From xen-devel-bounces@lists.xenproject.org Wed May 01 02:52:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 02:52:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715225.1116742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s204V-000369-9z; Wed, 01 May 2024 02:51:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715225.1116742; Wed, 01 May 2024 02:51:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s204V-000362-4T; Wed, 01 May 2024 02:51:51 +0000
Received: by outflank-mailman (input) for mailman id 715225;
 Wed, 01 May 2024 02:51:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s204U-00035s-0a; Wed, 01 May 2024 02:51:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s204T-0003cd-Tm; Wed, 01 May 2024 02:51:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s204T-00009l-KI; Wed, 01 May 2024 02:51:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s204T-000181-Jr; Wed, 01 May 2024 02:51:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xAumekWbJMfbdZwDGm+Wfuu91t6tP4oqgKs5JlzJ4tw=; b=JYaqzQnqv/oeroQvLSfwSCTlpv
	nvNudhCjy9CsID7fLEJijkgXRLXsxO1O+clnQJGFJqxY6yXk3bmPHjD5Pif7OlYYvb9tROBRLItlA
	+Ecfz5M6EV+ork7vovtnsX0blg/z79ykraQE0VSw2DPyyzk2hUptmpt/f+gHNEXGYlVI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185872-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185872: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=be5b08dd6ea6ef0f01caf537bdae125fa66a2230
X-Osstest-Versions-That:
    xen=be5b08dd6ea6ef0f01caf537bdae125fa66a2230
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 02:51:49 +0000

flight 185872 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185872/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot                   fail pass in 185860

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185860 like 185839
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185860 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185860
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185860
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185860
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185860
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185860
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  be5b08dd6ea6ef0f01caf537bdae125fa66a2230
baseline version:
 xen                  be5b08dd6ea6ef0f01caf537bdae125fa66a2230

Last test of basis   185872  2024-04-30 01:55:16 Z    1 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed May 01 06:54:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 06:54:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715242.1116750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s23rM-00069y-Sl; Wed, 01 May 2024 06:54:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715242.1116750; Wed, 01 May 2024 06:54:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s23rM-00069r-Px; Wed, 01 May 2024 06:54:32 +0000
Received: by outflank-mailman (input) for mailman id 715242;
 Wed, 01 May 2024 06:54:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UTMh=ME=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s23rK-00069l-Kh
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 06:54:30 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 (mail-am7eur03on20600.outbound.protection.outlook.com
 [2a01:111:f403:260e::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a732249e-0787-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 08:54:28 +0200 (CEST)
Received: from AM0PR02CA0116.eurprd02.prod.outlook.com (2603:10a6:20b:28c::13)
 by FRZPR08MB10997.eurprd08.prod.outlook.com (2603:10a6:d10:136::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.36; Wed, 1 May
 2024 06:54:24 +0000
Received: from AMS1EPF00000047.eurprd04.prod.outlook.com
 (2603:10a6:20b:28c:cafe::3e) by AM0PR02CA0116.outlook.office365.com
 (2603:10a6:20b:28c::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28 via Frontend
 Transport; Wed, 1 May 2024 06:54:24 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS1EPF00000047.mail.protection.outlook.com (10.167.16.135) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Wed, 1 May 2024 06:54:23 +0000
Received: ("Tessian outbound af213ececc3d:v315");
 Wed, 01 May 2024 06:54:23 +0000
Received: from 01be3ca9c208.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3251D331-81D3-4D21-8DA4-2E54B2FA1F23.1; 
 Wed, 01 May 2024 06:54:17 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 01be3ca9c208.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 May 2024 06:54:17 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com (2603:10a6:208:ff::27)
 by GV1PR08MB10854.eurprd08.prod.outlook.com (2603:10a6:150:161::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34; Wed, 1 May
 2024 06:54:12 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::446b:ac86:e80c:ffda]) by AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::446b:ac86:e80c:ffda%6]) with mapi id 15.20.7519.035; Wed, 1 May 2024
 06:54:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a732249e-0787-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=NKW6VNwYg3nJs7JnHe5TrE0sqaE4r2/D1UlM0wQM+Nla4K05JyGYuSm5Uqc/520pVHCo+wVWhr8V0YSOYUljA/Pm07k6yak9bdljKRyVqhQlGw3JG3cRt5cBZJiP6PcQ5x+dXyNqisfvP3HxH57fYXcX1tU3rVgiFLiyp/dD3wHCDZe3Jt/m8qy2iIzPZG7cMXa9JNJe8qd1uRVaeTX4Mr60X0/qJ8s/CgLdyE8dnhpC5RxoQR8zF33DtJYUUGerpkYX3h+n825a3uIXU7tqP1LgovfAC5+oHoedJpflBZvHgSqEwEkVGj4HBVaITJNZOS8w3GkRonbW/Ix0y2sc8w==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=G6wnQxmYQEs6x1K3g2n0Y17L5Gq9s/LAONIJZAEUtJE=;
 b=VYPMZX3t6nDxK+16m/v+OtKtJ0s80tQYDRSnL5pukHah8HacDGsKHdqLuFyQWMmiTt3146QA7tW2rqwJJQAscWpL8An7JwE7n1e1t2OwV2qewKeaIguHYmBQE+MsqVTwNZ410m17olXVs1mNfbJ8b3n5RhBOoS7U+aPlN6UeWppbKTrfMXNV0Qimw/rLuciPewiqCkYRk23ubkQ1OlULcnU3XRlKroOjfd165FMvISC5X+lYJNXPrjqdKC0RsVDT/fsMuvWDwuIoB8RVwjmZj0R7Z8rb2YmYtKp9vYSVgnIlLD99mHKFxo0aYQBjCIwbfETcmxQbh+M4HJCSbjjzBQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G6wnQxmYQEs6x1K3g2n0Y17L5Gq9s/LAONIJZAEUtJE=;
 b=igOFEGp4QcjcQyGEBTRz+ncbYVUGzUFU1PLfCPo4sJclwoUDsIBEkcgnQ+WybYvPf182laZscypfgsr+UAYIttdCiEqLZf6zqDSNDFbpOuIqW9GPev1jlN3akIeY2284AMtbFtc0J1bt7e761lymmg+KxLHfoksRVjVXiSGKh3w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: c9d8f8adfb55b057
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kxJZH8gapmQ0MCh6OVhFu7ZXlNn1Jm0/SozpvRNxeeYegg/DWVbKUumsoeT8hshgFYJCPyIymZ0u4qcNsNG1itbIJSD2ehTMMrX1wZ8oXhH+apwkycxk3bQihHgpi6qqlDWXl4+ol+Q3mNuuqKFe2yBWS/pAUi1EaV3ll2M+bmu97N9qraAhDmwMjMLz80U922tFxBAK9iSHe3VIh5a/qxaINPbw9u1cw5vIyq/l7xBuktQQC12jyG4Hwnh64RRvhXGyLUtZwcvCR3mkuV6C+es/YP+39RonaVl31rJllHrWtgnPvH+jCU95HxiQLjet6Qvj6Be2OKaFjOlNqlXDwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=G6wnQxmYQEs6x1K3g2n0Y17L5Gq9s/LAONIJZAEUtJE=;
 b=AA4vTXtAL5sbIt5axdYnlf3YohyZWfLrKd+aqt+wvJ5xyhTLDdaESAZk6C06upvj9RBWpoxLYeA1J5rsIFdzpeBsuS1KKxYXm8bNCKDjRbZMjiC7bQNL050ywRh/XdeIkZl0iL3Ho+RCHFRdB5ed6alW+eJltHTb/20LUiLNe+mcMhwpS2bSnUp8xDMSi3g4nTCsvsdZPhXorZyEljVdBSCFOPn3EUbbzOq1q4ZSZR8h8QjBCMhfam/uTR/o6T0DnpvDb4rtRqRiOe0Js5udAgbW7W2AsH002/frR/8ly3ytrcoI3uGP4WLsAfZxzTSKzesbY8HweI6ACm9I//weYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G6wnQxmYQEs6x1K3g2n0Y17L5Gq9s/LAONIJZAEUtJE=;
 b=igOFEGp4QcjcQyGEBTRz+ncbYVUGzUFU1PLfCPo4sJclwoUDsIBEkcgnQ+WybYvPf182laZscypfgsr+UAYIttdCiEqLZf6zqDSNDFbpOuIqW9GPev1jlN3akIeY2284AMtbFtc0J1bt7e761lymmg+KxLHfoksRVjVXiSGKh3w=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Thread-Topic: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Thread-Index: AQHamu7vn3o+gufgI0eDMZNP6vgeorGAsTwAgAFBl4A=
Date: Wed, 1 May 2024 06:54:12 +0000
Message-ID: <2AB02E85-2746-4DD6-A821-B858E52A6954@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-2-luca.fancellu@arm.com>
 <cd2352aa-e94b-43ef-b3fc-359cc1f43fb0@suse.com>
In-Reply-To: <cd2352aa-e94b-43ef-b3fc-359cc1f43fb0@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM0PR08MB3745:EE_|GV1PR08MB10854:EE_|AMS1EPF00000047:EE_|FRZPR08MB10997:EE_
X-MS-Office365-Filtering-Correlation-Id: d6fd1e70-8969-400c-d00a-08dc69ab8902
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?S2ZNK09rb1l6bS9BWEFLelRkTGpJUEQxbjRLaEZoS0doUlYzYmxHVTV1NytV?=
 =?utf-8?B?M283em5SNTNMUWFhOHFMUGd5SjNOWmR3Z2dDSHgremZBWUJhS2s5RXJ0akow?=
 =?utf-8?B?U1B1V0trSitnNEp3SHNXdEJ0N1NRZHpuejhra1lKWGVVNCtWMThROXcrMmF2?=
 =?utf-8?B?a1VreVpxOWpLc0hnWVNQNTJNb3poaDRyd1BGNnJWMEVyOUk5TWwwR3ZSaHJH?=
 =?utf-8?B?eUVqeTU0SDVpaXVNb3J5OFpFRnNkUmpkaGhiRXhQRWs3UVpkUTdERG1wSkJB?=
 =?utf-8?B?UUVpYWVYS3pSVkZZNFduaEEydFB0N2hIVWpWcG1kT2s0NmVqVHlQdE0rT0dr?=
 =?utf-8?B?eVRrY1FFUVNIN1hnZi9BbS9OZnQ0cTIzOHoyN0V5T2s0ZFhZNk9pY2t2UFU2?=
 =?utf-8?B?RlVBbGtLamVHdmdHYmZ3UWEwanVaY2llYjZzWUwyRWRSU2dQeEcxL2xadTNh?=
 =?utf-8?B?eUxJdUJ3ZnkzLzFOMUtNR1RiN2RBYklJS3JacSs1cFdmVG42QUh2SzUzdFA1?=
 =?utf-8?B?blJPbytURmNHU25QcUlPMDEwbDc4THJVMzRESEtGdkJBZTVpbzhseUNaQitQ?=
 =?utf-8?B?MGdiU01pR3JyaW1DcVZwaWhkRjZxaWlsL21rU2tOb3dKemVmNGVhSm50SmtC?=
 =?utf-8?B?cVJqa1cxc1Fud1dvOEI3K0dsSzgwRHdyeCtxaUJyRXFZTHRrWVQ5UWRoNDJF?=
 =?utf-8?B?UTJiZnZYKzRqbXpKZDNJYThQYUx6R2ErYVAvMmNnSmdpUzY3VEJiMWlyRS9J?=
 =?utf-8?B?MmhDS3YrbElHWkFkbUlGYXN4eGQvZzhFTjk3Rmpxa3d1bGdOeEdjeDhRTGRs?=
 =?utf-8?B?Vk9jS3QzNGljUEs5ZlVUeXlWWlk2d092SnFab2drZGFxdHUwZkZvY2FydFdl?=
 =?utf-8?B?bWJzbWV0dUNSdkNmM1pXcCtvYTBBVFRsOHIrSUZWM2RiUFJCNXFNVHdpL1JK?=
 =?utf-8?B?S3ZLMERFaFNPeGZxcmt2Ty9DMm1XdDFORmMvOTNGWVJ1VTZ2U2RFUm0zaUly?=
 =?utf-8?B?TjZmdStCeGpZYnZWM0xCUlhaNmtJNVdnc2tPN2NnTFJ1VWpwSFNPTnlveUVS?=
 =?utf-8?B?Yi81WkVBb1BJMFRGbVYzVU5NSkxFbUEwRmsybG9pUXVEOUJLdjl5WXlkTmpP?=
 =?utf-8?B?WWpIY2ZjNmVrK2krSVc3aFVxZUdGQTlrWWkwT1NyQ05nVEo1a0dhRExYVHQ3?=
 =?utf-8?B?bksvWHUyYzVHZExYMFdFTkc1UDBWSXdQbEpsbGR1YmNtakZ5eUcwREF6R2hw?=
 =?utf-8?B?MTZ1WXdScC9weWNPUFlsZ3Q0Q0J2NG1ONjFvMzZQdDdlN0FOa3kzQjJaNUh5?=
 =?utf-8?B?Q2VjeW1nbTNYNGtHa2E1RUdaa3YzbVgwc0t2OEQrajd4WnpWbkd3QS84eTJW?=
 =?utf-8?B?ZDRUNVhRSG9QcU16bjZYdjdMRHUramZIVGQ0akdjb0RSbWRNS1pTWUozMzFV?=
 =?utf-8?B?cE54RTZDTzZmSC94NytYUnc5aG0za1ZNRVRLUDI2K0JUczhjVVdTNUhoVHVJ?=
 =?utf-8?B?dmY5NW9jSDlscXYwWTFuRmRjWUZRMjJWZlJOOUoxTHFTZUNUUk9haU8xYnVt?=
 =?utf-8?B?VytSdkg0MENCckRQRDdoeDNMa2tWOUk0dnFZc2wza1F2K3o4d0piVGFEUVVF?=
 =?utf-8?B?dVQ2Uk9VSlJQZFFLdFBCZURkaW5HeVljcGlVeUZIZWhxUy9sODgxaU1WajRu?=
 =?utf-8?B?VG5IZEtqTVNJVm5hRUVNRng2bXhzb2RuNzRONEJwanhEQUdzWGxOZ21nPT0=?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR08MB3745.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <CD6A70CA13D48E4DAABBF57EA3D64557@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10854
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS1EPF00000047.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e534cc95-a711-4097-2447-08dc69ab8215
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|35042699010|82310400014;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?MUNiZG0raGdhS1cvek5LYzk5aE5ma0NmTVgyRDJqVjBrWU9adkdCckdnOEJD?=
 =?utf-8?B?VEhzbUZnUDFGSnc2Y29xTGcyTUorWHlva1hHU3h5eFBqbXpSbFpFbnJsenZZ?=
 =?utf-8?B?ZGtsTm8yT2VtRkc3d0hiQXpKN0FEYmJLY2c0MnBXV0w1d3FQQWxoWGsxQnA3?=
 =?utf-8?B?bStpTU5pWTYrQmV1SGQ0STFRVXlDU1FiNGRLdU1WaUJCUlEzcnk3QjJsdmxh?=
 =?utf-8?B?VHZOYjhJT3FrZnFwNXdFTXBzaFlsTDRJSGlTVUVRSitwSkhPK2JDTkgrNkxq?=
 =?utf-8?B?VXlURFA1OEdTaWtkb3NwZGR4Rnp3UFcycGlDUGEwVW1QanlCV0RtZ1JiVEVo?=
 =?utf-8?B?YVFtcWVBMDZCV2pzRjBlb2FwUEg0bE5jcS8rRXBDbSswaUlkRGlhOVVpT3lR?=
 =?utf-8?B?ZXVCVjd3aitVVlB2K2pEK2dadFMxQU02YzlpcUlicHlnS0N6MGdCcjFYSUI5?=
 =?utf-8?B?VDgrV2drY1NqNkp6Z3BFN1BrUmd3YVJrYkZGcUxqcWRnRVVzc2RuTzRjN0JV?=
 =?utf-8?B?M2RNRi94elp3OUptK091RXphcE9HWFhpYnluZjNTS0lqRXBBRTM0SjdEQ2V4?=
 =?utf-8?B?OXZwTmdYcUhManpkcW1CVmF4UXNHRVNzWk5jY3hxWlZ6OUdNVmFhMXBQTkdD?=
 =?utf-8?B?ZXJkckZTaEFsUlNSdExCV2J3d0ltSm5wSXFuMWtRWnhPZyt0TGhhUCtNQ2pF?=
 =?utf-8?B?ODZWOEIwSFpsZXMzdTcrelR3UFo3MjFKUjRqVEZwS3pDSFltQ3dhZVBwMitW?=
 =?utf-8?B?UlZLRGhkUTQxOW1DT2NyVzdoM3hGV1ozSWw4cTJPeTlhcGNQMGVTNUlNUnhR?=
 =?utf-8?B?eFFUWTBCcDhwektwSG5CQnpJZmlPMHpqQ3g5WVVBNmhnTWxJVmt5VGZQRTlt?=
 =?utf-8?B?WEdHdUxqNHhUVTYraWlQZFRuV2p3QlNFajRYd2dJQklObTRoa1RUb29wT3kr?=
 =?utf-8?B?Z1YyYXZGSWJDWGM4cEhTRFQ1RWd3Rk43b0FHRkROZFVLNkFva2ZzVS9OQWFh?=
 =?utf-8?B?a1c3WG94ZnNhTWFkNmkvZzBFYjBvdnVyTWtrL2N6VTdiVEgwZ2N3TSt3RzI2?=
 =?utf-8?B?cS9WM1NWTmgySW05N0oveVcva044MkwxS3NGdy96cUV3R2owUTFOQXBJeHQ0?=
 =?utf-8?B?bVg2bDlRTEdZS3A3bndpR1VEY0N3akJ2MUFVK1hPRzB3ZFhxTHp6UDA3cUFr?=
 =?utf-8?B?RDBsd0x1N090NlREY3VUWXR0RXhEeEtOWXpjT0dIRUZBZlovRlNCOXZpMUQr?=
 =?utf-8?B?VVhENzZycXpGYS9IcXFEeWZuenJvM0x6Vi9Hc2pZWlgwdlA4RUFLWjBReU42?=
 =?utf-8?B?WW5hR3JDWi83M0R1VllCNWxrVFdoYi9XSWpZMTNTK0ZpSkRpa3cyZWJvbVNX?=
 =?utf-8?B?TEZPdGp2c2s0OWU0T3RNdXVpTm1odFkzeTBIOUkrNU1WenlTYklqYTJxaklR?=
 =?utf-8?B?cVNwZWVrZlROcmdDOUx3QTYyL3VRQTRaRzFhVW9BUjRvcFJrclB6ekk2SnNs?=
 =?utf-8?B?MWhqT1BEMnFHN0RFQmxlMUtrMjl0Y05tcGVDMlZmMlBLV2RvQXFGdG5KTjVN?=
 =?utf-8?B?UTc1SU8wazk3cFJxSFpWSTJ2d0g5aWlaNVNoc2lUS2tlL005V2dTek9DWHN5?=
 =?utf-8?B?Y2J2NUdtL09rYjZuRUNVa3prOU4ybEI4SGVScW1rdnR4Y3pDQStYdlZJS2V6?=
 =?utf-8?B?VnBSWnJlY3VGZjh4L1QrOFpFc0l3RWdUNHdrRU5qZzhIV0w3bzdUWTlaT21M?=
 =?utf-8?Q?qXCdBzMh9D9jk2W56/GbaA4brBzmgCG0LEcsbWJ?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(35042699010)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2024 06:54:23.8129
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d6fd1e70-8969-400c-d00a-08dc69ab8902
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS1EPF00000047.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRZPR08MB10997

DQpIaSBKYW4sDQoNCj4gT24gMzAgQXByIDIwMjQsIGF0IDEyOjQzLCBKYW4gQmV1bGljaCA8amJl
dWxpY2hAc3VzZS5jb20+IHdyb3RlOg0KPiANCj4gT24gMzAuMDQuMjAyNCAxMzowOSwgTHVjYSBG
YW5jZWxsdSB3cm90ZToNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9rZXJuZWwuaA0KPj4gKysr
IGIveGVuL2luY2x1ZGUveGVuL2tlcm5lbC5oDQo+PiBAQCAtNTQsNiArNTQsMjcgQEANCj4+ICAg
ICAgICAgdHlwZW9mX2ZpZWxkKHR5cGUsIG1lbWJlcikgKl9fbXB0ciA9IChwdHIpOyAgICAgICAg
ICAgICBcDQo+PiAgICAgICAgICh0eXBlICopKCAoY2hhciAqKV9fbXB0ciAtIG9mZnNldG9mKHR5
cGUsbWVtYmVyKSApO30pDQo+PiANCj4+ICsvKioNCj4+ICsgKiBfX3N0cnVjdF9ncm91cCgpIC0g
Q3JlYXRlIGEgbWlycm9yZWQgbmFtZWQgYW5kIGFub255b21vdXMgc3RydWN0DQo+PiArICoNCj4+
ICsgKiBAVEFHOiBUaGUgdGFnIG5hbWUgZm9yIHRoZSBuYW1lZCBzdWItc3RydWN0ICh1c3VhbGx5
IGVtcHR5KQ0KPj4gKyAqIEBOQU1FOiBUaGUgaWRlbnRpZmllciBuYW1lIG9mIHRoZSBtaXJyb3Jl
ZCBzdWItc3RydWN0DQo+PiArICogQEFUVFJTOiBBbnkgc3RydWN0IGF0dHJpYnV0ZXMgKHVzdWFs
bHkgZW1wdHkpDQo+PiArICogQE1FTUJFUlM6IFRoZSBtZW1iZXIgZGVjbGFyYXRpb25zIGZvciB0
aGUgbWlycm9yZWQgc3RydWN0cw0KPj4gKyAqDQo+PiArICogVXNlZCB0byBjcmVhdGUgYW4gYW5v
bnltb3VzIHVuaW9uIG9mIHR3byBzdHJ1Y3RzIHdpdGggaWRlbnRpY2FsIGxheW91dA0KPj4gKyAq
IGFuZCBzaXplOiBvbmUgYW5vbnltb3VzIGFuZCBvbmUgbmFtZWQuIFRoZSBmb3JtZXIncyBtZW1i
ZXJzIGNhbiBiZSB1c2VkDQo+PiArICogbm9ybWFsbHkgd2l0aG91dCBzdWItc3RydWN0IG5hbWlu
ZywgYW5kIHRoZSBsYXR0ZXIgY2FuIGJlIHVzZWQgdG8NCj4+ICsgKiByZWFzb24gYWJvdXQgdGhl
IHN0YXJ0LCBlbmQsIGFuZCBzaXplIG9mIHRoZSBncm91cCBvZiBzdHJ1Y3QgbWVtYmVycy4NCj4+
ICsgKiBUaGUgbmFtZWQgc3RydWN0IGNhbiBhbHNvIGJlIGV4cGxpY2l0bHkgdGFnZ2VkIGZvciBs
YXllciByZXVzZSwgYXMgd2VsbA0KPj4gKyAqIGFzIGJvdGggaGF2aW5nIHN0cnVjdCBhdHRyaWJ1
dGVzIGFwcGVuZGVkLg0KPj4gKyAqLw0KPj4gKyNkZWZpbmUgX19zdHJ1Y3RfZ3JvdXAoVEFHLCBO
QU1FLCBBVFRSUywgTUVNQkVSUy4uLikgXA0KPj4gKyAgICB1bmlvbiB7IFwNCj4+ICsgICAgICAg
IHN0cnVjdCB7IE1FTUJFUlMgfSBBVFRSUzsgXA0KPj4gKyAgICAgICAgc3RydWN0IFRBRyB7IE1F
TUJFUlMgfSBBVFRSUyBOQU1FOyBcDQo+PiArICAgIH0gQVRUUlMNCj4gDQo+IEJlc2lkZXMgbXkg
aGVzaXRhbmNlIHRvd2FyZHMgaGF2aW5nIHRoaXMgY29uc3RydWN0LCBjYW4geW91IGV4cGxhaW4g
d2h5DQo+IEFUVFIgbmVlZHMgdXNpbmcgMyB0aW1lcywgaS5lLiBhbHNvIG9uIHRoZSB3cmFwcGlu
ZyB1bmlvbj8NCg0KVGhlIG9yaWdpbmFsIGNvbW1pdCBkaWRu4oCZdCBoYXZlIHRoZSB0aGlyZCBB
VFRSUywgYnV0IGFmdGVyd2FyZHMgaXQgd2FzIGludHJvZHVjZWQgZHVlIHRvDQp0aGlzOg0KDQpo
dHRwczovL3BhdGNod29yay5rZXJuZWwub3JnL3Byb2plY3QvbGludXgtd2lyZWxlc3MvcGF0Y2gv
MjAyMzExMjAxMTA2MDcuOTg5NTYtMS1kbWFudGlwb3ZAeWFuZGV4LnJ1LyMyNTYxMDA0NQ0KDQpO
b3csIEkgaGF2ZSB0byBzYXkgdGhhdCBmb3IgdGhlIE9yaWdpbiB0YWcgSSB1c2VkIHRoZSBTSEEg
b2YgdGhlIGNvbW1pdCBpbnRyb2R1Y2luZyB0aGUgbWFjcm8NCmFuZCB0aGUgU0hBIGRvaW5nIHRo
aXMgbW9kaWZpY2F0aW9uIGlzIGRpZmZlcmVudCwgaG93IGFyZSB0aGVzZSBjYXNlcyBoYW5kbGVk
Pw0KDQpDaGVlcnMsDQpMdWNhDQoNCg0K


From xen-devel-bounces@lists.xenproject.org Wed May 01 06:58:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 06:58:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715245.1116761 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s23uu-0006hK-BI; Wed, 01 May 2024 06:58:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715245.1116761; Wed, 01 May 2024 06:58:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s23uu-0006hD-8b; Wed, 01 May 2024 06:58:12 +0000
Received: by outflank-mailman (input) for mailman id 715245;
 Wed, 01 May 2024 06:58:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UTMh=ME=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s23ut-0006h4-78
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 06:58:11 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2611::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b19ca7b-0788-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 08:58:09 +0200 (CEST)
Received: from DB3PR06CA0013.eurprd06.prod.outlook.com (2603:10a6:8:1::26) by
 AM9PR08MB6659.eurprd08.prod.outlook.com (2603:10a6:20b:30a::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.36; Wed, 1 May
 2024 06:57:55 +0000
Received: from DU6PEPF00009524.eurprd02.prod.outlook.com
 (2603:10a6:8:1:cafe::d1) by DB3PR06CA0013.outlook.office365.com
 (2603:10a6:8:1::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.25 via Frontend
 Transport; Wed, 1 May 2024 06:57:55 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU6PEPF00009524.mail.protection.outlook.com (10.167.8.5) with
 Microsoft SMTP
 Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18 via
 Frontend Transport; Wed, 1 May 2024 06:57:54 +0000
Received: ("Tessian outbound 9d9bf1c5d85a:v315");
 Wed, 01 May 2024 06:57:53 +0000
Received: from 50c7ed490037.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BAAD313D-9B92-44B3-892F-8EB2DCB3AC80.1; 
 Wed, 01 May 2024 06:57:42 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 50c7ed490037.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 May 2024 06:57:42 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com (2603:10a6:208:ff::27)
 by AS8PR08MB9119.eurprd08.prod.outlook.com (2603:10a6:20b:5b0::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34; Wed, 1 May
 2024 06:57:37 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::446b:ac86:e80c:ffda]) by AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::446b:ac86:e80c:ffda%6]) with mapi id 15.20.7519.035; Wed, 1 May 2024
 06:57:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b19ca7b-0788-11ef-909b-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=Sna6tCkGh7DLQdQ7GHmLTl/BhZFDJ42+SpZiUPjrZIsoPGxIvso9x5UQA/SxSVy/81kV5H+j4xPymZXILdS0C1gPPP1JydCqvwZMcnB/4mwJrvYB2nYc09QwzbL7Xfj7rXh5QzqfcsBlf+5NcfoWtWexMv7wmKOXFCiQpw5fENAcYvt3hhey2REevBP92v/SnetuQXYvQVVbwHip8Vg0I2QScZzHkD5oHzGB+3cOW+VyQz/xEjFvjI2ZXmFvXQhniFJ4Q36f7eJQyD4zmd/xYDQCS+khkcdpQkHvpVFWYSTZPKYEmUusxFKKrtzw400ciW8fOPfUHia0FhSAQgElBg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/HJA8n3NJ/TQmyBEtz7L3kCXnOjOxKdWvuquyWUVvJA=;
 b=mfQ9EsjiRB8d9GJZZ6H/xWU8UnH17wpWIRVJvqUkUiiM4zakAWO4MRM6p2mM1vC0Jd/2aJ0qlQTXnPwMdMXcbMgEeCKLmAXAR/jr+dp7nLthcGFTiRoSF0h8HMlbD3ImgP8Yanm7PdMPRgeBkJy4OPly+uXrLJjkLXtnKPqobk2yxFCZ2WfLpom2JlzhVkdx+SdPwFQlNMdcE5JZ9m/ASs4GW5kyOV7pmYVzRPHYllXmDXSqSQRbmaCRhYVIcvKCyLlC8ydti/U9wW1pxWnyZhWhQttTi7LTOiP90B2lAD6yiYEA0d6BKzxgTM03VzR838Q+ThX+PcJSXGzJCfuvMA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/HJA8n3NJ/TQmyBEtz7L3kCXnOjOxKdWvuquyWUVvJA=;
 b=rsEz8HyMvucczjakrT9p0Go4Ggig8g4bk+wth5ZGcMZCirukxCO9KduLcWBco7cYST5gCNez/lvrCnB13+NYUITj8AClgmfkFdcwUlww1/sIUh7NYdwh2pPKwsf3Bmls9j+XGG+bXweVmONHbAsvzDRw5pOyHyjHLMmeCzD26rI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 1be23a0ccc45f80c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FeUEljIv1IBl6+GEw/LMNCcA02hFy44d5v44mbnG04SVAR5YyKl2diK7r0UQrVO4/BdWIOtHuCWWI1nXzSJ0qWXb6eD0RBDYnsjS5MFrq67QNf47oOb3jueP4koy/O7e42TX5b7fSXGLC1lomwOS99PgsGaKs8U9wB6VXALr1q/YmGQohJVpQ7SH4NGw18eCzyyGKAo1zF9EbSy1bc1nLc1NhjsHEWnJ5m69zUlHMxK/d357+JTn2jZXZQAFs+g3EXPQ9KMW7vlrL0G9wTcoCUugO+W6bWlTokdlsM4+EEdzgyXaPjLTt1VqrWOOWZ0UVJSugcgyFtj/LCLl98EfTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/HJA8n3NJ/TQmyBEtz7L3kCXnOjOxKdWvuquyWUVvJA=;
 b=G8DM8CdrlAs6VSM3LtQF/H4WCihyLTbsd4v5PFpKdkaHuZf36v6nNQPQbwCeQLuCRevatsPN5HLvqZtvJ3KvzMfgm4DNJE0ESSYydIYRieoH/U/BbBGs04xa4je9DdU1xfwg+Bj/1V4SV4SnGiPJ2JCm8a7QCUgWr+A9Yxb80s+Wwg1/8J5v8Id2/3pkjSz3Kd05/qlY5ldkF5UCQ8ZykuSgXZ0ybgR0FF3CSq2Phovn7hk1+nRtaXvuzoQxZgN1vUt+HtRVlhT/XcUSHhoJJvXp+gO9jl1t/yeY1nf/WX1CmLh8saJg3wumuWMvgnOaNNb8kQ8usmObivVp5OYY8A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/HJA8n3NJ/TQmyBEtz7L3kCXnOjOxKdWvuquyWUVvJA=;
 b=rsEz8HyMvucczjakrT9p0Go4Ggig8g4bk+wth5ZGcMZCirukxCO9KduLcWBco7cYST5gCNez/lvrCnB13+NYUITj8AClgmfkFdcwUlww1/sIUh7NYdwh2pPKwsf3Bmls9j+XGG+bXweVmONHbAsvzDRw5pOyHyjHLMmeCzD26rI=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index: AQHamu72Eq1RpEGs60mNp9DIfY8mn7GAr6+AgAFEGQA=
Date: Wed, 1 May 2024 06:57:37 +0000
Message-ID: <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
In-Reply-To: <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM0PR08MB3745:EE_|AS8PR08MB9119:EE_|DU6PEPF00009524:EE_|AM9PR08MB6659:EE_
X-MS-Office365-Filtering-Correlation-Id: 9764ec14-642f-4d5e-20a6-08dc69ac065b
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|376005|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?Rkk1MFBndkxGVW5MazJVb1BxMEZKMXRvZms2MmQ3M3JrSkJMVkxZd3BCcDdS?=
 =?utf-8?B?T3VTRVNOMkVnc0JSdlpYdXdPNEs1VDNwY0c0Q2hjZ2l1WlkvUE12bGJFeEJ5?=
 =?utf-8?B?SURNZEZibDVtTXhsYXRrTFFFVHlXYUNTV2cvci9VWnduZWFQWThLSmdheW03?=
 =?utf-8?B?TkpjdnZ2cTNJMEtqL3R5UDlFbVJVU3hpY2wvMTRNTC9kQzQ1VWNqcmI0Qzh3?=
 =?utf-8?B?K1lIMy9zcHZjN1FSdlpIT1JzVGpTK0V1bXlocy9iN1hCVjhuNTJpR2l0YUow?=
 =?utf-8?B?WmlWZkM5alJoUFpsQzZvUWxCQjlSOFNZNmhsSUVjVlhsUEpZZ21xR2REWnF2?=
 =?utf-8?B?VjRCMUVIbDVERVlQNUMvdE9CdFJyQUUyNSt1WDBSL3lYMzd5b3R0aitSQklT?=
 =?utf-8?B?ZzFhNzAzdFBjcllCaG13MGN0dkFCZVFxYnBEL2wrSHdmUWFBT3pqUUZmQ1Fp?=
 =?utf-8?B?eCtrRS9HdHJBZFlOMElQUnFqQitZeTVKaXBHeFI4VHc0czg1SE5WY2FuQ24y?=
 =?utf-8?B?Nm8rS1puTkdtd3JQY21MRDBjU1VYRzdaRHZVMy8zK0Nwd0hTRytiakEyNjFO?=
 =?utf-8?B?NE1RNmxrUWhOdXZUVWdudUdrNktpc1VURDFiLzNYZDZWNTlRUTg5SHg1T1c1?=
 =?utf-8?B?RHpIa3QwY2d1VnhQMGpZZk9ldm9oZDBYdG9iemNVVDJLeE1QbU1CeS9xNmND?=
 =?utf-8?B?TXdmTlJRUmdlSU9rRlZmbG9UYnExNDBSdFRyNyt6c3prK1UxOFVSdnhnL1M2?=
 =?utf-8?B?TlhqS2U3Q0RCa1Q3SkpqTjRGWUI2NVVUNG9sYzE4MEpmNWZ5QzlBRjlzL2lN?=
 =?utf-8?B?blZKeUJTZnlqdStKVTNxekVtd0tBWlFiNWgzSEdCdHhUM0ZWUjdlOVc3Z0ZJ?=
 =?utf-8?B?UnV3MlFld0U5MC92eGd5VDRZVVFFMGVTUCtOeENzWmVNZEMrdE45QUtoZDk3?=
 =?utf-8?B?V3FlalhaRzJhY2hQTjIvMFAwazFvQzF6dGhmUnA3N2xyS1d2N1FBaStsQmdr?=
 =?utf-8?B?Tnd2cUZzUHNrZ21vSjhSWTJxZFlSRXgyWVg1alIxVnVtVnpwVXJ0V3NwTXli?=
 =?utf-8?B?RjNPL04wb3VwVXJhL2J4SjFqV2FtZzlqSm5zLzhZTll5WHo2eEtmbXcyRmdH?=
 =?utf-8?B?bGNxZDM1UXpTWklxdWZrNEZCREFkajN3SlMzcG0rWWQ4UHRTNERWSHV2bDVq?=
 =?utf-8?B?ZXN6bEhrZWRNZmZGQjNHQnJyOUJyZ1NUTTBMMWdWa0UweTQ3YkdvVlUwV1FG?=
 =?utf-8?B?a2JQeFE3Qkl1S3dkNlNoSTlGQ3AyRnZnOW9IU01kWThGbUtySzNXZXZvdlNE?=
 =?utf-8?B?azZHV2xJeTRZTWZTc1dnRHo1OGc2QVpuV1hqWFh2SEIrZ2pGa2VHS3RMelBD?=
 =?utf-8?B?ZEhlN3pGU01tbi93NWFEbDVjaXZZdTdrWTYxM2hGWjljdjRKNytMN254VGg0?=
 =?utf-8?B?OUF5U1B6QUp0SUJtZnNZZllLT0puYmFmU29UMHBmMVppM0w3UlJmeXN5N3h5?=
 =?utf-8?B?MTFoSUNJVVVlOGcvWmJHRTltdHg2KzMzQnRKdEQ5M2ovMElaWnZNaEdFdzEv?=
 =?utf-8?B?WTlGejlwS053aWlHbHZqb3JBRkI5bThoNGVYQTBSVnFDbkRIUWlqbS8zYjVW?=
 =?utf-8?B?UW9qd2d4QTFvd2VKaTRGTnVUWC9BWWYrVThFZGRsWkVGWkErR1VvaE0xN2Jh?=
 =?utf-8?B?a3l1c3oxT2VxTE5aSFYvTHdESUx5bWlXSzByeTNsM0FleHljYnRlekYyM0cx?=
 =?utf-8?B?ZUY1K0JrWmttOG9RKzdLaVdGRW1wVThYaE5iTDNGSHZIV2Vsc1NzQys1TFRY?=
 =?utf-8?B?MUZqejRubFBUeWx2cFNrQT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR08MB3745.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <F4BDB047E6412F469D4317891B77E22A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9119
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU6PEPF00009524.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0f5f89ae-0073-482b-5008-08dc69abfc48
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|35042699010|1800799015|376005|82310400014;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?R2hSZGFxSTAzZjhrWVdWOVdWYThVQkl1S3FINll1UU1BV1ppNS9wZTVTLzF4?=
 =?utf-8?B?dWw0WHFjT0QwbW4xTURIWFgybVBuMExIaDB0R1hqcjRUdVFZWEswVktaUjlY?=
 =?utf-8?B?UTk2d3NiVUx0QzlUWnlMMmRtMjVSRkxHVnJ0TUdMaElKVzdhYWJUMHNjZStn?=
 =?utf-8?B?a0F1cVlURlYvZjZLNFhjNUR1NzFYOFNlcDJ0OVFUVWxnaHY3N0p6YlprY3pU?=
 =?utf-8?B?VXQxQVpRa09SYW12VE9CSCtOMC9nT1NIL296bHBPZGZKYzY3dTNsQTZqQ3pT?=
 =?utf-8?B?Nmk2cGlUVExQWkJMT2Z1dXYyZ1hLZFdsNVpNNVlwNG0raUR3RzN5S1pVM3NJ?=
 =?utf-8?B?Q0NGVCtIWGtCVytFckg0eUNnb3BmV1dmdDVyMklaWjd1eXQwa0NDSlVjS3JW?=
 =?utf-8?B?bHpNNW5nMHJYa3BmMHQwcUhlWThFY1VZOUxIcWJSYVlwaEJNZ0hEWU5mUnZu?=
 =?utf-8?B?YlQvSjN4OUd0K2ZJZnV5TjRzWjRQMFk5VDM0TnBOcUtVWm10SVBqbFo2Rmhz?=
 =?utf-8?B?blhoNnBHWTJsM3U4bUZYcEN2cXZLeEZkL0k1UFRuc3ovWEFiVitnYTdhaW1Y?=
 =?utf-8?B?b0d5LzVYa2F6QjRSOGJEOFdreElhQVJBSjkvSnhtMEhyM1d4N1JOdlN0TFJs?=
 =?utf-8?B?SWtiMmE2Z1VXbDRsMm5tcnpGcE84b2pwSzJKZFF2MXNPcTVwSllhYlZnejFT?=
 =?utf-8?B?bmNXWjZObHZ0UFRqMm1CVDJXbWFOV1MxOElMR09WMHo2aUJDVGcyK3NaTTVC?=
 =?utf-8?B?dlhCTGVjUzFnaUdnT2lBUG9IN2dXdEhOOHlwYVZybmZqTXUyNTNUMFpSZGUx?=
 =?utf-8?B?dE5DVUp4eUZkQWZLWGhZdWhRNlJJM0V5K3VCeTc3dGJnVUUyRlZoQ1VmWUhm?=
 =?utf-8?B?RXRuTkx3VW1JcHROY2txNjVObDJiNUdUbWV2RXZoeENwUFdUUGVxeXdRL2FH?=
 =?utf-8?B?NURUNW1OdExMSFNmM0VaNTBmZHgzSFgyZGpMTzkzcFBaZ1N5bmtQY08yMk1E?=
 =?utf-8?B?TlM1bk9PQlJtWlVyc1FaNGdXWllXa0E2MFU2eXJlSzRrRTk0UXZQbnlOL0JI?=
 =?utf-8?B?MWRML2ZxRTh0OFB5ZUVNODdLQVFUVUNVbDFKSFNXSXNuSkNzTFdlTEJzbkxu?=
 =?utf-8?B?UlNqTWJyZkpMUzFOdXFhTVhMTmsvUUdIRjhTb0FKMkJ3L0s0OEVCdkNJVHlI?=
 =?utf-8?B?OEE4aFRneS9BaEJzVkFCTmdpUGptSytIYzE0SFpqcW1UWVNPYlU3d1BCS2I3?=
 =?utf-8?B?SHdrQVFkSWJsQUNDZ1g1MUlGTmR4UzFUWEd1UC9EUHovOXY5dktNNElFZXQ5?=
 =?utf-8?B?MndLeW5lSzhEV3lSQzg5ZXpZTGVRUUlKdnJrclBQUFpjb0phbkJpYXRmeFRl?=
 =?utf-8?B?ampkbkRrUnpyWXpTTC9nWXFNZGZmUGZwSkIybWFML2ZqVEwrOEtnWkZlTWc5?=
 =?utf-8?B?Vml1WjdCdWZjRk1zcS9MUUhZMkhHei9VNVB6US9qLzJmd2tESWdZeWdObTdZ?=
 =?utf-8?B?Q2FjU0taLzM4aitLUnNKY0lHcU9YdGQ3Ry9GZVo4dG1KQlhJcjh5ejdtL0Y2?=
 =?utf-8?B?dUQrbFNqbHdBdmRkWVpHVXF6UlRuUEc1eGpvMk0rVk9RcnZ6cFkwVFFzRkRV?=
 =?utf-8?B?eVptNTk0cm9tU2RESWdIRG5DUTcyQnNxREdhL0JwcmgyQ0tacFlmL0lQQ3pO?=
 =?utf-8?B?dDFKcU9XdkRYVXdtWWFtY1FySzd5ek01VkZ0YkNkZDZKc0QySlFBVHBGYUR4?=
 =?utf-8?B?UTlMbHpvQXpjSUhNeitkVXNHNUY1cGJTeFhCNnU1V2NWTjF2bGhMais5Ri9J?=
 =?utf-8?B?ZlRYcENKYkdIcGRCR040ZEJXdXk5NmFSTUQ4QzNmNDgzb0JURHdMVEQyY0Vl?=
 =?utf-8?Q?4YdUjHtulfghY?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(35042699010)(1800799015)(376005)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2024 06:57:54.1694
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9764ec14-642f-4d5e-20a6-08dc69ac065b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU6PEPF00009524.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6659

SGkgSmFuLA0KDQo+IE9uIDMwIEFwciAyMDI0LCBhdCAxMjozNywgSmFuIEJldWxpY2ggPGpiZXVs
aWNoQHN1c2UuY29tPiB3cm90ZToNCj4gDQo+IE9uIDMwLjA0LjIwMjQgMTM6MDksIEx1Y2EgRmFu
Y2VsbHUgd3JvdGU6DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaA0K
Pj4gKysrIGIveGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmgNCj4+IEBAIC02NCwxOCAr
NjQsMjAgQEAgc3RydWN0IG1lbWJhbmsgew0KPj4gfTsNCj4+IA0KPj4gc3RydWN0IG1lbWJhbmtz
IHsNCj4+IC0gICAgdW5zaWduZWQgaW50IG5yX2JhbmtzOw0KPj4gLSAgICB1bnNpZ25lZCBpbnQg
bWF4X2JhbmtzOw0KPj4gKyAgICBfX3N0cnVjdF9ncm91cChtZW1iYW5rc19oZHIsIGNvbW1vbiwg
LA0KPj4gKyAgICAgICAgdW5zaWduZWQgaW50IG5yX2JhbmtzOw0KPj4gKyAgICAgICAgdW5zaWdu
ZWQgaW50IG1heF9iYW5rczsNCj4+ICsgICAgKTsNCj4+ICAgICBzdHJ1Y3QgbWVtYmFuayBiYW5r
W107DQo+PiB9Ow0KPiANCj4gSSdtIGFmcmFpZCBJIGNhbid0IHNwb3Qgd2h5IF9fc3RydWN0X2dy
b3VwKCkgaXMgbmVlZGVkIGhlcmUuIFdoeSB3b3VsZCBqdXN0DQo+IG9uZSBvZiB0aGUgdHdvIG1v
cmUgc3RyYWlnaHRmb3J3YXJkDQo+IA0KPiBzdHJ1Y3QgbWVtYmFua3Mgew0KPiAgICBzdHJ1Y3Qg
bWVtYmFua3NfaGRyIHsNCj4gICAgICAgIHVuc2lnbmVkIGludCBucl9iYW5rczsNCj4gICAgICAg
IHVuc2lnbmVkIGludCBtYXhfYmFua3M7DQo+ICAgICk7DQo+ICAgIHN0cnVjdCBtZW1iYW5rIGJh
bmtbXTsNCj4gfTsNCj4gDQoNCkF0IHRoZSBmaXJzdCBzaWdodCBJIHRob3VnaHQgdGhpcyBzb2x1
dGlvbiBjb3VsZCBoYXZlIHdvcmtlZCwgaG93ZXZlciBHQ0MgYnJvdWdodCBtZSBiYWNrIGRvd24g
dG8gZWFydGgNCnJlbWVtYmVyaW5nIG1lIHRoYXQgZmxleGlibGUgYXJyYXkgbWVtYmVycyBjYW7i
gJl0IGJlIGxlZnQgYWxvbmUgaW4gYW4gZW1wdHkgc3RydWN0dXJlOg0KDQovZGF0YV9zZGMvbHVj
ZmFuMDEvZ2l0bGFiX21pY2tsZWRvcmVfeGVuL3hlbi94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c2V0dXAuaDo3MDo2OiBlcnJvcjogZGVjbGFyYXRpb24gZG9lcyBub3QgZGVjbGFyZSBhbnl0aGlu
ZyBbLVdlcnJvcl0NCjcwIHwgfTsNCnwgXg0KL2RhdGFfc2RjL2x1Y2ZhbjAxL2dpdGxhYl9taWNr
bGVkb3JlX3hlbi94ZW4veGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NzE6MjA6IGVy
cm9yOiBmbGV4aWJsZSBhcnJheSBtZW1iZXIgaW4gYSBzdHJ1Y3Qgd2l0aCBubyBuYW1lZCBtZW1i
ZXJzDQo3MSB8IHN0cnVjdCBtZW1iYW5rIGJhbmtbXTsNCnwgXn5+fg0KWy4uLl0NCg0KQ2hlZXJz
LA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Wed May 01 08:40:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 08:40:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715259.1116771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s25VB-0003gB-Vc; Wed, 01 May 2024 08:39:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715259.1116771; Wed, 01 May 2024 08:39:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s25VB-0003g4-Rt; Wed, 01 May 2024 08:39:45 +0000
Received: by outflank-mailman (input) for mailman id 715259;
 Wed, 01 May 2024 08:39:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s25VA-0003fu-SU; Wed, 01 May 2024 08:39:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s25VA-0002jg-OL; Wed, 01 May 2024 08:39:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s25VA-00049f-Az; Wed, 01 May 2024 08:39:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s25VA-0004A2-AR; Wed, 01 May 2024 08:39:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=503wu9zg5rVX7khBY27xcShrSCQ2hvMETeH8V5wly/I=; b=zRnMj1MTI6lzulfxuORZsjKdd4
	mqVc60ZjWc0HJr0NpwL31QZ+t4b9kqm/F7C5A5LvQx4W4Je3298csW65n7F5BNkDwyIxedYtD0Du8
	ms1cSLynnbFBhumSTfJBjwWi4Gm2FKpTeg9M2PaL/gWVyDZtQwLe4lHQAM0MVljSstVA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185877-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.15-testing test] 185877: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.15-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=056500003eb44314cb90f2e3e7b2d405e86b5657
X-Osstest-Versions-That:
    xen=0a1c1a01a8028a929d460a966c8f8e7384f5b3a6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 08:39:44 +0000

flight 185877 xen-4.15-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185877/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185709
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185709
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185709
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185709
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185709
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185709
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  056500003eb44314cb90f2e3e7b2d405e86b5657
baseline version:
 xen                  0a1c1a01a8028a929d460a966c8f8e7384f5b3a6

Last test of basis   185709  2024-04-17 07:07:02 Z   14 days
Testing same since   185866  2024-04-29 08:38:32 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0a1c1a01a8..056500003e  056500003eb44314cb90f2e3e7b2d405e86b5657 -> stable-4.15


From xen-devel-bounces@lists.xenproject.org Wed May 01 09:17:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 09:17:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715269.1116788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2658-0000IA-J0; Wed, 01 May 2024 09:16:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715269.1116788; Wed, 01 May 2024 09:16:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2658-0000I2-GQ; Wed, 01 May 2024 09:16:54 +0000
Received: by outflank-mailman (input) for mailman id 715269;
 Wed, 01 May 2024 09:16:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ji+M=ME=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1s2657-0000Hu-1V
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 09:16:53 +0000
Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com
 [2607:f8b0:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8b0af7c2-079b-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 11:16:51 +0200 (CEST)
Received: by mail-ot1-x334.google.com with SMTP id
 46e09a7af769-6ee1c0ecfa5so2141218a34.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 02:16:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b0af7c2-079b-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714555010; x=1715159810; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=COKrBMlPMlizpEaPks8BJRw7ImDVcG414j/cYDCxwB0=;
        b=Mz9Zp8kmuyUgkD0J2K9PMv+uRcaBQJ/g88dEcIMqyzVVO09Os75On8ZvGpAfBkxNS1
         NCAvA3NM/Q28KotxtKgU7Njyp8IHu3pp+2wGQERZ0x2PNGXxKuHQCDszScSP8EvNmfnU
         tDIPW6rsERB66KHr4elj2mSVflNhDJXEeQevs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714555010; x=1715159810;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=COKrBMlPMlizpEaPks8BJRw7ImDVcG414j/cYDCxwB0=;
        b=wDh6Jp5RYwKKWPD8N+r1lMFDyW6xdwem/rcf3zkX74dXHXLbvm0G0LouSEsLcDTvKE
         B+VYafI5s1ZTrIg3+z/iluM/hoGZveyTI7uzk3MemCLHw1Gaj8qObnihrvhDdK30i/GI
         LdUigKnuvFZmEpByqMqPiYsu+HIEocN4BZ/bor/vvTztKiiaWtV+BANcAeylL7kfhcmL
         PpTUPuKKbD1wgtGmpPEtOo4s8M+dHHiwKenHd4gcvdTl6YlZuEJp6XWqwUgKt4QWKjfz
         CThtG41aZhiGMEMr8J18hKzN5YGbvC2tZezxT92u4U8pYItL6udCJbm4reIDuVq/rObV
         WWqQ==
X-Forwarded-Encrypted: i=1; AJvYcCW7+HHe8HnpomT4YtiusLSrKpTvxGYtXb4cmo6fH3YZLP3m9lN5KseH9mkgzq/XqdrKfDgvMsdrghPmkbFNnFByeKLGD6o175FveSmaqW0=
X-Gm-Message-State: AOJu0YzmLFQl2PkRYSdrLQtDZklqdNn/sziQVWdf/YtjiT2CMBMaSZ5J
	EwjTUfea0TKUVOG6ThtgxMRqiIOrLaTOjVeCFyNs8alTHlRhOHcEbPtvfrRXR0g0i25lMyLNd5j
	t9jEijt2vFJfM0yjHsrR+5P0w362BHGp7tdk9XQ==
X-Google-Smtp-Source: AGHT+IHfXehXX/9e++Fr6oyG828pIdr7BriotzIkzUlvQ81XCJKrd1zahJCIq4ZVzOm+PWSMT+W04b1oIkxfBRKcYDw=
X-Received: by 2002:a05:6870:3287:b0:23c:550:ca37 with SMTP id
 q7-20020a056870328700b0023c0550ca37mr2031527oac.42.1714555009911; Wed, 01 May
 2024 02:16:49 -0700 (PDT)
MIME-Version: 1.0
References: <20240429151625.977884-1-andrew.cooper3@citrix.com>
 <20240429151625.977884-2-andrew.cooper3@citrix.com> <766b178a-5e32-437f-ac02-4cb9ad8cb248@suse.com>
 <1749cd23-ba17-450d-b5e0-29fc9ce9489e@citrix.com>
In-Reply-To: <1749cd23-ba17-450d-b5e0-29fc9ce9489e@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 1 May 2024 10:16:38 +0100
Message-ID: <CA+zSX=Z=y2VUXc3kr4VhV_1TyY27inesO3RBoZkKf4PUZLX1CA@mail.gmail.com>
Subject: Re: [PATCH 1/5] x86/cpu-policy: Infrastructure for the AMD SVM and
 SEV leaves
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>, Andrei Semenov <andrei.semenov@vates.fr>, 
	Vaishali Thakkar <vaishali.thakkar@vates.tech>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Apr 30, 2024 at 2:25=E2=80=AFPM Andrew Cooper <andrew.cooper3@citri=
x.com> wrote:
>
> On 30/04/2024 1:45 pm, Jan Beulich wrote:
> > On 29.04.2024 17:16, Andrew Cooper wrote:
> >> Allocate two new feature leaves, and extend cpu_policy with the non-fe=
ature
> >> fields too.
> >>
> >> The CPUID dependency between the SVM bit on the whole SVM leaf is
> >> intentionally deferred, to avoid transiently breaking nested virt.
> > In reply to this I meant to ask that you at least add those dependencie=
s in
> > commented-out form, such that from looking at gen-cpuid.py it becomes c=
lear
> > they're intentionally omitted. But you don't add feature identifiers ei=
ther,
> > making dependencies impossible to express. Maybe this sentence was real=
ly
> > meant for another of the patches? (Then my request would actually apply
> > there.)
>
> This is necessary because c/s 4f8b0e94d7ca is buggy.  Notice how it puts
> an edit to the policy object in the middle of a block of logic editing
> the featureset, which ends with writing the featureset back over the
> policy object.
>
> And it's not the first outstanding problem from what is a very small
> number of nested-virt patches so far...

I specifically raised this on the x86 maintainers call, and you said
to go ahead with it.

 -George


From xen-devel-bounces@lists.xenproject.org Wed May 01 10:00:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 10:00:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715275.1116799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s26lE-0006bb-Jm; Wed, 01 May 2024 10:00:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715275.1116799; Wed, 01 May 2024 10:00:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s26lE-0006bU-GY; Wed, 01 May 2024 10:00:24 +0000
Received: by outflank-mailman (input) for mailman id 715275;
 Wed, 01 May 2024 10:00:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ji+M=ME=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1s26lD-0006bO-Ci
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 10:00:23 +0000
Received: from mail-oa1-x30.google.com (mail-oa1-x30.google.com
 [2001:4860:4864:20::30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9e5df7cb-07a1-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 12:00:21 +0200 (CEST)
Received: by mail-oa1-x30.google.com with SMTP id
 586e51a60fabf-23d175cea45so793888fac.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 03:00:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e5df7cb-07a1-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714557619; x=1715162419; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4PgxCtXauH5+qhWuqX0AP7gWWK9Xjwp6PUSSNEOSfwk=;
        b=avS10P+veaboJSpscb5CXByzSXdSPamfdCCqvXoYmLzPvUSPZQ9/itysERbAjhInBP
         FumwueqvGbkQVzUTa60mCS1Huke6YdScVGj6i0CvAiJPiTADwOOabECjIvFsBBl5TrWj
         K4oeAfQh7QGMaXXkbmWiloNmLjLV2+y/hJBiE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714557619; x=1715162419;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4PgxCtXauH5+qhWuqX0AP7gWWK9Xjwp6PUSSNEOSfwk=;
        b=qn5dj2FUyRzWDjxMU/DTRqJhoxuLmmCCYJ4HxDTpjXb5K++gLKgUG7kOXZ9ezlOp+0
         kJBMviX6/QUxpOTfIjRIlu5AxUICd7EmLqLWlHhpPTqohfGImJOOg0XfDJkfrQE+6EXz
         tVeivHnNvjZ9Yx1Ck2GoXIqO2gtlGJwGfcIV4nRoI+5r87HxE6YQhXkcyjdBpgtMPH/c
         EpkOVsxQ30do9Kjp/AyxzpurkjY/8CbccYbRemMqARMnHMge8N0a83o3iXW+cvjb2l9B
         ZBVfRESipl9F1hVBrxt5qEFDy4ssaJm1+Yd27wRGaT9jNeTWbCoF/CCCLy+HhIJgiqsl
         zX7Q==
X-Gm-Message-State: AOJu0YyGbmwue5glaXVff+cHfllVPpbP+ooCBzl0FxR2jmPdkLuRVfuz
	l1K63Zmmq4r9dmt8Uj49T96XI+q0emIvSqApMgSD5TbfocMCEBogPZ6JJve6iiiyw66HrPjfkVl
	z/ENcVwLg5a/GxLXkQMMCfH6m0g/dnvT1bcqXXA==
X-Google-Smtp-Source: AGHT+IGRYQkhQmaNpChn5amcdOKu7dzGqhsYSIwudWBWFPpKMP0F3C6e6Xqf2UrTtikbXcUlsDTdopbpMwtf+Pa4F+E=
X-Received: by 2002:a05:6871:8783:b0:238:f6dc:6f7f with SMTP id
 td3-20020a056871878300b00238f6dc6f7fmr1949894oab.58.1714557619442; Wed, 01
 May 2024 03:00:19 -0700 (PDT)
MIME-Version: 1.0
References: <20240429151625.977884-1-andrew.cooper3@citrix.com> <20240429151625.977884-3-andrew.cooper3@citrix.com>
In-Reply-To: <20240429151625.977884-3-andrew.cooper3@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 1 May 2024 11:00:08 +0100
Message-ID: <CA+zSX=Y_fcDLonuRZZCaaKjK87tXEnnvJGjJHrKy5wzvaC2rzA@mail.gmail.com>
Subject: Re: [PATCH 2/5] x86/cpu-policy: Add SVM features already used by Xen
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Jan Beulich <JBeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>, Andrei Semenov <andrei.semenov@vates.fr>, 
	Vaishali Thakkar <vaishali.thakkar@vates.tech>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 29, 2024 at 4:16=E2=80=AFPM Andrew Cooper <andrew.cooper3@citri=
x.com> wrote:
>
> These will replace svm_feature_flags and the SVM_FEATURE_* constants over=
 the
> next few changes.  Take the opportunity to rationalise some names.
>
> Drop the opencoded "inherit from host" logic in calculate_hvm_max_policy(=
) and
> use 'h'/'!' annotations.  The logic needs to operate on fs, not the polic=
y
> object, given its position within the function.
>
> Drop some trailing whitespace introduced when this block of code was last
> moved.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Xenia Ragiadakou <xenia.ragiadakou@amd.com>
> CC: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: George Dunlap <george.dunlap@citrix.com>
> CC: Andrei Semenov <andrei.semenov@vates.fr>
> CC: Vaishali Thakkar <vaishali.thakkar@vates.tech>
> ---
>  tools/misc/xen-cpuid.c                      | 11 +++++++++++
>  xen/arch/x86/cpu-policy.c                   | 17 +++++------------
>  xen/include/public/arch-x86/cpufeatureset.h | 14 ++++++++++++++
>  xen/tools/gen-cpuid.py                      |  3 +++
>  4 files changed, 33 insertions(+), 12 deletions(-)
>
> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> index ab09410a05d6..0d01b0e797f1 100644
> --- a/tools/misc/xen-cpuid.c
> +++ b/tools/misc/xen-cpuid.c
> @@ -266,6 +266,17 @@ static const char *const str_m10Ah[32] =3D
>
>  static const char *const str_eAd[32] =3D
>  {
> +    [ 0] =3D "npt",                 [ 1] =3D "v-lbr",
> +    [ 2] =3D "svm-lock",            [ 3] =3D "nrips",
> +    [ 4] =3D "v-tsc-rate",          [ 5] =3D "vmcb-cleanbits",
> +    [ 6] =3D "flush-by-asid",       [ 7] =3D "decode-assist",
> +
> +    [10] =3D "pause-filter",
> +    [12] =3D "pause-thresh",
> +    /* 14 */                      [15] =3D "v-loadsave",
> +    [16] =3D "v-gif",
> +    /* 18 */                      [19] =3D "npt-sss",
> +    [20] =3D "v-spec-ctrl",
>  };
>
>  static const char *const str_e1Fa[32] =3D
> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
> index 4b6d96276399..da4401047e89 100644
> --- a/xen/arch/x86/cpu-policy.c
> +++ b/xen/arch/x86/cpu-policy.c
> @@ -9,7 +9,6 @@
>  #include <asm/amd.h>
>  #include <asm/cpu-policy.h>
>  #include <asm/hvm/nestedhvm.h>
> -#include <asm/hvm/svm/svm.h>
>  #include <asm/intel-family.h>
>  #include <asm/msr-index.h>
>  #include <asm/paging.h>
> @@ -748,22 +747,16 @@ static void __init calculate_hvm_max_policy(void)
>      if ( !cpu_has_vmx )
>          __clear_bit(X86_FEATURE_PKS, fs);
>
> -    /*
> +    /*
>       * Make adjustments to possible (nested) virtualization features exp=
osed
>       * to the guest
>       */
> -    if ( p->extd.svm )
> +    if ( test_bit(X86_FEATURE_SVM, fs) )
>      {
> -        /* Clamp to implemented features which require hardware support.=
 */
> -        p->extd.raw[0xa].d &=3D ((1u << SVM_FEATURE_NPT) |
> -                               (1u << SVM_FEATURE_LBRV) |
> -                               (1u << SVM_FEATURE_NRIPS) |
> -                               (1u << SVM_FEATURE_PAUSEFILTER) |
> -                               (1u << SVM_FEATURE_DECODEASSISTS));
> -        /* Enable features which are always emulated. */
> -        p->extd.raw[0xa].d |=3D (1u << SVM_FEATURE_VMCBCLEAN);
> +        /* Xen always emulates cleanbits. */
> +        __set_bit(X86_FEATURE_VMCB_CLEANBITS, fs);
>      }

What about this line, at the end of recalculate_cpuid_policy()?

  if ( !p->extd.svm )
        p->extd.raw[0xa] =3D EMPTY_LEAF;

Is there a reason to continue to operate directly on the policy here,
or would it be better to do this up in the featureset section?

 -George


From xen-devel-bounces@lists.xenproject.org Wed May 01 10:39:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 10:39:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715282.1116810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s27NO-000371-M1; Wed, 01 May 2024 10:39:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715282.1116810; Wed, 01 May 2024 10:39:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s27NO-00036u-Gz; Wed, 01 May 2024 10:39:50 +0000
Received: by outflank-mailman (input) for mailman id 715282;
 Wed, 01 May 2024 10:39:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f0p4=ME=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s27NN-00036o-DB
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 10:39:49 +0000
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com
 [2607:f8b0:4864:20::732])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 210ca086-07a7-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 12:39:48 +0200 (CEST)
Received: by mail-qk1-x732.google.com with SMTP id
 af79cd13be357-790f699eb2cso281325685a.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 03:39:48 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 i19-20020a05620a0a1300b00790f72d03c7sm2498032qka.126.2024.05.01.03.39.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 03:39:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 210ca086-07a7-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714559986; x=1715164786; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=l+LNw8JZHG9WJfE9wedst1re3+8Hol5mGi1tU7NxzNU=;
        b=jhmwf3NjcXl26/t23BtdS7Hhbt9rpCcHJ7yyL+l34bvgv1umkSt/L/H0AbTzZhjWZQ
         T4PvAYX72jpi5P6kFShWbJRFHPXbDwVXRdPn2oQjga/cgj3KHqwPQZBJxMRFzwD56rg+
         ukqVnV0kbIyt1RhKZqKiZk3F+1CZHednkYbmM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714559986; x=1715164786;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=l+LNw8JZHG9WJfE9wedst1re3+8Hol5mGi1tU7NxzNU=;
        b=tFnd+TjUf0dJW7+nfh5Qc8MpoiITJBfSprIYVf7vqTY5qAppmDIcGA1kVOaG2IkTw5
         BTu07mnz+iAnJ/3j6NEZ9gd5J9PNO+MfeTciLv3DkFivRMJiJptzAvYkwqP+vSL/BLPp
         m0RBKXFiUgfS4IGGAq38oIUgfXIBoWFuwSPBM+cTDPsg6po1gRg4Y82tKfBrgaDbk0S2
         tWQXws8i/HC80nBuVvjNB8TXchW0X3qTC4At0tn2cYwv4QH09as/pKvaKo2LtrQtaERk
         PYaY+np9FqXGXdDSFVQSeLljEsE/Kn5K56AQfvA5otqLLFblzh0s9Cg+b5zhdEjEti0C
         0LIg==
X-Gm-Message-State: AOJu0YzMAblhtdi/re9oe9Am391kVv1SkiOt0fXqMTo1N48rrvVnBqMI
	W4PCc1ydt4IjhbmUAx7W/rRPX5/OzkGfDIdzqo/IytLqr8KMeApX4YlUJOUDQ4c=
X-Google-Smtp-Source: AGHT+IHcA3ENrP5Sbnhq8D+gW9T469J7yAaO3FK18Tz14H0zR8tSOGxUxJ1+xntMF5Svn6IS9sVyMg==
X-Received: by 2002:a05:620a:2906:b0:790:8f24:51cc with SMTP id m6-20020a05620a290600b007908f2451ccmr2425178qkp.44.1714559985434;
        Wed, 01 May 2024 03:39:45 -0700 (PDT)
Message-ID: <c5efd98f-da49-414c-8427-8e4b2311f9bc@citrix.com>
Date: Wed, 1 May 2024 11:39:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/5] x86/cpu-policy: Add SVM features already used by Xen
To: George Dunlap <george.dunlap@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
 Andrei Semenov <andrei.semenov@vates.fr>,
 Vaishali Thakkar <vaishali.thakkar@vates.tech>
References: <20240429151625.977884-1-andrew.cooper3@citrix.com>
 <20240429151625.977884-3-andrew.cooper3@citrix.com>
 <CA+zSX=Y_fcDLonuRZZCaaKjK87tXEnnvJGjJHrKy5wzvaC2rzA@mail.gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <CA+zSX=Y_fcDLonuRZZCaaKjK87tXEnnvJGjJHrKy5wzvaC2rzA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01/05/2024 11:00 am, George Dunlap wrote:
> On Mon, Apr 29, 2024 at 4:16 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> These will replace svm_feature_flags and the SVM_FEATURE_* constants over the
>> next few changes.  Take the opportunity to rationalise some names.
>>
>> Drop the opencoded "inherit from host" logic in calculate_hvm_max_policy() and
>> use 'h'/'!' annotations.  The logic needs to operate on fs, not the policy
>> object, given its position within the function.
>>
>> Drop some trailing whitespace introduced when this block of code was last
>> moved.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Stefano Stabellini <sstabellini@kernel.org>
>> CC: Xenia Ragiadakou <xenia.ragiadakou@amd.com>
>> CC: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
>> CC: George Dunlap <george.dunlap@citrix.com>
>> CC: Andrei Semenov <andrei.semenov@vates.fr>
>> CC: Vaishali Thakkar <vaishali.thakkar@vates.tech>
>> ---
>>  tools/misc/xen-cpuid.c                      | 11 +++++++++++
>>  xen/arch/x86/cpu-policy.c                   | 17 +++++------------
>>  xen/include/public/arch-x86/cpufeatureset.h | 14 ++++++++++++++
>>  xen/tools/gen-cpuid.py                      |  3 +++
>>  4 files changed, 33 insertions(+), 12 deletions(-)
>>
>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>> index ab09410a05d6..0d01b0e797f1 100644
>> --- a/tools/misc/xen-cpuid.c
>> +++ b/tools/misc/xen-cpuid.c
>> @@ -266,6 +266,17 @@ static const char *const str_m10Ah[32] =
>>
>>  static const char *const str_eAd[32] =
>>  {
>> +    [ 0] = "npt",                 [ 1] = "v-lbr",
>> +    [ 2] = "svm-lock",            [ 3] = "nrips",
>> +    [ 4] = "v-tsc-rate",          [ 5] = "vmcb-cleanbits",
>> +    [ 6] = "flush-by-asid",       [ 7] = "decode-assist",
>> +
>> +    [10] = "pause-filter",
>> +    [12] = "pause-thresh",
>> +    /* 14 */                      [15] = "v-loadsave",
>> +    [16] = "v-gif",
>> +    /* 18 */                      [19] = "npt-sss",
>> +    [20] = "v-spec-ctrl",
>>  };
>>
>>  static const char *const str_e1Fa[32] =
>> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
>> index 4b6d96276399..da4401047e89 100644
>> --- a/xen/arch/x86/cpu-policy.c
>> +++ b/xen/arch/x86/cpu-policy.c
>> @@ -9,7 +9,6 @@
>>  #include <asm/amd.h>
>>  #include <asm/cpu-policy.h>
>>  #include <asm/hvm/nestedhvm.h>
>> -#include <asm/hvm/svm/svm.h>
>>  #include <asm/intel-family.h>
>>  #include <asm/msr-index.h>
>>  #include <asm/paging.h>
>> @@ -748,22 +747,16 @@ static void __init calculate_hvm_max_policy(void)
>>      if ( !cpu_has_vmx )
>>          __clear_bit(X86_FEATURE_PKS, fs);
>>
>> -    /*
>> +    /*
>>       * Make adjustments to possible (nested) virtualization features exposed
>>       * to the guest
>>       */
>> -    if ( p->extd.svm )
>> +    if ( test_bit(X86_FEATURE_SVM, fs) )
>>      {
>> -        /* Clamp to implemented features which require hardware support. */
>> -        p->extd.raw[0xa].d &= ((1u << SVM_FEATURE_NPT) |
>> -                               (1u << SVM_FEATURE_LBRV) |
>> -                               (1u << SVM_FEATURE_NRIPS) |
>> -                               (1u << SVM_FEATURE_PAUSEFILTER) |
>> -                               (1u << SVM_FEATURE_DECODEASSISTS));
>> -        /* Enable features which are always emulated. */
>> -        p->extd.raw[0xa].d |= (1u << SVM_FEATURE_VMCBCLEAN);
>> +        /* Xen always emulates cleanbits. */
>> +        __set_bit(X86_FEATURE_VMCB_CLEANBITS, fs);
>>      }
> What about this line, at the end of recalculate_cpuid_policy()?
>
>   if ( !p->extd.svm )
>         p->extd.raw[0xa] = EMPTY_LEAF;
>
> Is there a reason to continue to operate directly on the policy here,
> or would it be better to do this up in the featureset section?

That's still needed.

Otherwise in a !SVM VM you still see svm_rev and nr_asids in a leaf that
should be all zeroes.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 01 10:51:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 10:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715287.1116818 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s27Yj-0005kk-L4; Wed, 01 May 2024 10:51:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715287.1116818; Wed, 01 May 2024 10:51:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s27Yj-0005kd-IJ; Wed, 01 May 2024 10:51:33 +0000
Received: by outflank-mailman (input) for mailman id 715287;
 Wed, 01 May 2024 10:51:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ji+M=ME=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1s27Yj-0005kX-1s
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 10:51:33 +0000
Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com
 [2001:4860:4864:20::29])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c4e7f3fb-07a8-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 12:51:31 +0200 (CEST)
Received: by mail-oa1-x29.google.com with SMTP id
 586e51a60fabf-23d23a6123eso535361fac.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 03:51:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c4e7f3fb-07a8-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714560690; x=1715165490; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7tA0zl5xuECP+2FZlON78Z8DHZiCEpbBPmvXhYGexKI=;
        b=C8Iai2J9Vkxi31j9kzqH6dDSN19TfWg/a4NHCZ/PN3TIi9lZGfZw4N/QFO19vhBLXw
         oiGwGKSX97R7ZFKd8O5DT6n0u2XWNhq3ie/lvRg9Yk9kATLJvpp5U+jv/1Ey6rRsm21z
         3q7fne2ZLkCfmA4JdIOwZrlOdpapWDesGkCQE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714560690; x=1715165490;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7tA0zl5xuECP+2FZlON78Z8DHZiCEpbBPmvXhYGexKI=;
        b=HrrMbKeuFSRXC0nTLCXqB5+o+rlXU7uZZIk03zL/ITMf3sZBhSl/Dth8NO8dVe1EnZ
         8b/voH2ytrlxZAM2bvjKUIYt/Cipr8+hwmrur9Ms44VjI/tNfXZGISKhIGcxcjY3bc14
         IMFNLwYK/QnAtpoM4T2MQSRWHupb/VyqUiv/i1cEBDVLn/pthr6xbBqBAudO/mimWKPw
         AAiuMSD+cMJPhLh8ICIpGObSG2Gs3KP1fN0qdGG9qn6TzjBNBqQeDFEfMzDvedG2pu9F
         nf2WKEhRBXT0REgDUhIbv7gdYrg70GkPqmw9N3zJAn9mpxN2ZcWM/M/bZV0Po7bS8jPy
         Zw2A==
X-Gm-Message-State: AOJu0YwVpirUsRxJTsZC68wAxNYM79FlXk6uElbnQ/wDDS/dQvUN4Z2m
	tBjw0K/r4WhtqE3sTJdEsZwwLJYFDCuZUS8nMaCWZRcwo0eQZykjTDsxyOywhd3HKkLw7FKSjUQ
	pqwfr+KVlg3wtdtVmxbJFQkKItXDH8Ajuq1/cqQ==
X-Google-Smtp-Source: AGHT+IHnvid0r7b+lj7dBLs1TJun5tYlzTQASrYzxqlIeOFAxZICtgD+5ZtFMh7qCSeAKaRbJoIa1I+YXjLGn2/2khg=
X-Received: by 2002:a05:6870:6128:b0:229:f988:4305 with SMTP id
 s40-20020a056870612800b00229f9884305mr2321194oae.10.1714560690585; Wed, 01
 May 2024 03:51:30 -0700 (PDT)
MIME-Version: 1.0
References: <20240429151625.977884-1-andrew.cooper3@citrix.com>
 <20240429151625.977884-3-andrew.cooper3@citrix.com> <CA+zSX=Y_fcDLonuRZZCaaKjK87tXEnnvJGjJHrKy5wzvaC2rzA@mail.gmail.com>
 <c5efd98f-da49-414c-8427-8e4b2311f9bc@citrix.com>
In-Reply-To: <c5efd98f-da49-414c-8427-8e4b2311f9bc@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 1 May 2024 11:51:19 +0100
Message-ID: <CA+zSX=aVb2znDf5XZ+04_vTD2aRPF8t4XoO9ZicYeU0_A2hsMQ@mail.gmail.com>
Subject: Re: [PATCH 2/5] x86/cpu-policy: Add SVM features already used by Xen
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Jan Beulich <JBeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>, Andrei Semenov <andrei.semenov@vates.fr>, 
	Vaishali Thakkar <vaishali.thakkar@vates.tech>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 1, 2024 at 11:39=E2=80=AFAM Andrew Cooper <andrew.cooper3@citri=
x.com> wrote:
>
> On 01/05/2024 11:00 am, George Dunlap wrote:
> > On Mon, Apr 29, 2024 at 4:16=E2=80=AFPM Andrew Cooper <andrew.cooper3@c=
itrix.com> wrote:
> >> These will replace svm_feature_flags and the SVM_FEATURE_* constants o=
ver the
> >> next few changes.  Take the opportunity to rationalise some names.
> >>
> >> Drop the opencoded "inherit from host" logic in calculate_hvm_max_poli=
cy() and
> >> use 'h'/'!' annotations.  The logic needs to operate on fs, not the po=
licy
> >> object, given its position within the function.
> >>
> >> Drop some trailing whitespace introduced when this block of code was l=
ast
> >> moved.
> >>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >> ---
> >> CC: Jan Beulich <JBeulich@suse.com>
> >> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> >> CC: Stefano Stabellini <sstabellini@kernel.org>
> >> CC: Xenia Ragiadakou <xenia.ragiadakou@amd.com>
> >> CC: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> >> CC: George Dunlap <george.dunlap@citrix.com>
> >> CC: Andrei Semenov <andrei.semenov@vates.fr>
> >> CC: Vaishali Thakkar <vaishali.thakkar@vates.tech>
> >> ---
> >>  tools/misc/xen-cpuid.c                      | 11 +++++++++++
> >>  xen/arch/x86/cpu-policy.c                   | 17 +++++------------
> >>  xen/include/public/arch-x86/cpufeatureset.h | 14 ++++++++++++++
> >>  xen/tools/gen-cpuid.py                      |  3 +++
> >>  4 files changed, 33 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> >> index ab09410a05d6..0d01b0e797f1 100644
> >> --- a/tools/misc/xen-cpuid.c
> >> +++ b/tools/misc/xen-cpuid.c
> >> @@ -266,6 +266,17 @@ static const char *const str_m10Ah[32] =3D
> >>
> >>  static const char *const str_eAd[32] =3D
> >>  {
> >> +    [ 0] =3D "npt",                 [ 1] =3D "v-lbr",
> >> +    [ 2] =3D "svm-lock",            [ 3] =3D "nrips",
> >> +    [ 4] =3D "v-tsc-rate",          [ 5] =3D "vmcb-cleanbits",
> >> +    [ 6] =3D "flush-by-asid",       [ 7] =3D "decode-assist",
> >> +
> >> +    [10] =3D "pause-filter",
> >> +    [12] =3D "pause-thresh",
> >> +    /* 14 */                      [15] =3D "v-loadsave",
> >> +    [16] =3D "v-gif",
> >> +    /* 18 */                      [19] =3D "npt-sss",
> >> +    [20] =3D "v-spec-ctrl",
> >>  };
> >>
> >>  static const char *const str_e1Fa[32] =3D
> >> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
> >> index 4b6d96276399..da4401047e89 100644
> >> --- a/xen/arch/x86/cpu-policy.c
> >> +++ b/xen/arch/x86/cpu-policy.c
> >> @@ -9,7 +9,6 @@
> >>  #include <asm/amd.h>
> >>  #include <asm/cpu-policy.h>
> >>  #include <asm/hvm/nestedhvm.h>
> >> -#include <asm/hvm/svm/svm.h>
> >>  #include <asm/intel-family.h>
> >>  #include <asm/msr-index.h>
> >>  #include <asm/paging.h>
> >> @@ -748,22 +747,16 @@ static void __init calculate_hvm_max_policy(void=
)
> >>      if ( !cpu_has_vmx )
> >>          __clear_bit(X86_FEATURE_PKS, fs);
> >>
> >> -    /*
> >> +    /*
> >>       * Make adjustments to possible (nested) virtualization features =
exposed
> >>       * to the guest
> >>       */
> >> -    if ( p->extd.svm )
> >> +    if ( test_bit(X86_FEATURE_SVM, fs) )
> >>      {
> >> -        /* Clamp to implemented features which require hardware suppo=
rt. */
> >> -        p->extd.raw[0xa].d &=3D ((1u << SVM_FEATURE_NPT) |
> >> -                               (1u << SVM_FEATURE_LBRV) |
> >> -                               (1u << SVM_FEATURE_NRIPS) |
> >> -                               (1u << SVM_FEATURE_PAUSEFILTER) |
> >> -                               (1u << SVM_FEATURE_DECODEASSISTS));
> >> -        /* Enable features which are always emulated. */
> >> -        p->extd.raw[0xa].d |=3D (1u << SVM_FEATURE_VMCBCLEAN);
> >> +        /* Xen always emulates cleanbits. */
> >> +        __set_bit(X86_FEATURE_VMCB_CLEANBITS, fs);
> >>      }
> > What about this line, at the end of recalculate_cpuid_policy()?
> >
> >   if ( !p->extd.svm )
> >         p->extd.raw[0xa] =3D EMPTY_LEAF;
> >
> > Is there a reason to continue to operate directly on the policy here,
> > or would it be better to do this up in the featureset section?
>
> That's still needed.
>
> Otherwise in a !SVM VM you still see svm_rev and nr_asids in a leaf that
> should be all zeroes.

...Uh, yes of course we still need to clear the non-existent CPUID
bits.  I was asking if we should change *how* we should clear them.

In the hunk I responded to, when enabling VMCBCLEAN, we switch from
setting bits in the policy, to setting bits in the featureset.  I was
asking whether it would make sense to do something like

    if !test_bit(X86_FEATURE_SVM, fs)
        fs[FEATURESET_eAd]  =3D 0;

before the x86_cpu_featureset_to_policy() instead.

 -George


From xen-devel-bounces@lists.xenproject.org Wed May 01 12:15:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 12:15:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715297.1116830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s28rF-0007yW-DO; Wed, 01 May 2024 12:14:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715297.1116830; Wed, 01 May 2024 12:14:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s28rF-0007yP-8V; Wed, 01 May 2024 12:14:45 +0000
Received: by outflank-mailman (input) for mailman id 715297;
 Wed, 01 May 2024 12:14:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s28rE-0007yF-Nt; Wed, 01 May 2024 12:14:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s28rE-0006Nm-LL; Wed, 01 May 2024 12:14:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s28rE-0000ds-4z; Wed, 01 May 2024 12:14:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s28rE-0007aj-4P; Wed, 01 May 2024 12:14:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=twnzhJTQ1YWwlEm/tcZNlHG8ctIskDHh42TxVR8vEHw=; b=vvN97na3tG1Nu4sCvBY7MjbPvq
	oYisCYBL6VwKGrG0BqMOut2u0/ViMWcV1iSjXrVOzXA+MbHhtQ0xoSNglOk2OSwhift5iy0hXNs+d
	seflTt/eNsqEw/yV/+37ZIkUMQpBCZgd/pH7f0fGp5MyEoCjx8nBukKUg2qWn/xOnQnU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185888-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185888: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=18daea77cca626f590fb140fc11e3a43c5d41354
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 12:14:44 +0000

flight 185888 linux-linus real [real]
flight 185892 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185888/
http://logs.test-lab.xenproject.org/osstest/logs/185892/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 185870

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185870

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185870
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                18daea77cca626f590fb140fc11e3a43c5d41354
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    1 days
Testing same since   185888  2024-04-30 20:31:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Potapenko <glider@google.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Audra Mitchell <audra@redhat.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Chuck Lever <chuck.lever@oracle.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Hans de Goede <hdegoede@redhat.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  John Garry <john.g.garry@oracle.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Sven Schnelle <svens@linux.ibm.com>
  Tejun Heo <tj@kernel.org>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Xingyou Chen <rockrush@rockwork.org>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1064 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed May 01 13:29:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 13:29:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715310.1116838 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2A1M-000853-MR; Wed, 01 May 2024 13:29:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715310.1116838; Wed, 01 May 2024 13:29:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2A1M-00084w-Jt; Wed, 01 May 2024 13:29:16 +0000
Received: by outflank-mailman (input) for mailman id 715310;
 Wed, 01 May 2024 13:29:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RAI4=ME=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1s2A1K-00084q-Pb
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 13:29:14 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cbe81874-07be-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 15:29:13 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-349545c3eb8so4419563f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 06:29:12 -0700 (PDT)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 u12-20020a5d434c000000b0034ad657deccsm27440612wrr.71.2024.05.01.06.29.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 06:29:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbe81874-07be-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714570151; x=1715174951; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=y/T26vGuVC3gvXPmvBK68ArxSnuFJumdSZ5Y78/YZ0Y=;
        b=u4ozEo9YtUYo5obRgCYh9UNP0Db0TB6HDxFrkF962XH9HxRoR6pZT1B8DUHUM113Ov
         IJ+T/S5jqW9CqbUTIArJNGvM0CfuN1h8L5walKOmdZN7FMvwvpNZmpE75m9ueatK1+gh
         JvCaAZjgyG4MdjlYrxzbptpm8PU17sG5450KY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714570151; x=1715174951;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=y/T26vGuVC3gvXPmvBK68ArxSnuFJumdSZ5Y78/YZ0Y=;
        b=qC/zcFFKuhmR0aQb9t0YEatPo+I3Diio/jSRYWB26RqnF6MMwKvwpjUn3z0FbqbT+9
         /M+eHU0HNSFl06SYB4CNQiKljy9B6LyM1ZbVU9olSOUPDo40vW+TkuCAs6FrdJvjD9a7
         2iDeagrHWSan8xsEKCgtk4jL8j1RULLyAzA5ywqMuFm4MGt4MVlexVnR1PST7WqSHtmd
         1oUvP68Igvlgemf+HVjRjxNFgv4EkIGnxcBz+DFqDRKKd4N/HJNdDrz/QNl6+MeQA9zV
         50xzY92UwOwN/kaS2qM805OwBF7x6FkaQO2CZUBHqg+hfKEHVA95V4sQ0RmWg7dUvVE7
         JxfQ==
X-Gm-Message-State: AOJu0YwHiGhdVMQ+4R+WQHu7h/MJRwVw6SnbkucZK8I04fm28WUpkn8x
	2n5N+PUVUZTZo1V8pQmC7FTPW5kgp7RQrr2KktELBJt/BP3remAl+/Os95t3pSI=
X-Google-Smtp-Source: AGHT+IG48C/kl2KyLRmfWD0XUwlGKTxmp3GiM2I5RgiGFkb2ABOECmMNxJfVgF+yw6uMJesgjCEpTQ==
X-Received: by 2002:a5d:490e:0:b0:34c:5a37:b90c with SMTP id x14-20020a5d490e000000b0034c5a37b90cmr1891139wrq.58.1714570151296;
        Wed, 01 May 2024 06:29:11 -0700 (PDT)
Date: Wed, 1 May 2024 14:29:10 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	Edwin =?iso-8859-1?B?VPZy9ms=?= <edwin.torok@cloud.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 1/2] tools/{c,o}xenstored: Don't link against libsystemd
Message-ID: <83160ad7-19b1-496f-a560-c11db67b1caf@perard>
References: <20240425173216.410940-1-andrew.cooper3@citrix.com>
 <20240425173216.410940-2-andrew.cooper3@citrix.com>
 <b5e242fa-daf1-43a2-afd2-cc1ad1bd4dc1@perard>
 <341c5901-254e-4ad2-b935-6b586cd25f2e@citrix.com>
 <413a63de-07ec-43a9-ae60-6b0cfdd61312@perard>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <413a63de-07ec-43a9-ae60-6b0cfdd61312@perard>

On Fri, Apr 26, 2024 at 09:51:47AM +0100, Anthony PERARD wrote:
> Run `systemd-notify --ready` instead. Hopefully, that will be enough.
> ($NOTIFY_SOCKET is a socket, and a bit more complicated that I though,
> it can start with "@" for example)

FTR: If it turns out that calling systemd-notify binary isn't working
well enough, we could have an implementation of sd_notify() in our tree,
openssh are doing there own here:
https://bugzilla.mindrot.org/show_bug.cgi?id=2641
and there's an example implementation on systemd's documentation:
https://www.freedesktop.org/software/systemd/man/devel/sd_notify.html#Notes
(Nothing for ocaml)

But let's go with `systemd-notify --ready` as it is just easier to
write a bit of shell script.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed May 01 13:48:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 13:48:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715315.1116848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2AJS-0002ca-6X; Wed, 01 May 2024 13:47:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715315.1116848; Wed, 01 May 2024 13:47:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2AJS-0002cT-3i; Wed, 01 May 2024 13:47:58 +0000
Received: by outflank-mailman (input) for mailman id 715315;
 Wed, 01 May 2024 13:47:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RAI4=ME=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1s2AJQ-0002cM-RT
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 13:47:56 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6965ef4b-07c1-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 15:47:55 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-34da35cd01cso1247709f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 06:47:55 -0700 (PDT)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d15-20020adffbcf000000b0034b1bd76d30sm24688229wrs.28.2024.05.01.06.47.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 06:47:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6965ef4b-07c1-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714571274; x=1715176074; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=NRtw5x79fRbqV4pf16UTLA4er6R06VdHd7ndfBFJApw=;
        b=B/clO00Lm3s/pGVNYYruFyLviDyzVyn5a7zVwQjbWe3zU2Qnz3miT96Eq1agEe0h3+
         ksr4nmm96rqvPh1Lp9YHByc0oxtAcmCnYm+WYxGKOJP50/0eJP19uzdP+BxYafvV/qhA
         2K+IfAwTD0g7ylPpZheXxOjroA1dPCQDUui+g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714571274; x=1715176074;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NRtw5x79fRbqV4pf16UTLA4er6R06VdHd7ndfBFJApw=;
        b=oCbCUvdT6guAyP2hVZRGWoSO1DU6J9/R/Fkl0S7Oy5lTiC+4Ux4AWmscxElcU7tFMh
         CNPHGxQ8NdxcJKZomcXhS5gjX48iY/7HwwAZjZrBIAt0Bfn6xwbeWAWWzVSgREtTvjEw
         pN0ZcozhDDvevGxFWUFvG+0984gdHa8STW97sMBeoK7Y6zyUEzSnmbfig2kAUQbbjQMl
         QISgutjHqWja+nwsn39HByY1N3s4grZPaM1HA+MpUY1CY9cjicsKR/k60xYjppbar5T8
         /SffvWxT2SH5KX2eduKdXz2CSN2nud339mYihJPUbq5wKYDPpK7fnPr2xs6GTWu8PJXf
         IEtw==
X-Gm-Message-State: AOJu0Ywuzcb5dDliGvVloQSHFLyG5ghflLPywctonhLWT8oBRLjeXeuI
	sfmFBT80SWb6zQYsr5HwWSqChgk0/kM35jPbLaVIAxhxOS76YMhOWl2XUdRGq7w=
X-Google-Smtp-Source: AGHT+IEwSGHCd8GGkObAIKSeA8QCTrYgxO2jhQyGfc5hLY1cak7d/bebRnkG3Hs202gTq4sqeWgxZA==
X-Received: by 2002:a5d:4d42:0:b0:34d:b43d:6e24 with SMTP id a2-20020a5d4d42000000b0034db43d6e24mr2353184wru.32.1714571274562;
        Wed, 01 May 2024 06:47:54 -0700 (PDT)
Date: Wed, 1 May 2024 14:47:53 +0100
From: Anthony PERARD <anthony.perard@cloud.com>
To: Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 04/15] tools/libs/light: Always enable IOMMU
Message-ID: <7336a5a0-9df0-4e90-846f-9c1647ba5319@perard>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-5-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240424033449.168398-5-xin.wang2@amd.com>

On Wed, Apr 24, 2024 at 11:34:38AM +0800, Henry Wang wrote:
> For overlay with iommu functionality to work with running VMs, we need
> to enable IOMMU when iomem presents for the domains.
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>  tools/libs/light/libxl_arm.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 1cb89fa584..dd5c9f4917 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -222,6 +222,12 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>          config->arch.sve_vl = d_config->b_info.arch_arm.sve_vl / 128U;
>      }
>  
> +#ifdef LIBXL_HAVE_DT_OVERLAY

libxl_arm.c is only build on Arm, so this should be defined, so no need
to check.

> +    if (d_config->b_info.num_iomem) {
> +        config->flags |= XEN_DOMCTL_CDF_iommu;

Is this doing the same thing as the previous patch?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed May 01 13:58:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 13:58:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715320.1116859 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ATK-0004S2-0R; Wed, 01 May 2024 13:58:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715320.1116859; Wed, 01 May 2024 13:58:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ATJ-0004Rv-Tl; Wed, 01 May 2024 13:58:09 +0000
Received: by outflank-mailman (input) for mailman id 715320;
 Wed, 01 May 2024 13:58:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RAI4=ME=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1s2ATI-0004Rp-Po
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 13:58:08 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d67ef559-07c2-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 15:58:07 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a55b93f5540so930400366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 06:58:07 -0700 (PDT)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 t8-20020a170906a10800b00a4e253c8735sm16455948ejy.52.2024.05.01.06.58.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 06:58:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d67ef559-07c2-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714571887; x=1715176687; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=eUIicldlVO0xadZzH9Mvl0lxsvG1olhmR5vt5CaN7Q8=;
        b=Tr6gWOIRes/PtO5vgoXJQMjFT45nprGufxcmRZAqwrQVy8ZuvtW9vOBfoQiwJ/FmRL
         2BOIKq+CJ4qzqi3eK+/kmXm8rcSN/twfnE1tdr6iVYVPLouksMnSmaVWdQnX+qsQIuSK
         QRJudQRWYP2WOUrzggPwIvIv1Fb0xvP8Gr4C4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714571887; x=1715176687;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eUIicldlVO0xadZzH9Mvl0lxsvG1olhmR5vt5CaN7Q8=;
        b=YHh99nhp0tksckO9OVAs6kPhmnhDrepc6z01FXetAa2uId6SichAn2qyrV1VVUzRae
         P4hqtyS0QN9h8vJEvhRwQ/UFgc1SLi5vGmYS+Q5f2PbCF+6j/kl38Li2bKPo7UbUdYky
         cfGeDK0GarmgD96cXeDGh4RIdzyScMWEhYS/xAE1TvBFtGxGnzueA+mBSIF803D1KKQ/
         OkIJmKUUKtA811n/UUrtFSL0+JqlFoYzDHuZv/FqQxnBgRPh+oOYYHH0/4c+PGCGBGZq
         7wMkeuCH9giFCMWQRwPw8bM0llNJnvHJdc0zdzpSdNvbLkRotdS8eSYJcp5GqXZd/ipN
         CgMQ==
X-Gm-Message-State: AOJu0Yyh9x8ozRkT3j+WhD0tTzU+5EBxvEny8X6iH2BIbk/IhrXxAvjE
	nq6hFay30X0HXgBck44i1cSMBQORxY/c9OC1HHqWn0+X8Q3HCiXbQ0jSkledPheHUCZI5ttbIk7
	P
X-Google-Smtp-Source: AGHT+IFVvWK+ZoBVDDXTaL9t7iSGqxcwwq2EcxwXkhTVqSaccOukO48NCYGmP27jv28x2mRIHiACww==
X-Received: by 2002:a17:906:f8c9:b0:a55:b3e2:718f with SMTP id lh9-20020a170906f8c900b00a55b3e2718fmr1889820ejb.23.1714571887128;
        Wed, 01 May 2024 06:58:07 -0700 (PDT)
Date: Wed, 1 May 2024 14:58:05 +0100
From: Anthony PERARD <anthony.perard@cloud.com>
To: Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Re: [PATCH 05/15] tools/libs/light: Increase nr_spi to 160
Message-ID: <334eb040-234f-4492-9006-9faadfe28c59@perard>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-6-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240424033449.168398-6-xin.wang2@amd.com>

On Wed, Apr 24, 2024 at 11:34:39AM +0800, Henry Wang wrote:
> Increase number of spi to 160 i.e. gic_number_lines() for Xilinx ZynqMP - 32.
> This was done to allocate and assign IRQs to a running domain.
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>  tools/libs/light/libxl_arm.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index dd5c9f4917..50dbd0f2a9 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -181,7 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>  
>      LOG(DEBUG, "Configure the domain");
>  
> -    config->arch.nr_spis = nr_spis;
> +    /* gic_number_lines() is 192 for Xilinx ZynqMP. min nr_spis = 192 - 32. */
> +    config->arch.nr_spis = MAX(nr_spis, 160);

Is there a way that that Xen or libxl could find out what the minimum
number of SPI needs to be? Are we going to have to increase that minimum
number every time a new platform comes along?

It doesn't appear that libxl is using that `nr_spis` value and it is
probably just given to Xen. So my guess is that Xen could simply take
care of the minimum value, gic_number_lines() seems to be a Xen
function.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed May 01 14:04:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 14:04:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715326.1116869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2AZ4-00069m-Mk; Wed, 01 May 2024 14:04:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715326.1116869; Wed, 01 May 2024 14:04:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2AZ4-00069f-Ir; Wed, 01 May 2024 14:04:06 +0000
Received: by outflank-mailman (input) for mailman id 715326;
 Wed, 01 May 2024 14:04:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f0p4=ME=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2AZ3-00069Z-RY
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 14:04:05 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aa7a9de2-07c3-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 16:04:03 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-78f05341128so473794185a.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 07:04:03 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k4-20020a0cc784000000b006a0cb461f89sm3150190qvj.6.2024.05.01.07.04.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 07:04:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa7a9de2-07c3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714572243; x=1715177043; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=iMxejeIzjvoaJbOylrVM2d+rDBZmPm9tSSUkJk06FUA=;
        b=lx8kdjr50KpsNbHpRO78vnh1hPTMpo+26JF7V+RgQEz9fPf/Dm8ljgO+zKPybNzIap
         L3VXWIe21tO4cAk8lUdPfDct3aCkzZvFOqRg/rSjDxONhcCOdxi/FD32l6o4w7u4DBm8
         ZlUJu7HfA4qqnEAZwayCW//7Xjh7OR+We9THc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714572243; x=1715177043;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iMxejeIzjvoaJbOylrVM2d+rDBZmPm9tSSUkJk06FUA=;
        b=fGMIytXHHBEwsVyX92lduZaSeARAQa04IhkidiOsaTqM0irkCVBwlDOW7EQ1yyxqMA
         9ualO5B5IjG0adYeTUSJy3KKyhXq6SvNbgW/aZ5XByWibNH6VjFw7Y/N2RtVKkOtCmTp
         q4z5bDYHZAS85iI+BIg2bSbtS82N0alfFRv/k2hmTw3sY8NDDoOA4I4F68OG+a43hFnH
         JiTLjVl0bcP6nHQGc1+7YzN/aK591GyAg/4mF1TjFtqHJsBpSufFQWRbSNyMN10+GB8D
         FIu9seG6qqEGo5pE9P047wGytc3U2vF5ESnh4l7XjRtdWdvf33L835rh5bunWpoIsmD0
         9MgA==
X-Gm-Message-State: AOJu0Yw19vzOmIuXqrt2VRnUZ9hvUMhkpCB6LlC/pHZxaDKD+y5K3n+F
	9elB2GMXzKCKtM0CBuy20rfOyYWmg8arAVq+5VJ8m/JxTNGuVzg4Yi6F/R9+LaArS6iV9adlXDm
	p
X-Google-Smtp-Source: AGHT+IFswrEyF7UFjObQ/mTFSeZCz1PMV7DTg3l8LmAjjldKRt39C8khdkwldLKMZBsKsxvoWTXH9g==
X-Received: by 2002:ad4:5c8d:0:b0:6a0:8225:d89e with SMTP id o13-20020ad45c8d000000b006a08225d89emr3108746qvh.25.1714572242690;
        Wed, 01 May 2024 07:04:02 -0700 (PDT)
Message-ID: <dcb9a57f-b002-46bd-a7dc-1cd580a9ab69@citrix.com>
Date: Wed, 1 May 2024 15:04:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] tools/{c,o}xenstored: Don't link against libsystemd
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240425173216.410940-1-andrew.cooper3@citrix.com>
 <20240425173216.410940-2-andrew.cooper3@citrix.com>
 <b5e242fa-daf1-43a2-afd2-cc1ad1bd4dc1@perard>
 <341c5901-254e-4ad2-b935-6b586cd25f2e@citrix.com>
 <413a63de-07ec-43a9-ae60-6b0cfdd61312@perard>
 <83160ad7-19b1-496f-a560-c11db67b1caf@perard>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <83160ad7-19b1-496f-a560-c11db67b1caf@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01/05/2024 2:29 pm, Anthony PERARD wrote:
> On Fri, Apr 26, 2024 at 09:51:47AM +0100, Anthony PERARD wrote:
>> Run `systemd-notify --ready` instead. Hopefully, that will be enough.
>> ($NOTIFY_SOCKET is a socket, and a bit more complicated that I though,
>> it can start with "@" for example)
> FTR: If it turns out that calling systemd-notify binary isn't working
> well enough, we could have an implementation of sd_notify() in our tree,
> openssh are doing there own here:
> https://bugzilla.mindrot.org/show_bug.cgi?id=2641
> and there's an example implementation on systemd's documentation:
> https://www.freedesktop.org/software/systemd/man/devel/sd_notify.html#Notes
> (Nothing for ocaml)
>
> But let's go with `systemd-notify --ready` as it is just easier to
> write a bit of shell script.

I was already thinking of going down the small-library-function route.

Given that I miss-analysed the launch-xenstore, script, I'm not overly
enthused with just falling back to waiting on the pidfile, because
that's adding technical debt rather than removing it.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 01 14:46:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 14:46:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715333.1116880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2BE7-0003oq-Rj; Wed, 01 May 2024 14:46:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715333.1116880; Wed, 01 May 2024 14:46:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2BE7-0003oj-NF; Wed, 01 May 2024 14:46:31 +0000
Received: by outflank-mailman (input) for mailman id 715333;
 Wed, 01 May 2024 14:46:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RAI4=ME=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1s2BE6-0003od-Qf
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 14:46:30 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9769909e-07c9-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 16:46:28 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e0a2870bceso41471691fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 07:46:28 -0700 (PDT)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l2-20020a05600c4f0200b0041bf87e702asm2416912wmq.10.2024.05.01.07.46.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 07:46:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9769909e-07c9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714574788; x=1715179588; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=I4ojWbgGKeLlz2RX8smFFr1xJq/YLUKQi3LRjZQ4IgY=;
        b=fF9BX+Euvyz6LiVndbFB+DQmblGHCRTb4OFdZSKYBwzOYGrWOy2qKq0Re1noimF+5P
         PVJ3pNXvV4IFGXLTBip7SqGcPzUTlsM7TuXoz42L//m2LavfIx3WX1hY55onviabIMkp
         9syAopf5gBOIVhyDGdV+/aJhf2rozgheH9KaE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714574788; x=1715179588;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=I4ojWbgGKeLlz2RX8smFFr1xJq/YLUKQi3LRjZQ4IgY=;
        b=WPQ0dAaQ8eMbtAajMIIg96tltnj29Ba/qmUc+0RLmQJa7SnZoTjYoM6d4noIjQVhSY
         2j1A1gCIy1PSEkodJfPtQU5Tols8PSyyYfEgdpkzyG3MJSHOMwikuDLn0tr5LRXT3R6M
         A9cvkdONvjoIZ978vAA2rfPxRG8flHAFL4X0CZinAtZJt5iJ/mogWd5jpiA4e9lWpzey
         Xqo/PG9Ueok8tMC8mMaBtFk8RV3A6o4lKUa7ck5tDY5Qg9O+WYhh/Sx2yjzF9A6XeUEZ
         4W+5MeGWp9NpdiDMoml5aaJb1zXMgaUHb9rAvzT68CmqeuiPsQMgWx5398TyDJv+U6XW
         bRvg==
X-Gm-Message-State: AOJu0YwaoScV7MeLbGtxUC/7Pekb7AG1XyxV8EL6Oc+P/l73euVOTvUl
	+Vb3nvxuwVjkb6N7L+EFUyzh3Bsdf46Ewg1mq58v+8FGsebYbB9ldq2VGO27FAsFFboOfUBLagS
	k
X-Google-Smtp-Source: AGHT+IHcq4PJMeZBL+hUyz++eAPEuutXJJmWdYVboovCrPtK61FURQnmMKjxVwuK7+8IqZvb5onRyA==
X-Received: by 2002:a2e:a26a:0:b0:2df:ee50:3c35 with SMTP id k10-20020a2ea26a000000b002dfee503c35mr1860601ljm.0.1714574785728;
        Wed, 01 May 2024 07:46:25 -0700 (PDT)
Date: Wed, 1 May 2024 15:46:24 +0100
From: Anthony PERARD <anthony.perard@cloud.com>
To: Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Re: [PATCH 08/15] tools: Add domain_id and expert mode for overlay
 operations
Message-ID: <eaea1986-a27e-4d6c-932f-1d0a9918861f@perard>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-9-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240424033449.168398-9-xin.wang2@amd.com>

On Wed, Apr 24, 2024 at 11:34:42AM +0800, Henry Wang wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
> 
> Add domain_id and expert mode for overlay assignment. This enables dynamic
> programming of nodes during runtime.
> 
> Take the opportunity to fix the name mismatch in the xl command, the
> command name should be "dt-overlay" instead of "dt_overlay".

I don't like much these unrelated / opportunistic changes in a patch,
I'd rather have a separate patch. And in this case, if it was on a
separate patch, that separated patch could gain: Fixes: 61765a07e3d8
("tools/xl: Add new xl command overlay for device tree overlay support")
and potentially backported.

> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>  tools/include/libxl.h               |  8 +++++--
>  tools/include/xenctrl.h             |  5 +++--
>  tools/libs/ctrl/xc_dt_overlay.c     |  7 ++++--
>  tools/libs/light/libxl_dt_overlay.c | 17 +++++++++++----
>  tools/xl/xl_vmcontrol.c             | 34 ++++++++++++++++++++++++++---
>  5 files changed, 58 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index 62cb07dea6..59a3e1b37c 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -2549,8 +2549,12 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
>  void libxl_device_pci_list_free(libxl_device_pci* list, int num);
>  
>  #if defined(__arm__) || defined(__aarch64__)
> -int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
> -                     uint32_t overlay_size, uint8_t overlay_op);
> +#define LIBXL_DT_OVERLAY_ADD                   1
> +#define LIBXL_DT_OVERLAY_REMOVE                2
> +
> +int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay,
> +                     uint32_t overlay_size, uint8_t overlay_op, bool auto_mode,
> +                     bool domain_mapping);

Sorry, you cannot change the API of an existing libxl function without
providing something backward compatible. We have already a few example
of this changes in libxl.h, e.g.: fded24ea8315 ("libxl: Make libxl_set_vcpuonline async")
So, providing a wrapper called libxl_dt_overlay_0x041800() which call
the new function.

>  #endif
>  
>  /*
> diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
> index a6c709a6dc..cdb62b28cf 100644
> --- a/tools/libs/light/libxl_dt_overlay.c
> +++ b/tools/libs/light/libxl_dt_overlay.c
> @@ -57,10 +58,18 @@ int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
>          rc = 0;
>      }
>  
> -    r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
> +    /* Check if user entered a valid domain id. */
> +    rc = libxl_domain_info(CTX, NULL, domid);
> +    if (rc == ERROR_DOMAIN_NOTFOUND) {

Why do you check specifically for "domain not found", what about other
error?

> +        LOGD(ERROR, domid, "Non-existant domain.");
> +        return ERROR_FAIL;

Use `goto out`, and you can let the function return
ERROR_DOMAIN_NOTFOUND if that the error, we can just propagate the `rc`
from libxl_domain_info().

> +    }
> +
> +    r = xc_dt_overlay(ctx->xch, domid, overlay_dt, overlay_dt_size, overlay_op,
> +                      domain_mapping);
>  
>      if (r) {
> -        LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
> +        LOG(ERROR, "domain%d: Adding/Removing overlay dtb failed.", domid);

You could replace the macro by LOGD, instead of handwriting "domain%d".

>          rc = ERROR_FAIL;
>      }
>  
> diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
> index 98f6bd2e76..9674383ec3 100644
> --- a/tools/xl/xl_vmcontrol.c
> +++ b/tools/xl/xl_vmcontrol.c
> @@ -1270,21 +1270,48 @@ int main_dt_overlay(int argc, char **argv)
>  {
>      const char *overlay_ops = NULL;
>      const char *overlay_config_file = NULL;
> +    uint32_t domain_id = 0;
>      void *overlay_dtb = NULL;
>      int rc;
> +    bool auto_mode = true;
> +    bool domain_mapping = false;
>      uint8_t op;
>      int overlay_dtb_size = 0;
>      const int overlay_add_op = 1;
>      const int overlay_remove_op = 2;
>  
> -    if (argc < 2) {
> -        help("dt_overlay");
> +    if (argc < 3) {
> +        help("dt-overlay");
>          return EXIT_FAILURE;
>      }
>  
> +    if (argc > 5) {
> +        fprintf(stderr, "Too many arguments\n");
> +        return ERROR_FAIL;
> +    }
> +
>      overlay_ops = argv[1];
>      overlay_config_file = argv[2];
>  
> +    if (!strcmp(argv[argc - 1], "-e"))
> +        auto_mode = false;
> +
> +    if (argc == 4 || !auto_mode) {
> +        domain_id = find_domain(argv[argc-1]);
> +        domain_mapping = true;
> +    }
> +
> +    if (argc == 5 || !auto_mode) {
> +        domain_id = find_domain(argv[argc-2]);
> +        domain_mapping = true;
> +    }

Sorry, I can't review that changes, this needs a change in the help
message of dt-overlay, and something in the man page. (and that argument
parsing looks convoluted).

> +
> +    /* User didn't prove any overlay operation. */

I guess you meant "provide" instead of prove. But the comment can be
discarded, it doesn't explain anything useful that the error message
doesn't already explain.

> +    if (overlay_ops == NULL) {
> +        fprintf(stderr, "No overlay operation mode provided\n");
> +        return ERROR_FAIL;

That should be EXIT_FAILURE instead. (and I realise that the function
already return ERROR_FAIL by mistake in several places. (ERROR_FAIL is a
libxl error return value of -3, which isn't really a good exit value for
CLI programmes.))

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed May 01 15:09:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 15:09:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715339.1116889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2BaM-00078U-LH; Wed, 01 May 2024 15:09:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715339.1116889; Wed, 01 May 2024 15:09:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2BaM-00078N-GY; Wed, 01 May 2024 15:09:30 +0000
Received: by outflank-mailman (input) for mailman id 715339;
 Wed, 01 May 2024 15:09:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RAI4=ME=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1s2BaL-00078E-Nj
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 15:09:29 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd97ccc1-07cc-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 17:09:27 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-418e4cd2196so53410085e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 08:09:27 -0700 (PDT)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 b8-20020adfe308000000b003436a3cae6dsm34601838wrj.98.2024.05.01.08.09.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 08:09:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd97ccc1-07cc-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714576167; x=1715180967; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=PLBsUNaDoVONJfDnLGkcsX7757Jxo3cdl4Cr/gz+qUY=;
        b=JJPf82vZgEiWdokwEjP9DE8Bcm9zWZsJr+RCo3X9AxigLuhCUbOcQ6tg8O4SSaNbBi
         ROpWE8K9BWBPxCkgVbeYNjCEtOANu/jW1TQy+8cU4sR0H/IOEYdyjl8jB1jHwn3XUJiU
         AWiDXK3DMfh+8PWRk4xLsehDC2PMrB8VQrmW8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714576167; x=1715180967;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PLBsUNaDoVONJfDnLGkcsX7757Jxo3cdl4Cr/gz+qUY=;
        b=jV0RTNBWNtZfXMTtHbwUAcEqb0Sl0EdrOvAH5QXBrgzVAVuXjqYj30RARFdc/ej//E
         GbVNdKygjc9JY2EsaehRfE4FwRFcN0iW1YyeqsrUuGI0kDiUA6NF3vzG3y9SrrSN+dp/
         UAS8Dh/h3oENUEwNEdgBO6lYorAPB11Qp3SEcDOe0bpSLPxUzTRxRrLeg4xt4oCfNJZg
         dFtXjFDAzTSZokAkj7RmqLSR8pM5xcIz2Sf6myad9qaznmZ1E+iAeBu5WwgdtnhY4nRG
         wPO0CsOTb2WyzrOpq7xQY9KMV63+NV7ayfd8Ghr3GFy/jm5dwe32Uhyw+aTOA2nxpYQS
         AcZw==
X-Gm-Message-State: AOJu0YyZ/N/QmbupYd65MUaT/mrepKS2Mpd105oTj2vmB1T3G6X2VF1X
	Cz3C8uuDf9LsIZlLhed0FJqhVZEbixeTKVzYmaSnIg4uMabVAlhnl0Egm8h8iFU=
X-Google-Smtp-Source: AGHT+IETR9o/DrjmGB19ffxt4f98CuGA2/SSsethogQR9ejLqKKSMQYeo8nWqUGTAigPvDHcBhsQIQ==
X-Received: by 2002:a05:6000:e06:b0:34d:8190:72f4 with SMTP id dx6-20020a0560000e0600b0034d819072f4mr2067461wrb.1.1714576167105;
        Wed, 01 May 2024 08:09:27 -0700 (PDT)
Date: Wed, 1 May 2024 16:09:25 +0100
From: Anthony PERARD <anthony.perard@cloud.com>
To: Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Re: [PATCH 09/15] tools/libs/light: Modify dtbo to domU linux dtbo
 format
Message-ID: <2c5baf13-7511-4035-91da-02754a2f28c0@perard>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-10-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240424033449.168398-10-xin.wang2@amd.com>

On Wed, Apr 24, 2024 at 11:34:43AM +0800, Henry Wang wrote:
> diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
> index cdb62b28cf..eaf11a0f9c 100644
> --- a/tools/libs/light/libxl_dt_overlay.c
> +++ b/tools/libs/light/libxl_dt_overlay.c
> @@ -41,6 +42,69 @@ static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size)
>      return 0;
>  }
>  
> +static int modify_overlay_for_domU(libxl__gc *gc, void *overlay_dt_domU,
> +                                   size_t size)
> +{
> +    int rc = 0;
> +    int virtual_interrupt_parent = GUEST_PHANDLE_GIC;
> +    const struct fdt_property *fdt_prop_node = NULL;
> +    int overlay;
> +    int prop_len = 0;
> +    int subnode = 0;
> +    int fragment;
> +    const char *prop_name;
> +    const char *target_path = "/";
> +
> +    fdt_for_each_subnode(fragment, overlay_dt_domU, 0) {
> +        prop_name = fdt_getprop(overlay_dt_domU, fragment, "target-path",
> +                                &prop_len);
> +        if (prop_name == NULL) {
> +            LOG(ERROR, "target-path property not found\n");

LOG* macros already takes care of adding \n, no need to add an extra
one.

> +            rc = ERROR_FAIL;
> +            goto err;
> +        }
> +
> +        /* Change target path for domU dtb. */
> +        rc = fdt_setprop_string(overlay_dt_domU, fragment, "target-path",

fdt_setprop_string() isn't a libxl function, store the return value in a
variable named `r` instead.

> +                                target_path);
> +        if (rc) {
> +            LOG(ERROR, "Setting interrupt parent property failed for %s\n",
> +                prop_name);
> +            goto err;
> +        }
> +
> +        overlay = fdt_subnode_offset(overlay_dt_domU, fragment, "__overlay__");
> +
> +        fdt_for_each_subnode(subnode, overlay_dt_domU, overlay)
> +        {
> +            const char *node_name = fdt_get_name(overlay_dt_domU, subnode,
> +                                                 NULL);
> +
> +            fdt_prop_node = fdt_getprop(overlay_dt_domU, subnode,
> +                                        "interrupt-parent", &prop_len);
> +            if (fdt_prop_node == NULL) {
> +                LOG(DETAIL, "%s property not found for %s. Skip to next node\n",
> +                    "interrupt-parent", node_name);

Why do you have "interrupt-parent" in a separate argument? Do you meant
to do something like
    const char *some_name = "interrupt-parent";
and use that in the 4 different places that this string is used? (Using
a variable mean that we (or the compiler) can make sure that they are
all spelled correctly.

> +                continue;
> +            }
> +
> +            rc = fdt_setprop_inplace_u32(overlay_dt_domU, subnode,
> +                                         "interrupt-parent",
> +                                         virtual_interrupt_parent);
> +            if (rc) {
> +                LOG(ERROR, "Setting interrupt parent property failed for %s\n",
> +                    "interrupt-parent");
> +                goto err;
> +            }
> +        }
> +    }
> +
> +return 0;

Missed indentation.

> +
> +err:
> +    return rc;

A few things, looks like `rc` is always going to be ERROR_FAIL here,
unless you find an libxl_error code that better describe the error, so
you could forgo the `rc` variable.

Also, if you don't need to clean up anything in the function or have a
generic error message, you could simply "return " instead of using the
"goto" style.

> +}
> +
>  int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domid, void *overlay_dt,
>                       uint32_t overlay_dt_size, uint8_t overlay_op,
>                       bool auto_mode, bool domain_mapping)
> @@ -73,6 +137,15 @@ int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domid, void *overlay_dt,
>          rc = ERROR_FAIL;
>      }
>  
> +    /*
> +     * auto_mode doesn't apply to dom0 as dom0 can get the physical
> +     * description of the hardware.
> +     */
> +    if (domid && auto_mode) {
> +        if (overlay_op == LIBXL_DT_OVERLAY_ADD)

Shouldn't libxl complain if the operation is different?

> +            rc = modify_overlay_for_domU(gc, overlay_dt, overlay_dt_size);
> +    }
> +
>  out:
>      GC_FREE;
>      return rc;

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed May 01 16:06:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 16:06:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715346.1116899 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2CSp-0007Rb-Lt; Wed, 01 May 2024 16:05:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715346.1116899; Wed, 01 May 2024 16:05:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2CSp-0007RU-Ih; Wed, 01 May 2024 16:05:47 +0000
Received: by outflank-mailman (input) for mailman id 715346;
 Wed, 01 May 2024 16:05:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PNhw=ME=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s2CSo-0007RN-KQ
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 16:05:46 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aa52170b-07d4-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 18:05:44 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a58e787130fso158882366b.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 09:05:44 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r20-20020a170906705400b00a555ef55ab5sm16453997ejj.218.2024.05.01.09.05.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 09:05:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa52170b-07d4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714579544; x=1715184344; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1z4icKP8G1xe0BAbmiBPb6FvSILe576zu9Ds2KIzx08=;
        b=D8mHWSsJNKSCWxisHYsHOGcfJxv2reDz67HbyTSHM1RqHijYuesUGNKoP2SF5CyYtR
         cvIsaBBHocE6YYm8BKI3iKrnlebxY8Zov7cKIxkO9uN45YSQnHltFtpQm0f6Hvw9WDkq
         XNGTDl7evXZxv1ezwN8F6tlMUl8TsIxhxhFng=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714579544; x=1715184344;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1z4icKP8G1xe0BAbmiBPb6FvSILe576zu9Ds2KIzx08=;
        b=myuxYV9t/DKs/IcAFNSZ65gn0eWKN4uetWuopnVCAY621A2mgzQA0oVM+OU630iYYG
         VMgzyRrvJm28I9dcBkspgAdbLknNLPCuFiiG42O5Ca2EAwdun8+Q2l+2asuZuB0rnbkf
         mc5SfX1N6iLH+o2Tmt4jqfpa9TezmIfB3w+G+Y5J6cPxEXFWeSt3Aq122PNQ1/Ivq/NO
         WKZUHM7crGtZ7g7yHk1rpTJY+l3PHiaHi7Yy7MRs1vkfGxB/jPnBaixvVMrZzQW/DuEp
         iago4PbzlBI8QWbM/7CmnSYX1J6L6R0E6hfW7CWqfW54J+mbT6TIIyd8q9g7Clts0Dth
         Gz1A==
X-Gm-Message-State: AOJu0YwTClb8zTGyWeBztsIG92Uh6wPj8FCCCJfDaBk5+RlejObv5RrE
	hL6WvZznA94Be7ZV8/6RLwjKHJ6JEWzu2VZ7zG46LMsUErkmo84Yba8iCrrXrNtQHaS190W2oHs
	I
X-Google-Smtp-Source: AGHT+IFWnklqpO3fZj1EcG0HvQHmDJCOgJsCjZKhqvzw/p0NzoBxCpsXVIjtEMNJbAPVyQBWUKKtFw==
X-Received: by 2002:a17:907:7788:b0:a59:2141:5b0d with SMTP id ky8-20020a170907778800b00a5921415b0dmr45596ejc.35.1714579543982;
        Wed, 01 May 2024 09:05:43 -0700 (PDT)
Message-ID: <4171847c-6475-48e8-9de2-9c21c8af45b9@cloud.com>
Date: Wed, 1 May 2024 17:05:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] xen/x86: Derive topologically correct x2APIC IDs from
 the policy
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-6-alejandro.vallejo@cloud.com>
 <Zfq3VVbEA4ljxllI@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zfq3VVbEA4ljxllI@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi,

Ack and sure to everything on types, constness and variable names.

On 20/03/2024 10:15, Roger Pau Monné wrote:
>> +        const char *name;
>> +        uint32_t vcpu_id;
>> +        uint32_t x2apic_id;
>> +        struct cpu_policy policy;
>> +    } tests[] = {
>> +        {
>> +            .name = "3v: 3 t/c, 8 c/s", .vcpu_id = 3, .x2apic_id = 1 << 2,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_AMD,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
>> +                    [1] = { .nr_logical = 8, .level = 1, .type = 2, .id_shift = 5, },
>> +                },
> 
> Don't we need a helper that gets passed the number of cores per
> socket and threads per core and fills this up?
> 
> I would introduce this first, add a test for it, and then do this
> testing using the helper.

Funnily enough that's how I tested it initially. Alas, it felt silly to
put it where it will be linked against the hypervisor. I'm happy to put
it back there.

>> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
>> index a3b24e6879..2a50bc076a 100644
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -2,15 +2,78 @@
>>  
>>  #include <xen/lib/x86/cpu-policy.h>
>>  
>> -uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t vcpu_id)
>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>>  {
>>      /*
>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>> -     *       until all infra is in place to retrieve or derive the initial
>> -     *       x2APIC ID from migrated domains.
>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>> +     * the next topological scope. For example, assuming a system with 2
>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>> +     * the number of threads in a module.
>> +     *
>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>> +     * level (cores/complex, etc) so we can return it as-is.
>>       */
>> -    return vcpu_id * 2;
>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>> +        return p->topo.subleaf[lvl].nr_logical;
>> +
>> +    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;
> 
> I think this is an optimization because we only have 2 levels,
> otherwise you would need a loop like:
> 
> unsigned int nr = p->topo.subleaf[lvl].nr_logical
> for ( unsigned int i; i < lvl; i++ )
>     nr /= p->topo.subleaf[i].nr_logical;
> 
> If that's the case I think we need a
> BUILD_BUG_ON(ARRAY_SIZE(p->topo.subleaf) > 1);

It's not meant to be. That division should still hold for leaves 0x1f
and e26 where the level count typically exceeds 2. From the SDM.

```
Bits 15-00: The number of logical processors across all instances of
this domain within the next higherscoped domain. (For example, in a
processor socket/package comprising “M” dies of “N” cores each, where
each core has “L” logical processors, the “die” domain sub-leaf value of
this field would be M*N*L.) This number reflects configuration as
shipped by Intel. Note, software must not use this field to enumerate
processor topology*.
```

So. If level1 has nr_logical=X*Y and level2 has nr_logical=X*Y*Z then
dividing the latter by the former gives you Z, which is the number of
lvl1-parts for a single instance of lvl2 (i.e: cores/package, or whatever).

Unless I'm missing something?

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 01 16:36:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 16:36:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715353.1116909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Cw4-0003LG-3Q; Wed, 01 May 2024 16:36:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715353.1116909; Wed, 01 May 2024 16:36:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Cw3-0003L9-W2; Wed, 01 May 2024 16:35:59 +0000
Received: by outflank-mailman (input) for mailman id 715353;
 Wed, 01 May 2024 16:35:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PNhw=ME=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s2Cw2-0003L0-Fy
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 16:35:58 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e1f1e913-07d8-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 18:35:56 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a58fc650f8fso424601166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 09:35:56 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r20-20020a170906705400b00a555ef55ab5sm16480723ejj.218.2024.05.01.09.35.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 09:35:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1f1e913-07d8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714581355; x=1715186155; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j0cOQGeawFsj/qDO2oxPiX5pTKzMxIMJ9PgXl4/JbDc=;
        b=cUNgFlzTYFkxNg71AgFEGuylJWbjg5fuwt9Gg0DJmk38zH0JbTXF8ENuqG4RbJl+sh
         5vYgt6GzuaHbMDKyxzFMhjkXDM6r6juAf9Twh6aZgxty6nESbjy2XWb96zL8ZyCfXVuQ
         RoJVEZSuLtI4s5dC06/SyGKG5ei2XUHIwBQWY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714581355; x=1715186155;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=j0cOQGeawFsj/qDO2oxPiX5pTKzMxIMJ9PgXl4/JbDc=;
        b=iDyBnbfiajO/koB55C3uGj5lrWVMaOA6RADsRqgAOgm64NnN9w7W0OQnqy6DPyicDf
         92nsXdWSYeSS8McUncSrXNpn2Q5EC6ZfeLRbtPhNGAu01zzPLvRVBjKDD/IZeylY0P+T
         JSie/oVsULZRcCOuiRFzH1oQaQVx2YC4KNCDUlTConyJpvKO67mmEr9HAI5+FWbheEaT
         gXWRfTH7ololQycoN7NOA8Ip+pGXxwrJzRYwU+B+RnRP3TrNj1rwYsOCBhlZUBFGR108
         kls3ovaiVzM1FYO1PHNvS2V5fsU2MI6n+pS3Qt9PlsDSUwZXIbhLNRWIY3cgIWBaQ3lE
         Rbmg==
X-Forwarded-Encrypted: i=1; AJvYcCUaOlrYqC7zuRUz4fb7nxP8c6v65cnNCHIvB2wSN1kQE6uKaq/H6iv/qNhgwJILe0ZUo+CHwgXEE+f1c+QERQZT5cHGHkavMNhWDqs9UGw=
X-Gm-Message-State: AOJu0YyRDe+iG0LiQvDo3adHfQQmQJ4JdSI7dJ5G3gmpSn5QJ6PLuICi
	jDq56aAJiIJsZRqN5zUiYMS1mfmmVsQWialLD3CxA60krnERmGT2OV1O46DJDUo=
X-Google-Smtp-Source: AGHT+IFRqeHjZmL01dM2T7B8D53tVViIXN/SqzUr/06Z8uK0aGxJky7964qbn84g8OyJb4/K4KHuMg==
X-Received: by 2002:a17:906:da04:b0:a52:62a2:d727 with SMTP id fi4-20020a170906da0400b00a5262a2d727mr2345857ejb.52.1714581355277;
        Wed, 01 May 2024 09:35:55 -0700 (PDT)
Message-ID: <e0963ddd-c3b7-44dd-a3b7-67811f3e9d7a@cloud.com>
Date: Wed, 1 May 2024 17:35:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] xen/x86: Derive topologically correct x2APIC IDs from
 the policy
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-6-alejandro.vallejo@cloud.com>
 <14891d02-e9e9-4650-b572-e6f071f1506c@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <14891d02-e9e9-4650-b572-e6f071f1506c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi,

On 26/03/2024 16:41, Jan Beulich wrote:
> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -2,15 +2,78 @@
>>  
>>  #include <xen/lib/x86/cpu-policy.h>
>>  
>> -uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t vcpu_id)
>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>>  {
>>      /*
>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>> -     *       until all infra is in place to retrieve or derive the initial
>> -     *       x2APIC ID from migrated domains.
>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>> +     * the next topological scope. For example, assuming a system with 2
>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>> +     * the number of threads in a module.
>> +     *
>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>> +     * level (cores/complex, etc) so we can return it as-is.
>>       */
>> -    return vcpu_id * 2;
>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>> +        return p->topo.subleaf[lvl].nr_logical;
> 
> Is "!= Intel" really appropriate here? I'd rather see this being "AMD || Hygon".

Sure, I don't particularly mind, but why? As far as we know only Intel
has this interpretation for the part counts. I definitely haven't seen
any non-Intel CPUID dump in which the part count is the total number of
threads (Centaur/Zhaoxin are not multithreaded, and don't expose leaves
1f or e26, as far as I could see).

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 01 16:48:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 16:48:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715358.1116918 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2D8O-00058g-5G; Wed, 01 May 2024 16:48:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715358.1116918; Wed, 01 May 2024 16:48:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2D8O-00058Z-2b; Wed, 01 May 2024 16:48:44 +0000
Received: by outflank-mailman (input) for mailman id 715358;
 Wed, 01 May 2024 16:48:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=93GK=ME=redhat.com=peterx@srs-se1.protection.inumbo.net>)
 id 1s2D8N-00058D-P9
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 16:48:43 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.133.124])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9d02b9d-07da-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 18:48:41 +0200 (CEST)
Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com
 [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-310-mmJZd0p4NIaR1-oHJiddWQ-1; Wed, 01 May 2024 12:48:38 -0400
Received: by mail-qt1-f197.google.com with SMTP id
 d75a77b69052e-43a3632d56aso28221481cf.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 09:48:38 -0700 (PDT)
Received: from x1n (pool-99-254-121-117.cpe.net.cable.rogers.com.
 [99.254.121.117]) by smtp.gmail.com with ESMTPSA id
 h2-20020ac85142000000b00439a1bdb6e5sm9468549qtn.17.2024.05.01.09.48.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 May 2024 09:48:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9d02b9d-07da-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1714582120;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=W0rqb1II+fEQyts8xj8PiGw705f1XoyimiMNElU/2to=;
	b=HpvsD0Sn4IBb18Chv6mlys3QLv3az7QhlQg6grvlcG/hXKt9eSHT2ZR+LPtBnzhq+KxHsW
	+PcnlVuj6Z24N4APt1vmkUMksEUBY04jrA1VVLMY+zIjcqYNqbhtnaSUTckyNZqBrgMsZi
	GYBmC6THB3YbuATJOlJ6bVXR6+9yaa8=
X-MC-Unique: mmJZd0p4NIaR1-oHJiddWQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714582118; x=1715186918;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=W0rqb1II+fEQyts8xj8PiGw705f1XoyimiMNElU/2to=;
        b=ARKZ815GC3OvDFfTgCV0sbn3PX2bTvYh3OO1weIgdyRb92hTA9ZfnQD7ObWgST5mz5
         NNyw/nBZV8RjJ3YC5UKRghxD7TBHIhnUQJip3iTqb9zGRF3AYrDeDTLz0jtJ8KwTsdOr
         mEcv0Uyd+EfG2dQt2Kxi49YAk6cI3grzhJy5QLx6QeMgdt9WGkPrj59dYNwAfdnn+OpQ
         xI8bs4i7LZ1hJUqSPuOotBlPUXcYxGK4OhLj+/LZ53XZLbN2wL5hWUH8kL7X3zfHGmy8
         X0SUpnso9iONH2sa6IOGTo4GP9IZK/H3ydjdjlTAQ+YLD6JdTklHQnC2+buQZbGESISM
         Ztyg==
X-Forwarded-Encrypted: i=1; AJvYcCUtSR4I3yZvbNIk3T6K1bruIUFhVw96QwhMAI+7nsOSe7UnCCSeh3LMJ7O6lPyuUJhH7bclsJfOUUYAK9ZtAg93mCFsvoDEZkDHPdCgnaY=
X-Gm-Message-State: AOJu0YwqN18675Lym3XFR80Mm8Viuf+t3dnc2SaypMJQFo3L6C2jm1eC
	vJhjb0pJV8r3RL7V7LGLONPNSZSTOzDRBNo3FKsu0fQGg967W4kOpDe7z2LP0DKrch0/RdBpWvf
	K667Rj+Stw9kG1tvX+pchOMal0kql1I59tW4u+D5BtZAOxuhpPknr1jQd/Iqcy8Mr
X-Received: by 2002:a05:622a:454:b0:43b:e8:3858 with SMTP id o20-20020a05622a045400b0043b00e83858mr3335557qtx.5.1714582118200;
        Wed, 01 May 2024 09:48:38 -0700 (PDT)
X-Google-Smtp-Source: AGHT+IGlczwLQ1ydCbZMNGW1G2sQPdaOKLA7HRYhsgku2nl0OuzRoK0kJ+YcZfu1hxBiGpLYl+M3KQ==
X-Received: by 2002:a05:622a:454:b0:43b:e8:3858 with SMTP id o20-20020a05622a045400b0043b00e83858mr3335525qtx.5.1714582117703;
        Wed, 01 May 2024 09:48:37 -0700 (PDT)
Date: Wed, 1 May 2024 12:48:35 -0400
From: Peter Xu <peterx@redhat.com>
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write
 xen_map_cache()
Message-ID: <ZjJyYzr-Qq9Y71BM@x1n>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-14-edgar.iglesias@gmail.com>
MIME-Version: 1.0
In-Reply-To: <20240430164939.925307-14-edgar.iglesias@gmail.com>
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline

On Tue, Apr 30, 2024 at 06:49:35PM +0200, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Propagate MR and is_write to xen_map_cache().
> This is in preparation for adding support for grant mappings.
> 
> No functional change.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Acked-by: Peter Xu <peterx@redhat.com>

-- 
Peter Xu



From xen-devel-bounces@lists.xenproject.org Wed May 01 17:07:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 17:07:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715371.1116929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2DQ4-0008BP-Kq; Wed, 01 May 2024 17:07:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715371.1116929; Wed, 01 May 2024 17:07:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2DQ4-0008BI-ID; Wed, 01 May 2024 17:07:00 +0000
Received: by outflank-mailman (input) for mailman id 715371;
 Wed, 01 May 2024 17:06:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PNhw=ME=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s2DQ3-0008BC-Oc
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 17:06:59 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3834c156-07dd-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 19:06:58 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-57232e47a81so1633829a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 10:06:58 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l12-20020a056402028c00b00572a0550acbsm1654878edv.55.2024.05.01.10.06.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 10:06:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3834c156-07dd-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714583218; x=1715188018; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nNqOMSclx5ECvP0hOOQ73Nh6pW/CCOjh7lvZ0NrytSg=;
        b=RZLgo+ww2cNOywVaiUqdWRch6+f5zCo/xc+3nTiPuz0EFFZkBpm7Q/zT/QUvLXFJdl
         5pInjIGAr+EG2J9u7L7d8XpZCcls+dTQExzVE3lKfjG+9X2Vt7NRSIysBc/D8zQuy0P+
         ufQL2kwmMSDC+h8vjQi8qaNkk1/07eQ8+Xuas=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714583218; x=1715188018;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nNqOMSclx5ECvP0hOOQ73Nh6pW/CCOjh7lvZ0NrytSg=;
        b=Ywirx4Zq1dyh2gtlDlbDw3g2SZRTkU8DwlZZtPu5mLqtNzvX2ZoTG7KYOT8N13wdK2
         ebndT8ijUVHJOx8UKErcW8CK5n1lsvUt0f2f6PCc+oFKSeQ+UHctUNx2EwTSF/iIwt6C
         d2YrzKIdjW8C7P4guNcfoieR2Jx+NzDMd9c77XZ+lKLnpTUGS9f1i16NYUOt5hKOwESt
         voIMBhILiniBsIluCFOtFdE1jNo/yWJVnx/3/uuLURAUYiZn5geIwEay1lu8r3MXEI82
         Oc2YxZfVSznJeyqT4QkYXtF29qWnr6dkuUM/OzX/9W6ZE54n9sxEKBAcpAWIZ5jxkVZY
         ERwQ==
X-Forwarded-Encrypted: i=1; AJvYcCVlvOCvBxnF+CquvwTJYxJ/rI8eleXS5M5clqAFwFkufMKZn0e7LbxbTLhdwzDcPNqVhKUIPNAJdqOuRLvpctumXXYpAsUOHh6vMXJfgKg=
X-Gm-Message-State: AOJu0Ywi9T6Vm9Elc9kjBMc4z6zNgMxokC67W5Xiyg96q2jXL1mAs1o/
	MRcGM1jdlRys4QoWMvgqa6REkk1tB1FtNg4rBfbPFZFA9cmUhlu1rFs+FSIJdBDMMytCv0znRHd
	7
X-Google-Smtp-Source: AGHT+IGFAHbJU6ILWShuMQKUwi7FfYahY3fNGQqnQWuCq6v8bmX5Vme45tP1I1BMn7xqiBC1T8pe+g==
X-Received: by 2002:a50:8ad6:0:b0:571:da40:22d9 with SMTP id k22-20020a508ad6000000b00571da4022d9mr200219edk.10.1714583217825;
        Wed, 01 May 2024 10:06:57 -0700 (PDT)
Message-ID: <e17b2cb1-1482-46e1-844f-22be23a7190a@cloud.com>
Date: Wed, 1 May 2024 18:06:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/6] xen/x86: Add topology generator
To: Jan Beulich <jbeulich@suse.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-7-alejandro.vallejo@cloud.com>
 <c3b4bacd-47c8-403c-ab5c-7a3adf39eeeb@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <c3b4bacd-47c8-403c-ab5c-7a3adf39eeeb@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26/03/2024 17:02, Jan Beulich wrote:
> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>> --- a/tools/include/xenguest.h
>> +++ b/tools/include/xenguest.h
>> @@ -843,5 +843,20 @@ enum xc_static_cpu_featuremask {
>>      XC_FEATUREMASK_HVM_HAP_DEF,
>>  };
>>  const uint32_t *xc_get_static_cpu_featuremask(enum xc_static_cpu_featuremask);
>> +
>> +/**
>> + * Synthesise topology information in `p` given high-level constraints
>> + *
>> + * Topology is given in various fields accross several leaves, some of
>> + * which are vendor-specific. This function uses the policy itself to
>> + * derive such leaves from threads/core and cores/package.
>> + *
>> + * @param p                   CPU policy of the domain.
>> + * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
>> + * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
>> + */
>> +void xc_topo_from_parts(struct cpu_policy *p,
>> +                        uint32_t threads_per_core, uint32_t cores_per_pkg);
> 
> Do we really want to constrain things to just two (or in fact any fixed number
> of) levels? Iirc on AMD there already can be up to 4.

For the time being, I think we should keep it simple(ish).

Leaf 0xb is always 2 levels, and it implicitly defines the offset into
the x2APIC ID for the 3rd level (the package, or the die). This approach
can be used for a long time before we need to expose anything else. It
can already be used for exposing multi-socket configurations, but it's
not properly wired to xl.

I suspect we won't have a need to expose more complicated topologies
until hetero systems are more common, and by that time the generator
will need tweaking anyway.

> 
>> @@ -1028,3 +976,89 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
>>  
>>      return false;
>>  }
>> +
>> +static uint32_t order(uint32_t n)
>> +{
>> +    return 32 - __builtin_clz(n);
>> +}
> 
> This isn't really portable. __builtin_clz() takes an unsigned int, which may
> in principle be wider than 32 bits. I also can't see a reason for the
> function to have a fixed-width return type. Perhaps

Sure to unsigned int. I'll s/CHAR_BIT/8/ as that's mandated by POSIX
already.

> 
> static unsigned int order(unsigned int n)
> {
>     return sizeof(n) * CHAR_BIT - __builtin_clz(n);
> }
> 
> ?
> 
>> +void xc_topo_from_parts(struct cpu_policy *p,
>> +                        uint32_t threads_per_core, uint32_t cores_per_pkg)
>> +{
>> +    uint32_t threads_per_pkg = threads_per_core * cores_per_pkg;
>> +    uint32_t apic_id_size;
>> +
>> +    if ( p->basic.max_leaf < 0xb )
>> +        p->basic.max_leaf = 0xb;
>> +
>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>> +
>> +    /* thread level */
>> +    p->topo.subleaf[0].nr_logical = threads_per_core;
>> +    p->topo.subleaf[0].id_shift = 0;
>> +    p->topo.subleaf[0].level = 0;
>> +    p->topo.subleaf[0].type = 1;
>> +    if ( threads_per_core > 1 )
>> +        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
>> +
>> +    /* core level */
>> +    p->topo.subleaf[1].nr_logical = cores_per_pkg;
>> +    if ( p->x86_vendor == X86_VENDOR_INTEL )
>> +        p->topo.subleaf[1].nr_logical = threads_per_pkg;
> 
> Same concern as in the other patch regarding "== Intel".
> 

Can you please articulate the concern?

>> +    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
>> +    p->topo.subleaf[1].level = 1;
>> +    p->topo.subleaf[1].type = 2;
>> +    if ( cores_per_pkg > 1 )
>> +        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
> 
> Don't you want to return an error when any of the X_per_Y values is 0?

I'd rather not.

The checks on input parameters should be done wherever the inputs are
taken from. Currently the call site passes threads_per_core=1 and
cores_per_pkg=1+max_vcpus, so it's all guaranteed to work out.

Once it comes from xl, libxl should be in charge of the validations.
Furthermore there's validations the function simply cannot do (nor
should it) in its current form, like checking that...

    max_vcpus == threads_per_core * cores_per_pkg * n_pkgs.

> 
>> +    apic_id_size = p->topo.subleaf[1].id_shift;
>> +
>> +    /*
>> +     * Contrary to what the name might seem to imply. HTT is an enabler for
>> +     * SMP and there's no harm in setting it even with a single vCPU.
>> +     */
>> +    p->basic.htt = true;
>> +
>> +    p->basic.lppp = 0xff;
>> +    if ( threads_per_pkg < 0xff )
>> +        p->basic.lppp = threads_per_pkg;
>> +
>> +    switch ( p->x86_vendor )
>> +    {
>> +        case X86_VENDOR_INTEL:
>> +            struct cpuid_cache_leaf *sl = p->cache.subleaf;
>> +            for ( size_t i = 0; sl->type &&
>> +                                i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
>> +            {
>> +                sl->cores_per_package = cores_per_pkg - 1;
>> +                sl->threads_per_cache = threads_per_core - 1;
> 
> IOW the names in struct cpuid_cache_leaf aren't quite correct.

Because of the - 1, you mean? If anything our name is marginally clearer
than the SDM description. It goes on to say "Add one to the return value
to get the result" in a [**] note, so it's not something we made up.

Xen: threads_per_cache => SDM: Maximum number of addressable IDs for
logical processors sharing this cache

Xen: cores_per_package => SDM: Maximum number of addressable IDs for
processor cores in the physical package

> 
>> +                if ( sl->type == 3 /* unified cache */ )
>> +                    sl->threads_per_cache = threads_per_pkg - 1;
>> +            }
>> +            break;
>> +
>> +        case X86_VENDOR_AMD:
>> +        case X86_VENDOR_HYGON:
>> +            /* Expose p->basic.lppp */
>> +            p->extd.cmp_legacy = true;
>> +
>> +            /* Clip NC to the maximum value it can hold */
>> +            p->extd.nc = 0xff;
>> +            if ( threads_per_pkg <= 0xff )
>> +                p->extd.nc = threads_per_pkg - 1;
>> +
>> +            /* TODO: Expose leaf e1E */
>> +            p->extd.topoext = false;
>> +
>> +            /*
>> +             * Clip APIC ID to 8, as that's what high core-count machines do
> 
> Nit: "... to 8 bits, ..."

ack

> 
>> +             *
>> +             * That what AMD EPYC 9654 does with >256 CPUs
>> +             */
>> +            p->extd.apic_id_size = 8;
>> +            if ( apic_id_size < 8 )
>> +                p->extd.apic_id_size = apic_id_size;
> 
> Use min(), with apic_id_size's type suitably adjusted?

ack

> 
>> --- a/xen/arch/x86/cpu-policy.c
>> +++ b/xen/arch/x86/cpu-policy.c
>> @@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
>>  
>>      p->basic.raw[0x8] = EMPTY_LEAF;
>>  
>> -    /* TODO: Rework topology logic. */
>> -    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>> -
>>      p->basic.raw[0xc] = EMPTY_LEAF;
>>  
>>      p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
>> @@ -387,6 +384,9 @@ static void __init calculate_host_policy(void)
>>      recalculate_xstate(p);
>>      recalculate_misc(p);
>>  
>> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
> 
> I don't think this should be zapped from the host policy. It wants zapping
> from the guest ones instead, imo. The host policy may (will) want using in
> Xen itself, and hence it should reflect reality.
> 
> Jan

Shouldn't Xen be checking its own cached state from the raw policy
instead? My understanding was that to a first approximation the host
policy is a template for guest creation. It already has a bunch of
overrides that don't match the real hardware configuration.

Cheers,
Alejandro



From xen-devel-bounces@lists.xenproject.org Wed May 01 18:20:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 18:20:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715379.1116938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2EYz-0001h3-Qy; Wed, 01 May 2024 18:20:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715379.1116938; Wed, 01 May 2024 18:20:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2EYz-0001gw-OO; Wed, 01 May 2024 18:20:17 +0000
Received: by outflank-mailman (input) for mailman id 715379;
 Wed, 01 May 2024 18:20:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2EYy-0001gm-MA; Wed, 01 May 2024 18:20:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2EYy-0004wL-Jj; Wed, 01 May 2024 18:20:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2EYy-0004Xs-3Y; Wed, 01 May 2024 18:20:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2EYy-0008Bu-35; Wed, 01 May 2024 18:20:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=apfBilwRBGSHLj9blxdrjIW9pLP/wgw+8O4SVYWlWKQ=; b=nFv6rRZGcLcpEhAGoQJp5xQs8u
	ew6FGi/7Vf+jdcuTY2yrmfeWsoWZSB9IaEw8PKJk9SceMUCFy5GqlqWwKHroc8khg7cX5p1YLJFb3
	IgcAgpwWr3OvEOJOnKhv48lgSHAlew5TeQGtxb472tc+WuA8GUTlbWJcJfrmrDRnTx28=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185890-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185890: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit2:host-ping-check-xen:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9e30bd8f4a8cee83368276c5149c7e2304efe5a2
X-Osstest-Versions-That:
    xen=be5b08dd6ea6ef0f01caf537bdae125fa66a2230
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 18:20:16 +0000

flight 185890 xen-unstable real [real]
flight 185894 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185890/
http://logs.test-lab.xenproject.org/osstest/logs/185894/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2  10 host-ping-check-xen fail pass in 185894-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185894 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185894 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185860
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185872
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185872
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185872
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185872
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185872
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9e30bd8f4a8cee83368276c5149c7e2304efe5a2
baseline version:
 xen                  be5b08dd6ea6ef0f01caf537bdae125fa66a2230

Last test of basis   185872  2024-04-30 01:55:16 Z    1 days
Testing same since   185890  2024-05-01 02:54:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Sergiy Kibrik <Sergiy_Kibrik@epam.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   be5b08dd6e..9e30bd8f4a  9e30bd8f4a8cee83368276c5149c7e2304efe5a2 -> master


From xen-devel-bounces@lists.xenproject.org Wed May 01 18:36:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 18:36:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715394.1116977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Eoj-00041n-N0; Wed, 01 May 2024 18:36:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715394.1116977; Wed, 01 May 2024 18:36:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Eoj-00041g-IX; Wed, 01 May 2024 18:36:33 +0000
Received: by outflank-mailman (input) for mailman id 715394;
 Wed, 01 May 2024 18:36:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Eoi-00041W-3K; Wed, 01 May 2024 18:36:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Eoi-0005FW-1P; Wed, 01 May 2024 18:36:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Eoh-0004vv-L0; Wed, 01 May 2024 18:36:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Eoh-0007kU-KU; Wed, 01 May 2024 18:36:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=WCsHr+5W1epU4D7KW0uMFH3GcZoW8bTXY6fJIPrJGJ4=; b=mFH4cm6hBLhH3dVfBXb5Ymlf4N
	BdPaqwBPbBVUtRA+abuANS7ZtCrPNnhzIhZcqaCebucz+efS9wvEibJzSeDx2Qj3I5pUIdXuqso5y
	201G/ZALFGdxKyWjArWAtZZb4yzu2Uf/WMc6olDNNJ3VTC1YX9/fH1DnqKFa4hCn+0ZI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185891-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185891: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=63f00d09e3bf7442cddf3657b51b0bc645f8c434
X-Osstest-Versions-That:
    libvirt=51589f154d9ca1ea76db42784bbfee9752c5a72b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 18:36:31 +0000

flight 185891 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185891/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185829
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              63f00d09e3bf7442cddf3657b51b0bc645f8c434
baseline version:
 libvirt              51589f154d9ca1ea76db42784bbfee9752c5a72b

Last test of basis   185829  2024-04-27 04:18:47 Z    4 days
Testing same since   185891  2024-05-01 04:18:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  Göran Uddeborg <goeran@uddeborg.se>
  Remus-Gabriel Chelu <remusgabriel.chelu@disroot.org>
  Weblate <noreply-mt-weblate@weblate.org>
  Weblate <noreply@weblate.org>
  Yuri Chornoivan <yurchor@ukr.net>
  김인수 <simmon@nplob.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   51589f154d..63f00d09e3  63f00d09e3bf7442cddf3657b51b0bc645f8c434 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 01 19:45:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 19:45:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715401.1116987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fte-00044p-HU; Wed, 01 May 2024 19:45:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715401.1116987; Wed, 01 May 2024 19:45:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fte-00044i-EZ; Wed, 01 May 2024 19:45:42 +0000
Received: by outflank-mailman (input) for mailman id 715401;
 Wed, 01 May 2024 19:45:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Ftc-00044c-PG
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 19:45:40 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5f29e1c7-07f3-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 21:45:36 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 4E6F8CE12A3;
 Wed,  1 May 2024 19:45:31 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8C44FC4AF19;
 Wed,  1 May 2024 19:45:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f29e1c7-07f3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714592730;
	bh=UyPQCRw95DefHzzFpKOcfCzyAdIzf1ArezoRPBVab1k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VdAB6I7vQFX+n7WO+nKbouHMCFKXWW7yZI9hM83f7wof3T6aO/jblW8lE5HjcShS2
	 EjcP1OJOavEDFNCTzcwU7xw1BOkVM6V6XmjJ5LRuorL01qd+pi6LXxZV447XeWHDdp
	 VS4xG24BStnyufB9SQxvMrRD/GvhoPTkfhxs4l/GcQf9pnWTM2+8q7mBgqJ2yiq5yY
	 An2BF8bqFLDO9e3eSSTPhbB4xtSQdQqsvq2YVcVwGUMx9sI3LvDCOLVxBzC0BeUu9Q
	 5KuAzIrrerH2HebI/JBFyFGJMFCBPzyg1/vm31f/Mcs/WtDuwndtFX9uOu9g7qVvrh
	 Apw5xrUKniPSQ==
Date: Wed, 1 May 2024 12:45:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 1/3] automation/eclair: tag Rule 7.2 as clean and
 temporarily remove Rules 1.1 and 8.2
In-Reply-To: <69c412a2322151971bf2b2705cabe4a292882028.1714401209.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011245170.497719@ubuntu-linux-20-04-desktop>
References: <cover.1714401209.git.federico.serafini@bugseng.com> <69c412a2322151971bf2b2705cabe4a292882028.1714401209.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 29 Apr 2024, Federico Serafini wrote:
> Update ECLAIR configuration to consider Rule 7.2 as clean.
> 
> Temporarily remove the clean tag from Rules 1.1 and 8.2:
> when violations of such rules will be addressed, the clean tag will be
> reintroduced.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
> index d609b470eb..bdf94ed996 100644
> --- a/automation/eclair_analysis/ECLAIR/tagging.ecl
> +++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
> @@ -19,7 +19,7 @@
>  
>  -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
>  
> --service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
> +-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
>  }
>  
>  -setq=target,getenv("XEN_TARGET_ARCH")
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 19:46:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 19:46:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715402.1116997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fu5-0004UC-OR; Wed, 01 May 2024 19:46:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715402.1116997; Wed, 01 May 2024 19:46:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fu5-0004U5-Lu; Wed, 01 May 2024 19:46:09 +0000
Received: by outflank-mailman (input) for mailman id 715402;
 Wed, 01 May 2024 19:46:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Fu4-0004Sa-6g
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 19:46:08 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ed7bea4-07f3-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 21:46:01 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 35DBBCE1360;
 Wed,  1 May 2024 19:45:58 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C44CC072AA;
 Wed,  1 May 2024 19:45:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ed7bea4-07f3-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714592757;
	bh=D1z2Oq6Nr7q2jVU8pRYIWTBkfTHNnLQ/TVYIxW/fVfQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=D4J/E70PLN+jRsL5Crj8FNBRTIf8KuCrrc7vQm9v0MnPE+L5rLMEglXgJ56VIvtRF
	 SuxdB7ZWt46jEQkeLJClGBvcsneeQzmNzbcje6GsLtBt7XDBvv7E8/Ak/XblieolVI
	 t8d1coLbCVmW4EvxvWN+4hLLnziP20wkmoad1mX7pRefjdercWPSCZprETyJfKmcGt
	 A+yOOLbmRMHqsqsUD4FsbTTtHngAfnQUONpgGAPh53IQf6UfjLlJ6Pg6ImOogFShc/
	 8vmF4qyCIj4q1sOoTT8hM76nHG89oKC7ULXD3wDAioefvSm1VW7quCk/hTZk7FUoR0
	 +oXDgTV15V6bw==
Date: Wed, 1 May 2024 12:45:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 2/3] automation: do not allow failure for triggered
 analyses
In-Reply-To: <34d8782629fc8cb1fe09060eb1aac2d6efab8096.1714401209.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011245490.497719@ubuntu-linux-20-04-desktop>
References: <cover.1714401209.git.federico.serafini@bugseng.com> <34d8782629fc8cb1fe09060eb1aac2d6efab8096.1714401209.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 29 Apr 2024, Federico Serafini wrote:
> Do not allow_failure for triggered analyses:
> introducing regressions of clean guidelines will cause a CI failure.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/gitlab-ci/analyze.yaml | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
> index 46c9d8e2e5..32bf570149 100644
> --- a/automation/gitlab-ci/analyze.yaml
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -26,7 +26,6 @@
>  
>  .eclair-analysis:triggered:
>    extends: .eclair-analysis
> -  allow_failure: true
>    rules:
>      - if: $CI_PIPELINE_SOURCE == "schedule"
>        when: never
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 19:46:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 19:46:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715407.1117007 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fuo-000531-20; Wed, 01 May 2024 19:46:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715407.1117007; Wed, 01 May 2024 19:46:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Fun-00052u-Uc; Wed, 01 May 2024 19:46:53 +0000
Received: by outflank-mailman (input) for mailman id 715407;
 Wed, 01 May 2024 19:46:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Fum-0004Sa-V5
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 19:46:52 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a7b060d-07f3-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 21:46:52 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 96534CE138E;
 Wed,  1 May 2024 19:46:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BB61C072AA;
 Wed,  1 May 2024 19:46:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a7b060d-07f3-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714592803;
	bh=/zXAlhKA7w+byChvy/eKON15X7SqbuaEFwSCyq7WYuA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TNrlckORgxWPdBFmxDP8bo25bwpqdY67vjtOPYWB/eWOZ19KVQsOLVCldvb/drfks
	 wCiDAwz8hx9fPFG0TZI36T4WK48WBk5XkrmHjn4iavBFAtJJjS1CHQyhfEh+j9J1Ak
	 fdlIWZoZWkHynpyKJ2XTejyXBu3JSYX5m/nYiHnh5Qrr+MPWeS5qfePeZkV1FeM4YQ
	 IZXJ+17EeSEE5oPqDuRlkeKlEQNaDOk84aSJfmPn1YUj/nSfRCczg4TlxiS8zF53uD
	 tipTIqSe25NfBzSfgpN0rG08w95CzbWfI0j8V0UmPA1JUUWjl2hdCxcg3jwQe8dbIM
	 4q4vUFfr9U3Ug==
Date: Wed, 1 May 2024 12:46:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 3/3] automation/eclair: make explicit there are no
 adopted files by default
In-Reply-To: <9d09f9787becdc356c076865e095d0ea97fbba62.1714401209.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011246300.497719@ubuntu-linux-20-04-desktop>
References: <cover.1714401209.git.federico.serafini@bugseng.com> <9d09f9787becdc356c076865e095d0ea97fbba62.1714401209.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 29 Apr 2024, Federico Serafini wrote:
> Update ECLAIR configuration to consider no adopted files by default.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/analysis.ecl | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index a604582da3..66ed7f952c 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -21,6 +21,10 @@ map_strings("scheduled-analysis",analysis_kind)
>  
>  -eval_file=toolchain.ecl
>  -eval_file=public_APIs.ecl
> +
> +-doc="Initially, there are no files tagged as adopted."
> +-file_tag+={adopted,"none()"}
> +
>  if(not(scheduled_analysis),
>      eval_file("adopted.ecl")
>  )
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 19:54:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 19:54:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715413.1117017 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2G2U-0006z1-QI; Wed, 01 May 2024 19:54:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715413.1117017; Wed, 01 May 2024 19:54:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2G2U-0006yu-Mj; Wed, 01 May 2024 19:54:50 +0000
Received: by outflank-mailman (input) for mailman id 715413;
 Wed, 01 May 2024 19:54:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2G2U-0006yo-1K
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 19:54:50 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a94f0c50-07f4-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 21:54:47 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DAD7A619F7;
 Wed,  1 May 2024 19:54:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C51C5C4AF14;
 Wed,  1 May 2024 19:54:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a94f0c50-07f4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714593285;
	bh=xoJXXr1iAPU4YJ7eR2Ibl3K9W6AbUYpI6MGa15AYcFE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bB3YWcpXFxRfZkRvYHVlL//VX89GzHNfQ9dqt8Dsoi3Fi5/r4IWbCXBW5bPRpBjAU
	 t/21AYc8KyyiXj4/yJevpKq4JmFSEVn/d4I/644GxV8a8vq6SzYDl8tGK2pMO9/z1y
	 fvT14Wm9ZphFdbskch0YFEKe5NlwCEfADDkpuvCQZq4P5T+DNhOc16iTyNyis2aOo+
	 DQOHuv06M1XtDcJMVDfjd/i3FHixFQRXAC1hylYWBUvpAZjy4xo6ZRPdMrEwzkETwB
	 OM0IGoGNHC+XAU8tzcAfEPPkeaLY05Ly3DC5sLapkVY1A0qtDqNnoy2j6fOuay3gqK
	 qTRvx4yTLqBFA==
Date: Wed, 1 May 2024 12:54:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, michal.orzel@amd.com, 
    xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, bertrand.marquis@arm.com, julien@xen.org, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: Re: [XEN PATCH 03/10] automation/eclair_analysis: deviate macro
 count_args_ for MISRA Rule 20.7
In-Reply-To: <5b675fcdf688723bc6d4ea260d10004e@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011254290.497719@ubuntu-linux-20-04-desktop>
References: <cover.1713885065.git.nicola.vetrini@bugseng.com> <7de407c218f0911e28b7c3f609a55636165166a8.1713885065.git.nicola.vetrini@bugseng.com> <alpine.DEB.2.22.394.2404241727520.3940@ubuntu-linux-20-04-desktop>
 <5b675fcdf688723bc6d4ea260d10004e@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 29 Apr 2024, Nicola Vetrini wrote:
> On 2024-04-25 02:28, Stefano Stabellini wrote:
> > On Tue, 23 Apr 2024, Nicola Vetrini wrote:
> > > The count_args_ macro violates Rule 20.7, but it can't be made
> > > compliant with Rule 20.7 without breaking its functionality. Since
> > > it's very unlikely for this macro to be misused, it is deviated.
> > 
> > That is OK but can't we use the SAF- framework to do it, given that it
> > is just one macro?
> > 
> > If not, this is also OK.
> > 
> > 
> 
> It would be more fragile, for no substantial gain

OK

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Wed May 01 19:58:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 19:58:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715419.1117027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2G5b-0007Z4-7h; Wed, 01 May 2024 19:58:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715419.1117027; Wed, 01 May 2024 19:58:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2G5b-0007Yx-3t; Wed, 01 May 2024 19:58:03 +0000
Received: by outflank-mailman (input) for mailman id 715419;
 Wed, 01 May 2024 19:58:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2G5a-0007WT-1m
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 19:58:02 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1adfaa9f-07f5-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 21:58:00 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 14305CE138C;
 Wed,  1 May 2024 19:57:52 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0343C072AA;
 Wed,  1 May 2024 19:57:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1adfaa9f-07f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714593471;
	bh=VZoNuTPfViOoSZdutjXK3PMT5252Fbw5CeEXYvzJA8k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kc4OLJ3VjP3j2Vjxxys81xPn9b2HfOxojZQk58ZvoYR6uPh4g1ccTRiuKlfC3c6JU
	 qG0j2GuYOPNX6admphH+vFQ+9x7gQGKbHJJFUBR3/UAVcW1fw7Zw7/HhzpQiAgURre
	 /QaSkV4ov++9+Mk9++RdNvLXcQ6xW3zYKtO1lFfKVq427f6jjK6LrNLPe/7wjtlCQk
	 9pzBQDhmHpczUloJxUEhVpxtH6L5+crDUqnIGPVwQaBb/iwSF6eJX9LtjYGHZ4Rxr8
	 eQEy8StACYDqqytrxkNp1FaxyuxilMJt9HiDIczBskxDK/KRsrMYvDz373jnXYcaJ9
	 yWulCRvt3yaug==
Date: Wed, 1 May 2024 12:57:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, bertrand.marquis@arm.com, julien@xen.org, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH v2 1/3] drivers: char: address violation of MISRA C
 Rule 20.7
In-Reply-To: <3ae8aa13a5562440cf60e4b30f713e5b2f73c577.1714487169.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011257290.497719@ubuntu-linux-20-04-desktop>
References: <cover.1714487169.git.nicola.vetrini@bugseng.com> <3ae8aa13a5562440cf60e4b30f713e5b2f73c577.1714487169.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional chage.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v2:
> - drop excess parentheses from val parameter.
> ---
>  xen/drivers/char/omap-uart.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/drivers/char/omap-uart.c b/xen/drivers/char/omap-uart.c
> index 03b5b66e7acb..e0128225f927 100644
> --- a/xen/drivers/char/omap-uart.c
> +++ b/xen/drivers/char/omap-uart.c
> @@ -48,8 +48,9 @@
>  /* System configuration register */
>  #define UART_OMAP_SYSC_DEF_CONF   0x0d   /* autoidle mode, wakeup is enabled */
>  
> -#define omap_read(uart, off)       readl((uart)->regs + (off<<REG_SHIFT))
> -#define omap_write(uart, off, val) writel((val), (uart)->regs + (off<<REG_SHIFT))
> +#define omap_read(uart, off)       readl((uart)->regs + ((off) << REG_SHIFT))
> +#define omap_write(uart, off, val) writel(val, (uart)->regs + \
> +                                               ((off) << REG_SHIFT))

the alignment looks off but could be fixed on commit


>  static struct omap_uart {
>      u32 baud, clock_hz, data_bits, parity, stop_bits, fifo_size;



From xen-devel-bounces@lists.xenproject.org Wed May 01 20:34:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:34:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715427.1117037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gel-0005Lx-Ty; Wed, 01 May 2024 20:34:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715427.1117037; Wed, 01 May 2024 20:34:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gel-0005Lq-R6; Wed, 01 May 2024 20:34:23 +0000
Received: by outflank-mailman (input) for mailman id 715427;
 Wed, 01 May 2024 20:34:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Gek-0005Lg-Ax
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:34:22 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2f315341-07fa-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:34:20 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2328D619EE;
 Wed,  1 May 2024 20:34:18 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2916C072AA;
 Wed,  1 May 2024 20:34:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f315341-07fa-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714595657;
	bh=IKatUJRv6K3vHjk7WFy6qYHSbp/IxGkXxamIiozg9nU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=lBLj1Ch5JkfWCi5j57NPnShmnPP8Dfc4gUKVPnEaQSFCzyWudv0fHh9PN83+9OWwf
	 QbJVJSiI9mk53BDPVoYQv+/dKm9nawnn8I6XdrrVDujYfGqc4Ew/FPdgYZur3yW9JW
	 19fKAgsIhVGlQsrIk8EvatI5OJhVCl+auFkdhWhYVxiR/1XCTZ+AaRPDDPXpF2052L
	 OtqX3VipLWHQxgu9g2zWhOHuTu/hx+9WgkPVjfcN6iShLi/h1DGuQWtpXM7tXunJb8
	 h6SIhbINJCspjUax6KD6BKlhe0mw+S5bOrxanpLmUqBcHK8sfnhZvC0p6lbQLyXlqZ
	 UpsPIC8eUmmsQ==
Date: Wed, 1 May 2024 13:34:14 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH] automation/eclair_enalysis: amend configuration for
 some MISRA rules
In-Reply-To: <7e8f6f4e0e007656d8809c4d521e73e642747d7f.1714394212.git.alessandro.zucchelli@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405011333530.497719@ubuntu-linux-20-04-desktop>
References: <7e8f6f4e0e007656d8809c4d521e73e642747d7f.1714394212.git.alessandro.zucchelli@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 29 Apr 2024, Alessandro Zucchelli wrote:
> Adjust ECLAIR configuration for rules: R21.14, R21.15, R21.16 by taking
> into account mem* macros defined in the Xen sources as if they were
> equivalent to the ones in Standard Library.
> 
> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/analysis.ecl | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index a604582da3..f3b634baba 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -19,6 +19,23 @@ map_strings("scheduled-analysis",analysis_kind)
>  
>  -enable=B.EXPLAIN
>  
> +-doc_begin="These configurations serve the purpose of recognizing the 'mem*' macros as
> +their Standard Library equivalents."
> +
> +-config=MC3R1.R21.14,call_select+=
> +{"macro(^memcmp$)&&any_arg(1..2, skip(__non_syntactic_paren_cast_stmts, node(string_literal)))",
> + "any()", violation, "%{__callslct_any_base_fmt()}", {{arg, "%{__callslct_arg_fmt()}"}}}
> +
> +-config=MC3R1.R21.15,call_args+=
> +{"macro(^mem(cmp|move|cpy)$)", {1, 2}, "unqual_pointee_compatible",
> + "%{__argscmpr_culprit_fmt()}", "%{__argscmpr_evidence_fmt()}"}
> +
> +-config=MC3R1.R21.16,call_select+=
> +{"macro(^memcmp$)&&any_arg(1..2, skip(__non_syntactic_paren_stmts, type(canonical(__memcmp_pte_types))))",
> + "any()", violation, "%{__callslct_any_base_fmt()}", {{arg,"%{__callslct_arg_type_fmt()}"}}}
> +
> +-doc_end
> +
>  -eval_file=toolchain.ecl
>  -eval_file=public_APIs.ecl
>  if(not(scheduled_analysis),
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:39:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:39:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715432.1117046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gk0-00063B-Fo; Wed, 01 May 2024 20:39:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715432.1117046; Wed, 01 May 2024 20:39:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gk0-000634-DH; Wed, 01 May 2024 20:39:48 +0000
Received: by outflank-mailman (input) for mailman id 715432;
 Wed, 01 May 2024 20:39:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Gjy-00062y-Pd
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:39:46 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ee6e6828-07fa-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:39:44 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 27D46CE11C4;
 Wed,  1 May 2024 20:39:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18CD8C4AF48;
 Wed,  1 May 2024 20:39:33 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee6e6828-07fa-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714595976;
	bh=5YO3GsemN4rW6B3DAytQmffj94Xk9tr4gbh/SLOBsHg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GLA4JLQv/NLSloJATnzmE7UGLXbdc+YRL7PWuDpmsuVStnK6/wFWikcw+hH3IhoH2
	 mSFcU6w3+iUJZb8D2/FvYZLsCsBYVuROXk5gcAjyGZIAGSNjGrzDFS5lfCYxUZwN0x
	 jBLeOZhzYk6hI1jnu5BbYv3n7xaO07oWZvl74q/0nya1xjti+7jAKNSOZ+Jyi06pqt
	 Hes89G5+2+MuwyyV/0kDftApgbKnRDwGi8XB3rY/BBKxy/SawSfrwifptapw/QT2q8
	 fXpYV9jhPSFqaSj2ThOCFW9J3pIt2j6HTbPIyeglcfKe3wykBzY5QWEKH+u1r5qNde
	 XxYsyNTJbiXPg==
Date: Wed, 1 May 2024 13:39:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 03/17] xen: mapcache: Refactor lock functions for
 multi-instance
In-Reply-To: <20240430164939.925307-4-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011339230.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-4-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Make the lock functions take MapCache * as argument. This is
> in preparation for supporting multiple caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 34 +++++++++++++++++-----------------
>  1 file changed, 17 insertions(+), 17 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index b7cefb78f7..3f11562075 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -74,14 +74,14 @@ typedef struct MapCache {
>  
>  static MapCache *mapcache;
>  
> -static inline void mapcache_lock(void)
> +static inline void mapcache_lock(MapCache *mc)
>  {
> -    qemu_mutex_lock(&mapcache->lock);
> +    qemu_mutex_lock(&mc->lock);
>  }
>  
> -static inline void mapcache_unlock(void)
> +static inline void mapcache_unlock(MapCache *mc)
>  {
> -    qemu_mutex_unlock(&mapcache->lock);
> +    qemu_mutex_unlock(&mc->lock);
>  }
>  
>  static inline int test_bits(int nr, int size, const unsigned long *addr)
> @@ -369,9 +369,9 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
>  {
>      uint8_t *p;
>  
> -    mapcache_lock();
> +    mapcache_lock(mapcache);
>      p = xen_map_cache_unlocked(phys_addr, size, lock, dma);
> -    mapcache_unlock();
> +    mapcache_unlock(mapcache);
>      return p;
>  }
>  
> @@ -384,7 +384,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>      ram_addr_t raddr;
>      int found = 0;
>  
> -    mapcache_lock();
> +    mapcache_lock(mapcache);
>      QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
>          if (reventry->vaddr_req == ptr) {
>              paddr_index = reventry->paddr_index;
> @@ -394,7 +394,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>          }
>      }
>      if (!found) {
> -        mapcache_unlock();
> +        mapcache_unlock(mapcache);
>          return RAM_ADDR_INVALID;
>      }
>  
> @@ -409,7 +409,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>          raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
>               ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
>      }
> -    mapcache_unlock();
> +    mapcache_unlock(mapcache);
>      return raddr;
>  }
>  
> @@ -480,9 +480,9 @@ static void xen_invalidate_map_cache_entry_bh(void *opaque)
>  {
>      XenMapCacheData *data = opaque;
>  
> -    mapcache_lock();
> +    mapcache_lock(mapcache);
>      xen_invalidate_map_cache_entry_unlocked(data->buffer);
> -    mapcache_unlock();
> +    mapcache_unlock(mapcache);
>  
>      aio_co_wake(data->co);
>  }
> @@ -498,9 +498,9 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
>                                  xen_invalidate_map_cache_entry_bh, &data);
>          qemu_coroutine_yield();
>      } else {
> -        mapcache_lock();
> +        mapcache_lock(mapcache);
>          xen_invalidate_map_cache_entry_unlocked(buffer);
> -        mapcache_unlock();
> +        mapcache_unlock(mapcache);
>      }
>  }
>  
> @@ -512,7 +512,7 @@ void xen_invalidate_map_cache(void)
>      /* Flush pending AIO before destroying the mapcache */
>      bdrv_drain_all();
>  
> -    mapcache_lock();
> +    mapcache_lock(mapcache);
>  
>      QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
>          if (!reventry->dma) {
> @@ -546,7 +546,7 @@ void xen_invalidate_map_cache(void)
>  
>      mapcache->last_entry = NULL;
>  
> -    mapcache_unlock();
> +    mapcache_unlock(mapcache);
>  }
>  
>  static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
> @@ -606,8 +606,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
>  {
>      uint8_t *p;
>  
> -    mapcache_lock();
> +    mapcache_lock(mapcache);
>      p = xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_addr, size);
> -    mapcache_unlock();
> +    mapcache_unlock(mapcache);
>      return p;
>  }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715435.1117057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gl9-0007Qb-Ot; Wed, 01 May 2024 20:40:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715435.1117057; Wed, 01 May 2024 20:40:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gl9-0007QU-M6; Wed, 01 May 2024 20:40:59 +0000
Received: by outflank-mailman (input) for mailman id 715435;
 Wed, 01 May 2024 20:40:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Gl9-0007P3-AV
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:40:59 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c33cf47-07fb-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:40:57 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id ADA1D61794;
 Wed,  1 May 2024 20:40:55 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8A961C072AA;
 Wed,  1 May 2024 20:40:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c33cf47-07fb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596055;
	bh=j96WBr+ckAnpUbK5HVQ29fNPUdvYgDjm1+efAxcME4o=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=AyQCOV9bZWw7X5lI/JknkOTUGFJ2UwMM84H3RXHkjTrS9gkV7qZhRp4Om0qse5P4Z
	 5Ir0k/Q5EKxOSLKvN/W3mKyD5YhzelyVKlJ+lYtJBG76ypj2WDHVC3EwdTslruBGma
	 YZfBjjX9fb++VU2fCnZgpdE9UScdwLps7Q9gHNtbXTYP+eRBVlV8nS/NDXiiN7NleU
	 CBqY4PSmIj/x7reYitY4Kde/3rn9mO4DBIzaFDwKtSC2XduleO/eUbMfBZlP/WEAm+
	 eIy+WXIJCtobo+GdLvMCDQmPgMioCLzz8kFvRX/cQ6dQv9DzY7CwxsryecaYlAaL9J
	 vmo/oeRRN6O6w==
Date: Wed, 1 May 2024 13:40:52 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 04/17] xen: mapcache: Refactor xen_map_cache for
 multi-instance
In-Reply-To: <20240430164939.925307-5-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011340460.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-5-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Make xen_map_cache take a MapCache as argument. This is in
> prepaparation to support multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 35 ++++++++++++++++++-----------------
>  1 file changed, 18 insertions(+), 17 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 3f11562075..896021d86f 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -240,7 +240,8 @@ static void xen_remap_bucket(MapCacheEntry *entry,
>      g_free(err);
>  }
>  
> -static uint8_t *xen_map_cache_unlocked(hwaddr phys_addr, hwaddr size,
> +static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> +                                       hwaddr phys_addr, hwaddr size,
>                                         uint8_t lock, bool dma)
>  {
>      MapCacheEntry *entry, *pentry = NULL,
> @@ -269,16 +270,16 @@ tryagain:
>          test_bit_size = XC_PAGE_SIZE;
>      }
>  
> -    if (mapcache->last_entry != NULL &&
> -        mapcache->last_entry->paddr_index == address_index &&
> +    if (mc->last_entry != NULL &&
> +        mc->last_entry->paddr_index == address_index &&
>          !lock && !size &&
>          test_bits(address_offset >> XC_PAGE_SHIFT,
>                    test_bit_size >> XC_PAGE_SHIFT,
> -                  mapcache->last_entry->valid_mapping)) {
> +                  mc->last_entry->valid_mapping)) {
>          trace_xen_map_cache_return(
> -            mapcache->last_entry->vaddr_base + address_offset
> +            mc->last_entry->vaddr_base + address_offset
>          );
> -        return mapcache->last_entry->vaddr_base + address_offset;
> +        return mc->last_entry->vaddr_base + address_offset;
>      }
>  
>      /* size is always a multiple of MCACHE_BUCKET_SIZE */
> @@ -291,7 +292,7 @@ tryagain:
>          cache_size = MCACHE_BUCKET_SIZE;
>      }
>  
> -    entry = &mapcache->entry[address_index % mapcache->nr_buckets];
> +    entry = &mc->entry[address_index % mc->nr_buckets];
>  
>      while (entry && (lock || entry->lock) && entry->vaddr_base &&
>              (entry->paddr_index != address_index || entry->size != cache_size ||
> @@ -326,10 +327,10 @@ tryagain:
>      if(!test_bits(address_offset >> XC_PAGE_SHIFT,
>                  test_bit_size >> XC_PAGE_SHIFT,
>                  entry->valid_mapping)) {
> -        mapcache->last_entry = NULL;
> +        mc->last_entry = NULL;
>  #ifdef XEN_COMPAT_PHYSMAP
> -        if (!translated && mapcache->phys_offset_to_gaddr) {
> -            phys_addr = mapcache->phys_offset_to_gaddr(phys_addr, size);
> +        if (!translated && mc->phys_offset_to_gaddr) {
> +            phys_addr = mc->phys_offset_to_gaddr(phys_addr, size);
>              translated = true;
>              goto tryagain;
>          }
> @@ -342,7 +343,7 @@ tryagain:
>          return NULL;
>      }
>  
> -    mapcache->last_entry = entry;
> +    mc->last_entry = entry;
>      if (lock) {
>          MapCacheRev *reventry = g_new0(MapCacheRev, 1);
>          entry->lock++;
> @@ -352,16 +353,16 @@ tryagain:
>              abort();
>          }
>          reventry->dma = dma;
> -        reventry->vaddr_req = mapcache->last_entry->vaddr_base + address_offset;
> -        reventry->paddr_index = mapcache->last_entry->paddr_index;
> +        reventry->vaddr_req = mc->last_entry->vaddr_base + address_offset;
> +        reventry->paddr_index = mc->last_entry->paddr_index;
>          reventry->size = entry->size;
> -        QTAILQ_INSERT_HEAD(&mapcache->locked_entries, reventry, next);
> +        QTAILQ_INSERT_HEAD(&mc->locked_entries, reventry, next);
>      }
>  
>      trace_xen_map_cache_return(
> -        mapcache->last_entry->vaddr_base + address_offset
> +        mc->last_entry->vaddr_base + address_offset
>      );
> -    return mapcache->last_entry->vaddr_base + address_offset;
> +    return mc->last_entry->vaddr_base + address_offset;
>  }
>  
>  uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
> @@ -370,7 +371,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
>      uint8_t *p;
>  
>      mapcache_lock(mapcache);
> -    p = xen_map_cache_unlocked(phys_addr, size, lock, dma);
> +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma);
>      mapcache_unlock(mapcache);
>      return p;
>  }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:42:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:42:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715438.1117068 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GmW-00080g-4H; Wed, 01 May 2024 20:42:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715438.1117068; Wed, 01 May 2024 20:42:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GmV-00080Z-VD; Wed, 01 May 2024 20:42:23 +0000
Received: by outflank-mailman (input) for mailman id 715438;
 Wed, 01 May 2024 20:42:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2GmU-00080J-M2
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:42:22 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4d39d447-07fb-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 22:42:21 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id CD03CCE1397;
 Wed,  1 May 2024 20:42:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B17DC072AA;
 Wed,  1 May 2024 20:42:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d39d447-07fb-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596137;
	bh=aPj5Km8vv8/JMkrkcHnSlAuBWQc5h+3YCn0oWQVhKgM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pw9mN7wE5W0kBjURzYbZPaBpQgGclQKmXA5GVhbirwYjucLiZQSWH7SP3t5q0YiBM
	 80nhotmvGLNq5tsd9pMDfz+KTrGgOouy+4lq1zwK/eTa2xvcecYAIlNOQ1I7iL8g6P
	 /y5EoO/q/FuK8kScb+QtEGdus9bhzjjs0PqDzz0dOiPhXA0sTslGh/P77zDGgASjpt
	 dyWbkanAWsVhV7+zIK8f3pb4LvCKdzNFoRIwIYAQGl+D5T/eeY8dAxf+ZNwK6Ml3Ju
	 WV9eJ3D6Fu/Amw7iz5yMn2Ajf9Lniq6j3iJHgF3cynIsJiiHVpGTpGmN01+BqCBkH+
	 +0BBo1A0RjUaQ==
Date: Wed, 1 May 2024 13:42:12 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 05/17] xen: mapcache: Refactor xen_remap_bucket for
 multi-instance
In-Reply-To: <20240430164939.925307-6-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011341590.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-6-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add MapCache argument to xen_remap_bucket in preparation
> to support multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 896021d86f..326a9b61ca 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -139,7 +139,8 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>      mapcache->entry = g_malloc0(size);
>  }
>  
> -static void xen_remap_bucket(MapCacheEntry *entry,
> +static void xen_remap_bucket(MapCache *mc,
> +                             MapCacheEntry *entry,
>                               void *vaddr,
>                               hwaddr size,
>                               hwaddr address_index,
> @@ -313,14 +314,14 @@ tryagain:
>      if (!entry) {
>          entry = g_new0(MapCacheEntry, 1);
>          pentry->next = entry;
> -        xen_remap_bucket(entry, NULL, cache_size, address_index, dummy);
> +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
>      } else if (!entry->lock) {
>          if (!entry->vaddr_base || entry->paddr_index != address_index ||
>                  entry->size != cache_size ||
>                  !test_bits(address_offset >> XC_PAGE_SHIFT,
>                      test_bit_size >> XC_PAGE_SHIFT,
>                      entry->valid_mapping)) {
> -            xen_remap_bucket(entry, NULL, cache_size, address_index, dummy);
> +            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
>          }
>      }
>  
> @@ -587,7 +588,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
>  
>      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
>  
> -    xen_remap_bucket(entry, entry->vaddr_base,
> +    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
>                       cache_size, address_index, false);
>      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
>                  test_bit_size >> XC_PAGE_SHIFT,
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:44:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:44:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715443.1117076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Go8-0000e9-CU; Wed, 01 May 2024 20:44:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715443.1117076; Wed, 01 May 2024 20:44:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Go8-0000e2-9r; Wed, 01 May 2024 20:44:04 +0000
Received: by outflank-mailman (input) for mailman id 715443;
 Wed, 01 May 2024 20:44:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Go7-0000cC-89
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:44:03 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8895d4c5-07fb-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:44:00 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 30EFACE13A1;
 Wed,  1 May 2024 20:43:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7C9AC072AA;
 Wed,  1 May 2024 20:43:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8895d4c5-07fb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596236;
	bh=dMPFSioNpeKX2paci1uyX5JomVV7+chkLPnWnqSos4g=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kaD1Ev+gEHDi1RM3cxVcWSCs1dGKiF7D6a4EEPW01+V2qzesATeYhJtscbgpusFWj
	 WZ0b84Wlrnw+W+nPG/R8jiLTiRUihlQaFtdPpwJh7hq0MkXQpdmFaI4IERTa3c5xX0
	 +zoAUdmoHAA4Fz18zF82DnmDI6V6vs8ATDbtDaOoodBL9OmlYBkfKLY9psFSo6Rlbl
	 Qzpi5NoRJvRrf/7rq/0dithiLHVzGz7EtIgwPm2EeQWB0BuxWoTbj21x3oW3Eexp14
	 8V/Kxs7WITjcqm2EdaiwDRGwUsLhce7VVyBnSdkxaW0p9tP7a7632d/UZ0Y2WFv6Ok
	 lXiUWh9A3P23g==
Date: Wed, 1 May 2024 13:43:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 06/17] xen: mapcache: Break out
 xen_ram_addr_from_mapcache_single
In-Reply-To: <20240430164939.925307-7-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011343460.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-7-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Break out xen_ram_addr_from_mapcache_single(), a multi-cache
> aware version of xen_ram_addr_from_mapcache.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 326a9b61ca..d2deff70c8 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -377,7 +377,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
>      return p;
>  }
>  
> -ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> +static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
>  {
>      MapCacheEntry *entry = NULL;
>      MapCacheRev *reventry;
> @@ -386,8 +386,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>      ram_addr_t raddr;
>      int found = 0;
>  
> -    mapcache_lock(mapcache);
> -    QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
> +    mapcache_lock(mc);
> +    QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
>          if (reventry->vaddr_req == ptr) {
>              paddr_index = reventry->paddr_index;
>              size = reventry->size;
> @@ -396,11 +396,11 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>          }
>      }
>      if (!found) {
> -        mapcache_unlock(mapcache);
> +        mapcache_unlock(mc);
>          return RAM_ADDR_INVALID;
>      }
>  
> -    entry = &mapcache->entry[paddr_index % mapcache->nr_buckets];
> +    entry = &mc->entry[paddr_index % mc->nr_buckets];
>      while (entry && (entry->paddr_index != paddr_index || entry->size != size)) {
>          entry = entry->next;
>      }
> @@ -411,10 +411,15 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>          raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
>               ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
>      }
> -    mapcache_unlock(mapcache);
> +    mapcache_unlock(mc);
>      return raddr;
>  }
>  
> +ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> +{
> +    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
> +}
> +
>  static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
>  {
>      MapCacheEntry *entry = NULL, *pentry = NULL;
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:46:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715449.1117087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GqE-0001Ip-Pw; Wed, 01 May 2024 20:46:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715449.1117087; Wed, 01 May 2024 20:46:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GqE-0001Ii-Mq; Wed, 01 May 2024 20:46:14 +0000
Received: by outflank-mailman (input) for mailman id 715449;
 Wed, 01 May 2024 20:46:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2GqD-0001Ia-Dd
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:46:13 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d5656890-07fb-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:46:09 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id CC645CE139B;
 Wed,  1 May 2024 20:46:05 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4F93AC072AA;
 Wed,  1 May 2024 20:46:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5656890-07fb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596365;
	bh=NArfGwWOnY6xyqS3W5ubl410JY9KsJxvONK6riU3GOc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HHwNvdY2yKMVzPmRX8GkA9qORm2TkcII9FkOlpbsMceq9wUHFpeUG6DwZbn/75m7S
	 gSwRdvKsTLBYG0tW540NuLGw4WW32Sm2ojn5QCMbmQReQelL/s9u1cSsa6dOzCud+E
	 1RDEjW/MvHQdXBM8oNZOl9uOC+TCcP2cpFuknsM+7kd9eI/d3fnyILqbc5iRvnRKox
	 uCcTy3cCvjL7wn2P/DMFPzq81WK3qNmqY9Q3tcchT4N6juZoco/8ZoRhINDZXIHdCG
	 4/Ny13TpMRNKoqa/WH4whayB+NiNSmP/C87BkFDTXqDkWZhupOslvWrD/MYY3nddtQ
	 0+dEZAM3hh95Q==
Date: Wed, 1 May 2024 13:46:01 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 07/17] xen: mapcache: Refactor
 xen_replace_cache_entry_unlocked
In-Reply-To: <20240430164939.925307-8-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011345140.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-8-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add MapCache argument to xen_replace_cache_entry_unlocked in
> preparation for supporting multiple map caches.
> 
> No functional change.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  hw/xen/xen-mapcache.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index d2deff70c8..6e758eff94 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -556,7 +556,8 @@ void xen_invalidate_map_cache(void)
>      mapcache_unlock(mapcache);
>  }
>  
> -static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
> +static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> +                                                 hwaddr old_phys_addr,
>                                                   hwaddr new_phys_addr,
>                                                   hwaddr size)
>  {
> @@ -578,7 +579,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
>          cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
>      }
>  
> -    entry = &mapcache->entry[address_index % mapcache->nr_buckets];
> +    entry = &mc->entry[address_index % mc->nr_buckets];
>      while (entry && !(entry->paddr_index == address_index &&
>                        entry->size == cache_size)) {
>          entry = entry->next;

There is still a global mapcache pointer in use in this function:

  xen_remap_bucket(mapcache, entry, entry->vaddr_base,


> @@ -614,7 +615,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
>      uint8_t *p;
>  
>      mapcache_lock(mapcache);
> -    p = xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_addr, size);
> +    p = xen_replace_cache_entry_unlocked(mapcache, old_phys_addr,
> +                                         new_phys_addr, size);
>      mapcache_unlock(mapcache);
>      return p;
>  }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:47:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:47:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715458.1117096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GrE-0001rP-67; Wed, 01 May 2024 20:47:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715458.1117096; Wed, 01 May 2024 20:47:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GrE-0001rG-3X; Wed, 01 May 2024 20:47:16 +0000
Received: by outflank-mailman (input) for mailman id 715458;
 Wed, 01 May 2024 20:47:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2GrC-0001Ia-BK
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:47:14 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fbcdffd5-07fb-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:47:12 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DC33B61904;
 Wed,  1 May 2024 20:47:10 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C082C072AA;
 Wed,  1 May 2024 20:47:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbcdffd5-07fb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596430;
	bh=IVFlG7/OVVyRn1qxCbkB1eYEoi2tCROExBghLoORPn8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ZuKGz8T++r4J9yyT7gooAxbI6V4XYbjjj35r+Q65vhiG00BNIgnMIKVEEdbD00LiV
	 WTmpfZyWtwkyc5OaDFEuAKlTtJwnw7WMMRmTjaVuUXbxUYOgkS5HXHXGTbxg/uRXEq
	 pFuHoFqFlAEri3F2MrPGYRUcAtA/Ofj1g29kUMY4c0RS9j2p4AdQ6iONBQ1pJ684GX
	 I9T1wTkhm4x1Ryd3EVUN4GEAI+t1H0Y1piD53f4jB10ri7ENpPOS179FFMSZP7WYFw
	 YHjlz64D7KgBQVvcBR6xAvOB44IXdQrbhjAPG+PKOsq9pRiiIcux/PS0lk8nA4lZz8
	 K1FaO9fKKT5QQ==
Date: Wed, 1 May 2024 13:47:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 08/17] xen: mapcache: Refactor
 xen_invalidate_map_cache_entry_unlocked
In-Reply-To: <20240430164939.925307-9-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011347000.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-9-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add MapCache argument to xen_invalidate_map_cache_entry_unlocked.
> This is in preparation for supporting multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 21 +++++++++++----------
>  1 file changed, 11 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 6e758eff94..34454da2f6 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -420,7 +420,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>      return xen_ram_addr_from_mapcache_single(mapcache, ptr);
>  }
>  
> -static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
> +static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> +                                                    uint8_t *buffer)
>  {
>      MapCacheEntry *entry = NULL, *pentry = NULL;
>      MapCacheRev *reventry;
> @@ -428,7 +429,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
>      hwaddr size;
>      int found = 0;
>  
> -    QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
> +    QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
>          if (reventry->vaddr_req == buffer) {
>              paddr_index = reventry->paddr_index;
>              size = reventry->size;
> @@ -438,7 +439,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
>      }
>      if (!found) {
>          trace_xen_invalidate_map_cache_entry_unlocked_not_found(buffer);
> -        QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
> +        QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
>              trace_xen_invalidate_map_cache_entry_unlocked_found(
>                  reventry->paddr_index,
>                  reventry->vaddr_req
> @@ -446,15 +447,15 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
>          }
>          return;
>      }
> -    QTAILQ_REMOVE(&mapcache->locked_entries, reventry, next);
> +    QTAILQ_REMOVE(&mc->locked_entries, reventry, next);
>      g_free(reventry);
>  
> -    if (mapcache->last_entry != NULL &&
> -        mapcache->last_entry->paddr_index == paddr_index) {
> -        mapcache->last_entry = NULL;
> +    if (mc->last_entry != NULL &&
> +        mc->last_entry->paddr_index == paddr_index) {
> +        mc->last_entry = NULL;
>      }
>  
> -    entry = &mapcache->entry[paddr_index % mapcache->nr_buckets];
> +    entry = &mc->entry[paddr_index % mc->nr_buckets];
>      while (entry && (entry->paddr_index != paddr_index || entry->size != size)) {
>          pentry = entry;
>          entry = entry->next;
> @@ -488,7 +489,7 @@ static void xen_invalidate_map_cache_entry_bh(void *opaque)
>      XenMapCacheData *data = opaque;
>  
>      mapcache_lock(mapcache);
> -    xen_invalidate_map_cache_entry_unlocked(data->buffer);
> +    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
>      mapcache_unlock(mapcache);
>  
>      aio_co_wake(data->co);
> @@ -506,7 +507,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
>          qemu_coroutine_yield();
>      } else {
>          mapcache_lock(mapcache);
> -        xen_invalidate_map_cache_entry_unlocked(buffer);
> +        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
>          mapcache_unlock(mapcache);
>      }
>  }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:48:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:48:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715461.1117107 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GsQ-0002QC-Gu; Wed, 01 May 2024 20:48:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715461.1117107; Wed, 01 May 2024 20:48:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GsQ-0002Q5-DZ; Wed, 01 May 2024 20:48:30 +0000
Received: by outflank-mailman (input) for mailman id 715461;
 Wed, 01 May 2024 20:48:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2GsP-0002Pv-Ti
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:48:29 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 286287b2-07fc-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 22:48:28 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 7214FCE139C;
 Wed,  1 May 2024 20:48:25 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BA10C072AA;
 Wed,  1 May 2024 20:48:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 286287b2-07fc-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596504;
	bh=uB6XjGflA6a/Z9KWVzsfz+aOYvxcUGxBLL/TwR9pkFs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iwekIfW4ncfiIJmnmkdKgjvkep1Mtu/5PqdNUKPK4DjotSAhdrFGNF15yY5//N7/e
	 K1VxqOgnLkfoPkk7xTCOf3hqRoMNcNOp7lygw2QHtsdXBJxT6I2yvkqJ2Wo3nyYrk7
	 tPfeP5kbp9W0CJMn/JyMClbtqLB8fh0JTlQB+1ENg3lBinJIuSiqRZK8dxkT1IubrX
	 G89QC/1Yvu1uwQOGTyG1nnNkHcRIFpYGCoknkppuwhJhH5Oem/i8Lf5XIX+flZ7aeM
	 MIs9DuoSsHZ/Oejv41Dd4d6wxqPf4Pxldgze/ZZp1BayZUT/v86E5WlG5NS2gPUX8A
	 vNR0wxzo+Fi2Q==
Date: Wed, 1 May 2024 13:48:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 09/17] xen: mapcache: Break out
 xen_invalidate_map_cache_single()
In-Reply-To: <20240430164939.925307-10-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011348130.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-10-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Break out xen_invalidate_map_cache_single().
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 25 +++++++++++++++----------
>  1 file changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 34454da2f6..dd08cd296b 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -512,17 +512,14 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
>      }
>  }
>  
> -void xen_invalidate_map_cache(void)
> +static void xen_invalidate_map_cache_single(MapCache *mc)
>  {
>      unsigned long i;
>      MapCacheRev *reventry;
>  
> -    /* Flush pending AIO before destroying the mapcache */
> -    bdrv_drain_all();
> -
> -    mapcache_lock(mapcache);
> +    mapcache_lock(mc);
>  
> -    QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
> +    QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
>          if (!reventry->dma) {
>              continue;
>          }
> @@ -530,8 +527,8 @@ void xen_invalidate_map_cache(void)
>                                         reventry->vaddr_req);
>      }
>  
> -    for (i = 0; i < mapcache->nr_buckets; i++) {
> -        MapCacheEntry *entry = &mapcache->entry[i];
> +    for (i = 0; i < mc->nr_buckets; i++) {
> +        MapCacheEntry *entry = &mc->entry[i];
>  
>          if (entry->vaddr_base == NULL) {
>              continue;
> @@ -552,9 +549,17 @@ void xen_invalidate_map_cache(void)
>          entry->valid_mapping = NULL;
>      }
>  
> -    mapcache->last_entry = NULL;
> +    mc->last_entry = NULL;
>  
> -    mapcache_unlock(mapcache);
> +    mapcache_unlock(mc);
> +}
> +
> +void xen_invalidate_map_cache(void)
> +{
> +    /* Flush pending AIO before destroying the mapcache */
> +    bdrv_drain_all();
> +
> +    xen_invalidate_map_cache_single(mapcache);
>  }
>  
>  static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:52:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:52:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715467.1117117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gvk-0003u5-UJ; Wed, 01 May 2024 20:51:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715467.1117117; Wed, 01 May 2024 20:51:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Gvk-0003ty-RP; Wed, 01 May 2024 20:51:56 +0000
Received: by outflank-mailman (input) for mailman id 715467;
 Wed, 01 May 2024 20:51:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2Gvj-0003ts-DG
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:51:55 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a1eefdbd-07fc-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:51:52 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D12F2CE138D;
 Wed,  1 May 2024 20:51:48 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42476C4AF14;
 Wed,  1 May 2024 20:51:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1eefdbd-07fc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596707;
	bh=HR+a7o2ZkxIM6jzJQZXWaVSl9k5NgRnHGAuDGSr6slg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ALDUNZnM70fk4wzKcV3eCENzn3/uRATGRYE9br0E6W/ZkmBon8cgKpqJAmU9vbObQ
	 WlS/uOTMmhm1sKXutTcz4jFYzdWB+MgZzttC9peCq3XBbJ3/NDy+rYmH5YDqrvC+M2
	 AO8nSPOOA7xV9fbNpjAxhNiNxhjaEHkGD7xVT+tJnTWAhGEZysQEGOlIpHg2hBLl3d
	 5OSFAGdVu+ZqvWwC8XYYX/qBrXeYHYq2sII0WXKkIzG+IuGxK4/ZfLcfdZIbQtBGYs
	 b1lin2VhOMhQVDLPERnDqnzZsZslla+hsEZCFgvqDKhta23jaqJrc9HEQ46K9xZqtm
	 iE74nsTNQYaNg==
Date: Wed, 1 May 2024 13:51:44 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 10/17] xen: mapcache: Break out
 xen_map_cache_init_single()
In-Reply-To: <20240430164939.925307-11-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011351380.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-11-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Break out xen_map_cache_init_single() in preparation for
> adding multiple map caches.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 53 ++++++++++++++++++++++++++-----------------
>  1 file changed, 32 insertions(+), 21 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index dd08cd296b..72a7e25e3e 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -93,23 +93,44 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
>          return 0;
>  }
>  
> -void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
> +static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
> +                                           void *opaque,
> +                                           unsigned long max_size)
>  {
>      unsigned long size;
> -    struct rlimit rlimit_as;
> +    MapCache *mc;
> +
> +    mc = g_new0(MapCache, 1);
> +
> +    mc->phys_offset_to_gaddr = f;
> +    mc->opaque = opaque;
> +    qemu_mutex_init(&mc->lock);
> +
> +    QTAILQ_INIT(&mc->locked_entries);
>  
> -    mapcache = g_new0(MapCache, 1);
> +    mc->max_mcache_size = max_size;
>  
> -    mapcache->phys_offset_to_gaddr = f;
> -    mapcache->opaque = opaque;
> -    qemu_mutex_init(&mapcache->lock);
> +    mc->nr_buckets =
> +        (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
> +          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
> +         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
>  
> -    QTAILQ_INIT(&mapcache->locked_entries);
> +    size = mc->nr_buckets * sizeof(MapCacheEntry);
> +    size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
> +    trace_xen_map_cache_init(mc->nr_buckets, size);
> +    mc->entry = g_malloc0(size);
> +    return mc;
> +}
> +
> +void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
> +{
> +    struct rlimit rlimit_as;
> +    unsigned long max_mcache_size;
>  
>      if (geteuid() == 0) {
>          rlimit_as.rlim_cur = RLIM_INFINITY;
>          rlimit_as.rlim_max = RLIM_INFINITY;
> -        mapcache->max_mcache_size = MCACHE_MAX_SIZE;
> +        max_mcache_size = MCACHE_MAX_SIZE;
>      } else {
>          getrlimit(RLIMIT_AS, &rlimit_as);
>          rlimit_as.rlim_cur = rlimit_as.rlim_max;
> @@ -119,24 +140,14 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>                          " memory is not infinity");
>          }
>          if (rlimit_as.rlim_max < MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
> -            mapcache->max_mcache_size = rlimit_as.rlim_max -
> -                NON_MCACHE_MEMORY_SIZE;
> +            max_mcache_size = rlimit_as.rlim_max - NON_MCACHE_MEMORY_SIZE;
>          } else {
> -            mapcache->max_mcache_size = MCACHE_MAX_SIZE;
> +            max_mcache_size = MCACHE_MAX_SIZE;
>          }
>      }
>  
> +    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
>      setrlimit(RLIMIT_AS, &rlimit_as);
> -
> -    mapcache->nr_buckets =
> -        (((mapcache->max_mcache_size >> XC_PAGE_SHIFT) +
> -          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
> -         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
> -
> -    size = mapcache->nr_buckets * sizeof (MapCacheEntry);
> -    size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
> -    trace_xen_map_cache_init(mapcache->nr_buckets, size);
> -    mapcache->entry = g_malloc0(size);
>  }
>  
>  static void xen_remap_bucket(MapCache *mc,
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 20:55:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 20:55:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715472.1117126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GzG-0005CA-Bw; Wed, 01 May 2024 20:55:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715472.1117126; Wed, 01 May 2024 20:55:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2GzG-0005C3-9A; Wed, 01 May 2024 20:55:34 +0000
Received: by outflank-mailman (input) for mailman id 715472;
 Wed, 01 May 2024 20:55:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2GzF-0005Bw-5W
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 20:55:33 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 23aa7882-07fd-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 22:55:30 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 2E86ACE139C;
 Wed,  1 May 2024 20:55:25 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C551AC072AA;
 Wed,  1 May 2024 20:55:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23aa7882-07fd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714596924;
	bh=rRM5UhhS3g5g91tWUVSmHsobPuVReB0n8SEWKW0JLmw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UOWmAIlwD1ZizziNao5SYkynMT4BK4AhFsid3YYv5lEObtar0/K+d6G/k5KPHuXr5
	 W6Kyw9ojRmZdPPea64jVFbMlgCH/Af/ZmKVcmks+YFDWNX4o2NRbkphtlgzF9yVGip
	 MqakMAGMYpYmFO0c9JCZLeeBjtX4XNG4+W/sOeaP6uojE/wp2EM6EDhNlcK2mDNEun
	 qrkGRu+sQbO/hXaSQlUQGki9OD09QD9wbaI1+i46ZoO+c6TVolEXU/Pt1xsVwjOwUS
	 hLOurVZUunTL6LcXWL6mW1JB1USL+FWr+AES73mkitYKYmCwqBuCoYXCmW4y7m6GvY
	 GbFyjKMNT3NDw==
Date: Wed, 1 May 2024 13:55:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 11/17] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime
 configurable
In-Reply-To: <20240430164939.925307-12-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011355140.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-12-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 52 ++++++++++++++++++++++++++-----------------
>  1 file changed, 31 insertions(+), 21 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 72a7e25e3e..4f98d284dd 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -23,13 +23,10 @@
>  
>  
>  #if HOST_LONG_BITS == 32
> -#  define MCACHE_BUCKET_SHIFT 16
>  #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
>  #else
> -#  define MCACHE_BUCKET_SHIFT 20
>  #  define MCACHE_MAX_SIZE     (1UL<<35) /* 32GB Cap */
>  #endif
> -#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
>  
>  /* This is the size of the virtual address space reserve to QEMU that will not
>   * be use by MapCache.
> @@ -65,7 +62,8 @@ typedef struct MapCache {
>      /* For most cases (>99.9%), the page address is the same. */
>      MapCacheEntry *last_entry;
>      unsigned long max_mcache_size;
> -    unsigned int mcache_bucket_shift;
> +    unsigned int bucket_shift;
> +    unsigned long bucket_size;
>  
>      phys_offset_to_gaddr_t phys_offset_to_gaddr;
>      QemuMutex lock;
> @@ -95,6 +93,7 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
>  
>  static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
>                                             void *opaque,
> +                                           unsigned int bucket_shift,
>                                             unsigned long max_size)
>  {
>      unsigned long size;
> @@ -108,12 +107,14 @@ static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
>  
>      QTAILQ_INIT(&mc->locked_entries);
>  
> +    mc->bucket_shift = bucket_shift;
> +    mc->bucket_size = 1UL << bucket_shift;
>      mc->max_mcache_size = max_size;
>  
>      mc->nr_buckets =
>          (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
> -          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
> -         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
> +          (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
> +         (bucket_shift - XC_PAGE_SHIFT));
>  
>      size = mc->nr_buckets * sizeof(MapCacheEntry);
>      size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
> @@ -126,6 +127,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>  {
>      struct rlimit rlimit_as;
>      unsigned long max_mcache_size;
> +    unsigned int bucket_shift;
> +
> +    if (HOST_LONG_BITS == 32) {
> +        bucket_shift = 16;
> +    } else {
> +        bucket_shift = 20;
> +    }
>  
>      if (geteuid() == 0) {
>          rlimit_as.rlim_cur = RLIM_INFINITY;
> @@ -146,7 +154,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>          }
>      }
>  
> -    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
> +    mapcache = xen_map_cache_init_single(f, opaque,
> +                                         bucket_shift,
> +                                         max_mcache_size);
>      setrlimit(RLIMIT_AS, &rlimit_as);
>  }
>  
> @@ -195,7 +205,7 @@ static void xen_remap_bucket(MapCache *mc,
>      entry->valid_mapping = NULL;
>  
>      for (i = 0; i < nb_pfn; i++) {
> -        pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
> +        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
>      }
>  
>      /*
> @@ -266,8 +276,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>      bool dummy = false;
>  
>  tryagain:
> -    address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
> -    address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
> +    address_index  = phys_addr >> mc->bucket_shift;
> +    address_offset = phys_addr & (mc->bucket_size - 1);
>  
>      trace_xen_map_cache(phys_addr);
>  
> @@ -294,14 +304,14 @@ tryagain:
>          return mc->last_entry->vaddr_base + address_offset;
>      }
>  
> -    /* size is always a multiple of MCACHE_BUCKET_SIZE */
> +    /* size is always a multiple of mc->bucket_size */
>      if (size) {
>          cache_size = size + address_offset;
> -        if (cache_size % MCACHE_BUCKET_SIZE) {
> -            cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
> +        if (cache_size % mc->bucket_size) {
> +            cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
>          }
>      } else {
> -        cache_size = MCACHE_BUCKET_SIZE;
> +        cache_size = mc->bucket_size;
>      }
>  
>      entry = &mc->entry[address_index % mc->nr_buckets];
> @@ -419,7 +429,7 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
>          trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
>          raddr = RAM_ADDR_INVALID;
>      } else {
> -        raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
> +        raddr = (reventry->paddr_index << mc->bucket_shift) +
>               ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
>      }
>      mapcache_unlock(mc);
> @@ -582,8 +592,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>      hwaddr address_index, address_offset;
>      hwaddr test_bit_size, cache_size = size;
>  
> -    address_index  = old_phys_addr >> MCACHE_BUCKET_SHIFT;
> -    address_offset = old_phys_addr & (MCACHE_BUCKET_SIZE - 1);
> +    address_index  = old_phys_addr >> mc->bucket_shift;
> +    address_offset = old_phys_addr & (mc->bucket_size - 1);
>  
>      assert(size);
>      /* test_bit_size is always a multiple of XC_PAGE_SIZE */
> @@ -592,8 +602,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>          test_bit_size += XC_PAGE_SIZE - (test_bit_size % XC_PAGE_SIZE);
>      }
>      cache_size = size + address_offset;
> -    if (cache_size % MCACHE_BUCKET_SIZE) {
> -        cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
> +    if (cache_size % mc->bucket_size) {
> +        cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
>      }
>  
>      entry = &mc->entry[address_index % mc->nr_buckets];
> @@ -606,8 +616,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>          return NULL;
>      }
>  
> -    address_index  = new_phys_addr >> MCACHE_BUCKET_SHIFT;
> -    address_offset = new_phys_addr & (MCACHE_BUCKET_SIZE - 1);
> +    address_index  = new_phys_addr >> mc->bucket_shift;
> +    address_offset = new_phys_addr & (mc->bucket_size - 1);
>  
>      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
>  
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 21:01:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 21:01:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715480.1117137 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2H4v-0006yZ-Uh; Wed, 01 May 2024 21:01:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715480.1117137; Wed, 01 May 2024 21:01:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2H4v-0006yS-Rs; Wed, 01 May 2024 21:01:25 +0000
Received: by outflank-mailman (input) for mailman id 715480;
 Wed, 01 May 2024 21:01:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2H4u-0006yM-QH
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 21:01:24 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6777847-07fd-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 23:01:22 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C679861904;
 Wed,  1 May 2024 21:01:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B5CDC072AA;
 Wed,  1 May 2024 21:01:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6777847-07fd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714597280;
	bh=dtQxKmYbUUAzg4PTwI2TWD3iTTIdynxCbqah4NU3uMY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VdMU18FGFqxEyNw0aTaOPXv3Ul/pct0Iux+iBqKTrtj4cklV45ZUfU02ttl+Kcou1
	 v2UD5KcbOZPVKahSrDO4YFe+mv9tOe2afCiRtm+bw+Yf7rJPNwX0EgTJtwXMsTAhmz
	 J0GwF1MF6Ob/tMTgNFRoALk6BfmfY0VI6D8GKxWecK7bOfKwBlFFQAhIQI/00ciHYf
	 0eCkPb2QQpktbhCP4bXySCiz1vyREpiFg+NURL3RAN6m+s1+NnkdJvSxAf5rd3MO0e
	 bJCG9DCYMa6q0xVYkZgJq8Ou206Tkke9AEmhioil3oASgMlz6f18c/lpkR/NEMUKRJ
	 xAotpSCdUaKrQ==
Date: Wed, 1 May 2024 14:01:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 12/17] xen: mapcache: Unmap first entries in buckets
In-Reply-To: <20240430164939.925307-13-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011401110.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-13-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> When invalidating memory ranges, if we happen to hit the first
> entry in a bucket we were never unmapping it. This was harmless
> for foreign mappings but now that we're looking to reuse the
> mapcache for transient grant mappings, we must unmap entries
> when invalidated.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 4f98d284dd..0365311788 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -486,18 +486,22 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>          return;
>      }
>      entry->lock--;
> -    if (entry->lock > 0 || pentry == NULL) {
> +    if (entry->lock > 0) {
>          return;
>      }
>  
> -    pentry->next = entry->next;
>      ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
>      if (munmap(entry->vaddr_base, entry->size) != 0) {
>          perror("unmap fails");
>          exit(-1);
>      }
> -    g_free(entry->valid_mapping);
> -    g_free(entry);
> +    if (pentry) {
> +        pentry->next = entry->next;
> +        g_free(entry->valid_mapping);
> +        g_free(entry);
> +    } else {
> +        memset(entry, 0, sizeof *entry);
> +    }
>  }
>  
>  typedef struct XenMapCacheData {
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 21:03:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 21:03:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715484.1117146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2H6s-0007Zg-Dx; Wed, 01 May 2024 21:03:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715484.1117146; Wed, 01 May 2024 21:03:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2H6s-0007ZZ-Ah; Wed, 01 May 2024 21:03:26 +0000
Received: by outflank-mailman (input) for mailman id 715484;
 Wed, 01 May 2024 21:03:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2H6r-0007ZR-3N
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 21:03:25 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3ee31c6d-07fe-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 23:03:23 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 9B783619BA;
 Wed,  1 May 2024 21:03:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 52E7EC072AA;
 Wed,  1 May 2024 21:03:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ee31c6d-07fe-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714597402;
	bh=2WZWdYOx2khYwwOaLHZ6CakoHF0il4iSJg/sGKO5QtE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=nx5Np+7/aKvpOiUcx2oJgi2mzU3PYHgVuYzyb3Iqwl7JARmd9X4N7n8oxT2O6+2w3
	 VDq+EMHWrRdxOuIqoVli5GZiP8e+zDFrlXHXQs1ZnI6EskAjIfJVTU3TBil4ztZt0i
	 YNaCjcBaXqo3aHAPONO6oo740BP5/YfL1OZNFYHJn207gWtwv3lmk8Fp1uYTd2WoMG
	 6TpUB1Mec9Zkw9Iumm93EzqamFpn4E//rHZAyimCB0UFClAUaCqjIaqmBP8me18+LC
	 IBqKEWfr9NsHlyC5kFciUNsNnx+cqe8mNVK+JKT4PA/WXc5o314hzu47qyUebU0abg
	 ceO+Y9SdheO8Q==
Date: Wed, 1 May 2024 14:03:18 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write
 xen_map_cache()
In-Reply-To: <20240430164939.925307-14-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011403120.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-14-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Propagate MR and is_write to xen_map_cache().
> This is in preparation for adding support for grant mappings.
> 
> No functional change.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c         | 10 ++++++----
>  include/sysemu/xen-mapcache.h | 11 +++++++----
>  system/physmem.c              | 31 +++++++++++++++++++++++--------
>  3 files changed, 36 insertions(+), 16 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 0365311788..09b5f36d9c 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -264,7 +264,7 @@ static void xen_remap_bucket(MapCache *mc,
>  
>  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>                                         hwaddr phys_addr, hwaddr size,
> -                                       uint8_t lock, bool dma)
> +                                       uint8_t lock, bool dma, bool is_write)
>  {
>      MapCacheEntry *entry, *pentry = NULL,
>                    *free_entry = NULL, *free_pentry = NULL;
> @@ -387,13 +387,15 @@ tryagain:
>      return mc->last_entry->vaddr_base + address_offset;
>  }
>  
> -uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
> -                       uint8_t lock, bool dma)
> +uint8_t *xen_map_cache(MemoryRegion *mr,
> +                       hwaddr phys_addr, hwaddr size,
> +                       uint8_t lock, bool dma,
> +                       bool is_write)
>  {
>      uint8_t *p;
>  
>      mapcache_lock(mapcache);
> -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma);
> +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
>      mapcache_unlock(mapcache);
>      return p;
>  }
> diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> index 10c2e3082a..1ec9e66752 100644
> --- a/include/sysemu/xen-mapcache.h
> +++ b/include/sysemu/xen-mapcache.h
> @@ -18,8 +18,9 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
>  
>  void xen_map_cache_init(phys_offset_to_gaddr_t f,
>                          void *opaque);
> -uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
> -                       uint8_t lock, bool dma);
> +uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
> +                       uint8_t lock, bool dma,
> +                       bool is_write);
>  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
>  void xen_invalidate_map_cache_entry(uint8_t *buffer);
>  void xen_invalidate_map_cache(void);
> @@ -33,10 +34,12 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
>  {
>  }
>  
> -static inline uint8_t *xen_map_cache(hwaddr phys_addr,
> +static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> +                                     hwaddr phys_addr,
>                                       hwaddr size,
>                                       uint8_t lock,
> -                                     bool dma)
> +                                     bool dma,
> +                                     bool is_write)
>  {
>      abort();
>  }
> diff --git a/system/physmem.c b/system/physmem.c
> index f114b972a5..ad7a8c7d95 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2190,11 +2190,22 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
>  
>  /*
>   * Return a host pointer to guest's ram.
> + * For Xen, foreign mappings get created if they don't already exist.
> + *
> + * @block: block for the RAM to lookup (optional and may be NULL).
> + * @addr: address within the memory region.
> + * @size: pointer to requested size (optional and may be NULL).
> + *        size may get modified and return a value smaller than
> + *        what was requested.
> + * @lock: wether to lock the mapping in xen-mapcache until invalidated.
> + * @is_write: hint wether to map RW or RO in the xen-mapcache.
> + *            (optional and may always be set to true).
>   *
>   * Called within RCU critical section.
>   */
>  static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
> -                                 hwaddr *size, bool lock)
> +                                 hwaddr *size, bool lock,
> +                                 bool is_write)
>  {
>      hwaddr len = 0;
>  
> @@ -2217,10 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>           * In that case just map the requested area.
>           */
>          if (block->offset == 0) {
> -            return xen_map_cache(addr, len, lock, lock);
> +            return xen_map_cache(block->mr, addr, len, lock, lock,
> +                                 is_write);
>          }
>  
> -        block->host = xen_map_cache(block->offset, block->max_length, 1, lock);
> +        block->host = xen_map_cache(block->mr, block->offset,
> +                                    block->max_length, 1,
> +                                    lock, is_write);
>      }
>  
>      return ramblock_ptr(block, addr);
> @@ -2236,7 +2250,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>   */
>  void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr)
>  {
> -    return qemu_ram_ptr_length(ram_block, addr, NULL, false);
> +    return qemu_ram_ptr_length(ram_block, addr, NULL, false, true);
>  }
>  
>  /* Return the offset of a hostpointer within a ramblock */
> @@ -2746,7 +2760,7 @@ static MemTxResult flatview_write_continue_step(MemTxAttrs attrs,
>      } else {
>          /* RAM case */
>          uint8_t *ram_ptr = qemu_ram_ptr_length(mr->ram_block, mr_addr, l,
> -                                               false);
> +                                               false, true);
>  
>          memmove(ram_ptr, buf, *l);
>          invalidate_and_set_dirty(mr, mr_addr, *l);
> @@ -2839,7 +2853,7 @@ static MemTxResult flatview_read_continue_step(MemTxAttrs attrs, uint8_t *buf,
>      } else {
>          /* RAM case */
>          uint8_t *ram_ptr = qemu_ram_ptr_length(mr->ram_block, mr_addr, l,
> -                                               false);
> +                                               false, false);
>  
>          memcpy(buf, ram_ptr, *l);
>  
> @@ -3233,7 +3247,7 @@ void *address_space_map(AddressSpace *as,
>      *plen = flatview_extend_translation(fv, addr, len, mr, xlat,
>                                          l, is_write, attrs);
>      fuzz_dma_read_cb(addr, *plen, mr);
> -    return qemu_ram_ptr_length(mr->ram_block, xlat, plen, true);
> +    return qemu_ram_ptr_length(mr->ram_block, xlat, plen, true, is_write);
>  }
>  
>  /* Unmaps a memory region previously mapped by address_space_map().
> @@ -3329,7 +3343,8 @@ int64_t address_space_cache_init(MemoryRegionCache *cache,
>          l = flatview_extend_translation(cache->fv, addr, len, mr,
>                                          cache->xlat, l, is_write,
>                                          MEMTXATTRS_UNSPECIFIED);
> -        cache->ptr = qemu_ram_ptr_length(mr->ram_block, cache->xlat, &l, true);
> +        cache->ptr = qemu_ram_ptr_length(mr->ram_block, cache->xlat, &l, true,
> +                                         is_write);
>      } else {
>          cache->ptr = NULL;
>      }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 21:07:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 21:07:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715489.1117157 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2HAG-0008E9-RI; Wed, 01 May 2024 21:06:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715489.1117157; Wed, 01 May 2024 21:06:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2HAG-0008E2-OQ; Wed, 01 May 2024 21:06:56 +0000
Received: by outflank-mailman (input) for mailman id 715489;
 Wed, 01 May 2024 21:06:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2HAG-0008Dw-Cv
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 21:06:56 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b9ff9529-07fe-11ef-909b-e314d9c70b13;
 Wed, 01 May 2024 23:06:54 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 2BB15CE12A3;
 Wed,  1 May 2024 21:06:46 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 121B8C072AA;
 Wed,  1 May 2024 21:06:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9ff9529-07fe-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714597605;
	bh=xKWmAR8bRUG3hI2S8/E9BMmyLRlDJp0dhBplSrRXHPs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=B2AmvL80x1kupedRVlAr399URmuGcwn3cwpbwfyoUtVewfBPYmPJni0NCIjfAJcmd
	 Py1e7coDbmDim3EQ24z0YS3iR/u0WHj2J/OwYw9MTV93ZmNF9RIqI9lL2nJQLkTHLf
	 z1tKh29NeN8Pj2Q4swIjQ6Re68UkSXBwPptU6B2quqQ1+eRLNytsfaTc5WBapkGmEW
	 1BpkMSCAYKCPnlti+/eu6KPCd+8EmiOura+0WhNbduzalZxgqYBxP2KHU/jNtt1t5H
	 ka/XXZ85YcxpNqT3Zifd3SnrL7LfxvQvnhKIIDwKUpbKLerjSkYFsIs1tkl72YzKXF
	 8El8bd6GxMr7A==
Date: Wed, 1 May 2024 14:06:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 14/17] xen: Add xen_mr_is_memory()
In-Reply-To: <20240430164939.925307-15-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011405050.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-15-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add xen_mr_is_memory() to abstract away tests for the
> xen_memory MR.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

There is an important change in this patch below


> ---
>  hw/xen/xen-hvm-common.c | 8 +++++++-
>  include/sysemu/xen.h    | 8 ++++++++
>  system/physmem.c        | 2 +-
>  3 files changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> index 1627da7398..0267b88d26 100644
> --- a/hw/xen/xen-hvm-common.c
> +++ b/hw/xen/xen-hvm-common.c
> @@ -12,6 +12,12 @@
>  
>  MemoryRegion xen_memory;
>  
> +/* Check for xen memory.  */
> +bool xen_mr_is_memory(MemoryRegion *mr)
> +{
> +    return mr == &xen_memory;
> +}
> +
>  void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
>                     Error **errp)
>  {
> @@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
>          return;
>      }
>  
> -    if (mr == &xen_memory) {
> +    if (xen_mr_is_memory(mr)) {
>          return;
>      }
>  
> diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> index 754ec2e6cb..dc72f83bcb 100644
> --- a/include/sysemu/xen.h
> +++ b/include/sysemu/xen.h
> @@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length);
>  void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>                     struct MemoryRegion *mr, Error **errp);
>  
> +bool xen_mr_is_memory(MemoryRegion *mr);
> +
>  #else /* !CONFIG_XEN_IS_POSSIBLE */
>  
>  #define xen_enabled() 0
> @@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>      g_assert_not_reached();
>  }
>  
> +static inline bool xen_mr_is_memory(MemoryRegion *mr)
> +{
> +    g_assert_not_reached();
> +    return false;
> +}
> +
>  #endif /* CONFIG_XEN_IS_POSSIBLE */
>  
>  #endif
> diff --git a/system/physmem.c b/system/physmem.c
> index ad7a8c7d95..1a5ffcba2a 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>           * because we don't want to map the entire memory in QEMU.
>           * In that case just map the requested area.
>           */
> -        if (block->offset == 0) {
> +        if (xen_mr_is_memory(block->mr)) {

This changes this check from block->offset == 0 to block->mr ==
&xen_memory. I think that's correct in all cases (x86 machines, ARM
machines) but I wanted to highlight it.

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>





>              return xen_map_cache(block->mr, addr, len, lock, lock,
>                                   is_write);
>          }
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 21:24:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 21:24:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715496.1117166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2HRI-0004XO-8E; Wed, 01 May 2024 21:24:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715496.1117166; Wed, 01 May 2024 21:24:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2HRI-0004XH-5C; Wed, 01 May 2024 21:24:32 +0000
Received: by outflank-mailman (input) for mailman id 715496;
 Wed, 01 May 2024 21:24:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aa2W=ME=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2HRH-0004XB-GL
 for xen-devel@lists.xenproject.org; Wed, 01 May 2024 21:24:31 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2f931596-0801-11ef-b4bb-af5377834399;
 Wed, 01 May 2024 23:24:28 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 7A152CE12A3;
 Wed,  1 May 2024 21:24:24 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59437C072AA;
 Wed,  1 May 2024 21:24:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f931596-0801-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714598663;
	bh=FLVZmdHfgEb1TFuY1+2IjEROy5nvohlsD6fcTAlsMjI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=qkhr28GvYs0ZF1LSITNc9av5nYO9vOAQjhmuYVa/vhQcGcUm7WsiYKnDM/5T4E48q
	 8tAynT3npaYSeVi3iwz3I3WVyoe0koA/lFOv3t9zy5OZI6VLnzMzWX5B/K4W7jaEUF
	 JQzrSU+MSroUlYr/ke+Q2SkS0Iu1aTlg8tni+fgqRMUGs4u9eEsQkDLwwhICIBe4qf
	 O3c5l8zMPnAr/f1RSSnr3lwSee1xPbH2rkLKtVZo/D3MWtpKn+UX4K3x/KUrl/sDOh
	 60BSPPwoUcfXpBvGVhc+m8FJvaezCvzF/2fC5cu9m4RMEoJuXOISr9k9hS9kfj0Kxy
	 4WYJZd6HoHNIg==
Date: Wed, 1 May 2024 14:24:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
In-Reply-To: <20240430164939.925307-16-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-16-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> The current mapcache assumes that all memory is mapped
> in a single RAM MR (the first one with offset 0). Remove
> this assumption and propagate the offset to the mapcache
> so it can do reverse mappings (from hostptr -> ram_addr).
> 
> This is in preparation for adding grant mappings.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>


Looking at xen_remap_bucket, it is only using address_index (without
adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
understand that address_index already includes the ram_offset.

Meaning that if we want to map foreign mapping at address 0x5000, then
address_index would be 0x5000, even if ram_offset is 0x1000.

But then looking xen_ram_addr_from_mapcache_single ram_offset is added
to paddr_index to calculate the physical address. So in that case we
would want address_index to be 0x4000 and ram_offset to be 0x1000. But
xen_remap_bucket would have to sum address_index and ram_offset to map
foreign memory.

So I am a bit confused, did I get it wrong? One more comment below.


> ---
>  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
>  include/sysemu/xen-mapcache.h |  2 ++
>  system/physmem.c              |  8 ++++----
>  3 files changed, 24 insertions(+), 11 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 09b5f36d9c..1b32d0c003 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
>  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
>      uint8_t flags;
>      hwaddr size;
> +
> +    /* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr).  */
> +    ram_addr_t ram_offset;
>      struct MapCacheEntry *next;
>  } MapCacheEntry;
>  
> @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
>                               void *vaddr,
>                               hwaddr size,
>                               hwaddr address_index,
> -                             bool dummy)
> +                             bool dummy,
> +                             ram_addr_t ram_offset)
>  {
>      uint8_t *vaddr_base;
>      xen_pfn_t *pfns;
> @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
>      entry->size = size;
>      entry->valid_mapping = g_new0(unsigned long,
>                                    BITS_TO_LONGS(size >> XC_PAGE_SHIFT));
> +    entry->ram_offset = ram_offset;
>  
>      if (dummy) {
>          entry->flags |= XEN_MAPCACHE_ENTRY_DUMMY;
> @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
>  
>  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>                                         hwaddr phys_addr, hwaddr size,
> +                                       ram_addr_t ram_offset,
>                                         uint8_t lock, bool dma, bool is_write)
>  {
>      MapCacheEntry *entry, *pentry = NULL,
> @@ -335,14 +341,16 @@ tryagain:
>      if (!entry) {
>          entry = g_new0(MapCacheEntry, 1);
>          pentry->next = entry;
> -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> +                         ram_offset);
>      } else if (!entry->lock) {
>          if (!entry->vaddr_base || entry->paddr_index != address_index ||
>                  entry->size != cache_size ||
>                  !test_bits(address_offset >> XC_PAGE_SHIFT,
>                      test_bit_size >> XC_PAGE_SHIFT,
>                      entry->valid_mapping)) {
> -            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> +            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> +                             ram_offset);
>          }
>      }
>  
> @@ -389,13 +397,15 @@ tryagain:
>  
>  uint8_t *xen_map_cache(MemoryRegion *mr,
>                         hwaddr phys_addr, hwaddr size,
> +                       ram_addr_t ram_addr_offset,
>                         uint8_t lock, bool dma,
>                         bool is_write)
>  {
>      uint8_t *p;
>  
>      mapcache_lock(mapcache);
> -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
> +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> +                               lock, dma, is_write);
>      mapcache_unlock(mapcache);
>      return p;
>  }
> @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
>          raddr = RAM_ADDR_INVALID;
>      } else {
>          raddr = (reventry->paddr_index << mc->bucket_shift) +
> -             ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
> +             ((unsigned long) ptr - (unsigned long) entry->vaddr_base) +
> +             entry->ram_offset;
>      }
>      mapcache_unlock(mc);
>      return raddr;
> @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>  
>      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
>  
> -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> -                     cache_size, address_index, false);
> +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> +                     cache_size, address_index, false, entry->ram_offset);
>      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
>                  test_bit_size >> XC_PAGE_SHIFT,
>                  entry->valid_mapping)) {
> diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> index 1ec9e66752..b5e3ea1bc0 100644
> --- a/include/sysemu/xen-mapcache.h
> +++ b/include/sysemu/xen-mapcache.h
> @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
>  void xen_map_cache_init(phys_offset_to_gaddr_t f,
>                          void *opaque);
>  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
> +                       ram_addr_t ram_addr_offset,
>                         uint8_t lock, bool dma,
>                         bool is_write);
>  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
>  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
>                                       hwaddr phys_addr,
>                                       hwaddr size,
> +                                     ram_addr_t ram_addr_offset,
>                                       uint8_t lock,
>                                       bool dma,
>                                       bool is_write)
> diff --git a/system/physmem.c b/system/physmem.c
> index 1a5ffcba2a..5b16eeccca 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>           * In that case just map the requested area.
>           */
>          if (xen_mr_is_memory(block->mr)) {
> -            return xen_map_cache(block->mr, addr, len, lock, lock,
> -                                 is_write);
> +            return xen_map_cache(block->mr, addr, len, block->offset,
> +                                 lock, lock, is_write);

Have you considered not tracking offset and address separately and
simply do this?

            return xen_map_cache(block->mr, addr + block->offset, len, 
                                 lock, lock, is_write);


>          }
>  
>          block->host = xen_map_cache(block->mr, block->offset,
> -                                    block->max_length, 1,
> -                                    lock, is_write);
> +                                    block->max_length, 0,
> +                                    1, lock, is_write);
>      }
>  
>      return ramblock_ptr(block, addr);
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 01 23:07:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 May 2024 23:07:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715505.1117189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2J2o-0006ok-GP; Wed, 01 May 2024 23:07:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715505.1117189; Wed, 01 May 2024 23:07:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2J2o-0006od-DI; Wed, 01 May 2024 23:07:22 +0000
Received: by outflank-mailman (input) for mailman id 715505;
 Wed, 01 May 2024 23:07:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2J2n-0006oT-CS; Wed, 01 May 2024 23:07:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2J2n-00025S-7H; Wed, 01 May 2024 23:07:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2J2m-0002ix-Qv; Wed, 01 May 2024 23:07:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2J2m-0001N6-QQ; Wed, 01 May 2024 23:07:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PVcooQNYoYHTZ0oN5GFuXyLlAFZbjxJMtBLtsYidu4c=; b=GkSN3hTF5FTVNUqQh3EatvIe84
	BKZkp8GeusRhhiWn73ZXOSFSXqUxZLb81mXkyF51iZpzBo4c1f/jOKMFANL3exDIrwHgX7wFtQYyU
	4n0NFzO/T4B9PIaSEq7UPlqUIka8Gm7Ifz9pP6jfHFhWFq9c7qGyEdoOXMwu1/CsgUMw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185893-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185893: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=18daea77cca626f590fb140fc11e3a43c5d41354
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 May 2024 23:07:20 +0000

flight 185893 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185893/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot         fail in 185888 pass in 185893
 test-armhf-armhf-xl-credit2   8 xen-boot                   fail pass in 185888

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185888 blocked in 185870
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185888 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185888 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185888 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-libvirt      8 xen-boot                     fail  like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                18daea77cca626f590fb140fc11e3a43c5d41354
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    2 days
Testing same since   185888  2024-04-30 20:31:50 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Potapenko <glider@google.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Audra Mitchell <audra@redhat.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Chuck Lever <chuck.lever@oracle.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Hans de Goede <hdegoede@redhat.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  John Garry <john.g.garry@oracle.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Sven Schnelle <svens@linux.ibm.com>
  Tejun Heo <tj@kernel.org>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Xingyou Chen <rockrush@rockwork.org>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1064 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 02 01:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 01:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715513.1117199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2LZ2-00042o-8s; Thu, 02 May 2024 01:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715513.1117199; Thu, 02 May 2024 01:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2LZ2-00042h-4j; Thu, 02 May 2024 01:48:48 +0000
Received: by outflank-mailman (input) for mailman id 715513;
 Thu, 02 May 2024 01:48:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2LZ0-00041m-MQ; Thu, 02 May 2024 01:48:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2LZ0-0006VD-Gz; Thu, 02 May 2024 01:48:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2LZ0-0007Y4-5U; Thu, 02 May 2024 01:48:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2LZ0-0004wj-4M; Thu, 02 May 2024 01:48:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CAfa9HGOLAgWBlz/Kbzkmgfrs5+D5qRLzswyv0+VZrs=; b=YungIOxlA6Oz30TKUlD6SOmb5j
	ZDCY16zxpuFzzZhqSF9FaUdXhHoNkMyBYo+bKF8iH2ZskclRV5ceQr+zi39+bQLDmqG9CioiDPGKr
	Lzin5hyZx59BvMzq6ZXkdx1B2Tgr/i5wglKOgZxXNEJIpvSuxi81jSyS1veoUnhaqnsY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185895-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185895: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
X-Osstest-Versions-That:
    xen=9e30bd8f4a8cee83368276c5149c7e2304efe5a2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 01:48:46 +0000

flight 185895 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185895/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b
baseline version:
 xen                  9e30bd8f4a8cee83368276c5149c7e2304efe5a2

Last test of basis   185880  2024-04-30 14:03:49 Z    1 days
Testing same since   185895  2024-05-01 22:02:19 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Federico Serafini <federico.serafini@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9e30bd8f4a..feb9158a62  feb9158a620040846d76981acbe8ea9e2255a07b -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:10:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:10:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715525.1117210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Pdc-0001WT-4l; Thu, 02 May 2024 06:09:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715525.1117210; Thu, 02 May 2024 06:09:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Pdc-0001WM-00; Thu, 02 May 2024 06:09:48 +0000
Received: by outflank-mailman (input) for mailman id 715525;
 Thu, 02 May 2024 06:09:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Pdb-0001WG-3g
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:09:47 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9245ef7a-084a-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:09:44 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2dcc8d10d39so87918471fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:09:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 da11-20020a056000196b00b00349ff2e0345sm375751wrb.70.2024.05.01.23.09.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:09:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9245ef7a-084a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714630184; x=1715234984; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=A+46cTmuwJjSR/h4dKhc9adX6YxlzB3CKB2cLjGOi9Y=;
        b=DALnIoi1qV9AeemgiZqGYV5oxM+dTTqII5A7X2TiPv8rajx+G0inE5TFL8l1N8CzFR
         Q1NXXR7XE3BmNGvS81KWeyaJaee6FEMQz0gl4+wf8qZw12fvz3a6GTfGFRgMevZFEBcu
         44fKMADpoGFrIw8j+QpHVO7qHpL5PA8qLDk4TWtxUgVnl93se5FWIgFNxbhjDCGXLizB
         Vd50CdqoOSfudKtlxy0LEUFvBKQPbOQzoTtLtGV9UPLk0BRu45YFyVlQs7uTNhNkiOwx
         el1aBNOUeQKhXNgi0AtTkH/iXqdPwUNfZkg2xyE/IA2JZcipCxmSFRrXSNCNToz+Y4RX
         DCeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714630184; x=1715234984;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=A+46cTmuwJjSR/h4dKhc9adX6YxlzB3CKB2cLjGOi9Y=;
        b=KX4sNfgyVgBd/AQqRfAxY7XpnUzCGgveLUTjqNywMRksp7Iuuda6fexp938pZH2pn0
         HbYBsyx7zODZBmprgik0cfxZ3JF1nu1jsu3Lx0G2t1LJjgzOTnBc+GbM1s2LOotaqIAH
         jRiFyDu2bIlmrylJT1FuTybtaRLbz6Cv1yvwrA13Y04gTjMMEsYsA886ucEzFrdu0ZSk
         fDUFmoq3AeUHxh61YypsKoc6PuJbGtjMzLnk8KOelZoixaA5F1OnVibWhyceEbXzcVVQ
         2bhiwprNHiRFN/Rj15JOtRpjuhDqTNXPaMjeikx/RzVrESRPDJlYpH96mW0z5HGuarFQ
         hC8w==
X-Forwarded-Encrypted: i=1; AJvYcCXTYyxzhEU4YTNCytUiyxuE6w3DVnbp5ST5Fc2y1gaQtvs+mbQhrkW33NQoTujvfXFxAYSMNR3oUf2EYxZnJ5+XfzbsmpCBLWDLm/B66pU=
X-Gm-Message-State: AOJu0YxTiI64ZodJAeo1pL83VdFpCSSyN6OTRjrk1rUxFXVxAn+pLun0
	Di1ns1lfcdmCBpsY+iSsrPc8RbNx02FgYeu74vlJn3Sdn/ESTZLP321NlQ2rzw==
X-Google-Smtp-Source: AGHT+IG7QAKO6zUDxtwbKSWW29cukgU7JUMi5JHjdYqFlTHW+Sr30inMOp6lrfyOt7Ot2tMHqsLqDg==
X-Received: by 2002:a2e:b784:0:b0:2df:907e:6de3 with SMTP id n4-20020a2eb784000000b002df907e6de3mr3186593ljo.35.1714630184116;
        Wed, 01 May 2024 23:09:44 -0700 (PDT)
Message-ID: <16815d66-ffe7-45e9-8311-5d4ed8754d2f@suse.com>
Date: Thu, 2 May 2024 08:09:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Content-Language: en-US
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-2-luca.fancellu@arm.com>
 <cd2352aa-e94b-43ef-b3fc-359cc1f43fb0@suse.com>
 <2AB02E85-2746-4DD6-A821-B858E52A6954@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2AB02E85-2746-4DD6-A821-B858E52A6954@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.05.2024 08:54, Luca Fancellu wrote:
>> On 30 Apr 2024, at 12:43, Jan Beulich <jbeulich@suse.com> wrote:
>> On 30.04.2024 13:09, Luca Fancellu wrote:
>>> --- a/xen/include/xen/kernel.h
>>> +++ b/xen/include/xen/kernel.h
>>> @@ -54,6 +54,27 @@
>>>         typeof_field(type, member) *__mptr = (ptr);             \
>>>         (type *)( (char *)__mptr - offsetof(type,member) );})
>>>
>>> +/**
>>> + * __struct_group() - Create a mirrored named and anonyomous struct
>>> + *
>>> + * @TAG: The tag name for the named sub-struct (usually empty)
>>> + * @NAME: The identifier name of the mirrored sub-struct
>>> + * @ATTRS: Any struct attributes (usually empty)
>>> + * @MEMBERS: The member declarations for the mirrored structs
>>> + *
>>> + * Used to create an anonymous union of two structs with identical layout
>>> + * and size: one anonymous and one named. The former's members can be used
>>> + * normally without sub-struct naming, and the latter can be used to
>>> + * reason about the start, end, and size of the group of struct members.
>>> + * The named struct can also be explicitly tagged for layer reuse, as well
>>> + * as both having struct attributes appended.
>>> + */
>>> +#define __struct_group(TAG, NAME, ATTRS, MEMBERS...) \
>>> +    union { \
>>> +        struct { MEMBERS } ATTRS; \
>>> +        struct TAG { MEMBERS } ATTRS NAME; \
>>> +    } ATTRS
>>
>> Besides my hesitance towards having this construct, can you explain why
>> ATTR needs using 3 times, i.e. also on the wrapping union?
> 
> The original commit didn’t have the third ATTRS, but afterwards it was introduced due to
> this:
> 
> https://patchwork.kernel.org/project/linux-wireless/patch/20231120110607.98956-1-dmantipov@yandex.ru/#25610045

Hmm. Since generally packing propagates to containing compound types, I'd
say this cannot go without a code comment, or at the very least a mention
in the description. But: Do we use this old ABI in Xen at all? IOW can we
get away without this 3rd instance?

> Now, I have to say that for the Origin tag I used the SHA of the commit introducing the macro
> and the SHA doing this modification is different, how are these cases handled?

I'd say the hash of the original commit is enough even if the 3rd instance
needs keeping for whatever reason.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:14:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715529.1117219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Phm-00036I-IY; Thu, 02 May 2024 06:14:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715529.1117219; Thu, 02 May 2024 06:14:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Phm-00036B-G0; Thu, 02 May 2024 06:14:06 +0000
Received: by outflank-mailman (input) for mailman id 715529;
 Thu, 02 May 2024 06:14:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Phl-000365-R8
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:14:05 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2d615b12-084b-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 08:14:04 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d8a2cbe1baso98664891fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:14:04 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n19-20020a05600c4f9300b0041c14061c71sm710626wmq.15.2024.05.01.23.14.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:14:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d615b12-084b-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714630444; x=1715235244; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FwcLNS+C1DcjgGVhCSJg6/XZY1PNQQ9sD3B4uti7yck=;
        b=IOZ9I3awFibb7fnnGgoanP/2lIxsHsGGeKkbrjcj5gTbpxW+vhrDaoJ3cCtiTBnsc4
         nQtMw/xwC4w0VwDn0GJ+NJ93FSyo8ccZOxtVt+iP7WRDdVw43Ju6+N/monAuRnXBsqhd
         ICggrQ7ElE5yJBgyVWb5qKKRSGt9XIEgg4P5xd4grtZXdF6LsmlVXscvDwofBi2QYFVa
         unCVLj/jYa3ebaNAimzc9BGbs8TQxINXEhLtS1kNEqcai/4ntfyAxy8BKhM2zLazM5w2
         swz0/um7doJk2LPRWTd8/20VRjv14i4DtpXyG4X9oLo49ULgIZzFpJJcgFmdrJz96gqc
         twfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714630444; x=1715235244;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FwcLNS+C1DcjgGVhCSJg6/XZY1PNQQ9sD3B4uti7yck=;
        b=aevoCg8n3qk7i8Bz7d62z0TVwbcGFBHy1v3O1tE2UG54cypXvGfh8iAWkhjnw3U180
         +kSGy3yI6S1RjTGvFHcUb+6MJSS1eBiKEHfD88GHd/sdf/yL/BHYTypa0Pyyh6mDHwdm
         zuvsvAAi1a/yUl2GEYJ2hyiG1lOeLWHLQqRneoKBqG13HvBZ+2RMsujU0NeNwa/W3/kH
         2V7hV3QLl4kS8N8/WqPZdJKa8DelOEjqZQzLIoMHewUcMijrsuK8anKOFfNn/y7WWL20
         2gl1amorHlY4f+8adV67+WMSkz1sjboeXsQKyV0zCGNeR03poEh4onfOf/8kn86+4oim
         z3xQ==
X-Forwarded-Encrypted: i=1; AJvYcCXO9ij92fmd54cLOjiPtMWZgo/KqFRVXyVPGcz30tZrM+GKpuzOs3UHWr2S37ypYm0YYJ6kkLoAydsILNSOmoIqBQGwBAIHhO0vu20rKdg=
X-Gm-Message-State: AOJu0YxmLpzFiJNfqt28MQ8Ff//f8qgRKbQayKX2GEIt4iP+15FdGcDr
	M5r4ZSnfIChEDd8Ko4p+lTgoTFcYsNS7m/U6m3JjcVI1G9i7VZRkB6jLV5YJxw==
X-Google-Smtp-Source: AGHT+IH1fe5sxrvlFkdTSFmEC7kjbe9VBmIlwn/s1M1vuZdwO8ZBz6tEuwesvDgxMU5E/bCjHwDUoQ==
X-Received: by 2002:a2e:9b12:0:b0:2dd:d3a0:e096 with SMTP id u18-20020a2e9b12000000b002ddd3a0e096mr894362lji.31.1714630442516;
        Wed, 01 May 2024 23:14:02 -0700 (PDT)
Message-ID: <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
Date: Thu, 2 May 2024 08:14:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Content-Language: en-US
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.05.2024 08:57, Luca Fancellu wrote:
> Hi Jan,
> 
>> On 30 Apr 2024, at 12:37, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 30.04.2024 13:09, Luca Fancellu wrote:
>>> --- a/xen/arch/arm/include/asm/setup.h
>>> +++ b/xen/arch/arm/include/asm/setup.h
>>> @@ -64,18 +64,20 @@ struct membank {
>>> };
>>>
>>> struct membanks {
>>> -    unsigned int nr_banks;
>>> -    unsigned int max_banks;
>>> +    __struct_group(membanks_hdr, common, ,
>>> +        unsigned int nr_banks;
>>> +        unsigned int max_banks;
>>> +    );
>>>     struct membank bank[];
>>> };
>>
>> I'm afraid I can't spot why __struct_group() is needed here. Why would just
>> one of the two more straightforward
>>
>> struct membanks {
>>    struct membanks_hdr {
>>        unsigned int nr_banks;
>>        unsigned int max_banks;
>>    );
>>    struct membank bank[];
>> };
>>
> 
> At the first sight I thought this solution could have worked, however GCC brought me back down to earth
> remembering me that flexible array members can’t be left alone in an empty structure:
> 
> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:70:6: error: declaration does not declare anything [-Werror]
> 70 | };
> | ^
> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:71:20: error: flexible array member in a struct with no named members
> 71 | struct membank bank[];
> | ^~~~
> [...]

Since for patch 1 you looked at Linux'es uapi/linux/stddef.h, the solution
to this lies there, in __DECLARE_FLEX_ARRAY(). Alongside or instead of
borrowing __struct_group(), we could consider borrowing this as well. Or
open-code it just here, for the time being (perhaps my preference). Yet
it's not clear to me that doing so will actually be enough to make things
work for you.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:19:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:19:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715535.1117229 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2PnD-0003k5-5M; Thu, 02 May 2024 06:19:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715535.1117229; Thu, 02 May 2024 06:19:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2PnD-0003jy-2c; Thu, 02 May 2024 06:19:43 +0000
Received: by outflank-mailman (input) for mailman id 715535;
 Thu, 02 May 2024 06:19:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2PnB-0003js-NU
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:19:41 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f4639b44-084b-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:19:38 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41bab13ca81so57574815e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:19:39 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q8-20020a05600c46c800b0041ac4aafd3dsm731893wmo.12.2024.05.01.23.19.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:19:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4639b44-084b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714630779; x=1715235579; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PkggZ4OXXNUCn2qbM23wSBqc8fUtkVETEdYNVwITcrU=;
        b=KzBWwEcfH6PHqNkFXOUdhROd4TyjnjQqIvH1F0tkoWblABk5MDtoJ8YgYth6e7WjHR
         kZCxtgyzo2JCip7AP3bKV3qDIxFInRAdmNxmDzlR8gSmKOqtxY38zuPA/i4MFhHSl8Qq
         sfV9hPG6soChcl6aNSUpLxUrbk5kS62FAolTkyXS9u/keGNPansPmY3b50xE0eNG2v/C
         JKdyxsHaUaL2/1iI1ynbbp+cb7yrjoPIrUCPqvy1Rcj9YY+DHJXaz+zpSU4+xxm7C0Db
         2dUbEDI0qB/sNhP1pTXcKElHLKidXx0d5mDon4Qi7v1pu+azSjpO4yirvVHONHYrB0HE
         /LJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714630779; x=1715235579;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PkggZ4OXXNUCn2qbM23wSBqc8fUtkVETEdYNVwITcrU=;
        b=uXtAS91Zz444KtgWWe12EP/g9AQrbS1VFxBJ34YHyt6aw/Jq5fzv5c0Y/15v7lK677
         y95cG5z10LC5KPRXh3II0jCzYsr/+E3hy9QvkL3sDsOMVhncuO5C9o9R2iQSEUyI5CWu
         W1YnvQdgibr3GFG44aOCuq0ECLd4O0ddlpxWWHJKOapFfNdOFFSY+bD4ttbhwallahlW
         nHFm82NTRWJ3zMPnLjH7pPNepyxecXbWEbtpLXCW39YYQpTPM1huwGw205Vbe7xf2sfE
         0xN2Z4UUtmJK5a6APlq6x7kLSSb4GCFhgsnuxYDU1MmPTK56dgcxjkn1vVyULVFyzrT2
         l+wA==
X-Forwarded-Encrypted: i=1; AJvYcCViBIRR+kUHVTfgXm2via4dGEbvqoRsiDHXI+0NIq5pC5JfhjGuWcqRmTU+fxd7NpUn+W1j/3VdB47D2ZlBWtwB1QiY1ygFcnSkG5IKriA=
X-Gm-Message-State: AOJu0YyEKPT/SxlrGhzTV5xJC0a7wJ/E+QgNxY0O/HfybTI36aM0xrEW
	m44k8ktRgX2PQbXVcTWfz3Myk64Ti12Gndtn1EZSizJpAN7cFtO9mHRQFp61YQ==
X-Google-Smtp-Source: AGHT+IFh3nBzt9EpmT6izizC73WBCENrf8v1TcXH+lNctNZZvVpwF9f9t36cfYlp/yIwhXsUQyJb+A==
X-Received: by 2002:a05:600c:4587:b0:41b:d85c:d3e2 with SMTP id r7-20020a05600c458700b0041bd85cd3e2mr4215450wmo.38.1714630779142;
        Wed, 01 May 2024 23:19:39 -0700 (PDT)
Message-ID: <b6460ffc-9308-4acf-8894-259a9cdd0ee2@suse.com>
Date: Thu, 2 May 2024 08:19:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/7] x86: Make the maximum number of altp2m views
 configurable
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <cover.1714322424.git.w1benny@gmail.com>
 <0041438ff7a5d5b2fad59b676f4ece80470bf3b3.1714322424.git.w1benny@gmail.com>
 <5600e97e-5d44-4901-a867-b48192e9525f@suse.com>
 <CAKBKdXjsz_By_nZHDBpzTASSwMQUw61-7UA0YYrSTN+YDivqWQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAKBKdXjsz_By_nZHDBpzTASSwMQUw61-7UA0YYrSTN+YDivqWQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30.04.2024 18:00, Petr Beneš wrote:
> On Tue, Apr 30, 2024 at 4:27 PM Jan Beulich <jbeulich@suse.com> wrote:
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -685,6 +685,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>          return -EINVAL;
>>>      }
>>>
>>> +    if ( config->max_altp2m > MAX_EPTP )
>>> +    {
>>> +        dprintk(XENLOG_INFO, "max_altp2m must be <= %u\n", MAX_EPTP);
>>> +        return -EINVAL;
>>> +    }
>>
>> ... using MAX_EPTP here feels like a layering violation to me. Imo there want
>> to be separate constants, tied together with a suitably placed BUILD_BUG_ON().
>>
>> Furthermore comparisons like this (there are further ones elsewhere) suggest
>> there is a (continued) naming issue: A max_ or MAX_ prefix ought to name a
>> "maximum valid value", not "number of permitted values". This is not a
>> request to alter MAX_EPTP, but one to make sure the new struct fields really
>> have matching names and purposes.
> 
> Do you have any proposals? I was considering nr_altp2m. Another
> question is what it should be named in xl.cfg - also nr_altp2m? I was
> too hesitant to name it like that, since there aren't any nr_* fields
> currently.

Internally nr_ or num_ are going to be fine. For xl whether either of
those would be, or maybe altp2ms= (along the lines of e.g. vcpus=), or
altp2m_count, or yet something else I simply don't know. That'll be
the maintainers there to help with.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:23:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:23:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715540.1117239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Pqw-0005A8-LE; Thu, 02 May 2024 06:23:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715540.1117239; Thu, 02 May 2024 06:23:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Pqw-0005A1-Hi; Thu, 02 May 2024 06:23:34 +0000
Received: by outflank-mailman (input) for mailman id 715540;
 Thu, 02 May 2024 06:23:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=A7KU=MF=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s2Pqv-00059T-0d
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:23:33 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on2060a.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::60a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7ddec23a-084c-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:23:29 +0200 (CEST)
Received: from AS4P195CA0016.EURP195.PROD.OUTLOOK.COM (2603:10a6:20b:5d6::18)
 by AM8PR08MB5812.eurprd08.prod.outlook.com (2603:10a6:20b:1d5::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29; Thu, 2 May
 2024 06:23:26 +0000
Received: from AM4PEPF00027A6B.eurprd04.prod.outlook.com
 (2603:10a6:20b:5d6:cafe::1c) by AS4P195CA0016.outlook.office365.com
 (2603:10a6:20b:5d6::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28 via Frontend
 Transport; Thu, 2 May 2024 06:23:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00027A6B.mail.protection.outlook.com (10.167.16.89) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Thu, 2 May 2024 06:23:26 +0000
Received: ("Tessian outbound 082664cc04c1:v315");
 Thu, 02 May 2024 06:23:25 +0000
Received: from 7690f41ef32e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8029D8E6-2EB9-4B9C-A2EF-0F1884B3694C.1; 
 Thu, 02 May 2024 06:23:20 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7690f41ef32e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 May 2024 06:23:20 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by GVXPR08MB10863.eurprd08.prod.outlook.com (2603:10a6:150:154::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.35; Thu, 2 May
 2024 06:23:17 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7519.031; Thu, 2 May 2024
 06:23:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ddec23a-084c-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=NqDlpHHP8KsA8rXOJGVnC6RA8Ly0X+SjEZDfaRT+XMA4SpVKatRiDq2U7QmkhoIRbbU2a6PvmZt6q+JKE/3HDW8T10a+z0HnQIBaKcIlZZjWtLQsIHTYTjg+iUN7a+sRef7OFOAYSGQkj+U2hWC54vlnxBH5jzcCn/ftPKbe7aDDWqfwuh8bF6fHxVD9W4tzl/wo40ztHgNp9DZO0+Drp4c6kDC4RkWjVSBM5xA1MAX4FPI0rmKIOBlASjjHoee0V088XTBwP5ysv7zSgHws0Nwqcw1Y5zuuWRZeoGs6u084A24bHm5BwMsFuONN25bjfg4PP2gB10cbUlld9NaY8g==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SkpJDoPJAcjhomrzoWJJ+nLmvfL+EmAyZtxfxalFIaw=;
 b=Jy0QAdhrQ8EgpwynJvy9FcUNPvR9ipMkmA9xuyfKs6KwyFTE6AUADNEIQ25IU3jgTGYiaM/nZG8JXxhvTixdUHD2nb6Vb2mdAuV2D7H/bgvpfuC4B05s+cqSOoUvOMbOerBvZ3dimJIF3F0JMn9CnXM0b4TS9hbdoAGK8mmEjyBmKSM7utiYW37lBo3CM4BX5GV4vLTJi5ri0M4HOuVMA6HVmzmUEeM70nEPzyD99nBalwtAuyXOeLNZ0B9BD5DfidVEqmNkrSgz0JCzpaEq0tRjHp7pgmiDONJyAQUcC9Lim+gxYFbmpoySaMB27ct+sahyG+qFpWjOpPwnlmqpeA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SkpJDoPJAcjhomrzoWJJ+nLmvfL+EmAyZtxfxalFIaw=;
 b=GMxSCRMCXxdd7RsLfk1yuAVgRpzwEA8e9oi0DRUEjEbJ2mjpDaCLo3VAhHXRFf8yL5i4HNVgrMgCLaRNTq/HqKSBVht/QXBaMKzSDGMvUmBZJAiu6caThzn43MWV7AYZSr6MZ16gRHI66v8muW6wfiSH02JUlRQCHKP2dFo/P/M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 4df3f04030d602a4
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JwFish/pbUS3o95l4Cv+rMmxrK61WBNv284CfykR4HRj50GKqx06NSq9719kzmCTBKbG/TPIYcUYOTNNmbuqG0d4nGPdQqwq+dOY1dffb9ePaLXdxFYj10mlbgBPdmkwXYzfXFbGkElsJbLovj9ZyFZVlx377vKkBEYdcxyYW/CLqzmLGVwkxZvtJUoS3zlDd/nJeNQoYHtjEu0/UIuAm1xcf2rl2LGrYNaSviav/wml9RoPn2b5o/Em0AAtbngkbA6IAusRGbRWSyRssz0xNMDCJifgY4knUUhIgylVvmxWnLbwLWgCOoQ7SkZvgPaFsv/8GUHXp71gm/dmpJTVTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SkpJDoPJAcjhomrzoWJJ+nLmvfL+EmAyZtxfxalFIaw=;
 b=iXG0AFhmsVHdYMvHtD5HRUHpgYpVxeLeuYBEtKd4rfr3WVrxSxmBDE3D/PoL+GjY+B1GEAqWi6JGgoHlovpQ5IEPh9H6xbCSbQdK20qOxax/cDjJzfvdoh7OSwM2SGgRYIjH/THvTQ/H/8Um61R1k06rHec+W8R/224R7mGgj5FaZb5gr4Q41DuF/akjdxHl5YsQmjjod0fxJAleBzaE+9fTvKnLH9umNHCRGXyoh4hYXLzwa/cYiAbT76lpV8GzQP1X/7so+dTvLi9v2lUDNDWj4wYclidkih/DhLc1GpslSIZAOZAxhmGqfybzCZHx3Y5IXBg/F4Uixf/sYDcz3A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SkpJDoPJAcjhomrzoWJJ+nLmvfL+EmAyZtxfxalFIaw=;
 b=GMxSCRMCXxdd7RsLfk1yuAVgRpzwEA8e9oi0DRUEjEbJ2mjpDaCLo3VAhHXRFf8yL5i4HNVgrMgCLaRNTq/HqKSBVht/QXBaMKzSDGMvUmBZJAiu6caThzn43MWV7AYZSr6MZ16gRHI66v8muW6wfiSH02JUlRQCHKP2dFo/P/M=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Thread-Topic: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Thread-Index: AQHamu7vn3o+gufgI0eDMZNP6vgeorGAsTwAgAFBl4CAAYX0AIAAA74A
Date: Thu, 2 May 2024 06:23:17 +0000
Message-ID: <C6F3416D-785A-4C1F-96B9-1C7A53B927EA@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-2-luca.fancellu@arm.com>
 <cd2352aa-e94b-43ef-b3fc-359cc1f43fb0@suse.com>
 <2AB02E85-2746-4DD6-A821-B858E52A6954@arm.com>
 <16815d66-ffe7-45e9-8311-5d4ed8754d2f@suse.com>
In-Reply-To: <16815d66-ffe7-45e9-8311-5d4ed8754d2f@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|GVXPR08MB10863:EE_|AM4PEPF00027A6B:EE_|AM8PR08MB5812:EE_
X-MS-Office365-Filtering-Correlation-Id: d12aeefa-02c9-48cf-0c4a-08dc6a706044
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|376005|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?UmRxWjNFOTlzWG4wYkx5Y0lTNTVKWExlOHhlTUFmMTZhV05uWEtubGRpQzFY?=
 =?utf-8?B?S2MvdndLVEUyc0k0MmF3aVJYV3FlTlVmaW9VMS9GZW0rcUdaZDBDR1h2ak9Q?=
 =?utf-8?B?V05pRDlEQjFwV3FZdG0rQTRUS0lRN3RzMGg1ckJwYjNGYkllZStuMEdmSUlE?=
 =?utf-8?B?U09IaWFTMUE5N3o5UE1WZHkrRUwvRXF6YzFNTk50ZVlYdjVSaHVjbGRHd3Bj?=
 =?utf-8?B?anc5cTRuL0o3OERjY0JWUGJPQ2NEMy9vV2gvQjNaTy9HK2lmNjBtTE1vRDNr?=
 =?utf-8?B?TVM4TVgwQTh1VlRVS3RpZkh6UXlTQ2NtVXpheHVHYmdCY1BQNGlGRG90dlpL?=
 =?utf-8?B?YXlraFR4TVdvdjkvVjVpbi90bkdBZlJBV0hBYzdEYVZpRUFBSU5ybkQyT1F3?=
 =?utf-8?B?U2xKRi80RGRQQ095VDI0OFhEMXFzZWpBaTljZkhpdkdhdDdlYjRJK0swdXNo?=
 =?utf-8?B?MzBmWW9LajFtTjNWaUwreXllSTlzbVphMVdWWERkZmN0Qlo0R3hOQ09qbG1z?=
 =?utf-8?B?S2E3ZDRUZWhQUXVQUER6NDJIRklPYktrQVFjL0hQWlBkWEdTWFN5akpxSDNF?=
 =?utf-8?B?UjRLalNUR01YR2NETWpyRzhhTXV1Wi8vWU5ZU25VTlBLeklZRUttZi9WM2Qz?=
 =?utf-8?B?c3NwYnMxcVpoK1hEN25uZlQyRnhRUURKdzc3YTdzZ2x0d3lXZis1bEJ0blBh?=
 =?utf-8?B?a0VNdjlYa1NRTTREOXd1UGxJN0hKUTBDa0d6ZHJsT1NxNWxsSUhqeUJBcTMr?=
 =?utf-8?B?cWY5ZGhIMG5SRmljdUFieE9jQ0M0VUQwTkhueGdZbk8vZGlyWjI4b2VjRG1R?=
 =?utf-8?B?TEtkdEVMSC9FbkhVdC96OXNBcUZQNlJUYzlZZXFnclYwYUtWZDNhQlZidmor?=
 =?utf-8?B?aDkvUzNXcGduZndUNVZod3c4Nk82UTVNdXhXUTZyWGFLaGRnM1g5UFhMVTZS?=
 =?utf-8?B?dSsxcVdsSVlVYkxMM1lFTUI2SGNKTGpKbVFEM09DQ0IyVkJyaHM0bllpU3cx?=
 =?utf-8?B?Zmw1YTNtMEMrZEt5N2pyakRGYUMrdXFTTnRCYU9kcnFBZVJBYlhwa2lJd2pD?=
 =?utf-8?B?amJKdytRRHArTGJLME5SZkFyNUk5ajJzQytsa29UZFpaQVFseldJOVpmS2tB?=
 =?utf-8?B?L3QwUE9rbkhmVWo4MUg0MWNoYVEyQ3NkL1ZyeUNlOGxVR3NQYVYvemU4QjJi?=
 =?utf-8?B?M3dZS3hDZ29HZzJtY1hFek0ydWxWQ0dDa01wZG1jd0d2eGlyWUNSZ1FvYXd2?=
 =?utf-8?B?QjY2c3ZlZTlqT2FwYUhYdW0yaEM2dllzY1hBckN5dStVdFdlQTNEazJBb1NI?=
 =?utf-8?B?cjV2UlVvTVp1Z2lZRUpyY21sYUkwRXFQZkpJcHJTek8rYjdwajZMRG9QMXVD?=
 =?utf-8?B?cW1MOWxPWkt2VmNvOFlkbUF1V0ZCanliVGhXK2l3dFpoaVZmcjZnYUNWR0tI?=
 =?utf-8?B?MEo2Rys1SlpKVjl2SjFGVWdLTlZ0Wnc0bEovSDJDWXNjKzNhc3hLLzE0cUlY?=
 =?utf-8?B?c2JrdE1HcUIvZjZtalRMZWdnQ2FoMnBlUC9IeThHUVcvWmkwYlNLMkg5L3Ir?=
 =?utf-8?B?N3BMd0owWDdxc3dVY1o1WEpMUm1RNzM2bjNCSmNYS09Kb3BFVFRPOWNwQVF1?=
 =?utf-8?B?UTg4STg1QzBSOERBME90bGpISWUrbnUrQUU4SmFYZEFNbXFJMGxHc2YxT0J6?=
 =?utf-8?B?aUFLekJzcE1SNWdFeVNCNEFuYXRScmkxY0lGeHZVaVJvamNSMWZqdGRRPT0=?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <1B908C8A5061894E8241F4B9308EC9B9@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB10863
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00027A6B.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9aaef03a-3e55-4daa-0178-08dc6a705abd
X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?M3JFZ1U4V2dOdHdQN0ZhcEVTWDdLMTRHdXJmUi82bEZ5SUNSYlpFbWdydTN3?=
 =?utf-8?B?K045YklxRGFOV1QzMDlFNEdUU1IvcXlpZHNiRHdUN3J1RmhQbkd2OGpNaFMv?=
 =?utf-8?B?cHhPY2hJektCQzkzYlY4QWZoWUg2b2VKQVhMWDc2VE4xcTdSK2Q3YzViTTdI?=
 =?utf-8?B?MTRtKzdyNGRXSnBINFhYeW1xNWJqbXdqY1EzK25UY3FGYkN1RVBSWFJqT1Nq?=
 =?utf-8?B?bVZaaWp6TW9Vc1ZNUDk2OEhMV3hheXNSM2ZOcjB6WlkrV2NHTkQ0OXdjeDY3?=
 =?utf-8?B?WU5DMDFQdFMzZnUrMThVRDh0c0s0WE8yeHNiWEtLbW9jWTRNMTk5b2dlcUkr?=
 =?utf-8?B?ZlZYc0pxOHZ0ZTZBQWVmdDY3U2YwL2ppaUJENFIyZ3RGb2dOSE9ZZG5LUXZu?=
 =?utf-8?B?NmxwWVlXYTY1U1RBNnV0dS9IQVRCNkVDWkI4QUt1OVN4dHkzWmhQU0RHcHZK?=
 =?utf-8?B?SVY2ZXBVTUp5cEI1ODVWbFNHNmRVNmdSWGV4L0x5UGVib2JodDFjU2hDNjM1?=
 =?utf-8?B?ZTF0QklhdmJKNkoreTBhSFZrUVRlNXRqS2Y0d09XV2czR0tQZXdMVWxybnZ0?=
 =?utf-8?B?SWdzSFJqdytaa1VVVE1IY2JrUzF2Y3J4d2J3VUpzdGU0aEF4V0pXVWhXYVRP?=
 =?utf-8?B?MGhzMWJiRTlCYUhxZ1BWMk5CT2VZdFhBa0lnekxkM1phY3l4b3FFSVdsL0JJ?=
 =?utf-8?B?dFQvSGNXM3I5NjRwczhYSXNoZmhpMjZ4c1NzS3o3Ym9COTdHMXhYYms1dmxq?=
 =?utf-8?B?NG1LZW1Bd3lTeXFUaU0wejl3a0cyUVY1MXg0aVlhK3RqZGE5bHlWcXE0Um1w?=
 =?utf-8?B?eS9hKzRRcHJFSnk2RXhMRzVFOXl4VlBZak01SlFxZjVlZkgvZlBNY05QWVZX?=
 =?utf-8?B?blUyR2tVY2NaUG5RdllWbG1wbkhlQ2xnSnk0RE1idDFqbGJIMmRTNnkvbDAx?=
 =?utf-8?B?QlNuOFExRG1WWXVlUVo2RFR6OUZHMTdhblFxci9IVUVJNXo1c1FiQXovS0oz?=
 =?utf-8?B?OXViZ001b0hhaWdFbEhPYVlES09rY1pmd0FxQ0lPWEI0ekxBamE3MU84MjB0?=
 =?utf-8?B?RU9aWjcvVDB1NnlGWHJSWGV6NHZuMHRxUUdGVFlQL0haSVBXYkVUWDFUVTJT?=
 =?utf-8?B?S2paOGRPTjJzM09oWjVvQURQcm9nVlNNdkZVWUtlVHVmTWowWVNLdU8yb3h0?=
 =?utf-8?B?ZTBabzNsazFuNERCMmV3dXp5eEVXYXYxR0kzWXlUSjJUZnl1Um9FblNUcWdL?=
 =?utf-8?B?NFNsRlYzL3R4QzhsakdDVmVHNGdWMUsxZkkwUTV6V09VQTdUVFlla3lRbHlt?=
 =?utf-8?B?bTlBRzBORjJid2VuSzVyem5qdmoyTC9NVmZFUTdLRzhXQ3N3ZEJ1ZitDOWNB?=
 =?utf-8?B?Q0JXMVlxSlh2V2M2bzJpbGVOYU5uSFBGbVlJNFZKWmwrRVhNb0FIUWNRNkll?=
 =?utf-8?B?dndIQ3dGQWRKM2JUWFRQeXpyUHZlYTY4WGRRak5ablM0Ny9OV2tnRHhRdFIz?=
 =?utf-8?B?dUszNlE3dEZ6ZFZXQVcxWkxBQzNMVFk3dXBQUVMxTll5ODIvZHg5TDdzNjRL?=
 =?utf-8?B?SndacXd3SDFsbWhYaGJYUThrZms5TWtOZHhGYlF6OVZGcU1yZFF0Q2oxRWkx?=
 =?utf-8?B?THQzalVQcUVTOXZXeE9oT1BPNmxZYXFUbkNhc012aWQ4R1RFVHdKYTh5ZWJj?=
 =?utf-8?B?NTlSOElzTTRUU2ZvbzB2dlQySm52M0J4VUpEWDdibXZMNDQ0d3FwVjQwUXNV?=
 =?utf-8?Q?z/HBBuaVRtlsvRfbxVBe+BxwlbqHtdYaMpDRz5i?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 06:23:26.3096
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d12aeefa-02c9-48cf-0c4a-08dc6a706044
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00027A6B.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB5812

DQoNCj4gT24gMiBNYXkgMjAyNCwgYXQgMDc6MDksIEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNl
LmNvbT4gd3JvdGU6DQo+IA0KPiBPbiAwMS4wNS4yMDI0IDA4OjU0LCBMdWNhIEZhbmNlbGx1IHdy
b3RlOg0KPj4+IE9uIDMwIEFwciAyMDI0LCBhdCAxMjo0MywgSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPiB3cm90ZToNCj4+PiBPbiAzMC4wNC4yMDI0IDEzOjA5LCBMdWNhIEZhbmNlbGx1
IHdyb3RlOg0KPj4+PiAtLS0gYS94ZW4vaW5jbHVkZS94ZW4va2VybmVsLmgNCj4+Pj4gKysrIGIv
eGVuL2luY2x1ZGUveGVuL2tlcm5lbC5oDQo+Pj4+IEBAIC01NCw2ICs1NCwyNyBAQA0KPj4+PiAg
ICAgICAgdHlwZW9mX2ZpZWxkKHR5cGUsIG1lbWJlcikgKl9fbXB0ciA9IChwdHIpOyAgICAgICAg
ICAgICBcDQo+Pj4+ICAgICAgICAodHlwZSAqKSggKGNoYXIgKilfX21wdHIgLSBvZmZzZXRvZih0
eXBlLG1lbWJlcikgKTt9KQ0KPj4+PiANCj4+Pj4gKy8qKg0KPj4+PiArICogX19zdHJ1Y3RfZ3Jv
dXAoKSAtIENyZWF0ZSBhIG1pcnJvcmVkIG5hbWVkIGFuZCBhbm9ueW9tb3VzIHN0cnVjdA0KPj4+
PiArICoNCj4+Pj4gKyAqIEBUQUc6IFRoZSB0YWcgbmFtZSBmb3IgdGhlIG5hbWVkIHN1Yi1zdHJ1
Y3QgKHVzdWFsbHkgZW1wdHkpDQo+Pj4+ICsgKiBATkFNRTogVGhlIGlkZW50aWZpZXIgbmFtZSBv
ZiB0aGUgbWlycm9yZWQgc3ViLXN0cnVjdA0KPj4+PiArICogQEFUVFJTOiBBbnkgc3RydWN0IGF0
dHJpYnV0ZXMgKHVzdWFsbHkgZW1wdHkpDQo+Pj4+ICsgKiBATUVNQkVSUzogVGhlIG1lbWJlciBk
ZWNsYXJhdGlvbnMgZm9yIHRoZSBtaXJyb3JlZCBzdHJ1Y3RzDQo+Pj4+ICsgKg0KPj4+PiArICog
VXNlZCB0byBjcmVhdGUgYW4gYW5vbnltb3VzIHVuaW9uIG9mIHR3byBzdHJ1Y3RzIHdpdGggaWRl
bnRpY2FsIGxheW91dA0KPj4+PiArICogYW5kIHNpemU6IG9uZSBhbm9ueW1vdXMgYW5kIG9uZSBu
YW1lZC4gVGhlIGZvcm1lcidzIG1lbWJlcnMgY2FuIGJlIHVzZWQNCj4+Pj4gKyAqIG5vcm1hbGx5
IHdpdGhvdXQgc3ViLXN0cnVjdCBuYW1pbmcsIGFuZCB0aGUgbGF0dGVyIGNhbiBiZSB1c2VkIHRv
DQo+Pj4+ICsgKiByZWFzb24gYWJvdXQgdGhlIHN0YXJ0LCBlbmQsIGFuZCBzaXplIG9mIHRoZSBn
cm91cCBvZiBzdHJ1Y3QgbWVtYmVycy4NCj4+Pj4gKyAqIFRoZSBuYW1lZCBzdHJ1Y3QgY2FuIGFs
c28gYmUgZXhwbGljaXRseSB0YWdnZWQgZm9yIGxheWVyIHJldXNlLCBhcyB3ZWxsDQo+Pj4+ICsg
KiBhcyBib3RoIGhhdmluZyBzdHJ1Y3QgYXR0cmlidXRlcyBhcHBlbmRlZC4NCj4+Pj4gKyAqLw0K
Pj4+PiArI2RlZmluZSBfX3N0cnVjdF9ncm91cChUQUcsIE5BTUUsIEFUVFJTLCBNRU1CRVJTLi4u
KSBcDQo+Pj4+ICsgICAgdW5pb24geyBcDQo+Pj4+ICsgICAgICAgIHN0cnVjdCB7IE1FTUJFUlMg
fSBBVFRSUzsgXA0KPj4+PiArICAgICAgICBzdHJ1Y3QgVEFHIHsgTUVNQkVSUyB9IEFUVFJTIE5B
TUU7IFwNCj4+Pj4gKyAgICB9IEFUVFJTDQo+Pj4gDQo+Pj4gQmVzaWRlcyBteSBoZXNpdGFuY2Ug
dG93YXJkcyBoYXZpbmcgdGhpcyBjb25zdHJ1Y3QsIGNhbiB5b3UgZXhwbGFpbiB3aHkNCj4+PiBB
VFRSIG5lZWRzIHVzaW5nIDMgdGltZXMsIGkuZS4gYWxzbyBvbiB0aGUgd3JhcHBpbmcgdW5pb24/
DQo+PiANCj4+IFRoZSBvcmlnaW5hbCBjb21taXQgZGlkbuKAmXQgaGF2ZSB0aGUgdGhpcmQgQVRU
UlMsIGJ1dCBhZnRlcndhcmRzIGl0IHdhcyBpbnRyb2R1Y2VkIGR1ZSB0bw0KPj4gdGhpczoNCj4+
IA0KPj4gaHR0cHM6Ly9wYXRjaHdvcmsua2VybmVsLm9yZy9wcm9qZWN0L2xpbnV4LXdpcmVsZXNz
L3BhdGNoLzIwMjMxMTIwMTEwNjA3Ljk4OTU2LTEtZG1hbnRpcG92QHlhbmRleC5ydS8jMjU2MTAw
NDUNCj4gDQo+IEhtbS4gU2luY2UgZ2VuZXJhbGx5IHBhY2tpbmcgcHJvcGFnYXRlcyB0byBjb250
YWluaW5nIGNvbXBvdW5kIHR5cGVzLCBJJ2QNCj4gc2F5IHRoaXMgY2Fubm90IGdvIHdpdGhvdXQg
YSBjb2RlIGNvbW1lbnQsIG9yIGF0IHRoZSB2ZXJ5IGxlYXN0IGEgbWVudGlvbg0KPiBpbiB0aGUg
ZGVzY3JpcHRpb24uIEJ1dDogRG8gd2UgdXNlIHRoaXMgb2xkIEFCSSBpbiBYZW4gYXQgYWxsPyBJ
T1cgY2FuIHdlDQo+IGdldCBhd2F5IHdpdGhvdXQgdGhpcyAzcmQgaW5zdGFuY2U/DQoNClllcywg
SSB0aGluayBpdCB3b27igJl0IGJlIGEgcHJvYmxlbSBmb3IgWGVuLCBpcyBpdCBzb21ldGhpbmcg
dGhhdCBjYW4gYmUgZG9uZSBvbiBjb21taXQ/DQpBbnl3YXkgaW4gY2FzZSBvZiBjb21tZW50cyBv
biB0aGUgc2Vjb25kIHBhdGNoLCBJ4oCZbGwgcHVzaCB0aGUgY2hhbmdlIGFsc28gZm9yIHRoaXMg
b25lLg0KDQpDaGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:32:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:32:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715546.1117249 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2PzY-000769-Ho; Thu, 02 May 2024 06:32:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715546.1117249; Thu, 02 May 2024 06:32:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2PzY-000761-EY; Thu, 02 May 2024 06:32:28 +0000
Received: by outflank-mailman (input) for mailman id 715546;
 Thu, 02 May 2024 06:32:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2PzW-00075p-8D
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:32:26 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bd0aea44-084d-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 08:32:25 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-57232e47a81so2446643a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:32:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd0aea44-084d-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714631544; x=1715236344; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8EYclx16HNFs8YntLtI93PesjTHQNuEjlTaisfuEtSc=;
        b=J9B6rvZFd8X8yhJy0C34mD5BzX1niwsDKld2wIayulNt65xcQPKwlOsV1Uqm0rzJNc
         eLwhv/24IZfqW40Z5af/pCThcSDf1lXNyQ8sG59ppmfEMmuZ8EZ/jNcvipkv50sc0edV
         Ql8iLzf9/NSO80fKlxXkcg257aJwwJv1jR+REnAT8VfNY47LA5GuRgvRlEA4S+VvxmbA
         IumT/QCnr+NSbEkDz+7rf6aEp3w6qE7QmvcJu+oGQrXV/f179uuXmuWQzg0QfYIvrJCi
         hsFLvTc6wpwGA6TSb+3m53FsXuNXskIZItXaWtK8F1vR+QxJO/8Q3tpHfuGvS/vAiDWc
         b0DA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714631544; x=1715236344;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8EYclx16HNFs8YntLtI93PesjTHQNuEjlTaisfuEtSc=;
        b=YwHYwJmPYrOKSPmfVKEFVQkbfzEN4O/wRVxMBEcu4AWdyvPwF0onbrAjDKvWa+WXen
         RCwQeZmYnd4g91kyyn8ntFNEfCibJOmCETu2Pq7JdhrU0n5Z9pgFDEk8Pq8IQZJO/IRW
         4xfvgk7fHBO9f9rf9GpkkCSmSQCEITZsMvhVFnhK8sLacGMJTu1Wc/9vuFyLFXdxkS0B
         C6l4AAXpM/IfhKuTGTvdzXQW+9EKYGBbcpxjECu4m2FXbKJbJ4rLMTu8YmEo6Jxippv8
         oxKvyhTtT543TLayI2MUBSh78f8qOWP3gWTEgx6o3cgzjMYT4VJkfV8vaxTpbzPvh8hZ
         tZ9w==
X-Forwarded-Encrypted: i=1; AJvYcCVtWHYOwbWAWI1ieDsVhK5/6pcVr7zspg2yaq3LYFX2ZR3UAhFB6zHXJllallAAXzA/CZEosAzH0pFhq9ZkRXb2Usqw8fMTJamkQI94+mY=
X-Gm-Message-State: AOJu0YxI3EWZpqFDsh65JUMYfgcgewCSAXsMpJ0Xr0UgSusLS/griDwj
	BC7a5HcCrTsp0WHMV5ZWvZC84hE9pMppWn/5UWMf5SNArX9d70RUBsDSS/H7kXc51ZvAJNcr7BV
	OZh7PJIzjoNkua8I9+KVG74WMuoI=
X-Google-Smtp-Source: AGHT+IHa+J7ytWh1ojM1i3gs+HSyTqEKairXLG5kBCLQOfyxqSWNggYlnQsg4jVmTkw5jE+i7zSPz/srdxbFnfvcN9o=
X-Received: by 2002:a50:c054:0:b0:56e:7281:55eb with SMTP id
 u20-20020a50c054000000b0056e728155ebmr1387706edd.9.1714631544216; Wed, 01 May
 2024 23:32:24 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-8-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011345140.497719@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405011345140.497719@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 08:32:11 +0200
Message-ID: <CAJy5ezq2PBXMXGMFORfuDVeC_t4S=9AF6b_pur9kD26fhBPhwA@mail.gmail.com>
Subject: Re: [PATCH v4 07/17] xen: mapcache: Refactor xen_replace_cache_entry_unlocked
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 1, 2024 at 10:46=E2=80=AFPM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Add MapCache argument to xen_replace_cache_entry_unlocked in
> > preparation for supporting multiple map caches.
> >
> > No functional change.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > ---
> >  hw/xen/xen-mapcache.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index d2deff70c8..6e758eff94 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -556,7 +556,8 @@ void xen_invalidate_map_cache(void)
> >      mapcache_unlock(mapcache);
> >  }
> >
> > -static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
> > +static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> > +                                                 hwaddr old_phys_addr,
> >                                                   hwaddr new_phys_addr,
> >                                                   hwaddr size)
> >  {
> > @@ -578,7 +579,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hw=
addr old_phys_addr,
> >          cache_size +=3D MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCK=
ET_SIZE);
> >      }
> >
> > -    entry =3D &mapcache->entry[address_index % mapcache->nr_buckets];
> > +    entry =3D &mc->entry[address_index % mc->nr_buckets];
> >      while (entry && !(entry->paddr_index =3D=3D address_index &&
> >                        entry->size =3D=3D cache_size)) {
> >          entry =3D entry->next;
>
> There is still a global mapcache pointer in use in this function:
>
>   xen_remap_bucket(mapcache, entry, entry->vaddr_base,
>


Thanks! I had accidentally put the change to use mc in future patches.
Will fix in v5.

Cheers,
Edgar


>
> > @@ -614,7 +615,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_ad=
dr,
> >      uint8_t *p;
> >
> >      mapcache_lock(mapcache);
> > -    p =3D xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_add=
r, size);
> > +    p =3D xen_replace_cache_entry_unlocked(mapcache, old_phys_addr,
> > +                                         new_phys_addr, size);
> >      mapcache_unlock(mapcache);
> >      return p;
> >  }
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:33:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:33:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715549.1117259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q0y-00086y-RT; Thu, 02 May 2024 06:33:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715549.1117259; Thu, 02 May 2024 06:33:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q0y-00086r-Oa; Thu, 02 May 2024 06:33:56 +0000
Received: by outflank-mailman (input) for mailman id 715549;
 Thu, 02 May 2024 06:33:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=A7KU=MF=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s2Q0x-00085D-K9
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:33:55 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01on2053.outbound.protection.outlook.com [40.107.13.53])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f2029258-084d-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 08:33:54 +0200 (CEST)
Received: from AS9PR05CA0311.eurprd05.prod.outlook.com (2603:10a6:20b:491::28)
 by DB9PR08MB9801.eurprd08.prod.outlook.com (2603:10a6:10:462::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28; Thu, 2 May
 2024 06:33:23 +0000
Received: from AMS0EPF000001B7.eurprd05.prod.outlook.com
 (2603:10a6:20b:491:cafe::4b) by AS9PR05CA0311.outlook.office365.com
 (2603:10a6:20b:491::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29 via Frontend
 Transport; Thu, 2 May 2024 06:33:23 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF000001B7.mail.protection.outlook.com (10.167.16.171) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Thu, 2 May 2024 06:33:23 +0000
Received: ("Tessian outbound af213ececc3d:v315");
 Thu, 02 May 2024 06:33:22 +0000
Received: from 9d42171dc654.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 2D2D2663-D161-4D80-934D-3EA3F3091C3E.1; 
 Thu, 02 May 2024 06:33:16 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9d42171dc654.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 May 2024 06:33:16 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB4PR08MB8102.eurprd08.prod.outlook.com (2603:10a6:10:386::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.36; Thu, 2 May
 2024 06:33:13 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7519.031; Thu, 2 May 2024
 06:33:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2029258-084d-11ef-909b-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=D4lbCH20xwLmVm9/jrUKHO4M8dNC2VvXWC7VQkug9urURtDmYVIxbXb28PEkHdQKddKBJDNm4GjmTcPfjThJTJF+XFM1XQnZzKuvN3LCq2W6fl4ArKVxBL95i4UwS3FVQAUpegyguBDt7aNfMlMxw+AhBw/0BDexBwyCuNQ9FlWXGZWWW1T6GrN12xzAkuR7kCtKTHnPqAc4fAzQUSnvFlw4wqhlbf9OrNQE3C6HBjFWJQfa57Pghhe4PDpzYw7S2vDt/rk9zxOuqYhV5ADu8JVji4/eCnQ+2FQpol0DsoDwmwyleRJQgPGdXYCH2iI/hYgPxOcGJQZIViawqzWJDg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=W6FwU0Q3/gqWAiXf1Xgd8ehO2TYoveL/szN9rczxeDY=;
 b=M0t9jdZ/S5x/xivj1UXJlaob9fbMc0vP5ND7iGyBKyW3Z3yhgL35lqUYO2YvJd5d0vppvYRHfd80eTVeSFByIhsD6yDwCpEUDdmebQ31vzxGUB0NzAISfb9jJkmrVrVy3JgdHBjD7dzpqyWZ6fFkH2BQcvKKq8qaiLofjSGc6g4ewnWTOLWW6ufvvJQdemm/Yj8DGq8zFKFGl51QlFdrwWlXp7RRn1XoS/w4DmcB0d/1sD8vfIga/OtVTwbr2sF6goI3NWObgrNdJsxTlCU+SrJ8bZkE4tOjM8i9z/4h5CCEz2TBjRYV8NRvtuzIUxmopIgOBwu0kW/4b6YLThJdcg==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=W6FwU0Q3/gqWAiXf1Xgd8ehO2TYoveL/szN9rczxeDY=;
 b=pJ+gjDPCZR+4pBjM6/FDm1f+4lP5zp/LDeIFFLBbxctLCl8RnR3QqbuMZ8Qn8N30DjalbdJL76VNFe27fJNlwYrsPelhLuzB0IEAPklV3TtG/9YaJscjPV/0dWVLNqnhlTLD67ByiixmXILOQNpbvxAQeSCc6bQPU+w6aO7sTZI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: ddc738dbce6feb10
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nQc2oVk9e9mQncIHU6jpw+vMkqqdj62r99B7NSA+37vMRZ+Cz6zdJRH8qspSwScZRKWyeZrE088aSZwlnBkoZV+zxljsErPbXPyn6uKzzN4wCrO1qpeIZuuyFKiwxqWw8B+8vVlAqODtVJmAYeHJ7HjusiXjJfwG5a7lBhRJM4HGNL03RmiD4gXVK0eKMSna/08uL5VHG7YNcm4ckuOWUmGUGDPPAsqgtHfMNMG6ffVcVRPiwLm3FU8aTWqcloOat1DJEgwszRGewy3q4iFj7PKxl2JD4TNf2PVQ5K/YsEM8cc4MsJaVnpVBBumOUvWCSFUE/6E7WV0C0lFwoInW1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=W6FwU0Q3/gqWAiXf1Xgd8ehO2TYoveL/szN9rczxeDY=;
 b=KKvoOmKZNbU7aAA9LLYgh3nm1T+JbS7qrCvyOAPFnSj53KOZU2sRI8iekoftndGSsqZztWmCUKrTINjPyGCQ0xi/bsUVOCb33EGCr0fj7WDOtG8IdvwNb7NJp8DdlZ4CaVA1hmgcz2Oxjgv8207245KBbCOCjAgupMytSrmm80wU86VSs+SCQVAU79mWKL0FqlVg4nBcozts2x+5axb/b7AmX+qd8FBcoXsCfWUXKYd1pKzsAf1FmMFXidBa35WbIxQRIz6HgRciUvNZktksckEgRTM9oGFGjzGb1DIv7nRpcqZiEtGDJ9KEA52Dy7h7zZ0z2216cMhXUBDs6wyrSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=W6FwU0Q3/gqWAiXf1Xgd8ehO2TYoveL/szN9rczxeDY=;
 b=pJ+gjDPCZR+4pBjM6/FDm1f+4lP5zp/LDeIFFLBbxctLCl8RnR3QqbuMZ8Qn8N30DjalbdJL76VNFe27fJNlwYrsPelhLuzB0IEAPklV3TtG/9YaJscjPV/0dWVLNqnhlTLD67ByiixmXILOQNpbvxAQeSCc6bQPU+w6aO7sTZI=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index: AQHamu72Eq1RpEGs60mNp9DIfY8mn7GAr6+AgAFEGQCAAYYzgIAABVAA
Date: Thu, 2 May 2024 06:33:12 +0000
Message-ID: <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
In-Reply-To: <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB4PR08MB8102:EE_|AMS0EPF000001B7:EE_|DB9PR08MB9801:EE_
X-MS-Office365-Filtering-Correlation-Id: 5ba21527-031e-4e5f-3732-08dc6a71c3f8
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|366007|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?RGEwSW53MjFkRS8rOXRhWU0weWQ4dUtqYVZZMzRCdmJuWTZoeEdWVXlBbzZk?=
 =?utf-8?B?NG1hTmhSNHJEb3lTNGRvQWxnTUFXWm03ejU1YlR6eDJ1MUZhSDltRFFEZm11?=
 =?utf-8?B?djQvVEhnOHRxRVRhY29wek1rYjVyNUIvOUE0a2ZXZWVJd3hjSnU5TlpRUnFx?=
 =?utf-8?B?SjZUa1ZTWlZPd01jNnNGRlU4Zmw1UXJvbHd4VTJtU09KQ3VSUjJwNVV0ejlV?=
 =?utf-8?B?Tmw5RTFPMkJ0TlhYakQ2Y0I4ckVNWXgvYnRkMzRPeEQxRGF0Q1FDenI0SjZB?=
 =?utf-8?B?UUx4MVJqQmk5S2FhRy9STEZiaTdVa3AxTSt6bUs4RWFqYTlJSitXMVhlejNp?=
 =?utf-8?B?cjFqWHdDb09DSHZrajdXUjBQN1lIRGJjYTlibGQ3M2RhVWlZM0Z5V1lEWlVX?=
 =?utf-8?B?UGZsS0MrVUxqbUNaRkk1b1NocVRoYUE3M091Zkh3ZG5qdEZpcWlNL29TbTJ0?=
 =?utf-8?B?cWZpREdwOEh2SitzVHVBdTFKY2JnbndKajQxYnI5WnBDWDRHbzZMemVQbkZI?=
 =?utf-8?B?ZVQxUlhCdXBNaS95TWlxWGUweDFRSFY5QnVsLytKSUxSZ1o3NURBUmczVFZH?=
 =?utf-8?B?aTJDTjd3V0VmdFJuNDQrV0w0YVZyNms3Ykh6K1pYbTVOU00rNzRsYk9LUEht?=
 =?utf-8?B?NzJaZm1vRU5UcVFwRlZ4M1N4aFFzQ2JXbEhmMGZaSzQyaEo1QXFGUjlJOUd3?=
 =?utf-8?B?WXRjU3RjNk9GQXJIenpJTU5CeDF6Vzc5akVFc0NCUDlwR2RFVWJBclhqVFVS?=
 =?utf-8?B?VmpZMVE3ZklNY2VxMVF3WGU2WE1jb05lRWM3N1g5UkVVNE9mSzJRK080NVkx?=
 =?utf-8?B?bzUyVFNHcElkTStDT0JKczdlWXV6dWpaTFFXRWlWejMzRGZPTXp1NkhoZlg4?=
 =?utf-8?B?dnkxVDBZU1ltZDBEZjRsaTRqUFBNT2dVbi9mK1A1RW1jSmZhZ09lWXU5Ujho?=
 =?utf-8?B?bldpNnFKNlRVNzRpMTVaUkxxOEFEMTlISDg1SkVUeUtBMWdKTnVoU2czb3Bk?=
 =?utf-8?B?cG1GT09laFlIK2lONDA3Y0tRSkhiY1JOeStDdHc5SlltalRUTUpzanorMHc5?=
 =?utf-8?B?UStkTVhtWlZJN0VBUU5SVTlXZjJQUTBpMmdXY3hZQXdxNEU2eldHK1lDdnJh?=
 =?utf-8?B?SW5aVFZEVGY0UWYwY01ZUnpleXBEY3ozQitTNlJmei9icTZVR0Npa0pQWitt?=
 =?utf-8?B?Zm9FK1N0UmtaTVlUT202SXV2NmpOdTV3SzhTZGZ6czM0cTdJRnJLb3JsVno2?=
 =?utf-8?B?aTFwNnRPU2cxbTZJVE5nV2IvR250dldaRVY3Rmw3SG8xQlBQWnZTd003OXdt?=
 =?utf-8?B?WDNiNERPVDVPUUxlNWlYRVJ4NmJwVHVKVGxSNWxvU2R1WjdSOVNoRkMwRDdU?=
 =?utf-8?B?SzFJUURnTU9waU9WWFA0aUJrNHlrbVhiTW9NNFM4U2lDQzJjU2Y4ZExXU1Zi?=
 =?utf-8?B?bTBzZVJNS3l2eTA0S0Vqa2dGVktjdjBVbURDRlNHclZVMWRGUFlYVHF1T0c3?=
 =?utf-8?B?T05jRlVtKzlBd1JPaWhndzZNSXowSnROOEhhT3MxT0ZSZjZ0UzU2cWJhZUxm?=
 =?utf-8?B?L1IxOVNtYW0vck5zSDBxL2tra01uN1VibSs2dHg1N1JjSVprQXhlbWRvQnJj?=
 =?utf-8?B?NDZiOTQrRDZQM2xlM1plK0VoMmJuYXV5QklwTGxndVFKK1ZwUHk0K2VuZWNZ?=
 =?utf-8?B?UU05WlZwYUxRODl0dVdFQjRSKysxOUNIRHQ2Tkw0TGJhSEpUUzJvN2MvRWM2?=
 =?utf-8?B?OHdNVm03c1lOYXFGaTNxNXdSc1BncG8zRyt6bGNhZkpuT3MyMTRiZE5VOTNJ?=
 =?utf-8?B?NThaZ3VCNGZRV3pKMG80dz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <5BCB814F3B538F46BA42E049A0E23142@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB8102
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF000001B7.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	769d5853-aafc-4409-35eb-08dc6a71bde3
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZW9MU2k4MCtSazJRYVNkNXJmWkY2UWMya1dpeW02d2c2S0ZlTG5YWHl5UkFV?=
 =?utf-8?B?ZTloUnhkVmh0L0ZZYnhCZzFkdGJ6QXNxem9WUFBZdENQenRPSCttdENGUjRh?=
 =?utf-8?B?UkxDdld1S3pHTnhLUDhKTFJnRWFZR1RaaEQ1bGVjTkVkUmJIYnpQNW4vcitl?=
 =?utf-8?B?anNzakJvTzdZT1Z1N3BJRzJJTWIrV1U5LzZ2MkI0NVN1Y2lEZS96bXp4bTNn?=
 =?utf-8?B?VVVXbmpkSDdYOGJhN2U1WTMwSHdoN0JnbFJVWWtoR2NtUnBhME0xczNkRnla?=
 =?utf-8?B?VSs1cW5SWWthcVZoamtGRGE4VElkVlJQNjJjSGhLVHpxN21pSElTcWZsS1dv?=
 =?utf-8?B?VTNVYmVJNFk0NmhneHlJZTI3VXNiVm9vazUvSm9RbE9LNmlYMDBvNFBxbkV2?=
 =?utf-8?B?WFBvbWZ2MnB1ZEJwM1k3azM5UGRBUG9udnBlNHRlVGdzTUxnL0NqR1B5S2VS?=
 =?utf-8?B?TGVUeW1uZ0dQQUViQ09DTGNSSlNaS2hUTzFzcDl5SUJ2am5LM1lUTXQwakR1?=
 =?utf-8?B?Z2Jmb3pCUysxSVI0cDczZzVkYVUzUG1hUVBzRUhDVFBSbkRCaFNqcnNyaUo2?=
 =?utf-8?B?UHlxeDlBa1JBNWo0TnZPdmdkQ25qUlhCYkQwb1RoWUpmY0FtZ0Q2Z2lYaUNO?=
 =?utf-8?B?WURSUmQ3VmpRT09wSTE5eHM4U3pNeEhuSlNYallPenN2Z0xCeStqdkJ3RUpu?=
 =?utf-8?B?aHJqdVdZOGJTV1c1SG5ra3pIckJxdDdFL1FrdDVnUzV2UkhhRVZORk5MVnRN?=
 =?utf-8?B?QngyWkpwUGc4ZE91eGZTcngyUmh4MTdILzdwR2MrWVlWK2JqUHpSdUZxZngy?=
 =?utf-8?B?Y2hsU01qamxOM2VMT3BVVFlDaGxhYTU1SkNrVXlpUEdmbGFBNFNqSWJBOUNB?=
 =?utf-8?B?SW4zSTA1SitrZHN6aTJnN0VaNVJnc1U2OEZvd1FjYjZLVFZuUzRmSHRqTFpV?=
 =?utf-8?B?Y3pBVVFmTWFxWEduVFRIV3NVS3pkREtKZDAxNG84Qmc1NTBEM3prcitoMS9J?=
 =?utf-8?B?dVNRYmJJNXNuR0ROR05ONVU2cEZqVm14UktycnhlMDNvakttWi83V2w1VGVa?=
 =?utf-8?B?NlJjQkk5WlpSdTcyTFNGVXVFNlZwK1VjdWhwNnh5OW1jTE01SkwwQWQ2TlVR?=
 =?utf-8?B?WFZ3RVF2dDk5Z0JYTGJRcmg4Rngza3lQWWhRZzBzN0Y5aUttcGtoQ2NXOHhi?=
 =?utf-8?B?L3pvRUVXcmhRbStlQi9vUURnMHYzN01IVlh1T205OGRubzhBVnZwSTVBLzJu?=
 =?utf-8?B?bHZEUmxxRk1aR0MyZDloMHNCb1RnTFNLNUVVeExJeVBiN09QcEh4OTJ2OWow?=
 =?utf-8?B?WWlOYUo5WHhBZkdjNnZCYzQrTVNHVDNJV3poRUFORUNZaTEwUVJFQVQzc1Vu?=
 =?utf-8?B?bkI2OGlObDZBWGo4VVFuVm1TVllPbTRRNUMvRGhsVXFNQk1wY0IzN09Ga1Q1?=
 =?utf-8?B?NmlacENlL0VaUitmcjZoaUFBUDJZVWZFZ2JKUlhEdU1ZQTNJa1oyZzdDWjlo?=
 =?utf-8?B?S3YzcFhkSEpGNVVLRTcwbVVKSTF1VHJCRTJEQXIyemhTZmk0TzVoZ3YxdThh?=
 =?utf-8?B?bzNBaFIwUDVlU0lGbW13dnZPYjZsR3lWQWxhN0dkb1ZXZUJyeVJ6REZVZXBn?=
 =?utf-8?B?TFM3K2Z5SWNoUDc3M3AxLzNGbzRkajRQaFliL3ovc0EwbXQyU0paVDl5Vldw?=
 =?utf-8?B?UlZ4ZXpQdTJHU2t1YS90TnFSMWZHNWxac3BMZ2V2cTlvWC80N3hvQlc0RmFt?=
 =?utf-8?B?REQzVEFKS1ZNRmxZcUhHMG9xUWtQSGJWMnlNOStueXluUVYvU1g4TXlPVVBD?=
 =?utf-8?B?TitmODI5a2dremlGdHFNVVhYbndQR1hnZmk0L096TEZyNTRlQ21oYzdkcHRR?=
 =?utf-8?Q?J7SaH670wnMmG?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 06:33:23.0815
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ba21527-031e-4e5f-3732-08dc6a71c3f8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF000001B7.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9801

DQoNCj4gT24gMiBNYXkgMjAyNCwgYXQgMDc6MTQsIEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNl
LmNvbT4gd3JvdGU6DQo+IA0KPiBPbiAwMS4wNS4yMDI0IDA4OjU3LCBMdWNhIEZhbmNlbGx1IHdy
b3RlOg0KPj4gSGkgSmFuLA0KPj4gDQo+Pj4gT24gMzAgQXByIDIwMjQsIGF0IDEyOjM3LCBKYW4g
QmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+IHdyb3RlOg0KPj4+IA0KPj4+IE9uIDMwLjA0LjIw
MjQgMTM6MDksIEx1Y2EgRmFuY2VsbHUgd3JvdGU6DQo+Pj4+IC0tLSBhL3hlbi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zZXR1cC5oDQo+Pj4+ICsrKyBiL3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
ZXR1cC5oDQo+Pj4+IEBAIC02NCwxOCArNjQsMjAgQEAgc3RydWN0IG1lbWJhbmsgew0KPj4+PiB9
Ow0KPj4+PiANCj4+Pj4gc3RydWN0IG1lbWJhbmtzIHsNCj4+Pj4gLSAgICB1bnNpZ25lZCBpbnQg
bnJfYmFua3M7DQo+Pj4+IC0gICAgdW5zaWduZWQgaW50IG1heF9iYW5rczsNCj4+Pj4gKyAgICBf
X3N0cnVjdF9ncm91cChtZW1iYW5rc19oZHIsIGNvbW1vbiwgLA0KPj4+PiArICAgICAgICB1bnNp
Z25lZCBpbnQgbnJfYmFua3M7DQo+Pj4+ICsgICAgICAgIHVuc2lnbmVkIGludCBtYXhfYmFua3M7
DQo+Pj4+ICsgICAgKTsNCj4+Pj4gICAgc3RydWN0IG1lbWJhbmsgYmFua1tdOw0KPj4+PiB9Ow0K
Pj4+IA0KPj4+IEknbSBhZnJhaWQgSSBjYW4ndCBzcG90IHdoeSBfX3N0cnVjdF9ncm91cCgpIGlz
IG5lZWRlZCBoZXJlLiBXaHkgd291bGQganVzdA0KPj4+IG9uZSBvZiB0aGUgdHdvIG1vcmUgc3Ry
YWlnaHRmb3J3YXJkDQo+Pj4gDQo+Pj4gc3RydWN0IG1lbWJhbmtzIHsNCj4+PiAgIHN0cnVjdCBt
ZW1iYW5rc19oZHIgew0KPj4+ICAgICAgIHVuc2lnbmVkIGludCBucl9iYW5rczsNCj4+PiAgICAg
ICB1bnNpZ25lZCBpbnQgbWF4X2JhbmtzOw0KPj4+ICAgKTsNCj4+PiAgIHN0cnVjdCBtZW1iYW5r
IGJhbmtbXTsNCj4+PiB9Ow0KPj4+IA0KPj4gDQo+PiBBdCB0aGUgZmlyc3Qgc2lnaHQgSSB0aG91
Z2h0IHRoaXMgc29sdXRpb24gY291bGQgaGF2ZSB3b3JrZWQsIGhvd2V2ZXIgR0NDIGJyb3VnaHQg
bWUgYmFjayBkb3duIHRvIGVhcnRoDQo+PiByZW1lbWJlcmluZyBtZSB0aGF0IGZsZXhpYmxlIGFy
cmF5IG1lbWJlcnMgY2Fu4oCZdCBiZSBsZWZ0IGFsb25lIGluIGFuIGVtcHR5IHN0cnVjdHVyZToN
Cj4+IA0KPj4gL2RhdGFfc2RjL2x1Y2ZhbjAxL2dpdGxhYl9taWNrbGVkb3JlX3hlbi94ZW4veGVu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NzA6NjogZXJyb3I6IGRlY2xhcmF0aW9uIGRv
ZXMgbm90IGRlY2xhcmUgYW55dGhpbmcgWy1XZXJyb3JdDQo+PiA3MCB8IH07DQo+PiB8IF4NCj4+
IC9kYXRhX3NkYy9sdWNmYW4wMS9naXRsYWJfbWlja2xlZG9yZV94ZW4veGVuL3hlbi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9zZXR1cC5oOjcxOjIwOiBlcnJvcjogZmxleGlibGUgYXJyYXkgbWVtYmVy
IGluIGEgc3RydWN0IHdpdGggbm8gbmFtZWQgbWVtYmVycw0KPj4gNzEgfCBzdHJ1Y3QgbWVtYmFu
ayBiYW5rW107DQo+PiB8IF5+fn4NCj4+IFsuLi5dDQo+IA0KPiBTaW5jZSBmb3IgcGF0Y2ggMSB5
b3UgbG9va2VkIGF0IExpbnV4J2VzIHVhcGkvbGludXgvc3RkZGVmLmgsIHRoZSBzb2x1dGlvbg0K
PiB0byB0aGlzIGxpZXMgdGhlcmUsIGluIF9fREVDTEFSRV9GTEVYX0FSUkFZKCkuIEFsb25nc2lk
ZSBvciBpbnN0ZWFkIG9mDQo+IGJvcnJvd2luZyBfX3N0cnVjdF9ncm91cCgpLCB3ZSBjb3VsZCBj
b25zaWRlciBib3Jyb3dpbmcgdGhpcyBhcyB3ZWxsLiBPcg0KPiBvcGVuLWNvZGUgaXQganVzdCBo
ZXJlLCBmb3IgdGhlIHRpbWUgYmVpbmcgKHBlcmhhcHMgbXkgcHJlZmVyZW5jZSkuIFlldA0KPiBp
dCdzIG5vdCBjbGVhciB0byBtZSB0aGF0IGRvaW5nIHNvIHdpbGwgYWN0dWFsbHkgYmUgZW5vdWdo
IHRvIG1ha2UgdGhpbmdzDQo+IHdvcmsgZm9yIHlvdS4NCg0KSSBsb29rZWQgYWxzbyBpbnRvIF9f
REVDTEFSRV9GTEVYX0FSUkFZKCksIGJ1dCB0aGVuIGRlY2lkZWQgX19zdHJ1Y3RfZ3JvdXAoKQ0K
d2FzIGVub3VnaCBmb3IgbXkgcHVycG9zZSwgY2FuIEkgYXNrIHRoZSB0ZWNobmljYWwgcmVhc29u
cyB3aHkgaXQgd291bGQgYmUgeW91cg0KcHJlZmVyZW5jZT8gSXMgdGhlcmUgc29tZXRoaW5nIGlu
IHRoYXQgY29uc3RydWN0IHRoYXQgaXMgYSBjb25jZXJuIGZvciB5b3U/DQoNCg0K


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:36:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:36:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715552.1117269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q3J-0000NM-87; Thu, 02 May 2024 06:36:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715552.1117269; Thu, 02 May 2024 06:36:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q3J-0000NF-4Z; Thu, 02 May 2024 06:36:21 +0000
Received: by outflank-mailman (input) for mailman id 715552;
 Thu, 02 May 2024 06:36:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Q3H-0000N1-NA
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:36:19 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47951d1f-084e-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:36:17 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41ba1ba5592so40611775e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:36:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z18-20020adff752000000b0034e19861891sm430398wrp.33.2024.05.01.23.36.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:36:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47951d1f-084e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714631777; x=1715236577; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+zgd5O08lB3GVxDkuAEkT5TAjyO3f5o57F518B1Bh+c=;
        b=LH5SpfssKMDwPp6XkzhvXufPGtw/bENv9TaSOZlq+U/5FQWdNAJdqzj1sQeLBmxf3Z
         1x7fnkR+is6EJkEcVWciYYIGhd2lS5jIW+Z3XajdJ3pA8nGt89EvfdsW/wQLPrPiU5mt
         7Uk76x3GHaocp2U6MpWSXvXJBLcTL2iPE9h7Kn4h9RmQCrcFMLq5OpJ0i+8N8smnqbER
         gHO2qTuBEiEpwf6HjkfYgIHM1+DpcqdJVtBdjXFN63yfwb+ddpRUa0xVFjBqpUecvWu1
         twvz69KcMVIEN80giSD9i3leLvNqBaacSFUsJ1kymVnMe+6ikh6IA+8cySYT61TOhuRe
         9O3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714631777; x=1715236577;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+zgd5O08lB3GVxDkuAEkT5TAjyO3f5o57F518B1Bh+c=;
        b=lumq2KFBtr8VuMYrbixnaEwploAdRmtwmeILybActUwnkh6m2VjG92yybeC5emZ7ty
         K884rAXnL9Xtu1seQ6B8eBLlWKD9Me/8x9UMFm1JQ6u8s1eceYtWI1zGsSYLxo2Os/g6
         +l6CRKPQq5pU1bY1HXNcUwfLzDWm7upbAl8kO6l2tPkSScWYAwr6QmWK0IG0MgYkaP6t
         OW+5DMoOd+x+7x2cFQNArEds4SKWP8F23uqy5EkMtW4Ulk7PXKBtaijXiRHjVVcXXrG9
         CuHFB/5G6F1VeMhTUjPXSogfOvI2J0nwfObJ1rwcpzTqy/lWN6QY3gHAt82p7X5Yn7Ue
         kgQA==
X-Forwarded-Encrypted: i=1; AJvYcCXKdV0IngduRYBVdILWlgzWcuZWQpwI1JsajsVFZyeXY/SYSkg1QqstBknzAbA0xr8AycSPamECM2X7wVtIqSpKwhScPQVr973ZU+ZwvvU=
X-Gm-Message-State: AOJu0YxXByOkh8ZAZ1TD5K7zAwEIWaNjmzkdkDgghP8Z2qDD39KXJnrt
	kl7TzBuXKFvIbqMjLAfIQGLfjB05Y1qPPndbAcowdv+GS4cvPTtBMHXHkmCPMBwZ1EswKi0dBvY
	=
X-Google-Smtp-Source: AGHT+IFvbz4GmEz1n6lESs/vvvzNkIIt4udh1z310njh8y21NGKVHJF2AtrixRd6FlONhwUFq4FVwA==
X-Received: by 2002:a5d:4011:0:b0:347:3037:188d with SMTP id n17-20020a5d4011000000b003473037188dmr935046wrp.34.1714631776866;
        Wed, 01 May 2024 23:36:16 -0700 (PDT)
Message-ID: <115a6513-559c-44d9-a828-10d1e4e5c401@suse.com>
Date: Thu, 2 May 2024 08:36:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 7/7] x86/hap: Increase the number of initial
 mempool_size to 1024 pages
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714322424.git.w1benny@gmail.com>
 <a26bc4aeba89f7895c79df7e320adfc695b16d50.1714322424.git.w1benny@gmail.com>
 <4d9930d4-4379-4440-83b2-14ebffc7c03d@suse.com>
 <CAKBKdXgAgb=+Spr_nobWy_zuN-yXTUfAL+Q3MdLDCDVgshOS7Q@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAKBKdXgAgb=+Spr_nobWy_zuN-yXTUfAL+Q3MdLDCDVgshOS7Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30.04.2024 17:40, Petr Beneš wrote:
> On Tue, Apr 30, 2024 at 4:47 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 28.04.2024 18:52, Petr Beneš wrote:
>>> From: Petr Beneš <w1benny@gmail.com>
>>>
>>> This change anticipates scenarios where `max_altp2m` is set to its maximum
>>> supported value (i.e., 512), ensuring sufficient memory is allocated upfront
>>> to accommodate all altp2m tables without initialization failure.
>>
>> And guests with fewer or even no altp2m-s still need the same bump? You
>> know the number of altp2m-s upon domain creation, so why bump by any more
>> than what's strictly needed for that?
> 
> I have to admit I've considered computing the value which goes to
> hap_set_allocation
> by simply adding 256 + max_altp2m, but that felt so arbitrary - the
> 256 value itself
> feels arbitrary, as I haven't found any reasoning for it anywhere.
> 
> I have also tried to make code changes to make the initial allocation
> size configurable
> via libxl (possibly reusing the shadow_memkb) - which seemed to me
> like the "correct"
> solution, but those changes were more complicated than I had
> anticipated and I would
> definitely not make it till the 4.19 deadline.
> 
> Question is, what to do now? Should I change it to 256 + max_altp2m?

Counter question: Is accounting for just the root page table really
enough? Meaning to say: I'm not convinced that minimum would really
be appropriate for altp2m use even before your changes. You growing
the number of root page tables _always_ required just makes things
worse even without considering how (many) altp2m-s are then going
to be used. Such an issue, if I'm right with this, would imo want
addressing up front, in a separate patch.

You may also want to take a look at the shadow side of things, even
if for altp2m shadow mode may not be relevant. Things like
sh_min_allocation() and shadow_min_acceptable_pages() may well need
proper counterparts in HAP now.

>>> --- a/tools/tests/paging-mempool/test-paging-mempool.c
>>> +++ b/tools/tests/paging-mempool/test-paging-mempool.c
>>> @@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = {
>>>
>>>  static uint64_t default_mempool_size_bytes =
>>>  #if defined(__x86_64__) || defined(__i386__)
>>> -    256 << 12; /* Only x86 HAP for now.  x86 Shadow needs more work. */
>>> +    1024 << 12; /* Only x86 HAP for now.  x86 Shadow needs more work. */
>>
>> I also can't derive from the description why we'd need to go from 256 to
>> 1024 here and ...
> 
> It's explained in the code few lines below:
> 
>     /*
>      * Check that the domain has the expected default allocation size.  This
>      * will fail if the logic in Xen is altered without an equivalent
>      * adjustment here.
>      */
> 
> I have verified that the default_mempool_size_bytes must reflect the number
> of pages in the initial hap_set_allocation() call.
> 
> Is it something I should include in the commit message, too?

Well, yes and no. My question wasn't about the "why", but ...

>>> --- a/xen/arch/x86/mm/hap/hap.c
>>> +++ b/xen/arch/x86/mm/hap/hap.c
>>> @@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)
>>>      if ( old_pages == 0 )
>>>      {
>>>          paging_lock(d);
>>> -        rv = hap_set_allocation(d, 256, NULL);
>>> +        rv = hap_set_allocation(d, 1024, NULL);
>>
>> ... here. You talk of (up to) 512 pages there only.

... about the "by how many".

>> Also isn't there at least one more place where the tool stack (libxl I
>> think) would need changing, where Dom0 ballooning needs are calculated?
>> And/or doesn't the pool size have a default calculation in the tool
>> stack, too?
> 
> I have found places in libxl where the mempool_size is calculated, but
> that mempool
> size is then set AFTER the domain is created via xc_set_paging_mempool_size.
> 
> In my opinion it doesn't necessarily require change, since it's
> expected by the user
> to manually set it via shadow_memkb. The only current problem is (which this
> commit is trying to fix) that setting shadow_memkb doesn't help when
> max_altp2m > (256 - 1 + vcpus + MAX_NESTEDP2M), since the initial mempool
> size is hardcoded.

Wait - are you saying the guest config value isn't respected in certain
cases? That would be another thing wanting to be fixed separately, up
front.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:38:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:38:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715558.1117279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q50-0000y7-KB; Thu, 02 May 2024 06:38:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715558.1117279; Thu, 02 May 2024 06:38:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q50-0000y0-HY; Thu, 02 May 2024 06:38:06 +0000
Received: by outflank-mailman (input) for mailman id 715558;
 Thu, 02 May 2024 06:38:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Q4z-0000xu-2h
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:38:05 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8729118e-084e-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 08:38:04 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41bca450fa3so37820505e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:38:04 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n12-20020adfe34c000000b00343d6c7240fsm431229wrj.35.2024.05.01.23.38.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:38:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8729118e-084e-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714631883; x=1715236683; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IgNSONZvYMrYXUmiqnhAKX2GzQFE5R07z74If2PXkfM=;
        b=cbH3wfqY/TZi5k7gydIiDSmpmqkbRe7MVCw9rFoPENFbwevLfPM1LkhPopylw11J3W
         s+2zBLYUJPLdZhF2gqsjeA9wohGXlB5HI7oBZN6xKt3lWqteH7GfY7g45N8lHvzV84BW
         kQF5ND7Vt+MbgpmRixGiTMojK5KXPsf+4c4AzXmdPH+6UoLJ5ELLD/dr/JLI1rYmyLfn
         s7JzgS2yVKi0o+Zjq69V4vPXPd7jkayRxv8QdNpjjkZsBb45qISQTe+1BBuhAcIZYwps
         qV+GPNbJmrvEIQJ67YGz/s4x6mSOzTtjNwtmUMob0gSRP9c9TDYLgPZ92uLVu4uzs4SW
         qUDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714631883; x=1715236683;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IgNSONZvYMrYXUmiqnhAKX2GzQFE5R07z74If2PXkfM=;
        b=BV3OIGL0Y8UPYeQZqHYcydzqwATz0QjupCliNd0ElPCTJ/ajcPhiyUbncURtEqmuNS
         lIZSqf9YVl6S3o3bvH/W6ENscTW5cX0MwFVk8xur378piDRKxpd/xnFr1HgjZc+ko8mb
         9cRSmVzHhP+er/Xlm2MhwcqsHcppzzdVy5Giw5ZjaWO8mKXWjOjiv7s76r9sSNM2YiBZ
         aRWMfMJMsX6fVYKu9F4cLfM+mZ8i8YmeMv8SAa2WEFhTzlcPyYeN+MLgSQ54z33ehGq3
         6J0q+yFDDb543qIewlUwlaDVYphoGioCnGGpbMVnPWSDgbA4opfgiiv/KkDk05My/2Q+
         N1lQ==
X-Forwarded-Encrypted: i=1; AJvYcCWOyCa3ZpiovAbCJh0fJXlmIBkBdwuo/KP2F9awNfUzGdidNLfToDCRd4NB1cLjcU74Owdn9olOrWaKRO+MChNnRHWRFY2Qw9ZkfgemnTM=
X-Gm-Message-State: AOJu0YzaVPjCPxENS3ypvGrDvmL+4rcX7dHb27BjVZnFyzBfcRFKqXeZ
	bC5QaXyVYe2dNEKEssi1Qb+UkXJ5Y48c28o/IQ0RZTLuPEfOekzBPTCKNlZGhw==
X-Google-Smtp-Source: AGHT+IE3u5DmjeLLM+wW/7k4BjXiZZGTxrUm+IOyXWWfZyBnK1EaG9KUyOrQ6S/jkv7k7vVZxbJyvQ==
X-Received: by 2002:a05:600c:1ca6:b0:41b:f979:e359 with SMTP id k38-20020a05600c1ca600b0041bf979e359mr3280074wms.38.1714631883548;
        Wed, 01 May 2024 23:38:03 -0700 (PDT)
Message-ID: <26d11f73-8359-4ab7-9ac2-71b632fe3ef9@suse.com>
Date: Thu, 2 May 2024 08:38:02 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
Content-Language: en-US
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-2-luca.fancellu@arm.com>
 <cd2352aa-e94b-43ef-b3fc-359cc1f43fb0@suse.com>
 <2AB02E85-2746-4DD6-A821-B858E52A6954@arm.com>
 <16815d66-ffe7-45e9-8311-5d4ed8754d2f@suse.com>
 <C6F3416D-785A-4C1F-96B9-1C7A53B927EA@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <C6F3416D-785A-4C1F-96B9-1C7A53B927EA@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 08:23, Luca Fancellu wrote:
> 
> 
>> On 2 May 2024, at 07:09, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 01.05.2024 08:54, Luca Fancellu wrote:
>>>> On 30 Apr 2024, at 12:43, Jan Beulich <jbeulich@suse.com> wrote:
>>>> On 30.04.2024 13:09, Luca Fancellu wrote:
>>>>> --- a/xen/include/xen/kernel.h
>>>>> +++ b/xen/include/xen/kernel.h
>>>>> @@ -54,6 +54,27 @@
>>>>>        typeof_field(type, member) *__mptr = (ptr);             \
>>>>>        (type *)( (char *)__mptr - offsetof(type,member) );})
>>>>>
>>>>> +/**
>>>>> + * __struct_group() - Create a mirrored named and anonyomous struct
>>>>> + *
>>>>> + * @TAG: The tag name for the named sub-struct (usually empty)
>>>>> + * @NAME: The identifier name of the mirrored sub-struct
>>>>> + * @ATTRS: Any struct attributes (usually empty)
>>>>> + * @MEMBERS: The member declarations for the mirrored structs
>>>>> + *
>>>>> + * Used to create an anonymous union of two structs with identical layout
>>>>> + * and size: one anonymous and one named. The former's members can be used
>>>>> + * normally without sub-struct naming, and the latter can be used to
>>>>> + * reason about the start, end, and size of the group of struct members.
>>>>> + * The named struct can also be explicitly tagged for layer reuse, as well
>>>>> + * as both having struct attributes appended.
>>>>> + */
>>>>> +#define __struct_group(TAG, NAME, ATTRS, MEMBERS...) \
>>>>> +    union { \
>>>>> +        struct { MEMBERS } ATTRS; \
>>>>> +        struct TAG { MEMBERS } ATTRS NAME; \
>>>>> +    } ATTRS
>>>>
>>>> Besides my hesitance towards having this construct, can you explain why
>>>> ATTR needs using 3 times, i.e. also on the wrapping union?
>>>
>>> The original commit didn’t have the third ATTRS, but afterwards it was introduced due to
>>> this:
>>>
>>> https://patchwork.kernel.org/project/linux-wireless/patch/20231120110607.98956-1-dmantipov@yandex.ru/#25610045
>>
>> Hmm. Since generally packing propagates to containing compound types, I'd
>> say this cannot go without a code comment, or at the very least a mention
>> in the description. But: Do we use this old ABI in Xen at all? IOW can we
>> get away without this 3rd instance?
> 
> Yes, I think it won’t be a problem for Xen, is it something that can be done on commit?

Don't know, maybe. First you need an ack, and I remain unconvinced that we
actually need this construct.

Jan

> Anyway in case of comments on the second patch, I’ll push the change also for this one.
> 
> Cheers,
> Luca
> 



From xen-devel-bounces@lists.xenproject.org Thu May 02 06:43:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:43:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715565.1117288 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q9s-0002QT-5G; Thu, 02 May 2024 06:43:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715565.1117288; Thu, 02 May 2024 06:43:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Q9s-0002QM-2i; Thu, 02 May 2024 06:43:08 +0000
Received: by outflank-mailman (input) for mailman id 715565;
 Thu, 02 May 2024 06:43:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Q9q-0002QG-D7
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:43:06 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 398fedc4-084f-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:43:03 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41b4ff362a8so70651795e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:43:03 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p12-20020a05600c1d8c00b0041bcb898984sm791184wms.31.2024.05.01.23.43.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:43:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 398fedc4-084f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714632183; x=1715236983; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MvxunB0V+uViE8WQxSg4C7zZwRt/Mn7AaDi3tcb3hEU=;
        b=TCFvErmgyyJzJCfGwKyepdpEBKsEuh59Vj1WXxISly/1PtLKOh0ZDj11lIqi4TH68o
         7FXbNlrtJlKjARdoKFliAQPS/QnR+osxa+Vt2hOCHMEVhheEgUzI64IWuyYZL6u+6w+i
         SaNDgrOtZRwhZ8wtQgG/DEUnOQl2tKrC7M9QKIxxQrIcXAI5ZUc++doAWTaloqTEGHiJ
         jTprYgHRadnNCbcbbKTxr+NQKBb6XwuP8w2Bd/OA+cwfyH5238fT5EyBCAK2wv3ST4JQ
         YuTQE3xVsv2Ql9bQt4Ep0vfTp7B5jU9996ulk+Se7/TfqcKJRvk41OhFchf/+qGg/8lD
         GHbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714632183; x=1715236983;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MvxunB0V+uViE8WQxSg4C7zZwRt/Mn7AaDi3tcb3hEU=;
        b=MpuTkhES2rO4Vn1yAX4PSfO0uwhSKH26mCbSN8roi1zb8O2lcx8cAa9RDBjoIBYOow
         rR6WfuhyD49rR0i9ZRTHqVCAqUsW0qMFtYQe1QeCEsrxAroapPIViSqaNeai8atNi+SJ
         ljUjR4DU4VWSECf6Qvz2RLexIz8lGlP3sOV3wWE7TBH+/hGewOo4rf14GGrtWMuIT1SS
         rGlHBqeui2j5wPfSx2IiZQC68d1bbI0O1DBRAPluPrPVMCieZq/j2e3tQzodnUKq9Jyp
         N4S7DGqDjUmtEdG+p1eEnsHsN0Eam2IPvRe84WF3lwTZscQ4TK17xb45NaN43ZVc0CdX
         ho6Q==
X-Forwarded-Encrypted: i=1; AJvYcCW4/6lpKNjBEMz2FOJ1fxMYoUBTBwhH4Vss08uduTYx727iapiWRgd2rG+Bb3pbN/Hh2QBsAshQrZNigdbFhEt6/wLWkYAf7ADT1LqGJkQ=
X-Gm-Message-State: AOJu0YzqypoR5ttp1rZRaDrXrOmOBAvKpRcfgcSoYKy3Vlw3+kVA6D9L
	ieRb+izGREYHoUsBa7WaSw89a/FqKEcrbnUjIAKGROj//7kiHHFpLYgNL51t6w==
X-Google-Smtp-Source: AGHT+IEsAPHrsVZoklCsm2+0z+E5Q2OOycBmiWrl109MnjaYFJL5k2Cu4QR9B4K0je9XpquHusPvzg==
X-Received: by 2002:a05:600c:3516:b0:41b:eaf2:f7e6 with SMTP id h22-20020a05600c351600b0041beaf2f7e6mr4386004wmq.2.1714632182858;
        Wed, 01 May 2024 23:43:02 -0700 (PDT)
Message-ID: <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
Date: Thu, 2 May 2024 08:43:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Content-Language: en-US
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 08:33, Luca Fancellu wrote:
> 
> 
>> On 2 May 2024, at 07:14, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 01.05.2024 08:57, Luca Fancellu wrote:
>>> Hi Jan,
>>>
>>>> On 30 Apr 2024, at 12:37, Jan Beulich <jbeulich@suse.com> wrote:
>>>>
>>>> On 30.04.2024 13:09, Luca Fancellu wrote:
>>>>> --- a/xen/arch/arm/include/asm/setup.h
>>>>> +++ b/xen/arch/arm/include/asm/setup.h
>>>>> @@ -64,18 +64,20 @@ struct membank {
>>>>> };
>>>>>
>>>>> struct membanks {
>>>>> -    unsigned int nr_banks;
>>>>> -    unsigned int max_banks;
>>>>> +    __struct_group(membanks_hdr, common, ,
>>>>> +        unsigned int nr_banks;
>>>>> +        unsigned int max_banks;
>>>>> +    );
>>>>>    struct membank bank[];
>>>>> };
>>>>
>>>> I'm afraid I can't spot why __struct_group() is needed here. Why would just
>>>> one of the two more straightforward
>>>>
>>>> struct membanks {
>>>>   struct membanks_hdr {
>>>>       unsigned int nr_banks;
>>>>       unsigned int max_banks;
>>>>   );
>>>>   struct membank bank[];
>>>> };
>>>>
>>>
>>> At the first sight I thought this solution could have worked, however GCC brought me back down to earth
>>> remembering me that flexible array members can’t be left alone in an empty structure:
>>>
>>> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:70:6: error: declaration does not declare anything [-Werror]
>>> 70 | };
>>> | ^
>>> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:71:20: error: flexible array member in a struct with no named members
>>> 71 | struct membank bank[];
>>> | ^~~~
>>> [...]
>>
>> Since for patch 1 you looked at Linux'es uapi/linux/stddef.h, the solution
>> to this lies there, in __DECLARE_FLEX_ARRAY(). Alongside or instead of
>> borrowing __struct_group(), we could consider borrowing this as well. Or
>> open-code it just here, for the time being (perhaps my preference). Yet
>> it's not clear to me that doing so will actually be enough to make things
>> work for you.
> 
> I looked also into __DECLARE_FLEX_ARRAY(), but then decided __struct_group()
> was enough for my purpose, can I ask the technical reasons why it would be your
> preference? Is there something in that construct that is a concern for you?

I don't like either construct very much, but of the two __DECLARE_FLEX_ARRAY()
looks slightly more "natural" for what is wanted and how it's done.
__struct_group() introducing twice the (effectively) same structure feels
pretty odd, for now at least. It's not even entirely clear to me whether there
aren't pitfalls, seeing that the C spec differentiates named and unnamed
struct fields in a few cases. For __DECLARE_FLEX_ARRAY(), otoh, I can't
presently see any reason to suspect possible corner cases.

Yet as said before - I'm not sure __DECLARE_FLEX_ARRAY() alone would be enough
for what you want to achieve.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:48:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:48:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715571.1117298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QEq-0003l1-O3; Thu, 02 May 2024 06:48:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715571.1117298; Thu, 02 May 2024 06:48:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QEq-0003ku-LY; Thu, 02 May 2024 06:48:16 +0000
Received: by outflank-mailman (input) for mailman id 715571;
 Thu, 02 May 2024 06:48:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2QEp-0003iD-0k
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:48:15 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f279ba6c-084f-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 08:48:13 +0200 (CEST)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-41b5e74fa2fso48503405e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:48:13 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p17-20020a05600c359100b0041adf358058sm813451wmq.27.2024.05.01.23.48.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:48:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f279ba6c-084f-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714632493; x=1715237293; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gx1POZzfK9lHz+kf5dqZtJ/lY88gBcic1RTUVhENGYs=;
        b=W04zCkq3oRdKvqtVFFgMqY+bxhpfntnWnHQypAr9BoKJLyOgLObn9dijHXKE/q1AQk
         X7BiaQGfMOEjgqcxKQAwy7gBpegj7M36C7a3crIlI6jUng2suWqGrdoG7OczIQsRN9KV
         1rTeSIDkll/8FszGEXvH9E5TWwKkcWYe1W1ynQAYNU1tKJOutyU4iNstFVfMGQFrFqJM
         MIct0q5v7vc3vtqHIVxbNtgIJH1JZONBbOETpApZR3frpohaSZxuSzXGm9FA7c8fdUqU
         fQVK2uoVtOmZHyqiFePd79Vg6Zp/mcFZdTQ/tDTrX94N53Djndr/mWhHZI5s8GKeqlAd
         eH9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714632493; x=1715237293;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gx1POZzfK9lHz+kf5dqZtJ/lY88gBcic1RTUVhENGYs=;
        b=qeXnLWE9az8EUQMinGv4osI2GSC8a4urDuxSQyawhDYvvr3T54DU6f1GjV90v304oP
         vvxfusst0xLv1szfz6v4a0jD4FeexI+12APX1KFTvnN2tus6jndAxPo/Y7fM9JSDt9lj
         bNpaiooAA0MEqMzLS/mg1z1af2UFh5VZg4hrD+6rcPBH91WECLoQJugVZy3+XMr6mz7B
         gV5HWQ0grAokLBJgZIqPCxXjY/X6nh1nnf35hDf3vYMvRlUArNwwiIwM1jTVejCk0IEG
         usRU1CYH4Ac3ISreTgSgxjgnrqhkuSU1380y4I7304iOY/VECTqpaCgyaBcSMUx3x6aC
         57qg==
X-Forwarded-Encrypted: i=1; AJvYcCXbQHOxIFFg0nCk6HsD0J/kFrRgZhLM9yGUbCl0kjgk2sqiwo7ts8U5+u2HZSGIHiToNHOQ4At1XU+ASm8+Z5M4eL0LPtgU1Uf39YErxeI=
X-Gm-Message-State: AOJu0Ywsi+I/TbTDEGkwXO3S5ZMcTDJOf4jZBZsjcA++S2dESs2nGlv3
	Tl043k9OPNkCnKDE6h1jUWOwjQjcc5GSjmUwSxCypXYPP2cNykyUSg53DDTGtw==
X-Google-Smtp-Source: AGHT+IHxlwGKSsqYyp/KeOt2naWfx4KxvgB7Be3qsTChPDPAtJcz7sNxLF49gWJt+aW6Ag9lSMzTdQ==
X-Received: by 2002:a05:600c:3b06:b0:41c:190:2b94 with SMTP id m6-20020a05600c3b0600b0041c01902b94mr3291909wms.6.1714632493326;
        Wed, 01 May 2024 23:48:13 -0700 (PDT)
Message-ID: <4cd997f9-a517-41f7-862e-37ca5984d48c@suse.com>
Date: Thu, 2 May 2024 08:48:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 07/27] x86: Map/unmap pages in
 restore_all_guests
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-8-eliasely@amazon.com>
 <a2ce9820-c57a-4690-9dc6-c15d8a1489f4@suse.com>
 <a5ba2399-30a6-4ca4-bb8c-cf8773f862b9@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a5ba2399-30a6-4ca4-bb8c-cf8773f862b9@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 18:08, Elias El Yandouzi wrote:
>>> --- a/xen/arch/x86/pv/domain.c
>>> +++ b/xen/arch/x86/pv/domain.c
>>> @@ -288,6 +288,19 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
>>>                                 1U << GDT_LDT_VCPU_SHIFT);
>>>   }
>>>   
>>> +static int pv_create_shadow_root_pt_l1tab(struct vcpu *v)
>>> +{
>>> +    return create_perdomain_mapping(v->domain, SHADOW_ROOT_PT_VCPU_VIRT_START(v),
>>
>> This line looks to be too long. But ...
>>
>>> +                                    1, v->domain->arch.pv.shadow_root_pt_l1tab,
>>> +                                    NULL);
>>> +}
>>> +
>>> +static void pv_destroy_shadow_root_pt_l1tab(struct vcpu *v)
>>> +
>>> +{
>>> +    destroy_perdomain_mapping(v->domain, SHADOW_ROOT_PT_VCPU_VIRT_START(v), 1);
>>> +}
>>
>> ... I'm not convinced of the usefulness of these wrapper functions
>> anyway, even more so that each is used exactly once.
> 
> The wrappers have been introduced to remain consistent with what has 
> been done with GDT/LDT table. I would like to keep them if you don't mind.

Hmm, yes, I can see your point.

>>> @@ -371,6 +394,12 @@ int pv_domain_initialise(struct domain *d)
>>>           goto fail;
>>>       clear_page(d->arch.pv.gdt_ldt_l1tab);
>>>   
>>> +    d->arch.pv.shadow_root_pt_l1tab =
>>> +        alloc_xenheap_pages(0, MEMF_node(domain_to_node(d)));
>>> +    if ( !d->arch.pv.shadow_root_pt_l1tab )
>>> +        goto fail;
>>> +    clear_page(d->arch.pv.shadow_root_pt_l1tab);
>>
>> Looks like you simply cloned the GDT/LDT code. That's covering 128k
>> of VA space per vCPU, though, while here you'd using only 4k. Hence
>> using a full page looks like a factor 32 over-allocation. And once
>> using xzalloc() here instead a further question would be whether to
>> limit to the domain's actual needs - most domains will have far less
>> than 8k vCPU-s. In the common case (up to 512 vCPU-s) a single slot
>> will suffice, at which point a yet further question would be whether
>> to embed the "array" in struct pv_domain instead in that common case
>> (e.g. by using a union).
> 
> I have to admit I don't really understand your suggestion. Could you 
> elaborate a bit more?

The (per vCPU) GDT and LDT are together taking up 128k of VA space.
Whereas you need only 4k. Therefore I was asking why you're over-
allocating by so much.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:55:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:55:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715577.1117308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QLl-0005x3-E2; Thu, 02 May 2024 06:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715577.1117308; Thu, 02 May 2024 06:55:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QLl-0005wu-BP; Thu, 02 May 2024 06:55:25 +0000
Received: by outflank-mailman (input) for mailman id 715577;
 Thu, 02 May 2024 06:55:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2QLk-0005wo-EA
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:55:24 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f1f05bed-0850-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:55:22 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2e01d666c88so48273551fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:55:22 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f12-20020a05600c154c00b0041be609b1c7sm4556592wmg.47.2024.05.01.23.55.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:55:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1f05bed-0850-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714632922; x=1715237722; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RLW3A3W679+djf8LQqd0VGX3LYFdhxBCP7IKuy2SWvg=;
        b=YGTTSuQKK/KnPrycOojQfsF6TnXL1/bDCrMjo6vzGzeRhI8v8/H4I62KHhwVCO/LCX
         JFcqBmbIAg7BVGpqhMOnedIbNV/GMEif20JUYIGzaaNiTuhaeoS6GAFosOyRS0j/Bk9V
         Uu/qWjuuv4by1lH+NLR4SxbJiQ6ICHWTviSI581oRc7JKTc3186eEYbegIw+zVsQEn/X
         F76fqeCgtsLZBagUxRRDnO4AyTMxdx3fDxnw1zycTfuyLYtJrYlBBx7xQCAdjMsferLn
         9PbFofHoxX5uJBDG7r/qun6vokz+08WlJlnO/MRKosdsaO1RdULTzsJc7CpgxfJfZRGV
         8hQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714632922; x=1715237722;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RLW3A3W679+djf8LQqd0VGX3LYFdhxBCP7IKuy2SWvg=;
        b=qDpuW4ygtZutFKr3+hMDESqS5fs/0CV6sSJ/75ttpR08G/KMJnHRikqpVHr8FPy2iD
         8Cn6qgQle4QPPog/X/M12tNRFu38SIQ0usXnNF+DVd8V48xCpyk2wdZMEqSdAfFT7Xu+
         +ZWVvLKm1lPY2kPn353mKRclwtZ8kzttgzCedv1MUmfZnCtkL1i1UWmXAw8hmCVMrdIJ
         wHmTN0xIwYtXcDY4PLfc6l9VQswqBGcD/g3Qc37OoYiXmkPWsZkMAo2rzTHD47czSsLr
         eMFt1eZOPhFHwKM/xIXeV+7NKSXdJ8fZoWLjFASs8kbh3EV54dzH8PXzrlaf4zARQl/C
         n6Rg==
X-Forwarded-Encrypted: i=1; AJvYcCXkEiaWEw1q3kMOsKY4/wAsCJZRWERJY6otzhScYygP9gCxFx4TL4c2zuOqSNuEYBWcDwbl9nEkKpOcQ2bO+0DJq8QSBY/dRaUlrgnjw9Y=
X-Gm-Message-State: AOJu0Yy603jRDobSv31KFaIJW6Q5bGuR9DwBI5w6RWfDi+Vxb3j0AqXX
	59N7OM1ljNAUs8SQv22UzBHxxRyrTofLVpVtKcgd8GxmuP0qfp4MH7OOM9Uu/Q==
X-Google-Smtp-Source: AGHT+IGVOyu6L11lfIuoIjTVmrfoxWNT36SdASpjj+MglQQnCkoeTOWrP664Kg7flhGmDPLfNg/LNA==
X-Received: by 2002:a05:651c:a10:b0:2df:c0b8:2dd6 with SMTP id k16-20020a05651c0a1000b002dfc0b82dd6mr3465809ljq.21.1714632921584;
        Wed, 01 May 2024 23:55:21 -0700 (PDT)
Message-ID: <52741b69-8bcb-492f-a8bc-f035b2747fe8@suse.com>
Date: Thu, 2 May 2024 08:55:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] xen/x86: Derive topologically correct x2APIC IDs from
 the policy
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-6-alejandro.vallejo@cloud.com>
 <14891d02-e9e9-4650-b572-e6f071f1506c@suse.com>
 <e0963ddd-c3b7-44dd-a3b7-67811f3e9d7a@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e0963ddd-c3b7-44dd-a3b7-67811f3e9d7a@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.05.2024 18:35, Alejandro Vallejo wrote:
> Hi,
> 
> On 26/03/2024 16:41, Jan Beulich wrote:
>> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>>> --- a/xen/lib/x86/policy.c
>>> +++ b/xen/lib/x86/policy.c
>>> @@ -2,15 +2,78 @@
>>>  
>>>  #include <xen/lib/x86/cpu-policy.h>
>>>  
>>> -uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t vcpu_id)
>>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>>>  {
>>>      /*
>>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>>> -     *       until all infra is in place to retrieve or derive the initial
>>> -     *       x2APIC ID from migrated domains.
>>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>>> +     * the next topological scope. For example, assuming a system with 2
>>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>>> +     * the number of threads in a module.
>>> +     *
>>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>>> +     * level (cores/complex, etc) so we can return it as-is.
>>>       */
>>> -    return vcpu_id * 2;
>>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>>> +        return p->topo.subleaf[lvl].nr_logical;
>>
>> Is "!= Intel" really appropriate here? I'd rather see this being "AMD || Hygon".
> 
> Sure, I don't particularly mind, but why? As far as we know only Intel
> has this interpretation for the part counts. I definitely haven't seen
> any non-Intel CPUID dump in which the part count is the total number of
> threads (Centaur/Zhaoxin are not multithreaded, and don't expose leaves
> 1f or e26, as far as I could see).

Because of x86'es origin and perhaps other historical aspects, cloning
Intel behavior is far more likely. The fact that Hygon matches AMD is
simply because they took AMD's design wholesale.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 06:58:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 06:58:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715583.1117319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QOK-0006YI-U2; Thu, 02 May 2024 06:58:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715583.1117319; Thu, 02 May 2024 06:58:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QOK-0006YA-QT; Thu, 02 May 2024 06:58:04 +0000
Received: by outflank-mailman (input) for mailman id 715583;
 Thu, 02 May 2024 06:58:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2QOJ-0006Y4-J3
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 06:58:03 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 50ea51e6-0851-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 08:58:01 +0200 (CEST)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2dd6c14d000so92822761fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 May 2024 23:58:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f20-20020a05600c4e9400b0041b5500e438sm845449wmq.23.2024.05.01.23.58.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 May 2024 23:58:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50ea51e6-0851-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714633081; x=1715237881; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=HCOzvp6QxP/hXwE5Y9W1gwdgfBsEHmk7xOXrbcEFRKA=;
        b=Sg31uRkya5gB7IDV98peiQ4MTUaHd3ukBCf4uboZ+Wh3j3+1G8WLn57O/GaTIHkmss
         5/3wQHlxurELZMASrky42G/+AOgZw2f7ODzBXj1e6awF5LcdTb2e7zvS0gnGuGoOdaz8
         hk4y5nLmZqI6rvI4b7WogElis97HZzt4BmZHHnd5ewiBIYX/wFoxkNBVoj1Pq27PIw3H
         SmBfgApLunmrniIonF55LJEpV+/mZ5V1RbpfAb8ozseYgbTHFQ+i6YM0a/77Hp+E2v1H
         cMQBtZapYkTH7hrbeMjLczTk4yECnggbEnzpVnssXJwENOpTgtmrCcqN+EaOpPDem5Gi
         cVgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714633081; x=1715237881;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=HCOzvp6QxP/hXwE5Y9W1gwdgfBsEHmk7xOXrbcEFRKA=;
        b=q6bfmER6bpl5IviD6k0md4LeUUTPPFa96ZfPoD+AyHmvw6h/NFfRbAh8oIyfRNeP91
         e2PdlkmAwpTx9iGfI9BlViec4+8TXEA5+A/rCAQM8MYcgPnK923i/XWy59UM/mdG1S3d
         2vIOl4PrMPTv3aLVd0R2e9+Tyj1rgNvPJLpRvJ/3kWGSTp1VZK9DLxhsnA1gJZWjAZ2a
         yHXIjxDc9eCI8P22sK4UoLV4HBjwLESFqmzhipA9lhpzBxh/GztKYNdE3KcDIP61AerY
         YZTj00dIyNf7OXU/5wedYg5aV65vBNQLKUS23q3u0PZ57NHUwrWNo5Sqc6L1YR4bR81j
         GfwQ==
X-Forwarded-Encrypted: i=1; AJvYcCVZNrcS9cXIItVEwhpPngj8etOlsWPRBY+ogb+/ESjS28bnUbDI7ZR1c3+1SY8+ULJyUSCL5Mp8FYJM+HHFm6cHXSfwY/R+Vewih/FTN+A=
X-Gm-Message-State: AOJu0YxzO2qpHhDOxDRIk4GxbpuEebu2RkNA07kGBMxlU62lfCx8KdJr
	o1R9aIWrN4lhwEXdhjK8NKZrS1+Ah6OK8hxBulgFw83EujvEGShNX7gnyqtwAA==
X-Google-Smtp-Source: AGHT+IGwPphjtRzMFFKpSGhz7HxpTDjRFtdYxYlL7qSEyU5FvCZejfBTg8NTO8bOdoVhk1zzJ6MWig==
X-Received: by 2002:a2e:9d8d:0:b0:2d8:5b46:788c with SMTP id c13-20020a2e9d8d000000b002d85b46788cmr621431ljj.17.1714633081011;
        Wed, 01 May 2024 23:58:01 -0700 (PDT)
Message-ID: <4a280eab-81f7-4a87-b531-3633311a4c4a@suse.com>
Date: Thu, 2 May 2024 08:57:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] xen/x86: Derive topologically correct x2APIC IDs from
 the policy
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-6-alejandro.vallejo@cloud.com>
 <14891d02-e9e9-4650-b572-e6f071f1506c@suse.com>
 <e0963ddd-c3b7-44dd-a3b7-67811f3e9d7a@cloud.com>
 <52741b69-8bcb-492f-a8bc-f035b2747fe8@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <52741b69-8bcb-492f-a8bc-f035b2747fe8@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 08:55, Jan Beulich wrote:
> On 01.05.2024 18:35, Alejandro Vallejo wrote:
>> Hi,
>>
>> On 26/03/2024 16:41, Jan Beulich wrote:
>>> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>>>> --- a/xen/lib/x86/policy.c
>>>> +++ b/xen/lib/x86/policy.c
>>>> @@ -2,15 +2,78 @@
>>>>  
>>>>  #include <xen/lib/x86/cpu-policy.h>
>>>>  
>>>> -uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t vcpu_id)
>>>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>>>>  {
>>>>      /*
>>>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>>>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>>>> -     *       until all infra is in place to retrieve or derive the initial
>>>> -     *       x2APIC ID from migrated domains.
>>>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>>>> +     * the next topological scope. For example, assuming a system with 2
>>>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>>>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>>>> +     * the number of threads in a module.
>>>> +     *
>>>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>>>> +     * level (cores/complex, etc) so we can return it as-is.
>>>>       */
>>>> -    return vcpu_id * 2;
>>>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>>>> +        return p->topo.subleaf[lvl].nr_logical;
>>>
>>> Is "!= Intel" really appropriate here? I'd rather see this being "AMD || Hygon".
>>
>> Sure, I don't particularly mind, but why? As far as we know only Intel
>> has this interpretation for the part counts. I definitely haven't seen
>> any non-Intel CPUID dump in which the part count is the total number of
>> threads (Centaur/Zhaoxin are not multithreaded, and don't expose leaves
>> 1f or e26, as far as I could see).
> 
> Because of x86'es origin and perhaps other historical aspects, cloning
> Intel behavior is far more likely. The fact that Hygon matches AMD is
> simply because they took AMD's design wholesale.

Perhaps: See how many dead ends AMD have created, i.e. stuff they proudly
introduced into the architecture, but then gave up again (presumably for
diverging too far from Intel, and hence lacking long term acceptance):
3DNow!, LWP, and XOP just to name those that come to mind right away.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 07:13:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:13:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715589.1117329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QdL-0001P8-6h; Thu, 02 May 2024 07:13:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715589.1117329; Thu, 02 May 2024 07:13:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QdL-0001P1-2W; Thu, 02 May 2024 07:13:35 +0000
Received: by outflank-mailman (input) for mailman id 715589;
 Thu, 02 May 2024 07:13:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2QdJ-0001Or-Ns
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:13:33 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b407877-0853-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 09:13:31 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41b782405d5so64836635e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:13:31 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p20-20020a05600c469400b0041bc41287cesm889702wmo.16.2024.05.02.00.13.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 00:13:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b407877-0853-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714634011; x=1715238811; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Y/eop79hcQEtCziVc8UWDGi7axuiN6CblRErcvnW9g0=;
        b=BbVfR6mxRcrCDsjl2xA6WsngtCj2qyX7IfpIVEeaCLcLetVbhJfcAMgMU7gpwWBCja
         /f9saOSOf+2wBxooWmuMH8a94ghEcqGeC6w6bWNeskJx6Es7c+9xLhxYi36QAzCfJAkA
         TOursqywqULOQfFSLLectDAMIS6wKFAOEbspyNT/z3OAdVX8I3jG8/3Kw2p+tErd9TTo
         ymjDWldRtsjQ1KdclUgUo8CnzpL6jU4UnZRBms3tuBZCQ6TGs3lilGM2n3fEnZ/fCz46
         tEFfGyXICXmwQHSqSJ/XBIaWxT/OjrUFU3mefQ3+QHJFg+0hS18EZEVbjqaOPzO+vXLS
         lGNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714634011; x=1715238811;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Y/eop79hcQEtCziVc8UWDGi7axuiN6CblRErcvnW9g0=;
        b=T0F+otUdxGbPfE+Q2Ltwyp1DkA2tHHYH5nw3PphsJvwFVHI9nxtYRo+x0PXE82FBxT
         iUEro/KpYCbfD/NHEoOJV7REWUSpm4wknhbk9Ow9toUvfeZfzZNu+2UAiPau9amvYhAM
         T6eZ6a6jiKOzKUGYz5n1BKlJJ5dmNxZ4MSwMyXP+VJmvmilU7ErsBQ98Grf1/Zw58ult
         BGsdEtJIKRX8spZjeSIXo9yfhMN9cQ93xmkOunReT5GWy9BKTbfQB3uMBjrpSm58AwM0
         vebm2F9ZGCmK46RZOJTY1ruxepvWw3a0tM6ZW1KrMJoVxNInxirjo1ebBzIvsSm76PyZ
         ODOw==
X-Forwarded-Encrypted: i=1; AJvYcCXjuhU6IoAYgVTnlm6mUP9vyMYzLaixo1EDCyQ1amSDkfgySurwF73k9Phzuq6DV+Cme/BBvlXOkL/TX449BtfNPBg3woQWWgNes1t2Qkk=
X-Gm-Message-State: AOJu0Yycdt15uMg/C/aWZbVFDOe/42A5sSMmpfqwUF9YGb252DzG+qqh
	HDnFO4FAbDC/wpngbXlPMvNOPDpIQ1y9QHZOhHTLmKtmxIEHHYUzSh0Mhqo/XA==
X-Google-Smtp-Source: AGHT+IEdOiCmd1C0VD738ZsOsEpZ13iJYrJXJWtv/5wZy5oeMQ4oTCM5NmPlgnC89bXZpM6CXq1gGA==
X-Received: by 2002:a05:600c:4f06:b0:416:8efd:1645 with SMTP id l6-20020a05600c4f0600b004168efd1645mr4014043wmq.7.1714634011199;
        Thu, 02 May 2024 00:13:31 -0700 (PDT)
Message-ID: <1bad8afa-68a5-4bb3-92ee-2daefd36fcb5@suse.com>
Date: Thu, 2 May 2024 09:13:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/6] xen/x86: Add topology generator
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-7-alejandro.vallejo@cloud.com>
 <c3b4bacd-47c8-403c-ab5c-7a3adf39eeeb@suse.com>
 <e17b2cb1-1482-46e1-844f-22be23a7190a@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e17b2cb1-1482-46e1-844f-22be23a7190a@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.05.2024 19:06, Alejandro Vallejo wrote:
> On 26/03/2024 17:02, Jan Beulich wrote:
>> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>>> --- a/tools/include/xenguest.h
>>> +++ b/tools/include/xenguest.h
>>> @@ -843,5 +843,20 @@ enum xc_static_cpu_featuremask {
>>>      XC_FEATUREMASK_HVM_HAP_DEF,
>>>  };
>>>  const uint32_t *xc_get_static_cpu_featuremask(enum xc_static_cpu_featuremask);
>>> +
>>> +/**
>>> + * Synthesise topology information in `p` given high-level constraints
>>> + *
>>> + * Topology is given in various fields accross several leaves, some of
>>> + * which are vendor-specific. This function uses the policy itself to
>>> + * derive such leaves from threads/core and cores/package.
>>> + *
>>> + * @param p                   CPU policy of the domain.
>>> + * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
>>> + * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
>>> + */
>>> +void xc_topo_from_parts(struct cpu_policy *p,
>>> +                        uint32_t threads_per_core, uint32_t cores_per_pkg);
>>
>> Do we really want to constrain things to just two (or in fact any fixed number
>> of) levels? Iirc on AMD there already can be up to 4.
> 
> For the time being, I think we should keep it simple(ish).

Perhaps, with (briefly) stating the reason(s) for doing so.

>>> +void xc_topo_from_parts(struct cpu_policy *p,
>>> +                        uint32_t threads_per_core, uint32_t cores_per_pkg)
>>> +{
>>> +    uint32_t threads_per_pkg = threads_per_core * cores_per_pkg;
>>> +    uint32_t apic_id_size;
>>> +
>>> +    if ( p->basic.max_leaf < 0xb )
>>> +        p->basic.max_leaf = 0xb;
>>> +
>>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>>> +
>>> +    /* thread level */
>>> +    p->topo.subleaf[0].nr_logical = threads_per_core;
>>> +    p->topo.subleaf[0].id_shift = 0;
>>> +    p->topo.subleaf[0].level = 0;
>>> +    p->topo.subleaf[0].type = 1;
>>> +    if ( threads_per_core > 1 )
>>> +        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
>>> +
>>> +    /* core level */
>>> +    p->topo.subleaf[1].nr_logical = cores_per_pkg;
>>> +    if ( p->x86_vendor == X86_VENDOR_INTEL )
>>> +        p->topo.subleaf[1].nr_logical = threads_per_pkg;
>>
>> Same concern as in the other patch regarding "== Intel".
> 
> Can you please articulate the concern?

See my replies to patch 5. Exactly the same applies here.

>>> +    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
>>> +    p->topo.subleaf[1].level = 1;
>>> +    p->topo.subleaf[1].type = 2;
>>> +    if ( cores_per_pkg > 1 )
>>> +        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
>>
>> Don't you want to return an error when any of the X_per_Y values is 0?
> 
> I'd rather not.
> 
> The checks on input parameters should be done wherever the inputs are
> taken from. Currently the call site passes threads_per_core=1 and
> cores_per_pkg=1+max_vcpus, so it's all guaranteed to work out.

Hmm, and then have this function produce potentially bogus results?

> Once it comes from xl, libxl should be in charge of the validations.
> Furthermore there's validations the function simply cannot do (nor
> should it) in its current form, like checking that...
> 
>     max_vcpus == threads_per_core * cores_per_pkg * n_pkgs.

But that isn't an equality that needs to hold, is it? It would put
constraints on exposing e.g. HT to a guest with an odd number of
vCPU-s. Imo especially with core scheduling HT-ness wants properly
reflecting from underlying hardware, so core-sched-like behavior in
the guest itself can actually achieve its goals.

>>> +    apic_id_size = p->topo.subleaf[1].id_shift;
>>> +
>>> +    /*
>>> +     * Contrary to what the name might seem to imply. HTT is an enabler for
>>> +     * SMP and there's no harm in setting it even with a single vCPU.
>>> +     */
>>> +    p->basic.htt = true;
>>> +
>>> +    p->basic.lppp = 0xff;
>>> +    if ( threads_per_pkg < 0xff )
>>> +        p->basic.lppp = threads_per_pkg;
>>> +
>>> +    switch ( p->x86_vendor )
>>> +    {
>>> +        case X86_VENDOR_INTEL:
>>> +            struct cpuid_cache_leaf *sl = p->cache.subleaf;
>>> +            for ( size_t i = 0; sl->type &&
>>> +                                i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
>>> +            {
>>> +                sl->cores_per_package = cores_per_pkg - 1;
>>> +                sl->threads_per_cache = threads_per_core - 1;
>>
>> IOW the names in struct cpuid_cache_leaf aren't quite correct.
> 
> Because of the - 1, you mean?

Yes. The expressions above simply read as if there was an (obvious)
off-by-1.

> If anything our name is marginally clearer
> than the SDM description. It goes on to say "Add one to the return value
> to get the result" in a [**] note, so it's not something we made up.
> 
> Xen: threads_per_cache => SDM: Maximum number of addressable IDs for
> logical processors sharing this cache
> 
> Xen: cores_per_package => SDM: Maximum number of addressable IDs for
> processor cores in the physical package

I'm afraid I don't follow what you're trying to justify here. Following
SDM naming is generally advisable, yes, but only as far as no confusion
results. Otherwise imo a better name wants picking, with the struct
field declaration then accompanied by a comment clarifying the
difference wrt SDM.

>>> --- a/xen/arch/x86/cpu-policy.c
>>> +++ b/xen/arch/x86/cpu-policy.c
>>> @@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
>>>  
>>>      p->basic.raw[0x8] = EMPTY_LEAF;
>>>  
>>> -    /* TODO: Rework topology logic. */
>>> -    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>>> -
>>>      p->basic.raw[0xc] = EMPTY_LEAF;
>>>  
>>>      p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
>>> @@ -387,6 +384,9 @@ static void __init calculate_host_policy(void)
>>>      recalculate_xstate(p);
>>>      recalculate_misc(p);
>>>  
>>> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
>>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>>
>> I don't think this should be zapped from the host policy. It wants zapping
>> from the guest ones instead, imo. The host policy may (will) want using in
>> Xen itself, and hence it should reflect reality.
> 
> Shouldn't Xen be checking its own cached state from the raw policy
> instead? My understanding was that to a first approximation the host
> policy is a template for guest creation. It already has a bunch of
> overrides that don't match the real hardware configuration.

No, raw policy is what comes from hardware, entirely unadjusted for
e.g. command line options of Xen internal restrictions. Any decisions
Xen takes for itself should be based on the host policy (whereas guest
related decisions are to use the respective guest policy).

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 07:22:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:22:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715594.1117340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qm0-0003df-2i; Thu, 02 May 2024 07:22:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715594.1117340; Thu, 02 May 2024 07:22:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qlz-0003dY-Uq; Thu, 02 May 2024 07:22:31 +0000
Received: by outflank-mailman (input) for mailman id 715594;
 Thu, 02 May 2024 07:22:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2Qly-0003dS-J2
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:22:30 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bb2af934-0854-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:22:28 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-5709cb80b03so8049364a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:22:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb2af934-0854-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714634548; x=1715239348; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TBJFJ5wMx1BtKIT2P03my37zyaWn91PkqyxuO5aXPNk=;
        b=d66o55N8FW/h6DzzLHctEX1ZE9oIcWra8yzPIHSlvddSJHNqScbDcSMD9Pi3b+VKuK
         O3fvy9RMTYdczM8SOvBCArVSPapQc+VTl9BPEXJtRJuvc7tFi111wGoeYVWC9OFINxpu
         54VldijEgQH5K+RQfGiB7bDBxul+q5NDTgAziV95TAUoyeMh+DmBeslJhQOkfqzoUP3G
         FOfYM+0OTzoIM1RhnVIkF0ddJ7+8pnUv1oT+CDugZngOnpXVZL2idjS8igEVyz/m5768
         Sj6VRX2SCeHHlSL/g/+zjPrPHxOIKpRth+OnSAqp4AZkvnsFjxJu7XK2LuPUCrGCELT/
         GJWw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714634548; x=1715239348;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=TBJFJ5wMx1BtKIT2P03my37zyaWn91PkqyxuO5aXPNk=;
        b=a/83VWUMX62T9Ph5IeRHfi5ZIQHjZ3m6XjJcumGwdZSXzvT5LcZahXnsSihUL1FKqc
         jnrbU6R1Una3OFf6vkdySI1IR0DRXzGdd+mHXoKet4Hu/FKt7f2/B6Ho50khOvuN9UbZ
         2Gqh2XnJIdEUbMQvieDs7ebQm7Bz5b29dZT6fcUhZjCBXn6wCRFjDVm0CBlRFz6kQnpd
         //9EyoHiYLIxzR8X46EizO+IFgZw+9qNCpc7qklBZTu4da5urKC9nuX1s81Iu58zrhHO
         rHTNg5TDDuz7MkmLiN4wHq/3PTCBj2ScsIpciV7xZebhGIcbOvs6fB0FG70WKOiIf5ff
         Hb7Q==
X-Forwarded-Encrypted: i=1; AJvYcCUSFlwGm5riNo+YI66sEgwn+XP/h8K+xPr5zqG3WL3OotlylbSfqDETupfxJq+PmGHeUlgU5kstaNN2AF3YFlvQi/3rzb+h/E5UUSbGWqE=
X-Gm-Message-State: AOJu0Yx9hSX3yAnKklYfKb2igCBQPS3iEJ6hv2B6VN+e/2YnAJl5Xhb6
	w468pErX0v4/oX09x88opLx5CwmPPusLNIweREs3vM9oruS3BmJgp0YtyIT2+++AaJ5N/lXbWUS
	otp7XfesEMf3jyafdV2wn+yXTuMQ=
X-Google-Smtp-Source: AGHT+IHSlb8IZgbXfdiILZJA/hvGebr0LK4P+w6/TsM+AJ6nYP9B3xdfVB6EUd55X8OzdH8lqoW18BT4TFiV4dMG2EU=
X-Received: by 2002:a50:bb26:0:b0:572:4faf:ed68 with SMTP id
 y35-20020a50bb26000000b005724fafed68mr3115003ede.27.1714634547567; Thu, 02
 May 2024 00:22:27 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-16-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 09:22:15 +0200
Message-ID: <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com>
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
	David Hildenbrand <david@redhat.com>, =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 1, 2024 at 11:24=E2=80=AFPM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > The current mapcache assumes that all memory is mapped
> > in a single RAM MR (the first one with offset 0). Remove
> > this assumption and propagate the offset to the mapcache
> > so it can do reverse mappings (from hostptr -> ram_addr).
> >
> > This is in preparation for adding grant mappings.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>
>
> Looking at xen_remap_bucket, it is only using address_index (without
> adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
> understand that address_index already includes the ram_offset.
>
> Meaning that if we want to map foreign mapping at address 0x5000, then
> address_index would be 0x5000, even if ram_offset is 0x1000.
>
> But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> to paddr_index to calculate the physical address. So in that case we
> would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> xen_remap_bucket would have to sum address_index and ram_offset to map
> foreign memory.
>
> So I am a bit confused, did I get it wrong? One more comment below.
>

Thanks Stefano,

I think the confusion is that this ram_addr_offset is not related to
guest address-space.
It's a QEMU internal thing and it shouldn't be included in the address
used to map foreign memory.
The mapcache can treat this ram_addr offset like a cookie that we keep
around to be able to do
reverse mappings from host pointers into ram_addr space
(xen_ram_addr_from_mapcache).

The current mapcache implementation works because we've really only
been using foreign mappings
on RAMBlocks with offset 0. We're also creating RAM's such that the
offset into the RAM is also
the guest physical address, for x86 this is natural since RAM starts
at zero (for lowmem) but for
ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size) to
make this assumption true. Anyway, In this series I'm not addressing
this second assumption.

There's a second call in physmem.c to xen_map_cache using the
block->offset as an address.
I was considering removing that second call since I can't see how it can wo=
rk
(except perhaps in some specific use-case by luck?). Anyway, for now
I've left it unmodified.


>
> > ---
> >  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
> >  include/sysemu/xen-mapcache.h |  2 ++
> >  system/physmem.c              |  8 ++++----
> >  3 files changed, 24 insertions(+), 11 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index 09b5f36d9c..1b32d0c003 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> >      uint8_t flags;
> >      hwaddr size;
> > +
> > +    /* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr)=
.  */
> > +    ram_addr_t ram_offset;
> >      struct MapCacheEntry *next;
> >  } MapCacheEntry;
> >
> > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> >                               void *vaddr,
> >                               hwaddr size,
> >                               hwaddr address_index,
> > -                             bool dummy)
> > +                             bool dummy,
> > +                             ram_addr_t ram_offset)
> >  {
> >      uint8_t *vaddr_base;
> >      xen_pfn_t *pfns;
> > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> >      entry->size =3D size;
> >      entry->valid_mapping =3D g_new0(unsigned long,
> >                                    BITS_TO_LONGS(size >> XC_PAGE_SHIFT)=
);
> > +    entry->ram_offset =3D ram_offset;
> >
> >      if (dummy) {
> >          entry->flags |=3D XEN_MAPCACHE_ENTRY_DUMMY;
> > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> >
> >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> >                                         hwaddr phys_addr, hwaddr size,
> > +                                       ram_addr_t ram_offset,
> >                                         uint8_t lock, bool dma, bool is=
_write)
> >  {
> >      MapCacheEntry *entry, *pentry =3D NULL,
> > @@ -335,14 +341,16 @@ tryagain:
> >      if (!entry) {
> >          entry =3D g_new0(MapCacheEntry, 1);
> >          pentry->next =3D entry;
> > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, d=
ummy);
> > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, d=
ummy,
> > +                         ram_offset);
> >      } else if (!entry->lock) {
> >          if (!entry->vaddr_base || entry->paddr_index !=3D address_inde=
x ||
> >                  entry->size !=3D cache_size ||
> >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> >                      test_bit_size >> XC_PAGE_SHIFT,
> >                      entry->valid_mapping)) {
> > -            xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy);
> > +            xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy,
> > +                             ram_offset);
> >          }
> >      }
> >
> > @@ -389,13 +397,15 @@ tryagain:
> >
> >  uint8_t *xen_map_cache(MemoryRegion *mr,
> >                         hwaddr phys_addr, hwaddr size,
> > +                       ram_addr_t ram_addr_offset,
> >                         uint8_t lock, bool dma,
> >                         bool is_write)
> >  {
> >      uint8_t *p;
> >
> >      mapcache_lock(mapcache);
> > -    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma,=
 is_write);
> > +    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_o=
ffset,
> > +                               lock, dma, is_write);
> >      mapcache_unlock(mapcache);
> >      return p;
> >  }
> > @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcache_single=
(MapCache *mc, void *ptr)
> >          raddr =3D RAM_ADDR_INVALID;
> >      } else {
> >          raddr =3D (reventry->paddr_index << mc->bucket_shift) +
> > -             ((unsigned long) ptr - (unsigned long) entry->vaddr_base)=
;
> > +             ((unsigned long) ptr - (unsigned long) entry->vaddr_base)=
 +
> > +             entry->ram_offset;
> >      }
> >      mapcache_unlock(mc);
> >      return raddr;
> > @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(Ma=
pCache *mc,
> >
> >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> >
> > -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> > -                     cache_size, address_index, false);
> > +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> > +                     cache_size, address_index, false, entry->ram_offs=
et);
> >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> >                  test_bit_size >> XC_PAGE_SHIFT,
> >                  entry->valid_mapping)) {
> > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcach=
e.h
> > index 1ec9e66752..b5e3ea1bc0 100644
> > --- a/include/sysemu/xen-mapcache.h
> > +++ b/include/sysemu/xen-mapcache.h
> > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_=
offset,
> >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> >                          void *opaque);
> >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size=
,
> > +                       ram_addr_t ram_addr_offset,
> >                         uint8_t lock, bool dma,
> >                         bool is_write);
> >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_=
gaddr_t f,
> >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> >                                       hwaddr phys_addr,
> >                                       hwaddr size,
> > +                                     ram_addr_t ram_addr_offset,
> >                                       uint8_t lock,
> >                                       bool dma,
> >                                       bool is_write)
> > diff --git a/system/physmem.c b/system/physmem.c
> > index 1a5ffcba2a..5b16eeccca 100644
> > --- a/system/physmem.c
> > +++ b/system/physmem.c
> > @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *bloc=
k, ram_addr_t addr,
> >           * In that case just map the requested area.
> >           */
> >          if (xen_mr_is_memory(block->mr)) {
> > -            return xen_map_cache(block->mr, addr, len, lock, lock,
> > -                                 is_write);
> > +            return xen_map_cache(block->mr, addr, len, block->offset,
> > +                                 lock, lock, is_write);
>
> Have you considered not tracking offset and address separately and
> simply do this?
>
>             return xen_map_cache(block->mr, addr + block->offset, len,
>                                  lock, lock, is_write);
>

Unfortunately this won't work since block->offset is not related to where t=
his
ram is mapped in guest address-space. In the case of grant's, we'd get the
wrong grant ref. See my previous comment.

Cheers,
Edgar


>
> >          }
> >
> >          block->host =3D xen_map_cache(block->mr, block->offset,
> > -                                    block->max_length, 1,
> > -                                    lock, is_write);
> > +                                    block->max_length, 0,
> > +                                    1, lock, is_write);
> >      }
> >
> >      return ramblock_ptr(block, addr);
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Thu May 02 07:23:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:23:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715599.1117349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qn0-0004DA-Da; Thu, 02 May 2024 07:23:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715599.1117349; Thu, 02 May 2024 07:23:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qn0-0004D3-B0; Thu, 02 May 2024 07:23:34 +0000
Received: by outflank-mailman (input) for mailman id 715599;
 Thu, 02 May 2024 07:23:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Qn0-0004Ch-03
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:23:34 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e10f52e3-0854-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:23:31 +0200 (CEST)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-34d7b0dac54so871235f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:23:31 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v28-20020a5d591c000000b0034db47c7e6dsm516329wrd.115.2024.05.02.00.23.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 00:23:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e10f52e3-0854-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714634611; x=1715239411; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jnK5z9LmLAEqd86FdaoU8oE0NAXd83I44namarsqDQ4=;
        b=QNsDS1ghNEW/rPoGOUT3ElF9nYaIG9PsXXUvoEt/LDGJfMD56XU9R93KrU2owjBVaO
         0ZHnKIWwOQj1q0HRT7tPd/e2taooGpwdeL/jE2BMyDbdw5k2z42nmP6FmKyPEdVzt60+
         8hwzdFCuIpcPcK+2FdWPkrJktszppaATSb8sLUXqt7AsLWgRKvtWmX3j/Tlh52XW0VuI
         82S6UQNh7Q+odG1D8A7x2cAbC3EMGvwWOJcCQgGt3hG6MsWeHF4lECmXU7FRRGHCpsSq
         LEBQw84/LstCL1bGiYtDQhRYShNJ+kFFZ22VM/nugsxqzU5ioiKjijD5wNzEmdGt0pEl
         56/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714634611; x=1715239411;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jnK5z9LmLAEqd86FdaoU8oE0NAXd83I44namarsqDQ4=;
        b=wNZVV5IFz2icaUoMg9JR8xbh28o6LfxSWM4AJilwmTxq3+1OYsRsLIafZiRX7W4Dbe
         lGW5iOaKZ2wvMdCJe5+rA07ByoNuNgVk4W5qg9MUXrgl6PFqbq+Rjff2AYdK0HwdclAg
         IBQq2JOv5Yl/WfciQ0MWSQyVvzq31QY3ibHZKPz6VPKbnve2+PTbhXPUonAEQalDcGR0
         dhmtrF14rTU65wGf0i44vSIPYAASsJBqxr+Jd6pv0084C6PgVa3d0e2zElLIqFPMWpPA
         PbCKZspiCtTotHP1z4VINF07VKGsmPRqeLmy6oHqNs9MaGLdO2cub4jQuMVEbn0th9HU
         qQcA==
X-Forwarded-Encrypted: i=1; AJvYcCUlKOBT6To1kquoVQVLE5o4aDhTOu7/HsMCcYIJ1A9VwBKQGitYsssiZwNurChhyK6LrgREpoTwD71nJTLGjnzGelfothCm8YOQnGa5nPM=
X-Gm-Message-State: AOJu0YyyzHQYxVZT8ccjtdysHvK+e4irAEk0plEFEAt6qp3VLjIyeguJ
	1Ysag8ji4Dw12TM+iTTzQr9hjQyO+hFeYhjTVLvzAl6zGilGqylmJeI+md9rHg==
X-Google-Smtp-Source: AGHT+IEl9vSHt9jbD2VL4V1KRu8iHVSUSGBOGxZDil2TWdNNXEygH5DxID2cVj9FaWxfH753FGc3gQ==
X-Received: by 2002:a05:6000:1b0a:b0:34d:c8d4:3e63 with SMTP id f10-20020a0560001b0a00b0034dc8d43e63mr1255902wrz.24.1714634611433;
        Thu, 02 May 2024 00:23:31 -0700 (PDT)
Message-ID: <0f39a067-70d2-4652-910a-5d05db6a3ebc@suse.com>
Date: Thu, 2 May 2024 09:23:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] ppc/riscv: fix arch_acquire_resource_check()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
References: <20240430153423.80875-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430153423.80875-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30.04.2024 17:34, Roger Pau Monne wrote:
> None of the implementations support set_foreign_p2m_entry() yet, neither they
> have a p2m walk in domain_relinquish_resources() in order to remove the foreign
> mappings from the p2m and thus drop the extra refcounts.

While I don't mind the cod adjustment into the more safe direction, I find
this justification odd: RISC-V has no domain_relinquish_resources() at all
right now, and PPC has it properly as a stub only. Judgement on what there
is (or not) can only be made one non-stub implementations exist.

IOW provided PPC and RISC-V people agree, I'm fine putting this in, but
preferably with an adjusted description. To be honest with how you put it,
it's not even really clear to me what (practical) problem, if any, you're
trying to address.

Jan

> Adjust the arch helpers to return false and introduce a comment that clearly
> states it is not only taking extra refcounts that's needed, but also dropping
> them on domain teardown.
> 
> Fixes: 4988704e00d8 ('xen/riscv: introduce p2m.h')
> Fixes: 4a2f68f90930 ('xen/ppc: Define minimal stub headers required for full build')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>  xen/arch/ppc/include/asm/p2m.h   | 7 ++++---
>  xen/arch/riscv/include/asm/p2m.h | 7 ++++---
>  2 files changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/ppc/include/asm/p2m.h b/xen/arch/ppc/include/asm/p2m.h
> index 25ba05466853..f144ef8e1a54 100644
> --- a/xen/arch/ppc/include/asm/p2m.h
> +++ b/xen/arch/ppc/include/asm/p2m.h
> @@ -81,10 +81,11 @@ static inline mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn)
>  static inline bool arch_acquire_resource_check(struct domain *d)
>  {
>      /*
> -     * The reference counting of foreign entries in set_foreign_p2m_entry()
> -     * is supported on PPC.
> +     * Requires refcounting the foreign mappings and walking the p2m on
> +     * teardown in order to remove foreign pages from the p2m and drop the
> +     * extra reference counts.
>       */
> -    return true;
> +    return false;
>  }
>  
>  static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
> diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
> index 87b13f897926..387f372b5d26 100644
> --- a/xen/arch/riscv/include/asm/p2m.h
> +++ b/xen/arch/riscv/include/asm/p2m.h
> @@ -79,10 +79,11 @@ static inline mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn)
>  static inline bool arch_acquire_resource_check(struct domain *d)
>  {
>      /*
> -     * The reference counting of foreign entries in set_foreign_p2m_entry()
> -     * is supported on RISCV.
> +     * Requires refcounting the foreign mappings and walking the p2m on
> +     * teardown in order to remove foreign pages from the p2m and drop the
> +     * extra reference counts.
>       */
> -    return true;
> +    return false;
>  }
>  
>  static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)



From xen-devel-bounces@lists.xenproject.org Thu May 02 07:25:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:25:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715602.1117359 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QoR-00059a-OD; Thu, 02 May 2024 07:25:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715602.1117359; Thu, 02 May 2024 07:25:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2QoR-00059T-LY; Thu, 02 May 2024 07:25:03 +0000
Received: by outflank-mailman (input) for mailman id 715602;
 Thu, 02 May 2024 07:25:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=bu2w=MF=redhat.com=david@srs-se1.protection.inumbo.net>)
 id 1s2QoQ-00059J-K7
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:25:02 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.129.124])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 154b599d-0855-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:25:00 +0200 (CEST)
Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com
 [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-175-_dQ6MOkJMKu0f5nOAR6SMA-1; Thu, 02 May 2024 03:24:57 -0400
Received: by mail-wm1-f70.google.com with SMTP id
 5b1f17b1804b1-418f18458a0so38863165e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:24:57 -0700 (PDT)
Received: from ?IPV6:2003:cb:c71e:bf00:eba1:3ab9:ab0f:d676?
 (p200300cbc71ebf00eba13ab9ab0fd676.dip0.t-ipconnect.de.
 [2003:cb:c71e:bf00:eba1:3ab9:ab0f:d676])
 by smtp.gmail.com with ESMTPSA id
 z5-20020a05600c0a0500b0041bd85cd3f2sm917097wmp.19.2024.05.02.00.24.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 00:24:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 154b599d-0855-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1714634699;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=cVr6/WyBfNaOaFW1NnhNL8WBGrtCboTC6NZ9VaDLEH0=;
	b=MGwBKlspzKNPa+WThiShe+0pRxnDeU6m/g0MGfyvy7//ZEuIkSyqUZxFEdRzeRe3X+sDWd
	vmGOByY64h/4p7JYfEsfNpX3tZKPjLvhCPykA4YHX5uQdm/GCe2WCzlAuG3ZRddK/ayQPN
	A4hz+aMp+vVL9YDQq5av32dgNvI7iAE=
X-MC-Unique: _dQ6MOkJMKu0f5nOAR6SMA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714634696; x=1715239496;
        h=content-transfer-encoding:in-reply-to:organization:autocrypt
         :content-language:from:references:cc:to:subject:user-agent
         :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=cVr6/WyBfNaOaFW1NnhNL8WBGrtCboTC6NZ9VaDLEH0=;
        b=u+T4SoIeBNm3OwBt1E3JUUeyAljP5v5uZVfV+KCF1paeQBnml6WMAD2Y8ZZ2hqYKUN
         +RycuI7SUiqEwTYQuajXpzcO+kAGjzAY6f0fs0h573pppSaNWkg7kh7gzBOm8LJPpdV9
         9MEal65kK/X3vaw8OCxzzB/b7u+WRQLEjgOEtctKisbqw90CjEpibmUTKsgNo0JONTJq
         acgVvliqXXhgXfHQDafZ4N52f8R6HBR3zsoX2zqmF751CpJRIWDtSGlzwi+0NsVImOR4
         2rPCFD7q8y7wRLzxmfN0i2CaFovyCYib87UKyIANo0EaITqjEZdQxEAmExpjohhYPrET
         x3Tg==
X-Forwarded-Encrypted: i=1; AJvYcCUhJFaL52CBauLbvbrk3A2MREqFdh4F3lkfEZ2FXjmsMRcxvJrlNw4HM5a3/UuwKznIBPIsy17e5+EdOWrfDuPk/YSRet7+45QLjrk6Qus=
X-Gm-Message-State: AOJu0Yz6ElnLgT3b1yh4DztqoFGyYNQ1INXpyO3oBSKSqdIvuWa3CDs7
	+y5UdHHCwb+PrEZac9XA3kSOpS91oU9/AA33Vf/h3VtZ+dJoSrhTZk4RKJ/MLUBgMQbE5Dn49hl
	mIhA24LS5ZnpRoxw4l9GQMSU42P/ZA3Hogrv7xnhPOGQVC4+hv6CVz6KUvPOl8287
X-Received: by 2002:a05:600c:46cc:b0:41b:eaf2:f7da with SMTP id q12-20020a05600c46cc00b0041beaf2f7damr1051581wmo.6.1714634696493;
        Thu, 02 May 2024 00:24:56 -0700 (PDT)
X-Google-Smtp-Source: AGHT+IHAqD4icNunxfUpJGhVol0TOZfVPB7oj7bLkW3rBwu0G9HXbzU4JQedlaQW2eoh7DwWWRwQKA==
X-Received: by 2002:a05:600c:46cc:b0:41b:eaf2:f7da with SMTP id q12-20020a05600c46cc00b0041beaf2f7damr1051563wmo.6.1714634696110;
        Thu, 02 May 2024 00:24:56 -0700 (PDT)
Message-ID: <7172c93c-8c25-4934-b1d8-de0658c6f50f@redhat.com>
Date: Thu, 2 May 2024 09:24:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write
 xen_map_cache()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-14-edgar.iglesias@gmail.com>
From: David Hildenbrand <david@redhat.com>
Autocrypt: addr=david@redhat.com; keydata=
 xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ
 dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL
 QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp
 XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK
 Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9
 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt
 WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc
 UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv
 jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb
 B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk
 ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW
 AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q
 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp
 rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf
 wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4
 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l
 pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd
 KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE
 BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs
 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF
 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9
 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz
 Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb
 T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A
 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk
 CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G
 NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75
 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx
 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS
 lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv
 AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa
 N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3
 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB
 boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq
 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f
 XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ
 a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq
 Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6
 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8
 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E
 th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr
 jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt
 WNyWQQ==
Organization: Red Hat
In-Reply-To: <20240430164939.925307-14-edgar.iglesias@gmail.com>
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 30.04.24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Propagate MR and is_write to xen_map_cache().

I'm pretty sure the patch subject is missing a "to" :)

> This is in preparation for adding support for grant mappings.
> 
> No functional change.
> 

Reviewed-by: David Hildenbrand <david@redhat.com>
-- 
Cheers,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Thu May 02 07:26:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:26:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715608.1117369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qpu-000627-2f; Thu, 02 May 2024 07:26:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715608.1117369; Thu, 02 May 2024 07:26:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qpt-000620-Ve; Thu, 02 May 2024 07:26:33 +0000
Received: by outflank-mailman (input) for mailman id 715608;
 Thu, 02 May 2024 07:26:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=bu2w=MF=redhat.com=david@srs-se1.protection.inumbo.net>)
 id 1s2Qpt-0005vQ-0X
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:26:33 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.133.124])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4b802b9a-0855-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:26:31 +0200 (CEST)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-594-nWDx02wtN0uF5tcpReHdTA-1; Thu, 02 May 2024 03:26:29 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 5b1f17b1804b1-416ac21981dso7716765e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:26:28 -0700 (PDT)
Received: from ?IPV6:2003:cb:c71e:bf00:eba1:3ab9:ab0f:d676?
 (p200300cbc71ebf00eba13ab9ab0fd676.dip0.t-ipconnect.de.
 [2003:cb:c71e:bf00:eba1:3ab9:ab0f:d676])
 by smtp.gmail.com with ESMTPSA id
 i16-20020a05600c355000b0041b61504565sm933209wmq.28.2024.05.02.00.26.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 00:26:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b802b9a-0855-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1714634790;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=3FnHssCUF+JogpfMvQcIC1XSfS0ftTV0t9WTQ+Vga4k=;
	b=QpZfy/dxqPqt6alTqZ7cRcBgSKOa/wIVp4zgxBLsdobf1TtFWCtERispbujh8Jw23mrLVM
	278bIiY27QGvbB2stVmYpIBq63fs/Y6V9S3DV8ajfznva5kMdcYwe/8XbpY5CiZCkr+ykV
	aCj2S7PomJt5e7FNJRfmJTPFO4LkWgg=
X-MC-Unique: nWDx02wtN0uF5tcpReHdTA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714634788; x=1715239588;
        h=content-transfer-encoding:in-reply-to:organization:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3FnHssCUF+JogpfMvQcIC1XSfS0ftTV0t9WTQ+Vga4k=;
        b=NMA5r0qdn/KNVLz8Mw2Gib+6Lj9HwAVSMV5N9Q23a4QKgLa1Y8UhL/e9ioU4ElHtPu
         A9iiLP0laePUFde3zLlYgMwyocYzQUz5BmooxO6RzWyit0C6Dhx5etucaIumoN6pqAxd
         HIB7iFjLdw5Wx0A/w5Wo+qN321oL/JC2atxPNPFG9lyOUT9RJacWKr/vu6K6LoMscfnu
         wR9f2lHgL9bZ4MuYmL3bt10K/2lBcSWjPVWIkRN97dwzfu0p9K3BDW/njwyZhwFk9+ht
         9dTdJLdoCxJI8DOH9GTxoIYWizuXqyw9cYNKrFRXOz0/LZ8Xq3DyjX9lUQ2egJod4eft
         +LLA==
X-Forwarded-Encrypted: i=1; AJvYcCVhDhGgYRi0XUzwA3HtpL3yk/EsFtvHYKPSMKVku9RUmP/+dy3L3GGPGqOw4vvXtprZcsV09H4kBBQVbseAOK1+MIJmOXA3e6luPUYc8zQ=
X-Gm-Message-State: AOJu0YzQ2kUkjfkJzUf52S6KBikvnafcJ0zSl61c2GVzsw8BnzP5bRnq
	RGOnA6S4378DZyjUpNB+HImE5XIm/IjnUO1LQdvDU7A59Gu+axDJeuY70uKM4uiPI4Ol0xNjjjj
	FWqpWsPdPkF/7XaiPKoAi6cCv2vqi5x7EoZvmsh0ycmJE7C58mzmdOmAm6D4I2aP/
X-Received: by 2002:a05:600c:474b:b0:419:f533:ef88 with SMTP id w11-20020a05600c474b00b00419f533ef88mr1517961wmo.11.1714634787591;
        Thu, 02 May 2024 00:26:27 -0700 (PDT)
X-Google-Smtp-Source: AGHT+IGCDfTji+hFLp33zSZml2ea3sk683NpvOdxSS6BPcqoE+MBESg6R44Zwy0plsW/UyhT56xFcg==
X-Received: by 2002:a05:600c:474b:b0:419:f533:ef88 with SMTP id w11-20020a05600c474b00b00419f533ef88mr1517933wmo.11.1714634787077;
        Thu, 02 May 2024 00:26:27 -0700 (PDT)
Message-ID: <0263b7e8-5800-4f5a-9dc5-bc2b4fbbbbfe@redhat.com>
Date: Thu, 2 May 2024 09:26:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/17] xen: Add xen_mr_is_memory()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-15-edgar.iglesias@gmail.com>
From: David Hildenbrand <david@redhat.com>
Autocrypt: addr=david@redhat.com; keydata=
 xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ
 dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL
 QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp
 XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK
 Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9
 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt
 WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc
 UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv
 jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb
 B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk
 ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW
 AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q
 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp
 rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf
 wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4
 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l
 pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd
 KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE
 BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs
 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF
 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9
 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz
 Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb
 T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A
 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk
 CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G
 NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75
 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx
 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS
 lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv
 AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa
 N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3
 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB
 boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq
 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f
 XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ
 a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq
 Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6
 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8
 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E
 th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr
 jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt
 WNyWQQ==
Organization: Red Hat
In-Reply-To: <20240430164939.925307-15-edgar.iglesias@gmail.com>
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 30.04.24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add xen_mr_is_memory() to abstract away tests for the
> xen_memory MR.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---

[...]

>   #endif
> diff --git a/system/physmem.c b/system/physmem.c
> index ad7a8c7d95..1a5ffcba2a 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>            * because we don't want to map the entire memory in QEMU.
>            * In that case just map the requested area.
>            */
> -        if (block->offset == 0) {
> +        if (xen_mr_is_memory(block->mr)) {
>               return xen_map_cache(block->mr, addr, len, lock, lock,
>                                    is_write);
>           }

I'd have moved that into a separate patch, because this is not a simple 
abstraction here.

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Thu May 02 07:31:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:31:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715615.1117379 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Quj-0007bT-L8; Thu, 02 May 2024 07:31:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715615.1117379; Thu, 02 May 2024 07:31:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Quj-0007bM-IF; Thu, 02 May 2024 07:31:33 +0000
Received: by outflank-mailman (input) for mailman id 715615;
 Thu, 02 May 2024 07:31:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2Qui-0007bG-CO
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:31:32 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fe395625-0855-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:31:30 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5727dc6d3edso4852590a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:31:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe395625-0855-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714635090; x=1715239890; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=J6K+vQgBEJa5RBagBtZnrylNM7yhfiFaLALvIlNSgfg=;
        b=Mn7lzZ9nslwo3JyfW91cLREteIKC6kOkSfwBAXnIkyWazs7id3dRizsfuJCuwOHf0t
         tPyEnnyKx53PwnPVynjpijYiZ3UHE4hS5+hq+VKcHE3bakLaJ2+KeunoEdNBBm51OW2W
         XrV5dWLt36KRVgGXp3G1wbcCV7TlLY3Y88JJns8fiW5vizICmKXzvhG7YxgQ1NkDgah/
         uwVsJhxyzQ6hFrUtE/25LuJBEKbHXcz6Y5zxQnwTbCbZMBmzjMOTEaylBfvdLmKAdWNq
         UktakWJ/S4KOrFXtas+HtThDUqDI4px6MwPX61JhYzHJdwOpcK8iR8/c40Ty9rEdZSHQ
         ixZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714635090; x=1715239890;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=J6K+vQgBEJa5RBagBtZnrylNM7yhfiFaLALvIlNSgfg=;
        b=BHHWWPgit8IZqKGegdiTwM/1ivtnKVBxntEha2LDz1aBxqHPOflEJX6QSjjKom4t72
         80irLIo1m/F00+pIraj5O1I1LygE8sEUpzP+tHHHzU3FPPxOenfwKjktOSBqCbOCZbM1
         sUW1r89jWDWrcc+EPvKYFg/AJ2ZmeVamWlRC7o8QU8L8BbZI0orkLdTSZAlwjiPK3/sA
         nU6/OO04SEARJ8E4bFY+Hx8UtoAMf1/P2eAqciQe7klbYC5swoVwAODdvAslzCi8rQ30
         hmO22bHp7s4dVStnHEnGsGUmZBHd2xhbYeb0LTZcK2ZCLngiXKfiCncuZ6f2aYP7jrCH
         sHQA==
X-Forwarded-Encrypted: i=1; AJvYcCXCeFuRRrHy0h8r1OgjQOSvc7yWeyX3vpmXMPEdxOlGqO1hlTJwNG3hwHfoLGA8HjtWUG6hoyOF4ma8ycOERPjUE4gRzYHh4TGT/jV3GQg=
X-Gm-Message-State: AOJu0YzYY3afPEiRNqKpzhh9zgeivCm7vxzNaYVt51GfLftqk7KLeZq0
	9hAsRLeT4oTg/rPkaoL0huq75ujeylyFDNDIIrE96iqYwpIp3NLxy3I6k/RH4TAM7QyeDWm6vI5
	4AROzM0KYoxzkJwmSgcimUqjS0aE=
X-Google-Smtp-Source: AGHT+IGMHAiiyj/igyCydtEDbXlM1aLJwGa5/OER2+/iX842FcrCPNtsDsqsmkDTMJpWU8WX/JeJCxlIf+/12u8L+4k=
X-Received: by 2002:a50:aade:0:b0:571:c0e6:233c with SMTP id
 r30-20020a50aade000000b00571c0e6233cmr3365370edc.37.1714635089557; Thu, 02
 May 2024 00:31:29 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-14-edgar.iglesias@gmail.com> <7172c93c-8c25-4934-b1d8-de0658c6f50f@redhat.com>
In-Reply-To: <7172c93c-8c25-4934-b1d8-de0658c6f50f@redhat.com>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 09:31:17 +0200
Message-ID: <CAJy5ezrP0OQU7ZOfZUDg2QgVZ+zcicrACk=UoL68BWEhOaQWsQ@mail.gmail.com>
Subject: Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write xen_map_cache()
To: David Hildenbrand <david@redhat.com>
Cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
	=?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 2, 2024 at 9:24=E2=80=AFAM David Hildenbrand <david@redhat.com>=
 wrote:
>
> On 30.04.24 18:49, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Propagate MR and is_write to xen_map_cache().
>
> I'm pretty sure the patch subject is missing a "to" :)

Thanks David! I'll fix it in v5!

Cheers,
Edgar


>
> > This is in preparation for adding support for grant mappings.
> >
> > No functional change.
> >
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> --
> Cheers,
>
> David / dhildenb
>


From xen-devel-bounces@lists.xenproject.org Thu May 02 07:35:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:35:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715619.1117389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qy9-0000rj-8R; Thu, 02 May 2024 07:35:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715619.1117389; Thu, 02 May 2024 07:35:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qy9-0000rc-5Y; Thu, 02 May 2024 07:35:05 +0000
Received: by outflank-mailman (input) for mailman id 715619;
 Thu, 02 May 2024 07:35:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2Qy7-0000rW-J5
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:35:03 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7b84a9ee-0856-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 09:35:00 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-572adaa172cso2103138a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:35:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b84a9ee-0856-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714635300; x=1715240100; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AlUMOnUMKElu43V8N8NXHih/16FRYsa+DT2aHHqwJNE=;
        b=fxtLdDno0cNzUZmJ9P2LIj3H61O5CM5oACEUZLdkft/cOH3ZBVTTqwykGdX8GbRku/
         Eu7n6Wl8xCNIlksgCJHy1l9Fvd0HegENm1RnnIt+4LxdP3RDQXb0QBNgIOnicn97M3pd
         38Yr3QpiGDdnb36tBdO0GxvS0y6j2YD7J/3ke+yaYzvhZPfiN0+F4PgFs9oSMabveaaW
         pXpUBqTm/ubQVac41TjS5Rati1heyU8uCFC9iLCBwedKlzsT44k+edbCk+WMvYt1Ij1Y
         CGPsE5CfC0uejkPJz3i+TG+IALoPPase0HYBTQptVHEMJdqbeYfQBAMXPkx3vER5SMLI
         wV4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714635300; x=1715240100;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=AlUMOnUMKElu43V8N8NXHih/16FRYsa+DT2aHHqwJNE=;
        b=p2ZBclRMeVRFZkiWxFWYM2ZG02hnLYEae8KytW/SBb9+Fm6c+o9WCyf90W3V0uvYz/
         8shM1feKljturQB6TZaTzSuBe3S4T9j5sKX88sNoOMPXnV4bzPIj+1D5gDfnBOB+J295
         sKgJSBRfC71e4z/G818rlN+6Rf7SQRI/luKjon51wUqKwZmwpZIq5k8J/NZN3oLAIIhz
         x9Dd7njyAYGuN8GS44rmtybwa/0rUAlsKfHHE7RIe3glrFywitwzhI0VRaNKzK7XZZcS
         Xjzkk79xoAd5PMAIPu/4svCgNpKBuMt86SVfeZgMGd6/uOzo5xDXIuKgXeo73Ldh9coy
         E0lw==
X-Forwarded-Encrypted: i=1; AJvYcCV0/+wCaOutlikvH9wat9OPkqr0mihHD2Ms3F9Lqlly/PCqasUcA03TOI0e6Ks5Oz90JTQF0Gt/W7GyaCS10VVZhPs1vLU4KYdxxzuiG/k=
X-Gm-Message-State: AOJu0Yz4FFLXi0tWotea2NjSZyFbAgPa8FN+UsSSSi747bQ0yb7pLaXq
	qSgHH41+LAkO6zN4pVVrVCl4+rFtWwUXygmHNEFaxmP+LVl0BNnU+v5cZGlI9H0KOG0DjSZnTAd
	ZO2TBS/frD8fQ9BghsUsv111dEZw=
X-Google-Smtp-Source: AGHT+IFeFt+Pbra3NifN8rs1UXcD8UP6ZmjUZHfOEsk+O2tSlQVe5lCGAJsT/L6+37rrO+XVhxWqsOTvvoJ6Jvn3MsI=
X-Received: by 2002:a50:cd53:0:b0:570:4aea:4e1e with SMTP id
 d19-20020a50cd53000000b005704aea4e1emr1859829edj.15.1714635299936; Thu, 02
 May 2024 00:34:59 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-13-edgar.iglesias@gmail.com>
In-Reply-To: <20240430164939.925307-13-edgar.iglesias@gmail.com>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 09:34:47 +0200
Message-ID: <CAJy5ezrJ2YYgAXFhcyAVosnKL3a5E96oJCfkuS4y2OGSxdU0Fw@mail.gmail.com>
Subject: Re: [PATCH v4 12/17] xen: mapcache: Unmap first entries in buckets
To: qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Apr 30, 2024 at 6:50=E2=80=AFPM Edgar E. Iglesias
<edgar.iglesias@gmail.com> wrote:
>
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>
> When invalidating memory ranges, if we happen to hit the first
> entry in a bucket we were never unmapping it. This was harmless
> for foreign mappings but now that we're looking to reuse the
> mapcache for transient grant mappings, we must unmap entries
> when invalidated.
>
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  hw/xen/xen-mapcache.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 4f98d284dd..0365311788 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -486,18 +486,22 @@ static void xen_invalidate_map_cache_entry_unlocked=
(MapCache *mc,
>          return;
>      }
>      entry->lock--;
> -    if (entry->lock > 0 || pentry =3D=3D NULL) {
> +    if (entry->lock > 0) {
>          return;
>      }
>
> -    pentry->next =3D entry->next;
>      ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size)=
;
>      if (munmap(entry->vaddr_base, entry->size) !=3D 0) {
>          perror("unmap fails");
>          exit(-1);
>      }
> -    g_free(entry->valid_mapping);
> -    g_free(entry);
> +    if (pentry) {
> +        pentry->next =3D entry->next;
> +        g_free(entry->valid_mapping);
> +        g_free(entry);
> +    } else {
> +        memset(entry, 0, sizeof *entry);

I noticed that we're leaking entry->valid_mapping here. I'll fix this for v=
5.

Cheers,
Edgar


> +    }
>  }
>
>  typedef struct XenMapCacheData {
> --
> 2.40.1
>


From xen-devel-bounces@lists.xenproject.org Thu May 02 07:35:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 07:35:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715622.1117399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qyf-0001K1-Gk; Thu, 02 May 2024 07:35:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715622.1117399; Thu, 02 May 2024 07:35:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Qyf-0001Ju-DB; Thu, 02 May 2024 07:35:37 +0000
Received: by outflank-mailman (input) for mailman id 715622;
 Thu, 02 May 2024 07:35:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPCJ=MF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2Qye-0001Jd-AT
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 07:35:36 +0000
Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com
 [2607:f8b0:4864:20::b36])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 901d0bcd-0856-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 09:35:35 +0200 (CEST)
Received: by mail-yb1-xb36.google.com with SMTP id
 3f1490d57ef6-de61424f478so2820799276.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 00:35:35 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 g22-20020ac842d6000000b0043c97f30fa7sm225305qtm.77.2024.05.02.00.35.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 00:35:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 901d0bcd-0856-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714635334; x=1715240134; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=7NTaJf/kheUfuUyMCZYF3yIWqHiMxkIdVbpzQ+sC7Hg=;
        b=mTXyct08Jnkzy21vZOShJSUVLlXg2a3v+A4E+h1fvArOLmM8iPjqDCf4fRRhBw4pGN
         G6X0qvzsQm19mAC3Rt0Oav2gwMi+D8KsgxaEZKf6hjq8iwRGwBZyq2sbA0L+9V2TWVhT
         ssuBQEt9dytjXrbNUwhKCxRG6UEA19FoIjrfI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714635334; x=1715240134;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7NTaJf/kheUfuUyMCZYF3yIWqHiMxkIdVbpzQ+sC7Hg=;
        b=TDZW2wPx3jMYdQcCFIk2rNJABqaZd+c/ZTZKL+CUh5xbAJmTvtmQkKp+52uobeA2wN
         3tBV/MHlS5vLDAEOcrGbAtUXiXvnrhCqUVnG1/HGXHeGkY38enTZRqOSM9scK8BCYYZ9
         ujGidHJ48okWlhK8sn03fBpS200Q1gFiWK6ILz5svSqKcVcnM6/3RYoKfd3oghDxfd8x
         avzloS9EX5O36N8lw9X71fXv7atsU7074LTw4KcRuvFE+jv7wJVN8mcGyzfA4ys8DtuH
         fPb+nrEZMufoEg2MymTHiFWydJDZHGdG6kc87/pJwmR5+IS0pd1oufI142NrkLACsZ24
         XspA==
X-Forwarded-Encrypted: i=1; AJvYcCV8hovtxqPRhQ4xYCw4dJkB2w5xreMljulUr23gPVQMToh/9c7vrpYNqGSvI7uv7MG/+qtK/74f/EAkz9u4twYiwb7nnMGFwVhi8I1t5kU=
X-Gm-Message-State: AOJu0YzjvvshZBzDfR+Sal/+bzm+2BuG7hVNHtuJfkdlgoAAOyGCxcVe
	xwaLvsEzP3wdHNMVg1xeSjAKBdCUX005rGOrCvXT8I1m4LDfD5H11mIZu9Cgzys=
X-Google-Smtp-Source: AGHT+IEUqaHnHOBWiYzK60R9TTvVOL06sVbW91eIoJi9LU/GZkBpRs44D+hAYXeEcadIwOPCC1vN/g==
X-Received: by 2002:a05:6902:4e4:b0:dd1:40cf:942b with SMTP id w4-20020a05690204e400b00dd140cf942bmr4517732ybs.48.1714635332477;
        Thu, 02 May 2024 00:35:32 -0700 (PDT)
Date: Thu, 2 May 2024 09:35:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19] ppc/riscv: fix arch_acquire_resource_check()
Message-ID: <ZjNCQiR2uUR5Iz8Q@macbook>
References: <20240430153423.80875-1-roger.pau@citrix.com>
 <0f39a067-70d2-4652-910a-5d05db6a3ebc@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <0f39a067-70d2-4652-910a-5d05db6a3ebc@suse.com>

On Thu, May 02, 2024 at 09:23:30AM +0200, Jan Beulich wrote:
> On 30.04.2024 17:34, Roger Pau Monne wrote:
> > None of the implementations support set_foreign_p2m_entry() yet, neither they
> > have a p2m walk in domain_relinquish_resources() in order to remove the foreign
> > mappings from the p2m and thus drop the extra refcounts.
> 
> While I don't mind the cod adjustment into the more safe direction, I find
> this justification odd: RISC-V has no domain_relinquish_resources() at all
> right now, and PPC has it properly as a stub only. Judgement on what there
> is (or not) can only be made one non-stub implementations exist.

Right, hence stating that foreign mappings are properly handled
(arch_acquire_resource_check() returning true) is bogus to me because
there's no code yet.

> IOW provided PPC and RISC-V people agree, I'm fine putting this in, but
> preferably with an adjusted description. To be honest with how you put it,
> it's not even really clear to me what (practical) problem, if any, you're
> trying to address.

The current statement is at best misleading, because there's no
implementation of set_foreign_p2m_entry() or
domain_relinquish_resources(), and hence making claims that future
implementation of them will properly handle foreign mappings could
lead to the special requirements of those mappings not being taken
into account when implementing those functions just because
arch_acquire_resource_check() already returns true.

IMO arch_acquire_resource_check() can only return true once the code
is in place, and mappings are properly handled.  Making claims about
yet to be implemented code is wrong.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 02 08:14:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:14:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715641.1117409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2RZq-0001E8-PC; Thu, 02 May 2024 08:14:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715641.1117409; Thu, 02 May 2024 08:14:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2RZq-0001E1-Ly; Thu, 02 May 2024 08:14:02 +0000
Received: by outflank-mailman (input) for mailman id 715641;
 Thu, 02 May 2024 08:14:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=A7KU=MF=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s2RZp-0001Dv-Ag
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:14:01 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01on0624.outbound.protection.outlook.com
 [2a01:111:f400:fe02::624])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eceeb89b-085b-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:13:58 +0200 (CEST)
Received: from AM6PR0202CA0068.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::45) by AM8PR08MB6529.eurprd08.prod.outlook.com
 (2603:10a6:20b:354::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.30; Thu, 2 May
 2024 08:13:56 +0000
Received: from AMS0EPF000001A0.eurprd05.prod.outlook.com
 (2603:10a6:20b:3a:cafe::6) by AM6PR0202CA0068.outlook.office365.com
 (2603:10a6:20b:3a::45) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29 via Frontend
 Transport; Thu, 2 May 2024 08:13:56 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF000001A0.mail.protection.outlook.com (10.167.16.230) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Thu, 2 May 2024 08:13:55 +0000
Received: ("Tessian outbound af213ececc3d:v315");
 Thu, 02 May 2024 08:13:55 +0000
Received: from df9133db84a9.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5DFEA3C2-BEBA-4BFF-904C-43ABFFA09603.1; 
 Thu, 02 May 2024 08:13:48 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id df9133db84a9.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 May 2024 08:13:48 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DBBPR08MB5964.eurprd08.prod.outlook.com (2603:10a6:10:20c::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28; Thu, 2 May
 2024 08:13:46 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7519.031; Thu, 2 May 2024
 08:13:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eceeb89b-085b-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=RPbx0lNRbqCgn6scOdjUhrH4opZ7S5FDa+0st5LjEQbUke4hLE30zRT/+NXiE66MtouJ1DKqttoj8S1LKCsdqZ80x8elDqnFC+tJzlfw98RQg6+Z3uyD9xvkcUIhaXSejR8nXOHWQ5QkfZRCIg25CK0ZuXTIor+WaYuN9GIqDNOGdpx3/hTu1HFPx4Wvpn6SSxarj1M0ygTxA/2uOUqWtJF8DduzM10DrhCOxb6XeEdV39MuSISkQv1fmNI4mF9olAFW4lm1SSLmk3vm3GnB1PNG4egFhLgTOXBeIup5uqLrwG5eWBDPdv1DS5tsAiOpKCqNCUXglM+dPKP/6TlHtg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BAUECygnRCTvIC2GrB7oVlQdHqf7g+0FQEkkv0kuxuU=;
 b=EXTzPtkjXI3kEuMXovBwDD8CvLQKdllPdy4Bu8pbgw3DbkdA8vTTJ0zFR601FemeCVUnt1cI1el8llsRQ/v8f/5/KxAHFiT+RIAcA2kzJaxnuFRm4xg9Xp56DjBICD3uk8iH1325OeEl59LmT6NZyG0a+2RwnqZbVdOq7a5BtrE8AosO/KcK2sx69FV8gYrp8aZ0xfWoOsm36nuyebsoCnVl0b0tGMI9iNbpbGFoZmoTRSThOZun3F39S79lqDlRTbFmErVfNpo0OgiM5LjiyUefA3dqRQpdG5hpt4vRE+Eg6gn8ASV58+UrUJ5EoViyf9ixelXgYkmnr+2SCe0i/g==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BAUECygnRCTvIC2GrB7oVlQdHqf7g+0FQEkkv0kuxuU=;
 b=l4ZEFx3lwm2esztVOwZ+EfjUoHlR6nL7LxfCeLGvQXyvovv7qiCJ/hVtOsW+HwvcJQJ22YQ2yrXfHENmYadVet3LP6zN5icQEs3n/fufnaxLz51bZ1I+ZejmlSkuNEXKSG0Cx2mamxofx/bXrWKqQPFNMCjzRkVNhBBm4qxbFQE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 89316e26dcd4cc90
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dgmemJqaOA3YdFbaPnAbcXOYHrY/fq7PLevpALvqgzIQCUGAMekR63F8xBP3pG3vuPr2xtAK3MTPZVY9HAtGatMHk16nPrIMD1dJ363n68/IUWZ066EGKw1OJii6i/XxUfbOs6gbXlhDBc8L794tl51+cP38oOhcDCGOyXwYUIgTGg0/YOOEXXUSgWWdiMkyxAEnKpaxqG029BpXT94YfqeDWeSBuoBdewWRLUEJNiogT5YbIzgr4Quie2N8t3EBoSqKgG9neMurOqhcuXdw7jEPuBNCz8ZRAvRp4Ycms29XKRvCpsS5SAqt5ubNjk7Asa59VfTy6mokkhm0tFII2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BAUECygnRCTvIC2GrB7oVlQdHqf7g+0FQEkkv0kuxuU=;
 b=c+myNK9NQpRZ4UwreT5KlSYy7z6/fj6suwBYdA8kovsIf0eIJ4mkgsvqhO115UXKtuyUeLoDxwWI8wVdridYjIGu8ZisZBkR9F37zX0SMkSePTXkH5dlErd2tTSvZq65xmf1ZwdpxaLLg6GDpkNOU37/PMwuuUF4VnHYFJ+Kj4GYmd1fz8d0XBjtXg+92u1OkY50lP0qZB5/XewYRNy2oYdAZci92xIWPFnxS5evOumz1d7HSnbKSjc5npmys9WxBUbGyVma7x/NZY8L9lpU90wAhJXlg44SF5PqCl/cZuhWzJB0vwgiV5ag+YVBTYPWXQarJKikvARBBIqf7UfCiA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BAUECygnRCTvIC2GrB7oVlQdHqf7g+0FQEkkv0kuxuU=;
 b=l4ZEFx3lwm2esztVOwZ+EfjUoHlR6nL7LxfCeLGvQXyvovv7qiCJ/hVtOsW+HwvcJQJ22YQ2yrXfHENmYadVet3LP6zN5icQEs3n/fufnaxLz51bZ1I+ZejmlSkuNEXKSG0Cx2mamxofx/bXrWKqQPFNMCjzRkVNhBBm4qxbFQE=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index:
 AQHamu72Eq1RpEGs60mNp9DIfY8mn7GAr6+AgAFEGQCAAYYzgIAABVAAgAACyoCAABlOgA==
Date: Thu, 2 May 2024 08:13:46 +0000
Message-ID: <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
 <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
In-Reply-To: <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DBBPR08MB5964:EE_|AMS0EPF000001A0:EE_|AM8PR08MB6529:EE_
X-MS-Office365-Filtering-Correlation-Id: 302f8565-b9e9-40e6-62ae-08dc6a7fcfc8
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|366007|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?T3UxMWg0NmZQYzVXbVE0U2toT3E3TndmMzRQc3ltK3hUa1l2Z29hblROcE5Y?=
 =?utf-8?B?bzBWbGowelpjR3VqcWsycW56ajEvc3ZFYjA3RkZvRForZ3NRbnU0WTBzeDVv?=
 =?utf-8?B?UUpDWHFtZTg4WkgwU0FrTThISkFhQmlHV1VxbW51bnJJcVNiMGpMbTdHZUxx?=
 =?utf-8?B?ZkFpamNYNCt0c3EvK1Y5ZkpJL2hqdXVXTDlYMEdMd2FieURnYXk0cFlOQnN2?=
 =?utf-8?B?TEpZYTN0cTFteEZYVkpZN1FlSTR3ZTFmWFNuRSt5QlhoeDBkWFVXelc5Rkx0?=
 =?utf-8?B?b2VyZzNsTnhPVFMrajJmc2Q4U0lXZXk4RGJSN3J0dEhoZVN3ajdNR1N1RHR0?=
 =?utf-8?B?TzhnV2tmd1RiK1Rxb0xYWU9qRC9RNVZrNTUxelNNWmxCeU5XZW9EMUpQREJa?=
 =?utf-8?B?U0ZkYy8yR0EwU1RnaEg3VXpWb2U3USs0MGZjQk15WnIyVHdGTmQxVTMwOFVO?=
 =?utf-8?B?UC9NQVQ2UFRRdXFPMS95cGxpTTkydDdvYkVwK3RKODJrc01iYkRsSFY3Q0Jn?=
 =?utf-8?B?RVhNc1RKZ3k1VHl0dlR2aUtjeUVsdGkxZ05qODVXd2c1RWI4cDc4aUpGd0dU?=
 =?utf-8?B?T3NvN1FkQXM3YXprVjVNeHBZcHk0Tmg0NUdiYy90ZnZUTDNIUVdVUS9EeTMr?=
 =?utf-8?B?NDBGR1laVnY4UnUraTZLMm83MjgxeCtUWmczUDJ6L3FwUVlGMDU0b2h3SWNm?=
 =?utf-8?B?Z1Z6NEt1T2t3aTlpVU5qb01Nd29aU2FZSTZ1TWo5bmxPVGZwZWpneXcvNEhi?=
 =?utf-8?B?OUFTOUMwd2hIT3lnVi9JTlFLeGdmNU5LNDVpWWtqc2ZWMGxLRldQMm1IWDN1?=
 =?utf-8?B?RGRCVVlGdTljTjZrZkZjOHkyT0dFYXRvWDdLb3BNTU9sdFhsQW9RenRhVG43?=
 =?utf-8?B?RVlQV1U0MGZjYkMwVVRGZHd3ZWdyMVRwdFhJUGhaRGRUWlhGQUZJQW4zQXlQ?=
 =?utf-8?B?eW1XRlB3dkJnRHkzc0hSY1lUMDRoYW9SQ1ZDUU9CRTEwRCtvL2QzUU9nenln?=
 =?utf-8?B?RjZNRVNWS3FORVB1NWR4c1FlRGZ5Rks3bmlrSGJSMzVKTXBJZXhvWHRZeUJz?=
 =?utf-8?B?TExoSzMwWE9vTzVHOTVSVUllYXVrQlBiREkxR25JM2ljbWhUaS9wQXdyZVdW?=
 =?utf-8?B?SGhVeENka0Y0MEVMa3I4WEt0cXdhNjE3eXlkZC92elgyekhVYVUzRzhLN2da?=
 =?utf-8?B?NW15MTlXcnNMQmMvcE1IenlWUVdLZGhrTGMxS3RNa2Z4TlF3Z080N0lGWk5X?=
 =?utf-8?B?NW0yUk5yVExEak4xOTlDd1dJcEhGOEQzYUMvcTNqcHZESXJlbmFwaHI0UUgy?=
 =?utf-8?B?UFRLeE9YSDI0aHU3T2lUdmZVemFqVnhkdklYMnRYeG1Tbm9aTklCbEFqUE54?=
 =?utf-8?B?ejRFdUlNbnZ5YnJ1a2lrSlA2ZEM2cVo5cWN3V0xYeTh0L0o4UURwUURoeDI0?=
 =?utf-8?B?eVJ1Rkhuak54cmtHMFlrSTlqckxmdjRndVdJTktYT2pYa1BxZk5jbUVMaTVD?=
 =?utf-8?B?ZjNNdnVxSHMxU0k3RUpHUkdkZEtQSE8zVjlZdHlzWElVUGMzZUdjbGdaZzNv?=
 =?utf-8?B?M2ZlQ2ErbGpXR0pYNTM4Sjg1VEhRNEc4Vk43TjhrZ1RRVjhrVVlqMExXOEFx?=
 =?utf-8?B?WTdud0M2dFAyS05QSjRaU3NYT2NkRXlYSHdrYlZLQ1V5aktKNy9aSWtSWkdq?=
 =?utf-8?B?aUY4dkRRZU9XdFlpTXdOQVZzVlhwYzNoc3l5TDBtL2wxSy9HVnNxQkZWMis4?=
 =?utf-8?B?R1lYK2ZIN1hneWF3dTRSR0drTTZIaGVxbGZ1VW11VDVUWjNsajNZYWRoNS9H?=
 =?utf-8?B?cTdjWjVSQTRZVkE3TG03UT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <E48870000489764E9C951580C2DA0AD2@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5964
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF000001A0.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c60fd3c2-ef00-46c8-618f-08dc6a7fca2a
X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?SVo1a0JvTWx0UGlqZXFJZG1zeG0wbDVzbUpQcjM5cjhoK29SODdQSW9tZlBi?=
 =?utf-8?B?blRxSDl2citWc2FyeU1EUmF6NS9OOWlrNWVUWlpSMUhIRzZvUDRuNUdBczZo?=
 =?utf-8?B?WkZYeEk3aUwrT2NjKy9XZVphR2RtdmJvallMQVVJTUhlZ1hWcU9pRm9hM1FS?=
 =?utf-8?B?a0pLVDRzblF0RXdRRWlMd1J2SFhxb2FLbWkzaXQ2STVucVFGc0toUk5PbU9J?=
 =?utf-8?B?REtqNGlnRzBKdzVuRjJQdWNydXl1Z3dtY3RIbWxONmdYeUdvNEFPVzhMT08v?=
 =?utf-8?B?TzVqYk5JeFdjcUJ0djZnaWhrajdlUlk3aWpIbzBPRzVDSTZwVTFwQ25qYmYx?=
 =?utf-8?B?R1NkRUpaZi9XcGtPTG42QWVVQ2YrTHRiRGlwbjQvNUZqSWs3TUc3WVVZYktq?=
 =?utf-8?B?S0N1aHZDbkZLUzRRcVY5N2NtbVE5V0VmOVU5Q3g2c1ZLNHNockZBaWNMWnRE?=
 =?utf-8?B?clpVNGJFY3ZSNUlPbGVXQ0VnNVRaNER5dm5WanFqeER1VURMYnpNLzVtWUt1?=
 =?utf-8?B?eG5MZG1STkF6Sm9XcE81V0NCand3clB1cFNaU3JybXJXT05Dd0FoQUlwMHN6?=
 =?utf-8?B?Tks5WjNlZjN0WEVzWTAxdUhHalh2OUR0dTUzd0ZnYzZDUmV6bUxUdXhMT2Rr?=
 =?utf-8?B?MFUzVlBqd0hMMjRpL2pLMEFMZG1DcDZvSWNpNzZndTB2WWo5YmZCeWExNkVM?=
 =?utf-8?B?SldVVjhPY08reVlCZmY3VEQ2UHN5TEtqcWt0Q1JKa0d2RWVRVS91VDZ1a0Yr?=
 =?utf-8?B?Y3FnSmZ5VVVDSHZrNDJXSFNDTUREQkVyNit2TEE5YTE0aHhQMWdjdkJZVCtP?=
 =?utf-8?B?cHdLQi91dUlSclR5bmV5RWhaL283V1EreEdTY2tDMVRaY2Y3V3dMWU5ueW9Q?=
 =?utf-8?B?cmYxWVkzVHg5LzVRRytNL1Q4bGFpdnVVeFFjeUdVaklWVGFpTUFUY2wzcTlU?=
 =?utf-8?B?YlBWQi9MV24vQUo0VDlEQ2c2TkVzL3FSanNsV3hjSmdWZkMxbzVKWGxXZGt6?=
 =?utf-8?B?ckNqVmJOeHRnY0NkQk9hTldNUHQzZWlQcjJYYUJ6WllsT2xHMWJ2WExSOXVQ?=
 =?utf-8?B?bTJLN2N5N09uMS9CZm1XZTRLY1JVaGQ5a1JySGZ2VkxhK3NxQkVNRUI4TjFN?=
 =?utf-8?B?N0Z3dFRpaEd4WlZwOFJIZzRqZzd3ZmNFRFM2NEFzN0FJdGl2NE1QQmRGaVBk?=
 =?utf-8?B?UEdhektqMXRaSkd2b2R4YVVGeGxHaWprOUlrMm1iYkM2aVVmb1ZLekVkSllC?=
 =?utf-8?B?cGhDT0ZiY3M0dmZBSVl4Rk5ZN2svSnp1Z01NVzErb2tTaHJJTFZqU0lIL3p1?=
 =?utf-8?B?ZmJDU0YySVpQN2pSMEtUY1hBK2hQYzV1eG1DeXZuMGh3bm9NODJYUHM5aVZ6?=
 =?utf-8?B?Q2NWTVZlVUtlUmFKelkya3c4YXJ5cHkxeno1MXFYTkF1T3hlRGhaZ0xXNWUz?=
 =?utf-8?B?bldNbThGOVJHY0F3UStLQ054VytqOGhTbi9vOTVLQjdEajZqeHV2eE1jdEtB?=
 =?utf-8?B?REs2TjU1dVhSWkhGTVhRTkpCc29YYmxSK0psaXRTSjh0dVVwNmpIV3hSUnJJ?=
 =?utf-8?B?QjhDeENFVzBnaXV4SWF4dHdiZVFTV2t5UmVBV3B1RkRwbFpVblBIZm5xeEFV?=
 =?utf-8?B?UEIrd1Y5RlB6b3JnWTR2MjhtMDVGdkYxREV4dytGeUZiTzgrQkVNVXVEUGs4?=
 =?utf-8?B?UHowMFhaOG03MEtsU01mQU5Jbm5VV2xjTnE0YjRuaXBILzgybDdMNjVtUWp0?=
 =?utf-8?B?TnZMdVlIcmdPOEdpaHlvSDhMRlRnUHJFYzFaMXVraXpSeXU1dzZjUEo5bnRl?=
 =?utf-8?B?azREczZWVjZGYlMveVpYSFpOVVZwWnMwQnpqK3NMaXFWd2lvWnNGZTZVdk1D?=
 =?utf-8?Q?P18OS1pZ9Zhg1?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 08:13:55.8393
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 302f8565-b9e9-40e6-62ae-08dc6a7fcfc8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF000001A0.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6529

DQoNCj4gT24gMiBNYXkgMjAyNCwgYXQgMDc6NDMsIEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNl
LmNvbT4gd3JvdGU6DQo+IA0KPiBPbiAwMi4wNS4yMDI0IDA4OjMzLCBMdWNhIEZhbmNlbGx1IHdy
b3RlOg0KPj4gDQo+PiANCj4+PiBPbiAyIE1heSAyMDI0LCBhdCAwNzoxNCwgSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPiB3cm90ZToNCj4+PiANCj4+PiBPbiAwMS4wNS4yMDI0IDA4OjU3
LCBMdWNhIEZhbmNlbGx1IHdyb3RlOg0KPj4+PiBIaSBKYW4sDQo+Pj4+IA0KPj4+Pj4gT24gMzAg
QXByIDIwMjQsIGF0IDEyOjM3LCBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+IHdyb3Rl
Og0KPj4+Pj4gDQo+Pj4+PiBPbiAzMC4wNC4yMDI0IDEzOjA5LCBMdWNhIEZhbmNlbGx1IHdyb3Rl
Og0KPj4+Pj4+IC0tLSBhL3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zZXR1cC5oDQo+Pj4+Pj4g
KysrIGIveGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmgNCj4+Pj4+PiBAQCAtNjQsMTgg
KzY0LDIwIEBAIHN0cnVjdCBtZW1iYW5rIHsNCj4+Pj4+PiB9Ow0KPj4+Pj4+IA0KPj4+Pj4+IHN0
cnVjdCBtZW1iYW5rcyB7DQo+Pj4+Pj4gLSAgICB1bnNpZ25lZCBpbnQgbnJfYmFua3M7DQo+Pj4+
Pj4gLSAgICB1bnNpZ25lZCBpbnQgbWF4X2JhbmtzOw0KPj4+Pj4+ICsgICAgX19zdHJ1Y3RfZ3Jv
dXAobWVtYmFua3NfaGRyLCBjb21tb24sICwNCj4+Pj4+PiArICAgICAgICB1bnNpZ25lZCBpbnQg
bnJfYmFua3M7DQo+Pj4+Pj4gKyAgICAgICAgdW5zaWduZWQgaW50IG1heF9iYW5rczsNCj4+Pj4+
PiArICAgICk7DQo+Pj4+Pj4gICBzdHJ1Y3QgbWVtYmFuayBiYW5rW107DQo+Pj4+Pj4gfTsNCj4+
Pj4+IA0KPj4+Pj4gSSdtIGFmcmFpZCBJIGNhbid0IHNwb3Qgd2h5IF9fc3RydWN0X2dyb3VwKCkg
aXMgbmVlZGVkIGhlcmUuIFdoeSB3b3VsZCBqdXN0DQo+Pj4+PiBvbmUgb2YgdGhlIHR3byBtb3Jl
IHN0cmFpZ2h0Zm9yd2FyZA0KPj4+Pj4gDQo+Pj4+PiBzdHJ1Y3QgbWVtYmFua3Mgew0KPj4+Pj4g
IHN0cnVjdCBtZW1iYW5rc19oZHIgew0KPj4+Pj4gICAgICB1bnNpZ25lZCBpbnQgbnJfYmFua3M7
DQo+Pj4+PiAgICAgIHVuc2lnbmVkIGludCBtYXhfYmFua3M7DQo+Pj4+PiAgKTsNCj4+Pj4+ICBz
dHJ1Y3QgbWVtYmFuayBiYW5rW107DQo+Pj4+PiB9Ow0KPj4+Pj4gDQo+Pj4+IA0KPj4+PiBBdCB0
aGUgZmlyc3Qgc2lnaHQgSSB0aG91Z2h0IHRoaXMgc29sdXRpb24gY291bGQgaGF2ZSB3b3JrZWQs
IGhvd2V2ZXIgR0NDIGJyb3VnaHQgbWUgYmFjayBkb3duIHRvIGVhcnRoDQo+Pj4+IHJlbWVtYmVy
aW5nIG1lIHRoYXQgZmxleGlibGUgYXJyYXkgbWVtYmVycyBjYW7igJl0IGJlIGxlZnQgYWxvbmUg
aW4gYW4gZW1wdHkgc3RydWN0dXJlOg0KPj4+PiANCj4+Pj4gL2RhdGFfc2RjL2x1Y2ZhbjAxL2dp
dGxhYl9taWNrbGVkb3JlX3hlbi94ZW4veGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6
NzA6NjogZXJyb3I6IGRlY2xhcmF0aW9uIGRvZXMgbm90IGRlY2xhcmUgYW55dGhpbmcgWy1XZXJy
b3JdDQo+Pj4+IDcwIHwgfTsNCj4+Pj4gfCBeDQo+Pj4+IC9kYXRhX3NkYy9sdWNmYW4wMS9naXRs
YWJfbWlja2xlZG9yZV94ZW4veGVuL3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zZXR1cC5oOjcx
OjIwOiBlcnJvcjogZmxleGlibGUgYXJyYXkgbWVtYmVyIGluIGEgc3RydWN0IHdpdGggbm8gbmFt
ZWQgbWVtYmVycw0KPj4+PiA3MSB8IHN0cnVjdCBtZW1iYW5rIGJhbmtbXTsNCj4+Pj4gfCBefn5+
DQo+Pj4+IFsuLi5dDQo+Pj4gDQo+Pj4gU2luY2UgZm9yIHBhdGNoIDEgeW91IGxvb2tlZCBhdCBM
aW51eCdlcyB1YXBpL2xpbnV4L3N0ZGRlZi5oLCB0aGUgc29sdXRpb24NCj4+PiB0byB0aGlzIGxp
ZXMgdGhlcmUsIGluIF9fREVDTEFSRV9GTEVYX0FSUkFZKCkuIEFsb25nc2lkZSBvciBpbnN0ZWFk
IG9mDQo+Pj4gYm9ycm93aW5nIF9fc3RydWN0X2dyb3VwKCksIHdlIGNvdWxkIGNvbnNpZGVyIGJv
cnJvd2luZyB0aGlzIGFzIHdlbGwuIE9yDQo+Pj4gb3Blbi1jb2RlIGl0IGp1c3QgaGVyZSwgZm9y
IHRoZSB0aW1lIGJlaW5nIChwZXJoYXBzIG15IHByZWZlcmVuY2UpLiBZZXQNCj4+PiBpdCdzIG5v
dCBjbGVhciB0byBtZSB0aGF0IGRvaW5nIHNvIHdpbGwgYWN0dWFsbHkgYmUgZW5vdWdoIHRvIG1h
a2UgdGhpbmdzDQo+Pj4gd29yayBmb3IgeW91Lg0KPj4gDQo+PiBJIGxvb2tlZCBhbHNvIGludG8g
X19ERUNMQVJFX0ZMRVhfQVJSQVkoKSwgYnV0IHRoZW4gZGVjaWRlZCBfX3N0cnVjdF9ncm91cCgp
DQo+PiB3YXMgZW5vdWdoIGZvciBteSBwdXJwb3NlLCBjYW4gSSBhc2sgdGhlIHRlY2huaWNhbCBy
ZWFzb25zIHdoeSBpdCB3b3VsZCBiZSB5b3VyDQo+PiBwcmVmZXJlbmNlPyBJcyB0aGVyZSBzb21l
dGhpbmcgaW4gdGhhdCBjb25zdHJ1Y3QgdGhhdCBpcyBhIGNvbmNlcm4gZm9yIHlvdT8NCj4gDQo+
IEkgZG9uJ3QgbGlrZSBlaXRoZXIgY29uc3RydWN0IHZlcnkgbXVjaCwgYnV0IG9mIHRoZSB0d28g
X19ERUNMQVJFX0ZMRVhfQVJSQVkoKQ0KPiBsb29rcyBzbGlnaHRseSBtb3JlICJuYXR1cmFsIiBm
b3Igd2hhdCBpcyB3YW50ZWQgYW5kIGhvdyBpdCdzIGRvbmUuDQo+IF9fc3RydWN0X2dyb3VwKCkg
aW50cm9kdWNpbmcgdHdpY2UgdGhlIChlZmZlY3RpdmVseSkgc2FtZSBzdHJ1Y3R1cmUgZmVlbHMN
Cj4gcHJldHR5IG9kZCwgZm9yIG5vdyBhdCBsZWFzdC4gSXQncyBub3QgZXZlbiBlbnRpcmVseSBj
bGVhciB0byBtZSB3aGV0aGVyIHRoZXJlDQo+IGFyZW4ndCBwaXRmYWxscywgc2VlaW5nIHRoYXQg
dGhlIEMgc3BlYyBkaWZmZXJlbnRpYXRlcyBuYW1lZCBhbmQgdW5uYW1lZA0KPiBzdHJ1Y3QgZmll
bGRzIGluIGEgZmV3IGNhc2VzLiBGb3IgX19ERUNMQVJFX0ZMRVhfQVJSQVkoKSwgb3RvaCwgSSBj
YW4ndA0KPiBwcmVzZW50bHkgc2VlIGFueSByZWFzb24gdG8gc3VzcGVjdCBwb3NzaWJsZSBjb3Ju
ZXIgY2FzZXMuDQo+IA0KPiBZZXQgYXMgc2FpZCBiZWZvcmUgLSBJJ20gbm90IHN1cmUgX19ERUNM
QVJFX0ZMRVhfQVJSQVkoKSBhbG9uZSB3b3VsZCBiZSBlbm91Z2gNCj4gZm9yIHdoYXQgeW91IHdh
bnQgdG8gYWNoaWV2ZS4NCg0KTW1tIHllcywgSSB0aGluayBJIHdvdWxkIHN0aWxsIGhhdmUgcHJv
YmxlbXMgYmVjYXVzZSBjb250YWluZXJfb2Ygd2FudHMgYSBuYW1lZCBtZW1iZXIsDQpzbyBfX0RF
Q0xBUkVfRkxFWF9BUlJBWSgpIGRvZXNu4oCZdCBoZWxwIG11Y2ggYWxvbmUsIGlmIEnigJltIG5v
dCBtaXNzaW5nIHNvbWV0aGluZyBvYnZpb3VzLg0KSWYgeW91IGJlbGlldmUgaG93ZXZlciB0aGF0
IGltcG9ydGluZyBfX3N0cnVjdF9ncm91cCgpIG9ubHkgZm9yIHRoaXMgaW5zdGFuY2UgaXMgbm90
IGVub3VnaCB0byBqdXN0aWZ5DQppdHMgcHJlc2VuY2UgaW4gdGhlIGNvZGViYXNlLCBJIGNvdWxk
IG9wZW4tY29kZSBpdCwgcHJvdmlkZWQgdGhhdCBtYWludGFpbmVycyBhcmUgb2sgd2l0aCB0aGF0
Lg0KDQpJbiBhbnkgY2FzZSBpdCB3b3VsZCBiZSB1c2VkIHNvb24gYWxzbyBmb3Igb3RoZXIgYXJj
aGl0ZWN0dXJlcyB1c2luZyBib290aW5mby4NCg0K


From xen-devel-bounces@lists.xenproject.org Thu May 02 08:33:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:33:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715647.1117419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2RsI-0005Bi-G9; Thu, 02 May 2024 08:33:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715647.1117419; Thu, 02 May 2024 08:33:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2RsI-0005Bb-Bq; Thu, 02 May 2024 08:33:06 +0000
Received: by outflank-mailman (input) for mailman id 715647;
 Thu, 02 May 2024 08:33:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ua+q=MF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2RsH-0005BV-KK
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:33:05 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9750a8f5-085e-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:33:03 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51ab4ee9df8so9570441e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 01:33:03 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 o3-20020ac24943000000b0051e12a2c07bsm95803lfi.20.2024.05.02.01.33.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 01:33:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9750a8f5-085e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714638782; x=1715243582; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=6uVKy1VOU+bFcQfhFvZNp5OKrzQp5GFDpm3cDxnm8rI=;
        b=kA2mSkN7BbLeRo1d9ra46jG19EAGkvaEcZlcC0g8nI/DXh4Z+QbFIT8nJImqeOC6/2
         gQfQnMiqQuSr7VNFDnPUFBqvuiHiDWG1Q5wL3UdPomAZ17Zhxgm26NWcusmeIMwUPE4w
         sBG3cSm5c9pveSApSLuT+rymajSNzFSyu8legV13z9+edPW3Fg20ObscWQF4YTxIpj6w
         hpOojHwFjUhzbOjz0f6wk/qUI0yrMas6Ocg+4lbyE3Ion/gQqzgEGmocmPQVf2WBOkcx
         QwRiT8QTfSKsIezgDMvHHB1Mh2Mrmtskq2rmaiN61qQH+ugQ8nePYD9SkDc06MYK5AvZ
         M6vQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714638782; x=1715243582;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6uVKy1VOU+bFcQfhFvZNp5OKrzQp5GFDpm3cDxnm8rI=;
        b=Hso+4rlANTFH/TB1ddnATb2u8smnZTPQOisyKb9qrvFfMkrltSXF8a9s4VG0ngToq1
         NpHjsFaZKqKpFYFmQbUwRTP8hUpTv8TuRCDYWb8TrYKX+PU2WQrzQ86V/br1ghxCALuf
         0KVv9ZAzyPzCN3U33vrsGn3ISoyVXIhjZbpGu/LX8zNimJIt+Sgyxsnx76wFW9poAd3U
         HTF3fB58R/IcYnXcmb3pu5qK0G0icalJ6nuYA6oGaTcmiNDavmRIhqtVitC9Uet6OYhu
         X9M2x/YFYQAtSlLq6fc4x6ED3r176ZVwXx2mwZGaSDgV9TeHi4ld8rTGpS1qzW3pnAjV
         T2yA==
X-Forwarded-Encrypted: i=1; AJvYcCWi2rl3bVFBfSZJzDjBWgX/iBjLtdomJf9FzGRdnPl/Cx8rvs6zsnVq9OKbSSfa+m0g45xKQvdrTrsdTKml8sHG1YKwAZ+lh3VeeMR3MxA=
X-Gm-Message-State: AOJu0YwgrwoWGflvlrzqW61FrhRtV6iCl2gaan85FWbZvz1pyyBEzRTG
	E3cVYmicCE4U9YON40IoQeP0A/bpvKO93/+WDv0hJL+fhe04+NWL
X-Google-Smtp-Source: AGHT+IHhut/JYAmyK42SUVj9T+RxnUBAni7Z/MI2KEJKojOsjd5tkUEi06agrSY228IWRN8H16FZGw==
X-Received: by 2002:ac2:58e5:0:b0:51e:eac7:d69b with SMTP id v5-20020ac258e5000000b0051eeac7d69bmr1496378lfo.27.1714638782174;
        Thu, 02 May 2024 01:33:02 -0700 (PDT)
Message-ID: <845de338f4c77f31b91f03028235997b2d0386c8.camel@gmail.com>
Subject: Re: [PATCH v8 08/17] xen/riscv: introduce atomic.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
Date: Thu, 02 May 2024 10:33:00 +0200
In-Reply-To: <d40ec2a0-fb14-48f1-804b-9ef18f35b7d5@suse.com>
References: <cover.1713347222.git.oleksii.kurochko@gmail.com>
	 <4967a5fcbef0901ef34d8954a48ded369b1ea2b1.1713347222.git.oleksii.kurochko@gmail.com>
	 <d40ec2a0-fb14-48f1-804b-9ef18f35b7d5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

T24gTW9uLCAyMDI0LTA0LTI5IGF0IDE1OjQ1ICswMjAwLCBKYW4gQmV1bGljaCB3cm90ZToKPiBP
biAxNy4wNC4yMDI0IDEyOjA0LCBPbGVrc2lpIEt1cm9jaGtvIHdyb3RlOgo+ID4gLS0tIC9kZXYv
bnVsbAo+ID4gKysrIGIveGVuL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vYXRvbWljLmgKPiA+IEBA
IC0wLDAgKzEsMjgxIEBACj4gPiArLyogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAt
b25seSAqLwo+ID4gKy8qCj4gPiArICogVGFrZW4gYW5kIG1vZGlmaWVkIGZyb20gTGludXguCj4g
PiArICoKPiA+ICsgKiBUaGUgZm9sbG93aW5nIGNoYW5nZXMgd2VyZSBkb25lOgo+ID4gKyAqIC0g
KiBhdG9taWMjI3ByZWZpeCMjXyp4Y2hnXyooYXRvbWljIyNwcmVmaXgjI190ICp2LCBjX3Qgbikg
d2VyZQo+ID4gdXBkYXRlZAo+ID4gKyAqwqDCoMKgwqAgdG8gdXNlX18qeGNoZ19nZW5lcmljKCkK
PiA+ICsgKiAtIGRyb3AgY2FzdHMgaW4gd3JpdGVfYXRvbWljKCkgYXMgdGhleSBhcmUgdW5uZWNl
c3NhcnkKPiA+ICsgKiAtIGRyb3AgaW50cm9kdWN0aW9uIG9mIFdSSVRFX09OQ0UoKSBhbmQgUkVB
RF9PTkNFKCkuCj4gPiArICrCoMKgIFhlbiBwcm92aWRlcyBBQ0NFU1NfT05DRSgpCj4gPiArICog
LSByZW1vdmUgemVyby1sZW5ndGggYXJyYXkgYWNjZXNzIGluIHJlYWRfYXRvbWljKCkKPiA+ICsg
KiAtIGRyb3AgZGVmaW5lcyBzaW1pbGFyIHRvIHBhdHRlcm4KPiA+ICsgKsKgwqAgI2RlZmluZSBh
dG9taWNfYWRkX3JldHVybl9yZWxheGVkwqDCoCBhdG9taWNfYWRkX3JldHVybl9yZWxheGVkCj4g
PiArICogLSBtb3ZlIG5vdCBSSVNDLVYgc3BlY2lmaWMgZnVuY3Rpb25zIHRvIGFzbS1nZW5lcmlj
L2F0b21pY3MtCj4gPiBvcHMuaAo+ID4gKyAqIAo+ID4gKyAqIENvcHlyaWdodCAoQykgMjAwNyBS
ZWQgSGF0LCBJbmMuIEFsbCBSaWdodHMgUmVzZXJ2ZWQuCj4gPiArICogQ29weXJpZ2h0IChDKSAy
MDEyIFJlZ2VudHMgb2YgdGhlIFVuaXZlcnNpdHkgb2YgQ2FsaWZvcm5pYQo+ID4gKyAqIENvcHly
aWdodCAoQykgMjAxNyBTaUZpdmUKPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMjQgVmF0ZXMgU0FT
Cj4gPiArICovCj4gPiArCj4gPiArI2lmbmRlZiBfQVNNX1JJU0NWX0FUT01JQ19ICj4gPiArI2Rl
ZmluZSBfQVNNX1JJU0NWX0FUT01JQ19ICj4gPiArCj4gPiArI2luY2x1ZGUgPHhlbi9hdG9taWMu
aD4KPiA+ICsKPiA+ICsjaW5jbHVkZSA8YXNtL2NtcHhjaGcuaD4KPiA+ICsjaW5jbHVkZSA8YXNt
L2ZlbmNlLmg+Cj4gPiArI2luY2x1ZGUgPGFzbS9pby5oPgo+ID4gKyNpbmNsdWRlIDxhc20vc3lz
dGVtLmg+Cj4gPiArCj4gPiArdm9pZCBfX2JhZF9hdG9taWNfc2l6ZSh2b2lkKTsKPiA+ICsKPiA+
ICsvKgo+ID4gKyAqIExlZ2FjeSBmcm9tIExpbnV4IGtlcm5lbC4gRm9yIHNvbWUgcmVhc29uIHRo
ZXkgd2FudGVkIHRvIGhhdmUKPiA+IG9yZGVyZWQKPiA+ICsgKiByZWFkL3dyaXRlIGFjY2Vzcy4g
VGhlcmVieSByZWFkKiBpcyB1c2VkIGluc3RlYWQgb2YgcmVhZCpfY3B1KCkKPiA+ICsgKi8KPiA+
ICtzdGF0aWMgYWx3YXlzX2lubGluZSB2b2lkIHJlYWRfYXRvbWljX3NpemUoY29uc3Qgdm9sYXRp
bGUgdm9pZCAqcCwKPiA+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdm9pZCAqcmVzLAo+
ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1bnNpZ25lZCBpbnQgc2l6ZSkKPiA+ICt7
Cj4gPiArwqDCoMKgIHN3aXRjaCAoIHNpemUgKQo+ID4gK8KgwqDCoCB7Cj4gPiArwqDCoMKgIGNh
c2UgMTogKih1aW50OF90ICopcmVzID0gcmVhZGIocCk7IGJyZWFrOwo+ID4gK8KgwqDCoCBjYXNl
IDI6ICoodWludDE2X3QgKilyZXMgPSByZWFkdyhwKTsgYnJlYWs7Cj4gPiArwqDCoMKgIGNhc2Ug
NDogKih1aW50MzJfdCAqKXJlcyA9IHJlYWRsKHApOyBicmVhazsKPiA+ICsjaWZuZGVmIENPTkZJ
R19SSVNDVl8zMgo+ID4gK8KgwqDCoCBjYXNlIDg6ICoodWludDMyX3QgKilyZXMgPSByZWFkcShw
KTsgYnJlYWs7Cj4gPiArI2VuZGlmCj4gPiArwqDCoMKgIGRlZmF1bHQ6IF9fYmFkX2F0b21pY19z
aXplKCk7IGJyZWFrOwo+ID4gK8KgwqDCoCB9Cj4gPiArfQo+ID4gKwo+ID4gKyNkZWZpbmUgcmVh
ZF9hdG9taWMocCkgKHvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIHVuaW9uIHsgdHlwZW9mKCoo
cCkpIHZhbDsgY2hhciBjW3NpemVvZigqKHApKV07IH0geF87wqDCoCBcCj4gPiArwqDCoMKgIHJl
YWRfYXRvbWljX3NpemUocCwgeF8uYywgc2l6ZW9mKCoocCkpKTvCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB4Xy52YWw7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFwKPiA+ICt9KQo+ID4gKwo+ID4gK3N0YXRpYyBhbHdheXNfaW5saW5l
IHZvaWQgX3dyaXRlX2F0b21pYyh2b2xhdGlsZSB2b2lkICpwLAo+ID4gK8KgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgdW5zaWduZWQgbG9uZyB4LCB1bnNpZ25lZAo+ID4gaW50IHNpemUpCj4gPiArewo+ID4g
K8KgwqDCoCBzd2l0Y2ggKCBzaXplICkKPiA+ICvCoMKgwqAgewo+ID4gK8KgwqDCoCBjYXNlIDE6
IHdyaXRlYih4LCBwKTsgYnJlYWs7Cj4gPiArwqDCoMKgIGNhc2UgMjogd3JpdGV3KHgsIHApOyBi
cmVhazsKPiA+ICvCoMKgwqAgY2FzZSA0OiB3cml0ZWwoeCwgcCk7IGJyZWFrOwo+ID4gKyNpZm5k
ZWYgQ09ORklHX1JJU0NWXzMyCj4gPiArwqDCoMKgIGNhc2UgODogd3JpdGVxKHgsIHApOyBicmVh
azsKPiA+ICsjZW5kaWYKPiA+ICvCoMKgwqAgZGVmYXVsdDogX19iYWRfYXRvbWljX3NpemUoKTsg
YnJlYWs7Cj4gPiArwqDCoMKgIH0KPiA+ICt9Cj4gPiArCj4gPiArI2RlZmluZSB3cml0ZV9hdG9t
aWMocCwgeCnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIFwKPiA+ICsoe8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB0eXBlb2YoKihwKSkgeF8gPSAoeCk7wqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiAr
wqDCoMKgIF93cml0ZV9hdG9taWMoKHApLCB4Xywgc2l6ZW9mKCoocCkpKTvCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFwKPiAKPiBOaXQ6IFRoZXJlIGFyZSBzdGlsbCBleGNlc3MgcGFyZW50
aGVzZXMgaGVyZS4KPiAKPiA+ICvCoMKgwqAgeF87wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgIFwKPiAKPiBXaHkgaXMgdGhpcz8gVGhlIG1hY3JvIGlzbid0IHN1cHBvc2Vk
IHRvICJyZXR1cm4iIGEgdmFsdWUsIGlzIGl0PwpZb3UgYXJlIHJpZ2h0LiB3cml0ZV9hdG9taWMo
KSBpc24ndCBzdXBwb3NlZCB0byByZXR1cm4gYSB2YWx1ZSwgc28KInhfOyIgY2FuIGJlIGRyb3Bw
ZWQuCgo+IAo+ID4gK30pCj4gPiArCj4gPiArc3RhdGljIGFsd2F5c19pbmxpbmUgdm9pZCBfYWRk
X3NpemVkKHZvbGF0aWxlIHZvaWQgKnAsCj4gPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGxv
bmcgeCwgdW5zaWduZWQgaW50Cj4gPiBzaXplKQo+ID4gK3sKPiA+ICvCoMKgwqAgc3dpdGNoICgg
c2l6ZSApCj4gPiArwqDCoMKgIHsKPiA+ICvCoMKgwqAgY2FzZSAxOgo+ID4gK8KgwqDCoCB7Cj4g
PiArwqDCoMKgwqDCoMKgwqAgdm9sYXRpbGUgdWludDhfdCAqcHRyID0gKHZvbGF0aWxlIHVpbnQ4
X3QgKilwOwo+IAo+IEhlcmUgYW5kIGJlbG93OiBXaHkgdGhlIGNhc3RzPwpZZXMsIHRoZXkgYXJl
IG5vdCBuZWVkZWQuIEluaXRpYWxseSBJIGRlY2lkZWQgdGhhdCBpdCBjb3VsZCBiZSBhCmNvbXBp
bGF0aW9uIGlzc3VlLgoKfiBPbGVrc2lpCg==



From xen-devel-bounces@lists.xenproject.org Thu May 02 08:46:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:46:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715654.1117429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2S53-0008MG-It; Thu, 02 May 2024 08:46:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715654.1117429; Thu, 02 May 2024 08:46:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2S53-0008M9-F9; Thu, 02 May 2024 08:46:17 +0000
Received: by outflank-mailman (input) for mailman id 715654;
 Thu, 02 May 2024 08:46:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ua+q=MF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2S52-0008M3-FL
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:46:16 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6e859080-0860-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:46:13 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51c077cfc09so9118872e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 01:46:13 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 g10-20020a19ac0a000000b0051f2bc124b8sm98064lfc.282.2024.05.02.01.46.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 01:46:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e859080-0860-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714639573; x=1715244373; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=119SfVGplz4DEiOku4pP5MJJYMC0Kn1JeItPRGIgL6I=;
        b=U85BpGolPpegSz+AZZd83xu4+ctSA1Ay4GaxRKwPGumu7LXlJP2eM5r/MfT5g7vjtQ
         aLHHhfTsV2SMpRXwJqK4IvrOrNHW9snO7yHEyBUtscjHcBCEk1oNYXWUZqC9vBJM+Kbp
         FgvZeF1W2sdDzw/VhI7/l4PWZjWF88rS5/ROlnDVFjVh8edEsD5sCNjSoe9juOYWXdy4
         CoyLOTcvpkxphUqhlbPDUMntvazPwBTiFwtXyirhy9ZptqSCLbuFqxpqULCBIzAThkgT
         dC3gGU4a4FzrPeRBQyfXn4t5FnLoXHYOtQAjL9QqGDKbdFgCC0+kVHinyGGFgwKP03uR
         Rz4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714639573; x=1715244373;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=119SfVGplz4DEiOku4pP5MJJYMC0Kn1JeItPRGIgL6I=;
        b=TSAB49lGtVvkr5Uk0S90rJyVaSWEtkmUL8c4DuQpCHCsdPYeMDNCHjCj77/aFlUGu1
         Sy81YbNNDkL2LPcFfMi5UPxK11j6CoNbQdHeKyTcMxti5EHqAL6C06KCw1xgck2fy4Vt
         grrya13nwMKo1j7F+lB1TsrI1Dtz+CQABBqZ6gLEIBpaDPHqbee7HTQo2O5vY1kEE+mr
         xRphWttIF0237eCoqt4hzfQSMGEmYQmClnYtqi21b7s9TyQIBf2JxnrOt0CfyRuSoy2O
         flouznoU9XmgjrfMskCvuslSahxawz1bh1iNEOYdSZG8lfRd37JV1gq3Zxu/QyuHxK3v
         r5NA==
X-Forwarded-Encrypted: i=1; AJvYcCXwxsvrE9nUyipDqLz4rzGgAFFXIeb7bolkiFu7tVQRkwYqeTLCByYbEPmjdxFZxNIA0vaP9jHGENgEwwFlk/94UPNLATfWNfrPeyMlFV0=
X-Gm-Message-State: AOJu0Yw/YWu7jR1awspSAJn0cO6RCae74ebsHUqBE46jMwzdRfXoUc0r
	uCzVzOjs6yQ0b5jP2ihUoBPi1x3NtSaX86KQTGmqZ+uao2iPidohxS8JIQ==
X-Google-Smtp-Source: AGHT+IG59POMHBugH1TPVjU1L/yXCgOYEUuzRowyg8acD+st/dsd3VewFQerZYTtAMYt7kTu9LbFlw==
X-Received: by 2002:a19:550e:0:b0:51d:348b:80bb with SMTP id n14-20020a19550e000000b0051d348b80bbmr2610874lfe.55.1714639572959;
        Thu, 02 May 2024 01:46:12 -0700 (PDT)
Message-ID: <2cabc6a1448c6f18cab963bc463347f088d643dd.camel@gmail.com>
Subject: Re: [PATCH for-4.19] ppc/riscv: fix arch_acquire_resource_check()
From: Oleksii <oleksii.kurochko@gmail.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Shawn Anastasio <sanastasio@raptorengineering.com>, Alistair Francis
	 <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, Connor
	Davis <connojdavis@gmail.com>
Date: Thu, 02 May 2024 10:46:12 +0200
In-Reply-To: <20240430153423.80875-1-roger.pau@citrix.com>
References: <20240430153423.80875-1-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-04-30 at 17:34 +0200, Roger Pau Monne wrote:
> None of the implementations support set_foreign_p2m_entry() yet,
> neither they
> have a p2m walk in domain_relinquish_resources() in order to remove
> the foreign
> mappings from the p2m and thus drop the extra refcounts.
>=20
> Adjust the arch helpers to return false and introduce a comment that
> clearly
> states it is not only taking extra refcounts that's needed, but also
> dropping
> them on domain teardown.
I am okay with such adjustment for now as it is more safe and nothing
will be missed during implementation of p2m, but I am curious how then
Arm handles that, their implementation is also just returns true. ( I
planned to have p2m implementation similar to Arm )

Anyway, based on that it safer for RISC-V:
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Fixes: 4988704e00d8 ('xen/riscv: introduce p2m.h')
> Fixes: 4a2f68f90930 ('xen/ppc: Define minimal stub headers required
> for full build')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> =C2=A0xen/arch/ppc/include/asm/p2m.h=C2=A0=C2=A0 | 7 ++++---
> =C2=A0xen/arch/riscv/include/asm/p2m.h | 7 ++++---
> =C2=A02 files changed, 8 insertions(+), 6 deletions(-)
>=20
> diff --git a/xen/arch/ppc/include/asm/p2m.h
> b/xen/arch/ppc/include/asm/p2m.h
> index 25ba05466853..f144ef8e1a54 100644
> --- a/xen/arch/ppc/include/asm/p2m.h
> +++ b/xen/arch/ppc/include/asm/p2m.h
> @@ -81,10 +81,11 @@ static inline mfn_t gfn_to_mfn(struct domain *d,
> gfn_t gfn)
> =C2=A0static inline bool arch_acquire_resource_check(struct domain *d)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 /*
> -=C2=A0=C2=A0=C2=A0=C2=A0 * The reference counting of foreign entries in
> set_foreign_p2m_entry()
> -=C2=A0=C2=A0=C2=A0=C2=A0 * is supported on PPC.
> +=C2=A0=C2=A0=C2=A0=C2=A0 * Requires refcounting the foreign mappings and=
 walking the p2m
> on
> +=C2=A0=C2=A0=C2=A0=C2=A0 * teardown in order to remove foreign pages fro=
m the p2m and
> drop the
> +=C2=A0=C2=A0=C2=A0=C2=A0 * extra reference counts.
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> -=C2=A0=C2=A0=C2=A0 return true;
> +=C2=A0=C2=A0=C2=A0 return false;
> =C2=A0}
> =C2=A0
> =C2=A0static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
> diff --git a/xen/arch/riscv/include/asm/p2m.h
> b/xen/arch/riscv/include/asm/p2m.h
> index 87b13f897926..387f372b5d26 100644
> --- a/xen/arch/riscv/include/asm/p2m.h
> +++ b/xen/arch/riscv/include/asm/p2m.h
> @@ -79,10 +79,11 @@ static inline mfn_t gfn_to_mfn(struct domain *d,
> gfn_t gfn)
> =C2=A0static inline bool arch_acquire_resource_check(struct domain *d)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 /*
> -=C2=A0=C2=A0=C2=A0=C2=A0 * The reference counting of foreign entries in
> set_foreign_p2m_entry()
> -=C2=A0=C2=A0=C2=A0=C2=A0 * is supported on RISCV.
> +=C2=A0=C2=A0=C2=A0=C2=A0 * Requires refcounting the foreign mappings and=
 walking the p2m
> on
> +=C2=A0=C2=A0=C2=A0=C2=A0 * teardown in order to remove foreign pages fro=
m the p2m and
> drop the
> +=C2=A0=C2=A0=C2=A0=C2=A0 * extra reference counts.
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> -=C2=A0=C2=A0=C2=A0 return true;
> +=C2=A0=C2=A0=C2=A0 return false;
> =C2=A0}
> =C2=A0
> =C2=A0static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)



From xen-devel-bounces@lists.xenproject.org Thu May 02 08:50:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:50:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715659.1117439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2S9M-0001Mp-3Z; Thu, 02 May 2024 08:50:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715659.1117439; Thu, 02 May 2024 08:50:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2S9L-0001Mi-Vr; Thu, 02 May 2024 08:50:43 +0000
Received: by outflank-mailman (input) for mailman id 715659;
 Thu, 02 May 2024 08:50:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ua+q=MF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2S9K-0001Mc-P7
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:50:42 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0dbe21a2-0861-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:50:40 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57230faeb81so4691941a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 01:50:40 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 c17-20020a1709060fd100b00a526a992d82sm297321ejk.4.2024.05.02.01.50.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 01:50:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0dbe21a2-0861-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714639840; x=1715244640; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=FwQKVQZOC9HuXifBT4s+lUF7T1W9sP/iWQHSp2dc6hg=;
        b=eKBdrLL8RhEcKnrD/82Pgau/fYhrtDb7RwRsrS2QQ+PJCxOR0NdQmwpv10xJ0EHuJ9
         4sxwRxMf8Ki0FeaSEJokGdjtsAz9KXnjkuLvtjzh/784ZXYxreNKOgyr0UEk/BHY0/w0
         /4LefN/qNK53NfB/CJs52YP+fxIpNgwu/VJ39yAKwB/jJpqeXj/IFjSIcABdweVRZWXG
         Had6S9M//PSATwi9h4FA7b/U/R9LXE5OGGCcHDY/3PNVZRVSH2kaR24DLIoCCUJUSNXe
         logDxVIOUq3KOyHXmUonznFQDTtr8XxRJBo8TOmIju/Owla1XLO1jKClwLQWQXns8Dsm
         soAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714639840; x=1715244640;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FwQKVQZOC9HuXifBT4s+lUF7T1W9sP/iWQHSp2dc6hg=;
        b=JqoYnu1yE1zg4ldaMw5B6Y9wrQZPE9sLVaoUySITNImnuBzUeEiN5IE7PgsamImQ+7
         /PcvaaV+pSfRc+YTUq541i/3lLaLFDBLNs8qZK9SGeRHP2yJdDbPGxERG0nBhyWR0BNZ
         3Rjlo5TXL6mz4Uo7alkdL7Y8wEfberzMH3RQC7L/POODUOv17AmaQ14SeB99YIrbE15v
         43TSDU29WCu3gnUWoT4Yti2pzwZyCdT+YLkpuDTATzSKxsDLwDCmPMmzsHT1Ndpg2Jn5
         v15cjnhPHzXHJ+Glt519W2pfGkbrBM5BmFZTsaI0XyFRATNWe3O1Wu/Y09Z1wQT2q0R8
         XWrQ==
X-Forwarded-Encrypted: i=1; AJvYcCV8oFLMQIhldNBw7udFIHA1Zz7Al8yNnklfmaNd5t3iqn1d64ZfODH1My+wA4Mqb3X05QzuiNf+qgZZ9M2Gvhn9GTiRFyGvr2CseaPXfIQ=
X-Gm-Message-State: AOJu0YxH4isyXE5t8JSs86GyCuzwflptwMbU3zv3Tn6PX92eB4e/zcH2
	d1y2CKCGeW/l/5Th8X80db3n/RbAYejWmhVAwEpQ2PVcjlBgnqnP
X-Google-Smtp-Source: AGHT+IEFRr2bNheETiMxne1P0mHyjiVNUbUXoOY59uuCkTOEka3C5lhLPWWdD2vW2qU+/vvlgG0P0g==
X-Received: by 2002:a50:c353:0:b0:570:5bcc:f749 with SMTP id q19-20020a50c353000000b005705bccf749mr3586960edb.29.1714639840089;
        Thu, 02 May 2024 01:50:40 -0700 (PDT)
Message-ID: <a23399a560ac566e15a3881063d34ebc19fb524a.camel@gmail.com>
Subject: Re: [PATCH for-4.19? 0/2] xen/x86: support foreign mappings for HVM
From: Oleksii <oleksii.kurochko@gmail.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>,  George Dunlap <george.dunlap@citrix.com>,
 Community Manager <community.manager@xenproject.org>
Date: Thu, 02 May 2024 10:50:38 +0200
In-Reply-To: <20240430165845.81696-1-roger.pau@citrix.com>
References: <20240430165845.81696-1-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

Hi Roger,

On Tue, 2024-04-30 at 18:58 +0200, Roger Pau Monne wrote:
> Hello,
>=20
> The following series attempts to solve a shortcoming of HVM/PVH
> guests
> with the lack of support for foreign mappings.=C2=A0 Such lack of support
> prevents using PVH based guests as stubdomains for example.
>=20
> Add support in a way similar to how it was done on Arm, by iterating
> over the p2m based on the maximum gfn.
>=20
> Mostly untested, sending early in case it could be considered for
> 4.19.
In case of it will be properly tested I think we can consider this
patch as a candidate for 4.19

~ Oleksii

>=20
> Thanks, Roger.
>=20
> Roger Pau Monne (2):
> =C2=A0 xen/x86: account for max guest gfn and number of foreign mappings
> in
> =C2=A0=C2=A0=C2=A0 the p2m
> =C2=A0 xen/x86: remove foreign mappings from the p2m on teardown
>=20
> =C2=A0CHANGELOG.md=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/x86/domain.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 |=C2=A0 6 +++
> =C2=A0xen/arch/x86/include/asm/p2m.h | 28 ++++++++++----
> =C2=A0xen/arch/x86/mm/p2m.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 | 70
> ++++++++++++++++++++++++++++++++--
> =C2=A04 files changed, 94 insertions(+), 11 deletions(-)
>=20



From xen-devel-bounces@lists.xenproject.org Thu May 02 08:55:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:55:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715664.1117448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SDO-0002NK-IW; Thu, 02 May 2024 08:54:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715664.1117448; Thu, 02 May 2024 08:54:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SDO-0002ND-Fv; Thu, 02 May 2024 08:54:54 +0000
Received: by outflank-mailman (input) for mailman id 715664;
 Thu, 02 May 2024 08:54:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ua+q=MF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2SDN-0002LU-1f
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:54:53 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a35b1925-0861-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 10:54:51 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2e09138a2b1so52262731fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 01:54:51 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 u13-20020a2e9b0d000000b002e1a4aa051bsm109759lji.50.2024.05.02.01.54.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 01:54:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a35b1925-0861-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714640091; x=1715244891; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=E26Wv2TFlLS5AsUDdheItNFrMuNyZIrA1sETr9u6GEk=;
        b=eAVhiZJJywgV403aInRdJ/+dVg1SUEsAyH3KqElocNgbFkTsY8u4KP3BoHzqFj04C4
         N3Lb5mh4LNkWAhGGKQTY+FggTqWlqTfMmtbSXo0IVmUQJS90HnSMRwcOJnuhRkldMVcg
         JhiePScPoQXUmPNW22icmlX1BZ0TXhjWDdXS72hd4M9lrg1nVtsu/atv3ZVNmP0DPVa1
         M4TKoAY4OlZV0GxLnLMEoIW/InVJwh2Bsqsx0uagE6+a0ZFghNcchYu79sYwfXjszccK
         7rFpHEDNUNRggLUmdnDKJHqXkN7CZd+LdL3Xn3NIZX3GdjICgYVCkYmU6PZUiNoLe8BV
         77Pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714640091; x=1715244891;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E26Wv2TFlLS5AsUDdheItNFrMuNyZIrA1sETr9u6GEk=;
        b=SNXDKIyPJYjcYfWrtKliqxW9HQrIr5HQfjxxd0n79D1POxbbDyNBRy7bbaIvBJkSmZ
         MW8qGj/md/ppz4uh8WMQdFeoOeScEUWCbu2LS72vYQutIfLgag3Uipup11DsSewaDMIr
         qaiT5KNHBXkVEDIjTRwbNt3tKiYSmiKusyik+0lJgEZnD+dgu9qhJ/qEuOT/vrwQVAHc
         ObHGQEThjOWahRhnZWaECi4rQfVSDDdrfkyLFr5hDdD69I1duLUTbMANKVTQL7duKACz
         jV9W66X3JFNStszfyz1KmyTzPH4iKXv3JMdKKtSbzZJTrbruRmLHqwhTAiAOuI9lFOEr
         eQvQ==
X-Forwarded-Encrypted: i=1; AJvYcCXdcVnb38wdGxhltcPJ7mkKlG4yQhJKJVIohNwZrUfOmtqUENzhKXNQ6J89jF04qrstyxsi3fkghrtWNYB5endskQ7dfk12RoTERZYzOZk=
X-Gm-Message-State: AOJu0YwSnOT7Iiwq16s7ib9smEe235uOp0968RlLlpXQXtP0J1Lm/Iz+
	22p6SmHWVgw/CccVeMEdVOksvpb7QlX3pEJetlmPtvyyINu82FV4
X-Google-Smtp-Source: AGHT+IGHRbaEb9PxwXTllTOmxwL0HPLTR4C6bDnFv7mHTYiOsErQo3uDGSDpVdAVBN22i1yShfzB3A==
X-Received: by 2002:a2e:8017:0:b0:2e1:ebec:1ded with SMTP id j23-20020a2e8017000000b002e1ebec1dedmr514876ljg.25.1714640090975;
        Thu, 02 May 2024 01:54:50 -0700 (PDT)
Message-ID: <34ef8b7e5797643aa36080389d61eac4b16ed648.camel@gmail.com>
Subject: Re: [PATCH for-4.19] tools/xen-cpuid: switch to use cpu-policy
 defined names
From: Oleksii <oleksii.kurochko@gmail.com>
To: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Jan Beulich
	 <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Anthony PERARD
	 <anthony@xenproject.org>, xen-devel@lists.xenproject.org
Date: Thu, 02 May 2024 10:54:50 +0200
In-Reply-To: <ZjDVN6kexFD0vcT2@macbook>
References: <20240430082901.77662-1-roger.pau@citrix.com>
	 <78d009f4-6610-4fcf-9cdc-ee1081e6c763@suse.com> <ZjDVN6kexFD0vcT2@macbook>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

T24gVHVlLCAyMDI0LTA0LTMwIGF0IDEzOjI1ICswMjAwLCBSb2dlciBQYXUgTW9ubsOpIHdyb3Rl
Ogo+IE9uIFR1ZSwgQXByIDMwLCAyMDI0IGF0IDEyOjM3OjQ0UE0gKzAyMDAsIEphbiBCZXVsaWNo
IHdyb3RlOgo+ID4gT24gMzAuMDQuMjAyNCAxMDoyOSwgUm9nZXIgUGF1IE1vbm5lIHdyb3RlOgo+
ID4gPiBMaWtlIGl0IHdhcyBkb25lIHJlY2VudGx5IGZvciBsaWJ4bCwgc3dpdGNoIHRvIHVzaW5n
IHRoZSBhdXRvLQo+ID4gPiBnZW5lcmF0ZWQgZmVhdHVyZQo+ID4gPiBuYW1lcyBieSB0aGUgcHJv
Y2Vzc2luZyBvZiBjcHVmZWF0dXJlc2V0LmgsIHRoaXMgYWxsb3dzIHJlbW92aW5nCj4gPiA+IHRo
ZSBvcGVuLWNvZGVkCj4gPiA+IGZlYXR1cmUgbmFtZXMsIGFuZCB1bmlmaWVzIHRoZSBmZWF0dXJl
IG5hbWluZyB3aXRoIGxpYnhsIGFuZCB0aGUKPiA+ID4gaHlwZXJ2aXNvci4KPiA+ID4gCj4gPiA+
IE5vdGUgdGhhdCBsZWFmIG5hbWVzIG5lZWQgdG8gYmUga2VwdCwgYXMgdGhlIGN1cnJlbnQgYXV0
by0KPiA+ID4gZ2VuZXJhdGVkIGRhdGEKPiA+ID4gZG9lc24ndCBjb250YWluIHRoZSBsZWFmIG5h
bWVzLgo+ID4gPiAKPiA+ID4gU2lnbmVkLW9mZi1ieTogUm9nZXIgUGF1IE1vbm7DqSA8cm9nZXIu
cGF1QGNpdHJpeC5jb20+Cj4gPiA+IC0tLQo+ID4gPiBMYXRlIGZvciA0LjE5LCBidXQgSSB3b3Vs
ZCBzdGlsbCBsaWtlIGl0IHRvIGJlIGNvbnNpZGVyZWQgZm9yCj4gPiA+IGluY2x1c2lvbiBzaW5j
ZQo+ID4gPiBpdCdzIElNTyBhIG5pY2UgY2xlYW51cCBhbmQgcmVkdWNlcyB0aGUgYnVyZGVuIG9m
IGFkZGluZyBuZXcKPiA+ID4gZmVhdHVyZSBiaXRzIGludG8KPiA+ID4gdGhlIHBvbGljeS4KPiA+
IAo+ID4gSSBhZ3JlZSwgYnV0IGl0J3MgT2xla3NpaSdzIGNhbGwgKG5vdyBDYy1lZCkuClRoaXMg
Y2xlYW51cCBsb29rcyBnb29kIHRvIG1lIGFuZCBJTU8gd2UgY2FuIGNvbnNpZGVyIGl0IGZvciA0
LjE5CnJlbGVhc2UuCgp+IE9sZWtzaWkKCj4gPiAKPiA+ID4gLS0tIGEvdG9vbHMvbWlzYy94ZW4t
Y3B1aWQuYwo+ID4gPiArKysgYi90b29scy9taXNjL3hlbi1jcHVpZC5jCj4gPiA+IEBAIC0xMiwy
ODIgKzEyLDMzIEBACj4gPiA+IMKgCj4gPiA+IMKgI2luY2x1ZGUgPHhlbi10b29scy9jb21tb24t
bWFjcm9zLmg+Cj4gPiA+IMKgCj4gPiA+IC1zdGF0aWMgdWludDMyX3QgbnJfZmVhdHVyZXM7Cj4g
PiA+IC0KPiA+ID4gLXN0YXRpYyBjb25zdCBjaGFyICpjb25zdCBzdHJfMWRbMzJdID0KPiA+ID4g
LXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gImZwdSIswqAgWyAxXSA9ICJ2bWUiLAo+ID4gPiAtwqDC
oMKgIFsgMl0gPSAiZGUiLMKgwqAgWyAzXSA9ICJwc2UiLAo+ID4gPiAtwqDCoMKgIFsgNF0gPSAi
dHNjIizCoCBbIDVdID0gIm1zciIsCj4gPiA+IC3CoMKgwqAgWyA2XSA9ICJwYWUiLMKgIFsgN10g
PSAibWNlIiwKPiA+ID4gLcKgwqDCoCBbIDhdID0gImN4OCIswqAgWyA5XSA9ICJhcGljIiwKPiA+
ID4gLcKgwqDCoCAvKiBbMTBdICovwqDCoMKgwqAgWzExXSA9ICJzeXNlbnRlciIsCj4gPiA+IC3C
oMKgwqAgWzEyXSA9ICJtdHJyIiwgWzEzXSA9ICJwZ2UiLAo+ID4gPiAtwqDCoMKgIFsxNF0gPSAi
bWNhIizCoCBbMTVdID0gImNtb3YiLAo+ID4gPiAtwqDCoMKgIFsxNl0gPSAicGF0IizCoCBbMTdd
ID0gInBzZTM2IiwKPiA+ID4gLcKgwqDCoCBbMThdID0gInBzbiIswqAgWzE5XSA9ICJjbGZsdXNo
IiwKPiA+ID4gLcKgwqDCoCAvKiBbMjBdICovwqDCoMKgwqAgWzIxXSA9ICJkcyIsCj4gPiA+IC3C
oMKgwqAgWzIyXSA9ICJhY3BpIiwgWzIzXSA9ICJtbXgiLAo+ID4gPiAtwqDCoMKgIFsyNF0gPSAi
ZnhzciIsIFsyNV0gPSAic3NlIiwKPiA+ID4gLcKgwqDCoCBbMjZdID0gInNzZTIiLCBbMjddID0g
InNzIiwKPiA+ID4gLcKgwqDCoCBbMjhdID0gImh0dCIswqAgWzI5XSA9ICJ0bSIsCj4gPiA+IC3C
oMKgwqAgWzMwXSA9ICJpYTY0IiwgWzMxXSA9ICJwYmUiLAo+ID4gPiAtfTsKPiA+ID4gLQo+ID4g
PiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl8xY1szMl0gPQo+ID4gPiAtewo+ID4gPiAt
wqDCoMKgIFsgMF0gPSAic3NlMyIswqDCoMKgIFsgMV0gPSAicGNsbXVscWRxIiwKPiA+ID4gLcKg
wqDCoCBbIDJdID0gImR0ZXM2NCIswqAgWyAzXSA9ICJtb25pdG9yIiwKPiA+ID4gLcKgwqDCoCBb
IDRdID0gImRzLWNwbCIswqAgWyA1XSA9ICJ2bXgiLAo+ID4gPiAtwqDCoMKgIFsgNl0gPSAic214
IizCoMKgwqDCoCBbIDddID0gImVzdCIsCj4gPiA+IC3CoMKgwqAgWyA4XSA9ICJ0bTIiLMKgwqDC
oMKgIFsgOV0gPSAic3NzZTMiLAo+ID4gPiAtwqDCoMKgIFsxMF0gPSAiY250eC1pZCIsIFsxMV0g
PSAic2RnYiIsCj4gPiA+IC3CoMKgwqAgWzEyXSA9ICJmbWEiLMKgwqDCoMKgIFsxM10gPSAiY3gx
NiIsCj4gPiA+IC3CoMKgwqAgWzE0XSA9ICJ4dHByIizCoMKgwqAgWzE1XSA9ICJwZGNtIiwKPiA+
ID4gLcKgwqDCoCAvKiBbMTZdICovwqDCoMKgwqDCoMKgwqAgWzE3XSA9ICJwY2lkIiwKPiA+ID4g
LcKgwqDCoCBbMThdID0gImRjYSIswqDCoMKgwqAgWzE5XSA9ICJzc2U0MSIsCj4gPiA+IC3CoMKg
wqAgWzIwXSA9ICJzc2U0MiIswqDCoCBbMjFdID0gIngyYXBpYyIsCj4gPiA+IC3CoMKgwqAgWzIy
XSA9ICJtb3ZlYmUiLMKgIFsyM10gPSAicG9wY250IiwKPiA+ID4gLcKgwqDCoCBbMjRdID0gInRz
Yy1kbCIswqAgWzI1XSA9ICJhZXNuaSIsCj4gPiA+IC3CoMKgwqAgWzI2XSA9ICJ4c2F2ZSIswqDC
oCBbMjddID0gIm9zeHNhdmUiLAo+ID4gPiAtwqDCoMKgIFsyOF0gPSAiYXZ4IizCoMKgwqDCoCBb
MjldID0gImYxNmMiLAo+ID4gPiAtwqDCoMKgIFszMF0gPSAicmRybmQiLMKgwqAgWzMxXSA9ICJo
eXBlciIsCj4gPiA+IC19Owo+ID4gPiAtCj4gPiA+IC1zdGF0aWMgY29uc3QgY2hhciAqY29uc3Qg
c3RyX2UxZFszMl0gPQo+ID4gPiAtewo+ID4gPiAtwqDCoMKgIFsgMF0gPSAiZnB1IizCoMKgwqAg
WyAxXSA9ICJ2bWUiLAo+ID4gPiAtwqDCoMKgIFsgMl0gPSAiZGUiLMKgwqDCoMKgIFsgM10gPSAi
cHNlIiwKPiA+ID4gLcKgwqDCoCBbIDRdID0gInRzYyIswqDCoMKgIFsgNV0gPSAibXNyIiwKPiA+
ID4gLcKgwqDCoCBbIDZdID0gInBhZSIswqDCoMKgIFsgN10gPSAibWNlIiwKPiA+ID4gLcKgwqDC
oCBbIDhdID0gImN4OCIswqDCoMKgIFsgOV0gPSAiYXBpYyIsCj4gPiA+IC3CoMKgwqAgLyogWzEw
XSAqL8KgwqDCoMKgwqDCoCBbMTFdID0gInN5c2NhbGwiLAo+ID4gPiAtwqDCoMKgIFsxMl0gPSAi
bXRyciIswqDCoCBbMTNdID0gInBnZSIsCj4gPiA+IC3CoMKgwqAgWzE0XSA9ICJtY2EiLMKgwqDC
oCBbMTVdID0gImNtb3YiLAo+ID4gPiAtwqDCoMKgIFsxNl0gPSAiZmNtb3YiLMKgIFsxN10gPSAi
cHNlMzYiLAo+ID4gPiAtwqDCoMKgIC8qIFsxOF0gKi/CoMKgwqDCoMKgwqAgWzE5XSA9ICJtcCIs
Cj4gPiA+IC3CoMKgwqAgWzIwXSA9ICJueCIswqDCoMKgwqAgLyogWzIxXSAqLwo+ID4gPiAtwqDC
oMKgIFsyMl0gPSAibW14KyIswqDCoCBbMjNdID0gIm1teCIsCj4gPiA+IC3CoMKgwqAgWzI0XSA9
ICJmeHNyIizCoMKgIFsyNV0gPSAiZnhzcisiLAo+ID4gPiAtwqDCoMKgIFsyNl0gPSAicGcxZyIs
wqDCoCBbMjddID0gInJkdHNjcCIsCj4gPiA+IC3CoMKgwqAgLyogWzI4XSAqL8KgwqDCoMKgwqDC
oCBbMjldID0gImxtIiwKPiA+ID4gLcKgwqDCoCBbMzBdID0gIjNkbm93KyIsIFszMV0gPSAiM2Ru
b3ciLAo+ID4gPiAtfTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0
cl9lMWNbMzJdID0KPiA+ID4gLXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gImxhaGYtbG0iLMKgwqDC
oCBbIDFdID0gImNtcCIsCj4gPiA+IC3CoMKgwqAgWyAyXSA9ICJzdm0iLMKgwqDCoMKgwqDCoMKg
IFsgM10gPSAiZXh0YXBpYyIsCj4gPiA+IC3CoMKgwqAgWyA0XSA9ICJjcjhkIizCoMKgwqDCoMKg
wqAgWyA1XSA9ICJsemNudCIsCj4gPiA+IC3CoMKgwqAgWyA2XSA9ICJzc2U0YSIswqDCoMKgwqDC
oCBbIDddID0gIm1zc2UiLAo+ID4gPiAtwqDCoMKgIFsgOF0gPSAiM2Rub3dwZiIswqDCoMKgIFsg
OV0gPSAib3N2dyIsCj4gPiA+IC3CoMKgwqAgWzEwXSA9ICJpYnMiLMKgwqDCoMKgwqDCoMKgIFsx
MV0gPSAieG9wIiwKPiA+ID4gLcKgwqDCoCBbMTJdID0gInNraW5pdCIswqDCoMKgwqAgWzEzXSA9
ICJ3ZHQiLAo+ID4gPiAtwqDCoMKgIC8qIFsxNF0gKi/CoMKgwqDCoMKgwqDCoMKgwqDCoCBbMTVd
ID0gImx3cCIsCj4gPiA+IC3CoMKgwqAgWzE2XSA9ICJmbWE0IizCoMKgwqDCoMKgwqAgWzE3XSA9
ICJ0Y2UiLAo+ID4gPiAtwqDCoMKgIC8qIFsxOF0gKi/CoMKgwqDCoMKgwqDCoMKgwqDCoCBbMTld
ID0gIm5vZGVpZCIsCj4gPiA+IC3CoMKgwqAgLyogWzIwXSAqL8KgwqDCoMKgwqDCoMKgwqDCoMKg
IFsyMV0gPSAidGJtIiwKPiA+ID4gLcKgwqDCoCBbMjJdID0gInRvcG9leHQiLMKgwqDCoCBbMjNd
ID0gInBlcmZjdHItY29yZSIsCj4gPiA+IC3CoMKgwqAgWzI0XSA9ICJwZXJmY3RyLW5iIiwgLyog
WzI1XSAqLwo+ID4gPiAtwqDCoMKgIFsyNl0gPSAiZGJ4IizCoMKgwqDCoMKgwqDCoCBbMjddID0g
InBlcmZ0c2MiLAo+ID4gPiAtwqDCoMKgIFsyOF0gPSAicGN4LWwyaSIswqDCoMKgIFsyOV0gPSAi
bW9uaXRvcngiLAo+ID4gPiAtwqDCoMKgIFszMF0gPSAiYWRkci1tc2stZXh0IiwKPiA+ID4gLX07
Cj4gPiA+IC0KPiA+ID4gLXN0YXRpYyBjb25zdCBjaGFyICpjb25zdCBzdHJfN2IwWzMyXSA9Cj4g
PiA+IC17Cj4gPiA+IC3CoMKgwqAgWyAwXSA9ICJmc2dzYmFzZSIsIFsgMV0gPSAidHNjLWFkaiIs
Cj4gPiA+IC3CoMKgwqAgWyAyXSA9ICJzZ3giLMKgwqDCoMKgwqAgWyAzXSA9ICJibWkxIiwKPiA+
ID4gLcKgwqDCoCBbIDRdID0gImhsZSIswqDCoMKgwqDCoCBbIDVdID0gImF2eDIiLAo+ID4gPiAt
wqDCoMKgIFsgNl0gPSAiZmRwLWV4biIswqAgWyA3XSA9ICJzbWVwIiwKPiA+ID4gLcKgwqDCoCBb
IDhdID0gImJtaTIiLMKgwqDCoMKgIFsgOV0gPSAiZXJtcyIsCj4gPiA+IC3CoMKgwqAgWzEwXSA9
ICJpbnZwY2lkIizCoCBbMTFdID0gInJ0bSIsCj4gPiA+IC3CoMKgwqAgWzEyXSA9ICJwcW0iLMKg
wqDCoMKgwqAgWzEzXSA9ICJkZXBmcHAiLAo+ID4gPiAtwqDCoMKgIFsxNF0gPSAibXB4IizCoMKg
wqDCoMKgIFsxNV0gPSAicHFlIiwKPiA+ID4gLcKgwqDCoCBbMTZdID0gImF2eDUxMmYiLMKgIFsx
N10gPSAiYXZ4NTEyZHEiLAo+ID4gPiAtwqDCoMKgIFsxOF0gPSAicmRzZWVkIizCoMKgIFsxOV0g
PSAiYWR4IiwKPiA+ID4gLcKgwqDCoCBbMjBdID0gInNtYXAiLMKgwqDCoMKgIFsyMV0gPSAiYXZ4
NTEyLWlmbWEiLAo+ID4gPiAtwqDCoMKgIFsyMl0gPSAicGNvbW1pdCIswqAgWzIzXSA9ICJjbGZs
dXNob3B0IiwKPiA+ID4gLcKgwqDCoCBbMjRdID0gImNsd2IiLMKgwqDCoMKgIFsyNV0gPSAicHJv
Yy10cmFjZSIsCj4gPiA+IC3CoMKgwqAgWzI2XSA9ICJhdng1MTJwZiIsIFsyN10gPSAiYXZ4NTEy
ZXIiLAo+ID4gPiAtwqDCoMKgIFsyOF0gPSAiYXZ4NTEyY2QiLCBbMjldID0gInNoYSIsCj4gPiA+
IC3CoMKgwqAgWzMwXSA9ICJhdng1MTJidyIsIFszMV0gPSAiYXZ4NTEydmwiLAo+ID4gPiAtfTsK
PiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl9EYTFbMzJdID0KPiA+
ID4gLXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gInhzYXZlb3B0IiwgWyAxXSA9ICJ4c2F2ZWMiLAo+
ID4gPiAtwqDCoMKgIFsgMl0gPSAieGdldGJ2MSIswqAgWyAzXSA9ICJ4c2F2ZXMiLAo+ID4gPiAt
fTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl83YzBbMzJdID0K
PiA+ID4gLXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gInByZWZldGNod3QxIizCoMKgwqDCoMKgIFsg
MV0gPSAiYXZ4NTEyLXZibWkiLAo+ID4gPiAtwqDCoMKgIFsgMl0gPSAidW1pcCIswqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFsgM10gPSAicGt1IiwKPiA+ID4gLcKgwqDCoCBbIDRdID0gIm9zcGtl
IizCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsgNV0gPSAid2FpdHBrZyIsCj4gPiA+IC3CoMKgwqAg
WyA2XSA9ICJhdng1MTItdmJtaTIiLMKgwqDCoMKgIFsgN10gPSAiY2V0LXNzIiwKPiA+ID4gLcKg
wqDCoCBbIDhdID0gImdmbmkiLMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBbIDldID0gInZhZXMi
LAo+ID4gPiAtwqDCoMKgIFsxMF0gPSAidnBjbG11bHFkcSIswqDCoMKgwqDCoMKgIFsxMV0gPSAi
YXZ4NTEyLXZubmkiLAo+ID4gPiAtwqDCoMKgIFsxMl0gPSAiYXZ4NTEyLWJpdGFsZyIsCj4gPiA+
IC3CoMKgwqAgWzE0XSA9ICJhdng1MTItdnBvcGNudGRxIiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKg
IFsyMl0gPSAicmRwaWQiLAo+ID4gPiAtwqDCoMKgIC8qIDI0ICovwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIFsyNV0gPSAiY2xkZW1vdGUiLAo+ID4gPiAtwqDCoMKgIC8qIDI2
ICovwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyN10gPSAibW92ZGlyaSIs
Cj4gPiA+IC3CoMKgwqAgWzI4XSA9ICJtb3ZkaXI2NGIiLMKgwqDCoMKgwqDCoMKgIFsyOV0gPSAi
ZW5xY21kIiwKPiA+ID4gLcKgwqDCoCBbMzBdID0gInNneC1sYyIswqDCoMKgwqDCoMKgwqDCoMKg
wqAgWzMxXSA9ICJwa3MiLAo+ID4gPiAtfTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNo
YXIgKmNvbnN0IHN0cl9lN2RbMzJdID0KPiA+ID4gLXsKPiA+ID4gLcKgwqDCoCAvKiA2ICovwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgWyA3XSA9ICJody1wc3RhdGUiLAo+
ID4gPiAtwqDCoMKgIFsgOF0gPSAiaXRzYyIswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsgOV0g
PSAiY3BiIiwKPiA+ID4gLcKgwqDCoCBbMTBdID0gImVmcm8iLAo+ID4gPiAtfTsKPiA+ID4gLQo+
ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl9lOGJbMzJdID0KPiA+ID4gLXsKPiA+
ID4gLcKgwqDCoCBbIDBdID0gImNsemVybyIsCj4gPiA+IC3CoMKgwqAgWyAyXSA9ICJyc3RyLWZw
LWVyci1wdHJzIiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKgIC8qIFsgOF0gKi/CoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBbIDldID0gIndibm9pbnZkIiwKPiA+ID4gLQo+ID4gPiAtwqDC
oMKgIFsxMl0gPSAiaWJwYiIsCj4gPiA+IC3CoMKgwqAgWzE0XSA9ICJpYnJzIizCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgWzE1XSA9ICJhbWQtc3RpYnAiLAo+ID4gPiAtwqDCoMKgIFsxNl0gPSAi
aWJycy1hbHdheXMiLMKgwqDCoMKgwqAgWzE3XSA9ICJzdGlicC1hbHdheXMiLAo+ID4gPiAtwqDC
oMKgIFsxOF0gPSAiaWJycy1mYXN0IizCoMKgwqDCoMKgwqDCoCBbMTldID0gImlicnMtc2FtZS1t
b2RlIiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKgIFsyMF0gPSAibm8tbG1zbCIsCj4gPiA+IC3CoMKg
wqAgLyogWzIyXSAqL8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyM10gPSAicHBp
biIsCj4gPiA+IC3CoMKgwqAgWzI0XSA9ICJhbWQtc3NiZCIswqDCoMKgwqDCoMKgwqDCoCBbMjVd
ID0gInZpcnQtc3NiZCIsCj4gPiA+IC3CoMKgwqAgWzI2XSA9ICJzc2Itbm8iLAo+ID4gPiAtwqDC
oMKgIFsyOF0gPSAicHNmZCIswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyOV0gPSAiYnRjLW5v
IiwKPiA+ID4gLcKgwqDCoCBbMzBdID0gImlicGItcmV0IiwKPiA+ID4gLX07Cj4gPiA+IC0KPiA+
ID4gLXN0YXRpYyBjb25zdCBjaGFyICpjb25zdCBzdHJfN2QwWzMyXSA9Cj4gPiA+IC17Cj4gPiA+
IC3CoMKgwqAgWyAyXSA9ICJhdng1MTItNHZubml3IiwgWyAzXSA9ICJhdng1MTItNGZtYXBzIiwK
PiA+ID4gLcKgwqDCoCBbIDRdID0gImZzcm0iLAo+ID4gPiAtCj4gPiA+IC3CoMKgwqAgWyA4XSA9
ICJhdng1MTItdnAyaW50ZXJzZWN0IiwgWyA5XSA9ICJzcmJkcy1jdHJsIiwKPiA+ID4gLcKgwqDC
oCBbMTBdID0gIm1kLWNsZWFyIizCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsxMV0gPSAicnRtLWFs
d2F5cy1hYm9ydCIsCj4gPiA+IC3CoMKgwqAgLyogMTIgKi/CoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgWzEzXSA9ICJ0c3gtZm9yY2UtYWJvcnQiLAo+ID4gPiAtwqDCoMKgIFsxNF0gPSAi
c2VyaWFsaXplIizCoMKgwqDCoCBbMTVdID0gImh5YnJpZCIsCj4gPiA+IC3CoMKgwqAgWzE2XSA9
ICJ0c3hsZHRyayIsCj4gPiA+IC3CoMKgwqAgWzE4XSA9ICJwY29uZmlnIiwKPiA+ID4gLcKgwqDC
oCBbMjBdID0gImNldC1pYnQiLAo+ID4gPiAtwqDCoMKgIC8qIDIyICovwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFsyM10gPSAiYXZ4NTEyLWZwMTYiLAo+ID4gPiAtCj4gPiA+IC3CoMKg
wqAgWzI2XSA9ICJpYnJzYiIswqDCoMKgwqDCoMKgwqDCoCBbMjddID0gInN0aWJwIiwKPiA+ID4g
LcKgwqDCoCBbMjhdID0gImwxZC1mbHVzaCIswqDCoMKgwqAgWzI5XSA9ICJhcmNoLWNhcHMiLAo+
ID4gPiAtwqDCoMKgIFszMF0gPSAiY29yZS1jYXBzIizCoMKgwqDCoCBbMzFdID0gInNzYmQiLAo+
ID4gPiAtfTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl83YTFb
MzJdID0KPiA+ID4gLXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gInNoYTUxMiIswqDCoMKgwqDCoMKg
wqAgWyAxXSA9ICJzbTMiLAo+ID4gPiAtwqDCoMKgIFsgMl0gPSAic200IiwKPiA+ID4gLcKgwqDC
oCBbIDRdID0gImF2eC12bm5pIizCoMKgwqDCoMKgIFsgNV0gPSAiYXZ4NTEyLWJmMTYiLAo+ID4g
PiAtCj4gPiA+IC3CoMKgwqAgWzEwXSA9ICJmenJtIizCoMKgwqDCoMKgwqDCoMKgwqAgWzExXSA9
ICJmc3JzIiwKPiA+ID4gLcKgwqDCoCBbMTJdID0gImZzcmNzIiwKPiA+ID4gLQo+ID4gPiAtwqDC
oMKgIC8qIDE4ICovwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsxOV0gPSAid3Jtc3Ju
cyIsCj4gPiA+IC0KPiA+ID4gLcKgwqDCoCAvKiAyMiAqL8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCBbMjNdID0gImF2eC1pZm1hIiwKPiA+ID4gLX07Cj4gPiA+ICsjaW5jbHVkZSA8eGVu
L2xpYi94ODYvY3B1LXBvbGljeS5oPgo+ID4gPiDCoAo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIg
KmNvbnN0IHN0cl9lMjFhWzMyXSA9Cj4gPiA+IC17Cj4gPiA+IC3CoMKgwqAgWyAwXSA9ICJuby1u
ZXN0LWJwIizCoMKgwqAgWyAxXSA9ICJmcy1ncy1ucyIsCj4gPiA+IC3CoMKgwqAgWyAyXSA9ICJs
ZmVuY2UrIiwKPiA+ID4gLcKgwqDCoCBbIDZdID0gIm5zY2IiLAo+ID4gPiAtwqDCoMKgIFsgOF0g
PSAiYXV0by1pYnJzIiwKPiA+ID4gLcKgwqDCoCBbMTBdID0gImFtZC1mc3JzIizCoMKgwqDCoMKg
IFsxMV0gPSAiYW1kLWZzcmMiLAo+ID4gPiAtCj4gPiA+IC3CoMKgwqAgLyogMTYgKi/CoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgWzE3XSA9ICJjcHVpZC11c2VyLWRpcyIsCj4gPiA+IC3C
oMKgwqAgWzE4XSA9ICJlcHNmIizCoMKgwqDCoMKgwqDCoMKgwqAgWzE5XSA9ICJmc3JzYyIsCj4g
PiA+IC3CoMKgwqAgWzIwXSA9ICJhbWQtcHJlZmV0Y2hpIiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKg
IC8qIDI2ICovwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyN10gPSAic2JwYiIsCj4g
PiA+IC3CoMKgwqAgWzI4XSA9ICJpYnBiLWJydHlwZSIswqDCoCBbMjldID0gInNyc28tbm8iLAo+
ID4gPiAtfTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl83YjFb
MzJdID0KPiA+ID4gLXsKPiA+ID4gLcKgwqDCoCBbIDBdID0gInBwaW4iLAo+ID4gPiAtfTsKPiA+
ID4gLQo+ID4gPiAtc3RhdGljIGNvbnN0IGNoYXIgKmNvbnN0IHN0cl83YzFbMzJdID0KPiA+ID4g
LXsKPiA+ID4gLX07Cj4gPiA+IC0KPiA+ID4gLXN0YXRpYyBjb25zdCBjaGFyICpjb25zdCBzdHJf
N2QxWzMyXSA9Cj4gPiA+IC17Cj4gPiA+IC3CoMKgwqAgWyA0XSA9ICJhdngtdm5uaS1pbnQ4IizC
oMKgwqDCoMKgwqAgWyA1XSA9ICJhdngtbmUtY29udmVydCIsCj4gPiA+IC0KPiA+ID4gLcKgwqDC
oCBbMTBdID0gImF2eC12bm5pLWludDE2IiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKgIFsxNF0gPSAi
cHJlZmV0Y2hpIiwKPiA+ID4gLQo+ID4gPiAtwqDCoMKgIFsxOF0gPSAiY2V0LXNzcyIsCj4gPiA+
IC19Owo+ID4gPiAtCj4gPiA+IC1zdGF0aWMgY29uc3QgY2hhciAqY29uc3Qgc3RyXzdkMlszMl0g
PQo+ID4gPiAtewo+ID4gPiAtwqDCoMKgIFsgMF0gPSAiaW50ZWwtcHNmZCIswqDCoMKgIFsgMV0g
PSAiaXByZWQtY3RybCIsCj4gPiA+IC3CoMKgwqAgWyAyXSA9ICJycnNiYS1jdHJsIizCoMKgwqAg
WyAzXSA9ICJkZHAtY3RybCIsCj4gPiA+IC3CoMKgwqAgWyA0XSA9ICJiaGktY3RybCIswqDCoMKg
wqDCoCBbIDVdID0gIm1jZHQtbm8iLAo+ID4gPiAtfTsKPiA+ID4gLQo+ID4gPiAtc3RhdGljIGNv
bnN0IGNoYXIgKmNvbnN0IHN0cl9tMTBBbFszMl0gPQo+ID4gPiAtewo+ID4gPiAtwqDCoMKgIFsg
MF0gPSAicmRjbC1ubyIswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsgMV0gPSAiZWlicnMiLAo+
ID4gPiAtwqDCoMKgIFsgMl0gPSAicnNiYSIswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IFsgM10gPSAic2tpcC1sMWRmbCIsCj4gPiA+IC3CoMKgwqAgWyA0XSA9ICJpbnRlbC1zc2Itbm8i
LMKgwqDCoMKgwqDCoMKgIFsgNV0gPSAibWRzLW5vIiwKPiA+ID4gLcKgwqDCoCBbIDZdID0gImlm
LXBzY2hhbmdlLW1jLW5vIizCoMKgIFsgN10gPSAidHN4LWN0cmwiLAo+ID4gPiAtwqDCoMKgIFsg
OF0gPSAidGFhLW5vIizCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBbIDldID0gIm1jdS1jdHJs
IiwKPiA+ID4gLcKgwqDCoCBbMTBdID0gIm1pc2MtcGtnLWN0cmwiLMKgwqDCoMKgwqDCoCBbMTFd
ID0gImVuZXJneS1jdHJsIiwKPiA+ID4gLcKgwqDCoCBbMTJdID0gImRvaXRtIizCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIFsxM10gPSAic2Jkci1zc2RwLW5vIiwKPiA+ID4gLcKgwqDCoCBb
MTRdID0gImZic2RwLW5vIizCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsxNV0gPSAicHNkcC1ubyIs
Cj4gPiA+IC3CoMKgwqAgLyogMTYgKi/CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgWzE3XSA9ICJmYi1jbGVhciIsCj4gPiA+IC3CoMKgwqAgWzE4XSA9ICJmYi1jbGVh
ci1jdHJsIizCoMKgwqDCoMKgwqAgWzE5XSA9ICJycnNiYSIsCj4gPiA+IC3CoMKgwqAgWzIwXSA9
ICJiaGktbm8iLMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyMV0gPSAieGFwaWMtc3RhdHVz
IiwKPiA+ID4gLcKgwqDCoCAvKiAyMiAqL8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBbMjNdID0gIm92cmNsay1zdGF0dXMiLAo+ID4gPiAtwqDCoMKgIFsyNF0gPSAi
cGJyc2Itbm8iLMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgWzI1XSA9ICJnZHMtY3RybCIsCj4gPiA+
IC3CoMKgwqAgWzI2XSA9ICJnZHMtbm8iLMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFsyN10g
PSAicmZkcy1ubyIsCj4gPiA+IC3CoMKgwqAgWzI4XSA9ICJyZmRzLWNsZWFyIiwKPiA+ID4gLX07
Cj4gPiA+IC0KPiA+ID4gLXN0YXRpYyBjb25zdCBjaGFyICpjb25zdCBzdHJfbTEwQWhbMzJdID0K
PiA+ID4gLXsKPiA+ID4gLX07Cj4gPiA+ICtzdGF0aWMgdWludDMyX3QgbnJfZmVhdHVyZXM7Cj4g
PiA+IMKgCj4gPiA+IMKgc3RhdGljIGNvbnN0IHN0cnVjdCB7Cj4gPiA+IMKgwqDCoMKgIGNvbnN0
IGNoYXIgKm5hbWU7Cj4gPiA+IMKgwqDCoMKgIGNvbnN0IGNoYXIgKmFiYnI7Cj4gPiA+IC3CoMKg
wqAgY29uc3QgY2hhciAqY29uc3QgKnN0cnM7Cj4gPiAKPiA+IFdoaWxlIGhvdyB5b3UncmUgZG9p
bmcgaXQgbG9va3MgYWxsIHRlY2huaWNhbGx5IGNvcnJlY3QgKHNvIGV2ZW4KPiA+IHdpdGhvdXQK
PiA+IGNoYW5nZXMgSSBtYXkgbGF0ZXIgYWNrIHRoaXMgYXMgaXMpLCBJJ20gc3RpbGwgYSBsaXR0
bGUgcHV6emxlZC4gSQo+ID4gd2FzCj4gPiBraW5kIG9mIGV4cGVjdGluZyB4ZW4tY3B1aWQucHkg
dG8gYmUgZXh0ZW5kZWQgdG8gc3VwcGx5IG1vcmUKPiA+IGFub3RoZXIgKHNldAo+ID4gb2YpICNk
ZWZpbmUocykgbW9yZSBzdWl0YWJsZSBmb3IgdXNlIGhlcmUuIEluIHBhcnRpY3VsYXIsIHdoaWxl
Cj4gPiBwZXJmb3JtYW5jZSBzdXJlbHkgaXNuJ3Qgb2YgbXVjaCBjb25jZXJuIGluIHRoaXMgdG9v
bCwgLi4uCj4gPiAKPiA+ID4gQEAgLTMwMSwyMSArNTIsMzIgQEAgc3RhdGljIGNvbnN0IGNoYXIg
KmNvbnN0IGZzX25hbWVzW10gPSB7Cj4gPiA+IMKgwqDCoMKgIFtYRU5fU1lTQ1RMX2NwdV9mZWF0
dXJlc2V0X2h2bV9tYXhdID0gIkhWTSBNYXgiLAo+ID4gPiDCoH07Cj4gPiA+IMKgCj4gPiA+IC1z
dGF0aWMgdm9pZCBkdW1wX2xlYWYodWludDMyX3QgbGVhZiwgY29uc3QgY2hhciAqY29uc3QgKnN0
cnMpCj4gPiA+ICtzdGF0aWMgY29uc3QgY2hhciAqZmluZF9uYW1lKHVuc2lnbmVkIGludCBpbmRl
eCkKPiA+ID4gwqB7Cj4gPiA+IC3CoMKgwqAgdW5zaWduZWQgaTsKPiA+ID4gK8KgwqDCoCBzdGF0
aWMgY29uc3Qgc3RydWN0IGZlYXR1cmVfbmFtZSB7Cj4gPiA+ICvCoMKgwqDCoMKgwqDCoCBjb25z
dCBjaGFyICpuYW1lOwo+ID4gPiArwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgaW50IGJpdDsKPiA+
ID4gK8KgwqDCoCB9IGZlYXR1cmVfbmFtZXNbXSA9IElOSVRfRkVBVFVSRV9OQU1FUzsKPiA+ID4g
K8KgwqDCoCB1bnNpZ25lZCBpbnQgaTsKPiA+ID4gwqAKPiA+ID4gLcKgwqDCoCBpZiAoICFzdHJz
ICkKPiA+ID4gLcKgwqDCoCB7Cj4gPiA+IC3CoMKgwqDCoMKgwqDCoCBwcmludGYoIiA/Pz8iKTsK
PiA+ID4gLcKgwqDCoMKgwqDCoMKgIHJldHVybjsKPiA+ID4gLcKgwqDCoCB9Cj4gPiA+ICvCoMKg
wqAgZm9yICggaSA9IDA7IGkgPCBBUlJBWV9TSVpFKGZlYXR1cmVfbmFtZXMpOyBpKysgKQo+ID4g
PiArwqDCoMKgwqDCoMKgwqAgaWYgKCBmZWF0dXJlX25hbWVzW2ldLmJpdCA9PSBpbmRleCApCj4g
PiA+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiBmZWF0dXJlX25hbWVzW2ldLm5hbWU7
Cj4gPiAKPiA+IC4uLiBhIGxpbmVhciBzZWFyY2gsIHJlcGVhdGVkIHBlcmhhcHMgaHVuZHJlZHMg
b2YgdGltZXMsIGxvb2tzCj4gPiBzdGlsbCBhCj4gPiBsaXR0bGUgb2RkIHRvIG1lLgo+IAo+IEkg
ZGlkbid0IGJlbmNobWFyayB3aGF0IGtpbmQgb2YgcGVyZm9ybWFuY2UgaW1wYWN0IHRoaXMgY2hh
bmdlIHdvdWxkCj4gaGF2ZSBvbiB0aGUgdG9vbCwgYnV0IEkgZGlkbid0IHRoaW5rIGl0IHdhcyB0
aGF0IHJlbGV2YW50LCBhcyB0aGlzIGlzCj4gYSBkaWFnbm9zdGljL2RlYnVnIHRvb2wsIGFuZCBo
ZW5jZSBwZXJmb3JtYW5jZSAodW5sZXNzIGl0IHRvb2sKPiBzZWNvbmRzCj4gdG8gZXhlY3V0ZSkg
c2hvdWxkbid0IGJlIHRoYXQgaW1wb3J0YW50Lgo+IAo+IEkgY291bGQgc3dpdGNoIHRvIGEgbm9u
LWNvbnN0IGFycmF5IGFuZCBzb3J0IGl0IGF0IHRoZSBzdGFydCBpbiBvcmRlcgo+IHRvIGRvIGEg
YmluYXJ5IHNlYXJjaCwgYnV0IHRoYXQgbWlnaHQgYmUgb3ZlciBlbmdpbmVlcmluZyBpdC4KPiAK
PiA+ID4gK3N0YXRpYyB2b2lkIGR1bXBfbGVhZih1aW50MzJfdCBsZWFmLCB1bnNpZ25lZCBpbnQg
aW5kZXgpCj4gPiA+ICt7Cj4gPiA+ICvCoMKgwqAgdW5zaWduZWQgaTsKPiA+ID4gwqAKPiA+ID4g
wqDCoMKgwqAgZm9yICggaSA9IDA7IGkgPCAzMjsgKytpICkKPiA+ID4gwqDCoMKgwqDCoMKgwqDC
oCBpZiAoIGxlYWYgJiAoMXUgPDwgaSkgKQo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgIHsKPiA+ID4g
LcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaWYgKCBzdHJzW2ldICkKPiA+ID4gLcKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBwcmludGYoIiAlcyIsIHN0cnNbaV0pOwo+ID4gPiArwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCBjb25zdCBjaGFyICpuYW1lID0gZmluZF9uYW1lKGluZGV4ICogMzIg
KyBpKTsKPiA+ID4gKwo+ID4gPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpZiAoIG5hbWUgKQo+
ID4gPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHByaW50ZigiICVzIiwgbmFtZSk7
Cj4gPiA+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBlbHNlCj4gPiA+IMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIHByaW50ZigiIDwldT4iLCBpKTsKPiA+ID4gwqDCoMKgwqDCoMKg
wqDCoCB9Cj4gPiA+IEBAIC0zMjYsNiArODgsNyBAQCBzdGF0aWMgdm9pZCBkZWNvZGVfZmVhdHVy
ZXNldChjb25zdCB1aW50MzJfdAo+ID4gPiAqZmVhdHVyZXMsCj4gPiA+IMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBjaGFy
ICpuYW1lLAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgYm9vbCBkZXRhaWwpCj4gPiA+IMKgewo+ID4gPiArwqDCoMKgIHN0
YXRpYyBjb25zdCB1aW50MzJfdCBrbm93bl9mZWF0dXJlc1tdID0KPiA+ID4gSU5JVF9LTk9XTl9G
RUFUVVJFUzsKPiA+ID4gwqDCoMKgwqAgdW5zaWduZWQgaW50IGk7Cj4gPiAKPiA+IFNvIHRoaXMg
dmFyaWFibGUgZXhpc3RzIHNvbGVseSB0byAuLi4KPiA+IAo+ID4gPiBAQCAtMzM2LDExICs5OSwx
NCBAQCBzdGF0aWMgdm9pZCBkZWNvZGVfZmVhdHVyZXNldChjb25zdCB1aW50MzJfdAo+ID4gPiAq
ZmVhdHVyZXMsCj4gPiA+IMKgwqDCoMKgIGlmICggIWRldGFpbCApCj4gPiA+IMKgwqDCoMKgwqDC
oMKgwqAgcmV0dXJuOwo+ID4gPiDCoAo+ID4gPiAtwqDCoMKgIGZvciAoIGkgPSAwOyBpIDwgbGVu
Z3RoICYmIGkgPCBBUlJBWV9TSVpFKGRlY29kZXMpOyArK2kgKQo+ID4gPiArwqDCoMKgIC8qIEVu
c3VyZSBsZWFmIG5hbWVzIHN0YXkgaW4gc3luYyB3aXRoIHRoZSBwb2xpY3kgbGVhZgo+ID4gPiBj
b3VudC4gKi8KPiA+ID4gK8KgwqDCoCBCVUlMRF9CVUdfT04oQVJSQVlfU0laRShrbm93bl9mZWF0
dXJlcykgIT0KPiA+ID4gQVJSQVlfU0laRShsZWFmX25hbWVzKSk7Cj4gPiAKPiA+IC4uLiBjYWxj
dWxhdGUgaXRzIHNpemUgaGVyZS4gVGh1cyByZWx5aW5nIG9uIHRoZSBjb21waWxlciB0byBub3QK
PiA+IGZsYWcKPiA+IHN1Y2ggZWZmZWN0aXZlbHkgdW51c2VkIHN0YXRpYyBjb25zdCB2YXJpYWJs
ZXMuCj4gCj4gSSB3b25kZXJlZCB3aGV0aGVyIHRvIGFkZCB0aGUgdW51c2VkIGF0dHJpYnV0ZSwg
YnV0IHNlZWluZyBhcyBnaXRsYWIKPiBkaWRuJ3QgY29tcGxhaW4gSSd2ZSBmb3Jnb3QgdG8gYWRk
IGl0LsKgIEkgY291bGQgYWRkIGl0Lgo+IAo+ID4gPiArwqDCoMKgIGZvciAoIGkgPSAwOyBpIDwg
bGVuZ3RoICYmIGkgPCBBUlJBWV9TSVpFKGxlYWZfbmFtZXMpOyArK2kgKQo+ID4gPiDCoMKgwqDC
oCB7Cj4gPiA+IC3CoMKgwqDCoMKgwqDCoCBwcmludGYoIsKgIFslMDJ1XSAlLSJDT0xfQUxJR04i
cyIsIGksIGRlY29kZXNbaV0ubmFtZSA/Ogo+ID4gPiAiPFVOS05PV04+Iik7Cj4gPiA+IC3CoMKg
wqDCoMKgwqDCoCBpZiAoIGRlY29kZXNbaV0ubmFtZSApCj4gPiA+IC3CoMKgwqDCoMKgwqDCoMKg
wqDCoMKgIGR1bXBfbGVhZihmZWF0dXJlc1tpXSwgZGVjb2Rlc1tpXS5zdHJzKTsKPiA+ID4gK8Kg
wqDCoMKgwqDCoMKgIHByaW50ZigiwqAgWyUwMnVdICUtIkNPTF9BTElHTiJzIiwgaSwgbGVhZl9u
YW1lc1tpXS5uYW1lCj4gPiA+ID86ICI8VU5LTk9XTj4iKTsKPiA+IAo+ID4gSSByZWFsaXplIHlv
dSBtZXJlbHkgdHJhbnNmb3JtIHdoYXQgaGFzIGJlZW4gdGhlcmUsIGJ1dCBkbyB3ZQo+ID4gcmVh
bGx5Cj4gPiBuZWVkIHRoaXMgZmFsbGJhY2sgdG8gIjxVTktOT1dOPiIgaGVyZSBhbmQgLi4uCj4g
PiAKPiA+ID4gQEAgLTM1NSw4ICsxMjEsOCBAQCBzdGF0aWMgdm9pZCBkdW1wX2luZm8oeGNfaW50
ZXJmYWNlICp4Y2gsIGJvb2wKPiA+ID4gZGV0YWlsKQo+ID4gPiDCoMKgwqDCoCBpZiAoICFkZXRh
aWwgKQo+ID4gPiDCoMKgwqDCoCB7Cj4gPiA+IMKgwqDCoMKgwqDCoMKgwqAgcHJpbnRmKCLCoMKg
wqDCoMKgwqAgJSJDT0xfQUxJR04icyAiLCAiS0VZIik7Cj4gPiA+IC3CoMKgwqDCoMKgwqDCoCBm
b3IgKCBpID0gMDsgaSA8IEFSUkFZX1NJWkUoZGVjb2Rlcyk7ICsraSApCj4gPiA+IC3CoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIHByaW50ZigiJS04cyAiLCBkZWNvZGVzW2ldLmFiYnIgPzogIj8/PyIp
Owo+ID4gPiArwqDCoMKgwqDCoMKgwqAgZm9yICggaSA9IDA7IGkgPCBBUlJBWV9TSVpFKGxlYWZf
bmFtZXMpOyArK2kgKQo+ID4gPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwcmludGYoIiUtOHMg
IiwgbGVhZl9uYW1lc1tpXS5hYmJyID86ICI/Pz8iKTsKPiA+IAo+ID4gdG8gIj8/PyIgaGVyZT8g
VGhlIHRhYmxlIGVudHJpZXMgYXJlIGFsbCBmdWxseSBwb3B1bGF0ZWQsIGFuZCBJCj4gPiBkb24n
dAo+ID4gc2VlIHdoeSBmdXR1cmUgb25lcyB3b3VsZG4ndCBiZS4gSU9XIEkgZG9uJ3QgdGhpbmsg
SSBzZWUgd2hhdCBnb29kCj4gPiB0aGlzCj4gPiBpcyBkb2luZyB1cywgd2hlbiBhdCB0aGUgc2Ft
ZSB0aW1lIGl0IGRvZXNuJ3QgaGVscCByZWFkYWJpbGl0eS4KPiAKPiBUQkggSSB3YXNuJ3Qgc3Vy
ZSBhYm91dCB0aGUgaW50ZW50aW9uIGhlcmUsIHNvIGRlY2lkZWQgdG8gbGVhdmUgdGhvc2UKPiBh
bG9uZS7CoCBUaGUgaXRlcmF0aW9uIGlzIGRvbmUgYWdhaW5zdCBsZWFmX25hbWVzIChvciBkZWNv
ZGVzKSwgc28KPiB0aGlzCj4gd291bGQgbWVhbiB0aGVyZSdzIGFuIGFycmF5IGVudHJ5IGluIHRo
ZSB0YWJsZSB0aGF0IGV4cGxpY2l0bHkgaGFzCj4gbmFtZSBvciBhYmJyIGZpZWxkcyB1bnNldCwg
d2hpY2ggd291bGQgYmUgd2VpcmQgdG8gbWUuCj4gCj4gQnV0IGFnYWluIHdpdGhvdXQga25vd2lu
ZyB0aGUgb3JpZ2luYWwgaW50ZW50aW9uIG9mIHRob3NlLCBhbmQgdGhlCj4gY2hhbmdlIGJlaW5n
IHRhbmdlbnRpYWwgdG8gdGhlIHB1cnBvc2UgSSd2ZSBkZWNpZGVkIHRvIGxlYXZlIHRob3NlCj4g
YWxvbmUuCj4gCj4gVGhhbmtzLCBSb2dlci4KCg==



From xen-devel-bounces@lists.xenproject.org Thu May 02 08:57:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:57:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715669.1117459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SFj-0003PE-3B; Thu, 02 May 2024 08:57:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715669.1117459; Thu, 02 May 2024 08:57:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SFj-0003P7-02; Thu, 02 May 2024 08:57:19 +0000
Received: by outflank-mailman (input) for mailman id 715669;
 Thu, 02 May 2024 08:57:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=216C=MF=epam.com=prvs=1852dd919c=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s2SFh-0003P1-9Y
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:57:17 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f82d6c2c-0861-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:57:15 +0200 (CEST)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4427mjg8010110;
 Thu, 2 May 2024 08:57:04 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3xv6s0g6uj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 02 May 2024 08:57:04 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by VI1PR03MB9939.eurprd03.prod.outlook.com (2603:10a6:800:1ce::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.27; Thu, 2 May
 2024 08:57:00 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::b309:8ea5:85c3:d04f]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::b309:8ea5:85c3:d04f%5]) with mapi id 15.20.7544.029; Thu, 2 May 2024
 08:57:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f82d6c2c-0861-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nK3B0hBj2B9sZldpdWjD9odigTiylzTpKbZ0967gs6nZZAMd/VXqtirktlM/OuPL+YN5PAUQMEm1tmqzycfMwKp7x4UZ+Zh3EjkMqegXbnf89nDHLu54ePdUhVV9Q9eSzwddOxhR3qTez7DSs6bu6X9D9gvd9volPMGnjNhw3q3aDIuANoEVJh9Q/lAe1CWY4IliKVGfvehp5D43kC8DnczgUTm0uN7abT5n8i4sYydx+/NJ5OjTirB7XSQKYGTf1Hq9wq25QcsWlnkt4lK1qeyX0RC1zBW9WXQBUjznlTUwg9nAGSHTXU7yAczc9R6+i6RuzF/ngp0RDjGJCiB5HQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=nILuJksBCbpy8EVtH9xQtajrWAAhX1Jwrb90HNRwaF8=;
 b=B+JwxXIDGr2p/lGUT2MEIA4N2GJSiiGnClXh2u87GDTZ09iv/H3YwWlIZrWM5//HkcTxGNRz7uLBoyGomIdoct4e+Nr2qVN9ea2B+XnYv4ebDHn+U6MMOZE/6yOIYZbQdse+P0XdGACYW0PrwYCAJeIB+X+GYI9DxgE/HPyaJI0a92C/846crYc2fJrCYZ6eywfkwb5PztHhoYZUJaudqe8UWkjSBkfrHR9I6IYXFVE3TtCyr8bSUJYTvUBe7kPHuNxDwblGr5n8HdsF/AtonmiGali9crloVI6NaeSIocwo5Jn/X3yIyknmGenk0JfBXFvbASdEVKWUTMKBFJNwlQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nILuJksBCbpy8EVtH9xQtajrWAAhX1Jwrb90HNRwaF8=;
 b=nxmaUOc1GWG4v7HCyzUHI6ibZ8BfvVK4wSpkLOzlH6n2ziNqLKjT9wvnIBASlVX3bLuWRopUjrd++v0TATZU2rpp5y+xGOX76OMvmkPDv76nBBQQjCJPJLc7hZlZ8JIXEIwOhAa5uJCbTP9FARREHgSueHx8kMqGqsrNJMPSBiHsxdKWmKLEdG3XdmwokasXID0YPHAktseMRpMB+l78KYGJDikZMb4dV9SmeYGvRnu+crWQPybGw8bSQeBcYpByXhZb7zvi5NplPJnjs2HuWLeqfgf0s0+b+4JukjJGrnuAqTv+oQm92nCY/My7NH8ItcHlLqu75rVtukhnchBcow==
Message-ID: <4e7680b2-d165-4f35-9af2-3c2878eb59e4@epam.com>
Date: Thu, 2 May 2024 11:56:58 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v1 4/7] x86/MCE: guard lmce_support/cmci_support
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1713860310.git.Sergiy_Kibrik@epam.com>
 <d5fa50b3056b96f3046be39ed682a8b347f1b425.1713860310.git.Sergiy_Kibrik@epam.com>
 <79834875-d26e-42c7-9ffa-868a84a07dfb@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <79834875-d26e-42c7-9ffa-868a84a07dfb@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA0P291CA0002.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1::29) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|VI1PR03MB9939:EE_
X-MS-Office365-Filtering-Correlation-Id: f9a2980c-ac5c-418e-ac35-08dc6a85d456
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?NlZyY1ZxOGM5V0hFbDZzb29TUjdTajUrZDVCQlVlY0JLYStnYnNYd0FSZjhB?=
 =?utf-8?B?ZEJLS29Ha28xd1o3MTV6bWlkT3VUbmxwL3JFYllOdnNuSDIvTjZrQjZmNmFX?=
 =?utf-8?B?U0E0UFhGUXRSUlJsc2Z4Tk83eFpRWVBTN09FY1ZNejY2bUQ0NXE5THR2L3ZS?=
 =?utf-8?B?Nm5Mc3hVYWdKbUFkMDRjM2xhUDdzbGNvVzd4cFowTWxIUUZibTdBY1p3c2Ir?=
 =?utf-8?B?aGVZQ3VteDFpUDU1blZVZzZVTnc2TXNyL1ZVV0ZaUTJkRDZZaHlzeHU5RTFT?=
 =?utf-8?B?MktlUVZKOUVZS0R3S05sLzd4OFJKRmZzWTNlaG5OK3ZoNGs0ZG9oaVFjM2pt?=
 =?utf-8?B?OFRIbGhIV1ExbUhNKytaeEpUNmxwaVYzTE1vbnduajQ5YVd5V1FZZVp2M05B?=
 =?utf-8?B?UkJvSmZOV1Vrcllvbm5qNkdQSzYvakF3N01STVVIeHpxWjF4UEYySVhqZW9l?=
 =?utf-8?B?ZVJNUXErRnlGY05GSXRjVnM1aGtSdE1BN2VIU3kyOS8zSHk2a015cnBIWTNR?=
 =?utf-8?B?SnQ1TXRBVWZqTkdkVmxMOWk5T0FuSGI0bnpMcTBnUGRDd3AycXlSMWZ4Y1ZS?=
 =?utf-8?B?aWZ4NlVyaTVNU2pQVWx3NUZlVm4xWTU1NDJxUXhJZzMzR2VaL3F4Z1pwS3JF?=
 =?utf-8?B?ZXhMS21aOUQ4dUVFTk1NNnNYd243TkY0M3hRNkJNMnlDcTM5am9yMjVlRCtC?=
 =?utf-8?B?TTd0cGFOb3FkWHhVYVNTUXpLaVpNMzFubHdDaGx2RGFoVzJKYlpEZnFJTnJp?=
 =?utf-8?B?blJnN2pYUzFMei9JL1dMeW40aTVtZmhZWTR4V0VpNGE3aWVXU3dMUHRsVGwr?=
 =?utf-8?B?WjhxYUlIWHJkWVc1VlZtY3JJZlNmRUoxYWJVSEdkbUZlT0NCenRMRFlXUlNM?=
 =?utf-8?B?SmtIRTljR3dOTzIvUjNRVFdzL05ZZVROM1RqNkxsV1lDM1kzUXZrYms0S2Nk?=
 =?utf-8?B?ZjRVQ0FFS3lLWnlTUzczWnh1TnZhQTVMa01aVk9Bb3Y2aWlWbWIyazlwcXJJ?=
 =?utf-8?B?WGNCbVhHQmIwYjVUMHBpaGIrODlOZHlxeTVqQ096TE1xTlJDYkh5alBMYVRy?=
 =?utf-8?B?Kysvc3c1TURkRmZ5cGNDaTJTUjRXVno3U3ZSK2w0c2J0cEM3TWpJL2Q0QVVF?=
 =?utf-8?B?T0N6QWprMlRkTktEWDdkSFdwR09QRW5mTDNiL0tyOGtYbmJtK2VzQWVnR2dR?=
 =?utf-8?B?MlVyR2poc2N3U05maTI4WEVTZDBxMGJIdWdablpiRUdzWUptL1hBbTVPajZC?=
 =?utf-8?B?c01aTkd6RTJDazVtQ3JsZy9acDYyMlFzbkJ6U1ZTMXFzV3d6UGFNYlVlWWdB?=
 =?utf-8?B?cHorbE1PT0thc0Y1WVRIMm96c1U1SzJxZWlYVGM5d0R2SU1JbkVWZmI4WVpX?=
 =?utf-8?B?dWUxYkcrZmJhaHVmenhiSHpma3crNlF4Wm5kaEE2ZFNsQzBkcTFEdXNmVFBi?=
 =?utf-8?B?TWtnL1Z6Z0lLNzNGanJvWXErTmlYVkFaWTh2M3FBUGh3RkE4RFI2QmZ4UHVF?=
 =?utf-8?B?KzEzUDdDdDh6MzlqS3BKR2tGODhDSXJrVTdGQU4wVTluTXRaNnhiazRPMk9U?=
 =?utf-8?B?OXlnN1NRU283TkFpVXFoVmVHT0taSUdVT2xnRFpTbEVHaDlFUFMvUDl4NHpT?=
 =?utf-8?B?QzlkN1NWYkkwMUt6SHM0M0JlcnNUSlV0cHhIQlhkY2JLdUNXLzRuVWZud1Bp?=
 =?utf-8?B?NEhWTGpUWGpxSFpXMEM1TkJYNm9UUkFaRGtaUncxeVN5NWRSR1U5amdnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ajlzSXdQT090Mnp2bFczdzNZS1lDSUZNaUtlTk1IcTFXbUgzYVF0eCtOQUFu?=
 =?utf-8?B?NVZUTThpN21tblBjczQweXlYUjlEM3gxL01TYUxydWFaTks3R0owUnJKVm1u?=
 =?utf-8?B?SXRwYmNXcXN3SW14YTBTUlRGejdLeGV0WXBab3hCdkNaeU9BeE54MVpnc1Ex?=
 =?utf-8?B?alQ5UkM3SkNlUktHT2lrVnppaVI1cFpJM0E3YzlkMlpDVGtrT2syajRVdllx?=
 =?utf-8?B?TG83d2NvbmVsaXhGVk53Y2E2SmRtd3dneWxUcVN3Nk55OXNXMmpQWlNBQlZi?=
 =?utf-8?B?NVoyR2tjWXRzeWlWZ1pLTmVKRlpTRmt2NDQwQXdZTW5JQi8yRWtUUEVOdnhR?=
 =?utf-8?B?dGE4R1FjemorTDdUa1JXM1BaNnRTMDVvbWNzK08wT2ZPWmNoSjVrc3h2UHk4?=
 =?utf-8?B?QlhwQzd3UTNBRDRjSWJZa3ZmdnlKaXdZM09tKzJJRjlkMmNSTWsvZ2V5aW1R?=
 =?utf-8?B?RUV1aGpMVUMwV283SEczTmZFNEw1V050ZFh0VnU3c0pUbXZvVzlrNkJOMEZa?=
 =?utf-8?B?cXNua1FZQmhOK054VzgrYlFmMUtzTTFwU3lpeGFpWjRkL3R0MVpHRUJiWHRL?=
 =?utf-8?B?ZXBYZnBXUE54OXh6dmFYNXNDWDdRdjFFRUprYkpuSHNTcnQvUkxXQmdKZ1JN?=
 =?utf-8?B?QklCOUVDcXVuYllsTEZ2WWdrYTJ4QUZ0SWNlTG9IWHorSkZFd0YvbUtBRmY2?=
 =?utf-8?B?NGhvaC9pdytOeXQzQ3RaZmVEUnNQKzZPVU85NTd6RXpOcWJPZytsNTdaUGhI?=
 =?utf-8?B?SXdBWXk2bjhzUkkra0ZTUW9IajlGekJ3V1ZoWElneXFMU1NHdkhGQWhDbGtm?=
 =?utf-8?B?WVhvdm1nNWlYd3ZNYlcxYmxBek9vaFdNOEx4ZkdsSy9RVVpFc092TnF3YlNw?=
 =?utf-8?B?LzNoWmJLaFNYQTZRTlhoMitBd2RqekNLWkNCUWhzVDFVdCtFU3VjejJlZ0ZX?=
 =?utf-8?B?US9hUUp3M1NpUVFHYlFCVkU5VUZISTZIOGV0cGJIL2F3WEFQczNpNldpME1l?=
 =?utf-8?B?dUc2TXZPakJmMllPRHJTYzA5a3NhNzVNMUNiMTlpZ25mUXlRYUM1bFhzMEZn?=
 =?utf-8?B?UDZaOGRtNEJxeUtiMnhKOVlLaitLWHUrQXhhTDYvWEJKcXZ4anBYV25NUi82?=
 =?utf-8?B?aitnaEtucS9RMG9xKzRjVFVkRWZhQXlhdG5PRjJ0WkcvSTlidVB0VXdmbXFV?=
 =?utf-8?B?M2J3OVpDL0Uxc05oMTI2bVhYSDJ1YTU2cS9HZzBsMWdySnRpMmFPMkhPZjRT?=
 =?utf-8?B?bGwycFhsUWU2RGhBZWZvYXJoVmowQ2ZSNFF3WmFFcC8vQ1FzSmdCS0hGWGsx?=
 =?utf-8?B?Q1EvWkN5TnRnTlZ5dGtlU1hoRG1DNXBjMFZubXdOTEVobVVyMHRRUnFuKzlL?=
 =?utf-8?B?eUNZb2pvYzdiSHdSUm8xWDZWZUE1NWd4d0RCNVZyckpnUzk1bEVOWVpzODN2?=
 =?utf-8?B?Vm12b0ZSVFgwZlAwNUJhaEZoMGRSZjhkMGJiLzBHTDRVTHdaamw2dDlSYVVk?=
 =?utf-8?B?TUs0YjB4QkFhait2bmx1K1hRL2trREk2RVpIcUM2UU5kZjUvMVJadWk5dnlP?=
 =?utf-8?B?TXJSckpyT1Bma1ZDZzhsMVo0NThuNHpuV29ORFMzM3VEVXAvdzBlVWp5NGlL?=
 =?utf-8?B?RG1tWjhKR3BpOGZ1UUM0R2YwSWlkeVI5NUJsVlp1Ukk0R2JXcTVaSHJYQkpo?=
 =?utf-8?B?NGU4US9RMWFFajVFY2JoM0ZyV0VzaGdzSnh4OUVTaGlpQWw1dzBIMjFwVlFJ?=
 =?utf-8?B?b1VWVHloZE9FREluazFXM1VXNDZ3bkZiQk80VGI0a0drRm00M0N6RUlYdUdR?=
 =?utf-8?B?ODZ5RXFJOWFvVk40dnMrN1BqcW1udmltZmM4cDBwT1NyTmdwTzdLWFdlRzRn?=
 =?utf-8?B?QURKVzBtRU05OVRNS0VDMy9MQjlqbkpRUlV1b3VwZ3N0T25aTkozRW1HUi9r?=
 =?utf-8?B?WHREaHFCdEVIMWJQSUtYYnk0THRrcXovNGtKaFNlancwNDV2K0hQcG9QTmhK?=
 =?utf-8?B?NVh0U29zaUx1UndHYnhFQXZEbkRIYXVUWE9qVU1kMUFydXN1dVNJRTMvUEsy?=
 =?utf-8?B?QTZnOElmTWxjajFOWUwyK2RRZkUweUVOMFNTaGcyZUtoa0J2TDRxak1nZzdw?=
 =?utf-8?B?TFlabWFZaDJtYWFkakx2VllqVllWWk1wOFlzWm8xdDVNTXlKYjkxNHdlb2FE?=
 =?utf-8?B?UEE9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f9a2980c-ac5c-418e-ac35-08dc6a85d456
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 08:57:00.7652
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zWHTtPiHKI+qfOTimgY10A7DKcJG+WzFkQ9nl7iz4D4knHcHIKfZ3WDamuUd+m9DHcs26jDGiEEL+FpiXirmkg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB9939
X-Proofpoint-GUID: 3kQenP0jxiSRuPn8jKw8KxYjcg8xAhwo
X-Proofpoint-ORIG-GUID: 3kQenP0jxiSRuPn8jKw8KxYjcg8xAhwo
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-01_16,2024-05-02_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 suspectscore=0
 adultscore=0 priorityscore=1501 phishscore=0 malwarescore=0
 lowpriorityscore=0 spamscore=0 clxscore=1015 mlxlogscore=999
 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2404010003 definitions=main-2405020054

29.04.24 18:42, Jan Beulich:
> On 23.04.2024 10:54, Sergiy Kibrik wrote:
>> Guard access to Intel-specific lmce_support & cmci_support variables in
>> common MCE/VMCE code. These are set in Intel-specific parts of mcheck code
>> and can potentially be skipped if building for non-intel platform by
>> disabling CONFIG_INTEL option.
>>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> 
> See comments given for patch 2.
> 

I'll squash this patch into patch 7 then, as they depend on each other

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Thu May 02 08:59:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 08:59:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715672.1117470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SHf-0003yR-GL; Thu, 02 May 2024 08:59:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715672.1117470; Thu, 02 May 2024 08:59:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SHf-0003yK-Bf; Thu, 02 May 2024 08:59:19 +0000
Received: by outflank-mailman (input) for mailman id 715672;
 Thu, 02 May 2024 08:59:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=216C=MF=epam.com=prvs=1852dd919c=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s2SHe-0003yE-VI
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 08:59:18 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40f66568-0862-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 10:59:16 +0200 (CEST)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4427u0dL022407;
 Thu, 2 May 2024 08:59:12 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3xuh9fbg3j-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 02 May 2024 08:59:11 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by DU0PR03MB9731.eurprd03.prod.outlook.com (2603:10a6:10:44d::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29; Thu, 2 May
 2024 08:59:06 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::b309:8ea5:85c3:d04f]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::b309:8ea5:85c3:d04f%5]) with mapi id 15.20.7544.029; Thu, 2 May 2024
 08:59:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40f66568-0862-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YatD3XBSyoaUgMlh0vQoPJqumriNyB3sk2jskvyjVM2SkVvLvafkkXy4lBBtsitftx6+9HgrH0DZTFZl/n+6p2m4x+hHbfPmtyr6aibgzL+Kia1N9ECfFwI9pI9271xs/Si9Q5GvL6ad5iucsVf6Kn6K1tiYnUamVCGcl2bAbzgzO0zldHJigUymC/iav4OrJ5MPrklUGle+3JgTgltqXAJRhXULgqsXMDWzk4BkXwnT5t24FC9B81PE8xiWAyY0FYUh5lE3FsOHp9pacaTIEQ8pB6Ohy9BRbf7tgT0ZbOvrqCOJptCBBGH9mWqtU4Ch05BHCcikzHGWGiL8MX6MGg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NvcjSf2ot5E0HjkHo7YQYp8x3QeT0rou/3zsPr4sqDo=;
 b=mNAm3Gh5JrhDvDj5NW/tnSuPvbbflXR7JJiDGrKXvoJn7cJe+S+71/7js5VYXmnGOsbqMJepy2lum3l2GS0sBP4nb3GcqlAewr4RY8kR+11gAb1JjWBYrb5j+DANysKkb54HUHvXDS3G+irY6cn8LcpTB/vGbarW8pa8C9z1gmPXevJYhjth03mgst7aOFhjdkbOoq0q1NVUamtReQiHNKC3um/pptQnCNdm826C1H8EQOdu3NTIdirFrHaqaOC5KGQ3Oynacv0v65k2HfpQn73cGCwo6gWEMu4OhpZyXyY8vMO9QIgbqP4VrwhHSWy96gl8Bw6WGzO3Zc1vrKBFPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NvcjSf2ot5E0HjkHo7YQYp8x3QeT0rou/3zsPr4sqDo=;
 b=Zpio9p/n7MvRi9tzA1DJSzeiGUkxvNox78pLTGnqFjgRF1lf2yQ1fDrb1sc9ftiBS/yku0qy3tq06W6FJ3+F01o1ext5JCS+PvS+MAwwcOlNUNBjoVbAZEB2LLnQZ2KOEBKEQ/zGMpsMhhdyhjesVi3bj9bTnPvdPOkTh6co2GbH9/3FxSGOQ4GZ4fGdyutVyuFvM5RfvaYnOzj4jQDBvXp3afp6GDz1YhlOEG3KdpHXPyr4C641AM8zPC3TXmyy0sWc8liauwjjpp6zjpukHV9G0sSevg/sKaQa/jZepFAUk8qrBfcD8AJsGGqNdIhZ14ldvcU0rgrJH5TLqthILw==
Message-ID: <256d9aee-5790-4a86-b50e-80429a117724@epam.com>
Date: Thu, 2 May 2024 11:59:05 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v1 7/7] x86/MCE: optional build of AMD/Intel MCE code
To: Jan Beulich <jbeulich@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1713860310.git.Sergiy_Kibrik@epam.com>
 <82df6ef350a2b4f42ec7adf12a90ebeae1d133f6.1713860310.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2404261615250.3940@ubuntu-linux-20-04-desktop>
 <a0cde29c-2b30-4b6b-a6de-135b10d7f165@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <a0cde29c-2b30-4b6b-a6de-135b10d7f165@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA0P291CA0007.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1::11) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|DU0PR03MB9731:EE_
X-MS-Office365-Filtering-Correlation-Id: 6f090e63-246a-4dc3-338f-08dc6a861f7c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|376005|1800799015;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?TXRKRHlYaGQ0eVg5M1JsRkpRWExHV2NBNTRxOFpxdDhlbUVMSnNSZUZ3TmF5?=
 =?utf-8?B?eTlZZzdDamtmbExMWTdlNCtnSDNDNFdoZnNiRFllckxFOTgvREt4NkNHZVVM?=
 =?utf-8?B?eFplalhId2hMNkJ6Q2tKaEx4M0U3MTBUamNLVVdzdTdwSTlrTkhkNGZyTnFy?=
 =?utf-8?B?dGNHS1hpZWJUWFF3UFo4VGF6ZExwbmRjQ21XZnBOVitiTm9tZnU1VDRqRUJl?=
 =?utf-8?B?eHdjVU13NHNjcngwV2d0TWZpandkOEc4eEN1Qm43aXpqTWQxVU5DRVpkcEdB?=
 =?utf-8?B?bmFkVTRwYjZJYXh2clBIMkhuRS9Xa3g5K2V2cGYwWTQzZXVoNCtoQldSNFds?=
 =?utf-8?B?SWVsT2t5cCtlMHdBbnZPOGFCR0VrYnZoM2dwVE93ck5Kb2djcVpkWUIwRGMz?=
 =?utf-8?B?RHJhaysvaVYzeXpWeXFYSDZjOS92Vk8rcW9aQ1grbTVzZkhabVovTWs4NHhU?=
 =?utf-8?B?bG0vdHNmakhyZlh0ekNDOUxiYndMb0g5T0oxUzRSNHNaM1B0VURmSTRwNjdX?=
 =?utf-8?B?bndseHZ3bnU2ZFhQSmdnV3ZRQVlybm1lZkRYVm1tU085VGpBYXltVkxESHBa?=
 =?utf-8?B?b25ZNjhBRmN6MzFCVTExNXlBSXRSM0E2R2VJUjFPRitVRGNoYmRicXU2aUdw?=
 =?utf-8?B?Um0wQUxVRU9BejRWY3I5dGt1NTV2TUNKdndoeUNiM0Z4Q0hmSUcrVWVsc0Iz?=
 =?utf-8?B?K0tjc0hpZGNaN05MbFErTW5zbTM5MzZzcW1USnYzM29kQWRsQ2ZRSURNYmVR?=
 =?utf-8?B?T21FRWErY29sWHg2cmRqVmJVZC8xMkc3THFSaFdabzdYdHo0cVpoOWhiSkdr?=
 =?utf-8?B?djNwZElkWisyZVpmVWNnTkVndDJPWXRJOVd1dDB2ZWpmN0pmQlVURnpmbjU1?=
 =?utf-8?B?RU1jb2I1T3Q3QVZnZHM0bFZLbW95RWRTTjRFYy9pcXBFUnNJckJVdGpZR0Z5?=
 =?utf-8?B?Z0dFMitwc1ozM0l0eDRnUXFQTkR1RlhkbUNsU2srNHhIUmQzZDB2WlRzWFh4?=
 =?utf-8?B?N3R4QmVoKzBwOEN5bkdnaE91RHU0OWhBa3ZUaFRmdkFyeFF1R0dxOWRLb085?=
 =?utf-8?B?K2xneXZreXpseGVBbVd2SG9FOTZUcWF0NFIxc1QwdFNvN3oyN3o0QXU0UEg0?=
 =?utf-8?B?cG1JV2k2ZWx3cUx6c2QyYW9ZQ25lL0hpd0YwVlo0UHBFeWZrVUNhTVU3VGZ2?=
 =?utf-8?B?bDZzK2RoeEFFWTEwS2ZsdEIyZ1YwUW1aalBaRlBFUmJ6YUJhS2hTbFNBUytj?=
 =?utf-8?B?dm8waEdMenVUblV4aStCYmZ6ZTZCbWt6d0N5eHlXZDl3WktQdUFZaGVOQktZ?=
 =?utf-8?B?OXBNcUpUeDkwWWNRNUxLZFBnNHhzc2JIMzVqejE3UlkyS0VybFBoYXQvZC9t?=
 =?utf-8?B?TVAyVjdmQU9WLytHS0RmdldOR2FWNDV2MnRyaFRIakpUdHVrMm5ZcnJaazBZ?=
 =?utf-8?B?T1grbkRUOS9aaW83anFEWWVIODJ5UG9LMmtMQUZ2SmxvS0haTkFibFNWbWtW?=
 =?utf-8?B?RzhtT2xlVHlDWTc1MDJzZlBKb2g3UVV5ZTBid25HdVhqWllqWXFUM3RIVGFD?=
 =?utf-8?B?Nis0WU1EVjBMdStkQTBLTDRHNnlSeFZWTi9uczNoenZmMEhiR2UvVUhpOXZk?=
 =?utf-8?B?THcyT2k2dVptSDBzUkdPNzJEYm5BeU16U0I1Wm9EaGhUVVBKUmJWMEtDc29Y?=
 =?utf-8?B?Yk13YXRBRVg0NVU0MWRqNmlmQ2FPMk1HVHlhMXdMR1lTcVlnb3hsNUdnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UUlIUThLQThHbEU2SVRpaXpZOE1BVG1vTHJDTGVaVDZoaFF0QS90S2htL1Bi?=
 =?utf-8?B?UmdoMUZVSG82VHJqcllpUjl0OTZZMnFYRzFWazQvMlU0RjRmNHRhMXRQemFU?=
 =?utf-8?B?ZENJTlJ5ZGpybDIvQjl2S0VGUEJpTm0rQmpTYjEyUWVuVnd3TEV4RmNEUDBn?=
 =?utf-8?B?VVBqeExxaGNmd2IzSFB2V3JLZ241RDlaeWV6Slc1cFRZalBwRlp1S3VjT2Ix?=
 =?utf-8?B?SzgvWEJJTTBNbkdMRWlPRGdUdHlRd2kzNS9iV3M4cFRiKzArYnhyOHVBYVFP?=
 =?utf-8?B?MnhwU3gwbkdtRFo0M3JmeXlKSWY2OWtUQlhjVjJGZHdzTWg5K29oR1pJbVRv?=
 =?utf-8?B?QmJSVEJXYmNidGNrMmJHUmp0WjcxbGI3YjVVK24xWDRwMmVZS292Z2xDQ2dH?=
 =?utf-8?B?VW52bWQwaGYvN25FNTdJZ01XcUo4K3FjMHlyc3pZU0VPRUlodkZDM2FzY3ZL?=
 =?utf-8?B?WXhGWVhuRGVvREJ2UkpXeTM2TmlCMUZVVTZaYU1VTGdBbXNOa2JhRHY0b3J5?=
 =?utf-8?B?RGJGVHNKMTNhOFpTWDdteHkzUnBSeTlVd0ZYNWEvMTdRbGYybGkrU0g5NUl1?=
 =?utf-8?B?ZHpMV2NBYitiem01RVhOdDBNbEVqcGd4YVZlSnkwWVZUTzBicHlwMDFTeEJI?=
 =?utf-8?B?YWtqN2xOZGFSS1pWNW9ZcHAvRGVxdWprcWZ5S1h5L2wxU3Jkc3ZFZThxUFdm?=
 =?utf-8?B?N1J2TnJKTjR2bFJveXRYbWpxemYxUGk0aGtVNDc3Zk9wM2c4VXJZSTBjRzNF?=
 =?utf-8?B?RjBLUE93a3J5R0tEZUVrYnZwYy9SeFlzSlJ2RjdNTmJEYTNiY2hWcHdQTEIz?=
 =?utf-8?B?WFVtSUl6Q0ZGYXZSaFhiVEtpUnpZWjFnNloyY1FDR05aVnR1cDRrbHh0c0RT?=
 =?utf-8?B?MDFDeUhFNUY5RUFXL3dtUGNNcEtMUjczOHpkOWNPVVhLUnlUSUxjMnZnd3Fn?=
 =?utf-8?B?aDBESkNsa2FNM25jdWFDWFFMcFpRZkc2UlROU3F4RnpjZlIzQWhzSlo0NTBV?=
 =?utf-8?B?M0FDVm1wTjhrMStUU0x1QjNmK2VBWks5NHZwTnNZQVNlbGIrMjA3VUt2bjlY?=
 =?utf-8?B?MUZIWllhMU5QdlcvTjVPUXRHdDNRSUY1QTlaOXFNbndHbzd5RTVGajRIUFdl?=
 =?utf-8?B?cFFGeXRCMEgyZkRZMGZjS2MyQW5qZUdGQTJiTE5lUHhMR00raXhpanp0eVhk?=
 =?utf-8?B?V3M0WDcyZ0ppTmREYjdhNFJUMXMrdWJlVm40c0RmSURvcThhQnFpQndadTFy?=
 =?utf-8?B?SGpLUDZNNy9tY2ovQWlWc0FsbEV2WVF2VUk1cDVHNmtXam5oZ3dwVmxEVEE4?=
 =?utf-8?B?dndsYXVKVDNiMTNSMVk0TjZxcEw5RllKeUhvWnlmeEpYSFNWOWJiVjRoUjJ2?=
 =?utf-8?B?TkZibjdrOFB3WnhsajgwbmNRd2dmcm5BYkVlQ3hWSnVCOGtuOXZ2aEMySk1C?=
 =?utf-8?B?ZGVKdkwzdXdOWC90M1JvdmR4ZkppOFI1MXYrRlFiYzB1RDZzV01GMEVFYS9s?=
 =?utf-8?B?c1V1YlRuU3g4ZzlyREJLZ2NJd3JRZUY4VE1lRkVpMk1NdVpRWlFjbm5SL21q?=
 =?utf-8?B?Q0tSaG9NQWpOUlh3Z3ZZdTFuZytEYm9HNEtheERqZmVGVlZxSEl6M1JGOFRh?=
 =?utf-8?B?UEJmb2xMZmRCaEhJUEZrTFh2YWVtV3FCbmszUVBOdmhJdGlsRExxM3ZYdVA4?=
 =?utf-8?B?bkt5Z0lRVUt1R0NaU1V3U2pXbjczMFZKZUJzN1RmYWhvVnV2VG1KNW1ibDIv?=
 =?utf-8?B?ZWU4dXkxNTJrb2lnUlZKbUdTWnJCUW1idkYzTzJNdGxFcDZrQWtpRG83Q2RT?=
 =?utf-8?B?VWU2dkt6a0FoalhZR3BFb0ZBOUZ4b3ljQnliZml0Tk5hYVJEZGV4WWlzOGtC?=
 =?utf-8?B?Z1dWdlhIZjdSVTJxb3FuS0lJRUNGWE5ERSs0T3RBSEgxLzN6YThIbWJzcnAr?=
 =?utf-8?B?U1pmWGdVR2NSQXZCMUpmeW5YOXltQTVDVi9pWkRPUEVha0syaTdHQ29pSFdy?=
 =?utf-8?B?VTN1NTJJU0d3eG1oeUp1cFJ6cUNMVmw3aURRQTNUUVIrZ0tKbGpsUlpWWXhJ?=
 =?utf-8?B?NGt4UUNHUU14b1dFT2twNzJVZFljRDgrNksyWkEvMVByZDhxNlpndEQ5N2w3?=
 =?utf-8?B?bUhnUkM0bU51d241TERKZHhrOHgrTDYxNnlib1F2bXZEYlZ4YUs5cTdYVlRy?=
 =?utf-8?B?L0E9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6f090e63-246a-4dc3-338f-08dc6a861f7c
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 08:59:06.7300
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fTSFKdtArINgJAE+95e5YByjOkm9YuzLibhpH+iDyTocOtV3mcVf5vkZp+tQYNOLSg9ADGJnACZ1WAuq3CfZ1A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB9731
X-Proofpoint-GUID: inOkiGseoYVfHLKKXR_8xyjg9p45Ly_f
X-Proofpoint-ORIG-GUID: inOkiGseoYVfHLKKXR_8xyjg9p45Ly_f
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-01_16,2024-05-02_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015
 lowpriorityscore=0 malwarescore=0 bulkscore=0 phishscore=0 impostorscore=0
 mlxlogscore=999 spamscore=0 priorityscore=1501 suspectscore=0 adultscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2404010003 definitions=main-2405020054

29.04.24 18:54, Jan Beulich:
> On 27.04.2024 01:16, Stefano Stabellini wrote:
>> On Tue, 23 Apr 2024, Sergiy Kibrik wrote:
>>> --- a/xen/arch/x86/cpu/mcheck/Makefile
>>> +++ b/xen/arch/x86/cpu/mcheck/Makefile
>>> @@ -1,12 +1,10 @@
>>> -obj-y += amd_nonfatal.o
>>> -obj-y += mce_amd.o
>>>   obj-y += mcaction.o
>>>   obj-y += barrier.o
>>> -obj-y += intel-nonfatal.o
>>>   obj-y += mctelem.o
>>>   obj-y += mce.o
>>>   obj-y += mce-apei.o
>>> -obj-y += mce_intel.o
>>> +obj-$(CONFIG_AMD) += mce_amd.o amd_nonfatal.o
>>> +obj-$(CONFIG_INTEL) += mce_intel.o intel-nonfatal.o
>>>   obj-y += non-fatal.o
>>>   obj-y += util.o
>>>   obj-y += vmce.o
>>
>> Awesome!
> 
> Almost. I'd appreciate if the ordering of files would be retained. It's
> not quite alphabetic, but still. Moving mce_amd.o and mcaction.o to their
> designated slots may or may not be done right here.

sure, I'll leave ordering as before

> 
>>> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
>>> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
>>> @@ -24,14 +24,20 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>>>   	 * Check for non-fatal errors every MCE_RATE s
>>>   	 */
>>>   	switch (c->x86_vendor) {
>>> +#ifdef CONFIG_AMD
>>>   	case X86_VENDOR_AMD:
>>>   	case X86_VENDOR_HYGON:
>>>   		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>>>   		amd_nonfatal_mcheck_init(c);
>>>   		break;
>>> +#endif
>>> +#ifdef CONFIG_INTEL
>>>   	case X86_VENDOR_INTEL:
>>>   		intel_nonfatal_mcheck_init(c);
>>>   		break;
>>> +#endif
>>> +	default:
>>> +		return -ENODEV;
> 
> This, while perhaps desirable, doesn't fit ...
> 
>>>   	}
>>>   	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>>>   	return 0;
> 
> ... earlier behavior, and hence is somewhat unexpected in a change which, by
> its description, looks like a "no functional change" one.
> 

I see, will try to describe it a bit better then.

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Thu May 02 09:10:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:10:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715679.1117479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SSp-0007QY-D6; Thu, 02 May 2024 09:10:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715679.1117479; Thu, 02 May 2024 09:10:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SSp-0007QR-9I; Thu, 02 May 2024 09:10:51 +0000
Received: by outflank-mailman (input) for mailman id 715679;
 Thu, 02 May 2024 09:10:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2SSn-0007QL-TT
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:10:49 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dba3d15f-0863-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 11:10:47 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 79BD62F165;
 Thu,  2 May 2024 05:10:44 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 724A72F164;
 Thu,  2 May 2024 05:10:44 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 2D6712F163;
 Thu,  2 May 2024 05:10:40 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dba3d15f-0863-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:mime-version:content-transfer-encoding;
	 s=sasl; bh=ygPGeb/EjkduarrNYMv/b9FupPivWzhlS+l8I9Jci3U=; b=i2u+
	NlwsMyWE0auqctJHGd6DmtnjXYzqOZC1xpMFgQZd8GNujAEmj89GGUbC8l2OYbyw
	nYzzUzBohXY6nNYWNYcuH2tejCKV6o24H49JZR7Sa4R0WoM+D4IzF8EkYdtlcvjx
	+qUCciyX0MUFmolr5x9rRYm/no7ulTP9zaqrrx4=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v2 0/5] x86: make Intel/AMD vPMU & MCE support configurable
Date: Thu,  2 May 2024 12:10:36 +0300
Message-Id: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
X-Pobox-Relay-ID:
 D8B0E80E-0863-11EF-B181-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

Here's a second attempt to separate support of Intel & AMD CPUs in Xen bu=
ild.
The code to drive both platforms used to be built unconditionally, until =
recent
introduction of CONFIG_{AMD,INTEL} Kconfig options.

This series extends coverage of these options on vpmu and mcheck subsyste=
ms,
which allows not to build Intel or AMD vpmu/mcheck support if CPU vendor'=
s support
was explicitly disabled.

Since v1 series one patch had been merged independently, and one more got=
 to be
squashed, so series became shorter. Specific changes since v1 are provide=
d per-patch.

v1 series here:
https://lore.kernel.org/xen-devel/cover.1713860310.git.Sergiy_Kibrik@epam=
.com/

  -Sergiy

Sergiy Kibrik (5):
  x86/vpmu: separate amd/intel vPMU code
  x86/intel: move vmce_has_lmce() routine to header
  x86/MCE: guard access to Intel/AMD-specific MCA MSRs
  x86/MCE: guard {intel/amd}_mcheck_init() calls
  x86/MCE: optional build of AMD/Intel MCE code

 xen/arch/x86/cpu/Makefile           |  4 +++-
 xen/arch/x86/cpu/mcheck/Makefile    |  8 ++++----
 xen/arch/x86/cpu/mcheck/mce.c       |  6 ++++--
 xen/arch/x86/cpu/mcheck/mce.h       | 13 +++++++++++++
 xen/arch/x86/cpu/mcheck/mce_intel.c |  4 ----
 xen/arch/x86/cpu/mcheck/non-fatal.c |  6 ++++++
 xen/arch/x86/cpu/mcheck/vmce.c      | 17 ++++++++++-------
 xen/arch/x86/cpu/vpmu.c             |  6 ++++--
 xen/arch/x86/include/asm/mce.h      |  1 -
 xen/arch/x86/msr.c                  |  2 ++
 10 files changed, 46 insertions(+), 21 deletions(-)

--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:12:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:12:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715683.1117488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SUp-00081C-PL; Thu, 02 May 2024 09:12:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715683.1117488; Thu, 02 May 2024 09:12:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SUp-000815-MT; Thu, 02 May 2024 09:12:55 +0000
Received: by outflank-mailman (input) for mailman id 715683;
 Thu, 02 May 2024 09:12:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2SUo-00080h-69
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:12:54 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2708c599-0864-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:12:53 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 0E9731AFEE;
 Thu,  2 May 2024 05:12:51 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 066281AFED;
 Thu,  2 May 2024 05:12:51 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 528781AFEC;
 Thu,  2 May 2024 05:12:50 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2708c599-0864-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=n8lh5sI7kgYWqWm0ZWfIf98vp
	I8aDBlCayXCBhx7N2Q=; b=EnSbZrzqTTb8JKUMDhjijLQ0pnaq3vOJhuoLXgBTv
	/TIE3764Q+jQo2weoCZp0wLZIGnN80l7PsnNIL790o77y+IZL05JOoVd6g6H2AZW
	3c0ohGF70/UqqdGTsYYle96nyf8b5AC1Q0LpwE4ZyVuwi5UpbVA4wKVbkFXMFj+w
	ik=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [XEN PATCH v2 1/5] x86/vpmu: separate amd/intel vPMU code
Date: Thu,  2 May 2024 12:12:43 +0300
Message-Id: <ddbf4e24c41f48832bba4b11368cd5e6e9f79a20.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 26447D24-0864-11EF-B2B6-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Build AMD vPMU when CONFIG_AMD is on, and Intel vPMU when CONFIG_INTEL
is on respectively, allowing for a plaftorm-specific build.

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>

---
changes in v2:
 - drop static inline stubs, use #idef/#endif in vpmu_init)()
changes in v1:
 - switch to CONFIG_{AMD,INTEL} instead of CONFIG_{SVM,VMX}
---
 xen/arch/x86/cpu/Makefile | 4 +++-
 xen/arch/x86/cpu/vpmu.c   | 6 ++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/cpu/Makefile b/xen/arch/x86/cpu/Makefile
index 35561fe51d..eafce5f204 100644
--- a/xen/arch/x86/cpu/Makefile
+++ b/xen/arch/x86/cpu/Makefile
@@ -10,4 +10,6 @@ obj-y +=3D intel.o
 obj-y +=3D intel_cacheinfo.o
 obj-y +=3D mwait-idle.o
 obj-y +=3D shanghai.o
-obj-y +=3D vpmu.o vpmu_amd.o vpmu_intel.o
+obj-y +=3D vpmu.o
+obj-$(CONFIG_AMD) +=3D vpmu_amd.o
+obj-$(CONFIG_INTEL) +=3D vpmu_intel.o
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index b2e9881e06..3db90b7839 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -827,6 +827,7 @@ static int __init cf_check vpmu_init(void)
=20
     switch ( vendor )
     {
+#ifdef CONFIG_AMD
     case X86_VENDOR_AMD:
         ops =3D amd_vpmu_init();
         break;
@@ -834,11 +835,12 @@ static int __init cf_check vpmu_init(void)
     case X86_VENDOR_HYGON:
         ops =3D hygon_vpmu_init();
         break;
-
+#endif
+#ifdef CONFIG_INTEL
     case X86_VENDOR_INTEL:
         ops =3D core2_vpmu_init();
         break;
-
+#endif
     default:
         printk(XENLOG_WARNING "VPMU: Unknown CPU vendor: %d. "
                "Turning VPMU off.\n", vendor);
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:15:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:15:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715687.1117499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SWo-00016y-4w; Thu, 02 May 2024 09:14:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715687.1117499; Thu, 02 May 2024 09:14:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SWo-00016r-1W; Thu, 02 May 2024 09:14:58 +0000
Received: by outflank-mailman (input) for mailman id 715687;
 Thu, 02 May 2024 09:14:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2SWm-00013q-Mw
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:14:56 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7056ff40-0864-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:14:55 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 604951B003;
 Thu,  2 May 2024 05:14:54 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 57B601B002;
 Thu,  2 May 2024 05:14:54 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 8AEF71B001;
 Thu,  2 May 2024 05:14:53 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7056ff40-0864-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=NanAAy5pF1VN7K+y0zyRcYv8A
	H8YMR9HJ30IOBBsrB0=; b=pHpgY9rsF3YTATkTEsVfovwnFOTxdLwOkPj7kHf6c
	NuqCUsnAscGxu+gdTwOaUh1boWQkzCCDbVsrd9ktM7Z6+WwDPn11uyca1JM1m7XX
	jeHqFzq1FTgZrJSeH2RnOxhDbQyW5Pj0DaGg+J1UF1BnRwKRHSUnmbLZFLNjRZK0
	Sc=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v2 2/5] x86/intel: move vmce_has_lmce() routine to header
Date: Thu,  2 May 2024 12:14:51 +0300
Message-Id: <e759fc187f2e3f23f42666b2a8461ae851591d50.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 6FB88086-0864-11EF-A735-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Moving this function out of mce_intel.c would make it possible to disable
build of Intel MCE code later on, because the function gets called from
common x86 code.

Add internal check for CONFIG_INTEL option, as MCG_LMCE_P bit is currentl=
y
specific to Intel CPUs only.

Also replace boilerplate code that checks for MCG_LMCE_P flag with
vmce_has_lmce(), which might contribute to readability a bit.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - move vmce_has_lmce() to cpu/mcheck/mce.h
 - move IS_ENABLED(CONFIG_INTEL) check inside vmce_has_lmce()
 - changed description
---
 xen/arch/x86/cpu/mcheck/mce.h       | 5 +++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ----
 xen/arch/x86/cpu/mcheck/vmce.c      | 5 ++---
 xen/arch/x86/include/asm/mce.h      | 1 -
 xen/arch/x86/msr.c                  | 2 ++
 5 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/mce.h b/xen/arch/x86/cpu/mcheck/mce.=
h
index 4806405f96..d6d56aa232 100644
--- a/xen/arch/x86/cpu/mcheck/mce.h
+++ b/xen/arch/x86/cpu/mcheck/mce.h
@@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v,=
 uint32_t msr)
     return 0;
 }
=20
+static inline bool vmce_has_lmce(const struct vcpu *v)
+{
+    return IS_ENABLED(CONFIG_INTEL) && v->arch.vmce.mcg_cap & MCG_LMCE_P=
;
+}
+
 struct mce_callbacks {
     void (*handler)(const struct cpu_user_regs *regs);
     bool (*check_addr)(uint64_t status, uint64_t misc, int addr_type);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mchec=
k/mce_intel.c
index 3f5199b531..af43281cc6 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -1050,7 +1050,3 @@ int vmce_intel_rdmsr(const struct vcpu *v, uint32_t=
 msr, uint64_t *val)
     return 1;
 }
=20
-bool vmce_has_lmce(const struct vcpu *v)
-{
-    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
-}
diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmc=
e.c
index 353d4f19b2..94d1f021e1 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -199,7 +199,7 @@ int vmce_rdmsr(uint32_t msr, uint64_t *val)
          * bits are always set in guest MSR_IA32_FEATURE_CONTROL by Xen,=
 so it
          * does not need to check them here.
          */
-        if ( cur->arch.vmce.mcg_cap & MCG_LMCE_P )
+        if ( vmce_has_lmce(cur) )
         {
             *val =3D cur->arch.vmce.mcg_ext_ctl;
             mce_printk(MCE_VERBOSE, "MCE: %pv: rd MCG_EXT_CTL %#"PRIx64"=
\n",
@@ -324,8 +324,7 @@ int vmce_wrmsr(uint32_t msr, uint64_t val)
         break;
=20
     case MSR_IA32_MCG_EXT_CTL:
-        if ( (cur->arch.vmce.mcg_cap & MCG_LMCE_P) &&
-             !(val & ~MCG_EXT_CTL_LMCE_EN) )
+        if ( vmce_has_lmce(cur) && !(val & ~MCG_EXT_CTL_LMCE_EN) )
             cur->arch.vmce.mcg_ext_ctl =3D val;
         else
             ret =3D -1;
diff --git a/xen/arch/x86/include/asm/mce.h b/xen/arch/x86/include/asm/mc=
e.h
index 6ce56b5b85..2ec47a71ae 100644
--- a/xen/arch/x86/include/asm/mce.h
+++ b/xen/arch/x86/include/asm/mce.h
@@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
 extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu =
*ctxt);
 extern int vmce_wrmsr(uint32_t msr, uint64_t val);
 extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
-extern bool vmce_has_lmce(const struct vcpu *v);
 extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);
=20
 DECLARE_PER_CPU(unsigned int, nr_mce_banks);
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 9babd441f9..b0ec96f021 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -24,6 +24,8 @@
=20
 #include <public/hvm/params.h>
=20
+#include "cpu/mcheck/mce.h"
+
 DEFINE_PER_CPU(uint32_t, tsc_aux);
=20
 int init_vcpu_msr_policy(struct vcpu *v)
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715693.1117508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SYm-0001if-G0; Thu, 02 May 2024 09:17:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715693.1117508; Thu, 02 May 2024 09:17:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2SYm-0001iY-DO; Thu, 02 May 2024 09:17:00 +0000
Received: by outflank-mailman (input) for mailman id 715693;
 Thu, 02 May 2024 09:16:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2SYl-0001ht-6r
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:16:59 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b9ddb56c-0864-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:16:58 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id A2AB21B012;
 Thu,  2 May 2024 05:16:57 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 99F821B011;
 Thu,  2 May 2024 05:16:57 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 9E1F81B010;
 Thu,  2 May 2024 05:16:56 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9ddb56c-0864-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=lgp28eofjwhEroQqYCklB0e/t
	P+m22xf9XqOUZKnIoA=; b=iV5Q5v2lZrzH5K4MyUiyAT/abCGCVRTRvk2x/PVvu
	OeJz58/s4lt7FOHZFK5DXc/413Bm//7t8DKEEcLQyl6fwt/Ncoi3fIhHk3upbG6g
	PCL5hDPRI4LmNuISx8YI46eGeQ7YFZFEEMIbSDv0GYtDjIYs4azWUIIvghhHEgxc
	gE=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v2 3/5] x86/MCE: guard access to Intel/AMD-specific MCA MSRs
Date: Thu,  2 May 2024 12:16:54 +0300
Message-Id: <725b477a688ad17e5fa2a8bac78f4905170523c8.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 B914D3A6-0864-11EF-B9EA-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Add build-time checks for newly introduced INTEL/AMD config options when
calling vmce_{intel/amd}_{rdmsr/wrmsr}() routines.
This way a platform-specific code can be omitted in vmce code, if this
platform is disabled in config.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - use #ifdef/#endif in switch instead of IS_ENABLED
 - fallback to returning default 0 if vendor not recognized
---
 xen/arch/x86/cpu/mcheck/vmce.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmc=
e.c
index 94d1f021e1..373a8e2452 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -138,17 +138,19 @@ static int bank_mce_rdmsr(const struct vcpu *v, uin=
t32_t msr, uint64_t *val)
     default:
         switch ( boot_cpu_data.x86_vendor )
         {
+#ifdef CONFIG_INTEL
         case X86_VENDOR_CENTAUR:
         case X86_VENDOR_SHANGHAI:
         case X86_VENDOR_INTEL:
             ret =3D vmce_intel_rdmsr(v, msr, val);
             break;
-
+#endif
+#ifdef CONFIG_AMD
         case X86_VENDOR_AMD:
         case X86_VENDOR_HYGON:
             ret =3D vmce_amd_rdmsr(v, msr, val);
             break;
-
+#endif
         default:
             ret =3D 0;
             break;
@@ -271,15 +273,17 @@ static int bank_mce_wrmsr(struct vcpu *v, uint32_t =
msr, uint64_t val)
     default:
         switch ( boot_cpu_data.x86_vendor )
         {
+#ifdef CONFIG_INTEL
         case X86_VENDOR_INTEL:
             ret =3D vmce_intel_wrmsr(v, msr, val);
             break;
-
+#endif
+#ifdef CONFIG_AMD
         case X86_VENDOR_AMD:
         case X86_VENDOR_HYGON:
             ret =3D vmce_amd_wrmsr(v, msr, val);
             break;
-
+#endif
         default:
             ret =3D 0;
             break;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:19:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:19:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715698.1117518 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2San-0002Hs-Qe; Thu, 02 May 2024 09:19:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715698.1117518; Thu, 02 May 2024 09:19:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2San-0002Hl-O1; Thu, 02 May 2024 09:19:05 +0000
Received: by outflank-mailman (input) for mailman id 715698;
 Thu, 02 May 2024 09:19:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2Sam-0002Hf-Pv
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:19:04 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 038b4ee0-0865-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 11:19:02 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 862961B026;
 Thu,  2 May 2024 05:19:01 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 5473D1B025;
 Thu,  2 May 2024 05:19:01 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id C77E21B023;
 Thu,  2 May 2024 05:18:59 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 038b4ee0-0865-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=wfkSN8qgSwdu2/Uw/g3XHdw7o
	vhuKayF/PQY78NPsFw=; b=kSAmzF3NmmTz+1KvaAiBPC15BgbZ0kwibq9PTkHib
	brWg1t3SXhWxinar4awFs2ilaSdgZFIw0qCC9nnGEpPyvPY7NgtsFR4PLCmaG/f8
	egobJMbfq20EyxHin8hjsGs/L26RhO4ellJa1c2hZ0ivZ2HmGetuXM9eMUd6ZWef
	YA=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v2 4/5] x86/MCE: guard {intel/amd}_mcheck_init() calls
Date: Thu,  2 May 2024 12:18:57 +0300
Message-Id: <ef74d26d98cb533d148fcc83c354919443ebaa74.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 027E4D10-0865-11EF-ADD8-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Guard calls to CPU-specific mcheck init routines in common MCE code
using new INTEL/AMD config options.

The purpose is not to build platform-specific mcheck code and calls to it=
,
if this platform is disabled in config.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - use #ifdef/#endif in switch instead of IS_ENABLED
---
 xen/arch/x86/cpu/mcheck/mce.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.=
c
index d179e6b068..fb943addae 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -760,11 +760,13 @@ void mcheck_init(struct cpuinfo_x86 *c, bool bsp)
=20
     switch ( c->x86_vendor )
     {
+#ifdef CONFIG_AMD
     case X86_VENDOR_AMD:
     case X86_VENDOR_HYGON:
         inited =3D amd_mcheck_init(c, bsp);
         break;
-
+#endif
+#ifdef CONFIG_INTEL
     case X86_VENDOR_INTEL:
         switch ( c->x86 )
         {
@@ -774,7 +776,7 @@ void mcheck_init(struct cpuinfo_x86 *c, bool bsp)
             break;
         }
         break;
-
+#endif
     default:
         break;
     }
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:20:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:20:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715702.1117528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Sbz-0003gD-4E; Thu, 02 May 2024 09:20:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715702.1117528; Thu, 02 May 2024 09:20:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Sbz-0003g6-1b; Thu, 02 May 2024 09:20:19 +0000
Received: by outflank-mailman (input) for mailman id 715702;
 Thu, 02 May 2024 09:20:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Sby-0003fw-0O; Thu, 02 May 2024 09:20:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Sbx-0007dp-QB; Thu, 02 May 2024 09:20:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Sbx-00086d-8v; Thu, 02 May 2024 09:20:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Sbx-0008G8-8b; Thu, 02 May 2024 09:20:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fboJqV1lccN23oKqVo7jGyt0voWLGHuWngd6YXhot+Y=; b=MABide3wA43RUt/GyDO4iP5s/y
	ZUwREhjS0uwBXSp+51WFLj08zOBddCo7P1QPHDhLFPB1WTcFmnLHLZtIpJEadr7XHEL7ARSsaH9RE
	08XdUFju3zfvIwajXFh5JMCRKDZf6bZtRp0oQhYJW9N/74xeH2jfgNPdjRPxUzG7/Mrs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185896-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185896: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0106679839f7c69632b3b9833c3268c316c0a9fc
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 09:20:17 +0000

flight 185896 linux-linus real [real]
flight 185898 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185896/
http://logs.test-lab.xenproject.org/osstest/logs/185898/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1   8 xen-boot                 fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail pass in 185898-retest
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 185898-retest
 test-armhf-armhf-examine      8 reboot              fail pass in 185898-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185870
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185898 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185898 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                0106679839f7c69632b3b9833c3268c316c0a9fc
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    2 days
Failing since        185888  2024-04-30 20:31:50 Z    1 days    3 attempts
Testing same since   185896  2024-05-01 23:13:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Potapenko <glider@google.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Audra Mitchell <audra@redhat.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Chuck Lever <chuck.lever@oracle.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Hans de Goede <hdegoede@redhat.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  John Garry <john.g.garry@oracle.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Sven Schnelle <svens@linux.ibm.com>
  Tejun Heo <tj@kernel.org>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Xingyou Chen <rockrush@rockwork.org>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1219 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 02 09:21:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:21:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715708.1117539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Scm-0004Fk-I2; Thu, 02 May 2024 09:21:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715708.1117539; Thu, 02 May 2024 09:21:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Scm-0004Fd-FB; Thu, 02 May 2024 09:21:08 +0000
Received: by outflank-mailman (input) for mailman id 715708;
 Thu, 02 May 2024 09:21:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jUhA=MF=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s2Scl-00043A-6p
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:21:07 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4cc15598-0865-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:21:06 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id D9BDB1EF76;
 Thu,  2 May 2024 05:21:03 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id BA55A1EF74;
 Thu,  2 May 2024 05:21:03 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id CEBDA1EF73;
 Thu,  2 May 2024 05:21:02 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cc15598-0865-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=qhW4J/cTFeEpCYrlIcXjhJRer
	OeqjMFOR0GWdMpf6D0=; b=WSRxL6Ii5P734QIsNDFDUTACl7oppmt6R0wKYAJXf
	guqIFaD7Gc5rTIBoqWsfHT69cwkwk5zH+L09BcErNVfFDaxd623nrY5oWfaqIHWP
	Nh2mN+MFqeEkNTRgsj/P4RoWUbgJEieU12TdcNlnaTB7mDecEaxhLgUfVhUPNI++
	LI=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v2 5/5] x86/MCE: optional build of AMD/Intel MCE code
Date: Thu,  2 May 2024 12:21:00 +0300
Message-Id: <47d32ecff7b915bd23b6d13b76cedf4b39db71a2.1714640459.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 4BD369E6-0865-11EF-B450-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

Separate Intel/AMD-specific MCE code using CONFIG_{INTEL,AMD} config opti=
ons.
Now we can avoid build of mcheck code if support for specific platform is
intentionally disabled by configuration.

Add default return value to init_nonfatal_mce_checker() routine -- in cas=
e
of a build with both AMD and INTEL options are off (e.g. randconfig).

Also global Intel-specific variables lmce_support & cmci_support have to =
be
redefined if !INTEL, as they get checked in common code.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - fallback to original ordering in Makefile
 - redefine lmce_support & cmci_support global vars to false when !INTEL
 - changed patch description
---
 xen/arch/x86/cpu/mcheck/Makefile    | 8 ++++----
 xen/arch/x86/cpu/mcheck/mce.h       | 8 ++++++++
 xen/arch/x86/cpu/mcheck/non-fatal.c | 6 ++++++
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/Makefile b/xen/arch/x86/cpu/mcheck/M=
akefile
index f927f10b4d..e6cb4dd503 100644
--- a/xen/arch/x86/cpu/mcheck/Makefile
+++ b/xen/arch/x86/cpu/mcheck/Makefile
@@ -1,12 +1,12 @@
-obj-y +=3D amd_nonfatal.o
-obj-y +=3D mce_amd.o
+obj-$(CONFIG_AMD) +=3D amd_nonfatal.o
+obj-$(CONFIG_AMD) +=3D mce_amd.o
 obj-y +=3D mcaction.o
 obj-y +=3D barrier.o
-obj-y +=3D intel-nonfatal.o
+obj-$(CONFIG_INTEL) +=3D intel-nonfatal.o
 obj-y +=3D mctelem.o
 obj-y +=3D mce.o
 obj-y +=3D mce-apei.o
-obj-y +=3D mce_intel.o
+obj-$(CONFIG_INTEL) +=3D mce_intel.o
 obj-y +=3D non-fatal.o
 obj-y +=3D util.o
 obj-y +=3D vmce.o
diff --git a/xen/arch/x86/cpu/mcheck/mce.h b/xen/arch/x86/cpu/mcheck/mce.=
h
index d6d56aa232..7fbf1fa2ae 100644
--- a/xen/arch/x86/cpu/mcheck/mce.h
+++ b/xen/arch/x86/cpu/mcheck/mce.h
@@ -40,7 +40,11 @@ enum mcheck_type {
 };
=20
 extern uint8_t cmci_apic_vector;
+#ifdef CONFIG_INTEL
 extern bool lmce_support;
+#else
+#define lmce_support (false)
+#endif
=20
 /* Init functions */
 enum mcheck_type amd_mcheck_init(const struct cpuinfo_x86 *c, bool bsp);
@@ -120,7 +124,11 @@ DECLARE_PER_CPU(struct mca_banks *, poll_bankmask);
 DECLARE_PER_CPU(struct mca_banks *, no_cmci_banks);
 DECLARE_PER_CPU(struct mca_banks *, mce_clear_banks);
=20
+#ifdef CONFIG_INTEL
 extern bool cmci_support;
+#else
+#define cmci_support (false)
+#endif
 extern bool is_mc_panic;
 extern bool mce_broadcast;
 extern void mcheck_mca_clearbanks(struct mca_banks *bankmask);
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mchec=
k/non-fatal.c
index 33cacd15c2..2d91a3b1e0 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -24,14 +24,20 @@ static int __init cf_check init_nonfatal_mce_checker(=
void)
 	 * Check for non-fatal errors every MCE_RATE s
 	 */
 	switch (c->x86_vendor) {
+#ifdef CONFIG_AMD
 	case X86_VENDOR_AMD:
 	case X86_VENDOR_HYGON:
 		/* Assume we are on K8 or newer AMD or Hygon CPU here */
 		amd_nonfatal_mcheck_init(c);
 		break;
+#endif
+#ifdef CONFIG_INTEL
 	case X86_VENDOR_INTEL:
 		intel_nonfatal_mcheck_init(c);
 		break;
+#endif
+	default:
+		return -ENODEV;
 	}
 	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n")=
;
 	return 0;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:22:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:22:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715711.1117548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Sdy-0004nY-QJ; Thu, 02 May 2024 09:22:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715711.1117548; Thu, 02 May 2024 09:22:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Sdy-0004nR-Np; Thu, 02 May 2024 09:22:22 +0000
Received: by outflank-mailman (input) for mailman id 715711;
 Thu, 02 May 2024 09:22:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+3sG=MF=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s2Sdx-0004nL-HZ
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:22:21 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 790baa5f-0865-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 11:22:19 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 7C9731FBBF;
 Thu,  2 May 2024 09:22:18 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 53D1913957;
 Thu,  2 May 2024 09:22:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Q5D9EkpbM2aKdQAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 02 May 2024 09:22:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 790baa5f-0865-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714641738; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=cATNxw1g0F/LigKNEgEM99/Pu1Mz9opkA8rmEbHNY9E=;
	b=JKTWtgQjeS/EIhHah9mCKjBSqFAcLyDW0Fj8/4tiiESK9CJBMpWXoTsrHXL7SkFL8UjVZP
	DEgDuqy45csNeMgrWTMxMOF9gK9mXq+zrkE6yDEc4Vi+VuC/mRtjUzV2XGAGd/2oIJZLPc
	ZOPEDJtBoV8pduAUZ4HQzVkmBxL3rco=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=JKTWtgQj
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714641738; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=cATNxw1g0F/LigKNEgEM99/Pu1Mz9opkA8rmEbHNY9E=;
	b=JKTWtgQjeS/EIhHah9mCKjBSqFAcLyDW0Fj8/4tiiESK9CJBMpWXoTsrHXL7SkFL8UjVZP
	DEgDuqy45csNeMgrWTMxMOF9gK9mXq+zrkE6yDEc4Vi+VuC/mRtjUzV2XGAGd/2oIJZLPc
	ZOPEDJtBoV8pduAUZ4HQzVkmBxL3rco=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH] tools/tests: let test-xenstore exit with non-0 status in case of error
Date: Thu,  2 May 2024 11:22:13 +0200
Message-Id: <20240502092213.15163-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Flag: NO
X-Spam-Score: -2.02
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: 7C9731FBBF
X-Spam-Level: 
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-2.02 / 50.00];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	MID_CONTAINS_FROM(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_MISSING_CHARSET(0.50)[];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.10)[text/plain];
	MX_GOOD(-0.01)[];
	BAYES_HAM(-0.01)[45.29%];
	FROM_EQ_ENVFROM(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	ARC_NA(0.00)[];
	FROM_HAS_DN(0.00)[];
	TO_DN_SOME(0.00)[];
	MIME_TRACE(0.00)[0:+];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RCVD_COUNT_TWO(0.00)[2];
	RCPT_COUNT_THREE(0.00)[4];
	DKIM_TRACE(0.00)[suse.com:+]

In case a test is failing in test-xenstore, let the tool exit with an
exit status other than 0.

Fix a typo in an error message.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: 3afc5e4a5b75 ("tools/tests: add xenstore testing framework")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/tests/xenstore/test-xenstore.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/tests/xenstore/test-xenstore.c b/tools/tests/xenstore/test-xenstore.c
index d491dac53b..d7d44cc0fa 100644
--- a/tools/tests/xenstore/test-xenstore.c
+++ b/tools/tests/xenstore/test-xenstore.c
@@ -506,14 +506,14 @@ int main(int argc, char *argv[])
         stop = time(NULL) + randtime;
         srandom((unsigned int)stop);
 
-        while ( time(NULL) < stop )
+        while ( time(NULL) < stop && !ret )
         {
             t = random() % ARRAY_SIZE(tests);
             ret = call_test(tests + t, iters, true);
         }
     }
     else
-        for ( t = 0; t < ARRAY_SIZE(tests); t++ )
+        for ( t = 0; t < ARRAY_SIZE(tests) && !ret; t++ )
         {
             if ( !test || !strcmp(test, tests[t].name) )
                 ret = call_test(tests + t, iters, false);
@@ -525,10 +525,10 @@ int main(int argc, char *argv[])
     xs_close(xsh);
 
     if ( ta_loops )
-        printf("Exhaustive transaction retries (%d) occurrred %d times.\n",
+        printf("Exhaustive transaction retries (%d) occurred %d times.\n",
                MAX_TA_LOOPS, ta_loops);
 
-    return 0;
+    return ret ? 3 : 0;
 }
 
 /*
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu May 02 09:26:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:26:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715717.1117558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Si4-0006hj-9h; Thu, 02 May 2024 09:26:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715717.1117558; Thu, 02 May 2024 09:26:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Si4-0006hc-6z; Thu, 02 May 2024 09:26:36 +0000
Received: by outflank-mailman (input) for mailman id 715717;
 Thu, 02 May 2024 09:26:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2Si2-0006fv-Ku
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:26:34 +0000
Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com
 [2607:f8b0:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 10689965-0866-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:26:33 +0200 (CEST)
Received: by mail-oi1-x230.google.com with SMTP id
 5614622812f47-3c74a75d9adso5045051b6e.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 02:26:33 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l13-20020ad4408d000000b006a0ee5b6ee6sm213811qvp.123.2024.05.02.02.26.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 02:26:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10689965-0866-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714641992; x=1715246792; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=RmWMTJt8l3CMWxELPSlrqRMoXkOQLmVL9dGzeB82P5E=;
        b=Gb6pJG/zK4+rrH+G76xgo7vigwfOlYr2HdIlzJSMhLrbWTRfFDP0M0WzninreVjD3q
         mCQqS2G7TC8xH7rZiVpFdcoCySnHx+vpAYFHSneLDIGkeiQSgBiAMrNbKWb+M9oJZyz3
         kPG91A1cfLmu/iSE4pxfgWkVeUosrwxGj2uas=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714641992; x=1715246792;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RmWMTJt8l3CMWxELPSlrqRMoXkOQLmVL9dGzeB82P5E=;
        b=HeJLxqdEhgQ3b7fpE1vM2VGi7lKDeKY842ftdKaMltdRSS6w5XaMofTI05sw3Qegiu
         +AiHXM4uUMtXg9bS1yhCmFaLvGS4H/oKXVPuj2CcA6+hWULUnO7XS7o+8ER4NQXJN6gr
         2UiptOvgY4/nQ3l/c2u2gstoAui5ftnsurgKZXFRGiJIJXRdXm2zuAPFy+1WzhgMOR5U
         mKbwsKh3UbWUHG5BwAGNWAE09R7Zkp042kkwpJiunK9/VigIR7X6FQFCn7MUYZC9FJto
         J5JB4TxUbWZ97HASvRtfFnsMj6qVHl4UePixV5+EFZrxkp7rZ0WfszQDtcOKT/1Mff+U
         fk/w==
X-Forwarded-Encrypted: i=1; AJvYcCURxf/XiIZxQqvz/Z4EfqDWhPOjzo1COSzwmFrMW3tCW3myEq+rriH4xnErmVn5TdHFyPoUWlWnCgBJBsowfuV1gQ1H09UjSEQ4zD8Y2UM=
X-Gm-Message-State: AOJu0Yx29w/arO7BJaAnWqnX3yLu5SIV3TsgGVf+Ix79Z75kDH1vO64x
	ay1MZaf/xK/k2ipYhmhZ/tFRjpshlHwFU1rBBU6AW6jxNFnbJFkXfMQ13jgpQXPBIZPehcoJPTZ
	e
X-Google-Smtp-Source: AGHT+IGkYUVa3G+uYdAoqKz4MbBQNCM5Q8I+J+rPNmAHOU0cVKrVWvRSW7jaAXRJfzV2zGXfHTNa4w==
X-Received: by 2002:a05:6808:4405:b0:3c7:4f55:e731 with SMTP id eo5-20020a056808440500b003c74f55e731mr5269705oib.40.1714641992188;
        Thu, 02 May 2024 02:26:32 -0700 (PDT)
Message-ID: <29784550-166d-43dc-a56e-b88afdad7d90@citrix.com>
Date: Thu, 2 May 2024 10:26:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/tests: let test-xenstore exit with non-0 status in
 case of error
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>
References: <20240502092213.15163-1-jgross@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240502092213.15163-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 10:22 am, Juergen Gross wrote:
> In case a test is failing in test-xenstore, let the tool exit with an
> exit status other than 0.
>
> Fix a typo in an error message.
>
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Fixes: 3afc5e4a5b75 ("tools/tests: add xenstore testing framework")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Thanks.  Acked-by: Andrew Cooper <andrew.cooper3@citrix.com> although
this could do with waiting until after we've fixed whatever else is
causing it to fail.

I'll try to find someone to work on that bug too.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 09:53:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:53:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715727.1117569 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2T8A-0003ya-Ai; Thu, 02 May 2024 09:53:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715727.1117569; Thu, 02 May 2024 09:53:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2T8A-0003yT-7d; Thu, 02 May 2024 09:53:34 +0000
Received: by outflank-mailman (input) for mailman id 715727;
 Thu, 02 May 2024 09:53:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2T88-0003yK-JX
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:53:32 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d3e5b059-0869-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 11:53:29 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41adf3580dbso9601625e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 02:53:29 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w20-20020a05600c475400b0041befc2652csm5146326wmo.31.2024.05.02.02.53.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 02:53:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3e5b059-0869-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714643609; x=1715248409; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=06MTo3BFClf2Y5rzR+dRcYttIeBThahWk2BGF+WvEok=;
        b=OGKWn18LxzZpZ/D25UMbCy29RcGvs1SUlw3bI/w7h6yfT2iqTTjic1tfsTZATrezUQ
         7TFY25PgTPqYzLWuc8fKs4d278F7tcJUKURqsOKIZaBr3Z51M++ImQJMx4j32QiZsc+d
         kLOsAs849Y8PzT1YrtBK5isB2FT+epOwBif0ilGJH6FOyEW4DxR1wUvRh7MwgF+MnILv
         DY7w94hDYGJP9QIe4pu1Wu6v0v1X96BgJXs10duED7J6Ph3COILN+JIz+s/0mltnl09K
         0spX1Ph+7krSiqHZdPeITv+kiFPKBbgDaTWEV+3BfIcnYI4HleLIqCRZdNC5TTpLR/vB
         lyXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714643609; x=1715248409;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=06MTo3BFClf2Y5rzR+dRcYttIeBThahWk2BGF+WvEok=;
        b=is74zBLanPHl68tYta1xGjez9zo+IkrNIQ4Pblj8e+6cm+41avCGxNOsrfJcWeWvQz
         kQIym61K7Gkf/nwdneE7zHOW8UtyKI9Jbzggt4BokvJfYT+RtbxjcfXvqiky8rstowcH
         vEhUqUiV3M0lxjt9RF5UxgZEgrUtvrnSaz+96U2ffbMVzGgP3Ek9ZTZH1V4pIWeuOftN
         haSFANA5Z4Y8gdi4ONXlGa7Fm07hJX4GVL/k4EtmT0farfdXBYSKKqnJEnSwT6cIgj7j
         fxGBXn1nUhhr+66m2HdE+VjNJJwA5JVsp1mfla/jZFtWTQpSNPRegZLtNX1f+pagRRb6
         17BA==
X-Forwarded-Encrypted: i=1; AJvYcCVjl8C6MeDvxvHg0ox35EblB95TP8Gh8rxpa8ivHXqnao6fUy5ZXcxvDEtTkeDdzeNpJx/SElKV3chV7pZseWE4rrbwy5mtYRvlE8EkIGU=
X-Gm-Message-State: AOJu0Yx9vafdoNupUDN6gGUHMgGIlWTow4+539veIP916vk8muf7jC56
	XVO2ijlupDOIKZ9PPwMr1P4Rt/LiKfMInZ2gs4da6mItZZRjw3KP1QIsTJE68w==
X-Google-Smtp-Source: AGHT+IESk0CxrY2d1AWdT+b8PKtlzo8/eeapYwhMxA3YkEFsGux7th9XIEFhozvipU5RfUrvbZVq2A==
X-Received: by 2002:a05:600c:1383:b0:41a:8035:af77 with SMTP id u3-20020a05600c138300b0041a8035af77mr1865029wmf.12.1714643608718;
        Thu, 02 May 2024 02:53:28 -0700 (PDT)
Message-ID: <4fee2998-e29b-43bb-855a-8600dbef9f13@suse.com>
Date: Thu, 2 May 2024 11:53:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
 <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
 <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 10:13, Luca Fancellu wrote:
> 
> 
>> On 2 May 2024, at 07:43, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 02.05.2024 08:33, Luca Fancellu wrote:
>>>
>>>
>>>> On 2 May 2024, at 07:14, Jan Beulich <jbeulich@suse.com> wrote:
>>>>
>>>> On 01.05.2024 08:57, Luca Fancellu wrote:
>>>>> Hi Jan,
>>>>>
>>>>>> On 30 Apr 2024, at 12:37, Jan Beulich <jbeulich@suse.com> wrote:
>>>>>>
>>>>>> On 30.04.2024 13:09, Luca Fancellu wrote:
>>>>>>> --- a/xen/arch/arm/include/asm/setup.h
>>>>>>> +++ b/xen/arch/arm/include/asm/setup.h
>>>>>>> @@ -64,18 +64,20 @@ struct membank {
>>>>>>> };
>>>>>>>
>>>>>>> struct membanks {
>>>>>>> -    unsigned int nr_banks;
>>>>>>> -    unsigned int max_banks;
>>>>>>> +    __struct_group(membanks_hdr, common, ,
>>>>>>> +        unsigned int nr_banks;
>>>>>>> +        unsigned int max_banks;
>>>>>>> +    );
>>>>>>>   struct membank bank[];
>>>>>>> };
>>>>>>
>>>>>> I'm afraid I can't spot why __struct_group() is needed here. Why would just
>>>>>> one of the two more straightforward
>>>>>>
>>>>>> struct membanks {
>>>>>>  struct membanks_hdr {
>>>>>>      unsigned int nr_banks;
>>>>>>      unsigned int max_banks;
>>>>>>  );
>>>>>>  struct membank bank[];
>>>>>> };
>>>>>>
>>>>>
>>>>> At the first sight I thought this solution could have worked, however GCC brought me back down to earth
>>>>> remembering me that flexible array members can’t be left alone in an empty structure:
>>>>>
>>>>> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:70:6: error: declaration does not declare anything [-Werror]
>>>>> 70 | };
>>>>> | ^
>>>>> /data_sdc/lucfan01/gitlab_mickledore_xen/xen/xen/arch/arm/include/asm/setup.h:71:20: error: flexible array member in a struct with no named members
>>>>> 71 | struct membank bank[];
>>>>> | ^~~~
>>>>> [...]
>>>>
>>>> Since for patch 1 you looked at Linux'es uapi/linux/stddef.h, the solution
>>>> to this lies there, in __DECLARE_FLEX_ARRAY(). Alongside or instead of
>>>> borrowing __struct_group(), we could consider borrowing this as well. Or
>>>> open-code it just here, for the time being (perhaps my preference). Yet
>>>> it's not clear to me that doing so will actually be enough to make things
>>>> work for you.
>>>
>>> I looked also into __DECLARE_FLEX_ARRAY(), but then decided __struct_group()
>>> was enough for my purpose, can I ask the technical reasons why it would be your
>>> preference? Is there something in that construct that is a concern for you?
>>
>> I don't like either construct very much, but of the two __DECLARE_FLEX_ARRAY()
>> looks slightly more "natural" for what is wanted and how it's done.
>> __struct_group() introducing twice the (effectively) same structure feels
>> pretty odd, for now at least. It's not even entirely clear to me whether there
>> aren't pitfalls, seeing that the C spec differentiates named and unnamed
>> struct fields in a few cases. For __DECLARE_FLEX_ARRAY(), otoh, I can't
>> presently see any reason to suspect possible corner cases.
>>
>> Yet as said before - I'm not sure __DECLARE_FLEX_ARRAY() alone would be enough
>> for what you want to achieve.
> 
> Mmm yes, I think I would still have problems because container_of wants a named member,
> so __DECLARE_FLEX_ARRAY() doesn’t help much alone, if I’m not missing something obvious.
> If you believe however that importing __struct_group() only for this instance is not enough to justify
> its presence in the codebase, I could open-code it, provided that maintainers are ok with that.

I'm afraid I've even more strongly against open-coding. If you can get an
ack from another maintainer for the introduction of struct_group(), that
would still allow it to go in. I didn't NAK the change, I merely have
reservations.

> In any case it would be used soon also for other architectures using bootinfo.

Oh, would it?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 09:55:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 09:55:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715731.1117578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TAE-0005Ma-QZ; Thu, 02 May 2024 09:55:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715731.1117578; Thu, 02 May 2024 09:55:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TAE-0005MT-Nh; Thu, 02 May 2024 09:55:42 +0000
Received: by outflank-mailman (input) for mailman id 715731;
 Thu, 02 May 2024 09:55:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ua+q=MF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2TAD-0005LA-Ac
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 09:55:41 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2222659d-086a-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 11:55:40 +0200 (CEST)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2dd19c29c41so95843431fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 02:55:40 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 s7-20020a2e81c7000000b002e1bc0bcfc0sm126302ljg.52.2024.05.02.02.55.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 02:55:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2222659d-086a-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714643740; x=1715248540; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=ZikOErl9JAy5Pr+z4FV6j6OlYGYJ5+bO1a3fDp9xqIQ=;
        b=YAMmDuevWlKvRjf5wKLvqIej83ULusH5XcYzt6IWrMqAS5O3Pd6HiGlZd2ohEdx2+u
         LQUlm5kEbaS0Cq2Jgls/+VOdU15K8fpOHPNGhMhgVDSk2WP0Yb0mbzC7oV9A9gLdMKIr
         qHbMPmCsFV0k+jW/kLnX7Ns5LlU5rslvK0DhCcZTHN6mFYpH5vgQRagPbFq/LPUzz6K9
         zoI/Rqoo42xZsQRTbc8ffjMBkLiBLWCeEQrYOSLdqiIptKZEaz6hE1569QqBgbvb7h4B
         e3X0nVFbNvWNeXd+UuK5IY011OR6N3+xE3GXUVLUkR5YXd3rLOH3wt9aZ2TE/d5C2BKd
         jzJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714643740; x=1715248540;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ZikOErl9JAy5Pr+z4FV6j6OlYGYJ5+bO1a3fDp9xqIQ=;
        b=kY2c5CMLJJw/o5m2xnoz2xOF5eCx1Lo138BrOBKFqHxQlEjClzNoUIJLUs278Z83iP
         PEMOnsUInMhfBaPrvumi021NVjGeRdBN64F9oq/BLN7O7otBuTJldStDvgVSgNqzPAFW
         LvCSfM9Flh6tuu6KNtKZCRQmbX1dwKA0NfEVYAKT7m3+F87OzCMS/Op1F06/uG1Z1h7k
         8NCQXOTrlJlH44Ua8npwQAH8xUaot1xEfdWWBQgmryhdXgTG+eZNdb45XBfBNtoGunJs
         L3sVl6gRHflP1ll332FIwD+vqZiBBL07lJDnjWyIjxfkJzyGdXfCygJ7VJ2+m77oK0Sn
         6bpw==
X-Forwarded-Encrypted: i=1; AJvYcCVNEWBgGAnwfadS1QjXvIW8OSmkvflJzeCLGOnZykq9iDRxVK33SWG73JkXEme+G+j5FGZRWBCyfrGf7a9vAAF0Ga/GGuVFFOtu9JYd5lw=
X-Gm-Message-State: AOJu0YwruYBSxLePHZ966Cweo+/AbGHGX5fTPYBjXftVFaFJfeXyVO3+
	JjUbe262eKNC0hK1+p5YNXDMdFqLH/+MHqXXVn1TEw4Im5V2LMzd
X-Google-Smtp-Source: AGHT+IHqfBrVOAoNPaumpqHvBhV8j90VOt9yI888dE1C0di5BzXiCx2ZbZooisWola07kVCY7Vnk1A==
X-Received: by 2002:a2e:3313:0:b0:2e0:774b:706c with SMTP id d19-20020a2e3313000000b002e0774b706cmr3036308ljc.3.1714643739666;
        Thu, 02 May 2024 02:55:39 -0700 (PDT)
Message-ID: <f2e86e372487bc784be4771369fb71401a4b187f.camel@gmail.com>
Subject: Re: [PATCH v1] xen/riscv: improve check-extension() macro
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
Date: Thu, 02 May 2024 11:55:38 +0200
In-Reply-To: <7ffd4293-e0fb-4ba1-b538-7ad44a2897ce@suse.com>
References: 
	<6f37ce6b115b682118a8332b2a81b49358c88587.1714144943.git.oleksii.kurochko@gmail.com>
	 <7ffd4293-e0fb-4ba1-b538-7ad44a2897ce@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-04-29 at 15:30 +0200, Jan Beulich wrote:
> On 26.04.2024 17:23, Oleksii Kurochko wrote:
> > Now, the check-extension() macro has 1 argument instead of 2.
> > This change helps to reduce redundancy around usage of extensions
> > name (in the case of the zbb extension, the name was used 3 times).
> >=20
> > To implement this, a new variable was introduced:
> > =C2=A0 <extension name>-insn
> > which represents the instruction support that is being checked.
> >=20
> > Additionally, zbb-insn is updated to use $(comma) instead of ",".
>=20
> Which is merely just-in-case, I suppose, but not strictly necessary
> anymore?
Sorry for late reply, you are right, this is not strictly necessary
anymore.

~ Oleksii

>=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > Suggested-by: Jan Beulich <jbeulich@suse.com>
>=20
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>=20
> Just as a remark: Tags want to be put in chronological order.
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Thu May 02 10:13:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 10:13:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715738.1117589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TR6-0000fG-7p; Thu, 02 May 2024 10:13:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715738.1117589; Thu, 02 May 2024 10:13:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TR6-0000f9-4m; Thu, 02 May 2024 10:13:08 +0000
Received: by outflank-mailman (input) for mailman id 715738;
 Thu, 02 May 2024 10:13:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=A7KU=MF=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s2TR5-0000f3-7J
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 10:13:07 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 (mail-am7eur03on20601.outbound.protection.outlook.com
 [2a01:111:f403:260e::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 906390ad-086c-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 12:13:04 +0200 (CEST)
Received: from DUZPR01CA0051.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:469::15) by GV1PR08MB7316.eurprd08.prod.outlook.com
 (2603:10a6:150:1f::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.26; Thu, 2 May
 2024 10:13:01 +0000
Received: from DB5PEPF00014B93.eurprd02.prod.outlook.com
 (2603:10a6:10:469:cafe::6a) by DUZPR01CA0051.outlook.office365.com
 (2603:10a6:10:469::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34 via Frontend
 Transport; Thu, 2 May 2024 10:13:01 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5PEPF00014B93.mail.protection.outlook.com (10.167.8.231) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Thu, 2 May 2024 10:13:00 +0000
Received: ("Tessian outbound 9d9bf1c5d85a:v315");
 Thu, 02 May 2024 10:13:00 +0000
Received: from 0c867f77a522.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 032E1BD0-002A-4B0E-A59E-EFA00EBBE484.1; 
 Thu, 02 May 2024 10:12:49 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0c867f77a522.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 May 2024 10:12:49 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB9PR08MB6668.eurprd08.prod.outlook.com (2603:10a6:10:2a9::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28; Thu, 2 May
 2024 10:12:45 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7519.031; Thu, 2 May 2024
 10:12:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 906390ad-086c-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=VVovkJ9uo/P4xx73uz5nzAjFZUidaqbUwqBu74G7hSWUVpRKM6GMJ9Rv+ilVX0t988aA2cNBXjRms7NOl0U1TpxsxlgbNjS95mzbzLqPBe0Ia/0IW9jgN1F6sGQmLX8/l1NZtRq03pj6CKhsYlg2TNzPcIdEjXzC25CzL9BPiNhBdbAnRwjCCPWPxXDUWoqc2Jc6phXSnvfD0Hztml3UyF4Bt3cbaax26It0HE8eoxRda/ieVheSP2ekXuIAczbP0VbbofGle30OVYHsJ1+mgffydX7g0NfRqFeNRF+0uuY40GkGV2kmqrCN1LM+RDgxo0mYABOZVw49v4QAfkPOhA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=yZG4fzbsBK6H78NSJfjcH/SoVTOvmdzV5mJN/jMVWeI=;
 b=dL1noUEoGCV5reCMnkITXd5iJ4NO5XPOL1uqyvzyz7QqLvl6G5pKyYwaFyHwc0aRkUufOlOeIsvrYJ1OHOHQKsOOD+HwuATrogkakhzyXrQc1MZ7RlbpcnCOusNw9+a0TNQip+wIfUDDGQN4HFfxGwaEIyr5w8ufAFSvU7HsCvgSoBfG8mShlsgBssaXnc9aA/mJKWH8X6T2/zg7D+CZiFJhoYir1x5ZwAYVSzpNC7L4/yY5iRSuLHa1pTo/1Li/3OSkVLysRDi+XJe7rLaSb3cfcAdeHM+czoFkL9dDYXTS6OAM3QYLvqfsPjIaRQILzwKMkoIRGL+tQhqzASatCA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yZG4fzbsBK6H78NSJfjcH/SoVTOvmdzV5mJN/jMVWeI=;
 b=LthQDQ02SoZ1JffiQiIBekIZpcXhTyHejtUeyzE8XqmdAE3qTDcUwZHpofrdmKZsvn5SIWOE3GG8iGpy9oakh1qQjHhPYWRNF09A8YXWi+4WVu/2k8IMO1mjp5T3Cd1Z4+1YU7YDjeGiB1fwKp4UcEg3HYi6Cp1WCUdqUqr+HrE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 9b8003f86866b762
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rso8jZnT4keh+KkjLXBhkONCgR8RnliGNNWHF8YhFeVE7u3aN/2iOVLXxQEj1pehCeDFI7c0m1GGi+PfIxSpCPUv4THJSnieKFJKJCxwFGwxhWptgsxveAsNA4+QXqAO8BykYHw/I8lTunu81bU8R5WaCI0Ma/YvqKexp1rBjd+WErX/kA1/xLFeZQPjMNzWItK3J0cW42thneT/CqHV++O+Me3wcG+6DKo3WNG3z7ipawgiTDvXo2xq4vGvKuiceFgfnihI0ERNqorb5d+XqLzjkF4L+2XhrZmyCZXUHfFDDjxYnlMauUYAzD/jBx8JZXtu26LaW5dpSvgHCGJIpw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=yZG4fzbsBK6H78NSJfjcH/SoVTOvmdzV5mJN/jMVWeI=;
 b=Pk8RI5dM0ZD8xgKWJKR4FvU3RQTXhi5uKWCn6Fral89IbTxitYRcLUB7LS/vkjpHKb4o0IO/0EtvEvCNg7GxOh/NS6FHun3ZtmGCiJjtjV1Ta78oKNE7qlhWycEL/SPHY+dFeXwg6o3DDOcBEUzooo8RP+h36HxXAF4yFHqVkTxDCYVO2rHJDWnrH5KKyyorYXv6FmuUNMk/XlLPpbEtJSK8Hjm0c3tryMro3QpHbXuZBtCJeC5jqEJgapSl3cpPb9S1num07T6z1EdZhgq8qxxRCPXUq+gUNNtdhY2YoHor2PR2g1AUFUAktvzewlB7QAjA01Ij5hIWJhiDYtk7aA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yZG4fzbsBK6H78NSJfjcH/SoVTOvmdzV5mJN/jMVWeI=;
 b=LthQDQ02SoZ1JffiQiIBekIZpcXhTyHejtUeyzE8XqmdAE3qTDcUwZHpofrdmKZsvn5SIWOE3GG8iGpy9oakh1qQjHhPYWRNF09A8YXWi+4WVu/2k8IMO1mjp5T3Cd1Z4+1YU7YDjeGiB1fwKp4UcEg3HYi6Cp1WCUdqUqr+HrE=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index:
 AQHamu72Eq1RpEGs60mNp9DIfY8mn7GAr6+AgAFEGQCAAYYzgIAABVAAgAACyoCAABlOgIAAG+eAgAAFWAA=
Date: Thu, 2 May 2024 10:12:44 +0000
Message-ID: <6A50BE71-7F92-4B7B-9A58-1D3A084FDA3C@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
 <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
 <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
 <4fee2998-e29b-43bb-855a-8600dbef9f13@suse.com>
In-Reply-To: <4fee2998-e29b-43bb-855a-8600dbef9f13@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB9PR08MB6668:EE_|DB5PEPF00014B93:EE_|GV1PR08MB7316:EE_
X-MS-Office365-Filtering-Correlation-Id: d258a44f-9ca2-4438-291d-08dc6a90728d
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?U/Ma4xItv0dVxmznf+H1aJQisNCaW4kqIhVmjFBo18XLcKbjAMPO6FZTvDwD?=
 =?us-ascii?Q?8iTEP6y/ymoivtHgYUUYhn/0i5rG+T2hASJF0ed9jWx5sK+2d75bsOvagdGm?=
 =?us-ascii?Q?469KJySYmm9E/ZS+bJtK6WDe4kABIuPcbenPq328MZoiu9ZLtllsR0w8nUBB?=
 =?us-ascii?Q?uZMoaFidhu7vwM3t/2xxNtu+Oasu3JGgQm1Y9Iu/LzkWvFRov/1PNG2jxmdR?=
 =?us-ascii?Q?T1XwzxGHm8Vhk/BawcDwy3iWCw0sVEQFWHwJH4fOGXrY+Vty0kg4Rq2dHkyL?=
 =?us-ascii?Q?kcQw8+aYcO8lZ7ciy/wJ080g/KdiLE+Q54lzmVHVnX/6FBHrSQi/c+PGkQvB?=
 =?us-ascii?Q?iTBgSQSWotkJ0BCL0CUkKLchgLxEBFpmkQqJmtxkxy4iqJXF9JW7xHTF0IvG?=
 =?us-ascii?Q?rhUmUzEDNJPcyGzl90F7WjDdodOHfJOzMMSc7nPTsErPF8c6tAS4Pfy0zD6N?=
 =?us-ascii?Q?T/NFL9rkturniZv2dN+/dsYl+FN/Z5KFf0vuP6gnNuCSfh8ZNYutla4URRDe?=
 =?us-ascii?Q?b1afYlRIBZVKNNHlkcutYPCFgRAdRp3mPIULoRp58zm678f9peMR7ZbdPral?=
 =?us-ascii?Q?hKYCdPVQSyMgMhAEz8AT31O0/2iEOhRo04xC8zo2avqRyKn1pXax4Io0x78s?=
 =?us-ascii?Q?p6qYuZ7/wuh92JuyXbh9jn/et2RZoddLPYC0GMCc1YAFM1Aya90igZvy5TWE?=
 =?us-ascii?Q?MGiesLjwpLhZRQhmUabFFMKucp+ifyv5gLZIYzQIOfZDyQ3aWioH2gNdolEj?=
 =?us-ascii?Q?kguLiyhHzGiWPWOAM/aQm2ofKq04b/HZ6B5aXP90Kv8zZbCeld7lr4hh2iYZ?=
 =?us-ascii?Q?SLpYkDSzOfiep7dIkovUXDHkxtTyE5HZpLR1zkFyoFoSEwhBIrGXle7FOkKq?=
 =?us-ascii?Q?O4WuApdYEoUPdIFWloQeVokc76rtkc1TkLD5xzbRXMlT797sIRs6aUKBVMaX?=
 =?us-ascii?Q?y5F/NoNAx2mjW4DYnC9XmTFR6Yj2MVEpyMmNP70Hbt/qYmQbldLJUamI8Nlv?=
 =?us-ascii?Q?NXm9/yJCMM6s7CBk3TKlw7Yf7YMy90Vz3Z2c4t+u4kTZGoQMHGgKYElI3+Tb?=
 =?us-ascii?Q?/ztEU5erFG3Np9gFat4N/L/PG8GQwYfnVdABSVzoVQmJPPhRcD+eoOwxLYa2?=
 =?us-ascii?Q?c1HhgptQ+fA1sRd8dYsYTQUOy5mko9C+ADJg8teieQT6k14HaIuUwSfkghsk?=
 =?us-ascii?Q?kHOxpyp1gQWngqQewdSU6gv+dqrLEw++MYPfIQR+UDBfiJd0eQPw/Re731HZ?=
 =?us-ascii?Q?ldt2cbh6adn6bM/gNoANqeSy84bDFHYK+FeH/lc1fA=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F3A0B629EB9473409F21D539B3215503@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6668
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5PEPF00014B93.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9d952a93-58ef-4dc0-b3a1-08dc6a906917
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?vSu1um24WIa2338HZl9uPYMajtFmPAWVDdNO/dtpNYw34G9TjGBtEOx4l/Er?=
 =?us-ascii?Q?8sEaTvYIZrmbYj5TjImpov6p3GtItH9AvGpl9Z84PzeTDbz9eLnqTUJLYZXk?=
 =?us-ascii?Q?qPpHaz9ASM1hEqBxEpLA/PYK8twgoJiWeuQAH3rOqtEVMq4+NlIFp2Z/LCHZ?=
 =?us-ascii?Q?n5sgMkGvWNnaN9l8Luc26fHKrquft076yowxqv81qjkRYoSap+hmnYiJladm?=
 =?us-ascii?Q?TrpIrFQHMGw0CtwGzhASAAp2y38+Ji30DgESL8hbqqs/PFrcI7Y1XhQTCvhD?=
 =?us-ascii?Q?KsSF/zcV9Csikl5P39TQ0GWgu/qxL1/b0TjqxlqhfrSa5AS9OWvgLa81fI7n?=
 =?us-ascii?Q?rk52ZISVOhtA5RPBH+KEG0K5F8/mHgSUMbCeTK9eAesFdRidIWhWauDdPpZC?=
 =?us-ascii?Q?V1wjMeRTBsOHxwhXXDgckoogd9f5SQtJQicnHE+LZ0jUG8jK8R2IA2cy1GXg?=
 =?us-ascii?Q?t5C4j98phjRjAXpCwAiRQq0j7vt5uZ/b6w4nXNiTJzRNatek4dulSKtSOcib?=
 =?us-ascii?Q?wDqjp+jAqq9Sxr2iN603zUGe+7VxDjYS+pHPaXUeTL8+MkX2G5D42rZHWK7/?=
 =?us-ascii?Q?GYf77LqoOeM68KCGVVDymZeihUBzwizg1Zek1jjkskT493XpKIo3on1ul4Lb?=
 =?us-ascii?Q?hpvDrR06qRRx/uVd6ay53HcpnFFo3ZSFLZaeEBo3VwTrPSafZkRGyG3ZZAgX?=
 =?us-ascii?Q?kX7HzuNO7yxryGt9WpSVYpLKr4J45OXiR5L/A3VI4Zqo03vpFqZiTV2rclBe?=
 =?us-ascii?Q?bPOTcHXfLYKX0LqBTiBeaqkF0kPe4TbWBjWWQ60nhaCH0XT+G0YaH0gaOU9I?=
 =?us-ascii?Q?BCu0oZYFSuucZBdhTbh5NHRK6kk3OTt65SFEWRkq3FqTNhpO0J1SJjbEP3Lp?=
 =?us-ascii?Q?HJPpqbhlDrbdZtCCN1FbFu1Ul5oE5OXdItlph2AEbN6bV4UNtaHudtb4PzLC?=
 =?us-ascii?Q?V3FgrtdlzaO6aIl9YXlhOsIeVsYEbZcWKrCKnzV7Z0URfzi3rt4jJOt6N9gA?=
 =?us-ascii?Q?nfijNTr8tx3ogVvyRXMIV8amjpdz7a8MDyT//ObESjhzud5dgM5lS8drfVu9?=
 =?us-ascii?Q?0Ba9yf6EsbRlKyNbzJTofAdo2qEo67Q8x2gmuXPP6A4LmcB/wtO5qUBFjZCd?=
 =?us-ascii?Q?XaicLRIaIhp2qX7auRh1hPoCJI9ExIJCNtzLc1x3TydbAWBYbA5AQCnNQhL5?=
 =?us-ascii?Q?tP46goREjIhbQsi0t++bpDuEKAuTn7h0QbpXKdNbH22+7pASXOnC8eek+ztj?=
 =?us-ascii?Q?Q0qEi9RrRzl9MltKWRALzQViv35DYlmxhGDyuq/vp4PtG4m95sj2XaTj+1Ke?=
 =?us-ascii?Q?+1SJkkIG5oorUwcnwPGjG9CH?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 10:13:00.9300
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d258a44f-9ca2-4438-291d-08dc6a90728d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5PEPF00014B93.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7316


>=20
>> In any case it would be used soon also for other architectures using boo=
tinfo.
>=20
> Oh, would it?

PPC people have plans on putting that interface in common:

https://patchwork.kernel.org/project/xen-devel/patch/451705505ff7f80ec66c78=
cc2830196fa6e4090c.1712893887.git.sanastasio@raptorengineering.com/




From xen-devel-bounces@lists.xenproject.org Thu May 02 10:30:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 10:30:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715743.1117599 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Ti6-0004ou-LH; Thu, 02 May 2024 10:30:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715743.1117599; Thu, 02 May 2024 10:30:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Ti6-0004on-IH; Thu, 02 May 2024 10:30:42 +0000
Received: by outflank-mailman (input) for mailman id 715743;
 Thu, 02 May 2024 10:30:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2Ti5-0004oh-MS
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 10:30:41 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 05caf24c-086f-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 12:30:40 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41b5e74fa83so52147515e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 03:30:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j12-20020a05600c190c00b0041c120dd345sm1473860wmq.21.2024.05.02.03.30.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 03:30:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05caf24c-086f-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714645840; x=1715250640; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/rq7/ynN2o7FWKC7uxnqCcCkIGlhG+l7/ws7B6xYfa4=;
        b=VI2rMbXIrvRDwmA9PWelQ2/vsZVEobSyiZlAda2maEfhO8h7FQ3/lgpZV04STBjglu
         IC9b6Qp4Sf2LF7vKjlA6jkUXlSQwExEyTXq4ZmfDFgYR6XDjyML2KY9J2DYKeng7qdwm
         nBtGB0ay1qXAbVtHXm+CNfsTtM5AlLqEdFkVxbDW2iLm5AYsL6MAd+74NOOCXkz/U7SK
         x4i+6yqw3suVxicOetdOPnVJDaHkjDrNZu+DsNCmBMqqIEN7LpyXCVdzUDwO4Ua51P6k
         3DUO67grHL6gqEWT+cZf0Q25M1FMTH1f5Vetxcg2/QTZ2QmI4rjAeT87nUrD6hJIxhX4
         rhqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714645840; x=1715250640;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/rq7/ynN2o7FWKC7uxnqCcCkIGlhG+l7/ws7B6xYfa4=;
        b=kDo5CirRLQKTIsuxUBdZ8qd55xsexeLv7QhdgBFX5+v/eK3/5AgPeMyGwIcn3ySIKU
         hMxE1+jkuFO2J6VRL92BV3aldyty0ZQd3GdRPOq8a+oNHwyMNQi9cWhSNQAFS8uHZW+J
         AqajL5D5CkjSqYWL+qhZ6ZzGJ2iFjUN+X+6VUJs9VmbobDl6j+w90h44ww9EsPrUwsRs
         5tnn/b4M9k87O/5J9iyO76sTQ4HINxprrKjpCpIJI2vb3lKnjbzssFLW2sSSqls/UcDO
         HOXY4SgNVBgSuqy5BxEK4VnY7i8FMszNXrTzSskQQLo13fmBDaKn2HODCIAKAt7CqZNO
         mujQ==
X-Forwarded-Encrypted: i=1; AJvYcCUpvC2tvRrwq2S7WnW85lPRiOr/iOvf9tO04G6L0SDloJoSpRNcdBNCihdvSDyBcsbnwdsZq6rBzrPUreN8Q4lQsxAz6ZDZiTDLxI5uPcI=
X-Gm-Message-State: AOJu0Yw/ITEpqjhD6lLsceXXp2VjaYQ5RhbvicdCRF8Pj0zuMeq74Hl9
	w2xJ6Tyj3IV6Imuv4eoHH5csCtU6OrsAv3hcGbT+ClDjTngCEjbaTfmu7j6PnA==
X-Google-Smtp-Source: AGHT+IF6z6BXLMBnB4ym4My21VZCZnICg1ooYHpgMoNG9iwD3uOi34qk6r69v3vRKUERpkUUbEYqfA==
X-Received: by 2002:a05:600c:1c26:b0:41b:e84d:67a3 with SMTP id j38-20020a05600c1c2600b0041be84d67a3mr3272754wms.0.1714645839909;
        Thu, 02 May 2024 03:30:39 -0700 (PDT)
Message-ID: <be052201-8e3c-4bd4-800b-08a5f717dcdc@suse.com>
Date: Thu, 2 May 2024 12:30:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Content-Language: en-US
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: "consulting @ bugseng . com" <consulting@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
 <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
 <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
 <4fee2998-e29b-43bb-855a-8600dbef9f13@suse.com>
 <6A50BE71-7F92-4B7B-9A58-1D3A084FDA3C@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6A50BE71-7F92-4B7B-9A58-1D3A084FDA3C@arm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 12:12, Luca Fancellu wrote:
>>> In any case it would be used soon also for other architectures using bootinfo.
>>
>> Oh, would it?
> 
> PPC people have plans on putting that interface in common:

I'm aware, but ...

> https://patchwork.kernel.org/project/xen-devel/patch/451705505ff7f80ec66c78cc2830196fa6e4090c.1712893887.git.sanastasio@raptorengineering.com/

... I can't spot any flexible array members in this patch.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 10:31:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 10:31:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715744.1117609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TiY-0005Ec-Sy; Thu, 02 May 2024 10:31:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715744.1117609; Thu, 02 May 2024 10:31:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TiY-0005EV-Q8; Thu, 02 May 2024 10:31:10 +0000
Received: by outflank-mailman (input) for mailman id 715744;
 Thu, 02 May 2024 10:31:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPCJ=MF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2TiX-0004oh-He
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 10:31:09 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16bea8d7-086f-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 12:31:08 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-41b79451153so46368485e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 03:31:08 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 g20-20020a05600c311400b0041bf685921dsm1493208wmo.0.2024.05.02.03.31.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 03:31:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16bea8d7-086f-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714645868; x=1715250668; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=PytkBrjX316DhoxmSp+I/eclZGVor1QD1/Gxdd7FvgQ=;
        b=mfajLPCWwS2XYJgngGL1ecXeVuNuspSzQypGNc2Aw4nyc/Ytvpncz+OUiyBfkWflNG
         4Sr9Up/squVzN4TCyR3eRw2DwtTJdedXwLyrd8FGVy8RYQES2byjwVm3dgAAcLEpp7ck
         p4DutpYvyidWM3Ky18PNs4jzHhuI/wJluZLdg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714645868; x=1715250668;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PytkBrjX316DhoxmSp+I/eclZGVor1QD1/Gxdd7FvgQ=;
        b=r6uVGgk/M9P88wnOTPPm99c26R/IGS3itaCeC+0x+PuPhkiDVPGBsDUzkI0zRZz73p
         ksqyUfGGx8BZxkheP1ej5Kq9o2VfGFS/zxTU3LojCZfLyamDASh5RZ81ILKqKO8IsnhZ
         kmO2jUC47N/6smNLoaGxu+ypvQonxFRkSa5EaCTnHJDuh2VB+XBEXhI7NYGgjZXkg0LA
         qPaJw5JQQRHOVoI7HQ8GQ1rrPMEOfz/bSEjScIr90aJ3rJN5YMaj8xm/Uo2W9tQB++ai
         MITboiZJY0dyjUuqrkKGOdImws/56JxTWwPwnwsVfA2sBmgEkCsKmrO8SVWXEevIRlLX
         6q+g==
X-Gm-Message-State: AOJu0YyLEYMyGJnJVBSrWL/nOaFstPPZnkxq6CEOuO1VmI6IA7ro9nC9
	gE3i4y5+prfeTuoI93Em7rKYhWJOOQep/jG/2rHxjMsU/BGByWNGR48B9fYRfF/zm1iIBLzXL1a
	m
X-Google-Smtp-Source: AGHT+IFlmX7BTc8Hlvkj5v7xK3CbyvqjB32tqVPMFhKg33ITQlw3w6KaUa0XYR7YLOB8zR4+L8etOw==
X-Received: by 2002:a05:600c:358f:b0:41c:2992:a3bd with SMTP id p15-20020a05600c358f00b0041c2992a3bdmr3798812wmq.27.1714645868149;
        Thu, 02 May 2024 03:31:08 -0700 (PDT)
Date: Thu, 2 May 2024 12:31:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: xen-devel@lists.xenproject.org,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: Re: [PATCH for-4.19] ppc/riscv: fix arch_acquire_resource_check()
Message-ID: <ZjNraq889KamBWfE@macbook>
References: <20240430153423.80875-1-roger.pau@citrix.com>
 <2cabc6a1448c6f18cab963bc463347f088d643dd.camel@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <2cabc6a1448c6f18cab963bc463347f088d643dd.camel@gmail.com>

On Thu, May 02, 2024 at 10:46:12AM +0200, Oleksii wrote:
> On Tue, 2024-04-30 at 17:34 +0200, Roger Pau Monne wrote:
> > None of the implementations support set_foreign_p2m_entry() yet,
> > neither they
> > have a p2m walk in domain_relinquish_resources() in order to remove
> > the foreign
> > mappings from the p2m and thus drop the extra refcounts.
> > 
> > Adjust the arch helpers to return false and introduce a comment that
> > clearly
> > states it is not only taking extra refcounts that's needed, but also
> > dropping
> > them on domain teardown.
> I am okay with such adjustment for now as it is more safe and nothing
> will be missed during implementation of p2m, but I am curious how then
> Arm handles that, their implementation is also just returns true. ( I
> planned to have p2m implementation similar to Arm )

Arm does have an implementation of set_foreign_p2m_entry() and
domain_relinquish_resources() that handle foreign mappings correctly.

> 
> Anyway, based on that it safer for RISC-V:
> Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 02 10:42:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 10:42:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715752.1117619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TtP-0007pP-0h; Thu, 02 May 2024 10:42:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715752.1117619; Thu, 02 May 2024 10:42:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2TtO-0007pI-UC; Thu, 02 May 2024 10:42:22 +0000
Received: by outflank-mailman (input) for mailman id 715752;
 Thu, 02 May 2024 10:42:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=A7KU=MF=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s2TtN-0007pC-1U
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 10:42:21 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05on20601.outbound.protection.outlook.com
 [2a01:111:f403:2613::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a62b1db6-0870-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 12:42:19 +0200 (CEST)
Received: from AM0PR02CA0079.eurprd02.prod.outlook.com (2603:10a6:208:154::20)
 by DB3PR08MB9087.eurprd08.prod.outlook.com (2603:10a6:10:43c::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.36; Thu, 2 May
 2024 10:42:16 +0000
Received: from AM4PEPF00025F9C.EURPRD83.prod.outlook.com
 (2603:10a6:208:154:cafe::ea) by AM0PR02CA0079.outlook.office365.com
 (2603:10a6:208:154::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29 via Frontend
 Transport; Thu, 2 May 2024 10:42:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00025F9C.mail.protection.outlook.com (10.167.16.11) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7519.0
 via Frontend Transport; Thu, 2 May 2024 10:42:15 +0000
Received: ("Tessian outbound af213ececc3d:v315");
 Thu, 02 May 2024 10:42:15 +0000
Received: from 695135e593ff.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 135B1535-9E40-417E-BCEC-CBA6D6AB1349.1; 
 Thu, 02 May 2024 10:42:07 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 695135e593ff.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 May 2024 10:42:07 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by PAXPR08MB7441.eurprd08.prod.outlook.com (2603:10a6:102:2b9::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29; Thu, 2 May
 2024 10:42:05 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7519.031; Thu, 2 May 2024
 10:42:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a62b1db6-0870-11ef-909b-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=cSgG+JNfxwH8ex4qvgvGcZxMTdPVztMSKj4NT8vVJEv33uIUoaG5MOAZFrN7YKKfVTHytKLgbi79F0JyNbLrqhf9f3vVSzPxqYC3eIvSGEbVySAB2HjtVbiKuMgohyHEpUiSkQF2XWBx6qNdq+oy/Fym0yZ3z4IvBBxnE3cWSJxIqqNP4jZCfI1MYqlW7ZixcNTFmvCGHhw3L7vmfraKBZqm3kEpumngs8KXeecf7Dwqkatu2H7oZZ7809h3wX0cN1c84vQgpDOW9WZYSrXYoStpvsfCJFafDONzWfopC4v/XbUlb/3qtdundcKOG02JViraQ/k10iDunYVqB0QYVQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=9K0ZGamDqyoPVex3PZ9FO3vycwjLLcguT8T4UOwvcHg=;
 b=dasyRi1j6OYdZqc24E3lsOcWtFef7ezZW/C3Dvai1T4E3lkeORbGwjZKZauDKNTugxQnrv/R0+yynAka17PE0AYVX3six2Km45eIgnRxPWz83S8pQZYa9kK/uaqqJk26dlFvn1InCSZ1v08U1d3k/bn0aYcDmkf3uVsJyRvv9HjzHheEhCVvnFpO9bsyTiUJJkptKMkfMQYTzW8NJ48E7BtKRjvqtOVMe0omXtYqSW3VS2ebOxau/LMldKGGCKcvwWF2fsYGjMmpjcWnOVA3En5gg9ZoG9hfe/YgeFgsyjx/PKHCVUslXLE6PVGNJMoxHxq+JVmNWuUxVkVViQe3FQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9K0ZGamDqyoPVex3PZ9FO3vycwjLLcguT8T4UOwvcHg=;
 b=ZStWzAd2z7bc9LIrkQdg84aL4IvSAcb9Fo22Jb5VDW0sgD7XOjwS1YntnHLuVZ9iMZYNYsk78WpcjaVtVkMHFWZ39U3UjBhRz9IBDwqsnXi9wR7n+gn6gBUQjSiNXkLnP6hIAon6pg+x9U0yd7kiVr/vdB4UhFwaGbFVVL0UFVo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 106821810e08db42
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bPFt8iyJSKycwF6us0QYwB4Ii2kR3PmkoaTemiqhHRGHwFHkXs5iVMOxcTx49dwpFyCnTZVYi9vX1pEdj9oczZDP+0+55uF729++NZAbq8JwVgzr0jGgbf6uTE7s9QLvhAO+ZUxeO9ZqlHpexeOPKjOyDYPDQHjam4r6wOAv9bHyWjFEUItJQTay0ysrqtssUkXm+uFuVeTBQEgMcS1vlVYwY+OLeum+id/Q4klPRE8Tbych6NaTpfD3FuiRdjqYSRih7/o3HmQdnlRUjaHbnKmnQK9i2d+3xEZaZtehzN2Q753IhX0w+sDuwrtwjTy4f+4h4PK2G43kj5y+rKLcRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=9K0ZGamDqyoPVex3PZ9FO3vycwjLLcguT8T4UOwvcHg=;
 b=P07mmGsimz4qjgjRUJWJhbcTIQfNsou4DjlWPhAVm0+W3q139jNLFZzqriRvunQLTGVzI0K3kVTLT4fHDhY8w0FAQ/5L2VDeCY4qfdFK6GIS0vS9WEHvIYAk/SlhUsWPC/qMYKs58fd9SXNFmQ7/JI6UW4IvyjoTseR1yWFdJAnE86bIXblvZodfkv1Wot7pt09Uvfz+mYV/FOmoF7suVwPpAyApySmZM6TXl7i5y0lEmppqR4Q0Mf6xbwQdZhdy39Fwctv/Nj/N2tLqvljER8uw1pJ3P1SixBkGFjHKhubUYGSn6aQRkz5C9jQtEEoR1c1woQNy/hTWI4v446ncwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9K0ZGamDqyoPVex3PZ9FO3vycwjLLcguT8T4UOwvcHg=;
 b=ZStWzAd2z7bc9LIrkQdg84aL4IvSAcb9Fo22Jb5VDW0sgD7XOjwS1YntnHLuVZ9iMZYNYsk78WpcjaVtVkMHFWZ39U3UjBhRz9IBDwqsnXi9wR7n+gn6gBUQjSiNXkLnP6hIAon6pg+x9U0yd7kiVr/vdB4UhFwaGbFVVL0UFVo=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "consulting @ bugseng . com" <consulting@bugseng.com>, Nicola Vetrini
	<nicola.vetrini@bugseng.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index:
 AQHamu72Eq1RpEGs60mNp9DIfY8mn7GAr6+AgAFEGQCAAYYzgIAABVAAgAACyoCAABlOgIAAG+eAgAAFWACAAAUMAIAAAyeA
Date: Thu, 2 May 2024 10:42:05 +0000
Message-ID: <7DAAE080-61F7-49E9-9DA9-CF0C02237ADA@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <327c7e42-9b59-4925-b746-0b59dbb5a3fb@suse.com>
 <91B23E5F-7515-4C4E-9FBD-57654980905F@arm.com>
 <de09e134-0150-49ca-8338-a0f4759c218b@suse.com>
 <29573F6B-BE27-4A65-A310-35883A39F989@arm.com>
 <af1116ad-6401-403b-a3b8-abe421e2e3da@suse.com>
 <2DF15520-B0A4-4972-92F6-FCB6BB852292@arm.com>
 <4fee2998-e29b-43bb-855a-8600dbef9f13@suse.com>
 <6A50BE71-7F92-4B7B-9A58-1D3A084FDA3C@arm.com>
 <be052201-8e3c-4bd4-800b-08a5f717dcdc@suse.com>
In-Reply-To: <be052201-8e3c-4bd4-800b-08a5f717dcdc@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|PAXPR08MB7441:EE_|AM4PEPF00025F9C:EE_|DB3PR08MB9087:EE_
X-MS-Office365-Filtering-Correlation-Id: 467d8e35-1ed7-4b84-0ccc-08dc6a948867
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?DjcRIDICfPzaO41a33nfTBw7s09wnQWmIhuP3u6rZQu0+WWJgdKJgog4ZH/s?=
 =?us-ascii?Q?hgjWCWaLewHsUVGhs1Pjs4PHwNorrWltF/Ke+QVzRnnCUjy04Q6ipsOmKGmU?=
 =?us-ascii?Q?5v2S14cAoF0uZPopzBSixzcG5Rh1Dm07e6JVL7ApPh9mCvt5CiwRtORSBe+P?=
 =?us-ascii?Q?rI1+WadgY4Fi+X20QRPIMyeXT3SNDkReQjcvpIx0wE3CupRg7MRmfkpUBQ/l?=
 =?us-ascii?Q?NjXeClfM81pomxdU55jr6oWSPs0Flz1+AggNjRxPxFUFO4XpmFc/zRJhCWau?=
 =?us-ascii?Q?2LfU3KaXgArjSt1KfMzoX17xwwdCNjsDulwdQkEHrLtxbCFSLKx1zinZ0DAh?=
 =?us-ascii?Q?tugRO1Q5w4kQj2WIbEGhyBYhqOJDwVc+5sY9GlK0ZIuoTCXIQcXFoWVlZeo+?=
 =?us-ascii?Q?GC9dRMT+kVUUosugLt4yCNvgSFRX93SmkANw3bcwFpZzd4dX+/z0ACkBhEsQ?=
 =?us-ascii?Q?4mSY3Lg4/31/85dawDdSZi7EvTRb/XHieUtRZBYhSkAzblqNHjyf9CtmGh9J?=
 =?us-ascii?Q?XvsZOaJeAhbUhIbX0cG2o37zr+8pV5HQVWrMPCwKZEoEW1H6ObHsYFJWC6sH?=
 =?us-ascii?Q?clsM8qzFhZlhnGlBfu1X20wBuwGi3HcWWPqlshl72K32wYjnlRF7OuUwEwUs?=
 =?us-ascii?Q?QZd2tJSfIyDWyivO/6VoNuRttstjCF4HVgVTvu7vrFJK/j2priV4/ahfOPDN?=
 =?us-ascii?Q?Q9kUzTqxGlPQ76ayDuz3lIFR7HT4VA2bXQmmylhMlMUQZWNjxCxK/CGOI+3w?=
 =?us-ascii?Q?MvqUB11FJu69tZBbskef5+jL6+xFlPPxS/d5fzgXL6TS2znmMC83RtC16OOv?=
 =?us-ascii?Q?eFuX8JmMaoUufucv4NVlXESmNuSL/lnTt6RlUbIjdUxSJcVFoiD0UoAeAzKE?=
 =?us-ascii?Q?4MvH3CSNTOE6uGhkjfk7g37bhoewrfdxmao3QZBVjBPCvAXvM1Zux4ELgPs5?=
 =?us-ascii?Q?PwxLA8A4cX50c1h8CiQa/bBqTgrTnPEacVNiKQ+eI0DpG8h51O0Mf8FQEGau?=
 =?us-ascii?Q?QJKcnz8QttyOk+AGK4LTw8eGH8fwCCt858Hafo7lLBFKaDneJRLCHBpHV/PO?=
 =?us-ascii?Q?JhrlQeb/iLj6psmaA2xRCLojbBOrv55GBFvgjLGJAtJevg86UFwtChcBp3xB?=
 =?us-ascii?Q?QpGKYx+ZZZ/bf52f17KTZljrY5afBwgEV2sRXwLHkVbNB3kH7S0D/hOTsUqg?=
 =?us-ascii?Q?AWfEAAHbavyvmhyns8s/KO6tFbopEsDeUNFKu1PM0v8/E8JTtAiLCHGNk2ro?=
 =?us-ascii?Q?VcIyJElRIWgAhUGCR7UbZfZ+LA2JxWBwncZJA8Uuhg=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <CB924931D885334BB8958761197F91A9@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7441
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00025F9C.EURPRD83.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7b909418-877a-46a7-7991-08dc6a948266
X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?ShP6Y75tG88lrdph5L3xP9V6AY/Gbbg/XDRIh3xOpnzXp0CQaED/gqzYLFKO?=
 =?us-ascii?Q?pUBst8pVdm6rCiPC0yXVr5s4I5FdHNUJAm2MleyNH1tK769k3I/QTHA3d9Mv?=
 =?us-ascii?Q?wbnpOqbKCC/DaU2KomSzb+Fos3EJn3Sg23lT5/f61G5+5J7feiUMLCrCEEyC?=
 =?us-ascii?Q?J/YHf6lJHEfyJBzqUjfxj4IKrejIiraqLCriX+V8IVTMa2uca//wyR9z+71t?=
 =?us-ascii?Q?CF47WyYwoitbT021dCki6hOvY3S/3GRZ3cdb2NPnUq/E/7c2fl1nuy5FK+fj?=
 =?us-ascii?Q?Fk7TpjG76mF1XLOPMOIYYaAe2KaY9bWCO6/qd0J08LZ/4PlBSwGuDqSECeaI?=
 =?us-ascii?Q?vhGWq+bRL0n8MawQkOf8o4AdwJ9rsdHF3+/qZ7vzTg8f0IvEP+0taV5HkuVZ?=
 =?us-ascii?Q?OvhMWQ8k2/M8WBU/iYlXzLw6vyN7ACE3KxNkKBwOTJ2CZp+7BTiytTtw4a8f?=
 =?us-ascii?Q?D8rM3QkqmT60P1gk2ZkQE/MZ6/9hCHVv+lU8eeGCpD35d9IrfKYRNfgcLp3D?=
 =?us-ascii?Q?H3mMvQegwBsyugkmRrYOlORYgk1uTVZ/5E+DT3a9sS/oh0ALb+X5aHC0Yydj?=
 =?us-ascii?Q?fRsKOUho4CzQrHMJbIgNgcadKuQM2WuXc1uKEJVy5jZEKtC8tF/I4iK2lXiU?=
 =?us-ascii?Q?GLbpH1SvCuwG88rhBLRFXOl1wvSwR3n7xLsrIqCIYDFaqaBRqSvLHJ1ep3gl?=
 =?us-ascii?Q?tWQtmTDQW0jbORz8Y8fbsEhVXfWkjcZZvynNCMh8trwa5h3DraZzZwn1F0tF?=
 =?us-ascii?Q?ELAW9VVKlyTSuJdDEaDtiK6q/K57HQeUbyo2jkWws2/qLd5xFilMTVHToiUq?=
 =?us-ascii?Q?GMJGoTcP/9Gv0khchT07MOTX2MJku6tPQrmuTnom2uBoDWoVV7Jw4JAKIlQL?=
 =?us-ascii?Q?kwbBxAeOupeUlkaNbRmRpLM5nYHG2NeHRtzOT41gl/dROkne7p8TP3lKzZgU?=
 =?us-ascii?Q?zGZ0XTNDInF4neHzbtakPpuSbGPCN4iQAZgkpML+JvZ6ffz0frxGzv+Nog6b?=
 =?us-ascii?Q?2W7IkC1x6DKYlCFeMJ8i/akUKUwq/Ex1SJvelvdFt+ffE97iRqdLjnLw3YFP?=
 =?us-ascii?Q?gL+sgm5FLKeJKDX3TULIq397QSbosiA6CYahrHcorqlfX6JN1erBMqWKdEwL?=
 =?us-ascii?Q?KfEsVmhI7QSBFJ1CB4hDWHmE/jE5sYrOo9xzEFdzF9Lhst014mYCMGfqnEaQ?=
 =?us-ascii?Q?NOnDFTOL/5RrbkwxbUWZrXwgj8jVzyvztBcwWzf94lmpsIM8RfK8DfMIDEsq?=
 =?us-ascii?Q?hUZ/oVd+Vj3XtXp4qgdRdlySrjKSG4wmCZuS3/oHjeZT+xNPCJEgHeVT+MQs?=
 =?us-ascii?Q?2kYAFMnKCX9Xc0fZlQwxhOUt?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 10:42:15.5301
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 467d8e35-1ed7-4b84-0ccc-08dc6a948867
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00025F9C.EURPRD83.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR08MB9087



> On 2 May 2024, at 11:30, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 02.05.2024 12:12, Luca Fancellu wrote:
>>>> In any case it would be used soon also for other architectures using b=
ootinfo.
>>>=20
>>> Oh, would it?
>>=20
>> PPC people have plans on putting that interface in common:
>=20
> I'm aware, but ...
>=20
>> https://patchwork.kernel.org/project/xen-devel/patch/451705505ff7f80ec66=
c78cc2830196fa6e4090c.1712893887.git.sanastasio@raptorengineering.com/
>=20
> ... I can't spot any flexible array members in this patch.

I guess v5 of that patch will have that, because it would be rebased on the=
 latest state of
xen/arch/arm/include/asm/setup.h, moving that interface in xen/include/xen/=
bootfdt.h, in
order to use part of the code taken out from xen/arch/arm/setup.c and put i=
n
xen/common/device-tree/bootinfo.c




From xen-devel-bounces@lists.xenproject.org Thu May 02 11:49:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 11:49:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715757.1117628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Uvs-0001n5-Pp; Thu, 02 May 2024 11:49:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715757.1117628; Thu, 02 May 2024 11:49:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Uvs-0001my-N1; Thu, 02 May 2024 11:49:00 +0000
Received: by outflank-mailman (input) for mailman id 715757;
 Thu, 02 May 2024 11:49:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Uvs-0001mo-9R; Thu, 02 May 2024 11:49:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Uvs-0001yi-2c; Thu, 02 May 2024 11:49:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Uvr-0004uw-KD; Thu, 02 May 2024 11:48:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2Uvr-00048U-Jj; Thu, 02 May 2024 11:48:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=L6gu8bkIzMW1Cee4Cu/8kLkKTu1LP27rMifHJKHZ2YA=; b=u3IWh04GdfnwgjFL30eY9/qvyR
	A1Pz4u263WMYhG0iTQTUPnoyVBaZQ4Lw5mGEPib3xfB7cLAW2wJDEnaQOxt9n2tRCFZ3zMo7iDD1L
	v+RMLK9SicZsunWK0rSg9lMBEPZKubkIoRUzODnrey5csPivbZ13noFoFHZlWDgYmSqA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185897-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185897: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
X-Osstest-Versions-That:
    xen=9e30bd8f4a8cee83368276c5149c7e2304efe5a2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 11:48:59 +0000

flight 185897 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185897/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185890
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185890
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185890
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185890
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185890
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185890
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b
baseline version:
 xen                  9e30bd8f4a8cee83368276c5149c7e2304efe5a2

Last test of basis   185890  2024-05-01 02:54:12 Z    1 days
Testing same since   185897  2024-05-02 01:53:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Federico Serafini <federico.serafini@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9e30bd8f4a..feb9158a62  feb9158a620040846d76981acbe8ea9e2255a07b -> master


From xen-devel-bounces@lists.xenproject.org Thu May 02 11:50:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 11:50:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715762.1117639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Uxe-0003Ef-91; Thu, 02 May 2024 11:50:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715762.1117639; Thu, 02 May 2024 11:50:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Uxe-0003EY-5E; Thu, 02 May 2024 11:50:50 +0000
Received: by outflank-mailman (input) for mailman id 715762;
 Thu, 02 May 2024 11:50:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPCJ=MF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2Uxc-0003EJ-6l
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 11:50:48 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35edd9cd-087a-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 13:50:45 +0200 (CEST)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-34e0d8b737eso481267f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 04:50:45 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 p12-20020adfe60c000000b0034ccd06a6a3sm1091868wrm.18.2024.05.02.04.50.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 04:50:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35edd9cd-087a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714650645; x=1715255445; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=BkBrlgwIFh4FEHp4gI0dRjqZ/EeBYAdyEfWwsVz4QbU=;
        b=cXnZodqZVJoO6C1BjQdh1fYFR+FNQpzkMFMrAvF+k54foMKwAx8auC7hAa1mk2QHA8
         EdU3LL08B0U3ymeZEHYYC40usvQnOnmv8Muwwcm/7Niw6gCBoiTUFy8Fut9OnCivJUt6
         HYiQuqYYuaDxPqkRf5WiQF7BQZ04PP1hRWw2g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714650645; x=1715255445;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BkBrlgwIFh4FEHp4gI0dRjqZ/EeBYAdyEfWwsVz4QbU=;
        b=L7aEU2ywPrWHNY7pDPg3q2PeYx27+BEYCBY5HYdugi3r/gk0+SODzFqbRni84sXnuy
         6lO+ub+e2bLSMN5vmhV5q6tu8XDqMVdUFmJMFSRmXdxwa6fuqK3ETHh+k0Q/GIFnvDM/
         dnLSjtj7zXCNH5qIWbXYrJZu5mlfM4ZmcWgukbxc+E4ttV0KELmyfERIWF7czUyhApQX
         2GHjd3s3+VBgrS4RzBRphFQfTFNYIC9sQ0ufGUyLoFzFJ5mm0PFtWbRjOBbmRyKajUUT
         luhokM7xWEhdFjr4E+SLkRGB/fBlDroXKs9PXKdQUgDVL0uQcV2DYO1hNlMBZ3VS7upN
         DCpw==
X-Gm-Message-State: AOJu0Yx55eEmV+mHmE03MZDmc0hNFtajYOBsMHX1IwGrEfpnGSI3ryn5
	tkI3KYGNYgT1FIYNCx2RhTp9046SXx/Xs4VHGKwUDNIdHOMEswzNMp5Rj5pjTROtOZOvBIPTSeX
	w
X-Google-Smtp-Source: AGHT+IGnHCm7STGcxijw75NO54366Q4LybvB65nqrM/je2OsY+GSS0bkBL+ETGffElucilhYoOyERg==
X-Received: by 2002:a5d:5145:0:b0:34d:af19:2373 with SMTP id u5-20020a5d5145000000b0034daf192373mr1995958wrt.16.1714650644756;
        Thu, 02 May 2024 04:50:44 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy defined names
Date: Thu,  2 May 2024 13:49:22 +0200
Message-ID: <20240502114922.94288-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Like it was done recently for libxl, switch to using the auto-generated feature
names by the processing of cpufeatureset.h, this allows removing the open-coded
feature names, and unifies the feature naming with libxl and the hypervisor.

Introduce a newly auto-generated array that contains the feature names indexed
at featureset bit position, otherwise using the existing INIT_FEATURE_NAMES
would require iterating over the array elements until a match with the expected
bit position is found.

Note that leaf names need to be kept, as the current auto-generated data
doesn't contain the leaf names.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Modify gen-cpuid.py to generate an array of strings with the feature names.
 - Introduce and use __maybe_unused.
---
 tools/include/xen-tools/common-macros.h |   4 +
 tools/misc/xen-cpuid.c                  | 320 +++---------------------
 xen/tools/gen-cpuid.py                  |  26 ++
 3 files changed, 68 insertions(+), 282 deletions(-)

diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
index 07aed92684b5..3e6a66080a4f 100644
--- a/tools/include/xen-tools/common-macros.h
+++ b/tools/include/xen-tools/common-macros.h
@@ -83,6 +83,10 @@
 #define __packed __attribute__((__packed__))
 #endif
 
+#ifndef __maybe_unused
+# define __maybe_unused __attribute__((__unused__))
+#endif
+
 #define container_of(ptr, type, member) ({              \
     typeof(((type *)0)->member) *mptr__ = (ptr);        \
     (type *)((char *)mptr__ - offsetof(type, member));  \
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 8893547bebce..2a1ac0ee8326 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -12,282 +12,33 @@
 
 #include <xen-tools/common-macros.h>
 
-static uint32_t nr_features;
-
-static const char *const str_1d[32] =
-{
-    [ 0] = "fpu",  [ 1] = "vme",
-    [ 2] = "de",   [ 3] = "pse",
-    [ 4] = "tsc",  [ 5] = "msr",
-    [ 6] = "pae",  [ 7] = "mce",
-    [ 8] = "cx8",  [ 9] = "apic",
-    /* [10] */     [11] = "sysenter",
-    [12] = "mtrr", [13] = "pge",
-    [14] = "mca",  [15] = "cmov",
-    [16] = "pat",  [17] = "pse36",
-    [18] = "psn",  [19] = "clflush",
-    /* [20] */     [21] = "ds",
-    [22] = "acpi", [23] = "mmx",
-    [24] = "fxsr", [25] = "sse",
-    [26] = "sse2", [27] = "ss",
-    [28] = "htt",  [29] = "tm",
-    [30] = "ia64", [31] = "pbe",
-};
-
-static const char *const str_1c[32] =
-{
-    [ 0] = "sse3",    [ 1] = "pclmulqdq",
-    [ 2] = "dtes64",  [ 3] = "monitor",
-    [ 4] = "ds-cpl",  [ 5] = "vmx",
-    [ 6] = "smx",     [ 7] = "est",
-    [ 8] = "tm2",     [ 9] = "ssse3",
-    [10] = "cntx-id", [11] = "sdgb",
-    [12] = "fma",     [13] = "cx16",
-    [14] = "xtpr",    [15] = "pdcm",
-    /* [16] */        [17] = "pcid",
-    [18] = "dca",     [19] = "sse41",
-    [20] = "sse42",   [21] = "x2apic",
-    [22] = "movebe",  [23] = "popcnt",
-    [24] = "tsc-dl",  [25] = "aesni",
-    [26] = "xsave",   [27] = "osxsave",
-    [28] = "avx",     [29] = "f16c",
-    [30] = "rdrnd",   [31] = "hyper",
-};
-
-static const char *const str_e1d[32] =
-{
-    [ 0] = "fpu",    [ 1] = "vme",
-    [ 2] = "de",     [ 3] = "pse",
-    [ 4] = "tsc",    [ 5] = "msr",
-    [ 6] = "pae",    [ 7] = "mce",
-    [ 8] = "cx8",    [ 9] = "apic",
-    /* [10] */       [11] = "syscall",
-    [12] = "mtrr",   [13] = "pge",
-    [14] = "mca",    [15] = "cmov",
-    [16] = "fcmov",  [17] = "pse36",
-    /* [18] */       [19] = "mp",
-    [20] = "nx",     /* [21] */
-    [22] = "mmx+",   [23] = "mmx",
-    [24] = "fxsr",   [25] = "fxsr+",
-    [26] = "pg1g",   [27] = "rdtscp",
-    /* [28] */       [29] = "lm",
-    [30] = "3dnow+", [31] = "3dnow",
-};
-
-static const char *const str_e1c[32] =
-{
-    [ 0] = "lahf-lm",    [ 1] = "cmp",
-    [ 2] = "svm",        [ 3] = "extapic",
-    [ 4] = "cr8d",       [ 5] = "lzcnt",
-    [ 6] = "sse4a",      [ 7] = "msse",
-    [ 8] = "3dnowpf",    [ 9] = "osvw",
-    [10] = "ibs",        [11] = "xop",
-    [12] = "skinit",     [13] = "wdt",
-    /* [14] */           [15] = "lwp",
-    [16] = "fma4",       [17] = "tce",
-    /* [18] */           [19] = "nodeid",
-    /* [20] */           [21] = "tbm",
-    [22] = "topoext",    [23] = "perfctr-core",
-    [24] = "perfctr-nb", /* [25] */
-    [26] = "dbx",        [27] = "perftsc",
-    [28] = "pcx-l2i",    [29] = "monitorx",
-    [30] = "addr-msk-ext",
-};
-
-static const char *const str_7b0[32] =
-{
-    [ 0] = "fsgsbase", [ 1] = "tsc-adj",
-    [ 2] = "sgx",      [ 3] = "bmi1",
-    [ 4] = "hle",      [ 5] = "avx2",
-    [ 6] = "fdp-exn",  [ 7] = "smep",
-    [ 8] = "bmi2",     [ 9] = "erms",
-    [10] = "invpcid",  [11] = "rtm",
-    [12] = "pqm",      [13] = "depfpp",
-    [14] = "mpx",      [15] = "pqe",
-    [16] = "avx512f",  [17] = "avx512dq",
-    [18] = "rdseed",   [19] = "adx",
-    [20] = "smap",     [21] = "avx512-ifma",
-    [22] = "pcommit",  [23] = "clflushopt",
-    [24] = "clwb",     [25] = "proc-trace",
-    [26] = "avx512pf", [27] = "avx512er",
-    [28] = "avx512cd", [29] = "sha",
-    [30] = "avx512bw", [31] = "avx512vl",
-};
-
-static const char *const str_Da1[32] =
-{
-    [ 0] = "xsaveopt", [ 1] = "xsavec",
-    [ 2] = "xgetbv1",  [ 3] = "xsaves",
-};
-
-static const char *const str_7c0[32] =
-{
-    [ 0] = "prefetchwt1",      [ 1] = "avx512-vbmi",
-    [ 2] = "umip",             [ 3] = "pku",
-    [ 4] = "ospke",            [ 5] = "waitpkg",
-    [ 6] = "avx512-vbmi2",     [ 7] = "cet-ss",
-    [ 8] = "gfni",             [ 9] = "vaes",
-    [10] = "vpclmulqdq",       [11] = "avx512-vnni",
-    [12] = "avx512-bitalg",
-    [14] = "avx512-vpopcntdq",
-
-    [22] = "rdpid",
-    /* 24 */                   [25] = "cldemote",
-    /* 26 */                   [27] = "movdiri",
-    [28] = "movdir64b",        [29] = "enqcmd",
-    [30] = "sgx-lc",           [31] = "pks",
-};
-
-static const char *const str_e7d[32] =
-{
-    /* 6 */                    [ 7] = "hw-pstate",
-    [ 8] = "itsc",             [ 9] = "cpb",
-    [10] = "efro",
-};
-
-static const char *const str_e8b[32] =
-{
-    [ 0] = "clzero",
-    [ 2] = "rstr-fp-err-ptrs",
-
-    /* [ 8] */                 [ 9] = "wbnoinvd",
-
-    [12] = "ibpb",
-    [14] = "ibrs",             [15] = "amd-stibp",
-    [16] = "ibrs-always",      [17] = "stibp-always",
-    [18] = "ibrs-fast",        [19] = "ibrs-same-mode",
-
-    [20] = "no-lmsl",
-    /* [22] */                 [23] = "ppin",
-    [24] = "amd-ssbd",         [25] = "virt-ssbd",
-    [26] = "ssb-no",
-    [28] = "psfd",             [29] = "btc-no",
-    [30] = "ibpb-ret",
-};
-
-static const char *const str_7d0[32] =
-{
-    [ 2] = "avx512-4vnniw", [ 3] = "avx512-4fmaps",
-    [ 4] = "fsrm",
-
-    [ 8] = "avx512-vp2intersect", [ 9] = "srbds-ctrl",
-    [10] = "md-clear",            [11] = "rtm-always-abort",
-    /* 12 */                [13] = "tsx-force-abort",
-    [14] = "serialize",     [15] = "hybrid",
-    [16] = "tsxldtrk",
-    [18] = "pconfig",
-    [20] = "cet-ibt",
-    /* 22 */                [23] = "avx512-fp16",
-
-    [26] = "ibrsb",         [27] = "stibp",
-    [28] = "l1d-flush",     [29] = "arch-caps",
-    [30] = "core-caps",     [31] = "ssbd",
-};
-
-static const char *const str_7a1[32] =
-{
-    [ 0] = "sha512",        [ 1] = "sm3",
-    [ 2] = "sm4",
-    [ 4] = "avx-vnni",      [ 5] = "avx512-bf16",
-
-    [10] = "fzrm",          [11] = "fsrs",
-    [12] = "fsrcs",
-
-    /* 18 */                [19] = "wrmsrns",
+#include <xen/lib/x86/cpu-policy.h>
 
-    /* 22 */                [23] = "avx-ifma",
-};
-
-static const char *const str_e21a[32] =
-{
-    [ 0] = "no-nest-bp",    [ 1] = "fs-gs-ns",
-    [ 2] = "lfence+",
-    [ 6] = "nscb",
-    [ 8] = "auto-ibrs",
-    [10] = "amd-fsrs",      [11] = "amd-fsrc",
-
-    /* 16 */                [17] = "cpuid-user-dis",
-    [18] = "epsf",          [19] = "fsrsc",
-    [20] = "amd-prefetchi",
-
-    /* 26 */                [27] = "sbpb",
-    [28] = "ibpb-brtype",   [29] = "srso-no",
-};
-
-static const char *const str_7b1[32] =
-{
-    [ 0] = "ppin",
-};
-
-static const char *const str_7c1[32] =
-{
-};
-
-static const char *const str_7d1[32] =
-{
-    [ 4] = "avx-vnni-int8",       [ 5] = "avx-ne-convert",
-
-    [10] = "avx-vnni-int16",
-
-    [14] = "prefetchi",
-
-    [18] = "cet-sss",
-};
-
-static const char *const str_7d2[32] =
-{
-    [ 0] = "intel-psfd",    [ 1] = "ipred-ctrl",
-    [ 2] = "rrsba-ctrl",    [ 3] = "ddp-ctrl",
-    [ 4] = "bhi-ctrl",      [ 5] = "mcdt-no",
-};
-
-static const char *const str_m10Al[32] =
-{
-    [ 0] = "rdcl-no",             [ 1] = "eibrs",
-    [ 2] = "rsba",                [ 3] = "skip-l1dfl",
-    [ 4] = "intel-ssb-no",        [ 5] = "mds-no",
-    [ 6] = "if-pschange-mc-no",   [ 7] = "tsx-ctrl",
-    [ 8] = "taa-no",              [ 9] = "mcu-ctrl",
-    [10] = "misc-pkg-ctrl",       [11] = "energy-ctrl",
-    [12] = "doitm",               [13] = "sbdr-ssdp-no",
-    [14] = "fbsdp-no",            [15] = "psdp-no",
-    /* 16 */                      [17] = "fb-clear",
-    [18] = "fb-clear-ctrl",       [19] = "rrsba",
-    [20] = "bhi-no",              [21] = "xapic-status",
-    /* 22 */                      [23] = "ovrclk-status",
-    [24] = "pbrsb-no",            [25] = "gds-ctrl",
-    [26] = "gds-no",              [27] = "rfds-no",
-    [28] = "rfds-clear",
-};
-
-static const char *const str_m10Ah[32] =
-{
-};
+static uint32_t nr_features;
 
 static const struct {
     const char *name;
     const char *abbr;
-    const char *const *strs;
-} decodes[] =
+} leaf_names[] =
 {
-    { "CPUID 0x00000001.edx",        "1d", str_1d },
-    { "CPUID 0x00000001.ecx",        "1c", str_1c },
-    { "CPUID 0x80000001.edx",       "e1d", str_e1d },
-    { "CPUID 0x80000001.ecx",       "e1c", str_e1c },
-    { "CPUID 0x0000000d:1.eax",     "Da1", str_Da1 },
-    { "CPUID 0x00000007:0.ebx",     "7b0", str_7b0 },
-    { "CPUID 0x00000007:0.ecx",     "7c0", str_7c0 },
-    { "CPUID 0x80000007.edx",       "e7d", str_e7d },
-    { "CPUID 0x80000008.ebx",       "e8b", str_e8b },
-    { "CPUID 0x00000007:0.edx",     "7d0", str_7d0 },
-    { "CPUID 0x00000007:1.eax",     "7a1", str_7a1 },
-    { "CPUID 0x80000021.eax",      "e21a", str_e21a },
-    { "CPUID 0x00000007:1.ebx",     "7b1", str_7b1 },
-    { "CPUID 0x00000007:2.edx",     "7d2", str_7d2 },
-    { "CPUID 0x00000007:1.ecx",     "7c1", str_7c1 },
-    { "CPUID 0x00000007:1.edx",     "7d1", str_7d1 },
-    { "MSR_ARCH_CAPS.lo",         "m10Al", str_m10Al },
-    { "MSR_ARCH_CAPS.hi",         "m10Ah", str_m10Ah },
+    { "CPUID 0x00000001.edx",        "1d" },
+    { "CPUID 0x00000001.ecx",        "1c" },
+    { "CPUID 0x80000001.edx",       "e1d" },
+    { "CPUID 0x80000001.ecx",       "e1c" },
+    { "CPUID 0x0000000d:1.eax",     "Da1" },
+    { "CPUID 0x00000007:0.ebx",     "7b0" },
+    { "CPUID 0x00000007:0.ecx",     "7c0" },
+    { "CPUID 0x80000007.edx",       "e7d" },
+    { "CPUID 0x80000008.ebx",       "e8b" },
+    { "CPUID 0x00000007:0.edx",     "7d0" },
+    { "CPUID 0x00000007:1.eax",     "7a1" },
+    { "CPUID 0x80000021.eax",      "e21a" },
+    { "CPUID 0x00000007:1.ebx",     "7b1" },
+    { "CPUID 0x00000007:2.edx",     "7d2" },
+    { "CPUID 0x00000007:1.ecx",     "7c1" },
+    { "CPUID 0x00000007:1.edx",     "7d1" },
+    { "MSR_ARCH_CAPS.lo",         "m10Al" },
+    { "MSR_ARCH_CAPS.hi",         "m10Ah" },
 };
 
 #define COL_ALIGN "24"
@@ -305,12 +56,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
 {
     unsigned i;
 
-    if ( !strs )
-    {
-        printf(" ???");
-        return;
-    }
-
     for ( i = 0; i < 32; ++i )
         if ( leaf & (1u << i) )
         {
@@ -326,6 +71,7 @@ static void decode_featureset(const uint32_t *features,
                               const char *name,
                               bool detail)
 {
+    static const uint32_t __maybe_unused known_features[] = INIT_KNOWN_FEATURES;
     unsigned int i;
 
     printf("%-"COL_ALIGN"s        ", name);
@@ -336,11 +82,21 @@ static void decode_featureset(const uint32_t *features,
     if ( !detail )
         return;
 
-    for ( i = 0; i < length && i < ARRAY_SIZE(decodes); ++i )
+    /* Ensure leaf names stay in sync with the policy leaf count. */
+    BUILD_BUG_ON(ARRAY_SIZE(known_features) != ARRAY_SIZE(leaf_names));
+
+    for ( i = 0; i < length && i < ARRAY_SIZE(leaf_names); ++i )
     {
-        printf("  [%02u] %-"COL_ALIGN"s", i, decodes[i].name ?: "<UNKNOWN>");
-        if ( decodes[i].name )
-            dump_leaf(features[i], decodes[i].strs);
+        static const char *const feature_names[] = INIT_FEATURE_NAME_ARRAY;
+
+        /* Ensure feature names stays in sync with number of feature words. */
+        BUILD_BUG_ON(ARRAY_SIZE(feature_names) !=
+                     ARRAY_SIZE(leaf_names) * sizeof(*features) * 8);
+
+        printf("  [%02u] %-"COL_ALIGN"s", i, leaf_names[i].name ?: "<UNKNOWN>");
+
+        /* Attempt to print features anyway even if the leaf name is unknown. */
+        dump_leaf(features[i], &feature_names[i * 32]);
         printf("\n");
     }
 }
@@ -355,8 +111,8 @@ static void dump_info(xc_interface *xch, bool detail)
     if ( !detail )
     {
         printf("       %"COL_ALIGN"s ", "KEY");
-        for ( i = 0; i < ARRAY_SIZE(decodes); ++i )
-            printf("%-8s ", decodes[i].abbr ?: "???");
+        for ( i = 0; i < ARRAY_SIZE(leaf_names); ++i )
+            printf("%-8s ", leaf_names[i].abbr ?: "???");
         printf("\n");
     }
 
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 380b9d973a67..c0caa3bdebef 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -475,6 +475,32 @@ def write_results(state):
     state.output.write(
 """}
 
+""")
+
+    state.output.write(
+"""
+#define INIT_FEATURE_NAME_ARRAY { \\
+""")
+
+    try:
+        _tmp = state.names.iteritems()
+    except AttributeError:
+        _tmp = state.names.items()
+
+    for bit, name in sorted(_tmp):
+        state.output.write(
+            '    [%s] = "%s",\\\n' % (bit, name.lower().replace("_", "-"))
+            )
+
+    # Ensure array size matches featureset size.
+    if not state.names.get(state.nr_entries * 32 - 1, ""):
+        state.output.write(
+            '    [%s] = NULL,\\\n' % (state.nr_entries * 32 - 1)
+            )
+
+    state.output.write(
+"""}
+
 """)
 
     for idx, text in enumerate(state.bitfields):
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu May 02 11:59:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 11:59:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715772.1117661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2V6G-0004Zd-5T; Thu, 02 May 2024 11:59:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715772.1117661; Thu, 02 May 2024 11:59:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2V6G-0004ZW-2R; Thu, 02 May 2024 11:59:44 +0000
Received: by outflank-mailman (input) for mailman id 715772;
 Thu, 02 May 2024 11:59:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=qixW=MF=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s2V6E-0004ZQ-TW
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 11:59:42 +0000
Received: from mail-oa1-x34.google.com (mail-oa1-x34.google.com
 [2001:4860:4864:20::34])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73f8ca10-087b-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 13:59:40 +0200 (CEST)
Received: by mail-oa1-x34.google.com with SMTP id
 586e51a60fabf-23bd61fbd64so3197636fac.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 04:59:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73f8ca10-087b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714651179; x=1715255979; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AMoVTKWH18dJjINgtv7qyuGCrDSDOV+xIvp7psoPk/M=;
        b=bXcMUQK0NYIRDwJhCVgNnL21dHVgin89oKHKIH+/i6BKgaVix5BnfLvycBaxr3kMyq
         h9j8uAfanvJEpUdDn24kxeuwANTmOneuF4F/gCBpV5PmKWgcfbLLtgrLh3oX4bxMIc3P
         zBLrp4bCL6ZOEXbnlgfC0KMY8X1w5+ai7dKHw10nDWQO9oHJ/ZocoIITaAwMsIYocbRe
         ZFHQ8T9HNjXPv02wkf8vB7K69LvhsiZJU1aUXvm2CBdUEAV3j0CfLYJee1zPhrzHJ3hO
         EHvm1C0oniWHZvuzU6luo0OMsvf4BTUPDLmuinQihT6gcEY02AcgKvtrIJUEwKKke1yQ
         6OMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714651179; x=1715255979;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=AMoVTKWH18dJjINgtv7qyuGCrDSDOV+xIvp7psoPk/M=;
        b=CGLa89WnCTmWX191j6r9Rd52ZnAG2gEG+jKKqylkbH7zygQaMZwE3XnRN0Cns2xdkt
         FNrHtFg2E+b+NklLac1pEI8Z9DytRsoHGjYta4QkAZr19RY/s2lLRa6VowdcaXcFgsuQ
         7c2wWqusSdqPRCPa9EH06zC1aCkU55MN3sZbnsxlVqKWa8hePX7ESaapSi14Viv+Am/N
         Yldlp46Hv/QFcKbNL0WJvs7nXWHAAV2EPkDP2NOwTgm40SXSMtfPFWIcFm47a+J1QWZp
         uM2+74GvFHvaHCXBfRTpoVUkDf/OL41J4xM5TR+NRE76JzInsngg7UkP5/bD0d7BxfQq
         DToA==
X-Forwarded-Encrypted: i=1; AJvYcCUPp3ioBHpeEwVK94mt6skojAhESdupBvAbzp5E5l1alDGU9Xz1+ndmSdq2ivdZ1wpJeAFEMMc/GjGVxLqhIU6mpxU97DpFmBfoqwg8Ckk=
X-Gm-Message-State: AOJu0Ywwi5vEwAa7f1idEQY/Q+7JHZ11W06Dzphcg5f02LKEbYuZJYgU
	1+Um13zNAYiK5aaFb21UyWjXV7vz02uVReKjCpcfaUvjz5S5qQFZ7H7HgtLMmgL9F2MworBAHWt
	zBIZTMCPchKQOXjRoUtQwTXVtp18=
X-Google-Smtp-Source: AGHT+IEApEEvdHkGGz9ZaVXTp5QFSiyTY0QNOtZLM3nF8eKmcjdaUQ7iaLRv0kjVTgUn2cP9x+YIMZ7b/mdpPPQ9erM=
X-Received: by 2002:a05:6870:c0ce:b0:239:7008:b751 with SMTP id
 e14-20020a056870c0ce00b002397008b751mr6025496oad.34.1714651178792; Thu, 02
 May 2024 04:59:38 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1714322424.git.w1benny@gmail.com> <a26bc4aeba89f7895c79df7e320adfc695b16d50.1714322424.git.w1benny@gmail.com>
 <4d9930d4-4379-4440-83b2-14ebffc7c03d@suse.com> <CAKBKdXgAgb=+Spr_nobWy_zuN-yXTUfAL+Q3MdLDCDVgshOS7Q@mail.gmail.com>
 <115a6513-559c-44d9-a828-10d1e4e5c401@suse.com>
In-Reply-To: <115a6513-559c-44d9-a828-10d1e4e5c401@suse.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Thu, 2 May 2024 13:59:27 +0200
Message-ID: <CAKBKdXhE9hKiYZ=Kz34somE0U1HmoRCXpTREzMBEyiUTS4eNjg@mail.gmail.com>
Subject: Re: [PATCH v2 7/7] x86/hap: Increase the number of initial
 mempool_size to 1024 pages
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 2, 2024 at 8:36=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 30.04.2024 17:40, Petr Bene=C5=A1 wrote:
> > On Tue, Apr 30, 2024 at 4:47=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >>
> >> On 28.04.2024 18:52, Petr Bene=C5=A1 wrote:
> >>> From: Petr Bene=C5=A1 <w1benny@gmail.com>
> >>>
> >>> This change anticipates scenarios where `max_altp2m` is set to its ma=
ximum
> >>> supported value (i.e., 512), ensuring sufficient memory is allocated =
upfront
> >>> to accommodate all altp2m tables without initialization failure.
> >>
> >> And guests with fewer or even no altp2m-s still need the same bump? Yo=
u
> >> know the number of altp2m-s upon domain creation, so why bump by any m=
ore
> >> than what's strictly needed for that?
> >
> > I have to admit I've considered computing the value which goes to
> > hap_set_allocation
> > by simply adding 256 + max_altp2m, but that felt so arbitrary - the
> > 256 value itself
> > feels arbitrary, as I haven't found any reasoning for it anywhere.
> >
> > I have also tried to make code changes to make the initial allocation
> > size configurable
> > via libxl (possibly reusing the shadow_memkb) - which seemed to me
> > like the "correct"
> > solution, but those changes were more complicated than I had
> > anticipated and I would
> > definitely not make it till the 4.19 deadline.
> >
> > Question is, what to do now? Should I change it to 256 + max_altp2m?
>
> Counter question: Is accounting for just the root page table really
> enough? Meaning to say: I'm not convinced that minimum would really
> be appropriate for altp2m use even before your changes. You growing
> the number of root page tables _always_ required just makes things
> worse even without considering how (many) altp2m-s are then going
> to be used. Such an issue, if I'm right with this, would imo want
> addressing up front, in a separate patch.

It is enough - at least based on my experiments. I'll try to explain it bel=
ow.

> >> Also isn't there at least one more place where the tool stack (libxl I
> >> think) would need changing, where Dom0 ballooning needs are calculated=
?
> >> And/or doesn't the pool size have a default calculation in the tool
> >> stack, too?
> >
> > I have found places in libxl where the mempool_size is calculated, but
> > that mempool
> > size is then set AFTER the domain is created via xc_set_paging_mempool_=
size.
> >
> > In my opinion it doesn't necessarily require change, since it's
> > expected by the user
> > to manually set it via shadow_memkb. The only current problem is (which=
 this
> > commit is trying to fix) that setting shadow_memkb doesn't help when
> > max_altp2m > (256 - 1 + vcpus + MAX_NESTEDP2M), since the initial mempo=
ol
> > size is hardcoded.
>
> Wait - are you saying the guest config value isn't respected in certain
> cases? That would be another thing wanting to be fixed separately, up
> front.

The xc_set_paging_mempool_size is still called within domain_create.
So the value of shadow_memkb is respected before any of the guest code
is run. My point was that shadow_memkb isn't respected here:

>>> --- a/xen/arch/x86/mm/hap/hap.c
>>> +++ b/xen/arch/x86/mm/hap/hap.c
>>> @@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)
>>>      if ( old_pages =3D=3D 0 )
>>>      {
>>>          paging_lock(d);
>>> -        rv =3D hap_set_allocation(d, 256, NULL);
>>> +        rv =3D hap_set_allocation(d, 1024, NULL);

This code (+ the root altp2ms allocation) is executed before the libxl
sends the shadow_memkb.

In another words, the sequence is following:

libxl:
------

do_domain_create
    initiate_domain_create
        libxl__domain_make
            xc_domain_create // MAX_ALTP2M is passed here
                             // and hap_enable is called

        dcs->bl.callback =3D domcreate_bootloader_done

domcreate_bootloader_done
    libxl__domain_build
        libxl__build_pre
            libxl__arch_domain_create
                libxl__domain_set_paging_mempool_size
                    xc_set_paging_mempool_size(shadow_mem)

xen (xc_domain_create cont.):
-----------------------------
domain_create
    arch_domain_create
        hvm_domain_initialise
            paging_enable
                hap_enable
                    // note that we shadow_mem (from config) hasn't been
                    // provided yet
                    hap_set_allocation(d, 1024, NULL);
                    p2m_alloc_table(p2m_get_hostp2m(d));
                    p2m_alloc_table(d->arch.nested_p2m[i..MAX_NESTEDP2M]);
                    p2m_alloc_table(d->arch.altp2m_p2m[i..MAX_ALTP2M]);

(I hope the email will preserve the spacing...)

Based on this, I would argue that shadow_memkb should be also part of
xc_domain_create/xen_domctl_createdomain. Which is why I've said in
previous email:

> > I have also tried to make code changes to make the initial allocation
> > size configurable
> > via libxl (possibly reusing the shadow_memkb) - which seemed to me
> > like the "correct"
> > solution, but those changes were more complicated than I had
> > anticipated and I would
> > definitely not make it till the 4.19 deadline.

P.


From xen-devel-bounces@lists.xenproject.org Thu May 02 12:08:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 12:08:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715780.1117671 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VEs-0006Tq-5A; Thu, 02 May 2024 12:08:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715780.1117671; Thu, 02 May 2024 12:08:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VEs-0006Tj-1H; Thu, 02 May 2024 12:08:38 +0000
Received: by outflank-mailman (input) for mailman id 715780;
 Thu, 02 May 2024 12:08:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2VEq-0006Td-Nz
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 12:08:36 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b2c7d51f-087c-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 14:08:34 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41dc9f98e8dso4647485e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 05:08:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l3-20020a05600c4f0300b0041b43d2d745sm1753119wmq.7.2024.05.02.05.08.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 05:08:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2c7d51f-087c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714651714; x=1715256514; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zjTR5TfN761Kp62caqIjIcDClfoM4hNzhQzrtOmSxuA=;
        b=PwegUNA2y+0o0lqQV5xjuUjBEtTLuZeCHcb9pfQxgcBqCLGIq8cMsDzDO1WWOJAO9R
         N8DNkq/BrmcAGhjDX1DcnteEtJjoQOMnJcrojZd66RNdcqCsrCMbr8KH2OSLaOeJqNmw
         nQrqVoIxpMWd49FTb1MrwNtRKP6p2vtVYXPfKixJEuV5QxO8lN8RCNXkT9QVXjgaXgfk
         20q1uDZKjGggW+BZpSQBxaOHsS2hz2BhRgK/O1SspZqJlJU8tg7Xf06vfzqu4hqNSkN9
         AMnt2+XAkFZGuvEPnddTeA3PyA/Gmz+4ngTmDwiKTl/3mxhWULfNaqeiAD2rDhuiIlM9
         gSpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714651714; x=1715256514;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zjTR5TfN761Kp62caqIjIcDClfoM4hNzhQzrtOmSxuA=;
        b=o0aeIUhJjW6L9g2hA8ClY470NCyjEcG1AXtBJKGXnocJd37zQ5NelnOcdB4Bbce4gN
         VbbYtmo0irqaBSf8RAbjRTD26+Sqm2uJrdBmL+qph4OCNKbCjBziEtUZZcK2RsDLB92o
         486odUsUcOZa1lQYUV1pnheYIPLPA3jGApnIWLWk33NGidnYswZgM/zqbmxBS0DbeQRO
         G5A83hwQOaEdnd9jjRnOVlR99ShzqOcWV6hKaZQZKOjr3q0td/u79xXhdDthM2+vAGWT
         9wPLRhHkPcsWJXNaYnhrZE0yHDYV7k/iowQXCIUC2MX68Wki8zfauYuL1llBMO9T0zIO
         Vkbw==
X-Forwarded-Encrypted: i=1; AJvYcCVByT+qhmdxWHHSDXH/Bx/H0DRP07or0l3o4DwLWT2e8WSPU8qvsuYmEbCyyYaGuQiGiplCsnOy9rB47QFsJ3yLS63Cc/DWQI4atO79gwU=
X-Gm-Message-State: AOJu0Yzstf0IHQ+wnxA6Wv/zAzwIN5L/haflJUO+IcSPt6E6ubOcb7Wo
	EfhvmXuIJfXlY/RY4AThPgUJfQUDxBuP/vG/S9KMIgVJndhqmu7+eMP4g+llqg==
X-Google-Smtp-Source: AGHT+IFKbwb3ty5b9VYfHFiyV5qEil3AmHHO9dSZekpjWiISW/7ZW5MYY5fNgvY5IfJZzvo+78A/Ig==
X-Received: by 2002:a05:600c:1c83:b0:41a:4623:7ee9 with SMTP id k3-20020a05600c1c8300b0041a46237ee9mr2069310wms.10.1714651714076;
        Thu, 02 May 2024 05:08:34 -0700 (PDT)
Message-ID: <1c4d8a69-9711-4f87-870b-38ec0f62aec1@suse.com>
Date: Thu, 2 May 2024 14:08:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] x86/hvm: Defer the size calculation in
 hvm_save_cpu_xsave_states()
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-2-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240429182823.1130436-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.04.2024 20:28, Andrew Cooper wrote:
> HVM_CPU_XSAVE_SIZE() may rewrite %xcr0 twice.  Defer the caluclation it until
> after we've decided to write out an XSAVE record.
> 
> Note in hvm_load_cpu_xsave_states() that there were versions of Xen which
> wrote out a useless XSAVE record.  This sadly limits out ability to tidy up
> the existing infrastructure.  Also leave a note in xstate_ctxt_size() that 0
> still needs tolerating for now.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with a nit that there are a few spelling/grammar issues in the text above.

Jan



From xen-devel-bounces@lists.xenproject.org Thu May 02 12:14:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 12:14:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715784.1117680 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VKj-0008Lh-NP; Thu, 02 May 2024 12:14:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715784.1117680; Thu, 02 May 2024 12:14:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VKj-0008La-Kj; Thu, 02 May 2024 12:14:41 +0000
Received: by outflank-mailman (input) for mailman id 715784;
 Thu, 02 May 2024 12:14:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2VKi-0008LU-7H
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 12:14:40 +0000
Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com
 [2607:f8b0:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8bdaf03c-087d-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 14:14:39 +0200 (CEST)
Received: by mail-oi1-x231.google.com with SMTP id
 5614622812f47-3c6f785208dso1491999b6e.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 05:14:39 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ew9-20020a05622a514900b0043a7f359414sm419310qtb.19.2024.05.02.05.14.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 05:14:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bdaf03c-087d-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714652077; x=1715256877; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=P6UgqazcQrB/QSFU5qYZhBirP/WJ5vy9WnvmGqckGTA=;
        b=PaSfcmMSIbfF6mo85yWHQIs1kvj1xtPNZBpcwUYh47gw6oaAX9FSMtkDPT327q6sBm
         AV6saTQR0uCD0bs0+9630d7gG001yH6/JtLZs1Et3WkkHitS2C/F3j+8Das8nVfjZlIy
         RGs3RLVMBnzhsot8glnQqWL9zroN4ry0EDEhQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714652077; x=1715256877;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=P6UgqazcQrB/QSFU5qYZhBirP/WJ5vy9WnvmGqckGTA=;
        b=GtUSTZeqZqoc8FQl64gfc0FMQRco/o9APb2lOcgfHqEp2IIIDCcz+lTNeRRBdTxpof
         ny+zzK3XnWWWoCfQ1Q/h88ljhozVvRw9GyuokDlqpFS7s8myBmXjjjcyDalssifTedhd
         MBfqtmp00aWL3RZxYsbkv2BDRPEuLyIA6uz7RAYZao5G3/ttvBQF9KuuX6t/fw2b8Nr6
         /kVC09M6O9b6T/Fqfm+cvP/SbyV0ksATqAX4KmKh9JuZvA9zriPHFOFtUMBm3LezS3OQ
         WObZUX9d4uW14DGrGvH8eOHI8AD2DNqJ6L+YWuQkC5QLv7E8WM2IADaM9th8BPEAE4Cr
         +wmg==
X-Forwarded-Encrypted: i=1; AJvYcCVtB/MY5d7DDRWjeeyk2e6URMCetk7EwXbXyCoNFSuqUGEaSi0qC1dZv7n8j1P7PRytiT6C6q7bbgv9Qx3Pom3uB6v/UR64BcnAz5cAW88=
X-Gm-Message-State: AOJu0YzvaQfHvilBc0zEDcQ3L1fmIlRPgKqo255f8AQPOh1v7HpQPhU6
	54D3WxbKigtB1H+NyYn8gx4KF3NTB9XeFIOmEY28x8ll/8CKVW0eO+vqTVujCfo=
X-Google-Smtp-Source: AGHT+IFyGsBkiapu+KmVh9ucflrVZkDv11+1yyhhhZmBtAZhIkrdgxHcuOiJxc75+m7dCZq2rRqVRw==
X-Received: by 2002:a05:6808:f04:b0:3c7:2f77:a102 with SMTP id m4-20020a0568080f0400b003c72f77a102mr7595012oiw.26.1714652077676;
        Thu, 02 May 2024 05:14:37 -0700 (PDT)
Message-ID: <ee407b03-a1aa-4fa0-a700-71278f643489@citrix.com>
Date: Thu, 2 May 2024 13:14:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] tools/xen-cpuid: switch to use cpu-policy
 defined names
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>, xen-devel@lists.xenproject.org,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240430082901.77662-1-roger.pau@citrix.com>
 <78d009f4-6610-4fcf-9cdc-ee1081e6c763@suse.com> <ZjDVN6kexFD0vcT2@macbook>
 <f1e594f7-2bf2-4898-824f-abd407690644@suse.com> <ZjDqHfdjUE1CTk1W@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZjDqHfdjUE1CTk1W@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/04/2024 1:54 pm, Roger Pau Monné wrote:
> On Tue, Apr 30, 2024 at 02:06:38PM +0200, Jan Beulich wrote:
>> On 30.04.2024 13:25, Roger Pau Monné wrote:
>>> On Tue, Apr 30, 2024 at 12:37:44PM +0200, Jan Beulich wrote:
>>>> On 30.04.2024 10:29, Roger Pau Monne wrote:
>>>>> @@ -301,21 +52,32 @@ static const char *const fs_names[] = {
>>>>>      [XEN_SYSCTL_cpu_featureset_hvm_max] = "HVM Max",
>>>>>  };
>>>>>  
>>>>> -static void dump_leaf(uint32_t leaf, const char *const *strs)
>>>>> +static const char *find_name(unsigned int index)
>>>>>  {
>>>>> -    unsigned i;
>>>>> +    static const struct feature_name {
>>>>> +        const char *name;
>>>>> +        unsigned int bit;
>>>>> +    } feature_names[] = INIT_FEATURE_NAMES;
>>>>> +    unsigned int i;
>>>>>  
>>>>> -    if ( !strs )
>>>>> -    {
>>>>> -        printf(" ???");
>>>>> -        return;
>>>>> -    }
>>>>> +    for ( i = 0; i < ARRAY_SIZE(feature_names); i++ )
>>>>> +        if ( feature_names[i].bit == index )
>>>>> +            return feature_names[i].name;
>>>> ... a linear search, repeated perhaps hundreds of times, looks still a
>>>> little odd to me.
>>> I didn't benchmark what kind of performance impact this change would
>>> have on the tool, but I didn't think it was that relevant, as this is
>>> a diagnostic/debug tool, and hence performance (unless it took seconds
>>> to execute) shouldn't be that important.
>> As indicated, performance itself isn't much of a concern here. My earlier
>> question wants reading in relation to the other question raised, regarding
>> the script maybe wanting to produce macro(s) more suitable for the purpose
>> here.
> Hm, we could maybe produce an array of strings, one per feature bit
> (features without names would get NULL).
>
> I will see, albeit my python skills are very limited.

See how Xen's cmdline parsing uses feature_names; they're intentionally
sorted already.

But, having gen-cpuid.py write out something that's closer to what
xen-cpuid.c wants is also very easy to do.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 12:19:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 12:19:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715791.1117691 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VPU-0000VU-7V; Thu, 02 May 2024 12:19:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715791.1117691; Thu, 02 May 2024 12:19:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VPU-0000VN-3d; Thu, 02 May 2024 12:19:36 +0000
Received: by outflank-mailman (input) for mailman id 715791;
 Thu, 02 May 2024 12:19:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2VPS-0000VG-Sl
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 12:19:34 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3b6a523f-087e-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 14:19:32 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41adf3580dbso10563065e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 05:19:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l3-20020a05600c4f0300b0041b43d2d745sm1784794wmq.7.2024.05.02.05.19.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 05:19:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b6a523f-087e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714652372; x=1715257172; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fbTwUBrPlrMq/nr/N7RudbiJWugel33p8MkNTTCKAAs=;
        b=O9EprtJnl9kC+mrfeQJo57rAz/9l9+tsg134nd9Hqf5FGPy4D2yESPG5XTJbRFjFC+
         Zhm+YlGUxGrFkKpUrE03D9m3JIMNlaryB98xFBQqbaypFG1xgBXjzVXaJnuygJnm1Es6
         R+OyXkDvHNClkP+1+2lHw/quYvQQKTa/IQBhBca/Ri1kUqEhD8B4KeRkTF8odegk8Thw
         QUNXdYzxNYuy5myDhBCwf5d0n5dmk0D5awpCUcH/s+YGxFG5ku9SM7MMAIFfV/tXGEdO
         KTjQGIoCedQ9AJDIEB7uO/PP6cwk0ZBKtCcBAB76qtO8XGZPr36bIarTippRqCl/OB+D
         U6ZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714652372; x=1715257172;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=fbTwUBrPlrMq/nr/N7RudbiJWugel33p8MkNTTCKAAs=;
        b=EtTBRqDORJBmUbbTyQzLLJYxSZnBn5YG4SQvbeaE9m/1QphoqPweyq///JTIdqeV1M
         3VVfHhaYSw0VUGW9WvnMHppO3uKu2qKdHxLswxJJsj9wvt4ipHwuuWbkj/1hshoWliAs
         3P2duA2nGnKFxEosvOS6lFPtg8U98WBzDETYa4BMbEBg1J8lEaGBHXUneuZ5uzCseiKI
         XfyBjwslpK+i+nDq5IGdr7fuUgpq2LFy/oxaUGSJKzJUeoySvLOZ6JqF/AYAN3FdH1Or
         WdV95Aftjuch/A1c3iu/hDWtLkwJ69qgy3U60XGSMFobm3jKQRM1rCOC9yRpm+RKhUI7
         D0bQ==
X-Forwarded-Encrypted: i=1; AJvYcCVthqD6Xaug8VxLlvUvDZbD+quv4WxcrInS+iC1IRpaxxkTixaC8KHY4gnJEj2fK4eFh1Ym6SJj0NKazGlsCQ9kRvuSCK5KFYk9P7mJDMU=
X-Gm-Message-State: AOJu0YxzJVFrWjec4R1/Kea96PCyu+/EAiWV/9XEVNkk633v32xZrajG
	kz+7J14nhvHSsESbYD4Fw/4761jopjoMtqAWP1V7Yz1Q1srk11/9L+EQet1PUw==
X-Google-Smtp-Source: AGHT+IESY9Io1fGavdZkQauA6sRbcTO29ju++YgwNmc8oPtSjv2mhq9nyxhxWPueG1MlksYd1YgKOA==
X-Received: by 2002:a05:600c:198a:b0:41c:120d:d3d6 with SMTP id t10-20020a05600c198a00b0041c120dd3d6mr2139460wmq.1.1714652372324;
        Thu, 02 May 2024 05:19:32 -0700 (PDT)
Message-ID: <741d4bef-8711-4802-91ad-8a6b4459da61@suse.com>
Date: Thu, 2 May 2024 14:19:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] x86/xstate: Rework xstate_ctxt_size() as
 xstate_uncompressed_size()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240429182823.1130436-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.04.2024 20:28, Andrew Cooper wrote:
> @@ -567,16 +567,51 @@ static unsigned int hw_uncompressed_size(uint64_t xcr0)
>      return size;
>  }
>  
> -/* Fastpath for common xstate size requests, avoiding reloads of xcr0. */
> -unsigned int xstate_ctxt_size(u64 xcr0)
> +unsigned int xstate_uncompressed_size(uint64_t xcr0)
>  {
> +    unsigned int size = XSTATE_AREA_MIN_SIZE, i;
> +
>      if ( xcr0 == xfeature_mask )
>          return xsave_cntxt_size;
>  
>      if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */
>          return 0;
>  
> -    return hw_uncompressed_size(xcr0);
> +    if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) )

This is open-coded XSTATE_FP_SSE, which I wouldn't mind if ...

> +        return size;
> +
> +    /*
> +     * For the non-legacy states, search all activate states and find the
> +     * maximum offset+size.  Some states (e.g. LWP, APX_F) are out-of-order
> +     * with respect their index.
> +     */
> +    xcr0 &= ~XSTATE_FP_SSE;

... you didn't use that macro here (and once further down). IOW please
be consistent, no matter which way round.

> +    for_each_set_bit ( i, &xcr0, 63 )
> +    {
> +        unsigned int s;
> +
> +        ASSERT(xstate_offsets[i] && xstate_sizes[i]);
> +
> +        s = xstate_offsets[i] && xstate_sizes[i];

You mean + here, don't you? Then:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

I'm also inclined to suggest making this the initializer of s.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 12:20:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 12:20:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715795.1117700 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VQO-0001ro-Ej; Thu, 02 May 2024 12:20:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715795.1117700; Thu, 02 May 2024 12:20:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2VQO-0001rh-CD; Thu, 02 May 2024 12:20:32 +0000
Received: by outflank-mailman (input) for mailman id 715795;
 Thu, 02 May 2024 12:20:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2VQN-0000VG-Vd
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 12:20:31 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5da8e27a-087e-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 14:20:30 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41b4ff362a8so73283365e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 05:20:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l3-20020a05600c4f0300b0041b43d2d745sm1784794wmq.7.2024.05.02.05.20.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 05:20:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5da8e27a-087e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714652430; x=1715257230; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3TQCesLjOF/25eiD8fMMvT518zpci2PUnWHdF5a0guQ=;
        b=N8WKL00ZepChYoE16lUA83OAFoAUJsMQDfijOJQQK29XLM6jW4FHHGRx4ijuKnuew0
         Wl3iL+Mb0xHfjLEcoPBkPvblSpAvUrb+z7O3ZiluYAj1YBFWRs8vvc2rTwbxlfDHKvbs
         JCx3LVd/BwXjEnQvXbMW9th1KFvzEH8eO8SiYhkdyWyGodF4kTi4Fw4YS6N5jKdl9Kst
         fn6tAVsHLs/CdwMMUhTMHAcz9hA8FVTad4u4EGY14jkBBd07LzXUVn0XEzJ/91vuSEWy
         u4I85Cod88jojdGUMu+ZHK7h19wkadGIFNMk5nPkH7ssoOaOnu1vVxq1RG2N29AjAJwb
         Ymjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714652430; x=1715257230;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3TQCesLjOF/25eiD8fMMvT518zpci2PUnWHdF5a0guQ=;
        b=DkC5P88NPXigrmfqrmqI0n6uI5wB5s4g24RSv3HPvzwlCmPwOyfT5tCFkR8ApoIsXG
         IkPDls3tlkxqtHP2WlTKRevfTbkK180MX8TrzWLO/xFvSvSk1e6CHe/BWNdfgqNB39+a
         OXHsylOKwYCQJusP5kBjylxjU+2e+EtIUEVz2j2gMPkPTu8yTUBLJHqDJ4Z3QaZPVi+E
         OKFntQvpUIhQ663xdLKilnLKvhv8jk1WFcpEyyDT6oe9zWQGjxHeA54XBGeAGMjFWmsQ
         flfvIvPoE6R3wrAGvtbzToNwKuh9c/cPKL4q8x1AOaYN9cwOFYN3kOYvgDfhakbv2rHq
         TkzA==
X-Forwarded-Encrypted: i=1; AJvYcCVKLyLBF7ayNYriIrxxH+C2ismDAXyg0f3pixL56Wju/1/Ksgd2eOaDkZMx49hIZBJGjBgiySu544EmobLMR7zglBggPTwQY3Wn/LUI8lo=
X-Gm-Message-State: AOJu0YzGTuSc9fykXjtoVh+aFb48qlctxBZ6atbp9cMjTJGvFTZFUx31
	9GmZ+dQTFFmQ59tSKx606jwJgvpG4YEOZfAZjagkeiNiHxy9VxMpQeyYpjmgCg==
X-Google-Smtp-Source: AGHT+IGA9qvl10RoUcA+Ygs/mtcTcvRzaaaq+XX9uBwvkHrAOZhxsurHvxSeiuM1ZDXts4HbuvlUHg==
X-Received: by 2002:a05:600c:4587:b0:41b:d85c:d3e2 with SMTP id r7-20020a05600c458700b0041bd85cd3e2mr5188670wmo.38.1714652429809;
        Thu, 02 May 2024 05:20:29 -0700 (PDT)
Message-ID: <76089076-e041-4bcd-921d-c7a8f0c417e3@suse.com>
Date: Thu, 2 May 2024 14:20:28 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] tools/xen-cpuid: switch to use cpu-policy
 defined names
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>, xen-devel@lists.xenproject.org,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240430082901.77662-1-roger.pau@citrix.com>
 <78d009f4-6610-4fcf-9cdc-ee1081e6c763@suse.com> <ZjDVN6kexFD0vcT2@macbook>
 <f1e594f7-2bf2-4898-824f-abd407690644@suse.com> <ZjDqHfdjUE1CTk1W@macbook>
 <ee407b03-a1aa-4fa0-a700-71278f643489@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ee407b03-a1aa-4fa0-a700-71278f643489@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 14:14, Andrew Cooper wrote:
> On 30/04/2024 1:54 pm, Roger Pau Monné wrote:
>> On Tue, Apr 30, 2024 at 02:06:38PM +0200, Jan Beulich wrote:
>>> On 30.04.2024 13:25, Roger Pau Monné wrote:
>>>> On Tue, Apr 30, 2024 at 12:37:44PM +0200, Jan Beulich wrote:
>>>>> On 30.04.2024 10:29, Roger Pau Monne wrote:
>>>>>> @@ -301,21 +52,32 @@ static const char *const fs_names[] = {
>>>>>>      [XEN_SYSCTL_cpu_featureset_hvm_max] = "HVM Max",
>>>>>>  };
>>>>>>  
>>>>>> -static void dump_leaf(uint32_t leaf, const char *const *strs)
>>>>>> +static const char *find_name(unsigned int index)
>>>>>>  {
>>>>>> -    unsigned i;
>>>>>> +    static const struct feature_name {
>>>>>> +        const char *name;
>>>>>> +        unsigned int bit;
>>>>>> +    } feature_names[] = INIT_FEATURE_NAMES;
>>>>>> +    unsigned int i;
>>>>>>  
>>>>>> -    if ( !strs )
>>>>>> -    {
>>>>>> -        printf(" ???");
>>>>>> -        return;
>>>>>> -    }
>>>>>> +    for ( i = 0; i < ARRAY_SIZE(feature_names); i++ )
>>>>>> +        if ( feature_names[i].bit == index )
>>>>>> +            return feature_names[i].name;
>>>>> ... a linear search, repeated perhaps hundreds of times, looks still a
>>>>> little odd to me.
>>>> I didn't benchmark what kind of performance impact this change would
>>>> have on the tool, but I didn't think it was that relevant, as this is
>>>> a diagnostic/debug tool, and hence performance (unless it took seconds
>>>> to execute) shouldn't be that important.
>>> As indicated, performance itself isn't much of a concern here. My earlier
>>> question wants reading in relation to the other question raised, regarding
>>> the script maybe wanting to produce macro(s) more suitable for the purpose
>>> here.
>> Hm, we could maybe produce an array of strings, one per feature bit
>> (features without names would get NULL).
>>
>> I will see, albeit my python skills are very limited.
> 
> See how Xen's cmdline parsing uses feature_names; they're intentionally
> sorted already.

Sorted by name, when here we'd like to have it sorted by feature index.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 12:39:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 12:39:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715803.1117711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ViY-0004fY-1l; Thu, 02 May 2024 12:39:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715803.1117711; Thu, 02 May 2024 12:39:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ViX-0004fR-VQ; Thu, 02 May 2024 12:39:17 +0000
Received: by outflank-mailman (input) for mailman id 715803;
 Thu, 02 May 2024 12:39:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2ViX-0004fL-2b
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 12:39:17 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc73f02e-0880-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 14:39:15 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41dc9f98e8dso4857215e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 05:39:15 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o27-20020a05600c511b00b00418a386c17bsm5568772wms.12.2024.05.02.05.39.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 05:39:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc73f02e-0880-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714653555; x=1715258355; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/awNH/fDe7E4Ox9uD3aF+dalF3thUNmdxzqGEtHvNrc=;
        b=EpDSjJQcPbzIPLuhsikygILj7BC/wrKwLiBUM1pHDE+PlvfTX2azVWUjZbMoBZmzdn
         a2KDWq2wLYkf1RVdw+Usy1PXIdsht96qGtoiYD9l2PE4I1DY4vJ9SYkZiUOqXmDtZzhj
         kSGLd/4XDIaREz5cKrEFj8MHMkf3rmFQ4xSURz7RV+v/XkrmMI21QFVfrH9oPvsETa5D
         PtfhyioAw434aIr+O4ykM1tDlgsQ7QVUj9wQ+yahpG0kItV/y9eIUZXPMeqpDg9wYMtg
         4eFNjgwMWPUPBRhv5cQsa6yWJ38cPVg9c48jYT/daQuXOauKNU5Eqz225WaGPyPgwGkx
         6yCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714653555; x=1715258355;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/awNH/fDe7E4Ox9uD3aF+dalF3thUNmdxzqGEtHvNrc=;
        b=tzf4ruZEYMa2mfCxehaQAJ4LyXHBmoEoj8zpujyA230wKmetyArv2t1f3EFWhB5nUG
         r/sRBKpIbv4/xCeeuHSSj71uWXEr/qTwpFSj/K03SI2MvjX4yQ1hwXl1C50lgZ/znz8J
         uDZBzqHYNJEQSlF/Y5QKSLXeIyY+CivdtpccUqjdb64GV9FeKr7cm9WgmRsARYDCCC1W
         jt0KEyaCXpULoeg00A28B3/Fy33O70sUrLl2U1RxlqYOZUPEFCx7oFO/0RqZBB6Lhcxg
         AXTxZgvDMgRvNkB96ly5HFVDaTba7hYLbfhZjJ2V1VmOiYnQxrT0GqmKK27uUAuJvu4j
         vZqw==
X-Forwarded-Encrypted: i=1; AJvYcCW9+qqxNgLWdWwEnabPqiASVuwW4jM5WXUPCeEwq9q9CdU/MUMMbimeJwDnC41exBltq7yqMpLjWRH0FGngMo8BaJDg6AJEcP92C83fFdg=
X-Gm-Message-State: AOJu0YwY9hnoqTaTV1ik4OpPnXVTwgLbScZK6kVT68w4bPE89WTKmx8R
	OXaV4ly5ItPDUG88PbbcBeGmdSMf/1i7qf8qu5052o2OONZj92LlsaySbs+OxQ==
X-Google-Smtp-Source: AGHT+IGzYpgj+judUzZT+0x1OBwP2DKqGboOgTzGkFWPqLln42VByZYO8YpS+c5Z+hDKqEZlsSSclw==
X-Received: by 2002:a05:6000:188:b0:34d:c4c7:7ca with SMTP id p8-20020a056000018800b0034dc4c707camr2283014wrx.20.1714653555139;
        Thu, 02 May 2024 05:39:15 -0700 (PDT)
Message-ID: <ecd6e142-7a08-46e8-83c0-7d9b0bc6f66f@suse.com>
Date: Thu, 2 May 2024 14:39:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] x86/cpu-policy: Simplify recalculate_xstate()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-4-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240429182823.1130436-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.04.2024 20:28, Andrew Cooper wrote:
> Make use of the new xstate_uncompressed_size() helper rather than maintaining
> the running calculation while accumulating feature components.

xstate_uncompressed_size() isn't really a new function, but the re-work of
an earlier one. That, aiui, could have been used here, too, just that it
would have been inefficient to do so. IOW perhaps drop "the new"?

> The rest of the CPUID data can come direct from the raw cpu policy.  All
> per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}*
> instructions.
> 
> Use for_each_set_bit() rather than opencoding a slightly awkward version of
> it.  Mask the attributes in ecx down based on the visible features.  This
> isn't actually necessary for any components or attributes defined at the time
> of writing (up to AMX), but is added out of an abundance of caution.

As to this, ...

> @@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p)
>          return;
>  
>      if ( p->basic.avx )
> -    {
>          xstates |= X86_XCR0_YMM;
> -        xstate_size = max(xstate_size,
> -                          xstate_offsets[X86_XCR0_YMM_POS] +
> -                          xstate_sizes[X86_XCR0_YMM_POS]);
> -    }
>  
>      if ( p->feat.mpx )
> -    {
>          xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR;
> -        xstate_size = max(xstate_size,
> -                          xstate_offsets[X86_XCR0_BNDCSR_POS] +
> -                          xstate_sizes[X86_XCR0_BNDCSR_POS]);
> -    }
>  
>      if ( p->feat.avx512f )
> -    {
>          xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM;
> -        xstate_size = max(xstate_size,
> -                          xstate_offsets[X86_XCR0_HI_ZMM_POS] +
> -                          xstate_sizes[X86_XCR0_HI_ZMM_POS]);
> -    }
>  
>      if ( p->feat.pku )
> -    {
>          xstates |= X86_XCR0_PKRU;
> -        xstate_size = max(xstate_size,
> -                          xstate_offsets[X86_XCR0_PKRU_POS] +
> -                          xstate_sizes[X86_XCR0_PKRU_POS]);
> -    }
>  
> -    p->xstate.max_size  =  xstate_size;
> +    /* Subleaf 0 */
> +    p->xstate.max_size =
> +        xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY);
>      p->xstate.xcr0_low  =  xstates & ~XSTATE_XSAVES_ONLY;
>      p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32;
>  
> +    /* Subleaf 1 */
>      p->xstate.Da1 = Da1;
> +    if ( p->xstate.xsavec )
> +        ecx_mask |= XSTATE_ALIGN64;
> +
>      if ( p->xstate.xsaves )
>      {
> +        ecx_mask |= XSTATE_XSS;
>          p->xstate.xss_low   =  xstates & XSTATE_XSAVES_ONLY;
>          p->xstate.xss_high  = (xstates & XSTATE_XSAVES_ONLY) >> 32;
>      }
> -    else
> -        xstates &= ~XSTATE_XSAVES_ONLY;
>  
> -    for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i )
> +    /* Subleafs 2+ */
> +    xstates &= ~XSTATE_FP_SSE;
> +    BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63);
> +    for_each_set_bit ( i, &xstates, 63 )
>      {
> -        uint64_t curr_xstate = 1UL << i;
> -
> -        if ( !(xstates & curr_xstate) )
> -            continue;
> -
> -        p->xstate.comp[i].size   = xstate_sizes[i];
> -        p->xstate.comp[i].offset = xstate_offsets[i];
> -        p->xstate.comp[i].xss    = curr_xstate & XSTATE_XSAVES_ONLY;
> -        p->xstate.comp[i].align  = curr_xstate & xstate_align;

... for this bit, isn't the move from this ...

> +        /*
> +         * Pass through size (eax) and offset (ebx) directly.  Visbility of
> +         * attributes in ecx limited by visible features in Da1.
> +         */
> +        p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a;
> +        p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b;
> +        p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask;

... to this changing what guests get to see, i.e. (mildly?) incompatible?
(For .xss there's no issue because XSTATE_XSAVES_ONLY is still 0.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 13:04:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:04:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715810.1117721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2W7B-00013H-0a; Thu, 02 May 2024 13:04:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715810.1117721; Thu, 02 May 2024 13:04:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2W7A-00013A-SV; Thu, 02 May 2024 13:04:44 +0000
Received: by outflank-mailman (input) for mailman id 715810;
 Thu, 02 May 2024 13:04:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2W79-000134-Qz
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:04:43 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8931e3a4-0884-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 15:04:41 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-346359c8785so6366393f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 06:04:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m3-20020adffa03000000b0034db974d7a1sm1219207wrr.81.2024.05.02.06.04.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 06:04:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8931e3a4-0884-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714655080; x=1715259880; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MjzMlQ+UB0mzimV8PSmOXAwTjeKig6Rkw1588EBUfaM=;
        b=NuWQ5L+Pjl3SCnrzP+50bVDvp1C2nnGgezNDGyTOPU3KeQQEENNl71I6u9jtYi+RkV
         v1qVa8xxneACWhEfUKpzaD0RgauvJxOElI3/xg7rrxyFzsDNY8cHHbeCqkWQUIRXleBa
         m1wej6XnmI3gFRPkSZrZnekjcc8m7NNdymwoX7FozSON4x6XAOX/xor1og9NTfDmm+XM
         XKLRp3p0y/KAxuscKdV1pwV7rREFSQn7XbCOD33KiEUWzbUttsOz67hBGbdDPpKmOHXD
         YM/arRH7CqsV36RjTBhNrtP1P2vDCrNwTEceMM6dzLOTYQ1ebB7d/N/XS5Ur03A1Sl+r
         PZpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714655080; x=1715259880;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MjzMlQ+UB0mzimV8PSmOXAwTjeKig6Rkw1588EBUfaM=;
        b=BYEpNjtityaitV3/NeNM76Ia1iXFKo3b8okIisfxd4DNlPlww18lnBKg6WVT8S35CK
         b94GHF8r+Z98QWKdzbwNJO1vwmMxujGG6PpSqafRe3YZB8+QNfOUEMhu4hQ5KaeG9mY/
         sBbgE7R5wdF5SJeD1ufe2QIxzU2866oLUcwhyNd0gAiL5c0k8/znDBpdE/zTouo0Nbfr
         l4/zEaPFOf38HQ+PxFG1duCFpOEC0Rtj+DOUx45lBdTgyvcgM8PcBFtFSic/TcYA2xdP
         AB844YCnQsrMlqEKtfmA1re4S0NfLTxNWID9WvAegE2R3lX+i5YieXS7aP+puQ8BYDho
         yEFA==
X-Forwarded-Encrypted: i=1; AJvYcCXlQTO6ZUARDaNJOHdvc6cxnhn1o2aIz1KMA4t2PDrntSwQgEPJSH91Y8RuX1qbj6iu26PQtTF6qfuujwNG1JdJw5V8hH1ITzF7sUqEmC8=
X-Gm-Message-State: AOJu0YwQFhtOMXEA7BQz6raP3ZAiS6ny+ttoyp+yxU4zxnjLv0U7327p
	XEv5dLQcwzcxRrJs8pPVLeaVPaECCSG0cxIRWmTapyunuSlCqpySac/ynPlgAg==
X-Google-Smtp-Source: AGHT+IHw8AFL8VCNilQGKblSIl1CXF3gYHtTQGdG0DSXmS0QHATt06glj1ESCB1HIHX40h2Ij8LJfg==
X-Received: by 2002:a5d:4dc7:0:b0:34d:12c3:ffb0 with SMTP id f7-20020a5d4dc7000000b0034d12c3ffb0mr3508693wru.9.1714655079457;
        Thu, 02 May 2024 06:04:39 -0700 (PDT)
Message-ID: <d87b31be-6c52-4271-a61f-bf31043f507d@suse.com>
Date: Thu, 2 May 2024 15:04:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] x86/cpuid: Fix handling of xsave dynamic leaves
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240429182823.1130436-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.04.2024 20:28, Andrew Cooper wrote:
> If max leaf is greater than 0xd but xsave not available to the guest, then the
> current XSAVE size should not be filled in.  This is a latent bug for now as
> the guest max leaf is 0xd, but will become problematic in the future.

Why would this not be an issue when .max_leaf == 0xd, but .xsave == 0? Without
my "x86/CPUID: shrink max_{,sub}leaf fields according to actual leaf contents"
I don't think we shrink max_leaf to below 0xd when there's no xsave for the
guest?

> The comment concerning XSS state is wrong.  VT-x doesn't manage host/guest
> state automatically, but there is provision for "host only" bits to be set, so
> the implications are still accurate.
> 
> Introduce {xstate,hw}_compressed_size() helpers to mirror the uncompressed
> ones.
> 
> This in turn higlights a bug in xstate_init().  Defaulting this_cpu(xss) to ~0
> requires a forced write to clear it back out.  This in turn highlights that
> it's only a safe default on systems with XSAVES.

Well, yes, such an explicit write was expected to appear when some xsaves-
based component would actually be enabled. Much like the set_xcr0() there.

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> 
> The more I think about it, the more I think that cross-checking with hardware
> is a bad move.  It's horribly expensive, and for supervisor states in
> particular, liable to interfere with functionality.

I agree, but I'd also like to see the cross checking not dropped entirely.
Can't we arrange for such to happen during boot, before we enable any such
functionality? After all even in debug builds it's not overly useful to do
the same cross-checking (i.e. for identical inputs) over and over again.
Of course doing in an exhaustive manner may be okay for the uncompressed
values, but might be a little too much for all possible combinations in
order to check compressed values, too.

> --- a/xen/arch/x86/xstate.c
> +++ b/xen/arch/x86/xstate.c
> @@ -614,6 +614,65 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0)
>      return size;
>  }
>  
> +static unsigned int hw_compressed_size(uint64_t xstates)
> +{
> +    uint64_t curr_xcr0 = get_xcr0(), curr_xss = get_msr_xss();
> +    unsigned int size;
> +    bool ok;
> +
> +    ok = set_xcr0(xstates & ~XSTATE_XSAVES_ONLY);
> +    ASSERT(ok);
> +    set_msr_xss(xstates & XSTATE_XSAVES_ONLY);
> +
> +    size = cpuid_count_ebx(XSTATE_CPUID, 1);
> +
> +    ok = set_xcr0(curr_xcr0);
> +    ASSERT(ok);
> +    set_msr_xss(curr_xss);
> +
> +    return size;
> +}
> +
> +unsigned int xstate_compressed_size(uint64_t xstates)
> +{
> +    unsigned int i, size = XSTATE_AREA_MIN_SIZE;
> +
> +    if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */
> +        return 0;
> +
> +    if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) )

Same comment as on the earlier change regarding the (lack of) use of ....

> +        return size;
> +
> +    /*
> +     * For the compressed size, every component matters.  Some are
> +     * automatically rounded up to 64 first.
> +     */
> +    xstates &= ~XSTATE_FP_SSE;

... this constant up there.

> +    for_each_set_bit ( i, &xstates, 63 )
> +    {
> +        if ( test_bit(i, &xstate_align) )
> +            size = ROUNDUP(size, 64);
> +
> +        size += xstate_sizes[i];
> +    }

The comment is a little misleading: As you have it in code, it is not the
component's size that is rounded up, but its position. Maybe "Some have
their start automatically rounded up to 64 first"?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 13:11:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:11:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715815.1117731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WDp-000302-Mf; Thu, 02 May 2024 13:11:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715815.1117731; Thu, 02 May 2024 13:11:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WDp-0002zv-IJ; Thu, 02 May 2024 13:11:37 +0000
Received: by outflank-mailman (input) for mailman id 715815;
 Thu, 02 May 2024 13:11:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QyT6=MF=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2WDo-0002zp-Hw
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:11:36 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7fbbcf6f-0885-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 15:11:34 +0200 (CEST)
Received: from truciolo.homenet.telecomitalia.it
 (host-79-36-52-167.retail.telecomitalia.it [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 362954EE0738;
 Thu,  2 May 2024 15:11:24 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fbbcf6f-0885-11ef-b4bb-af5377834399
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Subject: [XEN PATCH v2] automation/eclair: add deviation of MISRA C:2012 Rule 14.4
Date: Thu,  2 May 2024 15:11:15 +0200
Message-Id: <ce16233b539f4101d97b3e2962607c87b188c415.1714655082.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to take into account the deviations
agreed during MISRA meetings.

Amend an existing entry of Rule 14.4 in deviations-rst:
it is not a project-wide deviation.

Tag Rule 14.4 as clean for arm.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
Changes in v2:
- reordered deviations to have the more general one as first;
- tagged the rule as clean for arm.
---
 automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++++
 automation/eclair_analysis/ECLAIR/tagging.ecl    | 2 +-
 docs/misra/deviations.rst                        | 8 +++++++-
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index d21f112a9b..db1fae3aa2 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -343,6 +343,10 @@ statements are deliberate"
 -config=MC3R1.R14.3,statements={deliberate , "wrapped(any(),node(if_stmt))" }
 -doc_end
 
+-doc_begin="A controlling expression of 'if' and iteration statements having integer, character or pointer type has a semantics that is well-known to all Xen developers."
+-config=MC3R1.R14.4,etypes+={deliberate, "any()", "src_type(integer||character)||src_expr(type(desugar(pointer(any()))))"}
+-doc_end
+
 -doc_begin="The XEN team relies on the fact that the enum is_dying has the
 constant with assigned value 0 act as false and the other ones as true,
 therefore have the same behavior of a boolean"
diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index bdf94ed996..f573b32da4 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -29,7 +29,7 @@ if(string_equal(target,"x86_64"),
 )
 
 if(string_equal(target,"arm64"),
-    service_selector({"additional_clean_guidelines","MC3R1.R16.6||MC3R1.R20.12||MC3R1.R2.1||MC3R1.R5.3||MC3R1.R7.2||MC3R1.R7.3||MC3R1.R8.6||MC3R1.R9.3"})
+    service_selector({"additional_clean_guidelines","MC3R1.R14.4||MC3R1.R16.6||MC3R1.R20.12||MC3R1.R2.1||MC3R1.R5.3||MC3R1.R7.2||MC3R1.R7.3||MC3R1.R8.6||MC3R1.R9.3"})
 )
 
 -reports+={clean:added,"service(clean_guidelines_common||additional_clean_guidelines)"}
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index ed0c1e8ed0..c4e304fc44 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -290,11 +290,17 @@ Deviations related to MISRA C:2012 Rules:
        statements are deliberate.
      - Project-wide deviation; tagged as `disapplied` for ECLAIR.
 
+   * - R14.4
+     - A controlling expression of 'if' and iteration statements having
+       integer, character or pointer type has a semantics that is well-known to
+       all Xen developers.
+     - Tagged as `deliberate` for ECLAIR.
+
    * - R14.4
      - The XEN team relies on the fact that the enum is_dying has the
        constant with assigned value 0 act as false and the other ones as true,
        therefore have the same behavior of a boolean.
-     - Project-wide deviation; tagged as `deliberate` for ECLAIR.
+     - Tagged as `deliberate` for ECLAIR.
 
    * - R16.2
      - Complying with the Rule would entail a lot of code duplication in the
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 13:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715819.1117741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WJ1-00046P-7x; Thu, 02 May 2024 13:16:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715819.1117741; Thu, 02 May 2024 13:16:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WJ1-00046H-50; Thu, 02 May 2024 13:16:59 +0000
Received: by outflank-mailman (input) for mailman id 715819;
 Thu, 02 May 2024 13:16:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2WJ0-00046B-2W
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:16:58 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3fb011f0-0886-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 15:16:56 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41b782405d5so67780295e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 06:16:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p20-20020a05600c469400b0041bc41287cesm1955755wmo.16.2024.05.02.06.16.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 06:16:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3fb011f0-0886-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714655815; x=1715260615; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bqFPop1z8PxPmfrFqSXPCSC5YolOrRav4f99mOoojAw=;
        b=c5IqEtOWvrh10lo+E6vdI22DpXEcEY/aW9zPH2mT8+eK+SCJmrI7VGx2ja3l1TnpbH
         NjebAmFm1ZOv61lFbfUUL0Ppc+0YXDjhfQcpHLRo0GCDYRgcE2HWxoaks86j1gGzK0pq
         HPTAv291x75ENEwG3GL731DsX642C/qzTENIMbz5GMUXi8eITjJYvt24I9EfjPjWp+eJ
         7FtzrWPPBY55b76TBOU8IWIOOA+XQUI0gIkGWD8SpI9SiKhtg6YjNCxpQcQ1+YULNv/m
         c4wChkmhCj0zGnTiwmHz3JsF9INJKJpL/Y/ci+4H7+YZUPwwiwM/nmVbNZrylBRP++/r
         pi/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714655815; x=1715260615;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bqFPop1z8PxPmfrFqSXPCSC5YolOrRav4f99mOoojAw=;
        b=bKan9ljsOU9XaAxsPcLZAEQtjQ4atYwHLuve9V++ZiaJ+bU+4BxM6j5NEuxdtb2ZSG
         gmXM9USzyer24TnkHWHZjcIgYToRKNknVeJxBaO2CPJuWcUuf1EE1iMWEENeGoFUwtkr
         wBd0TdJZCba15UTwE35eA9H5aOXZ5msvyZbjK0WhnSNFh8SlWTrv0znFnIOkq0gMDZwF
         pwzS03FS2oQjVbA+KttXJUmDa7jU/UfPOd7vriDXUouZ7+swHWCVgSvOv1MTGsrAM7/M
         F2UbKAvF1oLoxK+10Yvc9JKYijyigPx2qUuvawzekdH2F4wC6L2LHtUYO2HQHuUngPXe
         7GtA==
X-Forwarded-Encrypted: i=1; AJvYcCUYaSl5lPGZH1jm/VMb9i8k9CRaxBGNZ+/dYy9EYgcmYGSUXJkufMmIOculTeRrK4LhTgQIMhauPYpxXhAURWLLRRmwiFcf7EGbVQx54aM=
X-Gm-Message-State: AOJu0YzwMQCSt4N6lB/1kwOvRWGVQTbFvmirzL7Yw3rPdJeIpVeBVp5W
	qxbBH8rpqeHGFX8VWc5OqfRVUtIPzwfHQcX4QR8OEZp8c2K0D9hAyzLGmhrL+DR08lAChUJCgpo
	=
X-Google-Smtp-Source: AGHT+IEddnz0SBQ2g0qRwYCQa8V0am179NRP5IVIF5yGrgmAC8xSyhnzCAnXzZmFe4frEFx/rhAsEQ==
X-Received: by 2002:a05:600c:4748:b0:41a:47db:290c with SMTP id w8-20020a05600c474800b0041a47db290cmr4941137wmo.5.1714655815459;
        Thu, 02 May 2024 06:16:55 -0700 (PDT)
Message-ID: <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com>
Date: Thu, 2 May 2024 15:16:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy
 defined names
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240502114922.94288-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502114922.94288-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 13:49, Roger Pau Monne wrote:
> Like it was done recently for libxl, switch to using the auto-generated feature
> names by the processing of cpufeatureset.h, this allows removing the open-coded
> feature names, and unifies the feature naming with libxl and the hypervisor.
> 
> Introduce a newly auto-generated array that contains the feature names indexed
> at featureset bit position, otherwise using the existing INIT_FEATURE_NAMES
> would require iterating over the array elements until a match with the expected
> bit position is found.
> 
> Note that leaf names need to be kept, as the current auto-generated data
> doesn't contain the leaf names.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
except that ...

> --- a/xen/tools/gen-cpuid.py
> +++ b/xen/tools/gen-cpuid.py
> @@ -475,6 +475,32 @@ def write_results(state):
>      state.output.write(
>  """}
>  
> +""")
> +
> +    state.output.write(
> +"""
> +#define INIT_FEATURE_NAME_ARRAY { \\
> +""")
> +
> +    try:
> +        _tmp = state.names.iteritems()
> +    except AttributeError:
> +        _tmp = state.names.items()

... can't figure what this try/except is needed for. Hopefully someone with
better Python foo than mine can take a look.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 13:17:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:17:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715821.1117751 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WJB-0004OL-IH; Thu, 02 May 2024 13:17:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715821.1117751; Thu, 02 May 2024 13:17:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WJB-0004OC-FM; Thu, 02 May 2024 13:17:09 +0000
Received: by outflank-mailman (input) for mailman id 715821;
 Thu, 02 May 2024 13:17:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2WJA-00046N-KI
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:17:08 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 46bee23d-0886-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 15:17:07 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-4196c62bb4eso58101565e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 06:17:07 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k7-20020a7bc407000000b0041674bf7d4csm5646744wmi.48.2024.05.02.06.17.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 06:17:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46bee23d-0886-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714655827; x=1715260627; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=K+P0Gj5m26NK8WSz79zHfWPdpUEHx50wuUj1P9z801s=;
        b=Z2VMzwN7r+h6KDNbaOy4+n4lmSu3JTc8kbE0PVeHQsCyH27J7f8PCRQOCMLcVtyuqm
         XtfrvxsF0awMu90SligR3ROQsAkzMH6+V47sMS/tnDpA/1Zst77ewEeXwYusZDLppdwm
         WKMBDFnHHqKcn5hceEPCXLcTKORTLNieYK4C0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714655827; x=1715260627;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=K+P0Gj5m26NK8WSz79zHfWPdpUEHx50wuUj1P9z801s=;
        b=TldJ+09nPCEbWLbSZ/fnsG3D4j16zBUUWLJ8T6xbmbgI3/j3ylarxNPWV3Si3XwKJ5
         Dkcc8Frz8iGv9059k5YJU/dNQNflN8RVxSzv9GFsqMLMdCCAg9ZmI2/gXZCY0BNbcR9C
         4u3CunNMNmVcHSEm69JLKNHrk4gKYmIVvDPHMVy8HzBBI32075hM3KImaXNaYKVkgsOO
         Ut5RUSdPlevzeVL15GxiOxXmK2CLkR3hhgzIXJmBPmu1jsdw7BJkVax4KDav1p+hMn3w
         mlxRQqYllLCExJw83GQX0iKARQHlT6ygLUkz9yv9rUKppmO6mdY7aF5QC/e5/meTZFF6
         Zf3g==
X-Forwarded-Encrypted: i=1; AJvYcCXiXzhEpMQ7dOffWJ+Bioj5+YpzllpWyajTbZUK/KQcEMiLceF412IqRJfBwGVDqz4QIzo+qDTjYkdxxMXuugV9wYhoOfWcC70aZxErpW0=
X-Gm-Message-State: AOJu0YzzkCV7jt3Q8DSU2rFFcPexT8D4P7Obxn8a5ho4YqvTfgHNoEYp
	ogjmyczM5h2fFe99tXJzXyrqXSg3mWj4fIF4VGacpJGvzLlfR2ch5yiSDY25WS0=
X-Google-Smtp-Source: AGHT+IFNbpj6ZFIEsQRJQk3DrlVyfZehQ6N+xpND6IDc4S5NKywFsmUWQ+OH+zY+vPKjqQrv84+y+A==
X-Received: by 2002:a05:600c:3b26:b0:41d:9d8c:5a34 with SMTP id m38-20020a05600c3b2600b0041d9d8c5a34mr3412092wms.9.1714655827254;
        Thu, 02 May 2024 06:17:07 -0700 (PDT)
Message-ID: <1628a593-c028-401d-bdff-b21a8b21b324@citrix.com>
Date: Thu, 2 May 2024 14:17:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] x86/xstate: Rework xstate_ctxt_size() as
 xstate_uncompressed_size()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-3-andrew.cooper3@citrix.com>
 <741d4bef-8711-4802-91ad-8a6b4459da61@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <741d4bef-8711-4802-91ad-8a6b4459da61@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 1:19 pm, Jan Beulich wrote:
> On 29.04.2024 20:28, Andrew Cooper wrote:
>> @@ -567,16 +567,51 @@ static unsigned int hw_uncompressed_size(uint64_t xcr0)
>>      return size;
>>  }
>>  
>> -/* Fastpath for common xstate size requests, avoiding reloads of xcr0. */
>> -unsigned int xstate_ctxt_size(u64 xcr0)
>> +unsigned int xstate_uncompressed_size(uint64_t xcr0)
>>  {
>> +    unsigned int size = XSTATE_AREA_MIN_SIZE, i;
>> +
>>      if ( xcr0 == xfeature_mask )
>>          return xsave_cntxt_size;
>>  
>>      if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */
>>          return 0;
>>  
>> -    return hw_uncompressed_size(xcr0);
>> +    if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) )
> This is open-coded XSTATE_FP_SSE, which I wouldn't mind if ...
>
>> +        return size;
>> +
>> +    /*
>> +     * For the non-legacy states, search all activate states and find the
>> +     * maximum offset+size.  Some states (e.g. LWP, APX_F) are out-of-order
>> +     * with respect their index.
>> +     */
>> +    xcr0 &= ~XSTATE_FP_SSE;
> ... you didn't use that macro here (and once further down). IOW please
> be consistent, no matter which way round.

Oh yes - that's a consequence of these hunks having between written 3
years apart.

It's important for the first one (logical comparison against a bitmap)
that it's split apart.

>
>> +    for_each_set_bit ( i, &xcr0, 63 )
>> +    {
>> +        unsigned int s;
>> +
>> +        ASSERT(xstate_offsets[i] && xstate_sizes[i]);
>> +
>> +        s = xstate_offsets[i] && xstate_sizes[i];
> You mean + here, don't you?

Yes I do...  That was a victim of a last minute refactor.

It also shows that even the cross-check with hardware isn't terribly
effective.  More on that in the other thread.

>  Then:
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
> I'm also inclined to suggest making this the initializer of s.

Hmm, good point.  Will change.

Thanks,

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 13:21:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:21:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715830.1117760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WNb-00067l-2g; Thu, 02 May 2024 13:21:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715830.1117760; Thu, 02 May 2024 13:21:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WNb-00067e-08; Thu, 02 May 2024 13:21:43 +0000
Received: by outflank-mailman (input) for mailman id 715830;
 Thu, 02 May 2024 13:21:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+3sG=MF=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s2WNZ-00067Y-Fi
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:21:41 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e92fc707-0886-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 15:21:40 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id B50A31FBB6;
 Thu,  2 May 2024 13:21:38 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 876EC1386E;
 Thu,  2 May 2024 13:21:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id QZaTH2KTM2acTAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 02 May 2024 13:21:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e92fc707-0886-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714656099; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=W0UBg54t8/+8VJyJ7E60mws+UPuTMw8aFgHfUteifWk=;
	b=ocSqdOGL39EmmuLr8CRjlQIjzIQdRYgVRKyDAcnYRopWLdv8Ul0lWJn6t3Ub4Cd30CkEV0
	5pbCmHVmB5Z1GybWmHJieDozNdgCaIiyA0cDNW61oEqdksyeLn6RbdyidjHLnUgDQdGlf+
	B7XxU7fcZqCz70fzT988siomb3gGlfs=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=mIial2bY
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714656098; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=W0UBg54t8/+8VJyJ7E60mws+UPuTMw8aFgHfUteifWk=;
	b=mIial2bYLyrAa77O0BDevQzy/j9naVCaSEQA9tCJu4eRzVN/BWo+zCyjxLhZ+39yPIm55J
	uAQ5DvXW9p9+a5QoioQZLr87yPttX3/YHaWZ3FvN/pMmqfQdge2e4QmyeDQ/4vIfZ6VLOR
	M3HyOQhl5JQKBtDrq7PWLgxTCoqiOwc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH] tools/tests: don't let test-xenstore write nodes exceeding default size
Date: Thu,  2 May 2024 15:21:36 +0200
Message-Id: <20240502132136.532-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Result: default: False [-0.02 / 50.00];
	MID_CONTAINS_FROM(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_MISSING_CHARSET(0.50)[];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.10)[text/plain];
	MX_GOOD(-0.01)[];
	BAYES_HAM(-0.01)[47.29%];
	ARC_NA(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim,suse.com:email];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	FROM_HAS_DN(0.00)[];
	TO_DN_SOME(0.00)[];
	MIME_TRACE(0.00)[0:+];
	SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	FROM_EQ_ENVFROM(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	RCPT_COUNT_THREE(0.00)[3];
	RCVD_TLS_ALL(0.00)[];
	DKIM_TRACE(0.00)[suse.com:+]
X-Spam-Flag: NO
X-Spam-Score: -0.02
X-Spamd-Bar: /
X-Rspamd-Queue-Id: B50A31FBB6
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Rspamd-Action: no action

Today test-xenstore will write nodes with 3000 bytes node data. This
size is exceeding the default quota for the allowed node size. While
working in dom0 with C-xenstored, OCAML-xenstored does not like that.

Use a size of 2000 instead, which is lower than the allowed default
node size of 2048.

Fixes: 3afc5e4a5b75 ("tools/tests: add xenstore testing framework")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/tests/xenstore/test-xenstore.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/tests/xenstore/test-xenstore.c b/tools/tests/xenstore/test-xenstore.c
index d7d44cc0fa..7a9bd9afb3 100644
--- a/tools/tests/xenstore/test-xenstore.c
+++ b/tools/tests/xenstore/test-xenstore.c
@@ -408,9 +408,9 @@ static int test_ta3_deinit(uintptr_t par)
 #define TEST(s, f, p, l) { s, f ## _init, f, f ## _deinit, (uintptr_t)(p), l }
 struct test tests[] = {
 TEST("read 1", test_read, 1, "Read node with 1 byte data"),
-TEST("read 3000", test_read, 3000, "Read node with 3000 bytes data"),
+TEST("read 2000", test_read, 2000, "Read node with 2000 bytes data"),
 TEST("write 1", test_write, 1, "Write node with 1 byte data"),
-TEST("write 3000", test_write, 3000, "Write node with 3000 bytes data"),
+TEST("write 2000", test_write, 2000, "Write node with 2000 bytes data"),
 TEST("dir", test_dir, 0, "List directory"),
 TEST("rm node", test_rm, 0, "Remove single node"),
 TEST("rm dir", test_rm, WRITE_BUFFERS_N, "Remove node with sub-nodes"),
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu May 02 13:24:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:24:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715833.1117770 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WQC-0006mU-Es; Thu, 02 May 2024 13:24:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715833.1117770; Thu, 02 May 2024 13:24:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WQC-0006mN-CJ; Thu, 02 May 2024 13:24:24 +0000
Received: by outflank-mailman (input) for mailman id 715833;
 Thu, 02 May 2024 13:24:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2WQB-0006mF-CE
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:24:23 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 49257624-0887-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 15:24:21 +0200 (CEST)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-34d0aa589a4so2868219f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 06:24:21 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 dd3-20020a0560001e8300b0034c9f060a14sm1266076wrb.11.2024.05.02.06.24.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 06:24:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49257624-0887-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714656261; x=1715261061; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XJggx4Oe2H3ZRxBPQOTa7S2IqKQJawSMummjLXb+uhk=;
        b=Pz39zQLVmYQnp24NpH661GXZq6oHHOn4n0W1fzUbIbFibOSljRNZlcWnCjDuLvXeKT
         5DgtAqCDIeEP3j61dNHM8uHHS5e7rKqrSkxT6ASSDCnEFCWOodLS1/bKx+2QNAmO8HID
         foCSdStx0q9438sf4P/t3ZH/4+Md5qTorkUrw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714656261; x=1715261061;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XJggx4Oe2H3ZRxBPQOTa7S2IqKQJawSMummjLXb+uhk=;
        b=ZUDFSndftFGRHAqfSlY/YnqlhdsqNa0/pUkmJ2Gr4sr9SvQKwQtOSStEGHo3/6HKpF
         Q4DiiqrqwtYCu6m3d0XsrcKKRvsldr+CtVHNmJtL+SCEnkCQP1EhG1lLJa2F/Zbh2JHD
         SR5wxGJetoI0bgltmGJoA5E/I7mxEzLKpDe1f5C461XQyLFgAAzsyeaogqiVc4d5yGem
         GsvW8qvAnZAXlgtx6iaimBB5n4L1IHR6X+zkmBV+/hOcGzuTFoQ2yva/J2pl1/XZV5cq
         6JTiu17P4k+/ybQLXtbwQJBetLb6d4Wa7/FEO/6LwFawAUWYtMmD2c8zg47bW9y9vl6C
         QbhQ==
X-Forwarded-Encrypted: i=1; AJvYcCXUjpGTzulTfRceu7g+vMwsMO0DK9S5xSpe5CebS38jcdVwhedo7OUyvXHhfdxlLGHBB3+w7IUX7KfRkTMOeSq3ByKzDoJtO/ZxpECb768=
X-Gm-Message-State: AOJu0YxkdoxGgENKynrQPT5PlWGwweV2eMB6my78PKP9MTwA47h1clAS
	/c6zavItsyazJVY+BRAHTVBwehSPS/YPNzYxuQj/pL7wv+hwABNaUn3i1+z0EnA=
X-Google-Smtp-Source: AGHT+IHBCMzuoOZu8ZzsCiTgicmu+FsTYZfOaABcCNpwrd7AaLXDFq2zr4ExxfwaRaiqedtKbGw7IQ==
X-Received: by 2002:adf:fa4a:0:b0:34d:9fb2:5148 with SMTP id y10-20020adffa4a000000b0034d9fb25148mr1581433wrr.31.1714656260758;
        Thu, 02 May 2024 06:24:20 -0700 (PDT)
Message-ID: <b5560120-96c5-4b97-a4cc-36fbcfdd223c@citrix.com>
Date: Thu, 2 May 2024 14:24:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] x86/cpu-policy: Simplify recalculate_xstate()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-4-andrew.cooper3@citrix.com>
 <ecd6e142-7a08-46e8-83c0-7d9b0bc6f66f@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ecd6e142-7a08-46e8-83c0-7d9b0bc6f66f@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 1:39 pm, Jan Beulich wrote:
> On 29.04.2024 20:28, Andrew Cooper wrote:
>> Make use of the new xstate_uncompressed_size() helper rather than maintaining
>> the running calculation while accumulating feature components.
> xstate_uncompressed_size() isn't really a new function, but the re-work of
> an earlier one. That, aiui, could have been used here, too, just that it
> would have been inefficient to do so. IOW perhaps drop "the new"?

Ok.

>
>> The rest of the CPUID data can come direct from the raw cpu policy.  All
>> per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}*
>> instructions.
>>
>> Use for_each_set_bit() rather than opencoding a slightly awkward version of
>> it.  Mask the attributes in ecx down based on the visible features.  This
>> isn't actually necessary for any components or attributes defined at the time
>> of writing (up to AMX), but is added out of an abundance of caution.
> As to this, ...
>
>> @@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p)
>>          return;
>>  
>>      if ( p->basic.avx )
>> -    {
>>          xstates |= X86_XCR0_YMM;
>> -        xstate_size = max(xstate_size,
>> -                          xstate_offsets[X86_XCR0_YMM_POS] +
>> -                          xstate_sizes[X86_XCR0_YMM_POS]);
>> -    }
>>  
>>      if ( p->feat.mpx )
>> -    {
>>          xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR;
>> -        xstate_size = max(xstate_size,
>> -                          xstate_offsets[X86_XCR0_BNDCSR_POS] +
>> -                          xstate_sizes[X86_XCR0_BNDCSR_POS]);
>> -    }
>>  
>>      if ( p->feat.avx512f )
>> -    {
>>          xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM;
>> -        xstate_size = max(xstate_size,
>> -                          xstate_offsets[X86_XCR0_HI_ZMM_POS] +
>> -                          xstate_sizes[X86_XCR0_HI_ZMM_POS]);
>> -    }
>>  
>>      if ( p->feat.pku )
>> -    {
>>          xstates |= X86_XCR0_PKRU;
>> -        xstate_size = max(xstate_size,
>> -                          xstate_offsets[X86_XCR0_PKRU_POS] +
>> -                          xstate_sizes[X86_XCR0_PKRU_POS]);
>> -    }
>>  
>> -    p->xstate.max_size  =  xstate_size;
>> +    /* Subleaf 0 */
>> +    p->xstate.max_size =
>> +        xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY);
>>      p->xstate.xcr0_low  =  xstates & ~XSTATE_XSAVES_ONLY;
>>      p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32;
>>  
>> +    /* Subleaf 1 */
>>      p->xstate.Da1 = Da1;
>> +    if ( p->xstate.xsavec )
>> +        ecx_mask |= XSTATE_ALIGN64;
>> +
>>      if ( p->xstate.xsaves )
>>      {
>> +        ecx_mask |= XSTATE_XSS;
>>          p->xstate.xss_low   =  xstates & XSTATE_XSAVES_ONLY;
>>          p->xstate.xss_high  = (xstates & XSTATE_XSAVES_ONLY) >> 32;
>>      }
>> -    else
>> -        xstates &= ~XSTATE_XSAVES_ONLY;
>>  
>> -    for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i )
>> +    /* Subleafs 2+ */
>> +    xstates &= ~XSTATE_FP_SSE;
>> +    BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63);
>> +    for_each_set_bit ( i, &xstates, 63 )
>>      {
>> -        uint64_t curr_xstate = 1UL << i;
>> -
>> -        if ( !(xstates & curr_xstate) )
>> -            continue;
>> -
>> -        p->xstate.comp[i].size   = xstate_sizes[i];
>> -        p->xstate.comp[i].offset = xstate_offsets[i];
>> -        p->xstate.comp[i].xss    = curr_xstate & XSTATE_XSAVES_ONLY;
>> -        p->xstate.comp[i].align  = curr_xstate & xstate_align;
> ... for this bit, isn't the move from this ...
>
>> +        /*
>> +         * Pass through size (eax) and offset (ebx) directly.  Visbility of
>> +         * attributes in ecx limited by visible features in Da1.
>> +         */
>> +        p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a;
>> +        p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b;
>> +        p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask;
> ... to this changing what guests get to see, i.e. (mildly?) incompatible?

No.

The only "rows" in leaf 0xd we expose to guests are AVX, MPX, AVX512 and
PKU  (higher up in this hunk, selecting valid bits in xstates).  None of
these have a non-zero value in ecx.

This is a latent bug until we offer AMX or CET, hence why I wanted to
complete this series before your AMX series goes in.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 13:25:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 13:25:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715837.1117780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WRB-0007HC-OP; Thu, 02 May 2024 13:25:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715837.1117780; Thu, 02 May 2024 13:25:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2WRB-0007H5-Lv; Thu, 02 May 2024 13:25:25 +0000
Received: by outflank-mailman (input) for mailman id 715837;
 Thu, 02 May 2024 13:25:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2WRA-0007CS-8X
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 13:25:24 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6dbb6353-0887-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 15:25:22 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41adf155cffso54752715e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 06:25:22 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 j3-20020a05600c1c0300b004186f979543sm5785088wms.33.2024.05.02.06.25.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 06:25:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6dbb6353-0887-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714656322; x=1715261122; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=L20Y43NAiZ2l0D/I+x7e0KD3rY8AxANBIZEupH6VNk4=;
        b=IMxW9XqUGlmorNDY7rvWeyqLqf6olVtn3yG6XT+5n6c8H+uKTtvUhxYccJ494hZ5fJ
         gs+3nxyoh5w9Ayi7CtKVk9SxOB96OQtqG6F3bs42A+41b7noGYXUGInHyMj/M5LgXL/A
         ofZb+nxGj94/bcbZ8LE6DIUpN6UTSjscXuUdY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714656322; x=1715261122;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=L20Y43NAiZ2l0D/I+x7e0KD3rY8AxANBIZEupH6VNk4=;
        b=geLuWMcLKDWPC8+qsVM/RkLXxMIv/XAEl60Hh7Up8jhm4thhmePpSWX8e/GsZ93C7E
         7yeF2+LpiTBvaBi0etCThDoNCBkQAL2IRrdqQIqg4Pq33iTGNJg7tDgKHykpU0djvWYR
         af0nE8NAN4bkC42vkINalMCrG56pvVQVBr3Sdoj3wB0E81kg267W6M1OAraP70YYPdaJ
         HxHyXUMdCu5P96MV56aTczrA9GseB3LOPlmho8mgdOzadjEukWe0nwU3mXOQsOqyK7cy
         gQDeIZnoHGSdWCyG9OX3mJsgy5r5n057H/pv7Hu6HNMXqTkryEw/E0ERoAaqlEP9PzZL
         bp/A==
X-Forwarded-Encrypted: i=1; AJvYcCXczSTyNY1+cw7JDcTJ1RtoNl3V9+yGFLYiWPJfYLrlAh/eP16ZqG1XNZPE2uFwxkvIBBvOzb/ens8wtjwUBxiLRrzdN0D7Z0GzJbCwIAo=
X-Gm-Message-State: AOJu0Yw8rZd7iUS047/bg/BuPW4tPrfOt9eRuSEPz237E6AXj7FF/GV5
	HZIIRMVSqKDbDVqIExSo+XtXcsAtwqLXsI7aAEkz5YCvINqTOnM/qud6Y9Cv5h8=
X-Google-Smtp-Source: AGHT+IFat97fJEqwDrjZf2OF7QHtResJXHXIMqZa3mdJUsy6zL7KXEmdHpLIXU26ojrvTtdXQbv0SA==
X-Received: by 2002:a05:600c:1392:b0:41b:f116:87e0 with SMTP id u18-20020a05600c139200b0041bf11687e0mr4077394wmf.29.1714656322163;
        Thu, 02 May 2024 06:25:22 -0700 (PDT)
Message-ID: <3c5ea8e3-87ce-4048-a3f1-7a963706f5b2@citrix.com>
Date: Thu, 2 May 2024 14:25:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/tests: don't let test-xenstore write nodes
 exceeding default size
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>
References: <20240502132136.532-1-jgross@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240502132136.532-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 2:21 pm, Juergen Gross wrote:
> Today test-xenstore will write nodes with 3000 bytes node data. This
> size is exceeding the default quota for the allowed node size. While
> working in dom0 with C-xenstored, OCAML-xenstored does not like that.
>
> Use a size of 2000 instead, which is lower than the allowed default
> node size of 2048.
>
> Fixes: 3afc5e4a5b75 ("tools/tests: add xenstore testing framework")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thanks for figuring this out.  I'll give this and the return code fix a
spin through XenRT and check there's nothing else unexpected hiding.


From xen-devel-bounces@lists.xenproject.org Thu May 02 14:32:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:32:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715849.1117791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XU9-0001ms-OU; Thu, 02 May 2024 14:32:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715849.1117791; Thu, 02 May 2024 14:32:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XU9-0001ml-Kn; Thu, 02 May 2024 14:32:33 +0000
Received: by outflank-mailman (input) for mailman id 715849;
 Thu, 02 May 2024 14:32:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2XU8-0001mf-Bg
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:32:32 +0000
Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com
 [2607:f8b0:4864:20::f36])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd496ba0-0890-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 16:32:29 +0200 (CEST)
Received: by mail-qv1-xf36.google.com with SMTP id
 6a1803df08f44-6a071595d22so41288096d6.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:32:29 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 z7-20020a0cfec7000000b006a0d2a52017sm382796qvs.146.2024.05.02.07.32.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 07:32:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd496ba0-0890-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714660348; x=1715265148; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=nWmuVZtzqrHtgUIOTc0CdZVJQG5k8cyxHYUZVt6RUXI=;
        b=qYzimQwf/KI7dD612OBSbe87u1xAjqIyhnmfyJFGWA0L62E0tn/+wTaJiyK+YbRHci
         tAy5rxIv67nWMVys08Vm3/mzhysjs7Z7mcp3PEuBPBfRllG7OuwHMKyQ8CU54kTohe3E
         WEBNvs+VT/D7/FeShqyKWFREi+5aLftMN5okw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714660348; x=1715265148;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nWmuVZtzqrHtgUIOTc0CdZVJQG5k8cyxHYUZVt6RUXI=;
        b=gbDTjfs3gaJB3gUUwWCdZAhdrVlnSqKUSiMP4xfTI7kzEGPN1g4d+oLF/0CNPlS1LI
         UZxmDsk3sE+VIdyBAbOumxo62sPbnFpPWFbaFkS5ipk7tnJatCoZXuugXWtZvfbdr3wW
         B4tx/JzG5qoMLAdy5BkWP1w31G4RgU0WXrZuM7hWRp/Sf4fTz/YXFt4hGh5a+zB0MCnT
         PcSj3GaxlPCAGuaV+kZsbxIPf5tmSj5NuI/1gxQLsrBGEk6P211/Nb7Xf6CfxeO88C8w
         /XNJ34mScPv31Mvb6wvH3bVuhGtTnvY69Rq6941rZRsTRfb6KBjNJG+S7M9GMidTXHiS
         6zrA==
X-Forwarded-Encrypted: i=1; AJvYcCWVPmedYZNQ81bgWuP2VKYRzyCEWulwFBZxriXPEHNZyw/R2otq8nbgL9YoahlKRoGsbmqgwY8VB3Ug6AxLX7OmjmzPts0LH77dorw7Wis=
X-Gm-Message-State: AOJu0YxOnTB32saXdDd4aUENwT1BYXclCBrzLGwLjipbP2geRn5hzc8N
	2lIIDMNxZsYJuSLJgR7/xA9sUz7K0W2M/OlODvbf/3MUcs0fRl7Bp6LTYLI8ryA=
X-Google-Smtp-Source: AGHT+IHzYCmkq8hIAY2wDFDqxCGaEQMH+U6Z+/+KAzJ24/D552bXtXY8jx6SbO9qv00KGtuiVkwmRA==
X-Received: by 2002:a05:6214:27c5:b0:69b:5ecd:7c9a with SMTP id ge5-20020a05621427c500b0069b5ecd7c9amr5797751qvb.51.1714660347700;
        Thu, 02 May 2024 07:32:27 -0700 (PDT)
Message-ID: <f20dae24-02eb-4091-9ab5-816587eca43b@citrix.com>
Date: Thu, 2 May 2024 15:32:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] x86/cpuid: Fix handling of xsave dynamic leaves
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-5-andrew.cooper3@citrix.com>
 <d87b31be-6c52-4271-a61f-bf31043f507d@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d87b31be-6c52-4271-a61f-bf31043f507d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 2:04 pm, Jan Beulich wrote:
> On 29.04.2024 20:28, Andrew Cooper wrote:
>> If max leaf is greater than 0xd but xsave not available to the guest, then the
>> current XSAVE size should not be filled in.  This is a latent bug for now as
>> the guest max leaf is 0xd, but will become problematic in the future.
> Why would this not be an issue when .max_leaf == 0xd, but .xsave == 0?

Hmm, true.  I'll adjust the description.

>
>> The comment concerning XSS state is wrong.  VT-x doesn't manage host/guest
>> state automatically, but there is provision for "host only" bits to be set, so
>> the implications are still accurate.
>>
>> Introduce {xstate,hw}_compressed_size() helpers to mirror the uncompressed
>> ones.
>>
>> This in turn higlights a bug in xstate_init().  Defaulting this_cpu(xss) to ~0
>> requires a forced write to clear it back out.  This in turn highlights that
>> it's only a safe default on systems with XSAVES.
> Well, yes, such an explicit write was expected to appear when some xsaves-
> based component would actually be enabled. Much like the set_xcr0() there.

I stumbled over this because the debug logic ended up trying to restore
0xffffffff (the thing it read out as the "last" value) back into XSS. 
This works about as well as you'd expect.
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>>
>> The more I think about it, the more I think that cross-checking with hardware
>> is a bad move.  It's horribly expensive, and for supervisor states in
>> particular, liable to interfere with functionality.
> I agree, but I'd also like to see the cross checking not dropped entirely.
> Can't we arrange for such to happen during boot, before we enable any such
> functionality? After all even in debug builds it's not overly useful to do
> the same cross-checking (i.e. for identical inputs) over and over again.
> Of course doing in an exhaustive manner may be okay for the uncompressed
> values, but might be a little too much for all possible combinations in
> order to check compressed values, too.

Given the observation of patch 2 being buggy and my final sanity test
before posting didn't notice, I think doing this all at boot would be a
much better idea.

I think I'm going to do a new patch early in the series as an adjustment
to xstate_init().

We can't feasibly test every combination, but what ought to do is
linearly accumulate the xstates Xen knows about and checking that in
each case the size(s) increase as appropriate.

This will have a substantial impact on the remainder of the series, but
I think the end result will be all the better for it.
>> --- a/xen/arch/x86/xstate.c
>> +++ b/xen/arch/x86/xstate.c
>> @@ -614,6 +614,65 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0)
>>      return size;
>>  }
>>  
>> +static unsigned int hw_compressed_size(uint64_t xstates)
>> +{
>> +    uint64_t curr_xcr0 = get_xcr0(), curr_xss = get_msr_xss();
>> +    unsigned int size;
>> +    bool ok;
>> +
>> +    ok = set_xcr0(xstates & ~XSTATE_XSAVES_ONLY);
>> +    ASSERT(ok);
>> +    set_msr_xss(xstates & XSTATE_XSAVES_ONLY);
>> +
>> +    size = cpuid_count_ebx(XSTATE_CPUID, 1);
>> +
>> +    ok = set_xcr0(curr_xcr0);
>> +    ASSERT(ok);
>> +    set_msr_xss(curr_xss);
>> +
>> +    return size;
>> +}
>> +
>> +unsigned int xstate_compressed_size(uint64_t xstates)
>> +{
>> +    unsigned int i, size = XSTATE_AREA_MIN_SIZE;
>> +
>> +    if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */
>> +        return 0;
>> +
>> +    if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) )
> Same comment as on the earlier change regarding the (lack of) use of ....
>
>> +        return size;
>> +
>> +    /*
>> +     * For the compressed size, every component matters.  Some are
>> +     * automatically rounded up to 64 first.
>> +     */
>> +    xstates &= ~XSTATE_FP_SSE;
> ... this constant up there.
>
>> +    for_each_set_bit ( i, &xstates, 63 )
>> +    {
>> +        if ( test_bit(i, &xstate_align) )
>> +            size = ROUNDUP(size, 64);
>> +
>> +        size += xstate_sizes[i];
>> +    }
> The comment is a little misleading: As you have it in code, it is not the
> component's size that is rounded up, but its position. Maybe "Some have
> their start automatically rounded up to 64 first"?

Size in that sentence referees to the compressed size of everything, not
the size of the component.

But I'll try to make it clearer.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 14:34:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715852.1117801 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XVa-0002PC-29; Thu, 02 May 2024 14:34:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715852.1117801; Thu, 02 May 2024 14:34:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XVZ-0002P5-V7; Thu, 02 May 2024 14:34:01 +0000
Received: by outflank-mailman (input) for mailman id 715852;
 Thu, 02 May 2024 14:34:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=G3ja=MF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2XVY-0002Ov-Rs
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:34:00 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 03649f15-0891-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 16:33:59 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-34d7a32bdd3so1798205f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:33:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020adfef92000000b0034c71090653sm1393418wro.57.2024.05.02.07.33.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 07:33:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03649f15-0891-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714660439; x=1715265239; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nBcZ19biHtw7JmPakETY+HmASRLKwRBwP7yYtSvbndg=;
        b=HBB4WvNN6mFhJE8y2rEf52I253B2lX6/LjR+AwLvY3Bnnj010/7amV5+2tNBYzxP1P
         VFu/vop4+TgppqaCba5Pdls+LYIXiyqeYJ1GmjkooHdE49+URcIqxEGRi0Zb/gs5kUps
         c7oYet1q6EgBrTVnzqlYT6FruJhpFGkeKEhxu0GLyIAf0OMfv3izs9jC1G3MEHGOvx45
         EuhBr46p0qH3n9Y13AF8cO6L9GGzR35Kf0VkaDJEfSOGTJNu+FYS09+P2420dL3Ka0yP
         dRUYFvQ7RY5DQ6i1sGe8QmXqjkNZEpd0hniFbhujttF9EPFwy99rZiL/jKowyUr59vVP
         wKVg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714660439; x=1715265239;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nBcZ19biHtw7JmPakETY+HmASRLKwRBwP7yYtSvbndg=;
        b=Fouesxrk2vwSdDhgcbfV6bYNamT6hA+J0aWHJWgdS5GD7UpsbtlZMntVjifdzJ/nnq
         JnHiNjTDeXwLdYVE0aIraoNc/Y/fMdHzLyQ4ypIsFimlNejBJhKshS3wg7PC0lfRq54P
         aCFnxONohKZu1b6FYXm6HR2Bnqh1CaU3ayH/UzKUepQAyQB+FVpbJTrS/G5DDPURumuN
         WaPB5cLo5AsG/09+w6/2UiMf0bqJ6dHVYujjjgavtTkuHxHJbpbiMSzdd+sqTCIhBzlQ
         IhgRNLZSnku3L/5Zs3GZzU4WO+bX2jAch/0/DhaMxWyV2VyVf9+YDTJt4xA9CN5tLy47
         idtg==
X-Forwarded-Encrypted: i=1; AJvYcCXBC62NvvOergSxNShsAVq9ERRSiLamfTJ8uea5DU7KICHYY/sh29N5PCSewCEOIA+9Sj3nU1giaItcpkCpwL7grqImZxhQR0U0NEelgYI=
X-Gm-Message-State: AOJu0Yx9yh0sPL7Y7qFB/BXZsiABEgzhhso4HenA98OSXJ+PMLNfEdCV
	lP0k0uiEF/72FZrNV3iJxI2+kYDlFn72/mH1JzJ92JqaFGUuvBTbOj8am3fuLw==
X-Google-Smtp-Source: AGHT+IF3IhUroj06pFWq7wLgGIHKh19EZOY0W6AgDpgXhf5bFTmHZ8i62IhgS4sQZctFnOtHl2iviA==
X-Received: by 2002:a5d:6449:0:b0:349:eb59:c188 with SMTP id d9-20020a5d6449000000b00349eb59c188mr5510wrw.5.1714660438816;
        Thu, 02 May 2024 07:33:58 -0700 (PDT)
Message-ID: <dad023f2-c74b-4067-a3e3-4236d9424689@suse.com>
Date: Thu, 2 May 2024 16:33:57 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] x86/cpu-policy: Simplify recalculate_xstate()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240429182823.1130436-1-andrew.cooper3@citrix.com>
 <20240429182823.1130436-4-andrew.cooper3@citrix.com>
 <ecd6e142-7a08-46e8-83c0-7d9b0bc6f66f@suse.com>
 <b5560120-96c5-4b97-a4cc-36fbcfdd223c@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b5560120-96c5-4b97-a4cc-36fbcfdd223c@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.05.2024 15:24, Andrew Cooper wrote:
> On 02/05/2024 1:39 pm, Jan Beulich wrote:
>> On 29.04.2024 20:28, Andrew Cooper wrote:
>>> Make use of the new xstate_uncompressed_size() helper rather than maintaining
>>> the running calculation while accumulating feature components.
>> xstate_uncompressed_size() isn't really a new function, but the re-work of
>> an earlier one. That, aiui, could have been used here, too, just that it
>> would have been inefficient to do so. IOW perhaps drop "the new"?
> 
> Ok.
> 
>>
>>> The rest of the CPUID data can come direct from the raw cpu policy.  All
>>> per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}*
>>> instructions.
>>>
>>> Use for_each_set_bit() rather than opencoding a slightly awkward version of
>>> it.  Mask the attributes in ecx down based on the visible features.  This
>>> isn't actually necessary for any components or attributes defined at the time
>>> of writing (up to AMX), but is added out of an abundance of caution.
>> As to this, ...
>>
>>> @@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p)
>>>          return;
>>>  
>>>      if ( p->basic.avx )
>>> -    {
>>>          xstates |= X86_XCR0_YMM;
>>> -        xstate_size = max(xstate_size,
>>> -                          xstate_offsets[X86_XCR0_YMM_POS] +
>>> -                          xstate_sizes[X86_XCR0_YMM_POS]);
>>> -    }
>>>  
>>>      if ( p->feat.mpx )
>>> -    {
>>>          xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR;
>>> -        xstate_size = max(xstate_size,
>>> -                          xstate_offsets[X86_XCR0_BNDCSR_POS] +
>>> -                          xstate_sizes[X86_XCR0_BNDCSR_POS]);
>>> -    }
>>>  
>>>      if ( p->feat.avx512f )
>>> -    {
>>>          xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM;
>>> -        xstate_size = max(xstate_size,
>>> -                          xstate_offsets[X86_XCR0_HI_ZMM_POS] +
>>> -                          xstate_sizes[X86_XCR0_HI_ZMM_POS]);
>>> -    }
>>>  
>>>      if ( p->feat.pku )
>>> -    {
>>>          xstates |= X86_XCR0_PKRU;
>>> -        xstate_size = max(xstate_size,
>>> -                          xstate_offsets[X86_XCR0_PKRU_POS] +
>>> -                          xstate_sizes[X86_XCR0_PKRU_POS]);
>>> -    }
>>>  
>>> -    p->xstate.max_size  =  xstate_size;
>>> +    /* Subleaf 0 */
>>> +    p->xstate.max_size =
>>> +        xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY);
>>>      p->xstate.xcr0_low  =  xstates & ~XSTATE_XSAVES_ONLY;
>>>      p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32;
>>>  
>>> +    /* Subleaf 1 */
>>>      p->xstate.Da1 = Da1;
>>> +    if ( p->xstate.xsavec )
>>> +        ecx_mask |= XSTATE_ALIGN64;
>>> +
>>>      if ( p->xstate.xsaves )
>>>      {
>>> +        ecx_mask |= XSTATE_XSS;
>>>          p->xstate.xss_low   =  xstates & XSTATE_XSAVES_ONLY;
>>>          p->xstate.xss_high  = (xstates & XSTATE_XSAVES_ONLY) >> 32;
>>>      }
>>> -    else
>>> -        xstates &= ~XSTATE_XSAVES_ONLY;
>>>  
>>> -    for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i )
>>> +    /* Subleafs 2+ */
>>> +    xstates &= ~XSTATE_FP_SSE;
>>> +    BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63);
>>> +    for_each_set_bit ( i, &xstates, 63 )
>>>      {
>>> -        uint64_t curr_xstate = 1UL << i;
>>> -
>>> -        if ( !(xstates & curr_xstate) )
>>> -            continue;
>>> -
>>> -        p->xstate.comp[i].size   = xstate_sizes[i];
>>> -        p->xstate.comp[i].offset = xstate_offsets[i];
>>> -        p->xstate.comp[i].xss    = curr_xstate & XSTATE_XSAVES_ONLY;
>>> -        p->xstate.comp[i].align  = curr_xstate & xstate_align;
>> ... for this bit, isn't the move from this ...
>>
>>> +        /*
>>> +         * Pass through size (eax) and offset (ebx) directly.  Visbility of
>>> +         * attributes in ecx limited by visible features in Da1.
>>> +         */
>>> +        p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a;
>>> +        p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b;
>>> +        p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask;
>> ... to this changing what guests get to see, i.e. (mildly?) incompatible?
> 
> No.
> 
> The only "rows" in leaf 0xd we expose to guests are AVX, MPX, AVX512 and
> PKU  (higher up in this hunk, selecting valid bits in xstates).  None of
> these have a non-zero value in ecx.
> 
> This is a latent bug until we offer AMX or CET, hence why I wanted to
> complete this series before your AMX series goes in.

Oh, so finally a description of that very issue you kept mentioning. With
the small tweak to the description then
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 02 14:34:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:34:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715855.1117811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XWL-0003Gj-9c; Thu, 02 May 2024 14:34:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715855.1117811; Thu, 02 May 2024 14:34:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XWL-0003Gc-6x; Thu, 02 May 2024 14:34:49 +0000
Received: by outflank-mailman (input) for mailman id 715855;
 Thu, 02 May 2024 14:34:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPCJ=MF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2XWK-0003B2-AS
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:34:48 +0000
Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com
 [2607:f8b0:4864:20::731])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ff1fd41-0891-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 16:34:47 +0200 (CEST)
Received: by mail-qk1-x731.google.com with SMTP id
 af79cd13be357-78ecd752a7cso555524685a.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:34:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 e10-20020a05620a12ca00b0078d65fbde2bsm406490qkl.86.2024.05.02.07.34.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:34:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ff1fd41-0891-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714660486; x=1715265286; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=h33R/j8iwl+jeKLC2uWicBtIIQspILXeWPCqycXkLtg=;
        b=NXAje8FUAkS58yhG1oEq/ie8R/XNF9b0PrP0nZaPb8mfDaHlzaP1pGR+kQri6/OIvI
         DJIhXvT/xcy/yMlEPoUema+uhqDd/WvKS7UW8MzqntROzKpnjrh3lBDwDu4r0juJBsqL
         zvF7ljEqELv8euRPBupF5hsTLh2y9DxMGo9is=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714660486; x=1715265286;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=h33R/j8iwl+jeKLC2uWicBtIIQspILXeWPCqycXkLtg=;
        b=jY21lebnOUQfA1H/d0KYoOK4s+VT4Q5Nug6aKDNe8/lCTcaPOkf3j0MbGN8fgRLpcl
         d2dXttC/z7R6dDJgJwLnVU8QMeu1XEZGjWEe/VblBpTE0NTM99sM1qS09OQdpsFqpe7D
         vkKdyORtirnAkcMe1onB5BEkHDZMwnm/rsEjHpkVlvZmaahISM+JvPFUWnHJwaxhuY0l
         Fac4iw3iODdaUfTluima4p0xLW8//5X3mHLxNwz6tU8z2IAVGV/q93Uc6jqQo/v95ydZ
         txOgjUg+Np5MPEoEbO3IdxjCHOhmzrYAYlTKVdmAS5r91VHtDPfL9y2rUUDWuwJjqIg5
         BCzw==
X-Forwarded-Encrypted: i=1; AJvYcCVplxiQzMXdSZfSayBx6fA09aVIm0a7Wv8B2ahPEXhom/2FMneIVjw2nLPc3neVOPPJedNMY/5TeOCyNcYp/IFTHCkDOhPJ8BVPJjhrpfM=
X-Gm-Message-State: AOJu0Yzfi05qDGhpp1WJleBSPcu+J7VdM7J3Dj/acHYhMr6GQQkRlECp
	sA31RBx8ZufzZJuQADRKsydhxpU/JIcfhOapA0skRtmz22ALc0H+pZqtGLY6WRE=
X-Google-Smtp-Source: AGHT+IHKEwVIb8rtmfrTtO7fcepZoKajc4IyCRyfViGbhIgKyFC6qOjCUGZ72SSKkva65uBdtn89GA==
X-Received: by 2002:a05:620a:2158:b0:790:919c:2051 with SMTP id m24-20020a05620a215800b00790919c2051mr5840876qkm.5.1714660486577;
        Thu, 02 May 2024 07:34:46 -0700 (PDT)
Date: Thu, 2 May 2024 16:34:44 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy
 defined names
Message-ID: <ZjOkhMGzHWEEdsQf@macbook>
References: <20240502114922.94288-1-roger.pau@citrix.com>
 <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com>

On Thu, May 02, 2024 at 03:16:54PM +0200, Jan Beulich wrote:
> On 02.05.2024 13:49, Roger Pau Monne wrote:
> > Like it was done recently for libxl, switch to using the auto-generated feature
> > names by the processing of cpufeatureset.h, this allows removing the open-coded
> > feature names, and unifies the feature naming with libxl and the hypervisor.
> > 
> > Introduce a newly auto-generated array that contains the feature names indexed
> > at featureset bit position, otherwise using the existing INIT_FEATURE_NAMES
> > would require iterating over the array elements until a match with the expected
> > bit position is found.
> > 
> > Note that leaf names need to be kept, as the current auto-generated data
> > doesn't contain the leaf names.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> except that ...

Thanks.

> 
> > --- a/xen/tools/gen-cpuid.py
> > +++ b/xen/tools/gen-cpuid.py
> > @@ -475,6 +475,32 @@ def write_results(state):
> >      state.output.write(
> >  """}
> >  
> > +""")
> > +
> > +    state.output.write(
> > +"""
> > +#define INIT_FEATURE_NAME_ARRAY { \\
> > +""")
> > +
> > +    try:
> > +        _tmp = state.names.iteritems()
> > +    except AttributeError:
> > +        _tmp = state.names.items()
> 
> ... can't figure what this try/except is needed for. Hopefully someone with
> better Python foo than mine can take a look.

If I understood this correctly (which I might have not, I know very
little Python), iteritems() method was removed in Python 3, and what
used to be iteritems() in Python 2 is items() on Python 3.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 02 14:44:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:44:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715864.1117821 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xfj-00053j-6k; Thu, 02 May 2024 14:44:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715864.1117821; Thu, 02 May 2024 14:44:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xfj-00053c-2h; Thu, 02 May 2024 14:44:31 +0000
Received: by outflank-mailman (input) for mailman id 715864;
 Thu, 02 May 2024 14:44:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1QD4=MF=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s2Xfh-00051X-RE
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:44:29 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a1c7ad1-0892-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 16:44:28 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a5878caeb9eso1022449866b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:44:28 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 v21-20020aa7d9d5000000b005700ef75274sm605382eds.33.2024.05.02.07.44.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 07:44:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a1c7ad1-0892-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1714661067; x=1715265867; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RU/PusuG1kQtflZ+fLSuhghAx8tGgTkOn3ucNF+ThOc=;
        b=LHDzOwjxlmbjqtOkF5mqgmZFYi/qafNPXAVWXbjFhoXxRujii0UT8YKF96oHVtvIkx
         kElsOWwsNNOyZA2NA/jSca8VndGlktWpiWvu6vNc9pSFBOxei7So96jOq23Es0fyMTV8
         KKtYDiG00YVqnhIbO/9B7GwN642LcSUou69ds=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661067; x=1715265867;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RU/PusuG1kQtflZ+fLSuhghAx8tGgTkOn3ucNF+ThOc=;
        b=dSCrM+hmv2ciVnqQ1CebqhtgOUjd840lDeggGwgdXySlhi6c7BH6CT7xwC8kAvF8qg
         a2In7BKJHNbUlPQd6w177oWxftpdAhMkQYJ973/+tqRmD8iy4sZA9gpbiV3uTu/giDi7
         An+TqXUCkNesRA37qVJxv/Fu6wQxhv2hVQCPEn8B6EjpYLNGWLBmoTiMJr4e3wxEdp7G
         pwnz9URVFRKWH6mqZcJ24crTr/GvcJBID+uX+rXdg5PraY1/dbPrDtx3X4UMUzabmooK
         m1kAJ+fSHl0G6UnVN8h9qDtI+Bx23D2wmXcvydDBdnRRzZ8msdWzGV+CTdgQz098TqEl
         lG9A==
X-Forwarded-Encrypted: i=1; AJvYcCX3ta9eR3cm6aEmn/ymYfG9yEzLwIIvvnWKuGKBXUreHyAHwwjDJmt3lCQlfVzsjVJG7EDe9PJb0EYPHDcw0+Ee9kpB/gJzZqHVbEfCu2M=
X-Gm-Message-State: AOJu0YykDc8tYSmZ1cBajt3olm33gzW6ZJvJUhtVa0bGVROZMMOjTixp
	35MJ5Mw2TABE3rvkQz92WSgHQYZv4WP6C11UXEUyLFSbnb5G20WvyRC980F1ELI=
X-Google-Smtp-Source: AGHT+IHEhohMfu8ocz/drT1r/eukOk/Rvb1fuTY0WloD6RQadTOZVk50YqLmtnP0NUBArYpZEF02Mw==
X-Received: by 2002:a50:9e48:0:b0:572:947d:559f with SMTP id z66-20020a509e48000000b00572947d559fmr4128352ede.3.1714661067380;
        Thu, 02 May 2024 07:44:27 -0700 (PDT)
Message-ID: <adcc0c16-6f02-454e-8a2c-ae00a6cbebd2@cloud.com>
Date: Thu, 2 May 2024 15:44:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] xen/x86: Derive topologically correct x2APIC IDs from
 the policy
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240109153834.4192-1-alejandro.vallejo@cloud.com>
 <20240109153834.4192-6-alejandro.vallejo@cloud.com>
 <14891d02-e9e9-4650-b572-e6f071f1506c@suse.com>
 <e0963ddd-c3b7-44dd-a3b7-67811f3e9d7a@cloud.com>
 <52741b69-8bcb-492f-a8bc-f035b2747fe8@suse.com>
 <4a280eab-81f7-4a87-b531-3633311a4c4a@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <4a280eab-81f7-4a87-b531-3633311a4c4a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02/05/2024 07:57, Jan Beulich wrote:
> On 02.05.2024 08:55, Jan Beulich wrote:
>> On 01.05.2024 18:35, Alejandro Vallejo wrote:
>>> Hi,
>>>
>>> On 26/03/2024 16:41, Jan Beulich wrote:
>>>> On 09.01.2024 16:38, Alejandro Vallejo wrote:
>>>>> --- a/xen/lib/x86/policy.c
>>>>> +++ b/xen/lib/x86/policy.c
>>>>> @@ -2,15 +2,78 @@
>>>>>  
>>>>>  #include <xen/lib/x86/cpu-policy.h>
>>>>>  
>>>>> -uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t vcpu_id)
>>>>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>>>>>  {
>>>>>      /*
>>>>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>>>>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>>>>> -     *       until all infra is in place to retrieve or derive the initial
>>>>> -     *       x2APIC ID from migrated domains.
>>>>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>>>>> +     * the next topological scope. For example, assuming a system with 2
>>>>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>>>>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>>>>> +     * the number of threads in a module.
>>>>> +     *
>>>>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>>>>> +     * level (cores/complex, etc) so we can return it as-is.
>>>>>       */
>>>>> -    return vcpu_id * 2;
>>>>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>>>>> +        return p->topo.subleaf[lvl].nr_logical;
>>>>
>>>> Is "!= Intel" really appropriate here? I'd rather see this being "AMD || Hygon".
>>>
>>> Sure, I don't particularly mind, but why? As far as we know only Intel
>>> has this interpretation for the part counts. I definitely haven't seen
>>> any non-Intel CPUID dump in which the part count is the total number of
>>> threads (Centaur/Zhaoxin are not multithreaded, and don't expose leaves
>>> 1f or e26, as far as I could see).
>>
>> Because of x86'es origin and perhaps other historical aspects, cloning
>> Intel behavior is far more likely.

That claim doesn't hold very well seeing how...

>> The fact that Hygon matches AMD is
>> simply because they took AMD's design wholesale.

... this statement contradicts it. We can't predict which new vendor (if
any) will be cloned/mimicked next, so that's not a very plausible reason
to prioritise a specific vendor in conditionals.

It remains to be seen what a Zhaoxin actually looks like, because I
couldn't get ahold of a complete cpuid dump.

> 
> Perhaps: See how many dead ends AMD have created, i.e. stuff they proudly
> introduced into the architecture, but then gave up again (presumably for
> diverging too far from Intel, and hence lacking long term acceptance):
> 3DNow!, LWP, and XOP just to name those that come to mind right away.
> 
> Jan

I can't say I agree on the cause; Regardless I'd rather not discuss the
relative merits of vendors with regards to backwards compatibility, as
that's besides the point. The point is whether there's a credible
technical reason to prefer this...

  if ( !(a & (B | C)) )
      foo();

... to this...

  if ( a == A )
      foo();

..., as is the case in patch6.

I argue there's not, and in fact legibility-wise the latter is very
clearly superior.

There's also a compelling reason to keep the check coherent on both
generators to avoid bad surprises down the line.

Cheers,
Alejandro



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:50:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:50:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715870.1117830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XlC-0006rC-Tg; Thu, 02 May 2024 14:50:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715870.1117830; Thu, 02 May 2024 14:50:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2XlC-0006r5-QN; Thu, 02 May 2024 14:50:10 +0000
Received: by outflank-mailman (input) for mailman id 715870;
 Thu, 02 May 2024 14:50:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2XlB-0006qv-Dh; Thu, 02 May 2024 14:50:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2XlB-00054j-7b; Thu, 02 May 2024 14:50:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2XlA-0002r5-Q0; Thu, 02 May 2024 14:50:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2XlA-0002CY-PO; Thu, 02 May 2024 14:50:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lB41fuJP1dcyZbQ//RLuNDmHe8vbGQdf3MOf1p7Nopk=; b=MNMecqUGfg+WqlERWz2at6F+um
	/+yKpXJetou/0so1QOve95pU80zWpsU9V/Vz9G6Tg88004fL60Np4eg7iXA4XkF4f+n+EBvp3B5hn
	RmvvLyxCiJtGwfuFHhFIxuf89krPWXlft/4RWSBicasNDRLTcQSJx8UStgT+dM/U7oEs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185900-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185900: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=fecf55a66a1cf908c2f906bedb79fe2e8362d50f
X-Osstest-Versions-That:
    ovmf=5d4c5253e8bbc0baa8837fcd868925212df85201
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 14:50:08 +0000

flight 185900 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185900/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 fecf55a66a1cf908c2f906bedb79fe2e8362d50f
baseline version:
 ovmf                 5d4c5253e8bbc0baa8837fcd868925212df85201

Last test of basis   185879  2024-04-30 11:12:53 Z    2 days
Testing same since   185900  2024-05-02 13:14:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Michael Roth <michael.roth@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   5d4c5253e8..fecf55a66a  fecf55a66a1cf908c2f906bedb79fe2e8362d50f -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 02 14:56:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:56:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715877.1117841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrg-0007ur-Iq; Thu, 02 May 2024 14:56:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715877.1117841; Thu, 02 May 2024 14:56:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrg-0007uk-Fd; Thu, 02 May 2024 14:56:52 +0000
Received: by outflank-mailman (input) for mailman id 715877;
 Thu, 02 May 2024 14:56:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrf-0007uZ-6r
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:56:51 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3490b1b8-0894-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 16:56:50 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-572ba002a6bso952131a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:56:50 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3490b1b8-0894-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661809; x=1715266609; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=p1AqukxMMW4ychAZU18Fl1hEbPBi09j+hpTr9oqZL2s=;
        b=NNnH/ceu2M/Nh51anVDsgYkN9u1y6n/B64frP7ESaJwrxr51gU+DAefo9u+qVyKSSQ
         XDpfrdt99DffdC6Vwe80jClaTNIoo2PLh4Uw2KQNJk3Ve+lV3hBXaQso6H3dEPRKzeg8
         soZJwn/dZzsEbG+XY4bPxCTrn8QT2UMz0S6/2jjqJ0qMyQu6LYx034EcTYrgEOKEIozW
         UYKfVoAklib/sTX1myJ4bRXEJsruYGSuqs5g8jKrBiSV2tI3eLzOgtaqIFGFtoNGaRuU
         VDEFMdDFoDnppNBujnFDugfJWUYjweo8/xrq4Zy8Lx5jGJuCyaQHr3AFLq8vvAl+WWk9
         oWIw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661809; x=1715266609;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=p1AqukxMMW4ychAZU18Fl1hEbPBi09j+hpTr9oqZL2s=;
        b=B9LCT8iVn1mxicOkyWX290vnn78DH71shWUamiAqm26Tc+9KbXMtPb7VXXTk9hgLEc
         8bB3SPY5BzxbDCPztJahmSJnabdP4kvmDz0C49ZviKnXyl6t7jOh4FLXREl6+26pqmCu
         11SDDoMTZFbROARvZ/CBVHS8jPAXRjTv2AZe+s89ST9ULXAEM/QkZBh40kvNnrqu/nSC
         dgpt746KKNAwx8PhPK9Lz60tFrWYcBm/ldUvnc/QN4rzv1HJN1Oy98iiCO/0soVjQJ+2
         rnVZz6zf31UmslRsR4NLDWOvwdMu5BiCHqT4aH3iVz0W12xQ312k5Cu+5AwtrIW9VNr7
         1Q4Q==
X-Gm-Message-State: AOJu0Yzj5LcGHET6ne4gJGt0MnOX2k6b/vzAKPgUaSGmioURY2Eemf79
	apQe3Rkp9aWaKq3nXVxgFryFbev52keq9wAJVVBDGAVSutyZgHBjLkAbdaXJgSzMCgviJMEjReV
	X
X-Google-Smtp-Source: AGHT+IEGkYk9wI3zGIWbX2B9WIhp+uE9Mrl+WFz62FiMAOQ4blxFiLDBTE95rpIxVdIv464c3+ILBg==
X-Received: by 2002:a50:c355:0:b0:570:1ddc:5e4f with SMTP id q21-20020a50c355000000b005701ddc5e4fmr3528187edb.8.1714661809293;
        Thu, 02 May 2024 07:56:49 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v4 0/5] FF-A notifications
Date: Thu,  2 May 2024 16:56:40 +0200
Message-Id: <20240502145645.1201613-1-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi,

This patch set adds support for FF-A notifications. We only support
global notifications, per vCPU notifications remain unsupported.

The first three patches are further cleanup and can be merged before the
rest if desired.

A physical SGI is used to make Xen aware of pending FF-A notifications. The
physical SGI is selected by the SPMC in the secure world. Since it must not
already be used by Xen the SPMC is in practice forced to donate one of the
secure SGIs, but that's normally not a problem. The SGI handling in Xen is
updated to support registration of handlers for SGIs that aren't statically
assigned, that is, SGI IDs above GIC_SGI_MAX.

Two items stand out in the last patch, "xen/arm: ffa: support notification":
- The interrupt handler must be registered for each online CPU, but this
  will usually happen for the secondary CPUs after FF-A has been initialized
  so we may need to take some further action in the error path
- The interrupt handler need to get hold of the domain struct of a guest as
  indicated by the SPMC. In this patch set we use
  rcu_lock_live_remote_domain_by_id() to get a safe reference to the domain
  struct.

Thanks,
Jens

v3->v4:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.

v2->v3:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.

v1->v2:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.
- Added Bertrands R-B for "xen/arm: ffa: refactor ffa_handle_call()",
  "xen/arm: ffa: use ACCESS_ONCE()", and
  "xen/arm: ffa: simplify ffa_handle_mem_share()"

Jens Wiklander (5):
  xen/arm: ffa: refactor ffa_handle_call()
  xen/arm: ffa: use ACCESS_ONCE()
  xen/arm: ffa: simplify ffa_handle_mem_share()
  xen/arm: allow dynamically assigned SGI handlers
  xen/arm: ffa: support notification

 xen/arch/arm/gic.c              |  12 +-
 xen/arch/arm/include/asm/gic.h  |   2 +-
 xen/arch/arm/irq.c              |  18 +-
 xen/arch/arm/tee/Makefile       |   1 +
 xen/arch/arm/tee/ffa.c          | 108 +++++---
 xen/arch/arm/tee/ffa_notif.c    | 453 ++++++++++++++++++++++++++++++++
 xen/arch/arm/tee/ffa_partinfo.c |   9 +-
 xen/arch/arm/tee/ffa_private.h  |  60 ++++-
 xen/arch/arm/tee/ffa_shm.c      |  33 +--
 xen/include/public/arch-arm.h   |  14 +
 10 files changed, 647 insertions(+), 63 deletions(-)
 create mode 100644 xen/arch/arm/tee/ffa_notif.c

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:56:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:56:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715879.1117861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrk-0008Op-Vh; Thu, 02 May 2024 14:56:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715879.1117861; Thu, 02 May 2024 14:56:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrk-0008Oi-Sb; Thu, 02 May 2024 14:56:56 +0000
Received: by outflank-mailman (input) for mailman id 715879;
 Thu, 02 May 2024 14:56:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrj-0008Nr-VK
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:56:55 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 36ae0eb3-0894-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 16:56:53 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a58ebdd8b64so697119766b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:56:53 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36ae0eb3-0894-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661812; x=1715266612; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Injz1eHKMU34CsdRACoEV/3/mGn5muvQCz5MmACq6o4=;
        b=w8Bc0/IDV4vUvWfkLoImEK9miZxMLSwIKmWLQXHsN8yDDIWyiJu1+LurdkNbsjyxxH
         awPvRauHGFD3eQL8PYYfildBMfVS1PpL8euszyl94Xfya66JcPRohyS7qjJJbhE+LmiI
         jptlRDq2al1eErZEjNlMwKniQzvyZ2+NWuJpTm3m7Eww1Z5MP755VeeTcKc62EjXsGVB
         zY4fKkC1Y5LQu6CVJIWpopZe9aW6eCI3ittSvKm3hhJGPlG/JisLt1YWCP/7L2pOIhi8
         ryKe7EtoJYLtFegPL3oy7Sl+41tDrLzKkADQenFR953heTm71lCPpS2mKWyRJRnOVnNw
         b/cw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661812; x=1715266612;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Injz1eHKMU34CsdRACoEV/3/mGn5muvQCz5MmACq6o4=;
        b=tFOKKMtcGZ59yTZgJCzY6K0vH+ftUtEiJaB/ztxVhIvFfORpxKf/03f6g86t/HSrAI
         VvHAivBVwFFsE/unzs4PVAEhGQbqRFYzmOCzeAET6a+9uVxfAIlIHXIrD4HtgSdz5HKR
         KMhjtGcTFSy0/nfZjZpC/mtKUd6xsNTG3bBcFO2b1fJXWxi45xxw679M6LXXbpgSUNb3
         c/3PwUPD7BCM6SO7tw3Xu9FYrSard1a9YBOK7s1d2o1YktxdMUSysYAzThwebg1oQC96
         9oZoPaLmYyc2s6z+AB0GiSQlVtCKNsBThao/gmdZz7L5eG4GNbuuE1ibOVZCUbf1SjNW
         cWhg==
X-Gm-Message-State: AOJu0YzRhDGjTARIvyZbA0VO71hmv3JqmLWWFMA71APqHl1aXhuu43MR
	NC+Df42JPn9FjTwTq49YuLv7xdz5e1pX4CbL8jg8b/io0rtWQBo+7cfW/I5n+dipVea+PqWA1eM
	c
X-Google-Smtp-Source: AGHT+IG7UqvYZtt6Bo69ToLIf6CpiL0D2ojKKHUUFQFlcGZ729buhKvFkmi0kpX8z5ksuzb1kcj64Q==
X-Received: by 2002:a50:c31d:0:b0:56e:2b31:b111 with SMTP id a29-20020a50c31d000000b0056e2b31b111mr4073339edb.7.1714661812744;
        Thu, 02 May 2024 07:56:52 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v4 2/5] xen/arm: ffa: use ACCESS_ONCE()
Date: Thu,  2 May 2024 16:56:42 +0200
Message-Id: <20240502145645.1201613-3-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502145645.1201613-1-jens.wiklander@linaro.org>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Replace read_atomic() with ACCESS_ONCE() to match the intended use, that
is, to prevent the compiler from (via optimization) reading shared
memory more than once.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa_shm.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/tee/ffa_shm.c b/xen/arch/arm/tee/ffa_shm.c
index eed9ad2d2986..75a5b66aeb4c 100644
--- a/xen/arch/arm/tee/ffa_shm.c
+++ b/xen/arch/arm/tee/ffa_shm.c
@@ -7,6 +7,7 @@
 #include <xen/sizes.h>
 #include <xen/types.h>
 #include <xen/mm.h>
+#include <xen/lib.h>
 #include <xen/list.h>
 #include <xen/spinlock.h>
 
@@ -171,8 +172,8 @@ static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
 
     for ( n = 0; n < range_count; n++ )
     {
-        page_count = read_atomic(&range[n].page_count);
-        addr = read_atomic(&range[n].address);
+        page_count = ACCESS_ONCE(range[n].page_count);
+        addr = ACCESS_ONCE(range[n].address);
         for ( m = 0; m < page_count; m++ )
         {
             if ( pg_idx >= shm->page_count )
@@ -527,13 +528,13 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
 
     mem_access = ctx->tx + trans.mem_access_offs;
-    if ( read_atomic(&mem_access->access_perm.perm) != FFA_MEM_ACC_RW )
+    if ( ACCESS_ONCE(mem_access->access_perm.perm) != FFA_MEM_ACC_RW )
     {
         ret = FFA_RET_NOT_SUPPORTED;
         goto out_unlock;
     }
 
-    region_offs = read_atomic(&mem_access->region_offs);
+    region_offs = ACCESS_ONCE(mem_access->region_offs);
     if ( sizeof(*region_descr) + region_offs > frag_len )
     {
         ret = FFA_RET_NOT_SUPPORTED;
@@ -541,8 +542,8 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
     }
 
     region_descr = ctx->tx + region_offs;
-    range_count = read_atomic(&region_descr->address_range_count);
-    page_count = read_atomic(&region_descr->total_page_count);
+    range_count = ACCESS_ONCE(region_descr->address_range_count);
+    page_count = ACCESS_ONCE(region_descr->total_page_count);
 
     if ( !page_count )
     {
@@ -557,7 +558,7 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
     }
     shm->sender_id = trans.sender_id;
-    shm->ep_id = read_atomic(&mem_access->access_perm.endpoint_id);
+    shm->ep_id = ACCESS_ONCE(mem_access->access_perm.endpoint_id);
 
     /*
      * Check that the Composite memory region descriptor fits.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:56:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:56:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715878.1117850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrh-000893-Oe; Thu, 02 May 2024 14:56:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715878.1117850; Thu, 02 May 2024 14:56:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrh-00088w-M2; Thu, 02 May 2024 14:56:53 +0000
Received: by outflank-mailman (input) for mailman id 715878;
 Thu, 02 May 2024 14:56:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrg-0007uZ-5F
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:56:52 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 355ec5b0-0894-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 16:56:51 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-572adaa172cso557118a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:56:51 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 355ec5b0-0894-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661811; x=1715266611; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2VzeWK8x82YxZO3FACx+2RlqRVfXhRtSExk4PP34ai4=;
        b=FW5wAuh6FO40tbiFPWsahhPfdYQpptM8uZZ7bMX9IVW9fzP5enIV+jx70BaT0MNnpO
         nKPnm44seVjWLsNwho0Uc5o6uPyDcGcNvj5iJHG/kOoq+ZIZdZULsEwO19jYMIsjZXOO
         IeweHMarW6g7mT+LAZEiryLd3XkX8It9f08CpfkE0EDBDhe3iCyhYbJ8zfwmyDcEI0pQ
         37j+G33YXzZ9gmzHZMY7bNyQFzeaUvVUbsivL/qdAxYk+FkjHnoLhynGYBO/jzaNJUOX
         skq8LHzz5Cn999cj4RZ/hIJRRqXjQmyu/xOJgAHgDh8s+HSYt3czG5UH1dbCli6uG6lJ
         nQJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661811; x=1715266611;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2VzeWK8x82YxZO3FACx+2RlqRVfXhRtSExk4PP34ai4=;
        b=adi5Y2k/Pve3LXWKE4pSGEq7KqyXSK1MsVSzY5suyNIcz42sStozFpViNP/He9TuEm
         tNDPGfny+YlbxLe1Fx83J8GZuE4UlRA0yEWGMOqPf9QODMi1njh7VLN/wvIxaAmQ01bk
         vbhhB4cgiUGRkTk2NMUc+7+HVYn04PIMqJ9cL2eZAH7EY3UcBqepioAV1g6DgE+TPLb4
         1Pa3O+dtpUIqHnZlf0aqHJNSIDob1ywpfzgbTGptR41Qx5kQTtHdJN0aWIgJCFs0+jHP
         YmQrd/ex5oYL1guQt+THaaBqF2eNsM5Uhb39fpXXwhsw4RxDSI3NlbQD3Mupjf24aLhT
         9D0A==
X-Gm-Message-State: AOJu0YyFXu8HDYW/tAOHd0JceDrBRfjUW4vtsJ3IihrNtfedaw2M8PI8
	Ldju4KW6A/GVApvewcRzmvojAuYzYOPzIxko0NNeVjkuY9lvpbdXg+qa4M5TWuvaNga5djJvZee
	b
X-Google-Smtp-Source: AGHT+IG4PNAt8sZmx/Xvf451yAjWAXkcpUe0VMrJTcWHf4v9vwpgHknG1PyFhplfqM4XnRBS6+qB/Q==
X-Received: by 2002:a50:8ad6:0:b0:571:da40:22d9 with SMTP id k22-20020a508ad6000000b00571da4022d9mr2338474edk.10.1714661810749;
        Thu, 02 May 2024 07:56:50 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v4 1/5] xen/arm: ffa: refactor ffa_handle_call()
Date: Thu,  2 May 2024 16:56:41 +0200
Message-Id: <20240502145645.1201613-2-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502145645.1201613-1-jens.wiklander@linaro.org>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Refactors the large switch block in ffa_handle_call() to use common code
for the simple case where it's either an error code or success with no
further parameters.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa.c | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 8665201e34a9..5209612963e1 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -273,18 +273,10 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
     case FFA_RXTX_MAP_64:
         e = ffa_handle_rxtx_map(fid, get_user_reg(regs, 1),
 				get_user_reg(regs, 2), get_user_reg(regs, 3));
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_RXTX_UNMAP:
         e = ffa_handle_rxtx_unmap();
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_PARTITION_INFO_GET:
         e = ffa_handle_partition_info_get(get_user_reg(regs, 1),
                                           get_user_reg(regs, 2),
@@ -299,11 +291,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
         return true;
     case FFA_RX_RELEASE:
         e = ffa_handle_rx_release();
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_MSG_SEND_DIRECT_REQ_32:
     case FFA_MSG_SEND_DIRECT_REQ_64:
         handle_msg_send_direct_req(regs, fid);
@@ -316,17 +304,19 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
         e = ffa_handle_mem_reclaim(regpair_to_uint64(get_user_reg(regs, 2),
                                                      get_user_reg(regs, 1)),
                                    get_user_reg(regs, 3));
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
 
     default:
         gprintk(XENLOG_ERR, "ffa: unhandled fid 0x%x\n", fid);
         ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
         return true;
     }
+
+    if ( e )
+        ffa_set_regs_error(regs, e);
+    else
+        ffa_set_regs_success(regs, 0, 0);
+    return true;
 }
 
 static int ffa_domain_init(struct domain *d)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:57:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:57:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715880.1117871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xro-0000Fz-6x; Thu, 02 May 2024 14:57:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715880.1117871; Thu, 02 May 2024 14:57:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xro-0000Fn-4B; Thu, 02 May 2024 14:57:00 +0000
Received: by outflank-mailman (input) for mailman id 715880;
 Thu, 02 May 2024 14:56:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrn-0008Nr-9b
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:56:59 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39107826-0894-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 16:56:57 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f036e9f4dso1228115e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:56:57 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39107826-0894-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661817; x=1715266617; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XH0hx5FhOx9XD5Quim9d/dWx8ptFKhc8V8iHDPJ7zTg=;
        b=DDyb36gee6z8tn1gd7jBknUIJZzubBqhNVhopyBhnhH4b9LLKWnBxEb1T3IW4EzWkB
         qXga+JmkJeaeJfdfk9nZXNkf80oySwpcX8Xwlrm6hJpX2zteqWkFzQzasmPGlW7rYfIO
         bbChfZtjOVgAu7KR+Y81XxnYS1DrcMXNzStjgyxaKZpYsPvEcEohOUsnt0vygGY2VgSa
         ZUh7VVuR1KP6S1wDA4+sYTVpvbNxwF+IdU3U7nJWuHkbrbOaBYU/bucyJf1SNzk0u7zQ
         vn20OJMeHxGbvni6RZQ9pEAyQSW5Ql1hefSf4/GeirTgrUTqAxSjLYnxFejVnQHTiC2b
         T36A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661817; x=1715266617;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XH0hx5FhOx9XD5Quim9d/dWx8ptFKhc8V8iHDPJ7zTg=;
        b=mOAJ4LFc0o7rmhhLwDxLppUUEIjL7swT2tjiODLdc8H5iuDPMqqiSXNPNG159pQCkz
         lRPoEOnddwjx6N6IVq9ox9z6Ml5/ern3S+NCqGtC1BtuLjc525T6m6BLGS1hPTygr0vz
         qz6PiGoOLYQkgqvC0amksJEQ7WqsQ4/4EZs42HhcxefaX08CpLpKEl94rO0wOj+EaQK4
         Wgwgqp0MOtdxlRpVzImImjp+XIBKjk9AUrZgUww4O6MCTXTaUm7t7T8Xrll7lrZMn30a
         OI/8O4yQe0vVxPkrhm1rxRvzW3tFjry2MyrKVKyxj/HZvyu3DSSDaSLxm4OypO6K33gP
         RRiA==
X-Gm-Message-State: AOJu0YzZ5rE9mLzgqQ4TljNva65hNrzZqoAHU00hz666ePtpBTsg5Jvr
	TzAvV2XOukfRFfdZzlQRSf9aajdv5TqZHgN3bJ/x02ynu3rVBTjWoyUweXgAdirakTGzRXQidvq
	B
X-Google-Smtp-Source: AGHT+IHHEuhDATstiao+d7av8I+qaU4G8o2hMWEhtg5W6T+i17G5D6mSLLfR3apAoOc64/z05QBEtA==
X-Received: by 2002:a19:6454:0:b0:51e:eac7:d69b with SMTP id b20-20020a196454000000b0051eeac7d69bmr36948lfj.27.1714661814547;
        Thu, 02 May 2024 07:56:54 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v4 3/5] xen/arm: ffa: simplify ffa_handle_mem_share()
Date: Thu,  2 May 2024 16:56:43 +0200
Message-Id: <20240502145645.1201613-4-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502145645.1201613-1-jens.wiklander@linaro.org>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Simplify ffa_handle_mem_share() by removing the start_page_idx and
last_page_idx parameters from get_shm_pages() and check that the number
of pages matches expectations at the end of get_shm_pages().

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa_shm.c | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/tee/ffa_shm.c b/xen/arch/arm/tee/ffa_shm.c
index 75a5b66aeb4c..370d83ec5cf8 100644
--- a/xen/arch/arm/tee/ffa_shm.c
+++ b/xen/arch/arm/tee/ffa_shm.c
@@ -159,10 +159,9 @@ static int32_t ffa_mem_reclaim(uint32_t handle_lo, uint32_t handle_hi,
  */
 static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
                          const struct ffa_address_range *range,
-                         uint32_t range_count, unsigned int start_page_idx,
-                         unsigned int *last_page_idx)
+                         uint32_t range_count)
 {
-    unsigned int pg_idx = start_page_idx;
+    unsigned int pg_idx = 0;
     gfn_t gfn;
     unsigned int n;
     unsigned int m;
@@ -191,7 +190,9 @@ static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
         }
     }
 
-    *last_page_idx = pg_idx;
+    /* The ranges must add up */
+    if ( pg_idx < shm->page_count )
+            return FFA_RET_INVALID_PARAMETERS;
 
     return FFA_RET_OK;
 }
@@ -460,7 +461,6 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
     struct domain *d = current->domain;
     struct ffa_ctx *ctx = d->arch.tee;
     struct ffa_shm_mem *shm = NULL;
-    unsigned int last_page_idx = 0;
     register_t handle_hi = 0;
     register_t handle_lo = 0;
     int ret = FFA_RET_DENIED;
@@ -570,15 +570,9 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out;
     }
 
-    ret = get_shm_pages(d, shm, region_descr->address_range_array, range_count,
-                        0, &last_page_idx);
+    ret = get_shm_pages(d, shm, region_descr->address_range_array, range_count);
     if ( ret )
         goto out;
-    if ( last_page_idx != shm->page_count )
-    {
-        ret = FFA_RET_INVALID_PARAMETERS;
-        goto out;
-    }
 
     /* Note that share_shm() uses our tx buffer */
     spin_lock(&ffa_tx_buffer_lock);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:57:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:57:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715881.1117881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrq-0000YC-Ig; Thu, 02 May 2024 14:57:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715881.1117881; Thu, 02 May 2024 14:57:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrq-0000Y1-Eh; Thu, 02 May 2024 14:57:02 +0000
Received: by outflank-mailman (input) for mailman id 715881;
 Thu, 02 May 2024 14:57:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrp-0008Nr-5q
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:57:01 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3a1fb8c6-0894-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 16:56:59 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a55911bff66so1022060266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:56:59 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a1fb8c6-0894-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661818; x=1715266618; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oQ4pg/w8bSmRL+E3Xx38ujJg69A1ctGlBgRe1phhh4c=;
        b=IQbKVC8jqOFXn247TIW9IcJupZwTWJl2+/AtW3llvHDSlebkyF8nSzpPPOXecxJyhf
         rSGgqkaPJSn/12JDc4jxly6/Usn8Kv7UsosncsehsptxdDk2yM3v63tqQK/iQ8HKFWkF
         WTN8RhPZRw0H8T84D27halQqA51CC+7SjsjcTXZdd8yGQecrfNgpj7Ro3pDCrKUj9qGt
         v+JHYCTJKek6OvXBSjCev5jPFYeIqvbOH5ghZwHxfEjeRcCNdLpjuoC+ehRyo1hb1mtY
         FqN3L14ZzqZmLRFyPKrSWcsmegzkJDfq4XqFEE9w+0r0uo6aMzOV5RiO7Bo88zy7rqBi
         3zEA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661818; x=1715266618;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oQ4pg/w8bSmRL+E3Xx38ujJg69A1ctGlBgRe1phhh4c=;
        b=kMVvStg52jABNj7tQ05umYLN9xTBzYrS8CxxsXNPBILXreJ3+l/wnjKDSX7XCiobEd
         7iyd/zoNHX6790VgAzpXLZN48d0HvQpufq8QJDHrmQGOH59F4heqkLTJTm0Yn4EDo3oV
         36wxV6oOdWd25RcsxYyVInfMtFFqElEnY6UVwPUc9LiWknmtK73q+2VE9I56ww6US0ln
         gRc6eKBcOQFW11erqhx9kCDIWq5HOuxkovhkxMfDj8fA1b6Sm2J9XkHGt9+sgynf1wvF
         C7DCq/RUUyZbxZsj7F/q9nFlNwyiakwmpFnXPblTNpNRsVHQWut/Klpua035zq9BlHqE
         oR3g==
X-Gm-Message-State: AOJu0YxSN6lugrvtv1eEOsJQ/JL8lihbH6pjBomQXr2LqP6x6/cyP8rC
	0u1DDbUXcIMzOXIYRsVk+zQQLb3fexcVY60I7XxoR+wPrCz4rvs4QReKoxqZU8K9gPd+Q860vXd
	q
X-Google-Smtp-Source: AGHT+IFpPzIsWmfh/PLDEHEqquyefJY5D949+YxCRmBuJGZjQ9NOdEbnTp95aeqVKK3HWtO3bWmhXg==
X-Received: by 2002:a50:d597:0:b0:572:9d24:257a with SMTP id v23-20020a50d597000000b005729d24257amr4480385edi.23.1714661818693;
        Thu, 02 May 2024 07:56:58 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH v4 4/5] xen/arm: allow dynamically assigned SGI handlers
Date: Thu,  2 May 2024 16:56:44 +0200
Message-Id: <20240502145645.1201613-5-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502145645.1201613-1-jens.wiklander@linaro.org>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Updates so request_irq() can be used with a dynamically assigned SGI irq
as input. This prepares for a later patch where an FF-A schedule
receiver interrupt handler is installed for an SGI generated by the
secure world.

>From the Arm Base System Architecture v1.0C [1]:
"The system shall implement at least eight Non-secure SGIs, assigned to
interrupt IDs 0-7."

gic_route_irq_to_xen() don't gic_set_irq_type() for SGIs since they are
always edge triggered.

gic_interrupt() is updated to route the dynamically assigned SGIs to
do_IRQ() instead of do_sgi(). The latter still handles the statically
assigned SGI handlers like for instance GIC_SGI_CALL_FUNCTION.

[1] https://developer.arm.com/documentation/den0094/

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
v3->v4
- Use IRQ_TYPE_EDGE_RISING instead of DT_IRQ_TYPE_EDGE_RISING

v2->v3
- Rename GIC_SGI_MAX to GIC_SGI_STATIC_MAX and rename do_sgi() to
  do_static_sgi()
- Update comment in setup_irq() to mention that SGI irq_desc is banked
- Add ASSERT() in do_IRQ() that the irq isn't an SGI before injecting
  calling vgic_inject_irq()
- Initialize local_irqs_type[] range for SGIs as IRQ_TYPE_EDGE_RISING
- Adding link to the Arm Base System Architecture v1.0C

v1->v2
- Update patch description as requested
---
 xen/arch/arm/gic.c             | 12 +++++++-----
 xen/arch/arm/include/asm/gic.h |  2 +-
 xen/arch/arm/irq.c             | 18 ++++++++++++++----
 3 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86defe..882768252740 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -38,7 +38,7 @@ const struct gic_hw_operations *gic_hw_ops;
 static void __init __maybe_unused build_assertions(void)
 {
     /* Check our enum gic_sgi only covers SGIs */
-    BUILD_BUG_ON(GIC_SGI_MAX > NR_GIC_SGI);
+    BUILD_BUG_ON(GIC_SGI_STATIC_MAX > NR_GIC_SGI);
 }
 
 void register_gic_ops(const struct gic_hw_operations *ops)
@@ -117,7 +117,9 @@ void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
 
     desc->handler = gic_hw_ops->gic_host_irq_type;
 
-    gic_set_irq_type(desc, desc->arch.type);
+    /* SGIs are always edge-triggered, so there is need to set it */
+    if ( desc->irq >= NR_GIC_SGI)
+        gic_set_irq_type(desc, desc->arch.type);
     gic_set_irq_priority(desc, priority);
 }
 
@@ -330,7 +332,7 @@ void gic_disable_cpu(void)
     gic_hw_ops->disable_interface();
 }
 
-static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
+static void do_static_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
 {
     struct irq_desc *desc = irq_to_desc(sgi);
 
@@ -375,7 +377,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         /* Reading IRQ will ACK it */
         irq = gic_hw_ops->read_irq();
 
-        if ( likely(irq >= 16 && irq < 1020) )
+        if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) )
         {
             isb();
             do_IRQ(regs, irq, is_fiq);
@@ -387,7 +389,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         }
         else if ( unlikely(irq < 16) )
         {
-            do_sgi(regs, irq);
+            do_static_sgi(regs, irq);
         }
         else
         {
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index 03f209529b13..541f0eeb808a 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -285,7 +285,7 @@ enum gic_sgi {
     GIC_SGI_EVENT_CHECK,
     GIC_SGI_DUMP_STATE,
     GIC_SGI_CALL_FUNCTION,
-    GIC_SGI_MAX,
+    GIC_SGI_STATIC_MAX,
 };
 
 /* SGI irq mode types */
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index bcce80a4d624..5224898265a5 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -152,7 +152,13 @@ void __init init_IRQ(void)
 
     spin_lock(&local_irqs_type_lock);
     for ( irq = 0; irq < NR_LOCAL_IRQS; irq++ )
-        local_irqs_type[irq] = IRQ_TYPE_INVALID;
+    {
+        /* SGIs are always edge-triggered */
+        if ( irq < NR_GIC_SGI )
+            local_irqs_type[irq] = IRQ_TYPE_EDGE_RISING;
+        else
+            local_irqs_type[irq] = IRQ_TYPE_INVALID;
+    }
     spin_unlock(&local_irqs_type_lock);
 
     BUG_ON(init_local_irq_data(smp_processor_id()) < 0);
@@ -224,9 +230,12 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
 
     perfc_incr(irqs);
 
-    ASSERT(irq >= 16); /* SGIs do not come down this path */
+    /* Statically assigned SGIs do not come down this path */
+    ASSERT(irq >= GIC_SGI_STATIC_MAX);
 
-    if ( irq < 32 )
+    if ( irq < NR_GIC_SGI )
+        perfc_incr(ipis);
+    else if ( irq < NR_GIC_LOCAL_IRQS )
         perfc_incr(ppis);
     else
         perfc_incr(spis);
@@ -260,6 +269,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
          * The irq cannot be a PPI, we only support delivery of SPIs to
          * guests.
          */
+        ASSERT(irq >= NR_GIC_SGI);
         vgic_inject_irq(info->d, NULL, info->virq, true);
         goto out_no_end;
     }
@@ -396,7 +406,7 @@ int setup_irq(unsigned int irq, unsigned int irqflags, struct irqaction *new)
     {
         gic_route_irq_to_xen(desc, GIC_PRI_IRQ);
         /* It's fine to use smp_processor_id() because:
-         * For PPI: irq_desc is banked
+         * For SGI and PPI: irq_desc is banked
          * For SPI: we don't care for now which CPU will receive the
          * interrupt
          * TODO: Handle case where SPI is setup on different CPU than
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 14:57:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 14:57:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715882.1117891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrs-0000rx-T9; Thu, 02 May 2024 14:57:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715882.1117891; Thu, 02 May 2024 14:57:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Xrs-0000rj-Ne; Thu, 02 May 2024 14:57:04 +0000
Received: by outflank-mailman (input) for mailman id 715882;
 Thu, 02 May 2024 14:57:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZceG=MF=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2Xrr-0008Nr-8V
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 14:57:03 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3b1d6603-0894-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 16:57:01 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a58fc650f8fso605804466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 07:57:01 -0700 (PDT)
Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 i15-20020a0564020f0f00b005726dceb2easm621919eda.20.2024.05.02.07.56.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 07:56:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b1d6603-0894-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714661820; x=1715266620; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tQTO0VmCzQBRvUIku9qGGXKc6qxkFrBOGAl6XQBbOfE=;
        b=kmzsatuf3NFlobnQhvJQf+jKiFt7LWHnzVlQYrc6wX+X1IA8eSY9J0om6NDAAlDfPp
         cY8FWjnXliyoYy/uO86p35EMfPYo5RcxMW7NC+rF9W04VB8biMxHEdtFd8H01UkcQeji
         iuYv4j9ZBJLZAtVvD55IRv2twxcHv4FShIlwnlt/ZxecZJHvO/E5hUPCE9vVbJoosql9
         EXtRGJDDZnZ+dj3AQQi/dv9ELHFfo4aYJMsbuhTO/Tvxoyy0c3C2Gx5qRfqq8eBZVrC8
         gg5EwPecUBnJZKFoh7+S3icjXxeuSLqkMDiOIIYi6XPUL5oUpQGXL6W7ZGvhrcftc/Tq
         w2hQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714661820; x=1715266620;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=tQTO0VmCzQBRvUIku9qGGXKc6qxkFrBOGAl6XQBbOfE=;
        b=b+6K/hPVRkPZMb/3viQB/PAnaXxSTW+lUHQpvDmeaHiiZnzCcBhl3GJo7mlomxmkHg
         ilZw/l7AjCPrcNRIicbO+SiJJAXyqoPgfqpAstR1ZjG+MhakaAsYeP9vnVCqjiRWgsSu
         37yAt3qwrGfJbfz7/67X2eFlErt6haAnn1zMHaIcKO4ZEghv7c4hKmudea+JVToYJ79z
         bMFw4BBVouhh4gE+qTRt3yFuyMD5rooboWik2D+AqYJqImrmMfq2j1YIHi2abztIEUDC
         GCQ6QBNuK4NZ26LxsSA8oJHWuKx/qkz6tCUk7znJrKkjYBQg4vJSFJjzbwq7GP4/YQ/v
         b8wQ==
X-Gm-Message-State: AOJu0YyDljsSGcDlQawdHM6K8RMk+2kVENo18ROg2pIbFMWMyIxRRIcL
	2JZ7JsDjgs2sMaXLyj6K4VAPYfKI4Cef5rHxM8ssmuwcBF0AGpUZoy/bDddJs/JxcdF3reI+9N1
	v
X-Google-Smtp-Source: AGHT+IFsNR70vSdd2PjYgE6U2rZ1n12drFTrAkl8y4fktjwlIzaPQvCVbuCa7n6gX7Vb8Y9aOKfY5A==
X-Received: by 2002:a50:951c:0:b0:56e:232b:95cd with SMTP id u28-20020a50951c000000b0056e232b95cdmr3468965eda.41.1714661820005;
        Thu, 02 May 2024 07:57:00 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v4 5/5] xen/arm: ffa: support notification
Date: Thu,  2 May 2024 16:56:45 +0200
Message-Id: <20240502145645.1201613-6-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502145645.1201613-1-jens.wiklander@linaro.org>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add support for FF-A notifications, currently limited to an SP (Secure
Partition) sending an asynchronous notification to a guest.

Guests and Xen itself are made aware of pending notifications with an
interrupt. The interrupt handler retrieves the notifications using the
FF-A ABI and deliver them to their destinations.

Update ffa_partinfo_domain_init() to return error code like
ffa_notif_domain_init().

Initialize FF-A with ffa_init() registered in presmp_initcall().
ffa_probe() just returns the probed version which is 0 if initialization
failed. Early FF-A initialization is needed to install the interrupt
handler on each CPU through a function registered with
register_cpu_notifier().

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
v3->v4:
- Add another note on FF-A limitations
- Clear secure_pending in ffa_handle_notification_get() if both SP and SPM
  bitmaps are retrieved
- ASSERT that ffa_rcu_lock_domain_by_vm_id() isn't passed the vm_id FF-A
  uses for Xen itself
- Replace the get_domain_by_id() call done via ffa_get_domain_by_vm_id() in
  notif_irq_handler() with a call to rcu_lock_live_remote_domain_by_id() via
  ffa_rcu_lock_domain_by_vm_id()
- Remove spinlock in struct ffa_ctx_notif and use atomic functions as needed
  to access and update the secure_pending field
- In notif_irq_handler(), look for the first online CPU instead of assuming
  that the first CPU is online
- Initialize FF-A via presmp_initcall() before the other CPUs are online,
  use register_cpu_notifier() to install the interrupt handler
  notif_irq_handler()
- Update commit message to reflect recent updates

v2->v3:
- Add a GUEST_ prefix and move FFA_NOTIF_PEND_INTR_ID and
  FFA_SCHEDULE_RECV_INTR_ID to public/arch-arm.h
- Register the Xen SRI handler on each CPU using on_selected_cpus() and
  setup_irq()
- Check that the SGI ID retrieved with FFA_FEATURE_SCHEDULE_RECV_INTR
  doesn't conflict with static SGI handlers

v1->v2:
- Addressing review comments
- Change ffa_handle_notification_{bind,unbind,set}() to take struct
  cpu_user_regs *regs as argument.
- Update ffa_partinfo_domain_init() and ffa_notif_domain_init() to return
  an error code.
- Fixing a bug in handle_features() for FFA_FEATURE_SCHEDULE_RECV_INTR.
---
 xen/arch/arm/tee/Makefile       |   1 +
 xen/arch/arm/tee/ffa.c          |  80 +++++-
 xen/arch/arm/tee/ffa_notif.c    | 453 ++++++++++++++++++++++++++++++++
 xen/arch/arm/tee/ffa_partinfo.c |   9 +-
 xen/arch/arm/tee/ffa_private.h  |  60 ++++-
 xen/include/public/arch-arm.h   |  14 +
 6 files changed, 602 insertions(+), 15 deletions(-)
 create mode 100644 xen/arch/arm/tee/ffa_notif.c

diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
index f0112a2f922d..7c0f46f7f446 100644
--- a/xen/arch/arm/tee/Makefile
+++ b/xen/arch/arm/tee/Makefile
@@ -2,5 +2,6 @@ obj-$(CONFIG_FFA) += ffa.o
 obj-$(CONFIG_FFA) += ffa_shm.o
 obj-$(CONFIG_FFA) += ffa_partinfo.o
 obj-$(CONFIG_FFA) += ffa_rxtx.o
+obj-$(CONFIG_FFA) += ffa_notif.o
 obj-y += tee.o
 obj-$(CONFIG_OPTEE) += optee.o
diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 5209612963e1..0dc0db9bfca8 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -39,6 +39,12 @@
  *   - at most 32 shared memory regions per guest
  * o FFA_MSG_SEND_DIRECT_REQ:
  *   - only supported from a VM to an SP
+ * o FFA_NOTIFICATION_*:
+ *   - only supports global notifications, that is, per vCPU notifications
+ *     are not supported
+ *   - doesn't support signalling the secondary scheduler of pending
+ *     notification for secure partitions
+ *   - doesn't support notifications for Xen itself
  *
  * There are some large locked sections with ffa_tx_buffer_lock and
  * ffa_rx_buffer_lock. Especially the ffa_tx_buffer_lock spinlock used
@@ -194,6 +200,8 @@ out:
 
 static void handle_features(struct cpu_user_regs *regs)
 {
+    struct domain *d = current->domain;
+    struct ffa_ctx *ctx = d->arch.tee;
     uint32_t a1 = get_user_reg(regs, 1);
     unsigned int n;
 
@@ -240,6 +248,30 @@ static void handle_features(struct cpu_user_regs *regs)
         BUILD_BUG_ON(PAGE_SIZE != FFA_PAGE_SIZE);
         ffa_set_regs_success(regs, 0, 0);
         break;
+    case FFA_FEATURE_NOTIF_PEND_INTR:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, GUEST_FFA_NOTIF_PEND_INTR_ID, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
+    case FFA_FEATURE_SCHEDULE_RECV_INTR:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, GUEST_FFA_SCHEDULE_RECV_INTR_ID, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
+
+    case FFA_NOTIFICATION_BIND:
+    case FFA_NOTIFICATION_UNBIND:
+    case FFA_NOTIFICATION_GET:
+    case FFA_NOTIFICATION_SET:
+    case FFA_NOTIFICATION_INFO_GET_32:
+    case FFA_NOTIFICATION_INFO_GET_64:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, 0, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
     default:
         ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
         break;
@@ -305,6 +337,22 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
                                                      get_user_reg(regs, 1)),
                                    get_user_reg(regs, 3));
         break;
+    case FFA_NOTIFICATION_BIND:
+        e = ffa_handle_notification_bind(regs);
+        break;
+    case FFA_NOTIFICATION_UNBIND:
+        e = ffa_handle_notification_unbind(regs);
+        break;
+    case FFA_NOTIFICATION_INFO_GET_32:
+    case FFA_NOTIFICATION_INFO_GET_64:
+        ffa_handle_notification_info_get(regs);
+        return true;
+    case FFA_NOTIFICATION_GET:
+        ffa_handle_notification_get(regs);
+        return true;
+    case FFA_NOTIFICATION_SET:
+        e = ffa_handle_notification_set(regs);
+        break;
 
     default:
         gprintk(XENLOG_ERR, "ffa: unhandled fid 0x%x\n", fid);
@@ -322,6 +370,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
 static int ffa_domain_init(struct domain *d)
 {
     struct ffa_ctx *ctx;
+    int ret;
 
     if ( !ffa_version )
         return -ENODEV;
@@ -345,10 +394,11 @@ static int ffa_domain_init(struct domain *d)
      * error, so no need for cleanup in this function.
      */
 
-    if ( !ffa_partinfo_domain_init(d) )
-        return -EIO;
+    ret = ffa_partinfo_domain_init(d);
+    if ( ret )
+        return ret;
 
-    return 0;
+    return ffa_notif_domain_init(d);
 }
 
 static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_time)
@@ -423,6 +473,7 @@ static int ffa_domain_teardown(struct domain *d)
         return 0;
 
     ffa_rxtx_domain_destroy(d);
+    ffa_notif_domain_destroy(d);
 
     ffa_domain_teardown_continue(ctx, true /* first_time */);
 
@@ -434,7 +485,7 @@ static int ffa_relinquish_resources(struct domain *d)
     return 0;
 }
 
-static bool ffa_probe(void)
+static int __init ffa_init(void)
 {
     uint32_t vers;
     unsigned int major_vers;
@@ -460,16 +511,16 @@ static bool ffa_probe(void)
         printk(XENLOG_ERR
                "ffa: unsupported SMCCC version %#x (need at least %#x)\n",
                smccc_ver, ARM_SMCCC_VERSION_1_2);
-        return false;
+        return 0;
     }
 
     if ( !ffa_get_version(&vers) )
-        return false;
+        return 0;
 
     if ( vers < FFA_MIN_SPMC_VERSION || vers > FFA_MY_VERSION )
     {
         printk(XENLOG_ERR "ffa: Incompatible version %#x found\n", vers);
-        return false;
+        return 0;
     }
 
     major_vers = (vers >> FFA_VERSION_MAJOR_SHIFT) & FFA_VERSION_MAJOR_MASK;
@@ -492,26 +543,33 @@ static bool ffa_probe(void)
          !check_mandatory_feature(FFA_MEM_SHARE_32) ||
          !check_mandatory_feature(FFA_MEM_RECLAIM) ||
          !check_mandatory_feature(FFA_MSG_SEND_DIRECT_REQ_32) )
-        return false;
+        return 0;
 
     if ( !ffa_rxtx_init() )
-        return false;
+        return 0;
 
     ffa_version = vers;
 
     if ( !ffa_partinfo_init() )
         goto err_rxtx_destroy;
 
+    ffa_notif_init();
     INIT_LIST_HEAD(&ffa_teardown_head);
     init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0);
 
-    return true;
+    return 0;
 
 err_rxtx_destroy:
     ffa_rxtx_destroy();
     ffa_version = 0;
 
-    return false;
+    return 0;
+}
+presmp_initcall(ffa_init);
+
+static bool ffa_probe(void)
+{
+    return ffa_version;
 }
 
 static const struct tee_mediator_ops ffa_ops =
diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c
new file mode 100644
index 000000000000..caf630ce2166
--- /dev/null
+++ b/xen/arch/arm/tee/ffa_notif.c
@@ -0,0 +1,453 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024  Linaro Limited
+ */
+
+#include <xen/const.h>
+#include <xen/cpu.h>
+#include <xen/list.h>
+#include <xen/notifier.h>
+#include <xen/spinlock.h>
+#include <xen/types.h>
+
+#include <asm/smccc.h>
+#include <asm/regs.h>
+
+#include "ffa_private.h"
+
+static bool __ro_after_init notif_enabled;
+static unsigned int __ro_after_init notif_sri_irq;
+
+int ffa_handle_notification_bind(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst & 0xFFFFU) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    if ( flags )    /* Only global notifications are supported */
+        return FFA_RET_DENIED;
+
+    /*
+     * We only support notifications from SP so no need to check the sender
+     * endpoint ID, the SPMC will take care of that for us.
+     */
+    return ffa_simple_call(FFA_NOTIFICATION_BIND, src_dst, flags, bitmap_hi,
+                           bitmap_lo);
+}
+
+int ffa_handle_notification_unbind(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst & 0xFFFFU) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    /*
+     * We only support notifications from SP so no need to check the
+     * destination endpoint ID, the SPMC will take care of that for us.
+     */
+    return  ffa_simple_call(FFA_NOTIFICATION_UNBIND, src_dst, 0, bitmap_hi,
+                            bitmap_lo);
+}
+
+void ffa_handle_notification_info_get(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( !notif_enabled )
+    {
+        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        return;
+    }
+
+    if ( test_and_clear_bool(ctx->notif.secure_pending) )
+    {
+        /* A pending global notification for the guest */
+        ffa_set_regs(regs, FFA_SUCCESS_64, 0,
+                     1U << FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT, ffa_get_vm_id(d),
+                     0, 0, 0, 0);
+    }
+    else
+    {
+        /* Report an error if there where no pending global notification */
+        ffa_set_regs_error(regs, FFA_RET_NO_DATA);
+    }
+}
+
+void ffa_handle_notification_get(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t recv = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t w2 = 0;
+    uint32_t w3 = 0;
+    uint32_t w4 = 0;
+    uint32_t w5 = 0;
+    uint32_t w6 = 0;
+    uint32_t w7 = 0;
+
+    if ( !notif_enabled )
+    {
+        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        return;
+    }
+
+    if ( (recv & 0xFFFFU) != ffa_get_vm_id(d) )
+    {
+        ffa_set_regs_error(regs, FFA_RET_INVALID_PARAMETERS);
+        return;
+    }
+
+    if ( flags & ( FFA_NOTIF_FLAG_BITMAP_SP | FFA_NOTIF_FLAG_BITMAP_SPM ) )
+    {
+        struct arm_smccc_1_2_regs arg = {
+            .a0 = FFA_NOTIFICATION_GET,
+            .a1 = recv,
+            .a2 = flags & ( FFA_NOTIF_FLAG_BITMAP_SP |
+                            FFA_NOTIF_FLAG_BITMAP_SPM ),
+        };
+        struct arm_smccc_1_2_regs resp;
+        int32_t e;
+
+        /*
+         * Clear secure pending if both FFA_NOTIF_FLAG_BITMAP_SP and
+         * FFA_NOTIF_FLAG_BITMAP_SPM are set since secure world can't have
+         * any more pending notifications.
+         */
+        if ( ( flags  & FFA_NOTIF_FLAG_BITMAP_SP ) &&
+             ( flags & FFA_NOTIF_FLAG_BITMAP_SPM ) )
+        {
+                struct ffa_ctx *ctx = d->arch.tee;
+
+                ACCESS_ONCE(ctx->notif.secure_pending) = false;
+        }
+
+        arm_smccc_1_2_smc(&arg, &resp);
+        e = ffa_get_ret_code(&resp);
+        if ( e )
+        {
+            ffa_set_regs_error(regs, e);
+            return;
+        }
+
+        if ( flags & FFA_NOTIF_FLAG_BITMAP_SP )
+        {
+            w2 = resp.a2;
+            w3 = resp.a3;
+        }
+
+        if ( flags & FFA_NOTIF_FLAG_BITMAP_SPM )
+            w6 = resp.a6;
+    }
+
+    ffa_set_regs(regs, FFA_SUCCESS_32, 0, w2, w3, w4, w5, w6, w7);
+}
+
+int ffa_handle_notification_set(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst >> 16) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    /* Let the SPMC check the destination of the notification */
+    return ffa_simple_call(FFA_NOTIFICATION_SET, src_dst, flags, bitmap_lo,
+                           bitmap_hi);
+}
+
+/*
+ * Extract a 16-bit ID (index n) from the successful return value from
+ * FFA_NOTIFICATION_INFO_GET_64 or FFA_NOTIFICATION_INFO_GET_32. IDs are
+ * returned in registers 3 to 7 with four IDs per register for 64-bit
+ * calling convention and two IDs per register for 32-bit calling
+ * convention.
+ */
+static uint16_t get_id_from_resp(struct arm_smccc_1_2_regs *resp,
+                                 unsigned int n)
+{
+    unsigned int ids_per_reg;
+    unsigned int reg_idx;
+    unsigned int reg_shift;
+
+    if ( smccc_is_conv_64(resp->a0) )
+        ids_per_reg = 4;
+    else
+        ids_per_reg = 2;
+
+    reg_idx = n / ids_per_reg + 3;
+    reg_shift = ( n % ids_per_reg ) * 16;
+
+    switch ( reg_idx )
+    {
+    case 3:
+        return resp->a3 >> reg_shift;
+    case 4:
+        return resp->a4 >> reg_shift;
+    case 5:
+        return resp->a5 >> reg_shift;
+    case 6:
+        return resp->a6 >> reg_shift;
+    case 7:
+        return resp->a7 >> reg_shift;
+    default:
+        ASSERT(0); /* "Can't happen" */
+        return 0;
+    }
+}
+
+static void notif_irq_handler(int irq, void *data)
+{
+    const struct arm_smccc_1_2_regs arg = {
+        .a0 = FFA_NOTIFICATION_INFO_GET_64,
+    };
+    struct arm_smccc_1_2_regs resp;
+    unsigned int id_pos;
+    unsigned int list_count;
+    uint64_t ids_count;
+    unsigned int n;
+    int32_t res;
+
+    do {
+        arm_smccc_1_2_smc(&arg, &resp);
+        res = ffa_get_ret_code(&resp);
+        if ( res )
+        {
+            if ( res != FFA_RET_NO_DATA )
+                printk(XENLOG_ERR "ffa: notification info get failed: error %d\n",
+                       res);
+            return;
+        }
+
+        ids_count = resp.a2 >> FFA_NOTIF_INFO_GET_ID_LIST_SHIFT;
+        list_count = ( resp.a2 >> FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT ) &
+                     FFA_NOTIF_INFO_GET_ID_COUNT_MASK;
+
+        id_pos = 0;
+        for ( n = 0; n < list_count; n++ )
+        {
+            unsigned int count = ((ids_count >> 2 * n) & 0x3) + 1;
+            uint16_t vm_id = get_id_from_resp(&resp, id_pos);
+            struct domain *d;
+
+            /*
+             * vm_id == 0 means a notifications pending for Xen itself, but
+             * we don't support that yet.
+             */
+            if (vm_id)
+                d = ffa_rcu_lock_domain_by_vm_id(vm_id);
+            else
+                d = NULL;
+
+            if ( d )
+            {
+                struct ffa_ctx *ctx = d->arch.tee;
+                struct vcpu *v;
+
+                ACCESS_ONCE(ctx->notif.secure_pending) = true;
+
+                /*
+                 * Since we're only delivering global notification, always
+                 * deliver to the first online vCPU. It doesn't matter
+                 * which we chose, as long as it's available.
+                 */
+                for_each_vcpu(d, v)
+                {
+                    if ( is_vcpu_online(v) )
+                    {
+                        vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INTR_ID,
+                                        true);
+                        break;
+                    }
+                }
+                if ( !v )
+                    printk(XENLOG_ERR "ffa: can't inject NPI, all vCPUs offline\n");
+
+                rcu_unlock_domain(d);
+            }
+
+            id_pos += count;
+        }
+
+    } while (resp.a2 & FFA_NOTIF_INFO_GET_MORE_FLAG);
+}
+
+static int32_t ffa_notification_bitmap_create(uint16_t vm_id,
+                                              uint32_t vcpu_count)
+{
+    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_CREATE, vm_id, vcpu_count,
+                           0, 0);
+}
+
+static int32_t ffa_notification_bitmap_destroy(uint16_t vm_id)
+{
+    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0, 0);
+}
+
+struct notif_irq_info {
+    bool called;
+    struct irqaction *action;
+};
+
+static void notif_irq_enable(void *info)
+{
+    struct notif_irq_info *irq_info = info;
+    int ret;
+
+    ret = setup_irq(notif_sri_irq, 0, irq_info->action);
+    if ( ret )
+        printk(XENLOG_ERR "ffa: setup_irq %u failed: error %d\n",
+               notif_sri_irq, ret);
+    else
+        irq_info->action = NULL;
+    irq_info->called = true;
+}
+
+static int ffa_setup_irq_callback(struct notifier_block *nfb,
+                                  unsigned long action, void *hcpu)
+{
+    unsigned int cpu = (unsigned long)hcpu;
+    struct notif_irq_info irq_info = { };
+
+    switch ( action )
+    {
+    case CPU_ONLINE:
+        /*
+         * The notifier call is done on the primary or initiating CPU when
+         * the target CPU have come online, but the SGI must be setup on
+         * the target CPU.
+         *
+         * We make an IPI call to the target CPU to setup the SGI. The call
+         * is executed in interrupt context on the target CPU, so we can't
+         * call request_irq() directly since it allocates memory.
+         *
+         * We preallocate the needed irqaction here and pass it via the
+         * temporary struct notif_irq_info. The call is synchronous in the
+         * sense that when on_selected_cpus() returns the callback
+         * notif_irq_enable() has done the same on the target CPU.
+         *
+         * We deal with two errors, one where notif_irq_enable() hasn't
+         * been called for some reason, that error is logged below. The
+         * other where setup_irq() fails is logged in the callback. We must
+         * free the irqaction in both cases since it has failed to become
+         * registered.
+         *
+         * Failures leads to a problem notifications, the CPUs with failure
+         * will trigger on the SGI indicating that there are notifications
+         * pending, while the SPMC in the secure world will not notice that
+         * the interrupt was lost.
+         */
+        irq_info.action = xmalloc(struct irqaction);
+        if ( !irq_info.action )
+        {
+            printk(XENLOG_ERR "ffa: setup irq %u failed, out of memory\n",
+                   notif_sri_irq);
+            return -ENOMEM;
+        }
+
+        *irq_info.action = (struct irqaction){
+            .handler = notif_irq_handler,
+            .name = "FF-A notif",
+            .free_on_release = 1,
+        };
+
+        on_selected_cpus(cpumask_of(cpu), notif_irq_enable, &irq_info, 1);
+        if (!irq_info.called)
+            printk(XENLOG_ERR "ffa: on_selected_cpus(cpumask_of(%u)) failed\n",
+                   cpu);
+        /*
+         * The irqaction is unused and must be freed if irq_info.action is
+         * non-NULL at this stage.
+         */
+        XFREE(irq_info.action);
+        break;
+    default:
+        break;
+    }
+
+    return 0;
+}
+
+static struct notifier_block ffa_request_irq_nfb = {
+    .notifier_call = ffa_setup_irq_callback,
+};
+
+void ffa_notif_init(void)
+{
+    const struct arm_smccc_1_2_regs arg = {
+        .a0 = FFA_FEATURES,
+        .a1 = FFA_FEATURE_SCHEDULE_RECV_INTR,
+    };
+    struct arm_smccc_1_2_regs resp;
+    unsigned int irq;
+    int ret;
+
+    arm_smccc_1_2_smc(&arg, &resp);
+    if ( resp.a0 != FFA_SUCCESS_32 )
+        return;
+
+    irq = resp.a2;
+    notif_sri_irq = irq;
+    if ( irq >= NR_GIC_SGI )
+        irq_set_type(irq, IRQ_TYPE_EDGE_RISING);
+    ret = request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL);
+    if ( ret )
+    {
+        printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n",
+               irq, ret);
+        return;
+    }
+
+    notif_enabled = true;
+    if ( irq < NR_GIC_SGI )
+        register_cpu_notifier(&ffa_request_irq_nfb);
+}
+
+int ffa_notif_domain_init(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+    int32_t res;
+
+    if ( !notif_enabled )
+        return 0;
+
+    res = ffa_notification_bitmap_create(ffa_get_vm_id(d), d->max_vcpus);
+    if ( res )
+        return -ENOMEM;
+
+    ctx->notif.enabled = true;
+
+    return 0;
+}
+
+void ffa_notif_domain_destroy(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( ctx->notif.enabled )
+    {
+        ffa_notification_bitmap_destroy(ffa_get_vm_id(d));
+        ctx->notif.enabled = false;
+    }
+}
diff --git a/xen/arch/arm/tee/ffa_partinfo.c b/xen/arch/arm/tee/ffa_partinfo.c
index dc1059584828..93a03c6bc672 100644
--- a/xen/arch/arm/tee/ffa_partinfo.c
+++ b/xen/arch/arm/tee/ffa_partinfo.c
@@ -306,7 +306,7 @@ static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
     }
 }
 
-bool ffa_partinfo_domain_init(struct domain *d)
+int ffa_partinfo_domain_init(struct domain *d)
 {
     unsigned int count = BITS_TO_LONGS(subscr_vm_destroyed_count);
     struct ffa_ctx *ctx = d->arch.tee;
@@ -315,7 +315,7 @@ bool ffa_partinfo_domain_init(struct domain *d)
 
     ctx->vm_destroy_bitmap = xzalloc_array(unsigned long, count);
     if ( !ctx->vm_destroy_bitmap )
-        return false;
+        return -ENOMEM;
 
     for ( n = 0; n < subscr_vm_created_count; n++ )
     {
@@ -330,7 +330,10 @@ bool ffa_partinfo_domain_init(struct domain *d)
     }
     vm_destroy_bitmap_init(ctx, n);
 
-    return n == subscr_vm_created_count;
+    if ( n != subscr_vm_created_count )
+        return -EIO;
+
+    return 0;
 }
 
 bool ffa_partinfo_domain_destroy(struct domain *d)
diff --git a/xen/arch/arm/tee/ffa_private.h b/xen/arch/arm/tee/ffa_private.h
index 98236cbf14a3..2f786eac6bfc 100644
--- a/xen/arch/arm/tee/ffa_private.h
+++ b/xen/arch/arm/tee/ffa_private.h
@@ -25,6 +25,7 @@
 #define FFA_RET_DENIED                  -6
 #define FFA_RET_RETRY                   -7
 #define FFA_RET_ABORTED                 -8
+#define FFA_RET_NO_DATA                 -9
 
 /* FFA_VERSION helpers */
 #define FFA_VERSION_MAJOR_SHIFT         16U
@@ -175,6 +176,21 @@
  */
 #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U)
 
+/* Flags used in calls to FFA_NOTIFICATION_GET interface  */
+#define FFA_NOTIF_FLAG_BITMAP_SP        BIT(0, U)
+#define FFA_NOTIF_FLAG_BITMAP_VM        BIT(1, U)
+#define FFA_NOTIF_FLAG_BITMAP_SPM       BIT(2, U)
+#define FFA_NOTIF_FLAG_BITMAP_HYP       BIT(3, U)
+
+#define FFA_NOTIF_INFO_GET_MORE_FLAG        BIT(0, U)
+#define FFA_NOTIF_INFO_GET_ID_LIST_SHIFT    12
+#define FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT   7
+#define FFA_NOTIF_INFO_GET_ID_COUNT_MASK    0x1F
+
+/* Feature IDs used with FFA_FEATURES */
+#define FFA_FEATURE_NOTIF_PEND_INTR     0x1U
+#define FFA_FEATURE_SCHEDULE_RECV_INTR  0x2U
+
 /* Function IDs */
 #define FFA_ERROR                       0x84000060U
 #define FFA_SUCCESS_32                  0x84000061U
@@ -213,6 +229,24 @@
 #define FFA_MEM_FRAG_TX                 0x8400007BU
 #define FFA_MSG_SEND                    0x8400006EU
 #define FFA_MSG_POLL                    0x8400006AU
+#define FFA_NOTIFICATION_BITMAP_CREATE  0x8400007DU
+#define FFA_NOTIFICATION_BITMAP_DESTROY 0x8400007EU
+#define FFA_NOTIFICATION_BIND           0x8400007FU
+#define FFA_NOTIFICATION_UNBIND         0x84000080U
+#define FFA_NOTIFICATION_SET            0x84000081U
+#define FFA_NOTIFICATION_GET            0x84000082U
+#define FFA_NOTIFICATION_INFO_GET_32    0x84000083U
+#define FFA_NOTIFICATION_INFO_GET_64    0xC4000083U
+
+struct ffa_ctx_notif {
+    bool enabled;
+
+    /*
+     * True if domain is reported by FFA_NOTIFICATION_INFO_GET to have
+     * pending global notifications.
+     */
+    bool secure_pending;
+};
 
 struct ffa_ctx {
     void *rx;
@@ -228,6 +262,7 @@ struct ffa_ctx {
     struct list_head shm_list;
     /* Number of allocated shared memory object */
     unsigned int shm_count;
+    struct ffa_ctx_notif notif;
     /*
      * tx_lock is used to serialize access to tx
      * rx_lock is used to serialize access to rx
@@ -257,7 +292,7 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs);
 int ffa_handle_mem_reclaim(uint64_t handle, uint32_t flags);
 
 bool ffa_partinfo_init(void);
-bool ffa_partinfo_domain_init(struct domain *d);
+int ffa_partinfo_domain_init(struct domain *d);
 bool ffa_partinfo_domain_destroy(struct domain *d);
 int32_t ffa_handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,
                                       uint32_t w4, uint32_t w5, uint32_t *count,
@@ -271,12 +306,35 @@ uint32_t ffa_handle_rxtx_map(uint32_t fid, register_t tx_addr,
 uint32_t ffa_handle_rxtx_unmap(void);
 int32_t ffa_handle_rx_release(void);
 
+void ffa_notif_init(void);
+int ffa_notif_domain_init(struct domain *d);
+void ffa_notif_domain_destroy(struct domain *d);
+
+int ffa_handle_notification_bind(struct cpu_user_regs *regs);
+int ffa_handle_notification_unbind(struct cpu_user_regs *regs);
+void ffa_handle_notification_info_get(struct cpu_user_regs *regs);
+void ffa_handle_notification_get(struct cpu_user_regs *regs);
+int ffa_handle_notification_set(struct cpu_user_regs *regs);
+
 static inline uint16_t ffa_get_vm_id(const struct domain *d)
 {
     /* +1 since 0 is reserved for the hypervisor in FF-A */
     return d->domain_id + 1;
 }
 
+static inline struct domain *ffa_rcu_lock_domain_by_vm_id(uint16_t vm_id)
+{
+    struct domain *d;
+
+    ASSERT(vm_id);
+
+    /* -1 to match ffa_get_vm_id() */
+    if ( rcu_lock_live_remote_domain_by_id(vm_id - 1, &d) )
+        return NULL;
+
+    return d;
+}
+
 static inline void ffa_set_regs(struct cpu_user_regs *regs, register_t v0,
                                 register_t v1, register_t v2, register_t v3,
                                 register_t v4, register_t v5, register_t v6,
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 289af81bd69d..e2412a17474e 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -505,6 +505,7 @@ typedef uint64_t xen_callback_t;
 #define GUEST_MAX_VCPUS 128
 
 /* Interrupts */
+
 #define GUEST_TIMER_VIRT_PPI    27
 #define GUEST_TIMER_PHYS_S_PPI  29
 #define GUEST_TIMER_PHYS_NS_PPI 30
@@ -515,6 +516,19 @@ typedef uint64_t xen_callback_t;
 #define GUEST_VIRTIO_MMIO_SPI_FIRST   33
 #define GUEST_VIRTIO_MMIO_SPI_LAST    43
 
+/*
+ * SGI is the preferred delivery mechanism of FF-A pending notifications or
+ * schedule recveive interrupt. SGIs 8-15 are normally not used by a guest
+ * as they in a non-virtualized system typically are assigned to the secure
+ * world. Here we're free to use SGI 8-15 since they are virtual and have
+ * nothing to do with the secure world.
+ *
+ * For partitioning of SGIs see also Arm Base System Architecture v1.0C,
+ * https://developer.arm.com/documentation/den0094/
+ */
+#define GUEST_FFA_NOTIF_PEND_INTR_ID      8
+#define GUEST_FFA_SCHEDULE_RECV_INTR_ID   9
+
 /* PSCI functions */
 #define PSCI_cpu_suspend 0
 #define PSCI_cpu_off     1
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 15:59:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 15:59:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715922.1117901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2YqR-000511-HX; Thu, 02 May 2024 15:59:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715922.1117901; Thu, 02 May 2024 15:59:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2YqR-00050u-EL; Thu, 02 May 2024 15:59:39 +0000
Received: by outflank-mailman (input) for mailman id 715922;
 Thu, 02 May 2024 15:59:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2YqQ-00050k-WD; Thu, 02 May 2024 15:59:39 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2YqQ-0006CU-Q0; Thu, 02 May 2024 15:59:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2YqQ-0004RR-Cc; Thu, 02 May 2024 15:59:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2YqQ-0007QS-C9; Thu, 02 May 2024 15:59:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=vMCflJTLkbaneEy+eL+VtmHm92MO7ygpHd6IoWE7pxo=; b=cYfq4+LPPxDUc1FVweazoyekmW
	vD3fN1AkrsUtPUT7OdnUGt2D2eiUIFbUiNX9pz+CAPVVs4fJcXE3JuFj9XFnFs6eUcrg1x3ChZAG+
	k6Rz957QvYWt39QWBtrpDX2TTZI8DvE5Z1RXQi9nR1Fd9IEItqVGDmW7owLJ3bMpbMzQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185899-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185899: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0106679839f7c69632b3b9833c3268c316c0a9fc
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 15:59:38 +0000

flight 185899 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185899/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail in 185896 pass in 185899
 test-armhf-armhf-xl-credit2   8 xen-boot         fail in 185896 pass in 185899
 test-armhf-armhf-xl-credit1   8 xen-boot         fail in 185896 pass in 185899
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185896
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185896

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185870
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 185896 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 185896 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0106679839f7c69632b3b9833c3268c316c0a9fc
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    2 days
Failing since        185888  2024-04-30 20:31:50 Z    1 days    4 attempts
Testing same since   185896  2024-05-01 23:13:02 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Potapenko <glider@google.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Audra Mitchell <audra@redhat.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Chuck Lever <chuck.lever@oracle.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Hans de Goede <hdegoede@redhat.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  John Garry <john.g.garry@oracle.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Sven Schnelle <svens@linux.ibm.com>
  Tejun Heo <tj@kernel.org>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Xingyou Chen <rockrush@rockwork.org>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1219 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 02 16:38:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:38:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715930.1117910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ZS5-0003Ak-Ck; Thu, 02 May 2024 16:38:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715930.1117910; Thu, 02 May 2024 16:38:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ZS5-0003Ad-AB; Thu, 02 May 2024 16:38:33 +0000
Received: by outflank-mailman (input) for mailman id 715930;
 Thu, 02 May 2024 16:38:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2ZS3-0003AX-3T
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:38:31 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67725715-08a2-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:38:28 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a58fbbcd77aso661045966b.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:38:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67725715-08a2-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714667908; x=1715272708; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tdq2b0nhg5nyul35h5bkMt4asw7wNg6ha8u+AmWjAGc=;
        b=h4/712AiA/3B8ZcDl7nhljMtOcIxywN/YD6+E2GMJQoFXwIbrt3Y+okDc9+mj1/jpX
         yGurqgCC78t3ll42ULg6Q5ztrbbrRvIFJDwbl8dkEkR2BRf7JmiE6dmvyW3Skklv5umf
         QxZqg2tSUjSyTZKwZuk+iO1WhLhVA1e99v8vWyo3LWFcMR9NPOeBjgQon/9Eb8XAwMCN
         vcgZJHTEjRh+4xiYCvb8o39DpXYIEPZ9mrkB/MA2waqqjmcYPWgub5ULwKNvk+njauM0
         f/FLGKWrud+kDKjG84ESHCpfG0+lvDgK4ZdWlSeSlk7yU9C8PEM+manoI9ZKEjytIdDn
         Uxmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714667908; x=1715272708;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Tdq2b0nhg5nyul35h5bkMt4asw7wNg6ha8u+AmWjAGc=;
        b=pl9aDnhouetGbzsxj48k8jRyuY9KcYvgqHojbJNDpZAP9Dd60xTo0Mr1seWChSMEj2
         5140Hx+osRRWX5IM/Com3jVJfpZYhvkrp4WQHP7zaCIEckxJpW3ZjdPEgus+hmDGoB/T
         HoDqcU6afWRthDVnW1no79Q1Y4soAcn8cCAG6H+HjRRnNJ4HHgzDuVxP2JUlRF7GyR9M
         w5esaRZYrefvtYFDBhKaa1iioA8MWBSe+DkYQoIDlzJL2ls9HcN7q+R7I3Z1T6gsi5H5
         4J9+DrrnIozLn0ruMv+UgRhHD+Pwb5qbRvaIBuskpFaBQSeh2Fmk5vm/1qRCwG0yVT6h
         Rd6Q==
X-Gm-Message-State: AOJu0YzxWpHeGymGoMflk7G3xSVaVMvhf+uEDdcshPWlU2/PIeTbZ6KJ
	MuwwGnZFvmToKuIZvH7/htN8We1S3+2ZppdDu2ZlpAmHth9K+NW/QA3+7mAQLYRhqs4s+rH6q14
	pAAqSkzzQvuDeVCsR+HCxB3HOH1w=
X-Google-Smtp-Source: AGHT+IFVcTsj3sgUTVuf4RSliWYI8UVluVED1hKF9MWgodAcHSq6Usexhfi2jq/y16/YXoYU/oMdHCGBhNsUpjXhbR4=
X-Received: by 2002:a50:8e4f:0:b0:570:5214:f62 with SMTP id
 15-20020a508e4f000000b0057052140f62mr22737edx.0.1714667907978; Thu, 02 May
 2024 09:38:27 -0700 (PDT)
MIME-Version: 1.0
References: <20240415231541.4140052-1-edgar.iglesias@gmail.com>
 <20240415231541.4140052-2-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2404251608320.3940@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2404251608320.3940@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 18:38:16 +0200
Message-ID: <CAJy5ezoyCarP28SdXwArMM4d7AALPAURERM-c2fDTxu1Am3tNQ@mail.gmail.com>
Subject: Re: [PATCH v2 1/1] xen/arm64: entry: Add missing code symbol annotations
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, bertrand.marquis@arm.com, 
	michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, jbeulich@suse.com, 
	andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Apr 26, 2024 at 1:14=E2=80=AFAM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Tue, 16 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Use the generic xen/linkage.h macros when and add missing
>                                         ^ when what?
>
> > code symbol annotations.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>
> I am looking at the implementation of FUNC and as far as I can tell
> there is no change compared to ENTRY. So from that point of view we are
> good. I wonder if we should keep using "ENTRY" because it is nice to
> mark explicitely the entry points as such but at the same time I am also
> OK with this. I'll let the other ARM maintainers decide.
>
> On the other hand, FUNC_LOCAL does introduce a change: it adds a .align
> everywhere. Should not be harmful?
>
> With the commit message fixed:

Thanks Stefano, will fix the commit message in v3:

 "Use the generic xen/linkage.h macros to annotate code symbols
 and add missing annotations."

Cheers,
Edgar



>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>
>
> > ---
> >  xen/arch/arm/arm64/entry.S | 72 +++++++++++++++++++++++++-------------
> >  1 file changed, 48 insertions(+), 24 deletions(-)
> >
> > diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
> > index f963c923bb..af9a592cae 100644
> > --- a/xen/arch/arm/arm64/entry.S
> > +++ b/xen/arch/arm/arm64/entry.S
> > @@ -289,21 +289,25 @@
> >          b       do_bad_mode
> >          .endm
> >
> > -hyp_sync_invalid:
> > +FUNC_LOCAL(hyp_sync_invalid)
> >          entry   hyp=3D1
> >          invalid BAD_SYNC
> > +END(hyp_sync_invalid)
> >
> > -hyp_irq_invalid:
> > +FUNC_LOCAL(hyp_irq_invalid)
> >          entry   hyp=3D1
> >          invalid BAD_IRQ
> > +END(hyp_irq_invalid)
> >
> > -hyp_fiq_invalid:
> > +FUNC_LOCAL(hyp_fiq_invalid)
> >          entry   hyp=3D1
> >          invalid BAD_FIQ
> > +END(hyp_fiq_invalid)
> >
> > -hyp_error_invalid:
> > +FUNC_LOCAL(hyp_error_invalid)
> >          entry   hyp=3D1
> >          invalid BAD_ERROR
> > +END(hyp_error_invalid)
> >
> >  /*
> >   * SError received while running in the hypervisor mode.
> > @@ -313,11 +317,12 @@ hyp_error_invalid:
> >   * simplicity, as SError should be rare and potentially fatal,
> >   * all interrupts are kept masked.
> >   */
> > -hyp_error:
> > +FUNC_LOCAL(hyp_error)
> >          entry   hyp=3D1
> >          mov     x0, sp
> >          bl      do_trap_hyp_serror
> >          exit    hyp=3D1
> > +END(hyp_error)
> >
> >  /*
> >   * Synchronous exception received while running in the hypervisor mode=
.
> > @@ -327,7 +332,7 @@ hyp_error:
> >   * some of them. So we want to inherit the state from the interrupted
> >   * context.
> >   */
> > -hyp_sync:
> > +FUNC_LOCAL(hyp_sync)
> >          entry   hyp=3D1
> >
> >          /* Inherit interrupts */
> > @@ -338,6 +343,7 @@ hyp_sync:
> >          mov     x0, sp
> >          bl      do_trap_hyp_sync
> >          exit    hyp=3D1
> > +END(hyp_sync)
> >
> >  /*
> >   * IRQ received while running in the hypervisor mode.
> > @@ -352,7 +358,7 @@ hyp_sync:
> >   * would require some rework in some paths (e.g. panic, livepatch) to
> >   * ensure the ordering is enforced everywhere.
> >   */
> > -hyp_irq:
> > +FUNC_LOCAL(hyp_irq)
> >          entry   hyp=3D1
> >
> >          /* Inherit D, A, F interrupts and keep I masked */
> > @@ -365,8 +371,9 @@ hyp_irq:
> >          mov     x0, sp
> >          bl      do_trap_irq
> >          exit    hyp=3D1
> > +END(hyp_irq)
> >
> > -guest_sync:
> > +FUNC_LOCAL(guest_sync)
> >          /*
> >           * Save x0, x1 in advance
> >           */
> > @@ -413,8 +420,9 @@ fastpath_out_workaround:
> >          mov     x1, xzr
> >          eret
> >          sb
> > +END(guest_sync)
> >
> > -wa2_ssbd:
> > +FUNC_LOCAL(wa2_ssbd)
> >  #ifdef CONFIG_ARM_SSBD
> >  alternative_cb arm_enable_wa2_handling
> >          b       wa2_end
> > @@ -450,42 +458,55 @@ wa2_end:
> >          mov     x0, xzr
> >          eret
> >          sb
> > -guest_sync_slowpath:
> > +END(wa2_ssbd)
> > +
> > +FUNC_LOCAL(guest_sync_slowpath)
> >          /*
> >           * x0/x1 may have been scratch by the fast path above, so avoi=
d
> >           * to save them.
> >           */
> >          guest_vector compat=3D0, iflags=3DIFLAGS__AI_, trap=3Dguest_sy=
nc, save_x0_x1=3D0
> > +END(guest_sync_slowpath)
> >
> > -guest_irq:
> > +FUNC_LOCAL(guest_irq)
> >          guest_vector compat=3D0, iflags=3DIFLAGS__A__, trap=3Dirq
> > +END(guest_irq)
> >
> > -guest_fiq_invalid:
> > +FUNC_LOCAL(guest_fiq_invalid)
> >          entry   hyp=3D0, compat=3D0
> >          invalid BAD_FIQ
> > +END(guest_fiq_invalid)
> >
> > -guest_error:
> > +FUNC_LOCAL(guest_error)
> >          guest_vector compat=3D0, iflags=3DIFLAGS__AI_, trap=3Dguest_se=
rror
> > +END(guest_error)
> >
> > -guest_sync_compat:
> > +FUNC_LOCAL(guest_sync_compat)
> >          guest_vector compat=3D1, iflags=3DIFLAGS__AI_, trap=3Dguest_sy=
nc
> > +END(guest_sync_compat)
> >
> > -guest_irq_compat:
> > +FUNC_LOCAL(guest_irq_compat)
> >          guest_vector compat=3D1, iflags=3DIFLAGS__A__, trap=3Dirq
> > +END(guest_irq_compat)
> >
> > -guest_fiq_invalid_compat:
> > +FUNC_LOCAL(guest_fiq_invalid_compat)
> >          entry   hyp=3D0, compat=3D1
> >          invalid BAD_FIQ
> > +END(guest_fiq_invalid_compat)
> >
> > -guest_error_compat:
> > +FUNC_LOCAL(guest_error_compat)
> >          guest_vector compat=3D1, iflags=3DIFLAGS__AI_, trap=3Dguest_se=
rror
> > +END(guest_error_compat)
> >
> > -ENTRY(return_to_new_vcpu32)
> > +FUNC(return_to_new_vcpu32)
> >          exit    hyp=3D0, compat=3D1
> > -ENTRY(return_to_new_vcpu64)
> > +END(return_to_new_vcpu32)
> > +
> > +FUNC(return_to_new_vcpu64)
> >          exit    hyp=3D0, compat=3D0
> > +END(return_to_new_vcpu64)
> >
> > -return_from_trap:
> > +FUNC_LOCAL(return_from_trap)
> >          msr     daifset, #IFLAGS___I_ /* Mask interrupts */
> >
> >          ldr     x21, [sp, #UREGS_PC]            /* load ELR */
> > @@ -524,6 +545,7 @@ return_from_trap:
> >
> >          eret
> >          sb
> > +END(return_from_trap)
> >
> >  /*
> >   * Consume pending SError generated by the guest if any.
> > @@ -536,7 +558,7 @@ return_from_trap:
> >   * it. So the function will unmask SError exception for a small window=
 and
> >   * then mask it again.
> >   */
> > -check_pending_guest_serror:
> > +FUNC_LOCAL(check_pending_guest_serror)
> >          /*
> >           * Save elr_el2 to check whether the pending SError exception =
takes
> >           * place while we are doing this sync exception.
> > @@ -586,7 +608,7 @@ abort_guest_exit_end:
> >          cset    x19, ne
> >
> >          ret
> > -ENDPROC(check_pending_guest_serror)
> > +END(check_pending_guest_serror)
> >
> >  /*
> >   * Exception vectors.
> > @@ -597,7 +619,7 @@ ENDPROC(check_pending_guest_serror)
> >          .endm
> >
> >          .align  11
> > -ENTRY(hyp_traps_vector)
> > +FUNC(hyp_traps_vector)
> >          ventry  hyp_sync_invalid            /* Synchronous EL2t */
> >          ventry  hyp_irq_invalid             /* IRQ EL2t */
> >          ventry  hyp_fiq_invalid             /* FIQ EL2t */
> > @@ -617,6 +639,7 @@ ENTRY(hyp_traps_vector)
> >          ventry  guest_irq_compat            /* IRQ 32-bit EL0/EL1 */
> >          ventry  guest_fiq_invalid_compat    /* FIQ 32-bit EL0/EL1 */
> >          ventry  guest_error_compat          /* Error 32-bit EL0/EL1 */
> > +END(hyp_traps_vector)
> >
> >  /*
> >   * struct vcpu *__context_switch(struct vcpu *prev, struct vcpu *next)
> > @@ -626,7 +649,7 @@ ENTRY(hyp_traps_vector)
> >   *
> >   * Returns prev in x0
> >   */
> > -ENTRY(__context_switch)
> > +FUNC(__context_switch)
> >          add     x8, x0, #VCPU_arch_saved_context
> >          mov     x9, sp
> >          stp     x19, x20, [x8], #16         /* store callee-saved regi=
sters */
> > @@ -647,6 +670,7 @@ ENTRY(__context_switch)
> >          ldr     lr, [x8]
> >          mov     sp, x9
> >          ret
> > +END(__context_switch)
> >
> >  /*
> >   * Local variables:
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715949.1117952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zii-0006wL-C8; Thu, 02 May 2024 16:55:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715949.1117952; Thu, 02 May 2024 16:55:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zii-0006uI-3d; Thu, 02 May 2024 16:55:44 +0000
Received: by outflank-mailman (input) for mailman id 715949;
 Thu, 02 May 2024 16:55:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zig-0006Lf-F7
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:42 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cf1a9237-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:41 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2dd6a7ae2dcso134698611fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:41 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf1a9237-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668941; x=1715273741; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OQIt6EqAhVSChAJtwNddHFJtbh/THZTAMqTRnNcVJEs=;
        b=jSFX0KHkYBXIaoPeifOO/6mr6iU+PP9auXhL2gwBjQCjkh4dj+l/VxIPDamJtctki0
         D4FhyLWVzxaHTjTKF8aOjaZXtMtAfbZQLBB7rbuq4zCbn3EkqBYjYiHTlYjsBGlE3FCT
         LZ+zrXXx5vAU7AzQzbA2SXPdscKs0ZU3Va54675pHyKUimb0PGZ075GyIWAw2vzj9uca
         jKVKPyfftBh5usEXknMX64dpGH9jsNlTsbHqEVXOKA5E4fjAlfffGSdKykjnRpEGb+7+
         occ7+VgAvKJBg0/MidhOnLs94HidALAiQNzx3SujaGpjtncMfBDskKHPwg/iH/xgaksA
         Z/og==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668941; x=1715273741;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OQIt6EqAhVSChAJtwNddHFJtbh/THZTAMqTRnNcVJEs=;
        b=wzGJcA0WxZ30ABiRoECSJXLuE3/PC9NMoFO5f+elrL33U2Adk8jJt1PLpSYEmyT2sA
         BPiWb5qN67byBiQNyhQgN7iVIsFd+0EPGpQpM5rtsD0Emk7gA7bNafB9KacKXhO9MY0/
         ZFrayJ0O97fign5X3s7s2Ch0Nfbj5WvrZefW7bAJXWa6QoIeXZxwvYK6RtJmDcIqAjo/
         YLOh7yE8ElRdQOV0ozssoSWHVhU1bPwP6r4CpO2DJZHC3T5GsYQaJz0QGhOYEB32PKrf
         992lk4aR5Mq2iIwoWqqJL9KsxycIh7BnPpXzodadNpSZjsKJXcO7tgSUkBbVudHR2sru
         3wZw==
X-Gm-Message-State: AOJu0YzPvvuvmaOnMtbYLnXFaHsc0fZ+FACOuW9YsL1ILXX/+Qq5Qb8c
	FJh8JsZhWZTDwV1MEIbX69mZuhTooGXxwr0ht2WHiBuQcTkflxRUzI4U/lztED4KStTqsqsSMNt
	U
X-Google-Smtp-Source: AGHT+IFfTaHyfpFw1bJI3YbMS0k968SxK3ibfMEZ2lxT+qyoIgawCS1WBjyfcSqxwSdRD47+M/u78g==
X-Received: by 2002:a2e:b04a:0:b0:2df:b2d5:5935 with SMTP id d10-20020a2eb04a000000b002dfb2d55935mr287080ljl.28.1714668940688;
        Thu, 02 May 2024 09:55:40 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 04/13] xen/arm: add Dom0 cache coloring support
Date: Thu,  2 May 2024 18:55:24 +0200
Message-Id: <20240502165533.319988-5-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a command line parameter to allow the user to set the coloring
configuration for Dom0.
A common configuration syntax for cache colors is introduced and
documented.
Take the opportunity to also add:
 - default configuration notion.
 - function to check well-formed configurations.

Direct mapping Dom0 isn't possible when coloring is enabled, so
CDF_directmap flag is removed when creating it.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- added bound check on dom0_num_colors
- default colors array set just once
v7:
- parse_color_config() doesn't accept leading/trailing commas anymore
- removed alloc_colors() helper
v6:
- moved domain_llc_coloring_free() in this patch
- removed domain_alloc_colors() in favor of a more explicit allocation
- parse_color_config() now accepts the size of the array to be filled
- allocate_memory() moved in another patch
v5:
- Carlo Nonato as the new author
- moved dom0 colors parsing (parse_colors()) in this patch
- added dom0_set_llc_colors() to set dom0 colors after creation
- moved color allocation and checking in this patch
- error handling when allocating color arrays
- FIXME: copy pasted allocate_memory() cause it got moved
v4:
- dom0 colors are dynamically allocated as for any other domain
  (colors are duplicated in dom0_colors and in the new array, but logic
  is simpler)
---
 docs/misc/cache-coloring.rst      |  29 +++++++
 docs/misc/xen-command-line.pandoc |   9 +++
 xen/arch/arm/domain_build.c       |  10 ++-
 xen/common/domain.c               |   3 +
 xen/common/llc-coloring.c         | 128 +++++++++++++++++++++++++++++-
 xen/include/xen/llc-coloring.h    |   3 +
 6 files changed, 180 insertions(+), 2 deletions(-)

diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
index 404262b728..26b306a0ff 100644
--- a/docs/misc/cache-coloring.rst
+++ b/docs/misc/cache-coloring.rst
@@ -105,6 +105,35 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`.
 +----------------------+-------------------------------+
 | ``llc-nr-ways``      | Set the LLC number of ways    |
 +----------------------+-------------------------------+
+| ``dom0-llc-colors``  | Dom0 color configuration      |
++----------------------+-------------------------------+
+
+Colors selection format
+***********************
+
+Regardless of the memory pool that has to be colored (Xen, Dom0/DomUs),
+the color selection can be expressed using the same syntax. In particular a
+comma-separated list of colors or ranges of colors is used.
+Ranges are hyphen-separated intervals (such as `0-4`) and are inclusive on both
+sides.
+
+Note that:
+
+- no spaces are allowed between values.
+- no overlapping ranges or duplicated colors are allowed.
+- values must be written in ascending order.
+
+Examples:
+
++-------------------+-----------------------------+
+| **Configuration** | **Actual selection**        |
++-------------------+-----------------------------+
+| 1-2,5-8           | [1, 2, 5, 6, 7, 8]          |
++-------------------+-----------------------------+
+| 4-8,10,11,12      | [4, 5, 6, 7, 8, 10, 11, 12] |
++-------------------+-----------------------------+
+| 0                 | [0]                         |
++-------------------+-----------------------------+
 
 Auto-probing of LLC specs
 #########################
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 906822a491..2923e1fad4 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
 
 Specify a list of IO ports to be excluded from dom0 access.
 
+### dom0-llc-colors
+> `= List of [ <integer> | <integer>-<integer> ]`
+
+> Default: `All available LLC colors`
+
+Specify dom0 LLC color configuration. This option is available only when
+`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available
+colors are used.
+
 ### dom0_max_vcpus
 
 Either:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index ba2e48ecc7..c0ed07051d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2,6 +2,7 @@
 #include <xen/init.h>
 #include <xen/compile.h>
 #include <xen/lib.h>
+#include <xen/llc-coloring.h>
 #include <xen/mm.h>
 #include <xen/param.h>
 #include <xen/domain_page.h>
@@ -2250,6 +2251,7 @@ void __init create_dom0(void)
         .max_maptrack_frames = -1,
         .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
     };
+    unsigned int flags = CDF_privileged;
     int rc;
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
@@ -2277,10 +2279,16 @@ void __init create_dom0(void)
             panic("SVE vector length error\n");
     }
 
-    dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap);
+    if ( !llc_coloring_enabled )
+        flags |= CDF_directmap;
+
+    dom0 = domain_create(0, &dom0_cfg, flags);
     if ( IS_ERR(dom0) )
         panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
 
+    if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
+        panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc);
+
     if ( alloc_dom0_vcpu0(dom0) == NULL )
         panic("Error creating domain 0 vcpu0\n");
 
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6773f7fb90..a3cfb3bf37 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -33,6 +33,7 @@
 #include <xen/xenoprof.h>
 #include <xen/irq.h>
 #include <xen/argo.h>
+#include <xen/llc-coloring.h>
 #include <asm/p2m.h>
 #include <asm/processor.h>
 #include <public/sched.h>
@@ -1219,6 +1220,8 @@ void domain_destroy(struct domain *d)
 
     BUG_ON(!d->is_dying);
 
+    domain_llc_coloring_free(d);
+
     /* May be already destroyed, or get_domain() can race us. */
     if ( atomic_cmpxchg(&d->refcnt, 0, DOMAIN_DESTROYED) != 0 )
         return;
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index bcc651cc10..26270cda9c 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -19,6 +19,66 @@ static unsigned int __initdata llc_nr_ways;
 integer_param("llc-nr-ways", llc_nr_ways);
 /* Number of colors available in the LLC */
 static unsigned int __ro_after_init max_nr_colors;
+/* Default coloring configuration */
+static unsigned int __ro_after_init default_colors[NR_LLC_COLORS];
+
+static unsigned int __initdata dom0_colors[NR_LLC_COLORS];
+static unsigned int __initdata dom0_num_colors;
+
+/*
+ * Parse the coloring configuration given in the buf string, following the
+ * syntax below.
+ *
+ * COLOR_CONFIGURATION ::= COLOR | RANGE,...,COLOR | RANGE
+ * RANGE               ::= COLOR-COLOR
+ *
+ * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15,16.
+ */
+static int __init parse_color_config(const char *buf, unsigned int *colors,
+                                     unsigned int max_num_colors,
+                                     unsigned int *num_colors)
+{
+    const char *s = buf;
+
+    *num_colors = 0;
+
+    while ( *s != '\0' )
+    {
+        unsigned int color, start, end;
+
+        start = simple_strtoul(s, &s, 0);
+
+        if ( *s == '-' )    /* Range */
+        {
+            s++;
+            end = simple_strtoul(s, &s, 0);
+        }
+        else                /* Single value */
+            end = start;
+
+        if ( start > end || (end - start) > (UINT_MAX - *num_colors) ||
+             (*num_colors + (end - start)) >= max_num_colors )
+            return -EINVAL;
+
+        /* Colors are range checked in check_colors() */
+        for ( color = start; color <= end; color++ )
+            colors[(*num_colors)++] = color;
+
+        if ( *s == ',' )
+            s++;
+        else if ( *s != '\0' )
+            break;
+    }
+
+    return *s ? -EINVAL : 0;
+}
+
+static int __init parse_dom0_colors(const char *s)
+{
+    return parse_color_config(s, dom0_colors, ARRAY_SIZE(dom0_colors),
+                              &dom0_num_colors);
+}
+custom_param("dom0-llc-colors", parse_dom0_colors);
 
 static void print_colors(const unsigned int *colors, unsigned int num_colors)
 {
@@ -43,9 +103,26 @@ static void print_colors(const unsigned int *colors, unsigned int num_colors)
     printk(" }\n");
 }
 
+static bool __init check_colors(const unsigned int *colors,
+                                unsigned int num_colors)
+{
+    unsigned int i;
+
+    for ( i = 0; i < num_colors; i++ )
+    {
+        if ( colors[i] >= max_nr_colors )
+        {
+            printk(XENLOG_ERR "LLC color %u >= %u\n", colors[i], max_nr_colors);
+            return false;
+        }
+    }
+
+    return true;
+}
+
 void __init llc_coloring_init(void)
 {
-    unsigned int way_size;
+    unsigned int way_size, i;
 
     if ( llc_size && llc_nr_ways )
     {
@@ -79,6 +156,9 @@ void __init llc_coloring_init(void)
     } else if ( max_nr_colors < 2 )
         panic("Number of LLC colors %u < 2\n", max_nr_colors);
 
+    for ( i = 0; i < max_nr_colors; i++ )
+        default_colors[i] = i;
+
     arch_llc_coloring_init();
 }
 
@@ -100,6 +180,52 @@ void domain_dump_llc_colors(const struct domain *d)
     print_colors(d->llc_colors, d->num_llc_colors);
 }
 
+static void __init domain_set_default_colors(struct domain *d)
+{
+    printk(XENLOG_WARNING
+           "LLC color config not found for %pd, using all colors\n", d);
+
+    d->llc_colors = default_colors;
+    d->num_llc_colors = max_nr_colors;
+}
+
+int __init dom0_set_llc_colors(struct domain *d)
+{
+    typeof(*dom0_colors) *colors;
+
+    if ( !dom0_num_colors )
+    {
+        domain_set_default_colors(d);
+        return 0;
+    }
+
+    if ( dom0_num_colors > max_nr_colors ||
+         !check_colors(dom0_colors, dom0_num_colors) )
+    {
+        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
+        return -EINVAL;
+    }
+
+    colors = xmalloc_array(typeof(*dom0_colors), dom0_num_colors);
+    if ( !colors )
+        return -ENOMEM;
+
+    memcpy(colors, dom0_colors, sizeof(*colors) * dom0_num_colors);
+    d->llc_colors = colors;
+    d->num_llc_colors = dom0_num_colors;
+
+    return 0;
+}
+
+void domain_llc_coloring_free(struct domain *d)
+{
+    if ( !llc_coloring_enabled || d->llc_colors == default_colors )
+        return;
+
+    /* free pointer-to-const using __va(__pa()) */
+    xfree(__va(__pa(d->llc_colors)));
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index 67b27c995b..ee82932266 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -16,16 +16,19 @@ extern bool llc_coloring_enabled;
 void llc_coloring_init(void);
 void dump_llc_coloring_info(void);
 void domain_dump_llc_colors(const struct domain *d);
+void domain_llc_coloring_free(struct domain *d);
 #else
 #define llc_coloring_enabled false
 
 static inline void llc_coloring_init(void) {}
 static inline void dump_llc_coloring_info(void) {}
 static inline void domain_dump_llc_colors(const struct domain *d) {}
+static inline void domain_llc_coloring_free(struct domain *d) {}
 #endif
 
 unsigned int get_llc_way_size(void);
 void arch_llc_coloring_init(void);
+int dom0_set_llc_colors(struct domain *d);
 
 #endif /* __COLORING_H__ */
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715950.1117969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zij-0007U9-LT; Thu, 02 May 2024 16:55:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715950.1117969; Thu, 02 May 2024 16:55:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zij-0007TA-HL; Thu, 02 May 2024 16:55:45 +0000
Received: by outflank-mailman (input) for mailman id 715950;
 Thu, 02 May 2024 16:55:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zih-0006N3-VT
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:43 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cfab38dc-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:42 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f3a49ff7dso919918e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:42 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfab38dc-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668942; x=1715273742; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9lNNmDdINfoHlllIflrjGEio9WD33GRBGAZScKnxGPA=;
        b=vIo2zAhWo5eT4uj4bhJHsRWmJ96FC1KhMzNfgWUQHJRriW1+BZI6M9DL5NQbPrNtcZ
         5fKhFpddbpBBcuVtO8QgeExlq0N6fSh8Sd7z9JW5vTH8PtYCI9v9wV9fdbwsXQWy0PAl
         MXLAeUNhdQ/nl9v229ZGa6vDLMob3/X0JgFumCdMMCOLjMntwpit5KZTINPC5t6uQ+e2
         +f3hwH2CXU+vepsngrNG++7j366Pv8af7st6jUbto74wfFD5cMp+I9iEefraM2e/T4BC
         gmgcD8QKw/Zp28T3EiS2W2YI7z8PdYWeGawI1q9fCNDBAqHhBVcmiKzhCT6h8prqYLn2
         8I0g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668942; x=1715273742;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9lNNmDdINfoHlllIflrjGEio9WD33GRBGAZScKnxGPA=;
        b=Cx366My83m+MdyBKQgu+MEFX20N4SOVNJLrKvWYk4sWryJZcYn2JhYAto2SBL7sr/h
         /hQCGfxLGHnE3o9blCCX7niJ1BFWwntkiFsqrYmc8xFZ+WJrJ0pLu2aYzELjl5bCOwdd
         QppQmVCFPjne/Ixz+h6mg+EFelGFyCbUkygwL8xzIWs5g+4wlDXXto7SU0Bwtv/pQduI
         2f45dBbyzpbAH+miKHH3GqoLEVFRrGn7Wajb77sGXjLbjhICzvN4Jo0Q7+JHlGfBuzeo
         gPamylXEAvm5gsJtd1IBvQyaFVKxt8yTAp9JjNvFc+5lXzsDCkJc+fSridXuOdeN677T
         laTA==
X-Gm-Message-State: AOJu0YwnSCx5BeznvdWMGV1Zk8g1/cZ+uyLTwbmbzUBf+/In6h4QLcuG
	pGXcyD3DxYMZS63xYtvI9TgFt6vmP4OoatVrNV41Tp7QJSWxWxFK0QDZjQti69nasA89t4/exIo
	I
X-Google-Smtp-Source: AGHT+IFE87a06WYupkqdQKi0ZTarJY7Cax3giuwXsFKojmeBItsHdWnxzl4JvHy/bNxWSAvrjg0bwg==
X-Received: by 2002:a05:6512:32b9:b0:51e:11d5:bca5 with SMTP id q25-20020a05651232b900b0051e11d5bca5mr224999lfe.54.1714668941569;
        Thu, 02 May 2024 09:55:41 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 05/13] xen: extend domctl interface for cache coloring
Date: Thu,  2 May 2024 18:55:25 +0200
Message-Id: <20240502165533.319988-6-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new domctl hypercall to allow the user to set LLC coloring
configurations. Colors can be set only once, just after domain creation,
since recoloring isn't supported.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- fixed memory leak on error path of domain_set_llc_colors()
v7:
- -EOPNOTSUPP returned in case of hypercall called without llc_coloring_enabled
- domain_set_llc_colors_domctl() renamed to domain_set_llc_colors()
- added padding and input bound checks to domain_set_llc_colors()
- removed alloc_colors() helper usage from domain_set_llc_colors()
v6:
- reverted the XEN_DOMCTL_INTERFACE_VERSION bump
- reverted to uint32 for the guest handle
- explicit padding added to the domctl struct
- rewrote domain_set_llc_colors_domctl() to be more explicit
v5:
- added a new hypercall to set colors
- uint for the guest handle
v4:
- updated XEN_DOMCTL_INTERFACE_VERSION
---
 xen/common/domctl.c            | 10 ++++++++
 xen/common/llc-coloring.c      | 46 +++++++++++++++++++++++++++++++---
 xen/include/public/domctl.h    |  9 +++++++
 xen/include/xen/llc-coloring.h |  2 ++
 4 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 2c0331bb05..30cca9a4ae 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -8,6 +8,7 @@
 
 #include <xen/types.h>
 #include <xen/lib.h>
+#include <xen/llc-coloring.h>
 #include <xen/err.h>
 #include <xen/mm.h>
 #include <xen/sched.h>
@@ -864,6 +865,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
                 __HYPERVISOR_domctl, "h", u_domctl);
         break;
 
+    case XEN_DOMCTL_set_llc_colors:
+        if ( op->u.set_llc_colors.pad )
+            ret = -EINVAL;
+        else if ( llc_coloring_enabled )
+            ret = domain_set_llc_colors(d, &op->u.set_llc_colors);
+        else
+            ret = -EOPNOTSUPP;
+        break;
+
     default:
         ret = arch_do_domctl(op, d, u_domctl);
         break;
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index 26270cda9c..ecfeb0ce82 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -4,6 +4,7 @@
  *
  * Copyright (C) 2022 Xilinx Inc.
  */
+#include <xen/guest_access.h>
 #include <xen/keyhandler.h>
 #include <xen/llc-coloring.h>
 #include <xen/param.h>
@@ -103,8 +104,7 @@ static void print_colors(const unsigned int *colors, unsigned int num_colors)
     printk(" }\n");
 }
 
-static bool __init check_colors(const unsigned int *colors,
-                                unsigned int num_colors)
+static bool check_colors(const unsigned int *colors, unsigned int num_colors)
 {
     unsigned int i;
 
@@ -180,7 +180,7 @@ void domain_dump_llc_colors(const struct domain *d)
     print_colors(d->llc_colors, d->num_llc_colors);
 }
 
-static void __init domain_set_default_colors(struct domain *d)
+static void domain_set_default_colors(struct domain *d)
 {
     printk(XENLOG_WARNING
            "LLC color config not found for %pd, using all colors\n", d);
@@ -226,6 +226,46 @@ void domain_llc_coloring_free(struct domain *d)
     xfree(__va(__pa(d->llc_colors)));
 }
 
+int domain_set_llc_colors(struct domain *d,
+                          const struct xen_domctl_set_llc_colors *config)
+{
+    unsigned int *colors;
+
+    if ( d->num_llc_colors )
+        return -EEXIST;
+
+    if ( !config->num_llc_colors )
+    {
+        domain_set_default_colors(d);
+        return 0;
+    }
+
+    if ( config->num_llc_colors > max_nr_colors )
+        return -EINVAL;
+
+    colors = xmalloc_array(unsigned int, config->num_llc_colors);
+    if ( !colors )
+        return -ENOMEM;
+
+    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_colors) )
+    {
+        xfree(colors);
+        return -EFAULT;
+    }
+
+    if ( !check_colors(colors, config->num_llc_colors) )
+    {
+        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
+        xfree(colors);
+        return -EINVAL;
+    }
+
+    d->llc_colors = colors;
+    d->num_llc_colors = config->num_llc_colors;
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..d44eac8775 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1190,6 +1190,13 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+struct xen_domctl_set_llc_colors {
+    /* IN LLC coloring parameters */
+    uint32_t num_llc_colors;
+    uint32_t pad;
+    XEN_GUEST_HANDLE_64(uint32) llc_colors;
+};
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1284,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_set_llc_colors                87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1347,7 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+        struct xen_domctl_set_llc_colors    set_llc_colors;
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index ee82932266..b3801fca00 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -29,6 +29,8 @@ static inline void domain_llc_coloring_free(struct domain *d) {}
 unsigned int get_llc_way_size(void);
 void arch_llc_coloring_init(void);
 int dom0_set_llc_colors(struct domain *d);
+int domain_set_llc_colors(struct domain *d,
+                          const struct xen_domctl_set_llc_colors *config);
 
 #endif /* __COLORING_H__ */
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715945.1117920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zie-0006Ls-TK; Thu, 02 May 2024 16:55:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715945.1117920; Thu, 02 May 2024 16:55:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zie-0006Ll-Qo; Thu, 02 May 2024 16:55:40 +0000
Received: by outflank-mailman (input) for mailman id 715945;
 Thu, 02 May 2024 16:55:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zid-0006Lf-Pk
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:39 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd13bc1d-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:38 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a55bf737cecso1010509166b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:38 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd13bc1d-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668937; x=1715273737; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=0gkpKCmCscYcHI5K9J1ZWC0k2m8hrdGMWQYAyGZgqIU=;
        b=2HtycorCH4hcj0oLKQAjM7A8n7dJc1XvB2iszUDXTbpyCSAK9UndkQfKTU/uyJ5ug5
         mSrB1IcR4pau5BUT9OANta0dWtjx4K9GRsOaw5sM53clB0p32IkWlt5JIMUnR1HMNFQ1
         5k98fwq2rHSdcCsi9wF2kTxT24d+VVzAJUylX+8oczucOeDGj0zpZCDzIhC3EOBeEFyd
         ftk19A/WP4gni3k5teZhJE+oxqTQRoICWCPp/MtNI2mQ96lZPKJXEhKfCfQBoHLIDzQ3
         6mSHYvIPHjxpray8goWKgI+9EafgXJl1NCdo/Kfzurv3vpe3r/P4RXx6/oWiXT3qxWeg
         8FVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668937; x=1715273737;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0gkpKCmCscYcHI5K9J1ZWC0k2m8hrdGMWQYAyGZgqIU=;
        b=a0EsJxtbixR+2B6f5OIUl5CwjHJAmQSSlRn3fc/ryMOLWxmGKUZfiKpxzMO4/vqYAa
         hbcnd83JDWN5nAv4cj3jY0Kf7AThYOQS9ewADvRBQlNynA5LIza/5+kt8gyHBj0cH9Ba
         va6OXokFGYgj04oUjkOzBGXc+5fY0SmYOnheXqpxU3VrZlDMbRzY3skAbgUkPxMZwJGo
         0s2smHoauf6Qtt7GBUeMyzCRdhUbsDJ+PPK6SHbyVbEala1xBIejKqNSj8nQH8RzZqUj
         M115/3LEN7FhfG77CNA09ezgS0BRT0mA9ohVWPedF6jeZo3nExDSjhtWvnlNutcgt3u4
         HYNA==
X-Gm-Message-State: AOJu0YySOlX5z9SZFFC87DUrwTzlsNEHEuDlVTGg+xyt4xhEpNNiQ9m1
	DSOcWgDDyqReFSYG9OKVIInHwD9FqMoKhVbqiqg+dw6xFvFbbX56L0LWoqi7UreDHZewsN+ODzQ
	r
X-Google-Smtp-Source: AGHT+IF2ygDwkCaFzAIPf+pJIDcPHgiTFNxHs9rVsRZhxQkDr56yF0mglY0KehVMayvtxae9mTD2hg==
X-Received: by 2002:a17:906:f1c8:b0:a58:9c9b:61ad with SMTP id gx8-20020a170906f1c800b00a589c9b61admr68340ejb.43.1714668937383;
        Thu, 02 May 2024 09:55:37 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v8 00/13] Arm cache coloring
Date: Thu,  2 May 2024 18:55:20 +0200
Message-Id: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Shared caches in multi-core CPU architectures represent a problem for
predictability of memory access latency. This jeopardizes applicability
of many Arm platform in real-time critical and mixed-criticality
scenarios. We introduce support for cache partitioning with page
coloring, a transparent software technique that enables isolation
between domains and Xen, and thus avoids cache interference.

When creating a domain, a simple syntax (e.g. `0-3` or `4-11`) allows
the user to define assignments of cache partitions ids, called colors,
where assigning different colors guarantees no mutual eviction on cache
will ever happen. This instructs the Xen memory allocator to provide
the i-th color assignee only with pages that maps to color i, i.e. that
are indexed in the i-th cache partition.

The proposed implementation supports the dom0less feature.
The proposed implementation doesn't support the static-mem feature.
The solution has been tested in several scenarios, including Xilinx Zynq
MPSoCs.

Carlo Nonato (12):
  xen/common: add cache coloring common code
  xen/arm: add initial support for LLC coloring on arm64
  xen/arm: permit non direct-mapped Dom0 construction
  xen/arm: add Dom0 cache coloring support
  xen: extend domctl interface for cache coloring
  tools: add support for cache coloring configuration
  xen/arm: add support for cache coloring configuration via device-tree
  xen/page_alloc: introduce preserved page flags macro
  xen: add cache coloring allocator for domains
  xen/arm: use domain memory to allocate p2m page tables
  xen/arm: make consider_modules() available for xen relocation
  xen/arm: add cache coloring support for Xen

Luca Miccio (1):
  xen/arm: add Xen cache colors command line parameter

 SUPPORT.md                              |   7 +
 docs/man/xl.cfg.5.pod.in                |  10 +
 docs/misc/arm/device-tree/booting.txt   |   4 +
 docs/misc/cache-coloring.rst            | 246 +++++++++++++++
 docs/misc/xen-command-line.pandoc       |  70 +++++
 tools/include/libxl.h                   |   5 +
 tools/include/xenctrl.h                 |   9 +
 tools/libs/ctrl/xc_domain.c             |  35 +++
 tools/libs/light/libxl_create.c         |  13 +
 tools/libs/light/libxl_types.idl        |   1 +
 tools/xl/xl_parse.c                     |  38 ++-
 xen/arch/arm/Kconfig                    |   1 +
 xen/arch/arm/Makefile                   |   1 +
 xen/arch/arm/alternative.c              |  30 +-
 xen/arch/arm/arm32/mmu/mm.c             | 115 +------
 xen/arch/arm/arm64/mmu/head.S           |  58 +++-
 xen/arch/arm/arm64/mmu/mm.c             |  28 +-
 xen/arch/arm/dom0less-build.c           |  60 +---
 xen/arch/arm/domain_build.c             | 107 ++++++-
 xen/arch/arm/include/asm/domain_build.h |   1 +
 xen/arch/arm/include/asm/mm.h           |   5 +
 xen/arch/arm/include/asm/mmu/layout.h   |   3 +
 xen/arch/arm/include/asm/processor.h    |  16 +
 xen/arch/arm/include/asm/setup.h        |   3 +
 xen/arch/arm/llc-coloring.c             | 140 +++++++++
 xen/arch/arm/mmu/p2m.c                  |   4 +-
 xen/arch/arm/mmu/setup.c                | 221 +++++++++++++-
 xen/arch/arm/setup.c                    |  13 +-
 xen/common/Kconfig                      |  30 ++
 xen/common/Makefile                     |   1 +
 xen/common/domain.c                     |   3 +
 xen/common/domctl.c                     |  10 +
 xen/common/keyhandler.c                 |   3 +
 xen/common/llc-coloring.c               | 379 ++++++++++++++++++++++++
 xen/common/page_alloc.c                 | 204 ++++++++++++-
 xen/include/public/domctl.h             |   9 +
 xen/include/xen/llc-coloring.h          |  63 ++++
 xen/include/xen/sched.h                 |   5 +
 xen/include/xen/xmalloc.h               |  12 +
 39 files changed, 1768 insertions(+), 195 deletions(-)
 create mode 100644 docs/misc/cache-coloring.rst
 create mode 100644 xen/arch/arm/llc-coloring.c
 create mode 100644 xen/common/llc-coloring.c
 create mode 100644 xen/include/xen/llc-coloring.h

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715946.1117930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006au-4e; Thu, 02 May 2024 16:55:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715946.1117930; Thu, 02 May 2024 16:55:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006an-1o; Thu, 02 May 2024 16:55:43 +0000
Received: by outflank-mailman (input) for mailman id 715946;
 Thu, 02 May 2024 16:55:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zif-0006Lf-Bn
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce959dcb-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:40 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a4702457ccbso1157253466b.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:40 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce959dcb-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668939; x=1715273739; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=okh8duaE+EFMPEO4FhT3MzEyU6q3kS1zoVKRzTYnIo0=;
        b=YsLw4RvwdV3j1BdDTCzJ0VbrZynYZIzldbHHf4L9M7usWrlUNVQSWrWEU+XUG/sXiF
         Da63TUupIvjGKe0zFYseJdeIklpHgaeFFvC8TePKZOgXvPd5Z0Hd3R35EbHq8hLLxSnn
         /UG79/8YwLttDgQn8tb+IiW3SdTU6LaVl8eum5ocF58e0OHqQZFCvAm8SzNqWxdxQCEL
         5k2QIeL+7ryXu7/H+V/dhX1v5UX6+NFzyGP9fZHpaNXOJazG9QBCKdW7bn3fSagCu9GX
         MrkUKnTzecoIYgJ/N2/pTrBsGzO0tBb0TaGfHHcKwu5ZkaLO3W8KzpeNffKplPRBskUH
         8I2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668939; x=1715273739;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=okh8duaE+EFMPEO4FhT3MzEyU6q3kS1zoVKRzTYnIo0=;
        b=v7h9OmUe9XXsbsYAXBs0SQh597BoVhnbBqU87k+ktrTF/TGU5BT+SbZlecz6FxjNw1
         /dXqW/5kiJAF0+lAtae2kXRFUNV/4vTX+rbAJQylRRNbH/TBM7e2g09Z1n99dnTdAUZQ
         LqHHIzH3qLxoYB++DBbijoQTA+Z3fbbm2kk9PfsHhdnIKiK9tujPwsjknQ+10NWoNVwV
         GKcdYCKdvz00EinepZ8E8cM6h22kRWiGKz1URkUX3BQSKtWES4E65tp2xHhZNy3YJOFG
         prPJHTD8R6DMHzE5Hs5rTeXQNI3YyhMXGcjoAMbWRIDB8AVHztRtBu0XAqHHer2wKEz3
         +mAw==
X-Gm-Message-State: AOJu0YwEEbqCaapfzjXD3WYMvZNoygcjckW1XWV+RI+OKhus26+JoLiu
	dZZUg+IqjM81+UBxezxX1v2kboefuerNBCFeVrvG93/zV/O9IkH8HeghW+XCraGXZlkEcH91Mi9
	Y
X-Google-Smtp-Source: AGHT+IGHjOT87G7UTWtB/PiIPUk65gmJVKYGc/SIGIKgPC0DhCwF2tIf+GoOIE1hkPBxW5p8pnGwVw==
X-Received: by 2002:a17:906:110c:b0:a59:719d:2735 with SMTP id h12-20020a170906110c00b00a59719d2735mr54793eja.64.1714668939702;
        Thu, 02 May 2024 09:55:39 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v8 03/13] xen/arm: permit non direct-mapped Dom0 construction
Date: Thu,  2 May 2024 18:55:23 +0200
Message-Id: <20240502165533.319988-4-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Cache coloring requires Dom0 not to be direct-mapped because of its non
contiguous mapping nature, so allocate_memory() is needed in this case.
8d2c3ab18cc1 ("arm/dom0less: put dom0less feature code in a separate module")
moved allocate_memory() in dom0less_build.c. In order to use it
in Dom0 construction bring it back to domain_build.c and declare it in
domain_build.h.

Take the opportunity to adapt the implementation of allocate_memory() so
that it uses the host layout when called on the hwdom, via
find_unallocated_memory().

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
---
v8:
- patch adapted to new changes to allocate_memory()
v7:
- allocate_memory() now uses the host layout when called on the hwdom
v6:
- new patch
---
 xen/arch/arm/dom0less-build.c           | 44 -----------
 xen/arch/arm/domain_build.c             | 97 ++++++++++++++++++++++++-
 xen/arch/arm/include/asm/domain_build.h |  1 +
 3 files changed, 94 insertions(+), 48 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index ca7519cf3b..c6bc4ee59c 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -49,50 +49,6 @@ bool __init is_dom0less_mode(void)
     return ( !dom0found && domUfound );
 }
 
-static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
-{
-    struct membanks *mem = kernel_info_get_mem(kinfo);
-    unsigned int i;
-    paddr_t bank_size;
-
-    printk(XENLOG_INFO "Allocating mappings totalling %ldMB for %pd:\n",
-           /* Don't want format this as PRIpaddr (16 digit hex) */
-           (unsigned long)(kinfo->unassigned_mem >> 20), d);
-
-    mem->nr_banks = 0;
-    bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
-                               bank_size) )
-        goto fail;
-
-    bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
-                               bank_size) )
-        goto fail;
-
-    if ( kinfo->unassigned_mem )
-        goto fail;
-
-    for( i = 0; i < mem->nr_banks; i++ )
-    {
-        printk(XENLOG_INFO "%pd BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
-               d,
-               i,
-               mem->bank[i].start,
-               mem->bank[i].start + mem->bank[i].size,
-               /* Don't want format this as PRIpaddr (16 digit hex) */
-               (unsigned long)(mem->bank[i].size >> 20));
-    }
-
-    return;
-
-fail:
-    panic("Failed to allocate requested domain memory."
-          /* Don't want format this as PRIpaddr (16 digit hex) */
-          " %ldKB unallocated. Fix the VMs configurations.\n",
-          (unsigned long)kinfo->unassigned_mem >> 10);
-}
-
 #ifdef CONFIG_VGICV2
 static int __init make_gicv2_domU_node(struct kernel_info *kinfo)
 {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 0784e4c5e3..ba2e48ecc7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -416,7 +416,6 @@ static void __init allocate_memory_11(struct domain *d,
     }
 }
 
-#ifdef CONFIG_DOM0LESS_BOOT
 bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
                                  gfn_t sgfn, paddr_t tot_size)
 {
@@ -479,7 +478,6 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
 
     return true;
 }
-#endif
 
 /*
  * When PCI passthrough is available we want to keep the
@@ -968,6 +966,94 @@ out:
     return res;
 }
 
+void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
+{
+    struct membanks *mem = kernel_info_get_mem(kinfo);
+    unsigned int i, nr_banks = 2;
+    paddr_t bank_start, bank_size;
+    struct membanks *hwdom_ext_regions = NULL;
+
+    printk(XENLOG_INFO "Allocating mappings totalling %ldMB for %pd:\n",
+           /* Don't want format this as PRIpaddr (16 digit hex) */
+           (unsigned long)(kinfo->unassigned_mem >> 20), d);
+
+    mem->nr_banks = 0;
+    /*
+     * Use host memory layout for hwdom. Only case for this is when LLC coloring
+     * is enabled.
+     */
+    if ( is_hardware_domain(d) )
+    {
+        ASSERT(llc_coloring_enabled);
+
+        hwdom_ext_regions = xzalloc_flex_struct(struct membanks, bank,
+                                                NR_MEM_BANKS);
+        if ( !hwdom_ext_regions )
+            goto fail;
+        hwdom_ext_regions->max_banks = NR_MEM_BANKS;
+
+        if ( find_unallocated_memory(kinfo, hwdom_ext_regions) )
+            goto fail;
+
+        nr_banks = hwdom_ext_regions->nr_banks;
+    }
+
+    for ( i = 0; kinfo->unassigned_mem > 0 && nr_banks > 0; i++, nr_banks-- )
+    {
+        if ( is_hardware_domain(d) )
+        {
+            bank_start = hwdom_ext_regions->bank[i].start;
+            bank_size = hwdom_ext_regions->bank[i].size;
+
+            if ( bank_size < min_t(paddr_t, kinfo->unassigned_mem, MB(128)) )
+                continue;
+        }
+        else
+        {
+            if ( i == 0 )
+            {
+                bank_start = GUEST_RAM0_BASE;
+                bank_size = GUEST_RAM0_SIZE;
+            }
+            else if ( i == 1 )
+            {
+                bank_start = GUEST_RAM1_BASE;
+                bank_size = GUEST_RAM1_SIZE;
+            }
+            else
+                goto fail;
+        }
+
+        bank_size = MIN(bank_size, kinfo->unassigned_mem);
+        if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(bank_start),
+                                   bank_size) )
+            goto fail;
+    }
+
+    if ( kinfo->unassigned_mem )
+        goto fail;
+
+    for( i = 0; i < mem->nr_banks; i++ )
+    {
+        printk(XENLOG_INFO "%pd BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
+               d,
+               i,
+               mem->bank[i].start,
+               mem->bank[i].start + mem->bank[i].size,
+               /* Don't want format this as PRIpaddr (16 digit hex) */
+               (unsigned long)(mem->bank[i].size >> 20));
+    }
+
+    xfree(hwdom_ext_regions);
+    return;
+
+fail:
+    panic("Failed to allocate requested domain memory."
+          /* Don't want format this as PRIpaddr (16 digit hex) */
+          " %ldKB unallocated. Fix the VMs configurations.\n",
+          (unsigned long)kinfo->unassigned_mem >> 10);
+}
+
 static int __init handle_pci_range(const struct dt_device_node *dev,
                                    uint64_t addr, uint64_t len, void *data)
 {
@@ -1188,7 +1274,7 @@ int __init make_hypervisor_node(struct domain *d,
 
         ext_regions->max_banks = NR_MEM_BANKS;
 
-        if ( is_domain_direct_mapped(d) )
+        if ( domain_use_host_layout(d) )
         {
             if ( !is_iommu_enabled(d) )
                 res = find_unallocated_memory(kinfo, ext_regions);
@@ -2117,7 +2203,10 @@ static int __init construct_dom0(struct domain *d)
     /* type must be set before allocate_memory */
     d->arch.type = kinfo.type;
 #endif
-    allocate_memory_11(d, &kinfo);
+    if ( is_domain_direct_mapped(d) )
+        allocate_memory_11(d, &kinfo);
+    else
+        allocate_memory(d, &kinfo);
     find_gnttab_region(d, &kinfo);
 
     rc = process_shm_chosen(d, &kinfo);
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 45936212ca..943a6d438e 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -8,6 +8,7 @@ typedef __be32 gic_interrupt_t[3];
 
 bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
                           gfn_t sgfn, paddr_t tot_size);
+void allocate_memory(struct domain *d, struct kernel_info *kinfo);
 int construct_domain(struct domain *d, struct kernel_info *kinfo);
 int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit);
 int make_chosen_node(const struct kernel_info *kinfo);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715951.1117973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zik-0007Yu-3l; Thu, 02 May 2024 16:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715951.1117973; Thu, 02 May 2024 16:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zij-0007Xo-RW; Thu, 02 May 2024 16:55:45 +0000
Received: by outflank-mailman (input) for mailman id 715951;
 Thu, 02 May 2024 16:55:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zii-0006Lf-2b
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:44 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d01e4368-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:43 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5966f5a76bso182206866b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:43 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d01e4368-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668942; x=1715273742; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3EUvo/9M8ITy6G3rHvhG7NG2DwAgBy3coT8I6ydQ2x4=;
        b=VS2N/fqYnnpQrOoXFuh4WnWwAMyilj52BvYFWiutiFyzngHCUGVMQo/ZO+mQ26abUp
         3kqzhL/mFzAFqNNYXFJD2Mg4NHdMkkLk53mLoKVvn97sud+Vk+bAtdejAMybIkeeHdox
         bvgvazr/8lihylx4dFf1WkNxlqToN8qjq0/m2DCglGYBpm+tPtnmI2M5mXVNyhWTow/C
         ETIpbJ2DLEzSJNK6FSrk2KcICOx9po1PGh9ZSk4RscbbursZ2vP6CR2LtFDlX6VTyupd
         xtp7nmhpdhtoll48efxOIgvWN4GimRicXVY5FiM5792fQqjukKbGpsc05wjWo0H2DU1s
         wPFA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668942; x=1715273742;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3EUvo/9M8ITy6G3rHvhG7NG2DwAgBy3coT8I6ydQ2x4=;
        b=iOZLc3ERsqWRh7yM68RHux/WINVGJf+H9hPe1I4aZV3NXoVl08u2vlhFKl/qoSsnT/
         3FSZ4PcuID36/H4SRiEILzgGciaR+mvcRQFNN1INiiKKQwXYN0YweAM0eb3VwaF84TYx
         lCtkVyzju1ikNIBZ1dZ6dtBd2FN9wJ53QCfbXLudoQOJDr7WTScZwOJad0oqvqq8eOi2
         I0ecpW/nSfrFatfkivSNRgtQg3WpWChOgBTDXDBeYVUZwe4fWk1J94Nt44Zegi/82jee
         KCaqLGdQi5EdY0Bx+YU5Jole588T4dbmhELCGo48rfBVwFIc/GAzqXudLY01WW4mGdEZ
         NBKg==
X-Gm-Message-State: AOJu0Yz8IAvEFSklRsrEysq3hcS+FtGNy9ijk2fwK+x7tKZTqs6QtN1W
	Vm/1vHzv93M/x6YOl6fezr0HepPUidLZZMDi+sbiBJTCZQsJKlTtj3lN/AuVNdAaGYZ7lOjnrmc
	5
X-Google-Smtp-Source: AGHT+IEaaUlZhh8N1o1seHM3dS9FR8bI1vP2vECN7M/57vevuCIUxNoc+9g8AGbYJ3CQ/WfMUyAYvw==
X-Received: by 2002:a17:906:4906:b0:a55:5958:cb00 with SMTP id b6-20020a170906490600b00a555958cb00mr2219308ejq.38.1714668942177;
        Thu, 02 May 2024 09:55:42 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 06/13] tools: add support for cache coloring configuration
Date: Thu,  2 May 2024 18:55:26 +0200
Message-Id: <20240502165533.319988-7-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new "llc_colors" parameter that defines the LLC color assignment for
a domain. The user can specify one or more color ranges using the same
syntax used everywhere else for color config described in the
documentation.
The parameter is defined as a list of strings that represent the color
ranges.

Documentation is also added.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- warn the user in case of coloring not supported at hypervisor level
v7:
- removed unneeded NULL check before xc_hypercall_buffer_free() in
  xc_domain_set_llc_colors()
v6:
- no edits
v5:
- added LIBXL_HAVE_BUILDINFO_LLC_COLORS
- moved color configuration in xc_domain_set_llc_colors() cause of the new
  hypercall
v4:
- removed overlapping color ranges checks during parsing
- moved hypercall buffer initialization in libxenctrl
---
 docs/man/xl.cfg.5.pod.in         | 10 +++++++++
 tools/include/libxl.h            |  5 +++++
 tools/include/xenctrl.h          |  9 ++++++++
 tools/libs/ctrl/xc_domain.c      | 35 +++++++++++++++++++++++++++++
 tools/libs/light/libxl_create.c  | 13 +++++++++++
 tools/libs/light/libxl_types.idl |  1 +
 tools/xl/xl_parse.c              | 38 +++++++++++++++++++++++++++++++-
 7 files changed, 110 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..320644701b 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,16 @@ raised.
 
 =back
 
+=over 4
+
+=item B<llc_colors=[ "RANGE", "RANGE", ...]>
+
+Specify the Last Level Cache (LLC) color configuration for the guest.
+B<RANGE> can be either a single color value or a hypen-separated closed
+interval of colors (such as "0-4").
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..49521e5da4 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1368,6 +1368,11 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
  */
 #define LIBXL_HAVE_BUILDINFO_HVM_SYSTEM_FIRMWARE
 
+/*
+ * The libxl_domain_build_info has the llc_colors array.
+ */
+#define LIBXL_HAVE_BUILDINFO_LLC_COLORS 1
+
 /*
  * ERROR_REMUS_XXX error code only exists from Xen 4.5, Xen 4.6 and it
  * is changed to ERROR_CHECKPOINT_XXX in Xen 4.7
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..2ad8e2066b 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2654,6 +2654,15 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32
 int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
                          xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
 
+/*
+ * Set LLC colors for a domain.
+ * It can only be used directly after domain creation. An attempt to use it
+ * afterwards will result in an error.
+ */
+int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
+                             const unsigned int *llc_colors,
+                             unsigned int num_llc_colors);
+
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index f2d9d14b4d..d315cfa6c1 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -2180,6 +2180,41 @@ int xc_domain_soft_reset(xc_interface *xch,
     domctl.domain = domid;
     return do_domctl(xch, &domctl);
 }
+
+int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
+                             const unsigned int *llc_colors,
+                             unsigned int num_llc_colors)
+{
+    struct xen_domctl domctl = {};
+    DECLARE_HYPERCALL_BUFFER(uint32_t, local);
+    int ret = -1;
+
+    if ( num_llc_colors )
+    {
+        size_t bytes = sizeof(uint32_t) * num_llc_colors;
+
+        local = xc_hypercall_buffer_alloc(xch, local, bytes);
+        if ( local == NULL )
+        {
+            PERROR("Could not allocate LLC colors for set_llc_colors");
+            ret = -ENOMEM;
+            goto out;
+        }
+        memcpy(local, llc_colors, bytes);
+        set_xen_guest_handle(domctl.u.set_llc_colors.llc_colors, local);
+    }
+
+    domctl.cmd = XEN_DOMCTL_set_llc_colors;
+    domctl.domain = domid;
+    domctl.u.set_llc_colors.num_llc_colors = num_llc_colors;
+
+    ret = do_domctl(xch, &domctl);
+
+out:
+    xc_hypercall_buffer_free(xch, local);
+
+    return ret;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 41252ec553..ff80625bd6 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -726,6 +726,19 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
             /* A new domain now exists */
             *domid = local_domid;
 
+            ret = xc_domain_set_llc_colors(ctx->xch, local_domid,
+                                           b_info->llc_colors,
+                                           b_info->num_llc_colors);
+            if (ret < 0 && errno == EOPNOTSUPP) {
+                if (b_info->num_llc_colors > 0)
+                    LOGED(WARN, local_domid,
+                          "LLC coloring not enabled in the hypervisor");
+            } else if (ret < 0) {
+                LOGED(ERROR, local_domid, "LLC colors allocation failed");
+                rc = ERROR_FAIL;
+                goto out;
+            }
+
             rc = libxl__is_domid_recent(gc, local_domid, &recent);
             if (rc)
                 goto out;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 470122e768..79118e1582 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -616,6 +616,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("ioports",          Array(libxl_ioport_range, "num_ioports")),
     ("irqs",             Array(uint32, "num_irqs")),
     ("iomem",            Array(libxl_iomem_range, "num_iomem")),
+    ("llc_colors",       Array(uint32, "num_llc_colors")),
     ("claim_mode",	     libxl_defbool),
     ("event_channels",   uint32),
     ("kernel",           string),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index ab09d0288b..ddfbcb00fd 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1294,7 +1294,7 @@ void parse_config_data(const char *config_source,
     XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms,
                    *usbctrls, *usbdevs, *p9devs, *vdispls, *pvcallsifs_devs;
     XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian, *dtdevs,
-                   *mca_caps, *smbios;
+                   *mca_caps, *smbios, *llc_colors;
     int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian, num_mca_caps;
     int num_smbios;
     int pci_power_mgmt = 0;
@@ -1302,6 +1302,7 @@ void parse_config_data(const char *config_source,
     int pci_permissive = 0;
     int pci_seize = 0;
     int i, e;
+    int num_llc_colors;
     char *kernel_basename;
 
     libxl_domain_create_info *c_info = &d_config->c_info;
@@ -1445,6 +1446,41 @@ void parse_config_data(const char *config_source,
     if (!xlu_cfg_get_long (config, "maxmem", &l, 0))
         b_info->max_memkb = l * 1024;
 
+    if (!xlu_cfg_get_list(config, "llc_colors", &llc_colors, &num_llc_colors, 0)) {
+        int cur_index = 0;
+
+        b_info->num_llc_colors = 0;
+        for (i = 0; i < num_llc_colors; i++) {
+            uint32_t start = 0, end = 0, k;
+
+            buf = xlu_cfg_get_listitem(llc_colors, i);
+            if (!buf) {
+                fprintf(stderr,
+                        "xl: Can't get element %d in LLC color list\n", i);
+                exit(1);
+            }
+
+            if (sscanf(buf, "%" SCNu32 "-%" SCNu32, &start, &end) != 2) {
+                if (sscanf(buf, "%" SCNu32, &start) != 1) {
+                    fprintf(stderr, "xl: Invalid LLC color range: %s\n", buf);
+                    exit(1);
+                }
+                end = start;
+            } else if (start > end) {
+                fprintf(stderr,
+                        "xl: Start LLC color is greater than end: %s\n", buf);
+                exit(1);
+            }
+
+            b_info->num_llc_colors += (end - start) + 1;
+            b_info->llc_colors = (uint32_t *)realloc(b_info->llc_colors,
+                        sizeof(*b_info->llc_colors) * b_info->num_llc_colors);
+
+            for (k = start; k <= end; k++)
+                b_info->llc_colors[cur_index++] = k;
+        }
+    }
+
     if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) {
         vcpus = l;
         if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715947.1117934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006cv-Ax; Thu, 02 May 2024 16:55:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715947.1117934; Thu, 02 May 2024 16:55:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006cS-7r; Thu, 02 May 2024 16:55:43 +0000
Received: by outflank-mailman (input) for mailman id 715947;
 Thu, 02 May 2024 16:55:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zif-0006N3-92
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd95efbb-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:39 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a56d7d457a1so1078769566b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:39 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd95efbb-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668938; x=1715273738; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7GREHaaEsQD6UfemfISve1bYOvLW8rS5iYyct68ZwSw=;
        b=EqkbhmjnI5QoSSieWG72gWd8+4WMPVc+PQY79pbvAjCKVb6Cc2krGEkzrqTZt15UJ7
         D/aaGH+7NEZzmVqH1z3BrB0rHKdZlXBlWryDohpTShvVsCBXanIN+c++GDzNGjR8QJlP
         n64GzWgrorq8uEyWT32Hi3bwQN+OKwW2ARDTvzn8oBZUqAUpQO6jtFjKbZ7K1d1lGT8P
         UwkdWiDVxFgcoTlgj3Mi4TR/rp9KuImRqyBgYUDFkeOGy0rJct8agmydMDG1ZSQh655S
         dDzafiqspXMKc91FI3GAuceW3dzWbfuvz6Q1XDZu7QykuDPw20C2CTUHUmeklbmv2lm9
         vldQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668938; x=1715273738;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7GREHaaEsQD6UfemfISve1bYOvLW8rS5iYyct68ZwSw=;
        b=Bjj3gM3UJnpkQ+q1sIc9h11aTWHD+becgTq4cvBHcoID0x9Kxb7hjy4/4ht6etqpZn
         qaV1AXIaYIAo51rPCdDeU3ghSJoLyXZ19YLwXm5GhzoTwrXXSeUfzdjk3Tuw2mEgyd1V
         BHjtQ1m03CxjG0Of5H5uZP2IZwPaA8tUpW0Xs1hMhnsZTMUN7bFkbM8Yuey+/hpX3dVW
         mi8ZqSnukSgpi5VemtEb/u3UjY6gCsVtW7PAKf2pMpSF6wnpCbwMrYxQNzCe40x6htNj
         uWSh1LoCjC9Xgr2DvDYdJa2MVzh8PieySuEvPvjozmcdAuRn5bPV3C9wk94lpsvZ5oYc
         6CJg==
X-Gm-Message-State: AOJu0Yzkftn3aKF60QzL+a0DpOjAB9fR1aVoRxluDd54KOfbo5Yb6j9l
	eYqvdsNgroWWdgrBwYcKzK4rGpZ3rLLXJOL4kr7aRQ+4g4ImBLZdJYfcyq/BAd12HF6UEO1IaA7
	E
X-Google-Smtp-Source: AGHT+IGWDc2DdScTkoTEys0xZwfnpda0hlZJeAncLA0A4bzwwXZSpr1rHSW9ZFbwc0DxtB10fwE7BA==
X-Received: by 2002:a17:906:c048:b0:a59:146:1c4e with SMTP id bm8-20020a170906c04800b00a5901461c4emr53013ejb.58.1714668938113;
        Thu, 02 May 2024 09:55:38 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 01/13] xen/common: add cache coloring common code
Date: Thu,  2 May 2024 18:55:21 +0200
Message-Id: <20240502165533.319988-2-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Last Level Cache (LLC) coloring allows to partition the cache in smaller
chunks called cache colors.

Since not all architectures can actually implement it, add a HAS_LLC_COLORING
Kconfig option.
MAX_LLC_COLORS_ORDER Kconfig option has a range maximum of 10 (2^10 = 1024)
because that's the number of colors that fit in a 4 KiB page when integers
are 4 bytes long.

LLC colors are a property of the domain, so struct domain has to be extended.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- minor documentation fixes
- "llc-coloring=on" is inferred from "llc-nr-ways" and "llc-size" usage
- turned CONFIG_NR_LLC_COLORS to CONFIG_MAX_LLC_COLORS_ORDER, base-2 exponent
- moved Kconfig options to common/Kconfig
- don't crash if computed max_nr_colors is too large
v7:
- SUPPORT.md changes added to this patch
- extended documentation to better address applicability of cache coloring
- "llc-nr-ways" and "llc-size" params introduced in favor of "llc-way-size"
- moved dump_llc_coloring_info() call in 'm' keyhandler (pagealloc_info())
v6:
- moved almost all code in common
- moved documentation in this patch
- reintroduced range for CONFIG_NR_LLC_COLORS
- reintroduced some stub functions to reduce the number of checks on
  llc_coloring_enabled
- moved domain_llc_coloring_free() in same patch where allocation happens
- turned "d->llc_colors" to pointer-to-const
- llc_coloring_init() now returns void and panics if errors are found
v5:
- used - instead of _ for filenames
- removed domain_create_llc_colored()
- removed stub functions
- coloring domain fields are now #ifdef protected
v4:
- Kconfig options moved to xen/arch
- removed range for CONFIG_NR_LLC_COLORS
- added "llc_coloring_enabled" global to later implement the boot-time
  switch
- added domain_create_llc_colored() to be able to pass colors
- added is_domain_llc_colored() macro
---
 SUPPORT.md                        |   7 ++
 docs/misc/cache-coloring.rst      | 116 ++++++++++++++++++++++++++++++
 docs/misc/xen-command-line.pandoc |  37 ++++++++++
 xen/common/Kconfig                |  22 ++++++
 xen/common/Makefile               |   1 +
 xen/common/keyhandler.c           |   3 +
 xen/common/llc-coloring.c         | 111 ++++++++++++++++++++++++++++
 xen/common/page_alloc.c           |   3 +
 xen/include/xen/llc-coloring.h    |  36 ++++++++++
 xen/include/xen/sched.h           |   5 ++
 10 files changed, 341 insertions(+)
 create mode 100644 docs/misc/cache-coloring.rst
 create mode 100644 xen/common/llc-coloring.c
 create mode 100644 xen/include/xen/llc-coloring.h

diff --git a/SUPPORT.md b/SUPPORT.md
index e10d46d924..94048adef2 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -381,6 +381,13 @@ by maintaining multiple physical to machine (p2m) memory mappings.
     Status, x86 HVM: Tech Preview
     Status, ARM: Tech Preview
 
+### Cache coloring
+
+Allows to reserve Last Level Cache (LLC) partitions for Dom0, DomUs and Xen
+itself.
+
+    Status, Arm64: Experimental
+
 ## Resource Management
 
 ### CPU Pools
diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
new file mode 100644
index 0000000000..bcb45e9344
--- /dev/null
+++ b/docs/misc/cache-coloring.rst
@@ -0,0 +1,116 @@
+Xen cache coloring user guide
+=============================
+
+The cache coloring support in Xen allows to reserve Last Level Cache (LLC)
+partitions for Dom0, DomUs and Xen itself. Currently only ARM64 is supported.
+Cache coloring realizes per-set cache partitioning in software and is applicable
+to shared LLCs as implemented in Cortex-A53, Cortex-A72 and similar CPUs.
+
+To compile LLC coloring support set ``CONFIG_LLC_COLORING=y``.
+
+If needed, change the maximum number of colors with
+``CONFIG_MAX_LLC_COLORS_ORDER=<n>``.
+
+Runtime configuration is done via `Command line parameters`_.
+
+Background
+**********
+
+Cache hierarchy of a modern multi-core CPU typically has first levels dedicated
+to each core (hence using multiple cache units), while the last level is shared
+among all of them. Such configuration implies that memory operations on one
+core (e.g. running a DomU) are able to generate interference on another core
+(e.g. hosting another DomU). Cache coloring realizes per-set cache-partitioning
+in software and mitigates this, guaranteeing more predictable performances for
+memory accesses.
+Software-based cache coloring is particularly useful in those situations where
+no hardware mechanisms (e.g., DSU-based way partitioning) are available to
+partition caches. This is the case for e.g., Cortex-A53, A57 and A72 CPUs that
+feature a L2 LLC cache shared among all cores.
+
+The key concept underlying cache coloring is a fragmentation of the memory
+space into a set of sub-spaces called colors that are mapped to disjoint cache
+partitions. Technically, the whole memory space is first divided into a number
+of subsequent regions. Then each region is in turn divided into a number of
+subsequent sub-colors. The generic i-th color is then obtained by all the
+i-th sub-colors in each region.
+
+::
+
+                            Region j            Region j+1
+                .....................   ............
+                .                     . .
+                .                       .
+            _ _ _______________ _ _____________________ _ _
+                |     |     |     |     |     |     |
+                | c_0 | c_1 |     | c_n | c_0 | c_1 |
+           _ _ _|_____|_____|_ _ _|_____|_____|_____|_ _ _
+                    :                       :
+                    :                       :...         ... .
+                    :                            color 0
+                    :...........................         ... .
+                                                :
+          . . ..................................:
+
+How colors are actually defined depends on the function that maps memory to
+cache lines. In case of physically-indexed, physically-tagged caches with linear
+mapping, the set index is found by extracting some contiguous bits from the
+physical address. This allows colors to be defined as shown in figure: they
+appear in memory as subsequent blocks of equal size and repeats themselves after
+``n`` different colors, where ``n`` is the total number of colors.
+
+If some kind of bit shuffling appears in the mapping function, then colors
+assume a different layout in memory. Those kind of caches aren't supported by
+the current implementation.
+
+**Note**: Finding the exact cache mapping function can be a really difficult
+task since it's not always documented in the CPU manual. As said Cortex-A53, A57
+and A72 are known to work with the current implementation.
+
+How to compute the number of colors
+###################################
+
+Given the linear mapping from physical memory to cache lines for granted, the
+number of available colors for a specific platform is computed using three
+parameters:
+
+- the size of the LLC.
+- the number of the LLC ways.
+- the page size used by Xen.
+
+The first two parameters can be found in the processor manual, while the third
+one is the minimum mapping granularity. Dividing the cache size by the number of
+its ways we obtain the size of a way. Dividing this number by the page size,
+the number of total cache colors is found. So for example an Arm Cortex-A53
+with a 16-ways associative 1 MiB LLC can isolate up to 16 colors when pages are
+4 KiB in size.
+
+Effective colors assignment
+###########################
+
+When assigning colors, if one wants to avoid cache interference between two
+domains, different colors needs to be used for their memory.
+
+Command line parameters
+***********************
+
+Specific documentation is available at `docs/misc/xen-command-line.pandoc`.
+
++----------------------+-------------------------------+
+| **Parameter**        | **Description**               |
++----------------------+-------------------------------+
+| ``llc-coloring``     | Enable coloring at runtime    |
++----------------------+-------------------------------+
+| ``llc-size``         | Set the LLC size              |
++----------------------+-------------------------------+
+| ``llc-nr-ways``      | Set the LLC number of ways    |
++----------------------+-------------------------------+
+
+Auto-probing of LLC specs
+#########################
+
+LLC size and number of ways are probed automatically by default.
+
+LLC specs can be manually set via the above command line parameters. This
+bypasses any auto-probing and it's used to overcome failing situations, such as
+flawed probing logic, or for debugging/testing purposes.
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index e760f3266e..906822a491 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1706,6 +1706,43 @@ This option is intended for debugging purposes only.  Enable MSR_DEBUGCTL.LBR
 in hypervisor context to be able to dump the Last Interrupt/Exception To/From
 record with other registers.
 
+### llc-coloring
+> `= <boolean>`
+
+> Default: `false`
+
+Flag to enable or disable LLC coloring support at runtime. This option is
+available only when `CONFIG_LLC_COLORING` is enabled. See the general
+cache coloring documentation for more info.
+
+### llc-nr-ways
+> `= <integer>`
+
+> Default: `Obtained from hardware`
+
+Specify the number of ways of the Last Level Cache. This option is available
+only when `CONFIG_LLC_COLORING` is enabled. LLC size and number of ways are used
+to find the number of supported cache colors. By default the value is
+automatically computed by probing the hardware, but in case of specific needs,
+it can be manually set. Those include failing probing and debugging/testing
+purposes so that it's possible to emulate platforms with different number of
+supported colors. If set, also "llc-size" must be set, otherwise the default
+will be used. Note that using these two options implies "llc-coloring=on".
+
+### llc-size
+> `= <size>`
+
+> Default: `Obtained from hardware`
+
+Specify the size of the Last Level Cache. This option is available only when
+`CONFIG_LLC_COLORING` is enabled. LLC size and number of ways are used to find
+the number of supported cache colors. By default the value is automatically
+computed by probing the hardware, but in case of specific needs, it can be
+manually set. Those include failing probing and debugging/testing purposes so
+that it's possible to emulate platforms with different number of supported
+colors. If set, also "llc-nr-ways" must be set, otherwise the default will be
+used. Note that using these two options implies "llc-coloring=on".
+
 ### lock-depth-size
 > `= <integer>`
 
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index cff3166ff9..276ef4e0f9 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -71,6 +71,9 @@ config HAS_IOPORTS
 config HAS_KEXEC
 	bool
 
+config HAS_LLC_COLORING
+	bool
+
 config HAS_PIRQ
 	bool
 
@@ -513,4 +516,23 @@ config TRACEBUFFER
 	  to be collected at run time for debugging or performance analysis.
 	  Memory and execution overhead when not active is minimal.
 
+config LLC_COLORING
+	bool "Last Level Cache (LLC) coloring" if EXPERT
+	depends on HAS_LLC_COLORING
+	depends on !NUMA
+
+config MAX_LLC_COLORS_ORDER
+	int "Maximum number of LLC colors (base-2 exponent)"
+	range 1 10
+	default 7
+	depends on LLC_COLORING
+	help
+	  Controls the build-time size of various arrays associated with LLC
+	  coloring. The value is a base-2 exponent. Refer to cache coloring
+	  documentation for how to compute the number of colors supported by the
+	  platform. This is only an upper bound. The runtime value is autocomputed
+	  or manually set via cmdline parameters.
+	  The default value corresponds to an 8 MiB 16-ways LLC, which should be
+	  more than what's needed in the general case.
+
 endmenu
diff --git a/xen/common/Makefile b/xen/common/Makefile
index d512cad524..7aa74a74ce 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -23,6 +23,7 @@ obj-y += keyhandler.o
 obj-$(CONFIG_KEXEC) += kexec.o
 obj-$(CONFIG_KEXEC) += kimage.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o livepatch_elf.o
+obj-$(CONFIG_LLC_COLORING) += llc-coloring.o
 obj-$(CONFIG_MEM_ACCESS) += mem_access.o
 obj-y += memory.o
 obj-y += multicall.o
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 127ca50696..778f93e063 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -5,6 +5,7 @@
 #include <asm/regs.h>
 #include <xen/delay.h>
 #include <xen/keyhandler.h>
+#include <xen/llc-coloring.h>
 #include <xen/param.h>
 #include <xen/shutdown.h>
 #include <xen/event.h>
@@ -303,6 +304,8 @@ static void cf_check dump_domains(unsigned char key)
 
         arch_dump_domain_info(d);
 
+        domain_dump_llc_colors(d);
+
         rangeset_domain_printk(d);
 
         dump_pageframe_info(d);
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
new file mode 100644
index 0000000000..65ed96603a
--- /dev/null
+++ b/xen/common/llc-coloring.c
@@ -0,0 +1,111 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Last Level Cache (LLC) coloring common code
+ *
+ * Copyright (C) 2022 Xilinx Inc.
+ */
+#include <xen/keyhandler.h>
+#include <xen/llc-coloring.h>
+#include <xen/param.h>
+
+#define NR_LLC_COLORS          (1 << CONFIG_MAX_LLC_COLORS_ORDER)
+
+static bool __ro_after_init llc_coloring_enabled;
+boolean_param("llc-coloring", llc_coloring_enabled);
+
+static unsigned int __initdata llc_size;
+size_param("llc-size", llc_size);
+static unsigned int __initdata llc_nr_ways;
+integer_param("llc-nr-ways", llc_nr_ways);
+/* Number of colors available in the LLC */
+static unsigned int __ro_after_init max_nr_colors;
+
+static void print_colors(const unsigned int *colors, unsigned int num_colors)
+{
+    unsigned int i;
+
+    printk("{ ");
+    for ( i = 0; i < num_colors; i++ )
+    {
+        unsigned int start = colors[i], end = start;
+
+        printk("%u", start);
+
+        for ( ; i < num_colors - 1 && end + 1 == colors[i + 1]; i++, end++ )
+            ;
+
+        if ( start != end )
+            printk("-%u", end);
+
+        if ( i < num_colors - 1 )
+            printk(", ");
+    }
+    printk(" }\n");
+}
+
+void __init llc_coloring_init(void)
+{
+    unsigned int way_size;
+
+    if ( llc_size && llc_nr_ways )
+    {
+        llc_coloring_enabled = true;
+        way_size = llc_size / llc_nr_ways;
+    }
+    else if ( !llc_coloring_enabled )
+        return;
+    else
+    {
+        way_size = get_llc_way_size();
+        if ( !way_size )
+            panic("LLC probing failed and 'llc-size' or 'llc-nr-ways' missing\n");
+    }
+
+    /*
+     * The maximum number of colors must be a power of 2 in order to correctly
+     * map them to bits of an address.
+     */
+    max_nr_colors = way_size >> PAGE_SHIFT;
+
+    if ( max_nr_colors & (max_nr_colors - 1) )
+        panic("Number of LLC colors (%u) isn't a power of 2\n", max_nr_colors);
+
+    if ( max_nr_colors > NR_LLC_COLORS )
+    {
+        printk(XENLOG_WARNING
+               "Number of LLC colors (%u) too big. Using configured max %u\n",
+               max_nr_colors, NR_LLC_COLORS);
+        max_nr_colors = NR_LLC_COLORS;
+    } else if ( max_nr_colors < 2 )
+        panic("Number of LLC colors %u < 2\n", max_nr_colors);
+
+    arch_llc_coloring_init();
+}
+
+void dump_llc_coloring_info(void)
+{
+    if ( !llc_coloring_enabled )
+        return;
+
+    printk("LLC coloring info:\n");
+    printk("    Number of LLC colors supported: %u\n", max_nr_colors);
+}
+
+void domain_dump_llc_colors(const struct domain *d)
+{
+    if ( !llc_coloring_enabled )
+        return;
+
+    printk("%u LLC colors: ", d->num_llc_colors);
+    print_colors(d->llc_colors, d->num_llc_colors);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 7c1bdfc046..d2032a79b0 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -126,6 +126,7 @@
 #include <xen/irq.h>
 #include <xen/keyhandler.h>
 #include <xen/lib.h>
+#include <xen/llc-coloring.h>
 #include <xen/mm.h>
 #include <xen/nodemask.h>
 #include <xen/numa.h>
@@ -2625,6 +2626,8 @@ static void cf_check pagealloc_info(unsigned char key)
     }
 
     printk("    Dom heap: %lukB free\n", total << (PAGE_SHIFT-10));
+
+    dump_llc_coloring_info();
 }
 
 static __init int cf_check pagealloc_keyhandler_init(void)
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
new file mode 100644
index 0000000000..c60c8050c5
--- /dev/null
+++ b/xen/include/xen/llc-coloring.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Last Level Cache (LLC) coloring common header
+ *
+ * Copyright (C) 2022 Xilinx Inc.
+ */
+#ifndef __COLORING_H__
+#define __COLORING_H__
+
+#include <xen/sched.h>
+#include <public/domctl.h>
+
+#ifdef CONFIG_LLC_COLORING
+void llc_coloring_init(void);
+void dump_llc_coloring_info(void);
+void domain_dump_llc_colors(const struct domain *d);
+#else
+static inline void llc_coloring_init(void) {}
+static inline void dump_llc_coloring_info(void) {}
+static inline void domain_dump_llc_colors(const struct domain *d) {}
+#endif
+
+unsigned int get_llc_way_size(void);
+void arch_llc_coloring_init(void);
+
+#endif /* __COLORING_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b841995..1bdb4767dd 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -629,6 +629,11 @@ struct domain
 
     /* Holding CDF_* constant. Internal flags for domain creation. */
     unsigned int cdf;
+
+#ifdef CONFIG_LLC_COLORING
+    unsigned int num_llc_colors;
+    const unsigned int *llc_colors;
+#endif
 };
 
 static inline struct page_list_head *page_to_list(
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715948.1117941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006kn-MB; Thu, 02 May 2024 16:55:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715948.1117941; Thu, 02 May 2024 16:55:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zih-0006ij-HT; Thu, 02 May 2024 16:55:43 +0000
Received: by outflank-mailman (input) for mailman id 715948;
 Thu, 02 May 2024 16:55:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zif-0006N3-VL
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ce01ee55-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:39 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-572babec735so938445a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:39 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce01ee55-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668939; x=1715273739; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=MCgujxNjMSkmhKzG/UmU41GDMYeB6voszmPxr4Qdo+o=;
        b=Yl1jixmoGr0qtFNK9iMplMDmSp+DGO0hnjggMSKf/sqMTMZSDB7xOAnausgc5rCZE8
         fo9fpAD+1o0vFSfdt617ommsQaETQByiSL/qPaCbAmKQPSMlo/NsTlE5lbNku5JRn5kM
         9RReyMmnK0N2U1g10osGaSfARxqRyitC2ApoQT+zdsKtpEeBbqASDh5PdKpdOIn0PA+1
         HGD6U0TAveG9odokZ8sVSivxBjuIkMbwZORwmkXdqFFiCbzFyFlFnRit2LeWRfAJZ9lX
         XEYkE0ShYyllGrgO48kNwG9xRsmW2X0KWIkzcTT6Sk9JnIgYiCdPEhM2MBbGm/lQvsq0
         up9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668939; x=1715273739;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=MCgujxNjMSkmhKzG/UmU41GDMYeB6voszmPxr4Qdo+o=;
        b=UnGxarK5N1LatJpnbiZYnbQSEsUiyurEcZ14Txe403xsexk6DXYA9ql5LDdi0NN4Hk
         2bHxHVpsFCwsngGyWfF4PVQwNPvy0BKj8eb7liMrS1WDAZM6jJXh8wBZKevsR/NnSUZY
         vo2EYJLTgA5bPefmeA3+LJ2qsPozSbmxAGebT61+UCZzo/HvFJBjxAqOLpGfuJ4IqDZ2
         kQBz+PT+SwnnCM458n2DdAraoT9haiTDYHd1lT5zY9HmRsWvczFEAi2SSGEyxnwKSL1X
         cv6TcCAZAN1RaWtGYuCwsuzHrShRln8dVjCl3AzyGocsbZibk6DVAg+v+qFlwy/XLtYZ
         4HwQ==
X-Gm-Message-State: AOJu0YxAK4tRqGmYtgRqRGJr5uQMmzQ5DSKwq8ERb7eD9R5Pg2jzhXzf
	ew6uhQwFmRrjXzjo6bsLurKvRNCYEdGIq8QZc9WhI7+sjQZjig4fEGaXvsFrWy5wQbh6eS++ANk
	Z
X-Google-Smtp-Source: AGHT+IFUMs2AimGGb+Nekd0DlA/Awn9vWPGIcCYkVDdvV6g6eekdDqZurnlmAEDoy1p37abPu6BM/w==
X-Received: by 2002:a17:906:5604:b0:a58:fcd2:759c with SMTP id f4-20020a170906560400b00a58fcd2759cmr212910ejq.34.1714668938966;
        Thu, 02 May 2024 09:55:38 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 02/13] xen/arm: add initial support for LLC coloring on arm64
Date: Thu,  2 May 2024 18:55:22 +0200
Message-Id: <20240502165533.319988-3-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

LLC coloring needs to know the last level cache layout in order to make the
best use of it. This can be probed by inspecting the CLIDR_EL1 register,
so the Last Level is defined as the last level visible by this register.
Note that this excludes system caches in some platforms.

Static memory allocation and cache coloring are incompatible because static
memory can't be guaranteed to use only colors assigned to the domain.
Panic during DomUs creation when both are enabled.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- no changes
v7:
- only minor changes
v6:
- get_llc_way_size() now checks for at least separate I/D caches
v5:
- used - instead of _ for filenames
- moved static-mem check in this patch
- moved dom0 colors parsing in next patch
- moved color allocation and configuration in next patch
- moved check_colors() in next patch
- colors are now printed in short form
v4:
- added "llc-coloring" cmdline option for the boot-time switch
- dom0 colors are now checked during domain init as for any other domain
- fixed processor.h masks bit width
- check for overflow in parse_color_config()
- check_colors() now checks also that colors are sorted and unique
---
 docs/misc/cache-coloring.rst         | 14 +++++
 xen/arch/arm/Kconfig                 |  1 +
 xen/arch/arm/Makefile                |  1 +
 xen/arch/arm/dom0less-build.c        |  6 +++
 xen/arch/arm/include/asm/processor.h | 16 ++++++
 xen/arch/arm/llc-coloring.c          | 77 ++++++++++++++++++++++++++++
 xen/arch/arm/setup.c                 |  3 ++
 xen/common/llc-coloring.c            |  2 +-
 xen/include/xen/llc-coloring.h       |  4 ++
 9 files changed, 123 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/llc-coloring.c

diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
index bcb45e9344..404262b728 100644
--- a/docs/misc/cache-coloring.rst
+++ b/docs/misc/cache-coloring.rst
@@ -111,6 +111,20 @@ Auto-probing of LLC specs
 
 LLC size and number of ways are probed automatically by default.
 
+In the Arm implementation, this is done by inspecting the CLIDR_EL1 register.
+This means that other system caches that aren't visible there are ignored.
+
 LLC specs can be manually set via the above command line parameters. This
 bypasses any auto-probing and it's used to overcome failing situations, such as
 flawed probing logic, or for debugging/testing purposes.
+
+Known issues and limitations
+****************************
+
+"xen,static-mem" isn't supported when coloring is enabled
+#########################################################
+
+In the domain configuration, "xen,static-mem" allows memory to be statically
+allocated to the domain. This isn't possible when LLC coloring is enabled,
+because that memory can't be guaranteed to use only colors assigned to the
+domain.
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index f8139a773a..36dfdc53c4 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -8,6 +8,7 @@ config ARM_64
 	depends on !ARM_32
 	select 64BIT
 	select HAS_FAST_MULTIPLY
+	select HAS_LLC_COLORING
 
 config ARM
 	def_bool y
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 7b1350e2ef..18ae566521 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -37,6 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
 obj-y += irq.o
 obj-y += kernel.init.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o
+obj-$(CONFIG_LLC_COLORING) += llc-coloring.o
 obj-y += mem_access.o
 obj-y += mm.o
 obj-y += monitor.o
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..ca7519cf3b 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -5,6 +5,7 @@
 #include <xen/grant_table.h>
 #include <xen/iocap.h>
 #include <xen/libfdt/libfdt.h>
+#include <xen/llc-coloring.h>
 #include <xen/sched.h>
 #include <xen/serial.h>
 #include <xen/sizes.h>
@@ -880,7 +881,12 @@ void __init create_domUs(void)
             panic("No more domain IDs available\n");
 
         if ( dt_find_property(node, "xen,static-mem", NULL) )
+        {
+            if ( llc_coloring_enabled )
+                panic("LLC coloring and static memory are incompatible\n");
+
             flags |= CDF_staticmem;
+        }
 
         if ( dt_property_read_bool(node, "direct-map") )
         {
diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h
index 8e02410465..ef33ea198c 100644
--- a/xen/arch/arm/include/asm/processor.h
+++ b/xen/arch/arm/include/asm/processor.h
@@ -18,6 +18,22 @@
 #define CTR_IDC_SHIFT       28
 #define CTR_DIC_SHIFT       29
 
+/* CCSIDR Current Cache Size ID Register */
+#define CCSIDR_LINESIZE_MASK            _AC(0x7, UL)
+#define CCSIDR_NUMSETS_SHIFT            13
+#define CCSIDR_NUMSETS_MASK             _AC(0x3fff, UL)
+#define CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX 32
+#define CCSIDR_NUMSETS_MASK_FEAT_CCIDX  _AC(0xffffff, UL)
+
+/* CSSELR Cache Size Selection Register */
+#define CSSELR_LEVEL_MASK  _AC(0x7, UL)
+#define CSSELR_LEVEL_SHIFT 1
+
+/* CLIDR Cache Level ID Register */
+#define CLIDR_CTYPEn_SHIFT(n) (3 * ((n) - 1))
+#define CLIDR_CTYPEn_MASK     _AC(0x7, UL)
+#define CLIDR_CTYPEn_LEVELS   7
+
 #define ICACHE_POLICY_VPIPT  0
 #define ICACHE_POLICY_AIVIVT 1
 #define ICACHE_POLICY_VIPT   2
diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c
new file mode 100644
index 0000000000..66c8db2baf
--- /dev/null
+++ b/xen/arch/arm/llc-coloring.c
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Last Level Cache (LLC) coloring support for ARM
+ *
+ * Copyright (C) 2022 Xilinx Inc.
+ */
+#include <xen/llc-coloring.h>
+#include <xen/types.h>
+
+#include <asm/processor.h>
+#include <asm/sysregs.h>
+
+/* Return the LLC way size by probing the hardware */
+unsigned int __init get_llc_way_size(void)
+{
+    register_t ccsidr_el1;
+    register_t clidr_el1 = READ_SYSREG(CLIDR_EL1);
+    register_t csselr_el1 = READ_SYSREG(CSSELR_EL1);
+    register_t id_aa64mmfr2_el1 = READ_SYSREG(ID_AA64MMFR2_EL1);
+    uint32_t ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT;
+    uint32_t ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK;
+    unsigned int n, line_size, num_sets;
+
+    for ( n = CLIDR_CTYPEn_LEVELS; n != 0; n-- )
+    {
+        uint8_t ctype_n = (clidr_el1 >> CLIDR_CTYPEn_SHIFT(n)) &
+                          CLIDR_CTYPEn_MASK;
+
+        /* Unified cache (see Arm ARM DDI 0487J.a D19.2.27) */
+        if ( ctype_n == 0b100 )
+            break;
+    }
+
+    if ( n == 0 )
+        return 0;
+
+    WRITE_SYSREG((n - 1) << CSSELR_LEVEL_SHIFT, CSSELR_EL1);
+    isb();
+
+    ccsidr_el1 = READ_SYSREG(CCSIDR_EL1);
+
+    /* Arm ARM: (Log2(Number of bytes in cache line)) - 4 */
+    line_size = 1U << ((ccsidr_el1 & CCSIDR_LINESIZE_MASK) + 4);
+
+    /* If FEAT_CCIDX is enabled, CCSIDR_EL1 has a different bit layout */
+    if ( (id_aa64mmfr2_el1 >> ID_AA64MMFR2_CCIDX_SHIFT) & 0x7 )
+    {
+        ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX;
+        ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK_FEAT_CCIDX;
+    }
+
+    /* Arm ARM: (Number of sets in cache) - 1 */
+    num_sets = ((ccsidr_el1 >> ccsidr_numsets_shift) & ccsidr_numsets_mask) + 1;
+
+    printk(XENLOG_INFO "LLC found: L%u (line size: %u bytes, sets num: %u)\n",
+           n, line_size, num_sets);
+
+    /* Restore value in CSSELR_EL1 */
+    WRITE_SYSREG(csselr_el1, CSSELR_EL1);
+    isb();
+
+    return line_size * num_sets;
+}
+
+void __init arch_llc_coloring_init(void)
+{
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d242674381..bbb3e1eea5 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -12,6 +12,7 @@
 #include <xen/device_tree.h>
 #include <xen/domain_page.h>
 #include <xen/grant_table.h>
+#include <xen/llc-coloring.h>
 #include <xen/types.h>
 #include <xen/string.h>
 #include <xen/serial.h>
@@ -776,6 +777,8 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
     printk("Command line: %s\n", cmdline);
     cmdline_parse(cmdline);
 
+    llc_coloring_init();
+
     setup_mm();
 
     vm_init();
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index 65ed96603a..bcc651cc10 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -10,7 +10,7 @@
 
 #define NR_LLC_COLORS          (1 << CONFIG_MAX_LLC_COLORS_ORDER)
 
-static bool __ro_after_init llc_coloring_enabled;
+bool __ro_after_init llc_coloring_enabled;
 boolean_param("llc-coloring", llc_coloring_enabled);
 
 static unsigned int __initdata llc_size;
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index c60c8050c5..67b27c995b 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -11,10 +11,14 @@
 #include <public/domctl.h>
 
 #ifdef CONFIG_LLC_COLORING
+extern bool llc_coloring_enabled;
+
 void llc_coloring_init(void);
 void dump_llc_coloring_info(void);
 void domain_dump_llc_colors(const struct domain *d);
 #else
+#define llc_coloring_enabled false
+
 static inline void llc_coloring_init(void) {}
 static inline void dump_llc_coloring_info(void) {}
 static inline void domain_dump_llc_colors(const struct domain *d) {}
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715952.1117981 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zik-0007kE-M6; Thu, 02 May 2024 16:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715952.1117981; Thu, 02 May 2024 16:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zik-0007hp-Ga; Thu, 02 May 2024 16:55:46 +0000
Received: by outflank-mailman (input) for mailman id 715952;
 Thu, 02 May 2024 16:55:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zij-0006Lf-8G
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:45 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d1013486-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:44 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a51addddbd4so947878966b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:44 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1013486-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668944; x=1715273744; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jzaTNPxmLK/IqCNOK/3y/1wQIjSUY/Zy4G4Lp5Bcs1Q=;
        b=sgDBb+zj8XWJ3PJG/IdcqIhmZwejruC7M8BY0Q/Ric7F5upCVlZBicUv1Gzjv0XWUF
         hHGYY0AEXym6WALeb62wA+jjB+jR/l0PafNaqpwEswZPs6/yEt7IfdcH0nLCt92BFTqZ
         /4zaItUxlTdta6gq4j0NcMKPvqV8dtRkyNs/qr3XC6HJoenViLo6lHaZA5AVtr+7Ov1c
         CilMIG6mXKK/fsepZ4F0ULULDvfn2fnUkiEtFHQuoMaar3g3UjDH8fIvQv3mq+ENdxCA
         0Vcyvthw+u/M0wTsKPXFZbVK6E5U4eGzigSj8BqP/5ZACF3hJ1MeN9G6siS33m0YXp/S
         ttrQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668944; x=1715273744;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jzaTNPxmLK/IqCNOK/3y/1wQIjSUY/Zy4G4Lp5Bcs1Q=;
        b=lL0yma5FdFVGiMsv73YeyxGNTAGRuQAT1hokl0il9QalF8c6A1Hr+2NuKak0AH69nm
         EMqD4TmbUKiEhCq3Yc/UEj2rNp5wKCZv6OPw8+XWfpb7VgEzNDAtttur0Lq2y0kiX3f8
         M2PqyXUNuEP3hYQgQjpO4M0u6yMED3BggmHChsCPbJtJZVqLv0W5QEPbuMB4YmXp1VyF
         JGL9b+6G2w85MVZ+zhWkkWXtjcXDipb5Jt06DKMtAZb1MTsqRAEwjdm+flKPAPzXpsKi
         uD29fQwduafemHj83m5nwMcNFkCqdrIjs/bWjzdmw2gEJ7OHb0laJhzALCzl2PRgpiKy
         KRxg==
X-Gm-Message-State: AOJu0Yz4mIlR3o7FBDw9lOJNNOdAuhDIq8rj5e1OOWTmmdvostmEM/gz
	7yuLpEvNVQMvauKZ8SpPv3bAdP5SzJC7NlJLoFFxnVaSCnXt3pLE+5rne5IUEuHD4elbvUG0pen
	U
X-Google-Smtp-Source: AGHT+IH8hfIaaSrExuqTbxGoNRBRtFoen/HYOr7HDoVeH46zsllKhxPxkt9DdQj/sz+Sb4NSSHJlqA==
X-Received: by 2002:a17:906:7fd2:b0:a51:8145:6877 with SMTP id r18-20020a1709067fd200b00a5181456877mr65635ejs.37.1714668944045;
        Thu, 02 May 2024 09:55:44 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v8 08/13] xen/page_alloc: introduce preserved page flags macro
Date: Thu,  2 May 2024 18:55:28 +0200
Message-Id: <20240502165533.319988-9-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

PGC_static and PGC_extra needs to be preserved when assigning a page.
Define a new macro that groups those flags and use it instead of or'ing
every time.

To make preserved flags even more meaningful, they are kept also when
switching state in mark_page_free().
Enforce the removal of PGC_extra before freeing new pages as this is
considered an error and can cause ASSERT violations.

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
---
v8:
- fixed PGC_extra ASSERT fail in alloc_domheap_pages() by removing PGC_extra
  before freeing
v7:
- PGC_preserved used also in mark_page_free()
v6:
- preserved_flags renamed to PGC_preserved
- PGC_preserved is used only in assign_pages()
v5:
- new patch
---
 xen/common/page_alloc.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index d2032a79b0..e71b571a3b 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -159,6 +159,7 @@
 #endif
 
 #define PGC_no_buddy_merge PGC_static
+#define PGC_preserved (PGC_extra | PGC_static)
 
 #ifndef PGT_TYPE_INFO_INITIALIZER
 #define PGT_TYPE_INFO_INITIALIZER 0
@@ -1426,11 +1427,11 @@ static bool mark_page_free(struct page_info *pg, mfn_t mfn)
     {
     case PGC_state_inuse:
         BUG_ON(pg->count_info & PGC_broken);
-        pg->count_info = PGC_state_free;
+        pg->count_info = PGC_state_free | (pg->count_info & PGC_preserved);
         break;
 
     case PGC_state_offlining:
-        pg->count_info = (pg->count_info & PGC_broken) |
+        pg->count_info = (pg->count_info & (PGC_broken | PGC_preserved)) |
                          PGC_state_offlined;
         pg_offlined = true;
         break;
@@ -2365,7 +2366,7 @@ int assign_pages(
 
         for ( i = 0; i < nr; i++ )
         {
-            ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_static)));
+            ASSERT(!(pg[i].count_info & ~PGC_preserved));
             if ( pg[i].count_info & PGC_extra )
                 extra_pages++;
         }
@@ -2425,7 +2426,7 @@ int assign_pages(
         page_set_owner(&pg[i], d);
         smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
         pg[i].count_info =
-            (pg[i].count_info & (PGC_extra | PGC_static)) | PGC_allocated | 1;
+            (pg[i].count_info & PGC_preserved) | PGC_allocated | 1;
 
         page_list_add_tail(&pg[i], page_to_list(d, &pg[i]));
     }
@@ -2484,6 +2485,11 @@ struct page_info *alloc_domheap_pages(
         }
         if ( assign_page(pg, order, d, memflags) )
         {
+            unsigned long i;
+
+            for ( i = 0; i < (1UL << order); i++ )
+                pg[i].count_info &= ~PGC_extra;
+
             free_heap_pages(pg, order, memflags & MEMF_no_scrub);
             return NULL;
         }
@@ -2538,6 +2544,7 @@ void free_domheap_pages(struct page_info *pg, unsigned int order)
                 {
                     ASSERT(d->extra_pages);
                     d->extra_pages--;
+                    pg[i].count_info &= ~PGC_extra;
                 }
             }
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:55:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:55:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715953.1117998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zim-0008DM-2o; Thu, 02 May 2024 16:55:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715953.1117998; Thu, 02 May 2024 16:55:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zil-0008Cp-Rk; Thu, 02 May 2024 16:55:47 +0000
Received: by outflank-mailman (input) for mailman id 715953;
 Thu, 02 May 2024 16:55:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zij-0006N3-K0
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:45 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d09ee2bf-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:44 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-5176f217b7bso14370030e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:44 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d09ee2bf-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668943; x=1715273743; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wPiz9qg4rfggjlyChZDk0/7m5I+1YfXXIS22BJekSDM=;
        b=nFgSGVw9nfUaItCkluN54sq0cUwTgEvjU6xqtpLXk9cQkDYscp4tuW4JUdsBixnm+N
         Yeye2kS0+ttLMNM0gO5kF874+vgg75IheBqMfGe1Z6yLQ5neWxaKkA8L0CQxvdlG3Y1N
         WFiCXZtL72XXZszqyqKMiyF+XQFQMVGTP9HOr0b2lvFhQ3jYdHmM7nIceEaxAF35WpdM
         DidBo+WOt1/15lwSlbdiMGVms59BQ8OFl+e0Alj7NCygFtilkdiUC9yOb+PzDu2aHZ+D
         suWue6iue6WffqThlloOenAdlZsKn3DguLL4XE4KE5PwBxMeySBZ+n/FvLAZTyTrLKOM
         sWPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668943; x=1715273743;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wPiz9qg4rfggjlyChZDk0/7m5I+1YfXXIS22BJekSDM=;
        b=acH4kB8YzlNN7UMt4ZGlUzjrK7nV+pCfA3bTcA08HfsBHWGgCClfvsBY4KYEGAJfeD
         BnHgrDzJ7Py+GSZsOZIZlcjbMpokT/5XAWp32I1sJa4JbrtyzV741UzOwZBXh2UcmBiR
         OJFIseiuZQe5lOlTbSg5Wd4zv+VhxyAq11KNUM78doQ9KEDSEHExMWXhhM4ACOG8KgsW
         2+qkK7FQOHUIFCKHp+jHWC4KNzxia2/2kThx9jX7LPoY5R3IHvD7qFIkaPSgrvMWahu5
         rShLWBLZXQm62HhhcZR/5p9E4cei+E0gIr4Fjx1oSJN7TL6oI5/OoK6abI0pqVU56rvI
         66Kg==
X-Gm-Message-State: AOJu0YyZ/XzaCz/0cevTpytCcGl9mGoGKQHafIZ7dr63ga254UQhguw9
	a357cz2VRxsagFxD9K8EORedbGC0EAaKTzkbrjO7n/hYHp/XcSfRMGDSacjIldC9qY031SVFtf9
	n
X-Google-Smtp-Source: AGHT+IH3s4EhQ3jyWS2fovvunTeH+2VNzXP7zab5Chz7JFeYqEuat8WHR2Ki4VySNT+9d/sxhbjarg==
X-Received: by 2002:a05:6512:36cf:b0:51d:4c8a:bbdb with SMTP id e15-20020a05651236cf00b0051d4c8abbdbmr310059lfs.3.1714668943288;
        Thu, 02 May 2024 09:55:43 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 07/13] xen/arm: add support for cache coloring configuration via device-tree
Date: Thu,  2 May 2024 18:55:27 +0200
Message-Id: <20240502165533.319988-8-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add the "llc-colors" Device Tree attribute to express DomUs and Dom0less
color configurations.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- fixed memory leak on error path of domain_set_llc_colors_from_str()
- realloc colors array after parsing from string to reduce memory usage
v7:
- removed alloc_colors() helper usage from domain_set_llc_colors_from_str()
v6:
- rewrote domain_set_llc_colors_from_str() to be more explicit
v5:
- static-mem check has been moved in a previous patch
- added domain_set_llc_colors_from_str() to set colors after domain creation
---
 docs/misc/arm/device-tree/booting.txt |  4 +++
 docs/misc/cache-coloring.rst          | 48 +++++++++++++++++++++++++++
 xen/arch/arm/dom0less-build.c         | 10 ++++++
 xen/common/llc-coloring.c             | 42 +++++++++++++++++++++++
 xen/include/xen/llc-coloring.h        |  1 +
 xen/include/xen/xmalloc.h             | 12 +++++++
 6 files changed, 117 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..bbe49faadc 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -162,6 +162,10 @@ with the following properties:
 
     An integer specifying the number of vcpus to allocate to the guest.
 
+- llc-colors
+    A string specifying the LLC color configuration for the guest.
+    Refer to docs/misc/cache_coloring.rst for syntax.
+
 - vpl011
 
     An empty property to enable/disable a virtual pl011 for the guest to
diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
index 26b306a0ff..9527d0511e 100644
--- a/docs/misc/cache-coloring.rst
+++ b/docs/misc/cache-coloring.rst
@@ -12,6 +12,7 @@ If needed, change the maximum number of colors with
 ``CONFIG_MAX_LLC_COLORS_ORDER=<n>``.
 
 Runtime configuration is done via `Command line parameters`_.
+For DomUs follow `DomUs configuration`_.
 
 Background
 **********
@@ -147,6 +148,53 @@ LLC specs can be manually set via the above command line parameters. This
 bypasses any auto-probing and it's used to overcome failing situations, such as
 flawed probing logic, or for debugging/testing purposes.
 
+DomUs configuration
+*******************
+
+DomUs colors can be set either in the ``xl`` configuration file (documentation
+at `docs/man/xl.cfg.pod.5.in`) or via Device Tree, also for Dom0less
+configurations (documentation at `docs/misc/arm/device-tree/booting.txt`) using
+the ``llc-colors`` option. For example:
+
+::
+
+    xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=1G dom0_max_vcpus=1 sched=null llc-coloring=on dom0-llc-colors=2-6";
+    xen,dom0-bootargs "console=hvc0 earlycon=xen earlyprintk=xen root=/dev/ram0"
+
+    dom0 {
+        compatible = "xen,linux-zimage" "xen,multiboot-module";
+        reg = <0x0 0x1000000 0x0 15858176>;
+    };
+
+    dom0-ramdisk {
+        compatible = "xen,linux-initrd" "xen,multiboot-module";
+        reg = <0x0 0x2000000 0x0 20638062>;
+    };
+
+    domU0 {
+        #address-cells = <0x1>;
+        #size-cells = <0x1>;
+        compatible = "xen,domain";
+        memory = <0x0 0x40000>;
+        llc-colors = "4-8,10,11,12";
+        cpus = <0x1>;
+        vpl011 = <0x1>;
+
+        module@2000000 {
+            compatible = "multiboot,kernel", "multiboot,module";
+            reg = <0x2000000 0xffffff>;
+            bootargs = "console=ttyAMA0";
+        };
+
+        module@30000000 {
+            compatible = "multiboot,ramdisk", "multiboot,module";
+            reg = <0x3000000 0xffffff>;
+        };
+    };
+
+**Note:** If no color configuration is provided for a domain, the default one,
+which corresponds to all available colors is used instead.
+
 Known issues and limitations
 ****************************
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index c6bc4ee59c..b77df9b642 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -807,6 +807,7 @@ void __init create_domUs(void)
     struct dt_device_node *node;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
+    const char *llc_colors_str = NULL;
 
     BUG_ON(chosen == NULL);
     dt_for_each_child_node(chosen, node)
@@ -950,6 +951,10 @@ void __init create_domUs(void)
 #endif
         }
 
+        dt_property_read_string(node, "llc-colors", &llc_colors_str);
+        if ( !llc_coloring_enabled && llc_colors_str )
+            panic("'llc-colors' found, but LLC coloring is disabled\n");
+
         /*
          * The variable max_init_domid is initialized with zero, so here it's
          * very important to use the pre-increment operator to call
@@ -960,6 +965,11 @@ void __init create_domUs(void)
             panic("Error creating domain %s (rc = %ld)\n",
                   dt_node_name(node), PTR_ERR(d));
 
+        if ( llc_coloring_enabled &&
+             (rc = domain_set_llc_colors_from_str(d, llc_colors_str)) )
+            panic("Error initializing LLC coloring for domain %s (rc = %d)\n",
+                  dt_node_name(node), rc);
+
         d->is_console = true;
         dt_device_set_used_by(node, d->domain_id);
 
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index ecfeb0ce82..000cafbc74 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -4,6 +4,7 @@
  *
  * Copyright (C) 2022 Xilinx Inc.
  */
+#include "xen/xmalloc.h"
 #include <xen/guest_access.h>
 #include <xen/keyhandler.h>
 #include <xen/llc-coloring.h>
@@ -266,6 +267,47 @@ int domain_set_llc_colors(struct domain *d,
     return 0;
 }
 
+int __init domain_set_llc_colors_from_str(struct domain *d, const char *str)
+{
+    int err;
+    unsigned int *colors, num_colors;
+
+    if ( !str )
+    {
+        domain_set_default_colors(d);
+        return 0;
+    }
+
+    colors = xmalloc_array(unsigned int, max_nr_colors);
+    if ( !colors )
+        return -ENOMEM;
+
+    err = parse_color_config(str, colors, max_nr_colors, &num_colors);
+    if ( err )
+    {
+        printk(XENLOG_ERR "Error parsing LLC color configuration");
+        xfree(colors);
+        return err;
+    }
+
+    if ( !check_colors(colors, num_colors) )
+    {
+        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
+        xfree(colors);
+        return -EINVAL;
+    }
+
+    /* Adjust the size cause it was initially set to max_nr_colors */
+    colors = xrealloc_array(colors, num_colors);
+    if ( !colors )
+        return -ENOMEM;
+
+    d->llc_colors = colors;
+    d->num_llc_colors = num_colors;
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index b3801fca00..49ebd1e712 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -31,6 +31,7 @@ void arch_llc_coloring_init(void);
 int dom0_set_llc_colors(struct domain *d);
 int domain_set_llc_colors(struct domain *d,
                           const struct xen_domctl_set_llc_colors *config);
+int domain_set_llc_colors_from_str(struct domain *d, const char *str);
 
 #endif /* __COLORING_H__ */
 
diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h
index 1b88a83be8..4bbf6ab7f3 100644
--- a/xen/include/xen/xmalloc.h
+++ b/xen/include/xen/xmalloc.h
@@ -34,6 +34,9 @@
     ((_type *)_xmalloc_array(sizeof(_type), __alignof__(_type), _num))
 #define xzalloc_array(_type, _num) \
     ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num))
+#define xrealloc_array(_ptr, _num)                                  \
+    ((typeof(_ptr))_xrealloc_array(_ptr, sizeof(typeof(*(_ptr))),   \
+                                   __alignof__(typeof(*(_ptr))), _num))
 
 /* Allocate space for a structure with a flexible array of typed objects. */
 #define xzalloc_flex_struct(type, field, nr) \
@@ -95,6 +98,15 @@ static inline void *_xzalloc_array(
     return _xzalloc(size * num, align);
 }
 
+static inline void *_xrealloc_array(
+    void *ptr, unsigned long size, unsigned long align, unsigned long num)
+{
+    /* Check for overflow. */
+    if ( size && num > UINT_MAX / size )
+        return NULL;
+    return _xrealloc(ptr, size * num, align);
+}
+
 /*
  * Pooled allocator interface.
  */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:56:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:56:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715955.1118011 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zin-0000CN-T2; Thu, 02 May 2024 16:55:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715955.1118011; Thu, 02 May 2024 16:55:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zin-0000B7-MX; Thu, 02 May 2024 16:55:49 +0000
Received: by outflank-mailman (input) for mailman id 715955;
 Thu, 02 May 2024 16:55:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zil-0006Lf-Lc
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:47 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d1b26224-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:45 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a5557e3ebcaso367139466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:45 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1b26224-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668945; x=1715273745; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=70o7z5P/u9lC96jCkfPVLMG5TJG4lGrDIqh9ufbB1CA=;
        b=uckBMfCez2QqZ6UXAyGe67ysoXGSuaU5Ottdn691+jp1N2PLaedanOX7jkuSQILqYp
         ZU3X/fvoifSavT07cOV5R+uzrO4gCnadTvliWCZ0zMy1DwJFxAsEEXuqi9CTluBvY946
         sibvyVjVVq+eToz8ctkHw5l9mOb8IHkVvJNHqQGHlInzqT5SufUThngHlAQrKuLoWRY7
         DFX2j2Rm0Sq/y3HfLbMnJC3crosBVfYnIHRF1tFnfBmA2Rvo2Gp1UXrEwt1AADDh9C8K
         2RUgVos+FbfMvwoQtDDpgg6bPplsBNwsDKPttK6o92Ck8H13KU2KpUSohn/A1CNXgfoW
         hJHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668945; x=1715273745;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=70o7z5P/u9lC96jCkfPVLMG5TJG4lGrDIqh9ufbB1CA=;
        b=ofo3pDgmvvVqaHdI6PZSXhqyMb34qCJBWMOquWN3OYG65/3Xvm03Ty0kmT28csxonz
         91aw0usPDmvOLSoLCH6Fh4XtiBHVnngljtiIG6id2pW1IA0QVTlsqeRejWJ6hcNqi3S7
         RenH07mtKV9gOHRlvDw8cRq/cbxEYkffrBNdI6+biUjToVj0cRGohmoiv1TQDOvywqAY
         ODW46Am70YySsV7qiWtVI8JY+zqQqWb6LKUfgBlK9NFVlBcjM25fTlK0CUATVUku/7DA
         37wUI1yc/AEc05ldY9qP2GuNQAd0/SaV4v07O922c6HXKp4+4zGS29/XXrbufE1zcL0s
         BBCQ==
X-Gm-Message-State: AOJu0YxWQk9ppPfsaSyQRmDSpF1eYEP5qzd6ptk8zPSacw9EiOGdKIpR
	JfrZ4qDyVE+IgvQsRcV4DXA5HOidIrfPHmQYV6l66ad+Gu3okSQYVgGl9u4HtZzi5F7TbVjy0m5
	X
X-Google-Smtp-Source: AGHT+IGOA87V2E1Mf+hIXpVnN2hqZ/qwFf4DIW7+mPHoXDUmy4CYWAlF760gNoYDHNj0pS1S1hhl8g==
X-Received: by 2002:a17:906:d293:b0:a51:abd8:8621 with SMTP id ay19-20020a170906d29300b00a51abd88621mr2580847ejb.19.1714668944927;
        Thu, 02 May 2024 09:55:44 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 09/13] xen: add cache coloring allocator for domains
Date: Thu,  2 May 2024 18:55:29 +0200
Message-Id: <20240502165533.319988-10-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new memory page allocator that implements the cache coloring mechanism.
The allocation algorithm enforces equal frequency distribution of cache
partitions, following the coloring configuration of a domain. This allows
for an even utilization of cache sets for every domain.

Pages are stored in a color-indexed array of lists. Those lists are filled
by a simple init function which computes the color of each page.
When a domain requests a page, the allocator extracts the page from the list
with the maximum number of free pages among those that the domain can access,
given its coloring configuration.

The allocator can only handle requests of order-0 pages. This allows for
easier implementation and since cache coloring targets only embedded systems,
it's assumed not to be a major problem.

The buddy allocator must coexist with the colored one because the Xen heap
isn't colored. For this reason a new Kconfig option and a command line
parameter are added to let the user set the amount of memory reserved for
the buddy allocator. Even when cache coloring is enabled, this memory
isn't managed by the colored allocator.

Colored heap information is dumped in the dump_heap() debug-key function.

Based on original work from: Luca Miccio <lucmiccio@gmail.com>

Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
---
v8:
- requests that uses MEMF_* flags that can't be served are now going to fail
- free_color_heap_page() is called directly from free_heap_pages()
v7:
- requests to alloc_color_heap_page() now fail if MEMF_bits is used
v6:
- colored allocator functions are now static
v5:
- Carlo Nonato as the new author
- the colored allocator balances color usage for each domain and it searches
  linearly only in the number of colors (FIXME removed)
- addedd scrub functionality
- removed stub functions (still requires some macro definition)
- addr_to_color turned to mfn_to_color for easier operations
- removed BUG_ON in init_color_heap_pages() in favor of panic()
- only non empty page lists are logged in dump_color_heap()
v4:
- moved colored allocator code after buddy allocator because it now has
  some dependencies on buddy functions
- buddy_alloc_size is now used only by the colored allocator
- fixed a bug that allowed the buddy to merge pages when they were colored
- free_color_heap_page() now calls mark_page_free()
- free_color_heap_page() uses of the frametable array for faster searches
- added FIXME comment for the linear search in free_color_heap_page()
- removed alloc_color_domheap_page() to let the colored allocator exploit
  some more buddy allocator code
- alloc_color_heap_page() now allocs min address pages first
- reduced the mess in end_boot_allocator(): use the first loop for
  init_color_heap_pages()
- fixed page_list_add_prev() (list.h) since it was doing the opposite of
  what it was supposed to do
- fixed page_list_add_prev() (non list.h) to check also for next existence
- removed unused page_list_add_next()
- moved p2m code in another patch
---
 docs/misc/cache-coloring.rst      |  37 ++++++
 docs/misc/xen-command-line.pandoc |  14 +++
 xen/arch/arm/include/asm/mm.h     |   5 +
 xen/common/Kconfig                |   8 ++
 xen/common/llc-coloring.c         |  13 +++
 xen/common/page_alloc.c           | 188 +++++++++++++++++++++++++++++-
 xen/include/xen/llc-coloring.h    |   4 +
 7 files changed, 265 insertions(+), 4 deletions(-)

diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
index 9527d0511e..f996627d0d 100644
--- a/docs/misc/cache-coloring.rst
+++ b/docs/misc/cache-coloring.rst
@@ -11,6 +11,9 @@ To compile LLC coloring support set ``CONFIG_LLC_COLORING=y``.
 If needed, change the maximum number of colors with
 ``CONFIG_MAX_LLC_COLORS_ORDER=<n>``.
 
+If needed, change the buddy allocator reserved size with
+``CONFIG_BUDDY_ALLOCATOR_SIZE=<n>``.
+
 Runtime configuration is done via `Command line parameters`_.
 For DomUs follow `DomUs configuration`_.
 
@@ -108,6 +111,8 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`.
 +----------------------+-------------------------------+
 | ``dom0-llc-colors``  | Dom0 color configuration      |
 +----------------------+-------------------------------+
+| ``buddy-alloc-size`` | Buddy allocator reserved size |
++----------------------+-------------------------------+
 
 Colors selection format
 ***********************
@@ -195,6 +200,17 @@ the ``llc-colors`` option. For example:
 **Note:** If no color configuration is provided for a domain, the default one,
 which corresponds to all available colors is used instead.
 
+Colored allocator and buddy allocator
+*************************************
+
+The colored allocator distributes pages based on color configurations of
+domains so that each domains only gets pages of its own colors.
+The colored allocator is meant as an alternative to the buddy allocator because
+its allocation policy is by definition incompatible with the generic one. Since
+the Xen heap is not colored yet, we need to support the coexistence of the two
+allocators and some memory must be left for the buddy one. Buddy memory
+reservation is configured via Kconfig or via command-line.
+
 Known issues and limitations
 ****************************
 
@@ -205,3 +221,24 @@ In the domain configuration, "xen,static-mem" allows memory to be statically
 allocated to the domain. This isn't possible when LLC coloring is enabled,
 because that memory can't be guaranteed to use only colors assigned to the
 domain.
+
+Cache coloring is intended only for embedded systems
+####################################################
+
+The current implementation aims to satisfy the need of predictability in
+embedded systems with small amount of memory to be managed in a colored way.
+Given that, some shortcuts are taken in the development. Expect worse
+performances on larger systems.
+
+Colored allocator can only make use of order-0 pages
+####################################################
+
+The cache coloring technique relies on memory mappings and on the smallest
+mapping granularity to achieve the maximum number of colors (cache partitions)
+possible. This granularity is what is normally called a page and, in Xen
+terminology, the order-0 page is the smallest one. The fairly simple
+colored allocator currently implemented, makes use only of such pages.
+It must be said that a more complex one could, in theory, adopt higher order
+pages if the colors selection contained adjacent colors. Two subsequent colors,
+for example, can be represented by an order-1 page, four colors correspond to
+an order-2 page, etc.
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 2923e1fad4..d4d1e37272 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -270,6 +270,20 @@ and not running softirqs. Reduce this if softirqs are not being run frequently
 enough. Setting this to a high value may cause boot failure, particularly if
 the NMI watchdog is also enabled.
 
+### buddy-alloc-size (arm64)
+> `= <size>`
+
+> Default: `64M`
+
+Amount of memory reserved for the buddy allocator when colored allocator is
+active. This options is available only when LLC coloring support is enabled.
+The colored allocator is meant as an alternative to the buddy allocator,
+because its allocation policy is by definition incompatible with the generic
+one. Since the Xen heap systems is not colored yet, we need to support the
+coexistence of the two allocators for now. This parameter, which is optional
+and for expert only, it's used to set the amount of memory reserved to the
+buddy allocator.
+
 ### cet
     = List of [ shstk=<bool>, ibt=<bool> ]
 
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 48538b5337..68b7754bec 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -145,6 +145,11 @@ struct page_info
 #else
 #define PGC_static     0
 #endif
+#ifdef CONFIG_LLC_COLORING
+/* Page is cache colored */
+#define _PGC_colored      PG_shift(4)
+#define PGC_colored       PG_mask(1, 4)
+#endif
 /* ... */
 /* Page is broken? */
 #define _PGC_broken       PG_shift(7)
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 276ef4e0f9..16573400ed 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -535,4 +535,12 @@ config MAX_LLC_COLORS_ORDER
 	  The default value corresponds to an 8 MiB 16-ways LLC, which should be
 	  more than what's needed in the general case.
 
+config BUDDY_ALLOCATOR_SIZE
+	int "Buddy allocator reserved memory size (MiB)"
+	default "64"
+	depends on LLC_COLORING
+	help
+	  Amount of memory reserved for the buddy allocator to serve Xen heap,
+	  working alongside the colored one.
+
 endmenu
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index 000cafbc74..c598eb4bf7 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -27,6 +27,9 @@ static unsigned int __ro_after_init default_colors[NR_LLC_COLORS];
 static unsigned int __initdata dom0_colors[NR_LLC_COLORS];
 static unsigned int __initdata dom0_num_colors;
 
+#define mfn_color_mask              (max_nr_colors - 1)
+#define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
+
 /*
  * Parse the coloring configuration given in the buf string, following the
  * syntax below.
@@ -308,6 +311,16 @@ int __init domain_set_llc_colors_from_str(struct domain *d, const char *str)
     return 0;
 }
 
+unsigned int page_to_llc_color(const struct page_info *pg)
+{
+    return mfn_to_color(page_to_mfn(pg));
+}
+
+unsigned int get_max_nr_llc_colors(void)
+{
+    return max_nr_colors;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index e71b571a3b..6a0f1262af 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -158,8 +158,12 @@
 #define PGC_static 0
 #endif
 
-#define PGC_no_buddy_merge PGC_static
-#define PGC_preserved (PGC_extra | PGC_static)
+#ifndef PGC_colored
+#define PGC_colored 0
+#endif
+
+#define PGC_no_buddy_merge (PGC_static | PGC_colored)
+#define PGC_preserved (PGC_extra | PGC_static | PGC_colored)
 
 #ifndef PGT_TYPE_INFO_INITIALIZER
 #define PGT_TYPE_INFO_INITIALIZER 0
@@ -1457,6 +1461,8 @@ static bool mark_page_free(struct page_info *pg, mfn_t mfn)
     return pg_offlined;
 }
 
+static void free_color_heap_page(struct page_info *pg, bool need_scrub);
+
 /* Free 2^@order set of pages. */
 static void free_heap_pages(
     struct page_info *pg, unsigned int order, bool need_scrub)
@@ -1481,6 +1487,14 @@ static void free_heap_pages(
             pg[i].count_info |= PGC_need_scrub;
             poison_one_page(&pg[i]);
         }
+
+        if ( pg->count_info & PGC_colored )
+        {
+            /* Colored pages can be 0-order only, so ignore it */
+            free_color_heap_page(pg, need_scrub);
+            spin_unlock(&heap_lock);
+            return;
+        }
     }
 
     avail[node][zone] += 1 << order;
@@ -1945,6 +1959,156 @@ static unsigned long avail_heap_pages(
     return free_pages;
 }
 
+/*************************
+ * COLORED SIDE-ALLOCATOR
+ *
+ * Pages are grouped by LLC color in lists which are globally referred to as the
+ * color heap. Lists are populated in end_boot_allocator().
+ * After initialization there will be N lists where N is the number of
+ * available colors on the platform.
+ */
+static struct page_list_head *__ro_after_init _color_heap;
+#define color_heap(color) (&_color_heap[color])
+
+static unsigned long *__ro_after_init free_colored_pages;
+
+/* Memory required for buddy allocator to work with colored one */
+#ifdef CONFIG_LLC_COLORING
+static unsigned long __initdata buddy_alloc_size =
+    MB(CONFIG_BUDDY_ALLOCATOR_SIZE);
+size_param("buddy-alloc-size", buddy_alloc_size);
+
+#define domain_num_llc_colors(d) (d)->num_llc_colors
+#define domain_llc_color(d, i)   (d)->llc_colors[i]
+#else
+static unsigned long __initdata buddy_alloc_size;
+
+#define domain_num_llc_colors(d) 0
+#define domain_llc_color(d, i)   0
+#endif
+
+static void free_color_heap_page(struct page_info *pg, bool need_scrub)
+{
+    unsigned int color = page_to_llc_color(pg);
+
+    free_colored_pages[color]++;
+    /*
+     * Head insertion allows re-using cache-hot pages in configurations without
+     * sharing of colors.
+     */
+    page_list_add(pg, color_heap(color));
+}
+
+static struct page_info *alloc_color_heap_page(unsigned int memflags,
+                                               const struct domain *d)
+{
+    struct page_info *pg = NULL;
+    unsigned int i, color = 0;
+    unsigned long max = 0;
+    bool need_tlbflush = false;
+    uint32_t tlbflush_timestamp = 0;
+    bool need_scrub;
+
+    if ( memflags & ~(MEMF_no_owner | MEMF_no_refcount | MEMF_no_scrub |
+                      MEMF_no_tlbflush) )
+        return NULL;
+
+    spin_lock(&heap_lock);
+
+    for ( i = 0; i < domain_num_llc_colors(d); i++ )
+    {
+        unsigned long free = free_colored_pages[domain_llc_color(d, i)];
+
+        if ( free > max )
+        {
+            color = domain_llc_color(d, i);
+            pg = page_list_first(color_heap(color));
+            max = free;
+        }
+    }
+
+    if ( !pg )
+    {
+        spin_unlock(&heap_lock);
+        return NULL;
+    }
+
+    need_scrub = pg->count_info & (PGC_need_scrub);
+    pg->count_info = PGC_state_inuse | (pg->count_info & PGC_preserved);
+    free_colored_pages[color]--;
+    page_list_del(pg, color_heap(color));
+
+    if ( !(memflags & MEMF_no_tlbflush) )
+        accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp);
+
+    init_free_page_fields(pg);
+
+    spin_unlock(&heap_lock);
+
+    if ( !(memflags & MEMF_no_scrub) )
+    {
+        if ( need_scrub )
+            scrub_one_page(pg);
+        else
+            check_one_page(pg);
+    }
+
+    if ( need_tlbflush )
+        filtered_flush_tlb_mask(tlbflush_timestamp);
+
+    flush_page_to_ram(mfn_x(page_to_mfn(pg)),
+                      !(memflags & MEMF_no_icache_flush));
+
+    return pg;
+}
+
+static void __init init_color_heap_pages(struct page_info *pg,
+                                         unsigned long nr_pages)
+{
+    unsigned int i;
+    bool need_scrub = opt_bootscrub == BOOTSCRUB_IDLE;
+
+    if ( buddy_alloc_size >= PAGE_SIZE )
+    {
+        unsigned long buddy_pages = min(PFN_DOWN(buddy_alloc_size), nr_pages);
+
+        init_heap_pages(pg, buddy_pages);
+        nr_pages -= buddy_pages;
+        buddy_alloc_size -= buddy_pages << PAGE_SHIFT;
+        pg += buddy_pages;
+    }
+
+    if ( !_color_heap )
+    {
+        unsigned int max_nr_colors = get_max_nr_llc_colors();
+
+        _color_heap = xmalloc_array(struct page_list_head, max_nr_colors);
+        free_colored_pages = xzalloc_array(unsigned long, max_nr_colors);
+        if ( !_color_heap || !free_colored_pages )
+            panic("Can't allocate colored heap. Buddy reserved size is too low");
+
+        for ( i = 0; i < max_nr_colors; i++ )
+            INIT_PAGE_LIST_HEAD(color_heap(i));
+    }
+
+    for ( i = 0; i < nr_pages; i++ )
+    {
+        pg[i].count_info = PGC_colored;
+        free_color_heap_page(&pg[i], need_scrub);
+    }
+}
+
+static void dump_color_heap(void)
+{
+    unsigned int color;
+
+    printk("Dumping color heap info\n");
+    for ( color = 0; color < get_max_nr_llc_colors(); color++ )
+        if ( free_colored_pages[color] > 0 )
+            printk("Color heap[%u]: %lu pages\n",
+                   color, free_colored_pages[color]);
+}
+
 void __init end_boot_allocator(void)
 {
     unsigned int i;
@@ -1964,7 +2128,13 @@ void __init end_boot_allocator(void)
     for ( i = nr_bootmem_regions; i-- > 0; )
     {
         struct bootmem_region *r = &bootmem_region_list[i];
-        if ( r->s < r->e )
+
+        if ( r->s >= r->e )
+            continue;
+
+        if ( llc_coloring_enabled )
+            init_color_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s);
+        else
             init_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s);
     }
     nr_bootmem_regions = 0;
@@ -2460,7 +2630,14 @@ struct page_info *alloc_domheap_pages(
     if ( memflags & MEMF_no_owner )
         memflags |= MEMF_no_refcount;
 
-    if ( !dma_bitsize )
+    /* Only domains are supported for coloring */
+    if ( d && llc_coloring_enabled )
+    {
+        /* Colored allocation must be done on 0 order */
+        if ( order || (pg = alloc_color_heap_page(memflags, d)) == NULL )
+            return NULL;
+    }
+    else if ( !dma_bitsize )
         memflags &= ~MEMF_no_dma;
     else if ( (dma_zone = bits_to_zone(dma_bitsize)) < zone_hi )
         pg = alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d);
@@ -2683,6 +2860,9 @@ static void cf_check dump_heap(unsigned char key)
             continue;
         printk("Node %d has %lu unscrubbed pages\n", i, node_need_scrub[i]);
     }
+
+    if ( llc_coloring_enabled )
+        dump_color_heap();
 }
 
 static __init int cf_check register_heap_trigger(void)
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index 49ebd1e712..7f8218bfb2 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -33,6 +33,10 @@ int domain_set_llc_colors(struct domain *d,
                           const struct xen_domctl_set_llc_colors *config);
 int domain_set_llc_colors_from_str(struct domain *d, const char *str);
 
+struct page_info;
+unsigned int page_to_llc_color(const struct page_info *pg);
+unsigned int get_max_nr_llc_colors(void);
+
 #endif /* __COLORING_H__ */
 
 /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:56:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:56:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715954.1118014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zio-0000GK-9b; Thu, 02 May 2024 16:55:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715954.1118014; Thu, 02 May 2024 16:55:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zio-0000Em-1o; Thu, 02 May 2024 16:55:50 +0000
Received: by outflank-mailman (input) for mailman id 715954;
 Thu, 02 May 2024 16:55:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zil-0006N3-Ob
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:47 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1eae1c0-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:46 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a58a36008ceso1024925566b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:46 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1eae1c0-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668945; x=1715273745; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=F0HjLGde2czBrFNRqDQC6Fk0V43qRnD7BBzryFWROfc=;
        b=IvPCEZixSLI1xDOU6cbItVZYOj02qlFkX/iDCbyJeqIFEBpy1Q+VIVicj5+L/QIc44
         BSXZmd/I1mD70SiMImkXQdexfD6my7rNex0qm3KlKX6iHWhY8ZOYeFPd+o84YvBe3oRA
         jlGMlhAeVcFO8xlqO4MfRJOS8Zla4cbyZRCHffydAkPNFwelQ3CAYEpyD6iUxOBaCv/b
         +fHMutPn2+1kytFBBbmV+6o9jRzLl0L/Apz0PigGPiHgpchDTGqVwU3n9rzVkI0WWAuR
         Y1OV3VuFCbwRo2T2fFTHsGQ3iw5g2E9MRtBKmiwEekjFy+TnqLipsRBPZkxJ2xk/8BqV
         Wjfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668945; x=1715273745;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=F0HjLGde2czBrFNRqDQC6Fk0V43qRnD7BBzryFWROfc=;
        b=i5QCRtl/eGMHcKU5Bx3cWdTxw1S9l/PtMP0riRg8wbqtOSHAgsMLWTNALaGLyuV+aA
         7GWuCAFY6zFqFTwAj9SGd6Ghnnhjzs6Ti/LHPHhHzinfYslf488aoBntuFbHK0/N9kp6
         LdjjXEO2mptp6SIqsNQIkz2uByduka6O5QgWrw04XrdYPAFy+Wl+oKMyicqTmzoYJqZE
         VP28onXFtDQoYe8x2M3hDkCHYicXw8XXAv2UcKlpIT50/1VfcHDbV3AHMT0z5im4BDEh
         BC+F2LdOoqLa3M33zvEX+x3UGLDlJTJe35Mu7d0E3RdZOxiGmiz5cEcTy2R63DQ4gTE/
         Odkw==
X-Gm-Message-State: AOJu0YzWXoU2tO6pI5En+IfP4vgWt+pONwpDxfWAY7wrGy+joNNOCG1e
	L8b38KcXticPlw5TEgyRJoldPDPSC2944JY8TzlARI0+jCDmcGuHGDGKyzQHoeIWuC9DPmwpwXG
	c
X-Google-Smtp-Source: AGHT+IFj86uc1obXj9Kx3zkvsvH8RpMBzwS/P2urE04hZd8oYKwJRTnSvEd+QNKgfKfXBfspNETP/w==
X-Received: by 2002:a17:906:eca8:b0:a58:c639:9518 with SMTP id qh8-20020a170906eca800b00a58c6399518mr47832ejb.76.1714668945596;
        Thu, 02 May 2024 09:55:45 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v8 10/13] xen/arm: use domain memory to allocate p2m page tables
Date: Thu,  2 May 2024 18:55:30 +0200
Message-Id: <20240502165533.319988-11-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Cache colored domains can benefit from having p2m page tables allocated
with the same coloring schema so that isolation can be achieved also for
those kind of memory accesses.
In order to do that, the domain struct is passed to the allocator and the
MEMF_no_owner flag is used.

This will be useful also when NUMA will be supported on Arm.

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Acked-by: Julien Grall <julien@xen.org>
---
v8:
- no changes
v7:
- no changes
v6:
- Carlo Nonato as the only signed-off-by
v5:
- new patch
---
 xen/arch/arm/mmu/p2m.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 41fcca011c..d02a478cb8 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -32,7 +32,7 @@ static struct page_info *p2m_alloc_page(struct domain *d)
      */
     if ( is_hardware_domain(d) )
     {
-        pg = alloc_domheap_page(NULL, 0);
+        pg = alloc_domheap_page(d, MEMF_no_owner);
         if ( pg == NULL )
             printk(XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n");
     }
@@ -81,7 +81,7 @@ int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted)
         if ( d->arch.paging.p2m_total_pages < pages )
         {
             /* Need to allocate more memory from domheap */
-            pg = alloc_domheap_page(NULL, 0);
+            pg = alloc_domheap_page(d, MEMF_no_owner);
             if ( pg == NULL )
             {
                 printk(XENLOG_ERR "Failed to allocate P2M pages.\n");
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:56:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:56:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715956.1118021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zip-0000PR-20; Thu, 02 May 2024 16:55:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715956.1118021; Thu, 02 May 2024 16:55:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zio-0000NP-MH; Thu, 02 May 2024 16:55:50 +0000
Received: by outflank-mailman (input) for mailman id 715956;
 Thu, 02 May 2024 16:55:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zim-0006N3-Ow
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:48 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d254c69c-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:46 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a58a36008ceso1024928566b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:46 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d254c69c-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668946; x=1715273746; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=N2LM23qNbULSHEplCwn2ygOn0A5pB1f/lEILoWDwehY=;
        b=EdhfhOOl01LJETvsGwmmXcWY/iJqkxkb1SF6GlwzkCVjor1eauEh/lWdehSJx7a0x9
         OY1l/w3PrF+/WnydKz7nyyloQw3UN5j4bvbVTASYnUHcbKMO8mVMHX+eiINN1gtVTRTS
         CQZGsggcUbEurwfTOt21+pDImh56p6k9bHy6wotISDRX5voS3yKhlxsy5zRpDA5Sa7H+
         DcQaa15gmFdPQSfdSwYPvfgw9l2Jx/zN7M64jUmoA1VazzIe/iwvgVzApT6VAhLvyN8M
         hsVpw862VfuqMq3J9HYVySbVAKtI9cE1ati9KbH2I0LU9bmNqJnsKlDskPZSW85w/krR
         YYNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668946; x=1715273746;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=N2LM23qNbULSHEplCwn2ygOn0A5pB1f/lEILoWDwehY=;
        b=A8063wtrAGAol44CPP3ppYahdjkwdtyx86X3hdce3TOdPWuUUEtGClCedLvvr3IZhC
         BiU5KEftGYQFtAR89lrAdeamVRyA+JHCC0CPsu8fdCkp7mca52CQjteQPPfQxXfhSRBr
         3GcABCzCtSmAL4YLRO+emOo+TwXU+76o2sA/byjoC0kCl05zwUXUftC5MftHpAfBmG7W
         O0p3z1GtyzgIylJivrIaJ+unp/wSo0Z/UBQe/1iXdedC5aKM3UnVw8/BgDKhZMIuWf0k
         KEh5Z2X3BGNUfwBDECu28CFPZyy1M/iiSf33kLlasU9vAU3yAfpLbIIHFzuo/7YuS4ur
         Oe2Q==
X-Gm-Message-State: AOJu0Yxv8Or+oNWTg9ptj3naAvhMHjLxyH9uJNw/tmu5eGDuEV5ybqj3
	Yg/xAIK+51MJVI7HStiXyNeIHQHu3E06lMvLi/wT1t4MHZ+QpFEqaytnkZcGHjTHXH1eaRL/n+n
	Q
X-Google-Smtp-Source: AGHT+IF4jfiVFArle54ykP0fTOLLkWR9rD9jCanhWbQyFsRvUeSfTo1P5MXi9h0x0Akki+DHOaZYOA==
X-Received: by 2002:a17:906:a159:b0:a58:e71d:d74 with SMTP id bu25-20020a170906a15900b00a58e71d0d74mr73203ejb.13.1714668946329;
        Thu, 02 May 2024 09:55:46 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Luca Miccio <lucmiccio@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Marco Solieri <marco.solieri@minervasys.tech>,
	Carlo Nonato <carlo.nonato@minervasys.tech>
Subject: [PATCH v8 11/13] xen/arm: add Xen cache colors command line parameter
Date: Thu,  2 May 2024 18:55:31 +0200
Message-Id: <20240502165533.319988-12-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Luca Miccio <lucmiccio@gmail.com>

Add a new command line parameter to configure Xen cache colors.
These colors are dumped together with other coloring info.

Benchmarking the VM interrupt response time provides an estimation of
LLC usage by Xen's most latency-critical runtime task. Results on Arm
Cortex-A53 on Xilinx Zynq UltraScale+ XCZU9EG show that one color, which
reserves 64 KiB of L2, is enough to attain best responsiveness:
- Xen 1 color latency:  3.1 us
- Xen 2 color latency:  3.1 us

Since this is the most common target for Arm cache coloring, the default
amount of Xen colors is set to one.

More colors are instead very likely to be needed on processors whose L1
cache is physically-indexed and physically-tagged, such as Cortex-A57.
In such cases, coloring applies to L1 also, and there typically are two
distinct L1-colors. Therefore, reserving only one color for Xen would
senselessly partitions a cache memory that is already private, i.e.
underutilize it.

Signed-off-by: Luca Miccio <lucmiccio@gmail.com>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
---
v8:
- added bound check on xen_colors in llc_coloring_init()
v7:
- removed XEN_DEFAULT_COLOR
- XEN_DEFAULT_NUM_COLORS is now used in a for loop to set xen default colors
---
 docs/misc/cache-coloring.rst      |  2 ++
 docs/misc/xen-command-line.pandoc | 10 ++++++++++
 xen/common/llc-coloring.c         | 29 +++++++++++++++++++++++++++++
 3 files changed, 41 insertions(+)

diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
index f996627d0d..ad42d6b811 100644
--- a/docs/misc/cache-coloring.rst
+++ b/docs/misc/cache-coloring.rst
@@ -113,6 +113,8 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`.
 +----------------------+-------------------------------+
 | ``buddy-alloc-size`` | Buddy allocator reserved size |
 +----------------------+-------------------------------+
+| ``xen-llc-colors``   | Xen color configuration       |
++----------------------+-------------------------------+
 
 Colors selection format
 ***********************
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index d4d1e37272..2427f865c5 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2915,6 +2915,16 @@ mode.
 **WARNING: `x2apic_phys` is deprecated and superseded by `x2apic-mode`.
 The latter takes precedence if both are set.**
 
+### xen-llc-colors (arm64)
+> `= List of [ <integer> | <integer>-<integer> ]`
+
+> Default: `0: the lowermost color`
+
+Specify Xen LLC color configuration. This options is available only when
+`CONFIG_LLC_COLORING` is enabled.
+Two colors are most likely needed on platforms where private caches are
+physically indexed, e.g. the L1 instruction cache of the Arm Cortex-A57.
+
 ### xenheap_megabytes (arm32)
 > `= <size>`
 
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index c598eb4bf7..0ddd6946f1 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -11,6 +11,7 @@
 #include <xen/param.h>
 
 #define NR_LLC_COLORS          (1 << CONFIG_MAX_LLC_COLORS_ORDER)
+#define XEN_DEFAULT_NUM_COLORS 1
 
 bool __ro_after_init llc_coloring_enabled;
 boolean_param("llc-coloring", llc_coloring_enabled);
@@ -27,6 +28,9 @@ static unsigned int __ro_after_init default_colors[NR_LLC_COLORS];
 static unsigned int __initdata dom0_colors[NR_LLC_COLORS];
 static unsigned int __initdata dom0_num_colors;
 
+static unsigned int __ro_after_init xen_colors[NR_LLC_COLORS];
+static unsigned int __ro_after_init xen_num_colors;
+
 #define mfn_color_mask              (max_nr_colors - 1)
 #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
 
@@ -85,6 +89,13 @@ static int __init parse_dom0_colors(const char *s)
 }
 custom_param("dom0-llc-colors", parse_dom0_colors);
 
+static int __init parse_xen_colors(const char *s)
+{
+    return parse_color_config(s, xen_colors, ARRAY_SIZE(xen_colors),
+                              &xen_num_colors);
+}
+custom_param("xen-llc-colors", parse_xen_colors);
+
 static void print_colors(const unsigned int *colors, unsigned int num_colors)
 {
     unsigned int i;
@@ -163,6 +174,22 @@ void __init llc_coloring_init(void)
     for ( i = 0; i < max_nr_colors; i++ )
         default_colors[i] = i;
 
+    if ( !xen_num_colors )
+    {
+        unsigned int i;
+
+        xen_num_colors = MIN(XEN_DEFAULT_NUM_COLORS, max_nr_colors);
+
+        printk(XENLOG_WARNING
+               "Xen LLC color config not found. Using first %u colors\n",
+               xen_num_colors);
+        for ( i = 0; i < xen_num_colors; i++ )
+            xen_colors[i] = i;
+    }
+    else if ( xen_num_colors > max_nr_colors ||
+              !check_colors(xen_colors, xen_num_colors) )
+        panic("Bad LLC color config for Xen\n");
+
     arch_llc_coloring_init();
 }
 
@@ -173,6 +200,8 @@ void dump_llc_coloring_info(void)
 
     printk("LLC coloring info:\n");
     printk("    Number of LLC colors supported: %u\n", max_nr_colors);
+    printk("    Xen LLC colors (%u): ", xen_num_colors);
+    print_colors(xen_colors, xen_num_colors);
 }
 
 void domain_dump_llc_colors(const struct domain *d)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:56:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:56:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715957.1118029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Ziq-0000bD-1T; Thu, 02 May 2024 16:55:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715957.1118029; Thu, 02 May 2024 16:55:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Zip-0000ZI-Je; Thu, 02 May 2024 16:55:51 +0000
Received: by outflank-mailman (input) for mailman id 715957;
 Thu, 02 May 2024 16:55:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zin-0006N3-Oz
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:49 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d2f792ba-08a4-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 18:55:48 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a595c61553cso245322166b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:48 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2f792ba-08a4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668947; x=1715273747; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=js2dKSvePAqOWBjeqArRJcnowTogkyqLrYPgrRYmXag=;
        b=RRXcZgWPMNtOTswwIbUkP7F+MWzCBWMztxI3r820+EGbYvBRm2BOB945+xFe3uMGJd
         RuQ+1ba75qFq+90sqIv/VU3LijYiEKPfikMRiExZ0P+ySe5C3oR4ykYSnVT6FnTmhA3i
         VmrGVC3SAwhBRgkPP62xIYv/W7Uu+umMIxo3DzrjPH6ck+0ZF6BRlRwSEoz48yULKmm6
         tqj2dtNz5fM2nHKpyvgr8OkUlntuHQXEUmkmQX3IoJ3Eyn60nXIWkzj3DeqIzxjl+OiF
         1VE+A0gdixMuVkYoqr0bZfd3ZGmUh933f6Xduv0fB8cd/Zc9J7KS+/0eL0aIIMdJMPra
         n4Fg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668947; x=1715273747;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=js2dKSvePAqOWBjeqArRJcnowTogkyqLrYPgrRYmXag=;
        b=TlOPt6rjg5t72vDLtsL7o+Faro/eQF0x2mhXdIZuBy3AE5ky8oF7RtqZCbBrKrB3GY
         F12r5GSZ1ClN3CipXsJuKcuNdxyAdBmv0HQJKxDCQeidd0A3O+Dvb4RK4u1uwmrMxeFS
         LVPm0JshHa0iv8k5c6IaeDTXVoVD1lAp8aJAscip3rqfxc7stIGE+gqjkGc0clipEcLl
         Y6WEv6TsY3iW1PTIDnWEUegEnQ1BTSfhUOLnwSqnUfKDwW6f5OrEYdQR1P16L7FlQgpk
         LTaxY3dWowC9KNk9OJLrFXcAA7POTdAtVVKZp9Pvn7fTcROCj2NcLedaSQwJUHOg+vpn
         Suuw==
X-Gm-Message-State: AOJu0YyXLAwjEkBhtFCn6pvQwmdG0NM/VmTqbi+M8wY4sH4csOiA09mp
	uPM9hp+A1/WiDfojJkKGIWJ2C2K5RKJsSJm2GJVYvOYoc18x/x1LsXRF1rNKLaFK/VgObsZexlQ
	W
X-Google-Smtp-Source: AGHT+IHgyNFFlUmdIrnd/ng6aOPddiEn0HB2PTkDujnG/vTVRVBCqMLVj1yOyJ1Lyw3K+l8ARC4uGg==
X-Received: by 2002:a17:906:fcc3:b0:a59:7766:e4ed with SMTP id qx3-20020a170906fcc300b00a597766e4edmr70061ejb.32.1714668947016;
        Thu, 02 May 2024 09:55:47 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v8 12/13] xen/arm: make consider_modules() available for xen relocation
Date: Thu,  2 May 2024 18:55:32 +0200
Message-Id: <20240502165533.319988-13-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Cache coloring must physically relocate Xen in order to color the hypervisor
and consider_modules() is a key function that is needed to find a new
available physical address.

672d67f339c0 ("xen/arm: Split MMU-specific setup_mm() and related code out")
moved consider_modules() under arm32. Move it to mmu/setup.c and make it
non-static so that it can be used outside.

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
---
v8:
- patch adapted to new changes to consider_modules()
v7:
- moved consider_modules() to arm/mmu/setup.c
v6:
- new patch
---
 xen/arch/arm/arm32/mmu/mm.c      | 115 +-----------------------------
 xen/arch/arm/include/asm/setup.h |   3 +
 xen/arch/arm/mmu/setup.c         | 117 +++++++++++++++++++++++++++++++
 3 files changed, 121 insertions(+), 114 deletions(-)

diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c
index 23150122f7..5d50db32eb 100644
--- a/xen/arch/arm/arm32/mmu/mm.c
+++ b/xen/arch/arm/arm32/mmu/mm.c
@@ -9,6 +9,7 @@
 #include <asm/fixmap.h>
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
+#include <asm/setup.h>
 
 static unsigned long opt_xenheap_megabytes __initdata;
 integer_param("xenheap_megabytes", opt_xenheap_megabytes);
@@ -31,120 +32,6 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,
     directmap_virt_end = XENHEAP_VIRT_START + nr_mfns * PAGE_SIZE;
 }
 
-/*
- * Returns the end address of the highest region in the range s..e
- * with required size and alignment that does not conflict with the
- * modules from first_mod to nr_modules.
- *
- * For non-recursive callers first_mod should normally be 0 (all
- * modules and Xen itself) or 1 (all modules but not Xen).
- */
-static paddr_t __init consider_modules(paddr_t s, paddr_t e,
-                                       uint32_t size, paddr_t align,
-                                       int first_mod)
-{
-    const struct membanks *reserved_mem = bootinfo_get_reserved_mem();
-#ifdef CONFIG_STATIC_SHM
-    const struct membanks *shmem = bootinfo_get_shmem();
-#endif
-    const struct bootmodules *mi = &bootinfo.modules;
-    int i;
-    int nr;
-
-    s = (s+align-1) & ~(align-1);
-    e = e & ~(align-1);
-
-    if ( s > e ||  e - s < size )
-        return 0;
-
-    /* First check the boot modules */
-    for ( i = first_mod; i < mi->nr_mods; i++ )
-    {
-        paddr_t mod_s = mi->module[i].start;
-        paddr_t mod_e = mod_s + mi->module[i].size;
-
-        if ( s < mod_e && mod_s < e )
-        {
-            mod_e = consider_modules(mod_e, e, size, align, i+1);
-            if ( mod_e )
-                return mod_e;
-
-            return consider_modules(s, mod_s, size, align, i+1);
-        }
-    }
-
-    /* Now check any fdt reserved areas. */
-
-    nr = fdt_num_mem_rsv(device_tree_flattened);
-
-    for ( ; i < mi->nr_mods + nr; i++ )
-    {
-        paddr_t mod_s, mod_e;
-
-        if ( fdt_get_mem_rsv_paddr(device_tree_flattened,
-                                   i - mi->nr_mods,
-                                   &mod_s, &mod_e ) < 0 )
-            /* If we can't read it, pretend it doesn't exist... */
-            continue;
-
-        /* fdt_get_mem_rsv_paddr returns length */
-        mod_e += mod_s;
-
-        if ( s < mod_e && mod_s < e )
-        {
-            mod_e = consider_modules(mod_e, e, size, align, i+1);
-            if ( mod_e )
-                return mod_e;
-
-            return consider_modules(s, mod_s, size, align, i+1);
-        }
-    }
-
-    /*
-     * i is the current bootmodule we are evaluating, across all
-     * possible kinds of bootmodules.
-     *
-     * When retrieving the corresponding reserved-memory addresses, we
-     * need to index the reserved_mem bank starting from 0, and only counting
-     * the reserved-memory modules. Hence, we need to use i - nr.
-     */
-    nr += mi->nr_mods;
-    for ( ; i - nr < reserved_mem->nr_banks; i++ )
-    {
-        paddr_t r_s = reserved_mem->bank[i - nr].start;
-        paddr_t r_e = r_s + reserved_mem->bank[i - nr].size;
-
-        if ( s < r_e && r_s < e )
-        {
-            r_e = consider_modules(r_e, e, size, align, i + 1);
-            if ( r_e )
-                return r_e;
-
-            return consider_modules(s, r_s, size, align, i + 1);
-        }
-    }
-
-#ifdef CONFIG_STATIC_SHM
-    nr += reserved_mem->nr_banks;
-    for ( ; i - nr < shmem->nr_banks; i++ )
-    {
-        paddr_t r_s = shmem->bank[i - nr].start;
-        paddr_t r_e = r_s + shmem->bank[i - nr].size;
-
-        if ( s < r_e && r_s < e )
-        {
-            r_e = consider_modules(r_e, e, size, align, i + 1);
-            if ( r_e )
-                return r_e;
-
-            return consider_modules(s, r_s, size, align, i + 1);
-        }
-    }
-#endif
-
-    return e;
-}
-
 /*
  * Find a contiguous region that fits in the static heap region with
  * required size and alignment, and return the end address of the region
diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
index 28fb659fe9..a72b6d5c63 100644
--- a/xen/arch/arm/include/asm/setup.h
+++ b/xen/arch/arm/include/asm/setup.h
@@ -278,6 +278,9 @@ struct init_info
     unsigned int cpuid;
 };
 
+paddr_t consider_modules(paddr_t s, paddr_t e, uint32_t size, paddr_t align,
+                         int first_mod);
+
 #endif
 /*
  * Local variables:
diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c
index f4bb424c3c..a8f93d4f82 100644
--- a/xen/arch/arm/mmu/setup.c
+++ b/xen/arch/arm/mmu/setup.c
@@ -6,7 +6,10 @@
  */
 
 #include <xen/init.h>
+#include <xen/lib.h>
 #include <xen/libfdt/libfdt.h>
+#include <xen/libfdt/libfdt-xen.h>
+#include <xen/llc-coloring.h>
 #include <xen/sections.h>
 #include <xen/sizes.h>
 #include <xen/vmap.h>
@@ -218,6 +221,120 @@ static void xen_pt_enforce_wnx(void)
     flush_xen_tlb_local();
 }
 
+/*
+ * Returns the end address of the highest region in the range s..e
+ * with required size and alignment that does not conflict with the
+ * modules from first_mod to nr_modules.
+ *
+ * For non-recursive callers first_mod should normally be 0 (all
+ * modules and Xen itself) or 1 (all modules but not Xen).
+ */
+paddr_t __init consider_modules(paddr_t s, paddr_t e,
+                                uint32_t size, paddr_t align,
+                                int first_mod)
+{
+    const struct membanks *reserved_mem = bootinfo_get_reserved_mem();
+#ifdef CONFIG_STATIC_SHM
+    const struct membanks *shmem = bootinfo_get_shmem();
+#endif
+    const struct bootmodules *mi = &bootinfo.modules;
+    int i;
+    int nr;
+
+    s = (s+align-1) & ~(align-1);
+    e = e & ~(align-1);
+
+    if ( s > e ||  e - s < size )
+        return 0;
+
+    /* First check the boot modules */
+    for ( i = first_mod; i < mi->nr_mods; i++ )
+    {
+        paddr_t mod_s = mi->module[i].start;
+        paddr_t mod_e = mod_s + mi->module[i].size;
+
+        if ( s < mod_e && mod_s < e )
+        {
+            mod_e = consider_modules(mod_e, e, size, align, i+1);
+            if ( mod_e )
+                return mod_e;
+
+            return consider_modules(s, mod_s, size, align, i+1);
+        }
+    }
+
+    /* Now check any fdt reserved areas. */
+
+    nr = fdt_num_mem_rsv(device_tree_flattened);
+
+    for ( ; i < mi->nr_mods + nr; i++ )
+    {
+        paddr_t mod_s, mod_e;
+
+        if ( fdt_get_mem_rsv_paddr(device_tree_flattened,
+                                   i - mi->nr_mods,
+                                   &mod_s, &mod_e ) < 0 )
+            /* If we can't read it, pretend it doesn't exist... */
+            continue;
+
+        /* fdt_get_mem_rsv_paddr returns length */
+        mod_e += mod_s;
+
+        if ( s < mod_e && mod_s < e )
+        {
+            mod_e = consider_modules(mod_e, e, size, align, i+1);
+            if ( mod_e )
+                return mod_e;
+
+            return consider_modules(s, mod_s, size, align, i+1);
+        }
+    }
+
+    /*
+     * i is the current bootmodule we are evaluating, across all
+     * possible kinds of bootmodules.
+     *
+     * When retrieving the corresponding reserved-memory addresses, we
+     * need to index the reserved_mem bank starting from 0, and only counting
+     * the reserved-memory modules. Hence, we need to use i - nr.
+     */
+    nr += mi->nr_mods;
+    for ( ; i - nr < reserved_mem->nr_banks; i++ )
+    {
+        paddr_t r_s = reserved_mem->bank[i - nr].start;
+        paddr_t r_e = r_s + reserved_mem->bank[i - nr].size;
+
+        if ( s < r_e && r_s < e )
+        {
+            r_e = consider_modules(r_e, e, size, align, i + 1);
+            if ( r_e )
+                return r_e;
+
+            return consider_modules(s, r_s, size, align, i + 1);
+        }
+    }
+
+#ifdef CONFIG_STATIC_SHM
+    nr += reserved_mem->nr_banks;
+    for ( ; i - nr < shmem->nr_banks; i++ )
+    {
+        paddr_t r_s = shmem->bank[i - nr].start;
+        paddr_t r_e = r_s + shmem->bank[i - nr].size;
+
+        if ( s < r_e && r_s < e )
+        {
+            r_e = consider_modules(r_e, e, size, align, i + 1);
+            if ( r_e )
+                return r_e;
+
+            return consider_modules(s, r_s, size, align, i + 1);
+        }
+    }
+#endif
+
+    return e;
+}
+
 /*
  * Boot-time pagetable setup.
  * Changes here may need matching changes in head.S
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 16:56:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 16:56:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.715958.1118035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Ziq-0000gl-Dy; Thu, 02 May 2024 16:55:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 715958.1118035; Thu, 02 May 2024 16:55:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2Ziq-0000f9-1M; Thu, 02 May 2024 16:55:52 +0000
Received: by outflank-mailman (input) for mailman id 715958;
 Thu, 02 May 2024 16:55:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8/Nh=MF=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s2Zin-0006Lf-UO
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:50 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d35bf878-08a4-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 18:55:48 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a587831809eso1015932966b.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 09:55:48 -0700 (PDT)
Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it.
 [155.185.85.8]) by smtp.gmail.com with ESMTPSA id
 mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 09:55:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d35bf878-08a4-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668948; x=1715273748; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WECgJiEg23J0x5KrTcZN6y0EvfDCy6GafmGxq0QIyEk=;
        b=lKJSluUBuvLwkgOk1Qq3XHcz7etstDbGtFKlkYZ0y/7mz47mk6V9M09vgOc+63JFSh
         zGafN/fwLNfnmXz9ZhiJwNzK03NbIjWSfuID6V8jjUILwaWyd0mmXR65EQOK1pR6dEvo
         SU6qNhs6koDQHvlQw27lLtRO5u3rXQPVqOWVOaPgnmH9np3lOu901EQhc5C57Gc0QtSI
         yHnzmvkQF9cEAU9kuOsokkJC76sjykTwxZE6TJvUaiAojHLTw3y/zSbMX0x6IADnq3oW
         wnXauT4lgMMqR/EPDPov9sFIaa013ufKQ+TH5iCzTcu65An76Srn+x7o+dbboo9FJq+u
         E8Iw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714668948; x=1715273748;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=WECgJiEg23J0x5KrTcZN6y0EvfDCy6GafmGxq0QIyEk=;
        b=UpuV5KR0nZFJOQ/h0bOyUv/9soOGoLV39HPaj1Ipb9Fa/FaVsnmzHq2F2w7+DmGm3t
         YB9MMl7HlGaTmE3w8yMxldc6AP4Nnwc1MrVbDmxQHAB9XW12qV4ELjPVj4pVN/x4neKR
         2J+JoYDRxmtkU7A5lXGSU7YEogiQM/Muo45XCFbG6ZYrneizhKBe9HHtfTBtkiTZ7vXw
         nzUhcXs/642F84yz8xyJw3IHVsQpyclMI2f1Yk+o+0SNx19jo79ZxObmbUatXKDglM7S
         JAeYroFVu83w32kfAf1+TKZJ10Gni1L0XsTEuIOxA14ozCvta3NoQJdUc2lhUjeIzFfY
         gJ+w==
X-Gm-Message-State: AOJu0YyQD/u1IZI9P0rHdpoW/j0xs9N697JnBAgfin0ayJoHSoEQnfkS
	Ga6Art1xvrnK+fLiishaH+zlJO66R4xVJu661w4ig69VaBFxH5aDUyKXEnHBMSvRX+rmpTkwLFu
	v
X-Google-Smtp-Source: AGHT+IFxhylZqKSSf0raeqaYai/+rXwGChzuHGphLorcC4+60j+kdNQeosjYOMCjdKBfd3yfWGD+5g==
X-Received: by 2002:a17:906:6c1:b0:a58:fabc:4a02 with SMTP id v1-20020a17090606c100b00a58fabc4a02mr48827ejb.39.1714668947881;
        Thu, 02 May 2024 09:55:47 -0700 (PDT)
From: Carlo Nonato <carlo.nonato@minervasys.tech>
To: xen-devel@lists.xenproject.org
Cc: andrea.bastoni@minervasys.tech,
	Carlo Nonato <carlo.nonato@minervasys.tech>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: [PATCH v8 13/13] xen/arm: add cache coloring support for Xen
Date: Thu,  2 May 2024 18:55:33 +0200
Message-Id: <20240502165533.319988-14-carlo.nonato@minervasys.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add the cache coloring support for Xen physical space.

Since Xen must be relocated to a new physical space, some relocation
functionalities must be brought back:
- the virtual address of the new space is taken from 0c18fb76323b
  ("xen/arm: Remove unused BOOT_RELOC_VIRT_START").
- relocate_xen() and get_xen_paddr() are taken from f60658c6ae47
  ("xen/arm: Stop relocating Xen").

setup_pagetables() must be adapted for coloring and for relocation. Runtime
page tables are used to map the colored space, but they are also linked in
boot tables so that the new space is temporarily available for relocation.
This implies that Xen protection must happen after the copy.

Finally, since the alternative framework needs to remap the Xen text and
inittext sections, this operation must be done in a coloring-aware way.
The function xen_remap_colored() is introduced for that.

Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
---
v8:
- moved xen_colored_map_size() to arm/llc-coloring.c
v7:
- added BUG_ON() checks to arch_llc_coloring_init() and
  create_llc_coloring_mappings()
v6:
- squashed with BOOT_RELOC_VIRT_START patch
- consider_modules() moved in another patch
- removed psci and smpboot code because of new idmap work already handles that
- moved xen_remap_colored() in alternative.c since it's only used there
- removed xen_colored_temp[] in favor of xen_xenmap[] usage for mapping
- use of boot_module_find_by_kind() to remove the need of extra parameter in
  setup_pagetables()
- moved get_xen_paddr() in arm/llc-coloring.c since it's only used there
v5:
- FIXME: consider_modules copy pasted since it got moved
v4:
- removed set_value_for_secondary() because it was wrongly cleaning cache
- relocate_xen() now calls switch_ttbr_id()
---
 xen/arch/arm/alternative.c            |  30 +++++++-
 xen/arch/arm/arm64/mmu/head.S         |  58 +++++++++++++-
 xen/arch/arm/arm64/mmu/mm.c           |  28 ++++++-
 xen/arch/arm/include/asm/mmu/layout.h |   3 +
 xen/arch/arm/llc-coloring.c           |  63 ++++++++++++++++
 xen/arch/arm/mmu/setup.c              | 104 ++++++++++++++++++++++----
 xen/arch/arm/setup.c                  |  10 ++-
 xen/common/llc-coloring.c             |  18 +++++
 xen/include/xen/llc-coloring.h        |  13 ++++
 9 files changed, 305 insertions(+), 22 deletions(-)

diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c
index 016e66978b..8ca649b55e 100644
--- a/xen/arch/arm/alternative.c
+++ b/xen/arch/arm/alternative.c
@@ -9,6 +9,7 @@
 #include <xen/init.h>
 #include <xen/types.h>
 #include <xen/kernel.h>
+#include <xen/llc-coloring.h>
 #include <xen/mm.h>
 #include <xen/vmap.h>
 #include <xen/smp.h>
@@ -191,6 +192,27 @@ static int __apply_alternatives_multi_stop(void *xenmap)
     return 0;
 }
 
+static void __init *xen_remap_colored(mfn_t xen_mfn, paddr_t xen_size)
+{
+    unsigned int i;
+    void *xenmap;
+    mfn_t *xen_colored_mfns, mfn;
+
+    xen_colored_mfns = xmalloc_array(mfn_t, xen_size >> PAGE_SHIFT);
+    if ( !xen_colored_mfns )
+        panic("Can't allocate LLC colored MFNs\n");
+
+    for_each_xen_colored_mfn ( xen_mfn, mfn, i )
+    {
+        xen_colored_mfns[i] = mfn;
+    }
+
+    xenmap = vmap(xen_colored_mfns, xen_size >> PAGE_SHIFT);
+    xfree(xen_colored_mfns);
+
+    return xenmap;
+}
+
 /*
  * This function should only be called during boot and before CPU0 jump
  * into the idle_loop.
@@ -209,8 +231,12 @@ void __init apply_alternatives_all(void)
      * The text and inittext section are read-only. So re-map Xen to
      * be able to patch the code.
      */
-    xenmap = __vmap(&xen_mfn, 1U << xen_order, 1, 1, PAGE_HYPERVISOR,
-                    VMAP_DEFAULT);
+    if ( llc_coloring_enabled )
+        xenmap = xen_remap_colored(xen_mfn, xen_size);
+    else
+        xenmap = __vmap(&xen_mfn, 1U << xen_order, 1, 1, PAGE_HYPERVISOR,
+                        VMAP_DEFAULT);
+
     /* Re-mapping Xen is not expected to fail during boot. */
     BUG_ON(!xenmap);
 
diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S
index fa40b696dd..7ad2c00fd5 100644
--- a/xen/arch/arm/arm64/mmu/head.S
+++ b/xen/arch/arm/arm64/mmu/head.S
@@ -427,6 +427,61 @@ fail:   PRINT("- Boot failed -\r\n")
         b     1b
 ENDPROC(fail)
 
+/*
+ * Copy Xen to new location and switch TTBR
+ * x0    ttbr
+ * x1    source address
+ * x2    destination address
+ * x3    length
+ *
+ * Source and destination must be word aligned, length is rounded up
+ * to a 16 byte boundary.
+ *
+ * MUST BE VERY CAREFUL when saving things to RAM over the copy
+ */
+ENTRY(relocate_xen)
+        /*
+         * Copy 16 bytes at a time using:
+         *   x9: counter
+         *   x10: data
+         *   x11: data
+         *   x12: source
+         *   x13: destination
+         */
+        mov     x9, x3
+        mov     x12, x1
+        mov     x13, x2
+
+1:      ldp     x10, x11, [x12], #16
+        stp     x10, x11, [x13], #16
+
+        subs    x9, x9, #16
+        bgt     1b
+
+        /*
+         * Flush destination from dcache using:
+         *   x9: counter
+         *   x10: step
+         *   x11: vaddr
+         *
+         * This is to ensure data is visible to the instruction cache
+         */
+        dsb   sy
+
+        mov   x9, x3
+        ldr   x10, =dcache_line_bytes /* x10 := step */
+        ldr   x10, [x10]
+        mov   x11, x2
+
+1:      dc    cvac, x11
+
+        add   x11, x11, x10
+        subs  x9, x9, x10
+        bgt   1b
+
+        /* No need for dsb/isb because they are alredy done in switch_ttbr_id */
+        b switch_ttbr_id
+
 /*
  * Switch TTBR
  *
@@ -452,7 +507,8 @@ ENTRY(switch_ttbr_id)
 
         /*
          * 5) Flush I-cache
-         * This should not be necessary but it is kept for safety.
+         * This should not be necessary in the general case, but it's needed
+         * for cache coloring because code is relocated in that case.
          */
         ic     iallu
         isb
diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 293acb67e0..076264b541 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 
 #include <xen/init.h>
+#include <xen/llc-coloring.h>
 #include <xen/mm.h>
 #include <xen/pfn.h>
 
@@ -126,27 +127,46 @@ void update_identity_mapping(bool enable)
 }
 
 extern void switch_ttbr_id(uint64_t ttbr);
+extern void relocate_xen(uint64_t ttbr, void *src, void *dst, size_t len);
 
 typedef void (switch_ttbr_fn)(uint64_t ttbr);
+typedef void (relocate_xen_fn)(uint64_t ttbr, void *src, void *dst, size_t len);
 
 void __init switch_ttbr(uint64_t ttbr)
 {
-    vaddr_t id_addr = virt_to_maddr(switch_ttbr_id);
-    switch_ttbr_fn *fn = (switch_ttbr_fn *)id_addr;
+    vaddr_t vaddr, id_addr;
     lpae_t pte;
 
+    if ( llc_coloring_enabled )
+        vaddr = (vaddr_t)relocate_xen;
+    else
+        vaddr = (vaddr_t)switch_ttbr_id;
+
+    id_addr = virt_to_maddr(vaddr);
+
     /* Enable the identity mapping in the boot page tables */
     update_identity_mapping(true);
 
     /* Enable the identity mapping in the runtime page tables */
-    pte = pte_of_xenaddr((vaddr_t)switch_ttbr_id);
+    pte = pte_of_xenaddr(vaddr);
     pte.pt.table = 1;
     pte.pt.xn = 0;
     pte.pt.ro = 1;
     write_pte(&xen_third_id[third_table_offset(id_addr)], pte);
 
     /* Switch TTBR */
-    fn(ttbr);
+    if ( llc_coloring_enabled )
+    {
+        relocate_xen_fn *fn = (relocate_xen_fn *)id_addr;
+
+        fn(ttbr, _start, (void *)BOOT_RELOC_VIRT_START, _end - _start);
+    }
+    else
+    {
+        switch_ttbr_fn *fn = (switch_ttbr_fn *)id_addr;
+
+        fn(ttbr);
+    }
 
     /*
      * Disable the identity mapping in the runtime page tables.
diff --git a/xen/arch/arm/include/asm/mmu/layout.h b/xen/arch/arm/include/asm/mmu/layout.h
index a3b546465b..19c0ec63a5 100644
--- a/xen/arch/arm/include/asm/mmu/layout.h
+++ b/xen/arch/arm/include/asm/mmu/layout.h
@@ -30,6 +30,7 @@
  *  10M -  12M   Fixmap: special-purpose 4K mapping slots
  *  12M -  16M   Early boot mapping of FDT
  *  16M -  18M   Livepatch vmap (if compiled in)
+ *  16M -  24M   Cache-colored Xen text, data, bss (temporary, if compiled in)
  *
  *   1G -   2G   VMAP: ioremap and early_ioremap
  *
@@ -74,6 +75,8 @@
 #define BOOT_FDT_VIRT_START     (FIXMAP_VIRT_START + FIXMAP_VIRT_SIZE)
 #define BOOT_FDT_VIRT_SIZE      _AT(vaddr_t, MB(4))
 
+#define BOOT_RELOC_VIRT_START   (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE)
+
 #ifdef CONFIG_LIVEPATCH
 #define LIVEPATCH_VMAP_START    (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE)
 #define LIVEPATCH_VMAP_SIZE    _AT(vaddr_t, MB(2))
diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c
index 66c8db2baf..c8b183da84 100644
--- a/xen/arch/arm/llc-coloring.c
+++ b/xen/arch/arm/llc-coloring.c
@@ -9,6 +9,7 @@
 
 #include <asm/processor.h>
 #include <asm/sysregs.h>
+#include <asm/setup.h>
 
 /* Return the LLC way size by probing the hardware */
 unsigned int __init get_llc_way_size(void)
@@ -62,8 +63,70 @@ unsigned int __init get_llc_way_size(void)
     return line_size * num_sets;
 }
 
+/**
+ * get_xen_paddr - get physical address to relocate Xen to
+ *
+ * Xen is relocated to as near to the top of RAM as possible and
+ * aligned to a XEN_PADDR_ALIGN boundary.
+ */
+static paddr_t __init get_xen_paddr(paddr_t xen_size)
+{
+    const struct membanks *mem = bootinfo_get_mem();
+    paddr_t min_size, paddr = 0;
+    unsigned int i;
+
+    min_size = (xen_size + (XEN_PADDR_ALIGN-1)) & ~(XEN_PADDR_ALIGN-1);
+
+    /* Find the highest bank with enough space. */
+    for ( i = 0; i < mem->nr_banks; i++ )
+    {
+        const struct membank *bank = &mem->bank[i];
+        paddr_t s, e;
+
+        if ( bank->size >= min_size )
+        {
+            e = consider_modules(bank->start, bank->start + bank->size,
+                                 min_size, XEN_PADDR_ALIGN, 0);
+            if ( !e )
+                continue;
+
+#ifdef CONFIG_ARM_32
+            /* Xen must be under 4GB */
+            if ( e > GB(4) )
+                e = GB(4);
+            if ( e < bank->start )
+                continue;
+#endif
+
+            s = e - min_size;
+
+            if ( s > paddr )
+                paddr = s;
+        }
+    }
+
+    if ( !paddr )
+        panic("Not enough memory to relocate Xen\n");
+
+    printk("Placing Xen at 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
+           paddr, paddr + min_size);
+
+    return paddr;
+}
+
+static paddr_t __init xen_colored_map_size(void)
+{
+    return ROUNDUP((_end - _start) * get_max_nr_llc_colors(), XEN_PADDR_ALIGN);
+}
+
 void __init arch_llc_coloring_init(void)
 {
+    struct bootmodule *xen_bootmodule = boot_module_find_by_kind(BOOTMOD_XEN);
+
+    BUG_ON(!xen_bootmodule);
+
+    xen_bootmodule->size = xen_colored_map_size();
+    xen_bootmodule->start = get_xen_paddr(xen_bootmodule->size);
 }
 
 /*
diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c
index a8f93d4f82..1a1b7f58a6 100644
--- a/xen/arch/arm/mmu/setup.c
+++ b/xen/arch/arm/mmu/setup.c
@@ -20,6 +20,11 @@
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef mfn_to_virt
 #define mfn_to_virt(mfn) __mfn_to_virt(mfn_x(mfn))
+#undef virt_to_mfn
+#define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
+
+#define virt_to_reloc_virt(virt) \
+    (((vaddr_t)virt) - XEN_VIRT_START + BOOT_RELOC_VIRT_START)
 
 /* Main runtime page tables */
 
@@ -72,6 +77,7 @@ static void __init __maybe_unused build_assertions(void)
     /* 2MB aligned regions */
     BUILD_BUG_ON(XEN_VIRT_START & ~SECOND_MASK);
     BUILD_BUG_ON(FIXMAP_ADDR(0) & ~SECOND_MASK);
+    BUILD_BUG_ON(BOOT_RELOC_VIRT_START & ~SECOND_MASK);
     /* 1GB aligned regions */
 #ifdef CONFIG_ARM_32
     BUILD_BUG_ON(XENHEAP_VIRT_START & ~FIRST_MASK);
@@ -135,7 +141,12 @@ static void __init __maybe_unused build_assertions(void)
 
 lpae_t __init pte_of_xenaddr(vaddr_t va)
 {
-    paddr_t ma = va + phys_offset;
+    paddr_t ma;
+
+    if ( llc_coloring_enabled )
+        ma = virt_to_maddr(virt_to_reloc_virt(va));
+    else
+        ma = va + phys_offset;
 
     return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL);
 }
@@ -335,9 +346,44 @@ paddr_t __init consider_modules(paddr_t s, paddr_t e,
     return e;
 }
 
+static void __init create_llc_coloring_mappings(void)
+{
+    lpae_t pte;
+    unsigned int i;
+    struct bootmodule *xen_bootmodule = boot_module_find_by_kind(BOOTMOD_XEN);
+    mfn_t start_mfn = maddr_to_mfn(xen_bootmodule->start), mfn;
+
+    for_each_xen_colored_mfn ( start_mfn, mfn, i )
+    {
+        pte = mfn_to_xen_entry(mfn, MT_NORMAL);
+        pte.pt.table = 1; /* level 3 mappings always have this bit set */
+        xen_xenmap[i] = pte;
+    }
+
+    for ( i = 0; i < XEN_NR_ENTRIES(2); i++ )
+    {
+        vaddr_t va = BOOT_RELOC_VIRT_START + (i << XEN_PT_LEVEL_SHIFT(2));
+
+        pte = mfn_to_xen_entry(virt_to_mfn(xen_xenmap +
+                                           i * XEN_PT_LPAE_ENTRIES),
+                               MT_NORMAL);
+        pte.pt.table = 1;
+        write_pte(&boot_second[second_table_offset(va)], pte);
+    }
+}
+
 /*
- * Boot-time pagetable setup.
+ * Boot-time pagetable setup with coloring support
  * Changes here may need matching changes in head.S
+ *
+ * The cache coloring support consists of:
+ * - Create colored mapping that conforms to Xen color selection in xen_xenmap[]
+ * - Link the mapping in boot page tables using BOOT_RELOC_VIRT_START as vaddr
+ * - pte_of_xenaddr() takes care of translating addresses to the new space
+ *   during runtime page tables creation
+ * - Relocate xen and update TTBR with the new address in the colored space
+ *   (see switch_ttbr())
+ * - Protect the new space
  */
 void __init setup_pagetables(unsigned long boot_phys_offset)
 {
@@ -347,6 +393,9 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
 
     phys_offset = boot_phys_offset;
 
+    if ( llc_coloring_enabled )
+        create_llc_coloring_mappings();
+
     arch_setup_page_tables();
 
 #ifdef CONFIG_ARM_64
@@ -374,13 +423,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
             break;
         pte = pte_of_xenaddr(va);
         pte.pt.table = 1; /* third level mappings always have this bit set */
-        if ( is_kernel_text(va) || is_kernel_inittext(va) )
-        {
-            pte.pt.xn = 0;
-            pte.pt.ro = 1;
-        }
-        if ( is_kernel_rodata(va) )
-            pte.pt.ro = 1;
+        pte.pt.xn = 0; /* Permissions will be enforced later. Allow execution */
         xen_xenmap[i] = pte;
     }
 
@@ -406,13 +449,48 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
     ttbr = (uintptr_t) cpu0_pgtable + phys_offset;
 #endif
 
-    switch_ttbr(ttbr);
-
-    xen_pt_enforce_wnx();
-
 #ifdef CONFIG_ARM_32
     per_cpu(xen_pgtable, 0) = cpu0_pgtable;
 #endif
+
+    if ( llc_coloring_enabled )
+        ttbr = virt_to_maddr(virt_to_reloc_virt(THIS_CPU_PGTABLE));
+
+    switch_ttbr(ttbr);
+
+    /* Protect Xen */
+    for ( i = 0; i < XEN_NR_ENTRIES(3); i++ )
+    {
+        vaddr_t va = XEN_VIRT_START + (i << PAGE_SHIFT);
+        lpae_t *entry = xen_xenmap + i;
+
+        if ( !is_kernel(va) )
+            break;
+
+        pte = read_atomic(entry);
+
+        if ( is_kernel_text(va) || is_kernel_inittext(va) )
+        {
+            pte.pt.xn = 0;
+            pte.pt.ro = 1;
+        } else if ( is_kernel_rodata(va) ) {
+            pte.pt.ro = 1;
+            pte.pt.xn = 1;
+        } else {
+            pte.pt.xn = 1;
+            pte.pt.ro = 0;
+        }
+
+        write_pte(entry, pte);
+    }
+
+    /*
+     * We modified live page-tables. Ensure the TLBs are invalidated
+     * before setting enforcing the WnX permissions.
+     */
+    flush_xen_tlb_local();
+
+    xen_pt_enforce_wnx();
 }
 
 void *__init arch_vmap_virt_end(void)
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index bbb3e1eea5..810dde3a49 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -754,8 +754,6 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
     /* Initialize traps early allow us to get backtrace when an error occurred */
     init_traps();
 
-    setup_pagetables(boot_phys_offset);
-
     smp_clear_cpu_maps();
 
     device_tree_flattened = early_fdt_map(fdt_paddr);
@@ -779,6 +777,14 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
 
     llc_coloring_init();
 
+    /*
+     * Page tables must be setup after LLC coloring initialization because
+     * coloring info are required in order to create colored mappings
+     */
+    setup_pagetables(boot_phys_offset);
+    /* Device-tree was mapped in boot page tables, remap it in the new tables */
+    device_tree_flattened = early_fdt_map(fdt_paddr);
+
     setup_mm();
 
     vm_init();
diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c
index 0ddd6946f1..323c125c48 100644
--- a/xen/common/llc-coloring.c
+++ b/xen/common/llc-coloring.c
@@ -33,6 +33,8 @@ static unsigned int __ro_after_init xen_num_colors;
 
 #define mfn_color_mask              (max_nr_colors - 1)
 #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
+#define get_mfn_with_color(mfn, color) \
+    (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color)))
 
 /*
  * Parse the coloring configuration given in the buf string, following the
@@ -350,6 +352,22 @@ unsigned int get_max_nr_llc_colors(void)
     return max_nr_colors;
 }
 
+mfn_t __init xen_colored_mfn(mfn_t mfn)
+{
+    unsigned int i, color = mfn_to_color(mfn);
+
+    for ( i = 0; i < xen_num_colors; i++ )
+    {
+        if ( color == xen_colors[i] )
+            return mfn;
+        else if ( color < xen_colors[i] )
+            return get_mfn_with_color(mfn, xen_colors[i]);
+    }
+
+    /* Jump to next color space (max_nr_colors mfns) and use the first color */
+    return get_mfn_with_color(mfn_add(mfn, max_nr_colors), xen_colors[0]);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h
index 7f8218bfb2..859ed2da4e 100644
--- a/xen/include/xen/llc-coloring.h
+++ b/xen/include/xen/llc-coloring.h
@@ -26,6 +26,17 @@ static inline void domain_dump_llc_colors(const struct domain *d) {}
 static inline void domain_llc_coloring_free(struct domain *d) {}
 #endif
 
+/**
+ * Iterate over each Xen mfn in the colored space.
+ * @start_mfn:  the first mfn that needs to be colored.
+ * @mfn:        the current mfn.
+ * @i:          loop index.
+ */
+#define for_each_xen_colored_mfn(start_mfn, mfn, i) \
+    for ( i = 0, mfn = xen_colored_mfn(start_mfn);  \
+          i < (_end - _start) >> PAGE_SHIFT;        \
+          i++, mfn = xen_colored_mfn(mfn_add(mfn, 1)) )
+
 unsigned int get_llc_way_size(void);
 void arch_llc_coloring_init(void);
 int dom0_set_llc_colors(struct domain *d);
@@ -37,6 +48,8 @@ struct page_info;
 unsigned int page_to_llc_color(const struct page_info *pg);
 unsigned int get_max_nr_llc_colors(void);
 
+mfn_t xen_colored_mfn(mfn_t mfn);
+
 #endif /* __COLORING_H__ */
 
 /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716036.1118072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU3-00015d-K5; Thu, 02 May 2024 17:44:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716036.1118072; Thu, 02 May 2024 17:44:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU3-00015S-Fi; Thu, 02 May 2024 17:44:39 +0000
Received: by outflank-mailman (input) for mailman id 716036;
 Thu, 02 May 2024 17:44:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU2-0000pj-7s
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:38 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a25438ad-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:32 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51bae805c56so9519587e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:32 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 i5-20020ac25b45000000b00516d2c05b3dsm242346lfp.299.2024.05.02.10.44.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a25438ad-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671872; x=1715276672; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=T5K1agmVtivL+Ttpsak7xYzIV14z+3VHjCsOZLq68vU=;
        b=SPM5S2WIHqqcFoSkNwVLyS3s9Q+9nh+mKe/OuLXZfQimd+P7C1Q1GlDJnfIXe0reI1
         5F27FnFIgtS1O85omlrlUoa9DyOOEPOYDsthrO/YB8l151x9JOhIo8J+gl0IymJ40t00
         WxLj2H1WtcqyagbK3qmw8F5WZGhtlASo432DgemlBgf7w8H/rt64iV9ORhAfz5InbTU1
         PRAHz1lLuUneqMnxkulYnnultCJeHtvWIkfnv+O/X7nVKDNdeu29U1pzpu32sFwkOmbE
         kRfRaqqkeiGO22ANXDTGgQjiJhhCd9w9aZkzq7wbCukZXuqpU9IG+rol/STyTdNkf3kZ
         hhWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671872; x=1715276672;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=T5K1agmVtivL+Ttpsak7xYzIV14z+3VHjCsOZLq68vU=;
        b=PPeB6nAvMVeSnw+rhFF5sCzyVDH5JVkucqIo1KzGKkbvjGpr+sO9fv3hwHSUzhi9iP
         iPYD9coa1mky5PE3W/KSuBu2GPX2qahD3GkYfKeCHYeEzP7B4DVJ9rzEzyoyuF1aFaBY
         87WksSrA/rEdDt0CtJDdjVuniLj1dMfFfggZgJLVHmPWzJ++XAO9pf2JIPCVZguonIo9
         9QDA8ayMtAcrGfRnhQg2oSvtE4+G7wBJLBGlD6hpOb4jhIIt3YMl06s8y5hF4N67njcc
         3XUVpilJBDZNjTax+JZjlD4EtlFnEjF0+4ckZkFWq+hW7td1Rm/txXhbeQ/U+D3G+/W2
         3tFg==
X-Gm-Message-State: AOJu0Yw9Mgx1KZgVaTZoAa/fpNQomeE5ciyfbPu5qOFUurlkS3FHdtMQ
	wdOm+V+0FEFQr9hYJdL6ietoDV9kX4gasszSj4NorZ5ZXgD08FWKSj/Jl0eS
X-Google-Smtp-Source: AGHT+IG2csGx9nxFG6gd+bfnXUUc3DkUJ2k/PLxHYh7cOXYh/SxizHlPKdD32MD5m9MV5cdVHehcWQ==
X-Received: by 2002:ac2:555a:0:b0:51e:f31b:1b45 with SMTP id l26-20020ac2555a000000b0051ef31b1b45mr320973lfk.50.1714671871317;
        Thu, 02 May 2024 10:44:31 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 0/9] xen/arm: arm64: Annotate code symbols
Date: Wed,  1 May 2024 05:54:39 +0200
Message-Id: <20240501035448.964625-1-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

On the way towards Xen safety certification we're evaluating the use
of tools to collect code-coverage/profiling information from execution
traces. Some tools rely on ELF symbols for code being declared with
type FUNC and having a symbol size.

We currently annotate some symbols but not all. Also, there seems to be
different ways to do the annotation.

This series now converts all the .S files under xen/arm/arm64 to use
the macros from xen/linkage.h and also adds missing code symbol
annotations when appropriate.

Best regards,
Edgar

ChangeLog:

v2 -> v3:
* Convert and add annotations for all of xen/arm/arm64.
* Fix brain freeze in commit message for entry.S.

v1 -> v2:
* Drop RFC.
* Squash into a single patch to avoid confusion.



Edgar E. Iglesias (9):
  xen/arm64: entry: Add missing code symbol annotations
  xen/arm64: smc: Add missing code symbol annotations
  xen/arm64: sve: Add missing code symbol annotations
  xen/arm64: head: Add missing code symbol annotations
  xen/arm64: debug: Add missing code symbol annotations
  xen/arm64: bpi: Add missing code symbol annotations
  xen/arm64: mmu/head: Add missing code symbol annotations
  xen/arm64: cache: Use the generic xen/linkage.h macros
  xen/arm64: lib: Use the generic xen/linkage.h macros

 xen/arch/arm/arm64/bpi.S         | 20 +++++----
 xen/arch/arm/arm64/cache.S       |  4 +-
 xen/arch/arm/arm64/debug.S       |  6 ++-
 xen/arch/arm/arm64/entry.S       | 72 +++++++++++++++++++++-----------
 xen/arch/arm/arm64/head.S        | 50 +++++++++++-----------
 xen/arch/arm/arm64/lib/memchr.S  |  4 +-
 xen/arch/arm/arm64/lib/memcmp.S  |  4 +-
 xen/arch/arm/arm64/lib/memcpy.S  |  4 +-
 xen/arch/arm/arm64/lib/memmove.S |  4 +-
 xen/arch/arm/arm64/lib/memset.S  |  4 +-
 xen/arch/arm/arm64/lib/strchr.S  |  4 +-
 xen/arch/arm/arm64/lib/strcmp.S  |  4 +-
 xen/arch/arm/arm64/lib/strlen.S  |  4 +-
 xen/arch/arm/arm64/lib/strncmp.S |  4 +-
 xen/arch/arm/arm64/lib/strnlen.S |  4 +-
 xen/arch/arm/arm64/lib/strrchr.S |  4 +-
 xen/arch/arm/arm64/mmu/head.S    | 28 ++++++-------
 xen/arch/arm/arm64/smc.S         |  6 ++-
 xen/arch/arm/arm64/sve-asm.S     |  9 ++--
 19 files changed, 138 insertions(+), 101 deletions(-)

-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716039.1118100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU5-0001mK-G8; Thu, 02 May 2024 17:44:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716039.1118100; Thu, 02 May 2024 17:44:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU5-0001k0-AP; Thu, 02 May 2024 17:44:41 +0000
Received: by outflank-mailman (input) for mailman id 716039;
 Thu, 02 May 2024 17:44:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU4-0000pk-65
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:40 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a66e57df-08ab-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 19:44:39 +0200 (CEST)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2dd615b6c44so93016481fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:39 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 x28-20020a056512131c00b0051979e3a586sm247764lfu.266.2024.05.02.10.44.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a66e57df-08ab-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671879; x=1715276679; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nJ6OnAjpVNUh/tzF6RUek+tV3ajhXFS6uCjmPKu/PhE=;
        b=Xt8vZsiU5G3AJ9ZV4GuI+OEv14qt2zT5zc5QhpjWPo3uL1K6fZ3g5h8NGzXlMmgurV
         mDRbawPjsBwuiTUEAy0KXXiKScEFbqlL6BVSFiOeF5NXmELUnPxoPFcuqKk1AqwMGh/A
         sEIkxzS8d0xhdIW2qMKoQx7CvkqMhLsokhShNPnTsguYxegJj79Udx9kpUGB/qC/ZsFY
         nFF2TkLvjxrsbFbQ84bglyXY/Q2IxjqGs5KjTuhWdEMoviLqs01P8+balJi33Jyl57I+
         S0MyohETlHiP2qB2EmRvZAH66flhchG/SUPR8Bgb3mVgX34j9Q5+IjQpHSCMffyz2ddS
         cdEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671879; x=1715276679;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nJ6OnAjpVNUh/tzF6RUek+tV3ajhXFS6uCjmPKu/PhE=;
        b=BfXjoQXCAjhvt9FR5l40c6sOHGdOJTIkCRxT2rSpavUNpyHEct//YjKR4FraPLrwSk
         ouvtF/k0kJMfdrbscge2hkxfFpXiyOgkJ1yM01rlLYYsgRLww5NwXNZ4x2PV2E2RxT9F
         IKNjCqfQOwG+nBswrz/59rLhB0oCHagavPupJnmHTqx96Hj1p+nw4k6MKKxbdWu26DFk
         6L9dbaUWlXULjDz54flTn5WtoS51ggj2mcDw9z5rcdryTrkCKEYwG6z7RTHlD7g5QjIp
         mH45IfVIE7BN+t/Y+SJOx4tlAXct+9Qd8wtNMDYDFaX578boZI7CGpOQkDi1L4yNKANc
         7M4A==
X-Gm-Message-State: AOJu0YyYLoSlEzBRTZIPVFsqTlZIwUyxUIXh8B0ItYuU3Uo1i+9cAShx
	I+E3nP/1+Gjezm/iirO/Mb6DOPxJPG8U8yoXnNIuHoVyBBpaxeGsmTTPgYpm
X-Google-Smtp-Source: AGHT+IEkj7gWz8+yVxln7UOE1GtV/L21PQXmlVr46kP9Y+fdNaA2Qeg7Jxs1ExUr363P8Z6AF5jU8g==
X-Received: by 2002:a05:6512:3119:b0:516:d18b:eae8 with SMTP id n25-20020a056512311900b00516d18beae8mr367383lfb.41.1714671878805;
        Thu, 02 May 2024 10:44:38 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 5/9] xen/arm64: debug: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:44 +0200
Message-Id: <20240501035448.964625-6-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/debug.S | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/debug.S b/xen/arch/arm/arm64/debug.S
index 71cad9d762..c3d02c33d7 100644
--- a/xen/arch/arm/arm64/debug.S
+++ b/xen/arch/arm/arm64/debug.S
@@ -27,17 +27,19 @@
  * Print a character on the UART - this function is called by C
  * x0: character to print
  */
-GLOBAL(early_putch)
+FUNC(early_putch)
         ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS
         early_uart_ready x15, 1
         early_uart_transmit x15, w0
         ret
+END(early_putch)
 
 /* Flush the UART - this function is called by C */
-GLOBAL(early_flush)
+FUNC(early_flush)
         ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS  /* x15 := VA UART base address */
         early_uart_ready x15, 1
         ret
+END(early_flush)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716035.1118061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU1-0000py-Bh; Thu, 02 May 2024 17:44:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716035.1118061; Thu, 02 May 2024 17:44:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU1-0000pr-8B; Thu, 02 May 2024 17:44:37 +0000
Received: by outflank-mailman (input) for mailman id 716035;
 Thu, 02 May 2024 17:44:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aTz-0000pk-If
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:35 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a2e17859-08ab-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 19:44:33 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51967f75729so9994573e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:33 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 b15-20020ac2562f000000b0051884ea2a32sm244102lff.31.2024.05.02.10.44.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2e17859-08ab-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671872; x=1715276672; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qOE/YLLSPbJgtqh9FaZ/GX1mmt9IgzaYsZgTTrSiNM8=;
        b=RaLH4dfs3KJ5VikZ3O2qPpKGgG44sblzksb6c17Zb8tZBjdXTuMymAPOSlBMu6JtZl
         rL8Za+2WqhjEH/C44MnGCsHHzVK1OP0Bsre7gx0GFHgxft8mFG4yXtkpFZLEVpAQ03ty
         SJJWo7goN9+bkzQ4ma0SZb7kfp0O+j5+WNalbT7MgtK8YZDFuSuS6DkS1E1YDLuoZ4Z8
         BVWKb3FSxjd3bqL/krnfsUaL4C/cjMnC04u4+Vj7/BvGlET7gFkg7cGAg8E8zgn5ZcsT
         updqZtiALPaWVKnRaCqWjSR8ymzKmwF427xFjvs4nzQ9hUV/KoD4mFQ4h5NVo7jOQCNR
         OmIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671872; x=1715276672;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qOE/YLLSPbJgtqh9FaZ/GX1mmt9IgzaYsZgTTrSiNM8=;
        b=paUYRzZzpp7RP7JSn3XXbRMmloB8Jdxu3kCCaBB/sjdgc8GjF4TUwl5+HyRVs87zwg
         0pwOvvbbQJ3XDm1hjwLKec/xY8yM43MAIGRAu8wzZIhCvXMOG1+Q593yuySMOIO8BGb2
         Qhax4QjknnNAhRqF8PuqzKOpd//z4FjqkSZkDu6atVambWZJ8FQtUuedS36kDTTUgvLs
         p8G+qelIhJ7YK1Gstamueudk7NnGr7sCiVmXz5s8DW1I7/R1vaEKpXnxomF7QcQ2uiin
         CRlyl3Fec4+1ua4lztDlMy0t+2SPeX8vMACKpO8lSKQwznJCIYUs8vcZTXPypS2lKHQ1
         C1LA==
X-Gm-Message-State: AOJu0YwYC9e711Nq/NlkFT7DYUZKs3oormZjYGWaC9QrTSLecqcWIPJC
	qVBXsM+Sl7qWNmZbffkKwnJo3L1NZfWOUhvxmqhcWynbBzqFF1MGR0fXSsgR
X-Google-Smtp-Source: AGHT+IGbcCm5fc38UKPK1zDJdfmBtwkHYVUiZ0vyqiprQuSB5aWnfRR1u8dWIJi9bSESJAk8I6IWfg==
X-Received: by 2002:ac2:520b:0:b0:518:b144:9020 with SMTP id a11-20020ac2520b000000b00518b1449020mr309556lfl.61.1714671872244;
        Thu, 02 May 2024 10:44:32 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 1/9] xen/arm64: entry: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:40 +0200
Message-Id: <20240501035448.964625-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/entry.S | 72 +++++++++++++++++++++++++-------------
 1 file changed, 48 insertions(+), 24 deletions(-)

diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index f963c923bb..af9a592cae 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -289,21 +289,25 @@
         b       do_bad_mode
         .endm
 
-hyp_sync_invalid:
+FUNC_LOCAL(hyp_sync_invalid)
         entry   hyp=1
         invalid BAD_SYNC
+END(hyp_sync_invalid)
 
-hyp_irq_invalid:
+FUNC_LOCAL(hyp_irq_invalid)
         entry   hyp=1
         invalid BAD_IRQ
+END(hyp_irq_invalid)
 
-hyp_fiq_invalid:
+FUNC_LOCAL(hyp_fiq_invalid)
         entry   hyp=1
         invalid BAD_FIQ
+END(hyp_fiq_invalid)
 
-hyp_error_invalid:
+FUNC_LOCAL(hyp_error_invalid)
         entry   hyp=1
         invalid BAD_ERROR
+END(hyp_error_invalid)
 
 /*
  * SError received while running in the hypervisor mode.
@@ -313,11 +317,12 @@ hyp_error_invalid:
  * simplicity, as SError should be rare and potentially fatal,
  * all interrupts are kept masked.
  */
-hyp_error:
+FUNC_LOCAL(hyp_error)
         entry   hyp=1
         mov     x0, sp
         bl      do_trap_hyp_serror
         exit    hyp=1
+END(hyp_error)
 
 /*
  * Synchronous exception received while running in the hypervisor mode.
@@ -327,7 +332,7 @@ hyp_error:
  * some of them. So we want to inherit the state from the interrupted
  * context.
  */
-hyp_sync:
+FUNC_LOCAL(hyp_sync)
         entry   hyp=1
 
         /* Inherit interrupts */
@@ -338,6 +343,7 @@ hyp_sync:
         mov     x0, sp
         bl      do_trap_hyp_sync
         exit    hyp=1
+END(hyp_sync)
 
 /*
  * IRQ received while running in the hypervisor mode.
@@ -352,7 +358,7 @@ hyp_sync:
  * would require some rework in some paths (e.g. panic, livepatch) to
  * ensure the ordering is enforced everywhere.
  */
-hyp_irq:
+FUNC_LOCAL(hyp_irq)
         entry   hyp=1
 
         /* Inherit D, A, F interrupts and keep I masked */
@@ -365,8 +371,9 @@ hyp_irq:
         mov     x0, sp
         bl      do_trap_irq
         exit    hyp=1
+END(hyp_irq)
 
-guest_sync:
+FUNC_LOCAL(guest_sync)
         /*
          * Save x0, x1 in advance
          */
@@ -413,8 +420,9 @@ fastpath_out_workaround:
         mov     x1, xzr
         eret
         sb
+END(guest_sync)
 
-wa2_ssbd:
+FUNC_LOCAL(wa2_ssbd)
 #ifdef CONFIG_ARM_SSBD
 alternative_cb arm_enable_wa2_handling
         b       wa2_end
@@ -450,42 +458,55 @@ wa2_end:
         mov     x0, xzr
         eret
         sb
-guest_sync_slowpath:
+END(wa2_ssbd)
+
+FUNC_LOCAL(guest_sync_slowpath)
         /*
          * x0/x1 may have been scratch by the fast path above, so avoid
          * to save them.
          */
         guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_sync, save_x0_x1=0
+END(guest_sync_slowpath)
 
-guest_irq:
+FUNC_LOCAL(guest_irq)
         guest_vector compat=0, iflags=IFLAGS__A__, trap=irq
+END(guest_irq)
 
-guest_fiq_invalid:
+FUNC_LOCAL(guest_fiq_invalid)
         entry   hyp=0, compat=0
         invalid BAD_FIQ
+END(guest_fiq_invalid)
 
-guest_error:
+FUNC_LOCAL(guest_error)
         guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_serror
+END(guest_error)
 
-guest_sync_compat:
+FUNC_LOCAL(guest_sync_compat)
         guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_sync
+END(guest_sync_compat)
 
-guest_irq_compat:
+FUNC_LOCAL(guest_irq_compat)
         guest_vector compat=1, iflags=IFLAGS__A__, trap=irq
+END(guest_irq_compat)
 
-guest_fiq_invalid_compat:
+FUNC_LOCAL(guest_fiq_invalid_compat)
         entry   hyp=0, compat=1
         invalid BAD_FIQ
+END(guest_fiq_invalid_compat)
 
-guest_error_compat:
+FUNC_LOCAL(guest_error_compat)
         guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_serror
+END(guest_error_compat)
 
-ENTRY(return_to_new_vcpu32)
+FUNC(return_to_new_vcpu32)
         exit    hyp=0, compat=1
-ENTRY(return_to_new_vcpu64)
+END(return_to_new_vcpu32)
+
+FUNC(return_to_new_vcpu64)
         exit    hyp=0, compat=0
+END(return_to_new_vcpu64)
 
-return_from_trap:
+FUNC_LOCAL(return_from_trap)
         msr     daifset, #IFLAGS___I_ /* Mask interrupts */
 
         ldr     x21, [sp, #UREGS_PC]            /* load ELR */
@@ -524,6 +545,7 @@ return_from_trap:
 
         eret
         sb
+END(return_from_trap)
 
 /*
  * Consume pending SError generated by the guest if any.
@@ -536,7 +558,7 @@ return_from_trap:
  * it. So the function will unmask SError exception for a small window and
  * then mask it again.
  */
-check_pending_guest_serror:
+FUNC_LOCAL(check_pending_guest_serror)
         /*
          * Save elr_el2 to check whether the pending SError exception takes
          * place while we are doing this sync exception.
@@ -586,7 +608,7 @@ abort_guest_exit_end:
         cset    x19, ne
 
         ret
-ENDPROC(check_pending_guest_serror)
+END(check_pending_guest_serror)
 
 /*
  * Exception vectors.
@@ -597,7 +619,7 @@ ENDPROC(check_pending_guest_serror)
         .endm
 
         .align  11
-ENTRY(hyp_traps_vector)
+FUNC(hyp_traps_vector)
         ventry  hyp_sync_invalid            /* Synchronous EL2t */
         ventry  hyp_irq_invalid             /* IRQ EL2t */
         ventry  hyp_fiq_invalid             /* FIQ EL2t */
@@ -617,6 +639,7 @@ ENTRY(hyp_traps_vector)
         ventry  guest_irq_compat            /* IRQ 32-bit EL0/EL1 */
         ventry  guest_fiq_invalid_compat    /* FIQ 32-bit EL0/EL1 */
         ventry  guest_error_compat          /* Error 32-bit EL0/EL1 */
+END(hyp_traps_vector)
 
 /*
  * struct vcpu *__context_switch(struct vcpu *prev, struct vcpu *next)
@@ -626,7 +649,7 @@ ENTRY(hyp_traps_vector)
  *
  * Returns prev in x0
  */
-ENTRY(__context_switch)
+FUNC(__context_switch)
         add     x8, x0, #VCPU_arch_saved_context
         mov     x9, sp
         stp     x19, x20, [x8], #16         /* store callee-saved registers */
@@ -647,6 +670,7 @@ ENTRY(__context_switch)
         ldr     lr, [x8]
         mov     sp, x9
         ret
+END(__context_switch)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716041.1118120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU8-0002Ml-8G; Thu, 02 May 2024 17:44:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716041.1118120; Thu, 02 May 2024 17:44:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU8-0002MW-5O; Thu, 02 May 2024 17:44:44 +0000
Received: by outflank-mailman (input) for mailman id 716041;
 Thu, 02 May 2024 17:44:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU7-0000pj-9D
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:43 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a7a1a43f-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:41 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-51ac5923ef6so1939533e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:41 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 cf9-20020a056512280900b0051efeb3994asm245518lfb.288.2024.05.02.10.44.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7a1a43f-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671880; x=1715276680; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+btECkGvGdaMDpadFUE07lhZtvml0qD3bZm2bwgsu5s=;
        b=KloZ3SnP/n8Yzcy/qtJjYPihZjESjPvMDTDL6BF2fSnAwySbO7zvGQOJ0cL7tT69ho
         bfZQKoUkog2ns0alfn1aLhm2Cpj104NDm83TldGC34zwCJfAQcFn9eQa66XhIey8GqfP
         5cYFOzkPjmEkbw5jhQLAxHXoN980H/gnpXssIRjMQt2vjPzbCsMKFHfhQalwsgM2lHPA
         UBpXYML1w9xNm1E3sppTOJztyKCj8+UTXx186A3S9eqAIBqzZa/S16hp8IxQbHTES43l
         6HmKYVsH0n//067t0QEAi/UZL/HojfJUszg2rNjyYbBtX6LOZA7n7Z5UZu8MfL0E4P36
         E+VA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671880; x=1715276680;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+btECkGvGdaMDpadFUE07lhZtvml0qD3bZm2bwgsu5s=;
        b=K9nXLp0n+QYMSdSeycn4H8TADSxltCvVfwl2XGtqsXiC9K8gtmtT4ZA4QldPDuaubU
         wDnAyPJyGFB+w+NaWEBlbE99VH/H0KO8EIWMxhszzE229OWusI8sZkJmInsnSf1YPJxB
         eQIFfSACCRQ8mA5o4+wczxysMjcEB4fb0xbgy1+r3hgcKBmeSGXOUvDyFo/VlDY6hy5w
         B6KVA1i/lTBz2dCprTnx69NriqtBmbJOmPRcZIEh8FmJTDvygyodhAcNfvurF0W+LS1l
         kDuI0lKYNSjDKyDm2LC77eH0DcSPiH5BnR+fjoE9nd7uk5Da6wJpYtzJ1MIaVLqeesuV
         rioQ==
X-Gm-Message-State: AOJu0YwwmvyEGmH8l9pQUAVrk81xyGVZ7t2lRv4lAvP19o0qHq6yPLjh
	FN0vfht61ZC4OeDAiC3Gu2iMUHzVRvu+MvOAkexYRSacbefDt7FAsjAc0Tbx
X-Google-Smtp-Source: AGHT+IEeaDRkvnyZ84ZEqX2g2gP1bsOAOy3rLwUj3q9QOWJgKyu8kPC1ZQjmXZ0ZCcY6wSReM+ACtA==
X-Received: by 2002:ac2:4572:0:b0:513:c47e:d811 with SMTP id k18-20020ac24572000000b00513c47ed811mr102176lfm.33.1714671880241;
        Thu, 02 May 2024 10:44:40 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:45 +0200
Message-Id: <20240501035448.964625-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index 4e63825220..b16e4d1e29 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -52,14 +52,15 @@
  * micro-architectures in a system.
  */
     .align	11
-ENTRY(__bp_harden_hyp_vecs_start)
+FUNC(__bp_harden_hyp_vecs_start)
     .rept 4
     vectors hyp_traps_vector
     .endr
-ENTRY(__bp_harden_hyp_vecs_end)
+GLOBAL(__bp_harden_hyp_vecs_end)
+END(__bp_harden_hyp_vecs_start)
 
 .macro mitigate_spectre_bhb_loop count
-ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+FUNC(__mitigate_spectre_bhb_loop_start_\count)
     stp     x0, x1, [sp, #-16]!
     mov     x0, \count
 .Lspectre_bhb_loop\@:
@@ -68,11 +69,12 @@ ENTRY(__mitigate_spectre_bhb_loop_start_\count)
     b.ne    .Lspectre_bhb_loop\@
     sb
     ldp     x0, x1, [sp], #16
-ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+GLOBAL(__mitigate_spectre_bhb_loop_end_\count)
+END(__mitigate_spectre_bhb_loop_start_\count)
 .endm
 
 .macro smccc_workaround num smcc_id
-ENTRY(__smccc_workaround_smc_start_\num)
+FUNC(__smccc_workaround_smc_start_\num)
     sub     sp, sp, #(8 * 4)
     stp     x0, x1, [sp, #(8 * 2)]
     stp     x2, x3, [sp, #(8 * 0)]
@@ -81,13 +83,15 @@ ENTRY(__smccc_workaround_smc_start_\num)
     ldp     x2, x3, [sp, #(8 * 0)]
     ldp     x0, x1, [sp, #(8 * 2)]
     add     sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_smc_end_\num)
+GLOBAL(__smccc_workaround_smc_end_\num)
+END(__smccc_workaround_smc_start_\num)
 .endm
 
-ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+FUNC(__mitigate_spectre_bhb_clear_insn_start)
     clearbhb
     isb
-ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+GLOBAL(__mitigate_spectre_bhb_clear_insn_end)
+END(__mitigate_spectre_bhb_clear_insn_start)
 
 mitigate_spectre_bhb_loop 8
 mitigate_spectre_bhb_loop 24
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716040.1118105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU5-0001qL-OO; Thu, 02 May 2024 17:44:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716040.1118105; Thu, 02 May 2024 17:44:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU5-0001pU-KK; Thu, 02 May 2024 17:44:41 +0000
Received: by outflank-mailman (input) for mailman id 716040;
 Thu, 02 May 2024 17:44:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU4-0000pj-F5
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:40 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a586fdf4-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:38 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e0a2870bceso59013851fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:38 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 bp42-20020a05651215aa00b0051f131c08c4sm247133lfb.183.2024.05.02.10.44.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a586fdf4-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671877; x=1715276677; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nmk0wrFUeJk71sTQLd3UkMRJK8JdRU2ZvbHWSXIjxGE=;
        b=fQ+dTFfZmd4zbRLCDE5eC01RylnZeDdbjUmaXA7/a2lALPe3fzMIWRpocpEaWOcELR
         AFOppQGy63r6Gw5NG9JWwGDAPbf+d8S8IcAQDQpOOkPRKN8AX5JhEfOTa8DHAfhAzS/n
         vrFHrLAfvSFrgNdkw2QdRNlwWSBWU3QIhTR0nOnEsd4KlJNOuaQqNz33kbbPg1Ae13uT
         +kTzA8t4HP+CUENaWwpic1f5Ea6r84LI3wCZp9gnrTLkMpwFdOmlD60UctsfmiEqJGh1
         MgQLU1cQV6Qp0qgNqBV3Ir0dMkRqGonRQav9ack8em/DGJWTht2HR9S+ImJxoPXBpSIU
         wTxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671877; x=1715276677;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nmk0wrFUeJk71sTQLd3UkMRJK8JdRU2ZvbHWSXIjxGE=;
        b=EsEBHEpulOU6YbkacLtLhb2MbH2vqDFwsBh23xyjtYCy/W8sr8sedPXqyzq7Uwx8AY
         S4rBZNnK4BfP5fgI+njQpaiNsObzwS/r6HegJEFyVjpi4REBC3cyzC+2ZLEVDgUy7wHN
         KULrWSX7O8I0HQy5Wwv/IBD38/XAIvsopZGzTUagAnPOzNGSNt4f97zVlPA5gWz9rWOy
         Mi5X7LwAAszd6tJwWKGnf78ibzqxise87UaOmE+jRuJ4xayzfqjGspoHytzM0+jTSmek
         Wd+vwuRG2twHUcy+0BhKBN3ZP5ECDL9KDRD7fyg4ktY/+s09KhvtIrVW6Gh45rFywONz
         aEog==
X-Gm-Message-State: AOJu0YzWlux7lp9kxPq+xyhWno54ZfFzv0LfjCZXVXtvpaACHuohNVIk
	aqGJtrzp6G3qj6Z3pX47qmgsSi16XJQwSccTHjDeDetvO2VH3imkB2wGNaok
X-Google-Smtp-Source: AGHT+IH5+T7s8J2KvprU48kD+WjKFIdBK1vELw+ethbsndwadvILH2gUZvISJTaosMo8adsitW8pTw==
X-Received: by 2002:ac2:4942:0:b0:516:582:2348 with SMTP id o2-20020ac24942000000b0051605822348mr346795lfi.54.1714671876830;
        Thu, 02 May 2024 10:44:36 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 4/9] xen/arm64: head: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:43 +0200
Message-Id: <20240501035448.964625-5-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/head.S | 50 ++++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 24 deletions(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index fb297e9eb5..7acedb4f8f 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -90,7 +90,7 @@
  * 4K-aligned address.
  */
 
-GLOBAL(start)
+FUNC(start)
         /*
          * DO NOT MODIFY. Image header expected by Linux boot-loaders.
          */
@@ -102,6 +102,7 @@ efi_head:
          */
         add     x13, x18, #0x16
         b       real_start           /* branch to kernel start */
+END(start)
         .quad   0                    /* Image load offset from start of RAM */
         .quad   _end - start         /* Effective size of kernel image, little-endian */
         .quad   __HEAD_FLAGS         /* Informative flags, little-endian */
@@ -223,7 +224,7 @@ section_table:
         .align  5
 #endif /* CONFIG_ARM_EFI */
 
-real_start:
+FUNC_LOCAL(real_start)
         /* BSS should be zeroed when booting without EFI */
         mov   x26, #0                /* x26 := skip_zero_bss */
 
@@ -263,9 +264,9 @@ primary_switched:
         mov   x1, x21                /* x1 := paddr(FDT) */
         ldr   x2, =start_xen
         b     launch
-ENDPROC(real_start)
+END(real_start)
 
-GLOBAL(init_secondary)
+FUNC(init_secondary)
         msr   DAIFSet, 0xf           /* Disable all interrupts */
 
         /* Find out where we are */
@@ -304,7 +305,7 @@ secondary_switched:
         /* Jump to C world */
         ldr   x2, =start_secondary
         b     launch
-ENDPROC(init_secondary)
+END(init_secondary)
 
 /*
  * Check if the CPU has been booted in Hypervisor mode.
@@ -313,7 +314,7 @@ ENDPROC(init_secondary)
  *
  * Clobbers x0 - x5
  */
-check_cpu_mode:
+FUNC_LOCAL(check_cpu_mode)
         PRINT_ID("- Current EL ")
         mrs   x5, CurrentEL
         print_reg x5
@@ -329,7 +330,7 @@ check_cpu_mode:
         PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n")
         PRINT_ID("- Please update the bootloader -\r\n")
         b fail
-ENDPROC(check_cpu_mode)
+END(check_cpu_mode)
 
 /*
  * Zero BSS
@@ -339,7 +340,7 @@ ENDPROC(check_cpu_mode)
  *
  * Clobbers x0 - x3
  */
-zero_bss:
+FUNC_LOCAL(zero_bss)
         /* Zero BSS only when requested */
         cbnz  x26, skip_bss
 
@@ -353,14 +354,14 @@ zero_bss:
 
 skip_bss:
         ret
-ENDPROC(zero_bss)
+END(zero_bss)
 
 /*
  * Initialize the processor for turning the MMU on.
  *
  * Clobbers x0 - x3
  */
-cpu_init:
+FUNC_LOCAL(cpu_init)
         PRINT_ID("- Initialize CPU -\r\n")
 
         /* Set up memory attribute type tables */
@@ -399,7 +400,7 @@ cpu_init:
          */
         msr spsel, #1
         ret
-ENDPROC(cpu_init)
+END(cpu_init)
 
 /*
  * Setup the initial stack and jump to the C world
@@ -411,7 +412,7 @@ ENDPROC(cpu_init)
  *
  * Clobbers x3
  */
-launch:
+FUNC_LOCAL(launch)
         ldr   x3, =init_data
         add   x3, x3, #INITINFO_stack /* Find the boot-time stack */
         ldr   x3, [x3]
@@ -421,13 +422,13 @@ launch:
 
         /* Jump to C world */
         br    x2
-ENDPROC(launch)
+END(launch)
 
 /* Fail-stop */
-fail:   PRINT_ID("- Boot failed -\r\n")
+FUNC_LOCAL(fail)   PRINT_ID("- Boot failed -\r\n")
 1:      wfe
         b     1b
-ENDPROC(fail)
+END(fail)
 
 #ifdef CONFIG_EARLY_PRINTK
 /*
@@ -438,14 +439,14 @@ ENDPROC(fail)
  *
  * Clobbers x0 - x1
  */
-init_uart:
+FUNC_LOCAL(init_uart)
         ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS
 #ifdef CONFIG_EARLY_UART_INIT
         early_uart_init x23, 0
 #endif
         PRINT("- UART enabled -\r\n")
         ret
-ENDPROC(init_uart)
+END(init_uart)
 
 /*
  * Print early debug messages.
@@ -454,7 +455,7 @@ ENDPROC(init_uart)
  * x23: Early UART base address
  * Clobbers x0-x1
  */
-ENTRY(asm_puts)
+FUNC(asm_puts)
         early_uart_ready x23, 1
         ldrb  w1, [x0], #1           /* Load next char */
         cbz   w1, 1f                 /* Exit on nul */
@@ -462,7 +463,7 @@ ENTRY(asm_puts)
         b     asm_puts
 1:
         ret
-ENDPROC(asm_puts)
+END(asm_puts)
 
 /*
  * Print a 64-bit number in hex.
@@ -471,7 +472,7 @@ ENDPROC(asm_puts)
  * x23: Early UART base address
  * Clobbers x0-x3
  */
-ENTRY(asm_putn)
+FUNC(asm_putn)
         adr_l x1, hex
         mov   x3, #16
 1:
@@ -484,7 +485,7 @@ ENTRY(asm_putn)
         subs  x3, x3, #1
         b.ne  1b
         ret
-ENDPROC(asm_putn)
+END(asm_putn)
 
 RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
 
@@ -493,16 +494,17 @@ RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
 /* This provides a C-API version of __lookup_processor_type
  * TODO: For now, the implementation return NULL every time
  */
-ENTRY(lookup_processor_type)
+FUNC(lookup_processor_type)
         mov  x0, #0
         ret
+END(lookup_processor_type)
 
 #ifdef CONFIG_ARM_EFI
 /*
  *  Function to transition from EFI loader in C, to Xen entry point.
  *  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
  */
-ENTRY(efi_xen_start)
+FUNC(efi_xen_start)
         /*
          * Preserve x0 (fdt pointer) across call to __flush_dcache_area,
          * restore for entry into Xen.
@@ -554,7 +556,7 @@ ENTRY(efi_xen_start)
         mov   x26, #1               /* x26 := skip_zero_bss */
 
         b     real_start_efi
-ENDPROC(efi_xen_start)
+END(efi_xen_start)
 
 #endif /* CONFIG_ARM_EFI */
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716038.1118083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU4-0001FD-65; Thu, 02 May 2024 17:44:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716038.1118083; Thu, 02 May 2024 17:44:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU4-0001Ee-1K; Thu, 02 May 2024 17:44:40 +0000
Received: by outflank-mailman (input) for mailman id 716038;
 Thu, 02 May 2024 17:44:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU3-0000pj-F2
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:39 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a47c8a4e-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:36 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2df848f9325so71748051fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:36 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 w9-20020a05651c102900b002e1c8a2f3d1sm255777ljm.75.2024.05.02.10.44.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a47c8a4e-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671875; x=1715276675; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OqCCbbv/MtZy1u/M+ApEL4BOvMHDwRPRM9TWyEa/hOE=;
        b=ShDznb8ZXMke04ud9GgP268RqFmqpicyjjeFqxLdcPFFroLKAnl/MG7k2Pe1tta7T6
         NhmBWCqVEuXMcU+MciNCyhMTruqHKftZtnPI7qOuybUUvyoo3q4u4osSqFh9099eRFeQ
         9m4syWN8qgtAr+m7Ueji4n5t0y34oHIaawQZHmg7rCiY15+dOCccBb1FFFlDn544xKUj
         OP0V6dCHqD4O0Q/BnR5DtpFZc7tUjqXoQshkd+1LJU71NILiOc/raHEdIWcMxTC4oe8L
         SeR4aP5++Td/YKfiB/DDLZL9Gly2lftjg36cP7pfX19FjiG5K3T1B0OaS13YzWftlqnS
         L92A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671875; x=1715276675;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OqCCbbv/MtZy1u/M+ApEL4BOvMHDwRPRM9TWyEa/hOE=;
        b=plq8uEax6Vj6u2i65cpzwwhZqyOKBcTlWqvbYpKz5Y8qkzHEfPmLFWqfxzZUkncbxb
         olNdaGcEWagoyhqbdonjGyhoUl+9jpTH73AQvAe39zUHQw0jxLUPH6i5mxgdBzXBZBUd
         V7ywN4JcQpcFx/oGxh9+vhZKNpblLT3HMZsro+U9epAEnK8K55NIfNNYhINl2sC0jsKD
         oXKkbwcbG9PI95KJGYlbAna17rj/IRXeIXWbia3UO982MfVd/cR/m8fjHFB0rupUAISN
         5G7kHuLCYw9aaVN3q/9xjWl6jr0XOWOd1+eHHVcMReilUp2ZzamK64oFQ5IC+WtMmmGW
         0Z3w==
X-Gm-Message-State: AOJu0YzBzcmQ7wTX9WUg06rYXMDFNW3PNGkrCkcJqb8lf6grvzWVgipE
	unPKF5Aiti4f8lMNgfU2U1qVAyIZtjtxP8MJPcuPwMsAhQ8Gc0eNKYGsmVSZ
X-Google-Smtp-Source: AGHT+IGkgN8FO7nnrbF/W+13whQheJUvKQ/jtMk8hAv9qCI/Rb7h3opgGgtUjVjZnTDJ5j/cjNhTUw==
X-Received: by 2002:a2e:7204:0:b0:2df:7824:7d32 with SMTP id n4-20020a2e7204000000b002df78247d32mr248943ljc.50.1714671875373;
        Thu, 02 May 2024 10:44:35 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 3/9] xen/arm64: sve: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:42 +0200
Message-Id: <20240501035448.964625-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/sve-asm.S | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/arm64/sve-asm.S b/xen/arch/arm/arm64/sve-asm.S
index 59dbefbbb2..2d8b895f07 100644
--- a/xen/arch/arm/arm64/sve-asm.S
+++ b/xen/arch/arm/arm64/sve-asm.S
@@ -161,9 +161,10 @@
 .endm
 
 /* Gets the current vector register size in bytes */
-GLOBAL(sve_get_hw_vl)
+FUNC(sve_get_hw_vl)
     _sve_rdvl 0, 1
     ret
+END(sve_get_hw_vl)
 
 /*
  * Save the SVE context
@@ -172,9 +173,10 @@ GLOBAL(sve_get_hw_vl)
  * x1 - pointer to buffer for P0-15
  * x2 - Save FFR if non-zero
  */
-GLOBAL(sve_save_ctx)
+FUNC(sve_save_ctx)
     sve_save 0, 1, x2
     ret
+END(sve_save_ctx)
 
 /*
  * Load the SVE context
@@ -183,9 +185,10 @@ GLOBAL(sve_save_ctx)
  * x1 - pointer to buffer for P0-15
  * x2 - Restore FFR if non-zero
  */
-GLOBAL(sve_load_ctx)
+FUNC(sve_load_ctx)
     sve_load 0, 1, x2
     ret
+END(sve_load_ctx)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716037.1118077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU3-00018x-Te; Thu, 02 May 2024 17:44:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716037.1118077; Thu, 02 May 2024 17:44:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aU3-00017F-Mp; Thu, 02 May 2024 17:44:39 +0000
Received: by outflank-mailman (input) for mailman id 716037;
 Thu, 02 May 2024 17:44:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU2-0000pj-Es
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:38 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a36e0f3b-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:34 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2dfb4ea2bbfso69869821fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:34 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 d4-20020a2e3604000000b002d2697570fcsm253313lja.93.2024.05.02.10.44.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a36e0f3b-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671873; x=1715276673; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZtRPbe4p6iFUf2UKoVIr28035yPnYXOsQI6g0x3mrLU=;
        b=JomZM2ux1OzpCWzWiG00OEmQZLN3V5K61CrnzRfvqC26sHdNO4K1zSDiSLSUukYL6X
         HTAXQd+f7JrAUQEn8OdI4nA42NzvKPZai5pbPRRNVBx1poe8JvPICPNXVtSyJUU0iGpk
         3c1G+GZSMOZANj7JqaSimmiFxF9VMxThvwxDG1j9XJTKtM9yWfq0GPdtj6AH63dbHhNx
         xCkIkXFCcxaYLKCfxcjBBBdhDo6994xo7AV0k3PF6TykHg/F/6T5kb5cZ1q47U4SxlLl
         E6RmkyLNIntc3pjUKuaWKjlq2OumpCQ6xd92dqpWgZtJOV927+5emL/4U6zn+WffErMp
         OuLA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671873; x=1715276673;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ZtRPbe4p6iFUf2UKoVIr28035yPnYXOsQI6g0x3mrLU=;
        b=S82WvfzrWMFI71NJSgbm4a/E4X7imhdv4GWUnqMljbD4skkhJobm4cKhA5JFwJWmYg
         WqGL8/MFZHlvIqNGRha8rrVI9VN6YDGQetOIFhFvUwKdQh2QZpbfY4DRAnx64p2facr1
         Ddun5uQRYVp2s/50j6tLQHwPCr5tusk0yJHCnWcfek4WzNmUj4+Q+VZaTj2Nwt0hvjvY
         84tHwWqBAPMXKvxlVd4tGP29dFWgQQGhBFMn7pVjTV1+861mXBIvHnpBw6Buubd0Ec+c
         XUSMDydBP/DEpcpQOoB51mkURAL2ePsZBhi4EYVDp5J6QM0tidQeZ8BYkqCTIywQz2/R
         +gQw==
X-Gm-Message-State: AOJu0YwUoesOQpDDXdlSDxE+F2x5eNL7XJzq909tS1Tep8EIC/rQHiCL
	y5ALW74CQwqVmwOCKSXMkAEivqNSJmgTNetHtadvj746IdxoumUyiY2rU0LA
X-Google-Smtp-Source: AGHT+IEaRjAESGjBXQiqOoLjVWNKmHhiuq1qWVhA9bmKTCAhITIMcXRmoAp2jGz9dptYpYFF/Z2ZTA==
X-Received: by 2002:a2e:3510:0:b0:2d9:eb66:6d39 with SMTP id z16-20020a2e3510000000b002d9eb666d39mr295242ljz.19.1714671873212;
        Thu, 02 May 2024 10:44:33 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 2/9] xen/arm64: smc: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:41 +0200
Message-Id: <20240501035448.964625-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/smc.S | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/smc.S b/xen/arch/arm/arm64/smc.S
index fc6b676e2e..68b05e8ddd 100644
--- a/xen/arch/arm/arm64/smc.S
+++ b/xen/arch/arm/arm64/smc.S
@@ -19,7 +19,7 @@
  *                          register_t a6, register_t a7,
  *                          struct arm_smccc_res *res)
  */
-ENTRY(__arm_smccc_1_0_smc)
+FUNC(__arm_smccc_1_0_smc)
         smc     #0
         ldr     x4, [sp]
         cbz     x4, 1f          /* No need to store the result */
@@ -27,12 +27,13 @@ ENTRY(__arm_smccc_1_0_smc)
         stp     x2, x3, [x4, #SMCCC_RES_a2]
 1:
         ret
+END(__arm_smccc_1_0_smc)
 
 /*
  * void arm_smccc_1_2_smc(const struct arm_smccc_1_2_regs *args,
  *                        struct arm_smccc_1_2_regs *res)
  */
-ENTRY(arm_smccc_1_2_smc)
+FUNC(arm_smccc_1_2_smc)
     /* Save `res` and free a GPR that won't be clobbered by SMC call */
     stp     x1, x19, [sp, #-16]!
 
@@ -69,3 +70,4 @@ ENTRY(arm_smccc_1_2_smc)
     /* Restore original x19 */
     ldp     xzr, x19, [sp], #16
     ret
+END(arm_smccc_1_2_smc)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716042.1118131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUA-0002gf-KJ; Thu, 02 May 2024 17:44:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716042.1118131; Thu, 02 May 2024 17:44:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUA-0002gR-GP; Thu, 02 May 2024 17:44:46 +0000
Received: by outflank-mailman (input) for mailman id 716042;
 Thu, 02 May 2024 17:44:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aU8-0000pj-Ub
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:44 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a8a5b5f0-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:43 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2dd615b6c44so93017441fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:43 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 h3-20020a056512054300b0051d94297380sm246511lfl.241.2024.05.02.10.44.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8a5b5f0-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671882; x=1715276682; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dlS4R+HzAaSNTc1UbpABFZoTWu/mGL1X+XqG4cvjVE0=;
        b=F9R6ufdprF+Tj4Ul7mfpt03VlmdISYD9tR4/Da7gTnZprT6Km3bIVXgpjBqAqXKJ+p
         OtKFHQM17/sOJ2BV9Z3H3zp54hc1ochqGg2gPuLrIyen3y8Y0L3jFtV1dZSlVajWVpnh
         TKVBlY5I4Iuv3bevyP2v25AWjd8ycJcvU1jW3BvO1t6V7MPwQO54F4dsjPeEYPyXRxb6
         omYuhQps7AX+ZGub9vSQrB4E/RDVDSOhAiFu6nLa/ncL08D6LI+ByC6SCCCjBE0UEZoI
         H1MLZ7XEg2WnAuWrhbUFI8aqTF+Ck9TBlzdocHVORc95Pf+mdAnQ20QApY97IcYBjeh5
         ueQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671882; x=1715276682;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dlS4R+HzAaSNTc1UbpABFZoTWu/mGL1X+XqG4cvjVE0=;
        b=brY201RW4GF2rHigqcEl2wuH5dafECwHkktHwL5/CDqgHjvo0/59Uk11lcakUhl0Nl
         cAvAv8KV+heG+w+j0ymqzD+yg/wi+Ex58xv0nlCOGIJU5W/CfCOPJA8AK1mWv3JO70l1
         SvuDPtZLfMl+YYbDxSOo68U71GHG9MjRlkeDGG6SUz5YWBS2QFVsu/REF98kweR4F83r
         wLYWPoQxZuXOBIUcjxdv8vXeIMA/A6yQ6gH/2ZUMrGd6+5CLvZ2l88dAMJCczxncVgks
         pW6Zn8pjl/5X4xnA+EGApW96XHvJIzwPMx8pU631ShinQbf3fA/X2Mdep5MjKqkq48a2
         te8g==
X-Gm-Message-State: AOJu0YzlB1eKRUH58gR8plrJyWxKM00XmpbLBdLlAT3tkag9LAzwKd+K
	mY34stmZjHt1zr2MMBt/3D5/7hGbEvd3fA2JMI6/mKQxkfA+rS0m2FEBee6G
X-Google-Smtp-Source: AGHT+IGjRyaLWSg4QpF1AbHHDfSCxyUL9ZKMbaLfp1r20NYD3hObym3Qg7SvgzXrGR8cJopa5nRmnQ==
X-Received: by 2002:a19:2d5c:0:b0:51f:5760:dd34 with SMTP id t28-20020a192d5c000000b0051f5760dd34mr307685lft.55.1714671882354;
        Thu, 02 May 2024 10:44:42 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 7/9] xen/arm64: mmu/head: Add missing code symbol annotations
Date: Wed,  1 May 2024 05:54:46 +0200
Message-Id: <20240501035448.964625-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/mmu/head.S | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S
index fa40b696dd..7788bb95e5 100644
--- a/xen/arch/arm/arm64/mmu/head.S
+++ b/xen/arch/arm/arm64/mmu/head.S
@@ -138,7 +138,7 @@
  *
  * Clobbers x0 - x4
  */
-create_page_tables:
+FUNC_LOCAL(create_page_tables)
         /* Prepare the page-tables for mapping Xen */
         ldr   x0, =XEN_VIRT_START
         create_table_entry boot_pgtable, boot_first, x0, 0, x1, x2, x3
@@ -260,7 +260,7 @@ virtphys_clash:
         /* Identity map clashes with boot_third, which we cannot handle yet */
         PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
         b     fail
-ENDPROC(create_page_tables)
+END(create_page_tables)
 
 /*
  * Turn on the Data Cache and the MMU. The function will return on the 1:1
@@ -273,7 +273,7 @@ ENDPROC(create_page_tables)
  *
  * Clobbers x0 - x5
  */
-enable_mmu:
+FUNC_LOCAL(enable_mmu)
         mov   x4, x0
         mov   x5, x1
         PRINT_ID("- Turning on paging -\r\n")
@@ -304,7 +304,7 @@ enable_mmu:
         PRINT_ID("- Paging turned on -\r\n")
 
         ret
-ENDPROC(enable_mmu)
+END(enable_mmu)
 
 /*
  * Enable mm (turn on the data cache and the MMU) for secondary CPUs.
@@ -316,7 +316,7 @@ ENDPROC(enable_mmu)
  *
  * Clobbers x0 - x6
  */
-ENTRY(enable_secondary_cpu_mm)
+FUNC(enable_secondary_cpu_mm)
         mov   x6, lr
 
         load_paddr x0, init_ttbr
@@ -328,7 +328,7 @@ ENTRY(enable_secondary_cpu_mm)
 
         /* Return to the virtual address requested by the caller. */
         ret
-ENDPROC(enable_secondary_cpu_mm)
+END(enable_secondary_cpu_mm)
 
 /*
  * Enable mm (turn on the data cache and the MMU) for the boot CPU.
@@ -340,7 +340,7 @@ ENDPROC(enable_secondary_cpu_mm)
  *
  * Clobbers x0 - x6
  */
-ENTRY(enable_boot_cpu_mm)
+FUNC(enable_boot_cpu_mm)
         mov   x6, lr
 
         bl    create_page_tables
@@ -365,7 +365,7 @@ ENTRY(enable_boot_cpu_mm)
          * by the caller.
          */
         b     remove_identity_mapping
-ENDPROC(enable_boot_cpu_mm)
+END(enable_boot_cpu_mm)
 
 /*
  * Remove the 1:1 map from the page-tables. It is not easy to keep track
@@ -377,7 +377,7 @@ ENDPROC(enable_boot_cpu_mm)
  *
  * Clobbers x0 - x1
  */
-remove_identity_mapping:
+FUNC_LOCAL(remove_identity_mapping)
         /*
          * Find the zeroeth slot used. Remove the entry from zeroeth
          * table if the slot is not XEN_ZEROETH_SLOT.
@@ -419,20 +419,20 @@ identity_mapping_removed:
         flush_xen_tlb_local
 
         ret
-ENDPROC(remove_identity_mapping)
+END(remove_identity_mapping)
 
 /* Fail-stop */
-fail:   PRINT("- Boot failed -\r\n")
+FUNC_LOCAL(fail)   PRINT("- Boot failed -\r\n")
 1:      wfe
         b     1b
-ENDPROC(fail)
+END(fail)
 
 /*
  * Switch TTBR
  *
  * x0    ttbr
  */
-ENTRY(switch_ttbr_id)
+FUNC(switch_ttbr_id)
         /* 1) Ensure any previous read/write have completed */
         dsb    ish
         isb
@@ -464,7 +464,7 @@ ENTRY(switch_ttbr_id)
         isb
 
         ret
-ENDPROC(switch_ttbr_id)
+END(switch_ttbr_id)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:44:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:44:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716043.1118142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUC-00031C-Uy; Thu, 02 May 2024 17:44:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716043.1118142; Thu, 02 May 2024 17:44:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUC-00030v-Pl; Thu, 02 May 2024 17:44:48 +0000
Received: by outflank-mailman (input) for mailman id 716043;
 Thu, 02 May 2024 17:44:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aUA-0000pj-Rx
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:46 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9d5dbe9-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:45 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2e0a34b2899so55621371fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:45 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 i8-20020a2e8648000000b002d8744903ebsm250080ljj.68.2024.05.02.10.44.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9d5dbe9-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671884; x=1715276684; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ThNztN9oL4lNlCs74CGjUu+Eh6PWfK8/gTCcea/eeeo=;
        b=IWZbgbC0CrS3LJPnjhxaINZlxd9qBKghXX37g/OqTUJCJcsgR7djK5jgRp+zAysiHr
         vHhy/9r7xFKQNiJkV/RfzbNMczLXDmT8yecaobw2bGobaX+8GzhX/NbclLwMeWHF0+GE
         52Zjvpch63ahHAgDwtmfxk7ZEUxiO+BE75WO5DIL+MQhmIRgnSb1xMPd1upUWVd5sUtg
         jgIhC+Hgvb3eg9KgIQXuR0ghqmHyZ3K17k400wReS7Xrh920+g7sQiRyCgYnZrFnIkRa
         +cmsCn7m0JOhPNGjJiJe8cMHv9WJxafpOjesY8S2LJ4cRCYyyrZZkvRcQZ0gTeqjxHZH
         BZvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671884; x=1715276684;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ThNztN9oL4lNlCs74CGjUu+Eh6PWfK8/gTCcea/eeeo=;
        b=rEgQQ7MxdWj0GPWNjWAvcfCQSdTJmmMgHiUlVg85cL1vmfDWrqI0bief+YRRNZFdkb
         BKTO9AXYs91mehUZ2gw6aggJ2OFOwbqjwkIrAe7HMqf07X9snyYeY5N0fLItzqQULKxz
         UjEcbRQwm+5YCbFT3cLRhJQgSVeaPkRdpJMn27F3b0bplesct/X1QIK6K9RUMnDE8UiW
         oQ/ADMsTgFqfvQfh1R5DN6MFUXCmKFqm7k5mIuKDYb2fF41UCKfvSwNkFmbPz1My4k/9
         6wfkhf7J7KWPAsSKMf/M/8MnpHG7nRSWMR8rpCzd/z+NF3q5oMV4uV/NUdJoxSoaF/mQ
         iuIw==
X-Gm-Message-State: AOJu0YwLEvz+IfO1padooZQDADyBrHehzNfucToLvCSbtD096TzXgROW
	ONbCVE5QfohtlGCaUhq/fuaQ8ZS4j+FuJheRywmsSVq1oGbgke895tkmiCUf
X-Google-Smtp-Source: AGHT+IGIPSpDcYfIeG4Mae6HaT2hgvKQP35pzI9a9QRegeGYj/o4Trlp9ayoLwS+0VIZ78qk6JEwrQ==
X-Received: by 2002:a05:651c:105c:b0:2df:1e3e:3280 with SMTP id x28-20020a05651c105c00b002df1e3e3280mr350557ljm.28.1714671884108;
        Thu, 02 May 2024 10:44:44 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 8/9] xen/arm64: cache: Use the generic xen/linkage.h macros
Date: Wed,  1 May 2024 05:54:47 +0200
Message-Id: <20240501035448.964625-9-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/cache.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/cache.S b/xen/arch/arm/arm64/cache.S
index 9a88a2b497..66ed85f735 100644
--- a/xen/arch/arm/arm64/cache.S
+++ b/xen/arch/arm/arm64/cache.S
@@ -40,7 +40,7 @@
  *	- kaddr   - kernel address
  *	- size    - size in question
  */
-ENTRY(__flush_dcache_area)
+FUNC(__flush_dcache_area)
 	dcache_line_size x2, x3
 	add	x1, x0, x1
 	sub	x3, x2, #1
@@ -51,4 +51,4 @@ ENTRY(__flush_dcache_area)
 	b.lo	1b
 	dsb	sy
 	ret
-ENDPROC(__flush_dcache_area)
+END(__flush_dcache_area)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 17:45:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 17:45:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716044.1118151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUE-0003J7-8F; Thu, 02 May 2024 17:44:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716044.1118151; Thu, 02 May 2024 17:44:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2aUE-0003Hv-44; Thu, 02 May 2024 17:44:50 +0000
Received: by outflank-mailman (input) for mailman id 716044;
 Thu, 02 May 2024 17:44:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2aUD-0000pj-83
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 17:44:49 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ab266fc8-08ab-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 19:44:47 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51f29e80800so1228221e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 10:44:47 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 f10-20020a19ae0a000000b0051f4ab12ee3sm190497lfc.248.2024.05.02.10.44.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 10:44:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab266fc8-08ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714671886; x=1715276686; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zHs4bshwNa+29DjSZp/GoXZPGT+4z3HWsZCQXLRsJBk=;
        b=ZHAPfkp0TQ4sThBvrNzSY6h2RcpTqhNpSMSyxvCYu4OCo7uJSqIia5/YpZxiHGkB71
         Xk2VhvpW/Ld5dy9pBps51AU2Yitts4fgqJH3uLxU6EPlDssYTxcPDDCVaTPk2Ds08Mq8
         2QiDMKE5FyHg1BrI+fhCyJhZxxRU1uBq5WITXuCzYuPH5kxXVmY1z9PW/r8euSzaytXm
         89iMjVwyxhT0KtamfcUyAImEfCYxbfsVYYHe6D+Nts9U/AhvifN8XlIlzcD6gVOuJlIv
         jU1qfrE/WaRk6vPESa4Yme6yBVQLbGvuQbXMFlCCe9Qvq9UWqABjmvN/EwsqMV55CRq7
         RWxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714671886; x=1715276686;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zHs4bshwNa+29DjSZp/GoXZPGT+4z3HWsZCQXLRsJBk=;
        b=GtyY6733WFVhpIH9afrFbpGvgPAVZw6zo7fBCT5HoHY8uKUOhi66Wvm6jdAfyLKQ6W
         zbscE2Pjcl8zOVlarpae+7MZVmxnszaq9LtlrpqNmUWGZnlDYH9WCHPqgfmQp3nRZe8M
         hlRT3CcYRSZns+FoocjXmHJQm55S7///98k/Do6EcaezVbdyMrKbQGTpsFNQA3AgKP1S
         vCdbmFac0S4CAp2eWUewvGpwFDhf3fiuoW19XH1//QPceFQHnsivCHZn6YXuYVy6proH
         hVXkMvaO9c1oXDQ10Lz+rHEO/SkfE72CNVGpX7VDDr6yx0hL4vtGVGEbloRKwF0w1Qb9
         U2CQ==
X-Gm-Message-State: AOJu0Yyt0bdI8ouu//QEiS1ipWNjXTXZAVPQR5gfU2VRwa/w5RsZp9WW
	zOSK4OBJuiMVwSdkWO+xczqfBfeNZ8OOYujoYZXaxH89PYXiTt/8B/7jMAcz
X-Google-Smtp-Source: AGHT+IHTUje8yIPxKFZYWW3hfLgmp4LDhlggTVeZyQmVCbaHU7mc8OlD2vNztC903H84MmHD22J8ew==
X-Received: by 2002:a05:6512:786:b0:51a:c3f2:69a8 with SMTP id x6-20020a056512078600b0051ac3f269a8mr332574lfr.53.1714671886039;
        Thu, 02 May 2024 10:44:46 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v3 9/9] xen/arm64: lib: Use the generic xen/linkage.h macros
Date: Wed,  1 May 2024 05:54:48 +0200
Message-Id: <20240501035448.964625-10-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240501035448.964625-1-edgar.iglesias@gmail.com>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/lib/memchr.S  | 4 ++--
 xen/arch/arm/arm64/lib/memcmp.S  | 4 ++--
 xen/arch/arm/arm64/lib/memcpy.S  | 4 ++--
 xen/arch/arm/arm64/lib/memmove.S | 4 ++--
 xen/arch/arm/arm64/lib/memset.S  | 4 ++--
 xen/arch/arm/arm64/lib/strchr.S  | 4 ++--
 xen/arch/arm/arm64/lib/strcmp.S  | 4 ++--
 xen/arch/arm/arm64/lib/strlen.S  | 4 ++--
 xen/arch/arm/arm64/lib/strncmp.S | 4 ++--
 xen/arch/arm/arm64/lib/strnlen.S | 4 ++--
 xen/arch/arm/arm64/lib/strrchr.S | 4 ++--
 11 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/xen/arch/arm/arm64/lib/memchr.S b/xen/arch/arm/arm64/lib/memchr.S
index 81f113bb1c..3d8aeca3ca 100644
--- a/xen/arch/arm/arm64/lib/memchr.S
+++ b/xen/arch/arm/arm64/lib/memchr.S
@@ -29,7 +29,7 @@
  * Returns:
  *	x0 - address of first occurrence of 'c' or 0
  */
-ENTRY(memchr)
+FUNC(memchr)
 	and	w1, w1, #0xff
 1:	subs	x2, x2, #1
 	b.mi	2f
@@ -40,4 +40,4 @@ ENTRY(memchr)
 	ret
 2:	mov	x0, #0
 	ret
-ENDPROC(memchr)
+END(memchr)
diff --git a/xen/arch/arm/arm64/lib/memcmp.S b/xen/arch/arm/arm64/lib/memcmp.S
index 87c2537ffe..d77dd4ce52 100644
--- a/xen/arch/arm/arm64/lib/memcmp.S
+++ b/xen/arch/arm/arm64/lib/memcmp.S
@@ -57,7 +57,7 @@ pos		.req	x11
 limit_wd	.req	x12
 mask		.req	x13
 
-ENTRY(memcmp)
+FUNC(memcmp)
 	cbz	limit, .Lret0
 	eor	tmp1, src1, src2
 	tst	tmp1, #7
@@ -254,4 +254,4 @@ CPU_LE( rev	data2, data2 )
 .Lret0:
 	mov	result, #0
 	ret
-ENDPROC(memcmp)
+END(memcmp)
diff --git a/xen/arch/arm/arm64/lib/memcpy.S b/xen/arch/arm/arm64/lib/memcpy.S
index d90d20ef3e..1e04b79010 100644
--- a/xen/arch/arm/arm64/lib/memcpy.S
+++ b/xen/arch/arm/arm64/lib/memcpy.S
@@ -55,7 +55,7 @@ C_h	.req	x12
 D_l	.req	x13
 D_h	.req	x14
 
-ENTRY(memcpy)
+FUNC(memcpy)
 	mov	dst, dstin
 	cmp	count, #16
 	/*When memory length is less than 16, the accessed are not aligned.*/
@@ -197,4 +197,4 @@ ENTRY(memcpy)
 	tst	count, #0x3f
 	b.ne	.Ltail63
 	ret
-ENDPROC(memcpy)
+END(memcpy)
diff --git a/xen/arch/arm/arm64/lib/memmove.S b/xen/arch/arm/arm64/lib/memmove.S
index a49de845d0..14438dbe9c 100644
--- a/xen/arch/arm/arm64/lib/memmove.S
+++ b/xen/arch/arm/arm64/lib/memmove.S
@@ -56,7 +56,7 @@ C_h	.req	x12
 D_l	.req	x13
 D_h	.req	x14
 
-ENTRY(memmove)
+FUNC(memmove)
 	cmp	dstin, src
 	b.lo	memcpy
 	add	tmp1, src, count
@@ -193,4 +193,4 @@ ENTRY(memmove)
 	tst	count, #0x3f
 	b.ne	.Ltail63
 	ret
-ENDPROC(memmove)
+END(memmove)
diff --git a/xen/arch/arm/arm64/lib/memset.S b/xen/arch/arm/arm64/lib/memset.S
index 5bf751521b..367fa60175 100644
--- a/xen/arch/arm/arm64/lib/memset.S
+++ b/xen/arch/arm/arm64/lib/memset.S
@@ -53,7 +53,7 @@ dst		.req	x8
 tmp3w		.req	w9
 tmp3		.req	x9
 
-ENTRY(memset)
+FUNC(memset)
 	mov	dst, dstin	/* Preserve return value.  */
 	and	A_lw, val, #255
 	orr	A_lw, A_lw, A_lw, lsl #8
@@ -212,4 +212,4 @@ ENTRY(memset)
 	ands	count, count, zva_bits_x
 	b.ne	.Ltail_maybe_long
 	ret
-ENDPROC(memset)
+END(memset)
diff --git a/xen/arch/arm/arm64/lib/strchr.S b/xen/arch/arm/arm64/lib/strchr.S
index 0506b0ff7f..83fd81e8ef 100644
--- a/xen/arch/arm/arm64/lib/strchr.S
+++ b/xen/arch/arm/arm64/lib/strchr.S
@@ -27,7 +27,7 @@
  * Returns:
  *	x0 - address of first occurrence of 'c' or 0
  */
-ENTRY(strchr)
+FUNC(strchr)
 	and	w1, w1, #0xff
 1:	ldrb	w2, [x0], #1
 	cmp	w2, w1
@@ -37,4 +37,4 @@ ENTRY(strchr)
 	cmp	w2, w1
 	csel	x0, x0, xzr, eq
 	ret
-ENDPROC(strchr)
+END(strchr)
diff --git a/xen/arch/arm/arm64/lib/strcmp.S b/xen/arch/arm/arm64/lib/strcmp.S
index c6f42dd255..7677108e26 100644
--- a/xen/arch/arm/arm64/lib/strcmp.S
+++ b/xen/arch/arm/arm64/lib/strcmp.S
@@ -59,7 +59,7 @@ tmp3		.req	x9
 zeroones	.req	x10
 pos		.req	x11
 
-ENTRY(strcmp)
+FUNC(strcmp)
 	eor	tmp1, src1, src2
 	mov	zeroones, #REP8_01
 	tst	tmp1, #7
@@ -230,4 +230,4 @@ CPU_BE(	orr	syndrome, diff, has_nul )
 	lsr	data1, data1, #56
 	sub	result, data1, data2, lsr #56
 	ret
-ENDPROC(strcmp)
+END(strcmp)
diff --git a/xen/arch/arm/arm64/lib/strlen.S b/xen/arch/arm/arm64/lib/strlen.S
index fb6aaf1a6a..10feedaf81 100644
--- a/xen/arch/arm/arm64/lib/strlen.S
+++ b/xen/arch/arm/arm64/lib/strlen.S
@@ -56,7 +56,7 @@ pos		.req	x12
 #define REP8_7f 0x7f7f7f7f7f7f7f7f
 #define REP8_80 0x8080808080808080
 
-ENTRY(strlen)
+FUNC(strlen)
 	mov	zeroones, #REP8_01
 	bic	src, srcin, #15
 	ands	tmp1, srcin, #15
@@ -123,4 +123,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp1 )	/* Shift (tmp1 & 63).  */
 	csinv	data1, data1, xzr, le
 	csel	data2, data2, data2a, le
 	b	.Lrealigned
-ENDPROC(strlen)
+END(strlen)
diff --git a/xen/arch/arm/arm64/lib/strncmp.S b/xen/arch/arm/arm64/lib/strncmp.S
index a4a0f779f5..c00a641fc7 100644
--- a/xen/arch/arm/arm64/lib/strncmp.S
+++ b/xen/arch/arm/arm64/lib/strncmp.S
@@ -64,7 +64,7 @@ limit_wd	.req	x13
 mask		.req	x14
 endloop		.req	x15
 
-ENTRY(strncmp)
+FUNC(strncmp)
 	cbz	limit, .Lret0
 	eor	tmp1, src1, src2
 	mov	zeroones, #REP8_01
@@ -307,4 +307,4 @@ CPU_BE( orr	syndrome, diff, has_nul )
 .Lret0:
 	mov	result, #0
 	ret
-ENDPROC(strncmp)
+END(strncmp)
diff --git a/xen/arch/arm/arm64/lib/strnlen.S b/xen/arch/arm/arm64/lib/strnlen.S
index 81c8e8b54e..2059ba782c 100644
--- a/xen/arch/arm/arm64/lib/strnlen.S
+++ b/xen/arch/arm/arm64/lib/strnlen.S
@@ -59,7 +59,7 @@ limit_wd	.req	x14
 #define REP8_7f 0x7f7f7f7f7f7f7f7f
 #define REP8_80 0x8080808080808080
 
-ENTRY(strnlen)
+FUNC(strnlen)
 	cbz	limit, .Lhit_limit
 	mov	zeroones, #REP8_01
 	bic	src, srcin, #15
@@ -168,4 +168,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp4 )	/* Shift (tmp1 & 63).  */
 .Lhit_limit:
 	mov	len, limit
 	ret
-ENDPROC(strnlen)
+END(strnlen)
diff --git a/xen/arch/arm/arm64/lib/strrchr.S b/xen/arch/arm/arm64/lib/strrchr.S
index 07059983f8..81033c0822 100644
--- a/xen/arch/arm/arm64/lib/strrchr.S
+++ b/xen/arch/arm/arm64/lib/strrchr.S
@@ -26,7 +26,7 @@
  * Returns:
  *	x0 - address of last occurrence of 'c' or 0
  */
-ENTRY(strrchr)
+FUNC(strrchr)
 	mov	x3, #0
 	and	w1, w1, #0xff
 1:	ldrb	w2, [x0], #1
@@ -37,4 +37,4 @@ ENTRY(strrchr)
 	b	1b
 2:	mov	x0, x3
 	ret
-ENDPROC(strrchr)
+END(strrchr)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 02 18:02:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:02:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716097.1118161 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2alT-00036H-1G; Thu, 02 May 2024 18:02:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716097.1118161; Thu, 02 May 2024 18:02:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2alS-00036A-Uy; Thu, 02 May 2024 18:02:38 +0000
Received: by outflank-mailman (input) for mailman id 716097;
 Thu, 02 May 2024 18:02:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2alR-000364-Rd
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:02:37 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 26f126d0-08ae-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 20:02:35 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D50AD61A1D;
 Thu,  2 May 2024 18:02:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89341C113CC;
 Thu,  2 May 2024 18:02:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26f126d0-08ae-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714672953;
	bh=/ZteNs6x9OubUJqG1B50uqZmMsB3ugNtVG566Vfz8sY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=SVu7/9084mtuCkBHG7NyEZnlldSwDsUHWHJxe4sKxla7k1M4lKwB0EPZiJMOURKtd
	 iFeHG0+mb2q6U2ZMW2qp1JxqckPBzcX6h0l+iyp1uCf1foMqfNOmPuj+gvkeexoq5t
	 /V5+Tox5R+9v5oSqQ8sPH/HtdrGoVoC4zNNDVfZAH7uYq1+Nkq0mu6LEj1QXPHUaWQ
	 DY6ij8f5hgdnO0nEyR3I0z6SnkbPhdBjGQ/yuhyFOQbxhtSRT/ReEHmPHZUrQmTXOb
	 VtGbJ1PN6uSp+qGg15/IBBKSQincxMlVB+O+Eu9yjk+gd/WxX7AHOM4HOMihxMQbaT
	 eT9ai3vYLUVuQ==
Date: Thu, 2 May 2024 11:02:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 07/15] xen/overlay: Enable device tree overlay assignment
 to running domains
In-Reply-To: <e3c7098b-a876-48f3-8ccb-18a2ced8521c@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405021101200.624854@ubuntu-linux-20-04-desktop>
References: <20240424033449.168398-1-xin.wang2@amd.com> <20240424033449.168398-8-xin.wang2@amd.com> <da67f90f-6807-4fdc-b65b-f7a4ba9f78ad@suse.com> <97a401ff-4b5c-4e6a-8d9c-e36305ec64f5@amd.com> <128c7e68-0431-44e9-b4fc-96cc46e158ad@xen.org>
 <e3c7098b-a876-48f3-8ccb-18a2ced8521c@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-878510409-1714672954=:624854"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-878510409-1714672954=:624854
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 30 Apr 2024, Henry Wang wrote:
> Hi Julien,
> 
> On 4/30/2024 1:34 AM, Julien Grall wrote:
> > On 29/04/2024 04:36, Henry Wang wrote:
> > > Hi Jan, Julien, Stefano,
> > 
> > Hi Henry,
> > 
> > > On 4/24/2024 2:05 PM, Jan Beulich wrote:
> > > > On 24.04.2024 05:34, Henry Wang wrote:
> > > > > --- a/xen/include/public/sysctl.h
> > > > > +++ b/xen/include/public/sysctl.h
> > > > > @@ -1197,7 +1197,9 @@ struct xen_sysctl_dt_overlay {
> > > > >   #define XEN_SYSCTL_DT_OVERLAY_ADD                   1
> > > > >   #define XEN_SYSCTL_DT_OVERLAY_REMOVE                2
> > > > >       uint8_t overlay_op;                     /* IN: Add or remove. */
> > > > > -    uint8_t pad[3];                         /* IN: Must be zero. */
> > > > > +    bool domain_mapping;                    /* IN: True of False. */
> > > > > +    uint8_t pad[2];                         /* IN: Must be zero. */
> > > > > +    uint32_t domain_id;
> > > > >   };
> > > > If you merely re-purposed padding fields, all would be fine without
> > > > bumping the interface version. Yet you don't, albeit for an unclear
> > > > reason: Why uint32_t rather than domid_t? And on top of that - why a
> > > > separate boolean when you could use e.g. DOMID_INVALID to indicate
> > > > "no domain mapping"?
> > > 
> > > I think both of your suggestion make great sense. I will follow the
> > > suggestion in v2.
> > > 
> > > > That said - anything taking a domain ID is certainly suspicious in a
> > > > sysctl. Judging from the description you really mean this to be a
> > > > domctl. Anything else will require extra justification.
> > > 
> > > I also think a domctl is better. I had a look at the history of the
> > > already merged series, it looks like in the first version of merged part 1
> > > [1], the hypercall was implemented as the domctl in the beginning but
> > > later in v2 changed to sysctl. I think this makes sense as the scope of
> > > that time is just to make Xen aware of the device tree node via Xen device
> > > tree.
> > > 
> > > However this is now a problem for the current part where the scope (and
> > > the end goal) is extended to assign the added device to Linux Dom0/DomU
> > > via device tree overlays. I am not sure which way is better, should we
> > > repurposing the sysctl to domctl or maybe add another domctl (I am
> > > worrying about the duplication because basically we need the same sysctl
> > > functionality but now with a domid in it)? What do you think?
> > 
> > I am not entirely sure this is a good idea to try to add the device in Xen
> > and attach it to the guests at the same time. Imagine the following
> > situation:
> > 
> > 1) Add and attach devices
> > 2) The domain is rebooted
> > 3) Detach and remove devices
> > 
> > After step 2, you technically have a new domain. You could have also a case
> > where this is a completely different guest. So the flow would look a little
> > bit weird (you create the DT overlay with domain A but remove with domain
> > B).
> > 
> > So, at the moment, it feels like the add/attach (resp detech/remove)
> > operations should happen separately.
> > 
> > Can you clarify why you want to add devices to Xen and attach to a guest
> > within a single hypercall?
> 
> Sorry I don't know if there is any specific thoughts on the design of using a
> single hypercall to do both add devices to Xen device tree and assign the
> device to the guest. In fact seeing your above comments, I think separating
> these two functionality to two xl commands using separated hypercalls would
> indeed be a better idea. Thank you for the suggestion!
> 
> To make sure I understand correctly, would you mind confirming if below
> actions for v2 make sense to you? Thanks!
> - Only use the XEN_SYSCTL_DT_OVERLAY_{ADD, REMOVE} sysctls to add/remove
> overlay to Xen device tree
> - Introduce the xl dt-overlay attach <domid> command and respective domctls to
> do the device assignment for the overlay to domain.

I think two hypercalls is OK. The original idea was to have a single xl
command to do the operation for user convenience (even that is not a
hard requirement) but that can result easily in two hypercalls.
--8323329-878510409-1714672954=:624854--


From xen-devel-bounces@lists.xenproject.org Thu May 02 18:08:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:08:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716102.1118170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2arN-0004Vp-L1; Thu, 02 May 2024 18:08:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716102.1118170; Thu, 02 May 2024 18:08:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2arN-0004Vi-IK; Thu, 02 May 2024 18:08:45 +0000
Received: by outflank-mailman (input) for mailman id 716102;
 Thu, 02 May 2024 18:08:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2arL-0004UP-Kh
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:08:43 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ffeb29d4-08ae-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 20:08:40 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 7D9F2CE1788;
 Thu,  2 May 2024 18:08:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1E7FC4AF1B;
 Thu,  2 May 2024 18:08:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffeb29d4-08ae-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714673312;
	bh=KKKjylRbEkEZP4mHEvG+hl/hr/LZ5arERC+XSWzHRSc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OIc7/DeVq3NldQyZVjmB5zS5CNntGJ49i0uOTOuFmy5AWdVp+Nf7SRWMlIunZAukD
	 VqsoTWBc7+M8LJhAIokiM2SvaJm0foazg9Ac56rZLGqAVbi3gXkK/dCO1sCuyIeqzs
	 4+LRdmufApixNJwALsiTige93L+Tne/r305P7P4hL5ZM/tVA1i/yVnnls6rXz/Cze5
	 VpnoeKSdZ3Pr63cgjbPUiR6Vd/iuCxgUPG91sHnB6gMTuEZBF63ZmXYXn0hPy29uZd
	 I/WSy4Rr0XYpfhZIgkKOZSV77/EzP4R+xO6Hh0OAkI6s8iv3gBlq3Z8ekd/KdfcH4X
	 cIdTY9plCvTFA==
Date: Thu, 2 May 2024 11:08:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Alec Kwapis <alec.kwapis@medtronic.com>
Subject: Re: [PATCH 2/3] xen/arm, tools: Add a new HVM_PARAM_MAGIC_BASE_PFN
 key in HVMOP
In-Reply-To: <20240426031455.579637-3-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405021104390.624854@ubuntu-linux-20-04-desktop>
References: <20240426031455.579637-1-xin.wang2@amd.com> <20240426031455.579637-3-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 26 Apr 2024, Henry Wang wrote:
> For use cases such as Dom0less PV drivers, a mechanism to communicate
> Dom0less DomU's static data with the runtime control plane (Dom0) is
> needed. Since on Arm HVMOP is already the existing approach to address
> such use cases (for example the allocation of HVM_PARAM_CALLBACK_IRQ),
> add a new HVMOP key HVM_PARAM_MAGIC_BASE_PFN for storing the magic
> page region base PFN. The value will be set at Dom0less DomU
> construction time after Dom0less DomU's magic page region has been
> allocated.
> 
> To keep consistent, also set the value for HVM_PARAM_MAGIC_BASE_PFN
> for libxl guests in alloc_magic_pages().
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>  tools/libs/guest/xg_dom_arm.c   | 2 ++
>  xen/arch/arm/dom0less-build.c   | 2 ++
>  xen/arch/arm/hvm.c              | 1 +
>  xen/include/public/hvm/params.h | 1 +
>  4 files changed, 6 insertions(+)
> 
> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
> index 8cc7f27dbb..3c08782d1d 100644
> --- a/tools/libs/guest/xg_dom_arm.c
> +++ b/tools/libs/guest/xg_dom_arm.c
> @@ -74,6 +74,8 @@ static int alloc_magic_pages(struct xc_dom_image *dom)
>      xc_clear_domain_page(dom->xch, dom->guest_domid, base + MEMACCESS_PFN_OFFSET);
>      xc_clear_domain_page(dom->xch, dom->guest_domid, dom->vuart_gfn);
>  
> +    xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_MAGIC_BASE_PFN,
> +            base);
>      xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_CONSOLE_PFN,
>              dom->console_pfn);
>      xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_STORE_PFN,
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 40dc85c759..72187c167d 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -861,6 +861,8 @@ static int __init construct_domU(struct domain *d,
>              free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
>              return rc;
>          }
> +
> +        d->arch.hvm.params[HVM_PARAM_MAGIC_BASE_PFN] = gfn_x(gfn);

I apologize as I have not read the whole email thread in reply to this
patch.

Why do we need to introduce a new hvm param instead of just setting
HVM_PARAM_CONSOLE_PFN and HVM_PARAM_STORE_PFN directly here?



From xen-devel-bounces@lists.xenproject.org Thu May 02 18:10:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:10:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716106.1118182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2atU-0006O4-2j; Thu, 02 May 2024 18:10:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716106.1118182; Thu, 02 May 2024 18:10:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2atT-0006Nx-U2; Thu, 02 May 2024 18:10:55 +0000
Received: by outflank-mailman (input) for mailman id 716106;
 Thu, 02 May 2024 18:10:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2atS-0006Np-MJ
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:10:54 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4fd92eb0-08af-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:10:52 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a597c192246so53273366b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 11:10:52 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 el4-20020a056402360400b005725ffd7305sm750255edb.75.2024.05.02.11.10.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 May 2024 11:10:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fd92eb0-08af-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714673451; x=1715278251; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=vsQHNid/N6RXFYLsFWeAhF7QhEAHIkMrktwhE6dS5a4=;
        b=GdbxStwPIVinYKPjVVaby2HlEGgcQqfV/wFXnMu7953kizK8clAG/QDtOvjmB13V6m
         9/gY4wz+dxtHgsh7OfXlJvHfzj0yqoZ8O2QRuBtGpf6fNr1FuPeEd5Ilh5nM72gCYBa4
         Nd/JYPWzG2vcx+XFkXiBtN4JdIaKurHXSm+Q0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714673451; x=1715278251;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vsQHNid/N6RXFYLsFWeAhF7QhEAHIkMrktwhE6dS5a4=;
        b=YyA8BEBMESwEc2u4woXV1NOQW7oijn39DFCFHhZHEqXjGOxJSCtLi01536EVpnD0Ft
         cDUrfY/iN5Y33W19zEOuoDB3ZFUx+oJZioF1AzOD3EqcBWcDf2bJMz4ci0caYrkKRS1Z
         FM8jdT9niP5ULfOZd18vCECmtkNCsNfqpngNnLMqIYBy4aQHwg6QjDdEDm6zuTqNw7HI
         PJ/CI4WbjKxF7UHS1RE2bO5ylcph0H1hFD4hZVkdLjCUUWSGUaUP+RcoXEa303ksqiXg
         WtVwkOrnQWlPxdAjfBoS41bUqAJOcCwlCE0m7GLItYkUuX0q6wLYnWES4HhVgypZ1EIJ
         Rqzw==
X-Gm-Message-State: AOJu0Yw9l8qYLhsLDQdjWu6eCmIvH946rDEJTN4xpwP/1L+rw+Ssi9jV
	ZKaI5r0ZTLqRX1uYnhN8nnaslWlmRW2y8y5jiFCXBS4qdg+90ytLX81LYNYsq/1i44WSd8SFY13
	a
X-Google-Smtp-Source: AGHT+IEayK8m84wT3uQX6fk9BhEtM+x6MA6TGfiapeoV5erhx30/LRiCTbi6zw417BHBiwDP4/hjxQ==
X-Received: by 2002:a50:aad5:0:b0:572:a089:75cc with SMTP id r21-20020a50aad5000000b00572a08975ccmr216378edc.5.1714673451517;
        Thu, 02 May 2024 11:10:51 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH] xen/Kconfig: Drop the final remnants of ---help---
Date: Thu,  2 May 2024 19:10:49 +0100
Message-Id: <20240502181049.1361384-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

We deprecated the use of ---help--- a while ago, but a lot of new content
copy&pastes bad examples.  Convert the remaining instances, and update
Kconfig's parser to no longer recongise it.

This now causes builds to fail with:

  Kconfig.debug:8: syntax error
  Kconfig.debug:7: unknown statement "---help---"

which short circuits one common piece of churn in new content.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>

For 4.19.  This cleans up a legacy we've been wanting to get rid of for a
while, and will be least disruptive on people if it gets in ahead of most
people starting work for 4.20.
---
 xen/Kconfig                     |  2 +-
 xen/Kconfig.debug               | 28 +++++++++----------
 xen/arch/arm/Kconfig            |  8 +++---
 xen/arch/arm/platforms/Kconfig  | 12 ++++-----
 xen/arch/x86/Kconfig            | 32 +++++++++++-----------
 xen/common/Kconfig              | 48 ++++++++++++++++-----------------
 xen/common/sched/Kconfig        | 10 +++----
 xen/drivers/passthrough/Kconfig |  8 +++---
 xen/drivers/video/Kconfig       |  2 +-
 xen/tools/kconfig/lexer.l       |  2 +-
 10 files changed, 76 insertions(+), 76 deletions(-)

diff --git a/xen/Kconfig b/xen/Kconfig
index 1e1b041fd52f..e459cdac0cd7 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -84,7 +84,7 @@ config UNSUPPORTED
 config LTO
 	bool "Link Time Optimisation"
 	depends on BROKEN
-	---help---
+	help
 	  Enable Link Time Optimisation.
 
 	  If unsure, say N.
diff --git a/xen/Kconfig.debug b/xen/Kconfig.debug
index fa81853e9385..61b24ac552cd 100644
--- a/xen/Kconfig.debug
+++ b/xen/Kconfig.debug
@@ -4,7 +4,7 @@ menu "Debugging Options"
 config DEBUG
 	bool "Developer Checks"
 	default y
-	---help---
+	help
 	  If you say Y here this will enable developer checks such as asserts
 	  and extra printks. This option is intended for development purposes
 	  only, and not for production use.
@@ -17,14 +17,14 @@ config GDBSX
 	bool "Guest debugging with gdbsx"
 	depends on X86
 	default y
-	---help---
+	help
 	  If you want to enable support for debugging guests from dom0 via
 	  gdbsx then say Y.
 
 config FRAME_POINTER
 	bool "Compile Xen with frame pointers"
 	default DEBUG
-	---help---
+	help
 	  If you say Y here the resulting Xen will be slightly larger and
 	  maybe slower, but it gives very useful debugging information
 	  in case of any Xen bugs.
@@ -33,7 +33,7 @@ config COVERAGE
 	bool "Code coverage support"
 	depends on !LIVEPATCH
 	select SUPPRESS_DUPLICATE_SYMBOL_WARNINGS if !ENFORCE_UNIQUE_SYMBOLS
-	---help---
+	help
 	  Enable code coverage support.
 
 	  If unsure, say N here.
@@ -41,7 +41,7 @@ config COVERAGE
 config DEBUG_LOCK_PROFILE
 	bool "Lock Profiling"
 	select DEBUG_LOCKS
-	---help---
+	help
 	  Lock profiling allows you to see how often locks are taken and blocked.
 	  You can use serial console to print (and reset) using 'l' and 'L'
 	  respectively, or the 'xenlockprof' tool.
@@ -49,13 +49,13 @@ config DEBUG_LOCK_PROFILE
 config DEBUG_LOCKS
 	bool "Lock debugging"
 	default DEBUG
-	---help---
+	help
 	  Enable debugging features of lock handling.  Some additional
 	  checks will be performed when acquiring and releasing locks.
 
 config PERF_COUNTERS
 	bool "Performance Counters"
-	---help---
+	help
 	  Enables software performance counters that allows you to analyze
 	  bottlenecks in the system.  To access this data you can use serial
 	  console to print (and reset) using 'p' and 'P' respectively, or
@@ -64,21 +64,21 @@ config PERF_COUNTERS
 config PERF_ARRAYS
 	bool "Performance Counter Array Histograms"
 	depends on PERF_COUNTERS
-	---help---
+	help
 	  Enables software performance counter array histograms.
 
 
 config VERBOSE_DEBUG
 	bool "Verbose debug messages"
 	default DEBUG
-	---help---
+	help
 	  Guest output from HYPERVISOR_console_io and hypervisor parsing
 	  ELF images (dom0) will be logged in the Xen ring buffer.
 
 config DEVICE_TREE_DEBUG
 	bool "Device tree debug messages"
 	depends on HAS_DEVICE_TREE
-	---help---
+	help
 	  Device tree parsing and DOM0 device tree building messages are
 	  logged in the Xen ring buffer.
 	  If unsure, say N here.
@@ -86,14 +86,14 @@ config DEVICE_TREE_DEBUG
 config SCRUB_DEBUG
 	bool "Page scrubbing test"
 	default DEBUG
-	---help---
+	help
 	  Verify that pages that need to be scrubbed before being allocated to
 	  a guest are indeed scrubbed.
 
 config UBSAN
 	bool "Undefined behaviour sanitizer"
 	depends on HAS_UBSAN
-	---help---
+	help
 	  Enable undefined behaviour sanitizer. It uses compiler to insert code
 	  snippets so that undefined behaviours in C are detected during runtime.
 	  This requires a UBSAN capable compiler and it is a debug only feature.
@@ -109,7 +109,7 @@ config UBSAN_FATAL
 
 config DEBUG_TRACE
 	bool "Debug trace support"
-	---help---
+	help
 	  Debug trace enables to record debug trace messages which are printed
 	  either directly to the console or are printed to console in case of
 	  a system crash.
@@ -117,7 +117,7 @@ config DEBUG_TRACE
 config XMEM_POOL_POISON
 	bool "Poison free xenpool blocks"
 	default DEBUG
-	---help---
+	help
 	  Poison free blocks with 0xAA bytes and verify them when a block is
 	  allocated in order to spot use-after-free issues.
 
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index f8139a773a43..21d03d9f4424 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -67,7 +67,7 @@ source "arch/Kconfig"
 config ACPI
 	bool "ACPI (Advanced Configuration and Power Interface) Support (UNSUPPORTED)" if UNSUPPORTED
 	depends on ARM_64 && ARM_EFI
-	---help---
+	help
 
 	  Advanced Configuration and Power Interface (ACPI) support for Xen is
 	  an alternative to device tree on ARM64. This requires UEFI.
@@ -102,7 +102,7 @@ config GICV3
 	depends on !NEW_VGIC
 	default n if ARM_32
 	default y if ARM_64
-	---help---
+	help
 
 	  Driver for the ARM Generic Interrupt Controller v3.
 	  If unsure, use the default setting.
@@ -132,7 +132,7 @@ config HVM
 config NEW_VGIC
 	bool "Use new VGIC implementation"
 	select GICV2
-	---help---
+	help
 
 	This is an alternative implementation of the ARM GIC interrupt
 	controller emulation, based on the Linux/KVM VGIC. It has a better
@@ -147,7 +147,7 @@ config NEW_VGIC
 config SBSA_VUART_CONSOLE
 	bool "Emulated SBSA UART console support"
 	default y
-	---help---
+	help
 	  Allows a guest to use SBSA Generic UART as a console. The
 	  SBSA Generic UART implements a subset of ARM PL011 UART.
 
diff --git a/xen/arch/arm/platforms/Kconfig b/xen/arch/arm/platforms/Kconfig
index c93a6b275620..76f7e76b1bf4 100644
--- a/xen/arch/arm/platforms/Kconfig
+++ b/xen/arch/arm/platforms/Kconfig
@@ -1,14 +1,14 @@
 choice
 	prompt "Platform Support"
 	default ALL_PLAT
-	---help---
+	help
 	Choose which hardware platform to enable in Xen.
 
 	If unsure, choose ALL_PLAT.
 
 config ALL_PLAT
 	bool "All Platforms"
-	---help---
+	help
 	Enable support for all available hardware platforms. It doesn't
 	automatically select any of the related drivers.
 
@@ -17,7 +17,7 @@ config QEMU
 	depends on ARM_64
 	select GICV3
 	select HAS_PL011
-	---help---
+	help
 	Enable all the required drivers for QEMU aarch64 virt emulated
 	machine.
 
@@ -26,7 +26,7 @@ config RCAR3
 	depends on ARM_64
 	select HAS_SCIF
 	select IPMMU_VMSA
-	---help---
+	help
 	Enable all the required drivers for Renesas RCar3
 
 config MPSOC
@@ -34,12 +34,12 @@ config MPSOC
 	depends on ARM_64
 	select HAS_CADENCE_UART
 	select ARM_SMMU
-	---help---
+	help
 	Enable all the required drivers for Xilinx Ultrascale+ MPSoC
 
 config NO_PLAT
 	bool "No Platforms"
-	---help---
+	help
 	Do not enable specific support for any platform.
 
 endchoice
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 2b6248774d8f..7e03e4bc5546 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -58,7 +58,7 @@ source "arch/Kconfig"
 config PV
 	def_bool y
 	prompt "PV support"
-	---help---
+	help
 	  Interfaces to support PV domains. These require guest kernel support
 	  to run as a PV guest, but don't require any specific hardware support.
 
@@ -71,7 +71,7 @@ config PV32
 	depends on PV
 	default PV_SHIM
 	select COMPAT
-	---help---
+	help
 	  The 32bit PV ABI uses Ring1, an area of the x86 architecture which
 	  was deprecated and mostly removed in the AMD64 spec.  As a result,
 	  it occasionally conflicts with newer x86 hardware features, causing
@@ -90,7 +90,7 @@ config PV_LINEAR_PT
        bool "Support for PV linear pagetables"
        depends on PV
        default y
-       ---help---
+       help
          Linear pagetables (also called "recursive pagetables") refers
          to the practice of a guest operating system having pagetable
          entries pointing to other pagetables of the same level (i.e.,
@@ -113,7 +113,7 @@ config HVM
 	select COMPAT
 	select IOREQ_SERVER
 	select MEM_ACCESS_ALWAYS_ON
-	---help---
+	help
 	  Interfaces to support HVM domains.  HVM domains require hardware
 	  virtualisation extensions (e.g. Intel VT-x, AMD SVM), but can boot
 	  guests which have no specific Xen knowledge.
@@ -126,7 +126,7 @@ config XEN_SHSTK
 	bool "Supervisor Shadow Stacks"
 	depends on HAS_AS_CET_SS
 	default y
-	---help---
+	help
 	  Control-flow Enforcement Technology (CET) is a set of features in
 	  hardware designed to combat Return-oriented Programming (ROP, also
 	  call/jump COP/JOP) attacks.  Shadow Stacks are one CET feature
@@ -152,7 +152,7 @@ config SHADOW_PAGING
 	bool "Shadow Paging"
 	default !PV_SHIM_EXCLUSIVE
 	depends on PV || HVM
-	---help---
+	help
 
           Shadow paging is a software alternative to hardware paging support
           (Intel EPT, AMD NPT).
@@ -171,7 +171,7 @@ config SHADOW_PAGING
 config BIGMEM
 	bool "big memory support"
 	default n
-	---help---
+	help
 	  Allows Xen to support up to 123Tb of memory.
 
 	  This requires enlarging struct page_info as well as shrinking
@@ -183,7 +183,7 @@ config HVM_FEP
 	bool "HVM Forced Emulation Prefix support (UNSUPPORTED)" if UNSUPPORTED
 	default DEBUG
 	depends on HVM
-	---help---
+	help
 
 	  Compiles in a feature that allows HVM guest to arbitrarily
 	  exercise the instruction emulator.
@@ -203,7 +203,7 @@ config TBOOT
 	depends on UNSUPPORTED
 	default !PV_SHIM_EXCLUSIVE
 	select CRYPTO
-	---help---
+	help
 	  Allows support for Trusted Boot using the Intel(R) Trusted Execution
 	  Technology (TXT)
 
@@ -213,14 +213,14 @@ choice
 	prompt "Alignment of Xen image"
 	default XEN_ALIGN_2M if PV_SHIM_EXCLUSIVE
 	default XEN_ALIGN_DEFAULT
-	---help---
+	help
 	  Specify alignment for Xen image.
 
 	  If unsure, choose "default".
 
 config XEN_ALIGN_DEFAULT
 	bool "Default alignment"
-	---help---
+	help
 	  Pick alignment according to build variants.
 
 	  For EFI build the default alignment is 2M. For ELF build
@@ -281,7 +281,7 @@ config GUEST
 config XEN_GUEST
 	bool "Xen Guest"
 	select GUEST
-	---help---
+	help
 	  Support for Xen detecting when it is running under Xen.
 
 	  If unsure, say N.
@@ -290,7 +290,7 @@ config PVH_GUEST
 	def_bool y
 	prompt "PVH Guest"
 	depends on XEN_GUEST
-	---help---
+	help
 	  Support booting using the PVH ABI.
 
 	  If unsure, say Y.
@@ -299,7 +299,7 @@ config PV_SHIM
 	def_bool y
 	prompt "PV Shim"
 	depends on PV && XEN_GUEST
-	---help---
+	help
 	  Build Xen with a mode which acts as a shim to allow PV guest to run
 	  in an HVM/PVH container. This mode can only be enabled with command
 	  line option.
@@ -309,7 +309,7 @@ config PV_SHIM
 config PV_SHIM_EXCLUSIVE
 	bool "PV Shim Exclusive"
 	depends on PV_SHIM
-	---help---
+	help
 	  Build Xen in a way which unconditionally assumes PV_SHIM mode.  This
 	  option is only intended for use when building a dedicated PV Shim
 	  firmware, and will not function correctly in other scenarios.
@@ -321,7 +321,7 @@ if !PV_SHIM_EXCLUSIVE
 config HYPERV_GUEST
 	bool "Hyper-V Guest"
 	select GUEST
-	---help---
+	help
 	  Support for Xen detecting when it is running under Hyper-V.
 
 	  If unsure, say N.
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index cff3166ff923..565ceda741b9 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -15,7 +15,7 @@ config CORE_PARKING
 config GRANT_TABLE
 	bool "Grant table support" if EXPERT
 	default y
-	---help---
+	help
 	  Grant table provides a generic mechanism to memory sharing
 	  between domains. This shared memory interface underpins the
 	  split device drivers for block and network IO in a classic
@@ -90,7 +90,7 @@ config MEM_ACCESS
 	def_bool MEM_ACCESS_ALWAYS_ON
 	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
 	depends on HVM
-	---help---
+	help
 
 	  Framework to configure memory access types for guests and receive
 	  related events in userspace.
@@ -133,7 +133,7 @@ config INDIRECT_THUNK
 config SPECULATIVE_HARDEN_ARRAY
 	bool "Speculative Array Hardening"
 	default y
-	---help---
+	help
 	  Contemporary processors may use speculative execution as a
 	  performance optimisation, but this can potentially be abused by an
 	  attacker to leak data via speculative sidechannels.
@@ -154,7 +154,7 @@ config SPECULATIVE_HARDEN_BRANCH
 	bool "Speculative Branch Hardening"
 	default y
 	depends on X86
-        ---help---
+        help
 	  Contemporary processors may use speculative execution as a
 	  performance optimisation, but this can potentially be abused by an
 	  attacker to leak data via speculative sidechannels.
@@ -228,7 +228,7 @@ config DIT_DEFAULT
 config HYPFS
 	bool "Hypervisor file system support"
 	default y
-	---help---
+	help
 	  Support Xen hypervisor file system. This file system is used to
 	  present various hypervisor internal data to dom0 and in some
 	  cases to allow modifying settings. Disabling the support will
@@ -241,7 +241,7 @@ config HYPFS_CONFIG
 	bool "Provide hypervisor .config via hypfs entry"
 	default y
 	depends on HYPFS
-	---help---
+	help
 	  When enabled the contents of the .config file used to build the
 	  hypervisor are provided via the hypfs entry /buildinfo/config.
 
@@ -252,7 +252,7 @@ config IOREQ_SERVER
 	bool "IOREQ support (EXPERT)" if EXPERT && !X86
 	default X86
 	depends on HVM
-	---help---
+	help
 	  Enables generic mechanism for providing emulated devices to the guests.
 
 	  If unsure, say N.
@@ -261,7 +261,7 @@ config KEXEC
 	bool "kexec support"
 	default y
 	depends on HAS_KEXEC
-	---help---
+	help
 	  Allows a running Xen hypervisor to be replaced with another OS
 	  without rebooting. This is primarily used to execute a crash
 	  environment to collect information on a Xen hypervisor or dom0 crash.
@@ -270,7 +270,7 @@ config KEXEC
 
 config EFI_SET_VIRTUAL_ADDRESS_MAP
     bool "EFI: call SetVirtualAddressMap()" if EXPERT
-    ---help---
+    help
       Call EFI SetVirtualAddressMap() runtime service to setup memory map for
       further runtime services. According to UEFI spec, it isn't strictly
       necessary, but many UEFI implementations misbehave when this call is
@@ -292,7 +292,7 @@ config XENOPROF
 config XSM
 	bool "Xen Security Modules support"
 	default ARM
-	---help---
+	help
 	  Enables the security framework known as Xen Security Modules which
 	  allows administrators fine-grained control over a Xen domain and
 	  its capabilities by defining permissible interactions between domains,
@@ -305,7 +305,7 @@ config XSM_FLASK
 	def_bool y
 	prompt "FLux Advanced Security Kernel support"
 	depends on XSM
-	---help---
+	help
 	  Enables FLASK (FLux Advanced Security Kernel) as the access control
 	  mechanism used by the XSM framework.  This provides a mandatory access
 	  control framework by which security enforcement, isolation, and
@@ -318,7 +318,7 @@ config XSM_FLASK_AVC_STATS
 	def_bool y
 	prompt "Maintain statistics on the FLASK access vector cache" if EXPERT
 	depends on XSM_FLASK
-	---help---
+	help
 	  Maintain counters on the access vector cache that can be viewed using
 	  the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.  Disabling
 	  this will save a tiny amount of memory and time to update the stats.
@@ -329,7 +329,7 @@ config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
 	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
 	depends on XSM_FLASK
-	---help---
+	help
 	  This includes a default XSM policy in the hypervisor so that the
 	  bootloader does not need to load a policy to get sane behavior from an
 	  XSM-enabled hypervisor.  If this is disabled, a policy must be
@@ -345,7 +345,7 @@ config XSM_SILO
 	def_bool y
 	prompt "SILO support"
 	depends on XSM
-	---help---
+	help
 	  Enables SILO as the access control mechanism used by the XSM framework.
 	  This is not the default module, add boot parameter xsm=silo to choose
 	  it. This will deny any unmediated communication channels (grant tables
@@ -372,7 +372,7 @@ config LATE_HWDOM
 	bool "Dedicated hardware domain"
 	default n
 	depends on XSM && X86
-	---help---
+	help
 	  Allows the creation of a dedicated hardware domain distinct from
 	  domain 0 that manages devices without needing access to other
 	  privileged functionality such as the ability to manage domains.
@@ -390,7 +390,7 @@ config LATE_HWDOM
 
 config ARGO
 	bool "Argo: hypervisor-mediated interdomain communication (UNSUPPORTED)" if UNSUPPORTED
-	---help---
+	help
 	  Enables a hypercall for domains to ask the hypervisor to perform
 	  data transfer of messages between domains.
 
@@ -417,7 +417,7 @@ config LIVEPATCH
 	default X86
 	depends on "$(XEN_HAS_BUILD_ID)" = "y"
 	select CC_SPLIT_SECTIONS
-	---help---
+	help
 	  Allows a running Xen hypervisor to be dynamically patched using
 	  binary patches without rebooting. This is primarily used to binarily
 	  patch in the field an hypervisor with XSA fixes.
@@ -428,7 +428,7 @@ config FAST_SYMBOL_LOOKUP
 	bool "Fast symbol lookup (bigger binary)"
 	default y
 	depends on LIVEPATCH
-	---help---
+	help
 	  When searching for symbol addresses we can use the built-in system
 	  that is optimized for searching symbols using addresses as the key.
 	  However using it for the inverse (find address using the symbol name)
@@ -440,7 +440,7 @@ config FAST_SYMBOL_LOOKUP
 config ENFORCE_UNIQUE_SYMBOLS
 	bool "Enforce unique symbols"
 	default LIVEPATCH
-	---help---
+	help
 	  Multiple symbols with the same name aren't generally a problem
 	  unless livepatching is to be used.
 
@@ -454,7 +454,7 @@ config ENFORCE_UNIQUE_SYMBOLS
 config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
 	bool "Suppress duplicate symbol warnings"
 	depends on !ENFORCE_UNIQUE_SYMBOLS
-	---help---
+	help
 	  Multiple symbols with the same name aren't generally a problem
 	  unless Live patching is to be used, so these warnings can be
 	  suppressed by enabling this option.  Certain other options (known
@@ -464,7 +464,7 @@ config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
 config CMDLINE
 	string "Built-in hypervisor command string" if EXPERT
 	default ""
-	---help---
+	help
 	  Enter arguments here that should be compiled into the hypervisor
 	  image and used at boot time. When the system boots, this string
 	  will be parsed prior to the bootloader command line. So if a
@@ -475,7 +475,7 @@ config CMDLINE_OVERRIDE
 	bool "Built-in command line overrides bootloader arguments"
 	default n
 	depends on CMDLINE != ""
-	---help---
+	help
 	  Set this option to 'Y' to have the hypervisor ignore the bootloader
 	  command line, and use ONLY the built-in command line.
 
@@ -485,7 +485,7 @@ config CMDLINE_OVERRIDE
 config DOM0_MEM
 	string "Default value for dom0_mem boot parameter"
 	default ""
-	---help---
+	help
 	  Sets a default value for dom0_mem, e.g. "512M".
 	  The specified string will be used for the dom0_mem parameter in
 	  case it was not specified on the command line.
@@ -507,7 +507,7 @@ config DTB_FILE
 config TRACEBUFFER
 	bool "Enable tracing infrastructure" if EXPERT
 	default y
-	---help---
+	help
 	  Enable tracing infrastructure and pre-defined tracepoints within Xen.
 	  This will allow live information about Xen's execution and performance
 	  to be collected at run time for debugging or performance analysis.
diff --git a/xen/common/sched/Kconfig b/xen/common/sched/Kconfig
index b2ef0c99a3f8..18ca1ce7ab9f 100644
--- a/xen/common/sched/Kconfig
+++ b/xen/common/sched/Kconfig
@@ -4,20 +4,20 @@ menu "Schedulers"
 config SCHED_CREDIT
 	bool "Credit scheduler support"
 	default y
-	---help---
+	help
 	  The traditional credit scheduler is a general purpose scheduler.
 
 config SCHED_CREDIT2
 	bool "Credit2 scheduler support"
 	default y
-	---help---
+	help
 	  The credit2 scheduler is a general purpose scheduler that is
 	  optimized for lower latency and higher VM density.
 
 config SCHED_RTDS
 	bool "RTDS scheduler support (UNSUPPORTED)" if UNSUPPORTED
 	default DEBUG
-	---help---
+	help
 	  The RTDS scheduler is a soft and firm real-time scheduler for
 	  multicore, targeted for embedded, automotive, graphics and gaming
 	  in the cloud, and general low-latency workloads.
@@ -25,14 +25,14 @@ config SCHED_RTDS
 config SCHED_ARINC653
 	bool "ARINC653 scheduler support (UNSUPPORTED)" if UNSUPPORTED
 	default DEBUG
-	---help---
+	help
 	  The ARINC653 scheduler is a hard real-time scheduler for single
 	  cores, targeted for avionics, drones, and medical devices.
 
 config SCHED_NULL
 	bool "Null scheduler support (UNSUPPORTED)" if UNSUPPORTED
 	default PV_SHIM || DEBUG
-	---help---
+	help
 	  The null scheduler is a static, zero overhead scheduler,
 	  for when there always are less vCPUs than pCPUs, typically
 	  in embedded or HPC scenarios.
diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index 864fcf3b0cef..78edd805365e 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -6,7 +6,7 @@ if ARM
 config ARM_SMMU
 	bool "ARM SMMUv1 and v2 driver"
 	default y
-	---help---
+	help
 	  Support for implementations of the ARM System MMU architecture
 	  versions 1 and 2.
 
@@ -16,7 +16,7 @@ config ARM_SMMU
 config ARM_SMMU_V3
 	bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support" if EXPERT
 	depends on ARM_64 && (!ACPI || BROKEN)
-	---help---
+	help
 	 Support for implementations of the ARM System MMU architecture
 	 version 3. Driver is in experimental stage and should not be used in
 	 production.
@@ -27,7 +27,7 @@ config ARM_SMMU_V3
 config IPMMU_VMSA
 	bool "Renesas IPMMU-VMSA found in R-Car Gen3/Gen4 SoCs"
 	depends on ARM_64
-	---help---
+	help
 	  Support for implementations of the Renesas IPMMU-VMSA found
 	  in R-Car Gen3/Gen4 SoCs.
 
@@ -68,7 +68,7 @@ choice
 	prompt "IOMMU device quarantining default behavior"
 	depends on HAS_PCI
 	default IOMMU_QUARANTINE_BASIC
-	---help---
+	help
 	  When a PCI device is assigned to an untrusted domain, it is possible
 	  for that domain to program the device to DMA to an arbitrary address.
 	  The IOMMU is used to protect the host from malicious DMA by making
diff --git a/xen/drivers/video/Kconfig b/xen/drivers/video/Kconfig
index 41ca503cc961..245030beeaa2 100644
--- a/xen/drivers/video/Kconfig
+++ b/xen/drivers/video/Kconfig
@@ -7,7 +7,7 @@ config VGA
 	select VIDEO
 	depends on X86
 	default y if !PV_SHIM_EXCLUSIVE
-	---help---
+	help
 	  Enable VGA output for the Xen hypervisor.
 
 	  If unsure, say Y.
diff --git a/xen/tools/kconfig/lexer.l b/xen/tools/kconfig/lexer.l
index 6354c905b006..4b7339ff4c8b 100644
--- a/xen/tools/kconfig/lexer.l
+++ b/xen/tools/kconfig/lexer.l
@@ -105,7 +105,7 @@ n	[A-Za-z0-9_-]
 "endchoice"		return T_ENDCHOICE;
 "endif"			return T_ENDIF;
 "endmenu"		return T_ENDMENU;
-"help"|"---help---"	return T_HELP;
+"help"			return T_HELP;
 "hex"			return T_HEX;
 "if"			return T_IF;
 "imply"			return T_IMPLY;

base-commit: feb9158a620040846d76981acbe8ea9e2255a07b
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 02 18:11:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:11:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716109.1118191 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2atu-00070Y-Cz; Thu, 02 May 2024 18:11:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716109.1118191; Thu, 02 May 2024 18:11:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2atu-00070R-9e; Thu, 02 May 2024 18:11:22 +0000
Received: by outflank-mailman (input) for mailman id 716109;
 Thu, 02 May 2024 18:11:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2ats-0006Np-Vt
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:11:20 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5fbbf117-08af-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:11:20 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 6267F61BFB;
 Thu,  2 May 2024 18:11:18 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7333BC113CC;
 Thu,  2 May 2024 18:11:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5fbbf117-08af-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714673478;
	bh=XhYOxmZWkP66YztGtxmVj1HWIYgzpvO5VBxe2rJtMBc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=s4y7zGWDzJenlan5uDQq3mzmQ+ocer6Fpxr84npjHlzsY2E9VWt9aRWRYF+KdfHlF
	 yIMJehZ8Ge2C3mFrusdiLt0ioFlqWGGJD3k8rgPE+ua7cEUkGmEvxXJcyVSFglErF/
	 rpmE7poR0aT+A/08w8gQgqTOu1Y1vYy31xUDXUnESd0ea6Ns9/DxsIqz6laVurDNCK
	 4XJYfeF6P8UPolOAH94ULAkiCjrYNTJIPgv/p7+oIHOnNx/B1ui7Esf6DRYsDARKtm
	 Lr/hdVeGXISk4ADZhKawCdl6AlRWiuCsuxHKa6VWvcx609bL0R9Sk8nRzjMIQ6HtDv
	 dLyTsgHd8hktw==
Date: Thu, 2 May 2024 11:11:15 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v1.1] xen/commom/dt-overlay: Fix missing lock when remove
 the device
In-Reply-To: <20240426015550.577986-1-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405021111070.624854@ubuntu-linux-20-04-desktop>
References: <20240426015550.577986-1-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1925617102-1714673478=:624854"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1925617102-1714673478=:624854
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 26 Apr 2024, Henry Wang wrote:
> If CONFIG_DEBUG=y, below assertion will be triggered:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
> (XEN) CPU:    0
> (XEN) PC:     00000a0000257418 iommu_remove_dt_device+0x8c/0xd4
> (XEN) LR:     00000a00002573a0
> (XEN) SP:     00008000fff7fb30
> (XEN) CPSR:   0000000000000249 MODE:64-bit EL2h (Hypervisor, handler)
> [...]
> 
> (XEN) Xen call trace:
> (XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
> (XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
> (XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
> (XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
> (XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
> (XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
> (XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
> (XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
> (XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
> (XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ****************************************
> 
> This is because iommu_remove_dt_device() is called without taking the
> dt_host_lock. Fix the issue by taking and releasing the lock properly.
> 
> Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> v1.1:
> - Move the unlock position before the check of rc.
> ---
>  xen/common/dt-overlay.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
> index 1b197381f6..ab8f43aea2 100644
> --- a/xen/common/dt-overlay.c
> +++ b/xen/common/dt-overlay.c
> @@ -381,7 +381,9 @@ static int remove_node_resources(struct dt_device_node *device_node)
>      {
>          if ( dt_device_is_protected(device_node) )
>          {
> +            write_lock(&dt_host_lock);
>              rc = iommu_remove_dt_device(device_node);
> +            write_unlock(&dt_host_lock);
>              if ( rc < 0 )
>                  return rc;
>          }
> -- 
> 2.34.1
> 
--8323329-1925617102-1714673478=:624854--


From xen-devel-bounces@lists.xenproject.org Thu May 02 18:13:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:13:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716118.1118201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2avo-0007rp-Nl; Thu, 02 May 2024 18:13:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716118.1118201; Thu, 02 May 2024 18:13:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2avo-0007ri-Kp; Thu, 02 May 2024 18:13:20 +0000
Received: by outflank-mailman (input) for mailman id 716118;
 Thu, 02 May 2024 18:13:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2avn-0007ra-AY
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:13:19 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a50c34f3-08af-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:13:17 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 541C7CE16DB;
 Thu,  2 May 2024 18:13:14 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 039A8C113CC;
 Thu,  2 May 2024 18:13:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a50c34f3-08af-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714673593;
	bh=fiyeFheWqFGsN3qj2iTlgwJMs98wz0J8HkiJWqRbeWA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GPPSIyx810EdZRJytMCosF0vzRwA+P7eUuYaqjLuW/fJ85AJjD2lFb1trzoyW5osb
	 8Z+qhMC8wxTkqwn6HrZOmAfaZZ+5oobyfbXwEF4LcWZeOvU5/c2UuVn4PoU2BtyXp9
	 WcL8untZ2UUzmhXgDwXqHHrW2VXwCBhcau2oNTOUxmZbMzqEkVxi8OTuKMg1J8Y7aG
	 hX2jGeSWSij2QJMcOHwl5r08/ffvWKNVszcelwN7W2JFC4smgjr9Ulz6nCydBsKzWi
	 AeT2C+LIwYpf92gO8lmBofNhkIQeD0IhPZu9MiNQ16Cgr+SKutEOOktTp26VN4gOCP
	 d68sH/fh3m1rg==
Date: Thu, 2 May 2024 11:13:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: =?UTF-8?Q?J=C3=BCrgen_Gro=C3=9F?= <jgross@suse.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
In-Reply-To: <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
References: <20240327152229.25847-1-jgross@suse.com> <20240327152229.25847-9-jgross@suse.com> <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com> <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com> <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com> <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-2084856487-1714673594=:624854"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2084856487-1714673594=:624854
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 29 Apr 2024, Julien Grall wrote:
> Hi Juergen,
> 
> On 29/04/2024 12:28, Jürgen Groß wrote:
> > On 29.04.24 13:04, Julien Grall wrote:
> > > Hi Juergen,
> > > 
> > > Sorry for the late reply.
> > > 
> > > On 29/04/2024 11:33, Juergen Gross wrote:
> > > > On 08.04.24 09:10, Jan Beulich wrote:
> > > > > On 27.03.2024 16:22, Juergen Gross wrote:
> > > > > > With lock handling now allowing up to 16384 cpus (spinlocks can
> > > > > > handle
> > > > > > 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit
> > > > > > for
> > > > > > the number of cpus to be configured to 16383.
> > > > > > 
> > > > > > The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
> > > > > > QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
> > > > > > 
> > > > > > Signed-off-by: Juergen Gross <jgross@suse.com>
> > > > > 
> > > > > Acked-by: Jan Beulich <jbeulich@suse.com>
> > > > > 
> > > > > I'd prefer this to also gain an Arm ack, though.
> > > > 
> > > > Any comment from Arm side?
> > > 
> > > Can you clarify what the new limits mean in term of (security) support?
> > > Are we now claiming that Xen will work perfectly fine on platforms with up
> > > to 16383?
> > > 
> > > If so, I can't comment for x86, but for Arm, I am doubtful that it would
> > > work without any (at least performance) issues. AFAIK, this is also an
> > > untested configuration. In fact I would be surprised if Xen on Arm was
> > > tested with more than a couple of hundreds cores (AFAICT the Ampere CPUs
> > > has 192 CPUs).
> > 
> > I think we should add a security support limit for the number of physical
> > cpus similar to the memory support limit we already have in place.
> > 
> > For x86 I'd suggest 4096 cpus for security support (basically the limit we
> > have with this patch), but I'm open for other suggestions, too.
> > 
> > I have no idea about any sensible limits for Arm32/Arm64.
> 
> I am not entirely. Bertrand, Michal, Stefano, should we use 192 (the number of
> CPUs from Ampere)?

I am OK with that. If we want to be a bit more future proof we could say
256 or 512. I leave this to you, as all the boards I have have a much
smaller CPU count.
--8323329-2084856487-1714673594=:624854--


From xen-devel-bounces@lists.xenproject.org Thu May 02 18:16:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:16:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716126.1118210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2az5-0001wp-5J; Thu, 02 May 2024 18:16:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716126.1118210; Thu, 02 May 2024 18:16:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2az5-0001wi-2d; Thu, 02 May 2024 18:16:43 +0000
Received: by outflank-mailman (input) for mailman id 716126;
 Thu, 02 May 2024 18:16:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2az3-0001wc-Qg
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:16:41 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1e4736b4-08b0-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:16:39 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id E689C61C00;
 Thu,  2 May 2024 18:16:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20C3AC113CC;
 Thu,  2 May 2024 18:16:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e4736b4-08b0-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714673797;
	bh=rwNEAoQNXpcPPM83mK5yiJdPzVUoLdLwJ399JpicAp8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=T5nOwUxTKVqb4UFVkVHc1Xh82DB3ObUayoC68S0UWzD9N9CRSTDBZv6WjOXTZ9K1/
	 +eda8uqphVEPq7NE8CdYAomc6IlgfGbcLcZTTIkM5FrAbCTEvDDnJ6seI67mitB3DZ
	 PRPd0NZvwg6K4Y6B0IJd2GMtoBo+Z5pxOYQ75moKVzCJ43UJkRz+rrcZkOLEbQRxg/
	 K3Zk+JMUQpWzoMKqxLOYrEVyIebpTUdD86Mczp3BG3EsT004va3Op++qvh1Oo4lTKv
	 AeQwfkE0z7NOVb8surCs+I+QsJm55wWoL1eQylCbq1NPKxv4W/qJmX8m/pBCtPeMCG
	 kMsIl/XWr6dCA==
Date: Thu, 2 May 2024 11:16:32 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH] xen/Kconfig: Drop the final remnants of ---help---
In-Reply-To: <20240502181049.1361384-1-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405021116250.624854@ubuntu-linux-20-04-desktop>
References: <20240502181049.1361384-1-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 2 May 2024, Andrew Cooper wrote:
> We deprecated the use of ---help--- a while ago, but a lot of new content
> copy&pastes bad examples.  Convert the remaining instances, and update
> Kconfig's parser to no longer recongise it.
> 
> This now causes builds to fail with:
> 
>   Kconfig.debug:8: syntax error
>   Kconfig.debug:7: unknown statement "---help---"
> 
> which short circuits one common piece of churn in new content.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> For 4.19.  This cleans up a legacy we've been wanting to get rid of for a
> while, and will be least disruptive on people if it gets in ahead of most
> people starting work for 4.20.
> ---
>  xen/Kconfig                     |  2 +-
>  xen/Kconfig.debug               | 28 +++++++++----------
>  xen/arch/arm/Kconfig            |  8 +++---
>  xen/arch/arm/platforms/Kconfig  | 12 ++++-----
>  xen/arch/x86/Kconfig            | 32 +++++++++++-----------
>  xen/common/Kconfig              | 48 ++++++++++++++++-----------------
>  xen/common/sched/Kconfig        | 10 +++----
>  xen/drivers/passthrough/Kconfig |  8 +++---
>  xen/drivers/video/Kconfig       |  2 +-
>  xen/tools/kconfig/lexer.l       |  2 +-
>  10 files changed, 76 insertions(+), 76 deletions(-)
> 
> diff --git a/xen/Kconfig b/xen/Kconfig
> index 1e1b041fd52f..e459cdac0cd7 100644
> --- a/xen/Kconfig
> +++ b/xen/Kconfig
> @@ -84,7 +84,7 @@ config UNSUPPORTED
>  config LTO
>  	bool "Link Time Optimisation"
>  	depends on BROKEN
> -	---help---
> +	help
>  	  Enable Link Time Optimisation.
>  
>  	  If unsure, say N.
> diff --git a/xen/Kconfig.debug b/xen/Kconfig.debug
> index fa81853e9385..61b24ac552cd 100644
> --- a/xen/Kconfig.debug
> +++ b/xen/Kconfig.debug
> @@ -4,7 +4,7 @@ menu "Debugging Options"
>  config DEBUG
>  	bool "Developer Checks"
>  	default y
> -	---help---
> +	help
>  	  If you say Y here this will enable developer checks such as asserts
>  	  and extra printks. This option is intended for development purposes
>  	  only, and not for production use.
> @@ -17,14 +17,14 @@ config GDBSX
>  	bool "Guest debugging with gdbsx"
>  	depends on X86
>  	default y
> -	---help---
> +	help
>  	  If you want to enable support for debugging guests from dom0 via
>  	  gdbsx then say Y.
>  
>  config FRAME_POINTER
>  	bool "Compile Xen with frame pointers"
>  	default DEBUG
> -	---help---
> +	help
>  	  If you say Y here the resulting Xen will be slightly larger and
>  	  maybe slower, but it gives very useful debugging information
>  	  in case of any Xen bugs.
> @@ -33,7 +33,7 @@ config COVERAGE
>  	bool "Code coverage support"
>  	depends on !LIVEPATCH
>  	select SUPPRESS_DUPLICATE_SYMBOL_WARNINGS if !ENFORCE_UNIQUE_SYMBOLS
> -	---help---
> +	help
>  	  Enable code coverage support.
>  
>  	  If unsure, say N here.
> @@ -41,7 +41,7 @@ config COVERAGE
>  config DEBUG_LOCK_PROFILE
>  	bool "Lock Profiling"
>  	select DEBUG_LOCKS
> -	---help---
> +	help
>  	  Lock profiling allows you to see how often locks are taken and blocked.
>  	  You can use serial console to print (and reset) using 'l' and 'L'
>  	  respectively, or the 'xenlockprof' tool.
> @@ -49,13 +49,13 @@ config DEBUG_LOCK_PROFILE
>  config DEBUG_LOCKS
>  	bool "Lock debugging"
>  	default DEBUG
> -	---help---
> +	help
>  	  Enable debugging features of lock handling.  Some additional
>  	  checks will be performed when acquiring and releasing locks.
>  
>  config PERF_COUNTERS
>  	bool "Performance Counters"
> -	---help---
> +	help
>  	  Enables software performance counters that allows you to analyze
>  	  bottlenecks in the system.  To access this data you can use serial
>  	  console to print (and reset) using 'p' and 'P' respectively, or
> @@ -64,21 +64,21 @@ config PERF_COUNTERS
>  config PERF_ARRAYS
>  	bool "Performance Counter Array Histograms"
>  	depends on PERF_COUNTERS
> -	---help---
> +	help
>  	  Enables software performance counter array histograms.
>  
>  
>  config VERBOSE_DEBUG
>  	bool "Verbose debug messages"
>  	default DEBUG
> -	---help---
> +	help
>  	  Guest output from HYPERVISOR_console_io and hypervisor parsing
>  	  ELF images (dom0) will be logged in the Xen ring buffer.
>  
>  config DEVICE_TREE_DEBUG
>  	bool "Device tree debug messages"
>  	depends on HAS_DEVICE_TREE
> -	---help---
> +	help
>  	  Device tree parsing and DOM0 device tree building messages are
>  	  logged in the Xen ring buffer.
>  	  If unsure, say N here.
> @@ -86,14 +86,14 @@ config DEVICE_TREE_DEBUG
>  config SCRUB_DEBUG
>  	bool "Page scrubbing test"
>  	default DEBUG
> -	---help---
> +	help
>  	  Verify that pages that need to be scrubbed before being allocated to
>  	  a guest are indeed scrubbed.
>  
>  config UBSAN
>  	bool "Undefined behaviour sanitizer"
>  	depends on HAS_UBSAN
> -	---help---
> +	help
>  	  Enable undefined behaviour sanitizer. It uses compiler to insert code
>  	  snippets so that undefined behaviours in C are detected during runtime.
>  	  This requires a UBSAN capable compiler and it is a debug only feature.
> @@ -109,7 +109,7 @@ config UBSAN_FATAL
>  
>  config DEBUG_TRACE
>  	bool "Debug trace support"
> -	---help---
> +	help
>  	  Debug trace enables to record debug trace messages which are printed
>  	  either directly to the console or are printed to console in case of
>  	  a system crash.
> @@ -117,7 +117,7 @@ config DEBUG_TRACE
>  config XMEM_POOL_POISON
>  	bool "Poison free xenpool blocks"
>  	default DEBUG
> -	---help---
> +	help
>  	  Poison free blocks with 0xAA bytes and verify them when a block is
>  	  allocated in order to spot use-after-free issues.
>  
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index f8139a773a43..21d03d9f4424 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -67,7 +67,7 @@ source "arch/Kconfig"
>  config ACPI
>  	bool "ACPI (Advanced Configuration and Power Interface) Support (UNSUPPORTED)" if UNSUPPORTED
>  	depends on ARM_64 && ARM_EFI
> -	---help---
> +	help
>  
>  	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>  	  an alternative to device tree on ARM64. This requires UEFI.
> @@ -102,7 +102,7 @@ config GICV3
>  	depends on !NEW_VGIC
>  	default n if ARM_32
>  	default y if ARM_64
> -	---help---
> +	help
>  
>  	  Driver for the ARM Generic Interrupt Controller v3.
>  	  If unsure, use the default setting.
> @@ -132,7 +132,7 @@ config HVM
>  config NEW_VGIC
>  	bool "Use new VGIC implementation"
>  	select GICV2
> -	---help---
> +	help
>  
>  	This is an alternative implementation of the ARM GIC interrupt
>  	controller emulation, based on the Linux/KVM VGIC. It has a better
> @@ -147,7 +147,7 @@ config NEW_VGIC
>  config SBSA_VUART_CONSOLE
>  	bool "Emulated SBSA UART console support"
>  	default y
> -	---help---
> +	help
>  	  Allows a guest to use SBSA Generic UART as a console. The
>  	  SBSA Generic UART implements a subset of ARM PL011 UART.
>  
> diff --git a/xen/arch/arm/platforms/Kconfig b/xen/arch/arm/platforms/Kconfig
> index c93a6b275620..76f7e76b1bf4 100644
> --- a/xen/arch/arm/platforms/Kconfig
> +++ b/xen/arch/arm/platforms/Kconfig
> @@ -1,14 +1,14 @@
>  choice
>  	prompt "Platform Support"
>  	default ALL_PLAT
> -	---help---
> +	help
>  	Choose which hardware platform to enable in Xen.
>  
>  	If unsure, choose ALL_PLAT.
>  
>  config ALL_PLAT
>  	bool "All Platforms"
> -	---help---
> +	help
>  	Enable support for all available hardware platforms. It doesn't
>  	automatically select any of the related drivers.
>  
> @@ -17,7 +17,7 @@ config QEMU
>  	depends on ARM_64
>  	select GICV3
>  	select HAS_PL011
> -	---help---
> +	help
>  	Enable all the required drivers for QEMU aarch64 virt emulated
>  	machine.
>  
> @@ -26,7 +26,7 @@ config RCAR3
>  	depends on ARM_64
>  	select HAS_SCIF
>  	select IPMMU_VMSA
> -	---help---
> +	help
>  	Enable all the required drivers for Renesas RCar3
>  
>  config MPSOC
> @@ -34,12 +34,12 @@ config MPSOC
>  	depends on ARM_64
>  	select HAS_CADENCE_UART
>  	select ARM_SMMU
> -	---help---
> +	help
>  	Enable all the required drivers for Xilinx Ultrascale+ MPSoC
>  
>  config NO_PLAT
>  	bool "No Platforms"
> -	---help---
> +	help
>  	Do not enable specific support for any platform.
>  
>  endchoice
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 2b6248774d8f..7e03e4bc5546 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -58,7 +58,7 @@ source "arch/Kconfig"
>  config PV
>  	def_bool y
>  	prompt "PV support"
> -	---help---
> +	help
>  	  Interfaces to support PV domains. These require guest kernel support
>  	  to run as a PV guest, but don't require any specific hardware support.
>  
> @@ -71,7 +71,7 @@ config PV32
>  	depends on PV
>  	default PV_SHIM
>  	select COMPAT
> -	---help---
> +	help
>  	  The 32bit PV ABI uses Ring1, an area of the x86 architecture which
>  	  was deprecated and mostly removed in the AMD64 spec.  As a result,
>  	  it occasionally conflicts with newer x86 hardware features, causing
> @@ -90,7 +90,7 @@ config PV_LINEAR_PT
>         bool "Support for PV linear pagetables"
>         depends on PV
>         default y
> -       ---help---
> +       help
>           Linear pagetables (also called "recursive pagetables") refers
>           to the practice of a guest operating system having pagetable
>           entries pointing to other pagetables of the same level (i.e.,
> @@ -113,7 +113,7 @@ config HVM
>  	select COMPAT
>  	select IOREQ_SERVER
>  	select MEM_ACCESS_ALWAYS_ON
> -	---help---
> +	help
>  	  Interfaces to support HVM domains.  HVM domains require hardware
>  	  virtualisation extensions (e.g. Intel VT-x, AMD SVM), but can boot
>  	  guests which have no specific Xen knowledge.
> @@ -126,7 +126,7 @@ config XEN_SHSTK
>  	bool "Supervisor Shadow Stacks"
>  	depends on HAS_AS_CET_SS
>  	default y
> -	---help---
> +	help
>  	  Control-flow Enforcement Technology (CET) is a set of features in
>  	  hardware designed to combat Return-oriented Programming (ROP, also
>  	  call/jump COP/JOP) attacks.  Shadow Stacks are one CET feature
> @@ -152,7 +152,7 @@ config SHADOW_PAGING
>  	bool "Shadow Paging"
>  	default !PV_SHIM_EXCLUSIVE
>  	depends on PV || HVM
> -	---help---
> +	help
>  
>            Shadow paging is a software alternative to hardware paging support
>            (Intel EPT, AMD NPT).
> @@ -171,7 +171,7 @@ config SHADOW_PAGING
>  config BIGMEM
>  	bool "big memory support"
>  	default n
> -	---help---
> +	help
>  	  Allows Xen to support up to 123Tb of memory.
>  
>  	  This requires enlarging struct page_info as well as shrinking
> @@ -183,7 +183,7 @@ config HVM_FEP
>  	bool "HVM Forced Emulation Prefix support (UNSUPPORTED)" if UNSUPPORTED
>  	default DEBUG
>  	depends on HVM
> -	---help---
> +	help
>  
>  	  Compiles in a feature that allows HVM guest to arbitrarily
>  	  exercise the instruction emulator.
> @@ -203,7 +203,7 @@ config TBOOT
>  	depends on UNSUPPORTED
>  	default !PV_SHIM_EXCLUSIVE
>  	select CRYPTO
> -	---help---
> +	help
>  	  Allows support for Trusted Boot using the Intel(R) Trusted Execution
>  	  Technology (TXT)
>  
> @@ -213,14 +213,14 @@ choice
>  	prompt "Alignment of Xen image"
>  	default XEN_ALIGN_2M if PV_SHIM_EXCLUSIVE
>  	default XEN_ALIGN_DEFAULT
> -	---help---
> +	help
>  	  Specify alignment for Xen image.
>  
>  	  If unsure, choose "default".
>  
>  config XEN_ALIGN_DEFAULT
>  	bool "Default alignment"
> -	---help---
> +	help
>  	  Pick alignment according to build variants.
>  
>  	  For EFI build the default alignment is 2M. For ELF build
> @@ -281,7 +281,7 @@ config GUEST
>  config XEN_GUEST
>  	bool "Xen Guest"
>  	select GUEST
> -	---help---
> +	help
>  	  Support for Xen detecting when it is running under Xen.
>  
>  	  If unsure, say N.
> @@ -290,7 +290,7 @@ config PVH_GUEST
>  	def_bool y
>  	prompt "PVH Guest"
>  	depends on XEN_GUEST
> -	---help---
> +	help
>  	  Support booting using the PVH ABI.
>  
>  	  If unsure, say Y.
> @@ -299,7 +299,7 @@ config PV_SHIM
>  	def_bool y
>  	prompt "PV Shim"
>  	depends on PV && XEN_GUEST
> -	---help---
> +	help
>  	  Build Xen with a mode which acts as a shim to allow PV guest to run
>  	  in an HVM/PVH container. This mode can only be enabled with command
>  	  line option.
> @@ -309,7 +309,7 @@ config PV_SHIM
>  config PV_SHIM_EXCLUSIVE
>  	bool "PV Shim Exclusive"
>  	depends on PV_SHIM
> -	---help---
> +	help
>  	  Build Xen in a way which unconditionally assumes PV_SHIM mode.  This
>  	  option is only intended for use when building a dedicated PV Shim
>  	  firmware, and will not function correctly in other scenarios.
> @@ -321,7 +321,7 @@ if !PV_SHIM_EXCLUSIVE
>  config HYPERV_GUEST
>  	bool "Hyper-V Guest"
>  	select GUEST
> -	---help---
> +	help
>  	  Support for Xen detecting when it is running under Hyper-V.
>  
>  	  If unsure, say N.
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index cff3166ff923..565ceda741b9 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -15,7 +15,7 @@ config CORE_PARKING
>  config GRANT_TABLE
>  	bool "Grant table support" if EXPERT
>  	default y
> -	---help---
> +	help
>  	  Grant table provides a generic mechanism to memory sharing
>  	  between domains. This shared memory interface underpins the
>  	  split device drivers for block and network IO in a classic
> @@ -90,7 +90,7 @@ config MEM_ACCESS
>  	def_bool MEM_ACCESS_ALWAYS_ON
>  	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
>  	depends on HVM
> -	---help---
> +	help
>  
>  	  Framework to configure memory access types for guests and receive
>  	  related events in userspace.
> @@ -133,7 +133,7 @@ config INDIRECT_THUNK
>  config SPECULATIVE_HARDEN_ARRAY
>  	bool "Speculative Array Hardening"
>  	default y
> -	---help---
> +	help
>  	  Contemporary processors may use speculative execution as a
>  	  performance optimisation, but this can potentially be abused by an
>  	  attacker to leak data via speculative sidechannels.
> @@ -154,7 +154,7 @@ config SPECULATIVE_HARDEN_BRANCH
>  	bool "Speculative Branch Hardening"
>  	default y
>  	depends on X86
> -        ---help---
> +        help
>  	  Contemporary processors may use speculative execution as a
>  	  performance optimisation, but this can potentially be abused by an
>  	  attacker to leak data via speculative sidechannels.
> @@ -228,7 +228,7 @@ config DIT_DEFAULT
>  config HYPFS
>  	bool "Hypervisor file system support"
>  	default y
> -	---help---
> +	help
>  	  Support Xen hypervisor file system. This file system is used to
>  	  present various hypervisor internal data to dom0 and in some
>  	  cases to allow modifying settings. Disabling the support will
> @@ -241,7 +241,7 @@ config HYPFS_CONFIG
>  	bool "Provide hypervisor .config via hypfs entry"
>  	default y
>  	depends on HYPFS
> -	---help---
> +	help
>  	  When enabled the contents of the .config file used to build the
>  	  hypervisor are provided via the hypfs entry /buildinfo/config.
>  
> @@ -252,7 +252,7 @@ config IOREQ_SERVER
>  	bool "IOREQ support (EXPERT)" if EXPERT && !X86
>  	default X86
>  	depends on HVM
> -	---help---
> +	help
>  	  Enables generic mechanism for providing emulated devices to the guests.
>  
>  	  If unsure, say N.
> @@ -261,7 +261,7 @@ config KEXEC
>  	bool "kexec support"
>  	default y
>  	depends on HAS_KEXEC
> -	---help---
> +	help
>  	  Allows a running Xen hypervisor to be replaced with another OS
>  	  without rebooting. This is primarily used to execute a crash
>  	  environment to collect information on a Xen hypervisor or dom0 crash.
> @@ -270,7 +270,7 @@ config KEXEC
>  
>  config EFI_SET_VIRTUAL_ADDRESS_MAP
>      bool "EFI: call SetVirtualAddressMap()" if EXPERT
> -    ---help---
> +    help
>        Call EFI SetVirtualAddressMap() runtime service to setup memory map for
>        further runtime services. According to UEFI spec, it isn't strictly
>        necessary, but many UEFI implementations misbehave when this call is
> @@ -292,7 +292,7 @@ config XENOPROF
>  config XSM
>  	bool "Xen Security Modules support"
>  	default ARM
> -	---help---
> +	help
>  	  Enables the security framework known as Xen Security Modules which
>  	  allows administrators fine-grained control over a Xen domain and
>  	  its capabilities by defining permissible interactions between domains,
> @@ -305,7 +305,7 @@ config XSM_FLASK
>  	def_bool y
>  	prompt "FLux Advanced Security Kernel support"
>  	depends on XSM
> -	---help---
> +	help
>  	  Enables FLASK (FLux Advanced Security Kernel) as the access control
>  	  mechanism used by the XSM framework.  This provides a mandatory access
>  	  control framework by which security enforcement, isolation, and
> @@ -318,7 +318,7 @@ config XSM_FLASK_AVC_STATS
>  	def_bool y
>  	prompt "Maintain statistics on the FLASK access vector cache" if EXPERT
>  	depends on XSM_FLASK
> -	---help---
> +	help
>  	  Maintain counters on the access vector cache that can be viewed using
>  	  the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.  Disabling
>  	  this will save a tiny amount of memory and time to update the stats.
> @@ -329,7 +329,7 @@ config XSM_FLASK_POLICY
>  	bool "Compile Xen with a built-in FLASK security policy"
>  	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
>  	depends on XSM_FLASK
> -	---help---
> +	help
>  	  This includes a default XSM policy in the hypervisor so that the
>  	  bootloader does not need to load a policy to get sane behavior from an
>  	  XSM-enabled hypervisor.  If this is disabled, a policy must be
> @@ -345,7 +345,7 @@ config XSM_SILO
>  	def_bool y
>  	prompt "SILO support"
>  	depends on XSM
> -	---help---
> +	help
>  	  Enables SILO as the access control mechanism used by the XSM framework.
>  	  This is not the default module, add boot parameter xsm=silo to choose
>  	  it. This will deny any unmediated communication channels (grant tables
> @@ -372,7 +372,7 @@ config LATE_HWDOM
>  	bool "Dedicated hardware domain"
>  	default n
>  	depends on XSM && X86
> -	---help---
> +	help
>  	  Allows the creation of a dedicated hardware domain distinct from
>  	  domain 0 that manages devices without needing access to other
>  	  privileged functionality such as the ability to manage domains.
> @@ -390,7 +390,7 @@ config LATE_HWDOM
>  
>  config ARGO
>  	bool "Argo: hypervisor-mediated interdomain communication (UNSUPPORTED)" if UNSUPPORTED
> -	---help---
> +	help
>  	  Enables a hypercall for domains to ask the hypervisor to perform
>  	  data transfer of messages between domains.
>  
> @@ -417,7 +417,7 @@ config LIVEPATCH
>  	default X86
>  	depends on "$(XEN_HAS_BUILD_ID)" = "y"
>  	select CC_SPLIT_SECTIONS
> -	---help---
> +	help
>  	  Allows a running Xen hypervisor to be dynamically patched using
>  	  binary patches without rebooting. This is primarily used to binarily
>  	  patch in the field an hypervisor with XSA fixes.
> @@ -428,7 +428,7 @@ config FAST_SYMBOL_LOOKUP
>  	bool "Fast symbol lookup (bigger binary)"
>  	default y
>  	depends on LIVEPATCH
> -	---help---
> +	help
>  	  When searching for symbol addresses we can use the built-in system
>  	  that is optimized for searching symbols using addresses as the key.
>  	  However using it for the inverse (find address using the symbol name)
> @@ -440,7 +440,7 @@ config FAST_SYMBOL_LOOKUP
>  config ENFORCE_UNIQUE_SYMBOLS
>  	bool "Enforce unique symbols"
>  	default LIVEPATCH
> -	---help---
> +	help
>  	  Multiple symbols with the same name aren't generally a problem
>  	  unless livepatching is to be used.
>  
> @@ -454,7 +454,7 @@ config ENFORCE_UNIQUE_SYMBOLS
>  config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
>  	bool "Suppress duplicate symbol warnings"
>  	depends on !ENFORCE_UNIQUE_SYMBOLS
> -	---help---
> +	help
>  	  Multiple symbols with the same name aren't generally a problem
>  	  unless Live patching is to be used, so these warnings can be
>  	  suppressed by enabling this option.  Certain other options (known
> @@ -464,7 +464,7 @@ config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
>  config CMDLINE
>  	string "Built-in hypervisor command string" if EXPERT
>  	default ""
> -	---help---
> +	help
>  	  Enter arguments here that should be compiled into the hypervisor
>  	  image and used at boot time. When the system boots, this string
>  	  will be parsed prior to the bootloader command line. So if a
> @@ -475,7 +475,7 @@ config CMDLINE_OVERRIDE
>  	bool "Built-in command line overrides bootloader arguments"
>  	default n
>  	depends on CMDLINE != ""
> -	---help---
> +	help
>  	  Set this option to 'Y' to have the hypervisor ignore the bootloader
>  	  command line, and use ONLY the built-in command line.
>  
> @@ -485,7 +485,7 @@ config CMDLINE_OVERRIDE
>  config DOM0_MEM
>  	string "Default value for dom0_mem boot parameter"
>  	default ""
> -	---help---
> +	help
>  	  Sets a default value for dom0_mem, e.g. "512M".
>  	  The specified string will be used for the dom0_mem parameter in
>  	  case it was not specified on the command line.
> @@ -507,7 +507,7 @@ config DTB_FILE
>  config TRACEBUFFER
>  	bool "Enable tracing infrastructure" if EXPERT
>  	default y
> -	---help---
> +	help
>  	  Enable tracing infrastructure and pre-defined tracepoints within Xen.
>  	  This will allow live information about Xen's execution and performance
>  	  to be collected at run time for debugging or performance analysis.
> diff --git a/xen/common/sched/Kconfig b/xen/common/sched/Kconfig
> index b2ef0c99a3f8..18ca1ce7ab9f 100644
> --- a/xen/common/sched/Kconfig
> +++ b/xen/common/sched/Kconfig
> @@ -4,20 +4,20 @@ menu "Schedulers"
>  config SCHED_CREDIT
>  	bool "Credit scheduler support"
>  	default y
> -	---help---
> +	help
>  	  The traditional credit scheduler is a general purpose scheduler.
>  
>  config SCHED_CREDIT2
>  	bool "Credit2 scheduler support"
>  	default y
> -	---help---
> +	help
>  	  The credit2 scheduler is a general purpose scheduler that is
>  	  optimized for lower latency and higher VM density.
>  
>  config SCHED_RTDS
>  	bool "RTDS scheduler support (UNSUPPORTED)" if UNSUPPORTED
>  	default DEBUG
> -	---help---
> +	help
>  	  The RTDS scheduler is a soft and firm real-time scheduler for
>  	  multicore, targeted for embedded, automotive, graphics and gaming
>  	  in the cloud, and general low-latency workloads.
> @@ -25,14 +25,14 @@ config SCHED_RTDS
>  config SCHED_ARINC653
>  	bool "ARINC653 scheduler support (UNSUPPORTED)" if UNSUPPORTED
>  	default DEBUG
> -	---help---
> +	help
>  	  The ARINC653 scheduler is a hard real-time scheduler for single
>  	  cores, targeted for avionics, drones, and medical devices.
>  
>  config SCHED_NULL
>  	bool "Null scheduler support (UNSUPPORTED)" if UNSUPPORTED
>  	default PV_SHIM || DEBUG
> -	---help---
> +	help
>  	  The null scheduler is a static, zero overhead scheduler,
>  	  for when there always are less vCPUs than pCPUs, typically
>  	  in embedded or HPC scenarios.
> diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
> index 864fcf3b0cef..78edd805365e 100644
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -6,7 +6,7 @@ if ARM
>  config ARM_SMMU
>  	bool "ARM SMMUv1 and v2 driver"
>  	default y
> -	---help---
> +	help
>  	  Support for implementations of the ARM System MMU architecture
>  	  versions 1 and 2.
>  
> @@ -16,7 +16,7 @@ config ARM_SMMU
>  config ARM_SMMU_V3
>  	bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support" if EXPERT
>  	depends on ARM_64 && (!ACPI || BROKEN)
> -	---help---
> +	help
>  	 Support for implementations of the ARM System MMU architecture
>  	 version 3. Driver is in experimental stage and should not be used in
>  	 production.
> @@ -27,7 +27,7 @@ config ARM_SMMU_V3
>  config IPMMU_VMSA
>  	bool "Renesas IPMMU-VMSA found in R-Car Gen3/Gen4 SoCs"
>  	depends on ARM_64
> -	---help---
> +	help
>  	  Support for implementations of the Renesas IPMMU-VMSA found
>  	  in R-Car Gen3/Gen4 SoCs.
>  
> @@ -68,7 +68,7 @@ choice
>  	prompt "IOMMU device quarantining default behavior"
>  	depends on HAS_PCI
>  	default IOMMU_QUARANTINE_BASIC
> -	---help---
> +	help
>  	  When a PCI device is assigned to an untrusted domain, it is possible
>  	  for that domain to program the device to DMA to an arbitrary address.
>  	  The IOMMU is used to protect the host from malicious DMA by making
> diff --git a/xen/drivers/video/Kconfig b/xen/drivers/video/Kconfig
> index 41ca503cc961..245030beeaa2 100644
> --- a/xen/drivers/video/Kconfig
> +++ b/xen/drivers/video/Kconfig
> @@ -7,7 +7,7 @@ config VGA
>  	select VIDEO
>  	depends on X86
>  	default y if !PV_SHIM_EXCLUSIVE
> -	---help---
> +	help
>  	  Enable VGA output for the Xen hypervisor.
>  
>  	  If unsure, say Y.
> diff --git a/xen/tools/kconfig/lexer.l b/xen/tools/kconfig/lexer.l
> index 6354c905b006..4b7339ff4c8b 100644
> --- a/xen/tools/kconfig/lexer.l
> +++ b/xen/tools/kconfig/lexer.l
> @@ -105,7 +105,7 @@ n	[A-Za-z0-9_-]
>  "endchoice"		return T_ENDCHOICE;
>  "endif"			return T_ENDIF;
>  "endmenu"		return T_ENDMENU;
> -"help"|"---help---"	return T_HELP;
> +"help"			return T_HELP;
>  "hex"			return T_HEX;
>  "if"			return T_IF;
>  "imply"			return T_IMPLY;
> 
> base-commit: feb9158a620040846d76981acbe8ea9e2255a07b
> -- 
> 2.30.2
> 


From xen-devel-bounces@lists.xenproject.org Thu May 02 18:19:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:19:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716134.1118221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2b1R-0003QD-Lf; Thu, 02 May 2024 18:19:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716134.1118221; Thu, 02 May 2024 18:19:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2b1R-0003Q6-IS; Thu, 02 May 2024 18:19:09 +0000
Received: by outflank-mailman (input) for mailman id 716134;
 Thu, 02 May 2024 18:19:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2b1P-0003Pw-EH
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:19:07 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 75e444b5-08b0-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:19:06 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 6328A61BDC;
 Thu,  2 May 2024 18:19:05 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69EF0C113CC;
 Thu,  2 May 2024 18:19:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75e444b5-08b0-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714673945;
	bh=I9dmzi4oI4UvbL5usXmBvXgN8cMVznKW1QDxn6+HsW4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kmKt46/w6AGDVZPept6ExXpUIn++D9uGM+6RspNV40UR5n1lR28HDW3C5kD+aVDk/
	 bidIYANYjiorxfjgG1xy+r0AWPKi7dnDww/ESzUa8zpqgrTrsrbuOR/1Ps9KszvTo/
	 oLZn7nNC3ZPhAisQDWZEBPISo/9eexD9YASv4X4RgDE0cjb0AKFvfTdWodtpejRygp
	 pxisq7D1mfNOuKZpHxcv69tSt13AAtLjqrjCwr8PJ0TTLQ3ELDaiEogOp772nAt1rU
	 8j7PEaIZR0jrq25tNh74rfHH1XcZD+XghyU7Sdaz189m1oR0yjl1sy5iGoIrc0+LNr
	 zgAFF9yN0iZRg==
Date: Thu, 2 May 2024 11:19:02 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>
Subject: Re: [XEN PATCH v2] automation/eclair: add deviation of MISRA C:2012
 Rule 14.4
In-Reply-To: <ce16233b539f4101d97b3e2962607c87b188c415.1714655082.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405021118550.624854@ubuntu-linux-20-04-desktop>
References: <ce16233b539f4101d97b3e2962607c87b188c415.1714655082.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 2 May 2024, Federico Serafini wrote:
> Update ECLAIR configuration to take into account the deviations
> agreed during MISRA meetings.
> 
> Amend an existing entry of Rule 14.4 in deviations-rst:
> it is not a project-wide deviation.
> 
> Tag Rule 14.4 as clean for arm.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 02 18:23:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:23:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716139.1118231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2b5h-0005I2-5Y; Thu, 02 May 2024 18:23:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716139.1118231; Thu, 02 May 2024 18:23:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2b5h-0005Hv-2M; Thu, 02 May 2024 18:23:33 +0000
Received: by outflank-mailman (input) for mailman id 716139;
 Thu, 02 May 2024 18:23:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2b5g-0005Hp-Ab
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:23:32 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13330339-08b1-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 20:23:30 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 0A1EF61BFB;
 Thu,  2 May 2024 18:23:29 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C0435C113CC;
 Thu,  2 May 2024 18:23:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13330339-08b1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714674208;
	bh=XLZHrc7PmFH8DYvNL8xM8HEj1o0d+jV/2GKWhhxap2Y=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=m4LAy+GZyqkDCvfr1nexdW320RQUoY3hdjFuqTUY9YUXcZ/nevXLIieVR6h8R40U0
	 +4Jlo8dJnQJpWGyc7YnFc4tUM6V5jfx6Nsw3d8T9FH5geB/pxW/xzfAMqy1m3JtX1L
	 lg7Pc6+Et8uRUCeTJltQW/XM6G3NAAr0EixMLcS3mMddailYdreW8Vn12JTo51Y10L
	 oqNaBnT3zx3J1HnU5HucwklrQuiRbXBUHz3OrLA1hAx74nW+z12Z6VAIcmleNJvncX
	 dquABQ/w4CpOnvy/UCOCjYj/aviU1lfQcgeVs7Q2LY1830Hj3OGBiznTb5o7lI2f2D
	 eF+McxIfcYHnQ==
Date: Thu, 2 May 2024 11:23:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    nicola.vetrini@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 1/2] xen/kernel.h: Import __struct_group from Linux
In-Reply-To: <20240430110922.15052-2-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.22.394.2405021123160.624854@ubuntu-linux-20-04-desktop>
References: <20240430110922.15052-1-luca.fancellu@arm.com> <20240430110922.15052-2-luca.fancellu@arm.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Luca Fancellu wrote:
> Import __struct_group from Linux, commit 50d7bd38c3aa
> ("stddef: Introduce struct_group() helper macro"), in order to
> allow the access through the anonymous structure to the members
> without having to write also the name, e.g:
> 
> struct foo {
>     int one;
>     struct {
>         int two;
>         int three, four;
>     } thing;
>     int five;
> };
> 
> would become:
> 
> struct foo {
>     int one;
>     __struct_group(/* None */, thing, /* None */,
>         int two;
>         int three, four;
>     );
>     int five;
> };
> 
> Allowing the users of this structure to access the .thing members by
> using .two/.three/.four on the struct foo.
> This construct will become useful in order to have some generalized
> interfaces that shares some common members.
> 
> Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 50d7bd38c3aa
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 02 18:35:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:35:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716149.1118240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bHE-0001Jp-4w; Thu, 02 May 2024 18:35:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716149.1118240; Thu, 02 May 2024 18:35:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bHE-0001Ji-2M; Thu, 02 May 2024 18:35:28 +0000
Received: by outflank-mailman (input) for mailman id 716149;
 Thu, 02 May 2024 18:35:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2bHC-0001Jc-Nb
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:35:26 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bcfe59ee-08b2-11ef-909b-e314d9c70b13;
 Thu, 02 May 2024 20:35:24 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5D5F761A32;
 Thu,  2 May 2024 18:35:23 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 159C7C113CC;
 Thu,  2 May 2024 18:35:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcfe59ee-08b2-11ef-909b-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714674923;
	bh=0yDd7DiaX/tc3dq2DQMO4BYmIbyavMLPVq7RLiQanFs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mtEL4HcIdsRFFRbpxki+sFk97j7xLKYlZiw4mneGu0MLEHyfxnJvJ0XYjoU2Q2r10
	 /DaE81p4pMMzXAscBouSyD19qZr8maRJCkQ4/p0dj88ve7qE6VGtwx7EyzXKv78teV
	 i3CJa3Uf6aP/IMHscPA/gaRxXFZGmEw8iqMkaeCTqicus8y2lkzyX4zR3uH0QLeJhB
	 Oxo8QIsESFc8AXJGNV8Og2NlcOK7CMMfJ12QhwJiBdsz0vpuVQXmoymwcRTU5x3l+E
	 2C9+DPjKKLXVxsse6ki2SSdUD3m0sV5G2BnseL5zAkefDXfmrxVkO8W7YPAZhydUiO
	 /6RzEF9IQXTFQ==
Date: Thu, 2 May 2024 11:35:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    nicola.vetrini@bugseng.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
In-Reply-To: <20240430110922.15052-3-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.22.394.2405021135060.624854@ubuntu-linux-20-04-desktop>
References: <20240430110922.15052-1-luca.fancellu@arm.com> <20240430110922.15052-3-luca.fancellu@arm.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Luca Fancellu wrote:
> Commit 2209c1e35b47 ("xen/arm: Introduce a generic way to access memory
> bank structures") introduced a MISRA regression for Rule 1.1 because a
> flexible array member is introduced in the middle of a struct, furthermore
> this is using a GCC extension that is going to be deprecated in GCC 14 and
> a warning to identify such cases will be present
> (-Wflex-array-member-not-at-end) to identify such cases.
> 
> In order to fix this issue, use the macro __struct_group to create a
> structure 'struct membanks_hdr' which will hold the common data among
> structures using the 'struct membanks' interface.
> 
> Modify the 'struct shared_meminfo' and 'struct meminfo' to use this new
> structure, effectively removing the flexible array member from the middle
> of the structure and modify the code accessing the .common field to use
> the macro container_of to maintain the functionality of the interface.
> 
> Given this change, container_of needs to be supplied with a type and so
> the macro 'kernel_info_get_mem' inside arm/include/asm/kernel.h can't be
> an option since it uses const and non-const types for struct membanks, so
> introduce two static inline, one of which will keep the const qualifier.
> 
> Given the complexity of the interface, which carries a lot of benefit but
> on the other hand could be prone to developer confusion if the access is
> open-coded, introduce two static inline helper for the
> 'struct kernel_info' .shm_mem member and get rid the open-coding
> shm_mem.common access.
> 
> Fixes: 2209c1e35b47 ("xen/arm: Introduce a generic way to access memory bank structures")
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/acpi/domain_build.c        |  2 +-
>  xen/arch/arm/domain_build.c             |  6 +++---
>  xen/arch/arm/include/asm/kernel.h       | 11 ++++++++++-
>  xen/arch/arm/include/asm/setup.h        | 18 ++++++++++--------
>  xen/arch/arm/include/asm/static-shmem.h | 12 ++++++++++++
>  xen/arch/arm/static-shmem.c             | 19 +++++++++----------
>  6 files changed, 45 insertions(+), 23 deletions(-)
> 
> diff --git a/xen/arch/arm/acpi/domain_build.c b/xen/arch/arm/acpi/domain_build.c
> index ed895dd8f926..2ce75543d004 100644
> --- a/xen/arch/arm/acpi/domain_build.c
> +++ b/xen/arch/arm/acpi/domain_build.c
> @@ -451,7 +451,7 @@ static int __init estimate_acpi_efi_size(struct domain *d,
>      struct acpi_table_rsdp *rsdp_tbl;
>      struct acpi_table_header *table;
>  
> -    efi_size = estimate_efi_size(kernel_info_get_mem(kinfo)->nr_banks);
> +    efi_size = estimate_efi_size(kernel_info_get_mem_const(kinfo)->nr_banks);
>  
>      acpi_size = ROUNDUP(sizeof(struct acpi_table_fadt), 8);
>      acpi_size += ROUNDUP(sizeof(struct acpi_table_stao), 8);
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 0784e4c5e315..f6550809cfdf 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -805,7 +805,7 @@ int __init make_memory_node(const struct kernel_info *kinfo, int addrcells,
>       * static shared memory banks need to be listed as /memory node, so when
>       * this function is handling the normal memory, add the banks.
>       */
> -    if ( mem == kernel_info_get_mem(kinfo) )
> +    if ( mem == kernel_info_get_mem_const(kinfo) )
>          shm_mem_node_fill_reg_range(kinfo, reg, &nr_cells, addrcells,
>                                      sizecells);
>  
> @@ -884,7 +884,7 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
>  {
>      const struct membanks *mem = bootinfo_get_mem();
>      const struct membanks *mem_banks[] = {
> -        kernel_info_get_mem(kinfo),
> +        kernel_info_get_mem_const(kinfo),
>          bootinfo_get_reserved_mem(),
>  #ifdef CONFIG_STATIC_SHM
>          bootinfo_get_shmem(),
> @@ -1108,7 +1108,7 @@ static int __init find_domU_holes(const struct kernel_info *kinfo,
>      uint64_t bankend;
>      const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
>      const uint64_t banksize[] = GUEST_RAM_BANK_SIZES;
> -    const struct membanks *kinfo_mem = kernel_info_get_mem(kinfo);
> +    const struct membanks *kinfo_mem = kernel_info_get_mem_const(kinfo);
>      int res = -ENOENT;
>  
>      for ( i = 0; i < GUEST_RAM_BANKS; i++ )
> diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/kernel.h
> index 16a2bfc01e27..7e6e3c82a477 100644
> --- a/xen/arch/arm/include/asm/kernel.h
> +++ b/xen/arch/arm/include/asm/kernel.h
> @@ -80,7 +80,16 @@ struct kernel_info {
>      };
>  };
>  
> -#define kernel_info_get_mem(kinfo) (&(kinfo)->mem.common)
> +static inline struct membanks *kernel_info_get_mem(struct kernel_info *kinfo)
> +{
> +    return container_of(&kinfo->mem.common, struct membanks, common);
> +}
> +
> +static inline const struct membanks *
> +kernel_info_get_mem_const(const struct kernel_info *kinfo)
> +{
> +    return container_of(&kinfo->mem.common, const struct membanks, common);
> +}
>  
>  #ifdef CONFIG_STATIC_SHM
>  #define KERNEL_INFO_SHM_MEM_INIT .shm_mem.common.max_banks = NR_SHMEM_BANKS,
> diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
> index 28fb659fe946..61c15806a7c4 100644
> --- a/xen/arch/arm/include/asm/setup.h
> +++ b/xen/arch/arm/include/asm/setup.h
> @@ -64,18 +64,20 @@ struct membank {
>  };
>  
>  struct membanks {
> -    unsigned int nr_banks;
> -    unsigned int max_banks;
> +    __struct_group(membanks_hdr, common, ,
> +        unsigned int nr_banks;
> +        unsigned int max_banks;
> +    );
>      struct membank bank[];
>  };
>  
>  struct meminfo {
> -    struct membanks common;
> +    struct membanks_hdr common;
>      struct membank bank[NR_MEM_BANKS];
>  };
>  
>  struct shared_meminfo {
> -    struct membanks common;
> +    struct membanks_hdr common;
>      struct membank bank[NR_SHMEM_BANKS];
>      struct shmem_membank_extra extra[NR_SHMEM_BANKS];
>  };
> @@ -166,25 +168,25 @@ extern domid_t max_init_domid;
>  
>  static inline struct membanks *bootinfo_get_mem(void)
>  {
> -    return &bootinfo.mem.common;
> +    return container_of(&bootinfo.mem.common, struct membanks, common);
>  }
>  
>  static inline struct membanks *bootinfo_get_reserved_mem(void)
>  {
> -    return &bootinfo.reserved_mem.common;
> +    return container_of(&bootinfo.reserved_mem.common, struct membanks, common);
>  }
>  
>  #ifdef CONFIG_ACPI
>  static inline struct membanks *bootinfo_get_acpi(void)
>  {
> -    return &bootinfo.acpi.common;
> +    return container_of(&bootinfo.acpi.common, struct membanks, common);
>  }
>  #endif
>  
>  #ifdef CONFIG_STATIC_SHM
>  static inline struct membanks *bootinfo_get_shmem(void)
>  {
> -    return &bootinfo.shmem.common;
> +    return container_of(&bootinfo.shmem.common, struct membanks, common);
>  }
>  
>  static inline struct shmem_membank_extra *bootinfo_get_shmem_extra(void)
> diff --git a/xen/arch/arm/include/asm/static-shmem.h b/xen/arch/arm/include/asm/static-shmem.h
> index 3b6569e5703f..806ee41cfc37 100644
> --- a/xen/arch/arm/include/asm/static-shmem.h
> +++ b/xen/arch/arm/include/asm/static-shmem.h
> @@ -45,6 +45,18 @@ int make_shm_resv_memory_node(const struct kernel_info *kinfo, int addrcells,
>  void shm_mem_node_fill_reg_range(const struct kernel_info *kinfo, __be32 *reg,
>                                   int *nr_cells, int addrcells, int sizecells);
>  
> +static inline struct membanks *
> +kernel_info_get_shm_mem(struct kernel_info *kinfo)
> +{
> +    return container_of(&kinfo->shm_mem.common, struct membanks, common);
> +}
> +
> +static inline const struct membanks *
> +kernel_info_get_shm_mem_const(const struct kernel_info *kinfo)
> +{
> +    return container_of(&kinfo->shm_mem.common, const struct membanks, common);
> +}
> +
>  #else /* !CONFIG_STATIC_SHM */
>  
>  /* Worst case /memory node reg element: (addrcells + sizecells) */
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 09f474ec6050..78881dd1d3f7 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -172,16 +172,16 @@ static int __init assign_shared_memory(struct domain *d,
>  }
>  
>  static int __init
> -append_shm_bank_to_domain(struct shared_meminfo *kinfo_shm_mem, paddr_t start,
> +append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
>                            paddr_t size, const char *shm_id)
>  {
> -    struct membanks *shm_mem = &kinfo_shm_mem->common;
> +    struct membanks *shm_mem = kernel_info_get_shm_mem(kinfo);
>      struct shmem_membank_extra *shm_mem_extra;
>  
>      if ( shm_mem->nr_banks >= shm_mem->max_banks )
>          return -ENOMEM;
>  
> -    shm_mem_extra = &kinfo_shm_mem->extra[shm_mem->nr_banks];
> +    shm_mem_extra = &kinfo->shm_mem.extra[shm_mem->nr_banks];
>  
>      shm_mem->bank[shm_mem->nr_banks].start = start;
>      shm_mem->bank[shm_mem->nr_banks].size = size;
> @@ -289,8 +289,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>           * Record static shared memory region info for later setting
>           * up shm-node in guest device tree.
>           */
> -        ret = append_shm_bank_to_domain(&kinfo->shm_mem, gbase, psize,
> -                                        shm_id);
> +        ret = append_shm_bank_to_domain(kinfo, gbase, psize, shm_id);
>          if ( ret )
>              return ret;
>      }
> @@ -301,7 +300,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>  int __init make_shm_resv_memory_node(const struct kernel_info *kinfo,
>                                       int addrcells, int sizecells)
>  {
> -    const struct membanks *mem = &kinfo->shm_mem.common;
> +    const struct membanks *mem = kernel_info_get_shm_mem_const(kinfo);
>      void *fdt = kinfo->fdt;
>      unsigned int i = 0;
>      int res = 0;
> @@ -517,7 +516,7 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>  int __init make_resv_memory_node(const struct kernel_info *kinfo, int addrcells,
>                                   int sizecells)
>  {
> -    const struct membanks *mem = &kinfo->shm_mem.common;
> +    const struct membanks *mem = kernel_info_get_shm_mem_const(kinfo);
>      void *fdt = kinfo->fdt;
>      int res = 0;
>      /* Placeholder for reserved-memory\0 */
> @@ -579,7 +578,7 @@ void __init init_sharedmem_pages(void)
>  int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
>                                      struct rangeset *rangeset)
>  {
> -    const struct membanks *shm_mem = &kinfo->shm_mem.common;
> +    const struct membanks *shm_mem = kernel_info_get_shm_mem_const(kinfo);
>      unsigned int i;
>  
>      /* Remove static shared memory regions */
> @@ -607,7 +606,7 @@ int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
>  int __init remove_shm_holes_for_domU(const struct kernel_info *kinfo,
>                                       struct membanks *ext_regions)
>  {
> -    const struct membanks *shm_mem = &kinfo->shm_mem.common;
> +    const struct membanks *shm_mem = kernel_info_get_shm_mem_const(kinfo);
>      struct rangeset *guest_holes;
>      unsigned int i;
>      paddr_t start;
> @@ -673,7 +672,7 @@ void __init shm_mem_node_fill_reg_range(const struct kernel_info *kinfo,
>                                          __be32 *reg, int *nr_cells,
>                                          int addrcells, int sizecells)
>  {
> -    const struct membanks *mem = &kinfo->shm_mem.common;
> +    const struct membanks *mem = kernel_info_get_shm_mem_const(kinfo);
>      unsigned int i;
>      __be32 *cells;
>  
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 02 18:53:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 18:53:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716156.1118250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bYa-0007RY-HJ; Thu, 02 May 2024 18:53:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716156.1118250; Thu, 02 May 2024 18:53:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bYa-0007RR-El; Thu, 02 May 2024 18:53:24 +0000
Received: by outflank-mailman (input) for mailman id 716156;
 Thu, 02 May 2024 18:53:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2bYY-0007RK-SX
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 18:53:22 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d5000a6-08b5-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 20:53:19 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id BF60361C06;
 Thu,  2 May 2024 18:53:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99659C4AF1A;
 Thu,  2 May 2024 18:53:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d5000a6-08b5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714675997;
	bh=AfimSmHkaoZU3iA7iLwFL/fwp3B+C3d33xlz1Ppucto=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=p8PjtnAXAKKL1MIaab9KmHIPBa48bjyhx+vGrYdcMJ393cdQ73/riZ1Q9DzyoqBzJ
	 C7WmIO2KAYVW7sejgzXEtdORiVww0mKnA48OekoGSjwM9vGu1DCTxpyKwalRJNO88C
	 9To05A0iBu4DL51lJt5noboy6Y+nckcv96K0ev0BP13dV0mwylNWugw5s/Ps91M7qg
	 iOQOgqw1DcVUzYJkMYXIc1kZAZYsxWJ7ZZJKe2w8umTaLy5dCZ/6dr5gGt9+Qaip7a
	 eD6v2I7hGqcyRaMNuiBjXTbMKxj4qjW/33eoVt5fudGZ5XuSTpptZY6Wm3+tRFVb7+
	 SJt5Y0cVxmsAg==
Date: Thu, 2 May 2024 11:53:11 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, qemu-devel@nongnu.org, 
    jgross@suse.com, "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org, Xenia.Ragiadakou@amd.com
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
In-Reply-To: <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405021144270.624854@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-16-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop> <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1125765863-1714675481=:624854"
Content-ID: <alpine.DEB.2.22.394.2405021144490.624854@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1125765863-1714675481=:624854
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2405021144491.624854@ubuntu-linux-20-04-desktop>

+Xenia

On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> On Wed, May 1, 2024 at 11:24 PM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
> >
> > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> > >
> > > The current mapcache assumes that all memory is mapped
> > > in a single RAM MR (the first one with offset 0). Remove
> > > this assumption and propagate the offset to the mapcache
> > > so it can do reverse mappings (from hostptr -> ram_addr).
> > >
> > > This is in preparation for adding grant mappings.
> > >
> > > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> >
> >
> > Looking at xen_remap_bucket, it is only using address_index (without
> > adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
> > understand that address_index already includes the ram_offset.
> >
> > Meaning that if we want to map foreign mapping at address 0x5000, then
> > address_index would be 0x5000, even if ram_offset is 0x1000.
> >
> > But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> > to paddr_index to calculate the physical address. So in that case we
> > would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> > xen_remap_bucket would have to sum address_index and ram_offset to map
> > foreign memory.
> >
> > So I am a bit confused, did I get it wrong? One more comment below.
> >
> 
> Thanks Stefano,
> 
> I think the confusion is that this ram_addr_offset is not related to
> guest address-space.
> It's a QEMU internal thing and it shouldn't be included in the address
> used to map foreign memory.
> The mapcache can treat this ram_addr offset like a cookie that we keep
> around to be able to do
> reverse mappings from host pointers into ram_addr space
> (xen_ram_addr_from_mapcache).
> 
> The current mapcache implementation works because we've really only
> been using foreign mappings
> on RAMBlocks with offset 0. We're also creating RAM's such that the
> offset into the RAM is also
> the guest physical address, for x86 this is natural since RAM starts
> at zero (for lowmem) but for
> ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size) to
> make this assumption true. Anyway, In this series I'm not addressing
> this second assumption.

Let's see if I understand correctly.

The ram_addr space is an internal QEMU address space which is different
from the guest physical address space and thus cannot and should not be
used to do foreign mappings (foreign mapping hypercalls take a guest
physical or a real physical address to map). Is that correct?

If so, then I understand.



> There's a second call in physmem.c to xen_map_cache using the
> block->offset as an address.
> I was considering removing that second call since I can't see how it can work
> (except perhaps in some specific use-case by luck?). Anyway, for now
> I've left it unmodified.

Yes, that code was written with the assumption that block->offset is an
offset in the guest physical address space and could be used as a guest
physical address. Actually, you might have spotted a real bug.
 
The intent was for smaller regions (not the bit RAM region, things like
a ROM region for instance) we could map them in full. So here we were
trying to map the whole thing from start to finish using block->offset
as start.


> > > ---
> > >  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
> > >  include/sysemu/xen-mapcache.h |  2 ++
> > >  system/physmem.c              |  8 ++++----
> > >  3 files changed, 24 insertions(+), 11 deletions(-)
> > >
> > > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > > index 09b5f36d9c..1b32d0c003 100644
> > > --- a/hw/xen/xen-mapcache.c
> > > +++ b/hw/xen/xen-mapcache.c
> > > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> > >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > >      uint8_t flags;
> > >      hwaddr size;
> > > +
> > > +    /* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr).  */
> > > +    ram_addr_t ram_offset;
> > >      struct MapCacheEntry *next;
> > >  } MapCacheEntry;
> > >
> > > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> > >                               void *vaddr,
> > >                               hwaddr size,
> > >                               hwaddr address_index,
> > > -                             bool dummy)
> > > +                             bool dummy,
> > > +                             ram_addr_t ram_offset)
> > >  {
> > >      uint8_t *vaddr_base;
> > >      xen_pfn_t *pfns;
> > > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> > >      entry->size = size;
> > >      entry->valid_mapping = g_new0(unsigned long,
> > >                                    BITS_TO_LONGS(size >> XC_PAGE_SHIFT));
> > > +    entry->ram_offset = ram_offset;
> > >
> > >      if (dummy) {
> > >          entry->flags |= XEN_MAPCACHE_ENTRY_DUMMY;
> > > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> > >
> > >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > >                                         hwaddr phys_addr, hwaddr size,
> > > +                                       ram_addr_t ram_offset,
> > >                                         uint8_t lock, bool dma, bool is_write)
> > >  {
> > >      MapCacheEntry *entry, *pentry = NULL,
> > > @@ -335,14 +341,16 @@ tryagain:
> > >      if (!entry) {
> > >          entry = g_new0(MapCacheEntry, 1);
> > >          pentry->next = entry;
> > > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> > > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> > > +                         ram_offset);
> > >      } else if (!entry->lock) {
> > >          if (!entry->vaddr_base || entry->paddr_index != address_index ||
> > >                  entry->size != cache_size ||
> > >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> > >                      test_bit_size >> XC_PAGE_SHIFT,
> > >                      entry->valid_mapping)) {
> > > -            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> > > +            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> > > +                             ram_offset);
> > >          }
> > >      }
> > >
> > > @@ -389,13 +397,15 @@ tryagain:
> > >
> > >  uint8_t *xen_map_cache(MemoryRegion *mr,
> > >                         hwaddr phys_addr, hwaddr size,
> > > +                       ram_addr_t ram_addr_offset,
> > >                         uint8_t lock, bool dma,
> > >                         bool is_write)
> > >  {
> > >      uint8_t *p;
> > >
> > >      mapcache_lock(mapcache);
> > > -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
> > > +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> > > +                               lock, dma, is_write);
> > >      mapcache_unlock(mapcache);
> > >      return p;
> > >  }
> > > @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
> > >          raddr = RAM_ADDR_INVALID;
> > >      } else {
> > >          raddr = (reventry->paddr_index << mc->bucket_shift) +
> > > -             ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
> > > +             ((unsigned long) ptr - (unsigned long) entry->vaddr_base) +
> > > +             entry->ram_offset;
> > >      }
> > >      mapcache_unlock(mc);
> > >      return raddr;
> > > @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> > >
> > >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> > >
> > > -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> > > -                     cache_size, address_index, false);
> > > +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> > > +                     cache_size, address_index, false, entry->ram_offset);
> > >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> > >                  test_bit_size >> XC_PAGE_SHIFT,
> > >                  entry->valid_mapping)) {
> > > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> > > index 1ec9e66752..b5e3ea1bc0 100644
> > > --- a/include/sysemu/xen-mapcache.h
> > > +++ b/include/sysemu/xen-mapcache.h
> > > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
> > >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > >                          void *opaque);
> > >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
> > > +                       ram_addr_t ram_addr_offset,
> > >                         uint8_t lock, bool dma,
> > >                         bool is_write);
> > >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> > >                                       hwaddr phys_addr,
> > >                                       hwaddr size,
> > > +                                     ram_addr_t ram_addr_offset,
> > >                                       uint8_t lock,
> > >                                       bool dma,
> > >                                       bool is_write)
> > > diff --git a/system/physmem.c b/system/physmem.c
> > > index 1a5ffcba2a..5b16eeccca 100644
> > > --- a/system/physmem.c
> > > +++ b/system/physmem.c
> > > @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
> > >           * In that case just map the requested area.
> > >           */
> > >          if (xen_mr_is_memory(block->mr)) {
> > > -            return xen_map_cache(block->mr, addr, len, lock, lock,
> > > -                                 is_write);
> > > +            return xen_map_cache(block->mr, addr, len, block->offset,
> > > +                                 lock, lock, is_write);
> >
> > Have you considered not tracking offset and address separately and
> > simply do this?
> >
> >             return xen_map_cache(block->mr, addr + block->offset, len,
> >                                  lock, lock, is_write);
> >
> 
> Unfortunately this won't work since block->offset is not related to where this
> ram is mapped in guest address-space. In the case of grant's, we'd get the
> wrong grant ref. See my previous comment.

OK, this code below (the second xen_map_cache call passing block->offset
as start address) was wrong before this patch. Can we fix it before
changing it further with this patch? I worry about making things even
worse.


> > >          }
> > >
> > >          block->host = xen_map_cache(block->mr, block->offset,
> > > -                                    block->max_length, 1,
> > > -                                    lock, is_write);
> > > +                                    block->max_length, 0,
> > > +                                    1, lock, is_write);
> > >      }
> > >
> > >      return ramblock_ptr(block, addr);
> > > --
> > > 2.40.1
> > >
> 
--8323329-1125765863-1714675481=:624854--


From xen-devel-bounces@lists.xenproject.org Thu May 02 19:18:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 19:18:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716163.1118261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bwk-0006xn-IR; Thu, 02 May 2024 19:18:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716163.1118261; Thu, 02 May 2024 19:18:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2bwk-0006xg-Ew; Thu, 02 May 2024 19:18:22 +0000
Received: by outflank-mailman (input) for mailman id 716163;
 Thu, 02 May 2024 19:18:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2bwj-0006xa-6W
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 19:18:21 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9ed244b-08b8-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 21:18:18 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 91139CE1791;
 Thu,  2 May 2024 19:18:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 278C8C113CC;
 Thu,  2 May 2024 19:18:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9ed244b-08b8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714677490;
	bh=x9X9JAlQJrR7tb6OdOoyjzyrYN9440N5cC/SzcgJewQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gms6kZdc2jftebu2v69jaOz0bCewYaSGpkhmKkztDo80a+HZaCyREJMX+nh9odIrI
	 dynCuUuXMd+rZezP2zhEu5BtVtdH8ia0tqQY3Qbpw5abxS/VsM77Yy3OGj1poT1ZVO
	 MzJbWYiQ5oBwkBvq0XZTxQfkLH6iHJ2NGfyYco/M1XfMLN9W3wlo1GJSKbkuLg4rcK
	 Y1QMyb+dociJM4CEDoIVUHsi2H73P8i0Zkl564WZXVhwY0rQv9zfxcI5eMcjow65HN
	 AeXcjELc1Zg9BXIRDAQWHOTWXsWe5yCBj4ekHXMVhWSLXSwT5j6bYFdETrxU+eugwd
	 AIo2b2pMop5fA==
Date: Thu, 2 May 2024 12:18:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v4 16/17] xen: mapcache: Add support for grant mappings
In-Reply-To: <20240430164939.925307-17-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405021156440.624854@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-17-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add a second mapcache for grant mappings. The mapcache for
> grants needs to work with XC_PAGE_SIZE granularity since
> we can't map larger ranges than what has been granted to us.
> 
> Like with foreign mappings (xen_memory), machines using grants
> are expected to initialize the xen_grants MR and map it
> into their address-map accordingly.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  hw/xen/xen-hvm-common.c         |  12 ++-
>  hw/xen/xen-mapcache.c           | 158 +++++++++++++++++++++++++-------
>  include/hw/xen/xen-hvm-common.h |   3 +
>  include/sysemu/xen.h            |   7 ++
>  4 files changed, 145 insertions(+), 35 deletions(-)
> 
> diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> index 0267b88d26..fdec400491 100644
> --- a/hw/xen/xen-hvm-common.c
> +++ b/hw/xen/xen-hvm-common.c
> @@ -10,12 +10,18 @@
>  #include "hw/boards.h"
>  #include "hw/xen/arch_hvm.h"
>  
> -MemoryRegion xen_memory;
> +MemoryRegion xen_memory, xen_grants;
>  
> -/* Check for xen memory.  */
> +/* Check for any kind of xen memory, foreign mappings or grants.  */
>  bool xen_mr_is_memory(MemoryRegion *mr)
>  {
> -    return mr == &xen_memory;
> +    return mr == &xen_memory || mr == &xen_grants;
> +}
> +
> +/* Check specifically for grants.  */
> +bool xen_mr_is_grants(MemoryRegion *mr)
> +{
> +    return mr == &xen_grants;
>  }
>  
>  void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 1b32d0c003..96cd68e28d 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -14,6 +14,7 @@
>  
>  #include <sys/resource.h>
>  
> +#include "hw/xen/xen-hvm-common.h"
>  #include "hw/xen/xen_native.h"
>  #include "qemu/bitmap.h"
>  
> @@ -21,6 +22,8 @@
>  #include "sysemu/xen-mapcache.h"
>  #include "trace.h"
>  
> +#include <xenevtchn.h>
> +#include <xengnttab.h>
>  
>  #if HOST_LONG_BITS == 32
>  #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
> @@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
>      unsigned long *valid_mapping;
>      uint32_t lock;
>  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> +#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
>      uint8_t flags;
>      hwaddr size;
>  
> @@ -74,6 +78,8 @@ typedef struct MapCache {
>  } MapCache;
>  
>  static MapCache *mapcache;
> +static MapCache *mapcache_grants;
> +static xengnttab_handle *xen_region_gnttabdev;
>  
>  static inline void mapcache_lock(MapCache *mc)
>  {
> @@ -132,6 +138,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>      unsigned long max_mcache_size;
>      unsigned int bucket_shift;
>  
> +    xen_region_gnttabdev = xengnttab_open(NULL, 0);
> +    if (xen_region_gnttabdev == NULL) {
> +        error_report("mapcache: Failed to open gnttab device");
> +        exit(EXIT_FAILURE);
> +    }
> +
>      if (HOST_LONG_BITS == 32) {
>          bucket_shift = 16;
>      } else {
> @@ -160,6 +172,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>      mapcache = xen_map_cache_init_single(f, opaque,
>                                           bucket_shift,
>                                           max_mcache_size);
> +
> +    /*
> +     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
> +     * map anything beyond the number of pages granted to us.
> +     */
> +    mapcache_grants = xen_map_cache_init_single(f, opaque,
> +                                                XC_PAGE_SHIFT,
> +                                                max_mcache_size);
> +
>      setrlimit(RLIMIT_AS, &rlimit_as);
>  }
>  
> @@ -169,17 +190,25 @@ static void xen_remap_bucket(MapCache *mc,
>                               hwaddr size,
>                               hwaddr address_index,
>                               bool dummy,
> +                             bool grant,
> +                             bool grant_is_write,
> +                             hwaddr grant_ref,
>                               ram_addr_t ram_offset)

Any chance we could pass grant_ref as address_index ?

Also instead of grant_is_write we could have a generic is_write that
applies to both.

I am not sure about this, but instead of bool grant, we could check on
address_index using XEN_GRANT_ADDR_OFF? This one might not work.

I admit that there is no real advantage on these suggestions except to
consolidate the parameters and make them look a bit more similar in the
two cases.



>  {
>      uint8_t *vaddr_base;
> -    xen_pfn_t *pfns;
> +    uint32_t *refs = NULL;
> +    xen_pfn_t *pfns = NULL;
>      int *err;
>      unsigned int i;
>      hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
>  
>      trace_xen_remap_bucket(address_index);
>  
> -    pfns = g_new0(xen_pfn_t, nb_pfn);
> +    if (grant) {
> +        refs = g_new0(uint32_t, nb_pfn);
> +    } else {
> +        pfns = g_new0(xen_pfn_t, nb_pfn);
> +    }
>      err = g_new0(int, nb_pfn);
>  
>      if (entry->vaddr_base != NULL) {
> @@ -208,21 +237,45 @@ static void xen_remap_bucket(MapCache *mc,
>      g_free(entry->valid_mapping);
>      entry->valid_mapping = NULL;
>  
> -    for (i = 0; i < nb_pfn; i++) {
> -        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
> +    if (grant) {
> +        for (i = 0; i < nb_pfn; i++) {
> +            refs[i] = grant_ref + i;
> +        }
> +    } else {
> +        for (i = 0; i < nb_pfn; i++) {
> +            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
> +        }
>      }
>  
> -    /*
> -     * If the caller has requested the mapping at a specific address use
> -     * MAP_FIXED to make sure it's honored.
> -     */
> +    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
> +
>      if (!dummy) {
> -        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
> -                                           PROT_READ | PROT_WRITE,
> -                                           vaddr ? MAP_FIXED : 0,
> -                                           nb_pfn, pfns, err);
> +        if (grant) {
> +            int prot = PROT_READ;
> +
> +            if (grant_is_write) {
> +                prot |= PROT_WRITE;
> +            }
> +
> +            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
> +            assert(vaddr == NULL);
> +            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
> +                                                         nb_pfn,
> +                                                         xen_domid, refs,
> +                                                         prot);
> +        } else {
> +            /*
> +             * If the caller has requested the mapping at a specific address use
> +             * MAP_FIXED to make sure it's honored.
> +             */
> +            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
> +                                               PROT_READ | PROT_WRITE,
> +                                               vaddr ? MAP_FIXED : 0,
> +                                               nb_pfn, pfns, err);
> +        }
>          if (vaddr_base == NULL) {
> -            perror("xenforeignmemory_map2");
> +            perror(grant ? "xengnttab_map_domain_grant_refs"
> +                           : "xenforeignmemory_map2");
>              exit(-1);
>          }
>      } else {
> @@ -263,6 +316,7 @@ static void xen_remap_bucket(MapCache *mc,
>          }
>      }
>  
> +    g_free(refs);
>      g_free(pfns);
>      g_free(err);
>  }
> @@ -270,10 +324,12 @@ static void xen_remap_bucket(MapCache *mc,
>  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>                                         hwaddr phys_addr, hwaddr size,
>                                         ram_addr_t ram_offset,
> -                                       uint8_t lock, bool dma, bool is_write)
> +                                       uint8_t lock, bool dma,
> +                                       bool grant, bool is_write)
>  {
>      MapCacheEntry *entry, *pentry = NULL,
>                    *free_entry = NULL, *free_pentry = NULL;
> +    hwaddr grant_ref = phys_addr >> XC_PAGE_SHIFT;
>      hwaddr address_index;
>      hwaddr address_offset;
>      hwaddr cache_size = size;
> @@ -342,7 +398,7 @@ tryagain:
>          entry = g_new0(MapCacheEntry, 1);
>          pentry->next = entry;
>          xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> -                         ram_offset);
> +                         grant, is_write, grant_ref, ram_offset);
>      } else if (!entry->lock) {
>          if (!entry->vaddr_base || entry->paddr_index != address_index ||
>                  entry->size != cache_size ||
> @@ -350,7 +406,7 @@ tryagain:
>                      test_bit_size >> XC_PAGE_SHIFT,
>                      entry->valid_mapping)) {
>              xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> -                             ram_offset);
> +                             grant, is_write, grant_ref, ram_offset);
>          }
>      }
>  
> @@ -401,12 +457,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
>                         uint8_t lock, bool dma,
>                         bool is_write)
>  {
> +    bool grant = xen_mr_is_grants(mr);
> +    MapCache *mc = grant ? mapcache_grants : mapcache;
>      uint8_t *p;
>  
> -    mapcache_lock(mapcache);
> -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> -                               lock, dma, is_write);
> -    mapcache_unlock(mapcache);
> +    if (grant) {
> +        /*
> +         * Grants are only supported via address_space_map(). Anything
> +         * else is considered a user/guest error.
> +         *
> +         * QEMU generally doesn't expect these mappings to ever fail, so
> +         * if this happens we report an error message and abort().
> +         */
> +        if (!lock) {
> +            error_report("Trying access a grant reference without mapping it.");
> +            abort();
> +        }
> +    }
> +
> +    mapcache_lock(mc);
> +    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
> +                               lock, dma, grant, is_write);
> +    mapcache_unlock(mc);
>      return p;
>  }
>  
> @@ -451,7 +523,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
>  
>  ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
>  {
> -    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
> +    ram_addr_t addr;
> +
> +    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
> +    if (addr == RAM_ADDR_INVALID) {
> +        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
> +    }
> +
> +    return addr;
>  }
>  
>  static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> @@ -504,9 +583,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>      }
>  
>      ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
> -    if (munmap(entry->vaddr_base, entry->size) != 0) {
> -        perror("unmap fails");
> -        exit(-1);
> +    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
> +        xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
> +                    (entry->size + mc->bucket_size - 1) >> mc->bucket_shift);

Am I getting this right that the + mc->bucket_size - 1 is unnecessary
because the bucket size is PAGE_SIZE and we can only map at page
granularity?

Also can we check for return errors?


> +    } else {
> +        if (munmap(entry->vaddr_base, entry->size) != 0) {
> +            perror("unmap fails");
> +            exit(-1);
> +        }
>      }
>      if (pentry) {
>          pentry->next = entry->next;
> @@ -522,14 +606,24 @@ typedef struct XenMapCacheData {
>      uint8_t *buffer;
>  } XenMapCacheData;
>  
> +static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
> +{
> +    mapcache_lock(mc);
> +    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
> +    mapcache_unlock(mc);
> +}
> +
> +static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
> +{
> +    xen_invalidate_map_cache_entry_single(mapcache, buffer);
> +    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
> +}
> +
>  static void xen_invalidate_map_cache_entry_bh(void *opaque)
>  {
>      XenMapCacheData *data = opaque;
>  
> -    mapcache_lock(mapcache);
> -    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
> -    mapcache_unlock(mapcache);
> -
> +    xen_invalidate_map_cache_entry_all(data->buffer);
>      aio_co_wake(data->co);
>  }
>  
> @@ -544,9 +638,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
>                                  xen_invalidate_map_cache_entry_bh, &data);
>          qemu_coroutine_yield();
>      } else {
> -        mapcache_lock(mapcache);
> -        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
> -        mapcache_unlock(mapcache);
> +        xen_invalidate_map_cache_entry_all(buffer);
>      }
>  }
>  
> @@ -598,6 +690,7 @@ void xen_invalidate_map_cache(void)
>      bdrv_drain_all();
>  
>      xen_invalidate_map_cache_single(mapcache);
> +    xen_invalidate_map_cache_single(mapcache_grants);
>  }
>  
>  static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> @@ -639,7 +732,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
>  
>      xen_remap_bucket(mc, entry, entry->vaddr_base,
> -                     cache_size, address_index, false, entry->ram_offset);
> +                     cache_size, address_index, false,
> +                     false, false, 0, entry->ram_offset);

If I understand correctly, xen_replace_cache_entry_unlocked cannot be
called on grants because xen_replace_cache_entry_unlocked is always
called on unlocked entries while grants are always locked. Should we
have an assert on !entry->lock and/or !(entry->flags & XEN_MAPCACHE_ENTRY_GRANT)?



>      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
>                  test_bit_size >> XC_PAGE_SHIFT,
>                  entry->valid_mapping)) {
> diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
> index 65a51aac2e..3d796235dc 100644
> --- a/include/hw/xen/xen-hvm-common.h
> +++ b/include/hw/xen/xen-hvm-common.h
> @@ -16,6 +16,7 @@
>  #include <xen/hvm/ioreq.h>
>  
>  extern MemoryRegion xen_memory;
> +extern MemoryRegion xen_grants;
>  extern MemoryListener xen_io_listener;
>  extern DeviceListener xen_device_listener;
>  
> @@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
>      do { } while (0)
>  #endif
>  
> +#define XEN_GRANT_ADDR_OFF (1ULL << 63)
> +
>  static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
>  {
>      return shared_page->vcpu_ioreq[i].vp_eport;
> diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> index dc72f83bcb..19dccf4d71 100644
> --- a/include/sysemu/xen.h
> +++ b/include/sysemu/xen.h
> @@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>                     struct MemoryRegion *mr, Error **errp);
>  
>  bool xen_mr_is_memory(MemoryRegion *mr);
> +bool xen_mr_is_grants(MemoryRegion *mr);
>  
>  #else /* !CONFIG_XEN_IS_POSSIBLE */
>  
> @@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
>      return false;
>  }
>  
> +static inline bool xen_mr_is_grants(MemoryRegion *mr)
> +{
> +    g_assert_not_reached();
> +    return false;
> +}
> +
>  #endif /* CONFIG_XEN_IS_POSSIBLE */
>  
>  #endif
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 02 19:43:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 19:43:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716169.1118271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cKa-0004o8-EO; Thu, 02 May 2024 19:43:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716169.1118271; Thu, 02 May 2024 19:43:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cKa-0004o1-AD; Thu, 02 May 2024 19:43:00 +0000
Received: by outflank-mailman (input) for mailman id 716169;
 Thu, 02 May 2024 19:42:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2cKZ-0004nv-Hk
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 19:42:59 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2cdc448c-08bc-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 21:42:57 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a597c192246so62336666b.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 12:42:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cdc448c-08bc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714678977; x=1715283777; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=FOZllDi8OvGMdRL5OmEg4WfFDW3bpic0V8Dsz4KaBgE=;
        b=hO6jdVTKHdZ89Puf/YPwLvM6r2zy/g6gxvHKtX0G3dVyUPZrIj6M8LbQaJ0PcJG9UT
         aqMl+lcDCIjBMyiLOKXvJ8ain33LDqXYdMq6cckHn3Vk3DZ1XOwVE6en5DFKt82IYTye
         UdDPEnr94fvOenDOffILtLljXWb6WteosHquAazpxjDLMaQ78wxVBYtuWrCvyRhmacUz
         VYmHCAowLyX5PSyQ7KXhiY/3yo7lAG+pZu/0+5hEY1AYBogx4O5QLpJwerorwp7yhR03
         mcqzVNKyoxE3QFrydmkRPnIc80v2H7FVQ+E/0diBiaE3tRpWmTMdqSGDomyski0BwxCA
         jy4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714678977; x=1715283777;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=FOZllDi8OvGMdRL5OmEg4WfFDW3bpic0V8Dsz4KaBgE=;
        b=rQSsX3Hc2UovFnc3H4eFt4OnHiSFQmsxA+JIrlmoeEtSkGOd6YpmHiO7Ajal+De2gb
         LYQ3Rpttgd+O7YQKBq3UQlfyDMMUvSFVxpFNj1Rs3z5S52NF0ZBeYdTaHUzxiUzuf28g
         6JNtreBzrQMdoEHl4eJ//5JdWrVOPyjSaplpKSr+tQmmGYCLIjv0OpvsSNsc1A7B+soY
         sIocjB3K4R43kvb3v85SSPo1kN7D6Dgebwe0pkl/WAPndGbFPMtq+yEFpF3ceRBaGB1x
         oGUYlQWf1QS1VfIhSI8o12wZ97jY3Ms2Ur/ghOnrd3fFXPKLvuGmNsLkxwjw8OnjXVxb
         ighA==
X-Forwarded-Encrypted: i=1; AJvYcCWIFaD7a5evOCaVhOT29XT120W1pHZ5RpE6ApEWY6XeWniT5umovDMkMMiSTxMIihgAfqQGRwwwiW2s4ymAJG6lkvZbm8WdB++LcJFCf78=
X-Gm-Message-State: AOJu0Yz0xazf4Hqdli4i3k1V6TDxZ8Owlim5YcrNgvjHiUB8yQssaxaD
	tyUmA52Dzg6+wxN7W2Af04oAsFUJN/q+gGp5igd7VEfsBURv5yPYR/vxvN5QRMwSyNDTW+ryqTB
	xGGm5qxKf67gdzzbG8iYVfov+wPs=
X-Google-Smtp-Source: AGHT+IH8Va0CQyduyPWxrVuJV6TU8ZZbOVVj3Kog1k7DHkXdH98rvYhW1sV2IsOOL52e0uNbd06M3XW6wXFR0BTz7YY=
X-Received: by 2002:a50:8e5d:0:b0:572:3f41:25aa with SMTP id
 29-20020a508e5d000000b005723f4125aamr300748edx.11.1714678976373; Thu, 02 May
 2024 12:42:56 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-16-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop>
 <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com> <alpine.DEB.2.22.394.2405021144270.624854@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405021144270.624854@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 21:42:44 +0200
Message-ID: <CAJy5ezq+wd4T91YPEarea4eUXp_CJftvPYMJT2AzA66DWjy34w@mail.gmail.com>
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
	David Hildenbrand <david@redhat.com>, =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>, 
	xen-devel@lists.xenproject.org, Xenia.Ragiadakou@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 2, 2024 at 8:53=E2=80=AFPM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> +Xenia
>
> On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > On Wed, May 1, 2024 at 11:24=E2=80=AFPM Stefano Stabellini
> > <sstabellini@kernel.org> wrote:
> > >
> > > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> > > >
> > > > The current mapcache assumes that all memory is mapped
> > > > in a single RAM MR (the first one with offset 0). Remove
> > > > this assumption and propagate the offset to the mapcache
> > > > so it can do reverse mappings (from hostptr -> ram_addr).
> > > >
> > > > This is in preparation for adding grant mappings.
> > > >
> > > > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > >
> > >
> > > Looking at xen_remap_bucket, it is only using address_index (without
> > > adding ram_offset) to map foreign memory. From xen_remap_bucket, I wo=
uld
> > > understand that address_index already includes the ram_offset.
> > >
> > > Meaning that if we want to map foreign mapping at address 0x5000, the=
n
> > > address_index would be 0x5000, even if ram_offset is 0x1000.
> > >
> > > But then looking xen_ram_addr_from_mapcache_single ram_offset is adde=
d
> > > to paddr_index to calculate the physical address. So in that case we
> > > would want address_index to be 0x4000 and ram_offset to be 0x1000. Bu=
t
> > > xen_remap_bucket would have to sum address_index and ram_offset to ma=
p
> > > foreign memory.
> > >
> > > So I am a bit confused, did I get it wrong? One more comment below.
> > >
> >
> > Thanks Stefano,
> >
> > I think the confusion is that this ram_addr_offset is not related to
> > guest address-space.
> > It's a QEMU internal thing and it shouldn't be included in the address
> > used to map foreign memory.
> > The mapcache can treat this ram_addr offset like a cookie that we keep
> > around to be able to do
> > reverse mappings from host pointers into ram_addr space
> > (xen_ram_addr_from_mapcache).
> >
> > The current mapcache implementation works because we've really only
> > been using foreign mappings
> > on RAMBlocks with offset 0. We're also creating RAM's such that the
> > offset into the RAM is also
> > the guest physical address, for x86 this is natural since RAM starts
> > at zero (for lowmem) but for
> > ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size=
) to
> > make this assumption true. Anyway, In this series I'm not addressing
> > this second assumption.
>
> Let's see if I understand correctly.
>
> The ram_addr space is an internal QEMU address space which is different
> from the guest physical address space and thus cannot and should not be
> used to do foreign mappings (foreign mapping hypercalls take a guest
> physical or a real physical address to map). Is that correct?
>
> If so, then I understand.
>

Yes, that matches my understanding.

>
>
> > There's a second call in physmem.c to xen_map_cache using the
> > block->offset as an address.
> > I was considering removing that second call since I can't see how it ca=
n work
> > (except perhaps in some specific use-case by luck?). Anyway, for now
> > I've left it unmodified.
>
> Yes, that code was written with the assumption that block->offset is an
> offset in the guest physical address space and could be used as a guest
> physical address. Actually, you might have spotted a real bug.
>
> The intent was for smaller regions (not the bit RAM region, things like
> a ROM region for instance) we could map them in full. So here we were
> trying to map the whole thing from start to finish using block->offset
> as start.
>
>
> > > > ---
> > > >  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
> > > >  include/sysemu/xen-mapcache.h |  2 ++
> > > >  system/physmem.c              |  8 ++++----
> > > >  3 files changed, 24 insertions(+), 11 deletions(-)
> > > >
> > > > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > > > index 09b5f36d9c..1b32d0c003 100644
> > > > --- a/hw/xen/xen-mapcache.c
> > > > +++ b/hw/xen/xen-mapcache.c
> > > > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> > > >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > > >      uint8_t flags;
> > > >      hwaddr size;
> > > > +
> > > > +    /* Keep ram_addr offset for reverse mappings (hostptr -> ram_a=
ddr).  */
> > > > +    ram_addr_t ram_offset;
> > > >      struct MapCacheEntry *next;
> > > >  } MapCacheEntry;
> > > >
> > > > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> > > >                               void *vaddr,
> > > >                               hwaddr size,
> > > >                               hwaddr address_index,
> > > > -                             bool dummy)
> > > > +                             bool dummy,
> > > > +                             ram_addr_t ram_offset)
> > > >  {
> > > >      uint8_t *vaddr_base;
> > > >      xen_pfn_t *pfns;
> > > > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > >      entry->size =3D size;
> > > >      entry->valid_mapping =3D g_new0(unsigned long,
> > > >                                    BITS_TO_LONGS(size >> XC_PAGE_SH=
IFT));
> > > > +    entry->ram_offset =3D ram_offset;
> > > >
> > > >      if (dummy) {
> > > >          entry->flags |=3D XEN_MAPCACHE_ENTRY_DUMMY;
> > > > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > >
> > > >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > > >                                         hwaddr phys_addr, hwaddr si=
ze,
> > > > +                                       ram_addr_t ram_offset,
> > > >                                         uint8_t lock, bool dma, boo=
l is_write)
> > > >  {
> > > >      MapCacheEntry *entry, *pentry =3D NULL,
> > > > @@ -335,14 +341,16 @@ tryagain:
> > > >      if (!entry) {
> > > >          entry =3D g_new0(MapCacheEntry, 1);
> > > >          pentry->next =3D entry;
> > > > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy);
> > > > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy,
> > > > +                         ram_offset);
> > > >      } else if (!entry->lock) {
> > > >          if (!entry->vaddr_base || entry->paddr_index !=3D address_=
index ||
> > > >                  entry->size !=3D cache_size ||
> > > >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> > > >                      test_bit_size >> XC_PAGE_SHIFT,
> > > >                      entry->valid_mapping)) {
> > > > -            xen_remap_bucket(mc, entry, NULL, cache_size, address_=
index, dummy);
> > > > +            xen_remap_bucket(mc, entry, NULL, cache_size, address_=
index, dummy,
> > > > +                             ram_offset);
> > > >          }
> > > >      }
> > > >
> > > > @@ -389,13 +397,15 @@ tryagain:
> > > >
> > > >  uint8_t *xen_map_cache(MemoryRegion *mr,
> > > >                         hwaddr phys_addr, hwaddr size,
> > > > +                       ram_addr_t ram_addr_offset,
> > > >                         uint8_t lock, bool dma,
> > > >                         bool is_write)
> > > >  {
> > > >      uint8_t *p;
> > > >
> > > >      mapcache_lock(mapcache);
> > > > -    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, lock, =
dma, is_write);
> > > > +    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, ram_ad=
dr_offset,
> > > > +                               lock, dma, is_write);
> > > >      mapcache_unlock(mapcache);
> > > >      return p;
> > > >  }
> > > > @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcache_si=
ngle(MapCache *mc, void *ptr)
> > > >          raddr =3D RAM_ADDR_INVALID;
> > > >      } else {
> > > >          raddr =3D (reventry->paddr_index << mc->bucket_shift) +
> > > > -             ((unsigned long) ptr - (unsigned long) entry->vaddr_b=
ase);
> > > > +             ((unsigned long) ptr - (unsigned long) entry->vaddr_b=
ase) +
> > > > +             entry->ram_offset;
> > > >      }
> > > >      mapcache_unlock(mc);
> > > >      return raddr;
> > > > @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocke=
d(MapCache *mc,
> > > >
> > > >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_ad=
dr);
> > > >
> > > > -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> > > > -                     cache_size, address_index, false);
> > > > +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> > > > +                     cache_size, address_index, false, entry->ram_=
offset);
> > > >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> > > >                  test_bit_size >> XC_PAGE_SHIFT,
> > > >                  entry->valid_mapping)) {
> > > > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-map=
cache.h
> > > > index 1ec9e66752..b5e3ea1bc0 100644
> > > > --- a/include/sysemu/xen-mapcache.h
> > > > +++ b/include/sysemu/xen-mapcache.h
> > > > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr p=
hys_offset,
> > > >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > > >                          void *opaque);
> > > >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr =
size,
> > > > +                       ram_addr_t ram_addr_offset,
> > > >                         uint8_t lock, bool dma,
> > > >                         bool is_write);
> > > >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > > > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset=
_to_gaddr_t f,
> > > >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> > > >                                       hwaddr phys_addr,
> > > >                                       hwaddr size,
> > > > +                                     ram_addr_t ram_addr_offset,
> > > >                                       uint8_t lock,
> > > >                                       bool dma,
> > > >                                       bool is_write)
> > > > diff --git a/system/physmem.c b/system/physmem.c
> > > > index 1a5ffcba2a..5b16eeccca 100644
> > > > --- a/system/physmem.c
> > > > +++ b/system/physmem.c
> > > > @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *=
block, ram_addr_t addr,
> > > >           * In that case just map the requested area.
> > > >           */
> > > >          if (xen_mr_is_memory(block->mr)) {
> > > > -            return xen_map_cache(block->mr, addr, len, lock, lock,
> > > > -                                 is_write);
> > > > +            return xen_map_cache(block->mr, addr, len, block->offs=
et,
> > > > +                                 lock, lock, is_write);
> > >
> > > Have you considered not tracking offset and address separately and
> > > simply do this?
> > >
> > >             return xen_map_cache(block->mr, addr + block->offset, len=
,
> > >                                  lock, lock, is_write);
> > >
> >
> > Unfortunately this won't work since block->offset is not related to whe=
re this
> > ram is mapped in guest address-space. In the case of grant's, we'd get =
the
> > wrong grant ref. See my previous comment.
>
> OK, this code below (the second xen_map_cache call passing block->offset
> as start address) was wrong before this patch. Can we fix it before
> changing it further with this patch? I worry about making things even
> worse.
>

I'll dig around and see if we can find something that explains more.
There's some older code that implements some sort of address-translation
for x86 between ram_addr space and guest physical addresses but
that code is turned off with newer Xen versions (disabled in my build).

https://github.com/qemu/qemu/blob/master/hw/xen/xen-mapcache.c#L330
https://github.com/qemu/qemu/blob/master/hw/i386/xen/xen-hvm.c#L193

Cheers,
Edgar


> > > >          }
> > > >
> > > >          block->host =3D xen_map_cache(block->mr, block->offset,
> > > > -                                    block->max_length, 1,
> > > > -                                    lock, is_write);
> > > > +                                    block->max_length, 0,
> > > > +                                    1, lock, is_write);
> > > >      }
> > > >
> > > >      return ramblock_ptr(block, addr);
> > > > --
> > > > 2.40.1
> > > >
> >


From xen-devel-bounces@lists.xenproject.org Thu May 02 19:50:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 19:50:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716177.1118281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cRR-0007Yd-7L; Thu, 02 May 2024 19:50:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716177.1118281; Thu, 02 May 2024 19:50:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cRR-0007YV-3m; Thu, 02 May 2024 19:50:05 +0000
Received: by outflank-mailman (input) for mailman id 716177;
 Thu, 02 May 2024 19:50:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1TOG=MF=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s2cRP-0007Dh-Ql
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 19:50:03 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 29d5838c-08bd-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 21:50:01 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5883518135so1018947366b.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 12:50:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29d5838c-08bd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714679401; x=1715284201; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=QfaVwS92YREa0mSSqG+DnBTPDjMpnFYk44qaBNVAgrw=;
        b=cGATaWeKtvtfIvTFcIhI9YmVV3UqPSzo95B2OIaGELjCYSc2ZpvRdKvlSN5J+L/1QL
         wYvcBw8ThR3DfhSMkTRchaOVrkB9VjAXn0FMy1ZWxPlKezOpnWZieJkBdJMXpN+uKT60
         C2WorFo9HCFZ5Mv0PCbNuMsadHUXBOsUcwaF+dds6i/DtkYGPqhFpA+/iXI9QwfYaNQO
         o1W/wQD1sApie4olMiHFSI5ano7lr7sXenZF2BsXkkzzjkDL1sEGF2DMMUE/KSWto0Bq
         5seB7TUY5M8q377bQgnPn5RHofeTONkt6mdRVNc9tzJGwbQ6GxE++HOy5NGocOk/mWUL
         QhHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714679401; x=1715284201;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=QfaVwS92YREa0mSSqG+DnBTPDjMpnFYk44qaBNVAgrw=;
        b=qiJ/3vHx+tPKtfsaDL52yGGly2UwFCs+MmJQiYQzbqePlJ4MnNxSUl7Cft/gPoufCm
         6gO0dWYu5huQyOoZILT7r4X8kncNkrIiVxxIwx9jRSJHTqDJzDKq3ZiKMbsJeU95wlk/
         sbAWNRQ48+DWwt92PPbQMJkTYMyFa0w6/s8HrybkNHAOJ8pC1/JrfdxBPxwDtyaAx+ug
         HrJimZr8dUdRO8Zj9pWuzHpC7h6fdSrpJrjuY8NNA5tqjuQq6RBBIGGPxr6JhmxU2R+4
         zzOsFmihZxBYo3gpU5eTnDcLMaJQSgSUArN9gMyNmA1Q0xks++7rYfsWkrk7yYkP1ecv
         opZQ==
X-Forwarded-Encrypted: i=1; AJvYcCVqaxY0bX+tWsl9PPxVYSMIOtxNYVzrhxMtlDtStnHMD4FNeeJfq/6cehSHVgaQj0GTv5zlC8ZC2XfbGpkmeiBZ/hyUYaXTM2yJuMPC+EY=
X-Gm-Message-State: AOJu0Yyt+4uzRCPabf9e/1u26pafzOmtuLmH8R/Iw0GqEs8+o5nAC1SI
	Y/PbBp7LriuWY67aSypSnLZ9DqBQbOoQ7GiLzFJHcg4zXMW8MJWkNrBpF6uD+CYoBG+rz51g+dV
	RtO7+8CDG/Nw5DX2NLXQBL3LcJ9M=
X-Google-Smtp-Source: AGHT+IF3UKceKxMn0GLxHk1LXZQB6a3XIa8JX/oAlwmFKq0ZStuQgwHoCnXoQRtfAg5/HJ44bZUqOiXmrttJqaMWg58=
X-Received: by 2002:a50:d741:0:b0:572:727f:d0db with SMTP id
 i1-20020a50d741000000b00572727fd0dbmr286422edj.15.1714679400730; Thu, 02 May
 2024 12:50:00 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-17-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405021156440.624854@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405021156440.624854@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 2 May 2024 21:49:49 +0200
Message-ID: <CAJy5ezoMYmf3HbEraqcK775t59TWH+BUzbPnxkWMMFMNUh6d2w@mail.gmail.com>
Subject: Re: [PATCH v4 16/17] xen: mapcache: Add support for grant mappings
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 2, 2024 at 9:18=E2=80=AFPM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Add a second mapcache for grant mappings. The mapcache for
> > grants needs to work with XC_PAGE_SIZE granularity since
> > we can't map larger ranges than what has been granted to us.
> >
> > Like with foreign mappings (xen_memory), machines using grants
> > are expected to initialize the xen_grants MR and map it
> > into their address-map accordingly.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > ---
> >  hw/xen/xen-hvm-common.c         |  12 ++-
> >  hw/xen/xen-mapcache.c           | 158 +++++++++++++++++++++++++-------
> >  include/hw/xen/xen-hvm-common.h |   3 +
> >  include/sysemu/xen.h            |   7 ++
> >  4 files changed, 145 insertions(+), 35 deletions(-)
> >
> > diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> > index 0267b88d26..fdec400491 100644
> > --- a/hw/xen/xen-hvm-common.c
> > +++ b/hw/xen/xen-hvm-common.c
> > @@ -10,12 +10,18 @@
> >  #include "hw/boards.h"
> >  #include "hw/xen/arch_hvm.h"
> >
> > -MemoryRegion xen_memory;
> > +MemoryRegion xen_memory, xen_grants;
> >
> > -/* Check for xen memory.  */
> > +/* Check for any kind of xen memory, foreign mappings or grants.  */
> >  bool xen_mr_is_memory(MemoryRegion *mr)
> >  {
> > -    return mr =3D=3D &xen_memory;
> > +    return mr =3D=3D &xen_memory || mr =3D=3D &xen_grants;
> > +}
> > +
> > +/* Check specifically for grants.  */
> > +bool xen_mr_is_grants(MemoryRegion *mr)
> > +{
> > +    return mr =3D=3D &xen_grants;
> >  }
> >
> >  void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion =
*mr,
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index 1b32d0c003..96cd68e28d 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -14,6 +14,7 @@
> >
> >  #include <sys/resource.h>
> >
> > +#include "hw/xen/xen-hvm-common.h"
> >  #include "hw/xen/xen_native.h"
> >  #include "qemu/bitmap.h"
> >
> > @@ -21,6 +22,8 @@
> >  #include "sysemu/xen-mapcache.h"
> >  #include "trace.h"
> >
> > +#include <xenevtchn.h>
> > +#include <xengnttab.h>
> >
> >  #if HOST_LONG_BITS =3D=3D 32
> >  #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
> > @@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
> >      unsigned long *valid_mapping;
> >      uint32_t lock;
> >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > +#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
> >      uint8_t flags;
> >      hwaddr size;
> >
> > @@ -74,6 +78,8 @@ typedef struct MapCache {
> >  } MapCache;
> >
> >  static MapCache *mapcache;
> > +static MapCache *mapcache_grants;
> > +static xengnttab_handle *xen_region_gnttabdev;
> >
> >  static inline void mapcache_lock(MapCache *mc)
> >  {
> > @@ -132,6 +138,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, =
void *opaque)
> >      unsigned long max_mcache_size;
> >      unsigned int bucket_shift;
> >
> > +    xen_region_gnttabdev =3D xengnttab_open(NULL, 0);
> > +    if (xen_region_gnttabdev =3D=3D NULL) {
> > +        error_report("mapcache: Failed to open gnttab device");
> > +        exit(EXIT_FAILURE);
> > +    }
> > +
> >      if (HOST_LONG_BITS =3D=3D 32) {
> >          bucket_shift =3D 16;
> >      } else {
> > @@ -160,6 +172,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, =
void *opaque)
> >      mapcache =3D xen_map_cache_init_single(f, opaque,
> >                                           bucket_shift,
> >                                           max_mcache_size);
> > +
> > +    /*
> > +     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
> > +     * map anything beyond the number of pages granted to us.
> > +     */
> > +    mapcache_grants =3D xen_map_cache_init_single(f, opaque,
> > +                                                XC_PAGE_SHIFT,
> > +                                                max_mcache_size);
> > +
> >      setrlimit(RLIMIT_AS, &rlimit_as);
> >  }
> >
> > @@ -169,17 +190,25 @@ static void xen_remap_bucket(MapCache *mc,
> >                               hwaddr size,
> >                               hwaddr address_index,
> >                               bool dummy,
> > +                             bool grant,
> > +                             bool grant_is_write,
> > +                             hwaddr grant_ref,
> >                               ram_addr_t ram_offset)
>
> Any chance we could pass grant_ref as address_index ?
>

Yes, good catch :-)
grant_ref is already the same as address_index.


> Also instead of grant_is_write we could have a generic is_write that
> applies to both.

Sounds good.

>
> I am not sure about this, but instead of bool grant, we could check on
> address_index using XEN_GRANT_ADDR_OFF? This one might not work.
>

Yeah, this won't work since we're only getting the offset into the
xen_grants memory region.

> I admit that there is no real advantage on these suggestions except to
> consolidate the parameters and make them look a bit more similar in the
> two cases.
>
>
>
> >  {
> >      uint8_t *vaddr_base;
> > -    xen_pfn_t *pfns;
> > +    uint32_t *refs =3D NULL;
> > +    xen_pfn_t *pfns =3D NULL;
> >      int *err;
> >      unsigned int i;
> >      hwaddr nb_pfn =3D size >> XC_PAGE_SHIFT;
> >
> >      trace_xen_remap_bucket(address_index);
> >
> > -    pfns =3D g_new0(xen_pfn_t, nb_pfn);
> > +    if (grant) {
> > +        refs =3D g_new0(uint32_t, nb_pfn);
> > +    } else {
> > +        pfns =3D g_new0(xen_pfn_t, nb_pfn);
> > +    }
> >      err =3D g_new0(int, nb_pfn);
> >
> >      if (entry->vaddr_base !=3D NULL) {
> > @@ -208,21 +237,45 @@ static void xen_remap_bucket(MapCache *mc,
> >      g_free(entry->valid_mapping);
> >      entry->valid_mapping =3D NULL;
> >
> > -    for (i =3D 0; i < nb_pfn; i++) {
> > -        pfns[i] =3D (address_index << (mc->bucket_shift - XC_PAGE_SHIF=
T)) + i;
> > +    if (grant) {
> > +        for (i =3D 0; i < nb_pfn; i++) {
> > +            refs[i] =3D grant_ref + i;
> > +        }
> > +    } else {
> > +        for (i =3D 0; i < nb_pfn; i++) {
> > +            pfns[i] =3D (address_index << (mc->bucket_shift - XC_PAGE_=
SHIFT)) + i;
> > +        }
> >      }
> >
> > -    /*
> > -     * If the caller has requested the mapping at a specific address u=
se
> > -     * MAP_FIXED to make sure it's honored.
> > -     */
> > +    entry->flags &=3D ~XEN_MAPCACHE_ENTRY_GRANT;
> > +
> >      if (!dummy) {
> > -        vaddr_base =3D xenforeignmemory_map2(xen_fmem, xen_domid, vadd=
r,
> > -                                           PROT_READ | PROT_WRITE,
> > -                                           vaddr ? MAP_FIXED : 0,
> > -                                           nb_pfn, pfns, err);
> > +        if (grant) {
> > +            int prot =3D PROT_READ;
> > +
> > +            if (grant_is_write) {
> > +                prot |=3D PROT_WRITE;
> > +            }
> > +
> > +            entry->flags |=3D XEN_MAPCACHE_ENTRY_GRANT;
> > +            assert(vaddr =3D=3D NULL);
> > +            vaddr_base =3D xengnttab_map_domain_grant_refs(xen_region_=
gnttabdev,
> > +                                                         nb_pfn,
> > +                                                         xen_domid, re=
fs,
> > +                                                         prot);
> > +        } else {
> > +            /*
> > +             * If the caller has requested the mapping at a specific a=
ddress use
> > +             * MAP_FIXED to make sure it's honored.
> > +             */
> > +            vaddr_base =3D xenforeignmemory_map2(xen_fmem, xen_domid, =
vaddr,
> > +                                               PROT_READ | PROT_WRITE,
> > +                                               vaddr ? MAP_FIXED : 0,
> > +                                               nb_pfn, pfns, err);
> > +        }
> >          if (vaddr_base =3D=3D NULL) {
> > -            perror("xenforeignmemory_map2");
> > +            perror(grant ? "xengnttab_map_domain_grant_refs"
> > +                           : "xenforeignmemory_map2");
> >              exit(-1);
> >          }
> >      } else {
> > @@ -263,6 +316,7 @@ static void xen_remap_bucket(MapCache *mc,
> >          }
> >      }
> >
> > +    g_free(refs);
> >      g_free(pfns);
> >      g_free(err);
> >  }
> > @@ -270,10 +324,12 @@ static void xen_remap_bucket(MapCache *mc,
> >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> >                                         hwaddr phys_addr, hwaddr size,
> >                                         ram_addr_t ram_offset,
> > -                                       uint8_t lock, bool dma, bool is=
_write)
> > +                                       uint8_t lock, bool dma,
> > +                                       bool grant, bool is_write)
> >  {
> >      MapCacheEntry *entry, *pentry =3D NULL,
> >                    *free_entry =3D NULL, *free_pentry =3D NULL;
> > +    hwaddr grant_ref =3D phys_addr >> XC_PAGE_SHIFT;
> >      hwaddr address_index;
> >      hwaddr address_offset;
> >      hwaddr cache_size =3D size;
> > @@ -342,7 +398,7 @@ tryagain:
> >          entry =3D g_new0(MapCacheEntry, 1);
> >          pentry->next =3D entry;
> >          xen_remap_bucket(mc, entry, NULL, cache_size, address_index, d=
ummy,
> > -                         ram_offset);
> > +                         grant, is_write, grant_ref, ram_offset);
> >      } else if (!entry->lock) {
> >          if (!entry->vaddr_base || entry->paddr_index !=3D address_inde=
x ||
> >                  entry->size !=3D cache_size ||
> > @@ -350,7 +406,7 @@ tryagain:
> >                      test_bit_size >> XC_PAGE_SHIFT,
> >                      entry->valid_mapping)) {
> >              xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy,
> > -                             ram_offset);
> > +                             grant, is_write, grant_ref, ram_offset);
> >          }
> >      }
> >
> > @@ -401,12 +457,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
> >                         uint8_t lock, bool dma,
> >                         bool is_write)
> >  {
> > +    bool grant =3D xen_mr_is_grants(mr);
> > +    MapCache *mc =3D grant ? mapcache_grants : mapcache;
> >      uint8_t *p;
> >
> > -    mapcache_lock(mapcache);
> > -    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_o=
ffset,
> > -                               lock, dma, is_write);
> > -    mapcache_unlock(mapcache);
> > +    if (grant) {
> > +        /*
> > +         * Grants are only supported via address_space_map(). Anything
> > +         * else is considered a user/guest error.
> > +         *
> > +         * QEMU generally doesn't expect these mappings to ever fail, =
so
> > +         * if this happens we report an error message and abort().
> > +         */
> > +        if (!lock) {
> > +            error_report("Trying access a grant reference without mapp=
ing it.");
> > +            abort();
> > +        }
> > +    }
> > +
> > +    mapcache_lock(mc);
> > +    p =3D xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
> > +                               lock, dma, grant, is_write);
> > +    mapcache_unlock(mc);
> >      return p;
> >  }
> >
> > @@ -451,7 +523,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_singl=
e(MapCache *mc, void *ptr)
> >
> >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> >  {
> > -    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
> > +    ram_addr_t addr;
> > +
> > +    addr =3D xen_ram_addr_from_mapcache_single(mapcache, ptr);
> > +    if (addr =3D=3D RAM_ADDR_INVALID) {
> > +        addr =3D xen_ram_addr_from_mapcache_single(mapcache_grants, pt=
r);
> > +    }
> > +
> > +    return addr;
> >  }
> >
> >  static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> > @@ -504,9 +583,14 @@ static void xen_invalidate_map_cache_entry_unlocke=
d(MapCache *mc,
> >      }
> >
> >      ram_block_notify_remove(entry->vaddr_base, entry->size, entry->siz=
e);
> > -    if (munmap(entry->vaddr_base, entry->size) !=3D 0) {
> > -        perror("unmap fails");
> > -        exit(-1);
> > +    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
> > +        xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
> > +                    (entry->size + mc->bucket_size - 1) >> mc->bucket_=
shift);
>
> Am I getting this right that the + mc->bucket_size - 1 is unnecessary
> because the bucket size is PAGE_SIZE and we can only map at page
> granularity?
>

Yes, you're right.
I'll fix this up in the next version.


> Also can we check for return errors?

Yes, I'll add error checking.


>
>
> > +    } else {
> > +        if (munmap(entry->vaddr_base, entry->size) !=3D 0) {
> > +            perror("unmap fails");
> > +            exit(-1);
> > +        }
> >      }
> >      if (pentry) {
> >          pentry->next =3D entry->next;
> > @@ -522,14 +606,24 @@ typedef struct XenMapCacheData {
> >      uint8_t *buffer;
> >  } XenMapCacheData;
> >
> > +static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_=
t *buffer)
> > +{
> > +    mapcache_lock(mc);
> > +    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
> > +    mapcache_unlock(mc);
> > +}
> > +
> > +static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
> > +{
> > +    xen_invalidate_map_cache_entry_single(mapcache, buffer);
> > +    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
> > +}
> > +
> >  static void xen_invalidate_map_cache_entry_bh(void *opaque)
> >  {
> >      XenMapCacheData *data =3D opaque;
> >
> > -    mapcache_lock(mapcache);
> > -    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
> > -    mapcache_unlock(mapcache);
> > -
> > +    xen_invalidate_map_cache_entry_all(data->buffer);
> >      aio_co_wake(data->co);
> >  }
> >
> > @@ -544,9 +638,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_en=
try(uint8_t *buffer)
> >                                  xen_invalidate_map_cache_entry_bh, &da=
ta);
> >          qemu_coroutine_yield();
> >      } else {
> > -        mapcache_lock(mapcache);
> > -        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
> > -        mapcache_unlock(mapcache);
> > +        xen_invalidate_map_cache_entry_all(buffer);
> >      }
> >  }
> >
> > @@ -598,6 +690,7 @@ void xen_invalidate_map_cache(void)
> >      bdrv_drain_all();
> >
> >      xen_invalidate_map_cache_single(mapcache);
> > +    xen_invalidate_map_cache_single(mapcache_grants);
> >  }
> >
> >  static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> > @@ -639,7 +732,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(Ma=
pCache *mc,
> >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> >
> >      xen_remap_bucket(mc, entry, entry->vaddr_base,
> > -                     cache_size, address_index, false, entry->ram_offs=
et);
> > +                     cache_size, address_index, false,
> > +                     false, false, 0, entry->ram_offset);
>
> If I understand correctly, xen_replace_cache_entry_unlocked cannot be
> called on grants because xen_replace_cache_entry_unlocked is always
> called on unlocked entries while grants are always locked. Should we
> have an assert on !entry->lock and/or !(entry->flags & XEN_MAPCACHE_ENTRY=
_GRANT)?
>

Sounds good, I'll add this in the next version as well.

>
>
> >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> >                  test_bit_size >> XC_PAGE_SHIFT,
> >                  entry->valid_mapping)) {
> > diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-c=
ommon.h
> > index 65a51aac2e..3d796235dc 100644
> > --- a/include/hw/xen/xen-hvm-common.h
> > +++ b/include/hw/xen/xen-hvm-common.h
> > @@ -16,6 +16,7 @@
> >  #include <xen/hvm/ioreq.h>
> >
> >  extern MemoryRegion xen_memory;
> > +extern MemoryRegion xen_grants;
> >  extern MemoryListener xen_io_listener;
> >  extern DeviceListener xen_device_listener;
> >
> > @@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
> >      do { } while (0)
> >  #endif
> >
> > +#define XEN_GRANT_ADDR_OFF (1ULL << 63)
> > +
> >  static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, in=
t i)
> >  {
> >      return shared_page->vcpu_ioreq[i].vp_eport;
> > diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> > index dc72f83bcb..19dccf4d71 100644
> > --- a/include/sysemu/xen.h
> > +++ b/include/sysemu/xen.h
> > @@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t si=
ze,
> >                     struct MemoryRegion *mr, Error **errp);
> >
> >  bool xen_mr_is_memory(MemoryRegion *mr);
> > +bool xen_mr_is_grants(MemoryRegion *mr);
> >
> >  #else /* !CONFIG_XEN_IS_POSSIBLE */
> >
> > @@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr=
)
> >      return false;
> >  }
> >
> > +static inline bool xen_mr_is_grants(MemoryRegion *mr)
> > +{
> > +    g_assert_not_reached();
> > +    return false;
> > +}
> > +
> >  #endif /* CONFIG_XEN_IS_POSSIBLE */
> >
> >  #endif
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Thu May 02 20:02:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 20:02:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716183.1118291 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cdH-00028M-8w; Thu, 02 May 2024 20:02:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716183.1118291; Thu, 02 May 2024 20:02:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2cdH-00028F-51; Thu, 02 May 2024 20:02:19 +0000
Received: by outflank-mailman (input) for mailman id 716183;
 Thu, 02 May 2024 20:02:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Qsp=MF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2cdF-000289-8F
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 20:02:17 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id de657a06-08be-11ef-b4bb-af5377834399;
 Thu, 02 May 2024 22:02:14 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5AA6761BEC;
 Thu,  2 May 2024 20:02:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3804C113CC;
 Thu,  2 May 2024 20:02:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de657a06-08be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714680133;
	bh=SZanZoajSudqQOI7FJeCs/pL1/q88SY5w1jmpAUiivU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MZvK7MzojHqLYEy7+Vln2N/sDo3tht2Vgm8l9nKvuduWJKKocxlVBi2uCc43eWzWe
	 /Az7RHJs91nLJ+9+UVFRCZUvifCyL+NV6vDR83unspPzXoxni3yoOYzsehXfSSQdI2
	 6wDsF1ygbV6rc8Hnv+UGzWqAFKINNI5mU/qQhCKhczPL3VpyGUhd+YGUc0rKZk/Esm
	 YieMvxMZvqii91++iWmoTLR0csFH9LbVOUByzNsdGnJPnWxFuMhNsmp79lDqvuOWbX
	 c+LRX5yfxN6xHj93CdmLA17Mqn7TptFyFmvgOLwdMicKckGapD9C6FV36DU60tUYJJ
	 n2xWIGCjyt0lw==
Date: Thu, 2 May 2024 13:02:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, qemu-devel@nongnu.org, 
    jgross@suse.com, "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org, Xenia.Ragiadakou@amd.com
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
In-Reply-To: <CAJy5ezq+wd4T91YPEarea4eUXp_CJftvPYMJT2AzA66DWjy34w@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405021301400.624854@ubuntu-linux-20-04-desktop>
References: <20240430164939.925307-1-edgar.iglesias@gmail.com> <20240430164939.925307-16-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop> <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com>
 <alpine.DEB.2.22.394.2405021144270.624854@ubuntu-linux-20-04-desktop> <CAJy5ezq+wd4T91YPEarea4eUXp_CJftvPYMJT2AzA66DWjy34w@mail.gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1380255118-1714680133=:624854"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1380255118-1714680133=:624854
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> On Thu, May 2, 2024 at 8:53 PM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
> >
> > +Xenia
> >
> > On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > > On Wed, May 1, 2024 at 11:24 PM Stefano Stabellini
> > > <sstabellini@kernel.org> wrote:
> > > >
> > > > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > > > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> > > > >
> > > > > The current mapcache assumes that all memory is mapped
> > > > > in a single RAM MR (the first one with offset 0). Remove
> > > > > this assumption and propagate the offset to the mapcache
> > > > > so it can do reverse mappings (from hostptr -> ram_addr).
> > > > >
> > > > > This is in preparation for adding grant mappings.
> > > > >
> > > > > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > > >
> > > >
> > > > Looking at xen_remap_bucket, it is only using address_index (without
> > > > adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
> > > > understand that address_index already includes the ram_offset.
> > > >
> > > > Meaning that if we want to map foreign mapping at address 0x5000, then
> > > > address_index would be 0x5000, even if ram_offset is 0x1000.
> > > >
> > > > But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> > > > to paddr_index to calculate the physical address. So in that case we
> > > > would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> > > > xen_remap_bucket would have to sum address_index and ram_offset to map
> > > > foreign memory.
> > > >
> > > > So I am a bit confused, did I get it wrong? One more comment below.
> > > >
> > >
> > > Thanks Stefano,
> > >
> > > I think the confusion is that this ram_addr_offset is not related to
> > > guest address-space.
> > > It's a QEMU internal thing and it shouldn't be included in the address
> > > used to map foreign memory.
> > > The mapcache can treat this ram_addr offset like a cookie that we keep
> > > around to be able to do
> > > reverse mappings from host pointers into ram_addr space
> > > (xen_ram_addr_from_mapcache).
> > >
> > > The current mapcache implementation works because we've really only
> > > been using foreign mappings
> > > on RAMBlocks with offset 0. We're also creating RAM's such that the
> > > offset into the RAM is also
> > > the guest physical address, for x86 this is natural since RAM starts
> > > at zero (for lowmem) but for
> > > ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size) to
> > > make this assumption true. Anyway, In this series I'm not addressing
> > > this second assumption.
> >
> > Let's see if I understand correctly.
> >
> > The ram_addr space is an internal QEMU address space which is different
> > from the guest physical address space and thus cannot and should not be
> > used to do foreign mappings (foreign mapping hypercalls take a guest
> > physical or a real physical address to map). Is that correct?
> >
> > If so, then I understand.
> >
> 
> Yes, that matches my understanding.
> 
> >
> >
> > > There's a second call in physmem.c to xen_map_cache using the
> > > block->offset as an address.
> > > I was considering removing that second call since I can't see how it can work
> > > (except perhaps in some specific use-case by luck?). Anyway, for now
> > > I've left it unmodified.
> >
> > Yes, that code was written with the assumption that block->offset is an
> > offset in the guest physical address space and could be used as a guest
> > physical address. Actually, you might have spotted a real bug.
> >
> > The intent was for smaller regions (not the bit RAM region, things like
> > a ROM region for instance) we could map them in full. So here we were
> > trying to map the whole thing from start to finish using block->offset
> > as start.
> >
> >
> > > > > ---
> > > > >  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
> > > > >  include/sysemu/xen-mapcache.h |  2 ++
> > > > >  system/physmem.c              |  8 ++++----
> > > > >  3 files changed, 24 insertions(+), 11 deletions(-)
> > > > >
> > > > > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > > > > index 09b5f36d9c..1b32d0c003 100644
> > > > > --- a/hw/xen/xen-mapcache.c
> > > > > +++ b/hw/xen/xen-mapcache.c
> > > > > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> > > > >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > > > >      uint8_t flags;
> > > > >      hwaddr size;
> > > > > +
> > > > > +    /* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr).  */
> > > > > +    ram_addr_t ram_offset;
> > > > >      struct MapCacheEntry *next;
> > > > >  } MapCacheEntry;
> > > > >
> > > > > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> > > > >                               void *vaddr,
> > > > >                               hwaddr size,
> > > > >                               hwaddr address_index,
> > > > > -                             bool dummy)
> > > > > +                             bool dummy,
> > > > > +                             ram_addr_t ram_offset)
> > > > >  {
> > > > >      uint8_t *vaddr_base;
> > > > >      xen_pfn_t *pfns;
> > > > > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > > >      entry->size = size;
> > > > >      entry->valid_mapping = g_new0(unsigned long,
> > > > >                                    BITS_TO_LONGS(size >> XC_PAGE_SHIFT));
> > > > > +    entry->ram_offset = ram_offset;
> > > > >
> > > > >      if (dummy) {
> > > > >          entry->flags |= XEN_MAPCACHE_ENTRY_DUMMY;
> > > > > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > > >
> > > > >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > > > >                                         hwaddr phys_addr, hwaddr size,
> > > > > +                                       ram_addr_t ram_offset,
> > > > >                                         uint8_t lock, bool dma, bool is_write)
> > > > >  {
> > > > >      MapCacheEntry *entry, *pentry = NULL,
> > > > > @@ -335,14 +341,16 @@ tryagain:
> > > > >      if (!entry) {
> > > > >          entry = g_new0(MapCacheEntry, 1);
> > > > >          pentry->next = entry;
> > > > > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> > > > > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> > > > > +                         ram_offset);
> > > > >      } else if (!entry->lock) {
> > > > >          if (!entry->vaddr_base || entry->paddr_index != address_index ||
> > > > >                  entry->size != cache_size ||
> > > > >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> > > > >                      test_bit_size >> XC_PAGE_SHIFT,
> > > > >                      entry->valid_mapping)) {
> > > > > -            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> > > > > +            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> > > > > +                             ram_offset);
> > > > >          }
> > > > >      }
> > > > >
> > > > > @@ -389,13 +397,15 @@ tryagain:
> > > > >
> > > > >  uint8_t *xen_map_cache(MemoryRegion *mr,
> > > > >                         hwaddr phys_addr, hwaddr size,
> > > > > +                       ram_addr_t ram_addr_offset,
> > > > >                         uint8_t lock, bool dma,
> > > > >                         bool is_write)
> > > > >  {
> > > > >      uint8_t *p;
> > > > >
> > > > >      mapcache_lock(mapcache);
> > > > > -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
> > > > > +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> > > > > +                               lock, dma, is_write);
> > > > >      mapcache_unlock(mapcache);
> > > > >      return p;
> > > > >  }
> > > > > @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
> > > > >          raddr = RAM_ADDR_INVALID;
> > > > >      } else {
> > > > >          raddr = (reventry->paddr_index << mc->bucket_shift) +
> > > > > -             ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
> > > > > +             ((unsigned long) ptr - (unsigned long) entry->vaddr_base) +
> > > > > +             entry->ram_offset;
> > > > >      }
> > > > >      mapcache_unlock(mc);
> > > > >      return raddr;
> > > > > @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> > > > >
> > > > >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> > > > >
> > > > > -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> > > > > -                     cache_size, address_index, false);
> > > > > +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> > > > > +                     cache_size, address_index, false, entry->ram_offset);
> > > > >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> > > > >                  test_bit_size >> XC_PAGE_SHIFT,
> > > > >                  entry->valid_mapping)) {
> > > > > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> > > > > index 1ec9e66752..b5e3ea1bc0 100644
> > > > > --- a/include/sysemu/xen-mapcache.h
> > > > > +++ b/include/sysemu/xen-mapcache.h
> > > > > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
> > > > >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > > > >                          void *opaque);
> > > > >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
> > > > > +                       ram_addr_t ram_addr_offset,
> > > > >                         uint8_t lock, bool dma,
> > > > >                         bool is_write);
> > > > >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > > > > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > > > >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> > > > >                                       hwaddr phys_addr,
> > > > >                                       hwaddr size,
> > > > > +                                     ram_addr_t ram_addr_offset,
> > > > >                                       uint8_t lock,
> > > > >                                       bool dma,
> > > > >                                       bool is_write)
> > > > > diff --git a/system/physmem.c b/system/physmem.c
> > > > > index 1a5ffcba2a..5b16eeccca 100644
> > > > > --- a/system/physmem.c
> > > > > +++ b/system/physmem.c
> > > > > @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
> > > > >           * In that case just map the requested area.
> > > > >           */
> > > > >          if (xen_mr_is_memory(block->mr)) {
> > > > > -            return xen_map_cache(block->mr, addr, len, lock, lock,
> > > > > -                                 is_write);
> > > > > +            return xen_map_cache(block->mr, addr, len, block->offset,
> > > > > +                                 lock, lock, is_write);
> > > >
> > > > Have you considered not tracking offset and address separately and
> > > > simply do this?
> > > >
> > > >             return xen_map_cache(block->mr, addr + block->offset, len,
> > > >                                  lock, lock, is_write);
> > > >
> > >
> > > Unfortunately this won't work since block->offset is not related to where this
> > > ram is mapped in guest address-space. In the case of grant's, we'd get the
> > > wrong grant ref. See my previous comment.
> >
> > OK, this code below (the second xen_map_cache call passing block->offset
> > as start address) was wrong before this patch. Can we fix it before
> > changing it further with this patch? I worry about making things even
> > worse.
> >
> 
> I'll dig around and see if we can find something that explains more.
> There's some older code that implements some sort of address-translation
> for x86 between ram_addr space and guest physical addresses but
> that code is turned off with newer Xen versions (disabled in my build).
> 
> https://github.com/qemu/qemu/blob/master/hw/xen/xen-mapcache.c#L330
> https://github.com/qemu/qemu/blob/master/hw/i386/xen/xen-hvm.c#L193

I don't have any more insights but I think Xenia might have a better
idea as she has fixed bugs related to this to get virtio-gpu to work.
--8323329-1380255118-1714680133=:624854--


From xen-devel-bounces@lists.xenproject.org Thu May 02 21:26:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 21:26:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716194.1118301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2dwn-0002QN-9K; Thu, 02 May 2024 21:26:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716194.1118301; Thu, 02 May 2024 21:26:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2dwn-0002QG-5p; Thu, 02 May 2024 21:26:33 +0000
Received: by outflank-mailman (input) for mailman id 716194;
 Thu, 02 May 2024 21:26:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/qc=MF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2dwm-0002QA-QY
 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 21:26:32 +0000
Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com
 [2607:f8b0:4864:20::82f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a4717131-08ca-11ef-909c-e314d9c70b13;
 Thu, 02 May 2024 23:26:31 +0200 (CEST)
Received: by mail-qt1-x82f.google.com with SMTP id
 d75a77b69052e-43b0b09d012so22998271cf.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 May 2024 14:26:31 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 fe13-20020a05622a4d4d00b0043842dc662esm847101qtb.4.2024.05.02.14.26.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 May 2024 14:26:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4717131-08ca-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714685190; x=1715289990; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=puzZpPNPRb+k9eiyXOVXkia9Jhh8mI5WHhsTAdJSu5A=;
        b=TwNK4CD5j+RWEXZfzo/tWy6Djgdibeh3GQp3+s2vAf6Gpa1S3ptwEMZbBh0FxbEdj8
         CyCUvCs5mGmOHRMX9kIBtRY6XBpvPACRaq/1Sf+9gotr6hm7UwSP7zOBNqEtCRL5Kb2g
         h9NWHwUHxxcYj56zFNDAV78o5vkLAxXKZlBcE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714685190; x=1715289990;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=puzZpPNPRb+k9eiyXOVXkia9Jhh8mI5WHhsTAdJSu5A=;
        b=LANAZ7ObW5dWarR5mplOTmUu2imTvCOKWU6qGq6KHYSfT0y60ZHksexhizbUxuh6+B
         5gfTyqWlPk/Ww1gebpvgnXW2nLH9RhUFfJk9KQvpAbNPAJ1lVHJRpiCJDEjtBxUFT+zr
         mh+BlhBmzipMt5pg6r1hg8W74U3wbpCQ18E3+qg4EtkhnRrZ59fluWsgt9fD+GNYndbh
         GicBOWpEkz2uqsVYWclrRcu2Q8VLm7hiOU7fj+v/O4Z966S2gZXC4YmP4qVlNHpZVv9T
         rQ5WfskjOR8ZYprkx2sWrSCqK6i58/pqXQoQkbCYyQ6x+VIdJpQuLyiIr28gB/L8yz5M
         lslA==
X-Forwarded-Encrypted: i=1; AJvYcCVa3bp/wfxc3eX8n33vdOP2KU9cYSdHQL2SeeTXaykkB/2mZSta++r7JXpb1NYHHUqF7MYq9Dn2s4xF9YUVdwbtyuW5sCK3XSQ67ZqpHJw=
X-Gm-Message-State: AOJu0YynQme3v2XgQ90fLyo/ZH6gjm/oEpE6voxoBuWzWDVBO1CzGoRB
	+SPD+vP4EU1otOycSmrF0Z+axwLu0gaAzRbIqQWZxOtyh4UDhCXNpOmtFUbnzFmlr0BhKXDcR0h
	w
X-Google-Smtp-Source: AGHT+IEQnOofbtvgxyovuEHj+AaVfJaKZ9Mk/cnvTyaAcKk0zU/yIbTZDKKFlw4TFIx5ZQl2xFleJQ==
X-Received: by 2002:ac8:5f0b:0:b0:43b:7:7d81 with SMTP id x11-20020ac85f0b000000b0043b00077d81mr847617qta.66.1714685190221;
        Thu, 02 May 2024 14:26:30 -0700 (PDT)
Message-ID: <e2eac84f-eb83-4658-9b48-6315f6418ac0@citrix.com>
Date: Thu, 2 May 2024 22:26:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/tests: don't let test-xenstore write nodes
 exceeding default size
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>
References: <20240502132136.532-1-jgross@suse.com>
 <3c5ea8e3-87ce-4048-a3f1-7a963706f5b2@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <3c5ea8e3-87ce-4048-a3f1-7a963706f5b2@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 2:25 pm, Andrew Cooper wrote:
> On 02/05/2024 2:21 pm, Juergen Gross wrote:
>> Today test-xenstore will write nodes with 3000 bytes node data. This
>> size is exceeding the default quota for the allowed node size. While
>> working in dom0 with C-xenstored, OCAML-xenstored does not like that.
>>
>> Use a size of 2000 instead, which is lower than the allowed default
>> node size of 2048.
>>
>> Fixes: 3afc5e4a5b75 ("tools/tests: add xenstore testing framework")
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
> Thanks for figuring this out.  I'll give this and the return code fix a
> spin through XenRT and check there's nothing else unexpected hiding.

All seems happy.  I'll put these in in due course.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 02 22:01:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 May 2024 22:01:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716199.1118311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2eUI-0000Nz-QS; Thu, 02 May 2024 22:01:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716199.1118311; Thu, 02 May 2024 22:01:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2eUI-0000Ns-NX; Thu, 02 May 2024 22:01:10 +0000
Received: by outflank-mailman (input) for mailman id 716199;
 Thu, 02 May 2024 22:01:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2eUI-0000Ni-5h; Thu, 02 May 2024 22:01:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2eUH-0004W5-Ur; Thu, 02 May 2024 22:01:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2eUH-0003ui-F6; Thu, 02 May 2024 22:01:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2eUH-0008Gr-Em; Thu, 02 May 2024 22:01:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ad61JDsn2a0PKz3bAXdqZFLi5AZeoyRb00ArUdkmZoo=; b=ABTvQaXWIIZrlvzrdEQgSWW3yA
	7BGP1xyeQ6cvez2+H13pAX2Q3sBuL5jlMckDglLC9hkhoz2puxlcycFqQnrm79Ai3eNevCpZ/W4AX
	86Px/pKT6ISNI5H2gmFGBIPFhWZlJotP3p6tM3/rKA+mHNA8fMTtuz47GMdIp2yGvIJs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185901-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 185901: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=909ba1f1b4146de529469910c1bd0b1248964536
X-Osstest-Versions-That:
    linux=dcbc050cb0d304c3427d6583384eebcaf0e3caee
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 May 2024 22:01:09 +0000

flight 185901 linux-6.1 real [real]
flight 185903 linux-6.1 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185901/
http://logs.test-lab.xenproject.org/osstest/logs/185903/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185903-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185871
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185871
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185871
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185871
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185871
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185871
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                909ba1f1b4146de529469910c1bd0b1248964536
baseline version:
 linux                dcbc050cb0d304c3427d6583384eebcaf0e3caee

Last test of basis   185871  2024-04-29 20:44:32 Z    3 days
Testing same since   185901  2024-05-02 15:12:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Li <adamli@os.amperecomputing.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Akhil R <akhilrajeev@nvidia.com>
  Aleksandr Loktionov <aleksandr.loktionov@intel.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Zubkov <green@qrator.net>
  Alexey Brodkin <abrodkin@synopsys.com>
  Alexey Brodkin <Alexey.Brodkin@synopsys.com>
  Alice Ryhl <aliceryhl@google.com>
  Andreas Taschner <andreas.taschner@suse.com>
  Andrei Simion <andrei.simion@microchip.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Ryabinin <ryabinin.a.a@gmail.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Arınç ÜNAL <arinc.unal@arinc9.com>
  Aswin Unnikrishnan <aswinunni01@gmail.com>
  Avraham Stern <avraham.stern@intel.com>
  Baoquan He <bhe@redhat.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Baruch Siach <baruch@tkos.co.il>
  Borislav Petkov (AMD) <bp@alien8.de>
  Breno Leitao <leitao@debian.org>
  Chen-Yu Tsai <wenst@chromium.org>
  Chun-Yi Lee <jlee@suse.com>
  Claudiu Beznea <claudiu.beznea@tuxon.dev>
  Conor Dooley <conor.dooley@microchip.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Bauer <mail@david-bauer.net>
  David Kaplan <david.kaplan@amd.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Dragan Simic <dsimic@manjaro.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Ed Trexel <ed.trexel@hp.com>
  Eric Dumazet <edumazet@google.com>
  Erwan Velu <e.velu@criteo.com>
  Fenghua Yu <fenghua.yu@intel.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guanrui Huang <guanrui.huang@linux.alibaba.com>
  Gustavo A. R. Silva <gustavoars@kernel.org>
  Hangbin Liu <liuhangbin@gmail.com>
  Hans de Goede <hdegoede@redhat.com>
  Harshit Mogalapalli <harshit.m.mogalapalli@oracle.com>
  Heiko Stuebner <heiko@sntech.de>
  Heiner Kallweit <hkallweit1@gmail.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Hyunwoo Kim <v4bel@theori.io>
  Ido Schimmel <idosch@nvidia.com>
  Ikjoon Jang <ikjn@chromium.org>
  Iskander Amara <iskander.amara@theobroma-systems.com>
  Ismael Luceno <iluceno@suse.de>
  Jakub Kicinski <kuba@kernel.org>
  Jarred White <jarredwhite@linux.microsoft.com>
  Jason Reeder <jreeder@ti.com>
  Jiantao Shan <shanjiantao@loongson.cn>
  Jiri Kosina <jkosina@suse.com>
  Johan Hovold <johan+linaro@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Johannes Thumshirn <Johannes.thumshirn@wdc.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Julian Anastasov <ja@ssi.bg>
  Kees Cook <keescook@chromium.org>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Konstantin Ovsepian <ovs@ovs.to>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Larry Finger <Larry.Finger@lwfinger.net>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Lucas Stach <l.stach@pengutronix.de>
  Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
  Mantas Pucka <mantas@8devices.com>
  Marcel Ziswiler <marcel.ziswiler@toradex.com>
  Marek Vasut <marex@denx.de>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Chan <michael.chan@broadcom.com>
  Michal Tomek <mtdev79b@gmail.com>
  Miguel Ojeda <ojeda@kernel.org>
  Mikhail Kobuk <m.kobuk@ispras.ru>
  Mineri Bhange <minerix.bhange@intel.com> (A Contingent Worker at Intel)
  Miquel Raynal <miquel.raynal@bootlin.com>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Mukul Joshi <mukul.joshi@amd.com>
  Nam Cao <namcao@linutronix.de>
  Nathan Chancellor <nathan@kernel.org>
  Nicolas Ferre <nicolas.ferre@microchip.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Palmer Dabbelt <palmer@rivosinc.com>
  Paolo Abeni <pabeni@redhat.com>
  Pascal Ernster <git@hardfalcon.net>
  Paul Geurts <paul_geurts@live.nl>
  Paulo Alcantara (Red Hat) <pc@manguebit.com>
  Paulo Alcantara <pc@manguebit.com>
  Pavel Machek (CIP) <pavel@denx.de>
  Peter Münster <pm@a16n.net>
  Petr Machata <petrm@nvidia.com>
  Pin-yen Lin <treapking@chromium.org>
  Quentin Schulz <quentin.schulz@theobroma-systems.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rafał Miłecki <rafal@milecki.pl>
  Rahul Rameshbabu <rrameshbabu@nvidia.com>
  Randy Dunlap <rdunlap@infradead.org>
  Randy Dunlap <rdunlap@infradead.org> # build-tested
  Ravi Gunasekaran <r-gunasekaran@ti.com>
  Richard Leitner <richard.leitner@skidata.com>
  Richard Zhu <hongxing.zhu@nxp.com>
  Robert Ganzynkowicz <robert.ganzynkowicz@intel.com>
  Ron Economos <re@w6rz.net>
  Ronnie Sahlberg <lsahlber@redhat.com>
  Salvatore Bonaccorso <carnil@debian.org>
  Samuel Holland <samuel.holland@sifive.com>
  Sasha Levin <sashal@kernel.org>
  Sean Anderson <sean.anderson@linux.dev>
  Sean Christopherson <seanjc@google.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  SeongJae Park <sj@kernel.org>
  Shuah Khan <skhan@linuxfoundation.org>
  Sindhu Devale <sindhu.devale@intel.com>
  Soheil Hassas Yeganeh <soheil@google.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Steve French <stfrench@microsoft.com>
  Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
  Takayuki Nagata <tnagata@redhat.com>
  Terrence Xu <terrence.xu@intel.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Zimmermann <tzimmermann@suse.de>
  Tom Talpey <tom@talpey.com>
  Tony Brelinski <tony.brelinski@intel.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Vaclav Svoboda <svoboda@neng.cz>
  Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
  Vikas Gupta <vikas.gupta@broadcom.com>
  Vineet Gupta <vgupta@kernel.org>
  Vinod Koul <vkoul@kernel.org>
  WangYuli <wangyuli@uniontech.com>
  Weiyi Lu <weiyi.lu@mediatek.com>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Wren Turkal <wt@penguintechs.org>
  Xiubo Li <xiubli@redhat.com>
  Yaraslau Furman <yaro330@gmail.com>
  Yick Xie <yick.xie@gmail.com>
  Zhang Lixu <lixu.zhang@intel.com>
  Ильфат Гаптрахманов <i.gaptrakhmanov@rosalinux.ru>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   dcbc050cb0d3..909ba1f1b414  909ba1f1b4146de529469910c1bd0b1248964536 -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Fri May 03 02:27:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 02:27:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716211.1118322 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ids-0008IZ-Cx; Fri, 03 May 2024 02:27:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716211.1118322; Fri, 03 May 2024 02:27:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ids-0008IS-7u; Fri, 03 May 2024 02:27:20 +0000
Received: by outflank-mailman (input) for mailman id 716211;
 Fri, 03 May 2024 02:27:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2idr-0008II-Oa; Fri, 03 May 2024 02:27:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2idp-00010W-W6; Fri, 03 May 2024 02:27:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2idp-00023M-HM; Fri, 03 May 2024 02:27:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2idp-0007J1-Gw; Fri, 03 May 2024 02:27:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=02WJRYHn0Kj0V9rI4BmR3u/h2gdAEI1O68vkfxPLfcg=; b=5YsrmCYkWXlsCXl2w0Ir6iLsjg
	fEHHNgRrO3eXjD6COzBCcvDkOc3891PyLIaTHqvc8VZdcj4QVOKGlUXterVtvTNjvqWs44tgMr0Vb
	0d90+kZHpIMZPP46lYrp5AZpLYEuBhqF2mDAMC8m+WIzoIyLUbSDwU9tHzkaKMetItfw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185904-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185904: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=248aa153f65866f46b5370ac2ef7dfaf3af72480
X-Osstest-Versions-That:
    ovmf=fecf55a66a1cf908c2f906bedb79fe2e8362d50f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 02:27:17 +0000

flight 185904 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185904/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 248aa153f65866f46b5370ac2ef7dfaf3af72480
baseline version:
 ovmf                 fecf55a66a1cf908c2f906bedb79fe2e8362d50f

Last test of basis   185900  2024-05-02 13:14:46 Z    0 days
Testing same since   185904  2024-05-03 00:44:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Duggapu Chinni B <chinni.b.duggapu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   fecf55a66a..248aa153f6  248aa153f65866f46b5370ac2ef7dfaf3af72480 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 03 02:40:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 02:40:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716219.1118331 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2iqK-0002UJ-Er; Fri, 03 May 2024 02:40:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716219.1118331; Fri, 03 May 2024 02:40:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2iqK-0002UC-BL; Fri, 03 May 2024 02:40:12 +0000
Received: by outflank-mailman (input) for mailman id 716219;
 Fri, 03 May 2024 02:40:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2iqI-0002Tz-Gq; Fri, 03 May 2024 02:40:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2iqI-0001C7-2j; Fri, 03 May 2024 02:40:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2iqH-0002OB-PM; Fri, 03 May 2024 02:40:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2iqH-0004Xs-Oz; Fri, 03 May 2024 02:40:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MGkf7iCs/7xLRBNUtBZE849XRulFqc45eIL/KCM4/q0=; b=qLY8lFSyBFIU+Gw4FKNXCCvIGv
	3+97groK5R6fO0aM+8rkR8HyrdWZZmuMlcxWijyheU7iSVcR5A8wn1thF3MNP8TmVaCpGXPAheirn
	QQuzc32C/N6uNCdaTbIPu+yOJY7aDcAOkVAo58xCtA6zB/BojVUSdhfSB9d+t33h48F8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185902-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185902: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-pvshim:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=49a73b1652c58ef2a81776a12ad9ac0795f38de0
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 02:40:09 +0000

flight 185902 linux-linus real [real]
flight 185905 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185902/
http://logs.test-lab.xenproject.org/osstest/logs/185905/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1   8 xen-boot                 fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-pvshim 20 guest-localmigrate/x10 fail pass in 185905-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185905-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-libvirt      8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                49a73b1652c58ef2a81776a12ad9ac0795f38de0
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    3 days
Failing since        185888  2024-04-30 20:31:50 Z    2 days    5 attempts
Testing same since   185902  2024-05-02 16:13:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Potapenko <glider@google.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Protopopov <aspsk@isovalent.com>
  Arnd Bergmann <arnd@arndb.de>
  Asbjørn Sloth Tønnesen <ast@fiberby.net>
  Audra Mitchell <audra@redhat.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Björn Töpel <bjorn@kernel.org>
  Björn Töpel <bjorn@rivosinc.com>
  Bui Quang Minh <minhquangbui99@gmail.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Christoph Paasch <cpaasch@apple.com>
  Chuck Lever <chuck.lever@oracle.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Borkmann <daniel@iogearbox.net>
  David Bauer <mail@david-bauer.net>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  Dima Ruinskiy <dima.ruinskiy@intel.com>
  Donald Hunter <donald.hunter@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guillaume Nault <gnault@redhat.com>
  Hans de Goede <hdegoede@redhat.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Ivan Babrou <ivan@cloudflare.com>
  Jakub Kicinski <kuba@kernel.org>
  James Chapman <jchapman@katalix.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  Jason Xing <kernelxing@tencent.com>
  Jeffrey Altman <jaltman@auristor.com>
  Jesper Dangaard Brouer <hawk@kernel.org>
  John Garry <john.g.garry@oracle.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Marc Dionne <marc.dionne@auristor.com>
  Marek Behún <kabel@kernel.org>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <martin.lau@kernel.org>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Abeni <pabeni@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Puranjay Mohan <puranjay12@gmail.com>
  Puranjay Mohan <puranjay@kernel.org>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Richard Gobert <richardbgobert@gmail.com>
  Sasha Neftin <sasha.neftin@intel.com>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislav Fomichev <sdf@google.com>
  Sven Schnelle <svens@linux.ibm.com>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tejun Heo <tj@kernel.org>
  Thanassis Avgerinos <thanassis.avgerinos@gmail.com>
  Toke Høiland-Jørgensen <toke@redhat.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Vitaly Lifshits <vitaly.lifshits@intel.com>
  Xin Long <lucien.xin@gmail.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Kuohai <xukuohai@huawei.com>
  Yunsheng Lin <linyunsheng@huawei.com>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3466 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri May 03 07:10:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 07:10:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716246.1118340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2n3C-0001NC-N9; Fri, 03 May 2024 07:09:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716246.1118340; Fri, 03 May 2024 07:09:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2n3C-0001N5-KD; Fri, 03 May 2024 07:09:46 +0000
Received: by outflank-mailman (input) for mailman id 716246;
 Fri, 03 May 2024 07:09:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o9xE=MG=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s2n3B-0001Mz-Tc
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 07:09:45 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1d90164f-091c-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 09:09:43 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id B8A774EE0738;
 Fri,  3 May 2024 09:09:42 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d90164f-091c-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Fri, 03 May 2024 09:09:42 +0200
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Tamas K Lengyel <tamas@tklengyel.com>, Alexandru
 Isaila <aisaila@bitdefender.com>, Petre Pircalabu
 <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org, julien@xen.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, volodymyr_babchuk@epam.com
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
Reply-To: alessandro.zucchelli@bugseng.com
Mail-Reply-To: alessandro.zucchelli@bugseng.com
In-Reply-To: <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com>
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com>
Message-ID: <834460aca34e6167a3c4620616c145b8@bugseng.com>
X-Sender: alessandro.zucchelli@bugseng.com
Organization: BUGSENG Srl
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-04-29 17:58, Jan Beulich wrote:
> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
>> allowing asm/mem_access.h to be included in all ARM build 
>> configurations.
>> This is to address the violation of MISRA C: 2012 Rule 8.4 which 
>> states:
>> "A compatible declaration shall be visible when an object or function
>> with external linkage is defined". Functions p2m_mem_access_check
>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is not
>> defined in ARM builds don't have visible declarations in the file
>> containing their definitions.
>> 
>> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
>> ---
>>  xen/include/xen/mem_access.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/xen/include/xen/mem_access.h 
>> b/xen/include/xen/mem_access.h
>> index 87d93b31f6..ec0630677d 100644
>> --- a/xen/include/xen/mem_access.h
>> +++ b/xen/include/xen/mem_access.h
>> @@ -33,7 +33,7 @@
>>   */
>>  struct vm_event_st;
>> 
>> -#ifdef CONFIG_MEM_ACCESS
>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
>>  #include <asm/mem_access.h>
>>  #endif
> 
> This doesn't look quite right. If Arm supports mem-access, why would it
> not set MEM_ACCESS=y? Whereas if it's only stubs that Arm supplies, 
> then
> those would better move here, thus eliminating the need for a per-arch
> stub header (see what was e.g. done for numa.h). This way RISC-V and 
> PPC
> (and whatever is to come) would then be taken care of as well.
> 
ARM does support mem-access, so I don't think this is akin to the 
changes done to handle numa.h.
ARM also allows users to set MEM_ACCESS=n (e.g. 
xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however, 
the implementation file mem_access.c is compiled unconditionally in 
ARM's makefile, hence why the violation was spotted.
This is a bit unusual, so I was also hoping to get some feedback from 
mem-access maintainers as to why this discrepancy from x86 exists. I 
probably should have also included some ARM maintainers as well, so I'm 
going to loop them in now.

An alternative option I think is to make the compilation of arm's 
mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and 
common).

-- 
Alessandro Zucchelli, B.Sc.

Software Engineer, BUGSENG (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 03 07:29:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 07:29:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716258.1118351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2nM9-0004H2-8l; Fri, 03 May 2024 07:29:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716258.1118351; Fri, 03 May 2024 07:29:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2nM9-0004Gv-5I; Fri, 03 May 2024 07:29:21 +0000
Received: by outflank-mailman (input) for mailman id 716258;
 Fri, 03 May 2024 07:29:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GLdc=MG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s2nM7-0004Gp-M1
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 07:29:19 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d98a99f8-091e-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 09:29:17 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 125B44EE0738;
 Fri,  3 May 2024 09:29:17 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d98a99f8-091e-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Fri, 03 May 2024 09:29:17 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 bertrand.marquis@arm.com, julien@xen.org, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH v2 1/3] drivers: char: address violation of MISRA C
 Rule 20.7
In-Reply-To: <alpine.DEB.2.22.394.2405011257290.497719@ubuntu-linux-20-04-desktop>
References: <cover.1714487169.git.nicola.vetrini@bugseng.com>
 <3ae8aa13a5562440cf60e4b30f713e5b2f73c577.1714487169.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405011257290.497719@ubuntu-linux-20-04-desktop>
Message-ID: <25f8aec2da7e963f80c12e23fc5f725a@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-01 21:57, Stefano Stabellini wrote:
> On Tue, 30 Apr 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that 
>> all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>> 
>> No functional chage.
>> 
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> 
>> ---
>> Changes in v2:
>> - drop excess parentheses from val parameter.
>> ---
>>  xen/drivers/char/omap-uart.c | 5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>> 
>> diff --git a/xen/drivers/char/omap-uart.c 
>> b/xen/drivers/char/omap-uart.c
>> index 03b5b66e7acb..e0128225f927 100644
>> --- a/xen/drivers/char/omap-uart.c
>> +++ b/xen/drivers/char/omap-uart.c
>> @@ -48,8 +48,9 @@
>>  /* System configuration register */
>>  #define UART_OMAP_SYSC_DEF_CONF   0x0d   /* autoidle mode, wakeup is 
>> enabled */
>> 
>> -#define omap_read(uart, off)       readl((uart)->regs + 
>> (off<<REG_SHIFT))
>> -#define omap_write(uart, off, val) writel((val), (uart)->regs + 
>> (off<<REG_SHIFT))
>> +#define omap_read(uart, off)       readl((uart)->regs + ((off) << 
>> REG_SHIFT))
>> +#define omap_write(uart, off, val) writel(val, (uart)->regs + \
>> +                                               ((off) << REG_SHIFT))
> 
> the alignment looks off but could be fixed on commit
> 

Can you clarify what you mean here? I aligned readl and writeln and the 
operands in writel to avoid the line being too long.

Thanks,

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 03 07:45:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 07:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716265.1118361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2nbn-0007Oe-N3; Fri, 03 May 2024 07:45:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716265.1118361; Fri, 03 May 2024 07:45:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2nbn-0007OX-Ix; Fri, 03 May 2024 07:45:31 +0000
Received: by outflank-mailman (input) for mailman id 716265;
 Fri, 03 May 2024 07:45:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1aGk=MG=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2nbm-0007OR-8J
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 07:45:30 +0000
Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com
 [2607:f8b0:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1bcd7789-0921-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 09:45:28 +0200 (CEST)
Received: by mail-ot1-x333.google.com with SMTP id
 46e09a7af769-6eb93ec8804so4935563a34.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 00:45:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bcd7789-0921-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714722327; x=1715327127; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=lNJa2Jm70diUip67ciyH7ehDCuPj0ZhHiJYE6JP/oGY=;
        b=q79r7L657hxNFSOo+j5ibDf2+4cx37VdRGwwcDlHHSmfl1A1iyxvt8eyCesQP8GiEB
         sIiMLW8xWv3UOaNeqtmd7NJ2QtskZi4pICTTDIzqtNmqVUemLhNjZh+QWbxQFeZxdvhs
         De55qa2AE8I4RoEt87N6yaXjA2tXwrG7R3SNT7b2SwOwiouX8WRqtzi1qs6cRSqpHAA0
         jwt9sepWV1V4N27NeY6aFIrtz13+I7jTwygO5rCMFzGWjIUCfm0FYMlnifQHghviedSQ
         i8xCge9SUDFqTxXLpl/16EcsgB+Kx0/AIzvO0NwR5PWRr+27Irww3FEWCqkFfxeNxH73
         nWdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714722327; x=1715327127;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lNJa2Jm70diUip67ciyH7ehDCuPj0ZhHiJYE6JP/oGY=;
        b=p5xzQWU+bwhiffPw65aXe0bkA0+rDOS6s60a4oFfoOemL5QMeqw7lsCk3dn3rTIrfk
         JQVkG3cIxxS99894LU3b6EVy1jvy4R6GZ1VM8Z5K4lqmStU0lsl0ldXlWo9DHe2onvaU
         9QcxXZZEIbKvFIABOX0MQ38MHwRHcFXdQ+5ghXi0nOlsHuVdVpzvxN9N2pK9zMiccmYE
         mNTKVx/rXvpNS+oxR5BDkaLdVA0iWYrc4qzVLW3iuphbrXZhcFTtmztVHNrBOJOQLA5B
         lKAxMrDWHmSUwsjgzirD6oOycZktgj0PjVE1XFD3/hmtn5yrUKkl6+g2NxkYy/dYpiTT
         aqGw==
X-Gm-Message-State: AOJu0Yx/ndFHopxMWORQfcQuX+dD85sZGLtBBcASRRzxrH5TNiuwFlEz
	yTeagJrG51G137laR8TuIkiSf1MZkfAtIVZUP47SNzardk0B8/u1k1mB5/vZmHnkeIFeZ+CctOU
	N8ckCG+rSybOADCRo8C48oL+ACpdDh8KCccxn9Q==
X-Google-Smtp-Source: AGHT+IGpzQm1nxrYlFMaLissBILcV7ybWqVSQw3QdxEceqh1ZDtbiSUCXQ8ndgJRMogkMTxUm/raAUl4o4olr/7grjU=
X-Received: by 2002:a05:6830:91b:b0:6ed:d4bc:1b72 with SMTP id
 v27-20020a056830091b00b006edd4bc1b72mr2962404ott.5.1714722327163; Fri, 03 May
 2024 00:45:27 -0700 (PDT)
MIME-Version: 1.0
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Fri, 3 May 2024 09:45:15 +0200
Message-ID: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
Subject: Referencing domain struct from interrupt handler
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Jan Beulich <jbeulich@suse.com>, George Dunlap <george.dunlap@citrix.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Michal Orzel <michal.orzel@amd.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"

Hi Xen maintainers,

In my patch series [XEN PATCH v4 0/5] FF-A notifications [1] I need to
get a reference to a domain struct from a domain ID and keep it from
being destroyed while using it in the interrupt handler
notif_irq_handler() (introduced in the last patch "[XEN PATCH v4 5/5]
xen/arm: ffa: support notification"). In my previous patch set [2] I
used get_domain_by_id(), but from the following discussion
rcu_lock_live_remote_domain_by_id() seems to be a better choice so
that's what I'm using now in the v4 patch set. The domain lock is held
during a call to vgic_inject_irq() and unlocked right after.

While we're reviewing the patch set in [1] I'd like to check the
approach with rcu_lock_live_remote_domain_by_id() here.

What do you think? Is using rcu_lock_live_remote_domain_by_id() the
best approach?

[1] https://patchew.org/Xen/20240502145645.1201613-1-jens.wiklander@linaro.org/
[2] https://patchew.org/Xen/20240426084723.4149648-1-jens.wiklander@linaro.org/

Thanks,
Jens


From xen-devel-bounces@lists.xenproject.org Fri May 03 08:34:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 08:34:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716279.1118372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2oMS-00069T-GZ; Fri, 03 May 2024 08:33:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716279.1118372; Fri, 03 May 2024 08:33:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2oMS-00069M-CO; Fri, 03 May 2024 08:33:44 +0000
Received: by outflank-mailman (input) for mailman id 716279;
 Fri, 03 May 2024 08:33:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZLhg=MG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2oMR-00069G-2w
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 08:33:43 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8391e11-0927-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 10:33:40 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e22a1bed91so8283331fa.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 01:33:40 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 f19-20020a05600c4e9300b0041bab13cd60sm8582372wmq.3.2024.05.03.01.33.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 01:33:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8391e11-0927-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714725220; x=1715330020; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=kWjmxi2cOzc79QflylxF/WlInTn2tT2FGY/gI+Gmvec=;
        b=dTRV9BgaPZ7HX09z704JxJutNUAKBMXgXbE3OGApOYLfbg1yPHAImC6/+3al5ujKV1
         E+zypAj18ezFtBIMukToszSNlMlyehFOgtl3g7v34SF1JJFPeUheSD6pVgXmZru10szg
         wIbtDI7R83AZoCc3lqjAHSHh6wiEylZs/rmdY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714725220; x=1715330020;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kWjmxi2cOzc79QflylxF/WlInTn2tT2FGY/gI+Gmvec=;
        b=cmEI0XiCokuI7gWbziEJRtMfOsmYZMot9CXCbrpZ4TdPgKR0hPeBLTrSf5lVhpPjsY
         OP7XQdgNJHCYyaJSmrQCrsD7nRObyc+vRvjBGkD0c/stUN3c1QRj/ZWHbCZgjkb8+Y4n
         yMbaBrHyBLCtPljAd+VphFCTwl3zxdxHcUyRWpnL4Qi9vEFdFRDacPVAUFRmlmY7WnJQ
         QulONF20/hlnsp1cXU9PdVjRJ1X5yIgF3mMiWCyJ/loaj2EG+9hGcaQeDNItycVkbalO
         01T0aq3+UiyjsHDCYu37RhPoiJjyMuYjwjmQHSB0uNnbOzM/ML2ONFkFDoGI40dVEV7C
         aePA==
X-Gm-Message-State: AOJu0Ywrd8L5XXLqREirZBfnrDrYhgLKgr7YR4b+poyZEFbKmBNOcqhw
	JmvhrJmeRb1L+ISUsWENhU2qKqBOkkViAHvZo2XEm6LmdNiLZBOJdN2Bii/JcF0=
X-Google-Smtp-Source: AGHT+IFZZfAEAwcZyZj1SwVnYywLJASTIN7o9Kcl8NxB6jUYw0S3+HK7NAe1PD+8QC8UIrM0iC+acw==
X-Received: by 2002:a2e:9857:0:b0:2df:4a6a:a195 with SMTP id e23-20020a2e9857000000b002df4a6aa195mr1147585ljj.11.1714725220008;
        Fri, 03 May 2024 01:33:40 -0700 (PDT)
Date: Fri, 3 May 2024 10:33:38 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v6 3/7] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <ZjShYhUvx694rPEp@macbook>
References: <cover.3491f479d5b6238106af8ddc56ec6834090b330f.1714154036.git-series.marmarek@invisiblethingslab.com>
 <a9b04e2224e97a27a127a003e8ccf5edfd4922c7.1714154036.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a9b04e2224e97a27a127a003e8ccf5edfd4922c7.1714154036.git-series.marmarek@invisiblethingslab.com>

On Fri, Apr 26, 2024 at 07:54:00PM +0200, Marek Marczykowski-Górecki wrote:
> Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> on the same page as MSI-X table. Device model (especially one in
> stubdomain) cannot really handle those, as direct writes to that page is
> refused (page is on the mmio_ro_ranges list). Instead, extend
> msixtbl_mmio_ops to handle such accesses too.
> 
> Doing this, requires correlating read/write location with guest
> of MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
  ^ extra 'of'?
> it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> for PV would need to be done separately.
> 
> This will be also used to read Pending Bit Array, if it lives on the same
> page, making QEMU not needing /dev/mem access at all (especially helpful
> with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> map it to the guest directly.
> If PBA lives on the same page, discard writes and log a message.
> Technically, writes outside of PBA could be allowed, but at this moment
> the precise location of PBA isn't saved, and also no known device abuses
> the spec in this way (at least yet).
> 
> To access those registers, msixtbl_mmio_ops need the relevant page
> mapped. MSI handling already has infrastructure for that, using fixmap,
> so try to map first/last page of the MSI-X table (if necessary) and save
> their fixmap indexes. Note that msix_get_fixmap() does reference
> counting and reuses existing mapping, so just call it directly, even if
> the page was mapped before. Also, it uses a specific range of fixmap
> indexes which doesn't include 0, so use 0 as default ("not mapped")
> value - which simplifies code a bit.
> 
> GCC 12.2.1 gets confused about 'desc' variable:
> 
>     arch/x86/hvm/vmsi.c: In function ‘msixtbl_range’:
>     arch/x86/hvm/vmsi.c:553:8: error: ‘desc’ may be used uninitialized [-Werror=maybe-uninitialized]
>       553 |     if ( desc )
>           |        ^
>     arch/x86/hvm/vmsi.c:537:28: note: ‘desc’ was declared here
>       537 |     const struct msi_desc *desc;
>           |                            ^~~~
> 
> It's conditional initialization is actually correct (in the case where
> it isn't initialized, function returns early), but to avoid
> build failure initialize it explicitly to NULL anyway.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> Changes in v6:
> - use MSIX_CHECK_WARN macro
> - extend assert on fixmap_idx
> - add break in default label, after ASSERT_UNREACHABLE(), and move
>   setting default there
> - style fixes
> Changes in v5:
> - style fixes
> - include GCC version in the commit message
> - warn only once (per domain, per device) about failed adjacent access
> Changes in v4:
> - drop same_page parameter of msixtbl_find_entry(), distinguish two
>   cases in relevant callers
> - rename adj_access_table_idx to adj_access_idx
> - code style fixes
> - drop alignment check in adjacent_{read,write}() - all callers already
>   have it earlier
> - delay mapping first/last MSI-X pages until preparing device for a
>   passthrough
> v3:
>  - merge handling into msixtbl_mmio_ops
>  - extend commit message
> v2:
>  - adjust commit message
>  - pass struct domain to msixtbl_page_handler_get_hwaddr()
>  - reduce local variables used only once
>  - log a warning if write is forbidden if MSI-X and PBA lives on the same
>    page
>  - do not passthrough unaligned accesses
>  - handle accesses both before and after MSI-X table
> ---
>  xen/arch/x86/hvm/vmsi.c        | 200 ++++++++++++++++++++++++++++++++--
>  xen/arch/x86/include/asm/msi.h |   5 +-
>  xen/arch/x86/msi.c             |  41 +++++++-
>  3 files changed, 236 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> index 999917983789..230e3a5dee3f 100644
> --- a/xen/arch/x86/hvm/vmsi.c
> +++ b/xen/arch/x86/hvm/vmsi.c
> @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
>      return d->arch.hvm.msixtbl_list.next;
>  }
>  
> +/*
> + * Lookup an msixtbl_entry on the same page as given addr. It's up to the
> + * caller to check if address is strictly part of the table - if relevant.
> + */
>  static struct msixtbl_entry *msixtbl_find_entry(
>      struct vcpu *v, unsigned long addr)
>  {
> @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
>      struct domain *d = v->domain;
>  
>      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> -        if ( addr >= entry->gtable &&
> -             addr < entry->gtable + entry->table_len )
> +        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
> +             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
>              return entry;
>  
>      return NULL;
> @@ -213,6 +217,138 @@ static struct msi_desc *msixtbl_addr_to_desc(
>      return NULL;
>  }
>  
> +/*
> + * Returns:
> + *  - ADJACENT_DONT_HANDLE if no handling should be done
> + *  - ADJACENT_DISCARD_WRITE if write should be discarded
> + *  - a fixmap idx to use for handling
> + */
> +#define ADJACENT_DONT_HANDLE UINT_MAX
> +#define ADJACENT_DISCARD_WRITE (UINT_MAX - 1)

I think this could be simpler, there's no need to signal with so fine
grained detail about the action to be performed.

Any adjacent access to the MSI-X table should be handled by the logic
you are adding, so anything that falls in those ranges should
terminate here.

adjacent_handle() should IMO just return whether the access is
replayed against the hardware, or if it's just dropped.

> +static unsigned int adjacent_handle(
> +    const struct msixtbl_entry *entry, unsigned long addr, bool write)
> +{
> +    unsigned int adj_type;
> +    struct arch_msix *msix;
> +
> +    if ( !entry || !entry->pdev )
> +        return ADJACENT_DONT_HANDLE;
> +
> +    if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable )
> +        adj_type = ADJ_IDX_FIRST;
> +    else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) &&
> +              addr >= entry->gtable + entry->table_len )
> +        adj_type = ADJ_IDX_LAST;
> +    else
> +        return ADJACENT_DONT_HANDLE;
> +
> +    msix = entry->pdev->msix;
> +    ASSERT(msix);
> +
> +    if ( !msix->adj_access_idx[adj_type] )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_not_initialized) )
> +            gprintk(XENLOG_WARNING,
> +                    "Page for adjacent(%d) MSI-X table access not initialized for %pp (addr %#lx, gtable %#lx\n",
> +                    adj_type, &entry->pdev->sbdf, addr, entry->gtable);
> +        return ADJACENT_DONT_HANDLE;
> +    }
> +
> +    /* If PBA lives on the same page too, discard writes. */
> +    if ( write &&
> +         ((adj_type == ADJ_IDX_LAST &&
> +           msix->table.last == msix->pba.first) ||
> +          (adj_type == ADJ_IDX_FIRST &&
> +           msix->table.first == msix->pba.last)) )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_pba) )
> +            gprintk(XENLOG_WARNING,
> +                    "MSI-X table and PBA of %pp live on the same page, "
> +                    "writing to other registers there is not implemented\n",
> +                    &entry->pdev->sbdf);
> +        return ADJACENT_DISCARD_WRITE;
> +    }
> +
> +    return msix->adj_access_idx[adj_type];
> +}
> +
> +static int adjacent_read(
> +    unsigned int fixmap_idx,
> +    paddr_t address, unsigned int len, uint64_t *pval)
> +{
> +    const void __iomem *hwaddr;
> +
> +    ASSERT(fixmap_idx <= FIX_MSIX_IO_RESERV_END);
> +
> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);

IMO adjacent_handle() should be called here (and in adjacent_write()),
and adjacent_{read,write}() called unconditionally from
msixtbl_{read,write}() when an access that doesn't fall in the MSI-X
table is handled.  See comment below in msixtbl_read().

> +
> +    switch ( len )
> +    {
> +    case 1:
> +        *pval = readb(hwaddr);
> +        break;
> +
> +    case 2:
> +        *pval = readw(hwaddr);
> +        break;
> +
> +    case 4:
> +        *pval = readl(hwaddr);
> +        break;
> +
> +    case 8:
> +        *pval = readq(hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +        *pval = ~0UL;
> +        break;
> +    }
> +
> +    return X86EMUL_OKAY;
> +}
> +
> +static int adjacent_write(
> +    unsigned int fixmap_idx,
> +    paddr_t address, unsigned int len, uint64_t val)
> +{
> +    void __iomem *hwaddr;
> +
> +    if ( fixmap_idx == ADJACENT_DISCARD_WRITE )
> +        return X86EMUL_OKAY;
> +
> +    ASSERT(fixmap_idx <= FIX_MSIX_IO_RESERV_END);

Since you check the idx is sane, shouldn't you also assert idx >=
FIX_MSIX_IO_RESERV_BASE?

> +
> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        writeb(val, hwaddr);
> +        break;
> +
> +    case 2:
> +        writew(val, hwaddr);
> +        break;
> +
> +    case 4:
> +        writel(val, hwaddr);
> +        break;
> +
> +    case 8:
> +        writeq(val, hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +    }
> +
> +    return X86EMUL_OKAY;
> +}
> +
>  static int cf_check msixtbl_read(
>      const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
>      uint64_t *pval)
> @@ -220,9 +356,10 @@ static int cf_check msixtbl_read(
>      unsigned long offset;
>      struct msixtbl_entry *entry;
>      unsigned int nr_entry, index;
> +    unsigned int adjacent_fixmap;
>      int r = X86EMUL_UNHANDLEABLE;
>  
> -    if ( (len != 4 && len != 8) || (address & (len - 1)) )
> +    if ( !IS_ALIGNED(address, len) )
>          return r;
>  
>      rcu_read_lock(&msixtbl_rcu_lock);
> @@ -230,6 +367,21 @@ static int cf_check msixtbl_read(
>      entry = msixtbl_find_entry(current, address);
>      if ( !entry )
>          goto out;
> +
> +    adjacent_fixmap = adjacent_handle(entry, address, false);

This seems overly complicated, but is possible I'm missing some logic.

IMO it would seem way less convoluted to simply do:

entry = msixtbl_find_entry(current, address);
if ( !entry )
    goto out;

if ( address < entry->gtable ||
     address >= entry->gtable + entry->table_len )
{
    adjacent_read(...);
    goto out;
}

And put all the logic in adjacent_{read,write}() directly rather than
having both adjacent_{read,write}() plus adjacent_handle() calls here?

If the access doesn't fall between the boundaries of the MSI-X table
it's either going to be a handled adjacent access, or it's going to be
discarded.

> +    if ( adjacent_fixmap != ADJACENT_DONT_HANDLE )
> +    {
> +        r = adjacent_read(adjacent_fixmap, address, len, pval);
> +        goto out;
> +    }
> +
> +    if ( address < entry->gtable ||
> +         address >= entry->gtable + entry->table_len )
> +        goto out;
> +
> +    if ( len != 4 && len != 8 )
> +        goto out;
> +
>      offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
>  
>      if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
> @@ -282,6 +434,7 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
>      int r = X86EMUL_UNHANDLEABLE;
>      unsigned long flags;
>      struct irq_desc *desc;
> +    unsigned int adjacent_fixmap;
>  
>      if ( !IS_ALIGNED(address, len) )
>          return X86EMUL_OKAY;
> @@ -291,6 +444,21 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
>      entry = msixtbl_find_entry(v, address);
>      if ( !entry )
>          goto out;
> +
> +    adjacent_fixmap = adjacent_handle(entry, address, true);
> +    if ( adjacent_fixmap != ADJACENT_DONT_HANDLE )
> +    {
> +        r = adjacent_write(adjacent_fixmap, address, len, val);
> +        goto out;
> +    }
> +
> +    if ( address < entry->gtable ||
> +         address >= entry->gtable + entry->table_len )
> +        goto out;
> +
> +    if ( len != 4 && len != 8 )
> +        goto out;
> +
>      nr_entry = array_index_nospec(((address - entry->gtable) /
>                                     PCI_MSIX_ENTRY_SIZE),
>                                    MAX_MSIX_TABLE_ENTRIES);
> @@ -356,8 +524,8 @@ static int cf_check _msixtbl_write(
>      const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
>      uint64_t val)
>  {
> -    /* Ignore invalid length or unaligned writes. */
> -    if ( (len != 4 && len != 8) || !IS_ALIGNED(address, len) )
> +    /* Ignore unaligned writes. */
> +    if ( !IS_ALIGNED(address, len) )
>          return X86EMUL_OKAY;
>  
>      /*
> @@ -374,14 +542,22 @@ static bool cf_check msixtbl_range(
>  {
>      struct vcpu *curr = current;
>      unsigned long addr = r->addr;
> -    const struct msi_desc *desc;
> +    const struct msixtbl_entry *entry;
> +    const struct msi_desc *desc = NULL;
> +    unsigned int adjacent_fixmap;
>  
>      ASSERT(r->type == IOREQ_TYPE_COPY);
>  
>      rcu_read_lock(&msixtbl_rcu_lock);
> -    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
> +    entry = msixtbl_find_entry(curr, addr);
> +    adjacent_fixmap = adjacent_handle(entry, addr, false);
> +    if ( adjacent_fixmap == ADJACENT_DONT_HANDLE )
> +        desc = msixtbl_addr_to_desc(entry, addr);

I'm not sure you need adjacent_handle() here, I would think that any
address adjacent to the MSI-X table Xen would want to handle
unconditionally, and hence msixtbl_range() should return true:

entry = msixtbl_find_entry(curr, addr);
if ( !entry )
    return 0;

if ( addr < entry->gtable || addr >= entry->gtable + entry->table_len )
    /* Possibly handle adjacent access. */
    return 1;

desc = msixtbl_find_entry(curr, addr);
...

(Missing the _unlock calls in the chunk above)

There's no other processing that can happen for an adjacent access
unless it's are handled here.  Otherwise such accesses will be
discarded anyway?  Hence better short-circuit the MMIO handler search
earlier.

>      rcu_read_unlock(&msixtbl_rcu_lock);
>  
> +    if ( adjacent_fixmap != ADJACENT_DONT_HANDLE )
> +        return 1;
> +
>      if ( desc )
>          return 1;
>  
> @@ -622,12 +798,16 @@ void msix_write_completion(struct vcpu *v)
>           v->arch.hvm.hvm_io.msix_snoop_gpa )
>      {
>          unsigned int token = hvmemul_cache_disable(v);
> -        const struct msi_desc *desc;
> +        const struct msi_desc *desc = NULL;
> +        const struct msixtbl_entry *entry;
>          uint32_t data;
>  
>          rcu_read_lock(&msixtbl_rcu_lock);
> -        desc = msixtbl_addr_to_desc(msixtbl_find_entry(v, snoop_addr),
> -                                    snoop_addr);
> +        entry = msixtbl_find_entry(v, snoop_addr);
> +        if ( entry &&
> +             snoop_addr >= entry->gtable &&
> +             snoop_addr < entry->gtable + entry->table_len )
> +            desc = msixtbl_addr_to_desc(entry, snoop_addr);

This would be easier if you added the MSI-X table boundary checks in
msixtbl_addr_to_desc() itself, rather than open-coding here.  That way
you don't need to modify msix_write_completion() at all.  It also
makes msixtbl_addr_to_desc() more robust in case it gets called with
bogus addresses.

>          rcu_read_unlock(&msixtbl_rcu_lock);
>  
>          if ( desc &&
> diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
> index bcfdfd35345d..923b730d48b8 100644
> --- a/xen/arch/x86/include/asm/msi.h
> +++ b/xen/arch/x86/include/asm/msi.h
> @@ -224,6 +224,9 @@ struct arch_msix {
>      } table, pba;
>      int table_refcnt[MAX_MSIX_TABLE_PAGES];
>      int table_idx[MAX_MSIX_TABLE_PAGES];
> +#define ADJ_IDX_FIRST 0
> +#define ADJ_IDX_LAST  1
> +    unsigned int adj_access_idx[2];
>      spinlock_t table_lock;
>      bool host_maskall, guest_maskall;
>      domid_t warned_domid;
> @@ -231,6 +234,8 @@ struct arch_msix {
>          uint8_t all;
>          struct {
>              bool maskall                   : 1;
> +            bool adjacent_not_initialized  : 1;
> +            bool adjacent_pba              : 1;
>          };
>      } warned_kind;
>  };
> diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> index 42c793426da3..c77b81896269 100644
> --- a/xen/arch/x86/msi.c
> +++ b/xen/arch/x86/msi.c
> @@ -913,6 +913,36 @@ static int msix_capability_init(struct pci_dev *dev,
>          list_add_tail(&entry->list, &dev->msi_list);
>          *desc = entry;
>      }
> +    else
> +    {
> +        /*
> +         * If the MSI-X table doesn't start at the page boundary, map the first page for
> +         * passthrough accesses.
> +         */
> +        if ( PAGE_OFFSET(table_paddr) )
> +        {
> +            int idx = msix_get_fixmap(msix, table_paddr, table_paddr);
> +
> +            if ( idx > 0 )
> +                msix->adj_access_idx[ADJ_IDX_FIRST] = idx;
> +            else
> +                gprintk(XENLOG_ERR, "Failed to map first MSI-X table page: %d\n", idx);
> +        }
> +        /*
> +         * If the MSI-X table doesn't end on the page boundary, map the last page
> +         * for passthrough accesses.
> +         */
> +        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE) )

This check is correct ....

> +        {
> +            uint64_t entry_paddr = table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE;

... however for correctness you want to use msix->nr_entries - 1 here,
otherwise you are requesting an address that's past the last MSI-X
table entry, and hence msix_get_fixmap() could refuse to provide an
idx if it ever does boundary checking.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 03 08:59:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 08:59:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716287.1118381 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2olW-00011O-Fs; Fri, 03 May 2024 08:59:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716287.1118381; Fri, 03 May 2024 08:59:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2olW-00011H-Bn; Fri, 03 May 2024 08:59:38 +0000
Received: by outflank-mailman (input) for mailman id 716287;
 Fri, 03 May 2024 08:59:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2olV-000117-BL; Fri, 03 May 2024 08:59:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2olV-0000Rz-1m; Fri, 03 May 2024 08:59:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2olU-0002Er-Ls; Fri, 03 May 2024 08:59:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2olU-00077K-LV; Fri, 03 May 2024 08:59:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MMD/rcFoCa4YT2ruZoFwXpOLAmOK7FO/Zs5uGXwF/tw=; b=FCPrrdRggNLoznZocThEQHpYGW
	UsELIsYfh4ObS40ZREwQTVlszI1HARl5AigAPur/5oEqj5qs7gIUepFpf2VkG3bPQ5oC76cU8w+/u
	E1+i/Q6UAqtQrGpOK0KHH0rHzvlapp4b1nRdxhCObr6gvxNMxSH8jK6kLAZBfFBNfr90=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185906-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185906: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
X-Osstest-Versions-That:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 08:59:36 +0000

flight 185906 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185906/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185897

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185897
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185897
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185897
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185897
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185897
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185897
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b
baseline version:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b

Last test of basis   185906  2024-05-03 01:53:38 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Fri May 03 09:33:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:33:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716303.1118394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pHq-0006O4-3e; Fri, 03 May 2024 09:33:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716303.1118394; Fri, 03 May 2024 09:33:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pHq-0006Nx-1C; Fri, 03 May 2024 09:33:02 +0000
Received: by outflank-mailman (input) for mailman id 716303;
 Fri, 03 May 2024 09:33:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s2pHo-0006Nr-SF
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:33:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2pHm-00011D-1X; Fri, 03 May 2024 09:32:58 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2pHl-0003my-NU; Fri, 03 May 2024 09:32:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=geCwVCE6Vmcs1nQ3ULn3lQtitAxH+3tQFuZxraC/tgs=; b=t0PHbhsqNRxHHgfQX9+8B/ZRC2
	bGmX7NM2v2IeLt4Ok55ER16fUsnHB/Yc5cKvZ3NLIe4eo1QIk0NpH681DLP1IfMvrkOeIqaG1YGxB
	rzD5KpJ1nK4LWSU3z8f91v58MWM9jCPMaptZ+DJhhg8wMiuZrUkt8QCgyJQGwsg+vUQk=;
Message-ID: <47033435-c621-40f6-b5a9-a385f323f382@xen.org>
Date: Fri, 3 May 2024 10:32:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
Content-Language: en-GB
To: alessandro.zucchelli@bugseng.com, Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 bertrand.marquis@arm.com, michal.orzel@amd.com, volodymyr_babchuk@epam.com
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com>
 <834460aca34e6167a3c4620616c145b8@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <834460aca34e6167a3c4620616c145b8@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 03/05/2024 08:09, Alessandro Zucchelli wrote:
> On 2024-04-29 17:58, Jan Beulich wrote:
>> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
>>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
>>> allowing asm/mem_access.h to be included in all ARM build 
>>> configurations.
>>> This is to address the violation of MISRA C: 2012 Rule 8.4 which states:
>>> "A compatible declaration shall be visible when an object or function
>>> with external linkage is defined". Functions p2m_mem_access_check
>>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is not
>>> defined in ARM builds don't have visible declarations in the file
>>> containing their definitions.
>>>
>>> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
>>> ---
>>>  xen/include/xen/mem_access.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/xen/include/xen/mem_access.h b/xen/include/xen/mem_access.h
>>> index 87d93b31f6..ec0630677d 100644
>>> --- a/xen/include/xen/mem_access.h
>>> +++ b/xen/include/xen/mem_access.h
>>> @@ -33,7 +33,7 @@
>>>   */
>>>  struct vm_event_st;
>>>
>>> -#ifdef CONFIG_MEM_ACCESS
>>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
>>>  #include <asm/mem_access.h>
>>>  #endif
>>
>> This doesn't look quite right. If Arm supports mem-access, why would it
>> not set MEM_ACCESS=y? Whereas if it's only stubs that Arm supplies, then
>> those would better move here, thus eliminating the need for a per-arch
>> stub header (see what was e.g. done for numa.h). This way RISC-V and PPC
>> (and whatever is to come) would then be taken care of as well.
>>
> ARM does support mem-access, so I don't think this is akin to the 
> changes done to handle numa.h.
> ARM also allows users to set MEM_ACCESS=n (e.g. 
> xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however, 
> the implementation file mem_access.c is compiled unconditionally in 
> ARM's makefile, hence why the violation was spotted.
> This is a bit unusual, so I was also hoping to get some feedback from 
> mem-access maintainers as to why this discrepancy from x86 exists. I 
> probably should have also included some ARM maintainers as well, so I'm 
> going to loop them in now.
> 
> An alternative option I think is to make the compilation of arm's 
> mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and common).

I can't think of a reason to have mem_access.c unconditional compiled 
in. So I think it should be conditional like on x86.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 09:40:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:40:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716307.1118405 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pOF-0007HO-PE; Fri, 03 May 2024 09:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716307.1118405; Fri, 03 May 2024 09:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pOF-0007HH-MI; Fri, 03 May 2024 09:39:39 +0000
Received: by outflank-mailman (input) for mailman id 716307;
 Fri, 03 May 2024 09:39:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s2pOD-0007HB-Um
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:39:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2pO9-00018K-TT; Fri, 03 May 2024 09:39:33 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2pO9-0004KX-Lt; Fri, 03 May 2024 09:39:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zA9LaweF8nQdsYtFBtByudj/2+oq7tg35tHYk5ib3dI=; b=PnG7FLtCpBt+yKNjjYmyartmfl
	IkBEF8+6z75z4eEED/w5M67kiDhMeAp5l/nb7Wr7BxeQe9QCyxFScvKn570FSJOiqMz3CdqJuOCQj
	WpmFMZtcYaT79qRL5qeMwVy9jgs+7Zx8tJObzcajfVI/QfUnlm3Wz6HoL2Zctfzq27ZY=;
Message-ID: <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
Date: Fri, 3 May 2024 10:39:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Stefano,

On 02/05/2024 19:13, Stefano Stabellini wrote:
> On Mon, 29 Apr 2024, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 29/04/2024 12:28, Jürgen Groß wrote:
>>> On 29.04.24 13:04, Julien Grall wrote:
>>>> Hi Juergen,
>>>>
>>>> Sorry for the late reply.
>>>>
>>>> On 29/04/2024 11:33, Juergen Gross wrote:
>>>>> On 08.04.24 09:10, Jan Beulich wrote:
>>>>>> On 27.03.2024 16:22, Juergen Gross wrote:
>>>>>>> With lock handling now allowing up to 16384 cpus (spinlocks can
>>>>>>> handle
>>>>>>> 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit
>>>>>>> for
>>>>>>> the number of cpus to be configured to 16383.
>>>>>>>
>>>>>>> The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
>>>>>>> QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
>>>>>>>
>>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>>
>>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>>>
>>>>>> I'd prefer this to also gain an Arm ack, though.
>>>>>
>>>>> Any comment from Arm side?
>>>>
>>>> Can you clarify what the new limits mean in term of (security) support?
>>>> Are we now claiming that Xen will work perfectly fine on platforms with up
>>>> to 16383?
>>>>
>>>> If so, I can't comment for x86, but for Arm, I am doubtful that it would
>>>> work without any (at least performance) issues. AFAIK, this is also an
>>>> untested configuration. In fact I would be surprised if Xen on Arm was
>>>> tested with more than a couple of hundreds cores (AFAICT the Ampere CPUs
>>>> has 192 CPUs).
>>>
>>> I think we should add a security support limit for the number of physical
>>> cpus similar to the memory support limit we already have in place.
>>>
>>> For x86 I'd suggest 4096 cpus for security support (basically the limit we
>>> have with this patch), but I'm open for other suggestions, too.
>>>
>>> I have no idea about any sensible limits for Arm32/Arm64.
>>
>> I am not entirely. Bertrand, Michal, Stefano, should we use 192 (the number of
>> CPUs from Ampere)?
> 
> I am OK with that. If we want to be a bit more future proof we could say
> 256 or 512.

Sorry, I don't follow your argument. A limit can be raised at time point 
in the future. The question is more whether we are confident that Xen on 
Arm will run well if a user has a platform with 256/512 pCPUs.

So are you saying that from Xen point of view, you are expecting no 
difference between 256 and 512. And therefore you would be happy if to 
backport patches if someone find differences (or even security issues) 
when using > 256 pCPUs?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 09:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716315.1118415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgt-0002Bk-9B; Fri, 03 May 2024 09:58:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716315.1118415; Fri, 03 May 2024 09:58:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgt-0002Bd-68; Fri, 03 May 2024 09:58:55 +0000
Received: by outflank-mailman (input) for mailman id 716315;
 Fri, 03 May 2024 09:58:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2pgr-0002BR-Dc
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:58:53 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bed4c179-0933-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 11:58:52 +0200 (CEST)
Received: from truciolo.homenet.telecomitalia.it
 (host-79-36-52-167.retail.telecomitalia.it [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 6137A4EE0738;
 Fri,  3 May 2024 11:58:51 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bed4c179-0933-11ef-909c-e314d9c70b13
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH 0/2] misra: deviations of Rule 16.4
Date: Fri,  3 May 2024 11:58:42 +0200
Message-Id: <cover.1714727807.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Define "switch-clause" in terms of the C specification.
Deviate Rule 16.4.

Federico Serafini (2):
  docs/misra: add Terms & Definitions section to rules.rst
  automation/eclair: add deviation for Rule 16.4

 automation/eclair_analysis/ECLAIR/deviations.ecl |  8 ++++++++
 docs/misra/deviations.rst                        | 13 +++++++++++++
 docs/misra/rules.rst                             | 14 ++++++++++++--
 3 files changed, 33 insertions(+), 2 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 03 09:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716316.1118419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgt-0002F7-Gl; Fri, 03 May 2024 09:58:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716316.1118419; Fri, 03 May 2024 09:58:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgt-0002EG-DZ; Fri, 03 May 2024 09:58:55 +0000
Received: by outflank-mailman (input) for mailman id 716316;
 Fri, 03 May 2024 09:58:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2pgs-0002BX-Oq
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:58:54 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bef5dcdc-0933-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 11:58:52 +0200 (CEST)
Received: from truciolo.homenet.telecomitalia.it
 (host-79-36-52-167.retail.telecomitalia.it [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id EEE5F4EE074A;
 Fri,  3 May 2024 11:58:51 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bef5dcdc-0933-11ef-b4bb-af5377834399
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH 1/2] docs/misra: add Terms & Definitions section to rules.rst
Date: Fri,  3 May 2024 11:58:43 +0200
Message-Id: <05d9d4b4319f28d602b7366f2964c451a3a50ce3.1714727807.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1714727807.git.federico.serafini@bugseng.com>
References: <cover.1714727807.git.federico.serafini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a section for terms and definitions used by MISRA but expressed
in terms of the C specification.

Add a definition of "switch clause" to the newly-introduced section.

Link the first use of the term "switch clause" in the document to its
definition.

Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
Jan you were not completely satisfied by the definition but I didn't find
a better one.
---
 docs/misra/rules.rst | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index b7b447e152..d3b70fdf04 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -489,8 +489,7 @@ maintainers if you want to suggest a change.
 
    * - `Rule 16.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_03.c>`_
      - Required
-     - An unconditional break statement shall terminate every
-       switch-clause
+     - An unconditional break statement shall terminate every switch-clause_
      - In addition to break, also other unconditional flow control statements
        such as continue, return, goto are allowed.
 
@@ -712,3 +711,14 @@ maintainers if you want to suggest a change.
      - The value of a pointer to a FILE shall not be used after the associated
        stream has been closed
      -
+
+Terms & Definitions
+-------------------
+
+.. _switch-clause:
+
+A *switch clause* can be defined as:
+"the non-empty list of statements which follows a non-empty list of
+case/default labels".
+A formal definition is available within the amplification of MISRA C:2012
+Rule 16.1.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 03 09:59:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:59:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716317.1118434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgu-0002eL-MT; Fri, 03 May 2024 09:58:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716317.1118434; Fri, 03 May 2024 09:58:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pgu-0002eE-Jv; Fri, 03 May 2024 09:58:56 +0000
Received: by outflank-mailman (input) for mailman id 716317;
 Fri, 03 May 2024 09:58:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2pgt-0002BX-DB
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:58:55 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf38cf39-0933-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 11:58:52 +0200 (CEST)
Received: from truciolo.homenet.telecomitalia.it
 (host-79-36-52-167.retail.telecomitalia.it [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 5A7FE4EE074D;
 Fri,  3 May 2024 11:58:52 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf38cf39-0933-11ef-b4bb-af5377834399
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Subject: [XEN PATCH 2/2] automation/eclair: add deviation for Rule 16.4
Date: Fri,  3 May 2024 11:58:44 +0200
Message-Id: <d4670fa5bd7e33a915d5642cb78eb2eb33988819.1714727807.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1714727807.git.federico.serafini@bugseng.com>
References: <cover.1714727807.git.federico.serafini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C:2012 Rule 16.4 states that "Every switch statement shall have a
default label".
Update ECLAIR configuration to take into account the deviations
agreed during MISRA meetings.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/deviations.ecl |  8 ++++++++
 docs/misra/deviations.rst                        | 13 +++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index d21f112a9b..f09ad71acf 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -384,6 +384,14 @@ explicit comment indicating the fallthrough intention is present."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
 -doc_end
 
+-doc_begin="Switch statements having a controlling expression of enum type deliberately do not have a default case: gcc -Wall enables -Wswitch which warns (and breaks the build as we use -Werror) if one of the enum labels is missing from the switch."
+-config=MC3R1.R16.4,reports+={deliberate,'any_area(kind(context)&&^.* has no `default.*$&&stmt(node(switch_stmt)&&child(cond,skip(__non_syntactic_paren_stmts,type(canonical(enum_underlying_type(any())))))))'}
+-doc_end
+
+-doc_begin="A switch statement with a single switch clause and no default label may be used in place of an equivalent if statement if it is considered to improve readability."
+-config=MC3R1.R16.4,switch_clauses+={deliberate,"switch(1)&&default(0)"}
+-doc_end
+
 -doc_begin="A switch statement with a single switch clause and no default label may be used in place of an equivalent if statement if it is considered to improve readability."
 -config=MC3R1.R16.6,switch_clauses+={deliberate, "default(0)"}
 -doc_end
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index ed0c1e8ed0..39cc321a27 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -334,6 +334,19 @@ Deviations related to MISRA C:2012 Rules:
          - /\* Fallthrough \*/
          - /\* Fallthrough. \*/
 
+   * - R16.4
+     - Switch statements having a controlling expression of enum type
+       deliberately do not have a default case: gcc -Wall enables -Wswitch
+       which warns (and breaks the build as we use -Werror) if one of the enum
+       labels is missing from the switch.
+     - Tagged as `deliberate` for ECLAIR.
+
+   * - R16.4
+     - A switch statement with a single switch clause and no default label may
+       be used in place of an equivalent if statement if it is considered to
+       improve readability.
+     - Tagged as `deliberate` for ECLAIR.
+
    * - R16.6
      - A switch statement with a single switch clause and no default label may
        be used in place of an equivalent if statement if it is considered to
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 03 09:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 09:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716323.1118445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2phX-0003ct-V1; Fri, 03 May 2024 09:59:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716323.1118445; Fri, 03 May 2024 09:59:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2phX-0003cm-SM; Fri, 03 May 2024 09:59:35 +0000
Received: by outflank-mailman (input) for mailman id 716323;
 Fri, 03 May 2024 09:59:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s2phW-0003cT-Lo
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 09:59:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2phV-0001SC-Sv; Fri, 03 May 2024 09:59:33 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2phV-0005oK-Mq; Fri, 03 May 2024 09:59:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=dcybJbfBlY7UIodb/FocVxDnAnLFfxC08d+QFS8S/58=; b=cIeM4ya/LW+vGoxdTy+eJzQ49z
	D6J6iGxOXxE7SwyZneXfTRnAfDzkSUCqrrIjJZKARQXb46HGMORvMmYxtvAEGFjV03bJ25lHe5TRN
	Fq59+nQXicv4OhUw6/gpAmokqnxoY4lmEhcEpPk9exZ0UU8zhoEHM+ekTs+UDJ1njRRg=;
Message-ID: <935e3947-53ee-48c5-a10c-5fa7f572ae65@xen.org>
Date: Fri, 3 May 2024 10:59:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 5/5] xen/arm: ffa: support notification
Content-Language: en-GB
To: Jens Wiklander <jens.wiklander@linaro.org>, xen-devel@lists.xenproject.org
Cc: patches@linaro.org, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
 <20240502145645.1201613-6-jens.wiklander@linaro.org>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240502145645.1201613-6-jens.wiklander@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jens,

On 02/05/2024 15:56, Jens Wiklander wrote:
> -static bool ffa_probe(void)
> +static int __init ffa_init(void)
>   {
>       uint32_t vers;
>       unsigned int major_vers;
> @@ -460,16 +511,16 @@ static bool ffa_probe(void)
>           printk(XENLOG_ERR
>                  "ffa: unsupported SMCCC version %#x (need at least %#x)\n",
>                  smccc_ver, ARM_SMCCC_VERSION_1_2);
> -        return false;
> +        return 0;
>       }
>   
>       if ( !ffa_get_version(&vers) )
> -        return false;
> +        return 0;
>   
>       if ( vers < FFA_MIN_SPMC_VERSION || vers > FFA_MY_VERSION )
>       {
>           printk(XENLOG_ERR "ffa: Incompatible version %#x found\n", vers);
> -        return false;
> +        return 0;
>       }
>   
>       major_vers = (vers >> FFA_VERSION_MAJOR_SHIFT) & FFA_VERSION_MAJOR_MASK;
> @@ -492,26 +543,33 @@ static bool ffa_probe(void)
>            !check_mandatory_feature(FFA_MEM_SHARE_32) ||
>            !check_mandatory_feature(FFA_MEM_RECLAIM) ||
>            !check_mandatory_feature(FFA_MSG_SEND_DIRECT_REQ_32) )
> -        return false;
> +        return 0;
>   
>       if ( !ffa_rxtx_init() )
> -        return false;
> +        return 0;
>   
>       ffa_version = vers;
>   
>       if ( !ffa_partinfo_init() )
>           goto err_rxtx_destroy;
>   
> +    ffa_notif_init();
>       INIT_LIST_HEAD(&ffa_teardown_head);
>       init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0);
>   
> -    return true;
> +    return 0;
>   
>   err_rxtx_destroy:
>       ffa_rxtx_destroy();
>       ffa_version = 0;
>   
> -    return false;
> +    return 0;
> +}
> +presmp_initcall(ffa_init);
I would rather prefer if tee_init() is called from presmp_initcall(). 
This would avoid FFA to have to try to initialize itself before all the 
others.

This is what I had in mind with my original request, but I guess I 
wasn't clear enough. Sorry for that.

[...]

> +static void notif_irq_handler(int irq, void *data)
> +{
> +    const struct arm_smccc_1_2_regs arg = {
> +        .a0 = FFA_NOTIFICATION_INFO_GET_64,
> +    };
> +    struct arm_smccc_1_2_regs resp;
> +    unsigned int id_pos;
> +    unsigned int list_count;
> +    uint64_t ids_count;
> +    unsigned int n;
> +    int32_t res;
> +
> +    do {
> +        arm_smccc_1_2_smc(&arg, &resp);
> +        res = ffa_get_ret_code(&resp);
> +        if ( res )
> +        {
> +            if ( res != FFA_RET_NO_DATA )
> +                printk(XENLOG_ERR "ffa: notification info get failed: error %d\n",
> +                       res);
> +            return;
> +        }
> +
> +        ids_count = resp.a2 >> FFA_NOTIF_INFO_GET_ID_LIST_SHIFT;
> +        list_count = ( resp.a2 >> FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT ) &
> +                     FFA_NOTIF_INFO_GET_ID_COUNT_MASK;
> +
> +        id_pos = 0;
> +        for ( n = 0; n < list_count; n++ )
> +        {
> +            unsigned int count = ((ids_count >> 2 * n) & 0x3) + 1;
> +            uint16_t vm_id = get_id_from_resp(&resp, id_pos);
> +            struct domain *d;
> +
> +            /*
> +             * vm_id == 0 means a notifications pending for Xen itself, but
> +             * we don't support that yet.
> +             */
> +            if (vm_id)
> +                d = ffa_rcu_lock_domain_by_vm_id(vm_id);

I am still unconvinced that this is sufficient. This will prevent 
"struct domain *" to be freed. But ...

> +            else
> +                d = NULL;
> +
> +            if ( d )
> +            {
> +                struct ffa_ctx *ctx = d->arch.tee;

... I don't think it will protect d->arch.tee to be freed as this is 
happening during teardorwn. You mostly need some other sort of locking 
to avoid d->arch.tee been freed behind your back.

> +                struct vcpu *v;
> +
> +                ACCESS_ONCE(ctx->notif.secure_pending) = true;
> +
> +                /*
> +                 * Since we're only delivering global notification, always
> +                 * deliver to the first online vCPU. It doesn't matter
> +                 * which we chose, as long as it's available.
> +                 */
> +                for_each_vcpu(d, v)
> +                {
> +                    if ( is_vcpu_online(v) )
> +                    {
> +                        vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INTR_ID,
> +                                        true);
> +                        break;
> +                    }
> +                }
> +                if ( !v )
> +                    printk(XENLOG_ERR "ffa: can't inject NPI, all vCPUs offline\n");
> +
> +                rcu_unlock_domain(d);
> +            }
> +
> +            id_pos += count;
> +        }
> +
> +    } while (resp.a2 & FFA_NOTIF_INFO_GET_MORE_FLAG);
> +}

[...]

> +static int ffa_setup_irq_callback(struct notifier_block *nfb,
> +                                  unsigned long action, void *hcpu)
> +{
> +    unsigned int cpu = (unsigned long)hcpu;
> +    struct notif_irq_info irq_info = { };
> +
> +    switch ( action )
> +    {
> +    case CPU_ONLINE:

Can't you execute the notifier in CPU_STARTING? This will be called on 
the CPU directly, so you should be able to use request_irq(...).

> +        /*
> +         * The notifier call is done on the primary or initiating CPU when
> +         * the target CPU have come online, but the SGI must be setup on
> +         * the target CPU.
> +         *
> +         * We make an IPI call to the target CPU to setup the SGI. The call
> +         * is executed in interrupt context on the target CPU, so we can't
> +         * call request_irq() directly since it allocates memory.
> +         *
> +         * We preallocate the needed irqaction here and pass it via the
> +         * temporary struct notif_irq_info. The call is synchronous in the
> +         * sense that when on_selected_cpus() returns the callback
> +         * notif_irq_enable() has done the same on the target CPU.
> +         *
> +         * We deal with two errors, one where notif_irq_enable() hasn't
> +         * been called for some reason, that error is logged below. The
> +         * other where setup_irq() fails is logged in the callback. We must
> +         * free the irqaction in both cases since it has failed to become
> +         * registered.
> +         *
> +         * Failures leads to a problem notifications, the CPUs with failure
> +         * will trigger on the SGI indicating that there are notifications
> +         * pending, while the SPMC in the secure world will not notice that
> +         * the interrupt was lost.
> +         */
> +        irq_info.action = xmalloc(struct irqaction);
> +        if ( !irq_info.action )
> +        {
> +            printk(XENLOG_ERR "ffa: setup irq %u failed, out of memory\n",
> +                   notif_sri_irq);
> +            return -ENOMEM;
> +        }
> +
> +        *irq_info.action = (struct irqaction){
> +            .handler = notif_irq_handler,
> +            .name = "FF-A notif",
> +            .free_on_release = 1,
> +        };
> +
> +        on_selected_cpus(cpumask_of(cpu), notif_irq_enable, &irq_info, 1);
> +        if (!irq_info.called)
> +            printk(XENLOG_ERR "ffa: on_selected_cpus(cpumask_of(%u)) failed\n",
> +                   cpu);
> +        /*
> +         * The irqaction is unused and must be freed if irq_info.action is
> +         * non-NULL at this stage.
> +         */
> +        XFREE(irq_info.action);
> +        break;
> +    default:
> +        break;
> +    }
> +
> +    return 0;
> +}

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 10:10:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 10:10:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716338.1118454 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ps9-0006o3-1o; Fri, 03 May 2024 10:10:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716338.1118454; Fri, 03 May 2024 10:10:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ps8-0006nw-VH; Fri, 03 May 2024 10:10:32 +0000
Received: by outflank-mailman (input) for mailman id 716338;
 Fri, 03 May 2024 10:10:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wM3W=MG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2ps7-0006no-9q
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 10:10:31 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ea9c8da-0935-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 12:10:30 +0200 (CEST)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-34b3374ae22so8257518f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 03:10:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r8-20020a5d4988000000b0034cceee9051sm3367182wrq.105.2024.05.03.03.10.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 03:10:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ea9c8da-0935-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714731029; x=1715335829; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=6hTusBcHi9YxURLdUO/WLl8txt8QQtV1zWj91AU31ts=;
        b=RHGnZWjBpuYFCEUVAM5pNiumCNW1CKR/ZV2/J8a04uVEcaYY7kfdxRdUGTMTuQIuD6
         8rl0NBQ4hczCJr3CxE9Wnr22W3SiD9v0FS+HarFv44hgLtTR2LrDR/+IRKZTb+LjS2SB
         mllghl67c8huqWMRAlW9vxVIbnQZyoS9pHJLwQScVvjIDU5NZk9rcRG6mJZTtOJXbJz3
         HQj4NpG02IrtgCwkWaDO0m7C8ILR2eyy6209QCJWivhin0bRHAu2eT/gzSfE+UrPeUew
         MQ7a4teUjeZP2RhLQWGGb1gfUV0eW0VVND0ZpwKslQFiDCNLIVqPPL46Buv9mHh91apv
         pyJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714731029; x=1715335829;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6hTusBcHi9YxURLdUO/WLl8txt8QQtV1zWj91AU31ts=;
        b=oC7pCxOp4ZZ2FTM9YNGQXJJOM1c50lX0YMzD8IV4PrOmi4JRItpbWMITBD7xVHVEO1
         p+bFO/Gn89GCxaP6fskUJsyiesuUm6g/ns5cIZsC0NjNvaitFcRtkBNHULWDhxk4hBDe
         l8e/9lSq7F3OiwTyGXT0oWSfo4CauHxuW/S5NfQXcICYjuPkVA3NFQI4tIFF4OsWIMvK
         0EuHbw2dE8wZl0D/VHFvNKG9hEHEZp1In3Y+HfHgbiGuABlHJeQpMhvM3sB6w1BjU8Nq
         EDHjeSTyy4RPETP9ayW5l51knLpYDqZ8UxyV5VXli3OfYVfMy9GZC55oHNH15tOaApsE
         Ht/Q==
X-Gm-Message-State: AOJu0YxFqS1qjNK7vpN4Ko9PdlraVFzyBtZ60QMdTKVpBhU67+8faUUM
	LGRcKB1Xz5W/MUoLAu9N5Qxflamf5m+fLsNbP1F16AO57DqCiQvXC6zXp2F2uQ==
X-Google-Smtp-Source: AGHT+IEjldV8dM6Mk24Eykv59A4quGmLQM1oMuK2rNayan+XPKnnuDSueH7CAf/NOkAcYTq3G2OJZQ==
X-Received: by 2002:adf:c6c2:0:b0:34c:b1a9:1816 with SMTP id c2-20020adfc6c2000000b0034cb1a91816mr2036149wrh.31.1714731029364;
        Fri, 03 May 2024 03:10:29 -0700 (PDT)
Message-ID: <83741df3-b951-4351-88c1-2b7bb7b1e16e@suse.com>
Date: Fri, 3 May 2024 12:10:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 1/3] drivers: char: address violation of MISRA C
 Rule 20.7
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 bertrand.marquis@arm.com, julien@xen.org,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714487169.git.nicola.vetrini@bugseng.com>
 <3ae8aa13a5562440cf60e4b30f713e5b2f73c577.1714487169.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405011257290.497719@ubuntu-linux-20-04-desktop>
 <25f8aec2da7e963f80c12e23fc5f725a@bugseng.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25f8aec2da7e963f80c12e23fc5f725a@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 03.05.2024 09:29, Nicola Vetrini wrote:
> On 2024-05-01 21:57, Stefano Stabellini wrote:
>> On Tue, 30 Apr 2024, Nicola Vetrini wrote:
>>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>>> of macro parameters shall be enclosed in parentheses". Therefore, some
>>> macro definitions should gain additional parentheses to ensure that 
>>> all
>>> current and future users will be safe with respect to expansions that
>>> can possibly alter the semantics of the passed-in macro parameter.
>>>
>>> No functional chage.
>>>
>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>>
>>
>>> ---
>>> Changes in v2:
>>> - drop excess parentheses from val parameter.
>>> ---
>>>  xen/drivers/char/omap-uart.c | 5 +++--
>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/xen/drivers/char/omap-uart.c 
>>> b/xen/drivers/char/omap-uart.c
>>> index 03b5b66e7acb..e0128225f927 100644
>>> --- a/xen/drivers/char/omap-uart.c
>>> +++ b/xen/drivers/char/omap-uart.c
>>> @@ -48,8 +48,9 @@
>>>  /* System configuration register */
>>>  #define UART_OMAP_SYSC_DEF_CONF   0x0d   /* autoidle mode, wakeup is 
>>> enabled */
>>>
>>> -#define omap_read(uart, off)       readl((uart)->regs + 
>>> (off<<REG_SHIFT))
>>> -#define omap_write(uart, off, val) writel((val), (uart)->regs + 
>>> (off<<REG_SHIFT))
>>> +#define omap_read(uart, off)       readl((uart)->regs + ((off) << 
>>> REG_SHIFT))
>>> +#define omap_write(uart, off, val) writel(val, (uart)->regs + \
>>> +                                               ((off) << REG_SHIFT))
>>
>> the alignment looks off but could be fixed on commit
>>
> 
> Can you clarify what you mean here? I aligned readl and writeln and the 
> operands in writel to avoid the line being too long.

#define omap_write(uart, off, val) writel(val, \
                                          (uart)->regs + ((off) << REG_SHIFT))

The main point being that before you start splitting an argument following
another one on the same line, you'd move that argument to a new line.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 03 10:18:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 10:18:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716344.1118465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pze-0007w7-PV; Fri, 03 May 2024 10:18:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716344.1118465; Fri, 03 May 2024 10:18:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2pze-0007w0-Mi; Fri, 03 May 2024 10:18:18 +0000
Received: by outflank-mailman (input) for mailman id 716344;
 Fri, 03 May 2024 10:18:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GLdc=MG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s2pze-0007vu-9R
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 10:18:18 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 746bb75d-0936-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 12:18:15 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 587954EE0738;
 Fri,  3 May 2024 12:18:15 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 746bb75d-0936-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Fri, 03 May 2024 12:18:15 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 bertrand.marquis@arm.com, julien@xen.org, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH v2 1/3] drivers: char: address violation of MISRA C
 Rule 20.7
In-Reply-To: <83741df3-b951-4351-88c1-2b7bb7b1e16e@suse.com>
References: <cover.1714487169.git.nicola.vetrini@bugseng.com>
 <3ae8aa13a5562440cf60e4b30f713e5b2f73c577.1714487169.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405011257290.497719@ubuntu-linux-20-04-desktop>
 <25f8aec2da7e963f80c12e23fc5f725a@bugseng.com>
 <83741df3-b951-4351-88c1-2b7bb7b1e16e@suse.com>
Message-ID: <82c375da8e1144b150e73c265ae206d6@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-03 12:10, Jan Beulich wrote:
> On 03.05.2024 09:29, Nicola Vetrini wrote:
>> On 2024-05-01 21:57, Stefano Stabellini wrote:
>>> On Tue, 30 Apr 2024, Nicola Vetrini wrote:
>>>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>>>> of macro parameters shall be enclosed in parentheses". Therefore, 
>>>> some
>>>> macro definitions should gain additional parentheses to ensure that
>>>> all
>>>> current and future users will be safe with respect to expansions 
>>>> that
>>>> can possibly alter the semantics of the passed-in macro parameter.
>>>> 
>>>> No functional chage.
>>>> 
>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>> 
>>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>>> 
>>> 
>>>> ---
>>>> Changes in v2:
>>>> - drop excess parentheses from val parameter.
>>>> ---
>>>>  xen/drivers/char/omap-uart.c | 5 +++--
>>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>> 
>>>> diff --git a/xen/drivers/char/omap-uart.c
>>>> b/xen/drivers/char/omap-uart.c
>>>> index 03b5b66e7acb..e0128225f927 100644
>>>> --- a/xen/drivers/char/omap-uart.c
>>>> +++ b/xen/drivers/char/omap-uart.c
>>>> @@ -48,8 +48,9 @@
>>>>  /* System configuration register */
>>>>  #define UART_OMAP_SYSC_DEF_CONF   0x0d   /* autoidle mode, wakeup 
>>>> is
>>>> enabled */
>>>> 
>>>> -#define omap_read(uart, off)       readl((uart)->regs +
>>>> (off<<REG_SHIFT))
>>>> -#define omap_write(uart, off, val) writel((val), (uart)->regs +
>>>> (off<<REG_SHIFT))
>>>> +#define omap_read(uart, off)       readl((uart)->regs + ((off) <<
>>>> REG_SHIFT))
>>>> +#define omap_write(uart, off, val) writel(val, (uart)->regs + \
>>>> +                                               ((off) << 
>>>> REG_SHIFT))
>>> 
>>> the alignment looks off but could be fixed on commit
>>> 
>> 
>> Can you clarify what you mean here? I aligned readl and writeln and 
>> the
>> operands in writel to avoid the line being too long.
> 
> #define omap_write(uart, off, val) writel(val, \
>                                           (uart)->regs + ((off) << 
> REG_SHIFT))
> 
> The main point being that before you start splitting an argument 
> following
> another one on the same line, you'd move that argument to a new line.
> 
> Jan

Oh, ok. Didn't think of that. Thanks, can amend it

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 03 10:32:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 10:32:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716351.1118475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2qDb-0002QC-Ui; Fri, 03 May 2024 10:32:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716351.1118475; Fri, 03 May 2024 10:32:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2qDb-0002Q5-Ro; Fri, 03 May 2024 10:32:43 +0000
Received: by outflank-mailman (input) for mailman id 716351;
 Fri, 03 May 2024 10:32:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wM3W=MG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s2qDa-0002Pz-R2
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 10:32:42 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 786de246-0938-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 12:32:41 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-51ab4ee9df8so11029494e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 03:32:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d10-20020a05600c3aca00b00418e4cc9de7sm8897606wms.7.2024.05.03.03.32.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 03:32:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 786de246-0938-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714732361; x=1715337161; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0LwtdhgJMCWtx/g+IZW/rcdUO2Tt5I4SDONdXJl8gls=;
        b=IdYAUaPl1ee7/ma2Cw/G78dR9WoO99uWR0atMb60xXi1IaeNdt1LgCn6MJ6AuzHVRB
         TiA/FMoDwz8GDuC2OIs/Pk5D0K0rVgx6InAwTCL80Ta+9wAAADOaQYb5vXbjBgBko2Lu
         PUuMRkFKQ7X84EhethCVYmOE3RPJgtXDBWtdqcPg+ZeF0AZkLfhP0w/HoIv1v0NOJ+4q
         lnAOdF5tHlQC+mnSOjLYgoihma59nbN+E4gFIxN8JTjGUNj67ImV93RGV+00H4DVgq0t
         cZx8Etubcx9JX2Rjd1FFFVWf9MkrqHouwIa5ZgE2t9TEP5OjrJG9dusWN8XYTqjSWeXS
         YKLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714732361; x=1715337161;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0LwtdhgJMCWtx/g+IZW/rcdUO2Tt5I4SDONdXJl8gls=;
        b=cUdOxT0BIuOiuuveJ67zn3jJecfNJKdEd6/GqDjf5V4J3lWjyGyOn+IQD2wzaC5YKS
         G7dOXnBuzP7uY//FueUmMi6+egXp+ub/uX+sWIUGyAxuuKint8l3Rb4yE2F5z+1HSds9
         zivLQj/ikAYuneC1UoeAcq8rLTwzRX/OiAE4aPhBk/nLJpPsBjNYXMI1izd5TSn2WvYx
         r7YJuTJ0KoZIG7wvzxetdzt8QHaYZ2HYzIFVbFpiUaWDTY9W1+QY5kcCH3+ozfgNlEeY
         g81QwphzVJgvvWZpZRk9P1FyNuhwD2OQNx0zAwg4+6LH5iTqzgj7OfUBAr3eVCuIEhke
         EguQ==
X-Gm-Message-State: AOJu0Yxb/gPiQILL7eBrugvWEiPOY98kcf4pEik6F93CbEgo5H6JvgRl
	IjS13564oM8Eksl3GIAa8hhDLwJiduWNKrCSdSGxXQAcwIpBnYAbUWO4nZRhZQ==
X-Google-Smtp-Source: AGHT+IGCjl/51ZWB5mk+Di29fz92dVWfq3PJlYg8rEjSRXj/Y2Udv2M72vv5Mohlz8rvMiRfBs8DLg==
X-Received: by 2002:ac2:522c:0:b0:51c:d876:710c with SMTP id i12-20020ac2522c000000b0051cd876710cmr1404026lfl.37.1714732361014;
        Fri, 03 May 2024 03:32:41 -0700 (PDT)
Message-ID: <b965ee57-c6fc-459f-a5fd-fae47dc6ea9d@suse.com>
Date: Fri, 3 May 2024 12:32:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Referencing domain struct from interrupt handler
Content-Language: en-US
To: Jens Wiklander <jens.wiklander@linaro.org>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 George Dunlap <george.dunlap@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 03.05.2024 09:45, Jens Wiklander wrote:
> Hi Xen maintainers,
> 
> In my patch series [XEN PATCH v4 0/5] FF-A notifications [1] I need to
> get a reference to a domain struct from a domain ID and keep it from
> being destroyed while using it in the interrupt handler
> notif_irq_handler() (introduced in the last patch "[XEN PATCH v4 5/5]
> xen/arm: ffa: support notification"). In my previous patch set [2] I
> used get_domain_by_id(), but from the following discussion
> rcu_lock_live_remote_domain_by_id() seems to be a better choice so
> that's what I'm using now in the v4 patch set. The domain lock is held
> during a call to vgic_inject_irq() and unlocked right after.
> 
> While we're reviewing the patch set in [1] I'd like to check the
> approach with rcu_lock_live_remote_domain_by_id() here.
> 
> What do you think? Is using rcu_lock_live_remote_domain_by_id() the
> best approach?

Is it guaranteed that the IRQ handler won't ever run in the context of a
vCPU belonging to the domain in question? If not, why the "remote" form
of the function?

Furthermore, is it guaranteed that the IRQ handler won't interrupt code
fiddling with the domain list? I don't think it is, since
domlist_update_lock isn't acquired in an IRQ-safe manner. Looks like
you need to defer the operation on the domain until softirq or tasklet
context.

Jan

> [1] https://patchew.org/Xen/20240502145645.1201613-1-jens.wiklander@linaro.org/
> [2] https://patchew.org/Xen/20240426084723.4149648-1-jens.wiklander@linaro.org/
> 
> Thanks,
> Jens



From xen-devel-bounces@lists.xenproject.org Fri May 03 12:20:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 12:20:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716397.1118493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2rtw-0000oo-Hu; Fri, 03 May 2024 12:20:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716397.1118493; Fri, 03 May 2024 12:20:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2rtw-0000oh-FN; Fri, 03 May 2024 12:20:32 +0000
Received: by outflank-mailman (input) for mailman id 716397;
 Fri, 03 May 2024 12:20:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=UPFJ=MG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s2rtu-0000oa-R3
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 12:20:30 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 879d40bb-0947-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 14:20:29 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C743B20359;
 Fri,  3 May 2024 12:20:28 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9334613991;
 Fri,  3 May 2024 12:20:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id axdxIozWNGbmagAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 03 May 2024 12:20:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 879d40bb-0947-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714738828; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ut+JW89x87VZepZnO2x2XzUTpzW28xGmJv03rNS6b4A=;
	b=IChCKY5tWm5MO4ZrqNegxopgfKarmL+AoRo62F1aGEhECBlAjh5e05TZZsjxEo/DQfWPQe
	Da2XiUWsxBGIlm35Nbx9WSQolWOk5hQClMYWL56KBqChfWIxh3GTaQR5x7D/AQ6TfnOpIY
	KNVFeHHrjQIQeMePtkp97hCiregmZYs=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=IChCKY5t
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714738828; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ut+JW89x87VZepZnO2x2XzUTpzW28xGmJv03rNS6b4A=;
	b=IChCKY5tWm5MO4ZrqNegxopgfKarmL+AoRo62F1aGEhECBlAjh5e05TZZsjxEo/DQfWPQe
	Da2XiUWsxBGIlm35Nbx9WSQolWOk5hQClMYWL56KBqChfWIxh3GTaQR5x7D/AQ6TfnOpIY
	KNVFeHHrjQIQeMePtkp97hCiregmZYs=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	sstabellini@kernel.org
Subject: [GIT PULL] xen: branch for v6.9-rc7
Date: Fri,  3 May 2024 14:20:28 +0200
Message-Id: <20240503122028.16437-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Flag: NO
X-Spam-Score: -5.01
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: C743B20359
X-Spam-Level: 
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-5.01 / 50.00];
	BAYES_HAM(-3.00)[99.99%];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	MID_CONTAINS_FROM(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_MISSING_CHARSET(0.50)[];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.10)[text/plain];
	MX_GOOD(-0.01)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	MIME_TRACE(0.00)[0:+];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	ARC_NA(0.00)[];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	RCVD_TLS_ALL(0.00)[];
	DKIM_TRACE(0.00)[suse.com:+];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	FROM_HAS_DN(0.00)[];
	SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	TO_DN_NONE(0.00)[];
	RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received];
	DNSWL_BLOCKED(0.00)[2a07:de40:b281:106:10:150:64:167:received];
	RCPT_COUNT_THREE(0.00)[4];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim]

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.9a-rc7-tag

xen: branch for v6.9-rc7

It contains two fixes when running as Xen PV guests for issues
introduced in the 6.9 merge window, both related to apic id handling.


Thanks.

Juergen

 arch/x86/xen/enlighten_pv.c | 11 ++++++++++-
 arch/x86/xen/smp_pv.c       |  4 ++--
 2 files changed, 12 insertions(+), 3 deletions(-)

Juergen Gross (1):
      x86/xen: return a sane initial apic id when running as PV guest

Thomas Gleixner (1):
      x86/xen/smp_pv: Register the boot CPU APIC properly


From xen-devel-bounces@lists.xenproject.org Fri May 03 13:04:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:04:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716407.1118502 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2saf-0006dv-Oe; Fri, 03 May 2024 13:04:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716407.1118502; Fri, 03 May 2024 13:04:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2saf-0006do-Lc; Fri, 03 May 2024 13:04:41 +0000
Received: by outflank-mailman (input) for mailman id 716407;
 Fri, 03 May 2024 13:04:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s2sae-0006di-E1
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:04:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2sae-0004dB-1v; Fri, 03 May 2024 13:04:40 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2sad-00084r-SH; Fri, 03 May 2024 13:04:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6Msoz6Zv678H7c5aiwWQ9ATzQgshUlA6DxmcE0GJf/w=; b=SzgnfhV2qa632CONvy3EnpSp4G
	386EqF8kSrClU9y3PFvh0oS0CKLv1U2etK98yufilH6K07cDFqeTAL14o+fG8JPa3lNv86yWNmUul
	KKW7M8Ox+kDsd0U9kseSQtDye+BcjAWL9Am8rhJUIHwXG4UKm1yEYGMG5s8mmYCOFDng=;
Message-ID: <835099c8-6cf0-4f6d-899b-07388df89319@xen.org>
Date: Fri, 3 May 2024 14:04:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1.1] xen/commom/dt-overlay: Fix missing lock when remove
 the device
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Jan Beulich <jbeulich@suse.com>
References: <20240426015550.577986-1-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240426015550.577986-1-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 26/04/2024 02:55, Henry Wang wrote:
> If CONFIG_DEBUG=y, below assertion will be triggered:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
> (XEN) CPU:    0
> (XEN) PC:     00000a0000257418 iommu_remove_dt_device+0x8c/0xd4
> (XEN) LR:     00000a00002573a0
> (XEN) SP:     00008000fff7fb30
> (XEN) CPSR:   0000000000000249 MODE:64-bit EL2h (Hypervisor, handler)
> [...]
> 
> (XEN) Xen call trace:
> (XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
> (XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
> (XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
> (XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
> (XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
> (XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
> (XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
> (XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
> (XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
> (XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ****************************************
> 
> This is because iommu_remove_dt_device() is called without taking the
> dt_host_lock. Fix the issue by taking and releasing the lock properly.
> 
> Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v1.1:
> - Move the unlock position before the check of rc.
> ---
>   xen/common/dt-overlay.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
> index 1b197381f6..ab8f43aea2 100644
> --- a/xen/common/dt-overlay.c
> +++ b/xen/common/dt-overlay.c
> @@ -381,7 +381,9 @@ static int remove_node_resources(struct dt_device_node *device_node)
>       {
>           if ( dt_device_is_protected(device_node) )
>           {
> +            write_lock(&dt_host_lock);

Looking at the code, we are not modifying the device_node, so shouldn't 
this be a read_lock()?

That said, even though either fix your issue, I am not entirely 
convinced this is the correct position for the lock. From my 
understanding, dt_host_lock is meant to ensure that the DT node will not 
disappear behind your back. So in theory, shouldn't the lock be taken as 
soon as you get hold of device_node?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 13:10:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:10:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716413.1118513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sgU-0008AU-E9; Fri, 03 May 2024 13:10:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716413.1118513; Fri, 03 May 2024 13:10:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sgU-0008AN-AY; Fri, 03 May 2024 13:10:42 +0000
Received: by outflank-mailman (input) for mailman id 716413;
 Fri, 03 May 2024 13:10:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZLhg=MG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2sgS-0008AH-KN
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:10:40 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 89c68a1f-094e-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:10:39 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41dc9c83e57so11618245e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 06:10:39 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 k7-20020a7bc407000000b0041674bf7d4csm9232860wmi.48.2024.05.03.06.10.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 06:10:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89c68a1f-094e-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714741838; x=1715346638; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=HdAyjxljkTuPZLMR/LrbpYRYi58+ssgwbp2Exp/5Dnk=;
        b=b3UH7ZD7C5HpLurXW4zZe18u/WdoVVkfzS+JY7qn4UnIrCo/PNNAvacUOdPDl1hsRC
         nL3Hucc1ap2SaobRyyHTkBvPi8XJD44qRCw4GSDYzZt5URQptvwHFA8gk6CEO3z2jKG5
         lnkTFLUqsAu/tuqCiEtcflXMO0okpfjfwy590=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714741838; x=1715346638;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HdAyjxljkTuPZLMR/LrbpYRYi58+ssgwbp2Exp/5Dnk=;
        b=ULRXaC8YPz7caaUuZ4mEMy248GL7MQYOn/gevjARpF8fAY3Qzs5lMvDeot/B2VAeVz
         b5tGE/YLnpIJH2kfnND7pZgfnsFdGfr01aaefm/Ot0NQ8hk0GLxyAB7YahazUTwB2Qi0
         Ewrk4nbwf4BDi/OiBYJmRa2E9+2pFyF8UV3gYx6ld6Fi9OeTYKpMmVEn5VuTXgua2/Su
         1mKnauEdxzSseg9HXOOkzE9m8sxAPkpxzIJsyNH8vYA2ZadeGBMFlU42ok8oM3mcSthP
         OvdBsK6IPOu4ohyF5AXC+BD24kz/1EvfBeWLhyHP1oLXDzhBHSNt4+3ZlyVyOj/kbuwN
         G1KQ==
X-Gm-Message-State: AOJu0YwQQo/MEr37fbC+WIp/VXGkSrlzh5C44c+StTNjhsLWY9sYAXAJ
	dHb7Zewb3Jr9MDMMuTqTRF63bPd4yyzBY75LJUQZctfPebt8p2EIocdqM3yVM/xne/ZKbaueSgX
	o
X-Google-Smtp-Source: AGHT+IGxalY00lVB22PNmZRhd/SFzBdfSMjXYxLNh57K5lBTAFZODUMQ5q4dR9usGMOtVmObMJDWaQ==
X-Received: by 2002:a05:600c:1f81:b0:418:9d4a:1ba5 with SMTP id je1-20020a05600c1f8100b004189d4a1ba5mr5458255wmb.6.1714741838364;
        Fri, 03 May 2024 06:10:38 -0700 (PDT)
Date: Fri, 3 May 2024 15:10:37 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>
Subject: Re: [PATCH for-4.19? 0/2] xen/x86: support foreign mappings for HVM
Message-ID: <ZjTiTYGzEHYXpncq@macbook>
References: <20240430165845.81696-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240430165845.81696-1-roger.pau@citrix.com>

On Tue, Apr 30, 2024 at 06:58:43PM +0200, Roger Pau Monne wrote:
> Hello,
> 
> The following series attempts to solve a shortcoming of HVM/PVH guests
> with the lack of support for foreign mappings.  Such lack of support
> prevents using PVH based guests as stubdomains for example.
> 
> Add support in a way similar to how it was done on Arm, by iterating
> over the p2m based on the maximum gfn.
> 
> Mostly untested, sending early in case it could be considered for 4.19.

I've now tested this using the following dummy XTF test:

void test_main(void)
{
    unsigned long idxs = 0;
    xen_pfn_t gpfns = 0;
    int errs = 0, error;
    struct xen_add_to_physmap_batch add = {
        .domid = DOMID_SELF,
        .space = XENMAPSPACE_gmfn_foreign,
        .u.foreign_domid = 0,
        .size = 1,
        .idxs.p = &idxs,
        .gpfns.p = &gpfns,
        .errs.p = &errs,
    };

    error = hypercall_memory_op(XENMEM_add_to_physmap_batch, &add);
    if ( error || errs )
        xtf_error("add_to_physmap_batch failed: %d (errs: %d)\n", error, errs);

    while ( 1 );

    xtf_success(NULL);
}

And avoiding some of the permissions checks in Xen so that an
arbitrary domU could map dom0 gfn 0.  I see count_info increasing by 1
until the XTF guest is killed, at which point the count_info goes back
to the value previous to the map.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 03 13:10:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:10:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716414.1118523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sgl-0008U2-Lz; Fri, 03 May 2024 13:10:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716414.1118523; Fri, 03 May 2024 13:10:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sgl-0008Tv-Iw; Fri, 03 May 2024 13:10:59 +0000
Received: by outflank-mailman (input) for mailman id 716414;
 Fri, 03 May 2024 13:10:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rrlW=MG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2sgk-0008TX-Ix
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:10:58 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 92839da4-094e-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 15:10:55 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51f40b5e059so2019426e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 06:10:54 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 h2-20020a19ca42000000b005189964a79dsm542848lfj.172.2024.05.03.06.10.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 06:10:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92839da4-094e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714741854; x=1715346654; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=vPkatx33e4RXhWnky40tRoYb8cVZymug/xfvKXxqeyk=;
        b=Nv1PbvrbvlsUdTw8XHGGt7pXlnYE7WnM1dZDmMHd0xHycT8p1WH8zjnPvFo90ocvWr
         WSswQXdvnU5twtovXGFLrVGInmSCod9VxZYGZ4THHbH+hE6oV9IMV/Rf7U+MswKfC2ao
         ybvg97khDHIsNdM7jgsuXfllSjs3zH7KtuUAKUw/PmrEWc2yygWpFojt6oVf1HmdynxB
         llwZAHZOU6GaZYozS90aUCR8OJYBBJY2LIOfveXnq7VSDwQ8DozhUNzkHpTMyWaDH4XW
         lmeZD/fVI1IguibPxZ5UpzyC56VIRJNmIyisxaH/eg7wGmvsD0k1rEOKnPFv/W1/4ldM
         2afw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714741854; x=1715346654;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vPkatx33e4RXhWnky40tRoYb8cVZymug/xfvKXxqeyk=;
        b=oBtsUpZHMjPWsj2oPYewekad1yJ6qX09bMFPPT4j60uhrVqX8sziXnKq1N556Req9p
         nKtAnQ1Sh/+XV1vvUQYnmYIOFIYtpWynT/80SP99EfX4gmbP4S9d4oJx/IyR3exZMKZ6
         T/oxD/kgGaJ+phNQDumgVnXrnk3Itchf/zgjYdXQoGbm/foAfJANy7wBmXwjGdxYTBrf
         pKjnnTtcMnUDxcdAJy0WTLNan1LTyQNjfj8fV/SJ5RMOjQoYuEsjiu66zy0kKZru5ByA
         4TwsE9rknY6CKIIiwCdDhLVsqTriHkYnVnpxVbfCt6xoccaZNgD2W9UnYmCRh6GGrtYv
         GyZw==
X-Forwarded-Encrypted: i=1; AJvYcCXrmOzkv6LfUcbzVU3B8K/UYS7nstRtkA77udsp3hNhi+Yb4YqBThpVNyjkO3KpWv3Neo3k3cSL0iuNEEYyksRKiXP8s/hpad/HZWvc2kk=
X-Gm-Message-State: AOJu0Yzp66Q5fpOv6lS3xlsOrDVZ3zmsMmD31LV8EVjNyWbI95ObFnwb
	FksoNlIC80zC1kZ4iyU+by3xuOHEo7gDj9TOi39jq+ZXtIHak28Qo0zOig==
X-Google-Smtp-Source: AGHT+IGVv5A0VRacjLYP22YxqE6z3/sRcj86Kjd308oWNkB4T0aJ0+LJKYTpcKZqiEhvW46FgttkkQ==
X-Received: by 2002:ac2:44af:0:b0:51b:5c40:da74 with SMTP id c15-20020ac244af000000b0051b5c40da74mr1867044lfm.1.1714741853384;
        Fri, 03 May 2024 06:10:53 -0700 (PDT)
Message-ID: <bf162e123f0124730bf6572c415cf5eb700fe316.camel@gmail.com>
Subject: Re: [PATCH] xen/Kconfig: Drop the final remnants of ---help---
From: Oleksii <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>,  Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>
Date: Fri, 03 May 2024 15:10:52 +0200
In-Reply-To: <20240502181049.1361384-1-andrew.cooper3@citrix.com>
References: <20240502181049.1361384-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-05-02 at 19:10 +0100, Andrew Cooper wrote:
> We deprecated the use of ---help--- a while ago, but a lot of new
> content
> copy&pastes bad examples.=C2=A0 Convert the remaining instances, and
> update
> Kconfig's parser to no longer recongise it.
>=20
> This now causes builds to fail with:
>=20
> =C2=A0 Kconfig.debug:8: syntax error
> =C2=A0 Kconfig.debug:7: unknown statement "---help---"
>=20
> which short circuits one common piece of churn in new content.
>=20
> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> For 4.19.=C2=A0 This cleans up a legacy we've been wanting to get rid of
> for a
> while, and will be least disruptive on people if it gets in ahead of
> most
> people starting work for 4.20.
I am okay with commiting the patch during 4.19 release:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> =C2=A0xen/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 =
2 +-
> =C2=A0xen/Kconfig.debug=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 28 +++++++++----------
> =C2=A0xen/arch/arm/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 8 +++---
> =C2=A0xen/arch/arm/platforms/Kconfig=C2=A0 | 12 ++++-----
> =C2=A0xen/arch/x86/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 | 32 +++++++++++-----------
> =C2=A0xen/common/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 48 ++++++++++++++++---------------
> --
> =C2=A0xen/common/sched/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
| 10 +++----
> =C2=A0xen/drivers/passthrough/Kconfig |=C2=A0 8 +++---
> =C2=A0xen/drivers/video/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=
=A0 2 +-
> =C2=A0xen/tools/kconfig/lexer.l=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=
=A0 2 +-
> =C2=A010 files changed, 76 insertions(+), 76 deletions(-)
>=20
> diff --git a/xen/Kconfig b/xen/Kconfig
> index 1e1b041fd52f..e459cdac0cd7 100644
> --- a/xen/Kconfig
> +++ b/xen/Kconfig
> @@ -84,7 +84,7 @@ config UNSUPPORTED
> =C2=A0config LTO
> =C2=A0	bool "Link Time Optimisation"
> =C2=A0	depends on BROKEN
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable Link Time Optimisation.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say N.
> diff --git a/xen/Kconfig.debug b/xen/Kconfig.debug
> index fa81853e9385..61b24ac552cd 100644
> --- a/xen/Kconfig.debug
> +++ b/xen/Kconfig.debug
> @@ -4,7 +4,7 @@ menu "Debugging Options"
> =C2=A0config DEBUG
> =C2=A0	bool "Developer Checks"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 If you say Y here this will enable developer checks such
> as asserts
> =C2=A0	=C2=A0 and extra printks. This option is intended for development
> purposes
> =C2=A0	=C2=A0 only, and not for production use.
> @@ -17,14 +17,14 @@ config GDBSX
> =C2=A0	bool "Guest debugging with gdbsx"
> =C2=A0	depends on X86
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 If you want to enable support for debugging guests from
> dom0 via
> =C2=A0	=C2=A0 gdbsx then say Y.
> =C2=A0
> =C2=A0config FRAME_POINTER
> =C2=A0	bool "Compile Xen with frame pointers"
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 If you say Y here the resulting Xen will be slightly
> larger and
> =C2=A0	=C2=A0 maybe slower, but it gives very useful debugging
> information
> =C2=A0	=C2=A0 in case of any Xen bugs.
> @@ -33,7 +33,7 @@ config COVERAGE
> =C2=A0	bool "Code coverage support"
> =C2=A0	depends on !LIVEPATCH
> =C2=A0	select SUPPRESS_DUPLICATE_SYMBOL_WARNINGS if
> !ENFORCE_UNIQUE_SYMBOLS
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable code coverage support.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say N here.
> @@ -41,7 +41,7 @@ config COVERAGE
> =C2=A0config DEBUG_LOCK_PROFILE
> =C2=A0	bool "Lock Profiling"
> =C2=A0	select DEBUG_LOCKS
> -	---help---
> +	help
> =C2=A0	=C2=A0 Lock profiling allows you to see how often locks are taken
> and blocked.
> =C2=A0	=C2=A0 You can use serial console to print (and reset) using 'l'
> and 'L'
> =C2=A0	=C2=A0 respectively, or the 'xenlockprof' tool.
> @@ -49,13 +49,13 @@ config DEBUG_LOCK_PROFILE
> =C2=A0config DEBUG_LOCKS
> =C2=A0	bool "Lock debugging"
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable debugging features of lock handling.=C2=A0 Some
> additional
> =C2=A0	=C2=A0 checks will be performed when acquiring and releasing
> locks.
> =C2=A0
> =C2=A0config PERF_COUNTERS
> =C2=A0	bool "Performance Counters"
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables software performance counters that allows you to
> analyze
> =C2=A0	=C2=A0 bottlenecks in the system.=C2=A0 To access this data you ca=
n
> use serial
> =C2=A0	=C2=A0 console to print (and reset) using 'p' and 'P'
> respectively, or
> @@ -64,21 +64,21 @@ config PERF_COUNTERS
> =C2=A0config PERF_ARRAYS
> =C2=A0	bool "Performance Counter Array Histograms"
> =C2=A0	depends on PERF_COUNTERS
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables software performance counter array histograms.
> =C2=A0
> =C2=A0
> =C2=A0config VERBOSE_DEBUG
> =C2=A0	bool "Verbose debug messages"
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 Guest output from HYPERVISOR_console_io and hypervisor
> parsing
> =C2=A0	=C2=A0 ELF images (dom0) will be logged in the Xen ring buffer.
> =C2=A0
> =C2=A0config DEVICE_TREE_DEBUG
> =C2=A0	bool "Device tree debug messages"
> =C2=A0	depends on HAS_DEVICE_TREE
> -	---help---
> +	help
> =C2=A0	=C2=A0 Device tree parsing and DOM0 device tree building messages
> are
> =C2=A0	=C2=A0 logged in the Xen ring buffer.
> =C2=A0	=C2=A0 If unsure, say N here.
> @@ -86,14 +86,14 @@ config DEVICE_TREE_DEBUG
> =C2=A0config SCRUB_DEBUG
> =C2=A0	bool "Page scrubbing test"
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 Verify that pages that need to be scrubbed before being
> allocated to
> =C2=A0	=C2=A0 a guest are indeed scrubbed.
> =C2=A0
> =C2=A0config UBSAN
> =C2=A0	bool "Undefined behaviour sanitizer"
> =C2=A0	depends on HAS_UBSAN
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable undefined behaviour sanitizer. It uses compiler to
> insert code
> =C2=A0	=C2=A0 snippets so that undefined behaviours in C are detected
> during runtime.
> =C2=A0	=C2=A0 This requires a UBSAN capable compiler and it is a debug
> only feature.
> @@ -109,7 +109,7 @@ config UBSAN_FATAL
> =C2=A0
> =C2=A0config DEBUG_TRACE
> =C2=A0	bool "Debug trace support"
> -	---help---
> +	help
> =C2=A0	=C2=A0 Debug trace enables to record debug trace messages which
> are printed
> =C2=A0	=C2=A0 either directly to the console or are printed to console
> in case of
> =C2=A0	=C2=A0 a system crash.
> @@ -117,7 +117,7 @@ config DEBUG_TRACE
> =C2=A0config XMEM_POOL_POISON
> =C2=A0	bool "Poison free xenpool blocks"
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 Poison free blocks with 0xAA bytes and verify them when a
> block is
> =C2=A0	=C2=A0 allocated in order to spot use-after-free issues.
> =C2=A0
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index f8139a773a43..21d03d9f4424 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -67,7 +67,7 @@ source "arch/Kconfig"
> =C2=A0config ACPI
> =C2=A0	bool "ACPI (Advanced Configuration and Power Interface)
> Support (UNSUPPORTED)" if UNSUPPORTED
> =C2=A0	depends on ARM_64 && ARM_EFI
> -	---help---
> +	help
> =C2=A0
> =C2=A0	=C2=A0 Advanced Configuration and Power Interface (ACPI) support
> for Xen is
> =C2=A0	=C2=A0 an alternative to device tree on ARM64. This requires
> UEFI.
> @@ -102,7 +102,7 @@ config GICV3
> =C2=A0	depends on !NEW_VGIC
> =C2=A0	default n if ARM_32
> =C2=A0	default y if ARM_64
> -	---help---
> +	help
> =C2=A0
> =C2=A0	=C2=A0 Driver for the ARM Generic Interrupt Controller v3.
> =C2=A0	=C2=A0 If unsure, use the default setting.
> @@ -132,7 +132,7 @@ config HVM
> =C2=A0config NEW_VGIC
> =C2=A0	bool "Use new VGIC implementation"
> =C2=A0	select GICV2
> -	---help---
> +	help
> =C2=A0
> =C2=A0	This is an alternative implementation of the ARM GIC
> interrupt
> =C2=A0	controller emulation, based on the Linux/KVM VGIC. It has a
> better
> @@ -147,7 +147,7 @@ config NEW_VGIC
> =C2=A0config SBSA_VUART_CONSOLE
> =C2=A0	bool "Emulated SBSA UART console support"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows a guest to use SBSA Generic UART as a console. The
> =C2=A0	=C2=A0 SBSA Generic UART implements a subset of ARM PL011 UART.
> =C2=A0
> diff --git a/xen/arch/arm/platforms/Kconfig
> b/xen/arch/arm/platforms/Kconfig
> index c93a6b275620..76f7e76b1bf4 100644
> --- a/xen/arch/arm/platforms/Kconfig
> +++ b/xen/arch/arm/platforms/Kconfig
> @@ -1,14 +1,14 @@
> =C2=A0choice
> =C2=A0	prompt "Platform Support"
> =C2=A0	default ALL_PLAT
> -	---help---
> +	help
> =C2=A0	Choose which hardware platform to enable in Xen.
> =C2=A0
> =C2=A0	If unsure, choose ALL_PLAT.
> =C2=A0
> =C2=A0config ALL_PLAT
> =C2=A0	bool "All Platforms"
> -	---help---
> +	help
> =C2=A0	Enable support for all available hardware platforms. It
> doesn't
> =C2=A0	automatically select any of the related drivers.
> =C2=A0
> @@ -17,7 +17,7 @@ config QEMU
> =C2=A0	depends on ARM_64
> =C2=A0	select GICV3
> =C2=A0	select HAS_PL011
> -	---help---
> +	help
> =C2=A0	Enable all the required drivers for QEMU aarch64 virt
> emulated
> =C2=A0	machine.
> =C2=A0
> @@ -26,7 +26,7 @@ config RCAR3
> =C2=A0	depends on ARM_64
> =C2=A0	select HAS_SCIF
> =C2=A0	select IPMMU_VMSA
> -	---help---
> +	help
> =C2=A0	Enable all the required drivers for Renesas RCar3
> =C2=A0
> =C2=A0config MPSOC
> @@ -34,12 +34,12 @@ config MPSOC
> =C2=A0	depends on ARM_64
> =C2=A0	select HAS_CADENCE_UART
> =C2=A0	select ARM_SMMU
> -	---help---
> +	help
> =C2=A0	Enable all the required drivers for Xilinx Ultrascale+ MPSoC
> =C2=A0
> =C2=A0config NO_PLAT
> =C2=A0	bool "No Platforms"
> -	---help---
> +	help
> =C2=A0	Do not enable specific support for any platform.
> =C2=A0
> =C2=A0endchoice
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 2b6248774d8f..7e03e4bc5546 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -58,7 +58,7 @@ source "arch/Kconfig"
> =C2=A0config PV
> =C2=A0	def_bool y
> =C2=A0	prompt "PV support"
> -	---help---
> +	help
> =C2=A0	=C2=A0 Interfaces to support PV domains. These require guest
> kernel support
> =C2=A0	=C2=A0 to run as a PV guest, but don't require any specific
> hardware support.
> =C2=A0
> @@ -71,7 +71,7 @@ config PV32
> =C2=A0	depends on PV
> =C2=A0	default PV_SHIM
> =C2=A0	select COMPAT
> -	---help---
> +	help
> =C2=A0	=C2=A0 The 32bit PV ABI uses Ring1, an area of the x86
> architecture which
> =C2=A0	=C2=A0 was deprecated and mostly removed in the AMD64 spec.=C2=A0 =
As a
> result,
> =C2=A0	=C2=A0 it occasionally conflicts with newer x86 hardware
> features, causing
> @@ -90,7 +90,7 @@ config PV_LINEAR_PT
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bool "Support for PV linear pa=
getables"
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 depends on PV
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 default y
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ---help---
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 help
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Linear pagetables =
(also called "recursive pagetables")
> refers
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to the practice of=
 a guest operating system having
> pagetable
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 entries pointing t=
o other pagetables of the same level
> (i.e.,
> @@ -113,7 +113,7 @@ config HVM
> =C2=A0	select COMPAT
> =C2=A0	select IOREQ_SERVER
> =C2=A0	select MEM_ACCESS_ALWAYS_ON
> -	---help---
> +	help
> =C2=A0	=C2=A0 Interfaces to support HVM domains.=C2=A0 HVM domains requir=
e
> hardware
> =C2=A0	=C2=A0 virtualisation extensions (e.g. Intel VT-x, AMD SVM), but
> can boot
> =C2=A0	=C2=A0 guests which have no specific Xen knowledge.
> @@ -126,7 +126,7 @@ config XEN_SHSTK
> =C2=A0	bool "Supervisor Shadow Stacks"
> =C2=A0	depends on HAS_AS_CET_SS
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Control-flow Enforcement Technology (CET) is a set of
> features in
> =C2=A0	=C2=A0 hardware designed to combat Return-oriented Programming
> (ROP, also
> =C2=A0	=C2=A0 call/jump COP/JOP) attacks.=C2=A0 Shadow Stacks are one CET
> feature
> @@ -152,7 +152,7 @@ config SHADOW_PAGING
> =C2=A0	bool "Shadow Paging"
> =C2=A0	default !PV_SHIM_EXCLUSIVE
> =C2=A0	depends on PV || HVM
> -	---help---
> +	help
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Shadow pagin=
g is a software alternative to hardware paging
> support
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (Intel EPT, =
AMD NPT).
> @@ -171,7 +171,7 @@ config SHADOW_PAGING
> =C2=A0config BIGMEM
> =C2=A0	bool "big memory support"
> =C2=A0	default n
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows Xen to support up to 123Tb of memory.
> =C2=A0
> =C2=A0	=C2=A0 This requires enlarging struct page_info as well as
> shrinking
> @@ -183,7 +183,7 @@ config HVM_FEP
> =C2=A0	bool "HVM Forced Emulation Prefix support (UNSUPPORTED)" if
> UNSUPPORTED
> =C2=A0	default DEBUG
> =C2=A0	depends on HVM
> -	---help---
> +	help
> =C2=A0
> =C2=A0	=C2=A0 Compiles in a feature that allows HVM guest to arbitrarily
> =C2=A0	=C2=A0 exercise the instruction emulator.
> @@ -203,7 +203,7 @@ config TBOOT
> =C2=A0	depends on UNSUPPORTED
> =C2=A0	default !PV_SHIM_EXCLUSIVE
> =C2=A0	select CRYPTO
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows support for Trusted Boot using the Intel(R) Trusted
> Execution
> =C2=A0	=C2=A0 Technology (TXT)
> =C2=A0
> @@ -213,14 +213,14 @@ choice
> =C2=A0	prompt "Alignment of Xen image"
> =C2=A0	default XEN_ALIGN_2M if PV_SHIM_EXCLUSIVE
> =C2=A0	default XEN_ALIGN_DEFAULT
> -	---help---
> +	help
> =C2=A0	=C2=A0 Specify alignment for Xen image.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, choose "default".
> =C2=A0
> =C2=A0config XEN_ALIGN_DEFAULT
> =C2=A0	bool "Default alignment"
> -	---help---
> +	help
> =C2=A0	=C2=A0 Pick alignment according to build variants.
> =C2=A0
> =C2=A0	=C2=A0 For EFI build the default alignment is 2M. For ELF build
> @@ -281,7 +281,7 @@ config GUEST
> =C2=A0config XEN_GUEST
> =C2=A0	bool "Xen Guest"
> =C2=A0	select GUEST
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support for Xen detecting when it is running under Xen.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say N.
> @@ -290,7 +290,7 @@ config PVH_GUEST
> =C2=A0	def_bool y
> =C2=A0	prompt "PVH Guest"
> =C2=A0	depends on XEN_GUEST
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support booting using the PVH ABI.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say Y.
> @@ -299,7 +299,7 @@ config PV_SHIM
> =C2=A0	def_bool y
> =C2=A0	prompt "PV Shim"
> =C2=A0	depends on PV && XEN_GUEST
> -	---help---
> +	help
> =C2=A0	=C2=A0 Build Xen with a mode which acts as a shim to allow PV
> guest to run
> =C2=A0	=C2=A0 in an HVM/PVH container. This mode can only be enabled
> with command
> =C2=A0	=C2=A0 line option.
> @@ -309,7 +309,7 @@ config PV_SHIM
> =C2=A0config PV_SHIM_EXCLUSIVE
> =C2=A0	bool "PV Shim Exclusive"
> =C2=A0	depends on PV_SHIM
> -	---help---
> +	help
> =C2=A0	=C2=A0 Build Xen in a way which unconditionally assumes PV_SHIM
> mode.=C2=A0 This
> =C2=A0	=C2=A0 option is only intended for use when building a dedicated
> PV Shim
> =C2=A0	=C2=A0 firmware, and will not function correctly in other
> scenarios.
> @@ -321,7 +321,7 @@ if !PV_SHIM_EXCLUSIVE
> =C2=A0config HYPERV_GUEST
> =C2=A0	bool "Hyper-V Guest"
> =C2=A0	select GUEST
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support for Xen detecting when it is running under Hyper-
> V.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say N.
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index cff3166ff923..565ceda741b9 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -15,7 +15,7 @@ config CORE_PARKING
> =C2=A0config GRANT_TABLE
> =C2=A0	bool "Grant table support" if EXPERT
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Grant table provides a generic mechanism to memory sharing
> =C2=A0	=C2=A0 between domains. This shared memory interface underpins
> the
> =C2=A0	=C2=A0 split device drivers for block and network IO in a classic
> @@ -90,7 +90,7 @@ config MEM_ACCESS
> =C2=A0	def_bool MEM_ACCESS_ALWAYS_ON
> =C2=A0	prompt "Memory Access and VM events" if
> !MEM_ACCESS_ALWAYS_ON
> =C2=A0	depends on HVM
> -	---help---
> +	help
> =C2=A0
> =C2=A0	=C2=A0 Framework to configure memory access types for guests and
> receive
> =C2=A0	=C2=A0 related events in userspace.
> @@ -133,7 +133,7 @@ config INDIRECT_THUNK
> =C2=A0config SPECULATIVE_HARDEN_ARRAY
> =C2=A0	bool "Speculative Array Hardening"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Contemporary processors may use speculative execution as a
> =C2=A0	=C2=A0 performance optimisation, but this can potentially be
> abused by an
> =C2=A0	=C2=A0 attacker to leak data via speculative sidechannels.
> @@ -154,7 +154,7 @@ config SPECULATIVE_HARDEN_BRANCH
> =C2=A0	bool "Speculative Branch Hardening"
> =C2=A0	default y
> =C2=A0	depends on X86
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ---help---
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 help
> =C2=A0	=C2=A0 Contemporary processors may use speculative execution as a
> =C2=A0	=C2=A0 performance optimisation, but this can potentially be
> abused by an
> =C2=A0	=C2=A0 attacker to leak data via speculative sidechannels.
> @@ -228,7 +228,7 @@ config DIT_DEFAULT
> =C2=A0config HYPFS
> =C2=A0	bool "Hypervisor file system support"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support Xen hypervisor file system. This file system is
> used to
> =C2=A0	=C2=A0 present various hypervisor internal data to dom0 and in
> some
> =C2=A0	=C2=A0 cases to allow modifying settings. Disabling the support
> will
> @@ -241,7 +241,7 @@ config HYPFS_CONFIG
> =C2=A0	bool "Provide hypervisor .config via hypfs entry"
> =C2=A0	default y
> =C2=A0	depends on HYPFS
> -	---help---
> +	help
> =C2=A0	=C2=A0 When enabled the contents of the .config file used to
> build the
> =C2=A0	=C2=A0 hypervisor are provided via the hypfs entry
> /buildinfo/config.
> =C2=A0
> @@ -252,7 +252,7 @@ config IOREQ_SERVER
> =C2=A0	bool "IOREQ support (EXPERT)" if EXPERT && !X86
> =C2=A0	default X86
> =C2=A0	depends on HVM
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables generic mechanism for providing emulated devices
> to the guests.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say N.
> @@ -261,7 +261,7 @@ config KEXEC
> =C2=A0	bool "kexec support"
> =C2=A0	default y
> =C2=A0	depends on HAS_KEXEC
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows a running Xen hypervisor to be replaced with
> another OS
> =C2=A0	=C2=A0 without rebooting. This is primarily used to execute a
> crash
> =C2=A0	=C2=A0 environment to collect information on a Xen hypervisor or
> dom0 crash.
> @@ -270,7 +270,7 @@ config KEXEC
> =C2=A0
> =C2=A0config EFI_SET_VIRTUAL_ADDRESS_MAP
> =C2=A0=C2=A0=C2=A0=C2=A0 bool "EFI: call SetVirtualAddressMap()" if EXPER=
T
> -=C2=A0=C2=A0=C2=A0 ---help---
> +=C2=A0=C2=A0=C2=A0 help
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Call EFI SetVirtualAddressMap() runt=
ime service to setup
> memory map for
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 further runtime services. According =
to UEFI spec, it isn't
> strictly
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 necessary, but many UEFI implementat=
ions misbehave when this
> call is
> @@ -292,7 +292,7 @@ config XENOPROF
> =C2=A0config XSM
> =C2=A0	bool "Xen Security Modules support"
> =C2=A0	default ARM
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables the security framework known as Xen Security
> Modules which
> =C2=A0	=C2=A0 allows administrators fine-grained control over a Xen
> domain and
> =C2=A0	=C2=A0 its capabilities by defining permissible interactions
> between domains,
> @@ -305,7 +305,7 @@ config XSM_FLASK
> =C2=A0	def_bool y
> =C2=A0	prompt "FLux Advanced Security Kernel support"
> =C2=A0	depends on XSM
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables FLASK (FLux Advanced Security Kernel) as the
> access control
> =C2=A0	=C2=A0 mechanism used by the XSM framework.=C2=A0 This provides a
> mandatory access
> =C2=A0	=C2=A0 control framework by which security enforcement,
> isolation, and
> @@ -318,7 +318,7 @@ config XSM_FLASK_AVC_STATS
> =C2=A0	def_bool y
> =C2=A0	prompt "Maintain statistics on the FLASK access vector
> cache" if EXPERT
> =C2=A0	depends on XSM_FLASK
> -	---help---
> +	help
> =C2=A0	=C2=A0 Maintain counters on the access vector cache that can be
> viewed using
> =C2=A0	=C2=A0 the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.=C2=
=A0
> Disabling
> =C2=A0	=C2=A0 this will save a tiny amount of memory and time to update
> the stats.
> @@ -329,7 +329,7 @@ config XSM_FLASK_POLICY
> =C2=A0	bool "Compile Xen with a built-in FLASK security policy"
> =C2=A0	default y if "$(XEN_HAS_CHECKPOLICY)" =3D "y"
> =C2=A0	depends on XSM_FLASK
> -	---help---
> +	help
> =C2=A0	=C2=A0 This includes a default XSM policy in the hypervisor so
> that the
> =C2=A0	=C2=A0 bootloader does not need to load a policy to get sane
> behavior from an
> =C2=A0	=C2=A0 XSM-enabled hypervisor.=C2=A0 If this is disabled, a policy
> must be
> @@ -345,7 +345,7 @@ config XSM_SILO
> =C2=A0	def_bool y
> =C2=A0	prompt "SILO support"
> =C2=A0	depends on XSM
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables SILO as the access control mechanism used by the
> XSM framework.
> =C2=A0	=C2=A0 This is not the default module, add boot parameter
> xsm=3Dsilo to choose
> =C2=A0	=C2=A0 it. This will deny any unmediated communication channels
> (grant tables
> @@ -372,7 +372,7 @@ config LATE_HWDOM
> =C2=A0	bool "Dedicated hardware domain"
> =C2=A0	default n
> =C2=A0	depends on XSM && X86
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows the creation of a dedicated hardware domain
> distinct from
> =C2=A0	=C2=A0 domain 0 that manages devices without needing access to
> other
> =C2=A0	=C2=A0 privileged functionality such as the ability to manage
> domains.
> @@ -390,7 +390,7 @@ config LATE_HWDOM
> =C2=A0
> =C2=A0config ARGO
> =C2=A0	bool "Argo: hypervisor-mediated interdomain communication
> (UNSUPPORTED)" if UNSUPPORTED
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enables a hypercall for domains to ask the hypervisor to
> perform
> =C2=A0	=C2=A0 data transfer of messages between domains.
> =C2=A0
> @@ -417,7 +417,7 @@ config LIVEPATCH
> =C2=A0	default X86
> =C2=A0	depends on "$(XEN_HAS_BUILD_ID)" =3D "y"
> =C2=A0	select CC_SPLIT_SECTIONS
> -	---help---
> +	help
> =C2=A0	=C2=A0 Allows a running Xen hypervisor to be dynamically patched
> using
> =C2=A0	=C2=A0 binary patches without rebooting. This is primarily used
> to binarily
> =C2=A0	=C2=A0 patch in the field an hypervisor with XSA fixes.
> @@ -428,7 +428,7 @@ config FAST_SYMBOL_LOOKUP
> =C2=A0	bool "Fast symbol lookup (bigger binary)"
> =C2=A0	default y
> =C2=A0	depends on LIVEPATCH
> -	---help---
> +	help
> =C2=A0	=C2=A0 When searching for symbol addresses we can use the built-
> in system
> =C2=A0	=C2=A0 that is optimized for searching symbols using addresses as
> the key.
> =C2=A0	=C2=A0 However using it for the inverse (find address using the
> symbol name)
> @@ -440,7 +440,7 @@ config FAST_SYMBOL_LOOKUP
> =C2=A0config ENFORCE_UNIQUE_SYMBOLS
> =C2=A0	bool "Enforce unique symbols"
> =C2=A0	default LIVEPATCH
> -	---help---
> +	help
> =C2=A0	=C2=A0 Multiple symbols with the same name aren't generally a
> problem
> =C2=A0	=C2=A0 unless livepatching is to be used.
> =C2=A0
> @@ -454,7 +454,7 @@ config ENFORCE_UNIQUE_SYMBOLS
> =C2=A0config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
> =C2=A0	bool "Suppress duplicate symbol warnings"
> =C2=A0	depends on !ENFORCE_UNIQUE_SYMBOLS
> -	---help---
> +	help
> =C2=A0	=C2=A0 Multiple symbols with the same name aren't generally a
> problem
> =C2=A0	=C2=A0 unless Live patching is to be used, so these warnings can
> be
> =C2=A0	=C2=A0 suppressed by enabling this option.=C2=A0 Certain other opt=
ions
> (known
> @@ -464,7 +464,7 @@ config SUPPRESS_DUPLICATE_SYMBOL_WARNINGS
> =C2=A0config CMDLINE
> =C2=A0	string "Built-in hypervisor command string" if EXPERT
> =C2=A0	default ""
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enter arguments here that should be compiled into the
> hypervisor
> =C2=A0	=C2=A0 image and used at boot time. When the system boots, this
> string
> =C2=A0	=C2=A0 will be parsed prior to the bootloader command line. So if
> a
> @@ -475,7 +475,7 @@ config CMDLINE_OVERRIDE
> =C2=A0	bool "Built-in command line overrides bootloader arguments"
> =C2=A0	default n
> =C2=A0	depends on CMDLINE !=3D ""
> -	---help---
> +	help
> =C2=A0	=C2=A0 Set this option to 'Y' to have the hypervisor ignore the
> bootloader
> =C2=A0	=C2=A0 command line, and use ONLY the built-in command line.
> =C2=A0
> @@ -485,7 +485,7 @@ config CMDLINE_OVERRIDE
> =C2=A0config DOM0_MEM
> =C2=A0	string "Default value for dom0_mem boot parameter"
> =C2=A0	default ""
> -	---help---
> +	help
> =C2=A0	=C2=A0 Sets a default value for dom0_mem, e.g. "512M".
> =C2=A0	=C2=A0 The specified string will be used for the dom0_mem
> parameter in
> =C2=A0	=C2=A0 case it was not specified on the command line.
> @@ -507,7 +507,7 @@ config DTB_FILE
> =C2=A0config TRACEBUFFER
> =C2=A0	bool "Enable tracing infrastructure" if EXPERT
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable tracing infrastructure and pre-defined tracepoints
> within Xen.
> =C2=A0	=C2=A0 This will allow live information about Xen's execution and
> performance
> =C2=A0	=C2=A0 to be collected at run time for debugging or performance
> analysis.
> diff --git a/xen/common/sched/Kconfig b/xen/common/sched/Kconfig
> index b2ef0c99a3f8..18ca1ce7ab9f 100644
> --- a/xen/common/sched/Kconfig
> +++ b/xen/common/sched/Kconfig
> @@ -4,20 +4,20 @@ menu "Schedulers"
> =C2=A0config SCHED_CREDIT
> =C2=A0	bool "Credit scheduler support"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 The traditional credit scheduler is a general purpose
> scheduler.
> =C2=A0
> =C2=A0config SCHED_CREDIT2
> =C2=A0	bool "Credit2 scheduler support"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 The credit2 scheduler is a general purpose scheduler that
> is
> =C2=A0	=C2=A0 optimized for lower latency and higher VM density.
> =C2=A0
> =C2=A0config SCHED_RTDS
> =C2=A0	bool "RTDS scheduler support (UNSUPPORTED)" if UNSUPPORTED
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 The RTDS scheduler is a soft and firm real-time scheduler
> for
> =C2=A0	=C2=A0 multicore, targeted for embedded, automotive, graphics and
> gaming
> =C2=A0	=C2=A0 in the cloud, and general low-latency workloads.
> @@ -25,14 +25,14 @@ config SCHED_RTDS
> =C2=A0config SCHED_ARINC653
> =C2=A0	bool "ARINC653 scheduler support (UNSUPPORTED)" if
> UNSUPPORTED
> =C2=A0	default DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 The ARINC653 scheduler is a hard real-time scheduler for
> single
> =C2=A0	=C2=A0 cores, targeted for avionics, drones, and medical devices.
> =C2=A0
> =C2=A0config SCHED_NULL
> =C2=A0	bool "Null scheduler support (UNSUPPORTED)" if UNSUPPORTED
> =C2=A0	default PV_SHIM || DEBUG
> -	---help---
> +	help
> =C2=A0	=C2=A0 The null scheduler is a static, zero overhead scheduler,
> =C2=A0	=C2=A0 for when there always are less vCPUs than pCPUs, typically
> =C2=A0	=C2=A0 in embedded or HPC scenarios.
> diff --git a/xen/drivers/passthrough/Kconfig
> b/xen/drivers/passthrough/Kconfig
> index 864fcf3b0cef..78edd805365e 100644
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -6,7 +6,7 @@ if ARM
> =C2=A0config ARM_SMMU
> =C2=A0	bool "ARM SMMUv1 and v2 driver"
> =C2=A0	default y
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support for implementations of the ARM System MMU
> architecture
> =C2=A0	=C2=A0 versions 1 and 2.
> =C2=A0
> @@ -16,7 +16,7 @@ config ARM_SMMU
> =C2=A0config ARM_SMMU_V3
> =C2=A0	bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support" if
> EXPERT
> =C2=A0	depends on ARM_64 && (!ACPI || BROKEN)
> -	---help---
> +	help
> =C2=A0	 Support for implementations of the ARM System MMU
> architecture
> =C2=A0	 version 3. Driver is in experimental stage and should not
> be used in
> =C2=A0	 production.
> @@ -27,7 +27,7 @@ config ARM_SMMU_V3
> =C2=A0config IPMMU_VMSA
> =C2=A0	bool "Renesas IPMMU-VMSA found in R-Car Gen3/Gen4 SoCs"
> =C2=A0	depends on ARM_64
> -	---help---
> +	help
> =C2=A0	=C2=A0 Support for implementations of the Renesas IPMMU-VMSA
> found
> =C2=A0	=C2=A0 in R-Car Gen3/Gen4 SoCs.
> =C2=A0
> @@ -68,7 +68,7 @@ choice
> =C2=A0	prompt "IOMMU device quarantining default behavior"
> =C2=A0	depends on HAS_PCI
> =C2=A0	default IOMMU_QUARANTINE_BASIC
> -	---help---
> +	help
> =C2=A0	=C2=A0 When a PCI device is assigned to an untrusted domain, it
> is possible
> =C2=A0	=C2=A0 for that domain to program the device to DMA to an
> arbitrary address.
> =C2=A0	=C2=A0 The IOMMU is used to protect the host from malicious DMA
> by making
> diff --git a/xen/drivers/video/Kconfig b/xen/drivers/video/Kconfig
> index 41ca503cc961..245030beeaa2 100644
> --- a/xen/drivers/video/Kconfig
> +++ b/xen/drivers/video/Kconfig
> @@ -7,7 +7,7 @@ config VGA
> =C2=A0	select VIDEO
> =C2=A0	depends on X86
> =C2=A0	default y if !PV_SHIM_EXCLUSIVE
> -	---help---
> +	help
> =C2=A0	=C2=A0 Enable VGA output for the Xen hypervisor.
> =C2=A0
> =C2=A0	=C2=A0 If unsure, say Y.
> diff --git a/xen/tools/kconfig/lexer.l b/xen/tools/kconfig/lexer.l
> index 6354c905b006..4b7339ff4c8b 100644
> --- a/xen/tools/kconfig/lexer.l
> +++ b/xen/tools/kconfig/lexer.l
> @@ -105,7 +105,7 @@ n	[A-Za-z0-9_-]
> =C2=A0"endchoice"		return T_ENDCHOICE;
> =C2=A0"endif"			return T_ENDIF;
> =C2=A0"endmenu"		return T_ENDMENU;
> -"help"|"---help---"	return T_HELP;
> +"help"			return T_HELP;
> =C2=A0"hex"			return T_HEX;
> =C2=A0"if"			return T_IF;
> =C2=A0"imply"			return T_IMPLY;
>=20
> base-commit: feb9158a620040846d76981acbe8ea9e2255a07b



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:14:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:14:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716424.1118532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sk9-0001Gu-8M; Fri, 03 May 2024 13:14:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716424.1118532; Fri, 03 May 2024 13:14:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sk9-0001G9-5X; Fri, 03 May 2024 13:14:29 +0000
Received: by outflank-mailman (input) for mailman id 716424;
 Fri, 03 May 2024 13:14:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2sk7-0001FM-Tt
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:14:27 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 110cc71b-094f-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:14:26 +0200 (CEST)
Received: from truciolo.homenet.telecomitalia.it
 (host-79-36-52-167.retail.telecomitalia.it [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 9ECE44EE0738;
 Fri,  3 May 2024 15:14:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 110cc71b-094f-11ef-909c-e314d9c70b13
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH] automation/eclair: hide reports coming from adopted code in scheduled analysis
Date: Fri,  3 May 2024 15:14:11 +0200
Message-Id: <65364e87637d342b6b6ca5bd5ca256828ecb7a55.1714741931.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To improve clarity and ease of navigation do not show reports related
to adopted code in the scheduled analysis.
Configuration options are commented out because they may be useful
in the future.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 .../eclair_analysis/ECLAIR/analysis.ecl       | 24 +++++++++++--------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
index 66ed7f952c..67be38f03c 100644
--- a/automation/eclair_analysis/ECLAIR/analysis.ecl
+++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
@@ -4,11 +4,11 @@
 
 setq(data_dir,getenv("ECLAIR_DATA_DIR"))
 setq(analysis_kind,getenv("ANALYSIS_KIND"))
-setq(scheduled_analysis,nil)
+# setq(scheduled_analysis,nil)
 
-strings_map("scheduled-analysis",500,"","^.*scheduled$",0,setq(scheduled_analysis,t))
-strings_map("scheduled-analysis",500,"","^.*$",0)
-map_strings("scheduled-analysis",analysis_kind)
+# strings_map("scheduled-analysis",500,"","^.*scheduled$",0,setq(scheduled_analysis,t))
+# strings_map("scheduled-analysis",500,"","^.*$",0)
+# map_strings("scheduled-analysis",analysis_kind)
 
 -verbose
 
@@ -25,12 +25,16 @@ map_strings("scheduled-analysis",analysis_kind)
 -doc="Initially, there are no files tagged as adopted."
 -file_tag+={adopted,"none()"}
 
-if(not(scheduled_analysis),
-    eval_file("adopted.ecl")
-)
-if(not(scheduled_analysis),
-    eval_file("out_of_scope.ecl")
-)
+# if(not(scheduled_analysis),
+#     eval_file("adopted.ecl")
+# )
+# if(not(scheduled_analysis),
+#     eval_file("out_of_scope.ecl")
+# )
+
+-eval_file=adopted.ecl
+-eval_file=out_of_scope.ecl
+
 -eval_file=deviations.ecl
 -eval_file=call_properties.ecl
 -eval_file=tagging.ecl
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716431.1118543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sou-0001zZ-Qn; Fri, 03 May 2024 13:19:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716431.1118543; Fri, 03 May 2024 13:19:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sou-0001zR-Mz; Fri, 03 May 2024 13:19:24 +0000
Received: by outflank-mailman (input) for mailman id 716431;
 Fri, 03 May 2024 13:19:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2sos-0001z6-Sf
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:23 +0000
Received: from smtp-190e.mail.infomaniak.ch (smtp-190e.mail.infomaniak.ch
 [2001:1600:4:17::190e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bfe5f73b-094f-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 15:19:20 +0200 (CEST)
Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch
 [10.7.10.108])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGb01yhzPm8;
 Fri,  3 May 2024 15:19:19 +0200 (CEST)
Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGX3sC9zSyt; Fri,  3 May 2024 15:19:16 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfe5f73b-094f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742358;
	bh=tHNEADY7nUlqkaLY/BU1vFawJAcOU9Jd1B14MfEsU3g=;
	h=From:To:Cc:Subject:Date:From;
	b=pqC5bnOXSMYl6cDukE7A0UKRhT7LQYSKJf2OTKKA1+szdGStmWJsMTAeo8dyxqnpg
	 QwcAnPcYKS+divRw8dAghSFyO/xO9AmY+H03rDDSeG3bCQA3YIqzsYijS39CyKQSg+
	 TI9U6B0vwRusuOQxqe43g5kbuVsb+pRQFz/+RSE8=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 0/5] Hypervisor-Enforced Kernel Integrity - CR pinning
Date: Fri,  3 May 2024 15:19:05 +0200
Message-ID: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

Hi,

This patch series implements control-register (CR) pinning for KVM and
provides an hypervisor-agnostic API to protect guests.  It includes the
guest interface, the host interface, and the KVM implementation.

It's not ready for mainline yet (see the current limitations), but we
think the overall design and interfaces are good and we'd like to have
some feedback on that.

# Changes since previous version

We choose to remove as much as possible from the previous version of
this patch series to only keep the CR pinning feature and the API.  This
makes the patches simpler and brings the foundation for future
enhancement.  This will also enables us to quickly iterate on new
versions.  We are still working on memory protection but that should be
part of another patch series, if possible once this one land.

We implemented proper KUnit tests and we are also improving the test
framework to make it easier to run tests (and another series is planed):
https://lore.kernel.org/r/20240408074625.65017-1-mic@digikod.net
It makes sense to use KUnit for hypervisor-agnostic features.

This series is rebased on top of v6.9-rc6 . guest_memfd is now merged
in mainline, which will help upcoming memory-related changes.

# Overview

The main idea being that kernel self-protection mechanisms should be
delegated to a more privileged part of the system, that is the
hypervisor (see the Threat model below for more details).  It is still
the role of the guest kernel to request such restrictions according to
its configuration. The high-level security guarantees provided by the
hypervisor are semantically the same as a subset of those the kernel
already enforces on itself (CR pinning hardening), but with much strong
guarantees.

The guest kernel API layer contains a global struct heki_hypervisor to
share data and functions between the common code and the hypervisor
support code.  The struct heki_hypervisor enables to plug in different
backend implementations that are initialized with the heki_early_init()
and heki_late_init() calls.

We took inspiration from previous patches, mainly the KVMI [1] [2] and
KVM CR-pinning [3] series, revamped and simplified relevant parts to fit
well with our goal, added one hypercall, and created a kernel API for
VMs to request protection in a generic way that can be leveraged by any
hypervisor.

When a guest request to change one of its previously protected CR, KVM
creates a GP fault.

Because the VMM needs to be involved and know the guests' requested
memory permissions, we implemented two new kind of VM exits to be able
to notify the VMM about guests' Heki configurations and policy
violations.  Indeed, forwarding such signals to the VMM could help
improve attack detection, and react to such attempt (e.g. log events,
stop the VM).  Giving visibility to the VMM would also enable us to
migrate VMs.

# Threat model

The main threat model is a malicious user space process exploiting a
kernel vulnerability to gain more privileges or to bypass the
access-control system.  This threat also covers attacks coming from
network or storage data (e.g., malformed network packet, inconsistent
drive content).

Considering all potential ways to compromise a kernel, Heki's goal is to
harden a sane kernel before a runtime attack to make it more difficult,
and potentially to cause such an attack to fail. Because current attack
mitigations are only mitigations, we consider the kernel itself to be
partially malicious during its lifetime e.g., because a ROP attack that
could disable kernel self-protection mechanisms and make kernel
exploitation much easier. Indeed, an exploit is often split into several
stages, each bypassing some security measures (including CFI).

CR pinning should already be enforced by the guest kernel and the reason
to pin such registers is the same.  With this patch series it
significantly improve such protection.

Getting the guarantee that these control registers cannot be changed
increases the cost of an attack.

# Prerequisites

For this new security layer to be effective, guest kernels must be
trusted by the VM owners at boot time, before launching any user space
processes nor receiving potentially malicious network packets. It is
then required to have a security mechanism to provide or check this
initial trust (e.g., secure boot, kernel module signing).  To protect
against persistent attacks, complementary security mechanisms should be
used (e.g., IMA, IPE, Lockdown).

# How does it work?

The KVM_HC_LOCK_CR_UPDATE hypercall enables guests to pin some of its
CPU control register flags (e.g., X86_CR0_WP, X86_CR4_SMEP,
X86_CR4_SMAP).

Two new kinds of VM exits are implemented: one for a guest Heki request
(i.e. hypercall), and another for a guest attempt to change its pinned
CRs.  When the guest attempts to update pinned CRs or to access memory
in a way that is not allowed, the VMM can then be notified and react to
such attack attempt. After that, if the VM is still running, KVM sends
a GP fault to the guest. The guest could then send a signal to the user
space process that triggered this policy violation (not implemented).

Heki can be enabled with the heki=1 boot command argument.

# Similar implementations

Here is a non-exhaustive list of similar implementations that we looked
at and took some ideas from. Linux mainline doesn't support such
security features, let's change that!

Windows's Virtualization-Based Security is a proprietary technology
that provides a superset of this kind of security mechanism, relying on
Hyper-V and Virtual Trust Levels which enables to have light and secure
VM enforcing restrictions on a full guest VM. This includes several
components such as HVCI for code authenticity, or HyperGuard for
monitoring and protecting kernel code and data.

Samsung's Real-time Kernel Protection (RKP) and Huawei Hypervisor
Execution Environment (HHEE) rely on proprietary hypervisors to protect
some Android devices. They monitor critical kernel data (e.g., page
tables, credentials, selinux_enforcing).

The iOS Kernel Patch Protection (KPP/Watchtower) is a proprietary
solution running in EL3 that monitors and protects critical parts of the
kernel. It is now replaced with a hardware-based mechanism: KTTR/RoRgn.

Bitdefender's Hypervisor Memory Introspection (HVMI) is an open-source
(but out of tree) set of components leveraging virtualization. HVMI
implementation is very complex, and this approach implies potential
semantic gap issues (i.e., kernel data structures may change from one
version to another).

Linux Kernel Runtime Guard is an open-source kernel module that can
detect some kernel data illegitimate modifications. Because it is the
same kernel as the compromised one, an attacker could also bypass or
disable these checks.

Intel's Virtualization Based Hardening [4] [5] is an open-source
proof-of-concept of a thin hypervisor dedicated to guest protection. As
such, it cannot be used to manage several VMs.

# Similar Linux patches

Paravirtualized Control Register pinning [3] added a set of KVM IOCTLs
to restrict some flags to be set. Heki doesn't implement such user space
interface, but only a dedicated hypercall to lock such registers. A
superset of these flags is configurable with Heki.

The Hypervisor Based Integrity patches [6] [7] only contain a generic
IPC mechanism (KVM_HC_UCALL hypercall) to request protection to the VMM.
The idea was to extend the KVM_SET_USER_MEMORY_REGION IOCTL to support
more permission than read-only.

# Current limitations

This patch series doesn't handle VM reboot, kexec, nor hybernate yet.
We'd like to leverage the realated feature from KVM CR-pinning patch
series [3].  Help appreciated!

We noticed that the KUnit tests don't work on AMD because the exception
table seems to not be properly handled (i.e. a double fault is
received).  Any reason why this would differ from an Intel's CPU?

What about extending register pinning to MSRs?  This should first be
implemented as a kernel self-protection though.

This patch series is also a call for collaboration. There is a lot to
do, either on hypervisors, guest kernels or VMMs sides.

# Resources

You can find related resources, including previous versions, and
conference talks about this work and the related LVBS project here:
https://github.com/heki-linux

[1] https://lore.kernel.org/all/20211006173113.26445-1-alazar@bitdefender.com/
[2] https://www.linux-kvm.org/images/7/72/KVMForum2017_Introspection.pdf
[3] https://lore.kernel.org/all/20200617190757.27081-1-john.s.andersen@intel.com/
[4] https://github.com/intel/vbh
[5] https://sched.co/TmwN
[6] https://sched.co/eE3f
[7] https://lore.kernel.org/all/20200501185147.208192-1-yuanyu@google.com/

Please reach out to us by replying to this thread, we're looking for
people to join and collaborate on this project!

Previous versions:
v2: https://lore.kernel.org/r/20231113022326.24388-1-mic@digikod.net
v1: https://lore.kernel.org/r/20230505152046.6575-1-mic@digikod.net

Regards,

Madhavan T. Venkataraman (1):
  virt: Introduce Hypervisor Enforced Kernel Integrity (Heki)

Mickaël Salaün (4):
  KVM: x86: Add new hypercall to lock control registers
  KVM: x86: Add notifications for Heki policy configuration and
    violation
  heki: Lock guest control registers at the end of guest kernel init
  virt: Add Heki KUnit tests

 Documentation/virt/kvm/x86/hypercalls.rst |  17 ++
 Kconfig                                   |   2 +
 arch/x86/Kconfig                          |   1 +
 arch/x86/include/asm/x86_init.h           |   1 +
 arch/x86/include/uapi/asm/kvm_para.h      |   2 +
 arch/x86/kernel/cpu/common.c              |   7 +-
 arch/x86/kernel/cpu/hypervisor.c          |   1 +
 arch/x86/kernel/kvm.c                     |  56 +++++++
 arch/x86/kvm/Kconfig                      |   1 +
 arch/x86/kvm/vmx/vmx.c                    |   6 +
 arch/x86/kvm/x86.c                        | 180 ++++++++++++++++++++++
 arch/x86/kvm/x86.h                        |  23 +++
 include/linux/heki.h                      |  54 +++++++
 include/linux/kvm_host.h                  |   7 +
 include/uapi/linux/kvm.h                  |  22 +++
 include/uapi/linux/kvm_para.h             |   1 +
 init/main.c                               |   3 +
 mm/mm_init.c                              |   1 +
 virt/Makefile                             |   1 +
 virt/heki/.kunitconfig                    |   9 ++
 virt/heki/Kconfig                         |  43 ++++++
 virt/heki/Makefile                        |   4 +
 virt/heki/common.h                        |  16 ++
 virt/heki/heki-test.c                     | 114 ++++++++++++++
 virt/heki/main.c                          |  68 ++++++++
 25 files changed, 638 insertions(+), 2 deletions(-)
 create mode 100644 include/linux/heki.h
 create mode 100644 virt/heki/.kunitconfig
 create mode 100644 virt/heki/Kconfig
 create mode 100644 virt/heki/Makefile
 create mode 100644 virt/heki/common.h
 create mode 100644 virt/heki/heki-test.c
 create mode 100644 virt/heki/main.c


base-commit: e67572cd2204894179d89bd7b984072f19313b03
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716434.1118563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sov-0002KO-VA; Fri, 03 May 2024 13:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716434.1118563; Fri, 03 May 2024 13:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sov-0002IB-Mr; Fri, 03 May 2024 13:19:25 +0000
Received: by outflank-mailman (input) for mailman id 716434;
 Fri, 03 May 2024 13:19:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2sou-0001zB-Iu
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:24 +0000
Received: from smtp-bc0c.mail.infomaniak.ch (smtp-bc0c.mail.infomaniak.ch
 [45.157.188.12]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c1f4b22d-094f-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:19:23 +0200 (CEST)
Received: from smtp-4-0000.mail.infomaniak.ch (smtp-4-0000.mail.infomaniak.ch
 [10.7.10.107])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGg1gbwzQgV;
 Fri,  3 May 2024 15:19:23 +0200 (CEST)
Received: from unknown by smtp-4-0000.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGf1DjgzGSs; Fri,  3 May 2024 15:19:22 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1f4b22d-094f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742363;
	bh=UoeYf5nSk52yPkAQOPZ4IYRYsP6lzs+qv6PV8R5PQV8=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=HFWITQU/9X3b9ffsqT8cXmFLNi73I0373rD/PFtktQpoOH+2zmgwV5Yfw+E/F3R8K
	 ZQMa5hgDHaXOz1twAAsinnIZ+pVrQ6lX74KgknaVZVKex/5MbuFdKHrG4JhjfAKeSk
	 26YP9U0qWz7UdHB+iwnqs+toDdVpb4+HaghZoTfY=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy configuration and violation
Date: Fri,  3 May 2024 15:19:08 +0200
Message-ID: <20240503131910.307630-4-mic@digikod.net>
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

Add an interface for user space to be notified about guests' Heki policy
and related violations.

Extend the KVM_ENABLE_CAP IOCTL with KVM_CAP_HEKI_CONFIGURE and
KVM_CAP_HEKI_DENIAL. Each one takes a bitmask as first argument that can
contains KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4. The
returned value is the bitmask of known Heki exit reasons, for now:
KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4.

If KVM_CAP_HEKI_CONFIGURE is set, a VM exit will be triggered for each
KVM_HC_LOCK_CR_UPDATE hypercalls according to the requested control
register. This enables to enlighten the VMM with the guest
auto-restrictions.

If KVM_CAP_HEKI_DENIAL is set, a VM exit will be triggered for each
pinned CR violation. This enables the VMM to react to a policy
violation.

Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Link: https://lore.kernel.org/r/20240503131910.307630-4-mic@digikod.net
---

Changes since v1:
* New patch. Making user space aware of Heki properties was requested by
  Sean Christopherson.
---
 arch/x86/kvm/vmx/vmx.c   |   5 +-
 arch/x86/kvm/x86.c       | 114 +++++++++++++++++++++++++++++++++++----
 arch/x86/kvm/x86.h       |   7 +--
 include/linux/kvm_host.h |   2 +
 include/uapi/linux/kvm.h |  22 ++++++++
 5 files changed, 136 insertions(+), 14 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 7ba970b525f7..5869a1ed7866 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -5445,6 +5445,7 @@ static int handle_cr(struct kvm_vcpu *vcpu)
 	int reg;
 	int err;
 	int ret;
+	bool exit = false;
 
 	exit_qualification = vmx_get_exit_qual(vcpu);
 	cr = exit_qualification & 15;
@@ -5454,8 +5455,8 @@ static int handle_cr(struct kvm_vcpu *vcpu)
 		val = kvm_register_read(vcpu, reg);
 		trace_kvm_cr_write(cr, val);
 
-		ret = heki_check_cr(vcpu, cr, val);
-		if (ret)
+		ret = heki_check_cr(vcpu, cr, val, &exit);
+		if (exit)
 			return ret;
 
 		switch (cr) {
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a5f47be59abc..865e88f2b0fc 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -119,6 +119,10 @@ static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS;
 
 #define KVM_CAP_PMU_VALID_MASK KVM_PMU_CAP_DISABLE
 
+#define KVM_HEKI_EXIT_REASON_VALID_MASK ( \
+	KVM_HEKI_EXIT_REASON_CR0 | \
+	KVM_HEKI_EXIT_REASON_CR4)
+
 #define KVM_X2APIC_API_VALID_FLAGS (KVM_X2APIC_API_USE_32BIT_IDS | \
                                     KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK)
 
@@ -4836,6 +4840,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		if (kvm_is_vm_type_supported(KVM_X86_SW_PROTECTED_VM))
 			r |= BIT(KVM_X86_SW_PROTECTED_VM);
 		break;
+	case KVM_CAP_HEKI_CONFIGURE:
+	case KVM_CAP_HEKI_DENIAL:
+		r = KVM_HEKI_EXIT_REASON_VALID_MASK;
+		break;
 	default:
 		break;
 	}
@@ -6729,6 +6737,22 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		}
 		mutex_unlock(&kvm->lock);
 		break;
+#ifdef CONFIG_HEKI
+	case KVM_CAP_HEKI_CONFIGURE:
+		r = -EINVAL;
+		if (cap->args[0] & ~KVM_HEKI_EXIT_REASON_VALID_MASK)
+			break;
+		kvm->heki_configure_exit_reason = cap->args[0];
+		r = 0;
+		break;
+	case KVM_CAP_HEKI_DENIAL:
+		r = -EINVAL;
+		if (cap->args[0] & ~KVM_HEKI_EXIT_REASON_VALID_MASK)
+			break;
+		kvm->heki_denial_exit_reason = cap->args[0];
+		r = 0;
+		break;
+#endif
 	default:
 		r = -EINVAL;
 		break;
@@ -8283,11 +8307,60 @@ static unsigned long emulator_get_cr(struct x86_emulate_ctxt *ctxt, int cr)
 
 #ifdef CONFIG_HEKI
 
+static int complete_heki_configure_exit(struct kvm_vcpu *const vcpu)
+{
+	kvm_rax_write(vcpu, 0);
+	++vcpu->stat.hypercalls;
+	return kvm_skip_emulated_instruction(vcpu);
+}
+
+static int complete_heki_denial_exit(struct kvm_vcpu *const vcpu)
+{
+	kvm_inject_gp(vcpu, 0);
+	return 1;
+}
+
+/* Returns true if the @exit_reason is handled by @vcpu->kvm. */
+static bool heki_exit_cr(struct kvm_vcpu *const vcpu, const __u32 exit_reason,
+			 const u64 heki_reason, unsigned long value)
+{
+	switch (exit_reason) {
+	case KVM_EXIT_HEKI_CONFIGURE:
+		if (!(vcpu->kvm->heki_configure_exit_reason & heki_reason))
+			return false;
+
+		vcpu->run->heki_configure.reason = heki_reason;
+		memset(vcpu->run->heki_configure.reserved, 0,
+		       sizeof(vcpu->run->heki_configure.reserved));
+		vcpu->run->heki_configure.cr_pinned = value;
+		vcpu->arch.complete_userspace_io = complete_heki_configure_exit;
+		break;
+	case KVM_EXIT_HEKI_DENIAL:
+		if (!(vcpu->kvm->heki_denial_exit_reason & heki_reason))
+			return false;
+
+		vcpu->run->heki_denial.reason = heki_reason;
+		memset(vcpu->run->heki_denial.reserved, 0,
+		       sizeof(vcpu->run->heki_denial.reserved));
+		vcpu->run->heki_denial.cr_value = value;
+		vcpu->arch.complete_userspace_io = complete_heki_denial_exit;
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		return false;
+	}
+
+	vcpu->run->exit_reason = exit_reason;
+	return true;
+}
+
 #define HEKI_ABI_VERSION 1
 
 static int heki_lock_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
-			unsigned long pin, unsigned long flags)
+			unsigned long pin, unsigned long flags, bool *exit)
 {
+	*exit = false;
+
 	if (flags) {
 		if ((flags == KVM_LOCK_CR_UPDATE_VERSION) && !cr && !pin)
 			return HEKI_ABI_VERSION;
@@ -8307,6 +8380,8 @@ static int heki_lock_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
 			return -KVM_EINVAL;
 
 		atomic_long_or(pin, &vcpu->kvm->heki_pinned_cr0);
+		*exit = heki_exit_cr(vcpu, KVM_EXIT_HEKI_CONFIGURE,
+				     KVM_HEKI_EXIT_REASON_CR0, pin);
 		return 0;
 	case 4:
 		/* Checks for irrelevant bits. */
@@ -8316,24 +8391,37 @@ static int heki_lock_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
 		/* Ignores bits not present in host. */
 		pin &= __read_cr4();
 		atomic_long_or(pin, &vcpu->kvm->heki_pinned_cr4);
+		*exit = heki_exit_cr(vcpu, KVM_EXIT_HEKI_CONFIGURE,
+				     KVM_HEKI_EXIT_REASON_CR4, pin);
 		return 0;
 	}
 	return -KVM_EINVAL;
 }
 
+/*
+ * Sets @exit to true if the caller must exit (i.e. denied access) with the
+ * returned value:
+ * - 0 when kvm_run is configured;
+ * - 1 when there is no user space handler.
+ */
 int heki_check_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
-		  const unsigned long val)
+		  const unsigned long val, bool *exit)
 {
 	unsigned long pinned;
 
+	*exit = false;
+
 	switch (cr) {
 	case 0:
 		pinned = atomic_long_read(&vcpu->kvm->heki_pinned_cr0);
 		if ((val & pinned) != pinned) {
 			pr_warn_ratelimited(
 				"heki: Blocked CR0 update: 0x%lx\n", val);
-			kvm_inject_gp(vcpu, 0);
-			return 1;
+			*exit = true;
+			if (heki_exit_cr(vcpu, KVM_EXIT_HEKI_DENIAL,
+					 KVM_HEKI_EXIT_REASON_CR0, val))
+				return 0;
+			return complete_heki_denial_exit(vcpu);
 		}
 		return 0;
 	case 4:
@@ -8341,8 +8429,11 @@ int heki_check_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
 		if ((val & pinned) != pinned) {
 			pr_warn_ratelimited(
 				"heki: Blocked CR4 update: 0x%lx\n", val);
-			kvm_inject_gp(vcpu, 0);
-			return 1;
+			*exit = true;
+			if (heki_exit_cr(vcpu, KVM_EXIT_HEKI_DENIAL,
+					 KVM_HEKI_EXIT_REASON_CR4, val))
+				return 0;
+			return complete_heki_denial_exit(vcpu);
 		}
 		return 0;
 	}
@@ -8356,9 +8447,10 @@ static int emulator_set_cr(struct x86_emulate_ctxt *ctxt, int cr, ulong val)
 {
 	struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
 	int res = 0;
+	bool exit = false;
 
-	res = heki_check_cr(vcpu, cr, val);
-	if (res)
+	res = heki_check_cr(vcpu, cr, val, &exit);
+	if (exit)
 		return res;
 
 	switch (cr) {
@@ -10222,7 +10314,11 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 		if (a0 > U32_MAX) {
 			ret = -KVM_EINVAL;
 		} else {
-			ret = heki_lock_cr(vcpu, a0, a1, a2);
+			bool exit = false;
+
+			ret = heki_lock_cr(vcpu, a0, a1, a2, &exit);
+			if (exit)
+				return ret;
 		}
 		break;
 #endif /* CONFIG_HEKI */
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index ade7d68ddaff..2740b74ab583 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -292,18 +292,19 @@ static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk)
 
 #ifdef CONFIG_HEKI
 
-int heki_check_cr(struct kvm_vcpu *vcpu, unsigned long cr, unsigned long val);
+int heki_check_cr(struct kvm_vcpu *vcpu, unsigned long cr, unsigned long val,
+		  bool *exit);
 
 #else /* CONFIG_HEKI */
 
 static inline int heki_check_cr(struct kvm_vcpu *vcpu, unsigned long cr,
-				unsigned long val)
+				unsigned long val, bool *exit)
 {
 	return 0;
 }
 
 static inline int heki_lock_cr(struct kvm_vcpu *const vcpu, unsigned long cr,
-			       unsigned long pin)
+			       unsigned long pin, bool *exit)
 {
 	return 0;
 }
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6ff13937929a..cf8e271d47aa 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -839,6 +839,8 @@ struct kvm {
 #ifdef CONFIG_HEKI
 	atomic_long_t heki_pinned_cr0;
 	atomic_long_t heki_pinned_cr4;
+	u64 heki_configure_exit_reason;
+	u64 heki_denial_exit_reason;
 #endif /* CONFIG_HEKI */
 
 #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 2190adbe3002..1051c2f817ba 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -178,6 +178,8 @@ struct kvm_xen_exit {
 #define KVM_EXIT_NOTIFY           37
 #define KVM_EXIT_LOONGARCH_IOCSR  38
 #define KVM_EXIT_MEMORY_FAULT     39
+#define KVM_EXIT_HEKI_CONFIGURE   40
+#define KVM_EXIT_HEKI_DENIAL      41
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -433,6 +435,24 @@ struct kvm_run {
 			__u64 gpa;
 			__u64 size;
 		} memory_fault;
+		/* KVM_EXIT_HEKI_CONFIGURE */
+		struct {
+#define KVM_HEKI_EXIT_REASON_CR0	(1ULL << 0)
+#define KVM_HEKI_EXIT_REASON_CR4	(1ULL << 1)
+			__u64 reason;
+			union {
+				__u64 cr_pinned;
+				__u64 reserved[7]; /* ignored */
+			};
+		} heki_configure;
+		/* KVM_EXIT_HEKI_DENIAL */
+		struct {
+			__u64 reason;
+			union {
+				__u64 cr_value;
+				__u64 reserved[7]; /* ignored */
+			};
+		} heki_denial;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -917,6 +937,8 @@ struct kvm_enable_cap {
 #define KVM_CAP_MEMORY_ATTRIBUTES 233
 #define KVM_CAP_GUEST_MEMFD 234
 #define KVM_CAP_VM_TYPES 235
+#define KVM_CAP_HEKI_CONFIGURE 236
+#define KVM_CAP_HEKI_DENIAL 237
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716435.1118584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sox-0002tp-7g; Fri, 03 May 2024 13:19:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716435.1118584; Fri, 03 May 2024 13:19:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sox-0002tg-1a; Fri, 03 May 2024 13:19:27 +0000
Received: by outflank-mailman (input) for mailman id 716435;
 Fri, 03 May 2024 13:19:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2sov-0001zB-PM
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:25 +0000
Received: from smtp-42aa.mail.infomaniak.ch (smtp-42aa.mail.infomaniak.ch
 [84.16.66.170]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c2d2fc61-094f-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:19:24 +0200 (CEST)
Received: from smtp-3-0000.mail.infomaniak.ch (smtp-3-0000.mail.infomaniak.ch
 [10.4.36.107])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGh4fbkzS3b;
 Fri,  3 May 2024 15:19:24 +0200 (CEST)
Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGg5Ql1zZFN; Fri,  3 May 2024 15:19:23 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2d2fc61-094f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742364;
	bh=5C2oJwkaABGy0N3TgDchTsaJRP0NaN1sG1zefdnqG5Y=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=YqIvlgiE19ek2V9SMYQfmDo9Wa87WmiGMnbGB2jn2HxiItp8TNw5d9eeeBTU8LpK3
	 AZT0MKxeoxXmSYrZ0EETcC/IslnkqX1RyIwnBoKBpYfJ00vOsRqigZFHpCXr5dOfgY
	 YVXF5SDClUmkftFfBhN3Uo9dR3fSNJeaCw/yx6WM=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 4/5] heki: Lock guest control registers at the end of guest kernel init
Date: Fri,  3 May 2024 15:19:09 +0200
Message-ID: <20240503131910.307630-5-mic@digikod.net>
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

The hypervisor needs to provide some functions to support Heki. These
form the Heki-Hypervisor API.

Define a heki_hypervisor structure to house the API functions. A
hypervisor that supports Heki must instantiate a heki_hypervisor
structure and pass it to the Heki common code. This allows the common
code to access these functions in a hypervisor-agnostic way.

The first function that is implemented is lock_crs() (lock control
registers). That is, certain flags in the control registers are pinned
so that they can never be changed for the lifetime of the guest.

Implement Heki support in the guest:

- Each supported hypervisor in x86 implements a set of functions for the
  guest kernel. Add an init_heki() function to that set.  This function
  initializes Heki-related stuff. Call init_heki() for the detected
  hypervisor in init_hypervisor_platform().

- Implement init_heki() for the guest.

- Implement kvm_lock_crs() in the guest to lock down control registers.
  This function calls a KVM hypercall to do the job.

- Instantiate a heki_hypervisor structure that contains a pointer to
  kvm_lock_crs().

- Pass the heki_hypervisor structure to Heki common code in init_heki().

Implement a heki_late_init() function and call it at the end of kernel
init. This function calls lock_crs(). In other words, control registers
of a guest are locked down at the end of guest kernel init.

Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Wanpeng Li <wanpengli@tencent.com>
Co-developed-by: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
Signed-off-by: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Link: https://lore.kernel.org/r/20240503131910.307630-5-mic@digikod.net
---

Changes since v2:
* Hide CONFIG_HYPERVISOR_SUPPORTS_HEKI from users.

Changes since v1:
* Shrinked the patch to only manage the CR pinning.
---
 arch/x86/include/asm/x86_init.h  |  1 +
 arch/x86/kernel/cpu/hypervisor.c |  1 +
 arch/x86/kernel/kvm.c            | 56 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/Kconfig             |  1 +
 include/linux/heki.h             | 22 +++++++++++++
 init/main.c                      |  1 +
 virt/heki/Kconfig                |  8 ++++-
 virt/heki/main.c                 | 25 ++++++++++++++
 8 files changed, 114 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index 6149eabe200f..113998799473 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -128,6 +128,7 @@ struct x86_hyper_init {
 	bool (*msi_ext_dest_id)(void);
 	void (*init_mem_mapping)(void);
 	void (*init_after_bootmem)(void);
+	void (*init_heki)(void);
 };
 
 /**
diff --git a/arch/x86/kernel/cpu/hypervisor.c b/arch/x86/kernel/cpu/hypervisor.c
index 553bfbfc3a1b..6085c8129e0c 100644
--- a/arch/x86/kernel/cpu/hypervisor.c
+++ b/arch/x86/kernel/cpu/hypervisor.c
@@ -106,4 +106,5 @@ void __init init_hypervisor_platform(void)
 
 	x86_hyper_type = h->type;
 	x86_init.hyper.init_platform();
+	x86_init.hyper.init_heki();
 }
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 7f0732bc0ccd..a54f2c0d7cd0 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -29,6 +29,7 @@
 #include <linux/syscore_ops.h>
 #include <linux/cc_platform.h>
 #include <linux/efi.h>
+#include <linux/heki.h>
 #include <asm/timer.h>
 #include <asm/cpu.h>
 #include <asm/traps.h>
@@ -999,6 +1000,60 @@ static bool kvm_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs)
 }
 #endif
 
+#ifdef CONFIG_HEKI
+
+extern unsigned long cr4_pinned_mask;
+
+/*
+ * TODO: Check SMP policy consistency, e.g. with
+ * this_cpu_read(cpu_tlbstate.cr4)
+ */
+static int kvm_lock_crs(void)
+{
+	unsigned long cr4;
+	int err;
+
+	err = kvm_hypercall3(KVM_HC_LOCK_CR_UPDATE, 0, X86_CR0_WP, 0);
+	if (err)
+		return err;
+
+	cr4 = __read_cr4();
+	err = kvm_hypercall3(KVM_HC_LOCK_CR_UPDATE, 4, cr4 & cr4_pinned_mask,
+			     0);
+	return err;
+}
+
+static struct heki_hypervisor kvm_heki_hypervisor = {
+	.lock_crs = kvm_lock_crs,
+};
+
+static void kvm_init_heki(void)
+{
+	long err;
+
+	if (!kvm_para_available()) {
+		/* Cannot make KVM hypercalls. */
+		return;
+	}
+
+	err = kvm_hypercall3(KVM_HC_LOCK_CR_UPDATE, 0, 0,
+			     KVM_LOCK_CR_UPDATE_VERSION);
+	if (err < 1) {
+		/* Ignores host not supporting at least the first version. */
+		return;
+	}
+
+	heki.hypervisor = &kvm_heki_hypervisor;
+}
+
+#else /* CONFIG_HEKI */
+
+static void kvm_init_heki(void)
+{
+}
+
+#endif /* CONFIG_HEKI */
+
 const __initconst struct hypervisor_x86 x86_hyper_kvm = {
 	.name				= "KVM",
 	.detect				= kvm_detect,
@@ -1007,6 +1062,7 @@ const __initconst struct hypervisor_x86 x86_hyper_kvm = {
 	.init.x2apic_available		= kvm_para_available,
 	.init.msi_ext_dest_id		= kvm_msi_ext_dest_id,
 	.init.init_platform		= kvm_init_platform,
+	.init.init_heki			= kvm_init_heki,
 #if defined(CONFIG_AMD_MEM_ENCRYPT)
 	.runtime.sev_es_hcall_prepare	= kvm_sev_es_hcall_prepare,
 	.runtime.sev_es_hcall_finish	= kvm_sev_es_hcall_finish,
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index 0ebdd088f28b..68e0e8d7230a 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -44,6 +44,7 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_PM_NOTIFIER if PM
 	select KVM_GENERIC_HARDWARE_ENABLING
+	select HYPERVISOR_SUPPORTS_HEKI
 	help
 	  Support hosting fully virtualized guest machines using hardware
 	  virtualization extensions.  You will need a fairly recent
diff --git a/include/linux/heki.h b/include/linux/heki.h
index 4c18d2283392..96ccb17657e5 100644
--- a/include/linux/heki.h
+++ b/include/linux/heki.h
@@ -9,6 +9,7 @@
 #define __HEKI_H__
 
 #include <linux/types.h>
+#include <linux/bug.h>
 #include <linux/cache.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -16,15 +17,36 @@
 
 #ifdef CONFIG_HEKI
 
+/*
+ * A hypervisor that supports Heki will instantiate this structure to
+ * provide hypervisor specific functions for Heki.
+ */
+struct heki_hypervisor {
+	int (*lock_crs)(void); /* Lock control registers. */
+};
+
+/*
+ * If the active hypervisor supports Heki, it will plug its heki_hypervisor
+ * pointer into this heki structure.
+ */
+struct heki {
+	struct heki_hypervisor *hypervisor;
+};
+
+extern struct heki heki;
 extern bool heki_enabled;
 
 void heki_early_init(void);
+void heki_late_init(void);
 
 #else /* !CONFIG_HEKI */
 
 static inline void heki_early_init(void)
 {
 }
+static inline void heki_late_init(void)
+{
+}
 
 #endif /* CONFIG_HEKI */
 
diff --git a/init/main.c b/init/main.c
index bec2c8d939aa..c2dc663ab4b5 100644
--- a/init/main.c
+++ b/init/main.c
@@ -1454,6 +1454,7 @@ static int __ref kernel_init(void *unused)
 	exit_boot_config();
 	free_initmem();
 	mark_readonly();
+	heki_late_init();
 
 	/*
 	 * Kernel mappings are now finalized - update the userspace page-table
diff --git a/virt/heki/Kconfig b/virt/heki/Kconfig
index 66e73d212856..0c764e342f48 100644
--- a/virt/heki/Kconfig
+++ b/virt/heki/Kconfig
@@ -8,6 +8,12 @@ config ARCH_SUPPORTS_HEKI
 	# and run with CONFIG_HEKI. That is, it should provide all of the
 	# architecture support required for the HEKI feature.
 
+config HYPERVISOR_SUPPORTS_HEKI
+	bool
+	# A hypervisor should select this when it can successfully build
+	# and run with CONFIG_HEKI. That is, it should provide all of the
+	# hypervisor support required for the Heki feature.
+
 menuconfig HEKI_MENU
 	bool "Virtualization hardening"
 
@@ -15,7 +21,7 @@ if HEKI_MENU
 
 config HEKI
 	bool "Hypervisor Enforced Kernel Integrity (Heki)"
-	depends on ARCH_SUPPORTS_HEKI
+	depends on ARCH_SUPPORTS_HEKI && HYPERVISOR_SUPPORTS_HEKI
 	help
 	  This feature enhances guest virtual machine security by taking
 	  advantage of security features provided by the hypervisor for guests.
diff --git a/virt/heki/main.c b/virt/heki/main.c
index 25c25f5700f7..ef0530a03e09 100644
--- a/virt/heki/main.c
+++ b/virt/heki/main.c
@@ -11,6 +11,7 @@
 #include "common.h"
 
 bool heki_enabled __ro_after_init = true;
+struct heki heki;
 
 /*
  * Must be called after kmem_cache_init().
@@ -22,6 +23,30 @@ __init void heki_early_init(void)
 		return;
 	}
 	pr_warn("Heki is enabled\n");
+
+	if (!heki.hypervisor) {
+		/* This happens for kernels running on bare metal as well. */
+		pr_warn("No support for Heki in the active hypervisor\n");
+		return;
+	}
+	pr_warn("Heki is supported by the active Hypervisor\n");
+}
+
+/*
+ * Must be called after mark_readonly().
+ */
+void heki_late_init(void)
+{
+	struct heki_hypervisor *hypervisor = heki.hypervisor;
+
+	if (!heki_enabled || !heki.hypervisor)
+		return;
+
+	/* Locks control registers so a compromised guest cannot change them. */
+	if (WARN_ON(hypervisor->lock_crs()))
+		return;
+
+	pr_warn("Control registers locked\n");
 }
 
 static int __init heki_parse_config(char *str)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716432.1118549 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sov-00022p-4E; Fri, 03 May 2024 13:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716432.1118549; Fri, 03 May 2024 13:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sou-00021y-UR; Fri, 03 May 2024 13:19:24 +0000
Received: by outflank-mailman (input) for mailman id 716432;
 Fri, 03 May 2024 13:19:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2sot-0001z6-NA
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:23 +0000
Received: from smtp-bc0d.mail.infomaniak.ch (smtp-bc0d.mail.infomaniak.ch
 [45.157.188.13]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c04244f9-094f-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 15:19:20 +0200 (CEST)
Received: from smtp-3-0000.mail.infomaniak.ch (smtp-3-0000.mail.infomaniak.ch
 [10.4.36.107])
 by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGc2g3hzS10;
 Fri,  3 May 2024 15:19:20 +0200 (CEST)
Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGb2zTRzbPY; Fri,  3 May 2024 15:19:19 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c04244f9-094f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742360;
	bh=obtQipiNRMFv28P9TKvF5pV8leBuQgSBtBcEQW3pxVc=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=NvcPqXJFBA2qsjuhqiLltCK88ANNHdMNpuz17Tcd63LbmHTm/KkiZOaiSif0w7f+4
	 cftpGAbztXjbX5M6Zeun4BIRd81twRoYYKlWzYdwI+wJzU1gYbqcjIInB4q/5H0LNP
	 PPfyGOD1eQbV3Tnb/7BmebaPzISSc/RgHtpbrUuw=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 1/5] virt: Introduce Hypervisor Enforced Kernel Integrity (Heki)
Date: Fri,  3 May 2024 15:19:06 +0200
Message-ID: <20240503131910.307630-2-mic@digikod.net>
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

From: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>

Hypervisor Enforced Kernel Integrity (Heki) is a feature that will use
the hypervisor to enhance guest virtual machine security.

Implement minimal code to introduce Heki:

- Define the config variables.

- Define a kernel command line parameter "heki" to turn the feature
  on or off. By default, Heki is on.

- Define heki_early_init() and call it in start_kernel(). Currently,
  this function only prints the value of the "heki" command
  line parameter.

Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Wanpeng Li <wanpengli@tencent.com>
Co-developed-by: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
Link: https://lore.kernel.org/r/20240503131910.307630-2-mic@digikod.net
---

Changes since v2:
* Move CONFIG_HEKI under a new CONFIG_HEKI_MENU to group it with the
  test configuration (see following patches).
* Hide CONFIG_ARCH_SUPPORS_HEKI from users.

Changes since v1:
* Shrinked this patch to only contain the minimal common parts.
* Moved heki_early_init() to start_kernel().
* Use kstrtobool().
---
 Kconfig              |  2 ++
 arch/x86/Kconfig     |  1 +
 include/linux/heki.h | 31 +++++++++++++++++++++++++++++++
 init/main.c          |  2 ++
 mm/mm_init.c         |  1 +
 virt/Makefile        |  1 +
 virt/heki/Kconfig    | 25 +++++++++++++++++++++++++
 virt/heki/Makefile   |  3 +++
 virt/heki/common.h   | 16 ++++++++++++++++
 virt/heki/main.c     | 33 +++++++++++++++++++++++++++++++++
 10 files changed, 115 insertions(+)
 create mode 100644 include/linux/heki.h
 create mode 100644 virt/heki/Kconfig
 create mode 100644 virt/heki/Makefile
 create mode 100644 virt/heki/common.h
 create mode 100644 virt/heki/main.c

diff --git a/Kconfig b/Kconfig
index 745bc773f567..0c844d9bcb03 100644
--- a/Kconfig
+++ b/Kconfig
@@ -29,4 +29,6 @@ source "lib/Kconfig"
 
 source "lib/Kconfig.debug"
 
+source "virt/heki/Kconfig"
+
 source "Documentation/Kconfig"
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 928820e61cb5..d2fba63c289b 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -34,6 +34,7 @@ config X86_64
 	select SWIOTLB
 	select ARCH_HAS_ELFCORE_COMPAT
 	select ZONE_DMA32
+	select ARCH_SUPPORTS_HEKI
 
 config FORCE_DYNAMIC_FTRACE
 	def_bool y
diff --git a/include/linux/heki.h b/include/linux/heki.h
new file mode 100644
index 000000000000..4c18d2283392
--- /dev/null
+++ b/include/linux/heki.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hypervisor Enforced Kernel Integrity (Heki) - Definitions
+ *
+ * Copyright © 2023 Microsoft Corporation
+ */
+
+#ifndef __HEKI_H__
+#define __HEKI_H__
+
+#include <linux/types.h>
+#include <linux/cache.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/printk.h>
+
+#ifdef CONFIG_HEKI
+
+extern bool heki_enabled;
+
+void heki_early_init(void);
+
+#else /* !CONFIG_HEKI */
+
+static inline void heki_early_init(void)
+{
+}
+
+#endif /* CONFIG_HEKI */
+
+#endif /* __HEKI_H__ */
diff --git a/init/main.c b/init/main.c
index 5dcf5274c09c..bec2c8d939aa 100644
--- a/init/main.c
+++ b/init/main.c
@@ -102,6 +102,7 @@
 #include <linux/randomize_kstack.h>
 #include <linux/pidfs.h>
 #include <linux/ptdump.h>
+#include <linux/heki.h>
 #include <net/net_namespace.h>
 
 #include <asm/io.h>
@@ -1059,6 +1060,7 @@ void start_kernel(void)
 	uts_ns_init();
 	key_init();
 	security_init();
+	heki_early_init();
 	dbg_late_init();
 	net_ns_init();
 	vfs_caches_init();
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 549e76af8f82..89d9f97bd471 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -27,6 +27,7 @@
 #include <linux/swap.h>
 #include <linux/cma.h>
 #include <linux/crash_dump.h>
+#include <linux/heki.h>
 #include "internal.h"
 #include "slab.h"
 #include "shuffle.h"
diff --git a/virt/Makefile b/virt/Makefile
index 1cfea9436af9..856b5ccedb5a 100644
--- a/virt/Makefile
+++ b/virt/Makefile
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-y	+= lib/
+obj-$(CONFIG_HEKI_MENU) += heki/
diff --git a/virt/heki/Kconfig b/virt/heki/Kconfig
new file mode 100644
index 000000000000..66e73d212856
--- /dev/null
+++ b/virt/heki/Kconfig
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Hypervisor Enforced Kernel Integrity (Heki)
+
+config ARCH_SUPPORTS_HEKI
+	bool
+	# An architecture should select this when it can successfully build
+	# and run with CONFIG_HEKI. That is, it should provide all of the
+	# architecture support required for the HEKI feature.
+
+menuconfig HEKI_MENU
+	bool "Virtualization hardening"
+
+if HEKI_MENU
+
+config HEKI
+	bool "Hypervisor Enforced Kernel Integrity (Heki)"
+	depends on ARCH_SUPPORTS_HEKI
+	help
+	  This feature enhances guest virtual machine security by taking
+	  advantage of security features provided by the hypervisor for guests.
+	  This feature is helpful in maintaining guest virtual machine security
+	  even after the guest kernel has been compromised.
+
+endif
diff --git a/virt/heki/Makefile b/virt/heki/Makefile
new file mode 100644
index 000000000000..8b10e73a154b
--- /dev/null
+++ b/virt/heki/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_HEKI) += main.o
diff --git a/virt/heki/common.h b/virt/heki/common.h
new file mode 100644
index 000000000000..edd98fc650a8
--- /dev/null
+++ b/virt/heki/common.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hypervisor Enforced Kernel Integrity (Heki) - Common header
+ *
+ * Copyright © 2023 Microsoft Corporation
+ */
+
+#ifndef _HEKI_COMMON_H
+
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+
+#define pr_fmt(fmt) "heki-guest: " fmt
+
+#endif /* _HEKI_COMMON_H */
diff --git a/virt/heki/main.c b/virt/heki/main.c
new file mode 100644
index 000000000000..25c25f5700f7
--- /dev/null
+++ b/virt/heki/main.c
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hypervisor Enforced Kernel Integrity (Heki) - Common code
+ *
+ * Copyright © 2023 Microsoft Corporation
+ */
+
+#include <linux/heki.h>
+#include <linux/kstrtox.h>
+
+#include "common.h"
+
+bool heki_enabled __ro_after_init = true;
+
+/*
+ * Must be called after kmem_cache_init().
+ */
+__init void heki_early_init(void)
+{
+	if (!heki_enabled) {
+		pr_warn("Heki is not enabled\n");
+		return;
+	}
+	pr_warn("Heki is enabled\n");
+}
+
+static int __init heki_parse_config(char *str)
+{
+	if (kstrtobool(str, &heki_enabled))
+		pr_warn("Invalid option string for heki: '%s'\n", str);
+	return 1;
+}
+__setup("heki=", heki_parse_config);
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716433.1118553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sov-00029A-DW; Fri, 03 May 2024 13:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716433.1118553; Fri, 03 May 2024 13:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sov-00025T-6O; Fri, 03 May 2024 13:19:25 +0000
Received: by outflank-mailman (input) for mailman id 716433;
 Fri, 03 May 2024 13:19:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2sot-0001zB-TP
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:23 +0000
Received: from smtp-42ab.mail.infomaniak.ch (smtp-42ab.mail.infomaniak.ch
 [84.16.66.171]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c1139cf1-094f-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:19:22 +0200 (CEST)
Received: from smtp-3-0000.mail.infomaniak.ch (smtp-3-0000.mail.infomaniak.ch
 [10.4.36.107])
 by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGd5P4YzRYW;
 Fri,  3 May 2024 15:19:21 +0200 (CEST)
Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGc5y1CzZ8K; Fri,  3 May 2024 15:19:20 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1139cf1-094f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742361;
	bh=8e4dNhg8cJXTgu6Tduk1ij3wqe1j+rj2Pu+QmetQRF0=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=Fmwx5YE+iW+x7PpDaGWblZGnZb7wurP6ci4Q4GCwAcRMfopzd8vkWkcKtNJT8f84N
	 Z/3rVqdkcn7fTd12tvbQSPZygf9ZLVEBMTWje4Ww7ynKrlLNQKpa+wyL1oCNT0eN5W
	 u0DxKz/b4BPdXsWJpFOgkARSwFxi9QaRNlXFLyRM=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 2/5] KVM: x86: Add new hypercall to lock control registers
Date: Fri,  3 May 2024 15:19:07 +0200
Message-ID: <20240503131910.307630-3-mic@digikod.net>
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

This enables guests to lock their CR0 and CR4 registers with a subset of
X86_CR0_WP, X86_CR4_SMEP, X86_CR4_SMAP, X86_CR4_UMIP, X86_CR4_FSGSBASE
and X86_CR4_CET flags.

The new KVM_HC_LOCK_CR_UPDATE hypercall takes three arguments.  The
first is to identify the control register, the second is a bit mask to
pin (i.e. mark as read-only), and the third is for optional flags.

These register flags should already be pinned by Linux guests, but once
compromised, this self-protection mechanism could be disabled, which is
not the case with this dedicated hypercall.

Once the CRs are pinned by the guest, if it attempts to change them,
then a general protection fault is sent to the guest.

This hypercall may evolve and support new kind of registers or pinning.
The optional KVM_LOCK_CR_UPDATE_VERSION flag enables guests to know the
supported abilities by mapping the returned version with the related
features.

Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Link: https://lore.kernel.org/r/20240503131910.307630-3-mic@digikod.net
---

Changes since v1:
* Guard KVM_HC_LOCK_CR_UPDATE hypercall with CONFIG_HEKI.
* Move extern cr4_pinned_mask to x86.h (suggested by Kees Cook).
* Move VMX CR checks from vmx_set_cr*() to handle_cr() to make it
  possible to return to user space (see next commit).
* Change the heki_check_cr()'s first argument to vcpu.
* Don't use -KVM_EPERM in heki_check_cr().
* Generate a fault when the guest requests a denied CR update.
* Add a flags argument to get the version of this hypercall. Being able
  to do a preper version check was suggested by Wei Liu.
---
 Documentation/virt/kvm/x86/hypercalls.rst | 17 +++++
 arch/x86/include/uapi/asm/kvm_para.h      |  2 +
 arch/x86/kernel/cpu/common.c              |  7 +-
 arch/x86/kvm/vmx/vmx.c                    |  5 ++
 arch/x86/kvm/x86.c                        | 84 +++++++++++++++++++++++
 arch/x86/kvm/x86.h                        | 22 ++++++
 include/linux/kvm_host.h                  |  5 ++
 include/uapi/linux/kvm_para.h             |  1 +
 8 files changed, 141 insertions(+), 2 deletions(-)

diff --git a/Documentation/virt/kvm/x86/hypercalls.rst b/Documentation/virt/kvm/x86/hypercalls.rst
index 10db7924720f..3178576f4c47 100644
--- a/Documentation/virt/kvm/x86/hypercalls.rst
+++ b/Documentation/virt/kvm/x86/hypercalls.rst
@@ -190,3 +190,20 @@ the KVM_CAP_EXIT_HYPERCALL capability. Userspace must enable that capability
 before advertising KVM_FEATURE_HC_MAP_GPA_RANGE in the guest CPUID.  In
 addition, if the guest supports KVM_FEATURE_MIGRATION_CONTROL, userspace
 must also set up an MSR filter to process writes to MSR_KVM_MIGRATION_CONTROL.
+
+9. KVM_HC_LOCK_CR_UPDATE
+------------------------
+
+:Architecture: x86
+:Status: active
+:Purpose: Request some control registers to be restricted.
+
+- a0: identify a control register
+- a1: bit mask to make some flags read-only
+- a2: optional KVM_LOCK_CR_UPDATE_VERSION flag that will return the version of
+      this hypercall. Version 1 supports CR0 and CR4 pinning.
+
+The hypercall lets a guest request control register flags to be pinned for
+itself.
+
+Returns 0 on success or a KVM error code otherwise.
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h
index a1efa7907a0b..cfc17f3d1877 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -149,4 +149,6 @@ struct kvm_vcpu_pv_apf_data {
 #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
 #define KVM_PV_EOI_DISABLED 0x0
 
+#define KVM_LOCK_CR_UPDATE_VERSION (1 << 0)
+
 #endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 605c26c009c8..69695d9d6e2a 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -398,8 +398,11 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 }
 
 /* These bits should not change their value after CPU init is finished. */
-static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
-					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED;
+const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP |
+				      X86_CR4_UMIP | X86_CR4_FSGSBASE |
+				      X86_CR4_CET | X86_CR4_FRED;
+EXPORT_SYMBOL_GPL(cr4_pinned_mask);
+
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 22411f4aff53..7ba970b525f7 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -5453,6 +5453,11 @@ static int handle_cr(struct kvm_vcpu *vcpu)
 	case 0: /* mov to cr */
 		val = kvm_register_read(vcpu, reg);
 		trace_kvm_cr_write(cr, val);
+
+		ret = heki_check_cr(vcpu, cr, val);
+		if (ret)
+			return ret;
+
 		switch (cr) {
 		case 0:
 			err = handle_set_cr0(vcpu, val);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 91478b769af0..a5f47be59abc 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8281,11 +8281,86 @@ static unsigned long emulator_get_cr(struct x86_emulate_ctxt *ctxt, int cr)
 	return value;
 }
 
+#ifdef CONFIG_HEKI
+
+#define HEKI_ABI_VERSION 1
+
+static int heki_lock_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
+			unsigned long pin, unsigned long flags)
+{
+	if (flags) {
+		if ((flags == KVM_LOCK_CR_UPDATE_VERSION) && !cr && !pin)
+			return HEKI_ABI_VERSION;
+		return -KVM_EINVAL;
+	}
+
+	if (!pin)
+		return -KVM_EINVAL;
+
+	switch (cr) {
+	case 0:
+		/* Cf. arch/x86/kernel/cpu/common.c */
+		if (!(pin & X86_CR0_WP))
+			return -KVM_EINVAL;
+
+		if ((pin & read_cr0()) != pin)
+			return -KVM_EINVAL;
+
+		atomic_long_or(pin, &vcpu->kvm->heki_pinned_cr0);
+		return 0;
+	case 4:
+		/* Checks for irrelevant bits. */
+		if ((pin & cr4_pinned_mask) != pin)
+			return -KVM_EINVAL;
+
+		/* Ignores bits not present in host. */
+		pin &= __read_cr4();
+		atomic_long_or(pin, &vcpu->kvm->heki_pinned_cr4);
+		return 0;
+	}
+	return -KVM_EINVAL;
+}
+
+int heki_check_cr(struct kvm_vcpu *const vcpu, const unsigned long cr,
+		  const unsigned long val)
+{
+	unsigned long pinned;
+
+	switch (cr) {
+	case 0:
+		pinned = atomic_long_read(&vcpu->kvm->heki_pinned_cr0);
+		if ((val & pinned) != pinned) {
+			pr_warn_ratelimited(
+				"heki: Blocked CR0 update: 0x%lx\n", val);
+			kvm_inject_gp(vcpu, 0);
+			return 1;
+		}
+		return 0;
+	case 4:
+		pinned = atomic_long_read(&vcpu->kvm->heki_pinned_cr4);
+		if ((val & pinned) != pinned) {
+			pr_warn_ratelimited(
+				"heki: Blocked CR4 update: 0x%lx\n", val);
+			kvm_inject_gp(vcpu, 0);
+			return 1;
+		}
+		return 0;
+	}
+	return 0;
+}
+EXPORT_SYMBOL_GPL(heki_check_cr);
+
+#endif /* CONFIG_HEKI */
+
 static int emulator_set_cr(struct x86_emulate_ctxt *ctxt, int cr, ulong val)
 {
 	struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
 	int res = 0;
 
+	res = heki_check_cr(vcpu, cr, val);
+	if (res)
+		return res;
+
 	switch (cr) {
 	case 0:
 		res = kvm_set_cr0(vcpu, mk_cr_64(kvm_read_cr0(vcpu), val));
@@ -10142,6 +10217,15 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
 		return 0;
 	}
+#ifdef CONFIG_HEKI
+	case KVM_HC_LOCK_CR_UPDATE:
+		if (a0 > U32_MAX) {
+			ret = -KVM_EINVAL;
+		} else {
+			ret = heki_lock_cr(vcpu, a0, a1, a2);
+		}
+		break;
+#endif /* CONFIG_HEKI */
 	default:
 		ret = -KVM_ENOSYS;
 		break;
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index a8b71803777b..ade7d68ddaff 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -290,6 +290,26 @@ static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk)
 	return !(kvm->arch.disabled_quirks & quirk);
 }
 
+#ifdef CONFIG_HEKI
+
+int heki_check_cr(struct kvm_vcpu *vcpu, unsigned long cr, unsigned long val);
+
+#else /* CONFIG_HEKI */
+
+static inline int heki_check_cr(struct kvm_vcpu *vcpu, unsigned long cr,
+				unsigned long val)
+{
+	return 0;
+}
+
+static inline int heki_lock_cr(struct kvm_vcpu *const vcpu, unsigned long cr,
+			       unsigned long pin)
+{
+	return 0;
+}
+
+#endif /* CONFIG_HEKI */
+
 void kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip);
 
 u64 get_kvmclock_ns(struct kvm *kvm);
@@ -327,6 +347,8 @@ extern u64 host_xcr0;
 extern u64 host_xss;
 extern u64 host_arch_capabilities;
 
+extern const unsigned long cr4_pinned_mask;
+
 extern struct kvm_caps kvm_caps;
 
 extern bool enable_pmu;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 48f31dcd318a..6ff13937929a 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -836,6 +836,11 @@ struct kvm {
 	bool vm_bugged;
 	bool vm_dead;
 
+#ifdef CONFIG_HEKI
+	atomic_long_t heki_pinned_cr0;
+	atomic_long_t heki_pinned_cr4;
+#endif /* CONFIG_HEKI */
+
 #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
 	struct notifier_block pm_notifier;
 #endif
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index 960c7e93d1a9..2ed418704603 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -30,6 +30,7 @@
 #define KVM_HC_SEND_IPI		10
 #define KVM_HC_SCHED_YIELD		11
 #define KVM_HC_MAP_GPA_RANGE		12
+#define KVM_HC_LOCK_CR_UPDATE		13
 
 /*
  * hypercalls use architecture specific
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:19:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:19:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716436.1118593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sp0-0003GV-FA; Fri, 03 May 2024 13:19:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716436.1118593; Fri, 03 May 2024 13:19:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2sp0-0003GC-Am; Fri, 03 May 2024 13:19:30 +0000
Received: by outflank-mailman (input) for mailman id 716436;
 Fri, 03 May 2024 13:19:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0Za+=MG=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s2soy-0001z6-CL
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:19:28 +0000
Received: from smtp-190f.mail.infomaniak.ch (smtp-190f.mail.infomaniak.ch
 [2001:1600:7:10::190f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c3a0112d-094f-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 15:19:26 +0200 (CEST)
Received: from smtp-3-0001.mail.infomaniak.ch (smtp-3-0001.mail.infomaniak.ch
 [10.4.36.108])
 by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VWBGk08rZzRxX;
 Fri,  3 May 2024 15:19:26 +0200 (CEST)
Received: from unknown by smtp-3-0001.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VWBGj1GSJzcWL; Fri,  3 May 2024 15:19:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3a0112d-094f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1714742365;
	bh=4ONNBgWSam9aJRjdro/4dfXdqBvgud7e8GDkEnOey/s=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=KMEY6pkEAKgoExfEmKVQNyCUEruknH1ap7Ou5Y419jIaRi8bmkoHU2rC0/FMv7d7x
	 rFR+BO0giuToCc4a3wUUdcvtmQ+cooljaMVQXO/XQSIZria32QbPa/X8LojG8X9+C1
	 h838ru/DHL68SkgW/jHR0XIi48e+6J+Z2qvxBa4k=
From: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>
To: Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>
Cc: =?UTF-8?q?Micka=C3=ABl=20Sala=C3=BCn?= <mic@digikod.net>,
	"Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
	Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>,
	Anna Trikalinou <atrikalinou@microsoft.com>,
	Chao Peng <chao.p.peng@linux.intel.com>,
	Forrest Yuan Yu <yuanyu@google.com>,
	James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>,
	John Andersen <john.s.andersen@intel.com>,
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
	Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?UTF-8?q?Mihai=20Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?UTF-8?q?Nicu=C8=99or=20C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>,
	Thara Gopinath <tgopinath@microsoft.com>,
	Trilok Soni <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>,
	Will Deacon <will@kernel.org>,
	Yu Zhang <yu.c.zhang@linux.intel.com>,
	=?UTF-8?q?=C8=98tefan=20=C8=98icleru?= <ssicleru@bitdefender.com>,
	dev@lists.cloudhypervisor.org,
	kvm@vger.kernel.org,
	linux-hardening@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-security-module@vger.kernel.org,
	qemu-devel@nongnu.org,
	virtualization@lists.linux-foundation.org,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH v3 5/5] virt: Add Heki KUnit tests
Date: Fri,  3 May 2024 15:19:10 +0200
Message-ID: <20240503131910.307630-6-mic@digikod.net>
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Infomaniak-Routing: alpha

The new CONFIG_HEKI_KUNIT_TEST option enables to run tests in a a kernel
module.  The minimal required configuration is listed in the
virt/heki-test/.kunitconfig file.

test_cr_disable_smep checks control-register pinning by trying to
disable SMEP.  This test should then failed on a non-protected kernel,
and only succeed with a kernel protected by Heki.

This test doesn't rely on native_write_cr4() because of the
cr4_pinned_mask hardening, which means that this *test* module loads a
valid kernel code to arbitrary change CR4.  This simulate an attack
scenario where an attaker would use ROP to directly jump to the related
cr4 instruction.

As for any KUnit test, the kernel is tainted with TAINT_TEST when the
test is executed.

It is interesting to create new KUnit tests instead of extending KVM's
Kselftests because Heki is design to be hypervisor-agnostic, it relies
on a set of hypercalls (for KVM or others), and we also want to test
kernel's configuration (actual pinned CR).  However, new KVM's
Kselftests would be useful to test KVM's interface with the host.

When using Qemu, we need to pass the following arguments: -cpu host
-enable-kvm

For now, it is not possible to run these tests as built-in but we are
working on that [1].  If tests are built-in anyway, they will just be
skipped because Heki would not be enabled.

Run Heki tests with:
  insmod heki-test.ko

  KTAP version 1
  1..1
      KTAP version 1
      # Subtest: heki_x86
      # module: heki_test
      1..1
      ok 1 test_cr_disable_smep
  ok 1 heki_x86

Link: https://lore.kernel.org/r/20240229170409.365386-2-mic@digikod.net [1]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Link: https://lore.kernel.org/r/20240503131910.307630-6-mic@digikod.net
---

Changes since v2:
* Make tests standalone (e.g. don't depends on CONFIG_HEKI).
* Enable to create a test kernel module.
* Don't rely on private kernel symbols.
* Handle GP fault for CR-pinning test case.
* Rename option to CONFIG_HEKI_KUNIT_TEST.
* Add the list of required kernel options.
* Move tests to virt/heki-test/ [FIXME]
* Only keep CR pinning test.
* Restore previous state (with SMEP enabled).
* Add a Kconfig menu for Heki and update the description.
* Skip tests if Heki is not protecting the running kernel.

Changes since v1:
* Move all tests to virt/heki/tests.c
---
 include/linux/heki.h   |   1 +
 virt/heki/.kunitconfig |   9 ++++
 virt/heki/Kconfig      |  12 +++++
 virt/heki/Makefile     |   1 +
 virt/heki/heki-test.c  | 114 +++++++++++++++++++++++++++++++++++++++++
 virt/heki/main.c       |  10 ++++
 6 files changed, 147 insertions(+)
 create mode 100644 virt/heki/.kunitconfig
 create mode 100644 virt/heki/heki-test.c

diff --git a/include/linux/heki.h b/include/linux/heki.h
index 96ccb17657e5..3294c4d583e5 100644
--- a/include/linux/heki.h
+++ b/include/linux/heki.h
@@ -35,6 +35,7 @@ struct heki {
 
 extern struct heki heki;
 extern bool heki_enabled;
+extern bool heki_enforcing;
 
 void heki_early_init(void);
 void heki_late_init(void);
diff --git a/virt/heki/.kunitconfig b/virt/heki/.kunitconfig
new file mode 100644
index 000000000000..ad4454800579
--- /dev/null
+++ b/virt/heki/.kunitconfig
@@ -0,0 +1,9 @@
+CONFIG_HEKI=y
+CONFIG_HEKI_KUNIT_TEST=m
+CONFIG_HEKI_MENU=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_KUNIT=y
+CONFIG_KVM=y
+CONFIG_KVM_GUEST=y
+CONFIG_PARAVIRT=y
diff --git a/virt/heki/Kconfig b/virt/heki/Kconfig
index 0c764e342f48..18895a81a9af 100644
--- a/virt/heki/Kconfig
+++ b/virt/heki/Kconfig
@@ -28,4 +28,16 @@ config HEKI
 	  This feature is helpful in maintaining guest virtual machine security
 	  even after the guest kernel has been compromised.
 
+config HEKI_KUNIT_TEST
+	tristate "KUnit tests for Heki" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	depends on X86
+	default KUNIT_ALL_TESTS
+	help
+	  Build KUnit tests for Landlock.
+
+	  See the KUnit documentation in Documentation/dev-tools/kunit
+
+	  If you are unsure how to answer this question, answer N.
+
 endif
diff --git a/virt/heki/Makefile b/virt/heki/Makefile
index 8b10e73a154b..7133545eb5ae 100644
--- a/virt/heki/Makefile
+++ b/virt/heki/Makefile
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 obj-$(CONFIG_HEKI) += main.o
+obj-$(CONFIG_HEKI_KUNIT_TEST) += heki-test.o
diff --git a/virt/heki/heki-test.c b/virt/heki/heki-test.c
new file mode 100644
index 000000000000..b4e11c21ac5d
--- /dev/null
+++ b/virt/heki/heki-test.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hypervisor Enforced Kernel Integrity (Heki) - Tests
+ *
+ * Copyright © 2023-2024 Microsoft Corporation
+ */
+
+#include <asm/asm.h>
+#include <asm/processor.h>
+#include <asm/special_insns.h>
+#include <kunit/test.h>
+#include <linux/heki.h>
+
+/* Returns true on error (i.e. GP fault), false otherwise. */
+static __always_inline bool set_cr4(unsigned long value)
+{
+	int err = 0;
+
+	might_sleep();
+	/* clang-format off */
+	asm volatile("1: mov %[value],%%cr4 \n"
+		     "2: \n"
+		     _ASM_EXTABLE_TYPE_REG(1b, 2b, EX_TYPE_ONE_REG, %[err])
+		     : [value] "+r" (value), [err] "+r" (err)
+		     :);
+	/* clang-format on */
+	return err;
+}
+
+/* Control register pinning tests with SMEP check. */
+static void test_cr_disable_smep(struct kunit *const test)
+{
+	bool is_vme_set;
+
+	/* SMEP should be initially enabled. */
+	KUNIT_ASSERT_TRUE(test, __read_cr4() & X86_CR4_SMEP);
+
+	/*
+	 * Trying to disable SMEP, bypassing kernel self-protection by not
+	 * using cr4_clear_bits(X86_CR4_SMEP), and checking GP fault.
+	 */
+	KUNIT_EXPECT_TRUE(test, set_cr4(__read_cr4() & ~X86_CR4_SMEP));
+
+	/* SMEP should still be enabled. */
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_SMEP);
+
+	/* Re-enabling SMEP doesn't throw a GP fault. */
+	KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() | X86_CR4_SMEP));
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_SMEP);
+
+	/* We are allowed to set and unset VME. */
+	is_vme_set = __read_cr4() & X86_CR4_VME;
+	KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() | X86_CR4_VME));
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_VME);
+
+	KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() & ~X86_CR4_VME));
+	KUNIT_EXPECT_FALSE(test, __read_cr4() & X86_CR4_VME);
+
+	KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() | X86_CR4_VME));
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_VME);
+
+	/* SMEP and VME changes should be consistent when setting both. */
+	KUNIT_EXPECT_TRUE(test, set_cr4(__read_cr4() &
+					~(X86_CR4_SMEP | X86_CR4_VME)));
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_SMEP);
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_VME);
+
+	/* Unset VME. */
+	KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() & ~X86_CR4_VME));
+	KUNIT_EXPECT_FALSE(test, __read_cr4() & X86_CR4_VME);
+
+	/* SMEP and VME changes should be consistent when only setting SMEP. */
+	KUNIT_EXPECT_TRUE(test, set_cr4(__read_cr4() &
+					~(X86_CR4_SMEP | X86_CR4_VME)));
+	KUNIT_EXPECT_TRUE(test, __read_cr4() & X86_CR4_SMEP);
+	KUNIT_EXPECT_FALSE(test, __read_cr4() & X86_CR4_VME);
+
+	/* Restores VME. */
+	if (is_vme_set)
+		KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() | X86_CR4_VME));
+	else
+		KUNIT_EXPECT_FALSE(test, set_cr4(__read_cr4() & ~X86_CR4_VME));
+}
+
+/* clang-format off */
+static struct kunit_case test_cases_x86[] = {
+	KUNIT_CASE(test_cr_disable_smep),
+	{}
+};
+/* clang-format on */
+
+static int test_init(struct kunit *test)
+{
+#ifdef CONFIG_HEKI
+	if (heki_enforcing)
+		return 0;
+#else /* CONFIG_HEKI */
+	kunit_skip(test, "Heki is not enforced");
+#endif /* CONFIG_HEKI */
+
+	return 0;
+}
+
+static struct kunit_suite test_suite_x86 = {
+	.name = "heki_x86",
+	.init = test_init,
+	.test_cases = test_cases_x86,
+};
+
+kunit_test_suite(test_suite_x86);
+
+MODULE_IMPORT_NS(HEKI_KUNIT_TEST);
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Tests for Hypervisor Enforced Kernel Integrity (Heki)");
diff --git a/virt/heki/main.c b/virt/heki/main.c
index ef0530a03e09..dcc89befaf66 100644
--- a/virt/heki/main.c
+++ b/virt/heki/main.c
@@ -11,6 +11,12 @@
 #include "common.h"
 
 bool heki_enabled __ro_after_init = true;
+
+#if IS_ENABLED(CONFIG_KUNIT)
+bool heki_enforcing = false;
+EXPORT_SYMBOL_NS_GPL(heki_enforcing, HEKI_KUNIT_TEST);
+#endif /* IS_ENABLED(CONFIG_KUNIT) */
+
 struct heki heki;
 
 /*
@@ -47,6 +53,10 @@ void heki_late_init(void)
 		return;
 
 	pr_warn("Control registers locked\n");
+
+#if IS_ENABLED(CONFIG_KUNIT)
+	heki_enforcing = true;
+#endif /* IS_ENABLED(CONFIG_KUNIT) */
 }
 
 static int __init heki_parse_config(char *str)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 03 13:50:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:50:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716470.1118602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tIP-0002ni-VU; Fri, 03 May 2024 13:49:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716470.1118602; Fri, 03 May 2024 13:49:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tIP-0002nb-Sp; Fri, 03 May 2024 13:49:53 +0000
Received: by outflank-mailman (input) for mailman id 716470;
 Fri, 03 May 2024 13:49:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1yCT=MG=flex--seanjc.bounces.google.com=3fes0ZgYKCWAQC8LHAEMMEJC.AMKVCL-BCTCJJGQRQ.VCLNPMHCAR.MPE@srs-se1.protection.inumbo.net>)
 id 1s2tIO-0002nT-Aq
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:49:52 +0000
Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com
 [2607:f8b0:4864:20::44a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02c61719-0954-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 15:49:51 +0200 (CEST)
Received: by mail-pf1-x44a.google.com with SMTP id
 d2e1a72fcca58-6f446a1ec59so1008744b3a.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 06:49:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02c61719-0954-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20230601; t=1714744189; x=1715348989; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id
         :reply-to;
        bh=rQGv5UfxS6jX/aTPX2JQQkA4rH0rnz4JsGbkCfvMiug=;
        b=xip0XDkZd2mt7GFPFKpe/Xyv9d85aE2dsBihmjWZ+xbDDoGgZWodKRDMhb/5V+8pkB
         fvnmIZV31JfI43aUWZ0qvTmHlPm75E8vJofwyf74i1kqDvW6F8qZ79TZl9DwbGZNwLK+
         kWdHw6ZI6HiwQAEMF/mNQ7Gr4FlPsxEKURsvC2fkcJcm10VQUfmkyMIxqf38l0g31nQu
         nBiRyNOPkL1gMFpxfnKy1PQBj4tCnKN2u73A/UCt7YJAWHYz1BmYpVMn4Sjmsj1zcFF8
         IYmyLzXEdPwaDeG87cHoURIBXfetzXrRhhEUNNjplFJL2Wi+I22PUG83L1ySYTBeoMx9
         oISg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714744189; x=1715348989;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=rQGv5UfxS6jX/aTPX2JQQkA4rH0rnz4JsGbkCfvMiug=;
        b=Ip0gI0vGsMjW7Vg+nngzB4g/WKzV1XSiG4+1P/nTF6M68xqn/fn6dScrFWYeo+HL4A
         CDG37qn0EHMg/PHsi8kLeqTYnhHKfM06CahsEqVc0tJ5CqQp/2KfBCtUDuCSwg1SI+uR
         zZfabpZIgXvE4Skp8DmPLpDSFjFn5QHnhPvxrlMFV37RmfG/AXsKmthFK0s5oaEVbaJH
         uK+v7Vi0OLoqkIVJOrfL65l1NQmOyXXZ9OjEpv2G2tyOIL7EBTtd1sX8WPEXYyPV8aid
         3VxAGIELBddKRRpwvZymBzjKKx7Opgy9t49TR7sqcHmw0V5HeGx8bpE/6ojcbmf6WuWG
         Im4w==
X-Forwarded-Encrypted: i=1; AJvYcCXlSBeDv4/BAM2eC4vvr02Ii/mzlSqg8VxXy58n9zF6T+WB5pr1L6AeQ5t1VfmbiJzQ10YytjXf2Nc9BbE/7zuTBQpLwsFJRR9ICpXNbJ0=
X-Gm-Message-State: AOJu0Ywu8piL8YzwL0lVsrFKQWhtyTUHJVjRx80AyGpZSvA1D4kDs00F
	alV4qt2HT8FoWIBHz6Z1ISwaxWcKEEYadJwuisZx/hb9b83rFpbiyHQNUTsGCTxIpmpWfQGnYk7
	nzQ==
X-Google-Smtp-Source: AGHT+IGc0eGZplb+oZL3dkGKqelUQj57uqxGLcLhw0noqTEVy1iaqZaCu7zCqnCr5z1/yKblcKIpSFxnXkk=
X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37])
 (user=seanjc job=sendgmr) by 2002:a05:6a00:2e07:b0:6ea:baf6:57a3 with SMTP id
 fc7-20020a056a002e0700b006eabaf657a3mr154355pfb.6.1714744189252; Fri, 03 May
 2024 06:49:49 -0700 (PDT)
Date: Fri, 3 May 2024 06:49:47 -0700
In-Reply-To: <20240503131910.307630-1-mic@digikod.net>
Mime-Version: 1.0
References: <20240503131910.307630-1-mic@digikod.net>
Message-ID: <ZjTre6BYRpkI_H4o@google.com>
Subject: Re: [RFC PATCH v3 0/5] Hypervisor-Enforced Kernel Integrity - CR pinning
From: Sean Christopherson <seanjc@google.com>
To: "=?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?=" <mic@digikod.net>
Cc: Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	"Mihai =?utf-8?B?RG9uyJt1?=" <mdontu@bitdefender.com>, 
	"=?utf-8?B?TmljdciZb3IgQ8OuyJt1?=" <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, 
	Will Deacon <will@kernel.org>, Yu Zhang <yu.c.zhang@linux.intel.com>, 
	"=?utf-8?Q?=C8=98tefan_=C8=98icleru?=" <ssicleru@bitdefender.com>, dev@lists.cloudhypervisor.org, 
	kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, 
	virtualization@lists.linux-foundation.org, x86@kernel.org, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 03, 2024, Micka=C3=ABl Sala=C3=BCn wrote:
> Hi,
>=20
> This patch series implements control-register (CR) pinning for KVM and
> provides an hypervisor-agnostic API to protect guests.  It includes the
> guest interface, the host interface, and the KVM implementation.
>=20
> It's not ready for mainline yet (see the current limitations), but we
> think the overall design and interfaces are good and we'd like to have
> some feedback on that.

...

> # Current limitations
>=20
> This patch series doesn't handle VM reboot, kexec, nor hybernate yet.
> We'd like to leverage the realated feature from KVM CR-pinning patch
> series [3].  Help appreciated!

Until you have a story for those scenarios, I don't expect you'll get a lot=
 of
valuable feedback, or much feedback at all.  They were the hot topic for KV=
M CR
pinning, and they'll likely be the hot topic now.


From xen-devel-bounces@lists.xenproject.org Fri May 03 13:54:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 13:54:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716474.1118613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tN4-00052h-GT; Fri, 03 May 2024 13:54:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716474.1118613; Fri, 03 May 2024 13:54:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tN4-00052a-Dn; Fri, 03 May 2024 13:54:42 +0000
Received: by outflank-mailman (input) for mailman id 716474;
 Fri, 03 May 2024 13:54:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1aGk=MG=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s2tN2-00051Z-Ph
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 13:54:40 +0000
Received: from mail-oo1-xc36.google.com (mail-oo1-xc36.google.com
 [2607:f8b0:4864:20::c36])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ae41d7ed-0954-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 15:54:38 +0200 (CEST)
Received: by mail-oo1-xc36.google.com with SMTP id
 006d021491bc7-5acb737b508so3947247eaf.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 06:54:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae41d7ed-0954-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714744477; x=1715349277; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=76gaEj8j93eOO7a7Mnb1OIG9aYIF5RI94rklGIt0fBU=;
        b=CMZV6v/i/OXGzJAB2pa296aOdOOnYTrBDfiyv9ZTJZWwBCaix74tyEYYf+Dy4hxiJL
         P8WbSs6n8Yom1CbDGN9D6FiRymP+W2WX2cTLoBIHIDzxGYs1Vv2gSIVXhFUEdYKXeTb1
         KOjfgJMCvGFlEh4eqPj5eIHXoMdQVW/S2lbcM744eVFE6+bQouMHqw370SB8SBzafr7D
         Axz1QQrI8XOuISi00uN9QvateYZ5EHcKspzxGkqTYFW0Q7AR+njSIKT/H1dfo43s58xA
         BMkBD34/+UCXIxjl1lmJ/PWHIsB4qocyu1qQkwLpqyrRNJ7vwwNSlP8V3XxeG5PbfKxB
         sk5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714744477; x=1715349277;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=76gaEj8j93eOO7a7Mnb1OIG9aYIF5RI94rklGIt0fBU=;
        b=vEqTxo7ol+4hSM1jvXs7FcfIw1DYgilJWBjtS71BMNEHhnNbS2dfDqMAlMB86Gltx4
         3pkFQggkSWOWHMG/1GGzF6Uxmfm1A6hnFWt/8eFzsZV2ji8lFhzOzUAJ/T77zjrrYy91
         2GxsqleX2TAqdRHVUxsvnfhtOjNHDJ3FTKMJ92gHiwre4pJcgKbdgmq8OeI3UkYBPGZu
         otnlK21DVOZeHatdrarWaYNWdhE3nUHt77gEIkQB22A9YlAlyCODZbZxoT/CFL7gt0HY
         0pe7/jZ+tQbocaAdaN8+j4KwIUYxr6fnxyseIMxnTzyxT5x55HDDnq660/OoQZEvqGeO
         3RTA==
X-Gm-Message-State: AOJu0Yx53sXcIREu7imkJwORc5hvm3RjcJqO7cPTt3kCQ+bMxA74NO6N
	QM0j68b9vj0y1qgGQuvMVu/IRwksliAk5pyxSjERAMBHOKFtjZ8WvBEz93ydjv3PVOGiK15mymu
	CmiHBj4GSLUjiC0Mn290AJEMNciOBKvNY6nEJOQ==
X-Google-Smtp-Source: AGHT+IGzMllgkhGbaihZspMy4AJHOCme0u1LbVbCDf4J7Y30fu/sxNZ4J4VRPNhc7SnUJTsqdbhq1Jy7BuHEszUSQCg=
X-Received: by 2002:a4a:4804:0:b0:5a7:c78e:410e with SMTP id
 p4-20020a4a4804000000b005a7c78e410emr2868046ooa.2.1714744477124; Fri, 03 May
 2024 06:54:37 -0700 (PDT)
MIME-Version: 1.0
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
 <20240502145645.1201613-6-jens.wiklander@linaro.org> <935e3947-53ee-48c5-a10c-5fa7f572ae65@xen.org>
In-Reply-To: <935e3947-53ee-48c5-a10c-5fa7f572ae65@xen.org>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Fri, 3 May 2024 15:54:25 +0200
Message-ID: <CAHUa44HdA_cdZcjbVL+xYXnLVkdK1hK5hfU_k+fZkS8=VAOy1A@mail.gmail.com>
Subject: Re: [XEN PATCH v4 5/5] xen/arm: ffa: support notification
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org, patches@linaro.org, 
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Julien,

On Fri, May 3, 2024 at 11:59=E2=80=AFAM Julien Grall <julien@xen.org> wrote=
:
>
> Hi Jens,
>
> On 02/05/2024 15:56, Jens Wiklander wrote:
> > -static bool ffa_probe(void)
> > +static int __init ffa_init(void)
> >   {
> >       uint32_t vers;
> >       unsigned int major_vers;
> > @@ -460,16 +511,16 @@ static bool ffa_probe(void)
> >           printk(XENLOG_ERR
> >                  "ffa: unsupported SMCCC version %#x (need at least %#x=
)\n",
> >                  smccc_ver, ARM_SMCCC_VERSION_1_2);
> > -        return false;
> > +        return 0;
> >       }
> >
> >       if ( !ffa_get_version(&vers) )
> > -        return false;
> > +        return 0;
> >
> >       if ( vers < FFA_MIN_SPMC_VERSION || vers > FFA_MY_VERSION )
> >       {
> >           printk(XENLOG_ERR "ffa: Incompatible version %#x found\n", ve=
rs);
> > -        return false;
> > +        return 0;
> >       }
> >
> >       major_vers =3D (vers >> FFA_VERSION_MAJOR_SHIFT) & FFA_VERSION_MA=
JOR_MASK;
> > @@ -492,26 +543,33 @@ static bool ffa_probe(void)
> >            !check_mandatory_feature(FFA_MEM_SHARE_32) ||
> >            !check_mandatory_feature(FFA_MEM_RECLAIM) ||
> >            !check_mandatory_feature(FFA_MSG_SEND_DIRECT_REQ_32) )
> > -        return false;
> > +        return 0;
> >
> >       if ( !ffa_rxtx_init() )
> > -        return false;
> > +        return 0;
> >
> >       ffa_version =3D vers;
> >
> >       if ( !ffa_partinfo_init() )
> >           goto err_rxtx_destroy;
> >
> > +    ffa_notif_init();
> >       INIT_LIST_HEAD(&ffa_teardown_head);
> >       init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL=
, 0);
> >
> > -    return true;
> > +    return 0;
> >
> >   err_rxtx_destroy:
> >       ffa_rxtx_destroy();
> >       ffa_version =3D 0;
> >
> > -    return false;
> > +    return 0;
> > +}
> > +presmp_initcall(ffa_init);
> I would rather prefer if tee_init() is called from presmp_initcall().
> This would avoid FFA to have to try to initialize itself before all the
> others.

OK, I'll update.

>
> This is what I had in mind with my original request, but I guess I
> wasn't clear enough. Sorry for that.

No worries.

>
> [...]
>
> > +static void notif_irq_handler(int irq, void *data)
> > +{
> > +    const struct arm_smccc_1_2_regs arg =3D {
> > +        .a0 =3D FFA_NOTIFICATION_INFO_GET_64,
> > +    };
> > +    struct arm_smccc_1_2_regs resp;
> > +    unsigned int id_pos;
> > +    unsigned int list_count;
> > +    uint64_t ids_count;
> > +    unsigned int n;
> > +    int32_t res;
> > +
> > +    do {
> > +        arm_smccc_1_2_smc(&arg, &resp);
> > +        res =3D ffa_get_ret_code(&resp);
> > +        if ( res )
> > +        {
> > +            if ( res !=3D FFA_RET_NO_DATA )
> > +                printk(XENLOG_ERR "ffa: notification info get failed: =
error %d\n",
> > +                       res);
> > +            return;
> > +        }
> > +
> > +        ids_count =3D resp.a2 >> FFA_NOTIF_INFO_GET_ID_LIST_SHIFT;
> > +        list_count =3D ( resp.a2 >> FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT =
) &
> > +                     FFA_NOTIF_INFO_GET_ID_COUNT_MASK;
> > +
> > +        id_pos =3D 0;
> > +        for ( n =3D 0; n < list_count; n++ )
> > +        {
> > +            unsigned int count =3D ((ids_count >> 2 * n) & 0x3) + 1;
> > +            uint16_t vm_id =3D get_id_from_resp(&resp, id_pos);
> > +            struct domain *d;
> > +
> > +            /*
> > +             * vm_id =3D=3D 0 means a notifications pending for Xen it=
self, but
> > +             * we don't support that yet.
> > +             */
> > +            if (vm_id)
> > +                d =3D ffa_rcu_lock_domain_by_vm_id(vm_id);
>
> I am still unconvinced that this is sufficient. This will prevent
> "struct domain *" to be freed. But ...
>
> > +            else
> > +                d =3D NULL;
> > +
> > +            if ( d )
> > +            {
> > +                struct ffa_ctx *ctx =3D d->arch.tee;
>
> ... I don't think it will protect d->arch.tee to be freed as this is
> happening during teardorwn. You mostly need some other sort of locking
> to avoid d->arch.tee been freed behind your back.

OK, I'll need to work more on this. The outcome of the [1] thread may
affect this too.

[1] https://lists.xenproject.org/archives/html/xen-devel/2024-05/msg00156.h=
tml

>
> > +                struct vcpu *v;
> > +
> > +                ACCESS_ONCE(ctx->notif.secure_pending) =3D true;
> > +
> > +                /*
> > +                 * Since we're only delivering global notification, al=
ways
> > +                 * deliver to the first online vCPU. It doesn't matter
> > +                 * which we chose, as long as it's available.
> > +                 */
> > +                for_each_vcpu(d, v)
> > +                {
> > +                    if ( is_vcpu_online(v) )
> > +                    {
> > +                        vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INT=
R_ID,
> > +                                        true);
> > +                        break;
> > +                    }
> > +                }
> > +                if ( !v )
> > +                    printk(XENLOG_ERR "ffa: can't inject NPI, all vCPU=
s offline\n");
> > +
> > +                rcu_unlock_domain(d);
> > +            }
> > +
> > +            id_pos +=3D count;
> > +        }
> > +
> > +    } while (resp.a2 & FFA_NOTIF_INFO_GET_MORE_FLAG);
> > +}
>
> [...]
>
> > +static int ffa_setup_irq_callback(struct notifier_block *nfb,
> > +                                  unsigned long action, void *hcpu)
> > +{
> > +    unsigned int cpu =3D (unsigned long)hcpu;
> > +    struct notif_irq_info irq_info =3D { };
> > +
> > +    switch ( action )
> > +    {
> > +    case CPU_ONLINE:
>
> Can't you execute the notifier in CPU_STARTING? This will be called on
> the CPU directly, so you should be able to use request_irq(...).

I tried that first but it failed with the ASSERT_ALLOC_CONTEXT() in _xmallo=
c().

I've also tested a three-step solution with CPU_UP_PREPARE,
CPU_STARTING, and CPU_UP_CANCELED.
My approach here is more direct, but it still suffers from a weakness
in error handling even if it seems quite unlikely to run out of heap
or for setup_irq() to fail at this stage.

Thanks,
Jens

>
> > +        /*
> > +         * The notifier call is done on the primary or initiating CPU =
when
> > +         * the target CPU have come online, but the SGI must be setup =
on
> > +         * the target CPU.
> > +         *
> > +         * We make an IPI call to the target CPU to setup the SGI. The=
 call
> > +         * is executed in interrupt context on the target CPU, so we c=
an't
> > +         * call request_irq() directly since it allocates memory.
> > +         *
> > +         * We preallocate the needed irqaction here and pass it via th=
e
> > +         * temporary struct notif_irq_info. The call is synchronous in=
 the
> > +         * sense that when on_selected_cpus() returns the callback
> > +         * notif_irq_enable() has done the same on the target CPU.
> > +         *
> > +         * We deal with two errors, one where notif_irq_enable() hasn'=
t
> > +         * been called for some reason, that error is logged below. Th=
e
> > +         * other where setup_irq() fails is logged in the callback. We=
 must
> > +         * free the irqaction in both cases since it has failed to bec=
ome
> > +         * registered.
> > +         *
> > +         * Failures leads to a problem notifications, the CPUs with fa=
ilure
> > +         * will trigger on the SGI indicating that there are notificat=
ions
> > +         * pending, while the SPMC in the secure world will not notice=
 that
> > +         * the interrupt was lost.
> > +         */
> > +        irq_info.action =3D xmalloc(struct irqaction);
> > +        if ( !irq_info.action )
> > +        {
> > +            printk(XENLOG_ERR "ffa: setup irq %u failed, out of memory=
\n",
> > +                   notif_sri_irq);
> > +            return -ENOMEM;
> > +        }
> > +
> > +        *irq_info.action =3D (struct irqaction){
> > +            .handler =3D notif_irq_handler,
> > +            .name =3D "FF-A notif",
> > +            .free_on_release =3D 1,
> > +        };
> > +
> > +        on_selected_cpus(cpumask_of(cpu), notif_irq_enable, &irq_info,=
 1);
> > +        if (!irq_info.called)
> > +            printk(XENLOG_ERR "ffa: on_selected_cpus(cpumask_of(%u)) f=
ailed\n",
> > +                   cpu);
> > +        /*
> > +         * The irqaction is unused and must be freed if irq_info.actio=
n is
> > +         * non-NULL at this stage.
> > +         */
> > +        XFREE(irq_info.action);
> > +        break;
> > +    default:
> > +        break;
> > +    }
> > +
> > +    return 0;
> > +}
>
> Cheers,
>
> --
> Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:00:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:00:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716481.1118623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tSr-0007GW-1h; Fri, 03 May 2024 14:00:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716481.1118623; Fri, 03 May 2024 14:00:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tSq-0007GP-V0; Fri, 03 May 2024 14:00:40 +0000
Received: by outflank-mailman (input) for mailman id 716481;
 Fri, 03 May 2024 14:00:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tSp-0007GD-EA; Fri, 03 May 2024 14:00:39 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tSp-0005r1-4O; Fri, 03 May 2024 14:00:39 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tSo-0004cD-Is; Fri, 03 May 2024 14:00:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tSo-0007gM-IH; Fri, 03 May 2024 14:00:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8Q9Qonx2DSU6f+AX+qpnI7oUcX32U59WfIHmjI6pzg0=; b=sA5ZKnJtcduJsry/ffVeGESurk
	vry28fK4ApJeFwmX1Xe+4RTlBsh4BqFuYsI7kCOS0uphBIdVGc6W2k9RDGqRhmXWCdkkZechVEYiu
	GIlIg7ltmXkIv+r5QiZsx1tEmQuwuhqlW4R3h0Lr+aHr/PDf3c3H1H9C46Aeq6q8HA24=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185908-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185908: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=3146305fd3a610573963fe4858cc12ec1c4cf5c7
X-Osstest-Versions-That:
    libvirt=63f00d09e3bf7442cddf3657b51b0bc645f8c434
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 14:00:38 +0000

flight 185908 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185908/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185891
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              3146305fd3a610573963fe4858cc12ec1c4cf5c7
baseline version:
 libvirt              63f00d09e3bf7442cddf3657b51b0bc645f8c434

Last test of basis   185891  2024-05-01 04:18:46 Z    2 days
Testing same since   185908  2024-05-03 04:20:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Jim Fehlig <jfehlig@suse.com>
  Jiri Denemark <jdenemar@redhat.com>
  Kristina Hanicova <khanicov@redhat.com>
  Martin Kletzander <mkletzan@redhat.com>
  Martin Shirokov <shirokovmartin@gmail.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Rayhan Faizel <rayhan.faizel@gmail.com>
  Tim Wiederhake <twiederh@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   63f00d09e3..3146305fd3  3146305fd3a610573963fe4858cc12ec1c4cf5c7 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:01:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:01:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716486.1118632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tTf-0007l6-AP; Fri, 03 May 2024 14:01:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716486.1118632; Fri, 03 May 2024 14:01:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tTf-0007kz-7w; Fri, 03 May 2024 14:01:31 +0000
Received: by outflank-mailman (input) for mailman id 716486;
 Fri, 03 May 2024 14:01:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZLhg=MG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s2tTd-0007kg-VP
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:01:29 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a38481c2-0955-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:01:29 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-34d8d11a523so2207482f8f.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 07:01:29 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 cx14-20020a056000092e00b0034a2d0b9a4fsm3864981wrb.17.2024.05.03.07.01.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 07:01:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a38481c2-0955-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714744889; x=1715349689; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=A9cDuwhBHs1gnvDfwUBcGu+H7/TOZIK+IHm/APTdY5U=;
        b=ObuA4q4S0hgxw8NQ/S/mo/Wz5eOHmEGmAbNb2raodQO3kiT83efn+TOobDdZ9QEgzp
         5a86Z+CqhTEjc3tnfLhi6exXd1KmESwAMuKMFpURzq9sDZN24PCWAhdq7x8SMk+CwQ2m
         nQmGWTC4EhWz+Q/qNz+8bbrJv63FXzp9e3mxY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714744889; x=1715349689;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=A9cDuwhBHs1gnvDfwUBcGu+H7/TOZIK+IHm/APTdY5U=;
        b=aLTkid7XeVv1wA43dp7/3CHrld0ovA65c5qjG/vX5Kt8TZu58XDhtE/brKlwjfJhmn
         UJBbFjsvwH7PAYhkKrLNizfqa1NeBEPgvHgYgTO3D9JTvNMrQ0Sh/mcnceBYMWRvtg36
         MkfwrH2/5zsKB9hWGqT4C8NPZqf+03yqmVbWSIN7/4Ava79NbUJTMMFbuLanAsQDSbGC
         70+DxcbfKehmbWx/Uw6xFeAwbGdJIdQy07+iQ4K+rQDb49HnzcpPpLNNDywkUbKdrW00
         QGQLKjFuSvl6yjrHJcFFKtQh6NZh5zJvawtvsqc1vYmRo/hq3vUJDWSSvrFx8QiTy3LV
         NZsQ==
X-Gm-Message-State: AOJu0YwYe5TutgzzsqRBAU9TPyHhF76RXWQLQ3++2dHaQvvWIq3HxG8X
	5ESNr9bHwdNaUP+81fL1LtXM8CD8oohx3gGKxdav8WKK0EyGD9gWUoCyGNqj190=
X-Google-Smtp-Source: AGHT+IF/ileQ8zUAa2ZSVXa2HqDO9W+DmPGx0vnvkkUXGIOlD6ppabL47IYCGrvgHyno0BxDv/3VMg==
X-Received: by 2002:a5d:4690:0:b0:34d:91d:8f93 with SMTP id u16-20020a5d4690000000b0034d091d8f93mr2051359wrq.47.1714744888738;
        Fri, 03 May 2024 07:01:28 -0700 (PDT)
Date: Fri, 3 May 2024 16:01:27 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 01/12] VT-d: correct ATS checking for root complex
 integrated devices
Message-ID: <ZjTuN8Ydh0pVCoNZ@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <3686dae7-e005-47b5-9235-14208a68eec5@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <3686dae7-e005-47b5-9235-14208a68eec5@suse.com>

On Thu, Feb 15, 2024 at 11:13:24AM +0100, Jan Beulich wrote:
> Spec version 4.1 says
> 
> "The ATSR structures identifies PCI Express Root-Ports supporting
>  Address Translation Services (ATS) transactions. Software must enable
>  ATS on endpoint devices behind a Root Port only if the Root Port is
>  reported as supporting ATS transactions."
> 
> Clearly root complex integrated devices aren't "behind root ports",
> matching my observation on a SapphireRapids system having an ATS-
> capable root complex integrated device. Hence for such devices we
> shouldn't try to locate a corresponding ATSR.
> 
> Since both pci_find_ext_capability() and pci_find_cap_offset() return
> "unsigned int", change "pos" to that type at the same time.
> 
> Fixes: 903b93211f56 ("[VTD] laying the ground work for ATS")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

> ---
> v2: New.
> 
> --- a/xen/drivers/passthrough/vtd/x86/ats.c
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c
> @@ -44,7 +44,7 @@ struct acpi_drhd_unit *find_ats_dev_drhd
>  int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
>  {
>      struct acpi_drhd_unit *ats_drhd;
> -    int pos;
> +    unsigned int pos, expfl = 0;
>  
>      if ( !ats_enabled || !iommu_qinval )
>          return 0;
> @@ -53,7 +53,12 @@ int ats_device(const struct pci_dev *pde
>           !ecap_dev_iotlb(drhd->iommu->ecap) )
>          return 0;
>  
> -    if ( !acpi_find_matched_atsr_unit(pdev) )
> +    pos = pci_find_cap_offset(pdev->sbdf, PCI_CAP_ID_EXP);
> +    if ( pos )
> +        expfl = pci_conf_read16(pdev->sbdf, pos + PCI_EXP_FLAGS);
> +
> +    if ( MASK_EXTR(expfl, PCI_EXP_FLAGS_TYPE) != PCI_EXP_TYPE_RC_END &&
> +         !acpi_find_matched_atsr_unit(pdev) )

Given the spec quote above, it might also be helpful to check that the
type is PCI_EXP_TYPE_ENDPOINT before attempting the ATSR check?

I would assume a well formed ATSR won't list non-endpoint devices.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:03:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:03:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716493.1118643 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tVX-0008PM-Rc; Fri, 03 May 2024 14:03:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716493.1118643; Fri, 03 May 2024 14:03:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tVX-0008PF-NO; Fri, 03 May 2024 14:03:27 +0000
Received: by outflank-mailman (input) for mailman id 716493;
 Fri, 03 May 2024 14:03:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MkFU=MG=flex--seanjc.bounces.google.com=3q-40ZgYKCZQG2yB704CC492.0CAL2B-12J2996GHG.L2BDFC720H.CF4@srs-se1.protection.inumbo.net>)
 id 1s2tVW-0008P5-HI
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:03:26 +0000
Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com
 [2607:f8b0:4864:20::b4a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e87e86d9-0955-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:03:25 +0200 (CEST)
Received: by mail-yb1-xb4a.google.com with SMTP id
 3f1490d57ef6-de59ff8af0bso13577809276.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 07:03:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e87e86d9-0955-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20230601; t=1714745004; x=1715349804; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id
         :reply-to;
        bh=a136SOKV90n/XcmvCvD6zgQKrJ5kR/Q+OhiUt3FbOiE=;
        b=zvhRf2DDgMqroQU5HA7kOmbkw7yxHrj/yEbbtbWLAXQ7fyOyIVPpJvos59L39N/fuY
         cYLFeaWEGjIpunhb+m+I2h18VW00dK8vnoQbbkXzHutSG0O9CbUE2/dmdz32vIahuAVU
         1AIzyoRzvxFm4vic8c7ZbuMW7LiCGe6inUoTkLeC7ziJosDCcvOBCZBdRB8WqAgvuJ1t
         dnl5FjAzOM45sk39VO1pKAt2yBbipr2Fq2HGW/vZa8Jd+TRX6lKi31U82E7kvE+cAmzx
         RMIEOe9oFDuwtZ1KM1267QbhWeP34duM1gXGlwNW24lK1ZvlonXpedOCuHMZW1dSDfZL
         m04w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714745004; x=1715349804;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=a136SOKV90n/XcmvCvD6zgQKrJ5kR/Q+OhiUt3FbOiE=;
        b=bZYa6PRCjlLsIz3SmWS7suD8Uo0DSRhQJtf8lnIaJ7eigcu0B69CPx1qkhiH/0XIOY
         tFvt8ylqETGjVsuQI+egyApF3JJhW8bBm947+v2KUchjFkHT1z5PFbyKytQzff2kHH3x
         vPTyBimUcDwtPl7Vs1cjo+hdnRoVpFMVC18SVi+JJQeyPLjqHH8mCuw6Gfm8R8KjMeKY
         Eg3dr1PTGKwoZSeA0RFA8JkshNKzLIdThUgZ/N93jhHEQJSF6EbMmkAbO2yI702e6jGV
         ydEEo7oUYEHhkU8+3+8H7HSBr8n+ocyZ2BsqCqzkWZE/i2B76FtpViCgvpryqEtYtPKk
         xKXA==
X-Forwarded-Encrypted: i=1; AJvYcCUDC9mU0loYUtI0+bIvg9hJz8IaJXxNDq0y5wpYipOtKAT74jVJMeD4B9YkfpOB/A6900MguGSCUAY9FP4VGimtbdC5xhN/rvr2+MDMsqc=
X-Gm-Message-State: AOJu0YzidlMtp+HA2SHj+dZ1kr28+HnwDlL9MqX6Lw3jH8wHAmdyCfVd
	0f/sN5B5woYWTwDtGelvh7Gh+sXVuaRhG28ZBfqttF8fjrFUoB6Ou4lNt17Ksf9pDFvojyEKaGm
	IsA==
X-Google-Smtp-Source: AGHT+IG8Se+FDx3icVQiBS8y3Ql+XygTIGhFwTldpFeLgMxDQJNLd2QsRcCHI+DmhaCm9MeTHgIbmJIbKcw=
X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37])
 (user=seanjc job=sendgmr) by 2002:a05:6902:1201:b0:de6:141a:b0de with SMTP id
 s1-20020a056902120100b00de6141ab0demr337561ybu.10.1714745003300; Fri, 03 May
 2024 07:03:23 -0700 (PDT)
Date: Fri, 3 May 2024 07:03:21 -0700
In-Reply-To: <20240503131910.307630-4-mic@digikod.net>
Mime-Version: 1.0
References: <20240503131910.307630-1-mic@digikod.net> <20240503131910.307630-4-mic@digikod.net>
Message-ID: <ZjTuqV-AxQQRWwUW@google.com>
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
From: Sean Christopherson <seanjc@google.com>
To: "=?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?=" <mic@digikod.net>
Cc: Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	"Mihai =?utf-8?B?RG9uyJt1?=" <mdontu@bitdefender.com>, 
	"=?utf-8?B?TmljdciZb3IgQ8OuyJt1?=" <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, 
	Will Deacon <will@kernel.org>, Yu Zhang <yu.c.zhang@linux.intel.com>, 
	"=?utf-8?Q?=C8=98tefan_=C8=98icleru?=" <ssicleru@bitdefender.com>, dev@lists.cloudhypervisor.org, 
	kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, 
	virtualization@lists.linux-foundation.org, x86@kernel.org, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 03, 2024, Micka=C3=ABl Sala=C3=BCn wrote:
> Add an interface for user space to be notified about guests' Heki policy
> and related violations.
>=20
> Extend the KVM_ENABLE_CAP IOCTL with KVM_CAP_HEKI_CONFIGURE and
> KVM_CAP_HEKI_DENIAL. Each one takes a bitmask as first argument that can
> contains KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4. The
> returned value is the bitmask of known Heki exit reasons, for now:
> KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4.
>=20
> If KVM_CAP_HEKI_CONFIGURE is set, a VM exit will be triggered for each
> KVM_HC_LOCK_CR_UPDATE hypercalls according to the requested control
> register. This enables to enlighten the VMM with the guest
> auto-restrictions.
>=20
> If KVM_CAP_HEKI_DENIAL is set, a VM exit will be triggered for each
> pinned CR violation. This enables the VMM to react to a policy
> violation.
>=20
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Sean Christopherson <seanjc@google.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
> Cc: Wanpeng Li <wanpengli@tencent.com>
> Signed-off-by: Micka=C3=ABl Sala=C3=BCn <mic@digikod.net>
> Link: https://lore.kernel.org/r/20240503131910.307630-4-mic@digikod.net
> ---
>=20
> Changes since v1:
> * New patch. Making user space aware of Heki properties was requested by
>   Sean Christopherson.

No, I suggested having userspace _control_ the pinning[*], not merely be no=
tified
of pinning.

 : IMO, manipulation of protections, both for memory (this patch) and CPU s=
tate
 : (control registers in the next patch) should come from userspace.  I hav=
e no
 : objection to KVM providing plumbing if necessary, but I think userspace =
needs to
 : to have full control over the actual state.
 :=20
 : One of the things that caused Intel's control register pinning series to=
 stall
 : out was how to handle edge cases like kexec() and reboot.  Deferring to =
userspace
 : means the kernel doesn't need to define policy, e.g. when to unprotect m=
emory,
 : and avoids questions like "should userspace be able to overwrite pinned =
control
 : registers".
 :=20
 : And like the confidential VM use case, keeping userspace in the loop is =
a big
 : beneifit, e.g. the guest can't circumvent protections by coercing usersp=
ace into
 : writing to protected memory.

I stand by that suggestion, because I don't see a sane way to handle things=
 like
kexec() and reboot without having a _much_ more sophisticated policy than w=
ould
ever be acceptable in KVM.

I think that can be done without KVM having any awareness of CR pinning wha=
tsoever.
E.g. userspace just needs to ability to intercept CR writes and inject #GPs=
.  Off
the cuff, I suspect the uAPI could look very similar to MSR filtering.  E.g=
. I bet
userspace could enforce MSR pinning without any new KVM uAPI at all.

[*] https://lore.kernel.org/all/ZFUyhPuhtMbYdJ76@google.com


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:25:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:25:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716503.1118653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tqc-0005mu-I5; Fri, 03 May 2024 14:25:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716503.1118653; Fri, 03 May 2024 14:25:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2tqc-0005mn-DE; Fri, 03 May 2024 14:25:14 +0000
Received: by outflank-mailman (input) for mailman id 716503;
 Fri, 03 May 2024 14:25:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tqb-0005l7-4M; Fri, 03 May 2024 14:25:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tqa-0006Fh-Q7; Fri, 03 May 2024 14:25:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tqa-00058c-AW; Fri, 03 May 2024 14:25:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2tqa-0005ow-A0; Fri, 03 May 2024 14:25:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aJrmcKvUWL/SvCITpP3lHewix4bp560kZ1bDK5a/DPU=; b=k3KBOdjFKyQNHhQ2/YGvyJPlW5
	P+5oqcqyI+bknjgzmFU7UGOWXMmvHEdThIITZ4QXs6clMd3h68r7iuhVxzF6bCOmoOppepN9zYJDp
	rzTVNtHSfOxxt4u0E9FdEwguFvTSMzfMGbIPT0e+J6Y0L0eNxE/YcwuDlhREns48uUGE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185907-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185907: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f03359bca01bf4372cf2c118cd9a987a5951b1c8
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 14:25:12 +0000

flight 185907 linux-linus real [real]
flight 185910 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185907/
http://logs.test-lab.xenproject.org/osstest/logs/185910/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit2   8 xen-boot                 fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-vhd  8 xen-boot            fail pass in 185910-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185910-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 185910-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185870

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185870
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185910 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185910 never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check fail in 185910 never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check fail in 185910 never pass
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                f03359bca01bf4372cf2c118cd9a987a5951b1c8
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    3 days
Failing since        185888  2024-04-30 20:31:50 Z    2 days    6 attempts
Testing same since   185907  2024-05-03 02:43:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Potapenko <glider@google.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Protopopov <aspsk@isovalent.com>
  Arnd Bergmann <arnd@arndb.de>
  Asbjørn Sloth Tønnesen <ast@fiberby.net>
  Audra Mitchell <audra@redhat.com>
  Barry Song <v-songbaohua@oppo.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Björn Töpel <bjorn@kernel.org>
  Björn Töpel <bjorn@rivosinc.com>
  Bui Quang Minh <minhquangbui99@gmail.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Christoph Paasch <cpaasch@apple.com>
  Chuck Lever <chuck.lever@oracle.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Borkmann <daniel@iogearbox.net>
  David Bauer <mail@david-bauer.net>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Dima Ruinskiy <dima.ruinskiy@intel.com>
  Dominique Martinet <dominique.martinet@atmark-techno.com>
  Donald Hunter <donald.hunter@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guillaume Nault <gnault@redhat.com>
  Hans de Goede <hdegoede@redhat.com>
  Harald Freudenberger <freude@linux.ibm.com>
  Heiko Carstens <hca@linux.ibm.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Ivan Babrou <ivan@cloudflare.com>
  Jakub Kicinski <kuba@kernel.org>
  James Chapman <jchapman@katalix.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  Jason Xing <kernelxing@tencent.com>
  Jeffrey Altman <jaltman@auristor.com>
  Jens Remus <jremus@linux.ibm.com>
  Jesper Dangaard Brouer <hawk@kernel.org>
  John Garry <john.g.garry@oracle.com>
  Josef Bacik <josef@toxicpanda.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Marc Dionne <marc.dionne@auristor.com>
  Marek Behún <kabel@kernel.org>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <martin.lau@kernel.org>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Max Filippov <jcmvbkbc@gmail.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Abeni <pabeni@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Puranjay Mohan <puranjay12@gmail.com>
  Puranjay Mohan <puranjay@kernel.org>
  Qu Wenruo <wqu@suse.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Richard Gobert <richardbgobert@gmail.com>
  Sasha Neftin <sasha.neftin@intel.com>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislav Fomichev <sdf@google.com>
  Sven Schnelle <svens@linux.ibm.com>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tejun Heo <tj@kernel.org>
  Thanassis Avgerinos <thanassis.avgerinos@gmail.com>
  Thorsten Blum <thorsten.blum@toblux.com>
  Toke Høiland-Jørgensen <toke@redhat.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Vitaly Lifshits <vitaly.lifshits@intel.com>
  Xin Long <lucien.xin@gmail.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Kuohai <xukuohai@huawei.com>
  Yunsheng Lin <linyunsheng@huawei.com>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4031 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:25:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:25:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716506.1118664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2trD-0006SE-Sp; Fri, 03 May 2024 14:25:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716506.1118664; Fri, 03 May 2024 14:25:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2trD-0006S7-NW; Fri, 03 May 2024 14:25:51 +0000
Received: by outflank-mailman (input) for mailman id 716506;
 Fri, 03 May 2024 14:25:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s2trD-0006S1-Eg
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:25:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2trD-0006Hh-2A; Fri, 03 May 2024 14:25:51 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s2trC-0004Vl-QT; Fri, 03 May 2024 14:25:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=bPVlUAKs7UW93A0XEt85U0EiIRHfPEAdaURMKVqObvo=; b=Hius+Pc+V5D6Q93SoatW4VAE+u
	fUimwVxaNXHqfX/kkbzR4MVYdAk5dH2fJtBR97ptmPWkxq1T9V1Ofl7kNmcfB0lHlijcMNRDoiXI0
	t8SCY+kVQiJmo2Kb6IqyEKRkicBO2X47lIzQoLhcc7LERcCsjfzleosZg33AZMQzAKY0=;
Message-ID: <447340c3-d57a-411b-8d4d-29e8683b27d9@xen.org>
Date: Fri, 3 May 2024 15:25:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 5/5] xen/arm: ffa: support notification
Content-Language: en-GB
To: Jens Wiklander <jens.wiklander@linaro.org>
Cc: xen-devel@lists.xenproject.org, patches@linaro.org,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
 <20240502145645.1201613-6-jens.wiklander@linaro.org>
 <935e3947-53ee-48c5-a10c-5fa7f572ae65@xen.org>
 <CAHUa44HdA_cdZcjbVL+xYXnLVkdK1hK5hfU_k+fZkS8=VAOy1A@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAHUa44HdA_cdZcjbVL+xYXnLVkdK1hK5hfU_k+fZkS8=VAOy1A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jens,

On 03/05/2024 14:54, Jens Wiklander wrote:
>>> +static int ffa_setup_irq_callback(struct notifier_block *nfb,
>>> +                                  unsigned long action, void *hcpu)
>>> +{
>>> +    unsigned int cpu = (unsigned long)hcpu;
>>> +    struct notif_irq_info irq_info = { };
>>> +
>>> +    switch ( action )
>>> +    {
>>> +    case CPU_ONLINE:
>>
>> Can't you execute the notifier in CPU_STARTING? This will be called on
>> the CPU directly, so you should be able to use request_irq(...).
> 
> I tried that first but it failed with the ASSERT_ALLOC_CONTEXT() in _xmalloc().
> 
> I've also tested a three-step solution with CPU_UP_PREPARE,
> CPU_STARTING, and CPU_UP_CANCELED.
> My approach here is more direct, but it still suffers from a weakness
> in error handling even if it seems quite unlikely to run out of heap
> or for setup_irq() to fail at this stage.

Ah I didn't notice that notify_cpu_starting() is called with IRQ 
disabled. I assumed they would be enabled.

Then I would consider to do:

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 6efed876782e..db322672e508 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -389,6 +389,7 @@ void asmlinkage start_secondary(void)
       */
      init_maintenance_interrupt();
      init_timer_interrupt();
+    init_tee_interrupt();

      local_abort_enable();

And plumb through the TEE subsystem.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716523.1118723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6f-0003fN-CG; Fri, 03 May 2024 14:41:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716523.1118723; Fri, 03 May 2024 14:41:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6f-0003ej-9Z; Fri, 03 May 2024 14:41:49 +0000
Received: by outflank-mailman (input) for mailman id 716523;
 Fri, 03 May 2024 14:41:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6e-0002U8-84
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:48 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 41cc7170-095b-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:41:44 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id DF4528009F;
 Fri,  3 May 2024 15:41:41 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id ACED82018B; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cc7170-095b-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747301;
	bh=t0vnuYLk59oT/1dlHfshijBhS9X5czFtKDiWa/l8Kes=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=ooJVneUZTiXWaJZJAdl794ELI5sa5+myNhpJEMY/4OrE4hQ2pYvW+sgx/FuPRdLJJ
	 ILZLQRguSOz6HlG5PubvaGkKXAazeNqAnkOJOiiABst7mFGZ8kWMxsnwu/oLLxjlbp
	 xfdffJrTef0+bl/YRyKs2yCjvo4k5QfRs6KuM1NM=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 3/5] tools/hotplug/Linux: Add bridge VLAN support
Date: Fri,  3 May 2024 15:41:22 +0100
Message-Id: <20240503144124.12931-4-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240503144124.12931-1-leigh@solinno.co.uk>
References: <20240503144124.12931-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update add_to_bridge shell function to read the vid parameter
from xenstore and set the bridge LAN for the VID to the given
value. This only works when using the iproute2 bridge command,
so a warning is issued if using the legacy brctl command and a
vid is set.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/hotplug/Linux/xen-network-common.sh | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Linux/xen-network-common.sh
index 42fa704e8d..19a8b3c7e5 100644
--- a/tools/hotplug/Linux/xen-network-common.sh
+++ b/tools/hotplug/Linux/xen-network-common.sh
@@ -125,14 +125,23 @@ create_bridge () {
 add_to_bridge () {
     local bridge=$1
     local dev=$2
+    local vid=$(xenstore_read_default "$XENBUS_PATH/vid" "")
 
     # Don't add $dev to $bridge if it's already on the bridge.
     if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
         log debug "adding $dev to bridge $bridge"
         if which brctl >&/dev/null; then
             brctl addif ${bridge} ${dev}
+            if [ -n "${vid}" ] ;then
+                log warning "bridge command not available, ignoring vid"
+            fi
         else
             ip link set ${dev} master ${bridge}
+            if [ -n "${vid}" ] ;then
+                log debug "Assigning $dev to vid $vid"
+                bridge vlan del dev ${dev} vid 1
+                bridge vlan add dev ${dev} vid ${vid} pvid untagged
+            fi
         fi
     else
         log debug "$dev already on bridge $bridge"
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716520.1118682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002b7-Lu; Fri, 03 May 2024 14:41:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716520.1118682; Fri, 03 May 2024 14:41:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002ZN-IC; Fri, 03 May 2024 14:41:46 +0000
Received: by outflank-mailman (input) for mailman id 716520;
 Fri, 03 May 2024 14:41:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6b-0002U7-Je
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:45 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 41cc299f-095b-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 16:41:42 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id D1DED80064;
 Fri,  3 May 2024 15:41:41 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id A727620175; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cc299f-095b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747301;
	bh=cdGs513PBK2hbMNX29q5EIK8D//y8QcA4L1hJTCLoOY=;
	h=From:To:Cc:Subject:Date:From;
	b=FbAE+OGdlIjs0P/Qwc1fwKPXRjmbs8YQScxXKVL6jX3GhnArzk7Y/sZZKmVaVT0rB
	 lK8DB5qhRGmwcLsmO5quYE88nRad6WZYyeRDiU45JIJ9RultUG7dYBcuCP0d+RFexy
	 WuxpqEEEVTvNzQXwEdyFOxuKKqVyHfXhY0NHZCo4=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 0/5] Add bridge VLAN support
Date: Fri,  3 May 2024 15:41:19 +0100
Message-Id: <20240503144124.12931-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

For many years I have been configuring VLANs on my Linux Dom0 by
creating VLAN interfaces for each VLAN I wanted to connect a domain
to and then a corresponding bridge. So I would tend to have things
like:

enp0s0	  -> br0     -> vif1, vif2
enp0s0.10 -> br0vl10 -> vif3, vif4
enp0s0.20 -> br0vl20 -> vif5
dummy0    -> br1     -> vif6

I recently discovered that iproute2 supports creating bridge VLANs that
allows you to assign a VLAN to each of the interfaces associated to a 
bridge. This allows a greatly simplified configuration where a single 
bridge can support all the domains, and the iproute2 bridge command can 
assign each VIF to the required VLAN.  This looks like this:

# bridge vlan
port              vlan-id  
enp0s0            1 PVID Egress Untagged
                  10
                  20
br0               1 PVID Egress Untagged
vif1.0            1 PVID Egress Untagged
vif2.0            10 PVID Egress Untagged
vif3.0            10 PVID Egress Untagged
vif4.0            20 PVID Egress Untagged
vif5.0            20 PVID Egress Untagged
vif6.0            30 PVID Egress Untagged

This patch set enables this capability as follows:

1. Adds `vid' as a new member of the libxl_device_nic structure;
2. Adds support to read and write vid from the xenstore;
3. Adds `vid' as a new keyword for the vif configuration option;
4. Adds support for assign the bridge VLAN in the Linux hotplug scripts.

I don't believe NetBSD or FreeBSD support this capability, but if they
do please point me in the direction of some documentation and/or examples.

NB: I'm not very familiar with Xen code base so may have missed
something important, although I have tested it and it is working well
for me.

Cheers,

Leigh.


leigh@solinno.co.uk (5):
  tools/libs/light: Add vid field to libxl_device_nic
  tools/xl: add vid keyword vif option
  tools/hotplug/Linux: Add bridge VLAN support
  docs/man: document VIF vid keyword
  tools/examples: Examples Linux bridge VLAN config

 docs/man/xl-network-configuration.5.pod.in    |  6 +++
 tools/examples/linux-bridge-vlan/README       | 52 +++++++++++++++++++
 tools/examples/linux-bridge-vlan/br0.netdev   |  7 +++
 tools/examples/linux-bridge-vlan/br0.network  |  8 +++
 .../examples/linux-bridge-vlan/enp0s0.network | 16 ++++++
 tools/hotplug/Linux/xen-network-common.sh     |  9 ++++
 tools/libs/light/libxl_nic.c                  | 20 +++++++
 tools/libs/light/libxl_types.idl              |  1 +
 tools/xl/xl_parse.c                           |  2 +
 9 files changed, 121 insertions(+)
 create mode 100644 tools/examples/linux-bridge-vlan/README
 create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
 create mode 100644 tools/examples/linux-bridge-vlan/br0.network
 create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network

-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716522.1118708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6e-0003Dw-8X; Fri, 03 May 2024 14:41:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716522.1118708; Fri, 03 May 2024 14:41:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6e-0003Cc-1D; Fri, 03 May 2024 14:41:48 +0000
Received: by outflank-mailman (input) for mailman id 716522;
 Fri, 03 May 2024 14:41:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6d-0002U8-80
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:47 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 41cc7177-095b-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:41:44 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id D40B88009E;
 Fri,  3 May 2024 15:41:41 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id A86C020171; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cc7177-095b-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747301;
	bh=QhPCF+Da6a6rJ80ZqmX2rlEn+1VBdptYD+Uv3wW6YvA=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=vf4ivSKnOtFHHamPhhsa9RrmD62thFAMLYjCSsHRbp37Nr7lydHqxJ+RpEYMfFpkp
	 jQPt019Su7UiiRVXWTHIGlmJcNxnYXK7v95BKn3Wa2c6sQsnyZoVg5LTWFeq1i3K7H
	 7eGqRtT/4a2rfc9rBj1h+D619duHqGPviuoy0Oto=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 1/5] tools/libs/light: Add vid field to libxl_device_nic
Date: Fri,  3 May 2024 15:41:20 +0100
Message-Id: <20240503144124.12931-2-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240503144124.12931-1-leigh@solinno.co.uk>
References: <20240503144124.12931-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add integer member `vid' to libxl_device_nic, to represent the
VLAN ID to assign to the VIF when adding it to the bridge device.

Update libxl_nic.c to read and write the vid field from the
xenstore.

This provides the capability for supported operating systems (e.g.
Linux) to perform VLAN filtering on bridge ports.  The Xen
hotplug scripts need to be updated to read this information from
then xenstore and perform the required configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/libs/light/libxl_nic.c     | 20 ++++++++++++++++++++
 tools/libs/light/libxl_types.idl |  1 +
 2 files changed, 21 insertions(+)

diff --git a/tools/libs/light/libxl_nic.c b/tools/libs/light/libxl_nic.c
index d6bf06fc34..e28b9101ee 100644
--- a/tools/libs/light/libxl_nic.c
+++ b/tools/libs/light/libxl_nic.c
@@ -233,6 +233,11 @@ static int libxl__set_xenstore_nic(libxl__gc *gc, uint32_t domid,
         flexarray_append(back, GCSPRINTF("%u", nic->mtu));
     }
     
+    if (nic->vid) {
+        flexarray_append(back, "vid");
+        flexarray_append(back, GCSPRINTF("%u", nic->vid));
+    }
+
     flexarray_append(back, "bridge");
     flexarray_append(back, libxl__strdup(gc, nic->bridge));
     flexarray_append(back, "handle");
@@ -313,6 +318,21 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const char *libxl_path,
         nic->mtu = LIBXL_DEVICE_NIC_MTU_DEFAULT;
     }
 
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                GCSPRINTF("%s/vid", libxl_path), &tmp);
+    if (rc) goto out;
+    if (tmp) {
+        char *endptr;
+
+        nic->vid = strtol(tmp, &endptr, 10);
+        if (*endptr != '\0') {
+            rc = ERROR_INVAL;
+            goto out;
+        }
+    } else {
+        nic->vid = 0;
+    }
+
     rc = libxl__xs_read_checked(gc, XBT_NULL,
                                 GCSPRINTF("%s/mac", libxl_path), &tmp);
     if (rc) goto out;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 7d8bd5d216..df5185128c 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -809,6 +809,7 @@ libxl_device_nic = Struct("device_nic", [
     ("backend_domname", string),
     ("devid", libxl_devid),
     ("mtu", integer),
+    ("vid", integer),
     ("model", string),
     ("mac", libxl_mac),
     ("ip", string),
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716518.1118673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002UV-73; Fri, 03 May 2024 14:41:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716518.1118673; Fri, 03 May 2024 14:41:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002UO-47; Fri, 03 May 2024 14:41:46 +0000
Received: by outflank-mailman (input) for mailman id 716518;
 Fri, 03 May 2024 14:41:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6a-0002U7-VG
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:45 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 41cbcce7-095b-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 16:41:42 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id E5FE8800A5;
 Fri,  3 May 2024 15:41:41 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id AF1072018C; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cbcce7-095b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747301;
	bh=KhHuubhLIPBWRZvjJf4xnGUWWnqkkuWA6mNwGxiVRt8=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=yIHTe500zhFyoQtaZ4zkjaCkZJa6Q5BneUmw1Ijkt/ZikST7LrcEkv+oFVFan5vzS
	 bbMcZAeKEttsSMKI8zH3AbBoJ6rOaM6Xi1a6B5i74cX3NDlGLuXllCTrOGuinevZu4
	 xNX6FFJ4xgcgNuBu++xFPPYyBBdXoVW7Saa/25jc=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 4/5] docs/man: document VIF vid keyword
Date: Fri,  3 May 2024 15:41:23 +0100
Message-Id: <20240503144124.12931-5-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240503144124.12931-1-leigh@solinno.co.uk>
References: <20240503144124.12931-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Document the new `vid' keyword in xl-network-configuration(5).

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 docs/man/xl-network-configuration.5.pod.in | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/docs/man/xl-network-configuration.5.pod.in b/docs/man/xl-network-configuration.5.pod.in
index f3e379bcf8..fe2615ae30 100644
--- a/docs/man/xl-network-configuration.5.pod.in
+++ b/docs/man/xl-network-configuration.5.pod.in
@@ -259,6 +259,12 @@ Specifies the MTU (i.e. the maximum size of an IP payload, exclusing headers). T
 default value is 1500 but, if the VIF is attached to a bridge, it will be set to match
 unless overridden by this parameter.
 
+=head2 vid
+
+Specifies the VLAN ID. If this is set to a non-zero value, it will be specified
+when attaching the VIF to a bridge.  This can be used on operating systems that
+support bridge VLANs (e.g. Linux using iproute2).
+
 =head2 trusted / untrusted
 
 An advisory setting for the frontend driver on whether the backend should be
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716519.1118677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002X3-Ey; Fri, 03 May 2024 14:41:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716519.1118677; Fri, 03 May 2024 14:41:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6c-0002Vy-A4; Fri, 03 May 2024 14:41:46 +0000
Received: by outflank-mailman (input) for mailman id 716519;
 Fri, 03 May 2024 14:41:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6b-0002U8-JE
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:45 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 41cc7167-095b-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:41:44 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id E26C1800A4;
 Fri,  3 May 2024 15:41:41 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id AAA762018A; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cc7167-095b-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747301;
	bh=y7+Cz3k/mrxS1xulR9+YhBk7BokYkEDmmQsPxFpElbg=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=lJi3qh4AzWO5q9AsQrDONF2oTzG3702q0mdtoTkVL8nMrWVE6FiAi5RKy16DYKvLI
	 K8bF5MLkVpKkLkM9SnoC824e6fhd1b5xwbdhbvlV+2a1Lu5XPpyDsfUh/5IuhUhWdI
	 ETt2YJGJYe/y20Ea6kvV+GDzEpRR8zBKyTxf7GNI=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 2/5] tools/xl: add vid keyword vif option
Date: Fri,  3 May 2024 15:41:21 +0100
Message-Id: <20240503144124.12931-3-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240503144124.12931-1-leigh@solinno.co.uk>
References: <20240503144124.12931-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update parse_nic_config() to support a new `vid' keyword. This
keyword specifies the numeric VLAN ID to assign to the VIF when
attaching it to the bridge port, on operating systems that support
the capability (e.g. Linux).

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/xl/xl_parse.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index ed983200c3..5d5dd4ec04 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -565,6 +565,8 @@ int parse_nic_config(libxl_device_nic *nic, XLU_Config **config, char *token)
         nic->devid = parse_ulong(oparg);
     } else if (MATCH_OPTION("mtu", token, oparg)) {
         nic->mtu = parse_ulong(oparg);
+    } else if (MATCH_OPTION("vid", token, oparg)) {
+        nic->vid = parse_ulong(oparg);
     } else if (!strcmp("trusted", token)) {
         libxl_defbool_set(&nic->trusted, true);
     } else if (!strcmp("untrusted", token)) {
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 14:41:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 14:41:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716521.1118703 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6d-0003BK-UR; Fri, 03 May 2024 14:41:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716521.1118703; Fri, 03 May 2024 14:41:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2u6d-0003B2-RI; Fri, 03 May 2024 14:41:47 +0000
Received: by outflank-mailman (input) for mailman id 716521;
 Fri, 03 May 2024 14:41:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8QA1=MG=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s2u6c-0002U8-7x
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 14:41:46 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 43139d67-095b-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 16:41:44 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 0A049800AB;
 Fri,  3 May 2024 15:41:42 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id B133E2018D; Fri,  3 May 2024 15:41:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43139d67-095b-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1714747302;
	bh=a0ZpqrsC8JFCevX5tRnZgArUyWq4u2VycjvhQNh9u1w=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=OEfqaY478oIg2vS+Gun7DbFYJ1k3n+O0upYaUFHKMuyVbnqsCJJum8fmDve9LmCMr
	 znkAJm/f0g+aq7YPKGvmflVdOm76LdBA1nLHCQz15e24Uln2nrAQStclLwh0Afr/b9
	 c0T0QShXykumfgqFhAdJL7Mf+X0g7+cCxQ9kVQHw=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH 5/5] tools/examples: Examples Linux bridge VLAN config
Date: Fri,  3 May 2024 15:41:24 +0100
Message-Id: <20240503144124.12931-6-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240503144124.12931-1-leigh@solinno.co.uk>
References: <20240503144124.12931-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new directory linux-bridge-vlan showing how to configure
systemd-networkd to support a bridge VLAN configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/examples/linux-bridge-vlan/README       | 52 +++++++++++++++++++
 tools/examples/linux-bridge-vlan/br0.netdev   |  7 +++
 tools/examples/linux-bridge-vlan/br0.network  |  8 +++
 .../examples/linux-bridge-vlan/enp0s0.network | 16 ++++++
 4 files changed, 83 insertions(+)
 create mode 100644 tools/examples/linux-bridge-vlan/README
 create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
 create mode 100644 tools/examples/linux-bridge-vlan/br0.network
 create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network

diff --git a/tools/examples/linux-bridge-vlan/README b/tools/examples/linux-bridge-vlan/README
new file mode 100644
index 0000000000..b287710e0f
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/README
@@ -0,0 +1,52 @@
+Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
+=====================================================================
+
+Introduction
+------------
+
+This directory contains example files to be placed in /etc/systemd/network
+to enable a single bridge with multiple VLAN support.
+
+The example is to support the scenario where the Xen host network interface
+is connected to an Ethernet switch configured as a trunk port. Each domain
+VIF can then be configured with the VLAN id (vid) of the required VLAN.
+
+The example files create a bridge device called br0, with a physical interface 
+called enp0s0. You will need to update this with your system's device name.
+
+Key points of the configuration are:
+
+1. In br0.netdev, VLANFiltering=on is set. This is required to ensure the
+   VLAN tags are handled correctly.  If it is not set then the packets
+   from the vif interfaces will not have the correct VLAN tags set.  I
+   observed them with the pvid in the switch MAC address table.
+
+2. In br0.network, a system IPv4 address is configured that can be updated
+   according to your local network settings.
+
+3. In enp0s0.network, Bridge=br0 sets the bridge device to connect to and
+   there is a [BridgeVLAN] section for each VLAN you want to give access
+   to the switch. Note, if you want to create an internal VLAN private to
+   the host, do not include that VLAN id in this file.
+
+
+Domain configuration
+--------------------
+
+Add the vid= keyword to the vif definition in the domain. For example:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vid=10' ]
+
+
+Hints and tips
+--------------
+
+1. To check if vlan_filtering is enabled, run:
+   # cat /sys/devices/virtual/net/<name>/bridge/vlan_filtering
+
+2. To check the bridge port VLAN assignments, run:
+   # bridge vlan
+
+3. To check the vid setting in the xenstore, run:
+   # xenstore-ls -f | grep 'vid ='
+
diff --git a/tools/examples/linux-bridge-vlan/br0.netdev b/tools/examples/linux-bridge-vlan/br0.netdev
new file mode 100644
index 0000000000..ae1fe487c3
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/br0.netdev
@@ -0,0 +1,7 @@
+[NetDev]
+Name=br0
+Kind=bridge
+MACAddress=xx:xx:xx:xx:xx:xx
+
+[Bridge]
+VLANFiltering=on
diff --git a/tools/examples/linux-bridge-vlan/br0.network b/tools/examples/linux-bridge-vlan/br0.network
new file mode 100644
index 0000000000..b56203b66a
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/br0.network
@@ -0,0 +1,8 @@
+[Match]
+Name=br0
+
+[Network]
+DNS=8.8.8.8
+#Domains=example.com
+Address=10.1.1.10/24
+Gateway=10.1.1.1
diff --git a/tools/examples/linux-bridge-vlan/enp0s0.network b/tools/examples/linux-bridge-vlan/enp0s0.network
new file mode 100644
index 0000000000..6ee3154dfc
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/enp0s0.network
@@ -0,0 +1,16 @@
+[Match]
+Name=enp0s0
+
+[Network]
+Bridge=br0
+
+# If Jumbo frames are required
+#[Link]
+#MTUBytes=9000
+
+[BridgeVLAN]
+VLAN=10
+
+[BridgeVLAN]
+VLAN=20
+
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 03 16:55:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 16:55:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716599.1118733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2wBK-0000m0-Hv; Fri, 03 May 2024 16:54:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716599.1118733; Fri, 03 May 2024 16:54:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2wBK-0000lS-El; Fri, 03 May 2024 16:54:46 +0000
Received: by outflank-mailman (input) for mailman id 716599;
 Fri, 03 May 2024 16:54:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rrlW=MG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2wBI-0000lM-NB
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 16:54:44 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d70b7299-096d-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 18:54:43 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59a17fcc6bso9027766b.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 09:54:43 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 oz19-20020a170906cd1300b00a599165a96esm859398ejb.23.2024.05.03.09.54.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 09:54:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d70b7299-096d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714755283; x=1715360083; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=qniKA5uv84tUxFNOf3yqEYy7oL/o+0y9C1S9PMYnU6A=;
        b=SV493vGrdbcgW30zOiAIVNIjwf4SriGxIu6KYGBuihjkXtHb8I1ui0CUvKSn+FpYmk
         79fNzJQbrUA6VYLfYy1Wdqb7YWc/IeIQISnujBRJsBVz7MrP+L2Z9vD87xj9Ni/SbQlX
         GSwuz+6YGsvaU779xXg8J0XqngQaObFUmjNPncw2eBG00AwuESYpcKwi60659pToVLNZ
         JPQiBSayEpWs52gQ3r6xmDURKfzsDaWNtbSyQblkDIg9KX58GfK1IRLk7jc6hyM0g8Eg
         TsMTo+I4Q7CjyKkmt289+qoAGPINWrqdnt3yv8FKuzb5iIBGgSwLDQFnI+OQCiIBN4o5
         QyJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714755283; x=1715360083;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qniKA5uv84tUxFNOf3yqEYy7oL/o+0y9C1S9PMYnU6A=;
        b=KtARv5ryYTCYl4N2aPPBOD7mwly/Ly2oXwyBHdjmXOAQ79NTciWobP3zeGl6jtyZuy
         MWY+0mwiDZkLyWX8guJrdnoqBTd8Zw0XCK93vkqJRHc/Y5N3RR/qeTGkvRGjNVFwEGBh
         JPnkUb8XTme9WOZqPotDzjkScl0qBfnG2Km4nCpkVnzoJme6O6kBDzGtxB/X215AAdkn
         JptCAyuR8uX3FH0aREJNEuznXVdODoQOEVgURIPZg0vFOeIQf+10+RHRz7XhRGeZW8KA
         W6Vp2PtD2siuZaIXOchWn/XDfw5eXPz+HhYtO09gnyXIItSbAYKZMKfw4yQBQ0H/K2Fh
         4c5g==
X-Gm-Message-State: AOJu0YzXL2Q1ndy+cetoxUdwahio8i5X0IlijpemQ2XU4JUySIGaKqGR
	Y9ZNiC8qVr3hZ39rPPlfk/5SdlwdtXj5zdChZyitib7YnqNXy9yUaPxLpw==
X-Google-Smtp-Source: AGHT+IF0vYB5VeZvt3MCe9FqwZ2rLEyEWO3Vkk0i49AX4QzY/2wFIecq0iAL1B1Ndq2YbPDB/KUs+g==
X-Received: by 2002:a17:906:3ad3:b0:a58:f186:8af with SMTP id z19-20020a1709063ad300b00a58f18608afmr2150467ejd.60.1714755282493;
        Fri, 03 May 2024 09:54:42 -0700 (PDT)
Message-ID: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
Subject: Xen 4.19 release status tracking list [ May ]
From: Oleksii <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: :, committers@xenproject.org, community.manager@xenproject.org, Kelly
 Choi <kelly.choi@cloud.com>
Date: Fri, 03 May 2024 18:54:40 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

Hello everyone,

I would like to share with you a list for status tracking based on Xen
ML and community members comments:

*** Arm ***:
  * Arm cache coloring [
https://lore.kernel.org/xen-devel/20240502165533.319988-1-carlo.nonato@mine=
rvasys.tech/
]:
    - new patch series version [v8] was sent

  * PCI devices passthrough on Arm, part 3 [
https://lore.kernel.org/xen-devel/20240202213321.1920347-1-stewart.hildebra=
nd@amd.com/
]
=20
  * DOMCTL-based guest magic region allocation for 11
domUs [
https://lore.kernel.org/xen-devel/20240409045357.236802-1-xin.wang2@amd.com=
/
]
     - new patch series verstion [v4] was sent
=20
  * [XEN v6 0/3] xen/arm: Add emulation of Debug Data Transfer
Registers [
https://patchew.org/Xen/20240307123943.1991755-1-ayan.kumar.halder@amd.com/
]

*** PPC ***:
  * [PATCH v4 0/6] Early Boot Allocation on Power [
https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptoreng=
ineering.com/
]:
    new patch series version [v4] was sent

*** RISC-V ***:
  * [PATCH v8 00/17] Enable build of full Xen for RISC-V
https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@gma=
il.com/
]:
    - several patches were merged
    - new patch series version [v8] were sent


*** x86 ***:
  * [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks [
https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citrix.c=
om/
]:
    - almost patch series have been merged already except the patch:
        [PATCH 4/4] iommu/x86: make unity range checking more strict

  * [PATCH 0/8] x86: support AVX10.1 [
https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@suse=
.com/
]:
    - two patches of patch series are waitng to merged/reviewed:
      [PATCH 1/4] amd-vi: fix IVMD memory type checks
      [PATCH 4/4] iommu/x86: make unity range checking more strict=20
 =20
  * APX support?

  * [PATCH v4 0/8] x86emul: misc additions [
https://lore.kernel.org/xen-devel/9dd23064-c79e-4a50-9c71-c0e73b189944@suse=
.com/
]
  =20
  * [PATCH v2 00/12] VT-d: SATC handling; ATS: tidying[
https://lore.kernel.org/xen-devel/64b028be-2197-4951-ae5b-32f9eabfa84a@suse=
.com/
]:
    new version was sent [ v2 ]

  * [XEN PATCH 0/9] x86: parallelize AP bring-up during boot [
https://lore.kernel.org/xen-devel/cover.1699982111.git.krystian.hebel@3mdeb=
.com/
]

  * [PATCH v2 00/12] x86: memcpy() / memset() (non-)ERMS flavors plus
fallout [
https://lore.kernel.org/xen-devel/8f56a8f4-0482-932f-96a9-c791bebb4610@suse=
.com/
]
    - 6/12 are merged.
=20
  * [PATCH v6 0/4] x86/pvh: Support relocating dom0 kernel [
https://patchew.org/Xen/20240327215102.136001-1-jason.andryuk@amd.com/
]
=20
  * x86/spec-ctrl: IBPB improvements [
https://patchew.org/Xen/06591b64-2f05-a4cc-a2f3-a74c3c4a76d6@suse.com/
]


*** common ***:
  * annotate entry points with type and size" series:
    The bulk of this has gone in, but there'll want to be follow-ups.

  * [PATCH v2 (resend) 00/27] Remove the directmap [
https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazon.co=
m/
]
    - 7/27 were merged.
=20
  * [PATCH] move __read_mostly to xen/cache.h [
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse=
.com/
]

  * [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [
https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.=
1706259490.git.federico.serafini@bugseng.com/
]

  * MISRA rules updates:
   - [PATCH v2] docs/misra/rules.rst update [
https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2402131431070.1925432=
@ubuntu-linux-20-04-desktop/T/#maded3df1bebe68d0fe53c73e89f996ec395a39e5
]: 2/3 were merged.

   - [XEN PATCH v3 0/7] address violations of MISRA C Rule 20.7[
https://patchew.org/Xen/cover.1711700095.git.nicola.vetrini@bugseng.com/
]: new patch series version (v3) were sent.

   - [XEN PATCH v3 00/16] xen: address violation of MISRA C:2012
Directive 4.10 [
https://patchew.org/Xen/cover.1710145041.git.simone.ballarin@bugseng.com/
]: 2/16 were merged.

  * [PATCH v6 00/8] xen/spinlock: make recursive spinlocks a dedicated
type [ https://patchew.org/Xen/20240327152229.25847-1-jgross@suse.com/
]:
    - only one patch required to be merged:
    =20
https://patchew.org/Xen/20240327152229.25847-1-jgross@suse.com/202403271522=
29.25847-9-jgross@suse.com/
  =20
  * [PATCH 0/7] GRUB: Supporting Secure Boot of xen.gz [
https://patchew.org/Xen/20240313150748.791236-1-ross.lagerwall@citrix.com/
]:
  =20
  * [PATCH v6 0/7] MSI-X support with qemu in stubdomain, and other
related changes:
    - new patch version was sent
=20
  * [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [
https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/
]:
=20
  * [PATCH 0/7] xen/trace: Treewide API cleanup [
https://patchew.org/Xen/20240318163552.3808695-1-andrew.cooper3@citrix.com/
]:
    patches were merged to staging:
     - [PATCH 3/7] xen/rt: Clean up trace handling
     - [PATCH 4/7] xen/sched: Clean up trace handling
=20
  * [PATCH v3 0/4] xenwatchdogd bugfixes and enhancements [
https://lore.kernel.org/xen-devel/20240411182023.56309-1-leigh@solinno.co.u=
k/
]:
    new patch series were sent.

  * [RFC XEN PATCH v7 0/5] Support device passthrough when dom0 is PVH
on Xen [
https://patchew.org/Xen/20240419035340.608833-1-Jiqian.Chen@amd.com/ ]:
    - new patch series were sent [v7].


*** Completed ***:
  *** Arm ***:
     * xen/arm64: Rework the MMU-off code (idmap) so it is self-
contained

  *** x86 ***:
     * tools: enable xenstore-stubdom to use 9pfs

  *** common ***:
     * NUMA: no need for asm/numa.h when !NUMA
     * xen: move BUG_ON(), WARN_ON(), ASSERT(), ASSERT_UNREACHABLE() to
xen/bug.h
     * xen/lib: introduce generic find next bit operations
     * Introduce generic headers
     * xen/livepatch: fixes for the pre-apply / post-revert hooks
     * limit passing around of cpu_user_regs

Please reply with items you would like to see in 4.19 so that people
know what is happening and prioritize accordingly.
You're welcome to provide a description and use cases of the feature
you're working on.

Have a nice week!

Best regards,
 Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 03 17:15:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 17:15:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716621.1118754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2wVI-0004ZL-Bk; Fri, 03 May 2024 17:15:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716621.1118754; Fri, 03 May 2024 17:15:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2wVI-0004ZE-9A; Fri, 03 May 2024 17:15:24 +0000
Received: by outflank-mailman (input) for mailman id 716621;
 Fri, 03 May 2024 17:15:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rrlW=MG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s2wVG-0004Z8-S1
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 17:15:22 +0000
Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com
 [2607:f8b0:4864:20::72d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b8bfa968-0970-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 19:15:22 +0200 (CEST)
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-792639cf4faso271558085a.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 10:15:21 -0700 (PDT)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 bp10-20020a05620a458a00b00790f7c28019sm1392913qkb.83.2024.05.03.10.15.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 10:15:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8bfa968-0970-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714756520; x=1715361320; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=nHhDPQFODDTSRPzwFDqZrnLgjpslzU8jT3iE1YAlVrg=;
        b=A/0qTA8ged8P3qVzYxwpKF8S7e8jj4xyUBX3NVpaeIfe5uzqTEyHWobTCDRaCSe6tN
         jtCrPJcibth6OM/e9/Gzrb2RgS+WWt8aqAs+ffoTwnbVXJ8ypbpQSl+2vHnqUprSMhNc
         Y0eCgHyPfsondVYDZWkvu649rV6mL5C/7AZlqhGsG4jLUzBZsZnnCZU4jbhDgEOPcFE5
         qU3r8Opn0o4dKeVDKVH0PVEZ+inbXYScGdby2wNSvJZmCPuj3OJ+PvuI1++lShOPkUC6
         m+nvgWsTaF+eIjGBmn+lF1l7FcPm95fOJ/+psd27UQPSci8aPt26nhY9pM4lypmBsKet
         s+7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714756520; x=1715361320;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nHhDPQFODDTSRPzwFDqZrnLgjpslzU8jT3iE1YAlVrg=;
        b=aXs6aMDRFIWcQWilZsxV/fQEHTPs+ECMyy+djzhLqR05+5Ef7eYWsM8XGgW95bh7rW
         1TfOUZGSAz5Edy462mInDdBQoTyTwyf2PmW3Tk0eNf4syUSLcXWq5iLZkpAMTKj3kRUV
         oH5Q0BfEB4EFJTtpMRp9S76aot7mewNWYM4ATDShtF2lAIM+rxTeho3YNTRmn3nT7lWR
         tpt1foZBUhxn2s/iiDqK0Uu23ZutmZePxPJQXehH4wT3lSekmdbn4hjUU6/2buXQC+WO
         J5V3RC4mOoWxgg3ARhB7eGxBxzYshu3u/W32Wh2o2OCqBE00uY/WpkM03LCjCCs5R+NS
         BUWg==
X-Forwarded-Encrypted: i=1; AJvYcCVpjlOiGQn0nrT3Sk1JnbKhJeqEjgGMf7WhD5ot2pfSMhr6ugD6377WvB/aq0vCjfd4Cx9TuyvKR9wV80UhpxDu3JRmnMEyZ0THWWY+5D4=
X-Gm-Message-State: AOJu0Ywf9s5ulDUAug0is09nXIueal1fLy7Yk6RKFOfSPOTl8qbsyQxu
	+KmMmDl/dZDiZOFHoSrRoE9/d26DI8K/K+VcaNxahR8Lo8sxGIoF
X-Google-Smtp-Source: AGHT+IHgB1ff01//xX1SHLUX6dnh+sR6UhOUAYbvSHppPFKINWQXkW2DrdL8DfuFC1M5nIGHpMC1hA==
X-Received: by 2002:a05:620a:40c3:b0:790:9688:4450 with SMTP id g3-20020a05620a40c300b0079096884450mr3736814qko.53.1714756520585;
        Fri, 03 May 2024 10:15:20 -0700 (PDT)
Message-ID: <940f11ee23b43ada1dba50bc0236c4764eb13d71.camel@gmail.com>
Subject: Re: [PATCH v8 02/17] xen: introduce generic non-atomic test_*bit()
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Fri, 03 May 2024 19:15:16 +0200
In-Reply-To: <3827c11c-6d47-411d-a356-871def4e5b30@suse.com>
References: <cover.1713347222.git.oleksii.kurochko@gmail.com>
	 <1a0977e3cf5a2de9f760ca5ec89a0d096894a9e3.1713347222.git.oleksii.kurochko@gmail.com>
	 <3827c11c-6d47-411d-a356-871def4e5b30@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-04-25 at 17:35 +0200, Jan Beulich wrote:
> > =C2=A0 #include <asm/bitops.h>
> > =C2=A0=20
> > +#ifndef arch_check_bitop_size
> > +#define arch_check_bitop_size(addr)
>=20
> Can this really do nothing? Passing the address of an object smaller
> than
> bitop_uint_t will read past the object in the generic__*_bit()
> functions.
It seems RISC-V isn' happy with the following generic definition:
   extern void __bitop_bad_size(void);
  =20
   /* --------------------- Please tidy above here --------------------
   - */
  =20
   #include <asm/bitops.h>
  =20
   #ifndef arch_check_bitop_size
  =20
   #define bitop_bad_size(addr) sizeof(*(addr)) < sizeof(bitop_uint_t)
  =20
   #define arch_check_bitop_size(addr) \
       if ( bitop_bad_size(addr) ) __bitop_bad_size();
  =20
   #endif /* arch_check_bitop_size */

The following errors occurs. bitop_uint_t for RISC-V is defined as
unsigned long for now:
    ./common/symbols-dummy.o -o ./.xen-syms.0
riscv64-linux-gnu-ld: prelink.o: in function `atomic_sub':
/build/xen/./arch/riscv/include/asm/atomic.h:152: undefined reference
to `__bitop_bad_size'
riscv64-linux-gnu-ld: prelink.o: in function `evtchn_check_pollers':
/build/xen/common/event_channel.c:1531: undefined reference to
`__bitop_bad_size'
riscv64-linux-gnu-ld: /build/xen/common/event_channel.c:1521: undefined
reference to `__bitop_bad_size'
riscv64-linux-gnu-ld: prelink.o: in function `evtchn_init':
/build/xen/common/event_channel.c:1541: undefined reference to
`__bitop_bad_size'
riscv64-linux-gnu-ld: prelink.o: in function `_read_lock':
/build/xen/./include/xen/rwlock.h:94: undefined reference to
`__bitop_bad_size'
riscv64-linux-gnu-ld:
prelink.o:/build/xen/./arch/riscv/include/asm/atomic.h:195: more
undefined references to `__bitop_bad_size' follow
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `__bitop_bad_size'
isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:15: xen-syms] Error 1

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 03 19:07:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 19:07:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716642.1118764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yFw-0002tQ-4G; Fri, 03 May 2024 19:07:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716642.1118764; Fri, 03 May 2024 19:07:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yFw-0002tJ-1m; Fri, 03 May 2024 19:07:40 +0000
Received: by outflank-mailman (input) for mailman id 716642;
 Fri, 03 May 2024 19:07:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s2yFv-0002tD-DK
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 19:07:39 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 66b52c7a-0980-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 21:07:36 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D350361E0C;
 Fri,  3 May 2024 19:07:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7838C116B1;
 Fri,  3 May 2024 19:07:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66b52c7a-0980-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714763254;
	bh=iiNH2csW/wExCYmnJzB9cYm+1os13nvklaTb7La4veo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=YGEPZRcqoHmlvqub8K5sX0+3i4n/J4Ib59/yDCnFQC9LPHdAosIOUxDvNcPbqVyF/
	 YsnrqDvADqAi+deuyDlJYIW0V2VLQh18GD7dRYFZESfp4ZB/vpzrHsnso4enfSY/BR
	 NgAZE814gcFPm0KxF+kpFx1N0Wyxy2mtU78X9R+tBkktis2UEFGLFKkm6G7N2Pm7mh
	 VV5BSJ7+5Kdvw2OY1P+TrgKQtIlw0LbLwXxZHY0wB7TOJb8wEIe7szrqiKIrEadAb2
	 VAQuggZrV+SsjAv1jf/VD8idWjdqSCkOODqXA4SjuU6ni+Bb/YiSkGYgBBDHbupEz/
	 8dx9i/OAHCNQQ==
Date: Fri, 3 May 2024 12:07:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?J=C3=BCrgen_Gro=C3=9F?= <jgross@suse.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
In-Reply-To: <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
References: <20240327152229.25847-1-jgross@suse.com> <20240327152229.25847-9-jgross@suse.com> <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com> <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com> <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com> <146fbd43-d8b8-4d73-8650-c60024498324@xen.org> <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop> <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1617437709-1714763254=:1151289"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1617437709-1714763254=:1151289
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 3 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 02/05/2024 19:13, Stefano Stabellini wrote:
> > On Mon, 29 Apr 2024, Julien Grall wrote:
> > > Hi Juergen,
> > > 
> > > On 29/04/2024 12:28, Jürgen Groß wrote:
> > > > On 29.04.24 13:04, Julien Grall wrote:
> > > > > Hi Juergen,
> > > > > 
> > > > > Sorry for the late reply.
> > > > > 
> > > > > On 29/04/2024 11:33, Juergen Gross wrote:
> > > > > > On 08.04.24 09:10, Jan Beulich wrote:
> > > > > > > On 27.03.2024 16:22, Juergen Gross wrote:
> > > > > > > > With lock handling now allowing up to 16384 cpus (spinlocks can
> > > > > > > > handle
> > > > > > > > 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed
> > > > > > > > limit
> > > > > > > > for
> > > > > > > > the number of cpus to be configured to 16383.
> > > > > > > > 
> > > > > > > > The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
> > > > > > > > QINVAL_MAX_ENTRY_NR required to be larger than 2 *
> > > > > > > > CONFIG_NR_CPUS.
> > > > > > > > 
> > > > > > > > Signed-off-by: Juergen Gross <jgross@suse.com>
> > > > > > > 
> > > > > > > Acked-by: Jan Beulich <jbeulich@suse.com>
> > > > > > > 
> > > > > > > I'd prefer this to also gain an Arm ack, though.
> > > > > > 
> > > > > > Any comment from Arm side?
> > > > > 
> > > > > Can you clarify what the new limits mean in term of (security)
> > > > > support?
> > > > > Are we now claiming that Xen will work perfectly fine on platforms
> > > > > with up
> > > > > to 16383?
> > > > > 
> > > > > If so, I can't comment for x86, but for Arm, I am doubtful that it
> > > > > would
> > > > > work without any (at least performance) issues. AFAIK, this is also an
> > > > > untested configuration. In fact I would be surprised if Xen on Arm was
> > > > > tested with more than a couple of hundreds cores (AFAICT the Ampere
> > > > > CPUs
> > > > > has 192 CPUs).
> > > > 
> > > > I think we should add a security support limit for the number of
> > > > physical
> > > > cpus similar to the memory support limit we already have in place.
> > > > 
> > > > For x86 I'd suggest 4096 cpus for security support (basically the limit
> > > > we
> > > > have with this patch), but I'm open for other suggestions, too.
> > > > 
> > > > I have no idea about any sensible limits for Arm32/Arm64.
> > > 
> > > I am not entirely. Bertrand, Michal, Stefano, should we use 192 (the
> > > number of
> > > CPUs from Ampere)?
> > 
> > I am OK with that. If we want to be a bit more future proof we could say
> > 256 or 512.
> 
> Sorry, I don't follow your argument. A limit can be raised at time point in
> the future. The question is more whether we are confident that Xen on Arm will
> run well if a user has a platform with 256/512 pCPUs.
> 
> So are you saying that from Xen point of view, you are expecting no difference
> between 256 and 512. And therefore you would be happy if to backport patches
> if someone find differences (or even security issues) when using > 256 pCPUs?

It is difficult to be sure about anything that it is not regularly
tested. I am pretty sure someone in the community got Xen running on an
Ampere, so like you said 192 is a good number. However, that is not
regularly tested, so we don't have any regression checks in gitlab-ci or
OSSTest for it.

One approach would be to only support things regularly tested either by
OSSTest, Gitlab-ci, or also Xen community members. I am not sure what
would be the highest number with this way of thinking but likely no
more than 192, probably less. I don't know the CPU core count of the
biggest ARM machine in OSSTest.

Another approach is to support a "sensible" number: not something tested
but something we believe it should work. No regular testing. (In safety,
they only believe in things that are actually tested, so this would not
be OK. But this is security, not safety, just FYI.) With this approach,
we could round up the number to a limit we think it won't break. If 192
works, 256/512 should work? I don't know but couldn't think of something
that would break going from 192 to 256.

It depends on how strict we want to be on testing requirements. I am not
sure what approach was taken by x86 so far. I am OK either way.
--8323329-1617437709-1714763254=:1151289--


From xen-devel-bounces@lists.xenproject.org Fri May 03 19:15:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 19:15:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716646.1118776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yN6-0004d1-T5; Fri, 03 May 2024 19:15:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716646.1118776; Fri, 03 May 2024 19:15:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yN6-0004cu-OR; Fri, 03 May 2024 19:15:04 +0000
Received: by outflank-mailman (input) for mailman id 716646;
 Fri, 03 May 2024 19:15:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yvxW=MG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2yN5-0004co-OP
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 19:15:03 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 706117e0-0981-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 21:15:01 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-436ee76c3b8so71516191cf.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 12:15:01 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 bz6-20020a05622a1e8600b00439c1419553sm1866666qtb.44.2024.05.03.12.14.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 12:15:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 706117e0-0981-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714763701; x=1715368501; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Ic5ZcpGxZalHaogiQ0LRYBQHMovcd2GjwOkEN80fwKA=;
        b=feoKyDJ611WhmQZgPPJRjZqoB352LMp/s73qDIxSeo17cFvVJqCx1z3iv/Loobp9Ui
         LN2DknHiHKAKTLrfS4lS1i+rnzMhjWSiFo5uduJTooRJQ9upXSwdQHAMWocnuwGx+v1p
         6R0ppRB7JLpSj8gibY7eV/BTOu6ez43suNGDo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714763701; x=1715368501;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Ic5ZcpGxZalHaogiQ0LRYBQHMovcd2GjwOkEN80fwKA=;
        b=ItqfScp2Cu/OCcACPUF5ldlGOEbHRkx4fTm/MVHePhZqbgLe4JBNEr/+6MJ9YhQWmN
         GCRx/XxJksY5yplxFaozHejdn89yL4RMKwdm4ogsTLE8SnpdKb9s9yy5ciejJOlndmRS
         6iuNnyIMZrs7DeOh0ZiMjzl+qrOMMLvfVc453czMGLTQGi+QAUXlS+S1L+3tsXBhwGy2
         TgKQJYJux4t9cd4hdt7vY76c/CP0dnE94rpBAjzOdZIJ7gBJohGDQ+4dIkbGPbv0xY8Z
         18Y5/ykeV2RWCETFegW5gU4RcLdIIaSUqYyoA2dIpUSQic3lGgBRpzKR0o1/+pPPeNSy
         uRSA==
X-Forwarded-Encrypted: i=1; AJvYcCWpBQttckPEgNjgV3GG0ZHowiZwzFJBXoEAaDDCH3WQfvWTeQSGl4oULtnG6s92ATNQHp6GDNHW5+esM9WxKcUv+XXtt6fUTorJ68Y7b1Q=
X-Gm-Message-State: AOJu0Ywln+kQ2V4qKDCdfKWqHkU5XAbf3yB0QJJ2dk9+ukCdMZfJcERO
	IRO5HX8rZp22K7z7bZ8ILEG9sKAdsOk+6rxWrFWjsZjEwcKSE/CB/x4Otf4AGOE=
X-Google-Smtp-Source: AGHT+IEyElJeLt9WgWFSq1uv6Mv0MWu+jbBBRuyjd9jqcPrvXW3Ebk7xO4OYgawjwyM1G5F6ZRDWtA==
X-Received: by 2002:a05:622a:308:b0:439:d05b:764c with SMTP id q8-20020a05622a030800b00439d05b764cmr3572211qtw.46.1714763700543;
        Fri, 03 May 2024 12:15:00 -0700 (PDT)
Message-ID: <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
Date: Fri, 3 May 2024 20:14:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
To: Federico Serafini <federico.serafini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <cover.1714401209.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/04/2024 4:21 pm, Federico Serafini wrote:
> Patch 1/3 does some preparation work.
>
> Patch 2/3, as the title says, removes allow_failure = true for triggered
> analyses.
>
> Patch 3/3 makes explicit that initally no files are tagged as adopted, this
> is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of
blocked rather than passed.

If I manually start the Eclair jobs, then eventually the pipeline gets
to Passed.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 03 19:44:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 19:44:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716650.1118784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ypL-0000kW-UH; Fri, 03 May 2024 19:44:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716650.1118784; Fri, 03 May 2024 19:44:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2ypL-0000kP-Rn; Fri, 03 May 2024 19:44:15 +0000
Received: by outflank-mailman (input) for mailman id 716650;
 Fri, 03 May 2024 19:44:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2ypK-0000kJ-8i
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 19:44:14 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 845f1ee5-0985-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 21:44:13 +0200 (CEST)
Received: from [192.168.1.20] (host-79-36-52-167.retail.telecomitalia.it
 [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 3783F4EE0738;
 Fri,  3 May 2024 21:44:12 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 845f1ee5-0985-11ef-909c-e314d9c70b13
Message-ID: <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
Date: Fri, 3 May 2024 21:44:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
 <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG
In-Reply-To: <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 03/05/24 21:14, Andrew Cooper wrote:
> On 29/04/2024 4:21 pm, Federico Serafini wrote:
>> Patch 1/3 does some preparation work.
>>
>> Patch 2/3, as the title says, removes allow_failure = true for triggered
>> analyses.
>>
>> Patch 3/3 makes explicit that initally no files are tagged as adopted, this
>> is needed by the scheduled analysis.
> 
> I'm afraid that something in this series is broken.
> 
> Since these patches went in, all pipelines are now getting a status of
> blocked rather than passed.
> 
> If I manually start the Eclair jobs, then eventually the pipeline gets
> to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew
and everything seems ok.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 03 19:46:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 19:46:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716652.1118795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yrE-0001I1-8Q; Fri, 03 May 2024 19:46:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716652.1118795; Fri, 03 May 2024 19:46:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2yrE-0001Hu-5A; Fri, 03 May 2024 19:46:12 +0000
Received: by outflank-mailman (input) for mailman id 716652;
 Fri, 03 May 2024 19:46:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yvxW=MG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2yrC-0001Hk-Bq
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 19:46:10 +0000
Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com
 [2607:f8b0:4864:20::832])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c7d06ca2-0985-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 21:46:06 +0200 (CEST)
Received: by mail-qt1-x832.google.com with SMTP id
 d75a77b69052e-439dfa27003so54026401cf.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 12:46:06 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 hx1-20020a05622a668100b004364d940d3dsm1851441qtb.96.2024.05.03.12.46.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 12:46:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7d06ca2-0985-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714765565; x=1715370365; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=6l2ocCHrTmlkYtAUM2lYOYHGUB5zcXSqlU1nXMoXWA4=;
        b=FrWWbzJcTrTw47Ree3/PJUS4ndA3A9AUU/WHbNlAQxx2sQPZ52iP0Uc2p6ohaGl/oX
         f0YxMUtDQLMCr5Niulf4LYEmkNm8hNZo52NNEWILsIavz7p7vNWuqxwoEtsJ8WkWd7z9
         n/mwcexQJTOd5otu31QDzCmEXSwzxZFfmbipQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714765565; x=1715370365;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6l2ocCHrTmlkYtAUM2lYOYHGUB5zcXSqlU1nXMoXWA4=;
        b=p85ORd0aC+B4Zuhe3Ry0ANRBaGk0wjUqMvn3qVuSiLT9Ng6tgGXzcNtf/eY5pxteXS
         QLsVE7JI0nqwy/r3Uw3foEjeTxVPDZFOFOQxGbVD0KJsKdZAv3+tHRoXfcXWsQ89YmsS
         qChMN+JVeiYQbXHmiF00KxD7q02JRk7XVUYco6CBtWzSPQJGya4hAM6432DLqBWyL9zg
         ZVGmBT/wMED4JnwV0hUbhzbMOhKswU5dY9+hIDlWkkes4e9DwelhvjVKrISQZh9STOBX
         1cFLCLwqKgHGjnkBhJUdZASfm8zGGvWTs6bePtGIRjOLKPKoyzcPLHRnLZhbNDSJtXmq
         1XAg==
X-Forwarded-Encrypted: i=1; AJvYcCXw7ZWrKS08X7n+LfKi1D+bNABw5cO93bpzcpO9p7oXWIO7vAnNrxJkWT4bMcQAwPMJySLGwE8tV6HE2S/RdOgB6Z4CCxwjjR8y28Elcag=
X-Gm-Message-State: AOJu0YyVHLXXAM5vZl0qxpkp3N5k6BFr4RcjBCGOzL3GwnS9tKfB9Ckd
	bv/SmA4d4YVOYlzbzi5zhpj9SAuA6vD2Y11Kle21UJsmhbqu5QjTryMQb+E40p5haEl/2JycCGj
	O
X-Google-Smtp-Source: AGHT+IGv5rlxXhEDz89Xs/z7iANzLk4L1afdzsPkIw55DdB7LBdnF/bp52Bk+6DFLXB5WlU33HVxAA==
X-Received: by 2002:a05:622a:1ba7:b0:43c:562f:e07b with SMTP id bp39-20020a05622a1ba700b0043c562fe07bmr3743190qtb.7.1714765565433;
        Fri, 03 May 2024 12:46:05 -0700 (PDT)
Message-ID: <e4468587-318a-44f2-ba66-9a9c5e504052@citrix.com>
Date: Fri, 3 May 2024 20:46:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
To: Federico Serafini <federico.serafini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
 <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
 <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03/05/2024 8:44 pm, Federico Serafini wrote:
> On 03/05/24 21:14, Andrew Cooper wrote:
>> On 29/04/2024 4:21 pm, Federico Serafini wrote:
>>> Patch 1/3 does some preparation work.
>>>
>>> Patch 2/3, as the title says, removes allow_failure = true for
>>> triggered
>>> analyses.
>>>
>>> Patch 3/3 makes explicit that initally no files are tagged as
>>> adopted, this
>>> is needed by the scheduled analysis.
>>
>> I'm afraid that something in this series is broken.
>>
>> Since these patches went in, all pipelines are now getting a status of
>> blocked rather than passed.
>>
>> If I manually start the Eclair jobs, then eventually the pipeline gets
>> to Passed.
>
> Can you provide us a link to those failures?
> I am looking at gitlab xen-project/xen and xen-project/patchew
> and everything seems ok.
>

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
is the first one I noticed as blocked, and I manually ran.  That ended
up working.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
is still in the blocked state.  The build-each-commit failure is unrelated.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 03 20:02:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 20:02:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716658.1118805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2z6T-0004Jv-M0; Fri, 03 May 2024 20:01:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716658.1118805; Fri, 03 May 2024 20:01:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2z6T-0004Jo-J4; Fri, 03 May 2024 20:01:57 +0000
Received: by outflank-mailman (input) for mailman id 716658;
 Fri, 03 May 2024 20:01:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IauD=MG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1s2z6R-0004Ji-Fo
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 20:01:55 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fcf84a8b-0987-11ef-909c-e314d9c70b13;
 Fri, 03 May 2024 22:01:54 +0200 (CEST)
Received: from [192.168.1.20] (host-79-36-52-167.retail.telecomitalia.it
 [79.36.52.167])
 by support.bugseng.com (Postfix) with ESMTPSA id 602934EE0738;
 Fri,  3 May 2024 22:01:53 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcf84a8b-0987-11ef-909c-e314d9c70b13
Message-ID: <518e455a-4233-498c-b7b1-afa485508543@bugseng.com>
Date: Fri, 3 May 2024 22:01:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
 <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
 <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
 <e4468587-318a-44f2-ba66-9a9c5e504052@citrix.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG
In-Reply-To: <e4468587-318a-44f2-ba66-9a9c5e504052@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 03/05/24 21:46, Andrew Cooper wrote:
> On 03/05/2024 8:44 pm, Federico Serafini wrote:
>> On 03/05/24 21:14, Andrew Cooper wrote:
>>> On 29/04/2024 4:21 pm, Federico Serafini wrote:
>>>> Patch 1/3 does some preparation work.
>>>>
>>>> Patch 2/3, as the title says, removes allow_failure = true for
>>>> triggered
>>>> analyses.
>>>>
>>>> Patch 3/3 makes explicit that initally no files are tagged as
>>>> adopted, this
>>>> is needed by the scheduled analysis.
>>>
>>> I'm afraid that something in this series is broken.
>>>
>>> Since these patches went in, all pipelines are now getting a status of
>>> blocked rather than passed.
>>>
>>> If I manually start the Eclair jobs, then eventually the pipeline gets
>>> to Passed.
>>
>> Can you provide us a link to those failures?
>> I am looking at gitlab xen-project/xen and xen-project/patchew
>> and everything seems ok.
>>
> 
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
> is the first one I noticed as blocked, and I manually ran.  That ended
> up working.
> 
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
> is still in the blocked state.  The build-each-commit failure is unrelated.

This is intentional and was introduced by
commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6

The ECLAIR analysis under people/* need to be activated
manually.

Is this causing some problems to the CI?

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 03 20:06:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 20:06:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716662.1118815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2zB7-0005Dz-63; Fri, 03 May 2024 20:06:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716662.1118815; Fri, 03 May 2024 20:06:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2zB7-0005Ds-3O; Fri, 03 May 2024 20:06:45 +0000
Received: by outflank-mailman (input) for mailman id 716662;
 Fri, 03 May 2024 20:06:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yvxW=MG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s2zB6-0005Dm-IO
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 20:06:44 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a884ddfb-0988-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 22:06:42 +0200 (CEST)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-79100a90868so85985a.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 13:06:42 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 mi8-20020a056214558800b006a0f4921e8esm1473716qvb.119.2024.05.03.13.06.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 13:06:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a884ddfb-0988-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714766801; x=1715371601; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=SvrvTcBjNiOYqdsvXGCgHprTsvcyK02R/zm/8t8sx5Y=;
        b=YBXxaPv7HpDFlExiEnLNk7pHdsfrRrob8ZGe76sJxbUJC4g64mgBK5oOM4u26iT5l6
         pQKkJmS2dUGYCVPdSXMWAaG+8jkBzLe3oaoP1XNk9zWZJhlMjm+8pCeOKcD7ojWs9QP+
         zORBs20abG17UJnAL0EV49qh9tvSXeT1e/YIY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714766801; x=1715371601;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SvrvTcBjNiOYqdsvXGCgHprTsvcyK02R/zm/8t8sx5Y=;
        b=FjI657dxr+/MJT9YAqi8Vduyp00M7q1/rGZ39IwW/nIXkdtc3ubXwwDiEIlyvF57i7
         rPytDiN0MSYqBvgxy4ZiY4KRkSoeaEJBk4RcRhB8XRuxqbjnhXOBSkE/YGZ9zxcq9lRv
         UjSPgrCdQSVpJ6fDQxy1A27y94iQQTn6idxtcNn+A7KPz9QKVE2yvlWtifrvI7S7VtBc
         1FV07vifVyXBxog4Etae5AUyYWQNUNw4gHFlTwEd2spVMaIeOnNsz7/BO/E+SejagHjz
         +q9GGX6P9pVZ3pWM1LKI+2G8/Fbi5gJI4xaYz1n8TmvTmyE9E1s+F6pEMDut5F2e3/Dg
         sv4w==
X-Forwarded-Encrypted: i=1; AJvYcCVgG2RsiupWDxuM/u66KO49CW8ERxy1Vl3HZiM7IGRTSJCT/p7s1Dz1pRddKJOKO0oQg4w54YxP73WaRKdemPRr0EY9bWN7kjnA/aYNTMg=
X-Gm-Message-State: AOJu0YyhtAYNxZc694Y3ntkksGWv2WJug/yKc+pMD0wDTMwyAngsE/JY
	/lCK7BcUcAPBHTkxne/fVP+HQf4JJYYqlx0GIU+X0GBq9m0TmX+cpZnH4cANU7AOYE/Suv0f1zy
	q
X-Google-Smtp-Source: AGHT+IEQ5QoCl109htOi9NZWCTIaoULc38q7qxYXiUlqUOd71bDp9bz3BGJ79y3lrWutgcnJizmmgQ==
X-Received: by 2002:a05:6214:21c4:b0:6a0:cca2:6513 with SMTP id d4-20020a05621421c400b006a0cca26513mr4478794qvh.46.1714766801365;
        Fri, 03 May 2024 13:06:41 -0700 (PDT)
Message-ID: <b2857b11-9ae6-4768-bd0d-8c93b9a3d604@citrix.com>
Date: Fri, 3 May 2024 21:06:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
To: Federico Serafini <federico.serafini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
 <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
 <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
 <e4468587-318a-44f2-ba66-9a9c5e504052@citrix.com>
 <518e455a-4233-498c-b7b1-afa485508543@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <518e455a-4233-498c-b7b1-afa485508543@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03/05/2024 9:01 pm, Federico Serafini wrote:
> On 03/05/24 21:46, Andrew Cooper wrote:
>> On 03/05/2024 8:44 pm, Federico Serafini wrote:
>>> On 03/05/24 21:14, Andrew Cooper wrote:
>>>> On 29/04/2024 4:21 pm, Federico Serafini wrote:
>>>>> Patch 1/3 does some preparation work.
>>>>>
>>>>> Patch 2/3, as the title says, removes allow_failure = true for
>>>>> triggered
>>>>> analyses.
>>>>>
>>>>> Patch 3/3 makes explicit that initally no files are tagged as
>>>>> adopted, this
>>>>> is needed by the scheduled analysis.
>>>>
>>>> I'm afraid that something in this series is broken.
>>>>
>>>> Since these patches went in, all pipelines are now getting a status of
>>>> blocked rather than passed.
>>>>
>>>> If I manually start the Eclair jobs, then eventually the pipeline gets
>>>> to Passed.
>>>
>>> Can you provide us a link to those failures?
>>> I am looking at gitlab xen-project/xen and xen-project/patchew
>>> and everything seems ok.
>>>
>>
>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
>> is the first one I noticed as blocked, and I manually ran.  That ended
>> up working.
>>
>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
>> is still in the blocked state.  The build-each-commit failure is
>> unrelated.
>
> This is intentional and was introduced by
> commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6
>
> The ECLAIR analysis under people/* need to be activated
> manually.

Yes.  I know, and that matches the behaviour I saw.

>
> Is this causing some problems to the CI?
>

Yes.

See https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines

Prior to this series, the manual actions were not used but the pipeline
was overall in the Passed state.  Specifically, they ended up being skipped.

After this series, the manual actions are now blocking the pipeline, not
letting it complete, and not marking it as passed.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 03 20:26:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 20:26:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716669.1118825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2zUB-00009s-Oj; Fri, 03 May 2024 20:26:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716669.1118825; Fri, 03 May 2024 20:26:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s2zUB-00009l-Lr; Fri, 03 May 2024 20:26:27 +0000
Received: by outflank-mailman (input) for mailman id 716669;
 Fri, 03 May 2024 20:26:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2zUA-00009Y-6P; Fri, 03 May 2024 20:26:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2zUA-0004Zw-2Z; Fri, 03 May 2024 20:26:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s2zU9-0000ew-M4; Fri, 03 May 2024 20:26:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s2zU9-0000nF-La; Fri, 03 May 2024 20:26:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=kFtVpaAjZTjPnkJZkxEboQgOj4iGwEMr/VXbWg7g3NU=; b=xd0QHLdTMMSbOHK8gLv6B/umYh
	bMrv8x1mT3O4xPyPvFNpuBGbrII5ZsA02P/ZyRPVE1ziO24yPbPzV5mTki8NIuJRakooTX67VX/ou
	LusYFocodUHqS92y39dHzYvh91HYUsCtiuJtmLcGTgphdPuNvl9iVzTi4Wb4Hm4ab1IM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185912-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185912: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
X-Osstest-Versions-That:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 20:26:25 +0000

flight 185912 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185912/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb
baseline version:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b

Last test of basis   185895  2024-05-01 22:02:19 Z    1 days
Testing same since   185912  2024-05-03 17:02:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   feb9158a62..f95cd010cb  f95cd010cbf0914154a0c2775c979d9158b1a3cb -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 03 21:19:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 21:19:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716679.1118835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s30J1-0007bH-Ik; Fri, 03 May 2024 21:18:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716679.1118835; Fri, 03 May 2024 21:18:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s30J1-0007bA-Fd; Fri, 03 May 2024 21:18:59 +0000
Received: by outflank-mailman (input) for mailman id 716679;
 Fri, 03 May 2024 21:18:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s30J0-0007b0-E2; Fri, 03 May 2024 21:18:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s30J0-0005Ro-8x; Fri, 03 May 2024 21:18:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s30Iz-00023i-SL; Fri, 03 May 2024 21:18:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s30Iz-00015R-Rp; Fri, 03 May 2024 21:18:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5+hdTBmIy6+wX+vf12nY1ZDDxbPxv8DLWVK1isS+fhg=; b=ku6cak/DJKBiGVZTCfZt5RdRJ1
	Rkiwj0Fd4j6n5GGbDtneoS+z1YUXIicU1U2M6BCziQ+VZsgnzi3T89mpZajTy5kqYFhAqI2AQCynq
	89Y4XaVXTQxeL0ElOh04ipWI6/+K8fXJPlLynQQFpmjGdOR9dApTHcpmWRIouvo3Tc6U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185909-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.15-testing test] 185909: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.15-testing:test-amd64-amd64-qemuu-freebsd12-amd64:guest-start/freebsd.repeat:fail:heisenbug
    xen-4.15-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=543bbddf79f105682904c3b5a09c5a1692ae4b15
X-Osstest-Versions-That:
    xen=056500003eb44314cb90f2e3e7b2d405e86b5657
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 May 2024 21:18:57 +0000

flight 185909 xen-4.15-testing real [real]
flight 185913 xen-4.15-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185909/
http://logs.test-lab.xenproject.org/osstest/logs/185913/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 21 guest-start/freebsd.repeat fail pass in 185913-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185877
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185877
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185877
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185877
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185877
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185877
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  543bbddf79f105682904c3b5a09c5a1692ae4b15
baseline version:
 xen                  056500003eb44314cb90f2e3e7b2d405e86b5657

Last test of basis   185877  2024-04-30 10:01:57 Z    3 days
Testing same since   185909  2024-05-03 09:40:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   056500003e..543bbddf79  543bbddf79f105682904c3b5a09c5a1692ae4b15 -> stable-4.15


From xen-devel-bounces@lists.xenproject.org Fri May 03 21:44:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 21:44:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716688.1118848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s30hm-0003L7-MC; Fri, 03 May 2024 21:44:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716688.1118848; Fri, 03 May 2024 21:44:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s30hm-0003L0-JX; Fri, 03 May 2024 21:44:34 +0000
Received: by outflank-mailman (input) for mailman id 716688;
 Fri, 03 May 2024 21:44:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tYw2=MG=kernel.org=pr-tracker-bot@srs-se1.protection.inumbo.net>)
 id 1s30hl-0003Ku-SQ
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 21:44:33 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51085549-0996-11ef-b4bb-af5377834399;
 Fri, 03 May 2024 23:44:30 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 257E3CE1A4F;
 Fri,  3 May 2024 21:44:23 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPS id 5D86AC116B1;
 Fri,  3 May 2024 21:44:22 +0000 (UTC)
Received: from aws-us-west-2-korg-oddjob-1.ci.codeaurora.org
 (localhost.localdomain [127.0.0.1])
 by aws-us-west-2-korg-oddjob-1.ci.codeaurora.org (Postfix) with ESMTP id
 48585C4339F; Fri,  3 May 2024 21:44:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51085549-0996-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714772662;
	bh=PUzXMSbC+X5EcWRG/at4HlBAQNJNqZepjSdVbkYW9hw=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=gnDWUcIB+KvvqI0jQWF+yIL6sQYUyALL1OYAkDRfTTnTmrS+TXRgg9Lsi90PpdduQ
	 hnoPjYtg/ocLBNWcyynP0kHKEW68WX0ecx/4M/PaWOW6CQQpOPgkSy9suVYjIvSHr/
	 r2P8NZd6hGs1ll4kqvYNlh9RmySa/7rxX6WgoIoZ6Zvia7yTU07usXzCF+IABsqIEU
	 aIM7o6z+/Ehv++2CpOG3o9J+0NGavF2t1w0AuAGMlFc6MCWR6WytKq0Pq//qeejmOc
	 kP49CpU40vKB9pjpI1Tg7erW9ugdFkDblaWxgQlRnRXWZBRC2m4aGYFpZKnFpTX307
	 RJYSxYZVS2o0Q==
Subject: Re: [GIT PULL] xen: branch for v6.9-rc7
From: pr-tracker-bot@kernel.org
In-Reply-To: <20240503122028.16437-1-jgross@suse.com>
References: <20240503122028.16437-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20240503122028.16437-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.9a-rc7-tag
X-PR-Tracked-Commit-Id: 802600ebdf23371b893a51a4ad046213f112ea3b
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: ddb4c3f25b7b95df3d6932db0b379d768a6ebdf7
Message-Id: <171477266229.28875.12276129093596082835.pr-tracker-bot@kernel.org>
Date: Fri, 03 May 2024 21:44:22 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, sstabellini@kernel.org

The pull request you sent on Fri,  3 May 2024 14:20:28 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.9a-rc7-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/ddb4c3f25b7b95df3d6932db0b379d768a6ebdf7

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:08:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:08:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716697.1118858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s320i-00054p-Ca; Fri, 03 May 2024 23:08:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716697.1118858; Fri, 03 May 2024 23:08:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s320i-00054i-9p; Fri, 03 May 2024 23:08:12 +0000
Received: by outflank-mailman (input) for mailman id 716697;
 Fri, 03 May 2024 23:08:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s320h-00054c-Fe
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:08:11 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 019ff209-09a2-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 01:08:09 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 72CF061A7C;
 Fri,  3 May 2024 23:08:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B469C116B1;
 Fri,  3 May 2024 23:08:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 019ff209-09a2-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714777688;
	bh=aze18fnjaf5ygacC22/M/ZR7vPAv6YyCbckTPisnxSA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=c4F/gscEP0ZM9sN08SaDPDor6jc7WcfqIde2jKJwa09qZbirHV2CC0+fgGBMlxZMz
	 rS8VcVoT4hzbjvUv7Gv6EpmcutEknyccW+3uemZ7KHu144vIpCADBuXOsLCVaPa7jk
	 77Tz89wZ+6oMh6eORek4a3Mt4bYWN1rF9C2s8vd1qkMuyf3rHGgoTagnOGYsEvg6EZ
	 nY66Wlj+yMLRXZRxOHLqXVEpC7MQMsOz4R/YDOMzSgHAxVmLORLFHbYzbbJDoSpFJD
	 EneWced98KLQyQQBjGvzgS2QtTI+MnVDiVVz1z6X5n/D+PgGs/JJhPqWdM8SgiMXI9
	 t/0fD2zA2El/w==
Date: Fri, 3 May 2024 16:08:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH] automation/eclair: hide reports coming from adopted
 code in scheduled analysis
In-Reply-To: <65364e87637d342b6b6ca5bd5ca256828ecb7a55.1714741931.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405031607540.1151289@ubuntu-linux-20-04-desktop>
References: <65364e87637d342b6b6ca5bd5ca256828ecb7a55.1714741931.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 May 2024, Federico Serafini wrote:
> To improve clarity and ease of navigation do not show reports related
> to adopted code in the scheduled analysis.
> Configuration options are commented out because they may be useful
> in the future.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  .../eclair_analysis/ECLAIR/analysis.ecl       | 24 +++++++++++--------
>  1 file changed, 14 insertions(+), 10 deletions(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index 66ed7f952c..67be38f03c 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -4,11 +4,11 @@
>  
>  setq(data_dir,getenv("ECLAIR_DATA_DIR"))
>  setq(analysis_kind,getenv("ANALYSIS_KIND"))
> -setq(scheduled_analysis,nil)
> +# setq(scheduled_analysis,nil)
>  
> -strings_map("scheduled-analysis",500,"","^.*scheduled$",0,setq(scheduled_analysis,t))
> -strings_map("scheduled-analysis",500,"","^.*$",0)
> -map_strings("scheduled-analysis",analysis_kind)
> +# strings_map("scheduled-analysis",500,"","^.*scheduled$",0,setq(scheduled_analysis,t))
> +# strings_map("scheduled-analysis",500,"","^.*$",0)
> +# map_strings("scheduled-analysis",analysis_kind)
>  
>  -verbose
>  
> @@ -25,12 +25,16 @@ map_strings("scheduled-analysis",analysis_kind)
>  -doc="Initially, there are no files tagged as adopted."
>  -file_tag+={adopted,"none()"}
>  
> -if(not(scheduled_analysis),
> -    eval_file("adopted.ecl")
> -)
> -if(not(scheduled_analysis),
> -    eval_file("out_of_scope.ecl")
> -)
> +# if(not(scheduled_analysis),
> +#     eval_file("adopted.ecl")
> +# )
> +# if(not(scheduled_analysis),
> +#     eval_file("out_of_scope.ecl")
> +# )
> +
> +-eval_file=adopted.ecl
> +-eval_file=out_of_scope.ecl
> +
>  -eval_file=deviations.ecl
>  -eval_file=call_properties.ecl
>  -eval_file=tagging.ecl
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:08:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:08:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716698.1118868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3217-0005Ph-JY; Fri, 03 May 2024 23:08:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716698.1118868; Fri, 03 May 2024 23:08:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3217-0005Pa-H1; Fri, 03 May 2024 23:08:37 +0000
Received: by outflank-mailman (input) for mailman id 716698;
 Fri, 03 May 2024 23:08:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s3216-0005PI-DA
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:08:36 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 102824fa-09a2-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 01:08:34 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D9D0761D7F;
 Fri,  3 May 2024 23:08:32 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62C67C116B1;
 Fri,  3 May 2024 23:08:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 102824fa-09a2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714777712;
	bh=ucQuWwuCgUwlFtaL8XCybyxB/OxABFPKf8TbbSa01A8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=cPA7/IM7yOyF6GVpwA2KbPO3Q+rrJQwb2OM4Ts4PeHzs4v2Aso/MWj0WZoi2Fu9Yv
	 xIVpEyyctmQbgZEVwyNHUftqAG72nUnPrbrB4u7TTwZwaRslfC5khnLs26nNX9dSou
	 VNIjTkFCF3hSpb07fdFXpUqRSf52phFrktnAVw5yFsnqdVN4YtKOLGylKD5vxPrbyk
	 Z95m1ONp/G9L4jPADwdCjBOc4D4JAoqUBrzb1+x1DdnYHuxos5+sgmY+ImCz9nHdJY
	 VDBKyFc7aUsY0ln6N1D7hki7KqktpczxoSo4X0+hyNzqO2YzpWe6TpHcC1BiDQq8sR
	 KfwnAPhek0IdQ==
Date: Fri, 3 May 2024 16:08:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 1/2] docs/misra: add Terms & Definitions section to
 rules.rst
In-Reply-To: <05d9d4b4319f28d602b7366f2964c451a3a50ce3.1714727807.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405031608220.1151289@ubuntu-linux-20-04-desktop>
References: <cover.1714727807.git.federico.serafini@bugseng.com> <05d9d4b4319f28d602b7366f2964c451a3a50ce3.1714727807.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 May 2024, Federico Serafini wrote:
> Add a section for terms and definitions used by MISRA but expressed
> in terms of the C specification.
> 
> Add a definition of "switch clause" to the newly-introduced section.
> 
> Link the first use of the term "switch clause" in the document to its
> definition.
> 
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Jan you were not completely satisfied by the definition but I didn't find
> a better one.
> ---
>  docs/misra/rules.rst | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index b7b447e152..d3b70fdf04 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -489,8 +489,7 @@ maintainers if you want to suggest a change.
>  
>     * - `Rule 16.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_03.c>`_
>       - Required
> -     - An unconditional break statement shall terminate every
> -       switch-clause
> +     - An unconditional break statement shall terminate every switch-clause_
>       - In addition to break, also other unconditional flow control statements
>         such as continue, return, goto are allowed.
>  
> @@ -712,3 +711,14 @@ maintainers if you want to suggest a change.
>       - The value of a pointer to a FILE shall not be used after the associated
>         stream has been closed
>       -
> +
> +Terms & Definitions
> +-------------------
> +
> +.. _switch-clause:
> +
> +A *switch clause* can be defined as:
> +"the non-empty list of statements which follows a non-empty list of
> +case/default labels".
> +A formal definition is available within the amplification of MISRA C:2012
> +Rule 16.1.
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:10:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:10:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716702.1118879 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s322Z-0006yz-TY; Fri, 03 May 2024 23:10:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716702.1118879; Fri, 03 May 2024 23:10:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s322Z-0006ys-QS; Fri, 03 May 2024 23:10:07 +0000
Received: by outflank-mailman (input) for mailman id 716702;
 Fri, 03 May 2024 23:10:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s322Y-0006yl-TO
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:10:06 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 45a11edb-09a2-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 01:10:05 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id C60B6CE19F4;
 Fri,  3 May 2024 23:10:01 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5C634C116B1;
 Fri,  3 May 2024 23:09:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45a11edb-09a2-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714777801;
	bh=6PxtIfi8KoXxEHG86P/CF4wIiQfgD3UardnDwMZ6q9o=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hR+3LtXeZCaA7bBHIH+xChgx65hpOB/uq04427OidhSPVOC30DDTMe07jeiX15Yvr
	 5kPT9SHgLESxdknu8HsfFW7eRWcVOssN5PWAIhe/dCbBMw7mpbB+wSBqOoB3gwsgdJ
	 7Dvl4p7KK++KdbBiapUDWfRcqlSi58jx3A2/HhwToR94Zd49/MxbkPaRaMe8SxF20b
	 xM/ZKMYpKFXxgVJNRIRGogYbQmixrReJu1QqZnNZAxkDIpCZGLrvMDd/kYAbJMR3y+
	 UkOC8NVqBWaAmyZrKD9t3p1kVkt1zrO25QjGkrqTaHMlfjkpqwE6Js0cX6ma2g2bY0
	 qCkcZ3ojPh2kA==
Date: Fri, 3 May 2024 16:09:58 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>
Subject: Re: [XEN PATCH 2/2] automation/eclair: add deviation for Rule 16.4
In-Reply-To: <d4670fa5bd7e33a915d5642cb78eb2eb33988819.1714727807.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405031609400.1151289@ubuntu-linux-20-04-desktop>
References: <cover.1714727807.git.federico.serafini@bugseng.com> <d4670fa5bd7e33a915d5642cb78eb2eb33988819.1714727807.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 May 2024, Federico Serafini wrote:
> MISRA C:2012 Rule 16.4 states that "Every switch statement shall have a
> default label".
> Update ECLAIR configuration to take into account the deviations
> agreed during MISRA meetings.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/deviations.ecl |  8 ++++++++
>  docs/misra/deviations.rst                        | 13 +++++++++++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> index d21f112a9b..f09ad71acf 100644
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -384,6 +384,14 @@ explicit comment indicating the fallthrough intention is present."
>  -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
>  -doc_end
>  
> +-doc_begin="Switch statements having a controlling expression of enum type deliberately do not have a default case: gcc -Wall enables -Wswitch which warns (and breaks the build as we use -Werror) if one of the enum labels is missing from the switch."
> +-config=MC3R1.R16.4,reports+={deliberate,'any_area(kind(context)&&^.* has no `default.*$&&stmt(node(switch_stmt)&&child(cond,skip(__non_syntactic_paren_stmts,type(canonical(enum_underlying_type(any())))))))'}
> +-doc_end
> +
> +-doc_begin="A switch statement with a single switch clause and no default label may be used in place of an equivalent if statement if it is considered to improve readability."
> +-config=MC3R1.R16.4,switch_clauses+={deliberate,"switch(1)&&default(0)"}
> +-doc_end
> +
>  -doc_begin="A switch statement with a single switch clause and no default label may be used in place of an equivalent if statement if it is considered to improve readability."
>  -config=MC3R1.R16.6,switch_clauses+={deliberate, "default(0)"}
>  -doc_end
> diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
> index ed0c1e8ed0..39cc321a27 100644
> --- a/docs/misra/deviations.rst
> +++ b/docs/misra/deviations.rst
> @@ -334,6 +334,19 @@ Deviations related to MISRA C:2012 Rules:
>           - /\* Fallthrough \*/
>           - /\* Fallthrough. \*/
>  
> +   * - R16.4
> +     - Switch statements having a controlling expression of enum type
> +       deliberately do not have a default case: gcc -Wall enables -Wswitch
> +       which warns (and breaks the build as we use -Werror) if one of the enum
> +       labels is missing from the switch.
> +     - Tagged as `deliberate` for ECLAIR.
> +
> +   * - R16.4
> +     - A switch statement with a single switch clause and no default label may
> +       be used in place of an equivalent if statement if it is considered to
> +       improve readability.
> +     - Tagged as `deliberate` for ECLAIR.
> +
>     * - R16.6
>       - A switch statement with a single switch clause and no default label may
>         be used in place of an equivalent if statement if it is considered to
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:30:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:30:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716708.1118889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32M8-0002gy-FD; Fri, 03 May 2024 23:30:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716708.1118889; Fri, 03 May 2024 23:30:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32M8-0002gr-CX; Fri, 03 May 2024 23:30:20 +0000
Received: by outflank-mailman (input) for mailman id 716708;
 Fri, 03 May 2024 23:30:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s32M6-0002gS-8R
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:30:18 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1733652c-09a5-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 01:30:15 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 4492ACE19F4;
 Fri,  3 May 2024 23:30:09 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 186DFC116B1;
 Fri,  3 May 2024 23:30:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1733652c-09a5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714779008;
	bh=hpr3Bz1TvKG5VL7nQWHdkpSRBjm1bkfiDw1Na63ZTtg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=lhbaFcZhzHXu7ZQh6dyIlcTKloI6sLeKeL74JlHbzz9Me2HhjIE88IiG+P3OSXiHi
	 iqCSZde/56rxckcWwMMXWVP/7+If1qke1tdkitzUCtAUYM78xglyofb4MaIvpGqlx9
	 jRCmnWjLv+grjx7TjmbxYkpohgBKjlJwhtez8bcV4KqinbwVOD5tnV3rvbecJUoTwk
	 fcETO8gSH6Ae3xxZ0mDaq8pmcnMdYCGsy/huN36Inoo2rJMtJPwZlFYtzvjH8YQ7Og
	 7UOCUk4fMqTx44IHfWu48kVbzCsMtArBXVblR1N95TzCECjd9tylwHj+eY4+eehHpn
	 ShXF6B9/TB78Q==
Date: Fri, 3 May 2024 16:30:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 2/9] xen/arm64: smc: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-3-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031629450.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-3-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/smc.S | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/smc.S b/xen/arch/arm/arm64/smc.S
> index fc6b676e2e..68b05e8ddd 100644
> --- a/xen/arch/arm/arm64/smc.S
> +++ b/xen/arch/arm/arm64/smc.S
> @@ -19,7 +19,7 @@
>   *                          register_t a6, register_t a7,
>   *                          struct arm_smccc_res *res)
>   */
> -ENTRY(__arm_smccc_1_0_smc)
> +FUNC(__arm_smccc_1_0_smc)
>          smc     #0
>          ldr     x4, [sp]
>          cbz     x4, 1f          /* No need to store the result */
> @@ -27,12 +27,13 @@ ENTRY(__arm_smccc_1_0_smc)
>          stp     x2, x3, [x4, #SMCCC_RES_a2]
>  1:
>          ret
> +END(__arm_smccc_1_0_smc)
>  
>  /*
>   * void arm_smccc_1_2_smc(const struct arm_smccc_1_2_regs *args,
>   *                        struct arm_smccc_1_2_regs *res)
>   */
> -ENTRY(arm_smccc_1_2_smc)
> +FUNC(arm_smccc_1_2_smc)
>      /* Save `res` and free a GPR that won't be clobbered by SMC call */
>      stp     x1, x19, [sp, #-16]!
>  
> @@ -69,3 +70,4 @@ ENTRY(arm_smccc_1_2_smc)
>      /* Restore original x19 */
>      ldp     xzr, x19, [sp], #16
>      ret
> +END(arm_smccc_1_2_smc)
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:30:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:30:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716709.1118899 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32MH-0002xV-MH; Fri, 03 May 2024 23:30:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716709.1118899; Fri, 03 May 2024 23:30:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32MH-0002xO-JU; Fri, 03 May 2024 23:30:29 +0000
Received: by outflank-mailman (input) for mailman id 716709;
 Fri, 03 May 2024 23:30:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s32MG-0002gS-Uy
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:30:28 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1edf8d9d-09a5-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 01:30:27 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 9AA2E61DA8;
 Fri,  3 May 2024 23:30:25 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60E79C116B1;
 Fri,  3 May 2024 23:30:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1edf8d9d-09a5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714779025;
	bh=NcJXG97k3QRabzf6wKMUDp4/I8OXrPcxTtEtDPyCHDI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bPqUxNT8/2OrA3N6EtKkwZEXqfhNElyg4YlEGfMNnv72uHlbf+A2fXDPSyUcLuHhj
	 YARSSOoffL7NzszgXDcMnwDnR2NcvcPlqndnenpVvwmvdF4esxxsMAU3Jy0T64hEtV
	 6Jf8qSnH1SlV99LknZlssYlQKaws7rU5PGvuAByqeXhOxfyvSk8ut7M7x0Klg1cglS
	 yLQ7dR4DhHdwnYx4Mb23ieQg+uTm5LFzftga7lylR3SgCbgMLYpkbDd3Oq0EjC7FHQ
	 d55sxCmV/8rZTMzl2kZDjMeii+OL4W6X8dhNlRh042WdiU4dKntyiWG6UXA0zx3313
	 UD3hIuIHWO0vg==
Date: Fri, 3 May 2024 16:30:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 3/9] xen/arm64: sve: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-4-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031630120.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-4-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/sve-asm.S | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/sve-asm.S b/xen/arch/arm/arm64/sve-asm.S
> index 59dbefbbb2..2d8b895f07 100644
> --- a/xen/arch/arm/arm64/sve-asm.S
> +++ b/xen/arch/arm/arm64/sve-asm.S
> @@ -161,9 +161,10 @@
>  .endm
>  
>  /* Gets the current vector register size in bytes */
> -GLOBAL(sve_get_hw_vl)
> +FUNC(sve_get_hw_vl)
>      _sve_rdvl 0, 1
>      ret
> +END(sve_get_hw_vl)
>  
>  /*
>   * Save the SVE context
> @@ -172,9 +173,10 @@ GLOBAL(sve_get_hw_vl)
>   * x1 - pointer to buffer for P0-15
>   * x2 - Save FFR if non-zero
>   */
> -GLOBAL(sve_save_ctx)
> +FUNC(sve_save_ctx)
>      sve_save 0, 1, x2
>      ret
> +END(sve_save_ctx)
>  
>  /*
>   * Load the SVE context
> @@ -183,9 +185,10 @@ GLOBAL(sve_save_ctx)
>   * x1 - pointer to buffer for P0-15
>   * x2 - Restore FFR if non-zero
>   */
> -GLOBAL(sve_load_ctx)
> +FUNC(sve_load_ctx)
>      sve_load 0, 1, x2
>      ret
> +END(sve_load_ctx)
>  
>  /*
>   * Local variables:
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:56:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:56:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716718.1118908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32lL-0006vY-Ke; Fri, 03 May 2024 23:56:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716718.1118908; Fri, 03 May 2024 23:56:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32lL-0006vR-Hj; Fri, 03 May 2024 23:56:23 +0000
Received: by outflank-mailman (input) for mailman id 716718;
 Fri, 03 May 2024 23:56:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s32lK-0006vL-8P
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:56:22 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bcece092-09a8-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 01:56:20 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B31F960A70;
 Fri,  3 May 2024 23:56:19 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E45A4C116B1;
 Fri,  3 May 2024 23:56:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcece092-09a8-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714780579;
	bh=2kV/DeMp2zLh9dPDzzypZhC+DxMSHbXGzskQrR1Id3I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Bhf2UyecDta4OWmKAuCJdy1S/LR5WaspAnmN0Qem02vDAI3eU/RP1LybrbYx7K6jB
	 LJksISitV5NIGt3kHFjypeU5yOZItt1UHUSXK0z0DlsSm77sCFW1+YSJDfY+L2Oqli
	 ZXi+XYQY8eE18JESv86+tljJWwVeqp8liFPsb2VCzXnHEspFHp8Hx3ne+8nCoNN7Vs
	 QUgYgiXgOemiCq7mNeb6N6bJkRbAw+rw71V7YhebDZuwy6kY1XHPD7NWUEOdTqN+LN
	 we5EChdAzI8cK83WXgXagA1iuURikgXtbXvntWBK3RFKtc76noPE1eVo/bIR/lyqrg
	 3/7RUqLKYRTdw==
Date: Fri, 3 May 2024 16:56:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 4/9] xen/arm64: head: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-5-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031630280.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-5-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  xen/arch/arm/arm64/head.S | 50 ++++++++++++++++++++-------------------
>  1 file changed, 26 insertions(+), 24 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
> index fb297e9eb5..7acedb4f8f 100644
> --- a/xen/arch/arm/arm64/head.S
> +++ b/xen/arch/arm/arm64/head.S
> @@ -90,7 +90,7 @@
>   * 4K-aligned address.
>   */
>  
> -GLOBAL(start)
> +FUNC(start)
>          /*
>           * DO NOT MODIFY. Image header expected by Linux boot-loaders.
>           */
> @@ -102,6 +102,7 @@ efi_head:
>           */
>          add     x13, x18, #0x16
>          b       real_start           /* branch to kernel start */
> +END(start)
>          .quad   0                    /* Image load offset from start of RAM */
>          .quad   _end - start         /* Effective size of kernel image, little-endian */
>          .quad   __HEAD_FLAGS         /* Informative flags, little-endian */
> @@ -223,7 +224,7 @@ section_table:
>          .align  5
>  #endif /* CONFIG_ARM_EFI */
>  
> -real_start:
> +FUNC_LOCAL(real_start)
>          /* BSS should be zeroed when booting without EFI */
>          mov   x26, #0                /* x26 := skip_zero_bss */
>  
> @@ -263,9 +264,9 @@ primary_switched:
>          mov   x1, x21                /* x1 := paddr(FDT) */
>          ldr   x2, =start_xen
>          b     launch
> -ENDPROC(real_start)
> +END(real_start)
>  
> -GLOBAL(init_secondary)
> +FUNC(init_secondary)
>          msr   DAIFSet, 0xf           /* Disable all interrupts */
>  
>          /* Find out where we are */
> @@ -304,7 +305,7 @@ secondary_switched:
>          /* Jump to C world */
>          ldr   x2, =start_secondary
>          b     launch
> -ENDPROC(init_secondary)
> +END(init_secondary)
>  
>  /*
>   * Check if the CPU has been booted in Hypervisor mode.
> @@ -313,7 +314,7 @@ ENDPROC(init_secondary)
>   *
>   * Clobbers x0 - x5
>   */
> -check_cpu_mode:
> +FUNC_LOCAL(check_cpu_mode)
>          PRINT_ID("- Current EL ")
>          mrs   x5, CurrentEL
>          print_reg x5
> @@ -329,7 +330,7 @@ check_cpu_mode:
>          PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n")
>          PRINT_ID("- Please update the bootloader -\r\n")
>          b fail
> -ENDPROC(check_cpu_mode)
> +END(check_cpu_mode)
>  
>  /*
>   * Zero BSS
> @@ -339,7 +340,7 @@ ENDPROC(check_cpu_mode)
>   *
>   * Clobbers x0 - x3
>   */
> -zero_bss:
> +FUNC_LOCAL(zero_bss)
>          /* Zero BSS only when requested */
>          cbnz  x26, skip_bss
>  
> @@ -353,14 +354,14 @@ zero_bss:
>  
>  skip_bss:
>          ret
> -ENDPROC(zero_bss)
> +END(zero_bss)
>  
>  /*
>   * Initialize the processor for turning the MMU on.
>   *
>   * Clobbers x0 - x3
>   */
> -cpu_init:
> +FUNC_LOCAL(cpu_init)
>          PRINT_ID("- Initialize CPU -\r\n")
>  
>          /* Set up memory attribute type tables */
> @@ -399,7 +400,7 @@ cpu_init:
>           */
>          msr spsel, #1
>          ret
> -ENDPROC(cpu_init)
> +END(cpu_init)
>  
>  /*
>   * Setup the initial stack and jump to the C world
> @@ -411,7 +412,7 @@ ENDPROC(cpu_init)
>   *
>   * Clobbers x3
>   */
> -launch:
> +FUNC_LOCAL(launch)
>          ldr   x3, =init_data
>          add   x3, x3, #INITINFO_stack /* Find the boot-time stack */
>          ldr   x3, [x3]
> @@ -421,13 +422,13 @@ launch:
>  
>          /* Jump to C world */
>          br    x2
> -ENDPROC(launch)
> +END(launch)
>  
>  /* Fail-stop */
> -fail:   PRINT_ID("- Boot failed -\r\n")
> +FUNC_LOCAL(fail)   PRINT_ID("- Boot failed -\r\n")

Maybe we should move PRINT_ID to a newline?
I am not sure FUNC_LOCAL supports having a command on the same line.



>  1:      wfe
>          b     1b
> -ENDPROC(fail)
> +END(fail)
>  
>  #ifdef CONFIG_EARLY_PRINTK
>  /*
> @@ -438,14 +439,14 @@ ENDPROC(fail)
>   *
>   * Clobbers x0 - x1
>   */
> -init_uart:
> +FUNC_LOCAL(init_uart)
>          ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS
>  #ifdef CONFIG_EARLY_UART_INIT
>          early_uart_init x23, 0
>  #endif
>          PRINT("- UART enabled -\r\n")
>          ret
> -ENDPROC(init_uart)
> +END(init_uart)
>  
>  /*
>   * Print early debug messages.
> @@ -454,7 +455,7 @@ ENDPROC(init_uart)
>   * x23: Early UART base address
>   * Clobbers x0-x1
>   */
> -ENTRY(asm_puts)
> +FUNC(asm_puts)
>          early_uart_ready x23, 1
>          ldrb  w1, [x0], #1           /* Load next char */
>          cbz   w1, 1f                 /* Exit on nul */
> @@ -462,7 +463,7 @@ ENTRY(asm_puts)
>          b     asm_puts
>  1:
>          ret
> -ENDPROC(asm_puts)
> +END(asm_puts)
>  
>  /*
>   * Print a 64-bit number in hex.
> @@ -471,7 +472,7 @@ ENDPROC(asm_puts)
>   * x23: Early UART base address
>   * Clobbers x0-x3
>   */
> -ENTRY(asm_putn)
> +FUNC(asm_putn)
>          adr_l x1, hex
>          mov   x3, #16
>  1:
> @@ -484,7 +485,7 @@ ENTRY(asm_putn)
>          subs  x3, x3, #1
>          b.ne  1b
>          ret
> -ENDPROC(asm_putn)
> +END(asm_putn)
>  
>  RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
>  
> @@ -493,16 +494,17 @@ RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
>  /* This provides a C-API version of __lookup_processor_type
>   * TODO: For now, the implementation return NULL every time
>   */
> -ENTRY(lookup_processor_type)
> +FUNC(lookup_processor_type)
>          mov  x0, #0
>          ret
> +END(lookup_processor_type)
>  
>  #ifdef CONFIG_ARM_EFI
>  /*
>   *  Function to transition from EFI loader in C, to Xen entry point.
>   *  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>   */
> -ENTRY(efi_xen_start)
> +FUNC(efi_xen_start)
>          /*
>           * Preserve x0 (fdt pointer) across call to __flush_dcache_area,
>           * restore for entry into Xen.
> @@ -554,7 +556,7 @@ ENTRY(efi_xen_start)
>          mov   x26, #1               /* x26 := skip_zero_bss */
>  
>          b     real_start_efi
> -ENDPROC(efi_xen_start)
> +END(efi_xen_start)
>  
>  #endif /* CONFIG_ARM_EFI */
>  
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 03 23:56:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 May 2024 23:56:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716720.1118918 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32lp-0007SR-0H; Fri, 03 May 2024 23:56:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716720.1118918; Fri, 03 May 2024 23:56:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s32lo-0007SK-Ti; Fri, 03 May 2024 23:56:52 +0000
Received: by outflank-mailman (input) for mailman id 716720;
 Fri, 03 May 2024 23:56:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvfX=MG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s32lo-0006vL-1O
 for xen-devel@lists.xenproject.org; Fri, 03 May 2024 23:56:52 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce212e4f-09a8-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 01:56:51 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 28531CE1A16;
 Fri,  3 May 2024 23:56:46 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0287C116B1;
 Fri,  3 May 2024 23:56:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce212e4f-09a8-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714780605;
	bh=KC/SiMnjEbTCWCvbhfalffU+KxKur83OHaxBl2yilo8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JeZBjTr2uIL4cAj8QYFyxFkKQXaa63FfJr4OlzLjTR/tYP2Gu2dQnDL1u7GTcYWvM
	 AZhfReu7BncolOMFoVZhNlJP2u7FIlJuiSfgyagOqatSdo+jeWt85pmCpQ1V19Lg/P
	 hrAliurp8xlir/tbHhOAHNDBRaXKTqZ5m83/+Mhjj3HhFnG8+xCsQ02QRcHZPSavSB
	 On4dtNgzh1tFBkvABp91KOC8eU6OlUe4a0+Q6C5xiKdScn9Swwi7bjeWa4HIWiK1ot
	 +HZmyBeK4+ib7k7Tsqm2zkAanSmpy6hjDGrWzDNwR4bNyWRyueBvF8s4Z8Uj7Hg4io
	 Kx8/UWG3WDtcQ==
Date: Fri, 3 May 2024 16:56:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 5/9] xen/arm64: debug: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-6-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031656340.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-6-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/debug.S | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/debug.S b/xen/arch/arm/arm64/debug.S
> index 71cad9d762..c3d02c33d7 100644
> --- a/xen/arch/arm/arm64/debug.S
> +++ b/xen/arch/arm/arm64/debug.S
> @@ -27,17 +27,19 @@
>   * Print a character on the UART - this function is called by C
>   * x0: character to print
>   */
> -GLOBAL(early_putch)
> +FUNC(early_putch)
>          ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS
>          early_uart_ready x15, 1
>          early_uart_transmit x15, w0
>          ret
> +END(early_putch)
>  
>  /* Flush the UART - this function is called by C */
> -GLOBAL(early_flush)
> +FUNC(early_flush)
>          ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS  /* x15 := VA UART base address */
>          early_uart_ready x15, 1
>          ret
> +END(early_flush)
>  
>  /*
>   * Local variables:
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:15:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:15:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716726.1118930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s333Q-0002yZ-A4; Sat, 04 May 2024 00:15:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716726.1118930; Sat, 04 May 2024 00:15:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s333Q-0002yS-5p; Sat, 04 May 2024 00:15:04 +0000
Received: by outflank-mailman (input) for mailman id 716726;
 Sat, 04 May 2024 00:15:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=498C=MH=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s333O-0002yM-Bg
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 00:15:02 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 56a8bd51-09ab-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 02:14:59 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id CE3E1CE18D3;
 Sat,  4 May 2024 00:14:55 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9CCDBC4AF14;
 Sat,  4 May 2024 00:14:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56a8bd51-09ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714781695;
	bh=var572hoqFKe+f0Pqve4/c5c3qNtJLQh9coE/PP5jKU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Pumx1jMAzj8Ec6z7JrZw+thO2ivMdEYTHK3sFamcmFM0Ie5cQUNbXOnOeec//SOdI
	 yCWcaXRAabQNm+D3YnICYybDU18hVjarufI9KifATbtW7k/wpvbZsesEwTFK9PYsSj
	 7clNhCo/lNzC/PPbmNFQD3npB6aHXKczgDu9laDsh7sBdtxhi+LQt+4pfsLlGr20zW
	 dcpBbQVzLcMKkojvPsJaEVWpoy7LILjPg/L7mq8ueJWgN1CrBD/aAdtD0IH6xzFvq/
	 VfLgbVF5koCH+mfliNTzRoLwT8rfJFG0/OJWenWNUMbEJalhxRmu+p3ixZzeWM++T0
	 P+JtvTj/sLp1A==
Date: Fri, 3 May 2024 17:14:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-7-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-7-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
>  1 file changed, 12 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
> index 4e63825220..b16e4d1e29 100644
> --- a/xen/arch/arm/arm64/bpi.S
> +++ b/xen/arch/arm/arm64/bpi.S
> @@ -52,14 +52,15 @@
>   * micro-architectures in a system.
>   */
>      .align	11
> -ENTRY(__bp_harden_hyp_vecs_start)
> +FUNC(__bp_harden_hyp_vecs_start)
>      .rept 4
>      vectors hyp_traps_vector
>      .endr
> -ENTRY(__bp_harden_hyp_vecs_end)
> +GLOBAL(__bp_harden_hyp_vecs_end)
> +END(__bp_harden_hyp_vecs_start)

Shouldn't GLOBAL be changed to FUNC as well?


>  .macro mitigate_spectre_bhb_loop count
> -ENTRY(__mitigate_spectre_bhb_loop_start_\count)
> +FUNC(__mitigate_spectre_bhb_loop_start_\count)
>      stp     x0, x1, [sp, #-16]!
>      mov     x0, \count
>  .Lspectre_bhb_loop\@:
> @@ -68,11 +69,12 @@ ENTRY(__mitigate_spectre_bhb_loop_start_\count)
>      b.ne    .Lspectre_bhb_loop\@
>      sb
>      ldp     x0, x1, [sp], #16
> -ENTRY(__mitigate_spectre_bhb_loop_end_\count)
> +GLOBAL(__mitigate_spectre_bhb_loop_end_\count)

Also here?


> +END(__mitigate_spectre_bhb_loop_start_\count)
>  .endm
>  
>  .macro smccc_workaround num smcc_id
> -ENTRY(__smccc_workaround_smc_start_\num)
> +FUNC(__smccc_workaround_smc_start_\num)
>      sub     sp, sp, #(8 * 4)
>      stp     x0, x1, [sp, #(8 * 2)]
>      stp     x2, x3, [sp, #(8 * 0)]
> @@ -81,13 +83,15 @@ ENTRY(__smccc_workaround_smc_start_\num)
>      ldp     x2, x3, [sp, #(8 * 0)]
>      ldp     x0, x1, [sp, #(8 * 2)]
>      add     sp, sp, #(8 * 4)
> -ENTRY(__smccc_workaround_smc_end_\num)
> +GLOBAL(__smccc_workaround_smc_end_\num)

And here?


> +END(__smccc_workaround_smc_start_\num)
>  .endm
>  
> -ENTRY(__mitigate_spectre_bhb_clear_insn_start)
> +FUNC(__mitigate_spectre_bhb_clear_insn_start)
>      clearbhb
>      isb
> -ENTRY(__mitigate_spectre_bhb_clear_insn_end)
> +GLOBAL(__mitigate_spectre_bhb_clear_insn_end)

and here?


> +END(__mitigate_spectre_bhb_clear_insn_start)
>  
>  mitigate_spectre_bhb_loop 8
>  mitigate_spectre_bhb_loop 24
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:15:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:15:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716727.1118939 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s333o-0003Mh-He; Sat, 04 May 2024 00:15:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716727.1118939; Sat, 04 May 2024 00:15:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s333o-0003Ma-E8; Sat, 04 May 2024 00:15:28 +0000
Received: by outflank-mailman (input) for mailman id 716727;
 Sat, 04 May 2024 00:15:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=498C=MH=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s333n-0002yM-6D
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 00:15:27 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 670cf9d5-09ab-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 02:15:25 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id BEB9C60A70;
 Sat,  4 May 2024 00:15:23 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C25CAC116B1;
 Sat,  4 May 2024 00:15:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 670cf9d5-09ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714781723;
	bh=guBLwYmp0JPu0oz5oTLIWtqx2/I45mwLYCHyDSqcxjc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=BnJz/5VShPDrTQso7btV6ytg8hEGi9V+cS+bifxyD655jjgUETmfDbIXgdJDGyyGd
	 4tUAVyEprwBmU3PJjKQktP+prnY6qWHkakqgLb8yxhz1FJuCSfqK1hiPSfGPWXZGTw
	 5/i/kznuQNQi7R1kzCJ6myWce1G1kveqDWMaVp7XbjLYs5X1zGXtWAbhDh2Swbt4KD
	 U7SEWqoJuAGkA+M+oDxO1Lv3fM//PBp25aAzKKUQA47ytz43F8Vz7Dun5nPiC9ZW2T
	 UztKGBAcfO0ifcoETix6gihF4etCSFSrOlKdqrXqevHfUnH3H4V0vEUeCH4wLr1e1d
	 ZYZrZrEQrHaLg==
Date: Fri, 3 May 2024 17:15:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 7/9] xen/arm64: mmu/head: Add missing code symbol
 annotations
In-Reply-To: <20240501035448.964625-8-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031715090.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-8-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  xen/arch/arm/arm64/mmu/head.S | 28 ++++++++++++++--------------
>  1 file changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S
> index fa40b696dd..7788bb95e5 100644
> --- a/xen/arch/arm/arm64/mmu/head.S
> +++ b/xen/arch/arm/arm64/mmu/head.S
> @@ -138,7 +138,7 @@
>   *
>   * Clobbers x0 - x4
>   */
> -create_page_tables:
> +FUNC_LOCAL(create_page_tables)
>          /* Prepare the page-tables for mapping Xen */
>          ldr   x0, =XEN_VIRT_START
>          create_table_entry boot_pgtable, boot_first, x0, 0, x1, x2, x3
> @@ -260,7 +260,7 @@ virtphys_clash:
>          /* Identity map clashes with boot_third, which we cannot handle yet */
>          PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
>          b     fail
> -ENDPROC(create_page_tables)
> +END(create_page_tables)
>  
>  /*
>   * Turn on the Data Cache and the MMU. The function will return on the 1:1
> @@ -273,7 +273,7 @@ ENDPROC(create_page_tables)
>   *
>   * Clobbers x0 - x5
>   */
> -enable_mmu:
> +FUNC_LOCAL(enable_mmu)
>          mov   x4, x0
>          mov   x5, x1
>          PRINT_ID("- Turning on paging -\r\n")
> @@ -304,7 +304,7 @@ enable_mmu:
>          PRINT_ID("- Paging turned on -\r\n")
>  
>          ret
> -ENDPROC(enable_mmu)
> +END(enable_mmu)
>  
>  /*
>   * Enable mm (turn on the data cache and the MMU) for secondary CPUs.
> @@ -316,7 +316,7 @@ ENDPROC(enable_mmu)
>   *
>   * Clobbers x0 - x6
>   */
> -ENTRY(enable_secondary_cpu_mm)
> +FUNC(enable_secondary_cpu_mm)
>          mov   x6, lr
>  
>          load_paddr x0, init_ttbr
> @@ -328,7 +328,7 @@ ENTRY(enable_secondary_cpu_mm)
>  
>          /* Return to the virtual address requested by the caller. */
>          ret
> -ENDPROC(enable_secondary_cpu_mm)
> +END(enable_secondary_cpu_mm)
>  
>  /*
>   * Enable mm (turn on the data cache and the MMU) for the boot CPU.
> @@ -340,7 +340,7 @@ ENDPROC(enable_secondary_cpu_mm)
>   *
>   * Clobbers x0 - x6
>   */
> -ENTRY(enable_boot_cpu_mm)
> +FUNC(enable_boot_cpu_mm)
>          mov   x6, lr
>  
>          bl    create_page_tables
> @@ -365,7 +365,7 @@ ENTRY(enable_boot_cpu_mm)
>           * by the caller.
>           */
>          b     remove_identity_mapping
> -ENDPROC(enable_boot_cpu_mm)
> +END(enable_boot_cpu_mm)
>  
>  /*
>   * Remove the 1:1 map from the page-tables. It is not easy to keep track
> @@ -377,7 +377,7 @@ ENDPROC(enable_boot_cpu_mm)
>   *
>   * Clobbers x0 - x1
>   */
> -remove_identity_mapping:
> +FUNC_LOCAL(remove_identity_mapping)
>          /*
>           * Find the zeroeth slot used. Remove the entry from zeroeth
>           * table if the slot is not XEN_ZEROETH_SLOT.
> @@ -419,20 +419,20 @@ identity_mapping_removed:
>          flush_xen_tlb_local
>  
>          ret
> -ENDPROC(remove_identity_mapping)
> +END(remove_identity_mapping)
>  
>  /* Fail-stop */
> -fail:   PRINT("- Boot failed -\r\n")
> +FUNC_LOCAL(fail)   PRINT("- Boot failed -\r\n")

move PRINT to newline


>  1:      wfe
>          b     1b
> -ENDPROC(fail)
> +END(fail)
>  
>  /*
>   * Switch TTBR
>   *
>   * x0    ttbr
>   */
> -ENTRY(switch_ttbr_id)
> +FUNC(switch_ttbr_id)
>          /* 1) Ensure any previous read/write have completed */
>          dsb    ish
>          isb
> @@ -464,7 +464,7 @@ ENTRY(switch_ttbr_id)
>          isb
>  
>          ret
> -ENDPROC(switch_ttbr_id)
> +END(switch_ttbr_id)
>  
>  /*
>   * Local variables:
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:15:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:15:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716729.1118949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3345-0003po-ON; Sat, 04 May 2024 00:15:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716729.1118949; Sat, 04 May 2024 00:15:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3345-0003pf-Lj; Sat, 04 May 2024 00:15:45 +0000
Received: by outflank-mailman (input) for mailman id 716729;
 Sat, 04 May 2024 00:15:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=498C=MH=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s3344-0003oi-5i
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 00:15:44 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 70a12c86-09ab-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 02:15:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 6A578CE1A2D;
 Sat,  4 May 2024 00:15:39 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E57AC4AF19;
 Sat,  4 May 2024 00:15:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70a12c86-09ab-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714781738;
	bh=pSXP/4179EUBJj40KOXInEIVPWcP3gBPOrw7GTh+l8Q=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=I8/Hxr9HrXvsGkZYevCU1bm7QsJAQ9ZAtbPMLczt1iFyTUSDYC9bJA8KBQuindaAZ
	 yP5dAVauspM4LVZHJBOauhRqNo8TASGs1jG9ZhCzqCDMfcRVyo7Mo4wHpLfS0+OmeV
	 cGG1ySA9QQTM6+4PSLknnZgVDLQCbqSsJHpsjrN0BfwQkFbu4clVNNYwI8/pATyJH4
	 /Tr5uY5H+cN0PWLx2rF5T/BtUBvK9fVHZTFi8nKxtlUmLOqL/nJADIeZQHaQA+03Vd
	 B/r9iBw5ghIWkbbL3sG8qAqo1R2cIxuyEm1Sfz37A5YmuNQXtsss/IqdEuAxwvyyVK
	 MOddGHwNxROHA==
Date: Fri, 3 May 2024 17:15:36 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 8/9] xen/arm64: cache: Use the generic xen/linkage.h
 macros
In-Reply-To: <20240501035448.964625-9-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031715300.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-9-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/cache.S | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/cache.S b/xen/arch/arm/arm64/cache.S
> index 9a88a2b497..66ed85f735 100644
> --- a/xen/arch/arm/arm64/cache.S
> +++ b/xen/arch/arm/arm64/cache.S
> @@ -40,7 +40,7 @@
>   *	- kaddr   - kernel address
>   *	- size    - size in question
>   */
> -ENTRY(__flush_dcache_area)
> +FUNC(__flush_dcache_area)
>  	dcache_line_size x2, x3
>  	add	x1, x0, x1
>  	sub	x3, x2, #1
> @@ -51,4 +51,4 @@ ENTRY(__flush_dcache_area)
>  	b.lo	1b
>  	dsb	sy
>  	ret
> -ENDPROC(__flush_dcache_area)
> +END(__flush_dcache_area)
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:16:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:16:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716734.1118959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s334S-0004OC-VR; Sat, 04 May 2024 00:16:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716734.1118959; Sat, 04 May 2024 00:16:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s334S-0004O5-SY; Sat, 04 May 2024 00:16:08 +0000
Received: by outflank-mailman (input) for mailman id 716734;
 Sat, 04 May 2024 00:16:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=498C=MH=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s334R-0003oi-TY
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 00:16:07 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7fc64586-09ab-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 02:16:06 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 98F1A60A70;
 Sat,  4 May 2024 00:16:05 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11641C116B1;
 Sat,  4 May 2024 00:16:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fc64586-09ab-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1714781765;
	bh=D437OGtMj2WAwrmdMvTnLcahsVVHscSm2tKPd8vwKUQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=elGDZmCsRrS8UukoXr9mYw3v85wvy0+rSSSRU/vwjPwHkAndLydQCSqZ6pyLt5cq9
	 +10Zy8kbBvwcMRFcz48XIhzxoVoCsujs8/n+d8HjzuRJN0zxSLShKu0GcoYEgmITg5
	 L32G2vO9qTUq9z9YM+dCUNNuVviOyKY70g+dKIy+MG2QYvzEOWIkSBnPKxTMpZh0vl
	 K+EM4WOLVCYNTnYo+e3QJGPGdHo8lcPsAtLlTeMxGHrQJvVmdDTDgNGCuJ3/aVPE76
	 lHcMKpiPet+E/vjboX3dnQMYcidEvmInBZSZLZN1v2n6L0gwhMvKtLcnWLJGmWaUZC
	 t8ssXR3Oj8hgg==
Date: Fri, 3 May 2024 17:16:01 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v3 9/9] xen/arm64: lib: Use the generic xen/linkage.h
 macros
In-Reply-To: <20240501035448.964625-10-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405031715560.1151289@ubuntu-linux-20-04-desktop>
References: <20240501035448.964625-1-edgar.iglesias@gmail.com> <20240501035448.964625-10-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/lib/memchr.S  | 4 ++--
>  xen/arch/arm/arm64/lib/memcmp.S  | 4 ++--
>  xen/arch/arm/arm64/lib/memcpy.S  | 4 ++--
>  xen/arch/arm/arm64/lib/memmove.S | 4 ++--
>  xen/arch/arm/arm64/lib/memset.S  | 4 ++--
>  xen/arch/arm/arm64/lib/strchr.S  | 4 ++--
>  xen/arch/arm/arm64/lib/strcmp.S  | 4 ++--
>  xen/arch/arm/arm64/lib/strlen.S  | 4 ++--
>  xen/arch/arm/arm64/lib/strncmp.S | 4 ++--
>  xen/arch/arm/arm64/lib/strnlen.S | 4 ++--
>  xen/arch/arm/arm64/lib/strrchr.S | 4 ++--
>  11 files changed, 22 insertions(+), 22 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/lib/memchr.S b/xen/arch/arm/arm64/lib/memchr.S
> index 81f113bb1c..3d8aeca3ca 100644
> --- a/xen/arch/arm/arm64/lib/memchr.S
> +++ b/xen/arch/arm/arm64/lib/memchr.S
> @@ -29,7 +29,7 @@
>   * Returns:
>   *	x0 - address of first occurrence of 'c' or 0
>   */
> -ENTRY(memchr)
> +FUNC(memchr)
>  	and	w1, w1, #0xff
>  1:	subs	x2, x2, #1
>  	b.mi	2f
> @@ -40,4 +40,4 @@ ENTRY(memchr)
>  	ret
>  2:	mov	x0, #0
>  	ret
> -ENDPROC(memchr)
> +END(memchr)
> diff --git a/xen/arch/arm/arm64/lib/memcmp.S b/xen/arch/arm/arm64/lib/memcmp.S
> index 87c2537ffe..d77dd4ce52 100644
> --- a/xen/arch/arm/arm64/lib/memcmp.S
> +++ b/xen/arch/arm/arm64/lib/memcmp.S
> @@ -57,7 +57,7 @@ pos		.req	x11
>  limit_wd	.req	x12
>  mask		.req	x13
>  
> -ENTRY(memcmp)
> +FUNC(memcmp)
>  	cbz	limit, .Lret0
>  	eor	tmp1, src1, src2
>  	tst	tmp1, #7
> @@ -254,4 +254,4 @@ CPU_LE( rev	data2, data2 )
>  .Lret0:
>  	mov	result, #0
>  	ret
> -ENDPROC(memcmp)
> +END(memcmp)
> diff --git a/xen/arch/arm/arm64/lib/memcpy.S b/xen/arch/arm/arm64/lib/memcpy.S
> index d90d20ef3e..1e04b79010 100644
> --- a/xen/arch/arm/arm64/lib/memcpy.S
> +++ b/xen/arch/arm/arm64/lib/memcpy.S
> @@ -55,7 +55,7 @@ C_h	.req	x12
>  D_l	.req	x13
>  D_h	.req	x14
>  
> -ENTRY(memcpy)
> +FUNC(memcpy)
>  	mov	dst, dstin
>  	cmp	count, #16
>  	/*When memory length is less than 16, the accessed are not aligned.*/
> @@ -197,4 +197,4 @@ ENTRY(memcpy)
>  	tst	count, #0x3f
>  	b.ne	.Ltail63
>  	ret
> -ENDPROC(memcpy)
> +END(memcpy)
> diff --git a/xen/arch/arm/arm64/lib/memmove.S b/xen/arch/arm/arm64/lib/memmove.S
> index a49de845d0..14438dbe9c 100644
> --- a/xen/arch/arm/arm64/lib/memmove.S
> +++ b/xen/arch/arm/arm64/lib/memmove.S
> @@ -56,7 +56,7 @@ C_h	.req	x12
>  D_l	.req	x13
>  D_h	.req	x14
>  
> -ENTRY(memmove)
> +FUNC(memmove)
>  	cmp	dstin, src
>  	b.lo	memcpy
>  	add	tmp1, src, count
> @@ -193,4 +193,4 @@ ENTRY(memmove)
>  	tst	count, #0x3f
>  	b.ne	.Ltail63
>  	ret
> -ENDPROC(memmove)
> +END(memmove)
> diff --git a/xen/arch/arm/arm64/lib/memset.S b/xen/arch/arm/arm64/lib/memset.S
> index 5bf751521b..367fa60175 100644
> --- a/xen/arch/arm/arm64/lib/memset.S
> +++ b/xen/arch/arm/arm64/lib/memset.S
> @@ -53,7 +53,7 @@ dst		.req	x8
>  tmp3w		.req	w9
>  tmp3		.req	x9
>  
> -ENTRY(memset)
> +FUNC(memset)
>  	mov	dst, dstin	/* Preserve return value.  */
>  	and	A_lw, val, #255
>  	orr	A_lw, A_lw, A_lw, lsl #8
> @@ -212,4 +212,4 @@ ENTRY(memset)
>  	ands	count, count, zva_bits_x
>  	b.ne	.Ltail_maybe_long
>  	ret
> -ENDPROC(memset)
> +END(memset)
> diff --git a/xen/arch/arm/arm64/lib/strchr.S b/xen/arch/arm/arm64/lib/strchr.S
> index 0506b0ff7f..83fd81e8ef 100644
> --- a/xen/arch/arm/arm64/lib/strchr.S
> +++ b/xen/arch/arm/arm64/lib/strchr.S
> @@ -27,7 +27,7 @@
>   * Returns:
>   *	x0 - address of first occurrence of 'c' or 0
>   */
> -ENTRY(strchr)
> +FUNC(strchr)
>  	and	w1, w1, #0xff
>  1:	ldrb	w2, [x0], #1
>  	cmp	w2, w1
> @@ -37,4 +37,4 @@ ENTRY(strchr)
>  	cmp	w2, w1
>  	csel	x0, x0, xzr, eq
>  	ret
> -ENDPROC(strchr)
> +END(strchr)
> diff --git a/xen/arch/arm/arm64/lib/strcmp.S b/xen/arch/arm/arm64/lib/strcmp.S
> index c6f42dd255..7677108e26 100644
> --- a/xen/arch/arm/arm64/lib/strcmp.S
> +++ b/xen/arch/arm/arm64/lib/strcmp.S
> @@ -59,7 +59,7 @@ tmp3		.req	x9
>  zeroones	.req	x10
>  pos		.req	x11
>  
> -ENTRY(strcmp)
> +FUNC(strcmp)
>  	eor	tmp1, src1, src2
>  	mov	zeroones, #REP8_01
>  	tst	tmp1, #7
> @@ -230,4 +230,4 @@ CPU_BE(	orr	syndrome, diff, has_nul )
>  	lsr	data1, data1, #56
>  	sub	result, data1, data2, lsr #56
>  	ret
> -ENDPROC(strcmp)
> +END(strcmp)
> diff --git a/xen/arch/arm/arm64/lib/strlen.S b/xen/arch/arm/arm64/lib/strlen.S
> index fb6aaf1a6a..10feedaf81 100644
> --- a/xen/arch/arm/arm64/lib/strlen.S
> +++ b/xen/arch/arm/arm64/lib/strlen.S
> @@ -56,7 +56,7 @@ pos		.req	x12
>  #define REP8_7f 0x7f7f7f7f7f7f7f7f
>  #define REP8_80 0x8080808080808080
>  
> -ENTRY(strlen)
> +FUNC(strlen)
>  	mov	zeroones, #REP8_01
>  	bic	src, srcin, #15
>  	ands	tmp1, srcin, #15
> @@ -123,4 +123,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp1 )	/* Shift (tmp1 & 63).  */
>  	csinv	data1, data1, xzr, le
>  	csel	data2, data2, data2a, le
>  	b	.Lrealigned
> -ENDPROC(strlen)
> +END(strlen)
> diff --git a/xen/arch/arm/arm64/lib/strncmp.S b/xen/arch/arm/arm64/lib/strncmp.S
> index a4a0f779f5..c00a641fc7 100644
> --- a/xen/arch/arm/arm64/lib/strncmp.S
> +++ b/xen/arch/arm/arm64/lib/strncmp.S
> @@ -64,7 +64,7 @@ limit_wd	.req	x13
>  mask		.req	x14
>  endloop		.req	x15
>  
> -ENTRY(strncmp)
> +FUNC(strncmp)
>  	cbz	limit, .Lret0
>  	eor	tmp1, src1, src2
>  	mov	zeroones, #REP8_01
> @@ -307,4 +307,4 @@ CPU_BE( orr	syndrome, diff, has_nul )
>  .Lret0:
>  	mov	result, #0
>  	ret
> -ENDPROC(strncmp)
> +END(strncmp)
> diff --git a/xen/arch/arm/arm64/lib/strnlen.S b/xen/arch/arm/arm64/lib/strnlen.S
> index 81c8e8b54e..2059ba782c 100644
> --- a/xen/arch/arm/arm64/lib/strnlen.S
> +++ b/xen/arch/arm/arm64/lib/strnlen.S
> @@ -59,7 +59,7 @@ limit_wd	.req	x14
>  #define REP8_7f 0x7f7f7f7f7f7f7f7f
>  #define REP8_80 0x8080808080808080
>  
> -ENTRY(strnlen)
> +FUNC(strnlen)
>  	cbz	limit, .Lhit_limit
>  	mov	zeroones, #REP8_01
>  	bic	src, srcin, #15
> @@ -168,4 +168,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp4 )	/* Shift (tmp1 & 63).  */
>  .Lhit_limit:
>  	mov	len, limit
>  	ret
> -ENDPROC(strnlen)
> +END(strnlen)
> diff --git a/xen/arch/arm/arm64/lib/strrchr.S b/xen/arch/arm/arm64/lib/strrchr.S
> index 07059983f8..81033c0822 100644
> --- a/xen/arch/arm/arm64/lib/strrchr.S
> +++ b/xen/arch/arm/arm64/lib/strrchr.S
> @@ -26,7 +26,7 @@
>   * Returns:
>   *	x0 - address of last occurrence of 'c' or 0
>   */
> -ENTRY(strrchr)
> +FUNC(strrchr)
>  	mov	x3, #0
>  	and	w1, w1, #0xff
>  1:	ldrb	w2, [x0], #1
> @@ -37,4 +37,4 @@ ENTRY(strrchr)
>  	b	1b
>  2:	mov	x0, x3
>  	ret
> -ENDPROC(strrchr)
> +END(strrchr)
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:43:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:43:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716747.1118969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s33Ue-0001Jq-0T; Sat, 04 May 2024 00:43:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716747.1118969; Sat, 04 May 2024 00:43:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s33Ud-0001Jj-TQ; Sat, 04 May 2024 00:43:11 +0000
Received: by outflank-mailman (input) for mailman id 716747;
 Sat, 04 May 2024 00:43:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s33Uc-0001JZ-28; Sat, 04 May 2024 00:43:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s33Ub-0001hj-Rh; Sat, 04 May 2024 00:43:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s33Ub-0002QN-C4; Sat, 04 May 2024 00:43:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s33Ub-0002df-Ba; Sat, 04 May 2024 00:43:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=I0j0m2d/zSS/52kZBeK4m3sdDtQhRjI3BGwJGViPkxo=; b=yR4aUdApQxT0mGe3oCYl1hpdxQ
	18uwBuDfkMqrJWgrzgrJNN1PjSofR9Qw/ieSh+VsgD0iIZLeIYSOeWcP6RARTt5kP2HGeh83bP59I
	NSO7hTFwc21aiTz/WVPQctHlIFiMwZk1NWfyngCtk2fzbAA+YY+AzxocIUGhku7YH5Kc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185911-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185911: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f03359bca01bf4372cf2c118cd9a987a5951b1c8
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 May 2024 00:43:09 +0000

flight 185911 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185911/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 185870
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 185870

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot         fail in 185907 pass in 185911
 test-armhf-armhf-xl-rtds      8 xen-boot         fail in 185907 pass in 185911
 test-armhf-armhf-xl-credit1   8 xen-boot         fail in 185907 pass in 185911
 test-armhf-armhf-xl           8 xen-boot                   fail pass in 185907

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185870
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail in 185907 like 185870
 test-armhf-armhf-xl         15 migrate-support-check fail in 185907 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 185907 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                f03359bca01bf4372cf2c118cd9a987a5951b1c8
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    4 days
Failing since        185888  2024-04-30 20:31:50 Z    3 days    7 attempts
Testing same since   185907  2024-05-03 02:43:45 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Potapenko <glider@google.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Protopopov <aspsk@isovalent.com>
  Arnd Bergmann <arnd@arndb.de>
  Asbjørn Sloth Tønnesen <ast@fiberby.net>
  Audra Mitchell <audra@redhat.com>
  Barry Song <v-songbaohua@oppo.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Björn Töpel <bjorn@kernel.org>
  Björn Töpel <bjorn@rivosinc.com>
  Bui Quang Minh <minhquangbui99@gmail.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Christoph Paasch <cpaasch@apple.com>
  Chuck Lever <chuck.lever@oracle.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Borkmann <daniel@iogearbox.net>
  David Bauer <mail@david-bauer.net>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Dima Ruinskiy <dima.ruinskiy@intel.com>
  Dominique Martinet <dominique.martinet@atmark-techno.com>
  Donald Hunter <donald.hunter@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guillaume Nault <gnault@redhat.com>
  Hans de Goede <hdegoede@redhat.com>
  Harald Freudenberger <freude@linux.ibm.com>
  Heiko Carstens <hca@linux.ibm.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Ivan Babrou <ivan@cloudflare.com>
  Jakub Kicinski <kuba@kernel.org>
  James Chapman <jchapman@katalix.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  Jason Xing <kernelxing@tencent.com>
  Jeffrey Altman <jaltman@auristor.com>
  Jens Remus <jremus@linux.ibm.com>
  Jesper Dangaard Brouer <hawk@kernel.org>
  John Garry <john.g.garry@oracle.com>
  Josef Bacik <josef@toxicpanda.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Marc Dionne <marc.dionne@auristor.com>
  Marek Behún <kabel@kernel.org>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <martin.lau@kernel.org>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Max Filippov <jcmvbkbc@gmail.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Abeni <pabeni@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Puranjay Mohan <puranjay12@gmail.com>
  Puranjay Mohan <puranjay@kernel.org>
  Qu Wenruo <wqu@suse.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Richard Gobert <richardbgobert@gmail.com>
  Sasha Neftin <sasha.neftin@intel.com>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislav Fomichev <sdf@google.com>
  Sven Schnelle <svens@linux.ibm.com>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tejun Heo <tj@kernel.org>
  Thanassis Avgerinos <thanassis.avgerinos@gmail.com>
  Thorsten Blum <thorsten.blum@toblux.com>
  Toke Høiland-Jørgensen <toke@redhat.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Vitaly Lifshits <vitaly.lifshits@intel.com>
  Xin Long <lucien.xin@gmail.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Kuohai <xukuohai@huawei.com>
  Yunsheng Lin <linyunsheng@huawei.com>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4031 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat May 04 00:48:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 00:48:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716753.1118978 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s33Zj-0002XZ-NN; Sat, 04 May 2024 00:48:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716753.1118978; Sat, 04 May 2024 00:48:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s33Zj-0002XS-Kg; Sat, 04 May 2024 00:48:27 +0000
Received: by outflank-mailman (input) for mailman id 716753;
 Sat, 04 May 2024 00:48:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjfK=MH=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s33Zi-0002XM-3N
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 00:48:26 +0000
Received: from wfout7-smtp.messagingengine.com
 (wfout7-smtp.messagingengine.com [64.147.123.150])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ff5b10ca-09af-11ef-b4bb-af5377834399;
 Sat, 04 May 2024 02:48:22 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id 85E941C00122;
 Fri,  3 May 2024 20:48:16 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Fri, 03 May 2024 20:48:16 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 3 May 2024 20:48:14 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff5b10ca-09af-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1714783696;
	 x=1714870096; bh=8y9zp+lJs5L2nOx89qV9x4rcjsN+qZDo7GnH7Aoq+28=; b=
	5G65szjPp9UiDoAiqzCSvstriv2UAz1oL1o16gyXMFP7/pFZOBqcn1wWpOMo8/GR
	N3AC9gAdbElNwdUneGP59UOQDfS8gVkuBm0zsqOI0PbUsAPgQnWHRQZutPPE0Y0J
	GmIomaqcsbPdYKE6x3KglLC+dvKNNW4s+kx0RVdJ8+wWctPSuDHitouMIIF5aVbT
	euxBLh3DS0acic8lAZTidZp4vnCQx2kSLuYgMAVXwuriyOzkxgtK49QanZAreYtp
	kQ8hgGd0mcOBoiRBUN33kJ0VlF5nGxMXpxCJEzVGfHvUxfbEvQB+NoU4Cs6TZXay
	3Tneu8w2FDhpoqv0dNXniw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1714783696; x=1714870096; bh=8y9zp+lJs5L2nOx89qV9x4rcjsN+
	qZDo7GnH7Aoq+28=; b=PNcYXFSMH9zhgRnvAlZzdinlWoDYUcijPvjBYrpzANYc
	qDw3vN3xhkYUnCCFfTRHQyuNtDViGtJ4P0UOuRAKZ7QCcvjTgqMAHMqZSIVp0MVq
	yjjbQ80GkG9esMZcxX4blZgUUsU3GLBf41uTl+dOM0/o9K3UJKNmnsMoIZIZxjP+
	ZSAdNsMV+ws+Xd+RDwLjbaUJzLOoj6F933i+HrrpicjACLU6h0ssp7Dh7dS7peXq
	NTXp4qwT+Xx1TrtamOWUuY+VormeJz9RLF3ySz/Z4C5p7wfG8D2YA4IEap1FKEjO
	4X111STgRSO0S6KYSG8uMZ+wmvzGrWZAXE2qOC8KIQ==
X-ME-Sender: <xms:z4U1Zo8KzAh8s1YqfdB22bxsSSlxKO4WHrOpJ8_c8hx02CtcoNF4Kw>
    <xme:z4U1ZgsVJxYrKE2YJTadH28LoQ7rA0ieWSqcZ2PUWM1RN5jnqhiTOnsNMk9MDNqwo
    W5Xog6H-oOWgg>
X-ME-Received: <xmr:z4U1ZuAvEHOBSgowyDlJeOq3V2znKmKtbnBySb13Akg3kVuAqVCkdcpegYaaB_bEXeTEeHKHmLOuijrfpY9tTKMZZm50gldR7A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvuddgfeekucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepieff
    tdfgvefgkeehffekffduiedtfefhhffgvdfhgfeugfffffegleekveeiueffnecuffhomh
    grihhnpehmshhigihtsghlpghlihhsthdrnhgvgihtnecuvehluhhsthgvrhfuihiivgep
    tdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslh
    gvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:z4U1ZocOSzAoOL2OAVYPbW8ms0h4kEdbRzjWsYj_VKRArEL4qWaMvA>
    <xmx:z4U1ZtM9urGL9O_FGN7mJ5iq4E-XcYGVRxSq5JtPEqq0tLrG-AHujw>
    <xmx:z4U1ZimBQnn7-1pe7g0gPuwojqrP2VbNNohDVFTjpTVcjm8vnXlJxA>
    <xmx:z4U1ZvtlxXO6VHXJFLiDUZHczAjqlEb7ZcdY7b3J-O8ejwrTZeHofw>
    <xmx:0IU1ZuqCYPeQzeTfCohhCQAVYxuC_9Jackt9lzNLrJnpV0Hv2pfxTG35>
Feedback-ID: i1568416f:Fastmail
Date: Sat, 4 May 2024 02:48:12 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v6 3/7] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <ZjWFzEdJ9wiz2GMI@mail-itl>
References: <cover.3491f479d5b6238106af8ddc56ec6834090b330f.1714154036.git-series.marmarek@invisiblethingslab.com>
 <a9b04e2224e97a27a127a003e8ccf5edfd4922c7.1714154036.git-series.marmarek@invisiblethingslab.com>
 <ZjShYhUvx694rPEp@macbook>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="IjZAMjsYod9cGStt"
Content-Disposition: inline
In-Reply-To: <ZjShYhUvx694rPEp@macbook>


--IjZAMjsYod9cGStt
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sat, 4 May 2024 02:48:12 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v6 3/7] x86/hvm: Allow access to registers on the same
 page as MSI-X table

On Fri, May 03, 2024 at 10:33:38AM +0200, Roger Pau Monn=C3=A9 wrote:
> On Fri, Apr 26, 2024 at 07:54:00PM +0200, Marek Marczykowski-G=C3=B3recki=
 wrote:
> > Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> > on the same page as MSI-X table. Device model (especially one in
> > stubdomain) cannot really handle those, as direct writes to that page is
> > refused (page is on the mmio_ro_ranges list). Instead, extend
> > msixtbl_mmio_ops to handle such accesses too.
> >=20
> > Doing this, requires correlating read/write location with guest
> > of MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
>   ^ extra 'of'?
> > it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> > for PV would need to be done separately.
> >=20
> > This will be also used to read Pending Bit Array, if it lives on the sa=
me
> > page, making QEMU not needing /dev/mem access at all (especially helpful
> > with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> > map it to the guest directly.
> > If PBA lives on the same page, discard writes and log a message.
> > Technically, writes outside of PBA could be allowed, but at this moment
> > the precise location of PBA isn't saved, and also no known device abuses
> > the spec in this way (at least yet).
> >=20
> > To access those registers, msixtbl_mmio_ops need the relevant page
> > mapped. MSI handling already has infrastructure for that, using fixmap,
> > so try to map first/last page of the MSI-X table (if necessary) and save
> > their fixmap indexes. Note that msix_get_fixmap() does reference
> > counting and reuses existing mapping, so just call it directly, even if
> > the page was mapped before. Also, it uses a specific range of fixmap
> > indexes which doesn't include 0, so use 0 as default ("not mapped")
> > value - which simplifies code a bit.
> >=20
> > GCC 12.2.1 gets confused about 'desc' variable:
> >=20
> >     arch/x86/hvm/vmsi.c: In function =E2=80=98msixtbl_range=E2=80=99:
> >     arch/x86/hvm/vmsi.c:553:8: error: =E2=80=98desc=E2=80=99 may be use=
d uninitialized [-Werror=3Dmaybe-uninitialized]
> >       553 |     if ( desc )
> >           |        ^
> >     arch/x86/hvm/vmsi.c:537:28: note: =E2=80=98desc=E2=80=99 was declar=
ed here
> >       537 |     const struct msi_desc *desc;
> >           |                            ^~~~
> >=20
> > It's conditional initialization is actually correct (in the case where
> > it isn't initialized, function returns early), but to avoid
> > build failure initialize it explicitly to NULL anyway.
> >=20
> > Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblething=
slab.com>
> > ---
> > Changes in v6:
> > - use MSIX_CHECK_WARN macro
> > - extend assert on fixmap_idx
> > - add break in default label, after ASSERT_UNREACHABLE(), and move
> >   setting default there
> > - style fixes
> > Changes in v5:
> > - style fixes
> > - include GCC version in the commit message
> > - warn only once (per domain, per device) about failed adjacent access
> > Changes in v4:
> > - drop same_page parameter of msixtbl_find_entry(), distinguish two
> >   cases in relevant callers
> > - rename adj_access_table_idx to adj_access_idx
> > - code style fixes
> > - drop alignment check in adjacent_{read,write}() - all callers already
> >   have it earlier
> > - delay mapping first/last MSI-X pages until preparing device for a
> >   passthrough
> > v3:
> >  - merge handling into msixtbl_mmio_ops
> >  - extend commit message
> > v2:
> >  - adjust commit message
> >  - pass struct domain to msixtbl_page_handler_get_hwaddr()
> >  - reduce local variables used only once
> >  - log a warning if write is forbidden if MSI-X and PBA lives on the sa=
me
> >    page
> >  - do not passthrough unaligned accesses
> >  - handle accesses both before and after MSI-X table
> > ---
> >  xen/arch/x86/hvm/vmsi.c        | 200 ++++++++++++++++++++++++++++++++--
> >  xen/arch/x86/include/asm/msi.h |   5 +-
> >  xen/arch/x86/msi.c             |  41 +++++++-
> >  3 files changed, 236 insertions(+), 10 deletions(-)
> >=20
> > diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> > index 999917983789..230e3a5dee3f 100644
> > --- a/xen/arch/x86/hvm/vmsi.c
> > +++ b/xen/arch/x86/hvm/vmsi.c
> > @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domai=
n *d)
> >      return d->arch.hvm.msixtbl_list.next;
> >  }
> > =20
> > +/*
> > + * Lookup an msixtbl_entry on the same page as given addr. It's up to =
the
> > + * caller to check if address is strictly part of the table - if relev=
ant.
> > + */
> >  static struct msixtbl_entry *msixtbl_find_entry(
> >      struct vcpu *v, unsigned long addr)
> >  {
> > @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
> >      struct domain *d =3D v->domain;
> > =20
> >      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> > -        if ( addr >=3D entry->gtable &&
> > -             addr < entry->gtable + entry->table_len )
> > +        if ( PFN_DOWN(addr) >=3D PFN_DOWN(entry->gtable) &&
> > +             PFN_DOWN(addr) <=3D PFN_DOWN(entry->gtable + entry->table=
_len - 1) )
> >              return entry;
> > =20
> >      return NULL;
> > @@ -213,6 +217,138 @@ static struct msi_desc *msixtbl_addr_to_desc(
> >      return NULL;
> >  }
> > =20
> > +/*
> > + * Returns:
> > + *  - ADJACENT_DONT_HANDLE if no handling should be done
> > + *  - ADJACENT_DISCARD_WRITE if write should be discarded
> > + *  - a fixmap idx to use for handling
> > + */
> > +#define ADJACENT_DONT_HANDLE UINT_MAX
> > +#define ADJACENT_DISCARD_WRITE (UINT_MAX - 1)
>=20
> I think this could be simpler, there's no need to signal with so fine
> grained detail about the action to be performed.
>=20
> Any adjacent access to the MSI-X table should be handled by the logic
> you are adding, so anything that falls in those ranges should
> terminate here.
>=20
> adjacent_handle() should IMO just return whether the access is
> replayed against the hardware, or if it's just dropped.

The distinction here is to return X86EMUL_OKAY in case of adjacent write
that is ignored because PBA is somewhere near, but X86EMUL_UNHANDLABLE
for other/error cases (like fixmap indices not initialized).
But maybe this distinction doesn't make sense and X86EMUL_UNHANDLABLE is
okay in either case?=20

> > +static unsigned int adjacent_handle(
> > +    const struct msixtbl_entry *entry, unsigned long addr, bool write)
> > +{
> > +    unsigned int adj_type;
> > +    struct arch_msix *msix;
> > +
> > +    if ( !entry || !entry->pdev )
> > +        return ADJACENT_DONT_HANDLE;
> > +
> > +    if ( PFN_DOWN(addr) =3D=3D PFN_DOWN(entry->gtable) && addr < entry=
->gtable )
> > +        adj_type =3D ADJ_IDX_FIRST;
> > +    else if ( PFN_DOWN(addr) =3D=3D PFN_DOWN(entry->gtable + entry->ta=
ble_len - 1) &&
> > +              addr >=3D entry->gtable + entry->table_len )
> > +        adj_type =3D ADJ_IDX_LAST;
> > +    else
> > +        return ADJACENT_DONT_HANDLE;
> > +
> > +    msix =3D entry->pdev->msix;
> > +    ASSERT(msix);
> > +
> > +    if ( !msix->adj_access_idx[adj_type] )
> > +    {
> > +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> > +                             adjacent_not_initialized) )
> > +            gprintk(XENLOG_WARNING,
> > +                    "Page for adjacent(%d) MSI-X table access not init=
ialized for %pp (addr %#lx, gtable %#lx\n",
> > +                    adj_type, &entry->pdev->sbdf, addr, entry->gtable);
> > +        return ADJACENT_DONT_HANDLE;
> > +    }
> > +
> > +    /* If PBA lives on the same page too, discard writes. */
> > +    if ( write &&
> > +         ((adj_type =3D=3D ADJ_IDX_LAST &&
> > +           msix->table.last =3D=3D msix->pba.first) ||
> > +          (adj_type =3D=3D ADJ_IDX_FIRST &&
> > +           msix->table.first =3D=3D msix->pba.last)) )
> > +    {
> > +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> > +                             adjacent_pba) )
> > +            gprintk(XENLOG_WARNING,
> > +                    "MSI-X table and PBA of %pp live on the same page,=
 "
> > +                    "writing to other registers there is not implement=
ed\n",
> > +                    &entry->pdev->sbdf);
> > +        return ADJACENT_DISCARD_WRITE;
> > +    }
> > +
> > +    return msix->adj_access_idx[adj_type];
> > +}
> > +
> > +static int adjacent_read(
> > +    unsigned int fixmap_idx,
> > +    paddr_t address, unsigned int len, uint64_t *pval)
> > +{
> > +    const void __iomem *hwaddr;
> > +
> > +    ASSERT(fixmap_idx <=3D FIX_MSIX_IO_RESERV_END);
> > +
> > +    hwaddr =3D fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
>=20
> IMO adjacent_handle() should be called here (and in adjacent_write()),
> and adjacent_{read,write}() called unconditionally from
> msixtbl_{read,write}() when an access that doesn't fall in the MSI-X
> table is handled.  See comment below in msixtbl_read().

Makes sense.

> > +
> > +    switch ( len )
> > +    {
> > +    case 1:
> > +        *pval =3D readb(hwaddr);
> > +        break;
> > +
> > +    case 2:
> > +        *pval =3D readw(hwaddr);
> > +        break;
> > +
> > +    case 4:
> > +        *pval =3D readl(hwaddr);
> > +        break;
> > +
> > +    case 8:
> > +        *pval =3D readq(hwaddr);
> > +        break;
> > +
> > +    default:
> > +        ASSERT_UNREACHABLE();
> > +        *pval =3D ~0UL;
> > +        break;
> > +    }
> > +
> > +    return X86EMUL_OKAY;
> > +}
> > +
> > +static int adjacent_write(
> > +    unsigned int fixmap_idx,
> > +    paddr_t address, unsigned int len, uint64_t val)
> > +{
> > +    void __iomem *hwaddr;
> > +
> > +    if ( fixmap_idx =3D=3D ADJACENT_DISCARD_WRITE )
> > +        return X86EMUL_OKAY;
> > +
> > +    ASSERT(fixmap_idx <=3D FIX_MSIX_IO_RESERV_END);
>=20
> Since you check the idx is sane, shouldn't you also assert idx >=3D
> FIX_MSIX_IO_RESERV_BASE?

If moving adjacent_handle() here, I'd simply drop this assert.

> > +
> > +    hwaddr =3D fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> > +
> > +    switch ( len )
> > +    {
> > +    case 1:
> > +        writeb(val, hwaddr);
> > +        break;
> > +
> > +    case 2:
> > +        writew(val, hwaddr);
> > +        break;
> > +
> > +    case 4:
> > +        writel(val, hwaddr);
> > +        break;
> > +
> > +    case 8:
> > +        writeq(val, hwaddr);
> > +        break;
> > +
> > +    default:
> > +        ASSERT_UNREACHABLE();
> > +    }
> > +
> > +    return X86EMUL_OKAY;
> > +}
> > +
> >  static int cf_check msixtbl_read(
> >      const struct hvm_io_handler *handler, uint64_t address, uint32_t l=
en,
> >      uint64_t *pval)
> > @@ -220,9 +356,10 @@ static int cf_check msixtbl_read(
> >      unsigned long offset;
> >      struct msixtbl_entry *entry;
> >      unsigned int nr_entry, index;
> > +    unsigned int adjacent_fixmap;
> >      int r =3D X86EMUL_UNHANDLEABLE;
> > =20
> > -    if ( (len !=3D 4 && len !=3D 8) || (address & (len - 1)) )
> > +    if ( !IS_ALIGNED(address, len) )
> >          return r;
> > =20
> >      rcu_read_lock(&msixtbl_rcu_lock);
> > @@ -230,6 +367,21 @@ static int cf_check msixtbl_read(
> >      entry =3D msixtbl_find_entry(current, address);
> >      if ( !entry )
> >          goto out;
> > +
> > +    adjacent_fixmap =3D adjacent_handle(entry, address, false);
>=20
> This seems overly complicated, but is possible I'm missing some logic.
>=20
> IMO it would seem way less convoluted to simply do:
>=20
> entry =3D msixtbl_find_entry(current, address);
> if ( !entry )
>     goto out;
>=20
> if ( address < entry->gtable ||
>      address >=3D entry->gtable + entry->table_len )
> {
>     adjacent_read(...);
>     goto out;
> }
>=20
> And put all the logic in adjacent_{read,write}() directly rather than
> having both adjacent_{read,write}() plus adjacent_handle() calls here?
>=20
> If the access doesn't fall between the boundaries of the MSI-X table
> it's either going to be a handled adjacent access, or it's going to be
> discarded.

Discarded - should it return X86EMUL_OKAY in that case? Currently it
returns X86EMUL_UNHANDLABLE in case adjacent access isn't handled (for
any reason) either.

> > +    if ( adjacent_fixmap !=3D ADJACENT_DONT_HANDLE )
> > +    {
> > +        r =3D adjacent_read(adjacent_fixmap, address, len, pval);
> > +        goto out;
> > +    }
> > +
> > +    if ( address < entry->gtable ||
> > +         address >=3D entry->gtable + entry->table_len )
> > +        goto out;
> > +
> > +    if ( len !=3D 4 && len !=3D 8 )
> > +        goto out;
> > +
> >      offset =3D address & (PCI_MSIX_ENTRY_SIZE - 1);
> > =20
> >      if ( offset !=3D PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
> > @@ -282,6 +434,7 @@ static int msixtbl_write(struct vcpu *v, unsigned l=
ong address,
> >      int r =3D X86EMUL_UNHANDLEABLE;
> >      unsigned long flags;
> >      struct irq_desc *desc;
> > +    unsigned int adjacent_fixmap;
> > =20
> >      if ( !IS_ALIGNED(address, len) )
> >          return X86EMUL_OKAY;
> > @@ -291,6 +444,21 @@ static int msixtbl_write(struct vcpu *v, unsigned =
long address,
> >      entry =3D msixtbl_find_entry(v, address);
> >      if ( !entry )
> >          goto out;
> > +
> > +    adjacent_fixmap =3D adjacent_handle(entry, address, true);
> > +    if ( adjacent_fixmap !=3D ADJACENT_DONT_HANDLE )
> > +    {
> > +        r =3D adjacent_write(adjacent_fixmap, address, len, val);
> > +        goto out;
> > +    }
> > +
> > +    if ( address < entry->gtable ||
> > +         address >=3D entry->gtable + entry->table_len )
> > +        goto out;
> > +
> > +    if ( len !=3D 4 && len !=3D 8 )
> > +        goto out;
> > +
> >      nr_entry =3D array_index_nospec(((address - entry->gtable) /
> >                                     PCI_MSIX_ENTRY_SIZE),
> >                                    MAX_MSIX_TABLE_ENTRIES);
> > @@ -356,8 +524,8 @@ static int cf_check _msixtbl_write(
> >      const struct hvm_io_handler *handler, uint64_t address, uint32_t l=
en,
> >      uint64_t val)
> >  {
> > -    /* Ignore invalid length or unaligned writes. */
> > -    if ( (len !=3D 4 && len !=3D 8) || !IS_ALIGNED(address, len) )
> > +    /* Ignore unaligned writes. */
> > +    if ( !IS_ALIGNED(address, len) )
> >          return X86EMUL_OKAY;
> > =20
> >      /*
> > @@ -374,14 +542,22 @@ static bool cf_check msixtbl_range(
> >  {
> >      struct vcpu *curr =3D current;
> >      unsigned long addr =3D r->addr;
> > -    const struct msi_desc *desc;
> > +    const struct msixtbl_entry *entry;
> > +    const struct msi_desc *desc =3D NULL;
> > +    unsigned int adjacent_fixmap;
> > =20
> >      ASSERT(r->type =3D=3D IOREQ_TYPE_COPY);
> > =20
> >      rcu_read_lock(&msixtbl_rcu_lock);
> > -    desc =3D msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr=
);
> > +    entry =3D msixtbl_find_entry(curr, addr);
> > +    adjacent_fixmap =3D adjacent_handle(entry, addr, false);
> > +    if ( adjacent_fixmap =3D=3D ADJACENT_DONT_HANDLE )
> > +        desc =3D msixtbl_addr_to_desc(entry, addr);
>=20
> I'm not sure you need adjacent_handle() here, I would think that any
> address adjacent to the MSI-X table Xen would want to handle
> unconditionally, and hence msixtbl_range() should return true:

I put it here to not duplicate a set of checks for adjacent access. It
isn't only about the range, but also few other case (like if fixmap
indices are set).

> entry =3D msixtbl_find_entry(curr, addr);
> if ( !entry )
>     return 0;
>=20
> if ( addr < entry->gtable || addr >=3D entry->gtable + entry->table_len )
>     /* Possibly handle adjacent access. */
>     return 1;
>=20
> desc =3D msixtbl_find_entry(curr, addr);
> ...
>=20
> (Missing the _unlock calls in the chunk above)
>=20
> There's no other processing that can happen for an adjacent access
> unless it's are handled here.  Otherwise such accesses will be
> discarded anyway?  Hence better short-circuit the MMIO handler search
> earlier.

Can't there be some ioreq server that could theoretically handle them?

> >      rcu_read_unlock(&msixtbl_rcu_lock);
> > =20
> > +    if ( adjacent_fixmap !=3D ADJACENT_DONT_HANDLE )
> > +        return 1;
> > +
> >      if ( desc )
> >          return 1;
> > =20
> > @@ -622,12 +798,16 @@ void msix_write_completion(struct vcpu *v)
> >           v->arch.hvm.hvm_io.msix_snoop_gpa )
> >      {
> >          unsigned int token =3D hvmemul_cache_disable(v);
> > -        const struct msi_desc *desc;
> > +        const struct msi_desc *desc =3D NULL;
> > +        const struct msixtbl_entry *entry;
> >          uint32_t data;
> > =20
> >          rcu_read_lock(&msixtbl_rcu_lock);
> > -        desc =3D msixtbl_addr_to_desc(msixtbl_find_entry(v, snoop_addr=
),
> > -                                    snoop_addr);
> > +        entry =3D msixtbl_find_entry(v, snoop_addr);
> > +        if ( entry &&
> > +             snoop_addr >=3D entry->gtable &&
> > +             snoop_addr < entry->gtable + entry->table_len )
> > +            desc =3D msixtbl_addr_to_desc(entry, snoop_addr);
>=20
> This would be easier if you added the MSI-X table boundary checks in
> msixtbl_addr_to_desc() itself, rather than open-coding here.  That way
> you don't need to modify msix_write_completion() at all.  It also
> makes msixtbl_addr_to_desc() more robust in case it gets called with
> bogus addresses.

Makes sense I think.

> >          rcu_read_unlock(&msixtbl_rcu_lock);
> > =20
> >          if ( desc &&
> > diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/=
msi.h
> > index bcfdfd35345d..923b730d48b8 100644
> > --- a/xen/arch/x86/include/asm/msi.h
> > +++ b/xen/arch/x86/include/asm/msi.h
> > @@ -224,6 +224,9 @@ struct arch_msix {
> >      } table, pba;
> >      int table_refcnt[MAX_MSIX_TABLE_PAGES];
> >      int table_idx[MAX_MSIX_TABLE_PAGES];
> > +#define ADJ_IDX_FIRST 0
> > +#define ADJ_IDX_LAST  1
> > +    unsigned int adj_access_idx[2];
> >      spinlock_t table_lock;
> >      bool host_maskall, guest_maskall;
> >      domid_t warned_domid;
> > @@ -231,6 +234,8 @@ struct arch_msix {
> >          uint8_t all;
> >          struct {
> >              bool maskall                   : 1;
> > +            bool adjacent_not_initialized  : 1;
> > +            bool adjacent_pba              : 1;
> >          };
> >      } warned_kind;
> >  };
> > diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> > index 42c793426da3..c77b81896269 100644
> > --- a/xen/arch/x86/msi.c
> > +++ b/xen/arch/x86/msi.c
> > @@ -913,6 +913,36 @@ static int msix_capability_init(struct pci_dev *de=
v,
> >          list_add_tail(&entry->list, &dev->msi_list);
> >          *desc =3D entry;
> >      }
> > +    else
> > +    {
> > +        /*
> > +         * If the MSI-X table doesn't start at the page boundary, map =
the first page for
> > +         * passthrough accesses.
> > +         */
> > +        if ( PAGE_OFFSET(table_paddr) )
> > +        {
> > +            int idx =3D msix_get_fixmap(msix, table_paddr, table_paddr=
);
> > +
> > +            if ( idx > 0 )
> > +                msix->adj_access_idx[ADJ_IDX_FIRST] =3D idx;
> > +            else
> > +                gprintk(XENLOG_ERR, "Failed to map first MSI-X table p=
age: %d\n", idx);
> > +        }
> > +        /*
> > +         * If the MSI-X table doesn't end on the page boundary, map th=
e last page
> > +         * for passthrough accesses.
> > +         */
> > +        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENT=
RY_SIZE) )
>=20
> This check is correct ....
>=20
> > +        {
> > +            uint64_t entry_paddr =3D table_paddr + msix->nr_entries * =
PCI_MSIX_ENTRY_SIZE;
>=20
> ... however for correctness you want to use msix->nr_entries - 1 here,
> otherwise you are requesting an address that's past the last MSI-X
> table entry, and hence msix_get_fixmap() could refuse to provide an
> idx if it ever does boundary checking.

Ok.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--IjZAMjsYod9cGStt
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmY1hcwACgkQ24/THMrX
1yxHMgf/TIQoc01q6v4iUJ9JUi/woENjVARQ6ckn1lBk+6zGKFgFxYjSZpJXKRK/
+cBTvD5fZ6L1oWl4Dr50ECyroX5o6gr6tDHNFcrxkthypgi6cUnvCVwGJlJGSGYh
L2qrBGPYKodi3IUxjFn8KlyGfO2ACR7E3oWlVfmnUTiijftw1pX6/j3JbTZBsbQj
/Hq3OCsQ27WVYC3PsEzOYQTwra0IRVof8aq0crH+fkdAJC05frcF08roncHf6dc6
1yZJqCqMhi4MnWpB+ZUdxY4JWjHIOPsQk4EF26iHnbOf8heyf/bOhUVN3sj9kyzz
dpt1SYFXC4MPzj8P9cl/Wccjg7CsSQ==
=JW+V
-----END PGP SIGNATURE-----

--IjZAMjsYod9cGStt--


From xen-devel-bounces@lists.xenproject.org Sat May 04 01:16:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 01:16:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716761.1118989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s340i-0006MN-QW; Sat, 04 May 2024 01:16:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716761.1118989; Sat, 04 May 2024 01:16:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s340i-0006MG-My; Sat, 04 May 2024 01:16:20 +0000
Received: by outflank-mailman (input) for mailman id 716761;
 Sat, 04 May 2024 01:16:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kUU4=MH=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s340g-0006MA-Tc
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 01:16:18 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e869971c-09b3-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 03:16:17 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51f17ac14daso279508e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 18:16:17 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r7-20020a170906350700b00a57e2d39d56sm2352639eja.223.2024.05.03.18.16.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 18:16:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e869971c-09b3-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714785377; x=1715390177; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=AmgLG2/r7crf4VeZKFy4V6sV4ajE44kBgHnEhpJVc4s=;
        b=UoDnbu04hRFPUIlZ49zs/K/NR7vMhHdbtT7+KlqW9KTnmChWPycI4c6fLiuc6QvVQQ
         Qja4VG1kp1ytH5y77/ENolCxd9mK3ZHA4LVEjEXN7oQ8UM0wTzI5IV8KBbxwV6ORe8op
         nPJ/65Nx9nRK2LR/Z4u1MzEu9HwNKNZnVaUKk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714785377; x=1715390177;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=AmgLG2/r7crf4VeZKFy4V6sV4ajE44kBgHnEhpJVc4s=;
        b=wWpOeTBBfe4I3cXy75jWZX0ug4mInDykJtRQH+mI96E/egy2Fmj4BluTPpfJyHGsrM
         yuBaDjm4lapqKbb43EWBwCMd88aGOozEggnxyUN/dUov7wQwJzbYIGbgo0XRYc49wmXs
         Qkf3v9yOKf4m14BD18QE2HTsvfzwq8DOIKzFY55xF36+LhRcxMUV5Zml+DsFgFQ4bXHT
         HZ+E9l7D4iJOiKdBPnOpo973Dv8hYM5L98WAy8/vp20IRslcgkeF1uIuVqI/m4FwBTmX
         CINfs1OB0OQF+nlSYEDpyLhN6vQ/wKwKUWcsmxzk/CrpdeNxDK5KL+fjuFHpJratto97
         vHbA==
X-Gm-Message-State: AOJu0Yy5mmJjUqkrt5e8z1h5BMVuUvAZWW8BTo8KaVJVDcMGUrQ5Stoh
	5d5cBkkgNxmaMNVIBgbiXwwGrHPFkRH9bsQMSIR23uz7zH9NKjhaHDlGa0CROnDJ3OaGoWzhwxw
	i
X-Google-Smtp-Source: AGHT+IHqI0krkuqGwRYB+NnZPz4XCKxWW9sv2ZAYTyzVfM13n0PbKNPYSKmS1DOhIMjCipdqFX/H4A==
X-Received: by 2002:a05:6512:3108:b0:51d:4260:4bf8 with SMTP id n8-20020a056512310800b0051d42604bf8mr2815739lfb.35.1714785376732;
        Fri, 03 May 2024 18:16:16 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH] tools/libxs: Open /dev/xen/xenbus fds as O_CLOEXEC
Date: Sat,  4 May 2024 02:16:14 +0100
Message-Id: <20240504011614.1645851-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The header description for xs_open() goes as far as to suggest that the fd is
O_CLOEXEC, but it isn't actually.

`xl devd` has been observed leaking /dev/xen/xenbus into children.

Link: https://github.com/QubesOS/qubes-issues/issues/8292
Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: Demi Marie Obenour <demi@invisiblethingslab.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Entirely speculative patch based on a Matrix report
---
 tools/libs/store/xs.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/libs/store/xs.c b/tools/libs/store/xs.c
index 140b9a28395e..1f74fb3c44a2 100644
--- a/tools/libs/store/xs.c
+++ b/tools/libs/store/xs.c
@@ -54,6 +54,10 @@ struct xs_stored_msg {
 #include <dlfcn.h>
 #endif
 
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+#endif
+
 struct xs_handle {
 	/* Communications channel to xenstore daemon. */
 	int fd;
@@ -227,7 +231,7 @@ static int get_socket(const char *connect_to)
 static int get_dev(const char *connect_to)
 {
 	/* We cannot open read-only because requests are writes */
-	return open(connect_to, O_RDWR);
+	return open(connect_to, O_RDWR|O_CLOEXEC);
 }
 
 static int all_restrict_cb(Xentoolcore__Active_Handle *ah, domid_t domid) {

base-commit: feb9158a620040846d76981acbe8ea9e2255a07b
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Sat May 04 06:11:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 06:11:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716790.1118998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s38bh-0003Tf-Vd; Sat, 04 May 2024 06:10:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716790.1118998; Sat, 04 May 2024 06:10:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s38bh-0003TY-SS; Sat, 04 May 2024 06:10:49 +0000
Received: by outflank-mailman (input) for mailman id 716790;
 Sat, 04 May 2024 06:10:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xoDN=MH=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s38bg-0003TN-4o
 for xen-devel@lists.xenproject.org; Sat, 04 May 2024 06:10:48 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0b577af3-09dd-11ef-909c-e314d9c70b13;
 Sat, 04 May 2024 08:10:45 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a598e483ad1so58335866b.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 May 2024 23:10:45 -0700 (PDT)
Received: from ?IPV6:2003:e5:8738:9a00:3771:d6c0:aec6:f5ea?
 (p200300e587389a003771d6c0aec6f5ea.dip0.t-ipconnect.de.
 [2003:e5:8738:9a00:3771:d6c0:aec6:f5ea])
 by smtp.gmail.com with ESMTPSA id
 c25-20020a170906155900b00a599a2d9a45sm1285840ejd.100.2024.05.03.23.10.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 May 2024 23:10:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b577af3-09dd-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714803045; x=1715407845; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=A9eVR4zjjIY9w/PGSyA1izYw0Cg97sL4VqSLqb3KGSk=;
        b=DzAfV0Gi5XcFoJ3DLeRG3s8IJmRU/uAToJjrgMvUgfLMFMFMqjL4Wza2PEOlUVZmad
         pGTVxiKPc7FX7gMRqVbF4uMaqwGnppGu8P/4DRslYYnU+N891JpFZ7eWiYy+UZuoH6IE
         hfl6lJgt2mw8j8NWywJBt5uyZKCnbflWIHcZAuNjurtgUAs0E1DvU81iyAM/Sa/L5BIo
         W6CSc6hiGRgaWnQpzApVFKqW3baWd6xAl5eavvUxV53mGXktehXLD7RVaAZW7EChx1Sy
         ygmD8XEUpH6kJmu58J2kezkdk4EFc7ifQwK2TlKjQUk09EZ/NBhYoLnombL6VgfZdHzO
         eddA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714803045; x=1715407845;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=A9eVR4zjjIY9w/PGSyA1izYw0Cg97sL4VqSLqb3KGSk=;
        b=mRps+No7mhG5vLkGCUgoWGmig0if+viUExE/U20BslfDfAb8kZT2CMQb3KxrepDArx
         3y9dXsZ6Jl4NzIwSBeu5aFiF51CbeZ0LFeY9tB2ViTsrmVxoAz6PvgiL/PLWLlfhhLmb
         5/Nn3e8BFmT83K+TALOfKqZKnCpfndksjUFmXvG2Ifh2fzacfv0D0iSZ04uTOlBycP7g
         sxGy5QaOmqAbFPG5dROCo9/2jvHHKuzCUWgqHtQjqycWgLR7GMdxJGgz3IDYj95oiwoP
         0e0043lx4kbAwxofoi4kQLDiVhSjWiAZzFh37WkjuJL1sdp+kYVIQzrAljnMXPTtAm/g
         Dwfg==
X-Forwarded-Encrypted: i=1; AJvYcCVN3hc2JinrzjVUIHsrw/28+N9k0qEu3ZIGa0LJN95f3utcfcMXmGHYHxN8i+FFsgbLIa7CTIaTtQMbH3rW7KD86gM5vnvcrdnKj+NaBNw=
X-Gm-Message-State: AOJu0YxJzvuPACir6UpQJjrmcI//5yrrbHwxnGNg8STWZaMntWyhtLeY
	NyJmpXNmJmMw+9maOKvznHZWAstzE6PQRlBhEk4VUNWkoYgJ6I7YfhlY0Wd7m2I=
X-Google-Smtp-Source: AGHT+IE4kQTNWDOqbQEf0p1ZwrWBtTOiHje/SA4hzjOZkyx3uzQ2WsaOTTwKwVInTsqBxPEe3laobg==
X-Received: by 2002:a17:906:2b09:b0:a51:d23e:c53 with SMTP id a9-20020a1709062b0900b00a51d23e0c53mr3108363ejg.31.1714803045058;
        Fri, 03 May 2024 23:10:45 -0700 (PDT)
Message-ID: <db1b1151-ab02-4fc0-b28d-2a28e5fcd78f@suse.com>
Date: Sat, 4 May 2024 08:10:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/libxs: Open /dev/xen/xenbus fds as O_CLOEXEC
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Demi Marie Obenour <demi@invisiblethingslab.com>,
 Anthony PERARD <anthony@xenproject.org>,
 =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
References: <20240504011614.1645851-1-andrew.cooper3@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240504011614.1645851-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 04.05.24 03:16, Andrew Cooper wrote:
> The header description for xs_open() goes as far as to suggest that the fd is
> O_CLOEXEC, but it isn't actually.
> 
> `xl devd` has been observed leaking /dev/xen/xenbus into children.
> 
> Link: https://github.com/QubesOS/qubes-issues/issues/8292
> Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

With the style breakage below fixed:

Reviewed-by: Juergen Gross <jgross@suse.com>

> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: Demi Marie Obenour <demi@invisiblethingslab.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> Entirely speculative patch based on a Matrix report
> ---
>   tools/libs/store/xs.c | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libs/store/xs.c b/tools/libs/store/xs.c
> index 140b9a28395e..1f74fb3c44a2 100644
> --- a/tools/libs/store/xs.c
> +++ b/tools/libs/store/xs.c
> @@ -54,6 +54,10 @@ struct xs_stored_msg {
>   #include <dlfcn.h>
>   #endif
>   
> +#ifndef O_CLOEXEC
> +#define O_CLOEXEC 0
> +#endif
> +
>   struct xs_handle {
>   	/* Communications channel to xenstore daemon. */
>   	int fd;
> @@ -227,7 +231,7 @@ static int get_socket(const char *connect_to)
>   static int get_dev(const char *connect_to)
>   {
>   	/* We cannot open read-only because requests are writes */
> -	return open(connect_to, O_RDWR);
> +	return open(connect_to, O_RDWR|O_CLOEXEC);

Nit: spaces around the "|", please.


Juergen

>   }
>   
>   static int all_restrict_cb(Xentoolcore__Active_Handle *ah, domid_t domid) {
> 
> base-commit: feb9158a620040846d76981acbe8ea9e2255a07b



From xen-devel-bounces@lists.xenproject.org Sat May 04 06:37:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 06:37:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716798.1119008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s391G-0006nq-Tm; Sat, 04 May 2024 06:37:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716798.1119008; Sat, 04 May 2024 06:37:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s391G-0006nj-Qx; Sat, 04 May 2024 06:37:14 +0000
Received: by outflank-mailman (input) for mailman id 716798;
 Sat, 04 May 2024 06:37:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s391F-0006nZ-0n; Sat, 04 May 2024 06:37:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s391E-0007kz-LZ; Sat, 04 May 2024 06:37:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s391E-0006P3-7n; Sat, 04 May 2024 06:37:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s391E-0008HG-7B; Sat, 04 May 2024 06:37:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rL7LeP42V/WJIfED+Ewc+ue6+ZlU8Yc6iizMSLa/di8=; b=SniHDcPffNGYXll79wNFJJXICq
	0dtoyUWKEeUkNDxDn8RQ3bQ8KDKUR/JJg8NZQTk2eo3F98l1GxXFTVsImfrO04uO3fLowp2j63v4o
	qn3zB6Jyjnqvt0wLZJnJrp3YyKkFA8pKD4QDMKR9gZcVrDiAVC/lRnAOWFuQqLM+AJAM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185914-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185914: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
X-Osstest-Versions-That:
    xen=feb9158a620040846d76981acbe8ea9e2255a07b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 May 2024 06:37:12 +0000

flight 185914 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185914/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185906
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185906
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185906
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185906
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185906
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185906
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb
baseline version:
 xen                  feb9158a620040846d76981acbe8ea9e2255a07b

Last test of basis   185906  2024-05-03 01:53:38 Z    1 days
Testing same since   185914  2024-05-03 20:39:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   feb9158a62..f95cd010cb  f95cd010cbf0914154a0c2775c979d9158b1a3cb -> master


From xen-devel-bounces@lists.xenproject.org Sat May 04 06:44:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 06:44:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716803.1119019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s397k-00006z-La; Sat, 04 May 2024 06:43:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716803.1119019; Sat, 04 May 2024 06:43:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s397k-00006s-Io; Sat, 04 May 2024 06:43:56 +0000
Received: by outflank-mailman (input) for mailman id 716803;
 Sat, 04 May 2024 06:43:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s397j-00006i-8s; Sat, 04 May 2024 06:43:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s397j-0007r2-0V; Sat, 04 May 2024 06:43:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s397i-0006YL-F9; Sat, 04 May 2024 06:43:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s397i-000839-Eg; Sat, 04 May 2024 06:43:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cUNhhEjzLx0xMghPaZAZXnhS8P/JrJomyQeE6OgO/w0=; b=G6ke6MBw7BKj5wXgrnUY0nhORo
	7ppCBbF9Sk1Sr/q6xMWmxG3AcjGJOwSRV35sCcXBkkW9jRKjPV6HmxtduKhYKvtOP0Wh1kAqD1Pyj
	8XbbHKVfr/B605Ec+7zwj9rgKdyq/9obPo0EN9QSlXohpFxCCFt+FHhb++RwbWAZ+B6U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185916-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185916: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=24fa360857ce9e181cce0e5dad44cf0810119ff4
X-Osstest-Versions-That:
    ovmf=248aa153f65866f46b5370ac2ef7dfaf3af72480
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 May 2024 06:43:54 +0000

flight 185916 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185916/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 24fa360857ce9e181cce0e5dad44cf0810119ff4
baseline version:
 ovmf                 248aa153f65866f46b5370ac2ef7dfaf3af72480

Last test of basis   185904  2024-05-03 00:44:42 Z    1 days
Testing same since   185916  2024-05-04 04:13:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jeff Brasen <jbrasen@nvidia.com>
  Nickle Wang <nicklew@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   248aa153f6..24fa360857  24fa360857ce9e181cce0e5dad44cf0810119ff4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat May 04 11:28:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 11:28:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716857.1119041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3DZD-00018n-VA; Sat, 04 May 2024 11:28:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716857.1119041; Sat, 04 May 2024 11:28:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3DZD-00018f-Pv; Sat, 04 May 2024 11:28:35 +0000
Received: by outflank-mailman (input) for mailman id 716857;
 Sat, 04 May 2024 11:28:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3DZB-00018V-Uz; Sat, 04 May 2024 11:28:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3DZB-0005Jg-MR; Sat, 04 May 2024 11:28:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3DZB-0000Pe-9W; Sat, 04 May 2024 11:28:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3DZB-00014j-8R; Sat, 04 May 2024 11:28:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FvqhBxE4tN1swcbsPzBJMhveWkJAW7ySJaiNJjaR7Vc=; b=q3A2QQvVgjPRNd3S2LvP2Uf5pv
	n4cEm2dkdZPK3ohodNzkPvYM34XrBSiutI5rXCexuu/FmvP+OOciahZhn6BvnzRrPgnWf+WPX3fZI
	iM0+713xbZwLtP2/2FVSeHle5xM9cM2m9KaAga5+aUz2eN/kONeIcdATitzATJ0FqYo8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185915-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185915: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7367539ad4b0f8f9b396baf02110962333719a48
X-Osstest-Versions-That:
    linux=b947cc5bf6d793101135265352e205aeb30b54f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 May 2024 11:28:33 +0000

flight 185915 linux-linus real [real]
flight 185918 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185915/
http://logs.test-lab.xenproject.org/osstest/logs/185918/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl           8 xen-boot            fail pass in 185918-retest
 test-armhf-armhf-examine      8 reboot              fail pass in 185918-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185870

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 185918 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 185918 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185870
 test-armhf-armhf-libvirt      8 xen-boot                     fail  like 185870
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 185870
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185870
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185870
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7367539ad4b0f8f9b396baf02110962333719a48
baseline version:
 linux                b947cc5bf6d793101135265352e205aeb30b54f0

Last test of basis   185870  2024-04-29 17:43:51 Z    4 days
Failing since        185888  2024-04-30 20:31:50 Z    3 days    8 attempts
Testing same since   185915  2024-05-04 01:14:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Potapenko <glider@google.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Amadeusz Sławiński <amadeuszx.slawinski@linux.intel.com>
  Aman Dhoot <amandhoot12@gmail.com>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Protopopov <aspsk@isovalent.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arvind Yadav <arvind.yadav@amd.com>
  Asbjørn Sloth Tønnesen <ast@fiberby.net>
  Audra Mitchell <audra@redhat.com>
  Aurabindo Pillai <aurabindo.pillai@amd.com>
  Barry Song <v-songbaohua@oppo.com>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Björn Töpel <bjorn@kernel.org>
  Björn Töpel <bjorn@rivosinc.com>
  Bui Quang Minh <minhquangbui99@gmail.com>
  Cezary Rojewski <cezary.rojewski@intel.com>
  Chen Yu <yu.c.chen@intel.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Christian König <christian.koenig@amd.com>
  Christoph Paasch <cpaasch@apple.com>
  Chuck Lever <chuck.lever@oracle.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Dave Airlie <airlied@redhat.com>
  Dave Jiang <dave.jiang@intel.com>
  David Bauer <mail@david-bauer.net>
  David Howells <dhowells@redhat.com>
  David Lechner <dlechner@baylibre.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Derek Fang <derek.fang@realtek.com>
  Devyn Liu <liudingyuan@huawei.com>
  Dima Ruinskiy <dima.ruinskiy@intel.com>
  Dominique Martinet <dominique.martinet@atmark-techno.com>
  Donald Hunter <donald.hunter@gmail.com>
  Doug Berger <opendmb@gmail.com>
  end.to.start <end.to.start@mail.ru>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gabe Teeger <gabe.teeger@amd.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  George Shen <george.shen@amd.com>
  Guillaume Nault <gnault@redhat.com>
  Hannes Reinecke <hare@kernel.org>
  Hans de Goede <hdegoede@redhat.com>
  Harald Freudenberger <freude@linux.ibm.com>
  Harry Wentland <harry.wentland@amd.com>
  Heiko Carstens <hca@linux.ibm.com>
  Hersen Wu <hersenxs.wu@amd.com>
  Ian Forbes <ian.forbes@broadcom.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Ivan Babrou <ivan@cloudflare.com>
  Jack Yu <jack.yu@realtek.com>
  Jakub Kicinski <kuba@kernel.org>
  James Chapman <jchapman@katalix.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  Jani Nikula <jani.nikula@intel.com>
  Jason Xing <kernelxing@tencent.com>
  Jeffrey Altman <jaltman@auristor.com>
  Jens Axboe <axboe@kernel.dk>
  Jens Remus <jremus@linux.ibm.com>
  Jerome Brunet <jbrunet@baylibre.com>
  Jesper Dangaard Brouer <hawk@kernel.org>
  Jirong Feng <jirong.feng@easystack.cn>
  Joao Paulo Goncalves <joao.goncalves@toradex.com>
  John Garry <john.g.garry@oracle.com>
  Josef Bacik <josef@toxicpanda.com>
  Juergen Gross <jgross@suse.com>
  Kailang Yang <kailang@realtek.com>
  Keith Busch <kbusch@kernel.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Lancelot SIX <lancelot.six@amd.com>
  Leo Ma <hanghong.ma@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Lucas De Marchi <lucas.demarchi@intel.com>
  Lyude Paul <lyude@redhat.com>
  Mans Rullgard <mans@mansr.com>
  Marc Dionne <marc.dionne@auristor.com>
  Marek Behún <kabel@kernel.org>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <martin.lau@kernel.org>
  Matt Coster <matt.coster@imgtec.com>
  Matthew Auld <matthew.auld@intel.com>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Maurizio Lombardi <mlombard@redhat.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Max Filippov <jcmvbkbc@gmail.com>
  Meenakshikumar Somasundaram <meenakshikumar.somasundaram@amd.com>
  Mukul Joshi <mukul.joshi@amd.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nikolay Aleksandrov <razor@blackwall.org>
  Nilay Shroff <nilay@linux.ibm.com>
  Olga Kornievskaia <kolga@netapp.com>
  Oliver Upton <oliver.upton@linux.dev>
  Oswald Buddenhagen <oswald.buddenhagen@gmx.de>
  Paolo Abeni <pabeni@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Ujfalusi <peter.ujfalusi@gmail.com>
  Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
  Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
  Puranjay Mohan <puranjay12@gmail.com>
  Puranjay Mohan <puranjay@kernel.org>
  Qu Wenruo <wqu@suse.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
  Richard Fitzgerald <rf@opensource.cirrus.com>
  Richard Gobert <richardbgobert@gmail.com>
  Robert Richter <rrichter@amd.com>
  Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
  Sagi Grimberg <sagi@grimberg.me>
  Sameer Pujar <spujar@nvidia.com>
  Sasha Neftin <sasha.neftin@intel.com>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Shashank Sharma <shashank.sharma@amd.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Simon Trimmer <simont@opensource.cirrus.com>
  Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislav Fomichev <sdf@google.com>
  Stefan Binding <sbinding@opensource.cirrus.com>
  Sung Joon Kim <sungjoon.kim@amd.com>
  Sven Schnelle <svens@linux.ibm.com>
  Swapnil Patel <swapnil.patel@amd.com>
  Takashi Iwai <tiwai@suse.de>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tejun Heo <tj@kernel.org>
  Thanassis Avgerinos <thanassis.avgerinos@gmail.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Zimmermann <tzimmermann@suse.de>
  Thorsten Blum <thorsten.blum@toblux.com>
  Toke Høiland-Jørgensen <toke@redhat.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Uday Shankar <ushankar@purestorage.com>
  Vitaly Lifshits <vitaly.lifshits@intel.com>
  Wayne Lin <wayne.lin@amd.com>
  Xin Long <lucien.xin@gmail.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Kuohai <xukuohai@huawei.com>
  Yi Zhang <yi.zhang@redhat.com>
  Yunsheng Lin <linyunsheng@huawei.com>
  Zack Rusin <zack.rusin@broadcom.com>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   b947cc5bf6d79..7367539ad4b0f  7367539ad4b0f8f9b396baf02110962333719a48 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat May 04 15:32:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 May 2024 15:32:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.716920.1119052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3HN3-00058x-G9; Sat, 04 May 2024 15:32:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 716920.1119052; Sat, 04 May 2024 15:32:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3HN3-00058q-Aa; Sat, 04 May 2024 15:32:17 +0000
Received: by outflank-mailman (input) for mailman id 716920;
 Sat, 04 May 2024 15:32:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3HN2-00058g-KT; Sat, 04 May 2024 15:32:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3HN2-0001Oz-AK; Sat, 04 May 2024 15:32:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3HN2-000698-0a; Sat, 04 May 2024 15:32:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3HN2-0002Kh-0A; Sat, 04 May 2024 15:32:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qrE2nGiAEHBZ9nI7yPmRg6UR2nOOi8GiVMZWNxFPwlk=; b=E2XzCK/RFwMzWuH335hCzQ9gQe
	efS4twit1CxyLVdEXvnwe2NYPqCCWRZDxTT/+kryGr8tYXmypPh6aJ2PI/AC7s8evw8/A4rv25sab
	YVANBiyPnE9gvo2P3lVGyRzOaOsvjX58WZ3Fj/K9MrchUhyu5I7f5yS89UTXGMrOk1VA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185917-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185917: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
X-Osstest-Versions-That:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 May 2024 15:32:16 +0000

flight 185917 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185917/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot                   fail pass in 185914

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185914 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185914 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185914
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185914
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185914
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185914
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185914
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185914
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb
baseline version:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb

Last test of basis   185917  2024-05-04 06:41:35 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun May 05 08:43:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 May 2024 08:43:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717049.1119062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3XSS-0006Fn-8a; Sun, 05 May 2024 08:42:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717049.1119062; Sun, 05 May 2024 08:42:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3XSS-0006Fg-3q; Sun, 05 May 2024 08:42:56 +0000
Received: by outflank-mailman (input) for mailman id 717049;
 Sun, 05 May 2024 08:42:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3XSQ-0006FW-OD; Sun, 05 May 2024 08:42:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3XSQ-0004gR-IG; Sun, 05 May 2024 08:42:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3XSQ-0007ib-38; Sun, 05 May 2024 08:42:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3XSQ-0000c3-0m; Sun, 05 May 2024 08:42:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lDX0vC4ghtgXMaIuYohpECz/vYTK5rClWprSTmNHQ7A=; b=hgvcfNi6RE3ULsMtotDC8x59l8
	H7Ze/n4H7eB85z6NDabuj54SaF+79/ZQ5ScVYdupkiqsQ/Fms5CiJOkOsfVW2Wts2PxLzfUXNrr9J
	FYBpdUHV2tiG4IOE44lFLDx0S80QIAY/VVA/IsHag6N7qKGqLRl9GBU69m+n0V0WvOZ8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185919-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185919: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:xen-boot:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
X-Osstest-Versions-That:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 May 2024 08:42:54 +0000

flight 185919 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185919/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot                   fail pass in 185917
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185917

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 185917 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 185917 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185917
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185917
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185917
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185917
 test-armhf-armhf-xl-credit2   8 xen-boot                     fail  like 185917
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185917
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185917
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb
baseline version:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb

Last test of basis   185919  2024-05-05 01:57:20 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun May 05 22:13:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 May 2024 22:13:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717139.1119076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3k6V-0006YN-1O; Sun, 05 May 2024 22:13:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717139.1119076; Sun, 05 May 2024 22:13:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3k6U-0006YD-Sr; Sun, 05 May 2024 22:13:06 +0000
Received: by outflank-mailman (input) for mailman id 717139;
 Sun, 05 May 2024 22:13:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F67X=MI=raptorengineering.com=sanastasio@srs-se1.protection.inumbo.net>)
 id 1s3k6T-0006Y7-VP
 for xen-devel@lists.xenproject.org; Sun, 05 May 2024 22:13:06 +0000
Received: from raptorengineering.com (mail.raptorengineering.com
 [23.155.224.40]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a2994a59-0b2c-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 00:13:03 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id D00728285375;
 Sun,  5 May 2024 17:12:59 -0500 (CDT)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id pbSFnbYpwGOm; Sun,  5 May 2024 17:12:59 -0500 (CDT)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 0BDAE8285609;
 Sun,  5 May 2024 17:12:59 -0500 (CDT)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id 5QPnu0esl4hU; Sun,  5 May 2024 17:12:58 -0500 (CDT)
Received: from [10.11.0.2] (5.edge.rptsys.com [23.155.224.38])
 by mail.rptsys.com (Postfix) with ESMTPSA id 7E8DD8285375;
 Sun,  5 May 2024 17:12:58 -0500 (CDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2994a59-0b2c-11ef-909c-e314d9c70b13
DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 0BDAE8285609
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD;
	t=1714947179; bh=O29LkBtMJH19pAXLgx1JTabdFZARX87JoGWKXY6FNKk=;
	h=Message-ID:Date:MIME-Version:To:From;
	b=beP/xJkl+Bq66svUq/KnsnuckbhTdELmzYXz9mcNxluO/F1IdwRAH8lOOun5QqBOO
	 0z3YkQQroNJ2IJSjDZexZ2+iW7ylJ2hIxaKLXCuFRZiDNZlCiRLKQPrNMwiK1vd+N7
	 FnQYofFcM8MS43whBXR2vOKk9M3rDjxiUXbRik4k=
X-Virus-Scanned: amavisd-new at rptsys.com
Message-ID: <212940fc-4e3f-4763-934f-c8306d44ad7e@raptorengineering.com>
Date: Sun, 5 May 2024 17:12:57 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] ppc/riscv: fix arch_acquire_resource_check()
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240430153423.80875-1-roger.pau@citrix.com>
Content-Language: en-US
From: Shawn Anastasio <sanastasio@raptorengineering.com>
In-Reply-To: <20240430153423.80875-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Roger,

On 4/30/24 10:34 AM, Roger Pau Monne wrote:
> None of the implementations support set_foreign_p2m_entry() yet, neithe=
r they
> have a p2m walk in domain_relinquish_resources() in order to remove the=
 foreign
> mappings from the p2m and thus drop the extra refcounts.
>=20
> Adjust the arch helpers to return false and introduce a comment that cl=
early
> states it is not only taking extra refcounts that's needed, but also dr=
opping
> them on domain teardown.
>=20
> Fixes: 4988704e00d8 ('xen/riscv: introduce p2m.h')
> Fixes: 4a2f68f90930 ('xen/ppc: Define minimal stub headers required for=
 full build')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---

This makes sense to me. This stub implementation was definitely an
oversight on my part.

Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

Thanks,
Shawn


From xen-devel-bounces@lists.xenproject.org Mon May 06 00:34:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 00:34:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717146.1119106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIz-0006vP-3F; Mon, 06 May 2024 00:34:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717146.1119106; Mon, 06 May 2024 00:34:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIy-0006vI-Vf; Mon, 06 May 2024 00:34:08 +0000
Received: by outflank-mailman (input) for mailman id 717146;
 Mon, 06 May 2024 00:34:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MOxl=MJ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s3mIx-0006Sb-JW
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 00:34:07 +0000
Received: from wfhigh8-smtp.messagingengine.com
 (wfhigh8-smtp.messagingengine.com [64.147.123.159])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 573fe0fc-0b40-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 02:34:05 +0200 (CEST)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailfhigh.west.internal (Postfix) with ESMTP id 5B0331800118;
 Sun,  5 May 2024 20:34:03 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute4.internal (MEProxy); Sun, 05 May 2024 20:34:03 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
 5 May 2024 20:34:01 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 573fe0fc-0b40-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1714955642; x=1715042042; bh=cI21Y0/d9U
	N+EtZKm1PiuxpPhL0dJGUf3Jiak+yaf+w=; b=P+hKOONZHAC1/7jUPq0NLEncLE
	9SsVIpe3QiQt61zgMSKs9QV+mE2REkm3a7pQryn+NXuz67jWIqNhHhZsrXZUKi5i
	8MwdNxDHS3aw78bjbvz0D3gUE20Pr349MrphKrylyDs+jlgmWeddFQZvheqZcIBW
	ly2ILGNv58tXYAj9R/7Y8uv9lDVcUVLSBEybzalSuOtXoOkVG7+CNgqdMb21pFqj
	rbWH1U8urbGsZIBjCXgoUJ14t7CuW1F83UwdbaGp+95taQldmVi/G/X4WK10WjtO
	XOl9mFvJlWMxloFpbbQjRAWjt9Bma5e/CFd+ePm8Q1DU4Mgwr7qgDAofXRGQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1714955642; x=
	1715042042; bh=cI21Y0/d9UN+EtZKm1PiuxpPhL0dJGUf3Jiak+yaf+w=; b=V
	Ru6jrVIsSuUx0hCMWF5iRIb7YnWBTqdy0fAn10pVqZC2QcZ/Y+a8eZVEqIFiVXBQ
	Rolomz3yMdsVEa+9z/exwa2ubgASoeUx/jOQ64NkdtDd6BU2RwMfRr/E7iyH06ZT
	ltXUAfd9pEK1jIU/4oigfZGnFS68gcMciVtjrBLXioQIHTe3EpqeZVBe9l9Xo4aS
	E9RfcVzaw6Y/zZvhExSwerGQ2uY4kRXQIZkfmHSTU9+trDv1UtKz64SjQFAMeouq
	wUHSA3GXYFIe+KIxfCAxU1/Dyqqya7RQbQPWcH3hH3y+FksN3AqaoiNGjyvhiYxW
	203x17lIOQSoyBxu+/UrQ==
X-ME-Sender: <xms:eiU4Zg4QFAukm_mgo5fW5KSGrbfba2sxzY0wfMElbDQU85uAo2czBg>
    <xme:eiU4Zh6G1fZ9Yj7F5Gu2jMhq_8tUrZnvdDPEUI_l8QFo7R0N5yjQv0rLJgBYMg7gp
    RjEdwSt_H0bNA>
X-ME-Received: <xmr:eiU4ZvdmcY57yvPgoVjwt19eLRzPvaZkuxonZTIh3PNrpIl-uU5NXqu2FreUeUU2t1hTj1qNz38zOvhrxaY2BYZOoD8UajtI0UQmCU6--D3B13MCR_U>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvhedgfeejucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:eiU4ZlItYuu500441k-R5fXA4RRYrJg_Ew7cYiYufv2uRdvb9UbBqQ>
    <xmx:eiU4ZkK8YI0jhVrdCHcpJbnfR0teg9ACxS964yFXBZlKts3roPHbpg>
    <xmx:eiU4ZmwQEmWROEBvOSGehVnmezo1cuM1TzTdpWyhg5gTY5VrCoEsPg>
    <xmx:eiU4ZoKXn4HmboPfH52O22oYa8NsION4gsTFpKr3bv1rDSGlVyHaDQ>
    <xmx:eiU4Zui9JKZeKLq7eo5PRgZQhL2hw91FQl__z_rloBJSHSqgDfm_N0Q9>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH v3 3/3] Do not access /dev/mem in MSI-X PCI passthrough on Xen
Date: Mon,  6 May 2024 02:33:22 +0200
Message-ID: <ebeb8c419feedad9fe0e9f39d3ed3a9ff0f4c49b.1714955598.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
References: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The /dev/mem is used for two purposes:
 - reading PCI_MSIX_ENTRY_CTRL_MASKBIT
 - reading Pending Bit Array (PBA)

The first one was originally done because when Xen did not send all
vector ctrl writes to the device model, so QEMU might have outdated old
register value. If Xen is new enough, this has been changed, so QEMU can
now use its cached value of the register instead. Detect the "new
enough" based on XENFEAT_dm_msix_all_writes bit in XENVER_get_features.

The Pending Bit Array (PBA) handling is for the case where it lives on
the same page as the MSI-X table itself. Xen has been extended to handle
this case too (as well as other registers that may live on those pages),
so QEMU handling is not necessary anymore.

Additionally, reading from /dev/mem is trapped and emulated by Xen, so
QEMU doesn't see real values anyway. And if it did, this method is prone
to race conditions. Removing /dev/mem access is useful to work within
stubdomain (avoids emulated reads and potential races), and necessary
when dom0 kernel runs in lockdown mode (where /dev/mem is unavailable at
all).

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v2:
- Make change conditional on new Xen version (tested via
  XENFEAT_dm_msix_all_writes)
- add few comments
---
 hw/xen/xen_pt_msi.c | 94 ++++++++++++++++++++++++++++------------------
 1 file changed, 59 insertions(+), 35 deletions(-)

diff --git a/hw/xen/xen_pt_msi.c b/hw/xen/xen_pt_msi.c
index 09cca4e..836cc9c 100644
--- a/hw/xen/xen_pt_msi.c
+++ b/hw/xen/xen_pt_msi.c
@@ -460,15 +460,23 @@ static void pci_msix_write(void *opaque, hwaddr addr,
         entry->updated = true;
     } else if (msix->enabled && entry->updated &&
                !(val & PCI_MSIX_ENTRY_CTRL_MASKBIT)) {
-        const volatile uint32_t *vec_ctrl;
-
         /*
-         * If Xen intercepts the mask bit access, entry->vec_ctrl may not be
-         * up-to-date. Read from hardware directly.
+         * Reading mask bit from hardware directly is needed on older Xen only.
          */
-        vec_ctrl = s->msix->phys_iomem_base + entry_nr * PCI_MSIX_ENTRY_SIZE
-            + PCI_MSIX_ENTRY_VECTOR_CTRL;
-        xen_pt_msix_update_one(s, entry_nr, *vec_ctrl);
+        if (s->msix->phys_iomem_base) {
+            /* Memory mapped registers */
+            const volatile uint32_t *vec_ctrl;
+
+            /*
+             * If Xen intercepts the mask bit access, entry->vec_ctrl may not be
+             * up-to-date. Read from hardware directly.
+             */
+            vec_ctrl = s->msix->phys_iomem_base + entry_nr * PCI_MSIX_ENTRY_SIZE
+                + PCI_MSIX_ENTRY_VECTOR_CTRL;
+            xen_pt_msix_update_one(s, entry_nr, *vec_ctrl);
+        } else {
+            xen_pt_msix_update_one(s, entry_nr, entry->latch(VECTOR_CTRL));
+        }
     }
 
     set_entry_value(entry, offset, val);
@@ -493,7 +501,12 @@ static uint64_t pci_msix_read(void *opaque, hwaddr addr,
         return get_entry_value(&msix->msix_entry[entry_nr], offset);
     } else {
         /* Pending Bit Array (PBA) */
-        return *(uint32_t *)(msix->phys_iomem_base + addr);
+        if (s->msix->phys_iomem_base) {
+            return *(uint32_t *)(msix->phys_iomem_base + addr);
+        }
+        XEN_PT_LOG(&s->dev, "reading PBA, addr 0x%lx, offset 0x%lx\n",
+                   addr, addr - msix->total_entries * PCI_MSIX_ENTRY_SIZE);
+        return 0xFFFFFFFF;
     }
 }
 
@@ -528,8 +541,8 @@ int xen_pt_msix_init(XenPCIPassthroughState *s, uint32_t base)
     uint32_t table_off = 0;
     int i, total_entries, bar_index;
     XenHostPCIDevice *hd = &s->real_device;
+    xen_feature_info_t xc_version_info = { 0 };
     PCIDevice *d = &s->dev;
-    int fd = -1;
     XenPTMSIX *msix = NULL;
     int rc = 0;
 
@@ -543,6 +556,10 @@ int xen_pt_msix_init(XenPCIPassthroughState *s, uint32_t base)
         return -1;
     }
 
+    if (xc_version(xen_xc, XENVER_get_features, &xc_version_info) < 0) {
+        return -1;
+    }
+
     rc = xen_host_pci_get_word(hd, base + PCI_MSIX_FLAGS, &control);
     if (rc) {
         XEN_PT_ERR(d, "Failed to read PCI_MSIX_FLAGS field\n");
@@ -576,33 +593,40 @@ int xen_pt_msix_init(XenPCIPassthroughState *s, uint32_t base)
     msix->table_base = s->real_device.io_regions[bar_index].base_addr;
     XEN_PT_LOG(d, "get MSI-X table BAR base 0x%"PRIx64"\n", msix->table_base);
 
-    fd = open("/dev/mem", O_RDWR);
-    if (fd == -1) {
-        rc = -errno;
-        XEN_PT_ERR(d, "Can't open /dev/mem: %s\n", strerror(errno));
-        goto error_out;
-    }
-    XEN_PT_LOG(d, "table_off = 0x%x, total_entries = %d\n",
-               table_off, total_entries);
-    msix->table_offset_adjust = table_off & 0x0fff;
-    msix->phys_iomem_base =
-        mmap(NULL,
-             total_entries * PCI_MSIX_ENTRY_SIZE + msix->table_offset_adjust,
-             PROT_READ,
-             MAP_SHARED | MAP_LOCKED,
-             fd,
-             msix->table_base + table_off - msix->table_offset_adjust);
-    close(fd);
-    if (msix->phys_iomem_base == MAP_FAILED) {
-        rc = -errno;
-        XEN_PT_ERR(d, "Can't map physical MSI-X table: %s\n", strerror(errno));
-        goto error_out;
-    }
-    msix->phys_iomem_base = (char *)msix->phys_iomem_base
-        + msix->table_offset_adjust;
+    /* Accessing /dev/mem is needed only on older Xen. */
+    if (!(xc_version_info.submap & (1U << XENFEAT_dm_msix_all_writes))) {
+        int fd = -1;
+
+        fd = open("/dev/mem", O_RDWR);
+        if (fd == -1) {
+            rc = -errno;
+            XEN_PT_ERR(d, "Can't open /dev/mem: %s\n", strerror(errno));
+            goto error_out;
+        }
+        XEN_PT_LOG(d, "table_off = 0x%x, total_entries = %d\n",
+                   table_off, total_entries);
+        msix->table_offset_adjust = table_off & 0x0fff;
+        msix->phys_iomem_base =
+            mmap(NULL,
+                 total_entries * PCI_MSIX_ENTRY_SIZE
+                 + msix->table_offset_adjust,
+                 PROT_READ,
+                 MAP_SHARED | MAP_LOCKED,
+                 fd,
+                 msix->table_base + table_off - msix->table_offset_adjust);
+        close(fd);
+        if (msix->phys_iomem_base == MAP_FAILED) {
+            rc = -errno;
+            XEN_PT_ERR(d, "Can't map physical MSI-X table: %s\n",
+                       strerror(errno));
+            goto error_out;
+        }
+        msix->phys_iomem_base = (char *)msix->phys_iomem_base
+            + msix->table_offset_adjust;
 
-    XEN_PT_LOG(d, "mapping physical MSI-X table to %p\n",
-               msix->phys_iomem_base);
+        XEN_PT_LOG(d, "mapping physical MSI-X table to %p\n",
+                   msix->phys_iomem_base);
+    }
 
     memory_region_add_subregion_overlap(&s->bar[bar_index], table_off,
                                         &msix->mmio,
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Mon May 06 00:34:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 00:34:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717145.1119096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIx-0006hA-SC; Mon, 06 May 2024 00:34:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717145.1119096; Mon, 06 May 2024 00:34:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIx-0006h2-OE; Mon, 06 May 2024 00:34:07 +0000
Received: by outflank-mailman (input) for mailman id 717145;
 Mon, 06 May 2024 00:34:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MOxl=MJ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s3mIw-0006Sc-VW
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 00:34:06 +0000
Received: from wfout5-smtp.messagingengine.com
 (wfout5-smtp.messagingengine.com [64.147.123.148])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 55575c70-0b40-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 02:34:02 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id CA52B1C0015F;
 Sun,  5 May 2024 20:34:00 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Sun, 05 May 2024 20:34:01 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
 5 May 2024 20:33:58 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55575c70-0b40-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1714955640; x=1715042040; bh=Qs8kJgBcfw
	yVOBoQxWcf1KBTRIcqfqizvEFlALXjf5I=; b=OGX8DgO8lNe7otID6bNrAnu9YY
	KjGEIZo/Ne7N+S4xYFfXvsa4W2P3LvNayRzh2sg7+5XueAYDyB5ZzKHSdEOGNpLO
	EJH8nIXcq0o/XGW9/QD6LfQRsc36Nu7p72z1/x5e1ANkSRlS+5+qICb2tM/lElpZ
	0zLT83dd9YxNCGVa8aPCiDgdcUv3xhQ6478yiwUUvMloHIZ7EyGN8O+LMlTngoYh
	mv1+SXLB8yWB9NQ+0wDskBUxc9m8fLbD/dzcii8Wgibcob1y3Bmtht8zj0mIwr21
	xxxG2cbKkL2fI65Bao2ROJ4d1KrObiI5akm1iM2FD7fuPKNX/tPS+XV3hn3w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1714955640; x=
	1715042040; bh=Qs8kJgBcfwyVOBoQxWcf1KBTRIcqfqizvEFlALXjf5I=; b=Z
	nxaS/Ij6kfcXycAu1vAl/Q0d++lt1aUh2niCpqU7QVANb6PjJx67MsB7+ficiqkE
	bbX3p0MOtwDygs3nHnhdUT2XdhO4x6/uFZ0c2ExD1JX8eHYdTa/33FwXnjpDFK11
	onHN1cMStJ9VdeMuDPTWVJW4iPM1Csd2fu2qfKpaQrbCJe7jLTA2ufuQ29jeuwg/
	FyD+2JS4fE3Q3ZhahBOJFQRRiHPRRziVnw9PYM+wMWU39JdxqErU4PlnAxIVCVRv
	y/BkSdab3Jg/Ml5RTl0vpzPeNdS0yehtvVVitSo/2BG0fv4jIiDs0llow8jovAuz
	NNOUG5p42I2kV8RkcithQ==
X-ME-Sender: <xms:dyU4ZjDZIinHH1_higeCrLB99UJHlcRB0NdbdoKtwgIau3h-TPe5gQ>
    <xme:dyU4Zpi1LfPQF67CdXSW8hMaUAPx_8GxcDDqJmEiOnwW-_NRy2Y5ABZx7hxet6Pdq
    3z-VkBU35J_6g>
X-ME-Received: <xmr:dyU4Zuk5Hcu_YoRRfyZU1iEtBGRuj5cyauISpJM2aWhWhX1WbbUYOlBTm-XZMm5lX2a3uJ4u-XDYiHxZRmLbSIMxLj-6c9ecPCrlagKAGdhYCZ5uPeY>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvhedgfeeiucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:eCU4ZlxqKHPKIjWiwJiG9fqpoWLp5QL4t2xPc81FDn7nsFg4mX3Big>
    <xmx:eCU4ZoS112h0PQEaEr87XWs51M_2YfNs0dVbf0i66Scnc0DuEYCceA>
    <xmx:eCU4ZoZ9g7_iPAEiSGZTZCu8kkGcVIUm3qTt0T3zHFLlD34wnYiN3w>
    <xmx:eCU4ZpRn6AvV4io_V7hcRo8UN4AVC-TzEJu8S8nuyuuPSCV4iTeHHQ>
    <xmx:eCU4ZtKM3j44LbYJps7jLc0H9V6UoimAubG5li6GYHOf-3cMst5N3Uyj>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH v3 2/3] Update Xen's features.h header
Date: Mon,  6 May 2024 02:33:21 +0200
Message-ID: <e167b362448dc3abc677f3b925a58c55d6158da7.1714955598.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
References: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Update it to get XENFEAT_dm_msix_all_writes for the next patch.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 include/hw/xen/interface/features.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/include/hw/xen/interface/features.h b/include/hw/xen/interface/features.h
index d2a9175..8801930 100644
--- a/include/hw/xen/interface/features.h
+++ b/include/hw/xen/interface/features.h
@@ -111,6 +111,23 @@
 #define XENFEAT_not_direct_mapped         16
 #define XENFEAT_direct_mapped             17
 
+/*
+ * Signal whether the domain is able to use the following hypercalls:
+ *
+ * VCPUOP_register_runstate_phys_area
+ * VCPUOP_register_vcpu_time_phys_area
+ */
+#define XENFEAT_runstate_phys_area        18
+#define XENFEAT_vcpu_time_phys_area       19
+
+/*
+ * If set, Xen will passthrough all MSI-X vector ctrl writes to device model,
+ * not only those unmasking an entry. This allows device model to properly keep
+ * track of the MSI-X table without having to read it from the device behind
+ * Xen's backs. This information is relevant only for device models.
+ */
+#define XENFEAT_dm_msix_all_writes        20
+
 #define XENFEAT_NR_SUBMAPS 1
 
 #endif /* __XEN_PUBLIC_FEATURES_H__ */
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Mon May 06 00:34:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 00:34:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717144.1119086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIw-0006Sp-Jy; Mon, 06 May 2024 00:34:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717144.1119086; Mon, 06 May 2024 00:34:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3mIw-0006Si-H3; Mon, 06 May 2024 00:34:06 +0000
Received: by outflank-mailman (input) for mailman id 717144;
 Mon, 06 May 2024 00:34:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MOxl=MJ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s3mIv-0006Sb-TK
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 00:34:06 +0000
Received: from wfout5-smtp.messagingengine.com
 (wfout5-smtp.messagingengine.com [64.147.123.148])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5476dad6-0b40-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 02:34:01 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id EC2831C00152;
 Sun,  5 May 2024 20:33:57 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Sun, 05 May 2024 20:33:58 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
 5 May 2024 20:33:55 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5476dad6-0b40-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1714955637; x=1715042037; bh=GzIxndmltz
	Yd7Z3Vh6Vc99wk3Tg6uvlrTPvLOMRcggc=; b=UBOUVlq0sZA/UHdiYCg5UBV7Oh
	pRBfXnQE40nTRmActFjoSYgbVXWW2VyBr3sDxVs9Um+ZNyyD38haq7ScfHfKZEFX
	kaz8y6dT/WBBvmY+xlqdpzrGVgbTLemTOowGHiu0aG1/BSufOc5Eg1a8SM5g1djr
	j+sjstYgu7sV38FIpgHLh79bFR6kemcOCgdIiHphD121Og805Y129zgpunXXalmP
	Y/QlDpezaGWvyQzMsvNnaF2ESGa6Hz/XPobVJ0jHNQ+mp6wqVWq5RI+pyFVGkT6M
	Kf09aujSJ4aN8dPSSd3M53hDw3TY8KMtXpbJG2OdLdkAxvbdhgZ6wWFOxytA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1714955637; x=
	1715042037; bh=GzIxndmltzYd7Z3Vh6Vc99wk3Tg6uvlrTPvLOMRcggc=; b=K
	ShN7G5D9D5QrxGKzG9cNmuI1x47tnvZMIFY/c/i3hoD2Jc4x4knuD6EkhM7W03LZ
	RHyt3Xugy+36+ZjYtHuxO+989zv0vUwZRi3AP7jzRqmUpVy1n4uApM0gLJZUuxt3
	Kb0Viz7NRJr5O4MRk+jSx/Y0D/VDIj55b45uRVcbeC5IuwpLSWYbk1XKiVpk70km
	wpcIfjAZRDHNuy0XioI4rIW9KOqN/vNd+2wAhCZZO+UdtUCE2K1w6daosDAM/8rx
	w/NBlBmESZO7xNDiJkxj+0Xc+1MdoTaDca2thA6+t1NfjWnm7gZkrURi7dkd93ZJ
	HZ+/K/X0hHIlsQGFNRG4g==
X-ME-Sender: <xms:dSU4ZviwT1LkRKirUO9NbStaFbILtt6mj0LKFcRMtRtdlKO3mXGuQQ>
    <xme:dSU4ZsBFgyESZyYH9Z9Ve_uR-tPAH7sWv5x_J2ZY-WYwxuYVl5lPYc2E9UsPuAsLT
    1_puPJjJAUqLA>
X-ME-Received: <xmr:dSU4ZvGK7_mo87Vc-yheInRDfounq8U8PMLsbOBDH_iERF8jiripxpp123ZQGCEzolt6i-LPiEqq_5Pit6GLRX4kFp5Q4PEy4LbAPE9AEQ9UhNnkv0A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvhedgfeeiucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:dSU4ZsQa4vqnY-k7Ra-9DijZl8rWi5rOIwRjUW8xeYA-cATrKo3nGw>
    <xmx:dSU4ZsxBRddJIqg-y0CFRpzvdLiRZTia3mRLNbrnglxVJy24o3Hczg>
    <xmx:dSU4Zi7okPLiBl1sYhesIsSJD533OUNae4rTo6uklqg6OoTnE3r0zw>
    <xmx:dSU4ZhzBUdk4xxe3ME6haUCTMQFXESHLHeW4_buhxDL7A8sn-LIgPQ>
    <xmx:dSU4ZrrQfW9tKVP04a53QyayFTZM2zIthOp_70gFQdsovibaJO-1Y5n->
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH v3 1/3] hw/xen/xen_pt: Save back data only for declared registers
Date: Mon,  6 May 2024 02:33:20 +0200
Message-ID: <aa391652c33c8a4a64db8f27ad50ccb65600b293.1714955598.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
References: <cover.f5d45e3c2fb87552abfaf80982b0b724fca2134c.1714955598.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Call pci_default_write_config() only after resolving any handlers from
XenPTRegInfo structures, and only with a value updated with those
handlers. This is important for two reasons:
1. XenPTRegInfo has ro_mask which needs to be enforced - Xen-specific
   hooks do that on their own (especially xen_pt_*_reg_write()).
2. Not setting value early allows hooks to see the old value too.

If it would be only about the first point, setting PCIDevice.wmask would
probably be sufficient, but given the second point, change those
writes.

Relevant handlers already save data back to the emulated registers
space, call the pci_default_write_config() only for its side effects.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
v3:
 - use emulated register value for pci_default_write_config() call, not
   the one for writting back to the hardware
 - greatly simplify the patch by calling pci_default_write_config() on
   the whole value
v2:
 - rewrite commit message, previous one was very misleading
 - fix loop saving register values
 - fix int overflow when calculating write mask
---
 hw/xen/xen_pt.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index 3635d1b..5f12d3c 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -311,7 +311,6 @@ static void xen_pt_pci_write_config(PCIDevice *d, uint32_t addr,
     }
 
     memory_region_transaction_begin();
-    pci_default_write_config(d, addr, val, len);
 
     /* adjust the read and write value to appropriate CFC-CFF window */
     read_val <<= (addr & 3) << 3;
@@ -397,6 +396,12 @@ static void xen_pt_pci_write_config(PCIDevice *d, uint32_t addr,
     /* need to shift back before passing them to xen_host_pci_set_block. */
     val >>= (addr & 3) << 3;
 
+    /* Call default handler for its side effects only, with value already
+     * written by specific handlers. */
+    pci_default_write_config(d, addr,
+                             pci_default_read_config(d, addr, len),
+                             len);
+
     memory_region_transaction_commit();
 
 out:
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Mon May 06 01:43:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 01:43:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717158.1119116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3nNS-0007E5-1u; Mon, 06 May 2024 01:42:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717158.1119116; Mon, 06 May 2024 01:42:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3nNR-0007Dy-Uq; Mon, 06 May 2024 01:42:49 +0000
Received: by outflank-mailman (input) for mailman id 717158;
 Mon, 06 May 2024 01:42:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3nNQ-0007Do-Gd; Mon, 06 May 2024 01:42:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3nNQ-0005pC-Ag; Mon, 06 May 2024 01:42:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3nNP-0000bE-T9; Mon, 06 May 2024 01:42:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3nNP-0005dy-SI; Mon, 06 May 2024 01:42:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cvCe1Dk9plvUiXDFIM0KHDMOuW7OkmTJvGmy2fKlJig=; b=TEbLl9YBqdAuDwuDnycioxoPPY
	f5OTw9Dndhri5Ks7bTqsWzHNQraS6grZN8AAd4r1OFLbzmfxu1nLj7kV55K/f7l09ngHZz8vdQqbk
	1KEv9Wja9DbN4Z2+xUzTliqwVv86NoubuVxsG9LZRZl4ZzHI2lCrhceBz6aCEoK2HqN8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185920-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185920: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=b9158815de525572333d0499a681459f6b075f28
X-Osstest-Versions-That:
    linux=7367539ad4b0f8f9b396baf02110962333719a48
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 01:42:47 +0000

flight 185920 linux-linus real [real]
flight 185921 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185920/
http://logs.test-lab.xenproject.org/osstest/logs/185921/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 185921-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185915
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 185921 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 185921 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185915
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185915
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185915
 test-armhf-armhf-examine      8 reboot                       fail  like 185915
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185915
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185915
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                b9158815de525572333d0499a681459f6b075f28
baseline version:
 linux                7367539ad4b0f8f9b396baf02110962333719a48

Last test of basis   185915  2024-05-04 01:14:15 Z    2 days
Testing same since   185920  2024-05-05 17:44:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Skladowski <a39.skl@gmail.com>
  Alan Stern <stern@rowland.harvard.edu>
  Alexander Usyskin <alexander.usyskin@intel.com>
  Amit Sunil Dhamne <amitsd@google.com>
  Badhri Jagan Sridharan <badhri@google.com>
  Bjorn Andersson <andersson@kernel.org>
  Chad Wagner <wagnerch42@gmail.com>
  Chris Wulff <chris.wulff@biamp.com>
  Christoph Hellwig <hch@lst.de>
  Conor Dooley <conor.dooley@microchip.com>
  Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
  Diego Roversi <diegor@tiscali.it>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Frank Oltmanns <frank@oltmanns.dev>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Hans de Goede <hdegoede@redhat.com>
  Ivan Avdeev <me@provod.works>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jim Cromie <jim.cromie@gmail.com>
  Johan Hovold <johan+linaro@kernel.org>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  LuMingYin <lumingyindetect@126.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Maxime Ripard <mripard@kernel.org>
  Nikita Ioffe <ioffe@google.com>
  Peter Colberg <peter.colberg@intel.com>
  Peter Korsgaard <peter@korsgaard.com>
  Ramona Gradinariu <ramona.bolboaca13@gmail.com>
  RD Babiera <rdbabiera@google.com>
  Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
  Stephen Boyd <sboyd@kernel.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Tomas Winkler <tomas.winkler@intel.com>
  Tze-nan Wu (吳澤南) <Tze-nan.Wu@mediatek.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vasileios Amoiridis <vassilisamir@gmail.com>
  Vicki Pfau <vi@endrift.com>
  Viken Dadhaniya <quic_vdadhani@quicinc.com>
  Wesley Cheng <quic_wcheng@quicinc.com>
  Will Deacon <will@kernel.org>
  Xu Yilun <yilun.xu@intel.com>
  Xu Yilun <yilun.xu@linux.intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   7367539ad4b0f..b9158815de525  b9158815de525572333d0499a681459f6b075f28 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon May 06 02:02:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 02:02:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717165.1119125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ngR-0002F4-Gn; Mon, 06 May 2024 02:02:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717165.1119125; Mon, 06 May 2024 02:02:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ngR-0002Ex-EA; Mon, 06 May 2024 02:02:27 +0000
Received: by outflank-mailman (input) for mailman id 717165;
 Mon, 06 May 2024 02:02:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3ngP-0002Er-Rq
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 02:02:25 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ada0973a-0b4c-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 04:02:24 +0200 (CEST)
Received: from CH0PR03CA0236.namprd03.prod.outlook.com (2603:10b6:610:e7::31)
 by DS7PR12MB8203.namprd12.prod.outlook.com (2603:10b6:8:e1::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Mon, 6 May
 2024 02:02:18 +0000
Received: from CH1PEPF0000A34C.namprd04.prod.outlook.com
 (2603:10b6:610:e7:cafe::39) by CH0PR03CA0236.outlook.office365.com
 (2603:10b6:610:e7::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Mon, 6 May 2024 02:02:18 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000A34C.mail.protection.outlook.com (10.167.244.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 02:02:18 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 21:01:52 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 5 May 2024 21:01:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ada0973a-0b4c-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n7p0sa8JZr9MVBCGnuDn4lD19SEzb0ll7CmD5VVEwsD5hqYEWbggQ3jUQI18/FYfNQVqFBWnGMSanGkxC3PCo2hqe7kGsw2S0QtJqt0xCaZyavhFdwzS3UxN6yyKhKWTM3PitEJCY0ytJtHDpNo7AERx11yjJNweaIgx++B7McyzAJqLQENpmLXVQ8l4QxEfhdoQFLab3R2rPZUvaoC9ptPW7OEYkNTPpt0kNbLi1pXkFBGK7FiURw9AzoBNrNnqKOYmSoGs1c5eRK2pDKDSH8irlF1BWK5fhlnfDYoJAGv/eT5DwzUb5/zNhWficcM7ogS3VorlhMMUz2rD5R9lTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=rqkhaQ3j8+Kjb2g9lCV7+HCsqiGSl3S6Y6XiGdv9440=;
 b=nAVu5d5jowJw/6RpDK88drrTP6UU9UXeqqqVZplNJRLh3r30e1ZdvLZ7jUESCZVZZX66yoLtisxBHkLVhVqJzl+I+kVLa5qsDwHdtpN23yz6iZS8atXJ7AuzgOhJZMe9fkD5ujxOiL/oeUmsFR4D//P6i1XIfRdtzPuuMu5NZlB1QEufYwmm8mqre7Ufub99kDZg6r4PNoqVLqs8PSpTHFWG8AnQXvZN+XzmZDBB5Wk2PASnruUQTnIv1rzqYXDhhsngyfJXMxg+tmPCTf6afw+6S13gl++xrDBwe6pEz6gLJG7z2jFaqfsHuAjyzzTp++Z6TfGGLpc80AydSQT0JA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rqkhaQ3j8+Kjb2g9lCV7+HCsqiGSl3S6Y6XiGdv9440=;
 b=2xrVqG48jpB+IU3gEVGpEVC2uJcKmQ7/875aj4MwRFzspvV6foBj9uaQ+TKKk7xTcEH4cKRAVqUwUc1APK2uchgG8nxKKn0PI5HI2K3KnQqJJatHDalqB6FHNu6Q8qQQ3TBNTruVbNw7pfOT7yqfQpsjHCMKjhSJNTkXvYkHZ3w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <6d877f4c-b4a1-4f0e-9588-b035aaa71b4f@amd.com>
Date: Mon, 6 May 2024 10:01:44 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] xen/arm, tools: Add a new HVM_PARAM_MAGIC_BASE_PFN
 key in HVMOP
To: Stefano Stabellini <sstabellini@kernel.org>
CC: <xen-devel@lists.xenproject.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>
References: <20240426031455.579637-1-xin.wang2@amd.com>
 <20240426031455.579637-3-xin.wang2@amd.com>
 <alpine.DEB.2.22.394.2405021104390.624854@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2405021104390.624854@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000A34C:EE_|DS7PR12MB8203:EE_
X-MS-Office365-Filtering-Correlation-Id: 47951bdf-12e1-47ce-c24a-08dc6d708ef1
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|7416005|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VGhuSWNNbmRMNit3VXJOQ2U0RUNaaE0vTEFkWXRJQ1pJcDRPU1l1WjFCSStH?=
 =?utf-8?B?TG9XdllmUWI1aElkZ1VheU85UEExZ3J1SkdKN1F1TkRBZjEvWHNjUHlxL25m?=
 =?utf-8?B?THJxWmtiZnlZZ0FrYWNMVGNtc1ZqL1FEblRwWlVuUEVJYTE3akNxQW9IMXZ0?=
 =?utf-8?B?K2VCanJ2QXZIU1NiVE1CK0I3MFl2M1VvRVUveUQ2ZWVrcUY0WXQwYTIzd2FD?=
 =?utf-8?B?cGxHUC9XWlIvb3RieTNhdkE3WXBCVXI3QUxrYXNNc2ZlOW5JcWpZdmxNZWgz?=
 =?utf-8?B?QnpiRG13ZXZRS1VyZ04xOTJNOUh6b1N2QVpqVlFtTUhDRU1YQ1hNNU4vdFJY?=
 =?utf-8?B?Q2ZsTk04RjJCZ3lHdVRxak5Kc2ZTWEZueFpPT21NL1psU1NCT2gxQndDMXRn?=
 =?utf-8?B?TWZ6akdidnMzVFA0TStoWGFVOFBLc1BkTWhqbTNpbThLMGxhVHN5ejhkVWVF?=
 =?utf-8?B?SStaNjBMT25YQ1Jxb3kyQmtldzJVOHByelRBSmNac3FHaGhvbEpGblBnUGtR?=
 =?utf-8?B?MmQ3WVBzSFFKTERxeEJMb2FEZXl2SUVoc1J0ejRUNE95aGdzdnMxd1dxS25p?=
 =?utf-8?B?QkdXUWxpUG02RmJoa0xKazhJY1BMYlhrcDNiVVBUYVB5YjFjZ0x5Mm91SjR6?=
 =?utf-8?B?bk12RUZsV0ZnekdWMFFZUTlQMTJkYmlhMDhpL29Nd0RSSVZuQ2FZaXRVMjhH?=
 =?utf-8?B?VGQ0MG9kL2g0NDF1cWpJQzZVRFlzY3htSWduYnFFdW1IalFzUTRzekptUWJh?=
 =?utf-8?B?R0Z0SVpJRUZmclZuQ0QwRDlBcG5NTmd5ZWRCa1RsQ29NQ3hvTkYyNVRlVFRE?=
 =?utf-8?B?SzhSMFNmVHBUY0s4UWJqSmxiaW9SRGlMTFp1QjNrUFFoaU1hZTd2dlIrWkN0?=
 =?utf-8?B?eTJ4N2JkeXFyckhnLzlBZEVGOU55eDJvQW03WE9zaDBjdVZNWG9lQUZsVlZ3?=
 =?utf-8?B?M1Vha1hnS2JxWWU2NWdwL1VrVHBid0g1T1k4aDExWk1yaGpuVjVWcjBIanJP?=
 =?utf-8?B?cDBHMTZLWi83Q2R2dncxOHJMdUo2OUxmZEZLaytpdHFBTXRQRWlKMjQzR0dt?=
 =?utf-8?B?QVNacG1uT1YwM1JMVXpZQjZmR21UTHpEeUhjVGc1UXRLaHFOcFl4a3RHZjU1?=
 =?utf-8?B?TndxWkd5MDZnamdPeHo3Nk9OYllWN0JERGkrTlBtZTM2ZU1DMjFLTEFoZFgw?=
 =?utf-8?B?bnJraDFjWGV3Szc5VVJEVEl5RFRCRnFseUlWTFUweEorRFAyNGpsRDhCdEpB?=
 =?utf-8?B?aE1IdWU5eTBGci9KM2J5RUpjcVROMVdHUUNzaG5tZ0ZpbDFEN0RTR0ZjaTU4?=
 =?utf-8?B?Q3lEK0VhWGRtNFZ3LzdpUlpSTWg5RGRUeFJrZ0FKTGI5K2dVNDQ3TUh6dSt5?=
 =?utf-8?B?L0k3eUxRTTNrOGdmK2gzOWdXZ2NXQ2wySEwrR0FjWnV1ejZkUXYwaWtvTGNk?=
 =?utf-8?B?Q1NYcERXbUVSMUZBOVNtYWpZV0FqeEJ6c215MWJDT1A2MEhRY0NOSEpGc29C?=
 =?utf-8?B?STNnUHQrcnpUalBpdXArelVtcldXUCtLOVJqUC9rd1ZSU0E0bVYzV3d3bUJC?=
 =?utf-8?B?ZURWR2htSGdtc0syUnlyTkwxZEZ0V3lvdzNyVElTS0NnOVJqZ1BHU0FOSUJL?=
 =?utf-8?B?ODNYNjNybGZhSlpxZ1Q4Wk1KcEo1NWNQbmJ4ODBUcVJUVVhwVnFSTUdsK0x0?=
 =?utf-8?B?VFNBRTU5K3dYTEJFSEE2MEcxdEtsT1FscmZoRUlFY0FuNjVsUW5wS2NvM1RG?=
 =?utf-8?B?blBtYmNHMUp5bFpyNXVtUlhMa3Q3bGZjcUxDbGVFOWNKVHR6RGV6VUxBZjFN?=
 =?utf-8?B?eVdkekZpcnJMTXpWelRpb3V5TnQwalo5N0grditLaWRvWmR4WC84b0I2VlF3?=
 =?utf-8?Q?7R+i35vGxEXOi?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 02:02:18.1116
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 47951bdf-12e1-47ce-c24a-08dc6d708ef1
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000A34C.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8203

Hi Stefano,

On 5/3/2024 2:08 AM, Stefano Stabellini wrote:
> On Fri, 26 Apr 2024, Henry Wang wrote:
>> For use cases such as Dom0less PV drivers, a mechanism to communicate
>> Dom0less DomU's static data with the runtime control plane (Dom0) is
>> needed. Since on Arm HVMOP is already the existing approach to address
>> such use cases (for example the allocation of HVM_PARAM_CALLBACK_IRQ),
>> add a new HVMOP key HVM_PARAM_MAGIC_BASE_PFN for storing the magic
>> page region base PFN. The value will be set at Dom0less DomU
>> construction time after Dom0less DomU's magic page region has been
>> allocated.
>>
>> To keep consistent, also set the value for HVM_PARAM_MAGIC_BASE_PFN
>> for libxl guests in alloc_magic_pages().
>>
>> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>>   tools/libs/guest/xg_dom_arm.c   | 2 ++
>>   xen/arch/arm/dom0less-build.c   | 2 ++
>>   xen/arch/arm/hvm.c              | 1 +
>>   xen/include/public/hvm/params.h | 1 +
>>   4 files changed, 6 insertions(+)
>>
>> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
>> index 8cc7f27dbb..3c08782d1d 100644
>> --- a/tools/libs/guest/xg_dom_arm.c
>> +++ b/tools/libs/guest/xg_dom_arm.c
>> @@ -74,6 +74,8 @@ static int alloc_magic_pages(struct xc_dom_image *dom)
>>       xc_clear_domain_page(dom->xch, dom->guest_domid, base + MEMACCESS_PFN_OFFSET);
>>       xc_clear_domain_page(dom->xch, dom->guest_domid, dom->vuart_gfn);
>>   
>> +    xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_MAGIC_BASE_PFN,
>> +            base);
>>       xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_CONSOLE_PFN,
>>               dom->console_pfn);
>>       xc_hvm_param_set(dom->xch, dom->guest_domid, HVM_PARAM_STORE_PFN,
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>> index 40dc85c759..72187c167d 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -861,6 +861,8 @@ static int __init construct_domU(struct domain *d,
>>               free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
>>               return rc;
>>           }
>> +
>> +        d->arch.hvm.params[HVM_PARAM_MAGIC_BASE_PFN] = gfn_x(gfn);
> I apologize as I have not read the whole email thread in reply to this
> patch.
>
> Why do we need to introduce a new hvm param instead of just setting
> HVM_PARAM_CONSOLE_PFN and HVM_PARAM_STORE_PFN directly here?
>

Yeah this is a good question, I aIso thought about this but in the end 
didn't do that directly because I don't really want to break the current 
protocol between Linux, Xen and toolstack.
In docs/features/dom0less.pandoc, section "PV Drivers", there is a 
communication protocol saying that Xen should keep the 
HVM_PARAM_STORE_PFN to ~0ULL until the toolstack sets the 
HVM_PARAM_STORE_PFN.

I am open to change the protocol (changes might be needed in the Linux 
side too), if it is ok to do that, I can set the HVM params here 
directly and change the doc accordingly.

Kind regards,
Henry




From xen-devel-bounces@lists.xenproject.org Mon May 06 02:16:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 02:16:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717170.1119136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3nuB-0004WI-Pn; Mon, 06 May 2024 02:16:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717170.1119136; Mon, 06 May 2024 02:16:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3nuB-0004WB-NF; Mon, 06 May 2024 02:16:39 +0000
Received: by outflank-mailman (input) for mailman id 717170;
 Mon, 06 May 2024 02:16:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3nuB-0004W5-5j
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 02:16:39 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9b5acbc-0b4e-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 04:16:37 +0200 (CEST)
Received: from BL1PR13CA0129.namprd13.prod.outlook.com (2603:10b6:208:2bb::14)
 by PH0PR12MB8800.namprd12.prod.outlook.com (2603:10b6:510:26f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 02:16:31 +0000
Received: from BL02EPF00021F6B.namprd02.prod.outlook.com
 (2603:10b6:208:2bb:cafe::e4) by BL1PR13CA0129.outlook.office365.com
 (2603:10b6:208:2bb::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 02:16:31 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL02EPF00021F6B.mail.protection.outlook.com (10.167.249.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 02:16:30 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 21:16:30 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 5 May 2024 21:16:29 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9b5acbc-0b4e-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ady+4OdaaNv20tQ+VUMLzuQt5DArIp9PAeahFU5PiKpS2qeLvrrL9EyNTAnCbRwzu6Ox2U4OfcAALMM9w6ZZQd0BwQeVkww91hfel6wBxITZ7fYftevvZg7cavtZocO3QFo0UkDpXu9wb2aSOglZ+oCw2zAzygg4d0pJTrqHvve3h0t0fMloqymZc+gdikPdNPU64LTU6A+WgRLeo4IafQFrxiELWrBKB8MYtaXjB7IapSXY698fvN0x/pWJBm414Hphl8SDdC5TYScoaq84CBnUFJgkWya8RxhEsGibkeFjhrsnBNRu2Izv9WgH99P248tU13roD74v6UiSIawPRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JbEo5joN/4v2ow6t+LusWh6gNzrsasDkuzFWmFX5q4I=;
 b=mzW5blWIIYNkA4dIVzUA5krn2V69h3Fsq4MFX390RGVTcN1Q3e+N3S2wp89nc7BFt4k5iro84i+sID5wEmhutoRW6VZeXsx50HTWqtRUIOcxsWYy0fsS0KTigpUljE1GRjARGp9zFkAHZ1W2DjoF2WRDja0vtW3beTfezjj4fEGGtN72+pSWqafy4Yuf0uAA/UPjmHqb8ktXvAYwxC3huoNDxpoFL5zCSrvoufbho/+1FGGb96TRAzZ5PH9K3hu6d+HmxRw7P13WN6wmi3wIj66iAjmwYVCIhT5DLGaExu3qPT0PaiTO+fqLG86L/k9KD7mhl4t4jNrB0JTe/J21tg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JbEo5joN/4v2ow6t+LusWh6gNzrsasDkuzFWmFX5q4I=;
 b=wDm0lMFTI7CB3aADe+vuKtFYAS64HIdZj6spJEV1C+yn5XMaTdvWUW8WIuqPkDIlb6hls7YUAfwqaDccF4ckqw24LftviJHmKp2oJ8OUpeQCUAVOzC4ZvwKznicyPzX58IAfFw1GD9b8FxKTMMFirBLa8rd4fXVDVtjafSKkwtY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <b1db7fea-eb86-405a-9cde-e8c600e0c4b8@amd.com>
Date: Mon, 6 May 2024 10:16:28 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1.1] xen/commom/dt-overlay: Fix missing lock when remove
 the device
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Jan Beulich
	<jbeulich@suse.com>
References: <20240426015550.577986-1-xin.wang2@amd.com>
 <835099c8-6cf0-4f6d-899b-07388df89319@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <835099c8-6cf0-4f6d-899b-07388df89319@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF00021F6B:EE_|PH0PR12MB8800:EE_
X-MS-Office365-Filtering-Correlation-Id: 3394e723-e85f-4764-e81f-08dc6d728afd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NHVkY0FKY1RUNktCbllmWWt1Sm0wR2xqOFk4TEkvMlR0Sm8xNkhJa1NkTVVO?=
 =?utf-8?B?eDZkdVVkWGs3cEs0L3lJQU9FNkNtTGNqbEtsYlNFMVhCRnduMDE1aGxKOEN3?=
 =?utf-8?B?anpCaHZHZityb1pTMGsyc1k1dWZoczJqSXRwNUNUSlJKN1RNRmhuaVRDSnRw?=
 =?utf-8?B?NFdzczFHVDlyMmJhTjluMzV0TG9uZTJxeTNLb05qbzJMaHUzblFKY1FQQWIw?=
 =?utf-8?B?M3hUWGIzWDR1Y21zZWVNU1dlWjRsaklGek9ITHBwZkFTdG9yaXBqN1FjUnZX?=
 =?utf-8?B?bFFIK05CN1VyVm9MWlFUcDdxaFVEVTRXMFY0VjZpN1NmOFJDUjIvNGxTS25z?=
 =?utf-8?B?MElheHVOSE53V3k1aTJYZGh4TS9NcHJRcXJ5Tkl3aXhpK0dLMFc0VGZsbTZq?=
 =?utf-8?B?Z3hMU1pBVVZmUVlUWGhVd1dCaExuMkxyZmI0eXFIdUpnSVMzSnNTMmtVTEsy?=
 =?utf-8?B?TURYTGI5TzdGdURobDZvZDRqb3VZbHRBRGszL3oxSGUzVDh1WFhYZFRkcGtl?=
 =?utf-8?B?OE1JVUx0TjlqRXdSSTlYNmJxbWF2YkhLbG1HcUFhU00xYmoyaWJEZ2RXSlY4?=
 =?utf-8?B?eUlEaWkxdXN1cTNBbmtoMEtsaC9CQ2hUcm9WK3p5T2hlWTRWbWlDa0J1NTZT?=
 =?utf-8?B?RTRRVkc4eUVTN0hGT3NrUktqdTZMNU1TVWZXUGFZZ3FtUnNPQnA2NFlSWWtl?=
 =?utf-8?B?Z2MzSzh4c01MeW5HYitydnlkbUdOalc4ekhncGxkMnVicHRRTXh0djZyRUp4?=
 =?utf-8?B?MmJQU3JMdXhWKzdma2l5aGlGN2JaVGRMMzd3NEYyUWg4VmJjbHVuZ3BOUmFU?=
 =?utf-8?B?aFlIUC81WWdOWUY3akkyMGVSNzVvbXllTEJsNW5maHpXby96S3pXWGZIWkpl?=
 =?utf-8?B?SG8zQk9pNjB0dUZpeDgvZks1Z3JmcGluMExibWFzcExwV2FxRURabWxRUkpj?=
 =?utf-8?B?OTIxT1Y5dW9wSlo4YklQcnJIdnRLRDZ5NndhdEwxTHRqQklzVHF3VW9uMWp2?=
 =?utf-8?B?UjlIMytKSk0zNmFjODk3OVBHWXZEakJEbHVpeERqbHNLOUJaenBYQ2lleUVH?=
 =?utf-8?B?bVIxNEJYbFBNMTFWVjE1WHhZN0EzNlUrUjVGRGp2UW5Tbm9nSjBPZnRjVEln?=
 =?utf-8?B?ZVZteXNWSFltTFpCemlRbTUrcTJTZ2xFWFZYUjRaWFZDUVlBYlZzMmFvcXA3?=
 =?utf-8?B?Yzh4NHYwcWFOVHlsY2ZNVDZ4WUswSFAxSVY3OGhTbVJNalFWMDE4RVpCK3ZM?=
 =?utf-8?B?eG5VbTdGWnIySCt6NVp6OFJKNEhzTXBGQXpZTlllWmpCQ0h5MWppdWRRUVhs?=
 =?utf-8?B?NUxxTVpYWlFsVmhXYnFTUk5mY2lNSEpSYWJLK1ZsbmtJbUdUc0QycVg5RWVL?=
 =?utf-8?B?MjBtbml1UTBOMWphMVBGTU1hSDVMY2E0a293UWpFbmpuKzcyYThqU0NtaU1i?=
 =?utf-8?B?UERRSG5uZnRMMVRiVFpQb0l6QWZmbnVtYUtNYkhiOFhIR1NxNlpFK1VUeGhK?=
 =?utf-8?B?V0pDTDJ3VEtnNHFOOHBHVXVKWWFCVUhKK083c2h6M1YxeHRmb3ZNYjdnU0NB?=
 =?utf-8?B?T20vYnZlTUZkYnk5ZlJlcEY1RWc5MVdGd1pJOUlmMHNBUWl3Z3h3bUhxR2x4?=
 =?utf-8?B?bFVHcUtRU1R1M3ZZMUlmZ0pSYURyak04TTR1SVlQc3JtZEZZcGtnQnoyczlZ?=
 =?utf-8?B?NE5SZ051Tk13dDVQRm5mOHV4dG5jalVCMkpxWFF4WlRLdHd3T1Zia1NJUHF2?=
 =?utf-8?B?YkFteFJOendZeUlrckZKYXArK1BMSEU5R3R3TVBiRktNandlSTZKdzB6dmVV?=
 =?utf-8?B?NzBBSVNGM1FsRitoNjQwUm4vNHN3M0psd3QxRUJzS3ZVdkVwT1grbmFKQXBL?=
 =?utf-8?Q?5TDYqWwjnfqCp?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 02:16:30.4894
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3394e723-e85f-4764-e81f-08dc6d728afd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF00021F6B.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8800

Hi Julien,

On 5/3/2024 9:04 PM, Julien Grall wrote:
> Hi Henry,
>
> On 26/04/2024 02:55, Henry Wang wrote:
>> If CONFIG_DEBUG=y, below assertion will be triggered:
>> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at 
>> drivers/passthrough/device_tree.c:146
>> (XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
>> (XEN) CPU:    0
>> (XEN) PC:     00000a0000257418 iommu_remove_dt_device+0x8c/0xd4
>> (XEN) LR:     00000a00002573a0
>> (XEN) SP:     00008000fff7fb30
>> (XEN) CPSR:   0000000000000249 MODE:64-bit EL2h (Hypervisor, handler)
>> [...]
>>
>> (XEN) Xen call trace:
>> (XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
>> (XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
>> (XEN)    [<00000a000020797c>] 
>> dt-overlay.c#remove_node_resources+0x8c/0x90
>> (XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
>> (XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
>> (XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
>> (XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
>> (XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
>> (XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
>> (XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
>> (XEN)
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 0:
>> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at 
>> drivers/passthrough/device_tree.c:146
>> (XEN) ****************************************
>>
>> This is because iommu_remove_dt_device() is called without taking the
>> dt_host_lock. Fix the issue by taking and releasing the lock properly.
>>
>> Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal 
>> functionalities")
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>> v1.1:
>> - Move the unlock position before the check of rc.
>> ---
>>   xen/common/dt-overlay.c | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
>> index 1b197381f6..ab8f43aea2 100644
>> --- a/xen/common/dt-overlay.c
>> +++ b/xen/common/dt-overlay.c
>> @@ -381,7 +381,9 @@ static int remove_node_resources(struct 
>> dt_device_node *device_node)
>>       {
>>           if ( dt_device_is_protected(device_node) )
>>           {
>> +            write_lock(&dt_host_lock);
>
> Looking at the code, we are not modifying the device_node, so 
> shouldn't this be a read_lock()?

Hmm yes, however after seeing your comment...

>
> That said, even though either fix your issue, I am not entirely 
> convinced this is the correct position for the lock. From my 
> understanding, dt_host_lock is meant to ensure that the DT node will 
> not disappear behind your back. So in theory, shouldn't the lock be 
> taken as soon as you get hold of device_node?

...here. I believe you made a point here so I think I will just move the 
write_lock(&dt_host_lock) as soon as getting  overlay_node, i.e. on top 
of the call to remove_descendant_nodes_resources(). Therefore we can 
solve the assertion issue of this patch together.

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 03:13:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 03:13:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717176.1119146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3onB-0004ja-SG; Mon, 06 May 2024 03:13:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717176.1119146; Mon, 06 May 2024 03:13:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3onB-0004jT-Od; Mon, 06 May 2024 03:13:29 +0000
Received: by outflank-mailman (input) for mailman id 717176;
 Mon, 06 May 2024 03:13:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3onA-0004jN-HQ
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 03:13:28 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 98ea6420-0b56-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 05:13:24 +0200 (CEST)
Received: from PH7P221CA0019.NAMP221.PROD.OUTLOOK.COM (2603:10b6:510:32a::15)
 by DM4PR12MB7503.namprd12.prod.outlook.com (2603:10b6:8:111::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 03:13:19 +0000
Received: from CY4PEPF0000EE37.namprd05.prod.outlook.com
 (2603:10b6:510:32a:cafe::62) by PH7P221CA0019.outlook.office365.com
 (2603:10b6:510:32a::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Mon, 6 May 2024 03:13:19 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EE37.mail.protection.outlook.com (10.167.242.43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 03:13:19 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:13:18 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:13:07 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 5 May 2024 22:13:05 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98ea6420-0b56-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X3ZMf/WIM3Xfs7Lo2g0thkasbpcO5J+g/wkWFWCQUh49ZDRDtYrB5uyrz5TEbsnobenHsIlWRoYCkUuFlE4jgsJSkVg/8x6dW5TTUGSkbr70yDMUUHQobs6DlQW1alUNYiBUqmyszqG2A1mUYGGkJruz7Z5cU/w2BszanymI0jrRq75zByemCVcbbpobFAsUcMDz0lSU61E64q0A7wBQKg9SfCWGob6qDeikCLQRAh0jjDZBSIweXtRqlnnR0jCFP663/8rU+523JGPn+9uK5GhDlX77cBzjpEOVWLEgPjpVfgUli8G1FP42a1TYobA9qKn2AvlVnkUHLIiZt94MBQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=v1e21ooJqaxmkhXcun0n96GKnY1vVbBlfGXqRdJB2vo=;
 b=C8vpNqDrK/YTHmhTAmNI4/0AaNARN1DDirjzHu6xz6XbHzv2kd5LIiD53RSwled3/skgc4+KKAYIIKO667nubTB5Z7P0ZINIn2hchpyMWDiiPdRU5YJNy7WgIhB0f4ZhGkuB6h1AGYzu+SYevIDdKu+Y6QIWi5luyi6mmIidWQhBnaZXe7nzjd6VV8IEtVx22rP0IOdpezwzyTnxgprg1fPjmhlYF8V9jZIRvFXHVZdUb16T9HMP5Ns01ROXIYMqVUnvdP82+GNIKmxSAxGvXWtTByAvn2c5wskhKFI8UQFRlwpHLpcQg1JmeGQnBTTzQvnH2v7i6ipLa0d+I4rP/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=apertussolutions.com smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v1e21ooJqaxmkhXcun0n96GKnY1vVbBlfGXqRdJB2vo=;
 b=UOYO1r5P7BWdxJEMposoAO2Nq9c1zGW5g7bJI/BqlMfgEnFxS44qYatPJtsCxq5nljWEwozCWqSESDoehSNnX3/vvzX02mtZPNKoacHLkIIKQVuILY9Tabbz/eh8VLFK6863DIAm9UmKWBb5vhzmYAm/eX1aFXuMe10Zd5vsADE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <eda5fd5b-87de-4da1-a785-44da563176cb@amd.com>
Date: Mon, 6 May 2024 11:13:04 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] xen/arm/dom0less-build: Alloc magic pages for
 Dom0less DomUs from hypervisor
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, "Alec
 Kwapis" <alec.kwapis@medtronic.com>
References: <20240426031455.579637-1-xin.wang2@amd.com>
 <20240426031455.579637-2-xin.wang2@amd.com>
 <ee507cf7-b5e8-4141-85b6-c81955bcc348@apertussolutions.com>
 <03357204-0308-46a2-a215-142d681d2b6a@amd.com>
 <3054018d-1ae4-43c5-b44a-2ecf1b4cdfc7@apertussolutions.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <3054018d-1ae4-43c5-b44a-2ecf1b4cdfc7@apertussolutions.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE37:EE_|DM4PR12MB7503:EE_
X-MS-Office365-Filtering-Correlation-Id: 745825c3-bdf5-485a-9416-08dc6d7a7aaa
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?WWpaQmtnZE03RkJ0SHp0eDNzcmlZdHZOY0FERnJQd0FzMEp5cW5LUzh1OTA0?=
 =?utf-8?B?cG13Sk4rV3BnQXg3aXlBTlNldXpnclNtUnFJeFNJNjNsTjVCVFF1Q0pOYWJC?=
 =?utf-8?B?dm1mUWhZRGRkM3crR1BNQ2dNV2Vja1lBa1o3YXc5cVlVeXo0d1FSV3g1MGtI?=
 =?utf-8?B?WWxNUUNqbkw5NlpGRk95VkhEZ2tnV1cveEk0UFEycHptMm5FNmp6S0VXZmZT?=
 =?utf-8?B?MUtVWGY2dXFhT2t1ZnhyRDdwS04zV1JKWVpJcmNTZFVMOEtYWjgybGtzOHh0?=
 =?utf-8?B?YWlrZ1pCSVVHeUtsdi9jOHAvNFhZM3JvYlRDanZORDhXdklUZytRMnZIc3lZ?=
 =?utf-8?B?dTBnaDNEU2FXTzhlR0JQSnJaNUhkY0pSUmEwdXhST2xPRnl1eSthRTFOU3pJ?=
 =?utf-8?B?MHozUEJjcnh1NUhMbWpEM2tsYXFFbSs0Y2hmWEZ0bkRGYXV1bmFuVEF3WmJl?=
 =?utf-8?B?U3l6UXJndC9sOHZzN0dnV3NIVFhMeENqR01DRmFSUnFnRGsyL3RyT2VBVm1J?=
 =?utf-8?B?RjdqRExWUHZWT1BNYVFKY0ZNcG5JYzhqSHZuejBUNTZVb1Y3d3NEYU9DeVBN?=
 =?utf-8?B?Ky9CeXMxT3Z4YlQ3ZFZyZVpsQU9XQTdUTVZaZUlzQzVReklmNGIyL2VZd2g2?=
 =?utf-8?B?eHE4MFVCRjhMU3cwR0VGSHpLS0xsRUc5c01ySUVEeWNuYXJUVzY0aUpLbnhU?=
 =?utf-8?B?NFNORThmQXQ5c21RTDl1ZzZDeWxESms3RCtNYlF0dlg3RldKQStFMkw0RGxR?=
 =?utf-8?B?VDkxanF1UnhFVFo0eHZGTENUcG1wNGpvTkE1NlRVcjM5SmhLYWFwTkVxdUhx?=
 =?utf-8?B?Y2xyQU9mRDZDSVFYenJmeG4xeVZVUE0rSUN1ZVU1d0ZuZWpubE4xa0QydU5W?=
 =?utf-8?B?WkxpUjRvY0hnYTV0OXNKZzRaeURNc1BaT0J2UDB0REFPTmYzVm9FbzB2WmNW?=
 =?utf-8?B?WkRxRXN0RkVzMUJCbkxGYmt5SmoyMGxZcFljOTBRTkp0bktBaUdoMFIrY0t6?=
 =?utf-8?B?OW8zU05tYVRoRFJRY044c1NEcFcrQ0IyczhlUzFPbm1aMzVFVy84Mk5DMVU1?=
 =?utf-8?B?ZEdpVXFHeHBNL1ZsdU96VmpROEZ3c1NpRDBsRFl0Z2xsdG1oazVySGI1UjRC?=
 =?utf-8?B?UWl1WG5oZ2lrWlZaNGVYS2JnV3h4cFBCcVl5QzArelExY1Rtc1djTWo4UjNE?=
 =?utf-8?B?aUVjaWJUZXhSUFJJVFFiQWR1SUxRN3FEY1psSUhlbDlpaGNseWZRRkNUaHR0?=
 =?utf-8?B?dng0Smp2NzMwQW4rUFZwS2VoWG1OUG5sTkM3b3M2TVFod3VoVWdWSFlXWEpU?=
 =?utf-8?B?S2tOTkRQQURwT0Q0dXVxOU9HVnRjZ2w2T2RQQ1ZVVEJncUtPSTIwdmFENi9U?=
 =?utf-8?B?dXdmNHRMY2ZmZlU3NG5YTC9hWjZPeWc2SUJrTVVuMC80R05wYmJDSmxhb1hN?=
 =?utf-8?B?ZzEycXZxZXdYZW1HYVhJQWd2c3JNRFIyMUhSblBnaVhnWHFYYUZBTXpMSVRE?=
 =?utf-8?B?UGppblowc2tSMFI0cHRETGVZVUROVk1wY1N5Tnk4bFRFTU5EWHQvS2hkdnF3?=
 =?utf-8?B?MDB4d0NsYVNqbVNtUVJRbGlLTjMvRnpPYUVBcVZCZXZOVW1oamkrendLTkIx?=
 =?utf-8?B?bm1LRG9QbjR6SGQ5RTVqNDNLL2tTTkkyRWZGeEs3aHMvUXlHSjVCZlgvRUNp?=
 =?utf-8?B?SzZuUHp0YUFGNGN1R0JYT1RQdXJBNGdGS3ptd0hyYVQwbDlRR2RieVdxelV3?=
 =?utf-8?B?RFpQbjZ1bDJ2eDN0bFpnY0J1Tm1aRVVoNC9wTzNOTzYzdlZjMm5WaHl2UXIz?=
 =?utf-8?B?ZDRBL2o5U0Q0Mm5odDdxYzVsTmJzQkxzNnhReklpaFVpTWdKdGd5QjFuc0NZ?=
 =?utf-8?Q?HzFPzGkf4AB0D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 03:13:19.0122
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 745825c3-bdf5-485a-9416-08dc6d7a7aaa
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE37.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7503

Hi Daniel,

On 4/30/2024 6:22 PM, Daniel P. Smith wrote:
> On 4/29/24 22:55, Henry Wang wrote:
>> Hi Daniel,
>>
>> On 4/30/2024 8:27 AM, Daniel P. Smith wrote:
>>> On 4/25/24 23:14, Henry Wang wrote:
>>>> There are use cases (for example using the PV driver) in Dom0less
>>>> setup that require Dom0less DomUs start immediately with Dom0, but
>>>> initialize XenStore later after Dom0's successful boot and call to
>>>> the init-dom0less application.
>>>>
>>>> An error message can seen from the init-dom0less application on
>>>> 1:1 direct-mapped domains:
>>>> ```
>>>> Allocating magic pages
>>>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>>>> Error on alloc magic pages
>>>> ```
>>>> This is because currently the magic pages for Dom0less DomUs are
>>>> populated by the init-dom0less app through populate_physmap(), and
>>>> populate_physmap() automatically assumes gfn == mfn for 1:1 direct
>>>> mapped domains. This cannot be true for the magic pages that are
>>>> allocated later from the init-dom0less application executed in Dom0.
>>>> For domain using statically allocated memory but not 1:1 
>>>> direct-mapped,
>>>> similar error "failed to retrieve a reserved page" can be seen as the
>>>> reserved memory list is empty at that time.
>>>>
>>>> To solve above issue, this commit allocates the magic pages for
>>>> Dom0less DomUs at the domain construction time. The base address/PFN
>>>> of the magic page region will be noted and communicated to the
>>>> init-dom0less application in Dom0.
>>>
>>> Might I suggest we not refer to these as magic pages? I would 
>>> consider them as hypervisor reserved pages for the VM to have access 
>>> to virtual platform capabilities. We may see this expand in the 
>>> future for some unforeseen, new capability.
>>
>> I think magic page is a specific terminology to refer to these pages, 
>> see alloc_magic_pages() for both x86 and Arm. I will reword the last 
>> paragraph of the commit message to refer them as "hypervisor reserved 
>> pages (currently used as magic pages on Arm)" if this sounds good to 
>> you.
>
> I would highlight that is a term used in the toolstack, while is 
> probably not the best, there is no reason to change in there, but the 
> hypervisor does not carry that terminology. IMHO we should not 
> introduce it there and be explicit about why the pages are getting 
> reserved.

Thanks for the suggestion. I will rework the commit message.

Kind regards,
Henry

>
> v/r,
> dps
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 03:16:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 03:16:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717179.1119156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3opk-0005VF-5S; Mon, 06 May 2024 03:16:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717179.1119156; Mon, 06 May 2024 03:16:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3opk-0005V8-2F; Mon, 06 May 2024 03:16:08 +0000
Received: by outflank-mailman (input) for mailman id 717179;
 Mon, 06 May 2024 03:16:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3opj-0005V2-AY
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 03:16:07 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f8af4a30-0b56-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 05:16:05 +0200 (CEST)
Received: from DM6PR04CA0029.namprd04.prod.outlook.com (2603:10b6:5:334::34)
 by DS0PR12MB8478.namprd12.prod.outlook.com (2603:10b6:8:15a::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Mon, 6 May
 2024 03:15:59 +0000
Received: from CY4PEPF0000E9D3.namprd03.prod.outlook.com
 (2603:10b6:5:334:cafe::3) by DM6PR04CA0029.outlook.office365.com
 (2603:10b6:5:334::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Mon, 6 May 2024 03:15:59 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D3.mail.protection.outlook.com (10.167.241.138) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 03:15:58 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:15:58 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:15:04 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 5 May 2024 22:15:01 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8af4a30-0b56-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mrb73tqaDvGuWUtOc/oofGWff07c6PffgZ/u/89IHI16t1xorF/XUCUi7WzkJvnKiQ2RC5s5uZk0zAKDRsIIfV0T8aTqvoPyrC8GewOYwHu8TE5yPU/ckQQsWudUqo+sOPlN6gSQv45Ys612nw20uCIeRnxUcv8rhLDrW3HLja+fQZfZpW79D/nSsYsFtV1VgyzHj9cEAoHIcHw/x9EJlu8H0aNIIVe7iq+2ekQmmd3OwXrXNqTWweGpIG5yC5SY7g7yK0tPyu8RKgIkjGIhx2dJizNPFcWssLHQJ0EWzjHyj9fcOIL6A1QHMQQWhfLrYOyQgMWs5Nn2YVK+wN7/3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=i1bh3+wge0j+km//KlMypTl9qwGSWcm+TB01jpyzT2E=;
 b=e0nwLrVRLxne4r6TYuXIhw0u54odg/xsS10pei2T3qlvWC01b5dL7JHJh+NB3s1W71uSMmUVN9USwstlz/urDWWGKUjQXFmqMvA/sYv5ZhyWmixYhU8tIxdXU36Wu52zOqoPqvlJF1+L39BKsjltuhwD0K1BemUwvxYTjjXg7/hI8Zu9JWQdHleAzQvGG9KXM5lnYvJxdqMjzeEn3Y7pZhX63jeOKnZ/a+I8qV1Wk0ZStdJlozDf3yFf2lkaP71UZhrPE6KChND7XiVzqM0a/k48zGZa6BtFSzZtR/WysHLnyb7h/kLKkgPr9gATChRt+znViTzepHXqZuMS9UHrJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=i1bh3+wge0j+km//KlMypTl9qwGSWcm+TB01jpyzT2E=;
 b=EoI15oSY1KeIenxMXGOk4m4MwK10qDILtT1vtboBEgIoZjlZP18gojPsUHYFeo72zgioKkwHFtBq7lnZWRT/d7juCphN9yVLUqqlRtQZIeBZ/oKXpxru4HLDrTBYPHEVpjmA9Xhy2V/XL8tyLQb6viK5DIzQ7pem0xMTQCHm9ac=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <999ff94f-826e-4493-9902-b700bd9747e0@amd.com>
Date: Mon, 6 May 2024 11:14:56 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 07/15] xen/overlay: Enable device tree overlay assignment
 to running domains
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
CC: Jan Beulich <jbeulich@suse.com>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	<xen-devel@lists.xenproject.org>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-8-xin.wang2@amd.com>
 <da67f90f-6807-4fdc-b65b-f7a4ba9f78ad@suse.com>
 <97a401ff-4b5c-4e6a-8d9c-e36305ec64f5@amd.com>
 <128c7e68-0431-44e9-b4fc-96cc46e158ad@xen.org>
 <e3c7098b-a876-48f3-8ccb-18a2ced8521c@amd.com>
 <alpine.DEB.2.22.394.2405021101200.624854@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2405021101200.624854@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D3:EE_|DS0PR12MB8478:EE_
X-MS-Office365-Filtering-Correlation-Id: fd57067e-2880-4a3b-78c7-08dc6d7ad9eb
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QVF6ZkRGN3B5dzRGVFBTQ2lwbFpMMEdGR3l1TDJJM0xXeWNybzlBME5aQ1Fa?=
 =?utf-8?B?VHByOU5QR2Q3MWtmdkFZL3VUcUZXQnRTeXg5Ukc5eFVEYUdaalA4aXM3bFhC?=
 =?utf-8?B?SXN3SHVLL2NvTitmQnk2NlpyT2VWYUlzV2svSHBQMmFqOVM2ekhSNkYxSEJj?=
 =?utf-8?B?TVpldE4yc0pvVFdhc3JaVjFlNmNxbUxWNGptZ095TG04cE9FV0ErL2kzMFh3?=
 =?utf-8?B?VktpSWNOcXVPVDhhd2lGQUNYdEg1Qzg0VXZFSlNXWVVlNzVUTGRpKzAxcEdO?=
 =?utf-8?B?VTBsbzkwRlFSVWxKdWlCaDJEMmZEaERkK2RYTWl0SE5tYXZnQlREc2JqR0w0?=
 =?utf-8?B?cU9xaDhEdkFscjVseWFzRnB6Wk9VaFZjK3VEZ29WTlMwT2locHlnNWE0VE9n?=
 =?utf-8?B?aFFScVdXN2hEbVQ4dXp6SmhPZk9CeGdtdFhqaWlCS2JvcXNqdEhpRFFsZkZx?=
 =?utf-8?B?RzMxUU5LNUx4YUhYc1BVSG5PakRmTHNlakg2QVZBOUFWVXljb0t1bWlmUXFw?=
 =?utf-8?B?V1BuYjdYTXhMZjJLYkFlSForSUtCbkVUUFZ6aDlpMmJGNnB4Z0hYWFM4S1Rq?=
 =?utf-8?B?MUErNmpKaXAvRnNQU2JPdkdHUi80NXpWZ0FXdjhRaEI3TFVNdjRMRUQ4ZFRm?=
 =?utf-8?B?bTR5TEZ6bjU0MC9DRUV5dU9OUDJDeTlNRGNvc1ljK3hBVDlRWTJ0SEMwalhI?=
 =?utf-8?B?N0kzRXM3Sy9wSy9ORVlxc0psYzZ0Y3hPcFc0WEhxSHJDVzlEM0hRMFIzaXdM?=
 =?utf-8?B?cXpqZWFXb0RkVURxWUk4UDkzMGhMUVNCdVJrbmp6MlRTY3pPTis4Ny91L2d3?=
 =?utf-8?B?Unc4WS8xRnJnMk4zS25GRTNTdHU5Yk96S1VtZFd5MW1Db2lSUUx0OG5FTHlR?=
 =?utf-8?B?T3htVUpLS0MyNFlVQThBQ3U2UWNGUVdEMkFSWEFaZ3p6QjI3bGJIMkQrMVJK?=
 =?utf-8?B?TmUrT05qbnlkRGpLdEVNVFB0MVljS1pXb1VTU1czMWVFc3FaZzVVVVowT3ls?=
 =?utf-8?B?eE5Sem9TU00xcE5lK0FQSzQzZEp0b2Q2aVB0aUpaNXBkeEFuYUkvQUZHWDhJ?=
 =?utf-8?B?V0x6dE9UWlJHeDh6RTJkYjJpdmpZaCtmZzdicUFtMW1SUjlCOUNLZytFcDhz?=
 =?utf-8?B?enB2TEdxaHRXdkJ4aUpJRDBzc0hoa2NXRENQUmVqYVcrYTAzYU1Wb0VwYlli?=
 =?utf-8?B?N0kySWdpdDlWSnlLaW5GWG5VaE54c3VhZFk0QTRxM3J5N0xGcTRkU2RxUTl3?=
 =?utf-8?B?RFR0ZkZrS29jemtPTWVXeEkyOFltSG5sWldidXFhS1RUdW5pMXJyZFlKVkRQ?=
 =?utf-8?B?Wkd4ZW9TdXlpTGRjOTZzdUpEbEgzTFVFQ3pnSkVYY0d6emFnOGZoRUpEVGxy?=
 =?utf-8?B?MWpaUy92amx1NkZyK05hSk5aWDl5SDdxak1PWmdtanhRR0dHeEhnaUVabmhk?=
 =?utf-8?B?a0VtcFljeDArOFZwUHJqRlpkL1NlVkhlZGpBRm5mNVk5SXVYVUdJbzhPTy92?=
 =?utf-8?B?OGJwd05lKzFDTVFtc0FsTXNaNHl4ZzZIcHFubjdqRitJbGY0MWJpb2pPWGRl?=
 =?utf-8?B?a284ZjdyOENOS3Mwb3VTUWV0WVByRUhwZ0dlOEZMZmpwcU1TbFBUNmNzZk9x?=
 =?utf-8?B?VHBlRVdtTEVuZWErV21KS1NqTDFxZklBbEhCQU0wUWJUdUZ4ai8zdlpBeHls?=
 =?utf-8?B?cEl1TjJObkc1bFB3K3lUWEhoWVMzd3QxZlRHOG1WT2VuQTYvQ21lZ29jc3VJ?=
 =?utf-8?B?NXI3b3NOcmVUaW5iUDV2RmlmTDdrMHJ2c2thTGFOaUZzTVFnYnU3MXk0V2pW?=
 =?utf-8?B?MnpwOGxxbkVmYVRJSUVtT0phSEJ1c0dNMFk5ZEMrcWw2OXlSWDBTUWJ6RnVr?=
 =?utf-8?Q?9eRGQLUVpKvAO?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 03:15:58.8355
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fd57067e-2880-4a3b-78c7-08dc6d7ad9eb
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D3.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8478

Hi Stefano, Julien,

On 5/3/2024 2:02 AM, Stefano Stabellini wrote:
> On Tue, 30 Apr 2024, Henry Wang wrote:
>> Hi Julien,
>>
>> On 4/30/2024 1:34 AM, Julien Grall wrote:
>>> On 29/04/2024 04:36, Henry Wang wrote:
>>>> Hi Jan, Julien, Stefano,
>>> Hi Henry,
>>>
>>>> On 4/24/2024 2:05 PM, Jan Beulich wrote:
>>>>> On 24.04.2024 05:34, Henry Wang wrote:
>>>>>> --- a/xen/include/public/sysctl.h
>>>>>> +++ b/xen/include/public/sysctl.h
>>>>>> @@ -1197,7 +1197,9 @@ struct xen_sysctl_dt_overlay {
>>>>>>    #define XEN_SYSCTL_DT_OVERLAY_ADD                   1
>>>>>>    #define XEN_SYSCTL_DT_OVERLAY_REMOVE                2
>>>>>>        uint8_t overlay_op;                     /* IN: Add or remove. */
>>>>>> -    uint8_t pad[3];                         /* IN: Must be zero. */
>>>>>> +    bool domain_mapping;                    /* IN: True of False. */
>>>>>> +    uint8_t pad[2];                         /* IN: Must be zero. */
>>>>>> +    uint32_t domain_id;
>>>>>>    };
>>>>> If you merely re-purposed padding fields, all would be fine without
>>>>> bumping the interface version. Yet you don't, albeit for an unclear
>>>>> reason: Why uint32_t rather than domid_t? And on top of that - why a
>>>>> separate boolean when you could use e.g. DOMID_INVALID to indicate
>>>>> "no domain mapping"?
>>>> I think both of your suggestion make great sense. I will follow the
>>>> suggestion in v2.
>>>>
>>>>> That said - anything taking a domain ID is certainly suspicious in a
>>>>> sysctl. Judging from the description you really mean this to be a
>>>>> domctl. Anything else will require extra justification.
>>>> I also think a domctl is better. I had a look at the history of the
>>>> already merged series, it looks like in the first version of merged part 1
>>>> [1], the hypercall was implemented as the domctl in the beginning but
>>>> later in v2 changed to sysctl. I think this makes sense as the scope of
>>>> that time is just to make Xen aware of the device tree node via Xen device
>>>> tree.
>>>>
>>>> However this is now a problem for the current part where the scope (and
>>>> the end goal) is extended to assign the added device to Linux Dom0/DomU
>>>> via device tree overlays. I am not sure which way is better, should we
>>>> repurposing the sysctl to domctl or maybe add another domctl (I am
>>>> worrying about the duplication because basically we need the same sysctl
>>>> functionality but now with a domid in it)? What do you think?
>>> I am not entirely sure this is a good idea to try to add the device in Xen
>>> and attach it to the guests at the same time. Imagine the following
>>> situation:
>>>
>>> 1) Add and attach devices
>>> 2) The domain is rebooted
>>> 3) Detach and remove devices
>>>
>>> After step 2, you technically have a new domain. You could have also a case
>>> where this is a completely different guest. So the flow would look a little
>>> bit weird (you create the DT overlay with domain A but remove with domain
>>> B).
>>>
>>> So, at the moment, it feels like the add/attach (resp detech/remove)
>>> operations should happen separately.
>>>
>>> Can you clarify why you want to add devices to Xen and attach to a guest
>>> within a single hypercall?
>> Sorry I don't know if there is any specific thoughts on the design of using a
>> single hypercall to do both add devices to Xen device tree and assign the
>> device to the guest. In fact seeing your above comments, I think separating
>> these two functionality to two xl commands using separated hypercalls would
>> indeed be a better idea. Thank you for the suggestion!
>>
>> To make sure I understand correctly, would you mind confirming if below
>> actions for v2 make sense to you? Thanks!
>> - Only use the XEN_SYSCTL_DT_OVERLAY_{ADD, REMOVE} sysctls to add/remove
>> overlay to Xen device tree
>> - Introduce the xl dt-overlay attach <domid> command and respective domctls to
>> do the device assignment for the overlay to domain.
> I think two hypercalls is OK. The original idea was to have a single xl
> command to do the operation for user convenience (even that is not a
> hard requirement) but that can result easily in two hypercalls.

Ok, sounds good. I will break the command to two hypercalls and try to 
reuse the existing domctls for assign/remove IRQ/MMIO ranges.

Kind regards,
Henry



From xen-devel-bounces@lists.xenproject.org Mon May 06 03:17:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 03:17:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717184.1119166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3orK-000662-It; Mon, 06 May 2024 03:17:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717184.1119166; Mon, 06 May 2024 03:17:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3orK-00065v-GF; Mon, 06 May 2024 03:17:46 +0000
Received: by outflank-mailman (input) for mailman id 717184;
 Mon, 06 May 2024 03:17:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3orJ-00065p-Hk
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 03:17:45 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 333d6ad2-0b57-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 05:17:43 +0200 (CEST)
Received: from SJ0PR13CA0003.namprd13.prod.outlook.com (2603:10b6:a03:2c0::8)
 by CH3PR12MB9455.namprd12.prod.outlook.com (2603:10b6:610:1c1::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 03:17:39 +0000
Received: from CO1PEPF000066E8.namprd05.prod.outlook.com
 (2603:10b6:a03:2c0:cafe::6c) by SJ0PR13CA0003.outlook.office365.com
 (2603:10b6:a03:2c0::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 03:17:39 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066E8.mail.protection.outlook.com (10.167.249.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 03:17:38 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:17:35 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 5 May
 2024 22:17:17 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 5 May 2024 22:17:07 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 333d6ad2-0b57-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lbgZPesOC2rLzdSEt85MtUh/n6RTz/M74Wss+KVb7zUL4E8LkRAdqa4zzzgq65tgKu9zGwzyZecZSL40xqhP3X6Xp2LKZA6j7igougWAv9Tg2fXDzCuETkzF+HSXR8Mmo3zNng2Xcvmo34dRSrZ5i2hm0oMhfaEzjjQyhsPpH2L4ZeauyTSIOJXwyNUKZIOOiYnMiQNSDChVKHZEhqcZ1ZzF0ji866rnCYjct0dHjgAtt0pvj1yHxf9KgGH+OoykQucK7GXA7nA3YItxSmQU5jciIE28HjkWLQtUwTHsmqiD8gxTzEGNB0d0WVAn9mt4lqXOpcueCVDWixyLv+Bvrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=d3hk1YdDFiTAoBiixgOKcQVHRL9CcqXxHTnZ3dq1RBs=;
 b=Naux0+UvGuNJ3MWeLUGQGupYhirlQ+wd/+kztJh3DcyIbaXDG3I9R3NYuMivuKO6wiPQKZSSzHamimWdhv5TbmNhIjvEaacy6ANbVHaK06cENb3h64xKDy1DzpA95Mpxo4d77rIZVw/nFv8sUSyxUWOFRoh85XxPTdwPCvqdyvTHXGoBsakLD3OKK2NECAhyrsUfG+T7q0d4Ki793+upIpG7bbSj35PtmsNuk7pA88BEO2FYr4XPzNFV6XAkmsH7brbQKn+B3t8b5G7DK0i3+uGbT1f4a1XvlJ54kodSDhMwTewwk41tNjHyzI+pp+E8xC4IAevGEJFEeAWMJ+HVVA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=cloud.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=d3hk1YdDFiTAoBiixgOKcQVHRL9CcqXxHTnZ3dq1RBs=;
 b=HnUbYtU0+ScnUkKOf4b/Vg2QCLlE2T06lH+tRott7mQa0m+CD3BmoockZ8BIGLbSi/2Nmr4+FE1gjuCwNTRv4BTFam/H/248ilRzABJh+lsETS2ZSyIGW5SXNHB0yPpxhMliKFBQpM8aJTz2JL8L0amnKv+3F8duxy3p+r/4EGg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <5d1848c6-1dfd-4f8e-901d-cbba7c203f57@amd.com>
Date: Mon, 6 May 2024 11:17:06 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 04/15] tools/libs/light: Always enable IOMMU
To: Anthony PERARD <anthony.perard@cloud.com>
CC: <xen-devel@lists.xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-5-xin.wang2@amd.com>
 <7336a5a0-9df0-4e90-846f-9c1647ba5319@perard>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <7336a5a0-9df0-4e90-846f-9c1647ba5319@perard>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066E8:EE_|CH3PR12MB9455:EE_
X-MS-Office365-Filtering-Correlation-Id: f67f547b-5c71-45a9-2a4a-08dc6d7b1595
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?aFZGcFpmL25RdFFSQTY3QlMrcEF0anNVS1d4WGJlT3NuSGNIQWFvMlA0OVNq?=
 =?utf-8?B?eUo0bFJkUFNoS01BYnh4R3c3RFNGTUhWQVpwM0VOSldGVmtVOUl3SHVWTWg5?=
 =?utf-8?B?U25tdWJQeXVBTFp1KzdHYTF2N2ZuYTJReURnS1NPQ2RwOC9yeXFZZWJQK1Jp?=
 =?utf-8?B?VlNJSm00cmhNUEk1cCtJNjBpV1VYR3VCUEcwdjdNcVZwMmdENzREUUR6czVk?=
 =?utf-8?B?Y2hEU0ZwNWhSM2Nua2hxWUdDTW5DQTFLVVdmRXpQWGE0Nzlxd2JENzRQbTUz?=
 =?utf-8?B?am1PN3YyaGFDOVZzUnNPdzF6ZlErS3JIWjBOM2hHZkh0WmxKQW5udEJRemlK?=
 =?utf-8?B?aWZreVN4OTA3cVExTkU3a2dWMEcxR3Y5TktsL1dVN3cxUThQTnRnZFR4SjFs?=
 =?utf-8?B?RDZ1U2J5N3hDMjRITHA4aWdRc2xxK2lCcU5Dc2dybkJwbTFRU3ZZU0ZsVFpi?=
 =?utf-8?B?ZStxK0c4NHdkS2tDV2dFZjFYaWtnaWlsSU1Ra0lqZnozSVJjaTRUVTMvSmpH?=
 =?utf-8?B?eUw1c1BZMEorbDBSS2ZFYTZ4Z20wU2RWVGpRdmx1OEFVUXArdEFnT2xuOTlT?=
 =?utf-8?B?N3FkWkR6VVJBNEdINCtCbk1IWHhNVy9UZ3QveFE5TXhVdGNqT3RwdGtpalgz?=
 =?utf-8?B?ODA5Y3FKT2lhRm5Sa1YrWGJmWEdlZEQ0OFBZeUhPaEg5ak9LRCtYazk0MzVz?=
 =?utf-8?B?dnIzQllOWnlEQ2R5VHhzN2ZlUkcxSFBEK0drQytvTmJ2UGlVQnRuak5jN0VQ?=
 =?utf-8?B?NWhQamhQY3UwMng5RHVTZWgxN2xjUHA3SHJxdG0xY25ybnJtalQ5eHFHdDdh?=
 =?utf-8?B?SHVJeDVsTXlndmVRUHphaXZ4Nm5HQ0NRamgraFZGY1A0RE8zODBpL24zbjZM?=
 =?utf-8?B?eEEwYnBMVEN1LzlVZW1RemlOZVRFeXUxZ0NnRDBMREcvSlo5UUN0UDJFTlVo?=
 =?utf-8?B?bk04eSs2cXdOOXNnM2FYNFlaWUpVbFhTUXpKWktQdk1MUkdpWUEyejB2aUdy?=
 =?utf-8?B?bjZSbXYyWWxFV0ZuNisrU09SMEc0QWNkL2lUajM0T3hFTWk3ZlZzbHArbldG?=
 =?utf-8?B?QVFmdTdxK3NzRGtnQnhBck1IUUFOVjM4RDdlMlVUSlRvb1RrTTdpKzlNcFBG?=
 =?utf-8?B?OTVnUHB5eGNxc0RUc3pRLzdxVUt4Zy90cUd2WnNqeGt6ZzEzWFFhVDlxTkNn?=
 =?utf-8?B?cXVuakdkZC9GMVo2cWJjRng4QzN3MEx1NHJRUUlwUm5rbHpWSDR2OWZYcnRy?=
 =?utf-8?B?WFVnNDY3YmpMSlFyQzI5TU94SEF5WlhtTHlPSUZMR1F0MjBqWmlBU1NYVGdZ?=
 =?utf-8?B?MDNoU2x0VEFmQTBzRW5tRHJyYVBxcEVEUURJL0dGcTdxVEcwY1ZUQ1Y2Y014?=
 =?utf-8?B?NU03QVJ4c1hlNVRUWDV3LzRQS0E1NTBnb3dtVEI1emdqQ0xPTzU4OEQzcHNS?=
 =?utf-8?B?UlU2TWpQT3piVDlVMnV2NlF1c2FnbHc1LytISnhPb0tHQm9xWDVCeXhhdDk0?=
 =?utf-8?B?NFArNUNrY1dRRmlzOFBiNzErcVNob3hLNUZsQWE3MGtNdERXaDFuVHFRM1lJ?=
 =?utf-8?B?M3NUaDVZY0VrZ1Y0bTB1Y0lUdGtaZW9Qbmlkc0RNYmZXYjk5eW54UkFINUlU?=
 =?utf-8?B?NVF0VXREcWFaaENGY09WUlhYNHBldWpSdUFjNVA5K3I3UkkzMkc5THI3Q3Bu?=
 =?utf-8?B?VUk1VzNoUGc4Z3FteUlveFpEdU9YWnRyRTRSZHowVzdjYy94ZXNJc3h6aTF0?=
 =?utf-8?B?Q3hVMXlEcVNJT1dYZ0MvZ3FWMmdLc1oxRnlUUjVOZG9vU05oRlh3bEFXRnN0?=
 =?utf-8?B?Q2V2bTlpM041TlBGblR2cTZzRG93dldWcGJhUnBGcHd4d1VzdEpJZk1YNmxE?=
 =?utf-8?Q?7TRsBPO2CpOwH?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 03:17:38.8551
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f67f547b-5c71-45a9-2a4a-08dc6d7b1595
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066E8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9455

Hi Anthony,

On 5/1/2024 9:47 PM, Anthony PERARD wrote:
> On Wed, Apr 24, 2024 at 11:34:38AM +0800, Henry Wang wrote:
>> For overlay with iommu functionality to work with running VMs, we need
>> to enable IOMMU when iomem presents for the domains.
>>
>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>>   tools/libs/light/libxl_arm.c | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index 1cb89fa584..dd5c9f4917 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -222,6 +222,12 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>           config->arch.sve_vl = d_config->b_info.arch_arm.sve_vl / 128U;
>>       }
>>   
>> +#ifdef LIBXL_HAVE_DT_OVERLAY
> libxl_arm.c is only build on Arm, so this should be defined, so no need
> to check.

Ah sure, I was just thought in the future RISC-V/PPC may have the same, 
but you are correct. I will remove the check.

>> +    if (d_config->b_info.num_iomem) {
>> +        config->flags |= XEN_DOMCTL_CDF_iommu;
> Is this doing the same thing as the previous patch?

I think so, yes, we need the IOMMU flag to be set if we want to assign a 
device from a DT node protected by IOMMU.

Kind regards,
Henry

>
> Thanks,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 05:17:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 05:17:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717190.1119175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3qiy-0005KO-Kp; Mon, 06 May 2024 05:17:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717190.1119175; Mon, 06 May 2024 05:17:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3qiy-0005KH-IF; Mon, 06 May 2024 05:17:16 +0000
Received: by outflank-mailman (input) for mailman id 717190;
 Mon, 06 May 2024 05:17:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3qiw-0005KB-JP
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 05:17:14 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e37f4090-0b67-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 07:17:10 +0200 (CEST)
Received: from BL1PR13CA0374.namprd13.prod.outlook.com (2603:10b6:208:2c0::19)
 by DM4PR12MB6205.namprd12.prod.outlook.com (2603:10b6:8:a8::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.35; Mon, 6 May
 2024 05:17:06 +0000
Received: from BL02EPF0001A100.namprd03.prod.outlook.com
 (2603:10b6:208:2c0:cafe::30) by BL1PR13CA0374.outlook.office365.com
 (2603:10b6:208:2c0::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 05:17:06 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF0001A100.mail.protection.outlook.com (10.167.242.107) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 05:17:06 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:17:05 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:17:05 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 00:17:02 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e37f4090-0b67-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aPKwCA4a3uwF0v3DdYtyqEuNJJBJoIj7ell6THp9eRXIPM/cNd65A+RULXv5w2V3BxyapUV3x9cyfiMzpvm39vOKJ6FRAygA988r7v/+mN9wHdpLgEl34yjUquAsW1BNVlvEWKkjLPoq545dIbS+whEwuJo3Hp9mSlFTgQwunq6oEvjhL+BT2lWPzxnpriblWOaijS9ptLkVO1LCbeN0A73ddbf5anIDVn1RY31/7TqqoU64Kj3BBfpgYmPHLdEwqzJifDNc9Qq4I5dVlI2PcMPUhAF0gx6IU3+bUTrvkdDfRDRhnus2Z766FaOgY3GY7+M+7TvWyQo2xQ0984ZeQw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=l6c0h+MggjM7sWpoLtoclczD4wvLJnexioR+AiveNI4=;
 b=ftRgA9MrG6oqra166S4KqhZNVf0esKHHlUsKxT1glwW+zBjqYVTwwR+n4XgH0jD+DfIRCiAf1bkDDXfcePZJsmo91nsANqL89EKoJUhI6xhbflqv/nv25fYWKzn2FLjGnQo/3z4zabFoqc4g5YJAcLcJJ5H8G0pi5GN/vTtGCdezEHjpbnMjqLRoyNiN1iAarLgNu9gbkKdKoalXF5XPy3Qo1m/2j5Vh8wPBpGy81WVzurwZHqbWx+gVJOjD0tKe1Gy2/0ctsHaNxwOyBkkg0aJ58LES0IX+rPIk/vfHXpzV+4bMloZMA8QYeJoRSnn4lyr/NC1n/ynpRVasnwt22w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=cloud.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=l6c0h+MggjM7sWpoLtoclczD4wvLJnexioR+AiveNI4=;
 b=kD3Iu19krscQagypT98KUfNbbiCMqxwsZJxs9X4XwFXhYTOOfj0Xt+esbTCuM327Ap0BsMpAdA1X9PjRB0CNKsrYGb4KkNAOQWmaOV1l6asl4gxygfYVUZDZxxDDDV5pS0NSiyYPSimGiaPxGOa4BQHwk/LLPapigPX0NGX99qc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <71fc7673-e968-487d-8478-1c7aabef708b@amd.com>
Date: Mon, 6 May 2024 13:17:02 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 05/15] tools/libs/light: Increase nr_spi to 160
To: Anthony PERARD <anthony.perard@cloud.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal Orzel
	<michal.orzel@amd.com>, Bertrand Marquis <bertrand.marquis@arm.com>
CC: <xen-devel@lists.xenproject.org>, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-6-xin.wang2@amd.com>
 <334eb040-234f-4492-9006-9faadfe28c59@perard>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <334eb040-234f-4492-9006-9faadfe28c59@perard>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A100:EE_|DM4PR12MB6205:EE_
X-MS-Office365-Filtering-Correlation-Id: b19f68ab-a4d7-4ca2-eff8-08dc6d8bc5d7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?dTh1K1krbWJBbXo0THVtSW00R0YyekU3d085UmpZbWNwamI1ODgvQmxqckM0?=
 =?utf-8?B?dnlQRCt1NUw5TTFNSkZXSG0zeTdiUnRVNC9tUVVBVEIvWUhuOUNoVi8yYWlt?=
 =?utf-8?B?MDlXVmdzallKeWRhVG91VnhrTG01VzZsQ2JCQnNiemR6dy9kZnRyWTFnbzNa?=
 =?utf-8?B?QXREU2FwUy9sV1lkSllMODhGSlA5UEVRN1BlTG04eHRrSFVkV2dYcFluaDN2?=
 =?utf-8?B?ZHRTbmVtMno3Uyt3SzRDNWF1SUEvSko4Sk1VNUJleCttWTd5RXdpR2RRUEp5?=
 =?utf-8?B?d2Q0eFVmSDYxczJVRVAwUFQrek1yRm9jSG8zWU4zUzJ1N2kwUEY5dGYyYTR1?=
 =?utf-8?B?REs2WHBuU0J2eFNqbVZxWjduR1RxRGg1VytobXMyMjFLY2JtaVpZT3UvYXg0?=
 =?utf-8?B?ckovSmFvUnU3U1JZNDBLeFRxYmFCQ000ZUxwdmloVEFzVnJsUERDWFZZMEJu?=
 =?utf-8?B?Y243UUZET1ZvQ2NaWXNCMmJSK2dTNDN6WG1LUXBGZldMcU13clBPdllNaVA2?=
 =?utf-8?B?eGpvNUNNUjFycWRuenRLaG9DQ1hSZ3g2ZzdQVVE4a3JPeG10ZTY2RG9KZkpN?=
 =?utf-8?B?QXIvZk5VN1pBRW1nbmp2Y3ZXY0ovS0NvajJ4R05DRnVEdkNrRDk5cGpPc1FE?=
 =?utf-8?B?YjR4Z2ZmMHE3MTVDaGJ0a0wxL2VUbWVFSmN1ejZTZndUK2p2ZXFxbW5mVEUv?=
 =?utf-8?B?ekZicVBFdnhtaWJrZFNRU2kwRmRsa1N3Z1lrWXh5KzBPaFpHNStmVm9BSFRu?=
 =?utf-8?B?VmVKSnJrTjBRa1N2YjF4YXFLWVE1ZnB2Z0c3NTNXWDBhbGd3SC9pNDJZRHkr?=
 =?utf-8?B?U1hiemNTdW41SWlJelZXemVCTnJvZmxUWHpsUjE1YzBlaW1tdVF5YlFseXJk?=
 =?utf-8?B?OEFsdTY1Z042WVdNQldYSGVLemhGM3d3UjVqWDFTekJZWitPdDlwVC9LcWhG?=
 =?utf-8?B?VEViREVXaVY0OEtSNnVaR3dnaGRLYzl3RlJ5NlBRWklvT1VjMm1KTVVkbGR2?=
 =?utf-8?B?elZQaGdhcDhPZHNqM0ZnMmh4NTJOdWVVNlZjT3Jnblg5Y2VnVU5qL3FWMjFs?=
 =?utf-8?B?akVQR2hzS0xzekE3RGNtTXh0YlhGbEVBSzdoVGdYYkV4MER0Z2dzbForeXBk?=
 =?utf-8?B?eHEzWXVZYlMyV2hTMEpLZDN3TkZSNTAvQlorTDhYZXhUNWY2RWxlWkw1WFps?=
 =?utf-8?B?RTZKQ05rMHUxblRyOTFPNjFpdDVFbFNidlZpdGFPY1JHZWEraFA0TXZUZjRG?=
 =?utf-8?B?eG91dDRqYTVwQitKQk9RcmRmRzF0NXJNSmdLNnZpSDI2K0V5eExpS2w1ODJQ?=
 =?utf-8?B?S2M0LytpQlRkdGs2WUNEbTR0M0VvcTNNWFp1Tzk0V3UrME0xU2RtTlIweHVK?=
 =?utf-8?B?QnhyUkt5QVRFczVnUi9sTlQ2RWhMOCtYNHZseTkvdU90eFZZTGhFVlloNkla?=
 =?utf-8?B?d2tWRTZSK3pCM1F3VW1TMzA5OG9ITW5CN2VaWmk1Z1hHZVVMQlVWOGJPdi9F?=
 =?utf-8?B?QS9iano4SFJFNFYvNFU3ZDlhUnByK2hqUVRqMUNXVlBqQTJ3Wi96VngwdFRT?=
 =?utf-8?B?VWhveXRJRFZMMnVLYk9tbzQvb2F0U09pMmNjT1FTRDVyVXhvSnVqZmhsazZU?=
 =?utf-8?B?cy90Q0QyNlZzcWRWajhOdE1JWXE0OHZYQlBCTk9ZSEdXOHhVRnBKMm5laDU3?=
 =?utf-8?B?aHN4N0haSjlhNzZVNENRL1lqVTh6WVF0OWZEb1NZbXNWMHo4bjluM3lVd0dP?=
 =?utf-8?B?eXY3VkNuRjRmL3o1bUNsNkhGNWIvbjFtcElQOUZ5aW91YlRkU1oyeGQ3dWt3?=
 =?utf-8?B?L2grd28zMmx0bFIyU2FibTRzUElTN2pBODRtc0ZJT21SUXUyRVV3NTZvRTdR?=
 =?utf-8?Q?yTzirStsVdCqc?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 05:17:06.6444
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b19f68ab-a4d7-4ca2-eff8-08dc6d8bc5d7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A100.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6205

Hi Anthony,

(+Arm maintainers)

On 5/1/2024 9:58 PM, Anthony PERARD wrote:
> On Wed, Apr 24, 2024 at 11:34:39AM +0800, Henry Wang wrote:
>> Increase number of spi to 160 i.e. gic_number_lines() for Xilinx ZynqMP - 32.
>> This was done to allocate and assign IRQs to a running domain.
>>
>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>>   tools/libs/light/libxl_arm.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index dd5c9f4917..50dbd0f2a9 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -181,7 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>   
>>       LOG(DEBUG, "Configure the domain");
>>   
>> -    config->arch.nr_spis = nr_spis;
>> +    /* gic_number_lines() is 192 for Xilinx ZynqMP. min nr_spis = 192 - 32. */
>> +    config->arch.nr_spis = MAX(nr_spis, 160);
> Is there a way that that Xen or libxl could find out what the minimum
> number of SPI needs to be?

I am afraid currently there is none.

> Are we going to have to increase that minimum
> number every time a new platform comes along?
>
> It doesn't appear that libxl is using that `nr_spis` value and it is
> probably just given to Xen. So my guess is that Xen could simply take
> care of the minimum value, gic_number_lines() seems to be a Xen
> function.

Xen will take care of the value of nr_spis for dom0 in create_dom0()
dom0_cfg.arch.nr_spis = min(gic_number_lines(), (unsigned int) 992) - 32;
and also for dom0less domUs in create_domUs().

However, it looks like Xen will not take care of the mininum value for 
libxl guests, the value from config->arch.nr_spis in guest config file 
will be directly passed to the domain_vgic_init() function from 
arch_domain_create().

I agree with you that we shouldn't just bump the number everytime when 
we have a new platform. Therefore, would it be a good idea to move the 
logic in this patch to arch_sanitise_domain_config()?

Kind regards,
Henry

>
> Thanks,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 05:26:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 05:26:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717193.1119186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3qrz-00072l-Hq; Mon, 06 May 2024 05:26:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717193.1119186; Mon, 06 May 2024 05:26:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3qrz-00072e-Ep; Mon, 06 May 2024 05:26:35 +0000
Received: by outflank-mailman (input) for mailman id 717193;
 Mon, 06 May 2024 05:26:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3qry-00072Y-5i
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 05:26:34 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 31a9c99c-0b69-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 07:26:31 +0200 (CEST)
Received: from MN2PR07CA0028.namprd07.prod.outlook.com (2603:10b6:208:1a0::38)
 by DM3PR12MB9325.namprd12.prod.outlook.com (2603:10b6:0:46::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 05:26:27 +0000
Received: from BL02EPF00021F6D.namprd02.prod.outlook.com
 (2603:10b6:208:1a0:cafe::bf) by MN2PR07CA0028.outlook.office365.com
 (2603:10b6:208:1a0::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 05:26:27 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL02EPF00021F6D.mail.protection.outlook.com (10.167.249.9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 05:26:26 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:26:26 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 00:26:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31a9c99c-0b69-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gMFh8rpMaa46Cvhac+sOnnfS5g2HeC6I5X/7teiTMHUeUnFQn/tuZ7P06YS9krzdyI+B/pRYuvlVydfRNR8c2wKnICIKFj5WMUiVXJo+bu1wY2tRj6W1l1VygtWxZ55EeutrPIofRq785OsCSNqCJ/9oNfJYXh8Kpe8DaHzK2wsE6liMsv5s9C8WGNER9K+rvkfUncjqCaqcmhQVIy3YRlm+uHWjieSVE1xGwSsXLH5QzBubF9EFl1oR0P9QDe2OIRfr4RBdEshMPSfJnRNN+OhXKE7sPD5O7GkhDm+0h7Nh98P1/4EjjtB/j5ZnIyDoLilr+6Xsi4T19VJn6DB5/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dMvWbq8eGkuTUPMO+sbko5zSqJnpOZqKBbowDBeFt2A=;
 b=Bu5HEU7TxUWwc0cxw8YGuXaLw6mu1Ud6Ggu/qxcojff/5Ix8ucyU++rN8Uqc63AEJReX1h5qmjMic7fC0EQ/szoJW0xViOc4iAaUksINWe1R1IahZOeVK3aL6ZvNDz+CHqKjoaLWkcGdtDPiQ/7qlzX4V3CFbsMRDlrnByV+O1WwSjMOZnc9LAgWTXJJOa5hKbBPkWHBc40UZGsMWoh9zdgBg7BMMhKkEAZjwnQg2JXQC0xNvCfCam62SJO5wb4ZKQiJrwPc4/gcrZfEQnLE252Z09iIrx7Ww3cX24jO7dpeAx38U/LM8BMI6l+rrv7SkSch8nfYqYKEGggvcUu5UA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dMvWbq8eGkuTUPMO+sbko5zSqJnpOZqKBbowDBeFt2A=;
 b=SlONAVj3mIGuw+SHX6gogH8rNhjg1PrlBb6thvHrKN98HSJug+q/r7eNt402XmPt3dmUgKM3wchZaHUPPqc20Sk4pz96qODefHZUU1JlG+FMqfW5qO9Ca+TnxmJCfPqmivtu53UypiQ97KSG7UxpJbnPXJHdSCCoBt1/wD9tRz8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <7d7d2ac3-47dd-4c64-b16e-29d67958cfbc@amd.com>
Date: Mon, 6 May 2024 13:26:23 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 07/15] xen/overlay: Enable device tree overlay assignment
 to running domains
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, <xen-devel@lists.xenproject.org>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-8-xin.wang2@amd.com>
 <da67f90f-6807-4fdc-b65b-f7a4ba9f78ad@suse.com>
 <97a401ff-4b5c-4e6a-8d9c-e36305ec64f5@amd.com>
 <128c7e68-0431-44e9-b4fc-96cc46e158ad@xen.org>
 <e3c7098b-a876-48f3-8ccb-18a2ced8521c@amd.com>
 <63ad5cf3-5275-4fcd-a1d1-fbf5f3f167f4@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <63ad5cf3-5275-4fcd-a1d1-fbf5f3f167f4@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF00021F6D:EE_|DM3PR12MB9325:EE_
X-MS-Office365-Filtering-Correlation-Id: 93cc985e-f737-491b-ef4d-08dc6d8d1384
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bTFDYmJTNkYvTUNaeE1UQmVURjZ3a0laZnVGVHZsNVZaUEwxME1tdnEwREMr?=
 =?utf-8?B?aHQvTmMxZTVtcDVNZ3JjemNUMG5mNWNqZytYR2Z3dHhHYVhkb00zSG45YUQ5?=
 =?utf-8?B?WjNZNDlRSHdoNmo2R01Fbmk2aGhIaTVSSEVMM2UxTWdMMHd2SndIUEJQT05x?=
 =?utf-8?B?Y0xjbFdiMVpyclEwM0Z5am1MeDV0OWo4NkRid1lGb29VemZoRUtBbUhoUkdR?=
 =?utf-8?B?Mi9sTEdpM3V4ek4yK21XcHpoakkxY0p6VWtjNVk3VHFTMjhDNk1UOW1XYXRj?=
 =?utf-8?B?QnorcE5MaVNldVZuUUhOREJkNTJQRHpEaHo1cjZ6eWNMOVNvODcyVUNzcytr?=
 =?utf-8?B?MW50cEtnL3Z5bUlJTFZDL1hiOUlpNWwvT1NWczhjSzltYkpvZlBuRUd2YkZk?=
 =?utf-8?B?TFJBYXNBakc4dUgzb0VKekpQMTBjVW5GUUgvTm1BcnJGZ2Vhd3RLUi9VYkJX?=
 =?utf-8?B?d3JvV2p0akgxZG10NkplZFJNWjdKREVZWjZZSWJBdlk2MjZmRmZHZDEzeVdK?=
 =?utf-8?B?dHl0eHJJMDNKRGZPUlFzWlpFSW02bFpkcTBzUzJQdmQ2RmttZkdpZmJ5eXlJ?=
 =?utf-8?B?M28wNjFKakRBU2xCYkUydTlZMDFnZ1ZjQm84QTlKWGt3b2VBelpsQW9oMFFq?=
 =?utf-8?B?T2cxZC9kakoxYm91b1d2OE1CcExZVjQ0Z1NhaEJUZGQxSmh0eTRQUlhNUUdF?=
 =?utf-8?B?ZnFLMU1JODdWUEZCL0MzdWttV2F1MkVMcGkyb015V0xDWGhiTzRnTzdydDNV?=
 =?utf-8?B?UmtLaGhJbitjbExmeEhqVGVzbHhpcFVhb1duOUcyL01DRjFGZEpYQVBuK0th?=
 =?utf-8?B?cVFGOHR4Q284aHRhRzNLUGs3cVpCMUg2L1dGcGRKL3l0S3J6UUMxaFRxSGtM?=
 =?utf-8?B?aUdqMisweHBldVpjTXk4ZU50VGtqT3NLVktvZGF2cldIcmZxVGhlTldOMzFL?=
 =?utf-8?B?SzZSUm81OVdwWWhHRFBVVGlyTnhRQzhiMEVnS1R4YVVxaFZaNkRtRTl6OTl5?=
 =?utf-8?B?dG5lamw3dEQwdnNHTFc3eDk3NUhTamFMeDNqNHBRZitxVTRkVzh0U3NiYlpr?=
 =?utf-8?B?N2RhSWEzS2ZBM2pOc0dGR3Qxbm1lcDY1NkRqZGljTys3TFlKR3c5NUM4VmpD?=
 =?utf-8?B?WTNQcjAyT25yMWt6Tkp4VjUyS0lTbWlEN0h3aDNXQk5EM2RnYmFwOFhwQnJu?=
 =?utf-8?B?RVRRdmlqeDJlRWQ3S0RvQU8xQlhwWGxTeU9qd21KVndvZUFuT0VWM09SS3Ax?=
 =?utf-8?B?REMwc3ZsMzFxdVBRbHVtU3pMa01wREdvbmo2K0RBQTNsSXVHZE1wdXlpZEN3?=
 =?utf-8?B?L0w2a1Rna1AwSk8xK3JPZVk1ZHdxcTRHOHdraFdLbURzbnNicXFWY3hOL3Yy?=
 =?utf-8?B?dEdLaUJrWjczaTRGYkk2NVM2QlpoanRuSnhSNlhsZ1FqbW5VajhCZDFKWlRK?=
 =?utf-8?B?SDVycldUVkVQdUwvZkI5NWZleXBkWE5ZUUkwd2wyY0pKcVkrUVFoSFpod2Fm?=
 =?utf-8?B?TWp4bnNtdEFFZzRBK1FqdzA0MHlZaFRIUWg1bjhoUmJ1RHIvZklkRnE3aTJu?=
 =?utf-8?B?WEt5Q1l4YjF1UFdmaWpkTVVhbExuN2Q0QTZlQVFuOWpycGZXN2Z6dWRpaTJV?=
 =?utf-8?B?YkEyeXlFSE9seGZJaEh2aUJBeTdFZmZLckt2R2Z2TXVsUzJpWWhKdFZ5VXdM?=
 =?utf-8?B?eXZObmFyd2toT2hySjF5d01oSHlhRlR0REJhRXU0bGpCQ2RQVTJSWTYzT0xm?=
 =?utf-8?B?YzV2M05oaE1sL29KZ0VROWVIV0xJZG5kMVBPSDNPMEJ4T29XT3lQMUsrellh?=
 =?utf-8?B?REpqVkdJbHhpRFc3ZzNpd3NaeWVSR3NicFFLREYzUWZhNGZlUmM2WG5rT2ZK?=
 =?utf-8?Q?PPfb3vs9HRVAV?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 05:26:26.4750
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 93cc985e-f737-491b-ef4d-08dc6d8d1384
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF00021F6D.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9325

Hi Julien,

On 4/30/2024 5:47 PM, Julien Grall wrote:
> On 30/04/2024 05:00, Henry Wang wrote:
>> Hi Julien,
>
> Hi Henry,
>
>> On 4/30/2024 1:34 AM, Julien Grall wrote:
>>> On 29/04/2024 04:36, Henry Wang wrote:
>>>> Hi Jan, Julien, Stefano,
>>>
>>> Hi Henry,
>>>
>>>> On 4/24/2024 2:05 PM, Jan Beulich wrote:
>>>>> On 24.04.2024 05:34, Henry Wang wrote:
>>>>>> --- a/xen/include/public/sysctl.h
>>>>>> +++ b/xen/include/public/sysctl.h
>>>>>> @@ -1197,7 +1197,9 @@ struct xen_sysctl_dt_overlay {
>>>>>>   #define XEN_SYSCTL_DT_OVERLAY_ADD                   1
>>>>>>   #define XEN_SYSCTL_DT_OVERLAY_REMOVE                2
>>>>>>       uint8_t overlay_op;                     /* IN: Add or 
>>>>>> remove. */
>>>>>> -    uint8_t pad[3];                         /* IN: Must be zero. */
>>>>>> +    bool domain_mapping;                    /* IN: True of 
>>>>>> False. */
>>>>>> +    uint8_t pad[2];                         /* IN: Must be zero. */
>>>>>> +    uint32_t domain_id;
>>>>>>   };
>>>>> If you merely re-purposed padding fields, all would be fine without
>>>>> bumping the interface version. Yet you don't, albeit for an unclear
>>>>> reason: Why uint32_t rather than domid_t? And on top of that - why a
>>>>> separate boolean when you could use e.g. DOMID_INVALID to indicate
>>>>> "no domain mapping"?
>>>>
>>>> I think both of your suggestion make great sense. I will follow the 
>>>> suggestion in v2.
>>>>
>>>>> That said - anything taking a domain ID is certainly suspicious in a
>>>>> sysctl. Judging from the description you really mean this to be a
>>>>> domctl. Anything else will require extra justification.
>>>>
>>>> I also think a domctl is better. I had a look at the history of the 
>>>> already merged series, it looks like in the first version of merged 
>>>> part 1 [1], the hypercall was implemented as the domctl in the 
>>>> beginning but later in v2 changed to sysctl. I think this makes 
>>>> sense as the scope of that time is just to make Xen aware of the 
>>>> device tree node via Xen device tree.
>>>>
>>>> However this is now a problem for the current part where the 
>>>> scope (and the end goal) is extended to assign the added device to 
>>>> Linux Dom0/DomU via device tree overlays. I am not sure which way 
>>>> is better, should we repurposing the sysctl to domctl or maybe add 
>>>> another domctl (I am worrying about the duplication because 
>>>> basically we need the same sysctl functionality but now with a 
>>>> domid in it)? What do you think?
>>>
>>> I am not entirely sure this is a good idea to try to add the device 
>>> in Xen and attach it to the guests at the same time. Imagine the 
>>> following situation:
>>>
>>> 1) Add and attach devices
>>> 2) The domain is rebooted
>>> 3) Detach and remove devices
>>>
>>> After step 2, you technically have a new domain. You could have also 
>>> a case where this is a completely different guest. So the flow would 
>>> look a little bit weird (you create the DT overlay with domain A but 
>>> remove with domain B).
>>>
>>> So, at the moment, it feels like the add/attach (resp detech/remove) 
>>> operations should happen separately.
>
> Thinking a bit more about it, there is another problem with the single 
> hypercall appproach. The MMIOs will be mapped 1:1 to the guest. These 
> region may clash with other part of the layout for domain created by 
> the toolstack
> and dom0less (if the 1:1 option has not been enabled).
>
> I guess for that add, it would be possible to specify the mapping in 
> the Device-Tree. But that would not work for the removal (this may be 
> a different domain).
>
> On a somewhat similar topic, the number of IRQs supported by the vGIC 
> is fixed at boot. How would that work with this patch?

Seeing your comment here I now realized patch #5 is to address this 
issue. But I think we need to have a complete rework of the original 
patch to make the feature portable. We can continue the discussion in 
patch 5.

>>>
>>> Can you clarify why you want to add devices to Xen and attach to a 
>>> guest within a single hypercall?
>>
>> Sorry I don't know if there is any specific thoughts on the design of 
>> using a single hypercall to do both add devices to Xen device tree 
>> and assign the device to the guest. In fact seeing your above 
>> comments, I think separating these two functionality to two xl 
>> commands using separated hypercalls would indeed be a better idea. 
>> Thank you for the suggestion!
>>
>> To make sure I understand correctly, would you mind confirming if 
>> below actions for v2 make sense to you? Thanks!
>> - Only use the XEN_SYSCTL_DT_OVERLAY_{ADD, REMOVE} sysctls to 
>> add/remove overlay to Xen device tree
>
> Note that this would attach the devices to dom0 first. Maybe this is 
> why it was decided to merge the two operations? An option would be to 
> allow the devices to be attached to no-one.
>
>> - Introduce the xl dt-overlay attach <domid> command and respective 
>> domctls to do the device assignment for the overlay to domain.
>
> We already have domctls to route IRQs and map MMIOs. So do we actually 
> need new domctls?

No I don't think so, like you and Stefano said in the other thread, I 
think I need to split the command to different hypercalls instead of 
only one hypercall and reuse the existing domctl.

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 05:40:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 05:40:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717200.1119196 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3r5b-0001N5-Qy; Mon, 06 May 2024 05:40:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717200.1119196; Mon, 06 May 2024 05:40:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3r5b-0001My-O6; Mon, 06 May 2024 05:40:39 +0000
Received: by outflank-mailman (input) for mailman id 717200;
 Mon, 06 May 2024 05:40:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3r5a-0001Mo-Km
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 05:40:38 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 28aba8bb-0b6b-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 07:40:35 +0200 (CEST)
Received: from CH2PR08CA0018.namprd08.prod.outlook.com (2603:10b6:610:5a::28)
 by MW5PR12MB5624.namprd12.prod.outlook.com (2603:10b6:303:19d::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 05:40:33 +0000
Received: from DS3PEPF000099D9.namprd04.prod.outlook.com
 (2603:10b6:610:5a:cafe::14) by CH2PR08CA0018.outlook.office365.com
 (2603:10b6:610:5a::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 05:40:32 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF000099D9.mail.protection.outlook.com (10.167.17.10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 05:40:32 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:40:31 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:40:31 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 00:40:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28aba8bb-0b6b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gsnzpv6mISkJsjG+3p1PfzE9rZJZ7bcbVlEhIu86IR/4KmnECRMWyLIRxBH9pli9b3H2Wz/Wpe8jcGCLHNfjJXpr2oWj91MJLMqlD9TwZbXsARRfIlqI1KYcAToetZxT0mDYPSJ8MAy6XM57I7TSNFTN3l6nkPVo1tFice33nrhoXdnmSz3dLAt8VrJTxoLsxRwNHMXAxR5tqL1coKECbkuILGZkCLXXFIYYt8i/zycZjKUJfAmGIBHlPNxa10/ghvweN3c+Y3qVwdJW9vVA6HKfVCrYcD21PNd8BvlNJWWpEcMWVXcCRY+To53mpnD+pYAmariogvs0yTvfuo1FdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Cuqj2S6r6M1ACM+fj5csnjKIRemFWACC7B5ROJ6ZRqs=;
 b=kDMQlwB58+buozohYfN+BzBiolBNs2f7r9+rucTHi+6JW8DBxpV0yk68320Rqy63c55UarZgdVTbg9LMAHB6PAwd4+DZQ4lQofxQ+9qH+vwQXQjyVdynh6fM1brLJU+Yum7cJnoeQwCr9Kwvw/OFHlyOCmcezVb2+GhogrNA/J4udh53j8xR/BFjOQ/eMHflxGrpipoQXua7CUJlvkMLBnbNIYkxx8LN0Wm/4/5vuxPo5VjunGWvu2ThTor+rAPMm0bcH7vL84G10FGvcQvBuoUCOWSOYIB+42dPRg9C42wfI3FggvWYlwProlEh10n0bpWqdiVeNWxd02BPfJRrxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=cloud.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Cuqj2S6r6M1ACM+fj5csnjKIRemFWACC7B5ROJ6ZRqs=;
 b=Zd6pGSpVqJD5pbZkikJ7dfBp6Ri7SaREvU5GRwPgWcNJxCSrILl4DbGZLNtWjUDBDAUgqMGuH+NQVk7D1KGu0IVY3C/cYmYOEvjERCrKhNVXXeGF4KSkXJBcsUP+xPB8xoJh6hHl1bcqIYM1dQdn2bTbCRMQW2O/ESLoevfqP0w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <fd98b8b8-70f5-43dd-8edd-a78b47b31431@amd.com>
Date: Mon, 6 May 2024 13:40:29 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 09/15] tools/libs/light: Modify dtbo to domU linux dtbo
 format
To: Anthony PERARD <anthony.perard@cloud.com>
CC: <xen-devel@lists.xenproject.org>, Juergen Gross <jgross@suse.com>, Stefano
 Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-10-xin.wang2@amd.com>
 <2c5baf13-7511-4035-91da-02754a2f28c0@perard>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <2c5baf13-7511-4035-91da-02754a2f28c0@perard>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099D9:EE_|MW5PR12MB5624:EE_
X-MS-Office365-Filtering-Correlation-Id: 26aba952-cd4c-4caa-2390-08dc6d8f0bc4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VHNpc1dJdlFmTVk3dEVrbkRFSUNOVG5OZmFwOFlTU1JFWkRFRmlXbXQ0cmNk?=
 =?utf-8?B?V01rN0l6QzdDYmtxbHJsakN6ZW9OT2UyTEFRNTRQOWhqbEllZEpyY3FtYmlw?=
 =?utf-8?B?WWRZVnVvSWhTOUxLKzNuMXRqZVFmSlZRdlhPbHlRdWZqQU1ESG5QMENHcEtT?=
 =?utf-8?B?ZzgwZHBoQjd0MTdhemNZVytCTnVBeXB5ZTZkN3BmR01zdHJqMFlWamVvSWM1?=
 =?utf-8?B?NUJUTDN3SVovRGlrSGFpWFJsMzcxZGVYdFNhUmRHRldyOG0zb2pKSEdCQ3ND?=
 =?utf-8?B?YVVrQUZVSmUvMlNlUnBkNkQvRW9TKytZaElEQ3R3aE90ZmI4b1BnMUN0cTJS?=
 =?utf-8?B?TUdRK2tHMzcrd3dmN2lQOEVVMkhsYzRXaVlJZnN1cGp2QWVCczkvSFhJVXcy?=
 =?utf-8?B?M0xvd05GNHdMblowbzFhaU5FeVMrUmZiQzlJcmtxaFJOUzcxUkdxd1lLQnlu?=
 =?utf-8?B?cG53QXBBZytPT1BiWnZzM1hwMTd3NVZEanZPbmZnMWIrbkhKazBGUklFRFhY?=
 =?utf-8?B?N1NLbkpnL21yV25hMGVjUy9MS09tc2d4R29hY3VPLzdZdWlPdVhpdmpIN1Z1?=
 =?utf-8?B?dFJFNW5VZWtqZ2xEWG03dVgxRU56YlNoaCt0ZklvZlBBejF5OEpaT1FTdjB0?=
 =?utf-8?B?Si9zdVhTUGNiaDRkTk42YnEyMWN3S0ZoZFF0K0Evb0NvSGRKejdZanRMT00w?=
 =?utf-8?B?c3lVTzVRVEo4Z1lFZXN4eVEwem84ZGI3VHEwcDR1T0d3Vmt2TGdyLzgrVXV6?=
 =?utf-8?B?cjFvTDQyTGUyTmhPbWlUdllrU1hmYU1GYjVPMmRVMnJBWnNwNG95L0dpcmNq?=
 =?utf-8?B?T0h6TitpZmNSd2xyR1QzR0NRNVB5dnIrVFlEdDN3c2tNNG9LNEdHMndhcWRN?=
 =?utf-8?B?a0ZBcWZpZExhTVZ1WW9KVVV2NGxjRlN2djgwbmJQMUlrRUY3Q2p2RXZuQUR3?=
 =?utf-8?B?YnYzTFZBVnViOU9Fa2dqdHpvc1pVd2o5TEFoamFqeTJTTkJuRTdRK01qNWJE?=
 =?utf-8?B?ek9uM2pwTkFETVgxWTFSMjRyR0JEcnExSWZQUS9lSDRSSVRuNUNReUVqN0tG?=
 =?utf-8?B?bXphS1FDN1MyT0oxcTliUmhvSmNhMjFoSVgvMDZnMkRrdndvT2NlNHNLSVVX?=
 =?utf-8?B?Um1KNXBmWW13bWxwbE14QWpIMEJNck0wa2hPaFMwczZsMWl3b1pCTnh5WmFI?=
 =?utf-8?B?Z3JOQnprenBkSkNLbTdmbXd1ZVBvS252WllOSE9JSm5GdjBlQmgrN3hod1Vp?=
 =?utf-8?B?VCtMbVRXeGUvVTZJUUd4aVhadlFQa0lOZytQdC9CS2ZoMjdUL3hKUWd2TTZa?=
 =?utf-8?B?NklQdXNlVzF6Ni9wT1RsWEc1dGtkTU5KSnpoc0pDOEo5cWVkZWZzWUdWVUU3?=
 =?utf-8?B?MkRxZjEzSy9ad1JuTW5rQlRxV2hNQ3I0Mm1QdnVubDV6bHZEMG1lOVpUUyts?=
 =?utf-8?B?ZG85VU5JL1hCL0hwdHVhZDBFajlaVzQ2UzlpTEVtRkNFVFhpZU1xeGk3MlFm?=
 =?utf-8?B?dG5zR1FWT21RSTh5emZVbGJoRm1KUEo5RVFjWVlHWlFnUHJBR29KZkVVbDIz?=
 =?utf-8?B?czFENXNtNXNQbXZMZ3gvNnlrVGx3U3E5VW5kazFFMU4vdzNnNlN3U3J6ODBI?=
 =?utf-8?B?Wlp4enpMb2N2dHBrRE9mbjBIZEV0ZHpLOEF6aTNtdWk4RndHcmZjRWp0VXRE?=
 =?utf-8?B?eWJZUGcwUnVJa0poelQ4YnAxMi9wU0wvc0MxVWdDYmg0V1NldXZIUndiRUta?=
 =?utf-8?B?azQ2K25UUnFDTGNnakF5ZGYyNzZ3eUpqM0N2U3ZUMHVFbW1NQVNPSUFMeEJC?=
 =?utf-8?Q?j06p66vJAqEzckJtX0ZdfbnDRJZkjHBrtSON8=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 05:40:32.0234
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 26aba952-cd4c-4caa-2390-08dc6d8f0bc4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099D9.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5624

Hi Anthony,

On 5/1/2024 11:09 PM, Anthony PERARD wrote:
> On Wed, Apr 24, 2024 at 11:34:43AM +0800, Henry Wang wrote:
>> diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
>> index cdb62b28cf..eaf11a0f9c 100644
>> --- a/tools/libs/light/libxl_dt_overlay.c
>> +++ b/tools/libs/light/libxl_dt_overlay.c
>> @@ -41,6 +42,69 @@ static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size)
>>       return 0;
>>   }
>>   
>> +static int modify_overlay_for_domU(libxl__gc *gc, void *overlay_dt_domU,
>> +                                   size_t size)
>> +{
>> +    int rc = 0;
>> +    int virtual_interrupt_parent = GUEST_PHANDLE_GIC;
>> +    const struct fdt_property *fdt_prop_node = NULL;
>> +    int overlay;
>> +    int prop_len = 0;
>> +    int subnode = 0;
>> +    int fragment;
>> +    const char *prop_name;
>> +    const char *target_path = "/";
>> +
>> +    fdt_for_each_subnode(fragment, overlay_dt_domU, 0) {
>> +        prop_name = fdt_getprop(overlay_dt_domU, fragment, "target-path",
>> +                                &prop_len);
>> +        if (prop_name == NULL) {
>> +            LOG(ERROR, "target-path property not found\n");
> LOG* macros already takes care of adding \n, no need to add an extra
> one.

Sure, I will remove the "\n".

>
>> +            rc = ERROR_FAIL;
>> +            goto err;
>> +        }
>> +
>> +        /* Change target path for domU dtb. */
>> +        rc = fdt_setprop_string(overlay_dt_domU, fragment, "target-path",
> fdt_setprop_string() isn't a libxl function, store the return value in a
> variable named `r` instead.'

Thanks for spotting this. Will change it to `r`.

>> +                                target_path);
>> +        if (rc) {
>> +            LOG(ERROR, "Setting interrupt parent property failed for %s\n",
>> +                prop_name);
>> +            goto err;
>> +        }
>> +
>> +        overlay = fdt_subnode_offset(overlay_dt_domU, fragment, "__overlay__");
>> +
>> +        fdt_for_each_subnode(subnode, overlay_dt_domU, overlay)
>> +        {
>> +            const char *node_name = fdt_get_name(overlay_dt_domU, subnode,
>> +                                                 NULL);
>> +
>> +            fdt_prop_node = fdt_getprop(overlay_dt_domU, subnode,
>> +                                        "interrupt-parent", &prop_len);
>> +            if (fdt_prop_node == NULL) {
>> +                LOG(DETAIL, "%s property not found for %s. Skip to next node\n",
>> +                    "interrupt-parent", node_name);
> Why do you have "interrupt-parent" in a separate argument? Do you meant
> to do something like
>      const char *some_name = "interrupt-parent";
> and use that in the 4 different places that this string is used? (Using
> a variable mean that we (or the compiler) can make sure that they are
> all spelled correctly.

Great suggestion! I will do this way.

>> +                continue;
>> +            }
>> +
>> +            rc = fdt_setprop_inplace_u32(overlay_dt_domU, subnode,
>> +                                         "interrupt-parent",
>> +                                         virtual_interrupt_parent);
>> +            if (rc) {
>> +                LOG(ERROR, "Setting interrupt parent property failed for %s\n",
>> +                    "interrupt-parent");
>> +                goto err;
>> +            }
>> +        }
>> +    }
>> +
>> +return 0;
> Missed indentation.

Will correct it.

>> +
>> +err:
>> +    return rc;
> A few things, looks like `rc` is always going to be ERROR_FAIL here,
> unless you find an libxl_error code that better describe the error, so
> you could forgo the `rc` variable.
>
> Also, if you don't need to clean up anything in the function or have a
> generic error message, you could simply "return " instead of using the
> "goto" style.

Sure, I will simply use return because I don't really think there is 
anything to be cleaned up.

>> +}
>> +
>>   int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domid, void *overlay_dt,
>>                        uint32_t overlay_dt_size, uint8_t overlay_op,
>>                        bool auto_mode, bool domain_mapping)
>> @@ -73,6 +137,15 @@ int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domid, void *overlay_dt,
>>           rc = ERROR_FAIL;
>>       }
>>   
>> +    /*
>> +     * auto_mode doesn't apply to dom0 as dom0 can get the physical
>> +     * description of the hardware.
>> +     */
>> +    if (domid && auto_mode) {
>> +        if (overlay_op == LIBXL_DT_OVERLAY_ADD)
> Shouldn't libxl complain if the operation is different?

I will add corresponding error handling code here. Thanks!

Kind regards,
Henry

>> +            rc = modify_overlay_for_domU(gc, overlay_dt, overlay_dt_size);
>> +    }
>> +
>>   out:
>>       GC_FREE;
>>       return rc;
> Thanks,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 05:51:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 05:51:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717204.1119206 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3rGT-0003G3-Ps; Mon, 06 May 2024 05:51:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717204.1119206; Mon, 06 May 2024 05:51:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3rGT-0003Fw-M5; Mon, 06 May 2024 05:51:53 +0000
Received: by outflank-mailman (input) for mailman id 717204;
 Mon, 06 May 2024 05:51:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3rGS-0003Fq-HA
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 05:51:52 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bae8db28-0b6c-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 07:51:50 +0200 (CEST)
Received: from PH7PR17CA0014.namprd17.prod.outlook.com (2603:10b6:510:324::29)
 by CH3PR12MB8582.namprd12.prod.outlook.com (2603:10b6:610:163::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 05:51:42 +0000
Received: from SA2PEPF000015C7.namprd03.prod.outlook.com
 (2603:10b6:510:324:cafe::83) by PH7PR17CA0014.outlook.office365.com
 (2603:10b6:510:324::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Mon, 6 May 2024 05:51:42 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF000015C7.mail.protection.outlook.com (10.167.241.197) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 05:51:41 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 00:51:41 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 00:51:39 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bae8db28-0b6c-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WjKRRowkLLIY0qAijMRkK7blyZ7g19YORdlmSsECMHOWaK2XgtgsZEMG6lvdEvBLaP7BZrLFpxxcvgSJtbjCDU6p26jeC4P71RtpYsI17ouKuiFbErsRfskQ5jVZ/6gk7UPPxuX/E2gbpoIivkCdNdqTbLDM6/XOpn1Q1Q64OKgECRGICGl6QieCFw+scdaiF0/npt5+qs9CmdXr+bN3URXX2hc6feNGRQ4u3pTMEqEsawZpEUgne3l1LIyqr1L9LxoHdXiA7mZpFBT/G6cIwxWEySolFbCt0J7rs8x+mo5SJxD2Utxv0+LfyClN9gWHKL6xLaKqlJu/lGZcmOxWEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wvIWJc+cE+dWxpio20Uhn1m2SQAmGfkqcv+wkxPCj4o=;
 b=Hqa3iEZOC3i69nAA4zpKeqlsqYqe1iW12YGcCjkxpe3xw7qbVGwH4TIphVnDe/jKdJo9Bp3H2esL1lcoYAOzfn5GvVOY8QIW2JVuWV7Rsy7e4NWkxdtrrylDF5CCs4WaOQqgQ8PW08AbdqYZGss3WOHPTCeqbXWpXnx9tcJbuvMwBxLPTN5gCDJFOuDLB7jr9RWqqso64rNxruJpYAd+Y05QJuTmOF8hxhIv9l/pbFNqmf/TvRU1ZW8051FWdzIuGB5GECD35ccFpaAr7lmSHHjrgjXp8VNmaAEmqpA7aOyDglDeXI+5RrIoudb88O+YTd2Nk94NBizckjMrr/syeQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=cloud.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wvIWJc+cE+dWxpio20Uhn1m2SQAmGfkqcv+wkxPCj4o=;
 b=G5Xn19NSoYGq4o1NwW6HgOjF83LTa/EEnMI5mXamxfmONt24GnjuT7bNzcP52l77up4vK50MlB/SIjaripcknZFxlva81o66+h5nBZ8cSM6vuNWrusgwVFVsd5GdSgP/kDbdFw6v8yGNIKncTIj8tqCGppZ23C6vePfQd4tyKFo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <c2519443-1b22-496b-8cbc-0df29b55caf2@amd.com>
Date: Mon, 6 May 2024 13:51:39 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 08/15] tools: Add domain_id and expert mode for overlay
 operations
To: Anthony PERARD <anthony.perard@cloud.com>
CC: <xen-devel@lists.xenproject.org>, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-9-xin.wang2@amd.com>
 <eaea1986-a27e-4d6c-932f-1d0a9918861f@perard>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <eaea1986-a27e-4d6c-932f-1d0a9918861f@perard>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF000015C7:EE_|CH3PR12MB8582:EE_
X-MS-Office365-Filtering-Correlation-Id: 9cee19e7-b166-43f8-77ca-08dc6d909ace
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QVplWEwrOEJ6U01LNjZ2UXdqUjdpQ2xYVnJKVW9NUGhmSkRWb3hoY1lramIw?=
 =?utf-8?B?WExVa1QxN3Z3ZUtVT3FDZGg0bXNzWmJrS0xrWFpRbFI5MFhOQXlaUm1jeVNV?=
 =?utf-8?B?aDBtZUNXRnE0YlMwcnptMnhlUlEwd0FBMk9XSzlYakpuNENqb3ZGSVNnVGdL?=
 =?utf-8?B?U0xYWnUzUWpUQzB0eHNpWSsxdkNIdllsLzBUNzJncyt1eVU1QjQ3cW4rOUpa?=
 =?utf-8?B?aHY4NENHZ0JLSHB0c2NHVVB4MjlrcWs2cVI0cjFhMWhDYndpNHNKa3NWNHRs?=
 =?utf-8?B?bEQzTUVUZXNLdTdncDk2anhZU3RBZyt5bGpmTzFsaG5IcVVIZkRXQmgrcjRI?=
 =?utf-8?B?bjNvZ2xqMEhJN1dxUlpPc1lnaEdUZ0ZoTU1vdGQ2UitoVUpyYUFWcEg2T2xx?=
 =?utf-8?B?REp3L3poVnlmMWdwUVVQKzY0M0ZBREdBRHBCbnNPejVuZk55VzZDcExNY2Fq?=
 =?utf-8?B?dFpDdXFSOExHdUxlUUYxbXdSejZTQ1VxRnNwakZuQnVHUW5YZHJFZDRFZVN6?=
 =?utf-8?B?dVZyNjg2YjV5WmFLamZCaklBWHV6Und5Z1BZS0VPZ1gxWGVCdnZmdUdqOGtG?=
 =?utf-8?B?QjYycUlxWWtPajNIVWptOHpxMGV4V3pGMDBsc2FsMzZqc0ZNTSs0cm43cEdx?=
 =?utf-8?B?Q2RENm1Qc0x2VXROaWwwVmVTSG5MS3VnTm5aSmcyL0k2MTVQbkE4dnNEUjZN?=
 =?utf-8?B?MTVBN2xWS3VvSU14T1ViK0FWb3doOVVJc2xYRGszZHZFMFJyY0xQam5wUkJK?=
 =?utf-8?B?bzRqV0MvSDR5NkF2Y3E1V0ovRmx6RGwzb0ZJYURlS1Nta0E4Z0VPQ2xHUUZm?=
 =?utf-8?B?eCtRajI0U2RHRGNpWVdZWm8zMnFQYmxvTDdvQjlYaFkxM0VTb3psSURkbUx2?=
 =?utf-8?B?Y01XMnRqQ0UxR0F2dzhia1ZhSDhDQWR5UGwxVkZhTk9ROEVCLzIvMUI2clQ0?=
 =?utf-8?B?Vmh3UnhYZUw0SkpLbGlFcDhjODZTaVNXV2JZZHhLWmhLV3VITkhlT3ZBSmZD?=
 =?utf-8?B?WURwT0VSOGpmdWo2M090Q2pQUStDMisyRjVvSnoyN1UyQTV0OEdvVHdybXJq?=
 =?utf-8?B?eU12UDhjbTVJcHJOY1M2UDZvMnUzMWF6VUlIWmxwQUU5KzRuMmUzUjJlc2VN?=
 =?utf-8?B?eXRyUnczQmNLaEFlengwa1R4eGJpMmdmeThBc01vOFNyRFBrUURnVW9RUjlM?=
 =?utf-8?B?WHdUN3VtVVZLa1pFWnUrR29URUhmN1hyTGlRZ2E0VEZhWEJZUi9ZVmhXSjFD?=
 =?utf-8?B?ZGtYTnFwL0kyT1JqempocWgrTnhnaklQY1J2OU1MNHN1TnBxajlpRmJxWFdV?=
 =?utf-8?B?VXpNQlE1WEZSMlZVNnk1c0xDNDJuVkN1aGVzSnpxeElVZXYxUUpDemhWS2ty?=
 =?utf-8?B?UlNtczd1TjczNytlRUxWMjFrcGVHUWRsVVY2S0xxc1NOZWJra1pEbjVOaDlu?=
 =?utf-8?B?NGxaenE4dE9GTS9ERTJrTHZzVjgxZmhoYTR0bnQybkRWNXNwVC9HVUFMM2pT?=
 =?utf-8?B?bmthaFZnbmN5SStEZEhmQXNQaHpkckNRRXVvK1hyMU94cTR6cTdhcVB4NXha?=
 =?utf-8?B?cktZTVlhK0g2bTlRV0VSSVljWi9EWmRGcUtnWXZvMVdTVzd5NnQ5UkgxVjJC?=
 =?utf-8?B?c0wwS0tONW03eTl1eVRRbGc4K1ZmdFNXbGdqUCsxdDBuOGVMeFQwL3ZoRWdn?=
 =?utf-8?B?TFlEb3lNOUFUMU9RTVJkZVVZcnZHNHVFZ2EyOTNCQnBuWW9GRENlcE5pNU9l?=
 =?utf-8?B?eS9mbGozQWhBcDFSajV4aTg5MnZiRnh2OE9oSXNBTkVGTUM4OHRjekl5SDc4?=
 =?utf-8?B?b3JNU1JlM3FjSVlNZEpTMHpqSzB1OEFYcDcwQUo2QWhNaUtoK3dsUnJPSDZH?=
 =?utf-8?Q?HFVBihZ9dj87o?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 05:51:41.9083
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9cee19e7-b166-43f8-77ca-08dc6d909ace
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF000015C7.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8582

Hi Anthony,

On 5/1/2024 10:46 PM, Anthony PERARD wrote:
> On Wed, Apr 24, 2024 at 11:34:42AM +0800, Henry Wang wrote:
>> From: Vikram Garhwal <fnu.vikram@xilinx.com>
>>
>> Add domain_id and expert mode for overlay assignment. This enables dynamic
>> programming of nodes during runtime.
>>
>> Take the opportunity to fix the name mismatch in the xl command, the
>> command name should be "dt-overlay" instead of "dt_overlay".
> I don't like much these unrelated / opportunistic changes in a patch,
> I'd rather have a separate patch. And in this case, if it was on a
> separate patch, that separated patch could gain: Fixes: 61765a07e3d8
> ("tools/xl: Add new xl command overlay for device tree overlay support")
> and potentially backported.

Ok. I can split this part to a separated commit.

>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>>   tools/include/libxl.h               |  8 +++++--
>>   tools/include/xenctrl.h             |  5 +++--
>>   tools/libs/ctrl/xc_dt_overlay.c     |  7 ++++--
>>   tools/libs/light/libxl_dt_overlay.c | 17 +++++++++++----
>>   tools/xl/xl_vmcontrol.c             | 34 ++++++++++++++++++++++++++---
>>   5 files changed, 58 insertions(+), 13 deletions(-)
>>
>> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
>> index 62cb07dea6..59a3e1b37c 100644
>> --- a/tools/include/libxl.h
>> +++ b/tools/include/libxl.h
>> @@ -2549,8 +2549,12 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
>>   void libxl_device_pci_list_free(libxl_device_pci* list, int num);
>>   
>>   #if defined(__arm__) || defined(__aarch64__)
>> -int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
>> -                     uint32_t overlay_size, uint8_t overlay_op);
>> +#define LIBXL_DT_OVERLAY_ADD                   1
>> +#define LIBXL_DT_OVERLAY_REMOVE                2
>> +
>> +int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay,
>> +                     uint32_t overlay_size, uint8_t overlay_op, bool auto_mode,
>> +                     bool domain_mapping);
> Sorry, you cannot change the API of an existing libxl function without
> providing something backward compatible. We have already a few example
> of this changes in libxl.h, e.g.: fded24ea8315 ("libxl: Make libxl_set_vcpuonline async")
> So, providing a wrapper called libxl_dt_overlay_0x041800() which call
> the new function.

Ok, I will add an wrapper.

>>   #endif
>>   
>>   /*
>> diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
>> index a6c709a6dc..cdb62b28cf 100644
>> --- a/tools/libs/light/libxl_dt_overlay.c
>> +++ b/tools/libs/light/libxl_dt_overlay.c
>> @@ -57,10 +58,18 @@ int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
>>           rc = 0;
>>       }
>>   
>> -    r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
>> +    /* Check if user entered a valid domain id. */
>> +    rc = libxl_domain_info(CTX, NULL, domid);
>> +    if (rc == ERROR_DOMAIN_NOTFOUND) {
> Why do you check specifically for "domain not found", what about other
> error?

I agree this is indeed very confusing...I will rewrite this part 
properly in the next version.

>> +        LOGD(ERROR, domid, "Non-existant domain.");
>> +        return ERROR_FAIL;
> Use `goto out`, and you can let the function return
> ERROR_DOMAIN_NOTFOUND if that the error, we can just propagate the `rc`
> from libxl_domain_info().

Sure, will do the suggested way.

>> +    }
>> +
>> +    r = xc_dt_overlay(ctx->xch, domid, overlay_dt, overlay_dt_size, overlay_op,
>> +                      domain_mapping);
>>   
>>       if (r) {
>> -        LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
>> +        LOG(ERROR, "domain%d: Adding/Removing overlay dtb failed.", domid);
> You could replace the macro by LOGD, instead of handwriting "domain%d".

Great suggestion. I will use LOGD.

>>           rc = ERROR_FAIL;
>>       }
>>   
>> diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
>> index 98f6bd2e76..9674383ec3 100644
>> --- a/tools/xl/xl_vmcontrol.c
>> +++ b/tools/xl/xl_vmcontrol.c
>> @@ -1270,21 +1270,48 @@ int main_dt_overlay(int argc, char **argv)
>>   {
>>       const char *overlay_ops = NULL;
>>       const char *overlay_config_file = NULL;
>> +    uint32_t domain_id = 0;
>>       void *overlay_dtb = NULL;
>>       int rc;
>> +    bool auto_mode = true;
>> +    bool domain_mapping = false;
>>       uint8_t op;
>>       int overlay_dtb_size = 0;
>>       const int overlay_add_op = 1;
>>       const int overlay_remove_op = 2;
>>   
>> -    if (argc < 2) {
>> -        help("dt_overlay");
>> +    if (argc < 3) {
>> +        help("dt-overlay");
>>           return EXIT_FAILURE;
>>       }
>>   
>> +    if (argc > 5) {
>> +        fprintf(stderr, "Too many arguments\n");
>> +        return ERROR_FAIL;
>> +    }
>> +
>>       overlay_ops = argv[1];
>>       overlay_config_file = argv[2];
>>   
>> +    if (!strcmp(argv[argc - 1], "-e"))
>> +        auto_mode = false;
>> +
>> +    if (argc == 4 || !auto_mode) {
>> +        domain_id = find_domain(argv[argc-1]);
>> +        domain_mapping = true;
>> +    }
>> +
>> +    if (argc == 5 || !auto_mode) {
>> +        domain_id = find_domain(argv[argc-2]);
>> +        domain_mapping = true;
>> +    }
> Sorry, I can't review that changes, this needs a change in the help
> message of dt-overlay, and something in the man page. (and that argument
> parsing looks convoluted).

I will rework this part to see if I can make it better.

>> +
>> +    /* User didn't prove any overlay operation. */
> I guess you meant "provide" instead of prove. But the comment can be
> discarded, it doesn't explain anything useful that the error message
> doesn't already explain.

I think your comment is correct, I will just drop it.

>> +    if (overlay_ops == NULL) {
>> +        fprintf(stderr, "No overlay operation mode provided\n");
>> +        return ERROR_FAIL;
> That should be EXIT_FAILURE instead. (and I realise that the function
> already return ERROR_FAIL by mistake in several places. (ERROR_FAIL is a
> libxl error return value of -3, which isn't really a good exit value for
> CLI programmes.))

Thanks. Will use EXIT_FAILURE.

Kind regards,
Henry

>
> Thanks,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 06:33:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 06:33:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717213.1119216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3rub-0001CH-Oj; Mon, 06 May 2024 06:33:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717213.1119216; Mon, 06 May 2024 06:33:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3rub-0001CA-Lb; Mon, 06 May 2024 06:33:21 +0000
Received: by outflank-mailman (input) for mailman id 717213;
 Mon, 06 May 2024 06:33:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3rua-0001C4-52
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 06:33:20 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 861dbbab-0b72-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 08:33:17 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d8a24f8a3cso18590211fa.1
 for <xen-devel@lists.xenproject.org>; Sun, 05 May 2024 23:33:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ka21-20020a05600c585500b0041a9c3444a6sm18389949wmb.28.2024.05.05.23.33.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 05 May 2024 23:33:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 861dbbab-0b72-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714977197; x=1715581997; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MLjztR0JpobJ3JhIJikqgg4wdwh+y3Eegwp1h1Me+1Y=;
        b=K7qNQC0IAvbwbK9FIRIr58ZCRyPbD51t2OQ6EKFLhya4UoiYRytgS5OQCcZbl1Pl+E
         1MVDp0dq9P8NaQo+MA1xN+vORB7ou/5VhG80oArrp+NZ+ALHA+jKr93YlFzxINpjhbK5
         fDl1pFijZz9T2yYKzmmk8asiwYSHqJcS66T+UYQKJcLGyb0hqVVEZLGSeWnNixvM38G5
         mQVeluShaFje/uImdoo/kkC5QLewGz9vptl/RnSLON8s8KOSrq5jX6mL+Qfh60VA8Tgi
         gO5SOy1GINm4IdlYxGPfSFUqTFkUDSLA7nJ6/6aUF8IufwrOy1GRIyHKDm4nq8soxWga
         qnBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714977197; x=1715581997;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MLjztR0JpobJ3JhIJikqgg4wdwh+y3Eegwp1h1Me+1Y=;
        b=Zs86RDG1sHzVMGefnCFHGR8stwa5I7Vkvcdlr6uCUcSbsqKS68JAk/8IohP3KCv48x
         WWe22pelENJYRngjaYzbrkteiflCbRrTk6DXCw1hdjrgloznSZ2MgrzdT/DZx3sSWVMS
         3dHF0JfRVFKbRYD3xmOIe+fadjHEIa3mDOtWJSLXDmlf7UBivWrmtNIRpuIKXTNpGz9d
         H+qEnzuU0APDdCf7stSUMwK1x+7y26jOZVDKLcxMmJhP0I6jAS3hp6rb/QsYUDqW7qsR
         UkL+DmyuWyrUCnwK6WT8zfhoL6r+IJ/cIfkhVSQWpIGnuV5CTtunxN7FBCzCsQI2r9wI
         oAeQ==
X-Forwarded-Encrypted: i=1; AJvYcCUEvw0h9Ny9AjCCkh3ppQ6Bjha2yyidC7K7r+H3whjXbE7c453+2nseam9DtMl7Nbgf4fKDejO+ysHTywvj5fXsw3irIqnqF061GnIzqYY=
X-Gm-Message-State: AOJu0Yy9XvTfZrowLIIDyL6sEA0c9W+bXMfwMAUdzvlLnV/CC3TTNnVM
	ELMdW9i96GErzoTISZHGiqAFgXsVCXd2Jm1GvZyoI2ZuaXi3Ixehop/Rypg6ng==
X-Google-Smtp-Source: AGHT+IGwyuIzosbBu5vfVd9tV76AZPwCvf77BCcHCkSFXJrAWS+QbLpG/SLBmc+gdASv2S2zjSE2dA==
X-Received: by 2002:a05:651c:201c:b0:2dd:c9fc:c472 with SMTP id s28-20020a05651c201c00b002ddc9fcc472mr5387008ljo.26.1714977197436;
        Sun, 05 May 2024 23:33:17 -0700 (PDT)
Message-ID: <93eeb1d8-d41f-40e8-8ca2-e6828877b53d@suse.com>
Date: Mon, 6 May 2024 08:33:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 02/17] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1713347222.git.oleksii.kurochko@gmail.com>
 <1a0977e3cf5a2de9f760ca5ec89a0d096894a9e3.1713347222.git.oleksii.kurochko@gmail.com>
 <3827c11c-6d47-411d-a356-871def4e5b30@suse.com>
 <940f11ee23b43ada1dba50bc0236c4764eb13d71.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <940f11ee23b43ada1dba50bc0236c4764eb13d71.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.05.2024 19:15, Oleksii wrote:
> On Thu, 2024-04-25 at 17:35 +0200, Jan Beulich wrote:
>>>   #include <asm/bitops.h>
>>>   
>>> +#ifndef arch_check_bitop_size
>>> +#define arch_check_bitop_size(addr)
>>
>> Can this really do nothing? Passing the address of an object smaller
>> than
>> bitop_uint_t will read past the object in the generic__*_bit()
>> functions.
> It seems RISC-V isn' happy with the following generic definition:
>    extern void __bitop_bad_size(void);
>    
>    /* --------------------- Please tidy above here --------------------
>    - */
>    
>    #include <asm/bitops.h>
>    
>    #ifndef arch_check_bitop_size
>    
>    #define bitop_bad_size(addr) sizeof(*(addr)) < sizeof(bitop_uint_t)
>    
>    #define arch_check_bitop_size(addr) \
>        if ( bitop_bad_size(addr) ) __bitop_bad_size();
>    
>    #endif /* arch_check_bitop_size */

I'm afraid you've re-discovered something that was also found during the
original Arm porting effort. As nice and logical as it would (seem to) be
to have bitop_uint_t match machine word size, there are places ...

> The following errors occurs. bitop_uint_t for RISC-V is defined as
> unsigned long for now:

... where we assume such operations can be done on 32-bit quantities.

Jan

>     ./common/symbols-dummy.o -o ./.xen-syms.0
> riscv64-linux-gnu-ld: prelink.o: in function `atomic_sub':
> /build/xen/./arch/riscv/include/asm/atomic.h:152: undefined reference
> to `__bitop_bad_size'
> riscv64-linux-gnu-ld: prelink.o: in function `evtchn_check_pollers':
> /build/xen/common/event_channel.c:1531: undefined reference to
> `__bitop_bad_size'
> riscv64-linux-gnu-ld: /build/xen/common/event_channel.c:1521: undefined
> reference to `__bitop_bad_size'
> riscv64-linux-gnu-ld: prelink.o: in function `evtchn_init':
> /build/xen/common/event_channel.c:1541: undefined reference to
> `__bitop_bad_size'
> riscv64-linux-gnu-ld: prelink.o: in function `_read_lock':
> /build/xen/./include/xen/rwlock.h:94: undefined reference to
> `__bitop_bad_size'
> riscv64-linux-gnu-ld:
> prelink.o:/build/xen/./arch/riscv/include/asm/atomic.h:195: more
> undefined references to `__bitop_bad_size' follow
> riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `__bitop_bad_size'
> isn't defined
> riscv64-linux-gnu-ld: final link failed: bad value
> make[2]: *** [arch/riscv/Makefile:15: xen-syms] Error 1
> 
> ~ Oleksii



From xen-devel-bounces@lists.xenproject.org Mon May 06 06:42:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 06:42:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717219.1119226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3s3C-0003CL-LG; Mon, 06 May 2024 06:42:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717219.1119226; Mon, 06 May 2024 06:42:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3s3C-0003CE-Ia; Mon, 06 May 2024 06:42:14 +0000
Received: by outflank-mailman (input) for mailman id 717219;
 Mon, 06 May 2024 06:42:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3s3A-0003C8-OQ
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 06:42:12 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c36bb86e-0b73-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 08:42:10 +0200 (CEST)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-34d7d04808bso1136444f8f.0
 for <xen-devel@lists.xenproject.org>; Sun, 05 May 2024 23:42:10 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s14-20020adfe00e000000b0034e553f6174sm8312875wrh.98.2024.05.05.23.42.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 05 May 2024 23:42:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c36bb86e-0b73-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714977730; x=1715582530; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cN6g1BGIp4j/LoLsnnkV3ci966DWyGYF5iZ47Vj9pm8=;
        b=gg3HvPHjBQ1Bs8pxrmb06PdbdmZLEPq1AW2wQM6Tsf9On+CfzioXDE/5Y9JI8ADW5S
         zpSxozVGepvcKA488A+R/Jk0EqrU3LfyxNlSJMX7/hsIMeODXmhayQI0vxCImAiM04he
         0+bylt2CnFXOoZsH5XqoEYuJ1aakTxYXXvdlrIUtk907Bmm8MC7styCkNHC9WjUc3LgO
         9GfeCy1PHRfAcCOqedug//87DxMf77KD0UnRIB95Im4+5QGD+2sZ9Aq49rPDzmRZT+FG
         GLJQBVcLqI6gN6HAbtl7CfnGiz9aVp8mjDSELeN7kxrkmySXeuDmwpyU+Ac6UXhvyaXQ
         P5wQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714977730; x=1715582530;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cN6g1BGIp4j/LoLsnnkV3ci966DWyGYF5iZ47Vj9pm8=;
        b=r/O5U4woMNlK/qcfMH6cxzFWMehmiupjlXFYtWzGOedTIgIl1SjaUDnY4gSlrPKi4l
         OMvSTar5ESmqrNBGBpDc8NgWGczJn9OazTIl4AAcXsmcpQy1becN/8FpVJPX89RZ0/SF
         1ymGTDqyA6KzZQD6ONQxuG9ANdyCB0R0QvDlL4YMrmILLO6G15S8UVWzE5JzAiEdEkrl
         eWUEoe0gUgnkylDb6DxyAg2A7CzvVhye+RKYaa2rozXAQvmGWQHNVuFfSq2bVQ0hP9vF
         YaQIMrnupf5qXbSJjrBJ9b+5/22qLnUy7dAScVPtM9N4OHAQthr5Q0EIhk9mgeStM1M3
         Vqiw==
X-Forwarded-Encrypted: i=1; AJvYcCWNe1DU1bmtQd52Ra/mPi5GhXrnVQlzhTjB/0m7GwzOi7hxxAuIviVh9PivFSyUk37zVJY97vRd/JVQ7/v1pZP4geURkwD4F9MoPhmzlZY=
X-Gm-Message-State: AOJu0YzouTDeH9pTwkWkBPp/rYivr5pAwt6PUBAvrrxPQcxiyreEu9dY
	wTwd0TEgv0F67/CNqWmdLzjGkXvo8NNMEB8yEyD0smP/Eq3jstTEhaIchSSiYg==
X-Google-Smtp-Source: AGHT+IGuhe4X/fvtSWsahosVEUGDhRuomehpx7bMGYj0+y+748CxofBCY81b2CVj67sJCCEmxHmAFA==
X-Received: by 2002:a5d:4f8f:0:b0:34d:b265:6401 with SMTP id d15-20020a5d4f8f000000b0034db2656401mr8518262wru.46.1714977729852;
        Sun, 05 May 2024 23:42:09 -0700 (PDT)
Message-ID: <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
Date: Mon, 6 May 2024 08:42:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org,
 Julien Grall <julien@xen.org>
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
 <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
 <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.05.2024 21:07, Stefano Stabellini wrote:
> On Fri, 3 May 2024, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 02/05/2024 19:13, Stefano Stabellini wrote:
>>> On Mon, 29 Apr 2024, Julien Grall wrote:
>>>> Hi Juergen,
>>>>
>>>> On 29/04/2024 12:28, Jürgen Groß wrote:
>>>>> On 29.04.24 13:04, Julien Grall wrote:
>>>>>> Hi Juergen,
>>>>>>
>>>>>> Sorry for the late reply.
>>>>>>
>>>>>> On 29/04/2024 11:33, Juergen Gross wrote:
>>>>>>> On 08.04.24 09:10, Jan Beulich wrote:
>>>>>>>> On 27.03.2024 16:22, Juergen Gross wrote:
>>>>>>>>> With lock handling now allowing up to 16384 cpus (spinlocks can
>>>>>>>>> handle
>>>>>>>>> 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed
>>>>>>>>> limit
>>>>>>>>> for
>>>>>>>>> the number of cpus to be configured to 16383.
>>>>>>>>>
>>>>>>>>> The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
>>>>>>>>> QINVAL_MAX_ENTRY_NR required to be larger than 2 *
>>>>>>>>> CONFIG_NR_CPUS.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>>>>
>>>>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>
>>>>>>>> I'd prefer this to also gain an Arm ack, though.
>>>>>>>
>>>>>>> Any comment from Arm side?
>>>>>>
>>>>>> Can you clarify what the new limits mean in term of (security)
>>>>>> support?
>>>>>> Are we now claiming that Xen will work perfectly fine on platforms
>>>>>> with up
>>>>>> to 16383?
>>>>>>
>>>>>> If so, I can't comment for x86, but for Arm, I am doubtful that it
>>>>>> would
>>>>>> work without any (at least performance) issues. AFAIK, this is also an
>>>>>> untested configuration. In fact I would be surprised if Xen on Arm was
>>>>>> tested with more than a couple of hundreds cores (AFAICT the Ampere
>>>>>> CPUs
>>>>>> has 192 CPUs).
>>>>>
>>>>> I think we should add a security support limit for the number of
>>>>> physical
>>>>> cpus similar to the memory support limit we already have in place.
>>>>>
>>>>> For x86 I'd suggest 4096 cpus for security support (basically the limit
>>>>> we
>>>>> have with this patch), but I'm open for other suggestions, too.
>>>>>
>>>>> I have no idea about any sensible limits for Arm32/Arm64.
>>>>
>>>> I am not entirely. Bertrand, Michal, Stefano, should we use 192 (the
>>>> number of
>>>> CPUs from Ampere)?
>>>
>>> I am OK with that. If we want to be a bit more future proof we could say
>>> 256 or 512.
>>
>> Sorry, I don't follow your argument. A limit can be raised at time point in
>> the future. The question is more whether we are confident that Xen on Arm will
>> run well if a user has a platform with 256/512 pCPUs.
>>
>> So are you saying that from Xen point of view, you are expecting no difference
>> between 256 and 512. And therefore you would be happy if to backport patches
>> if someone find differences (or even security issues) when using > 256 pCPUs?
> 
> It is difficult to be sure about anything that it is not regularly
> tested. I am pretty sure someone in the community got Xen running on an
> Ampere, so like you said 192 is a good number. However, that is not
> regularly tested, so we don't have any regression checks in gitlab-ci or
> OSSTest for it.
> 
> One approach would be to only support things regularly tested either by
> OSSTest, Gitlab-ci, or also Xen community members. I am not sure what
> would be the highest number with this way of thinking but likely no
> more than 192, probably less. I don't know the CPU core count of the
> biggest ARM machine in OSSTest.
> 
> Another approach is to support a "sensible" number: not something tested
> but something we believe it should work. No regular testing. (In safety,
> they only believe in things that are actually tested, so this would not
> be OK. But this is security, not safety, just FYI.) With this approach,
> we could round up the number to a limit we think it won't break. If 192
> works, 256/512 should work? I don't know but couldn't think of something
> that would break going from 192 to 256.

I would suggest to aim at sticking to power-of-2 values. There are still
some calculations in Xen which can  be translated to more efficient code
that way (mainly: using shifts rather than multiplications or a
combination of shifts and adds). Of course those calculations depend on
what people choose as actual values, but giving an upper bound being a
power of 2 may at least serve as a hint to them.

> It depends on how strict we want to be on testing requirements. I am not
> sure what approach was taken by x86 so far. I am OK either way.

The bumping of the limit here clearly is forward-looking for x86, i.e. is
unlikely to be even possible to test right now (except maybe when running
Xen itself virtualized). I actually think there need to be two separate
considerations: One is towards for how many CPUs Xen can be built (and
such a build can be validated on a much smaller system), while another is
to limit what is supported (in ./SUPPORT.md).

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 06:45:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 06:45:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717226.1119237 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3s6f-0004EI-5p; Mon, 06 May 2024 06:45:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717226.1119237; Mon, 06 May 2024 06:45:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3s6f-0004EB-0p; Mon, 06 May 2024 06:45:49 +0000
Received: by outflank-mailman (input) for mailman id 717226;
 Mon, 06 May 2024 06:45:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VNYQ=MJ=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s3s6e-0004E5-C1
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 06:45:48 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 43d4e18f-0b74-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 08:45:45 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 52CBF4EE074D;
 Mon,  6 May 2024 08:45:45 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43d4e18f-0b74-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Mon, 06 May 2024 08:45:45 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Federico Serafini <federico.serafini@bugseng.com>,
 xen-devel@lists.xenproject.org, consulting@bugseng.com, Simone Ballarin
 <simone.ballarin@bugseng.com>, Doug Goldstein <cardoe@cardoe.com>, Stefano
 Stabellini <sstabellini@kernel.org>
Subject: Re: [REGRESSION] Re: [XEN PATCH 0/3] automation/eclair: do not allow
 failure for triggered analyses
In-Reply-To: <b2857b11-9ae6-4768-bd0d-8c93b9a3d604@citrix.com>
References: <cover.1714401209.git.federico.serafini@bugseng.com>
 <d24c9ef6-a7c8-4ee0-baff-b1585c788db1@citrix.com>
 <ecebd04a-1b95-4055-9f1c-a5468cc3bb1a@bugseng.com>
 <e4468587-318a-44f2-ba66-9a9c5e504052@citrix.com>
 <518e455a-4233-498c-b7b1-afa485508543@bugseng.com>
 <b2857b11-9ae6-4768-bd0d-8c93b9a3d604@citrix.com>
Message-ID: <8ddfbc0d6b20238cb902ccfccb2310fc@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-05-03 22:06, Andrew Cooper wrote:
> On 03/05/2024 9:01 pm, Federico Serafini wrote:
>> On 03/05/24 21:46, Andrew Cooper wrote:
>>> On 03/05/2024 8:44 pm, Federico Serafini wrote:
>>>> On 03/05/24 21:14, Andrew Cooper wrote:
>>>>> On 29/04/2024 4:21 pm, Federico Serafini wrote:
>>>>>> Patch 1/3 does some preparation work.
>>>>>> 

Hi,

>>>>>> Patch 2/3, as the title says, removes allow_failure = true for
>>>>>> triggered
>>>>>> analyses.
>>>>>> 
>>>>>> Patch 3/3 makes explicit that initally no files are tagged as
>>>>>> adopted, this
>>>>>> is needed by the scheduled analysis.
>>>>> 
>>>>> I'm afraid that something in this series is broken.
>>>>> 
>>>>> Since these patches went in, all pipelines are now getting a status 
>>>>> of
>>>>> blocked rather than passed.
>>>>> 
>>>>> If I manually start the Eclair jobs, then eventually the pipeline 
>>>>> gets
>>>>> to Passed.
>>>> 
>>>> Can you provide us a link to those failures?
>>>> I am looking at gitlab xen-project/xen and xen-project/patchew
>>>> and everything seems ok.
>>>> 
>>> 
>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
>>> is the first one I noticed as blocked, and I manually ran.  That 
>>> ended
>>> up working.
>>> 
>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
>>> is still in the blocked state.  The build-each-commit failure is
>>> unrelated.
>> 
>> This is intentional and was introduced by
>> commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6
>> 
>> The ECLAIR analysis under people/* need to be activated
>> manually.
> 
> Yes.  I know, and that matches the behaviour I saw.
> 
>> 
>> Is this causing some problems to the CI?
>> 
> 
> Yes.
> 
> See https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines
> 
> Prior to this series, the manual actions were not used but the pipeline
> was overall in the Passed state.  Specifically, they ended up being 
> skipped.
> 
> After this series, the manual actions are now blocking the pipeline, 
> not
> letting it complete, and not marking it as passed.

one way to fix this can be to set allow_failure when in the people/* 
branches, which will cause the stage to be skipped and thus mark the 
pipeline as Passed when all other steps have completed. See 
https://gitlab.com/xen-project/people/bugseng/xen/-/pipelines/1278550868


-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon May 06 06:53:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 06:53:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717231.1119246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3sDt-0005jR-PT; Mon, 06 May 2024 06:53:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717231.1119246; Mon, 06 May 2024 06:53:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3sDt-0005jK-Mc; Mon, 06 May 2024 06:53:17 +0000
Received: by outflank-mailman (input) for mailman id 717231;
 Mon, 06 May 2024 06:53:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ndfh=MJ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s3sDs-0005jD-Sy
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 06:53:17 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4fae18c8-0b75-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 08:53:15 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 8B0365F8C5;
 Mon,  6 May 2024 06:53:14 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3E3961386E;
 Mon,  6 May 2024 06:53:14 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Ms9YDVp+OGZITgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 06 May 2024 06:53:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fae18c8-0b75-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714978394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=n0uDFq1JdIdpjvRx2YlMSlAflID/a3NCbpUN7X2V+DA=;
	b=OLZVEMpdPjeYqfKPqJh4+H4Yg0jWfAmwSpyqz4n0I3dfcZLhFAfslxvEaIT2cq65zi/PHW
	mxBo5mCdlJI91IGGSDFx73Ed0ZHPbnYHsbueRu/q6FrcMI7g3L5BAWwHoRT/B9hoauB3Ps
	x3FiNrVOswlf4kM58MzC+Pd8aI0HlsY=
Authentication-Results: smtp-out2.suse.de;
	none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1714978394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=n0uDFq1JdIdpjvRx2YlMSlAflID/a3NCbpUN7X2V+DA=;
	b=OLZVEMpdPjeYqfKPqJh4+H4Yg0jWfAmwSpyqz4n0I3dfcZLhFAfslxvEaIT2cq65zi/PHW
	mxBo5mCdlJI91IGGSDFx73Ed0ZHPbnYHsbueRu/q6FrcMI7g3L5BAWwHoRT/B9hoauB3Ps
	x3FiNrVOswlf4kM58MzC+Pd8aI0HlsY=
Message-ID: <f31db1df-0d06-4a62-9761-86165a6d986c@suse.com>
Date: Mon, 6 May 2024 08:53:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org,
 Julien Grall <julien@xen.org>
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
 <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
 <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
 <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------xL5qX0QquxcPCH6DBRNX9Mu9"
X-Spam-Flag: NO
X-Spam-Score: -6.18
X-Spam-Level: 
X-Spamd-Result: default: False [-6.18 / 50.00];
	BAYES_HAM(-3.00)[100.00%];
	SIGNED_PGP(-2.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.19)[-0.971];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	XM_UA_NO_VERSION(0.01)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	RCPT_COUNT_SEVEN(0.00)[8];
	MID_RHS_MATCH_FROM(0.00)[];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	ARC_NA(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	TO_DN_SOME(0.00)[];
	FROM_EQ_ENVFROM(0.00)[];
	FROM_HAS_DN(0.00)[];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RCVD_COUNT_TWO(0.00)[2];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	HAS_ATTACHMENT(0.00)[]

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------xL5qX0QquxcPCH6DBRNX9Mu9
Content-Type: multipart/mixed; boundary="------------fKn1louHxgGR7JmYlewZgRWC";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org,
 Julien Grall <julien@xen.org>
Message-ID: <f31db1df-0d06-4a62-9761-86165a6d986c@suse.com>
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
 <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
 <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
 <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
In-Reply-To: <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
Autocrypt-Gossip: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJ3BBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AAIQkQoDSui/t3IH4WIQQ+pJkfkcoLMCa4X6CgNK6L+3cgfgn7AJ9DmMd0SMJE
 ePbc7/m22D2v04iu7ACffXTdZQhNl557tJuDXZSBxDmW/tLOwU0EWTecRBAIAIK5OMKMU5R2
 Lk2bbjgX7vyQuCFFyKf9rC/4itNwhYWFSlKzVj3WJBDsoi2KvPm7AI+XB6NIkNAkshL5C0kd
 pcNd5Xo0jRR5/WE/bT7LyrJ0OJWS/qUit5eNNvsO+SxGAk28KRa1ieVLeZi9D03NL0+HIAtZ
 tecfqwgl3Y72UpLUyt+r7LQhcI/XR5IUUaD4C/chB4Vq2QkDKO7Q8+2HJOrFIjiVli4lU+Sf
 OBp64m//Y1xys++Z4ODoKh7tkh5DxiO3QBHG7bHK0CSQsJ6XUvPVYubAuy1XfSDzSeSBl//C
 v78Fclb+gi9GWidSTG/4hsEzd1fY5XwCZG/XJJY9M/sAAwUH/09Ar9W2U1Qm+DwZeP2ii3Ou
 14Z9VlVVPhcEmR/AFykL9dw/OV2O/7cdi52+l00reUu6Nd4Dl8s4f5n8b1YFzmkVVIyhwjvU
 jxtPyUgDOt6DRa+RaDlXZZmxQyWcMv2anAgYWGVszeB8Myzsw8y7xhBEVV1S+1KloCzw4V8Z
 DSJrcsZlyMDoiTb7FyqxwQnM0f6qHxWbmOOnbzJmBqpNpFuDcz/4xNsymJylm6oXiucHQBAP
 Xb/cE1YNHpuaH4SRhIxwQilCYEznWowQphNAbJtEKOmcocY7EbSt8VjXTzmYENkIfkrHRyXQ
 dUm5AoL51XZljkCqNwrADGkTvkwsWSvCSQQYEQIACQUCWTecRAIbDAAKCRCgNK6L+3cgfuef
 AJ9wlZQNQUp0KwEf8Tl37RmcxCL4bQCcC5alCSMzUBJ5DBIcR4BY+CyQFAs=

--------------fKn1louHxgGR7JmYlewZgRWC
Content-Type: multipart/mixed; boundary="------------1gFfrJwB838flN80oaPDckCS"

--------------1gFfrJwB838flN80oaPDckCS
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMDYuMDUuMjQgMDg6NDIsIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBPbiAwMy4wNS4yMDI0
IDIxOjA3LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+PiBPbiBGcmksIDMgTWF5IDIw
MjQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+PiBIaSBTdGVmYW5vLA0KPj4+DQo+Pj4gT24g
MDIvMDUvMjAyNCAxOToxMywgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPj4+PiBPbiBN
b24sIDI5IEFwciAyMDI0LCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+Pj4+PiBIaSBKdWVyZ2Vu
LA0KPj4+Pj4NCj4+Pj4+IE9uIDI5LzA0LzIwMjQgMTI6MjgsIErDvHJnZW4gR3Jvw58gd3Jv
dGU6DQo+Pj4+Pj4gT24gMjkuMDQuMjQgMTM6MDQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+
Pj4+Pj4gSGkgSnVlcmdlbiwNCj4+Pj4+Pj4NCj4+Pj4+Pj4gU29ycnkgZm9yIHRoZSBsYXRl
IHJlcGx5Lg0KPj4+Pj4+Pg0KPj4+Pj4+PiBPbiAyOS8wNC8yMDI0IDExOjMzLCBKdWVyZ2Vu
IEdyb3NzIHdyb3RlOg0KPj4+Pj4+Pj4gT24gMDguMDQuMjQgMDk6MTAsIEphbiBCZXVsaWNo
IHdyb3RlOg0KPj4+Pj4+Pj4+IE9uIDI3LjAzLjIwMjQgMTY6MjIsIEp1ZXJnZW4gR3Jvc3Mg
d3JvdGU6DQo+Pj4+Pj4+Pj4+IFdpdGggbG9jayBoYW5kbGluZyBub3cgYWxsb3dpbmcgdXAg
dG8gMTYzODQgY3B1cyAoc3BpbmxvY2tzIGNhbg0KPj4+Pj4+Pj4+PiBoYW5kbGUNCj4+Pj4+
Pj4+Pj4gNjU1MzUgY3B1cywgcndsb2NrcyBjYW4gaGFuZGxlIDE2Mzg0IGNwdXMpLCByYWlz
ZSB0aGUgYWxsb3dlZA0KPj4+Pj4+Pj4+PiBsaW1pdA0KPj4+Pj4+Pj4+PiBmb3INCj4+Pj4+
Pj4+Pj4gdGhlIG51bWJlciBvZiBjcHVzIHRvIGJlIGNvbmZpZ3VyZWQgdG8gMTYzODMuDQo+
Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+IFRoZSBuZXcgbGltaXQgaXMgaW1wb3NlZCBieSBJT01N
VV9DTURfQlVGRkVSX01BWF9FTlRSSUVTIGFuZA0KPj4+Pj4+Pj4+PiBRSU5WQUxfTUFYX0VO
VFJZX05SIHJlcXVpcmVkIHRvIGJlIGxhcmdlciB0aGFuIDIgKg0KPj4+Pj4+Pj4+PiBDT05G
SUdfTlJfQ1BVUy4NCj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4gU2lnbmVkLW9mZi1ieTogSnVl
cmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPg0KPj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4gQWNr
ZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4+Pj4+Pj4+Pg0KPj4+
Pj4+Pj4+IEknZCBwcmVmZXIgdGhpcyB0byBhbHNvIGdhaW4gYW4gQXJtIGFjaywgdGhvdWdo
Lg0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IEFueSBjb21tZW50IGZyb20gQXJtIHNpZGU/DQo+Pj4+
Pj4+DQo+Pj4+Pj4+IENhbiB5b3UgY2xhcmlmeSB3aGF0IHRoZSBuZXcgbGltaXRzIG1lYW4g
aW4gdGVybSBvZiAoc2VjdXJpdHkpDQo+Pj4+Pj4+IHN1cHBvcnQ/DQo+Pj4+Pj4+IEFyZSB3
ZSBub3cgY2xhaW1pbmcgdGhhdCBYZW4gd2lsbCB3b3JrIHBlcmZlY3RseSBmaW5lIG9uIHBs
YXRmb3Jtcw0KPj4+Pj4+PiB3aXRoIHVwDQo+Pj4+Pj4+IHRvIDE2MzgzPw0KPj4+Pj4+Pg0K
Pj4+Pj4+PiBJZiBzbywgSSBjYW4ndCBjb21tZW50IGZvciB4ODYsIGJ1dCBmb3IgQXJtLCBJ
IGFtIGRvdWJ0ZnVsIHRoYXQgaXQNCj4+Pj4+Pj4gd291bGQNCj4+Pj4+Pj4gd29yayB3aXRo
b3V0IGFueSAoYXQgbGVhc3QgcGVyZm9ybWFuY2UpIGlzc3Vlcy4gQUZBSUssIHRoaXMgaXMg
YWxzbyBhbg0KPj4+Pj4+PiB1bnRlc3RlZCBjb25maWd1cmF0aW9uLiBJbiBmYWN0IEkgd291
bGQgYmUgc3VycHJpc2VkIGlmIFhlbiBvbiBBcm0gd2FzDQo+Pj4+Pj4+IHRlc3RlZCB3aXRo
IG1vcmUgdGhhbiBhIGNvdXBsZSBvZiBodW5kcmVkcyBjb3JlcyAoQUZBSUNUIHRoZSBBbXBl
cmUNCj4+Pj4+Pj4gQ1BVcw0KPj4+Pj4+PiBoYXMgMTkyIENQVXMpLg0KPj4+Pj4+DQo+Pj4+
Pj4gSSB0aGluayB3ZSBzaG91bGQgYWRkIGEgc2VjdXJpdHkgc3VwcG9ydCBsaW1pdCBmb3Ig
dGhlIG51bWJlciBvZg0KPj4+Pj4+IHBoeXNpY2FsDQo+Pj4+Pj4gY3B1cyBzaW1pbGFyIHRv
IHRoZSBtZW1vcnkgc3VwcG9ydCBsaW1pdCB3ZSBhbHJlYWR5IGhhdmUgaW4gcGxhY2UuDQo+
Pj4+Pj4NCj4+Pj4+PiBGb3IgeDg2IEknZCBzdWdnZXN0IDQwOTYgY3B1cyBmb3Igc2VjdXJp
dHkgc3VwcG9ydCAoYmFzaWNhbGx5IHRoZSBsaW1pdA0KPj4+Pj4+IHdlDQo+Pj4+Pj4gaGF2
ZSB3aXRoIHRoaXMgcGF0Y2gpLCBidXQgSSdtIG9wZW4gZm9yIG90aGVyIHN1Z2dlc3Rpb25z
LCB0b28uDQo+Pj4+Pj4NCj4+Pj4+PiBJIGhhdmUgbm8gaWRlYSBhYm91dCBhbnkgc2Vuc2li
bGUgbGltaXRzIGZvciBBcm0zMi9Bcm02NC4NCj4+Pj4+DQo+Pj4+PiBJIGFtIG5vdCBlbnRp
cmVseS4gQmVydHJhbmQsIE1pY2hhbCwgU3RlZmFubywgc2hvdWxkIHdlIHVzZSAxOTIgKHRo
ZQ0KPj4+Pj4gbnVtYmVyIG9mDQo+Pj4+PiBDUFVzIGZyb20gQW1wZXJlKT8NCj4+Pj4NCj4+
Pj4gSSBhbSBPSyB3aXRoIHRoYXQuIElmIHdlIHdhbnQgdG8gYmUgYSBiaXQgbW9yZSBmdXR1
cmUgcHJvb2Ygd2UgY291bGQgc2F5DQo+Pj4+IDI1NiBvciA1MTIuDQo+Pj4NCj4+PiBTb3Jy
eSwgSSBkb24ndCBmb2xsb3cgeW91ciBhcmd1bWVudC4gQSBsaW1pdCBjYW4gYmUgcmFpc2Vk
IGF0IHRpbWUgcG9pbnQgaW4NCj4+PiB0aGUgZnV0dXJlLiBUaGUgcXVlc3Rpb24gaXMgbW9y
ZSB3aGV0aGVyIHdlIGFyZSBjb25maWRlbnQgdGhhdCBYZW4gb24gQXJtIHdpbGwNCj4+PiBy
dW4gd2VsbCBpZiBhIHVzZXIgaGFzIGEgcGxhdGZvcm0gd2l0aCAyNTYvNTEyIHBDUFVzLg0K
Pj4+DQo+Pj4gU28gYXJlIHlvdSBzYXlpbmcgdGhhdCBmcm9tIFhlbiBwb2ludCBvZiB2aWV3
LCB5b3UgYXJlIGV4cGVjdGluZyBubyBkaWZmZXJlbmNlDQo+Pj4gYmV0d2VlbiAyNTYgYW5k
IDUxMi4gQW5kIHRoZXJlZm9yZSB5b3Ugd291bGQgYmUgaGFwcHkgaWYgdG8gYmFja3BvcnQg
cGF0Y2hlcw0KPj4+IGlmIHNvbWVvbmUgZmluZCBkaWZmZXJlbmNlcyAob3IgZXZlbiBzZWN1
cml0eSBpc3N1ZXMpIHdoZW4gdXNpbmcgPiAyNTYgcENQVXM/DQo+Pg0KPj4gSXQgaXMgZGlm
ZmljdWx0IHRvIGJlIHN1cmUgYWJvdXQgYW55dGhpbmcgdGhhdCBpdCBpcyBub3QgcmVndWxh
cmx5DQo+PiB0ZXN0ZWQuIEkgYW0gcHJldHR5IHN1cmUgc29tZW9uZSBpbiB0aGUgY29tbXVu
aXR5IGdvdCBYZW4gcnVubmluZyBvbiBhbg0KPj4gQW1wZXJlLCBzbyBsaWtlIHlvdSBzYWlk
IDE5MiBpcyBhIGdvb2QgbnVtYmVyLiBIb3dldmVyLCB0aGF0IGlzIG5vdA0KPj4gcmVndWxh
cmx5IHRlc3RlZCwgc28gd2UgZG9uJ3QgaGF2ZSBhbnkgcmVncmVzc2lvbiBjaGVja3MgaW4g
Z2l0bGFiLWNpIG9yDQo+PiBPU1NUZXN0IGZvciBpdC4NCj4+DQo+PiBPbmUgYXBwcm9hY2gg
d291bGQgYmUgdG8gb25seSBzdXBwb3J0IHRoaW5ncyByZWd1bGFybHkgdGVzdGVkIGVpdGhl
ciBieQ0KPj4gT1NTVGVzdCwgR2l0bGFiLWNpLCBvciBhbHNvIFhlbiBjb21tdW5pdHkgbWVt
YmVycy4gSSBhbSBub3Qgc3VyZSB3aGF0DQo+PiB3b3VsZCBiZSB0aGUgaGlnaGVzdCBudW1i
ZXIgd2l0aCB0aGlzIHdheSBvZiB0aGlua2luZyBidXQgbGlrZWx5IG5vDQo+PiBtb3JlIHRo
YW4gMTkyLCBwcm9iYWJseSBsZXNzLiBJIGRvbid0IGtub3cgdGhlIENQVSBjb3JlIGNvdW50
IG9mIHRoZQ0KPj4gYmlnZ2VzdCBBUk0gbWFjaGluZSBpbiBPU1NUZXN0Lg0KPj4NCj4+IEFu
b3RoZXIgYXBwcm9hY2ggaXMgdG8gc3VwcG9ydCBhICJzZW5zaWJsZSIgbnVtYmVyOiBub3Qg
c29tZXRoaW5nIHRlc3RlZA0KPj4gYnV0IHNvbWV0aGluZyB3ZSBiZWxpZXZlIGl0IHNob3Vs
ZCB3b3JrLiBObyByZWd1bGFyIHRlc3RpbmcuIChJbiBzYWZldHksDQo+PiB0aGV5IG9ubHkg
YmVsaWV2ZSBpbiB0aGluZ3MgdGhhdCBhcmUgYWN0dWFsbHkgdGVzdGVkLCBzbyB0aGlzIHdv
dWxkIG5vdA0KPj4gYmUgT0suIEJ1dCB0aGlzIGlzIHNlY3VyaXR5LCBub3Qgc2FmZXR5LCBq
dXN0IEZZSS4pIFdpdGggdGhpcyBhcHByb2FjaCwNCj4+IHdlIGNvdWxkIHJvdW5kIHVwIHRo
ZSBudW1iZXIgdG8gYSBsaW1pdCB3ZSB0aGluayBpdCB3b24ndCBicmVhay4gSWYgMTkyDQo+
PiB3b3JrcywgMjU2LzUxMiBzaG91bGQgd29yaz8gSSBkb24ndCBrbm93IGJ1dCBjb3VsZG4n
dCB0aGluayBvZiBzb21ldGhpbmcNCj4+IHRoYXQgd291bGQgYnJlYWsgZ29pbmcgZnJvbSAx
OTIgdG8gMjU2Lg0KPiANCj4gSSB3b3VsZCBzdWdnZXN0IHRvIGFpbSBhdCBzdGlja2luZyB0
byBwb3dlci1vZi0yIHZhbHVlcy4gVGhlcmUgYXJlIHN0aWxsDQo+IHNvbWUgY2FsY3VsYXRp
b25zIGluIFhlbiB3aGljaCBjYW4gIGJlIHRyYW5zbGF0ZWQgdG8gbW9yZSBlZmZpY2llbnQg
Y29kZQ0KPiB0aGF0IHdheSAobWFpbmx5OiB1c2luZyBzaGlmdHMgcmF0aGVyIHRoYW4gbXVs
dGlwbGljYXRpb25zIG9yIGENCj4gY29tYmluYXRpb24gb2Ygc2hpZnRzIGFuZCBhZGRzKS4g
T2YgY291cnNlIHRob3NlIGNhbGN1bGF0aW9ucyBkZXBlbmQgb24NCj4gd2hhdCBwZW9wbGUg
Y2hvb3NlIGFzIGFjdHVhbCB2YWx1ZXMsIGJ1dCBnaXZpbmcgYW4gdXBwZXIgYm91bmQgYmVp
bmcgYQ0KPiBwb3dlciBvZiAyIG1heSBhdCBsZWFzdCBzZXJ2ZSBhcyBhIGhpbnQgdG8gdGhl
bS4NCj4gDQo+PiBJdCBkZXBlbmRzIG9uIGhvdyBzdHJpY3Qgd2Ugd2FudCB0byBiZSBvbiB0
ZXN0aW5nIHJlcXVpcmVtZW50cy4gSSBhbSBub3QNCj4+IHN1cmUgd2hhdCBhcHByb2FjaCB3
YXMgdGFrZW4gYnkgeDg2IHNvIGZhci4gSSBhbSBPSyBlaXRoZXIgd2F5Lg0KPiANCj4gVGhl
IGJ1bXBpbmcgb2YgdGhlIGxpbWl0IGhlcmUgY2xlYXJseSBpcyBmb3J3YXJkLWxvb2tpbmcg
Zm9yIHg4NiwgaS5lLiBpcw0KPiB1bmxpa2VseSB0byBiZSBldmVuIHBvc3NpYmxlIHRvIHRl
c3QgcmlnaHQgbm93IChleGNlcHQgbWF5YmUgd2hlbiBydW5uaW5nDQo+IFhlbiBpdHNlbGYg
dmlydHVhbGl6ZWQpLiBJIGFjdHVhbGx5IHRoaW5rIHRoZXJlIG5lZWQgdG8gYmUgdHdvIHNl
cGFyYXRlDQo+IGNvbnNpZGVyYXRpb25zOiBPbmUgaXMgdG93YXJkcyBmb3IgaG93IG1hbnkg
Q1BVcyBYZW4gY2FuIGJlIGJ1aWx0IChhbmQNCj4gc3VjaCBhIGJ1aWxkIGNhbiBiZSB2YWxp
ZGF0ZWQgb24gYSBtdWNoIHNtYWxsZXIgc3lzdGVtKSwgd2hpbGUgYW5vdGhlciBpcw0KPiB0
byBsaW1pdCB3aGF0IGlzIHN1cHBvcnRlZCAoaW4gLi9TVVBQT1JULm1kKS4NCg0KTXkgc3Vn
Z2VzdGlvbiB3b3VsZCBiZSB0byBhZGQgdGhlIGZvbGxvd2luZyB0byBteSBwYXRjaDoNCg0K
LSBpbnRyb2R1Y2luZyB0aGUgbnVtYmVyIG9mIHNlY3VyaXR5IHN1cHBvcnRlZCBwaHlzaWNh
bCBjcHVzIHRvIFNVUFBPUlQubWQNCiAgICg0MDk2IGZvciB4ODYsIDI1NiBmb3IgQXJtNjQg
YW5kIEFybTMyKQ0KDQotIGFkZGluZyB0aGUgbmV3IHVwcGVyIGJvdW5kIHRvIENIQU5HRUxP
Ry5tZA0KDQpJbiBjYXNlIEkgZG9uJ3QgaGVhciBhbnkgb2JqZWN0aW9ucyBJJ2xsIHNlbmQg
aXQgb3V0IHRvbW9ycm93Lg0KDQoNCkp1ZXJnZW4NCg==
--------------1gFfrJwB838flN80oaPDckCS
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------1gFfrJwB838flN80oaPDckCS--

--------------fKn1louHxgGR7JmYlewZgRWC--

--------------xL5qX0QquxcPCH6DBRNX9Mu9
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmY4flkFAwAAAAAACgkQsN6d1ii/Ey/7
iQf/S08DBejk8ZNLNYUNaZd0Ae5AlwJu+fxpURcEXYBjoLp2tbbyyu0pGwC0b4oKffUErb1/v/0y
mSJkAS0OifFj0uUla37Y4igBrU3x8MY4QHHT5hk6KOA9t+CLf4lvwpGMoTH4vKmtCuatJd6pyEOl
tBqmLUZe4VZnXkmR4FrqiS4tzzV3MCs+L7gGDPtaLUNb/4gH64gUerBE/UFPJxk+8VNcY1mRIRxY
gCclGvBghaReIfR86Bhs6baQ/87vMd7ZyeC0uhYGi3zsksCcg8FzbsGRiIbwGBtuxN4iO4au+jPa
B28KkAF+0/2wgNJ+11p22kXNeqtqrRs20leN5Lytjw==
=KoFb
-----END PGP SIGNATURE-----

--------------xL5qX0QquxcPCH6DBRNX9Mu9--


From xen-devel-bounces@lists.xenproject.org Mon May 06 07:11:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 07:11:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717248.1119260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3sVn-0000yr-D5; Mon, 06 May 2024 07:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717248.1119260; Mon, 06 May 2024 07:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3sVn-0000yk-AZ; Mon, 06 May 2024 07:11:47 +0000
Received: by outflank-mailman (input) for mailman id 717248;
 Mon, 06 May 2024 07:11:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3sVl-0000ye-NW
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 07:11:45 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e50fec72-0b77-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 09:11:44 +0200 (CEST)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2e34e85ebf4so5038441fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 00:11:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r13-20020a05600c35cd00b0041bf45c0665sm18690774wmq.15.2024.05.06.00.11.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 00:11:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e50fec72-0b77-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714979504; x=1715584304; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=q7/gYtPmo8Zt50VkQkmr9V5PyAFFkrgi1XLQy4kWdE0=;
        b=B4BLMBVAIjSdqaTCuNLOcN0QT6JrewBKjI8NKO6vIPvyJwBpPW9wqsa84q3JsqNST4
         CFBovl7NZ8uE/I0q9zPL71sD4TGHM2nK0Ot4ClnI7NHKCfXGD1UQe+Oc4R9cuytuMZLQ
         hrYaAePoeY2rK+5OG4wW+Ia+WNtXj5dk07bxrIDsEAFEgl5qbmS1eCRyYd9z383Mvhy2
         1yDi+VT9gRRk9O+CcRBsv47xrVsrgfYNE7rFLlToUc4UtezqcqneeMHxa9RQvlRysLkk
         aHAG2TvdDz0/A/Tm72tOx0IlW1fc/PCWk0ickMzRWKr0oOaEjUDjsP6vhOEZLw7G4OE2
         GXRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714979504; x=1715584304;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q7/gYtPmo8Zt50VkQkmr9V5PyAFFkrgi1XLQy4kWdE0=;
        b=If3mUB8zZ57ZtiXKL3cscwcaiI/oEUcUlbljaHHhaVGW82VIYtiH9reMg5AxRSxxP7
         L89B6nR9WH3FtnKe3kPr+KMIEQX/LWzsfejvq7Rct0mCttAddrAlmO56pZzGH7QwisTG
         bMkzdofYUUw34zuKIWeN1fcKk04E1lb+IPDhAPDSsnVu+q+Zb8WMC1XO31/ppUneu42+
         /Y8Lq43HnTE9uU6VavtbkZRRLbFqAXrH0UoJdq9dFbkYfDbrirL3cX2u/4HQFa+kXJOB
         QY8DRV77RFcQz2xEonRyXlOpgv+irh+m9Pgm3BLLv9y4z5z/M/ayJ4KTMWBDmipGziF/
         pdNA==
X-Forwarded-Encrypted: i=1; AJvYcCUrxguKE+9Bsxp1Rfi76OzzQszRCzvh0hH715tYHQdn+Tjf18vf9xCWUPleVXxrEyTNf3sNbJE56NWisNXq+lW2t0NLdUD0gMipKdok1Vc=
X-Gm-Message-State: AOJu0Yyw6Cxy6El6E0zAnruH5SrjkVt3ul6hyraueWhh7KHMrCuhXvC/
	rChfxemubS1N9kggQeNj5urLkFsERSGq6BwKFNdA8oM2ayKtX6FduI6FD6lorw==
X-Google-Smtp-Source: AGHT+IFbBmoOdv+4VkzeQmjUmBEO14EpVLCzIZxMLHZRsCBBiK5Yw8BL7UOt80ewzpJuswnSB4erwQ==
X-Received: by 2002:a2e:812:0:b0:2e3:6ef5:d567 with SMTP id 18-20020a2e0812000000b002e36ef5d567mr805813lji.30.1714979504204;
        Mon, 06 May 2024 00:11:44 -0700 (PDT)
Message-ID: <8c0bc745-05f5-4839-b920-7830e2c55688@suse.com>
Date: Mon, 6 May 2024 09:11:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.19 release status tracking list [ May ]
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
References: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 03.05.2024 18:54, Oleksii wrote:
> *** x86 ***:
>   * [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks [
> https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citrix.com/
> ]:
>     - almost patch series have been merged already except the patch:
>         [PATCH 4/4] iommu/x86: make unity range checking more strict
> 
>   * [PATCH 0/8] x86: support AVX10.1 [
> https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@suse.com/
> ]:
>     - two patches of patch series are waitng to merged/reviewed:
>       [PATCH 1/4] amd-vi: fix IVMD memory type checks
>       [PATCH 4/4] iommu/x86: make unity range checking more strict 

This sub-item doesn't really fit the topic, and seems to rather belong
to the earlier one? Where (correctly) 1/4 isn't listed anymore, for
having gone in. As to 4/4 - Roger, I'm not sure I can conclude what the
plan here was: Are you meaning to submit an updated version, or did we
rather settle on not further pursuing this?

>   * APX support?

I think you should drop this now. I'm throttling further work on the insn
emulator, as long as I have so many other patches there pending review.

>   * [PATCH v6 0/4] x86/pvh: Support relocating dom0 kernel [
> https://patchew.org/Xen/20240327215102.136001-1-jason.andryuk@amd.com/
> ]

There had been a v7, and all of that plus a follow-on adjustment have gone
in.

> *** common ***:
>   * annotate entry points with type and size" series:
>     The bulk of this has gone in, but there'll want to be follow-ups.

I think these follow-ups now want tracking on a per-arch basis (x86 and
Arm, with PPC and RISC-V filly done).

>   * [PATCH v2 (resend) 00/27] Remove the directmap [
> https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazon.com/
> ]
>     - 7/27 were merged.

Hmm, no, I don't think that one was. Other were, yes.

>   * [PATCH v6 0/7] MSI-X support with qemu in stubdomain, and other
> related changes:
>     - new patch version was sent

1/7 was committed.

>   * [PATCH v3 0/4] xenwatchdogd bugfixes and enhancements [
> https://lore.kernel.org/xen-devel/20240411182023.56309-1-leigh@solinno.co.uk/
> ]:
>     new patch series were sent.

Was there anything left from that series?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 07:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 07:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717265.1119290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3svc-00057v-H0; Mon, 06 May 2024 07:38:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717265.1119290; Mon, 06 May 2024 07:38:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3svc-00057o-EJ; Mon, 06 May 2024 07:38:28 +0000
Received: by outflank-mailman (input) for mailman id 717265;
 Mon, 06 May 2024 07:38:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3svb-00057i-Cs
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 07:38:27 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e02e2ae-0b7b-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 09:38:23 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41bab13ca81so17215005e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 00:38:23 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 m9-20020a05600c4f4900b0041bf7da4200sm15141161wmq.33.2024.05.06.00.38.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 00:38:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e02e2ae-0b7b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714981103; x=1715585903; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=tjYCOSdMFmFeFYv00bvqUGHB4qSVO8Xv+3DtlFur9B0=;
        b=k0jEjrbK1U+2hKHOhoyoG4zqakLalryJqSIlr1QFei57lhBQQyunQFPfdvPyXYbq8f
         BMlyUtVH2mweuvxrHfZECYKRHmMMPXE1XDjjqrybw32E+0xxyVwtIrPDcnfkdxrjg17G
         A28fBdAKZc2cSoHFGtZPvniyD5zrAicVvxRUY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714981103; x=1715585903;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tjYCOSdMFmFeFYv00bvqUGHB4qSVO8Xv+3DtlFur9B0=;
        b=cj4cIM+UrW6kfVb6msqGB2VHnL0w8rzy0DEGP2Y3UzKl8DPZxABjpZFnUvyLIRsRh1
         nAAsn4oRhWkGIp9upTov0+32npUT2HpHYBV2By6bEGx0Alk0GuNqQBlri4P544aTCe8O
         SeVuNE07qgjuu9iTwacfvraBlx8k+04fO6uDDSoaIpoxd7im2Ir0rOcEjohB+4XrzKQF
         D/L80nDDc89+YcqTtS0ow1AQPhSjDulc0dSKXm0X/c/tqaeVlFF9PlRFb7dsNKUdaDmc
         O9Iz52x3Zd/QKIabaXPCpLXkhPtUHf8vT2/0kIOHRiEEINCwFAWgC/yhhYleM4NBVdGP
         qakA==
X-Gm-Message-State: AOJu0YzxNBUPCwjePeZlYqkurhu/aGCrktGiqH0hC/66p3uf+CN4KBRm
	xr2ufsm5kC3pVI/34ridG0PJQbqC9CjeQsdcf1IELBdp9vuL9a1nM5NGESQH/1Y=
X-Google-Smtp-Source: AGHT+IGYx3Dd8wZbnJ3Oizz+3OPNRKZZjqh9SO2K+wrAoSLUgKPZHikdK3L0k/dpjUbsnVCAUIG55g==
X-Received: by 2002:a05:600c:450e:b0:41b:edf4:4071 with SMTP id t14-20020a05600c450e00b0041bedf44071mr9417881wmo.35.1714981102682;
        Mon, 06 May 2024 00:38:22 -0700 (PDT)
Date: Mon, 6 May 2024 09:38:21 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v6 3/7] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <ZjiI7cLtiJTzc1El@macbook>
References: <cover.3491f479d5b6238106af8ddc56ec6834090b330f.1714154036.git-series.marmarek@invisiblethingslab.com>
 <a9b04e2224e97a27a127a003e8ccf5edfd4922c7.1714154036.git-series.marmarek@invisiblethingslab.com>
 <ZjShYhUvx694rPEp@macbook>
 <ZjWFzEdJ9wiz2GMI@mail-itl>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZjWFzEdJ9wiz2GMI@mail-itl>

On Sat, May 04, 2024 at 02:48:12AM +0200, Marek Marczykowski-Górecki wrote:
> On Fri, May 03, 2024 at 10:33:38AM +0200, Roger Pau Monné wrote:
> > On Fri, Apr 26, 2024 at 07:54:00PM +0200, Marek Marczykowski-Górecki wrote:
> > > Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> > > on the same page as MSI-X table. Device model (especially one in
> > > stubdomain) cannot really handle those, as direct writes to that page is
> > > refused (page is on the mmio_ro_ranges list). Instead, extend
> > > msixtbl_mmio_ops to handle such accesses too.
> > > 
> > > Doing this, requires correlating read/write location with guest
> > > of MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
> >   ^ extra 'of'?
> > > it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> > > for PV would need to be done separately.
> > > 
> > > This will be also used to read Pending Bit Array, if it lives on the same
> > > page, making QEMU not needing /dev/mem access at all (especially helpful
> > > with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> > > map it to the guest directly.
> > > If PBA lives on the same page, discard writes and log a message.
> > > Technically, writes outside of PBA could be allowed, but at this moment
> > > the precise location of PBA isn't saved, and also no known device abuses
> > > the spec in this way (at least yet).
> > > 
> > > To access those registers, msixtbl_mmio_ops need the relevant page
> > > mapped. MSI handling already has infrastructure for that, using fixmap,
> > > so try to map first/last page of the MSI-X table (if necessary) and save
> > > their fixmap indexes. Note that msix_get_fixmap() does reference
> > > counting and reuses existing mapping, so just call it directly, even if
> > > the page was mapped before. Also, it uses a specific range of fixmap
> > > indexes which doesn't include 0, so use 0 as default ("not mapped")
> > > value - which simplifies code a bit.
> > > 
> > > GCC 12.2.1 gets confused about 'desc' variable:
> > > 
> > >     arch/x86/hvm/vmsi.c: In function ‘msixtbl_range’:
> > >     arch/x86/hvm/vmsi.c:553:8: error: ‘desc’ may be used uninitialized [-Werror=maybe-uninitialized]
> > >       553 |     if ( desc )
> > >           |        ^
> > >     arch/x86/hvm/vmsi.c:537:28: note: ‘desc’ was declared here
> > >       537 |     const struct msi_desc *desc;
> > >           |                            ^~~~
> > > 
> > > It's conditional initialization is actually correct (in the case where
> > > it isn't initialized, function returns early), but to avoid
> > > build failure initialize it explicitly to NULL anyway.
> > > 
> > > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > > ---
> > > Changes in v6:
> > > - use MSIX_CHECK_WARN macro
> > > - extend assert on fixmap_idx
> > > - add break in default label, after ASSERT_UNREACHABLE(), and move
> > >   setting default there
> > > - style fixes
> > > Changes in v5:
> > > - style fixes
> > > - include GCC version in the commit message
> > > - warn only once (per domain, per device) about failed adjacent access
> > > Changes in v4:
> > > - drop same_page parameter of msixtbl_find_entry(), distinguish two
> > >   cases in relevant callers
> > > - rename adj_access_table_idx to adj_access_idx
> > > - code style fixes
> > > - drop alignment check in adjacent_{read,write}() - all callers already
> > >   have it earlier
> > > - delay mapping first/last MSI-X pages until preparing device for a
> > >   passthrough
> > > v3:
> > >  - merge handling into msixtbl_mmio_ops
> > >  - extend commit message
> > > v2:
> > >  - adjust commit message
> > >  - pass struct domain to msixtbl_page_handler_get_hwaddr()
> > >  - reduce local variables used only once
> > >  - log a warning if write is forbidden if MSI-X and PBA lives on the same
> > >    page
> > >  - do not passthrough unaligned accesses
> > >  - handle accesses both before and after MSI-X table
> > > ---
> > >  xen/arch/x86/hvm/vmsi.c        | 200 ++++++++++++++++++++++++++++++++--
> > >  xen/arch/x86/include/asm/msi.h |   5 +-
> > >  xen/arch/x86/msi.c             |  41 +++++++-
> > >  3 files changed, 236 insertions(+), 10 deletions(-)
> > > 
> > > diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> > > index 999917983789..230e3a5dee3f 100644
> > > --- a/xen/arch/x86/hvm/vmsi.c
> > > +++ b/xen/arch/x86/hvm/vmsi.c
> > > @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
> > >      return d->arch.hvm.msixtbl_list.next;
> > >  }
> > >  
> > > +/*
> > > + * Lookup an msixtbl_entry on the same page as given addr. It's up to the
> > > + * caller to check if address is strictly part of the table - if relevant.
> > > + */
> > >  static struct msixtbl_entry *msixtbl_find_entry(
> > >      struct vcpu *v, unsigned long addr)
> > >  {
> > > @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
> > >      struct domain *d = v->domain;
> > >  
> > >      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> > > -        if ( addr >= entry->gtable &&
> > > -             addr < entry->gtable + entry->table_len )
> > > +        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
> > > +             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
> > >              return entry;
> > >  
> > >      return NULL;
> > > @@ -213,6 +217,138 @@ static struct msi_desc *msixtbl_addr_to_desc(
> > >      return NULL;
> > >  }
> > >  
> > > +/*
> > > + * Returns:
> > > + *  - ADJACENT_DONT_HANDLE if no handling should be done
> > > + *  - ADJACENT_DISCARD_WRITE if write should be discarded
> > > + *  - a fixmap idx to use for handling
> > > + */
> > > +#define ADJACENT_DONT_HANDLE UINT_MAX
> > > +#define ADJACENT_DISCARD_WRITE (UINT_MAX - 1)
> > 
> > I think this could be simpler, there's no need to signal with so fine
> > grained detail about the action to be performed.
> > 
> > Any adjacent access to the MSI-X table should be handled by the logic
> > you are adding, so anything that falls in those ranges should
> > terminate here.
> > 
> > adjacent_handle() should IMO just return whether the access is
> > replayed against the hardware, or if it's just dropped.
> 
> The distinction here is to return X86EMUL_OKAY in case of adjacent write
> that is ignored because PBA is somewhere near, but X86EMUL_UNHANDLABLE
> for other/error cases (like fixmap indices not initialized).
> But maybe this distinction doesn't make sense and X86EMUL_UNHANDLABLE is
> okay in either case? 

That's my suggestion, yes.  I don't think it's expected for ioreqs to
handle those adjacent accesses, the more with the limitation that some
of them might not even have access to such region in the first place.

> > > @@ -220,9 +356,10 @@ static int cf_check msixtbl_read(
> > >      unsigned long offset;
> > >      struct msixtbl_entry *entry;
> > >      unsigned int nr_entry, index;
> > > +    unsigned int adjacent_fixmap;
> > >      int r = X86EMUL_UNHANDLEABLE;
> > >  
> > > -    if ( (len != 4 && len != 8) || (address & (len - 1)) )
> > > +    if ( !IS_ALIGNED(address, len) )
> > >          return r;
> > >  
> > >      rcu_read_lock(&msixtbl_rcu_lock);
> > > @@ -230,6 +367,21 @@ static int cf_check msixtbl_read(
> > >      entry = msixtbl_find_entry(current, address);
> > >      if ( !entry )
> > >          goto out;
> > > +
> > > +    adjacent_fixmap = adjacent_handle(entry, address, false);
> > 
> > This seems overly complicated, but is possible I'm missing some logic.
> > 
> > IMO it would seem way less convoluted to simply do:
> > 
> > entry = msixtbl_find_entry(current, address);
> > if ( !entry )
> >     goto out;
> > 
> > if ( address < entry->gtable ||
> >      address >= entry->gtable + entry->table_len )
> > {
> >     adjacent_read(...);
> >     goto out;
> > }
> > 
> > And put all the logic in adjacent_{read,write}() directly rather than
> > having both adjacent_{read,write}() plus adjacent_handle() calls here?
> > 
> > If the access doesn't fall between the boundaries of the MSI-X table
> > it's either going to be a handled adjacent access, or it's going to be
> > discarded.
> 
> Discarded - should it return X86EMUL_OKAY in that case? Currently it
> returns X86EMUL_UNHANDLABLE in case adjacent access isn't handled (for
> any reason) either.

Yes, I think we want to terminate all adjacent accesses here - there's
so far no need to forward them to any other handler, and that would
simplify the logic.  I don't see an use case for handling those
elsewhere, but if that ever arises we can always add the support them.
There's no need to cater for a non-existent use-case that just makes
the code more complicated.

> > > +    if ( adjacent_fixmap != ADJACENT_DONT_HANDLE )
> > > +    {
> > > +        r = adjacent_read(adjacent_fixmap, address, len, pval);
> > > +        goto out;
> > > +    }
> > > +
> > > +    if ( address < entry->gtable ||
> > > +         address >= entry->gtable + entry->table_len )
> > > +        goto out;
> > > +
> > > +    if ( len != 4 && len != 8 )
> > > +        goto out;
> > > +
> > >      offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
> > >  
> > >      if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
> > > @@ -282,6 +434,7 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
> > >      int r = X86EMUL_UNHANDLEABLE;
> > >      unsigned long flags;
> > >      struct irq_desc *desc;
> > > +    unsigned int adjacent_fixmap;
> > >  
> > >      if ( !IS_ALIGNED(address, len) )
> > >          return X86EMUL_OKAY;
> > > @@ -291,6 +444,21 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
> > >      entry = msixtbl_find_entry(v, address);
> > >      if ( !entry )
> > >          goto out;
> > > +
> > > +    adjacent_fixmap = adjacent_handle(entry, address, true);
> > > +    if ( adjacent_fixmap != ADJACENT_DONT_HANDLE )
> > > +    {
> > > +        r = adjacent_write(adjacent_fixmap, address, len, val);
> > > +        goto out;
> > > +    }
> > > +
> > > +    if ( address < entry->gtable ||
> > > +         address >= entry->gtable + entry->table_len )
> > > +        goto out;
> > > +
> > > +    if ( len != 4 && len != 8 )
> > > +        goto out;
> > > +
> > >      nr_entry = array_index_nospec(((address - entry->gtable) /
> > >                                     PCI_MSIX_ENTRY_SIZE),
> > >                                    MAX_MSIX_TABLE_ENTRIES);
> > > @@ -356,8 +524,8 @@ static int cf_check _msixtbl_write(
> > >      const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
> > >      uint64_t val)
> > >  {
> > > -    /* Ignore invalid length or unaligned writes. */
> > > -    if ( (len != 4 && len != 8) || !IS_ALIGNED(address, len) )
> > > +    /* Ignore unaligned writes. */
> > > +    if ( !IS_ALIGNED(address, len) )
> > >          return X86EMUL_OKAY;
> > >  
> > >      /*
> > > @@ -374,14 +542,22 @@ static bool cf_check msixtbl_range(
> > >  {
> > >      struct vcpu *curr = current;
> > >      unsigned long addr = r->addr;
> > > -    const struct msi_desc *desc;
> > > +    const struct msixtbl_entry *entry;
> > > +    const struct msi_desc *desc = NULL;
> > > +    unsigned int adjacent_fixmap;
> > >  
> > >      ASSERT(r->type == IOREQ_TYPE_COPY);
> > >  
> > >      rcu_read_lock(&msixtbl_rcu_lock);
> > > -    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
> > > +    entry = msixtbl_find_entry(curr, addr);
> > > +    adjacent_fixmap = adjacent_handle(entry, addr, false);
> > > +    if ( adjacent_fixmap == ADJACENT_DONT_HANDLE )
> > > +        desc = msixtbl_addr_to_desc(entry, addr);
> > 
> > I'm not sure you need adjacent_handle() here, I would think that any
> > address adjacent to the MSI-X table Xen would want to handle
> > unconditionally, and hence msixtbl_range() should return true:
> 
> I put it here to not duplicate a set of checks for adjacent access. It
> isn't only about the range, but also few other case (like if fixmap
> indices are set).

Right, but as discussed above, we likely want to terminate those
accesses here anyway, since there's no other handler that cares about
MSI-X table adjacent regions.

> > entry = msixtbl_find_entry(curr, addr);
> > if ( !entry )
> >     return 0;
> > 
> > if ( addr < entry->gtable || addr >= entry->gtable + entry->table_len )
> >     /* Possibly handle adjacent access. */
> >     return 1;
> > 
> > desc = msixtbl_find_entry(curr, addr);
> > ...
> > 
> > (Missing the _unlock calls in the chunk above)
> > 
> > There's no other processing that can happen for an adjacent access
> > unless it's are handled here.  Otherwise such accesses will be
> > discarded anyway?  Hence better short-circuit the MMIO handler search
> > earlier.
> 
> Can't there be some ioreq server that could theoretically handle them?

I don't think any of the current ioreq implementations care about
those ATM, and with your work to make QEMU not depend on /dev/mem
it's even more unlikely that we might gain such in the future.

As said above, I wouldn't mind allowing such forwarding if it made the
code easier, but seeing how it makes the logic more complicated I
think it's best to terminate all MSI-X table adjacent accesses here.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 07:46:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 07:46:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717271.1119300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3t3S-00071o-7e; Mon, 06 May 2024 07:46:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717271.1119300; Mon, 06 May 2024 07:46:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3t3S-00071h-52; Mon, 06 May 2024 07:46:34 +0000
Received: by outflank-mailman (input) for mailman id 717271;
 Mon, 06 May 2024 07:46:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3t3R-00071b-Bk
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 07:46:33 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c08477ed-0b7c-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 09:46:30 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b21ed19f5so10596195e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 00:46:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f19-20020a05600c4e9300b0041bab13cd60sm18892475wmq.3.2024.05.06.00.46.29
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 00:46:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c08477ed-0b7c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714981590; x=1715586390; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Ivowa02Zz09uluRnS4pkLodP0hzWYK2w++sjkL+XuTY=;
        b=OJKiHGyv1zpMUuJ8H+gfhAwioShKB6UHycXem+aObNRhnBHbPcBTftGLQM6LRhTEa1
         29d1y2z5OBd2HAmVC/I+udZhnVJB8n82nfueKm20lvpgNASZjUB0cvwEX/Po0//snctn
         9Tyl9nRy3PS4QTI3Up40lpUbIperyHfGTQOIi9qeHnmsO4wVVayLbjF20PozHx/uTnW9
         CO1EDAjC3hwZDOk99jw8Yx3/BeVcQwfypmyDuONAELrbn+JkZ1jjzGxvU8dNW/AIux+V
         99iMDp3H7nIJSke3WnGOvdQE20/yhzDWC0E47JmVaKN8SZNEecBL5WBWIG5idvwB8NKV
         2VWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714981590; x=1715586390;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ivowa02Zz09uluRnS4pkLodP0hzWYK2w++sjkL+XuTY=;
        b=QpGsbErujzvVm7/T6PyI15vQjaTClSJM3aFn+A42aDiBam1j0Nte3rkfzJNLeXeubB
         NHxZooTeGtLj2B6IIOPfU9oWlt0jOWRt+U6a/Jo6q3ux84NiR1AV6fD+m/zQjrCXB6gF
         Wt72Zz2SRv3BTfMg7DlBBGzmfzM9CXs2zTVzYhnkOeLy88hJH2LeJ4bQz0SQlUFe45Fk
         xjw6MvqsaFxIQYCz9s+CVpfq87ZqF1wzTjf607CFGw/CeCHX4pmHv01NFWSNUV5twK1k
         2MR5CnXxW2OhnfRYO/F9whsTa6y2jJIi3OmhCqnLp/gg+ywaMcMukFjA+stuz3WjC+XE
         OZ5Q==
X-Gm-Message-State: AOJu0Yw1NV/ZCDAbgta89Ol7tpKnUK4RQJvIjZleyVTczwp0+NK+MZXD
	IvsCfR5Kcv4eDy5pqPNY6nQMlaswK5za5V2onNwofzY9pOauOBHt+7QPDcRgZ4vNdAJ+/OK1faE
	=
X-Google-Smtp-Source: AGHT+IEO759mocuHoAgJC6kS/uBBM+NLYTt7MgkriaBqHke8h+jP3A8itTi5JBtoQUYXMVO2+mORZQ==
X-Received: by 2002:a05:600c:4fc3:b0:418:969a:b316 with SMTP id o3-20020a05600c4fc300b00418969ab316mr6666492wmq.1.1714981590250;
        Mon, 06 May 2024 00:46:30 -0700 (PDT)
Message-ID: <0794e145-aeb0-4b33-a53b-5a940958cbad@suse.com>
Date: Mon, 6 May 2024 09:46:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: New Defects reported by Coverity Scan for XenProject
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6637576caf98c_10d9e42c57d37559ac60499@prd-scan-dashboard-0.mail>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6637576caf98c_10d9e42c57d37559ac60499@prd-scan-dashboard-0.mail>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.05.2024 11:54, scan-admin@coverity.com wrote:
> Hi,
> 
> Please find the latest report on new defect(s) introduced to XenProject found with Coverity Scan.
> 
> 2 new defect(s) introduced to XenProject found with Coverity Scan.
> 1 defect(s), reported by Coverity Scan earlier, were marked fixed in the recent build analyzed by Coverity Scan.
> 
> New defect(s) Reported-by: Coverity Scan
> Showing 2 of 2 defect(s)
> 
> 
> ** CID 1596837:    (USE_AFTER_FREE)
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 935 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 935 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 935 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 935 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 935 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> 
> 
> ________________________________________________________________________________________________________
> *** CID 1596837:    (USE_AFTER_FREE)
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> 937             goto out;
> 938         }
> 939     
> 940         DEBG("dyn6 ");
> 941     
> 942         /* decompress until an end-of-block code */
>>>>     CID 1596837:    (USE_AFTER_FREE)
>>>>     Calling "inflate_codes" dereferences freed pointer "tl".
> 943         if (inflate_codes(tl, td, bl, bd)) {
> 944             ret = 1;
> 945             goto out;
> 946         }

While first I thought the tool may be confused by the earlier huft_free()
(matching an earlier huft_build()), ...

> ** CID 1596836:    (USE_AFTER_FREE)
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> /tools/firmware/xen-dir/xen-root/xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> 
> 
> ________________________________________________________________________________________________________
> *** CID 1596836:    (USE_AFTER_FREE)
> /xen/common/gzip/inflate.c: 943 in inflate_dynamic()
> 937             goto out;
> 938         }
> 939     
> 940         DEBG("dyn6 ");
> 941     
> 942         /* decompress until an end-of-block code */
>>>>     CID 1596836:    (USE_AFTER_FREE)
>>>>     Calling "inflate_codes" dereferences freed pointer "td".
> 943         if (inflate_codes(tl, td, bl, bd)) {
> 944             ret = 1;
> 945             goto out;
> 946         }

... no dual usage exists for td. Hence I'm utterly confused as to what the
tool is "thinking". In fact it looks like there is an opposite issue in
both inflate_fixed() and inflate_dynamic(): tl and td are leaked when
inflate_codes() fails. I guess I'll make a patch ...

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 07:47:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 07:47:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717273.1119311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3t3v-0007XY-Io; Mon, 06 May 2024 07:47:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717273.1119311; Mon, 06 May 2024 07:47:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3t3v-0007XR-Fw; Mon, 06 May 2024 07:47:03 +0000
Received: by outflank-mailman (input) for mailman id 717273;
 Mon, 06 May 2024 07:47:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8Q7K=MJ=inria.fr=fonyuy-asheri.caleb@srs-se1.protection.inumbo.net>)
 id 1s3t3u-00071b-0d
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 07:47:02 +0000
Received: from mail2-relais-roc.national.inria.fr
 (mail2-relais-roc.national.inria.fr [192.134.164.83])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1a303e0-0b7c-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 09:46:59 +0200 (CEST)
Received: from zcs2-store8.inria.fr ([128.93.142.6])
 by mail2-relais-roc.national.inria.fr with ESMTP; 06 May 2024 09:46:59 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1a303e0-0b7c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=inria.fr; s=dc;
  h=date:from:to:message-id:subject:mime-version;
  bh=Odxatie3Hdqs5x7nve4Krl7bVieqJ8Cx2jixE3fO7pg=;
  b=bt+PDPkn2gBNxqy1v1R33U7rNofFjAYhWNpyKhcPpc2EThZ3UbAXa6bE
   V58Pdw8Edoh8f1ax1ZyrkvtP+dA2lP8mS4EC4xqWtb57WOvSmRte79sFG
   81tX3zpS5Gm1d4dRwQkYjKSoUmr2oS/SXkfY+N5KWnCJCQaCs5Y0/peq4
   w=;
Authentication-Results: mail2-relais-roc.national.inria.fr; dkim=none (message not signed) header.i=none; spf=Pass smtp.mailfrom=fonyuy-asheri.caleb@inria.fr; spf=None smtp.helo=postmaster@zcs2-store8.inria.fr
Received-SPF: Pass (mail2-relais-roc.national.inria.fr: domain of
  fonyuy-asheri.caleb@inria.fr designates 128.93.142.6 as
  permitted sender) identity=mailfrom; client-ip=128.93.142.6;
  receiver=mail2-relais-roc.national.inria.fr;
  envelope-from="fonyuy-asheri.caleb@inria.fr";
  x-sender="fonyuy-asheri.caleb@inria.fr";
  x-conformance=spf_only; x-record-type="v=spf1";
  x-record-text="v=spf1 include:mailout.safebrands.com
  a:basic-mail.safebrands.com a:basic-mail01.safebrands.com
  a:basic-mail02.safebrands.com ip4:128.93.142.0/24
  ip4:192.134.164.0/24 ip4:128.93.162.160 ip4:89.107.174.7 mx
  ~all"
Received-SPF: None (mail2-relais-roc.national.inria.fr: no sender
  authenticity information available from domain of
  postmaster@zcs2-store8.inria.fr) identity=helo;
  client-ip=128.93.142.6;
  receiver=mail2-relais-roc.national.inria.fr;
  envelope-from="fonyuy-asheri.caleb@inria.fr";
  x-sender="postmaster@zcs2-store8.inria.fr";
  x-conformance=spf_only
X-IronPort-AV: E=Sophos;i="6.07,257,1708383600"; 
   d="scan'208,217";a="164611360"
X-MGA-submission: =?us-ascii?q?MDE54H6sFu4LecgeLOfnLcNkdwMj+lYbAHQ4O1?=
 =?us-ascii?q?l6X0G6tlqFtEOrhZNSumeiwB0yvzRZ31vlHIg0HNXiRmZ4RU4c3SCCRr?=
 =?us-ascii?q?7I6pnPSVwio6yFbUgLjSmHnI+7MmHfMOZDbCaYv17x/OBsV2jHLvCOMj?=
 =?us-ascii?q?1ywXKXrrihGH0KVV98v6rcpw=3D=3D?=
Date: Mon, 6 May 2024 09:46:58 +0200 (CEST)
From: Fonyuy-Asheri Caleb <fonyuy-asheri.caleb@inria.fr>
To: xen-devel@lists.xenproject.org
Message-ID: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>
Subject: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID
 constants
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="=_5971adc4-77e8-49f7-ba82-ae59d490cabd"
X-Originating-IP: [131.254.23.3]
X-Mailer: Zimbra 10.0.8_GA_4611 (ZimbraWebClient - GC124 (Linux)/10.0.8_GA_4611)
Thread-Index: 9o3XUyoiP5VQRv5KMMDzNA7BwO9FFw==
Thread-Topic: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID constants

--=_5971adc4-77e8-49f7-ba82-ae59d490cabd
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi, 
I am currently doing a study on the way xen handles CPUID information. 

I came across these constants in the code (xen/include/xen/lib/x86/cpu-policy.h file) but no explanation of why they have been set that way. 

#define CPUID_GUEST_NR_BASIC (0xdu + 1) 
#define CPUID_GUEST_NR_CACHE (5u + 1) 
#define CPUID_GUEST_NR_FEAT (2u + 1) 
#define CPUID_GUEST_NR_TOPO (1u + 1) 
#define CPUID_GUEST_NR_XSTATE (62u + 1) 
#define CPUID_GUEST_NR_EXTD_INTEL (0x8u + 1) 
#define CPUID_GUEST_NR_EXTD_AMD (0x21u + 1) 

Please can someone explain to me why we have these constants or point to a documentation which explains it? 
I am particularly interested in the CPUID_GUEST_NR_BASIC given that for intel processors for example, we have 
basic leaves running up to 0x21u already for recent processors. This value sort of forces a particular max leaf value. 


Caleb 

--=_5971adc4-77e8-49f7-ba82-ae59d490cabd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<html><body><div style=3D"font-family: arial, helvetica, sans-serif; font-s=
ize: 12pt; color: #000000"><div>Hi,</div><div>I am currently doing a study =
on the way xen handles CPUID information. </div><div><br data-mce-bogus=3D"=
1"></div><div>I came across these constants in the code (xen/include/xen/li=
b/x86/cpu-policy.h file) but no explanation of why they have been set that =
way.</div><div><br data-mce-bogus=3D"1"></div><div><div>#define CPUID_GUEST=
_NR_BASIC&nbsp; &nbsp; &nbsp; (0xdu + 1)</div><div>#define CPUID_GUEST_NR_C=
ACHE&nbsp; &nbsp; &nbsp; (5u + 1)</div><div>#define CPUID_GUEST_NR_FEAT&nbs=
p; &nbsp; &nbsp; &nbsp;(2u + 1)</div><div>#define CPUID_GUEST_NR_TOPO&nbsp;=
 &nbsp; &nbsp; &nbsp;(1u + 1)</div><div>#define CPUID_GUEST_NR_XSTATE&nbsp;=
 &nbsp; &nbsp;(62u + 1)</div><div>#define CPUID_GUEST_NR_EXTD_INTEL (0x8u +=
 1)</div><div>#define CPUID_GUEST_NR_EXTD_AMD&nbsp; &nbsp;(0x21u + 1)</div>=
<div><br data-mce-bogus=3D"1"></div><div>Please can someone explain to me w=
hy we have these constants or point to a documentation which explains it?&n=
bsp;</div><div>I am particularly interested in the CPUID_GUEST_NR_BASIC giv=
en that for intel processors for example, we have</div><div>basic leaves ru=
nning up to 0x21u already for recent processors. This value sort of forces =
a particular max leaf value.&nbsp;</div><div><br data-mce-bogus=3D"1"></div=
><div><br data-mce-bogus=3D"1"></div><div>Caleb</div></div></div></body></h=
tml>
--=_5971adc4-77e8-49f7-ba82-ae59d490cabd--


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:08:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:08:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717285.1119321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tOu-0003Bb-Iy; Mon, 06 May 2024 08:08:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717285.1119321; Mon, 06 May 2024 08:08:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tOu-0003BU-GE; Mon, 06 May 2024 08:08:44 +0000
Received: by outflank-mailman (input) for mailman id 717285;
 Mon, 06 May 2024 08:08:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3tOs-0003BO-JG
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:08:42 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d8f33e79-0b7f-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:08:40 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41ecffed96cso5626805e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:08:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t3-20020a05600001c300b0034df7313bf1sm10150992wrx.0.2024.05.06.01.08.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:08:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8f33e79-0b7f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714982920; x=1715587720; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SJoKfjr9VcDjkUuiasQBoBbYQ10TDbtiRDBcCFiZOno=;
        b=aKcm4dluLWXlUPjZdfutp550xxJk86kxxdB17SKB3Qh80KbRCWCnA81bLhE+Vzmwh1
         D8BoY1AKZ7Yu/kCbsscNU7fFT6fdet7imGZ/wp2C9inKePAPxC0bw3iBzHVgu/wl5FcK
         LlRtXChSLO2EEhmVJIWp98K7PIhgoI3YuAumlfMLUwCpt3A3ff3ykkYyhN0W72g69LmZ
         kXK7SIwUAOk+KKpwrwbFe8PrS5A2RLEaRG6TNwtFksb6Lr9raEQG5jyx+048AunLzKqi
         DcLxZ5qpqbUWXZORoBJatOSnL6TEBCwzNLepFwpmu76yEYZZCvMhtrBFJ/dmOAq33gQ1
         GKkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714982920; x=1715587720;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SJoKfjr9VcDjkUuiasQBoBbYQ10TDbtiRDBcCFiZOno=;
        b=aVBS6NPMeAezX7neP2qcztRCLu8nAxEmltedtDhf77GD2YdwVTVu7U3SqQz+aglhDp
         ver6hvgQJ2BlJly/TuDFDKQOHYgBoGXpxOvsPj5WbV/x1fOst8o3EVTOSzP4CgZHDWOZ
         p1C+Ioc3osphZLqwRUi1Pnn+GTbI2xS2tIkZpB6I/emvHYIY03vUOM8uJ2f9xH41v4WY
         ZFgNR5T2HSvrG5uWKqkSn8vyIBjs1skhwqRzybmcv+Wrswj7bf+o03x/dQQV3iUcG7kv
         vKeTBg08tioEv55Jwc7dLqP4p74GtaN8bbilaraskwx6OgsBDH5jmzBOGF9tGy9H1JXC
         UkXw==
X-Gm-Message-State: AOJu0YyXISXybAqQ+OL0Gsj/kKom0Rp6uGSigjSPFSoEl1+UDVqCAp3n
	f4FrXN/g9AgCF3j2evAZPmux+i1IB7cmb1rd2lvz6wbdAn72HgZl3cXeskbiiHsFrjl6PCpWNw4
	=
X-Google-Smtp-Source: AGHT+IHzqrb5PuWE3VdQ8R8NnA2MV8PFiXfJtnIFGE1rHi/Bg1HEDxQxA8YwNop4JjmA3VxSeEchzA==
X-Received: by 2002:adf:e947:0:b0:34d:b75c:c7ec with SMTP id m7-20020adfe947000000b0034db75cc7ecmr6933615wrn.8.1714982919612;
        Mon, 06 May 2024 01:08:39 -0700 (PDT)
Message-ID: <33b93fdf-bf16-49b8-aec2-0b2c19f5c471@suse.com>
Date: Mon, 6 May 2024 10:08:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] gunzip: don't leak memory on error paths
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

While decompression errors are likely going to be fatal to Xen's boot
process anyway, the latest with the goal of doing multiple decompressor
runs it is likely better to avoid leaks even on error paths. All the
more when this way code size actually shrinks a tiny bit.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
This is quite the opposite of Coverity reporting use-after-free-s and
free-after-free-s in inflate_dynamic() for tl and td, for an unclear to
me reason.

--- a/xen/common/gzip/inflate.c
+++ b/xen/common/gzip/inflate.c
@@ -757,16 +757,14 @@ static int noinline __init inflate_fixed
     }
 
     /* decompress until an end-of-block code */
-    if (inflate_codes(tl, td, bl, bd)) {
-        free(l);
-        return 1;
-    }
+    i = inflate_codes(tl, td, bl, bd);
 
     /* free the decoding tables, return */
     free(l);
     huft_free(tl);
     huft_free(td);
-    return 0;
+
+    return !!i;
 }
 
 /*
@@ -940,19 +938,17 @@ static int noinline __init inflate_dynam
     DEBG("dyn6 ");
 
     /* decompress until an end-of-block code */
-    if (inflate_codes(tl, td, bl, bd)) {
-        ret = 1;
-        goto out;
-    }
+    ret = !!inflate_codes(tl, td, bl, bd);
 
-    DEBG("dyn7 ");
+    if (!ret)
+       DEBG("dyn7 ");
 
     /* free the decoding tables, return */
     huft_free(tl);
     huft_free(td);
 
-    DEBG(">");
-    ret = 0;
+    if (!ret)
+       DEBG(">");
  out:
     free(ll);
     return ret;


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:13:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:13:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717289.1119330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tTl-0004sE-4a; Mon, 06 May 2024 08:13:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717289.1119330; Mon, 06 May 2024 08:13:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tTl-0004s7-1n; Mon, 06 May 2024 08:13:45 +0000
Received: by outflank-mailman (input) for mailman id 717289;
 Mon, 06 May 2024 08:13:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3tTj-0004qb-TJ
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:13:43 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d4f4492-0b80-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:13:42 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e2a4c20870so21164591fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:13:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 je8-20020a05600c1f8800b0041bf28aa11dsm15132210wmb.42.2024.05.06.01.13.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:13:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d4f4492-0b80-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714983222; x=1715588022; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Oa41dDjxU7rF/t2uSqi3aUDKSelv+d9yen6SWI3wiag=;
        b=ZhMA+/ScXGIZl1SGSlTPn6kwyV1nHkQuq2gW217fxvhc/QU7DNAf5/UXKjunfHdXmJ
         MYgxqgEFa0zv9Z7OKBs72N1Uotj6tPHT598m9PkVrUmWu6Fsyq1+Db7i5XNTxlYhNx1I
         6sNArM1gKScU5xFdurNaQ67ueMpWSU3K1/gxm8SQEKY2O/vMNYGIXK9FdtVMPUTJW0ZP
         myS3cu7Y1HZrVOPZMVrHHG9cc2gw5juneFXZBg/uH5a090qjWjny7ga9BINtL5bjLrhi
         am1EnbJvke3DdGEtN8i5OQes6YOSDD+3csv5Q34QE2LCRgyZ2RWbehk7OVVkHdWj2qcG
         +0Pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983222; x=1715588022;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Oa41dDjxU7rF/t2uSqi3aUDKSelv+d9yen6SWI3wiag=;
        b=P6n5qLLvhnWlfAqqj15mIhHg79hNnFDJthSP8L+BNrh2au5hWSTN4+7chM+AueO9dz
         oMXLc6FuSO8If07KnO8K5C5teFk1KOVEoYOn7gfMdyY4HxHRS8RHchxmSX8RyfCFC2N9
         47XEbeCDwYUMRpd4TaN0ktYHVGUCwihAXbXIllh9SWES+XCRn/IZumN12oEBmHXHcsOr
         ltYuyVcw8oLrR0FpclHh19m3KGobPlkxoXKejYoIs3FxP5e99nZEDtqlP2cVtM5t+XX9
         k/Cx21GZV3a1cOcPmj3MIu1LLMs28P9yfFTS2i63f7jH4dn9HMNn+8Ne5Bhuy8pECyDv
         oEUQ==
X-Gm-Message-State: AOJu0Yxtv7PVOCyMOfZVM7/+tzxMkzWm06u+wOzC7s8BwrQ4QU81qdFz
	slbuTr6utzsTGRLl6DXJ0kJmjZJut2XKY4EISdivmfaUoC7h1ELd3KbOJc2dbg==
X-Google-Smtp-Source: AGHT+IG89GaC2yK1oIXI1RdsdeHVODJUmOs4OwfJDpDJwMBwGlUEDdWyOHLuOQRuSi2t2VwgXvH7JQ==
X-Received: by 2002:a2e:3619:0:b0:2e2:891d:5f62 with SMTP id d25-20020a2e3619000000b002e2891d5f62mr4306487lja.29.1714983222093;
        Mon, 06 May 2024 01:13:42 -0700 (PDT)
Message-ID: <7fbe2757-99dd-486b-940f-4d30379f5c23@suse.com>
Date: Mon, 6 May 2024 10:13:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID
 constants
To: Fonyuy-Asheri Caleb <fonyuy-asheri.caleb@inria.fr>
References: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>
Content-Language: en-US
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 09:46, Fonyuy-Asheri Caleb wrote:
> I came across these constants in the code (xen/include/xen/lib/x86/cpu-policy.h file) but no explanation of why they have been set that way. 
> 
> #define CPUID_GUEST_NR_BASIC (0xdu + 1) 
> #define CPUID_GUEST_NR_CACHE (5u + 1) 
> #define CPUID_GUEST_NR_FEAT (2u + 1) 
> #define CPUID_GUEST_NR_TOPO (1u + 1) 
> #define CPUID_GUEST_NR_XSTATE (62u + 1) 
> #define CPUID_GUEST_NR_EXTD_INTEL (0x8u + 1) 
> #define CPUID_GUEST_NR_EXTD_AMD (0x21u + 1) 
> 
> Please can someone explain to me why we have these constants or point to a documentation which explains it? 

These specify the number of (sub)leaves Xen supports, first and foremost to
dimension internal arrays accordingly. I.e. the "why" lies in the way they're
used.

> I am particularly interested in the CPUID_GUEST_NR_BASIC given that for intel processors for example, we have 
> basic leaves running up to 0x21u already for recent processors. This value sort of forces a particular max leaf value. 

Right, but support (for guests) for these needs properly enabling. Hence why
that limit is the way it is, for the time being.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:16:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:16:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717293.1119341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tWo-0005gx-IH; Mon, 06 May 2024 08:16:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717293.1119341; Mon, 06 May 2024 08:16:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tWo-0005gq-Fe; Mon, 06 May 2024 08:16:54 +0000
Received: by outflank-mailman (input) for mailman id 717293;
 Mon, 06 May 2024 08:16:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3tWn-0005gk-A3
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:16:53 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fd979ec8-0b80-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:16:51 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51f1bf83f06so1905567e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:16:51 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 b16-20020a0565120b9000b0051c4e9ebc71sm1551840lfv.210.2024.05.06.01.16.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 01:16:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd979ec8-0b80-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714983411; x=1715588211; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=6ltznsBNEm1qvEPIFxf6ALehqFUyCcgDjD4oZEHMjg8=;
        b=LALemgNe2DkFaYjgzob0rlmpJsEVPvDavq2kZqxaheZw7lsnz5FeEmYz9gCN8IxoIj
         t5wp6fm/jabZnMUT/jSDkzp1/Ue0WTLJ/ootfYbPqWZGhHYd/3zMrvr1zHiv8WO8tDcB
         /gQmG5FCciZ14RcdOR55uP4X8BTwPDLxJo2gjLAe8efS1lQ/aW0YPIpBo1u7VPmXndxV
         cEsxST1zzm20+GYU0cWFEwUN4kzferDJByj/vZWY26KBx2KZW1ZS+YAwdnv+dbjff+fU
         UQCwyum4cxbj8Yhqd2vbXgNltpEKLhq6YyyDFMDGI7JdRB8XMHQEOlv87hKwz68X0c1p
         F0iw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983411; x=1715588211;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6ltznsBNEm1qvEPIFxf6ALehqFUyCcgDjD4oZEHMjg8=;
        b=Am4BQKoU0iMc21VaP0OLfc/CH8e7RNzkLtwK7iGOK3r4ZJ3XuJT60L/1lzeYH0/h4z
         H8GsSiQqNbXagE/B11J2ziFyGLQWU9iZKWfrF+fUCqk8veWsaSkryIWH4ueY6Tspk/RE
         CHRvEBLRN9NJnp+V7RUBDNXmaKISseo/JsrpGiuqTA34xRmsWsYcZrrg352HssXPHbpD
         O/B52j1vSKvSqQbcJBVqF6E1jVE2yLltV3IosxTMzrEbeLSmIbUQvJaGKLCAnawmbUaT
         B2ojqkdiab+CQrgBozLjDZIV40Av6umajhwaMSPtcJmEfyyqup8mMl/pq6vQehFm0AhN
         tmJQ==
X-Forwarded-Encrypted: i=1; AJvYcCUaMEOhOwDdP7JSYAAPsK6Fl1JtQb/Utg4elk90Z8J/5AJafnTCWLgceuxCD/0wakz0/OmaCQpmHe0wSy+Lg/viXzFeU+28uPJgRBAGmQo=
X-Gm-Message-State: AOJu0Yz+ivS3L/QZY9SopHAZCHlzzqqmQrwSCTg0ou2lw/pTpgckH0TE
	lJneHl8PBuCKYdgcJWIZjk8hQiBnr8sHxlqr5O9zPVQCjeeQXlpf
X-Google-Smtp-Source: AGHT+IFDVyYtHfiZu1KbCQvjbacSqSyptEw8QaSVmLR7h/n247yQeXey5mnAhx3IjY2mwz6XdmSq3Q==
X-Received: by 2002:a05:6512:3f09:b0:51e:ff32:16a8 with SMTP id y9-20020a0565123f0900b0051eff3216a8mr8830963lfa.62.1714983410531;
        Mon, 06 May 2024 01:16:50 -0700 (PDT)
Message-ID: <2d81e4700075b55f1885a4b1c7ee44ad046b35f2.camel@gmail.com>
Subject: Re: [PATCH v8 02/17] xen: introduce generic non-atomic test_*bit()
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Mon, 06 May 2024 10:16:49 +0200
In-Reply-To: <93eeb1d8-d41f-40e8-8ca2-e6828877b53d@suse.com>
References: <cover.1713347222.git.oleksii.kurochko@gmail.com>
	 <1a0977e3cf5a2de9f760ca5ec89a0d096894a9e3.1713347222.git.oleksii.kurochko@gmail.com>
	 <3827c11c-6d47-411d-a356-871def4e5b30@suse.com>
	 <940f11ee23b43ada1dba50bc0236c4764eb13d71.camel@gmail.com>
	 <93eeb1d8-d41f-40e8-8ca2-e6828877b53d@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-06 at 08:33 +0200, Jan Beulich wrote:
> On 03.05.2024 19:15, Oleksii wrote:
> > On Thu, 2024-04-25 at 17:35 +0200, Jan Beulich wrote:
> > > > =C2=A0 #include <asm/bitops.h>
> > > > =C2=A0=20
> > > > +#ifndef arch_check_bitop_size
> > > > +#define arch_check_bitop_size(addr)
> > >=20
> > > Can this really do nothing? Passing the address of an object
> > > smaller
> > > than
> > > bitop_uint_t will read past the object in the generic__*_bit()
> > > functions.
> > It seems RISC-V isn' happy with the following generic definition:
> > =C2=A0=C2=A0 extern void __bitop_bad_size(void);
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 /* --------------------- Please tidy above here ----------=
------
> > ----
> > =C2=A0=C2=A0 - */
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 #include <asm/bitops.h>
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 #ifndef arch_check_bitop_size
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 #define bitop_bad_size(addr) sizeof(*(addr)) <
> > sizeof(bitop_uint_t)
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 #define arch_check_bitop_size(addr) \
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bito=
p_bad_size();
> > =C2=A0=C2=A0=20
> > =C2=A0=C2=A0 #endif /* arch_check_bitop_size */
>=20
> I'm afraid you've re-discovered something that was also found during
> the
> original Arm porting effort. As nice and logical as it would (seem
> to) be
> to have bitop_uint_t match machine word size, there are places ...
>=20
> > The following errors occurs. bitop_uint_t for RISC-V is defined as
> > unsigned long for now:
>=20
> ... where we assume such operations can be done on 32-bit quantities.
Based on RISC-V spec machine word is 32-bit, so then I can just drop
re-definition of bitop_uint_t in riscv/asm/types.h and use the
definition of bitop_uint_t in xen/types.h.
Also it will be needed to update __AMO() macros in <riscv>/asm/bitops.h
in the following way:
   #if BITOP_BITS_PER_WORD =3D=3D 64
   #define __AMO(op)   "amo" #op ".d"
   #elif BITOP_BITS_PER_WORD =3D=3D 32
   #define __AMO(op)   "amo" #op ".w"
   #else
   #error "Unexpected BITS_PER_LONG"
   #endif
Note: BITS_PER_LONG was changed to BITOP_BITS_PER_WORD !

Only one question remains for me. Given that there are some operations whic=
hcan be performed on 32-bit quantities, it seems to me that bitop_uint_t
can only be uint32_t.
Am I correct? If yes, do we need to have ability to redefine
bitop_uint_t and=C2=A0BITOP_BITS_PER_WORD in xen/types.h:
   #ifndef BITOP_TYPE
   #define BITOP_BITS_PER_WORD 32
  =20
   typedef uint32_t bitop_uint_t;
   #endif

~ Oleksii

>=20
> Jan
>=20
> > =C2=A0=C2=A0=C2=A0 ./common/symbols-dummy.o -o ./.xen-syms.0
> > riscv64-linux-gnu-ld: prelink.o: in function `atomic_sub':
> > /build/xen/./arch/riscv/include/asm/atomic.h:152: undefined
> > reference
> > to `__bitop_bad_size'
> > riscv64-linux-gnu-ld: prelink.o: in function
> > `evtchn_check_pollers':
> > /build/xen/common/event_channel.c:1531: undefined reference to
> > `__bitop_bad_size'
> > riscv64-linux-gnu-ld: /build/xen/common/event_channel.c:1521:
> > undefined
> > reference to `__bitop_bad_size'
> > riscv64-linux-gnu-ld: prelink.o: in function `evtchn_init':
> > /build/xen/common/event_channel.c:1541: undefined reference to
> > `__bitop_bad_size'
> > riscv64-linux-gnu-ld: prelink.o: in function `_read_lock':
> > /build/xen/./include/xen/rwlock.h:94: undefined reference to
> > `__bitop_bad_size'
> > riscv64-linux-gnu-ld:
> > prelink.o:/build/xen/./arch/riscv/include/asm/atomic.h:195: more
> > undefined references to `__bitop_bad_size' follow
> > riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol
> > `__bitop_bad_size'
> > isn't defined
> > riscv64-linux-gnu-ld: final link failed: bad value
> > make[2]: *** [arch/riscv/Makefile:15: xen-syms] Error 1
> >=20
> > ~ Oleksii
>=20




From xen-devel-bounces@lists.xenproject.org Mon May 06 08:20:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:20:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717297.1119351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ta7-00077u-15; Mon, 06 May 2024 08:20:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717297.1119351; Mon, 06 May 2024 08:20:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ta6-00077n-U2; Mon, 06 May 2024 08:20:18 +0000
Received: by outflank-mailman (input) for mailman id 717297;
 Mon, 06 May 2024 08:20:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3ta5-00077h-Vh
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:20:17 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 782abf08-0b81-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:20:16 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-34e0d47bd98so2204210f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:20:16 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i15-20020adfb64f000000b0034af40b2efdsm10059143wre.108.2024.05.06.01.20.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:20:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 782abf08-0b81-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714983616; x=1715588416; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=s2rPXEsoE9bbs26kSdurMoDF4s8RZXdUIvKwrhiHXQs=;
        b=SJhjf+gWleW0Q8Po+oQO/wyR0LGLFgYXFXZ7I6p7reHG4P+0VM3XD9U54BNd80Tg+h
         IEtZ+ogPZ4ZUBuLaaecBVq4pRIMw3UJfnw9CcSdW9pIokXtmQGXNGmObEGrt6dT3w78X
         /+Md/yrjjLms7FXb1BVuRRVYgpHHUYkvlVxj6W+/FhsQ+s517iefi5jFLKyyloqjWZv2
         ug/HoSY2lXwj26KtET2rTiYyVeY//oIWRB/8YquYYRZoV6S+L5RuHlgcqBkdYBhKsCxA
         gQtZIXwXju+3nf15doLMxa8QbsJZOeKq9vlm1HBOsXj+eqlmHi+u64s8wOdawZjbnUWx
         VXbQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983616; x=1715588416;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=s2rPXEsoE9bbs26kSdurMoDF4s8RZXdUIvKwrhiHXQs=;
        b=u5LoEQj49sEbdvPIIYytUeROqsPUJ4cR6wzoX14Q8APzpxLTd3u8Cewqk9Y+hNYETX
         TAH13mn0aDo8E+aSgZhb2vJfZbHJCFyj7Z8T2zRbT/1J+qnq18FT2TTsW3RjIIe7g8Ud
         QX5H9UpzRsl4D590wbNjWmOjwtF73Q9AL5MUzPGcQn573Yt7WTbUrDRRwbcmSZSU6Eqa
         mVbuwgeOjXUB+ZMONCxXloC5a5iQsO9cOQeFjEB0I5eiM2ruDQEIglYaoz4O0bcHOj2A
         cq8E3kCjK5rboIWEJEfqhnVZshNfNoaeBikpQNjEE1JsmTh68THcTltUTsPRZNgrAjnE
         5ZRg==
X-Forwarded-Encrypted: i=1; AJvYcCWVf4hpb7nddVdatBGzwm5BD3LsBfJh62h/9EeCdbKSVP5ARnTiZNMuJzIuBUNxGKOsLphB74xkZtck/8KN0B/mtJQ72QmC71s0ziN+1As=
X-Gm-Message-State: AOJu0Yw8pOlX6LWAHgXHQFApsMSpKKkFxloCyOnFvu89eCrJ5Qcarbi/
	VMBr/h+WnDCx8L8O46/LPv5SRxOqYuf9lm9S9w4/eg8rd8q8h2yKtqt0tbfGaA==
X-Google-Smtp-Source: AGHT+IGImdOw5vmpKIpDmL62hzspCxmOJQS2Eof0CXMuQUtVvywxBvsiif33w5bhse7bZSi82gPu7Q==
X-Received: by 2002:a05:6000:c0a:b0:34e:de3b:3ccb with SMTP id dn10-20020a0560000c0a00b0034ede3b3ccbmr3835159wrb.14.1714983616250;
        Mon, 06 May 2024 01:20:16 -0700 (PDT)
Message-ID: <cc1f42bf-7629-41fa-99b5-f77f15a371be@suse.com>
Date: Mon, 6 May 2024 10:20:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/5] x86: Add usage() to print out usage message
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-4-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430124709.865183-4-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 14:47, Fouad Hilly wrote:
> Refactor xen-ucode tool by adding usage() to handle usage\help messages.
> As we add more command options this will keep help\usage messages in a common block.
> Only generic error message is printed to stderr. usage and show_curr_cpu are printed to stdout.
> 
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> ---
> [v3]
> 1- Reverted usage message.
> 2- Utilized usage().
> 
> [v2]
> 1- Improved message description.
> 2- Fixed formatting and indentation.
> 3- Error message to print to stderr.

Already on v2 I pointed out that this change wasn't really done. And now,
in v3, ...

>  tools/misc/xen-ucode.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
> index c6ae6498d659..005bf85b6551 100644
> --- a/tools/misc/xen-ucode.c
> +++ b/tools/misc/xen-ucode.c
> @@ -17,6 +17,13 @@ static xc_interface *xch;
>  static const char intel_id[] = "GenuineIntel";
>  static const char   amd_id[] = "AuthenticAMD";
>  
> +static void usage(const char *name)
> +{
> +    printf("%s: Xen microcode updating tool\n"
> +           "Usage: %s [<microcode file> | show-cpu-info]\n",
> +           name, name);
> +}

... you're still unconditionally printing to stdout, when ...

>  static void show_curr_cpu(FILE *f)
>  {
>      int ret;
> @@ -89,9 +96,8 @@ int main(int argc, char *argv[])
>      if ( argc < 2 )
>      {
>          fprintf(stderr,
> -                "xen-ucode: Xen microcode updating tool\n"
> -                "Usage: %s [<microcode file> | show-cpu-info]\n", argv[0]);
> -        show_curr_cpu(stderr);
> +                "%s: unable to process command line arguments\n", argv[0]);
> +        usage(argv[0]);
>          exit(2);
>      }

... originally the usage message (correctly) went to stderr in this case.
It ought to go to stdout only in response to something like a --help
command line option.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:21:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:21:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717301.1119361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tbM-0007hO-DG; Mon, 06 May 2024 08:21:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717301.1119361; Mon, 06 May 2024 08:21:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tbM-0007hH-A4; Mon, 06 May 2024 08:21:36 +0000
Received: by outflank-mailman (input) for mailman id 717301;
 Mon, 06 May 2024 08:21:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3tbK-0007h9-Us
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:21:34 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a5830ae7-0b81-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:21:32 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2dd6a7ae2dcso27067431fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:21:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i15-20020adfb64f000000b0034af40b2efdsm10059143wre.108.2024.05.06.01.21.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:21:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5830ae7-0b81-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714983692; x=1715588492; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DTGGRFlH0AZmaLfikDiNRxLL7Y55HnvFSN6jh6IGiwk=;
        b=UwSE30QGHozCUSZX4sCOPVTzpITmh+SB+MWSf/WmKrI3Hwk+QHTWahPCPS/QRm36hf
         ygN7rY6NEkuPs605zNHn89HXhDXIqjQIoC7LQ4zdnl/0OviHSNFWkmsoJw0iXsnoviXD
         m8YwZQF5daChe3Y3v+pC+kw0xB60BgTefDWwRWUy/z2GE5Jv1SbN1R6LbZrKv/3/e8ti
         vlNMHsqqsHxaXuv7fF8utB27Kv0Qvu4NqcMzB9szgvyU1v6Hwybs4rw2bueYqgTPnVbO
         4p2OglYgY7VOnmSurjJYjIlEyOdeK79fZw7CNDKi7UXSmY4O6LpoFDCynW56g6UgdlpC
         ot0g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983692; x=1715588492;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DTGGRFlH0AZmaLfikDiNRxLL7Y55HnvFSN6jh6IGiwk=;
        b=eBeiBvLLxaViYnpZvQTF4ASuy+pMHAkFCsly12XVzoQ9pdPuCCCJvlHsebOB2EKfiN
         IMWaJCTHFvoK6F45iFqMtTfzYR71OsBbJwjixKlez9eeAyJ7rLYEJ/Iey/nf0lkkkYfV
         nnFferdTbgj0F3Hqu095WBeq5pZZy+RelRX/bhEhGvIPwKLL5qG+NDd9bLoebkHwUqfI
         dFL60m9S+iX6PMJtaCBUPZ0FJwOZOqSXc7oGOdLxYmK+mGZf65vdVB0tWWWr17NdADaV
         iRsKgAWgA8UMiIiXch9yqhVPb6TmzD5InDIBAl6/nI1I5MWUbsffQz8oV85suMVRAKiQ
         q9nA==
X-Forwarded-Encrypted: i=1; AJvYcCUjocIiKtbw5H3AXERN6UdGNdzdbswk5xmFG43vpan9iM3+jsNNVgo9JqnJzA2NsdQvtMhCs+OEF+lbGEyosa5XqfD4EPd33o/jyij6te8=
X-Gm-Message-State: AOJu0YyMvDDVh8AeSXdJX1Np6Z2002idpUrLKIxLUiuYSoQwWugAOJuq
	z1aKDG+Hn6UWephE0gDIAYHfQn5E5TYfXZ6Rin87u22P2cyeZLeXLy9zgwXxg53pBer5k3wwR3E
	=
X-Google-Smtp-Source: AGHT+IE9k99HxbRquyraMmYVMixXRU4J+AVLJ+zF+y+Wjem1E4i6e+g82tl0CO6qCBjO5Gji+PaWAw==
X-Received: by 2002:a2e:f02:0:b0:2e2:9842:a9d3 with SMTP id 2-20020a2e0f02000000b002e29842a9d3mr5644533ljp.46.1714983692468;
        Mon, 06 May 2024 01:21:32 -0700 (PDT)
Message-ID: <fc0c86f5-837a-45e8-807e-6861063be85f@suse.com>
Date: Mon, 6 May 2024 10:21:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/5] x86: Use getopt to handle command line args
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-5-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430124709.865183-5-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 14:47, Fouad Hilly wrote:
> Use getopt_long() to handle command line arguments.
> Introduce ext_err for common exit with errors.
> 
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>

Nit: Neither subject nor description make clear ...

>  tools/misc/xen-ucode.c | 45 +++++++++++++++++++++++++++++++-----------
>  1 file changed, 33 insertions(+), 12 deletions(-)

... what tool is actually being touched.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:24:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:24:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717305.1119371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tdu-0000Aq-Oi; Mon, 06 May 2024 08:24:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717305.1119371; Mon, 06 May 2024 08:24:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tdu-0000Aj-Ll; Mon, 06 May 2024 08:24:14 +0000
Received: by outflank-mailman (input) for mailman id 717305;
 Mon, 06 May 2024 08:24:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3tdt-0000Ad-CP
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:24:13 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 04990f20-0b82-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:24:12 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51ae2e37a87so2159826e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:24:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g4-20020a5d5404000000b0034ca136f0e9sm10078398wrv.88.2024.05.06.01.24.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:24:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04990f20-0b82-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714983852; x=1715588652; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3RnKgDBuRQFq9ioB+Ph2oGOdUSZAJ31Q/yo5SWdrVVQ=;
        b=UwJbX/adTFzsKrCxZ+D3tyabCZv6ZM/NdjfMsSvrmsPcZ+NwGG1fSEDX11aThrWIVI
         bXg2eKkkMyMcnl75NtVaTMfAEt4q53HasUlwpqKgC7n1OUFPLkL+TkShDPqFU89GENOb
         i0E6ee2zUfYHSH55OujK1E+No8/8KXaL6sSA0K9jdK9r8aUBUIJA0FvtMsHRbyt07YTR
         MKXrAPzCReA9+K82C8dyvVl3VzAyqK31d2KuJArGuZVudSiOJ8MVinCo3E818MR9NouG
         V242wlVkD5SaazKTjapZDMfY/bKcqyN0sVEAA/XYKrJtkTJss3QHv1pA8M/Ht7MhbI/k
         2NUQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983852; x=1715588652;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3RnKgDBuRQFq9ioB+Ph2oGOdUSZAJ31Q/yo5SWdrVVQ=;
        b=SFFJAaGU9nNYSOZnC6DiFqFwEbelYjiKgsVqPHjUrz7b/6Lh/h/XJKadYXhbA8OrIp
         9kNPDG2Frl0J4jaUcc4fKpFLGF8FADE7d0oFHZZUsIaKWbkYYzFGvBSmAOZUQymBKJSA
         bmqohLIrcdPGxqNfUdfoW+EixCOe3PurGIIu0nwOWk7NkmW8t2mc/kmhMIkzLGM5P/wj
         zjqvD9zhIIBGaVjoCxGxfu5XHGzhXRL3/S62CNlsA5cw0ugYmAS8DMisAn4hjE2oK1V0
         6CMGgn2k+8URaEyiUHC4B0yugpop6R3lBOJajOlpHJb7Sz5C5teaMaw1FzP70H1pZQoj
         Brhg==
X-Forwarded-Encrypted: i=1; AJvYcCU6hQylR4ENPbI6ENWGRvRnrSCRWTT1Tq/k0nMUlOErtvjlHYfyG2HdydM6sV3lEXYz635M33+h8VffOwcn8Pr4v3F53t8wzKkQp8CwGts=
X-Gm-Message-State: AOJu0YxyS4yY6IqiFitZD5EGHOiJaEqL9SODkZYPaXSFM6fP8YXg+Nc3
	V7DxCOPzcBzsQjcYGqEWSqiIpZIXDC2zj8b4UcXrSe+AVpR8byHgNMIrTu/1DA==
X-Google-Smtp-Source: AGHT+IFRaAv70T/zkgmRKpffOfiooMo4XeSxJdaVqlPvZkjHzcghW0dd5LQf3AGCdz9tE/OgbDbEZw==
X-Received: by 2002:ac2:46d4:0:b0:51e:f8ae:db35 with SMTP id p20-20020ac246d4000000b0051ef8aedb35mr5782212lfo.43.1714983851870;
        Mon, 06 May 2024 01:24:11 -0700 (PDT)
Message-ID: <7f2eb0fb-3ed1-4248-90bd-2bc3551a97f1@suse.com>
Date: Mon, 6 May 2024 10:24:12 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 02/17] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1713347222.git.oleksii.kurochko@gmail.com>
 <1a0977e3cf5a2de9f760ca5ec89a0d096894a9e3.1713347222.git.oleksii.kurochko@gmail.com>
 <3827c11c-6d47-411d-a356-871def4e5b30@suse.com>
 <940f11ee23b43ada1dba50bc0236c4764eb13d71.camel@gmail.com>
 <93eeb1d8-d41f-40e8-8ca2-e6828877b53d@suse.com>
 <2d81e4700075b55f1885a4b1c7ee44ad046b35f2.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2d81e4700075b55f1885a4b1c7ee44ad046b35f2.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 10:16, Oleksii wrote:
> On Mon, 2024-05-06 at 08:33 +0200, Jan Beulich wrote:
>> On 03.05.2024 19:15, Oleksii wrote:
>>> On Thu, 2024-04-25 at 17:35 +0200, Jan Beulich wrote:
>>>>>   #include <asm/bitops.h>
>>>>>   
>>>>> +#ifndef arch_check_bitop_size
>>>>> +#define arch_check_bitop_size(addr)
>>>>
>>>> Can this really do nothing? Passing the address of an object
>>>> smaller
>>>> than
>>>> bitop_uint_t will read past the object in the generic__*_bit()
>>>> functions.
>>> It seems RISC-V isn' happy with the following generic definition:
>>>    extern void __bitop_bad_size(void);
>>>    
>>>    /* --------------------- Please tidy above here ----------------
>>> ----
>>>    - */
>>>    
>>>    #include <asm/bitops.h>
>>>    
>>>    #ifndef arch_check_bitop_size
>>>    
>>>    #define bitop_bad_size(addr) sizeof(*(addr)) <
>>> sizeof(bitop_uint_t)
>>>    
>>>    #define arch_check_bitop_size(addr) \
>>>        if ( bitop_bad_size(addr) ) __bitop_bad_size();
>>>    
>>>    #endif /* arch_check_bitop_size */
>>
>> I'm afraid you've re-discovered something that was also found during
>> the
>> original Arm porting effort. As nice and logical as it would (seem
>> to) be
>> to have bitop_uint_t match machine word size, there are places ...
>>
>>> The following errors occurs. bitop_uint_t for RISC-V is defined as
>>> unsigned long for now:
>>
>> ... where we assume such operations can be done on 32-bit quantities.
> Based on RISC-V spec machine word is 32-bit, so then I can just drop
> re-definition of bitop_uint_t in riscv/asm/types.h and use the
> definition of bitop_uint_t in xen/types.h.
> Also it will be needed to update __AMO() macros in <riscv>/asm/bitops.h
> in the following way:
>    #if BITOP_BITS_PER_WORD == 64
>    #define __AMO(op)   "amo" #op ".d"
>    #elif BITOP_BITS_PER_WORD == 32
>    #define __AMO(op)   "amo" #op ".w"
>    #else
>    #error "Unexpected BITS_PER_LONG"
>    #endif
> Note: BITS_PER_LONG was changed to BITOP_BITS_PER_WORD !
> 
> Only one question remains for me. Given that there are some operations whichcan be performed on 32-bit quantities, it seems to me that bitop_uint_t
> can only be uint32_t.
> Am I correct? If yes, do we need to have ability to redefine
> bitop_uint_t and BITOP_BITS_PER_WORD in xen/types.h:
>    #ifndef BITOP_TYPE
>    #define BITOP_BITS_PER_WORD 32
>    
>    typedef uint32_t bitop_uint_t;
>    #endif

Probably not, right now. Hence me having said "As nice and logical as it
would (seem to) be" in the earlier reply.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:25:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:25:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717308.1119381 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tfM-00013u-2L; Mon, 06 May 2024 08:25:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717308.1119381; Mon, 06 May 2024 08:25:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tfL-00013n-VT; Mon, 06 May 2024 08:25:43 +0000
Received: by outflank-mailman (input) for mailman id 717308;
 Mon, 06 May 2024 08:25:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3tfL-00013h-9E
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:25:43 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3a5115fb-0b82-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:25:42 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-518931f8d23so1546444e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:25:42 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 c3-20020a056512238300b0051cd6c9e9a2sm1532357lfv.152.2024.05.06.01.25.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 01:25:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a5115fb-0b82-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714983942; x=1715588742; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=puH2uGZgKYQUwSAx1rYEmvYiiuS0k81HyWFgtxzTHHY=;
        b=lPYKOR2btVCCowD5ELq90w25j8eiztKhD/jSLPzLjYrJHGAQKLx/+9y0CHNiihfGf2
         hVVhkmCpZBl1FCYDr5rSlgwDfSqNdkx9RAT0hI/wkiC4ZBdbzMsqsDaXGArv5BK6/faM
         2ulLjC6PS6Z9Zy1C/BipOyMC3Xu2W/WtLbvn8v5i3AxP4Qn7zgz/N43pj+sXooItTEif
         XE3rN/p5Se0L9ChwfONGYnW6ZbqbmKMwNk1nCQ/fvZpp3ZosO3jpsDSdsEbixtf9BuGg
         BJwiQT4Gp7NduU5541773v3dbzs8dI/s6w5UoOwkQAkLd3ZXtD7LUFzUpeMPf8TrSp45
         IDCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714983942; x=1715588742;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=puH2uGZgKYQUwSAx1rYEmvYiiuS0k81HyWFgtxzTHHY=;
        b=cheNyl+QFqvuvVoIrNwFlch9M7WrJjkIlCY1pAiXec0GiVJU3Fmte48LAZLOKwtgg5
         AxqjaX072+V6SxoqxEV/SUETI3ZzeJU8w4dE4xXzjzL4c2fIbx/pLIR/hl6MStCkj7dE
         e6lQekwOj40BpE34puZ5JhVvmQqJyRUVaVyPW1blqDdSjIwrKCUC6ig3NO8wLy6l9oit
         ESWNvFldrXOR0Fp8a5fEDawh+Af38qBcUZvZTs7nAu8VQXkaGzhefW0QRumBdZz8NTu+
         Cn75DRBb4lmSxmK7IWNPAJEO08FfcW2qbAbTLnWnK20YPx6Td/znqV8nlMhR0ULsna2f
         nvfw==
X-Forwarded-Encrypted: i=1; AJvYcCVfl0KFjcO+NGsMwu1fGjuPnktzscVya6yVe0blxQPNy/k5cJ34D3jMSkd+8I6Pncz5WEzWnJ9tV7ZgE1c5c1tO+U91paSGGiCXqJHaHN0=
X-Gm-Message-State: AOJu0YwjUaK+cXYjGxMajNLruGr7zaOwOmBkyaS/Hs9K8OoE7CKRAlg7
	l8eZCS6T6Xg0loy3YWOaAGOT59z0b59UFaIqSVxbs2HfmzffLHoz
X-Google-Smtp-Source: AGHT+IE8K8rwmBVSUmxgfm3wxQpwyx/B1T8lvnAmjmG7fbLYIU9sEDnKKptpJ0FDqsJ8pqLPhTCplQ==
X-Received: by 2002:ac2:43a7:0:b0:51c:348:3ba9 with SMTP id t7-20020ac243a7000000b0051c03483ba9mr5670038lfl.22.1714983941693;
        Mon, 06 May 2024 01:25:41 -0700 (PDT)
Message-ID: <609d3d5da1c95b14281233860f7619e6d15de195.camel@gmail.com>
Subject: Re: Xen 4.19 release status tracking list [ May ]
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
Date: Mon, 06 May 2024 10:25:40 +0200
In-Reply-To: <8c0bc745-05f5-4839-b920-7830e2c55688@suse.com>
References: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
	 <8c0bc745-05f5-4839-b920-7830e2c55688@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-06 at 09:11 +0200, Jan Beulich wrote:
> On 03.05.2024 18:54, Oleksii wrote:
> > *** x86 ***:
> > =C2=A0 * [PATCH 0/4] iommu/x86: fixes/improvements for unity range
> > checks [
> > https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citr=
ix.com/
> > ]:
> > =C2=A0=C2=A0=C2=A0 - almost patch series have been merged already excep=
t the
> > patch:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [PATCH 4/4] iommu/x86: make =
unity range checking more
> > strict
> >=20
> > =C2=A0 * [PATCH 0/8] x86: support AVX10.1 [
> > https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@=
suse.com/
> > ]:
> > =C2=A0=C2=A0=C2=A0 - two patches of patch series are waitng to merged/r=
eviewed:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [PATCH 1/4] amd-vi: fix IVMD memory type=
 checks
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [PATCH 4/4] iommu/x86: make unity range =
checking more strict=20
>=20
> This sub-item doesn't really fit the topic, and seems to rather
> belong
> to the earlier one?=C2=A0
Thanks for correction, it is belonged to previous topic.

> Where (correctly) 1/4 isn't listed anymore, for
> having gone in. As to 4/4 - Roger, I'm not sure I can conclude what
> the
> plan here was: Are you meaning to submit an updated version, or did
> we
> rather settle on not further pursuing this?
>=20
> > =C2=A0 * APX support?
>=20
> I think you should drop this now. I'm throttling further work on the
> insn
> emulator, as long as I have so many other patches there pending
> review.
>=20
> > =C2=A0 * [PATCH v6 0/4] x86/pvh: Support relocating dom0 kernel [
> > https://patchew.org/Xen/20240327215102.136001-1-jason.andryuk@amd.com/
> > ]
>=20
> There had been a v7, and all of that plus a follow-on adjustment have
> gone
> in.
>=20
> > *** common ***:
> > =C2=A0 * annotate entry points with type and size" series:
> > =C2=A0=C2=A0=C2=A0 The bulk of this has gone in, but there'll want to b=
e follow-
> > ups.
>=20
> I think these follow-ups now want tracking on a per-arch basis (x86
> and
> Arm, with PPC and RISC-V filly done).
>=20
> > =C2=A0 * [PATCH v2 (resend) 00/27] Remove the directmap [
> > https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazo=
n.com/
> > ]
> > =C2=A0=C2=A0=C2=A0 - 7/27 were merged.
>=20
> Hmm, no, I don't think that one was. Other were, yes.
>=20
> > =C2=A0 * [PATCH v6 0/7] MSI-X support with qemu in stubdomain, and othe=
r
> > related changes:
> > =C2=A0=C2=A0=C2=A0 - new patch version was sent
>=20
> 1/7 was committed.
>=20
> > =C2=A0 * [PATCH v3 0/4] xenwatchdogd bugfixes and enhancements [
> > https://lore.kernel.org/xen-devel/20240411182023.56309-1-leigh@solinno.=
co.uk/
> > ]:
> > =C2=A0=C2=A0=C2=A0 new patch series were sent.
>=20
> Was there anything left from that series?
I double-checked and everything has been merged. Thanks for correction.

I'm considering writing a script to automate the process of checking
whether the patches in a patch series have been merged or not.
Currently, I'm doing it manually, so there's a chance I might miss
something. It would be helpful if such a script already exists.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:32:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:32:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717316.1119391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tlt-0002Y0-OW; Mon, 06 May 2024 08:32:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717316.1119391; Mon, 06 May 2024 08:32:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tlt-0002Xt-Ks; Mon, 06 May 2024 08:32:29 +0000
Received: by outflank-mailman (input) for mailman id 717316;
 Mon, 06 May 2024 08:32:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=pn4q=MJ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s3tls-0002Xn-K5
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:32:28 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 29f28838-0b83-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:32:25 +0200 (CEST)
Received: from PH8PR02CA0020.namprd02.prod.outlook.com (2603:10b6:510:2d0::9)
 by PH7PR12MB8124.namprd12.prod.outlook.com (2603:10b6:510:2ba::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 08:32:20 +0000
Received: from CY4PEPF0000EE3A.namprd03.prod.outlook.com
 (2603:10b6:510:2d0:cafe::ce) by PH8PR02CA0020.outlook.office365.com
 (2603:10b6:510:2d0::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Mon, 6 May 2024 08:32:19 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EE3A.mail.protection.outlook.com (10.167.242.14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 08:32:19 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 03:32:19 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 03:32:17 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29f28838-0b83-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=STlMwvOnX1xW94KeO/qXF82bVT4wwlTkUpJtzx04PgDRmC755zr6stKnXTHylTd2OSeIW3izOJtYeTcpnTKWH1zzUGRr8so4xjBsrVkfex6gzLFepFbY0I1XkKn9Hm4Hi5XCDLt+Xtc6lgO61MIP3Jp4kAezzvNIzi1PZwX0n8rkO4wvEX0uF0Q4jHtb6LCPGYD7/H9z5jk9JFh4zMUuu+4GRap1dfk+RtbD8adkcGC52kW43fLbPzmxOIDGlitm9nX7TuijxXdzRsVQ8BPdUMDLUDfdsMaPox5+qLuQCmeDSaIaQ2gcL9u0zIQUIyuDJka2CfUq9V1SEcREHQnkmA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=f9xnphQ9L0vnxCKKlR624TQkPTFXuE84/afIalgczIs=;
 b=i0Id4On33VUrxYLtWtH39htpKkvAgDhsDZQrGeJ2c5Ja2TUEEcR0NdTjMCSVvzGF4iDdJVGEYokOS/1u8H50uvkwZb0uWTbx7pB8wf1vwmd1Zyaz0VpOLNb2a2zsElqkkESUFtFgCvZyZCQLGV90nf4lztdkX53lGMEudV0INF4S6lQpOflixlDdg9xcwVMz+FIQfhMVaVKzFT1fnJqnnnPdiy0a1ITbCKtmt5wxjZVtoKi76n2GEBPwlhWcKlUaHz9kB9ekI+X+LAKs/VZ4GJsvqnOyY5dMLMumfW8TF6PG30ie8vqkj41QyyqVhqC5LSuQCL+DPIFx2Iq2T5hUrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=f9xnphQ9L0vnxCKKlR624TQkPTFXuE84/afIalgczIs=;
 b=qbcynnFbET0LlWsSzkESzx/FkO9ycNxNgy6NsuaeD5zTrY/FPoLAKiTsTyFlxXmRXujYOLHOP0UzKUUiY/KnbXZcJjJELmYxl8bER1pONsBpKvrm00RBTXGpfJJO+xnldHGepoTfvvb/Ejalb2/xi57MVrfi8aT4fwMCeIa4ArA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
Date: Mon, 6 May 2024 16:32:16 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3A:EE_|PH7PR12MB8124:EE_
X-MS-Office365-Filtering-Correlation-Id: 771196be-39ee-4eb8-10e4-08dc6da70b40
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NVBzSVJ4RE4ySnU2MGlmNzVnWm9rR09UQzM5ZWpMd1laS0FaaEorcUVGby9Q?=
 =?utf-8?B?aG9jTytwaWc2MktIWWhkSUtUYlZpNHJ6YkZWTXZDNGQxOTQ1QzVIWjZnU3NK?=
 =?utf-8?B?ZGVMZ2lYalpYVDhsdkNNY2pjd3kxM3pwNzJpQUE1YzRuVE1lQTlUSlZ5Qkkx?=
 =?utf-8?B?UXZxSFlyaDFxSHVaeHdiYlN5bG9xMGdUZkwxOVQ4N3JVRjhpSHFub29KM0dh?=
 =?utf-8?B?SFM4SjRXcVIzeU5HUm1FemN5Ni85anMrdXN6bW5ZalNUZXd1RGhIY3puSFJy?=
 =?utf-8?B?SWljVUk3a0VaRng4QmZjUmR1TVdrNVN5V05JSG9OdHdUVEwvR1duVXNxaE9m?=
 =?utf-8?B?VnJJRTcyRWpQMnRvKzhndDlwQnNHQ0U1QnVSaW9VOUtMMlRQWHgvU2dWbHdB?=
 =?utf-8?B?UWROQTZVZDNjdTkrSE83bHh3bUd5OE90d0JNcEpEVS96NEdUdkpPckcxNjdP?=
 =?utf-8?B?QUFuY2xuRHVsKzBjZDViMGs1QlNaU003cEE5VndRS01hTFM4bzR1MGo3bldm?=
 =?utf-8?B?aVRKUnZXMGpuSzVDRW9WZjFBakVDRjdBOVBYVVFZWlFsektWM2Q5dkxvWm1L?=
 =?utf-8?B?cEVqRUlMajZkeG5YQ3JhcnRaMXVQWTVDUHI2UndSOTRuRElkOHpHZ2pETkFt?=
 =?utf-8?B?MmtMcUpiZ3FjeVZGOTBMTHU3ZXFPL1lwM3U0aWhRUDBvUldzTS9kUCtBNU1X?=
 =?utf-8?B?N01qR3JzSlVwZXZHYWlqSGNFNmxVa3VxbjR2UjVzVmxnL0NXRG9qNUZDM1FG?=
 =?utf-8?B?SHI1L0xJQVJMdVkyVVRSOXNjd0hCY1N5QjN6YTYrUnJKNXllNlEzYXMvSllM?=
 =?utf-8?B?bE0rZjdnRGhaVzVTWERoUnVEbUl1QU5mVG8zZXZCRG5EbkFOa09aRjlUVEhS?=
 =?utf-8?B?RHgzcEdKZURicFBOQ2FHeEVBMGtMUFhJY05Cc2ViRnRBcHIxUTE3Q0UrMXl6?=
 =?utf-8?B?bUJHWHg0YXY4eXM3Y3dqOXBwbzBkMzM2S3IrK0J5NmVqU2VnejF3ekVuZ2pM?=
 =?utf-8?B?aFBmczVHSzZtN1JpRkl0eENJc3VEcGV1T2MrUG1tdFExbUI1blQwVWEzK3ZY?=
 =?utf-8?B?TUV1Wk9WSjJSM05mUzFWUys5M1RaMFNYV0NiVjlEdk4yOTFsVVd6WUl4T3kw?=
 =?utf-8?B?d3Y4b0Fub1UvNzRlRUdSblp6cGN3dnVxWS9Jbi9EZDVYVkhtWHNPYzZhREw3?=
 =?utf-8?B?NFBuSmM1dmh2MGlkY1JjYmIraTUrRS9ZNjNvRVpXdmxOTUo1aHcyTkZTTVlL?=
 =?utf-8?B?RHFER3BpZU44U2E2SGhpTS8yWUpRRTBEWWhTS29RZUFWbmxPUlF4UG1wNUJM?=
 =?utf-8?B?MlRVYzFQdmE0WmpvbkFuSmpTZHFBZjVzRGNLSmlVQ2dnR2pYMDRSSnhFWHpq?=
 =?utf-8?B?eURzNkxQcDc1c2xOSzZxWnVIY2RDeFF3dkQzQWU3NVFkMlJvRm5XQnlwRlNh?=
 =?utf-8?B?ODRmdDZnYWxWZjFBYTk3NEpVVEJsc0ZQKzBvdWtudkNJVHQwYVpVcDJGZ3VL?=
 =?utf-8?B?T201NmRvNWNkdTQ0dFFuM21vTUVKRGM4QldvVkhQdGhmWkZ3SHRZeW5RcVhM?=
 =?utf-8?B?cGxUa1E2NXpzbmU1WlhhZ2tnMnYrdDFPT3N1c0xyaXZ4ekhFa213MGF0NXRP?=
 =?utf-8?B?RDhzeGZIczVJRW11V3hrOUlzc3NIQ3kwU1dSaUtmODRqZjE3bm1RVXA3VVVB?=
 =?utf-8?B?NGtZS1BnT0dKS0FvbVVIcUVqeS82bzEvQWxMaXZTclQzU2tPWDAwaG9rTDNO?=
 =?utf-8?B?Q3labDluUFUvRW84dlZXY3V4bkRHaFNmUDd4UVluUzBud2hNNU13RmlDRG9X?=
 =?utf-8?Q?ReEpEoWO3j2oKwGURUr15dKjqTVvoSesIv4rM=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 08:32:19.4388
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 771196be-39ee-4eb8-10e4-08dc6da70b40
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE3A.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8124

Hi Julien,

On 5/1/2024 4:13 AM, Julien Grall wrote:
> Hi Henry,
>
> On 30/04/2024 04:50, Henry Wang wrote:
>> On 4/25/2024 10:28 PM, Julien Grall wrote:
>>>> Thanks for your feeedback. After checking the b8577547236f commit 
>>>> message I think I now understand your point. Do you have any 
>>>> suggestion about how can I properly add the support to route/remove 
>>>> the IRQ to running domains? Thanks.
>>
>> I spent some time going through the GIC/vGIC code and had some 
>> discussions with Stefano and Stewart during the last couple of days, 
>> let me see if I can describe the use case properly now to continue 
>> the discussion:
>>
>> We have some use cases that requires assigning devices to domains 
>> after domain boot time. For example, suppose there is an FPGA on the 
>> board which can simulate a device, and the bitstream for the FPGA is 
>> provided and programmed after domain boot. So we need a way to assign 
>> the device to the running domain. This series tries to implement this 
>> use case by using device tree overlay - users can firstly add the 
>> overlay to Xen dtb, assign the device in the overlay to a domain by 
>> the xl command, then apply the overlay to Linux.
>
> Thanks for the description! This helps to understand your goal :).

Thank you very much for spending your time on discussing this and 
provide these valuable comments!

>>
>>> I haven't really look at that code in quite a while. I think we need 
>>> to make sure that the virtual and physical IRQ state matches at the 
>>> time we do the routing.
>>>
>>> I am undecided on whether we want to simply prevent the action to 
>>> happen or try to reset the state.
>>>
>>> There is also the question of what to do if the guest is enabling 
>>> the vIRQ before it is routed.
>>
>> Sorry for bothering, would you mind elaborating a bit more about the 
>> two cases that you mentioned above? Commit b8577547236f ("xen/arm: 
>> Restrict when a physical IRQ can be routed/removed from/to a domain") 
>> only said there will be undesirable effects, so I am not sure if I 
>> understand the concerns raised above and the consequences of these 
>> two use cases.
>
> I will try to explain them below after I answer the rest.
>
>> I am probably wrong, I think when we add the overlay, we are probably 
>> fine as the interrupt is not being used before. 
>
> What if the DT overlay is unloaded and then reloaded? Wouldn't the 
> same interrupt be re-used? As a more generic case, this could also be 
> a new bitstream for the FPGA.
>
> But even if the interrupt is brand new every time for the DT overlay, 
> you are effectively relaxing the check for every user (such as 
> XEN_DOMCTL_bind_pt_irq). So the interrupt re-use case needs to be 
> taken into account.

I agree. I think IIUC, with your explanation here and below, could we 
simplify the problem to how to properly handle the removal of the IRQ 
from a running guest, if we always properly remove and clean up the 
information when remove the IRQ from the guest? In this way, the IRQ can 
always be viewed as a brand new one when we add it back. Then the only 
corner case that we need to take care of would be...

>> Also since we only load the device driver after the IRQ is routed to 
>> the guest, 
>
> This is what a well-behave guest will do. However, we need to think 
> what will happen if a guest misbehaves. I am not concerned about a 
> guest only impacting itself, I am more concerned about the case where 
> the rest of the system is impacted.
>
>> I am not sure the guest can enable the vIRQ before it is routed.
>
> Xen allows the guest to enable a vIRQ even if there is no pIRQ 
> assigned. Thanksfully, it looks like the vgic_connect_hw_irq(), in 
> both the current and new vGIC, will return an error if we are trying 
> to route a pIRQ to an already enabled vIRQ.
>
> But we need to investigate all the possible scenarios to make sure 
> that any inconsistencies between the physical state and virtual state 
> (including the LRs) will not result to bigger problem.
>
> The one that comes to my mind is: The physical interrupt is 
> de-assigned from the guest before it was EOIed. In this case, the 
> interrupt will still be in the LR with the HW bit set. This would 
> allow the guest to EOI the interrupt even if it is routed to someone 
> else. It is unclear what would be the impact on the other guest.

...same as this case, i.e.
test_bit(_IRQ_INPROGRESS, &desc->status) || !test_bit(_IRQ_DISABLED, 
&desc->status)) when we try to remove the IRQ from a running domain.

we have 3 possible states which can be read from LR for this case : 
active, pending, pending and active.
- I don't think we can do anything about the active state, so we should 
return -EBUSY and reject the whole operation of removing the IRQ from 
running guest, and user can always retry this operation.
- For the pending (and active) case, can we clear the LR and point the 
LR for the pending_irq to invalid?

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Mon May 06 08:34:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:34:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717320.1119401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tnk-0003Mo-8I; Mon, 06 May 2024 08:34:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717320.1119401; Mon, 06 May 2024 08:34:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tnk-0003Mh-5R; Mon, 06 May 2024 08:34:24 +0000
Received: by outflank-mailman (input) for mailman id 717320;
 Mon, 06 May 2024 08:34:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3tni-0003Mb-UW
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:34:22 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ffa7319-0b83-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:34:22 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41ba1ba55ffso8591605e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:34:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 p8-20020a5d4588000000b00341ce80ea66sm10118640wrq.82.2024.05.06.01.34.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 01:34:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ffa7319-0b83-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714984461; x=1715589261; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=QwgDWwoaxTcN56Er8JLtUEke87wX8zbb94LYyK8GmOg=;
        b=p5sFfBuov7fhzr7lgIQiquT8umeLuHfuzb8fpIa3axRuXKRuH9I7fxuAQJtReRJlVq
         g7e3McW2tSV9YnyDPgvuB2O+yCTcm/0m4iIyJD1jFmJgj0w5rjkARvOknBSB8exPADQT
         2k8du8v5tFmGuyCBA3lOTn9ypvW2BAK6Z0JEk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714984461; x=1715589261;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=QwgDWwoaxTcN56Er8JLtUEke87wX8zbb94LYyK8GmOg=;
        b=jDqHWseas/bVu6pQh0qhtXtWBH4vDLyKArU/cJ34gPeX3NMazeNf61vbgL7FAKYlAm
         SdEtc5YRI6FyWa+zHXvoqBH97b5O4O+KAImc3YgBJjv7QoElWM/HKci8rGNnVC8es4Fw
         39Tcv9ZkwvpX/gXTefXvYihYxPMzp6Xnr3N6QUDEUt7smVGHJ3R7Qfh8lr4NthfTA+76
         grWSIwfPdR4QIXeMzE2OO2S1uA5cYGQdTgpwMKqMsytwBeW8Nm5wGdcOWBgR9iwT8NTS
         ecbhefZGG9WuEFZL9o22xNqRk6/npOZiK0BNeslYP3+QoYzojoLhDuYQTgwbx6O+654+
         3Mug==
X-Gm-Message-State: AOJu0Yw1nTKFnaio5OhszidWPRFi7fMWBLBI8QsSpU2m0irW7YDalmV4
	Ns0XcJ1q82TtfewGrmdrK+KI4OvtG9+v6BMYvKNeGipn7wWyFAEagW58c76i4dU=
X-Google-Smtp-Source: AGHT+IE1RCjHpW3xXL0Otsjm29GPRvmWbM+Z6ai4/nkhmQPTzvJCJOtILnnNLc5i0ZXrTW1UxITluQ==
X-Received: by 2002:a05:600c:3c89:b0:41b:34d3:42a5 with SMTP id bg9-20020a05600c3c8900b0041b34d342a5mr8860077wmb.1.1714984461444;
        Mon, 06 May 2024 01:34:21 -0700 (PDT)
Date: Mon, 6 May 2024 10:34:20 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Fonyuy-Asheri Caleb <fonyuy-asheri.caleb@inria.fr>
Cc: xen-devel@lists.xenproject.org
Subject: Re: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID
 constants
Message-ID: <ZjiWDMXFNmwSzAxQ@macbook>
References: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>

On Mon, May 06, 2024 at 09:46:58AM +0200, Fonyuy-Asheri Caleb wrote:
> Hi, 
> I am currently doing a study on the way xen handles CPUID information. 
> 
> I came across these constants in the code (xen/include/xen/lib/x86/cpu-policy.h file) but no explanation of why they have been set that way. 
> 
> #define CPUID_GUEST_NR_BASIC (0xdu + 1) 
> #define CPUID_GUEST_NR_CACHE (5u + 1) 
> #define CPUID_GUEST_NR_FEAT (2u + 1) 
> #define CPUID_GUEST_NR_TOPO (1u + 1) 
> #define CPUID_GUEST_NR_XSTATE (62u + 1) 
> #define CPUID_GUEST_NR_EXTD_INTEL (0x8u + 1) 
> #define CPUID_GUEST_NR_EXTD_AMD (0x21u + 1) 
> 
> Please can someone explain to me why we have these constants or point to a documentation which explains it? 
> I am particularly interested in the CPUID_GUEST_NR_BASIC given that for intel processors for example, we have 
> basic leaves running up to 0x21u already for recent processors. This value sort of forces a particular max leaf value. 

This is related to the maximum leaves supported in the cpu_policy
structure:

https://elixir.bootlin.com/xen/latest/source/xen/include/xen/lib/x86/cpu-policy.h#L122

For basic leaves (0x000000xx) we support up to leaf 0xd (XSTATE).  It
doesn't mean there are no further leaves behind it, but we likely
still have no use for them, and hence they are not part of the policy.
The cpu-policy is used to store a (cpuid) policy object for guests,
so if the information exposed in those leaves are related to features
that are never exposed to guests is makes no sense to have space for
them.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:46:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:46:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717327.1119422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tz2-0006cW-Ge; Mon, 06 May 2024 08:46:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717327.1119422; Mon, 06 May 2024 08:46:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tz2-0006cP-C1; Mon, 06 May 2024 08:46:04 +0000
Received: by outflank-mailman (input) for mailman id 717327;
 Mon, 06 May 2024 08:46:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8Q7K=MJ=inria.fr=fonyuy-asheri.caleb@srs-se1.protection.inumbo.net>)
 id 1s3tz0-0006Kb-Aa
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:46:02 +0000
Received: from mail2-relais-roc.national.inria.fr
 (mail2-relais-roc.national.inria.fr [192.134.164.83])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 101eb759-0b85-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:46:00 +0200 (CEST)
Received: from zcs2-store8.inria.fr ([128.93.142.6])
 by mail2-relais-roc.national.inria.fr with ESMTP; 06 May 2024 10:46:00 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 101eb759-0b85-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=inria.fr; s=dc;
  h=date:from:to:cc:message-id:in-reply-to:references:
   subject:mime-version:content-transfer-encoding;
  bh=3YzvehrpGhuFqIC/wsqReooTMwVJmiiSvD4Ye90gxdU=;
  b=BvBRwtZqgfG4WqgQFe7VuQKN+eRRxRU02FB4qzwgDlb4KDoWFTgZvROw
   Kj5NEs1W7Gavd8B60AP9CEu4rIg174+Du82mtAbYjDurytnynXvSWZ0Dc
   9SpKZjDOa15MgqqASOo0hXikVWLLhfuhyLK+aVvjt8VKu8GnsZToVc7le
   A=;
Authentication-Results: mail2-relais-roc.national.inria.fr; dkim=none (message not signed) header.i=none; spf=Pass smtp.mailfrom=fonyuy-asheri.caleb@inria.fr; spf=None smtp.helo=postmaster@zcs2-store8.inria.fr
Received-SPF: Pass (mail2-relais-roc.national.inria.fr: domain of
  fonyuy-asheri.caleb@inria.fr designates 128.93.142.6 as
  permitted sender) identity=mailfrom; client-ip=128.93.142.6;
  receiver=mail2-relais-roc.national.inria.fr;
  envelope-from="fonyuy-asheri.caleb@inria.fr";
  x-sender="fonyuy-asheri.caleb@inria.fr";
  x-conformance=spf_only; x-record-type="v=spf1";
  x-record-text="v=spf1 include:mailout.safebrands.com
  a:basic-mail.safebrands.com a:basic-mail01.safebrands.com
  a:basic-mail02.safebrands.com ip4:128.93.142.0/24
  ip4:192.134.164.0/24 ip4:128.93.162.160 ip4:89.107.174.7 mx
  ~all"
Received-SPF: None (mail2-relais-roc.national.inria.fr: no sender
  authenticity information available from domain of
  postmaster@zcs2-store8.inria.fr) identity=helo;
  client-ip=128.93.142.6;
  receiver=mail2-relais-roc.national.inria.fr;
  envelope-from="fonyuy-asheri.caleb@inria.fr";
  x-sender="postmaster@zcs2-store8.inria.fr";
  x-conformance=spf_only
X-IronPort-AV: E=Sophos;i="6.07,257,1708383600"; 
   d="scan'208";a="164628874"
X-MGA-submission: =?us-ascii?q?MDEmAluPrGZbeGbY2zZzPkIZW/UO20cBb4070q?=
 =?us-ascii?q?Fp21AzsCMXN87/g9v5E0Lq3Jwa0vpoalWr1hgmnNn84mqmcLrMfBv4cS?=
 =?us-ascii?q?VJ6vJtPALmqPKlxbpWg1r2FQjtDeYGOreW/eDkXjN1QpYu8HE9/gkSBP?=
 =?us-ascii?q?kaByGhP6jr6mqzANm5ILR5nw=3D=3D?=
Date: Mon, 6 May 2024 10:45:59 +0200 (CEST)
From: Fonyuy-Asheri Caleb <fonyuy-asheri.caleb@inria.fr>
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>
Message-ID: <1049993614.4894172.1714985159910.JavaMail.zimbra@inria.fr>
In-Reply-To: <ZjiWDMXFNmwSzAxQ@macbook>
References: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr> <ZjiWDMXFNmwSzAxQ@macbook>
Subject: Re: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID
 constants
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Originating-IP: [131.254.23.3]
X-Mailer: Zimbra 10.0.8_GA_4611 (ZimbraWebClient - GC124 (Linux)/10.0.8_GA_4611)
Thread-Topic: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID constants
Thread-Index: H7fOWAst3uymhR9+0n74Xxv6FcRcAQ==

> From: "Roger Pau Monn=E9" <roger.pau@citrix.com>
> To: "Fonyuy-Asheri Caleb" <fonyuy-asheri.caleb@inria.fr>
> Cc: "xen-devel" <xen-devel@lists.xenproject.org>
> Sent: Monday, May 6, 2024 10:34:20 AM
> Subject: Re: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID =
constants

> On Mon, May 06, 2024 at 09:46:58AM +0200, Fonyuy-Asheri Caleb wrote:
>> Hi,
>> I am currently doing a study on the way xen handles CPUID information.
>>=20
>> I came across these constants in the code (xen/include/xen/lib/x86/cpu-p=
olicy.h
>> file) but no explanation of why they have been set that way.
>>=20
>> #define CPUID_GUEST_NR_BASIC (0xdu + 1)
>> #define CPUID_GUEST_NR_CACHE (5u + 1)
>> #define CPUID_GUEST_NR_FEAT (2u + 1)
>> #define CPUID_GUEST_NR_TOPO (1u + 1)
>> #define CPUID_GUEST_NR_XSTATE (62u + 1)
>> #define CPUID_GUEST_NR_EXTD_INTEL (0x8u + 1)
>> #define CPUID_GUEST_NR_EXTD_AMD (0x21u + 1)
>>=20
>> Please can someone explain to me why we have these constants or point to=
 a
>> documentation which explains it?
>> I am particularly interested in the CPUID_GUEST_NR_BASIC given that for =
intel
>> processors for example, we have
>> basic leaves running up to 0x21u already for recent processors. This val=
ue sort
>> of forces a particular max leaf value.
>=20
> This is related to the maximum leaves supported in the cpu_policy
> structure:
>=20
> https://elixir.bootlin.com/xen/latest/source/xen/include/xen/lib/x86/cpu-=
policy.h#L122
>=20
> For basic leaves (0x000000xx) we support up to leaf 0xd (XSTATE).  It
> doesn't mean there are no further leaves behind it, but we likely
> still have no use for them, and hence they are not part of the policy.
> The cpu-policy is used to store a (cpuid) policy object for guests,
> so if the information exposed in those leaves are related to features
> that are never exposed to guests is makes no sense to have space for
> them.

So if I understand you well, you mean before extending this, we need to per=
form a study on the=20
leaves to confirm how useful they are to the guests depending on the cpu in=
formation they carry.=20

>=20
> Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:46:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:46:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717326.1119411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tym-0006Kt-7y; Mon, 06 May 2024 08:45:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717326.1119411; Mon, 06 May 2024 08:45:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3tym-0006Km-4u; Mon, 06 May 2024 08:45:48 +0000
Received: by outflank-mailman (input) for mailman id 717326;
 Mon, 06 May 2024 08:45:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3tyk-0006Kb-Ji
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:45:46 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 051e6a7f-0b85-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:45:41 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-34da04e44a2so935882f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:45:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v10-20020a5d4a4a000000b0034c462f920csm10117631wrs.40.2024.05.06.01.45.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:45:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 051e6a7f-0b85-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714985141; x=1715589941; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=imwH2Ri8RCLMiz9DnGr+M3SBSVp5vLJ7kgYsVMFNZtw=;
        b=EhVL8DbZ68HptR4QCR8GEOWzVe8Mmscn49Bvw3dW8IYzLHb/yZL7Be/i7fZLoNxo3y
         WqVVtuHV9v0YIlVjbvp+KAfNm3mLu1XF/WsTYB9NwO4tNphrdeWFt8fKFWaS1Eia0Z7p
         3Lm/tjiG5eCf+TwSFEkLh/btT+N35wIv7syqd7lHGWY99PJxGRkO2GY8T1Rdbxe3GWik
         ZEjABMMgLvOVmBxbsyFOcBi/VXimUlTtvwnluDu9CLeeQidBhNsanuNmKhF0DDeW16Cu
         3aIkk9740DnUs8DG8zDHMk45Fe8KQDT5h9xkxqVXFhbgOHsD68czx+BofNzO1jfzxsKc
         qQVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714985141; x=1715589941;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=imwH2Ri8RCLMiz9DnGr+M3SBSVp5vLJ7kgYsVMFNZtw=;
        b=wK0IfrTNvDloVmv/VVUXu/U1nf2ETKBWDcetoVBVBmmt2AvTqXbgJe7UMvVT6xLs5T
         Tz3pveuHlS2ijL8JA4TP+YA1QdYc9gVykTHeLaDQKgKuhTp6C6qQB2M+RblHvylm1Q83
         nJgspnRHB8PpzYKgWqaFm3KRdAx/LDJXktUD4Ex36MqH6c7uVfRQULsu368Npn+5liBx
         e2g8QMqUR0+M9hH4USxDrOYdl9gR3h33dZVo6rX3BAIfxK6rncCqw4BcJwqV7mPvC31N
         zNHhfpUjT88dzKrLLhNHN0qE1GMPLNdgUEYC3F02tM4KMQZE5LvPj37X6sjIXKq7S3eU
         GeWA==
X-Forwarded-Encrypted: i=1; AJvYcCUbv1Zd9GCEO1nXfYXYY7jDa4Mvy0RsMk7V/5KnlPMQUstIKUB27izJwE9dqeuhweYsKATwycDjAv9gArmpjiMYdWU2qwznORIkC8uiKaA=
X-Gm-Message-State: AOJu0YyGY+slL1FNS1tICvB2znY5+UxAew7xaZV6nFXzbOTHOLtOKk68
	0YsBbQ1Amkx8E9Sz+x1S6/P5dLEAIx5UtfIa28AlhsrrFMuERpGtbvmrddl3lA==
X-Google-Smtp-Source: AGHT+IHUe+fOVf+VKr3lWY+T/YsVUlOlfuJ5i0qtmc4YmNDMw4g+YVSuiWGbQTpO1CSLU1+H1DFGIA==
X-Received: by 2002:a5d:6146:0:b0:34c:e409:49d9 with SMTP id y6-20020a5d6146000000b0034ce40949d9mr5658388wrt.68.1714985141378;
        Mon, 06 May 2024 01:45:41 -0700 (PDT)
Message-ID: <1f16e73f-a5a9-4816-8054-81ad0c186030@suse.com>
Date: Mon, 6 May 2024 10:45:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/5] x86: Update x86 low level version check of
 microcode
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-2-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430124709.865183-2-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 14:47, Fouad Hilly wrote:
> Update microcode version check at Intel and AMD Level by:
> Preventing the low level code from sending errors if the microcode
> version is not a newer version. this is required to allow developers to do
> ucode loading testing, including the introduction of Intel "min rev" field,
> which requires an override mechanism for newer version checks.

Won't "min rev" checking, for being Intel-only, require quite the opposite,
i.e. leaving more of the checking to vendor specific code?

> Even though
> the check for newer is removed at this level, it still exists at higher
> common level, where by default only newer version will be processed.
> The option to override version check, will be added as part of this patch
> series.

Please avoid wording like "this patch", "this commit", and all the more
"this patch series". Especially this last one will become completely
meaningless once part of a commit message in the tree.

> --- a/xen/arch/x86/cpu/microcode/amd.c
> +++ b/xen/arch/x86/cpu/microcode/amd.c
> @@ -384,11 +384,10 @@ static struct microcode_patch *cf_check cpu_request_microcode(
>              }
>  
>              /*
> -             * If the new ucode covers current CPU, compare ucodes and store the
> -             * one with higher revision.
> +             * If the microcode covers current CPU, then store its
> +             * revision.
>               */

Nit: This can become a single line comment now, can't it? (Again then in Intel
code.)

> --- a/xen/arch/x86/cpu/microcode/intel.c
> +++ b/xen/arch/x86/cpu/microcode/intel.c
> @@ -294,8 +294,7 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
>  
>      result = microcode_update_match(patch);
>  
> -    if ( result != NEW_UCODE &&
> -         !(opt_ucode_allow_same && result == SAME_UCODE) )
> +    if ( result == MIS_UCODE )
>          return -EINVAL;

I continue to be in trouble with this change, despite the v3 adjustment
you make: If this is needed here, why would a similar change not be needed
for AMD?

Plus the original question remains: Is this actually valid to be changed
right here? IOW despite the somewhat improved patch description I'm still
having difficulty identifying which vendor-independent check this is
redundant with. As opposed to the AMD change further up and ...

> @@ -355,11 +354,10 @@ static struct microcode_patch *cf_check cpu_request_microcode(
>              break;
>  
>          /*
> -         * If the new update covers current CPU, compare updates and store the
> -         * one with higher revision.
> +         * If the microcode covers current CPU, then store its
> +         * revision.
>           */
> -        if ( (microcode_update_match(mc) != MIS_UCODE) &&
> -             (!saved || compare_revisions(saved->rev, mc->rev) == NEW_UCODE) )
> +        if ( (microcode_update_match(mc) != MIS_UCODE) && !saved )
>              saved = mc;

... this one, where I can see that they are about caching of ucode blobs,
which looks to be dealt with by cache maintenance logic in
microcode_update_helper() and microcode_update_cache().

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:47:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:47:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717333.1119431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3u0c-0007MI-QV; Mon, 06 May 2024 08:47:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717333.1119431; Mon, 06 May 2024 08:47:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3u0c-0007MB-Nj; Mon, 06 May 2024 08:47:42 +0000
Received: by outflank-mailman (input) for mailman id 717333;
 Mon, 06 May 2024 08:47:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3u0b-0007M3-K9
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:47:41 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4b34c599-0b85-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 10:47:39 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41b79450f78so11132855e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 01:47:39 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 b17-20020a05600c4e1100b0041be3383a2fsm19105345wmq.19.2024.05.06.01.47.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 01:47:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b34c599-0b85-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714985259; x=1715590059; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SM18yENDR0bUqzBoEPLR+Iws2RxetPS+i/++V2FfZRM=;
        b=NVnwkXK4RwsEN2alkgffIbe2jE5qefG4dz4n8W+8arsE/yeF5y4oI7AcgzvS0RQerk
         CCSR0OIr42jnawyqH5lNBKwPcpeb2VGKhRdr3VsgOqmqRGvvmF6m9142ZyDogEZa/XgT
         P/OGURiaBTb8EirS8ab9AHuwa8SLgUWKOG+h0uGmP5FMqH4XUhZcbRW2m2ynm5UJkXfm
         RLXZPLQSJ76daa5lvVvvTxJW0flLHcDg82JQW6AbiembdJ+Hja82yE+8SFIfkkxNxhq3
         C9dvql52KGpyBMGmE0X6rI2PsIV2AWszm0ryvJdQOcyTbXjQHMNFgeQb39+YF53d5VVi
         wKQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714985259; x=1715590059;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SM18yENDR0bUqzBoEPLR+Iws2RxetPS+i/++V2FfZRM=;
        b=Y7IdBzdI/3ygInmljCyUCk9P4TgBQ1+DNtDxYvfw/kQUT6oo5FywDBl5cOp+uET20s
         pHi8WsG7IMzLk6mt8vTOOhfZV2dyAZ3cfPOXDCGidyqdsYGeLPEra5vTac7iuFoB0xhD
         Tohu3VWVXt/XSepqRYnCVSoPzYw+c5MaOsfMNpBTyDQx9zH7jHZAbkrTsGbc2CcpxWqi
         1NMlfG/iT1G6Vbd1ormsT9rRo0SoOcpMJgVDWN9fuI8Xv3ZXhnBDs0NTdgJgXAHIGPuA
         P0PtS1rCo3OsPa1+LXQHov3xfxWjD31P0cp49ShtmcHXT943+nf/mIbad13ydDLjFajI
         /b3A==
X-Gm-Message-State: AOJu0Yyc+gqBU9mVVJxUoI3Nvv2jPC4LMUM7IzsoLn5l6h+NUWZv7Wp2
	8GpVf32wiCrDBf2kBbpFE0XYL8C42s2PGIv97TdgBIWIj4RSV2XyvdraA1aMwA==
X-Google-Smtp-Source: AGHT+IEAi8zI3a6GLf0OxXK+sowmFP5nqM6KmKxOfNk/Zenq1iOoyO7iLmPjApCEzx8ywfBHGzgt9g==
X-Received: by 2002:a05:600c:3109:b0:41b:9e4f:d2b2 with SMTP id g9-20020a05600c310900b0041b9e4fd2b2mr9924623wmo.2.1714985259029;
        Mon, 06 May 2024 01:47:39 -0700 (PDT)
Message-ID: <b892301a-0f45-4bc3-a117-b96672cc36a6@suse.com>
Date: Mon, 6 May 2024 10:47:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: file xen/include/xen/lib/x86/cpu-policy.h: Meaning of CPUID
 constants
Content-Language: en-US
To: Fonyuy-Asheri Caleb <fonyuy-asheri.caleb@inria.fr>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <2066842119.4829764.1714981618946.JavaMail.zimbra@inria.fr>
 <ZjiWDMXFNmwSzAxQ@macbook>
 <1049993614.4894172.1714985159910.JavaMail.zimbra@inria.fr>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1049993614.4894172.1714985159910.JavaMail.zimbra@inria.fr>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 10:45, Fonyuy-Asheri Caleb wrote:
>> From: "Roger Pau Monné" <roger.pau@citrix.com>
>> Sent: Monday, May 6, 2024 10:34:20 AM
> 
>> For basic leaves (0x000000xx) we support up to leaf 0xd (XSTATE).  It
>> doesn't mean there are no further leaves behind it, but we likely
>> still have no use for them, and hence they are not part of the policy.
>> The cpu-policy is used to store a (cpuid) policy object for guests,
>> so if the information exposed in those leaves are related to features
>> that are never exposed to guests is makes no sense to have space for
>> them.
> 
> So if I understand you well, you mean before extending this, we need to perform a study on the 
> leaves to confirm how useful they are to the guests depending on the cpu information they carry. 

It's not only question of usefulness, but of correctness. We can't blindly
expose leaves.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 08:52:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 08:52:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717335.1119441 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3u5P-0000P7-Bk; Mon, 06 May 2024 08:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717335.1119441; Mon, 06 May 2024 08:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3u5P-0000P0-9D; Mon, 06 May 2024 08:52:39 +0000
Received: by outflank-mailman (input) for mailman id 717335;
 Mon, 06 May 2024 08:52:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VNYQ=MJ=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s3u5O-0000Ou-7m
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 08:52:38 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc89a03c-0b85-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 10:52:37 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 24D244EE074D;
 Mon,  6 May 2024 10:52:35 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc89a03c-0b85-11ef-909c-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	andrew.cooper3@citrix.com,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH] automation/eclair_analysis: unblock pipelines from certain repositories
Date: Mon,  6 May 2024 10:52:31 +0200
Message-Id: <b599441c1678d11cfaa43b679bacbd5a1b886d3f.1714983088.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Repositories under people/* only execute the analyze step if manually
triggered, but in order to avoid blocking the rest of the pipeline
if such step is not run, allow it to fail.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
See https://gitlab.com/xen-project/people/bugseng/xen/-/pipelines/1278550868
for a sample pipeline with this change applied.
---
 automation/gitlab-ci/analyze.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
index 32bf57014961..02e0ea692c66 100644
--- a/automation/gitlab-ci/analyze.yaml
+++ b/automation/gitlab-ci/analyze.yaml
@@ -31,6 +31,7 @@
       when: never
     - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
       when: manual
+      allow_failure: true
     - !reference [.eclair-analysis, rules]
 
 eclair-x86_64:
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:06:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:06:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717349.1119452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uIU-0003mE-Lc; Mon, 06 May 2024 09:06:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717349.1119452; Mon, 06 May 2024 09:06:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uIU-0003m7-HV; Mon, 06 May 2024 09:06:10 +0000
Received: by outflank-mailman (input) for mailman id 717349;
 Mon, 06 May 2024 09:06:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3uIT-0003m1-54
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:06:09 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df5267d8-0b87-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:06:06 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51fc01b6fe7so1332816e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:06:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ay2-20020a05600c1e0200b00418948a5eb0sm19179739wmb.32.2024.05.06.02.06.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:06:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df5267d8-0b87-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714986366; x=1715591166; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UNbmiClhu3TCiYvFLpf5IzYM43ZV+ft5M+6hFSbisss=;
        b=Vfzdaq2W8O5q/tm8aeJz7E64bd2SkUoj7racLp88BTau4xRtl3l36kDsFcjtPkFbXL
         mzC4A1w0ifSjF7Ybn2WrsNm+rfeQvANfIcBRnT+Hc7i8FIhI3dUVb3hd7qpK+iN5DB59
         5dKDnbYma/V9ciq1Mj06BM2063F/pXLAPe0tBD9bjKu4RoHAdPZnqpMOLZT7VFzrme53
         CaTwvRPgLsOxx6JrEXvZsIga/dH6RrdSpwe+22ota0P6iObfFMDgJEDUWcPawrhuhFjp
         w+6wLj42GzJO8SUIDu7ok5B/c6aXNPophocJqMdp38gkd+DclpqyPTXqkpnUkJjOdmG+
         nRzw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714986366; x=1715591166;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UNbmiClhu3TCiYvFLpf5IzYM43ZV+ft5M+6hFSbisss=;
        b=rrJdBCzZDxE+5g5h4D+tEfl/ihA9rAEGWmqbd4y4bMkZM6GcNIkinN3jNAa08w+jXI
         hY3CNIn2kxh6zbPzyPGDsg1T1/9KX+4nPx0+kZKxBFgUjEoJ/ONRKONuvBTXSj+Jtf2n
         GtiKU5Zfpk8ioD9BNisUexOpL4RA4xMhRTyPl5Rg3c1+H9ebowU0zeu9jWNg+240Su4x
         UReWqe7NPUvBHa2KJdCH/NV6N1yppSGE3BPw6lQy8w64P6EwXhI2lhutqIIim01OhO4A
         8zkrhjQimnfphACmZ0pPb3Z+/cETXNZCzUIHRtssBonLxRD1dpsqB02MgKDJiSw0ljP6
         SYiA==
X-Gm-Message-State: AOJu0Yw4ih1/jDSXrzf1+wFU22U+clIAHCiMugpK2RjdarA+hyYovPmY
	ggpy9CdyqJgLTUggM41SEEh1i4RIE8/sLQMwCWPkkrGDJrGwEjgSTjUxo3FXlbAoebyd5dZPkRI
	=
X-Google-Smtp-Source: AGHT+IHLqLekucEteWtsCFTcLedzrG+NyRzUN+rSt2f7qJXgWmjtThA5tMJzebd7PqqaUqHPt0bGOQ==
X-Received: by 2002:a2e:888b:0:b0:2e0:1414:5eb9 with SMTP id k11-20020a2e888b000000b002e014145eb9mr6578281lji.25.1714986366316;
        Mon, 06 May 2024 02:06:06 -0700 (PDT)
Message-ID: <92e5df23-0bdf-42d4-9ab0-c668110174f0@suse.com>
Date: Mon, 6 May 2024 11:06:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/platform: correct #undef in compat checking
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

A stray 'p' was there, rendering the #undef ineffectual.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/x86_64/platform_hypercall.c
+++ b/xen/arch/x86/x86_64/platform_hypercall.c
@@ -30,7 +30,7 @@ CHECK_pf_pcpu_version;
 
 #define xen_pf_ucode_revision xenpf_ucode_revision
 CHECK_pf_ucode_revision;
-#undef xen_pf_pucode_revision
+#undef xen_pf_ucode_revision
 
 #define xen_pf_enter_acpi_sleep xenpf_enter_acpi_sleep
 CHECK_pf_enter_acpi_sleep;


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:12:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:12:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717352.1119460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uOx-0005Ev-9R; Mon, 06 May 2024 09:12:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717352.1119460; Mon, 06 May 2024 09:12:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uOx-0005Eo-6k; Mon, 06 May 2024 09:12:51 +0000
Received: by outflank-mailman (input) for mailman id 717352;
 Mon, 06 May 2024 09:12:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3uOv-0005Ei-1s
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:12:49 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce568b2b-0b88-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:12:47 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e3737697d9so7516391fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:12:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 l8-20020a05600c4f0800b0041bfa349cadsm18990477wmq.16.2024.05.06.02.12.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 02:12:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce568b2b-0b88-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714986767; x=1715591567; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=AEh57JYKqowRxni3RBYf/sUdlKCxJ/2VHnMkJSMDgvs=;
        b=X4FSzF8j6FIco650i8r/VZLKb2AIc+GjFVYCcU0el3ISqocbwngNc4MYPJEZU45Oqg
         2ipUHk2A9ECxjbQ9jnd6SbHDT3IjRWLQkhiMJ9YDApjQasAoJVo8dIN6ciiWUPXA0HnN
         p7fbevLE+AxRaVSd6MN+/MIJYEU+5EwLKeREc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714986767; x=1715591567;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=AEh57JYKqowRxni3RBYf/sUdlKCxJ/2VHnMkJSMDgvs=;
        b=CAZ5ISfGK9DT81OvK0wlj0LgTXOuzuGFDSZFNFhYGUaF+McvtAbePJTHV5sbZaPcDD
         I//schaXgQOMPWL5Aejgq91jVA7COUVZDfOU0cRbw9bwX/kXYrUoodLPHMdwDT6q0Xnx
         EMS3T6QfxlCYLrouCOuojH0UIZdG8tDUUgxzlfVMUDKUdDyIzaiSDdoBoMzPvD2DsBtU
         XMQNYuXIQKUl+57uLJVBGsZFrLVl/i+uHTJ8WncYx828LJZ360HnjVXbZhYcdtw1qVd0
         WOLv+S6cLLQssG3qDPcNYH4Wc/VoH5A2qdav5XZ67m0BGa1rT/5cwMIwwmGhgx+jv5T/
         NIjw==
X-Gm-Message-State: AOJu0YyoR+XhhiBysr5EGm+V5G8ZLUT6zyIxkId9kSXCjax8seNTXOkb
	EDoRTy7yIsBUH7CKuidqD9/Gr4KLGffrySSgdN+z5xdztl302nUMeSDhM1ImhS8=
X-Google-Smtp-Source: AGHT+IHPBH82g9x76HH+l4k0wkhCuMH6VRrGNYGujzBlomAUqCiV9LWXKIMRs/z3D8HqLMUeY0sc0Q==
X-Received: by 2002:a2e:7007:0:b0:2de:883d:1aef with SMTP id l7-20020a2e7007000000b002de883d1aefmr5785586ljc.46.1714986767235;
        Mon, 06 May 2024 02:12:47 -0700 (PDT)
Date: Mon, 6 May 2024 11:12:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 02/12] VT-d: tidy error handling of RMRR parsing
Message-ID: <ZjifDsIeRseZD27n@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <97577ebf-3871-47be-97c5-8164701be756@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <97577ebf-3871-47be-97c5-8164701be756@suse.com>

On Thu, Feb 15, 2024 at 11:14:02AM +0100, Jan Beulich wrote:
> It's acpi_parse_one_rmrr() where the allocation is coming from (by way
> of invoking acpi_parse_dev_scope()), or in add_one_user_rmrr()'s case
> allocation is even open-coded there, so freeing would better also happen
> there. Care needs to be taken to preserve acpi_parse_one_rmrr()'s
> ultimate return value.
> 
> While fiddling with callers also move scope_devices_free() to .init and
> have it use XFREE() instead of open-coding it.
> 
> In register_one_rmrr() also have the "ignore" path take the main
> function return path.
> 
> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

> ---
> v2: New.
> 
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -82,14 +82,13 @@ static int __init acpi_register_rmrr_uni
>      return 0;
>  }
>  
> -static void scope_devices_free(struct dmar_scope *scope)
> +static void __init scope_devices_free(struct dmar_scope *scope)
>  {
>      if ( !scope )
>          return;
>  
>      scope->devices_cnt = 0;
> -    xfree(scope->devices);
> -    scope->devices = NULL;
> +    XFREE(scope->devices);
>  }
>  
>  static void __init disable_all_dmar_units(void)
> @@ -595,17 +594,13 @@ static int register_one_rmrr(struct acpi

register_one_rmrr() could also be made __init AFAICT? (even before
this patch)

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:15:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:15:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717355.1119471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uR4-00069B-Kj; Mon, 06 May 2024 09:15:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717355.1119471; Mon, 06 May 2024 09:15:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uR4-000694-I6; Mon, 06 May 2024 09:15:02 +0000
Received: by outflank-mailman (input) for mailman id 717355;
 Mon, 06 May 2024 09:15:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3uR3-00068s-8w
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:15:01 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c9d9ecc-0b89-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:14:59 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e2a4c20870so21773691fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:14:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h11-20020a05600c314b00b00418f72d9027sm19069461wmo.18.2024.05.06.02.14.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:14:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c9d9ecc-0b89-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714986899; x=1715591699; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2tyxXSkCHlpN1H8CHNsRUNsWeZzOljFYfMS4j355/3k=;
        b=WH++tKntYCZJOQoJtC9QEjxgFgqESbeq1h52eNgCjTArY0MNe9OjDmzECCg4ngMnAv
         IbXq4kallsIx2gQ0EEE5Ym9fDA96ZHBwJmsGMERHiroFGxHLmzyfRJrSO0yyvshoCRNv
         M3IGcL+2ISYG7lwj7Z3s6YjZ/NrBrVtnOq/MKpXWHAOA5Dt0mpo+iSteA4Y6cM7NPVqS
         0RSAf9w2XES9giAy5N/bgdEFR4u1AG1mZloIysPMITPrVDMswOI6es4A8L9YucmOWACd
         aS+pepp7IfCQbXIq6nBIuofx/P5vkG5ntfXHd8nJ4ZwIacWbBpNawc8mfNKs9D34HxQT
         BK2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714986899; x=1715591699;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2tyxXSkCHlpN1H8CHNsRUNsWeZzOljFYfMS4j355/3k=;
        b=jrELEMPibvaHOwjyQuI0Hg2OXQFx7ggCzkVwgXx/SkrzgKTET6z3MzLCcZ5l72JfkA
         1mTEIiElhmGuDtTB0WE85PsTCp3sUtf+INUSOQFXq7FUMgfJoi2dlyYZ8/VhM2JNLxr4
         5lTMPdTR1FKRpK5Gv0pgIDNf58ax0weos1OeZv32A/CVHuIDgRw35VLzNU7FDTOiFuc6
         W+11ahJUlQdKJtaYFhMEh8Ad0Ym/9jdJc5pAiCEFjQpt1LqCUe/X8ECCnJMH0ebkqtxK
         pWgi+ZLnsIJT440uKZcUU12D2BL8Macxzj+XP+CqPGBhEfO8kVhApXlFvXx3soXfEjfn
         RK4w==
X-Forwarded-Encrypted: i=1; AJvYcCVF4DUI9X1+ojFhWX1vJGacz04sVmVwbIjAZ6ZNmDVIaK9etWhMwsTdWieK9slI7AZiF3wxLaZDobLSodAKfBb+USIUVWBvzF6JAM+OVT8=
X-Gm-Message-State: AOJu0YyI+h+7LuamPhPdFQfF45bnpSlNttFuItGQgCInNswYSDiU/VAr
	ezXBhwAd7pOmSb9hz73nY1X9paRp92df3//TCs/9ZV6N4obKj/PkgHJDDhPiJ9n/1W6OKGfMJFc
	=
X-Google-Smtp-Source: AGHT+IGxAhOnVIZrg5m27XIDi6E9jyltSEdVBtBzncF4zbR0jjOsJLJZvbhbg546JOYuqUkYOVSTTg==
X-Received: by 2002:a05:651c:b13:b0:2df:49b:27cc with SMTP id b19-20020a05651c0b1300b002df049b27ccmr6461992ljr.22.1714986898702;
        Mon, 06 May 2024 02:14:58 -0700 (PDT)
Message-ID: <ec48e32a-30e5-45ab-8f11-7d3d6ce6122e@suse.com>
Date: Mon, 6 May 2024 11:14:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/5] x86: Refactor microcode_update() hypercall with
 flags
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-3-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430124709.865183-3-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 14:47, Fouad Hilly wrote:
> @@ -633,12 +637,12 @@ static long cf_check microcode_update_helper(void *data)
>                                    microcode_cache);
>  
>          if ( result != NEW_UCODE &&
> -             !(opt_ucode_allow_same && result == SAME_UCODE) )
> +             !((opt_ucode_allow_same || ucode_force_flag) && result == SAME_UCODE) )

Why would "force" not also allow a downgrade?

> @@ -708,11 +712,15 @@ static long cf_check microcode_update_helper(void *data)
>      return ret;
>  }
>  
> -int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long len)
> +int microcode_update(XEN_GUEST_HANDLE(const_void) buf,
> +                     unsigned long len, unsigned int flags)
>  {
>      int ret;
>      struct ucode_buf *buffer;
>  
> +    if ( flags > 1 )

How is one to connect this literal 1 with what is really meant here? Also
would be nice if this check fit with other similar checks we do, i.e.

    if ( flags & ~XENPF_UCODE_FLAG_FORCE_SET )

> +        return -EINVAL;
> +
>      if ( len != (uint32_t)len )
>          return -E2BIG;

As an aside: Isn't this dead code, with the respective hypercall interface
struct fields (now) both being uint32_t?

> --- a/xen/arch/x86/platform_hypercall.c
> +++ b/xen/arch/x86/platform_hypercall.c
> @@ -311,7 +311,17 @@ ret_t do_platform_op(
>  
>          guest_from_compat_handle(data, op->u.microcode.data);
>  
> -        ret = microcode_update(data, op->u.microcode.length);
> +        ret = microcode_update(data, op->u.microcode.length, 0);
> +        break;
> +    }
> +
> +    case XENPF_microcode_update2:
> +    {
> +        XEN_GUEST_HANDLE(const_void) data;
> +
> +        guest_from_compat_handle(data, op->u.microcode2.data);
> +
> +        ret = microcode_update(data, op->u.microcode2.length, op->u.microcode2.flags);

Nit (style): Overlong line.

> --- a/xen/include/public/platform.h
> +++ b/xen/include/public/platform.h
> @@ -624,6 +624,19 @@ struct xenpf_ucode_revision {
>  typedef struct xenpf_ucode_revision xenpf_ucode_revision_t;
>  DEFINE_XEN_GUEST_HANDLE(xenpf_ucode_revision_t);
>  
> +/* Hypercall to microcode_update with flags */
> +#define XENPF_microcode_update2    66
> +struct xenpf_microcode_update2 {
> +    /* IN variables. */
> +    uint32_t flags;                   /* Flags to be passed with ucode. */
> +/* Force to skip microcode version check when set */
> +#define XENPF_UCODE_FLAG_FORCE_SET     1

Nit: What is "SET" in the identifier intended to mean?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:21:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:21:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717361.1119481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uWy-00083F-9I; Mon, 06 May 2024 09:21:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717361.1119481; Mon, 06 May 2024 09:21:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uWy-000838-4i; Mon, 06 May 2024 09:21:08 +0000
Received: by outflank-mailman (input) for mailman id 717361;
 Mon, 06 May 2024 09:21:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3uWx-000832-Ca
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:21:07 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f76a09ef-0b89-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:21:06 +0200 (CEST)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-34f0e55787aso718604f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:21:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n15-20020adff08f000000b00343eac2acc4sm10226254wro.111.2024.05.06.02.21.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:21:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f76a09ef-0b89-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714987265; x=1715592065; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=S/zSWmaKj8EIn/E2UAkfYgHX661SSODSWokmoxNghQs=;
        b=KKTnET+IzaNie73vOApLkc6pKvjuUQ80jrW4tUAKWNut1f2aTDIaa5tNGERNrPiaO2
         7nPlqmz7neOXe+aso71f2y8QpTceSvdgq1f7DOAHV6SULOVyBYHWEQuzdshA5P8gYbzK
         IXppxpUR2t92kY/MvDA3dCgNvuuZOeuXDRIyuC9nl8IxaE0UiwXtNz4VJhiFhLUG/RT/
         uJcBwrSaA4WRCA4NqAn+jpBWviM9oDpxX4Fp8oJIW1peumHJLnWFdurNdnkKqUhihX3j
         fYpvFbWu+9nhG09IVXrcbZ34feEuBFYjA1PIYPNSlB/nu9INPKVYh7R4Ii8s2KOAxs1Y
         U/4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714987265; x=1715592065;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=S/zSWmaKj8EIn/E2UAkfYgHX661SSODSWokmoxNghQs=;
        b=LQmsVV/rsUPXmF1ElmmNZ+SGotI2fph1bNKNDYIjrUoKXmR0eGju/jJIPzzt61whFS
         6Lyw3t8kHXU3xJojfSsrBlFg5Zz6mPDgD1HCaj3FQshViMf1QgJl3lY0cQM5pDlYKZNM
         +afEQxPstu+Uoc2E4TxQT7OvgRtcP8/0aXPL+xwWrOG7aYvMg2HbgZL6jNQMABb5jjot
         +Dw0H7f0tvm44wuX+g9ynn3LXMlDor8cs239cTs9rFl1wfmUfjLdPJ32kI00PMHHiQEs
         JCGRzl/wT9KPE+jekqYVeEFklp9CKAI/qbEFU1Z3rKjUelsUgyD/8e4vsjpxN5xg5ZSz
         5x6Q==
X-Gm-Message-State: AOJu0YxE1nwNzwiI05iz07ajYt3Vyt6GMxh9jtvdCbWkltTcXMud703T
	WNNfFuaXC/64hDTck3TTTbShXd/8k4YvnX1k1BJhu0vai49kc5Ygf1F2j0LgUw==
X-Google-Smtp-Source: AGHT+IE0exLuM3sVoNVOz2FWZa24HP9L83VkmE6/B+LMKjt/zP8WH5DEG+WDOAmUICJu2FydEZ7AnA==
X-Received: by 2002:a5d:410e:0:b0:34d:bbcf:11fe with SMTP id l14-20020a5d410e000000b0034dbbcf11femr6159230wrp.62.1714987265641;
        Mon, 06 May 2024 02:21:05 -0700 (PDT)
Message-ID: <77e6c510-b5bd-4cb5-a29c-44bf0cf7a521@suse.com>
Date: Mon, 6 May 2024 11:21:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 02/12] VT-d: tidy error handling of RMRR parsing
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <97577ebf-3871-47be-97c5-8164701be756@suse.com> <ZjifDsIeRseZD27n@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjifDsIeRseZD27n@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 11:12, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:14:02AM +0100, Jan Beulich wrote:
>> It's acpi_parse_one_rmrr() where the allocation is coming from (by way
>> of invoking acpi_parse_dev_scope()), or in add_one_user_rmrr()'s case
>> allocation is even open-coded there, so freeing would better also happen
>> there. Care needs to be taken to preserve acpi_parse_one_rmrr()'s
>> ultimate return value.
>>
>> While fiddling with callers also move scope_devices_free() to .init and
>> have it use XFREE() instead of open-coding it.
>>
>> In register_one_rmrr() also have the "ignore" path take the main
>> function return path.
>>
>> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

>> --- a/xen/drivers/passthrough/vtd/dmar.c
>> +++ b/xen/drivers/passthrough/vtd/dmar.c
>> @@ -82,14 +82,13 @@ static int __init acpi_register_rmrr_uni
>>      return 0;
>>  }
>>  
>> -static void scope_devices_free(struct dmar_scope *scope)
>> +static void __init scope_devices_free(struct dmar_scope *scope)
>>  {
>>      if ( !scope )
>>          return;
>>  
>>      scope->devices_cnt = 0;
>> -    xfree(scope->devices);
>> -    scope->devices = NULL;
>> +    XFREE(scope->devices);
>>  }
>>  
>>  static void __init disable_all_dmar_units(void)
>> @@ -595,17 +594,13 @@ static int register_one_rmrr(struct acpi
> 
> register_one_rmrr() could also be made __init AFAICT? (even before
> this patch)

Indeed, all the more when it calls acpi_register_rmrr_unit(), which is
__init. With scope_devices_free() becoming __init here, it would seem
quite logical to fold that adjustment right into here. I'll do so,
unless you'd indicate that this would then invalidate your R-b.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:21:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:21:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717364.1119491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uX8-0008KM-FE; Mon, 06 May 2024 09:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717364.1119491; Mon, 06 May 2024 09:21:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uX8-0008KF-CF; Mon, 06 May 2024 09:21:18 +0000
Received: by outflank-mailman (input) for mailman id 717364;
 Mon, 06 May 2024 09:21:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3uX7-0008JZ-5r; Mon, 06 May 2024 09:21:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3uX6-00071Z-Ou; Mon, 06 May 2024 09:21:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3uX6-00026w-7K; Mon, 06 May 2024 09:21:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3uX6-0007lP-6Q; Mon, 06 May 2024 09:21:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7OCjHbIbki4SrdgVazdJycmhy+Li9Vu+Mo+ZS5+AiJw=; b=RThoXGmZmRgBy0VPVbjFFbV/MS
	3W8sgv3UeNFGpADiCkqzHuKAl0AAOt2mrPqdNO2jbTvZCbHhZMyHSQ2xn+jjiP3i3jsrlCJ8LZJ4f
	EjkBLacaHx7gvxE4UGmfjpjP9QCHip6m92kADSho5K3Ix54NAIaHwbPKueNsFh6PjEDk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185922-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185922: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
X-Osstest-Versions-That:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 09:21:16 +0000

flight 185922 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185922/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185919
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185919
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185919
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185919
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185919
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185919
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb
baseline version:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb

Last test of basis   185922  2024-05-06 01:55:17 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:22:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:22:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717372.1119502 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uYg-0000i1-WE; Mon, 06 May 2024 09:22:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717372.1119502; Mon, 06 May 2024 09:22:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uYg-0000hu-Rw; Mon, 06 May 2024 09:22:54 +0000
Received: by outflank-mailman (input) for mailman id 717372;
 Mon, 06 May 2024 09:22:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3uYf-0000hm-Ms
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:22:53 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 36b26a17-0b8a-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:22:52 +0200 (CEST)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-34e0d8b737eso2310837f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:22:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 k3-20020adff5c3000000b00349a5b8eba6sm10162622wrp.34.2024.05.06.02.22.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 02:22:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36b26a17-0b8a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714987372; x=1715592172; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=CAbwoPt6z9g2WCw1vc/+Qz8gKP4dT1K/gzliHd+2JA4=;
        b=qxiH5DG2FI5e05fIZS8fOgBi+jbzsVbXTv1cENPkwYFLu5wNWCxb/ZbQmkb3uJz9fy
         anjmcTum0M+f/n7/D/B54tPYfuEo1m/m9oKxqx/bc/5P1i7cuuKgZ+TAiU2FVPgPAFet
         HFeYOfVAP2V2Im0/fZxwbOWUKkmh19xlaBpA0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714987372; x=1715592172;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=CAbwoPt6z9g2WCw1vc/+Qz8gKP4dT1K/gzliHd+2JA4=;
        b=wuw0yiQCq/AXms/qIn0EYY7J/RcKAd4fsb75LyAIa6EP4ZArKwWra56E30+t7Po1Wm
         TUnsvBkMm3YuSn8dFMmTp1CfZok4WkItWxo5HCSno4lZ6boKw9zTVvmycwHEUAqoGiij
         RoOikpsxA/sfxyWb8H9Uw825mpYbGdp6NGFYA6ImPD7jQKeawM2fcVwEIRqbaDiai3ar
         aF/B5k+1iiqH5WMa/+gtVi8fVbCkSS+ch/ME+1S7x0F9JYGpHfEdUEcwhmzz35SeoJ1+
         P/r2a3YqV8bR6JAqQdQVyjkvJ/GAhhqWlOoNsAAyi+2DCKsFLSi20L7WrZh2/ipsY4GH
         oTxQ==
X-Gm-Message-State: AOJu0Yyz1T6jg3wBEnTpH7xSKy4aFFIdJsaJ2w/8OggkT7JJ7/PmREO4
	sMTm3Zy//Izd20huOw+Ly4TKdlc6xZiVI8razjtlPsvyiXWxPcoB1JQqKKnpzHY=
X-Google-Smtp-Source: AGHT+IGGbH+kH+AgnANIy4TqQOu4BUSKZYt1MR07KgA5vAcZ2pV6NrVdC+cY3Xd7aRUEUjJUzH7MtQ==
X-Received: by 2002:a5d:6d82:0:b0:346:47a6:e77e with SMTP id l2-20020a5d6d82000000b0034647a6e77emr13786818wrs.27.1714987371927;
        Mon, 06 May 2024 02:22:51 -0700 (PDT)
Date: Mon, 6 May 2024 11:22:50 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] x86/platform: correct #undef in compat checking
Message-ID: <Zjihaq9OIprv9EfN@macbook>
References: <92e5df23-0bdf-42d4-9ab0-c668110174f0@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <92e5df23-0bdf-42d4-9ab0-c668110174f0@suse.com>

On Mon, May 06, 2024 at 11:06:07AM +0200, Jan Beulich wrote:
> A stray 'p' was there, rendering the #undef ineffectual.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

No Fixes tag?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:26:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:26:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717380.1119520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uc9-0002QL-In; Mon, 06 May 2024 09:26:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717380.1119520; Mon, 06 May 2024 09:26:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uc9-0002QE-GF; Mon, 06 May 2024 09:26:29 +0000
Received: by outflank-mailman (input) for mailman id 717380;
 Mon, 06 May 2024 09:26:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3uc8-0002PT-3e
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:26:28 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b5e6fe77-0b8a-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:26:25 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-34eb52bfca3so1148740f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:26:25 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j3-20020a05600c1c0300b004186f979543sm19267940wms.33.2024.05.06.02.26.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:26:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5e6fe77-0b8a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714987585; x=1715592385; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2f1IgiICCuXTJTyRGcHrtMI1Jc3g5GHfO3/ktl7Cb0U=;
        b=NnfjD6wGO5zenuQbRksLBy+k7Qce93PkFdMQ0dhFZbAtfgdj67xJrPij+eb1UXOQFS
         /nPr6uv7xWfVdHFSkCK36bgym3xhWlpbD8hOkDFiaiqKbn3YzrBhbxuEhErg1SCXWoeg
         KvPXrOgh2i9yuaMgQQDPGNEYNKNBJwmspmiZ6btSI7N4MsQk6jOcgvnqZ16DR9K55RVO
         70iqEdhI6bDA62TRrqQaym/8OnXxM0qLwr41F0+BaHbZ6FsZsgMJ4Sd7bFOn4i52S3u+
         uBG3lMNs969+uTC3jWK0BYgwzY5FbCkGj9cg771LcyjT95cXWbeYHg/Bmv6TUFDktt+j
         Klkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714987585; x=1715592385;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2f1IgiICCuXTJTyRGcHrtMI1Jc3g5GHfO3/ktl7Cb0U=;
        b=Dx5n/7hnJR0ipHc5OJrHaX2sktNvEuyxkEzlVsfL4t9iKVI1z66oErGoeffRsjeYdN
         goG/XKeixQc+r/RRlZErtar1VcUupRi336oppbV1cYpkkwk5eecZi42NdfUypZNI/eBW
         Hi7D14okNR6i9A7WnJaQjX086mks3Bgtg+5WLegFiE4pN69oPZGFCGKZCvJyLjMBTHIH
         EQtvPgJtiUKu6/ha/pk+xVODmcqY2ocn1dRRUn9CShkmEr0wydJVLDL2Mbr00OPK4dv+
         iVhkl+yQ2ohf9eeO44RUHWGxPARaVXYbgiTEMeDDsfKNFZjzsyOwBsuVjBjAHzDTPjvz
         AGfA==
X-Gm-Message-State: AOJu0YyS4vAK/iFf8/ds+9yMj3TLP52SskmKeGDXCHWTu3NgnNPelSXB
	Gi2qPR4BUfCD5FEupLt/dYpCanq0qr0e1mfvGElYLDgQQUsS2aN98vqqgqo41A==
X-Google-Smtp-Source: AGHT+IFm2ui+kxSjbcy0zlfAfFGpR3j9IuhtqqHBe9w7/1HXb1U9Smk20ubMyI3jwiO1gHp6GccaYw==
X-Received: by 2002:a5d:4903:0:b0:34a:beb3:f6dc with SMTP id x3-20020a5d4903000000b0034abeb3f6dcmr9544930wrq.2.1714987585385;
        Mon, 06 May 2024 02:26:25 -0700 (PDT)
Message-ID: <08354db7-92ef-45ee-81a9-7162f770194e@suse.com>
Date: Mon, 6 May 2024 11:26:26 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/platform: correct #undef in compat checking
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <92e5df23-0bdf-42d4-9ab0-c668110174f0@suse.com>
 <Zjihaq9OIprv9EfN@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zjihaq9OIprv9EfN@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 11:22, Roger Pau Monné wrote:
> On Mon, May 06, 2024 at 11:06:07AM +0200, Jan Beulich wrote:
>> A stray 'p' was there, rendering the #undef ineffectual.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

> No Fixes tag?

I didn't think it was worthwhile digging out the offending commit, as
there's no real bug here. Assuming that a colliding identifier might be
introduced elsewhere seems pretty far fetched to me. And Misra, which
might have something to say here, would want us to avoid #undef
altogether, iirc.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:26:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:26:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717379.1119511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uc3-0002AG-CY; Mon, 06 May 2024 09:26:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717379.1119511; Mon, 06 May 2024 09:26:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uc3-0002A9-A0; Mon, 06 May 2024 09:26:23 +0000
Received: by outflank-mailman (input) for mailman id 717379;
 Mon, 06 May 2024 09:26:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3uc1-0002A3-O2
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:26:21 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b2af1702-0b8a-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:26:20 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41bab13ca81so18102685e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:26:20 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 fc20-20020a05600c525400b00418accde252sm15317484wmb.30.2024.05.06.02.26.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 02:26:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2af1702-0b8a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714987580; x=1715592380; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=ypYgY7/acmMjSfvzNvh+PS5/dhdYt9UhirRx9JlwSFo=;
        b=Smh9vHPJWZVF51D0zHmOI8iXDNjYg9/cIpwab+VyYDLpgq/1EUlnKrLhSJP1ZVxNFG
         /vx2ig4PQzaz6AZ29LXVQp0xnpPMK9czOf8i+2oyeU1e04gfqqBYSjP1cee5sdrGXDB0
         FaOVcq7MequB6i/QqsPUF+fHU7K1h+WO2ZXoo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714987580; x=1715592380;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ypYgY7/acmMjSfvzNvh+PS5/dhdYt9UhirRx9JlwSFo=;
        b=EWcZ9Up+jOAUjJJGhG+1RXh+T9yTiOdUVlAd8GebL/3hUhRyma5zS8jOkGnzY5z2ul
         y3OzvaL5byO33NPFor8sPp+NaihFQvZXhfVI1KvabJg4nKZEsTXi4ZIheJdOhTQZfcT8
         OEStdG93ricv52PIUkpa6g0XVF4TpeM51UqW14UpMDVXY8xHsAQMU15EuzW+oBqnUrja
         D6PIPNi+pIQw9fO2pnvbjOW9jr3eP3G7M0thP/gPhKPAdXXulSWJodUyqEHyfNNMn5+B
         GhOfbCdjxTV60qhxUREbr7bGItOaL7bdmDkC88zEWTaznWV0ggk/sHshlmL074R2smMw
         M9cw==
X-Gm-Message-State: AOJu0YwYTKGcGCcSZF5jgP1f5FxOa94iOAlEYQ0S9Jjn7c5BlDBciQ+O
	wO/XkcKBXJ+5Kf1R1DGIjXBm03eXJRtdMCNZgNd705z6sy24VKLA1tC67YQusGVlqjR14Q39TJW
	J
X-Google-Smtp-Source: AGHT+IGCK+ZiAgLW+S8Af5odW7em2DXVEZNuFT4mNgnIgTJQr2/fBL2FUQZcDQkuF4BJScKcKmEZVw==
X-Received: by 2002:a05:600c:3c9f:b0:418:f6b7:429a with SMTP id bg31-20020a05600c3c9f00b00418f6b7429amr8582990wmb.27.1714987579844;
        Mon, 06 May 2024 02:26:19 -0700 (PDT)
Date: Mon, 6 May 2024 11:26:18 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: xen-devel@lists.xenproject.org, committers@xenproject.org,
	community.manager@xenproject.org, Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Xen 4.19 release status tracking list [ May ]
Message-ID: <ZjiiOtvJq1KjXo5F@macbook>
References: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>

On Fri, May 03, 2024 at 06:54:40PM +0200, Oleksii wrote:
> Hello everyone,
> 
> I would like to share with you a list for status tracking based on Xen
> ML and community members comments:
> 
> *** Arm ***:
>   * Arm cache coloring [
> https://lore.kernel.org/xen-devel/20240502165533.319988-1-carlo.nonato@minervasys.tech/
> ]:
>     - new patch series version [v8] was sent
> 
>   * PCI devices passthrough on Arm, part 3 [
> https://lore.kernel.org/xen-devel/20240202213321.1920347-1-stewart.hildebrand@amd.com/
> ]
>  
>   * DOMCTL-based guest magic region allocation for 11
> domUs [
> https://lore.kernel.org/xen-devel/20240409045357.236802-1-xin.wang2@amd.com/
> ]
>      - new patch series verstion [v4] was sent
>  
>   * [XEN v6 0/3] xen/arm: Add emulation of Debug Data Transfer
> Registers [
> https://patchew.org/Xen/20240307123943.1991755-1-ayan.kumar.halder@amd.com/
> ]
> 
> *** PPC ***:
>   * [PATCH v4 0/6] Early Boot Allocation on Power [
> https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptorengineering.com/
> ]:
>     new patch series version [v4] was sent
> 
> *** RISC-V ***:
>   * [PATCH v8 00/17] Enable build of full Xen for RISC-V
> https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@gmail.com/
> ]:
>     - several patches were merged
>     - new patch series version [v8] were sent
> 
> 
> *** x86 ***:
>   * [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks [
> https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citrix.com/
> ]:
>     - almost patch series have been merged already except the patch:
>         [PATCH 4/4] iommu/x86: make unity range checking more strict
> 
>   * [PATCH 0/8] x86: support AVX10.1 [
> https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@suse.com/
> ]:
>     - two patches of patch series are waitng to merged/reviewed:
>       [PATCH 1/4] amd-vi: fix IVMD memory type checks
>       [PATCH 4/4] iommu/x86: make unity range checking more strict 
>   
>   * APX support?
> 
>   * [PATCH v4 0/8] x86emul: misc additions [
> https://lore.kernel.org/xen-devel/9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com/
> ]
>    
>   * [PATCH v2 00/12] VT-d: SATC handling; ATS: tidying[
> https://lore.kernel.org/xen-devel/64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com/
> ]:
>     new version was sent [ v2 ]
> 
>   * [XEN PATCH 0/9] x86: parallelize AP bring-up during boot [
> https://lore.kernel.org/xen-devel/cover.1699982111.git.krystian.hebel@3mdeb.com/
> ]
> 
>   * [PATCH v2 00/12] x86: memcpy() / memset() (non-)ERMS flavors plus
> fallout [
> https://lore.kernel.org/xen-devel/8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com/
> ]
>     - 6/12 are merged.
>  
>   * [PATCH v6 0/4] x86/pvh: Support relocating dom0 kernel [
> https://patchew.org/Xen/20240327215102.136001-1-jason.andryuk@amd.com/
> ]

Isn't this done? (at least the Xen side):

dfc9fab00378 x86/PVH: Support relocatable dom0 kernels

I have some minor stuff that was posted one day after feature
freeze:

xen/x86: support foreign mappings for HVM
https://lore.kernel.org/xen-devel/20240430165845.81696-1-roger.pau@citrix.com/

and

tools/xen-cpuid: switch to use cpu-policy defined names
https://lore.kernel.org/xen-devel/20240502114922.94288-1-roger.pau@citrix.com/

Which I think would be nice to have for the release.

The first one because it enables work on PVH-based QEMU stubdomains,
the second it's a nice cleanup that makes it easier to introduce new
CPUID features.

Some series that I posted before the freeze:

livepatch: minor bug fixes and improvements
https://lore.kernel.org/xen-devel/20240424081957.34326-1-roger.pau@citrix.com/
Patches 3 and 4 are pending review.

xen: introduce Kconfig function alignment option
https://lore.kernel.org/xen-devel/20240207145547.89689-1-roger.pau@citrix.com/
Patch 2 is missing review.

And then some patches that I don't expect to make progress:

x86/shutdown: change default reboot method preference
https://lore.kernel.org/xen-devel/20230915074347.94712-1-roger.pau@citrix.com/

x86/time: prefer CMOS over EFI_GET_TIME
https://lore.kernel.org/xen-devel/20240315114242.33309-1-roger.pau@citrix.com/

I find it quite funny how everyone complains about all downstreams
carrying similar patches to deal with firmware bugs, yet when I
attempt to formally post them I get zero interest from such
downstreams.  Maybe raising them here will help get some eyes.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:26:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:26:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717384.1119532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ucZ-000377-Sj; Mon, 06 May 2024 09:26:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717384.1119532; Mon, 06 May 2024 09:26:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ucZ-000370-Nj; Mon, 06 May 2024 09:26:55 +0000
Received: by outflank-mailman (input) for mailman id 717384;
 Mon, 06 May 2024 09:26:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3ucY-0002PT-Fl
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:26:54 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5f8b57d-0b8a-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:26:52 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41adf155cffso10873355e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:26:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 j3-20020a05600c1c0300b004186f979543sm19269133wms.33.2024.05.06.02.26.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 02:26:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5f8b57d-0b8a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714987612; x=1715592412; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Wy4+P0aSKp9sb0zjzlVOE4/kxyRE9YvkU63k0Vlw5HU=;
        b=TAFm1C5dRlWvKaAMtMA5ekJPldWu0q73GpIK/M1xiBPxNiwqrT3cAJzQlne04hfVdC
         UGRdSxpNQWwzSL/TAtJRjXG1MvaBaHULT7JHLfcV1eoe4Z4ZeG2SR30KnPwu4+ahU8JJ
         MmRcB9XTtVHKWLhkpXj5pdHJEzpA3ivdJPWLM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714987612; x=1715592412;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Wy4+P0aSKp9sb0zjzlVOE4/kxyRE9YvkU63k0Vlw5HU=;
        b=Go7iBC4T7ZNr9OJ7+idJ5zXfhvuz1mhu9J8JHg6kvGec/jbzq85JwjtfHvGszSU4m4
         JtcEd6CL578vsF1iw7tTgtwVmwzxsTSUdNW4intIxcUI8KCQBXw0yO2/26yIAQg/afLu
         m88qoYNPqBfvnK11kK3DdW9qoSBh1MlS8d6WgosiqOxPOh0zP0kcpSOW19DE7L13aQiX
         1oTmOQ98CJR+VGyJBF/Z9lmrotLYYuLCWosnIY6cHsVPa/qt/3cGDAREaoscva2EL5B6
         t+uR9iHpQ7ogbp50eykcErBurO7yNinYZs1+KqLIaVzmBDwS53ZzK00U6J66NuKSDz7M
         UhHQ==
X-Gm-Message-State: AOJu0Yzo9h1q66xG+ZjtZl0BUqXYYWv2+4cgIH6rWHgSvJaubL55kkN3
	B7kLme1Dn9+Ll/2SNs703unVL0HkRoeu3q0IgruAySOpag8Du6Oxb/WdoBGLrPg=
X-Google-Smtp-Source: AGHT+IHtyKfYDz+jLI++pGxzapbKNhU18SZK1bty5s2bfrMu/GM+X/F053AkW8Ux0q9JwymolbOncw==
X-Received: by 2002:a05:600c:3554:b0:41b:6dbb:52f9 with SMTP id i20-20020a05600c355400b0041b6dbb52f9mr7523731wmq.39.1714987612384;
        Mon, 06 May 2024 02:26:52 -0700 (PDT)
Date: Mon, 6 May 2024 11:26:51 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 02/12] VT-d: tidy error handling of RMRR parsing
Message-ID: <ZjiiW7ZeZCIiMiFP@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <97577ebf-3871-47be-97c5-8164701be756@suse.com>
 <ZjifDsIeRseZD27n@macbook>
 <77e6c510-b5bd-4cb5-a29c-44bf0cf7a521@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <77e6c510-b5bd-4cb5-a29c-44bf0cf7a521@suse.com>

On Mon, May 06, 2024 at 11:21:06AM +0200, Jan Beulich wrote:
> On 06.05.2024 11:12, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 11:14:02AM +0100, Jan Beulich wrote:
> >> It's acpi_parse_one_rmrr() where the allocation is coming from (by way
> >> of invoking acpi_parse_dev_scope()), or in add_one_user_rmrr()'s case
> >> allocation is even open-coded there, so freeing would better also happen
> >> there. Care needs to be taken to preserve acpi_parse_one_rmrr()'s
> >> ultimate return value.
> >>
> >> While fiddling with callers also move scope_devices_free() to .init and
> >> have it use XFREE() instead of open-coding it.
> >>
> >> In register_one_rmrr() also have the "ignore" path take the main
> >> function return path.
> >>
> >> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.
> 
> >> --- a/xen/drivers/passthrough/vtd/dmar.c
> >> +++ b/xen/drivers/passthrough/vtd/dmar.c
> >> @@ -82,14 +82,13 @@ static int __init acpi_register_rmrr_uni
> >>      return 0;
> >>  }
> >>  
> >> -static void scope_devices_free(struct dmar_scope *scope)
> >> +static void __init scope_devices_free(struct dmar_scope *scope)
> >>  {
> >>      if ( !scope )
> >>          return;
> >>  
> >>      scope->devices_cnt = 0;
> >> -    xfree(scope->devices);
> >> -    scope->devices = NULL;
> >> +    XFREE(scope->devices);
> >>  }
> >>  
> >>  static void __init disable_all_dmar_units(void)
> >> @@ -595,17 +594,13 @@ static int register_one_rmrr(struct acpi
> > 
> > register_one_rmrr() could also be made __init AFAICT? (even before
> > this patch)
> 
> Indeed, all the more when it calls acpi_register_rmrr_unit(), which is
> __init. With scope_devices_free() becoming __init here, it would seem
> quite logical to fold that adjustment right into here. I'll do so,
> unless you'd indicate that this would then invalidate your R-b.

Sure, feel free to fold here.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:35:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:35:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717393.1119541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ukR-00059x-Hu; Mon, 06 May 2024 09:35:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717393.1119541; Mon, 06 May 2024 09:35:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ukR-00059q-Ej; Mon, 06 May 2024 09:35:03 +0000
Received: by outflank-mailman (input) for mailman id 717393;
 Mon, 06 May 2024 09:35:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3ukQ-00059k-II
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:35:02 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e8a135bd-0b8b-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:35:00 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41e82b78387so10906995e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:35:00 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g18-20020a5d5412000000b00346bda84bf9sm10261442wrv.78.2024.05.06.02.34.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:34:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8a135bd-0b8b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714988100; x=1715592900; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=amZL2ATlg3xNvQ3/IvNYNfGTONfI++zlDrkaHJkdsvw=;
        b=UYVcGxkHhN8Wlzb73fjpnfnLVDSXcJS2VS/3QxvKBv+cnCTGUiIlAcvHX1NINxvoAE
         Ok5tVnFMMB4KQjxWHwnAibixEn79QH9MOgsNS95MhBVEUoz58ZpwVyBwqERKiUQkqKYW
         c+3jzjrB7ioVbjiPjmRFuwokrzClyG7x7cFEDrz/+rKW5Yxl9AVjfNcYhOxmCTkNcUp/
         hvVGTdHPqCAFhdSHNPAjKW/Ot3IgoVAUcMkMaGoCeudxX/m9t4eWCjIM7Gi5va8ZUey+
         k1trymykJdsOUIljThtSrM6EOk7e7jHADNEWKu4d8N7wX2G4ZJzc33Tz/sKB8+SjAPPt
         EMuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714988100; x=1715592900;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=amZL2ATlg3xNvQ3/IvNYNfGTONfI++zlDrkaHJkdsvw=;
        b=a8UUKa0ozO2ck2lriUe9YYC/8J0ORlWfmnH+2YFWil1zCmeMrHAAns24SYTBZQTbcg
         oxD3NEmVB33cpwvlLDON9o+UYv2Vk3w4SuZFKSEwPTS5BL4H54L8DIG7rIapuGljGsi3
         K5StiVNzjeCWAY+gNsgGdMvMHcmlbh7aXGI92k6Hsa4ofcAPyyKAHxZFm/+FuBTDNyIN
         krjaO+rAhoxab7oXOt75BOlven081+sNX6wOFgiDGSuuM8FnCGxeCRtZpGhyq8EJGfE2
         Fw6JVgI+JhcYVGVG5kokEKhzcPkx99NQL6dyPda1fTve7UZVwVwdKt6Uvg3j3mx+d87N
         SpIQ==
X-Gm-Message-State: AOJu0YxbErs0UWyR/uVOvsO7uAhF7L4HcFvHlLWBZTwPGmoGl25bVB1N
	zSA0MODK4hMW+QDg/DVl1RiIH6Xy70xUS0yPQzlq9xo0dGDjllJPTu1k6FIjBg==
X-Google-Smtp-Source: AGHT+IGYA4wldbHo2Jg6kKK9VlI8Zytyb2g4rclyI1eSbO4rt1dSM5egO0Cuc3907lMg84/exo2TMQ==
X-Received: by 2002:a5d:5960:0:b0:34c:f5d2:528d with SMTP id e32-20020a5d5960000000b0034cf5d2528dmr6802471wri.43.1714988099881;
        Mon, 06 May 2024 02:34:59 -0700 (PDT)
Message-ID: <a8be9d16-36e1-4528-b14c-42746bae605c@suse.com>
Date: Mon, 6 May 2024 11:35:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.19 release status tracking list [ May ]
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, committers@xenproject.org,
 community.manager@xenproject.org, Kelly Choi <kelly.choi@cloud.com>,
 Oleksii <oleksii.kurochko@gmail.com>
References: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
 <ZjiiOtvJq1KjXo5F@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjiiOtvJq1KjXo5F@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 11:26, Roger Pau Monné wrote:
> And then some patches that I don't expect to make progress:
> 
> x86/shutdown: change default reboot method preference
> https://lore.kernel.org/xen-devel/20230915074347.94712-1-roger.pau@citrix.com/
> 
> x86/time: prefer CMOS over EFI_GET_TIME
> https://lore.kernel.org/xen-devel/20240315114242.33309-1-roger.pau@citrix.com/
> 
> I find it quite funny how everyone complains about all downstreams
> carrying similar patches to deal with firmware bugs, yet when I
> attempt to formally post them I get zero interest from such
> downstreams.  Maybe raising them here will help get some eyes.

Well, as indicated on the Community Call I've meanwhile proposed a design
session (which won't really be about design) for Lisbon, as to maintenance
of our EFI interfacing. I think I had provided feedback on the shutdown
patch, and I didn't think there was much of a point repeating almost the
same for the time one.

As a minor remark on what you say above: It's clearly not "all"
downstreams, as we don't carry anything similar to either of the two
patches. But I'm pretty sure you meant "most" or "many" anyway.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:39:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:39:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717398.1119551 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uoT-0006GL-4H; Mon, 06 May 2024 09:39:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717398.1119551; Mon, 06 May 2024 09:39:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3uoT-0006GE-1h; Mon, 06 May 2024 09:39:13 +0000
Received: by outflank-mailman (input) for mailman id 717398;
 Mon, 06 May 2024 09:39:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3uoR-0006G5-N9
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:39:11 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7d953571-0b8c-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:39:10 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e0a34b2899so23722551fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:39:10 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r9-20020a05600c35c900b0041bf5b9fb93sm15477300wmq.5.2024.05.06.02.39.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:39:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d953571-0b8c-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714988350; x=1715593150; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1n2DVMBt5fEB5Tp437HSYwkEpJgwZ2SQY1adK8Ybwdg=;
        b=Rm+JG2JO+hVs5cVH/2Cta/qkBkbjX/WQw5cnAOXobjMrhMIp/wUVOAULTjFDX/zJcy
         lDlUSApUfYVDi2nYpxbE7LoGaCxqCfFR/l4dVxuhWX8r8L5fA/r/fd5dxY1kKvdIPIsM
         UhJb8odqmGAq+sTsijya31w3+oqXSWu+yBFK+w4JlAK5BIFnQsrww7BqJ1TobPiT6Gb0
         /vbuYRkdcL/Qu3M+ALMCNH2I64PYqz9zf5o7oWYd8N6V8EYkuU8gkZ1UYFW7KfdDMSwX
         F9jowK219Tv4HE4tTbGcgTFIk5p3cDuv+csPmTIVXfTqgwyEsB0sCO+itKzawnnVVqTT
         MevA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714988350; x=1715593150;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1n2DVMBt5fEB5Tp437HSYwkEpJgwZ2SQY1adK8Ybwdg=;
        b=INGLgL18M/j+735SiowmosDgLCQic8Gg8tMk5/WGX+paD7QvO/C25yylBEJXJ4fJoF
         E7I7w6Sd4cZWfoBHXDNBSkLM39necL2++q08JSFLX9pfZFSlJIPM5gdgUAEfdgPw1j7g
         DX5QMcTqF+EjyrPADZymvVlWniErNtFNHyYGTD+UzQkoqWpxBf76YqUJYdMQZsN5yl4m
         4mDzzUImOIb/mz8kg8+l11U25g7cotNAFbWKPW6/CYfN7fPcvUua7jnsC/RCclYxYr4N
         lpMoYRYRBM4ze/B0aTpXFMkLsEgW5yFYHAZrJqn46WS34y3Kz96YHlteAZTzmo6+ancW
         OJow==
X-Forwarded-Encrypted: i=1; AJvYcCXYq33IrgXQAyaD0URRa6K142EcLgpt0iAS1j67u+ZDuEZnReleeu8mLZxN1uSL6W2TCZ3YrEH8bW4+iIr3DI25AzvvP+RckoCizkbL5RM=
X-Gm-Message-State: AOJu0YzXp2eYCHayK5Efa4V/DsIxFriig/mKjCJoFePM36lX5JBEPMeZ
	7FwFt5+PMfUpw5mxG2gx/XpKRJPRHr1N21NHHMV8Xv/oaKUECZd0ycUQdd3bKA==
X-Google-Smtp-Source: AGHT+IFTeY5bY37/AXMMNxuSh63QwqRpurC1a2f4LPh4w0eq4TqVcgRj7JxS3ppD/IcB7ovmiQ7Oqw==
X-Received: by 2002:a2e:3a1a:0:b0:2e1:a8ca:6166 with SMTP id h26-20020a2e3a1a000000b002e1a8ca6166mr8272740lja.43.1714988350182;
        Mon, 06 May 2024 02:39:10 -0700 (PDT)
Message-ID: <8b433feb-467c-4569-9426-cc245fc9da50@suse.com>
Date: Mon, 6 May 2024 11:39:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/5] Add --force option to xen-ucode to override
 microcode version check
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-6-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430124709.865183-6-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 14:47, Fouad Hilly wrote:
> @@ -21,10 +23,11 @@ static const char   amd_id[] = "AuthenticAMD";
>  static void usage(const char *name)
>  {
>      printf("%s: Xen microcode updating tool\n"
> -           "Usage: %s [<microcode file> | Options]\n"
> +           "Usage: %s [microcode file] [options]\n"
>             "options:\n"
>             "  -h, --help            display this help and exit\n"
> -           "  -s, --show-cpu-info   show CPU information and exit\n",
> +           "  -s, --show-cpu-info   show CPU information and exit\n"
> +           "  -f, --force           force to skip micorocde version check\n",

Besides (nit) the typo, isn't "version check" too broad here? Elsewhere in the
series you mention Intel's "min_rev" addition. _That_ kind of version check
probably shouldn't be permitted to skip? In which case maybe better "certain
version checking" or some such?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 09:53:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:53:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717404.1119560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v1t-0001II-9Y; Mon, 06 May 2024 09:53:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717404.1119560; Mon, 06 May 2024 09:53:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v1t-0001IB-6g; Mon, 06 May 2024 09:53:05 +0000
Received: by outflank-mailman (input) for mailman id 717404;
 Mon, 06 May 2024 09:53:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v1s-0001I5-5r
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:53:04 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d67731f-0b8e-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:53:02 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f300b318cso2036528e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:53:02 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 s20-20020adfa294000000b0034e285d818dsm10325541wra.32.2024.05.06.02.52.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:53:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d67731f-0b8e-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989181; x=1715593981; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TPCl+61dRvUZ43xraXeNfSbdX7BxsyMvGiFdXcYoCwY=;
        b=pwYfrPLqtrci2e7yfF7JVk4A+Ir0uYFoCKlVwZgsu6F1igJTowc0VJSL+A/9EDBJ4k
         oenf0CZZLUR5/1xh/yXRiFdrnIMXVQmQWQf36bpTWBrUjZ75L3jGA9KKUKsTzlEaqba7
         rIx0rx4x7FV5NpkAaDyUJg7rgn72o4VDn4Gggs0MyRpZ7M3m6ATQm/5GYoez5o/CMxyF
         cfpp4d1foWuF+FCaGs1WrMUxHMT6FERly/6yZ8R9y4aPkvIp+gNXW7Qo65tEA9VsleoX
         syVjeUYcr2owo5nc+7e3767NDbrbQKcAMD4rH+ulbNnBxbSmJruMM9Q6yYmyA7GRh5pa
         UETw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989181; x=1715593981;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TPCl+61dRvUZ43xraXeNfSbdX7BxsyMvGiFdXcYoCwY=;
        b=ajKmLTf0Vkuv1M2SaqRcvwMrIn60hGPw0oinwT+5gzy7IKFY82cdgTbieohCaBPBwO
         A1xugA/nJySdBUgySya/8bxtamd6rEEU1WEXwlHiwWgjLgKzilDxzhsfKLnGwRkf4iBf
         Allnzu55t3FvIr4DMAj5SkcwlhiLag0/9R5ZqshhmXmis0LobZzGLrLwkFqlzX5K2MrY
         0QU8t1v23M6bulJhZvjcVd+o/2Oszf3gQk9qjIVEdfhxqtDCYFOq6zaZzdtL0PECbrjL
         Z6IxII4sQLeycpQme5yt+bFC/N0RJz1vbz0IgKhhYkSYLnSi6zjM2jT+VxZWG0N7INdl
         pqLw==
X-Forwarded-Encrypted: i=1; AJvYcCXASKhMYd5lvGHP9j2dkaJ+EIet4TMPl3dp+HGvZ9u5huQUYp2VgqQKHMtI3Z3ziDWr5B6VbLJLgE+NU3qdvZzJXxZClqHUNg8B/B6jLAo=
X-Gm-Message-State: AOJu0YwlrAoPzi+L1T+wjDeAm+b8VI/zgMOD4H3vCp8DEn/+W1h0mOGM
	uL6REm6/6EBMqvp/1quyBVBo5XJS2AsTzGH4MBpMolr19YYetvsFZBWVamUWiWI=
X-Google-Smtp-Source: AGHT+IE6VEab3Nb48zWWttMEjaLYBNaijuvXZUWASOQIGE7LaA6yDT6c1ODOLkonnCdy4GCZ52lmBg==
X-Received: by 2002:ac2:43bc:0:b0:51c:15fa:b08 with SMTP id t28-20020ac243bc000000b0051c15fa0b08mr5054250lfl.69.1714989181528;
        Mon, 06 May 2024 02:53:01 -0700 (PDT)
Message-ID: <e43695d7-e13c-4bb3-a9ff-0a5739cbeb79@linaro.org>
Date: Mon, 6 May 2024 11:52:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 03/17] xen: mapcache: Refactor lock functions for
 multi-instance
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-4-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-4-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Make the lock functions take MapCache * as argument. This is
> in preparation for supporting multiple caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 34 +++++++++++++++++-----------------
>   1 file changed, 17 insertions(+), 17 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:53:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:53:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717406.1119571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v2b-0001sK-IB; Mon, 06 May 2024 09:53:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717406.1119571; Mon, 06 May 2024 09:53:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v2b-0001sD-FV; Mon, 06 May 2024 09:53:49 +0000
Received: by outflank-mailman (input) for mailman id 717406;
 Mon, 06 May 2024 09:53:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v2Z-0001pw-To
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:53:47 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 875cd8b5-0b8e-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:53:45 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-41ba1ba5592so11550755e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:53:45 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 f19-20020a05600c4e9300b0041bab13cd60sm19278191wmq.3.2024.05.06.02.53.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:53:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 875cd8b5-0b8e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989225; x=1715594025; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=R3MXBECi884L5d0aW1y4r22YZZuFnhM5zOy+PcayxA4=;
        b=Q2L/085Lg08gqjXmr+TjunWgxPXsSmiv7dDNJYJipR1M/eXGOL8EDmidrBfMPaf7JI
         XCpYM8kk0OS9vZu96XUvcXbnTUOZBXo8/KyOBn0J4qs00lZUL4a9XbdIRyMTCHhhQ/9n
         DZ7GFrEgsYGHmXQP2CxxmQ5VioP4Xs2UK40W7wBPB/ej+xwpk+4WxHS+F4PD/an6rOWy
         rS/Y05ezWjyz9ATC80Q4dNZMeO+iAwRKn20ar4CuLC7rjRcHFGOQgvZMtaGBaJpCSJ1a
         mrCSTbOB4Jx6BAUWxpG9uv4CZvZ2ykVjEYOledOGUatITUa8QgmbWYfpxpINRhdUYnOD
         fHhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989225; x=1715594025;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=R3MXBECi884L5d0aW1y4r22YZZuFnhM5zOy+PcayxA4=;
        b=ftMuvtp8z7DEFKRmqIZBHsF/jfrsuriZ/PVJdiezpD2ONtr+gGz5QuI3TLdPfBonJC
         q6AxBqgwUoaCXyRcSA5CyU95UZ8/5X2LERndmq1k5OB79asRHLLGnpG+KmeWCcN8noe7
         woJh96jVMLkjpzKxF0Fd1HskrsVnO+QfgowudLaxU5INFtU+Z080vXlCco+v70ROKQJB
         6tUjsTI0f9//P9t03VTIeRP/hdRxh+CmT2W5Ka5brbjomES+RVKZtkOm07ED+N0GzM8O
         EDoZyiaBRAWlMOXUzwBIoiWDlV4MWyjuIVH6BlpKUskMKq5tVkFdSZrnp8pXzifR/3pL
         G2nQ==
X-Forwarded-Encrypted: i=1; AJvYcCWRETgCHUdkUyRYdQI6VbwD398m2zirTjjrU2Ix9JUeiSAUpTTk3Yi53ql+TYJUhH02F4pedKgeXDXLknhnks0v6ec41Q/ojFbciAMtWec=
X-Gm-Message-State: AOJu0YySs7d8Q/KGUecCh/Ucy1cXlOQlVH2g9opKNkfhAgTE5GMeoFPq
	53vpg0QrufrR/At3TNS7La9RRPVtT2t8dxDquVS+nsEBKKr/3YzIKiA0u+KsGFU=
X-Google-Smtp-Source: AGHT+IF16S4BEzauRTbWJC0iDOTjOLBHJaJTfN9M9AvFa6XZRi9Sy9mQaph2LBneCeJ++5YUspTtfQ==
X-Received: by 2002:a05:600c:3150:b0:419:f241:633b with SMTP id h16-20020a05600c315000b00419f241633bmr9657877wmo.8.1714989225374;
        Mon, 06 May 2024 02:53:45 -0700 (PDT)
Message-ID: <c42978d2-cc54-413c-bdba-3d0adf1cc6ab@linaro.org>
Date: Mon, 6 May 2024 11:53:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 04/17] xen: mapcache: Refactor xen_map_cache for
 multi-instance
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-5-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-5-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Make xen_map_cache take a MapCache as argument. This is in
> prepaparation to support multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 35 ++++++++++++++++++-----------------
>   1 file changed, 18 insertions(+), 17 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:54:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:54:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717409.1119580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v3M-0002hS-QP; Mon, 06 May 2024 09:54:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717409.1119580; Mon, 06 May 2024 09:54:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v3M-0002hL-Nm; Mon, 06 May 2024 09:54:36 +0000
Received: by outflank-mailman (input) for mailman id 717409;
 Mon, 06 May 2024 09:54:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v3K-0002fU-Lt
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:54:34 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a352faf7-0b8e-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 11:54:32 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41b7a26326eso12140975e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:54:32 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 w15-20020a05600c474f00b0041ea90d11edsm5565124wmo.31.2024.05.06.02.54.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:54:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a352faf7-0b8e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989272; x=1715594072; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0OcltWuGCNsBV8ToL0ISlYCII+KDrnOLMgxIJIBcpP0=;
        b=qPYmNMekrovadYmOvMYgWtFHrNfcG5Dmkzl3jBnjcbv0vMFHE4i8x6bNYocyPLweBJ
         6i3OzMldBZhnyxZhW4E+VSOf7n6SiiC6Iu36zF2Yq7ifc+7h6YvTnLLKtw4zLBXMCBFM
         QVKMXnsRGTlrXPfATW9fBAhphd3vYr5p0yiDaApdoqR7tXdFOpA/kTbtlBzL8rZfjZXj
         kGWz4WtsnJGBRFAaswiGfoF3cO9q9/s46ErCe/fxvhdxS2fiu4s4GHw2RoN9ZO549OYK
         fK27UzDysfcmeJfySMAzveB/pRzbfSIJuv1/CYqkIx0zZAOTLDd3svP58uG4dQBLaQOj
         QKNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989272; x=1715594072;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0OcltWuGCNsBV8ToL0ISlYCII+KDrnOLMgxIJIBcpP0=;
        b=qduYCRIPDXlDprfeWyZPk5tC0RSs0kVssWNiBW6FzU4q+khlX0Bpj2mwe7RlggtxNu
         zp0fdpa/CGNpFvu/cMAv/kMDapAoQi/fRXtj0lI8kyFjNwMEqBMs1JhzI9BX3vwxejqe
         1VgxfmHPzpXPZhCV16DCNLzRTvtnHpbL29zAdyVz1eqdBKQJawxVN2cGql3H0G/opbj8
         ia7M7LnpyBDwwbHKMw5lsUMqNNHbtU6e1MvtKpceUWF0JX1B6r95K+aLaBXYAf/L8AJR
         NYPuAJUFDd29xdnh31o1SqYQSpMeMG+9tGe3MjBl3S9DkOpep72lrm5kmiVKFoGtPHXR
         vMTw==
X-Forwarded-Encrypted: i=1; AJvYcCX2KAXtB42yV5u3aiUM7vVQsjUCTJzhPm56xS5D6RaFt2bnz4vCPr7wm7mT0DuiSABhGhwJSm+J/O9Rxbi0TaOt9S/+pM0CFhVCPLfo4rQ=
X-Gm-Message-State: AOJu0YxoCP3Y+gDw1J/caFhBYPQ2TsMEi7VY6+/QkyxtndAkzscSLjIb
	lzFxeTq5GA+7vOPLpcGyBM5iqHKjVMzBoMf30gw9fOSFAg/adKhLGSI2h11MqN8=
X-Google-Smtp-Source: AGHT+IFylmkAkvUEoW5CaLT55WPTEepSZxq4J7hnQLGB3yYct+OA5rehhF3zB38+LrV0wF4/FHmvcw==
X-Received: by 2002:a05:600c:1546:b0:41b:4c6a:fa24 with SMTP id f6-20020a05600c154600b0041b4c6afa24mr7329339wmg.40.1714989272270;
        Mon, 06 May 2024 02:54:32 -0700 (PDT)
Message-ID: <28f9b937-454f-4a99-a362-5e50756e0e1b@linaro.org>
Date: Mon, 6 May 2024 11:54:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 05/17] xen: mapcache: Refactor xen_remap_bucket for
 multi-instance
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-6-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-6-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add MapCache argument to xen_remap_bucket in preparation
> to support multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 9 +++++----
>   1 file changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:55:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:55:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717411.1119590 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v47-0003R3-2q; Mon, 06 May 2024 09:55:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717411.1119590; Mon, 06 May 2024 09:55:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v47-0003Qw-0D; Mon, 06 May 2024 09:55:23 +0000
Received: by outflank-mailman (input) for mailman id 717411;
 Mon, 06 May 2024 09:55:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v45-0003QW-Tg
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:55:21 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c04305b3-0b8e-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:55:21 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51f71e4970bso1937915e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:55:21 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 o3-20020a05600c4fc300b0041aa570bcd3sm19284538wmq.35.2024.05.06.02.55.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:55:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c04305b3-0b8e-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989321; x=1715594121; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yGt1PH+8XRnS4VKauyPJDcQu8WqGev12CrWRiGOjLZk=;
        b=vWsJqoDYzHgfs3BZezlYsCQ3phJV9Es0FbmG6MtxsR00EZEFupbkq/zpzajyENhubJ
         SlAEe9Q1rJpqlPNdMU6ZgPpmCcjAyp4Lrm2YWA2sNyvFaF5km2cXVcsrk2ztbSDbsrTm
         J3ayzJqwrnrC96V0pkcWfh8oqLEqCdHKi08CjkPFA4JXVo8wwdDlnxrRv6RseYXPiEro
         LFFdhmmkOMJ3q+hFpEr9TYDe8T7I0svRBQNmQQ0vMCqVQk63ZRRr+pRIJEuKH6qt/Qj1
         nQrkErdiUhZPAZNz9lYT1UT8mQgPt9rxex8PFEIGvIbDa4ehjgAIzwZiiXyYTpoMsxJx
         AHQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989321; x=1715594121;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yGt1PH+8XRnS4VKauyPJDcQu8WqGev12CrWRiGOjLZk=;
        b=QMQ0fyNxgtbtyoZzJ7OgdcIV8HYDP7du4zVteWpQlqxQRNCI908dMCm/VdhyDCIIBm
         gw3M07YGlREx47O6xO4XYKd28ymR4q8AnaNpKx5yelF3ibgcB99w8cQvx0coRLW2O4n+
         QmQ/NsMCQ0jlmlmHEaA43iUyFI+9kPK2f5hu/ib/ctf3A4+0IgLKWgPKkb6vbuMB3Mpu
         1erDZF62zZY/j5g/wvAgzbqNZHuQa/9+RpB0wL0JINVtOnQAWaV4auxeQrBAntBSXCY+
         q0Jl3ZxwyFZh2xaB9e0DZbEemo2RiHK4Hi7cX237PuuuNsjOmggOjbtvUxbe66eKCeMx
         phJA==
X-Forwarded-Encrypted: i=1; AJvYcCX4D26d+QOKZMCA9gdktcR2KxXQtNxcRd/omEkKiTXsPLAUzAHI+zfNJYEf12GrDzls8PzbSkeIRyAHEth39scTjLnY66FBO1nUcTHFwB0=
X-Gm-Message-State: AOJu0YzGMJlAdIBe44ma6rqz8+S3CeJgJ9jpZrRNs3YBIkw0Wowp9NO3
	kxG4Thh2u3rCu9/9txz9+h622UIa9y0WgiaGbr1jAdXY6cOLGiI/QVI0Zus/wp8=
X-Google-Smtp-Source: AGHT+IHTsGsHu6hSDWz3r2Lzee35mbjMEuzTmmAqHLkTQ0LClXys4F43/xSUizKz37UX1rL2+ksRSg==
X-Received: by 2002:ac2:488d:0:b0:517:870b:a13d with SMTP id x13-20020ac2488d000000b00517870ba13dmr7623294lfc.37.1714989320691;
        Mon, 06 May 2024 02:55:20 -0700 (PDT)
Message-ID: <560f1ca7-02b2-4305-a7fe-5d1ed59b1d6c@linaro.org>
Date: Mon, 6 May 2024 11:55:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 08/17] xen: mapcache: Refactor
 xen_invalidate_map_cache_entry_unlocked
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-9-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-9-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add MapCache argument to xen_invalidate_map_cache_entry_unlocked.
> This is in preparation for supporting multiple map caches.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 21 +++++++++++----------
>   1 file changed, 11 insertions(+), 10 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:56:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:56:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717418.1119600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v5R-0004Kx-EK; Mon, 06 May 2024 09:56:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717418.1119600; Mon, 06 May 2024 09:56:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v5R-0004Kq-BN; Mon, 06 May 2024 09:56:45 +0000
Received: by outflank-mailman (input) for mailman id 717418;
 Mon, 06 May 2024 09:56:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v5Q-0003QW-Hd
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:56:44 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f193b721-0b8e-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:56:44 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-34ddc9fe497so1059881f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:56:44 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0034dcc70929dsm10241487wrw.83.2024.05.06.02.56.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:56:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f193b721-0b8e-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989403; x=1715594203; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=s2OtRlS9dnyJBli43HIli/2qfBfNu58F4VWyD11DfUE=;
        b=Jiw5UHhy3w3WwogIy73wEVaZEVX64RHlgHbUBwdp5UKkIRMK025F325Edv3nAxsOma
         m6kN5ARNOq2r0JleBJXUzRKSopAiUW+Bv8AU8t/cfAbsa9nowfjbJJdPTQAy3hYZXaAQ
         6frX7XdEv7GRwpdxTBc6GydtME/Z+hXhHecxAxPunbRt23CijdpBgvaREhqQ7jcQF5fQ
         xNyuINoYHTY/zneuOsjh+sOKgnu4bnbslhqqzcwSQhaecUi7kZ/fCg/yJQN8LDezd1af
         b1TXOiLuGJnlfcfeBH/ut+jOOiEKmpi6k+L6l0CbMty1ZJsBMbk4hON71kZ+j5VgYOwP
         QrRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989403; x=1715594203;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=s2OtRlS9dnyJBli43HIli/2qfBfNu58F4VWyD11DfUE=;
        b=ZzItR74Zma6yIM4ZFT2IHu6rcb223w8XQ/JYbHoQwNFb6vvB4OO3hugkAPgtSbHbje
         oyzpbhKtsk/grwVQPw3T23p43ePHAS8+XGzxAPYna+0jx02ZYoRMFO65A7Jx6ub4qYxG
         WLIzfUuMwzt9fdqI8MhRTQqNooFo3nW1+fx0yVGYrqfhHPvgdna/8U262aPuwyIQ5Ypb
         fvLWA2KnOF3Dn8UJmZG8cmeugdBEPjqUSMFQUICkoOOTIxqsaLbPFg5zFq4aTH/+PyrB
         3RzMNIww1gGl5c+tOe55W4dkXsBHB8x03AgFnMtA7IoQiHSsamX+qI/MTq47TRwtO5IJ
         nZ/Q==
X-Forwarded-Encrypted: i=1; AJvYcCUXxge3bqvjaUzrV/oAPf7eA8G0bGCB1ZuZTyFhrYnTIjilJMclWS7x49sxwl76xZpV+DsCT3ImlqH+BdlplyVT5uylXlgIShf/G0b1fB0=
X-Gm-Message-State: AOJu0Yxr+DvZJJC4yVI7HLLTdZxdfKpU/uCI5KpS6nLVw0kaYdoD2qaW
	zsavPBIexYYmAXmVcVCVPPwQfUNq3BVhvPEjnNUAEjUtbeWWsrE93C6ftHCsWlU=
X-Google-Smtp-Source: AGHT+IEpGXuliabLS0yW1g8DkYrIw9Px9sWJY4J0BCzBVjcaQz/xa9RWsgkxol5KxbdMskmUDHBnXQ==
X-Received: by 2002:a5d:6dcc:0:b0:34c:f507:84b6 with SMTP id d12-20020a5d6dcc000000b0034cf50784b6mr7858286wrz.41.1714989403409;
        Mon, 06 May 2024 02:56:43 -0700 (PDT)
Message-ID: <47bd7984-39b9-44af-964c-d04620cb6e7e@linaro.org>
Date: Mon, 6 May 2024 11:56:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write
 xen_map_cache()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 David Hildenbrand <david@redhat.com>, xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-14-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-14-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Propagate MR and is_write to xen_map_cache().
> This is in preparation for adding support for grant mappings.
> 
> No functional change.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c         | 10 ++++++----
>   include/sysemu/xen-mapcache.h | 11 +++++++----
>   system/physmem.c              | 31 +++++++++++++++++++++++--------
>   3 files changed, 36 insertions(+), 16 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 09:59:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 09:59:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717421.1119611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v7p-00051b-W5; Mon, 06 May 2024 09:59:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717421.1119611; Mon, 06 May 2024 09:59:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3v7p-00051U-Su; Mon, 06 May 2024 09:59:13 +0000
Received: by outflank-mailman (input) for mailman id 717421;
 Mon, 06 May 2024 09:59:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3v7o-00051N-Hh
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 09:59:12 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 499b99b4-0b8f-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 11:59:11 +0200 (CEST)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-34d7d04808bso1246090f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 02:59:11 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 f3-20020adfb603000000b0034dced2c5bdsm10269498wre.80.2024.05.06.02.59.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 02:59:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 499b99b4-0b8f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714989551; x=1715594351; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RM2AUVZhCrZRBdwl2qu154tGUSj3YdbWUJhDj2uZ1jg=;
        b=ubdxtiJEpjb8LyCDW0EMYCkXdIkX35BfDBa/nXOqQno7PaxiIr9uHFBnzI3+Imvh3S
         Lkf6ZGmFQy1p9rEC/pT8WTiCSnqqhZiSB82yHzWx8Z5d7JgpQlOSBynax6c+KyJrEft1
         ZYkuc89wy2STjb3KvT4j7TU/K7F4AkEBS8Jkj1MS5fIm7Fhq1fWVLvvX6aXa9oD/GZOM
         K/wOyP6aWtdK8FHMK2cjUOcthPDk6eMbmCMmNqLCB29MoFQyUKvf+tKxcjWLtScDFmlR
         YmfG64tcXixI7kcyqDKxg7j1GVg5uM+V8OCDkvso9dIoRFteAvS6cBHSY711y57Ym9QQ
         PY7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714989551; x=1715594351;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RM2AUVZhCrZRBdwl2qu154tGUSj3YdbWUJhDj2uZ1jg=;
        b=EzZy4wSn0tYbblOFsbZuQzoSUB7+WHGm3MiRryIgWpxXr/lOMKZtydyGIWqaifERAf
         pxqbdsJJW2w1nWeTDCSGBW9JKHwdNq6dsgUEXPzcznx4ALXf2jjuy7iS4tZm5IvuRq/h
         kcibQtJz1K/pZqEuf7wuO6zUHJBOgw/AtuyI9TnUn7W97vGt93fCsdr0ooMFkBVBSM94
         Wfw3d0yUbMyj62LamdLxctFoJfmTYfwl/xBTlc3G/YoPCTKD21wT5FxPvaDvYVt2em4f
         y5hV7p5C6F5aRjA8lZBnVEPHSwY83dZBh2vQwttZDL+q1vewdhu7XYkioPGtv2n6CQxm
         6I/w==
X-Forwarded-Encrypted: i=1; AJvYcCVx3u0XuYIPIS+OvBR/vrwD5SGAJE+OawldX2CVqQHGTsoA8ke/F3OnXMSJTJPIHhJjH108Ivzt5Xfb1n3/bAA1WaSjn+sB13G4Lnc7b5s=
X-Gm-Message-State: AOJu0YxwkR4Bk8Z4tpLUJCIiJ0lrtIG5xvVWuLiS2xKZwVr7fpX9MtZq
	gQKmFHBu4RPQ8WwWgJoBeqPF+ZeSA0ZknEVdSCyu4UtvF68cLm+Jlxz0vKTGi0M=
X-Google-Smtp-Source: AGHT+IGQhLat87LYAh2IwfqAZwyzJJcGQTjnVUUxq0f0jcHzkbbN/ooQNq+MPvlN954LLRpUdApp6w==
X-Received: by 2002:a5d:456e:0:b0:34d:8d55:739a with SMTP id a14-20020a5d456e000000b0034d8d55739amr9345620wrc.57.1714989551149;
        Mon, 06 May 2024 02:59:11 -0700 (PDT)
Message-ID: <d4008042-6f68-42f6-8ade-fbb0ba115a33@linaro.org>
Date: Mon, 6 May 2024 11:59:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/17] xen: Add xen_mr_is_memory()
To: David Hildenbrand <david@redhat.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-15-edgar.iglesias@gmail.com>
 <0263b7e8-5800-4f5a-9dc5-bc2b4fbbbbfe@redhat.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <0263b7e8-5800-4f5a-9dc5-bc2b4fbbbbfe@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 2/5/24 09:26, David Hildenbrand wrote:
> On 30.04.24 18:49, Edgar E. Iglesias wrote:
>> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>>
>> Add xen_mr_is_memory() to abstract away tests for the
>> xen_memory MR.
>>
>> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>> ---
> 
> [...]
> 
>>   #endif
>> diff --git a/system/physmem.c b/system/physmem.c
>> index ad7a8c7d95..1a5ffcba2a 100644
>> --- a/system/physmem.c
>> +++ b/system/physmem.c
>> @@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock 
>> *block, ram_addr_t addr,
>>            * because we don't want to map the entire memory in QEMU.
>>            * In that case just map the requested area.
>>            */
>> -        if (block->offset == 0) {
>> +        if (xen_mr_is_memory(block->mr)) {
>>               return xen_map_cache(block->mr, addr, len, lock, lock,
>>                                    is_write);
>>           }
> 
> I'd have moved that into a separate patch, because this is not a simple 
> abstraction here.

Yes please, maybe using Stefano review comment in the description.

> 
> Acked-by: David Hildenbrand <david@redhat.com>
> 



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:07:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:07:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717426.1119621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vFy-0006sR-OO; Mon, 06 May 2024 10:07:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717426.1119621; Mon, 06 May 2024 10:07:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vFy-0006sK-Kc; Mon, 06 May 2024 10:07:38 +0000
Received: by outflank-mailman (input) for mailman id 717426;
 Mon, 06 May 2024 10:07:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3vFw-0006sE-N5
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:07:36 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7473b897-0b90-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:07:33 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41b79451145so12316475e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:07:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cm11-20020a5d5f4b000000b0034dd27adb2fsm10296294wrb.107.2024.05.06.03.07.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 03:07:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7473b897-0b90-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714990052; x=1715594852; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rRfGvW0aOUXUA/OnoM19hA59ktYGB4Ifflpn7fMvZYg=;
        b=HXbbnMYb5RDivnCdxRhFYpxOpQKgl/rF+4kIUU6KVXC4jNe9fiFoSyoy3PHirjzQge
         dgmpVu/hjPnYbBNJB1NBJSXpWPAlqPvOEOm8GLrpyX8WR30Mb6D+S2HPaFPeWaATsq3y
         U8hZ5ev/5KUuAPfMWWu3GtIgRzEmhir5bg5a/5fYSAdYP6wUQ9kK6aeRUzuosCvvzuXE
         ajh9TzfCnRWAXWrdGdeg3xM0JrUyeX1A1HTBFO5PY/MJ/TzCq74Scy6uEsXVADSn5+dU
         cFvoZcUaNisLQ2lY43tLWTl+K6clAm0D0gCRJm1aFkMhmSqu4W4Uo3N5rbSNGZa8jUZB
         o5yA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990052; x=1715594852;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rRfGvW0aOUXUA/OnoM19hA59ktYGB4Ifflpn7fMvZYg=;
        b=vxEjA0KbSDBHD6+DYbEYXm3TlP/0NqhvSDFr6rjvnJ/RPSMniZpaWSxveJ8MZuP7WL
         8ulBEwcoghfImGoY6pkPk9PF3GRWlGiMrfxqFrVJjCkUgE5KoLGxIcNhBImEx91eoBdT
         q4nY/0QyBpmBmlil4+2K5hUZhLJbAAFr5lqitg4plP4R9qnFMJrAHCqfpM4Pk/kklvxV
         VU166yQCWxa+BUd/bfp2SvduwaC+7ZayW6w8BuX15bvxEmc8FAymb+gmN5xAG6vsZGiR
         baqkK74VALOOR7TBHL+T4cDecFdN+e5GkWiMsx1ovYljtiPI9boEjQeqohYbQpqXINom
         kf+A==
X-Forwarded-Encrypted: i=1; AJvYcCV8WcxPKaYmKQ3Ym3ewEg+POdkXQU7JCS5CXxDWydSPsY2Ncw19s8OpxICRqLS1LMvkOI9ukmqro8FHdxUXXzZjBJEcLJcmwrtUV3EysM4=
X-Gm-Message-State: AOJu0YzbIj06I3nm0fusHp4SV+VFOm9/AHC1E4bsoHZdDTfT2FcN6u4e
	4yDQzZuU68GdkzTnJfNBQm3KvUfVxHWoGWxjoDfzWHTtfGKAeB/40LgMvcazZw==
X-Google-Smtp-Source: AGHT+IEf19JWnuxwQOqsLOBnerM5Ugif5QAnX0DK2p5XEXmNF2VgsIQZySwm/PuBqEbkeUw/esv8Rg==
X-Received: by 2002:adf:cc8a:0:b0:34b:147f:df6c with SMTP id p10-20020adfcc8a000000b0034b147fdf6cmr6055812wrj.48.1714990052564;
        Mon, 06 May 2024 03:07:32 -0700 (PDT)
Message-ID: <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>
Date: Mon, 6 May 2024 12:07:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430165845.81696-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 18:58, Roger Pau Monne wrote:
> Keep track of the maximum gfn that has ever been populated into the p2m, and
> also account for the number of foreign mappings.  Such information will be
> needed in order to remove foreign mappings during teardown for HVM guests.

Is "needed" the right term? We could e.g. traverse the P2M tree (didn't look
at patch 2 yet as to how exactly you use these two new fields there), at which
point we might get away without either or both of these extra statistics,
while at the same time also not needing to iterate over a gigantic range of
GFNs. Going from populated page tables would roughly match "max_gfn", with the
benefit of certain removals of P2M entries then also shrinking the upper bound.

> @@ -1049,6 +1057,8 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
>          if ( !page_get_owner_and_reference(mfn_to_page(nfn)) )
>              return -EBUSY;
>  
> +        p2m->nr_foreign++;
> +
>          break;
>  
>      default:
> @@ -1069,6 +1079,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
>              return -EINVAL;
>          }
>          put_page(mfn_to_page(ofn));
> +        p2m->nr_foreign--;
>          break;

Like for the ioreq accounting I'm a little worried of putting this here,
especially with the decrement thus coming ahead of the actual page table
update, but probably I'm overly concerned here. The put_page() living here
would clearly be doing bigger damage if not unconditionally followed by a page
table write. IOW - just a remark, no request for any kind of change.

> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -413,6 +413,8 @@ int p2m_set_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
>          set_rc = p2m->set_entry(p2m, gfn, mfn, order, p2mt, p2ma, -1);
>          if ( set_rc )
>              rc = set_rc;
> +        else
> +            p2m->max_gfn = gfn_max(gfn_add(gfn, 1u << order), p2m->max_gfn);

For one a (new) field named "max_..." wants to record the maximum value, not
one above. And then you want to use 1UL, to match ...

>          gfn = gfn_add(gfn, 1UL << order);
>          if ( !mfn_eq(mfn, INVALID_MFN) )

... surrounding code (more just out of context).

Further I can't really convince myself that doing the update just here is
enough, or whether alternatively the update wouldn't want to be further
constrained to happen just on newly set foreign entries. In that latter
case it would be far easier to reason whether doing the update just here is
sufficient. Plus iirc foreign entries are also necessarily order-0 (else
p2m_entry_modify() wouldn't be correct as is), which would allow to store
just the gfn we have in hands, thus resulting in the field then being
properly named (as to its prefix; it would likely want to become
"max_foreign_gfn" then).

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717434.1119667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNf-0002Rl-V6; Mon, 06 May 2024 10:15:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717434.1119667; Mon, 06 May 2024 10:15:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNf-0002Qn-NY; Mon, 06 May 2024 10:15:35 +0000
Received: by outflank-mailman (input) for mailman id 717434;
 Mon, 06 May 2024 10:15:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNd-0001cU-PX
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:33 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 92a33655-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:33 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59b49162aeso318484966b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:33 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92a33655-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990532; x=1715595332; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HJ4ONEZyJqQciNKYxaBQg/mppiyTg6XZa2ET+ksHj5Y=;
        b=dbZekPFR769FL1tAFtTjP0EW7I19ctISNgUPasRG3A75WbM+k7ud9WOahfQtifQjxy
         atnyXfZqr/WVUsOREhxc72VL0S2jany/TiFlr2lnZAo2BRspJd/Blo2PfQuHsyVtaKCo
         VOHDDoNLRBl+/km6ghRtLRQnCpRMNCP/C4Dt9lwrYeckCifi1xMs8F82VNy9m+P1SfWj
         tpT/4A4JyNO3hQ2oepxPm0ZDKcepxUxfjAEIT3pVFl4yBNNQgSmlI5xfVMDImQB2d1Jo
         afKHMLrTjqwYLkHsLr+MqeniEll7RDsT6MUITDLz/FS5T1yoJ6btWlpTpZmlCK421gB2
         iirA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990532; x=1715595332;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=HJ4ONEZyJqQciNKYxaBQg/mppiyTg6XZa2ET+ksHj5Y=;
        b=XO/f5DrkyJGqEfcIG6CebQfnsvGRoMUDyxjpvla5EGHCgpPPEvg/kCzOlWF06pYE/i
         buRPgMqOc2WtYuZd6pMWzkQiFYMdOBQvL6JmGCa1R+QR/bSaWx+hg1LziJYsyfmuMWlZ
         CSGmMT3vZB9CiljBlz7M6GXkU8YGgC4DVovbPCJ/ktriq6ZVbAQaCQOkoXo37A4ZOOFX
         CCww0Z2xxl2vGo+xkpynOxeJzyDpMRR060Bc6GbIZKUym6ZSXMNBIU8SLfsYVE/bloiC
         tqoCs/TzvXXU/NphkyUBwvaiYb/fIWXVdLsrFy4nnhNmavp7wwrc/dtF8X7xKEzl1lRt
         D+cg==
X-Gm-Message-State: AOJu0YyWrFO7iaPRi4qp706HkemiC6D7MrgA6D+e+FKCB53em4+48RPf
	9G2Nhh3BXrAgVAXc6m2kkpYF6w7dG9cj7qk+5ZD/XcUVrAnxZ00t1VrDYA==
X-Google-Smtp-Source: AGHT+IEyiSX0ZrAYQlbQ+yXOPTKDOcWShyOAiEUWqGNjgpbNhCquuXTWMg5liaftvBSjkrVr5aOkOA==
X-Received: by 2002:a17:907:9493:b0:a59:bacc:b086 with SMTP id dm19-20020a170907949300b00a59baccb086mr3381844ejc.50.1714990532394;
        Mon, 06 May 2024 03:15:32 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Rahul Singh <rahul.singh@arm.com>
Subject: [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
Date: Mon,  6 May 2024 12:15:13 +0200
Message-ID: <4ef575389520479cee0a79fccfc20e6891594d61.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The mentioned macros exist only because of Linux compatible purpose.

The patch defines __ffs() in terms of Xen bitops and it is safe
to define in this way ( as __ffs() - 1 ) as considering that __ffs()
was defined as __builtin_ctzl(x), which has undefined behavior when x=0,
so it is assumed that such cases are not encountered in the current code.

To not include <xen/linux-compat.h> to Xen library files __ffs() and __ffz()
were defined locally in find-next-bit.c.

Except __ffs() usage in find-next-bit.c only one usage of __ffs() leave
in smmu-v3.c. It seems that it __ffs can be changed to ffsl(x)-1 in
this file, but to keep smmu-v3.c looks close to linux it was deciced just
to define __ffs() in xen/linux-compat.h and include it in smmu-v3.c

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V9:
 - update the defintion of __ffs in xen/linux-compat.h.
 - add Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - add Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com> for PPC part which
   I missed to add in the previous patch version.
---
Changes in V8:
 - drop ffz() for PPC as there is no any usage of it and it seems to me that it was
   introduced only because Arm has it, and Arm uses it only in find-next-bit.c where
   ffz() was moved to.
 - add Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com> for PPC part.
---
Changes in V7:
 - introduce ffz(),__ffs() locally in find-next-bit.c
 - drop inclusion of <xen/linux-compat.h> in find-next-bit.c.
 - update the commit message.
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm64/bitops.h | 21 ---------------------
 xen/arch/ppc/include/asm/bitops.h       | 21 ---------------------
 xen/drivers/passthrough/arm/smmu-v3.c   |  2 ++
 xen/include/xen/linux-compat.h          |  2 ++
 xen/lib/find-next-bit.c                 |  3 +++
 5 files changed, 7 insertions(+), 42 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 5f5d97faa0..2deb134388 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -1,27 +1,6 @@
 #ifndef _ARM_ARM64_BITOPS_H
 #define _ARM_ARM64_BITOPS_H
 
-/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */
-/**
- * __ffs - find first bit in word.
- * @word: The word to search
- *
- * Undefined if no bit exists, so code should check against 0 first.
- */
-static /*__*/always_inline unsigned long __ffs(unsigned long word)
-{
-        return __builtin_ctzl(word);
-}
-
-/* Based on linux/include/asm-generic/bitops/ffz.h */
-/*
- * ffz - find first zero in word.
- * @word: The word to search
- *
- * Undefined if no zero exists, so code should check against ~0UL first.
- */
-#define ffz(x)  __ffs(~(x))
-
 static inline int arch_flsl(unsigned long x)
 {
         uint64_t ret;
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index ca308fd62b..2237b9f8f4 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -119,15 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/* Based on linux/include/asm-generic/bitops/ffz.h */
-/*
- * ffz - find first zero in word.
- * @word: The word to search
- *
- * Undefined if no zero exists, so code should check against ~0UL first.
- */
-#define ffz(x) __ffs(~(x))
-
 /**
  * hweightN - returns the hamming weight of a N-bit word
  * @x: the word to weigh
@@ -139,16 +130,4 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
 #define hweight16(x) __builtin_popcount((uint16_t)(x))
 #define hweight8(x)  __builtin_popcount((uint8_t)(x))
 
-/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */
-/**
- * __ffs - find first bit in word.
- * @word: The word to search
- *
- * Undefined if no bit exists, so code should check against 0 first.
- */
-static always_inline unsigned long __ffs(unsigned long word)
-{
-    return __builtin_ctzl(word);
-}
-
 #endif /* _ASM_PPC_BITOPS_H */
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index b1c40c2c0a..6904962467 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -72,12 +72,14 @@
  */
 
 #include <xen/acpi.h>
+#include <xen/bitops.h>
 #include <xen/config.h>
 #include <xen/delay.h>
 #include <xen/errno.h>
 #include <xen/err.h>
 #include <xen/irq.h>
 #include <xen/lib.h>
+#include <xen/linux-compat.h>
 #include <xen/list.h>
 #include <xen/mm.h>
 #include <xen/rbtree.h>
diff --git a/xen/include/xen/linux-compat.h b/xen/include/xen/linux-compat.h
index 62ba71485c..b289dfd894 100644
--- a/xen/include/xen/linux-compat.h
+++ b/xen/include/xen/linux-compat.h
@@ -19,4 +19,6 @@ typedef int64_t __s64;
 
 typedef paddr_t phys_addr_t;
 
+#define __ffs(x) (ffsl(x) - 1UL)
+
 #endif /* __XEN_LINUX_COMPAT_H__ */
diff --git a/xen/lib/find-next-bit.c b/xen/lib/find-next-bit.c
index ca6f82277e..761b027398 100644
--- a/xen/lib/find-next-bit.c
+++ b/xen/lib/find-next-bit.c
@@ -12,6 +12,9 @@
 
 #include <asm/byteorder.h>
 
+#define __ffs(x) (ffsl(x) - 1)
+#define ffz(x) __ffs(~(x))
+
 #ifndef find_next_bit
 /*
  * Find the next set bit in a memory region.
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717432.1119648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNe-0001x1-61; Mon, 06 May 2024 10:15:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717432.1119648; Mon, 06 May 2024 10:15:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNd-0001vl-Rr; Mon, 06 May 2024 10:15:33 +0000
Received: by outflank-mailman (input) for mailman id 717432;
 Mon, 06 May 2024 10:15:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNc-0001cU-M7
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:32 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 919f05ad-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:31 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso33035531fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:31 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 919f05ad-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990531; x=1715595331; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Rn7sRl5y5flPj8K4Q3SklEy2SXb6R+KXG+lim5CHAxw=;
        b=dTR9sBLMlXPkQhAi4ID6dOOiIrTRryiEAGqVVtlg5xzMIRR6atNp2JRREBUpfjD9KN
         EXhKOX0g52AgKRk6WM5cjDr+Fo4ZY1WM2Ob0V2eMEn6q2Q3ky3nBclCq5diysim64QAV
         YoWIlyskrXditJAFbe6e2VAgYBSm5rliYgYA5hfOyqH7FmdThabnr4x2OyZRkHAXPb+5
         yBTooXZCjif9F9dGU9iwhz/x8qzpmYLWFrjBEMtFVQ0cNNl/L2BbSOkGngV2zHZJYu2m
         x+Es6uBSgi6VpSTyFRUZQCCNdAiBqi0eEWBMaEYl0jwprm6uQQTEPUXtOr/fJQBl3fMY
         8CLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990531; x=1715595331;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Rn7sRl5y5flPj8K4Q3SklEy2SXb6R+KXG+lim5CHAxw=;
        b=p5DtvvGR5qzGaPHy530EzkrT4MF8lfKskeh5Uqym+myVKPQvbzSfXBcX7+toKpiloQ
         261GF8bVm56yYpxXpY3Y6k8EFphIWoqI3s6Vi9pAPrAadiLWO73Sj8PSLVcYVPHvXyBX
         loD2Xf6ZtRICPejccXENYGZXl5f938AqiZgy39z8Bx2f+3aqNMkFtAl0TjsjnkqHY4Qy
         D9FxfSgr+mV6ZMdVxnmpck9FrGLGxFbuYs2A3Py9toDM26iL8IQjlabx6kE5VyZ/UXdH
         QRNrBmhM74C+8fRWGUYyWyys7fL9GXHviJ8joD0pb0yRSJ84Si74p4tfRVhFSNkIPucY
         xhBA==
X-Gm-Message-State: AOJu0Yyy1qzyLOGhxdcEuapT/495uAKa4POI5V8N7kU1vkWXVjNP6+I4
	JppolAvvXScS1g3KEwothXQHONuBjSEX/M/ac7z7hBWzc+qv5CF39hvajw==
X-Google-Smtp-Source: AGHT+IENFl389NbTRpubiCLv3s9XJPPInqH5vSJK+hwpcjNWxDQaA8dKnWJ+RyKFPo8+5DJkW+6C/g==
X-Received: by 2002:a2e:a551:0:b0:2d8:5af9:90c5 with SMTP id e17-20020a2ea551000000b002d85af990c5mr10187437ljn.39.1714990530265;
        Mon, 06 May 2024 03:15:30 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
Date: Mon,  6 May 2024 12:15:11 +0200
Message-ID: <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Because of that x86 has the following check in the macros test_bit(),
__test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit():
    if ( bitop_bad_size(addr) ) __bitop_bad_size();
It was necessary to make bitop bad size check generic too, so
arch_check_bitop_size() was introduced.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
   The context ("* Find First Set bit.  Bits are labelled from 1." in xen/bitops.h )
   suggests there's a dependency on an uncommitted patch. It happens becuase the current patch
   series is based on Andrew's patch series ( https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t ),
   but if everything is okay with the current one patch it can be merged without Andrew's patch series being merged.
---
Changes in V9:
  - move up xen/bitops.h in ppc/asm/page.h.
  - update defintion of arch_check_bitop_size.
    And drop correspondent macros from x86/asm/bitops.h
  - drop parentheses in generic__test_and_set_bit() for definition of
    local variable p.
  - fix indentation inside #ifndef BITOP_TYPE...#endif
  - update the commit message.
---
 Changes in V8:
  - drop __pure for function which uses volatile.
  - drop unnessary () in generic__test_and_change_bit() for addr casting.
  - update prototype of generic_test_bit() and test_bit(): now it returns bool
    instead of int.
  - update generic_test_bit() to use BITOP_MASK().
  - Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
  - add a footer with explanation of dependency on an uncommitted patch after Signed-off.
  - abstract bitop_size().
  - move BITOP_TYPE define to <xen/types.h>.
---
 Changes in V7:
  - move everything to xen/bitops.h to follow the same approach for all generic
    bit ops.
  - put together BITOP_BITS_PER_WORD and bitops_uint_t.
  - make BITOP_MASK more generic.
  - drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
    enough.
  - drop "_" for generic__{test_and_set_bit,...}().
  - drop " != 0" for functions which return bool.
  - add volatile during the cast for generic__{...}().
  - update the commit message.
  - update arch related code to follow the proposed generic approach.
---
 Changes in V6:
  - Nothing changed ( only rebase )
---
 Changes in V5:
   - new patch
---
 xen/arch/arm/arm64/livepatch.c    |   1 -
 xen/arch/arm/include/asm/bitops.h |  67 ---------------
 xen/arch/ppc/include/asm/bitops.h |  52 ------------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  31 ++-----
 xen/include/xen/bitops.h          | 134 ++++++++++++++++++++++++++++++
 xen/include/xen/types.h           |   6 ++
 8 files changed, 151 insertions(+), 144 deletions(-)

diff --git a/xen/arch/arm/arm64/livepatch.c b/xen/arch/arm/arm64/livepatch.c
index df2cebedde..4bc8ed9be5 100644
--- a/xen/arch/arm/arm64/livepatch.c
+++ b/xen/arch/arm/arm64/livepatch.c
@@ -10,7 +10,6 @@
 #include <xen/mm.h>
 #include <xen/vmap.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/insn.h>
 #include <asm/livepatch.h>
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 5104334e48..8e16335e76 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -22,9 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 #define ADDR (*(volatile int *) addr)
@@ -76,70 +73,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 /*
  * On ARMv5 and above those functions can be implemented around
  * the clz instruction for much better code efficiency.
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 989d341a44..e2b6473c8c 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -15,9 +15,6 @@
 #define __set_bit(n, p)         set_bit(n, p)
 #define __clear_bit(n, p)       clear_bit(n, p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 /* PPC bit number conversion */
@@ -69,17 +66,6 @@ static inline void clear_bit(int nr, volatile void *addr)
     clear_bits(BITOP_MASK(nr), (volatile unsigned int *)addr + BITOP_WORD(nr));
 }
 
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-    const volatile unsigned int *p = addr;
-    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
-}
-
 static inline unsigned int test_and_clear_bits(
     unsigned int mask,
     volatile unsigned int *p)
@@ -133,44 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old | mask;
-    return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old & ~mask;
-    return (old & mask) != 0;
-}
-
 #define flsl(x) generic_flsl(x)
 #define fls(x) generic_fls(x)
 
diff --git a/xen/arch/ppc/include/asm/page.h b/xen/arch/ppc/include/asm/page.h
index 890e285051..6d4cd2611c 100644
--- a/xen/arch/ppc/include/asm/page.h
+++ b/xen/arch/ppc/include/asm/page.h
@@ -2,9 +2,9 @@
 #ifndef _ASM_PPC_PAGE_H
 #define _ASM_PPC_PAGE_H
 
+#include <xen/bitops.h>
 #include <xen/types.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
diff --git a/xen/arch/ppc/mm-radix.c b/xen/arch/ppc/mm-radix.c
index ab5a10695c..9055730997 100644
--- a/xen/arch/ppc/mm-radix.c
+++ b/xen/arch/ppc/mm-radix.c
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index dd439b69a0..23f09fdb7a 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -19,9 +19,6 @@
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
-extern void __bitop_bad_size(void);
-#define bitop_bad_size(addr) (sizeof(*(addr)) < 4)
-
 /**
  * set_bit - Atomically set a bit in memory
  * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index f14ad0d33a..4f3399273a 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -65,10 +65,144 @@ static inline int generic_flsl(unsigned long x)
  * scope
  */
 
+#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
+
+#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
+
+extern void __bitop_bad_size(void);
+
 /* --------------------- Please tidy above here --------------------- */
 
 #include <asm/bitops.h>
 
+#ifndef arch_check_bitop_size
+
+#define bitop_bad_size(addr) sizeof(*(addr)) < sizeof(bitop_uint_t)
+
+#define arch_check_bitop_size(addr) \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();
+
+#endif /* arch_check_bitop_size */
+
+/**
+ * generic__test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask);
+}
+
+/**
+ * generic__test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask);
+}
+
+/* WARNING: non atomic and it can be reordered! */
+static always_inline bool
+generic__test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask);
+}
+/**
+ * generic_test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static always_inline bool generic_test_bit(int nr, const volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+
+    return (*p & mask);
+}
+
+static always_inline bool
+__test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+#ifndef arch__test_and_set_bit
+#define arch__test_and_set_bit generic__test_and_set_bit
+#endif
+
+    return arch__test_and_set_bit(nr, addr);
+}
+#define __test_and_set_bit(nr, addr) ({             \
+    arch_check_bitop_size(addr);                    \
+    __test_and_set_bit(nr, addr);                   \
+})
+
+static always_inline bool
+__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
+{
+#ifndef arch__test_and_clear_bit
+#define arch__test_and_clear_bit generic__test_and_clear_bit
+#endif
+
+    return arch__test_and_clear_bit(nr, addr);
+}
+#define __test_and_clear_bit(nr, addr) ({           \
+    arch_check_bitop_size(addr);                    \
+    __test_and_clear_bit(nr, addr);                 \
+})
+
+static always_inline bool
+__test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+#ifndef arch__test_and_change_bit
+#define arch__test_and_change_bit generic__test_and_change_bit
+#endif
+
+    return arch__test_and_change_bit(nr, addr);
+}
+#define __test_and_change_bit(nr, addr) ({              \
+    arch_check_bitop_size(addr);                        \
+    __test_and_change_bit(nr, addr);                    \
+})
+
+static always_inline bool test_bit(int nr, const volatile void *addr)
+{
+#ifndef arch_test_bit
+#define arch_test_bit generic_test_bit
+#endif
+
+    return arch_test_bit(nr, addr);
+}
+#define test_bit(nr, addr) ({                           \
+    arch_check_bitop_size(addr);                        \
+    test_bit(nr, addr);                                 \
+})
+
 /*
  * Find First Set bit.  Bits are labelled from 1.
  */
diff --git a/xen/include/xen/types.h b/xen/include/xen/types.h
index 449947b353..2d63d984eb 100644
--- a/xen/include/xen/types.h
+++ b/xen/include/xen/types.h
@@ -64,6 +64,12 @@ typedef __u64 __be64;
 
 typedef unsigned int __attribute__((__mode__(__pointer__))) uintptr_t;
 
+#ifndef BITOP_TYPE
+#define BITOP_BITS_PER_WORD 32
+
+typedef uint32_t bitop_uint_t;
+#endif
+
 #define test_and_set_bool(b)   xchg(&(b), true)
 #define test_and_clear_bool(b) xchg(&(b), false)
 
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717430.1119631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNb-0001dy-FM; Mon, 06 May 2024 10:15:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717430.1119631; Mon, 06 May 2024 10:15:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNb-0001dr-CG; Mon, 06 May 2024 10:15:31 +0000
Received: by outflank-mailman (input) for mailman id 717430;
 Mon, 06 May 2024 10:15:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNa-0001cU-9z
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:30 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90447dda-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:29 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so140112066b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:29 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90447dda-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990528; x=1715595328; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=V2mq6PYp1Rzsja4sLq2SCNUxiJkS+6Sedd3YBnZM19A=;
        b=YQx6xNnTnejzLlK02lvqMxSvvcnCzlMrOJVNAuUIYg6WPEfSD4dQ0v3SkI4aC7Jh1V
         KlY590ZQ0a6XwO3CiIJ5BPdwUdn5eaIBMvsA8aWRDwrUrlikefHpMJ0zIrfclD5SMMEp
         n6UYtVMs9rI4JLdc1RDU8YvY4SOJfmPvel7ysDHDLtAo4EYNOSWNxqFSEHxzbBPrL1nG
         bD6977VuEcKySyfzwBN322OyLg0QZgOHIMRn4AwEQj+LRbhChJe0s93cx+MFOmSeArT2
         MXjcAJ8eLyn00ETtNTZeIPlLva9yRHfBMFuLpJsbOFEPW7d1Rl5RcjIrbI8nL2ipXeK7
         /USA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990528; x=1715595328;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=V2mq6PYp1Rzsja4sLq2SCNUxiJkS+6Sedd3YBnZM19A=;
        b=eOGCggmJHWO9YvMOJoqLyRTdSa+w8F4TT2oafSLUkW4/BGRe2fEcc8tONDxBfPUYuQ
         6L5yJWsXmXPnpWa7EPJTaAUxuFvJOrcqb/R6vmqdZ0TlfIKSDL0UuynhLq0TpCEAzRiN
         6ROCPJkSTMyHXLGJlCkWfl4fRxycTLl+xlN1mJyQUja/u9g4K+9+9Ihw9UqbkZs9HAQ9
         Fj94jZO9/MF4jgAFmpOAqOQ8o6hulQSCcUW2ebRP50FnJV/glFPwB2ckW2MX1MZ3vTEW
         9QSNnkO7HW8LvAbokLzjlvyY5ZtQwdNOor4ej53egMX62a1WIsjDczVfqbQRvbHnoxUB
         4bJA==
X-Gm-Message-State: AOJu0Yx4HhhicNPOOBdi18qlnm6ffKVacdLz/ym+zly3ajRc932qGNGQ
	14EHjKZ1j4FBhJNMcedf6mjaKTKkBTet40SXdQTiC1MOcRcgBMwWrQ/oUg==
X-Google-Smtp-Source: AGHT+IFowQgxlDgro5aXRiLb7cmn2K3coY5/eRQxyiFzhMYsEZA8lWL5Po7CE3U0keaLo1XnZH0Ygw==
X-Received: by 2002:a17:906:f754:b0:a59:b490:e77b with SMTP id jp20-20020a170906f75400b00a59b490e77bmr3032366ejb.40.1714990528099;
        Mon, 06 May 2024 03:15:28 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Rahul Singh <rahul.singh@arm.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v9 00/15] Enable build of full Xen for RISC-V
Date: Mon,  6 May 2024 12:15:09 +0200
Message-ID: <cover.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.
As an option, it can be considered to merge arch-specific patch and then just
rebase [2] and drop arch-specific changes.

[1] https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

---
Changes in V9:
 - Patch was merged to staging:
    - [PATCH v8 07/17] xen/riscv: introduce io.h
  	- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V8:
 - Patch was merged to staging:
    - [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
    - [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
 - Update the commit message:
     - drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
     - Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (15):
  xen/riscv: disable unnecessary configs
  xen: introduce generic non-atomic test_*bit()
  xen/bitops: implement fls{l}() in common logic
  xen/bitops: put __ffs() into linux compatible header
  xen/riscv: introduce bitops.h
  xen/riscv: introduce cmpxchg.h
  xen/riscv: introduce atomic.h
  xen/riscv: introduce monitor.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add required things to current.h
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: introduce vm_event_*() functions
  xen/riscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                  |   4 +
 automation/gitlab-ci/build.yaml         |   4 +
 xen/arch/arm/arm64/livepatch.c          |   1 -
 xen/arch/arm/include/asm/arm32/bitops.h |   2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  27 +-
 xen/arch/arm/include/asm/bitops.h       |  74 +----
 xen/arch/ppc/include/asm/bitops.h       |  76 -----
 xen/arch/ppc/include/asm/page.h         |   2 +-
 xen/arch/ppc/mm-radix.c                 |   2 +-
 xen/arch/riscv/Makefile                 |  18 +-
 xen/arch/riscv/arch.mk                  |   4 -
 xen/arch/riscv/configs/tiny64_defconfig |  12 +-
 xen/arch/riscv/early_printk.c           | 168 ----------
 xen/arch/riscv/include/asm/atomic.h     | 280 ++++++++++++++++
 xen/arch/riscv/include/asm/bitops.h     | 137 ++++++++
 xen/arch/riscv/include/asm/cache.h      |   2 +
 xen/arch/riscv/include/asm/cmpxchg.h    | 239 ++++++++++++++
 xen/arch/riscv/include/asm/config.h     |   2 +
 xen/arch/riscv/include/asm/current.h    |  19 ++
 xen/arch/riscv/include/asm/domain.h     |   2 +
 xen/arch/riscv/include/asm/mm.h         | 240 ++++++++++++++
 xen/arch/riscv/include/asm/monitor.h    |  26 ++
 xen/arch/riscv/include/asm/p2m.h        |   2 +
 xen/arch/riscv/mm.c                     |  52 ++-
 xen/arch/riscv/setup.c                  |  10 +-
 xen/arch/riscv/stubs.c                  | 415 ++++++++++++++++++++++++
 xen/arch/riscv/traps.c                  |  25 ++
 xen/arch/riscv/vm_event.c               |  19 ++
 xen/arch/x86/include/asm/bitops.h       |  37 +--
 xen/common/bitops.c                     |  22 ++
 xen/common/page_alloc.c                 |   4 +-
 xen/drivers/passthrough/arm/smmu-v3.c   |   2 +
 xen/include/asm-generic/atomic-ops.h    |  97 ++++++
 xen/include/xen/bitops.h                | 158 +++++++++
 xen/include/xen/linux-compat.h          |   2 +
 xen/include/xen/types.h                 |   6 +
 xen/lib/find-next-bit.c                 |   3 +
 37 files changed, 1810 insertions(+), 385 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
 create mode 100644 xen/arch/riscv/include/asm/monitor.h
 create mode 100644 xen/arch/riscv/stubs.c
 create mode 100644 xen/arch/riscv/vm_event.c
 create mode 100644 xen/include/asm-generic/atomic-ops.h

-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717433.1119661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNf-0002OC-Ix; Mon, 06 May 2024 10:15:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717433.1119661; Mon, 06 May 2024 10:15:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNf-0002O3-EY; Mon, 06 May 2024 10:15:35 +0000
Received: by outflank-mailman (input) for mailman id 717433;
 Mon, 06 May 2024 10:15:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNd-0001cU-Cl
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:33 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9263e977-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:32 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59b178b75bso231065766b.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:32 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9263e977-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990532; x=1715595332; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GwX58G2PYucDxTd1xZ6gewg6BqcdHjkMPEio9G/R6/g=;
        b=W7UpMGCERnKa59Hjv66TwSO7jIqjTnFsHRNvKOOKeJHtIIFRxcPzOoJMLqhpCuEA0b
         5r4yAeFxPHcZYiPjgM6o18QcF1DkFlwM1pHLtjojyhipXbj4Nujyqx+JyD12foVR61ik
         eM6/lzJm+j+qEUWbHNW9r2WOE4hYtyY4et6f3inYM5vpEI24pHf4VsMt0Ydyr/aow6OT
         zmnOiwctTU3WjKufmTJVPWuC9pvVrQ+9aDpeRlTWzENYtPUGS3b2KvIED1ILmz3Mp391
         tJ+omlyM8yA+tKJRYobNhaon2MhDMYMqBwIWha6Us/dvvEeU5c4EOnNGMOVJ3buxgBCf
         ctwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990532; x=1715595332;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GwX58G2PYucDxTd1xZ6gewg6BqcdHjkMPEio9G/R6/g=;
        b=bay7K8i1+f0Ap5fCqe6sUGlUMQW3F3mT99PgO13bKwO4oJE+7fJzV1O06a47/x8Ay8
         vY9+INRJIOHd3HetEtkMl+iPe0YDFU9MRVF0Ebnv2xiHHFIuK2KeoSE6vimwtxbWbGIH
         iAEYI55UKUAAkdPd9FQajSRdTuFb9JqbYnHOHdZiBKQ/qs7zWDLfnS+qTGZhzXGXOTNP
         yBwK2xg2kyHMpBgbYzIAsGeAR7LCXcGFBJxn1+rQp2/Peu4/COvAIldKfrvho1c1BFqc
         YWbKk46lWhssc19RIl8NuiDy9CSNpA3ISA/bVJTvr5pA76qwmlTWt8ijN1aabAkCwaGo
         r+Ag==
X-Gm-Message-State: AOJu0Yw5wXCAn3xyMUWTcSdWttWngdNAMc0qoM2VFdxewUIgmmFH9Mdj
	0Uk5a3ZB6kTic3CODykTS0q3d5/M+THlO9Se0L4DFvM0K0X73jhdn75DdA==
X-Google-Smtp-Source: AGHT+IFc90sRqQ7a5yx1Wt+wBItRldSAvqCOT4TbDzIMWDfj+lR0G5ORNitsKnj7p8VY1ZMkBVKShw==
X-Received: by 2002:a17:907:e86:b0:a59:c6fd:5160 with SMTP id ho6-20020a1709070e8600b00a59c6fd5160mr2766166ejc.76.1714990531465;
        Mon, 06 May 2024 03:15:31 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
Date: Mon,  6 May 2024 12:15:12 +0200
Message-ID: <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To avoid the compilation error below, it is needed to update to places
in common/page_alloc.c where flsl() is used as now flsl() returns unsigned int:

./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
       18 |         (void) (&_x == &_y);            \
          |                     ^~
    common/page_alloc.c:1843:34: note: in expansion of macro 'min'
     1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);

generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0,
the result in undefined.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 The patch is almost independent from Andrew's patch series
 ( https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t)
 except test_fls() function which IMO can be merged as a separate patch after Andrew's patch
 will be fully ready.
---
Changes in V9:
 - update return type of fls and flsl() to unsigned int to be aligned with other
   bit ops.
 - update places where return value of fls() and flsl() is compared with int.
 - update the commit message.
---
Changes in V8:
 - do proper rebase: back definition of fls{l} to the current patch.
 - drop the changes which removed ffz() in PPC. it should be done not
   in this patch.
 - add a message after Signed-off.
---
Changes in V7:
 - Code style fixes
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm32/bitops.h |  2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  6 ++----
 xen/arch/arm/include/asm/bitops.h       |  7 ++-----
 xen/arch/ppc/include/asm/bitops.h       |  3 ---
 xen/arch/x86/include/asm/bitops.h       |  6 ++++--
 xen/common/bitops.c                     | 22 ++++++++++++++++++++++
 xen/common/page_alloc.c                 |  4 ++--
 xen/include/xen/bitops.h                | 24 ++++++++++++++++++++++++
 8 files changed, 57 insertions(+), 17 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/bitops.h b/xen/arch/arm/include/asm/arm32/bitops.h
index d0309d47c1..5552d4e945 100644
--- a/xen/arch/arm/include/asm/arm32/bitops.h
+++ b/xen/arch/arm/include/asm/arm32/bitops.h
@@ -1,7 +1,7 @@
 #ifndef _ARM_ARM32_BITOPS_H
 #define _ARM_ARM32_BITOPS_H
 
-#define flsl fls
+#define arch_flsl fls
 
 /*
  * Little endian assembly bitops.  nr = 0 -> byte 0 bit 0.
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 0efde29068..5f5d97faa0 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long __ffs(unsigned long word)
  */
 #define ffz(x)  __ffs(~(x))
 
-static inline int flsl(unsigned long x)
+static inline int arch_flsl(unsigned long x)
 {
         uint64_t ret;
 
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
         asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
 
         return BITS_PER_LONG - ret;
 }
+#define arch_flsl arch_flsl
 
 /* Based on linux/include/asm-generic/bitops/find.h */
 
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 8e16335e76..860d6d4689 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -78,17 +78,14 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p,
  * the clz instruction for much better code efficiency.
  */
 
-static inline int fls(unsigned int x)
+static inline int arch_fls(unsigned int x)
 {
         int ret;
 
-        if (__builtin_constant_p(x))
-               return generic_fls(x);
-
         asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
         return 32 - ret;
 }
-
+#define arch_fls arch_fls
 
 #define arch_ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
 #define arch_ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index e2b6473c8c..ca308fd62b 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -119,9 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-#define flsl(x) generic_flsl(x)
-#define fls(x) generic_fls(x)
-
 /* Based on linux/include/asm-generic/bitops/ffz.h */
 /*
  * ffz - find first zero in word.
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 23f09fdb7a..8f3d76fe44 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -425,7 +425,7 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
  *
  * This is defined the same way as ffs.
  */
-static inline int flsl(unsigned long x)
+static always_inline int arch_flsl(unsigned long x)
 {
     long r;
 
@@ -435,8 +435,9 @@ static inline int flsl(unsigned long x)
           "1:" : "=r" (r) : "rm" (x));
     return (int)r+1;
 }
+#define arch_flsl arch_flsl
 
-static inline int fls(unsigned int x)
+static always_inline int arch_fls(unsigned int x)
 {
     int r;
 
@@ -446,6 +447,7 @@ static inline int fls(unsigned int x)
           "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_fls arch_fls
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index a8c32f6767..95bc47176b 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -62,9 +62,31 @@ static void test_ffs(void)
     CHECK(ffs64, (uint64_t)0x8000000000000000, 64);
 }
 
+static void test_fls(void)
+{
+    /* unsigned int ffs(unsigned int) */
+    CHECK(fls, 1, 1);
+    CHECK(fls, 3, 2);
+    CHECK(fls, 3U << 30, 32);
+
+    /* unsigned int flsl(unsigned long) */
+    CHECK(flsl, 1, 1);
+    CHECK(flsl, 1UL << (BITS_PER_LONG - 1), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    CHECK(flsl, 3UL << 32, 34);
+#endif
+
+    /* unsigned int fls64(uint64_t) */
+    CHECK(fls64, 1, 1);
+    CHECK(fls64, 0x00000000C0000000ULL, 32);
+    CHECK(fls64, 0x0000000180000000ULL, 33);
+    CHECK(fls64, 0xC000000000000000ULL, 64);
+}
+
 static int __init cf_check test_bitops(void)
 {
     test_ffs();
+    test_fls();
 
     return 0;
 }
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index be4ba3962a..eed6b2a901 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1842,7 +1842,7 @@ static void _init_heap_pages(const struct page_info *pg,
          * Note that the value of ffsl() and flsl() starts from 1 so we need
          * to decrement it by 1.
          */
-        unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
+        unsigned int inc_order = min(MAX_ORDER + 0U, flsl(e - s) - 1);
 
         if ( s )
             inc_order = min(inc_order, ffsl(s) - 1U);
@@ -2266,7 +2266,7 @@ void __init xenheap_max_mfn(unsigned long mfn)
     ASSERT(!first_node_initialised);
     ASSERT(!xenheap_bits);
     BUILD_BUG_ON((PADDR_BITS - PAGE_SHIFT) >= BITS_PER_LONG);
-    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS);
+    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS + 0U);
     printk(XENLOG_INFO "Xen heap: %u bits\n", xenheap_bits);
 }
 
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 4f3399273a..f5be10b928 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -203,6 +203,30 @@ static always_inline bool test_bit(int nr, const volatile void *addr)
     test_bit(nr, addr);                                 \
 })
 
+static always_inline __pure unsigned int fls(unsigned int x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_fls(x);
+
+#ifndef arch_fls
+#define arch_fls generic_fls
+#endif
+
+    return arch_fls(x);
+}
+
+static always_inline __pure unsigned int flsl(unsigned long x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_flsl(x);
+
+#ifndef arch_flsl
+#define arch_flsl generic_flsl
+#endif
+
+    return arch_flsl(x);
+}
+
 /*
  * Find First Set bit.  Bits are labelled from 1.
  */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717435.1119673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNg-0002ac-Fg; Mon, 06 May 2024 10:15:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717435.1119673; Mon, 06 May 2024 10:15:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNg-0002Yo-7l; Mon, 06 May 2024 10:15:36 +0000
Received: by outflank-mailman (input) for mailman id 717435;
 Mon, 06 May 2024 10:15:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNe-0001cU-NW
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:34 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 93317354-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:34 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59cdd185b9so106588966b.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:34 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93317354-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990533; x=1715595333; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=c72THRDPJytFxUouq/3yaM0Y1kQGbdnojUTJoNIkwb0=;
        b=LRmb6jIZN9EedVUExHaQwXYSDB9Q+Q1YMyT0G2B6soYBRLQ/Mnymz3hNS/0ytFR2bw
         orv4Qmr06sndHPhkQMT9AHxXuWNI+QfP+AJxc0nT7KYViml239nTRJzzIdHeHURxhT6e
         SMoJyLVBUfOpReIavP4NIvExXy2TrlQyTbJXAeznMKALSb6ULhtte2P2oDteC+eF8o76
         fT7AlmEx+bAFrN0cRPX/K562vB+qtJ42OlDUM7bIyTteP6D1wI3VVPpTbkVBpdhb1oBI
         RuGCJg+uvrjh/bnIObkgEAvyMU5zp+w0IoFyDN+4UmXcxGa9cw8oRm+BGi0zML19RaZe
         /gJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990533; x=1715595333;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=c72THRDPJytFxUouq/3yaM0Y1kQGbdnojUTJoNIkwb0=;
        b=WZa5kpEFzU0teU5qUbbVi2yyHR6k/HOY/BJqsFlLmQpkm76mlqjdux9l0GsMD6uFm/
         R4gTqTRhXguVQ2TK38AJYWocZAqzvuWkDnVNmnEQIGRBzFJvPDutonINh/RkzRDu0STw
         LQ3RtixdoVHCtC9TrGJ0xmJAo8OgbkmT199p8lHFmKNnBnUghKCmz7qTLyWAzfFpcHEs
         uBqeasuLFIrAJgtnx4VYKdWQRq46athYvXAB+al18hjHus2swryQZen/toOFzrTbnjMY
         DKaExuGsW/RxHpYWl8wDfh71MLuTAkNmYmsoo2MHKTw/FAhn4zuauYRN5KWeA3qH36zO
         rYGw==
X-Gm-Message-State: AOJu0Yw94oZNDW4DqdE3aENzLhKqeQL2XP6Wgpj4D774FXwSSopbzMvN
	8YfLehpg3g+7T+CjfWXrtIvSl5YUrh06/rQNvzB9ht0JxPrXuduEjExrdw==
X-Google-Smtp-Source: AGHT+IFempjP+TE6sNlnduMEa1bJE6kAMEUQJ9XE7S1bYbydX4IjJqj2e+Vf/Euk0dAkVpEHX85xNg==
X-Received: by 2002:a17:906:4a4b:b0:a59:908e:42d with SMTP id a11-20020a1709064a4b00b00a59908e042dmr6542249ejv.32.1714990533267;
        Mon, 06 May 2024 03:15:33 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 05/15] xen/riscv: introduce bitops.h
Date: Mon,  6 May 2024 12:15:14 +0200
Message-ID: <c81251aa51abe383f60a24770a07a9aa6d673a21.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V9:
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects
   to work with 32-bit quantities.
 - s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
 - define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of
   "introduce generic non-atomic test_*bit()".
 - drop duplicated __set_bit() and __clear_bit().
 - drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
 - update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
 - drop 1 padding blank in test_and_op_bit_ord().
 - update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
   change return type to bool.
 - change addr argument type of test_and_change_bit(): unsigned long * -> void *.
 - move test_and_change_bit() closer to other test_and-s function.
 - Code style fixes: tabs -> space.
 - s/#undef __op_bit/#undef op_bit.
 - update the commit message: delete information about generic-non-atomic.h changes as now
   it is a separate patch.
---
Changes in V7:
 - Update the commit message.
 - Drop "__" for __op_bit and __op_bit_ord as they are atomic.
 - add comment above __set_bit and __clear_bit about why they are defined as atomic.
 - align bitops_uint_t with __AMO().
 - make changes after  generic non-atomic test_*bit() were changed.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
 - rebase clean ups were done: drop unused asm-generic includes
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..9f796bf859
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITS_PER_LONG"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline bool test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline bool test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline bool test_and_change_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717431.1119641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNd-0001sc-Mz; Mon, 06 May 2024 10:15:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717431.1119641; Mon, 06 May 2024 10:15:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNd-0001sV-JL; Mon, 06 May 2024 10:15:33 +0000
Received: by outflank-mailman (input) for mailman id 717431;
 Mon, 06 May 2024 10:15:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNc-0001n2-91
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:32 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 90c7cc01-0b91-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:15:30 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59a9d66a51so353711866b.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:30 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90c7cc01-0b91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990529; x=1715595329; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GS70VGt/vkRIGFqYUodDia2ytY8yKMbU0vSOnyuvPs0=;
        b=FEK1rtCHBibIkRfqMrYP1ttJrjXNRoyOG8vR0zx8+fzv8mQuB827OZbAJD4mwnNohl
         C1uJHzgQ5uoXN7eIcfqIAgZCrFYp8khOWqa7iyNdY8gVCKz+rmqlNLM/CQvXANRhA7FE
         hAdpwGFbWWaKp5qnwGIUysbj3d19ReiMsAGcqXr+DckHRSeJfB0r8IIrwhzUwPPDT+0O
         +qGo8V+6G2EDEEOFRRqfUFnG2xKzMsPXJEpFXx26XZFZEnv2Hm3vORlUoTBLN4N9Aoiz
         MGI3NC4Zleos9DI8u2SUp1/DLi/522rwFqTJXVxSHnXV1WQ+v30gwyimQtyL94UX8rQB
         eApA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990529; x=1715595329;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GS70VGt/vkRIGFqYUodDia2ytY8yKMbU0vSOnyuvPs0=;
        b=JkCysEtqJr9TQiWOBGRxIpthX9SmOCAkW0ADVh8hXcUIcKqy65uMhK+b8zQvXo96e1
         wwege+1Yr+X4l0TvA4xWt2IFhKabIlGWPNz12KW7ueqMeTc3hAHwDvgL0WO5pjjApP6p
         RxgjXheNKhwcWJiViYGSGjupCyOChJ73ObEfN4MSoKffMnI0rAifY7DAfhD3zvKxdb54
         u4p7OH3won0Q/SqPVpzOvAtCPUOth8SyshB7y/hxW2YEEbFk1yeOkZsgUD2cEUCLllqV
         0549Sp4oI3PzWPH8Lus8TSIDwp5jTEW6kZqyKIP1QuQcvcYC11MZ6a+5pRlM8Dm+nbgK
         /j1A==
X-Gm-Message-State: AOJu0YypOb9C8Mxk0KS5jsrvJ1EDNg+nWzYx8r/TyJS8x02bF33ANW97
	irsa2Ba1hsBjFRqftDbsBUx1T0iTkzBdjUwDz9LTDUP2/zamyPOCAo+d+g==
X-Google-Smtp-Source: AGHT+IFNfQs1aGRBM2aNpOJpnBHabE19zaC1aCJMK8HRlERQab4UdfftQY1oTH5pLYL0Ztec1KuYhg==
X-Received: by 2002:a17:907:7291:b0:a59:bfab:b25a with SMTP id dt17-20020a170907729100b00a59bfabb25amr3063761ejc.63.1714990529180;
        Mon, 06 May 2024 03:15:29 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v9 01/15] xen/riscv: disable unnecessary configs
Date: Mon,  6 May 2024 12:15:10 +0200
Message-ID: <f3fd3f5ba8a8cd5ead43880c337a23e62cf8c1e3.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Only configs which lead to compilation issues were disabled.

Remove lines related to disablement of configs which aren't affected
compilation:
 -# CONFIG_SCHED_CREDIT is not set
 -# CONFIG_SCHED_RTDS is not set
 -# CONFIG_SCHED_NULL is not set
 -# CONFIG_SCHED_ARINC653 is not set
 -# CONFIG_TRACEBUFFER is not set
 -# CONFIG_HYPFS is not set
 -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set

To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
check_get_page_from_gfn() from xen/p2m-common.h.

Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was
included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
aren't declared started to occur.

CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.

CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
acpi_disabled, so it is disabled for now.

PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
also disabled for now, as RISC-V hasn't introduced this headers yet.

LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V9:
 - update the commit message: add info about LIVEPATCH and PERF_COUNTERS.
---
Changes in V8:
 - disabled CPU_BOOT_TIME_CPUPOOLS as it requires an introduction of cpu_physical_id() and acpi_disabled.
 - leave XSM disabled, add explanation in the commit message.
 - drop HYPFS as the patch was provided to resolve compilation issue when this condif is enabled for RISC-V.
 - include asm/p2m.h to asm/domain.h, and xen/errno.h to asm/p2m.h to drop ARGO config from
   tiny64_defconfing and build.yaml.
 - update the commit message.
---
Changes in V7:
 - Disable only configs which cause compilation issues.
 - Update the commit message.
---
Changes in V6:
 - Nothing changed. Only rebase.
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         |  4 ++++
 xen/arch/riscv/configs/tiny64_defconfig | 12 +++++-------
 xen/arch/riscv/include/asm/domain.h     |  2 ++
 xen/arch/riscv/include/asm/p2m.h        |  2 ++
 4 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5..ff5c9055d1 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -494,10 +494,14 @@ alpine-3.18-gcc-debug-arm64-earlyprintk:
 .riscv-fixed-randconfig:
   variables: &riscv-fixed-randconfig
     EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_BOOT_TIME_CPUPOOLS=n
       CONFIG_COVERAGE=n
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..fc7a04872f 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -1,12 +1,10 @@
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
+# CONFIG_BOOT_TIME_CPUPOOLS is not set
 # CONFIG_GRANT_TABLE is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_XSM is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 027bfa8a93..16a9dd57aa 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -5,6 +5,8 @@
 #include <xen/xmalloc.h>
 #include <public/hvm/params.h>
 
+#include <asm/p2m.h>
+
 struct hvm_domain
 {
     uint64_t              params[HVM_NR_PARAMS];
diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
index 87b13f8979..aa86fa10a7 100644
--- a/xen/arch/riscv/include/asm/p2m.h
+++ b/xen/arch/riscv/include/asm/p2m.h
@@ -2,6 +2,8 @@
 #ifndef __ASM_RISCV_P2M_H__
 #define __ASM_RISCV_P2M_H__
 
+#include <xen/errno.h>
+
 #include <asm/page-bits.h>
 
 #define paddr_bits PADDR_BITS
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717436.1119690 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNh-000344-Rk; Mon, 06 May 2024 10:15:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717436.1119690; Mon, 06 May 2024 10:15:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNh-00032P-It; Mon, 06 May 2024 10:15:37 +0000
Received: by outflank-mailman (input) for mailman id 717436;
 Mon, 06 May 2024 10:15:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNg-0001cU-34
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:36 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 93d432dd-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:35 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59d0810f59so128947566b.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:35 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93d432dd-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990534; x=1715595334; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AHvwTI4dLlSQ4prD7bg4j4h3gUxydNi/2fMUullhh2s=;
        b=JgYH+b6SOokMK4qckuR61Zl7Dzzd7LBFnXFYTfCWjoO2vYLWWRdMii0kwOMLibLZMO
         mc0DT8YgC2iAx0EY7VziTLQ7OwwfrtJwENTR+5wM8jJVE/cBULxVBNqUvZqu6K10dFo8
         v2oxigw2o+rDbiwyd56rB/49dFpsUq2ZqKrwgekhX6kDaSStsabd35GIIAgF6d1uI8ef
         d0eV/OcKjQ3wYa6TJpMX1l+tsx0txRZFIPWYtcvINlY4ekGz21x09prs2XU3MXfq4gd2
         V7IEYWGlYk2RmDNxzRFkXjoDz8rfG0j8D3/DU4fP3N153Czj0AzeAB9cT90Uayc5d2GG
         Hjpg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990534; x=1715595334;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=AHvwTI4dLlSQ4prD7bg4j4h3gUxydNi/2fMUullhh2s=;
        b=a0rXyo/TvBKbP9L3xKIsswNcwYd+Zey6pMo/cFpZTYW1+DhtQxsUx/puWCMKHF6xuS
         jtUoGhpLxVW1OwrbV+A19hZDOnuYgrEuBIfg5wXLtGgc3j23vQbiKhfBqN3+YLD3QTz9
         V/XK+/GbQFiwVLC/HVC7hmzf2xeUglo1uLAFBXCqDLNpfphs6medt22u/scPBaXyEr+i
         C1rKd+2thNajuA5OqIzJcjr3g1fAuU/fFn+pI2cqAppUMBLFSSOPiuwRol1oLPoIJV7t
         K8e25gLgne0X05zBqoW5g4ocR/5YYTVaZuA0N9mjkvZyBsCJmZ/b50XYBZW4LPnm2iec
         3ICA==
X-Gm-Message-State: AOJu0Yz1/g3ujIMDNbpIJonCDITxBybZaayhhJ+6wwy0QGvQxOCcWwoy
	wSUz4c5dslpVqOpe6W5cfoYdQBUPj4uf9VBybNxcWv66ynzZi35MIlqF5g==
X-Google-Smtp-Source: AGHT+IFMnIrkRESammmSMr6XhiTPPiwWooB/9XlpBT1o73mwSSCSx2Uzllgt+7Wsk/2jSCSQfdi0tA==
X-Received: by 2002:a17:907:7f86:b0:a59:d5f7:a697 with SMTP id qk6-20020a1709077f8600b00a59d5f7a697mr1088704ejc.9.1714990534183;
        Mon, 06 May 2024 03:15:34 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 06/15] xen/riscv: introduce cmpxchg.h
Date: Mon,  6 May 2024 12:15:15 +0200
Message-ID: <37eb9cd10cc10b84a3314039f6de6563aaabe793.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header was taken from Linux kernl 6.4.0-rc1.

Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic
  access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use them
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )

Implementation of 4- and 8-byte cases were updated according to the spec:
```
              ....
Linux Construct         RVWMO AMO Mapping
    ...
atomic <op>             amo<op>.{w|d}.aqrl
Linux Construct         RVWMO LR/SC Mapping
    ...
atomic <op>             loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop

Table A.5: Mappings from Linux memory primitives to RISC-V primitives

```

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
          setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V9:
 - update return type of __bad_xchg();
 - update the comment above __bad_cmpxchg();
 - update the default case inside __xchg() to be aligned with similar default
   case in __cmpxchg().
---
Changes in V8:
 - use __bad_{xchg,cmpxch}(ptr,size) insetead of STATIC_ASSERT_UNREACHABLE() to
   make this patch be independent from the macros that haven't been committed yet
   and may never be.
---
Changes in V7:
 - replace __*() -> _*() in cmpxchg.h
 - add () around ptr in _amoswap_generic(), emulate_xchg_1_2()
 - fix typos
 - code style fixes.
 - refactor emulate_xcgh_1_2():
   - add parentheses for new argument.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - s/CONFIG_32BIT/CONFIG_RISCV_32
 - drop unnecessary parentheses in xchg()
 - drop register in _generic_cmpxchg()
 - refactor and update prototype of _generic_cmpxchg():
   add named operands, return value instead of passing ret as an argument, drop %z and J
   constraints for mask operand as it can't be zero
 - refactor and code style fixes in emulate_cmpxchg_1_2():
   - add explanatory comment for emulate_cmpxchg_1_2().
   - add parentheses for old and new arguments.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - drop unnessary parenthesses in cmpxchg().
 - update the commit message.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
-  update the commit message? ( As before I don't understand this point. Can you give an example of what sort of opcode / instruction is missing?)
 - Code style fixes
 - change sizeof(*ptr) -> sizeof(*(ptr))
 - update operands names and some local variables for macros emulate_xchg_1_2() and emulate_cmpxchg_1_2()
 - drop {cmp}xchg_{relaxed,acquire,release) versions as they aren't needed for Xen
 - update __amoswap_generic() prototype and defintion: drop pre and post barries.
 - update emulate_xchg_1_2() prototype and definion: add lr_sfx, drop pre and post barries.
 - rename __xchg_generic to __xchg(), make __xchg as static inline function to be able to "#ifndef CONFIG_32BIT case 8:... "
---
Changes in V5:
 - update the commit message.
 - drop ALIGN_DOWN().
 - update the definition of emulate_xchg_1_2():
   - lr.d -> lr.w, sc.d -> sc.w.
   - drop ret argument.
   - code style fixes around asm volatile.
   - update prototype.
   - use asm named operands.
   - rename local variables.
   - add comment above the macros
 - update the definition of __xchg_generic:
   - rename to __xchg()
   - transform it to static inline
   - code style fixes around switch()
   - update prototype.
 - redefine cmpxchg()
 - update emulate_cmpxchg_1_2():
   - update prototype
   - update local variables names and usage of them
   - use name asm operands.
   - add comment above the macros
 - drop pre and post, and use .aq,.rl, .aqrl suffixes.
 - drop {cmp}xchg_{relaxed, aquire, release} as they are not used by Xen.
 - drop unnessary details in comment above emulate_cmpxchg_1_2()
---
Changes in V4:
 - Code style fixes.
 - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
   was removed at the end of asm instruction.
 - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
 - drop cmpxcg{32,64}_{local} as they aren't used.
 - introduce generic version of xchg_* and cmpxchg_*.
 - update the commit message.
---
Changes in V3:
 - update the commit message
 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
 - update the comment at the top of the header.
 - change xen/lib.h to xen/bug.h.
 - sort inclusion of headers properly.
---
 xen/arch/riscv/include/asm/cmpxchg.h | 239 +++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/config.h  |   2 +
 2 files changed, 241 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
new file mode 100644
index 0000000000..f5a23d5cbf
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -0,0 +1,239 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2014 Regents of the University of California */
+
+#ifndef _ASM_RISCV_CMPXCHG_H
+#define _ASM_RISCV_CMPXCHG_H
+
+#include <xen/compiler.h>
+#include <xen/lib.h>
+
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#define _amoswap_generic(ptr, new, ret, sfx) \
+    asm volatile ( \
+        " amoswap" sfx " %0, %2, %1" \
+        : "=r" (ret), "+A" (*(ptr)) \
+        : "r" (new) \
+        : "memory" );
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits.
+ * - for 2-byte xchg access the containing word by clearing bit 1.
+ *
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */
+#define emulate_xchg_1_2(ptr, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    unsigned int new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)(ptr) & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
+        "   andn  %[scratch], %[old], %[mask]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        : [old] "=&r" (old), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [new_] "rJ" (new_), [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((old & mask) >> new_val_bit); \
+})
+
+/*
+ * This function doesn't exist, so you'll get a linker error
+ * if something tries to do an invalid xchg().
+ */
+extern unsigned long __bad_xchg(volatile void *ptr, int size);
+
+static always_inline unsigned long __xchg(volatile void *ptr, unsigned long new, int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_xchg_1_2((volatile uint8_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_xchg_1_2((volatile uint16_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 4:
+        _amoswap_generic((volatile uint32_t *)ptr, new, ret, ".w.aqrl");
+        break;
+#ifndef CONFIG_RISCV_32
+    case 8:
+        _amoswap_generic((volatile uint64_t *)ptr, new, ret, ".d.aqrl");
+        break;
+#endif
+    default:
+        return __bad_xchg(ptr, size);
+    }
+
+    return ret;
+}
+
+#define xchg(ptr, x) \
+({ \
+    __typeof__(*(ptr)) n_ = (x); \
+    (__typeof__(*(ptr))) \
+        __xchg((ptr), (unsigned long)n_, sizeof(*(ptr))); \
+})
+
+#define _generic_cmpxchg(ptr, old, new, lr_sfx, sc_sfx) \
+ ({ \
+    unsigned int rc; \
+    unsigned long ret; \
+    unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0); \
+    asm volatile ( \
+        "0: lr" lr_sfx " %[ret], %[ptr_]\n" \
+        "   and  %[ret], %[ret], %[mask]\n" \
+        "   bne  %[ret], %z[old_], 1f\n" \
+        "   sc" sc_sfx " %[rc], %z[new_], %[ptr_]\n" \
+        "   bnez %[rc], 0b\n" \
+        "1:\n" \
+        : [ret] "=&r" (ret), [rc] "=&r" (rc), [ptr_] "+A" (*ptr) \
+        : [old_] "rJ" (old), [new_] "rJ" (new), [mask] "r" (mask)  \
+        : "memory" ); \
+    ret; \
+ })
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits
+ * - for 2-byte xchg ccess the containing word by clearing first bit.
+ * 
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ *
+ * old_val was casted to unsigned long for cmpxchgptr()
+ */
+#define emulate_cmpxchg_1_2(ptr, old, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    uint8_t new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int old_ = (old) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old_val; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)ptr & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[scratch], %[ptr_]\n" \
+        "   and  %[old_val], %[scratch], %[mask]\n" \
+        "   bne  %[old_val], %z[old_], 1f\n" \
+        /* the following line is an equivalent to: \
+         *     scratch = old_val & ~mask; \
+         * And to elimanate one ( likely register ) input it was decided \
+         * to use: \
+         *     scratch = old_val ^ scratch \
+         */ \
+        "   xor  %[scratch], %[old_val], %[scratch]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        "1:\n" \
+        : [old_val] "=&r" (old_val), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [old_] "rJ" (old_), [new_] "rJ" (new_), \
+          [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((unsigned long)old_val >> new_val_bit); \
+})
+
+/*
+ * This function doesn't exist, so you'll get a linker error
+ * if something tries to do an invalid cmpxchg().
+ */
+extern unsigned long __bad_cmpxchg(volatile void *ptr, int size);
+
+/*
+ * Atomic compare and exchange.  Compare OLD with MEM, if identical,
+ * store NEW in MEM.  Return the initial value in MEM.  Success is
+ * indicated by comparing RETURN with OLD.
+ */
+static always_inline unsigned long __cmpxchg(volatile void *ptr,
+                                             unsigned long old,
+                                             unsigned long new,
+                                             int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_cmpxchg_1_2((volatile uint8_t *)ptr, old, new,
+                                  ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_cmpxchg_1_2((volatile uint16_t *)ptr, old, new,
+                                   ".aq", ".aqrl");
+        break;
+    case 4:
+        ret = _generic_cmpxchg((volatile uint32_t *)ptr, old, new,
+                          ".w.aq", ".w.aqrl");
+        break;
+#ifndef CONFIG_32BIT
+    case 8:
+        ret = _generic_cmpxchg((volatile uint64_t *)ptr, old, new,
+                           ".d.aq", ".d.aqrl");
+        break;
+#endif
+    default:
+        return __bad_cmpxchg(ptr, size);
+    }
+
+    return ret;
+}
+
+#define cmpxchg(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr))) \
+    __cmpxchg((ptr), (unsigned long)o_, (unsigned long)n_, \
+              sizeof(*(ptr))); \
+})
+
+#endif /* _ASM_RISCV_CMPXCHG_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index c5f93e6a01..50583aafdc 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -119,6 +119,8 @@
 
 #define BITS_PER_LLONG 64
 
+#define BITS_PER_BYTE 8
+
 /* xen_ulong_t is always 64 bits */
 #define BITS_PER_XEN_ULONG 64
 
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717437.1119701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNj-0003TB-Du; Mon, 06 May 2024 10:15:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717437.1119701; Mon, 06 May 2024 10:15:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNj-0003S8-9Q; Mon, 06 May 2024 10:15:39 +0000
Received: by outflank-mailman (input) for mailman id 717437;
 Mon, 06 May 2024 10:15:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNh-0001cU-3n
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:37 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 945c66a3-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:36 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso227499566b.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:36 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 945c66a3-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990535; x=1715595335; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PDyjivvlEUajhlGlhnu2GxTfqYEGut5Q4+O15ImgrLo=;
        b=Bh4uasLPuei6T+dcn4SErSq8Af3R+8AagnBUAHifuRGlg6HBEDas0NpP3DsFF9yJCs
         DhuvIDQqiAXPmTk74EAEdwcQPxkGyUrNkMiNAgu/5WaF1a2s2zDyAVfgabXuilF4767O
         la8hXo2IP2xeu8tQR2tNv3s2ORwNZpyo6y9ShOkU8GpRZVP8gTwMYLEpHcQFSlsymmBo
         b+mJy+B4uhcSggsS82o4Khq0Sp5lWDC+NRqEHt2bVkxpG25zDqmMfdJHAtGloiJMCimr
         kMyFONHy1S0zpw8XG1nKK16aJoOb/k5wsVsn65+R9do3Wg/ayLlApRt3ux3SUtc5Kq9l
         BHQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990535; x=1715595335;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PDyjivvlEUajhlGlhnu2GxTfqYEGut5Q4+O15ImgrLo=;
        b=D2zpWJUvsHTbJZ2qUIm4OZ088/qfQ1IKTYuORC/AeIaNEuI7t+IvyKmJKpd1VybxTa
         QxPKb+EV10lUDAenjmCWqENRYYyRrlEKARMm+nvcdKo/WTWlUulKmVtTQsRMmU9N0G1V
         OCHoDXMrzii26l/mFTbYu4b7+TUsJt1MOeFwJyIQ92YpKO1ZIYqZ6+Qe7f0Z6FBDIK0M
         03FaCTn80HHRuGw/gFKs4C89ERonbI3AclmuiKfG4omVWf+BDTQafpvrYqHD4ep4eSAI
         ZO2tN/VcQ9hme1etZ/Q9V7j/w9Kik54yA/d+KRBdimkWVfoz+I9QeJy5JsBFc486xgZU
         1LMw==
X-Gm-Message-State: AOJu0Yy/o8XyxT/ma06ZM1MzQTSl1OUDgPyvANUKs0tVGY2fUyDIqgaK
	xsrInnRXrTJ/VAIGw/NAW4QgF1aKn+nQxGHP5DC8DyO4I1btPQnM66Jk0A==
X-Google-Smtp-Source: AGHT+IEY9DIUYyry5vXcx4xo2XAeKDNEjLER6Q69hYI4eQuaLoI8Gil7153BtXz6puW3HgOjx7hvJg==
X-Received: by 2002:a17:907:6d10:b0:a59:bd5b:a0f3 with SMTP id sa16-20020a1709076d1000b00a59bd5ba0f3mr3104781ejc.38.1714990535190;
        Mon, 06 May 2024 03:15:35 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 07/15] xen/riscv: introduce atomic.h
Date: Mon,  6 May 2024 12:15:16 +0200
Message-ID: <1b8d0d3e665adc10222a9e07cedab7bcb40d88f2.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Initially the patch was introduced by Bobby, who takes the header from
Linux kernel.

The following changes were done on top of Bobby's changes:
 - atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
   to use__*xchg_generic()
 - drop casts in write_atomic() as they are unnecessary
 - drop introduction of WRITE_ONCE() and READ_ONCE().
   Xen provides ACCESS_ONCE()
 - remove zero-length array access in read_atomic()
 - drop defines similar to pattern:
   #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
 - drop  atomic##prefix##_{cmp}xchg_{release, aquire, release}() as they
   are not used in Xen.
 - update the defintion of  atomic##prefix##_{cmp}xchg according to
   {cmp}xchg() implementation in Xen.
 - some ATOMIC_OP() macros were updated:
   - drop size argument for ATOMIC_OP which defines atomic##prefix##_xchg()
     and atomic##prefix##_cmpxchg().
   - drop c_op argument for ATOMIC_OPS which defines ATOMIC_OPS(and, and),
     ATOMIC_OPS( or,  or), ATOMIC_OPS(xor, xor), ATOMIC_OPS(add, add, +),
     ATOMIC_OPS(sub, add, -) as c_op is always "+" for them.
   - drop "" from definition of __atomic_{acquire/release"}_fence.

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
              setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Bobby Eshleman <bobbyeshleman@gmail.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V9:
 - update the defintion of write_atomic macros:
   drop the return value as this macros isn't expeceted to return something
   drop unnessary parentheses around p.
 - drop casts inside _add_sized() for ptr variable as they aren't necessary.
---
Changes in V8:
 - ???? add the explanatory comment to _add_sized().
 - drop "" in __atomic_{acquire, release}_fence().
 - code style fixes in atomic##prefix##_##op##_return(): indentation.
 - drop an unary_op argument ("+") for ATOMIC_OPS(and, and), ATOMIC_OPS( or,  or), ATOMIC_OPS(xor, xor)
   and use "+" directly inside definition of ATOMIC_OPS().
 - drop c_op for ATOMIC_OPS(add, add, +) and ATOMIC_OPS(sub, add, -) as it is always "+" for now.
   Just use "+" inside definition of ATOMIC_OPS().
 - drop size argument for ATOMIC_OP() defintions of atomic##prefix##_{xchg,cmpxchg}()
 - update the commit message.
---
Changes in V7:
 - drop relaxed version of atomic ops as they are not used.
 - update the commit message
 - code style fixes
 - refactor functions write_atomic(), add_sized() to be able to use #ifdef CONFIG_RISCV_32 ... #endif
   for {write,read}q().
 - update ATOMIC_OPS to receive unary operator.
 - update the header on top of atomic-ops.h.
 - some minor movements of function inside atomic-ops.h header.
---
Changes in V6:
 - drop atomic##prefix##_{cmp}xchg_{release, aquire, relaxed} as they aren't used
   by Xen
 - code style fixes.
 - %s/__asm__ __volatile__/asm volatile
 - add explanational comments.
 - move inclusion of "#include <asm-generic/atomic-ops.h>" further down in atomic.h
   header.
---
Changes in V5:
 - fence.h changes were moved to separate patch as patches related to io.h and cmpxchg.h,
   which are dependecies for this patch, also needed changes in fence.h
 - remove accessing of zero-length array
 - drops cast in write_atomic()
 - drop introduction of WRITE_ONCE() and READ_ONCE().
 - drop defines similar to pattern #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - Xen code style fixes
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
---
Changes in V4:
 - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
 - drop casts in read_atomic_size(), write_atomic(), add_sized()
 - tabs -> spaces
 - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
   the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
  - update the commit message
  - add SPDX for fence.h
  - code style fixes
  - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
  - re-order the patch
  - merge to this patch fence.h
---
Changes in V2:
 - Change an author of commit. I got this header from Bobby's old repo.
---

 xen/arch/riscv/include/asm/atomic.h  | 280 +++++++++++++++++++++++++++
 xen/include/asm-generic/atomic-ops.h |  97 ++++++++++
 2 files changed, 377 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/include/asm-generic/atomic-ops.h

diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
new file mode 100644
index 0000000000..097e27c51b
--- /dev/null
+++ b/xen/arch/riscv/include/asm/atomic.h
@@ -0,0 +1,280 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Taken and modified from Linux.
+ *
+ * The following changes were done:
+ * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
+ *     to use__*xchg_generic()
+ * - drop casts in write_atomic() as they are unnecessary
+ * - drop introduction of WRITE_ONCE() and READ_ONCE().
+ *   Xen provides ACCESS_ONCE()
+ * - remove zero-length array access in read_atomic()
+ * - drop defines similar to pattern
+ *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
+ * - move not RISC-V specific functions to asm-generic/atomics-ops.h
+ * 
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2024 Vates SAS
+ */
+
+#ifndef _ASM_RISCV_ATOMIC_H
+#define _ASM_RISCV_ATOMIC_H
+
+#include <xen/atomic.h>
+
+#include <asm/cmpxchg.h>
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+void __bad_atomic_size(void);
+
+/*
+ * Legacy from Linux kernel. For some reason they wanted to have ordered
+ * read/write access. Thereby read* is used instead of read*_cpu()
+ */
+static always_inline void read_atomic_size(const volatile void *p,
+                                           void *res,
+                                           unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: *(uint8_t *)res = readb(p); break;
+    case 2: *(uint16_t *)res = readw(p); break;
+    case 4: *(uint32_t *)res = readl(p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: *(uint32_t *)res = readq(p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define read_atomic(p) ({                                   \
+    union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_;   \
+    read_atomic_size(p, x_.c, sizeof(*(p)));                \
+    x_.val;                                                 \
+})
+
+static always_inline void _write_atomic(volatile void *p,
+                                       unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: writeb(x, p); break;
+    case 2: writew(x, p); break;
+    case 4: writel(x, p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: writeq(x, p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define write_atomic(p, x)                              \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _write_atomic(p, x_, sizeof(*(p)));                 \
+})
+
+static always_inline void _add_sized(volatile void *p,
+                                     unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1:
+    {
+        volatile uint8_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+    case 2:
+    {
+        volatile uint16_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+    case 4:
+    {
+        volatile uint32_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+#ifndef CONFIG_RISCV_32
+    case 8:
+    {
+        volatile uint64_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define add_sized(p, x)                                 \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _add_sized((p), x_, sizeof(*(p)));                  \
+})
+
+#define __atomic_acquire_fence() \
+    asm volatile ( RISCV_ACQUIRE_BARRIER ::: "memory" )
+
+#define __atomic_release_fence() \
+    asm volatile ( RISCV_RELEASE_BARRIER ::: "memory" )
+
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set.  These don't return anything, so there's only
+ * one version to worry about.
+ */
+#define ATOMIC_OP(op, asm_op, unary_op, asm_type, c_type, prefix)  \
+static inline                                               \
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{                                                           \
+    asm volatile (                                          \
+        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
+        : "+A" (v->counter)                                 \
+        : "r" (unary_op i)                                  \
+        : "memory" );                                       \
+}                                                           \
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument for ATOMIC_OP isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, unary_op)                    \
+        ATOMIC_OP (op, asm_op, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +)
+ATOMIC_OPS(sub, add, -)
+ATOMIC_OPS(and, and, +)
+ATOMIC_OPS( or,  or, +)
+ATOMIC_OPS(xor, xor, +)
+
+#undef ATOMIC_OP
+#undef ATOMIC_OPS
+
+#include <asm-generic/atomic-ops.h>
+
+/*
+ * Atomic ops that have ordered variant.
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#define ATOMIC_FETCH_OP(op, asm_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{                                                                   \
+    register c_type ret;                                            \
+    asm volatile (                                                  \
+        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (unary_op i)                                          \
+        : "memory" );                                               \
+    return ret;                                                     \
+}
+
+#define ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
+{                                                                       \
+    return atomic##prefix##_fetch_##op(i, v) c_op (unary_op i);         \
+}
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, unary_op)                        \
+        ATOMIC_FETCH_OP( op, asm_op,    unary_op, w, int,   )   \
+        ATOMIC_OP_RETURN(op, asm_op, +, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +)
+ATOMIC_OPS(sub, add, -)
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, asm_op) \
+        ATOMIC_FETCH_OP(op, asm_op, +, w, int,   )
+
+ATOMIC_OPS(and, and)
+ATOMIC_OPS( or,  or)
+ATOMIC_OPS(xor, xor)
+
+#undef ATOMIC_OPS
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+
+/* This is required to provide a full barrier on success. */
+static inline int atomic_add_unless(atomic_t *v, int a, int u)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   beq      %[p],  %[u], 1f\n"
+        "   add      %[rc], %[p], %[a]\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [a] "r" (a), [u] "r" (u)
+        : "memory");
+    return prev;
+}
+
+static inline int atomic_sub_if_positive(atomic_t *v, int offset)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   sub      %[rc], %[p], %[o]\n"
+        "   bltz     %[rc], 1f\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [o] "r" (offset)
+        : "memory" );
+    return prev - offset;
+}
+
+/*
+ * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return.
+ */
+#define ATOMIC_OP(c_t, prefix)                                  \
+static inline                                                   \
+c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
+{                                                               \
+    return __xchg(&v->counter, n, sizeof(c_t));                 \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
+{                                                               \
+    return __cmpxchg(&v->counter, o, n, sizeof(c_t));           \
+}
+
+#define ATOMIC_OPS() \
+    ATOMIC_OP(int,   )
+
+ATOMIC_OPS()
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP
+
+#endif /* _ASM_RISCV_ATOMIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-generic/atomic-ops.h b/xen/include/asm-generic/atomic-ops.h
new file mode 100644
index 0000000000..98dd907942
--- /dev/null
+++ b/xen/include/asm-generic/atomic-ops.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * The header provides default implementations for every xen/atomic.h-provided
+ * forward inline declaration that can be synthesized from other atomic
+ * functions or being created from scratch.
+ */
+#ifndef _ASM_GENERIC_ATOMIC_OPS_H_
+#define _ASM_GENERIC_ATOMIC_OPS_H_
+
+#include <xen/atomic.h>
+#include <xen/lib.h>
+
+#ifndef ATOMIC_READ
+static inline int atomic_read(const atomic_t *v)
+{
+    return ACCESS_ONCE(v->counter);
+}
+#endif
+
+#ifndef _ATOMIC_READ
+static inline int _atomic_read(atomic_t v)
+{
+    return v.counter;
+}
+#endif
+
+#ifndef ATOMIC_SET
+static inline void atomic_set(atomic_t *v, int i)
+{
+    ACCESS_ONCE(v->counter) = i;
+}
+#endif
+
+#ifndef _ATOMIC_SET
+static inline void _atomic_set(atomic_t *v, int i)
+{
+    v->counter = i;
+}
+#endif
+
+#ifndef ATOMIC_SUB_AND_TEST
+static inline int atomic_sub_and_test(int i, atomic_t *v)
+{
+    return atomic_sub_return(i, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC_AND_TEST
+static inline int atomic_inc_and_test(atomic_t *v)
+{
+    return atomic_add_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC
+static inline void atomic_inc(atomic_t *v)
+{
+    atomic_add(1, v);
+}
+#endif
+
+#ifndef ATOMIC_INC_RETURN
+static inline int atomic_inc_return(atomic_t *v)
+{
+    return atomic_add_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC
+static inline void atomic_dec(atomic_t *v)
+{
+    atomic_sub(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_RETURN
+static inline int atomic_dec_return(atomic_t *v)
+{
+    return atomic_sub_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_AND_TEST
+static inline int atomic_dec_and_test(atomic_t *v)
+{
+    return atomic_sub_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_ADD_NEGATIVE
+static inline int atomic_add_negative(int i, atomic_t *v)
+{
+    return atomic_add_return(i, v) < 0;
+}
+#endif
+
+#endif /* _ASM_GENERIC_ATOMIC_OPS_H_ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717438.1119708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNk-0003Xf-1j; Mon, 06 May 2024 10:15:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717438.1119708; Mon, 06 May 2024 10:15:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNj-0003XA-Kq; Mon, 06 May 2024 10:15:39 +0000
Received: by outflank-mailman (input) for mailman id 717438;
 Mon, 06 May 2024 10:15:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNh-0001cU-Mx
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:37 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 94dcaa4f-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:36 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a64db066so393437066b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:36 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94dcaa4f-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990536; x=1715595336; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Jf9UofV0vIWQlCtbQGQy8Ck6PFeBmxijL+bZKvWBsVk=;
        b=ghCWMiuN2+grDHY8T2rXY6Sdzykafex+KcV1GwX5FneyOhP2ApULZUe6KJ/RzcUwZ1
         5530FGkYHvy2e/kxEq+Rez/h+numDfNSnRUa8beCQC9vDWtIWakD4ok2J8rzx9qFTyoG
         LSgiXiQWV11eJHPb5cAnHcQQ7k0ZZr+FpykTSYPQadNQhXTT5LIUN9PKYTevK+B3VzPB
         N9JmDQ3JywbiAZFby2dMmGe7JYoXa9k4Q23AFKqLtSF60L3teRxUAPUWULKoBUvwdhsv
         odaT/Sj1a8fRkifN8IkbUrvCXbP9JlYEobgLWERja3Xk8zcDo3kpPHPfK5YcmVYKwv2I
         5b0w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990536; x=1715595336;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Jf9UofV0vIWQlCtbQGQy8Ck6PFeBmxijL+bZKvWBsVk=;
        b=n8+iT2q9tiZJbOTu7Hr/80GmT+N7lE6ZObPW0xUb3sbDiTb1tzDBXVlSwWAE8sKLec
         No1Gj/kqkIDKJXxm3XZTAuvrnJNka+vCuXxwcQDr+v6dtbhFzgqILutOo0eZHqX2yRpD
         /MoXpOdayG+85sNUTzboIdtkEniLSv251/mTrm5KO7ZWA8qtFkxJDS3U76IbVYv5qsd0
         MQDyiRuBnS/bRXQKeg6sgjhUX3/jJyFqrWMUqZp8FvNCsHnAWtwIY15RcGuJ9n566clr
         uH0yp+TOdwUHM5uBugkosmQrzBKGRMSXAYdjpw9olXAeLToARW/Q42vdk0XedrTwPFbi
         61CQ==
X-Gm-Message-State: AOJu0YwO94albt9tLbUg0+BXDAw8bHsW0cDcIHBav9wgX2PFh/WGjxsd
	6JzXzEMil3U6vmKwKI9etna2mMZ2AeyR5JHIgK+VRB677BAbIi6vVbD/zg==
X-Google-Smtp-Source: AGHT+IHxi8paI+7rmQfjcJQcWkUjlIL7zhwGgI71wFUh0EOdA+5cg0YJ6HlMYOOi1/TJrC6dxQ+91g==
X-Received: by 2002:a17:906:b08b:b0:a59:a3ef:21f9 with SMTP id x11-20020a170906b08b00b00a59a3ef21f9mr3956496ejy.52.1714990536058;
        Mon, 06 May 2024 03:15:36 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v9 08/15] xen/riscv: introduce monitor.h
Date: Mon,  6 May 2024 12:15:17 +0200
Message-ID: <e5fbacd0a8ef222087dbe969a912dec0e2bc3af9.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4-V9:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - new patch.
---
 xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/monitor.h

diff --git a/xen/arch/riscv/include/asm/monitor.h b/xen/arch/riscv/include/asm/monitor.h
new file mode 100644
index 0000000000..f4fe2c0690
--- /dev/null
+++ b/xen/arch/riscv/include/asm/monitor.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_MONITOR_H__
+#define __ASM_RISCV_MONITOR_H__
+
+#include <xen/bug.h>
+
+#include <asm-generic/monitor.h>
+
+struct domain;
+
+static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+#endif /* __ASM_RISCV_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717439.1119711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNk-0003f6-Gd; Mon, 06 May 2024 10:15:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717439.1119711; Mon, 06 May 2024 10:15:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNk-0003ch-2V; Mon, 06 May 2024 10:15:40 +0000
Received: by outflank-mailman (input) for mailman id 717439;
 Mon, 06 May 2024 10:15:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNj-0001cU-0i
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:39 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 95e7b675-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:38 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso33038681fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:38 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95e7b675-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990538; x=1715595338; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OJvBUFAikfpz7OucuBwBiLsAEZFo3YhoCLIIuuKYp90=;
        b=V+UxY4opuLh2ap/XGQl1qSvNMNhWvIt7RiErQDbTjmCUkNSgKpZ+GQA+HOj94YIrJU
         IgcxpTNVTD5IJhjtY1WYReNiSHpMipMOaE3q5rou2l+vVDBkS7qSsqnJX1usuytZRSXt
         iky2Qh0mvZrknkmvxBey52Hb1SL+/HOmWJldUDS9n3KQIpJZQGm6OcP2uVZ/SLlQiRsk
         82GXjcTNr6LyVRRI/e6JQjj0GiVtt42lfTSuv5ldUtg0aun7ugUJczMTyz3AegfE50jy
         3w7VHcqBWRZFpKJ95UzMUTiljSW6IL2B8BUD2Z3Qpwax8xi2bEg8ggGZHyLyLI0WBEb5
         RWrg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990538; x=1715595338;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OJvBUFAikfpz7OucuBwBiLsAEZFo3YhoCLIIuuKYp90=;
        b=pogXoWFaZnY2uIp7cnV3axoewwcHx3pioCu1NqDULh3N5ciuYj3K4ZpNLygwARCAn9
         kiLF208VqHIFr/4OslY3fqz9/6jc3fbkhg95fGGOFUaMmm1anM18gYCObD2JFfDbIomE
         jV3uxf1BpcS2tnYfoG9GNs4W/JkHkQoSeGv2avUUe83AN60D2Y9gqNgN3C3FCL8sLDaV
         ewbbqIJL11rGtTgIJ6G2dm9cvx1q48VVC2WMEZrqgt+g+KDQhL7L38Nzy2EjGJgH0Yck
         YZ7cy6zectXdKPnpFPbGClZZKyE3xv+OvyQTCyMYMifut3NhN3jWQii5dvFjk/ip9Syu
         Exeg==
X-Gm-Message-State: AOJu0YxUCWGDtdA5KecLp1HRZLHJSunGLSAYofSwkkeM3JSzmCT4NdRa
	YP/SpooXQIK7PlGgCAkMhb+NwurVH1C5M9ObkwkzUg6cPzJ7g+A1u5sppg==
X-Google-Smtp-Source: AGHT+IFeV2fdGTFPUqK1aGKrd8GUTySyeD0VJX0V4TFsFPrwd6qKIcK4SgxWip01sNT5PTe+gUQbWA==
X-Received: by 2002:a2e:9482:0:b0:2e1:d48e:d5b3 with SMTP id c2-20020a2e9482000000b002e1d48ed5b3mr7502821ljh.20.1714990537078;
        Mon, 06 May 2024 03:15:37 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 09/15] xen/riscv: add definition of __read_mostly
Date: Mon,  6 May 2024 12:15:18 +0200
Message-ID: <7fa7b60b4b66f2801de28195de8872050643a741.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

Considering that there is still no still final decision regarding patch [2] my suggestion
is to merge RISC-V specific patch and just drop the changes in patch [2].

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V9:
 - Only rebase was done.
---
Change in V8:
 - update the footer after Signed-off.
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717440.1119729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNm-0004DS-IB; Mon, 06 May 2024 10:15:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717440.1119729; Mon, 06 May 2024 10:15:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNm-0004Cq-6Q; Mon, 06 May 2024 10:15:42 +0000
Received: by outflank-mailman (input) for mailman id 717440;
 Mon, 06 May 2024 10:15:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNk-0001n2-GK
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:40 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 95f52bdb-0b91-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:15:38 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso283056066b.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:38 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95f52bdb-0b91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990538; x=1715595338; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=u9aRyETJM++e8QaOVBfsR9DXY+H55bqORXnss0g7U94=;
        b=Vj6N//DgZ6M7A5/U+cAUxc7UatQTZ7mfZg3wrzK8oxmrzK44w0Vp1aVKer3lKePuKZ
         RaLkDamDGoNew10UmxcWb+Ut0kf/kWWFWf2L8zjEG68k+ZdOwWkzkBd61/L67I1jHlLO
         NuVxb0t4as5w0Pz8VTWwxrVX5D9jLfK0nksHqGVemprxpEqr5H/mB1FPrZYceA7/y2UZ
         1IXrU83naBYK6KquA07kxzBmPQY4/5B87s3p0HLjqBvbgBHG2nKsjTGy0GAYhE07qW45
         +f/fr1Sb+SiUe/1XUG+461lTPLaRYiu4Fz1+0asrw33AfCAA1bpOzP9WNT2SAR2sM2Gb
         U2NQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990538; x=1715595338;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=u9aRyETJM++e8QaOVBfsR9DXY+H55bqORXnss0g7U94=;
        b=YmWDo07/BQ6pUJaZaXNc46P4z88+rEset4+oGN0fSeShYAIbxDpdfNl8MYaGczGAis
         LUAIzw/5zA7HjAI8e0ItFRoPHzGxzIdXB5QgIyBxURSX9tAIeHcca0TjihhelLmwN7uX
         3c5Uz9O3Qb2RreWVtjZq2DGVp5f8TpWQrRvfxlVmDrcRWkFq2sTGv/1aCRIaYFBQVGE6
         JeP9HoDSkmEj+tML8948S1tZ2pUwJkZaSQwhLAHirzi3t9nz8A6CNDGfR84Kd0QsObmc
         JlLzE8UPAS2N48+4O5GTA9jLIovpqgjQs5dsgXWFr4C3F0ohppCjxGeKQPpiL/T7GalX
         mK/g==
X-Gm-Message-State: AOJu0YyPP9JNV1tHXDggXFZhQcCrfxNa4q3l/KoUmMnGtaXrhwrJ7nRY
	QiaPzPqTapafkwWaEHbnI1gCLsYjIZ/IoezSqWHlXsgexcXlRWZJv7QTnQ==
X-Google-Smtp-Source: AGHT+IHI93x4RqSFxuW56ORAdgjkjFHZj7dEuWqCEvHuMmmbdXeYP0LpN0rimu9dVLlOWXtWCB67jg==
X-Received: by 2002:a17:906:2492:b0:a59:9f4e:4e3d with SMTP id e18-20020a170906249200b00a599f4e4e3dmr4246517ejb.3.1714990537955;
        Mon, 06 May 2024 03:15:37 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 10/15] xen/riscv: add required things to current.h
Date: Mon,  6 May 2024 12:15:19 +0200
Message-ID: <083df7371fa500056fe160261e83b5f89236e3f8.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add minimal requied things to be able to build full Xen.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V9:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - BUG() was changed to BUG_ON("unimplemented");
 - Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - drop a forward declaration of struct vcpu;
 - update guest_cpu_user_regs() macros
 - replace get_processor_id with smp_processor_id
 - update the commit message
 - code style fixes
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/current.h b/xen/arch/riscv/include/asm/current.h
index d84f15dc50..aedb6dc732 100644
--- a/xen/arch/riscv/include/asm/current.h
+++ b/xen/arch/riscv/include/asm/current.h
@@ -3,6 +3,21 @@
 #ifndef __ASM_CURRENT_H
 #define __ASM_CURRENT_H
 
+#include <xen/lib.h>
+#include <xen/percpu.h>
+#include <asm/processor.h>
+
+#ifndef __ASSEMBLY__
+
+/* Which VCPU is "current" on this PCPU. */
+DECLARE_PER_CPU(struct vcpu *, curr_vcpu);
+
+#define current            this_cpu(curr_vcpu)
+#define set_current(vcpu)  do { current = (vcpu); } while (0)
+#define get_cpu_current(cpu)  per_cpu(curr_vcpu, cpu)
+
+#define guest_cpu_user_regs() ({ BUG_ON("unimplemented"); NULL; })
+
 #define switch_stack_and_jump(stack, fn) do {               \
     asm volatile (                                          \
             "mv sp, %0\n"                                   \
@@ -10,4 +25,8 @@
     unreachable();                                          \
 } while ( false )
 
+#define get_per_cpu_offset() __per_cpu_offset[smp_processor_id()]
+
+#endif /* __ASSEMBLY__ */
+
 #endif /* __ASM_CURRENT_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717441.1119733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNn-0004Kn-6U; Mon, 06 May 2024 10:15:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717441.1119733; Mon, 06 May 2024 10:15:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNm-0004I6-OC; Mon, 06 May 2024 10:15:42 +0000
Received: by outflank-mailman (input) for mailman id 717441;
 Mon, 06 May 2024 10:15:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNk-0001cU-K0
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:40 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 969ede62-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:39 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59d0810f59so128966066b.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:39 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 969ede62-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990539; x=1715595339; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lk6VYyilFs08cUPdzoh60bmbO+59bOcEn0s9QuoGbXk=;
        b=lRYjD1N/cMmXGWOMFiSHf/X4RIcYw+qxIHGwLchTuEbejRtzCOYoXJzm9bWTMUYr9o
         wulXY4XLdnqlUk08Jmxul1e5Fr3TW+X3GkOlD4Ad/D3dug4Z7kaB8zdjOfa/DHKYLWrx
         Q7Pk7HbBAbUeppNMFGrpKC4fzpVI+TbapDHLrUdnssgJjIDOAThYvuo/4anpWCVbBSiP
         +5z/gMfkqstmNhZW3Vt3V6rSx/bu9jnHh8DhREWfgMD0MM0JferElN41JCzOrzDEhojO
         a2/aM7EHnL+W8qg+O4YAdxzRaJuDJv4VDFfjo5mNEw0B1P0Y6lWW6ycQlVwwoZr4h/rQ
         +ZaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990539; x=1715595339;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=lk6VYyilFs08cUPdzoh60bmbO+59bOcEn0s9QuoGbXk=;
        b=Bd6A3U83bDKXC4JIgj7JS3Q5kq/NwB2pVl3jfSIQyihNQaLS7Csed/326/yr5TsgDH
         rvqMDkZaDePEerSHNDvGYpVze97sKA5iypNxfcLrfNAKcNPgEDy31dNkuQwWQI1uT0c+
         3htN3fiCqYmIUvHBUtXdQK6zlhsaJCJXAewHGEVShuZhIHe1qpkgS9kZNWn49PrF+yLC
         lOqm979n1RymBlABAr25dU1jcC/gpAkUrqc8w+ttd12U8Z+W6XHo2D6udTAya/FKInwX
         W9niuvZJgWcys51FPoAVMVK8lOW8BAh79+y+uTxDcQSFn6kdFg8hWGVZIeVJ8hclaqDI
         JZIg==
X-Gm-Message-State: AOJu0YyXSKjgV2i1voVUy6JkaD94W255O4YiZu5Ci/lIxd7S6lbq6ANc
	EOrSk06g6r5SGpYEBaRZGbImKCqM30RaXFO3TBbzPZw++C49EYxScjF0NQ==
X-Google-Smtp-Source: AGHT+IGXXf4PJZWtyl5eyDzLpRVjcbHq+4Z1XT5ssfDhCAQYWWpoFfUnmSbW6OyckZh0vFg22++kxQ==
X-Received: by 2002:a17:907:3f18:b0:a59:cbff:a05 with SMTP id hq24-20020a1709073f1800b00a59cbff0a05mr2228667ejc.20.1714990538989;
        Mon, 06 May 2024 03:15:38 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 11/15] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Mon,  6 May 2024 12:15:20 +0200
Message-ID: <4240462c3d34afb62df56859a06a2199647a1f5a.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V8-V9:
 - Nothing changed only rebase.
---
Changes in V7:
 - update argument type of maddr_to_virt() function: unsigned long -> paddr_t
 - rename argument of PFN_ORDER(): pfn -> pg.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V6:
 - drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
 - parenthesize va in definition of vmap_to_mfn().
 - Code style fixes.
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 242 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..cc4a07a71c 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,246 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717442.1119743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNp-0004oN-2j; Mon, 06 May 2024 10:15:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717442.1119743; Mon, 06 May 2024 10:15:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNo-0004lC-Lq; Mon, 06 May 2024 10:15:44 +0000
Received: by outflank-mailman (input) for mailman id 717442;
 Mon, 06 May 2024 10:15:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNm-0001n2-MQ
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:42 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 975989c0-0b91-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:15:41 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59b58fe083so183322666b.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:41 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 975989c0-0b91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990540; x=1715595340; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jJEKHrBIx5KBw7idMYpZ76iQLa/FXNMtEBbm85wlAkQ=;
        b=ffGCpWoMoHPoAbf2L/IykxhXjay17Xkc4J9iApc8O9jhalQZNssqdAS+yL8cVsfG1b
         F8tzZNaS0Qdyk5nefMyNQ5SZBCwTc69uJCMRH6wd7LJHQ87ULuri5wkn5ZOvRAAaPffJ
         Y4V91ijdlOlqDaTEYEWGi/DSpESBmihsZIsfgEywEGpCQHq1i+hMkFvvAE43Ea9xXLkX
         +uZaEQcCnU2JUBIi6iDDSRekka8YoV6iLXMY0ZH2VL1kpCMekLc7D++3sN+3wAr4eFr2
         pPuUEHfip3YcX45q3kR014Fb5oL9uRYkKmaXpcq3CZNFkFbJOj/7vVYAvZk/gcGg6KKE
         b3pg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990540; x=1715595340;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jJEKHrBIx5KBw7idMYpZ76iQLa/FXNMtEBbm85wlAkQ=;
        b=nsW7NzutQknIrxYFpE38ZV3TyW+7+z2ZYIpoMB35sfqPcxjV43n6s37D6shORSy7c0
         rn5yJmFsFoGJUbHkmYNySugOpfsBEWD20aBUCEDODx/AiBzcPOGf9+Tmk6sBW/UiZd5d
         QYZUIa9aXqjk1vw8U7/fE9mFgM0gDjWObSZzj+ZF2w9Y9PAbHeMP+3GvwuFTk6BHawI3
         PgW3/iXN40kfx1b4Jv8DYRvvrmWixkejb4umhlPASaERWpO3rXaKJsNnRGa04yYiL2zW
         R/yIs+fdCrQ0BlalbMSM+rhB8QlsB1ZkD1ywTiram1cZvdPPa16zIA2YijvftETFxOSL
         Vl8A==
X-Gm-Message-State: AOJu0Yyhhc+Hu9g/2DxGw1OWW0hjAAbsezTavrdHbEt+yfSNIoEsVpLq
	DFZdkAsMd/CYPzFQoCehjji50sZURiEApf/66pGElWLy6EHd1/ps5iTxQw==
X-Google-Smtp-Source: AGHT+IGUvhxo85V6byRYRu8Q9mthG/6eb+72vYDKutlqNDRpiMEt+cDk1cNzkb80SvbNx0GUFzt0Ew==
X-Received: by 2002:a17:906:b52:b0:a59:a857:85ce with SMTP id v18-20020a1709060b5200b00a59a85785cemr3409654ejg.52.1714990539813;
        Mon, 06 May 2024 03:15:39 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v9 12/15] xen/riscv: introduce vm_event_*() functions
Date: Mon,  6 May 2024 12:15:21 +0200
Message-ID: <e157c2c6895f5b4939d461920d4e10dab9563312.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V9:
 - Only rebase was done.
---
Changes in V4:
  - New patch.
---
 xen/arch/riscv/Makefile   |  1 +
 xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 xen/arch/riscv/vm_event.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 2fefe14e7c..1ed1a8369b 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
 obj-y += traps.o
+obj-y += vm_event.o
 
 $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
diff --git a/xen/arch/riscv/vm_event.c b/xen/arch/riscv/vm_event.c
new file mode 100644
index 0000000000..bb1fc73bc1
--- /dev/null
+++ b/xen/arch/riscv/vm_event.c
@@ -0,0 +1,19 @@
+#include <xen/bug.h>
+
+struct vm_event_st;
+struct vcpu;
+
+void vm_event_fill_regs(struct vm_event_st *req)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_set_registers(struct vcpu *v, struct vm_event_st *rsp)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_monitor_next_interrupt(struct vcpu *v)
+{
+    /* Not supported on RISCV. */
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717443.1119753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNq-00059X-NX; Mon, 06 May 2024 10:15:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717443.1119753; Mon, 06 May 2024 10:15:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNq-00057F-DN; Mon, 06 May 2024 10:15:46 +0000
Received: by outflank-mailman (input) for mailman id 717443;
 Mon, 06 May 2024 10:15:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNo-0001n2-Bg
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:44 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97f52fc1-0b91-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:15:42 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so350968166b.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:42 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97f52fc1-0b91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990541; x=1715595341; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZZjuT02oKj/RNzIaHenTN7083fSBrh0r9COrszoJb68=;
        b=A5Hz11eOC3hQKGxNNOjRL7rHfMa7POf+IQg/cdqc8uRNmxe33ta4hgqtm0fXXKQjC9
         a8dIzpe4+wHKkFArW+uo1ucmjplILbQ+3PBGdtotKjvYBw1DFH2VOnSK1UYIkQqZvdGx
         MKg+eV9A+7aXKhfpMUHMsum3h/EfoLCCUnFnkqUtxE5JOc9+xQ2qGVk8v9a+DDmdAfKN
         9WloPpLhLOhrxN8N5gw7MYw+0PcHPufBi+0MTv5O5fjzCDVjwknZHCgiJ5IC4uaYoCYJ
         w6aFOBeh2OQs+KnIajNWvQmgri0Bm6tHMMLZE4r0Oqh3rY2tPcJOGJM8kYfiDOdTaw6d
         3DMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990541; x=1715595341;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ZZjuT02oKj/RNzIaHenTN7083fSBrh0r9COrszoJb68=;
        b=nKPsZUk06hmfxb4N3qKB0wyNp+slp3xVOfZoQd2/8OdJrk00hPQ9UdIvVhxy1AECqb
         9P5SsA9irhjrJYzW04F/38Ubp+gqsRAbAkcYOV0ZVt7ZFKX7sn0Q4LAJIkBBPUWgCU/+
         0J0ihbYQOdMOF3Zsh7e76SnigqcrxN9Sx2GlSebNOSivKpKSv0I3Lkt9GstSE04tJcCJ
         i5qe+Q4/hX5Kye3zv3Hu+sI7q5ZJTSUQG6rZKqspeXPKEBFD89OKNxjv5LRJW1hXPiE4
         UdAzQh4MrKD4gpZUA1uX7wTY4XV4o4UhsEhzSF1C1j3FcTGiuaSzPgyNdqizdEbXNamX
         oVrA==
X-Gm-Message-State: AOJu0YxJHRz8O1TzQQOxEdGUh198zmI2Gis2Q/rt/Tx5Bib3DVSRdpnB
	7tGTi60NM4qlBb7cAWNxdXVS0EVKzjmvfbYBFj04mjnY1+eXzZNPWFVxcg==
X-Google-Smtp-Source: AGHT+IHluUVNcXc+vVe0Wb82E8iX3zqWdREVmUtANne8QryX8Kmk4y6OpJgoXRaajh9/5VbBYusxWQ==
X-Received: by 2002:a17:906:81d8:b0:a59:cd46:fe89 with SMTP id e24-20020a17090681d800b00a59cd46fe89mr1486001ejx.59.1714990541330;
        Mon, 06 May 2024 03:15:41 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 13/15] xen/riscv: add minimal amount of stubs to build full Xen
Date: Mon,  6 May 2024 12:15:22 +0200
Message-ID: <5a3c661c6d8623531fee9e13b32f8d38a4d390a8.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7-V9:
 - Only rebase was done.
---
Changes in V6:
 - update the commit in stubs.c around /* ... common/irq.c ... */
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 439 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 523 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..8285bcffef
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, but common/irq.c isn't
+ * built for now. These changes will be removed there when common/irq.c is
+ * ready.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:15:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:15:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717444.1119763 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNs-0005O9-9M; Mon, 06 May 2024 10:15:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717444.1119763; Mon, 06 May 2024 10:15:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vNr-0005Ln-Fm; Mon, 06 May 2024 10:15:47 +0000
Received: by outflank-mailman (input) for mailman id 717444;
 Mon, 06 May 2024 10:15:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNp-0001n2-HI
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:45 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 990e55ba-0b91-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:15:43 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso33040831fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:44 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 990e55ba-0b91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990543; x=1715595343; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UGf7j+VFITNzuSDPQfzUoszvG5jtrP41Tc7vySyHgmQ=;
        b=EO5HQWFRSu15BMkt3HHRKJpyXjfLubCr5uhoRzzmlTMtm9rX2UEETm0uLVTSGxPBbk
         wvrSM5NxBTd8CcnSvwwmeM5jFTnsvvdY0QKLzgFDrvueYYpONsj118rYakQM6laBXG2v
         jtpdQVL3QWXyuaN1o0v37p94q8btB2vzeajl1Cl/0y18y7DLTmufTSicHYJtXpPWXdCT
         KNzHUC8yyiM3VXxyrs4LsjRIYRxyz/C07fX3BnI8+dpG/NIN3tdmQMf5acqjk5BRdVKG
         lRH1o7yWobdPlj8wmXfqzdXLYTyNbgodz6nZoRkOL2/pZ24LfnQ3pQSpcZFrmK3t71Vx
         P4sA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990543; x=1715595343;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UGf7j+VFITNzuSDPQfzUoszvG5jtrP41Tc7vySyHgmQ=;
        b=E5OVDVmZL6bDbZo2/B5qUJEPXuern3dLRqav9JLnU/LLc8SwkCyZLf0znTAiiqxcE9
         GTZiEWiXjTsy+Cd4YXNzG9QWjiy/0xvYUeamjXO4xh42fO/tllEI5NfMZeBvbyuxyczn
         CDsa5H30Rqgh3zTq8OapEer1EvZop/V9lRiV2hoIjIjGhiQhW+dGPyjE0/2wnX0JUNPq
         mRGoLOsVYyUWSaMJBF/nt2TBHN3VpjaVpYzVMd/irriDe2JagjuuLvj+oG3khQb1KlV4
         cX9ktZn7ZZ+q5HzTQRVO/ZZW2Z/5cOM7LUcsmyzEa9NwTvqa4Kw9zioxaLlYq15FRwL3
         NeUQ==
X-Gm-Message-State: AOJu0YwiFElLg6jlMcEVQsYgSlzOBlw2bdL+jE27ygGKEUQi5audyUqb
	cWYLLzPrJJEbTJGH2yhds7EKXYrZwxesxjrcFfRTd7Jk3ItZSboYM9ztlw==
X-Google-Smtp-Source: AGHT+IHx8T/bh0+URP+QGniO8cp/ei7wx+5dDWfqjBQz4Hqg6uZTjSu7pleLSaZVT/B2pSCt9fg4eg==
X-Received: by 2002:a05:6512:6d5:b0:51f:6ab6:9e5b with SMTP id u21-20020a05651206d500b0051f6ab69e5bmr11075796lff.36.1714990543163;
        Mon, 06 May 2024 03:15:43 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 15/15] xen/README: add compiler and binutils versions for RISC-V64
Date: Mon,  6 May 2024 12:15:24 +0200
Message-ID: <0a110c3f0d31c86147797c4d41c2d63a32127e04.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V9:
 - Nothing changed. Only rebase.
---
 Changes in V6:
  - update the message in README.
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README b/README
index c8a108449e..30da5ff9c0 100644
--- a/README
+++ b/README
@@ -48,6 +48,10 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+          Older GCC and GNU Binutils would work, but this is not a guarantee.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:21:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717468.1119781 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vTB-0002ZD-De; Mon, 06 May 2024 10:21:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717468.1119781; Mon, 06 May 2024 10:21:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vTB-0002Z6-Ag; Mon, 06 May 2024 10:21:17 +0000
Received: by outflank-mailman (input) for mailman id 717468;
 Mon, 06 May 2024 10:21:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3vT9-0002Z0-UO
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:21:15 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5d95d9e6-0b92-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:21:13 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-34e663aa217so1434917f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:21:13 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 h12-20020adff4cc000000b0034dc2f96ed9sm10310991wrp.89.2024.05.06.03.21.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 03:21:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d95d9e6-0b92-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714990873; x=1715595673; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zRWhy3FvEBWh75I4fZPH9hqvwRcqSer8jaFzZFu5XCA=;
        b=axXKjUgHBkWOWlI3QhaMBFoqahKJSIjHZ/b7b4d4yHkMnDeC3VqrDP6E3lWSC8uHmh
         S+N0Aw9IwgirmzYskZuiQ1EJDNco1fLwSNP0WBXLvE2ZeKE1+25lrWMoeZ8vBpKaqdnz
         /sOzwx/Ga1jxEk7UXmhct6jukVPlJNiCChyUdNnhANPbhCJUyGjT2fnlZBEbCPc87wVc
         BjPbNq3RPkynvRi1GUOF4DYOnqXqV4hUr10Jvd5mAbLYfaoiG6XD/tR1GY/3bACX/Mau
         Vw7ndu9qKjhUM72JN1/RbQWJpA5Da/SMI3ER3eri6U9/KaXlkEjX28R3aU4VkuLPGrgC
         7Jfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990873; x=1715595673;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zRWhy3FvEBWh75I4fZPH9hqvwRcqSer8jaFzZFu5XCA=;
        b=pHq/30A6hVpVsMKBY7ZiY1pDhQOmmyN8Ixkf7BUQqIfN+ytFYbM3+oFC+OJRaFghET
         uaXwN6/9i+msbw+xUFw8v808FivmiXbV7cT/m1p3w8jXcou5nr35tz3W1oeRwgOVfZL1
         Ta9XauutbP0f1Q/qeFSVQaiC4F5Eo30p69SdGAQWJFDRq+tCIaCh4ViTDc8beT2SkCsA
         qeKXD7YN7gRf/WsALZ6EP89xDXcD+WrUdqEBEhsIn1g6G56H//5nuUR77olTQ6lEFOMK
         EVKnhPwTqmThxCo5e5NpCSkrUlRwA4s8o951zdUV/lw6b0vJx/mMML4QWqCc+kO1c/r9
         tiKQ==
X-Forwarded-Encrypted: i=1; AJvYcCViQ4XohFvC+85ul2546CwRus5gdU1ds3D+VBdKjwJVvMoKdOBYy2MO8W453rDV7l12pIntImiXHR2eQoNXOEG0BD6n56E9fxICixHhViM=
X-Gm-Message-State: AOJu0YxaxGq3i9aJ0ey4tHlK4dZngFdmMuv2b6CW6xb0NpBETvgoz9Xh
	d8ExGL/x/crbyPnjaboWJVIviQRpFeWtVw5aAo07suYAif15Q6WD32OuoIbBw8c=
X-Google-Smtp-Source: AGHT+IGt5MWUOqpRv+5GL6b8+MuIpkcuA5UrQZ8tXaEGWG6Fd1RBPPz/jWQxP3k8yGeKShKeyH9v4A==
X-Received: by 2002:a5d:6449:0:b0:34c:97e7:af6c with SMTP id d9-20020a5d6449000000b0034c97e7af6cmr6958400wrw.59.1714990873099;
        Mon, 06 May 2024 03:21:13 -0700 (PDT)
Message-ID: <43dbec7b-1a84-4d7d-a151-65a348277961@linaro.org>
Date: Mon, 6 May 2024 12:21:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/17] xen: mapcache: Break out
 xen_invalidate_map_cache_single()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-10-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-10-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Break out xen_invalidate_map_cache_single().
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 25 +++++++++++++++----------
>   1 file changed, 15 insertions(+), 10 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:21:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:21:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717470.1119790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vTa-0002zZ-Jx; Mon, 06 May 2024 10:21:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717470.1119790; Mon, 06 May 2024 10:21:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vTa-0002zS-HJ; Mon, 06 May 2024 10:21:42 +0000
Received: by outflank-mailman (input) for mailman id 717470;
 Mon, 06 May 2024 10:21:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3vTY-0002zG-VO
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:21:40 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d581570-0b92-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:21:40 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41c7ac71996so14461645e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:21:40 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 n5-20020a5d5985000000b0034e0ff3e6dasm10304139wri.93.2024.05.06.03.21.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 03:21:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d581570-0b92-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714990899; x=1715595699; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3R19P3OyRCxMOQp4ONqe+KyB+udrpxcbfQ17ucAD1/A=;
        b=gPYLtaYlVWjvt4jL5kLIridf8NjLATKJuf++4vpEHEDMjbcZeo3ba36hQp8f2oZ17K
         bWnwXWtn4zwaa3oGbWr8SMjwXXfzE8aELropghdfNTMZITWmN0yXGv3T+0Clg+bFFUgF
         l9U+QjHBy84hu5gJqA+XdY9YefkdwqmcHeH3Lkt4zKnHKNbgU6PRvDcFPfRWoPqXbPrD
         AtlrPPnBWwgv+PW6BJX/9WrYvoyD2Lp7qU771kltlbkDTGtq7n1RLuQC7x5wGaPQtrjA
         T8Wjwe92TcrG0QdOKUUlE/8qZmEkLYPBZJKj3SEr6ZXoY+1tIYBf0YFTnLf11c8StEQR
         PxMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990899; x=1715595699;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3R19P3OyRCxMOQp4ONqe+KyB+udrpxcbfQ17ucAD1/A=;
        b=wxpO18CXOKYrJU0NcXgRbx2SjJZMbsPjJpLc3j148RayBXsoN9tl+7ckaKH4ytY6fZ
         6UVWf8DEt/m+3v0Icia03kWs34fB89/WUYDStNmjSKdaIthPyFbOyDvtFdEGBp48BxB8
         9O3HPnWEZwUSTKqpIib2OmIsMT9RueqolUO74C9Bfp9PalpJSbbuas0bGXgdm3iuIAzA
         e07EKAeXfW+XQaf0f/Kg8UGWkWhcKeO52Jd5Xlp+H19aRA/eWTJ8+6zBJKJGTcC4p3hQ
         d3P9j/C1AHejeI1BKA/A98I7YMjvOHICqCVhUQt3AluTSfjFWJGojbq9t4JFUGaCDNTM
         qqVw==
X-Forwarded-Encrypted: i=1; AJvYcCVnCGsbYebFQmVvZX4YUZ1QLRHnfCi2WUyFQ4mtI8j+qdHHT4lre7zBAxxYiAIiN/R96HgPldw30x04MDaUqbs54d8cXvRRb60r9Xw8iJM=
X-Gm-Message-State: AOJu0YwiYDkYqAZJjMoRrD3wuOUJIXwBEZ6eHr/8gDeiq93udrx4sZHq
	L0h+OtvyzpiP8p5snY/YTKjqq9VCqZ+/2HkA9d03WC+huoLRX3huLuNpnjRkvmc=
X-Google-Smtp-Source: AGHT+IF1SoDbHOXgpG8N/21hSx48uhBBIxbLFUYZmZQ6dqIA0cRCB6rMozrTycEpVZUACIKnO+CDuw==
X-Received: by 2002:a5d:4444:0:b0:34d:b284:9540 with SMTP id x4-20020a5d4444000000b0034db2849540mr6615328wrr.4.1714990899512;
        Mon, 06 May 2024 03:21:39 -0700 (PDT)
Message-ID: <b386a0b6-9cf0-4503-85be-f7b48d90d381@linaro.org>
Date: Mon, 6 May 2024 12:21:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/17] xen: mapcache: Refactor
 xen_replace_cache_entry_unlocked
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-8-edgar.iglesias@gmail.com>
 <alpine.DEB.2.22.394.2405011345140.497719@ubuntu-linux-20-04-desktop>
 <CAJy5ezq2PBXMXGMFORfuDVeC_t4S=9AF6b_pur9kD26fhBPhwA@mail.gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <CAJy5ezq2PBXMXGMFORfuDVeC_t4S=9AF6b_pur9kD26fhBPhwA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 2/5/24 08:32, Edgar E. Iglesias wrote:
> On Wed, May 1, 2024 at 10:46 PM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
>>
>> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
>>> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>>>
>>> Add MapCache argument to xen_replace_cache_entry_unlocked in
>>> preparation for supporting multiple map caches.
>>>
>>> No functional change.
>>>
>>> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>>> ---
>>>   hw/xen/xen-mapcache.c | 8 +++++---
>>>   1 file changed, 5 insertions(+), 3 deletions(-)


>>> -static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
>>> +static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>>> +                                                 hwaddr old_phys_addr,
>>>                                                    hwaddr new_phys_addr,
>>>                                                    hwaddr size)
>>>   {
>>> @@ -578,7 +579,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
>>>           cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
>>>       }
>>>
>>> -    entry = &mapcache->entry[address_index % mapcache->nr_buckets];
>>> +    entry = &mc->entry[address_index % mc->nr_buckets];
>>>       while (entry && !(entry->paddr_index == address_index &&
>>>                         entry->size == cache_size)) {
>>>           entry = entry->next;
>>
>> There is still a global mapcache pointer in use in this function:
>>
>>    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
>>
> 
> 
> Thanks! I had accidentally put the change to use mc in future patches.
> Will fix in v5.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:22:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:22:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717472.1119801 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vU0-0003Uu-TS; Mon, 06 May 2024 10:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717472.1119801; Mon, 06 May 2024 10:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vU0-0003Ul-Pa; Mon, 06 May 2024 10:22:08 +0000
Received: by outflank-mailman (input) for mailman id 717472;
 Mon, 06 May 2024 10:22:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=a0G4=MJ=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s3vTz-0003UC-1m
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:22:07 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c6d330f-0b92-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 12:22:05 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41ecd60bb16so8872715e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:22:05 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.211.4])
 by smtp.gmail.com with ESMTPSA id
 i14-20020a05600c354e00b004169836bf9asm19292525wmq.23.2024.05.06.03.22.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 03:22:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c6d330f-0b92-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1714990925; x=1715595725; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ilbeC+ygufH5V3ZDcZlT+wqSJLkVPE09rEah8GFOVCk=;
        b=xGQNF5z+QfvukIsJsl9Wrei59lt3BJpEKRw1j5JKXUBJ0dG8SEKxSm4W8BOczOHEDT
         G3KOhMRZUBhpkd3fRVxZUCP/1NSlblNGwU+O0wpOcQzHGy165i4WbLnYB/NcMB5iW7vQ
         QMnX5W7/8g3UEMG4VWToeFSB+TlRarXG4InN8XDiq7ZN5l7VsUS0k5pQvIew9TNZDQCq
         TFDyFnyE9x3upB4I5ciEgc9QSZBpQryja9QqJOY5/6bl0IJ9T1Jir20YkNK8GbZoiyYZ
         B7+kOniCY5hSowBllWPX+vet7TUSADvBP7Woq2nEk7i/oxytmgSK7KcqQi2ybV71wtVY
         K8BA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990925; x=1715595725;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ilbeC+ygufH5V3ZDcZlT+wqSJLkVPE09rEah8GFOVCk=;
        b=R/J/5WnBcaevSls1S4dicckayD3f9G7R1Eae7g4WbkeM4LLKV3qVjibYR4k18F1PwD
         aKJaUkVBpBMIXOD7UUMBdE5xLNlW6pP9LNjF3UTzgTqkDmJxuh9b6lTkc/IPwJ40mmTR
         RzkHv3pYT4BqY7g2gV2QDSDb0GXiW4TE+qdSbn3FOixqNrA15dRRxVxwoAaUmnXOeO4x
         bCE9j/qZ5jeF4js49svOjgYJ3AFF/gaAEpKRkoEMwf8DWP1LdW6e5iYo+lqZ4KB6sNY1
         wnaRDp3tm3u2kW2nG3TcH0F7BFXquK6b56kNMNSQAUgJ0S3xTfoL8J1wvcsnZWEMG/a5
         EdBA==
X-Forwarded-Encrypted: i=1; AJvYcCUMF2vJYUhE5PfoYsiyLoh1jB9m/5Ayc43avmWw5JelAIPZu2lKypOzZPYW6DU/PqlexyWweWlhQzXZTw85gTps1N/DeGvGRPJlr4sM0ug=
X-Gm-Message-State: AOJu0YwWzq7GWPEQouskSyuqf51/rAbVvCdVdyO5H716NwoDJDtSMguh
	Paurto3Ut6wj8BZa135REe4Vyyyi0U6V+aKu8ZY+b+ETDtUaTAosCec3lOPMf3Q=
X-Google-Smtp-Source: AGHT+IEz3G/PqHe9VAmMjR3NKpgGTo5GDm13K4iOusz17D1sWq1x/nyWqplNUACy0Iv4lUzLKKVUdA==
X-Received: by 2002:a05:600c:1e09:b0:418:a706:3209 with SMTP id ay9-20020a05600c1e0900b00418a7063209mr10481700wmb.31.1714990924971;
        Mon, 06 May 2024 03:22:04 -0700 (PDT)
Message-ID: <c8194252-5048-4b7c-8890-0f275e61aee2@linaro.org>
Date: Mon, 6 May 2024 12:22:02 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/17] xen: mapcache: Break out
 xen_ram_addr_from_mapcache_single
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-7-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240430164939.925307-7-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 30/4/24 18:49, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Break out xen_ram_addr_from_mapcache_single(), a multi-cache
> aware version of xen_ram_addr_from_mapcache.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>   hw/xen/xen-mapcache.c | 17 +++++++++++------
>   1 file changed, 11 insertions(+), 6 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:25:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:25:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717487.1119811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vXc-0004uJ-CB; Mon, 06 May 2024 10:25:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717487.1119811; Mon, 06 May 2024 10:25:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vXc-0004uC-9B; Mon, 06 May 2024 10:25:52 +0000
Received: by outflank-mailman (input) for mailman id 717487;
 Mon, 06 May 2024 10:25:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loHM=MJ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s3vNn-0001cU-O4
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:15:43 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9887fea6-0b91-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:15:43 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59a0e4b773so386281166b.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:15:43 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 h8-20020a17090634c800b00a59b8e16ac7sm2160242ejb.36.2024.05.06.03.15.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:15:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9887fea6-0b91-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714990542; x=1715595342; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Grm55+UlLdSg8iPa212zyFukbCQoJ6LukCxV4Nbb5CQ=;
        b=D9VF19e1MOcpVmsMRYzgRFRHCFuaMhf8deUyMK2awQfz7pNCYHBbTwdvJ1hU/dqWMM
         tgY90nEPSag52qpdBecUjmOZhyBAIpgvlLLMP3L21ajJywVYvgkClhIjb8/v6Mf8DDcl
         PtMtaIr1DqLQJb77haedvJeZ5QQ9/OyCUQZByYbzmDmUBM3a0nd82Dssg10un4xGDA+L
         znR3R7mKZRnTOBpPNZj68I7NchTXevxoQ4q5djcz4JtOM2wfb7fFDNITQTQ8bd1JDAm5
         dhDbtnTnwyPaDo+255cwAwDXEXXHewcZVdcKKBtBkE21ql9DkBjVpPXkuLLQcpKb0YbM
         zNLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714990542; x=1715595342;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Grm55+UlLdSg8iPa212zyFukbCQoJ6LukCxV4Nbb5CQ=;
        b=Wat/MdCpwzz/xAOx00SMKO36wmkE1Xk+/pIfpC97PkPhPoCpctj7kHEH+hsqWDaUuz
         xsPHzJIfzhdEluFfLjA1sEP4n+A1aaVaCCqeyQ6/vgqx0NXvmHuPFo/LjdzFEOXZcXTq
         VwHLcOgIHWMvTdgA3SVWeqGw7ImVf3FMho/87aGmZFKZSPf6iUbUdrDtibFu6e0k98tp
         fEhCoLzoaRgXJJcOR2xXhWT4DZhY1fxmKjSVthL3XtU5bUXIXIbeTnUOn5W8R5sBUdIY
         QDaZMrUCxitTDpyHzkYzvtjV9Bw6gEduuAI4aD68HETMZo3yymJSVEBVfLumSipJ4qbc
         Rq8Q==
X-Gm-Message-State: AOJu0YyMxpHZqGWpIqJAGHGQNAe8DjdO5tY85/W1DkOWqLPSPluyqaQ7
	SAsG/fsavmoKjiUkZdK9uRWZY1DrwcgRUgyNAJt1iV8joO56pLRzZPVDXg==
X-Google-Smtp-Source: AGHT+IFzLEClSBkXx/oT/A66IDhMwEWVcF6ooUAuAGhrfaOkibPNXs3ZkwMN612FgS3zhQbb1fHJCg==
X-Received: by 2002:a17:907:9493:b0:a59:bae0:b12a with SMTP id dm19-20020a170907949300b00a59bae0b12amr2994421ejc.63.1714990542252;
        Mon, 06 May 2024 03:15:42 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v9 14/15] xen/riscv: enable full Xen build
Date: Mon,  6 May 2024 12:15:23 +0200
Message-ID: <c6c40f3149c0f07d7fb4ba2aa481c6a489f4a2f0.1714988096.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1714988096.git.oleksii.kurochko@gmail.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V9:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---

 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  24 -----
 4 files changed, 15 insertions(+), 197 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8c071aff65..17827c302c 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 8285bcffef..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, but common/irq.c isn't
- * built for now. These changes will be removed there when common/irq.c is
- * ready.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Mon May 06 10:29:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:29:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717512.1119821 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vap-0006LA-TU; Mon, 06 May 2024 10:29:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717512.1119821; Mon, 06 May 2024 10:29:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vap-0006L3-QD; Mon, 06 May 2024 10:29:11 +0000
Received: by outflank-mailman (input) for mailman id 717512;
 Mon, 06 May 2024 10:29:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3vao-0006Ku-8q
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:29:10 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7901f49f-0b93-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:29:09 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-34dc129accaso1386250f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:29:09 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 dn15-20020a0560000c0f00b0034d9012e74bsm10443802wrb.72.2024.05.06.03.29.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:29:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7901f49f-0b93-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714991348; x=1715596148; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=YcGCF7Q0lu5pxj6ACuevUFrYdYThmWpEF7jjA1jhM2E=;
        b=M7AzAPt1VFurnlr69Aw0BBu3EpZDGNn6B4HJCvSHBtaRvdPTB8hJ6QRI9hMYS2oDd0
         aefbqp5X8lmV/gzkCZ3sZfXsZDO3/UZ6scMNE4ydFNp6dEH4tZ7PHysvrP9ceVZkJZ6G
         QGXkxW2jMh6N7p9bsAB3YL6YY+YoRQOx6YEMc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714991348; x=1715596148;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YcGCF7Q0lu5pxj6ACuevUFrYdYThmWpEF7jjA1jhM2E=;
        b=nrXs/EQbCeQhGx4lPHMM+b64ixCAB6u9hyNKMA1tjZCWIkMIz+Mp4RVnRwM5h+GRb8
         ITpyUyAdvaez4TVg94oB7qVjEFz1/hzTFPSqTewxKRaq7unViP/aI4+4aZUTYORyyby1
         7S3q3pksnvlTJ2x6I0UVBocvGOtLzTgeUrF7m5Y7r36M2rquUapkMyKa+AktRBD94/Cz
         v6WrvgQW1JtlDC43WlnVMB5Uuovlx7GAfuJ2FVFh4lr1F6Z5+me4avSSG0ErCrYNJDfE
         eWNufMObtA0cJYIpFlTOCRvSlKGdTPfK5JcYziMtdKq/Oiw/TeIYHmAiYhuBLQSNa9y1
         Vbkg==
X-Gm-Message-State: AOJu0YwxciKFrOefHal5nKHb54i/TNkYxSIXNrRQSWI1KeG5VlsvIdz4
	YrGfmoRZeM3tH0LToJ2wEaiICDONTXiR7iIouDaCkLYBYr1E30V8jH/SrqcmXmk=
X-Google-Smtp-Source: AGHT+IG4kgIUjJxuDfyYFyAfju6Y246/ifMlX/Ke2KT7cRQT4SFAxUB750pQJIK2XOzw60sODbd4ng==
X-Received: by 2002:a5d:67c4:0:b0:34c:c1c:8413 with SMTP id n4-20020a5d67c4000000b0034c0c1c8413mr7162485wrw.58.1714991348495;
        Mon, 06 May 2024 03:29:08 -0700 (PDT)
Date: Mon, 6 May 2024 12:29:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 03/12] VT-d: parse ACPI "SoC Integrated Address
 Translation Cache Reporting Structure"s
Message-ID: <Zjiw89WKvy6vJAPn@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <c70b250d-2ec4-4254-89cf-d3241dac0d35@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <c70b250d-2ec4-4254-89cf-d3241dac0d35@suse.com>

On Thu, Feb 15, 2024 at 11:14:31AM +0100, Jan Beulich wrote:
> This is a prereq to us, in particular, respecting the "ATC required"
> flag.
> 
> Note that ACPI_SATC_ATC_REQUIRED has its #define put in dmar.h, as we
> try to keep actbl*.h in sync what Linux (who in turn inherit from ACPI
> CA) has.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Lovely: On the SPR system with the SATC I tried passing "ats" (the
> "required" flag is clear there), just to then hit "IOMMU#4: QI dev wait
> descriptor taking too long" while setting up Dom0. The 2nd message there
> doesn't ever appear, so the request never completes. Not sure whether
> that's us doing something wrong or the hardware acting up. In the former
> case I'd generally expect an IOMMU fault to be raised, though. FTR same
> on 4.18 with just "VT-d: correct ATS checking for root complex
> integrated devices" backported there.

Great, so we likely have a bug in our ATS implementation?

> 
> Should we check scope entries for appropriate types? (If so, then also
> for e.g. ATSR.)
> ---
> v2: Move error case freeing to acpi_parse_one_satc(). Introduce #define
>     for the flag bit. Style.
> 
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
>  LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
>  static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
>  static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
> +static LIST_HEAD_READ_MOSTLY(acpi_satc_units);

We could even make this one RO after init.

>  
>  static struct acpi_table_header *__read_mostly dmar_table;
>  static int __read_mostly dmar_flags;
> @@ -750,6 +751,93 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
>      return ret;
>  }
>  
> +static int __init register_one_satc(struct acpi_satc_unit *satcu)
> +{
> +    bool ignore = false;
> +    unsigned int i = 0;
> +    int ret = 0;
> +
> +    /* Skip checking if segment is not accessible yet. */
> +    if ( !pci_known_segment(satcu->segment) )
> +        i = UINT_MAX;
> +
> +    for ( ; i < satcu->scope.devices_cnt; i++ )
> +    {
> +        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
> +        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
> +        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
> +
> +        if ( !pci_device_detect(satcu->segment, b, d, f) )
> +        {
> +            dprintk(XENLOG_WARNING VTDPREFIX,
> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
> +                    &PCI_SBDF(satcu->segment, b, d, f));
> +            ignore = true;
> +        }
> +        else
> +        {
> +            ignore = false;
> +            break;
> +        }
> +    }
> +
> +    if ( ignore )
> +    {
> +        dprintk(XENLOG_WARNING VTDPREFIX,
> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable\n",
> +                satcu->segment);

Re the error messages: won't it be better to print them using plain
printk and gate on iommu_verbose being enabled if anything?

It does seem a bit odd that such messages won't be printed when
iommu={debug,verbose} is enabled on the command line.

> +        return 1;
> +    }
> +
> +    if ( iommu_verbose )
> +        printk(VTDPREFIX " ATC required: %d\n", satcu->atc_required);
> +
> +    list_add(&satcu->list, &acpi_satc_units);
> +
> +    return ret;
> +}
> +
> +static int __init
> +acpi_parse_one_satc(const struct acpi_dmar_header *header)
> +{
> +    const struct acpi_dmar_satc *satc =
> +        container_of(header, const struct acpi_dmar_satc, header);
> +    struct acpi_satc_unit *satcu;
> +    const void *dev_scope_start, *dev_scope_end;
> +    int ret = acpi_dmar_check_length(header, sizeof(*satc));
> +
> +    if ( ret )
> +        return ret;
> +
> +    satcu = xzalloc(struct acpi_satc_unit);
> +    if ( !satcu )
> +        return -ENOMEM;
> +
> +    satcu->segment = satc->segment;
> +    satcu->atc_required = satc->flags & ACPI_SATC_ATC_REQUIRED;
> +
> +    dev_scope_start = (const void *)(satc + 1);
> +    dev_scope_end   = (const void *)satc + header->length;

Isn't it enough to just cast to void * and inherit the const from the
left side variable declaration?

You could even initialize dev_scope_{start,end} at definition.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 10:35:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:35:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717532.1119831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vgQ-0000z6-IA; Mon, 06 May 2024 10:34:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717532.1119831; Mon, 06 May 2024 10:34:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vgQ-0000yz-DX; Mon, 06 May 2024 10:34:58 +0000
Received: by outflank-mailman (input) for mailman id 717532;
 Mon, 06 May 2024 10:34:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3vgP-0000yt-IS
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:34:57 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 481639f2-0b94-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:34:56 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-34eb52bfca3so1194741f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:34:56 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 u5-20020a05600c138500b0041ac5f19213sm15647564wmf.8.2024.05.06.03.34.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 03:34:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 481639f2-0b94-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714991696; x=1715596496; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=wiclyWMdqkQ7SzHiYugOC5HvIxzP8WwGlPvhETChz/s=;
        b=rSJGrrh4N59ciftotwTJXvtUOMnPNrC6HOvHd6t5IdhbnHbUiyPFWp+tn/PqLZbSXF
         VKU7Q69lzd765oeC+GYBaT1WTLqhqWoL/94BZCo/GfznlR+nB8JsyGL/R+hcRBoyN9jc
         3relKrE7GhTbRPvA1jdrKOmjc0C/v6jAHt2uU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714991696; x=1715596496;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wiclyWMdqkQ7SzHiYugOC5HvIxzP8WwGlPvhETChz/s=;
        b=LxKjJpsKzLfYqZlgT5crLGAaFexRKoCZbQWh92rAABLXkXtTzUO28c7+czfZzYP38z
         Yd5vihIZ+GMKkQIxfhKSwR2j9r/TFYMkMKkv+osQVhedT3ZDcnXZCToJX+K6iB1nF5Ty
         Fqv5XQWuVFlZAralyEZgI+Pl1gNOnR1+ihP3M9DYL2PpoPYha1RniW4LEjjabGgIZo02
         YzqQlXQifAZgPMGsDbXclW9rZsQIOhMCByRiBCeidwdkPI3TEDlKDYaH8POOaZrd4+Si
         DP/yRdplZ/WSTmiZkcsjsqoCtF1NOjADCpv30k3Y5LUgM5uP7MylRg2677s6gZODXUeM
         S48A==
X-Gm-Message-State: AOJu0YwTgZwJeASe+PCZOPUFqFcxZbnIlRYAiSR8/7x19vTKut6uTsrN
	uukeYmty3UIS6tEM9WNP4KEsYlO40BHRYfvQPDTmA0hPWxk72tqNpTt2CjAt9oc=
X-Google-Smtp-Source: AGHT+IE6jinlnyJfDz8MX4gSaw2yYikTgjDvz4mPmZmg1aFRyCTPcztXI34kcbX1x8CONwCQDeHpig==
X-Received: by 2002:a05:6000:c0c:b0:345:663f:cd79 with SMTP id dn12-20020a0560000c0c00b00345663fcd79mr8358570wrb.14.1714991695977;
        Mon, 06 May 2024 03:34:55 -0700 (PDT)
Date: Mon, 6 May 2024 12:34:54 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, committers@xenproject.org,
	community.manager@xenproject.org, Kelly Choi <kelly.choi@cloud.com>,
	Oleksii <oleksii.kurochko@gmail.com>
Subject: Re: Xen 4.19 release status tracking list [ May ]
Message-ID: <ZjiyTp-1qKEOnGVN@macbook>
References: <32e2d9dddfaa80e8edfaa5a7207c4e7fb7862a45.camel@gmail.com>
 <ZjiiOtvJq1KjXo5F@macbook>
 <a8be9d16-36e1-4528-b14c-42746bae605c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a8be9d16-36e1-4528-b14c-42746bae605c@suse.com>

On Mon, May 06, 2024 at 11:35:00AM +0200, Jan Beulich wrote:
> On 06.05.2024 11:26, Roger Pau Monné wrote:
> > And then some patches that I don't expect to make progress:
> > 
> > x86/shutdown: change default reboot method preference
> > https://lore.kernel.org/xen-devel/20230915074347.94712-1-roger.pau@citrix.com/
> > 
> > x86/time: prefer CMOS over EFI_GET_TIME
> > https://lore.kernel.org/xen-devel/20240315114242.33309-1-roger.pau@citrix.com/
> > 
> > I find it quite funny how everyone complains about all downstreams
> > carrying similar patches to deal with firmware bugs, yet when I
> > attempt to formally post them I get zero interest from such
> > downstreams.  Maybe raising them here will help get some eyes.
> 
> Well, as indicated on the Community Call I've meanwhile proposed a design
> session (which won't really be about design) for Lisbon, as to maintenance
> of our EFI interfacing. I think I had provided feedback on the shutdown
> patch, and I didn't think there was much of a point repeating almost the
> same for the time one.

I know you are likely not OK with those, so I wasn't expecting input
from you.  We had discussions in the past where people complained that
our defaults are not OK because they lead to errors and panics on
production systems, and I'm surprised to find that none seem to have
cared about the above patches that attempt to address at least part of
this.

> As a minor remark on what you say above: It's clearly not "all"
> downstreams, as we don't carry anything similar to either of the two
> patches. But I'm pretty sure you meant "most" or "many" anyway.

Yes, indeed not all.  I don't carry those either on the FreeBSD
packages.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 10:42:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 10:42:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717539.1119842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vn8-0003t9-8k; Mon, 06 May 2024 10:41:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717539.1119842; Mon, 06 May 2024 10:41:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3vn8-0003t2-3u; Mon, 06 May 2024 10:41:54 +0000
Received: by outflank-mailman (input) for mailman id 717539;
 Mon, 06 May 2024 10:41:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3vn6-0003sw-Pp
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 10:41:52 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f3a8b0e-0b95-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 12:41:51 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51f71e4970bso1983753e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 03:41:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hc17-20020a05622a2a1100b00437b4048972sm4984776qtb.18.2024.05.06.03.41.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 03:41:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f3a8b0e-0b95-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714992111; x=1715596911; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WI60SXQx2PFmM3yU66qoV6vyK5R/kzpLS52lw1aCV3s=;
        b=glbtTpcqB+wN6nnENwgo+EgAjWpJNitu8fP4MLSuIsI6AKBcbFWkgxOS0YL9kwkkbv
         IgzIWJQ9ffOCu+OVBzNSCsNeTGkaKQ8HGFqtzRyVGTrtoOCjIgfgLHD8yiA6G4MmuYuz
         FjP1uzjnjvBzAOJBdDfVRKHl9LOxoYgwuUY7IjM1k0VPiyFGWDDMOJV9UZuvIEjmXz9F
         PS5XOM9zKo8wwOAYFv1RAm+N4T0d+uQrzYYsDYn4TWw0rLf5oiQiWlp6l1LfF2/APcYH
         INfEjk6akGOvMyLQT9w8xYEC75qvr1tjaXjsIFFbmnqBM9xKqsvpe3/vULIt5pBjemWA
         Sg+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714992111; x=1715596911;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WI60SXQx2PFmM3yU66qoV6vyK5R/kzpLS52lw1aCV3s=;
        b=dX4ETdBkAxF8NTX6baeIXshtaOdYZbpYHd8a7ptYw90GvJpzDg3FTDxBRUkEuS4Nkq
         8FF9NkD/QYjYNrgalcEoOi/e3SJgKMNlzY5ydrWdpIjfgjVjPvhMg2XcythYUh0HxHIU
         ZgoPSOQX6LWkER1sO2iDa0A6KPAzn7VfyM2w/8eXnf99tu8Chmkv8wUOoBmWmyTgsndw
         +cBP/uRJDfvBZZrIPOsh818yRxYzJg6Cc7nlgwy2HqcO6JctAGkEtZNbZ0impD2eBr8X
         Yi1QwyosVQwwrn+PTHnEaJ1TvG3koxyJbPnScT+hhj4+CH7Z2vu3RGqaWHaGT70YgccP
         KjmQ==
X-Forwarded-Encrypted: i=1; AJvYcCVZ8vwPNJDhAw9F7A1imyID5E4jvROm7eFrWq1dlvntFBF0WVNQ/45YQBfypYXH3cWZMzxzS6t/bAkDbZSJ8n7aG7Y8undA+j8KLEpu55g=
X-Gm-Message-State: AOJu0YzGq7jpz91A0rImkn88mPxPtFwgvshuwZFze2rFDtiOcYe7Eiri
	JRn4ltChr+R0QTvq9MuI58XcSfRrAUvL3yTvh8emizxBfZdmtZY05pPDiCzc6Q==
X-Google-Smtp-Source: AGHT+IEcve25x17bLq2o01bUwZioghN2LiADhk7vN13wnfyoFJVy/ven90nKGG+aR2ROUi1T2tEm3w==
X-Received: by 2002:a19:4311:0:b0:51c:d8f6:4e6f with SMTP id q17-20020a194311000000b0051cd8f64e6fmr7521827lfa.40.1714992110572;
        Mon, 06 May 2024 03:41:50 -0700 (PDT)
Message-ID: <54a52cc8-4c36-4086-a10e-114e9a733f45@suse.com>
Date: Mon, 6 May 2024 12:41:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? 2/2] xen/x86: remove foreign mappings from the
 p2m on teardown
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Community Manager <community.manager@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240430165845.81696-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 18:58, Roger Pau Monne wrote:
> @@ -2695,6 +2691,70 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
>      return rc;
>  }
>  
> +/*
> + * Remove foreign mappings from the p2m, as that drops the page reference taken
> + * when mapped.
> + */
> +int relinquish_p2m_mapping(struct domain *d)
> +{
> +    struct p2m_domain *p2m = p2m_get_hostp2m(d);

Are there any guarantees made anywhere that altp2m-s and nested P2Ms can't
hold foreign mappings? p2m_entry_modify() certainly treats them all the same.

> +    unsigned long gfn = gfn_x(p2m->max_gfn);
> +    int rc = 0;
> +
> +    if ( !paging_mode_translate(d) )
> +        return 0;
> +
> +    BUG_ON(!d->is_dying);
> +
> +    p2m_lock(p2m);
> +
> +    /* Iterate over the whole p2m on debug builds to ensure correctness. */
> +    while ( gfn && (IS_ENABLED(CONFIG_DEBUG) || p2m->nr_foreign) )
> +    {
> +        unsigned int order;
> +        p2m_type_t t;
> +        p2m_access_t a;
> +
> +        _get_gfn_type_access(p2m, _gfn(gfn - 1), &t, &a, 0, &order, 0);
> +        ASSERT(IS_ALIGNED(gfn, 1u << order));

This heavily relies on the sole place where max_gfn is updated being indeed
sufficient.

> +        gfn -= 1 << order;

Please be consistent with the kind of 1 you shift left. Perhaps anyway both
better as 1UL.

> +        if ( t == p2m_map_foreign )
> +        {
> +            ASSERT(p2m->nr_foreign);
> +            ASSERT(order == 0);
> +            /*
> +             * Foreign mappings can only be of order 0, hence there's no need
> +             * to align the gfn to the entry order.  Otherwise we would need to
> +             * adjust gfn to point to the start of the page if order > 0.
> +             */

I'm a little irritated by this comment. Ahead of the enclosing if() you
already rely on (and assert) GFN being suitably aligned.

> +            rc = p2m_set_entry(p2m, _gfn(gfn), INVALID_MFN, order, p2m_invalid,
> +                               p2m->default_access);
> +            if ( rc )
> +            {
> +                printk(XENLOG_ERR
> +                       "%pd: failed to unmap foreign page %" PRI_gfn " order %u error %d\n",
> +                       d, gfn, order, rc);
> +                ASSERT_UNREACHABLE();
> +                break;
> +            }

Together with the updating of ->max_gfn further down, for a release build
this means: A single attempt to clean up the domain would fail when such a
set-entry fails. However, another attempt to clean up despite the earlier
error would then not retry for the failed GFN, but continue one below.
That's unexpected: I'd either see such a domain remain as a zombie forever,
or a best effort continuation of all cleanup right away.

> +        }
> +
> +        if ( !(gfn & 0xfff) && hypercall_preempt_check() )

By going from gfn's low bits you may check way more often than necessary
when encountering large pages.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:02:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:02:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717544.1119850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3w6R-0001oO-Ou; Mon, 06 May 2024 11:01:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717544.1119850; Mon, 06 May 2024 11:01:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3w6R-0001oH-LU; Mon, 06 May 2024 11:01:51 +0000
Received: by outflank-mailman (input) for mailman id 717544;
 Mon, 06 May 2024 11:01:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3w6R-0001oB-CR
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:01:51 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 099cfb21-0b98-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 13:01:49 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d8b2389e73so21787671fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:01:49 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z11-20020a05600c0a0b00b0041c097e20f9sm19322123wmp.25.2024.05.06.04.01.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:01:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 099cfb21-0b98-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714993309; x=1715598109; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vPisyKAC+kDGazaUgedMDr6re5zca91pODfF7/dPcps=;
        b=W0MGuR+0Oj4hdJSiJ45FckPaJRg2/qtwdpxtzoMAQ18rU6lFRmoGHJUL0d0iu+67YV
         PwQxuokomwzPa6XE4vVxVnkCuYx0qdiM8lLGsIJ444HxdhSOFCqRCRlhOiUnLz1CDPxk
         gtrcUbUYNpxNAIKSMaVLXdLDQ1lbCx9pbv5f41sGUn+1fLi1LsPvV+t1UOE0B9iUTG/i
         EKToVlSzrE/2KU5qcw8zrI206MOasx5aojDyD6eKQemIVaPg2MLqe/TfOfsCWXA6DCCH
         LfL6i1m1GibqLllccJf+jznQyXPIEjsByu+evMIHZ+5jWZiY/h5eJjhGIdYt/IvIGo9p
         3+vA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714993309; x=1715598109;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vPisyKAC+kDGazaUgedMDr6re5zca91pODfF7/dPcps=;
        b=SnHRtjm/hPDG16//ZPBU26npq+Im8uFWsag11SF9D85OU3o1rm5JvNGr1quZDs8bI/
         6IiWEMaV/H5tyDPEPO/nNAqDsbDSMeDX1Ol0ORfubw32CWL31vNTj5YWz9vEM0Oq5Y2t
         9pj4K+Rr9qPi2iX4wn4i1OoFZZe11CKD3HxpTiwRjM/dSFj32GsRCS50SckF8Hm2Hqql
         OQKN8ktcOjkEjKTrplOi2MMPsBjzBUnWyYI6WbYRleKdXJthNJyN2P1ETgNRM1Sau2FL
         JpMVPQqrvMVJP7H3629PVZcH+zDV2tuzjROT4vDBCklz4Rr9Y4vZ+0KrYkkSf8Jak4Cx
         2wfw==
X-Gm-Message-State: AOJu0YzSM1Y0jzoKjv8YpVvnMsGVsR1N55aeEWF+1PArGAZojgygeeZ3
	6jtTCi9FowvIYw0WKWfnc64dXpa9N1B/HadYB0k/I37/gNpAbsE74w+6MRsbc2bS7HB+q+QgCEw
	=
X-Google-Smtp-Source: AGHT+IH94xAQRz012cal+dtTIW78sAGXd6hKfHgDijGJnPM5PXBWmIrYOpBIfTC2T3+iyjIC+3M3Sg==
X-Received: by 2002:a2e:b8d2:0:b0:2e1:e8fa:4f4b with SMTP id s18-20020a2eb8d2000000b002e1e8fa4f4bmr7740843ljp.32.1714993309049;
        Mon, 06 May 2024 04:01:49 -0700 (PDT)
Message-ID: <fb687016-6979-4294-8426-a70d579dfb50@suse.com>
Date: Mon, 6 May 2024 13:01:48 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 03/12] VT-d: parse ACPI "SoC Integrated Address
 Translation Cache Reporting Structure"s
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <c70b250d-2ec4-4254-89cf-d3241dac0d35@suse.com> <Zjiw89WKvy6vJAPn@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zjiw89WKvy6vJAPn@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 12:29, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:14:31AM +0100, Jan Beulich wrote:
>> This is a prereq to us, in particular, respecting the "ATC required"
>> flag.
>>
>> Note that ACPI_SATC_ATC_REQUIRED has its #define put in dmar.h, as we
>> try to keep actbl*.h in sync what Linux (who in turn inherit from ACPI
>> CA) has.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> Lovely: On the SPR system with the SATC I tried passing "ats" (the
>> "required" flag is clear there), just to then hit "IOMMU#4: QI dev wait
>> descriptor taking too long" while setting up Dom0. The 2nd message there
>> doesn't ever appear, so the request never completes. Not sure whether
>> that's us doing something wrong or the hardware acting up. In the former
>> case I'd generally expect an IOMMU fault to be raised, though. FTR same
>> on 4.18 with just "VT-d: correct ATS checking for root complex
>> integrated devices" backported there.
> 
> Great, so we likely have a bug in our ATS implementation?

Or there's a hardware / firmware issue. As said in the remark, while I'm
not really sure which one it is, I'd kind of expect some form of error
indication rather than just a hang if we did something wrong.

>> --- a/xen/drivers/passthrough/vtd/dmar.c
>> +++ b/xen/drivers/passthrough/vtd/dmar.c
>> @@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
>>  LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
>>  static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
>>  static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
>> +static LIST_HEAD_READ_MOSTLY(acpi_satc_units);
> 
> We could even make this one RO after init.

Maybe, after first introducing LIST_HEAD_RO_AFTER_INIT() and then
perhaps switching the others up front. IOW I'd prefer to keep those
consistent and then (if so desired) update them all in one go.

>> @@ -750,6 +751,93 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
>>      return ret;
>>  }
>>  
>> +static int __init register_one_satc(struct acpi_satc_unit *satcu)
>> +{
>> +    bool ignore = false;
>> +    unsigned int i = 0;
>> +    int ret = 0;
>> +
>> +    /* Skip checking if segment is not accessible yet. */
>> +    if ( !pci_known_segment(satcu->segment) )
>> +        i = UINT_MAX;
>> +
>> +    for ( ; i < satcu->scope.devices_cnt; i++ )
>> +    {
>> +        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
>> +        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
>> +        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
>> +
>> +        if ( !pci_device_detect(satcu->segment, b, d, f) )
>> +        {
>> +            dprintk(XENLOG_WARNING VTDPREFIX,
>> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
>> +                    &PCI_SBDF(satcu->segment, b, d, f));
>> +            ignore = true;
>> +        }
>> +        else
>> +        {
>> +            ignore = false;
>> +            break;
>> +        }
>> +    }
>> +
>> +    if ( ignore )
>> +    {
>> +        dprintk(XENLOG_WARNING VTDPREFIX,
>> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable\n",
>> +                satcu->segment);
> 
> Re the error messages: won't it be better to print them using plain
> printk and gate on iommu_verbose being enabled if anything?
> 
> It does seem a bit odd that such messages won't be printed when
> iommu={debug,verbose} is enabled on the command line.

Well, perhaps yes. Yet I'm trying here to stay (largely) in sync with how
in particular register_one_rmrr() behaves. Do you strictly think I should
diverge here?

>> +static int __init
>> +acpi_parse_one_satc(const struct acpi_dmar_header *header)
>> +{
>> +    const struct acpi_dmar_satc *satc =
>> +        container_of(header, const struct acpi_dmar_satc, header);
>> +    struct acpi_satc_unit *satcu;
>> +    const void *dev_scope_start, *dev_scope_end;
>> +    int ret = acpi_dmar_check_length(header, sizeof(*satc));
>> +
>> +    if ( ret )
>> +        return ret;
>> +
>> +    satcu = xzalloc(struct acpi_satc_unit);
>> +    if ( !satcu )
>> +        return -ENOMEM;
>> +
>> +    satcu->segment = satc->segment;
>> +    satcu->atc_required = satc->flags & ACPI_SATC_ATC_REQUIRED;
>> +
>> +    dev_scope_start = (const void *)(satc + 1);
>> +    dev_scope_end   = (const void *)satc + header->length;
> 
> Isn't it enough to just cast to void * and inherit the const from the
> left side variable declaration?

Misra won't like the (transient) removal of const, afaict. Personally I
also consider it bad practice to omit such const.

> You could even initialize dev_scope_{start,end} at definition.

Right. This is again the way it is to be in sync with other
acpi_parse_one_...() functions. It's always hard to judge where to diverge
and where consistency is weighed higher. Whichever way you do it, you may
get comment asking for the opposite ...

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:09:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:09:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717549.1119861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wE2-0004Cc-KB; Mon, 06 May 2024 11:09:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717549.1119861; Mon, 06 May 2024 11:09:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wE2-0004CV-GW; Mon, 06 May 2024 11:09:42 +0000
Received: by outflank-mailman (input) for mailman id 717549;
 Mon, 06 May 2024 11:09:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3wE1-0004By-3w
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:09:41 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 214646c3-0b99-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:09:39 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-34de61b7ca4so1251129f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:09:39 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 z18-20020adff1d2000000b0034df178a9acsm10388104wro.99.2024.05.06.04.09.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 04:09:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 214646c3-0b99-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714993778; x=1715598578; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=SN6cXPrjM8zvabuWv90fC9A8MK0G1jsVP2M5Dz/QHXM=;
        b=af3H1eUVJUnAkzHljEdX3jBe+UUUIVRFEA4Q+bSisKkt0N9Uwz6YxbY4o4Ohot4sSe
         UaULU4Gu3MsE9LvHRxRe2ZHDENl+Rvyb/AikiFxuKSrD4LR3/Fsmw1tt8pQ8ggymYgyv
         CrgS4XQDUBR/W5ulGbp5qlqtHGNzi3zTTVTbg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714993778; x=1715598578;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SN6cXPrjM8zvabuWv90fC9A8MK0G1jsVP2M5Dz/QHXM=;
        b=uSnpiCykuZyaL/ZNERFGv9K+7LfIMKNGjF17riv8v9oeTlGX6EsJtcpvpDnJw2T3TH
         QA4+jF37QksI8PxpaYC2ESzoez/VBaZdaVtzbTKPG0h6bAeasX0QuBXYCbvrqVWdK9Hb
         V0UL1qEATpu1JYsopS5oWlCB1mClWbg/4sUj+q8HPx+icjuuhRfLfTLLBfRxs9ltobQJ
         WSa7T/BH4b7D1ilWJ2TOrNQWHAWQnyj15/X9KrU2YhhiP/Ixyhjpf3yPswQMF7DzDmsH
         UM3kMsTZqcLUOe+l3rOP8SI7y6cvugFfgHkaBW2QqTe0DmVtQrZJBp4FH+0fk8whH5sI
         +UuQ==
X-Gm-Message-State: AOJu0Yw3SOHl0KLJnqfacYf71CNr5etB4tbSOtohFjlmoAa2ioOYSYho
	Si87JPW44BCfDoUhwOs7BGVS35ASBnvCaYVry4QJIWPNRoAT2PpsOK9RD7hg7qg=
X-Google-Smtp-Source: AGHT+IHLJ/8wloe8izuJOeQqefxaHn6wmY/aRSzCnUC1+AqirbyKx6DNeQdaWFoJq4BA0tuYAdUg3w==
X-Received: by 2002:a5d:678b:0:b0:34b:44d7:f3eb with SMTP id v11-20020a5d678b000000b0034b44d7f3ebmr9305830wru.3.1714993778311;
        Mon, 06 May 2024 04:09:38 -0700 (PDT)
Date: Mon, 6 May 2024 13:09:37 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 03/12] VT-d: parse ACPI "SoC Integrated Address
 Translation Cache Reporting Structure"s
Message-ID: <Zji6cWcAQD1RsQWD@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <c70b250d-2ec4-4254-89cf-d3241dac0d35@suse.com>
 <Zjiw89WKvy6vJAPn@macbook>
 <fb687016-6979-4294-8426-a70d579dfb50@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <fb687016-6979-4294-8426-a70d579dfb50@suse.com>

On Mon, May 06, 2024 at 01:01:48PM +0200, Jan Beulich wrote:
> On 06.05.2024 12:29, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 11:14:31AM +0100, Jan Beulich wrote:
> >> This is a prereq to us, in particular, respecting the "ATC required"
> >> flag.
> >>
> >> Note that ACPI_SATC_ATC_REQUIRED has its #define put in dmar.h, as we
> >> try to keep actbl*.h in sync what Linux (who in turn inherit from ACPI
> >> CA) has.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

I think however it should be mentioned in the description that
introduced code attempts to stay in sync with the existing
register_one_satc and acpi_parse_one_*() functions.

> >> ---
> >> Lovely: On the SPR system with the SATC I tried passing "ats" (the
> >> "required" flag is clear there), just to then hit "IOMMU#4: QI dev wait
> >> descriptor taking too long" while setting up Dom0. The 2nd message there
> >> doesn't ever appear, so the request never completes. Not sure whether
> >> that's us doing something wrong or the hardware acting up. In the former
> >> case I'd generally expect an IOMMU fault to be raised, though. FTR same
> >> on 4.18 with just "VT-d: correct ATS checking for root complex
> >> integrated devices" backported there.
> > 
> > Great, so we likely have a bug in our ATS implementation?
> 
> Or there's a hardware / firmware issue. As said in the remark, while I'm
> not really sure which one it is, I'd kind of expect some form of error
> indication rather than just a hang if we did something wrong.
> 
> >> --- a/xen/drivers/passthrough/vtd/dmar.c
> >> +++ b/xen/drivers/passthrough/vtd/dmar.c
> >> @@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
> >>  LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
> >>  static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
> >>  static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
> >> +static LIST_HEAD_READ_MOSTLY(acpi_satc_units);
> > 
> > We could even make this one RO after init.
> 
> Maybe, after first introducing LIST_HEAD_RO_AFTER_INIT() and then
> perhaps switching the others up front. IOW I'd prefer to keep those
> consistent and then (if so desired) update them all in one go.
> 
> >> @@ -750,6 +751,93 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
> >>      return ret;
> >>  }
> >>  
> >> +static int __init register_one_satc(struct acpi_satc_unit *satcu)
> >> +{
> >> +    bool ignore = false;
> >> +    unsigned int i = 0;
> >> +    int ret = 0;
> >> +
> >> +    /* Skip checking if segment is not accessible yet. */
> >> +    if ( !pci_known_segment(satcu->segment) )
> >> +        i = UINT_MAX;
> >> +
> >> +    for ( ; i < satcu->scope.devices_cnt; i++ )
> >> +    {
> >> +        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
> >> +        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
> >> +        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
> >> +
> >> +        if ( !pci_device_detect(satcu->segment, b, d, f) )
> >> +        {
> >> +            dprintk(XENLOG_WARNING VTDPREFIX,
> >> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
> >> +                    &PCI_SBDF(satcu->segment, b, d, f));
> >> +            ignore = true;
> >> +        }
> >> +        else
> >> +        {
> >> +            ignore = false;
> >> +            break;
> >> +        }
> >> +    }
> >> +
> >> +    if ( ignore )
> >> +    {
> >> +        dprintk(XENLOG_WARNING VTDPREFIX,
> >> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable\n",
> >> +                satcu->segment);
> > 
> > Re the error messages: won't it be better to print them using plain
> > printk and gate on iommu_verbose being enabled if anything?
> > 
> > It does seem a bit odd that such messages won't be printed when
> > iommu={debug,verbose} is enabled on the command line.
> 
> Well, perhaps yes. Yet I'm trying here to stay (largely) in sync with how
> in particular register_one_rmrr() behaves. Do you strictly think I should
> diverge here?
> 
> >> +static int __init
> >> +acpi_parse_one_satc(const struct acpi_dmar_header *header)
> >> +{
> >> +    const struct acpi_dmar_satc *satc =
> >> +        container_of(header, const struct acpi_dmar_satc, header);
> >> +    struct acpi_satc_unit *satcu;
> >> +    const void *dev_scope_start, *dev_scope_end;
> >> +    int ret = acpi_dmar_check_length(header, sizeof(*satc));
> >> +
> >> +    if ( ret )
> >> +        return ret;
> >> +
> >> +    satcu = xzalloc(struct acpi_satc_unit);
> >> +    if ( !satcu )
> >> +        return -ENOMEM;
> >> +
> >> +    satcu->segment = satc->segment;
> >> +    satcu->atc_required = satc->flags & ACPI_SATC_ATC_REQUIRED;
> >> +
> >> +    dev_scope_start = (const void *)(satc + 1);
> >> +    dev_scope_end   = (const void *)satc + header->length;
> > 
> > Isn't it enough to just cast to void * and inherit the const from the
> > left side variable declaration?
> 
> Misra won't like the (transient) removal of const, afaict. Personally I
> also consider it bad practice to omit such const.
> 
> > You could even initialize dev_scope_{start,end} at definition.
> 
> Right. This is again the way it is to be in sync with other
> acpi_parse_one_...() functions. It's always hard to judge where to diverge
> and where consistency is weighed higher. Whichever way you do it, you may
> get comment asking for the opposite ...

Oh, yes, IIRC you already mentioned this in v1, yet I've forgot when
reviewing this one.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:09:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:09:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717550.1119871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wEB-0004Uq-QI; Mon, 06 May 2024 11:09:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717550.1119871; Mon, 06 May 2024 11:09:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wEB-0004Uh-Na; Mon, 06 May 2024 11:09:51 +0000
Received: by outflank-mailman (input) for mailman id 717550;
 Mon, 06 May 2024 11:09:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3wEA-0004By-Gr
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:09:50 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 272a9598-0b99-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:09:48 +0200 (CEST)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-34d8f6cfe5bso1283991f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:09:48 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dn2-20020a0560000c0200b0034e14d59f07sm10491751wrb.73.2024.05.06.04.09.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:09:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 272a9598-0b99-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714993788; x=1715598588; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gsHoXsKWZlQKbQRTc+3UPbRnjtnVIkWe+22IWxBFW7c=;
        b=aXKzGEL4QcLdu7fqOe5y4ZJ9cGqrOpRfbTGHL5ZgMKuvZyGzevpbZeRvwVuHpfZmTU
         qiIFXE0xDSUTwXB8gixRxdjNThRedOx+6HbS1cHzAx0roWp/58fB2OXS9mAZFWzlxRkK
         yC+JmrAyLVhsm6/E20cHNC21GwlhEd9L02j0cgYUgi5l8jH1xchw0BZnu3ciQoyeUl1f
         /ZS5eYm8X+8jfDz1w08KFNKq58jjdN+dPqSr2Q5Z1Bs2opD08mj15MPOgzTPD+8Jc8kw
         5MaWCgFW1HiPxU78KJrugz0pP0B6j4gtqFWls1F4J85HDl9Pec4SudDLVePUSu6dEjMQ
         JDtg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714993788; x=1715598588;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gsHoXsKWZlQKbQRTc+3UPbRnjtnVIkWe+22IWxBFW7c=;
        b=jLauXhwW9hjp13UG3cUsLHxaODuow1yQP1DkrfdzKLkHp6xz105m9WiTl9YSf9J/cb
         jAN+pm/OxLgAADeQgOAv7ibLJhWSBTgE9RiLbgTjCzzwa02yNIyi5QTIiVSkER+gJ0iJ
         1qcCuOsOX+wzLkdZJBM46vVSglmabUjPuLkMx6grWXD9YYzQCD+bnBpNUMgQs/Oo7RfW
         Pdu7Qb34ZY1USI7HUr7vSwUntRvqeJFeFu7fCsaCKYusyVkZVwziLBCZZdEsPn6OCHms
         IdxYwV+db0b+b+ax0yL8RsALA/hTvyiMk9E0GLXANf9oDl+Agte8v3w1eizg3fxHUKvP
         GUmA==
X-Forwarded-Encrypted: i=1; AJvYcCVUZk2GTisMD5ipGtW8P9hKCIBHRkmFNwa5MnT3LCfJcZ9htDgv2FhJWM8ksIXl5m7kQqAlNOLUBSKEEzfzhE6QXYkVfOfU0QOOwuqJEuU=
X-Gm-Message-State: AOJu0Yw8ONpzJ5otubWXVdbKCNWjdNnbTZFElLCbatCV1pKkQlqKUM8p
	Bj6y/wxHrc3keVWJbX8Jj1nzI80lArvDrPuAupsp20JMWqEhmSvRKMzpJCFcgA==
X-Google-Smtp-Source: AGHT+IHxA4I7mEO4IbqZAXu4j8B+iLCzfpQGSRpqTch6z6IlewZWOsSKErblzLKw4zhR0fVIDXc28g==
X-Received: by 2002:a5d:6503:0:b0:349:fc93:1e2 with SMTP id x3-20020a5d6503000000b00349fc9301e2mr7454624wru.9.1714993788282;
        Mon, 06 May 2024 04:09:48 -0700 (PDT)
Message-ID: <326643bc-2d8c-4cff-87ff-e1abb7b78e8d@suse.com>
Date: Mon, 6 May 2024 13:09:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 1/5] x86/vpmu: separate amd/intel vPMU code
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <ddbf4e24c41f48832bba4b11368cd5e6e9f79a20.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ddbf4e24c41f48832bba4b11368cd5e6e9f79a20.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 11:12, Sergiy Kibrik wrote:
> Build AMD vPMU when CONFIG_AMD is on, and Intel vPMU when CONFIG_INTEL
> is on respectively, allowing for a plaftorm-specific build.
> 
> No functional change intended.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

I can only guess that Stefano is likely fine with

> changes in v2:
>  - drop static inline stubs, use #idef/#endif in vpmu_init)()

this.

Acked-by: Jan Beulich <jbeulich@suse.com>
with ...

> --- a/xen/arch/x86/cpu/vpmu.c
> +++ b/xen/arch/x86/cpu/vpmu.c
> @@ -827,6 +827,7 @@ static int __init cf_check vpmu_init(void)
>  
>      switch ( vendor )
>      {
> +#ifdef CONFIG_AMD
>      case X86_VENDOR_AMD:
>          ops = amd_vpmu_init();
>          break;
> @@ -834,11 +835,12 @@ static int __init cf_check vpmu_init(void)
>      case X86_VENDOR_HYGON:
>          ops = hygon_vpmu_init();
>          break;
> -
> +#endif
> +#ifdef CONFIG_INTEL
>      case X86_VENDOR_INTEL:
>          ops = core2_vpmu_init();
>          break;
> -
> +#endif
>      default:
>          printk(XENLOG_WARNING "VPMU: Unknown CPU vendor: %d. "
>                 "Turning VPMU off.\n", vendor);

... neither of the blank lines dropped.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:18:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:18:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717557.1119881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wMF-000766-LP; Mon, 06 May 2024 11:18:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717557.1119881; Mon, 06 May 2024 11:18:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wMF-00075z-HQ; Mon, 06 May 2024 11:18:11 +0000
Received: by outflank-mailman (input) for mailman id 717557;
 Mon, 06 May 2024 11:18:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3wME-00075t-1x
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:18:10 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5132ddb6-0b9a-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 13:18:08 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41b782405bbso11752075e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:18:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 iv16-20020a05600c549000b0041bff91ea43sm15702641wmb.37.2024.05.06.04.18.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:18:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5132ddb6-0b9a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714994288; x=1715599088; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bbPEahgx1/JOW7KB6YqGwbxrg9vduTiudeqA5BcZtsc=;
        b=BHzsFoDf9oSLrl5GBlIXwg6WtVfOBIh41cDSr/GraarNBduaveGg7IHVnio2QNBmWv
         ihnwceXVrTKqqueLkm/LjgdHoNlc7CL+VD4VHUv5JW0uNv3bgsoT1cl/JBFpqNoXmqzI
         Tv9xIhd6MXcpV4xUeXyQKQxecMgDgiNqBgKmH2ZFH+qQi5aP067o8TKrEwWFHxXauWJJ
         15BYs5Nbpl6gjWofeOvsp2cj6gpWldiVk+9hZ3ZiBSF0uBnbY4YwpHIKatLkt6Jc65oa
         bog69H6qcRrxlZgkzY9VGax2OFlMa+eOnlNk3oDCdCWgcmsZIwoVXVXLLUuytDvh2do4
         uCUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714994288; x=1715599088;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bbPEahgx1/JOW7KB6YqGwbxrg9vduTiudeqA5BcZtsc=;
        b=R9sSkvBsomo7JjNgxHvGF0yiA+diGwbLLGPrUWEVqiOpaZGNmH2GSPhz/7H+hNgvNa
         UKYRlDeXNpSK4wx+RrQJfRv4NXmS0D5NV345uK+mxPH0ExDYhPFY4YHegEdKmWeFCIs9
         9qwM4p8HxGfKcKN6HxXZOseK0wPkd1MMQtbqm/SpDuPSFuZYBi5PfiaKsE9RfWi71kH/
         e59+i0VDoFidTAgGgtHAXmfo60bpff7dpyy681Ih4mGdCSMmP2Qf6C0iLeuHaakFgQzh
         c34LkKsHIvL16n4qGhpvGCqZuDJtu2hiHKn8O5Vw5H9PTvwzh32PZTibs7+KfY9LUUm2
         NbOA==
X-Forwarded-Encrypted: i=1; AJvYcCXs1XUk+k8BwHSOxrqOSHcSAi6WzjHIT9/NOz1+65bTUkxUp6HWVq6coujmuAYEwOGrtgymKKODTeRe4RqXKJg3kpgeNzTym67hxH6Ucr0=
X-Gm-Message-State: AOJu0YwoIbeE6MsZro1TW2aXtX20+Jn/DCucwRjwMUBoXDyv+3dzzt3S
	A8BJtTULh2ZzwzgRZUFvO1H+whON1nbXuUBUVnm37u0+iCCHxYjE5Yohj4mMhg==
X-Google-Smtp-Source: AGHT+IEPvgX1YR01s3+KyrpN9dhm8u3W/Unz/twTIDZtWHlIouDEJkkSgV8dx2J5/2hiUbBlMXIXvA==
X-Received: by 2002:a05:600c:34d2:b0:41c:3e1:9dbd with SMTP id d18-20020a05600c34d200b0041c03e19dbdmr6829468wmq.40.1714994288279;
        Mon, 06 May 2024 04:18:08 -0700 (PDT)
Message-ID: <12d7600c-ccfa-430c-9c62-a2402b9620b8@suse.com>
Date: Mon, 6 May 2024 13:18:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/5] x86/intel: move vmce_has_lmce() routine to
 header
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <e759fc187f2e3f23f42666b2a8461ae851591d50.1714640459.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e759fc187f2e3f23f42666b2a8461ae851591d50.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 11:14, Sergiy Kibrik wrote:
> Moving this function out of mce_intel.c would make it possible to disable
> build of Intel MCE code later on, because the function gets called from
> common x86 code.
> 
> Add internal check for CONFIG_INTEL option, as MCG_LMCE_P bit is currently
> specific to Intel CPUs only.

My previously voiced concern regarding this was not addressed. If ...

> --- a/xen/arch/x86/cpu/mcheck/mce.h
> +++ b/xen/arch/x86/cpu/mcheck/mce.h
> @@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v, uint32_t msr)
>      return 0;
>  }
>  
> +static inline bool vmce_has_lmce(const struct vcpu *v)
> +{
> +    return IS_ENABLED(CONFIG_INTEL) && v->arch.vmce.mcg_cap & MCG_LMCE_P;

... the IS_ENABLED() indeed is to stay, the & wants parenthesizing.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:18:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:18:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717559.1119891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wMe-0007Y0-S0; Mon, 06 May 2024 11:18:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717559.1119891; Mon, 06 May 2024 11:18:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wMe-0007Xt-Oe; Mon, 06 May 2024 11:18:36 +0000
Received: by outflank-mailman (input) for mailman id 717559;
 Mon, 06 May 2024 11:18:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3wMc-0007V6-VO; Mon, 06 May 2024 11:18:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3wMc-0000vt-Jc; Mon, 06 May 2024 11:18:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s3wMc-00057b-AT; Mon, 06 May 2024 11:18:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s3wMc-0002ca-9z; Mon, 06 May 2024 11:18:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sFkflJwd4LSroBZKIQbHodnifyw9rjB4cxysMuFVCZk=; b=RUwraBylhshFDp4yvpVZUP8ErT
	c9M5My62NrYjj0NzUnHu/JSSNdJikYht5HbUNvUEeBZLU3yyw7w1AK44OnxUDL22NZUoeNkeJZ/pj
	Rz194EO3j9XgnwLW5iFO8w5nYpo5S6WvXQiHksKxpUcs8ekw6JedfqP8Fqr7wEsrfW/A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185924-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185924: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=692f6199bd9a45c93952142c610f0a7dd3b3ef8e
X-Osstest-Versions-That:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 11:18:34 +0000

flight 185924 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185924/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  692f6199bd9a45c93952142c610f0a7dd3b3ef8e
baseline version:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb

Last test of basis   185912  2024-05-03 17:02:06 Z    2 days
Testing same since   185924  2024-05-06 08:02:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Federico Serafini <federico.serafini@bugseng.com>
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f95cd010cb..692f6199bd  692f6199bd9a45c93952142c610f0a7dd3b3ef8e -> smoke


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:20:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:20:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717565.1119900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wOP-0000vc-77; Mon, 06 May 2024 11:20:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717565.1119900; Mon, 06 May 2024 11:20:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wOP-0000vV-4a; Mon, 06 May 2024 11:20:25 +0000
Received: by outflank-mailman (input) for mailman id 717565;
 Mon, 06 May 2024 11:20:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3wON-0000vJ-HN
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:20:23 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1093097-0b9a-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 13:20:22 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51fb14816f6so2114688e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:20:22 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l30-20020a05600c1d1e00b0041b086d664fsm15745352wms.6.2024.05.06.04.20.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:20:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1093097-0b9a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714994422; x=1715599222; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Xk6skj+2N00M6JOjJM3GFYM/7DpvW5dXFYBY/ROTOiM=;
        b=SKl59TnNFR8IOrG+IY+29KvO6l4nLwCWSL0x5jkUcVPWe2LigHviZrlozYn/zXFNZ/
         r6wc0QQO3RuAu6QZNReW7tYYuLR1radYEIkFfTGyxKAWl63mXODyllUj3XdtVocaywfq
         5Lm7v91g/qjcLwycX/kkC2mtwXsd/BlmdIyKUAkPvPlY8WO8083aux7ckXsvS3ARhS1q
         9lydPJXkNIO/IjMJ9ehpvw8QggDGD+7J+y9UlvA0ZMc3fJMCjsyRje2ALXUlh/f0TlCY
         lE1/vBNBbdBBkTRb09/wz+XEEZedJ8MkmAclR2Px2PPhO0cDjMA2gmfZzl1eWOeMx7Gk
         4WoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714994422; x=1715599222;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Xk6skj+2N00M6JOjJM3GFYM/7DpvW5dXFYBY/ROTOiM=;
        b=hVgS+FO5V22wlZH86iH+Rd6b1i6G3B5aWzSFQ6pt46ayBueDkXq02vn+Qhvd5AzC1I
         DMMpCMiaUnTyI8NYLz3owmkBrLbQvTfM/HGvVkbYfTLE/AJl4nKR8H+yQNElpWOMpeUR
         896w9LSzT92E+AJU1T+mfEru0BliMcLON9nAZMfPdVCYiPKQoDlgJH1ouYykf/gQ17g6
         gxE62yru08sx7molyVy1jBen7soctkx1qpPV4o+VYSpll1SX3HvBa7CRvAVwjMlcC8lZ
         X2hKIT4LOjE3v24BvGaO47L91K9dOQYf/ik9eBfrwBnJoGjMjE+IrAu+xpm2us/uY+Nb
         wrFw==
X-Forwarded-Encrypted: i=1; AJvYcCVmYrgaj5BArTCnS3PdQot4MI6SCnakNSTqqO8BisPl1/MMmgNC81d9rvyIRFk3CwNSS8lI0m0hz0sT25MYL0QTbmwXolerLexjb25jXmw=
X-Gm-Message-State: AOJu0Yxkfm8ms0f60bq8STe1bAjWqp35VG//f3JGYV8fskX0M4yjwzbG
	V3WMh0GQ6KU4TX5jLWGromvrb7QO4/Cn5cOo5rTozVM3bGk7t0hKvJYT0Wq/Wg==
X-Google-Smtp-Source: AGHT+IEQ+BuYF/yKwkkQIm5qZ5Cm2Frm+wwIbJdt2J2D3MNQ8TGmyM1AuRGRFP0yJnLSw7IdCZEqrg==
X-Received: by 2002:a05:6512:1256:b0:51d:4c0a:a512 with SMTP id fb22-20020a056512125600b0051d4c0aa512mr8054779lfb.68.1714994422158;
        Mon, 06 May 2024 04:20:22 -0700 (PDT)
Message-ID: <4de79253-158e-4aff-a3f0-54a1fc131ec1@suse.com>
Date: Mon, 6 May 2024 13:20:23 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 3/5] x86/MCE: guard access to Intel/AMD-specific
 MCA MSRs
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <725b477a688ad17e5fa2a8bac78f4905170523c8.1714640459.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <725b477a688ad17e5fa2a8bac78f4905170523c8.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 11:16, Sergiy Kibrik wrote:
> Add build-time checks for newly introduced INTEL/AMD config options when
> calling vmce_{intel/amd}_{rdmsr/wrmsr}() routines.
> This way a platform-specific code can be omitted in vmce code, if this
> platform is disabled in config.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

With the same remark and the same constraint as on patch 1:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Mon May 06 11:22:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:22:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717568.1119911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wQ6-0001t7-Lp; Mon, 06 May 2024 11:22:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717568.1119911; Mon, 06 May 2024 11:22:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wQ6-0001t0-I7; Mon, 06 May 2024 11:22:10 +0000
Received: by outflank-mailman (input) for mailman id 717568;
 Mon, 06 May 2024 11:22:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3wQ5-0001su-G2
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:22:09 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df577d81-0b9a-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:22:07 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41ba1ba55ebso12756965e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:22:07 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o10-20020a5d47ca000000b0034d9e5411ebsm10519637wrc.45.2024.05.06.04.22.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:22:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df577d81-0b9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714994527; x=1715599327; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yyYOXV41CuLhX/Vi5skclwIvLF4VyHxTMCAtmuC7KmA=;
        b=OwOG2TnCtKyEB9bv6Wa+TLEyO0+de5CNGDX/q5IQWvgR73JHz+/BFbXtE6g73Z2jBV
         dEoW5DIReSaqZvzLlmfa1tFEkoBdGMvMis0QoCAXNaVn+toexWvDdtCw1nN//WQzXn6k
         bWSQYI8eawuAfQSkeX4vs0a176IFXr5972ZWPq3pGSn8p8cPkdQs6dpne/0D4maqa0I9
         voz4zGe/BRvzz3ibjCITnmGigh7gcisBS2sHYYxni5ysyUCnPZW+NBPsdVSf7FlyO5/g
         LsSqjtDyH3D/oJRHT4gBo5hhMSuy0OBI9VZYnipVWSGZQm1KPnJO8NqGyXubkmuEJWik
         51RA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714994527; x=1715599327;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yyYOXV41CuLhX/Vi5skclwIvLF4VyHxTMCAtmuC7KmA=;
        b=juaucPUCgu0e83BoZM80tKhexnbRr7LojM5S/Z0T7eMg+PlsMCLGquG3Q6gYGlAK4u
         +u7NTd14uhyZdntx2h3WJu5oJtJZdW7PEC6J1NDwjuVOrTp9lT1rXj4GNmMI5GpHCsP3
         i7tGXVuZtIclHDx1Cws9LkiNCnECcT2MVNmFhB1XdEKvn/qOVVgVHxF9IYjAnXU43Pxv
         t4LriHhGNedYV359bSDOIwEb2gCbjlAhlGgEGnx54PkFM4bWxUaXOXXAl2/l+N5nX+Uo
         UK3pJnOcNmdMUutDEO7iZESjmikBcBiWbqKtdeTRa0d6+z+NUxt7ddIHwQjug4klZQEv
         Ulkw==
X-Forwarded-Encrypted: i=1; AJvYcCVZ3nB4DHXAiajXtxX9r2SihpBaHU1EMYNHuGbo4MluLmw2HjY/l4wORKgZDR04wxfKc8FUiPbkTiWEb3gTnUQ1eHt1n1tbl5ouL6oyidE=
X-Gm-Message-State: AOJu0YytS+LqsIS8POR5gUoiqsQuPpZG6KVgRtZS0FAErSUJgljDgq07
	N7GMEC/vldgQq0OdcdRxGUPEAqs41TWx8PKlRU4ldlL03JjeCHoIG9+gVTIdPg==
X-Google-Smtp-Source: AGHT+IGdkcSjykiRn7ClIHtiPYhkfyim0yWkgLhyNWQMRTA14P3YuTKF+QU8prc+np//z52GRyPM8Q==
X-Received: by 2002:a05:600c:1d24:b0:41b:fea6:6526 with SMTP id l36-20020a05600c1d2400b0041bfea66526mr8305217wms.33.1714994526874;
        Mon, 06 May 2024 04:22:06 -0700 (PDT)
Message-ID: <30a55580-2ebc-47ca-8e62-a518c1aca2d3@suse.com>
Date: Mon, 6 May 2024 13:22:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 4/5] x86/MCE: guard {intel/amd}_mcheck_init() calls
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <ef74d26d98cb533d148fcc83c354919443ebaa74.1714640459.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ef74d26d98cb533d148fcc83c354919443ebaa74.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 11:18, Sergiy Kibrik wrote:
> Guard calls to CPU-specific mcheck init routines in common MCE code
> using new INTEL/AMD config options.
> 
> The purpose is not to build platform-specific mcheck code and calls to it,
> if this platform is disabled in config.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Once again with the same remark and the same constraint as on patches 1 and 3:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:27:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717573.1119921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wV3-0003aV-6m; Mon, 06 May 2024 11:27:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717573.1119921; Mon, 06 May 2024 11:27:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wV3-0003aO-4E; Mon, 06 May 2024 11:27:17 +0000
Received: by outflank-mailman (input) for mailman id 717573;
 Mon, 06 May 2024 11:27:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3wV2-0003aI-Fo
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:27:16 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9672260b-0b9b-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:27:14 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41b79451153so14942065e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:27:14 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 u17-20020a05600c19d100b0041bb11ff5a7sm19561968wmq.8.2024.05.06.04.27.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 04:27:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9672260b-0b9b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714994834; x=1715599634; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=pP7CcC8wKLVjW69QigxatZDum9+U92H/3B9J1+uPzh0=;
        b=LtL+v5nqh8NiyWKth5ZbYsInoaimKa6g6GXdsMjItFCyMXJyIE5OmWqlwJIY4ksj2G
         TOaVdRFohsjwLsS53pwpt/pA0wTsNbPTX9docB0jwcRWoQr45FjBdyuOGdgdSWNtswza
         m+g2PqUcRiZJ/SQ5gMompisk33ONl5lIGSHPo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714994834; x=1715599634;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pP7CcC8wKLVjW69QigxatZDum9+U92H/3B9J1+uPzh0=;
        b=tpTL867paHxi0iPZVJUDwJV3dU/xrjDCO4MS0wjRqiTZRlnv6s2vm8uY4j4ox0ksa7
         UHdWR4qoip5uCRJo8pSxImmuoK6zQI9GOUhlK0qvTEoF0n2QpqPrp31DH5/Hb7X7WbR5
         T4Ryvi9LZXM8rbpDW3XpAZJRyIulm5ptQIlfdg61ZF0To3BEr5CKN309sFoChBWNuuC+
         TwyTK2xFohZ2gAl7OT5M+mRaZk8wCNgY21ZnInZXHdLlWRXZOodj09UBYRmrhRb8xHib
         SBqqA7qQAEH6kqXER7XdLgUmuuc4f1uoumEA/fVJ/R0VEMa6UFSK5Yc3fd9lRoXQ3yFQ
         040Q==
X-Gm-Message-State: AOJu0YwRL7fl6ucs8a/vugZiqQER5SQ48CVhZN/+pGkVbYgkUOXCTuMb
	5sT+WE4O7sYKxOPty9qv0OYRTbbkRU6ISHQigViU4Xh+oLSEJxJotb1ipPQlCbs=
X-Google-Smtp-Source: AGHT+IGuuZwQJSX82zAwuhnIT2ue8FPBPpi8Zlm7WQ2/i/fnr/l6arirfl6i4/Zd3pkbx5iZyOLABg==
X-Received: by 2002:a05:600c:46cc:b0:41a:fa9a:d86b with SMTP id q12-20020a05600c46cc00b0041afa9ad86bmr7438560wmo.11.1714994833854;
        Mon, 06 May 2024 04:27:13 -0700 (PDT)
Date: Mon, 6 May 2024 13:27:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 04/12] AMD/IOMMU: add helper to check whether ATS is
 to be used for a device
Message-ID: <Zji-kEKvpoGLhFxp@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <bd89275f-915a-4925-afbb-494133bb6e65@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bd89275f-915a-4925-afbb-494133bb6e65@suse.com>

On Thu, Feb 15, 2024 at 11:15:12AM +0100, Jan Beulich wrote:
> The same set of conditions is used in three places, requiring to be kept
> in sync. Introduce a helper to centralize these checks.
> 
> To allow all parameters of the new helper be pointer-to-const,
> iommu_has_cap() also needs its 1st parameter to be constified. Beyond
> that further "modernize" that function.
> 
> Requested-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:32:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:32:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717576.1119930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3waD-0005fF-Om; Mon, 06 May 2024 11:32:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717576.1119930; Mon, 06 May 2024 11:32:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3waD-0005f8-M9; Mon, 06 May 2024 11:32:37 +0000
Received: by outflank-mailman (input) for mailman id 717576;
 Mon, 06 May 2024 11:32:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3waC-0005f2-6I
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:32:36 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 54bf47b0-0b9c-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:32:33 +0200 (CEST)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-41b79451153so14983235e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:32:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r2-20020adfce82000000b0034dd7984d7fsm10450975wrn.94.2024.05.06.04.32.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:32:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54bf47b0-0b9c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714995153; x=1715599953; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sbwKYqc4ObxT5PtsWwJ+fQdTLqENUDfgL5UhlzbL1a0=;
        b=QywJJThyBumbLeVdl6SjH0g0cYrI6u3qsFCY5Fz6Yp2AcaWIy3KaMcCWiMbSwodsjJ
         fqUnbqz2htwziWRPENFd8uChVONEKB/LhoA+8FIuERgkKiYaw8BCBv0FTcaa+Q19W4i6
         5v+Ks+tmLLvP1T1I6KtiYqoe4F7ti5ax4VnqUagOlCP9RwRJdyGkxgoLzjFZTQxKYOTW
         hxqJdGBxqoVWdBT/Q9pKqpdm2dp53P1hZg9++QV+F8xbB6kUBs4hhnvf+Hcbml9N5sQD
         i9uucJvSpXXyqPPTIAxgoj1X42pbw0Or27SNGNW7zI6I8fP9ItFE6brfyyvzDKNGsXiu
         mvdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714995153; x=1715599953;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sbwKYqc4ObxT5PtsWwJ+fQdTLqENUDfgL5UhlzbL1a0=;
        b=SMqWjnVIN0s4bSTjbdMz9eitDJtF8+PS4yRmSlQs+gudeWM/Jiet9dTucjZ83oz+5V
         ID4iyJsLWdGhy6GpaCeWVA4rGNKn9oPWCnDc+hlSYy+WT0anXB9pAPluoXmMrtWKnIMI
         ifbd4KaJ+Xz1hMO7L3jIzBeTFIFwHzj1eOdS+NJqbrBynjB5/4MG1eo1J4QD9uKV0Wfi
         9rwgTm5SRR8tSKV8s7NIzerwv+UbyzgXjGHB4URFaziWJBY175sNqZ+8+EvQ5yIDSoUl
         ruecl1YXALGCajEsLDW8oJUTt+CH6EIZDxBq/YbPVw5T28CYdaglGHhF4ok4SPUk0G7c
         ktOA==
X-Forwarded-Encrypted: i=1; AJvYcCVVJT2V0J4B4ioJCWFmMBNRTipgjblifLdzlBpkqhRQuawoZTtdWbU2kplvDPpvXbWqwVHfPJeht/8+CK+iHWQvbpAs31hx1nJbgacdm1U=
X-Gm-Message-State: AOJu0Yzpmm8lKSEEho7GdkR+c6fZNR8Iaflb3jLDKW/j2weAfvDNrAI6
	hMIKi0lxkK1yW76JRZHBYrQDEkAzjKwINeFjQ9kb7UegdJoR+Getv7af5Oeatw==
X-Google-Smtp-Source: AGHT+IF0lYVjmW/3y7oemnIhn9f0F2zG9N/m9PTI+XSBjHavdQcZ+hJ9Y/yM43qJ5lpw9A5k9i32eA==
X-Received: by 2002:a05:600c:3b02:b0:418:f760:abfb with SMTP id m2-20020a05600c3b0200b00418f760abfbmr6528723wms.5.1714995153288;
        Mon, 06 May 2024 04:32:33 -0700 (PDT)
Message-ID: <dc14d45e-b2b2-4414-a937-ae95115aa2cd@suse.com>
Date: Mon, 6 May 2024 13:32:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 5/5] x86/MCE: optional build of AMD/Intel MCE code
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <47d32ecff7b915bd23b6d13b76cedf4b39db71a2.1714640459.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <47d32ecff7b915bd23b6d13b76cedf4b39db71a2.1714640459.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 11:21, Sergiy Kibrik wrote:
> Separate Intel/AMD-specific MCE code using CONFIG_{INTEL,AMD} config options.
> Now we can avoid build of mcheck code if support for specific platform is
> intentionally disabled by configuration.
> 
> Add default return value to init_nonfatal_mce_checker() routine -- in case
> of a build with both AMD and INTEL options are off (e.g. randconfig).

I'm afraid that, as before, I can't accept this as a justification for the
addition. The addition likely is wanted, but perhaps in a separate up-front
patch and explaining what's wrong when that's missing.

> Also global Intel-specific variables lmce_support & cmci_support have to be
> redefined if !INTEL, as they get checked in common code.

Them being checked in common code may have different resolution strategies.
The justification here imo is that, right now, both variables are only ever
written by mce_intel.c. As mentioned for vmce_has_lmce(), there's nothing
fundamentally preventing MCG_CAP from having respective bits set on a non-
Intel CPU.

> --- a/xen/arch/x86/cpu/mcheck/mce.h
> +++ b/xen/arch/x86/cpu/mcheck/mce.h
> @@ -40,7 +40,11 @@ enum mcheck_type {
>  };
>  
>  extern uint8_t cmci_apic_vector;
> +#ifdef CONFIG_INTEL
>  extern bool lmce_support;
> +#else
> +#define lmce_support (false)

Nit: Neither here nor ...

> @@ -120,7 +124,11 @@ DECLARE_PER_CPU(struct mca_banks *, poll_bankmask);
>  DECLARE_PER_CPU(struct mca_banks *, no_cmci_banks);
>  DECLARE_PER_CPU(struct mca_banks *, mce_clear_banks);
>  
> +#ifdef CONFIG_INTEL
>  extern bool cmci_support;
> +#else
> +#define cmci_support (false)

... here parentheses are really needed.

> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
> @@ -24,14 +24,20 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>  	 * Check for non-fatal errors every MCE_RATE s
>  	 */
>  	switch (c->x86_vendor) {
> +#ifdef CONFIG_AMD
>  	case X86_VENDOR_AMD:
>  	case X86_VENDOR_HYGON:
>  		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>  		amd_nonfatal_mcheck_init(c);
>  		break;
> +#endif
> +#ifdef CONFIG_INTEL
>  	case X86_VENDOR_INTEL:
>  		intel_nonfatal_mcheck_init(c);
>  		break;
> +#endif
> +	default:
> +		return -ENODEV;
>  	}
>  	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>  	return 0;

Along the lines of remarks on earlier patches, it would be a good opportunity
here to add missing blank lines between non-fall-through case blocks.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 11:54:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 11:54:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717584.1119941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wvd-000369-GE; Mon, 06 May 2024 11:54:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717584.1119941; Mon, 06 May 2024 11:54:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3wvd-000362-Cv; Mon, 06 May 2024 11:54:45 +0000
Received: by outflank-mailman (input) for mailman id 717584;
 Mon, 06 May 2024 11:54:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3wvc-00035C-Ji
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 11:54:44 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c9be461-0b9f-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 13:54:42 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2df9af57b5eso23478321fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 04:54:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v15-20020a05600c444f00b0041be78ae1f0sm15933188wmn.2.2024.05.06.04.54.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 04:54:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c9be461-0b9f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714996482; x=1715601282; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ybB+OrEafTfvjTdX0ZF5+FOZSJMERe67vWiSCm+uFhQ=;
        b=NsCzXzTHD42beaKhXN4YZnXMsXQjTydsaegfUEL1giuWvDkwQR177MTM5n++ueOQSC
         RD1PI6GJOFNXlWzUwwUHlg9LDxe57P4yxVPDiZ+DaLdSsz4AGsdIsRHhuQoobe1RxDlS
         2c195yy+M4pj2vsiFDzzsiYbux/JSssYvh82dyyP819Skr4GAfvhG4VwgtxuWXGMb2GD
         Tmb9gVqCNNNVx5nSczdkcaVeL0NjGsMU95Zk3i4lEJOPzsIHEcnatO7uOFC4gYFTYsut
         xkgMBmQU9GwpdLmjiOHId9QmV5Q/H+k8SXFnagSoawUST3VH0PMp/NnKomo/fUKYdR24
         zPVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714996482; x=1715601282;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ybB+OrEafTfvjTdX0ZF5+FOZSJMERe67vWiSCm+uFhQ=;
        b=Mba8d+Zw0coLKle17aFSeeDW5j5a3NL9KXy2f/kDcon7vDfncpHYxlsbicC/o47nq+
         FJfwnAdEqYPSVvxCCp061yxtZf9o464GRmKP8QuRNeCeaAy6698NueDa3LpVbLlCS8K/
         aIFCyZZF02xEvUEEOu/aOPwMCtC//qonW8aBh3PLXWHFX2SmLGxrV25EOKYOixSSptqa
         q/xaK9JrVTAWWeF1RYe6r4WQm24isIY/BcBlwa0YbYW9qLcx9AoUgZ/VQxHjsezI9JKo
         QNos3Q4+ceCWC/h9VGrX+po67lJGN8MwC7KwfJzR1jB9B5t3q90xI6TpTaKeqUtMqY6p
         njcg==
X-Forwarded-Encrypted: i=1; AJvYcCX+Gqxn4P+iHfLADQshE70tfRd+RgSwZPHvJn8z+VBAPfXLEh1ltKHxEm3aw3yjs9mVF/JDk6cSgr//q4Wv//myFfuQvDtR9v8r9pEgk4w=
X-Gm-Message-State: AOJu0YwRIFzCRHjgUWX8TdzJyRwkNo1kCTICRHVyffy+ZHW20bHXaAuj
	xZY6behlzHFLXJLWNtC7zlppfOm/fWOkmPaUuEDGQ+/VjDB8fup3ovtUUy+k8A==
X-Google-Smtp-Source: AGHT+IF9dsI6Q8fhz+IexT2MYp4Iub5+RHx6SAzPoXOQ5lprLNg5G4WXPI7H1BvzUWFoEz/xDgwIFA==
X-Received: by 2002:a2e:6a01:0:b0:2dd:66ea:5fbd with SMTP id f1-20020a2e6a01000000b002dd66ea5fbdmr7944661ljc.44.1714996481698;
        Mon, 06 May 2024 04:54:41 -0700 (PDT)
Message-ID: <bcfc0da1-5d8a-4e14-be34-89876d668b86@suse.com>
Date: Mon, 6 May 2024 13:54:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 01/13] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-2-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-2-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -71,6 +71,9 @@ config HAS_IOPORTS
>  config HAS_KEXEC
>  	bool
>  
> +config HAS_LLC_COLORING
> +	bool
> +
>  config HAS_PIRQ
>  	bool
>  
> @@ -513,4 +516,23 @@ config TRACEBUFFER
>  	  to be collected at run time for debugging or performance analysis.
>  	  Memory and execution overhead when not active is minimal.
>  
> +config LLC_COLORING
> +	bool "Last Level Cache (LLC) coloring" if EXPERT
> +	depends on HAS_LLC_COLORING
> +	depends on !NUMA
> +
> +config MAX_LLC_COLORS_ORDER

May I ask that you consider dropping MAX_ from here (but keeping "maximum"
in prompt and text), thus ...

> --- /dev/null
> +++ b/xen/common/llc-coloring.c
> @@ -0,0 +1,111 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Last Level Cache (LLC) coloring common code
> + *
> + * Copyright (C) 2022 Xilinx Inc.
> + */
> +#include <xen/keyhandler.h>
> +#include <xen/llc-coloring.h>
> +#include <xen/param.h>
> +
> +#define NR_LLC_COLORS          (1 << CONFIG_MAX_LLC_COLORS_ORDER)

... making this look less strange?

To match up with e.g. max_nr_colors you may also want to use 1U here.

> +void __init llc_coloring_init(void)
> +{
> +    unsigned int way_size;
> +
> +    if ( llc_size && llc_nr_ways )
> +    {
> +        llc_coloring_enabled = true;
> +        way_size = llc_size / llc_nr_ways;
> +    }
> +    else if ( !llc_coloring_enabled )
> +        return;
> +    else
> +    {
> +        way_size = get_llc_way_size();
> +        if ( !way_size )
> +            panic("LLC probing failed and 'llc-size' or 'llc-nr-ways' missing\n");

Since you won't accept way_size == 0 here, how about it ending up zero in
the initial if()'s body? Even more, don't you want to demand
llc_size % llc_nr_ways == 0 there (thus, together with the enclosing
condition, guaranteeing way_size != 0)?

> +    }
> +
> +    /*
> +     * The maximum number of colors must be a power of 2 in order to correctly
> +     * map them to bits of an address.
> +     */
> +    max_nr_colors = way_size >> PAGE_SHIFT;
> +
> +    if ( max_nr_colors & (max_nr_colors - 1) )
> +        panic("Number of LLC colors (%u) isn't a power of 2\n", max_nr_colors);
> +
> +    if ( max_nr_colors > NR_LLC_COLORS )
> +    {
> +        printk(XENLOG_WARNING
> +               "Number of LLC colors (%u) too big. Using configured max %u\n",
> +               max_nr_colors, NR_LLC_COLORS);
> +        max_nr_colors = NR_LLC_COLORS;
> +    } else if ( max_nr_colors < 2 )
> +        panic("Number of LLC colors %u < 2\n", max_nr_colors);

Ah, here's a check guaranteeing at least the first of the two things asked
about above.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:01:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:01:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717593.1119951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3x2Q-0005Dw-CF; Mon, 06 May 2024 12:01:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717593.1119951; Mon, 06 May 2024 12:01:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3x2Q-0005Dp-9J; Mon, 06 May 2024 12:01:46 +0000
Received: by outflank-mailman (input) for mailman id 717593;
 Mon, 06 May 2024 12:01:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3x2O-0005Dj-UD
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:01:44 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 67506f42-0ba0-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 14:01:42 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41ba1ba55e8so11004215e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:01:43 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e14-20020adfe38e000000b0034eba48cd17sm5804009wrm.117.2024.05.06.05.01.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 05:01:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67506f42-0ba0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714996902; x=1715601702; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wn9f2TTvLG3kyrz8hnqjPXnNRo322gO541SYHGNn/VU=;
        b=OuyQ+16KWlo0qX/z+RMJkfqE5EfW6hXJWr7ZOon929dM0Cgv++5Md9Hq9ABT1sFs7T
         jZ4rkA080+0G4dyVVPL+7Fg+IEqTgNLmO4dfy3M0m0JFhKrCL9fuO+S7sfErb3pnvY8T
         jwUYkFfOQjdzHZX4iINNHgkrtWhf1ZGciwAaJjhb2u1nO4F49gIx9rBcydxmsBCrXPyv
         I+ACQ1XGbBCj3Si9KIu7S50xNBDDraRu6ppfQR3UtaL8dzRlyZJv84NS+vOQfO19NS2N
         LNCKOf0aYoa9ZBPGF9lmnHhZCmZg2gi6ScqUmHkN1J+jdgbiBhG6nV9cbaqvquaK8hYr
         tooQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714996902; x=1715601702;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wn9f2TTvLG3kyrz8hnqjPXnNRo322gO541SYHGNn/VU=;
        b=XfYW4FNIgCDS9P+jue1OTn8QYJG1geSkMcR+7tmf7nozcVAnSg1raLW6Hv6zSwr5i/
         2Jt9+iLKHFCHZ2jGHIP4AbeGeX9RE6sKtzV8avTiUuIv0UOwnPAZKQThxdXQA+YLjK7J
         VWTLvCPiAW+OA6DPTYjuqiuc4d8GN9gfSTsc6GvvmYsaHGSM+/L/QMNU6Z4l6RQOtgVe
         k14eM41cCtm9SbFquFk1gCJGTWT9ra6catckP0DqjVXYjcJgL+43zu50YtBZ2nPAppoi
         l2ff7MXvm3mnsneITlebIIArmzVp1FTOK7OE2lnrz8jorT5gmdGyPsrFGKN3ovpn3c3N
         H9wg==
X-Forwarded-Encrypted: i=1; AJvYcCUaxvB8G+jxBSdoMOFkUqztCPzq10acvAdZlk1SgV8fQLyvzUP8jOmgUgQ0XGUqmX/UBSAGwFvfJ7F9AcNuX0I/uv7f1Ik4utp7qYyGpfc=
X-Gm-Message-State: AOJu0Yy3zgYO+uG6rEiS5Xc3s+V5MzURPQd7WV+4hbV4tw5w+x+W/79I
	CwxTdPCRC8mChxgFTbZR/1D0e5YWb7T+Qx/Npdk8jT2mY1XgMv8Q1/IxpJ3kGw==
X-Google-Smtp-Source: AGHT+IHbcN2efsltGEzBrytA6ECwgsHScr6/ca0HcKPzaGq9thQFa7/O+xzlLQEFc8o5DTWGOaUNxw==
X-Received: by 2002:a5d:4f09:0:b0:34d:8c1f:3194 with SMTP id c9-20020a5d4f09000000b0034d8c1f3194mr6694080wru.35.1714996902417;
        Mon, 06 May 2024 05:01:42 -0700 (PDT)
Message-ID: <8fbd50df-6c9a-4177-8001-39c0f15e715e@suse.com>
Date: Mon, 6 May 2024 14:01:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 04/13] xen/arm: add Dom0 cache coloring support
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-5-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-5-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> Add a command line parameter to allow the user to set the coloring
> configuration for Dom0.
> A common configuration syntax for cache colors is introduced and
> documented.
> Take the opportunity to also add:
>  - default configuration notion.
>  - function to check well-formed configurations.
> 
> Direct mapping Dom0 isn't possible when coloring is enabled, so
> CDF_directmap flag is removed when creating it.
> 
> Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> 
> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit ...

> v8:
> - added bound check on dom0_num_colors
> - default colors array set just once
> v7:
> - parse_color_config() doesn't accept leading/trailing commas anymore
> - removed alloc_colors() helper
> v6:
> - moved domain_llc_coloring_free() in this patch

... I'm having trouble seeing why this was done: Dom0 isn't ever cleaned up
after, is it?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:05:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:05:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717595.1119961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3x5m-00068u-RE; Mon, 06 May 2024 12:05:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717595.1119961; Mon, 06 May 2024 12:05:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3x5m-00068n-Oe; Mon, 06 May 2024 12:05:14 +0000
Received: by outflank-mailman (input) for mailman id 717595;
 Mon, 06 May 2024 12:05:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3x5l-00068h-HD
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:05:13 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e3ca27ac-0ba0-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:05:12 +0200 (CEST)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-41ba1ba55e8so11026035e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:05:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l8-20020a05600c4f0800b0041bfa349cadsm19496151wmq.16.2024.05.06.05.05.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 05:05:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3ca27ac-0ba0-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714997111; x=1715601911; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+P+kuDOoW/O68w1duUHJDOYwUCEcePMI8GDNDSdlUQc=;
        b=UgFQhSJxwywsQwaBIE0qcAush4GR6El/t0DUDjcP2oLGAnNg7rXqVbV2n4Cbr6APZF
         sQJVgn7Q+ht0VrpKnc37EZ7/qh+D8/EyYWflJzSU2Lt6s0lFV1tRCc9af8G+JWTBCwla
         TL4zZ9vgzY4b8AewyOQeDKTxcN05jl9L7ExvKnUEgBKUPTWFea4bZgS/PUpNxJ8OxsiF
         ABHPKw/+y/TYPq5ZAnJHK0V41i7mEAbOqqESNPObpvr3v8r1ZVxEm+I0Je6X9vD8G/82
         jeUuNz7ayuE3OgnETXk6bQrnPJ47I4tKMksAF24C0mNiI1N0RSXFYdJANcajBNIf8ax8
         snFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714997111; x=1715601911;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+P+kuDOoW/O68w1duUHJDOYwUCEcePMI8GDNDSdlUQc=;
        b=UqmWUtyzLRGWiw835cvMEuOr6wxC3KK4+3H6LXRnVX19H9h+h83eNqCgP+uDsdR8H+
         YsCGm/BAOhnYqV3X4P87iCcxvTA9gdHicNQIfyxqyNggWhLaHQW17B6m8RctC0FVYTUL
         FmvTpKQKW+VSetJ+2vdtMKwcQ9WkM3+nRuTvm4MEcT9kBhjEUsUb6FZ8l6Pe8aPV2jAx
         Eh32cZGD7l2Kct30KIUyXOyKovTaxz1310Ukt3kHHfmRm/4N5J6usizp0ZNb2jpVjMHb
         9PJsVD9/ejlNMKH31B1FIvQrEqocSG9KAMytW138zXgZna8eJVlPSVrWcLkDVSdupha8
         1MFQ==
X-Forwarded-Encrypted: i=1; AJvYcCXommGgEZJeP6rbieHJ4fRGh2E44JtT0qJd/H6qoaMOPCLXYyv8vZJDkEE3+dX7vjvdCsL8ky58alBro4nH1tVfIRl4lVCVEdgJkMies6g=
X-Gm-Message-State: AOJu0Yy+F5oiccPJjAqe6957u/svhACNCXTNobCG9oHSFysIU2EB/Wxb
	u0SNtWk1MVuHZnausc0dUxt94cCBxXFLXkXdorGk81wZ2YEE3y82mH8c/6gs6vCYpdN8yLoy2Ho
	=
X-Google-Smtp-Source: AGHT+IGtEp9O52Q5V4v4MTeXqoQtciGtDqHEhNmWCT1ioSfMzg37Dhy/Xi/WfxpLOBFVZmc5yW8cHw==
X-Received: by 2002:a05:600c:310e:b0:41b:f979:e359 with SMTP id g14-20020a05600c310e00b0041bf979e359mr6958693wmo.38.1714997111248;
        Mon, 06 May 2024 05:05:11 -0700 (PDT)
Message-ID: <6c37e82b-5116-4856-9a83-8ed20c2d1d0d@suse.com>
Date: Mon, 6 May 2024 14:05:12 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 05/13] xen: extend domctl interface for cache coloring
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-6-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-6-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> Add a new domctl hypercall to allow the user to set LLC coloring
> configurations. Colors can be set only once, just after domain creation,
> since recoloring isn't supported.
> 
> Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> 
> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one remark:

> @@ -226,6 +226,46 @@ void domain_llc_coloring_free(struct domain *d)
>      xfree(__va(__pa(d->llc_colors)));
>  }
>  
> +int domain_set_llc_colors(struct domain *d,
> +                          const struct xen_domctl_set_llc_colors *config)
> +{
> +    unsigned int *colors;
> +
> +    if ( d->num_llc_colors )
> +        return -EEXIST;
> +
> +    if ( !config->num_llc_colors )
> +    {
> +        domain_set_default_colors(d);
> +        return 0;
> +    }
> +
> +    if ( config->num_llc_colors > max_nr_colors )
> +        return -EINVAL;
> +
> +    colors = xmalloc_array(unsigned int, config->num_llc_colors);
> +    if ( !colors )
> +        return -ENOMEM;
> +
> +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_colors) )
> +    {
> +        xfree(colors);
> +        return -EFAULT;
> +    }
> +
> +    if ( !check_colors(colors, config->num_llc_colors) )
> +    {
> +        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);

Slightly shorter (and more in line with what we have elsewhere) as

       printk(XENLOG_ERR "%pd: bad LLC color config\n", d);

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:11:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:11:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717601.1119971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xBw-0008Se-BF; Mon, 06 May 2024 12:11:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717601.1119971; Mon, 06 May 2024 12:11:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xBw-0008SX-81; Mon, 06 May 2024 12:11:36 +0000
Received: by outflank-mailman (input) for mailman id 717601;
 Mon, 06 May 2024 12:11:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3xBu-0008SR-UR
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:11:34 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c78c8f98-0ba1-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:11:33 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-34d9c9f2cf0so1386113f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:11:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n13-20020a5d420d000000b00346f9071405sm10577419wrq.21.2024.05.06.05.11.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 05:11:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c78c8f98-0ba1-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714997493; x=1715602293; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=B5rhoF8q7VZIX3oI/NbOVreFCdgRgYaV1D6l9CEVgVM=;
        b=E4nn0BPUTlZNh2nFXNpiX6zYjIB2zVNJeAqfJtCi26S3z51SBZjFExPfK2SfxNfJMh
         pt32GzXuW+sIHr9ePlN9O+14GRXjMrWzkv5GLmtMNujIWuZRrwYH+FPoySMfxnL9RCx0
         4gNmER7g/qBwOj00rwa2YHcPw3erb3grSIZl3VCxvfh/fvKBfsmaE0vTIMS+MndXgdmk
         Qy77kFAnXBMDFm5vyVV8iwz4kA2GNyfEp0GAqPiVxxUHqIkBxsC8NBsE64/+JSmFTZjP
         4V3W13B+Bf5E+Tr6H66F+V7gJAr4YtEJXWDP0zsYiFeAs3b5O8X7olT2fcy+Qxosu4kv
         wcRw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714997493; x=1715602293;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=B5rhoF8q7VZIX3oI/NbOVreFCdgRgYaV1D6l9CEVgVM=;
        b=leqMlvFqM47jIfq1G56r1fJmyyt6Bkw7LjS1pchtwC+66fH4e4631o5mE8XYeRJly3
         53SI1G3Wjh702wl5YBPwE399/Go4Vo9xES1r8l1GMNkgf/gS2Q05EA3wklPTFbPgpKtZ
         RHh8zfHsFdS+ydNOCZqImhyLVHG9Aatjv+HhmP/+PH2l/43GEtQ4YaRZQPQvQQRJlsWK
         dyxbj6/cK67bDLn0d4CW7gJtgfg3htY7gJF+/nr16SYmrgffIDkyly9FZyHgGPWJ2duW
         +3/qvoCCqhBrNFVNvev2ZXwntJTHc4uacctFRtQws1lCg3m40IEFj2oNjTo5wAlmavn6
         1dtA==
X-Forwarded-Encrypted: i=1; AJvYcCUADPk3fAdhATiW+aqJUGMiD5I9cBs2R2pv1M73BpwGEzbkZNifwVk2kbYRPBKgI7L5Fz65ojAM80gXE2992iwlt8asSh8G2Of0YnpyTk8=
X-Gm-Message-State: AOJu0YxQh2FHMb2Ej3al/4ywG4c4tDeL1oEbldtzWYu4mUOrrPWogjr/
	zXhHJPV1uOkRiouA5+KqwpwQcdxohsfMHPZ9dvB4LuwtOs8R3A6oHHtcSv36JA==
X-Google-Smtp-Source: AGHT+IGELMoGWj7XS02yT090pj1Pgn/rogR7hLMWYYhdwdpxYWLGQAocEfBILhm0HrAgl3v3QgvNhg==
X-Received: by 2002:a05:6000:c0c:b0:345:663f:cd79 with SMTP id dn12-20020a0560000c0c00b00345663fcd79mr8542837wrb.14.1714997493318;
        Mon, 06 May 2024 05:11:33 -0700 (PDT)
Message-ID: <99f6db77-23b8-4a18-b0b1-be4b9cd7f522@suse.com>
Date: Mon, 6 May 2024 14:11:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 07/13] xen/arm: add support for cache coloring
 configuration via device-tree
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-8-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-8-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> @@ -266,6 +267,47 @@ int domain_set_llc_colors(struct domain *d,
>      return 0;
>  }
>  
> +int __init domain_set_llc_colors_from_str(struct domain *d, const char *str)
> +{
> +    int err;
> +    unsigned int *colors, num_colors;
> +
> +    if ( !str )
> +    {
> +        domain_set_default_colors(d);
> +        return 0;
> +    }
> +
> +    colors = xmalloc_array(unsigned int, max_nr_colors);
> +    if ( !colors )
> +        return -ENOMEM;
> +
> +    err = parse_color_config(str, colors, max_nr_colors, &num_colors);
> +    if ( err )
> +    {
> +        printk(XENLOG_ERR "Error parsing LLC color configuration");
> +        xfree(colors);
> +        return err;
> +    }
> +
> +    if ( !check_colors(colors, num_colors) )
> +    {
> +        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
> +        xfree(colors);
> +        return -EINVAL;
> +    }
> +
> +    /* Adjust the size cause it was initially set to max_nr_colors */
> +    colors = xrealloc_array(colors, num_colors);
> +    if ( !colors )
> +        return -ENOMEM;

In the error case you're leaking the (too large) array. IMO you want this to
be best effort anyway - try to shrink, but if shrinking fails use what you
already have:

    d->llc_colors = xrealloc_array(colors, num_colors);
    if ( !d->llc_colors )
        d->llc_colors = colors;

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:22:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:22:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717606.1119980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xM8-0002c1-7D; Mon, 06 May 2024 12:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717606.1119980; Mon, 06 May 2024 12:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xM8-0002bu-4O; Mon, 06 May 2024 12:22:08 +0000
Received: by outflank-mailman (input) for mailman id 717606;
 Mon, 06 May 2024 12:22:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3xM7-0002bo-H3
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:22:07 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 409c3f33-0ba3-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:22:06 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2db17e8767cso23237951fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:22:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h11-20020a05600c314b00b00418f72d9027sm19623521wmo.18.2024.05.06.05.22.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 05:22:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 409c3f33-0ba3-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714998126; x=1715602926; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Bxo0c2JnSEh32fqZxBsLZSfN57DBcB8XaU4pEtQxhYI=;
        b=HfojG50/Rkw9T3NEU1wbd8jN8HRH3QcWwjVC33pDWP/jdu/tkIzt117Wd/v7ZEdhDL
         V0O7bPJ8mhot0/YH/1cMYGO9C7i2UZwiZIOO8yIsA5CTlLF4vu2ppgQNQAESaHWEDN/Z
         HTk2jmtlbnxOSo260MiFwUNhKyLjUoc6NqzJaLHwIItGLc40hVqzZ2FtItrOoo5wMBCs
         y7pHWOc+CW9XNZVL+W3ap/fNXPXHDeC0u9h7Yt6tEDlmbik4AK4A9fYB+z1Sr+VMO+8S
         9uIAs+LeoxjMWrafZ7Quh2pI3fogcxTiv6EHNotJx01EUEcbUopPWO3Lg9BTq4YrDuDF
         J98g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714998126; x=1715602926;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Bxo0c2JnSEh32fqZxBsLZSfN57DBcB8XaU4pEtQxhYI=;
        b=srynDeYytfGiv95XhPCnIZh3zT+3jX27wU3K43r8TI/YM3LXuvdmTyKpCAVzatRIh3
         BG+Xjr8bvqZKQjIOcVrOBiJFaluOxmnU9n3jw0bKY0oV5UBWsPYdm+EXG1Rbk4nUXIju
         nuGSzszhIrifcJuwcjPD+lcK/CwJJAlDKgkYdLLHqW5aitHCwn78CBs2p8pwZVW2tz4f
         NcDYFNLhMLi2v0EzdxwjoJ+LAZfsC/Vxds19B7aB/6sic42UTutmjEoRZbJWkRAgVw4M
         1TITfHu85uKQ+Bf21lZjY2bVs8B81XkKChkJZ2H/21u+JrtCzsjVblih9tcFO8rmbZJ1
         sq6Q==
X-Forwarded-Encrypted: i=1; AJvYcCUfwt45lhS0xY16ZyRdJCCB9V60VTIBcEcXakXkNaSaemUNe1iVJp47fHpugW9fY8rBfOXCFvacLijUFk++ui0yuV9nWvbllW5d+fZQh9Y=
X-Gm-Message-State: AOJu0YwcbrSvUaFz5JwDSOKC7MAefOfWg93V6oheQjtydWY7hp7l/nsI
	ySdYL9W2BZDP89dLg8NPzL+OCAUXZ3Sas/R0zKB9L8J/nF5TbRHmMo0hP9DCZg==
X-Google-Smtp-Source: AGHT+IHWCFIwLkDwGzs54+64v6fwBdW6o42d7rP4gg8wPh4kgwmzYg12fCeRwZcZkLqs5cp3zFVjIw==
X-Received: by 2002:a2e:bc02:0:b0:2d8:6054:a1bd with SMTP id b2-20020a2ebc02000000b002d86054a1bdmr8093495ljf.40.1714998125902;
        Mon, 06 May 2024 05:22:05 -0700 (PDT)
Message-ID: <674578c8-8243-40f6-ad90-28f9b76a42fd@suse.com>
Date: Mon, 6 May 2024 14:22:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 08/13] xen/page_alloc: introduce preserved page flags
 macro
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-9-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-9-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -159,6 +159,7 @@
>  #endif
>  
>  #define PGC_no_buddy_merge PGC_static
> +#define PGC_preserved (PGC_extra | PGC_static)

Seeing this again and its use ...

> @@ -1426,11 +1427,11 @@ static bool mark_page_free(struct page_info *pg, mfn_t mfn)
>      {
>      case PGC_state_inuse:
>          BUG_ON(pg->count_info & PGC_broken);
> -        pg->count_info = PGC_state_free;
> +        pg->count_info = PGC_state_free | (pg->count_info & PGC_preserved);
>          break;
>  
>      case PGC_state_offlining:
> -        pg->count_info = (pg->count_info & PGC_broken) |
> +        pg->count_info = (pg->count_info & (PGC_broken | PGC_preserved)) |
>                           PGC_state_offlined;
>          pg_offlined = true;
>          break;

... here: Shouldn't PGC_broken also be included in PGC_preserved?

> @@ -2484,6 +2485,11 @@ struct page_info *alloc_domheap_pages(
>          }
>          if ( assign_page(pg, order, d, memflags) )
>          {
> +            unsigned long i;
> +
> +            for ( i = 0; i < (1UL << order); i++ )
> +                pg[i].count_info &= ~PGC_extra;

For larger order this loop is non-trivial and may have a fair effect on
caches. Looking at the code just outside of upper patch context, is this
loop needed at all when MEMF_no_refcount is clear in memflags?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:42:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:42:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717614.1119990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xfa-0006hS-Tm; Mon, 06 May 2024 12:42:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717614.1119990; Mon, 06 May 2024 12:42:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xfa-0006hL-Qn; Mon, 06 May 2024 12:42:14 +0000
Received: by outflank-mailman (input) for mailman id 717614;
 Mon, 06 May 2024 12:42:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3xfZ-0006hF-FG
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:42:13 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0e8bcfa4-0ba6-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 14:42:11 +0200 (CEST)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-41c1b75ca31so13285125e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:42:11 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d10-20020a05600c3aca00b00418e4cc9de7sm19628689wms.7.2024.05.06.05.42.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 05:42:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e8bcfa4-0ba6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1714999330; x=1715604130; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=q6S8tN0PqUfWdBDz4LYEnqY2T5KjG/5O0y44awgmZBk=;
        b=mA6fE7RQipQDTPBwzdhz1Ejc4htv5ML9LgNIWhARp5hqij+VgnF87IHQZMdiDwrlFQ
         H6P7NmND8Oi3NMg/yDhzySHwZFEjl9kUUNPXTsCgrW5pP+o3XMjQi4y4QyZ39SUg7NiZ
         63uI3m6m/Rp+9EYpfYQGudCKYiPxSfTPRYIYo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714999330; x=1715604130;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=q6S8tN0PqUfWdBDz4LYEnqY2T5KjG/5O0y44awgmZBk=;
        b=P5lfrX8NzVAToQ0sJz1xvkqm8JkbSgWe3ZYgduWVDhx/Ki6rHdv1Y+oz/gajaLUeC3
         Gq39BtekN74sLRH6m9Io45LvE3sYcOL4x0SawGHk30YRSMTSfKPHBW50IX/hS1f4RMa7
         dH3qrs11xDSM8l8rc17nc977TjZeupYuchS32evxXi/BtaBWVXeRYC++tJ7xuEARDC9v
         sp7tsIrGKJbMEzBXMLthupxrrb9JgJcDz3h8G0Ppru9nHzHZYYkVkgh65//gkzcRvtll
         JdYJ9zHC25jB9kSv+uiZ7+M3vWODSMDALiN+b9hYgSA4Ax+gVKg9bvAfiJ/KNZM8/7wl
         1nUw==
X-Gm-Message-State: AOJu0YzGq2vuffyz5U+V1WYBwVuKOfto84u0vhcmoJwMiX4NNB0GYXFD
	ilyXnkzYQvPJ0wh0b4gSe4+gS1+fS+svyKjaIYb67MCTfZRrOc8F+Nm+TKWm0ng=
X-Google-Smtp-Source: AGHT+IH9EKQc2r2CMZ2jiGPEoRkY88zsw2uViGhGuKiiPjYbA4remP977CoWSC3O6ec50nyxwUoWaA==
X-Received: by 2002:a05:600c:4711:b0:41a:c170:701f with SMTP id v17-20020a05600c471100b0041ac170701fmr7637618wmo.38.1714999330380;
        Mon, 06 May 2024 05:42:10 -0700 (PDT)
Date: Mon, 6 May 2024 14:42:09 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Message-ID: <ZjjQIaxEwS6b-swj@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>

On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
> Make the variable a tristate, with (as done elsewhere) a negative value
> meaning "default". Since all use sites need looking at, also rename it
> to match our usual "opt_*" pattern. While touching it, also move it to
> .data.ro_after_init.

I guess I need to look at further patches, as given the feedback on
the past version I think we agreed we want to set ATS unconditionally
disabled by default, and hence I'm not sure I see the point of the
tri-state if enabling ATS will require an explicit opt-in on the
command line (ats=1).

> The only place it retains boolean nature is pci_ats_device(), for now.
> 
> In AMD code re-order conditionals to have the config space accesses
> after (cheaper) flag checks.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> In domain_context_mapping_one() I'm a little puzzled that translation
> type is selected based on only IOMMU and global properties, i.e. not
> taking the device itself into account.

Is it maybe fine do set DEV_IOTLB unconditionally as long as the IOMMU
supports it, and then let the device decide whether it wants to issue
translated or non-translated requests depending on whether it supports
(and enables) ATS?

I think it would be best to limit this strictly to devices that have
ATS enabled.

> ---
> v2: Re-base over new earlier patches.
> 
> --- a/xen/drivers/passthrough/amd/iommu_cmd.c
> +++ b/xen/drivers/passthrough/amd/iommu_cmd.c
> @@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
>      struct amd_iommu *iommu;
>      unsigned int req_id, queueid, maxpend;
>  
> -    if ( !ats_enabled )
> +    if ( opt_ats <= 0 )

If having a tri-state is required, won't it be best to decide on a
binary value at init time, so that runtime functions can handle
opt_ats as a boolean?

Otherwise we are open coding the default expectation of what -1
implies (disabled) in all use sites.

>          return;
>  
>      if ( !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) )
> @@ -340,7 +340,7 @@ static void _amd_iommu_flush_pages(struc
>          flush_command_buffer(iommu, 0);
>      }
>  
> -    if ( ats_enabled )
> +    if ( opt_ats > 0 )
>      {
>          amd_iommu_flush_all_iotlbs(d, daddr, order);
>  
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -119,7 +119,7 @@ static bool use_ats(
>      const struct amd_iommu *iommu,
>      const struct ivrs_mappings *ivrs_dev)
>  {
> -    return !ivrs_dev->block_ats &&
> +    return opt_ats > 0 && !ivrs_dev->block_ats &&
>             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>             pci_ats_device(iommu->seg, pdev->bus, pdev->devfn);
>  }
> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>  
>          if ( use_ats(pdev, iommu, ivrs_dev) )
> -            dte->i = ats_enabled;
> +            dte->i = true;

Might be easier to just use:

dte->i = use_ats(pdev, iommu, ivrs_dev);

>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> @@ -257,7 +257,7 @@ static int __must_check amd_iommu_setup_
>                                           ACPI_IVHD_SYSTEM_MGMT));
>  
>          if ( use_ats(pdev, iommu, ivrs_dev) )
> -            ASSERT(dte->i == ats_enabled);
> +            ASSERT(dte->i);

ASSERT(dte->i == use_ats(pdev, iommu, ivrs_dev));

>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> --- a/xen/drivers/passthrough/ats.c
> +++ b/xen/drivers/passthrough/ats.c
> @@ -18,8 +18,8 @@
>  #include <xen/pci_regs.h>
>  #include "ats.h"
>  
> -bool __read_mostly ats_enabled;
> -boolean_param("ats", ats_enabled);
> +int8_t __ro_after_init opt_ats = -1;
> +boolean_param("ats", opt_ats);
>  
>  int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list)
>  {
> --- a/xen/drivers/passthrough/ats.h
> +++ b/xen/drivers/passthrough/ats.h
> @@ -22,7 +22,7 @@
>  #define ATS_QUEUE_DEPTH_MASK     0x1f
>  #define ATS_ENABLE               (1<<15)
>  
> -extern bool ats_enabled;
> +extern int8_t opt_ats;
>  
>  int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list);
>  void disable_ats_device(struct pci_dev *pdev);
> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
>  
>  static inline int pci_ats_device(int seg, int bus, int devfn)
>  {
> -    if ( !ats_enabled )
> +    if ( !opt_ats )
>          return 0;

Can't you remove that check altogether now, since you are adding an
opt_ats check to use_ats()?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:46:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:46:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717618.1120001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xjY-0007u2-Cz; Mon, 06 May 2024 12:46:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717618.1120001; Mon, 06 May 2024 12:46:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xjY-0007tv-AA; Mon, 06 May 2024 12:46:20 +0000
Received: by outflank-mailman (input) for mailman id 717618;
 Mon, 06 May 2024 12:46:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3xjW-0007tp-Hh
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:46:18 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1697a2b-0ba6-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:46:17 +0200 (CEST)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-34da35cd01cso1760877f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:46:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n12-20020a5d588c000000b0034cfb79220asm10719760wrf.116.2024.05.06.05.46.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 05:46:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1697a2b-0ba6-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1714999577; x=1715604377; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ghM6KZl18bBETFhHKssHmHIUmInwONy1mb9Y8rJH74Q=;
        b=MkHsBn3mAJnQsLAL/Z6W91ols+ZPcIsGhFCgPnU7WMgQJAhHlvxLcFKTePEbYPgsjf
         O/K3SMPsuNKzIfqmiuUTfjBlb+FW1RiLD5OKvoKZLqxz6KyXV3gu3+BuC+dZRb+6bhG5
         7lWbaD4aQaIUVfxAt831nELOCoLJAybgAosM/P4Y7zXRwO/2QISDx8Fs5MIRznEpkA1Z
         ccjksq7vPitlWBoeyZCx9ak6IMesAL9+uT3lB5TNw3XRpLaFD8RqUiBQIJhtiPTUy22W
         z/LUqOvlvl7mUosRksoPJWmTbQ94DygywWPgsUcBmrnHfsytDiURV6Sn2UszFVouVhBB
         cIrg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714999577; x=1715604377;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ghM6KZl18bBETFhHKssHmHIUmInwONy1mb9Y8rJH74Q=;
        b=GAuNeeJgQzeXn5Bhk/bpE+mUPs7JOUCi9lb5Wkr/lPhbwKTvXojID8CoSaLXa/q5XM
         hxUqfwNnZvqIH7kOjZ63dBMplwzMB2pcbzftXXf4xHCY80IA73sSznbssQz9gj6lRUzE
         v7WZgyiZGMoprEGdpnBSKb7Vf3LFmwD3nkm7NKEUoz91jKkO2g5WTXrhY5vYCCWbRo0q
         v04f66s12d/VNBkOMxyMxZodsP4hvtWaDSifkKvmeWlVtay8f8+iyPGp6+HOugeNTMJQ
         Qjp04LhA12+Jv51iP7tg6G8CTTPBNjHQbgRRS4QWUfbOOVTDrekDF8Uy8ifLMmLA73gO
         fgFA==
X-Forwarded-Encrypted: i=1; AJvYcCUy/5yLuenzhjM8+Da91xeIdZuf7gnpHh41qSn+3/Cyp8BymXW0gvaBSud44TAADsyfqDZn4cbdHJ0Bp2b998oC+pOZ1FhuVsG2MwGCnbY=
X-Gm-Message-State: AOJu0YzQFc5H3TfmGb+WP5ODJoKKCIC/7KjZA8jDNuxtiRlxuJKrxzF5
	3TuwmRjq5AC4ExFrka/Ew9wPd+9hU3geSDvQWmWkaUPrjiS2w7IoV/UXCaZlXQ==
X-Google-Smtp-Source: AGHT+IFgqd6doj7PUBIKYw8MzPQahd+NuX1yuzvoYqBfoTOWBHTMg6LJYi9m88B6ZPQuGNCwvFvMMg==
X-Received: by 2002:adf:e8d0:0:b0:34d:a650:46f8 with SMTP id k16-20020adfe8d0000000b0034da65046f8mr9789135wrn.9.1714999576812;
        Mon, 06 May 2024 05:46:16 -0700 (PDT)
Message-ID: <5c16f53f-3bb0-49d6-b174-b0e8c6ceff4c@suse.com>
Date: Mon, 6 May 2024 14:46:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 09/13] xen: add cache coloring allocator for domains
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-10-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-10-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -270,6 +270,20 @@ and not running softirqs. Reduce this if softirqs are not being run frequently
>  enough. Setting this to a high value may cause boot failure, particularly if
>  the NMI watchdog is also enabled.
>  
> +### buddy-alloc-size (arm64)

I'd like to ask for consistency in the command line doc additions: Either
"(arm64)" here and then also in patch 1, or (less desirable) nowhere.

> @@ -1481,6 +1487,14 @@ static void free_heap_pages(
>              pg[i].count_info |= PGC_need_scrub;
>              poison_one_page(&pg[i]);
>          }
> +
> +        if ( pg->count_info & PGC_colored )
> +        {
> +            /* Colored pages can be 0-order only, so ignore it */
> +            free_color_heap_page(pg, need_scrub);
> +            spin_unlock(&heap_lock);
> +            return;
> +        }

Why "ignore it"? You pass the page to free_color_heap_page() after all.
(later) Oh, you may mean to ignore "order". Yet besides this not really
becoming clear from the wording, I think that this constraint could do
with asserting here.

> @@ -1945,6 +1959,156 @@ static unsigned long avail_heap_pages(
>      return free_pages;
>  }
>  
> +/*************************
> + * COLORED SIDE-ALLOCATOR
> + *
> + * Pages are grouped by LLC color in lists which are globally referred to as the
> + * color heap. Lists are populated in end_boot_allocator().
> + * After initialization there will be N lists where N is the number of
> + * available colors on the platform.
> + */
> +static struct page_list_head *__ro_after_init _color_heap;
> +#define color_heap(color) (&_color_heap[color])
> +
> +static unsigned long *__ro_after_init free_colored_pages;
> +
> +/* Memory required for buddy allocator to work with colored one */
> +#ifdef CONFIG_LLC_COLORING
> +static unsigned long __initdata buddy_alloc_size =
> +    MB(CONFIG_BUDDY_ALLOCATOR_SIZE);

I think it would be quite nice if this and ...

> +size_param("buddy-alloc-size", buddy_alloc_size);
> +
> +#define domain_num_llc_colors(d) (d)->num_llc_colors
> +#define domain_llc_color(d, i)   (d)->llc_colors[i]
> +#else
> +static unsigned long __initdata buddy_alloc_size;

... this were folded. Which I think would be possible if the Kconfig
addition went like this:

config BUDDY_ALLOCATOR_SIZE
	int "Buddy allocator reserved memory size (MiB)" if LLC_COLORING
	default "0" if !LLC_COLORING
	default "64"

> +static struct page_info *alloc_color_heap_page(unsigned int memflags,
> +                                               const struct domain *d)
> +{
> +    struct page_info *pg = NULL;
> +    unsigned int i, color = 0;
> +    unsigned long max = 0;
> +    bool need_tlbflush = false;
> +    uint32_t tlbflush_timestamp = 0;
> +    bool need_scrub;
> +
> +    if ( memflags & ~(MEMF_no_owner | MEMF_no_refcount | MEMF_no_scrub |
> +                      MEMF_no_tlbflush) )
> +        return NULL;
> +
> +    spin_lock(&heap_lock);
> +
> +    for ( i = 0; i < domain_num_llc_colors(d); i++ )
> +    {
> +        unsigned long free = free_colored_pages[domain_llc_color(d, i)];
> +
> +        if ( free > max )
> +        {
> +            color = domain_llc_color(d, i);
> +            pg = page_list_first(color_heap(color));
> +            max = free;
> +        }
> +    }
> +
> +    if ( !pg )
> +    {
> +        spin_unlock(&heap_lock);
> +        return NULL;
> +    }
> +
> +    need_scrub = pg->count_info & (PGC_need_scrub);

Nit: No need for parentheses here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:49:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:49:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717621.1120010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xmx-0008T8-QG; Mon, 06 May 2024 12:49:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717621.1120010; Mon, 06 May 2024 12:49:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xmx-0008T1-NV; Mon, 06 May 2024 12:49:51 +0000
Received: by outflank-mailman (input) for mailman id 717621;
 Mon, 06 May 2024 12:49:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3cb6=MJ=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s3xmx-0008Sv-AR
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:49:51 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2050c584-0ba7-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:49:50 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51f57713684so2292805e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:49:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2050c584-0ba7-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1714999790; x=1715604590; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=M5NJFsBrj9SVXEBCDMWdoSrQkLoJ16wxEkHgXpymkP0=;
        b=ZdpsTlMuCBm7jSbsz/i8A2hHz7ieHlhHKLK52BqYz8QjAofVhk8o3mpjOTkRXm6L83
         8wSOU61TUJuX03sszM+Le8GezeHTUTr//sQUiz0rszJc7DLIi0jHedi4LeqProTA0szZ
         jLgHu/9RHWjH4ZFgZaEVDR+//6Jf7VQ393sSLPUcaKQ1MrElbdaafEUEKpRCjSboz/s0
         ltHs7Yrvn1KMCKyJaEE+dyYbKRtRs1nUMkUIwDphlTbzwPWQ1XxDDm7BSb/J5wkzyQkJ
         wNmGFEHhbuM/+achoZ0Td9q/+Ba6KXew8hpzFMH0KwykExeSAvjkOF2WI8fSq24BUD8y
         UHog==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1714999790; x=1715604590;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=M5NJFsBrj9SVXEBCDMWdoSrQkLoJ16wxEkHgXpymkP0=;
        b=AwRD2AE75YgpRgXuKiU0O26yy8Eqz0j0tbpxuiCyypLQuG2VIVpDDZ4xrqPtWFjtMb
         Ot2mIt4pHAUqbeTS4J3lMu7RGa5YZzk+GQISNW34AGIhRaxq1AKBo/e5OYkh+Snu+xW3
         NyjhyspsxVYdsYBOYGXaum4hc4AP5pVwwK2i965Mz6Oc1QbG09IANTd3iS20oSRcalHL
         dNg8jk1AOCtEMql40+pLPUYrUDjjBYgWOXeQRsc/AN/fAr9jq6RM0R9tWmoWVfL9PrJg
         0oeo0Et0Qoc19Wh8W2FFHg4w6tqFkKLkKvMtUy/f9BrfuM5/Qs3Quhbo9Lrgo0dezFf1
         X6DQ==
X-Gm-Message-State: AOJu0YyYIdBgdB4+ndCljNDi1WDauPQ5G19SbD9CD652/uMhW4qoTumL
	8FgcvrxLTQZYl7Aq7x8lezsXFqQYtk0yNk650IWHMYQzD/nCdmoOcTfK8PmAhRcta363/XpIvVO
	wClbDZTL8zqYg5q9INNZGfv3tfE4=
X-Google-Smtp-Source: AGHT+IEEB0aF6cWX8VP5LbU11br+4/MYlgzLx9i5IwgCvYzDNUlzmCezJSD2ELCMi4/Uzcaru8oBDBNx9KIzZzcAC3E=
X-Received: by 2002:a19:644b:0:b0:51c:cd8d:2865 with SMTP id
 b11-20020a19644b000000b0051ccd8d2865mr6434141lfj.44.1714999789372; Mon, 06
 May 2024 05:49:49 -0700 (PDT)
MIME-Version: 1.0
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-5-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405031630280.1151289@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405031630280.1151289@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Mon, 6 May 2024 14:49:37 +0200
Message-ID: <CAJy5ezpMvTb2Jv-pPMzeRGLxdo5EJSA4RboWj6D41UuTv6mAiA@mail.gmail.com>
Subject: Re: [PATCH v3 4/9] xen/arm64: head: Add missing code symbol annotations
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, bertrand.marquis@arm.com, 
	michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, jbeulich@suse.com, 
	andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, May 4, 2024 at 1:56=E2=80=AFAM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Use the generic xen/linkage.h macros to annotate code symbols
> > and add missing annotations.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > ---
> >  xen/arch/arm/arm64/head.S | 50 ++++++++++++++++++++-------------------
> >  1 file changed, 26 insertions(+), 24 deletions(-)
> >
> > diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
> > index fb297e9eb5..7acedb4f8f 100644
> > --- a/xen/arch/arm/arm64/head.S
> > +++ b/xen/arch/arm/arm64/head.S
> > @@ -90,7 +90,7 @@
> >   * 4K-aligned address.
> >   */
> >
> > -GLOBAL(start)
> > +FUNC(start)
> >          /*
> >           * DO NOT MODIFY. Image header expected by Linux boot-loaders.
> >           */
> > @@ -102,6 +102,7 @@ efi_head:
> >           */
> >          add     x13, x18, #0x16
> >          b       real_start           /* branch to kernel start */
> > +END(start)
> >          .quad   0                    /* Image load offset from start o=
f RAM */
> >          .quad   _end - start         /* Effective size of kernel image=
, little-endian */
> >          .quad   __HEAD_FLAGS         /* Informative flags, little-endi=
an */
> > @@ -223,7 +224,7 @@ section_table:
> >          .align  5
> >  #endif /* CONFIG_ARM_EFI */
> >
> > -real_start:
> > +FUNC_LOCAL(real_start)
> >          /* BSS should be zeroed when booting without EFI */
> >          mov   x26, #0                /* x26 :=3D skip_zero_bss */
> >
> > @@ -263,9 +264,9 @@ primary_switched:
> >          mov   x1, x21                /* x1 :=3D paddr(FDT) */
> >          ldr   x2, =3Dstart_xen
> >          b     launch
> > -ENDPROC(real_start)
> > +END(real_start)
> >
> > -GLOBAL(init_secondary)
> > +FUNC(init_secondary)
> >          msr   DAIFSet, 0xf           /* Disable all interrupts */
> >
> >          /* Find out where we are */
> > @@ -304,7 +305,7 @@ secondary_switched:
> >          /* Jump to C world */
> >          ldr   x2, =3Dstart_secondary
> >          b     launch
> > -ENDPROC(init_secondary)
> > +END(init_secondary)
> >
> >  /*
> >   * Check if the CPU has been booted in Hypervisor mode.
> > @@ -313,7 +314,7 @@ ENDPROC(init_secondary)
> >   *
> >   * Clobbers x0 - x5
> >   */
> > -check_cpu_mode:
> > +FUNC_LOCAL(check_cpu_mode)
> >          PRINT_ID("- Current EL ")
> >          mrs   x5, CurrentEL
> >          print_reg x5
> > @@ -329,7 +330,7 @@ check_cpu_mode:
> >          PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n")
> >          PRINT_ID("- Please update the bootloader -\r\n")
> >          b fail
> > -ENDPROC(check_cpu_mode)
> > +END(check_cpu_mode)
> >
> >  /*
> >   * Zero BSS
> > @@ -339,7 +340,7 @@ ENDPROC(check_cpu_mode)
> >   *
> >   * Clobbers x0 - x3
> >   */
> > -zero_bss:
> > +FUNC_LOCAL(zero_bss)
> >          /* Zero BSS only when requested */
> >          cbnz  x26, skip_bss
> >
> > @@ -353,14 +354,14 @@ zero_bss:
> >
> >  skip_bss:
> >          ret
> > -ENDPROC(zero_bss)
> > +END(zero_bss)
> >
> >  /*
> >   * Initialize the processor for turning the MMU on.
> >   *
> >   * Clobbers x0 - x3
> >   */
> > -cpu_init:
> > +FUNC_LOCAL(cpu_init)
> >          PRINT_ID("- Initialize CPU -\r\n")
> >
> >          /* Set up memory attribute type tables */
> > @@ -399,7 +400,7 @@ cpu_init:
> >           */
> >          msr spsel, #1
> >          ret
> > -ENDPROC(cpu_init)
> > +END(cpu_init)
> >
> >  /*
> >   * Setup the initial stack and jump to the C world
> > @@ -411,7 +412,7 @@ ENDPROC(cpu_init)
> >   *
> >   * Clobbers x3
> >   */
> > -launch:
> > +FUNC_LOCAL(launch)
> >          ldr   x3, =3Dinit_data
> >          add   x3, x3, #INITINFO_stack /* Find the boot-time stack */
> >          ldr   x3, [x3]
> > @@ -421,13 +422,13 @@ launch:
> >
> >          /* Jump to C world */
> >          br    x2
> > -ENDPROC(launch)
> > +END(launch)
> >
> >  /* Fail-stop */
> > -fail:   PRINT_ID("- Boot failed -\r\n")
> > +FUNC_LOCAL(fail)   PRINT_ID("- Boot failed -\r\n")
>
> Maybe we should move PRINT_ID to a newline?
> I am not sure FUNC_LOCAL supports having a command on the same line.

Thanks, yes I checked that it works but it feels a little fragile so
in v4 I'll move the print to a new line (here and in mmu/head.S).

Thanks,
Edgar


>
>
>
> >  1:      wfe
> >          b     1b
> > -ENDPROC(fail)
> > +END(fail)
> >
> >  #ifdef CONFIG_EARLY_PRINTK
> >  /*
> > @@ -438,14 +439,14 @@ ENDPROC(fail)
> >   *
> >   * Clobbers x0 - x1
> >   */
> > -init_uart:
> > +FUNC_LOCAL(init_uart)
> >          ldr   x23, =3DCONFIG_EARLY_UART_BASE_ADDRESS
> >  #ifdef CONFIG_EARLY_UART_INIT
> >          early_uart_init x23, 0
> >  #endif
> >          PRINT("- UART enabled -\r\n")
> >          ret
> > -ENDPROC(init_uart)
> > +END(init_uart)
> >
> >  /*
> >   * Print early debug messages.
> > @@ -454,7 +455,7 @@ ENDPROC(init_uart)
> >   * x23: Early UART base address
> >   * Clobbers x0-x1
> >   */
> > -ENTRY(asm_puts)
> > +FUNC(asm_puts)
> >          early_uart_ready x23, 1
> >          ldrb  w1, [x0], #1           /* Load next char */
> >          cbz   w1, 1f                 /* Exit on nul */
> > @@ -462,7 +463,7 @@ ENTRY(asm_puts)
> >          b     asm_puts
> >  1:
> >          ret
> > -ENDPROC(asm_puts)
> > +END(asm_puts)
> >
> >  /*
> >   * Print a 64-bit number in hex.
> > @@ -471,7 +472,7 @@ ENDPROC(asm_puts)
> >   * x23: Early UART base address
> >   * Clobbers x0-x3
> >   */
> > -ENTRY(asm_putn)
> > +FUNC(asm_putn)
> >          adr_l x1, hex
> >          mov   x3, #16
> >  1:
> > @@ -484,7 +485,7 @@ ENTRY(asm_putn)
> >          subs  x3, x3, #1
> >          b.ne  1b
> >          ret
> > -ENDPROC(asm_putn)
> > +END(asm_putn)
> >
> >  RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
> >
> > @@ -493,16 +494,17 @@ RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef=
")
> >  /* This provides a C-API version of __lookup_processor_type
> >   * TODO: For now, the implementation return NULL every time
> >   */
> > -ENTRY(lookup_processor_type)
> > +FUNC(lookup_processor_type)
> >          mov  x0, #0
> >          ret
> > +END(lookup_processor_type)
> >
> >  #ifdef CONFIG_ARM_EFI
> >  /*
> >   *  Function to transition from EFI loader in C, to Xen entry point.
> >   *  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
> >   */
> > -ENTRY(efi_xen_start)
> > +FUNC(efi_xen_start)
> >          /*
> >           * Preserve x0 (fdt pointer) across call to __flush_dcache_are=
a,
> >           * restore for entry into Xen.
> > @@ -554,7 +556,7 @@ ENTRY(efi_xen_start)
> >          mov   x26, #1               /* x26 :=3D skip_zero_bss */
> >
> >          b     real_start_efi
> > -ENDPROC(efi_xen_start)
> > +END(efi_xen_start)
> >
> >  #endif /* CONFIG_ARM_EFI */
> >
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Mon May 06 12:55:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 12:55:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717625.1120021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xs1-00029p-CE; Mon, 06 May 2024 12:55:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717625.1120021; Mon, 06 May 2024 12:55:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3xs1-00029i-8P; Mon, 06 May 2024 12:55:05 +0000
Received: by outflank-mailman (input) for mailman id 717625;
 Mon, 06 May 2024 12:55:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3cb6=MJ=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s3xs0-00029W-39
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 12:55:04 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dad0ec0f-0ba7-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 14:55:03 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so185984966b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 05:55:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dad0ec0f-0ba7-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715000102; x=1715604902; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EhH/7Q5eRDHZgsJelZf736Vi0Pan7ufpWetR5CiOK+A=;
        b=gdlvWalALhjbKJIuhGrR0QAyyAzcWD4PYhs1VYgoLNWJYmlLQaOcFOIlk9BEnLTOxq
         2TMcgKZFCGIrxiJVY65HO4TX+m9U4xIIkNIwVvLsVuYIM/I0Vbunv3PwfnlKxRpUd95l
         uvhGr4vgZhpJX5paNJ8rKjw3bNU4ccD1hfFC2Q2D14JzXjrdR3x4h6P01Sdr09GRzJXs
         KB/bQ7rZRj22T37TF6dL/BvT59WZDgIBote0J72yi3O0GR1sdubhmO4RXMs8iqNV4CVe
         yJsBnZMTq3peHRieFTzlNRU83+oqT6ScyNqdLw+NQ7W3uTdSSbgNRekQq8D+6KsxADco
         ZXxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715000102; x=1715604902;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=EhH/7Q5eRDHZgsJelZf736Vi0Pan7ufpWetR5CiOK+A=;
        b=qz6ZJxKpUzS2QAKyob7tk5CxY0eGwsqgeqWLPk/FD18x8AtGdOKgOM4WWAyvL4R00z
         hESYHHDi7CEMKA7nDs5P83cWYWlBLPMJWClqFtNpROA64K0DQKMMYEiH/gthYDQKAwJG
         Lp7VGi9pg03tR3fbi9Xw7NqCga2EhJzRW/7kMDKtFfTg438B3bLpBlClwrxLlzoZ2MB3
         Lu9emVLRebA2ptl1HQWcGxG3IaZ7G6C8DpPqev4fy1XT6LH53XYWGipHO3Stnbrpc5T9
         K6TPEhh8xcC+spHvj3a9dAJ/raqTv9YYL7N1612i6mb3SC6W61kqqRY5SltZN0dsQV76
         0HBA==
X-Gm-Message-State: AOJu0Yw/NH6UwbdEs7P/n1Fxl/PcIcM8dS0hB44o1jV/Iam3Wbx4w0TI
	+jXwf1h1Nh9gRDzsARyMiqQEM93jwyk425QC/BJDelDMHXokp3WJjm6LXyOAoe/AXZjrp9qVsJ0
	ZYN/Hf/YnbswfAfvOCru2PkHlR00=
X-Google-Smtp-Source: AGHT+IGGY3pick5A2OFm6u0newd657HwtCQFnmrAmJ767RsGkoGB4nk/StQ01Wlb3Gn8OXFqkh3i5Vzh2taLsTduMZo=
X-Received: by 2002:a50:9b53:0:b0:572:a711:3daf with SMTP id
 a19-20020a509b53000000b00572a7113dafmr6044694edj.40.1715000102574; Mon, 06
 May 2024 05:55:02 -0700 (PDT)
MIME-Version: 1.0
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Mon, 6 May 2024 14:54:51 +0200
Message-ID: <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol annotations
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, bertrand.marquis@arm.com, 
	michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, jbeulich@suse.com, 
	andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, May 4, 2024 at 2:14=E2=80=AFAM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Use the generic xen/linkage.h macros to annotate code symbols
> > and add missing annotations.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > ---
> >  xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
> >  1 file changed, 12 insertions(+), 8 deletions(-)
> >
> > diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
> > index 4e63825220..b16e4d1e29 100644
> > --- a/xen/arch/arm/arm64/bpi.S
> > +++ b/xen/arch/arm/arm64/bpi.S
> > @@ -52,14 +52,15 @@
> >   * micro-architectures in a system.
> >   */
> >      .align   11
> > -ENTRY(__bp_harden_hyp_vecs_start)
> > +FUNC(__bp_harden_hyp_vecs_start)
> >      .rept 4
> >      vectors hyp_traps_vector
> >      .endr
> > -ENTRY(__bp_harden_hyp_vecs_end)
> > +GLOBAL(__bp_harden_hyp_vecs_end)
> > +END(__bp_harden_hyp_vecs_start)
>
> Shouldn't GLOBAL be changed to FUNC as well?
>

I was a bit unsure but went for GLOBAL since the _end labels point to
addresses after and outside of the code sequence.
But I don't have a strong opinion and am happy to change them to FUNC
if you feel that's better.

Cheers,
Edgar


>
> >  .macro mitigate_spectre_bhb_loop count
> > -ENTRY(__mitigate_spectre_bhb_loop_start_\count)
> > +FUNC(__mitigate_spectre_bhb_loop_start_\count)
> >      stp     x0, x1, [sp, #-16]!
> >      mov     x0, \count
> >  .Lspectre_bhb_loop\@:
> > @@ -68,11 +69,12 @@ ENTRY(__mitigate_spectre_bhb_loop_start_\count)
> >      b.ne    .Lspectre_bhb_loop\@
> >      sb
> >      ldp     x0, x1, [sp], #16
> > -ENTRY(__mitigate_spectre_bhb_loop_end_\count)
> > +GLOBAL(__mitigate_spectre_bhb_loop_end_\count)
>
> Also here?
>
>
> > +END(__mitigate_spectre_bhb_loop_start_\count)
> >  .endm
> >
> >  .macro smccc_workaround num smcc_id
> > -ENTRY(__smccc_workaround_smc_start_\num)
> > +FUNC(__smccc_workaround_smc_start_\num)
> >      sub     sp, sp, #(8 * 4)
> >      stp     x0, x1, [sp, #(8 * 2)]
> >      stp     x2, x3, [sp, #(8 * 0)]
> > @@ -81,13 +83,15 @@ ENTRY(__smccc_workaround_smc_start_\num)
> >      ldp     x2, x3, [sp, #(8 * 0)]
> >      ldp     x0, x1, [sp, #(8 * 2)]
> >      add     sp, sp, #(8 * 4)
> > -ENTRY(__smccc_workaround_smc_end_\num)
> > +GLOBAL(__smccc_workaround_smc_end_\num)
>
> And here?
>
>
> > +END(__smccc_workaround_smc_start_\num)
> >  .endm
> >
> > -ENTRY(__mitigate_spectre_bhb_clear_insn_start)
> > +FUNC(__mitigate_spectre_bhb_clear_insn_start)
> >      clearbhb
> >      isb
> > -ENTRY(__mitigate_spectre_bhb_clear_insn_end)
> > +GLOBAL(__mitigate_spectre_bhb_clear_insn_end)
>
> and here?
>
>
> > +END(__mitigate_spectre_bhb_clear_insn_start)
> >
> >  mitigate_spectre_bhb_loop 8
> >  mitigate_spectre_bhb_loop 24
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:21:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:21:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717641.1120047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yGn-0007As-9P; Mon, 06 May 2024 13:20:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717641.1120047; Mon, 06 May 2024 13:20:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yGn-0007Al-6B; Mon, 06 May 2024 13:20:41 +0000
Received: by outflank-mailman (input) for mailman id 717641;
 Mon, 06 May 2024 13:20:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3yGl-0007Af-Vv
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:20:39 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6e0d09c1-0bab-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 15:20:38 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51f57713684so2345161e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 06:20:38 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bi15-20020a05600c3d8f00b0041c24321934sm19905853wmb.41.2024.05.06.06.20.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 06:20:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e0d09c1-0bab-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715001638; x=1715606438; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6tpCpntxm/g/0s9FtaKk969D+NT36dYnCjWS35vjXoE=;
        b=VlWBHkLSVjlbVxOUao7uxKNgTP9qcNBTWCMmZDJnDaDdPvxkhoz1tqfEF2MkdKwDau
         mEfzmTz66PM5LQi18gCluxgup5ld+KsIEtvHVcnZC0kOzuLA80tn7JLDIEX0mTJiURkJ
         Y2fv2FbFVxVasuynkVVORDLPZFpF2jVK9YU53S4aEzEG+buTvLdlRAyJDtPGpxs4Gik+
         mWEOlMugIPpNcXiej9nDvc2C/3LsknhdT+NYbLbWnsfXZZzQmjh2pHpDDcqQdSxZWYUD
         xPwqAGBNnmVu8VwRT/bz/HBkBYRErHmkbrDbut/3YZ37jXjMzscjd8D0iNad48CpNgZv
         N7xw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715001638; x=1715606438;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6tpCpntxm/g/0s9FtaKk969D+NT36dYnCjWS35vjXoE=;
        b=O498RPNTLBn7u/qtd3iALdG1mMhrnnsw9xTRbD21GfqF5DliUu4AnEw7Rypo8lMU76
         Wzaw4jItObtJp6hbwPTJ49/B37f83R1SxxuY+wiJN8h+8ycjDkJOEaIf2XhoiMLSmqCd
         1RLpxXpJ7ULiAFFhz4YJss+XWQ4EqArDFaeLyhtUPWzKgiYtoVtrSVHrYL7wCpw39mBo
         MmGY0txCX8ExogHALlzW9T6Z2Q0QN2WifB59lqe1S2bfyoXzKxypBbp+631B9bEB68nW
         3BrC+xeCQ61jVRFTqQBmrMyV0h7EenO83RONK4pGJ9gb/GjHjgHIKct3eJauMJiU+vQr
         bJDQ==
X-Gm-Message-State: AOJu0YyX8gxuUO5QnqexH1Ak7A/JPRaSL32FHcMzyNH2TW4lm5C27++f
	ylNxS1CuPeYsizOM1470++l9DxKqV65IQP+MT+mHfSewz7liNY1STZCN5JanPg==
X-Google-Smtp-Source: AGHT+IFmObMCg8x9w17HSxTXZDj5JgpRBtQeNbZ2omns0SU6j0I0wbdOXN21W8UEtj9kj6+R3rB9Ww==
X-Received: by 2002:a19:644b:0:b0:51c:cd8d:2865 with SMTP id b11-20020a19644b000000b0051ccd8d2865mr6496432lfj.44.1715001638018;
        Mon, 06 May 2024 06:20:38 -0700 (PDT)
Message-ID: <24d52bbb-1329-4f8a-81be-505a35969875@suse.com>
Date: Mon, 6 May 2024 15:20:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com> <ZjjQIaxEwS6b-swj@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjjQIaxEwS6b-swj@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 14:42, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
>> Make the variable a tristate, with (as done elsewhere) a negative value
>> meaning "default". Since all use sites need looking at, also rename it
>> to match our usual "opt_*" pattern. While touching it, also move it to
>> .data.ro_after_init.
> 
> I guess I need to look at further patches, as given the feedback on
> the past version I think we agreed we want to set ATS unconditionally
> disabled by default, and hence I'm not sure I see the point of the
> tri-state if enabling ATS will require an explicit opt-in on the
> command line (ats=1).

With the present wording in the VT-d spec (which we've now had vague
indication that it may not be meant that way) there needs to be
tristate behavior:
- With "ats=0" ATS won't be used.
- With "ats=1" ATS will be used for all ATS-capable devices.
- Without either option ATS will be used for devices where firmware
  mandates its use.
If the alternative reading was confirmed (and preferably the text also
adjusted), we should be able to get away with a simple boolean again.
At which point this patch may end up being purely renaming, and hence
could then as well be left out.

>> The only place it retains boolean nature is pci_ats_device(), for now.
>>
>> In AMD code re-order conditionals to have the config space accesses
>> after (cheaper) flag checks.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> In domain_context_mapping_one() I'm a little puzzled that translation
>> type is selected based on only IOMMU and global properties, i.e. not
>> taking the device itself into account.
> 
> Is it maybe fine do set DEV_IOTLB unconditionally as long as the IOMMU
> supports it, and then let the device decide whether it wants to issue
> translated or non-translated requests depending on whether it supports
> (and enables) ATS?

This might be the reason why it is what it is now.

> I think it would be best to limit this strictly to devices that have
> ATS enabled.

And this was the reason for me putting the remark here.

>> --- a/xen/drivers/passthrough/amd/iommu_cmd.c
>> +++ b/xen/drivers/passthrough/amd/iommu_cmd.c
>> @@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
>>      struct amd_iommu *iommu;
>>      unsigned int req_id, queueid, maxpend;
>>  
>> -    if ( !ats_enabled )
>> +    if ( opt_ats <= 0 )
> 
> If having a tri-state is required, won't it be best to decide on a
> binary value at init time, so that runtime functions can handle
> opt_ats as a boolean?

As per above, unlike for other options we can't consolidate into a
boolean, as runtime behavior wants to be different with all three
possible settings.

> Otherwise we are open coding the default expectation of what -1
> implies (disabled) in all use sites.

That's pretty much unavoidable with the different meaning of 1 and -1.

>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>  
>>          if ( use_ats(pdev, iommu, ivrs_dev) )
>> -            dte->i = ats_enabled;
>> +            dte->i = true;
> 
> Might be easier to just use:
> 
> dte->i = use_ats(pdev, iommu, ivrs_dev);

I'm hesitant here, as in principle we might be overwriting a "true" by
"false" then.

>> @@ -257,7 +257,7 @@ static int __must_check amd_iommu_setup_
>>                                           ACPI_IVHD_SYSTEM_MGMT));
>>  
>>          if ( use_ats(pdev, iommu, ivrs_dev) )
>> -            ASSERT(dte->i == ats_enabled);
>> +            ASSERT(dte->i);
> 
> ASSERT(dte->i == use_ats(pdev, iommu, ivrs_dev));

I'm okay switching here, but better to the precise logical equivalent of
the earlier code:

ASSERT(dte->i || !use_ats(pdev, iommu, ivrs_dev));

>> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
>>  
>>  static inline int pci_ats_device(int seg, int bus, int devfn)
>>  {
>> -    if ( !ats_enabled )
>> +    if ( !opt_ats )
>>          return 0;
> 
> Can't you remove that check altogether now, since you are adding an
> opt_ats check to use_ats()?

Two reasons why not: For one this isn't AMD-specific code, and hence
shouldn't be tied to the AMD-specific use_ats(). In principle VT-d
code should be okay to call here, too. And then
amd_iommu_disable_domain_device() doesn't use use_ats(), but does call
here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:24:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:24:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717648.1120057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yKk-0008Mb-Oy; Mon, 06 May 2024 13:24:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717648.1120057; Mon, 06 May 2024 13:24:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yKk-0008MU-M4; Mon, 06 May 2024 13:24:46 +0000
Received: by outflank-mailman (input) for mailman id 717648;
 Mon, 06 May 2024 13:24:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lkt3=MJ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s3yKj-0008MO-Et
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:24:45 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id feafdc20-0bab-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 15:24:42 +0200 (CEST)
Received: from DS7PR05CA0009.namprd05.prod.outlook.com (2603:10b6:5:3b9::14)
 by DM6PR12MB4219.namprd12.prod.outlook.com (2603:10b6:5:217::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Mon, 6 May
 2024 13:24:38 +0000
Received: from DS1PEPF0001709D.namprd05.prod.outlook.com
 (2603:10b6:5:3b9:cafe::b3) by DS7PR05CA0009.outlook.office365.com
 (2603:10b6:5:3b9::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Mon, 6 May 2024 13:24:38 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS1PEPF0001709D.mail.protection.outlook.com (10.167.18.107) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 13:24:38 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 08:24:37 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 08:24:36 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: feafdc20-0bab-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VUe7yHXCpA2Z6emPQnmF8zbqoy2MaJ8fKCwd6J0f1QKXRdejTq39rD+YyKli7iKJCieAtii08kNb6Qxk2r9/zU1sc0EG2Qtmw3KkPFAvBMUHUUDTEOfvRr4AwteJ4RpOrqTSxKD9MoZpLMEewPw/MMflCaGYfDGdsWcEMFn6Q6/eCbIKviLVvBArvKwvbaMFYUNDFqq79F36aba+7NdRAVoZ1NlkMDcOvIipBXOSR/J41OfN71cnN6YQowMXzLdgKjdZTboxb3GVy7cB2/0VMtw3jTf5F8nlk7g7t1Iod686FzTnPk9HwgRfeN4cPtaRorewkC/dmKWdJzpZ3bmksQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=oRoXz1QYnq0OFNHm9Okr8mA1DLAHSVfztTFO/qbJjbw=;
 b=DPISDOhe1SfiNBsN7eQh9Uj3NN2oTFSOh/pR5KFW/gmxlwk6qzVT1CUkLBx+8wPD9qKhpNfEoGL9BbIvd5gWHZrx802BucupTZWN4Z+ukyzeXgOYZgypCn+4qtvM6XIPD/o62ambgEUlY82s8xsE+pDAEp5KiBI9kwT/nt4INk4Q5MOMfVLomzbTfhcFGdIYU5H1tJRCh8o9E1nz+4KV23yYR35zzkNURyYJaLHxrI2PC29X2vTwk47gXcGDYW3BENPEamNAfoKbuivgCi1SYzkyPX8Re80BPWivg2j7lCgGWUTfX6rbkOfUNreblwZFCu6zN5KiY9LdhMIvQA54yA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oRoXz1QYnq0OFNHm9Okr8mA1DLAHSVfztTFO/qbJjbw=;
 b=HGS1XuwARMs8me1nOCRMCWPmFvftYqWtl07XclUAkOztTuMhP/vJsrTDtkYKrKss39JArgTukw9qznLUePi3PZLutflk8/nPpfEHxiqhyZyo9sSMJaQe4jv48V94T4K3JVHTnprYa9dUDcLMn/BP5NPKk/fwkDzNPcqnTfZ1qvg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <28322f07-99fc-4428-bcf2-b8c2af04d38b@amd.com>
Date: Mon, 6 May 2024 15:24:36 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-2-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-2-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF0001709D:EE_|DM6PR12MB4219:EE_
X-MS-Office365-Filtering-Correlation-Id: 51c5e88d-a379-4e9a-4bb1-08dc6dcfe14b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QW9YWmozTkRLNWdKL08ycFJSdjdlZklPUFBNV09ZOFhZTHJyRFl3b0dSUUd6?=
 =?utf-8?B?d3g2RmFvQXU0dTJGRDdOYkVDd2Z0a2FrTzZYQzJBSnpHeVdHTkJhMEJjUVlU?=
 =?utf-8?B?M01YUlZMWXVHMEJJdjlxcGY0OUY2cWx6enRVWlQzK0J5RUZoSjVzK2JDVUJ2?=
 =?utf-8?B?d2dRUHZDTTdOU1V2RW9hTjNBVWc0cFhDM2RoZHg4ekFKdVE4dVkwT0ttcnls?=
 =?utf-8?B?dytEaVdBYmhUb3V1RHhLdG81MHZMdWlrYVFuT1Q0NEUzd2dYL1JuNVNNTEpl?=
 =?utf-8?B?enZaTWRRU0kxQUg3QW1MTURXRnhYWEhNdWhhQ2Fra2lha2pCNVZ0cVlCallF?=
 =?utf-8?B?Q2RoQjc2MmdYNHZLNURDcmk0eXlQaUNXTDM1Mm1hOFBJWkNZbktTTDRKRS96?=
 =?utf-8?B?YjdaTUlkTlkyb2k4WnJQeUkyTUkwU3lCd08xb0lOU25oY0ZFaEd0MGFQU1My?=
 =?utf-8?B?ZURRcXVEYTNETjdsV2xRanNrUmF6bkFhdjhtNUF1cnBibFd3TWRJaXJjR2sv?=
 =?utf-8?B?WVBuck1PazdwblNBaUdiMjBpVG81R0FTUElMeW1mTERwOHdMU3lsM04zYk9p?=
 =?utf-8?B?TVBrcDBSRVBDSFRmRG4zSktZRTRwM1licFZHcnRhTE5saDVvMmJ1aFFxNU1h?=
 =?utf-8?B?NFNwRVR2aVZHckhINi9JNGcrQWlUREJ1YllxT1ZMd2J0US96NGp6R3YzTDdz?=
 =?utf-8?B?RGd3RjZUd2xnMEMrQll0bnl0b0dtaTYrN3ZHOGJqUWNMNlpRcTQxVDUvU1BW?=
 =?utf-8?B?c3hBcXpjdW1SZlM4dG4wUk9TZ29xVHVmOUQrYjNWL0Rwc2MySkRVSGdwUGV6?=
 =?utf-8?B?MDRUeUdNVG01Ky9kRlhnU0huaHM3d01YUHdYellDTEwrU1VoY1FQT1JrZ1FB?=
 =?utf-8?B?eVBxRHl0SFlPUzRnQlluMWNEZnl1WFlkSDA0VDlsSEZLQUFDak9Rck0yOEd6?=
 =?utf-8?B?SkpPdEpGd0xpQWFBRm5CYVlHSk9zNjdXeWJDWlJJbTJDMU1sZncvWU1ERkhn?=
 =?utf-8?B?eEVHdHdmRExJWjZDVXNpeENCV2tXVVFxbXZxaUVOY1Y2RDBhR1ZiTTJrelJL?=
 =?utf-8?B?YjRXMERkNTYxWTM3R08yQ2pkY3poNVliNXhQSWpqQmltSjlnMFcvWXNLdEVr?=
 =?utf-8?B?YVJsVWR6dEJpZzQ0ZHRFd1dlVCtkYWxSWnUzL1B1WFh5R0xUQjF6SUNCYzFZ?=
 =?utf-8?B?UWttOFpkRWRhUE00dzk4aFV3SkNCcFN3RlZXUGh3SytLNVdrb1BzWm1UYzls?=
 =?utf-8?B?azN4ZzFKMnlWbVJDVjRHKzAzVzRXSHBYQXdOb2hFSVJiNU11R3A3TDR0MlZi?=
 =?utf-8?B?TURQOGhaVlZsSlFNdG1zenh4cEpFaEFmVlJYcjc0cy94ZzNMbytpZGY5MURT?=
 =?utf-8?B?aXN1dURPY1dwL2tBQzk3WmxpaVdyNXdzQnVMN1kwOXFxdVkvTy9SMytQUzk4?=
 =?utf-8?B?ZjNzaTZBM0RFaFRjY3R5RnoyVmJtWHF6WTVsaDg2WnhGTExWbjZzYnUvMFpr?=
 =?utf-8?B?TXFZajZVb0xWbHFROElUVkpsTFg3aTFGRENGQUhkWTdoMEJKUW00YlZOcW0v?=
 =?utf-8?B?Tm1HNXhEUFFRN2UrZXJDU2VTZVl3L05pbktSRUVLdDRDSERGU1hyQWZNWWNV?=
 =?utf-8?B?NTR6Z2U4TTZwbGVDdFUwYTl4OWEra0taOW9tNW5HV1Z6ODZtUmh1R3JVcjBD?=
 =?utf-8?B?REhnb2p5a1I4L2tSUnV3aFhqRDEzbVdQSDBVWklhOEVkQW9BNmlyK2lkN3p1?=
 =?utf-8?B?VEdielBVa3FJbHFNQmd1cEY1SzJkNjVvbGJ2SXQ5UTlTRUljdkFIQzVudnA3?=
 =?utf-8?B?djNrbldKc3ZualZsRUZ2bURFUVlFL1NnYnNIbVdUMFBNd1lYZCtKYVRiNXhP?=
 =?utf-8?Q?uXG7XP3qgu621?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 13:24:38.4452
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 51c5e88d-a379-4e9a-4bb1-08dc6dcfe14b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF0001709D.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4219

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> The current static shared memory code is using bootinfo banks when it
> needs to find the number of borrower, so every time assign_shared_memory
s/borrower/borrowers

> is called, the bank is searched in the bootinfo.shmem structure.
> 
> There is nothing wrong with it, however the bank can be used also to
> retrieve the start address and size and also to pass less argument to
> assign_shared_memory. When retrieving the information from the bootinfo
> bank, it's also possible to move the checks on alignment to
> process_shm_node in the early stages.
Is this change really required for what you want to achieve? At the moment the alignment checks
are done before first use, which requires these values to be aligned. FDT processing part does not need it.

> 
> So create a new function find_shm() which takes a 'struct shared_meminfo'
Can we name it find_shm_bank() or find_shm_bank_by_id()?
I agree that it's better to use a unique ID rather than matching by address/size

> structure and the shared memory ID, to look for a bank with a matching ID,
> take the physical host address and size from the bank, pass the bank to
> assign_shared_memory() removing the now unnecessary arguments and finally
> remove the acquire_nr_borrower_domain() function since now the information
> can be extracted from the passed bank.
> Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
> case of errors (unlikely), as said above, move the checks on alignment
> to process_shm_node.
> 
> Drawback of this change is that now the bootinfo are used also when the
> bank doesn't need to be allocated, however it will be convinient later
> to use it as an argument for assign_shared_memory when dealing with
> the use case where the Host physical address is not supplied by the user.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  xen/arch/arm/static-shmem.c | 105 ++++++++++++++++++++----------------
>  1 file changed, 58 insertions(+), 47 deletions(-)
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 09f474ec6050..f6cf74e58a83 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -19,29 +19,24 @@ static void __init __maybe_unused build_assertions(void)
>                   offsetof(struct shared_meminfo, bank)));
>  }
> 
> -static int __init acquire_nr_borrower_domain(struct domain *d,
> -                                             paddr_t pbase, paddr_t psize,
> -                                             unsigned long *nr_borrowers)
> +static const struct membank __init *find_shm(const struct membanks *shmem,
> +                                             const char *shm_id)
>  {
> -    const struct membanks *shmem = bootinfo_get_shmem();
>      unsigned int bank;
> 
> -    /* Iterate reserved memory to find requested shm bank. */
> +    BUG_ON(!shmem || !shm_id);
Is it really necessary? For example, before calling find_shm(), strlen is used on shm_id

> +
>      for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
>      {
> -        paddr_t bank_start = shmem->bank[bank].start;
> -        paddr_t bank_size = shmem->bank[bank].size;
> -
> -        if ( (pbase == bank_start) && (psize == bank_size) )
> +        if ( strncmp(shm_id, shmem->bank[bank].shmem_extra->shm_id,
> +                     MAX_SHM_ID_LENGTH) == 0 )
Why not strcmp? AFAICS it's been validated many times already

>              break;
>      }
> 
>      if ( bank == shmem->nr_banks )
> -        return -ENOENT;
> -
> -    *nr_borrowers = shmem->bank[bank].shmem_extra->nr_shm_borrowers;
> +        return NULL;
> 
> -    return 0;
> +    return &shmem->bank[bank];
>  }
> 
>  /*
> @@ -103,14 +98,20 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
>      return smfn;
>  }
> 
> -static int __init assign_shared_memory(struct domain *d,
> -                                       paddr_t pbase, paddr_t psize,
> -                                       paddr_t gbase)
> +static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
> +                                       const struct membank *shm_bank)
>  {
>      mfn_t smfn;
>      int ret = 0;
>      unsigned long nr_pages, nr_borrowers, i;
>      struct page_info *page;
> +    paddr_t pbase, psize;
> +
> +    BUG_ON(!shm_bank || !shm_bank->shmem_extra);
Is it really necessary? Isn't shm_bank already validated? It's not very common to have NULL checks in internal functions.

> +
> +    pbase = shm_bank->start;
> +    psize = shm_bank->size;
> +    nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
> 
>      printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
>             d, pbase, pbase + psize);
> @@ -135,14 +136,6 @@ static int __init assign_shared_memory(struct domain *d,
>          }
>      }
> 
> -    /*
> -     * Get the right amount of references per page, which is the number of
> -     * borrower domains.
> -     */
> -    ret = acquire_nr_borrower_domain(d, pbase, psize, &nr_borrowers);
> -    if ( ret )
> -        return ret;
> -
>      /*
>       * Instead of letting borrower domain get a page ref, we add as many
>       * additional reference as the number of borrowers when the owner
> @@ -199,6 +192,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
> 
>      dt_for_each_child_node(node, shm_node)
>      {
> +        const struct membank *boot_shm_bank;
>          const struct dt_property *prop;
>          const __be32 *cells;
>          uint32_t addr_cells, size_cells;
> @@ -212,6 +206,23 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
>              continue;
> 
> +        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
> +        {
> +            printk("%pd: invalid \"xen,shm-id\" property", d);
> +            return -EINVAL;
> +        }
> +        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
> +
> +        boot_shm_bank = find_shm(bootinfo_get_shmem(), shm_id);
> +        if ( !boot_shm_bank )
> +        {
> +            printk("%pd: static shared memory bank not found: '%s'", d, shm_id);
> +            return -ENOENT;
> +        }
> +
> +        pbase = boot_shm_bank->start;
> +        psize = boot_shm_bank->size;
> +
>          /*
>           * xen,shared-mem = <pbase, gbase, size>;
>           * TODO: pbase is optional.
> @@ -221,20 +232,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
>          BUG_ON(!prop);
>          cells = (const __be32 *)prop->value;
> -        device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
> -        psize = dt_read_paddr(cells, size_cells);
> -        if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
> -        {
> -            printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
> -                   d, pbase, gbase);
> -            return -EINVAL;
> -        }
> -        if ( !IS_ALIGNED(psize, PAGE_SIZE) )
> -        {
> -            printk("%pd: size 0x%"PRIpaddr" is not suitably aligned\n",
> -                   d, psize);
> -            return -EINVAL;
> -        }
> +        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
> 
>          for ( i = 0; i < PFN_DOWN(psize); i++ )
>              if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
> @@ -251,13 +249,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
>              owner_dom_io = false;
> 
> -        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
> -        {
> -            printk("%pd: invalid \"xen,shm-id\" property", d);
> -            return -EINVAL;
> -        }
> -        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
> -
>          /*
>           * DOMID_IO is a fake domain and is not described in the Device-Tree.
>           * Therefore when the owner of the shared region is DOMID_IO, we will
> @@ -270,8 +261,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>               * We found the first borrower of the region, the owner was not
>               * specified, so they should be assigned to dom_io.
>               */
> -            ret = assign_shared_memory(owner_dom_io ? dom_io : d,
> -                                       pbase, psize, gbase);
> +            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
> +                                       boot_shm_bank);
>              if ( ret )
>                  return ret;
>          }
> @@ -440,6 +431,26 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>      device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
>      size = dt_next_cell(size_cells, &cell);
> 
> +    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
> +    {
> +        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
> +               paddr);
> +        return -EINVAL;
> +    }
> +
> +    if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
> +    {
> +        printk("fdt: guest address 0x%"PRIpaddr" is not suitably aligned.\n",
> +               gaddr);
> +        return -EINVAL;
> +    }
> +
> +    if ( !IS_ALIGNED(size, PAGE_SIZE) )
What sense does it make to check for size being aligned before checking for size being 0? It would pass this check.

> +    {
> +        printk("fdt: size 0x%"PRIpaddr" is not suitably aligned\n", size);
> +        return -EINVAL;
> +    }
> +
>      if ( !size )
>      {
>          printk("fdt: the size for static shared memory region can not be zero\n");
> --
> 2.34.1
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:25:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:25:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717649.1120066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yKq-0000BO-Ue; Mon, 06 May 2024 13:24:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717649.1120066; Mon, 06 May 2024 13:24:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yKq-0000BH-S4; Mon, 06 May 2024 13:24:52 +0000
Received: by outflank-mailman (input) for mailman id 717649;
 Mon, 06 May 2024 13:24:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3yKp-0008MO-IH
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:24:51 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 03ce3cc9-0bac-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 15:24:49 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51f8211c588so1926150e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 06:24:49 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h11-20020adfe98b000000b00343d1d09550sm10711581wrm.60.2024.05.06.06.24.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 06:24:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03ce3cc9-0bac-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715001889; x=1715606689; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KiJBdPPRY/NUJaEPZJ4xZh1WqKJ1mjEkLa8gfRm5A6Y=;
        b=UlVSrjDVNUhau509H15lwlJvQ8V23Dl+RDWxuvaYw0iXwv6S+vW6aSxFu6CRHwb0dN
         ILjGkSug+ZpHNS1pcgN/LHGetnGNY4LdIfW6S0jrnZVJSHDIAyWDBT1VhN1hMQ5sMsMw
         121JVGgVGBgd7tQFkPIcQmm0jKWWVIUyMsZYwZgdhZHdvfFQcbRfxrfjkfRf7jHlnBTx
         kaIhKacBa2N6jYh38MD0rK2WdTenvzLvyoodtVUT8BqOGD2XR7Zz2AMoAV3vFW0DGsta
         W03h2GzshuKfKiKS5qUbcvZvTwSBYL+LTLFolhbB5MMseEWidQCuBJGm9b1FVHOYOscO
         rfRw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715001889; x=1715606689;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KiJBdPPRY/NUJaEPZJ4xZh1WqKJ1mjEkLa8gfRm5A6Y=;
        b=CqWM0/E4FLVymoY/L4r8v6DVPX/qFfgbVPXvm3mtqRxBBdphtKdCCFRvgZwCcTam+B
         o/J4ntQoKHfs5VWBvU5zykiDfydBW71YypB1sRVCrKqTlN7Hqtkcf9TmwHFt3f+00lI4
         AqGUgrRH8jXJG/fsoUSafu67p6qUGTJ0T+RCadDcdUaFjduAyVkjr+fLPDVkNhh2HBy6
         it56Cpek7GyJNGdtY3C6hdZ4WYpI6tx33tXDBNyhrsOwVn6sPTAoxoRreYDQia1D4A3v
         iIpOg6sZ3k6Lqm4R23f9JVfkBxOpDtAR6pFCzGpM5uKQXPNUxImICO+wnxYMjVC9neg/
         CVig==
X-Forwarded-Encrypted: i=1; AJvYcCVpkaXL7RVUpF+RmQj8BTkqWfBVE3FHzLsztDidophgDhFMfeudixFn2raQNg+YBUYDpXWfY3bFvlLpjSeFu7pwXbAWVaTaaiwSuVpYmZ0=
X-Gm-Message-State: AOJu0YwI87VRruSQBmBSYHYIth2K/ScVebytgzjRpDtA2zf6huQykoGf
	cA1Ad8jeZdFhPyjVJNjsc/sNVHoXKELfZFNOwniXlVe5q4ZhaOfUHJHLGHDH5w==
X-Google-Smtp-Source: AGHT+IGewT+MutigeIuDr5hJAY1fGM1J73lQvtbidYmsNdhZ9inG1Z/Zt9Hiebjk7BgodK5Iz+Y8SA==
X-Received: by 2002:ac2:4959:0:b0:51d:abb3:d701 with SMTP id o25-20020ac24959000000b0051dabb3d701mr6568286lfi.5.1715001889409;
        Mon, 06 May 2024 06:24:49 -0700 (PDT)
Message-ID: <01bd95f8-9122-41dd-9bc4-0fb6bbea1f6c@suse.com>
Date: Mon, 6 May 2024 15:24:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 11/13] xen/arm: add Xen cache colors command line
 parameter
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Luca Miccio <lucmiccio@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-12-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240502165533.319988-12-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.05.2024 18:55, Carlo Nonato wrote:
> From: Luca Miccio <lucmiccio@gmail.com>
> 
> Add a new command line parameter to configure Xen cache colors.
> These colors are dumped together with other coloring info.
> 
> Benchmarking the VM interrupt response time provides an estimation of
> LLC usage by Xen's most latency-critical runtime task. Results on Arm
> Cortex-A53 on Xilinx Zynq UltraScale+ XCZU9EG show that one color, which
> reserves 64 KiB of L2, is enough to attain best responsiveness:
> - Xen 1 color latency:  3.1 us
> - Xen 2 color latency:  3.1 us
> 
> Since this is the most common target for Arm cache coloring, the default
> amount of Xen colors is set to one.
> 
> More colors are instead very likely to be needed on processors whose L1
> cache is physically-indexed and physically-tagged, such as Cortex-A57.
> In such cases, coloring applies to L1 also, and there typically are two
> distinct L1-colors. Therefore, reserving only one color for Xen would
> senselessly partitions a cache memory that is already private, i.e.
> underutilize it.
> 
> Signed-off-by: Luca Miccio <lucmiccio@gmail.com>
> Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 06 13:26:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:26:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717655.1120076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yMN-00011w-CI; Mon, 06 May 2024 13:26:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717655.1120076; Mon, 06 May 2024 13:26:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yMN-00011p-9U; Mon, 06 May 2024 13:26:27 +0000
Received: by outflank-mailman (input) for mailman id 717655;
 Mon, 06 May 2024 13:26:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3cb6=MJ=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s3yMM-00011h-59
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:26:26 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3ca3daf9-0bac-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 15:26:25 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59a9d66a51so405880066b.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 06:26:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ca3daf9-0bac-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715001985; x=1715606785; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iXjEFMRzTzj3KldPoNPBs9HkkDMftiy61B6AF2CyXKk=;
        b=NMZC1GpRiuCI2vCfxAUFcjCQ0ai0CZCBEWEMHIL5Jyna42GvTYu6j+grOkcz8nex45
         WGrU/LFW5MbSUDvPGTpmLpPufZYp0JvlCRdyHjVRzRwZqkiPyfkQff3dagPedyXZZnNW
         ys5MolhMHWUoS4XU6TYUzmdc2CW9g5jVYFb8jV6Da0TGw4I7V28ZSBTrYNUNzCH4LW+I
         rKd8EdKrNvqEoC2vni71JLyvtUDoRzdEdvEBvh7LL74w3HHGlBl1i2+AMoI6WJRp2POJ
         8OpK+BUaQyh33SW51zHV8ZZcFdnxoEpFZUZh/9jodkNQwzAsvE/i7TqJU0KYx7biSS4u
         STvg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715001985; x=1715606785;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iXjEFMRzTzj3KldPoNPBs9HkkDMftiy61B6AF2CyXKk=;
        b=t37E/aBt5EqsHr50KWPEMZ5uGcBJzPpklK3c2EOXkYBCya+D9NCHemQAmHtej0jOEy
         DoVIv4X1W8ETFzkyLSu8gGWGRFqrSFLyoQJRSOaaaPrLiV63Hewv7JsBb6R+2M4m2Ier
         foGJSj//GtdA8VNJmo84TUhNZH9BuAKucioREd4CXJF9L3QIPd2Szwd1DqKWRiOw9EjS
         bJUKVtdeECoQiectg5Nw8w4pqTd3hCNsN9/vJ+MtZl4+3PTpQbU66vTVoM2UeYGz6eE+
         5tCki/FZk+b7rEogTS6icQfbdR9GMOXaEV4r/RlRfiWoWuHju6aEXXdb+qqhwzC0nPal
         91BQ==
X-Forwarded-Encrypted: i=1; AJvYcCUlkDRQfdCwioVLT6FdY3b532qTrKnKQfT47f5ZVpE3qwkXu8mfBBJNNSmocGkdTVxxNfazG2upMd3YLOFP1rknG1paBFb5Ea6ezxJ0iyU=
X-Gm-Message-State: AOJu0YylnkMZQlvgGq4Rj4NIiGKNzjouFLc/VdM+WDTZDdl3STR1gCYT
	rqsfHf1PMG5l586ZDOn4ESjBMNGqdNKZ8ARRCB3EGgmzwwkeyHc2eZneIDgXzObFr+fXgeXkr5G
	6OVjg7dNMUrN1DxAknQoUTxDcymc=
X-Google-Smtp-Source: AGHT+IFjRGcd4vevXd4YmEJIpnLbj+EtQPldgB+Qq/+tP0OUO7ic/p9jXzqaReEl+6jNOvGvJR1QQdDnYUkvaMPNk/A=
X-Received: by 2002:a50:d495:0:b0:56d:c928:ad76 with SMTP id
 s21-20020a50d495000000b0056dc928ad76mr6259743edi.26.1715001984479; Mon, 06
 May 2024 06:26:24 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-15-edgar.iglesias@gmail.com> <0263b7e8-5800-4f5a-9dc5-bc2b4fbbbbfe@redhat.com>
 <d4008042-6f68-42f6-8ade-fbb0ba115a33@linaro.org>
In-Reply-To: <d4008042-6f68-42f6-8ade-fbb0ba115a33@linaro.org>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Mon, 6 May 2024 15:26:12 +0200
Message-ID: <CAJy5ezoVOR-OB7HVEAAkmwdC7jerCVatNS=VdP1jTMZpgUm_fg@mail.gmail.com>
Subject: Re: [PATCH v4 14/17] xen: Add xen_mr_is_memory()
To: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>
Cc: David Hildenbrand <david@redhat.com>, qemu-devel@nongnu.org, sstabellini@kernel.org, 
	jgross@suse.com, "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 11:59=E2=80=AFAM Philippe Mathieu-Daud=C3=A9
<philmd@linaro.org> wrote:
>
> On 2/5/24 09:26, David Hildenbrand wrote:
> > On 30.04.24 18:49, Edgar E. Iglesias wrote:
> >> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >>
> >> Add xen_mr_is_memory() to abstract away tests for the
> >> xen_memory MR.
> >>
> >> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> >> ---
> >
> > [...]
> >
> >>   #endif
> >> diff --git a/system/physmem.c b/system/physmem.c
> >> index ad7a8c7d95..1a5ffcba2a 100644
> >> --- a/system/physmem.c
> >> +++ b/system/physmem.c
> >> @@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock
> >> *block, ram_addr_t addr,
> >>            * because we don't want to map the entire memory in QEMU.
> >>            * In that case just map the requested area.
> >>            */
> >> -        if (block->offset =3D=3D 0) {
> >> +        if (xen_mr_is_memory(block->mr)) {
> >>               return xen_map_cache(block->mr, addr, len, lock, lock,
> >>                                    is_write);
> >>           }
> >
> > I'd have moved that into a separate patch, because this is not a simple
> > abstraction here.
>
> Yes please, maybe using Stefano review comment in the description.
>

Thanks, for v5 I've split out this particular change into a separate patch:

    softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

    For xen, when checking for the first RAM (xen_memory), use
    xen_mr_is_memory() rather than checking for a RAMBlock with
    offset 0.

    All Xen machines create xen_memory first so this has no
    functional change for existing machines.

    Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

diff --git a/system/physmem.c b/system/physmem.c
index ad7a8c7d95..1a5ffcba2a 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock
*block, ram_addr_t addr,
          * because we don't want to map the entire memory in QEMU.
          * In that case just map the requested area.
          */
-        if (block->offset =3D=3D 0) {
+        if (xen_mr_is_memory(block->mr)) {
             return xen_map_cache(block->mr, addr, len, lock, lock,
                                  is_write);
         }



> >
> > Acked-by: David Hildenbrand <david@redhat.com>
> >
>


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:38:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:38:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717664.1120086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yYN-0003X2-DE; Mon, 06 May 2024 13:38:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717664.1120086; Mon, 06 May 2024 13:38:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yYN-0003Wv-Ab; Mon, 06 May 2024 13:38:51 +0000
Received: by outflank-mailman (input) for mailman id 717664;
 Mon, 06 May 2024 13:38:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3yYM-0003Wp-51
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:38:50 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f74b8557-0bad-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 15:38:47 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41e82b78387so12843775e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 06:38:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 u9-20020adfa189000000b0034bc5934bf8sm10759564wru.31.2024.05.06.06.38.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 06:38:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f74b8557-0bad-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715002727; x=1715607527; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=WA3PJewGWcJ0vSMf1oiWHFOn6W9zm43WZOEYmSc5W+k=;
        b=tt1VfIEtxULEd+5P2mzS4rDGezvFC3On1FU8e4eJISmSR0U/xoJgTbph1J7IAo+8qt
         wDKB7lysewwy6r+maml88vhe58Qo7QsdYeglUiP/m/EkW8hzAS/cAKA8w7exr7jHnrDp
         qXTvKdgZZqQVMM2un7E+M8ny+X0QLaOVy4HVQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715002727; x=1715607527;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WA3PJewGWcJ0vSMf1oiWHFOn6W9zm43WZOEYmSc5W+k=;
        b=d5xo+Y4aiGuJ6t/koi97OQDpbsNqGyJgecA3M88Sarig6lmYgOiATPrBfhST3I/Sga
         fL9cdYLSQkfjPIKHyqgzp18Heq3s5wA9mPOXCcdD6uIWoBpVVvC5iJGa7XJuFAR4j8xS
         q9UsRVC4AN2+GeU/j8XAeIgbgAqlVlfP3d60yqZbHHmJokZqJn8rj1qWWl+3JyzHQ2F3
         OJxsikDzHVjg9iy1PtLowFqBoDEJ30v09GBScyIZ4Cz0pLOUIAuGrOdbfAYshri/Dj7R
         miPknesheeWA0bKvEsmP9T3wtECu1tELGQnQvSrEopESl05Gn9IyjqnGqbDTEbNZe4Ff
         F5Ow==
X-Gm-Message-State: AOJu0YxOWdtDqQhJgR87dZOmfTKFaLIib/z+BPRpx8jUw7bRFcuUhfKj
	nmoQY4o8ZLhL5DZnQhyTNH5IFq5yIPbf6QuAqcsf/UPE8tg15KJg6fb+/sQLK3lJiK3UbyiOe4R
	B
X-Google-Smtp-Source: AGHT+IEo/chnw6rYpHAYwTydwUAMg61gZeznYY3GDrNYu59TykV32hwiqFrNrKpksXj9V7I/PI1bLA==
X-Received: by 2002:a05:6000:49:b0:34a:3f3d:bb22 with SMTP id k9-20020a056000004900b0034a3f3dbb22mr8007514wrx.26.1715002727272;
        Mon, 06 May 2024 06:38:47 -0700 (PDT)
Date: Mon, 6 May 2024 15:38:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 06/12] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Message-ID: <ZjjdZRPluS0YIazc@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com>

On Thu, Feb 15, 2024 at 11:16:11AM +0100, Jan Beulich wrote:
> When the flag is set, permit Dom0 to control the device (no worse than
> what we had before and in line with other "best effort" behavior we use
> when it comes to Dom0),

I think we should somehow be able to signal dom0 that this device
might not operate as expected, otherwise dom0 might use it and the
device could silently malfunction due to ATS not being enabled.

Otherwise we should just hide the device from dom0.

I assume setting the IOMMU context entry to passthrough mode would
also be fine for such devices that require ATS?

> but suppress passing through to DomU-s unless
> ATS can actually be enabled for such devices (and was explicitly enabled
> on the command line).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v2: Re-base over new earlier patches.
> 
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -225,7 +225,11 @@ exceptions (watchdog NMIs and unexpected
>  > Default: `false`
>  
>  Permits Xen to set up and use PCI Address Translation Services.  This is a
> -performance optimisation for PCI Passthrough.
> +performance optimisation for PCI Passthrough.  Note that firmware may indicate
> +that certain devices need to have ATS enabled for proper operation. For such
> +devices ATS will be enabled by default, unless the option is used in its
> +negative form.  Such devices will still not be eligible for passing through to
> +guests, unless the option is used in its positive form.
>  
>  **WARNING: Xen cannot currently safely use ATS because of its synchronous wait
>  loops for Queued Invalidation completions.**
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -253,6 +253,24 @@ struct acpi_atsr_unit *acpi_find_matched
>      return all_ports;
>  }
>  
> +const struct acpi_satc_unit *acpi_find_matched_satc_unit(
> +    const struct pci_dev *pdev)
> +{
> +    const struct acpi_satc_unit *satc;
> +
> +    list_for_each_entry ( satc, &acpi_satc_units, list )
> +    {
> +        if ( satc->segment != pdev->seg )
> +            continue;
> +
> +        for ( unsigned int i = 0; i < satc->scope.devices_cnt; ++i )
> +            if ( satc->scope.devices[i] == pdev->sbdf.bdf )
> +                return satc;
> +    }
> +
> +    return NULL;
> +}
> +
>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd)
>  {
>      struct acpi_rhsa_unit *rhsa;
> --- a/xen/drivers/passthrough/vtd/dmar.h
> +++ b/xen/drivers/passthrough/vtd/dmar.h
> @@ -112,6 +112,8 @@ struct acpi_satc_unit {
>  
>  struct acpi_drhd_unit *acpi_find_matched_drhd_unit(const struct pci_dev *);
>  struct acpi_atsr_unit *acpi_find_matched_atsr_unit(const struct pci_dev *);
> +const struct acpi_satc_unit *acpi_find_matched_satc_unit(
> +    const struct pci_dev *pdev);
>  
>  #define DMAR_TYPE 1
>  #define RMRR_TYPE 2
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2364,6 +2364,26 @@ static int cf_check intel_iommu_add_devi
>      if ( ret )
>          dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
>                  pdev->domain);
> +    else if ( !pdev->broken )
> +    {
> +        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
> +        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
> +
> +        /*
> +         * Prevent the device from getting assigned to an unprivileged domain
> +         * when firmware indicates ATS is required, but ATS could not be enabled
> +         * or was not explicitly enabled via command line option.
> +         */
> +        if ( satc && satc->atc_required &&
> +             (!drhd || ats_device(pdev, drhd) <= 0 ||
> +              !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) ||
> +              opt_ats < 0) )

Do you need the opt_ats check here?

I don't think it's possible for pci_ats_enabled() to return true if
opt_ats is <= 0, and hence the opt_ats < 0 check can be dropped from
the conditional?

> +        {
> +            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
> +                   &pdev->sbdf);
> +            pdev->broken = true;
> +        }
> +    }
>  
>      return ret;
>  }
> @@ -2375,12 +2395,26 @@ static int cf_check intel_iommu_enable_d
>  
>      pci_vtd_quirk(pdev);
>  
> -    if ( ret <= 0 )
> -        return ret;
> +    if ( ret <= 0 ||
> +         (ret = enable_ats_device(pdev, &drhd->iommu->ats_devices)) < 0 ||
> +         opt_ats < 0 )

Shouldn't this be opt_ats <= 0?

> +    {
> +        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
> +
> +        /*
> +         * Besides in error cases also prevent the device from getting assigned
> +         * to an unprivileged domain when firmware indicates ATS is required,
> +         * but ATS use was not explicitly enabled via command line option.
> +         */
> +        if ( satc && satc->atc_required && !pdev->broken )
> +        {
> +            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
> +                   &pdev->sbdf);
> +            pdev->broken = true;
> +        }

I think the code here could be easier to read if this was put in an
label at the end, and the early return above that you remove becomes a
goto.  But that's a question of taste.

> +    }
>  
> -    ret = enable_ats_device(pdev, &drhd->iommu->ats_devices);
> -
> -    return ret >= 0 ? 0 : ret;
> +    return ret <= 0 ? ret : 0;
>  }
>  
>  static int cf_check intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
> --- a/xen/drivers/passthrough/vtd/x86/ats.c
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c
> @@ -45,8 +45,9 @@ int ats_device(const struct pci_dev *pde
>  {
>      struct acpi_drhd_unit *ats_drhd;
>      unsigned int pos, expfl = 0;
> +    const struct acpi_satc_unit *satc;
>  
> -    if ( opt_ats <= 0 || !iommu_qinval )
> +    if ( !opt_ats || !iommu_qinval )
>          return 0;

FWIW, I find this change confusing, hence my request earlier that
opt_ats must be set to 0 or 1 by the point it gets used.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:39:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:39:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717667.1120096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yZH-00042I-Lg; Mon, 06 May 2024 13:39:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717667.1120096; Mon, 06 May 2024 13:39:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yZH-00042B-Iv; Mon, 06 May 2024 13:39:47 +0000
Received: by outflank-mailman (input) for mailman id 717667;
 Mon, 06 May 2024 13:39:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lkt3=MJ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s3yZG-00040r-4a
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:39:46 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on20631.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 183db192-0bae-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 15:39:44 +0200 (CEST)
Received: from CH2PR12CA0011.namprd12.prod.outlook.com (2603:10b6:610:57::21)
 by PH7PR12MB5854.namprd12.prod.outlook.com (2603:10b6:510:1d5::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Mon, 6 May
 2024 13:39:40 +0000
Received: from CH1PEPF0000A347.namprd04.prod.outlook.com
 (2603:10b6:610:57:cafe::3c) by CH2PR12CA0011.outlook.office365.com
 (2603:10b6:610:57::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Mon, 6 May 2024 13:39:40 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000A347.mail.protection.outlook.com (10.167.244.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Mon, 6 May 2024 13:39:40 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 08:39:39 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 6 May
 2024 08:39:39 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 6 May 2024 08:39:38 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 183db192-0bae-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Eg6Hfu5OFqGX7zrph32luwwCrT8EyX9QlgWhU325e44az7w7JCvxLnXMZbhb2D4fdSnB8ToNmawpHUPZgN0LGN2kBe3BqUp/qJ4/60u4Zpbs6t4+LQJ/rvts6WLaTVWs05A9ey7NXrD+YrAn71J/pp7ArjvYR8cue+N3JNWkGz7XpNGutcKGZ5ahka2m7wRqVLwzRf8/WR/CBISI6Dnmb8b+ITFPPYMYmDJe1U5aCow1aFh04AfATRDn0V2fRaWnYIANXylfp2eSPy03V3+ggw78w2YOTsuoTrb8jgV2FX95gxVHEnAxvB6YzLggYKNYC+4fOWswu5KpsENlmCNhlA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=MPCkmmAp+IGVKzc9GL8irMKVoCZzCPhBnUM5yusPrno=;
 b=AmZUISqzX2I4tBekxW071qfFlZZVrwiPIKII2By/RmaJyEQGUirciE7EC+sxmYHyY9sn5GkJAKsFw0QveZsfW+5fIwusi4Ah4KZqrZ4C6P0sZhb9WRC3MJn+xd/3bQlZHzr5+kellUl2w0iei0gvQpizSGuafaxR9Mr6WAY+p8yBp4shJwsGFZFyhimzDAVLy3lds5ZK2LXImehg2ngo5+j7/EpS+pL8a7Q97dDrjafPmIujYKq804+Wyh+zvZdCUYONoMgEe5m7fmqhIws1oRMj8xHbXPc800YA/DabLMldPj7HdMRwykd6XoEw2uylZiJy5nsOHIQN+6qFBMuOUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MPCkmmAp+IGVKzc9GL8irMKVoCZzCPhBnUM5yusPrno=;
 b=HvsSahIalLb4ew6aTMp4IjdT1N/+yUPpgMXRGCtPfUqAbkM/WVep73w7lSiUK9Z2pSqP/vxXFdy0udmda3DaOC1n7Iy/uA4wseJl26D6oKtxSigXP9yEVE1ncDDUB9vmljtiJOcop4t1n4smt7dp/IlnM9noMOpE2BPzEiawI6I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
Date: Mon, 6 May 2024 15:39:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-3-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000A347:EE_|PH7PR12MB5854:EE_
X-MS-Office365-Filtering-Correlation-Id: 5cc2a445-0d63-4d5c-6000-08dc6dd1fabd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VDZjYVNpbmhwZ1dYcXRCZm1JV3pjQUZXNmlLOHMvRFZyZFRkK3hLcHZDTUtE?=
 =?utf-8?B?UkQzbXhCTHk2dytYMEgwek5yM3Q1Ni9aWHF5VWRoQ2FjanEyMlljR2gzNU1U?=
 =?utf-8?B?ZVdNeFhVd1JNWjZaT1piOEJ4eHVWZTNoSlFLVlk3Nm15VG90SWN0S2FsTVVW?=
 =?utf-8?B?Y2dTenFiSDZOY1JFSW1CcjdFSFp3RUhJN1JTTEhWZzdMZkxKZzFnbG9yNXhj?=
 =?utf-8?B?UW1wVldwbCtheEdJT3FxeDVnSm9kZXdsd2hXdDNJTVZWRmxyVTdQOVJqdm5j?=
 =?utf-8?B?cHk3SXlBTkdSQkMrT2ZWRG1pdk9jQWFiL1pReVVkT1hqLzhLQVZ3a2YwRE1V?=
 =?utf-8?B?enF6ek9mUTd0T3BDRkNma3lMSURoeCsybGE0OXNHVXVyU0ZyUTAzWU9BZEJC?=
 =?utf-8?B?LzdwSDhpeGZiREYzNkVxMHczbWVSWGMxRkdTSjJpMERlS3N1aWtWQXFlMTNF?=
 =?utf-8?B?TjlkVmRXRE1uVmpyemdOSzRoSlgwZ0MwT2N6WmoraEhiUzA5STA4VklUaEdK?=
 =?utf-8?B?UldkTTVaSFNRS3dkUkxPRU1XSnJidjk2MEZuam9jMGkrQWowT0hraXhvZEtQ?=
 =?utf-8?B?UFN2KzM1TUkwazVEU0RQa2FJcHpjcEZFeHp4S3BrMUlQZEZFQnRReVA3aVVX?=
 =?utf-8?B?ZGxHb2NOUEhhWjZwL05tNlllWnJ3YWkyRnpyQ3RBSEVOcHNPc0FYRmFXZ3A3?=
 =?utf-8?B?Y2dnUFZFRkNaNUREY1hHOVpNYU50ejFSdEJER3JjM2cvaGFrdy92dTFMUGxG?=
 =?utf-8?B?dTloVUhRZklhbURNY1IrSUU5dXU3T21PMzJmc2cxSGRCa2owbU5HM1QvOW9w?=
 =?utf-8?B?YXNFY0tBZWduVEx2VVhHZTk4c0Rlbitxd2FrODNoWFd1eXBkMyt4Wk8yK3I1?=
 =?utf-8?B?V0MvcTltTXNpcVNSVkhiM1l2Q0VBS2NIRGtZeXdKVXdNK09zVVZlVVIwc0dZ?=
 =?utf-8?B?Q1pxM1p2c1NtR1k1ck1qVHZtRk9RRjRCQU1NTjBJem9aYXBidnorMmg0cDFP?=
 =?utf-8?B?OXlmTWtzNjVYWml6ODhOWGN0c0syd3hNb3dzVGJQU1NLSlplSUVYZkRDVk1D?=
 =?utf-8?B?SHRKV1M3eE9QMWVFM3o4SjNjVnlPQkU3ZmdOZzhidkdmUXNpOS93VHFjVG1w?=
 =?utf-8?B?bUk4LzJoaWFKOGQrR2tnUU4zNnVtYWVCOEFMNW9WaHp1YS9tZXBvRlh5MkZ4?=
 =?utf-8?B?OGxUQzVQS1NWa2laMWl5elpKZ2JoaW03WkxuMG5ZQjhUUFhTdk5tSUcybnBj?=
 =?utf-8?B?MkRUVWRFK1ZPQ0YvT3FWRFhjYVlpSTBMSWJsc2N0WVcrZ1NydVpKS2x2QWxM?=
 =?utf-8?B?NXYwVkxEaGM0cnltak5DdVM5eUpqdkFrR3dPR096ZllxUUZkMlRJdmdUV0FT?=
 =?utf-8?B?bjVsaDc1dHdDcHFGZ1Jabk9VTTFrazFCNFhiL3JHcGs0NWZGL2VNODhLVUNW?=
 =?utf-8?B?R2xHbHVhVW5QSnhkakRoQXVBQ284elUzSDVZeXk4VTNwRlhiblpPYkFSODM0?=
 =?utf-8?B?VHRQQlhjbEJpT29PY0wxak4yNFM5VlBFTW5vVWtrbTZXSWVzcVhWaUp2dllS?=
 =?utf-8?B?amZUd3MwbGtybFU4Vm1sNnFFNHRuSTRnMDBRMitjZnBtME42VFpCa1huU2hl?=
 =?utf-8?B?azFIK09qKzNrSFVSZHBUVS94amFhMnUrVmhlaUZsMVdKb0krQk5nSmYxa3NP?=
 =?utf-8?B?WjVqdW83NDJoQTZmZUttMFdRR2htUVR2bzJZd2l3YXJ4VUNrZG1od1B6S1VX?=
 =?utf-8?B?eWNteXpGQjN4TjRqT0lucTRWSmFKYWxtdkNtcGxZaDBtYVBLTGJIUzIwZ0x6?=
 =?utf-8?B?UE1yb3lzRmEvTFMxSnFDYmtueFVrTVpqN3ZNSlNzY05RcStZcHk4VjEzY2ZK?=
 =?utf-8?Q?my8TH13qm4ZPd?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 13:39:40.1450
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5cc2a445-0d63-4d5c-6000-08dc6dd1fabd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000A347.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5854

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> Wrap the code and logic that is calling assign_shared_memory
> and map_regions_p2mt into a new function 'handle_shared_mem_bank',
> it will become useful later when the code will allow the user to
> don't pass the host physical address.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  xen/arch/arm/static-shmem.c | 71 +++++++++++++++++++++++--------------
>  1 file changed, 45 insertions(+), 26 deletions(-)
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index f6cf74e58a83..24e40495a481 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -185,6 +185,47 @@ append_shm_bank_to_domain(struct shared_meminfo *kinfo_shm_mem, paddr_t start,
>      return 0;
>  }
> 
> +static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
> +                                         bool owner_dom_io,
> +                                         const char *role_str,
> +                                         const struct membank *shm_bank)
> +{
> +    paddr_t pbase, psize;
> +    int ret;
> +
> +    BUG_ON(!shm_bank);
not needed

> +
> +    pbase = shm_bank->start;
> +    psize = shm_bank->size;
please add empty line here

> +    /*
> +     * DOMID_IO is a fake domain and is not described in the Device-Tree.
> +     * Therefore when the owner of the shared region is DOMID_IO, we will
> +     * only find the borrowers.
> +     */
> +    if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
> +         (!owner_dom_io && strcmp(role_str, "owner") == 0) )
> +    {
> +        /*
> +         * We found the first borrower of the region, the owner was not
> +         * specified, so they should be assigned to dom_io.
> +         */
> +        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
> +    {
> +        /* Set up P2M foreign mapping for borrower domain. */
> +        ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
> +                               _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
>  int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>                         const struct dt_device_node *node)
>  {
> @@ -249,32 +290,10 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
>              owner_dom_io = false;
Looking at owner_dom_io, why don't you move parsing role and setting owner_dom_io accordingly to handle_shared_mem_bank()?

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 06 13:53:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 13:53:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717673.1120106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ymd-0007UL-PY; Mon, 06 May 2024 13:53:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717673.1120106; Mon, 06 May 2024 13:53:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3ymd-0007UE-N2; Mon, 06 May 2024 13:53:35 +0000
Received: by outflank-mailman (input) for mailman id 717673;
 Mon, 06 May 2024 13:53:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3ymc-0007Tp-8N
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 13:53:34 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 061c6ecc-0bb0-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 15:53:31 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b79451145so14110835e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 06:53:31 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 p12-20020adfe60c000000b0034ccd06a6a3sm10781264wrm.18.2024.05.06.06.53.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 06:53:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 061c6ecc-0bb0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715003611; x=1715608411; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=IzWay8SvDuFicV4e9SxZxNL2/VKrxE94L4zRP/ZhYDo=;
        b=XfiluBt3L8LzrXx44oKBzqxE9qYS4AdGITyUpe/KSDbtCI3JPwzjp4HSnl1B4YYq9b
         pKVIZTKvQm9SW54pFSUVM9rSSPS80/EDSF/aN8FZdf1xcEphvkg+92IaG5njKH1/0WBd
         JhQ4P0DJxVxWrNiCuAqeGTKfi6RboGljInE5Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715003611; x=1715608411;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IzWay8SvDuFicV4e9SxZxNL2/VKrxE94L4zRP/ZhYDo=;
        b=YSYBgpf728/kW4vzARetMuGUF9IVzMZDR6H+652sdq4IbsiTk7E1pAsw6CWuEpnuFR
         a1y9vO0PM0cc3TT8HMk2mqDjZ/mkL7mRNoSBRjF4FU20+uVS0+JNH7vabR50BMPDRNr8
         dr3vHIqzvdH5lNH7JvSyyG4MPo27y+fAkwlc08h+23eskNVTE/3SByvcW2hmxYiMLmoo
         ldvr9iX+8UJP5iHeRdDsTWG8XMjJ07cflD+4YfAP6FbWjGTwSOtilyj8Rg5o10bdcvl6
         krapo91R3W0tKmfXcd2tbtzSj5qcU3Bq4vcPtSb2BXrUK05aXHFJVuyIr+Ko4H11t6gH
         it3A==
X-Gm-Message-State: AOJu0Yw+CdGuDrrfY906Hzy+Mq29m8kfnbdWMVnCh+i5RSKf23QytTgA
	RnC9DLyIAUbDLkt17dZwPnv49/PaVOnNrl1VR6bod5C2M8+NjDFRO4JCm8HzQBk=
X-Google-Smtp-Source: AGHT+IECXGdbR1PYnK/N65eDMHZTdJDAYaRVnuIIuJiDv+0mZpEiiAj6edJsFQ7AJXcAOdeddrVUfA==
X-Received: by 2002:a05:6000:36f:b0:343:7b6b:dcc6 with SMTP id f15-20020a056000036f00b003437b6bdcc6mr6306378wrf.30.1715003611140;
        Mon, 06 May 2024 06:53:31 -0700 (PDT)
Date: Mon, 6 May 2024 15:53:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Message-ID: <Zjjg2ueqgjmn-MS3@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
 <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <24d52bbb-1329-4f8a-81be-505a35969875@suse.com>

On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
> On 06.05.2024 14:42, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
> >> Make the variable a tristate, with (as done elsewhere) a negative value
> >> meaning "default". Since all use sites need looking at, also rename it
> >> to match our usual "opt_*" pattern. While touching it, also move it to
> >> .data.ro_after_init.
> > 
> > I guess I need to look at further patches, as given the feedback on
> > the past version I think we agreed we want to set ATS unconditionally
> > disabled by default, and hence I'm not sure I see the point of the
> > tri-state if enabling ATS will require an explicit opt-in on the
> > command line (ats=1).
> 
> With the present wording in the VT-d spec (which we've now had vague
> indication that it may not be meant that way) there needs to be
> tristate behavior:
> - With "ats=0" ATS won't be used.
> - With "ats=1" ATS will be used for all ATS-capable devices.
> - Without either option ATS will be used for devices where firmware
>   mandates its use.

I'm afraid I don't agree to this behavior.  Regardless of what the
firmware requests ATS must only be enabled on user-request (iow: when
the ats=1 command line option is passed).  Otherwise ATS must remain
disabled for all devices.  It's not fine for firmware to trigger the
enabling of a feature that's not supported on Xen.

> >> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
> >>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
> >>  
> >>          if ( use_ats(pdev, iommu, ivrs_dev) )
> >> -            dte->i = ats_enabled;
> >> +            dte->i = true;
> > 
> > Might be easier to just use:
> > 
> > dte->i = use_ats(pdev, iommu, ivrs_dev);
> 
> I'm hesitant here, as in principle we might be overwriting a "true" by
> "false" then.

Hm, but that would be fine, what's the point in enabling the IOMMU to
reply to ATS requests if ATS is not enabled on the device?

IOW: overwriting a "true" with a "false" seem like the correct
behavior if it's based on the output of use_ats().

> >> @@ -257,7 +257,7 @@ static int __must_check amd_iommu_setup_
> >>                                           ACPI_IVHD_SYSTEM_MGMT));
> >>  
> >>          if ( use_ats(pdev, iommu, ivrs_dev) )
> >> -            ASSERT(dte->i == ats_enabled);
> >> +            ASSERT(dte->i);
> > 
> > ASSERT(dte->i == use_ats(pdev, iommu, ivrs_dev));
> 
> I'm okay switching here, but better to the precise logical equivalent of
> the earlier code:
> 
> ASSERT(dte->i || !use_ats(pdev, iommu, ivrs_dev));

Hm, I see.  I think we should be more strict with this (see my
previous comment), but we could defer to a later change.

> 
> >> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
> >>  
> >>  static inline int pci_ats_device(int seg, int bus, int devfn)
> >>  {
> >> -    if ( !ats_enabled )
> >> +    if ( !opt_ats )
> >>          return 0;
> > 
> > Can't you remove that check altogether now, since you are adding an
> > opt_ats check to use_ats()?
> 
> Two reasons why not: For one this isn't AMD-specific code, and hence
> shouldn't be tied to the AMD-specific use_ats(). In principle VT-d
> code should be okay to call here, too. And then
> amd_iommu_disable_domain_device() doesn't use use_ats(), but does call
> here.

Oh, that's confusing, I didn't realize use_ats was AMD specific code.
It should have some kind of prefix to avoid this kind of confusion.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 14:06:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 14:06:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717680.1120116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yys-0001iE-Tx; Mon, 06 May 2024 14:06:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717680.1120116; Mon, 06 May 2024 14:06:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3yys-0001i7-RF; Mon, 06 May 2024 14:06:14 +0000
Received: by outflank-mailman (input) for mailman id 717680;
 Mon, 06 May 2024 14:06:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3yys-0001i1-HN
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 14:06:14 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c9b18b58-0bb1-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 16:06:09 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41bab13ca81so21066895e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 07:06:09 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 n17-20020a05600c4f9100b0041668162b45sm19852740wmq.26.2024.05.06.07.06.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 07:06:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9b18b58-0bb1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715004369; x=1715609169; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=W8gS7mFS/qALjpKtFdLhPxwy976dDTHPycWvDXjs3u8=;
        b=V404p4rF2mp7VFNjhQyUiXsr+gutR71bRvQV1CRT14PXL/6Q+TDAPbeb/0tGMsFhrA
         ytcejDwO9wuX5Z7fzWn+gywcGptSNrkK0iC61NyLRol+lXJt6DIHTBfep3jOj2U6ymrA
         BPD8lz/FL5puLcDHJJnRfuPg9b4CphGH1rcSg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715004369; x=1715609169;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=W8gS7mFS/qALjpKtFdLhPxwy976dDTHPycWvDXjs3u8=;
        b=PSQQq+8EKIes76LqbAGcCBH0Q/U3muvFp7lOWF7ZCOmbCDxKgF7BTZqc2OeB3N52Qq
         8COCPXUr8PDMYYeapeK+caSsWUu3ZrmmkaXf/Zu95TejkJkI5laK5SK+kM5cej2VjPM9
         aO/aruGmQqdI3hNxpV3e5CMRG11w4vX7Ecy4s6cGzHLW9Qcl0mMXpi8QPYcD8QcAAQCW
         t9/JvL2+Old8urusGjqe6bysZneNOhryFotBm7pxc2hQYbIirzkDB89y/bCjXKSveMN7
         3mLYWXgYfqUhbMt+XifL0dYpzxYZETdVERG/mkatlF84k9maWRm116uRk+sj9QETA4Iq
         QTvg==
X-Gm-Message-State: AOJu0Yygo3e+lwhjagZrBcACKt8sl6hISnX3lDbnqytViSwSnoLR9hlP
	UmuJXyECwjEOpc/AtpEGunyBMajqL/8zocDdTG/z0NUGCTSaopNJW+hn4LL7wkY=
X-Google-Smtp-Source: AGHT+IG9Y+R6JkI4ECHkJFrqVZ1SHI3aRYioSWNHwdt0G7SrVaVTUoNrZypiOt5uBTPWeQkgQLjS/w==
X-Received: by 2002:a05:600c:500b:b0:41b:f6b6:46cf with SMTP id n11-20020a05600c500b00b0041bf6b646cfmr10443649wmr.11.1715004368878;
        Mon, 06 May 2024 07:06:08 -0700 (PDT)
Date: Mon, 6 May 2024 16:06:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 11/12] VT-d: drop flush_dev_iotlb parameter from IOTLB
 flush hook
Message-ID: <Zjjjz8q1hHyagOQG@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <d63065bd-517b-4b6a-a554-7991bdabc3c2@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d63065bd-517b-4b6a-a554-7991bdabc3c2@suse.com>

On Thu, Feb 15, 2024 at 11:18:52AM +0100, Jan Beulich wrote:
> All call sites pass it using the flag from the IOMMU which they also
> pass.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 14:10:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 14:10:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717682.1120126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3z32-00040t-EQ; Mon, 06 May 2024 14:10:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717682.1120126; Mon, 06 May 2024 14:10:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3z32-00040m-BG; Mon, 06 May 2024 14:10:32 +0000
Received: by outflank-mailman (input) for mailman id 717682;
 Mon, 06 May 2024 14:10:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3z31-00040g-El
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 14:10:31 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6538683a-0bb2-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 16:10:30 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-34da4d6f543so1620788f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 07:10:30 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 g17-20020adfa491000000b0034de87e81c7sm10895688wrb.23.2024.05.06.07.10.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 07:10:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6538683a-0bb2-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715004630; x=1715609430; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=JHNKIOJC2nBVBjalRj5ZvSAiBNsF3fzv9026i5Rh2Ww=;
        b=Z9fUmnFJF+7QDltF0yhoPPrWGYM+W8v2+3mi0Y2qUbejc1SfGalFBkNvCVk+RkCr8z
         FXFwth4JHIQv70fZ4oWZg+FCNXkfV+0Jmk2quuOyyUFydTw6e6ysNXvl1WRwnVJOphmB
         +smSD7UvURR2FCSNWbEBhWrVlxffItysLH5tQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715004630; x=1715609430;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JHNKIOJC2nBVBjalRj5ZvSAiBNsF3fzv9026i5Rh2Ww=;
        b=k2NmbnuUCkXppr0DFUtpIlkKl94kXSmP1aer8+CXPyIWpfIHDPjyt89XRUQ4F/5bOi
         8F0HHQFsdEFZXOMy8sg1qqaUhLhs6rZDDesKl6tW4B/ovxauDlF1BiIRJsIsFeThtwOc
         4e6LqYOjUZWdxZmvqc5J63N8lvuHnUy96d6nkL91MRKGRZNQMZ64AKMk4cLJBPPRTyzf
         iU4LDVWC3+wb4HFCtaS2aZrnVNS1JQu1/D4JJMYBxhzdmyavfI6yTlRPjyKiw1PfmQb8
         9APR52yPNbbvloRGJXfU2YvxsX2VFsoH6gNSvAUJaG4Smfb2Z0mY3/soAUcucI4REgJZ
         ovFQ==
X-Gm-Message-State: AOJu0YwbONgz1iL4VayTCv3W/DN/hTl4bf0himCBSL8Cn2fcXE2yxQ+1
	CADXeTNWjQX+AJxAC9w21ZSCi/mVMHF405vacANJ3iew3N05W/9q1gqoG1n7MeM=
X-Google-Smtp-Source: AGHT+IE7rG2/g32CPYKAv4eA0DehOawrIG6WaReqhZ1Nt0rzfC5tlkdl37IwezvjwkTTMwAkHd9zGQ==
X-Received: by 2002:a05:6000:e42:b0:349:8a7e:e0af with SMTP id dy2-20020a0560000e4200b003498a7ee0afmr7326422wrb.11.1715004629734;
        Mon, 06 May 2024 07:10:29 -0700 (PDT)
Date: Mon, 6 May 2024 16:10:28 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 12/12] PCI/ATS: tidy {en,dis}able_ats_device() a little
Message-ID: <Zjjk1Fya82MGzyHA@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <edec0260-e338-49aa-a21f-c507ef9e35cb@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <edec0260-e338-49aa-a21f-c507ef9e35cb@suse.com>

On Thu, Feb 15, 2024 at 11:19:46AM +0100, Jan Beulich wrote:
> Use appropriate types for the control register value as well as the
> capability position. Constify a pointer. Use "else" in favor of encoding
> the opposite condition of the earlier if().
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 14:33:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 14:33:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717686.1120136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zOg-0008L5-4s; Mon, 06 May 2024 14:32:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717686.1120136; Mon, 06 May 2024 14:32:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zOg-0008Ky-1t; Mon, 06 May 2024 14:32:54 +0000
Received: by outflank-mailman (input) for mailman id 717686;
 Mon, 06 May 2024 14:32:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3zOe-0008Ks-Ix
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 14:32:52 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 83f16cc7-0bb5-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 16:32:50 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41ba1ba55e9so16277395e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 07:32:50 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 fl24-20020a05600c0b9800b00418916f5848sm16396865wmb.43.2024.05.06.07.32.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 07:32:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83f16cc7-0bb5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715005970; x=1715610770; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=/AUScGi7SIqtNwoGhkYK1uoA/9TpdAqlDm5MCvmqSZ8=;
        b=isxPu6cj21mRZ7YFA9woz+2hFGPcCMzqlZTioVxZhNnBGZ1JN/az19VrLz5dM2ViEI
         1Za/mIYJTujAi+XIK/h3MNhnj2mX3aRGajPMua3Ff+3auk9o4xnZn/KoskGxQv61UTfX
         K678BLEKzig0tSJ1KIUtevvgHvjuH69nDAe1U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715005970; x=1715610770;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/AUScGi7SIqtNwoGhkYK1uoA/9TpdAqlDm5MCvmqSZ8=;
        b=Y74xQTvn7G0Q2vX1El95Q5cSD1JOog37NTSYjFPfa4w9mDsJnkYh1VBuTUaS0HxrgQ
         nn/KwV30LFU4lff/abNkII1BdllJKse9Et4LfHOOQ0q/c2/LoCoxL8ZT6R6RialuvCmp
         eO1fhR8sttyfcmMVL+sipNKXnFWwcm+WIED2KMykaRXvb8qtySElaEpWgA1Qe61eZCiG
         +X0h7x9Jzmuq0yVk7FbOgzAO951TRU9Zk1cU/xdIvDTOCdm29gj75Yv31+HJ19P3IBQp
         8Ex2Tb4etlbcHNwAPt8VLzz5pmQnC6emPYJ8gkS6sCRfd1+RkeoEblLU+kBVKHVSK70+
         gkyQ==
X-Forwarded-Encrypted: i=1; AJvYcCUlbsLyANmH6lfHXiiT1P49Ry73yQEwsvzb/rcQvCmcNvYboGri1np8J9jUwojNlvwOiSsmu2EX7ZbvwM7kvFkgvigBmlFSAyCQCkBHyG8=
X-Gm-Message-State: AOJu0YyXrtBpa4hnI4ihsn1ZlYptcmgTPafVfJpsn58CaI24x0wAEo/Z
	riOsyV3+V8GaPVsAXmLYhlMxSCOMyfiepGIl/TSfz3j9qmogw0mb/LXXVvcEuQg=
X-Google-Smtp-Source: AGHT+IEur79MizdH5sIWiHr00QL5pFl2fCc4aXr38mCKPADyUGiY738bWeUAijMOObMWLOKhlRFbkg==
X-Received: by 2002:a05:600c:1e20:b0:41b:fc3a:f1ef with SMTP id ay32-20020a05600c1e2000b0041bfc3af1efmr7967156wmb.33.1715005969793;
        Mon, 06 May 2024 07:32:49 -0700 (PDT)
Date: Mon, 6 May 2024 16:32:48 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Message-ID: <ZjjqEPRycO-ZIkAe@macbook>
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
 <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>

On Mon, May 06, 2024 at 12:07:33PM +0200, Jan Beulich wrote:
> On 30.04.2024 18:58, Roger Pau Monne wrote:
> > Keep track of the maximum gfn that has ever been populated into the p2m, and
> > also account for the number of foreign mappings.  Such information will be
> > needed in order to remove foreign mappings during teardown for HVM guests.
> 
> Is "needed" the right term? We could e.g. traverse the P2M tree (didn't look
> at patch 2 yet as to how exactly you use these two new fields there), at which
> point we might get away without either or both of these extra statistics,
> while at the same time also not needing to iterate over a gigantic range of
> GFNs. Going from populated page tables would roughly match "max_gfn", with the
> benefit of certain removals of P2M entries then also shrinking the upper bound.

The nr_foreign field is also used as a way to signal whether iteration
over the p2m is needed in the first place.  If there are no foreign
entries the iteration can be avoided (which is likely the case for a
lot of domains).

Note that in 2/2 max_gfn is also used as the cursor for the teardown
iteration, and points to the last processed p2m entry.  So even if the
maximum gfn is obtained from the p2m page-tables directly, we would
still need some kind of cursor to signal the position during teardown.
Or alternatively remove all entries from the p2m, regardless of their
type, so that the p2m shrinks.

> > --- a/xen/arch/x86/mm/p2m.c
> > +++ b/xen/arch/x86/mm/p2m.c
> > @@ -413,6 +413,8 @@ int p2m_set_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
> >          set_rc = p2m->set_entry(p2m, gfn, mfn, order, p2mt, p2ma, -1);
> >          if ( set_rc )
> >              rc = set_rc;
> > +        else
> > +            p2m->max_gfn = gfn_max(gfn_add(gfn, 1u << order), p2m->max_gfn);
> 
> For one a (new) field named "max_..." wants to record the maximum value, not
> one above. And then you want to use 1UL, to match ...

So gfn + (1UL << order) - 1.

> >          gfn = gfn_add(gfn, 1UL << order);
> >          if ( !mfn_eq(mfn, INVALID_MFN) )
> 
> ... surrounding code (more just out of context).

Oh, indeed.

> Further I can't really convince myself that doing the update just here is
> enough, or whether alternatively the update wouldn't want to be further
> constrained to happen just on newly set foreign entries. In that latter
> case it would be far easier to reason whether doing the update just here is
> sufficient. Plus iirc foreign entries are also necessarily order-0 (else
> p2m_entry_modify() wouldn't be correct as is), which would allow to store
> just the gfn we have in hands, thus resulting in the field then being
> properly named (as to its prefix; it would likely want to become
> "max_foreign_gfn" then).

I didn't want to limit this to foreign entries exclusively, as it
could be useful for other purposes.  My initial intention was to do it
in p2m_entry_modify() so that nr_foreign and max_gfn where set in the
same function, but that requires passing yet another parameter to the
function.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 14:55:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 14:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717696.1120146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zkp-0004Yh-Q0; Mon, 06 May 2024 14:55:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717696.1120146; Mon, 06 May 2024 14:55:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zkp-0004Ya-NK; Mon, 06 May 2024 14:55:47 +0000
Received: by outflank-mailman (input) for mailman id 717696;
 Mon, 06 May 2024 14:55:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s3zko-0004YU-Ac
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 14:55:46 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b737e018-0bb8-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 16:55:45 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-34eb52bfca3so1407582f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 07:55:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 c12-20020adffb4c000000b0034f3e5452a5sm2251140wrs.1.2024.05.06.07.55.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 07:55:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b737e018-0bb8-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715007344; x=1715612144; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MJsqUtpAuWNoDoam7qidS6qcyOXY7ErXJ60/WNrj9qo=;
        b=aJrz01BP23tiuPezKni/YyqIE1KE0/p8qxg70jDuZZ4OgVzKyAqn7lzPT06ssPjjsZ
         A28cbgNzZ9d/XpWyQcmKY2EqXsaYiMWOJdYrdvK4gvf3zNhGmKgAotgqfBzJpfiHuus1
         YSAjIdOxershTTAB8WahptIrsC44SAVqZqNxO4Gv06YK0RtFGjAJHpl+/7tNMrbW7ap7
         P1gPfwnUvyYFFCxl89tDa/LBm9Vs6smwzjnbbfBVnCKBX7lGmBGCKiUuN8u4PlcYAzfp
         pUJu1fqLlzFP95QRIEYJdAanUrR97xXjzRdSd4IfzeQkT9cR7zpQw2GSQGj9G+LbAi3G
         EsPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715007344; x=1715612144;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MJsqUtpAuWNoDoam7qidS6qcyOXY7ErXJ60/WNrj9qo=;
        b=A39ypBboo2fUAuMB2YvsW7EXpmYtjEvVMdBIIpqL31Yf6PAIWb4o3T39SymWYs3TiI
         xf6Sls7sCzUegBZvIct++3r0VaOSgnG3EJ/dy0nCEjSrXroi3K8LHpaZaO9WPeWt0jTU
         RupvHbcyUt803RWsEZcldwFbqEDuML1zwptmYj6TxDA0OCxn1rPVDSskAKMMTrK7agoA
         wzZk3CT2DkNyUZO8Fz+Gn+XBZM/eZuQRsoCHkJBW2gA/3IEUeXBV9pTDhWiAXzLezgGN
         q67Vd7e1uiS0gfvDZIUsmPsmsiV8xnKjWij9+cvoQYLIud6760OgfXmQ1Yh9Mf+D6Tri
         Z4Lg==
X-Forwarded-Encrypted: i=1; AJvYcCXpJQYtnEiBx/vXgkSdPliQh+3wueKpNGMfKMiW2LO89u0NFQzb0SVL1o+J8tAGvn+/ycHjs6ckiMdIel97FXvfOI6pqcIndp4Ch2r8IEE=
X-Gm-Message-State: AOJu0YwVcoO/pdrnZmh61xrLCEFSs1lslhYjV3zJD0TIa40V5CK6bu+E
	/pi/Jd7urxobNc0OHEvbH/km2qJDNeM+NcjLUHx7JDB/lWZ2T23HiJzV8q1Lzw==
X-Google-Smtp-Source: AGHT+IGF9KHwkGVl8uuKLAgyLhhqXVIWYRt6LByMZnQBu1wAdwMZbYMi2Q7aiBPnMqx39lOCqqEF9Q==
X-Received: by 2002:adf:f5cb:0:b0:34d:a7bc:e647 with SMTP id k11-20020adff5cb000000b0034da7bce647mr9700250wrp.62.1715007344357;
        Mon, 06 May 2024 07:55:44 -0700 (PDT)
Message-ID: <4ef79e54-deb1-4242-bcf6-af2b324326b4@suse.com>
Date: Mon, 6 May 2024 16:55:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
 <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com> <ZjjqEPRycO-ZIkAe@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjjqEPRycO-ZIkAe@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 16:32, Roger Pau Monné wrote:
> On Mon, May 06, 2024 at 12:07:33PM +0200, Jan Beulich wrote:
>> On 30.04.2024 18:58, Roger Pau Monne wrote:
>>> Keep track of the maximum gfn that has ever been populated into the p2m, and
>>> also account for the number of foreign mappings.  Such information will be
>>> needed in order to remove foreign mappings during teardown for HVM guests.
>>
>> Is "needed" the right term? We could e.g. traverse the P2M tree (didn't look
>> at patch 2 yet as to how exactly you use these two new fields there), at which
>> point we might get away without either or both of these extra statistics,
>> while at the same time also not needing to iterate over a gigantic range of
>> GFNs. Going from populated page tables would roughly match "max_gfn", with the
>> benefit of certain removals of P2M entries then also shrinking the upper bound.
> 
> The nr_foreign field is also used as a way to signal whether iteration
> over the p2m is needed in the first place.  If there are no foreign
> entries the iteration can be avoided (which is likely the case for a
> lot of domains).
> 
> Note that in 2/2 max_gfn is also used as the cursor for the teardown
> iteration, and points to the last processed p2m entry.  So even if the
> maximum gfn is obtained from the p2m page-tables directly, we would
> still need some kind of cursor to signal the position during teardown.
> Or alternatively remove all entries from the p2m, regardless of their
> type, so that the p2m shrinks.

Having such a cursor just for teardown wouldn't be a big deal, I think.

>>> --- a/xen/arch/x86/mm/p2m.c
>>> +++ b/xen/arch/x86/mm/p2m.c
>>> @@ -413,6 +413,8 @@ int p2m_set_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
>>>          set_rc = p2m->set_entry(p2m, gfn, mfn, order, p2mt, p2ma, -1);
>>>          if ( set_rc )
>>>              rc = set_rc;
>>> +        else
>>> +            p2m->max_gfn = gfn_max(gfn_add(gfn, 1u << order), p2m->max_gfn);
>>
>> For one a (new) field named "max_..." wants to record the maximum value, not
>> one above. And then you want to use 1UL, to match ...
> 
> So gfn + (1UL << order) - 1.

Right, or give the field a different name.

>>>          gfn = gfn_add(gfn, 1UL << order);
>>>          if ( !mfn_eq(mfn, INVALID_MFN) )
>>
>> ... surrounding code (more just out of context).
> 
> Oh, indeed.
> 
>> Further I can't really convince myself that doing the update just here is
>> enough, or whether alternatively the update wouldn't want to be further
>> constrained to happen just on newly set foreign entries. In that latter
>> case it would be far easier to reason whether doing the update just here is
>> sufficient. Plus iirc foreign entries are also necessarily order-0 (else
>> p2m_entry_modify() wouldn't be correct as is), which would allow to store
>> just the gfn we have in hands, thus resulting in the field then being
>> properly named (as to its prefix; it would likely want to become
>> "max_foreign_gfn" then).
> 
> I didn't want to limit this to foreign entries exclusively, as it
> could be useful for other purposes.

I see.

>  My initial intention was to do it
> in p2m_entry_modify() so that nr_foreign and max_gfn where set in the
> same function, but that requires passing yet another parameter to the
> function.

I was indeed implying that would have been the reason for you to not have
put it there.

What you don't answer though is the question of how you determined that
none of the other ->set_entry() invocations would need to have similar
code added. There are quite a few of them, after all.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 14:56:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 14:56:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717700.1120158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zle-00054B-3k; Mon, 06 May 2024 14:56:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717700.1120158; Mon, 06 May 2024 14:56:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s3zld-000542-Vn; Mon, 06 May 2024 14:56:37 +0000
Received: by outflank-mailman (input) for mailman id 717700;
 Mon, 06 May 2024 14:56:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s3zlc-00053M-56
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 14:56:36 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0b3b5c4-0bb8-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 16:56:27 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-34dc9065606so1117655f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 07:56:34 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 gb30-20020a056000459e00b0034e0346317dsm10875061wrb.13.2024.05.06.07.56.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 07:56:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0b3b5c4-0bb8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715007394; x=1715612194; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Pb6o4xcA50FNFY7MOE6kYEZuFMP9IHg+DGUAMLrgFTg=;
        b=nsOSvJCGRcA3R1Jpso6JdIZqHl2Wr1iWmLdqQ8E9vE8Pp30GxK81pFkv1cgWS6Xojl
         NB+F+6jyDs8xgmz44P9NC7pZ5p8UR8a3RU5MpQY1Xxanw3FDEtp7fKmv8KW6qz8pBLOv
         ADA0PvQDq2cd4J+7jSnNtntDpbEyIcsA8ybzY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715007394; x=1715612194;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Pb6o4xcA50FNFY7MOE6kYEZuFMP9IHg+DGUAMLrgFTg=;
        b=sOLshOnaccm2/YJi5ATppWhmqTq+ooiwDzMv5jyWdpBIOziAV0xT6A1ebkZfEngt1P
         VpZNTF82566RpTHYjD8rQ6v6sCsOl+c77gTSCQfb05vGlh6MWT6poHWRiHJOvyw8Z7vx
         VEnRmnsLBn3tzbxdUuJ80+WYmljsv1UFT9YLAz619qhnePJovj/W1Q4aoVSYwtdr+ysh
         vT1QYlasetb1DVVMadR0nuZwxID5RZvPs9Py9t0ONkgYavqITKY1XgNNJk8ja4lVzcI3
         cnZkcx7OKV1LSPnhiPJyoPOZ/9ja9Zo63ulOxEOcSGUAC1lfBiTnRBcJnaCZfdugSN0o
         MTyg==
X-Forwarded-Encrypted: i=1; AJvYcCW73jyWUXvw1QyC4piruwFj6fxrOZ+zIwJG7JUtPryHcE09+Byq8QUYgfeTxV+y8QkpoXPdb+Nb7oFU7z615YmC6BTGxZuvS0CVwvSEQxs=
X-Gm-Message-State: AOJu0YxgPpF4Ezmi6Qp5TM3HBzd54/MihoGppbEpaTaoSQJ8QsuRieli
	CTa56xWGT7pyWTWLjHgTN/ewHrfbHk7MUWRcXXvOnOYvGiI5jBNMSfJyMxRC238=
X-Google-Smtp-Source: AGHT+IFbyGkn2kF8b+GifpcQjgpWWXSTRirQMqfCe5M7IydErtyysMQrxmbULxejkVG72qrQbJAiUA==
X-Received: by 2002:a05:6000:188d:b0:34e:5284:4753 with SMTP id a13-20020a056000188d00b0034e52844753mr11336441wri.1.1715007393792;
        Mon, 06 May 2024 07:56:33 -0700 (PDT)
Date: Mon, 6 May 2024 16:56:32 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19? 2/2] xen/x86: remove foreign mappings from the
 p2m on teardown
Message-ID: <ZjjvoFS2jPy_tt95@macbook>
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-3-roger.pau@citrix.com>
 <54a52cc8-4c36-4086-a10e-114e9a733f45@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <54a52cc8-4c36-4086-a10e-114e9a733f45@suse.com>

On Mon, May 06, 2024 at 12:41:49PM +0200, Jan Beulich wrote:
> On 30.04.2024 18:58, Roger Pau Monne wrote:
> > @@ -2695,6 +2691,70 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> >      return rc;
> >  }
> >  
> > +/*
> > + * Remove foreign mappings from the p2m, as that drops the page reference taken
> > + * when mapped.
> > + */
> > +int relinquish_p2m_mapping(struct domain *d)
> > +{
> > +    struct p2m_domain *p2m = p2m_get_hostp2m(d);
> 
> Are there any guarantees made anywhere that altp2m-s and nested P2Ms can't
> hold foreign mappings? p2m_entry_modify() certainly treats them all the same.

Good point, I will disable those initially, as I don't think there's a
need right now for foreign mapping in altp2m-s and nested p2ms.

> > +    unsigned long gfn = gfn_x(p2m->max_gfn);
> > +    int rc = 0;
> > +
> > +    if ( !paging_mode_translate(d) )
> > +        return 0;
> > +
> > +    BUG_ON(!d->is_dying);
> > +
> > +    p2m_lock(p2m);
> > +
> > +    /* Iterate over the whole p2m on debug builds to ensure correctness. */
> > +    while ( gfn && (IS_ENABLED(CONFIG_DEBUG) || p2m->nr_foreign) )
> > +    {
> > +        unsigned int order;
> > +        p2m_type_t t;
> > +        p2m_access_t a;
> > +
> > +        _get_gfn_type_access(p2m, _gfn(gfn - 1), &t, &a, 0, &order, 0);
> > +        ASSERT(IS_ALIGNED(gfn, 1u << order));
> 
> This heavily relies on the sole place where max_gfn is updated being indeed
> sufficient.
> 
> > +        gfn -= 1 << order;
> 
> Please be consistent with the kind of 1 you shift left. Perhaps anyway both
> better as 1UL.
> 
> > +        if ( t == p2m_map_foreign )
> > +        {
> > +            ASSERT(p2m->nr_foreign);
> > +            ASSERT(order == 0);
> > +            /*
> > +             * Foreign mappings can only be of order 0, hence there's no need
> > +             * to align the gfn to the entry order.  Otherwise we would need to
> > +             * adjust gfn to point to the start of the page if order > 0.
> > +             */
> 
> I'm a little irritated by this comment. Ahead of the enclosing if() you
> already rely on (and assert) GFN being suitably aligned.

Oh, I've added that outer assert later, and then didn't remove this
comment.

> > +            rc = p2m_set_entry(p2m, _gfn(gfn), INVALID_MFN, order, p2m_invalid,
> > +                               p2m->default_access);
> > +            if ( rc )
> > +            {
> > +                printk(XENLOG_ERR
> > +                       "%pd: failed to unmap foreign page %" PRI_gfn " order %u error %d\n",
> > +                       d, gfn, order, rc);
> > +                ASSERT_UNREACHABLE();
> > +                break;
> > +            }
> 
> Together with the updating of ->max_gfn further down, for a release build
> this means: A single attempt to clean up the domain would fail when such a
> set-entry fails. However, another attempt to clean up despite the earlier
> error would then not retry for the failed GFN, but continue one below.
> That's unexpected: I'd either see such a domain remain as a zombie forever,
> or a best effort continuation of all cleanup right away.

I see, thanks for spotting that.  Will change the logic to ensure the
index is not updated past the failed to remove entry.

> > +        }
> > +
> > +        if ( !(gfn & 0xfff) && hypercall_preempt_check() )
> 
> By going from gfn's low bits you may check way more often than necessary
> when encountering large pages.

Yeah, it's difficult to strike a good balance, short of counting by
processed entries rather than using the gfn value.  I'm fine with
checking more often than required, as long as we always ensure that
progress is made on each function call.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:11:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717711.1120167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s400I-0008J7-Et; Mon, 06 May 2024 15:11:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717711.1120167; Mon, 06 May 2024 15:11:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s400I-0008Ix-9w; Mon, 06 May 2024 15:11:46 +0000
Received: by outflank-mailman (input) for mailman id 717711;
 Mon, 06 May 2024 15:11:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s400H-0008Im-IO; Mon, 06 May 2024 15:11:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s400H-0005na-71; Mon, 06 May 2024 15:11:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s400G-00061S-Rc; Mon, 06 May 2024 15:11:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s400G-0006XO-RE; Mon, 06 May 2024 15:11:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NAaTPuW0WalQCAbLLuhtfrr51LzgVsCgqi4ptt70CgM=; b=yrUBlNoMrL8dMGXlfzPlaVM9RX
	PXlDCbrsHmsXnNgTAlZim7EOhnceaUj6v6lbiqietHG3oxBTZlw8jLO18hzs9XLJ4rcPP0GMCUuEV
	56rOdN+FxYST1OxhOn08O+inoag1c2ZCSobpn2VY6Lrz81ttgqG/AaQB27vJzzOhsZC0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185923-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185923: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:debian-di-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=dd5a440a31fae6e459c0d6271dddd62825505361
X-Osstest-Versions-That:
    linux=b9158815de525572333d0499a681459f6b075f28
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 15:11:44 +0000

flight 185923 linux-linus real [real]
flight 185927 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185923/
http://logs.test-lab.xenproject.org/osstest/logs/185927/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 185920

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot            fail pass in 185927-retest
 test-armhf-armhf-xl-raw      12 debian-di-install   fail pass in 185927-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185920

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 185927 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 185927 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 185927 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 185927 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185920
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185920
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185920
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 linux                dd5a440a31fae6e459c0d6271dddd62825505361
baseline version:
 linux                b9158815de525572333d0499a681459f6b075f28

Last test of basis   185920  2024-05-05 17:44:03 Z    0 days
Testing same since   185923  2024-05-06 01:57:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adrian Huang <ahuang12@lenovo.com>
  Andy Lutomirski <luto@kernel.org>
  Ashish Kalra <ashish.kalra@amd.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Gaurav Batra <gbatra@linux.ibm.com>
  Ingo Molnar <mingo@kernel.org>
  Jiri Olsa <jolsa@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Nageswara R Sastry <rnsastry@linux.ibm.com>
  Nayna Jain <nayna@linux.ibm.com>
  Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
  Thomas Gleixner <tglx@linutronix.de>
  Zqiang <qiang.zhang1211@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 477 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:13:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:13:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717716.1120178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4025-0000Ry-Pa; Mon, 06 May 2024 15:13:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717716.1120178; Mon, 06 May 2024 15:13:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4025-0000Rp-LH; Mon, 06 May 2024 15:13:37 +0000
Received: by outflank-mailman (input) for mailman id 717716;
 Mon, 06 May 2024 15:13:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4025-0000PP-8V
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:13:37 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3061ef17-0bbb-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 17:13:27 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-346359c8785so1697280f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:13:33 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 i15-20020adfb64f000000b0034af40b2efdsm10895356wre.108.2024.05.06.08.13.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 08:13:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3061ef17-0bbb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715008413; x=1715613213; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Fh1uLXLzofTd7zjk31n+Gl39sS6WkpHfgVe3CHmMTh0=;
        b=agD0c10thd3qJNPiXl4TvKPjA0xQ6Ofw0vhmZ9qNE3A5sOaLeRO52ipeEkTmw+doYH
         TIuy7hoU8hzKnRnxegY9MRQEhjQJAZm9kOBtsVcl3J7aS1CKfWRwjdrkqerSpRHS430b
         m4fzyIBngq19BQ45wBcZAPU0wLtJW76NIXprA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715008413; x=1715613213;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Fh1uLXLzofTd7zjk31n+Gl39sS6WkpHfgVe3CHmMTh0=;
        b=ErbpwNTIbrx8Xe0IreL7ImEeZ/ZOtg67ZJ6bI/bUZDjai1ch3thrITuQtMqUmN8Cwe
         LB7P2o/ETrtu6T84HDSrIbfgLyQ2mOpfh8LM6LeP9ZtVqWHj5PhlzQ1JBs5IzEvH3KM+
         fjut/opNtBaiz1gknj7mZZ8lcBbwmASQZHaaM4UyZH8pw4WaHF//VRqSZJ0zor7OoiWK
         T9UH/OnVz/xrpUj+ZOL0JEdx/TNvrwfPl6BFveuq2KJdrWQy04mr2c+RQA8y7zByAm68
         l1f38yy1HNmeUYgMVWOVS3kU6R2pOy5/ql33Cn2/S7UzTGKe9apIjdo7T/hTnU14hhY3
         iJYA==
X-Forwarded-Encrypted: i=1; AJvYcCXXOHTOi3ZwFQL49RqVAUWrpsX6kY5EMI/XZJJJ7IsVhRyjgvJ9iUMn4wzwT0RM1qaAh3DB/FuULysr7vSdMUj19Fxwv3HvOLwF24mK6BI=
X-Gm-Message-State: AOJu0YxcuaM6gyDwmZa2ZlPtNp4ojeOAGESObs54jGOQQBZAZYqPueA6
	uIne0koMmNAMwskElOxvchf5qqHrBEd+yNmomdh+SmIp2yBAeJ7yhTZfa3SQp6M=
X-Google-Smtp-Source: AGHT+IFYJ3oRUHNmGejMJYGPtvnRDN2GCmlkI0ukot09ZFENVj3AxpAjq/SKPs+IDdpOGLEJNqEYNw==
X-Received: by 2002:a5d:678b:0:b0:34c:77bc:6c51 with SMTP id v11-20020a5d678b000000b0034c77bc6c51mr7278265wru.1.1715008413347;
        Mon, 06 May 2024 08:13:33 -0700 (PDT)
Date: Mon, 6 May 2024 17:13:32 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Message-ID: <ZjjznLLJq-w3Kob8@macbook>
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
 <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>
 <ZjjqEPRycO-ZIkAe@macbook>
 <4ef79e54-deb1-4242-bcf6-af2b324326b4@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4ef79e54-deb1-4242-bcf6-af2b324326b4@suse.com>

On Mon, May 06, 2024 at 04:55:45PM +0200, Jan Beulich wrote:
> On 06.05.2024 16:32, Roger Pau Monné wrote:
> > On Mon, May 06, 2024 at 12:07:33PM +0200, Jan Beulich wrote:
> >> On 30.04.2024 18:58, Roger Pau Monne wrote:
> >  My initial intention was to do it
> > in p2m_entry_modify() so that nr_foreign and max_gfn where set in the
> > same function, but that requires passing yet another parameter to the
> > function.
> 
> I was indeed implying that would have been the reason for you to not have
> put it there.
> 
> What you don't answer though is the question of how you determined that
> none of the other ->set_entry() invocations would need to have similar
> code added. There are quite a few of them, after all.

Aside from the mem_sharing copying/forking usages, the rest of the
uses of ->set_entry() looked like changes over existing entries, and
strictly not adding new entries.  I might be wrong however, I see that
some of the altp2m usages could also end up populating altp2m entries
(not that the teardown will work with altp2m-s anyway).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:14:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:14:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717718.1120187 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s402f-0000zR-1T; Mon, 06 May 2024 15:14:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717718.1120187; Mon, 06 May 2024 15:14:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s402e-0000zK-U6; Mon, 06 May 2024 15:14:12 +0000
Received: by outflank-mailman (input) for mailman id 717718;
 Mon, 06 May 2024 15:14:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UTOr=MJ=outlook.com=mhklinux@srs-se1.protection.inumbo.net>)
 id 1s402d-0000xq-MA
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:14:11 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12olkn20801.outbound.protection.outlook.com
 [2a01:111:f403:2805::801])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 44cdc2fa-0bbb-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 17:14:02 +0200 (CEST)
Received: from SN6PR02MB4157.namprd02.prod.outlook.com (2603:10b6:805:33::23)
 by CO1PR02MB8361.namprd02.prod.outlook.com (2603:10b6:303:150::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Mon, 6 May
 2024 15:14:05 +0000
Received: from SN6PR02MB4157.namprd02.prod.outlook.com
 ([fe80::cedd:1e64:8f61:b9df]) by SN6PR02MB4157.namprd02.prod.outlook.com
 ([fe80::cedd:1e64:8f61:b9df%2]) with mapi id 15.20.7544.041; Mon, 6 May 2024
 15:14:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44cdc2fa-0bbb-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iaeiV0VKL/k2Yv4fKGGwEssr80vJINOXq6m1xDPFjQuWBb21gbLJrK68jx+KlY1G28AN2l5hRO5/YREmNhMnBTd1q/MwkNXoUws2pM2P7k/bwUs9gWPU6nfjvYq5HsMJK/fxckIiuS2spNPydiuQ+NWcmyqqVzidrlJpd560LQBGNF2at4JnYMvpun0zWjP7UJ1jwuE5yRwO0G5aEIaeZ6ZN8fo8IAB9nNn2YZHXS3fyBhX5iluwxH1YqocIDSXsBz5NBQNtzvuk5ygtboj6hejzbnrJT2lYaU2BRQBe+E3hgt6klNZk2Uka4AVv1JAOzdP9CaKbDqdJH7TpVolfAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=nCIhy+mGfDg/UuUP0MtuKeUQ1yxJ3lv0/2Qdtu52IVY=;
 b=MOGjrQgFllslAOM1oy/MySBiVEegkEnPSJFBZt/lKLBra6SiwG4DYQ66Qb7p+3Yz3GcFGokf+mME1vxckhWk8TZ7q+2z0ze7hA0GDN9KMF55NYjr2DZGZY+ZQyO5CnjM1wqSNRDKLBwPgsAK9tF8AB6S77jMWreVE68LCDtomPEwrXUwHHN9xSJY5+3L1vySf2hRDHMO/Q95Ry3JPVwOQgGAkqcb52WaKle9M2LrVT0qLb0cD0byuQzLKSAGNmt8nJ9PUkG/zUC8b8PbKosvHZiFZ9LNWyUVfZ532PA0abD0LbKmmBJRE9hcKjGX4CQvRtGqSBqOUqMGdzeDrJ0wdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none;
 dkim=none; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com;
 s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nCIhy+mGfDg/UuUP0MtuKeUQ1yxJ3lv0/2Qdtu52IVY=;
 b=PM69548+jhjIEebdDslbKNcDze5jD6ViqihU+RTFCC+rBW3KaCGxcYaVJkVkZM9WAmjYghUI9DHm5Aqe2dhJMr6dKZaEFZjOSXXhY2SuKt15k3KFUbk+TV9K51X9bHZyZFDS7xgoBmI+OeIIv5FvfbOs2F/lUFbEaSxIOTcLLLdMOSIzxkelgf3kef3mBPagp4XTnEcLob3TX+Y3l2i5yJWR8ZvAWU1nefy0aSuiopV22hxjpWESgtTXDSfGRMLKe1tiIOabvH8/LPRWn+ZpNHCLBpHHvAPYHjO1wWvEVDxM6F/7u9+XH4jyYgGcggsNbXocCXYJdkAtou+cWWuYEw==
From: Michael Kelley <mhklinux@outlook.com>
To: Michael Kelley <mhklinux@outlook.com>, "robin.murphy@arm.com"
	<robin.murphy@arm.com>, "joro@8bytes.org" <joro@8bytes.org>,
	"will@kernel.org" <will@kernel.org>, "jgross@suse.com" <jgross@suse.com>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"oleksandr_tyshchenko@epam.com" <oleksandr_tyshchenko@epam.com>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "petr@tesarici.cz" <petr@tesarici.cz>, "roberto.sassu@huaweicloud.com"
	<roberto.sassu@huaweicloud.com>
Subject: RE: [PATCH 1/2] swiotlb: Remove alloc_size argument to
 swiotlb_tbl_map_single()
Thread-Topic: [PATCH 1/2] swiotlb: Remove alloc_size argument to
 swiotlb_tbl_map_single()
Thread-Index: AQHaiWsCp8JQSGA74EyasRMxVkBb5LGKee/Q
Date: Mon, 6 May 2024 15:14:05 +0000
Message-ID:
 <SN6PR02MB4157FD86DFB5E305836EF209D41C2@SN6PR02MB4157.namprd02.prod.outlook.com>
References: <20240408041142.665563-1-mhklinux@outlook.com>
In-Reply-To: <20240408041142.665563-1-mhklinux@outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-tmn: [NnHOC9kgMwi4lxQfxWVHoFjvRL7exwSt]
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SN6PR02MB4157:EE_|CO1PR02MB8361:EE_
x-ms-office365-filtering-correlation-id: ebe81af2-bb5c-48a2-3bc8-08dc6ddf2b51
x-microsoft-antispam:
 BCL:0;ARA:14566002|461199019|102099023|440099019|3412199016|1710799017;
x-microsoft-antispam-message-info:
 hxLYDpDNf6DkpgAyLj88sO2Uif6D89H6852hmwQREl/4/4C8U8Nf0dfWEN5iw9uVUZfLkTTZKSFcswAsZxrdQvMyXEqaGmWBiIjjsoBubwYfb4lWjknf7GAOGoPPJKUwvBPImwcin9PTh3NQgovGp513X1aZFKsWK4qP0BzO2E2TfbHH3xP6nC83+bvPBjKyGF1sFdYHRoGJEFIfmSC7M1XvP5lj6qbFCA9B3m1uefMmDbinCL69wCq4ORYXNmZCdk4mthZGWlr9lKg9bCH74SBjKPLCoiIwtIR4J6UCkQiTa8HnXphlB06Xd2sn7J5bg0UtU5QBq8uc+p/BNqdkLakLftgNw8Nmh/m6AbZ87C4yNXdbJI+OMa/4R/uNhmobJOblcvCOVDAgD32ftqns7eoC39fLAAM87iSPfhDopnSOCInO4c5kRynxRHJR0Nl8/vCTC7ng5FuRGy3ISQUbOXCDRYj4HLARCQd4m0wTbbuaMqTKS8xgoHmdG2VHt4/ZkGAjCewlDukdzbt75Fx8IuQeUKabAV2AvtWWpqnXiyvxVKsJigedLLs64y3Vs/HV4NSwS+VLlYa7dXh9Xfu9YdIpIHXHw4+H/+4taNLQhq1IJ+lMrgWgu1Qsl9e9MERu
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?rTnYaO5e7xysz7/CacTatmtlLBdAHCIx5lBVZ07nPCd0Rfj6TcHCWbCN4ziI?=
 =?us-ascii?Q?eZONax+0FQ3TrUvo5unKQzEWtdaP+GF6YmjgkfBnEsd2SnbY403eo8kp+f7E?=
 =?us-ascii?Q?nNxZbria8mTxDmHI/BZSialLhgYJIa5oGk8rvgJQsLEbiLgyA8uKW5aPlsgp?=
 =?us-ascii?Q?sjSEn5KlOr/xyC3Xw/HjrD9nEo2obrlnLmkuklDlnl9kwn5TM7UHooX4QDUo?=
 =?us-ascii?Q?WcinPJcixXTMI4wzwAkyT64PATsPeiXOQ0qXGkHcA7sCUzk7H7CMw34pR5xn?=
 =?us-ascii?Q?KQ8OGJ2iRffQqEkuzhcnZ2gqL4WqGNenJee0WfVs1OI0xq/wYLjtUAG37p79?=
 =?us-ascii?Q?iMDC3yDLk+evZibuZKveictDYXldVHA7rv1Zx8yvhIT4tr+tC+fifmLrowWh?=
 =?us-ascii?Q?qTP1eOMopyl4nm1KgNUU3sgL529XEu/KoFFsKU+T4xc6fkfJT/U3XvJzqaLM?=
 =?us-ascii?Q?wLTg1xxYZL8GGMmvpyicJIWUgyN4QNIaH7Rt7XNygcHhdAowDGmA4C3eTuVC?=
 =?us-ascii?Q?gft5+ezqLCEIhj10DeyXTD3paSwUj3Yq1aao6Runz0PTZzvPFQMRrmVLWfcd?=
 =?us-ascii?Q?TbdD2BlX8cDthFcUasNhA4LG0uDtuH/c57CIqPFbdVylbB47EBu2yS8lvckF?=
 =?us-ascii?Q?E8pYo8SiO1xqKRqcfT8gWXIrKQ2MRAq1DWTdcOt7Sz16GbTImd2rvHaq5SSI?=
 =?us-ascii?Q?Gajr+7E8NyIfgyPx39fow7ervaFfItyo2+L4R25LE8XosZEV7Yl6nUyNXV5p?=
 =?us-ascii?Q?ld+Wwf9WFmAwbQJTp59dTQ+Ux0OH2cBOB3pVffS5NpJcKZXeoYGI9c0vUj6k?=
 =?us-ascii?Q?5kItralhH52MHlUTp+B8N8VYs9N+ddsj/ZoPv3ox5HPYd/eYadEgYLROiYRA?=
 =?us-ascii?Q?ZaV8FO7HsYNKhZVcAOpC3InvTtsiF9c9E8/nHel3/MN2f/A2+XpGW/4ZsI7Z?=
 =?us-ascii?Q?Azj7+BuM7gtfAmSwZLwQz+rOGiVymOO2DdsEVvF7i7bliaZHgkrLteQVSXWK?=
 =?us-ascii?Q?nLUaXWh4L1TTH9cmd5OuWliUH83fEqljVRNhLHcxSU2l32opuPdBSfYBH2xc?=
 =?us-ascii?Q?keQOvE7tPT+Dd+Cy1viG+szptD4Esd0x0tqUyVsI+zjY8nc+iXbu0sHBEQaw?=
 =?us-ascii?Q?+zv3U2xHJMcGd64m0Ll+WRXQfk/u1cNUtbFfO74YDfauyfREOQ+4Icbpwafn?=
 =?us-ascii?Q?+tws7ciKSwtJU+bj4+7dRoLHLfFjdD/bDDh8JrDHyYJVVDEjYbXMvmzODrQ?=
 =?us-ascii?Q?=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR02MB4157.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: ebe81af2-bb5c-48a2-3bc8-08dc6ddf2b51
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 May 2024 15:14:05.1019
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8361

From: mhkelley58@gmail.com <mhkelley58@gmail.com>
>=20

Gentle ping ...

Anyone interested in reviewing this series of two patches?  It fixes
an edge case bug in the size of the swiotlb request coming from
dma-iommu, and plugs a hole that allows untrusted devices to see
kernel data unrelated to the intended DMA transfer.  I think these are
the last "known bugs" that came out of the extensive swiotlb discussion
and patches for 6.9.

Michael

> Currently swiotlb_tbl_map_single() takes alloc_align_mask and
> alloc_size arguments to specify an swiotlb allocation that is
> larger than mapping_size. This larger allocation is used solely
> by iommu_dma_map_single() to handle untrusted devices that should
> not have DMA visibility to memory pages that are partially used
> for unrelated kernel data.
>=20
> Having two arguments to specify the allocation is redundant. While
> alloc_align_mask naturally specifies the alignment of the starting
> address of the allocation, it can also implicitly specify the size
> by rounding up the mapping_size to that alignment.
>=20
> Additionally, the current approach has an edge case bug.
> iommu_dma_map_page() already does the rounding up to compute the
> alloc_size argument. But swiotlb_tbl_map_single() then calculates
> the alignment offset based on the DMA min_align_mask, and adds
> that offset to alloc_size. If the offset is non-zero, the addition
> may result in a value that is larger than the max the swiotlb can
> allocate. If the rounding up is done _after_ the alignment offset is
> added to the mapping_size (and the original mapping_size conforms to
> the value returned by swiotlb_max_mapping_size), then the max that the
> swiotlb can allocate will not be exceeded.
>=20
> In view of these issues, simplify the swiotlb_tbl_map_single() interface
> by removing the alloc_size argument. Most call sites pass the same
> value for mapping_size and alloc_size, and they pass alloc_align_mask
> as zero. Just remove the redundant argument from these callers, as they
> will see no functional change. For iommu_dma_map_page() also remove
> the alloc_size argument, and have swiotlb_tbl_map_single() compute
> the alloc_size by rounding up mapping_size after adding the offset
> based on min_align_mask. This has the side effect of fixing the
> edge case bug but with no other functional change.
>=20
> Also add a sanity test on the alloc_align_mask. While IOMMU code
> currently ensures the granule is not larger than PAGE_SIZE, if
> that guarantee were to be removed in the future, the downstream
> effect on the swiotlb might go unnoticed until strange allocation
> failures occurred.
>=20
> Tested on an ARM64 system with 16K page size and some kernel
> test-only hackery to allow modifying the DMA min_align_mask and
> the granule size that becomes the alloc_align_mask. Tested these
> combinations with a variety of original memory addresses and
> sizes, including those that reproduce the edge case bug:
>=20
> * 4K granule and 0 min_align_mask
> * 4K granule and 0xFFF min_align_mask (4K - 1)
> * 16K granule and 0xFFF min_align_mask
> * 64K granule and 0xFFF min_align_mask
> * 64K granule and 0x3FFF min_align_mask (16K - 1)
>=20
> With the changes, all combinations pass.
>=20
> Signed-off-by: Michael Kelley <mhklinux@outlook.com>
> ---
> I've haven't used any "Fixes:" tags. This patch really should be
> backported only if all the other recent swiotlb fixes get backported,
> and I'm unclear on whether that will happen.
>=20
> I saw the brief discussion about removing the "dir" parameter from
> swiotlb_tbl_map_single(). That removal could easily be done as part
> of this patch, since it's already changing the swiotlb_tbl_map_single()
> parameters. But I think the conclusion of the discussion was to leave
> the "dir" parameter for symmetry with the swiotlb_sync_*() functions.
> Please correct me if that's wrong, and I'll respin this patch to do
> the removal.
>=20
>  drivers/iommu/dma-iommu.c |  2 +-
>  drivers/xen/swiotlb-xen.c |  2 +-
>  include/linux/swiotlb.h   |  2 +-
>  kernel/dma/swiotlb.c      | 56 +++++++++++++++++++++++++++++----------
>  4 files changed, 45 insertions(+), 17 deletions(-)
>=20
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 07d087eecc17..c21ef1388499 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -1165,7 +1165,7 @@ static dma_addr_t iommu_dma_map_page(struct device =
*dev, struct page *page,
>  		trace_swiotlb_bounced(dev, phys, size);
>=20
>  		aligned_size =3D iova_align(iovad, size);
> -		phys =3D swiotlb_tbl_map_single(dev, phys, size, aligned_size,
> +		phys =3D swiotlb_tbl_map_single(dev, phys, size,
>  					      iova_mask(iovad), dir, attrs);
>=20
>  		if (phys =3D=3D DMA_MAPPING_ERROR)
> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> index 1c4ef5111651..6579ae3f6dac 100644
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -216,7 +216,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device =
*dev, struct page *page,
>  	 */
>  	trace_swiotlb_bounced(dev, dev_addr, size);
>=20
> -	map =3D swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
> +	map =3D swiotlb_tbl_map_single(dev, phys, size, 0, dir, attrs);
>  	if (map =3D=3D (phys_addr_t)DMA_MAPPING_ERROR)
>  		return DMA_MAPPING_ERROR;
>=20
> diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
> index ea23097e351f..14bc10c1bb23 100644
> --- a/include/linux/swiotlb.h
> +++ b/include/linux/swiotlb.h
> @@ -43,7 +43,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
>  extern void __init swiotlb_update_mem_attributes(void);
>=20
>  phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phy=
s,
> -		size_t mapping_size, size_t alloc_size,
> +		size_t mapping_size,
>  		unsigned int alloc_aligned_mask, enum dma_data_direction dir,
>  		unsigned long attrs);
>=20
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index a5e0dfc44d24..046da973a7e2 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -1340,15 +1340,40 @@ static unsigned long mem_used(struct io_tlb_mem
> *mem)
>=20
>  #endif /* CONFIG_DEBUG_FS */
>=20
> +/**
> + * swiotlb_tbl_map_single() - bounce buffer map a single contiguous phys=
ical area
> + * @dev:		Device which maps the buffer.
> + * @orig_addr:		Original (non-bounced) physical IO buffer address
> + * @mapping_size:	Requested size of the actual bounce buffer, excluding
> + *			any pre- or post-padding for alignment
> + * @alloc_align_mask:	Required start and end alignment of the allocated =
buffer
> + * @dir:		DMA direction
> + * @attrs:		Optional DMA attributes for the map operation
> + *
> + * Find and allocate a suitable sequence of IO TLB slots for the request=
.
> + * The allocated space starts at an alignment specified by alloc_align_m=
ask,
> + * and the size of the allocated space is rounded up so that the total a=
mount
> + * of allocated space is a multiple of (alloc_align_mask + 1). If
> + * alloc_align_mask is zero, the allocated space may be at any alignment=
 and
> + * the size is not rounded up.
> + *
> + * The returned address is within the allocated space and matches the bi=
ts
> + * of orig_addr that are specified in the DMA min_align_mask for the dev=
ice. As
> + * such, this returned address may be offset from the beginning of the a=
llocated
> + * space. The bounce buffer space starting at the returned address for
> + * mapping_size bytes is initialized to the contents of the original IO =
buffer
> + * area. Any pre-padding (due to an offset) and any post-padding (due to
> + * rounding-up the size) is not initialized.
> + */
>  phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_=
addr,
> -		size_t mapping_size, size_t alloc_size,
> -		unsigned int alloc_align_mask, enum dma_data_direction dir,
> -		unsigned long attrs)
> +		size_t mapping_size, unsigned int alloc_align_mask,
> +		enum dma_data_direction dir, unsigned long attrs)
>  {
>  	struct io_tlb_mem *mem =3D dev->dma_io_tlb_mem;
>  	unsigned int offset;
>  	struct io_tlb_pool *pool;
>  	unsigned int i;
> +	size_t size;
>  	int index;
>  	phys_addr_t tlb_addr;
>  	unsigned short pad_slots;
> @@ -1362,20 +1387,24 @@ phys_addr_t swiotlb_tbl_map_single(struct device =
*dev, phys_addr_t orig_addr,
>  	if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
>  		pr_warn_once("Memory encryption is active and system is using DMA boun=
ce buffers\n");
>=20
> -	if (mapping_size > alloc_size) {
> -		dev_warn_once(dev, "Invalid sizes (mapping: %zd bytes, alloc: %zd byte=
s)",
> -			      mapping_size, alloc_size);
> -		return (phys_addr_t)DMA_MAPPING_ERROR;
> -	}
> +	/*
> +	 * The default swiotlb memory pool is allocated with PAGE_SIZE
> +	 * alignment. If a mapping is requested with larger alignment,
> +	 * the mapping may be unable to use the initial slot(s) in all
> +	 * sets of IO_TLB_SEGSIZE slots. In such case, a mapping request
> +	 * of or near the maximum mapping size would always fail.
> +	 */
> +	dev_WARN_ONCE(dev, alloc_align_mask > ~PAGE_MASK,
> +		"Alloc alignment may prevent fulfilling requests with max mapping_size=
\n");
>=20
>  	offset =3D swiotlb_align_offset(dev, alloc_align_mask, orig_addr);
> -	index =3D swiotlb_find_slots(dev, orig_addr,
> -				   alloc_size + offset, alloc_align_mask, &pool);
> +	size =3D ALIGN(mapping_size + offset, alloc_align_mask + 1);
> +	index =3D swiotlb_find_slots(dev, orig_addr, size, alloc_align_mask, &p=
ool);
>  	if (index =3D=3D -1) {
>  		if (!(attrs & DMA_ATTR_NO_WARN))
>  			dev_warn_ratelimited(dev,
>  	"swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (s=
lots)\n",
> -				 alloc_size, mem->nslabs, mem_used(mem));
> +				 size, mem->nslabs, mem_used(mem));
>  		return (phys_addr_t)DMA_MAPPING_ERROR;
>  	}
>=20
> @@ -1388,7 +1417,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *d=
ev, phys_addr_t orig_addr,
>  	offset &=3D (IO_TLB_SIZE - 1);
>  	index +=3D pad_slots;
>  	pool->slots[index].pad_slots =3D pad_slots;
> -	for (i =3D 0; i < nr_slots(alloc_size + offset); i++)
> +	for (i =3D 0; i < (nr_slots(size) - pad_slots); i++)
>  		pool->slots[index + i].orig_addr =3D slot_addr(orig_addr, i);
>  	tlb_addr =3D slot_addr(pool->start, index) + offset;
>  	/*
> @@ -1543,8 +1572,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_add=
r_t paddr, size_t size,
>=20
>  	trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
>=20
> -	swiotlb_addr =3D swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
> -			attrs);
> +	swiotlb_addr =3D swiotlb_tbl_map_single(dev, paddr, size, 0, dir, attrs=
);
>  	if (swiotlb_addr =3D=3D (phys_addr_t)DMA_MAPPING_ERROR)
>  		return DMA_MAPPING_ERROR;
>=20
> --
> 2.25.1
>=20



From xen-devel-bounces@lists.xenproject.org Mon May 06 15:33:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:33:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717725.1120197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40LN-0004pR-Gl; Mon, 06 May 2024 15:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717725.1120197; Mon, 06 May 2024 15:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40LN-0004pK-Cz; Mon, 06 May 2024 15:33:33 +0000
Received: by outflank-mailman (input) for mailman id 717725;
 Mon, 06 May 2024 15:33:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VpOg=MJ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s40LM-0004pE-Fd
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:33:32 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe1dd36b-0bbd-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 17:33:31 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b21ed19f5so13944905e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:33:31 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 m37-20020a05600c3b2500b0041816c3049csm16412025wms.11.2024.05.06.08.33.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 May 2024 08:33:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe1dd36b-0bbd-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715009611; x=1715614411; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=snLg2+vaAI3TQzbaqfZHEFjB/gbPR3kotBOnp48T8pE=;
        b=h8nzGGpuj8XsvzVWVugr4ddobdszpJ0Zh1UGv32NbSNZf5yKcgDvUgPU8kkLXqyLNc
         UnANnoeHVOM+XAafwV1KpdPTlgiUnf+GZMq5AWCHmCt53DP6hWFUYhidgqhRmhb014oF
         i1aZ94NK3QmQjFOkiRhWqM2vymLARV4V+fA2E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715009611; x=1715614411;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=snLg2+vaAI3TQzbaqfZHEFjB/gbPR3kotBOnp48T8pE=;
        b=dAlwC7ffDQECk+oaki/fmakmsx6WaVX+aXpQdshBYlEfDM0cL4LtEqis6FyLtlHF+y
         4QaW0yuVmerX4Vb/jvouOrg8qBVvEOvdl9m9kWpkcanTE7lZqh1nGFiZcNGNo3bA72vR
         sB00Os6xOI17aM7ccpaWUgrc7k2vWn0pNUJlJSVm5EFaSNTVvCqU309ZoLBRMFGRll1y
         A0lmI9g1veTbDn5MPc9BMR2/sY4lgOFI1Su8v1OAiK5FUci84B0e+MRfHSMz0iOZy5un
         RiKpueuQlC7+mSwdZ5iBvaJiJK8amwUM9e1hCEPif8RSO9ovNBKJ5fjdTxJvwX2crV4X
         ib3A==
X-Forwarded-Encrypted: i=1; AJvYcCW3GokAji63BGV/FpwkWwdQqzsLaEf5JkswBkF5RE+xKeY7h5Z80/CdTSFFep5zUGtDispAD5I/lvPHYq6ZjbNHPk4UnlOMLCVsKdlT5MI=
X-Gm-Message-State: AOJu0Yy6qw+XIhzHpveO3ffJfxpbXBpMzlf3/BXrcrBSZ+gJAtm1eKiq
	LPuFQaKIFFhHKRLjrO9V/v0m3K8dM7YJOrlS99ONHUfoptCSQsyHRmLL/iLprvM=
X-Google-Smtp-Source: AGHT+IEy9wOLNe44HrklfgjrIgywsx5JALd4urAnXNyYtged7xRQLiBhOy7K0yMM7PmsTDDnpDPWfg==
X-Received: by 2002:a05:600c:4f08:b0:41a:2044:1b3e with SMTP id l8-20020a05600c4f0800b0041a20441b3emr7568873wmq.32.1715009610678;
        Mon, 06 May 2024 08:33:30 -0700 (PDT)
Date: Mon, 6 May 2024 17:33:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Message-ID: <Zjj4SQ3r9ZtjjjKn@macbook>
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
 <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com>

On Mon, May 06, 2024 at 12:07:33PM +0200, Jan Beulich wrote:
> On 30.04.2024 18:58, Roger Pau Monne wrote:
> > Keep track of the maximum gfn that has ever been populated into the p2m, and
> > also account for the number of foreign mappings.  Such information will be
> > needed in order to remove foreign mappings during teardown for HVM guests.
> 
> Is "needed" the right term? We could e.g. traverse the P2M tree (didn't look
> at patch 2 yet as to how exactly you use these two new fields there), at which
> point we might get away without either or both of these extra statistics,
> while at the same time also not needing to iterate over a gigantic range of
> GFNs. Going from populated page tables would roughly match "max_gfn", with the
> benefit of certain removals of P2M entries then also shrinking the upper bound.

One note about traversing the p2m tree that I forgot to add earlier:
AFAICT we would need one implementation for EPT and one for NPT, as I
expect the different page-table format won't allow us to use the same
code against both EPT and NPT page-tables (I really need to check).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:35:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:35:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717728.1120207 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40Mo-0005VH-Pu; Mon, 06 May 2024 15:35:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717728.1120207; Mon, 06 May 2024 15:35:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40Mo-0005VA-N6; Mon, 06 May 2024 15:35:02 +0000
Received: by outflank-mailman (input) for mailman id 717728;
 Mon, 06 May 2024 15:35:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gbjR=MJ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s40Mn-0005Uz-BV
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:35:01 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2e738e1d-0bbe-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 17:34:52 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e0933d3b5fso26661331fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:34:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n17-20020a05600c4f9100b0041668162b45sm20089734wmq.26.2024.05.06.08.34.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 06 May 2024 08:34:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e738e1d-0bbe-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715009698; x=1715614498; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WUiUBKu9XEb28QbAb0HZrV56jxAQv7um+hXO5u53PNE=;
        b=P785VbQrvuj5fLplNzpd9BlIBiDqP8kuqG9xIQS5Iuf3/2Eoot8p+BNa1r54/xSizn
         +hIURgVQhQWW36VN0oYAr01k6LHpMLYUcNPsYv1WVoS7Rljh3jh0czghfhnsZeZA8tVv
         SBec2f2H6LHHSBFMxLtiZvRN2Z+w55mbc/qaAgUk61YjOypXRSXS51gRIzEHuQct6uZA
         TekXfNoPWw0sQmDMo8LaM65bZnt7cmdAe/J1/HRtCNDIwvWPVwFeiJ/l3Hd/CJFWzqu8
         h7co8//coqgyqunebs8tV+kaUrQWgkRShIA1cmFTAHb62ZlYeU3aku+CrfcWvORPytff
         A8lA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715009698; x=1715614498;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WUiUBKu9XEb28QbAb0HZrV56jxAQv7um+hXO5u53PNE=;
        b=W36QNrhKL+TSyviByNxF625mpO8xcIpjZXqFZ8rpIaQFvG0qb5t6DkLyem3J3A3EzR
         rnCA2to7cvAZf1V669ioTSXsY0kgixmwtp/6StCC09zMVgPW20YXd7qD3YctcbfL0uhA
         Wy9C4p4nHAZ+n6uEktovejnNLSbveKoh3L+Y1XaznUp3/jCpw+m9hvXCtRp4nCpqNY8G
         dF9nHEbwYVsL40XfyAt9ANe+h/leUqgh0PtCWc362ism/ZGenDaiG9K5lZ6EEfpFW/XM
         yxB3LEYioxm/EBemG5twuUgMHZR7yKZpyd+cmgYRxEgi+oArTn5WWpvNO4QblDv3/Y7I
         sL3w==
X-Forwarded-Encrypted: i=1; AJvYcCUMOQbQA215s8+BbXlxrPIEsP8A0ePzCDYhI+J0Z7PoKA0266auzAMwM38J+a2K0pnO7q8f33jRSINtjsB4b9xgqydPrngZPPRoNuh7K8g=
X-Gm-Message-State: AOJu0Yy5aD2X6WOd5hNccrexR4p4zHCqPz1gOlqvp8+mvW6nAhVDvVZ8
	n5z4i9k+d8TZir25WprQG8nvw1xLFo0qe+BL9+O+759U2+akxAtNzcNAuNmCqA==
X-Google-Smtp-Source: AGHT+IFq+0/nLuXGQDq6UZfPIqUf56hXBLcuLmZxE+IWoFTSmESnYCdq/DsBDJKby5ySpl1EBmZ+fQ==
X-Received: by 2002:a2e:3c0b:0:b0:2e1:dc63:9f9 with SMTP id j11-20020a2e3c0b000000b002e1dc6309f9mr7887870lja.23.1715009698523;
        Mon, 06 May 2024 08:34:58 -0700 (PDT)
Message-ID: <a2c6ca3c-4d34-4423-bfb2-13bb98dff56e@suse.com>
Date: Mon, 6 May 2024 17:34:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and
 number of foreign mappings in the p2m
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240430165845.81696-1-roger.pau@citrix.com>
 <20240430165845.81696-2-roger.pau@citrix.com>
 <45c52348-e821-4e36-9bd6-7dda00eeb7d3@suse.com> <Zjj4SQ3r9ZtjjjKn@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zjj4SQ3r9ZtjjjKn@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 17:33, Roger Pau Monné wrote:
> On Mon, May 06, 2024 at 12:07:33PM +0200, Jan Beulich wrote:
>> On 30.04.2024 18:58, Roger Pau Monne wrote:
>>> Keep track of the maximum gfn that has ever been populated into the p2m, and
>>> also account for the number of foreign mappings.  Such information will be
>>> needed in order to remove foreign mappings during teardown for HVM guests.
>>
>> Is "needed" the right term? We could e.g. traverse the P2M tree (didn't look
>> at patch 2 yet as to how exactly you use these two new fields there), at which
>> point we might get away without either or both of these extra statistics,
>> while at the same time also not needing to iterate over a gigantic range of
>> GFNs. Going from populated page tables would roughly match "max_gfn", with the
>> benefit of certain removals of P2M entries then also shrinking the upper bound.
> 
> One note about traversing the p2m tree that I forgot to add earlier:
> AFAICT we would need one implementation for EPT and one for NPT, as I
> expect the different page-table format won't allow us to use the same
> code against both EPT and NPT page-tables (I really need to check).

Yes, that would be pretty much unavoidable, I agree.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:36:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:36:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717731.1120216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40Nv-0006aW-3C; Mon, 06 May 2024 15:36:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717731.1120216; Mon, 06 May 2024 15:36:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40Nv-0006aP-04; Mon, 06 May 2024 15:36:11 +0000
Received: by outflank-mailman (input) for mailman id 717731;
 Mon, 06 May 2024 15:36:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vH2P=MJ=tesarici.cz=petr@srs-se1.protection.inumbo.net>)
 id 1s40Ns-0006aC-Ko
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:36:09 +0000
Received: from bee.tesarici.cz (bee.tesarici.cz [37.205.15.56])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57ceba89-0bbe-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 17:36:02 +0200 (CEST)
Received: from meshulam.tesarici.cz
 (dynamic-2a00-1028-83b8-1e7a-4427-cc85-6706-c595.ipv6.o2.cz
 [IPv6:2a00:1028:83b8:1e7a:4427:cc85:6706:c595])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest
 SHA256) (No client certificate requested)
 by bee.tesarici.cz (Postfix) with ESMTPSA id D62611BDAD9;
 Mon,  6 May 2024 17:36:00 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57ceba89-0bbe-11ef-909c-e314d9c70b13
Authentication-Results: mail.tesarici.cz; dmarc=fail (p=quarantine dis=none) header.from=tesarici.cz
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tesarici.cz; s=mail;
	t=1715009761; bh=9PdFWcg++iSIYS3ak0VBMnwVN/MLlIYFQAFux6B9M38=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=KcBAbNefOP2c7ZBclkQ0mP2koZWxiiEgG54z7TjRE3kn1ciXZjNVMinJ0QM6v7imM
	 69s4to8n1w3lSbYPyAowkec/10OMBgLg5/mJ3iyYZMBg8L7/BXCbiLgc35Jjtvuqy/
	 nE7VY+E43DBKSMEjKz49KG1WgtzTERR/hYWsLVbkR21NVXyAwP36vQxLmsGV7NYv8H
	 jieXogVsQ03aNtm786Z6uM6pjSEnIxKmRZCKo/vNDZTCO8JbCQTI+SykDqa4rJb/cG
	 uHAm/XWiDFIBhc5+aO3W/Od2xsxcArsLmWPAU1hvQn4uE+2sQ9I8ct4moR9zCaxW7t
	 yvumtk7HqmKBg==
Date: Mon, 6 May 2024 17:35:59 +0200
From: Petr =?UTF-8?B?VGVzYcWZw61r?= <petr@tesarici.cz>
To: Michael Kelley <mhklinux@outlook.com>
Cc: "robin.murphy@arm.com" <robin.murphy@arm.com>, "joro@8bytes.org"
 <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>, "jgross@suse.com"
 <jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "oleksandr_tyshchenko@epam.com" <oleksandr_tyshchenko@epam.com>,
 "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
 <m.szyprowski@samsung.com>, "iommu@lists.linux.dev"
 <iommu@lists.linux.dev>, "linux-kernel@vger.kernel.org"
 <linux-kernel@vger.kernel.org>, "xen-devel@lists.xenproject.org"
 <xen-devel@lists.xenproject.org>, "roberto.sassu@huaweicloud.com"
 <roberto.sassu@huaweicloud.com>
Subject: Re: [PATCH 1/2] swiotlb: Remove alloc_size argument to
 swiotlb_tbl_map_single()
Message-ID: <20240506173559.4e9af047@meshulam.tesarici.cz>
In-Reply-To: <SN6PR02MB4157FD86DFB5E305836EF209D41C2@SN6PR02MB4157.namprd02.prod.outlook.com>
References: <20240408041142.665563-1-mhklinux@outlook.com>
	<SN6PR02MB4157FD86DFB5E305836EF209D41C2@SN6PR02MB4157.namprd02.prod.outlook.com>
X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-suse-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Mon, 6 May 2024 15:14:05 +0000
Michael Kelley <mhklinux@outlook.com> wrote:

> From: mhkelley58@gmail.com <mhkelley58@gmail.com>
> >   
> 
> Gentle ping ...
> 
> Anyone interested in reviewing this series of two patches?  It fixes
> an edge case bug in the size of the swiotlb request coming from
> dma-iommu, and plugs a hole that allows untrusted devices to see
> kernel data unrelated to the intended DMA transfer.  I think these are
> the last "known bugs" that came out of the extensive swiotlb discussion
> and patches for 6.9.
> 
> Michael
> 
> > Currently swiotlb_tbl_map_single() takes alloc_align_mask and
> > alloc_size arguments to specify an swiotlb allocation that is
> > larger than mapping_size. This larger allocation is used solely
> > by iommu_dma_map_single() to handle untrusted devices that should
> > not have DMA visibility to memory pages that are partially used
> > for unrelated kernel data.
> > 
> > Having two arguments to specify the allocation is redundant. While
> > alloc_align_mask naturally specifies the alignment of the starting
> > address of the allocation, it can also implicitly specify the size
> > by rounding up the mapping_size to that alignment.
> > 
> > Additionally, the current approach has an edge case bug.
> > iommu_dma_map_page() already does the rounding up to compute the
> > alloc_size argument. But swiotlb_tbl_map_single() then calculates
> > the alignment offset based on the DMA min_align_mask, and adds
> > that offset to alloc_size. If the offset is non-zero, the addition
> > may result in a value that is larger than the max the swiotlb can
> > allocate. If the rounding up is done _after_ the alignment offset is
> > added to the mapping_size (and the original mapping_size conforms to
> > the value returned by swiotlb_max_mapping_size), then the max that the
> > swiotlb can allocate will not be exceeded.
> > 
> > In view of these issues, simplify the swiotlb_tbl_map_single() interface
> > by removing the alloc_size argument. Most call sites pass the same
> > value for mapping_size and alloc_size, and they pass alloc_align_mask
> > as zero. Just remove the redundant argument from these callers, as they
> > will see no functional change. For iommu_dma_map_page() also remove
> > the alloc_size argument, and have swiotlb_tbl_map_single() compute
> > the alloc_size by rounding up mapping_size after adding the offset
> > based on min_align_mask. This has the side effect of fixing the
> > edge case bug but with no other functional change.
> > 
> > Also add a sanity test on the alloc_align_mask. While IOMMU code
> > currently ensures the granule is not larger than PAGE_SIZE, if
> > that guarantee were to be removed in the future, the downstream
> > effect on the swiotlb might go unnoticed until strange allocation
> > failures occurred.
> > 
> > Tested on an ARM64 system with 16K page size and some kernel
> > test-only hackery to allow modifying the DMA min_align_mask and
> > the granule size that becomes the alloc_align_mask. Tested these
> > combinations with a variety of original memory addresses and
> > sizes, including those that reproduce the edge case bug:
> > 
> > * 4K granule and 0 min_align_mask
> > * 4K granule and 0xFFF min_align_mask (4K - 1)
> > * 16K granule and 0xFFF min_align_mask
> > * 64K granule and 0xFFF min_align_mask
> > * 64K granule and 0x3FFF min_align_mask (16K - 1)
> > 
> > With the changes, all combinations pass.
> > 
> > Signed-off-by: Michael Kelley <mhklinux@outlook.com>

Looks good to me. My previous discussion was not related to this
change; I was merely trying to find an answer to your question whether
anything else should be changed, and IIUC the result was that not.

Reviewed-by: Petr Tesarik <petr@tesarici.cz>

Petr T

> > ---
> > I've haven't used any "Fixes:" tags. This patch really should be
> > backported only if all the other recent swiotlb fixes get backported,
> > and I'm unclear on whether that will happen.
> > 
> > I saw the brief discussion about removing the "dir" parameter from
> > swiotlb_tbl_map_single(). That removal could easily be done as part
> > of this patch, since it's already changing the swiotlb_tbl_map_single()
> > parameters. But I think the conclusion of the discussion was to leave
> > the "dir" parameter for symmetry with the swiotlb_sync_*() functions.
> > Please correct me if that's wrong, and I'll respin this patch to do
> > the removal.
> > 
> >  drivers/iommu/dma-iommu.c |  2 +-
> >  drivers/xen/swiotlb-xen.c |  2 +-
> >  include/linux/swiotlb.h   |  2 +-
> >  kernel/dma/swiotlb.c      | 56 +++++++++++++++++++++++++++++----------
> >  4 files changed, 45 insertions(+), 17 deletions(-)
> > 
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 07d087eecc17..c21ef1388499 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -1165,7 +1165,7 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
> >  		trace_swiotlb_bounced(dev, phys, size);
> > 
> >  		aligned_size = iova_align(iovad, size);
> > -		phys = swiotlb_tbl_map_single(dev, phys, size, aligned_size,
> > +		phys = swiotlb_tbl_map_single(dev, phys, size,
> >  					      iova_mask(iovad), dir, attrs);
> > 
> >  		if (phys == DMA_MAPPING_ERROR)
> > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> > index 1c4ef5111651..6579ae3f6dac 100644
> > --- a/drivers/xen/swiotlb-xen.c
> > +++ b/drivers/xen/swiotlb-xen.c
> > @@ -216,7 +216,7 @@ static dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
> >  	 */
> >  	trace_swiotlb_bounced(dev, dev_addr, size);
> > 
> > -	map = swiotlb_tbl_map_single(dev, phys, size, size, 0, dir, attrs);
> > +	map = swiotlb_tbl_map_single(dev, phys, size, 0, dir, attrs);
> >  	if (map == (phys_addr_t)DMA_MAPPING_ERROR)
> >  		return DMA_MAPPING_ERROR;
> > 
> > diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
> > index ea23097e351f..14bc10c1bb23 100644
> > --- a/include/linux/swiotlb.h
> > +++ b/include/linux/swiotlb.h
> > @@ -43,7 +43,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
> >  extern void __init swiotlb_update_mem_attributes(void);
> > 
> >  phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t phys,
> > -		size_t mapping_size, size_t alloc_size,
> > +		size_t mapping_size,
> >  		unsigned int alloc_aligned_mask, enum dma_data_direction dir,
> >  		unsigned long attrs);
> > 
> > diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> > index a5e0dfc44d24..046da973a7e2 100644
> > --- a/kernel/dma/swiotlb.c
> > +++ b/kernel/dma/swiotlb.c
> > @@ -1340,15 +1340,40 @@ static unsigned long mem_used(struct io_tlb_mem
> > *mem)
> > 
> >  #endif /* CONFIG_DEBUG_FS */
> > 
> > +/**
> > + * swiotlb_tbl_map_single() - bounce buffer map a single contiguous physical area
> > + * @dev:		Device which maps the buffer.
> > + * @orig_addr:		Original (non-bounced) physical IO buffer address
> > + * @mapping_size:	Requested size of the actual bounce buffer, excluding
> > + *			any pre- or post-padding for alignment
> > + * @alloc_align_mask:	Required start and end alignment of the allocated buffer
> > + * @dir:		DMA direction
> > + * @attrs:		Optional DMA attributes for the map operation
> > + *
> > + * Find and allocate a suitable sequence of IO TLB slots for the request.
> > + * The allocated space starts at an alignment specified by alloc_align_mask,
> > + * and the size of the allocated space is rounded up so that the total amount
> > + * of allocated space is a multiple of (alloc_align_mask + 1). If
> > + * alloc_align_mask is zero, the allocated space may be at any alignment and
> > + * the size is not rounded up.
> > + *
> > + * The returned address is within the allocated space and matches the bits
> > + * of orig_addr that are specified in the DMA min_align_mask for the device. As
> > + * such, this returned address may be offset from the beginning of the allocated
> > + * space. The bounce buffer space starting at the returned address for
> > + * mapping_size bytes is initialized to the contents of the original IO buffer
> > + * area. Any pre-padding (due to an offset) and any post-padding (due to
> > + * rounding-up the size) is not initialized.
> > + */
> >  phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
> > -		size_t mapping_size, size_t alloc_size,
> > -		unsigned int alloc_align_mask, enum dma_data_direction dir,
> > -		unsigned long attrs)
> > +		size_t mapping_size, unsigned int alloc_align_mask,
> > +		enum dma_data_direction dir, unsigned long attrs)
> >  {
> >  	struct io_tlb_mem *mem = dev->dma_io_tlb_mem;
> >  	unsigned int offset;
> >  	struct io_tlb_pool *pool;
> >  	unsigned int i;
> > +	size_t size;
> >  	int index;
> >  	phys_addr_t tlb_addr;
> >  	unsigned short pad_slots;
> > @@ -1362,20 +1387,24 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
> >  	if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
> >  		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
> > 
> > -	if (mapping_size > alloc_size) {
> > -		dev_warn_once(dev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
> > -			      mapping_size, alloc_size);
> > -		return (phys_addr_t)DMA_MAPPING_ERROR;
> > -	}
> > +	/*
> > +	 * The default swiotlb memory pool is allocated with PAGE_SIZE
> > +	 * alignment. If a mapping is requested with larger alignment,
> > +	 * the mapping may be unable to use the initial slot(s) in all
> > +	 * sets of IO_TLB_SEGSIZE slots. In such case, a mapping request
> > +	 * of or near the maximum mapping size would always fail.
> > +	 */
> > +	dev_WARN_ONCE(dev, alloc_align_mask > ~PAGE_MASK,
> > +		"Alloc alignment may prevent fulfilling requests with max mapping_size\n");
> > 
> >  	offset = swiotlb_align_offset(dev, alloc_align_mask, orig_addr);
> > -	index = swiotlb_find_slots(dev, orig_addr,
> > -				   alloc_size + offset, alloc_align_mask, &pool);
> > +	size = ALIGN(mapping_size + offset, alloc_align_mask + 1);
> > +	index = swiotlb_find_slots(dev, orig_addr, size, alloc_align_mask, &pool);
> >  	if (index == -1) {
> >  		if (!(attrs & DMA_ATTR_NO_WARN))
> >  			dev_warn_ratelimited(dev,
> >  	"swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
> > -				 alloc_size, mem->nslabs, mem_used(mem));
> > +				 size, mem->nslabs, mem_used(mem));
> >  		return (phys_addr_t)DMA_MAPPING_ERROR;
> >  	}
> > 
> > @@ -1388,7 +1417,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
> >  	offset &= (IO_TLB_SIZE - 1);
> >  	index += pad_slots;
> >  	pool->slots[index].pad_slots = pad_slots;
> > -	for (i = 0; i < nr_slots(alloc_size + offset); i++)
> > +	for (i = 0; i < (nr_slots(size) - pad_slots); i++)
> >  		pool->slots[index + i].orig_addr = slot_addr(orig_addr, i);
> >  	tlb_addr = slot_addr(pool->start, index) + offset;
> >  	/*
> > @@ -1543,8 +1572,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
> > 
> >  	trace_swiotlb_bounced(dev, phys_to_dma(dev, paddr), size);
> > 
> > -	swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, size, 0, dir,
> > -			attrs);
> > +	swiotlb_addr = swiotlb_tbl_map_single(dev, paddr, size, 0, dir, attrs);
> >  	if (swiotlb_addr == (phys_addr_t)DMA_MAPPING_ERROR)
> >  		return DMA_MAPPING_ERROR;
> > 
> > --
> > 2.25.1
> >   
> 



From xen-devel-bounces@lists.xenproject.org Mon May 06 15:48:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:48:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717737.1120226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40ZR-0000Pj-3n; Mon, 06 May 2024 15:48:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717737.1120226; Mon, 06 May 2024 15:48:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40ZR-0000Pc-1D; Mon, 06 May 2024 15:48:05 +0000
Received: by outflank-mailman (input) for mailman id 717737;
 Mon, 06 May 2024 15:48:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vH2P=MJ=tesarici.cz=petr@srs-se1.protection.inumbo.net>)
 id 1s40ZP-0000PW-JY
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:48:03 +0000
Received: from bee.tesarici.cz (bee.tesarici.cz [37.205.15.56])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 052ce61e-0bc0-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 17:48:02 +0200 (CEST)
Received: from meshulam.tesarici.cz
 (dynamic-2a00-1028-83b8-1e7a-4427-cc85-6706-c595.ipv6.o2.cz
 [IPv6:2a00:1028:83b8:1e7a:4427:cc85:6706:c595])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest
 SHA256) (No client certificate requested)
 by bee.tesarici.cz (Postfix) with ESMTPSA id 47CBA1BCF87;
 Mon,  6 May 2024 17:48:01 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 052ce61e-0bc0-11ef-909c-e314d9c70b13
Authentication-Results: mail.tesarici.cz; dmarc=fail (p=quarantine dis=none) header.from=tesarici.cz
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tesarici.cz; s=mail;
	t=1715010481; bh=srKPK1Kv8jhLdWZU61PvI/zI+CBs9Hz1Jinnh4TWed0=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=ECcH26Kdga6BoqJP6f+Wr+EsWEcynNVu+MHNOUyx0ZeT1iuDJEzAAL7byUxVrwX1M
	 UExDmtTSdATmlM+3jKvFhbWmH05siHZcrDSPyu7vTSesVho3ko0xZvWv3cNZ1UER4k
	 aBpoT8yMJqZXisepfMjMbeC2fxdN1P15ao5K71IT1OPdOydZtC7mmi4r42Zvd+fK5A
	 15SxAgr7mippmhpjnBqZWIjY58fcg228e/mSsR25JZy9DK7uRcA5jQkwGWR9kCFsP1
	 yXD42vVTQnYkoTfVLxJmu6PYVWR9KwXyCOSWpkWTXcnlKecTFtXhvPRPV7q4SoO/+g
	 AUzlzM0tRqd6Q==
Date: Mon, 6 May 2024 17:48:00 +0200
From: Petr =?UTF-8?B?VGVzYcWZw61r?= <petr@tesarici.cz>
To: mhkelley58@gmail.com
Cc: mhklinux@outlook.com, robin.murphy@arm.com, joro@8bytes.org,
 will@kernel.org, jgross@suse.com, sstabellini@kernel.org,
 oleksandr_tyshchenko@epam.com, hch@lst.de, m.szyprowski@samsung.com,
 iommu@lists.linux.dev, linux-kernel@vger.kernel.org,
 xen-devel@lists.xenproject.org, roberto.sassu@huaweicloud.com
Subject: Re: [PATCH 2/2] iommu/dma: Fix zero'ing of bounce buffer padding
 used by untrusted devices
Message-ID: <20240506174800.74ce570b@meshulam.tesarici.cz>
In-Reply-To: <20240408041142.665563-2-mhklinux@outlook.com>
References: <20240408041142.665563-1-mhklinux@outlook.com>
	<20240408041142.665563-2-mhklinux@outlook.com>
X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-suse-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

V Sun,  7 Apr 2024 21:11:42 -0700
mhkelley58@gmail.com naps=C3=A1no:

> From: Michael Kelley <mhklinux@outlook.com>
>=20
> iommu_dma_map_page() allocates swiotlb memory as a bounce buffer when
> an untrusted device wants to map only part of the memory in an
> granule. The goal is to disallow the untrusted device having
> DMA access to unrelated kernel data that may be sharing the granule.
> To meet this goal, the bounce buffer itself is zero'ed, and any
> additional swiotlb memory up to alloc_size after the bounce buffer
> end (i.e., "post-padding") is also zero'ed.
>=20
> However, as of commit 901c7280ca0d ("Reinstate some of "swiotlb: rework
> "fix info leak with DMA_FROM_DEVICE"""), swiotlb_tbl_map_single()
> always initializes the contents of the bounce buffer to the original
> memory. Zero'ing the bounce buffer is redundant and probably wrong per
> the discussion in that commit. Only the post-padding needs to be
> zero'ed.
>=20
> Also, when the DMA min_align_mask is non-zero, the allocated bounce
> buffer space may not start on a granule boundary. The swiotlb memory
> from the granule boundary to the start of the allocated bounce buffer
> might belong to some unrelated bounce buffer. So as described in the
> "second issue" in [1], it can't be zero'ed to protect against untrusted
> devices. But as of commit XXXXXXXXXXXX ("swiotlb: extend buffer
> pre-padding to alloc_align_mask if necessary"), swiotlb_tbl_map_single()

This is now commit af133562d5af.

> allocates pre-padding slots when necessary to meet min_align_mask
> requirements, making it possible to zero the pre-padding area as well.
>=20
> Finally, iommu_dma_map_page() uses the swiotlb for untrusted devices
> and also for certain kmalloc() memory. Current code does the zero'ing
> for both cases, but it is needed only for the untrusted device case.
>=20
> Fix all of this by updating iommu_dma_map_page() to zero both the
> pre-padding and post-padding areas, but not the actual bounce buffer.
> Do this only in the case where the bounce buffer is used because
> of an untrusted device.
>=20
> [1] https://lore.kernel.org/all/20210929023300.335969-1-stevensd@google.c=
om/
>=20
> Signed-off-by: Michael Kelley <mhklinux@outlook.com>
> ---
> I've wondered if this code for zero'ing the pre- and post-padding
> should go in swiotlb_tbl_map_single(). The bounce buffer proper is
> already being initialized there. But swiotlb_tbl_map_single()
> would need to test for an untrusted device (or have a "zero the
> padding" flag passed in as part of the "attrs" argument), which
> adds complexity. Thoughts?

Historically, swiotlb has never cared about exposing data from a
previous user of a bounce buffer. I assume that's because it was
pointless to make an attempt at protecting system memory from a
malicious device that can do DMA to any address anyway. The situation
has changed with hardware IOMMUs, and that could be why the zeroing is
only done in the IOMMU path.

In short, if anybody can explain the value of concealing potentially
sensitive data from devices that are not behind an IOMMU, let's move
the zeroing to swiotlb. Otherwise, let's keep what we have.

Other than that (and the missing commit id), the patch looks good to me.

Reviewed-by: Petr Tesarik <petr@tesarici.cz>

Petr T

>=20
> The commit ID of Petr's patch is X'ed out above because Petr's patch
> hasn't gone into Linus' tree yet. We can add the real commit ID once
> this patch is ready to go in.
>=20
> Also I've haven't used any "Fixes:" tags. This patch really should
> be backported only if all the other recent swiotlb fixes get
> backported, and I'm unclear on whether that will happen.
>=20
>  drivers/iommu/dma-iommu.c | 29 ++++++++++++++++-------------
>  include/linux/iova.h      |  5 +++++
>  2 files changed, 21 insertions(+), 13 deletions(-)
>=20
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index c21ef1388499..ecac39b3190d 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -1154,9 +1154,6 @@ static dma_addr_t iommu_dma_map_page(struct device =
*dev, struct page *page,
>  	 */
>  	if (dev_use_swiotlb(dev, size, dir) &&
>  	    iova_offset(iovad, phys | size)) {
> -		void *padding_start;
> -		size_t padding_size, aligned_size;
> -
>  		if (!is_swiotlb_active(dev)) {
>  			dev_warn_once(dev, "DMA bounce buffers are inactive, unable to map un=
aligned transaction.\n");
>  			return DMA_MAPPING_ERROR;
> @@ -1164,24 +1161,30 @@ static dma_addr_t iommu_dma_map_page(struct devic=
e *dev, struct page *page,
> =20
>  		trace_swiotlb_bounced(dev, phys, size);
> =20
> -		aligned_size =3D iova_align(iovad, size);
>  		phys =3D swiotlb_tbl_map_single(dev, phys, size,
>  					      iova_mask(iovad), dir, attrs);
> =20
>  		if (phys =3D=3D DMA_MAPPING_ERROR)
>  			return DMA_MAPPING_ERROR;
> =20
> -		/* Cleanup the padding area. */
> -		padding_start =3D phys_to_virt(phys);
> -		padding_size =3D aligned_size;
> +		/*
> +		 * Untrusted devices should not see padding areas with random
> +		 * leftover kernel data, so zero the pre- and post-padding.
> +		 * swiotlb_tbl_map_single() has initialized the bounce buffer
> +		 * proper to the contents of the original memory buffer.
> +		 */
> +		if (dev_is_untrusted(dev)) {
> +			size_t start, virt =3D (size_t)phys_to_virt(phys);
> =20
> -		if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
> -		    (dir =3D=3D DMA_TO_DEVICE || dir =3D=3D DMA_BIDIRECTIONAL)) {
> -			padding_start +=3D size;
> -			padding_size -=3D size;
> -		}
> +			/* Pre-padding */
> +			start =3D iova_align_down(iovad, virt);
> +			memset((void *)start, 0, virt - start);
> =20
> -		memset(padding_start, 0, padding_size);
> +			/* Post-padding */
> +			start =3D virt + size;
> +			memset((void *)start, 0,
> +			       iova_align(iovad, start) - start);
> +		}
>  	}
> =20
>  	if (!coherent && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> diff --git a/include/linux/iova.h b/include/linux/iova.h
> index 83c00fac2acb..d2c4fd923efa 100644
> --- a/include/linux/iova.h
> +++ b/include/linux/iova.h
> @@ -65,6 +65,11 @@ static inline size_t iova_align(struct iova_domain *io=
vad, size_t size)
>  	return ALIGN(size, iovad->granule);
>  }
> =20
> +static inline size_t iova_align_down(struct iova_domain *iovad, size_t s=
ize)
> +{
> +	return ALIGN_DOWN(size, iovad->granule);
> +}
> +
>  static inline dma_addr_t iova_dma_addr(struct iova_domain *iovad, struct=
 iova *iova)
>  {
>  	return (dma_addr_t)iova->pfn_lo << iova_shift(iovad);



From xen-devel-bounces@lists.xenproject.org Mon May 06 15:53:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:53:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717742.1120237 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40ew-00023j-Ns; Mon, 06 May 2024 15:53:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717742.1120237; Mon, 06 May 2024 15:53:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40ew-00023c-Ks; Mon, 06 May 2024 15:53:46 +0000
Received: by outflank-mailman (input) for mailman id 717742;
 Mon, 06 May 2024 15:53:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2hIh=MJ=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s40ev-00022I-Nt
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:53:45 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d114953e-0bc0-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 17:53:44 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so524449766b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:53:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d114953e-0bc0-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1715010824; x=1715615624; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BF8NClv1jPLw3wlVZskeQ6pJjW/iX57O7dzan0zx8K0=;
        b=ApFq60RkP9gHF4vGVlar+wPlEZNT9KmARWzszuhvQtOwh088Tqn1qMFSdF18D5PW63
         rJdvz6MCdf6s1ovY1MkasfprSl+g39FnhNSOSyphAwVSc5eccSAjxftGhSHLZKFIkCK9
         sSf4rrXr498yn2PwjgUy9PQFguRwLaS9qHJVz3C6TJBQ4+saDgS1kq28PNx6ENCbrYI/
         R2h6OULtjEkDGSDSE29Grkq7P8Vaw1pgUoTtIazMTBhKqWKOeOmgUw+haCxouHLUsghN
         Tc9KsIyznSMCVhxmrw+QOsxd4jRFSfHtzF+e0M3scX4SgmIHf8hF0GLh9noxoHkn5XWq
         f9ow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715010824; x=1715615624;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BF8NClv1jPLw3wlVZskeQ6pJjW/iX57O7dzan0zx8K0=;
        b=rwuTlWIvjFHe1HYPhp2w80LYniAgdVxolwgwsEy0TEefvKtQwpohhGdIhAbDW9RAD7
         YmhvAQkL6ZeFBnhPzHIxxZJAc8t4xBy78FPLjgiNELczGTeVBExtcVySowuUnHLzTiH2
         3Pj0EBjs5v5qyTBo2Ne8SVeEN6wEGsvAWNAGz+YaHvgNbHQ0xcIAdLGlcEJ1D4e5bVD1
         mx0xQrUKblUlCtmy5ErtCOixoTs9MtUzqhMPVKECNB0N7S55q/xdhia+vQivyAS0vya4
         1ZNPFonrOjuXrUjJ8Znz7A5cx+lR11zPuKxzMTV0jIG13/oNIRqpxF1X3qfku7LbHK5c
         oWiQ==
X-Forwarded-Encrypted: i=1; AJvYcCU+Sk3hjROEI/ADyTnqSqxmWyguF4DdPrd9KmCNK+3jVJgMNR/DUjqkhi7yq4aAcX7sJyjk0lgKPoDfVuepAG6U7A234eW67a4K1wNvQCg=
X-Gm-Message-State: AOJu0YwHCU7TuYOun4hwLTYWSsFEXsjWmdRED5AJ1MNb5KgDKPJ5LIiG
	+P7j11+/+xfaiMSZVSnW7cyFpjmk6LeF14KWluC3VFCRNv9Y7JFQLyUjctx/tFF8ZGxf7Zk+qwb
	pw5F+LyfdhPvjZpdegpHr8VRTWfvTsTE52L7i0Q==
X-Google-Smtp-Source: AGHT+IGPd2PvGoli0JwBV4Xf5joV1u9PoWFLIS1puMiNKG6zVUpEF7ZLNN4M/62zuFVbA3Xuyf+XsQRCqoZqwOrMF2I=
X-Received: by 2002:a17:907:3f07:b0:a59:c9b1:cb68 with SMTP id
 hq7-20020a1709073f0700b00a59c9b1cb68mr3493553ejc.7.1715010823607; Mon, 06 May
 2024 08:53:43 -0700 (PDT)
MIME-Version: 1.0
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-2-carlo.nonato@minervasys.tech> <bcfc0da1-5d8a-4e14-be34-89876d668b86@suse.com>
In-Reply-To: <bcfc0da1-5d8a-4e14-be34-89876d668b86@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 6 May 2024 17:53:32 +0200
Message-ID: <CAG+AhRXj6GeyAXvWyVPymB9eD=YkzcuoB2GTAj2uKb_q_RWa4g@mail.gmail.com>
Subject: Re: [PATCH v8 01/13] xen/common: add cache coloring common code
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Marco Solieri <marco.solieri@minervasys.tech>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Mon, May 6, 2024 at 1:54=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 02.05.2024 18:55, Carlo Nonato wrote:
> > --- a/xen/common/Kconfig
> > +++ b/xen/common/Kconfig
> > @@ -71,6 +71,9 @@ config HAS_IOPORTS
> >  config HAS_KEXEC
> >       bool
> >
> > +config HAS_LLC_COLORING
> > +     bool
> > +
> >  config HAS_PIRQ
> >       bool
> >
> > @@ -513,4 +516,23 @@ config TRACEBUFFER
> >         to be collected at run time for debugging or performance analys=
is.
> >         Memory and execution overhead when not active is minimal.
> >
> > +config LLC_COLORING
> > +     bool "Last Level Cache (LLC) coloring" if EXPERT
> > +     depends on HAS_LLC_COLORING
> > +     depends on !NUMA
> > +
> > +config MAX_LLC_COLORS_ORDER
>
> May I ask that you consider dropping MAX_ from here (but keeping "maximum=
"
> in prompt and text), thus ...
>
> > --- /dev/null
> > +++ b/xen/common/llc-coloring.c
> > @@ -0,0 +1,111 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Last Level Cache (LLC) coloring common code
> > + *
> > + * Copyright (C) 2022 Xilinx Inc.
> > + */
> > +#include <xen/keyhandler.h>
> > +#include <xen/llc-coloring.h>
> > +#include <xen/param.h>
> > +
> > +#define NR_LLC_COLORS          (1 << CONFIG_MAX_LLC_COLORS_ORDER)
>
> ... making this look less strange?

Ok.

> To match up with e.g. max_nr_colors you may also want to use 1U here.

Ok.

> > +void __init llc_coloring_init(void)
> > +{
> > +    unsigned int way_size;
> > +
> > +    if ( llc_size && llc_nr_ways )
> > +    {
> > +        llc_coloring_enabled =3D true;
> > +        way_size =3D llc_size / llc_nr_ways;
> > +    }
> > +    else if ( !llc_coloring_enabled )
> > +        return;
> > +    else
> > +    {
> > +        way_size =3D get_llc_way_size();
> > +        if ( !way_size )
> > +            panic("LLC probing failed and 'llc-size' or 'llc-nr-ways' =
missing\n");
>
> Since you won't accept way_size =3D=3D 0 here, how about it ending up zer=
o in
> the initial if()'s body? Even more, don't you want to demand
> llc_size % llc_nr_ways =3D=3D 0 there (thus, together with the enclosing
> condition, guaranteeing way_size !=3D 0)?

Not sure why I would need it. way_size =3D=3D 0 is checked later via max_nr=
_colors
as you said. llc_size % llc_nr_ways =3D=3D 0 doesn't add anything imo. What=
 matters
the most is that max_nr_colors must be a power of 2.

Thanks.


> > +    }
> > +
> > +    /*
> > +     * The maximum number of colors must be a power of 2 in order to c=
orrectly
> > +     * map them to bits of an address.
> > +     */
> > +    max_nr_colors =3D way_size >> PAGE_SHIFT;
> > +
> > +    if ( max_nr_colors & (max_nr_colors - 1) )
> > +        panic("Number of LLC colors (%u) isn't a power of 2\n", max_nr=
_colors);
> > +
> > +    if ( max_nr_colors > NR_LLC_COLORS )
> > +    {
> > +        printk(XENLOG_WARNING
> > +               "Number of LLC colors (%u) too big. Using configured ma=
x %u\n",
> > +               max_nr_colors, NR_LLC_COLORS);
> > +        max_nr_colors =3D NR_LLC_COLORS;
> > +    } else if ( max_nr_colors < 2 )
> > +        panic("Number of LLC colors %u < 2\n", max_nr_colors);
>
> Ah, here's a check guaranteeing at least the first of the two things aske=
d
> about above.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:53:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:53:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717743.1120248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40f8-0002QG-Vj; Mon, 06 May 2024 15:53:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717743.1120248; Mon, 06 May 2024 15:53:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40f8-0002Q6-RB; Mon, 06 May 2024 15:53:58 +0000
Received: by outflank-mailman (input) for mailman id 717743;
 Mon, 06 May 2024 15:53:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2hIh=MJ=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s40f8-0002PZ-0Y
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:53:58 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d81063de-0bc0-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 17:53:56 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so524509666b.3
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:53:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d81063de-0bc0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1715010835; x=1715615635; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CQu0cUZw8AUQ4Wyu0ySWc7HzzX+i8CY7BjnLCCYsj5c=;
        b=kpoKVyDmOsNJsv/qLDEpJ7eoxKyHtIeRU//e6fx1WiGMij8BQRkudYMDdqUH+iQGQ8
         9iSoZCoLlgF4YVSOi+rD002yeiLHWJETJrz8lGSi1d1TGHplUwS4fzdv3HT4giNFNg2I
         y9lkZahmFXbwZmLC3AFaKI8Ms5WBr+5yG+9X3ad2u0+k9ydVROa6803iETEjf69X67b8
         CCa3DlcJsLjjaH7GUQZi9udu4XVTI4RihnAajDP85iahkuODSvJT+YdUxXPxuCz1yyiV
         XcHy/w04tkCq2JTMZL1+hbchZIlUNEM/T8sBVjjniR7oXgduowkuMSeL2QexQ2AYAKky
         GCcw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715010835; x=1715615635;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=CQu0cUZw8AUQ4Wyu0ySWc7HzzX+i8CY7BjnLCCYsj5c=;
        b=BLpAukxGqYQsFv45J/SXg2Aw5aoHfJo5BS6xrrba6UGwmufrKbwcVLaAPjxh4Qlvre
         0Ttbd++qkG8QSdnaMYry45nCJX0E1Md1XK2sNWP5QIPqUK3HTm1XXbNwBUwiozf1PzgH
         A7jNPj+Dk2ppipPTv+pzju/PwjjnGcZOQSZb9jsFtKd3M1g/y341dt/D6GHrk5eepSTw
         1bR5knrwMWC0va9zBfGZ0Jp2waCVgw7xVhc83Nq891NF9EEiGg3a2ebgfpqUhMbJh5tV
         ziZPgkxyD7Aj9+iBsNbAnkC0UxaVnw58xp2zeU39NcCdP2HlPQGZPBNt7ezmi+/PuT0M
         JVuw==
X-Forwarded-Encrypted: i=1; AJvYcCUWfeb8Rz0hP9e0vtRJK1ep8REI7MK6aZMncUfOoZ90OY/CbI/WDI/n6r1V1UvY4TZuZG3RYsM0zA3eITpZDBmUE8dPv2qtNE9CC/+sNe8=
X-Gm-Message-State: AOJu0YyFyGTe1JRToR621yFbcLi8yaNoQNzb3UAmbMXO34TClusISdOt
	pDGs5NcxjmCMYeOPP6kldD1/lrJ13/DvxSzrFAaLQ6npS8VOIlbADqqV7CHzKW3lN1oC4PjwDTO
	bKP1xqqsVQUmPX7L0rOZ9f7JGHShiLVf/2miVvQ==
X-Google-Smtp-Source: AGHT+IEDKgCWSD6Vf5aYEhpIsjg5XbmSLtKFLK/obXv5f4bSlhLj9p3fYMaLLJQIqjOvv4ldUWZufFiNwr4yxWU9gCs=
X-Received: by 2002:a17:907:6e87:b0:a59:d063:f602 with SMTP id
 sh7-20020a1709076e8700b00a59d063f602mr1982800ejc.29.1715010835480; Mon, 06
 May 2024 08:53:55 -0700 (PDT)
MIME-Version: 1.0
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-5-carlo.nonato@minervasys.tech> <8fbd50df-6c9a-4177-8001-39c0f15e715e@suse.com>
In-Reply-To: <8fbd50df-6c9a-4177-8001-39c0f15e715e@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 6 May 2024 17:53:44 +0200
Message-ID: <CAG+AhRX3oUiGxwYphQDcKo_W+4V3kzMoDh_NVGQcCBXxYT0YAg@mail.gmail.com>
Subject: Re: [PATCH v8 04/13] xen/arm: add Dom0 cache coloring support
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Mon, May 6, 2024 at 2:01=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 02.05.2024 18:55, Carlo Nonato wrote:
> > Add a command line parameter to allow the user to set the coloring
> > configuration for Dom0.
> > A common configuration syntax for cache colors is introduced and
> > documented.
> > Take the opportunity to also add:
> >  - default configuration notion.
> >  - function to check well-formed configurations.
> >
> > Direct mapping Dom0 isn't possible when coloring is enabled, so
> > CDF_directmap flag is removed when creating it.
> >
> > Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> >
> > Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> > Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> albeit ...
>
> > v8:
> > - added bound check on dom0_num_colors
> > - default colors array set just once
> > v7:
> > - parse_color_config() doesn't accept leading/trailing commas anymore
> > - removed alloc_colors() helper
> > v6:
> > - moved domain_llc_coloring_free() in this patch
>
> ... I'm having trouble seeing why this was done: Dom0 isn't ever cleaned =
up
> after, is it?

domain_destroy() is never called for dom0, right? If so, yes this is wrongl=
y
placed here.

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 15:54:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 15:54:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717748.1120257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40fZ-00039b-5s; Mon, 06 May 2024 15:54:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717748.1120257; Mon, 06 May 2024 15:54:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s40fZ-00039U-36; Mon, 06 May 2024 15:54:25 +0000
Received: by outflank-mailman (input) for mailman id 717748;
 Mon, 06 May 2024 15:54:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2hIh=MJ=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1s40fY-00022I-BI
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 15:54:24 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e8ae7452-0bc0-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 17:54:23 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5200afe39eso536211766b.1
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 08:54:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8ae7452-0bc0-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1715010863; x=1715615663; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XRtgzqc0FbKV9K3Jo+wyMI1HnW4CFqr0A/4aRlfTRjw=;
        b=kqr+tM3W1TXvjku12srnXQxEFepH915LsmkWX3PxTeZT2HwWB6Oz1zKAB5+8xWZOP9
         7LTZSuwumx/5iW1+nJpUU8PagnELfp1fIgLmEaR4FoYZMz3viMIDWsjXiJZmGqyaiXDJ
         saTAeHl3csxbO/E07n/yUufKRAMa4SxPooje7Q1YH/rl43FufthNPg88YAhQ6n3RyL/N
         nDQ2UTCfTHUXDdmZtRPjDkqupUY2XYB5ChuaxvhnTSW+uJE4M/QqskhRw/sfN7dU9BTa
         zjEj7FnMM0IFV7tAo8zc+mV4R0deecNubve8+nAsPQIwh1yPTib+nh2AFGIIRQskFPvO
         oQxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715010863; x=1715615663;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XRtgzqc0FbKV9K3Jo+wyMI1HnW4CFqr0A/4aRlfTRjw=;
        b=GY97g1YUz8GKp2WUp31RvzUBmYsqBa90KhVE8NIRjsrsdYH6AAI/24TB3c2GPPP7wB
         ZuEQZZ+cwyYo9cLUA6Ae8RtZc+AmNbaJmVwhnw6xHar6466oJkn8ZNtdgYxH9woXi9n6
         Gqk4i67S5xm3goGqTIes16qF4axQ83f1dxUhGpNxbdYRixkufpew3M+X6QNrDFC/Abgq
         xZu77oSRnHmG0mfx6G74OlDEZylrd8Eofd2S/JlXSRemGstHKjMHD9+StTgLYVTyfno8
         8GCcud643Gqknz2/D0YTgObTsLqAFVkcTdZpkWe70t+5z2A6JY06tCUH4vut4i6ts4Lr
         +xyg==
X-Forwarded-Encrypted: i=1; AJvYcCWKhiyapjtXlVv0wN5WKtYxr0Qwf2s3RB5J+jT9tZfMZZALxxmzx8c25b0JwT+1EMySU+qOeX8z/jUzeZbsgLLAVdk5q4+9K8hPBlpAHuE=
X-Gm-Message-State: AOJu0YwZ6XPJmMHNmqa5DCDGhirejIfrEiKOn9cZgnEJHx2KNFtYwy/z
	dwPQXAm77AlnFb/Z7inWQi8mg+dozDymQhn1MT1AAuq50ikWSss7XCqM/R1sxK8Toumu+0o6wj6
	GJ6Okt/7r6YQLcPIFsX8Mai380eAh75fkmbxvtA==
X-Google-Smtp-Source: AGHT+IFIYlFrBPt6c5eh54nU/XVUqczBUnaMSB29KBVhqtB6a1kce1oJ0Rk4BthBMe7u+ybROSup9zp+0Wmp8jhtCk4=
X-Received: by 2002:a17:906:6a05:b0:a59:c3d0:5508 with SMTP id
 qw5-20020a1709066a0500b00a59c3d05508mr5372568ejc.35.1715010863454; Mon, 06
 May 2024 08:54:23 -0700 (PDT)
MIME-Version: 1.0
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-9-carlo.nonato@minervasys.tech> <674578c8-8243-40f6-ad90-28f9b76a42fd@suse.com>
In-Reply-To: <674578c8-8243-40f6-ad90-28f9b76a42fd@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 6 May 2024 17:54:12 +0200
Message-ID: <CAG+AhRVo74yzSRuRe8g16a+ySnmmLw11R8DARfacafdn1M9+FQ@mail.gmail.com>
Subject: Re: [PATCH v8 08/13] xen/page_alloc: introduce preserved page flags macro
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Mon, May 6, 2024 at 2:22=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 02.05.2024 18:55, Carlo Nonato wrote:
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -159,6 +159,7 @@
> >  #endif
> >
> >  #define PGC_no_buddy_merge PGC_static
> > +#define PGC_preserved (PGC_extra | PGC_static)
>
> Seeing this again and its use ...
>
> > @@ -1426,11 +1427,11 @@ static bool mark_page_free(struct page_info *pg=
, mfn_t mfn)
> >      {
> >      case PGC_state_inuse:
> >          BUG_ON(pg->count_info & PGC_broken);
> > -        pg->count_info =3D PGC_state_free;
> > +        pg->count_info =3D PGC_state_free | (pg->count_info & PGC_pres=
erved);
> >          break;
> >
> >      case PGC_state_offlining:
> > -        pg->count_info =3D (pg->count_info & PGC_broken) |
> > +        pg->count_info =3D (pg->count_info & (PGC_broken | PGC_preserv=
ed)) |
> >                           PGC_state_offlined;
> >          pg_offlined =3D true;
> >          break;
>
> ... here: Shouldn't PGC_broken also be included in PGC_preserved?

I hope there are no other problems like the one with PGC_extra.

> > @@ -2484,6 +2485,11 @@ struct page_info *alloc_domheap_pages(
> >          }
> >          if ( assign_page(pg, order, d, memflags) )
> >          {
> > +            unsigned long i;
> > +
> > +            for ( i =3D 0; i < (1UL << order); i++ )
> > +                pg[i].count_info &=3D ~PGC_extra;
>
> For larger order this loop is non-trivial and may have a fair effect on
> caches. Looking at the code just outside of upper patch context, is this
> loop needed at all when MEMF_no_refcount is clear in memflags?

Nope. I will wrap it in a if.

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Mon May 06 16:40:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 16:40:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717756.1120266 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s41Ny-0004AC-Bj; Mon, 06 May 2024 16:40:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717756.1120266; Mon, 06 May 2024 16:40:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s41Ny-0004A5-95; Mon, 06 May 2024 16:40:18 +0000
Received: by outflank-mailman (input) for mailman id 717756;
 Mon, 06 May 2024 16:40:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s41Nx-00049s-5Z; Mon, 06 May 2024 16:40:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s41Nw-0007kH-U4; Mon, 06 May 2024 16:40:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s41Nw-0000OH-Hy; Mon, 06 May 2024 16:40:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s41Nw-0008OI-Hb; Mon, 06 May 2024 16:40:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4c2xBaXrSTuzNHHJJeqnvmvp4gPE/sDHXp7TXHiaO0U=; b=wWSKIRm2PZavNs3rxAqrOghaFo
	s0lUr3rxza2HqeHvVy1tzvU1s1KL+fv5yXWgzMnylf4T3AozIjtxnIF6xnst9Taa1UAlBUTMpeBaf
	pJR/BBZtDTsF8R+Rxr5aDQ64Mcv8R7O1/1SVL+/OfYRXFsGODnC8tzj/Sfn+VmcxALRo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185926-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185926: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
X-Osstest-Versions-That:
    xen=692f6199bd9a45c93952142c610f0a7dd3b3ef8e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 16:40:16 +0000

flight 185926 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185926/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
baseline version:
 xen                  692f6199bd9a45c93952142c610f0a7dd3b3ef8e

Last test of basis   185924  2024-05-06 08:02:15 Z    0 days
Testing same since   185926  2024-05-06 13:02:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   692f6199bd..ebab808eb1  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon May 06 17:50:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 17:50:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717779.1120277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s42Tn-0005qB-VZ; Mon, 06 May 2024 17:50:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717779.1120277; Mon, 06 May 2024 17:50:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s42Tn-0005q4-Sl; Mon, 06 May 2024 17:50:23 +0000
Received: by outflank-mailman (input) for mailman id 717779;
 Mon, 06 May 2024 17:50:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Sp3F=MJ=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s42Tm-0005py-Lp
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 17:50:23 +0000
Received: from smtp-8fab.mail.infomaniak.ch (smtp-8fab.mail.infomaniak.ch
 [83.166.143.171]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 19a3e8bd-0bd1-11ef-b4bb-af5377834399;
 Mon, 06 May 2024 19:50:18 +0200 (CEST)
Received: from smtp-3-0001.mail.infomaniak.ch (smtp-3-0001.mail.infomaniak.ch
 [10.4.36.108])
 by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VY87s1JrnzP8k;
 Mon,  6 May 2024 19:50:17 +0200 (CEST)
Received: from unknown by smtp-3-0001.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VY87p5F0GzB95; Mon,  6 May 2024 19:50:14 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19a3e8bd-0bd1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1715017816;
	bh=FeafhCaExu57qsacTj62dWRUdrBXKjPBuLT9vOLipXU=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=mGwJTwDHmCYyRVABfvivVlR645+s55YE3ikqtGU8J06uhEpc53bUE7Ls8lVECmSJY
	 ZoGZR6nkUaCWIF86rf6UUOQ30vsnFTU+oyKVa5juk1cYq6aZPMMxOD7Adtf6s6Qe8Z
	 /vj4j5MGtH+BC89lqOLDnwO6/+q161VQ+b28m63o=
Date: Mon, 6 May 2024 19:50:13 +0200
From: =?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
To: Sean Christopherson <seanjc@google.com>
Cc: Borislav Petkov <bp@alien8.de>, 
	Dave Hansen <dave.hansen@linux.intel.com>, "H . Peter Anvin" <hpa@zytor.com>, 
	Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, Anna Trikalinou <atrikalinou@microsoft.com>, 
	Chao Peng <chao.p.peng@linux.intel.com>, Forrest Yuan Yu <yuanyu@google.com>, 
	James Gowans <jgowans@amazon.com>, James Morris <jamorris@linux.microsoft.com>, 
	John Andersen <john.s.andersen@intel.com>, "Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, 
	Marian Rotariu <marian.c.rotariu@gmail.com>, Mihai =?utf-8?B?RG9uyJt1?= <mdontu@bitdefender.com>, 
	=?utf-8?B?TmljdciZb3IgQ8OuyJt1?= <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, 
	Yu Zhang <yu.c.zhang@linux.intel.com>, =?utf-8?Q?=C8=98tefan_=C8=98icleru?= <ssicleru@bitdefender.com>, 
	dev@lists.cloudhypervisor.org, kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, 
	x86@kernel.org, xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
Message-ID: <20240506.ohwe7eewu0oB@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZjTuqV-AxQQRWwUW@google.com>
X-Infomaniak-Routing: alpha

On Fri, May 03, 2024 at 07:03:21AM GMT, Sean Christopherson wrote:
> On Fri, May 03, 2024, Mickaël Salaün wrote:
> > Add an interface for user space to be notified about guests' Heki policy
> > and related violations.
> > 
> > Extend the KVM_ENABLE_CAP IOCTL with KVM_CAP_HEKI_CONFIGURE and
> > KVM_CAP_HEKI_DENIAL. Each one takes a bitmask as first argument that can
> > contains KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4. The
> > returned value is the bitmask of known Heki exit reasons, for now:
> > KVM_HEKI_EXIT_REASON_CR0 and KVM_HEKI_EXIT_REASON_CR4.
> > 
> > If KVM_CAP_HEKI_CONFIGURE is set, a VM exit will be triggered for each
> > KVM_HC_LOCK_CR_UPDATE hypercalls according to the requested control
> > register. This enables to enlighten the VMM with the guest
> > auto-restrictions.
> > 
> > If KVM_CAP_HEKI_DENIAL is set, a VM exit will be triggered for each
> > pinned CR violation. This enables the VMM to react to a policy
> > violation.
> > 
> > Cc: Borislav Petkov <bp@alien8.de>
> > Cc: Dave Hansen <dave.hansen@linux.intel.com>
> > Cc: H. Peter Anvin <hpa@zytor.com>
> > Cc: Ingo Molnar <mingo@redhat.com>
> > Cc: Kees Cook <keescook@chromium.org>
> > Cc: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Sean Christopherson <seanjc@google.com>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
> > Cc: Wanpeng Li <wanpengli@tencent.com>
> > Signed-off-by: Mickaël Salaün <mic@digikod.net>
> > Link: https://lore.kernel.org/r/20240503131910.307630-4-mic@digikod.net
> > ---
> > 
> > Changes since v1:
> > * New patch. Making user space aware of Heki properties was requested by
> >   Sean Christopherson.
> 
> No, I suggested having userspace _control_ the pinning[*], not merely be notified
> of pinning.
> 
>  : IMO, manipulation of protections, both for memory (this patch) and CPU state
>  : (control registers in the next patch) should come from userspace.  I have no
>  : objection to KVM providing plumbing if necessary, but I think userspace needs to
>  : to have full control over the actual state.
>  : 
>  : One of the things that caused Intel's control register pinning series to stall
>  : out was how to handle edge cases like kexec() and reboot.  Deferring to userspace
>  : means the kernel doesn't need to define policy, e.g. when to unprotect memory,
>  : and avoids questions like "should userspace be able to overwrite pinned control
>  : registers".
>  : 
>  : And like the confidential VM use case, keeping userspace in the loop is a big
>  : beneifit, e.g. the guest can't circumvent protections by coercing userspace into
>  : writing to protected memory.
> 
> I stand by that suggestion, because I don't see a sane way to handle things like
> kexec() and reboot without having a _much_ more sophisticated policy than would
> ever be acceptable in KVM.
> 
> I think that can be done without KVM having any awareness of CR pinning whatsoever.
> E.g. userspace just needs to ability to intercept CR writes and inject #GPs.  Off
> the cuff, I suspect the uAPI could look very similar to MSR filtering.  E.g. I bet
> userspace could enforce MSR pinning without any new KVM uAPI at all.
> 
> [*] https://lore.kernel.org/all/ZFUyhPuhtMbYdJ76@google.com

OK, I had concern about the control not directly coming from the guest,
especially in the case of pKVM and confidential computing, but I get you
point.  It should indeed be quite similar to the MSR filtering on the
userspace side, except that we need another interface for the guest to
request such change (i.e. self-protection).

Would it be OK to keep this new KVM_HC_LOCK_CR_UPDATE hypercall but
forward the request to userspace with a VM exit instead?  That would
also enable userspace to get the request and directly configure the CR
pinning with the same VM exit.


From xen-devel-bounces@lists.xenproject.org Mon May 06 17:55:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 17:55:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717793.1120287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s42Z7-0006rl-LM; Mon, 06 May 2024 17:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717793.1120287; Mon, 06 May 2024 17:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s42Z7-0006re-I3; Mon, 06 May 2024 17:55:53 +0000
Received: by outflank-mailman (input) for mailman id 717793;
 Mon, 06 May 2024 17:55:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GMTE=MJ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s42Z6-0006rY-Il
 for xen-devel@lists.xenproject.org; Mon, 06 May 2024 17:55:52 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0155e00-0bd1-11ef-909c-e314d9c70b13;
 Mon, 06 May 2024 19:55:51 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 0BC046141C;
 Mon,  6 May 2024 17:55:50 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id DD241C116B1;
 Mon,  6 May 2024 17:55:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0155e00-0bd1-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715018149;
	bh=EdWtqKvcYWA8XAs2OydI8w9jKT90C+jXPrvnw8usLT0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=SCT1qtXiceWF3Dhbx04mBJktHjHzNOmRzzbiVFNMqRQ97Mokt6Gv6OHl0gst+bZT6
	 rlpMRZZiGIhpyAbpISR1paNrR46dN25YS4LW9TUR25GITv2cGacWJ6iB8h/3oCs0Y5
	 a1IQpGf/Ce0TGynukgKsZHYjZX46KZYAR2z1znVfEY1SmNPRz6KOgl9U9ohQc8eiyq
	 5P4jSTmyrvYDW9dYlsYVqg1mpXssVyt3z9seMcZ4Xwr4s5ssNJvG/kibFEoJnMO9TN
	 up/S3ByboXArBJahaI8R+sgMO1830HdgLFDpeKvCEravWX9sh9YhuEvW1E7UKmqXSy
	 Hu1/oZsqVXXjA==
Date: Mon, 6 May 2024 10:55:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, bertrand.marquis@arm.com, julien@xen.org, 
    andrew.cooper3@citrix.com, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [XEN PATCH] automation/eclair_analysis: unblock pipelines from
 certain repositories
In-Reply-To: <b599441c1678d11cfaa43b679bacbd5a1b886d3f.1714983088.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405061055370.1151289@ubuntu-linux-20-04-desktop>
References: <b599441c1678d11cfaa43b679bacbd5a1b886d3f.1714983088.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 6 May 2024, Nicola Vetrini wrote:
> Repositories under people/* only execute the analyze step if manually
> triggered, but in order to avoid blocking the rest of the pipeline
> if such step is not run, allow it to fail.
> 
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> See https://gitlab.com/xen-project/people/bugseng/xen/-/pipelines/1278550868
> for a sample pipeline with this change applied.
> ---
>  automation/gitlab-ci/analyze.yaml | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
> index 32bf57014961..02e0ea692c66 100644
> --- a/automation/gitlab-ci/analyze.yaml
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -31,6 +31,7 @@
>        when: never
>      - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
>        when: manual
> +      allow_failure: true
>      - !reference [.eclair-analysis, rules]
>  
>  eclair-x86_64:
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Mon May 06 19:10:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 19:10:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717809.1120297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s43ib-0000EB-Su; Mon, 06 May 2024 19:09:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717809.1120297; Mon, 06 May 2024 19:09:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s43ib-0000E4-PH; Mon, 06 May 2024 19:09:45 +0000
Received: by outflank-mailman (input) for mailman id 717809;
 Mon, 06 May 2024 19:09:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s43ia-0000Du-1G; Mon, 06 May 2024 19:09:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s43iZ-0001tT-Mo; Mon, 06 May 2024 19:09:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s43iZ-0006zz-Ag; Mon, 06 May 2024 19:09:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s43iZ-0005Jv-AA; Mon, 06 May 2024 19:09:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=K3Hm4s1tbNYOva5CSrfVFrKfTneQJKa3rmk1sFzPWBc=; b=tRJiLX+4lWbmCRlPj3J30Cl8wR
	UwtC/EIcA2WFHjGj9xTcIblK4nmHGM/1CbHU7+PNzcUUD+CXrmCf5WA97qm0sn/WF+tgAYKAN/sfy
	XG12GdZFJSciKIlVNiKxpLoKEcRD3UAtHkUPnmKFJUst4zFP20lEGyjW8Qcopgn9UZyE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185925-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185925: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=692f6199bd9a45c93952142c610f0a7dd3b3ef8e
X-Osstest-Versions-That:
    xen=f95cd010cbf0914154a0c2775c979d9158b1a3cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 19:09:43 +0000

flight 185925 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185925/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185922
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185922
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185922
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185922
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185922
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185922
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  692f6199bd9a45c93952142c610f0a7dd3b3ef8e
baseline version:
 xen                  f95cd010cbf0914154a0c2775c979d9158b1a3cb

Last test of basis   185922  2024-05-06 01:55:17 Z    0 days
Testing same since   185925  2024-05-06 11:40:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Federico Serafini <federico.serafini@bugseng.com>
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f95cd010cb..692f6199bd  692f6199bd9a45c93952142c610f0a7dd3b3ef8e -> master


From xen-devel-bounces@lists.xenproject.org Mon May 06 23:09:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 May 2024 23:09:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717831.1120327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s47Sb-000323-8O; Mon, 06 May 2024 23:09:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717831.1120327; Mon, 06 May 2024 23:09:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s47Sb-00031w-5I; Mon, 06 May 2024 23:09:29 +0000
Received: by outflank-mailman (input) for mailman id 717831;
 Mon, 06 May 2024 23:09:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s47Sa-00031j-RR; Mon, 06 May 2024 23:09:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s47Sa-0006lG-G3; Mon, 06 May 2024 23:09:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s47SZ-00082n-V5; Mon, 06 May 2024 23:09:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s47SZ-0000wk-Un; Mon, 06 May 2024 23:09:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1HVXGRRbB5tBbMXvVVnF2EnKMRdw5KiocCEGv7CqCGA=; b=Bc2ydaiG38QoBRMC2x0OKDcQJe
	ZuwHNH/OUm/1YzJufi7XhLSUTBOGoA6gLtGI7eUL5Z1P2U1PQyoK07OOBW7dCH0zz7wKMOBGTLo2c
	65Ed13AEsYzowCwoWE/qzfAF6xMliTk+mEClJNxf4cO7CNlWMVYLozOkfR6GJHvy/8t4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185928-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185928: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:debian-di-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=dd5a440a31fae6e459c0d6271dddd62825505361
X-Osstest-Versions-That:
    linux=b9158815de525572333d0499a681459f6b075f28
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 May 2024 23:09:27 +0000

flight 185928 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185928/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot         fail in 185923 pass in 185928
 test-armhf-armhf-libvirt-vhd  8 xen-boot         fail in 185923 pass in 185928
 test-armhf-armhf-xl-raw     12 debian-di-install fail in 185923 pass in 185928
 test-armhf-armhf-xl          10 host-ping-check-xen        fail pass in 185923
 test-armhf-armhf-xl-credit2   8 xen-boot                   fail pass in 185923

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185920

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 185923 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 185923 never pass
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185923 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185923 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185920
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185920
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185920
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185920
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                dd5a440a31fae6e459c0d6271dddd62825505361
baseline version:
 linux                b9158815de525572333d0499a681459f6b075f28

Last test of basis   185920  2024-05-05 17:44:03 Z    1 days
Testing same since   185923  2024-05-06 01:57:00 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adrian Huang <ahuang12@lenovo.com>
  Andy Lutomirski <luto@kernel.org>
  Ashish Kalra <ashish.kalra@amd.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Gaurav Batra <gbatra@linux.ibm.com>
  Ingo Molnar <mingo@kernel.org>
  Jiri Olsa <jolsa@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Nageswara R Sastry <rnsastry@linux.ibm.com>
  Nayna Jain <nayna@linux.ibm.com>
  Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
  Thomas Gleixner <tglx@linutronix.de>
  Zqiang <qiang.zhang1211@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   b9158815de525..dd5a440a31fae  dd5a440a31fae6e459c0d6271dddd62825505361 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 07 01:35:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 01:35:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717841.1120336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s49jQ-0002wl-4s; Tue, 07 May 2024 01:35:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717841.1120336; Tue, 07 May 2024 01:35:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s49jQ-0002we-22; Tue, 07 May 2024 01:35:00 +0000
Received: by outflank-mailman (input) for mailman id 717841;
 Tue, 07 May 2024 01:34:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M4Rs=MK=flex--seanjc.bounces.google.com=3P4U5ZgYKCWgYKGTPIMUUMRK.IUSdKT-JKbKRROYZY.dKTVXUPKIZ.UXM@srs-se1.protection.inumbo.net>)
 id 1s49jP-0002wY-2y
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 01:34:59 +0000
Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com
 [2607:f8b0:4864:20::1149])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 020584d2-0c12-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 03:34:56 +0200 (CEST)
Received: by mail-yw1-x1149.google.com with SMTP id
 00721157ae682-61e0949fc17so51016017b3.0
 for <xen-devel@lists.xenproject.org>; Mon, 06 May 2024 18:34:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 020584d2-0c12-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20230601; t=1715045695; x=1715650495; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id
         :reply-to;
        bh=VvGkjH9SPvsjHvBH/WrCy4dSbt+wcqHhkwU5LfzuS0Q=;
        b=rSbNxVVJZQW8h9Rc3NCEYE5rj86k7b/JKWp+u8a06cl1R5LCF6LbCQTBVaCnagZ1dh
         K0+AXioYpy5VsH2jU0LbqE1HXu53NgA1YkcaHRQXqv+nNgX8/RohytAe6x7DNKLMz+EN
         j+9smKgfeY2RhS1A57zCvz8JU9OGfhatePqZ6EwJ/m4AONcTpMP89PQMi1UfcnRyFxno
         2xPfMMSWHIldAuoV1Lhf9THQNmwfFl9QBrB7V9vFSWI/ogiikhWBuH93tpVY00wxJ8Dr
         cgF8Bn5H6bgYQj4r7318FzWVIqPkraEX3s8yHcyHYS9Za9sWjXoTiLsYGKsIRClD+r1/
         5caA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715045695; x=1715650495;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=VvGkjH9SPvsjHvBH/WrCy4dSbt+wcqHhkwU5LfzuS0Q=;
        b=FBYEphw08/jHZ3rI597+oVCePTor8h2KFgrHbnLbw5Z6fqnfUC0t7M6y5wBayM5FTA
         cKHRzpM+xHpOGTFBBqdAt0RiQv9cLLzVUmxC9nH1tCMRnXUzl/2u2/pThL5CZR8FuaiH
         xl4XkHelB+axs7DY8PrSH4fBkeZQPs3r9o2l51eRlMI6YrvI3k4wlBqK/ScfFlRNsk9P
         MFKf76IeVAFR6Nn6LosiO0wuCb5muy/eNB5YHukMknD4tHbfQ+c0bjfnkEgJPTu1Vsw1
         Ire6yHoMdWNjCUuBp0qDgsqT1Z4Ko0S1r6Je16v3QuTrhtL1UTRJd5NCXyDiTi8yxHqq
         HssQ==
X-Forwarded-Encrypted: i=1; AJvYcCXbyRexej6TtBDGA8iG/+FidZ6w9aO4/a/o4quxFM+ntv/zoZX15cNZdwwd9b5WXg4k1uXFU3wgAoyUiw/mdJ/BaOUsHT88EWsxktX2tzY=
X-Gm-Message-State: AOJu0YwkUDjE+zBIM1dKCwk5Ug8eCjorUxBxkpJCmKND0OwOzA9i0vwE
	q2PSYbBf6n3KNCOsnfAVD5Vr3s9hYWsLwtw289LEhWz/a3gTYINsp2l1hA8X7ZEJdpcV0g1YhFg
	Y2w==
X-Google-Smtp-Source: AGHT+IEuRRNgC6bmH2Ym7Hdh0kKkoS5NCGgVEW4kTiZkpma4/j2gQOn7KVKqRXFpAim0ldEq7Pn05YmuCaI=
X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37])
 (user=seanjc job=sendgmr) by 2002:a05:6902:1893:b0:de4:67d9:a2c6 with SMTP id
 cj19-20020a056902189300b00de467d9a2c6mr1291648ybb.2.1715045695256; Mon, 06
 May 2024 18:34:55 -0700 (PDT)
Date: Mon, 6 May 2024 18:34:53 -0700
In-Reply-To: <20240506.ohwe7eewu0oB@digikod.net>
Mime-Version: 1.0
References: <20240503131910.307630-1-mic@digikod.net> <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com> <20240506.ohwe7eewu0oB@digikod.net>
Message-ID: <ZjmFPZd5q_hEBdBz@google.com>
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
From: Sean Christopherson <seanjc@google.com>
To: "=?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?=" <mic@digikod.net>
Cc: Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	"Mihai =?utf-8?B?RG9uyJt1?=" <mdontu@bitdefender.com>, 
	"=?utf-8?B?TmljdciZb3IgQ8OuyJt1?=" <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, 
	Will Deacon <will@kernel.org>, Yu Zhang <yu.c.zhang@linux.intel.com>, 
	"=?utf-8?Q?=C8=98tefan_=C8=98icleru?=" <ssicleru@bitdefender.com>, dev@lists.cloudhypervisor.org, 
	kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, 
	virtualization@lists.linux-foundation.org, x86@kernel.org, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 06, 2024, Micka=C3=ABl Sala=C3=BCn wrote:
> On Fri, May 03, 2024 at 07:03:21AM GMT, Sean Christopherson wrote:
> > > ---
> > >=20
> > > Changes since v1:
> > > * New patch. Making user space aware of Heki properties was requested=
 by
> > >   Sean Christopherson.
> >=20
> > No, I suggested having userspace _control_ the pinning[*], not merely b=
e notified
> > of pinning.
> >=20
> >  : IMO, manipulation of protections, both for memory (this patch) and C=
PU state
> >  : (control registers in the next patch) should come from userspace.  I=
 have no
> >  : objection to KVM providing plumbing if necessary, but I think usersp=
ace needs to
> >  : to have full control over the actual state.
> >  :=20
> >  : One of the things that caused Intel's control register pinning serie=
s to stall
> >  : out was how to handle edge cases like kexec() and reboot.  Deferring=
 to userspace
> >  : means the kernel doesn't need to define policy, e.g. when to unprote=
ct memory,
> >  : and avoids questions like "should userspace be able to overwrite pin=
ned control
> >  : registers".
> >  :=20
> >  : And like the confidential VM use case, keeping userspace in the loop=
 is a big
> >  : beneifit, e.g. the guest can't circumvent protections by coercing us=
erspace into
> >  : writing to protected memory.
> >=20
> > I stand by that suggestion, because I don't see a sane way to handle th=
ings like
> > kexec() and reboot without having a _much_ more sophisticated policy th=
an would
> > ever be acceptable in KVM.
> >=20
> > I think that can be done without KVM having any awareness of CR pinning=
 whatsoever.
> > E.g. userspace just needs to ability to intercept CR writes and inject =
#GPs.  Off
> > the cuff, I suspect the uAPI could look very similar to MSR filtering. =
 E.g. I bet
> > userspace could enforce MSR pinning without any new KVM uAPI at all.
> >=20
> > [*] https://lore.kernel.org/all/ZFUyhPuhtMbYdJ76@google.com
>=20
> OK, I had concern about the control not directly coming from the guest,
> especially in the case of pKVM and confidential computing, but I get you

Hardware-based CoCo is completely out of scope, because KVM has zero visibi=
lity
into the guest (well, SNP technically allows trapping CR0/CR4, but KVM real=
ly
shouldn't intercept CR0/CR4 for SNP guests).

And more importantly, _KVM_ doesn't define any policies for CoCo VMs.  KVM =
might
help enforce policies that are defined by hardware/firmware, but KVM doesn'=
t
define any of its own.

If pKVM on x86 comes along, then KVM will likely get in the business of def=
ining
policy, but until that happens, KVM needs to stay firmly out of the picture=
.

> point.  It should indeed be quite similar to the MSR filtering on the
> userspace side, except that we need another interface for the guest to
> request such change (i.e. self-protection).
>=20
> Would it be OK to keep this new KVM_HC_LOCK_CR_UPDATE hypercall but
> forward the request to userspace with a VM exit instead?  That would
> also enable userspace to get the request and directly configure the CR
> pinning with the same VM exit.

No?  Maybe?  I strongly suspect that full support will need a richer set of=
 APIs
than a single hypercall.  E.g. to handle kexec(), suspend+resume, emulated =
SMM,
and so on and so forth.  And that's just for CR pinning.

And hypercalls are hampered by the fact that VMCALL/VMMCALL don't allow for
delegation or restriction, i.e. there's no way for the guest to communicate=
 to
the hypervisor that a less privileged component is allowed to perform some =
action,
nor is there a way for the guest to say some chunk of CPL0 code *isn't* all=
owed
to request transition.  Delegation and restriction all has to be done out-o=
f-band.

It'd probably be more annoying to setup initially, but I think a synthetic =
device
with an MMIO-based interface would be more powerful and flexible in the lon=
g run.
Then userspace can evolve without needing to wait for KVM to catch up.

Actually, potential bad/crazy idea.  Why does the _host_ need to define pol=
icy?
Linux already knows what assets it wants to (un)protect and when.  What's m=
issing
is a way for the guest kernel to effectively deprivilege and re-authenticat=
e
itself as needed.  We've been tossing around the idea of paired VMs+vCPUs t=
o
support VTLs and SEV's VMPLs, what if we usurped/piggybacked those ideas, w=
ith a
bit of pKVM mixed in?

Borrowing VTL terminology, where VTL0 is the least privileged, userspace la=
unches
the VM at VTL0.  At some point, the guest triggers the deprivileging sequen=
ce and
userspace creates VTL1.  Userpace also provides a way for VTL0 restrict acc=
ess to
its memory, e.g. to effectively make the page tables for the kernel's direc=
t map
writable only from VTL1, to make kernel text RO (or XO), etc.  And VTL0 cou=
ld then
also completely remove its access to code that changes CR0/CR4.

It would obviously require a _lot_ more upfront work, e.g. to isolate the k=
ernel
text that modifies CR0/CR4 so that it can be removed from VTL0, but that sh=
ould
be doable with annotations, e.g. tag relevant functions with __magic or wha=
tever,
throw them in a dedicated section, and then free/protect the section(s) at =
the
appropriate time.

KVM would likely need to provide the ability to switch VTLs (or whatever th=
ey get
called), and host userspace would need to provide a decent amount of the ba=
ckend
mechanisms and "core" policies, e.g. to manage VTL0 memory, teardown (turn =
off?)
VTL1 on kexec(), etc.  But everything else could live in the guest kernel i=
tself.
E.g. to have CR pinning play nice with kexec(), toss the relevant kexec() c=
ode into
VTL1.  That way VTL1 can verify the kexec() target and tear itself down bef=
ore
jumping into the new kernel.=20

This is very off the cuff and have-wavy, e.g. I don't have much of an idea =
what
it would take to harden kernel text patching, but keeping the policy in the=
 guest
seems like it'd make everything more tractable than trying to define an ABI
between Linux and a VMM that is rich and flexible enough to support all the
fancy things Linux does (and will do in the future).

Am I crazy?  Or maybe reinventing whatever that McAfee thing was that led t=
o
Intel implementing EPTP switching?


From xen-devel-bounces@lists.xenproject.org Tue May 07 02:42:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 02:42:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717849.1120347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Am9-0003rw-PZ; Tue, 07 May 2024 02:41:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717849.1120347; Tue, 07 May 2024 02:41:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Am9-0003rp-MZ; Tue, 07 May 2024 02:41:53 +0000
Received: by outflank-mailman (input) for mailman id 717849;
 Tue, 07 May 2024 02:41:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Am8-0003rf-9f; Tue, 07 May 2024 02:41:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Am7-0002GO-Tz; Tue, 07 May 2024 02:41:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Am7-00087E-DP; Tue, 07 May 2024 02:41:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Am7-0002Iu-C8; Tue, 07 May 2024 02:41:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=38PuD+31bKoUEAi+cxXs+u8cibaTyswGUc6uns9xiQ4=; b=A0gG6x7lym3rdHo2OEXMoDF+rj
	enpnXUW5fI6UzPs/U+9wJZTGMjYfeZKS79zNgAShpiEPgusBAru0AaDb94Qo/Yokl2nvQVWRBcU9s
	uXMez/QtHdjnehB6VzmV7B7UDiUMZLo4a3av0e316oqqmRu3NcI/BCdY4IzXsbBm2/5c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185931-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185931: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=17f333f2a450656101aa4cb46d24b7cf4ee80ebf
X-Osstest-Versions-That:
    ovmf=24fa360857ce9e181cce0e5dad44cf0810119ff4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 02:41:51 +0000

flight 185931 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185931/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 17f333f2a450656101aa4cb46d24b7cf4ee80ebf
baseline version:
 ovmf                 24fa360857ce9e181cce0e5dad44cf0810119ff4

Last test of basis   185916  2024-05-04 04:13:05 Z    2 days
Testing same since   185931  2024-05-07 00:43:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dionna Glaze <dionnaglaze@google.com>
  Liming Gao <gaoliming@byosoft.com.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   24fa360857..17f333f2a4  17f333f2a450656101aa4cb46d24b7cf4ee80ebf -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 07 04:58:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 04:58:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717862.1120357 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Cty-0002PP-Rl; Tue, 07 May 2024 04:58:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717862.1120357; Tue, 07 May 2024 04:58:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Cty-0002PI-Nc; Tue, 07 May 2024 04:58:06 +0000
Received: by outflank-mailman (input) for mailman id 717862;
 Tue, 07 May 2024 04:58:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Ctx-0002P8-9W; Tue, 07 May 2024 04:58:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Ctx-0004yw-63; Tue, 07 May 2024 04:58:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Ctw-0003pf-Q9; Tue, 07 May 2024 04:58:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Ctw-000204-Pj; Tue, 07 May 2024 04:58:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4bwWf7J6xbuqf3yvZ3kxg/zWKPbWv8XpiaJez+sK0C0=; b=gg7MSryDmuMj9pAwU2j1cxd+EP
	Hh/nYguUYKaQoIv6rivBfGsDoIPVJ9ynUU5IKSA4Ao9sZ47/mcz8po+YfAUqMXVk/6ih7Y4gIATN6
	jip36OBnCyBmQYtY8JlQlecYtluZxODiHNuq0tVcsQmrudokD4L4oyiKic3dOy0mcPyE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185932-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185932: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=c12bbc14900aa5c70eec8c0576757c2182db3d01
X-Osstest-Versions-That:
    ovmf=17f333f2a450656101aa4cb46d24b7cf4ee80ebf
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 04:58:04 +0000

flight 185932 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185932/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 c12bbc14900aa5c70eec8c0576757c2182db3d01
baseline version:
 ovmf                 17f333f2a450656101aa4cb46d24b7cf4ee80ebf

Last test of basis   185931  2024-05-07 00:43:02 Z    0 days
Testing same since   185932  2024-05-07 02:44:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Xianglei Cai <xianglei.cai@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   17f333f2a4..c12bbc1490  c12bbc14900aa5c70eec8c0576757c2182db3d01 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 07 05:36:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 05:36:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717870.1120367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4DUr-0007ew-Dg; Tue, 07 May 2024 05:36:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717870.1120367; Tue, 07 May 2024 05:36:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4DUr-0007ep-B6; Tue, 07 May 2024 05:36:13 +0000
Received: by outflank-mailman (input) for mailman id 717870;
 Tue, 07 May 2024 05:36:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=yupB=MK=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1s4DUp-0007ej-V2
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 05:36:11 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b3fc7131-0c33-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 07:36:07 +0200 (CEST)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 66D67227A87; Tue,  7 May 2024 07:36:04 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3fc7131-0c33-11ef-b4bb-af5377834399
Date: Tue, 7 May 2024 07:36:04 +0200
From: Christoph Hellwig <hch@lst.de>
To: mhklinux@outlook.com
Cc: robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, jgross@suse.com,
	sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, hch@lst.de,
	m.szyprowski@samsung.com, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
	petr@tesarici.cz, roberto.sassu@huaweicloud.com
Subject: Re: [PATCH 2/2] iommu/dma: Fix zero'ing of bounce buffer padding
 used by untrusted devices
Message-ID: <20240507053604.GA31731@lst.de>
References: <20240408041142.665563-1-mhklinux@outlook.com> <20240408041142.665563-2-mhklinux@outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240408041142.665563-2-mhklinux@outlook.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Sun, Apr 07, 2024 at 09:11:42PM -0700, mhkelley58@gmail.com wrote:
> I've wondered if this code for zero'ing the pre- and post-padding
> should go in swiotlb_tbl_map_single(). The bounce buffer proper is
> already being initialized there. But swiotlb_tbl_map_single()
> would need to test for an untrusted device (or have a "zero the
> padding" flag passed in as part of the "attrs" argument), which
> adds complexity. Thoughts?

If we want to go down that route it should be the latter.  I'm
not sure if it is an improvement, but we'd have to implement it
to see if it does.

> The commit ID of Petr's patch is X'ed out above because Petr's patch
> hasn't gone into Linus' tree yet. We can add the real commit ID once
> this patch is ready to go in.

I've fixed that up and commit the series.

Thanks a lot!



From xen-devel-bounces@lists.xenproject.org Tue May 07 05:51:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 05:51:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717874.1120377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4DjJ-0001vr-Kf; Tue, 07 May 2024 05:51:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717874.1120377; Tue, 07 May 2024 05:51:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4DjJ-0001vk-HV; Tue, 07 May 2024 05:51:09 +0000
Received: by outflank-mailman (input) for mailman id 717874;
 Tue, 07 May 2024 05:51:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4DjI-0001va-7e; Tue, 07 May 2024 05:51:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4DjH-0006T0-Ph; Tue, 07 May 2024 05:51:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4DjH-0005Ks-Cz; Tue, 07 May 2024 05:51:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4DjH-0002V9-CX; Tue, 07 May 2024 05:51:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wfPliQej7yOyL1X91ps4doOfxWZBUUlb8EAIGhK9qdg=; b=MN7COb56ify4vqffyJKX/cjNge
	cukxyd5cVgc6tqHCIQzHCaTsgw+jwCB7UifJ/ML80mIIIOpICsXtaFsJMwnswf+GAPCiiW6edd1do
	mi1PT0N8AsD2MUs5isoWvdDIq4bGd17QxHVtzy85dWwOU12FWuOjHvBh6UEa04iddaaM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185929-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185929: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
X-Osstest-Versions-That:
    xen=692f6199bd9a45c93952142c610f0a7dd3b3ef8e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 05:51:07 +0000

flight 185929 xen-unstable real [real]
flight 185933 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185929/
http://logs.test-lab.xenproject.org/osstest/logs/185933/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185933-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 185933-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185933 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185933 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185925
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185925
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185925
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185925
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185925
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185925
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
baseline version:
 xen                  692f6199bd9a45c93952142c610f0a7dd3b3ef8e

Last test of basis   185925  2024-05-06 11:40:31 Z    0 days
Testing same since   185929  2024-05-06 19:40:27 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   692f6199bd..ebab808eb1  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81 -> master


From xen-devel-bounces@lists.xenproject.org Tue May 07 07:13:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 07:13:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717894.1120403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4F0r-0003GE-IH; Tue, 07 May 2024 07:13:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717894.1120403; Tue, 07 May 2024 07:13:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4F0r-0003G7-F8; Tue, 07 May 2024 07:13:21 +0000
Received: by outflank-mailman (input) for mailman id 717894;
 Tue, 07 May 2024 07:13:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4F0q-0003Fx-9u; Tue, 07 May 2024 07:13:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4F0q-0007xA-1k; Tue, 07 May 2024 07:13:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4F0p-0007wQ-Oh; Tue, 07 May 2024 07:13:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4F0p-0001no-O5; Tue, 07 May 2024 07:13:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pws/QV6/jXb7Tidv4Qd7WLwrkMDXpmylBNpOmbLdusQ=; b=I0UlHcAH1PbfOTNqhHpR6kVyAB
	hWgUa10BOTLaP35hdMVL/5Okkh9Bka3P9tQHnYT79BSH3hum5S3sWkm7qTbOjI4z1Kd5385bYKlpO
	1AbGOQHnkSyLlubCzEFUp08uOkH5ceFXhMUVuZlgIRHBBCrlRAt06GyyMexaC6F7vFCI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185935-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185935: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=1c0d4ae2c0fd24164873947c2e262c499ecf13b5
X-Osstest-Versions-That:
    ovmf=c12bbc14900aa5c70eec8c0576757c2182db3d01
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 07:13:19 +0000

flight 185935 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185935/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 1c0d4ae2c0fd24164873947c2e262c499ecf13b5
baseline version:
 ovmf                 c12bbc14900aa5c70eec8c0576757c2182db3d01

Last test of basis   185932  2024-05-07 02:44:06 Z    0 days
Testing same since   185935  2024-05-07 05:12:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Xianglei Cai <xianglei.cai@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   c12bbc1490..1c0d4ae2c0  1c0d4ae2c0fd24164873947c2e262c499ecf13b5 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 07 08:24:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 08:24:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717919.1120433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7Z-0004G9-CD; Tue, 07 May 2024 08:24:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717919.1120433; Tue, 07 May 2024 08:24:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7Z-0004G2-8x; Tue, 07 May 2024 08:24:21 +0000
Received: by outflank-mailman (input) for mailman id 717919;
 Tue, 07 May 2024 08:24:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dSAm=MK=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s4G7X-0003kv-A7
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 08:24:19 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 316d029e-0c4b-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 10:24:17 +0200 (CEST)
Received: from BY5PR17CA0010.namprd17.prod.outlook.com (2603:10b6:a03:1b8::23)
 by SA3PR12MB7877.namprd12.prod.outlook.com (2603:10b6:806:31b::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 08:24:12 +0000
Received: from CO1PEPF000066EB.namprd05.prod.outlook.com
 (2603:10b6:a03:1b8:cafe::dc) by BY5PR17CA0010.outlook.office365.com
 (2603:10b6:a03:1b8::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Tue, 7 May 2024 08:24:12 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066EB.mail.protection.outlook.com (10.167.249.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 08:24:11 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:10 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:10 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 03:24:08 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 316d029e-0c4b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cir1bWtVTDDZiusjo6Y2irTyEaA96THmEn15TBtAcvuTY5AfHv3ZGPbf57uoqAQLfYlMmFNty8QLU3WKyHDiiR+JLk1ElMELPnigFFx62AWMfbgBKO8pL0mUdZujAekwLMDz7PFKN8JaHE3igiPsOEKSQD8j3+pVbx2To8j9MAtZkJgzEqmc8a+WadKj1DoiZCJrSWFZXLrGPC0MMov4GbJdTLi81pQj7BbtSnR9OpDwfXl9Ka4DRiT3rrZJXBO7x5mH0icVXmnkwnuzoRtE0Mrnli5aeEbuNU6Oh4J3Ib5bG5UV8NrpheRPJcOzluAyvaPSrENtgH+2m+onDSsZbA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UCoREhGdiGdJh4nqENuQ2tKLWDW2JKQq7+BvW87Gh/k=;
 b=fP0JsAyw9MdF+mZ3W/l+XokiHx7jTAavRMKbmi3KZSfcaoOhTp4+Gbr1T+XTSSEv1g94XIkndLag7PKwuibueaUERgL6SMgealrD2ptsdU+/uvUfrU4y2+NB9coIScQ7VeCW/IZkUpxqGmOa/8014q2ilC6aktMF2cgAd/d6NhibUdwRSNgkPdn5YPz2XuTQn3b6vSYbsi+IgwguIuYsV2gAJYYj0gZRtGFOpHm9fd4U6lXApK7beE2KWFV0BRbsGRZjcbxCp3iOT/x1qZt8zOwdpA9MZE107+nHkRdas3cGWk1A8FWgxXSFLbpRsFRSn/TgrXgZG2yR1/63JeaR/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UCoREhGdiGdJh4nqENuQ2tKLWDW2JKQq7+BvW87Gh/k=;
 b=lBmaY/qh33tdt1A44CaQYAPcL5ZpldxmpbgxVLzChrlM322DIA3OlZUI3SS9mPUX84B02i07J0wAVDzwUypaOxdgK3N4X4RrPukpRHQF+jSQSEglwPT51VKIC7MiHBxTfEbBTqt4a/hTxjfyWl0BF0E8JMKrZ+gJcnf9kaNozuM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Anthony PERARD <anthony.perard@cloud.com>
Subject: [PATCH v2 2/2] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Tue, 7 May 2024 16:24:00 +0800
Message-ID: <20240507082400.25146-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240507082400.25146-1-xin.wang2@amd.com>
References: <20240507082400.25146-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EB:EE_|SA3PR12MB7877:EE_
X-MS-Office365-Filtering-Correlation-Id: 1814a71d-a3d2-44dc-f6f5-08dc6e6f12b3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?ir3m0+WSeaR40l1L9cgqXPwAMqnGeCluPMNwBDXUx1E6V4qHocQAdjFzquYn?=
 =?us-ascii?Q?rQmjIKKGA1gt3eDZudDOqdElOIIVSqSalcKL8I5cLMJxeNes0mdm4w7ZkTFk?=
 =?us-ascii?Q?CZgen6rxde14AbC1lMYIT+uE3oCu13Q2KwsfwUU3rWBgELJH37olUoCcENT+?=
 =?us-ascii?Q?NxyVO3YqV+sX0ane9qFp4G55yo+8WOup9A4Olxi9vquZLAqPHpZOx3IG38JW?=
 =?us-ascii?Q?FtaeESnzUvjVvajOHIXCBZaaEEQGKswFfFh0/Xstkzr723kT1X3DnPb+Q6eo?=
 =?us-ascii?Q?ugC0e8SRMLwFJ5qt2d9KxSpJpkRGSQqKSTH4+mdyTRrUBnGkRc9ypI1fkdHQ?=
 =?us-ascii?Q?fXmz1nmApLStVJnGS8rLnAkxxswkjuDJUelWCSwKlk/N2tQIb15gUJLiNPXv?=
 =?us-ascii?Q?51iAl5BF8zm808KzF+IVV1xpv9R6dHwPQOWxh3hi6uYTVfIhxLu9jUOeQy7m?=
 =?us-ascii?Q?Atrl0/nlEAFAAtWNaY/4yC9GitFLzhOf2eAglGbUHIC/uaPhXahfAYCONYxP?=
 =?us-ascii?Q?b2fwU+BeBbv2BaGeuk5zr5AirtBpO1dkgwpp9LQE6lsyZrhQr5m585PagGru?=
 =?us-ascii?Q?m8vDDswuIryfK+uyEVI3hSpGu9Cyyg8KzgmDaKVAMmTPnpumFKcZ2y9mXcDR?=
 =?us-ascii?Q?1kvbYKqslbHEqAA7p8wIJ3itXnR8F/wpi2c+Th193eCG+3Jfgn1fIuRZUYon?=
 =?us-ascii?Q?7Tr51Q1EaKtiIQfkm7ZP1qvlxqGZ9CcDPphcqFhWxAhzoslbiCFx9vRMMQxk?=
 =?us-ascii?Q?mG7AbQvADZ1bEF4hC/C33EaJ93o3ZhdiQLts5UC5gR7ZH9NnIELHX14ydPeK?=
 =?us-ascii?Q?rihqApX6vU80EIQm8IPvpZay+6iIXt8MWFLK0WXy8ZytEwe+9XCAImcBeQH8?=
 =?us-ascii?Q?qmi6KEcbR2ud9+ZUKh/hi7BblFqe9lYz7H5Av/1YrSYXwX3SFIA1y+UhyCWx?=
 =?us-ascii?Q?QhMI2ZQZH5L1Fx0orKeXsQ8hEVvvZXVWreJpgtR23vUkBMVq6b/6HskLD8Se?=
 =?us-ascii?Q?SZAgduGbmBpPbVUKg/3Op1Lp+oPNEFPXKv4RP4GMDHVQf8HnYnqc7GXSzsdT?=
 =?us-ascii?Q?i3xrY5DU8JkTp7JbLNAVAabEKjOZ9H1RF+QwT1QdRKzVvU5BUqOUpREmob0O?=
 =?us-ascii?Q?NEEQg1Xzy3IjEB/8ptvivE0Y+2mRHyqQZF0Drw6keAHiIAJoWOYrTqNpExRa?=
 =?us-ascii?Q?Dfgkc3rGKRrb+WWggzyotFtoeugwtaejTOBrpuSoE7EonlwrRRS9lY7sHC+N?=
 =?us-ascii?Q?JmuyW+eAXTVZckXdfaxXo1hI3puiKW/Qi4ORQRg1g7JIASmke9Kzxhnihpnj?=
 =?us-ascii?Q?C0/M4brDdXTWPH6/bzHCOcMJ1MBUKooih+OBM0TLPPaVvZXWm6MPbLPwtzvd?=
 =?us-ascii?Q?IewL1dM=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 08:24:11.2414
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1814a71d-a3d2-44dc-f6f5-08dc6e6f12b3
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7877

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony.perard@cloud.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch
---
 tools/xl/xl_vmcontrol.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 07 08:24:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 08:24:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717917.1120413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7S-0003lC-TD; Tue, 07 May 2024 08:24:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717917.1120413; Tue, 07 May 2024 08:24:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7S-0003l5-Pd; Tue, 07 May 2024 08:24:14 +0000
Received: by outflank-mailman (input) for mailman id 717917;
 Tue, 07 May 2024 08:24:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dSAm=MK=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s4G7R-0003kv-NO
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 08:24:13 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2deb08e5-0c4b-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 10:24:11 +0200 (CEST)
Received: from PH7PR17CA0059.namprd17.prod.outlook.com (2603:10b6:510:325::25)
 by DS7PR12MB6335.namprd12.prod.outlook.com (2603:10b6:8:94::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Tue, 7 May
 2024 08:24:07 +0000
Received: from CY4PEPF0000E9D9.namprd05.prod.outlook.com
 (2603:10b6:510:325:cafe::93) by PH7PR17CA0059.outlook.office365.com
 (2603:10b6:510:325::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43 via Frontend
 Transport; Tue, 7 May 2024 08:24:07 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D9.mail.protection.outlook.com (10.167.241.72) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 08:24:06 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:05 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:05 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 03:24:04 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2deb08e5-0c4b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ywr12NUKpueLDd9M8AaVDQEopdNkALRQY+SDvCn1FEcL3zCoGOT+9VYQiPdyTPXRNdqn+XEmUBZZiUJjou4sOHYWImiYjZ4wUHi2HbxpWFyKDzwsK24cA1ehwATDyNkDMcHwZlRA3NSJVf4Ztr3EPQqbDNls+5281Sb9KyTqif7xh2+f/AZ1kVodEY+g55WV6eiFvUxrnfKXaRA45PNELaRiYFAMlAUJZsz0g8t5CjE/squPn+tFZzcQUnpFr7AU1/BLw+QvS4dAWuMs5FZoKWEWS4vhqXA3XWhk0NZo6v74zuhlDWFOWWOQGVglCnT63sA7etbGOg7tGO6YeSE1Qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wxmswW5MKaVNrwibgvBTKnW3hc/vi2J/QjjoxBoYTLU=;
 b=jhSiq2Qkfjvbq+dTKU62fOXaUVGf4bhYocr4TH/+WMxecSRolWPS9bMT4F/nvWWtLI0seZ4ztiHGY79fujK+hCZK9dJq+sexGrRB+TKSfsgLhVeG0SIdFCvNBmE6ntGOexBK7Pgb31NopMqq+61wWN2qPhkW2BGM91O6oJ9F10KBzzPIY0GtZmMRJmrc44z1BsytcFzkBd3hvj99UcNun6ol7onm6j/7OjbmFIpFBNrS2PkLqFa+O+R7OsHZSgSHT+gGY9jZRAcEpqaRfUX+CSopbvxhsFLzaC2tAX5Y7hsY/nPIQOQgka9R5p85heyBTnhXYqvEjvUhdq5sKq/usQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wxmswW5MKaVNrwibgvBTKnW3hc/vi2J/QjjoxBoYTLU=;
 b=PqQOrdgHRBGZ7jBB3XOwL8aGDBY5iedEdCl+C74NlQRk59ZnOloPjHq/rabH5roENwhQduj3b+HdRS5u147Mv84U2YmRsaViJW8H/IoYU+7JHqF28XSKA2GcUUdLIWdIsCbSNKs0WKPPiwqq1DsVGx04PMDav2s5iPQm8sqp1ow=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Anthony PERARD
	<anthony@xenproject.org>
Subject: [PATCH v2 0/2] Some fixes for the existing dynamic dtbo code
Date: Tue, 7 May 2024 16:23:58 +0800
Message-ID: <20240507082400.25146-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D9:EE_|DS7PR12MB6335:EE_
X-MS-Office365-Filtering-Correlation-Id: 9ecc9d5b-5c1f-4d64-8793-08dc6e6f0fd5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?OR+ZMjCg85PvQuLNuq6Kbgi1/LtA5c15E85bgHpOM2oTzTAKhUdtVW0HWQcz?=
 =?us-ascii?Q?DdRp2VwsLE6u29KCOZj16m0EwYY+GomYfTaWEUt831CGfUyzHewPXRP78SPm?=
 =?us-ascii?Q?wa2PcLhKC6fGi9hlxcxWu7klAipGEUbmCIBaZGpfaZhpndGsUQqE6Z0X7NSj?=
 =?us-ascii?Q?dAECqBpGQlWUS08+RjeyJn1Wsyy7Z7gvbgShEIvxzYc5+vopRIgLBkQXu2qw?=
 =?us-ascii?Q?UpBZdS9SFQzVNRkrpPT683lswg2SkEe73rWvtD6YgBUqM8/yjo8E/wN+2Tuq?=
 =?us-ascii?Q?MdXUA/yDC3ZetgA350kjLbP0IePZkfLVP8wBWyOg7G1m/BcQfRYQp82VQVS4?=
 =?us-ascii?Q?4IT4LSR2hzlUPvkPrD9CQpzwRTe8z0r12jehgoYz6ns0VI4Fg0rN+4dMrog0?=
 =?us-ascii?Q?ngpv/2HLjSi3OOB5OeNGSPnKXNkhcJ9Fte85Ge75lwj0sePJLA8oM2p3rjtU?=
 =?us-ascii?Q?bqzCzDev95NdAk8uLNYBMJPmy1YJxGQ1bXvOsq31s3PCmjKKTw4tl6T5x8Nb?=
 =?us-ascii?Q?DRU3s55YXFJHB3B99wHhrvfQk+2FjjDFsy708MhO7Vrieuvi3mFVXmdKz93O?=
 =?us-ascii?Q?IuIvrLvWdr6l6gnW8vCQCWZaaS5uqhNFz1uMNJVr351bCSHZ1mPYHJ31Lk9k?=
 =?us-ascii?Q?cR2NTsOrsAtqoxLzLLb8n6zdbRMpotMjpqJQYDir+X+Eeg5Gx0/nyTTg1eNt?=
 =?us-ascii?Q?vRX7YqMc1mrRJkXD2Ju29NcZblVVeG8iSAbJl6A5s7hSnw1nqNOpqBJTHIAF?=
 =?us-ascii?Q?WW9TkAu+nzoHH6ffjE2MBKSK8CIYY7fpXTjhFxQMiNZmowUQAP3Q1E2FxbTV?=
 =?us-ascii?Q?LIA8GVfAaacd4r2h/AmrzZf0T+xl1SpyFNAcyCPD1Q182FAleXfVb3OlVXw5?=
 =?us-ascii?Q?rvH90zlXFXxpjOrqE7nMGMx2skxs4QXBBJ+40Q0vekeuBycpSF3FgCOeimUU?=
 =?us-ascii?Q?EdWVCzgByV7USsGsxidq08QvhuDCvUN0EoO+rRhshdQL2GwdrE2IOhP1NgR7?=
 =?us-ascii?Q?dJlIdzdjQsDuf+N6x4N0796OAooupPE9+FfODoSAFJR2Zqcd4flCEOGW+4d0?=
 =?us-ascii?Q?L5dvKm/gWZI3vY+x4d3Fdh6YCQacTirPZuwz9yK+0cKEiVOJMIiKk065AEL3?=
 =?us-ascii?Q?DXCrMXAu8I23+fpFHufk6+xGbbu7FE0L4SYgH8cb6h3td3LkTYbuGrawd3x0?=
 =?us-ascii?Q?1Eby7u3Gj0TX61Dok8nOseeg0lysoTpsWY9VQWyrCziPjRFaUpE6V1/zY7eY?=
 =?us-ascii?Q?vQSiA6nEqXe3yuGKYtZaiRbOn9Frb/e57GJ08UpmqlNH+meWm8AND/hevyO2?=
 =?us-ascii?Q?ekXQen450AmDrch+Z1mXEkyEiHHVGHu7CfwHwqgBSAXjpPA+ERgc0HWCrQEI?=
 =?us-ascii?Q?IOWvGjX+Zaal7W+0G8vgoM0q8y5y?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 08:24:06.4595
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ecc9d5b-5c1f-4d64-8793-08dc6e6f0fd5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D9.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6335

During the review process for the v1 of the dynamic dtbo series, some
issues of the existing code were identified. Discussions of them can
be found in [1] (for the first patch) and [2] (for the second patch).

Since the main part of the remaining dynamic dtbo series requires more
rework, just send these fixes for now.

[1] https://lore.kernel.org/xen-devel/835099c8-6cf0-4f6d-899b-07388df89319@xen.org/
[2] https://lore.kernel.org/xen-devel/eaea1986-a27e-4d6c-932f-1d0a9918861f@perard/

Henry Wang (2):
  xen/common/dt-overlay: Fix missing lock when remove the device
  tools/xl: Correct the help information and exit code of the dt-overlay
    command

 tools/xl/xl_vmcontrol.c | 6 +++---
 xen/common/dt-overlay.c | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 07 08:24:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 08:24:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717918.1120423 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7U-0003zO-5T; Tue, 07 May 2024 08:24:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717918.1120423; Tue, 07 May 2024 08:24:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4G7U-0003zH-1P; Tue, 07 May 2024 08:24:16 +0000
Received: by outflank-mailman (input) for mailman id 717918;
 Tue, 07 May 2024 08:24:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dSAm=MK=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s4G7S-0003kv-NM
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 08:24:14 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ee27881-0c4b-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 10:24:13 +0200 (CEST)
Received: from BY5PR17CA0004.namprd17.prod.outlook.com (2603:10b6:a03:1b8::17)
 by DS7PR12MB6046.namprd12.prod.outlook.com (2603:10b6:8:85::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Tue, 7 May
 2024 08:24:09 +0000
Received: from CO1PEPF000066ED.namprd05.prod.outlook.com
 (2603:10b6:a03:1b8:cafe::f) by BY5PR17CA0004.outlook.office365.com
 (2603:10b6:a03:1b8::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Tue, 7 May 2024 08:24:09 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 08:24:09 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:08 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 03:24:07 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 03:24:06 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ee27881-0c4b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DBZ36jiPbkTm799GQponF1AV0KXQ0h8xTCvyihe3HEzrVbM0qHS9kxzYgPqGrrkemqbcLx+N8kaZSks/QQjAD8EETQJ6VH3vi+kyrMG348bTdH7uMT4gsxLH4UFo5rFKhnKIkB7VZIhhxWJOh0oPPn5USPIYROaNO0LSgwltm9iEqhVc4JGCAKGYgbKxZiAzUa+dgar7Ded7yO1WYq3qymwtiOFQC+ECtff2kdXbX0eiN/sE8SMWazrwwBLHZxnROlH3kp1gRe8SrVX7tmlvh+5CQ1L5uZ2bVYUxAfa4p7Blw8wXpG2zYBD+80SNnvOx75lUOo88xV23KKtfQ4N6Mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wD4Edd/ZUQf6Rvz7ZHLdHdBtw+JBBmdZ+XcaUxuIswo=;
 b=fJUGebMYw3QKaf8MVhrI4sWQ61lDNyNMFSgqIaht8vkVjoI4msse874aeUjKzWpTGpLUxhbvJrnrKW5Z++FwObtMIHqnBPqjFzLs/6Le6HrHw+w6i3I2QEHg2Yup37bvOdzhyk6U2II1NMfVxNTwap5GDofq3dlpC0PQJzhIoIzVjZnFQP9zK6IdD1T71tDnKUX94GbwFauauo8/k5Ydtnvl+rUrugeeLcvVlYGAq5BFhJquba63CiDKnetnbCgzb1tLMwl44FlUP3Vxz430H+qrRRkkJCpEgomLi/he8iJuBRUzdtCMMLY/etSc00IeWOl+vXhhjloCO911l4rQYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wD4Edd/ZUQf6Rvz7ZHLdHdBtw+JBBmdZ+XcaUxuIswo=;
 b=iLT1x5zNabal3zNcpLs2N+k84rObxRzW30yoV6H8RcR9T0WejI779Gp0N062nzsVk3ZZwRo8O+rxeJ6zyQCarVKb18ToYwl+S6XJyaDRnMvC9zlcwteS6ewzVa7b9HdHwLIpn4fsDiKD9qZd7tuPSnlUplOPgrMfueSQ2vBysZk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Subject: [PATCH v2 1/2] xen/common/dt-overlay: Fix missing lock when remove the device
Date: Tue, 7 May 2024 16:23:59 +0800
Message-ID: <20240507082400.25146-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240507082400.25146-1-xin.wang2@amd.com>
References: <20240507082400.25146-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|DS7PR12MB6046:EE_
X-MS-Office365-Filtering-Correlation-Id: 4de37103-84ff-4aef-8b8a-08dc6e6f1162
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?25w2L+oTI6tJh9+1ACt4/ZjMLfP34T3DdYnO7/+YykwAbXBof3YcS7frECyS?=
 =?us-ascii?Q?sv1aIqis/gj4VPNaNHln3ap/Gqji3Nx820JU4DRPC8JJH2G0fQrGGYIkv3/O?=
 =?us-ascii?Q?EGm4yDvpM9l5TTGmpC52q778P12WdKvzNKBA2b2wnZdiY78phSy+pLg7lHQY?=
 =?us-ascii?Q?Y9C7PxQ8xxkLlrkoZj8pYmbVABi358PlGZan3SrFyWEr+A5mRrNQk7JXNguh?=
 =?us-ascii?Q?xUtJBHCtxi2ZGWTFXjDBjF4hrOLGHH531ZUyk8ev3SRaP+HMltiosbd2+Rde?=
 =?us-ascii?Q?yclL2uIxE8IFlSDbPiVq09QbmypMxHq2gz2cqcdq7P3ljn8QcLacNvKnOmh7?=
 =?us-ascii?Q?ZHr4QpgqykkJb3jZue8CUVV4WCMonc5qV+89f/3Kjxhe/ZT3yPU3+/7zQBII?=
 =?us-ascii?Q?fRlEryX1zLvNgniSmRrnjmcbQtlk9GDVDxqNNOpHMqHS0sAdTkt3LGLMcXau?=
 =?us-ascii?Q?wi7qpMfIcqmm8aVMZN4O2Q8b+b4fE4n/Glws/WWefs+1TRC/AsMFgPEypB8j?=
 =?us-ascii?Q?N5smep32w+yPST14H14xs79i9ZwtVRHXf3+hx6Gd2lQjYxLLkKvxHZ3bgcxr?=
 =?us-ascii?Q?riVKT7CFMav9+L+u8LseX8FSs5EVko15KoN4wgX0WeUG22aZ2sq5h/NpZAjs?=
 =?us-ascii?Q?vY+J5cXCbw6LBUnrPZCo9Jo/Z7k2tTNY3Jx8fjQgEt0rDClQ2k7E1Loirv7E?=
 =?us-ascii?Q?evPflTUnvy5LHZ1H9VI3H2OiFrwVpyQzOFyrHywl/RygQHR8SX3HWljTawvV?=
 =?us-ascii?Q?Bo20HbcP16T+bf6LW39PNxq1b76ckR/qQ+i2R5zIcpp4YJuvXerYLt5pkNx2?=
 =?us-ascii?Q?O0ak3wsCG5k4Fb3tI2WExToel9Fxnr+fzL2I+v2pLcXluIuGbMz/fnCzUmwp?=
 =?us-ascii?Q?xoawOE1faebF5uHqZ+2fiIgj8PfdrpwzdDNoIcTPhcp0G+k1epx4RY8BaBrT?=
 =?us-ascii?Q?6qkM8rRIKyVwoxPg7Ow+X0BS4bOp/vmcyvWNxsH4nXGDUOe2F/+Gko3t4Qyt?=
 =?us-ascii?Q?YQ5TtIpFUBphE6Vp3KvgemNsLTGHsIWMy8RstPDpoCmttmS3l8XfVKsmzPmr?=
 =?us-ascii?Q?/zXRgboh6aoFKuW1gbKkQx0QDQ/DSvhf3J8hTUFLRmeyPHfHUlRixm5hs8RQ?=
 =?us-ascii?Q?YCnsI6dwD3DNfzLiFVoUzvGPJAzxI8yqOS6qdLh4PpjmkslOG1F14tIIw/mc?=
 =?us-ascii?Q?pu4Y8f5p8ouZJXVY8QwWXDILeBp+hXAYXb4WYt4HDDHqdEVPRMMVCFg/s/Va?=
 =?us-ascii?Q?tLAD+v+JltmyerKfBTP+DKKKrtzQEsJeWlh7zctsWNjQLMC6T+WzpwmJBBuh?=
 =?us-ascii?Q?RNRqdQiX+fnIGvfLYsnOXOMBGR/w6o41PczP9EGF7uGww3PqAQL1pScr0DvA?=
 =?us-ascii?Q?srJwgUurMDs3Sc8sjH9EKv50CrCk?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 08:24:09.0145
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4de37103-84ff-4aef-8b8a-08dc6e6f1162
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066ED.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6046

If CONFIG_DEBUG=y, below assertion will be triggered:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     00000a0000257418 iommu_remove_dt_device+0x8c/0xd4
(XEN) LR:     00000a00002573a0
(XEN) SP:     00008000fff7fb30
(XEN) CPSR:   0000000000000249 MODE:64-bit EL2h (Hypervisor, handler)
[...]

(XEN) Xen call trace:
(XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
(XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
(XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
(XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
(XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
(XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
(XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
(XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
(XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
(XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ****************************************

This is because iommu_remove_dt_device() is called without taking the
dt_host_lock. dt_host_lock is meant to ensure that the DT node will not
disappear behind back. So fix the issue by taking the lock as soon as getting
hold of overlay_node.

Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Take the lock as soon as getting hold of overlay_node.
v1.1:
- Move the unlock position before the check of rc.
---
 xen/common/dt-overlay.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 1b197381f6..25d15cbcb1 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -429,6 +429,8 @@ static int remove_nodes(const struct overlay_track *tracker)
         if ( overlay_node == NULL )
             return -EINVAL;
 
+        write_lock(&dt_host_lock);
+
         rc = remove_descendant_nodes_resources(overlay_node);
         if ( rc )
             return rc;
@@ -439,8 +441,6 @@ static int remove_nodes(const struct overlay_track *tracker)
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
 
-        write_lock(&dt_host_lock);
-
         rc = dt_overlay_remove_node(overlay_node);
         if ( rc )
         {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 07 09:30:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 09:30:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717942.1120443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4H9c-0005XU-Ad; Tue, 07 May 2024 09:30:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717942.1120443; Tue, 07 May 2024 09:30:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4H9c-0005XN-7n; Tue, 07 May 2024 09:30:32 +0000
Received: by outflank-mailman (input) for mailman id 717942;
 Tue, 07 May 2024 09:30:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SDgQ=MK=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s4H9b-0005XH-AI
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 09:30:31 +0000
Received: from smtp-190e.mail.infomaniak.ch (smtp-190e.mail.infomaniak.ch
 [185.125.25.14]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 71aeadee-0c54-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 11:30:29 +0200 (CEST)
Received: from smtp-4-0000.mail.infomaniak.ch (smtp-4-0000.mail.infomaniak.ch
 [10.7.10.107])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VYY0h64rLzhcB;
 Tue,  7 May 2024 11:30:28 +0200 (CEST)
Received: from unknown by smtp-4-0000.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VYY0d4NWlzhxd; Tue,  7 May 2024 11:30:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71aeadee-0c54-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=digikod.net;
	s=20191114; t=1715074228;
	bh=N7C083XW9HErJ1+R52TKnyEZNzil5v9FCazYyr/JQx0=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=g4cR8t/iRRSN5giqF0Iwy3XXWNCfhqE/qqqfoxy0lSmFuyqhdQ4FyOhVfsBJ5BFWv
	 xFaM80SdAPMBtmnCReKLZkFRcwvBisebzGQz+PahxsjXN1dm1r13uXB5fKC058GUn5
	 E40tFjEfU9DKYpAc1PGGy+RZ3L2pJN9bZLXwuqBg=
Date: Tue, 7 May 2024 11:30:24 +0200
From: =?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
To: Sean Christopherson <seanjc@google.com>
Cc: Borislav Petkov <bp@alien8.de>, 
	Dave Hansen <dave.hansen@linux.intel.com>, "H . Peter Anvin" <hpa@zytor.com>, 
	Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, Anna Trikalinou <atrikalinou@microsoft.com>, 
	Chao Peng <chao.p.peng@linux.intel.com>, Forrest Yuan Yu <yuanyu@google.com>, 
	James Gowans <jgowans@amazon.com>, James Morris <jamorris@linux.microsoft.com>, 
	John Andersen <john.s.andersen@intel.com>, "Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, 
	Marian Rotariu <marian.c.rotariu@gmail.com>, Mihai =?utf-8?B?RG9uyJt1?= <mdontu@bitdefender.com>, 
	=?utf-8?B?TmljdciZb3IgQ8OuyJt1?= <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, 
	Yu Zhang <yu.c.zhang@linux.intel.com>, =?utf-8?Q?=C8=98tefan_=C8=98icleru?= <ssicleru@bitdefender.com>, 
	dev@lists.cloudhypervisor.org, kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, 
	x86@kernel.org, xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
Message-ID: <20240507.ieghomae0UoC@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com>
 <20240506.ohwe7eewu0oB@digikod.net>
 <ZjmFPZd5q_hEBdBz@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZjmFPZd5q_hEBdBz@google.com>
X-Infomaniak-Routing: alpha

On Mon, May 06, 2024 at 06:34:53PM GMT, Sean Christopherson wrote:
> On Mon, May 06, 2024, Mickaël Salaün wrote:
> > On Fri, May 03, 2024 at 07:03:21AM GMT, Sean Christopherson wrote:
> > > > ---
> > > > 
> > > > Changes since v1:
> > > > * New patch. Making user space aware of Heki properties was requested by
> > > >   Sean Christopherson.
> > > 
> > > No, I suggested having userspace _control_ the pinning[*], not merely be notified
> > > of pinning.
> > > 
> > >  : IMO, manipulation of protections, both for memory (this patch) and CPU state
> > >  : (control registers in the next patch) should come from userspace.  I have no
> > >  : objection to KVM providing plumbing if necessary, but I think userspace needs to
> > >  : to have full control over the actual state.
> > >  : 
> > >  : One of the things that caused Intel's control register pinning series to stall
> > >  : out was how to handle edge cases like kexec() and reboot.  Deferring to userspace
> > >  : means the kernel doesn't need to define policy, e.g. when to unprotect memory,
> > >  : and avoids questions like "should userspace be able to overwrite pinned control
> > >  : registers".
> > >  : 
> > >  : And like the confidential VM use case, keeping userspace in the loop is a big
> > >  : beneifit, e.g. the guest can't circumvent protections by coercing userspace into
> > >  : writing to protected memory.
> > > 
> > > I stand by that suggestion, because I don't see a sane way to handle things like
> > > kexec() and reboot without having a _much_ more sophisticated policy than would
> > > ever be acceptable in KVM.
> > > 
> > > I think that can be done without KVM having any awareness of CR pinning whatsoever.
> > > E.g. userspace just needs to ability to intercept CR writes and inject #GPs.  Off
> > > the cuff, I suspect the uAPI could look very similar to MSR filtering.  E.g. I bet
> > > userspace could enforce MSR pinning without any new KVM uAPI at all.
> > > 
> > > [*] https://lore.kernel.org/all/ZFUyhPuhtMbYdJ76@google.com
> > 
> > OK, I had concern about the control not directly coming from the guest,
> > especially in the case of pKVM and confidential computing, but I get you
> 
> Hardware-based CoCo is completely out of scope, because KVM has zero visibility
> into the guest (well, SNP technically allows trapping CR0/CR4, but KVM really
> shouldn't intercept CR0/CR4 for SNP guests).
> 
> And more importantly, _KVM_ doesn't define any policies for CoCo VMs.  KVM might
> help enforce policies that are defined by hardware/firmware, but KVM doesn't
> define any of its own.
> 
> If pKVM on x86 comes along, then KVM will likely get in the business of defining
> policy, but until that happens, KVM needs to stay firmly out of the picture.
> 
> > point.  It should indeed be quite similar to the MSR filtering on the
> > userspace side, except that we need another interface for the guest to
> > request such change (i.e. self-protection).
> > 
> > Would it be OK to keep this new KVM_HC_LOCK_CR_UPDATE hypercall but
> > forward the request to userspace with a VM exit instead?  That would
> > also enable userspace to get the request and directly configure the CR
> > pinning with the same VM exit.
> 
> No?  Maybe?  I strongly suspect that full support will need a richer set of APIs
> than a single hypercall.  E.g. to handle kexec(), suspend+resume, emulated SMM,
> and so on and so forth.  And that's just for CR pinning.
> 
> And hypercalls are hampered by the fact that VMCALL/VMMCALL don't allow for
> delegation or restriction, i.e. there's no way for the guest to communicate to
> the hypervisor that a less privileged component is allowed to perform some action,
> nor is there a way for the guest to say some chunk of CPL0 code *isn't* allowed
> to request transition.  Delegation and restriction all has to be done out-of-band.
> 
> It'd probably be more annoying to setup initially, but I think a synthetic device
> with an MMIO-based interface would be more powerful and flexible in the long run.
> Then userspace can evolve without needing to wait for KVM to catch up.
> 
> Actually, potential bad/crazy idea.  Why does the _host_ need to define policy?
> Linux already knows what assets it wants to (un)protect and when.  What's missing
> is a way for the guest kernel to effectively deprivilege and re-authenticate
> itself as needed.  We've been tossing around the idea of paired VMs+vCPUs to
> support VTLs and SEV's VMPLs, what if we usurped/piggybacked those ideas, with a
> bit of pKVM mixed in?
> 
> Borrowing VTL terminology, where VTL0 is the least privileged, userspace launches
> the VM at VTL0.  At some point, the guest triggers the deprivileging sequence and
> userspace creates VTL1.  Userpace also provides a way for VTL0 restrict access to
> its memory, e.g. to effectively make the page tables for the kernel's direct map
> writable only from VTL1, to make kernel text RO (or XO), etc.  And VTL0 could then
> also completely remove its access to code that changes CR0/CR4.
> 
> It would obviously require a _lot_ more upfront work, e.g. to isolate the kernel
> text that modifies CR0/CR4 so that it can be removed from VTL0, but that should
> be doable with annotations, e.g. tag relevant functions with __magic or whatever,
> throw them in a dedicated section, and then free/protect the section(s) at the
> appropriate time.
> 
> KVM would likely need to provide the ability to switch VTLs (or whatever they get
> called), and host userspace would need to provide a decent amount of the backend
> mechanisms and "core" policies, e.g. to manage VTL0 memory, teardown (turn off?)
> VTL1 on kexec(), etc.  But everything else could live in the guest kernel itself.
> E.g. to have CR pinning play nice with kexec(), toss the relevant kexec() code into
> VTL1.  That way VTL1 can verify the kexec() target and tear itself down before
> jumping into the new kernel. 
> 
> This is very off the cuff and have-wavy, e.g. I don't have much of an idea what
> it would take to harden kernel text patching, but keeping the policy in the guest
> seems like it'd make everything more tractable than trying to define an ABI
> between Linux and a VMM that is rich and flexible enough to support all the
> fancy things Linux does (and will do in the future).

Yes, we agree that the guest needs to manage its own policy.  That's why
we implemented Heki for KVM this way, but without VTLs because KVM
doesn't support them.

To sum up, is the VTL approach the only one that would be acceptable for
KVM?  If yes, that would indeed require a *lot* of work for something
we're not sure will be accepted later on.

> 
> Am I crazy?  Or maybe reinventing whatever that McAfee thing was that led to
> Intel implementing EPTP switching?
> 


From xen-devel-bounces@lists.xenproject.org Tue May 07 09:55:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 09:55:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717955.1120452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HXe-0000W9-5n; Tue, 07 May 2024 09:55:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717955.1120452; Tue, 07 May 2024 09:55:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HXe-0000W2-3C; Tue, 07 May 2024 09:55:22 +0000
Received: by outflank-mailman (input) for mailman id 717955;
 Tue, 07 May 2024 09:55:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HXc-0000Vs-AE; Tue, 07 May 2024 09:55:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HXc-0002xk-0U; Tue, 07 May 2024 09:55:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HXb-0006fL-JS; Tue, 07 May 2024 09:55:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HXb-00065a-Iu; Tue, 07 May 2024 09:55:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=c+mP6+gJUNEZM2XGvpaxqGKSllbgtdnCJ9UMpdx/0ug=; b=6mK+6B3vrIIio9bUeFgMqLjsvk
	b4nu9Ko23akuH9AyWeHVlyFotAzKNCfZuYgwLGB/FCuAtts2fuPo/lxDg0Iowk7i6OgeR5gnziXpI
	Dxzh0TFEdMrsciyxuhz3JTowdomfK8e6bOxWzjpuSUHbj+iyknOq1Uhq0nlwBZAqoSX4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185937-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185937: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=987bea6525d70cd01649472c93d19f89d41d83a2
X-Osstest-Versions-That:
    ovmf=1c0d4ae2c0fd24164873947c2e262c499ecf13b5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 09:55:19 +0000

flight 185937 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185937/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 987bea6525d70cd01649472c93d19f89d41d83a2
baseline version:
 ovmf                 1c0d4ae2c0fd24164873947c2e262c499ecf13b5

Last test of basis   185935  2024-05-07 05:12:56 Z    0 days
Testing same since   185937  2024-05-07 07:43:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abdul Lateef Attar <AbdulLateef.Attar@amd.com>
  Ray Ni <ray.ni@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   1c0d4ae2c0..987bea6525  987bea6525d70cd01649472c93d19f89d41d83a2 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 07 09:57:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 09:57:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717963.1120463 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HZm-00012y-Iq; Tue, 07 May 2024 09:57:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717963.1120463; Tue, 07 May 2024 09:57:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HZm-00012r-Fm; Tue, 07 May 2024 09:57:34 +0000
Received: by outflank-mailman (input) for mailman id 717963;
 Tue, 07 May 2024 09:57:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4HZk-00012j-OP
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 09:57:32 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4HZj-00031K-R8; Tue, 07 May 2024 09:57:31 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4HZj-00069J-G4; Tue, 07 May 2024 09:57:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=FxG6s7sf/5Zp7OQTdwIn5Zm+iEX8TkB+QnNL89TCXpY=; b=GSKhzQNE5tyy92R1izzT+WtUUG
	H4JHEM5lZBj2YfdKFgREN7Q0MpoLSLhWUEPAmnGGaloHkqvPvwdrbM7mosU5qwNrJvnfkYB9T94zq
	AAswUSIS2rKZU1mKufsF2SnXLweyp/76YS7G0AvuRXFxS7B1Bkdt4GTfsg0j3E54+U2E=;
Message-ID: <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org>
Date: Tue, 7 May 2024 10:57:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol
 annotations
Content-Language: en-GB
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, jbeulich@suse.com,
 andrew.cooper3@citrix.com, edgar.iglesias@amd.com
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com>
 <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
 <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 06/05/2024 13:54, Edgar E. Iglesias wrote:
> On Sat, May 4, 2024 at 2:14 AM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
>>
>> On Wed, 1 May 2024, Edgar E. Iglesias wrote:
>>> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>>>
>>> Use the generic xen/linkage.h macros to annotate code symbols
>>> and add missing annotations.
>>>
>>> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>>> ---
>>>   xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
>>>   1 file changed, 12 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
>>> index 4e63825220..b16e4d1e29 100644
>>> --- a/xen/arch/arm/arm64/bpi.S
>>> +++ b/xen/arch/arm/arm64/bpi.S
>>> @@ -52,14 +52,15 @@
>>>    * micro-architectures in a system.
>>>    */
>>>       .align   11
>>> -ENTRY(__bp_harden_hyp_vecs_start)
>>> +FUNC(__bp_harden_hyp_vecs_start)
>>>       .rept 4
>>>       vectors hyp_traps_vector
>>>       .endr
>>> -ENTRY(__bp_harden_hyp_vecs_end)
>>> +GLOBAL(__bp_harden_hyp_vecs_end)
>>> +END(__bp_harden_hyp_vecs_start)
>>
>> Shouldn't GLOBAL be changed to FUNC as well?
>>
> 
> I was a bit unsure but went for GLOBAL since the _end labels point to
> addresses after and outside of the code sequence.
> But I don't have a strong opinion and am happy to change them to FUNC
> if you feel that's better.

I don't think it should be FUNC as this is not meant to be called 
directly. I am also under the impression, we were planning to get rid of 
GLOBAL() as well.

Furthermore, __bp_harden_hyp_vec_start is not a function per say. It is 
a pointer to the vector table.

 From the brief look, the same remarks would apply to the rest of bpi.S. 
So I think we want to switch all the ENTRY() to LABEL().

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 10:15:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 10:15:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717974.1120473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HrM-0004FD-2m; Tue, 07 May 2024 10:15:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717974.1120473; Tue, 07 May 2024 10:15:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4HrL-0004F6-Uy; Tue, 07 May 2024 10:15:43 +0000
Received: by outflank-mailman (input) for mailman id 717974;
 Tue, 07 May 2024 10:15:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HrK-0004Ew-FJ; Tue, 07 May 2024 10:15:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HrK-0003Ys-5H; Tue, 07 May 2024 10:15:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HrJ-0007Fh-LZ; Tue, 07 May 2024 10:15:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4HrJ-0004jF-L9; Tue, 07 May 2024 10:15:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ku7Cfid0qIlbbwM8/Cy066IFH18sl/vlgJPSOi2arAA=; b=bc5EE2P1rW466jE55bnGGkYT47
	MkZDa2/LK3GLgbRYNBOF13MWsVdtjayz1xPqPWrJmF7Pr2ak1cKEKQJXXVjV33Je6rFUSJn5CRXLM
	1kUZEvnWabjJBwpp7A8bJ/LPsYlZtm5BNqt+49NjsCj/Fz1idIMF/C48gQhyE8NlpGP4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185930-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185930: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=dccb07f2914cdab2ac3a5b6c98406f765acab803
X-Osstest-Versions-That:
    linux=dd5a440a31fae6e459c0d6271dddd62825505361
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 10:15:41 +0000

flight 185930 linux-linus real [real]
flight 185938 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185930/
http://logs.test-lab.xenproject.org/osstest/logs/185938/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 185938-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 185938 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 185938 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185928
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185928
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185928
 test-armhf-armhf-xl-credit2   8 xen-boot                     fail  like 185928
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185928
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185928
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185928
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                dccb07f2914cdab2ac3a5b6c98406f765acab803
baseline version:
 linux                dd5a440a31fae6e459c0d6271dddd62825505361

Last test of basis   185928  2024-05-06 15:43:38 Z    0 days
Testing same since   185930  2024-05-06 23:44:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Chengming Zhou <chengming.zhou@linux.dev>
  Dan Carpenter <dan.carpenter@linaro.org>
  David Rientjes <rientjes@google.com>
  David Sterba <dsterba@suse.com>
  Geert Uytterhoeven <geert@linux-m68k.org>
  Josef Bacik <josef@toxicpanda.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maxime Ripard <mripard@kernel.org>
  Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
  Qu Wenruo <wqu@suse.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vlastimil Babka <vbabka@suse.cz>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   dd5a440a31fae..dccb07f2914cd  dccb07f2914cdab2ac3a5b6c98406f765acab803 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 07 10:31:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 10:31:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.717991.1120483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4I6O-00073B-FM; Tue, 07 May 2024 10:31:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 717991.1120483; Tue, 07 May 2024 10:31:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4I6O-000734-BG; Tue, 07 May 2024 10:31:16 +0000
Received: by outflank-mailman (input) for mailman id 717991;
 Tue, 07 May 2024 10:31:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4I6M-00072s-HW; Tue, 07 May 2024 10:31:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4I6M-0003w9-5T; Tue, 07 May 2024 10:31:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4I6L-0007dZ-OI; Tue, 07 May 2024 10:31:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4I6L-0001vw-Np; Tue, 07 May 2024 10:31:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=I+rQHLDwmjJA4HP8JpzYmWw8qub0MqBHpgDvzyW0k+U=; b=C42AbTWa2RzwO0ivxNQhQ2d+LJ
	BzHOP1AtBp/EApVoRbHakGrRczu7D6KFITafp5ZNTIPxsbGX8abqfWpYnvt996wd8qgAOM7SVXdXy
	zsDyFNuE50iRzN/0PIHTI1e7R84gZcv8tZ9AvyNyshMETXGVAclI3hZtgjanGqy7J1QE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185934-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185934: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=da95bcb6b2d9b04958e0f2603202801dd29debb8
X-Osstest-Versions-That:
    libvirt=3146305fd3a610573963fe4858cc12ec1c4cf5c7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 10:31:13 +0000

flight 185934 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185934/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185908
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              da95bcb6b2d9b04958e0f2603202801dd29debb8
baseline version:
 libvirt              3146305fd3a610573963fe4858cc12ec1c4cf5c7

Last test of basis   185908  2024-05-03 04:20:40 Z    4 days
Testing same since   185934  2024-05-07 04:20:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Julis <ajulis@redhat.com>
  Ján Tomko <jtomko@redhat.com>
  Kristina Hanicova <khanicov@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Oleg Sviridov <oleg.sviridov@red-soft.ru>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   3146305fd3..da95bcb6b2  da95bcb6b2d9b04958e0f2603202801dd29debb8 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 07 11:08:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 11:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718009.1120493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Ig9-00034A-67; Tue, 07 May 2024 11:08:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718009.1120493; Tue, 07 May 2024 11:08:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Ig9-000343-3B; Tue, 07 May 2024 11:08:13 +0000
Received: by outflank-mailman (input) for mailman id 718009;
 Tue, 07 May 2024 11:08:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4Ig7-00033x-EY
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 11:08:11 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 164a0621-0c62-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 13:08:09 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-572669fd9f9so182499a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 04:08:09 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ig13-20020a056402458d00b0056bc0c44f02sm5710900edb.96.2024.05.07.04.08.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 04:08:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 164a0621-0c62-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715080088; x=1715684888; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=vAJJDFqhGr764DmADsx5a1oN1JBCRdtQwgQVTLAGJcQ=;
        b=pUsuV21O4GtmwYn9xRkkZA/HQEwao6095lbE0zOiJ445AgX/wZfu/w43/eYtwPPgxy
         W2TK1zuRY+ix5J+dPd8bWqKSvRFOwR+B0nTTcyuCn1nyTy/EgIQsarNAdx/xcuPFCxb2
         OE5ZgWKb0raZAlGdtEv+Ob2UUkfnE8wfvC84M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715080088; x=1715684888;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vAJJDFqhGr764DmADsx5a1oN1JBCRdtQwgQVTLAGJcQ=;
        b=fIJZ6P49PefyjBHGi0X7Ru+r2b0A/8Da6iWGbEp3Ys1fhW71xyWmJ+wv9ZT4OXtYND
         cFIOL/BqpC3RzbhOCMY6F2KVuoHcDVxCjuV7eNuta0AIzgxHyz6xfvzwyVswvySMrBAM
         lBq/6iFhH6keu1FA7dCDWblY74P617dvtmrWPvY4oMSPOtGsqoa/IwYjJVQV1UBkmLtE
         foxWRXlbvMHjNMtCE5vVtubY4bfnBNKanhoCRk1wK9pBMC9YtfgjOraYyrBm2ToWAvMI
         FZlgG2mSXOwhWuNKzLtx331P0C0q0ID7Hc6ArKiSMKsx450lYl9YolaGWtSeYkyofvdL
         Sc+g==
X-Gm-Message-State: AOJu0YxFYrRmmUsGCuCp53Z7tyBtlkJeKH66kIr6aRSwwO9LnJCUjKzy
	WteeUKMjcwKztn8QBpp48bzSicIflzF2+b5AEEUeiYQ2apaa7/pfppCxWSXyVWtpsJt8c2Fqbkp
	y
X-Google-Smtp-Source: AGHT+IHPINCMaGIC5WIiPHeKp+Mfb7tXVN1pbWnI2M4wLw3sBt8Uh2jIu226664I6aaGxji56nsMbw==
X-Received: by 2002:a50:cd15:0:b0:570:2198:b3ca with SMTP id z21-20020a50cd15000000b005702198b3camr9284848edi.14.1715080088452;
        Tue, 07 May 2024 04:08:08 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
Date: Tue,  7 May 2024 12:08:06 +0100
Message-Id: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

`xl devd` has been observed leaking /var/log/xldevd.log into children.

Link: https://github.com/QubesOS/qubes-issues/issues/8292
Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: Demi Marie Obenour <demi@invisiblethingslab.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Also entirely speculative based on the QubesOS ticket.
---
 tools/xl/xl_utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
index 17489d182954..060186db3a59 100644
--- a/tools/xl/xl_utils.c
+++ b/tools/xl/xl_utils.c
@@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pidfile)
         exit(-1);
     }
 
-    CHK_SYSCALL(logfile = open(fullname, O_WRONLY|O_CREAT|O_APPEND, 0644));
+    CHK_SYSCALL(logfile = open(fullname, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0644));
     free(fullname);
     assert(logfile >= 3);
 

base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
prerequisite-patch-id: 212e50457e9b6bdfd06a97da545a5aa7155bb919
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 07 11:17:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 11:17:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718014.1120503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4IpB-0004qH-2N; Tue, 07 May 2024 11:17:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718014.1120503; Tue, 07 May 2024 11:17:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4IpA-0004qA-Uh; Tue, 07 May 2024 11:17:32 +0000
Received: by outflank-mailman (input) for mailman id 718014;
 Tue, 07 May 2024 11:17:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7h5X=MK=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s4Ip9-0004q4-3I
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 11:17:31 +0000
Received: from mail-oo1-xc36.google.com (mail-oo1-xc36.google.com
 [2607:f8b0:4864:20::c36])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 63d531db-0c63-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 13:17:29 +0200 (CEST)
Received: by mail-oo1-xc36.google.com with SMTP id
 006d021491bc7-5b2058b16d3so1050982eaf.1
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 04:17:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63d531db-0c63-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715080648; x=1715685448; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EnxMVQLxN6lfLXdxL4OeHbWXPiJqXf0eZqdPM9jpIPY=;
        b=oXiQRlPdl5ZYuc7oocACoGEV0R1S0PKxfNhM1ppNaNUrodbVTv6DqChRcunn+Y0wBP
         oK0ihX6xqvZHrmZAPj0u2i8xtVGi1MsyMJ4GhxjrM4MpO/83uK9EC9dCqU5N2ssB6woH
         A2sAN6yo7n0fCSz3piYrfugPLZUlEZ68NpCaBRhNBcEsEBjfYf9vZ0aWyUiJtbjvfFFo
         rX11HSghdyOFktWvbokNoIvUmUmUtYaq2fQj1Cx1t7reuCL28QsLDVYAljOVVGEHmgjx
         oe1X/woDyRaqkikl2TWZuEmNmR6PAWQgu4BNqUNUk0JQXUaQoJTxGvjjZkwN3luMi1eO
         b3/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715080648; x=1715685448;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=EnxMVQLxN6lfLXdxL4OeHbWXPiJqXf0eZqdPM9jpIPY=;
        b=axvmwZ9mVez9IyuC3gSOfx66u4wSYP/mjuEW1IVR9JjGhmtjBD5Df57bnFnom/+R5J
         NZnq58UyPBf3FrBHuy5yTKNcDgqFEuRKQcmI0OyTAOedQ0b8ipt+AdugiTeodAfj6l0o
         16lUjxPyFXTEyw0gTo6KQlDAWb8O60QA44EkbGmwSpdxH+9zsUrArz1fnTLSBofHYTUn
         Gh9a9T6Ltc1HB2Q5lbeVn8F0V7C9FfvS8ifnxzXrxqlumU4QGKJwIujdrHNig8Q23u9s
         hr8jpM4z6c9F7Uo2gt2IWXe6iUmX/feDz7b7tt9aZfQg8E3+54p8zHiS2PnWKK6KDbmz
         L2Qw==
X-Gm-Message-State: AOJu0YyROBhMKjtlkXE+952z9zu/Kv2Q4WFecLO2FEUcXia81AkdviA6
	Mh/C0EVpkCsO4KsxUjaxdVyvnAfRNIRB2krJDqMVBImZd+ZHiC5ur8z14nj1rnptJAJT4jjbtoQ
	Z1y3+EzXsq33pH9vpQR/OQ66Zjsc7M8vGNGaP9g==
X-Google-Smtp-Source: AGHT+IFPb40kBdSq6e2UlwPFFb2+PpFqpZdbqA6mfmmn7A46+dNh903vNpriL6FBENJs88KoAeW1eleHY+B2kxa4fTE=
X-Received: by 2002:a4a:5441:0:b0:5aa:6404:936a with SMTP id
 t62-20020a4a5441000000b005aa6404936amr12790871ooa.5.1715080648523; Tue, 07
 May 2024 04:17:28 -0700 (PDT)
MIME-Version: 1.0
References: <20240502145645.1201613-1-jens.wiklander@linaro.org>
 <20240502145645.1201613-6-jens.wiklander@linaro.org> <935e3947-53ee-48c5-a10c-5fa7f572ae65@xen.org>
 <CAHUa44HdA_cdZcjbVL+xYXnLVkdK1hK5hfU_k+fZkS8=VAOy1A@mail.gmail.com> <447340c3-d57a-411b-8d4d-29e8683b27d9@xen.org>
In-Reply-To: <447340c3-d57a-411b-8d4d-29e8683b27d9@xen.org>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Tue, 7 May 2024 13:17:16 +0200
Message-ID: <CAHUa44Fc6pRDjUr1HiW7r=QB4kxCEdRz_G9ATLqEg2KqNp+gFA@mail.gmail.com>
Subject: Re: [XEN PATCH v4 5/5] xen/arm: ffa: support notification
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org, patches@linaro.org, 
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Julien,

On Fri, May 3, 2024 at 4:25=E2=80=AFPM Julien Grall <julien@xen.org> wrote:
>
> Hi Jens,
>
> On 03/05/2024 14:54, Jens Wiklander wrote:
> >>> +static int ffa_setup_irq_callback(struct notifier_block *nfb,
> >>> +                                  unsigned long action, void *hcpu)
> >>> +{
> >>> +    unsigned int cpu =3D (unsigned long)hcpu;
> >>> +    struct notif_irq_info irq_info =3D { };
> >>> +
> >>> +    switch ( action )
> >>> +    {
> >>> +    case CPU_ONLINE:
> >>
> >> Can't you execute the notifier in CPU_STARTING? This will be called on
> >> the CPU directly, so you should be able to use request_irq(...).
> >
> > I tried that first but it failed with the ASSERT_ALLOC_CONTEXT() in _xm=
alloc().
> >
> > I've also tested a three-step solution with CPU_UP_PREPARE,
> > CPU_STARTING, and CPU_UP_CANCELED.
> > My approach here is more direct, but it still suffers from a weakness
> > in error handling even if it seems quite unlikely to run out of heap
> > or for setup_irq() to fail at this stage.
>
> Ah I didn't notice that notify_cpu_starting() is called with IRQ
> disabled. I assumed they would be enabled.
>
> Then I would consider to do:
>
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 6efed876782e..db322672e508 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -389,6 +389,7 @@ void asmlinkage start_secondary(void)
>        */
>       init_maintenance_interrupt();
>       init_timer_interrupt();
> +    init_tee_interrupt();
>
>       local_abort_enable();
>
> And plumb through the TEE subsystem.

I'll use that in the next version, it should remove a lot of complex code.

Thanks,
Jens


From xen-devel-bounces@lists.xenproject.org Tue May 07 11:30:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 11:30:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718019.1120513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4J1G-0006rP-45; Tue, 07 May 2024 11:30:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718019.1120513; Tue, 07 May 2024 11:30:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4J1G-0006qt-0h; Tue, 07 May 2024 11:30:02 +0000
Received: by outflank-mailman (input) for mailman id 718019;
 Tue, 07 May 2024 11:30:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4J1F-0006i8-O5
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 11:30:01 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2380978a-0c65-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 13:30:00 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59b49162aeso611372566b.3
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 04:30:00 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bf14-20020a170907098e00b00a59c3e28917sm2963527ejc.70.2024.05.07.04.29.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 04:29:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2380978a-0c65-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715081399; x=1715686199; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=rUxoYmJy59sYBP/lL0GePoG6vQYveONH9tj5bH4Vfzc=;
        b=BC4CJtqXejJUGCAgslmeFMg0wTxJksdjPtpOl+N6UC3S51uoHv4NvesOGY3fkTkAjO
         1i3LujELi9zRMqXzP1V/S9lfSFLTdLE71mx+ERMAIYJc/LMbRczbqEA1KWDnx+D5W3KL
         VjBeO1vt2PdsN+iFKsQugtm08Oefm4F80pHmI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715081399; x=1715686199;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=rUxoYmJy59sYBP/lL0GePoG6vQYveONH9tj5bH4Vfzc=;
        b=JLDuk9hLv9MblirSI+YTKTj7Wt7mOb0UUhRMhwUV/CLJvLnAeEZYLi7TKYIYl4iERh
         Qne16Ik3rI3u63Min0WJkaq9iITPUv7MVugk3/hMXF+m85V2kA1TeEU60beoJdAtdRsK
         aqadg5gj2Em3RGsyWPj+HuMUfj3k2X0DsVlbyzNAiO/ju9i6suxW8w7qGkoTFg+fT7hK
         vLxK23Lhi8u4vZ177/WZhl/t1IvunnZ3k9ixOONkLOpvc60LIELVM+v1dyvtJGPhJwCy
         Uadhf1Vw0lRfSLBEE1xmruICXCVuW7N9M3qYKpIyAjYTFJW1n/7ubz5kR7Kh0Qfs7TLt
         He2w==
X-Gm-Message-State: AOJu0YyF4BMt/8DfI/imzky+Yazq18MtGFeRCG9TQ8NqYEKCqvlAuzY/
	7MwRKcf2UeZv7fMcVRL3bQ+Na92Icd+xMUo7JPPtk9EZ+A+y6rCBvct85IW91DVn/xDgHTmDEhR
	V
X-Google-Smtp-Source: AGHT+IEbS1utK86wR8VYYWGam4Ib5kTVH82qOPukxrTfu+ou1gmt1UdnSSShFNEnjMrzLC3EukyocA==
X-Received: by 2002:a17:907:938a:b0:a59:9b52:cfc4 with SMTP id cm10-20020a170907938a00b00a599b52cfc4mr6520827ejc.44.1715081399207;
        Tue, 07 May 2024 04:29:59 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] x86/cpu-policy: Fix migration from Ice Lake to Cascade Lake
Date: Tue,  7 May 2024 12:29:57 +0100
Message-Id: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Ever since Xen 4.14, there has been a latent bug with migration.

While some toolstacks can level the features properly, they don't shink
feat.max_subleaf when all features have been dropped.  This is because
we *still* have not completed the toolstack side work for full CPU Policy
objects.

As a consequence, even when properly feature levelled, VMs can't migrate
"backwards" across hardware which reduces feat.max_subleaf.  One such example
is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).

Extend the host policy's feat.max_subleaf to the hightest number Xen knows
about, similarly to how we extend extd.max_leaf for LFENCE_DISPATCH.  This
will allow VMs with a higher feat.max_subleaf than strictly necessary to
migrate in.

Eventually we'll manage to teach the toolstack how to avoid creating such VMs
in the first place, but there's still more work to do there.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/cpu-policy.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 4b6d96276399..a216fc8b886f 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -373,8 +373,13 @@ static void __init calculate_host_policy(void)
 
     p->basic.max_leaf =
         min_t(uint32_t, p->basic.max_leaf,   ARRAY_SIZE(p->basic.raw) - 1);
-    p->feat.max_subleaf =
-        min_t(uint32_t, p->feat.max_subleaf, ARRAY_SIZE(p->feat.raw) - 1);
+
+    /*
+     * p->feat is "just" featureset information.  We know about more than may
+     * be present in this hardware.  Also, VMs may have a higher max_subleaf
+     * than strictly necessary, and we can accept those too.
+     */
+    p->feat.max_subleaf = ARRAY_SIZE(p->feat.raw) - 1;
 
     max_extd_leaf = p->extd.max_leaf;
 

base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 07 11:32:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 11:32:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718022.1120522 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4J3I-00085d-FD; Tue, 07 May 2024 11:32:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718022.1120522; Tue, 07 May 2024 11:32:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4J3I-00085W-CZ; Tue, 07 May 2024 11:32:08 +0000
Received: by outflank-mailman (input) for mailman id 718022;
 Tue, 07 May 2024 11:32:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4J3G-00085Q-VP
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 11:32:07 +0000
Received: from fout2-smtp.messagingengine.com (fout2-smtp.messagingengine.com
 [103.168.172.145]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6d706293-0c65-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 13:32:04 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.nyi.internal (Postfix) with ESMTP id 701CA13808C4;
 Tue,  7 May 2024 07:32:03 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Tue, 07 May 2024 07:32:03 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 07:32:02 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d706293-0c65-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715081523;
	 x=1715167923; bh=XH1RoyO3KTqa5pRCLmz9Yhq5hK7bnWr/v1fAmjKrMsE=; b=
	ZOgmjlD3wNLb+4qkNSfWOTEq+GhXslq8SDeF7lXBt4c4nkanu0HFjG+91xygpson
	nbz2vXR6mHC8a6MMkLefcOoWBuHnmfDFFozsFpOEcUID0x1AR+NX/DSj5Ap+KVat
	S/RK9mlt+Cf3BsbiWioTWoPBDItN2sME8EwxXpwJGNajYEvfyOTjMxcDK9PaASSh
	boTQ/Y5e7igL9ov3YOR7EHq4CfhVYUBeIl0uxwx8DRIP9f3HgUdDWOFGjoydltqF
	eGn/3XWPm/Nd4C8JbqFE+LQoOsYBBXpdVHmHjnm3HVu0SZ77PmaI5NzzT6GjRPiV
	+dVcx56+++dfxIO8ee45qg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715081523; x=1715167923; bh=XH1RoyO3KTqa5pRCLmz9Yhq5hK7b
	nWr/v1fAmjKrMsE=; b=T6m3VsteyzqfMmaJGiXHX+R7/34Yjij2f1nvQJavU6Pr
	e/9xha130u/NYR0VHz5Hht95c5KQlVt8yo7fS5SBO6yjHs6I2hdwk4EuWduLzkjj
	SLD5mVhjhPStO6rnxurmdGdJCKhY0roUvR+nskkEmcETt3kvwgCKCMj+zzuNazjO
	Fo3SB6GwFu1f3WJSKf0tH59yoyI0RSsj8GslitLtKXIk0HF3ngkDlX8I5wyyUwt6
	YWmj4ymwJ2s5MIPeIEEQ/qxYtCs9/4Rgwh2XQxk0pApdEL+M8jdJleitPkF3UrRk
	QLWU2Vv6ZA9T0X4RrS0AtVfgahr9cX4sJL2cvyNU+g==
X-ME-Sender: <xms:MxE6ZhRhjSwe4jDeokO8-LbqkbL8VSeU-CGLCPAFtLSDknLYauzI5g>
    <xme:MxE6Zqxp0BspZ_zHQ4ze3b3qmeHh6VrigxDmqNfW2LOXpTAyf_eKJWhKgcB7h7g_r
    HL673fP_Lj8tQ>
X-ME-Received: <xmr:MxE6Zm3enaBQixlencohOrnPU8V7wZkScPqHlPdCXEeSLW6xSWIPlovPTEHDVtn1E4vSYRouSy1FqFyUDaocGLpL7CmE_5cEFg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgfeelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueek
    teetgefggfekudehteegieeljeejieeihfejgeevhfetgffgteeuteetueetnecuffhomh
    grihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:MxE6ZpAObRDQP-PwKAsUY5EI28u8EF7NYTERGDTCLHrH-sL8psKUrg>
    <xmx:MxE6ZqhXiILHfySaG0GKBzIdVXl12Y8sfSR-ap9RfhMW0GbnkLHoRg>
    <xmx:MxE6ZtrelBLY03oFjtzIAaUo0ldjMkOoz3T_nRwTRUB88cb4zwfphg>
    <xmx:MxE6ZlhQMJlIgl16gY9Ciw-2I7g56tXD9MLP-5OXkQ2fet9jqWuDYg>
    <xmx:MxE6ZieSylpj26gLbkb18D7SOz8NnfCVszKQMSsFF6nEt6VV7MLtCaiB>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 7 May 2024 13:32:00 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
Message-ID: <ZjoRMHmL8_K9_lsL@mail-itl>
References: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="vwnm9mLjO5513wFX"
Content-Disposition: inline
In-Reply-To: <20240507110806.1692135-1-andrew.cooper3@citrix.com>


--vwnm9mLjO5513wFX
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 7 May 2024 13:32:00 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC

On Tue, May 07, 2024 at 12:08:06PM +0100, Andrew Cooper wrote:
> `xl devd` has been observed leaking /var/log/xldevd.log into children.
>=20
> Link: https://github.com/QubesOS/qubes-issues/issues/8292
> Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: Demi Marie Obenour <demi@invisiblethingslab.com>
> CC: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>
>=20
> Also entirely speculative based on the QubesOS ticket.
> ---
>  tools/xl/xl_utils.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>=20
> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
> index 17489d182954..060186db3a59 100644
> --- a/tools/xl/xl_utils.c
> +++ b/tools/xl/xl_utils.c
> @@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pidfil=
e)
>          exit(-1);
>      }
> =20
> -    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY|O_CREAT|O_APPEND, 06=
44));
> +    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY | O_CREAT | O_APPEND=
 | O_CLOEXEC, 0644));

This one might be not enough, as the FD gets dup2()-ed to stdout/stderr
just outside of the context here, and then inherited by various hotplug
script. Just adding O_CLOEXEC here means the hotplug scripts will run
with stdout/stderr closed. The scripts shipped with Xen do redirect
stderr to a log quite early, but a) it doesn't do it for stdout, and b)
custom hotplug scripts are a valid use case.
Without that, I see at least few potential issues:
- some log messages may be lost (minor, but annoying)
- something might simply fail on writing to a closed FD, breaking the
  hotplug script
- FD 1 will be used as first free FD for any open() or similar call - if
  a tool later tries writing something to stdout, it will gets written
  to that FD - worse of all three

What should be the behavior of hotplug scripts logging? Should they
always take care of their own logging? If so, the hotplug calling part
should redirect stdout/stderr to /dev/null IMO. But if `xl` should
provide some default logging for them (like, the xldevd.log here?), then
the O_CLOEXEC should be set only after duplicating logfile over stdout/err.

>      free(fullname);
>      assert(logfile >=3D 3);
> =20
>=20
> base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
> prerequisite-patch-id: 212e50457e9b6bdfd06a97da545a5aa7155bb919

Which one is this? I don't see it in staging, nor in any of your
branches on xenbits. Lore finds "tools/libxs: Open /dev/xen/xenbus fds
as O_CLOEXEC" which I guess is correct, but I have no idea how it
correlates it, as this hash doesn't appear anywhere in the message, nor
its headers...

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--vwnm9mLjO5513wFX
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmY6ETAACgkQ24/THMrX
1yzRCwgAglPZYtGvsBtnApKrtka1bc81ZR7VRbolYJToBIGc0Mokx281fmyP4zwx
savkunZD/3tQK3cdxExdrsRV7fZvkzMMQDMYGmwlX7cJfukJe3RHsBzPzPWbk/j0
egNZhDfMCbq2fdLujxyPq+bCOBAa+IEVvxYIBmHZI+tq9XnHuzh/xGNBTfZl7+Wy
ZderiPNg7SoZEs8tXai5iausrjCr6AQS0TiwElAHVxTicXhmC3Gu3lw4P0UaWluM
6a3QENmwDTjuJiBXYQsjDtL22przyxS8fqMqVKQH59u/xlzuVXbHSyo9+TIuZ7i2
/diSgmMQHwmRr50TnAuQI7PxlqpdXA==
=D35V
-----END PGP SIGNATURE-----

--vwnm9mLjO5513wFX--


From xen-devel-bounces@lists.xenproject.org Tue May 07 11:51:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 11:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718033.1120533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4JLQ-0002vC-14; Tue, 07 May 2024 11:50:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718033.1120533; Tue, 07 May 2024 11:50:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4JLP-0002v5-Ub; Tue, 07 May 2024 11:50:51 +0000
Received: by outflank-mailman (input) for mailman id 718033;
 Tue, 07 May 2024 11:50:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J4VX=MK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4JLO-0002uv-CW
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 11:50:50 +0000
Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com
 [2607:f8b0:4864:20::829])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0a04f2d0-0c68-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 13:50:46 +0200 (CEST)
Received: by mail-qt1-x829.google.com with SMTP id
 d75a77b69052e-4349685c845so18649991cf.0
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 04:50:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 fb20-20020a05622a481400b00434efa0feaasm6361001qtb.1.2024.05.07.04.50.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 04:50:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a04f2d0-0c68-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715082646; x=1715687446; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=9MBrMJr0YxXM16vUgS6E+E+CVsMKyrrJZEQyXQwyNy0=;
        b=qtYu/qO/vsWM82kWqGdv9o+iYSdVZWRSY1GoAjAHgwSWUDfBYKpuwrA3Lr11YfsY8d
         VYZfRP1hZgrF/f64rOomMkoq5VvGD0W23kTGzXB6j1t15csSsd7qMtxOzuToe8eh7sIH
         /ekrYoqLrOKZW7BeDvt17IoLWUZgzFlY9TZmQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715082646; x=1715687446;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9MBrMJr0YxXM16vUgS6E+E+CVsMKyrrJZEQyXQwyNy0=;
        b=kkrYoJ6tnCNPlnFPH48vRv06yDd/a71NH/84BRAc2kdzWUQn2E/mvsFFbGAprRBiFU
         GjZJjJPm+69T6czY0z4n79eQOExAVRnfpYjHCEOWOpVJDVY6veRZRO8GlOKwlAFufcll
         2viNrqMjvMXm3E9PXU9yeoW1B5ftRmGWu01aijrgiiPTL+8+6tvNxlSsIB9ASu9KXI1l
         tj5ragc/Jvk/Q2rOg0yHQSdXlU6EbBx6FAXuJvoXUb/HT7hO82i2VPqP7M1v4Zf/eb2i
         53z4XuffBFAN6lND7Ej7Yy/drHNAxieklnzDncdnLXbz1DZCnvAryXrJpFE2hFECkN/F
         XQhg==
X-Gm-Message-State: AOJu0YzLpOr8aEc8QEWYpDM0fc3AbZR5IwUhUvAr+YVd60eJrtpJp5HV
	rZB/nXzsVz/ScKFsWNr8XEOSp62om6LiB4sLSE6pFzCcxNi8ZZ9jIEcpSdJuJTCJwsqUV+SMuwu
	C
X-Google-Smtp-Source: AGHT+IEPpq//f2byQ6AaKtUYrs5MxbiERCv1xZ157TKLa4mYLY2yDEAldx3HYi9FN3apX1GVVZ7Vew==
X-Received: by 2002:a05:622a:1aaa:b0:43b:7ac:ac79 with SMTP id s42-20020a05622a1aaa00b0043b07acac79mr16395081qtc.21.1715082646091;
        Tue, 07 May 2024 04:50:46 -0700 (PDT)
Date: Tue, 7 May 2024 13:50:43 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH] x86/cpu-policy: Fix migration from Ice Lake to Cascade
 Lake
Message-ID: <ZjoVk6JMjHGAB-EB@macbook>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240507112957.1701824-1-andrew.cooper3@citrix.com>

On Tue, May 07, 2024 at 12:29:57PM +0100, Andrew Cooper wrote:
> Ever since Xen 4.14, there has been a latent bug with migration.
> 
> While some toolstacks can level the features properly, they don't shink
> feat.max_subleaf when all features have been dropped.  This is because
> we *still* have not completed the toolstack side work for full CPU Policy
> objects.
> 
> As a consequence, even when properly feature levelled, VMs can't migrate
> "backwards" across hardware which reduces feat.max_subleaf.  One such example
> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
> 
> Extend the host policy's feat.max_subleaf to the hightest number Xen knows
> about, similarly to how we extend extd.max_leaf for LFENCE_DISPATCH.  This
> will allow VMs with a higher feat.max_subleaf than strictly necessary to
> migrate in.

Seeing what we do for max_extd_leaf, shouldn't we switch to doing what
you propose for feat.max_subleaf to max_extd_leaf also?

To allow migration between hosts that have 0x80000021.eax and hosts
that don't have such extended leaf.

cpu_has_lfence_dispatch kind of does that, but if lfence cannot be
made serializing then the max extended leaf is not expanded.  And we
should also likely account for more feature leafs possibly appearing
after 0x80000021?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:20:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:20:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718045.1120542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Jnh-0007Ii-Bl; Tue, 07 May 2024 12:20:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718045.1120542; Tue, 07 May 2024 12:20:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Jnh-0007IA-8z; Tue, 07 May 2024 12:20:05 +0000
Received: by outflank-mailman (input) for mailman id 718045;
 Tue, 07 May 2024 12:20:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4Jng-0006yi-3S
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:20:04 +0000
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com
 [2607:f8b0:4864:20::732])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1fc869c9-0c6c-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 14:20:00 +0200 (CEST)
Received: by mail-qk1-x732.google.com with SMTP id
 af79cd13be357-7928c5cb63fso215047585a.1
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 05:20:01 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 n22-20020ae9c316000000b007925fc27b1bsm4839542qkg.124.2024.05.07.05.19.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 05:20:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1fc869c9-0c6c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715084400; x=1715689200; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YRLVjkv17D9aBha14p2zSizQpD/IHwVWplpoCtaVJyk=;
        b=lbtJLP9NOOAhnd9uPi5BBvje7I7omfk4HJcUV0z782deJfW3xFaUthiLT0ZGVq0g6n
         p5rduN94SKYGpdpsYxHhOsNn/eOLJw4IRNfsDYOtGUHQrr5hvxhXDZFg6n1BXBpKtJz0
         s8hzlzHe79ymk/jDevdYVK8GxR/Dxxu5hmEcc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715084400; x=1715689200;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YRLVjkv17D9aBha14p2zSizQpD/IHwVWplpoCtaVJyk=;
        b=RM0Ogq5+CoTwyGnKgtkwwC91pw6kZ6OKT5yS+qPcQauJnuyAIAnnebCPyxSFMkF/zY
         Ses8S0GG+8QrIKCgLsgGWVdMV8dX39Ol6limi1piEcBJAQ8TvILhmnGaBgurUPbq4XPp
         xNOFlFw+fmREOj5lUSMkdqqznykTQ/ldKfAbdzZazD1Cb+U+uL4V5RQeF1lCT7pKSUW3
         UQ2OpKmw3gkWJeiVFPyrqHN+Qyvb2bvF6U0ArQdWAHzopYePaDqGKDl91je+W4brZi6d
         1KxnTepZfgLrPzfDPpRqyXevHveATjGA5U5V/lFrZ0wh4w1IZFcj2HlXFAcHECvZzNdE
         7knA==
X-Gm-Message-State: AOJu0Yway3Uqi47uNBUoCClB1oop66DI7kQVhT8X18gukaO9bURKiXOT
	DbIQkljUcdVEvrLF/AUx+DblPTmUo2AkkdKwq8eF79MdmEOWmbXukXRaB0w1/H4=
X-Google-Smtp-Source: AGHT+IEwdWF+ybwuXBQaBxCgzcQ7qQ0nOFVlTuEbMZ5zt0GyuDSonTxlr0DQrl2WmENPxRrGy9dfkA==
X-Received: by 2002:a05:620a:2490:b0:78d:5d86:ee3a with SMTP id i16-20020a05620a249000b0078d5d86ee3amr4862746qkn.27.1715084400641;
        Tue, 07 May 2024 05:20:00 -0700 (PDT)
Message-ID: <dd15568e-c987-4636-810b-ca81d86f6867@citrix.com>
Date: Tue, 7 May 2024 13:19:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix migration from Ice Lake to Cascade
 Lake
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <ZjoVk6JMjHGAB-EB@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZjoVk6JMjHGAB-EB@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/05/2024 12:50 pm, Roger Pau Monné wrote:
> On Tue, May 07, 2024 at 12:29:57PM +0100, Andrew Cooper wrote:
>> Ever since Xen 4.14, there has been a latent bug with migration.
>>
>> While some toolstacks can level the features properly, they don't shink
>> feat.max_subleaf when all features have been dropped.  This is because
>> we *still* have not completed the toolstack side work for full CPU Policy
>> objects.
>>
>> As a consequence, even when properly feature levelled, VMs can't migrate
>> "backwards" across hardware which reduces feat.max_subleaf.  One such example
>> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
>>
>> Extend the host policy's feat.max_subleaf to the hightest number Xen knows
>> about, similarly to how we extend extd.max_leaf for LFENCE_DISPATCH.  This
>> will allow VMs with a higher feat.max_subleaf than strictly necessary to
>> migrate in.
> Seeing what we do for max_extd_leaf, shouldn't we switch to doing what
> you propose for feat.max_subleaf to max_extd_leaf also?
>
> To allow migration between hosts that have 0x80000021.eax and hosts
> that don't have such extended leaf.
>
> cpu_has_lfence_dispatch kind of does that, but if lfence cannot be
> made serializing then the max extended leaf is not expanded.  And we
> should also likely account for more feature leafs possibly appearing
> after 0x80000021?

On second thoughts, this adjustment ought to be in the max policies only.

It's slightly different to LFENCE_DISPATCH, in that we don't actually
have any set bits in those leaves.

I'll do a different patch.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:27:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718051.1120553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4JuO-0008MW-1Y; Tue, 07 May 2024 12:27:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718051.1120553; Tue, 07 May 2024 12:27:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4JuN-0008MP-VF; Tue, 07 May 2024 12:26:59 +0000
Received: by outflank-mailman (input) for mailman id 718051;
 Tue, 07 May 2024 12:26:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r/6G=MK=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s4JuM-0008MJ-Ht
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:26:58 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 182e8ba7-0c6d-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:26:57 +0200 (CEST)
Received: from BN8PR04CA0035.namprd04.prod.outlook.com (2603:10b6:408:70::48)
 by PH0PR12MB7815.namprd12.prod.outlook.com (2603:10b6:510:28a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 12:26:50 +0000
Received: from BN2PEPF0000449D.namprd02.prod.outlook.com
 (2603:10b6:408:70:cafe::6) by BN8PR04CA0035.outlook.office365.com
 (2603:10b6:408:70::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43 via Frontend
 Transport; Tue, 7 May 2024 12:26:49 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF0000449D.mail.protection.outlook.com (10.167.243.148) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 12:26:49 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 07:26:49 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 07:26:49 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 07:26:47 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 182e8ba7-0c6d-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gx3w/6ERGkLHJIQXcakBDAMn9UdfAjlcYOTs4hhHLeMXIGrMqCmoLIZr1YVEM1OSQ/z+HFdgX0OfC5OmeNbPo78auBFPkvlCej+/e4bsfexuczdTibuhn2HnZ1i/HcQJVS7i+4VVgvVbGQDL+DddPJkwm4T4oB/ArSEAfaduRIUeuwzuAZbXg3WUug+dGPJXoqVo3fg32D5YUOU71/2TNvJNuBTMl7DOTNhFLIWjko3CImrv4Bou4mYATe27fZI9q8gWhqXsPRnD3Pyv4gCZP4+on5SenCzSHnI7fXaw+oxcZ8bVl6om0j/5L9SlLo/PmpVOdSbrKfJ/f1U18xmeUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=MzHcNyRSKZ7Zn+9CUKTIZM2VTaJxubh0xWFmsKJUM4Y=;
 b=Kzp3wFdEUKnWndON5teyx6/4Zo6AZ6/DjaL4D5BGSIAHMwY8Yi9FsG1FJDghkeGldDb27ktCKeKuBl8qeex6h3YY1cdIg2XdX0MrgkpyE9z5ZZ/Tz9dCvq7+Sbr01oUpWIKVhvXZDSpM0HXKL8uI3jbHUqTxgbJYrzv22GseqoHNPr/WDBguQA2W6cntrruVTxYvIQmg05IQf/3TeXeKfQExIAu+sAwpadaxfPn4gkLn2Z4qbTCMdSeM9A6d8p3gu0KAS5zk6il5b9zQpYVMMa4Ip87NVNFUbiCeRx2ubLGwSkD+io5qEscvN/ec6kDkK/azoP/LqrgWcoLcCMSTAQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MzHcNyRSKZ7Zn+9CUKTIZM2VTaJxubh0xWFmsKJUM4Y=;
 b=2w9F7THaQoN7AvkkmGESHAJwMo+19tZ5dpPU/rMNdZ1z7gv3DXOtSZwe9K8JFAy6HfEgncT3GDqnoy3bfcyY7GK14rxN36IT/JyIqHkUdIGuYXh+yFqJVPqQp14sGXP4whAWkxw2ghzt1W2zGl67taN7yskEPy6SUkhgsF0LrSI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <c8976284-1a27-4182-a214-9f5a10de6f95@amd.com>
Date: Tue, 7 May 2024 14:26:47 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Penny Zheng <Penny.Zheng@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF0000449D:EE_|PH0PR12MB7815:EE_
X-MS-Office365-Filtering-Correlation-Id: 9375707a-261e-47cb-4a39-08dc6e90f828
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?UC9jcm1NSmVOSU9SM3FGWVhjemR2OWM5WStUTG9lL3hyOEdDbkMzRFhGY2sv?=
 =?utf-8?B?UUkwMUxWT212dlZEMGV4bjFxOENjSFkrNklwbUtpVEoyNDJ5eXRob3lqUFhs?=
 =?utf-8?B?NUlvVUhwczFGd05TT3BLTTU2SnB1c1p2WEZ6bUJCN2xZNnJVTWV5blFhSkxC?=
 =?utf-8?B?VnhMcFBKMkp1R0pjREVMSys5aTVhWEZZSEg3R2hKMmo1RGNlMHpILzc2REVV?=
 =?utf-8?B?MTZUKzN4VTNrZVhob2tnTHhWUGJxMmRvNE5qdmliM00wRUdEeGpUdGpYaUFN?=
 =?utf-8?B?UHVNMU5CUStPOFRPUTYxNEp3eVA2NmJSNFozYllsdGFxOEN5TUVybnhFZXNT?=
 =?utf-8?B?c00yMWhoVnFYc0xnNFhFRDl6SENlTUZCVW9ucWw0N3FEZXVGWlo0Wk82cEg5?=
 =?utf-8?B?MG95SUErQk1nMndYdm9SMkZWa0FsYTVaMVoyQlU5Yll4RVFzRG5MZmx2VzVh?=
 =?utf-8?B?LytiTlJzUnMzMmhzeVdTVG9ka1V1d1F6a0ZoMk8vR2lQTGk4eFJ3MGJqTlY1?=
 =?utf-8?B?SmsvdWFPdlZRT2ZibmliWDJES1oyd2VnTGZVcmJTSmMwc2JNeWNwd2VuUFVM?=
 =?utf-8?B?ZFVpeXJDbjZVTWs3UVBWNTYxOERURXR6d1VLR2ZRUE5sZmw2OUxzY2FiR1FW?=
 =?utf-8?B?MlZuZlcxeWZ2akhOVWxEajZMYmQybnkwZTZsVUZLa3Y2TFMyWUM4U0oxTVZm?=
 =?utf-8?B?KzdYMElqYm5ZL3lhNWhFWHE2MnNWNm13R2t0UUZpdExGaWlpMmZEY01XZkM1?=
 =?utf-8?B?S05EZW9JY1h1WnFqbDVPY1I1YmJhcFRYakNSN08vMHJSV3dkVWhneHdlckYy?=
 =?utf-8?B?SnR1a0VGZHpuaDVyQWprOTJOZnZuSTlyeFpaWGZWZmZUVVFwRWRaeFBtb214?=
 =?utf-8?B?NXk5U3FpZ1AvdSt6Vkd3RDE0emdUSUd0VzdLQmxQSSt3OUVtOEE2UDIrTkFH?=
 =?utf-8?B?RFZLKzQxYVpYaEZGUm5RejdOSjFDTWVkK1llQmtFZXY2SmVkWWxyR1V2VWhw?=
 =?utf-8?B?dkZ0ZkNlbmlSNmxSRDlKS1BPbHpib2hvZ1ViQkZJTVNtU0VLcTVlV2Nkc0Vq?=
 =?utf-8?B?b09Yenp5dDE2d3dBaCtabUFMQlBkV0VHS1YwRHQ4eUw4SDFkS0JkbFlUdG1y?=
 =?utf-8?B?eVRYTWlldXpMcjV2QjlLc01vTUNWY29rUXhPS3pUYjZ1Tm1xT1hXTlp2RHRh?=
 =?utf-8?B?eHdFd2NJYUpVOFA0NzdrZURRVGJtY0NoU0lsNmNtZytzZmJFbHhqR0NRL0pk?=
 =?utf-8?B?Q0F3bmZXeWlHS2JtSTQ4eGt0eHdEeDRwVVBYKzY1UHMzUXBuRHVET1V2VVVS?=
 =?utf-8?B?cm1zbTZENnBpeUpXdko2QmJwTVBTWWMyeW5HV0F0Z2VsRWpmZHBkVjIwNGJC?=
 =?utf-8?B?SkJjdEsrUm9rNlFkWmdyN3VzSUdpQnlZRnFTTXV1RjdEamx2VXRZRXVpbU0v?=
 =?utf-8?B?MVhQekFVV3NxWUsydlBVSzlFTEJtb2tha0hBdzdQbkJnelBKcHlQc0RKZUc2?=
 =?utf-8?B?SW0wOHNEY0NhRmVNQm5yRU5RMzZtODF5MjNlcEphbDNyL1RrNGhqS3ZBY0t1?=
 =?utf-8?B?cUQzK1VPR3FqbXVkVnNtUlFLR3Q0SXd4UWlMZk00eUtwQzh0bEpMVXNCL3dE?=
 =?utf-8?B?Qi84STExZmNiNnhib09IRDF2NlZWd1pSVjZWL0RDMU83RHUxaE9Xa0pISkxz?=
 =?utf-8?B?OU1xdFlINFhPRTNqOE9qckZyT3dlU2VZQVlvTU5KcmhLbC9KM3NsRmdmRlk4?=
 =?utf-8?Q?nQoefUCrHbwpZyGwGwp2JZ2hNh38aRGJaBV2vu6?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 12:26:49.7035
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9375707a-261e-47cb-4a39-08dc6e90f828
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF0000449D.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7815

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
> 
> This commits implements a new function p2m_put_superpage to handle superpages,
> specifically for helping put extra references for foreign superpages.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v1:
>  - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
> ---
>  xen/arch/arm/mmu/p2m.c | 58 +++++++++++++++++++++++++++++++-----------
>  1 file changed, 43 insertions(+), 15 deletions(-)
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index 41fcca011cf4..479a80fbd4cf 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>      return rc;
>  }
> 
> -/*
> - * Put any references on the single 4K page referenced by pte.
> - * TODO: Handle superpages, for now we only take special references for leaf
> - * pages (specifically foreign ones, which can't be super mapped today).
> - */
> -static void p2m_put_l3_page(const lpae_t pte)
> +/* Put any references on the single 4K page referenced by mfn. */
> +static void p2m_put_l3_page(mfn_t mfn, unsigned type)
Shouldn't type be of p2m_type_t?

>  {
> -    mfn_t mfn = lpae_get_mfn(pte);
> -
> -    ASSERT(p2m_is_valid(pte));
> -
>      /*
>       * TODO: Handle other p2m types
>       *
> @@ -771,16 +763,53 @@ static void p2m_put_l3_page(const lpae_t pte)
>       * flush the TLBs if the page is reallocated before the end of
>       * this loop.
>       */
> -    if ( p2m_is_foreign(pte.p2m.type) )
> +    if ( p2m_is_foreign(type) )
>      {
>          ASSERT(mfn_valid(mfn));
>          put_page(mfn_to_page(mfn));
>      }
>      /* Detect the xenheap page and mark the stored GFN as invalid. */
> -    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
> +    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
>          page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
>  }
> 
> +/* Put any references on the superpage referenced by mfn. */
> +static void p2m_put_superpage(mfn_t mfn, unsigned int next_level, unsigned type)
Shouldn't type be of p2m_type_t?

> +{
> +    unsigned int i;
> +    unsigned int level_order = XEN_PT_LEVEL_ORDER(next_level);
> +
> +    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
> +    {
> +        if ( next_level == 3 )
> +            p2m_put_l3_page(mfn, type);
> +        else
> +            p2m_put_superpage(mfn, next_level + 1, type);
> +
> +        mfn = mfn_add(mfn, 1 << level_order);
> +    }
> +}
> +
> +/* Put any references on the page referenced by pte. */
> +static void p2m_put_page(const lpae_t pte, unsigned int level)
> +{
> +    mfn_t mfn = lpae_get_mfn(pte);
> +
> +    ASSERT(p2m_is_valid(pte));
> +
> +    /*
> +     * We are either having a first level 1G superpage or a
> +     * second level 2M superpage.
> +     */
> +    if ( p2m_is_superpage(pte, level) )
> +        return p2m_put_superpage(mfn, level + 1, pte.p2m.type);
> +    else
No need for this else

> +    {
> +        ASSERT(level == 3);
> +        return p2m_put_l3_page(mfn, pte.p2m.type);
> +    }
> +}
> +
>  /* Free lpae sub-tree behind an entry */
>  static void p2m_free_entry(struct p2m_domain *p2m,
>                             lpae_t entry, unsigned int level)
> @@ -809,9 +838,8 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>  #endif
> 
>          p2m->stats.mappings[level]--;
> -        /* Nothing to do if the entry is a super-page. */
> -        if ( level == 3 )
> -            p2m_put_l3_page(entry);
> +        p2m_put_page(entry, level);
> +
>          return;
>      }
> 
> --
> 2.34.1
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718063.1120562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBj-00037v-E7; Tue, 07 May 2024 12:44:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718063.1120562; Tue, 07 May 2024 12:44:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBj-00037o-BL; Tue, 07 May 2024 12:44:55 +0000
Received: by outflank-mailman (input) for mailman id 718063;
 Tue, 07 May 2024 12:44:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBi-00037i-NK
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:44:54 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97ab5713-0c6f-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 14:44:51 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id 66FDE1C001A6;
 Tue,  7 May 2024 08:44:48 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Tue, 07 May 2024 08:44:48 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:47 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97ab5713-0c6f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1715085888; x=1715172288; bh=T1P1ggud7vkTcl80BAOrqh+6kawfaaIO
	q43P1qzsTfU=; b=2f4ZS0MsrQxnuY00dLw6PYBxpW5faxoTLHQLT73G8LoUFzQk
	7bRJvcTzUvVmwNEG1nUGTcMKNES2f56uY5GRdUNbnTkAl6HnXNTfg7foYtKHsnDE
	/bOn+0NeAAtLtMSXgHMQnhLkKindDKNkcm31coDmK2qv1cbksP1gryGjLOmkcQRr
	V2r4eZmXYh6nmZcZUspDrBq2IwUJqVKW36+FHZUdtdAp/PvWsUy3D/xLSPAACM1j
	Uyz0sF/Kzw5gay8dZAtN9FPDbNDHHWUba7NYJHNM/Mc5U7o3+0BjgWVQ6RcwcAVZ
	eskULlJlRwgz6CGFeliAVU5991ngYKYdHk7QvA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm3; t=1715085888; x=1715172288; bh=T1P1ggud7vkTc
	l80BAOrqh+6kawfaaIOq43P1qzsTfU=; b=X7DbEQ7AhaPrQilG7heFSMlWRK7A/
	LfvIYjPCrZYFCymSnkg2QIXEBUEuIUM2YxiM6mHGRxkfF75EovV0CwjLLTEn8rMd
	ZeuaOY4lGFRcH7jpc2EbOVgZ9RIchdoDfCJbGv5ppe69wje3ohM+ZmiqYNZFmCGn
	wPJdmS772DbV/yXE7R3lJQ3QVCohJqfWTFRDDVpojW5b111K5q1l8fmKTYFMlCda
	B3Y52wgyjY9HayqqcVZrp0glmgKn6j/Arrur0g4iZS7hls6EVZw2ZX+bP2zAldOt
	kKHRobYYLBV4etZN+rMYl9W53CWR8lwI5tcDBpfCGYHSouU01diUO15vg==
X-ME-Sender: <xms:PyI6Zo_Dwj50ez3u4dfK-Y8sje6Z921FAiZDKt9VB6WaXVgiWrFtqQ>
    <xme:PyI6ZgvaaUbXbmtw18AtRZm6CDRBSx-KBVw049kyBt13Hs1N2K5gOmU57ZbteRFj6
    9WnJ9V1vEeiUg>
X-ME-Received: <xmr:PyI6ZuAQfJhNlxRPtjmsTLOHCBtFnPETH6L7L-qs5uxW1MJDVDsk7KuhfVNQy_knPuY3iy7A3bAY8Jod3I65G_bkNduUYa-UrnfEJk9w9n4Atdj3ANQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggt
    khhiuceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
    eqnecuggftrfgrthhtvghrnhephfefvdektdeivdfgieevteffveefgfdtheevtedtveej
    udffveethefhveeghfehnecuffhomhgrihhnpehgihhthhhusgdrtghomhdpghhithhlrg
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:PyI6ZofbX3fl2qpacOGA4XNZUQHVlFFk3V0THkIHmDySOvn8_QaqRA>
    <xmx:PyI6ZtOGw6EiS2uCF599xHFUY9-5CbppPo1eM_iMMuJ0Okx-kdsZFQ>
    <xmx:PyI6ZimWzFAk0pIsDGIammq3WptxF6k_tP7SFrpxa0Az0476mO4rSQ>
    <xmx:PyI6ZvtWbqSzMvNXWCyAEIUqRNYTxRkH7yGX_2Jk2uo_fcjojApIBw>
    <xmx:QCI6Zr0wjptRiUkcYXDjFcBMISn09tNBtwvsk4pX49peYUKNcW84wV-D>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH v7 0/6] MSI-X support with qemu in stubdomain, and other related changes
Date: Tue,  7 May 2024 14:44:00 +0200
Message-ID: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This series includes changes to make MSI-X working with Linux stubdomain and
especially Intel Wifi 6 AX210 card. This takes care of remaining reasons for
QEMU to access /dev/mem, but also the Intel Wifi card violating spec by putting
some registers on the same page as the MSI-X table.
Besides the stubdomain case (of which I care more), this is also necessary for
PCI-passthrough to work with lockdown enabled in dom0 (when QEMU runs in dom0).

See individual patches for details.

This series include also tests for MSI-X using new approach (by preventing QEMU
access to /dev/mem). But for it to work, it needs QEMU change that
makes use of the changes introduced here. It can be seen at
https://github.com/marmarek/qemu/commits/msix

Here is the pipeline that used the QEMU fork above:
https://gitlab.com/xen-project/people/marmarek/xen/-/pipelines/1280152273
(the build failures are due to issues with fetching or building newer QEMU
 discussed on Matrix)

v7:
 - "x86/msi: passthrough all MSI-X vector ctrl writes to device model" is already applied

Marek Marczykowski-Górecki (6):
  x86/msi: Extend per-domain/device warning mechanism
  x86/hvm: Allow access to registers on the same page as MSI-X table
  automation: prevent QEMU access to /dev/mem in PCI passthrough tests
  automation: switch to a wifi card on ADL system
  [DO NOT APPLY] switch to qemu fork
  [DO NOT APPLY] switch to alternative artifact repo

 Config.mk                                           |   4 +-
 automation/gitlab-ci/build.yaml                     |   4 +-
 automation/gitlab-ci/test.yaml                      |   4 +-
 automation/scripts/qubes-x86-64.sh                  |   9 +-
 automation/tests-artifacts/alpine/3.18.dockerfile   |   7 +-
 automation/tests-artifacts/kernel/6.1.19.dockerfile |   2 +-
 xen/arch/x86/hvm/vmsi.c                             | 205 ++++++++++++-
 xen/arch/x86/include/asm/msi.h                      |  22 +-
 xen/arch/x86/msi.c                                  |  47 ++-
 9 files changed, 285 insertions(+), 19 deletions(-)

base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718066.1120593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBo-0003qb-7K; Tue, 07 May 2024 12:45:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718066.1120593; Tue, 07 May 2024 12:45:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBo-0003qS-47; Tue, 07 May 2024 12:45:00 +0000
Received: by outflank-mailman (input) for mailman id 718066;
 Tue, 07 May 2024 12:44:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBm-00037i-CO
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:44:58 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9b1d384d-0c6f-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 14:44:56 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id CE04A1C000B3;
 Tue,  7 May 2024 08:44:54 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Tue, 07 May 2024 08:44:55 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:53 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b1d384d-0c6f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715085894; x=1715172294; bh=hZ6VYL08E0
	Yxr0n/W2twXlHVhmqK+fKxAH0QY/uH+Wo=; b=E7AzwpSCa7RJFAipYDZ3jCQw8U
	jjxqm4GQB2I/bfs42ihOCBTLat9qrYlxXOMjR8bQlWdnKseCAhsSadphICOrCrdN
	BjHUbzEMXOzMlpymCmT/n6Lwc3+Q90JK8n8zBqcMgJKDdLv/haysg/er7n+Go+Ya
	wLr/ZQVRHGFOoFa0E04VOEN3GLu003xNOq//1iZio/KXnk9jIkQ5oZ+xxSoWrpRm
	kXGgYPJaYHENFt1F+sVtEzhRDyJdmln5DZvNxOnFUG7FsBUBHZda7gS/99nFB2yJ
	wRiIl7Im7o3GKVVMMNcRwI/00ZZa30h8Agf/P49lSmV+JbKVqAIRw695vDhA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085894; x=
	1715172294; bh=hZ6VYL08E0Yxr0n/W2twXlHVhmqK+fKxAH0QY/uH+Wo=; b=R
	6Kgq3hm3cCwJhJ/y8diYXiEPrwDuROJuZo0xnozeKI5rqLeIS9tDH7bsgWvv3fIW
	8cVSDVAD/d57rUYl24DXXaS19oOHirqhSPdIy9nTvoQwT3oaIxh9a5NVLyWvmvl7
	OFHWlO2j4EarZasWgk20o6E3s1aBmuIzz6vS70jeFBOkWH9DEKeZmIW6BKhilXv8
	DOtjU/wb3et5DstvmN6uavRFvu6An3DIyk47MrCSoSAP3J9KfvZLSBb5IInzJGK3
	09lHBlfSnjNZqSVggqiwdgSC8zaSB56eRghNdKxdCVLDFAT+K2LYRw7FGx6wDLRo
	mO7dLDT/QSp9QuzIY0YsQ==
X-ME-Sender: <xms:RiI6ZvFk4IHbNcRDXRa6C6kDZ4Ds_8yuPoJegUEk5VsxhUQos6AFnw>
    <xme:RiI6ZsWcXTuynUD8jxqSVZap9zX-KPM30j4mv94JlpoowMafwh6Ffw4zkveCGKLXq
    IVBBlGK9eL64Q>
X-ME-Received: <xmr:RiI6ZhIWsbQusce-zgzxgrc29SlCquYFZ5vDm4LagfLWy3-WT6192UTHO-p5_BpH853oiYAAcYNI810mEzEtKhQmWi2mmlfpmxyLxnvVPxQAqDUX5Qw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:RiI6ZtH9tUIWHwFUjhLJtXHQa3lApTfNYTyq5MxHlnuN7yMW9sSx6Q>
    <xmx:RiI6ZlWcdmBRSeuV5_6RAty83r1TJgy2Bfxhj4lSgOVlTe0lKJCIyQ>
    <xmx:RiI6ZoOKN_jcBfDrlZg8ba_i0sgqlisdBHKcvmUWhrlV7nlPWjM87Q>
    <xmx:RiI6Zk09eVNfedsrKu7ENldn3xQt8RnNxH2_-prJU1Eqmfb8NBmdAw>
    <xmx:RiI6ZoSdx8UiGg8Mv0SzBmankFhc6clJH7lBWq5DwPbOghMUIdGr4paf>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH v7 3/6] automation: prevent QEMU access to /dev/mem in PCI passthrough tests
Date: Tue,  7 May 2024 14:44:03 +0200
Message-ID: <2b6ed802c83f7e6d7d03437b8ecc693628f85651.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

/dev/mem access doesn't work in dom0 in lockdown and in stubdomain.
Simulate this environment with removing /dev/mem device node. Full test
for lockdown and stubdomain will come later, when all requirements will
be in place.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
This can be applied only after QEMU change is committed. Otherwise the
test will fail.
---
 automation/scripts/qubes-x86-64.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index d81ed7b931cf..7eabc1bd6ad4 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -163,6 +163,8 @@ ifconfig eth0 up
 ifconfig xenbr0 up
 ifconfig xenbr0 192.168.0.1
 
+# ensure QEMU wont have access /dev/mem
+rm -f /dev/mem
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
 xl create /etc/xen/domU.cfg
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718064.1120573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBk-0003MB-LD; Tue, 07 May 2024 12:44:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718064.1120573; Tue, 07 May 2024 12:44:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBk-0003M4-I2; Tue, 07 May 2024 12:44:56 +0000
Received: by outflank-mailman (input) for mailman id 718064;
 Tue, 07 May 2024 12:44:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBj-0003AI-SJ
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:44:55 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 988f90d1-0c6f-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:44:53 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id 7B30A1C001C3;
 Tue,  7 May 2024 08:44:50 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Tue, 07 May 2024 08:44:50 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:48 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 988f90d1-0c6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715085890; x=1715172290; bh=lLrdK6DtjD
	VUONkQTnGTL9QCNQzxUIg1GxNz3rQOZI8=; b=jlsdqPycvSAtttPd9k2N0Vf9MT
	tHInZyY+y8waJLqLJj2P2qjZ2AR+DKc/CGRh0sK5lBv7blTm4meaY75ktmcrsH5J
	g60hpl+ONHYI/PArkU5M9Bobhvi+hnjdk8HZD8qwR8uhnyQfyQAWZPZRCMk0mDdy
	H92K3moJcV6pEqikxnWzreihIOkjpKyit1ngF/pHl1UYoX4bxyU2xIIfuEuah3rj
	nFVV4bCbfR9aaW2QSOk7gpQttgnHPM9Cpap6fPYcOy46ZAurk7ZwDOmK3QUdjhFd
	Pda9fSgTc99UwIXUohJLZR8XtiVwyS1XIcQnNU1BVtuAHAH+Kqe5ZNCIUqEQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085890; x=
	1715172290; bh=lLrdK6DtjDVUONkQTnGTL9QCNQzxUIg1GxNz3rQOZI8=; b=B
	KpxfY/Wgp6m5U3JzKUOxzFk4TkNst2YgLI58YlZIQyRh1piAscLI+ZLhcgfLh+Eo
	UEnNVxh8+Aa2eTt1xpGLHFNpXQoKM5ZicKZAh5uBHefSr5aXhgVgKWpVSqNkgjX+
	xWPTbN/W3pjm8/XHGfYuu22dG9AVLOzfSV0OY6/1GFZMiC0xINJSn8SGbV75MLZJ
	fpzZpL5YUWVUv3ysiUjrUFOGrJaPXhNNnFeZhAl0zLoKAqvoNXY+R1R9uP8FeiyF
	ix41szVICf3bTz/7ilJ4zNwgE/beiX0rpSkinB35zbOIHRPL+W6rczGNIfrAyQUb
	pYKWyDjxaldRpxPs2Fo1Q==
X-ME-Sender: <xms:QSI6Zrl5bta65n9BYZFfnc3plhAdkB7qyvlQHGIGc3CZ8ZOneJrs0A>
    <xme:QSI6Zu0l6HrL6DMW8Z45SxilizYz2V7MAwlfSkakSbKK5smdPb54CQRP5eaO-WLMk
    B9GQdv23bFdQQ>
X-ME-Received: <xmr:QSI6Zhpsx1qVGEiEz7kpzU8Djzy8Lsf3CpUR3gACAPmGBjGxoCgIPaUvunlK-caY1aHFFjWLyv5QgCjdMHHgZiG_aNlf8BfAa4-aa4qwhKyq_l1OPn4>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:QSI6ZjnP40qUKd4Z-XLB0DdrVW5AIzH-b0YZb-7-OK0a_oWg7Kra1g>
    <xmx:QSI6Zp01_EFaxsrcW55OjdzikrKFdo3zLwiBKxpYC_zkkqH0QiIibw>
    <xmx:QSI6Zit-MMSZKwj_JhyzEJ5dPNizBiXVXyY-jBCRiGbH4rIbqMjEBQ>
    <xmx:QSI6ZtXrQSZDtL12vC6ohMifrP66wHgnz2Ia85qLIm_GEODh4aGWew>
    <xmx:QiI6ZkRAs5VXOv_jKFLCNsD9pNSZZqNtRA5tpum0T5bIFAKh4GLXzCAv>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v7 1/6] x86/msi: Extend per-domain/device warning mechanism
Date: Tue,  7 May 2024 14:44:01 +0200
Message-ID: <0ff02aa7d6f78106a69158794d51c0b32a75431d.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The arch_msix struct had a single "warned" field with a domid for which
warning was issued. Upcoming patch will need similar mechanism for few
more warnings, so change it to save a bit field of issued warnings.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v6:
- add MSIX_CHECK_WARN macro (Jan)
- drop struct name from warned_kind union (Jan)

New in v5
---
 xen/arch/x86/include/asm/msi.h | 17 ++++++++++++++++-
 xen/arch/x86/msi.c             |  5 +----
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
index 997ccb87be0c..bcfdfd35345d 100644
--- a/xen/arch/x86/include/asm/msi.h
+++ b/xen/arch/x86/include/asm/msi.h
@@ -208,6 +208,15 @@ struct msg_address {
                                        PCI_MSIX_ENTRY_SIZE + \
                                        (~PCI_MSIX_BIRMASK & (PAGE_SIZE - 1)))
 
+#define MSIX_CHECK_WARN(msix, domid, which) ({ \
+    if ( (msix)->warned_domid != (domid) ) \
+    { \
+        (msix)->warned_domid = (domid); \
+        (msix)->warned_kind.all = 0; \
+    } \
+    (msix)->warned_kind.which ? false : ((msix)->warned_kind.which = true); \
+})
+
 struct arch_msix {
     unsigned int nr_entries, used_entries;
     struct {
@@ -217,7 +226,13 @@ struct arch_msix {
     int table_idx[MAX_MSIX_TABLE_PAGES];
     spinlock_t table_lock;
     bool host_maskall, guest_maskall;
-    domid_t warned;
+    domid_t warned_domid;
+    union {
+        uint8_t all;
+        struct {
+            bool maskall                   : 1;
+        };
+    } warned_kind;
 };
 
 void early_msi_init(void);
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index e721aaf5c001..42c793426da3 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -364,13 +364,10 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)
             domid_t domid = pdev->domain->domain_id;
 
             maskall = true;
-            if ( pdev->msix->warned != domid )
-            {
-                pdev->msix->warned = domid;
+            if ( MSIX_CHECK_WARN(pdev->msix, domid, maskall) )
                 printk(XENLOG_G_WARNING
                        "cannot mask IRQ %d: masking MSI-X on Dom%d's %pp\n",
                        desc->irq, domid, &pdev->sbdf);
-            }
         }
         pdev->msix->host_maskall = maskall;
         if ( maskall || pdev->msix->guest_maskall )
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718067.1120603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBp-00046P-F5; Tue, 07 May 2024 12:45:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718067.1120603; Tue, 07 May 2024 12:45:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBp-00046I-BG; Tue, 07 May 2024 12:45:01 +0000
Received: by outflank-mailman (input) for mailman id 718067;
 Tue, 07 May 2024 12:44:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBn-0003AI-FO
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:44:59 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9c561c91-0c6f-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:44:58 +0200 (CEST)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailfout.west.internal (Postfix) with ESMTP id D40251C001A6;
 Tue,  7 May 2024 08:44:56 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute4.internal (MEProxy); Tue, 07 May 2024 08:44:57 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:55 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c561c91-0c6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715085896; x=1715172296; bh=Uac2qx6Vbh
	R5d6LSLBww4OHHHB4xRISCtst2qrKvDik=; b=NZ3nJJayXQFTqA+mUvCpiC9TVg
	gD7W0R0+PciJkLArYtoaAdQk84tR+KchXX9U+GziNDtiauWVtn6lHxk4CE3eQn78
	95bQcuwfcwm+cesruC+s/1LWd+o1eIXrxEro2oIVKQ2zp/MgUY7t+PbbNB0UBlPi
	jcBGEFZvvD99VbQJsv+Ppjg1R64lDguVWXjMhzLZ3SfzbO31h3CiuFRkmDGZV3QL
	xpcQmdj8+RHIMeUFX60QAycx7Y2ZQ5hKIpN+JNkjcUmJZ5Rc4q+Gs5pwU99yE8Dk
	GqB1LaiOR05dof6bcCUBnUuXbvp8dKlhi2nQjifDESP5dPckcGAYTiZ+cYtw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085896; x=
	1715172296; bh=Uac2qx6VbhR5d6LSLBww4OHHHB4xRISCtst2qrKvDik=; b=M
	kpBVCHFyUPYgsWeu2RSfdoAbC6hMyrHx7HznkSDgdAMsmogGBwFSEIEj0/jmJYES
	spMCgyVn4xLtyMFffEhtu/tyWWJXA289ZQvz66oIgSBqXRC2758sXXq+G5uYlLvR
	VTKLniHznDfluiN5JipvUtdbXw7TncZJtjaCQjbVphK1d0fg2oX64pBrfqV35MAH
	hjFJOEtwawijNqlH3+t3FC9+x5DsRlvyOtFrBlZXuRkxY31wVBCcJ8+pjv6RAaXr
	DF3lvrQcTBz99iUWpkS/0QjlKIga/t1lY93JZjM+KWSbb/9rBuPYaraLRZoSMD1+
	AVfk2RlXGaWRqVwEn5uDw==
X-ME-Sender: <xms:SCI6ZkcrioHD4UUikHK1ABBzxLW_nZrhxumPUam0MMVBYWZQYyvEiw>
    <xme:SCI6ZmNTIAo1F1D-4Us48n5Jwk5eR7Fzog5akkaVrM3f61YZLUX8VGaS9-vcrPHEA
    uqIHK2d0-w8hQ>
X-ME-Received: <xmr:SCI6ZljLUguujsDD8LKeEiK89RC2dxvByyr-1qpvvpDNiZXY2SOgnu7u50Kojid37RaYTB-KCGHtcH9GtmXtRLsrXOy2xg6dRkQ6A4S9_vgAgvW-Gso>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeff
    iedtgeevffetledvgefhhfevgffhfeekleehueejjeegvddvgfffjeeutddvleenucffoh
    hmrghinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr
    rghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhngh
    hslhgrsgdrtghomh
X-ME-Proxy: <xmx:SCI6Zp8Q9T6_t4ByT_Uk1qu-7fdF_8LiPMYkm7kJzMFF8i_4bpqxXQ>
    <xmx:SCI6ZguyHmzeuqQtHYTid8kveP95-KYpEMF2XP8WF3REVah09bm-Rw>
    <xmx:SCI6ZgFNEspnXz_uLMNUZdVNikEvpSo8wwhH1hUSqrg1JOCqYLoJFw>
    <xmx:SCI6ZvNAg8iMMdXgMqcOaZDpImDM_PG7Hy5v6vQJoSLG4L_2GM9igg>
    <xmx:SCI6ZsLx_A8nMaAy_PIdAERGpop0Pj7otlC_nlFEiKI_Otk_JCgSLfIN>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH v7 4/6] automation: switch to a wifi card on ADL system
Date: Tue,  7 May 2024 14:44:04 +0200
Message-ID: <666976d22bb50574b5c2a91d8b41b48de944b38a.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Switch to a wifi card that has registers on a MSI-X page. This tests the
"x86/hvm: Allow writes to registers on the same page as MSI-X table"
feature. Switch it only for HVM test, because MSI-X adjacent write is
not supported on PV.

This requires also including drivers and firmware in system for tests.
Remove firmware unrelated to the test, to not increase initrd size too
much (all firmware takes over 100MB compressed).
And finally adjusts test script to handle not only eth0 as a test device,
but also wlan0 and connect it to the wifi network.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
This needs two new gitlab variables: WIFI_HW2_SSID and WIFI_HW2_PSK. I'll
provide them in private.

This change requires rebuilding test containers.

This can be applied only after QEMU change is committed. Otherwise the
test will fail.
---
 automation/gitlab-ci/test.yaml                      | 4 ++++
 automation/scripts/qubes-x86-64.sh                  | 7 +++++++
 automation/tests-artifacts/alpine/3.18.dockerfile   | 7 +++++++
 automation/tests-artifacts/kernel/6.1.19.dockerfile | 2 ++
 4 files changed, 20 insertions(+)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index ad249fa0a5d9..6803cae116b5 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -193,6 +193,10 @@ adl-pci-pv-x86-64-gcc-debug:
 
 adl-pci-hvm-x86-64-gcc-debug:
   extends: .adl-x86-64
+  variables:
+    PCIDEV: "00:14.3"
+    WIFI_SSID: "$WIFI_HW2_SSID"
+    WIFI_PSK: "$WIFI_HW2_PSK"
   script:
     - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE}
   needs:
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 7eabc1bd6ad4..60498ef1e89a 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -94,6 +94,13 @@ on_reboot = "destroy"
     domU_check="
 set -x -e
 interface=eth0
+if [ -e /sys/class/net/wlan0 ]; then
+    interface=wlan0
+    set +x
+    wpa_passphrase "$WIFI_SSID" "$WIFI_PSK" > /etc/wpa_supplicant.conf
+    set -x
+    wpa_supplicant -B -iwlan0 -c /etc/wpa_supplicant.conf
+fi
 ip link set \"\$interface\" up
 timeout 30s udhcpc -i \"\$interface\"
 pingip=\$(ip -o -4 r show default|cut -f 3 -d ' ')
diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile
index 9cde6c9ad4da..c323e266c7da 100644
--- a/automation/tests-artifacts/alpine/3.18.dockerfile
+++ b/automation/tests-artifacts/alpine/3.18.dockerfile
@@ -34,6 +34,13 @@ RUN \
   apk add udev && \
   apk add pciutils && \
   apk add libelf && \
+  apk add wpa_supplicant && \
+  # Select firmware for hardware tests
+  apk add linux-firmware-other && \
+  mkdir /lib/firmware-preserve && \
+  mv /lib/firmware/iwlwifi-so-a0-gf-a0* /lib/firmware-preserve/ && \
+  rm -rf /lib/firmware && \
+  mv /lib/firmware-preserve /lib/firmware && \
   \
   # Xen
   cd / && \
diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile
index 3a4096780d20..84ed5dff23ae 100644
--- a/automation/tests-artifacts/kernel/6.1.19.dockerfile
+++ b/automation/tests-artifacts/kernel/6.1.19.dockerfile
@@ -32,6 +32,8 @@ RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSI
     make xen.config && \
     scripts/config --enable BRIDGE && \
     scripts/config --enable IGC && \
+    scripts/config --enable IWLWIFI && \
+    scripts/config --enable IWLMVM && \
     cp .config .config.orig && \
     cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
     make -j$(nproc) bzImage && \
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718068.1120613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBq-0004NA-Tr; Tue, 07 May 2024 12:45:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718068.1120613; Tue, 07 May 2024 12:45:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBq-0004My-Qi; Tue, 07 May 2024 12:45:02 +0000
Received: by outflank-mailman (input) for mailman id 718068;
 Tue, 07 May 2024 12:45:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBp-0003AI-Rv
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:45:01 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9de48f33-0c6f-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:45:01 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.west.internal (Postfix) with ESMTP id 4D6C21C000B3;
 Tue,  7 May 2024 08:44:59 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Tue, 07 May 2024 08:44:59 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:57 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9de48f33-0c6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to; s=fm3; t=1715085898; x=1715172298; bh=EAaNxhwX2Xjwv3rkR/3PC
	xXHxt3VIaaNGAHnw+G6qes=; b=JY8d7J7lJXtyd2BPAJXkW+MmZsVnTi+/ACzIy
	Y+NMlHs31TO+025RNoxuN0vb8CkF1TJjeL6TVFwyaMoCrN0TVjPQ5lJUxtVqeVLM
	z4oGM172hEU976I8o1dY98DrDAWV6ZBlmov9sBTfQCAfOPLbTf7Jcf7rxOH8YXAU
	B7SpB2FOjJGSz4skgCuY2tF0URzUPazoV3/jFdh8CbZzsrhAj8qzoIISUjebTEgv
	hszOqraBrt4Mn28SHS5Wi+PiPg2C++JPWXjRmyv7oiLUdkRjca0AgWlaOCHr9U3y
	LYf2ZOVeDMOnT79W/ZOBaUeNsOFNZLToUUD7taTi6k2mI9KLQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:feedback-id:feedback-id:from:from
	:in-reply-to:in-reply-to:message-id:mime-version:references
	:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085898; x=
	1715172298; bh=EAaNxhwX2Xjwv3rkR/3PCxXHxt3VIaaNGAHnw+G6qes=; b=V
	GcLr9SqyAcXuKbD8BZAVKASwA6aTYPiqZy7+SFs9t9YM2m6pUbD7sUvy/U7kgMv+
	MuSlnNZoZwVDjOqXSVFIrbUKHg7X84Zt1okrAOrfnK9RjFiBBOD1Gw9vCfyBRJhg
	JNSaS74KzzMmao06hK+JzeMHe1iOPFdbqslmsH0NHhzBjBktvrNa3HIu3h4i3TmC
	0ZLnIXs8LpYZa6UBvZUd8qKHj4D6FDb1WdwAQ5MGQatyPXMa3Z0lxoPcooutO91O
	pz1G/uLLvTYF/rs2szULqU9vrrnL+UkFuDSUOuXBiUvbHEai3upNKDUYbNQD2o/l
	l6kD4ZN8cHwlqUokLLvSg==
X-ME-Sender: <xms:SiI6ZgwFxo9ICcdvk7FEN-f1vmTr6e-YCGmLKoysZQFRvY3wKZE_BA>
    <xme:SiI6ZkQK33bpi_qzihvVBUBDBV9DhGJVUSjnCmADF_zKd1CavQUperJc8HE48gCMQ
    OnqymRWdxNzjA>
X-ME-Received: <xmr:SiI6ZiXPQGbF2kcWjYtSobFA0wST2WouReY4PgR6oZVL2uKN0R-gyhoIEcClmMNSyILfHHBot9wEwYwSODkhThZVGZRdk7uSyHMgeL-dnpCoxfiD4z0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgeet
    tdeigefhkefhfeegvdeuiedvtdffkeeltdeivdfgteelueeuhfetleevueegnecuffhomh
    grihhnpeigvghnrdhorhhgpdhgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigv
    pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg
    hlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:SiI6ZujcshQqAw0wo47mPzVqONpbi4RuTBf6gh8QzqB6Vywa_GRupg>
    <xmx:SiI6ZiBDxJQ2yZbjTGnWfezsmi_DSYgG-kV0TC6A3G6S3y76cF3v8Q>
    <xmx:SiI6ZvLwRc7LTwCVXxytYsDjsp6UruLsywalI2klCdMQZ6rlKo8Rqg>
    <xmx:SiI6ZpCU9hdzHVDgO7dvUt4UO9U1DhJF5v9YDYNLzjrpJmZGH1RgCg>
    <xmx:SiI6Zk3FLoLf_L5ghq4e3oNkqAUrKr66pau44MqQFvrN4cSmeBGRFj7u>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v7 5/6] [DO NOT APPLY] switch to qemu fork
Date: Tue,  7 May 2024 14:44:05 +0200
Message-ID: <711d3d849a80657ae730daa16656770a2758862a.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This makes tests to use patched QEMU, to actually test the new behavior.
---
 Config.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Config.mk b/Config.mk
index a962f095ca16..5e220a1284e4 100644
--- a/Config.mk
+++ b/Config.mk
@@ -220,8 +220,8 @@ endif
 OVMF_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/ovmf.git
 OVMF_UPSTREAM_REVISION ?= ba91d0292e593df8528b66f99c1b0b14fadc8e16
 
-QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
-QEMU_UPSTREAM_REVISION ?= master
+QEMU_UPSTREAM_URL ?= https://github.com/marmarek/qemu
+QEMU_UPSTREAM_REVISION ?= origin/msix
 
 MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
 MINIOS_UPSTREAM_REVISION ?= b6a5b4d72b88e5c4faed01f5a44505de022860fc
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718065.1120583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBl-0003ak-Ui; Tue, 07 May 2024 12:44:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718065.1120583; Tue, 07 May 2024 12:44:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBl-0003ad-Ri; Tue, 07 May 2024 12:44:57 +0000
Received: by outflank-mailman (input) for mailman id 718065;
 Tue, 07 May 2024 12:44:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBk-0003AI-Ho
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:44:56 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 99e67496-0c6f-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:44:54 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id B7FAC1C001C8;
 Tue,  7 May 2024 08:44:52 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Tue, 07 May 2024 08:44:53 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:44:51 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99e67496-0c6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715085892; x=1715172292; bh=337mWtE1IZ
	WiZWyGhYmWNijg7Hos8zgGc/g42z5K3X8=; b=0vGjV7bFutsPrY8oS12ro60kcT
	rknupAk1K2MFRZKI7pWNww4xNELjAlNjseUaJ783dpXzbItUZfUQg0i+f2RfPsY2
	MxD4FHzgwDLI8gq43E5dwcQkULixLagXQfzHs0YYYeOn1uWHqpbNZjoE/GHOGVLP
	4IO9UKfl564uLinrOss5VciER2n5qiVnFWfd4dKEI3GABlR8cIpCm4gTlIFtrF4d
	kM/EPw8Q6UAmUkz6KhCdLpHxq8FNtq7m2S1T76lXExaYND7AC0oo7xPaLZISbsvp
	/ZH4fIYgpL3qh3RlI5mlZpjiWaz9ZGZBwjk8Fm8PFDKbJNrVHkZlAcSHuE/Q==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085892; x=
	1715172292; bh=337mWtE1IZWiZWyGhYmWNijg7Hos8zgGc/g42z5K3X8=; b=F
	T9eEdd2bJ3P3j9pPb3jWSDIYH6OD0DVM6qX1+my1RRcfmJYWr8cncCWBILp483Nk
	5FSgka4+GBh8Z3ktZTGM/fiUcW+RNWlV3NWbF8RPlOaJMQoYh6wV9AFDg3reE2Zc
	IG32umu/lLz9TFkt012DkHTP71G9IMN5d6lHqBRe7Cd/Jc7fDxBcAEjPY2TDI7SO
	42JVNOsJUEp6YLx5ySNVia1bt4whvRnLV22GL+PigNaAIOQ9txreRdhFB6T6lrqy
	29tVt2GPFKW0ipGHnhJfvbZxvRAuy/elswLG+rvcx/V+neqqNpgz2ugixuRI2qoE
	MZLDlGd/cllpaCa+yZA2Q==
X-ME-Sender: <xms:RCI6ZmYRbHaVfIspCXLCkOo1aFAaKSGqgHs0yXKb1dl01LgTX2dgLw>
    <xme:RCI6ZpYoo0mAaQrDgc6k2iUZoM4LN8ASYlJKL-stH7LtJUxbYnzsGby5uDyy7F9Ix
    pR9EXmUaBG1hA>
X-ME-Received: <xmr:RCI6Zg9kl2bvqv2y5Byw9N6LQjHEAXMy7P1gub8EfCiXAOftCO_AXAcGRFAilJ5i0goBQm-QU8XFdL6bNd3kpSHrGrmmRc6eqPiJL_wnSiJJVBo5IIs>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefh
    iefhgfehgfekteeigfdtiedvgfevhffhfedvudeltedvgfegtdekteeffeegheenucffoh
    hmrghinhepmhhsihigthgslhgplhhishhtrdhnvgigthenucevlhhushhtvghrufhiiigv
    pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg
    hlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:RCI6ZoqAEjchSYLpTeYR8szakiYZcH0mOFWB0O6sU_jRkSgFrb_SSA>
    <xmx:RCI6Zhp72HkswoEfh6k3y3RAhs2DB0fFAydf6Fg3ixnHCXbHr3mF9Q>
    <xmx:RCI6ZmRipOjQ_9M8hACUrn6zOZ2onnuPoah5OaaqXnhysTkuzcmQ8A>
    <xmx:RCI6ZhpOMZ28LTYpOIC4e1mjbnYgOf3ejZneyyJuK_zkJCtVAc7onw>
    <xmx:RCI6ZinGkKcR71aedwPyw0B80pjPTNbEqOrpW-fDnkkYQXsGcTq8MO8B>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v7 2/6] x86/hvm: Allow access to registers on the same page as MSI-X table
Date: Tue,  7 May 2024 14:44:02 +0200
Message-ID: <da180c8419882dc068512b69c6c7ad3ff3343fe0.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
on the same page as MSI-X table. Device model (especially one in
stubdomain) cannot really handle those, as direct writes to that page is
refused (page is on the mmio_ro_ranges list). Instead, extend
msixtbl_mmio_ops to handle such accesses too.

Doing this, requires correlating read/write location with guest
MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
for PV would need to be done separately.

This will be also used to read Pending Bit Array, if it lives on the same
page, making QEMU not needing /dev/mem access at all (especially helpful
with lockdown enabled in dom0). If PBA lives on another page, QEMU will
map it to the guest directly.
If PBA lives on the same page, discard writes and log a message.
Technically, writes outside of PBA could be allowed, but at this moment
the precise location of PBA isn't saved, and also no known device abuses
the spec in this way (at least yet).

To access those registers, msixtbl_mmio_ops need the relevant page
mapped. MSI handling already has infrastructure for that, using fixmap,
so try to map first/last page of the MSI-X table (if necessary) and save
their fixmap indexes. Note that msix_get_fixmap() does reference
counting and reuses existing mapping, so just call it directly, even if
the page was mapped before. Also, it uses a specific range of fixmap
indexes which doesn't include 0, so use 0 as default ("not mapped")
value - which simplifies code a bit.

Based on assumption that all MSI-X page accesses are handled by Xen, do
not forward adjacent accesses to other hypothetical ioreq servers, even
if the access wasn't handled for some reason (failure to map pages etc).
Relevant places log a message about that already.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v7:
- simplify logic based on assumption that all access to MSI-X pages are
  handled by Xen (Roger)
- move calling adjacent_handle() into adjacent_{read,write}() (Roger)
- move range check into msixtbl_addr_to_desc() (Roger)
- fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
- no longer distinguish between unhandled write due to PBA nearby and
  other reasons
- add missing break after ASSERT_UNREACHABLE (Jan)
Changes in v6:
- use MSIX_CHECK_WARN macro
- extend assert on fixmap_idx
- add break in default label, after ASSERT_UNREACHABLE(), and move
  setting default there
- style fixes
Changes in v5:
- style fixes
- include GCC version in the commit message
- warn only once (per domain, per device) about failed adjacent access
Changes in v4:
- drop same_page parameter of msixtbl_find_entry(), distinguish two
  cases in relevant callers
- rename adj_access_table_idx to adj_access_idx
- code style fixes
- drop alignment check in adjacent_{read,write}() - all callers already
  have it earlier
- delay mapping first/last MSI-X pages until preparing device for a
  passthrough
v3:
 - merge handling into msixtbl_mmio_ops
 - extend commit message
v2:
 - adjust commit message
 - pass struct domain to msixtbl_page_handler_get_hwaddr()
 - reduce local variables used only once
 - log a warning if write is forbidden if MSI-X and PBA lives on the same
   page
 - do not passthrough unaligned accesses
 - handle accesses both before and after MSI-X table
---
 xen/arch/x86/hvm/vmsi.c        | 205 ++++++++++++++++++++++++++++++++--
 xen/arch/x86/include/asm/msi.h |   5 +-
 xen/arch/x86/msi.c             |  42 +++++++-
 3 files changed, 242 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 999917983789..f7b7b4998b5e 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
     return d->arch.hvm.msixtbl_list.next;
 }
 
+/*
+ * Lookup an msixtbl_entry on the same page as given addr. It's up to the
+ * caller to check if address is strictly part of the table - if relevant.
+ */
 static struct msixtbl_entry *msixtbl_find_entry(
     struct vcpu *v, unsigned long addr)
 {
@@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
     struct domain *d = v->domain;
 
     list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
-        if ( addr >= entry->gtable &&
-             addr < entry->gtable + entry->table_len )
+        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
+             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
             return entry;
 
     return NULL;
@@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
     if ( !entry || !entry->pdev )
         return NULL;
 
+    if ( addr < entry->gtable ||
+         addr >= entry->gtable + entry->table_len )
+        return NULL;
+
     nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
 
     list_for_each_entry( desc, &entry->pdev->msi_list, list )
@@ -213,6 +221,152 @@ static struct msi_desc *msixtbl_addr_to_desc(
     return NULL;
 }
 
+/*
+ * Returns:
+ *  - ADJACENT_DONT_HANDLE if no handling should be done
+ *  - a fixmap idx to use for handling
+ */
+#define ADJACENT_DONT_HANDLE UINT_MAX
+static unsigned int adjacent_handle(
+    const struct msixtbl_entry *entry, unsigned long addr, bool write)
+{
+    unsigned int adj_type;
+    struct arch_msix *msix;
+
+    if ( !entry || !entry->pdev )
+    {
+        ASSERT_UNREACHABLE();
+        return ADJACENT_DONT_HANDLE;
+    }
+
+    if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable )
+        adj_type = ADJ_IDX_FIRST;
+    else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) &&
+              addr >= entry->gtable + entry->table_len )
+        adj_type = ADJ_IDX_LAST;
+    else
+    {
+        /* All callers should already do equivalent range checking. */
+        ASSERT_UNREACHABLE();
+        return ADJACENT_DONT_HANDLE;
+    }
+
+    msix = entry->pdev->msix;
+    ASSERT(msix);
+
+    if ( !msix->adj_access_idx[adj_type] )
+    {
+        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
+                             adjacent_not_initialized) )
+            gprintk(XENLOG_WARNING,
+                    "Page for adjacent(%d) MSI-X table access not initialized for %pp (addr %#lx, gtable %#lx\n",
+                    adj_type, &entry->pdev->sbdf, addr, entry->gtable);
+        return ADJACENT_DONT_HANDLE;
+    }
+
+    /* If PBA lives on the same page too, discard writes. */
+    if ( write &&
+         ((adj_type == ADJ_IDX_LAST &&
+           msix->table.last == msix->pba.first) ||
+          (adj_type == ADJ_IDX_FIRST &&
+           msix->table.first == msix->pba.last)) )
+    {
+        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
+                             adjacent_pba) )
+            gprintk(XENLOG_WARNING,
+                    "MSI-X table and PBA of %pp live on the same page, "
+                    "writing to other registers there is not implemented\n",
+                    &entry->pdev->sbdf);
+        return ADJACENT_DONT_HANDLE;
+    }
+
+    return msix->adj_access_idx[adj_type];
+}
+
+static int adjacent_read(
+    const struct msixtbl_entry *entry,
+    paddr_t address, unsigned int len, uint64_t *pval)
+{
+    const void __iomem *hwaddr;
+    unsigned int fixmap_idx;
+
+    fixmap_idx = adjacent_handle(entry, address, false);
+
+    if ( fixmap_idx == ADJACENT_DONT_HANDLE )
+    {
+        *pval = ~0UL;
+        return X86EMUL_OKAY;
+    }
+
+    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
+
+    switch ( len )
+    {
+    case 1:
+        *pval = readb(hwaddr);
+        break;
+
+    case 2:
+        *pval = readw(hwaddr);
+        break;
+
+    case 4:
+        *pval = readl(hwaddr);
+        break;
+
+    case 8:
+        *pval = readq(hwaddr);
+        break;
+
+    default:
+        ASSERT_UNREACHABLE();
+        *pval = ~0UL;
+        break;
+    }
+
+    return X86EMUL_OKAY;
+}
+
+static int adjacent_write(
+    const struct msixtbl_entry *entry,
+    paddr_t address, unsigned int len, uint64_t val)
+{
+    void __iomem *hwaddr;
+    unsigned int fixmap_idx;
+
+    fixmap_idx = adjacent_handle(entry, address, true);
+
+    if ( fixmap_idx == ADJACENT_DONT_HANDLE )
+        return X86EMUL_OKAY;
+
+    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
+
+    switch ( len )
+    {
+    case 1:
+        writeb(val, hwaddr);
+        break;
+
+    case 2:
+        writew(val, hwaddr);
+        break;
+
+    case 4:
+        writel(val, hwaddr);
+        break;
+
+    case 8:
+        writeq(val, hwaddr);
+        break;
+
+    default:
+        ASSERT_UNREACHABLE();
+        break;
+    }
+
+    return X86EMUL_OKAY;
+}
+
 static int cf_check msixtbl_read(
     const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
     uint64_t *pval)
@@ -222,7 +376,7 @@ static int cf_check msixtbl_read(
     unsigned int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
-    if ( (len != 4 && len != 8) || (address & (len - 1)) )
+    if ( !IS_ALIGNED(address, len) )
         return r;
 
     rcu_read_lock(&msixtbl_rcu_lock);
@@ -230,6 +384,17 @@ static int cf_check msixtbl_read(
     entry = msixtbl_find_entry(current, address);
     if ( !entry )
         goto out;
+
+    if ( address < entry->gtable ||
+         address >= entry->gtable + entry->table_len )
+    {
+        r = adjacent_read(entry, address, len, pval);
+        goto out;
+    }
+
+    if ( len != 4 && len != 8 )
+        goto out;
+
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
 
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
@@ -291,6 +456,17 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
     entry = msixtbl_find_entry(v, address);
     if ( !entry )
         goto out;
+
+    if ( address < entry->gtable ||
+         address >= entry->gtable + entry->table_len )
+    {
+        r = adjacent_write(entry, address, len, val);
+        goto out;
+    }
+
+    if ( len != 4 && len != 8 )
+        goto out;
+
     nr_entry = array_index_nospec(((address - entry->gtable) /
                                    PCI_MSIX_ENTRY_SIZE),
                                   MAX_MSIX_TABLE_ENTRIES);
@@ -356,8 +532,8 @@ static int cf_check _msixtbl_write(
     const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
     uint64_t val)
 {
-    /* Ignore invalid length or unaligned writes. */
-    if ( (len != 4 && len != 8) || !IS_ALIGNED(address, len) )
+    /* Ignore unaligned writes. */
+    if ( !IS_ALIGNED(address, len) )
         return X86EMUL_OKAY;
 
     /*
@@ -374,16 +550,25 @@ static bool cf_check msixtbl_range(
 {
     struct vcpu *curr = current;
     unsigned long addr = r->addr;
-    const struct msi_desc *desc;
+    const struct msixtbl_entry *entry;
+    bool ret = false;
 
     ASSERT(r->type == IOREQ_TYPE_COPY);
 
     rcu_read_lock(&msixtbl_rcu_lock);
-    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
+    entry = msixtbl_find_entry(curr, addr);
+    if ( entry )
+    {
+        if ( addr < entry->gtable || addr >= entry->gtable + entry->table_len )
+            /* Possibly handle adjacent access. */
+            ret = true;
+        else
+            ret = msixtbl_addr_to_desc(entry, addr) != NULL;
+    }
     rcu_read_unlock(&msixtbl_rcu_lock);
 
-    if ( desc )
-        return 1;
+    if ( ret )
+        return ret;
 
     if ( r->state == STATE_IOREQ_READY && r->dir == IOREQ_WRITE )
     {
@@ -429,7 +614,7 @@ static bool cf_check msixtbl_range(
         }
     }
 
-    return 0;
+    return false;
 }
 
 static const struct hvm_io_ops msixtbl_mmio_ops = {
diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
index bcfdfd35345d..923b730d48b8 100644
--- a/xen/arch/x86/include/asm/msi.h
+++ b/xen/arch/x86/include/asm/msi.h
@@ -224,6 +224,9 @@ struct arch_msix {
     } table, pba;
     int table_refcnt[MAX_MSIX_TABLE_PAGES];
     int table_idx[MAX_MSIX_TABLE_PAGES];
+#define ADJ_IDX_FIRST 0
+#define ADJ_IDX_LAST  1
+    unsigned int adj_access_idx[2];
     spinlock_t table_lock;
     bool host_maskall, guest_maskall;
     domid_t warned_domid;
@@ -231,6 +234,8 @@ struct arch_msix {
         uint8_t all;
         struct {
             bool maskall                   : 1;
+            bool adjacent_not_initialized  : 1;
+            bool adjacent_pba              : 1;
         };
     } warned_kind;
 };
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 42c793426da3..87190a88ed5d 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -913,6 +913,37 @@ static int msix_capability_init(struct pci_dev *dev,
         list_add_tail(&entry->list, &dev->msi_list);
         *desc = entry;
     }
+    else
+    {
+        /*
+         * If the MSI-X table doesn't start at the page boundary, map the first page for
+         * passthrough accesses.
+         */
+        if ( PAGE_OFFSET(table_paddr) )
+        {
+            int idx = msix_get_fixmap(msix, table_paddr, table_paddr);
+
+            if ( idx > 0 )
+                msix->adj_access_idx[ADJ_IDX_FIRST] = idx;
+            else
+                gprintk(XENLOG_ERR, "Failed to map first MSI-X table page: %d\n", idx);
+        }
+        /*
+         * If the MSI-X table doesn't end on the page boundary, map the last page
+         * for passthrough accesses.
+         */
+        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE) )
+        {
+            uint64_t entry_paddr = table_paddr +
+                (msix->nr_entries - 1) * PCI_MSIX_ENTRY_SIZE;
+            int idx = msix_get_fixmap(msix, table_paddr, entry_paddr);
+
+            if ( idx > 0 )
+                msix->adj_access_idx[ADJ_IDX_LAST] = idx;
+            else
+                gprintk(XENLOG_ERR, "Failed to map last MSI-X table page: %d\n", idx);
+        }
+    }
 
     if ( !msix->used_entries )
     {
@@ -1079,6 +1110,17 @@ static void _pci_cleanup_msix(struct arch_msix *msix)
         msix->table.first = 0;
         msix->table.last = 0;
 
+        if ( msix->adj_access_idx[ADJ_IDX_FIRST] )
+        {
+            msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_FIRST]);
+            msix->adj_access_idx[ADJ_IDX_FIRST] = 0;
+        }
+        if ( msix->adj_access_idx[ADJ_IDX_LAST] )
+        {
+            msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_LAST]);
+            msix->adj_access_idx[ADJ_IDX_LAST] = 0;
+        }
+
         if ( rangeset_remove_range(mmio_ro_ranges, msix->pba.first,
                                    msix->pba.last) )
             WARN();
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 12:45:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 12:45:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718069.1120623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBt-0004hF-6N; Tue, 07 May 2024 12:45:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718069.1120623; Tue, 07 May 2024 12:45:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KBt-0004h6-21; Tue, 07 May 2024 12:45:05 +0000
Received: by outflank-mailman (input) for mailman id 718069;
 Tue, 07 May 2024 12:45:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4KBs-0003AI-11
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 12:45:04 +0000
Received: from wfout4-smtp.messagingengine.com
 (wfout4-smtp.messagingengine.com [64.147.123.147])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9f18bff1-0c6f-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 14:45:03 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id 7BE8F1C001C6;
 Tue,  7 May 2024 08:45:01 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Tue, 07 May 2024 08:45:01 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 08:45:00 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f18bff1-0c6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to; s=fm3; t=1715085901; x=1715172301; bh=1Lf7PaioW5WU8mR8fRuXT
	r4i1fep7cAUzdBzJdvpuFY=; b=PdL5SGhdFetggvTuU790ekju/zGZL5Ukv44pe
	p4XqV5GinwU0oP3gGyJ9d2Fi2JIqF0mGyfd48FmLuHgaU5TXjh2UaABNN7sadMun
	oUqMilUGV+Ae12t2DLxvKSsnvC9d6ZgCeur0Y8yQn9CvxqWUTsg1fNrLepuujZIC
	TdjFCcXbQYc6MGp2Deq1yX8lbviDKmQgLaPj02sd6dKIG947bIJGC4toiNU3EECZ
	De5zWYxYhVjFyn51LiziErVZQXu1ecB2V6XghLZ8QKEWLFRRGHUy0681P1aRjwbQ
	cOtdbpZePbkrdkfOLl7HKdoAK3I8rwmkNexHa3sEQ14PBd6Uw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:feedback-id:feedback-id:from:from
	:in-reply-to:in-reply-to:message-id:mime-version:references
	:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715085901; x=
	1715172301; bh=1Lf7PaioW5WU8mR8fRuXTr4i1fep7cAUzdBzJdvpuFY=; b=R
	PJHGLXM/Nf7y2RXw118EP1uJfcAOAxrL7xGrTYnP1kTOvp+PixQRlED7JNaMNguy
	n3VTrRRnHIS9KyrMo8BnKMcsxtIBBuftXUzssjhLy4ZKBBFoUH5ChisMioGkpuRX
	JVHkw14NM84itBQB8xKghAPlSEgaqaYBVb/SCRJ5EHvXqleQ2mlJkluAn4VdddDL
	pm4sQOnzeiJSiqjN80pGyAo3ZlOsssXEZZR79RW9/HdjnBv1gwnJbSWqFsWwiMg4
	M+6XlRwXuU1EdL4s4NOtugtXV30Wm99MWUWCWgSITA7ViT+7a3LgZbATUJ550P72
	S32RnmcgqQ0k5QYTNnowQ==
X-ME-Sender: <xms:TCI6ZpcQ4edVLw_5-YE5kC-iMaLxXdb8MY8g8dzd3OxFmzB61L50wQ>
    <xme:TCI6ZnPW5XQvj_itJ4I1BXXLff77_Lk5ST0pT-0rtHsJjk9FhN9ihZTqYdhTt4l-e
    N0p8KOYTrJd4w>
X-ME-Received: <xmr:TCI6Zij566cHRS8fmg1O3N5O9FmvTwEzwAg7SxejzZm77babe2JL1EL52U2AA-kXnxu7eQtlRHSgE4BgeEPz8Hzsljv1wG4Ia2VOqhQU4wv5g7dIOBs>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgheefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueet
    hefgieehheeftdeuleduhffgieelueejhffggfeiieevtefhfeffffeftdehnecuffhomh
    grihhnpehgihhtlhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:TCI6Zi9yFicGlLpnoj1iDh1oVPrzcKuLpywwmzNKG7ojnFizlzz1Bw>
    <xmx:TCI6ZlucvBC_6FlopO6tsR6wf2HM4kvyONaDshImhBGeiT-iUnI-Gg>
    <xmx:TCI6ZhHxkiNQKPQ-ic1BcwIusBFyj-ZZk34ULtUFF5W65f7SeXbNHw>
    <xmx:TCI6ZsN7b5bmYPk6LfgDB3ojRGI-0HgjX2gCFDupTdAC2kzopD--Mw>
    <xmx:TSI6ZtIHyCVmZrNIssoSkwCPdSybAy9Pa3KiARzoBMRuoU73rGopsEvV>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v7 6/6] [DO NOT APPLY] switch to alternative artifact repo
Date: Tue,  7 May 2024 14:44:06 +0200
Message-ID: <5bb5167affa80458ce42be05b8d0057ce211667a.1715085837.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

For testing, switch to my containers registry that includes containers
rebuilt with changes in this series.
---
 automation/gitlab-ci/build.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5b4..0d7e311417d8 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -320,7 +320,7 @@ qemu-system-ppc64-8.1.0-ppc64-export:
 
 alpine-3.18-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/alpine:3.18
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
   artifacts:
@@ -331,7 +331,7 @@ alpine-3.18-rootfs-export:
 
 kernel-6.1.19-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/kernel:6.1.19
   script:
     - mkdir binaries && cp /bzImage binaries/bzImage
   artifacts:
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:09:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:09:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718094.1120632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KYy-0001qJ-0Z; Tue, 07 May 2024 13:08:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718094.1120632; Tue, 07 May 2024 13:08:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KYx-0001qC-Tj; Tue, 07 May 2024 13:08:55 +0000
Received: by outflank-mailman (input) for mailman id 718094;
 Tue, 07 May 2024 13:08:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4KYw-0001ow-Nx
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:08:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KYq-0006bB-MT; Tue, 07 May 2024 13:08:48 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KYq-0000VR-F9; Tue, 07 May 2024 13:08:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=s5G8FN5uL7yRNPiJ3fk4NTa+OOMfgP91t9ubDbT1ESc=; b=a/TT4LtV86K3vMf5loeQRpKREX
	/dIOWDOTaxK40ER0DFfIg34tWMUZmaNEWfxqA7Ul8NrYc4I2dnHKY0hkljOM0I00qRexwQbMOz/qD
	61BKFI/ZNbT/PtR8r0S5yyg7BOEoEXFLxSTs/6XLQ/Ol2ko0kXa187/gKO8+/S3Qe7Mw=;
Message-ID: <0eff541b-51c9-4764-b8c2-b686d5e602ce@xen.org>
Date: Tue, 7 May 2024 14:08:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
 <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
 <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 03/05/2024 20:07, Stefano Stabellini wrote:
> On Fri, 3 May 2024, Julien Grall wrote:

[...]

>> So are you saying that from Xen point of view, you are expecting no difference
>> between 256 and 512. And therefore you would be happy if to backport patches
>> if someone find differences (or even security issues) when using > 256 pCPUs?
> 
> It is difficult to be sure about anything that it is not regularly
> tested. I am pretty sure someone in the community got Xen running on an
> Ampere, so like you said 192 is a good number. However, that is not
> regularly tested, so we don't have any regression checks in gitlab-ci or
> OSSTest for it.
> 
> One approach would be to only support things regularly tested either by
> OSSTest, Gitlab-ci, or also Xen community members. I am not sure what
> would be the highest number with this way of thinking but likely no
> more than 192, probably less. I don't know the CPU core count of the
> biggest ARM machine in OSSTest.

This would be rochester* (Cavium Thunder-X). They have 96 pCPUs which, 
IIRC, are split across two numa nodes.

> 
> Another approach is to support a "sensible" number: not something tested
> but something we believe it should work. No regular testing. (In safety,
> they only believe in things that are actually tested, so this would not
> be OK. But this is security, not safety, just FYI.) With this approach,
> we could round up the number to a limit we think it won't break. If 192
> works, 256/512 should work? I don't know but couldn't think of something
> that would break going from 192 to 256.

It depends what you mean by work/break. Strictly speaking, Xen should 
run (i.e. not crash). However, it is unclear how well as if you increase 
the number of physical CPUs, you will increase contention and may find 
some bottleneck.

I haven't done any performance testing with that many CPUs and I haven't 
seen any so far with Xen. But I have some areas of concerns.

* Xenstored: At least the C version is single-threaded. Technically the 
limit here is not based on the number of pCPUs, but as you increase it, 
you indirectly increase the number of domains that can run. I doubt it 
will behave well if you have 4096 domains running (I am thinking about 
the x86 limit...).

* Locking
   * How Xen use the locks: I don't think we have many places where we 
have global locks (one is the memory subsystem). If a lock is already 
taken, the others will spin. It is unclear if we could high contending.
   * How Xen implements the locks: At the moment, we are using LL/SC. My 
take of XSA-295 is there is a lack of fairness with them. I am not sure 
what would happen if they get contented (as we support more pCPUs). It 
is also probably time to finally implement LSE atomics.

* TLB flush: The TLB flush are broadcasted. There are some suggestions 
on the Linux ML [1] that they don't perform well on some processors. The 
discussion seems to have gone nowhere in Linux. But I think it is 
propably worth to take into account when we decide to update the limit 
we (security) support.

> 
> It depends on how strict we want to be on testing requirements.
 From above, I am rather worry about claiming that Xen can supports up 
to 256 (and TBH even 192) without any proper testing. This could end up 
to backfire as we may need to do (in a rush) and backport some rather 
large work (unless we decide to remove support after the fact).

I think I would prefer if we have a low number until someone can do some 
testing (including potentially malicious guest). If we want for a 
power-of-two, I would go with 128 because this is closer to the HW we 
have in testing. If in the future someone can show some data on other 
platforms (e.g. Ampere), then we can up the limit.

 > I am not
 > sure what approach was taken by x86 so far.

It is unclear to me. I don't see how we can claim to support up to 4096 
CPUs. But that's for the x86 folks to decide.

Cheers,

[1] 
https://lore.kernel.org/linux-arm-kernel/20190617143255.10462-1-indou.takao@jp.fujitsu.com/

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:12:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:12:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718097.1120642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KcT-0003Vh-F8; Tue, 07 May 2024 13:12:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718097.1120642; Tue, 07 May 2024 13:12:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KcT-0003Va-Ce; Tue, 07 May 2024 13:12:33 +0000
Received: by outflank-mailman (input) for mailman id 718097;
 Tue, 07 May 2024 13:12:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4KcR-0003UR-Oa
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:12:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KcO-0006ha-28; Tue, 07 May 2024 13:12:28 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KcN-0000q2-So; Tue, 07 May 2024 13:12:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=ALX2nUnpQGSejcJ0GbvP/TSWhy/TzTrDvkd6ybX/1CM=; b=FEsZkdH+QXqN2zb8TWzKKi3N2d
	VIdF4lkZOCkwqJlPG97bwmoJu/ODyLKR8j0aZ8Dn1lUF9JS1KZ1eAJ+e5czZIHw43NQcaCCkEXY68
	Iia7YoX3ppwFR5uGxMiZ9d302XvR+77x8lKssPrMWX812X9wugsEkqKwwXenelhs6sJs=;
Message-ID: <be1c1109-f814-4716-981f-1e926c655f3c@xen.org>
Date: Tue, 7 May 2024 14:12:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240327152229.25847-1-jgross@suse.com>
 <20240327152229.25847-9-jgross@suse.com>
 <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com>
 <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com>
 <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com>
 <146fbd43-d8b8-4d73-8650-c60024498324@xen.org>
 <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop>
 <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org>
 <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
 <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ccd19870-d44b-44a7-9a5c-72a13d4f74a2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 06/05/2024 07:42, Jan Beulich wrote:
> Of course those calculations depend on
> what people choose as actual values, but giving an upper bound being a
> power of 2 may at least serve as a hint to them.

This is rather a weak hint. If you want to encourage users to chose a 
power-of-2 value, then let's spell it out in the description.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:20:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:20:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718104.1120653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KkK-00068X-83; Tue, 07 May 2024 13:20:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718104.1120653; Tue, 07 May 2024 13:20:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KkK-00068Q-56; Tue, 07 May 2024 13:20:40 +0000
Received: by outflank-mailman (input) for mailman id 718104;
 Tue, 07 May 2024 13:20:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4KkJ-00067A-8a
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:20:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KkI-0006t3-QM; Tue, 07 May 2024 13:20:38 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4KkI-00013E-Jw; Tue, 07 May 2024 13:20:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Vq5D31ZcLBl3pVFEjwUufVPv5XrfimQqt7yfe2KP65g=; b=nvLL9naYIjMk1Hvosgl1vJJsnj
	LhyuXKQD7/JF3QuwLsYdrQvA5HRMN/TXOaZoa9cjkdMZBS7Fgbv9wfJaeF8IskUpzOIuGiGrgHUnv
	9Z1mNjCoY1nybgqyOTzgH261ANpU+//+tfErSPtsk9yc+9ALUGJDpIcFGTEuj2H99i54=;
Message-ID: <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
Date: Tue, 7 May 2024 14:20:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240423082532.776623-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Luca,

On 23/04/2024 09:25, Luca Fancellu wrote:
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.

Is this because we are mapping more than one page at the time? Can you 
point me to the code?

> But today, p2m_put_l3_page could not handle superpages.

This was done on purpose. Xen is not preemptible and therefore we need 
to be cautious how much work is done within the p2m code.

With the below proposal, for 1GB mapping, we may end up to call 
put_page() up to 512 * 512 = 262144 times. put_page() can free memory. 
This could be a very long operation.

Have you benchmark how long it would take?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:31:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:31:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718118.1120663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KuR-0008Li-73; Tue, 07 May 2024 13:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718118.1120663; Tue, 07 May 2024 13:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4KuR-0008Lb-3H; Tue, 07 May 2024 13:31:07 +0000
Received: by outflank-mailman (input) for mailman id 718118;
 Tue, 07 May 2024 13:31:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6rSc=MK=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4KuQ-0008LV-76
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:31:06 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05on20601.outbound.protection.outlook.com
 [2a01:111:f403:2612::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0bfd2d87-0c76-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 15:31:02 +0200 (CEST)
Received: from AM0PR02CA0183.eurprd02.prod.outlook.com (2603:10a6:20b:28e::20)
 by AS8PR08MB8826.eurprd08.prod.outlook.com (2603:10a6:20b:5bb::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 13:30:52 +0000
Received: from AMS0EPF0000019B.eurprd05.prod.outlook.com
 (2603:10a6:20b:28e:cafe::82) by AM0PR02CA0183.outlook.office365.com
 (2603:10a6:20b:28e::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Tue, 7 May 2024 13:30:52 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF0000019B.mail.protection.outlook.com (10.167.16.247) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Tue, 7 May 2024 13:30:51 +0000
Received: ("Tessian outbound e46bb127ed3d:v315");
 Tue, 07 May 2024 13:30:51 +0000
Received: from 6a051c7c62f0.3
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 79ED57C8-3CE3-41DA-A4E0-6AE3B1E5B583.1; 
 Tue, 07 May 2024 13:30:40 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6a051c7c62f0.3
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 May 2024 13:30:40 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DU5PR08MB10415.eurprd08.prod.outlook.com (2603:10a6:10:519::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 13:30:37 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Tue, 7 May 2024
 13:30:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0bfd2d87-0c76-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=UZudkOKp+gbdyMRBFhSGCE/8einnKGQc8LRJmHQZAdr5Q+bAvJFbeoiHDIXmRvkgXhozvTBf97VYKxd8cJ4NRfx5eucUO5jg9h4J3DV1iZEbNQ1SCiJCCU8Kyti3UR0CGe5Ou2U+u98nwtzUX6jhbJ3Rl0Q29USncBFgdjgXUlJ8QDScPsk4fd2mWPQbyryt+K2vS4ECIouZCE22W8OrbEekB82r+AjdneOPlI66NqLq4vPqqxi/clBUN6GM+ZLeluKCKyJ7A+uLClIiSr5B1Wg0Z15Yd3kn4t++5FwIomEZAZpcJi1fZjxsSOeP/fIBbSZ7Hu7BkOpEdNDoVNtcpw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=LLZe/krtQ6Xu4kVb/9R/+gGbvpxBggIpPorzOgR0Bhw=;
 b=XOQrsKwh4HDqJ/VDFegyTNH5S7w4NmmXaKaNjY/OZKnG5k5Q6zZMLtCfhmLwFcs9k5zP27riNA6WRUbw8n03iCAi4ztNc2j0saA2LZ75xzEug3TG5TFD1hZ/4WsngPSpJWzeFkwoS/3iyhDLHS7mztye29csmbMG+4xktIGaBfEEw+9SzlX8gQCOWpUs7DPrEejYINgKEIkpxSOLFIOsLQZlrQdxNxCxQ2asYgrb9iiMfkXfayWvCwsCyglQRFDrxe25eJ06ZyyAEOyD50BODXJiNGYgQLbiNo2EGf88X+Q7sq4a178Hwl1eiJnu/sYg+v1ZQZpvK2iVZiuhsNKncA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LLZe/krtQ6Xu4kVb/9R/+gGbvpxBggIpPorzOgR0Bhw=;
 b=aBOdcJIFpV/mfIV8S9n6o82RDBdpqw9h4jl8rjA4m6hUxKQIXvQYOj5uCAqf07LXx2LTxVGmsYmtiU34NsKl4O/bPrUu3v7xuiWFu5K4coj7Wr3kYytnHVVvzD20zttIqYq9p7WAy1zfqXENdto/xbGS7j/CO0c5Vqeq74wPRV4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 12cf3bd0580ecbd4
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QRNt4g4A7VSxJO3WUpZf9rjoOFENpUE1By8lly6tBntEQm6suXdQG3LSr/EjS66p6+AEjo9pA8aYYkojVGi4qlPiaB5E878/GZtSTR4UWGY6HtO7b0hbgcKjfL9w1R4e2/p0nBTSvxA5IzfSEcsQSiiggWEf9D9Noprh024SQp1f4r0FPrRLc7H0pTl9Tg8STH/GnFGINH5qZ5zYmJ30xdpEzKcrfCgSpH9VVbTdI8aDpXczISnYdZJUC/YGbOSU6UiOOvqvdwrupq9F4Gt+tylXiHUjrZR4+FqNhmI+qQJs9tEgNhkH14T/XCw6FBaYJjHpbZbnjdhcgrHjoD+qdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=LLZe/krtQ6Xu4kVb/9R/+gGbvpxBggIpPorzOgR0Bhw=;
 b=ByY9qVFQu9/SPoA6ZeOdjVj2A8sHiuAm0UMhr3F33UiSpJkPGPxxn2gTpf1Nzmyr/oFpiCyMt6wBF28qAUUILAINPAvaV44iI6tdbboxtC1rJgCoOdGHHpUC1VOJHgwb3jhLIV1Aeq8zjR9mzW7jHnrNYS2ffpaJcwRaboiP5QQMYJQYEKgOQ3JcJ3uwUN2WVSXcj8UOvSqnKl9SUejMGyW6rnMtxq0KRxkxoOUX34hCBLRTCwz4WH3zTHkKegx6FXs6D89Yr4CWnsoZUOHh1EyaMC1ZwPOhjlo7h4kRnDmWXvEOyH2Eb2moCtFJmJ3NcVvjPOp0YDBUCiDtxsmd5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LLZe/krtQ6Xu4kVb/9R/+gGbvpxBggIpPorzOgR0Bhw=;
 b=aBOdcJIFpV/mfIV8S9n6o82RDBdpqw9h4jl8rjA4m6hUxKQIXvQYOj5uCAqf07LXx2LTxVGmsYmtiU34NsKl4O/bPrUu3v7xuiWFu5K4coj7Wr3kYytnHVVvzD20zttIqYq9p7WAy1zfqXENdto/xbGS7j/CO0c5Vqeq74wPRV4=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
	<Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Bertrand
 Marquis <Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Thread-Topic: [PATCH 3/7] xen/p2m: put reference for superpage
Thread-Index: AQHalVftK+2d6c8qh0SLUJVfhafWO7GL2AEAgAACwYA=
Date: Tue, 7 May 2024 13:30:37 +0000
Message-ID: <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
In-Reply-To: <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DU5PR08MB10415:EE_|AMS0EPF0000019B:EE_|AS8PR08MB8826:EE_
X-MS-Office365-Filtering-Correlation-Id: 8f28889c-3db6-4745-e11e-08dc6e99ea3d
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?d0w5TDFpbmFWRjVOK0lpbVJFWkxpVWljMWxrOHU2TVY1Q2MxYmNkRlZuTTF6?=
 =?utf-8?B?dEJmekZQL1o1M1pIWm9LQ3loOE5lcExLOUxHZmpaTE9FRDdNKzlzaW1QOENq?=
 =?utf-8?B?WTZ1eVZnMDc5Um9IbnRDenVFa01GRlNhanEyQ09wNk13bExEQ0tzcEJ2c2RD?=
 =?utf-8?B?SG0yQ0oxQklpNEdWelBQdEtCUUxaZW95L2NIeGxIai85V3A2dmplRUtxdGFm?=
 =?utf-8?B?bm9TQmF4Mnppa2lISUhUWENUVWR2Yjl4ckgweEgxeHVKdVBZZVJOeG5MeW55?=
 =?utf-8?B?SnR4NUNCcktNZ3V5VEw4OHpCVStyaTdGdldSTFFrOTgvK2VFTmtVMm92Wnhn?=
 =?utf-8?B?Z1QxYUpvNHg0bGpaUjJCanhnNi9IVUp3T29sK045eUZOVVlRK3M1N1dRU0Ew?=
 =?utf-8?B?U216OG1BWUZMNWhUaVYyekM5anp2eHN4ZGJ1Q0lSWXMra1Q1Ky9LOGc3bTl6?=
 =?utf-8?B?RHVubWpQNWVKazUvdStKQTZ5RzNtRUVuUmRxMlQrUTI3TFovNGt0YjVyNlE3?=
 =?utf-8?B?R2J5M0JlU0VudWZkN29uNlRrYjdwK0pzT2VJVXArZ0VTWkR6end4ZnRuWnd4?=
 =?utf-8?B?WlhlRjhwRXZxMlAwRlRNMitLVXpaZDU1YkFJWUo5ZDRBNGVMays4VlN4Qkxz?=
 =?utf-8?B?Z3ZIUlhtZWRTNUx0YXhxaVdDT1ZuRDA2cFFFcnhjMkZGWm1rVTNOYkhQZ2ZG?=
 =?utf-8?B?T3I2K2dGdEdMcWg0cjREUUNFODZkQ3ZaUTg4U0c0S1IyTlE0OUJkTWFvb2Uy?=
 =?utf-8?B?OTRXalBaYmxtSHRwRU5pb2EvQTFXVjdFZ0s4NHJSM1E3WklVKzhPS0FzaGRZ?=
 =?utf-8?B?bjhZUGwrNnYyUHlBMUxGRnU1VGRhblpGemhvZ0ZzdG1ORktzSEUyUThhSDln?=
 =?utf-8?B?MFAvVUFDY1V0OFR3dVk3SXMwZS9oOS82b1dnVG45Zmc2ck1sN3FXNHpHYWJZ?=
 =?utf-8?B?aU1xZ0pOSUkxQlJ5aitzS1hVQWxvQXVqZW1wR0Q1QWhEbzRMSE1McDdQMXRz?=
 =?utf-8?B?Y0pTZm5vNE84YzN5WU5lNEdWT3BLNkgvQWdUaDJhMFlua2IrWUpGZEFJbEov?=
 =?utf-8?B?di9mK0gySWE0VDRpMzhZdlhjd3FIWktwM1Q2QWFITEVnM0VtYjh3Mm5BTnl2?=
 =?utf-8?B?M3J0SDNqaHBmNVVJQlZlQ3ZtNmlHNFZjbXlBeisrc2RQWmRxT3N5TmxaZVlR?=
 =?utf-8?B?bzZWekI2b2RLSy94azZHNVI4Y3pzYUFmRVlZV1lETXg3NlNsVURIYm9OMXU5?=
 =?utf-8?B?RFFic09jN29HZkRpV1h6aHd5bzJTUUcxa05GdmphaWZZblQ5NDNqOU9wNmFo?=
 =?utf-8?B?TC9rdWFxVE1QM3MvNlJQdGJUWDVOT2I4OXNONHNzZ1EyUnpkYzNtTTVLc1gv?=
 =?utf-8?B?cU93RG9mOFBBV2ZtREFnZ1dod1lPUytNbTZHWDZYVGJvZ2tJc3JlWTRkc2Jz?=
 =?utf-8?B?cmRyckhsL05uVE1JcUo4cDdHSzhXVFk3bkVMTFl5NnAxZ0J0QkVyYm9QWkY0?=
 =?utf-8?B?SGQ5bHFvdWJyemZQOXF1aFd5S1lnZDUzemhncVFBM0VTeVZZa1dIN0dybTlj?=
 =?utf-8?B?cVM3eXZkQ1ZHUHBYajB2ZS94MUhyMDVxcGlQWjljUHcxOWpMWmZJbXhiWGhD?=
 =?utf-8?B?OWVLYlhzTnlqSGRJelQ4WnJyb045bm9iNG9icWZDcVVsZlhvdS81c0d6NXcz?=
 =?utf-8?B?UndQWXdEU05VU0YyOWtNZkFZRG9oZlh3K0pPWmJobjJlZm1BWkd5RG5WRmV5?=
 =?utf-8?B?aDlhUEVnc1ZTdHJJbjZsc3VWSUhtczltYVlPeEl4ZEt2UDAyY0czUDBSOTJu?=
 =?utf-8?B?LzZaSzJIeG9JSkVSLzI3QT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <77AC5011AA01494FA545FE595E2026D0@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU5PR08MB10415
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF0000019B.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0412a9e2-e79e-489d-2f8c-08dc6e99e1ea
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?OW8rRmNWb1JITzc2OE5QcXZJb2h4UEJVYTRMSW9EdlZWSHAxVVJwMWh5MHlY?=
 =?utf-8?B?ZThLdkZoTHczRUdYaitlTGRLTGFqMVVsV2JnZ09Qb2pBMzlzZGI1Tm9vQ1RO?=
 =?utf-8?B?NHlaaHNmZnZxdlo4bVRHOEtRRjM2QnNnakdvN3B2KzRUNzROSnBhdkpTSVkr?=
 =?utf-8?B?dUtNNFgvdWsveTBLWWl1dStCR2UrVkl0MUFwN2hpNWdNUnFNVjhWOG9kdE5D?=
 =?utf-8?B?ZjQxWFdrdmc1T1FVallpSE9ocUFBVTUrYlVMbzBtWmdoYzNIS2lkZnkrRGww?=
 =?utf-8?B?WlZ3L21vOVlieXRacUEzaFhWRU1LNDhDMlMyNk5rRFl4RHN6eDNtNU1GUHBu?=
 =?utf-8?B?WXRhSWhIazZRSUlhTXQxdnhVNlVONnc1ak1HcmNDR25acFhYbmtiZ3RWTHVt?=
 =?utf-8?B?VHZUajNVVGJUUkJzNzlZQktORFNTcFFRQVlicGVCbCtBZ0tVMkJoa2l3QXBG?=
 =?utf-8?B?MzhmSThZeFFXOXQ3Nk9ONmhrU3BrWmRrajFOU0pTeVlWQTZFR1p5OC82azVZ?=
 =?utf-8?B?NVRoZWRYQ0tpejZZRDNDUlFxbnRGcXo5WG9nSks2alQ2Y0kwU0tEWk9oU3Zi?=
 =?utf-8?B?eWxlbEhnc3RvVTMxSnByVldlVzR6RFRTVm1ybnljcnF5Zy9QN0tGcFdLRFpZ?=
 =?utf-8?B?K3hDY2ltOUd3N0JKbXQyKzlSejBRUzZSRWhDUEcyOE9YOEJVV2Q2d0M2UVNI?=
 =?utf-8?B?QklYcEcyNy9OV0NJanUvcWpLZUtmNEErWmxJT0M1SG83VFhIbnMrT2RZLzZU?=
 =?utf-8?B?bjN2NHgwYUFacm0vd1JrNVZibXBZWWVRQmRzVVRjNFg0dXMyS1JBQS94QXNL?=
 =?utf-8?B?ZUlrbDhCeHBkVVB1WmxqWWZ6N0tpaVV5ZWprUDhGeENEWGJES3Fsa1I2amhs?=
 =?utf-8?B?RjVYNnNNUmRZTTVqT3VwdXNUN2kzWkY5UWlmYmpJdlhyNzEzSjU1aFRmd1FQ?=
 =?utf-8?B?T2xjVm1jM05sKy9xNzMxT1E0UUk3a0MrSUp6QSt2TE13Mko2N0lZSDQyaXJX?=
 =?utf-8?B?RmhNT2VzRm92THRNc2R2djhQV1dZN3JVVDZ5N283V3hxZ2hmLzlKTW1OL08x?=
 =?utf-8?B?ZWV0MnlWdUIrYlp4ZWdpckdBbGUyT3IxT0tZSUUzSFI1WXBxMlIzaktFakt1?=
 =?utf-8?B?Y002TUxwb2dlcHI3aytEQU5udko3T0U3dlo1TndWVzR4WjJUQ20rY09TQzJq?=
 =?utf-8?B?ZzJxZXZNU1Q5YmFYeDM3ZjRvQ1lWRzlEYlNWYndJVTc1UWw4TnVmT1hQRUhs?=
 =?utf-8?B?ZkQ4bnhNdS9rYWVzOStEa2E3Zngwb0FLRnpDbXpzaERBV0JjLzZmL0pFQXVP?=
 =?utf-8?B?NEVndWh3M0dDOElDd2M1ei9ONTdDeFI3NzcyQ0d4LzYyYnhUUjYxR21aSlBs?=
 =?utf-8?B?QlpkalI3QUFQdHgxZ3FwZmJTR2kxNHJiUXBuZkRoNHk2NGVNVEJpYkhabXd4?=
 =?utf-8?B?Y2NjNVRxRTB3b3l1Y0N1QXZtd2E0OWt4YXJzSE90NFFwZmc4aml4c3lUdFBo?=
 =?utf-8?B?M0lpZUJhZmRESGdYSTV0dGcwZXdFN1F5QjBVaG1xNnBzUEhyakNvZktSRWFI?=
 =?utf-8?B?bENGbnEzZWJtbGxzL0Nna0kwZWdadTduTWgwVUVXMnNOV3VyQWhVNVdxNmtt?=
 =?utf-8?B?MTZwNEFkOUlPa3BNRk43WHphVjFnMDlSRFRHOWl3M0lnanRDYy9SeUFMa1J2?=
 =?utf-8?B?ZnV6eVROYnp3bTVreHUvRHAxNGZKUnlJZ0JUbExPQm44ZHR0RThYemVueENp?=
 =?utf-8?B?Y2VQOGVPVnVsZHBQZ3JtMEJLdWFvL3JOcnhIYUhUMCt3dHRGNEVqRlFLUmtp?=
 =?utf-8?B?djZqNngxSEJDVzNlZy9BRFRZcnZzL2JCajBWMlMxSkxHd2JoOGdxS3lublhD?=
 =?utf-8?Q?UfUv4tDuA3vjY?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 13:30:51.8118
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f28889c-3db6-4745-e11e-08dc6e99ea3d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF0000019B.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8826

SGkgSnVsaWVuLA0KDQo+IE9uIDcgTWF5IDIwMjQsIGF0IDE0OjIwLCBKdWxpZW4gR3JhbGwgPGp1
bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4gDQo+IEhpIEx1Y2EsDQo+IA0KPiBPbiAyMy8wNC8yMDI0
IDA5OjI1LCBMdWNhIEZhbmNlbGx1IHdyb3RlOg0KPj4gRnJvbTogUGVubnkgWmhlbmcgPFBlbm55
LlpoZW5nQGFybS5jb20+DQo+PiBXZSBhcmUgZG9pbmcgZm9yZWlnbiBtZW1vcnkgbWFwcGluZyBm
b3Igc3RhdGljIHNoYXJlZCBtZW1vcnksIGFuZA0KPj4gdGhlcmUgaXMgYSBncmVhdCBwb3NzaWJp
bGl0eSB0aGF0IGl0IGNvdWxkIGJlIHN1cGVyIG1hcHBlZC4NCj4gDQo+IElzIHRoaXMgYmVjYXVz
ZSB3ZSBhcmUgbWFwcGluZyBtb3JlIHRoYW4gb25lIHBhZ2UgYXQgdGhlIHRpbWU/IENhbiB5b3Ug
cG9pbnQgbWUgdG8gdGhlIGNvZGU/DQoNClNvLCB0byBiZSBob25lc3QgdGhpcyBwYXRjaCB3YXMg
b3JpZ2luYWxseSBpbiBQZW5ueeKAmXMgc2VyaWUsIG15IGtub3dsZWRnZSBvZiB0aGlzIHNpZGUg
b2YgdGhlIGNvZGViYXNlDQppcyB2ZXJ5IGxpbWl0ZWQgYW5kIHNvIEkgcHVzaGVkIHRoaXMgb25l
IGJhc2ljYWxseSB1bnRvdWNoZWQuDQoNCkZyb20gd2hhdCBJIGNhbiBzZWUgaW4gdGhlIHNlcmll
IHRoZSBtYXBwaW5ncyBhcmUgbWFkZSBpbiBoYW5kbGVfc2hhcmVkX21lbV9iYW5rLCBhbmQgbWFw
X3JlZ2lvbnNfcDJtdA0KaXMgY2FsbGVkIGZvciBvbmUgcGFnZSBhdCB0aGUgdGltZSAoYWxsb2Nh
dGVkIHRocm91Z2ggdGhlIGZ1bmN0aW9uIGFsbG9jYXRlX2RvbWhlYXBfbWVtb3J5IChuZXcgZnVu
Y3Rpb24gaW50cm9kdWNlZCBpbg0KdGhlIHNlcmllKS4NCg0KU28gaXMgdGhhdCB0aGUgY2FzZSB0
aGF0IHRoaXMgcGF0Y2ggaXMgbm90IG5lZWRlZD8NCg0KDQo+IA0KPj4gQnV0IHRvZGF5LCBwMm1f
cHV0X2wzX3BhZ2UgY291bGQgbm90IGhhbmRsZSBzdXBlcnBhZ2VzLg0KPiANCj4gVGhpcyB3YXMg
ZG9uZSBvbiBwdXJwb3NlLiBYZW4gaXMgbm90IHByZWVtcHRpYmxlIGFuZCB0aGVyZWZvcmUgd2Ug
bmVlZCB0byBiZSBjYXV0aW91cyBob3cgbXVjaCB3b3JrIGlzIGRvbmUgd2l0aGluIHRoZSBwMm0g
Y29kZS4NCj4gDQo+IFdpdGggdGhlIGJlbG93IHByb3Bvc2FsLCBmb3IgMUdCIG1hcHBpbmcsIHdl
IG1heSBlbmQgdXAgdG8gY2FsbCBwdXRfcGFnZSgpIHVwIHRvIDUxMiAqIDUxMiA9IDI2MjE0NCB0
aW1lcy4gcHV0X3BhZ2UoKSBjYW4gZnJlZSBtZW1vcnkuIFRoaXMgY291bGQgYmUgYSB2ZXJ5IGxv
bmcgb3BlcmF0aW9uLg0KPiANCj4gSGF2ZSB5b3UgYmVuY2htYXJrIGhvdyBsb25nIGl0IHdvdWxk
IHRha2U/DQoNCkkgZGlkIG5vdCwgc2luY2UgaXRzIHB1cnBvc2Ugd2FzIHVuY2xlYXIgdG8gbWUg
YW5kIHdhcyBub3QgY29tbWVudGVkIGluIHRoZSBsYXN0IHNlcmllIGZyb20gUGVubnkuDQoNCkNo
ZWVycywNCkx1Y2ENCg0K


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:45:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:45:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718128.1120673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4L80-0002d9-E0; Tue, 07 May 2024 13:45:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718128.1120673; Tue, 07 May 2024 13:45:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4L80-0002d2-An; Tue, 07 May 2024 13:45:08 +0000
Received: by outflank-mailman (input) for mailman id 718128;
 Tue, 07 May 2024 13:45:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6rSc=MK=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4L7z-0002cw-6Q
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:45:07 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on062f.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02eadff9-0c78-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 15:45:06 +0200 (CEST)
Received: from AS4P190CA0003.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5de::8)
 by GV1PR08MB8355.eurprd08.prod.outlook.com (2603:10a6:150:a5::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34; Tue, 7 May
 2024 13:44:59 +0000
Received: from AM4PEPF00027A67.eurprd04.prod.outlook.com
 (2603:10a6:20b:5de:cafe::5d) by AS4P190CA0003.outlook.office365.com
 (2603:10a6:20b:5de::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Tue, 7 May 2024 13:44:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00027A67.mail.protection.outlook.com (10.167.16.84) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Tue, 7 May 2024 13:44:59 +0000
Received: ("Tessian outbound 85e363419cbe:v315");
 Tue, 07 May 2024 13:44:58 +0000
Received: from 082676b6661e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B8041C56-2D6F-4BDA-A6BC-1158593B2F67.1; 
 Tue, 07 May 2024 13:44:48 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 082676b6661e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 May 2024 13:44:48 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by PAWPR08MB8910.eurprd08.prod.outlook.com (2603:10a6:102:33b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 13:44:46 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Tue, 7 May 2024
 13:44:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02eadff9-0c78-11ef-909c-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=OJXePcf35FueMV6ZOAEI7ig+WXHOg9TGJ7sntuwZ1tZcBH2q27OjCvnl6tZK3lF9nUNxQOk/P9+mlRUgmSWItaw5unRYBZxJTo1n0gY68r62TWD0Ega+YGDkk64MBombLbOku4Yk7W2yuCYocFI9CKZebQgygK7wkkcVf4SwR3764RwbgEp+jeWm2hqixw/CdhOMrYUx4JZ+t8OIAv4IjXwh9mYa0iaI6RyHD22wMR6B5wZ0hpfvy/jKnBURqGyRcl0rdwdrRk0h8bAXuG0qfj6oCY2ltJqj0APKW+J8KrJBhoUb9nGIoFXGOP4QoW7tem37zYabpE1qUgmxhS7Svg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RurwdJ6fzazxDFFVx47I1keTCoWhiTFJ/e3dKHXsu6g=;
 b=VLJ3U7Mwr+LC4KS1GC7K8XDpkXyqf7es6dpXeBED2clj/+evEIiBOfRVUzspwQ8acvMBGtfcvkDo7c7D+Wf6n9SHhjuVUpr38j8eJSAOiuMp8ZYBELrqCBGAXQ/KJfNnwuU7idgTsPvy9e3BLSTCX/bcC7tnVk2LitlxCGqVqlzEZvj0eRkR2bFWlx0gjf8hhtxutOeHYSnIFIU1aScuSA5PvUwxxw1iB860gH40Mk2EDpH/ZP8IEqKF0kylB4EKhmBJUnJktMhNKQYNF4Q0hLZQiw15sIJjjfiqA3iLta+5V8ka6n6v+2U1Uke7oDWPZT7OkxdwoiZJilLxw7Pg5w==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RurwdJ6fzazxDFFVx47I1keTCoWhiTFJ/e3dKHXsu6g=;
 b=CUofkBkx/AlyjdEWsDW4ujCscL3guRnQRsep5XtS3gcvdR1xMwIjD/jmZjl/KCKSTFKgJ1AsUtcnf/BfEM9sT9n0b2luZlOgK5FwsB5N66rIcb7SNVo068Z826xsxm/lyhsVQhjO7uiW4yFHvn3NZD27gtnxmYpWQA3j6KnWHZ8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: f77c75a8c54e75a9
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ic7cO5u1K+lz3eATedlvmNIwgxyAUQWTz/UjvmdbnivTE7OEsjkwIgfWj3pbYG+Ww/z5zgRng3ZFpxoOjhzkh4y9Tn1zE6Rnm7uyWDU/734kjkyQ0XSkOtat3w8yKq8ezEyvXcav0/efcAOdvm254BWTEJv4WFuUE/DN+HNwchAGYPo9u7uw5j2drrDJDtK6kYlVDrhj2ct+DKqMKmyHoPWhoqhW7VNvQ3qKYA96NSZdK5XXDO+qBA97wjtUXY6nFUpfIBvuMXMUZAvwSa23jXJzc7DwlWyb2baCNKrghMa99KPQEuE6GFwiH6isgbiuoQUH5YhOdVTXxreisAh9bw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RurwdJ6fzazxDFFVx47I1keTCoWhiTFJ/e3dKHXsu6g=;
 b=hmn60C74V5Wm0RuLDpJwXHqhAIsRhaBMYYevz54jlJCnpLNuv6Me5QDNj7LF1xdWv3UHaIf+rcoXdS0kGkVvDH1HbCmDItsdTdMiCVXXjeoaU9q7wjWIXVCd4Ih25cRbKIoBvXHpZRyAxCAMDEFNeTYi+vaVaFyKvWqId/uAcz2SaOlWucxja+7CdSEhvCQps6qVj4fh5qv4rpfN2CdsEiwSeh6rTbsw+I/MU19LFUnsM0+vSYP2GXFIC4ea+MW7YKDWhAbF/A8s+conNK/r+SrZ3A3Y+dl59FNSDupv4R4IisGKLgxoviAEFwJiZSRl6NMkbqksMrmkT3JjrSuZzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RurwdJ6fzazxDFFVx47I1keTCoWhiTFJ/e3dKHXsu6g=;
 b=CUofkBkx/AlyjdEWsDW4ujCscL3guRnQRsep5XtS3gcvdR1xMwIjD/jmZjl/KCKSTFKgJ1AsUtcnf/BfEM9sT9n0b2luZlOgK5FwsB5N66rIcb7SNVo068Z826xsxm/lyhsVQhjO7uiW4yFHvn3NZD27gtnxmYpWQA3j6KnWHZ8=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Thread-Topic: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Thread-Index: AQHalVftWBiV4yp+FUmOns8otkWCO7GKRsoAgAGX6oA=
Date: Tue, 7 May 2024 13:44:45 +0000
Message-ID: <882A5390-64AC-4FB6-AA53-4CE466CF9062@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-2-luca.fancellu@arm.com>
 <28322f07-99fc-4428-bcf2-b8c2af04d38b@amd.com>
In-Reply-To: <28322f07-99fc-4428-bcf2-b8c2af04d38b@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|PAWPR08MB8910:EE_|AM4PEPF00027A67:EE_|GV1PR08MB8355:EE_
X-MS-Office365-Filtering-Correlation-Id: e275f028-ce93-4579-603e-08dc6e9be338
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?RTZ3UGdRU2pHdGRoK1BzbFY2cE82aUxLb2d4VDhtc1FWZXh5emFZWmJ0WStR?=
 =?utf-8?B?MWtPelIybzlIb1Q1Wk44M0tPZisyZDFua3d1TzdzeG8rUytkZEt4SGFuZU5V?=
 =?utf-8?B?OEpTVEcyVk9EUjhuUStaVXVlTzI3TWZ3bW5uMitmQlAzeEVjRU9TeGZMZERr?=
 =?utf-8?B?SytIekRvbHFvU1ZXb2hTMUg5eWxMTnYwY0ZzVjFTSVIxSTRja2oyT09HZDd5?=
 =?utf-8?B?SnRTQ2RQMzk2VTNwMEgxQVhlV2J2VDBVb2owOW80cUFlNXBISFJSdXZUTGwy?=
 =?utf-8?B?ZnZ6OFVDQWhmaXFVV2s5SjIwV08zUEN4M2VQK3NlRTFScHBoSWxOeUlqQUNo?=
 =?utf-8?B?RFdQRVdYLzk0UFduUEVHaloyNjZHcWNlWDQ2T2lNVEdNRzJzaTRpSWszMGh6?=
 =?utf-8?B?N1paVWVwTEZ0L25FRFByZWViNU1KMkYvT1NkMEhJUDZBT2NMYXJXMW50dWpy?=
 =?utf-8?B?S2xnZEpjNkcvclJjSWVIZHFoSEkvV0wzOS9JVTN3OGlnR2ZDRXh2Rk9CQTdS?=
 =?utf-8?B?NlNFUCtDYWNDSHpYZGp5bHJhL0hEZms0blpKWWxwZmNkakFTcUhTeFNzTkd0?=
 =?utf-8?B?eW5YMGZnbWVHVVhza09JV2JmbS9ya0FkaXpJOFFnNGRTU1IvTGsxSms4MmZx?=
 =?utf-8?B?MnRlZ0hONVMrRUlOQjExSENCU0E1OThwdUwvV0kzcWhhU0Rlamo4NjVlYVha?=
 =?utf-8?B?WEJsdGlZMm1IR2d0ZzF3SklqMTV5TDgrUmtXbW5NVVRCdVlhSEhGK245Rnlw?=
 =?utf-8?B?a3NHKzRVaGkvM0JaTmkwSzkrekR1bHhHNHh1TDJ4aHpoSWsyUCtveVd3ei9x?=
 =?utf-8?B?QnFKNDNrenJyYXNWekVHNFR0MW5PWDAxSlphUDFJMjE0WTZsUjFKZzVrZW8x?=
 =?utf-8?B?dnc0RVR6UlN5ZWRsK0VpWExKMEJuZFVvbnhCVXZsZWx2QWxpb3pNOFRQOU5H?=
 =?utf-8?B?bUlhck1QL2VqYi9EK3dyQW5Qb1VYNzQ0eGhUcGV2VjFQbHBZT1dyYUlZUlMw?=
 =?utf-8?B?UEdlTkZiOTRiNGxYb1AzRDNSMU9XNWsrTEJkbmFwbUJ5SysrL2tQSG4weHk0?=
 =?utf-8?B?SXNFamtBMkZsMDg4VWhnK2ZGYWNpOHdheVJQYkxDTGJ2VmNTdWVnQWhQZy9n?=
 =?utf-8?B?WFFTSGFVTnZDOVUrS0RLMWoyeW1icXprWnhYWUNmNWMraDcvRnFIbmU4Q0NT?=
 =?utf-8?B?YkxLUnJhYXE4RWFITHlRMVd2bWo2NEFSNi9QTGFHM3hhWXQyUkYyTGFtbXln?=
 =?utf-8?B?MzNEYllLZ3h5NlRVVU0vcWtvMndOQjV1SUhMWFNHblBWVCtwOEZBNW44enlk?=
 =?utf-8?B?N3NlNVVUek8vOWZLdzJNVE16ZmMrZHNya2o2REY2UmI3WmxPbExoemlRSFdy?=
 =?utf-8?B?dXA3aWhnVG4zSUgxVzRoV29Sc3R3RWpvQnYvVVhocDlGOHBkT2FJRDBTVk9W?=
 =?utf-8?B?MVg1blJoL25Jd0hIdFpibkNUZEdFemhJUlI1bXpxR3hLMk9TeEs3ZUFKUStD?=
 =?utf-8?B?bnVLUmxNNHJKY09reFRkbHJ3N3U5Z08yNW1lQ1hNWlpXZms1Q0liN3NyOVpq?=
 =?utf-8?B?cVB4R3FjdVo4a1BmOFlmOWV6RGd2VGtmUyswVmJHYnhkbjVOZXJ3Tjlqdjd6?=
 =?utf-8?B?cUZPUUNqdkh5T3BJRCt6ejgyNFYzWkppOVQzQXhETWpLSG1hWVhzL1lkd1B1?=
 =?utf-8?B?UVlTdDJaNXkrdnlRNms3UlgvS2lTNU9oTWNqTkFSZ0FoSlZERU13SU9FdEZI?=
 =?utf-8?B?MzArdnBYbVVJbjRpTmgwUGp2YjZoSnB2angvVjA5Sk5BZFl5TXMyd21wOWE4?=
 =?utf-8?B?UVNaTEhuTHc4Vmg2S2lQdz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <A0EE7CB667B56F47A604BDAAB23E5ACF@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR08MB8910
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00027A67.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	aacf5cdf-5dc4-43b4-8673-08dc6e9bdb76
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Z21PVnB4ZXVMekJXVFl2WjBzY2xSYzFXVzYrSSt0L25WQmVDMXRtTTJ4Q09t?=
 =?utf-8?B?aStOZS9MUGh0ZS9EVk1PNHltdW9TZXBLcVliYWtqU1NqdEtiZkxUOUxrQ285?=
 =?utf-8?B?ZnJKY2JVUFN5ZTEyOGc5aDJ0b3ltOG9aK0pHK1ZBV3cwRTRYSXBheWNJT2lZ?=
 =?utf-8?B?eEFiNEtuVDJ5S2VZYVlub3ZNZVR3UlJFNHl4ODBkZXNRcWRCazVRMmxzaFZP?=
 =?utf-8?B?TU1LRGVLenRyZmVCZ0tYMkl0UnFYN0xTSnZrRmxHMER5cGhHaFNyYzQraHFE?=
 =?utf-8?B?WHljV1Y0QWlRaURqYWJsUlo3TkJQMzM5eE1XS05hQUVrT1dyQS9vdjVCL2Mz?=
 =?utf-8?B?dlY4N1FJa0M0TkE3SCtWNWJsQUpHWXNEdS9NVlRxQTM1bitrSzV0V2FPVTF1?=
 =?utf-8?B?U0VNOTJENUp6a0ZvQjh3WklmRW5VRjE0Um9Nck9zRVY3TnRwRlppTnNKSUpC?=
 =?utf-8?B?YUpYdVZJUUcxVmE2WnBuUThKSHk2OUpxUEs1amliUGhBbldjYTRHOHlpRGJV?=
 =?utf-8?B?SkE4em9Bc1RkMnJvNFhQdEhnSTkrYzlkeGlRY2lWRGFxYWxWaUtlV3FoMHFE?=
 =?utf-8?B?VFpnVjlMSDdSSmtwbVUrNnVqdDBnaGJjUFRDeDdqTHRocG1xV3N6Yzh2YXJ5?=
 =?utf-8?B?Q1JhQ3FzdmpXNW5OK1AvWEY0YUtReEtaRXRWOXV2Y0o3NjdIdDNtRTN2OUlD?=
 =?utf-8?B?MGV5cm00RmJXT2ZvWEU3UWMvN2RNeG9MUnFack0zbnAveFRiME5zUEU5cW5q?=
 =?utf-8?B?Y3ZmWERkRXBrd3NqSzRhdEVxcUNOR0JWYXF5NnRSc0xIaVp2WW1ZNlMrNm1Z?=
 =?utf-8?B?elRVNGJ0MDJUTjN3aHY1MUs1SlpQTGZoK28rWHUrQnJlVU9uZUlrSUpUZlQ2?=
 =?utf-8?B?dzhpdkpXOUJVazJvbU9LejJFWWJLOTNNajlXeWl6clYwMHMzbWFRRG41Um0y?=
 =?utf-8?B?Zjk1ZkxJYzc5czJSdUNEeWVBeEU3YlB3ZUtaU3V5YXI1OFQ4Y3pPbkJvOTR1?=
 =?utf-8?B?Z3RydFFLZk9jbGt6S0JFSy9YNVZSdG8weTV1Uk9zdHkxVDlYc1lJS2xXMXdL?=
 =?utf-8?B?NU0wL09haWFGak93dWpyMUFxVEY2Znc5ZndsY1NvM01CQ1RtWCt1WmluU0Ur?=
 =?utf-8?B?TWR5QzVDN2srZ2xwTkhKd0xwOThzMGN4TkkwVk5iZ1hlYkJxKzVidVBUZWJq?=
 =?utf-8?B?TkZ5cXV3OTQxOW5NSUE2RVVLZnFRdkVuMzhKQ3JNRWZFQ21ZWDJSalcxUndl?=
 =?utf-8?B?T0RzcXRRK1RmRTN2RSs5UmRUcGIwVGVnOHdpL0M4UlgraGg5RTdyZGkyNWdC?=
 =?utf-8?B?NVA1b1g4aUZkV1hzZ3o4NkJsL1I0U0pqSFF2Vll6ZFFpemg3T2Rqc1pVWGpK?=
 =?utf-8?B?V09SVFBkSnJ1S0ZUY25FMXVSQ25UeUJLY3hJMmt3WGxpeTN4dFhvUTFsaG1t?=
 =?utf-8?B?M2FDdE53RVllSFRiL0hSM2hvNzcwcWpYcHg5NlBEUlhxVnYwN1d1L1RCOVU0?=
 =?utf-8?B?MjE0Y1YvTUFWb3lMd1hIRnFtVjJ0R25nck1EcUpGemIvN0xkcTR2cGpOUWFC?=
 =?utf-8?B?d1lwU2RxWS9UYzYyek5VbHhHU0x6VUlLWEpaQkhKZkVVZVhrd1R1bFpHZ0R6?=
 =?utf-8?B?Ym9jVndxOGgyOGFFQ0tSS2pMKzMyMytLT2JOWG5qR0ZFMWVlVXR3bnErcUFt?=
 =?utf-8?B?VHlORnNlSjlBU3kvL3dWVStDNXBtdWQvQWU5K2hpcHhhSWdPWCtFSHkvRitj?=
 =?utf-8?B?MVFuRHVTYTRtUktXeG01cGRnL0FGYXNCUkhoYmdXL2twaFE1Ym1LdVNMZW5K?=
 =?utf-8?B?djNGa3R3cmFVQnM5UjI2UFYvV2VQNXBOYk9CbVpsbFhPcHE4NnZvZUQ3d2pm?=
 =?utf-8?Q?Ys0tNTt8fbr6+?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 13:44:59.0251
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e275f028-ce93-4579-603e-08dc6e9be338
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00027A67.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB8355

SGkgTWljaGFsLA0KDQpUaGFua3MgZm9yIHlvdXIgcmV2aWV3Lg0KDQo+IE9uIDYgTWF5IDIwMjQs
IGF0IDE0OjI0LCBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnplbEBhbWQuY29tPiB3cm90ZToNCj4g
DQo+IEhpIEx1Y2EsDQo+IA0KPiBPbiAyMy8wNC8yMDI0IDEwOjI1LCBMdWNhIEZhbmNlbGx1IHdy
b3RlOg0KPj4gDQo+PiANCj4+IFRoZSBjdXJyZW50IHN0YXRpYyBzaGFyZWQgbWVtb3J5IGNvZGUg
aXMgdXNpbmcgYm9vdGluZm8gYmFua3Mgd2hlbiBpdA0KPj4gbmVlZHMgdG8gZmluZCB0aGUgbnVt
YmVyIG9mIGJvcnJvd2VyLCBzbyBldmVyeSB0aW1lIGFzc2lnbl9zaGFyZWRfbWVtb3J5DQo+IHMv
Ym9ycm93ZXIvYm9ycm93ZXJzDQoNCldpbGwgZml4DQoNCj4gDQo+PiBpcyBjYWxsZWQsIHRoZSBi
YW5rIGlzIHNlYXJjaGVkIGluIHRoZSBib290aW5mby5zaG1lbSBzdHJ1Y3R1cmUuDQo+PiANCj4+
IFRoZXJlIGlzIG5vdGhpbmcgd3Jvbmcgd2l0aCBpdCwgaG93ZXZlciB0aGUgYmFuayBjYW4gYmUg
dXNlZCBhbHNvIHRvDQo+PiByZXRyaWV2ZSB0aGUgc3RhcnQgYWRkcmVzcyBhbmQgc2l6ZSBhbmQg
YWxzbyB0byBwYXNzIGxlc3MgYXJndW1lbnQgdG8NCj4+IGFzc2lnbl9zaGFyZWRfbWVtb3J5LiBX
aGVuIHJldHJpZXZpbmcgdGhlIGluZm9ybWF0aW9uIGZyb20gdGhlIGJvb3RpbmZvDQo+PiBiYW5r
LCBpdCdzIGFsc28gcG9zc2libGUgdG8gbW92ZSB0aGUgY2hlY2tzIG9uIGFsaWdubWVudCB0bw0K
Pj4gcHJvY2Vzc19zaG1fbm9kZSBpbiB0aGUgZWFybHkgc3RhZ2VzLg0KPiBJcyB0aGlzIGNoYW5n
ZSByZWFsbHkgcmVxdWlyZWQgZm9yIHdoYXQgeW91IHdhbnQgdG8gYWNoaWV2ZT8gQXQgdGhlIG1v
bWVudCB0aGUgYWxpZ25tZW50IGNoZWNrcw0KPiBhcmUgZG9uZSBiZWZvcmUgZmlyc3QgdXNlLCB3
aGljaCByZXF1aXJlcyB0aGVzZSB2YWx1ZXMgdG8gYmUgYWxpZ25lZC4gRkRUIHByb2Nlc3Npbmcg
cGFydCBkb2VzIG5vdCBuZWVkIGl0Lg0KDQpUaGF04oCZcyB0cnVlLCBidXQgaXQgd291bGQgc2Vw
YXJhdGUgYmV0dGVyIHRoZSBwYXJzaW5nIHBhcnQsIGluIHRoZSBlbmQgd2hhdCBpcyB0aGUgcG9p
bnQgb2YgZmFpbGluZyBsYXRlciBpZiwgZm9yIGV4YW1wbGUsDQpzb21lIHZhbHVlIGFyZSBwYXNz
ZWQgYnV0IG5vdCBhbGlnbmVkPyANCg0KPiANCj4+IA0KPj4gU28gY3JlYXRlIGEgbmV3IGZ1bmN0
aW9uIGZpbmRfc2htKCkgd2hpY2ggdGFrZXMgYSAnc3RydWN0IHNoYXJlZF9tZW1pbmZvJw0KPiBD
YW4gd2UgbmFtZSBpdCBmaW5kX3NobV9iYW5rKCkgb3IgZmluZF9zaG1fYmFua19ieV9pZCgpPw0K
PiBJIGFncmVlIHRoYXQgaXQncyBiZXR0ZXIgdG8gdXNlIGEgdW5pcXVlIElEIHJhdGhlciB0aGFu
IG1hdGNoaW5nIGJ5IGFkZHJlc3Mvc2l6ZQ0KDQpZZXMgZWl0aGVyIG5hbWVzIGFyZSBnb29kIGZv
ciBtZSwgSSB3b3VsZCB1c2UgZmluZF9zaG1fYmFua19ieV9pZA0KDQo+IA0KPj4gc3RydWN0dXJl
IGFuZCB0aGUgc2hhcmVkIG1lbW9yeSBJRCwgdG8gbG9vayBmb3IgYSBiYW5rIHdpdGggYSBtYXRj
aGluZyBJRCwNCj4+IHRha2UgdGhlIHBoeXNpY2FsIGhvc3QgYWRkcmVzcyBhbmQgc2l6ZSBmcm9t
IHRoZSBiYW5rLCBwYXNzIHRoZSBiYW5rIHRvDQo+PiBhc3NpZ25fc2hhcmVkX21lbW9yeSgpIHJl
bW92aW5nIHRoZSBub3cgdW5uZWNlc3NhcnkgYXJndW1lbnRzIGFuZCBmaW5hbGx5DQo+PiByZW1v
dmUgdGhlIGFjcXVpcmVfbnJfYm9ycm93ZXJfZG9tYWluKCkgZnVuY3Rpb24gc2luY2Ugbm93IHRo
ZSBpbmZvcm1hdGlvbg0KPj4gY2FuIGJlIGV4dHJhY3RlZCBmcm9tIHRoZSBwYXNzZWQgYmFuay4N
Cj4+IE1vdmUgdGhlICJ4ZW4sc2htLWlkIiBwYXJzaW5nIGVhcmx5IGluIHByb2Nlc3Nfc2htIHRv
IGJhaWwgb3V0IHF1aWNrbHkgaW4NCj4+IGNhc2Ugb2YgZXJyb3JzICh1bmxpa2VseSksIGFzIHNh
aWQgYWJvdmUsIG1vdmUgdGhlIGNoZWNrcyBvbiBhbGlnbm1lbnQNCj4+IHRvIHByb2Nlc3Nfc2ht
X25vZGUuDQo+PiANCj4+IERyYXdiYWNrIG9mIHRoaXMgY2hhbmdlIGlzIHRoYXQgbm93IHRoZSBi
b290aW5mbyBhcmUgdXNlZCBhbHNvIHdoZW4gdGhlDQo+PiBiYW5rIGRvZXNuJ3QgbmVlZCB0byBi
ZSBhbGxvY2F0ZWQsIGhvd2V2ZXIgaXQgd2lsbCBiZSBjb252aW5pZW50IGxhdGVyDQo+PiB0byB1
c2UgaXQgYXMgYW4gYXJndW1lbnQgZm9yIGFzc2lnbl9zaGFyZWRfbWVtb3J5IHdoZW4gZGVhbGlu
ZyB3aXRoDQo+PiB0aGUgdXNlIGNhc2Ugd2hlcmUgdGhlIEhvc3QgcGh5c2ljYWwgYWRkcmVzcyBp
cyBub3Qgc3VwcGxpZWQgYnkgdGhlIHVzZXIuDQo+PiANCj4+IFNpZ25lZC1vZmYtYnk6IEx1Y2Eg
RmFuY2VsbHUgPGx1Y2EuZmFuY2VsbHVAYXJtLmNvbT4NCj4+IC0tLQ0KPj4geGVuL2FyY2gvYXJt
L3N0YXRpYy1zaG1lbS5jIHwgMTA1ICsrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLS0t
LQ0KPj4gMSBmaWxlIGNoYW5nZWQsIDU4IGluc2VydGlvbnMoKyksIDQ3IGRlbGV0aW9ucygtKQ0K
Pj4gDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3N0YXRpYy1zaG1lbS5jIGIveGVuL2Fy
Y2gvYXJtL3N0YXRpYy1zaG1lbS5jDQo+PiBpbmRleCAwOWY0NzRlYzYwNTAuLmY2Y2Y3NGU1OGE4
MyAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9zdGF0aWMtc2htZW0uYw0KPj4gKysrIGIv
eGVuL2FyY2gvYXJtL3N0YXRpYy1zaG1lbS5jDQo+PiBAQCAtMTksMjkgKzE5LDI0IEBAIHN0YXRp
YyB2b2lkIF9faW5pdCBfX21heWJlX3VudXNlZCBidWlsZF9hc3NlcnRpb25zKHZvaWQpDQo+PiAg
ICAgICAgICAgICAgICAgIG9mZnNldG9mKHN0cnVjdCBzaGFyZWRfbWVtaW5mbywgYmFuaykpKTsN
Cj4+IH0NCj4+IA0KPj4gLXN0YXRpYyBpbnQgX19pbml0IGFjcXVpcmVfbnJfYm9ycm93ZXJfZG9t
YWluKHN0cnVjdCBkb21haW4gKmQsDQo+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcGFkZHJfdCBwYmFzZSwgcGFkZHJfdCBwc2l6ZSwNCj4+IC0gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nICpucl9i
b3Jyb3dlcnMpDQo+PiArc3RhdGljIGNvbnN0IHN0cnVjdCBtZW1iYW5rIF9faW5pdCAqZmluZF9z
aG0oY29uc3Qgc3RydWN0IG1lbWJhbmtzICpzaG1lbSwNCj4+ICsgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBjaGFyICpzaG1faWQpDQo+PiB7DQo+PiAt
ICAgIGNvbnN0IHN0cnVjdCBtZW1iYW5rcyAqc2htZW0gPSBib290aW5mb19nZXRfc2htZW0oKTsN
Cj4+ICAgICB1bnNpZ25lZCBpbnQgYmFuazsNCj4+IA0KPj4gLSAgICAvKiBJdGVyYXRlIHJlc2Vy
dmVkIG1lbW9yeSB0byBmaW5kIHJlcXVlc3RlZCBzaG0gYmFuay4gKi8NCj4+ICsgICAgQlVHX09O
KCFzaG1lbSB8fCAhc2htX2lkKTsNCj4gSXMgaXQgcmVhbGx5IG5lY2Vzc2FyeT8gRm9yIGV4YW1w
bGUsIGJlZm9yZSBjYWxsaW5nIGZpbmRfc2htKCksIHN0cmxlbiBpcyB1c2VkIG9uIHNobV9pZA0K
DQpTbywgSSBndWVzcyBJIGRpZCB0aGF0IHRvIGhhdmUgbW9yZSByb2J1c3QgY29kZSwgaW4gY2Fz
ZSBzb21lb25lIGNoYW5nZXMgdGhlIGNvZGUgaW4gdGhlDQpmdXR1cmUgYW5kIHBlcmhhcHMgcmVt
b3ZlcyBzb21ldGhpbmcgd2UgcmVseSBvbi4gSWYgeW91IG9iamVjdCB0byB0aGVtIEkgd2lsbCBy
ZW1vdmUgdGhvdWdoLA0KaGVyZSBhbmQgdGhlIG90aGVyIHJlbGF0ZWQgcG9pbnRzIGJlbG93Lg0K
DQo+IA0KPj4gKw0KPj4gICAgIGZvciAoIGJhbmsgPSAwIDsgYmFuayA8IHNobWVtLT5ucl9iYW5r
czsgYmFuaysrICkNCj4+ICAgICB7DQo+PiAtICAgICAgICBwYWRkcl90IGJhbmtfc3RhcnQgPSBz
aG1lbS0+YmFua1tiYW5rXS5zdGFydDsNCj4+IC0gICAgICAgIHBhZGRyX3QgYmFua19zaXplID0g
c2htZW0tPmJhbmtbYmFua10uc2l6ZTsNCj4+IC0NCj4+IC0gICAgICAgIGlmICggKHBiYXNlID09
IGJhbmtfc3RhcnQpICYmIChwc2l6ZSA9PSBiYW5rX3NpemUpICkNCj4+ICsgICAgICAgIGlmICgg
c3RybmNtcChzaG1faWQsIHNobWVtLT5iYW5rW2JhbmtdLnNobWVtX2V4dHJhLT5zaG1faWQsDQo+
PiArICAgICAgICAgICAgICAgICAgICAgTUFYX1NITV9JRF9MRU5HVEgpID09IDAgKQ0KPiBXaHkg
bm90IHN0cmNtcD8gQUZBSUNTIGl0J3MgYmVlbiB2YWxpZGF0ZWQgbWFueSB0aW1lcyBhbHJlYWR5
DQo+IA0KPj4gICAgICAgICAgICAgYnJlYWs7DQo+PiAgICAgfQ0KPj4gDQo+PiAgICAgaWYgKCBi
YW5rID09IHNobWVtLT5ucl9iYW5rcyApDQo+PiAtICAgICAgICByZXR1cm4gLUVOT0VOVDsNCj4+
IC0NCj4+IC0gICAgKm5yX2JvcnJvd2VycyA9IHNobWVtLT5iYW5rW2JhbmtdLnNobWVtX2V4dHJh
LT5ucl9zaG1fYm9ycm93ZXJzOw0KPj4gKyAgICAgICAgcmV0dXJuIE5VTEw7DQo+PiANCj4+IC0g
ICAgcmV0dXJuIDA7DQo+PiArICAgIHJldHVybiAmc2htZW0tPmJhbmtbYmFua107DQo+PiB9DQo+
PiANCj4+IC8qDQo+PiBAQCAtMTAzLDE0ICs5OCwyMCBAQCBzdGF0aWMgbWZuX3QgX19pbml0IGFj
cXVpcmVfc2hhcmVkX21lbW9yeV9iYW5rKHN0cnVjdCBkb21haW4gKmQsDQo+PiAgICAgcmV0dXJu
IHNtZm47DQo+PiB9DQo+PiANCj4+IC1zdGF0aWMgaW50IF9faW5pdCBhc3NpZ25fc2hhcmVkX21l
bW9yeShzdHJ1Y3QgZG9tYWluICpkLA0KPj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHBhZGRyX3QgcGJhc2UsIHBhZGRyX3QgcHNpemUsDQo+PiAtICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcGFkZHJfdCBnYmFzZSkNCj4+ICtzdGF0aWMgaW50
IF9faW5pdCBhc3NpZ25fc2hhcmVkX21lbW9yeShzdHJ1Y3QgZG9tYWluICpkLCBwYWRkcl90IGdi
YXNlLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBtZW1iYW5rICpzaG1fYmFuaykNCj4+IHsNCj4+ICAgICBtZm5fdCBzbWZuOw0KPj4gICAg
IGludCByZXQgPSAwOw0KPj4gICAgIHVuc2lnbmVkIGxvbmcgbnJfcGFnZXMsIG5yX2JvcnJvd2Vy
cywgaTsNCj4+ICAgICBzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlOw0KPj4gKyAgICBwYWRkcl90IHBi
YXNlLCBwc2l6ZTsNCj4+ICsNCj4+ICsgICAgQlVHX09OKCFzaG1fYmFuayB8fCAhc2htX2Jhbmst
PnNobWVtX2V4dHJhKTsNCj4gSXMgaXQgcmVhbGx5IG5lY2Vzc2FyeT8gSXNuJ3Qgc2htX2Jhbmsg
YWxyZWFkeSB2YWxpZGF0ZWQ/IEl0J3Mgbm90IHZlcnkgY29tbW9uIHRvIGhhdmUgTlVMTCBjaGVj
a3MgaW4gaW50ZXJuYWwgZnVuY3Rpb25zLg0KPiANCg0KWy4uLl0NCg0KPj4gDQo+PiBAQCAtNDQw
LDYgKzQzMSwyNiBAQCBpbnQgX19pbml0IHByb2Nlc3Nfc2htX25vZGUoY29uc3Qgdm9pZCAqZmR0
LCBpbnQgbm9kZSwgdWludDMyX3QgYWRkcmVzc19jZWxscywNCj4+ICAgICBkZXZpY2VfdHJlZV9n
ZXRfcmVnKCZjZWxsLCBhZGRyZXNzX2NlbGxzLCBhZGRyZXNzX2NlbGxzLCAmcGFkZHIsICZnYWRk
cik7DQo+PiAgICAgc2l6ZSA9IGR0X25leHRfY2VsbChzaXplX2NlbGxzLCAmY2VsbCk7DQo+PiAN
Cj4+ICsgICAgaWYgKCAhSVNfQUxJR05FRChwYWRkciwgUEFHRV9TSVpFKSApDQo+PiArICAgIHsN
Cj4+ICsgICAgICAgIHByaW50aygiZmR0OiBwaHlzaWNhbCBhZGRyZXNzIDB4JSJQUklwYWRkciIg
aXMgbm90IHN1aXRhYmx5IGFsaWduZWQuXG4iLA0KPj4gKyAgICAgICAgICAgICAgIHBhZGRyKTsN
Cj4+ICsgICAgICAgIHJldHVybiAtRUlOVkFMOw0KPj4gKyAgICB9DQo+PiArDQo+PiArICAgIGlm
ICggIUlTX0FMSUdORUQoZ2FkZHIsIFBBR0VfU0laRSkgKQ0KPj4gKyAgICB7DQo+PiArICAgICAg
ICBwcmludGsoImZkdDogZ3Vlc3QgYWRkcmVzcyAweCUiUFJJcGFkZHIiIGlzIG5vdCBzdWl0YWJs
eSBhbGlnbmVkLlxuIiwNCj4+ICsgICAgICAgICAgICAgICBnYWRkcik7DQo+PiArICAgICAgICBy
ZXR1cm4gLUVJTlZBTDsNCj4+ICsgICAgfQ0KPj4gKw0KPj4gKyAgICBpZiAoICFJU19BTElHTkVE
KHNpemUsIFBBR0VfU0laRSkgKQ0KPiBXaGF0IHNlbnNlIGRvZXMgaXQgbWFrZSB0byBjaGVjayBm
b3Igc2l6ZSBiZWluZyBhbGlnbmVkIGJlZm9yZSBjaGVja2luZyBmb3Igc2l6ZSBiZWluZyAwPyBJ
dCB3b3VsZCBwYXNzIHRoaXMgY2hlY2suDQoNClllcywgYnV0IGluIHRoZSBlbmQgd2UgYXJlIGRv
aW5nIHRoYXQgdG8gcHJpbnQgYSBkaWZmZXJlbnQgZXJyb3IgbWVzc2FnZSwgc28gaXQgd291bGQg
cGFzcw0KZm9yIDAgYW5kIGl04oCZcyB0b3RhbGx5IGZpbmUsIGJ1dCBpbiB0aGUgZW5kIGl0IHdp
bGwgZmFpbCBhZnRlcndhcmRzLiBJIGRvbuKAmXQgc2VlIGZ1bmN0aW9uYWwgZGlzcnVwdGlvbnMN
CmhhdmluZyB0aGlzIG9uZSBiZWZvcmUgdGhlIG90aGVyLCB3aGF0IGlzIHRoZSBjb25jZXJuIGhl
cmU/DQoNCj4gDQo+PiArICAgIHsNCj4+ICsgICAgICAgIHByaW50aygiZmR0OiBzaXplIDB4JSJQ
UklwYWRkciIgaXMgbm90IHN1aXRhYmx5IGFsaWduZWRcbiIsIHNpemUpOw0KPj4gKyAgICAgICAg
cmV0dXJuIC1FSU5WQUw7DQo+PiArICAgIH0NCj4+ICsNCj4+ICAgICBpZiAoICFzaXplICkNCj4+
ICAgICB7DQo+PiAgICAgICAgIHByaW50aygiZmR0OiB0aGUgc2l6ZSBmb3Igc3RhdGljIHNoYXJl
ZCBtZW1vcnkgcmVnaW9uIGNhbiBub3QgYmUgemVyb1xuIik7DQo+PiAtLQ0KPj4gMi4zNC4xDQo+
PiANCg0KQ2hlZXJzLA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue May 07 13:45:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:45:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718130.1120683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4L8b-0003Ba-M5; Tue, 07 May 2024 13:45:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718130.1120683; Tue, 07 May 2024 13:45:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4L8b-0003BT-JF; Tue, 07 May 2024 13:45:45 +0000
Received: by outflank-mailman (input) for mailman id 718130;
 Tue, 07 May 2024 13:45:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4L8a-000385-Lr
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:45:44 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19959997-0c78-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 15:45:43 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59cf0bda27so242386466b.0
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 06:45:43 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 wr8-20020a170907700800b00a59be52ff98sm3390524ejb.191.2024.05.07.06.45.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 06:45:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19959997-0c78-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715089543; x=1715694343; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TMHwRoPKPO1P7SvUfr7ww6qhGQBp9aeUszh+KmkrJg4=;
        b=UAWpAOH3TLz3OIaHxcTNbvhcw9JaI2xxDAfpp0rMBWKwjZfPR8Wd5dCkMdCPDR8aka
         Ivrl2/35Xbe/L3go/nLBBqbv0Np4eOFTha9onwOPRQMb5Uk0VqXqEp9qxODJU6mMxZXR
         j0wmF4lMwQehkMk1nU0G2bzTDL6L3wXkj/STo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715089543; x=1715694343;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=TMHwRoPKPO1P7SvUfr7ww6qhGQBp9aeUszh+KmkrJg4=;
        b=nqkyaPvdtvq1Yo+3j3FRL7yAhqUZ7qansijh/wbfzA331VpAPUMkDEvIo5GaCrbSul
         /PDV+lObBEdtD7oNzpfRSaurM2JbkNNkajUL+osRSFyxPZi2YOcPgltc0WDnliPdgG04
         ZXZ9wv2rF0vfjnYd1EVDJlkya6J0lGjvn//NMmCbDCo2fWSuldlnf9SpeahpIfkjOtHO
         NCLnTnc+rK1di3nLYAmbmar/rVXx/1upqYa15R+ZLyDUdWYKQPSGqQwUtixcxZ5l4r9u
         0QNXK/TAt1/ls/0Git0Mi+E5jO9ekLhfL/hNXg4xfdDIa2w+dxXWZRHBR137UPvdjwRD
         9rbw==
X-Gm-Message-State: AOJu0YygT9klCURrHCj3y80R+jK8ub4K8PI6KTn1bxdO0W8eYkBliEmB
	dINIhIwQA4Hv2fHnB2Pv/DXVc19sn4eqBz9+fYsoGcBk5v+2Jr/UrxBuKHFcMIyvCbPN2rm/6Au
	j
X-Google-Smtp-Source: AGHT+IGrAL8dcn9M7kkpZ9qMXztsIHogfUW/seJi3mKgm1ABBsd2rethPRIB+NdBhqYeAgITRYFVoQ==
X-Received: by 2002:a50:9f66:0:b0:572:67d9:6c3b with SMTP id b93-20020a509f66000000b0057267d96c3bmr10685330edf.26.1715089543043;
        Tue, 07 May 2024 06:45:43 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to Cascade Lake
Date: Tue,  7 May 2024 14:45:40 +0100
Message-Id: <20240507134540.1714274-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Ever since Xen 4.14, there has been a latent bug with migration.

While some toolstacks can level the features properly, they don't shink
feat.max_subleaf when all features have been dropped.  This is because
we *still* have not completed the toolstack side work for full CPU Policy
objects.

As a consequence, even when properly feature levelled, VMs can't migrate
"backwards" across hardware which reduces feat.max_subleaf.  One such example
is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).

Extend the max policies feat.max_subleaf to the hightest number Xen knows
about, but leave the default policies matching the host.  This will allow VMs
with a higher feat.max_subleaf than strictly necessary to migrate in.

Eventually we'll manage to teach the toolstack how to avoid creating such VMs
in the first place, but there's still more work to do there.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v2:
 * Adjust max policies rather than the host policy.
---
 xen/arch/x86/cpu-policy.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 4b6d96276399..f7e2910c01b5 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -590,6 +590,13 @@ static void __init calculate_pv_max_policy(void)
     unsigned int i;
 
     *p = host_cpu_policy;
+
+    /*
+     * Some VMs may have a larger-than-necessary feat max_leaf.  Allow them to
+     * migrate in.
+     */
+    p->feat.max_subleaf = ARRAY_SIZE(p->feat.raw) - 1;
+
     x86_cpu_policy_to_featureset(p, fs);
 
     for ( i = 0; i < ARRAY_SIZE(fs); ++i )
@@ -630,6 +637,10 @@ static void __init calculate_pv_def_policy(void)
     unsigned int i;
 
     *p = pv_max_cpu_policy;
+
+    /* Default to the same max_subleaf as the host. */
+    p->feat.max_subleaf = host_cpu_policy.feat.max_subleaf;
+
     x86_cpu_policy_to_featureset(p, fs);
 
     for ( i = 0; i < ARRAY_SIZE(fs); ++i )
@@ -666,6 +677,13 @@ static void __init calculate_hvm_max_policy(void)
     const uint32_t *mask;
 
     *p = host_cpu_policy;
+
+    /*
+     * Some VMs may have a larger-than-necessary feat max_leaf.  Allow them to
+     * migrate in.
+     */
+    p->feat.max_subleaf = ARRAY_SIZE(p->feat.raw) - 1;
+
     x86_cpu_policy_to_featureset(p, fs);
 
     mask = hvm_hap_supported() ?
@@ -783,6 +801,10 @@ static void __init calculate_hvm_def_policy(void)
     const uint32_t *mask;
 
     *p = hvm_max_cpu_policy;
+
+    /* Default to the same max_subleaf as the host. */
+    p->feat.max_subleaf = host_cpu_policy.feat.max_subleaf;
+
     x86_cpu_policy_to_featureset(p, fs);
 
     mask = hvm_hap_supported() ?

base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 07 13:57:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:57:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718138.1120692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LJj-0005g9-Pt; Tue, 07 May 2024 13:57:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718138.1120692; Tue, 07 May 2024 13:57:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LJj-0005g2-NM; Tue, 07 May 2024 13:57:15 +0000
Received: by outflank-mailman (input) for mailman id 718138;
 Tue, 07 May 2024 13:57:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4LJj-0005fw-Eh
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:57:15 +0000
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [2607:f8b0:4864:20::72e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b44d26d0-0c79-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 15:57:13 +0200 (CEST)
Received: by mail-qk1-x72e.google.com with SMTP id
 af79cd13be357-78f049ddd7dso260505385a.1
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 06:57:13 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a6-20020a0cc586000000b006a0cf4808dfsm4700891qvj.45.2024.05.07.06.57.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 06:57:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b44d26d0-0c79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715090232; x=1715695032; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=z2A08Jr20kBVPb86rfVv1HDi3tAUKWnBkwp6nOuq83I=;
        b=mNk/J3d1w71cjjgpq/0VJv4ir26did6TZK8vGEwvdqtgDib8kX/zYH6TMjiFH4Vnv/
         Y6HIFEPNOjyYDyfZ02OdEHNYj445l5FRuBlY3JRjW2Sy2tR31GSDx9UpMZjsQC9l9TcA
         qwxLllW6dyHpOGxa7Fz3g6qEdEsnCiDsJRU2I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715090232; x=1715695032;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=z2A08Jr20kBVPb86rfVv1HDi3tAUKWnBkwp6nOuq83I=;
        b=Sbpo4qcyv4zIwEtb01T7+fmAUFkUfQNg7zPgHI+TtFZjvTqd7vWMdgnJ4x2ii7680U
         PfFnKgFpAlqVA4qHrWu+BgcUD63QJFJufZawlBc8S/fUu6V4onTy8nlxKt/I7ly874BF
         41qG/UsSSri7p0gEq2Dp3aI225ZHG1kA27KlpgwNI6two9/z6AjO6Cybuk49Rn4PpDE1
         SKrGv4A1S2WmI9xbeJVXQXcfpmq4Y89lvE/EdLzq1Gf1KhBML3DzmusHFTuNAU50Z55E
         FFhYpEnHF3sRR4ayRk5xOIJbDk+cJk5I1r71zD3XjxP5c0l1Q0YV4KITU1T1jTeY8vs0
         rfPA==
X-Forwarded-Encrypted: i=1; AJvYcCX6DNK58h7fTVp+iPjlxqMA8FO9KOaJhLmp3mJOIN8UfjZ3ZKKB+bjItCLx7zTVjr9qrI7VCHPOfMvj7qtZa0aYdwks9+7lKVcpwMcb2p8=
X-Gm-Message-State: AOJu0Yzb/qgp5zzQJmmFrqFWiPtTTWUXBiYKdZYaG7707MwonIWlOS3/
	Z2Y4WYEcEgyVaJqj8EMRjSjlmRVr26cxg/px2YtsWss2K2wYHL0v14TUSv2k73c=
X-Google-Smtp-Source: AGHT+IFcJuvYTFSI+zDsF1MoUWcvxSd5HlkYQG7GeSf4Qz7eqqiv4oOMLXO6CXMsVB4ZbbfOXjGzXQ==
X-Received: by 2002:a05:6214:2683:b0:6a0:69ef:a264 with SMTP id gm3-20020a056214268300b006a069efa264mr19311294qvb.23.1715090232283;
        Tue, 07 May 2024 06:57:12 -0700 (PDT)
Message-ID: <9a2018c6-4efb-4bfe-b90f-531a072f0ef8@citrix.com>
Date: Tue, 7 May 2024 14:57:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net] xen-netfront: Add missing skb_mark_for_recycle
To: cve@kernel.org
Cc: Jesper Dangaard Brouer <hawk@kernel.org>, netdev@vger.kernel.org,
 Ilias Apalodimas <ilias.apalodimas@linaro.org>, wei.liu@kernel.org,
 paul@xen.org, Jakub Kicinski <kuba@kernel.org>, kirjanov@gmail.com,
 dkirjanov@suse.de, kernel-team@cloudflare.com, security@xenproject.org,
 xen-devel@lists.xenproject.org, George Dunlap <dunlapg@umich.edu>,
 Greg KH <gregkh@linuxfoundation.org>
References: <171154167446.2671062.9127105384591237363.stgit@firesoul>
 <CALUcmU=xOR1j9Asdv0Ny7x=o4Ckz80mDjbuEnJC0Z_Aepu0Zzw@mail.gmail.com>
 <CALUcmUkvpnq+CKSCn=cuAfxXOGU22fkBx4QD4u2nZYGM16DD6A@mail.gmail.com>
 <CALUcmUn0__izGAS-8gDL2h2Ceg9mdkFnLmdOgvAfO7sqxXK1-Q@mail.gmail.com>
 <CAFLBxZaLKGgrZRUDMQ+kCAYKD7ypzsjO55mWvkZHtMTBxdw51A@mail.gmail.com>
 <2024042544-jockstrap-cycle-ed93@gregkh>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <2024042544-jockstrap-cycle-ed93@gregkh>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hello,

Please could we request a CVE for "xen-netfront: Add missing
skb_mark_for_recycle" which is 037965402a010898d34f4e35327d22c0a95cd51f
in Linus' tree.

This is a kernel memory leak trigger-able from unprivileged userspace.

I can't see any evidence of this fix having been assigned a CVE thus far
on the linux-cve-annouce mailing list.

Thanks,

~Andrew


On 25/04/2024 4:13 pm, Greg KH wrote:
> On Thu, Apr 25, 2024 at 02:39:38PM +0100, George Dunlap wrote:
>> Greg,
>>
>> We're issuing an XSA for this; can you issue a CVE?
> To ask for a cve, please contact cve@kernel.org as per our
> documentation.  Please provide the git id of the commit you wish to have
> the cve assigned to.
>
> thanks,
>
> greg k-h



From xen-devel-bounces@lists.xenproject.org Tue May 07 13:58:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 13:58:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718140.1120703 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LKX-0006AD-3G; Tue, 07 May 2024 13:58:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718140.1120703; Tue, 07 May 2024 13:58:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LKW-0006A6-WE; Tue, 07 May 2024 13:58:05 +0000
Received: by outflank-mailman (input) for mailman id 718140;
 Tue, 07 May 2024 13:58:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6rSc=MK=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4LKW-00067T-DM
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 13:58:04 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 (mail-am7eur03on20600.outbound.protection.outlook.com
 [2a01:111:f403:260e::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d241ec5b-0c79-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 15:58:03 +0200 (CEST)
Received: from DUZPR01CA0028.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:46b::7) by AS2PR08MB10055.eurprd08.prod.outlook.com
 (2603:10a6:20b:645::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 13:57:57 +0000
Received: from DU2PEPF00028D01.eurprd03.prod.outlook.com
 (2603:10a6:10:46b:cafe::97) by DUZPR01CA0028.outlook.office365.com
 (2603:10a6:10:46b::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend
 Transport; Tue, 7 May 2024 13:57:57 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028D01.mail.protection.outlook.com (10.167.242.185) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Tue, 7 May 2024 13:57:57 +0000
Received: ("Tessian outbound af213ececc3d:v315");
 Tue, 07 May 2024 13:57:57 +0000
Received: from 127378893e86.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7C5AB285-C85E-437C-A335-3B533FD96C88.1; 
 Tue, 07 May 2024 13:57:51 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 127378893e86.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 May 2024 13:57:51 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AM9PR08MB6145.eurprd08.prod.outlook.com (2603:10a6:20b:2dd::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Tue, 7 May
 2024 13:57:45 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Tue, 7 May 2024
 13:57:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d241ec5b-0c79-11ef-909c-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=h0I1I5DPCO5S9fLGKY5Fl08PlnMxuoblxKmr/3iGAgjXgJFENL99asHSnqIXUAZqVRWq6w9khNgzVFswB2p3kniEJ/TGRKjjlDZYPx7AvwW/OSB+wUKhulJg/NQxUcOZ/ANqj5LqQcqsVlu62kSH8/LjvliqBNFupvDW55hPuxneinNhwtcU5AQOoX4SUeiinj4dEa1ju1a6AwOvaFtBIPh0KfBwddOR2VGwPK0EfGhEcDMqad7yZ6geM5aywyJLSbfmyFSrCkQSNN+DK6OtZu07cY48Lvmr1Dng61JAs5vT+DIvd/yDt4ygqO9qmtz9HSD8Qe0523Pdi6VErunefA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Qj95v0F4jEBZJBlEeYiyEY3/PdOwE+UuPuzRIxR02Zc=;
 b=bF3iPwx16wtGib6F0x6KTkt7mW48jvHbNHLLlmPDKOoSqMeHbdFwzM4wAcw6NxhcXOwKKsSabrlAbO1yqI01QsT5oiP/0OpWQGO9evqvGaS2a0Jh2bPrV9jkGlw1XYhe5U4G2Zwj7g8OcMFfFJC5emYs8gMxlDiv48PflZ+AZUIlIWhkhWmEzW9B35FNNU5s1Yot+HRS+vhPwjAGdwDLY91u5MIrUUOCcIGogw35v4557nUQ6GjfumUnRcNkntlRZuUot86Js0ns62SlrJsjytQwpID+D1XBjb4ojeYL89KwWmYnoZZj/MBJvysD+inKzXNld1yaJP0ePGMsa/bMeQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qj95v0F4jEBZJBlEeYiyEY3/PdOwE+UuPuzRIxR02Zc=;
 b=BRX+Ix4CuzZRtyKVdE6aSo2SO4ugZPKlz+mSUXdw5dfeW8vPTbzKptYHgcbrpWF8MhPHwNS990DBzmijElioBcWUHB/X7PyzvIFcIES2YTleHBuSyu69/WOqc2v1CQiAMH6XsJcU1meew8OWKQ/86oUMT3lg+c9kgDCXBZ9X9Wk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0db1e7dbc6cf367c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xgf96QGKJCQD+b3sow/to73uTTq4selMrsbLRjnDQ2LgFEU5RH/cniGgrQhEOy+DC4DUNsQAXVixdkMpMaYRBTz+mf49Byq9vctbMtmQcFR4/CxnD5bPG6CAo9ulwjgJXEJIsCG5PtUP0JFR2oiQvVdL1/LdJMPD5BcAst3zSjMeoLhC66X9uzAC1D49MzBqcn1OmBrTKg1daQymnexDIeAHmQK0OECWAaoGxfGKIy621mfgKrzFl6rF4jXOThdP5jdM+81dlvz1JYmiQLIh3etTAZgHBxmeRKHEOw56Hiah9oEuvKV43Lewn4/OaGsfHE7Wt2NQd5t5ilZEFoEZGQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Qj95v0F4jEBZJBlEeYiyEY3/PdOwE+UuPuzRIxR02Zc=;
 b=X8lfrAE0ck3Vbtqix2uhsOZN8DbuFMl9D+lP1qetVLIJRdqV0w8OlLyh055ae1uLOzmJjhqAkL1OACC0iTQ8kv+rOciUX5ouNaKWpKHKul3LvxTeiY7LR0eP7y0PIGrw5MwV4jxsSnNJyguroGYd8TFxJTMTk1sOtXWT2ds0GzDYqLl9aV3pilumyye4ABO36ZmysFbb0IgLLhmpBqzQi9Zlh8ylzdt4rcIR+Gh1A1NEFomWJQCYhT1x342CGzNM/GtaBRhf78KpzYhLbi50rwi8PJUMY4Eu139LYCc71TQb4VJzfrjbY9gAkml3V/Qsog5tBdSvSWA7mLC/KJXcrg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qj95v0F4jEBZJBlEeYiyEY3/PdOwE+UuPuzRIxR02Zc=;
 b=BRX+Ix4CuzZRtyKVdE6aSo2SO4ugZPKlz+mSUXdw5dfeW8vPTbzKptYHgcbrpWF8MhPHwNS990DBzmijElioBcWUHB/X7PyzvIFcIES2YTleHBuSyu69/WOqc2v1CQiAMH6XsJcU1meew8OWKQ/86oUMT3lg+c9kgDCXBZ9X9Wk=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Topic: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Index: AQHalVfkdpwTrVVpI0ikGFOyHt7ILLGKSvyAgAGXWoA=
Date: Tue, 7 May 2024 13:57:45 +0000
Message-ID: <E429579D-8698-4A92-BD9A-CCD686D1720B@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-3-luca.fancellu@arm.com>
 <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
In-Reply-To: <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AM9PR08MB6145:EE_|DU2PEPF00028D01:EE_|AS2PR08MB10055:EE_
X-MS-Office365-Filtering-Correlation-Id: fa1fc7b6-0572-4485-155a-08dc6e9db319
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|366007|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?eWpmZ2JrUmxuVWlZWlFjSGNJWWNFVmVDN0Z4S0Q4SStFWFpsMytFaGhXa0Fu?=
 =?utf-8?B?dC90bncvU05VNld2SFo0MEpuclBid0VxVHBxVUhLemNKY1o3ZGVZQ2ltYkpM?=
 =?utf-8?B?dnROWkF1ZWk0d0RPa2FvcFlwblFzN0l3SGF6ZnVCcldDbzltNWZYdHpkWjZB?=
 =?utf-8?B?d0VENzhpM1lYNGdOb2FKSEk4MTF4MjNLdDRnZlRodms5MzEvU0xyemlVTytW?=
 =?utf-8?B?MkRsdytjNU1MdGdMVU5EVUJ2UEs1QmV6VFRGTGl3RlRBS1k3U0JTekduVkFP?=
 =?utf-8?B?QlFvN2dwMjRZYU9SV0c5blB3R0dUS1h3ZWZSdUh0MTN4WS9TbXJhSGMra1BL?=
 =?utf-8?B?dmtzaUZqUGV4TlhIMWcyZEVTTk5hc0VOeUJ6czM2UTQvUjk0d2g1eTE4R21n?=
 =?utf-8?B?THBLcHRSMndEYVlGRVF6c01LQUY4VWZ1dTJUYTVuOG9hdlludy9zNDhIaWZs?=
 =?utf-8?B?bmIrR3ZpVU9TUWErWUNLczVZQkt0dklsNXpGcjBSZEJvZWlPYkpKNWpoR2hk?=
 =?utf-8?B?ZXF3TXczeHZMVnNYa1BJeGxDdUp0SDhDakdmeTExeDdaa0ptUDRhR3pSbFk2?=
 =?utf-8?B?Ymg2ZWsrZ3JIeC95eG9kYXVRYUZnWEdpN2FYb1VmWDNKb3VUL3dUVFJyWGZt?=
 =?utf-8?B?OGNWaDNsRk55NVpoanZOQkUzSnQveHA1YXZRZXhJM0x0c0c0TkY0SGdORnJv?=
 =?utf-8?B?ZHd3ZHJuMmlNRDlhYjVJS3BhZWJWcSs4MGJ5WEhXcDFqMDYzYUdPaDVQUGNs?=
 =?utf-8?B?RVVSWGpyQVZTZXV4YnBPRyswY0laU242V1IrdmZlMW5nVE1KaHF1SUt6YmRQ?=
 =?utf-8?B?R2d6RzRkYWY4TlpJaFQwSnJBZ3lubEZ0RmVFSG03VGRsRkN6aWtWWWFncGo2?=
 =?utf-8?B?eGRTRXhqeVMvVjdGRUh5cnF2TXVFdFp4WDF0MHBxL2kvZnBLQTVQMkFTR3Rl?=
 =?utf-8?B?VXA5Rk04Ui9QVUE3dzZsTmV6b0dIZ0JWNm9HNDBHMGpsNmVvajZyWDlqd0dm?=
 =?utf-8?B?cFd4NEVkdDB6T0RTd3hhaUY2VkoxVmlGWnYyMzl5dlRnRlMweEtRQUFvc0lM?=
 =?utf-8?B?RUEyNDFTbEFkRktkMXU4Z0RJOEZMVTgwRTk2UmRUU3VkRnArR2RpY3hjeWFL?=
 =?utf-8?B?ZEFSSXh5ZEM2cnNUeHRVRlN0R2lHL29XN0JWYXY1aERSRUhhUGJ2Q1U2ZXJH?=
 =?utf-8?B?Z0dJNGpLbk9zdnVBSWhGZGg1MXhUVVR5WE1QbXdWNzRPd2N6aWNRa0JZUE0x?=
 =?utf-8?B?NzJQcDVRZ1ZGZGRTUlM1SE9iQnBEUHdGSkZBSFloWWs1Rkd2SGxaOEYzNXZW?=
 =?utf-8?B?eDJaRlU2bVVpdGFoUDZKc0NOeVd6Q1ovRlRLUG1RTHY3Vkwva1NkQkVjR0kw?=
 =?utf-8?B?amF1QkRHcXNmWkF0RDgyVnVrL1FRcE5rcTJiRVJuLyswd25XN0Z2L3Mxb0VP?=
 =?utf-8?B?OEhYR01JdXU5T2VUL1daeW1tK0UzWmhObUQyV0dHVG8zelBkRDk5Q3gyMDYw?=
 =?utf-8?B?cGkwaTdacXhwV3VkOVFpVWZSbFhpSW9ublMydG1wNkFYL2Q3V0lVT3g5TUZt?=
 =?utf-8?B?NEQwbEUyTm56eU9mS1VkZmRrOVFBdXNSZW11SjNIMGlwdm5xcHlCTlpSTkpq?=
 =?utf-8?B?UTdlUW5RSlovZ29GUDJFRWQwWXdZSHFJYU01cWFmRGlGdWZYUDBEZkpDWWwz?=
 =?utf-8?B?b04vV2llY2d3NS94ekV6eVZUcWVRcEFwYmIzS3R0dUd1aElvN0daaXdodnBV?=
 =?utf-8?B?anp6TVlyNXVQNlZMRnVSclZkMUZoYlNHdFNOOW5oK0hjSkdkMXFRak12S3di?=
 =?utf-8?B?MGJGMWtGVWx4cjZINWdLdz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <D5904377BF04B848AF02081B4EF97252@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6145
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028D01.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2b52b9e2-9447-42aa-713c-08dc6e9dac46
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|376005|82310400017|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NWl0SmI2M0gyaVFLNDdTdnNwU1M5MjU2UXlaaExLRko0dUFvcmRUQ1ZZblpw?=
 =?utf-8?B?Q3dnRk1sR1Z1ZytabEV0OVp6UkdaRGp0QWtkVWhqNmh4dlFmVGl3M201L3Rq?=
 =?utf-8?B?U2U2OXRscmw5b3dVSElxSGNLOW1pK0psaFpySDU3RTdtNTk5Y2pPQUtDRUh4?=
 =?utf-8?B?Ty9EQ0JwZG9zdlZ5NGp6ZkQ5dkgyTE1oT3hOSnJIZjZldUttZVJCTHFPemVO?=
 =?utf-8?B?Ym0zbzdnZW4xSUZ2ZXFHU1EwV04wa05CUFRscU9ucy81ODNVanh1YmtvUllQ?=
 =?utf-8?B?VXd6b3Rib1krUEQ0UXJObk5mUzlQazBSaUF4MkVSRjRYU2I5R3RUSm9IQS9K?=
 =?utf-8?B?T1NBeE0xWVRiNWNIN2RzaGowUUhERGc3ZGoxV3BNclJ1NzlJcEVkcXJSSXpD?=
 =?utf-8?B?T0Z6eWZ2bkt5STdZUU1tVDF5NVMvSnFGSjhTL0dDMkE2Zm9OdWVxZXc2M2h6?=
 =?utf-8?B?QTMzdSsxSmFqd0VHUWVsZi9DMzFYYklEV01NRUxSYWNTbWpUcDFLOHBxa1pv?=
 =?utf-8?B?QkZrS1F5Y2Zjd0l3bDAxelc5RUpOU2F1SW5FbHROVXkvZTdZU3JpWmVEMWZV?=
 =?utf-8?B?c0lkS1V4UGExaXd1S1lLaGlhcW56ZmpxNnozZTZDWjFnTlU2MzU3b29XSXhJ?=
 =?utf-8?B?REpWeWREM0c2akJLOGpoZ1UyRXRObmlyV0N1a3lLU0MyVTJ3eE56MmlnU1Bw?=
 =?utf-8?B?SGViVU1jd253Z0lxZjlEV3A1aC9wejVlenJEejZCN0JDUm5nZHRKazZxdkRx?=
 =?utf-8?B?REk2VC8zQS83TE5ZQ05pYWFWNzdRUllQVkkxNXJaNmdJSXZQNHAycFdIZXRX?=
 =?utf-8?B?YWRoNTBUblVlcTBSWmtJZVFiVUttalJ5c0JSTTFKY2ZZTktmeWFqdWcra2xU?=
 =?utf-8?B?Wlk0QzhLZWhZSzUxS01Lc3ByVFRKb3ZrUmxJVlBoUmkxMHFzdlhMdzYvQ2Fq?=
 =?utf-8?B?dHhkQmZYUUpUM3ZxZ0hRNThIZ0NSWjhQeFhpTVBaallCZGN3ZVk5MkRlREJs?=
 =?utf-8?B?TUdjTEkvZHlsMVY5MUQ3K0ZiT0ZQSXZockFxUnFRWjFYQVdpc2duSStIelB4?=
 =?utf-8?B?Z202U0F5VmVPcnhLU1ZkV3NYd1RvZ0h1aFd6SXp0Wk14V2lMelN0b1ZyWkJE?=
 =?utf-8?B?UkgwT3dQLzYyK3kzeFBuTmpFZ2orTDFRN1kxWGV4Z1FBbzU2eklWTkY1UTc4?=
 =?utf-8?B?OEY1R2JQU0dqMjJUZjY1NWJyakNLNWJpZGdBNEVVQ0R0cVRZaFI5ZkdTKzlp?=
 =?utf-8?B?dEV1cjJONUdmNHFBUmZ4V0tUU20xTWdzaU56bCtqRWZSNWdjb045S2gyUGlY?=
 =?utf-8?B?U1FPNnIvN0ZQSjdvRUR1QnIvcEJzb29raHdQOFhyZllnSXNXTVFuSUZvNERw?=
 =?utf-8?B?VmZ0NGptSnRmSitDa2dxRGNGYlpxYVBaZXVOMWNIVGpNQlJPZnhaWUt1RnZI?=
 =?utf-8?B?dEx2dG5yeVEvZWhSYmhNTnpvWnE1Q09BaGEzYnR0L0tPaTZ6elBpMlZJTGlj?=
 =?utf-8?B?cml4SDF4VDlCejZaZzMybzVxeHVQLysvSy9KYjNZd050cFE0MjNvSVJWamxO?=
 =?utf-8?B?YkVzOE42eUliMXdCeVdOVm5ReGhkWkg0NHE2elh1Z0diMm5wT2ZHWlBTb0w2?=
 =?utf-8?B?OHU1eXZ6ZitCL3RXSVpvVmlyd3BWOFIrTE9hZ2pVcTM4OFFlbi9QSXFJeEsv?=
 =?utf-8?B?SDdYVzF6S2pxUFU4dnIvcWp4SFFDZ2xzK25ydm9UdXFyQUFNazQxK1ZCSmd4?=
 =?utf-8?B?MkV6UmI1ZERCUDFwbGJhTnluS3IrV1NUcW0wRnpGMVpDUkFHVWswTWdWOFRq?=
 =?utf-8?Q?hGoL+y/YvvsA4nXXeQKTBd996rcN9B2D+t2LY=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(376005)(82310400017)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 13:57:57.3454
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fa1fc7b6-0572-4485-155a-08dc6e9db319
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028D01.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB10055

SGkgTWljaGFsLA0KDQo+PiANCj4+ICtzdGF0aWMgaW50IF9faW5pdCBoYW5kbGVfc2hhcmVkX21l
bV9iYW5rKHN0cnVjdCBkb21haW4gKmQsIHBhZGRyX3QgZ2Jhc2UsDQo+PiArICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBib29sIG93bmVyX2RvbV9pbywNCj4+ICsgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IGNoYXIgKnJvbGVfc3Ry
LA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IG1lbWJhbmsgKnNobV9iYW5rKQ0KPj4gK3sNCj4+ICsgICAgcGFkZHJfdCBwYmFzZSwgcHNp
emU7DQo+PiArICAgIGludCByZXQ7DQo+PiArDQo+PiArICAgIEJVR19PTighc2htX2JhbmspOw0K
PiBub3QgbmVlZGVkDQo+IA0KPj4gKw0KPj4gKyAgICBwYmFzZSA9IHNobV9iYW5rLT5zdGFydDsN
Cj4+ICsgICAgcHNpemUgPSBzaG1fYmFuay0+c2l6ZTsNCj4gcGxlYXNlIGFkZCBlbXB0eSBsaW5l
IGhlcmUNCg0KV2lsbCBkbw0KPj4gDQo+PiBpbnQgX19pbml0IHByb2Nlc3Nfc2htKHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCBrZXJuZWxfaW5mbyAqa2luZm8sDQo+PiAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqbm9kZSkNCj4+IHsNCj4+IEBAIC0y
NDksMzIgKzI5MCwxMCBAQCBpbnQgX19pbml0IHByb2Nlc3Nfc2htKHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBrZXJuZWxfaW5mbyAqa2luZm8sDQo+PiAgICAgICAgIGlmICggZHRfcHJvcGVydHlf
cmVhZF9zdHJpbmcoc2htX25vZGUsICJyb2xlIiwgJnJvbGVfc3RyKSA9PSAwICkNCj4+ICAgICAg
ICAgICAgIG93bmVyX2RvbV9pbyA9IGZhbHNlOw0KPiBMb29raW5nIGF0IG93bmVyX2RvbV9pbywg
d2h5IGRvbid0IHlvdSBtb3ZlIHBhcnNpbmcgcm9sZSBhbmQgc2V0dGluZyBvd25lcl9kb21faW8g
YWNjb3JkaW5nbHkgdG8gaGFuZGxlX3NoYXJlZF9tZW1fYmFuaygpPw0KDQpJIHRoaW5rIEkgd2Fu
dGVkIHRvIGtlZXAgYWxsIGR0XyogZnVuY3Rpb25zIG9uIHRoZSBzYW1lIGxldmVsIGluc2lkZSBw
cm9jZXNzX3NobSwgb3RoZXJ3aXNlIHllcywgSSBjb3VsZA0KcGFzcyBkb3duIHNobV9ub2RlIGFu
ZCBkbyB0aGUgcmVhZGluZyBvZiByb2xlX3N0ciBpbiBoYW5kbGVfc2hhcmVkX21lbV9iYW5rLCBv
ciBJIGNvdWxkIGRlcml2ZQ0Kb3duZXJfZG9tX2lvIGZyb20gcm9sZV9zdHIgYmVpbmcgcGFzc2Vk
IG9yIG5vdCwgc29tZXRoaW5nIGxpa2U6DQoNCnJvbGVfc3RyID0gTlVMTDsNCmR0X3Byb3BlcnR5
X3JlYWRfc3RyaW5nKHNobV9ub2RlLCAicm9sZSIsICZyb2xlX3N0cikNCg0KW2luc2lkZSBoYW5k
bGVfc2hhcmVkX21lbV9iYW5rXToNCklmICggcm9sZV9zdHIgKQ0KICAgIG93bmVyX2RvbV9pbyA9
IGZhbHNlOw0KDQpBbmQgcGFzcyBvbmx5IHJvbGVfc3RyIHRvIGhhbmRsZV9zaGFyZWRfbWVtX2Jh
bmsuDQoNCklzIHRoaXMgY29tbWVudCB0byByZWR1Y2UgdGhlIG51bWJlciBvZiBwYXJhbWV0ZXJz
IHBhc3NlZD8gSSBndWVzcyBpdOKAmXMgbm90IGZvciB3aGVyZSB3ZSBjYWxsDQpkdF9wcm9wZXJ0
eV9yZWFkX3N0cmluZyBpc27igJl0IGl0Pw0KDQpDaGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:03:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:03:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718144.1120712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LPO-0007j8-Js; Tue, 07 May 2024 14:03:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718144.1120712; Tue, 07 May 2024 14:03:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LPO-0007j1-HA; Tue, 07 May 2024 14:03:06 +0000
Received: by outflank-mailman (input) for mailman id 718144;
 Tue, 07 May 2024 14:03:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r/6G=MK=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s4LPN-0007iv-Qo
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:03:05 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 85abfdc8-0c7a-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:03:04 +0200 (CEST)
Received: from BY3PR04CA0022.namprd04.prod.outlook.com (2603:10b6:a03:217::27)
 by DS0PR12MB8564.namprd12.prod.outlook.com (2603:10b6:8:167::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Tue, 7 May
 2024 14:01:58 +0000
Received: from SJ5PEPF000001CE.namprd05.prod.outlook.com
 (2603:10b6:a03:217:cafe::3e) by BY3PR04CA0022.outlook.office365.com
 (2603:10b6:a03:217::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43 via Frontend
 Transport; Tue, 7 May 2024 14:01:54 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ5PEPF000001CE.mail.protection.outlook.com (10.167.242.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 14:01:54 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 09:01:53 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 09:01:53 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 09:01:52 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85abfdc8-0c7a-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N9XS40Nziemi3exM6yV8MzXHJ4iyXhq5E6+selPat4WlKxvB+B77Rw8+Id3voaLBcTGDavxV+CSH/JQExMdULPVFCkx3GR2llLXW0giqoaH07mFBXqLgDr4Sw04uZVPI+7cm0FNQ7IMEpeLf+NcXkuCATErZ9rW+9uuEjZSoXCB4EeNz12rqxQxFW+2W64tEjxFrnO+PZ9Y3aYekFQnJECzrogl3uAR0Dzm1OBD37+KdXK5tjcov0XGtZ1ZmukSlLVk4MZypyFvaouN7MSiQz4VBhhyDhC51KCt1szUfeRi//K/RnkGKp28A9zEEv2FSYbR0gRuNKR9wq2IDgRzhnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6kNXQpdsUYcRt+L3wFhEZg3RZIrnIi2H4upecjFXPRw=;
 b=afLBeSPjNsV11Kd2WogOUqDXam2Mt/UBD/+vGBeZaN1tgdXckFxYI7CVFKfjcjLZVxHHnOXO7Jc8CbmdEEfpDlXrjVNH7ZJOlWkKzti7OFaKKw88MnIdqR4GgoiPuC22Zc6L5ifrkVIzrNGvb8XfUZTDd4VMJPGpS/6BVc7fYqfmM6Wh4A+Dm6NZyXIMIJgnSxQR5l+9oQgDmIyD+pRmIUIGezAOtMFwi/rcD6t07UcaZN85qgxdZ3cDblkily5HNOWAp/1GSMF8kTBOYIUAyE8m/f1y1M7mEjcj5balPAOXOL3dj+8+n9p4K0L9GP/+6vdNdoUHTZdEQJEeiBzCqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6kNXQpdsUYcRt+L3wFhEZg3RZIrnIi2H4upecjFXPRw=;
 b=cWzj3y3igJ4h/bLNc+dZlSzKrghWC+w6Kjt/5p7pc/k+Pp6CHKzdQgljAsI27YdtRaCr2IsDO7Svnd3sGfiDgaYL7KGTOATgfTj9aIrg6nsj4zxLLOzpU++4oGtOZHb4DhX2Z31S2+ZDgXkIw8UyK9Ojzg0zJIMItoUf4/JKtbA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <2a33e7b2-c958-4a63-a9ea-40e9940b4253@amd.com>
Date: Tue, 7 May 2024 16:01:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-2-luca.fancellu@arm.com>
 <28322f07-99fc-4428-bcf2-b8c2af04d38b@amd.com>
 <882A5390-64AC-4FB6-AA53-4CE466CF9062@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <882A5390-64AC-4FB6-AA53-4CE466CF9062@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001CE:EE_|DS0PR12MB8564:EE_
X-MS-Office365-Filtering-Correlation-Id: d80d82e4-05a5-4f7d-0aef-08dc6e9e409d
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?TjAwcklWbStjeWRUaHd2SWpGZGc3Q2ZyZHpubWxQQmdMU3FTT0xVMFBKUVBl?=
 =?utf-8?B?aE5hdDFyb1JPR2l1S2hIT3BnSlZlcnNhN0Zlb1l1K2p5OFhmTWQ4NE9oV3Vx?=
 =?utf-8?B?NUtIRC81dmtxbTdNM1FYRDBqbWJMOXFjSlZIbHJVM3YvQktLenFXRDREYmFL?=
 =?utf-8?B?RWJNMENpc3hIbTRHVk94VzlCYnpabHpzalJjbzBDNXJTUlFIcitOK251S2V1?=
 =?utf-8?B?Ymc2blpWYnFiK0FlT244UzBVWkV0dXkxVjhSNklHTC9KVTBIc0RZRGJlbVdW?=
 =?utf-8?B?TG5CTnZ2c0dmR01XSUxkYVo1bkdiSHlkS3lZMlZHZDZMTlhtWUVIdjI3bURR?=
 =?utf-8?B?YVgvRWNCcDBqb2lCN2FqRnNaMVUxOUpyNnFuOFZoaFNLd2JEYVlrVFlQVC9G?=
 =?utf-8?B?ZXFaQXE4eGdvZFA5SWhaeWRXdlFWWHZ1RnRTM1JjUHVaWFJNTHRHby9DNm8z?=
 =?utf-8?B?Q29VbTYreEVIU3VTK0duZzZjR1M0Q2lOejcvNmpDWkRSb1UyaUJMRXRkN0FJ?=
 =?utf-8?B?RnZiUFRWOEpVVVFYcmZwbE1iWmVXSlJXOFE0eUhydG5PTGpkaHNCaWRsWlI5?=
 =?utf-8?B?S1Uyb1NWV2tHZkVHQ0p3R1Btb3RVbDFsdDRHOFMydXd0eExXc0RuMHVPdTFa?=
 =?utf-8?B?Yk01emJTbENxOU5ORjZtaXpIbVdFcithY3d1UE1iYk1XWWozbGtrSGtKUUdD?=
 =?utf-8?B?TTFScHdJY1Q2c01pMzNpMTV6WGpycW1vSzZ5NGhaZFlwOEx4K2RXY2VGeDFj?=
 =?utf-8?B?dWZyMnNtbGtoQ2RLNWt3eHB4Vi9mVlhNUXloV0M0cjYydng5STBvNHBsQkR0?=
 =?utf-8?B?WUhCOFRnemN3QmtTSWV1Yk9aZ2pLUGd1Z3dSOUp5eVZLeDdXZkt6SVlBN3hC?=
 =?utf-8?B?VUtsTzRDc3RGTFAwZE8vUjdmamd1ek96bmJpVzZsTGUzOS8yYjA2aWdFbVdU?=
 =?utf-8?B?cDdwRzJORzNXQjFiRWxGM3IyWVgrdHBUaWRCbE5HaUpLaGpEbEk3OEFCV2hp?=
 =?utf-8?B?Rm9WUDltUUxoYmMrQnc1NGxhTTJNaEg1MS9heC90SGJMajZhRVEzQTV0SHZU?=
 =?utf-8?B?b2pVZFZtSjYxZkI1QmRBTC9vc2lvUlZCaG8wMXZMNk1ORmRHNStocXRudWpO?=
 =?utf-8?B?ejRZSGtZVmNkSmh3Uzhqd3Z5MHA0WXpwR01FN3QyUThaS2wwR0FtY1ZPWGhv?=
 =?utf-8?B?TWpiOEFlekg4eHFlakRqNStMQlBpSkxldzZlOVRpdEJHUHR4Ymc5SDRjSkJN?=
 =?utf-8?B?WG8yUmg3azFyZnlXMGJBM01SWDdnR2tzOGNtc2lUZDVZQ2pFRkgzOFVKeEI0?=
 =?utf-8?B?OUJqVmR3WSs1UURoWUt2dEtqTW93VnR3aTJFWElxcmhXMzZPQXlIS2FCQUw5?=
 =?utf-8?B?ZEtFSHhITnl0QUs3QVdYWmhzdWhRQ3Y5b3MxQ0hEZG8vYWN4WVpvVm9iZzNG?=
 =?utf-8?B?VXIrS09jdlFLYm9nMFE3WVg3eTZpc2g0Q0ZmUWZtRURDRUtmZTFreHR4ZDMv?=
 =?utf-8?B?c2RReURudzBuUW5RZFlQQ29LSlBtdTlJdzZDN2Y4VnhlTCtranZDclRHVkYr?=
 =?utf-8?B?NXlyeEpqeVkyWnV5bkZ2R0w1bGhvNkFIcllvSmhzaXBzQ09nbXYwVmJwd0F0?=
 =?utf-8?B?ZHQvRTBBOWpQNEpmMmxtMjBBLytDYmNoN3k4VGFZWEJESFY4SDBGRllQWVVY?=
 =?utf-8?B?Vm15akZSVHcrdVhxYytMTE1DdFUwY2N3MGlpcWdWa09lOHpyckExakROMjl0?=
 =?utf-8?B?K09qRXV6cE9ES09xUm54aGdqZDdFV0p5K29QczRtRFBvNHdLcEFQU3lyMWND?=
 =?utf-8?Q?xHOwmu2iam6ziCaiJPgkDkvCgOneDGmpEfxTk=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 14:01:54.6439
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d80d82e4-05a5-4f7d-0aef-08dc6e9e409d
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001CE.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8564



On 07/05/2024 15:44, Luca Fancellu wrote:
> 
> 
> Hi Michal,
> 
> Thanks for your review.
> 
>> On 6 May 2024, at 14:24, Michal Orzel <michal.orzel@amd.com> wrote:
>>
>> Hi Luca,
>>
>> On 23/04/2024 10:25, Luca Fancellu wrote:
>>>
>>>
>>> The current static shared memory code is using bootinfo banks when it
>>> needs to find the number of borrower, so every time assign_shared_memory
>> s/borrower/borrowers
> 
> Will fix
> 
>>
>>> is called, the bank is searched in the bootinfo.shmem structure.
>>>
>>> There is nothing wrong with it, however the bank can be used also to
>>> retrieve the start address and size and also to pass less argument to
>>> assign_shared_memory. When retrieving the information from the bootinfo
>>> bank, it's also possible to move the checks on alignment to
>>> process_shm_node in the early stages.
>> Is this change really required for what you want to achieve? At the moment the alignment checks
>> are done before first use, which requires these values to be aligned. FDT processing part does not need it.
> 
> That’s true, but it would separate better the parsing part, in the end what is the point of failing later if, for example,
> some value are passed but not aligned?
> 
>>
>>>
>>> So create a new function find_shm() which takes a 'struct shared_meminfo'
>> Can we name it find_shm_bank() or find_shm_bank_by_id()?
>> I agree that it's better to use a unique ID rather than matching by address/size
> 
> Yes either names are good for me, I would use find_shm_bank_by_id
> 
>>
>>> structure and the shared memory ID, to look for a bank with a matching ID,
>>> take the physical host address and size from the bank, pass the bank to
>>> assign_shared_memory() removing the now unnecessary arguments and finally
>>> remove the acquire_nr_borrower_domain() function since now the information
>>> can be extracted from the passed bank.
>>> Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
>>> case of errors (unlikely), as said above, move the checks on alignment
>>> to process_shm_node.
>>>
>>> Drawback of this change is that now the bootinfo are used also when the
>>> bank doesn't need to be allocated, however it will be convinient later
>>> to use it as an argument for assign_shared_memory when dealing with
>>> the use case where the Host physical address is not supplied by the user.
>>>
>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>> ---
>>> xen/arch/arm/static-shmem.c | 105 ++++++++++++++++++++----------------
>>> 1 file changed, 58 insertions(+), 47 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
>>> index 09f474ec6050..f6cf74e58a83 100644
>>> --- a/xen/arch/arm/static-shmem.c
>>> +++ b/xen/arch/arm/static-shmem.c
>>> @@ -19,29 +19,24 @@ static void __init __maybe_unused build_assertions(void)
>>>                  offsetof(struct shared_meminfo, bank)));
>>> }
>>>
>>> -static int __init acquire_nr_borrower_domain(struct domain *d,
>>> -                                             paddr_t pbase, paddr_t psize,
>>> -                                             unsigned long *nr_borrowers)
>>> +static const struct membank __init *find_shm(const struct membanks *shmem,
>>> +                                             const char *shm_id)
>>> {
>>> -    const struct membanks *shmem = bootinfo_get_shmem();
>>>     unsigned int bank;
>>>
>>> -    /* Iterate reserved memory to find requested shm bank. */
>>> +    BUG_ON(!shmem || !shm_id);
>> Is it really necessary? For example, before calling find_shm(), strlen is used on shm_id
> 
> So, I guess I did that to have more robust code, in case someone changes the code in the
> future and perhaps removes something we rely on. If you object to them I will remove though,
> here and the other related points below.
> 
>>
>>> +
>>>     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
>>>     {
>>> -        paddr_t bank_start = shmem->bank[bank].start;
>>> -        paddr_t bank_size = shmem->bank[bank].size;
>>> -
>>> -        if ( (pbase == bank_start) && (psize == bank_size) )
>>> +        if ( strncmp(shm_id, shmem->bank[bank].shmem_extra->shm_id,
>>> +                     MAX_SHM_ID_LENGTH) == 0 )
>> Why not strcmp? AFAICS it's been validated many times already
>>
>>>             break;
>>>     }
>>>
>>>     if ( bank == shmem->nr_banks )
>>> -        return -ENOENT;
>>> -
>>> -    *nr_borrowers = shmem->bank[bank].shmem_extra->nr_shm_borrowers;
>>> +        return NULL;
>>>
>>> -    return 0;
>>> +    return &shmem->bank[bank];
>>> }
>>>
>>> /*
>>> @@ -103,14 +98,20 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
>>>     return smfn;
>>> }
>>>
>>> -static int __init assign_shared_memory(struct domain *d,
>>> -                                       paddr_t pbase, paddr_t psize,
>>> -                                       paddr_t gbase)
>>> +static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
>>> +                                       const struct membank *shm_bank)
>>> {
>>>     mfn_t smfn;
>>>     int ret = 0;
>>>     unsigned long nr_pages, nr_borrowers, i;
>>>     struct page_info *page;
>>> +    paddr_t pbase, psize;
>>> +
>>> +    BUG_ON(!shm_bank || !shm_bank->shmem_extra);
>> Is it really necessary? Isn't shm_bank already validated? It's not very common to have NULL checks in internal functions.
>>
> 
> [...]
> 
>>>
>>> @@ -440,6 +431,26 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>>>     device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
>>>     size = dt_next_cell(size_cells, &cell);
>>>
>>> +    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
>>> +    {
>>> +        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
>>> +               paddr);
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
>>> +    {
>>> +        printk("fdt: guest address 0x%"PRIpaddr" is not suitably aligned.\n",
>>> +               gaddr);
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if ( !IS_ALIGNED(size, PAGE_SIZE) )
>> What sense does it make to check for size being aligned before checking for size being 0? It would pass this check.
> 
> Yes, but in the end we are doing that to print a different error message, so it would pass
> for 0 and it’s totally fine, but in the end it will fail afterwards. I don’t see functional disruptions
> having this one before the other, what is the concern here?
It does not cause the functional disruption. It is more about code readability and writing cleaner code.
It makes more sense to first check for size being 0 rather than whether it's page aligned, since the latter can
pass if former is true and thus not making much sense.

~Michal


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:08:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:08:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718153.1120723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LUn-0000Oj-Bo; Tue, 07 May 2024 14:08:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718153.1120723; Tue, 07 May 2024 14:08:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LUn-0000Oc-83; Tue, 07 May 2024 14:08:41 +0000
Received: by outflank-mailman (input) for mailman id 718153;
 Tue, 07 May 2024 14:08:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r/6G=MK=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s4LUl-0000OW-Ij
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:08:39 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4caafadb-0c7b-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:08:38 +0200 (CEST)
Received: from BL0PR05CA0018.namprd05.prod.outlook.com (2603:10b6:208:91::28)
 by CY8PR12MB7219.namprd12.prod.outlook.com (2603:10b6:930:59::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29; Tue, 7 May
 2024 14:08:29 +0000
Received: from BL6PEPF00020E62.namprd04.prod.outlook.com
 (2603:10b6:208:91:cafe::54) by BL0PR05CA0018.outlook.office365.com
 (2603:10b6:208:91::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43 via Frontend
 Transport; Tue, 7 May 2024 14:08:28 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF00020E62.mail.protection.outlook.com (10.167.249.23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Tue, 7 May 2024 14:08:28 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 09:08:17 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 09:08:16 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 09:08:15 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4caafadb-0c7b-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CDrCSjtDeQHKuhOOGU5mmOyPMNgADY2Tf75XodKp58x4DepGGIuAuVsJSuAxYH/wX69H53NA8idjDUnFj2AzF7a/6RJRv3ZExsttsvPQg0JvKPKIS/K4G5LqV6bED+BsoQGxqzTVWrAydf+8UtnlsnGhAm3CQ64t2CE5ePXc3sZKs3tUXiPBP5XL99HpEDVNgR9L40LJofKlHcG7xG6b5BzP0e2GoT/LAlFjpFukyNA0qaQggd9OLq0aI4rROchP8iRMIsnUx6y8e5ip74To9wSyetDBUBlTOtypLQa6MoZ7lgnqp5OTOYP65n0barw8XzkT2HkdF84CT0VhCDg4CA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=W5NEyJuvreuOJv1SiUQGdPNtvBALU/HTlg5HwYrgl8k=;
 b=Kj6fKt+zszLRz81J/64ZDl4V20NJO5Pl3VJQp/raFM9twvE3KmVOx+rS+eMspNOCn16SAozA7zIpZYuDxCAwknok2+IEhjHpBvin/vmIYIHS02nymnw/+kL31YMtQYtAYMZX/2/IPuR2G0FzOxdy2/75pxqAOiPDEAsZyr1IQt6+JfiMRMToBHL5krtraiZ4GG906dhUOQTZu7c/j51aJiYXqS92E3npg/5iZdLwm5oKCN6mDR7+eYoVyBgumW3oBIh4mR0zsZciHjfcXY5ThOAUt2s5xtauqH9H72cL7FxSFvu/7I4i2fUEJn3b2cfbYnIkaOK77uakDmoPcNIzDQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=W5NEyJuvreuOJv1SiUQGdPNtvBALU/HTlg5HwYrgl8k=;
 b=qN11mdBFCNU/mNmIAghx4y6jLFXfMAzyLJ8v81tFenB0hXDxbi9mCwcI+jsF8N0J5Hr9KTVkBRI+S19n81py4AaR65rMfFwevfXBRl475F1ahrOnfYQFfMwWToP7RV0UyAlq8RVHXBl1MeZcH+ZZMQBFFlDIFSB3I4omLbnLrNI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <86e0857b-32f3-4e53-b869-604f7c6c82c6@amd.com>
Date: Tue, 7 May 2024 16:08:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-3-luca.fancellu@arm.com>
 <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
 <E429579D-8698-4A92-BD9A-CCD686D1720B@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <E429579D-8698-4A92-BD9A-CCD686D1720B@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF00020E62:EE_|CY8PR12MB7219:EE_
X-MS-Office365-Filtering-Correlation-Id: 89a331fc-4531-4274-e670-08dc6e9f2b42
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?UEJNNXpoS0RkMWpXWHJmTE1XQ1VwMHA2eEZzd0ViaXQ5TGJJdlpGNW1scFg3?=
 =?utf-8?B?aGF1NjZmWnZRZUtrV2NEdFdRcU5wZkJkZVU4ZE5DclZKY3NGSCtMWHRGQnFn?=
 =?utf-8?B?czZ5ajN1d0t2eW5LcEN0NzJTNStiMEw4dEpTTjkwRUJnbUVuQjVmNWFSQnB3?=
 =?utf-8?B?STJyZjNQTDhyamN1STB5N25ObmlPMStQOGkrQnZzenBycVdhcDJZQ21xRXJa?=
 =?utf-8?B?TFkvVmRqVmFMZjJpeXdCQXJ2ZFVUeEhjU0phem9mZklxMDJtUklQamVpeVNX?=
 =?utf-8?B?VThncmZBZnpiaFNhYUljN3Y3c0lQOXh6WGpSUUtxWi9yZGhya2ZBYlNpUmt2?=
 =?utf-8?B?a0ZWc0FacEoyei9tVDhsMURMQ0ZRalpNZDhRZ1RWZXpMdUxUR1RKZjlUTDVE?=
 =?utf-8?B?YzJWMEgyVkh6RjBKbGhqdDhmK1drbTJrODhFM0djUmd2aWl0Y3hFbG83L0RV?=
 =?utf-8?B?dzkxNXFaWG9QcWd2Snk1NzNVaUpRNUJ1TzdDeXFzdy9lRk5BR3pzS2paS1l6?=
 =?utf-8?B?dkNNazFJR081VjlrWG9ESHhyT21YbFhnK2VyRXh1TW04RVJaT2tmWEFTcWFs?=
 =?utf-8?B?MjdyMTNJYWtzL2pmWmVOL0xXQ1FUY0pIVWcvUkhRUWxuNXh6NkJoY3JReE9P?=
 =?utf-8?B?N1NvV2hNa3k0U0UzK0xSNW1CWjBTTS9ZTUNMSWkycWVmalcyOGN3eVB0a25v?=
 =?utf-8?B?ZlNsc3dKTUtNT3BaOThIdW5QdVZwd1lQN1ZaU3ZVSTEySVNvblRrOExLZmVB?=
 =?utf-8?B?QlhKdEczQUZ1MjRXYVN5Y1U2VTBjN0hEYXY5ZHAwK0Q3akZpT1cyRUxhS0wy?=
 =?utf-8?B?Q3BLTFh1d1dtY1lOaldseGVBb0FaclBCWHNlaG9lSmxDeWpOQlRRVEtsYXRG?=
 =?utf-8?B?bWtmYkJxYzcxYjhydWdwRjdmc0lNbFBadzcxUzNUNUh0aVJ5VnNNS3h5ZFht?=
 =?utf-8?B?ekhncFFmRGhUZTN2NjhWNzlBUkpkRGVoRUZOd0t1aDRHcmVDVHJlbG1ZWHFx?=
 =?utf-8?B?dlIxRUNFWlJoRmdUb2dUbS9LcVRTdkVkYnBuSjlxWFZ6VFdaMzJOaGxsK2da?=
 =?utf-8?B?MHJPSlk5YUVhVDZCSEJyOUhVMUxZTXE1blRSR05KSjZXY0h6WmdEMnBCcFZt?=
 =?utf-8?B?cFpKNUhCMElqRENBZTNFa2crbUQ5RTNrM3EyeGhTNS9STVhTU0UvTVZodStw?=
 =?utf-8?B?cVU0QisyWnJmSUMyL0pjakRQUlJpeTlwY0VoS3VrdmNjWWFxMGd1aVdwaG1F?=
 =?utf-8?B?TUJqQ3FKcU4xUjZteVBQZCs3bzRKOHNXWXlWUHY3WW9aZjY1VWkxR0lLTUtz?=
 =?utf-8?B?TVU3NHdIaHp4cXJ2Tm44RGtoSUdacVYxSUk0TTVRb1AxYTVESE0wT084aXI4?=
 =?utf-8?B?WDJSTFVYZ2ZRZFdJSkhDWmZUV2JyQitUemJtc0V6ZXdHMjd1Z2lYUXZjYit4?=
 =?utf-8?B?aFoxVE5acnovTGNTOURtaHF3amJBbXE1aE1xTWErMFVqdTBmUDVoZC9NTXhQ?=
 =?utf-8?B?ZHgwTDIrcGJYRFdxeVZPZGZOQnV6UG5RVy96TU5zQW5BcjFpY2ZsTGlUaVV0?=
 =?utf-8?B?T01pUUJnT2VmbGZoQmg1c2k2cFVXa3JxbC9Qc01vU3J6SmI4eXVvc28wU1p0?=
 =?utf-8?B?VEk2eENITnpBSmo5VVhpSmg3Z2ZmamNJYUpqU3FMQWtMR3oybmdYMGRlNXpn?=
 =?utf-8?B?YXF2TC9LOHdRWDZDd2gxS2VYOTN2bUlucFRyczU1OGFOeFZrZ1NBWnNWaGpm?=
 =?utf-8?B?QVZ6cE1yUkxyVVpvN1lZd082L3QrcEFJWlliVy8zZUl3ak1scVdMNXI4Q3Bz?=
 =?utf-8?B?dXNuL3oxUWFTZ2xvaHJiT3JJY1NvWmdlUmZaUHk4a2pva0ZTMnp1S01ubDlS?=
 =?utf-8?Q?aqtUUznhUSLTe?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 14:08:28.3951
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 89a331fc-4531-4274-e670-08dc6e9f2b42
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF00020E62.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7219



On 07/05/2024 15:57, Luca Fancellu wrote:
> 
> 
> Hi Michal,
> 
>>>
>>> +static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>>> +                                         bool owner_dom_io,
>>> +                                         const char *role_str,
>>> +                                         const struct membank *shm_bank)
>>> +{
>>> +    paddr_t pbase, psize;
>>> +    int ret;
>>> +
>>> +    BUG_ON(!shm_bank);
>> not needed
>>
>>> +
>>> +    pbase = shm_bank->start;
>>> +    psize = shm_bank->size;
>> please add empty line here
> 
> Will do
>>>
>>> int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>                        const struct dt_device_node *node)
>>> {
>>> @@ -249,32 +290,10 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>         if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
>>>             owner_dom_io = false;
>> Looking at owner_dom_io, why don't you move parsing role and setting owner_dom_io accordingly to handle_shared_mem_bank()?
> 
> I think I wanted to keep all dt_* functions on the same level inside process_shm, otherwise yes, I could
> pass down shm_node and do the reading of role_str in handle_shared_mem_bank, or I could derive
> owner_dom_io from role_str being passed or not, something like:
> 
> role_str = NULL;
> dt_property_read_string(shm_node, "role", &role_str)
> 
> [inside handle_shared_mem_bank]:
> If ( role_str )
>     owner_dom_io = false;
> 
> And pass only role_str to handle_shared_mem_bank.
> 
> Is this comment to reduce the number of parameters passed? I guess it’s not for where we call
In this series as well as the previous one you limit the number of arguments passed to quite a few functions.
So naturally I would expect it to be done here as well. owner_dom_io is used only by handle_shared_mem_bank, so it makes more sense to move parsing to this
function so that it is self-contained.

~Michal


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:12:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:12:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718170.1120733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LYW-00026M-QY; Tue, 07 May 2024 14:12:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718170.1120733; Tue, 07 May 2024 14:12:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LYW-00026F-NP; Tue, 07 May 2024 14:12:32 +0000
Received: by outflank-mailman (input) for mailman id 718170;
 Tue, 07 May 2024 14:12:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6rSc=MK=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4LYV-00024W-Cl
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:12:31 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 (mail-dbaeur03on20600.outbound.protection.outlook.com
 [2a01:111:f403:260d::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d68aaf5c-0c7b-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 16:12:29 +0200 (CEST)
Received: from AM6P195CA0089.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:86::30)
 by GV1PR08MB7804.eurprd08.prod.outlook.com (2603:10a6:150:5b::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 14:12:27 +0000
Received: from AM3PEPF00009BA1.eurprd04.prod.outlook.com
 (2603:10a6:209:86:cafe::a3) by AM6P195CA0089.outlook.office365.com
 (2603:10a6:209:86::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Tue, 7 May 2024 14:12:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF00009BA1.mail.protection.outlook.com (10.167.16.26) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Tue, 7 May 2024 14:12:26 +0000
Received: ("Tessian outbound b7675f20d34d:v315");
 Tue, 07 May 2024 14:12:26 +0000
Received: from ac1c0323418b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E8036E1E-7F29-4152-86EB-DBCFB612FF0F.1; 
 Tue, 07 May 2024 14:12:15 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ac1c0323418b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 May 2024 14:12:15 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS4PR08MB8046.eurprd08.prod.outlook.com (2603:10a6:20b:586::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 14:12:12 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Tue, 7 May 2024
 14:12:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d68aaf5c-0c7b-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=UtTm13LDuSpDP+vDkOwLugl/PwQE+IGIvJtOgWU1yOBZr6HKGns6Lvp33+kNp8PdxSC4TPh9O49k58Melae0J13XjjjcWGstbNL7uuO9J7tYPIiKgEstJDVW39QSjZUOORn7hJN1Q87l4UPBdqaxQeJV/gYtap1leNMyY0DVzlsXZKlBjp3xHMvB9sasDZgCNjn9HxaDrjGqiAhiPxShcrTLeetOPLOqrck882imgH4C3guxF2vqzJfzb/+zp3M1KgB7eYBx1yXZh3R5mp7qqaU2l9k56Q9FHswkuJAUvzi+rZTIVLrTltnI+ELqSMmhiye4zbWRbBlu95d/Iy0cnA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qTbXAxOSkw4qUW8xcnWH29JWsqQgbakDLygqvGzklhI=;
 b=EqiKoRPc96OfnKjN9QTmt4RWN1kPhUboVW6tnGb/3CcwsPqsGb6qqE+oIu4P0A2Fvcb1VyF/kz3XsETWwNZksUUfQGzGt1t1GWptTPO3r2jnDKlPkHb2d2wtDTHWM2z3XdXM83/LzpZJYZ+E8GMwLq8bsqRtfDL9gSAXYlAE1ez5W1D+W7eaYvz1eGXEWlKTeysJQrC58JZWEXtN8RTJQrhfwALxRRutdRUSlFniXtfu+KEhuJV5DCQNEkc5w7M4pTzVTVjYSZqONOvm55Z38BSuu8Sn1yxQXNaobSx6DDSGNevbLgk3ZSTUVawMkMBgzUcJHSkw+wbCcl6YQFwLeA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qTbXAxOSkw4qUW8xcnWH29JWsqQgbakDLygqvGzklhI=;
 b=QAqoAZchFepHexAXTOjfV1a9ceQBVNZhNY0TOm6/4O9Sc6z6VhtX4GQGBYEKnrev8GpmbwdFWuMLqpVrdhxCqLthbY+cyLNGqT//FE3RWgSvOyjh6pgPDQxYtuEGdyOueb41/FyzVxzlcm5G3IR9om+eOxKqbbbTveMqa7f/hJs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 80355d62aeaa91a6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GAXi/9m2G41n/1Q8wnyDPSPtrr1GfU7tmcKMK4Pjwz0P1PToKOGaYFmDl5DQmiCvE8E51V6IxLp1rBv4aKGEytIfw8vsMhv3He3zML599GU+IDTicMEseqj12LTIQO8iwIZIRZATFKNRN/StCCOhlIWP3rMJCdtvJ8iuC3569zaVRQo9NigVpt0X5rIGSuoZv9tCi2mj8j1VYDmwRJsJISOZLjehFmtgqKXIU18xPbn4IDU1v+lzG0V9uCVM7wU8rQcbaGnYxurMD6FpKINdA4f8gyq0Awjb47xjw9dOJ8UzlSKrbr3EVXi1z2sT8S2l04KGF0BYJX5UQaTPGjqBNg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qTbXAxOSkw4qUW8xcnWH29JWsqQgbakDLygqvGzklhI=;
 b=hscYlr5hUkGieuYtTUbrzW99jBGVJf0llQfHmUk4oU36nyaRlf26fUCW4N+QsVzzX7+MMGBSD+PoBnYUmbKwrSadQtqerGpf24P0aYZW+hH7A7wGZ/QlZoAjXnMTBK5RSOQ9HBOsitvO7HOMIHpqfwL+KRZV/SU+SAplVsV6GI/lVujv/LC7M3WRkUklmIbUWKohXoFnF6kmLVWBlTMrft/9gKzgMcnI4JcfCOUXhcmdozSxfTjR2wOcTS20EgaqFjHmY9QDpr4yjc0TUw3teLw47vzRQsDqd5/eMbPVLyNcys6vaFdUffa7sMSJJfi61lTznJ86ucoQv2Pd4IQkUQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qTbXAxOSkw4qUW8xcnWH29JWsqQgbakDLygqvGzklhI=;
 b=QAqoAZchFepHexAXTOjfV1a9ceQBVNZhNY0TOm6/4O9Sc6z6VhtX4GQGBYEKnrev8GpmbwdFWuMLqpVrdhxCqLthbY+cyLNGqT//FE3RWgSvOyjh6pgPDQxYtuEGdyOueb41/FyzVxzlcm5G3IR9om+eOxKqbbbTveMqa7f/hJs=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Thread-Topic: [PATCH 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Thread-Index: AQHalVftWBiV4yp+FUmOns8otkWCO7GKRsoAgAGX6oCAAATTgIAAAtiA
Date: Tue, 7 May 2024 14:12:12 +0000
Message-ID: <2121ADBD-87D6-48B7-AFDF-9F79DBCFBFD1@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-2-luca.fancellu@arm.com>
 <28322f07-99fc-4428-bcf2-b8c2af04d38b@amd.com>
 <882A5390-64AC-4FB6-AA53-4CE466CF9062@arm.com>
 <2a33e7b2-c958-4a63-a9ea-40e9940b4253@amd.com>
In-Reply-To: <2a33e7b2-c958-4a63-a9ea-40e9940b4253@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS4PR08MB8046:EE_|AM3PEPF00009BA1:EE_|GV1PR08MB7804:EE_
X-MS-Office365-Filtering-Correlation-Id: 1ce6135e-0663-45ee-0e60-08dc6e9fb957
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?clFYRHZOQS9nOTJ5NUhIdXJPZXBHR2F3TjF3NU9mbEFyZlVSYzIySEw4MFds?=
 =?utf-8?B?dWpOQXFhaWpIVXQzaWRHNkFZelJhVjVOVkZBUnZSQUlmdlN1UDhod1l5WjVT?=
 =?utf-8?B?N2JDT1VTYkJuQWlNTCtIamo0bDJPWitGZi9MYlM3a0U0T3kybm5ZSG9zNHpl?=
 =?utf-8?B?d0ZEUkNXb01BaHlCREZscTJXZ2tkUWVzMFlQY0d5UjBIcXVIK0xKOU5pUDZH?=
 =?utf-8?B?ZGdiNnZLUDBXRXNlLzlHdkhQbDQrbTg1VTErYS9TdmMraVVpenlFZDY4aC9M?=
 =?utf-8?B?L2lzcDhER0EybkdGdytXS3R3dWdua1RISVdZOHhxb3RqSFBJRHlVaXdrenU2?=
 =?utf-8?B?L2lOSkV1V1ZlUkxTV0w4S0RadVBvQWdMQ3gwQzhtVU9ib3BwVUwrL3ZSMVBU?=
 =?utf-8?B?MitzMXVDSXNFb0ZxZ3ViY0VvZXBZSEZCSmFGK3NjTG1QMnNsTVVlczVmZ3RY?=
 =?utf-8?B?d3JoUk1PN056OXorU3paVmpid3BZTk5pbzVjTjM2ZXhnVzI0NTBuOHUxeG02?=
 =?utf-8?B?dFBuQmo0eitLMDBQS2YzUDdWcG83RWpwQXNFVUtieU9RYWVUUWk4UUVORnRN?=
 =?utf-8?B?OVk3bXhRY0NWdldKOVAzSm5TT2NyTDZtRldac01DRHNEU2dXL0RvY0lNNHZx?=
 =?utf-8?B?TytBdCtMczM1U2prR3FlaDEwWXhodHozajNPNy9icWJ5eG95WEp2eFVpVCtO?=
 =?utf-8?B?MStzTUNSVC9YYkVjMGRxUUQ5YTMweHFpVzJRM2MvbzI2MXJhOW9XV1BMS2ZS?=
 =?utf-8?B?THNzdzNFQndONWZZNHJOYVlYOWZUSkZXNTdpR1p6QTVwRUdSRTFHQ0FScHlW?=
 =?utf-8?B?VkdqeUEyWHoyaTRITlBTV3kxUkllbUdidTFZWGVsa1ozVW5YaThReGdoTnRV?=
 =?utf-8?B?bkx1VVl3SnhodWNFZkE0OUh0dEdjT1ozSG9ERy9xcDZ5c2J4UVZ3OERWVjlK?=
 =?utf-8?B?eG82dVpKYkIyQWsrTUl6aThjSThHcWMrWGIxbk1zcDNnd0RMTElJR1BTWWRX?=
 =?utf-8?B?aHRQSkVkSHZHcjFEaUsxVURaWXVHc3hiU2d0VE5HWHk5TFJ4SXI0VEJ0ZjNt?=
 =?utf-8?B?WVYza1B0R3lSNzdra1A0eldia2RwVDJZWmM4MFFVUVB5cVVZRkVNK3FUMEdZ?=
 =?utf-8?B?NTFDUWRoV2ZPSWx1YzZsVldTUGQvMk4veXBDdGc1YllKNjBoOHJFRWtUbnFI?=
 =?utf-8?B?bERmbHJrUzczd0lDOHNldGpUKzRRcDl4NlYrdjFscjg3MjlQRmtxa0xhWk9t?=
 =?utf-8?B?Y1VWZ0ovTmxDbTlPNExWYkw2NHJlbXpETUdTbnFCZllGbTc2T3BkUkVTWEh1?=
 =?utf-8?B?RWk5MzJkeUNIQUE4UGFmL0w2cHJ6UGVNQWxMMEZLZUJrb0s2WENYTzZWWjBp?=
 =?utf-8?B?K2poTHVNUXJBZjRoUjhnblJIV3U2Y2pPUk9kTlBsVlRrMmVESW1sSFFHSzJt?=
 =?utf-8?B?NnM1YklCbEN4czdyUENxQy82SXpUaXdXdWtkbklqS05sY2g4QUd2MEd3QnJR?=
 =?utf-8?B?bUpoZEJnNE1ieFZCR0toWXBQck9aK1JIM2gwMHNtS2VLTktwd2hhU0UxejJE?=
 =?utf-8?B?M3FIS2l5TU82eDVVQmNZby92RmRvMUJWREcwbVpIU1pNNWtjZnNMcGJvNXRp?=
 =?utf-8?B?alp1LzFkMDBGQkZpSEdnMHJUVGZ5TC9FWVU1TXV0emlWMnVCdkdiemhNL2I1?=
 =?utf-8?B?VHFOZ3BGRGNCeVlNU2pIbW1pY3N5VEpvbUdieVRTeVNkM1lmeUU4NnpucDkv?=
 =?utf-8?B?SEo0OFVzUkdtSTZ2RnZ4K2NMeVhBdWlTSjF3cDQ2c255ZkJ1a3diTDNmemNi?=
 =?utf-8?B?RVEzOFAyU2xOTWY5TEt3Zz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <9DC5C736EDCCC24394A2EE7F96FB8BF3@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8046
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF00009BA1.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1892db06-1343-4812-4c57-08dc6e9fb0ac
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|35042699013|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZnpCU0tZTWlzYVdEeGhIaWhsSThIemRGS0t6blNKVGljY3pTcFd4WnFEeUgx?=
 =?utf-8?B?SlpZK1dkU21tRFdJVEFIU0I3azRQSDFUR3pnL0R0UUhPQzdpTk5KVEg5SHY5?=
 =?utf-8?B?cForNHl0NTZDT05BYXVmb1MwYmJPWWRKWWxkbTIwTmowRnhlMkJ6NEtxbUNY?=
 =?utf-8?B?OFlIdXYwWU5YakU2a2NZVFBjcjVHakovSGxJTlkyY0hUOFBoblcwdUllRGYv?=
 =?utf-8?B?THI1SllsWHBuaVJuUC9tdTltNjJCZFR3MmdJRlQwSFNaSnZJMTV5dWZibVBS?=
 =?utf-8?B?WkEzZXBkL0tDOGZjTk51UlNYRGhVbUhielFnZlBsWlpDRkJkMVVDTGlzdHRu?=
 =?utf-8?B?WjJSTVNVOWNwaUtNZmE1WERlb3JVZXpQd3BRdGtwNUlCYzBMRUtadkJKVjBI?=
 =?utf-8?B?aktmdGxQNUhlb0ZlZkxEeEdXdlhKaHE2dHVkVGoxb2htSDNQY0lJdTZ6eTJk?=
 =?utf-8?B?NVRZRUhvbjRzN3JSNVc4RHVWWFpaaElhUHNmdmpUazRGTmpYQlJ0QzhEcTZ0?=
 =?utf-8?B?d01SRDhyUmlpU2Vja0FLK28rbTdVNGxVSytHMDJLTFhRb0hxdDl4UXF3R3Vj?=
 =?utf-8?B?aG9PN3hVd2s2L21nbTRHTVRlaC9sV0NsdjZpV0REVkE4Q0NHSzEvVVJzVG43?=
 =?utf-8?B?R3F2eHppKzlqTlR6T01jRU9zUjlqLzJnVmowZFpPK0xuZ1EzcWdvU2xmRnQ2?=
 =?utf-8?B?TXBQLzE1TDVjRGM1UnBIYnc2V3E2WUU4Q3NCL0Q3aUVpaWFQUkFjRzAyTHBJ?=
 =?utf-8?B?eEJ6YXFIUnJUZW1yS0hEanJucUlKbjlmM3MrSERhaHBjR0ttRkZOSjd3YURO?=
 =?utf-8?B?Zmx1aTVzYW1KdmhodmJqekI2YUZXMUJmOHd6eWx0akhSZzZWUnB1d1BZR3VU?=
 =?utf-8?B?dkE2QVU5M2t4U3ZlNm91VldsaFdvc2pZTkJBWjRQZTY3NjVLbVJpQjNyUDMx?=
 =?utf-8?B?ZFJQRE0zWUxMV0l1aWxNamxEdGFUVnZUTkczMzVBQVVpamROUXBBRW1UOGdY?=
 =?utf-8?B?SThXc1JMV1dNZEh4NnRNdkpMN2ZNalA0Y29VVWtwMCtJYzRhcXk1dHBLa295?=
 =?utf-8?B?Q0hWUTdSSW9YYk9jWStFeUM2eFNJdmMxVE1FK3o5ZzQ0TFp5WnpuSndxVE5w?=
 =?utf-8?B?S1R1dWhHNEs2MHdMeC9qMGJ6ZGhHa2t4b0tvWTZ5MWMvRWs0dTUzNkVMSTNM?=
 =?utf-8?B?N05VS1gxVkZrR1VzUTNOcllQbElBbXU2NmlDZEZmb2R4Y2JiNUg2RFE2dWhX?=
 =?utf-8?B?WERhWTF6WkVwelozdVlJNCtCR21UYlF1eGp0TmtvYmowcmRwbWlZMlBQTGd6?=
 =?utf-8?B?MlRYS3Nod3JOeDZZYmNqY3Noa2xnT0NHUVZQa01pK3VoQTNBemQrVlN3N3cy?=
 =?utf-8?B?L0NpSUF2S0JiNkhoZEFkdk1oblNNVDdHZ0lsbnIwWTVFNW0zZjhWTGxzOVY1?=
 =?utf-8?B?WGdmb1NaK0J3RHJVSzlzU3N0eGhMQUhkV1pObVhXYVpBY2JCUWpZRnhxZWNW?=
 =?utf-8?B?Nkh0QWV1bHVRbE1BcFE4eEVwVGVTMDZmU3BLeDdPd1hkWG1KdjJ5alRVT3I2?=
 =?utf-8?B?UjI3eXB4TDJZYUw4ZWx4aXdmRDZkN1UrZFVUSGlDZXhFWngvRmtFWFBLTk9v?=
 =?utf-8?B?K0xFZGcyV3VuM3hYaU9DOVdLK2dZakUxaDhzaVBveDYwamdSMnNaYjd2ZnNP?=
 =?utf-8?B?YVdNV2MyMUxtRzFSVW5YSVo1Z1ZnS2xsVW1CT2tPejM0MXRDQS8waEgyTjEw?=
 =?utf-8?B?UkV4YlRpMkhWWFZ1WU9PaUptYm9FT2tCZ0hhM2xFTkMxYjdpT2pOb2VqWG5u?=
 =?utf-8?Q?Dd2bNZTrgHBhUGBJbPxa9GzRw5fgLMXeHz4T0=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(35042699013)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 14:12:26.7644
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ce6135e-0663-45ee-0e60-08dc6e9fb957
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF00009BA1.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7804

SGkgTWljaGFsLA0KDQo+PiANCj4+Pj4gDQo+Pj4+IEBAIC00NDAsNiArNDMxLDI2IEBAIGludCBf
X2luaXQgcHJvY2Vzc19zaG1fbm9kZShjb25zdCB2b2lkICpmZHQsIGludCBub2RlLCB1aW50MzJf
dCBhZGRyZXNzX2NlbGxzLA0KPj4+PiAgICBkZXZpY2VfdHJlZV9nZXRfcmVnKCZjZWxsLCBhZGRy
ZXNzX2NlbGxzLCBhZGRyZXNzX2NlbGxzLCAmcGFkZHIsICZnYWRkcik7DQo+Pj4+ICAgIHNpemUg
PSBkdF9uZXh0X2NlbGwoc2l6ZV9jZWxscywgJmNlbGwpOw0KPj4+PiANCj4+Pj4gKyAgICBpZiAo
ICFJU19BTElHTkVEKHBhZGRyLCBQQUdFX1NJWkUpICkNCj4+Pj4gKyAgICB7DQo+Pj4+ICsgICAg
ICAgIHByaW50aygiZmR0OiBwaHlzaWNhbCBhZGRyZXNzIDB4JSJQUklwYWRkciIgaXMgbm90IHN1
aXRhYmx5IGFsaWduZWQuXG4iLA0KPj4+PiArICAgICAgICAgICAgICAgcGFkZHIpOw0KPj4+PiAr
ICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4+Pj4gKyAgICB9DQo+Pj4+ICsNCj4+Pj4gKyAgICBp
ZiAoICFJU19BTElHTkVEKGdhZGRyLCBQQUdFX1NJWkUpICkNCj4+Pj4gKyAgICB7DQo+Pj4+ICsg
ICAgICAgIHByaW50aygiZmR0OiBndWVzdCBhZGRyZXNzIDB4JSJQUklwYWRkciIgaXMgbm90IHN1
aXRhYmx5IGFsaWduZWQuXG4iLA0KPj4+PiArICAgICAgICAgICAgICAgZ2FkZHIpOw0KPj4+PiAr
ICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4+Pj4gKyAgICB9DQo+Pj4+ICsNCj4+Pj4gKyAgICBp
ZiAoICFJU19BTElHTkVEKHNpemUsIFBBR0VfU0laRSkgKQ0KPj4+IFdoYXQgc2Vuc2UgZG9lcyBp
dCBtYWtlIHRvIGNoZWNrIGZvciBzaXplIGJlaW5nIGFsaWduZWQgYmVmb3JlIGNoZWNraW5nIGZv
ciBzaXplIGJlaW5nIDA/IEl0IHdvdWxkIHBhc3MgdGhpcyBjaGVjay4NCj4+IA0KPj4gWWVzLCBi
dXQgaW4gdGhlIGVuZCB3ZSBhcmUgZG9pbmcgdGhhdCB0byBwcmludCBhIGRpZmZlcmVudCBlcnJv
ciBtZXNzYWdlLCBzbyBpdCB3b3VsZCBwYXNzDQo+PiBmb3IgMCBhbmQgaXTigJlzIHRvdGFsbHkg
ZmluZSwgYnV0IGluIHRoZSBlbmQgaXQgd2lsbCBmYWlsIGFmdGVyd2FyZHMuIEkgZG9u4oCZdCBz
ZWUgZnVuY3Rpb25hbCBkaXNydXB0aW9ucw0KPj4gaGF2aW5nIHRoaXMgb25lIGJlZm9yZSB0aGUg
b3RoZXIsIHdoYXQgaXMgdGhlIGNvbmNlcm4gaGVyZT8NCj4gSXQgZG9lcyBub3QgY2F1c2UgdGhl
IGZ1bmN0aW9uYWwgZGlzcnVwdGlvbi4gSXQgaXMgbW9yZSBhYm91dCBjb2RlIHJlYWRhYmlsaXR5
IGFuZCB3cml0aW5nIGNsZWFuZXIgY29kZS4NCj4gSXQgbWFrZXMgbW9yZSBzZW5zZSB0byBmaXJz
dCBjaGVjayBmb3Igc2l6ZSBiZWluZyAwIHJhdGhlciB0aGFuIHdoZXRoZXIgaXQncyBwYWdlIGFs
aWduZWQsIHNpbmNlIHRoZSBsYXR0ZXIgY2FuDQo+IHBhc3MgaWYgZm9ybWVyIGlzIHRydWUgYW5k
IHRodXMgbm90IG1ha2luZyBtdWNoIHNlbnNlLg0KDQpPayB0aGVuIEkgd2lsbCBzd2l0Y2ggdGhl
bSBhbmQgY2hlY2sgaXQgYmVpbmcgZGlmZmVyZW50IGZyb20gMCBiZWZvcmUgdGhlIGFsaWdubWVu
dCBjaGVjay4NCg0KQ2hlZXJzLA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:16:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:16:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718202.1120742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lbm-0003VJ-7W; Tue, 07 May 2024 14:15:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718202.1120742; Tue, 07 May 2024 14:15:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lbm-0003VC-4p; Tue, 07 May 2024 14:15:54 +0000
Received: by outflank-mailman (input) for mailman id 718202;
 Tue, 07 May 2024 14:15:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4Lbl-0003V6-AE
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:15:53 +0000
Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com
 [2607:f8b0:4864:20::72a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4f3379a3-0c7c-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:15:52 +0200 (CEST)
Received: by mail-qk1-x72a.google.com with SMTP id
 af79cd13be357-7928c351c6bso283076485a.0
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 07:15:52 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vk21-20020a05620a70d500b00792938d0f02sm2588423qkn.39.2024.05.07.07.15.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 07:15:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f3379a3-0c7c-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715091351; x=1715696151; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=wpVCwRfr9ubipiDlF+tgfX3af1LVpIRSlrT9PfCJ7d8=;
        b=vP0PN91C8YKrS+jPbdMyBa8uL8uQ/h+2g9u1AkAp46n0/ckqdVgj7wsbhMR7eJYCw1
         yNnUyaENe+J/T6R4vClePRMn8qwgdum7zeDrlhUS4oYmqpwnLTm9lzEbPUggWaXvErj4
         psuA9H7/jq6s4zhIMiRVN1t8o82a5jS9s9pY0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715091351; x=1715696151;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wpVCwRfr9ubipiDlF+tgfX3af1LVpIRSlrT9PfCJ7d8=;
        b=IV8CBw8x3DiE0xhIbV36NTt4j5i+4Ugg88jHELe/edab9dCbJFTa7QyCKHyn2/moNR
         /L8g7kQphCxJCDSf+srhPh6NF+IyYTIeTY24LzeJlO2LfS00r0Fe+08cuXmXzHApsbum
         FJjF/fUU4vUKRnWdXbJ/y/YMp1HPKk9x1AW6ZJZibrBFPOUK8+zqVRFwYJMRwDAagcyg
         t+WaUGestg7sPjSqWrrp3bhuuKhDOgbAJaaCvtyTISz/zAFsnVtFRE2jb1kbogoQqGEa
         CjygRTbe1X037a8T3ccXctiw2+d/iYoSf1p3McDyZlu53OH/Dmg+ufkBplQnvgy8b2AC
         Y+8Q==
X-Gm-Message-State: AOJu0YyMKnpIO2zXHrOw5Kuukvm2oXmAyGWcjcvIekuJAhk0HD6l9AIi
	HeU74sR23nbX4jO+h09rhZbvCuJxz9SYFvTWXt3K9Cjw38WqfHV/PLU/thL0pp1LwU96VdnS2YK
	j
X-Google-Smtp-Source: AGHT+IEe7o+xgwE6/oWoj19AXPdS2WYbq+F0c2bT91ca+kojtZNs/D8viPQ6PYswj3PCmWOvazXhsg==
X-Received: by 2002:a05:620a:4047:b0:790:fd04:5a1b with SMTP id i7-20020a05620a404700b00790fd045a1bmr17449643qko.9.1715091351039;
        Tue, 07 May 2024 07:15:51 -0700 (PDT)
Message-ID: <549a101f-b9b6-47fb-a782-c716f1ce49ec@citrix.com>
Date: Tue, 7 May 2024 15:15:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Demi Marie Obenour <demi@invisiblethingslab.com>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
 <ZjoRMHmL8_K9_lsL@mail-itl>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZjoRMHmL8_K9_lsL@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/05/2024 12:32 pm, Marek Marczykowski-Górecki wrote:
> On Tue, May 07, 2024 at 12:08:06PM +0100, Andrew Cooper wrote:
>> `xl devd` has been observed leaking /var/log/xldevd.log into children.
>>
>> Link: https://github.com/QubesOS/qubes-issues/issues/8292
>> Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Anthony PERARD <anthony@xenproject.org>
>> CC: Juergen Gross <jgross@suse.com>
>> CC: Demi Marie Obenour <demi@invisiblethingslab.com>
>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>
>> Also entirely speculative based on the QubesOS ticket.
>> ---
>>  tools/xl/xl_utils.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
>> index 17489d182954..060186db3a59 100644
>> --- a/tools/xl/xl_utils.c
>> +++ b/tools/xl/xl_utils.c
>> @@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pidfile)
>>          exit(-1);
>>      }
>>  
>> -    CHK_SYSCALL(logfile = open(fullname, O_WRONLY|O_CREAT|O_APPEND, 0644));
>> +    CHK_SYSCALL(logfile = open(fullname, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0644));
> This one might be not enough, as the FD gets dup2()-ed to stdout/stderr
> just outside of the context here, and then inherited by various hotplug
> script. Just adding O_CLOEXEC here means the hotplug scripts will run
> with stdout/stderr closed.

Lovely :(  Yes - this won't work.  I guess what we want instead is:

diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
index 060186db3a59..a0ce7dd7fa21 100644
--- a/tools/xl/xl_utils.c
+++ b/tools/xl/xl_utils.c
@@ -282,6 +282,7 @@ int do_daemonize(const char *name, const char *pidfile)
     dup2(logfile, 2);
 
     close(nullfd);
+    close(logfile);
 
     CHK_SYSCALL(daemon(0, 1));
 
which at least means there's not a random extra fd attached to the logfile.
>>      free(fullname);
>>      assert(logfile >= 3);
>>  
>>
>> base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
>> prerequisite-patch-id: 212e50457e9b6bdfd06a97da545a5aa7155bb919
> Which one is this? I don't see it in staging, nor in any of your
> branches on xenbits. Lore finds "tools/libxs: Open /dev/xen/xenbus fds
> as O_CLOEXEC" which I guess is correct, but I have no idea how it
> correlates it, as this hash doesn't appear anywhere in the message, nor
> its headers...

It's the libxs patch, but rebased over yesterday's push to staging. 
auto-base doesn't work quite so well in cases like this.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:16:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:16:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718204.1120753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lby-0003nj-Hz; Tue, 07 May 2024 14:16:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718204.1120753; Tue, 07 May 2024 14:16:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lby-0003nc-Ec; Tue, 07 May 2024 14:16:06 +0000
Received: by outflank-mailman (input) for mailman id 718204;
 Tue, 07 May 2024 14:16:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6rSc=MK=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4Lbw-0003mw-Sj
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:16:04 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05on20601.outbound.protection.outlook.com
 [2a01:111:f403:2613::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 55b9eb64-0c7c-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 16:16:02 +0200 (CEST)
Received: from DUZPR01CA0149.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:4bd::25) by AS2PR08MB9714.eurprd08.prod.outlook.com
 (2603:10a6:20b:606::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 14:15:58 +0000
Received: from DU2PEPF00028D02.eurprd03.prod.outlook.com
 (2603:10a6:10:4bd:cafe::7d) by DUZPR01CA0149.outlook.office365.com
 (2603:10a6:10:4bd::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29 via Frontend
 Transport; Tue, 7 May 2024 14:15:58 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028D02.mail.protection.outlook.com (10.167.242.186) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Tue, 7 May 2024 14:15:57 +0000
Received: ("Tessian outbound b7675f20d34d:v315");
 Tue, 07 May 2024 14:15:57 +0000
Received: from 70aa33a8b9ee.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4BB05AE4-046F-48DF-AE65-C94E4C18A604.1; 
 Tue, 07 May 2024 14:15:46 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 70aa33a8b9ee.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 May 2024 14:15:46 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS4PR08MB8046.eurprd08.prod.outlook.com (2603:10a6:20b:586::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Tue, 7 May
 2024 14:15:43 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Tue, 7 May 2024
 14:15:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55b9eb64-0c7c-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=SF39ZeanH4pz/8iEVER6c/q3QV9tBl9AbFg+OWwReqcpZiD0fQ1+szOHSptjOV3gSpH+hyTYB4uG7FywS5IxcQe78tvL7/syBFM+uJ2jQxXchqrv2LnxZwAuLRe6XV+uT+LkUsbLbPoCCPjV2t2nkI1pxh7BGW7ZtbiDRZzEFH1AVLarvKsZs/ZMmYzhXv1eY1T5QMXFxUrLBwIdaHKICd68ZN1kbmTcOzz+M24aRkTlHEPIRZyx89nkNc7oFiXSSTc9IdM8ufHIY67YfJ05jINa3n3H/ktJLyDLxwO5OpgEROZNBYf3cg7QGlAunk8XWOjuEuPPRrPESCl7fFl0bg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HDhY/V3mlF24qJu1Y0rzcDgTa62AsaR53Ul8cRp69tg=;
 b=ag0RZbEMvgMAHiMzWVaStb8VaKtrL81PPIVbtCrWtJeTImdTD/BoKHxAC32fN3PLksOMEUazqY+ErEJX6OtuA5P4L0hqkOBQI4O+S+yWqQzrIJ8SMkZJAFGwIOgPQAEU59EdrtsKEjtrouyPmgKRtdF0bOt+xwQGcN0KmqkI60gfPtg/7kh5U59IXd1nIhTUKO6oYzHcN46nvRZAbuI3A1mFPjEo3vSy2fUsaLwlRN9ZnRlJQhfMHV5U4VUJrhwm/ipzcvUcIjoiZYpATY3sN4cZ12a3JBhCy87nFIpi0qVFEhnTba5L0dkQZjizy468gTGSYYFzG0fS0gOBVQzUzQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HDhY/V3mlF24qJu1Y0rzcDgTa62AsaR53Ul8cRp69tg=;
 b=WFAUBK8Kq6Gp4hUR25LmUZuM8iVnpApkpXE3Pkt8MBRv+V74UEeUG0ALWwjIOxQjX0RknZDNnvMYdGQTRhCJUKEd/SJfvX9aC0hqpsJouV1V7TA+BNaTKCNrBQwNddZ/WFxRPz6f8wfWV/yfNzkID5bKTNOqBVcsgW8q/Z80zxg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0597d7055ea7f21e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Zg3LTHNgKhoqDReRD7zzTtrEdB6N6CaPtQRMjYUSoIC19v+4xpmIGmNrUlme8G3Yz/Tbrr74gCoqaTOF7+mAhiDY28GIqWOjpYRuRsiV4PeizSFjAspw7/jzWY7NjyjQZ4Hzj1WGigqcw/9N3BsQhGiNn/ppkfiyd1GOsu8OGPQs1JD8oEljpu+BIHEg1OEyZq36HBanvr+M5xvW4rHp2IeyQMRbzTZPPYjcjodbz6o3CtQuyFynj7W/FVM1gFyO+Ivyp63qKR+2SF6uwhxpqC5wqsr2LDLsv79vxXYP7eXlv4KOyHWnZ9FH6yxnMaVnrVQERQ1f5t3kJjkdDA8QFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HDhY/V3mlF24qJu1Y0rzcDgTa62AsaR53Ul8cRp69tg=;
 b=oXn34NpzNg5QVPUnH0P0jLXWHaA4iK6dDgfJLZnJF/3HfnFTcecfXGzjXFA35DkL7eLxVxWHsp88W5Na4sAzLpHqN3hi8AHi6b1MRWtbLokr3FkEZglAO43IKRRTMTJlm8q6mVmjK6MawX/ZQ44ka0SxH+uEVp9g0uTVDSZm2Hzo7VdSZ20AoUVk0DJpHj/edMPg6Nm6SScWFeUXF0ocE1ZZhwnHyCvCb2W214D/0GvlpT6QRNTUtrEMooThrdhMn8eLe5LJKWBKBWQkZbvHPEMQU4UV9Nd8JgdGkDBqAvyC3sAEemS4gRNRbhlpLbbqBL1eM1MDkRnggTwPDDtH8w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HDhY/V3mlF24qJu1Y0rzcDgTa62AsaR53Ul8cRp69tg=;
 b=WFAUBK8Kq6Gp4hUR25LmUZuM8iVnpApkpXE3Pkt8MBRv+V74UEeUG0ALWwjIOxQjX0RknZDNnvMYdGQTRhCJUKEd/SJfvX9aC0hqpsJouV1V7TA+BNaTKCNrBQwNddZ/WFxRPz6f8wfWV/yfNzkID5bKTNOqBVcsgW8q/Z80zxg=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Topic: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Index: AQHalVfkdpwTrVVpI0ikGFOyHt7ILLGKSvyAgAGXWoCAAAL7gIAAAgkA
Date: Tue, 7 May 2024 14:15:43 +0000
Message-ID: <3CCC6BD3-0A1A-4AE4-BF81-DFBE3A11364C@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-3-luca.fancellu@arm.com>
 <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
 <E429579D-8698-4A92-BD9A-CCD686D1720B@arm.com>
 <86e0857b-32f3-4e53-b869-604f7c6c82c6@amd.com>
In-Reply-To: <86e0857b-32f3-4e53-b869-604f7c6c82c6@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS4PR08MB8046:EE_|DU2PEPF00028D02:EE_|AS2PR08MB9714:EE_
X-MS-Office365-Filtering-Correlation-Id: 7598b5fc-7265-4596-d41e-08dc6ea03709
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?NDhlbDd4NVIxNmVGWFN4Rm1md3F2cVAzQUEwQWl6d3BrRldwZ2xiamhhQUNJ?=
 =?utf-8?B?TkhUQ0RwTTgvUmJ5SHBMRy9tWDBJWU5QdEdUNCtpVUZwWFlaMjhOZlNxczR3?=
 =?utf-8?B?blhKNTF2M2YyTWxRcUltbHVydElyUWExWitMNmJJNjk5ZWRLbjljRE5EWEla?=
 =?utf-8?B?UnhaK1A0QTZSUTdYYTMxa3VGZzJ1TWVUN3U3dEZlNkZmZWdxWnQveVBXTmZE?=
 =?utf-8?B?NFJma0RvNVNXWlpEcDlUMEt1UzVucmxYQko2cnJmeVo5UjFNS2dZZjR0UXJ4?=
 =?utf-8?B?dThRUnFwamlGWkVKSm5KVjlLYnhKakFxOThzL29penl3Snd5c05zMWhEdnVD?=
 =?utf-8?B?WEhkdXZBOXdwRVlxSTJMcW9ZbURFTnE5L0RpNlZBSlBZVUFDc2JENW04b0Yv?=
 =?utf-8?B?SXBTZWFxS1J6T05vWURTd096ZWNjTnhSdzhIQnlrdnlJRGdDNjFOcGJCR0xi?=
 =?utf-8?B?VmMxT01xbExubklkVVk2SGhyancxU2xzemNvb09PQUhsUU9DOGlNakdXRkNm?=
 =?utf-8?B?UzVXTjZJdmEyS1IrVVBlVlRIRm1xSnNXQXRYRTRmZHRLOEVQS1ZwcmtvRmt1?=
 =?utf-8?B?OVNhUkR1YnJZTUFYMHF2Ni92cEQvWWxGUXk1SllXQVpWejBrTWU3bW9nMk9x?=
 =?utf-8?B?Y3kycWNCWTdjMWhwRjcwWDRFZ0R6ZFpWN0tzV01nL2hNcXBxM0JFYUVkeWkr?=
 =?utf-8?B?OCtjOGVRNHpHa0UxMGVFQ1ZzdUVVdjl4bnRMVm5semVZOW9XWS9DN2lvNGFu?=
 =?utf-8?B?a2NXdDhZRHF3TXZyZ1JxTHJ4QWE1ZmhDM1p0UFQ3ZEdiamRZRlpabDJrZ1dL?=
 =?utf-8?B?YlExSWxRL0YzeWlNaFlmakdMRlo2WjZudFE3cndpd0NkNDBndXdiVXBSZ0V1?=
 =?utf-8?B?K3ZhZytmbHY0NWZhOTJWWkZqcUxEc0ZmeHArM2dCNnN1Q2krWnMvbXlsV2xQ?=
 =?utf-8?B?aWkzbmVwVnlPSnFTaGFKdUJNd2xEaVVWMkN1ckV4T1NaYTRQRm00cno2S3JV?=
 =?utf-8?B?NEQ5a0pQYXVyMjZyTVg2UEozOGV2Vk1EMFVycmJkM1N5WS9HUVJJeUN2aito?=
 =?utf-8?B?cnB1SnJmTjZPSnNYZVZIR0gzUi9sc3h2RUV1cXUrZzFhSWNSeEF5S2VFN1RM?=
 =?utf-8?B?NXMzaC84bjM1aUVveGJjeU9RYVRwTFhSN0pSOEtCQnQvdUFBMG9MZFdaTGdk?=
 =?utf-8?B?VG5lUXV5MGZsWHN1MUloVEU5VlMrS2VtU0J3bDBVbDVJWXlaZGs3WnAraDND?=
 =?utf-8?B?YUlPRFFiYWNyMFhWeUEyUjVWWXlOOTM5Slh1dTBJWW42QXpsbWFwY3pLQy9r?=
 =?utf-8?B?Q3NBOVVidGYraVZvTVZ0YlN0cHpYOFJmYXVuVHB6aklZQ1lvUU9SeWQ1b25n?=
 =?utf-8?B?VTRyRXpXdkVXQkh2a3lhMHVwQ1ZWam56S1NBTStXOVllYmRzNGNnUUNZQ0I0?=
 =?utf-8?B?MkptQUlkbGNaamptVERVZWZSQ2tSdEVsT3VNcGxqVEYyR2V2N0NxNXdwaXZP?=
 =?utf-8?B?S3F6eU1jUU9XbHdWN2RydGw5UDNNM0RoOEVmdE9tc011ZUpVTExlSmE3cTJP?=
 =?utf-8?B?WFhhdzZlUFRSQmJCN0FxbUdCV1JoRXpSTDZhR2RFYmhqd0J4ZlpBTjdBYmNs?=
 =?utf-8?B?emZvSERnMVJCdXBsTHhlZjN3MnZidEZabzBEOHQ5ZGdmVHlXNktBaElsUEVM?=
 =?utf-8?B?K2FxU0tMazVudUlLaldFMXZSTndRMjZTNXQ3ckJodjVnam11TWpLTHlZY3RB?=
 =?utf-8?B?dEk0R1FGSWZpdVZQZU16TVJ2eC9yRk5keHF6blorUlltSHV3aFlydkFacU5z?=
 =?utf-8?B?UjdIcmNteWFVbThiV1NiZz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <EFC1163C432FF8488B83842DE93B887A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8046
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028D02.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	46c2aac2-1d64-4c1a-6bfc-08dc6ea02e58
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|35042699013|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Nis0V2tuSGMrdFdaZVo3MmhITldaeVh3Q0p0cmtRWkRPNldyNDlZbG9Cb1Bh?=
 =?utf-8?B?L0NFemkrQlRDNjgxamZ2S3c5ZHVmMW9wQVdUM3VyZkxxeDZqZmdUL3ZrS3Jx?=
 =?utf-8?B?VVJwRmt5YUtZY2FtYzdUbDFXdDF6ZFgvZERzcnhLL1M0RFd0THlSVjk3TFIz?=
 =?utf-8?B?ait1SDlXWG8yU3JGdkgwbkI2bUhMTGpSYWxHSVVjOUpLRGxrSkhXc2JPQlMz?=
 =?utf-8?B?R2pna3l4NWhBTUpyQTlFKzZMNGVGL2tHaFFZR0F6UDhsZHdRMUlqbHVNZWVM?=
 =?utf-8?B?MmliY2NmSkpZQjFHY3ZGdmpOU0t5NktzN2hvdmlYc1grSWVHbmFGVnl0OXUy?=
 =?utf-8?B?ZjV5V3luL0hzYVJMRFRUMXdCSWE0bUNMVzV1Y2dKRUt6U2pVelhsSyt5ZmZr?=
 =?utf-8?B?VnpJbmNOL0ZiMnBDOFhYLzYrcWJHS1MvVUszN3UyQkZrNmJxMlJDdnk5Y1NP?=
 =?utf-8?B?S1UxTmxUZ3RRTnI2czFnaml6SStVTGVVN1BJZ2JraHlzMmtza0V4a2k0bkFl?=
 =?utf-8?B?cTU1ZjBmaFdseVFGajBRdFVoaXV1TnUvT01NSEJRMzJ6ZERuMmVqNEE3RHQz?=
 =?utf-8?B?UHBxWmM0NlBjY1JHNFhhZ0hoTkZ3YmtNb1cvYXFiblR6clg3NTYzS3VSNkYw?=
 =?utf-8?B?Q3ZORXlLRGZOU0grTzZTWFpQbm9ZRks5MXB5QzAwKzVFU3l5ZFpYWmJHRmxm?=
 =?utf-8?B?bVp1OHBxcGE1akRhM1E3M0ZQR2tCRGwvbko3VTBPQUJjbVg5NE95WkxneTRH?=
 =?utf-8?B?dWM2WmVtY0lsMkJLZms4ODR5ZGRPaC9PLzgwRGxybitqVzE3aE12ZGlBV2xl?=
 =?utf-8?B?U0pSTDZNZnlORlR5UUZSekswTkwrSnJxL0d1Rm93ZHZaWmRMeWJvL2NoR2FS?=
 =?utf-8?B?Y0IrRUlRTmdyRnNoNWJxRGdjaVcxcFphWUhaVmFkMWNTWmorWTFzbWhYOFM3?=
 =?utf-8?B?SUJMbUF2U3FNZWpNem04OXlaTElOOWVubEt1azVkZUZYREZPZktMa2hUelJv?=
 =?utf-8?B?cGVBb0Ewck5SWnlhK2I5YjRJSzNmSGZabVMwVkFiMTVsbDBhamtLRDlVaDVX?=
 =?utf-8?B?T0Vva0xtTmFsSnVJNjdaaHdkZGUzN1RsQ2Z2alFRek1CMHdnUjVwa2JqblpF?=
 =?utf-8?B?WnFPT3ppUEZuVnR1WUZKY0hSN1dXK2NkR25ESE1YZnYxV2pVTGVqdWhRQXVu?=
 =?utf-8?B?b2I5OEZEcGpUZEFiTWg1L2lwZ2xXcGtNR3hQb1RPbGtNNU52NlptTUQremZa?=
 =?utf-8?B?OStaWVhyU2d4eGdML05QK0t1Y1hxdVBGazNrRVd3bk9YK1lMRjR0bVZrSWF4?=
 =?utf-8?B?UzNkVjZja2drcCtzNnR3UXkyVFIwVk5uMm1vQ1FablYvSzdmT01QYW9KRTZJ?=
 =?utf-8?B?dkVjb3VmbUNndjQvYUhORjlRZXkxWEp6N04wMTg2c25HQktobmx0bkpTT3Iz?=
 =?utf-8?B?TFQvTVNYTTlqTHNXbEZuYkppRGFBZ0Q3MjVLSjk5aWZPeEFXSmcrOFFKdXZt?=
 =?utf-8?B?eFZJWnJLVG8zbzlCdStxOEhSSE5zK0RlYnhOOHBtdXR1TysxYStHYmYwRnN4?=
 =?utf-8?B?QWVtNkdsU0E1cDQxTStaTnhtcjdCMDBwSCtpU0xxL2VlTlJCLzlkdFByVHQw?=
 =?utf-8?B?R284NHROd1ZZdC9taVdSRkpyYXNUUzBhVDlMbjByZkV4cVZwc2pzbFRway9x?=
 =?utf-8?B?TnpXZmdiNHBlbVFKYUJsM3NUMFNCM042SzR1ZUVMN01uQjZoZk5tOVVYelJR?=
 =?utf-8?B?WWJKNytDdC9jWTloNlMyak8wN3l2enM2M09mbXhEcWpSZ2VDUGZQaTBSaEov?=
 =?utf-8?B?S0JVaCtpUUxuZjE3bEVkRUFIQnB4cnV4VGE4TXF1enVQaGl0czgrQTZqTHlm?=
 =?utf-8?Q?f6a/BlABw+Ze/?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(35042699013)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 14:15:57.6825
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7598b5fc-7265-4596-d41e-08dc6ea03709
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028D02.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9714

SGkgTWljaGFsLA0KDQoNCj4+Pj4gDQo+Pj4+IGludCBfX2luaXQgcHJvY2Vzc19zaG0oc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IGtlcm5lbF9pbmZvICpraW5mbywNCj4+Pj4gICAgICAgICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqbm9kZSkNCj4+Pj4gew0KPj4+
PiBAQCAtMjQ5LDMyICsyOTAsMTAgQEAgaW50IF9faW5pdCBwcm9jZXNzX3NobShzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3Qga2VybmVsX2luZm8gKmtpbmZvLA0KPj4+PiAgICAgICAgaWYgKCBkdF9w
cm9wZXJ0eV9yZWFkX3N0cmluZyhzaG1fbm9kZSwgInJvbGUiLCAmcm9sZV9zdHIpID09IDAgKQ0K
Pj4+PiAgICAgICAgICAgIG93bmVyX2RvbV9pbyA9IGZhbHNlOw0KPj4+IExvb2tpbmcgYXQgb3du
ZXJfZG9tX2lvLCB3aHkgZG9uJ3QgeW91IG1vdmUgcGFyc2luZyByb2xlIGFuZCBzZXR0aW5nIG93
bmVyX2RvbV9pbyBhY2NvcmRpbmdseSB0byBoYW5kbGVfc2hhcmVkX21lbV9iYW5rKCk/DQo+PiAN
Cj4+IEkgdGhpbmsgSSB3YW50ZWQgdG8ga2VlcCBhbGwgZHRfKiBmdW5jdGlvbnMgb24gdGhlIHNh
bWUgbGV2ZWwgaW5zaWRlIHByb2Nlc3Nfc2htLCBvdGhlcndpc2UgeWVzLCBJIGNvdWxkDQo+PiBw
YXNzIGRvd24gc2htX25vZGUgYW5kIGRvIHRoZSByZWFkaW5nIG9mIHJvbGVfc3RyIGluIGhhbmRs
ZV9zaGFyZWRfbWVtX2JhbmssIG9yIEkgY291bGQgZGVyaXZlDQo+PiBvd25lcl9kb21faW8gZnJv
bSByb2xlX3N0ciBiZWluZyBwYXNzZWQgb3Igbm90LCBzb21ldGhpbmcgbGlrZToNCj4+IA0KPj4g
cm9sZV9zdHIgPSBOVUxMOw0KPj4gZHRfcHJvcGVydHlfcmVhZF9zdHJpbmcoc2htX25vZGUsICJy
b2xlIiwgJnJvbGVfc3RyKQ0KPj4gDQo+PiBbaW5zaWRlIGhhbmRsZV9zaGFyZWRfbWVtX2Jhbmtd
Og0KPj4gSWYgKCByb2xlX3N0ciApDQo+PiAgICBvd25lcl9kb21faW8gPSBmYWxzZTsNCj4+IA0K
Pj4gQW5kIHBhc3Mgb25seSByb2xlX3N0ciB0byBoYW5kbGVfc2hhcmVkX21lbV9iYW5rLg0KPj4g
DQo+PiBJcyB0aGlzIGNvbW1lbnQgdG8gcmVkdWNlIHRoZSBudW1iZXIgb2YgcGFyYW1ldGVycyBw
YXNzZWQ/IEkgZ3Vlc3MgaXTigJlzIG5vdCBmb3Igd2hlcmUgd2UgY2FsbA0KPiBJbiB0aGlzIHNl
cmllcyBhcyB3ZWxsIGFzIHRoZSBwcmV2aW91cyBvbmUgeW91IGxpbWl0IHRoZSBudW1iZXIgb2Yg
YXJndW1lbnRzIHBhc3NlZCB0byBxdWl0ZSBhIGZldyBmdW5jdGlvbnMuDQo+IFNvIG5hdHVyYWxs
eSBJIHdvdWxkIGV4cGVjdCBpdCB0byBiZSBkb25lIGhlcmUgYXMgd2VsbC4gb3duZXJfZG9tX2lv
IGlzIHVzZWQgb25seSBieSBoYW5kbGVfc2hhcmVkX21lbV9iYW5rLCBzbyBpdCBtYWtlcyBtb3Jl
IHNlbnNlIHRvIG1vdmUgcGFyc2luZyB0byB0aGlzDQo+IGZ1bmN0aW9uIHNvIHRoYXQgaXQgaXMg
c2VsZi1jb250YWluZWQuDQoNCk9rIEkgd2lsbCwganVzdCB0byBiZSBvbiB0aGUgc2FtZSBwYWdl
IGhlcmUsIHlvdSBtZWFuIGhhdmluZyBkdF9wcm9wZXJ0eV9yZWFkX3N0cmluZyBpbnNpZGUgaGFu
ZGxlX3NoYXJlZF9tZW1fYmFuaz8NCk9yIHRoZSBhYm92ZSBleGFtcGxlIHdvdWxkIHdvcmsgZm9y
IHlvdSBhcyB3ZWxsPyBUaGF0IG9uZSB3b3VsZCBoYXZlIHJvbGVfc3RyIHBhc3NlZCBpbnN0ZWFk
IG9mIHNobV9ub2RlLg0KDQpDaGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:24:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:24:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718211.1120764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Ljd-0005iG-Cr; Tue, 07 May 2024 14:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718211.1120764; Tue, 07 May 2024 14:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Ljd-0005i9-86; Tue, 07 May 2024 14:24:01 +0000
Received: by outflank-mailman (input) for mailman id 718211;
 Tue, 07 May 2024 14:23:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4Ljb-0005i3-FD
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:23:59 +0000
Received: from wfhigh7-smtp.messagingengine.com
 (wfhigh7-smtp.messagingengine.com [64.147.123.158])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6f224551-0c7d-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:23:57 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfhigh.west.internal (Postfix) with ESMTP id 34FB318000EA;
 Tue,  7 May 2024 10:23:53 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Tue, 07 May 2024 10:23:53 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 10:23:51 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f224551-0c7d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715091832;
	 x=1715178232; bh=bGrzyVzYoeNKTDEvT1NrsJFQfSswRRZA4mlVkdChOC4=; b=
	Cm7+rs0r8ymQVUtc8inDSyNVEkJLu5ayDCehlqhXXc71cIwoHE/Nz0AAZIOgvFrJ
	hEtpAsyNgX0wxsgdvSTzCslDdwqtHNfYxx6SMps6Sx+Xgb9KKXJKzjaOp7/i2Gud
	Zf58qfY+HYmaCCxNEH/CVG2sWja3Jlu8zU9r3qZOM7lQ0RBZFJ5AwkhvfPQj/8FL
	8GHCsaRDWJ0kOqB5yCT0G84YfvaexyOXRPwYcp8gT69m4Ts3qxN95EYFfgvzMwpI
	eEifKZljeyJLAtMZ0M3g4u8NLAv3L99KXgdhBj+731jE8nMbMY7roiVLsZAW2kgS
	zko2qpKXUJlTvmopHDuJsg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715091832; x=1715178232; bh=bGrzyVzYoeNKTDEvT1NrsJFQfSsw
	RRZA4mlVkdChOC4=; b=hzXkfExpFRItLKuQ9AYF3TPLX7FGI4GAfk5wAGNQkq82
	k2F/yBC4ZYzJqNK5mbfHoXaai1Xe/DgILBQjrLdxhPTkEPYRl5U1F2IY0E+tsSD+
	4epJVQMaDDkcxnMBufXLQmChpKwqXGWF1R9+OzD/livRj3MWDXJcRW9DjQuGsbOK
	wIFqIr/7315sOzcHNnflTQnybIjw/LvqdODk8LYeU/LQKal0fvg7uzAI6b9Q5YT8
	V2Mtshd5hf4vPIBdT+CU/MxRcbmx8B5Fj9CccQXvSvqm7Awduh7rp+wbtVTFJNqS
	PUidUHfs8NkwZKE80ohgBWqYig8/mXjeyd7gm9orUw==
X-ME-Sender: <xms:eDk6ZlfdjMhVvyzSofE3nMe8ghxQHsDDrrlD3OJX9j0tfEd7qeShAQ>
    <xme:eDk6ZjOEALkEnnbQAvlMqR34wh5e_GLI8sixzmQOtbSPs-TeX8HkbBYukgH5Ki1Y3
    GSkhWiaMKHB5g>
X-ME-Received: <xmr:eDk6ZuhzYkzZNQ5WD8IJHxOWdCStzfiSth38jQPQnc0HezENhQ4B3VbCn_CwgYj6IL1VfNzQy7JkR5MrY0XZmFDYpkQErCnrtg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgjeefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueek
    teetgefggfekudehteegieeljeejieeihfejgeevhfetgffgteeuteetueetnecuffhomh
    grihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:eDk6Zu810UixtwjIhVt40vcfCGxwmWA60AlPhY7qVjBjE-J7zZ34xg>
    <xmx:eDk6Zht1IC5qsvT6Qv5NibcsrmZLYwPdZ3cyPFq-tk6A1QsBMs1WLA>
    <xmx:eDk6ZtG60qxqPRfFGJsmUv9HtXJ5bXDCUMpVB8eaHv6_Cwpqa1Kgnw>
    <xmx:eDk6ZoPi2tJtTbcBNmxU0_7GrhDJ2h2nvvbR6a77ekgJrML8QNFhDQ>
    <xmx:eDk6ZrIex4-1S7RnuC_XK2g3uA-G9vsb1xV9gGaaLC-oC97WhJINCFfH>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 7 May 2024 16:23:48 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
Message-ID: <Zjo5dbC6iaFWUA8R@mail-itl>
References: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
 <ZjoRMHmL8_K9_lsL@mail-itl>
 <549a101f-b9b6-47fb-a782-c716f1ce49ec@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="eURIAwtQ8LuabfNF"
Content-Disposition: inline
In-Reply-To: <549a101f-b9b6-47fb-a782-c716f1ce49ec@citrix.com>


--eURIAwtQ8LuabfNF
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 7 May 2024 16:23:48 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC

On Tue, May 07, 2024 at 03:15:48PM +0100, Andrew Cooper wrote:
> On 07/05/2024 12:32 pm, Marek Marczykowski-G=C3=B3recki wrote:
> > On Tue, May 07, 2024 at 12:08:06PM +0100, Andrew Cooper wrote:
> >> `xl devd` has been observed leaking /var/log/xldevd.log into children.
> >>
> >> Link: https://github.com/QubesOS/qubes-issues/issues/8292
> >> Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >> ---
> >> CC: Anthony PERARD <anthony@xenproject.org>
> >> CC: Juergen Gross <jgross@suse.com>
> >> CC: Demi Marie Obenour <demi@invisiblethingslab.com>
> >> CC: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>
> >>
> >> Also entirely speculative based on the QubesOS ticket.
> >> ---
> >>  tools/xl/xl_utils.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
> >> index 17489d182954..060186db3a59 100644
> >> --- a/tools/xl/xl_utils.c
> >> +++ b/tools/xl/xl_utils.c
> >> @@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pid=
file)
> >>          exit(-1);
> >>      }
> >> =20
> >> -    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY|O_CREAT|O_APPEND,=
 0644));
> >> +    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY | O_CREAT | O_APP=
END | O_CLOEXEC, 0644));
> > This one might be not enough, as the FD gets dup2()-ed to stdout/stderr
> > just outside of the context here, and then inherited by various hotplug
> > script. Just adding O_CLOEXEC here means the hotplug scripts will run
> > with stdout/stderr closed.
>=20
> Lovely :(=C2=A0 Yes - this won't work.=C2=A0 I guess what we want instead=
 is:
>=20
> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
> index 060186db3a59..a0ce7dd7fa21 100644
> --- a/tools/xl/xl_utils.c
> +++ b/tools/xl/xl_utils.c
> @@ -282,6 +282,7 @@ int do_daemonize(const char *name, const char *pidfil=
e)
> =C2=A0=C2=A0=C2=A0=C2=A0 dup2(logfile, 2);
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0 close(nullfd);
> +=C2=A0=C2=A0=C2=A0 close(logfile);
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0 CHK_SYSCALL(daemon(0, 1));
> =C2=A0
> which at least means there's not a random extra fd attached to the logfil=
e.

But logfile is a global variable, and it looks to be used in dolog()...

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--eURIAwtQ8LuabfNF
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmY6OXUACgkQ24/THMrX
1yxZ+Af9EwBLKKb20LbZ2RZhnpeodi5iGSRiJSHeDv1QDydizP1PDna7iZ2K3zjq
SvsHLknSCxfdZL/pgmG0ZB55/5LFLumX4vg70D1x/Uk14vBQ8Xrcp1LxjYaONjoa
nqlPNIkjBCSgMs5tZAsGa8ghe7uaTIEWvPaEYob+DO2eILvfAZuy4I8os700YhkS
sYsBAF3CD0wYlpBzpSQFh0zM3v6kf8d03b4Yh/p0q5LVmGL5wTKQhPUUCg6y7qCI
VLQLKofqY3B1ydhWCplBQ/TqljUNR/WMYyefDHmTxSOrlx7J8AwYaKgbjPXJAuai
HF4Fm4UhDjxDbBgLtr2aOSKgSUhkhA==
=2Skc
-----END PGP SIGNATURE-----

--eURIAwtQ8LuabfNF--


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:24:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:24:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718212.1120773 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lju-00062Z-IM; Tue, 07 May 2024 14:24:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718212.1120773; Tue, 07 May 2024 14:24:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lju-00062S-Fh; Tue, 07 May 2024 14:24:18 +0000
Received: by outflank-mailman (input) for mailman id 718212;
 Tue, 07 May 2024 14:24:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J4VX=MK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4Ljt-00061o-68
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:24:17 +0000
Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com
 [2607:f8b0:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7acf9374-0c7d-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 16:24:15 +0200 (CEST)
Received: by mail-oi1-x235.google.com with SMTP id
 5614622812f47-3c86f066256so1799189b6e.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 07:24:15 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 r9-20020a0cf809000000b006a0ecb2333esm4735874qvn.99.2024.05.07.07.24.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 07:24:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7acf9374-0c7d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715091854; x=1715696654; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+HzMSaFTxW/BpNWP9DdNLYLF5hdPWgxP7OVUISaXBrU=;
        b=WvSCr0/7ECf+wpT3bR/xmPXb4K30C4pceJnMydLESpmoOgvW3a02j5YcogBmIO/kMy
         k4aUGotmFj+Weugygg8mXbzr7B/iLn130LX8imyIUYmca86v60i5urZRp+C94AWBbJMl
         83yE7AhqMcoYxofVinmPxMP5cbwnd8FpudSnk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715091854; x=1715696654;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+HzMSaFTxW/BpNWP9DdNLYLF5hdPWgxP7OVUISaXBrU=;
        b=pn2I2Cbk1JYjOZJoOK6weRhJUjdYihEwOPk395BRRUkn0tH4vpZw2H2Jn+I1tH8Kak
         pX1b83pPdZMhQNkXOS4tnaz8ttndh+G7bx4pIp3IQtD9suiR8DN7Pm32HqT82I4Q35ED
         +YfUI5HTjGas4Y4XUc+yvL9B2cl/mSaMSRNjVGNhDUZad0SE0Yz7E0PRYME5YxAu7gU/
         2V4BGGsw8icOuOtfXAviria+/RYT6zFk3uGG7wj9nm/Tqg+wjiK5uOuYgbKAM4LALCB2
         cd6sgm1fcom9kmkDdKvuR2zBcP+qGYXDEriJnpmh5+frPwgS1W1HW8AtMsKs34pjIO7t
         6Dfg==
X-Gm-Message-State: AOJu0Yx3R3PmpzerxeuL4XQIVR/VyS5REai5pUFpmQxJlxy9hPh4kir4
	hEUYLryFpXZVGHlIUs4qHclNBdsavX7WL9sss+GzG4zanDbhvxDrdEadgZkoPjQ=
X-Google-Smtp-Source: AGHT+IHdu0GbLBLq8xrWB7P1x0kX0CqfibMyyb+mstpbF5UTNoLMg64Yh3Dd9ZTvP4qbCrEBQNRZCA==
X-Received: by 2002:aca:90a:0:b0:3c9:7003:af7e with SMTP id 10-20020aca090a000000b003c97003af7emr5226265oij.18.1715091853672;
        Tue, 07 May 2024 07:24:13 -0700 (PDT)
Date: Tue, 7 May 2024 16:24:11 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to
 Cascade Lake
Message-ID: <Zjo5iwmEvD3RWtsx@macbook>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <20240507134540.1714274-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240507134540.1714274-1-andrew.cooper3@citrix.com>

On Tue, May 07, 2024 at 02:45:40PM +0100, Andrew Cooper wrote:
> Ever since Xen 4.14, there has been a latent bug with migration.
> 
> While some toolstacks can level the features properly, they don't shink
> feat.max_subleaf when all features have been dropped.  This is because
> we *still* have not completed the toolstack side work for full CPU Policy
> objects.
> 
> As a consequence, even when properly feature levelled, VMs can't migrate
> "backwards" across hardware which reduces feat.max_subleaf.  One such example
> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
> 
> Extend the max policies feat.max_subleaf to the hightest number Xen knows
> about, but leave the default policies matching the host.  This will allow VMs
> with a higher feat.max_subleaf than strictly necessary to migrate in.
> 
> Eventually we'll manage to teach the toolstack how to avoid creating such VMs
> in the first place, but there's still more work to do there.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Even if we have just found one glitch with PSFD and Ice Lake vs
Cascade Lack, wouldn't it be safer to always extend the max policies
max leafs and subleafs to match the known array sizes?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:26:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:26:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718219.1120783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LmC-0007Mg-1N; Tue, 07 May 2024 14:26:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718219.1120783; Tue, 07 May 2024 14:26:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LmB-0007MZ-Tn; Tue, 07 May 2024 14:26:39 +0000
Received: by outflank-mailman (input) for mailman id 718219;
 Tue, 07 May 2024 14:26:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w3sU=MK=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4LmA-0007HX-72
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:26:38 +0000
Received: from wfout5-smtp.messagingengine.com
 (wfout5-smtp.messagingengine.com [64.147.123.148])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdb64a1d-0c7d-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 16:26:36 +0200 (CEST)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailfout.west.internal (Postfix) with ESMTP id A39BF1C0010F;
 Tue,  7 May 2024 10:26:32 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Tue, 07 May 2024 10:26:33 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 7 May 2024 10:26:30 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdb64a1d-0c7d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715091992;
	 x=1715178392; bh=thdWYW9Axd3S2TbzcgPEsebhY8W24WIZD54DTU8mwcc=; b=
	nA5KnFuHO6K9cJtYbMOGtqDsm76JyiJBP1TEhs5YliBG734dI1/znAOqMBA+1d08
	UJHdQ3LCPPquYFyKS1OjmsKtvbVpA59lFqQIvCuRY3QfYRPn2XPWw3uiI43U7cwh
	lDcDHiT6Zpw99wPzXSACTjQZ/2Hi3EW2M6FyyocKZ4fMqyFLwr692i9O2bUD6hIt
	5jelM8U68o60tVbrMdyjBV6JnWvkX/iWVHfAwwp0YfXg67hOXoH9CztPrMr1Kk9L
	DGylXWXXFdfk2P8u/kn2+5RnZ4oZtHZUk5puMUvIC4gEDl5XCasfnsfnRyQIkZxh
	dezKCbIdkSQG1HLk8IEfsA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715091992; x=1715178392; bh=thdWYW9Axd3S2TbzcgPEsebhY8W2
	4WIZD54DTU8mwcc=; b=SXUeizl+W7owp4VctUFPfE59NQr298xuoAZnyzgAg65B
	+GKTcV414TDY0iZY3Wzj0ysAwhnmllDzLSVGu8087YoRsQNd6QqyR0wwOTXmoM6X
	84Ib8mwOu7yIVM+y4F4x+IV+ZMWLuRJcjWPz+YThvobwzE8ZBLtI4sqqB1XqcWTz
	61zgjiqhZSgZza7D7MArCvBjnWTNT69saRIjOyUcaw9jD3JY0BFDqTQEkGUE/jSU
	o1OkbybCC4PyyBHA/GWoHlv4qx2bggof/DIyrVR91nfXWhnoxi2O5OYW9r0qChzU
	FYpX2oFb8iqBc9k1v4DIsL9+s1+JEDWm6/XaO+zoHg==
X-ME-Sender: <xms:Fzo6ZhjtPA2xk_TXchwOloeqmEXPY3Kv4MQh3AzjgNWYwU1XiI4Qrg>
    <xme:Fzo6ZmCfCgp39eTo0LFhEjl6lcqXu2ZrurCMWPIGaqp0ZLPGjRlxuCrlCj617HWBO
    5rQ6uTKMerqUA>
X-ME-Received: <xmr:Fzo6ZhF0lvVrgM92idJ-NorMNZQq8ad9tyXa2zPg0rXyP88zh8OdQmsxPkQxbAUZxuRYXthHUP6fKxMNTqDRvVEM5u3KuAYFwQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddvkedgjeegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueek
    teetgefggfekudehteegieeljeejieeihfejgeevhfetgffgteeuteetueetnecuffhomh
    grihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:Fzo6ZmQd5r9Qh7YFLalW0qCjrYmjQczJurjHbbssi-u1sD8EdjtHqw>
    <xmx:Fzo6Zuwf1n8OG1bKlCzeWZA1XmIjby_nrHofUqod9NLEckADNpi5Kg>
    <xmx:Fzo6Zs47UGWyVMNhkAADgDhG-4MjgrJmMul26DU4ktY_E7rxA4aN9g>
    <xmx:Fzo6ZjwcanFtziJR8rokr8oItvYHs06Gl6FbTXXU4VE-5NutqWS4Ag>
    <xmx:GDo6Zot21DMmRhIlf19tTvxBUX-nQSEvtlYIt8dEV9qAtLxlY75Plfld>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 7 May 2024 16:26:28 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
Message-ID: <Zjo6FNSnL6JNfK9l@mail-itl>
References: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
 <ZjoRMHmL8_K9_lsL@mail-itl>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="M/M/fcStNov/GPjV"
Content-Disposition: inline
In-Reply-To: <ZjoRMHmL8_K9_lsL@mail-itl>


--M/M/fcStNov/GPjV
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 7 May 2024 16:26:28 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Demi Marie Obenour <demi@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC

On Tue, May 07, 2024 at 01:32:00PM +0200, Marek Marczykowski-G=C3=B3recki w=
rote:
> On Tue, May 07, 2024 at 12:08:06PM +0100, Andrew Cooper wrote:
> > `xl devd` has been observed leaking /var/log/xldevd.log into children.
> >=20
> > Link: https://github.com/QubesOS/qubes-issues/issues/8292
> > Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > ---
> > CC: Anthony PERARD <anthony@xenproject.org>
> > CC: Juergen Gross <jgross@suse.com>
> > CC: Demi Marie Obenour <demi@invisiblethingslab.com>
> > CC: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>
> >=20
> > Also entirely speculative based on the QubesOS ticket.
> > ---
> >  tools/xl/xl_utils.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >=20
> > diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
> > index 17489d182954..060186db3a59 100644
> > --- a/tools/xl/xl_utils.c
> > +++ b/tools/xl/xl_utils.c
> > @@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pidf=
ile)
> >          exit(-1);
> >      }
> > =20
> > -    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY|O_CREAT|O_APPEND, =
0644));
> > +    CHK_SYSCALL(logfile =3D open(fullname, O_WRONLY | O_CREAT | O_APPE=
ND | O_CLOEXEC, 0644));
>=20
> This one might be not enough, as the FD gets dup2()-ed to stdout/stderr
> just outside of the context here, and then inherited by various hotplug
> script. Just adding O_CLOEXEC here means the hotplug scripts will run
> with stdout/stderr closed. The scripts shipped with Xen do redirect
> stderr to a log quite early, but a) it doesn't do it for stdout, and b)
> custom hotplug scripts are a valid use case.
> Without that, I see at least few potential issues:
> - some log messages may be lost (minor, but annoying)
> - something might simply fail on writing to a closed FD, breaking the
>   hotplug script
> - FD 1 will be used as first free FD for any open() or similar call - if
>   a tool later tries writing something to stdout, it will gets written
>   to that FD - worse of all three

Wait, the above is wrong, dup does not copy the O_CLOEXEC flag over to
the new FD. So, maybe your patch is correct after all.

> What should be the behavior of hotplug scripts logging? Should they
> always take care of their own logging? If so, the hotplug calling part
> should redirect stdout/stderr to /dev/null IMO. But if `xl` should
> provide some default logging for them (like, the xldevd.log here?), then
> the O_CLOEXEC should be set only after duplicating logfile over stdout/er=
r.
>=20
> >      free(fullname);
> >      assert(logfile >=3D 3);
> > =20
> >=20
> > base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
> > prerequisite-patch-id: 212e50457e9b6bdfd06a97da545a5aa7155bb919
>=20
> Which one is this? I don't see it in staging, nor in any of your
> branches on xenbits. Lore finds "tools/libxs: Open /dev/xen/xenbus fds
> as O_CLOEXEC" which I guess is correct, but I have no idea how it
> correlates it, as this hash doesn't appear anywhere in the message, nor
> its headers...
>=20
> --=20
> Best Regards,
> Marek Marczykowski-G=C3=B3recki
> Invisible Things Lab



--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--M/M/fcStNov/GPjV
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmY6OhQACgkQ24/THMrX
1yzYxwf/VMal/oF542eF20bsWJ4/D1LGZkkLrWwHBJ7WUll7OfeorN3XU5J1uWuc
Hsbh/rw1VcN1GHsm9BKSuyOdTHarxz2nmueFtTPSC93tLGOwd3E0I5cuD1IJB/tm
TLYbI30cduPZVDpIpbmU5E+7s/fjbWz5KT0Z5yHbUub0CGErF6hqx3YIBcyNut/X
V2reBK/hU97uymaG6r5TfqT0t0ws/2gWYyj331ITkJLwQXd0PRrE9mRFWjxS4l92
nf1/CB53o96HV+wfGKoXSQ4ZgCl81ctFkYDNKcCULs0JMDcIMHBEyVGW+TNKgoG9
WlAw4ani2Qw7LKYwUbsC96G8ts366Q==
=AMNU
-----END PGP SIGNATURE-----

--M/M/fcStNov/GPjV--


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:26:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:26:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718222.1120793 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LmP-0007ij-7h; Tue, 07 May 2024 14:26:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718222.1120793; Tue, 07 May 2024 14:26:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LmP-0007ia-4c; Tue, 07 May 2024 14:26:53 +0000
Received: by outflank-mailman (input) for mailman id 718222;
 Tue, 07 May 2024 14:26:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4LmO-0007dK-5P
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:26:52 +0000
Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com
 [2607:f8b0:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8124229-0c7d-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:26:51 +0200 (CEST)
Received: by mail-ot1-x32a.google.com with SMTP id
 46e09a7af769-6f054c567e2so1506629a34.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 07:26:51 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 fe13-20020a05622a4d4d00b0043842dc662esm6481076qtb.4.2024.05.07.07.26.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 07:26:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8124229-0c7d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715092010; x=1715696810; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ACYAsvZOXFthKYevSl+c77kLIndM3633AwMnmyqidpc=;
        b=SH3TduKA7Gjt6fZN3h/Vt2x41SmJwge2oSffsKEh51fRWq0hWHpDGrPAtCrnoQhnjE
         Q0sjuSWUpkDltDVPzE/b7pIjJUq1R3BKkP9FCmniCCyRprPCewoHVyLcel/ceZahxsAH
         jajb0tqQ5OMX7/A+4KMNxEqW49u+4C5wJzLkc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715092010; x=1715696810;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ACYAsvZOXFthKYevSl+c77kLIndM3633AwMnmyqidpc=;
        b=vjwxL6pPaMkOD1YVoZJxHxaA3otWnms03pWWKmEHUgT/bdo4bbD84d9U/vlYnhZ5F6
         mAdAW1otCW3saebIRROAZ50edaXlQ5g8LV27aTfXC1JcNUH9Otz1TQZCyDY5ZywaarMK
         GX7sn01mTp3+iC65f1t9obW0n6bsY4yhxxkNNgHNIS+aEUFnH+Df3j++TGaeLUA2Kmxl
         Ft9G/4jfb/wCseEccS70c2kakCd6IE2GszjiOGpoRoZGDgg0gucp5TmfJLcncqjeUFC8
         BbL+CaLRRYqsUnpOvKZ3dZ+GWd2ExndV+5U7D9+ysssBCIZ3qfNnyh/0zvLZlARFRW5I
         7kgg==
X-Gm-Message-State: AOJu0YzaK1ooXeFF6MO+end1osHRYk4tCHCyROHgaTh3zoa4aGRMHav6
	FyF0eKr5qpZ7Bg5fenMPrzHzW5DrWFWRw6tiH0rt6WmnssYV0rbbkBRfjd7ylw8=
X-Google-Smtp-Source: AGHT+IGuU0Ihj0Eu7pJDfvAYKKVDddqLWBIAzBGFv1BV6rJOiy/Fao8B2gcbZmDp+oAXp78Oh6oPlA==
X-Received: by 2002:a05:6808:2013:b0:3c8:6238:8c38 with SMTP id q19-20020a056808201300b003c862388c38mr18011769oiw.11.1715092010243;
        Tue, 07 May 2024 07:26:50 -0700 (PDT)
Message-ID: <27a67c41-6dd3-416a-8be1-cd0daa600449@citrix.com>
Date: Tue, 7 May 2024 15:26:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xl: Open xldevd.log with O_CLOEXEC
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Demi Marie Obenour <demi@invisiblethingslab.com>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240507110806.1692135-1-andrew.cooper3@citrix.com>
 <ZjoRMHmL8_K9_lsL@mail-itl> <549a101f-b9b6-47fb-a782-c716f1ce49ec@citrix.com>
 <Zjo5dbC6iaFWUA8R@mail-itl>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zjo5dbC6iaFWUA8R@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/05/2024 3:23 pm, Marek Marczykowski-Górecki wrote:
> On Tue, May 07, 2024 at 03:15:48PM +0100, Andrew Cooper wrote:
>> On 07/05/2024 12:32 pm, Marek Marczykowski-Górecki wrote:
>>> On Tue, May 07, 2024 at 12:08:06PM +0100, Andrew Cooper wrote:
>>>> `xl devd` has been observed leaking /var/log/xldevd.log into children.
>>>>
>>>> Link: https://github.com/QubesOS/qubes-issues/issues/8292
>>>> Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> ---
>>>> CC: Anthony PERARD <anthony@xenproject.org>
>>>> CC: Juergen Gross <jgross@suse.com>
>>>> CC: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>>>
>>>> Also entirely speculative based on the QubesOS ticket.
>>>> ---
>>>>  tools/xl/xl_utils.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
>>>> index 17489d182954..060186db3a59 100644
>>>> --- a/tools/xl/xl_utils.c
>>>> +++ b/tools/xl/xl_utils.c
>>>> @@ -270,7 +270,7 @@ int do_daemonize(const char *name, const char *pidfile)
>>>>          exit(-1);
>>>>      }
>>>>  
>>>> -    CHK_SYSCALL(logfile = open(fullname, O_WRONLY|O_CREAT|O_APPEND, 0644));
>>>> +    CHK_SYSCALL(logfile = open(fullname, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0644));
>>> This one might be not enough, as the FD gets dup2()-ed to stdout/stderr
>>> just outside of the context here, and then inherited by various hotplug
>>> script. Just adding O_CLOEXEC here means the hotplug scripts will run
>>> with stdout/stderr closed.
>> Lovely :(  Yes - this won't work.  I guess what we want instead is:
>>
>> diff --git a/tools/xl/xl_utils.c b/tools/xl/xl_utils.c
>> index 060186db3a59..a0ce7dd7fa21 100644
>> --- a/tools/xl/xl_utils.c
>> +++ b/tools/xl/xl_utils.c
>> @@ -282,6 +282,7 @@ int do_daemonize(const char *name, const char *pidfile)
>>      dup2(logfile, 2);
>>  
>>      close(nullfd);
>> +    close(logfile);
>>  
>>      CHK_SYSCALL(daemon(0, 1));
>>  
>> which at least means there's not a random extra fd attached to the logfile.
> But logfile is a global variable, and it looks to be used in dolog()...

Urgh, fine.  Lets go back to your suggestion of setting CLOEXEC after
dup()ing.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:31:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:31:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718227.1120802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lqn-00017O-OW; Tue, 07 May 2024 14:31:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718227.1120802; Tue, 07 May 2024 14:31:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Lqn-00017H-Lt; Tue, 07 May 2024 14:31:25 +0000
Received: by outflank-mailman (input) for mailman id 718227;
 Tue, 07 May 2024 14:31:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4Lqm-00017B-LQ
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:31:24 +0000
Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com
 [2001:4860:4864:20::2f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 79ace0fc-0c7e-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 16:31:22 +0200 (CEST)
Received: by mail-oa1-x2f.google.com with SMTP id
 586e51a60fabf-23f55e2e908so1342863fac.3
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 07:31:22 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ez7-20020a05622a4c8700b0043496744c5dsm6510755qtb.52.2024.05.07.07.31.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 07:31:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79ace0fc-0c7e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715092281; x=1715697081; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=j0BTVGvHEKyiBbxz3RZmU5S9st2y1z9qXUwG+d5JHcE=;
        b=tPK52cHsNEudlPHtFREIqfgq1mNyWQYZBvSpkyfgbrTFq59hRi6yKmY3jqsQ/a6idZ
         l5umflP0LO2kCvlwZdtsfQOejyhWWBSsHI9vMPNpGEiA0pt/gSYuYYTpV87VVYKWxOsd
         K/ha3W6bv9pYy+2ECNEbbsysPciE80XyaXbfg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715092281; x=1715697081;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=j0BTVGvHEKyiBbxz3RZmU5S9st2y1z9qXUwG+d5JHcE=;
        b=WMKnc+izrpfiH9TCk3jGAqRxIvRyThUo2e09VN1PA07Xaq0OQBGyPosMlrktGxc/8Y
         L5YuRBR6Nd90CGcaXKex9JHzNKHhUzdp09bzpoEA9ypngeu6CxgzrM6gwaeW+MQp9SN4
         6aypG2JR6M9rxhYBfOvUM8isEDzvOA8hpRt7CW7jCaYIPzbJFqknvY+S/udSFsgVIRin
         BFWE66CIaWt0+q9nvDQS8evlG6brT5m8J7yyp0LDKq4dQQXueY21lwi1g2e5ftn8nvZM
         yM3KTPvO0ffPTS2acdmxReR+brAtMm3FpjbATXWXyj0zczT0ScSg2vHpphHl+gzrwxzg
         JQvg==
X-Gm-Message-State: AOJu0Yws9axnc2pPKlu7hkR4N6li3T6MpLjUdATSK4NL+imhlNyiZqcG
	I8vev6G55ZEDMIVUv8YGlHfN3OpBlOw5w6B8S10ARUf6FnGHLEpDvnSDxLuITSE=
X-Google-Smtp-Source: AGHT+IHkP42kZ17vHRLPYZ1ANwuxu48Jg7HK/BVc4n6dZ2+mqkyEUiMxMtAWcmhCUh4l06m+Iw6xfg==
X-Received: by 2002:a05:6870:2b0a:b0:229:e6c6:1f13 with SMTP id ld10-20020a0568702b0a00b00229e6c61f13mr15348545oab.14.1715092281354;
        Tue, 07 May 2024 07:31:21 -0700 (PDT)
Message-ID: <f0272d79-7c59-42b2-b185-2aaf84f1b196@citrix.com>
Date: Tue, 7 May 2024 15:31:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to Cascade
 Lake
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <20240507134540.1714274-1-andrew.cooper3@citrix.com>
 <Zjo5iwmEvD3RWtsx@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zjo5iwmEvD3RWtsx@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/05/2024 3:24 pm, Roger Pau Monné wrote:
> On Tue, May 07, 2024 at 02:45:40PM +0100, Andrew Cooper wrote:
>> Ever since Xen 4.14, there has been a latent bug with migration.
>>
>> While some toolstacks can level the features properly, they don't shink
>> feat.max_subleaf when all features have been dropped.  This is because
>> we *still* have not completed the toolstack side work for full CPU Policy
>> objects.
>>
>> As a consequence, even when properly feature levelled, VMs can't migrate
>> "backwards" across hardware which reduces feat.max_subleaf.  One such example
>> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
>>
>> Extend the max policies feat.max_subleaf to the hightest number Xen knows
>> about, but leave the default policies matching the host.  This will allow VMs
>> with a higher feat.max_subleaf than strictly necessary to migrate in.
>>
>> Eventually we'll manage to teach the toolstack how to avoid creating such VMs
>> in the first place, but there's still more work to do there.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

>
> Even if we have just found one glitch with PSFD and Ice Lake vs
> Cascade Lack, wouldn't it be safer to always extend the max policies
> max leafs and subleafs to match the known array sizes?

This is the final max leaf (containing feature information) to gain
custom handling, I think?

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:35:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:35:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718230.1120812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LuP-000271-7z; Tue, 07 May 2024 14:35:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718230.1120812; Tue, 07 May 2024 14:35:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4LuP-00026u-54; Tue, 07 May 2024 14:35:09 +0000
Received: by outflank-mailman (input) for mailman id 718230;
 Tue, 07 May 2024 14:35:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4LuO-00025M-0P
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:35:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4LuL-0008MV-Fl; Tue, 07 May 2024 14:35:05 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4LuL-0005cF-8R; Tue, 07 May 2024 14:35:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EoKL+Z0kQgMFXQExJ9CRpDqJzIJUhs2/XE5rGxXj090=; b=GyYOJNT8QrlEuUqPnHygH4YZun
	Fn1MoLoDxVtkVjtcPjJJ3HDonRSpSiaOG1eW/Z0qxvu1uA/H+Zaa/r481RE9ObViASV3BxcRFN9PC
	sEFs5s9oSA/dqn1aCRIXjGqVIQtVY3heqVnxSgheL9mXJH42Dmp7FGPVcC1l2bXAw3Vg=;
Message-ID: <bca3098b-4f0b-48e1-8e21-51c66c63e4f3@xen.org>
Date: Tue, 7 May 2024 15:35:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 05/15] tools/libs/light: Increase nr_spi to 160
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, Anthony PERARD
 <anthony.perard@cloud.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
Cc: xen-devel@lists.xenproject.org, Vikram Garhwal <fnu.vikram@xilinx.com>,
 Juergen Gross <jgross@suse.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-6-xin.wang2@amd.com>
 <334eb040-234f-4492-9006-9faadfe28c59@perard>
 <71fc7673-e968-487d-8478-1c7aabef708b@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <71fc7673-e968-487d-8478-1c7aabef708b@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 06/05/2024 06:17, Henry Wang wrote:
> On 5/1/2024 9:58 PM, Anthony PERARD wrote:
>> On Wed, Apr 24, 2024 at 11:34:39AM +0800, Henry Wang wrote:
>>> Increase number of spi to 160 i.e. gic_number_lines() for Xilinx 
>>> ZynqMP - 32.
>>> This was done to allocate and assign IRQs to a running domain.
>>>
>>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>>> ---
>>>   tools/libs/light/libxl_arm.c | 3 ++-
>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>> index dd5c9f4917..50dbd0f2a9 100644
>>> --- a/tools/libs/light/libxl_arm.c
>>> +++ b/tools/libs/light/libxl_arm.c
>>> @@ -181,7 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>>       LOG(DEBUG, "Configure the domain");
>>> -    config->arch.nr_spis = nr_spis;
>>> +    /* gic_number_lines() is 192 for Xilinx ZynqMP. min nr_spis = 
>>> 192 - 32. */
>>> +    config->arch.nr_spis = MAX(nr_spis, 160);
>> Is there a way that that Xen or libxl could find out what the minimum
>> number of SPI needs to be?
> 
> I am afraid currently there is none.
> 
>> Are we going to have to increase that minimum
>> number every time a new platform comes along?
>>
>> It doesn't appear that libxl is using that `nr_spis` value and it is
>> probably just given to Xen. So my guess is that Xen could simply take
>> care of the minimum value, gic_number_lines() seems to be a Xen
>> function.
> 
> Xen will take care of the value of nr_spis for dom0 in create_dom0()
> dom0_cfg.arch.nr_spis = min(gic_number_lines(), (unsigned int) 992) - 32;
> and also for dom0less domUs in create_domUs().
> 
> However, it looks like Xen will not take care of the mininum value for 
> libxl guests, the value from config->arch.nr_spis in guest config file 
> will be directly passed to the domain_vgic_init() function from 
> arch_domain_create().
> 
> I agree with you that we shouldn't just bump the number everytime when 
> we have a new platform. Therefore, would it be a good idea to move the 
> logic in this patch to arch_sanitise_domain_config()?

Xen domains are supposed to be platform agnostics and therefore the 
numbers of SPIs should not be based on the HW.

Furthermore, with your proposal we would end up to allocate data 
structure for N SPIs when a domain may never needs any SPIs (such as if 
passthrough is not in-use). This is more likely for domain created by 
the toolstack than from Xen directly.

Instead, we should introduce a new XL configuration to let the user 
decide the number of SPIs. I would suggest to name "nr_spis" to match 
the DT bindings.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 14:46:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 14:46:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718251.1120823 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4M4y-0004zm-67; Tue, 07 May 2024 14:46:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718251.1120823; Tue, 07 May 2024 14:46:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4M4y-0004zf-36; Tue, 07 May 2024 14:46:04 +0000
Received: by outflank-mailman (input) for mailman id 718251;
 Tue, 07 May 2024 14:46:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J4VX=MK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4M4w-0004yM-HD
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 14:46:02 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 855de1ab-0c80-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 16:46:01 +0200 (CEST)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-78f04924a96so280884285a.0
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 07:46:01 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 j10-20020a0cf9ca000000b006a105b217b5sm4713072qvo.112.2024.05.07.07.45.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 May 2024 07:45:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 855de1ab-0c80-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715093160; x=1715697960; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Gc4LwRfvx8MFYxJkMuKFV+WreREFsbPeRoETwV9Pld8=;
        b=tLvr3C10pqONFhlPtDnGXMDShWPiqyEyzgzhuALOwzjkklF5wZMgApcL/ovrnHGvwe
         azZTW9HaM9tllE6aN8vAld95CxwKAIJ/6pN706TPzA3t25ivhe5rnMq+PUsyjo+BZeIK
         up97xwNg0Fw8r3pOWShqAt8OAutccho7PeMYE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715093160; x=1715697960;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Gc4LwRfvx8MFYxJkMuKFV+WreREFsbPeRoETwV9Pld8=;
        b=N7pthbMa2+zFRYwTnGjAWqxSoaX2lRkGboFJjqB3QlA4paaJIhTnOltR7j6VCjuCwd
         Iy5wwvhkx+MvOyUjGHiX4hhBWGDCDocHS21SCJwGdco70nQaucjTKwsKSzaxdauMvRt2
         9bgDppmjFI8s6ndddZYmpFn9e0WrT+TdikIu6WHOcWgz1iS5sHG09UU7umzuMhcEYKU+
         +CJbTFPAGGegd/XccEDdNbmIE+u1WbPqttqZPQ1jB0f1MuAc7jEe2jf8pLc6m4YWTEf1
         t0cVXm26HJca0+JFfP4/5zUngP6BnNa41HJ+AKnKwBKQjxvHreL/EFN4ZUeMCvhkKexF
         MMeg==
X-Gm-Message-State: AOJu0YyezYHCrwYoq2q0QPd/VtBS221LjhLoItTpDmaJB3nQENCxseVK
	bb3kIYcwkXwtBhXyGEicZw3FAaIwEU4vp3ERAFDlpQ5K8CO2npJw0QaMreZwtYk=
X-Google-Smtp-Source: AGHT+IEHSJmsIeqE4A9Ovx26s4fzTti3J56PWp6L7ISnazuu6Be0YFvO68bdbFHjr36dC3fs6D0lGg==
X-Received: by 2002:a05:6214:1c84:b0:6a0:b594:1744 with SMTP id 6a1803df08f44-6a1514e8bb7mr123666d6.62.1715093160064;
        Tue, 07 May 2024 07:46:00 -0700 (PDT)
Date: Tue, 7 May 2024 16:45:57 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to
 Cascade Lake
Message-ID: <Zjo-pWunEGru6Z_K@macbook>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <20240507134540.1714274-1-andrew.cooper3@citrix.com>
 <Zjo5iwmEvD3RWtsx@macbook>
 <f0272d79-7c59-42b2-b185-2aaf84f1b196@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <f0272d79-7c59-42b2-b185-2aaf84f1b196@citrix.com>

On Tue, May 07, 2024 at 03:31:19PM +0100, Andrew Cooper wrote:
> On 07/05/2024 3:24 pm, Roger Pau Monné wrote:
> > On Tue, May 07, 2024 at 02:45:40PM +0100, Andrew Cooper wrote:
> >> Ever since Xen 4.14, there has been a latent bug with migration.
> >>
> >> While some toolstacks can level the features properly, they don't shink
> >> feat.max_subleaf when all features have been dropped.  This is because
> >> we *still* have not completed the toolstack side work for full CPU Policy
> >> objects.
> >>
> >> As a consequence, even when properly feature levelled, VMs can't migrate
> >> "backwards" across hardware which reduces feat.max_subleaf.  One such example
> >> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
> >>
> >> Extend the max policies feat.max_subleaf to the hightest number Xen knows
> >> about, but leave the default policies matching the host.  This will allow VMs
> >> with a higher feat.max_subleaf than strictly necessary to migrate in.
> >>
> >> Eventually we'll manage to teach the toolstack how to avoid creating such VMs
> >> in the first place, but there's still more work to do there.
> >>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Acked-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.
> 
> >
> > Even if we have just found one glitch with PSFD and Ice Lake vs
> > Cascade Lack, wouldn't it be safer to always extend the max policies
> > max leafs and subleafs to match the known array sizes?
> 
> This is the final max leaf (containing feature information) to gain
> custom handling, I think?

Couldn't the same happen with extended leaves?  Some of the extended
leaves contain features, and hence for policy leveling toolstack might
decide to zero them, yet extd.max_leaf won't be adjusted.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 07 15:03:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 15:03:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718260.1120832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4MLp-0000qb-Iu; Tue, 07 May 2024 15:03:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718260.1120832; Tue, 07 May 2024 15:03:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4MLp-0000qU-Fy; Tue, 07 May 2024 15:03:29 +0000
Received: by outflank-mailman (input) for mailman id 718260;
 Tue, 07 May 2024 15:03:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9mcW=MK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4MLn-0000qO-Vp
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 15:03:27 +0000
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [2607:f8b0:4864:20::72e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3ff391d-0c82-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 17:03:25 +0200 (CEST)
Received: by mail-qk1-x72e.google.com with SMTP id
 af79cd13be357-79291262dc2so311286685a.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 08:03:25 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 e17-20020a056214111100b006a0d0c6ed22sm4764978qvs.98.2024.05.07.08.03.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 May 2024 08:03:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3ff391d-0c82-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715094204; x=1715699004; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=44FclAYB4asT1fRHDFchMJxjb3ZZ8KHEZCSsQpGlMRM=;
        b=YA9m2+LMy+YB091eCb7Qi2x6iz9Sd3B5jFHNcZBmTKG50Hw+Up4qAjIj06IE3+MKEf
         ZALuFO/dl4WI4aIYKVZ7ujX03gR4iMX2E0qrU+fP5rxXXq1x9KH08/H0Sp5xOBDJ1Jx2
         WspQ0QsO4XJl2ZQm/jWf4kMXuSADhruws47L8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715094204; x=1715699004;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=44FclAYB4asT1fRHDFchMJxjb3ZZ8KHEZCSsQpGlMRM=;
        b=dXf8XoUzAR5ko9qR5+zbKkcyMejCuGAYDP6n+4nqHZv58d/q/M92eZaHFFY52K5HSN
         A03Yd0Ymo3d6rYRbXcNhbjKm7Rw38W8GCVbPur7tYkAmMZKr5q4twXy6bA4or3/vwW0C
         lwPw+LT7tAi/XojqCmiI6h3X4hbaMwAOpO3snDadLrfp6vE5pTThUK4sGsohI8Tclawe
         bddQ3nO93DncJpRDn2o/fynGBESA3uV/PvqF8s4agbC6XetSCbUW8SkV3fbag8fuBavF
         EXMqpZY381by85iROhGf44nTCca4g7aMzlWVcNIYDzfOLDfBsnmQXhM/1QhR5qhjrW6B
         8FbQ==
X-Gm-Message-State: AOJu0YwzamfcEqlnvDHqyDlwlaOdc2ATYGozmTktIWuppmpVtRCRy3yK
	uGriY0uyC/QsVQ2EX/dEBTxo/ps8878fr7BPDepjCEdM5wGcYViz9NnLveWO2IOpY7lnM60MMIZ
	Z
X-Google-Smtp-Source: AGHT+IGnDp0OO4TxUU5ic6SEpb+aJ+ukXOGAefHNY2KRBX7Hk5btmAcOF44NF3spaJqyafultRO1LA==
X-Received: by 2002:a05:6214:21eb:b0:69b:3c90:400f with SMTP id 6a1803df08f44-6a151569f0bmr437586d6.32.1715094204542;
        Tue, 07 May 2024 08:03:24 -0700 (PDT)
Message-ID: <57c14302-5e12-4101-bf52-1e88d7dd2532@citrix.com>
Date: Tue, 7 May 2024 16:03:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to Cascade
 Lake
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <20240507134540.1714274-1-andrew.cooper3@citrix.com>
 <Zjo5iwmEvD3RWtsx@macbook> <f0272d79-7c59-42b2-b185-2aaf84f1b196@citrix.com>
 <Zjo-pWunEGru6Z_K@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zjo-pWunEGru6Z_K@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/05/2024 3:45 pm, Roger Pau Monné wrote:
> On Tue, May 07, 2024 at 03:31:19PM +0100, Andrew Cooper wrote:
>> On 07/05/2024 3:24 pm, Roger Pau Monné wrote:
>>> On Tue, May 07, 2024 at 02:45:40PM +0100, Andrew Cooper wrote:
>>>> Ever since Xen 4.14, there has been a latent bug with migration.
>>>>
>>>> While some toolstacks can level the features properly, they don't shink
>>>> feat.max_subleaf when all features have been dropped.  This is because
>>>> we *still* have not completed the toolstack side work for full CPU Policy
>>>> objects.
>>>>
>>>> As a consequence, even when properly feature levelled, VMs can't migrate
>>>> "backwards" across hardware which reduces feat.max_subleaf.  One such example
>>>> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
>>>>
>>>> Extend the max policies feat.max_subleaf to the hightest number Xen knows
>>>> about, but leave the default policies matching the host.  This will allow VMs
>>>> with a higher feat.max_subleaf than strictly necessary to migrate in.
>>>>
>>>> Eventually we'll manage to teach the toolstack how to avoid creating such VMs
>>>> in the first place, but there's still more work to do there.
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Acked-by: Roger Pau Monné <roger.pau@citrix.com>
>> Thanks.
>>
>>> Even if we have just found one glitch with PSFD and Ice Lake vs
>>> Cascade Lack, wouldn't it be safer to always extend the max policies
>>> max leafs and subleafs to match the known array sizes?
>> This is the final max leaf (containing feature information) to gain
>> custom handling, I think?
> Couldn't the same happen with extended leaves?  Some of the extended
> leaves contain features, and hence for policy leveling toolstack might
> decide to zero them, yet extd.max_leaf won't be adjusted.

Hmm.  Right now, extd max leaf is also the one with the bit that we
unconditionally advertise, and it's inherited all the way from the host
policy.

So yes, in principle, but anything that bumps this limit is going to
have other implications too, and I'd prefer not to second-guess them at
this point.

I hope we can get the toolstack side fixes before this becomes a real
problem...

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 07 15:07:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 15:07:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718263.1120844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4MPR-0001vf-4Y; Tue, 07 May 2024 15:07:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718263.1120844; Tue, 07 May 2024 15:07:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4MPQ-0001vY-VP; Tue, 07 May 2024 15:07:12 +0000
Received: by outflank-mailman (input) for mailman id 718263;
 Tue, 07 May 2024 15:07:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4MPP-0001vO-Cv; Tue, 07 May 2024 15:07:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4MPP-0000Us-4V; Tue, 07 May 2024 15:07:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4MPO-0005rU-Pt; Tue, 07 May 2024 15:07:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4MPO-0001Vo-PA; Tue, 07 May 2024 15:07:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2QSDDKRnX/DJGbGX+jwhcCSWvQXQzIlQ/Rym1RaoY1Y=; b=5rcAopQ04sx/CaC/7+5qR6gXwM
	lKknIKk+9fLK98saQKD2VCaHSMR2+Lx+ZvSa0CK7c02277u+7+yU/eezLxVMQea99+itOHv8Ln8Wx
	SAOclXHCYZ9JayXElVrg2vzzDvSptJDV2zWf96vkZd/OskN5wqFLnLBMtOcxVMHLE3CQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185936-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185936: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
X-Osstest-Versions-That:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 15:07:10 +0000

flight 185936 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185936/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185929
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185929
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185929
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185929
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 185929
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185929
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185929
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
baseline version:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81

Last test of basis   185936  2024-05-07 05:53:32 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue May 07 15:22:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 15:22:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718273.1120853 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Mdv-0005hO-C9; Tue, 07 May 2024 15:22:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718273.1120853; Tue, 07 May 2024 15:22:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Mdv-0005hH-9e; Tue, 07 May 2024 15:22:11 +0000
Received: by outflank-mailman (input) for mailman id 718273;
 Tue, 07 May 2024 15:22:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YimZ=MK=amazon.co.uk=prvs=8506a7df7=eliasely@srs-se1.protection.inumbo.net>)
 id 1s4Mdt-0005hB-ET
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 15:22:09 +0000
Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90b34196-0c85-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 17:22:08 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-6002.iad6.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2024 15:22:03 +0000
Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:9527]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.8.30:2525] with
 esmtp (Farcaster)
 id 6f031f60-57de-41f3-851e-2a71332a4210; Tue, 7 May 2024 15:22:01 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Tue, 7 May 2024 15:22:01 +0000
Received: from [192.168.15.40] (10.106.83.8) by EX19D018EUA002.ant.amazon.com
 (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Tue, 7 May
 2024 15:21:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90b34196-0c85-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715095329; x=1746631329;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=yaFPD2Ekn21BDjcaiX5C+jqap0rCbSx8lXe5MsuDjWw=;
  b=lJt67S+EEzbTUd8+dw9q7iYXzFGmlO6IVbPgNc8X/OnJYQBV/6IqPqkO
   MaZrPB0dxltd3j+EJAckGNmIAQNunwmJqVSfal/xMs9+WS16UbxMMBAeI
   lyZHvygagl1sbWQCpiZFwdZTX7PH80ln24LCV00+TB4fqTzlucQhJ1lZC
   o=;
X-IronPort-AV: E=Sophos;i="6.08,142,1712620800"; 
   d="scan'208";a="405335772"
X-Farcaster-Flow-ID: 6f031f60-57de-41f3-851e-2a71332a4210
Message-ID: <e465d95c-f1c8-4857-9bb5-1c37a3c4cc80@amazon.com>
Date: Tue, 7 May 2024 16:21:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 09/27] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Julien Grall <jgrall@amazon.com>,
	<xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-10-eliasely@amazon.com>
 <7031861b-f925-479c-b236-8b1809996ede@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <7031861b-f925-479c-b236-8b1809996ede@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.106.83.8]
X-ClientProxiedBy: EX19D032UWB003.ant.amazon.com (10.13.139.165) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

 > On 20/02/2024 10:28, Jan Beulich wrote:
>> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>> --- a/xen/arch/x86/pv/dom0_build.c
>> +++ b/xen/arch/x86/pv/dom0_build.c
>> @@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
>>       l3_pgentry_t *l3tab = NULL, *l3start = NULL;
>>       l2_pgentry_t *l2tab = NULL, *l2start = NULL;
>>       l1_pgentry_t *l1tab = NULL, *l1start = NULL;
>> +    mfn_t l4start_mfn = INVALID_MFN;
>> +    mfn_t l3start_mfn = INVALID_MFN;
>> +    mfn_t l2start_mfn = INVALID_MFN;
>> +    mfn_t l1start_mfn = INVALID_MFN;
> 
> The reason initializers are needed here is, aiui, the overly large scope
> of these variables. For example ...
> 

Correct, is it just an observation or do you want me to do anything?

>> @@ -708,22 +712,32 @@ int __init dom0_construct_pv(struct domain *d,
>>           v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
>>       }
>>   
>> +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
>> +do {                                                    \
>> +    unmap_domain_page(virt_var);                        \
>> +    mfn_var = maddr_to_mfn(maddr);                      \
>> +    maddr += PAGE_SIZE;                                 \
>> +    virt_var = map_domain_page(mfn_var);                \
>> +} while ( false )
>> +
>>       if ( !compat )
>>       {
>>           maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
>> -        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
>> +        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
>> +        l4tab = l4start;
>>           clear_page(l4tab);
>> -        init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)),
>> -                          d, INVALID_MFN, true);
>> -        v->arch.guest_table = pagetable_from_paddr(__pa(l4start));
>> +        init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true);
>> +        v->arch.guest_table = pagetable_from_mfn(l4start_mfn);
> 
> ... looks to be required only here, while ...
> 
>>       }
>>       else
>>       {
>>           /* Monitor table already created by switch_compat(). */
>> -        l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table));
>> +        l4start_mfn = pagetable_get_mfn(v->arch.guest_table);
>> +        l4start = l4tab = map_domain_page(l4start_mfn);
> 
> ... in principle the use of the variable could be avoided here. Below
> from here there's no further use of it.
> 
>> @@ -781,30 +797,34 @@ int __init dom0_construct_pv(struct domain *d,
>>   
>>       if ( compat )
>>       {
>> -        l2_pgentry_t *l2t;
>> -
>>           /* Ensure the first four L3 entries are all populated. */
>>           for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
>>           {
>>               if ( !l3e_get_intpte(*l3tab) )
>>               {
>>                   maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
>> -                l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
>> -                clear_page(l2tab);
>> -                *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT);
>> +                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
>> +                clear_page(l2start);
>> +                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
>>               }
> 
> The updating of l2start is only conditional here, yet ...
> 
>>               if ( i == 3 )
>>                   l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
>>           }
>>   
>> -        l2t = map_l2t_from_l3e(l3start[3]);
>> -        init_xen_pae_l2_slots(l2t, d);
>> -        unmap_domain_page(l2t);
>> +        init_xen_pae_l2_slots(l2start, d);
> 
> ... here you assume it points at the page referenced by the 3rd L3 entry.

Hmm, I missed it when sending the revision and indeed it doesn't look 
correct.

> Question is why the original code is being replaced here in the first
> place: It was already suitably mapping the page in question.

The code was already suitably mapping the pages in question. This patch 
doesn't aim to make any functional change, just to rework how the 
domheap pages are used. The goal of the series is to remove the mappings 
from the directmap, which means those pages needs to be mapped and 
unmapped when required.

This is all this patch do, see `UNMAP_MAP_AND_ADVANCE()` macro.

Elias


From xen-devel-bounces@lists.xenproject.org Tue May 07 15:22:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 15:22:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718274.1120863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Me0-0005wO-K1; Tue, 07 May 2024 15:22:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718274.1120863; Tue, 07 May 2024 15:22:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Me0-0005wH-HA; Tue, 07 May 2024 15:22:16 +0000
Received: by outflank-mailman (input) for mailman id 718274;
 Tue, 07 May 2024 15:22:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YimZ=MK=amazon.co.uk=prvs=8506a7df7=eliasely@srs-se1.protection.inumbo.net>)
 id 1s4Mdz-0005vv-QE
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 15:22:15 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 938368bb-0c85-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 17:22:13 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2024 15:22:08 +0000
Received: from EX19MTAEUC001.ant.amazon.com [10.0.43.254:57861]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.47.37:2525]
 with esmtp (Farcaster)
 id c5e04267-e561-4916-a0ae-9e823ef5524c; Tue, 7 May 2024 15:22:07 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUC001.ant.amazon.com (10.252.51.193) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Tue, 7 May 2024 15:22:07 +0000
Received: from [192.168.15.40] (10.106.83.8) by EX19D018EUA002.ant.amazon.com
 (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Tue, 7 May
 2024 15:22:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 938368bb-0c85-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715095333; x=1746631333;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=ccs95s/rZ0fo0CsY7mPxJ280lq33oO29To/SWJqvuJI=;
  b=jdpqn34xwSqRcK5PGlevPW+GTuB4LtxTTs7zZ6T+8pqRyrSmIBxV3hYu
   RHDgYwKBfay7/tW1KnF6YwWGqvwdWRYUMDP1rNp83tIyFuixCQsz4+ar2
   h3lqaS1Fyf9ADiTzhExNb3h8obttPJ6jyC5CbT4jxsRuDqu6hdYLn8yxa
   Y=;
X-IronPort-AV: E=Sophos;i="6.08,142,1712620800"; 
   d="scan'208";a="87520326"
X-Farcaster-Flow-ID: c5e04267-e561-4916-a0ae-9e823ef5524c
Message-ID: <06e162de-55d6-4612-aa45-b1a350571d2b@amazon.com>
Date: Tue, 7 May 2024 16:22:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 11/27] x86: Lift mapcache variable to the arch
 level
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Wei Liu
	<wei.liu2@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Wei Wang <wawei@amazon.de>, Hongyan Xia <hongyxia@amazon.com>,
	Julien Grall <jgrall@amazon.com>, <xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-12-eliasely@amazon.com>
 <1ff19cb2-851d-40cf-bb12-5b7074eee0b8@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <1ff19cb2-851d-40cf-bb12-5b7074eee0b8@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.106.83.8]
X-ClientProxiedBy: EX19D032UWB003.ant.amazon.com (10.13.139.165) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

>> This only lifts the mapcache variable up. Whether we populate the
>> mapcache for a domain is unchanged in this patch.
> 
> Is it? I wonder because of ...
>

I agree, the commit message doesn't completely reflect the changes below.

>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -843,6 +843,8 @@ int arch_domain_create(struct domain *d,
>>   
>>       psr_domain_init(d);
>>   
>> +    mapcache_domain_init(d);
>> +
>>       if ( is_hvm_domain(d) )
>>       {
>>           if ( (rc = hvm_domain_initialise(d, config)) != 0 )
>> @@ -850,8 +852,6 @@ int arch_domain_create(struct domain *d,
>>       }
>>       else if ( is_pv_domain(d) )
>>       {
>> -        mapcache_domain_init(d);
>> -
>>           if ( (rc = pv_domain_initialise(d)) != 0 )
>>               goto fail;
>>       }
> 
> ... this and ...
> 
>> --- a/xen/arch/x86/domain_page.c
>> +++ b/xen/arch/x86/domain_page.c
>> @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
>>   #endif
>>   
>>       v = mapcache_current_vcpu();
>> -    if ( !v || !is_pv_vcpu(v) )
>> +    if ( !v )
>>           return mfn_to_virt(mfn_x(mfn));
> 
> ... this and yet more changes indicating otherwise.
> 
> Yet if which domains have a mapcache set up is already changed here, I
> wonder whether the idle domain shouldn't be taken care of here as well.

Do you suggest to fold here the following patch where the mapcache gets 
initialized for idle domains?


Elias


From xen-devel-bounces@lists.xenproject.org Tue May 07 15:25:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 15:25:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718279.1120873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Mh5-00073t-0J; Tue, 07 May 2024 15:25:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718279.1120873; Tue, 07 May 2024 15:25:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Mh4-00073m-TR; Tue, 07 May 2024 15:25:26 +0000
Received: by outflank-mailman (input) for mailman id 718279;
 Tue, 07 May 2024 15:25:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YimZ=MK=amazon.co.uk=prvs=8506a7df7=eliasely@srs-se1.protection.inumbo.net>)
 id 1s4Mh3-00073e-L6
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 15:25:25 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 04d20811-0c86-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 17:25:23 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2024 15:25:19 +0000
Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:22996]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.11.44:2525]
 with esmtp (Farcaster)
 id 93c06852-df1b-488d-ac33-f5dfbd568114; Tue, 7 May 2024 15:25:18 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Tue, 7 May 2024 15:25:18 +0000
Received: from [192.168.15.40] (10.106.83.8) by EX19D018EUA002.ant.amazon.com
 (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Tue, 7 May
 2024 15:25:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04d20811-0c86-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715095523; x=1746631523;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=Gr9kwZ+RTjeaE5vmYv137GFBoQHoDJVWeG4hdNIu1V8=;
  b=YAvU4Nwp2CFIvLN3QtzWaYvqEw+OgL0IJRqE9TYFsBTMEvOhJiqWbTgV
   XZDNlHqTXstjusmiobYa5YFkT4aN/oivRCk598S4qci+Uo92c1pzAjqIl
   cu3QuEcDXrTzxCq/x1Z3yCRBHTv9bW9KYE2iwythlBMLQktJ7hGvyYNj2
   Q=;
X-IronPort-AV: E=Sophos;i="6.08,142,1712620800"; 
   d="scan'208";a="87615136"
X-Farcaster-Flow-ID: 93c06852-df1b-488d-ac33-f5dfbd568114
Message-ID: <5b1e5eac-df9b-48b0-ba14-9c56f32a5d9c@amazon.com>
Date: Tue, 7 May 2024 16:25:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 12/27] x86/mapcache: Initialise the mapcache
 for the idle domain
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Wei Wang <wawei@amazon.de>, Julien Grall <jgrall@amazon.com>,
	<xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-13-eliasely@amazon.com>
 <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.106.83.8]
X-ClientProxiedBy: EX19D031UWA002.ant.amazon.com (10.13.139.96) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

On 20/02/2024 10:51, Jan Beulich wrote:
> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -750,9 +750,16 @@ int arch_domain_create(struct domain *d,
>>   
>>       spin_lock_init(&d->arch.e820_lock);
>>   
>> +    if ( (rc = mapcache_domain_init(d)) != 0)
>> +    {
>> +        free_perdomain_mappings(d);
>> +        return rc;
>> +    }
>> +
>>       /* Minimal initialisation for the idle domain. */
>>       if ( unlikely(is_idle_domain(d)) )
>>       {
>> +        struct page_info *pg = d->arch.perdomain_l3_pg;
>>           static const struct arch_csw idle_csw = {
>>               .from = paravirt_ctxt_switch_from,
>>               .to   = paravirt_ctxt_switch_to,
>> @@ -763,6 +770,9 @@ int arch_domain_create(struct domain *d,
>>   
>>           d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>>   
>> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
>> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
>> +
>>           return 0;
>>       }
> 
> Why not add another call to mapcache_domain_init() right here, allowing
> a more specific panic() to be invoked in case of failure (compared to
> the BUG_ON() upon failure of creation of the idle domain as a whole)?
> Then the other mapcache_domain_init() call doesn't need moving a 2nd
> time in close succession.


Sorry but I don't get your point, why calling another time 
`mapcache_domain_init()`? What panic() are you referring to?

Elias


From xen-devel-bounces@lists.xenproject.org Tue May 07 16:16:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 16:16:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718297.1120882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4NUC-0008CF-L0; Tue, 07 May 2024 16:16:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718297.1120882; Tue, 07 May 2024 16:16:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4NUC-0008C8-ID; Tue, 07 May 2024 16:16:12 +0000
Received: by outflank-mailman (input) for mailman id 718297;
 Tue, 07 May 2024 16:16:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7cCq=MK=flex--seanjc.bounces.google.com=3x1M6ZgYKCZACyu73w08805y.w86Hy7-xyFy552CDC.Hy79B83ywD.8B0@srs-se1.protection.inumbo.net>)
 id 1s4NUA-0008C2-Uc
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 16:16:10 +0000
Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com
 [2607:f8b0:4864:20::b49])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ceca4df-0c8d-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 18:16:09 +0200 (CEST)
Received: by mail-yb1-xb49.google.com with SMTP id
 3f1490d57ef6-de59ff8af0bso5147616276.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 09:16:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ceca4df-0c8d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20230601; t=1715098568; x=1715703368; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id
         :reply-to;
        bh=IWxS0mCdKrqU6bVOz4n3NTmQy0CRO7Urzkw5a/fWZlQ=;
        b=AZOcz3z8AQqXNYWVb7Qf+KlHCB/MUVXVgKcNyqEJ5QtrLEmHvwRmqiz5x7YZ8Fe63U
         Fjzvut1HibbXzGBAFhGVDfn9iM1XMfc0RmCUKY4xI5ESKKMLurOcV7lzQG/3M0cSy5oU
         RnIM2MpfCh3k0Ntu09ZgL3pCP4OCoOVKVv4/NsFQWZbjJKZjK4XVM/609NMuR2E6JIx1
         TncTFU1KNSizpokwTHV/w1dfQUqlfE9f74mLfEvHymGfnmKxOi8l3n5/MD+TIPI2C4c4
         Mrd3lio5topPsRPO67WCvGEhLZiDd2DWxzKypiSOjQGk7Lt6aJiAxH17E6Bf2DkNi1q4
         fyrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715098568; x=1715703368;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=IWxS0mCdKrqU6bVOz4n3NTmQy0CRO7Urzkw5a/fWZlQ=;
        b=PmvyweHrkk7bJA6M/jiiRZGnYrGjzA520BcnTj3EQt0IRU6k87euYc4ihqzK8rPvRJ
         xlSjumEviayM7f4AzjvfU7GiQnoaX6Xzg0P/FrFtnc0ROCH6jcDlgEabM3FA9sx/INzy
         1Am+eI0Vpd2rbDeSkGAQFEHNgJK4SqWqsej2ERleyw6GOzFutgwSLpSiDQjPYydoQhop
         qs+DwB6jb6tgOAMYsdmCiujOJ4d/3pVpHkT8BvF/M6Gupg+l7QOVz+hsFEyYDQ1nOpte
         J2qvFCz03Rlila1J8NB/V6P9Si4v3KcUs7NAQFWEQnZXTQuYJsENpEbymTu+Uw6DLeCd
         XAHg==
X-Forwarded-Encrypted: i=1; AJvYcCXZjnV4dt26998G9XB7klDNu4gbL1hlwtQDtuyTgpu1yw2MUIPuLdpjottBEsy9ReZJWDIAvwfc7USLFEehSuowWSWZkcj6lRfBUjx6Yis=
X-Gm-Message-State: AOJu0Yy5MYzuy+Xh1rpiJmSf3+JkSnpxRNfXRAvO5lSfIEqB4agKbcDz
	5cLXfcJ1InWny2gbdK5sVbbFc3A5X+N3lbaUyZKW71aW7UZ1btWCos3KrC5jON+HOJ+VIM6KM8n
	kog==
X-Google-Smtp-Source: AGHT+IEEPkTwPJaOTGouRKDc7DVHMnqTRjb0yesgQ4YUUdFhPv8WpJzXBwRvcXmUBR39ssSE6I5KsNiqYrA=
X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37])
 (user=seanjc job=sendgmr) by 2002:a05:6902:707:b0:dbe:30cd:8fcb with SMTP id
 3f1490d57ef6-debb9c0032emr11950276.0.1715098567940; Tue, 07 May 2024 09:16:07
 -0700 (PDT)
Date: Tue, 7 May 2024 09:16:06 -0700
In-Reply-To: <20240507.ieghomae0UoC@digikod.net>
Mime-Version: 1.0
References: <20240503131910.307630-1-mic@digikod.net> <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com> <20240506.ohwe7eewu0oB@digikod.net>
 <ZjmFPZd5q_hEBdBz@google.com> <20240507.ieghomae0UoC@digikod.net>
Message-ID: <ZjpTxt-Bxia3bRwB@google.com>
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
From: Sean Christopherson <seanjc@google.com>
To: "=?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?=" <mic@digikod.net>
Cc: Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	"Mihai =?utf-8?B?RG9uyJt1?=" <mdontu@bitdefender.com>, 
	"=?utf-8?B?TmljdciZb3IgQ8OuyJt1?=" <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, 
	Will Deacon <will@kernel.org>, Yu Zhang <yu.c.zhang@linux.intel.com>, 
	"=?utf-8?Q?=C8=98tefan_=C8=98icleru?=" <ssicleru@bitdefender.com>, dev@lists.cloudhypervisor.org, 
	kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, 
	virtualization@lists.linux-foundation.org, x86@kernel.org, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 07, 2024, Micka=C3=ABl Sala=C3=BCn wrote:
> > Actually, potential bad/crazy idea.  Why does the _host_ need to define=
 policy?
> > Linux already knows what assets it wants to (un)protect and when.  What=
's missing
> > is a way for the guest kernel to effectively deprivilege and re-authent=
icate
> > itself as needed.  We've been tossing around the idea of paired VMs+vCP=
Us to
> > support VTLs and SEV's VMPLs, what if we usurped/piggybacked those idea=
s, with a
> > bit of pKVM mixed in?
> >=20
> > Borrowing VTL terminology, where VTL0 is the least privileged, userspac=
e launches
> > the VM at VTL0.  At some point, the guest triggers the deprivileging se=
quence and
> > userspace creates VTL1.  Userpace also provides a way for VTL0 restrict=
 access to
> > its memory, e.g. to effectively make the page tables for the kernel's d=
irect map
> > writable only from VTL1, to make kernel text RO (or XO), etc.  And VTL0=
 could then
> > also completely remove its access to code that changes CR0/CR4.
> >=20
> > It would obviously require a _lot_ more upfront work, e.g. to isolate t=
he kernel
> > text that modifies CR0/CR4 so that it can be removed from VTL0, but tha=
t should
> > be doable with annotations, e.g. tag relevant functions with __magic or=
 whatever,
> > throw them in a dedicated section, and then free/protect the section(s)=
 at the
> > appropriate time.
> >=20
> > KVM would likely need to provide the ability to switch VTLs (or whateve=
r they get
> > called), and host userspace would need to provide a decent amount of th=
e backend
> > mechanisms and "core" policies, e.g. to manage VTL0 memory, teardown (t=
urn off?)
> > VTL1 on kexec(), etc.  But everything else could live in the guest kern=
el itself.
> > E.g. to have CR pinning play nice with kexec(), toss the relevant kexec=
() code into
> > VTL1.  That way VTL1 can verify the kexec() target and tear itself down=
 before
> > jumping into the new kernel.=20
> >=20
> > This is very off the cuff and have-wavy, e.g. I don't have much of an i=
dea what
> > it would take to harden kernel text patching, but keeping the policy in=
 the guest
> > seems like it'd make everything more tractable than trying to define an=
 ABI
> > between Linux and a VMM that is rich and flexible enough to support all=
 the
> > fancy things Linux does (and will do in the future).
>=20
> Yes, we agree that the guest needs to manage its own policy.  That's why
> we implemented Heki for KVM this way, but without VTLs because KVM
> doesn't support them.
>=20
> To sum up, is the VTL approach the only one that would be acceptable for
> KVM? =20

Heh, that's not a question you want to be asking.  You're effectively askin=
g me
to make an authorative, "final" decision on a topic which I am only passing=
ly
familiar with.

But since you asked it... :-)  Probably?

I see a lot of advantages to a VTL/VSM-like approach:

 1. Provides Linux-as-a guest the flexibility it needs to meaningfully adva=
nce
    its security, with the least amount of policy built into the guest/host=
 ABI.

 2. Largely decouples guest policy from the host, i.e. should allow the gue=
st to
    evolve/update it's policy without needing to coordinate changes with th=
e host.

 3. The KVM implementation can be generic enough to be reusable for other f=
eatures.

 4. Other groups are already working on VTL-like support in KVM, e.g. for V=
SM
    itself, and potentially for VMPL/SVSM support.

IMO, #2 is a *huge* selling point.  Not having to coordinate changes across
multiple code bases and/or organizations and/or maintainers is a big win fo=
r
velocity, long term maintenance, and probably the very viability of HEKI.

Providing the guest with the tools to define and implement its own policy m=
eans
end users don't have to way for some third party, e.g. CSPs, to deploy the
accompanying host-side changes, because there are no host-side changes.

And encapsulating everything in the guest drastically reduces the friction =
with
changes in the kernel that interact with hardening, both from a technical a=
nd a
social perspective.  I.e. giving the kernel (near) complete control over it=
s
destiny minimizes the number of moving parts, and will be far, far easier t=
o sell
to maintainers.  I would expect maintainers to react much more favorably to=
 being
handed tools to harden the kernel, as opposed to being presented a set of A=
PIs
that can be used to make the kernel compliant with _someone else's_ vision =
of
what kernel hardening should look like.

E.g. imagine a new feature comes along that requires overriding CR0/CR4 pin=
ning
in a way that doesn't fit into existing policy.  If the VMM is involved in
defining/enforcing the CR pinning policy, then supporting said new feature =
would
require new guest/host ABI and an updated host VMM in order to make the new
feature compatible with HEKI.  Inevitably, even if everything goes smoothly=
 from
an upstreaming perspective, that will result in guests that have to choose =
between
HEKI and new feature X, because there is zero chance that all hosts that ru=
n Linux
as a guest will be updated in advance of new feature X being deployed.

And if/when things don't go smoothly, odds are very good that kernel mainta=
iners
will eventually tire of having to coordinate and negotiate with QEMU and ot=
her
VMMs, and will become resistant to continuing to support/extend HEKI.

> If yes, that would indeed require a *lot* of work for something we're not
> sure will be accepted later on.

Yes and no.  The AWS folks are pursuing VSM support in KVM+QEMU, and SVSM s=
upport
is trending toward the paired VM+vCPU model.  IMO, it's entirely feasible t=
o
design KVM support such that much of the development load can be shared bet=
ween
the projects.  And having 2+ use cases for a feature (set) makes it _much_ =
more
likely that the feature(s) will be accepted.

And similar to what Paolo said regarding HEKI not having a complete story, =
I
don't see a clear line of sight for landing host-defined policy enforcement=
, as
there are many open, non-trivial questions that need answers. I.e. upstream=
ing
HEKI in its current form is also far from a done deal, and isn't guaranteed=
 to
be substantially less work when all is said and done.


From xen-devel-bounces@lists.xenproject.org Tue May 07 16:55:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 16:55:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718320.1120905 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4O6M-0006B8-NO; Tue, 07 May 2024 16:55:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718320.1120905; Tue, 07 May 2024 16:55:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4O6M-0006B0-KF; Tue, 07 May 2024 16:55:38 +0000
Received: by outflank-mailman (input) for mailman id 718320;
 Tue, 07 May 2024 16:55:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0blL=MK=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s4O6L-0006Ar-Kf
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 16:55:37 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a02cb858-0c92-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 18:55:36 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59a0e4b773so813467166b.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 09:55:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a02cb858-0c92-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715100936; x=1715705736; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9r1PyM22ApUZwyfRX8L8ckcNtjRIFNBKB4u9h89oKmY=;
        b=RkpNj2mQxvl3JLmMIHucW6xkZflAeIVXBWGOLb+4TL5fJ0shkNw6VIfbQhFsr7q9kg
         1OOZqc2nInYi4QvqRGVOsjPHL0FAQukGk/6arFHviZsa+Zvcd1xyUk7jaIgREQgTKZAQ
         0V9V8wLFI+VZHWjSUn7TdB7RDGh4C+iIF55B80ehyWraRcEg4P4PhO3RNo7RWRjtoyIy
         pBf5kBCHcm1bdY3fVWbkuHEAyANNVegSLNXAKOa0/sXp1EuWSWwkScisHyH3pt9x8m/9
         tzSXazEiDNwdXtQ7yaHzvdx0gcZqz3g5DabhGpSY3bqkMpMJuZdc1rba+ijehUiSxWEU
         1HPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715100936; x=1715705736;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9r1PyM22ApUZwyfRX8L8ckcNtjRIFNBKB4u9h89oKmY=;
        b=SJjWw+XNQKARYahZs4HCake10MuisVZZRLQywANhNRTjRlN/F+xmqFyRTLc9VgX9Cq
         Mfya3PgHSAk98PvcvuEfVQjG5JVuqb1XnxP49KGmE8QqLPsPOPKRJM9C1KRYQAci3cDf
         L5UNPIEOZPqlsoDlByK583XtzYZZI385clkDtqhkK0U30l2PXkEW85EYANtg2fq7feqp
         99Vo5xBXyi/5Ka17uNdDZc1u+zIrVX+rl4e9fdN36ut8KQUSDL2thXFViXgxo8WFWvVn
         JLSMEJeNEd0dAUuiCtfRqYCmgqpay+zji+RYwcC1p4KI4KMAN7PCd3pW/9b8qwxeDobN
         JkPA==
X-Forwarded-Encrypted: i=1; AJvYcCUUDGvnAbumkMa7NPO8LbGCTDs5kWnO4kIaMUZV/37bLD7At4Fcs2tA6bm/DglfxQE2xLp475bfFYxqIMFQNEae0QMkJhZBYuACugJkOSE=
X-Gm-Message-State: AOJu0Yy9NznuNY+OhMOM7qiN2vnDLQhVwC7GQi1MUdKjZ1bgzkoQyo/v
	SAs0g1dMNPYK/TbBsQMJplfbGUZ8/lV0r+WqKCGz7xjjCT4CE/PKODJbnqYrvYptc6TwQ1Uu4GC
	AN2jg+tNiGmMRo3i6MBGMY77mFlg=
X-Google-Smtp-Source: AGHT+IGp2NBJhmznoejOk1f0f9GHTGN+JUTUgQodCAVivgpT9rDkTiLpV5SCdP3UumowcFdDnO9GD0s/BuHUhccKv6Y=
X-Received: by 2002:a50:a687:0:b0:56c:5a12:ac53 with SMTP id
 4fb4d7f45d1cf-5731d9f1788mr201250a12.24.1715100935761; Tue, 07 May 2024
 09:55:35 -0700 (PDT)
MIME-Version: 1.0
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
 <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com> <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org>
In-Reply-To: <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Tue, 7 May 2024 18:55:22 +0200
Message-ID: <CAJy5ezpNQP5q0NVaGdXtt5BbJSzCMfTfFt-0RRgVq3+DqMbKgA@mail.gmail.com>
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol annotations
To: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org, 
	bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
	jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 7, 2024 at 11:57=E2=80=AFAM Julien Grall <julien@xen.org> wrote=
:
>
> Hi,
>
> On 06/05/2024 13:54, Edgar E. Iglesias wrote:
> > On Sat, May 4, 2024 at 2:14=E2=80=AFAM Stefano Stabellini
> > <sstabellini@kernel.org> wrote:
> >>
> >> On Wed, 1 May 2024, Edgar E. Iglesias wrote:
> >>> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >>>
> >>> Use the generic xen/linkage.h macros to annotate code symbols
> >>> and add missing annotations.
> >>>
> >>> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> >>> ---
> >>>   xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
> >>>   1 file changed, 12 insertions(+), 8 deletions(-)
> >>>
> >>> diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
> >>> index 4e63825220..b16e4d1e29 100644
> >>> --- a/xen/arch/arm/arm64/bpi.S
> >>> +++ b/xen/arch/arm/arm64/bpi.S
> >>> @@ -52,14 +52,15 @@
> >>>    * micro-architectures in a system.
> >>>    */
> >>>       .align   11
> >>> -ENTRY(__bp_harden_hyp_vecs_start)
> >>> +FUNC(__bp_harden_hyp_vecs_start)
> >>>       .rept 4
> >>>       vectors hyp_traps_vector
> >>>       .endr
> >>> -ENTRY(__bp_harden_hyp_vecs_end)
> >>> +GLOBAL(__bp_harden_hyp_vecs_end)
> >>> +END(__bp_harden_hyp_vecs_start)
> >>
> >> Shouldn't GLOBAL be changed to FUNC as well?
> >>
> >
> > I was a bit unsure but went for GLOBAL since the _end labels point to
> > addresses after and outside of the code sequence.
> > But I don't have a strong opinion and am happy to change them to FUNC
> > if you feel that's better.
>
> I don't think it should be FUNC as this is not meant to be called
> directly. I am also under the impression, we were planning to get rid of
> GLOBAL() as well.
>
> Furthermore, __bp_harden_hyp_vec_start is not a function per say. It is
> a pointer to the vector table.
>
>  From the brief look, the same remarks would apply to the rest of bpi.S.
> So I think we want to switch all the ENTRY() to LABEL().

Hi Julien,

The reason I choose FUNC for the start of the symbol is because these
symbols contain
executable code (not only a table of pointers to code somewhere else)
and the ELF spec
says that STT_FUNC means the symbol contains functions or other executable
code (not only callable functions IIUC):

"STT_FUNC The symbol is associated with a function or other executable code=
."
https://refspecs.linuxbase.org/elf/elf.pdf
(Symbol Table 1-20).

I think using LABEL instead of GLOBAL for the _end labels of these
code sequences makes sense.
I'm happy to change the _start labels to LABEL too if you guys feel
that's better.

Cheers,
Edgar


From xen-devel-bounces@lists.xenproject.org Tue May 07 17:13:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 17:13:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718325.1120916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ON3-0000cE-Sl; Tue, 07 May 2024 17:12:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718325.1120916; Tue, 07 May 2024 17:12:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ON3-0000c6-Pr; Tue, 07 May 2024 17:12:53 +0000
Received: by outflank-mailman (input) for mailman id 718325;
 Tue, 07 May 2024 17:12:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=b+jw=MK=xenbits.xen.org=andrewcoop@srs-se1.protection.inumbo.net>)
 id 1s4ON2-0000bq-0C
 for xen-devel@lists.xen.org; Tue, 07 May 2024 17:12:52 +0000
Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07c5027c-0c95-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 19:12:50 +0200 (CEST)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1s4OMs-0003Fk-Qq; Tue, 07 May 2024 17:12:42 +0000
Received: from andrewcoop by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1s4OMs-0002AQ-Oo; Tue, 07 May 2024 17:12:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07c5027c-0c95-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=xm3rIbfTJrlQM5CEIPJ8p75FFBGnHoEQ30OngBr9mv0=; b=yVMUvwkxlJBei7PPdyHvy6+gdu
	mBNmeoDu86YQnBGCbKJ9ZzaOcrOSR2A+uZ0WJIKPQ6gvcXjD8SASdS2vgLg5IqHRFwy3QdRi46Ey/
	DiragXdDijWwZlz9ZhCCgzBlXbSOdEbUkw8kshtk1/3CviTW6DMIIvOpFeDkk6euQvRw=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 456 v3 (CVE-2024-2201) - x86: Native Branch
 History Injection
Message-Id: <E1s4OMs-0002AQ-Oo@xenbits.xenproject.org>
Date: Tue, 07 May 2024 17:12:42 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

            Xen Security Advisory CVE-2024-2201 / XSA-456
                              version 3

                 x86: Native Branch History Injection

UPDATES IN VERSION 3
====================

Issues were found with the original code changes.  See the bottom of the
Resolution section for how to obtain those.

ISSUE DESCRIPTION
=================

In August 2022, researchers at VU Amsterdam disclosed Spectre-BHB.

Spectre-BHB was discussed in XSA-398.  At the time, the susceptibility
of Xen to Spectre-BHB was uncertain so no specific action was taken in
XSA-398.  However, various changes were made thereafter in upstream Xen
as a consequence; more on these later.

VU Amsterdam have subsequently adjusted the attack to be pulled off
entirely from userspace, without the aid of a managed runtime in the
victim context.

For more details, see:
  https://vusec.net/projects/native-bhi
  https://vusec.net/projects/bhi-spectre-bhb
  https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html
  https://xenbits.xen.org/xsa/advisory-398.html

IMPACT
======

An attacker might be able to infer the contents of arbitrary host
memory, including memory assigned to other guests.

VULNERABLE SYSTEMS
==================

Systems running all versions of Xen are affected.

Only Intel x86 CPUs are potentially affected.  CPUs from other
manufacturers are not known to be affected.

A wide range of Intel CPUs employ Branch History prediction techniques.
However for older CPUs existing Spectre-v2 mitigations (XSA-254) are
believed to be sufficient to mitigate Native-BHI.

Therefore, the rest of the discussion will be limited in scope to the
CPUs for which a change in behaviour is expected.  These are believed to
be all CPUs with eIBRS (Enhanced IBRS, a.k.a. IBRS_ALL or IBRS_ATT).
eIBRS signifies a hardware adjustment (mode-tagged indirect predictions)
designed to combat Spectre-v2, available in CPUs from 2019 onwards.

To determine if a system has eIBRS, run `xen-cpuid -v` in dom0, looking for
the string "eibrs" in the Dynamic Raw block of information.  e.g.

  # xen-cpuid -v
  ...
  Dynamic sets:
  Raw                             ...
    ...
    [16] MSR_ARCH_CAPS.lo         ... eibrs ...
    ...
  ...

Be aware that the Static sets are compile time information so will include the
string "eibrs" irrespective of hardware support.  If there is no row for "[16]
MSR_ARCH_CAPS.lo" then the fixes for XSA-435 are missing.

MITIGATION
==========

There are no mitigations.

CREDITS
=======

This issue was discovered by VU Amsterdam.

RESOLUTION
==========

In Xen 4.17, in response to the original Spectre-BHB, CET-IBT support was
added to Xen to use on capable hardware.  It also came with work to remove
unnecessary function pointers, and to de-virtualise function pointers at boot,
as both a performance and hardening improvement.  This work has been steadily
continuing since, and every removed/de-virtualised function pointer reduces
the options available to an adversary trying to mount a Native-BHI attack.
All of this work has been backported to 4.17 and later for this advisory.

Beginning with the Intel Alder Lake (Client) and Sapphire Rapids (Server)
CPUs, a hardware control called BHI_DIS_S is available, which restricts
history-based predictions.  This control requires updated microcode on some
CPUs.  Look for "bhi-ctrl" in `xen-cpuid -v`, similar to eibrs above.

Xen has been updated to use this control when available, and to virtualise it
for guests to use.

For CPUs without BHI_DIS_S, BHB clearing sequences need using.  Out of an
abundance of caution, all sequences in the Intel whitepaper have been
implemented, although Xen will only use the "short" sequence by default.  The
others are available to opt in to.

The work to mitigate Native-BHI is extensive, and the backports are
more-extensive still.

Therefore, we have decided to produce new releases on all stable trees.
Please find fixes in the respective branches under the following release
tags:

  RELEASE-4.18.2
  RELEASE-4.17.4
  RELEASE-4.16.6
  RELEASE-4.15.6

Other release activities (tarballs, announcements, etc) will happen in
due course.

Issues were in those found subsequently.  To address those, newer commits
from the stable branches need updating to, in particular

stable-4.15	056500003eb44314cb90f2e3e7b2d405e86b5657
stable-4.16	d0e8f8ffbb19b5df5f767328baeb54c069b08e6a
stable-4.17	effcf70f020ff12d34c80e2abde0ecb00ce92bda
stable-4.18	f0ff1d9cb96041a84a24857a6464628240deed4f

For 4.15, since we're closing the branch, RELEASE-4.15.7 was tagged in
addition; other release activities - as per above - will follow.

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmY6YBEMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZxKYH/1nCaEsJ6TvDBFc3uB9kLuwcOye56KPa2PQ+9Cqs
5OG+zKG7jaMtPVvsyZQF/o2EHk+Ypj0LbZMQKwvPnV9ajZvACa7ZmJd32XamJpnf
+v4UzCWjKd4seCsdySneU3ap4MhBcOhKXzDMts29dGPNdeol67RrsMjAkuVeDl86
q/VSZnBn6VuiujYS0BtqG1ZB4vVA6nyQccbbprXyi4MMdikrIlaiQqFqCNRdh+lS
tGfhNDRVXcX8ipq+MkHKf+aE3qP/CTWollqgB0eEYi1IPjCJhMIqP4DP8DLvAA+L
Ab3gSVVfvp6IgaFZpVu+Y5T6Z5S6TlKU1YDtR7UE/fML8Ns=
=K66w
-----END PGP SIGNATURE-----

--=separator--


From xen-devel-bounces@lists.xenproject.org Tue May 07 17:13:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 17:13:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718331.1120975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ONK-00028O-8g; Tue, 07 May 2024 17:13:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718331.1120975; Tue, 07 May 2024 17:13:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ONK-00028H-4r; Tue, 07 May 2024 17:13:10 +0000
Received: by outflank-mailman (input) for mailman id 718331;
 Tue, 07 May 2024 17:13:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=b+jw=MK=xenbits.xen.org=andrewcoop@srs-se1.protection.inumbo.net>)
 id 1s4ONI-0000bw-M1
 for xen-devel@lists.xen.org; Tue, 07 May 2024 17:13:08 +0000
Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0fb129d1-0c95-11ef-b4bb-af5377834399;
 Tue, 07 May 2024 19:13:04 +0200 (CEST)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1s4ON4-0003GZ-Ao; Tue, 07 May 2024 17:12:54 +0000
Received: from andrewcoop by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1s4ON4-0002CG-9R; Tue, 07 May 2024 17:12:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fb129d1-0c95-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=bjJTywfWaY2JlU0ugZOq5fY/zAJLvPujOreQwU8D4R4=; b=X1KAGgf7pWupjNkulgkUiPk5f5
	OxvAaTXUYiW8WuXBN42V0BNdpswxKPcrlUVMLfd/VH5Is0O0MZISmDKqa9aaq4l17pKCJrcCuGsJ1
	UwnDNkDtHEupN/LOfx5MVAhTGN23ZE5TFJBIvnXnR06qvqJpRiS1Izo+9N3iLr53658Y=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 457 v1 - Linux/xen-netback: Memory leak due
 to missing cleanup function
Message-Id: <E1s4ON4-0002CG-9R@xenbits.xenproject.org>
Date: Tue, 07 May 2024 17:12:54 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

                    Xen Security Advisory XSA-457

    Linux/xen-netback: Memory leak due to missing cleanup function

ISSUE DESCRIPTION
=================

In netback, xennet_alloc_one_rx_buffer() failed to call the
appropriate clean-up function, resulting in a memory leak.

IMPACT
======

A malicious guest userspace process can exhaust memory resources
within the guest kernel, potentially leading to a system crash (Denial
of Service). It is not known whether it can be triggered remotely.

VULNERABLE SYSTEMS
==================

Systems with guests running Linux 5.9 and later with Xen PV network
devices are affected.

MITIGATION
==========

For HVM guests, using emulated network devices will avoid this issue.

RESOLUTION
==========

The following patch in Linux resolves the issue:

https://git.kernel.org/torvalds/c/037965402a010898d34f4e35327d22c0a95cd51f

A copy of which has attached.

xsa457.patch           Linux 5.9

$ sha256sum xsa457*
9d6ae3da27f1ff92f9f45c800822beecda603d6dea6726207cee6c768416114c  xsa457.patch
$


NOTE ON THE LACK OF EMBARGO
===========================

The issue was reported initially on a public bug tracker and fixed in
public before it was realized that there was a security aspect.
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmY6YN8MHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZq4kH/0BcaF/4dKqxQ/hYMMoLxcE1kzHn2kAdFPcvxcuu
Csk1yLugbvxHgwgp0lI9JjiqzSMt68pN8B9mWbcMBBvA7jGGsJ6Vjp25kQnUToLe
FPiAhW/TY+1YXOnhsfn9dHHk1Tv0W5D69QuUuj6zGUvRMdV+WPyA/mGPWnBrJgT+
5s6tKFxls1JiLdFxuJKqi8Ok8HrX1zE9unSWEUri8SNE2k3h5i29X2v+S8yBv2y0
XBnzr16kL9KKim0sNSErB1QU5BThnDBCFk+7FKAAYGAv5H6N3VLv66DLARCYfPhP
iXJU3/+yvAjwZjp5oYtbqHXzdd/m0b/IrF/0ZMLBaoDs0s4=
=vfs6
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa457.patch"
Content-Disposition: attachment; filename="xsa457.patch"
Content-Transfer-Encoding: base64

RnJvbSAwMzc5NjU0MDJhMDEwODk4ZDM0ZjRlMzUzMjdkMjJjMGE5NWNkNTFm
IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKZXNwZXIgRGFuZ2Fh
cmQgQnJvdWVyIDxoYXdrQGtlcm5lbC5vcmc+CkRhdGU6IFdlZCwgMjcgTWFy
IDIwMjQgMTM6MTQ6NTYgKzAxMDAKU3ViamVjdDogeGVuLW5ldGZyb250OiBB
ZGQgbWlzc2luZyBza2JfbWFya19mb3JfcmVjeWNsZQoKTm90aWNlIHRoYXQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKSBpcyBpbnRyb2R1Y2VkIGxhdGVyIHRo
YW4gZml4ZXMgdGFnIGluCmNvbW1pdCA2YTViY2Q4NGU4ODYgKCJwYWdlX3Bv
b2w6IEFsbG93IGRyaXZlcnMgdG8gaGludCBvbiBTS0IgcmVjeWNsaW5nIiku
CgpJdCBpcyBiZWxpZXZlZCB0aGF0IGZpeGVzIHRhZyB3ZXJlIG1pc3Npbmcg
YSBjYWxsIHRvIHBhZ2VfcG9vbF9yZWxlYXNlX3BhZ2UoKQpiZXR3ZWVuIHY1
LjkgdG8gdjUuMTQsIGFmdGVyIHdoaWNoIGlzIHNob3VsZCBoYXZlIHVzZWQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKS4KU2luY2UgdjYuNiB0aGUgY2FsbCBw
YWdlX3Bvb2xfcmVsZWFzZV9wYWdlKCkgd2VyZSByZW1vdmVkIChpbgpjb21t
aXQgNTM1YjljNjFiZGVmICgibmV0OiBwYWdlX3Bvb2w6IGhpZGUgcGFnZV9w
b29sX3JlbGVhc2VfcGFnZSgpIikKYW5kIHJlbWFpbmluZyBjYWxsZXJzIGNv
bnZlcnRlZCAoaW4gY29tbWl0IDZiZmVmMmVjMDE3MiAoIk1lcmdlIGJyYW5j
aAonbmV0LXBhZ2VfcG9vbC1yZW1vdmUtcGFnZV9wb29sX3JlbGVhc2VfcGFn
ZSciKSkuCgpUaGlzIGxlYWsgYmVjYW1lIHZpc2libGUgaW4gdjYuOCB2aWEg
Y29tbWl0IGRiYTFiOGE3YWI2OCAoIm1tL3BhZ2VfcG9vbDogY2F0Y2gKcGFn
ZV9wb29sIG1lbW9yeSBsZWFrcyIpLgoKQ2M6IHN0YWJsZUB2Z2VyLmtlcm5l
bC5vcmcKRml4ZXM6IDZjNWFhNmZjNGRlZiAoInhlbiBuZXR3b3JraW5nOiBh
ZGQgYmFzaWMgWERQIHN1cHBvcnQgZm9yIHhlbi1uZXRmcm9udCIpClJlcG9y
dGVkLWJ5OiBMZW9uaWRhcyBTcHlyb3BvdWxvcyA8YXJ0YWZpbmRlQGFyY2hs
aW51eC5jb20+Ckxpbms6IGh0dHBzOi8vYnVnemlsbGEua2VybmVsLm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9MjE4NjU0ClJlcG9ydGVkLWJ5OiBBcnRodXIgQm9y
c2Jvb20gPGFydGh1cmJvcnNib29tQGdtYWlsLmNvbT4KU2lnbmVkLW9mZi1i
eTogSmVzcGVyIERhbmdhYXJkIEJyb3VlciA8aGF3a0BrZXJuZWwub3JnPgpM
aW5rOiBodHRwczovL2xvcmUua2VybmVsLm9yZy9yLzE3MTE1NDE2NzQ0Ni4y
NjcxMDYyLjkxMjcxMDUzODQ1OTEyMzczNjMuc3RnaXRAZmlyZXNvdWwKU2ln
bmVkLW9mZi1ieTogSmFrdWIgS2ljaW5za2kgPGt1YmFAa2VybmVsLm9yZz4K
LS0tCiBkcml2ZXJzL25ldC94ZW4tbmV0ZnJvbnQuYyB8IDEgKwogMSBmaWxl
IGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCspCgpkaWZmIC0tZ2l0IGEvZHJpdmVy
cy9uZXQveGVuLW5ldGZyb250LmMgYi9kcml2ZXJzL25ldC94ZW4tbmV0ZnJv
bnQuYwppbmRleCBhZDI5ZjM3MDAzNGU0Zi4uOGQyYWVlODg1MjZjNjkgMTAw
NjQ0Ci0tLSBhL2RyaXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCisrKyBiL2Ry
aXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCkBAIC0yODUsNiArMjg1LDcgQEAg
c3RhdGljIHN0cnVjdCBza19idWZmICp4ZW5uZXRfYWxsb2Nfb25lX3J4X2J1
ZmZlcihzdHJ1Y3QgbmV0ZnJvbnRfcXVldWUgKnF1ZXVlKQogCQlyZXR1cm4g
TlVMTDsKIAl9CiAJc2tiX2FkZF9yeF9mcmFnKHNrYiwgMCwgcGFnZSwgMCwg
MCwgUEFHRV9TSVpFKTsKKwlza2JfbWFya19mb3JfcmVjeWNsZShza2IpOwog
CiAJLyogQWxpZ24gaXAgaGVhZGVyIHRvIGEgMTYgYnl0ZXMgYm91bmRhcnkg
Ki8KIAlza2JfcmVzZXJ2ZShza2IsIE5FVF9JUF9BTElHTik7Ci0tIApjZ2l0
IDEuMi4zLWtvcmcKCg==

--=separator--


From xen-devel-bounces@lists.xenproject.org Tue May 07 17:19:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 17:19:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718381.1120997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4OTC-0005s3-B7; Tue, 07 May 2024 17:19:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718381.1120997; Tue, 07 May 2024 17:19:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4OTC-0005rw-8K; Tue, 07 May 2024 17:19:14 +0000
Received: by outflank-mailman (input) for mailman id 718381;
 Tue, 07 May 2024 17:19:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0blL=MK=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s4OTA-0005rU-Tc
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 17:19:13 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ebdfab3c-0c95-11ef-909c-e314d9c70b13;
 Tue, 07 May 2024 19:19:12 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a5200afe39eso900944566b.1
 for <xen-devel@lists.xenproject.org>; Tue, 07 May 2024 10:19:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebdfab3c-0c95-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715102351; x=1715707151; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DJb6Fs5+m3mfHuEewj/2Zpd4X1a6+E0R+LSwyHP9T0Y=;
        b=BVW7+WU1EEwFM4CPtFljaRDY5IkPk2jIyS7BnXRlJewnwZskolvI7H/46Abc0aW+M1
         D4lkRkqIvjUte/bfTnR9Og8rVB0O7ngAAjT1sexBA4Xh14VBHuPW6g6XnhyGJ2MWEXQX
         zLfNo8fSDZI6ZEnTKXOlW/CIskBFaVMZBu0LbaZu8nb+tVHftSc05VNKQxsm1VdJM23Y
         4G3aoEhXjJZiS0Hx/DiiW/oOJuUsct9aeuddwPL/4I3QF5FXchiOv0Rnak2AiirISvVd
         z3+po8lTkxHBQNVrfO89b+36bMJ0Lye4AE64EGuWQi1Yr1rhzkU3KGuXH/r/SJRENWyD
         0Y1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715102351; x=1715707151;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DJb6Fs5+m3mfHuEewj/2Zpd4X1a6+E0R+LSwyHP9T0Y=;
        b=RqDW6VQQXEvtGpuYpfCbga16NLgUVv9DpwjcXBgJyyx7NDbAFRM7j7Y59bifGUsiIz
         ++XdYFyfpb7BfFpzM+WMw55FqxbD9b/3QJxn0ez9z7HrOH+Y3HMRExRBByQA+XoBUbqs
         ukdcQ2uJ8hKX3wG0hL4k5LUGJlUdM1aepsFAXcVp5jU9Tl1iXP30civQ58hIikTnX+kR
         SqChnrX4rLiecgyKvsKZpDGyAuWR6QM8mxDuaXjXzZRjKirfaenEG399mehhvJZgP/hE
         DL7s+HqXXIjZOZla6vUEe99Oki6d++P/4/h2CK8iUFSoei0hy7PimMNAgcAZr9XVrZ8A
         QDZg==
X-Forwarded-Encrypted: i=1; AJvYcCV7X1ruiupBa+GVcgPx7d0OkRImiiUDBTP6y7bDW9GsF6ktiaOqZU2hZw1TlHa+60bhiGg8CYuAUQTMEu4iazjedHqoctETDwYa+QX/Sbk=
X-Gm-Message-State: AOJu0YwHdI/lOgodyt6YDgYFF75av5mWlmt0QV3Dkyrjt8M1JwR5icHV
	2KgK1XzvcDQOxlf7b933uENjw8spZCLcVJnghSU7uChl6yhnLN1Ks83Cbii0K5IuIAhhZKr3cam
	f4b+47OkZIFWkECQ7yM6bz9Hz28U=
X-Google-Smtp-Source: AGHT+IGl4aWcvM+580d59kYVJt6J/TIw0/S9A88EwJy2nsKC4dankoJEo04a6Ct4IEezazd8WxDbcCDSrDREQxhRmgc=
X-Received: by 2002:a50:d78b:0:b0:568:cdd8:cf60 with SMTP id
 4fb4d7f45d1cf-5731d9b5e37mr305529a12.8.1715102351091; Tue, 07 May 2024
 10:19:11 -0700 (PDT)
MIME-Version: 1.0
References: <20240430164939.925307-1-edgar.iglesias@gmail.com>
 <20240430164939.925307-16-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405011414320.497719@ubuntu-linux-20-04-desktop>
 <CAJy5ezoQ4Q6kc9T8_P3nEU6N20e6Pvr3K_zuM0j95RcG-hDgfw@mail.gmail.com>
 <alpine.DEB.2.22.394.2405021144270.624854@ubuntu-linux-20-04-desktop>
 <CAJy5ezq+wd4T91YPEarea4eUXp_CJftvPYMJT2AzA66DWjy34w@mail.gmail.com> <alpine.DEB.2.22.394.2405021301400.624854@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405021301400.624854@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Tue, 7 May 2024 19:18:57 +0200
Message-ID: <CAJy5ezoTVyP5Rs=zM3jdY2NKhwQqDuCWD9oG72m4Xsdc-kmc+A@mail.gmail.com>
Subject: Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock
 with 0 offset
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
	David Hildenbrand <david@redhat.com>, =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>, 
	xen-devel@lists.xenproject.org, Xenia.Ragiadakou@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 2, 2024 at 10:02=E2=80=AFPM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > On Thu, May 2, 2024 at 8:53=E2=80=AFPM Stefano Stabellini
> > <sstabellini@kernel.org> wrote:
> > >
> > > +Xenia
> > >
> > > On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > > > On Wed, May 1, 2024 at 11:24=E2=80=AFPM Stefano Stabellini
> > > > <sstabellini@kernel.org> wrote:
> > > > >
> > > > > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > > > > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> > > > > >
> > > > > > The current mapcache assumes that all memory is mapped
> > > > > > in a single RAM MR (the first one with offset 0). Remove
> > > > > > this assumption and propagate the offset to the mapcache
> > > > > > so it can do reverse mappings (from hostptr -> ram_addr).
> > > > > >
> > > > > > This is in preparation for adding grant mappings.
> > > > > >
> > > > > > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > > > >
> > > > >
> > > > > Looking at xen_remap_bucket, it is only using address_index (with=
out
> > > > > adding ram_offset) to map foreign memory. From xen_remap_bucket, =
I would
> > > > > understand that address_index already includes the ram_offset.
> > > > >
> > > > > Meaning that if we want to map foreign mapping at address 0x5000,=
 then
> > > > > address_index would be 0x5000, even if ram_offset is 0x1000.
> > > > >
> > > > > But then looking xen_ram_addr_from_mapcache_single ram_offset is =
added
> > > > > to paddr_index to calculate the physical address. So in that case=
 we
> > > > > would want address_index to be 0x4000 and ram_offset to be 0x1000=
. But
> > > > > xen_remap_bucket would have to sum address_index and ram_offset t=
o map
> > > > > foreign memory.
> > > > >
> > > > > So I am a bit confused, did I get it wrong? One more comment belo=
w.
> > > > >
> > > >
> > > > Thanks Stefano,
> > > >
> > > > I think the confusion is that this ram_addr_offset is not related t=
o
> > > > guest address-space.
> > > > It's a QEMU internal thing and it shouldn't be included in the addr=
ess
> > > > used to map foreign memory.
> > > > The mapcache can treat this ram_addr offset like a cookie that we k=
eep
> > > > around to be able to do
> > > > reverse mappings from host pointers into ram_addr space
> > > > (xen_ram_addr_from_mapcache).
> > > >
> > > > The current mapcache implementation works because we've really only
> > > > been using foreign mappings
> > > > on RAMBlocks with offset 0. We're also creating RAM's such that the
> > > > offset into the RAM is also
> > > > the guest physical address, for x86 this is natural since RAM start=
s
> > > > at zero (for lowmem) but for
> > > > ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-=
size) to
> > > > make this assumption true. Anyway, In this series I'm not addressin=
g
> > > > this second assumption.
> > >
> > > Let's see if I understand correctly.
> > >
> > > The ram_addr space is an internal QEMU address space which is differe=
nt
> > > from the guest physical address space and thus cannot and should not =
be
> > > used to do foreign mappings (foreign mapping hypercalls take a guest
> > > physical or a real physical address to map). Is that correct?
> > >
> > > If so, then I understand.
> > >
> >
> > Yes, that matches my understanding.
> >
> > >
> > >
> > > > There's a second call in physmem.c to xen_map_cache using the
> > > > block->offset as an address.
> > > > I was considering removing that second call since I can't see how i=
t can work
> > > > (except perhaps in some specific use-case by luck?). Anyway, for no=
w
> > > > I've left it unmodified.
> > >
> > > Yes, that code was written with the assumption that block->offset is =
an
> > > offset in the guest physical address space and could be used as a gue=
st
> > > physical address. Actually, you might have spotted a real bug.
> > >
> > > The intent was for smaller regions (not the bit RAM region, things li=
ke
> > > a ROM region for instance) we could map them in full. So here we were
> > > trying to map the whole thing from start to finish using block->offse=
t
> > > as start.
> > >
> > >
> > > > > > ---
> > > > > >  hw/xen/xen-mapcache.c         | 25 ++++++++++++++++++-------
> > > > > >  include/sysemu/xen-mapcache.h |  2 ++
> > > > > >  system/physmem.c              |  8 ++++----
> > > > > >  3 files changed, 24 insertions(+), 11 deletions(-)
> > > > > >
> > > > > > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > > > > > index 09b5f36d9c..1b32d0c003 100644
> > > > > > --- a/hw/xen/xen-mapcache.c
> > > > > > +++ b/hw/xen/xen-mapcache.c
> > > > > > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> > > > > >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > > > > >      uint8_t flags;
> > > > > >      hwaddr size;
> > > > > > +
> > > > > > +    /* Keep ram_addr offset for reverse mappings (hostptr -> r=
am_addr).  */
> > > > > > +    ram_addr_t ram_offset;
> > > > > >      struct MapCacheEntry *next;
> > > > > >  } MapCacheEntry;
> > > > > >
> > > > > > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> > > > > >                               void *vaddr,
> > > > > >                               hwaddr size,
> > > > > >                               hwaddr address_index,
> > > > > > -                             bool dummy)
> > > > > > +                             bool dummy,
> > > > > > +                             ram_addr_t ram_offset)
> > > > > >  {
> > > > > >      uint8_t *vaddr_base;
> > > > > >      xen_pfn_t *pfns;
> > > > > > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > > > >      entry->size =3D size;
> > > > > >      entry->valid_mapping =3D g_new0(unsigned long,
> > > > > >                                    BITS_TO_LONGS(size >> XC_PAG=
E_SHIFT));
> > > > > > +    entry->ram_offset =3D ram_offset;
> > > > > >
> > > > > >      if (dummy) {
> > > > > >          entry->flags |=3D XEN_MAPCACHE_ENTRY_DUMMY;
> > > > > > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> > > > > >
> > > > > >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > > > > >                                         hwaddr phys_addr, hwadd=
r size,
> > > > > > +                                       ram_addr_t ram_offset,
> > > > > >                                         uint8_t lock, bool dma,=
 bool is_write)
> > > > > >  {
> > > > > >      MapCacheEntry *entry, *pentry =3D NULL,
> > > > > > @@ -335,14 +341,16 @@ tryagain:
> > > > > >      if (!entry) {
> > > > > >          entry =3D g_new0(MapCacheEntry, 1);
> > > > > >          pentry->next =3D entry;
> > > > > > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_=
index, dummy);
> > > > > > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_=
index, dummy,
> > > > > > +                         ram_offset);
> > > > > >      } else if (!entry->lock) {
> > > > > >          if (!entry->vaddr_base || entry->paddr_index !=3D addr=
ess_index ||
> > > > > >                  entry->size !=3D cache_size ||
> > > > > >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> > > > > >                      test_bit_size >> XC_PAGE_SHIFT,
> > > > > >                      entry->valid_mapping)) {
> > > > > > -            xen_remap_bucket(mc, entry, NULL, cache_size, addr=
ess_index, dummy);
> > > > > > +            xen_remap_bucket(mc, entry, NULL, cache_size, addr=
ess_index, dummy,
> > > > > > +                             ram_offset);
> > > > > >          }
> > > > > >      }
> > > > > >
> > > > > > @@ -389,13 +397,15 @@ tryagain:
> > > > > >
> > > > > >  uint8_t *xen_map_cache(MemoryRegion *mr,
> > > > > >                         hwaddr phys_addr, hwaddr size,
> > > > > > +                       ram_addr_t ram_addr_offset,
> > > > > >                         uint8_t lock, bool dma,
> > > > > >                         bool is_write)
> > > > > >  {
> > > > > >      uint8_t *p;
> > > > > >
> > > > > >      mapcache_lock(mapcache);
> > > > > > -    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, lo=
ck, dma, is_write);
> > > > > > +    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, ra=
m_addr_offset,
> > > > > > +                               lock, dma, is_write);
> > > > > >      mapcache_unlock(mapcache);
> > > > > >      return p;
> > > > > >  }
> > > > > > @@ -432,7 +442,8 @@ static ram_addr_t xen_ram_addr_from_mapcach=
e_single(MapCache *mc, void *ptr)
> > > > > >          raddr =3D RAM_ADDR_INVALID;
> > > > > >      } else {
> > > > > >          raddr =3D (reventry->paddr_index << mc->bucket_shift) =
+
> > > > > > -             ((unsigned long) ptr - (unsigned long) entry->vad=
dr_base);
> > > > > > +             ((unsigned long) ptr - (unsigned long) entry->vad=
dr_base) +
> > > > > > +             entry->ram_offset;
> > > > > >      }
> > > > > >      mapcache_unlock(mc);
> > > > > >      return raddr;
> > > > > > @@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unl=
ocked(MapCache *mc,
> > > > > >
> > > > > >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phy=
s_addr);
> > > > > >
> > > > > > -    xen_remap_bucket(mapcache, entry, entry->vaddr_base,
> > > > > > -                     cache_size, address_index, false);
> > > > > > +    xen_remap_bucket(mc, entry, entry->vaddr_base,
> > > > > > +                     cache_size, address_index, false, entry->=
ram_offset);
> > > > > >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> > > > > >                  test_bit_size >> XC_PAGE_SHIFT,
> > > > > >                  entry->valid_mapping)) {
> > > > > > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen=
-mapcache.h
> > > > > > index 1ec9e66752..b5e3ea1bc0 100644
> > > > > > --- a/include/sysemu/xen-mapcache.h
> > > > > > +++ b/include/sysemu/xen-mapcache.h
> > > > > > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwad=
dr phys_offset,
> > > > > >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> > > > > >                          void *opaque);
> > > > > >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwa=
ddr size,
> > > > > > +                       ram_addr_t ram_addr_offset,
> > > > > >                         uint8_t lock, bool dma,
> > > > > >                         bool is_write);
> > > > > >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > > > > > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_of=
fset_to_gaddr_t f,
> > > > > >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> > > > > >                                       hwaddr phys_addr,
> > > > > >                                       hwaddr size,
> > > > > > +                                     ram_addr_t ram_addr_offse=
t,
> > > > > >                                       uint8_t lock,
> > > > > >                                       bool dma,
> > > > > >                                       bool is_write)
> > > > > > diff --git a/system/physmem.c b/system/physmem.c
> > > > > > index 1a5ffcba2a..5b16eeccca 100644
> > > > > > --- a/system/physmem.c
> > > > > > +++ b/system/physmem.c
> > > > > > @@ -2228,13 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlo=
ck *block, ram_addr_t addr,
> > > > > >           * In that case just map the requested area.
> > > > > >           */
> > > > > >          if (xen_mr_is_memory(block->mr)) {
> > > > > > -            return xen_map_cache(block->mr, addr, len, lock, l=
ock,
> > > > > > -                                 is_write);
> > > > > > +            return xen_map_cache(block->mr, addr, len, block->=
offset,
> > > > > > +                                 lock, lock, is_write);
> > > > >
> > > > > Have you considered not tracking offset and address separately an=
d
> > > > > simply do this?
> > > > >
> > > > >             return xen_map_cache(block->mr, addr + block->offset,=
 len,
> > > > >                                  lock, lock, is_write);
> > > > >
> > > >
> > > > Unfortunately this won't work since block->offset is not related to=
 where this
> > > > ram is mapped in guest address-space. In the case of grant's, we'd =
get the
> > > > wrong grant ref. See my previous comment.
> > >
> > > OK, this code below (the second xen_map_cache call passing block->off=
set
> > > as start address) was wrong before this patch. Can we fix it before
> > > changing it further with this patch? I worry about making things even
> > > worse.
> > >
> >
> > I'll dig around and see if we can find something that explains more.
> > There's some older code that implements some sort of address-translatio=
n
> > for x86 between ram_addr space and guest physical addresses but
> > that code is turned off with newer Xen versions (disabled in my build).
> >
> > https://github.com/qemu/qemu/blob/master/hw/xen/xen-mapcache.c#L330
> > https://github.com/qemu/qemu/blob/master/hw/i386/xen/xen-hvm.c#L193
>
> I don't have any more insights but I think Xenia might have a better
> idea as she has fixed bugs related to this to get virtio-gpu to work.

Hi again,

I found the reason this works today is because for RAM's created after
xen_memory,
QEMU hypercalls into Xen to populate the guest memory map with RAM at the g=
iven
ram_offset.
https://github.com/qemu/qemu/blob/master/hw/xen/xen-hvm-common.c#L44

The current grant series does not affect this current behaviour but I
think it would be
good to make the offset handling a little more explicit by adding the
addr+ram_offset
in the map-cache (except for the special grant region) and always
passing them in
the same argument slots. I'll try to improve this in the next version
and add some
comments and we can discuss from there.

Cheers,
Edgar


From xen-devel-bounces@lists.xenproject.org Tue May 07 17:37:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 17:37:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718430.1121008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Okf-000165-U7; Tue, 07 May 2024 17:37:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718430.1121008; Tue, 07 May 2024 17:37:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Okf-00015y-R8; Tue, 07 May 2024 17:37:17 +0000
Received: by outflank-mailman (input) for mailman id 718430;
 Tue, 07 May 2024 17:37:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4Oke-00015s-87
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 17:37:16 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4Okc-0003pi-Vk; Tue, 07 May 2024 17:37:14 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4Okc-0003VO-Mh; Tue, 07 May 2024 17:37:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=wblJRTqKKZsdNoe9ayI7MRwbMHoVJ4a8NXxsfgPQbSM=; b=jqpwiMGzM6jdOpssEvE+cC92sF
	P6KE8iDG+GVxlaZ+S9zNjnPBCXsbANjwcHO27k5unUCpqbvmaa9TtUYnWjacxny2Xxvq5+0t0/Fcc
	7Lw6Pq3d/9R0Xz1/P0pKHhVttj2yOaeHccIczau9h5zLLVQJP790euiq3W2PT2+zOX/M=;
Message-ID: <35c8fbfb-8b09-4aae-8c98-402dcd65ca2e@xen.org>
Date: Tue, 7 May 2024 18:37:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol
 annotations
Content-Language: en-GB
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, jbeulich@suse.com,
 andrew.cooper3@citrix.com, edgar.iglesias@amd.com
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com>
 <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
 <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
 <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org>
 <CAJy5ezpNQP5q0NVaGdXtt5BbJSzCMfTfFt-0RRgVq3+DqMbKgA@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAJy5ezpNQP5q0NVaGdXtt5BbJSzCMfTfFt-0RRgVq3+DqMbKgA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 07/05/2024 17:55, Edgar E. Iglesias wrote:
> On Tue, May 7, 2024 at 11:57 AM Julien Grall <julien@xen.org> wrote:
> Hi Julien,

Hi Edgar,

> 
> The reason I choose FUNC for the start of the symbol is because these
> symbols contain
> executable code (not only a table of pointers to code somewhere else)
> and the ELF spec
> says that STT_FUNC means the symbol contains functions or other executable
> code (not only callable functions IIUC):
> 
> "STT_FUNC The symbol is associated with a function or other executable code."
> https://refspecs.linuxbase.org/elf/elf.pdf
> (Symbol Table 1-20).

Thanks for the pointer. I originally did intend to suggest the change, 
but then I saw the use of LABEL in x86 (such as svm_stgi_label). There 
are a few others example with LABEL_LOCAL.

AFAICT, this is also executable code which the only difference that it 
is not meant to be called by someone else. Furthermore, LABEL is using 
DO_CODE_ALIGN(...) for the alignment which imply that it is intended to 
be used by executable code. So I thought the only difference was whether 
the label was intended to be used as a function.

> 
> I think using LABEL instead of GLOBAL for the _end labels of these
> code sequences makes sense.
> I'm happy to change the _start labels to LABEL too if you guys feel
> that's better.

I have to admit I am little confused with the difference between LABEL 
vs FUNC. I think I will need some guidance from Jan (he introduced 
linkage.h).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 07 19:55:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 19:55:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718467.1121018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4QuL-0000cw-Du; Tue, 07 May 2024 19:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718467.1121018; Tue, 07 May 2024 19:55:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4QuL-0000cp-A9; Tue, 07 May 2024 19:55:25 +0000
Received: by outflank-mailman (input) for mailman id 718467;
 Tue, 07 May 2024 19:55:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4QuJ-0000cJ-Le; Tue, 07 May 2024 19:55:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4QuJ-00069G-IX; Tue, 07 May 2024 19:55:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4QuJ-0003oE-69; Tue, 07 May 2024 19:55:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4QuJ-0001Sv-5g; Tue, 07 May 2024 19:55:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eXXPtVhBy72P0ET3L6GIgiZjgaV3xAGKeAYYSvJFgEg=; b=pi3LAqaqqzZ616gdz5OPFcSJQ6
	slkw0lsMBaWJGmAOCFPhankrNLIWAugg6e82/UqFjk73aNPy0TtEYVp0yNgsCjHJ2e0gz3M0aD6l4
	xLmUlDLp5lNoV9p6w96OkNMmS9hGggeaoW0CRqKeoqJqC/SAxHgYOJZ6248BQI49Kyyo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185939-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185939: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
X-Osstest-Versions-That:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 May 2024 19:55:23 +0000

flight 185939 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185939/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b
baseline version:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81

Last test of basis   185926  2024-05-06 13:02:07 Z    1 days
Testing same since   185939  2024-05-07 17:03:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ebab808eb1..a2330b51df  a2330b51df267e20e66bbba6c5bf08f0570ed58b -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 07 21:54:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 May 2024 21:54:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718498.1121027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4SlI-0002ct-E8; Tue, 07 May 2024 21:54:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718498.1121027; Tue, 07 May 2024 21:54:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4SlI-0002cm-BI; Tue, 07 May 2024 21:54:12 +0000
Received: by outflank-mailman (input) for mailman id 718498;
 Tue, 07 May 2024 21:54:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4SlH-0002cb-Ey
 for xen-devel@lists.xenproject.org; Tue, 07 May 2024 21:54:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4SlG-00089O-8X; Tue, 07 May 2024 21:54:10 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4SlG-0000U4-1O; Tue, 07 May 2024 21:54:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=P9VUyF33uGGarJ9y2y7l+KKZkEIaubY68qvQcvVYqlM=; b=u7UCKS0jdtJ4beIoEjnhytbXOE
	d9o2Up4MX9Z2i5ZvwKrkQYNbLqFn4bP1L8/L93yCHdbFjE8wBPZiNrpyvRWYpKYAK4dzOJ9k4tVQV
	g21mKay9Z9EK5VA6Pxf/eytPCbpYeinEnbx5ft9eDzAt4d43JRXvjDkMNtxJkYc1Ok6Y=;
Message-ID: <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
Date: Tue, 7 May 2024 22:54:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 06/05/2024 09:32, Henry Wang wrote:
> On 5/1/2024 4:13 AM, Julien Grall wrote:
>> Hi Henry,
>>
>> On 30/04/2024 04:50, Henry Wang wrote:
>>> On 4/25/2024 10:28 PM, Julien Grall wrote:
>>>>> Thanks for your feeedback. After checking the b8577547236f commit 
>>>>> message I think I now understand your point. Do you have any 
>>>>> suggestion about how can I properly add the support to route/remove 
>>>>> the IRQ to running domains? Thanks.
>>>
>>> I spent some time going through the GIC/vGIC code and had some 
>>> discussions with Stefano and Stewart during the last couple of days, 
>>> let me see if I can describe the use case properly now to continue 
>>> the discussion:
>>>
>>> We have some use cases that requires assigning devices to domains 
>>> after domain boot time. For example, suppose there is an FPGA on the 
>>> board which can simulate a device, and the bitstream for the FPGA is 
>>> provided and programmed after domain boot. So we need a way to assign 
>>> the device to the running domain. This series tries to implement this 
>>> use case by using device tree overlay - users can firstly add the 
>>> overlay to Xen dtb, assign the device in the overlay to a domain by 
>>> the xl command, then apply the overlay to Linux.
>>
>> Thanks for the description! This helps to understand your goal :).
> 
> Thank you very much for spending your time on discussing this and 
> provide these valuable comments!
> 
>>>
>>>> I haven't really look at that code in quite a while. I think we need 
>>>> to make sure that the virtual and physical IRQ state matches at the 
>>>> time we do the routing.
>>>>
>>>> I am undecided on whether we want to simply prevent the action to 
>>>> happen or try to reset the state.
>>>>
>>>> There is also the question of what to do if the guest is enabling 
>>>> the vIRQ before it is routed.
>>>
>>> Sorry for bothering, would you mind elaborating a bit more about the 
>>> two cases that you mentioned above? Commit b8577547236f ("xen/arm: 
>>> Restrict when a physical IRQ can be routed/removed from/to a domain") 
>>> only said there will be undesirable effects, so I am not sure if I 
>>> understand the concerns raised above and the consequences of these 
>>> two use cases.
>>
>> I will try to explain them below after I answer the rest.
>>
>>> I am probably wrong, I think when we add the overlay, we are probably 
>>> fine as the interrupt is not being used before. 
>>
>> What if the DT overlay is unloaded and then reloaded? Wouldn't the 
>> same interrupt be re-used? As a more generic case, this could also be 
>> a new bitstream for the FPGA.
>>
>> But even if the interrupt is brand new every time for the DT overlay, 
>> you are effectively relaxing the check for every user (such as 
>> XEN_DOMCTL_bind_pt_irq). So the interrupt re-use case needs to be 
>> taken into account.
> 
> I agree. I think IIUC, with your explanation here and below, could we 
> simplify the problem to how to properly handle the removal of the IRQ 
> from a running guest, if we always properly remove and clean up the 
> information when remove the IRQ from the guest? In this way, the IRQ can 
> always be viewed as a brand new one when we add it back.

If we can make sure the virtual IRQ and physical IRQ is cleaned then yes.

> Then the only 
> corner case that we need to take care of would be...

Can you clarify whether you say the "only corner case" because you 
looked at the code? Or is it just because I mentioned only one?

> 
>>> Also since we only load the device driver after the IRQ is routed to 
>>> the guest, 
>>
>> This is what a well-behave guest will do. However, we need to think 
>> what will happen if a guest misbehaves. I am not concerned about a 
>> guest only impacting itself, I am more concerned about the case where 
>> the rest of the system is impacted.
>>
>>> I am not sure the guest can enable the vIRQ before it is routed.
>>
>> Xen allows the guest to enable a vIRQ even if there is no pIRQ 
>> assigned. Thanksfully, it looks like the vgic_connect_hw_irq(), in 
>> both the current and new vGIC, will return an error if we are trying 
>> to route a pIRQ to an already enabled vIRQ.
>>
>> But we need to investigate all the possible scenarios to make sure 
>> that any inconsistencies between the physical state and virtual state 
>> (including the LRs) will not result to bigger problem.
>>
>> The one that comes to my mind is: The physical interrupt is 
>> de-assigned from the guest before it was EOIed. In this case, the 
>> interrupt will still be in the LR with the HW bit set. This would 
>> allow the guest to EOI the interrupt even if it is routed to someone 
>> else. It is unclear what would be the impact on the other guest.
> 
> ...same as this case, i.e.
> test_bit(_IRQ_INPROGRESS, &desc->status) || !test_bit(_IRQ_DISABLED, 
> &desc->status)) when we try to remove the IRQ from a running domain.

We already call ->shutdown() which will disable the IRQ. So don't we 
only need to take care of _IRQ_INPROGRESS?

[...]

> we have 3 possible states which can be read from LR for this case : 
> active, pending, pending and active.
> - I don't think we can do anything about the active state, so we should 
> return -EBUSY and reject the whole operation of removing the IRQ from 
> running guest, and user can always retry this operation.

This would mean a malicious/buggy guest would be able to prevent a 
device to be de-assigned. This is not a good idea in particular when the 
domain is dying.

That said, I think you can handle this case. The LR has a bit to 
indicate whether the pIRQ needs to be EOIed. You can clear it and this 
would prevent the guest to touch the pIRQ. There might be other clean-up 
to do in the vGIC datastructure.

Anyway, we don't have to handle removing an active IRQ when the domain 
is still running (although we do when the domain is destroying). But I 
think this would need to be solved before the feature is (security) 
supported.

> - For the pending (and active) case,

Shouldn't the pending and active case handled the same way as the active 
case?

> can we clear the LR and point the 
> LR for the pending_irq to invalid?

LRs can be cleared. You will need to find which vCPU was used for the 
injection and then pause it so the LR can be safely updated.

There will also be some private state to clear. I don't know how easy it 
will be. However, we decided to not do anything for ICPENDR (which 
requires a similar behavior) as this was complex (?) to do with the 
existing vGIC.

I vaguely remember we had some discussions on the ML. I didn't look for 
them though.

Anyway, same as above, this could possibly handled later on. But this 
would probably need to be solved before the feature is (security supported).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 08 00:31:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 00:31:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718535.1121038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4VDE-0002Fp-SF; Wed, 08 May 2024 00:31:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718535.1121038; Wed, 08 May 2024 00:31:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4VDE-0002Fi-OK; Wed, 08 May 2024 00:31:12 +0000
Received: by outflank-mailman (input) for mailman id 718535;
 Wed, 08 May 2024 00:31:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RXYr=ML=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s4VDC-0002FX-Tv
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 00:31:10 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 407cf4c9-0cd2-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 02:31:06 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 08B3C619EF;
 Wed,  8 May 2024 00:31:03 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B59BC2BBFC;
 Wed,  8 May 2024 00:31:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 407cf4c9-0cd2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715128262;
	bh=Xzj0m2AH5anAzGDQNaSSzA3eqlk3g3E+XmrdfVrcHTw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Up/LuWU7ZTUooAzRuiX6Rhtb4XjIw8nRcaW5jxdYtOUx8iWLj/R+9PzrJ1NuVZ4Of
	 N9TjFE5cPInr8eZy8d/ARcm6u8tIEeuU/B1Axp03VWYfyFyvv8p4TwVjYkjYeUkWAV
	 RPqJxXw5r2eBgCXzgdDLMx5YLjpR+LzmOA1/6rRc58leaYqOMm/ZG5qjNd7bwJJnw9
	 IEh3eKbIs0NzSVVx+Ms3WZQ2rq0+XWiAe68JCE7ZCzIz/hmRjiqQ29iBAr+APhvRUD
	 eC7k4hESlGOR8gOJdRqpzjOd1V60EZ2ndxViNw6eYbRL324ogP+LOHzEq6CAF+gXlQ
	 wRY7Fo7yoF9ZQ==
Date: Tue, 7 May 2024 17:31:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?J=C3=BCrgen_Gro=C3=9F?= <jgross@suse.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 8/8] xen: allow up to 16383 cpus
In-Reply-To: <0eff541b-51c9-4764-b8c2-b686d5e602ce@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405071726000.1151289@ubuntu-linux-20-04-desktop>
References: <20240327152229.25847-1-jgross@suse.com> <20240327152229.25847-9-jgross@suse.com> <da1662ac-70f1-4ae4-9737-e10e617c8036@suse.com> <637c755a-9f24-4b86-af12-7c7d50bf40c8@suse.com> <d0714064-c544-47d3-84c0-a19391ccf496@xen.org>
 <29296e11-8d21-4867-9d31-fc94af828214@suse.com> <146fbd43-d8b8-4d73-8650-c60024498324@xen.org> <alpine.DEB.2.22.394.2405021111540.624854@ubuntu-linux-20-04-desktop> <b3299df2-e726-49eb-a893-cbc547b2419c@xen.org> <alpine.DEB.2.22.394.2405031159240.1151289@ubuntu-linux-20-04-desktop>
 <0eff541b-51c9-4764-b8c2-b686d5e602ce@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 03/05/2024 20:07, Stefano Stabellini wrote:
> > On Fri, 3 May 2024, Julien Grall wrote:
> 
> [...]
> 
> > > So are you saying that from Xen point of view, you are expecting no
> > > difference
> > > between 256 and 512. And therefore you would be happy if to backport
> > > patches
> > > if someone find differences (or even security issues) when using > 256
> > > pCPUs?
> > 
> > It is difficult to be sure about anything that it is not regularly
> > tested. I am pretty sure someone in the community got Xen running on an
> > Ampere, so like you said 192 is a good number. However, that is not
> > regularly tested, so we don't have any regression checks in gitlab-ci or
> > OSSTest for it.
> > 
> > One approach would be to only support things regularly tested either by
> > OSSTest, Gitlab-ci, or also Xen community members. I am not sure what
> > would be the highest number with this way of thinking but likely no
> > more than 192, probably less. I don't know the CPU core count of the
> > biggest ARM machine in OSSTest.
> 
> This would be rochester* (Cavium Thunder-X). They have 96 pCPUs which, IIRC,
> are split across two numa nodes.
> 
> > 
> > Another approach is to support a "sensible" number: not something tested
> > but something we believe it should work. No regular testing. (In safety,
> > they only believe in things that are actually tested, so this would not
> > be OK. But this is security, not safety, just FYI.) With this approach,
> > we could round up the number to a limit we think it won't break. If 192
> > works, 256/512 should work? I don't know but couldn't think of something
> > that would break going from 192 to 256.
> 
> It depends what you mean by work/break. Strictly speaking, Xen should run
> (i.e. not crash). However, it is unclear how well as if you increase the
> number of physical CPUs, you will increase contention and may find some
> bottleneck.
> 
> I haven't done any performance testing with that many CPUs and I haven't seen
> any so far with Xen. But I have some areas of concerns.
> 
> * Xenstored: At least the C version is single-threaded. Technically the limit
> here is not based on the number of pCPUs, but as you increase it, you
> indirectly increase the number of domains that can run. I doubt it will behave
> well if you have 4096 domains running (I am thinking about the x86 limit...).
> 
> * Locking
>   * How Xen use the locks: I don't think we have many places where we have
> global locks (one is the memory subsystem). If a lock is already taken, the
> others will spin. It is unclear if we could high contending.
>   * How Xen implements the locks: At the moment, we are using LL/SC. My take
> of XSA-295 is there is a lack of fairness with them. I am not sure what would
> happen if they get contented (as we support more pCPUs). It is also probably
> time to finally implement LSE atomics.
> 
> * TLB flush: The TLB flush are broadcasted. There are some suggestions on the
> Linux ML [1] that they don't perform well on some processors. The discussion
> seems to have gone nowhere in Linux. But I think it is propably worth to take
> into account when we decide to update the limit we (security) support.
> 
> > 
> > It depends on how strict we want to be on testing requirements.
> From above, I am rather worry about claiming that Xen can supports up to 256
> (and TBH even 192) without any proper testing. This could end up to backfire
> as we may need to do (in a rush) and backport some rather large work (unless
> we decide to remove support after the fact).

I agree with everything you said and I would also add that is not just
about backports: if we "support" something it is supposed to mean that
we strongly believe it is working. I think we should only make that
claim if we test regularly that configuration/feature.


> I think I would prefer if we have a low number until someone can do some
> testing (including potentially malicious guest). If we want for a
> power-of-two, I would go with 128 because this is closer to the HW we have in
> testing. If in the future someone can show some data on other platforms (e.g.
> Ampere), then we can up the limit.

I am OK with that. I wonder if we could use QEMU to add a test for this.


> > I am not
> > sure what approach was taken by x86 so far.
> 
> It is unclear to me. I don't see how we can claim to support up to 4096 CPUs.
> But that's for the x86 folks to decide.

Until not long ago, many things were "supported" in many Open Source
projects (including Linux, QEMU, etc.) without any automated tests at
all. Maybe it is time to revisit this practice.


From xen-devel-bounces@lists.xenproject.org Wed May 08 00:47:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 00:47:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718543.1121048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4VSX-0004HF-3H; Wed, 08 May 2024 00:47:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718543.1121048; Wed, 08 May 2024 00:47:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4VSW-0004H8-W6; Wed, 08 May 2024 00:47:00 +0000
Received: by outflank-mailman (input) for mailman id 718543;
 Wed, 08 May 2024 00:47:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/w98=ML=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s4VSW-0004H2-5x
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 00:47:00 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 78d7785f-0cd4-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 02:46:58 +0200 (CEST)
Received: from MW4PR03CA0094.namprd03.prod.outlook.com (2603:10b6:303:b7::9)
 by IA0PR12MB8088.namprd12.prod.outlook.com (2603:10b6:208:409::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Wed, 8 May
 2024 00:46:53 +0000
Received: from CO1PEPF000044FC.namprd21.prod.outlook.com
 (2603:10b6:303:b7:cafe::76) by MW4PR03CA0094.outlook.office365.com
 (2603:10b6:303:b7::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.40 via Frontend
 Transport; Wed, 8 May 2024 00:46:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Wed, 8 May 2024 00:46:50 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 19:46:47 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 7 May
 2024 19:46:47 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 7 May 2024 19:46:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78d7785f-0cd4-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gjkNqSJJI481/Vok0Ma6867BI7KDpI5RdwXpY5No07C98ZKE0DHjVgWFewDY5xt1IitNBhNWLdBMmZ6TokZz2YEGYERLdCmzl5coW9ElctjwFty7f9jhHu0RQEgvRCUeOBEdugg2kUfSUTiTyhyRblI9VchQmtopf5tnb48x8brGrK6x9Yl/LOnYkmbgAZaStA8+2d2S32a4GR9hyZjXhrXarRsUiAQICmOiQcjkfVdgc3ZyZz+JTSZnXVr3sIau7Un583kMWidHTWtSg77spu5qVnWNjBVJ9+SXvi35wRgSNKyZDU076WjHW0bmRkNP7ysuhaco2fKrFejDd5B1og==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=QPxKK61GnhaYA1csjyQ7sWqoh+eoqf9tfHNpwuWk7jE=;
 b=Ze3YYwm3ruLNSDCTnryPlTxjCKIRFTpTuuQmoSqoK/OYKeAfi2OzkBNbsxxIXJ1BddOnHPHNgRstK79T/BswoSzl3Epx/tXJmixYeJlGzQjVB5Sgk7SjRAVvfzEjwsm75t8qyl6vofirKvfP8pfwmtAle5ta7HWWOwzdEjooEqfE9F8qGHFVQdcwr7bNI+y0spfbT2lP4bpiaR8l7jHelBwIO+sKXdSTSVJglBmapN+U9pGbo/M867qb654KJK5Le0KAu1/eaKFfXyUvgJtqzNtntZ2zX3G+YJhK0QFuh6U0fO0ex4JQdBFJmfYnoY/M3d7mVo0RpGHUO/yITyacSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QPxKK61GnhaYA1csjyQ7sWqoh+eoqf9tfHNpwuWk7jE=;
 b=YTrtFdexO53RhjwGXjbGfm7jWvRbUCgSS1Icfjfhr+CCTa6trseq5BrkZ8bR6AYG1Xhc2mWgOfw22b09GUFAsETxjhDnrrs1X+VNsjL89wYDZXxLIHYC1mxkhNHUSzDBxNARKCos9Ed+uoML7dpntk5Ypktof1YHwQknvf9ae4A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <ad3af234-3b4b-48fe-9628-f205d8c07eb5@amd.com>
Date: Wed, 8 May 2024 08:46:45 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 05/15] tools/libs/light: Increase nr_spi to 160
To: Julien Grall <julien@xen.org>, Anthony PERARD <anthony.perard@cloud.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Michal Orzel
	<michal.orzel@amd.com>, Bertrand Marquis <bertrand.marquis@arm.com>
CC: <xen-devel@lists.xenproject.org>, Vikram Garhwal <fnu.vikram@xilinx.com>,
	Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-6-xin.wang2@amd.com>
 <334eb040-234f-4492-9006-9faadfe28c59@perard>
 <71fc7673-e968-487d-8478-1c7aabef708b@amd.com>
 <bca3098b-4f0b-48e1-8e21-51c66c63e4f3@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <bca3098b-4f0b-48e1-8e21-51c66c63e4f3@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|IA0PR12MB8088:EE_
X-MS-Office365-Filtering-Correlation-Id: 4bee10df-ce70-43fd-3de4-08dc6ef8595f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?WHNCOTA1bGsyM29MUzI2T2haMDk1aXZFRDVSajhScFBDeUtXQlArSm5OMlh5?=
 =?utf-8?B?blptcFA2amM0b3Z0TENZOUF3MFI0Y2J3WUd0czdhd29CeVdseWs3U2tPdTJo?=
 =?utf-8?B?RWgrZGo5YlhBWDdYNkJzL0NrUkVuM1RCTHNwcjk2ZW1neXFiQWNGV2N3Q0p0?=
 =?utf-8?B?VHBPakFCdHpLTVhvcnFjVEtxU093ekltaThGQUZ5NlhzWjdZSjA0bkpsTGkr?=
 =?utf-8?B?cDhLaDRHa3lRMmpSWHJrVWNUY24vaHNOVGtMQVhRLzlQeHRBR3VYU25MTGFI?=
 =?utf-8?B?N2JPSmExOVVFVmJpYy96cEoreGRnQlo4ZTM4RGw4d1pBWEIzSXVTRW9ieWdo?=
 =?utf-8?B?MFZGaVA3bFNncWtDTlpRQTlBSC9ROGR4aVlpWUJCMk5zWjJzQ3lPRDNBZVZ0?=
 =?utf-8?B?VEk4cnZvK0Q0NWdsOHoybjF3M09pL2JoVzBlVEZ0ZnYxUEV0djExRTRvajVa?=
 =?utf-8?B?N2hXMkEvRDhWQzVyQ3FCZ1orQmZtckVBWUlNYTk4c3FZUjg4WDR3WUxKNE5V?=
 =?utf-8?B?NmpUeTFXYWJ2eDlXRVhMZk9wdVZHd0dBcU5JTXdFTGlHMXJsa05vZnMzd0Vx?=
 =?utf-8?B?Z3VERGFRenRJVzl1SzNPNmszc3M3N3B0OEFoL05vbVJILzd4Z2YyMld3Qnhy?=
 =?utf-8?B?ZFR4SU03ajN4QWtlOWlMN0ZJaUN0dVArZ0RuZDF0UzBSU1hBbjlDSFpDSjVY?=
 =?utf-8?B?VDE2VEY1MWVTLzJzRW5aSlF2Wk5YUmxDVUFGSEFEUUo0T3lNSkVaajREdkxD?=
 =?utf-8?B?Z3ZiK3dtT2gvWjBSM3BjQXdpN29CdVFDcURKS0hjYXJoVWVPT1lLUDlSeTFo?=
 =?utf-8?B?bnFZWEFpTzN0Kzd3ZEMrTlFkTjBDcDU5MWp3QkR3WXVQSUlNN3YvNUxFVFhu?=
 =?utf-8?B?eVFGSXB4bU53K0lESUpQWm1zM3o4TVc5T05ORXh2ZTRFTjMvMDZyZHBGa3JJ?=
 =?utf-8?B?b3NnT05MS2RoUVkyQVBUTzhGby9NTzk1RmpaMkNMbzJrR3FYQWVDOVhGdnRB?=
 =?utf-8?B?Z3dOVHVoZzlva1pReVVyUjJuNHA2VXZScmVHMzk1aktZbXM1SkdzTEpPL3pH?=
 =?utf-8?B?Mnp2b3ZnMGhlNW9sMC9LaWpoZjhHUi9UOHAxZXIwQ0p0VDg1S3dzR1NrakxJ?=
 =?utf-8?B?aWs3NzN2dHZHaVZHenkvaTJsdWZBVU1mb2EwUDN1b2h6c2xjN1BmNnpTRlFo?=
 =?utf-8?B?YlVQcUNtWVZCSFFtTXluV3I5VWlvVlRwWXFRakdQRy9hbmhENFczWks5QkxZ?=
 =?utf-8?B?ZWNsRVUvb3oxSnRZbHh4U3dqNTVxRjlxYmtnVTZUbE5tUnFOaCs4UFg4djRN?=
 =?utf-8?B?Z20yVHpNb0VrTGx4RGp6dUgvQ3RLcEQ2MG1TVnJqZDMzNzV4N0tjOCs5cFZt?=
 =?utf-8?B?c29yZnZlL1V5d0U2d2F1UmpTRXlWdDN5V0JnYk05SVRUK1NtbHlzdTJpOUVy?=
 =?utf-8?B?ZkhYMjB1M1VNdEtqNW1LM1pBRXA5cnN6dFRTZk5UK0VtVWlEOFdtWlY3eW92?=
 =?utf-8?B?d0J4c21lWmZOaGFYVDFRdG9WSTlVN21xMXdkVWRxVFY0eHNDTXFxRGNSVmJk?=
 =?utf-8?B?aVMxVWpxNmtZQ2doSEh6SGRwdm1PMGdyem5tYllzdkh3VGc5dFdJWitNd3Mw?=
 =?utf-8?B?UFZmU2lyQzFaVTkzbzh3MTJJUlFzQVlWanRxR1pXcDZrWlZ1TXJYQ295cnlT?=
 =?utf-8?B?NElVQUJCbTBqRjh2L01Uc1RHd0E0Q0d0aG5OS0JaTUoxUE1sc2tZMmhpa2wv?=
 =?utf-8?B?QkdPeis4cHpIN25mTHUwOU11VDNWRFl5emMyL2E0eW5YL1J6elVwWlhuY3k3?=
 =?utf-8?B?dDU5MElBQTRWb09INGlPRHVwSVFFVzRMbGdMdVRQcldiWVp3elgzRmhueUpD?=
 =?utf-8?Q?iNAW+MTkPu5Ek?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 00:46:50.8720
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4bee10df-ce70-43fd-3de4-08dc6ef8595f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FC.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8088

Hi Julien,

On 5/7/2024 10:35 PM, Julien Grall wrote:
> Hi,
>
> On 06/05/2024 06:17, Henry Wang wrote:
>> On 5/1/2024 9:58 PM, Anthony PERARD wrote:
>>> On Wed, Apr 24, 2024 at 11:34:39AM +0800, Henry Wang wrote:
>>>> Increase number of spi to 160 i.e. gic_number_lines() for Xilinx 
>>>> ZynqMP - 32.
>>>> This was done to allocate and assign IRQs to a running domain.
>>>>
>>>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>>>> ---
>>>>   tools/libs/light/libxl_arm.c | 3 ++-
>>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/tools/libs/light/libxl_arm.c 
>>>> b/tools/libs/light/libxl_arm.c
>>>> index dd5c9f4917..50dbd0f2a9 100644
>>>> --- a/tools/libs/light/libxl_arm.c
>>>> +++ b/tools/libs/light/libxl_arm.c
>>>> @@ -181,7 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc 
>>>> *gc,
>>>>       LOG(DEBUG, "Configure the domain");
>>>> -    config->arch.nr_spis = nr_spis;
>>>> +    /* gic_number_lines() is 192 for Xilinx ZynqMP. min nr_spis = 
>>>> 192 - 32. */
>>>> +    config->arch.nr_spis = MAX(nr_spis, 160);
>>> Is there a way that that Xen or libxl could find out what the minimum
>>> number of SPI needs to be?
>>
>> I am afraid currently there is none.
>>
>>> Are we going to have to increase that minimum
>>> number every time a new platform comes along?
>>>
>>> It doesn't appear that libxl is using that `nr_spis` value and it is
>>> probably just given to Xen. So my guess is that Xen could simply take
>>> care of the minimum value, gic_number_lines() seems to be a Xen
>>> function.
>>
>> Xen will take care of the value of nr_spis for dom0 in create_dom0()
>> dom0_cfg.arch.nr_spis = min(gic_number_lines(), (unsigned int) 992) - 
>> 32;
>> and also for dom0less domUs in create_domUs().
>>
>> However, it looks like Xen will not take care of the mininum value 
>> for libxl guests, the value from config->arch.nr_spis in guest config 
>> file will be directly passed to the domain_vgic_init() function from 
>> arch_domain_create().
>>
>> I agree with you that we shouldn't just bump the number everytime 
>> when we have a new platform. Therefore, would it be a good idea to 
>> move the logic in this patch to arch_sanitise_domain_config()?
>
> Xen domains are supposed to be platform agnostics and therefore the 
> numbers of SPIs should not be based on the HW.
>
> Furthermore, with your proposal we would end up to allocate data 
> structure for N SPIs when a domain may never needs any SPIs (such as 
> if passthrough is not in-use). This is more likely for domain created 
> by the toolstack than from Xen directly.

Agreed on both comments.

> Instead, we should introduce a new XL configuration to let the user 
> decide the number of SPIs. I would suggest to name "nr_spis" to match 
> the DT bindings.

Sure, I will introduce a new xl config for this to replace this patch. 
Thank you for the suggestion.

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Wed May 08 02:56:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 02:56:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718570.1121065 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4XTr-0003Av-4W; Wed, 08 May 2024 02:56:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718570.1121065; Wed, 08 May 2024 02:56:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4XTr-0003Ao-1c; Wed, 08 May 2024 02:56:31 +0000
Received: by outflank-mailman (input) for mailman id 718570;
 Wed, 08 May 2024 02:56:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4XTp-0003Ae-Ug; Wed, 08 May 2024 02:56:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4XTp-0005LB-T2; Wed, 08 May 2024 02:56:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4XTp-0000ex-EN; Wed, 08 May 2024 02:56:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4XTp-0005zp-Dr; Wed, 08 May 2024 02:56:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=q4wQwcMxuMG785GCAIxqHZJeXgLwU/TKBQNhofx3DyQ=; b=c+phhJRhVVpQrl4lAQYNCMvsRD
	9TBJWgwXL3PmFR2QuDZ+ZfrGN9JXZ7xyqN+Y49GQVYnHz0dSdBolM31m850Co3bshD567AAkNNQH6
	9UqzYRg2GEiEUmK2ZtCp1W61kShiCgBV9GgAr+x+fHoJYhjgdYMZQ6zukbcwwdJKqwIc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185940-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185940: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
X-Osstest-Versions-That:
    xen=ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 02:56:29 +0000

flight 185940 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185940/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185936
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185936
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185936
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185936
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 185936
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185936
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185936
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b
baseline version:
 xen                  ebab808eb1bb8f24c7d0dd41b956e48cb1824b81

Last test of basis   185936  2024-05-07 05:53:32 Z    0 days
Testing same since   185940  2024-05-07 20:08:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ebab808eb1..a2330b51df  a2330b51df267e20e66bbba6c5bf08f0570ed58b -> master


From xen-devel-bounces@lists.xenproject.org Wed May 08 03:21:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 03:21:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718578.1121076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Xro-0006rK-1i; Wed, 08 May 2024 03:21:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718578.1121076; Wed, 08 May 2024 03:21:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4Xrn-0006rD-V7; Wed, 08 May 2024 03:21:15 +0000
Received: by outflank-mailman (input) for mailman id 718578;
 Wed, 08 May 2024 03:21:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Xrn-0006r3-2c; Wed, 08 May 2024 03:21:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Xrm-0005lM-Q7; Wed, 08 May 2024 03:21:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Xrm-0001Fy-I7; Wed, 08 May 2024 03:21:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4Xrm-00047f-Hg; Wed, 08 May 2024 03:21:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=U6wxujaZzaSEyfduomq65dGTMOgmGkGZLQWoIxamONI=; b=NsYO+vlLZ9dpyjQOCcfhAxekxp
	CIQh35y5nkWTafsjewymHevz2mM2zPAaNBerZTeJ9IXf0gfuqOdjAalL9pyW77WgZ7RW+paDi2fCD
	UDUmxRw3h6fMoaZ2kJO2kOOh9VROy/wCE2hgaibHaFHpV1V3EJFvb2fTSir9RB4XNKkA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185941-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185941: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=2727231b0a6fb4c043479d132df4d36cf9f751c2
X-Osstest-Versions-That:
    ovmf=987bea6525d70cd01649472c93d19f89d41d83a2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 03:21:14 +0000

flight 185941 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185941/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 2727231b0a6fb4c043479d132df4d36cf9f751c2
baseline version:
 ovmf                 987bea6525d70cd01649472c93d19f89d41d83a2

Last test of basis   185937  2024-05-07 07:43:05 Z    0 days
Testing same since   185941  2024-05-08 01:57:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>
  Jiaxin Wu <jiaxin.wu@intel.com>
  Jiewen Yao <Jiewen.yao@intel.com>
  Ray Ni <ray.ni@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   987bea6525..2727231b0a  2727231b0a6fb4c043479d132df4d36cf9f751c2 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 08 06:30:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 06:30:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718599.1121085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4aob-000422-2X; Wed, 08 May 2024 06:30:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718599.1121085; Wed, 08 May 2024 06:30:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4aob-00041v-02; Wed, 08 May 2024 06:30:09 +0000
Received: by outflank-mailman (input) for mailman id 718599;
 Wed, 08 May 2024 06:30:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4aoZ-00041l-UV; Wed, 08 May 2024 06:30:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4aoZ-0000x6-Kc; Wed, 08 May 2024 06:30:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4aoZ-0001AL-9f; Wed, 08 May 2024 06:30:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4aoZ-0001Jz-9A; Wed, 08 May 2024 06:30:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pJ9bPejv2pngdd8qC96+p4v5SmKfKzvYWlnOhPHcP80=; b=ONHwY8+OWeRIRv40UWv3w1wshN
	E6ilmqq1tNOxkvKATbxBg/NUtXPUJ2DkxxvO68A0KiuTveHjwPzFiPq99wnZ6h/6l5R2KmAzsGQLR
	6Z0yVqDI8+gw0EoUOx0dpazkLu5zSx8UjSiNOkWr+Y9H5Zjdf0kR5awKSL509vTZ0fk8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185944-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185944: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=952b5cf94c8727b65e04d3d507c0134743be2a66
X-Osstest-Versions-That:
    ovmf=2727231b0a6fb4c043479d132df4d36cf9f751c2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 06:30:07 +0000

flight 185944 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185944/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 952b5cf94c8727b65e04d3d507c0134743be2a66
baseline version:
 ovmf                 2727231b0a6fb4c043479d132df4d36cf9f751c2

Last test of basis   185941  2024-05-08 01:57:46 Z    0 days
Testing same since   185944  2024-05-08 05:11:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abner Chang <abner.chang@amd.com>
  Brit Chesley <brit.chesley@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   2727231b0a..952b5cf94c  952b5cf94c8727b65e04d3d507c0134743be2a66 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 08 06:33:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 06:33:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718604.1121095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4aru-0004bt-IS; Wed, 08 May 2024 06:33:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718604.1121095; Wed, 08 May 2024 06:33:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4aru-0004bI-Fp; Wed, 08 May 2024 06:33:34 +0000
Received: by outflank-mailman (input) for mailman id 718604;
 Wed, 08 May 2024 06:33:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ompu=ML=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s4art-0004Zq-5b
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 06:33:33 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2bb35f7-0d04-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 08:33:31 +0200 (CEST)
Received: from CH2PR11CA0026.namprd11.prod.outlook.com (2603:10b6:610:54::36)
 by CY5PR12MB6552.namprd12.prod.outlook.com (2603:10b6:930:40::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Wed, 8 May
 2024 06:33:28 +0000
Received: from DS2PEPF00003441.namprd04.prod.outlook.com
 (2603:10b6:610:54:cafe::3e) by CH2PR11CA0026.outlook.office365.com
 (2603:10b6:610:54::36) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.45 via Frontend
 Transport; Wed, 8 May 2024 06:33:28 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS2PEPF00003441.mail.protection.outlook.com (10.167.17.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Wed, 8 May 2024 06:33:28 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 8 May
 2024 01:33:27 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 8 May
 2024 01:33:27 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 8 May 2024 01:33:26 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2bb35f7-0d04-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gO60bE+H/DC3jZl4CDEnzU+Eg4fQ9nEMgQb6wgKUhKHAnJ24xutCIWsyGTrvPt7zkh3QfzgXu05qKyzNNkcylMasBBZiJv5EbjsMAa1bEDRLc2/Q/RNjzyODHJuTNi6PSoJgkAk/6OJA0bTKnJNqQQzWePT//QHVWffsB2Tp6f0JEFe0nUqIQx6rZV7Eb+imfVB9D1VlvkRmGdBkji8Zyf139+NX1ieGebr2N4lUx83fSgKkroHbEIPjVCNLZlB9f1JeQUBYzutO+Fu/Mkq/wxCLeSv02EnvihEpoE1j3HWdH9l7GmexlXTlb5cIYXD3/GzJ3WwkHmzxuPl9GkvjiA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fHocSXdyfuCzYVEwjGoiSjVB03rI4Z4IWRghwYVOdPM=;
 b=WvlHi9gXtaVPwiyAeNjo5kmmH2sMMJhdWJH6upibiUjri1wrA5ufVWOYyEkYiuv0aUMxEYAspQaalVSy/SXqwyuPBdLcoQfHACyzQ9YTJrOSlOQNueD+0i08Tk7s+XQYgLKt8K2rM/nMrj+vgKOE5LDdSZsQS3vshFVFWEzP/yO0r71cmLAG6fBhI/+O33iYRezaVxFa2ppWCJIN9QfhG8/F+8wfXaQ603MJATuVZQZG8MsjD8qzrBYd2eb56t//K20BaxQirRjly0FXZeDnAZ7UDn8Ef/NQSwqXgZY/rNUNT7FhzV81sepGaXRR1fuFj76j1rF+pcjMMSGbmTTHiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fHocSXdyfuCzYVEwjGoiSjVB03rI4Z4IWRghwYVOdPM=;
 b=AgSM+V7HI82sRjn3PfXq7KE0m3zOELK7t7P+V3b36f1L2Wpf2SeBbeTVtmkCh94W/oMbPcDAcAdmxPKlK5txD2D1Xw24qDUqB4A25P/iQw5wAM1HzO/lqLDT4Q4GiCiuRyBDsB3n7oi85UoNLq0/YU+d1pmMREZA2O6yYqKN9fY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <5ef0df0b-00c4-4a1e-8420-393d7ce1130a@amd.com>
Date: Wed, 8 May 2024 08:33:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] xen/arm: Wrap shared memory mapping code in one
 function
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-3-luca.fancellu@arm.com>
 <ef9b0a4d-134c-4eea-927d-5cce0b330aff@amd.com>
 <E429579D-8698-4A92-BD9A-CCD686D1720B@arm.com>
 <86e0857b-32f3-4e53-b869-604f7c6c82c6@amd.com>
 <3CCC6BD3-0A1A-4AE4-BF81-DFBE3A11364C@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <3CCC6BD3-0A1A-4AE4-BF81-DFBE3A11364C@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS2PEPF00003441:EE_|CY5PR12MB6552:EE_
X-MS-Office365-Filtering-Correlation-Id: dba08317-adef-4f4f-e729-08dc6f28c591
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bTNtcEZhblhRK2JjeHViMkZNOGorWmhER091RGY3TFFyckIxMjhMZlR0a0Rt?=
 =?utf-8?B?aWFEeFRoYlhsM2tVQkFzWS84VVJqUjEvYUd4QmVZZGxUNFVXNFlnSWc1R1VU?=
 =?utf-8?B?dUQvbmV6LzhUUWt2UDF3T1lHdVl0Vi9wNllJSFl4ajlZL2F4VGgzREQyakVz?=
 =?utf-8?B?Y1ZzcGExcCtNd2xDRnlEWEhsd0JWQlBNTS9PcXg1RnNvZDdKcXNiRGx3d2Vl?=
 =?utf-8?B?UkpQSzhLOXJmNEcrTEhtcEczZzRoUTJpbk1zQkdCL0NwaVp0cEEvYzRHbkM5?=
 =?utf-8?B?dWdheUVlUTdvdnAydUFLbGw3Nloxb0NPUWVqbGY5cWcyMFFtUm01NStBQ1Qr?=
 =?utf-8?B?blRwdUpSVUdQOE1aL3FpN0JkaFM1NmY0V2xGZVArMXVpMjNveXNmb20vdWtm?=
 =?utf-8?B?elNiUmVlZmk3d2NkU1dTRWJnNHVXNk5jaHJBTzFTLzFZVDBwMWhkNjF4ZTkr?=
 =?utf-8?B?ODdOQ2Nqc3htdFNpR0ZVcGphSUQ4dzF3bnZ4RldjeEpaUHAxOStRV244N2tK?=
 =?utf-8?B?NWh3eHhmL25vemFXd2pwa3FEVUNTQk1mdGdLMzBreklkWGJxYThCWEdlNDVq?=
 =?utf-8?B?V0J1cG9NYU5aT01yMnVvdkxjZkJ4eHFZZVpuM2FaeGpvZmJSdzZoR3Ywd2RX?=
 =?utf-8?B?R0ZoTFV2Z3JFZWMwWXFINmZ0ZUVPRjczVkEzU3Q1OTVta2FxZGxsY1hhZ0hI?=
 =?utf-8?B?VGZGdFM3Zko0SWRrTzJzQzNnY1VRd1RjYXY2WTRERTV5dVZzcUJoUDk3VXUv?=
 =?utf-8?B?d1FieU5pdVpEK1JQNWhNVjFHRWE2KzdvNVhyOTA1YWR3OFc2OGpqQmQzOVNN?=
 =?utf-8?B?RFI0RlJvbk5xQkQ0MWZMRmluL0FWNE1zSVVDT3lXN2pacnlKVG9NM3I3Umg2?=
 =?utf-8?B?UTJrSWs5Y0hNcUIveFhRQXVHenlyZUVRRGZmSmFnUW1ocGRDRUcrbEZWTjk3?=
 =?utf-8?B?RDc1K0NBWEE3V0dvQXdEVFQvVDVQem1ld0ttRnRSdERvM21zYkxjMGlJRHI3?=
 =?utf-8?B?MDJheEJzWGNQSXArWW5xcTU1OEF1WjFONTY0ZTZEcVI4WEhjOUV0eUZQaWpv?=
 =?utf-8?B?N0dDdzVuWFRiN09yR2dxdm1VYzZodnMzcTBoTTVKTGhQeUt6cmQxTzg5WnlR?=
 =?utf-8?B?MldhOGdPM3N5ZkJQaCt1ZkF3cndVaWtqZ0E3NXcxVzJnMW15V2plVnZLQzZ0?=
 =?utf-8?B?eEpBS1h0UFVmL0Nrd2hHN09oNmN3d0ZMMFpDNUFlV1FZWjV2MmRIbHFYdHVS?=
 =?utf-8?B?S0dXZEtDTXBPU0dJSnhQdG9DWlp4K0xGMDdMWC9jUEJEQm16L2YxQjJnZVNN?=
 =?utf-8?B?Rk1ZT0YwTWhacDZ2NE5LWHd4MGN1Q0lacnYxQldQY2R3RkxhQ0xjeDFhc3Rp?=
 =?utf-8?B?NUV4WHhCdWZNQUhZREVWVHVLS0Q2WFYvWkJUZVVneFJQeHdaVUdLamFKeHVY?=
 =?utf-8?B?UzMrSUhtOTJFVXUyUjZrSzBWWi9pelZnUGllSytHTlppTk00UGV3ckF4K1BF?=
 =?utf-8?B?eVJsK2lya2hxak05WEt2MVhNYlVnNUhxQUI1ZVJ1blZBdjk0RlFCY3NYR3U3?=
 =?utf-8?B?REJTbzdYNnBBMVVUWDhrZmh4Z3RFeU1xM21zQ24vSmFBWTVtdDRCNkZOeWlt?=
 =?utf-8?B?ZE5HVUpQQmpHWU9Pb3RRd2dsUFJZZi9JMzBZbkZPdEhZMmNnM3RFQmRpRk1n?=
 =?utf-8?B?czB4SStaYzBZWWdUMk90azJ3eWlab2hIT3l1ZlFRWWdSUy80aUViT1JNZHpJ?=
 =?utf-8?B?c0RlL1RwZWhHS256czgwbC9pNS9tTVp3OGYxcFJGRC9HSDRvbVNkemxkd3N1?=
 =?utf-8?B?TitBNkZ5ZzFKdXlEQVhUamJjNTNhVXZSUzZoYnZIZlN0dmV3R1VScW5vUm1L?=
 =?utf-8?Q?+fUH3fHlY48+N?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 06:33:28.3034
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dba08317-adef-4f4f-e729-08dc6f28c591
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF00003441.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6552



On 07/05/2024 16:15, Luca Fancellu wrote:
> 
> 
> Hi Michal,
> 
> 
>>>>>
>>>>> int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>>>                       const struct dt_device_node *node)
>>>>> {
>>>>> @@ -249,32 +290,10 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>>>        if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
>>>>>            owner_dom_io = false;
>>>> Looking at owner_dom_io, why don't you move parsing role and setting owner_dom_io accordingly to handle_shared_mem_bank()?
>>>
>>> I think I wanted to keep all dt_* functions on the same level inside process_shm, otherwise yes, I could
>>> pass down shm_node and do the reading of role_str in handle_shared_mem_bank, or I could derive
>>> owner_dom_io from role_str being passed or not, something like:
>>>
>>> role_str = NULL;
>>> dt_property_read_string(shm_node, "role", &role_str)
>>>
>>> [inside handle_shared_mem_bank]:
>>> If ( role_str )
>>>    owner_dom_io = false;
>>>
>>> And pass only role_str to handle_shared_mem_bank.
>>>
>>> Is this comment to reduce the number of parameters passed? I guess it’s not for where we call
>> In this series as well as the previous one you limit the number of arguments passed to quite a few functions.
>> So naturally I would expect it to be done here as well. owner_dom_io is used only by handle_shared_mem_bank, so it makes more sense to move parsing to this
>> function so that it is self-contained.
> 
> Ok I will, just to be on the same page here, you mean having dt_property_read_string inside handle_shared_mem_bank?
> Or the above example would work for you as well? That one would have role_str passed instead of shm_node.
I'm ok with the solution above to pass role_str.

~Michal


From xen-devel-bounces@lists.xenproject.org Wed May 08 07:11:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 07:11:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718614.1121105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4bS2-0001Po-DN; Wed, 08 May 2024 07:10:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718614.1121105; Wed, 08 May 2024 07:10:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4bS2-0001Ph-Ao; Wed, 08 May 2024 07:10:54 +0000
Received: by outflank-mailman (input) for mailman id 718614;
 Wed, 08 May 2024 07:10:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vH+U=ML=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s4bS0-0001Pa-Gd
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 07:10:52 +0000
Received: from mail-oo1-xc31.google.com (mail-oo1-xc31.google.com
 [2607:f8b0:4864:20::c31])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19cf6894-0d0a-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 09:10:51 +0200 (CEST)
Received: by mail-oo1-xc31.google.com with SMTP id
 006d021491bc7-5b20c4eda82so1596770eaf.3
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 00:10:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19cf6894-0d0a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715152250; x=1715757050; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Zi75ktYMuiFMsqMhE3oQ1E3tXNLWFdpqtaKZxL0bRec=;
        b=erJQhViCEd9QV0fz2vryXV4nVp3NxYjn2xEjivpKYM2sZWF5zNhywF3ZD5fyonP/vc
         eNzDrhxKD3KxNz8H9n6UyJdGcWYsm903ollQ172AmaayWA0wHAJy1YCfwRhZc/j3b1Ye
         IfObFOqPjCKllcqzcwIqxbhFLI4hTSQMebmXF0hxCfXf1GuvQvbBaGlKbMICAZXpgm3t
         gZU33DUOwWDVDbN1IKKvS19fmFNmdx0O/V0dqDQZ88rWTgjaqBWtsoC6OHM0TcV2XKrl
         0LoKKieDoclIRpR1qfk4Tz1q1ksODgyfBJ1Sfz8QirRuNMCsaKrToQQ/hctAvyOJUeV4
         PODQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715152250; x=1715757050;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Zi75ktYMuiFMsqMhE3oQ1E3tXNLWFdpqtaKZxL0bRec=;
        b=thRSUBR+FdyW07d42tsx9U4IS+9SQIJj7w9MGG4gWj9E8SldMEkx3tgOJt42d4CXy0
         pO4RWc6lYyc3w6kj9O01O2tNC3c6T3jC8XP5RhiF2LdTOvUPvdIYWts9Fhuh4QF211rO
         lfltoiNAeYjeNloJBnAmq//ybumMBzFp2p9coxJGpkaijMUKj/mLg9tr3U8buzcomLdo
         8zVX9JpXgECtsG70y9ADgwgYB25SbPTVqqykHqX98B5lZlT/PCHhGibTeQ0Anl71mP/2
         6PWyebxItv9ndW+s3oO544hluDFDHuox1ZgPEeXpjrArZ1mbwXVvFKsuxwGpiM7C0qeS
         kKIw==
X-Gm-Message-State: AOJu0Ywz7KtXahw3uLZbtqVRBI2AIWCBAyAtoixm6VdFROyS9h/p4IS1
	5VqYUJLH351ZXODlqwzFtuxkNz1mjMWt9j6Ok4G4I7hR8JVibIc65eFSg7203sx3iNbMnSzWRV3
	MNb0VOdyEAlI1ASoD9FqXP6XGHIVrab0UTs7UQg==
X-Google-Smtp-Source: AGHT+IEtaaIGSPo28NZBXRaOTyHq1x2iLQutmbFpfTb/gbinupTDsQ9DB+58YSI+/LvR3z8YZwXUjTOZo2UMuOaGwM4=
X-Received: by 2002:a4a:8c23:0:b0:5b2:2b9:a502 with SMTP id
 006d021491bc7-5b24d15bd67mr1503817eaf.2.1715152250006; Wed, 08 May 2024
 00:10:50 -0700 (PDT)
MIME-Version: 1.0
References: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
 <b965ee57-c6fc-459f-a5fd-fae47dc6ea9d@suse.com>
In-Reply-To: <b965ee57-c6fc-459f-a5fd-fae47dc6ea9d@suse.com>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Wed, 8 May 2024 09:10:37 +0200
Message-ID: <CAHUa44FsFi0F4tz3jN+d3WkR4dTPJ1HdUru+ME1YQyzMSbMG7Q@mail.gmail.com>
Subject: Re: Referencing domain struct from interrupt handler
To: Jan Beulich <jbeulich@suse.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	George Dunlap <george.dunlap@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Fri, May 3, 2024 at 12:32=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 03.05.2024 09:45, Jens Wiklander wrote:
> > Hi Xen maintainers,
> >
> > In my patch series [XEN PATCH v4 0/5] FF-A notifications [1] I need to
> > get a reference to a domain struct from a domain ID and keep it from
> > being destroyed while using it in the interrupt handler
> > notif_irq_handler() (introduced in the last patch "[XEN PATCH v4 5/5]
> > xen/arm: ffa: support notification"). In my previous patch set [2] I
> > used get_domain_by_id(), but from the following discussion
> > rcu_lock_live_remote_domain_by_id() seems to be a better choice so
> > that's what I'm using now in the v4 patch set. The domain lock is held
> > during a call to vgic_inject_irq() and unlocked right after.
> >
> > While we're reviewing the patch set in [1] I'd like to check the
> > approach with rcu_lock_live_remote_domain_by_id() here.
> >
> > What do you think? Is using rcu_lock_live_remote_domain_by_id() the
> > best approach?
>
> Is it guaranteed that the IRQ handler won't ever run in the context of a
> vCPU belonging to the domain in question? If not, why the "remote" form
> of the function?

No, that's my mistake.

>
> Furthermore, is it guaranteed that the IRQ handler won't interrupt code
> fiddling with the domain list? I don't think it is, since
> domlist_update_lock isn't acquired in an IRQ-safe manner. Looks like
> you need to defer the operation on the domain until softirq or tasklet
> context.

Thanks for the suggestion, I'm testing it as:
static DECLARE_TASKLET(notif_sri_tasklet, notif_sri_action, NULL);

static void notif_irq_handler(int irq, void *data)
{
    tasklet_schedule(&notif_sri_tasklet);
}

Where notif_sri_action() does what notif_irq_handler() did before
(using rcu_lock_domain_by_id()).

I have one more question regarding this.

Even with the RCU lock if I understand it correctly, it's possible for
domain_kill() to tear down the domain. Or as Julien explained it in
another thread [3]:
> CPU0: ffa_get_domain_by_vm_id() (return the domain as it is alive)
>
> CPU1: call domain_kill()
> CPU1: teardown is called, free d->arch.tee (the pointer is not set to NUL=
L)
>
> d->arch.tee is now a dangling pointer
>
> CPU0: access d->arch.tee
>
> This implies you may need to gain a global lock (I don't have a better
> idea so far) to protect the IRQ handler against domains teardown.

I'm trying to address that (now in a tasklet) with:
    /*
     * domain_kill() calls ffa_domain_teardown() which will free
     * d->arch.tee, but not set it to NULL. This can happen while holding
     * the RCU lock.
     *
     * domain_lock() will stop rspin_barrier() in domain_kill(), unless
     * we're already past rspin_barrier(), but then will d->is_dying be
     * non-zero.
     */
    domain_lock(d);
    if ( !d->is_dying )
    {
        struct ffa_ctx *ctx =3D d->arch.tee;

        ACCESS_ONCE(ctx->notif.secure_pending) =3D true;
    }
    domain_unlock(d);

It seems to work, but I'm worried I'm missing something or abusing
domain_lock(). I can do this in v5 of the patch set if that helps to
see what I mean.

[3] https://patchew.org/Xen/20240426084723.4149648-1-jens.wiklander@linaro.=
org/20240426084723.4149648-6-jens.wiklander@linaro.org/#c7a672a7-02f8-4d24-=
b87e-1b8439d7eb4c@xen.org

Thanks,
Jens

>
> Jan
>
> > [1] https://patchew.org/Xen/20240502145645.1201613-1-jens.wiklander@lin=
aro.org/
> > [2] https://patchew.org/Xen/20240426084723.4149648-1-jens.wiklander@lin=
aro.org/
> >
> > Thanks,
> > Jens
>


From xen-devel-bounces@lists.xenproject.org Wed May 08 07:50:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 07:50:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718621.1121115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4c3o-0005su-6i; Wed, 08 May 2024 07:49:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718621.1121115; Wed, 08 May 2024 07:49:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4c3o-0005sn-4D; Wed, 08 May 2024 07:49:56 +0000
Received: by outflank-mailman (input) for mailman id 718621;
 Wed, 08 May 2024 07:49:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/w98=ML=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s4c3n-0005sh-Mn
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 07:49:55 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8c488c20-0d0f-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 09:49:51 +0200 (CEST)
Received: from CH0PR03CA0017.namprd03.prod.outlook.com (2603:10b6:610:b0::22)
 by PH8PR12MB7375.namprd12.prod.outlook.com (2603:10b6:510:215::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May
 2024 07:49:47 +0000
Received: from CH1PEPF0000AD7D.namprd04.prod.outlook.com
 (2603:10b6:610:b0:cafe::fd) by CH0PR03CA0017.outlook.office365.com
 (2603:10b6:610:b0::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Wed, 8 May 2024 07:49:46 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000AD7D.mail.protection.outlook.com (10.167.244.86) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7452.22 via Frontend Transport; Wed, 8 May 2024 07:49:46 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 8 May
 2024 02:49:45 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 8 May 2024 02:49:44 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c488c20-0d0f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YssFiGC9sbeMVOmlbZjwkqChFvt7qw7+0H4hDX9HSg/qHwUjpom09cGm0/jWPEl2Ce0MwWUjvij9avsZAG26AGMk5mD1U63eVFyzmx61ZrX9+f244ocxaW+mf0690ThMNzQPL8PWWkElbIKdwRYKKwy1pMCAg0TUlq+grI2Q3za4gSYJSx3oArPjsm2h0GZ1qkOKJb6meobjUGC4yWgv8dtzOX5v7DQzLXJlFdzcp4dmopRyZxm0nFt0NReUznt44b2qh9Kzc6wMWN7G5VIV0uFVIf2bzevkFoLV7OPBYVJNPaDh+00JEiu+g5/d2mD/vNTTwa1l6ySnlGAif6ok5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ilLXy8IYzm5SkskMIgeJHFmvXi8IaiFQks8Xz1TbKks=;
 b=gf9MN/AS3vAJox+j29tyjI/7rqmH0L8NyvduEJpSIYwF4P7dj39mHZatbTAYqbO8zM1XlxQUJ3Uy9ITm0ZCn12nVjDdKEHj9Mez1zUGH+tsD0Gw0F7ju/qIroZeUBJkt0NEZitRtyEIv3UHGoY+K4rLcLoDLYyLzmmZzxMSdsCOdpxlIXUfplR5dpDP/4cszJlbi1C6zK1w+Y1T8RLqJ5OWXlEK7wuFjtKMQxn3C0IpbQepol7ECAkm8mHD3nqH8BaOPJ2JtUA2czG9wiBAfsO6/w1PcW1efNF9XbizUkBtyn9FELjQ3/X2JOfx2LjPt53XeF73b1XJ7hb8kKe3FKg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ilLXy8IYzm5SkskMIgeJHFmvXi8IaiFQks8Xz1TbKks=;
 b=MgkB/sFucPn6OH+vsAingFTcbHGgkpfleYO2jhje6xHukxM2b0JXrcckfGrLxJUflNZmoZbM0ESLW6JoE2yBoP9vAcreZB5vdLqqksHkFRLmcmSosweAe5PVRcMYJIlbxieA0j+0kXxw2yy02M6EP1inSzTGSy6o/mbZvpivf+A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
Date: Wed, 8 May 2024 15:49:43 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7D:EE_|PH8PR12MB7375:EE_
X-MS-Office365-Filtering-Correlation-Id: c075c5d5-3f0b-442c-4b55-08dc6f336e34
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Sml0a0ppczFKZVcvVWV4bmsrU3ZJc2t6VXNOTmtPR0U5YXFuKzQvOTdlSlB1?=
 =?utf-8?B?blBQZEZQNGlYUSs0L3FTelBqaTJab2NuQVJkQzdscGxDTkh4MEJiKzVJS3o1?=
 =?utf-8?B?SldlUnZwMDRraDdGcUZyc3o4cW82MGZiL3B1Z2RvejJuNFUrMDBTYnNrN0FR?=
 =?utf-8?B?Q2kxdU91NFV2dlR6UWxjd1BhajFuZUovUkVZVFNVYmw1dGhwWXczS3k0T2JW?=
 =?utf-8?B?WmdZVncwcFE3UkVFRTR2anhqejhsOEZHWHBTQ0Y0TXRuRGtmT2Y4akpCSm9J?=
 =?utf-8?B?SVhkdWVhbW1IYXRIZ0dpTUVsUStaeVBhaTlsWXo3S1hwV0lJK3dGUVhrc1J6?=
 =?utf-8?B?WVI3ejFwMlZsWktqUGlvSHJNU1pqZVdXODJZcjJHOElXOFIwNzcrV3YrRnI5?=
 =?utf-8?B?T0JEeHdKNlMrV0ZyV0lielp4Wkl6L0dVZFdDY2UycFpFazkxVjdkNEUyMkxq?=
 =?utf-8?B?MlNYQk82dG1VUzI4VXlWc1l6K2o2SVYzK2NMa2ViM2lEVncyL2ZPQUVURG1C?=
 =?utf-8?B?STVycTk5aXE4L0FGQUxnM0tEdVNnN1lvR0c0NXlUTVlsNUpOLy95S0F5bGJJ?=
 =?utf-8?B?Rkl0cWs0OXlVc1kxb0RtZXBZNG5ibDAzN0piblNobDBGME43S0FWdEhRRHB4?=
 =?utf-8?B?RmFsc0NvOE41eStBYnVzYVB6V2lUN3FmREE4S0o0aUE4em9rbzVralpwUWxH?=
 =?utf-8?B?OEowQmJmeWhWOTNTRlJhM0hWUnUrdUhrdk1zSzF0OGhsTnNIWExlUW5DM0Uv?=
 =?utf-8?B?SFArMU5CbVpGY2lqMU04T1UzemVMS0dyZXdiZjBoVkorYkhJSXFIMmFwd3ZJ?=
 =?utf-8?B?RFdPTEpRVFVUSHB6aXJmQThBTWh4OGNwK3V3c3kzL3RoeFRMeTVNV0Z4aDE0?=
 =?utf-8?B?UVpkQjdqa09MT1YrN1hRN0kvZTRvWWpqaXRnSFFXMVRDM1Y3SHRjNU9PY2h6?=
 =?utf-8?B?YkdkSlhPRWxDcXM3R1VYbC9mMGtiZUxZeitxTVJHaENYdlJJdWRnZnprRnBq?=
 =?utf-8?B?Y01PWXdHRnR2VXJSOWlqdzNITy9MekM2aWlqamJpay9LOFlRRXFPRXYyY1Nm?=
 =?utf-8?B?TXdIamoyT296Y1FiU3drZlBhbkVZQmJaYmVyam01OXkxVnB2eVNQZUd6TVZT?=
 =?utf-8?B?N0NRMkVUei83UWFyT1dFdXF6SGJKSmRpdG1SZkMyeGQwWHBQcGRXdVJEUzRB?=
 =?utf-8?B?RFhCNWlWbUMrSnM5Tmw5aThDUk1leGlEYlJ3N2xDWkFqOW1HQnh1ZjBPNG1p?=
 =?utf-8?B?ek1SejE0RFA3dDkvZjFjWm1YV1FVMmo5a0NJWVJqL1FsZmF0UkJlZXMzT0xk?=
 =?utf-8?B?VDRxL1RSUkpRUFM0TVBSRURHMVpZS3NjWWxTczRqMHlUYkNpN2lsZm80ZGNN?=
 =?utf-8?B?MGRzVmtHSG1HbHVTVHEvYzhWa0J1bkFXMUZWSVhPSkZJTUpKanIwNlNjcFB2?=
 =?utf-8?B?dWI0MjRmQzF0WWhWUy9OdWoydHFwRlJwLy9tKzJodDJRSEdFQkF5TENLTStz?=
 =?utf-8?B?anlTbnJwRmNiTTFodG41dmZDc0JQMzNkQUNwai9kbVlCUkFZQmI0ZlcxNnpW?=
 =?utf-8?B?NUx2NE9IUWU0QWxwcC9wU3hDTmhBSHJLNHBodFhrZWQvR1FOOTJYL2ZXNWhD?=
 =?utf-8?B?Qmt6dzF2cWEybk1FT0d2d0c2RWxCKzZIWEd2OFJHVEk4UGQwcjBJb004L1Zh?=
 =?utf-8?B?V2VwZStVbmhsRGFnRWphYTBqNUY3OExWTkRQc0NBUEM0d1dPckx0QnRtTDBN?=
 =?utf-8?B?RjNvSHA3dUMyWFRzbzFNMXkzakQrZW1zcUNpR3lYeTUzaDJmMDZ2ZXB0MUw1?=
 =?utf-8?B?eUFnQlFiUDRZcVJuZm9RUkZlOFZQVjBudnhyVHdLVFpKays3NklQMUNZRWdz?=
 =?utf-8?Q?R2YDuP/UaSlpg?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 07:49:46.2077
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c075c5d5-3f0b-442c-4b55-08dc6f336e34
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD7D.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7375

Hi Julien,

On 5/8/2024 5:54 AM, Julien Grall wrote:
> Hi Henry,
>>> What if the DT overlay is unloaded and then reloaded? Wouldn't the 
>>> same interrupt be re-used? As a more generic case, this could also 
>>> be a new bitstream for the FPGA.
>>>
>>> But even if the interrupt is brand new every time for the DT 
>>> overlay, you are effectively relaxing the check for every user (such 
>>> as XEN_DOMCTL_bind_pt_irq). So the interrupt re-use case needs to be 
>>> taken into account.
>>
>> I agree. I think IIUC, with your explanation here and below, could we 
>> simplify the problem to how to properly handle the removal of the IRQ 
>> from a running guest, if we always properly remove and clean up the 
>> information when remove the IRQ from the guest? In this way, the IRQ 
>> can always be viewed as a brand new one when we add it back.
>
> If we can make sure the virtual IRQ and physical IRQ is cleaned then yes.
>
>> Then the only corner case that we need to take care of would be...
>
> Can you clarify whether you say the "only corner case" because you 
> looked at the code? Or is it just because I mentioned only one?

Well, I indeed checked the code and to my best knowledge the corner case 
that you pointed out would be the only one I can think of.

>>> Xen allows the guest to enable a vIRQ even if there is no pIRQ 
>>> assigned. Thanksfully, it looks like the vgic_connect_hw_irq(), in 
>>> both the current and new vGIC, will return an error if we are trying 
>>> to route a pIRQ to an already enabled vIRQ.
>>>
>>> But we need to investigate all the possible scenarios to make sure 
>>> that any inconsistencies between the physical state and virtual 
>>> state (including the LRs) will not result to bigger problem.
>>>
>>> The one that comes to my mind is: The physical interrupt is 
>>> de-assigned from the guest before it was EOIed. In this case, the 
>>> interrupt will still be in the LR with the HW bit set. This would 
>>> allow the guest to EOI the interrupt even if it is routed to someone 
>>> else. It is unclear what would be the impact on the other guest.
>>
>> ...same as this case, i.e.
>> test_bit(_IRQ_INPROGRESS, &desc->status) || !test_bit(_IRQ_DISABLED, 
>> &desc->status)) when we try to remove the IRQ from a running domain.
>
> We already call ->shutdown() which will disable the IRQ. So don't we 
> only need to take care of _IRQ_INPROGRESS?

Yes you are correct.

>> we have 3 possible states which can be read from LR for this case : 
>> active, pending, pending and active.
>> - I don't think we can do anything about the active state, so we 
>> should return -EBUSY and reject the whole operation of removing the 
>> IRQ from running guest, and user can always retry this operation.
>
> This would mean a malicious/buggy guest would be able to prevent a 
> device to be de-assigned. This is not a good idea in particular when 
> the domain is dying.
>
> That said, I think you can handle this case. The LR has a bit to 
> indicate whether the pIRQ needs to be EOIed. You can clear it and this 
> would prevent the guest to touch the pIRQ. There might be other 
> clean-up to do in the vGIC datastructure.

I probably misunderstood this sentence, do you mean the EOI bit in the 
pINTID field? I think this bit is only available when the HW bit of LR 
is 0, but in our case the HW is supposed to be 1 (as indicated as your 
previous comment). Would you mind clarifying a bit more? Thanks!

> Anyway, we don't have to handle removing an active IRQ when the domain 
> is still running (although we do when the domain is destroying). But I 
> think this would need to be solved before the feature is (security) 
> supported.
>
>> - For the pending (and active) case,
>
> Shouldn't the pending and active case handled the same way as the 
> active case?

Sorry, yes you are correct.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Wed May 08 09:19:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 09:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718639.1121126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4dSH-0000xO-Qe; Wed, 08 May 2024 09:19:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718639.1121126; Wed, 08 May 2024 09:19:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4dSH-0000xH-NU; Wed, 08 May 2024 09:19:17 +0000
Received: by outflank-mailman (input) for mailman id 718639;
 Wed, 08 May 2024 09:19:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4dSG-0000x7-EO; Wed, 08 May 2024 09:19:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4dSG-0004O2-6S; Wed, 08 May 2024 09:19:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4dSF-0005TP-S4; Wed, 08 May 2024 09:19:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4dSF-0004J3-Rb; Wed, 08 May 2024 09:19:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=esGsPCRdMyX6oGlQa7SjglmtEcIUEUceDHDcn+kctB0=; b=2BiYrAYc0oA/8xq7vc+A6scISx
	r4z6YXHjStKWRB/qKbm0Y3c/YUe7PTh9syUMOsrHaDU2aJtCKWhaA+cLWe2otTooQOr1EId6wkwWA
	HMpXetz767typ3vvhTo69YDj9S3mc25KVlTpSc/IgjdCuwTv7IeeLfDbgM79QUFF3Lfg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185942-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185942: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
X-Osstest-Versions-That:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 09:19:15 +0000

flight 185942 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185942/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 185940 pass in 185942
 test-armhf-armhf-xl-credit1   8 xen-boot                   fail pass in 185940

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185940 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185940 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185940
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185940
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185940
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185940
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185940
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185940
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b
baseline version:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b

Last test of basis   185942  2024-05-08 03:00:34 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed May 08 10:30:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 10:30:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718661.1121141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4eYd-0001Ke-7F; Wed, 08 May 2024 10:29:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718661.1121141; Wed, 08 May 2024 10:29:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4eYd-0001KX-3e; Wed, 08 May 2024 10:29:55 +0000
Received: by outflank-mailman (input) for mailman id 718661;
 Wed, 08 May 2024 10:29:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eYc-0001KL-1b; Wed, 08 May 2024 10:29:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eYb-0005e1-Ub; Wed, 08 May 2024 10:29:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eYb-0007J7-LC; Wed, 08 May 2024 10:29:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eYb-00025Y-Kj; Wed, 08 May 2024 10:29:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=w9tJC/nvo2KCot4gEpkFduiMvQSzr/PeHQdu2HZVUn0=; b=GxBmqH4OdjNPCnmK4wqe33E1F3
	GgJiOXhek/HtApuypmjo8pdCtyaWPB9UwYIKemaL19QrKeUjO3e5rJwW/ktj6hGdDcogAEzc7/1sF
	f7Iaqwfzj2pnypONv3qNKernStNbXgq6O91LqzHmC9IzNxElligYDBWPwEgmaAnYICsw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185945-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185945: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=5cbfb93abea4410be396b5230da4ad2f6c972788
X-Osstest-Versions-That:
    ovmf=952b5cf94c8727b65e04d3d507c0134743be2a66
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 10:29:53 +0000

flight 185945 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185945/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 5cbfb93abea4410be396b5230da4ad2f6c972788
baseline version:
 ovmf                 952b5cf94c8727b65e04d3d507c0134743be2a66

Last test of basis   185944  2024-05-08 05:11:09 Z    0 days
Testing same since   185945  2024-05-08 08:44:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jiaxin Wu <jiaxin.wu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   952b5cf94c..5cbfb93abe  5cbfb93abea4410be396b5230da4ad2f6c972788 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 08 10:38:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 10:38:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718668.1121151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ehA-0002yR-0c; Wed, 08 May 2024 10:38:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718668.1121151; Wed, 08 May 2024 10:38:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4eh9-0002yK-U7; Wed, 08 May 2024 10:38:43 +0000
Received: by outflank-mailman (input) for mailman id 718668;
 Wed, 08 May 2024 10:38:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eh8-0002yA-5v; Wed, 08 May 2024 10:38:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eh8-0005ms-1M; Wed, 08 May 2024 10:38:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eh7-0007VI-Lz; Wed, 08 May 2024 10:38:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4eh7-0000mL-LV; Wed, 08 May 2024 10:38:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dXrlKTqYhpc2LX0CSc0DYKXpTD7HoilBRCZfmBHVgnA=; b=dIeIlwqZ1R9xbcJORq+4+8dPOE
	vzIPSO9sQAygwGswDFY4A6DSVdrpowhvtwnfhP0SIknWcdcUD1b+O1MRXOCMBNSQXKylGd+ZKziK+
	VBX4l1MepwXnnXkFTp6RWuYotcyhPRiAg0BLE9yNwo3j2ZEmuzqHHQWYC0TuB1NYuVb4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185943-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185943: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=df9ffb025646d649f981ca25600225bc8cf794d1
X-Osstest-Versions-That:
    libvirt=da95bcb6b2d9b04958e0f2603202801dd29debb8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 10:38:41 +0000

flight 185943 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185943/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185934
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              df9ffb025646d649f981ca25600225bc8cf794d1
baseline version:
 libvirt              da95bcb6b2d9b04958e0f2603202801dd29debb8

Last test of basis   185934  2024-05-07 04:20:24 Z    1 days
Testing same since   185943  2024-05-08 04:22:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   da95bcb6b2..df9ffb0256  df9ffb025646d649f981ca25600225bc8cf794d1 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 08 11:04:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:04:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718679.1121192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4f68-0007bd-JF; Wed, 08 May 2024 11:04:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718679.1121192; Wed, 08 May 2024 11:04:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4f68-0007bW-Fg; Wed, 08 May 2024 11:04:32 +0000
Received: by outflank-mailman (input) for mailman id 718679;
 Wed, 08 May 2024 11:04:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gEdx=ML=xenbits.xen.org=julieng@srs-se1.protection.inumbo.net>)
 id 1s4f66-0006tz-P8
 for xen-devel@lists.xen.org; Wed, 08 May 2024 11:04:30 +0000
Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bc7f667a-0d2a-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 13:04:28 +0200 (CEST)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julieng@xenbits.xen.org>)
 id 1s4f5w-0006DQ-PL; Wed, 08 May 2024 11:04:20 +0000
Received: from julieng by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <julieng@xenbits.xen.org>)
 id 1s4f5w-0005EO-KC; Wed, 08 May 2024 11:04:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc7f667a-0d2a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=6d+yGvKAp/ietMPZl7Ya/IGpIx85yuZHje+WdkXJkGk=; b=yrH6e/TTf/PeSSaP5rafYXKFiW
	2ARQx7toKnniWwOpGiOJcHHrj7hMGADfEcuq128Q3UiNv6crEHIR9wunmHNhFn5viuUmWFznVTgDO
	e0D8hn90KIOH+/uXar/DBMZeTpGV1QncR95n1WmXTiPHRTLHcYJ3QlL6LpvZtKW+fbf0=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 457 v2 - Linux/xen-netfront: Memory leak
 due to missing cleanup function
Message-Id: <E1s4f5w-0005EO-KC@xenbits.xenproject.org>
Date: Wed, 08 May 2024 11:04:20 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

                    Xen Security Advisory XSA-457
                              version 2

    Linux/xen-netfront: Memory leak due to missing cleanup function

UPDATES IN VERSION 2
====================

* Clarify the XSA is in netfront and *not* netback
* Clarify the impact: only the guest may crash

ISSUE DESCRIPTION
=================

In netfront, xennet_alloc_one_rx_buffer() failed to call the
appropriate clean-up function, resulting in a memory leak.

IMPACT
======

A malicious guest userspace process can exhaust memory resources
within the guest kernel, potentially leading to a guest crash (Denial
of Service). It is not known whether it can be triggered remotely.

VULNERABLE SYSTEMS
==================

Systems with guests running Linux 5.9 and later with Xen PV network
devices are affected.

MITIGATION
==========

For HVM guests, using emulated network devices will avoid this issue.

RESOLUTION
==========

The following patch in Linux resolves the issue:

https://git.kernel.org/torvalds/c/037965402a010898d34f4e35327d22c0a95cd51f

A copy of which is attached.

xsa457.patch           Linux 5.9

$ sha256sum xsa457*
9d6ae3da27f1ff92f9f45c800822beecda603d6dea6726207cee6c768416114c  xsa457.patch
$


NOTE ON THE LACK OF EMBARGO
===========================

The issue was reported initially on a public bug tracker and fixed in
public before it was realized that there was a security aspect.
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmY7W/gMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZnPQIAIPhOEXsSKutZJF776KKDmoNDmZ00SikkfZ9tZW8
LyiNNJ7l7tDN3A5EVJn4l8Xos+PFaadNIXdaLKemRt17nP4Qw+UzjvBTiTbou+m7
OGUGsRMCNkfpv8OEi/U91o3W3uEE/tL7ahws/wAnOzEfcbTFl5alTDfuDfrtOaiA
1Uz37QO0GNQSD+n91SyosqAljfbAvWNQMLJ+Iz9YB6BonVwsWWNeHjF1N9zDWv3k
pD+DVOa60FYIA3xxeJveZO3ZLA6oBo5wyKiQ8p3bun9X9W5+i6PrzWewnsWCvya+
Yyi0xTZ2YBzo+eNFpQ9OKqjDVoSREx9l9Ef0YvSStR0/aBw=
=/9cg
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa457.patch"
Content-Disposition: attachment; filename="xsa457.patch"
Content-Transfer-Encoding: base64

RnJvbSAwMzc5NjU0MDJhMDEwODk4ZDM0ZjRlMzUzMjdkMjJjMGE5NWNkNTFm
IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKZXNwZXIgRGFuZ2Fh
cmQgQnJvdWVyIDxoYXdrQGtlcm5lbC5vcmc+CkRhdGU6IFdlZCwgMjcgTWFy
IDIwMjQgMTM6MTQ6NTYgKzAxMDAKU3ViamVjdDogeGVuLW5ldGZyb250OiBB
ZGQgbWlzc2luZyBza2JfbWFya19mb3JfcmVjeWNsZQoKTm90aWNlIHRoYXQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKSBpcyBpbnRyb2R1Y2VkIGxhdGVyIHRo
YW4gZml4ZXMgdGFnIGluCmNvbW1pdCA2YTViY2Q4NGU4ODYgKCJwYWdlX3Bv
b2w6IEFsbG93IGRyaXZlcnMgdG8gaGludCBvbiBTS0IgcmVjeWNsaW5nIiku
CgpJdCBpcyBiZWxpZXZlZCB0aGF0IGZpeGVzIHRhZyB3ZXJlIG1pc3Npbmcg
YSBjYWxsIHRvIHBhZ2VfcG9vbF9yZWxlYXNlX3BhZ2UoKQpiZXR3ZWVuIHY1
LjkgdG8gdjUuMTQsIGFmdGVyIHdoaWNoIGlzIHNob3VsZCBoYXZlIHVzZWQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKS4KU2luY2UgdjYuNiB0aGUgY2FsbCBw
YWdlX3Bvb2xfcmVsZWFzZV9wYWdlKCkgd2VyZSByZW1vdmVkIChpbgpjb21t
aXQgNTM1YjljNjFiZGVmICgibmV0OiBwYWdlX3Bvb2w6IGhpZGUgcGFnZV9w
b29sX3JlbGVhc2VfcGFnZSgpIikKYW5kIHJlbWFpbmluZyBjYWxsZXJzIGNv
bnZlcnRlZCAoaW4gY29tbWl0IDZiZmVmMmVjMDE3MiAoIk1lcmdlIGJyYW5j
aAonbmV0LXBhZ2VfcG9vbC1yZW1vdmUtcGFnZV9wb29sX3JlbGVhc2VfcGFn
ZSciKSkuCgpUaGlzIGxlYWsgYmVjYW1lIHZpc2libGUgaW4gdjYuOCB2aWEg
Y29tbWl0IGRiYTFiOGE3YWI2OCAoIm1tL3BhZ2VfcG9vbDogY2F0Y2gKcGFn
ZV9wb29sIG1lbW9yeSBsZWFrcyIpLgoKQ2M6IHN0YWJsZUB2Z2VyLmtlcm5l
bC5vcmcKRml4ZXM6IDZjNWFhNmZjNGRlZiAoInhlbiBuZXR3b3JraW5nOiBh
ZGQgYmFzaWMgWERQIHN1cHBvcnQgZm9yIHhlbi1uZXRmcm9udCIpClJlcG9y
dGVkLWJ5OiBMZW9uaWRhcyBTcHlyb3BvdWxvcyA8YXJ0YWZpbmRlQGFyY2hs
aW51eC5jb20+Ckxpbms6IGh0dHBzOi8vYnVnemlsbGEua2VybmVsLm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9MjE4NjU0ClJlcG9ydGVkLWJ5OiBBcnRodXIgQm9y
c2Jvb20gPGFydGh1cmJvcnNib29tQGdtYWlsLmNvbT4KU2lnbmVkLW9mZi1i
eTogSmVzcGVyIERhbmdhYXJkIEJyb3VlciA8aGF3a0BrZXJuZWwub3JnPgpM
aW5rOiBodHRwczovL2xvcmUua2VybmVsLm9yZy9yLzE3MTE1NDE2NzQ0Ni4y
NjcxMDYyLjkxMjcxMDUzODQ1OTEyMzczNjMuc3RnaXRAZmlyZXNvdWwKU2ln
bmVkLW9mZi1ieTogSmFrdWIgS2ljaW5za2kgPGt1YmFAa2VybmVsLm9yZz4K
LS0tCiBkcml2ZXJzL25ldC94ZW4tbmV0ZnJvbnQuYyB8IDEgKwogMSBmaWxl
IGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCspCgpkaWZmIC0tZ2l0IGEvZHJpdmVy
cy9uZXQveGVuLW5ldGZyb250LmMgYi9kcml2ZXJzL25ldC94ZW4tbmV0ZnJv
bnQuYwppbmRleCBhZDI5ZjM3MDAzNGU0Zi4uOGQyYWVlODg1MjZjNjkgMTAw
NjQ0Ci0tLSBhL2RyaXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCisrKyBiL2Ry
aXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCkBAIC0yODUsNiArMjg1LDcgQEAg
c3RhdGljIHN0cnVjdCBza19idWZmICp4ZW5uZXRfYWxsb2Nfb25lX3J4X2J1
ZmZlcihzdHJ1Y3QgbmV0ZnJvbnRfcXVldWUgKnF1ZXVlKQogCQlyZXR1cm4g
TlVMTDsKIAl9CiAJc2tiX2FkZF9yeF9mcmFnKHNrYiwgMCwgcGFnZSwgMCwg
MCwgUEFHRV9TSVpFKTsKKwlza2JfbWFya19mb3JfcmVjeWNsZShza2IpOwog
CiAJLyogQWxpZ24gaXAgaGVhZGVyIHRvIGEgMTYgYnl0ZXMgYm91bmRhcnkg
Ki8KIAlza2JfcmVzZXJ2ZShza2IsIE5FVF9JUF9BTElHTik7Ci0tIApjZ2l0
IDEuMi4zLWtvcmcKCg==

--=separator--


From xen-devel-bounces@lists.xenproject.org Wed May 08 11:04:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:04:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718680.1121201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4f6D-0007tU-R2; Wed, 08 May 2024 11:04:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718680.1121201; Wed, 08 May 2024 11:04:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4f6D-0007tE-OB; Wed, 08 May 2024 11:04:37 +0000
Received: by outflank-mailman (input) for mailman id 718680;
 Wed, 08 May 2024 11:04:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Pq3G=ML=cloud.com=matthew.barnes@srs-se1.protection.inumbo.net>)
 id 1s4f6C-0006tz-Ez
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 11:04:36 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c17d6f75-0d2a-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 13:04:35 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51f60817e34so5030888e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 04:04:35 -0700 (PDT)
Received: from EMEAENGAAD91498.citrite.net ([2.223.45.79])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f882089cbsm18947595e9.48.2024.05.08.04.04.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 04:04:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c17d6f75-0d2a-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715166275; x=1715771075; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=zDHEY4Ssox3Ng1oWJa+kdpIzbH8xHGv/gTRq2AfViYw=;
        b=fd6bCGvmmlNUxOfwENrz3RRYurc7OWK+6s09HEuGoTpmZvBImXAaWUEw9/wX4csoZ5
         CtF/err6mwBS8dOI1pmPXKfe/gxoUYoC/2mnvLULZBZR2BnWonbWBXDFkmLA2XNx9w8o
         1h7PhjzkbR4dQzZgFaAXN9RgSRLZygersJKd8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715166275; x=1715771075;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=zDHEY4Ssox3Ng1oWJa+kdpIzbH8xHGv/gTRq2AfViYw=;
        b=U2AttsRIvXzGkWRFpWSaASPRBR84UoJOlBh4ltRkWqy5yajAFCOYoSTy6ZWpS7Lnv9
         oIlZQpsqS+LScfq0coxmfWJ2eDt8C4hkogrGZZHDk7oS8NaM9rjCTfvF4UZM7G+n8hZH
         a60PdQiJijJznMdKQz/YGxzA/ZjazkSp6anzl/2GnzEw6orNlwtehRtDyFYu9b1N8PNC
         LZKLr3CPi10/x+ZfbjWLfKRcli9uOIyetBy1l/tR3jLG+5Gbh9AsXZs2idU67JxMVv9A
         yGJfAQuSbVlwTzo5VsyY+JChvt7l8ZuRecsCquCAbvHtcBD5f53MmVDmFog/6Q2ZrKKN
         uHrQ==
X-Gm-Message-State: AOJu0YzTzcNJDp7/abcgjtjHzyTjZhwZP2WgBQHldEcpRFXgw6EZCDXz
	iLhIdfx4rm9ZBn5FpOYwOYGAjACEe2AX0RlkGUN0RTDZ+C51k7bDSdzVULl4L3/2ERI2BkKCHXw
	H
X-Google-Smtp-Source: AGHT+IGlp2WgRocMsshp5skYmJ5RS6HmNu+5bNPhbb7IDWErEQ01aV8I7xXgkRfMdAz7VGsNVlJfzg==
X-Received: by 2002:ac2:4109:0:b0:51f:4096:dbca with SMTP id 2adb3069b0e04-5217cd48834mr1457296e87.62.1715166275046;
        Wed, 08 May 2024 04:04:35 -0700 (PDT)
From: Matthew Barnes <matthew.barnes@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Matthew Barnes <matthew.barnes@cloud.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [XEN PATCH v3] tools/lsevtchn: Use errno macro to handle hypercall error cases
Date: Wed,  8 May 2024 12:04:28 +0100
Message-Id: <8bde5ee38597a86334b86822de920802483a7179.1715100071.git.matthew.barnes@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Currently, lsevtchn aborts its event channel enumeration when it hits
its first hypercall error, namely:
* When an event channel doesn't exist at the specified port
* When the event channel is owned by Xen

lsevtchn does not distinguish between different hypercall errors, which
results in lsevtchn missing potential relevant event channels with
higher port numbers.

Use the errno macro to distinguish between hypercall errors, and
continue event channel enumeration if the hypercall error is not
critical to enumeration.

Signed-off-by: Matthew Barnes <matthew.barnes@cloud.com>
---
 tools/xcutils/lsevtchn.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/tools/xcutils/lsevtchn.c b/tools/xcutils/lsevtchn.c
index d1710613ddc5..e4b3f88fe4ec 100644
--- a/tools/xcutils/lsevtchn.c
+++ b/tools/xcutils/lsevtchn.c
@@ -3,6 +3,7 @@
 #include <stdint.h>
 #include <string.h>
 #include <stdio.h>
+#include <errno.h>
 
 #include <xenctrl.h>
 
@@ -24,7 +25,18 @@ int main(int argc, char **argv)
         status.port = port;
         rc = xc_evtchn_status(xch, &status);
         if ( rc < 0 )
-            break;
+        {
+            if ( errno == ESRCH )
+            {
+                fprintf(stderr, "Domain ID '%d' does not correspond to valid domain.\n", domid);
+                break;
+            }
+
+            if ( errno == EINVAL )
+                break;
+
+            continue;
+        }
 
         if ( status.status == EVTCHNSTAT_closed )
             continue;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 11:27:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718735.1121232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRu-0004pZ-2u; Wed, 08 May 2024 11:27:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718735.1121232; Wed, 08 May 2024 11:27:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRu-0004pS-00; Wed, 08 May 2024 11:27:02 +0000
Received: by outflank-mailman (input) for mailman id 718735;
 Wed, 08 May 2024 11:27:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4fRs-0004Lt-Oi
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 11:27:00 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2b2ec57-0d2d-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 13:27:00 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59b178b75bso779042666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 04:27:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 b3-20020a170906d10300b00a59edf10304sm1858562ejz.76.2024.05.08.04.26.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 04:26:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2b2ec57-0d2d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715167619; x=1715772419; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PfaLi13cq6Jp00njiG3Oj5QQBnM03VocbDPA5xG0o9g=;
        b=WRPzU5RgQkevL0cFdnFNTcngzUsBZ3tfNratpyaKdpvZQdc2cgedCRyVrOknXcnG/M
         gejuU+NDIsNexFfo9CqEA+9LoanVKPtXFDEEDHGYUpNlsPaMpK6iW5ZiQ43cfWl2mt5O
         DjxxzZvmkIrzo7S5a0jv3tZPSDmpZxn3UHVcQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715167619; x=1715772419;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PfaLi13cq6Jp00njiG3Oj5QQBnM03VocbDPA5xG0o9g=;
        b=UjbAB6WfJwEh+xuHjlLmw6Iy83jouPLGn1v3sKFTLVKf7TSigFObgkL3+OQJWE2Dpa
         atH/yQdAaxKxTRSiDeqXCQVPytB6aLwD7bTJjwInxWzcjcdhlJJkkNb7JPwXslGNmott
         b3V8s6+HlZTu7qXIKge1h3j6FrIV1qUAaJCwKeG7cx61qS4vFVsqm+eBkOEoqHr3rRQk
         XZ7jv0oCwq4Ee3mUywNgxNNpoTfnqMWuW+2VjFnZ1icMdSVL1LNnk5SYSXAunXlBy1FB
         iriw5E9vLVwHnk6J2xT3CX+beomMH6EFlV3I74vAny7qJruIW8prXroPQSuNs9EvNPps
         EfWA==
X-Gm-Message-State: AOJu0Yw7DH1iXprdKVcfAFXAuBS3lqDXjY14pXk4ajuTjhp7xedKda7X
	KK0GzeVmtz5ot2GGAPzo/NXYn/p7gKKxErp/tblFToixc1JUbXX3JtE+v6aBqHEE8u60fTmcM6F
	d
X-Google-Smtp-Source: AGHT+IFu7vidAerhk41EYCSsbv5K0vQLhkaZwr+s2zbmNY5pScb1HCc3PxhVMj0gScVnH3E6/JOiGA==
X-Received: by 2002:a17:906:fb85:b0:a59:c577:c5cb with SMTP id a640c23a62f3a-a59fb94a413mr171281266b.10.1715167619317;
        Wed, 08 May 2024 04:26:59 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH for-4.19 v2 3/3] xen/x86: remove foreign mappings from the p2m on teardown
Date: Wed,  8 May 2024 13:23:23 +0200
Message-ID: <20240508112323.38946-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240508112323.38946-1-roger.pau@citrix.com>
References: <20240508112323.38946-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Iterate over the p2m up to the maximum recorded gfn and remove any foreign
mappings, in order to drop the underlying page references and thus don't keep
extra page references if a domain is destroyed while still having foreign
mappings on it's p2m.

The logic is similar to the one used on Arm.

Note that foreign mappings cannot be created by guests that have altp2m or
nested HVM enabled, as p2ms different than the host one are not currently
scrubbed when destroyed in order to drop references to any foreign maps.

It's unclear whether the right solution is to take an extra reference when
foreign maps are added to p2ms different than the host one, or just rely on the
host p2m already having a reference.  The mapping being removed from the host
p2m should cause it to be dropped on all domain p2ms.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Use existing p2m max_mapped_pfn field.
 - Prevent creating foreign mappings by guests that have altp2m or nestedhvm
   enabled.
---
 CHANGELOG.md                   |  1 +
 xen/arch/x86/domain.c          |  8 +++-
 xen/arch/x86/include/asm/p2m.h | 26 +++++++------
 xen/arch/x86/mm/p2m-basic.c    | 17 +++++++++
 xen/arch/x86/mm/p2m.c          | 68 ++++++++++++++++++++++++++++++++--
 5 files changed, 103 insertions(+), 17 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8041cfb7d243..09bdb9b97578 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    - HVM PIRQs are disabled by default.
    - Reduce IOMMU setup time for hardware domain.
  - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
+ - Allow HVM/PVH domains to map foreign pages.
 
 ### Added
  - On x86:
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index dff790060605..1cb3ccddab00 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -718,7 +718,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
-    if ( altp2m && (altp2m & (altp2m - 1)) )
+    if ( altp2m & (altp2m - 1) )
     {
         dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
                 config->flags);
@@ -2387,6 +2387,7 @@ int domain_relinquish_resources(struct domain *d)
         enum {
             PROG_iommu_pagetables = 1,
             PROG_shared,
+            PROG_mappings,
             PROG_paging,
             PROG_vcpu_pagetables,
             PROG_xen,
@@ -2435,6 +2436,11 @@ int domain_relinquish_resources(struct domain *d)
         }
 #endif
 
+    PROGRESS(mappings):
+        ret = relinquish_p2m_mapping(d);
+        if ( ret )
+            return ret;
+
     PROGRESS(paging):
 
         /* Tear down paging-assistance stuff. */
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 107b9f260848..c1478ffc3647 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -383,6 +383,8 @@ struct p2m_domain {
 
     /* Number of foreign mappings. */
     unsigned long      nr_foreign;
+    /* Cursor for iterating over the p2m on teardown. */
+    unsigned long      teardown_gfn;
 #endif /* CONFIG_HVM */
 };
 
@@ -395,16 +397,7 @@ struct p2m_domain {
 #endif
 #include <xen/p2m-common.h>
 
-static inline bool arch_acquire_resource_check(struct domain *d)
-{
-    /*
-     * FIXME: Until foreign pages inserted into the P2M are properly
-     * reference counted, it is unsafe to allow mapping of
-     * resource pages unless the caller is the hardware domain
-     * (see set_foreign_p2m_entry()).
-     */
-    return !paging_mode_translate(d) || is_hardware_domain(d);
-}
+bool arch_acquire_resource_check(const struct domain *d);
 
 /*
  * Updates vCPU's n2pm to match its np2m_base in VMCx12 and returns that np2m.
@@ -720,6 +713,10 @@ p2m_pod_offline_or_broken_hit(struct page_info *p);
 void
 p2m_pod_offline_or_broken_replace(struct page_info *p);
 
+/* Perform cleanup of p2m mappings ahead of teardown. */
+int
+relinquish_p2m_mapping(struct domain *d);
+
 #else
 
 static inline bool
@@ -748,6 +745,11 @@ static inline void p2m_pod_offline_or_broken_replace(struct page_info *p)
     ASSERT_UNREACHABLE();
 }
 
+static inline int relinquish_p2m_mapping(struct domain *d)
+{
+    return 0;
+}
+
 #endif
 
 
@@ -1043,7 +1045,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         break;
 
     case p2m_map_foreign:
-        if ( !mfn_valid(nfn) )
+        if ( !mfn_valid(nfn) || p2m != p2m_get_hostp2m(p2m->domain) )
         {
             ASSERT_UNREACHABLE();
             return -EINVAL;
@@ -1068,7 +1070,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         break;
 
     case p2m_map_foreign:
-        if ( !mfn_valid(ofn) )
+        if ( !mfn_valid(ofn) || p2m != p2m_get_hostp2m(p2m->domain) )
         {
             ASSERT_UNREACHABLE();
             return -EINVAL;
diff --git a/xen/arch/x86/mm/p2m-basic.c b/xen/arch/x86/mm/p2m-basic.c
index 8599bd15c61a..d374e1e824f1 100644
--- a/xen/arch/x86/mm/p2m-basic.c
+++ b/xen/arch/x86/mm/p2m-basic.c
@@ -13,6 +13,8 @@
 
 #include <xen/event.h>
 #include <xen/types.h>
+#include <asm/altp2m.h>
+#include <asm/hvm/nestedhvm.h>
 #include <asm/p2m.h>
 #include "mm-locks.h"
 #include "p2m.h"
@@ -207,6 +209,21 @@ void p2m_final_teardown(struct domain *d)
     p2m_teardown_hostp2m(d);
 }
 
+bool arch_acquire_resource_check(const struct domain *d)
+{
+    /*
+     * altp2m is not supported as we would otherwise also need to walk the
+     * altp2m tables and drop any foreign map entries in order to drop the page
+     * reference.
+     *
+     * The same applies to nestedhvm nested p2m tables, as the type from the L0
+     * p2m is replicated into the L1 p2m, and there's no filtering that
+     * prevents foreign mappings from being created in nestedp2m.
+     */
+    return d->arch.hvm.params[HVM_PARAM_ALTP2M] == XEN_ALTP2M_disabled &&
+           !nestedhvm_enabled(d);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index ce742c12e0de..23c6c42af275 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2333,10 +2333,6 @@ static int p2m_add_foreign(struct domain *tdom, unsigned long fgfn,
     int rc;
     struct domain *fdom;
 
-    /*
-     * hvm fixme: until support is added to p2m teardown code to cleanup any
-     * foreign entries, limit this to hardware domain only.
-     */
     if ( !arch_acquire_resource_check(tdom) )
         return -EPERM;
 
@@ -2693,6 +2689,70 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
     return rc;
 }
 
+/*
+ * Remove foreign mappings from the p2m, as that drops the page reference taken
+ * when mapped.
+ */
+int relinquish_p2m_mapping(struct domain *d)
+{
+    struct p2m_domain *p2m = p2m_get_hostp2m(d);
+    unsigned long gfn, count = 0;
+    int rc = 0;
+
+    if ( !paging_mode_translate(d) )
+        return 0;
+
+    BUG_ON(!d->is_dying);
+
+    p2m_lock(p2m);
+
+    gfn = p2m->teardown_gfn;
+
+    /* Iterate over the whole p2m on debug builds to ensure correctness. */
+    while ( gfn <= p2m->max_mapped_pfn &&
+            (IS_ENABLED(CONFIG_DEBUG) || p2m->nr_foreign) )
+    {
+        unsigned int order;
+        p2m_type_t t;
+        p2m_access_t a;
+
+        _get_gfn_type_access(p2m, _gfn(gfn), &t, &a, 0, &order, 0);
+        ASSERT(IS_ALIGNED(gfn, 1UL << order));
+
+        if ( t == p2m_map_foreign )
+        {
+            ASSERT(p2m->nr_foreign);
+            ASSERT(order == 0);
+
+            rc = p2m_set_entry(p2m, _gfn(gfn), INVALID_MFN, order, p2m_invalid,
+                               p2m->default_access);
+            if ( rc )
+            {
+                printk(XENLOG_ERR
+                       "%pd: failed to unmap foreign page %" PRI_gfn " order %u error %d\n",
+                       d, gfn, order, rc);
+                ASSERT_UNREACHABLE();
+                break;
+            }
+        }
+
+        gfn += 1UL << order;
+
+        if ( !(++count & 0xff) && hypercall_preempt_check() )
+        {
+            rc = -ERESTART;
+            break;
+        }
+    }
+
+    ASSERT(gfn <= p2m->max_mapped_pfn || !p2m->nr_foreign);
+    p2m->teardown_gfn = gfn;
+
+    p2m_unlock(p2m);
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 08 11:27:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718736.1121242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRv-00054q-Bd; Wed, 08 May 2024 11:27:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718736.1121242; Wed, 08 May 2024 11:27:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRv-00054h-7Z; Wed, 08 May 2024 11:27:03 +0000
Received: by outflank-mailman (input) for mailman id 718736;
 Wed, 08 May 2024 11:27:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4fRt-0004oy-EX
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 11:27:01 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e238001f-0d2d-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 13:26:59 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59c04839caso957236066b.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 04:26:59 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 g13-20020a1709063b0d00b00a59a17eb5d2sm5787521ejf.222.2024.05.08.04.26.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 04:26:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e238001f-0d2d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715167618; x=1715772418; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+9za8P4EPzVB+2QpoWeyBB/3pcjYoli4eAelV6qAoOM=;
        b=qhnKDssikiMPAbClzmJF0xaAbBBhToVoBGTVaTz87+0kg8G/0CRpwP7h6jLXoVQwaP
         nDzjZwWZETgqTEbONM18j6IHb7ncGwqHsfYDHuMlwPWYX8lEAPeo7PaypxdF2hxpoIkI
         YpMKIGMdTlh3gKr0tMYc0Kird6poGUoB8jmC8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715167618; x=1715772418;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+9za8P4EPzVB+2QpoWeyBB/3pcjYoli4eAelV6qAoOM=;
        b=l862eFv6rKCTcJNkclWffhWCuH16A7crw/uM7zJY/0jdb6x0aUoK+WHIgF6Ovwoow+
         MxowC5fA0OnxQ5X+5Z8bgMuDg+aS16uoldiCKdfugJ2lbG1Ixtzn2N0XPkVc7pa87kjZ
         F+rIvMqHA053pj50N0jZ1teCqxJtQEY6138fP72AuGw76CLyG7Mn7H12/pPJo4AxoRb1
         qZQ48EUUoF6wvbnEvgrcA9LEFHzOMkJngmGtnrEyobBPyiAMCriFHMewr/DAQOS8n7ro
         M7B3G//GD4cOA6LYrIRytnfRuTLF2PsXABmFhTB2GfMIa5bEV1BbAtjyDe7vxoddyFmC
         VEsQ==
X-Gm-Message-State: AOJu0YwVFrsOaJ4ZGr/ZoQQcVE9M3yILTDb2wf1ackaGXH3IAaz0acqt
	pSbfngWAqqX02J1K24cu+NXxkkcLbpP2rE9wgwohBvVbPIwZu/kXzseWzzb+d0FoJytgMTrmFMZ
	l
X-Google-Smtp-Source: AGHT+IECJsbn5pM4hjExb+nFNlRr5lvugFFZyFGIDU8Jd4fM7lAesXbatc8NcLMBMlYb2Pr8VUOoCw==
X-Received: by 2002:a17:906:f28c:b0:a59:a0eb:aeb0 with SMTP id a640c23a62f3a-a59fb952bb0mr153520466b.36.1715167617901;
        Wed, 08 May 2024 04:26:57 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH for-4.19 v2 2/3] xen/x86: enable altp2m at create domain domctl
Date: Wed,  8 May 2024 13:23:22 +0200
Message-ID: <20240508112323.38946-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240508112323.38946-1-roger.pau@citrix.com>
References: <20240508112323.38946-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Enabling it using an HVM param is fragile, and complicates the logic when
deciding whether options that interact with altp2m can also be enabled.

Leave the HVM param value for consumption by the guest, but prevent it from
being set.  Enabling is now done using the misc_flags field in
xen_arch_domainconfig.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
 tools/libs/light/libxl_x86.c      | 43 +++++++++++++++++++++----------
 xen/arch/x86/domain.c             | 25 +++++++++++++++++-
 xen/arch/x86/hvm/hvm.c            | 15 ++++++++++-
 xen/include/public/arch-x86/xen.h | 18 +++++++++++++
 xen/include/public/hvm/params.h   |  9 ++-----
 5 files changed, 87 insertions(+), 23 deletions(-)

diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
index a50ec37eb3eb..86ee8132536c 100644
--- a/tools/libs/light/libxl_x86.c
+++ b/tools/libs/light/libxl_x86.c
@@ -6,8 +6,21 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
                                       libxl_domain_config *d_config,
                                       struct xen_domctl_createdomain *config)
 {
+    unsigned int altp2m = d_config->b_info.altp2m;
+
     switch(d_config->c_info.type) {
     case LIBXL_DOMAIN_TYPE_HVM:
+        /*
+         * The config parameter "altp2m" replaces the parameter "altp2mhvm".
+         * For legacy reasons, both parameters are accepted on x86 HVM guests.
+         *
+         * If the legacy field info->u.hvm.altp2m is set, activate altp2m.
+         * Otherwise set altp2m based on the field info->altp2m.
+         */
+        if (altp2m == LIBXL_ALTP2M_MODE_DISABLED &&
+            libxl_defbool_val(d_config->b_info.u.hvm.altp2m))
+            altp2m = libxl_defbool_val(d_config->b_info.u.hvm.altp2m);
+
         config->arch.emulation_flags = (XEN_X86_EMU_ALL & ~XEN_X86_EMU_VPCI);
         if (!libxl_defbool_val(d_config->b_info.u.hvm.pirq))
             config->arch.emulation_flags &= ~XEN_X86_EMU_USE_PIRQ;
@@ -26,6 +39,22 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
     if (libxl_defbool_val(d_config->b_info.arch_x86.msr_relaxed))
         config->arch.misc_flags |= XEN_X86_MSR_RELAXED;
 
+    if (d_config->c_info.type != LIBXL_DOMAIN_TYPE_PV) {
+        switch (altp2m) {
+        case LIBXL_ALTP2M_MODE_MIXED:
+            config->arch.misc_flags |= XEN_X86_ALTP2M_MIXED;
+            break;
+
+        case LIBXL_ALTP2M_MODE_EXTERNAL:
+            config->arch.misc_flags |= XEN_X86_ALTP2M_EXT;
+            break;
+
+        case LIBXL_ALTP2M_MODE_LIMITED:
+            config->arch.misc_flags |= XEN_X86_ALTP2M_LIMITED;
+            break;
+        }
+    }
+
     return 0;
 }
 
@@ -407,19 +436,9 @@ static int hvm_set_conf_params(libxl__gc *gc, uint32_t domid,
     libxl_ctx *ctx = libxl__gc_owner(gc);
     xc_interface *xch = ctx->xch;
     int ret = ERROR_FAIL;
-    unsigned int altp2m = info->altp2m;
 
     switch(info->type) {
     case LIBXL_DOMAIN_TYPE_HVM:
-        /* The config parameter "altp2m" replaces the parameter "altp2mhvm". For
-         * legacy reasons, both parameters are accepted on x86 HVM guests.
-         *
-         * If the legacy field info->u.hvm.altp2m is set, activate altp2m.
-         * Otherwise set altp2m based on the field info->altp2m. */
-        if (info->altp2m == LIBXL_ALTP2M_MODE_DISABLED &&
-            libxl_defbool_val(info->u.hvm.altp2m))
-            altp2m = libxl_defbool_val(info->u.hvm.altp2m);
-
         if (xc_hvm_param_set(xch, domid, HVM_PARAM_HPET_ENABLED,
                              libxl_defbool_val(info->u.hvm.hpet))) {
             LOG(ERROR, "Couldn't set HVM_PARAM_HPET_ENABLED");
@@ -444,10 +463,6 @@ static int hvm_set_conf_params(libxl__gc *gc, uint32_t domid,
             LOG(ERROR, "Couldn't set HVM_PARAM_TIMER_MODE");
             goto out;
         }
-        if (xc_hvm_param_set(xch, domid, HVM_PARAM_ALTP2M, altp2m)) {
-            LOG(ERROR, "Couldn't set HVM_PARAM_ALTP2M");
-            goto out;
-        }
         break;
 
     default:
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 20e83cf38bbd..dff790060605 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -637,6 +637,9 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
     bool hap = config->flags & XEN_DOMCTL_CDF_hap;
     bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
     unsigned int max_vcpus;
+    unsigned int altp2m = config->arch.misc_flags & (XEN_X86_ALTP2M_MIXED |
+                                                     XEN_X86_ALTP2M_EXT |
+                                                     XEN_X86_ALTP2M_LIMITED);
 
     if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
     {
@@ -708,13 +711,33 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         }
     }
 
-    if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED )
+    if ( config->arch.misc_flags & ~XEN_X86_MISC_FLAGS_ALL )
     {
         dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n",
                 config->arch.misc_flags);
         return -EINVAL;
     }
 
+    if ( altp2m && (altp2m & (altp2m - 1)) )
+    {
+        dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
+                config->flags);
+        return -EINVAL;
+    }
+
+    if ( altp2m && nested_virt )
+    {
+        dprintk(XENLOG_INFO,
+                "Nested virt and altp2m are mutually incompatible\n");
+        return -EINVAL;
+    }
+
+    if ( altp2m && !hap )
+    {
+        dprintk(XENLOG_INFO, "altp2m requires HAP\n");
+        return -EINVAL;
+    }
+
     return 0;
 }
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 0ce45b177cf4..f3e1b9364588 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -659,6 +659,14 @@ int hvm_domain_initialise(struct domain *d,
 
     d->arch.hvm.params[HVM_PARAM_TRIPLE_FAULT_REASON] = SHUTDOWN_reboot;
 
+    /* Set altp2m based on domctl flags. */
+    if ( config->arch.misc_flags & XEN_X86_ALTP2M_MIXED )
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_mixed;
+    else if ( config->arch.misc_flags & XEN_X86_ALTP2M_EXT )
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_external;
+    else if ( config->arch.misc_flags & XEN_X86_ALTP2M_LIMITED )
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_limited;
+
     vpic_init(d);
 
     rc = vioapic_init(d);
@@ -4163,6 +4171,12 @@ static int hvm_allow_set_param(struct domain *d,
     case HVM_PARAM_CONSOLE_EVTCHN:
     case HVM_PARAM_X87_FIP_WIDTH:
         break;
+
+    /* The following parameters are read-only. */
+    case HVM_PARAM_ALTP2M:
+        rc = -EEXIST;
+        break;
+
     /* The following parameters are deprecated. */
     case HVM_PARAM_PAE_ENABLED:
     case HVM_PARAM_DM_DOMAIN:
@@ -4204,7 +4218,6 @@ static int hvm_allow_set_param(struct domain *d,
     case HVM_PARAM_BUFIOREQ_PFN:
     case HVM_PARAM_IOREQ_SERVER_PFN:
     case HVM_PARAM_NR_IOREQ_SERVER_PAGES:
-    case HVM_PARAM_ALTP2M:
     case HVM_PARAM_MCA_CAP:
         if ( value != 0 && new_value != value )
             rc = -EEXIST;
diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
index a9a87d9b50de..bc444e7744a5 100644
--- a/xen/include/public/arch-x86/xen.h
+++ b/xen/include/public/arch-x86/xen.h
@@ -300,6 +300,24 @@ struct xen_arch_domainconfig {
  * doesn't allow the guest to read or write to the underlying MSR.
  */
 #define XEN_X86_MSR_RELAXED (1u << 0)
+
+/* altp2m options.  Only one can be set.
+ *
+ * Note that 'mixed' mode has not been evaluated for safety from a
+ * security perspective.  Before using this mode in a
+ * security-critical environment, each subop should be evaluated for
+ * safety, with unsafe subops blacklisted in XSM.
+ *
+ * Enable altp2m mixed mode.
+ */
+#define XEN_X86_ALTP2M_MIXED   (1U << 1)
+/* Enable altp2m external mode. */
+#define XEN_X86_ALTP2M_EXT     (1U << 2)
+/* Enable altp2m limited mode. */
+#define XEN_X86_ALTP2M_LIMITED (1U << 3)
+
+#define XEN_X86_MISC_FLAGS_ALL (XEN_X86_MSR_RELAXED | XEN_X86_ALTP2M_MIXED | \
+                                XEN_X86_ALTP2M_EXT | XEN_X86_ALTP2M_LIMITED)
     uint32_t misc_flags;
 };
 
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index a22b4ed45d2e..99c40b4287f1 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -246,16 +246,11 @@
 #define HVM_PARAM_VM_GENERATION_ID_ADDR 34
 
 /*
- * Set mode for altp2m:
- *  disabled: don't activate altp2m (default)
+ * Get mode for altp2m:
+ *  disabled: altp2m not active (default)
  *  mixed: allow access to all altp2m ops for both in-guest and external tools
  *  external: allow access to external privileged tools only
  *  limited: guest only has limited access (ie. control VMFUNC and #VE)
- *
- * Note that 'mixed' mode has not been evaluated for safety from a
- * security perspective.  Before using this mode in a
- * security-critical environment, each subop should be evaluated for
- * safety, with unsafe subops blacklisted in XSM.
  */
 #define HVM_PARAM_ALTP2M       35
 #define XEN_ALTP2M_disabled      0
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 08 11:27:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718733.1121211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRq-0004MB-Hw; Wed, 08 May 2024 11:26:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718733.1121211; Wed, 08 May 2024 11:26:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRq-0004M4-FG; Wed, 08 May 2024 11:26:58 +0000
Received: by outflank-mailman (input) for mailman id 718733;
 Wed, 08 May 2024 11:26:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4fRp-0004Lt-Fj
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 11:26:57 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e08bd0da-0d2d-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 13:26:56 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so1255071a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 04:26:56 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 fk4-20020a056402398400b005701eaa2023sm7556600edb.72.2024.05.08.04.26.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 04:26:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e08bd0da-0d2d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715167616; x=1715772416; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=jh48XAjrxMpbuLHFvEvd7ild4VQqTjCd5Npj4n5W6mE=;
        b=vI5+xl4lFu2SNBwb1HDLu+ndm6U6gKHwlwl9V6y+e2KmuRTrp3KV1zBCO/nGZ0KJ/r
         S6oyHvxseaksZSqCFSKU9/hO7ZuCQkAQHFLI4UA5IuRBwltNjG4nfms2jI2XC+3dhv+F
         qUkHJ7Vnc0yfSywKePyRZ3GLi/RUHUD/3sDl8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715167616; x=1715772416;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=jh48XAjrxMpbuLHFvEvd7ild4VQqTjCd5Npj4n5W6mE=;
        b=aEXA1m0IPhE4Z22icEmXULN2QiqnbwHkIt7GIbfL/ZZ3V9Mprq6NDGq9olXYzQfIoU
         I58CJMuygdGWxLrdSHUy9Xyl0utbfRUWMvlSHiO6YFPovU5m8rQWy3kGntYHJx7gRKaR
         6VoqjTU5ghFx10BJeu1BrZiWinflEvjrtIV17bmt3cBeFmJ3kgD/83T4z8eiNSJwD/1R
         8ajBSfTokrjItrmxqmjRGkqRkw/FtrsP3qHz0128hclbYhqs05L0uYA+Tv6zriJQvo4x
         7H3oU825J9jcNLUc7rleRYvMkTAMjWTJc9gopwMScztj4juqcZnwD6i8jjDQ+dImyqMQ
         Ldyg==
X-Gm-Message-State: AOJu0YyC6ks4QFVS4zhQRj8DnggWafJhEBmDT8QNzWZjqWrwSQNPDu2U
	yLDi6l5FY74jh9zHzPsIgJac/bAz7O2PhGn59kfwkS26qIDeVC9+7u3R2SJMYr7as6g6sYDpJbT
	8
X-Google-Smtp-Source: AGHT+IF5qoFu591aP56KZr2TFcCwq4kgSVGLjEnTCEI8eZFDdvm1HBTc4Z/5N+uMDrpERxBl0GCZPQ==
X-Received: by 2002:a50:f681:0:b0:572:1fb7:2a77 with SMTP id 4fb4d7f45d1cf-5731d85cc02mr2099029a12.6.1715167615641;
        Wed, 08 May 2024 04:26:55 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>
Subject: [PATCH for-4.19 v2 0/3] xen/x86: support foreign mappings for HVM/PVH
Date: Wed,  8 May 2024 13:23:20 +0200
Message-ID: <20240508112323.38946-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following series attempts to solve a shortcoming of HVM/PVH guests
with the lack of support for foreign mappings.  Such lack of support
prevents using PVH based guests as stubdomains for example.

Add support in a way similar to how it's done on Arm, by iterating over
the p2m based on the maximum gfn.

Patch 2 is not strictly needed.  Moving the enablement of altp2m from an
HVM param to a create domctl flag avoids any possible race with the HVM
param changing after it's been evaluated.  Note the param can only be
set by the control domain, and libxl currently sets it at domain
create.  Also altp2m enablement is different from activation, as
activation does happen during runtime of the domain.

Thanks, Roger.

Roger Pau Monne (3):
  xen/x86: account number of foreign mappings in the p2m
  xen/x86: enable altp2m at create domain domctl
  xen/x86: remove foreign mappings from the p2m on teardown

 CHANGELOG.md                      |  1 +
 tools/libs/light/libxl_x86.c      | 43 ++++++++++++-------
 xen/arch/x86/domain.c             | 31 +++++++++++++-
 xen/arch/x86/hvm/hvm.c            | 15 ++++++-
 xen/arch/x86/include/asm/p2m.h    | 32 +++++++++------
 xen/arch/x86/mm/p2m-basic.c       | 17 ++++++++
 xen/arch/x86/mm/p2m.c             | 68 +++++++++++++++++++++++++++++--
 xen/include/public/arch-x86/xen.h | 18 ++++++++
 xen/include/public/hvm/params.h   |  9 +---
 9 files changed, 195 insertions(+), 39 deletions(-)

-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 08 11:27:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 11:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718734.1121222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRr-0004aW-RV; Wed, 08 May 2024 11:26:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718734.1121222; Wed, 08 May 2024 11:26:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4fRr-0004aP-OO; Wed, 08 May 2024 11:26:59 +0000
Received: by outflank-mailman (input) for mailman id 718734;
 Wed, 08 May 2024 11:26:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4fRq-0004Lt-4U
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 11:26:58 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e14a2c93-0d2d-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 13:26:57 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-573137ba8d7so1323641a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 04:26:57 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ds9-20020a0564021cc900b005705bfeeb27sm7483983edb.66.2024.05.08.04.26.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 04:26:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e14a2c93-0d2d-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715167617; x=1715772417; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=m3tygfqqTA1tpknR6/VpOTguLDppE0Q8yESUk/lbUEQ=;
        b=PeDhNHJc/G5PctJ0aQSeKah54LSD8i9qXqHBBRbbKldzm94S4iqdOR1ZiEOA7w6nNM
         K8evlx9UWTCCTzg/M9DMl2vW8HHhY+0Y4tTwk/jWlYhOakUVBzfmTXWHZSGDSIO6wxxi
         AlA5N1yW7/+ncFrIFlt1csNdBNbJRZZxAVZSU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715167617; x=1715772417;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=m3tygfqqTA1tpknR6/VpOTguLDppE0Q8yESUk/lbUEQ=;
        b=im1uBoAIkxQZpcJ/pnTy1hwPZINb98kkJmMlHSJKCynqkffVzEmflO41s/swAJb25b
         YxQPmOthc1B2Za8gbiYUE7cfFY3D6V+x7q3f9mQc2BfLs+77toQlbvawR6++4hivCick
         +rTVDJNq/kRQ5BQtqlh9HPjb1m+Bx/nvyGGciPbJE1WQYjG4XEWGXMDJYODQAJr8d7bd
         JPjz0MOtsYAc3KEHiS8Xt3nCcXYPpljMZQEjIfOhuoT5VHp8rHnH3rec+FzsNmd88ML2
         vDHQ+wepZiycaHzAycurTe0vw1m2Zf42Uz8nMBLRjg+wnQBo1a+50XJcmVSh1UUJxtV9
         GDrw==
X-Gm-Message-State: AOJu0YyGfKt9uJQzO9+BK1eLK1OTHDYgzQtivGEKOciQ3y+ldenHXdLT
	qb+xGecrMHBbyBqFi7PMY/ojC9GFDhkl57Kqw1BkH8bWQf4iwPwJlxGV7/PgcjPdOUBhw1A7eus
	8
X-Google-Smtp-Source: AGHT+IH/1U2J21bD3IY1H/G66eCkgtvsFEgF8vyRrrhwjx+EZKl3CjCkyvOuUWbjDz7v0WRnQtmQjw==
X-Received: by 2002:a05:6402:7d0:b0:572:e81d:6e70 with SMTP id 4fb4d7f45d1cf-57311084911mr4703148a12.15.1715167616786;
        Wed, 08 May 2024 04:26:56 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 v2 1/3] xen/x86: account number of foreign mappings in the p2m
Date: Wed,  8 May 2024 13:23:21 +0200
Message-ID: <20240508112323.38946-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240508112323.38946-1-roger.pau@citrix.com>
References: <20240508112323.38946-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Such information will be needed in order to remove foreign mappings during
teardown for HVM guests.

Right now the introduced counter is not consumed.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Drop max_gfn accounting.
---
 xen/arch/x86/include/asm/p2m.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 111badf89a6e..107b9f260848 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -380,6 +380,9 @@ struct p2m_domain {
         unsigned int flags;
         unsigned long entry_count;
     } ioreq;
+
+    /* Number of foreign mappings. */
+    unsigned long      nr_foreign;
 #endif /* CONFIG_HVM */
 };
 
@@ -1049,6 +1052,8 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         if ( !page_get_owner_and_reference(mfn_to_page(nfn)) )
             return -EBUSY;
 
+        p2m->nr_foreign++;
+
         break;
 
     default:
@@ -1069,6 +1074,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
             return -EINVAL;
         }
         put_page(mfn_to_page(ofn));
+        p2m->nr_foreign--;
         break;
 
     default:
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:09:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:09:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718771.1121252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4g71-00057b-2P; Wed, 08 May 2024 12:09:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718771.1121252; Wed, 08 May 2024 12:09:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4g70-00057U-Us; Wed, 08 May 2024 12:09:30 +0000
Received: by outflank-mailman (input) for mailman id 718771;
 Wed, 08 May 2024 12:09:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ompu=ML=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s4g6z-00057L-BY
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:09:29 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ceb2459d-0d33-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 14:09:24 +0200 (CEST)
Received: from SJ0PR03CA0021.namprd03.prod.outlook.com (2603:10b6:a03:33a::26)
 by MW6PR12MB8960.namprd12.prod.outlook.com (2603:10b6:303:23e::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Wed, 8 May
 2024 12:09:20 +0000
Received: from SJ1PEPF00001CE8.namprd03.prod.outlook.com
 (2603:10b6:a03:33a:cafe::bd) by SJ0PR03CA0021.outlook.office365.com
 (2603:10b6:a03:33a::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Wed, 8 May 2024 12:09:19 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ1PEPF00001CE8.mail.protection.outlook.com (10.167.242.24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Wed, 8 May 2024 12:09:19 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 8 May
 2024 07:09:18 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 8 May 2024 07:09:17 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ceb2459d-0d33-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BrsNPWbMss67TDAn1mJfgMgD34hAhSZGfWAmHTakd6xxFlLyTggAVzPjJ+l+INXgYM6ut8ZbDBY5R+m1thon8ShZZQhQ2BOnFS9ofRrhxhTad7eb3+Af0xYYa/qXi6knyr+ByqySSA1ZDlDIL9Wmc7qAVrYYl9uWC47oFHKu1wGg7lVZARx8dDudmZ3GFfvlij2jVoE6dtD7b7mGNtE616/gglWAaiidyR+/3eDBBGp6NWiLN+S+olMbf7Bv36MsXh/0gnAYs0dz9qj/x03DHX91NXFOdyeZ1Do5FF/GLZOtXDYn/yKEVn+vR1hGF9E9u5vlgZzlVGFZ+Z9fnqyI6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JQ97iZJtt9Thg0nMoKUUPhZ0X3mcr4cpwmkH/PCWogk=;
 b=GEn+gd+SeLZdO3m87bo8XUlFM+1/C3Mqt4eo3T3DB9/XSTO5O4X05mHriQDyAb7MDqfL4VsvN1cjrHKi1M7GQzmeho+HxkS9vBC32K3gw5gpRlWIMNdLeL75L/cgMIpVWEal0E7qjiRryr1WZZYKitpRAZNzxsHYqLB+pKVWgD/1KU8NRu7hWeF1XxB7NfM7hLv0Sa4al39zSY1lVP4TTm2Y/K+vdR9h529U27pV/Z4YgbmRExR12JQTOcpUnHb9NpebTCChQ3J4KJkOH8kIsB9v3Xm8Dr+SsNQ7+MwuUGw7V1n14SEEzmOnmzYoC6+QC6m7idYN1QULuXVTYLfxgw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JQ97iZJtt9Thg0nMoKUUPhZ0X3mcr4cpwmkH/PCWogk=;
 b=es2zWZqUEC79XP0BIrAPUKowIOtx3iJGxSGGRcyZjJqQDKln3I5clD2EgE4RPi7UpgPLsgXv06dehZQ+5xcmxPV54JJRfv+G3Yhsr6/bmFMqtk2dY8eGoYfs1+h5evAhn1dyeqwwuJf8LH1V2LxPe9xpxG387iFwVpaihvZYudw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <59b02ed2-e543-4bfd-946e-6a82919c582d@amd.com>
Date: Wed, 8 May 2024 14:09:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/7] xen/arm: Parse xen,shared-mem when host phys address
 is not provided
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-5-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-5-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE8:EE_|MW6PR12MB8960:EE_
X-MS-Office365-Filtering-Correlation-Id: 2ef0010e-17f0-47e0-bad3-08dc6f57b078
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?KzhaVEljS0VZRjN6dnkyVWlkMTVLN2M1dWVKNnUwcnN3SExYTmUyVHloWXVk?=
 =?utf-8?B?cnc5ekVRZDZjSHRvckRoaVNJOVg5M2puRHN5d2k0THdacDNWYjBYR0xHN3lq?=
 =?utf-8?B?K0ZENnY3bDVmNHQrdld5VXVHcUdKSER4TmZaZHk1R3kvK2xiYzZGVWxWWkR3?=
 =?utf-8?B?RnErcEZpaWo4ell6TkQ5TWh3OEppcFZ4VHRXRjkwN085QlE0d2VCd1RrKzh4?=
 =?utf-8?B?QjU0eStxU2crMitBRXUxOHRaMmVvMkovYk94OVZMVXRiaDg4OFY5VjBFTGVw?=
 =?utf-8?B?am1vSlcxTkZBVmM3MDJKQ3dQVDNYME5aS29pK2FnSmluOTZyN0xUQXVOMy9Z?=
 =?utf-8?B?bzUzRXZITmNLOXo5NThzNXBUVXRSbStYbXJScDd1R1Ftd20zRGloYmtZRGNt?=
 =?utf-8?B?TWVwRTNicmNsWk5ZRGFMRDVZTlkwVmQxYUlVcHowTkFibmJlV3FEd0RKZUtN?=
 =?utf-8?B?c2hYV3JpbVNha2RSdjhFT2NQRVMvc3hkMWJlMnJVT0NEa2hvRzJNdnN5Zkli?=
 =?utf-8?B?TWZCWkVtQXlXaUNWaDBFNWVGQlc5TC9aK1hIdlB6WUFJZjBiK2F4QndjK1lm?=
 =?utf-8?B?Ti80OG91cVRGOUdvNjFTUk1MSzhmV3pnYi8vaVhyTzBhcFQxYWxiMmNrRGpI?=
 =?utf-8?B?WFhWZkhnaFN3MWNIY0hqeUtsUUg2a3MwRlJmeDhFYjV2RzVQUnByK0dhT2k2?=
 =?utf-8?B?U0VIcXdoTDJidElHVzA3cmFzQzZKVmU3cExtYmRFTlAzQ1FYTHRGNEE4Yis4?=
 =?utf-8?B?VS9qK25iUytuaHlqWm5nQVl1UEhYZWFTOTVZUmU5ZFp0aU5KTmMyVzJJanVG?=
 =?utf-8?B?ZzM0RXVLTjdUR01wbkZNLzFBN0RaRlFoWnczTzRLWFFxQmhEc0J5MXpiMGYy?=
 =?utf-8?B?Q2ZmZGZ4SFZ0SVJwY3RhV0JQT1EyL3F1ZFpmejZ0a3U4VWl0MG16LzZyR1ZU?=
 =?utf-8?B?QWJRY0lNejhkYnJLQUw5c2JTdTlTZXl1VWRQMHVzejBzTGd4ZGhiMDQraVAw?=
 =?utf-8?B?NVU5UHNncTl4b1Nvdy9jVFZReEZsSTdXL0ptaEZJazkyOGJ4R2d0b29lQ1FL?=
 =?utf-8?B?SHZJNmdwMmtlNTBRbVNYeVRqdlVabnYrdEQ5RmtKTjN5ZnI3SDBlamJiMFY3?=
 =?utf-8?B?dUlPZ1FZb09STDA4TC9DYkVucnRRSHI1ekw0VW1aK0lPZlZjRWdBVW9hZDFY?=
 =?utf-8?B?cFpUc1hTWGFSbXBVVDFPNitqRk95OEh5MFFmbHB4aHd4cExTOXI3K0xHUUo2?=
 =?utf-8?B?LytkYUZHLzJRc1NpNllTd3ZHSXVGN0Y1eDN0TVRiZHk3OG5DWXFRMVAxRWhL?=
 =?utf-8?B?cEdaS1MvSkkwMXdkZzRKTFhwSDZXVGlxNTRadno2L3FRc3NTaC9uK1ZPQjZO?=
 =?utf-8?B?K2d6N09ETTBFcTdlUkZyRmdza2RCSUJWUXcrcUJtQ1VOY3M2UnJpem8zL3F5?=
 =?utf-8?B?NWlOczE0d3d0WXBkOTdVU01RNGovZ0QwcVFnaERUVEpsemRzTlFEcUpwRUlm?=
 =?utf-8?B?YzkrWlpVQkttelYyZVZPOHJxY1Z0aWFGRGFyWFVXRVlHbHpRS2JmQ0R5cFk3?=
 =?utf-8?B?aHY0UUlKRlg5QWZ4UXVQcllxTCtJQWgzWU41eWMzS3U2bVdRVVlLaUNLb2hh?=
 =?utf-8?B?SitVaS9NaWM3bUpBSlZHL3ZXV05HYzNUUlExTyt3SFdHUllqL2xGcmlJQkh1?=
 =?utf-8?B?VTdGcXRjWmprMHR1Vlc5Qk94MjNlMGNRUkdlTC9aWEhhSmVlL0RTblljK1Nj?=
 =?utf-8?B?LzlLOWozeVgvS3lWaGw0NzZHWGIzZ0x6Y21GcWZHMUJRRmNPa1hROXlVV1FY?=
 =?utf-8?B?TVRTeDU1VEpMc2JZd2h3K0EvYXpYUnpWdjM4dEUrbXdFNUQzSjUwYVd6Tjhz?=
 =?utf-8?Q?aERGeP9AV1QAE?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 12:09:19.2057
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ef0010e-17f0-47e0-bad3-08dc6f57b078
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE8.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8960

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> Handle the parsing of the 'xen,shared-mem' property when the host physical
> address is not provided, this commit is introducing the logic to parse it,
> but the functionality is still not implemented and will be part of future
> commits.
> 
> Rework the logic inside process_shm_node to check the shm_id before doing
> the other checks, because it ease the logic itself, add more comment on
> the logic.
> Now when the host physical address is not provided, the value
> INVALID_PADDR is chosen to signal this condition and it is stored as
> start of the bank, due to that change also early_print_info_shmem and
> init_sharedmem_pages are changed, to don't handle banks with start equal
> to INVALID_PADDR.
> 
> Another change is done inside meminfo_overlap_check, to skip banks that
> are starting with the start address INVALID_PADDR, that function is used
> to check banks from reserved memory and ACPI and it's unlikely for these
also from shmem

> bank to have the start address as INVALID_PADDR. The change holds
> because of this consideration.
On arm64 and LPAE arm32 we don't have this problem. In theory we could have a bank
starting at INVALID_PADDR if PA range was 32bit but as the comment above the function states,
wrapping around is not handled. You might want to use it as a justification to be clear.

> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  xen/arch/arm/setup.c        |   3 +-
>  xen/arch/arm/static-shmem.c | 129 +++++++++++++++++++++++++-----------
>  2 files changed, 93 insertions(+), 39 deletions(-)
> 
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index d242674381d4..f15d40a85a5f 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -297,7 +297,8 @@ static bool __init meminfo_overlap_check(const struct membanks *mem,
>          bank_start = mem->bank[i].start;
>          bank_end = bank_start + mem->bank[i].size;
> 
> -        if ( region_end <= bank_start || region_start >= bank_end )
> +        if ( INVALID_PADDR == bank_start || region_end <= bank_start ||
> +             region_start >= bank_end )
>              continue;
>          else
>          {
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 24e40495a481..1c03bb7f1882 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -264,6 +264,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          pbase = boot_shm_bank->start;
>          psize = boot_shm_bank->size;
> 
> +        if ( INVALID_PADDR == pbase )
> +        {
> +            printk("%pd: host physical address must be chosen by users at the moment.", d);
The dot at the end is not needed.

> +            return -EINVAL;
> +        }
> +
>          /*
>           * xen,shared-mem = <pbase, gbase, size>;
>           * TODO: pbase is optional.
> @@ -382,7 +388,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>  {
>      const struct fdt_property *prop, *prop_id, *prop_role;
>      const __be32 *cell;
> -    paddr_t paddr, gaddr, size, end;
> +    paddr_t paddr = INVALID_PADDR;
> +    paddr_t gaddr, size, end;
>      struct membanks *mem = bootinfo_get_shmem();
>      struct shmem_membank_extra *shmem_extra = bootinfo_get_shmem_extra();
>      unsigned int i;
> @@ -437,24 +444,37 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>      if ( !prop )
>          return -ENOENT;
> 
> +    cell = (const __be32 *)prop->data;
>      if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) )
>      {
> -        if ( len == dt_cells_to_size(size_cells + address_cells) )
> -            printk("fdt: host physical address must be chosen by users at the moment.\n");
> -
> -        printk("fdt: invalid `xen,shared-mem` property.\n");
> -        return -EINVAL;
> +        if ( len == dt_cells_to_size(address_cells + size_cells) )
> +            device_tree_get_reg(&cell, address_cells, size_cells, &gaddr,
> +                                &size);
> +        else
> +        {
> +            printk("fdt: invalid `xen,shared-mem` property.\n");
> +            return -EINVAL;
> +        }
>      }
> +    else
> +    {
> +        device_tree_get_reg(&cell, address_cells, address_cells, &paddr,
> +                            &gaddr);
> +        size = dt_next_cell(size_cells, &cell);
> 
> -    cell = (const __be32 *)prop->data;
> -    device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
> -    size = dt_next_cell(size_cells, &cell);
> +        if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
> +        {
> +            printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
> +                paddr);
> +            return -EINVAL;
> +        }
> 
> -    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
> -    {
> -        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
> -               paddr);
> -        return -EINVAL;
> +        end = paddr + size;
> +        if ( end <= paddr )
> +        {
> +            printk("fdt: static shared memory region %s overflow\n", shm_id);
> +            return -EINVAL;
> +        }
>      }
> 
>      if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
> @@ -476,41 +496,69 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>          return -EINVAL;
>      }
> 
> -    end = paddr + size;
> -    if ( end <= paddr )
> -    {
> -        printk("fdt: static shared memory region %s overflow\n", shm_id);
> -        return -EINVAL;
> -    }
> -
>      for ( i = 0; i < mem->nr_banks; i++ )
>      {
>          /*
>           * Meet the following check:
> +         * when host address is provided:
- when would read better

>           * 1) The shm ID matches and the region exactly match
>           * 2) The shm ID doesn't match and the region doesn't overlap
>           * with an existing one
> +         * when host address is not provided:
> +         * 1) The shm ID matches and the region size exactly match
>           */
> -        if ( paddr == mem->bank[i].start && size == mem->bank[i].size )
> +        bool paddr_assigned = INVALID_PADDR == paddr;
parenthesis around INVALID_PADDR == paddr

> +        bool shm_id_match = strncmp(shm_id, shmem_extra[i].shm_id,
> +                                    MAX_SHM_ID_LENGTH) == 0;
why not if ( strncmp... given no other use of this variable other than the one below?

> +        if ( shm_id_match )
>          {
> -            if ( strncmp(shm_id, shmem_extra[i].shm_id,
> -                         MAX_SHM_ID_LENGTH) == 0  )
> +            /*
> +             * Regions have same shm_id (cases):
> +             * 1) physical host address is supplied:
> +             *    - OK:   paddr is equal and size is equal (same region)
> +             *    - Fail: paddr doesn't match or size doesn't match (there
> +             *            cannot exists two shmem regions with same shm_id)
> +             * 2) physical host address is NOT supplied:
> +             *    - OK:   size is equal (same region)
> +             *    - Fail: size is not equal (same shm_id must identify only one
> +             *            region, there can't be two different regions with same
> +             *            shm_id)
> +             */
> +            bool start_match = paddr_assigned ? (paddr == mem->bank[i].start) :
> +                                                true;
> +
> +            if ( start_match && size == mem->bank[i].size )
>                  break;
>              else
>              {
> -                printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n",
> +                printk("fdt: different shared memory region could not share the same shm ID %s\n",
>                         shm_id);
>                  return -EINVAL;
>              }
>          }
> -        else if ( strncmp(shm_id, shmem_extra[i].shm_id,
> -                          MAX_SHM_ID_LENGTH) != 0 )
> -            continue;
>          else
>          {
There is no need for this else and entire block given that the block within if either calls break or return

> -            printk("fdt: different shared memory region could not share the same shm ID %s\n",
> -                   shm_id);
> -            return -EINVAL;
> +            /*
> +             * Regions have different shm_id (cases):
> +             * 1) physical host address is supplied:
> +             *    - OK:   paddr different, or size different (case where paddr
> +             *            is equal but psize is different are wrong, but they
> +             *            are handled later when checking for overlapping)
> +             *    - Fail: paddr equal and size equal (the same region can't be
> +             *            identified with different shm_id)
> +             * 2) physical host address is NOT supplied:
> +             *    - OK:   Both have different shm_id so even with same size they
> +             *            can exists
> +             */
> +            if ( !paddr_assigned || paddr != mem->bank[i].start ||
> +                 size != mem->bank[i].size )
> +                continue;
> +            else
> +            {
> +                printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n",
> +                       shm_id);
> +                return -EINVAL;
> +            }
>          }
>      }
> 
> @@ -518,7 +566,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>      {
>          if (i < mem->max_banks)
>          {
> -            if ( check_reserved_regions_overlap(paddr, size) )
> +            if ( (paddr != INVALID_PADDR) &&
> +                 check_reserved_regions_overlap(paddr, size) )
>                  return -EINVAL;
> 
>              /* Static shared memory shall be reserved from any other use. */
> @@ -588,13 +637,16 @@ void __init early_print_info_shmem(void)
>  {
>      const struct membanks *shmem = bootinfo_get_shmem();
>      unsigned int bank;
> +    unsigned int printed = 0;
> 
>      for ( bank = 0; bank < shmem->nr_banks; bank++ )
> -    {
> -        printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", bank,
> -               shmem->bank[bank].start,
> -               shmem->bank[bank].start + shmem->bank[bank].size - 1);
> -    }
> +        if ( shmem->bank[bank].start != INVALID_PADDR )
> +        {
> +            printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", printed,
> +                shmem->bank[bank].start,
> +                shmem->bank[bank].start + shmem->bank[bank].size - 1);
> +            printed++;
NIT: you could initialize and increment it as part of the for loop

> +        }
>  }
> 
>  void __init init_sharedmem_pages(void)
> @@ -603,7 +655,8 @@ void __init init_sharedmem_pages(void)
>      unsigned int bank;
> 
>      for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
> -        init_staticmem_bank(&shmem->bank[bank]);
> +        if ( shmem->bank[bank].start != INVALID_PADDR )
> +            init_staticmem_bank(&shmem->bank[bank]);
>  }
> 
>  int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
> --
> 2.34.1
> 

~Michal



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718782.1121321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaD-0003pQ-6U; Wed, 08 May 2024 12:39:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718782.1121321; Wed, 08 May 2024 12:39:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaC-0003o2-Tp; Wed, 08 May 2024 12:39:40 +0000
Received: by outflank-mailman (input) for mailman id 718782;
 Wed, 08 May 2024 12:39:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4gaB-0002bO-RN
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:39 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 084d133c-0d38-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 14:39:38 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59b097b202so851533466b.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:38 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 084d133c-0d38-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171977; x=1715776777; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=h0aXIfx/yfl8UuSfpAwTUvnGU7Teh/O7MEVINpnauu4=;
        b=hy++ymaiEXPVnLDqM4/WZ9q9u6ujYGhksxMMbxSuutD31RCbpI+RDAX3Q7UhQr26eR
         jz+T/mMiclLsfEp6Nfp9EYnDCS2vZjw0OZTXYXB8Adny3QxVEXr94wcVdkswRyEXevOa
         8ShZHR+sK+VbrQiI0N/a65oktZOpbR335J2z0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171977; x=1715776777;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=h0aXIfx/yfl8UuSfpAwTUvnGU7Teh/O7MEVINpnauu4=;
        b=xIeeh5DbkvtGDoDHwRyrwYgGDiCqR8YBk+vWo3lz8ie4XnLX1jpt301q2ZRYuy1vdw
         FiMuzos3WFpd/Z0V9/bFqLfwyuBbrVSEbZyz+GjhGe2lw9qZfsunwb9qaLSZNfU0UXP0
         OQMdwn9Tfn5zGDhGl4cUZA0HlHhO0yH4ydah3Pfryk5pbD/HpHnCXUEKXzT/Sdt2AjqZ
         IW4tBXxecnLKGUqaFPkrhMiN45CzKnUJJzPQAlpikz+p/Gfruod50pG2ra9Sz6o6XxQF
         2LQOGXNFcgtMtlzDmlgqm4GG5nJ9d03GtNK2BhC68lTOklX8C5JQpD64z2/iH2d7LZ++
         9Igw==
X-Gm-Message-State: AOJu0YzQqnxnVooQh7M367Vxg21ru6HHSTy+5HErgzYe/0OwuLm3gfnT
	+COfbgyjsy7QapxNzwC5JxHBSb9yvFTgAOwo9vTlYJKp4R5OQIGKMrDvNHoQsAoO4SiYKm1wmVL
	k
X-Google-Smtp-Source: AGHT+IGPNenRVYdkse7eucd1yFdgEvIk9xBV/nUjri4SqCLroW46G1BK17f6LxgMI3Mu+6BsAJeyxQ==
X-Received: by 2002:a17:906:b7cb:b0:a59:ba18:2fb9 with SMTP id a640c23a62f3a-a59fb9211d3mr151555366b.12.1715171977453;
        Wed, 08 May 2024 05:39:37 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v2 7/8] xen/x86: Derive topologically correct x2APIC IDs from the policy
Date: Wed,  8 May 2024 13:39:26 +0100
Message-Id: <87a2a4589e330472b7260ff6ab513744596a4488.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Implements the helper for mapping vcpu_id to x2apic_id given a valid
topology in a policy. The algo is written with the intention of extending
it to leaves 0x1f and e26 in the future.

Toolstack doesn't set leaf 0xb and the HVM default policy has it cleared,
so the leaf is not implemented. In that case, the new helper just returns
the legacy mapping.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * const-ify the test definitions
  * Cosmetic changes (newline + parameter name in prototype)
---
 tools/tests/cpu-policy/test-cpu-policy.c | 63 ++++++++++++++++++++
 xen/include/xen/lib/x86/cpu-policy.h     |  2 +
 xen/lib/x86/policy.c                     | 73 ++++++++++++++++++++++--
 3 files changed, 133 insertions(+), 5 deletions(-)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
index 0ba8c418b1b3..82a6aeb23317 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -776,6 +776,68 @@ static void test_topo_from_parts(void)
     }
 }
 
+static void test_x2apic_id_from_vcpu_id_success(void)
+{
+    static const struct test {
+        unsigned int vcpu_id;
+        unsigned int threads_per_core;
+        unsigned int cores_per_pkg;
+        uint32_t x2apic_id;
+        uint8_t x86_vendor;
+    } tests[] = {
+        {
+            .vcpu_id = 3, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 1 << 2,
+        },
+        {
+            .vcpu_id = 6, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 2 << 2,
+        },
+        {
+            .vcpu_id = 24, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 1 << 5,
+        },
+        {
+            .vcpu_id = 35, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = (35 % 3) | (((35 / 3) % 8)  << 2) | ((35 / 24) << 5),
+        },
+        {
+            .vcpu_id = 96, .threads_per_core = 7, .cores_per_pkg = 3,
+            .x2apic_id = (96 % 7) | (((96 / 7) % 3)  << 3) | ((96 / 21) << 5),
+        },
+    };
+
+    const uint8_t vendors[] = {
+        X86_VENDOR_INTEL,
+        X86_VENDOR_AMD,
+        X86_VENDOR_CENTAUR,
+        X86_VENDOR_SHANGHAI,
+        X86_VENDOR_HYGON,
+    };
+
+    printf("Testing x2apic id from vcpu id success:\n");
+
+    /* Perform the test run on every vendor we know about */
+    for ( size_t i = 0; i < ARRAY_SIZE(vendors); ++i )
+    {
+        struct cpu_policy policy = { .x86_vendor = vendors[i] };
+        for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+        {
+            const struct test *t = &tests[i];
+            uint32_t x2apic_id;
+            int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);
+
+            x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);
+            if ( rc || x2apic_id != t->x2apic_id )
+                fail("FAIL[%d] - '%s cpu%u %u t/c %u c/p'. bad x2apic_id: expected=%u actual=%u\n",
+                     rc,
+                     x86_cpuid_vendor_to_str(policy.x86_vendor),
+                     t->vcpu_id, t->threads_per_core, t->cores_per_pkg,
+                     t->x2apic_id, x2apic_id);
+        }
+    }
+}
+
 int main(int argc, char **argv)
 {
     printf("CPU Policy unit tests\n");
@@ -794,6 +856,7 @@ int main(int argc, char **argv)
     test_is_compatible_failure();
 
     test_topo_from_parts();
+    test_x2apic_id_from_vcpu_id_success();
 
     if ( nr_failures )
         printf("Done: %u failures\n", nr_failures);
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index f5df18e9f77c..2cbc2726a861 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -545,6 +545,8 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
 /**
  * Calculates the x2APIC ID of a vCPU given a CPU policy
  *
+ * If the policy lacks leaf 0xb falls back to legacy mapping of apic_id=cpu*2
+ *
  * @param p          CPU policy of the domain.
  * @param id         vCPU ID of the vCPU.
  * @returns x2APIC ID of the vCPU.
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index d033ee5398dd..e498e32f8fd7 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -2,15 +2,78 @@
 
 #include <xen/lib/x86/cpu-policy.h>
 
+static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
+{
+    /*
+     * `nr_logical` reported by Intel is the number of THREADS contained in
+     * the next topological scope. For example, assuming a system with 2
+     * threads/core and 3 cores/module in a fully symmetric topology,
+     * `nr_logical` at the core level will report 6. Because it's reporting
+     * the number of threads in a module.
+     *
+     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
+     * level (cores/complex, etc) so we can return it as-is.
+     */
+    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
+        return p->topo.subleaf[lvl].nr_logical;
+
+    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;
+}
+
 uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
 {
+    uint32_t shift = 0, x2apic_id = 0;
+
+    /* In the absence of topology leaves, fallback to traditional mapping */
+    if ( !p->topo.subleaf[0].type )
+        return id * 2;
+
     /*
-     * TODO: Derive x2APIC ID from the topology information inside `p`
-     *       rather than from vCPU ID. This bodge is a temporary measure
-     *       until all infra is in place to retrieve or derive the initial
-     *       x2APIC ID from migrated domains.
+     * `id` means different things at different points of the algo
+     *
+     * At lvl=0: global thread_id (same as vcpu_id)
+     * At lvl=1: global core_id
+     * At lvl=2: global socket_id (actually complex_id in AMD, module_id
+     *                             in Intel, but the name is inconsequential)
+     *
+     *                 +--+
+     *            ____ |#0| ______           <= 1 socket
+     *           /     +--+       \+--+
+     *       __#0__              __|#1|__    <= 2 cores/socket
+     *      /   |  \        +--+/  +-|+  \
+     *    #0   #1   #2      |#3|    #4    #5 <= 3 threads/core
+     *                      +--+
+     *
+     * ... and so on. Global in this context means that it's a unique
+     * identifier for the whole topology, and not relative to the level
+     * it's in. For example, in the diagram shown above, we're looking at
+     * thread #3 in the global sense, though it's #0 within its core.
+     *
+     * Note that dividing a global thread_id by the number of threads per
+     * core returns the global core id that contains it. e.g: 0, 1 or 2
+     * divided by 3 returns core_id=0. 3, 4 or 5 divided by 3 returns core
+     * 1, and so on. An analogous argument holds for higher levels. This is
+     * the property we exploit to derive x2apic_id from vcpu_id.
+     *
+     * NOTE: `topo` is currently derived from leaf 0xb, which is bound to
+     * two levels, but once we track leaves 0x1f (or e26) there will be a
+     * few more. The algorithm is written to cope with that case.
      */
-    return vcpu_id * 2;
+    for ( uint32_t i = 0; i < ARRAY_SIZE(p->topo.raw); i++ )
+    {
+        uint32_t nr_parts;
+
+        if ( !p->topo.subleaf[i].type )
+            /* sentinel subleaf */
+            break;
+
+        nr_parts = parts_per_higher_scoped_level(p, i);
+        x2apic_id |= (id % nr_parts) << shift;
+        id /= nr_parts;
+        shift = p->topo.subleaf[i].id_shift;
+    }
+
+    return (id << shift) | x2apic_id;
 }
 
 static unsigned int order(unsigned int n)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718780.1121303 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003TH-VS; Wed, 08 May 2024 12:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718780.1121303; Wed, 08 May 2024 12:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003SD-Ov; Wed, 08 May 2024 12:39:39 +0000
Received: by outflank-mailman (input) for mailman id 718780;
 Wed, 08 May 2024 12:39:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4gaA-0002b0-Jf
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:38 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0750ef9e-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:36 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59c04839caso977235566b.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:36 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0750ef9e-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171976; x=1715776776; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tjb9wDk03eu5cy1/rNtEigMH1xNhBsbk5Mon31Q8thQ=;
        b=E3lIERaoiBhvhymIlJVY5fSiUamqJXXkB5MeK37T9+Lx48NFtdYICzQ5oaGslclVBg
         0V1hW36VSnVNJuY1Nl0q+gh6slUTMmURmdyMo9u6bEbMVyMJm6Ff56l/BTNhXf0sTJMe
         t7TUGtNZNupX5YFJF5kwurW9uUFVlv4ExCGI4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171976; x=1715776776;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Tjb9wDk03eu5cy1/rNtEigMH1xNhBsbk5Mon31Q8thQ=;
        b=v34JS0yPekgUZVYkb2ZJaQ3JsRs6ErcrNXdTRwUlyrBzA19wlg569afnmNxhcMwdm5
         WZ/ioguIFu8dLz/2/4CVaQGCntF6IyR7UrUG2znToYZVmAnG+90yjC69SO4qW+RqBCGJ
         KqN8yQ2Ds+IpPeswnhImumqFZe3qlffHkdBw/yz3ie6zrzTonD5xoqdAeob74k/9JkBp
         CiQRGVnfGe0WRPAr4n+3X3mCm2qHFrLZfjOVNHKifUzASV0zHkOzHp9G698Biy2nR3qO
         QlzVULXgKXJaHOd2pYQelfjGYn4hZYcoIq6vzxyvx6k857EKaSSm6ZUoMVINCoy2yPGi
         qjGA==
X-Gm-Message-State: AOJu0YzsjjURn4l0VRqukhY+zkrG/asJAZFMxKNFE5MAaoqvCgm98vBK
	jHRkkFH1637MSEVzlQt10Amj9ZMlTUrNZ2jvZWxXev1LUTHLxTBrBkdyfBhlIG6SRx1kyGnZhCs
	b
X-Google-Smtp-Source: AGHT+IF9b75aKMImWVHAuQOHLUdGMkpVk6WnMpMg8aFYo7LFMr0V61yRnpql50wsVtLQWkTMQRkUOQ==
X-Received: by 2002:a17:907:3e13:b0:a59:ca33:6845 with SMTP id a640c23a62f3a-a59fb9d2c7emr178006266b.75.1715171975798;
        Wed, 08 May 2024 05:39:35 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the APs themselves
Date: Wed,  8 May 2024 13:39:24 +0100
Message-Id: <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
from hvmloader.

While at this also remove ap_callin, as writing the APIC ID may serve the same
purpose.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * New patch. Replaces adding cpu policy to hvmloader in v1.
---
 tools/firmware/hvmloader/config.h    |  6 ++++-
 tools/firmware/hvmloader/hvmloader.c |  4 +--
 tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
 tools/firmware/hvmloader/util.h      |  5 ++++
 xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
 5 files changed, 47 insertions(+), 9 deletions(-)

diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
index c82adf6dc508..edf6fa9c908c 100644
--- a/tools/firmware/hvmloader/config.h
+++ b/tools/firmware/hvmloader/config.h
@@ -4,6 +4,8 @@
 #include <stdint.h>
 #include <stdbool.h>
 
+#include <xen/hvm/hvm_info_table.h>
+
 enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
 extern enum virtual_vga virtual_vga;
 
@@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
 
 #define IOAPIC_ID           0x01
 
+extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
+
 #define LAPIC_BASE_ADDRESS  0xfee00000
-#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
+#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
 
 #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
 #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
index c58841e5b556..1eba92229925 100644
--- a/tools/firmware/hvmloader/hvmloader.c
+++ b/tools/firmware/hvmloader/hvmloader.c
@@ -342,11 +342,11 @@ int main(void)
 
     printf("CPU speed is %u MHz\n", get_cpu_mhz());
 
+    smp_initialise();
+
     apic_setup();
     pci_setup();
 
-    smp_initialise();
-
     perform_tests();
 
     if ( bios->bios_info_setup )
diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
index a668f15d7e1f..4d75f239c2f5 100644
--- a/tools/firmware/hvmloader/smp.c
+++ b/tools/firmware/hvmloader/smp.c
@@ -29,7 +29,34 @@
 
 #include <xen/vcpu.h>
 
-static int ap_callin, ap_cpuid;
+static int ap_cpuid;
+
+/**
+ * Lookup table of x2APIC IDs.
+ *
+ * Each entry is populated its respective CPU as they come online. This is required
+ * for generating the MADT with minimal assumptions about ID relationships.
+ */
+uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
+
+static uint32_t read_apic_id(void)
+{
+    uint32_t apic_id;
+
+    cpuid(1, NULL, &apic_id, NULL, NULL);
+    apic_id >>= 24;
+
+    /*
+     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
+     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
+     * but only if the x2APIC feature is present. If there are that many CPUs
+     * it's guaranteed to be there so we can avoid checking for it specifically
+     */
+    if ( apic_id == 255 )
+        cpuid(0xb, NULL, NULL, NULL, &apic_id);
+
+    return apic_id;
+}
 
 static void ap_start(void)
 {
@@ -37,12 +64,12 @@ static void ap_start(void)
     cacheattr_init();
     printf("done.\n");
 
+    wmb();
+    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();
+
     if ( !ap_cpuid )
         return;
 
-    wmb();
-    ap_callin = 1;
-
     while ( 1 )
         asm volatile ( "hlt" );
 }
@@ -86,10 +113,11 @@ static void boot_cpu(unsigned int cpu)
         BUG();
 
     /*
-     * Wait for the secondary processor to complete initialisation.
+     * Wait for the secondary processor to complete initialisation,
+     * which is signaled by its x2APIC ID being writted to the LUT.
      * Do not touch shared resources meanwhile.
      */
-    while ( !ap_callin )
+    while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) )
         cpu_relax();
 
     /* Take the secondary processor offline. */
diff --git a/tools/firmware/hvmloader/util.h b/tools/firmware/hvmloader/util.h
index 14078bde1e30..51e9003bc615 100644
--- a/tools/firmware/hvmloader/util.h
+++ b/tools/firmware/hvmloader/util.h
@@ -23,6 +23,11 @@ enum {
 #define __STR(...) #__VA_ARGS__
 #define STR(...) __STR(__VA_ARGS__)
 
+#define __ACCESS_ONCE(x) ({                             \
+            (void)(typeof(x))0; /* Scalar typecheck. */ \
+            (volatile typeof(x) *)&(x); })
+#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
+
 /* GDT selector values. */
 #define SEL_CODE16          0x0008
 #define SEL_DATA16          0x0010
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index 84911f3ebcb4..6c005f0b0b38 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -16,6 +16,7 @@
 #include <asm/current.h>
 #include <asm/x86_emulate.h>
 #include <asm/hvm/asid.h>
+#include <asm/hvm/vlapic.h>
 
 struct pirq; /* needed by pi_update_irte */
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718775.1121261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002bm-AA; Wed, 08 May 2024 12:39:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718775.1121261; Wed, 08 May 2024 12:39:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002bf-6y; Wed, 08 May 2024 12:39:37 +0000
Received: by outflank-mailman (input) for mailman id 718775;
 Wed, 08 May 2024 12:39:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4ga7-0002b0-VI
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:35 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0627193c-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:34 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59b097b202so851510766b.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:34 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0627193c-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171974; x=1715776774; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=W4Y4JBEHdpXX8zg2matRsfDF2skYhfYeN6rMH7JX82k=;
        b=HOFpu4lL7HXEZ5VfTG1TqhuWGlqhdBfaZLuzFWNW0hhUkxDJsQ+QjdqZuEKqiy649V
         6JaB0vhAK7GmSxAuqxo8KHfcIFeOU93AwdeudSB3kgUbxlEjy3yzhYmvdBgKWgbm8b2d
         Bj7Z/N2zHm1WgjPM5yUhPLrMhazAHdczpOxDw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171974; x=1715776774;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=W4Y4JBEHdpXX8zg2matRsfDF2skYhfYeN6rMH7JX82k=;
        b=sDOBvp9KZZWtRw3ACSRby5puOjok6fRERHV39e3a2oGcyxno5MEJ3Rlng2AH7j77MP
         MfF7uuLRAzvDo+Hf5lcCyZObjB5UhLgGxfNr32C+bapzUr5XRIBFmMy6ATIZtQ4EgO0e
         mPkvkCRuTXqHjm8F9aP2AmGmAiv6yqPnRi1Uj7K4Tfj9u8t+H+w+0IQLeJI9tn9ve86U
         C30xkKpA6+Hj2UXBQZOEdHAvegpbFWECJgyjj8n0p+Akxvwaj8d1Wqo6ITWryGz3mYec
         qY58PjhxRKVQ2awq88Bqbzxu7MKzAtQRX3JwN56clM2JyISKwJ+RkWK0bFH/TDftHXOC
         0nGQ==
X-Gm-Message-State: AOJu0YwYfc5lsv4VS0yB9xuf2N78bkGYllnG/f0CQQrp3RuEfVpiCFWg
	bk3AeQhKIm3D25drf+gSgslfY767lfVKVyH3+reXi+4rkIWxfG+a1Fy5FXr6kIcbUwCEeCWv4yt
	2
X-Google-Smtp-Source: AGHT+IFMTaH+tzF18MqnLL27Qd4OMZTc17yFE7oZP7EcHVUHMtbeJ5YuFFtK2PJD1WxS08Vm5LJymQ==
X-Received: by 2002:a17:906:f818:b0:a59:be8a:bd71 with SMTP id a640c23a62f3a-a59fb920ea5mr151770266b.16.1715171971178;
        Wed, 08 May 2024 05:39:31 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 0/8] x86: Expose consistent topology to guests
Date: Wed,  8 May 2024 13:39:19 +0100
Message-Id: <cover.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

v1 -> v2:

  * v1/patch 4 replaced by a different strategy (See patches 4 and 5 in v2):
      * Have hvmloader populate MADT with the real APIC IDs as read by the APs
        themselves rather than giving it knowledge on how to derive them.
  * Removed patches 2 and 3 in v1, as no longer relevant.
  * Split v1/patch6 in two parts ((a) creating the generator and (b) plugging it
    in) and use the generator in the unit tests of the vcpuid->apicid mapping
    function. Becomes patches 6 and 8 in v2.

Patch 1: Same as v1/patch1.
Patch 2: Header dependency cleanup in preparation for patch3.
Patch 3: Adds vlapic_hidden check for the newly introduced reserved area.
Patch 4: [hvmloader] Replaces INIT+SIPI+SIPI sequences with hypercalls.
Patch 5: [hvmloader] Retrieve the per-CPU APIC IDs from the APs themselves.
Patch 6: Split from v1/patch6.
Patch 7: Logically matching v1/patch5, but using v2/patch6 for testing.
Patch 8: Split from v1/patch6.

=== Original cover letter ===

Current topology handling is close to non-existent. As things stand, APIC
IDs are allocated through the apic_id=vcpu_id*2 relation without giving any
hints to the OS on how to parse the x2APIC ID of a given CPU and assuming
the guest will assume 2 threads per core.

This series involves bringing x2APIC IDs into the migration stream, so
older guests keep operating as they used to and enhancing Xen+toolstack so
new guests get topology information consistent with their x2APIC IDs. As a
side effect of this, x2APIC IDs are now packed and don't have (unless under
a pathological case) gaps.

Further work ought to allow combining this topology configurations with
gang-scheduling of guest hyperthreads into affine physical hyperthreads.
For the time being it purposefully keeps the configuration of "1 socket" +
"1 thread per core" + "1 core per vCPU".

Patch 1: Includes x2APIC IDs in the migration stream. This allows Xen to
         reconstruct the right x2APIC IDs on migrated-in guests, and
         future-proofs itself in the face of x2APIC ID derivation changes.
Patch 2: Minor refactor to expose xc_cpu_policy in libxl
Patch 3: Refactors xen/lib/x86 to work on non-Xen freestanding environments
         (e.g: hvmloader)
Patch 4: Remove old assumptions about vcpu_id<->apic_id relationship in hvmloader
Patch 5: Add logic to derive x2APIC IDs given a CPU policy and vCPU IDs
Patch 6: Includes a simple topology generator for toolstack so new guests
         have topologically consistent information in CPUID

Alejandro Vallejo (8):
  xen/x86: Add initial x2APIC ID to the per-vLAPIC save area
  xen/x86: Simplify header dependencies in x86/hvm
  x86/vlapic: Move lapic_load_hidden migration checks to the check hook
  tools/hvmloader: Wake APs with hypercalls and not with INIT+SIPI+SIPI
  tools/hvmloader: Retrieve (x2)APIC IDs from the APs themselves
  xen/lib: Add topology generator for x86
  xen/x86: Derive topologically correct x2APIC IDs from the policy
  xen/x86: Synthesise domain topologies

 tools/firmware/hvmloader/config.h        |   6 +-
 tools/firmware/hvmloader/hvmloader.c     |   4 +-
 tools/firmware/hvmloader/smp.c           | 151 +++++++++---------
 tools/firmware/hvmloader/util.h          |   5 +
 tools/libs/guest/xg_cpuid_x86.c          |  62 ++------
 tools/tests/cpu-policy/test-cpu-policy.c | 191 +++++++++++++++++++++++
 xen/arch/x86/cpu-policy.c                |   9 +-
 xen/arch/x86/cpuid.c                     |  15 +-
 xen/arch/x86/hvm/irq.c                   |   6 +-
 xen/arch/x86/hvm/vlapic.c                |  75 +++++++--
 xen/arch/x86/include/asm/hvm/hvm.h       |   8 +
 xen/arch/x86/include/asm/hvm/vlapic.h    |   8 +-
 xen/arch/x86/include/asm/hvm/vpt.h       |   1 -
 xen/include/public/arch-x86/hvm/save.h   |   2 +
 xen/include/xen/lib/x86/cpu-policy.h     |  27 ++++
 xen/lib/x86/policy.c                     | 160 +++++++++++++++++++
 16 files changed, 562 insertions(+), 168 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718779.1121298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003MN-KL; Wed, 08 May 2024 12:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718779.1121298; Wed, 08 May 2024 12:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003LY-Cr; Wed, 08 May 2024 12:39:39 +0000
Received: by outflank-mailman (input) for mailman id 718779;
 Wed, 08 May 2024 12:39:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4ga9-0002b0-J3
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:37 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06dbf4e2-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:35 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59b49162aeso949718866b.3
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:35 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06dbf4e2-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171975; x=1715776775; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SqD0+veAXWzP3pl5EqEMwlTj/kQBDP8F+ZMM9Cf6nOI=;
        b=b4Tqc86tE+8yM+fJ41o291lmU20XK9g5RjMWM9NIzFrYzQDr0Eb08XOHEaDvwKTwLA
         Y1Lm55N4yU+5VGsmyYtzz0l+1IF92TRxbFZcmCIRerHNO9IXlrDiGhyKmBKPIPPrUzxo
         mW2+ij4ZNIGGPMj8E4oABS4EaN2c+TKdWfFfs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171975; x=1715776775;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SqD0+veAXWzP3pl5EqEMwlTj/kQBDP8F+ZMM9Cf6nOI=;
        b=ueCeDGnBB8Jyp5OzakJ9K7xqlXBHYjySlY0d1vdrrVUJnR44T4HQIZao4kaReKCe8z
         +Hc3HvEbIsKrB6goNmXxXycgLU5zpi2p6Q3juAMe9it6LkkeJrEFqM5BRxIe3fZEw/gj
         SBbp4+s/YRGURgsf9Ej+yk66cf6NcfuGaQaf+XvUMN/ELlbywE15SN7cKhpOcfC6UvYC
         3AeBKW7ZVzt+j89bV0wvc0lgW812CB6n7KxJLu8FEn0vMn9fhViUVFKys68qlwwsrw+u
         Ty+6EvzLoa7Yx/MSVwOW6s9wjAxwl/fHKgdsYKBHgF+VhKWFdQ2f1jVggvXvVTLtO3R5
         04UA==
X-Gm-Message-State: AOJu0Ywswu5fsUUmsjUCivSaom1fxUgueOQShkO9yFVlnbDHb9zMohby
	hkt42Tp6bC9IfPLzF6xaS2b4Fpk66Ukbe1paT38Pgxz/MZcut9+gtFuBPY8lHifBMj15CeRJywO
	Y
X-Google-Smtp-Source: AGHT+IGg/M+0iKytNyTfXIK6S1aJWBIHRH7JNnNwznMpl2CuP/vkB2TIZRLcAfCZph/OLryXk56CSw==
X-Received: by 2002:a17:907:990b:b0:a59:c3d0:550c with SMTP id a640c23a62f3a-a59fb95a533mr160323066b.43.1715171975070;
        Wed, 08 May 2024 05:39:35 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v2 4/8] tools/hvmloader: Wake APs with hypercalls and not with INIT+SIPI+SIPI
Date: Wed,  8 May 2024 13:39:23 +0100
Message-Id: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Removes a needless assembly entry point and simplifies the codebase by allowing
hvmloader to wake APs it doesn't know the APIC ID of.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * New patch. Replaces adding cpu policy to hvmloader in v1.
---
 tools/firmware/hvmloader/smp.c | 111 +++++++++++++--------------------
 1 file changed, 44 insertions(+), 67 deletions(-)

diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
index 082b17f13818..a668f15d7e1f 100644
--- a/tools/firmware/hvmloader/smp.c
+++ b/tools/firmware/hvmloader/smp.c
@@ -22,88 +22,68 @@
 #include "util.h"
 #include "config.h"
 #include "apic_regs.h"
+#include "hypercall.h"
 
-#define AP_BOOT_EIP 0x1000
-extern char ap_boot_start[], ap_boot_end[];
+#include <xen/asm/x86-defns.h>
+#include <xen/hvm/hvm_vcpu.h>
+
+#include <xen/vcpu.h>
 
 static int ap_callin, ap_cpuid;
 
-asm (
-    "    .text                       \n"
-    "    .code16                     \n"
-    "ap_boot_start: .code16          \n"
-    "    mov   %cs,%ax               \n"
-    "    mov   %ax,%ds               \n"
-    "    lgdt  gdt_desr-ap_boot_start\n"
-    "    xor   %ax, %ax              \n"
-    "    inc   %ax                   \n"
-    "    lmsw  %ax                   \n"
-    "    ljmpl $0x08,$1f             \n"
-    "gdt_desr:                       \n"
-    "    .word gdt_end - gdt - 1     \n"
-    "    .long gdt                   \n"
-    "ap_boot_end: .code32            \n"
-    "1:  mov   $0x10,%eax            \n"
-    "    mov   %eax,%ds              \n"
-    "    mov   %eax,%es              \n"
-    "    mov   %eax,%ss              \n"
-    "    movl  $stack_top,%esp       \n"
-    "    movl  %esp,%ebp             \n"
-    "    call  ap_start              \n"
-    "1:  hlt                         \n"
-    "    jmp  1b                     \n"
-    "                                \n"
-    "    .align 8                    \n"
-    "gdt:                            \n"
-    "    .quad 0x0000000000000000    \n"
-    "    .quad 0x00cf9a000000ffff    \n" /* 0x08: Flat code segment */
-    "    .quad 0x00cf92000000ffff    \n" /* 0x10: Flat data segment */
-    "gdt_end:                        \n"
-    "                                \n"
-    "    .bss                        \n"
-    "    .align    8                 \n"
-    "stack:                          \n"
-    "    .skip    0x4000             \n"
-    "stack_top:                      \n"
-    "    .text                       \n"
-    );
-
-void ap_start(void); /* non-static avoids unused-function compiler warning */
-/*static*/ void ap_start(void)
+static void ap_start(void)
 {
     printf(" - CPU%d ... ", ap_cpuid);
     cacheattr_init();
     printf("done.\n");
+
+    if ( !ap_cpuid )
+        return;
+
     wmb();
     ap_callin = 1;
-}
 
-static void lapic_wait_ready(void)
-{
-    while ( lapic_read(APIC_ICR) & APIC_ICR_BUSY )
-        cpu_relax();
+    while ( 1 )
+        asm volatile ( "hlt" );
 }
 
 static void boot_cpu(unsigned int cpu)
 {
-    unsigned int icr2 = SET_APIC_DEST_FIELD(LAPIC_ID(cpu));
+    static uint8_t ap_stack[4 * PAGE_SIZE] __attribute__ ((aligned (16)));
+    static struct vcpu_hvm_context ap;
 
     /* Initialise shared variables. */
     ap_cpuid = cpu;
-    ap_callin = 0;
     wmb();
 
-    /* Wake up the secondary processor: INIT-SIPI-SIPI... */
-    lapic_wait_ready();
-    lapic_write(APIC_ICR2, icr2);
-    lapic_write(APIC_ICR, APIC_DM_INIT);
-    lapic_wait_ready();
-    lapic_write(APIC_ICR2, icr2);
-    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
-    lapic_wait_ready();
-    lapic_write(APIC_ICR2, icr2);
-    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
-    lapic_wait_ready();
+    /* Wake up the secondary processor */
+    ap = (struct vcpu_hvm_context) {
+        .mode = VCPU_HVM_MODE_32B,
+        .cpu_regs.x86_32 = {
+            .eip = (uint32_t)ap_start,
+            .esp = (uint32_t)ap_stack + ARRAY_SIZE(ap_stack),
+
+            /* Protected mode with MMU off */
+            .cr0 = X86_CR0_PE,
+
+            /* Prepopulate the GDT */
+            .cs_limit = -1U,
+            .ds_limit = -1U,
+            .ss_limit = -1U,
+            .es_limit = -1U,
+            .tr_limit = 0x67,
+            .cs_ar = 0xc9b,
+            .ds_ar = 0xc93,
+            .es_ar = 0xc93,
+            .ss_ar = 0xc93,
+            .tr_ar = 0x8b,
+        },
+    };
+
+    if ( hypercall_vcpu_op(VCPUOP_initialise, cpu, &ap) )
+        BUG();
+    if ( hypercall_vcpu_op(VCPUOP_up, cpu, NULL) )
+        BUG();
 
     /*
      * Wait for the secondary processor to complete initialisation.
@@ -113,17 +93,14 @@ static void boot_cpu(unsigned int cpu)
         cpu_relax();
 
     /* Take the secondary processor offline. */
-    lapic_write(APIC_ICR2, icr2);
-    lapic_write(APIC_ICR, APIC_DM_INIT);
-    lapic_wait_ready();    
+    if ( hypercall_vcpu_op(VCPUOP_down, cpu, NULL) )
+        BUG();
 }
 
 void smp_initialise(void)
 {
     unsigned int i, nr_cpus = hvm_info->nr_vcpus;
 
-    memcpy((void *)AP_BOOT_EIP, ap_boot_start, ap_boot_end - ap_boot_start);
-
     printf("Multiprocessor initialisation:\n");
     ap_start();
     for ( i = 1; i < nr_cpus; i++ )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718781.1121313 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaC-0003fM-JF; Wed, 08 May 2024 12:39:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718781.1121313; Wed, 08 May 2024 12:39:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaC-0003e7-EQ; Wed, 08 May 2024 12:39:40 +0000
Received: by outflank-mailman (input) for mailman id 718781;
 Wed, 08 May 2024 12:39:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4gaB-0002b0-JQ
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:39 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07d48e6c-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:37 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a599af16934so1088765366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:37 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07d48e6c-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171976; x=1715776776; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HgVeKZJJDa3o9g7T6dDnYuOFXwdr65sevSfeNvtvPGo=;
        b=etWXzobbzyIV+PYTlgoFsqPJR87wtyO1yzhz3f4p0KmYMPG9K5A7Uxa48fMhw0EV11
         jO+fBxW4Ek+DZbfehXsbLfea3sEGdi4G9skI0477CdibcRncQLkF1wXkMOlXYG9NsPCN
         R0j8z4R5Pz1CHU5+KZOffZGtY3kLq7RTP8L+o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171976; x=1715776776;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=HgVeKZJJDa3o9g7T6dDnYuOFXwdr65sevSfeNvtvPGo=;
        b=nYXNkLse6kNyy++hyhJAetq8DWk+uqSKA+pCVxW0oKOAqYuw09Wjx3+94YErGFCqvv
         9SYxZc6ByE0/Qpzo7VFmO0DZ+ss8cOEEZlaMAh6DQnjMp7obrdXX7k3HmtPm4ev47BDf
         M8EeS5Wp351slQpP/olKQQ59RtlORtsllqhAnZZ38Yf/k1pPLqgTi30Ya0DVlN3dLw9b
         3ftnIJHLitJGExOHuzgcCXihCE9X0JWboii3TbUyT8YguGrvDgchk5DiKI4OHMIfrafl
         SfKBYOZg+7C6uQ9/DiUbpmhpzo6Pe69CgW8VpqLC29r7e57EdtpljANOHBZenYVD4h8d
         7GXg==
X-Gm-Message-State: AOJu0Yz1Ad8oVZIS2hPUMFZBA2rJNeBTLJCkT9qE9VEKObC9UlIy5xLz
	qDwP6nnqhcJV929G5dU/GS7AnWa6q24oLGFK5WmsPb3oItN/qK655l2T+9YltiXfv+6E1OaomlC
	S
X-Google-Smtp-Source: AGHT+IEjhEqD6jSzOOxN3SFf1hIWRaJ0vLeIkZb+dYtRv4xOorgZ2g2626xib1oqp8QwkOANeImtag==
X-Received: by 2002:a17:906:3901:b0:a52:65bd:a19a with SMTP id a640c23a62f3a-a59fb9d6462mr167510066b.57.1715171976633;
        Wed, 08 May 2024 05:39:36 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v2 6/8] xen/lib: Add topology generator for x86
Date: Wed,  8 May 2024 13:39:25 +0100
Message-Id: <1ffad529d7fed10381df67215c747fc2d69f805e.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a helper to populate topology leaves in the cpu policy from
threads/core and cores/package counts.

No functional change, as it's not connected to anything yet.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * New patch. Extracted from v1/patch6
---
 tools/tests/cpu-policy/test-cpu-policy.c | 128 +++++++++++++++++++++++
 xen/include/xen/lib/x86/cpu-policy.h     |  16 +++
 xen/lib/x86/policy.c                     |  86 +++++++++++++++
 3 files changed, 230 insertions(+)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
index 301df2c00285..0ba8c418b1b3 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -650,6 +650,132 @@ static void test_is_compatible_failure(void)
     }
 }
 
+static void test_topo_from_parts(void)
+{
+    static const struct test {
+        unsigned int threads_per_core;
+        unsigned int cores_per_pkg;
+        struct cpu_policy policy;
+    } tests[] = {
+        {
+            .threads_per_core = 3, .cores_per_pkg = 1,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
+                    [1] = { .nr_logical = 1, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 1, .cores_per_pkg = 3,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
+                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 7, .cores_per_pkg = 5,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
+                    [1] = { .nr_logical = 5, .level = 1, .type = 2, .id_shift = 6, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 2, .cores_per_pkg = 128,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
+                    [1] = { .nr_logical = 128, .level = 1, .type = 2, .id_shift = 8, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 3, .cores_per_pkg = 1,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
+                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 1, .cores_per_pkg = 3,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
+                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 7, .cores_per_pkg = 5,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
+                    [1] = { .nr_logical = 35, .level = 1, .type = 2, .id_shift = 6, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 2, .cores_per_pkg = 128,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
+                    [1] = { .nr_logical = 256, .level = 1, .type = 2, .id_shift = 8, },
+                },
+            },
+        },
+    };
+
+    printf("Testing topology synthesis from parts:\n");
+
+    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+    {
+        const struct test *t = &tests[i];
+        struct cpu_policy actual = { .x86_vendor = t->policy.x86_vendor };
+        int rc = x86_topo_from_parts(&actual, t->threads_per_core, t->cores_per_pkg);
+
+        if ( rc || memcmp(&actual.topo, &t->policy.topo, sizeof(actual.topo)) )
+        {
+#define TOPO(n) topo.subleaf[(n)]
+            fail("FAIL[%d] - '%s %u t/c, %u c/p'\n",
+                 rc,
+                 x86_cpuid_vendor_to_str(t->policy.x86_vendor),
+                 t->threads_per_core, t->cores_per_pkg);
+            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
+                   "             expected_lvl=%u actual_lvl=%u\n"
+                   "             expected_type=%u actual_type=%u\n"
+                   "             expected_shift=%u actual_shift=%u\n",
+                   0, t->policy.TOPO(0).nr_logical, actual.TOPO(0).nr_logical,
+                      t->policy.TOPO(0).level,      actual.TOPO(0).level,
+                      t->policy.TOPO(0).type,       actual.TOPO(0).type,
+                      t->policy.TOPO(0).id_shift,   actual.TOPO(0).id_shift);
+
+            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
+                   "             expected_lvl=%u actual_lvl=%u\n"
+                   "             expected_type=%u actual_type=%u\n"
+                   "             expected_shift=%u actual_shift=%u\n",
+                   1, t->policy.TOPO(1).nr_logical, actual.TOPO(1).nr_logical,
+                      t->policy.TOPO(1).level,      actual.TOPO(1).level,
+                      t->policy.TOPO(1).type,       actual.TOPO(1).type,
+                      t->policy.TOPO(1).id_shift,   actual.TOPO(1).id_shift);
+#undef TOPO
+        }
+    }
+}
+
 int main(int argc, char **argv)
 {
     printf("CPU Policy unit tests\n");
@@ -667,6 +793,8 @@ int main(int argc, char **argv)
     test_is_compatible_success();
     test_is_compatible_failure();
 
+    test_topo_from_parts();
+
     if ( nr_failures )
         printf("Done: %u failures\n", nr_failures);
     else
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index 392320b9adbe..f5df18e9f77c 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -551,6 +551,22 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
  */
 uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
 
+/**
+ * Synthesise topology information in `p` given high-level constraints
+ *
+ * Topology is given in various fields accross several leaves, some of
+ * which are vendor-specific. This function uses the policy itself to
+ * derive such leaves from threads/core and cores/package.
+ *
+ * @param p                   CPU policy of the domain.
+ * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
+ * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
+ * @return                    0 on success; -errno on failure
+ */
+int x86_topo_from_parts(struct cpu_policy *p,
+                        unsigned int threads_per_core,
+                        unsigned int cores_per_pkg);
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index 4cef658feeb8..d033ee5398dd 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -13,6 +13,92 @@ uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
     return vcpu_id * 2;
 }
 
+static unsigned int order(unsigned int n)
+{
+    return 8 * sizeof(n) - __builtin_clz(n);
+}
+
+int x86_topo_from_parts(struct cpu_policy *p,
+                        unsigned int threads_per_core,
+                        unsigned int cores_per_pkg)
+{
+    unsigned int threads_per_pkg = threads_per_core * cores_per_pkg;
+    unsigned int apic_id_size;
+
+    if ( !p || !threads_per_core || !cores_per_pkg )
+        return -EINVAL;
+
+    p->basic.max_leaf = MAX(0xb, p->basic.max_leaf);
+
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
+
+    /* thread level */
+    p->topo.subleaf[0].nr_logical = threads_per_core;
+    p->topo.subleaf[0].id_shift = 0;
+    p->topo.subleaf[0].level = 0;
+    p->topo.subleaf[0].type = 1;
+    if ( threads_per_core > 1 )
+        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
+
+    /* core level */
+    p->topo.subleaf[1].nr_logical = cores_per_pkg;
+    if ( p->x86_vendor == X86_VENDOR_INTEL )
+        p->topo.subleaf[1].nr_logical = threads_per_pkg;
+    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
+    p->topo.subleaf[1].level = 1;
+    p->topo.subleaf[1].type = 2;
+    if ( cores_per_pkg > 1 )
+        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
+
+    apic_id_size = p->topo.subleaf[1].id_shift;
+
+    /*
+     * Contrary to what the name might seem to imply. HTT is an enabler for
+     * SMP and there's no harm in setting it even with a single vCPU.
+     */
+    p->basic.htt = true;
+    p->basic.lppp = MIN(0xff, p->basic.lppp);
+
+    switch ( p->x86_vendor )
+    {
+        case X86_VENDOR_INTEL: {
+            struct cpuid_cache_leaf *sl = p->cache.subleaf;
+            for ( size_t i = 0; sl->type &&
+                                i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
+            {
+                sl->cores_per_package = cores_per_pkg - 1;
+                sl->threads_per_cache = threads_per_core - 1;
+                if ( sl->type == 3 /* unified cache */ )
+                    sl->threads_per_cache = threads_per_pkg - 1;
+            }
+            break;
+        }
+        case X86_VENDOR_AMD:
+        case X86_VENDOR_HYGON:
+            /* Expose p->basic.lppp */
+            p->extd.cmp_legacy = true;
+
+            /* Clip NC to the maximum value it can hold */
+            p->extd.nc = 0xff;
+            if ( threads_per_pkg <= 0xff )
+                p->extd.nc = threads_per_pkg - 1;
+
+            /* TODO: Expose leaf e1E */
+            p->extd.topoext = false;
+
+            /*
+             * Clip APIC ID to 8 bits, as that's what high core-count machines do
+             *
+             * That what AMD EPYC 9654 does with >256 CPUs
+             */
+            p->extd.apic_id_size = MIN(8, apic_id_size);
+
+            break;
+    }
+
+    return 0;
+}
+
 int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718778.1121292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003Iv-6u; Wed, 08 May 2024 12:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718778.1121292; Wed, 08 May 2024 12:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaB-0003Io-3i; Wed, 08 May 2024 12:39:39 +0000
Received: by outflank-mailman (input) for mailman id 718778;
 Wed, 08 May 2024 12:39:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4ga9-0002bO-0s
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:37 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05c39b90-0d38-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 14:39:34 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59a0168c75so1133548366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:33 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05c39b90-0d38-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171973; x=1715776773; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Rt+Dpwuj06WuxiC7ieJ01ZuDEFxunnINhuSQIiORfm8=;
        b=BnbLM+com9EQbRs3rr8aKO/hO+jHeNe8qlkIMt8HllnArmrOWmC+t7Ndh9XnInzoK1
         cvAquPszSTCXmdpHFln7nfcxotUfYgQffWrpK4tU/H4H7p6OmlE199Yl/PcvFluHFgmm
         CIdt1Ff17yMipLQG3nuHBNaFoJYYBsEAsRtZU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171973; x=1715776773;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Rt+Dpwuj06WuxiC7ieJ01ZuDEFxunnINhuSQIiORfm8=;
        b=VDCmMYhOO/Nm65Qqm5CvSK6kLcqI7B1OwMXRMbF8MbcPkCEwvsLqpyF3kE8RuUFavR
         LpLxnVQLJYliCpqSBsWnA1F/ORdlcH+awevVFp0QffU+lWVfIFvdhM8jpt+R+GmtbTBG
         NQ7GHxzMqcwrV+qAMa0CCuOKHeGZlRBxHLk/oUNkWA5YUjx66W0i+dStPLGUhUmk1A/p
         4KWnUPWNVIesfJ2t6G+3Auj8K8EEaXIKj65LP4YHRiaJJU5t95LF1i/sRAZuJhc3txgn
         T96ynwWCznMJBlxivpka2R6rHFNUW6OmNNjhWzBi/fOw2DfGoZ0vDyrB/1tnFxEzBJdU
         YWQg==
X-Gm-Message-State: AOJu0YylppFUSqOJYs4al81mspX8Wpm8fLYhAWV//3nR1WR1a/trD4VE
	/kQaGqeNkMJgF8vnTjfFWvW79MBgraF+TAkUfUm6PwmyJqN9OEAUrYe8mu8aIl9OFT3e5k8cwOe
	1
X-Google-Smtp-Source: AGHT+IF3f0rpiBCbGK6d85BfHLVqHkydomk/mOPnkyuMz2szjNGaZF3xK+tRFQaovXxLP12m59c+Iw==
X-Received: by 2002:a17:907:6ea3:b0:a59:bc9d:a0ab with SMTP id a640c23a62f3a-a59fb9f3409mr169249366b.72.1715171972911;
        Wed, 08 May 2024 05:39:32 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 1/8] xen/x86: Add initial x2APIC ID to the per-vLAPIC save area
Date: Wed,  8 May 2024 13:39:20 +0100
Message-Id: <4095f31a88589ced2b620e8ebbb84cdc2fae8914.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This allows the initial x2APIC ID to be sent on the migration stream. The
hardcoded mapping x2apic_id=2*vcpu_id is maintained for the time being.
Given the vlapic data is zero-extended on restore, fix up migrations from
hosts without the field by setting it to the old convention if zero.

x2APIC IDs are calculated from the CPU policy where the guest topology is
defined. For the time being, the function simply returns the old
relationship, but will eventually return results consistent with the
topology.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * Removed usage of SET_xAPIC_ID().
  * Restored previous logic when exposing leaf 0xb, and gate it for HVM only.
  * Rewrote comment in lapic_load_fixup, including the implicit assumption.
  * Moved vlapic_cpu_policy_changed() into hvm_cpuid_policy_changed())
  * const-ified policy in vlapic_cpu_policy_changed()
---
 xen/arch/x86/cpuid.c                   | 15 ++++---------
 xen/arch/x86/hvm/vlapic.c              | 30 ++++++++++++++++++++++++--
 xen/arch/x86/include/asm/hvm/hvm.h     |  1 +
 xen/arch/x86/include/asm/hvm/vlapic.h  |  2 ++
 xen/include/public/arch-x86/hvm/save.h |  2 ++
 xen/include/xen/lib/x86/cpu-policy.h   |  9 ++++++++
 xen/lib/x86/policy.c                   | 11 ++++++++++
 7 files changed, 57 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 7a38e032146a..242c21ec5bb6 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -139,10 +139,9 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
         const struct cpu_user_regs *regs;
 
     case 0x1:
-        /* TODO: Rework topology logic. */
         res->b &= 0x00ffffffu;
         if ( is_hvm_domain(d) )
-            res->b |= (v->vcpu_id * 2) << 24;
+            res->b |= vlapic_x2apic_id(vcpu_vlapic(v)) << 24;
 
         /* TODO: Rework vPMU control in terms of toolstack choices. */
         if ( vpmu_available(v) &&
@@ -311,19 +310,13 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
         break;
 
     case 0xb:
-        /*
-         * In principle, this leaf is Intel-only.  In practice, it is tightly
-         * coupled with x2apic, and we offer an x2apic-capable APIC emulation
-         * to guests on AMD hardware as well.
-         *
-         * TODO: Rework topology logic.
-         */
-        if ( p->basic.x2apic )
+        /* Don't expose topology information to PV guests */
+        if ( is_hvm_domain(d) && p->basic.x2apic )
         {
             *(uint8_t *)&res->c = subleaf;
 
             /* Fix the x2APIC identifier. */
-            res->d = v->vcpu_id * 2;
+            res->d = vlapic_x2apic_id(vcpu_vlapic(v));
         }
         break;
 
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 05072a21bf38..61a96474006b 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1069,7 +1069,7 @@ static uint32_t x2apic_ldr_from_id(uint32_t id)
 static void set_x2apic_id(struct vlapic *vlapic)
 {
     const struct vcpu *v = vlapic_vcpu(vlapic);
-    uint32_t apic_id = v->vcpu_id * 2;
+    uint32_t apic_id = vlapic->hw.x2apic_id;
     uint32_t apic_ldr = x2apic_ldr_from_id(apic_id);
 
     /*
@@ -1083,6 +1083,22 @@ static void set_x2apic_id(struct vlapic *vlapic)
     vlapic_set_reg(vlapic, APIC_LDR, apic_ldr);
 }
 
+void vlapic_cpu_policy_changed(struct vcpu *v)
+{
+    struct vlapic *vlapic = vcpu_vlapic(v);
+    const struct cpu_policy *cp = v->domain->arch.cpu_policy;
+
+    /*
+     * Don't override the initial x2APIC ID if we have migrated it or
+     * if the domain doesn't have vLAPIC at all.
+     */
+    if ( !has_vlapic(v->domain) || vlapic->loaded.hw )
+        return;
+
+    vlapic->hw.x2apic_id = x86_x2apic_id_from_vcpu_id(cp, v->vcpu_id);
+    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
+}
+
 int guest_wrmsr_apic_base(struct vcpu *v, uint64_t val)
 {
     const struct cpu_policy *cp = v->domain->arch.cpu_policy;
@@ -1449,7 +1465,7 @@ void vlapic_reset(struct vlapic *vlapic)
     if ( v->vcpu_id == 0 )
         vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
 
-    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
+    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
     vlapic_do_init(vlapic);
 }
 
@@ -1514,6 +1530,16 @@ static void lapic_load_fixup(struct vlapic *vlapic)
     const struct vcpu *v = vlapic_vcpu(vlapic);
     uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id);
 
+    /*
+     * Loading record without hw.x2apic_id in the save stream, calculate using
+     * the traditional "vcpu_id * 2" relation. There's an implicit assumption
+     * that vCPU0 always has x2APIC0, which is true for the old relation, and
+     * still holds under the new x2APIC generation algorithm. While that case
+     * goes through the conditional it's benign because it still maps to zero.
+     */
+    if ( !vlapic->hw.x2apic_id )
+        vlapic->hw.x2apic_id = v->vcpu_id * 2;
+
     /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */
     if ( !vlapic_x2apic_mode(vlapic) ||
          (vlapic->loaded.ldr == good_ldr) )
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index 0c9e6f15645d..e1f0585d75a9 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -448,6 +448,7 @@ static inline void hvm_update_guest_efer(struct vcpu *v)
 static inline void hvm_cpuid_policy_changed(struct vcpu *v)
 {
     alternative_vcall(hvm_funcs.cpuid_policy_changed, v);
+    vlapic_cpu_policy_changed(v);
 }
 
 static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
index 88ef94524339..e8d41313abd3 100644
--- a/xen/arch/x86/include/asm/hvm/vlapic.h
+++ b/xen/arch/x86/include/asm/hvm/vlapic.h
@@ -44,6 +44,7 @@
 #define vlapic_xapic_mode(vlapic)                               \
     (!vlapic_hw_disabled(vlapic) && \
      !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD))
+#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id)
 
 /*
  * Generic APIC bitmap vector update & search routines.
@@ -107,6 +108,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack);
 
 int  vlapic_init(struct vcpu *v);
 void vlapic_destroy(struct vcpu *v);
+void vlapic_cpu_policy_changed(struct vcpu *v);
 
 void vlapic_reset(struct vlapic *vlapic);
 
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 7ecacadde165..1c2ec669ffc9 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -394,6 +394,8 @@ struct hvm_hw_lapic {
     uint32_t             disabled; /* VLAPIC_xx_DISABLED */
     uint32_t             timer_divisor;
     uint64_t             tdt_msr;
+    uint32_t             x2apic_id;
+    uint32_t             rsvd_zero;
 };
 
 DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic);
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index d5e447e9dc06..392320b9adbe 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -542,6 +542,15 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err);
 
+/**
+ * Calculates the x2APIC ID of a vCPU given a CPU policy
+ *
+ * @param p          CPU policy of the domain.
+ * @param id         vCPU ID of the vCPU.
+ * @returns x2APIC ID of the vCPU.
+ */
+uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index f033d22785be..4cef658feeb8 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -2,6 +2,17 @@
 
 #include <xen/lib/x86/cpu-policy.h>
 
+uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
+{
+    /*
+     * TODO: Derive x2APIC ID from the topology information inside `p`
+     *       rather than from vCPU ID. This bodge is a temporary measure
+     *       until all infra is in place to retrieve or derive the initial
+     *       x2APIC ID from migrated domains.
+     */
+    return vcpu_id * 2;
+}
+
 int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718777.1121274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002jG-TX; Wed, 08 May 2024 12:39:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718777.1121274; Wed, 08 May 2024 12:39:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002ha-Lq; Wed, 08 May 2024 12:39:37 +0000
Received: by outflank-mailman (input) for mailman id 718777;
 Wed, 08 May 2024 12:39:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4ga8-0002b0-Io
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:36 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06a4974f-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:35 +0200 (CEST)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso82820181fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:35 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06a4974f-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171974; x=1715776774; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=H1Ga1HBZRTHKI6NaCRCof9WPNhoGPFDq14GP1TJIEi4=;
        b=BHV9QNW2x0qZztkc5PVqx/fyWqk4cKREUIPyFLbTbO3/mqjEmTsbsvbdxUoYLqg3Xr
         YgkY7xf6sE1PaLSkm+QRKI6F/2mCVsBOvSICXZx5sW2SAx9p0sYDA0Nzy12AdzIvEipt
         mTkPGwcLccNm/jMcKX+DxzfgOPFyXcLVSlkGw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171974; x=1715776774;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=H1Ga1HBZRTHKI6NaCRCof9WPNhoGPFDq14GP1TJIEi4=;
        b=iPbKKupU/JH/J/P3CC4e1rQ/+oIzgggFy8afZ+D7o+38fjA/0ElxyLNZlkVG6SBqHM
         8WSaHEVjgtDv+LuOYq9tOwI4KV+LV6xwKBiZH9BXn1B/DoR2Vr+oTvmrq3guZA0QMSZf
         piFWjeYii/Z1iEaA7j8V9s+sVnbSnR9uZspvdIEdWuW5VHZwVNAdgOnxXGcXSrCGoI42
         XKbi6ubW2FrmTijPRBMc7BC5V1FcLjbJfkjRfdjpI20UtHK0jXz1mNfzgPsX9rz5KbiP
         m/3sYfAvxmQzclPqXHPsd2pMLGyM3xGCeSkN93MB2IMt6lercpWDXdKpbikI7FUMMcmq
         Tsvw==
X-Gm-Message-State: AOJu0YzI6kozmeRt4lCcqMhJul6Kg5YhLkkY6VxP2VnvF1lPjnvxdKuw
	tYe85I10SCBm/haulTFpnayUcO2Ox4Cz5IYDj2B5yIHMxLTGGcA6dtfsxmck1ZPtD7No7uSCa5p
	k
X-Google-Smtp-Source: AGHT+IEsG9Cq9YdVt5/5FaGqvZELl/kornKj55RPE+pogIXnucFJ/moPPpMWOe3cSMbfkfmVqhtlVQ==
X-Received: by 2002:a2e:a6a1:0:b0:2d8:8fb6:a53d with SMTP id 38308e7fff4ca-2e4476995aamr17025641fa.42.1715171974299;
        Wed, 08 May 2024 05:39:34 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration checks to the check hook
Date: Wed,  8 May 2024 13:39:22 +0100
Message-Id: <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

While at it, add a check for the reserved field in the hidden save area.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * New patch. Addresses the missing check for rsvd_zero in v1.
---
 xen/arch/x86/hvm/vlapic.c | 41 ++++++++++++++++++++++++++++-----------
 1 file changed, 30 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 8a244100009c..2f06bff1b2cc 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
                v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
 }
 
-static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
+static int cf_check lapic_check_hidden(const struct domain *d,
+                                       hvm_domain_context_t *h)
 {
     unsigned int vcpuid = hvm_load_instance(h);
-    struct vcpu *v;
-    struct vlapic *s;
+    struct hvm_hw_lapic s;
 
     if ( !has_vlapic(d) )
         return -ENODEV;
 
     /* Which vlapic to load? */
-    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
+    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )
     {
         dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
                 d->domain_id, vcpuid);
         return -EINVAL;
     }
-    s = vcpu_vlapic(v);
 
-    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
+    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )
+        return -ENODATA;
+
+    /* EN=0 with EXTD=1 is illegal */
+    if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) ==
+         APIC_BASE_EXTD )
+        return -EINVAL;
+
+    /*
+     * Fail migrations from newer versions of Xen where
+     * rsvd_zero is interpreted as something else.
+     */
+    if ( s.rsvd_zero )
         return -EINVAL;
 
+    return 0;
+}
+
+static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
+{
+    unsigned int vcpuid = hvm_load_instance(h);
+    struct vcpu *v = d->vcpu[vcpuid];
+    struct vlapic *s = vcpu_vlapic(v);
+
+    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
+        BUG();
+
     s->loaded.hw = 1;
     if ( s->loaded.regs )
         lapic_load_fixup(s);
 
-    if ( !(s->hw.apic_base_msr & APIC_BASE_ENABLE) &&
-         unlikely(vlapic_x2apic_mode(s)) )
-        return -EINVAL;
-
     hvm_update_vlapic_mode(v);
 
     return 0;
@@ -1643,7 +1662,7 @@ static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
     return 0;
 }
 
-HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, NULL,
+HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, lapic_check_hidden,
                           lapic_load_hidden, 1, HVMSR_PER_VCPU);
 HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs, NULL,
                           lapic_load_regs, 1, HVMSR_PER_VCPU);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718776.1121266 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002f0-Ig; Wed, 08 May 2024 12:39:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718776.1121266; Wed, 08 May 2024 12:39:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ga9-0002eR-Dw; Wed, 08 May 2024 12:39:37 +0000
Received: by outflank-mailman (input) for mailman id 718776;
 Wed, 08 May 2024 12:39:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4ga8-0002bO-Ax
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:36 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05fbb670-0d38-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 14:39:34 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59a64db066so1105824266b.3
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:34 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05fbb670-0d38-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171973; x=1715776773; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=IQGCph8OwoOZ7Au91nMnyWXL6z4q2/NNY3PC0JWpYgc=;
        b=fnJa2taKYoWqge5+mJ0+kKfqsnYWuCRIx2K5a5Fiu9Z55EhIDsATBNfrYtXDa0ZEvO
         8jnzQm6cmyoFq3g/9ghgdnPd0HI42TTf06ndqj3sVoLSFnqhfYB8I82KdLC26e2z6GUy
         SpEK/rGSGn+EAxsayzn4GddDPLxvYWbl8xRwA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171973; x=1715776773;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=IQGCph8OwoOZ7Au91nMnyWXL6z4q2/NNY3PC0JWpYgc=;
        b=ltYPRltG1GT63mHy4d19XHZaSla/9mq+cLrfwatwkRoG6v0CGj9TilqGtzAeXN6zLg
         XAUiRgPGzypopeJquWdKPodvrPG7HS8TIIN2TSqAjGCfnVVwkXBSr7WhDUSgMSV7MtZ/
         4mzWOGSH7r1lSeH2YZmZ0noxBClizd6mP40ZDxrRDZ0PRuAJX4H3Lkrc9Vpc/JLD5gYf
         9sVUYpgPTDAuo9d8PuC0Xr7ruMpOW6gNVV2XqEOcEa1Ar5to1hPLeIDvror5L2dNwS5S
         Hjh/7A6AS86/439mO0CC40PeYFRLgmAuvxD9wbuLo3tlm2pi3o3y4JK07BYen7HnQdfX
         G+4g==
X-Gm-Message-State: AOJu0YzWUxT0f9B603KaonHbLQHxsxRo0P0G/ND64BsNNzddF5H0unxq
	9yeonqHaoh9MnglnYqLvQ3mp2Y0tON4vNqMS9Jh7VGY6vqHT9nMLbzh9Sm0uW8AKCXc1eK/lJ4s
	v
X-Google-Smtp-Source: AGHT+IFy+1YrPs2ygTxchgGmLZ0UyoDFvdObqrbFi+6msM/QkJlrfa66CMKbsLLSE2VB1GiduBNJ5A==
X-Received: by 2002:a17:906:dfc1:b0:a59:bcfd:d950 with SMTP id a640c23a62f3a-a59fb95e30bmr160219566b.46.1715171973587;
        Wed, 08 May 2024 05:39:33 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
Date: Wed,  8 May 2024 13:39:21 +0100
Message-Id: <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Otherwise it's not possible to call functions described in hvm/vlapic.h from the
inline functions of hvm/hvm.h.

This is because a static inline in vlapic.h depends on hvm.h, and pulls it
transitively through vpt.h. The ultimate cause is having hvm.h included in any
of the "v*.h" headers, so break the cycle moving the guilty inline into hvm.h.

No functional change.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * New patch. Prereq to moving vlapic_cpu_policy_changed() onto hvm.h
---
 xen/arch/x86/hvm/irq.c                | 6 +++---
 xen/arch/x86/hvm/vlapic.c             | 4 ++--
 xen/arch/x86/include/asm/hvm/hvm.h    | 6 ++++++
 xen/arch/x86/include/asm/hvm/vlapic.h | 6 ------
 xen/arch/x86/include/asm/hvm/vpt.h    | 1 -
 5 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index 4a9fe82cbd8d..4f5479b12c98 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -512,13 +512,13 @@ struct hvm_intack hvm_vcpu_has_pending_irq(struct vcpu *v)
     int vector;
 
     /*
-     * Always call vlapic_sync_pir_to_irr so that PIR is synced into IRR when
-     * using posted interrupts. Note this is also done by
+     * Always call hvm_vlapic_sync_pir_to_irr so that PIR is synced into IRR
+     * when using posted interrupts. Note this is also done by
      * vlapic_has_pending_irq but depending on which interrupts are pending
      * hvm_vcpu_has_pending_irq will return early without calling
      * vlapic_has_pending_irq.
      */
-    vlapic_sync_pir_to_irr(v);
+    hvm_vlapic_sync_pir_to_irr(v);
 
     if ( unlikely(v->arch.nmi_pending) )
         return hvm_intack_nmi;
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 61a96474006b..8a244100009c 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -98,7 +98,7 @@ static void vlapic_clear_irr(int vector, struct vlapic *vlapic)
 
 static int vlapic_find_highest_irr(struct vlapic *vlapic)
 {
-    vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic));
+    hvm_vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic));
 
     return vlapic_find_highest_vector(&vlapic->regs->data[APIC_IRR]);
 }
@@ -1516,7 +1516,7 @@ static int cf_check lapic_save_regs(struct vcpu *v, hvm_domain_context_t *h)
     if ( !has_vlapic(v->domain) )
         return 0;
 
-    vlapic_sync_pir_to_irr(v);
+    hvm_vlapic_sync_pir_to_irr(v);
 
     return hvm_save_entry(LAPIC_REGS, v->vcpu_id, h, vcpu_vlapic(v)->regs);
 }
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index e1f0585d75a9..84911f3ebcb4 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
         alternative_vcall(hvm_funcs.update_vlapic_mode, v);
 }
 
+static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
+{
+    if ( hvm_funcs.sync_pir_to_irr )
+        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
+}
+
 #else  /* CONFIG_HVM */
 
 #define hvm_enabled false
diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
index e8d41313abd3..34f23cd38a20 100644
--- a/xen/arch/x86/include/asm/hvm/vlapic.h
+++ b/xen/arch/x86/include/asm/hvm/vlapic.h
@@ -139,10 +139,4 @@ bool vlapic_match_dest(
     const struct vlapic *target, const struct vlapic *source,
     int short_hand, uint32_t dest, bool dest_mode);
 
-static inline void vlapic_sync_pir_to_irr(struct vcpu *v)
-{
-    if ( hvm_funcs.sync_pir_to_irr )
-        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
-}
-
 #endif /* __ASM_X86_HVM_VLAPIC_H__ */
diff --git a/xen/arch/x86/include/asm/hvm/vpt.h b/xen/arch/x86/include/asm/hvm/vpt.h
index feb0bf43f14b..0b92b286252d 100644
--- a/xen/arch/x86/include/asm/hvm/vpt.h
+++ b/xen/arch/x86/include/asm/hvm/vpt.h
@@ -11,7 +11,6 @@
 #include <xen/timer.h>
 #include <xen/list.h>
 #include <xen/rwlock.h>
-#include <asm/hvm/hvm.h>
 
 /*
  * Abstract layer of periodic time, one short time.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 12:39:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 12:39:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718783.1121337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaE-0004Nq-HA; Wed, 08 May 2024 12:39:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718783.1121337; Wed, 08 May 2024 12:39:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4gaE-0004Li-9k; Wed, 08 May 2024 12:39:42 +0000
Received: by outflank-mailman (input) for mailman id 718783;
 Wed, 08 May 2024 12:39:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6nIi=ML=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s4gaC-0002b0-JS
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 12:39:40 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08ccfda2-0d38-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 14:39:39 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59a0168c75so1133579866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 05:39:39 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 uj4-20020a170907c98400b00a599f876c28sm5984439ejc.38.2024.05.08.05.39.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 05:39:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08ccfda2-0d38-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715171978; x=1715776778; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WBnTFB98oFX4thcLAgiTD7V4T2rkgzAOEvWeV1KCES8=;
        b=UdnJkS1Xey+jFVrHMVucdBDQL2SzZ4TR83e/tNSjcKcBV8XdzPbsUDFeeqIbA0zXUs
         7LB9lbKeGF2ACT95pL4gZe9h9f6LatQDMCvcUvvJP8ceU9w3sK2YJ+F6nv1zNPEkE8nW
         yGdkNsooVmP7y0ZPN/4KepW6Ha6i5ErQkZ4gE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715171978; x=1715776778;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=WBnTFB98oFX4thcLAgiTD7V4T2rkgzAOEvWeV1KCES8=;
        b=h+l/lku6MBcCaymWVfNWhh5zVvLCL5WZprm4D0ArFblpjTcTTQL48IFieRDWf3fePQ
         PrY9IyvvzOHmN72n9lxfU2K/mKFDgd4y4YUTTI2yU8TzkmAO+6v2m692KWjguDTcUiLb
         kAGb1LXK23dH6RCX7+VzsJg4/RipsqDCoXkdRn+0LoCjn3q0TUwcK4A6QqdsZTdATgd6
         VHm0RLY12lHc5YRm957cciIJKwhorPTOvcDcFOzzKJNAghFFnmd5mTe2nInq0AoSuZYz
         zoQvC6rSWTK/lB2VcWox/1ZrXPuH7F5KqndeMx0i1SdXViuIct5xi79tZza71VyG1kyl
         yqlg==
X-Gm-Message-State: AOJu0Yx89cirfJKIdjUKQHpJYzqQMvbXbThylkbJfTKBEHspsgBnpa0j
	Syw5Jy3SclAn4EDMho9cejyzjPkiJa+X23wLIQgJqqMx2BiMjmuvvdPuBpu7Mfv45ZKgiAOCL81
	x
X-Google-Smtp-Source: AGHT+IERL7rxnRlKAg01EIgOZcb957IgS/zzCgslJ25W7IBtQ+EVGdBOj/K4oGZ1jLeGW1fySAEirQ==
X-Received: by 2002:a17:906:7951:b0:a59:ce25:9b88 with SMTP id a640c23a62f3a-a59fb9dbc65mr193715266b.54.1715171978238;
        Wed, 08 May 2024 05:39:38 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 8/8] xen/x86: Synthesise domain topologies
Date: Wed,  8 May 2024 13:39:27 +0100
Message-Id: <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715102098.git.alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Expose sensible topologies in leaf 0xb. At the moment it synthesises non-HT
systems, in line with the previous code intent.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * Zap the topology leaves of (pv/hvm)_(def/max)_policy rather than the host policy
---
 tools/libs/guest/xg_cpuid_x86.c | 62 +++++----------------------------
 xen/arch/x86/cpu-policy.c       |  9 +++--
 2 files changed, 15 insertions(+), 56 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..8170769dbe43 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -584,7 +584,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     bool hvm;
     xc_domaininfo_t di;
     struct xc_cpu_policy *p = xc_cpu_policy_init();
-    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
+    unsigned int nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
     uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
     uint32_t len = ARRAY_SIZE(host_featureset);
@@ -727,59 +727,15 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     else
     {
-        /*
-         * Topology for HVM guests is entirely controlled by Xen.  For now, we
-         * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
-         */
-        p->policy.basic.htt = true;
-        p->policy.extd.cmp_legacy = false;
-
-        /*
-         * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
-         * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
-         * overflow.
-         */
-        if ( !p->policy.basic.lppp )
-            p->policy.basic.lppp = 2;
-        else if ( !(p->policy.basic.lppp & 0x80) )
-            p->policy.basic.lppp *= 2;
-
-        switch ( p->policy.x86_vendor )
+        /* TODO: Expose the ability to choose a custom topology for HVM/PVH */
+        unsigned int threads_per_core = 1;
+        unsigned int cores_per_pkg = di.max_vcpu_id + 1;
+        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);
+        if ( rc )
         {
-        case X86_VENDOR_INTEL:
-            for ( i = 0; (p->policy.cache.subleaf[i].type &&
-                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
-            {
-                p->policy.cache.subleaf[i].cores_per_package =
-                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
-                p->policy.cache.subleaf[i].threads_per_cache = 0;
-            }
-            break;
-
-        case X86_VENDOR_AMD:
-        case X86_VENDOR_HYGON:
-            /*
-             * Leaf 0x80000008 ECX[15:12] is ApicIdCoreSize.
-             * Leaf 0x80000008 ECX[7:0] is NumberOfCores (minus one).
-             * Update to reflect vLAPIC_ID = vCPU_ID * 2.  But avoid
-             * - overflow,
-             * - going out of sync with leaf 1 EBX[23:16],
-             * - incrementing ApicIdCoreSize when it's zero (which changes the
-             *   meaning of bits 7:0).
-             *
-             * UPDATE: I addition to avoiding overflow, some
-             * proprietary operating systems have trouble with
-             * apic_id_size values greater than 7.  Limit the value to
-             * 7 for now.
-             */
-            if ( p->policy.extd.nc < 0x7f )
-            {
-                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
-                    p->policy.extd.apic_id_size++;
-
-                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
-            }
-            break;
+            ERROR("Failed to generate topology: t/c=%u c/p=%u",
+                  threads_per_core, cores_per_pkg);
+            goto out;
         }
     }
 
diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 4b6d96276399..0ad871732ba0 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
 
     p->basic.raw[0x8] = EMPTY_LEAF;
 
-    /* TODO: Rework topology logic. */
-    memset(p->topo.raw, 0, sizeof(p->topo.raw));
-
     p->basic.raw[0xc] = EMPTY_LEAF;
 
     p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
@@ -621,6 +618,9 @@ static void __init calculate_pv_max_policy(void)
     recalculate_xstate(p);
 
     p->extd.raw[0xa] = EMPTY_LEAF; /* No SVM for PV guests. */
+
+    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
 }
 
 static void __init calculate_pv_def_policy(void)
@@ -773,6 +773,9 @@ static void __init calculate_hvm_max_policy(void)
 
     /* It's always possible to emulate CPUID faulting for HVM guests */
     p->platform_info.cpuid_faulting = true;
+
+    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
 }
 
 static void __init calculate_hvm_def_policy(void)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 13:28:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 13:28:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718821.1121352 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hLZ-0000ra-9p; Wed, 08 May 2024 13:28:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718821.1121352; Wed, 08 May 2024 13:28:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hLZ-0000rT-72; Wed, 08 May 2024 13:28:37 +0000
Received: by outflank-mailman (input) for mailman id 718821;
 Wed, 08 May 2024 13:28:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=guPL=ML=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4hLX-0000r7-L9
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 13:28:35 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on0604.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::604])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id de491bb3-0d3e-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 15:28:34 +0200 (CEST)
Received: from AM0PR10CA0110.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:e6::27)
 by DB9PR08MB7697.eurprd08.prod.outlook.com (2603:10a6:10:390::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41; Wed, 8 May
 2024 13:28:28 +0000
Received: from AM3PEPF0000A78E.eurprd04.prod.outlook.com
 (2603:10a6:208:e6:cafe::7e) by AM0PR10CA0110.outlook.office365.com
 (2603:10a6:208:e6::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Wed, 8 May 2024 13:28:27 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF0000A78E.mail.protection.outlook.com (10.167.16.117) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Wed, 8 May 2024 13:28:27 +0000
Received: ("Tessian outbound 9d9bf1c5d85a:v315");
 Wed, 08 May 2024 13:28:27 +0000
Received: from fb0a36088b4e.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F95F9527-53F6-4C55-BD1F-7AF6C9A6DC49.1; 
 Wed, 08 May 2024 13:28:14 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id fb0a36088b4e.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 May 2024 13:28:14 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB8804.eurprd08.prod.outlook.com (2603:10a6:20b:5ba::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Wed, 8 May
 2024 13:28:11 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.041; Wed, 8 May 2024
 13:28:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de491bb3-0d3e-11ef-909c-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=VvK6a8+TDXcEMppnbQzeG7LUrBpw4vq8celIzHam/6GPPXTBF3aqcxD2xgUTFw5g7ncy8l8/2i9YeAcMeoQn0rnLJS2jZE6PdUA/tkCjYoiKJbucb6DTG0LC8a2fxCEAZLkY+zLLShizRdY/AFTFPFHkTymJRqZGXCzfAaAWANgawZ55+GLRnaMsUtCaiSskuhoufkRIaYb2PvqPmYtoERpN+I5204BdKpxObKC7TrmOgX3d+sKmLWIodaOwi8qQVgJkg9EiIPi5zrU19nrZPnlxkmbdd1SoGu0ujYA0ZcYbnvwjmQ4vKPLzHsDVX4Kb8v+l1nACD7gsIKskoggSBg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NFgVM2Oo/y+HZPz/oTIqTdMyExdgg4M61NzSM1xXVEc=;
 b=b7D9PXdpqDRx6OOJyrnjZ4NThrJBn5rYMZ453eGgojhHPkWUnOIfhvVD4HQeqQCC6fzQDA8LMJ0JguFgeKYK6PI2gI5i9mjEn7eYDRoZqJ+4csimvKxGXf2uHvdGn3JUdWJOEXQuWpMLrifN3XbZlX6y2RfyV+2ylU5BrgUcsEfK1qXIIgSFPFCk62kB8yZnjt9pCheIWII2h67elVe6BplejxKHEOvLnJhu7H3mg+gvw398QZ5WTgF8vebDi0YA0cQPH/GtRpkaKXQMluqjD4HtqOl6NNLtGP+gEQf+cr8UodNnoNts1Eao4meDAOPMSXtD836NgxGm+EzCgroYbw==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NFgVM2Oo/y+HZPz/oTIqTdMyExdgg4M61NzSM1xXVEc=;
 b=mBf1abFGJz14DWzegcfqSUgW/cEUQ5yhP7gYc2egF3B6MbZY5gnhtjJB1ARczTNa0S/EJwzpffg50j3tQrQDsZuy3bHAcJIPhRLs9w/jzWafad1lhnha017zJeGx9z+wgc0c/dryKC20mUCt8W9UqHeRxG9FKCwMm/Zh0H3puKE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 09e6c53ee0b268f7
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nq9Lsz59euS5P99FKTTHnZhsVM018WytIhBtBbwYYIIfdqZkWdm1H8e7yVtr5UKwHbpbM+uR0uKjDvXucLon5FeKn/FJTv2OQGIXVz3V1aIIFpZ3+//Q5n6+YV7kEIdy9Dps0+5I2AzXWrVP9hG67zz9ftZAGfHPpOWldNIHIopgRBM26D/cOCBhOr97w7bp/5jagcUqj/ZRpOIKhtaXkDi2P/vVPPXpDi1luw/h2aq4pGtD0Vp77sfsXLoUdwGemNuFu4RqZIoA17Dww+NkIeWN2MoJtpvnY8MAKFMr34ls1o0ZfEiYe8jSQKlw7oyONlE4GFgDpzjYA20lcfo4Ng==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NFgVM2Oo/y+HZPz/oTIqTdMyExdgg4M61NzSM1xXVEc=;
 b=G8ZZUJY9+yqM5LKdkJEfW8S5wo/6QPKlorJJN9u1jb3E8kDLaTwy6uJlwsJDP4cM7L/Fw1WAzd89ZRQwoITtUtDmLjsXZ4z+V8uq2UQExBVrrudTRCmqqGeMmC8o46Rl+MprBH+oPGN9ExzqXFmp/x8/3bxf3da4x7Q7LMz/39maidVRggflDeqo4YMBx5JVLpX34akA2E45UEYAEfzh2fQADlADLsAOePY8PhgbX8o/haDlAMUolLuljOkrkHazIa87hpbin7rir8wMgtG5HFpb4GhMiCojj2/HviWMG3/9vuIYaI7yLghjufehBY27xDR/6DtOW6AluYlzZ/ukMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NFgVM2Oo/y+HZPz/oTIqTdMyExdgg4M61NzSM1xXVEc=;
 b=mBf1abFGJz14DWzegcfqSUgW/cEUQ5yhP7gYc2egF3B6MbZY5gnhtjJB1ARczTNa0S/EJwzpffg50j3tQrQDsZuy3bHAcJIPhRLs9w/jzWafad1lhnha017zJeGx9z+wgc0c/dryKC20mUCt8W9UqHeRxG9FKCwMm/Zh0H3puKE=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 4/7] xen/arm: Parse xen,shared-mem when host phys address
 is not provided
Thread-Topic: [PATCH 4/7] xen/arm: Parse xen,shared-mem when host phys address
 is not provided
Thread-Index: AQHalVfmskMuPI2TlEqCln/x5OWH6LGNVmgAgAAV/wA=
Date: Wed, 8 May 2024 13:28:11 +0000
Message-ID: <A9708869-7739-4306-8527-1A0978D542FF@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-5-luca.fancellu@arm.com>
 <59b02ed2-e543-4bfd-946e-6a82919c582d@amd.com>
In-Reply-To: <59b02ed2-e543-4bfd-946e-6a82919c582d@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB8804:EE_|AM3PEPF0000A78E:EE_|DB9PR08MB7697:EE_
X-MS-Office365-Filtering-Correlation-Id: 0b6d33fa-b1a0-458b-5bdf-08dc6f62beae
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?QlZmOU11cFpNV09KRkhtbEZMY05oQ2VKOElqeGU1RG5kajkyV2RsMnM0MHIr?=
 =?utf-8?B?NTJYMHlVcHhmcDNVb0ovQ3RyaTUxazY5ZWM1U2UwUjhFelQ5WGJkb3pqbVN2?=
 =?utf-8?B?KzkwdWw4dy9DWUR1bDRyU0RHamdPb0VoYmlKUUJjWXQvQ1Z0Z2ptMkt0dmZ0?=
 =?utf-8?B?Tk1zV3Yyc241R1J3cE00QUVRL01WS242VTROakgzY3p4eTVpWjhUUHZkN1Bh?=
 =?utf-8?B?SmRBSHpTYW9mZjhmTzR5OVk3RkVzNEdpc2JzVUFzZnhPOFRGUE9nVk1WYndq?=
 =?utf-8?B?UkM2RTFOaEk1Ni9nOXA0OUtyMjY3YW1sUlBhdTI5RjB5WXBuRksxUXczTHZF?=
 =?utf-8?B?VlVUbEJBUTV4N0xjM1VtckRTWUZYaW9BYTFZTGJta1VrcGkzU0c3c2pFV3U4?=
 =?utf-8?B?WTFRd1FNQkZLQytiWG50dDVtMGM3Y3RXdE5leDRjYjNiZWRVTW96ZkRrMWk0?=
 =?utf-8?B?OTBDTEFZbGlFc2M2QmtoRXNYZVBvV1VVQllJWEZoc2pkTmVIbW1VYnBZTVRE?=
 =?utf-8?B?VEgrUkpWSkxlOExLbVFBaVJqemFteGs4RUxjR3BIV3JnN3JmRkgySEpRbDFO?=
 =?utf-8?B?b09aTkR3UWVYY1JnTStWaXZYZGlXa0huaU5tWkJiUDJjdEQ0QzZuWE5lNDY5?=
 =?utf-8?B?cGlwWVQwMlBlUlZobkk1WGNGTDdjU2ZmWTV3aW5FWXdVWjhWaXNRNkV5eks0?=
 =?utf-8?B?M3ZjTDlmMkJyb3h5TEtZbXdWRUU5QmR2YkhVd3IzcWozYmsxWEMzNDhpZlpZ?=
 =?utf-8?B?T1VFMGMzdjhYUXVmSVZEWFh4bzN4aW1GTng0SnVrWkt4MzVvdGpKVDdGSWpi?=
 =?utf-8?B?a2ovQVBkSDByQnp3ZkZoZ2pCQjhYdmtKaWdmQzExc0RMTndrazRXdnV4ZlJF?=
 =?utf-8?B?bGo5SEw3SmVyaEVoN0xZY3lnYlpXL1VyNUlBY29qTWFtQWp3Sklac0QxOVFF?=
 =?utf-8?B?cVVlV0N0OHpNd3N0Mm9pN2UxMUJ3VkVzNDd6bnNIZlF4ZG85bTM1S0o5VWpk?=
 =?utf-8?B?cENSRXRHUVpCZjQ0eGtmcThLNVZkZnpEYWtuVnJZSXFuWk9vNEFEKzBWbWRG?=
 =?utf-8?B?bjhKOGlOaXY5TXRrWFVNZWt3RTBmV1dtSUxSUnlRUldwTWlmZExrV3VEYndN?=
 =?utf-8?B?QUIvbUpiSGQvVzJNbkFEQkprSmZsNEVZMTRwa3ZoYm9zMmJKUnVudk81NlV3?=
 =?utf-8?B?Zm1peHJkbHd2RnREMTBVZHNlcmhZRkhHT29tMnU5d3loVWwvdVduSFBaSC9O?=
 =?utf-8?B?bW9XK054Zi92RjZsWnZtTzNYdVNTZ1hCalBEazZPVklJTGxOUkZteDRFendk?=
 =?utf-8?B?c3RRNlhSbWp1OFQySTBVNlQzRURyQ1ZoQmxQYTJBREZQUlkvSkcxODdyek9s?=
 =?utf-8?B?eGxNUkRYOXhXbkhHQTExUlFPVHc3bzFEcFhERm1wTnZITXkyalZwMVF6NFdq?=
 =?utf-8?B?ZUdsbUU5TGNHejFGUngxaUtubi9GRmVjUTlvOFdCNFZJYWhLSWVOZm1GclRM?=
 =?utf-8?B?Y014eUUwcnRBYVZ4aEc4alA2RXcrNlcxYXY3NEdub3N4bDRUYXZTb0wwM294?=
 =?utf-8?B?OERzbHRXMTdjQjBMdUJUeFlaeDYwNTlCc1ZyVStZWU5zdEdFSGozV2pGSGVV?=
 =?utf-8?B?TCtPMk82MmthL1ErNk8zeGh2YTFsNEl5SnpGcG1mUis1ek1sREsyMU9INEFv?=
 =?utf-8?B?R1dBemRuTlFvRElmTzV2YnU5ZUI5NENNN2VMa0hyL0traXNEOFpvanpVMVph?=
 =?utf-8?B?dU5Ta3dkQmVQbnk3TUZRNVdiaGNSMmZIeU1rZFBJMUhWYVpOcU5HUzBrUkhG?=
 =?utf-8?B?eE1RUHNOdGhvcG1xcm9TUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <BB98CCD80B3929468311EC7AC7ADA1DD@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8804
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF0000A78E.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c9bd4288-bd05-426a-4321-08dc6f62b533
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|35042699013|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RjhCejFsTGNXLzZVZ3RVNjl2TjdMdzJxTjcwakkvSGJNdk8zd1lyNTVDTE93?=
 =?utf-8?B?U3VxRldHZmEyR0wrbjlEeERkNW96Q3V6cy9zb2N2QVdRNHh3ZVRFTWE4Zjds?=
 =?utf-8?B?cExkRGhGMmlYYm4yNit2WWNCb2dqZXlTWHFnNFpTdXlJZjJyUWNLaGdMbXVB?=
 =?utf-8?B?LzhaKy9Mam0zSGVQUWROcmRsNmoxM3VuWmtZVWs5em1WNTBvbFRBT2RjOW5B?=
 =?utf-8?B?c1JEclo0enE2Mk1LTFgvWjhSLzArVE15OFdWK0JDbEcyT1NXRldOY3Z4Uk9t?=
 =?utf-8?B?am5pUFJsYlgydDBHY1ROQXZGZS9yOG5lZzRwL2oxa1pHZ1pFMlV1U0YyRWdj?=
 =?utf-8?B?YVlRUFE5bUg5VytsSkQwT3owWGZaT3k4bjVUaDJ3MG1qcHlLMEw0L1lRZWZR?=
 =?utf-8?B?ajdBNlZjTUx4dFMxQ1JTTUFkQXNUemJUc2VaNTNtWmduZDEyWHV3aFUyMTVC?=
 =?utf-8?B?Z0l5OVA3UFN1ZU9sY3NOYVRTcENVVzZhdXVwUk9ZRlFaM2pRMExkQllIaTBl?=
 =?utf-8?B?REFDSVpZUUhHemhzWWoxSXNoeStKNmJ2YXBKZEVaL1lhOEgwVHZ1UHp6NXJt?=
 =?utf-8?B?R1B2VzE3cjB6OThIZGZqSG1nNVNhNmFJMFNhWTNrZFA5NDNLNnM1VmgrMkxi?=
 =?utf-8?B?Yzh6Q2JiczR5bGhmZThKcks5WVR5Q000Z0RtM05zWWRicjU2TWQ2Y2NJUzhE?=
 =?utf-8?B?eWhxRzJ5Zk8zVVJzS3dmNElia0JpRTJKR2RqODgvQkRiWlhnVXkrSVJQUjRG?=
 =?utf-8?B?bVR5RjUyZDB0aGpQT0h5bW5EWXM5MEFWZlEwQUN2TmcxMjJiSE1YZ3c4K3gy?=
 =?utf-8?B?czlXN0JMM002bk1TVnB6dmlnZmJLTXJZdEh0WTlsaVlBMlhzK2NtR1JWM0Jn?=
 =?utf-8?B?OW00ek50Ulp3ZVkxU2lGa2QvbVNpUHQydVAzU1pPaGQzc1pBZkRXYTJqZ0t1?=
 =?utf-8?B?RUpiVURkaU5qVzRTK0J2OGVFSld3NWVaN0wzL05JV0tDVzFWR0tmcVd2N3FF?=
 =?utf-8?B?YUVwUlVNR2JuNkdKVGxxM3BkbDRsempITWFkS2Z6bS8rdEtqd0NRSEpnemxG?=
 =?utf-8?B?Z3d5UnZKWmFjcVVXUVdkblMvVk5pR010cnloMU51dTdHV0JObDFwZFArZVh0?=
 =?utf-8?B?SFBENVorUEV6eEkySWhXd0JYeXR5eXdjMGNxbEtVYStSTDBKZkN4eDljblJz?=
 =?utf-8?B?blN1MmdqdzhlY3hZRk4veUpVTDB6aFdtWVBuV0M3aEtoTmdjSTBDSFVvNlpM?=
 =?utf-8?B?QmxLUEJmVy8vMnVzbVAwQ2dmYW1ZVEhBakRML0VxK0QvU0JuRXFDZ2tTakp5?=
 =?utf-8?B?d1JQZFdQVE9kNGZjbUsrUWNXeFIrSjlrWm91akhiaTdOUkJwV0tKYlAreEIr?=
 =?utf-8?B?MGx5TmxXVm5ZZlZvTHh5WUJSR3p4ZUNwZXNVREtQaU5wZm10OFhIMTJpc3Rv?=
 =?utf-8?B?WGVLRlJ3YWlFRXZhcWhLTzUxcVMxVldHK2tneHlCNlFHMldYMDdFeml2c0ZT?=
 =?utf-8?B?TW9UTEprVlRWNlAvbnE0Zmp5cnJsaUxTeG5EYzFHS2s1cUtWTEtLV25nZWUr?=
 =?utf-8?B?Y280K2hkbENiVUN5SUx1V0FjZ252Q2syRmdVUWdLbnNpU3ZNSEUwRGJobGh2?=
 =?utf-8?B?TVI1M1BNNnB0NDZObzBZamhDRDV2NldaY3ROV0gxTk9JZm12V1dCL2s1T2VQ?=
 =?utf-8?B?Ly9aZXduME50V0pOUWc5RnJaMWxwVzluM1p1ZGhiRjF3aFBieE42U1c3WXRL?=
 =?utf-8?B?UmJGazg0cGppWjdSdU9LNU1TMmFPMGZmeERseXhJUjZBWldSNWVTdWdkWHow?=
 =?utf-8?B?V0kxMS9kKzlMWTl5Q3Ayc2xJUk9sakVmZzA2U05ORUt3QWpUeWVOdis5M0pH?=
 =?utf-8?Q?NTrnwcRAp6+Ps?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(35042699013)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 13:28:27.5718
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0b6d33fa-b1a0-458b-5bdf-08dc6f62beae
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF0000A78E.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7697

SGkgTWljaGFsLA0KDQo+IE9uIDggTWF5IDIwMjQsIGF0IDEzOjA5LCBNaWNoYWwgT3J6ZWwgPG1p
Y2hhbC5vcnplbEBhbWQuY29tPiB3cm90ZToNCj4gDQo+IEhpIEx1Y2EsDQo+IA0KPiBPbiAyMy8w
NC8yMDI0IDEwOjI1LCBMdWNhIEZhbmNlbGx1IHdyb3RlOg0KPj4gDQo+PiANCj4+IEhhbmRsZSB0
aGUgcGFyc2luZyBvZiB0aGUgJ3hlbixzaGFyZWQtbWVtJyBwcm9wZXJ0eSB3aGVuIHRoZSBob3N0
IHBoeXNpY2FsDQo+PiBhZGRyZXNzIGlzIG5vdCBwcm92aWRlZCwgdGhpcyBjb21taXQgaXMgaW50
cm9kdWNpbmcgdGhlIGxvZ2ljIHRvIHBhcnNlIGl0LA0KPj4gYnV0IHRoZSBmdW5jdGlvbmFsaXR5
IGlzIHN0aWxsIG5vdCBpbXBsZW1lbnRlZCBhbmQgd2lsbCBiZSBwYXJ0IG9mIGZ1dHVyZQ0KPj4g
Y29tbWl0cy4NCj4+IA0KPj4gUmV3b3JrIHRoZSBsb2dpYyBpbnNpZGUgcHJvY2Vzc19zaG1fbm9k
ZSB0byBjaGVjayB0aGUgc2htX2lkIGJlZm9yZSBkb2luZw0KPj4gdGhlIG90aGVyIGNoZWNrcywg
YmVjYXVzZSBpdCBlYXNlIHRoZSBsb2dpYyBpdHNlbGYsIGFkZCBtb3JlIGNvbW1lbnQgb24NCj4+
IHRoZSBsb2dpYy4NCj4+IE5vdyB3aGVuIHRoZSBob3N0IHBoeXNpY2FsIGFkZHJlc3MgaXMgbm90
IHByb3ZpZGVkLCB0aGUgdmFsdWUNCj4+IElOVkFMSURfUEFERFIgaXMgY2hvc2VuIHRvIHNpZ25h
bCB0aGlzIGNvbmRpdGlvbiBhbmQgaXQgaXMgc3RvcmVkIGFzDQo+PiBzdGFydCBvZiB0aGUgYmFu
aywgZHVlIHRvIHRoYXQgY2hhbmdlIGFsc28gZWFybHlfcHJpbnRfaW5mb19zaG1lbSBhbmQNCj4+
IGluaXRfc2hhcmVkbWVtX3BhZ2VzIGFyZSBjaGFuZ2VkLCB0byBkb24ndCBoYW5kbGUgYmFua3Mg
d2l0aCBzdGFydCBlcXVhbA0KPj4gdG8gSU5WQUxJRF9QQUREUi4NCj4+IA0KPj4gQW5vdGhlciBj
aGFuZ2UgaXMgZG9uZSBpbnNpZGUgbWVtaW5mb19vdmVybGFwX2NoZWNrLCB0byBza2lwIGJhbmtz
IHRoYXQNCj4+IGFyZSBzdGFydGluZyB3aXRoIHRoZSBzdGFydCBhZGRyZXNzIElOVkFMSURfUEFE
RFIsIHRoYXQgZnVuY3Rpb24gaXMgdXNlZA0KPj4gdG8gY2hlY2sgYmFua3MgZnJvbSByZXNlcnZl
ZCBtZW1vcnkgYW5kIEFDUEkgYW5kIGl0J3MgdW5saWtlbHkgZm9yIHRoZXNlDQo+IGFsc28gZnJv
bSBzaG1lbQ0KPiANCj4+IGJhbmsgdG8gaGF2ZSB0aGUgc3RhcnQgYWRkcmVzcyBhcyBJTlZBTElE
X1BBRERSLiBUaGUgY2hhbmdlIGhvbGRzDQo+PiBiZWNhdXNlIG9mIHRoaXMgY29uc2lkZXJhdGlv
bi4NCj4gT24gYXJtNjQgYW5kIExQQUUgYXJtMzIgd2UgZG9uJ3QgaGF2ZSB0aGlzIHByb2JsZW0u
IEluIHRoZW9yeSB3ZSBjb3VsZCBoYXZlIGEgYmFuaw0KPiBzdGFydGluZyBhdCBJTlZBTElEX1BB
RERSIGlmIFBBIHJhbmdlIHdhcyAzMmJpdCBidXQgYXMgdGhlIGNvbW1lbnQgYWJvdmUgdGhlIGZ1
bmN0aW9uIHN0YXRlcywNCj4gd3JhcHBpbmcgYXJvdW5kIGlzIG5vdCBoYW5kbGVkLiBZb3UgbWln
aHQgd2FudCB0byB1c2UgaXQgYXMgYSBqdXN0aWZpY2F0aW9uIHRvIGJlIGNsZWFyLg0KDQpTdXJl
LCBJ4oCZbGwgcmVwaHJhc2UgaXQsIGlzIGl0IG9rIHNvbWV0aGluZyBsaWtlIHRoaXM6DQoNClsu
Li5dDQpBbm90aGVyIGNoYW5nZSBpcyBkb25lIGluc2lkZSBtZW1pbmZvX292ZXJsYXBfY2hlY2ss
IHRvIHNraXAgYmFua3MgdGhhdA0KYXJlIHN0YXJ0aW5nIHdpdGggdGhlIHN0YXJ0IGFkZHJlc3Mg
SU5WQUxJRF9QQUREUiwgdGhhdCBmdW5jdGlvbiBpcyB1c2VkDQp0byBjaGVjayBiYW5rcyBmcm9t
IHJlc2VydmVkIG1lbW9yeSwgc2hhcmVkIG1lbW9yeSBhbmQgQUNQSSBhbmQgc2luY2UNCnRoZSBj
b21tZW50IGFib3ZlIHRoZSBmdW5jdGlvbiBzdGF0ZXMgdGhhdCB3cmFwcGluZyBhcm91bmQgaXMg
bm90IGhhbmRsZWQsDQppdOKAmXMgdW5saWtlbHkgZm9yIHRoZXNlIGJhbmsgdG8gaGF2ZSB0aGUg
c3RhcnQgYWRkcmVzcyBhcyBJTlZBTElEX1BBRERSLg0KVGhlIGNoYW5nZSBob2xkcyBiZWNhdXNl
IG9mIHRoaXMgY29uc2lkZXJhdGlvbi4NCg0KPj4gDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gv
YXJtL3N0YXRpYy1zaG1lbS5jIGIveGVuL2FyY2gvYXJtL3N0YXRpYy1zaG1lbS5jDQo+PiBpbmRl
eCAyNGU0MDQ5NWE0ODEuLjFjMDNiYjdmMTg4MiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2Fy
bS9zdGF0aWMtc2htZW0uYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3N0YXRpYy1zaG1lbS5jDQo+
PiBAQCAtMjY0LDYgKzI2NCwxMiBAQCBpbnQgX19pbml0IHByb2Nlc3Nfc2htKHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBrZXJuZWxfaW5mbyAqa2luZm8sDQo+PiAgICAgICAgIHBiYXNlID0gYm9v
dF9zaG1fYmFuay0+c3RhcnQ7DQo+PiAgICAgICAgIHBzaXplID0gYm9vdF9zaG1fYmFuay0+c2l6
ZTsNCj4+IA0KPj4gKyAgICAgICAgaWYgKCBJTlZBTElEX1BBRERSID09IHBiYXNlICkNCj4+ICsg
ICAgICAgIHsNCj4+ICsgICAgICAgICAgICBwcmludGsoIiVwZDogaG9zdCBwaHlzaWNhbCBhZGRy
ZXNzIG11c3QgYmUgY2hvc2VuIGJ5IHVzZXJzIGF0IHRoZSBtb21lbnQuIiwgZCk7DQo+IFRoZSBk
b3QgYXQgdGhlIGVuZCBpcyBub3QgbmVlZGVkLg0KV2lsbCBmaXgNCg0KDQo+PiANCj4+IC0gICAg
ZW5kID0gcGFkZHIgKyBzaXplOw0KPj4gLSAgICBpZiAoIGVuZCA8PSBwYWRkciApDQo+PiAtICAg
IHsNCj4+IC0gICAgICAgIHByaW50aygiZmR0OiBzdGF0aWMgc2hhcmVkIG1lbW9yeSByZWdpb24g
JXMgb3ZlcmZsb3dcbiIsIHNobV9pZCk7DQo+PiAtICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4+
IC0gICAgfQ0KPj4gLQ0KPj4gICAgIGZvciAoIGkgPSAwOyBpIDwgbWVtLT5ucl9iYW5rczsgaSsr
ICkNCj4+ICAgICB7DQo+PiAgICAgICAgIC8qDQo+PiAgICAgICAgICAqIE1lZXQgdGhlIGZvbGxv
d2luZyBjaGVjazoNCj4+ICsgICAgICAgICAqIHdoZW4gaG9zdCBhZGRyZXNzIGlzIHByb3ZpZGVk
Og0KPiAtIHdoZW4gd291bGQgcmVhZCBiZXR0ZXINCk9rIEnigJlsbCB1c2UgaHlwaGVuIGluc3Rl
YWQgb2Ygc3RhciwgaGVyZSBhbmQgYmVsb3cNCg0KPiANCj4+ICAgICAgICAgICogMSkgVGhlIHNo
bSBJRCBtYXRjaGVzIGFuZCB0aGUgcmVnaW9uIGV4YWN0bHkgbWF0Y2gNCj4+ICAgICAgICAgICog
MikgVGhlIHNobSBJRCBkb2Vzbid0IG1hdGNoIGFuZCB0aGUgcmVnaW9uIGRvZXNuJ3Qgb3Zlcmxh
cA0KPj4gICAgICAgICAgKiB3aXRoIGFuIGV4aXN0aW5nIG9uZQ0KPj4gKyAgICAgICAgICogd2hl
biBob3N0IGFkZHJlc3MgaXMgbm90IHByb3ZpZGVkOg0KPj4gKyAgICAgICAgICogMSkgVGhlIHNo
bSBJRCBtYXRjaGVzIGFuZCB0aGUgcmVnaW9uIHNpemUgZXhhY3RseSBtYXRjaA0KPj4gICAgICAg
ICAgKi8NCj4+IC0gICAgICAgIGlmICggcGFkZHIgPT0gbWVtLT5iYW5rW2ldLnN0YXJ0ICYmIHNp
emUgPT0gbWVtLT5iYW5rW2ldLnNpemUgKQ0KPj4gKyAgICAgICAgYm9vbCBwYWRkcl9hc3NpZ25l
ZCA9IElOVkFMSURfUEFERFIgPT0gcGFkZHI7DQo+IHBhcmVudGhlc2lzIGFyb3VuZCBJTlZBTElE
X1BBRERSID09IHBhZGRyDQpPaw0KDQo+IA0KPj4gKyAgICAgICAgYm9vbCBzaG1faWRfbWF0Y2gg
PSBzdHJuY21wKHNobV9pZCwgc2htZW1fZXh0cmFbaV0uc2htX2lkLA0KPj4gKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIE1BWF9TSE1fSURfTEVOR1RIKSA9PSAwOw0KPiB3aHkg
bm90IGlmICggc3RybmNtcC4uLiBnaXZlbiBubyBvdGhlciB1c2Ugb2YgdGhpcyB2YXJpYWJsZSBv
dGhlciB0aGFuIHRoZSBvbmUgYmVsb3c/DQoNClllYWggSSB0aGluayBpbiBzb21lIHByZXZpb3Vz
IHJld29yayBJIHdhcyB1c2luZyBtdWx0aXBsZSB0aW1lcyBhbmQgdGhlbiBJIGZvcmdvdCB0bw0K
Y2hhbmdlIGhlcmUsIEnigJlsbCBmaXgNCg0KPiANCj4+ICsgICAgICAgIGlmICggc2htX2lkX21h
dGNoICkNCj4+ICAgICAgICAgew0KPj4gLSAgICAgICAgICAgIGlmICggc3RybmNtcChzaG1faWQs
IHNobWVtX2V4dHJhW2ldLnNobV9pZCwNCj4+IC0gICAgICAgICAgICAgICAgICAgICAgICAgTUFY
X1NITV9JRF9MRU5HVEgpID09IDAgICkNCj4+ICsgICAgICAgICAgICAvKg0KPj4gKyAgICAgICAg
ICAgICAqIFJlZ2lvbnMgaGF2ZSBzYW1lIHNobV9pZCAoY2FzZXMpOg0KPj4gKyAgICAgICAgICAg
ICAqIDEpIHBoeXNpY2FsIGhvc3QgYWRkcmVzcyBpcyBzdXBwbGllZDoNCj4+ICsgICAgICAgICAg
ICAgKiAgICAtIE9LOiAgIHBhZGRyIGlzIGVxdWFsIGFuZCBzaXplIGlzIGVxdWFsIChzYW1lIHJl
Z2lvbikNCj4+ICsgICAgICAgICAgICAgKiAgICAtIEZhaWw6IHBhZGRyIGRvZXNuJ3QgbWF0Y2gg
b3Igc2l6ZSBkb2Vzbid0IG1hdGNoICh0aGVyZQ0KPj4gKyAgICAgICAgICAgICAqICAgICAgICAg
ICAgY2Fubm90IGV4aXN0cyB0d28gc2htZW0gcmVnaW9ucyB3aXRoIHNhbWUgc2htX2lkKQ0KPj4g
KyAgICAgICAgICAgICAqIDIpIHBoeXNpY2FsIGhvc3QgYWRkcmVzcyBpcyBOT1Qgc3VwcGxpZWQ6
DQo+PiArICAgICAgICAgICAgICogICAgLSBPSzogICBzaXplIGlzIGVxdWFsIChzYW1lIHJlZ2lv
bikNCj4+ICsgICAgICAgICAgICAgKiAgICAtIEZhaWw6IHNpemUgaXMgbm90IGVxdWFsIChzYW1l
IHNobV9pZCBtdXN0IGlkZW50aWZ5IG9ubHkgb25lDQo+PiArICAgICAgICAgICAgICogICAgICAg
ICAgICByZWdpb24sIHRoZXJlIGNhbid0IGJlIHR3byBkaWZmZXJlbnQgcmVnaW9ucyB3aXRoIHNh
bWUNCj4+ICsgICAgICAgICAgICAgKiAgICAgICAgICAgIHNobV9pZCkNCj4+ICsgICAgICAgICAg
ICAgKi8NCj4+ICsgICAgICAgICAgICBib29sIHN0YXJ0X21hdGNoID0gcGFkZHJfYXNzaWduZWQg
PyAocGFkZHIgPT0gbWVtLT5iYW5rW2ldLnN0YXJ0KSA6DQo+PiArICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZTsNCj4+ICsNCj4+ICsgICAgICAgICAg
ICBpZiAoIHN0YXJ0X21hdGNoICYmIHNpemUgPT0gbWVtLT5iYW5rW2ldLnNpemUgKQ0KPj4gICAg
ICAgICAgICAgICAgIGJyZWFrOw0KPj4gICAgICAgICAgICAgZWxzZQ0KPj4gICAgICAgICAgICAg
ew0KPj4gLSAgICAgICAgICAgICAgICBwcmludGsoImZkdDogeGVuLHNobS1pZCAlcyBkb2VzIG5v
dCBtYXRjaCBmb3IgYWxsIHRoZSBub2RlcyB1c2luZyB0aGUgc2FtZSByZWdpb24uXG4iLA0KPj4g
KyAgICAgICAgICAgICAgICBwcmludGsoImZkdDogZGlmZmVyZW50IHNoYXJlZCBtZW1vcnkgcmVn
aW9uIGNvdWxkIG5vdCBzaGFyZSB0aGUgc2FtZSBzaG0gSUQgJXNcbiIsDQo+PiAgICAgICAgICAg
ICAgICAgICAgICAgIHNobV9pZCk7DQo+PiAgICAgICAgICAgICAgICAgcmV0dXJuIC1FSU5WQUw7
DQo+PiAgICAgICAgICAgICB9DQo+PiAgICAgICAgIH0NCj4+IC0gICAgICAgIGVsc2UgaWYgKCBz
dHJuY21wKHNobV9pZCwgc2htZW1fZXh0cmFbaV0uc2htX2lkLA0KPj4gLSAgICAgICAgICAgICAg
ICAgICAgICAgICAgTUFYX1NITV9JRF9MRU5HVEgpICE9IDAgKQ0KPj4gLSAgICAgICAgICAgIGNv
bnRpbnVlOw0KPj4gICAgICAgICBlbHNlDQo+PiAgICAgICAgIHsNCj4gVGhlcmUgaXMgbm8gbmVl
ZCBmb3IgdGhpcyBlbHNlIGFuZCBlbnRpcmUgYmxvY2sgZ2l2ZW4gdGhhdCB0aGUgYmxvY2sgd2l0
aGluIGlmIGVpdGhlciBjYWxscyBicmVhayBvciByZXR1cm4NCg0KVGhlcmUgd2FzIGEgTUlTUkEg
ZGlzY3Vzc2lvbiBhYm91dCBlbHNlIGF0IHRoZSBlbmQgb2YgaWYgLi4uIGVsc2UgaWYgLi4uIChS
MTUuNykgYW5kIEkgZG9u4oCZdCByZW1lbWJlcg0KdGhlIG91dGNvbWUNCj4+IA0KPj4gQEAgLTU4
OCwxMyArNjM3LDE2IEBAIHZvaWQgX19pbml0IGVhcmx5X3ByaW50X2luZm9fc2htZW0odm9pZCkN
Cj4+IHsNCj4+ICAgICBjb25zdCBzdHJ1Y3QgbWVtYmFua3MgKnNobWVtID0gYm9vdGluZm9fZ2V0
X3NobWVtKCk7DQo+PiAgICAgdW5zaWduZWQgaW50IGJhbms7DQo+PiArICAgIHVuc2lnbmVkIGlu
dCBwcmludGVkID0gMDsNCj4+IA0KPj4gICAgIGZvciAoIGJhbmsgPSAwOyBiYW5rIDwgc2htZW0t
Pm5yX2JhbmtzOyBiYW5rKysgKQ0KPj4gLSAgICB7DQo+PiAtICAgICAgICBwcmludGsoIiBTSE1F
TVsldV06ICUiUFJJcGFkZHIiIC0gJSJQUklwYWRkciJcbiIsIGJhbmssDQo+PiAtICAgICAgICAg
ICAgICAgc2htZW0tPmJhbmtbYmFua10uc3RhcnQsDQo+PiAtICAgICAgICAgICAgICAgc2htZW0t
PmJhbmtbYmFua10uc3RhcnQgKyBzaG1lbS0+YmFua1tiYW5rXS5zaXplIC0gMSk7DQo+PiAtICAg
IH0NCj4+ICsgICAgICAgIGlmICggc2htZW0tPmJhbmtbYmFua10uc3RhcnQgIT0gSU5WQUxJRF9Q
QUREUiApDQo+PiArICAgICAgICB7DQo+PiArICAgICAgICAgICAgcHJpbnRrKCIgU0hNRU1bJXVd
OiAlIlBSSXBhZGRyIiAtICUiUFJJcGFkZHIiXG4iLCBwcmludGVkLA0KPj4gKyAgICAgICAgICAg
ICAgICBzaG1lbS0+YmFua1tiYW5rXS5zdGFydCwNCj4+ICsgICAgICAgICAgICAgICAgc2htZW0t
PmJhbmtbYmFua10uc3RhcnQgKyBzaG1lbS0+YmFua1tiYW5rXS5zaXplIC0gMSk7DQo+PiArICAg
ICAgICAgICAgcHJpbnRlZCsrOw0KPiBOSVQ6IHlvdSBjb3VsZCBpbml0aWFsaXplIGFuZCBpbmNy
ZW1lbnQgaXQgYXMgcGFydCBvZiB0aGUgZm9yIGxvb3ANClN1cmUsIEnigJlsbCBkbw0KDQo=


From xen-devel-bounces@lists.xenproject.org Wed May 08 13:43:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 13:43:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718827.1121361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hZm-00043x-H5; Wed, 08 May 2024 13:43:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718827.1121361; Wed, 08 May 2024 13:43:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hZm-00043q-EJ; Wed, 08 May 2024 13:43:18 +0000
Received: by outflank-mailman (input) for mailman id 718827;
 Wed, 08 May 2024 13:43:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4hZl-00043g-KG; Wed, 08 May 2024 13:43:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4hZl-0000f7-Gt; Wed, 08 May 2024 13:43:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4hZl-0003Eg-76; Wed, 08 May 2024 13:43:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4hZl-0003TX-6e; Wed, 08 May 2024 13:43:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uXOu81C2KSAKZNz083heZlrgKfKGv/Wl92AvwM1g40I=; b=Hy48z/+NfnvVk3nPFDg+QIrw9z
	uFZlKHFPGIspErmvN/uLGCOy7vxfxJ5eySPXJPuEA/UaaCr5OYbzoU1Mm62w4dMVR2AAcXCOT5N/Q
	H/odiv2FirVI070v3ooxTES7GVbrzv99oBw3dMqVZF1PKBYaxIDQ33UFU4/qIE/7tByw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185946-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185946: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=b82c9631da39ca5a1f0702185a46fea60446dd0a
X-Osstest-Versions-That:
    ovmf=5cbfb93abea4410be396b5230da4ad2f6c972788
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 May 2024 13:43:17 +0000

flight 185946 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185946/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 b82c9631da39ca5a1f0702185a46fea60446dd0a
baseline version:
 ovmf                 5cbfb93abea4410be396b5230da4ad2f6c972788

Last test of basis   185945  2024-05-08 08:44:38 Z    0 days
Testing same since   185946  2024-05-08 11:45:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Sachin Ganesh <sachinganesh@ami.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   5cbfb93abe..b82c9631da  b82c9631da39ca5a1f0702185a46fea60446dd0a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 08 14:02:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 14:02:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718841.1121372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hsT-0007xq-3W; Wed, 08 May 2024 14:02:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718841.1121372; Wed, 08 May 2024 14:02:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4hsS-0007xj-Vs; Wed, 08 May 2024 14:02:36 +0000
Received: by outflank-mailman (input) for mailman id 718841;
 Wed, 08 May 2024 14:02:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TYmE=ML=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s4hsS-0007xN-4R
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 14:02:36 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9de50042-0d43-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 16:02:33 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id F31EF4EE0752;
 Wed,  8 May 2024 16:02:32 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9de50042-0d43-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Wed, 08 May 2024 16:02:32 +0200
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, Tamas K Lengyel
 <tamas@tklengyel.com>, Alexandru Isaila <aisaila@bitdefender.com>, Petre
 Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com,
 volodymyr_babchuk@epam.com
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
Reply-To: alessandro.zucchelli@bugseng.com
Mail-Reply-To: alessandro.zucchelli@bugseng.com
In-Reply-To: <47033435-c621-40f6-b5a9-a385f323f382@xen.org>
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com>
 <834460aca34e6167a3c4620616c145b8@bugseng.com>
 <47033435-c621-40f6-b5a9-a385f323f382@xen.org>
Message-ID: <5b84019c4e0111b3cc5e369b999fbfab@bugseng.com>
X-Sender: alessandro.zucchelli@bugseng.com
Organization: BUGSENG Srl
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-05-03 11:32, Julien Grall wrote:
> Hi,
> 
> On 03/05/2024 08:09, Alessandro Zucchelli wrote:
>> On 2024-04-29 17:58, Jan Beulich wrote:
>>> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
>>>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
>>>> allowing asm/mem_access.h to be included in all ARM build 
>>>> configurations.
>>>> This is to address the violation of MISRA C: 2012 Rule 8.4 which 
>>>> states:
>>>> "A compatible declaration shall be visible when an object or 
>>>> function
>>>> with external linkage is defined". Functions p2m_mem_access_check
>>>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is not
>>>> defined in ARM builds don't have visible declarations in the file
>>>> containing their definitions.
>>>> 
>>>> Signed-off-by: Alessandro Zucchelli 
>>>> <alessandro.zucchelli@bugseng.com>
>>>> ---
>>>>  xen/include/xen/mem_access.h | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>> 
>>>> diff --git a/xen/include/xen/mem_access.h 
>>>> b/xen/include/xen/mem_access.h
>>>> index 87d93b31f6..ec0630677d 100644
>>>> --- a/xen/include/xen/mem_access.h
>>>> +++ b/xen/include/xen/mem_access.h
>>>> @@ -33,7 +33,7 @@
>>>>   */
>>>>  struct vm_event_st;
>>>> 
>>>> -#ifdef CONFIG_MEM_ACCESS
>>>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
>>>>  #include <asm/mem_access.h>
>>>>  #endif
>>> 
>>> This doesn't look quite right. If Arm supports mem-access, why would 
>>> it
>>> not set MEM_ACCESS=y? Whereas if it's only stubs that Arm supplies, 
>>> then
>>> those would better move here, thus eliminating the need for a 
>>> per-arch
>>> stub header (see what was e.g. done for numa.h). This way RISC-V and 
>>> PPC
>>> (and whatever is to come) would then be taken care of as well.
>>> 
>> ARM does support mem-access, so I don't think this is akin to the 
>> changes done to handle numa.h.
>> ARM also allows users to set MEM_ACCESS=n (e.g. 
>> xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however, 
>> the implementation file mem_access.c is compiled unconditionally in 
>> ARM's makefile, hence why the violation was spotted.
>> This is a bit unusual, so I was also hoping to get some feedback from 
>> mem-access maintainers as to why this discrepancy from x86 exists. I 
>> probably should have also included some ARM maintainers as well, so 
>> I'm going to loop them in now.
>> 
>> An alternative option I think is to make the compilation of arm's 
>> mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and 
>> common).
> 
> I can't think of a reason to have mem_access.c unconditional compiled 
> in. So I think it should be conditional like on x86.

Hi,
attempting to build ARM with a configuration where MEM_ACCESS=n and 
mem_access.c is conditioned on CONFIG_MEM_ACCESS results in a fail as 
there are other pieces of code that call some mem_access.c functions 
(p2m_mem_access_check_and_get_page, p2m_mem_access_check).
In a Matrix chat Julien was suggesting adding stubs for the functions 
for this use case.
-- 
Alessandro Zucchelli, B.Sc.

Software Engineer, BUGSENG (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed May 08 15:27:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 15:27:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718849.1121382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jCK-000325-6P; Wed, 08 May 2024 15:27:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718849.1121382; Wed, 08 May 2024 15:27:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jCK-00031y-2Z; Wed, 08 May 2024 15:27:12 +0000
Received: by outflank-mailman (input) for mailman id 718849;
 Wed, 08 May 2024 15:27:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dFtT=ML=linux.intel.com=andriy.shevchenko@srs-se1.protection.inumbo.net>)
 id 1s4jCI-00031s-Kz
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 15:27:11 +0000
Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6da09837-0d4f-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 17:27:08 +0200 (CEST)
Received: from fmviesa008.fm.intel.com ([10.60.135.148])
 by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 08 May 2024 08:27:05 -0700
Received: from black.fi.intel.com ([10.237.72.28])
 by fmviesa008.fm.intel.com with ESMTP; 08 May 2024 08:27:04 -0700
Received: by black.fi.intel.com (Postfix, from userid 1003)
 id E52E2109; Wed, 08 May 2024 18:27:02 +0300 (EEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6da09837-0d4f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1715182028; x=1746718028;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=ItL86uGAqwtSHzvV7e4Z98MypI8As0zBjnzHJ4JD/+Y=;
  b=dihqrNeW4yJ4lGlK2+krNY901BGI1vF0WXQlAZxODAolSr5cP0kBDCXb
   a1FwrPkefBOrno+qcV2A5zwEDApP8wLO6cigGuB3vhQJKhyo2dFXe5FgY
   UHZ9JvIMU9SqmYneEBmo6F9UQTqQqnr3dF4w6oZPr3gq9TpgKezk2NDZ9
   u02I6Lwy5xNK7SLhoT6DDZEFUMLROtrEZo9CLVR1wnCu4nL8cOEedGY8P
   lG7F0ppmWuIEpZKaT1e3dHjSRH27lGWu1ue7Qj2fzM3Pe8dQtrfI8mBvH
   MGPoeHrYLEn5F7K6YXZpWkjLM6GXYD74CDhgLUWPJVc5g6femLFKtqTUj
   w==;
X-CSE-ConnectionGUID: SnFDFH67Qf2o/Y5+AnNbOw==
X-CSE-MsgGUID: VRNSCFwcQJq/GtO50mrVww==
X-IronPort-AV: E=McAfee;i="6600,9927,11067"; a="11429089"
X-IronPort-AV: E=Sophos;i="6.08,145,1712646000"; 
   d="scan'208";a="11429089"
X-CSE-ConnectionGUID: MYnSKQ2/RdCXHfHJiF7k9w==
X-CSE-MsgGUID: U96b3+4LRi2AqLykBsphWA==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="6.08,145,1712646000"; 
   d="scan'208";a="28851881"
From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
To: Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Subject: [PATCH v1 1/1] xen/xenbus: Use *-y instead of *-objs in Makefile
Date: Wed,  8 May 2024 18:26:47 +0300
Message-ID: <20240508152658.1445809-1-andriy.shevchenko@linux.intel.com>
X-Mailer: git-send-email 2.43.0.rc1.1336.g36b5255a03ac
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

*-objs suffix is reserved rather for (user-space) host programs while
usually *-y suffix is used for kernel drivers (although *-objs works
for that purpose for now).

Let's correct the old usages of *-objs in Makefiles.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---

Note, the original approach is weirdest from the existing.
Only a few drivers use this (-objs-y) one most likely by mistake.

 drivers/xen/xenbus/Makefile | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/xen/xenbus/Makefile b/drivers/xen/xenbus/Makefile
index 0c7532110815..b0d69602214e 100644
--- a/drivers/xen/xenbus/Makefile
+++ b/drivers/xen/xenbus/Makefile
@@ -1,15 +1,13 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y	+= xenbus.o
+
+xenbus-y := xenbus_client.o
+xenbus-y += xenbus_comms.o
+xenbus-y += xenbus_xs.o
+xenbus-y += xenbus_probe.o
+
+xenbus-$(CONFIG_XEN_BACKEND) += xenbus_probe_backend.o
+
 obj-y	+= xenbus_dev_frontend.o
-
-xenbus-objs =
-xenbus-objs += xenbus_client.o
-xenbus-objs += xenbus_comms.o
-xenbus-objs += xenbus_xs.o
-xenbus-objs += xenbus_probe.o
-
-xenbus-be-objs-$(CONFIG_XEN_BACKEND) += xenbus_probe_backend.o
-xenbus-objs += $(xenbus-be-objs-y)
-
 obj-$(CONFIG_XEN_BACKEND) += xenbus_dev_backend.o
 obj-$(CONFIG_XEN_XENBUS_FRONTEND) += xenbus_probe_frontend.o
-- 
2.43.0.rc1.1336.g36b5255a03ac



From xen-devel-bounces@lists.xenproject.org Wed May 08 16:02:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:02:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718864.1121392 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jkG-0000hP-PW; Wed, 08 May 2024 16:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718864.1121392; Wed, 08 May 2024 16:02:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jkG-0000hI-Mh; Wed, 08 May 2024 16:02:16 +0000
Received: by outflank-mailman (input) for mailman id 718864;
 Wed, 08 May 2024 16:02:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+6iU=ML=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s4jkE-0000hC-NZ
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:02:14 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 53fb880a-0d54-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 18:02:12 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715184127647979.96624377197;
 Wed, 8 May 2024 09:02:07 -0700 (PDT)
Received: by mail-yb1-f179.google.com with SMTP id
 3f1490d57ef6-debaa161ae3so2150549276.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 09:02:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53fb880a-0d54-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; t=1715184129; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=GGMICaZCndvaw4O66Vn7q/JTkjmsbDi27MxVeQWigqOMxp5+kXoQjnyyxnFUuzDWhVO4c21QPkDVd6OdXfOOSQv4EMvqul4/1rSxqBwNMouMxSZPBbVgyQxsNrTtdeDZAjCBjRuwX/8CgNPm8fiI22fJrDbuRoUlAud2b5/8G+w=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715184129; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=kdhTNtSLBgIcMB2+gf3nZE9Jc2bVl6O30it28R56h+I=; 
	b=Z6bYzMaV3KR6Km7gCg7bc4+HGm1xz9YVrovKoP3+J3rbh2gKIDMGy6dmmnJ0DZNXJN3UYHJ6gs9dOY452xRKsi/D3J8NvreB6ZPE7pjMNGXL4U8FGMpmOqgkZwNuwzZIvuZ+K2LS5uNcAMeMYgIR/D0uIqSnA6XpAkyEJzpPGBw=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715184129;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=kdhTNtSLBgIcMB2+gf3nZE9Jc2bVl6O30it28R56h+I=;
	b=TMaFu/T6rai3Kee4Qp3pF8lBl1RTKzadTw4sfta/eITbLiHoq3AjebdY896vSfZI
	2tD+TbDcN21e5ZFu34QOc2L+xixGnJ2bznayEJzIpLsUFREjUrBSyfT4HixSSgt/fb4
	Nx4cu7YebLoTl6ivd0NPbFWz3JQAfx+R+IaDPzv4=
X-Forwarded-Encrypted: i=1; AJvYcCX9PdzMhy4vAQrXawlBmCpOoP9D80W0AL7La0J6d0C0oqK5YwxlJs+4zyBtxJoi8R3QluCtf/3nYN6/h/8AIIh+kwUXj01iV80CDZ3PYCg=
X-Gm-Message-State: AOJu0YyIqgrLnj8mVEK8S1+4rdAdFSU9LW/BbNrD6eTZwi4/hcQnwF+8
	aOhzAl2/9Iq1PcStLYNIKl4jHHernrmUPsZSijbmSECAmNymUOwoKxd23UTqQZRdtljfWxMnnGS
	JsHMddMz1tE1aPb2Tl1LcdpCUEJg=
X-Google-Smtp-Source: AGHT+IF400K5awnmZh4PIQySVMEDEdn4BxZjvGUiTVh9+B7aEtMXBHrT/6RAVn5Xayjh7BIG0/tIxeT1NoDQ6F0lbOU=
X-Received: by 2002:a25:9786:0:b0:de5:53f3:7218 with SMTP id
 3f1490d57ef6-debb9d48616mr3752430276.38.1715184126603; Wed, 08 May 2024
 09:02:06 -0700 (PDT)
MIME-Version: 1.0
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com> <834460aca34e6167a3c4620616c145b8@bugseng.com>
 <47033435-c621-40f6-b5a9-a385f323f382@xen.org> <5b84019c4e0111b3cc5e369b999fbfab@bugseng.com>
In-Reply-To: <5b84019c4e0111b3cc5e369b999fbfab@bugseng.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Wed, 8 May 2024 12:01:27 -0400
X-Gmail-Original-Message-ID: <CABfawhkKAKAYhTrbRNyJcOaf=rQChy9eM=aVYQM9xv1ScO-UNg@mail.gmail.com>
Message-ID: <CABfawhkKAKAYhTrbRNyJcOaf=rQChy9eM=aVYQM9xv1ScO-UNg@mail.gmail.com>
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
To: alessandro.zucchelli@bugseng.com
Cc: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
	bertrand.marquis@arm.com, michal.orzel@amd.com, volodymyr_babchuk@epam.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 10:02=E2=80=AFAM Alessandro Zucchelli
<alessandro.zucchelli@bugseng.com> wrote:
>
> On 2024-05-03 11:32, Julien Grall wrote:
> > Hi,
> >
> > On 03/05/2024 08:09, Alessandro Zucchelli wrote:
> >> On 2024-04-29 17:58, Jan Beulich wrote:
> >>> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
> >>>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
> >>>> allowing asm/mem_access.h to be included in all ARM build
> >>>> configurations.
> >>>> This is to address the violation of MISRA C: 2012 Rule 8.4 which
> >>>> states:
> >>>> "A compatible declaration shall be visible when an object or
> >>>> function
> >>>> with external linkage is defined". Functions p2m_mem_access_check
> >>>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is not
> >>>> defined in ARM builds don't have visible declarations in the file
> >>>> containing their definitions.
> >>>>
> >>>> Signed-off-by: Alessandro Zucchelli
> >>>> <alessandro.zucchelli@bugseng.com>
> >>>> ---
> >>>>  xen/include/xen/mem_access.h | 2 +-
> >>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/xen/include/xen/mem_access.h
> >>>> b/xen/include/xen/mem_access.h
> >>>> index 87d93b31f6..ec0630677d 100644
> >>>> --- a/xen/include/xen/mem_access.h
> >>>> +++ b/xen/include/xen/mem_access.h
> >>>> @@ -33,7 +33,7 @@
> >>>>   */
> >>>>  struct vm_event_st;
> >>>>
> >>>> -#ifdef CONFIG_MEM_ACCESS
> >>>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
> >>>>  #include <asm/mem_access.h>
> >>>>  #endif
> >>>
> >>> This doesn't look quite right. If Arm supports mem-access, why would
> >>> it
> >>> not set MEM_ACCESS=3Dy? Whereas if it's only stubs that Arm supplies,
> >>> then
> >>> those would better move here, thus eliminating the need for a
> >>> per-arch
> >>> stub header (see what was e.g. done for numa.h). This way RISC-V and
> >>> PPC
> >>> (and whatever is to come) would then be taken care of as well.
> >>>
> >> ARM does support mem-access, so I don't think this is akin to the
> >> changes done to handle numa.h.
> >> ARM also allows users to set MEM_ACCESS=3Dn (e.g.
> >> xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however,
> >> the implementation file mem_access.c is compiled unconditionally in
> >> ARM's makefile, hence why the violation was spotted.
> >> This is a bit unusual, so I was also hoping to get some feedback from
> >> mem-access maintainers as to why this discrepancy from x86 exists. I
> >> probably should have also included some ARM maintainers as well, so
> >> I'm going to loop them in now.
> >>
> >> An alternative option I think is to make the compilation of arm's
> >> mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and
> >> common).
> >
> > I can't think of a reason to have mem_access.c unconditional compiled
> > in. So I think it should be conditional like on x86.
>
> Hi,
> attempting to build ARM with a configuration where MEM_ACCESS=3Dn and
> mem_access.c is conditioned on CONFIG_MEM_ACCESS results in a fail as
> there are other pieces of code that call some mem_access.c functions
> (p2m_mem_access_check_and_get_page, p2m_mem_access_check).
> In a Matrix chat Julien was suggesting adding stubs for the functions
> for this use case.

Perhaps just wrap the callers into #ifdef CONFIG_MEM_ACCESS blocks?

Tamas


From xen-devel-bounces@lists.xenproject.org Wed May 08 16:10:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:10:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718867.1121402 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jrg-0001kj-Ge; Wed, 08 May 2024 16:09:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718867.1121402; Wed, 08 May 2024 16:09:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4jrg-0001kc-DT; Wed, 08 May 2024 16:09:56 +0000
Received: by outflank-mailman (input) for mailman id 718867;
 Wed, 08 May 2024 16:09:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4jrf-0001kW-9Y
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:09:55 +0000
Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com
 [2607:f8b0:4864:20::832])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 66c386c6-0d55-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 18:09:52 +0200 (CEST)
Received: by mail-qt1-x832.google.com with SMTP id
 d75a77b69052e-43d3a33d41dso26553321cf.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 09:09:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 bz6-20020a05622a1e8600b00439c1419553sm7839119qtb.44.2024.05.08.09.09.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 09:09:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66c386c6-0d55-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715184591; x=1715789391; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=ERgYyoQWo4zQ/PeMvKAHRkgO4o9g0joMcpPbVkGfWr8=;
        b=Fg5wLiRe+Up0cdtU+gNRBk4IAvl9ozfIs0ugEVYUc2KVkxVUrkwIwYDlZ9E6cRQ3aS
         noQvD/Eb3jeNNX5kHzbyJpTqQQv8b/Gg+zxrHe+RZ7289sPr9VHD7kxO2U1fOMqUik8z
         4bvnZTgm2eW/Ly8GK+9iNjf5e0Snk1cUsKdt8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715184591; x=1715789391;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ERgYyoQWo4zQ/PeMvKAHRkgO4o9g0joMcpPbVkGfWr8=;
        b=VxdKcEkbYXL13iyY6jfywIc/7Kg4Dkk38+Oga1dOxf8AoSxGVn8jxDC1t369SVqMBS
         gm78yaVZTpc6LktCPE7BRX2dfUUvNJv9QgGFY1AOqCYzDe7MZ1ALhar2KAvnDhVR1D5l
         pXUkmJ8MeRWQLH8gfRzeCG4c82zZwobodtw5pALRlj8q35m8cVYlzBcR5VacDO5Ng3Oc
         bP7a9KLjh6j5lDKf6sv8iFy32u3Bl9wCTDngjzMkruDwXAu7H2MNl0oC0uRn+USPBh2d
         lwSZVRiDbYvhzj2iPNUzdLJo4sYa3BewcMDvj1DeFTw9K6zMvos/Zt4YXSkjvlWOnhF/
         d+Vw==
X-Gm-Message-State: AOJu0YzSmZGpSlmh6U+Abb0Drv2J0s3qGT+2NewHv9hqADntL8CAdl2g
	YCJdXjH4z5hxY7ASWEjbiNWEJC6RXOnKau6+/MRPSkK7nH0MRRH48SxnAMLw1hs=
X-Google-Smtp-Source: AGHT+IE5oM5oC35wdA9RcOoUYsujpFjSMvST0YLomhmVsj3c+bNqCFjIXOjuwGrux+PPpE/RH6xGRQ==
X-Received: by 2002:ac8:5987:0:b0:43a:e61a:ddc7 with SMTP id d75a77b69052e-43dbf73b39amr39150791cf.44.1715184591206;
        Wed, 08 May 2024 09:09:51 -0700 (PDT)
Date: Wed, 8 May 2024 18:09:48 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v7 2/6] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <ZjujzCnAfHye_eIp@macbook>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
 <da180c8419882dc068512b69c6c7ad3ff3343fe0.1715085837.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <da180c8419882dc068512b69c6c7ad3ff3343fe0.1715085837.git-series.marmarek@invisiblethingslab.com>

On Tue, May 07, 2024 at 02:44:02PM +0200, Marek Marczykowski-Górecki wrote:
> Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> on the same page as MSI-X table. Device model (especially one in
> stubdomain) cannot really handle those, as direct writes to that page is
> refused (page is on the mmio_ro_ranges list). Instead, extend
> msixtbl_mmio_ops to handle such accesses too.
> 
> Doing this, requires correlating read/write location with guest
> MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
> it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> for PV would need to be done separately.
> 
> This will be also used to read Pending Bit Array, if it lives on the same
> page, making QEMU not needing /dev/mem access at all (especially helpful
> with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> map it to the guest directly.
> If PBA lives on the same page, discard writes and log a message.
> Technically, writes outside of PBA could be allowed, but at this moment
> the precise location of PBA isn't saved, and also no known device abuses
> the spec in this way (at least yet).
> 
> To access those registers, msixtbl_mmio_ops need the relevant page
> mapped. MSI handling already has infrastructure for that, using fixmap,
> so try to map first/last page of the MSI-X table (if necessary) and save
> their fixmap indexes. Note that msix_get_fixmap() does reference
> counting and reuses existing mapping, so just call it directly, even if
> the page was mapped before. Also, it uses a specific range of fixmap
> indexes which doesn't include 0, so use 0 as default ("not mapped")
> value - which simplifies code a bit.
> 
> Based on assumption that all MSI-X page accesses are handled by Xen, do
> not forward adjacent accesses to other hypothetical ioreq servers, even
> if the access wasn't handled for some reason (failure to map pages etc).
> Relevant places log a message about that already.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Thanks, just a couple of minor comments, I think the only relevant one
is that you can drop ADJACENT_DONT_HANDLE unless there's something
I'm missing.  The rest are mostly cosmetic, but if you have to respin
and agree with them might be worth addressing.

Sorry for giving this feedback so late in the process, I should have
attempted to review earlier versions.

> ---
> Changes in v7:
> - simplify logic based on assumption that all access to MSI-X pages are
>   handled by Xen (Roger)
> - move calling adjacent_handle() into adjacent_{read,write}() (Roger)
> - move range check into msixtbl_addr_to_desc() (Roger)
> - fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
> - no longer distinguish between unhandled write due to PBA nearby and
>   other reasons
> - add missing break after ASSERT_UNREACHABLE (Jan)
> Changes in v6:
> - use MSIX_CHECK_WARN macro
> - extend assert on fixmap_idx
> - add break in default label, after ASSERT_UNREACHABLE(), and move
>   setting default there
> - style fixes
> Changes in v5:
> - style fixes
> - include GCC version in the commit message
> - warn only once (per domain, per device) about failed adjacent access
> Changes in v4:
> - drop same_page parameter of msixtbl_find_entry(), distinguish two
>   cases in relevant callers
> - rename adj_access_table_idx to adj_access_idx
> - code style fixes
> - drop alignment check in adjacent_{read,write}() - all callers already
>   have it earlier
> - delay mapping first/last MSI-X pages until preparing device for a
>   passthrough
> v3:
>  - merge handling into msixtbl_mmio_ops
>  - extend commit message
> v2:
>  - adjust commit message
>  - pass struct domain to msixtbl_page_handler_get_hwaddr()
>  - reduce local variables used only once
>  - log a warning if write is forbidden if MSI-X and PBA lives on the same
>    page
>  - do not passthrough unaligned accesses
>  - handle accesses both before and after MSI-X table
> ---
>  xen/arch/x86/hvm/vmsi.c        | 205 ++++++++++++++++++++++++++++++++--
>  xen/arch/x86/include/asm/msi.h |   5 +-
>  xen/arch/x86/msi.c             |  42 +++++++-
>  3 files changed, 242 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> index 999917983789..f7b7b4998b5e 100644
> --- a/xen/arch/x86/hvm/vmsi.c
> +++ b/xen/arch/x86/hvm/vmsi.c
> @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
>      return d->arch.hvm.msixtbl_list.next;
>  }
>  
> +/*
> + * Lookup an msixtbl_entry on the same page as given addr. It's up to the
> + * caller to check if address is strictly part of the table - if relevant.
> + */
>  static struct msixtbl_entry *msixtbl_find_entry(
>      struct vcpu *v, unsigned long addr)
>  {
> @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
>      struct domain *d = v->domain;
>  
>      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> -        if ( addr >= entry->gtable &&
> -             addr < entry->gtable + entry->table_len )
> +        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
> +             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
>              return entry;
>  
>      return NULL;
> @@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
>      if ( !entry || !entry->pdev )
>          return NULL;
>  
> +    if ( addr < entry->gtable ||
> +         addr >= entry->gtable + entry->table_len )
> +        return NULL;
> +
>      nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
>  
>      list_for_each_entry( desc, &entry->pdev->msi_list, list )
> @@ -213,6 +221,152 @@ static struct msi_desc *msixtbl_addr_to_desc(
>      return NULL;
>  }
>  
> +/*
> + * Returns:
> + *  - ADJACENT_DONT_HANDLE if no handling should be done
> + *  - a fixmap idx to use for handling
> + */
> +#define ADJACENT_DONT_HANDLE UINT_MAX

Isn't it fine to just return 0 to signal that the access is not
handled?

fixmap index 0 is reserved anyway (see FIX_RESERVED), so could be used
for this purpose and then you don't need to introduce
ADJACENT_DONT_HANDLE?

> +static unsigned int adjacent_handle(
> +    const struct msixtbl_entry *entry, unsigned long addr, bool write)

Now that it has been quite pruned, get_adjacent_idx() or some such
might be more inline with the function logic.

> +{
> +    unsigned int adj_type;
> +    struct arch_msix *msix;
> +
> +    if ( !entry || !entry->pdev )
> +    {
> +        ASSERT_UNREACHABLE();
> +        return ADJACENT_DONT_HANDLE;
> +    }
> +
> +    if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable )
> +        adj_type = ADJ_IDX_FIRST;
> +    else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) &&
> +              addr >= entry->gtable + entry->table_len )
> +        adj_type = ADJ_IDX_LAST;
> +    else
> +    {
> +        /* All callers should already do equivalent range checking. */
> +        ASSERT_UNREACHABLE();
> +        return ADJACENT_DONT_HANDLE;
> +    }
> +
> +    msix = entry->pdev->msix;
> +    ASSERT(msix);

Since you already do it above, would be best to do:

if ( !msix )
{
    ASSERT_UNREACHABLE();
    return 0;
}

> +
> +    if ( !msix->adj_access_idx[adj_type] )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_not_initialized) )
> +            gprintk(XENLOG_WARNING,
> +                    "Page for adjacent(%d) MSI-X table access not initialized for %pp (addr %#lx, gtable %#lx\n",

Do you really need to log an error here?  There's an error already
printed in msix_capability_init() if the adjacent pages can't be
mapped.

> +                    adj_type, &entry->pdev->sbdf, addr, entry->gtable);
> +        return ADJACENT_DONT_HANDLE;
> +    }
> +
> +    /* If PBA lives on the same page too, discard writes. */
> +    if ( write &&
> +         ((adj_type == ADJ_IDX_LAST &&
> +           msix->table.last == msix->pba.first) ||
> +          (adj_type == ADJ_IDX_FIRST &&
> +           msix->table.first == msix->pba.last)) )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_pba) )
> +            gprintk(XENLOG_WARNING,
> +                    "MSI-X table and PBA of %pp live on the same page, "
> +                    "writing to other registers there is not implemented\n",
> +                    &entry->pdev->sbdf);

I would usually start those errors with the SBDF, as that makes it
easier to identify error originating from the same device:

"%pp: MSI-X table and PBA share a page, discard write to adjacent memory (%#lx)\n",
&entry->pdev->sbdf, addr

> +        return ADJACENT_DONT_HANDLE;
> +    }
> +
> +    return msix->adj_access_idx[adj_type];
> +}
> +
> +static int adjacent_read(
> +    const struct msixtbl_entry *entry,
> +    paddr_t address, unsigned int len, uint64_t *pval)
> +{
> +    const void __iomem *hwaddr;
> +    unsigned int fixmap_idx;
> +

I would add an ASSERT(IS_ALIGNED(address, len)) (and in
adjacent_write()) just in case, as otherwise the access could cross a
page boundary.

> +    fixmap_idx = adjacent_handle(entry, address, false);
> +
> +    if ( fixmap_idx == ADJACENT_DONT_HANDLE )
> +    {
> +        *pval = ~0UL;
> +        return X86EMUL_OKAY;
> +    }

FWIW, I find it safer to unconditionally init *pval = ~0UL at the
start of the function, and then the return here and in the default
switch statement case can avoid setting it.  It's less easy to return
without the variable being set.

> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        *pval = readb(hwaddr);
> +        break;
> +
> +    case 2:
> +        *pval = readw(hwaddr);
> +        break;
> +
> +    case 4:
> +        *pval = readl(hwaddr);
> +        break;
> +
> +    case 8:
> +        *pval = readq(hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +        *pval = ~0UL;
> +        break;

We should likely move this to some kind of helpers, as it's now
exactly the same that's done in adjacent_{read,write}() in
vpci/msix.c (not that you should do it here, just a note).

> +    }
> +
> +    return X86EMUL_OKAY;
> +}
> +
> +static int adjacent_write(
> +    const struct msixtbl_entry *entry,
> +    paddr_t address, unsigned int len, uint64_t val)
> +{
> +    void __iomem *hwaddr;
> +    unsigned int fixmap_idx;
> +
> +    fixmap_idx = adjacent_handle(entry, address, true);
> +
> +    if ( fixmap_idx == ADJACENT_DONT_HANDLE )
> +        return X86EMUL_OKAY;
> +
> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        writeb(val, hwaddr);
> +        break;
> +
> +    case 2:
> +        writew(val, hwaddr);
> +        break;
> +
> +    case 4:
> +        writel(val, hwaddr);
> +        break;
> +
> +    case 8:
> +        writeq(val, hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +        break;
> +    }
> +
> +    return X86EMUL_OKAY;
> +}
> +
>  static int cf_check msixtbl_read(
>      const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
>      uint64_t *pval)
> @@ -222,7 +376,7 @@ static int cf_check msixtbl_read(
>      unsigned int nr_entry, index;
>      int r = X86EMUL_UNHANDLEABLE;
>  
> -    if ( (len != 4 && len != 8) || (address & (len - 1)) )
> +    if ( !IS_ALIGNED(address, len) )
>          return r;
>  
>      rcu_read_lock(&msixtbl_rcu_lock);
> @@ -230,6 +384,17 @@ static int cf_check msixtbl_read(
>      entry = msixtbl_find_entry(current, address);
>      if ( !entry )
>          goto out;
> +
> +    if ( address < entry->gtable ||
> +         address >= entry->gtable + entry->table_len )
> +    {
> +        r = adjacent_read(entry, address, len, pval);
> +        goto out;
> +    }
> +
> +    if ( len != 4 && len != 8 )
> +        goto out;
> +
>      offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
>  
>      if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
> @@ -291,6 +456,17 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
>      entry = msixtbl_find_entry(v, address);
>      if ( !entry )
>          goto out;
> +
> +    if ( address < entry->gtable ||
> +         address >= entry->gtable + entry->table_len )
> +    {
> +        r = adjacent_write(entry, address, len, val);
> +        goto out;
> +    }
> +
> +    if ( len != 4 && len != 8 )
> +        goto out;
> +
>      nr_entry = array_index_nospec(((address - entry->gtable) /
>                                     PCI_MSIX_ENTRY_SIZE),
>                                    MAX_MSIX_TABLE_ENTRIES);
> @@ -356,8 +532,8 @@ static int cf_check _msixtbl_write(
>      const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
>      uint64_t val)
>  {
> -    /* Ignore invalid length or unaligned writes. */
> -    if ( (len != 4 && len != 8) || !IS_ALIGNED(address, len) )
> +    /* Ignore unaligned writes. */
> +    if ( !IS_ALIGNED(address, len) )
>          return X86EMUL_OKAY;
>  
>      /*
> @@ -374,16 +550,25 @@ static bool cf_check msixtbl_range(
>  {
>      struct vcpu *curr = current;
>      unsigned long addr = r->addr;
> -    const struct msi_desc *desc;
> +    const struct msixtbl_entry *entry;
> +    bool ret = false;
>  
>      ASSERT(r->type == IOREQ_TYPE_COPY);
>  
>      rcu_read_lock(&msixtbl_rcu_lock);
> -    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
> +    entry = msixtbl_find_entry(curr, addr);
> +    if ( entry )
> +    {
> +        if ( addr < entry->gtable || addr >= entry->gtable + entry->table_len )
> +            /* Possibly handle adjacent access. */
> +            ret = true;
> +        else
> +            ret = msixtbl_addr_to_desc(entry, addr) != NULL;
> +    }

You could probably put all this into a single condition:

if ( entry &&
      /* Adjacent access. */
     (addr < entry->gtable || addr >= entry->gtable + entry->table_len ||
      /* Otherwise check if there's a matching msi_desc. */
      msixtbl_addr_to_desc(entry, addr)) )
    ret = true;

That's IMO easier to read by setting ret once only.

>      rcu_read_unlock(&msixtbl_rcu_lock);
>  
> -    if ( desc )
> -        return 1;
> +    if ( ret )
> +        return ret;
>  
>      if ( r->state == STATE_IOREQ_READY && r->dir == IOREQ_WRITE )
>      {
> @@ -429,7 +614,7 @@ static bool cf_check msixtbl_range(
>          }
>      }
>  
> -    return 0;
> +    return false;
>  }
>  
>  static const struct hvm_io_ops msixtbl_mmio_ops = {
> diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
> index bcfdfd35345d..923b730d48b8 100644
> --- a/xen/arch/x86/include/asm/msi.h
> +++ b/xen/arch/x86/include/asm/msi.h
> @@ -224,6 +224,9 @@ struct arch_msix {
>      } table, pba;
>      int table_refcnt[MAX_MSIX_TABLE_PAGES];
>      int table_idx[MAX_MSIX_TABLE_PAGES];
> +#define ADJ_IDX_FIRST 0
> +#define ADJ_IDX_LAST  1
> +    unsigned int adj_access_idx[2];
>      spinlock_t table_lock;
>      bool host_maskall, guest_maskall;
>      domid_t warned_domid;
> @@ -231,6 +234,8 @@ struct arch_msix {
>          uint8_t all;
>          struct {
>              bool maskall                   : 1;
> +            bool adjacent_not_initialized  : 1;

Not sure we need that, since we already warn of failure to map at
initialization time, not sure it's worth also printing another error
at access time.

> +            bool adjacent_pba              : 1;
>          };
>      } warned_kind;
>  };
> diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> index 42c793426da3..87190a88ed5d 100644
> --- a/xen/arch/x86/msi.c
> +++ b/xen/arch/x86/msi.c
> @@ -913,6 +913,37 @@ static int msix_capability_init(struct pci_dev *dev,
>          list_add_tail(&entry->list, &dev->msi_list);
>          *desc = entry;
>      }
> +    else
> +    {
> +        /*
> +         * If the MSI-X table doesn't start at the page boundary, map the first page for
> +         * passthrough accesses.
> +         */
> +        if ( PAGE_OFFSET(table_paddr) )
> +        {
> +            int idx = msix_get_fixmap(msix, table_paddr, table_paddr);
> +
> +            if ( idx > 0 )
> +                msix->adj_access_idx[ADJ_IDX_FIRST] = idx;
> +            else
> +                gprintk(XENLOG_ERR, "Failed to map first MSI-X table page: %d\n", idx);
> +        }
> +        /*
> +         * If the MSI-X table doesn't end on the page boundary, map the last page
> +         * for passthrough accesses.
> +         */
> +        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE) )
> +        {
> +            uint64_t entry_paddr = table_paddr +
> +                (msix->nr_entries - 1) * PCI_MSIX_ENTRY_SIZE;
> +            int idx = msix_get_fixmap(msix, table_paddr, entry_paddr);
> +
> +            if ( idx > 0 )
> +                msix->adj_access_idx[ADJ_IDX_LAST] = idx;
> +            else
> +                gprintk(XENLOG_ERR, "Failed to map last MSI-X table page: %d\n", idx);

Could you prefix the messages with the SBDF of the device please?
It's in the seg, bus, slot, func local variables AFAICT.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 08 16:18:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718872.1121412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k05-0003h2-Fb; Wed, 08 May 2024 16:18:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718872.1121412; Wed, 08 May 2024 16:18:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k05-0003gv-Br; Wed, 08 May 2024 16:18:37 +0000
Received: by outflank-mailman (input) for mailman id 718872;
 Wed, 08 May 2024 16:18:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=guPL=ML=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4k04-0003gp-2w
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:18:36 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on0626.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::626])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9d3a52e0-0d56-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 18:18:33 +0200 (CEST)
Received: from DUZPR01CA0066.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:3c2::14) by PAVPR08MB9379.eurprd08.prod.outlook.com
 (2603:10a6:102:300::5) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.45; Wed, 8 May
 2024 16:18:28 +0000
Received: from DU2PEPF00028D09.eurprd03.prod.outlook.com
 (2603:10a6:10:3c2:cafe::31) by DUZPR01CA0066.outlook.office365.com
 (2603:10a6:10:3c2::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.41 via Frontend
 Transport; Wed, 8 May 2024 16:18:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028D09.mail.protection.outlook.com (10.167.242.169) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7519.19
 via Frontend Transport; Wed, 8 May 2024 16:18:24 +0000
Received: ("Tessian outbound e46bb127ed3d:v315");
 Wed, 08 May 2024 16:18:24 +0000
Received: from cf8b32b6726a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 055DF03E-800D-44CC-A8D0-A73765D0CC25.1; 
 Wed, 08 May 2024 16:18:13 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cf8b32b6726a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 May 2024 16:18:13 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com (2603:10a6:208:ff::27)
 by PAVPR08MB9233.eurprd08.prod.outlook.com (2603:10a6:102:30a::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.45; Wed, 8 May
 2024 16:18:08 +0000
Received: from AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::d987:3f45:3a0a:f56a]) by AM0PR08MB3745.eurprd08.prod.outlook.com
 ([fe80::d987:3f45:3a0a:f56a%6]) with mapi id 15.20.7544.041; Wed, 8 May 2024
 16:18:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d3a52e0-0d56-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=ehqDRoyvk1S5vnKayQOFW5CSmDeCzdsY9Ms6OnY3SN2SN7eR22YgvupqtieehHiZ+aP4qj1QzHLUs2+VLAMKAuiIpjrdVEdgjhk/C43U8gAwkrnTqsgV1tsDS7ULCCZEGiMNPSR/JQg4NuWDUHrBj2PkLGzop+q2xqlbP0FV/K2s95EQM57a/OtIZgvC5MZEe36crcsKKOl/KlEF2AZ70pxBSJHLjW7mrQhwiA01CW5V2qolQQW1A6iWCzxrN1+VSaicXcLK9cNeY97wD4Bwqlgsza5RhOis3MWNAf+BRusKjqh23NWDwiXaApskTOIMyMlGL+E52+dCGz4PhY9E7Q==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4OH/AvRmA/KmgVttKawV+T8DCYZ/F9UF7C3uLjcrtSM=;
 b=f5jjZrp7OJUfkpIB7E+gj1sqE74w5V0aUKGLInodXgDFVg1PMa79ct+kXQW3u/QMB9P1ibMi8ahwTkbBRj4uMVp7Q0u5QcQT23nKMPTyCJ0KUmuG1eOz4hF9hYc8xxF3YWH/25eBIBJdJixSD8D90eRuVWyeM2ECtv01h7H/4MwjNUd2CBpdgO8oI/XZbBHWidk1nFgZ2oj4KSpFX3o9GMgH7RlBLz8UZJZTtMI4tjulraXHiyjwHabzEfVZidOBp/DPGT/W2JUS2ZjXFe7R5a1Gd53uGFIX4Yj/ivV83wCZraReOzcgfapcKUAdruXsWMrTXVHX3TCArSIdLLIMqQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4OH/AvRmA/KmgVttKawV+T8DCYZ/F9UF7C3uLjcrtSM=;
 b=V0iV8mrmIMpG2DUo04fllLgGeQaO3vWxR1i4S5kQ/o1NtbKisFdTHx2OstJFtJJapwCCAbDShOxXvZgv1S+cznGY8Y+lEjsvjauc/RFPAlNeH391yrdSKwE9AiRz3UOiFhpCweDIqUJThdsV6uO74g6/Xhx8Ae/0vYQtRZyQauw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 437280982cff6f78
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kv0rrcI3neHORlyJso6NAX2zrYVidb35zkJgI/eVipSkzJpejVcivYqcLq/qoPdQnCk9wQMs2zhbygXry8Lu8Z85ya+7ewEO4J0LbWzeEGyS05kdR7cIOC7xGHHfsZnptgFXjvIPL9xsgm5DtdCedPNTy5E8KFvSeQN0qDVX2/BSNl+MCawJcgru+VJpry6iPxNupYmTsl6h7J4Dkt8taoFXxslIq7RnV3slIim9FimuXvT8qwZx8bRudu+fptOCto19PhRFuu1/0CqmFqYlHv9bqu3XYnyfYoBqNpUYKvjlgkTX+y0DFPlbmNz9u4BIEZrqhAvpTN6SvSJszuFkFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4OH/AvRmA/KmgVttKawV+T8DCYZ/F9UF7C3uLjcrtSM=;
 b=NtRbUN0YrZ84LKLxYPXwiJtDW8y6TPE9xiJBBMi2+51zccaftTGrvM8IhGn8XL3j2xYmu5Iy/pvQNs98q8vl7yb65HlMN/npgLl+gFfItHX/4U8FgtpP5h0ZP9zgogPwpvmLVFMiTOQiHQznIpql9W65YF2VZ+sI3q4MVUVvBGliVmFcQuVc5bXJ60PYxQr4U3V9sbRZHmlf4F9S0gCIBG4JVATWC4F9Hg7k41M2Z+oZaYq6ALgROYj2uJNp8E8KV7TNv/SWimP3Rd40RWCNLZq0T9zde/HYR6xwBHKy3OTr9wgdfL+7ODeWm01K5hXlLU2jMpddSvtGLCfdFlisDg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4OH/AvRmA/KmgVttKawV+T8DCYZ/F9UF7C3uLjcrtSM=;
 b=V0iV8mrmIMpG2DUo04fllLgGeQaO3vWxR1i4S5kQ/o1NtbKisFdTHx2OstJFtJJapwCCAbDShOxXvZgv1S+cznGY8Y+lEjsvjauc/RFPAlNeH391yrdSKwE9AiRz3UOiFhpCweDIqUJThdsV6uO74g6/Xhx8Ae/0vYQtRZyQauw=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "consulting @ bugseng . com"
	<consulting@bugseng.com>, Nicola Vetrini <nicola.vetrini@bugseng.com>, Julien
 Grall <julien@xen.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible array
 member not at the end
Thread-Topic: [PATCH 2/2] xen/arm: Fix MISRA regression on R1.1, flexible
 array member not at the end
Thread-Index: AQHamu72Eq1RpEGs60mNp9DIfY8mn7GESRmAgAlHnIA=
Date: Wed, 8 May 2024 16:18:07 +0000
Message-ID: <57F6E89F-F670-49BA-B77E-6C6121A6A95F@arm.com>
References: <20240430110922.15052-1-luca.fancellu@arm.com>
 <20240430110922.15052-3-luca.fancellu@arm.com>
 <alpine.DEB.2.22.394.2405021135060.624854@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2405021135060.624854@ubuntu-linux-20-04-desktop>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM0PR08MB3745:EE_|PAVPR08MB9233:EE_|DU2PEPF00028D09:EE_|PAVPR08MB9379:EE_
X-MS-Office365-Filtering-Correlation-Id: 8cbcb262-7565-468b-df4b-08dc6f7a7cb1
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?GXo2rfW1TFbQG1P019xHklv2Cj/qWLH0HfslRrXkrq3dV21yN8hggAn65tsv?=
 =?us-ascii?Q?Mdk+DJTSG4/9eAgNpJjTBKu0ZEqDfq8j1ZS8zA18sHHqtBuuCqbh+UD8IyDg?=
 =?us-ascii?Q?5k1RvcE2pKjW/5LOzy855/ovVR1cZFhsegborcwJz5iMiGOdC5lvARebKzi8?=
 =?us-ascii?Q?2tLYe3Zs8P3x3qA9ME2MJg1K30UAXXpdD0WVZgh92T6r/zV1/FSfWa0Gu+I4?=
 =?us-ascii?Q?4+WdAHWMWnJiy7L9PLb95O46j4/6jWBcfEm9MP7ITb9DKw/ESL3wKHoMfsVj?=
 =?us-ascii?Q?wJZpHXDD5FKQFw9WjjXEROevijmXmk2Ey4do8yfWEx+RJTmT1CuG6xIldBO4?=
 =?us-ascii?Q?XOmisHhGaZ1039yQoIvt/P1RZQCTFAdkL2LReD0ec1KQXA1h1xGGdeiRbbda?=
 =?us-ascii?Q?10645rCUrYlQ739e2EMjGEhGQraucJVXbmd2n8P84N9MKwTtJjz6Zg/0zUfs?=
 =?us-ascii?Q?VnBlK0SSF7cPLJJcQ4HvD3qKr5wHZnxWM5uEQSsRpb1DiFVsBqSqY04HPuA7?=
 =?us-ascii?Q?WmekjM/pxC5wQP00TX9wzdvIbSgRSanhvCb8nbFJok/Oe9omF/Sa3v+GinOh?=
 =?us-ascii?Q?uz2vrYRv+fi689sVNTx8DWpKsbQF7eCR2eYLtfuYMreSu3Nv6iMzj2wKYE/E?=
 =?us-ascii?Q?DrE0e/nCqJe/UCOkjkQFHmybbMVUzrAvrMMvZrcmq0Wu/eaRRr32UxUyF2se?=
 =?us-ascii?Q?Qf/e5HPSOiK+9SFjednZBZGn1DyqwRy2Qnk1SUDp9FgJuX0bDtp1YS1lwCIJ?=
 =?us-ascii?Q?Pm2cupbjhayY0o2Lpe4OXCOSDsHk2Gq92/xSd4CRoFkrJMssu5rc9SwANz+C?=
 =?us-ascii?Q?VU96dhE6P28tMNwPeXo2+dlT3w0YRqyqsNf5chrXfXW/Yixo8nZwMBZPcfo4?=
 =?us-ascii?Q?I0c2hIRh3lS6N2tE6aXlejNOu2dlFiVZuBJFK+BX5d/D77ZatWSo9Dm9tl94?=
 =?us-ascii?Q?S+FmqjKBP9qNFyFsxx+y2DsIW2hfVUrLsC096jr9Ju6ub2KAVKyWjdE5Ki2O?=
 =?us-ascii?Q?1UxfrdN0H/VON4bEW+AZluJL1xdtrCVBBNyeEgQOGzZw493SRdiI402QcghI?=
 =?us-ascii?Q?VPQ1vfjH1ld+c1MGYZIPNoQkyrX4RYmMJU1/CEYId0pCX/zOHCf7wskKn8ea?=
 =?us-ascii?Q?0ETvhrOMtTpqVocB4to320gdIlwrJRrm5O3vCT6bO7YY87PMcJTnVg10BEk/?=
 =?us-ascii?Q?J0p4PxKwPBljmgLJX2u7n4M9ytjTOm30edcGCGo/26uuzX66rya27GzKTfpj?=
 =?us-ascii?Q?jDWwU5dpaWPYoAdj1dq60oUtrz+w5AZGN1uJQfUBZniWXjhRE45CLAkXDbSH?=
 =?us-ascii?Q?J63029YQdbCADrX8Mv57a8L92YNUtw76ggJD5xoYES4UWw=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR08MB3745.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <57781801558DEC4693A76E4030FD3453@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB9233
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028D09.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9122885c-749e-4340-3685-08dc6f7a727e
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|35042699013|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?OQC4mK5qQW2nevFFpZJ10BYBMDL1UMjP6BmJIX+Tkrb45UX6tgJuPrWGmOM2?=
 =?us-ascii?Q?26biQcRTcvPcKjWs9KymlpB3W5rJEoYzXeK7nd6f5sKui2fUHKpa5cJj/GG2?=
 =?us-ascii?Q?t4JVym0etwFQnO9gO4lDBnLF25Hn7PopoLuNM63SKJFHYiXBoLwyZkuVlxRq?=
 =?us-ascii?Q?tfpGLUQOPP8gc7qY1S64A4VYInp+8FY8GWlZzCQlA/kl//K8PdgLMrXCL0AD?=
 =?us-ascii?Q?3+eS/xSIQZ7BwDzp7CGOeZ/5NUCq7pe0fTvy2Ag0YFJoBpHhlwxQXsEsiWwv?=
 =?us-ascii?Q?8G2PCMDt6Rim+wLJF/1e+Ys7/GildlxVv6RGhALF6VFcZhPagsZ2xw+oKD9M?=
 =?us-ascii?Q?TAW9xd4WJsDaHVvU8DF0eu7UPb1FJdGE1scNW/9FK5U8irxPI8oL8jm+6DhW?=
 =?us-ascii?Q?dNvXK9bo6S9POmDDaAAYd03WIPcoGFrEV/edqPBG+Z5FUknqr0pp8t4yOp9R?=
 =?us-ascii?Q?yndD1XQhDfxQ+bpKG36MiQ8NDF3K2ZBUMx4R5yWbNDy1tkeq3HGdOpX7KiKV?=
 =?us-ascii?Q?3arVT1esEG0ml2Yn5eR2sYiwa+HVuExqOZ6xyjlwAtxu+PAwa772Q/LErCe3?=
 =?us-ascii?Q?yislCC2t1xpguofUiKYOW2tg6HjmQcWDODFWHqF6xZiE2cfdEE71tRsLj98u?=
 =?us-ascii?Q?PocCpNF+woWcAI+o801CfAetPNPMLYyaXlKU4COokcJvGPhhnuKUq7Ez0sLL?=
 =?us-ascii?Q?+3dHaM+AcLR3hxHFefccF+C+Tv4UUms6HoI/tIGzc3eXN1du8ImR3tGYy4z4?=
 =?us-ascii?Q?f71TCF0xfpykaU1Q2ecyd1vUgt8Y6A2sAH3oS5UPmE3k+C+6Fb/1BZPd0iSO?=
 =?us-ascii?Q?cnoz+Emlmg3m4fltrMHIXIYI4vl2Flb3tDbuHJOuNnd244/ww0qJY5KOw4hz?=
 =?us-ascii?Q?Kb8jOnNf92d+Vbl2Q51pb322HZPqRAPM2cGjKIlzrDCPTicpi7/DKaWUNvn5?=
 =?us-ascii?Q?4uM7HoyDbMJXE4ngG/hhkTHgO/Kad6VQWX5IdPIeZWW36zxNf8dyRxF/ZP5y?=
 =?us-ascii?Q?jYQddJf7sF/HXdEw0ms1nYOegFMZLifOF6EaFa+THmpL3lcO55ukt2IW2zxJ?=
 =?us-ascii?Q?U+6IkH7WCVvodF4kbGXME854neR/8ItaryrD9MkR9HZzjYjNC8U93NKbTe53?=
 =?us-ascii?Q?wEuhB7pNJH7kCCz7fFDnvP/9qmx3QwxLENDd61dfFBRQyIYKWj9Gv9HrOL5i?=
 =?us-ascii?Q?zo0dN29lDIujEIGJ9ZwvlCyrfjTDjqOMUB+vAmLf81vVa4HWeC6u7Kap8mDC?=
 =?us-ascii?Q?qOWlwF+woECjnGXNJU+qjHPcOGzxsj2d8VnWmT4SY1kMUfeU/Q0jCC6Apmmt?=
 =?us-ascii?Q?wvyM3Sb4YH9AnPxcAPzLWeNW8kzbu0qH17/WJndED8Ziva1Wnk+h9AlZixUU?=
 =?us-ascii?Q?Ww0fw79zgXTxjq7vgxhN1bTTjbmw?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(35042699013)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 16:18:24.8492
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8cbcb262-7565-468b-df4b-08dc6f7a7cb1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028D09.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB9379



> On 2 May 2024, at 19:35, Stefano Stabellini <sstabellini@kernel.org> wrot=
e:
>=20
> On Tue, 30 Apr 2024, Luca Fancellu wrote:
>> Commit 2209c1e35b47 ("xen/arm: Introduce a generic way to access memory
>> bank structures") introduced a MISRA regression for Rule 1.1 because a
>> flexible array member is introduced in the middle of a struct, furthermo=
re
>> this is using a GCC extension that is going to be deprecated in GCC 14 a=
nd
>> a warning to identify such cases will be present
>> (-Wflex-array-member-not-at-end) to identify such cases.
>>=20
>> In order to fix this issue, use the macro __struct_group to create a
>> structure 'struct membanks_hdr' which will hold the common data among
>> structures using the 'struct membanks' interface.
>>=20
>> Modify the 'struct shared_meminfo' and 'struct meminfo' to use this new
>> structure, effectively removing the flexible array member from the middl=
e
>> of the structure and modify the code accessing the .common field to use
>> the macro container_of to maintain the functionality of the interface.
>>=20
>> Given this change, container_of needs to be supplied with a type and so
>> the macro 'kernel_info_get_mem' inside arm/include/asm/kernel.h can't be
>> an option since it uses const and non-const types for struct membanks, s=
o
>> introduce two static inline, one of which will keep the const qualifier.
>>=20
>> Given the complexity of the interface, which carries a lot of benefit bu=
t
>> on the other hand could be prone to developer confusion if the access is
>> open-coded, introduce two static inline helper for the
>> 'struct kernel_info' .shm_mem member and get rid the open-coding
>> shm_mem.common access.
>>=20
>> Fixes: 2209c1e35b47 ("xen/arm: Introduce a generic way to access memory =
bank structures")
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>=20
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Hi Stefano,

Thanks!

Is it possible to add, eventually on commit, this tag?

Reported-by: Nicola Vetrini <nicola.vetrini@bugseng.com>




From xen-devel-bounces@lists.xenproject.org Wed May 08 16:18:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:18:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718874.1121422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k0F-0003yK-M8; Wed, 08 May 2024 16:18:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718874.1121422; Wed, 08 May 2024 16:18:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k0F-0003xv-J3; Wed, 08 May 2024 16:18:47 +0000
Received: by outflank-mailman (input) for mailman id 718874;
 Wed, 08 May 2024 16:18:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6EkY=ML=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4k0E-0003gp-9b
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:18:46 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a437eb9c-0d56-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 18:18:44 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59ae3efbb3so484935266b.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 09:18:44 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ds9-20020a0564021cc900b005705bfeeb27sm7730212edb.66.2024.05.08.09.18.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 09:18:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a437eb9c-0d56-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715185124; x=1715789924; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=mPXNK6VDI3eN9sDJaElz56WvTPg3GXhNKZT2KE+6/YI=;
        b=mLGZ4GQcbnY47pNL0Qqf4E5FyhWFtMNiIOP6tf9d7Prk+71lLXWLNHdZN7ERsXj463
         7vzUJF42JZR0NoKJ+3I1DcDmgYHSc8miiIp5VAiAbf5qwiy7WX3u/IPSkb180ELqv/Z1
         AeCJRXQ2o2KsOKc4z0pQLBV7olZhaGR7IYU4U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715185124; x=1715789924;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mPXNK6VDI3eN9sDJaElz56WvTPg3GXhNKZT2KE+6/YI=;
        b=A/fh8XBM9lw6gG2UzyoaVW/GK+OJspIcTrcOJ3jazxmRGLUP3jkpO2M1eeMeRlGI45
         x9MFMqAg9ej2+a6PxS7+jFzALHo+y44DCKOEEmCpKRTT/xXabMdc7TdrKfae7n9of6or
         DU3/KKfJB1TbgSNfLtkmg/X2b7ltFI5Ldy4AzfUCj4rW9ZFBFTHWmhWmW7nYOZRg8sCh
         8puDK/iAs/NwIYc4EBB8csd1k8ov0/8veY+n1W4SpF4ZHzAOBYQTN2u4i0xHce2/QF6n
         dw6hFmdG1koblMhcpBRL6t/CERfSLbONNUTrr8LyVRbg5eTEQY9DHWr20sd+Ckrns/B2
         SUHA==
X-Gm-Message-State: AOJu0YxxBKO7+5to/HbTrZ1g7TNqi3sc3Ek+jtLEzh5lOp5x05yPdtoL
	Nyg8xwKDyewjsr0IhUy68XN7kR1Iia3Py17txDc+DbpyMYGZ4zYVv+Q/5E/js8m5J7ItoqFYism
	d
X-Google-Smtp-Source: AGHT+IGLiUU+B2eT1NQivHN6bxUbo6EQVV8P+/oMCjRqOBPAXYL6QXvDCgvSqd1TWCeGNkLlfbKC8w==
X-Received: by 2002:a50:d60c:0:b0:572:9dc3:ce38 with SMTP id 4fb4d7f45d1cf-5731d9d1364mr2186063a12.19.1715185123741;
        Wed, 08 May 2024 09:18:43 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] x86/ucode: Distinguish "ucode already up to date"
Date: Wed,  8 May 2024 17:18:42 +0100
Message-Id: <20240508161842.1785414-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Right now, Xen returns -ENOENT for both "the provided blob isn't correct for
this CPU", and "the blob isn't newer than what's loaded".

This in turn causes xen-ucode to exit with an error, when "nothing to do" is
more commonly a success condition.

Handle EEXIST specially and exit cleanly.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
---
 tools/misc/xen-ucode.c            | 5 ++++-
 xen/arch/x86/cpu/microcode/core.c | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
index c6ae6498d659..390969db3d1c 100644
--- a/tools/misc/xen-ucode.c
+++ b/tools/misc/xen-ucode.c
@@ -125,8 +125,11 @@ int main(int argc, char *argv[])
         exit(1);
     }
 
+    errno = 0;
     ret = xc_microcode_update(xch, buf, len);
-    if ( ret )
+    if ( ret == -1 && errno == EEXIST )
+        printf("Microcode already up to date\n");
+    else if ( ret )
     {
         fprintf(stderr, "Failed to update microcode. (err: %s)\n",
                 strerror(errno));
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index e738a88f5cd1..762111838f5f 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -640,7 +640,7 @@ static long cf_check microcode_update_helper(void *data)
                    "microcode: couldn't find any newer%s revision in the provided blob!\n",
                    opt_ucode_allow_same ? " (or the same)" : "");
             microcode_free_patch(patch);
-            ret = -ENOENT;
+            ret = -EEXIST;
 
             goto put;
         }

base-commit: a2330b51df267e20e66bbba6c5bf08f0570ed58b
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 16:26:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:26:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718883.1121432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k7o-0006Ht-DN; Wed, 08 May 2024 16:26:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718883.1121432; Wed, 08 May 2024 16:26:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4k7o-0006Hm-AY; Wed, 08 May 2024 16:26:36 +0000
Received: by outflank-mailman (input) for mailman id 718883;
 Wed, 08 May 2024 16:26:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4k7m-0006Hg-Jl
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:26:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4k7j-0003vi-MI; Wed, 08 May 2024 16:26:31 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4k7j-0006Ac-Dm; Wed, 08 May 2024 16:26:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=RL0hQpSOclw6ycIWilwMPBbv4tY2zKX1p9TDawtqp2k=; b=zBua08FzZKDJcmPqW796m1dBQB
	YDV98uvENGfyC41yagCnlPezL9y6UDZaeSEVQlqW9TSb+nVsGdzYxQuoumHrZOL3vw44ebqLffWe0
	dJpjPb6kP5q3/XjfUI1e1OODVrChpCqhLK/9Vl5Y4InqJk4e3npZme2FFc32gGrmwGmI=;
Message-ID: <c1566cf8-91ce-4ba6-9f42-cedad744151a@xen.org>
Date: Wed, 8 May 2024 17:26:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
Content-Language: en-GB
To: Tamas K Lengyel <tamas@tklengyel.com>, alessandro.zucchelli@bugseng.com
Cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 bertrand.marquis@arm.com, michal.orzel@amd.com, volodymyr_babchuk@epam.com
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com>
 <834460aca34e6167a3c4620616c145b8@bugseng.com>
 <47033435-c621-40f6-b5a9-a385f323f382@xen.org>
 <5b84019c4e0111b3cc5e369b999fbfab@bugseng.com>
 <CABfawhkKAKAYhTrbRNyJcOaf=rQChy9eM=aVYQM9xv1ScO-UNg@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CABfawhkKAKAYhTrbRNyJcOaf=rQChy9eM=aVYQM9xv1ScO-UNg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Tamas,

On 08/05/2024 17:01, Tamas K Lengyel wrote:
> On Wed, May 8, 2024 at 10:02 AM Alessandro Zucchelli
> <alessandro.zucchelli@bugseng.com> wrote:
>>
>> On 2024-05-03 11:32, Julien Grall wrote:
>>> Hi,
>>>
>>> On 03/05/2024 08:09, Alessandro Zucchelli wrote:
>>>> On 2024-04-29 17:58, Jan Beulich wrote:
>>>>> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
>>>>>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
>>>>>> allowing asm/mem_access.h to be included in all ARM build
>>>>>> configurations.
>>>>>> This is to address the violation of MISRA C: 2012 Rule 8.4 which
>>>>>> states:
>>>>>> "A compatible declaration shall be visible when an object or
>>>>>> function
>>>>>> with external linkage is defined". Functions p2m_mem_access_check
>>>>>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is not
>>>>>> defined in ARM builds don't have visible declarations in the file
>>>>>> containing their definitions.
>>>>>>
>>>>>> Signed-off-by: Alessandro Zucchelli
>>>>>> <alessandro.zucchelli@bugseng.com>
>>>>>> ---
>>>>>>   xen/include/xen/mem_access.h | 2 +-
>>>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/xen/include/xen/mem_access.h
>>>>>> b/xen/include/xen/mem_access.h
>>>>>> index 87d93b31f6..ec0630677d 100644
>>>>>> --- a/xen/include/xen/mem_access.h
>>>>>> +++ b/xen/include/xen/mem_access.h
>>>>>> @@ -33,7 +33,7 @@
>>>>>>    */
>>>>>>   struct vm_event_st;
>>>>>>
>>>>>> -#ifdef CONFIG_MEM_ACCESS
>>>>>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
>>>>>>   #include <asm/mem_access.h>
>>>>>>   #endif
>>>>>
>>>>> This doesn't look quite right. If Arm supports mem-access, why would
>>>>> it
>>>>> not set MEM_ACCESS=y? Whereas if it's only stubs that Arm supplies,
>>>>> then
>>>>> those would better move here, thus eliminating the need for a
>>>>> per-arch
>>>>> stub header (see what was e.g. done for numa.h). This way RISC-V and
>>>>> PPC
>>>>> (and whatever is to come) would then be taken care of as well.
>>>>>
>>>> ARM does support mem-access, so I don't think this is akin to the
>>>> changes done to handle numa.h.
>>>> ARM also allows users to set MEM_ACCESS=n (e.g.
>>>> xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however,
>>>> the implementation file mem_access.c is compiled unconditionally in
>>>> ARM's makefile, hence why the violation was spotted.
>>>> This is a bit unusual, so I was also hoping to get some feedback from
>>>> mem-access maintainers as to why this discrepancy from x86 exists. I
>>>> probably should have also included some ARM maintainers as well, so
>>>> I'm going to loop them in now.
>>>>
>>>> An alternative option I think is to make the compilation of arm's
>>>> mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and
>>>> common).
>>>
>>> I can't think of a reason to have mem_access.c unconditional compiled
>>> in. So I think it should be conditional like on x86.
>>
>> Hi,
>> attempting to build ARM with a configuration where MEM_ACCESS=n and
>> mem_access.c is conditioned on CONFIG_MEM_ACCESS results in a fail as
>> there are other pieces of code that call some mem_access.c functions
>> (p2m_mem_access_check_and_get_page, p2m_mem_access_check).
>> In a Matrix chat Julien was suggesting adding stubs for the functions
>> for this use case.
> 
> Perhaps just wrap the callers into #ifdef CONFIG_MEM_ACCESS blocks?

In Xen, we tend prefer stubs over #ifdef-ing code blocks. I would rather 
use this approach here too.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 08 16:33:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 16:33:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718889.1121443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4kDw-0007lj-3b; Wed, 08 May 2024 16:32:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718889.1121443; Wed, 08 May 2024 16:32:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4kDv-0007lc-VO; Wed, 08 May 2024 16:32:55 +0000
Received: by outflank-mailman (input) for mailman id 718889;
 Wed, 08 May 2024 16:32:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0je0=ML=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4kDt-0007lW-SL
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 16:32:53 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9ce1f31c-0d58-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 18:32:51 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59cdd185b9so193353266b.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 09:32:51 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 r25-20020a170906351900b00a59b2683971sm5094559eja.187.2024.05.08.09.32.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 May 2024 09:32:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ce1f31c-0d58-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715185971; x=1715790771; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=O354KtvB6ahvVPuGywICPIp2MHx6KYPaz+66z+lEk80=;
        b=XCdPkna+F8Vvvwv2GXSgpTSy6+L0+qVE/VSa7N6SfHfKDk4cet8JNgL81NhHdKWzJh
         kjdNaN4FHJuttpNSH8XjIRrWlMt5OvMz93oT4k1OJLOgj87B1NBXlB+LpzAolN/XpS4z
         11LhxR3+DABonkzGlrCYFc44g3zfrJ7QwRbSg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715185971; x=1715790771;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=O354KtvB6ahvVPuGywICPIp2MHx6KYPaz+66z+lEk80=;
        b=lBAVz1kME/eJLwFz+rgL9RZVv0KpPdgkhtt5FbP8CT8U/gOLTc7rZ5ZWhWWTvyiNls
         LvdA22a2gOXWjVozSP+5Yv30+tFdkdFpVtKaSHN8cDgdBQaRo5B30j9uutjCQpxiWpSR
         xogtZF2PICdAn9qKVKjtrz/mnRgJkGoL9q0jZMg84dRlKV7XJgBaBw9HfQeColTILUNo
         JESnDhCn/SEh3M0NgVpzGf/kvAgES5V0wh6oOyLwa0yPvIYS5RtRVfEIWrzIcX6WKN8i
         joqftfhR/Ri/IoKSrrGsLmHpaM7BYj1Ndtav35mUY9uTS2JifTavjv4y5uYRUo7j1PZ9
         DRVg==
X-Gm-Message-State: AOJu0YxgCjVdyyHk3g7lubbxNblCavEZjbtqe1Wc0fZJQXOij1iJAGdQ
	eyOeyUUu6aQ7ZMtL2sXJtmEcLM51E81z/VoswArE7KSFJdjZRe0WyvJRg9FghvQ=
X-Google-Smtp-Source: AGHT+IHPN7HSPneklhkm0fU/q9NksB+sQP+wFawY+/NxTsc0M8V+UrLH5wwSIAKUUmgwxRAXYu6jDA==
X-Received: by 2002:a17:906:408a:b0:a59:dba0:1544 with SMTP id a640c23a62f3a-a5a117934famr11663666b.31.1715185970744;
        Wed, 08 May 2024 09:32:50 -0700 (PDT)
Date: Wed, 8 May 2024 18:32:49 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH] x86/ucode: Distinguish "ucode already up to date"
Message-ID: <ZjupMbwLGco9rSeI@macbook>
References: <20240508161842.1785414-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240508161842.1785414-1-andrew.cooper3@citrix.com>

On Wed, May 08, 2024 at 05:18:42PM +0100, Andrew Cooper wrote:
> Right now, Xen returns -ENOENT for both "the provided blob isn't correct for
> this CPU", and "the blob isn't newer than what's loaded".
> 
> This in turn causes xen-ucode to exit with an error, when "nothing to do" is
> more commonly a success condition.
> 
> Handle EEXIST specially and exit cleanly.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.


From xen-devel-bounces@lists.xenproject.org Wed May 08 17:00:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 17:00:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718894.1121452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4kew-0004UI-9m; Wed, 08 May 2024 17:00:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718894.1121452; Wed, 08 May 2024 17:00:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4kew-0004UB-6m; Wed, 08 May 2024 17:00:50 +0000
Received: by outflank-mailman (input) for mailman id 718894;
 Wed, 08 May 2024 17:00:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+6iU=ML=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s4kev-0004U0-57
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 17:00:49 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8105a80d-0d5c-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 19:00:43 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715187640156347.1766965633724;
 Wed, 8 May 2024 10:00:40 -0700 (PDT)
Received: by mail-yb1-f179.google.com with SMTP id
 3f1490d57ef6-de5b1e6beceso5334871276.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 10:00:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8105a80d-0d5c-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; t=1715187641; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=JJGEgOvmp/V135Bbi8Ctl3jdbCT9mwe/IWYozm7/m76LxYlHHTn7DCPn2S02bzgxDSOeFxt95i+kaigmT6pbzZ7OYko9TnvFwy1JNr7Yxa6EIYYGA/mxOehWVY61xVBih7lSrvfcpsylY9bssMpcfXwkoTE8G+NtH0pN4T+9fEM=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715187641; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=cqneTxxJSgPb2TrrC49/XhkSYxbRRGvFCNeMAOfl6Z8=; 
	b=AVG402j2/SqquE6ssJ/7iYGtxIr0/vsQjuAMc4iWK79mEyWVGSki10hzNAbbxqWuQnq2g8cSSLK18LXeIlucnrdOH+G2DCwLqRKi3gCOk2sbBkP3RzJ8nQFpHOrqpBsH/o+wBE8bJrInZ5u/IOhcmj+h51PhGFHNVKJntVCNFAE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715187641;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=cqneTxxJSgPb2TrrC49/XhkSYxbRRGvFCNeMAOfl6Z8=;
	b=AnKWH6PLykmpQwYCrM715pak2TWS716OgtIrzQHjFLVDhslNh26whXV8lbPS2fJP
	ejqkAHW6TR9bEX6DL33SNT8BjwD3HvCFBN0kNxZdYf5KQdSzOEsvlXKDvAu8wXUhExR
	a8/k+I+YcAZa6bpUyUWGO0Cw0oER4SEwC3BjQsYo=
X-Forwarded-Encrypted: i=1; AJvYcCXIgMfizpphrZ3eOldrS0V/mRnztxC9trpQKUbey5XFptPMBoylyLIcwWExwAyMmu133uqoaFd7l5tf3wvFdPTX3FXx8peiNGNg9ST74DI=
X-Gm-Message-State: AOJu0YzLi2dijekx98rIZWt4kcDVfIAP/8S3f+WEut7Nu3D0vQcvUTgW
	pQqPGTirQ7qeSqymWVfzpDIggsUBtH6sxFG8zl+ardX98xnicJyK4dnPAXX62RXE2ShhGc6oQ5i
	mTqwZ5LMr229KGG5rDOAQYW8Wk1M=
X-Google-Smtp-Source: AGHT+IGjkls0BYju8+WLsWDaNNdKjanTL6KRySc5nmdZf0byC7AJXq7BWcgPrn+qVMXdzDaS/O7KYGLZ4lozpxpOR2g=
X-Received: by 2002:a5b:a88:0:b0:dc6:ddfc:1736 with SMTP id
 3f1490d57ef6-debb9cfcd05mr3432789276.17.1715187639062; Wed, 08 May 2024
 10:00:39 -0700 (PDT)
MIME-Version: 1.0
References: <a3d4e07433932624266ac9b675daf0b70734696d.1714405386.git.alessandro.zucchelli@bugseng.com>
 <08eb5444-2bf0-48c0-9fcb-d8e4b801e8fa@suse.com> <834460aca34e6167a3c4620616c145b8@bugseng.com>
 <47033435-c621-40f6-b5a9-a385f323f382@xen.org> <5b84019c4e0111b3cc5e369b999fbfab@bugseng.com>
 <CABfawhkKAKAYhTrbRNyJcOaf=rQChy9eM=aVYQM9xv1ScO-UNg@mail.gmail.com> <c1566cf8-91ce-4ba6-9f42-cedad744151a@xen.org>
In-Reply-To: <c1566cf8-91ce-4ba6-9f42-cedad744151a@xen.org>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Wed, 8 May 2024 13:00:02 -0400
X-Gmail-Original-Message-ID: <CABfawhkrxjS=KwSRbdxtUmxBvrf3+TiU-vJ7oWaMqLyHCxhU1Q@mail.gmail.com>
Message-ID: <CABfawhkrxjS=KwSRbdxtUmxBvrf3+TiU-vJ7oWaMqLyHCxhU1Q@mail.gmail.com>
Subject: Re: [XEN PATCH] xen/mem_access: address violations of MISRA C: 2012
 Rule 8.4
To: Julien Grall <julien@xen.org>
Cc: alessandro.zucchelli@bugseng.com, Jan Beulich <jbeulich@suse.com>, 
	consulting@bugseng.com, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org, 
	sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
	volodymyr_babchuk@epam.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 12:26=E2=80=AFPM Julien Grall <julien@xen.org> wrote=
:
>
> Hi Tamas,
>
> On 08/05/2024 17:01, Tamas K Lengyel wrote:
> > On Wed, May 8, 2024 at 10:02=E2=80=AFAM Alessandro Zucchelli
> > <alessandro.zucchelli@bugseng.com> wrote:
> >>
> >> On 2024-05-03 11:32, Julien Grall wrote:
> >>> Hi,
> >>>
> >>> On 03/05/2024 08:09, Alessandro Zucchelli wrote:
> >>>> On 2024-04-29 17:58, Jan Beulich wrote:
> >>>>> On 29.04.2024 17:45, Alessandro Zucchelli wrote:
> >>>>>> Change #ifdef CONFIG_MEM_ACCESS by OR-ing defined(CONFIG_ARM),
> >>>>>> allowing asm/mem_access.h to be included in all ARM build
> >>>>>> configurations.
> >>>>>> This is to address the violation of MISRA C: 2012 Rule 8.4 which
> >>>>>> states:
> >>>>>> "A compatible declaration shall be visible when an object or
> >>>>>> function
> >>>>>> with external linkage is defined". Functions p2m_mem_access_check
> >>>>>> and p2m_mem_access_check_and_get_page when CONFIG_MEM_ACCESS is no=
t
> >>>>>> defined in ARM builds don't have visible declarations in the file
> >>>>>> containing their definitions.
> >>>>>>
> >>>>>> Signed-off-by: Alessandro Zucchelli
> >>>>>> <alessandro.zucchelli@bugseng.com>
> >>>>>> ---
> >>>>>>   xen/include/xen/mem_access.h | 2 +-
> >>>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>>>
> >>>>>> diff --git a/xen/include/xen/mem_access.h
> >>>>>> b/xen/include/xen/mem_access.h
> >>>>>> index 87d93b31f6..ec0630677d 100644
> >>>>>> --- a/xen/include/xen/mem_access.h
> >>>>>> +++ b/xen/include/xen/mem_access.h
> >>>>>> @@ -33,7 +33,7 @@
> >>>>>>    */
> >>>>>>   struct vm_event_st;
> >>>>>>
> >>>>>> -#ifdef CONFIG_MEM_ACCESS
> >>>>>> +#if defined(CONFIG_MEM_ACCESS) || defined(CONFIG_ARM)
> >>>>>>   #include <asm/mem_access.h>
> >>>>>>   #endif
> >>>>>
> >>>>> This doesn't look quite right. If Arm supports mem-access, why woul=
d
> >>>>> it
> >>>>> not set MEM_ACCESS=3Dy? Whereas if it's only stubs that Arm supplie=
s,
> >>>>> then
> >>>>> those would better move here, thus eliminating the need for a
> >>>>> per-arch
> >>>>> stub header (see what was e.g. done for numa.h). This way RISC-V an=
d
> >>>>> PPC
> >>>>> (and whatever is to come) would then be taken care of as well.
> >>>>>
> >>>> ARM does support mem-access, so I don't think this is akin to the
> >>>> changes done to handle numa.h.
> >>>> ARM also allows users to set MEM_ACCESS=3Dn (e.g.
> >>>> xen/arch/arm/configs/tiny64_defconfig) and builds just fine; however=
,
> >>>> the implementation file mem_access.c is compiled unconditionally in
> >>>> ARM's makefile, hence why the violation was spotted.
> >>>> This is a bit unusual, so I was also hoping to get some feedback fro=
m
> >>>> mem-access maintainers as to why this discrepancy from x86 exists. I
> >>>> probably should have also included some ARM maintainers as well, so
> >>>> I'm going to loop them in now.
> >>>>
> >>>> An alternative option I think is to make the compilation of arm's
> >>>> mem_access.c conditional on CONFIG_MEM_ACCESS (as for x86/mm and
> >>>> common).
> >>>
> >>> I can't think of a reason to have mem_access.c unconditional compiled
> >>> in. So I think it should be conditional like on x86.
> >>
> >> Hi,
> >> attempting to build ARM with a configuration where MEM_ACCESS=3Dn and
> >> mem_access.c is conditioned on CONFIG_MEM_ACCESS results in a fail as
> >> there are other pieces of code that call some mem_access.c functions
> >> (p2m_mem_access_check_and_get_page, p2m_mem_access_check).
> >> In a Matrix chat Julien was suggesting adding stubs for the functions
> >> for this use case.
> >
> > Perhaps just wrap the callers into #ifdef CONFIG_MEM_ACCESS blocks?
>
> In Xen, we tend prefer stubs over #ifdef-ing code blocks. I would rather
> use this approach here too.

I was looking at arch/x86/hvm/hvm.c for examples on how MEM_PAGING and
MEM_SHARING calls are handled and those were ifdef'd. I have no
preference for one vs the other, both work.

Tamas


From xen-devel-bounces@lists.xenproject.org Wed May 08 18:46:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 18:46:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718920.1121462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4mJD-0001qY-KR; Wed, 08 May 2024 18:46:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718920.1121462; Wed, 08 May 2024 18:46:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4mJD-0001qR-Gm; Wed, 08 May 2024 18:46:31 +0000
Received: by outflank-mailman (input) for mailman id 718920;
 Wed, 08 May 2024 18:46:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CW/7=ML=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s4mJC-0001qL-In
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 18:46:30 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 468dae99-0d6b-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 20:46:27 +0200 (CEST)
Received: from nico.home-life.hub (unknown [176.206.12.122])
 by support.bugseng.com (Postfix) with ESMTPSA id 311884EE0753;
 Wed,  8 May 2024 20:46:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 468dae99-0d6b-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	andrew.cooper3@citrix.com,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
	Jan Beulich <jbeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>
Subject: [XEN PATCH v3] x86/IOMMU: address violations of MISRA C:2012 Rule 14.4
Date: Wed,  8 May 2024 20:46:21 +0200
Message-Id: <eb64431143014d80e387861825df9e6d058e2a2d.1715193515.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>

The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
headline states:
"The controlling expression of an if statement and the controlling
expression of an iteration-statement shall have essentially Boolean type".

Add comparisons to avoid using enum constants as controlling expressions
to comply with Rule 14.4.

Amend the comment in the enum definition to reflect the fact that
boolean uses of iommu_intremap are no longer allowed.

No functional change.

Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
Signed-off-by: Nicola Vetrini  <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini  <sstabellini@kernel.org>
---
Changes in v2
- rename prefix from AMD/IOMMU to x86/IOMMU
- move changes on msi.c and hpet.c in this patch.
Changes in v3:
- amend comment in iommu_intremap definition
- add Stefano's R-by
---
 xen/arch/x86/hpet.c                      | 6 +++---
 xen/arch/x86/msi.c                       | 4 ++--
 xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
 xen/drivers/passthrough/vtd/iommu.c      | 4 ++--
 xen/drivers/passthrough/vtd/quirks.c     | 2 +-
 xen/include/xen/iommu.h                  | 4 ----
 6 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 0759eae055ce..2f54d3188966 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -278,7 +278,7 @@ static int hpet_msi_write(struct hpet_event_channel *ch, struct msi_msg *msg)
 {
     ch->msi.msg = *msg;
 
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
     {
         int rc = iommu_update_ire_from_msi(&ch->msi, msg);
 
@@ -352,7 +352,7 @@ static int __init hpet_setup_msi_irq(struct hpet_event_channel *ch)
     u32 cfg = hpet_read32(HPET_Tn_CFG(ch->idx));
     irq_desc_t *desc = irq_to_desc(ch->msi.irq);
 
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
     {
         ch->msi.hpet_id = hpet_blockid;
         ret = iommu_setup_hpet_msi(&ch->msi);
@@ -371,7 +371,7 @@ static int __init hpet_setup_msi_irq(struct hpet_event_channel *ch)
         ret = __hpet_setup_msi_irq(desc);
     if ( ret < 0 )
     {
-        if ( iommu_intremap )
+        if ( iommu_intremap != iommu_intremap_off )
             iommu_update_ire_from_msi(&ch->msi, NULL);
         return ret;
     }
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index e721aaf5c001..3009ce109f5e 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -189,7 +189,7 @@ static int write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
     entry->msg = *msg;
 
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
     {
         int rc;
 
@@ -555,7 +555,7 @@ int msi_free_irq(struct msi_desc *entry)
             destroy_irq(entry[nr].irq);
 
         /* Free the unused IRTE if intr remap enabled */
-        if ( iommu_intremap )
+        if ( iommu_intremap != iommu_intremap_off )
             iommu_update_ire_from_msi(entry + nr, NULL);
     }
 
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 358c8cbad99c..6c0dc2d5cb69 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1483,7 +1483,7 @@ int __init amd_iommu_init(bool xt)
             goto error_out;
     }
 
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
         register_keyhandler('V', &amd_iommu_dump_intremap_tables,
                             "dump IOMMU intremap tables", 0);
 
@@ -1501,7 +1501,7 @@ int __init amd_iommu_init_late(void)
 
     /* Further initialize the device table(s). */
     pci_init = true;
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
         rc = iterate_ivrs_mappings(amd_iommu_setup_device_table);
 
     for_each_amd_iommu ( iommu )
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index c7110af7c9cd..e13be244c130 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2540,7 +2540,7 @@ static int __must_check init_vtd_hw(bool resume)
     /*
      * Enable interrupt remapping
      */  
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
     {
         int apic;
         for ( apic = 0; apic < nr_ioapics; apic++ )
@@ -2556,7 +2556,7 @@ static int __must_check init_vtd_hw(bool resume)
             }
         }
     }
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
     {
         for_each_drhd_unit ( drhd )
         {
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 5a56565ea883..950dcd56ef8a 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -392,7 +392,7 @@ void __init platform_quirks_init(void)
     map_igd_reg();
 
     /* Tylersburg interrupt remap quirk */
-    if ( iommu_intremap )
+    if ( iommu_intremap != iommu_intremap_off )
         tylersburg_intremap_quirk();
 }
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 92db6f124f13..442ae5322d34 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -60,10 +60,6 @@ extern uint8_t iommu_quarantine;
 
 #ifdef CONFIG_X86
 extern enum __packed iommu_intremap {
-   /*
-    * In order to allow traditional boolean uses of the iommu_intremap
-    * variable, the "off" value has to come first (yielding a value of zero).
-    */
    iommu_intremap_off,
    /*
     * Interrupt remapping enabled, but only able to generate interrupts
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 08 19:13:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 19:13:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718924.1121472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4mj9-0005bM-Lu; Wed, 08 May 2024 19:13:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718924.1121472; Wed, 08 May 2024 19:13:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4mj9-0005bF-Ix; Wed, 08 May 2024 19:13:19 +0000
Received: by outflank-mailman (input) for mailman id 718924;
 Wed, 08 May 2024 19:13:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6EkY=ML=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4mj8-0005b9-7n
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 19:13:18 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 062209ae-0d6f-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 21:13:17 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41ebcf01013so810215e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 12:13:17 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 b12-20020a5d4d8c000000b0034e65b8b43fsm14020419wru.8.2024.05.08.12.13.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 08 May 2024 12:13:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 062209ae-0d6f-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715195596; x=1715800396; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=WcdIt/e4eYUzUbM9WS/sjSzHTYUNboHK5drX4E4qu5A=;
        b=OhJGudtWuqeNe+El1F+XLPRSqDr3GYQbNWz1Ebz4pfEHnzjiwrN2Bwpjpca08ZHyaY
         bdt4L2mluaB7JS+tv3vjMLoHSOg2vk1SCxEfDqg07Azb6pHSjF3J7gwdyPWjX0HUMhVs
         50f3WO0shjIr2uRvcMxGycQo38cbaAw+5GhnE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715195596; x=1715800396;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WcdIt/e4eYUzUbM9WS/sjSzHTYUNboHK5drX4E4qu5A=;
        b=enNzpFZgNIWqO0k0HUReV3BOhsx/S9Zj8l5BcRIoLCZoXSHb8iVap3du00gzSCS9CK
         QJxCfs6tbYKSs4YaVG5MNk0v2xMKahWTfYFVNhD0p5I1uuNapvXTUNVK44bpXWMkycZn
         Rj7ISWLJ+7LggMaUYtTdooHVv7S8iUeAbcgQGt5YReAXFX9D/5fEyUC4KM2ZhD4kax+v
         DCOLqOcWyNxarwpb5r4N7Zo1s7wYlZ1pnqYpz/ommEms3u2kY4t8yeDjaYpn8Rk90riY
         Ge71ZxoYaBGcEpTnwyZIHS1heiKnquxQ3zJhdGSJzIBqTe+Yi91uSJ4PlJ2PPwPSHG8A
         7Caw==
X-Forwarded-Encrypted: i=1; AJvYcCUtmpfLAdUu72x4SCD9rhzngFHMxB/aC/cVCNyCvt0LsTvJHvRIiuASyUX0Ry835pz1wIgQ83Avu7UdY/3UAkJSFIREdQ5lPSFVxj2C5nE=
X-Gm-Message-State: AOJu0YzSUG1SvFlr19C7Cvw4b/LV1JUjwkuAu6GZTZ3byYWbrRh0V0gR
	qCOqqv5lw6c0HArzdh5gixDKZcUyi8u4KHPNqyIkNs3sw9CP1spPtl7wWLZIpk0=
X-Google-Smtp-Source: AGHT+IEzmIGoyqAHb9LYk8U+9Ny/Aa5h1lpGH0cCl+JhElteTx124psZi6iNXZ0CuBlcYMuexnHNgA==
X-Received: by 2002:a05:600c:3b88:b0:419:f4d6:463b with SMTP id 5b1f17b1804b1-41fbcea20b7mr4765255e9.14.1715195596256;
        Wed, 08 May 2024 12:13:16 -0700 (PDT)
Message-ID: <45753c70-eef4-4ca7-b7f7-73b452310d4a@citrix.com>
Date: Wed, 8 May 2024 20:13:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/8] tools/hvmloader: Wake APs with hypercalls and not
 with INIT+SIPI+SIPI
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/05/2024 1:39 pm, Alejandro Vallejo wrote:
> Removes a needless assembly entry point and simplifies the codebase by allowing
> hvmloader to wake APs it doesn't know the APIC ID of.
>
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

This is basically independent of the rest of the series, and it would be
good to pull it in separately.  A few notes.

The commit message ought to note that this has a side effect of removing
an LMSW instruction which has fastpath at all on AMD CPUs, and requires
full emulation, and it gets rid of 13 vLAPIC emulations when 3
hypercalls would do.

The point being that this is borderline backport material, although it
does depend on the 32 vCPU bugfix.

> ---
> v2:
>   * New patch. Replaces adding cpu policy to hvmloader in v1.
> ---
>  tools/firmware/hvmloader/smp.c | 111 +++++++++++++--------------------
>  1 file changed, 44 insertions(+), 67 deletions(-)
>
> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
> index 082b17f13818..a668f15d7e1f 100644
> --- a/tools/firmware/hvmloader/smp.c
> +++ b/tools/firmware/hvmloader/smp.c
> @@ -22,88 +22,68 @@
>  #include "util.h"
>  #include "config.h"
>  #include "apic_regs.h"
> +#include "hypercall.h"
>  
> -#define AP_BOOT_EIP 0x1000
> -extern char ap_boot_start[], ap_boot_end[];
> +#include <xen/asm/x86-defns.h>
> +#include <xen/hvm/hvm_vcpu.h>
> +
> +#include <xen/vcpu.h>
>  
>  static int ap_callin, ap_cpuid;
>  
> -asm (
> -    "    .text                       \n"
> -    "    .code16                     \n"
> -    "ap_boot_start: .code16          \n"
> -    "    mov   %cs,%ax               \n"
> -    "    mov   %ax,%ds               \n"
> -    "    lgdt  gdt_desr-ap_boot_start\n"
> -    "    xor   %ax, %ax              \n"
> -    "    inc   %ax                   \n"
> -    "    lmsw  %ax                   \n"
> -    "    ljmpl $0x08,$1f             \n"
> -    "gdt_desr:                       \n"
> -    "    .word gdt_end - gdt - 1     \n"
> -    "    .long gdt                   \n"
> -    "ap_boot_end: .code32            \n"
> -    "1:  mov   $0x10,%eax            \n"
> -    "    mov   %eax,%ds              \n"
> -    "    mov   %eax,%es              \n"
> -    "    mov   %eax,%ss              \n"
> -    "    movl  $stack_top,%esp       \n"
> -    "    movl  %esp,%ebp             \n"
> -    "    call  ap_start              \n"
> -    "1:  hlt                         \n"
> -    "    jmp  1b                     \n"
> -    "                                \n"
> -    "    .align 8                    \n"
> -    "gdt:                            \n"
> -    "    .quad 0x0000000000000000    \n"
> -    "    .quad 0x00cf9a000000ffff    \n" /* 0x08: Flat code segment */
> -    "    .quad 0x00cf92000000ffff    \n" /* 0x10: Flat data segment */
> -    "gdt_end:                        \n"
> -    "                                \n"
> -    "    .bss                        \n"
> -    "    .align    8                 \n"
> -    "stack:                          \n"
> -    "    .skip    0x4000             \n"
> -    "stack_top:                      \n"
> -    "    .text                       \n"
> -    );
> -
> -void ap_start(void); /* non-static avoids unused-function compiler warning */
> -/*static*/ void ap_start(void)
> +static void ap_start(void)
>  {
>      printf(" - CPU%d ... ", ap_cpuid);
>      cacheattr_init();
>      printf("done.\n");
> +
> +    if ( !ap_cpuid )
> +        return;
> +
>      wmb();
>      ap_callin = 1;

/* After this point, the BSP will shut us down. */

> -}
>  
> -static void lapic_wait_ready(void)
> -{
> -    while ( lapic_read(APIC_ICR) & APIC_ICR_BUSY )
> -        cpu_relax();
> +    while ( 1 )

For this, we tend to favour "for ( ;; )".

> +        asm volatile ( "hlt" );
>  }
>  
>  static void boot_cpu(unsigned int cpu)
>  {
> -    unsigned int icr2 = SET_APIC_DEST_FIELD(LAPIC_ID(cpu));
> +    static uint8_t ap_stack[4 * PAGE_SIZE] __attribute__ ((aligned (16)));

I know you're just copying what was there, but 4 pages is stupidly large
for something that needs about 4 stack slots.

4K is absolutely plenty.

> +    static struct vcpu_hvm_context ap;
>  
>      /* Initialise shared variables. */
>      ap_cpuid = cpu;
> -    ap_callin = 0;
>      wmb();
>  
> -    /* Wake up the secondary processor: INIT-SIPI-SIPI... */
> -    lapic_wait_ready();
> -    lapic_write(APIC_ICR2, icr2);
> -    lapic_write(APIC_ICR, APIC_DM_INIT);
> -    lapic_wait_ready();
> -    lapic_write(APIC_ICR2, icr2);
> -    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
> -    lapic_wait_ready();
> -    lapic_write(APIC_ICR2, icr2);
> -    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
> -    lapic_wait_ready();
> +    /* Wake up the secondary processor */
> +    ap = (struct vcpu_hvm_context) {
> +        .mode = VCPU_HVM_MODE_32B,
> +        .cpu_regs.x86_32 = {
> +            .eip = (uint32_t)ap_start,
> +            .esp = (uint32_t)ap_stack + ARRAY_SIZE(ap_stack),

Not that it really matters, but these want to be unsigned long casts.

> +
> +            /* Protected mode with MMU off */
> +            .cr0 = X86_CR0_PE,
> +
> +            /* Prepopulate the GDT */

/* 32bit Flat Mode */

This isn't the GDT; it's the segment registers, but "Flat Mode" is the
more meaningful term to use.


I'm happy to fix all on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 08 19:33:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 19:33:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718931.1121481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4n2a-0008W7-76; Wed, 08 May 2024 19:33:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718931.1121481; Wed, 08 May 2024 19:33:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4n2a-0008W0-3j; Wed, 08 May 2024 19:33:24 +0000
Received: by outflank-mailman (input) for mailman id 718931;
 Wed, 08 May 2024 19:33:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LGkX=ML=linuxfoundation.org=gregkh@srs-se1.protection.inumbo.net>)
 id 1s4n2Y-0008Vd-9o
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 19:33:22 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d269f0ef-0d71-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 21:33:20 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 45AD7CE1A2E;
 Wed,  8 May 2024 19:33:15 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6D5DC113CC;
 Wed,  8 May 2024 19:33:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d269f0ef-0d71-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1715196794;
	bh=BqASjYSExQ+D7Yd+cxznVTfXPxXorUXWvTG6AeWlH3c=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=UQa1tWWSpnsQOi3dp5NKMdGe3cMEdHHphY+y/2nUymWnqqf/e8+UIEkdtLg5yPur6
	 0W4BBrcZkwt1SgmHsROH0hNeDsAJ+7hNCBDtKfr7OUUBtL1AeockbogA93jl4TBVpT
	 fIT8X+LXl2gz0CttL5Ciwzs3L6an8JFQKIIgmAmE=
Date: Wed, 8 May 2024 20:33:10 +0100
From: Greg KH <gregkh@linuxfoundation.org>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: cve@kernel.org, Jesper Dangaard Brouer <hawk@kernel.org>,
	netdev@vger.kernel.org,
	Ilias Apalodimas <ilias.apalodimas@linaro.org>, wei.liu@kernel.org,
	paul@xen.org, Jakub Kicinski <kuba@kernel.org>, kirjanov@gmail.com,
	dkirjanov@suse.de, kernel-team@cloudflare.com,
	security@xenproject.org, xen-devel@lists.xenproject.org,
	George Dunlap <dunlapg@umich.edu>
Subject: Re: [PATCH net] xen-netfront: Add missing skb_mark_for_recycle
Message-ID: <2024050802-playful-brick-0c67@gregkh>
References: <171154167446.2671062.9127105384591237363.stgit@firesoul>
 <CALUcmU=xOR1j9Asdv0Ny7x=o4Ckz80mDjbuEnJC0Z_Aepu0Zzw@mail.gmail.com>
 <CALUcmUkvpnq+CKSCn=cuAfxXOGU22fkBx4QD4u2nZYGM16DD6A@mail.gmail.com>
 <CALUcmUn0__izGAS-8gDL2h2Ceg9mdkFnLmdOgvAfO7sqxXK1-Q@mail.gmail.com>
 <CAFLBxZaLKGgrZRUDMQ+kCAYKD7ypzsjO55mWvkZHtMTBxdw51A@mail.gmail.com>
 <2024042544-jockstrap-cycle-ed93@gregkh>
 <9a2018c6-4efb-4bfe-b90f-531a072f0ef8@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <9a2018c6-4efb-4bfe-b90f-531a072f0ef8@citrix.com>

On Tue, May 07, 2024 at 02:57:08PM +0100, Andrew Cooper wrote:
> Hello,
> 
> Please could we request a CVE for "xen-netfront: Add missing
> skb_mark_for_recycle" which is 037965402a010898d34f4e35327d22c0a95cd51f
> in Linus' tree.
> 
> This is a kernel memory leak trigger-able from unprivileged userspace.
> 
> I can't see any evidence of this fix having been assigned a CVE thus far
> on the linux-cve-annouce mailing list.

CVE-2024-27393 is now created for this, thanks.

greg k-h


From xen-devel-bounces@lists.xenproject.org Wed May 08 19:38:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 19:38:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718934.1121492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4n7D-000103-SM; Wed, 08 May 2024 19:38:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718934.1121492; Wed, 08 May 2024 19:38:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4n7D-0000zw-Ox; Wed, 08 May 2024 19:38:11 +0000
Received: by outflank-mailman (input) for mailman id 718934;
 Wed, 08 May 2024 19:38:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6EkY=ML=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4n7D-0000zq-4g
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 19:38:11 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 801a4ea4-0d72-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 21:38:09 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-51f71e4970bso64111e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 12:38:09 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 q14-20020ac25a0e000000b0051e932297e2sm2628346lfn.297.2024.05.08.12.38.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 08 May 2024 12:38:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 801a4ea4-0d72-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715197089; x=1715801889; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=iqJvQsZ6lpxwkHhMuvOVW+fL3A34zOW2QxxsCIio7TM=;
        b=FbNhBJ79pXkP/Do85KIby1438DbjeEf/HMk/osqP9Wd8Kj8CUxdXUI9yk1G1Dz32l1
         1biCbCb0YKSeGGMzlzNdDT6MDDfe6rQUoCzqk6/cz18ZB/2Encb8/+u1ljtITBq9RjbK
         HYgqMi+mHEvT+54OENW7zwnchlTyH/ZsyBEKA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715197089; x=1715801889;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iqJvQsZ6lpxwkHhMuvOVW+fL3A34zOW2QxxsCIio7TM=;
        b=fDGRTlz350eDVa+fRKml5fzLb3YHLWeCDakZxbj+C7qt2EEr2822RLX4u9MM2zeuXt
         4RWDO1lqA3btaATsdg3jWC/1NAuBT64A24DpUuN7Cjzgi/uh8RuRZYNhCqXw8yCWeoNv
         5bZTyg+ZZCeKbvuydeOWx6L0TMfkK1nJs3fgiKmdtczu1yKonI80MYbbrg+JHdcCtAkl
         YbsHpqEVnJ5NxeewSMVEEoUE43R0d1LhxHiyli96Nm++mp4Ar+IMNBNEJce1NiW0MVTa
         G0YSlB+zdUI1D21oWzv9x31ssczJBVMLimKEXLcibAZNzKj2O2mlOKyZ+0uuWcwtqSjK
         Hb3Q==
X-Forwarded-Encrypted: i=1; AJvYcCVsijZUkoswpG0ATq+1g6JBBZcMZCuMMjscEz9Sd1zZuqiE8R+zONBBV83lsIQdSI8whpxnTha4yPluVpu9aywXAsPAD2hqBUBQQtRSTHs=
X-Gm-Message-State: AOJu0Yzw3HlD6awgNqON+HUoQpJr6zPl83j9iWKLazCiCxp66tvkfSPR
	4vnz25HlB0r5TmFg7WRjojtRSCwhxDgp+IDtJtkOlrVqwaWLrdC/mBLd3ARzUnE=
X-Google-Smtp-Source: AGHT+IH633bOTO1if1AIrdNr7B+rn7sl8WIprS74i2qagJs8y6TfsFW8imv2PT6q+vyzBLpJR7pKhA==
X-Received: by 2002:ac2:5388:0:b0:51a:e305:eb20 with SMTP id 2adb3069b0e04-5217c567012mr3047101e87.17.1715197089394;
        Wed, 08 May 2024 12:38:09 -0700 (PDT)
Message-ID: <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com>
Date: Wed, 8 May 2024 20:38:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2 2/3] xen/x86: enable altp2m at create domain
 domctl
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-3-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240508112323.38946-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/05/2024 12:23 pm, Roger Pau Monne wrote:
> Enabling it using an HVM param is fragile, and complicates the logic when
> deciding whether options that interact with altp2m can also be enabled.
>
> Leave the HVM param value for consumption by the guest, but prevent it from
> being set.  Enabling is now done using the misc_flags field in
> xen_arch_domainconfig.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - New in this version.

Ha.  So this is actually work that Petr has been wanting to do.

Petr has a series hoping to make it into 4.19 (x86: Make MAX_ALTP2M
configurable), which just missed out on this side of things.

altp2m is not architecture specific at all, and there's even support for
ARM out on the mailing list.  Therefore, the altp2m mode wants to be
common, just like the new MAX_ALTP2M setting already is.

Both fields can reasonably share uint32_t, but could you work with Petr
to make both halfs of this land cleanly.


As to the HVMPARAM, I'd really quite like to delete it.  It was always a
bodge, and there's a full set of HVMOP_altp2m_* for a guest to use.

> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index 20e83cf38bbd..dff790060605 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -708,13 +711,33 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          }
>      }
>  
> -    if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED )
> +    if ( config->arch.misc_flags & ~XEN_X86_MISC_FLAGS_ALL )
>      {
>          dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n",
>                  config->arch.misc_flags);
>          return -EINVAL;
>      }
>  
> +    if ( altp2m && (altp2m & (altp2m - 1)) )
> +    {
> +        dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
> +                config->flags);
> +        return -EINVAL;

I think this would be clearer to follow by having a 2 bit field called
altp2m_mode and check for <= 2.

> +    }
> +
> +    if ( altp2m && nested_virt )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "Nested virt and altp2m are mutually incompatible\n");

There's nothing inherently incompatible.  I think it's more that noone
had any interest in trying to make it work in combination with nested p2ms.

I'd phrase it as "not supported", rather than incompatible.

> +        return -EINVAL;
> +    }
> +
> +    if ( altp2m && !hap )
> +    {
> +        dprintk(XENLOG_INFO, "altp2m requires HAP\n");
> +        return -EINVAL;
> +    }

altp2m ought to work fine with shadow.  It's only if you want VMFUNC/#VE
acceleration that you depend on EPT.

Again, I'd phrase this as "not supported".

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 08 20:13:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 20:13:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718948.1121501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4nfO-0006cQ-FP; Wed, 08 May 2024 20:13:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718948.1121501; Wed, 08 May 2024 20:13:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4nfO-0006cJ-Ch; Wed, 08 May 2024 20:13:30 +0000
Received: by outflank-mailman (input) for mailman id 718948;
 Wed, 08 May 2024 20:13:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LtzS=ML=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s4nfN-0006cD-Dy
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 20:13:29 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6dd02e63-0d77-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 22:13:27 +0200 (CEST)
Received: from CH2PR16CA0012.namprd16.prod.outlook.com (2603:10b6:610:50::22)
 by SJ1PR12MB6265.namprd12.prod.outlook.com (2603:10b6:a03:458::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43; Wed, 8 May
 2024 20:13:20 +0000
Received: from CH3PEPF00000013.namprd21.prod.outlook.com
 (2603:10b6:610:50:cafe::9c) by CH2PR16CA0012.outlook.office365.com
 (2603:10b6:610:50::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46 via Frontend
 Transport; Wed, 8 May 2024 20:13:20 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH3PEPF00000013.mail.protection.outlook.com (10.167.244.118) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Wed, 8 May 2024 20:13:20 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 8 May
 2024 15:13:19 -0500
Received: from [172.30.155.127] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 8 May 2024 15:13:19 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6dd02e63-0d77-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hgjqxp57wULjH5aRXgKQ98eGXovk2wN3pamzJeXk4e/m32/mZMmu/qc+ZCPfDPoGWpGJMzIsvQAjVTnT0gloD/MYj2SsLG0RwDqDuGQJgVDc5NW944HxUqZgKR7q9kMH9+SmQRF2Xu6crlYUvpnHaDGnkfYqeh/igvgAfVGI7l9EDogKkYZa6/QlNZi2O6Es+CZdB1x5OiHlEJWnDI6KA8UxyU656q9FGAwhDqfgmb8SUh0aphEWFe+23GEh6fkO3bAMed16zZUCixeC/afbtBHIeDXG+IzhYf7ClUVJLISuB7IlxX+hdEXywlEjPbaDVFs4L+UgGEpSFS4XRjnJlw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JJfr/gIGRYHTbuChT5h9TCLU91Hqtc3OkwD+8w9q1dw=;
 b=eiQ+puA+cGg1Wa2W7z7sQSW+NMm2pN0ZZRj2zj39Fpg9pV6vTJrVpTet/kKInNZp0b4FOUWMhd3OqYImRLyHleiuHQ9z0OBX4WL2Dj0MeivZmIyXtSJZWwuLykmWAu9hdxLNuJNMAy5a9jA9XRD3quh3Dk0ulcWTjBpkdC0CyB/DPZi//rtjWKKtiYksPlmbSmxal/jvyOnQZWpG8IaMEdxKrG1or9ZI5Kpg/RSLYUqiXc5pBSVVRf0oRVP/fFEbVPW2Y3/qKMJMFeg+1bT36sgHhSjj5ycEJijcABuzP4bJHwHmDQDLBGg4hdGmIdO3ZZxNTJjmcjg5IOt4tPB/WA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JJfr/gIGRYHTbuChT5h9TCLU91Hqtc3OkwD+8w9q1dw=;
 b=kqQARjVTxlSuJ7lE77tDJS2Cn5kfwEE41Crfko1IimqnvufeiHYprmHdIyfZg3OGv5goXX20ChnpS1QR5QfcOmXfPy370vYl7oc9eKdr+DEnr6XfhMdXBgJKjoswRbSp2TgHk/PBYkDFcCFuQufpkZ+BurDYmq6OBhGUrECuZds=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ec9d5f1d-4686-4f7c-ad27-0a1b214050ed@amd.com>
Date: Wed, 8 May 2024 16:13:21 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v2 1/3] x86: allow to suppress port-alias probing
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <0c45155a-2beb-4e69-bca3-cdf42ba22f2b@suse.com>
 <bd116645-3451-47d7-8b8e-6e4b1af0680d@suse.com>
Content-Language: en-US
In-Reply-To: <bd116645-3451-47d7-8b8e-6e4b1af0680d@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH3PEPF00000013:EE_|SJ1PR12MB6265:EE_
X-MS-Office365-Filtering-Correlation-Id: 4d796059-08a5-4076-888f-08dc6f9b4e17
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Y0wwVWlwNTdPSmhtanM0VVNNNEY3RTZlaHhPVVlydCtxbzM2ZGVvOHNTQzV2?=
 =?utf-8?B?cGRpTDZUamFZd3AxSGxlSXdmM3VtUlk4ZlBqOVdYQ3F5YmFrYzNxdEpyRWZG?=
 =?utf-8?B?d05HSG5pNFdrV3ZNTXZIRjd6MVVWSjZuY3pHeUZQRVMvWWkvaTU2KzMxeUY4?=
 =?utf-8?B?RGgzbmJCZ0xHQWJoeGx0UWE3UUlnMTVEQ3BEaEhlTVE5RnBPZXkremEwLy9K?=
 =?utf-8?B?R010b3RuRGs3WkdNYTBQVkVTVjhEYktPR1FnSGhVc080aThweWIxUU1FaUo0?=
 =?utf-8?B?UG9lTTJuRzlOR3RBMU13cHFDalJUajQ4d1N3eXA3UFhKUWVFZUgzQWdBdThG?=
 =?utf-8?B?VmhmMldRVGFYanYrbWZ4NzdHS0FIaFdoVENheVlLaExXaWJBbXZwUjM0dktG?=
 =?utf-8?B?VEJLeGpLNVNYT0NOMzFUMVJCQkY1eHBMeEMvbUNWbFdDeXNaQUovdDJVMnl0?=
 =?utf-8?B?V3lrSlpjZ2dBSzUrWksyeHlTYVl5NDdiYjVMM0Zpa2d2QU9PbG9vU3NxRUN5?=
 =?utf-8?B?M3Mzc2ZHRmpEOXFzS0pSVVlMcGVVaHlQRmZ2NkgrU1dKcUw4MysxWkJpdW9I?=
 =?utf-8?B?Z2RPb2oveTRROUcvdHkwV3JveGpTdFRXUGVQUkk4dTlDVlkxN3JSeWpKbDNn?=
 =?utf-8?B?eUw0cVZzUDFVMHVsa3lEczhRclRPekFqOEkxT2Z4bHVRZGhHWUlkWGszYWhI?=
 =?utf-8?B?WFVVSWltYVBqbTNUVlJLYk5uRDIrY0gwWEw4T3BiVkp6ZG5IVkVyc25BTVRJ?=
 =?utf-8?B?d3R1cHFjVy9PejIxU0xBNnIxdTBtU05lRjBRTzB6blMyVXQ3YjF2OGRQM0Q2?=
 =?utf-8?B?SHdQU2J4TUdRQzE4eGw3MjFwRUFaSlVQUnlCbW1yS0tMOUpLWTB6TW14bDV6?=
 =?utf-8?B?Tk1UV1JNUm9EdkM1V2dzbDhBcGwvRW5IaU1pYVp6cUtHTFJ5d0o0TEpqK3Zr?=
 =?utf-8?B?Q0VDRk5taDM0OW9xTmZUdGRWMVVVNUZCUGNGUkdyMkJmVXlid0FtNzhpNFVw?=
 =?utf-8?B?VU5seDJCdExTZ1pCK1J3RUlPcVVmSm1ITzBrYklMbmRkWEpVZXU1a2liOXhU?=
 =?utf-8?B?S3JnZ1hPaXdKbFFYR0xLVXM5U1k5UWZVdFk1bmNDSE5qMmZWeTltNDMrMlE0?=
 =?utf-8?B?VzdheFpkblM4dS9uOFFTUzUzZnU5bEpobDFwZ28zUVl1L3hOYVB1VCtZZS80?=
 =?utf-8?B?dWZMWk84ZnBNZDU3SWt2U1hnNFpuTHlna2RWN1hmRjI5SmsxaENNM1psa0V1?=
 =?utf-8?B?RzY5aHVPNnRyTWlmMUhSOG9aL1oyQU92NjBTbXRiSVhMQVlSbGlGbzBSRCtU?=
 =?utf-8?B?Q0JSZUY5c3ZocXhEY2pIU0wvWlhDWnNSb1dYS3N6MVhFZFZac1RoNWpGMDJz?=
 =?utf-8?B?K0tIWDRhNStPWDQ0em9xcUtnN1R6dWw2c1dJWkFRUWoyWnNiTzQyOS8wTTk3?=
 =?utf-8?B?emt4QlJOaXlVMDVSSmFGWWVqcXVPMTFPU2piMTVzZ29yNkp6WVVWZFAwYU1S?=
 =?utf-8?B?Unh4S2dDWlpCUnpvblFNeFdyUkdud1Y5bDZ6c1ByREg4ZUErd2JSc205djRh?=
 =?utf-8?B?aXdSSXBudVlRMHpKZmJUNVJ1OExMdVN0ZjZTaWs2d3FNQTdPY0l0Ukw1VitS?=
 =?utf-8?B?a1h1bWU5MGcxUitXTTlYditLQTJDYmtZcHd6QW9EcXFGWVRGVUo4dkVRK0xl?=
 =?utf-8?B?M2JyRFcyS3dvdmswcURxNXJjZE9Bd1hPQ3lrOC94Q2YzRkZUcENxWjgwbVA0?=
 =?utf-8?B?akxuZDBIRmxXRDhoeDhTVEhTREMzckl0ZndFVGhqSXRsMUljZWQxM2o3WFJX?=
 =?utf-8?B?UUdhM0IzbkZlaHpqeVhoZ0ZScGtwbi85SGpiV0hhbkpSVy9iZ3pJRzQwcFVP?=
 =?utf-8?Q?3AVPDdnZEWghA?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 20:13:20.0078
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4d796059-08a5-4076-888f-08dc6f9b4e17
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH3PEPF00000013.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6265

On 2023-12-18 09:47, Jan Beulich wrote:
> By default there's already no use for this when we run in shim mode.
> Plus there may also be a need to suppress the probing in case of issues
> with it. Before introducing further port alias probing, introduce a
> command line option allowing to bypass it, default it to on when in shim
> mode, and gate RTC/CMOS port alias probing on it.
> 
> Requested-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>


From xen-devel-bounces@lists.xenproject.org Wed May 08 20:13:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 20:13:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718949.1121512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4nfp-00072l-OA; Wed, 08 May 2024 20:13:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718949.1121512; Wed, 08 May 2024 20:13:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4nfp-00072e-Ku; Wed, 08 May 2024 20:13:57 +0000
Received: by outflank-mailman (input) for mailman id 718949;
 Wed, 08 May 2024 20:13:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jfQ8=ML=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s4nfo-0006cD-Qo
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 20:13:56 +0000
Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com
 [2607:f8b0:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7edc3742-0d77-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 22:13:56 +0200 (CEST)
Received: by mail-ot1-x334.google.com with SMTP id
 46e09a7af769-6f05c253669so63692a34.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 13:13:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7edc3742-0d77-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715199234; x=1715804034; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wRggg1p/EbSSaYTs12aw4BJeQ5wwLzbWoK6Lvonni98=;
        b=MGZ4dbEIujxbOoHMbJvp069eh6TdVODBrVbJLSuCT1c/xFaiFEqfj0bCJqn56lzVMF
         x5XpiJlI22Cn2898oBQHuBOp1f59M1usXF27iiAgF1C43RE5IWBx4v/B/4OZpvxYdiYv
         kqj54Voy72hn8WsbsepWb9gSFjOIt/adqk935aojCjPEEgsuT6VREYXAKU5j2cSDp1Qj
         jqI96L0lYho0aAaE/STu/2Chz2UpWBIalsFy7QmvLQu5B6+ZWdW+qEHO+V+yyxeB6VbA
         zeZLWVQwvyuPN0aH9iqm8B1BOS1YjuU3RvUnLDDtBycCqpl35a/Ea+CLORAm90dRLGMd
         bXKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715199234; x=1715804034;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wRggg1p/EbSSaYTs12aw4BJeQ5wwLzbWoK6Lvonni98=;
        b=qfZeLGQ9AKjArF7b9n6HfCZJrr/HuJ3quEUQzUYk75BlpsymPvEsbXFVxvrYyKqYFB
         1ZOCa7bTi/fMB371KAFW87fq5V6kE/9nsC7QsSskdua1F0aAUg/C6rdvfBKRKCv8NB4s
         2Xf4BH0q63Pn8//H4IjQR6uBh09Te+IT34rs06L3knXhkwlfj42purokvljEpiZcWgDQ
         OtEoxCmy05uQKqI1SMM6/aDV3E9a9pR3eBHFsvb+2Wq568dhBZl2JAVs3ybZUh1w69WZ
         PJZBXp2mcQOlK+ehgL0xMFJQDfKsSElOAkvzIvjnu0RZI8TmsdWkw9ii+tyWuPYskN6b
         M2Gg==
X-Forwarded-Encrypted: i=1; AJvYcCXOviRRSy7MLbVImN1U3IYNbZHuwocaKtfzDc0CqYOZBhmcEcfXACQCxR8+8HJ8DCixSDn3QKXpf0R5oqhk9LbJuaJf8IxYxSGuayBmD1w=
X-Gm-Message-State: AOJu0YxbQso61m0cNGXUV45/AY2S+oDv1Qjcxp7sQfdxUfUiFaoJj44f
	CceDE0BtmY+m6AVvrBXz1h4OTQSPQabwwaa2us737eXRFTIb6gBW9PQq1Hi70PWxnTzwP+Bdd/B
	GvgdF7xqot3B4RFmRHVZvdOJFLHo=
X-Google-Smtp-Source: AGHT+IEe4eS8xFuTEwl8jPS+WU9ndSX+h6zDLmDeaWYPbe4GCu87M5JtEWMd490lMCuLVzYypMlXwFOaRbLRkxUyxtg=
X-Received: by 2002:a05:6870:724b:b0:229:7d01:7e03 with SMTP id
 586e51a60fabf-240985b7368mr4030866fac.43.1715199234609; Wed, 08 May 2024
 13:13:54 -0700 (PDT)
MIME-Version: 1.0
References: <20240508112323.38946-1-roger.pau@citrix.com> <20240508112323.38946-3-roger.pau@citrix.com>
 <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com>
In-Reply-To: <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Wed, 8 May 2024 22:13:43 +0200
Message-ID: <CAKBKdXjSvJMZb=Djd+DxObM311gwrZ4MkhzkSrx36spWnrd4-w@mail.gmail.com>
Subject: Re: [PATCH for-4.19 v2 2/3] xen/x86: enable altp2m at create domain domctl
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org, 
	Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, 
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	"Daniel P. Smith" <dpsmith@apertussolutions.com>, Tamas K Lengyel <tamas@tklengyel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 9:38=E2=80=AFPM Andrew Cooper <andrew.cooper3@citrix=
.com> wrote:
> Both fields can reasonably share uint32_t, but could you work with Petr
> to make both halfs of this land cleanly.

Hi, I think creating a new anonymous struct "altp2m" within `struct
domain` would be a good fit. uint16_t for my MAX_ALTP2M replacement
field will be enough, so the other uint16_t could be used for various
altp2m modes/flags.

P.


From xen-devel-bounces@lists.xenproject.org Wed May 08 20:47:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 20:47:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718954.1121521 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4oBh-0003js-6O; Wed, 08 May 2024 20:46:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718954.1121521; Wed, 08 May 2024 20:46:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4oBh-0003jl-3k; Wed, 08 May 2024 20:46:53 +0000
Received: by outflank-mailman (input) for mailman id 718954;
 Wed, 08 May 2024 20:46:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4oBf-0003jf-8F
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 20:46:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4oBe-0008Lt-PK; Wed, 08 May 2024 20:46:50 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4oBe-00088j-Ia; Wed, 08 May 2024 20:46:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=jqnr1r57jIullktxgnYkYrfprejWmFnMTsPi5pM5FEw=; b=balDH2KO80FfyQ/EKNgLBmR4Cg
	KljCRCJbKFLdmL1qUw1mCK/uQYLhsdF/jWmacqztteNcaA+AXKUmiKFJxSn9DDEEKBxU9c8IotXKN
	OWoAROhz4Pghvfh2y0lrxSZGMd3WnNmOhqRRtDfelvW9is275YBQNBeGgSCCzAk2lM/0=;
Message-ID: <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
Date: Wed, 8 May 2024 21:46:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 08/05/2024 08:49, Henry Wang wrote:
> On 5/8/2024 5:54 AM, Julien Grall wrote:
>> Hi Henry,
>>>> What if the DT overlay is unloaded and then reloaded? Wouldn't the 
>>>> same interrupt be re-used? As a more generic case, this could also 
>>>> be a new bitstream for the FPGA.
>>>>
>>>> But even if the interrupt is brand new every time for the DT 
>>>> overlay, you are effectively relaxing the check for every user (such 
>>>> as XEN_DOMCTL_bind_pt_irq). So the interrupt re-use case needs to be 
>>>> taken into account.
>>>
>>> I agree. I think IIUC, with your explanation here and below, could we 
>>> simplify the problem to how to properly handle the removal of the IRQ 
>>> from a running guest, if we always properly remove and clean up the 
>>> information when remove the IRQ from the guest? In this way, the IRQ 
>>> can always be viewed as a brand new one when we add it back.
>>
>> If we can make sure the virtual IRQ and physical IRQ is cleaned then yes.
>>
>>> Then the only corner case that we need to take care of would be...
>>
>> Can you clarify whether you say the "only corner case" because you 
>> looked at the code? Or is it just because I mentioned only one?
> 
> Well, I indeed checked the code and to my best knowledge the corner case 
> that you pointed out would be the only one I can think of.

Ok :). I was just checking you had a look as well.

[...]

>>> we have 3 possible states which can be read from LR for this case : 
>>> active, pending, pending and active.
>>> - I don't think we can do anything about the active state, so we 
>>> should return -EBUSY and reject the whole operation of removing the 
>>> IRQ from running guest, and user can always retry this operation.
>>
>> This would mean a malicious/buggy guest would be able to prevent a 
>> device to be de-assigned. This is not a good idea in particular when 
>> the domain is dying.
>>
>> That said, I think you can handle this case. The LR has a bit to 
>> indicate whether the pIRQ needs to be EOIed. You can clear it and this 
>> would prevent the guest to touch the pIRQ. There might be other 
>> clean-up to do in the vGIC datastructure.
> 
> I probably misunderstood this sentence, do you mean the EOI bit in the 
> pINTID field? I think this bit is only available when the HW bit of LR 
> is 0, but in our case the HW is supposed to be 1 (as indicated as your 
> previous comment). Would you mind clarifying a bit more? Thanks!

You are right, ICH_LR.HW will be 1 for physical IRQ routed to a guest. 
What I was trying to explain is this bit could be cleared (with 
ICH_LR.pINTD adjusted).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 08 21:05:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:05:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718959.1121532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4oTw-0006wW-RS; Wed, 08 May 2024 21:05:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718959.1121532; Wed, 08 May 2024 21:05:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4oTw-0006wP-NE; Wed, 08 May 2024 21:05:44 +0000
Received: by outflank-mailman (input) for mailman id 718959;
 Wed, 08 May 2024 21:05:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4oTv-0006wJ-8X
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:05:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4oTu-0000HW-Qn; Wed, 08 May 2024 21:05:42 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4oTu-0001Ah-LQ; Wed, 08 May 2024 21:05:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=uMKXVQiv+rd0LyM0gDIWnH+cpu+xUY0PsuYtzul+dEE=; b=lvMUbxF+GCm3t1/ndJtXpWaNQK
	ciz1SRMoxOAS0fHiK2xLNz4LbFbTASPG1bSbtzPinWzTj8qTxcMChJIqHQYQZPsFiO5UM8x0OsEE8
	CbaUd1tF0zaYeNLB3aW2C0VCm9E18k7Qpj1Lu42FIR4F5K7H2PZuORzO31yhnHSycLvM=;
Message-ID: <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
Date: Wed, 8 May 2024 22:05:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 07/05/2024 14:30, Luca Fancellu wrote:
> Hi Julien,

Hi Luca,

>> On 7 May 2024, at 14:20, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Luca,
>>
>> On 23/04/2024 09:25, Luca Fancellu wrote:
>>> From: Penny Zheng <Penny.Zheng@arm.com>
>>> We are doing foreign memory mapping for static shared memory, and
>>> there is a great possibility that it could be super mapped.
>>
>> Is this because we are mapping more than one page at the time? Can you point me to the code?
> 
> So, to be honest this patch was originally in Penny’s serie, my knowledge of this side of the codebase
> is very limited and so I pushed this one basically untouched.
> 
>  From what I can see in the serie the mappings are made in handle_shared_mem_bank, and map_regions_p2mt
> is called for one page at the time (allocated through the function allocate_domheap_memory (new function introduced in
> the serie).
> 
> So is that the case that this patch is not needed?

I looked at the code and, if I am not mistake, we are passing 
PFN_DOWN(psize) to map_regions_p2mt. At the moment, it is unclear to me 
why would psize be < PAGE_SIZE.

>>> But today, p2m_put_l3_page could not handle superpages.
>>
>> This was done on purpose. Xen is not preemptible and therefore we need to be cautious how much work is done within the p2m code.
>>
>> With the below proposal, for 1GB mapping, we may end up to call put_page() up to 512 * 512 = 262144 times. put_page() can free memory. This could be a very long operation.
>>
>> Have you benchmark how long it would take?
> 
> I did not, since its purpose was unclear to me and was not commented in the last serie from Penny.

Honestly, I can't remember why it wasn't commented.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718970.1121588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0D-0004wt-KZ; Wed, 08 May 2024 21:39:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718970.1121588; Wed, 08 May 2024 21:39:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0D-0004uc-Gk; Wed, 08 May 2024 21:39:05 +0000
Received: by outflank-mailman (input) for mailman id 718970;
 Wed, 08 May 2024 21:39:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4p0C-00044I-AS
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:39:04 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 612663b5-0d83-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 23:38:59 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2D09B800A5;
 Wed,  8 May 2024 22:38:59 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 06DB42017F; Wed,  8 May 2024 22:38:59 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 612663b5-0d83-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204339;
	bh=bIUl73UyV+ayA8Z1TZOYtaBLcFb3iD439KxaXODeE/U=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=tCaiftnLTPj7TR5XUPM/tF9fgWEtaFVpTkhb0aVmdLKPFmotpT+z0OUNDZ6adEaJH
	 BRybjeI8ofZSoM9mirE/VKFFGGaAQpuDnFRHbHNuoOBv8451uLn/TJ5aXsnRDk6Snq
	 wNxjXBZTaPN67mOpMMWl9GfeWbfVsyeLBx2I9Fkg=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 5/5] tools/examples: Example Linux bridge VLAN config
Date: Wed,  8 May 2024 22:38:24 +0100
Message-Id: <20240508213824.55824-6-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new directory linux-bridge-vlan with examples files showing
how to configure systemd-networkd to support a bridge VLAN
configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/examples/linux-bridge-vlan/README       | 68 +++++++++++++++++++
 tools/examples/linux-bridge-vlan/br0.netdev   |  7 ++
 tools/examples/linux-bridge-vlan/br0.network  |  8 +++
 .../examples/linux-bridge-vlan/enp0s0.network | 16 +++++
 4 files changed, 99 insertions(+)
 create mode 100644 tools/examples/linux-bridge-vlan/README
 create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
 create mode 100644 tools/examples/linux-bridge-vlan/br0.network
 create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network

diff --git a/tools/examples/linux-bridge-vlan/README b/tools/examples/linux-bridge-vlan/README
new file mode 100644
index 0000000000..83b9fa3fd6
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/README
@@ -0,0 +1,68 @@
+Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
+=====================================================================
+
+Introduction
+------------
+
+This directory contains example files to be placed in /etc/systemd/network
+to enable a single bridge with multiple VLAN support.
+
+The example is to support the scenario where the Xen host network interface
+is connected to an Ethernet switch configured as a trunk port. Each domain
+VIF can then be configured with one or more VLAN IDs, one of which will be
+the PVID.
+
+The example files create a bridge device called br0, with a physical interface 
+called enp0s0. You will need to update this with your system's device name.
+
+Key points of the configuration are:
+
+1. In br0.netdev, VLANFiltering=on is set. This is required to ensure the
+   VLAN tags are handled correctly.  If it is not set then the packets
+   from the VIF interfaces will not have the correct VLAN tags set.
+
+2. In br0.network, a system IPv4 address is configured that can be updated
+   according to your local network settings.
+
+3. In enp0s0.network, Bridge=br0 sets the bridge device to connect to. There
+   is also a [BridgeVLAN] section for each VLAN you want to give access
+   to the switch. Note, if you want to create an internal VLAN private to
+   the host, do not include its VLAN ID in this file.
+
+
+Domain configuration
+--------------------
+
+Add the vlan= keyword to the vif definition in the domain. The simplest
+and most common example is a domain that wishes to connect to a single VLAN:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10' ]
+
+If you wish to configure a domain to route between two VLANs, you have two
+options. Option 1 is to create multiple interfaces on different VLANs:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10',
+	'max=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=20' ]
+
+Alternatively, you can create single interface:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20' ]
+
+In the domain, you would, for example, use enX0 for VLAN 10 and enX0.20 for 
+VLAN 20.
+
+
+Hints and tips
+--------------
+
+You can run the following commands on dom0 or a driver domain:
+
+1. To check if vlan_filtering is enabled:
+   # cat /sys/devices/virtual/net/<name>/bridge/vlan_filtering
+
+2. To check the bridge port VLAN assignments:
+   # bridge vlan
+
+3. To check the vlan setting in the xenstore (dom0 only):
+   # xenstore-ls -f | grep 'vlan ='
+
diff --git a/tools/examples/linux-bridge-vlan/br0.netdev b/tools/examples/linux-bridge-vlan/br0.netdev
new file mode 100644
index 0000000000..ae1fe487c3
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/br0.netdev
@@ -0,0 +1,7 @@
+[NetDev]
+Name=br0
+Kind=bridge
+MACAddress=xx:xx:xx:xx:xx:xx
+
+[Bridge]
+VLANFiltering=on
diff --git a/tools/examples/linux-bridge-vlan/br0.network b/tools/examples/linux-bridge-vlan/br0.network
new file mode 100644
index 0000000000..b56203b66a
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/br0.network
@@ -0,0 +1,8 @@
+[Match]
+Name=br0
+
+[Network]
+DNS=8.8.8.8
+#Domains=example.com
+Address=10.1.1.10/24
+Gateway=10.1.1.1
diff --git a/tools/examples/linux-bridge-vlan/enp0s0.network b/tools/examples/linux-bridge-vlan/enp0s0.network
new file mode 100644
index 0000000000..6ee3154dfc
--- /dev/null
+++ b/tools/examples/linux-bridge-vlan/enp0s0.network
@@ -0,0 +1,16 @@
+[Match]
+Name=enp0s0
+
+[Network]
+Bridge=br0
+
+# If Jumbo frames are required
+#[Link]
+#MTUBytes=9000
+
+[BridgeVLAN]
+VLAN=10
+
+[BridgeVLAN]
+VLAN=20
+
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718968.1121571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0B-0004Yg-ST; Wed, 08 May 2024 21:39:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718968.1121571; Wed, 08 May 2024 21:39:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0B-0004Y3-OY; Wed, 08 May 2024 21:39:03 +0000
Received: by outflank-mailman (input) for mailman id 718968;
 Wed, 08 May 2024 21:39:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4p0A-00044I-A7
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:39:02 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 611416e4-0d83-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 23:38:59 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2010F8007B;
 Wed,  8 May 2024 22:38:59 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id EC76220166; Wed,  8 May 2024 22:38:58 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 611416e4-0d83-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204339;
	bh=NwExQyE50t0WzUcJcVbwpgO12YLzpS7FSJX0VovsD2w=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=0p+fVAK8o9JhCJg0v5dza9u9Fi5JpN2LJfm8aDeFs5HqXP3kQATXlN3LphiN+I8v9
	 I4A0NE276Pj55g6Dwknjzd5zrWUIygNHMKk4m++Tp4vtfaVHaYfuBqgcqAuoUDVUuE
	 RH/FHx9hXn/S7UVSt+9h6UYjOZZ0eD/DlRWWWOjM=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 1/5] tools/libs/light: Add vlan field to libxl_device_nic
Date: Wed,  8 May 2024 22:38:20 +0100
Message-Id: <20240508213824.55824-2-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add `vlan' string field to libxl_device_nic, to allow a VLAN
configuration to be specified for the VIF when adding it to the
bridge device.

Update libxl_nic.c to read and write the vlan field from the
xenstore.

This provides the capability for supported operating systems (e.g.
Linux) to perform VLAN filtering on bridge ports.  The Xen
hotplug scripts need to be updated to read this information from
the xenstore and perform the required configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/libs/light/libxl_nic.c     | 10 ++++++++++
 tools/libs/light/libxl_types.idl |  1 +
 2 files changed, 11 insertions(+)

diff --git a/tools/libs/light/libxl_nic.c b/tools/libs/light/libxl_nic.c
index d6bf06fc34..d861e3726d 100644
--- a/tools/libs/light/libxl_nic.c
+++ b/tools/libs/light/libxl_nic.c
@@ -233,6 +233,11 @@ static int libxl__set_xenstore_nic(libxl__gc *gc, uint32_t domid,
         flexarray_append(back, GCSPRINTF("%u", nic->mtu));
     }
     
+    if (nic->vlan) {
+        flexarray_append(back, "vlan");
+        flexarray_append(back, libxl__strdup(gc, nic->vlan));
+    }
+
     flexarray_append(back, "bridge");
     flexarray_append(back, libxl__strdup(gc, nic->bridge));
     flexarray_append(back, "handle");
@@ -313,6 +318,11 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const char *libxl_path,
         nic->mtu = LIBXL_DEVICE_NIC_MTU_DEFAULT;
     }
 
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                GCSPRINTF("%s/vlan", libxl_path),
+				(const char **)(&nic->vlan));
+    if (rc) goto out;
+
     rc = libxl__xs_read_checked(gc, XBT_NULL,
                                 GCSPRINTF("%s/mac", libxl_path), &tmp);
     if (rc) goto out;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 7d8bd5d216..5c510dc272 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -809,6 +809,7 @@ libxl_device_nic = Struct("device_nic", [
     ("backend_domname", string),
     ("devid", libxl_devid),
     ("mtu", integer),
+    ("vlan", string),
     ("model", string),
     ("mac", libxl_mac),
     ("ip", string),
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718965.1121541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ozz-0003oT-5g; Wed, 08 May 2024 21:38:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718965.1121541; Wed, 08 May 2024 21:38:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4ozz-0003oM-2x; Wed, 08 May 2024 21:38:51 +0000
Received: by outflank-mailman (input) for mailman id 718965;
 Wed, 08 May 2024 21:38:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4ozx-0003oG-QA
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:38:50 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 59e204b1-0d83-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 23:38:47 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id B8C968007B;
 Wed,  8 May 2024 22:38:46 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 8E19520166; Wed,  8 May 2024 22:38:46 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59e204b1-0d83-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204326;
	bh=dzK53Up7iGU4Nhei0LzAbBYnoantiYERlMzBBwEAOiI=;
	h=From:To:Cc:Subject:Date:From;
	b=lLiS2XU8pLls9rdCTM73I+hGi7o7kCWmOZzWEa/w18nEb0L43gmFLTWhBn9McNu6K
	 XTmYx7dcfeTUP5CfztVHDd/7lhN+txeiFnr42AAwYbjvet3CYFs0Z6ev3H7SP4rlVr
	 R6Dam92xOJYPFKbZsROMCtLJ2WOV47CDCxoLeVi4=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 0/5] Add bridge VLAN support
Date: Wed,  8 May 2024 22:38:19 +0100
Message-Id: <20240508213824.55824-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hello all,

I realised over the weekend that there is a valid use case for providing
a VIF to a domain that has access to multiple VLANs, e.g. a router. Yes,
you can create a VIF per VLAN, but if you start having several VLANs (as
I do), it would be nicer to create a single interface that has access to
all the relevant VLANs (e.g. enX0.10, enX0.20, etc.).

So, version 2 changes the name and type of the parameter from an integer
called `vid' to a string called `vlan'. The vlan parameter is then
parsed by the vif-bridge script (actually, the functions called by it in
xen-network-common.sh).

As it quite a common practice to allocate VLANs in round numbers, I also
implemented the ability to specify contiguous or non-contiguous ranges.
You can specify whether a VLAN is tagged or untagged, and which VLAN is
the PVID (only one PVID is allowed).  For example,

vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20-29' ]

will setup the VIF so that 10 is the PVID and VLAN IDs 20 through 29
are permitted with tags. Another example:

vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=1p/10+10x9' ]

will setup the bridge to set 1 as the PVID and permit access with
tags for VLAN IDs 10, 20, 30, 40, 50, 60, 70, 80 and 90.

This patch set enables this capability as follows:

1. Adds `vlan' as a new member of the libxl_device_nic structure;
2. Adds support to read and write the vlan parameter from the xenstore;
3. Adds `vlan' as a new keyword for the vif configuration option;
4. Adds support to assign the bridge VLANs in the Linux hotplug scripts;
5. Updated xl-network-configuration(5) manpage and example configs.

Original blurb below:

For many years I have been configuring VLANs on my Linux Dom0 by
creating VLAN interfaces for each VLAN I wanted to connect a domain
to and then a corresponding bridge. So I would tend to have things
like:

enp0s0    -> br0     -> vif1, vif2
enp0s0.10 -> br0vl10 -> vif3, vif4
enp0s0.20 -> br0vl20 -> vif5
dummy0    -> br1     -> vif6

I recently discovered that iproute2 supports creating bridge VLANs that
allows you to assign a VLAN to each of the interfaces associated to a
bridge. This allows a greatly simplified configuration where a single
bridge can support all the domains, and the iproute2 bridge command can
assign each VIF to the required VLAN.  This looks like this:

# bridge vlan
port              vlan-id  
enp0s0            1 PVID Egress Untagged
                  10
                  20
br0               1 PVID Egress Untagged
vif1.0            1 PVID Egress Untagged
vif2.0            1 PVID Egress Untagged
vif3.0            10 PVID Egress Untagged
vif4.0            10 PVID Egress Untagged
vif5.0            20 PVID Egress Untagged
vif6.0            30 PVID Egress Untagged

This patch set enables this capability as follows:

1. Adds `vid' as a new member of the libxl_device_nic structure;
2. Adds support to read and write vid from the xenstore;
3. Adds `vid' as a new keyword for the vif configuration option;
4. Adds support for assign the bridge VLAN in the Linux hotplug scripts.

I don't believe NetBSD or FreeBSD support this capability, but if they
do please point me in the direction of some documentation and/or examples.

NB: I'm not very familiar with Xen code base so may have missed
something important, although I have tested it and it is working well
for me.

Cheers,

Leigh.


Leigh Brown (5):
  tools/libs/light: Add vlan field to libxl_device_nic
  tools/xl: add vlan keyword to vif option
  tools/hotplug/Linux: Add bridge VLAN support
  docs/man: document VIF vlan keyword
  tools/examples: Example Linux bridge VLAN config

 docs/man/xl-network-configuration.5.pod.in    |  38 ++++++
 tools/examples/linux-bridge-vlan/README       |  68 +++++++++++
 tools/examples/linux-bridge-vlan/br0.netdev   |   7 ++
 tools/examples/linux-bridge-vlan/br0.network  |   8 ++
 .../examples/linux-bridge-vlan/enp0s0.network |  16 +++
 tools/hotplug/Linux/xen-network-common.sh     | 111 ++++++++++++++++++
 tools/libs/light/libxl_nic.c                  |  10 ++
 tools/libs/light/libxl_types.idl              |   1 +
 tools/xl/xl_parse.c                           |   2 +
 9 files changed, 261 insertions(+)
 create mode 100644 tools/examples/linux-bridge-vlan/README
 create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
 create mode 100644 tools/examples/linux-bridge-vlan/br0.network
 create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network

-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718967.1121558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0A-00046d-OH; Wed, 08 May 2024 21:39:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718967.1121558; Wed, 08 May 2024 21:39:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0A-00045T-GT; Wed, 08 May 2024 21:39:02 +0000
Received: by outflank-mailman (input) for mailman id 718967;
 Wed, 08 May 2024 21:39:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4p09-00044I-KG
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:39:01 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 611b41a1-0d83-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 23:38:59 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2561B8009E;
 Wed,  8 May 2024 22:38:59 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id F01BE20167; Wed,  8 May 2024 22:38:58 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 611b41a1-0d83-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204339;
	bh=aK+rR4WSKsfVDFqSo21YMhoedb8+5SFZWoGq++M/1CY=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=z9Km2AAeaGZp8E5gVbHMQzWSZ/jxogBa0gsn7AqAHpBmYIJKLxo5nZbuQOgazXjGt
	 iyfZer4aO/4nefLHaj3Ga8/kZwRFta8c290LOz6Sqa8V801mBqVLH7eN1znO7hJm7X
	 1smoj4EfSIQ+IRk/HzDu82G0wjjuqOHaRLm2X8GI=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 2/5] tools/xl: add vlan keyword to vif option
Date: Wed,  8 May 2024 22:38:21 +0100
Message-Id: <20240508213824.55824-3-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update parse_nic_config() to support a new `vlan' keyword. This
keyword specifies the VLAN configuration to assign to the VIF when
attaching it to the bridge port, on operating systems that support
the capability (e.g. Linux). The vlan keyword will allow one or
more VLANs to be configured on the VIF when adding it to the bridge
port. This will be done by the vif-bridge script and functions.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/xl/xl_parse.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index ed983200c3..7546fe7e7a 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -565,6 +565,8 @@ int parse_nic_config(libxl_device_nic *nic, XLU_Config **config, char *token)
         nic->devid = parse_ulong(oparg);
     } else if (MATCH_OPTION("mtu", token, oparg)) {
         nic->mtu = parse_ulong(oparg);
+    } else if (MATCH_OPTION("vlan", token, oparg)) {
+        replace_string(&nic->vlan, oparg);
     } else if (!strcmp("trusted", token)) {
         libxl_defbool_set(&nic->trusted, true);
     } else if (!strcmp("untrusted", token)) {
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718969.1121582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0D-0004oc-48; Wed, 08 May 2024 21:39:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718969.1121582; Wed, 08 May 2024 21:39:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0D-0004oV-12; Wed, 08 May 2024 21:39:05 +0000
Received: by outflank-mailman (input) for mailman id 718969;
 Wed, 08 May 2024 21:39:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4p0B-00044I-A9
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:39:03 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 611f7b26-0d83-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 23:38:59 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 292668009F;
 Wed,  8 May 2024 22:38:59 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 030132017E; Wed,  8 May 2024 22:38:59 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 611f7b26-0d83-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204339;
	bh=+Btd9KVZELds4IWrVvm32cKo9jGdEkoWQ7GZ1KHgtYw=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=iHPObwfQZEAy1JyUCQI4BAtqbCaTT/dlCv37rO2Tt0tG39UPsSmaCC9mwPwyFt4IQ
	 +rimKjAj/Q+t6x+DEPJ27PJCTpSOZdmsPehguoNqyyneNPNuf2gvVcmRHkcrIvg8fC
	 q2CkZe+gi2jwfxPg+R2DcTX6MLSnk/wLidltzK4g=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 4/5] docs/man: document VIF vlan keyword
Date: Wed,  8 May 2024 22:38:23 +0100
Message-Id: <20240508213824.55824-5-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Document the new `vlan' keyword in xl-network-configuration(5).

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 docs/man/xl-network-configuration.5.pod.in | 38 ++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/docs/man/xl-network-configuration.5.pod.in b/docs/man/xl-network-configuration.5.pod.in
index f3e379bcf8..c35c0922b3 100644
--- a/docs/man/xl-network-configuration.5.pod.in
+++ b/docs/man/xl-network-configuration.5.pod.in
@@ -259,6 +259,44 @@ Specifies the MTU (i.e. the maximum size of an IP payload, exclusing headers). T
 default value is 1500 but, if the VIF is attached to a bridge, it will be set to match
 unless overridden by this parameter.
 
+=head2 vlan
+
+Specifies the VLAN configuration. The format of this parameter is one or more
+VLAN IDs or ranges separated by forward slashes. Each term can be:
+
+=over
+
+=item *
+
+B<vlan> - a single VLAN ID in the range 1 to 4094. This can optionally followed
+by a B<p> to indicate the PVID or by a B<u> to indicate an untagged VLAN. C<p>
+implies B<u>.
+
+=item *
+
+B<vlan1>-B<vlan2> - a range of VLAN IDs from B<vlan1> to B<vlan2>, both between
+1 and 4094 and B<vlan1> being less than or equal to B<vlan2>. This can be
+optionally followed by a B<u> to indicate that the range of VLANs are untagged.
+
+=item *
+
+B<vlan>+B<offset>xB<count> - describing a range of VLAN IDs starting at B<vlan>
+with B<count> additional entries, each incremented by B<offset>. This can be 
+optionally followed by a B<u> to indicate that the range of VLANs are untagged.
+
+=back
+
+Note, one VLAN ID must be marked as the PVID. In the case of a vlan 
+specification consisting of a single VLAN ID (e.g. C<vlan=10>), the B<p> suffix
+may be omitted. Specifying more than one untagged VLAN ID is an advanced 
+configuration - use with caution.
+
+For example:
+
+        'vlan=10' -- meaning a single VLAN that is the PVID.
+	'vlan=10p/20' -- VLAN 10 is the PVID and VLAN 20 is tagged.
+	'vlan=10p/100+10x4' -- VLANs 10, 100, 110, 120, 130, 140, 150.
+
 =head2 trusted / untrusted
 
 An advisory setting for the frontend driver on whether the backend should be
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718966.1121552 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0A-00044o-Dn; Wed, 08 May 2024 21:39:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718966.1121552; Wed, 08 May 2024 21:39:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4p0A-00044h-9M; Wed, 08 May 2024 21:39:02 +0000
Received: by outflank-mailman (input) for mailman id 718966;
 Wed, 08 May 2024 21:39:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nd4S=ML=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s4p09-0003oG-5k
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:39:01 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 61228ae4-0d83-11ef-b4bb-af5377834399;
 Wed, 08 May 2024 23:38:59 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2C084800A4;
 Wed,  8 May 2024 22:38:59 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id F408220174; Wed,  8 May 2024 22:38:58 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61228ae4-0d83-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715204339;
	bh=tu/eViqs6PEWINbRyoFYh4zDxVMHi/C0o7aOqYaz5Tc=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=k5+w8Px9oNv5TbaY4ri+l5AsGUbFSwrz1Wa8vA2/fe9HwFiUaR9ZQj6cG55Y2aMkH
	 WKTx2PCeTVvDpRvLRtbcNSadHZ9fkAzziz5raPHnD9oYNIZzYgiKP9SYR8eLYI7fgS
	 w+936cXMqEfkSAVs76rum/DTRlHQSkMHbRwvh4TY=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	anthony.perard@citrix.com,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [RFC PATCH v2 3/5] tools/hotplug/Linux: Add bridge VLAN support
Date: Wed,  8 May 2024 22:38:22 +0100
Message-Id: <20240508213824.55824-4-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update add_to_bridge shell function to read the vlan parameter
from xenstore and set the bridge VLAN configuration for the VID.

Add additional helper functions to parse the vlan specification,
which consists of one or more of the follow:

a) single VLAN (e.g. 10).
b) contiguous range of VLANs (e.g. 10-15).
c) discontiguous range with base, increment and count
   (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).

A single VLAN can be suffixed with "p" to indicate the PVID, or
"u" to indicate untagged. A range of VLANs can be suffixed with
"u" to indicate untagged.  A complex example would be:

   vlan=1p/10-15/20-25u

This capability only works when using the iproute2 bridge command,
so a warning is issued if the vlan parameter is set and the bridge
command is not available, as it will be ignored.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/hotplug/Linux/xen-network-common.sh | 111 ++++++++++++++++++++++
 1 file changed, 111 insertions(+)

diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Linux/xen-network-common.sh
index 42fa704e8d..d9fb4f7355 100644
--- a/tools/hotplug/Linux/xen-network-common.sh
+++ b/tools/hotplug/Linux/xen-network-common.sh
@@ -121,10 +121,113 @@ create_bridge () {
     fi
 }
 
+_vif_vlan_add() {
+    # References vlans, pvid and error variables from the calling function
+    local -i vid=$1
+    local flag=${2:-}
+
+    if (( vid < 1 || vid > 4094 )) ;then
+        error="vlan id $vid not between 1 and 4094"
+        return
+    fi
+    if [[ -n "${vlans[$vid]}" ]] ;then
+        error="vlan id $vid specified more than once"
+        return
+    fi
+    case $flag in
+     p) if (( pvid != 0 )) ;then
+            error="more than one pvid specified ($vid and $pvid)"
+            return
+        fi
+        pvid=$vid
+        vlans[$vid]=p ;;
+     u) vlans[$vid]=u ;;
+     *) vlans[$vid]=t ;;
+    esac
+}
+
+_vif_vlan_parse_term() {
+    # References error variable from the calling function
+    local vid incr last term=${1:-}
+
+    if [[ $term =~ ^([0-9]+)([pu])?$ ]] ;then
+        _vif_vlan_add ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}
+    elif [[ $term =~ ^([0-9]+)-([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        last=${BASH_REMATCH[2]}
+        if (( last >= vid )) ;then
+            for (( ; vid<=last; vid++ )) ;do
+                _vif_vlan_add $vid ${BASH_REMATCH[3]}
+            done
+        else
+	    error="invalid vlan id range: $term"
+        fi
+    elif [[ $term =~ ^([0-9]+)\+([0-9]+)x([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        incr=${BASH_REMATCH[2]}
+        for (( j=${BASH_REMATCH[3]}; j>0; --j, vid+=incr ))
+        do
+            _vif_vlan_add $vid ${BASH_REMATCH[4]}
+        done
+    else
+        error="invalid vlan specification: $term"
+    fi
+}
+
+_vif_vlan_validate_pvid() {
+    # References vlans and pvid variables from the calling function
+    if (( pvid == 0 )) ;then
+        if (( ${#vlans[@]} == 1 )) ;then
+            vlans[${!vlans[*]}]=p
+        else
+            error="pvid required for multiple vlan ids"
+        fi
+    fi
+}
+
+_vif_vlan_setup() {
+    # References vlans and dev variable from the calling function
+    local vid cmd
+
+    bridge vlan del dev "$dev" vid 1
+    for vid in ${!vlans[@]} ;do
+        cmd="bridge vlan add dev '$dev' vid $vid"
+        case ${vlans[$vid]} in
+             p) cmd="$cmd pvid untagged" ;;
+             u) cmd="$cmd untagged" ;;
+             t) ;;
+        esac
+        eval "$cmd"
+    done
+}
+
+_vif_vlan_membership() {
+    # The vlans, pvid, dev and error variables are used by sub-functions
+    local -A vlans=()
+    local -a terms=()
+    local -i i pvid=0
+    local dev=$1 error=""
+
+    # Split the vlan specification string into its terms
+    readarray -d / -t terms <<<$2
+    for (( i=0; i<${#terms[@]}; ++i )) ;do
+        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}
+        [[ -n "$error" ]] && break
+    done
+
+    [[ -z "$error" ]] && _vif_vlan_validate_pvid
+    [[ -z "$error" ]] && _vif_vlan_setup
+    [[ -z "$error" ]] && return 0
+
+    log error "$error"
+    return 1
+}
+
 # Usage: add_to_bridge bridge dev
 add_to_bridge () {
     local bridge=$1
     local dev=$2
+    local vlan=$(xenstore_read_default "$XENBUS_PATH/vlan" "")
 
     # Don't add $dev to $bridge if it's already on the bridge.
     if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
@@ -134,6 +237,14 @@ add_to_bridge () {
         else
             ip link set ${dev} master ${bridge}
         fi
+        if [ -n "${vlan}" ] ;then
+            if which bridge >&/dev/null; then
+                log debug "configuring VLANs for ${dev} on ${bridge}"
+                _vif_vlan_membership "${dev}" "${vlan}"
+            else
+                log warning "bridge command not available, ignoring vlan config"
+            fi
+        fi
     else
         log debug "$dev already on bridge $bridge"
     fi
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed May 08 21:52:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 21:52:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.718993.1121601 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pDE-0001JI-Pq; Wed, 08 May 2024 21:52:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 718993.1121601; Wed, 08 May 2024 21:52:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pDE-0001JB-N6; Wed, 08 May 2024 21:52:32 +0000
Received: by outflank-mailman (input) for mailman id 718993;
 Wed, 08 May 2024 21:52:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DEwc=ML=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s4pDC-0001Iz-Qo
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 21:52:31 +0000
Received: from wfout8-smtp.messagingengine.com
 (wfout8-smtp.messagingengine.com [64.147.123.151])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 410a3226-0d85-11ef-909c-e314d9c70b13;
 Wed, 08 May 2024 23:52:27 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id 105A21C0010D;
 Wed,  8 May 2024 17:52:22 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Wed, 08 May 2024 17:52:23 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 8 May 2024 17:52:21 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 410a3226-0d85-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715205142;
	 x=1715291542; bh=EhhU/8hFj1pYY+xpTRYiDtTgGET1KkwtT2kVDMcQm0A=; b=
	lYKZWkYmyVuhwoFvAJzm3zvM0L6k7Fz+Wb8qDNKIXfOJ9JFR2nlAHNI6PXilTHcv
	4lDFRuObmhRQnC/HslsGuFK++V33UILAV1jZRvDrq3Rww9ZknVKmh02iiGhGpzBB
	+dmUb019KPNyJ9hRLQ8KowprsyXwrSnwWPj8HMfsNH+4misK5nKcvTB2MVlcb0fH
	POCHkpTb37xxn01y0oP0Spk3WcHKHaobmgi0r0Nt1b1xD39Uag/FMqzauBDYfbnR
	hBpa6sN5ZP3bdBgT1Og3OcNiJ3MKOjOY5jDiXx/8rTfnsNvpfB3DFIwnVaH7qhQC
	ryeVMQOFEiA6ULG0jY3g2g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715205142; x=1715291542; bh=EhhU/8hFj1pYY+xpTRYiDtTgGET1
	KkwtT2kVDMcQm0A=; b=WOS4I+hEhwTyX5b9o0+l0Su/Q12ibU70aljgHgjkRM3y
	XEbtRBNUk8aaRfQrj7OG3d6Xdsr4eolllRZ9s0HXU9r6EZ7rdXNixqvbhOjsOPpW
	8y5GekiNXO1LY5ga+y3UtJ9B2TElX+H88fsfSe/L/PfpyAxqT3+Xttnwuk9wPWOF
	+jsrolFKzLfTMyozKlp0Rm7ObVAYy3w69b3D8SyAp1Aj+0ZAbvQCUogYuIPwGJXl
	sPUF6ofA4CvkMw4OiWffc6sAmZx4LAlz8/evtwe7CBWdCDcZUcVvFOdhgW1v5ufG
	HJTQXg3AuS88Xiy5WQiZKf+m2uTZLy/UN68Qi6Yv3Q==
X-ME-Sender: <xms:FvQ7Zu-cZuFyVnvj2w9GgVIexAt2coN6c35oyc1Zkb3NqKIBiFm8zw>
    <xme:FvQ7Zuun-V_LzMPDGlOiX0QamUjmuadttXqIFueqwFEEHLJ8zO_kvVKIeW9Den0BG
    DqnTMlvCoOSqw>
X-ME-Received: <xmr:FvQ7ZkD5LbiEp4lAy6Vl8G8szQKJr_BqlxkCj_UNnxULFbuGTRMe-R_OicHXVbwr7vYrigxUw1WHHAv6BOsAyTMkX3xmUQVDhQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefuddgtdefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepieff
    tdfgvefgkeehffekffduiedtfefhhffgvdfhgfeugfffffegleekveeiueffnecuffhomh
    grihhnpehmshhigihtsghlpghlihhsthdrnhgvgihtnecuvehluhhsthgvrhfuihiivgep
    tdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslh
    gvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:FvQ7ZmcMZkUMM1bUIIccUa8ND3vuYOLOqgP_U0QBGqSbiL0kCHiyqA>
    <xmx:FvQ7ZjOWzzjwgf8bF2A77zNymeHO6rMLhKTn9I2oo8x32tzdXWThVQ>
    <xmx:FvQ7ZgliP-PGtxxv3Dplq-2AwZp4nuwEAHwAzzxwZ2bQX5WZuNUiDQ>
    <xmx:FvQ7Zltl-lQR1T3SAOMLFRtEpsi9togG8BTzKqr0yTH6ZRhTOUj0Aw>
    <xmx:FvQ7ZsqfBFHyoQ6xLGSEg2uS5suueoGCGcZcfVX7yLtBT5OxQIIAc-Jj>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 8 May 2024 23:52:17 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v7 2/6] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <Zjv0Ea3hQyzFwpmc@mail-itl>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
 <da180c8419882dc068512b69c6c7ad3ff3343fe0.1715085837.git-series.marmarek@invisiblethingslab.com>
 <ZjujzCnAfHye_eIp@macbook>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="Je8spFxkYQg5lGsn"
Content-Disposition: inline
In-Reply-To: <ZjujzCnAfHye_eIp@macbook>


--Je8spFxkYQg5lGsn
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 8 May 2024 23:52:17 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v7 2/6] x86/hvm: Allow access to registers on the same
 page as MSI-X table

On Wed, May 08, 2024 at 06:09:48PM +0200, Roger Pau Monn=C3=A9 wrote:
> On Tue, May 07, 2024 at 02:44:02PM +0200, Marek Marczykowski-G=C3=B3recki=
 wrote:
> > Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> > on the same page as MSI-X table. Device model (especially one in
> > stubdomain) cannot really handle those, as direct writes to that page is
> > refused (page is on the mmio_ro_ranges list). Instead, extend
> > msixtbl_mmio_ops to handle such accesses too.
> >=20
> > Doing this, requires correlating read/write location with guest
> > MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
> > it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> > for PV would need to be done separately.
> >=20
> > This will be also used to read Pending Bit Array, if it lives on the sa=
me
> > page, making QEMU not needing /dev/mem access at all (especially helpful
> > with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> > map it to the guest directly.
> > If PBA lives on the same page, discard writes and log a message.
> > Technically, writes outside of PBA could be allowed, but at this moment
> > the precise location of PBA isn't saved, and also no known device abuses
> > the spec in this way (at least yet).
> >=20
> > To access those registers, msixtbl_mmio_ops need the relevant page
> > mapped. MSI handling already has infrastructure for that, using fixmap,
> > so try to map first/last page of the MSI-X table (if necessary) and save
> > their fixmap indexes. Note that msix_get_fixmap() does reference
> > counting and reuses existing mapping, so just call it directly, even if
> > the page was mapped before. Also, it uses a specific range of fixmap
> > indexes which doesn't include 0, so use 0 as default ("not mapped")
> > value - which simplifies code a bit.
> >=20
> > Based on assumption that all MSI-X page accesses are handled by Xen, do
> > not forward adjacent accesses to other hypothetical ioreq servers, even
> > if the access wasn't handled for some reason (failure to map pages etc).
> > Relevant places log a message about that already.
> >=20
> > Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblething=
slab.com>
>=20
> Thanks, just a couple of minor comments, I think the only relevant one
> is that you can drop ADJACENT_DONT_HANDLE unless there's something
> I'm missing.  The rest are mostly cosmetic, but if you have to respin
> and agree with them might be worth addressing.
>=20
> Sorry for giving this feedback so late in the process, I should have
> attempted to review earlier versions.
>=20
> > ---
> > Changes in v7:
> > - simplify logic based on assumption that all access to MSI-X pages are
> >   handled by Xen (Roger)
> > - move calling adjacent_handle() into adjacent_{read,write}() (Roger)
> > - move range check into msixtbl_addr_to_desc() (Roger)
> > - fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
> > - no longer distinguish between unhandled write due to PBA nearby and
> >   other reasons
> > - add missing break after ASSERT_UNREACHABLE (Jan)
> > Changes in v6:
> > - use MSIX_CHECK_WARN macro
> > - extend assert on fixmap_idx
> > - add break in default label, after ASSERT_UNREACHABLE(), and move
> >   setting default there
> > - style fixes
> > Changes in v5:
> > - style fixes
> > - include GCC version in the commit message
> > - warn only once (per domain, per device) about failed adjacent access
> > Changes in v4:
> > - drop same_page parameter of msixtbl_find_entry(), distinguish two
> >   cases in relevant callers
> > - rename adj_access_table_idx to adj_access_idx
> > - code style fixes
> > - drop alignment check in adjacent_{read,write}() - all callers already
> >   have it earlier
> > - delay mapping first/last MSI-X pages until preparing device for a
> >   passthrough
> > v3:
> >  - merge handling into msixtbl_mmio_ops
> >  - extend commit message
> > v2:
> >  - adjust commit message
> >  - pass struct domain to msixtbl_page_handler_get_hwaddr()
> >  - reduce local variables used only once
> >  - log a warning if write is forbidden if MSI-X and PBA lives on the sa=
me
> >    page
> >  - do not passthrough unaligned accesses
> >  - handle accesses both before and after MSI-X table
> > ---
> >  xen/arch/x86/hvm/vmsi.c        | 205 ++++++++++++++++++++++++++++++++--
> >  xen/arch/x86/include/asm/msi.h |   5 +-
> >  xen/arch/x86/msi.c             |  42 +++++++-
> >  3 files changed, 242 insertions(+), 10 deletions(-)
> >=20
> > diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> > index 999917983789..f7b7b4998b5e 100644
> > --- a/xen/arch/x86/hvm/vmsi.c
> > +++ b/xen/arch/x86/hvm/vmsi.c
> > @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domai=
n *d)
> >      return d->arch.hvm.msixtbl_list.next;
> >  }
> > =20
> > +/*
> > + * Lookup an msixtbl_entry on the same page as given addr. It's up to =
the
> > + * caller to check if address is strictly part of the table - if relev=
ant.
> > + */
> >  static struct msixtbl_entry *msixtbl_find_entry(
> >      struct vcpu *v, unsigned long addr)
> >  {
> > @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
> >      struct domain *d =3D v->domain;
> > =20
> >      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> > -        if ( addr >=3D entry->gtable &&
> > -             addr < entry->gtable + entry->table_len )
> > +        if ( PFN_DOWN(addr) >=3D PFN_DOWN(entry->gtable) &&
> > +             PFN_DOWN(addr) <=3D PFN_DOWN(entry->gtable + entry->table=
_len - 1) )
> >              return entry;
> > =20
> >      return NULL;
> > @@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
> >      if ( !entry || !entry->pdev )
> >          return NULL;
> > =20
> > +    if ( addr < entry->gtable ||
> > +         addr >=3D entry->gtable + entry->table_len )
> > +        return NULL;
> > +
> >      nr_entry =3D (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
> > =20
> >      list_for_each_entry( desc, &entry->pdev->msi_list, list )
> > @@ -213,6 +221,152 @@ static struct msi_desc *msixtbl_addr_to_desc(
> >      return NULL;
> >  }
> > =20
> > +/*
> > + * Returns:
> > + *  - ADJACENT_DONT_HANDLE if no handling should be done
> > + *  - a fixmap idx to use for handling
> > + */
> > +#define ADJACENT_DONT_HANDLE UINT_MAX
>=20
> Isn't it fine to just return 0 to signal that the access is not
> handled?
>=20
> fixmap index 0 is reserved anyway (see FIX_RESERVED), so could be used
> for this purpose and then you don't need to introduce
> ADJACENT_DONT_HANDLE?

It was this way before in v2 and you asked me to not use 0 for this
purpose...

> > +static unsigned int adjacent_handle(
> > +    const struct msixtbl_entry *entry, unsigned long addr, bool write)
>=20
> Now that it has been quite pruned, get_adjacent_idx() or some such
> might be more inline with the function logic.

makes sense

>=20
> > +{
> > +    unsigned int adj_type;
> > +    struct arch_msix *msix;
> > +
> > +    if ( !entry || !entry->pdev )
> > +    {
> > +        ASSERT_UNREACHABLE();
> > +        return ADJACENT_DONT_HANDLE;
> > +    }
> > +
> > +    if ( PFN_DOWN(addr) =3D=3D PFN_DOWN(entry->gtable) && addr < entry=
->gtable )
> > +        adj_type =3D ADJ_IDX_FIRST;
> > +    else if ( PFN_DOWN(addr) =3D=3D PFN_DOWN(entry->gtable + entry->ta=
ble_len - 1) &&
> > +              addr >=3D entry->gtable + entry->table_len )
> > +        adj_type =3D ADJ_IDX_LAST;
> > +    else
> > +    {
> > +        /* All callers should already do equivalent range checking. */
> > +        ASSERT_UNREACHABLE();
> > +        return ADJACENT_DONT_HANDLE;
> > +    }
> > +
> > +    msix =3D entry->pdev->msix;
> > +    ASSERT(msix);
>=20
> Since you already do it above, would be best to do:
>=20
> if ( !msix )
> {
>     ASSERT_UNREACHABLE();
>     return 0;
> }

Ok.

> > +
> > +    if ( !msix->adj_access_idx[adj_type] )
> > +    {
> > +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> > +                             adjacent_not_initialized) )
> > +            gprintk(XENLOG_WARNING,
> > +                    "Page for adjacent(%d) MSI-X table access not init=
ialized for %pp (addr %#lx, gtable %#lx\n",
>=20
> Do you really need to log an error here?  There's an error already
> printed in msix_capability_init() if the adjacent pages can't be
> mapped.

IMO it's better to keep this message, otherwise it might be pretty hard
to debug not working device - a message buried somewhere on startup
might be hard to correlate with an issue much later.

> > +                    adj_type, &entry->pdev->sbdf, addr, entry->gtable);
> > +        return ADJACENT_DONT_HANDLE;
> > +    }
> > +
> > +    /* If PBA lives on the same page too, discard writes. */
> > +    if ( write &&
> > +         ((adj_type =3D=3D ADJ_IDX_LAST &&
> > +           msix->table.last =3D=3D msix->pba.first) ||
> > +          (adj_type =3D=3D ADJ_IDX_FIRST &&
> > +           msix->table.first =3D=3D msix->pba.last)) )
> > +    {
> > +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> > +                             adjacent_pba) )
> > +            gprintk(XENLOG_WARNING,
> > +                    "MSI-X table and PBA of %pp live on the same page,=
 "
> > +                    "writing to other registers there is not implement=
ed\n",
> > +                    &entry->pdev->sbdf);
>=20
> I would usually start those errors with the SBDF, as that makes it
> easier to identify error originating from the same device:
>=20
> "%pp: MSI-X table and PBA share a page, discard write to adjacent memory =
(%#lx)\n",
> &entry->pdev->sbdf, addr

ok

> > +        return ADJACENT_DONT_HANDLE;
> > +    }
> > +
> > +    return msix->adj_access_idx[adj_type];
> > +}
> > +
> > +static int adjacent_read(
> > +    const struct msixtbl_entry *entry,
> > +    paddr_t address, unsigned int len, uint64_t *pval)
> > +{
> > +    const void __iomem *hwaddr;
> > +    unsigned int fixmap_idx;
> > +
>=20
> I would add an ASSERT(IS_ALIGNED(address, len)) (and in
> adjacent_write()) just in case, as otherwise the access could cross a
> page boundary.

ok

> > +    fixmap_idx =3D adjacent_handle(entry, address, false);
> > +
> > +    if ( fixmap_idx =3D=3D ADJACENT_DONT_HANDLE )
> > +    {
> > +        *pval =3D ~0UL;
> > +        return X86EMUL_OKAY;
> > +    }
>=20
> FWIW, I find it safer to unconditionally init *pval =3D ~0UL at the
> start of the function, and then the return here and in the default
> switch statement case can avoid setting it.  It's less easy to return
> without the variable being set.

It was this way in v5, but Jan asked me to move it to only relevant
branch.

> > +    hwaddr =3D fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> > +
> > +    switch ( len )
> > +    {
> > +    case 1:
> > +        *pval =3D readb(hwaddr);
> > +        break;
> > +
> > +    case 2:
> > +        *pval =3D readw(hwaddr);
> > +        break;
> > +
> > +    case 4:
> > +        *pval =3D readl(hwaddr);
> > +        break;
> > +
> > +    case 8:
> > +        *pval =3D readq(hwaddr);
> > +        break;
> > +
> > +    default:
> > +        ASSERT_UNREACHABLE();
> > +        *pval =3D ~0UL;
> > +        break;
>=20
> We should likely move this to some kind of helpers, as it's now
> exactly the same that's done in adjacent_{read,write}() in
> vpci/msix.c (not that you should do it here, just a note).
>=20
> > +    }
> > +
> > +    return X86EMUL_OKAY;
> > +}
> > +
> > +static int adjacent_write(
> > +    const struct msixtbl_entry *entry,
> > +    paddr_t address, unsigned int len, uint64_t val)
> > +{
> > +    void __iomem *hwaddr;
> > +    unsigned int fixmap_idx;
> > +
> > +    fixmap_idx =3D adjacent_handle(entry, address, true);
> > +
> > +    if ( fixmap_idx =3D=3D ADJACENT_DONT_HANDLE )
> > +        return X86EMUL_OKAY;
> > +
> > +    hwaddr =3D fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> > +
> > +    switch ( len )
> > +    {
> > +    case 1:
> > +        writeb(val, hwaddr);
> > +        break;
> > +
> > +    case 2:
> > +        writew(val, hwaddr);
> > +        break;
> > +
> > +    case 4:
> > +        writel(val, hwaddr);
> > +        break;
> > +
> > +    case 8:
> > +        writeq(val, hwaddr);
> > +        break;
> > +
> > +    default:
> > +        ASSERT_UNREACHABLE();
> > +        break;
> > +    }
> > +
> > +    return X86EMUL_OKAY;
> > +}
> > +
> >  static int cf_check msixtbl_read(
> >      const struct hvm_io_handler *handler, uint64_t address, uint32_t l=
en,
> >      uint64_t *pval)
> > @@ -222,7 +376,7 @@ static int cf_check msixtbl_read(
> >      unsigned int nr_entry, index;
> >      int r =3D X86EMUL_UNHANDLEABLE;
> > =20
> > -    if ( (len !=3D 4 && len !=3D 8) || (address & (len - 1)) )
> > +    if ( !IS_ALIGNED(address, len) )
> >          return r;
> > =20
> >      rcu_read_lock(&msixtbl_rcu_lock);
> > @@ -230,6 +384,17 @@ static int cf_check msixtbl_read(
> >      entry =3D msixtbl_find_entry(current, address);
> >      if ( !entry )
> >          goto out;
> > +
> > +    if ( address < entry->gtable ||
> > +         address >=3D entry->gtable + entry->table_len )
> > +    {
> > +        r =3D adjacent_read(entry, address, len, pval);
> > +        goto out;
> > +    }
> > +
> > +    if ( len !=3D 4 && len !=3D 8 )
> > +        goto out;
> > +
> >      offset =3D address & (PCI_MSIX_ENTRY_SIZE - 1);
> > =20
> >      if ( offset !=3D PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
> > @@ -291,6 +456,17 @@ static int msixtbl_write(struct vcpu *v, unsigned =
long address,
> >      entry =3D msixtbl_find_entry(v, address);
> >      if ( !entry )
> >          goto out;
> > +
> > +    if ( address < entry->gtable ||
> > +         address >=3D entry->gtable + entry->table_len )
> > +    {
> > +        r =3D adjacent_write(entry, address, len, val);
> > +        goto out;
> > +    }
> > +
> > +    if ( len !=3D 4 && len !=3D 8 )
> > +        goto out;
> > +
> >      nr_entry =3D array_index_nospec(((address - entry->gtable) /
> >                                     PCI_MSIX_ENTRY_SIZE),
> >                                    MAX_MSIX_TABLE_ENTRIES);
> > @@ -356,8 +532,8 @@ static int cf_check _msixtbl_write(
> >      const struct hvm_io_handler *handler, uint64_t address, uint32_t l=
en,
> >      uint64_t val)
> >  {
> > -    /* Ignore invalid length or unaligned writes. */
> > -    if ( (len !=3D 4 && len !=3D 8) || !IS_ALIGNED(address, len) )
> > +    /* Ignore unaligned writes. */
> > +    if ( !IS_ALIGNED(address, len) )
> >          return X86EMUL_OKAY;
> > =20
> >      /*
> > @@ -374,16 +550,25 @@ static bool cf_check msixtbl_range(
> >  {
> >      struct vcpu *curr =3D current;
> >      unsigned long addr =3D r->addr;
> > -    const struct msi_desc *desc;
> > +    const struct msixtbl_entry *entry;
> > +    bool ret =3D false;
> > =20
> >      ASSERT(r->type =3D=3D IOREQ_TYPE_COPY);
> > =20
> >      rcu_read_lock(&msixtbl_rcu_lock);
> > -    desc =3D msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr=
);
> > +    entry =3D msixtbl_find_entry(curr, addr);
> > +    if ( entry )
> > +    {
> > +        if ( addr < entry->gtable || addr >=3D entry->gtable + entry->=
table_len )
> > +            /* Possibly handle adjacent access. */
> > +            ret =3D true;
> > +        else
> > +            ret =3D msixtbl_addr_to_desc(entry, addr) !=3D NULL;
> > +    }
>=20
> You could probably put all this into a single condition:
>=20
> if ( entry &&
>       /* Adjacent access. */
>      (addr < entry->gtable || addr >=3D entry->gtable + entry->table_len =
||
>       /* Otherwise check if there's a matching msi_desc. */
>       msixtbl_addr_to_desc(entry, addr)) )
>     ret =3D true;
>=20
> That's IMO easier to read by setting ret once only.

Is multi-line "if" mixed with comments really easier to follow?

>=20
> >      rcu_read_unlock(&msixtbl_rcu_lock);
> > =20
> > -    if ( desc )
> > -        return 1;
> > +    if ( ret )
> > +        return ret;
> > =20
> >      if ( r->state =3D=3D STATE_IOREQ_READY && r->dir =3D=3D IOREQ_WRIT=
E )
> >      {
> > @@ -429,7 +614,7 @@ static bool cf_check msixtbl_range(
> >          }
> >      }
> > =20
> > -    return 0;
> > +    return false;
> >  }
> > =20
> >  static const struct hvm_io_ops msixtbl_mmio_ops =3D {
> > diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/=
msi.h
> > index bcfdfd35345d..923b730d48b8 100644
> > --- a/xen/arch/x86/include/asm/msi.h
> > +++ b/xen/arch/x86/include/asm/msi.h
> > @@ -224,6 +224,9 @@ struct arch_msix {
> >      } table, pba;
> >      int table_refcnt[MAX_MSIX_TABLE_PAGES];
> >      int table_idx[MAX_MSIX_TABLE_PAGES];
> > +#define ADJ_IDX_FIRST 0
> > +#define ADJ_IDX_LAST  1
> > +    unsigned int adj_access_idx[2];
> >      spinlock_t table_lock;
> >      bool host_maskall, guest_maskall;
> >      domid_t warned_domid;
> > @@ -231,6 +234,8 @@ struct arch_msix {
> >          uint8_t all;
> >          struct {
> >              bool maskall                   : 1;
> > +            bool adjacent_not_initialized  : 1;
>=20
> Not sure we need that, since we already warn of failure to map at
> initialization time, not sure it's worth also printing another error
> at access time.

See response earlier above.

> > +            bool adjacent_pba              : 1;
> >          };
> >      } warned_kind;
> >  };
> > diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> > index 42c793426da3..87190a88ed5d 100644
> > --- a/xen/arch/x86/msi.c
> > +++ b/xen/arch/x86/msi.c
> > @@ -913,6 +913,37 @@ static int msix_capability_init(struct pci_dev *de=
v,
> >          list_add_tail(&entry->list, &dev->msi_list);
> >          *desc =3D entry;
> >      }
> > +    else
> > +    {
> > +        /*
> > +         * If the MSI-X table doesn't start at the page boundary, map =
the first page for
> > +         * passthrough accesses.
> > +         */
> > +        if ( PAGE_OFFSET(table_paddr) )
> > +        {
> > +            int idx =3D msix_get_fixmap(msix, table_paddr, table_paddr=
);
> > +
> > +            if ( idx > 0 )
> > +                msix->adj_access_idx[ADJ_IDX_FIRST] =3D idx;
> > +            else
> > +                gprintk(XENLOG_ERR, "Failed to map first MSI-X table p=
age: %d\n", idx);
> > +        }
> > +        /*
> > +         * If the MSI-X table doesn't end on the page boundary, map th=
e last page
> > +         * for passthrough accesses.
> > +         */
> > +        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENT=
RY_SIZE) )
> > +        {
> > +            uint64_t entry_paddr =3D table_paddr +
> > +                (msix->nr_entries - 1) * PCI_MSIX_ENTRY_SIZE;
> > +            int idx =3D msix_get_fixmap(msix, table_paddr, entry_paddr=
);
> > +
> > +            if ( idx > 0 )
> > +                msix->adj_access_idx[ADJ_IDX_LAST] =3D idx;
> > +            else
> > +                gprintk(XENLOG_ERR, "Failed to map last MSI-X table pa=
ge: %d\n", idx);
>=20
> Could you prefix the messages with the SBDF of the device please?
> It's in the seg, bus, slot, func local variables AFAICT.

Ok (%pp + dev->sbdf).

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--Je8spFxkYQg5lGsn
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmY79BEACgkQ24/THMrX
1yxQtAf/QRS8CEzR8iZfvQsjKZ9mcbPIuLF3n05aLlcKbdGiFsHSqzFl5G6D+I0b
7Jz1TwPJlRvhuwBC5uplZRMZVDpvTYSYWLG6gPf7yA4lCSpjPp/joKYMCMYTQFi3
jccHEvkXhYzS4uQCeApa9rTnk8e1SAwDVTiO6zwnviJeZDdaSEaX7wE7HuN/v2PF
9089nmRvH2cd6d/zxrpQhaqX4FePc6bfSGx2nl1C5oTHyYQL0xJWg2dMAcF8SS5g
RZqjYcKaBAWLv7iFfjNJV5/mVI5yXZVxXcV8ukK6ED/5NOYGjpppUaY3RhK1mtp3
+5isYac3rvjgDIbGOU5xzaTWfTArwg==
=kW4n
-----END PGP SIGNATURE-----

--Je8spFxkYQg5lGsn--


From xen-devel-bounces@lists.xenproject.org Wed May 08 22:11:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 22:11:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719000.1121611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pVE-0005bq-EJ; Wed, 08 May 2024 22:11:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719000.1121611; Wed, 08 May 2024 22:11:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pVE-0005bj-As; Wed, 08 May 2024 22:11:08 +0000
Received: by outflank-mailman (input) for mailman id 719000;
 Wed, 08 May 2024 22:11:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s4pVD-0005bd-2s
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 22:11:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4pVC-0001NQ-GV; Wed, 08 May 2024 22:11:06 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s4pVC-0005S7-A5; Wed, 08 May 2024 22:11:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	References:Cc:To:From:Subject:MIME-Version:Date:Message-ID;
	bh=m1VVijBiFDc7GUlCDpqW1KovSksostA44Er1BhD/AXE=; b=JpjmK9fZpel7QCM2qTdy4LrgTn
	NSVK039078kJZTRmx0BxKvTw3Jxk+gc+eRAZr8daaTK48QS82My1zmMb9Bd+LbZsoi5Wd1bnaXMRG
	cZT04h42uup5FSYMdNLoVJ6Z1Elj3FBe8LaruWpBEico1KdXlEH6PagLro2nn4EcjoYM=;
Message-ID: <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>
Date: Wed, 8 May 2024 23:11:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
In-Reply-To: <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

CC-ing Roger as he is working on adding support for the foreign mapping 
on x86. Although, I am not expecting any implication as only 4KB mapping 
should be supported.

On 08/05/2024 22:05, Julien Grall wrote:
> On 07/05/2024 14:30, Luca Fancellu wrote:
>>> On 7 May 2024, at 14:20, Julien Grall <julien@xen.org> wrote:
>>>
>>> Hi Luca,
>>>
>>> On 23/04/2024 09:25, Luca Fancellu wrote:
>>>> From: Penny Zheng <Penny.Zheng@arm.com>
>>>> But today, p2m_put_l3_page could not handle superpages.
>>>
>>> This was done on purpose. Xen is not preemptible and therefore we 
>>> need to be cautious how much work is done within the p2m code.
>>>
>>> With the below proposal, for 1GB mapping, we may end up to call 
>>> put_page() up to 512 * 512 = 262144 times. put_page() can free 
>>> memory. This could be a very long operation.
>>>
>>> Have you benchmark how long it would take?
>>
>> I did not, since its purpose was unclear to me and was not commented 
>> in the last serie from Penny.
> 
> Honestly, I can't remember why it wasn't commented.

I skimmed through the code to check what we currently do for preemption.

{decrease, increase}_reservation() will allow to handle max_order() 
mapping at the time. On a default configuration, the max would be 4MB.

relinquish_p2m_mapping() is preempting every 512 iterations. One 
iteration is either a 4KB/2MB/1GB mapping.

relinquish_memory() is checking for preemption after every page.

So I think, it would be ok to allow 2MB mapping for static shared memory 
but not 1GB. relinquish_p2m_mapping() would also needs to be updated to 
take into account the larger foreign mapping.

I would consider to check for preemption if 't' is p2m_map_foreign and 
the order is above 9 (i.e. 2MB).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 08 22:20:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 22:20:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719007.1121630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pdz-00084Z-Ll; Wed, 08 May 2024 22:20:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719007.1121630; Wed, 08 May 2024 22:20:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4pdz-00083H-Gi; Wed, 08 May 2024 22:20:11 +0000
Received: by outflank-mailman (input) for mailman id 719007;
 Wed, 08 May 2024 22:20:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gEdx=ML=xenbits.xen.org=julieng@srs-se1.protection.inumbo.net>)
 id 1s4pdy-0007z7-HK
 for xen-devel@lists.xen.org; Wed, 08 May 2024 22:20:10 +0000
Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e8a195d-0d89-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 00:20:06 +0200 (CEST)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julieng@xenbits.xen.org>)
 id 1s4pdm-0001Uy-2T; Wed, 08 May 2024 22:19:58 +0000
Received: from julieng by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <julieng@xenbits.xen.org>)
 id 1s4pdm-0005hR-0X; Wed, 08 May 2024 22:19:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e8a195d-0d89-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=9h/2wzU0Mvhn2MKrTpCI2QVVZK0bZHd1iD7FuzxTA8E=; b=tSR60kzSnRej3nVh6mY4MjCHPJ
	KGGsGnSctV0d2rb8HT4jpEcnuRdfMKp9Z1J8A3DEjpqr7+0AZHEzOMNR/yZaapi9ERIyHckKxHtVl
	bpoSB5FVw+pjsdaSv7X0r9BS/wvgTMXcB4vLp6ibDCckxRGSjANPfeFrtu1n/1onX6aU=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 457 v3 (CVE-2024-27393) - Linux/xen-netfront:
 Memory leak due to missing cleanup function
Message-Id: <E1s4pdm-0005hR-0X@xenbits.xenproject.org>
Date: Wed, 08 May 2024 22:19:58 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

            Xen Security Advisory CVE-2024-27393 / XSA-457
                              version 3

    Linux/xen-netfront: Memory leak due to missing cleanup function

UPDATES IN VERSION 3
====================

CVE assigned.

ISSUE DESCRIPTION
=================

In netfront, xennet_alloc_one_rx_buffer() failed to call the
appropriate clean-up function, resulting in a memory leak.

IMPACT
======

A malicious guest userspace process can exhaust memory resources
within the guest kernel, potentially leading to a guest crash (Denial
of Service). It is not known whether it can be triggered remotely.

VULNERABLE SYSTEMS
==================

Systems with guests running Linux 5.9 and later with Xen PV network
devices are affected.

MITIGATION
==========

For HVM guests, using emulated network devices will avoid this issue.

RESOLUTION
==========

The following patch in Linux resolves the issue:

https://git.kernel.org/torvalds/c/037965402a010898d34f4e35327d22c0a95cd51f

A copy of which is attached.

xsa457.patch           Linux 5.9

$ sha256sum xsa457*
9d6ae3da27f1ff92f9f45c800822beecda603d6dea6726207cee6c768416114c  xsa457.patch
$


NOTE ON THE LACK OF EMBARGO
===========================

The issue was reported initially on a public bug tracker and fixed in
public before it was realized that there was a security aspect.
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmY7+mgMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZIygH/2qlkovJs5zZy4dTpsygoSnSiv6L31r2IGmMnR/c
qdgtfedzctQ/ibw0iaz/37w/d0F3lo/lg3iWnVgCcIfV384MvvoArFsOZ4v/RRXL
b0XiNCb0k5xLpw9R86f7oG7cDw59JU+sXVjBH6GcVo86yL+HKaeli7/FZb9zkz/D
VRushpxeA353u3FFdqHJcFlD68wA5nhM2JdjkPk1rrgPVc0sBLjHwrcFOrHHHuuq
epYSYzWEf5HGbOf+zg6NY9B0uD4Vb9J3xa+xcYaHfPlQ1Jexw5GA7vBMO82qcR57
lRwAOav844fHw+lNxizfg8+4ayFpOCyGX2WEag6qjN92qJE=
=mMwm
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa457.patch"
Content-Disposition: attachment; filename="xsa457.patch"
Content-Transfer-Encoding: base64

RnJvbSAwMzc5NjU0MDJhMDEwODk4ZDM0ZjRlMzUzMjdkMjJjMGE5NWNkNTFm
IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKZXNwZXIgRGFuZ2Fh
cmQgQnJvdWVyIDxoYXdrQGtlcm5lbC5vcmc+CkRhdGU6IFdlZCwgMjcgTWFy
IDIwMjQgMTM6MTQ6NTYgKzAxMDAKU3ViamVjdDogeGVuLW5ldGZyb250OiBB
ZGQgbWlzc2luZyBza2JfbWFya19mb3JfcmVjeWNsZQoKTm90aWNlIHRoYXQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKSBpcyBpbnRyb2R1Y2VkIGxhdGVyIHRo
YW4gZml4ZXMgdGFnIGluCmNvbW1pdCA2YTViY2Q4NGU4ODYgKCJwYWdlX3Bv
b2w6IEFsbG93IGRyaXZlcnMgdG8gaGludCBvbiBTS0IgcmVjeWNsaW5nIiku
CgpJdCBpcyBiZWxpZXZlZCB0aGF0IGZpeGVzIHRhZyB3ZXJlIG1pc3Npbmcg
YSBjYWxsIHRvIHBhZ2VfcG9vbF9yZWxlYXNlX3BhZ2UoKQpiZXR3ZWVuIHY1
LjkgdG8gdjUuMTQsIGFmdGVyIHdoaWNoIGlzIHNob3VsZCBoYXZlIHVzZWQg
c2tiX21hcmtfZm9yX3JlY3ljbGUoKS4KU2luY2UgdjYuNiB0aGUgY2FsbCBw
YWdlX3Bvb2xfcmVsZWFzZV9wYWdlKCkgd2VyZSByZW1vdmVkIChpbgpjb21t
aXQgNTM1YjljNjFiZGVmICgibmV0OiBwYWdlX3Bvb2w6IGhpZGUgcGFnZV9w
b29sX3JlbGVhc2VfcGFnZSgpIikKYW5kIHJlbWFpbmluZyBjYWxsZXJzIGNv
bnZlcnRlZCAoaW4gY29tbWl0IDZiZmVmMmVjMDE3MiAoIk1lcmdlIGJyYW5j
aAonbmV0LXBhZ2VfcG9vbC1yZW1vdmUtcGFnZV9wb29sX3JlbGVhc2VfcGFn
ZSciKSkuCgpUaGlzIGxlYWsgYmVjYW1lIHZpc2libGUgaW4gdjYuOCB2aWEg
Y29tbWl0IGRiYTFiOGE3YWI2OCAoIm1tL3BhZ2VfcG9vbDogY2F0Y2gKcGFn
ZV9wb29sIG1lbW9yeSBsZWFrcyIpLgoKQ2M6IHN0YWJsZUB2Z2VyLmtlcm5l
bC5vcmcKRml4ZXM6IDZjNWFhNmZjNGRlZiAoInhlbiBuZXR3b3JraW5nOiBh
ZGQgYmFzaWMgWERQIHN1cHBvcnQgZm9yIHhlbi1uZXRmcm9udCIpClJlcG9y
dGVkLWJ5OiBMZW9uaWRhcyBTcHlyb3BvdWxvcyA8YXJ0YWZpbmRlQGFyY2hs
aW51eC5jb20+Ckxpbms6IGh0dHBzOi8vYnVnemlsbGEua2VybmVsLm9yZy9z
aG93X2J1Zy5jZ2k/aWQ9MjE4NjU0ClJlcG9ydGVkLWJ5OiBBcnRodXIgQm9y
c2Jvb20gPGFydGh1cmJvcnNib29tQGdtYWlsLmNvbT4KU2lnbmVkLW9mZi1i
eTogSmVzcGVyIERhbmdhYXJkIEJyb3VlciA8aGF3a0BrZXJuZWwub3JnPgpM
aW5rOiBodHRwczovL2xvcmUua2VybmVsLm9yZy9yLzE3MTE1NDE2NzQ0Ni4y
NjcxMDYyLjkxMjcxMDUzODQ1OTEyMzczNjMuc3RnaXRAZmlyZXNvdWwKU2ln
bmVkLW9mZi1ieTogSmFrdWIgS2ljaW5za2kgPGt1YmFAa2VybmVsLm9yZz4K
LS0tCiBkcml2ZXJzL25ldC94ZW4tbmV0ZnJvbnQuYyB8IDEgKwogMSBmaWxl
IGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCspCgpkaWZmIC0tZ2l0IGEvZHJpdmVy
cy9uZXQveGVuLW5ldGZyb250LmMgYi9kcml2ZXJzL25ldC94ZW4tbmV0ZnJv
bnQuYwppbmRleCBhZDI5ZjM3MDAzNGU0Zi4uOGQyYWVlODg1MjZjNjkgMTAw
NjQ0Ci0tLSBhL2RyaXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCisrKyBiL2Ry
aXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCkBAIC0yODUsNiArMjg1LDcgQEAg
c3RhdGljIHN0cnVjdCBza19idWZmICp4ZW5uZXRfYWxsb2Nfb25lX3J4X2J1
ZmZlcihzdHJ1Y3QgbmV0ZnJvbnRfcXVldWUgKnF1ZXVlKQogCQlyZXR1cm4g
TlVMTDsKIAl9CiAJc2tiX2FkZF9yeF9mcmFnKHNrYiwgMCwgcGFnZSwgMCwg
MCwgUEFHRV9TSVpFKTsKKwlza2JfbWFya19mb3JfcmVjeWNsZShza2IpOwog
CiAJLyogQWxpZ24gaXAgaGVhZGVyIHRvIGEgMTYgYnl0ZXMgYm91bmRhcnkg
Ki8KIAlza2JfcmVzZXJ2ZShza2IsIE5FVF9JUF9BTElHTik7Ci0tIApjZ2l0
IDEuMi4zLWtvcmcKCg==

--=separator--


From xen-devel-bounces@lists.xenproject.org Wed May 08 23:44:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 May 2024 23:44:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719055.1121663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4qx4-0000Cr-QB; Wed, 08 May 2024 23:43:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719055.1121663; Wed, 08 May 2024 23:43:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4qx4-0000Ck-NV; Wed, 08 May 2024 23:43:58 +0000
Received: by outflank-mailman (input) for mailman id 719055;
 Wed, 08 May 2024 23:43:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6EkY=ML=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s4qx3-0000B9-T9
 for xen-devel@lists.xenproject.org; Wed, 08 May 2024 23:43:57 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d4ada7e1-0d94-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 01:43:54 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2dcc8d10d39so3264601fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 08 May 2024 16:43:54 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee94dasm3633745e9.32.2024.05.08.16.43.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 08 May 2024 16:43:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4ada7e1-0d94-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715211834; x=1715816634; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=tugmobpIslNnbTaUpaee9pv3NYraflTvT+qiaO/ws/U=;
        b=e9njp+RUdsMxqy1NQkePdLIQxUAVD3A/mRPqY8SVswleh/DZc1ekHwO1khGjOgEMmz
         llic7R9T/wt25XVybSCt5xw+9AdlKV1X0wgi4NZmtRSwgtv8Kdi9HPsDwNkeN9p6qLG+
         PPVO38J6Oiur4HisCei8GGap8Gp6Cyis7thws=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715211834; x=1715816634;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tugmobpIslNnbTaUpaee9pv3NYraflTvT+qiaO/ws/U=;
        b=sPKY6IUiYrl9CZqJBJgi3aEVU2ntUUSDFAiam3U6efwb6032xccs3JrniF7OhA/Mfg
         ih6F4SBrwcHbUmx7vY34nJ3dlWkMui7A1BdWyg0EcPVncqTUDM8lZbZBE1RuYcjs5qbI
         4rnGCpgVaIKJiRYZ/XR7dW7iOyT+94vBzsPWP7oghFrjzHoBxqo5mjCutksms4s24vqe
         +BTcCHw5tuwFdAW2ccG6E4d8WiPDkCqryX5rM6y02OiyDc08WeamUa2TMxpc/8NWQPYd
         y1Dn04JISSgB7EWceqbbbz36ENmXGX1/vLsr0/P39ZY9Baxjg7kJHhSbP1HxEAxLGn+6
         TqBQ==
X-Forwarded-Encrypted: i=1; AJvYcCXnQuis95hppVkwAiSeGKpfSaxkFKtVliXg0mDrANJVYjVixYZ+hPW/OahFJoUMltgYbpNL3UnyxxqfrsIF6O78wQB/zNS+dTdnYpusJgc=
X-Gm-Message-State: AOJu0Yxmyh18FJxM4ppggTB7bWz/csY/9Daa5WGgt5fVatoERESqzOc2
	tcXa+ZX4S4h9lIZRm169D9r8T4i3K2OPyEDBakyrpAcZ74kWzpnLm15FOKH+P/4=
X-Google-Smtp-Source: AGHT+IGVY7q/dqWrcA6ii6B9RUZefIhz/wRZveUN/O1Ceqau10PsZJ2w+Cs5Po5Vo5KIIj2re2pusw==
X-Received: by 2002:a05:651c:cb:b0:2e2:1a8b:e2f with SMTP id 38308e7fff4ca-2e446d83d31mr25608351fa.2.1715211834204;
        Wed, 08 May 2024 16:43:54 -0700 (PDT)
Message-ID: <a8628a12-9cd3-4182-a387-6bc659d0a09f@citrix.com>
Date: Thu, 9 May 2024 00:43:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] gunzip: don't leak memory on error paths
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <33b93fdf-bf16-49b8-aec2-0b2c19f5c471@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <33b93fdf-bf16-49b8-aec2-0b2c19f5c471@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06/05/2024 9:08 am, Jan Beulich wrote:
> While decompression errors are likely going to be fatal to Xen's boot
> process anyway, the latest with the goal of doing multiple decompressor
> runs it is likely better to avoid leaks even on error paths. All the
> more when this way code size actually shrinks a tiny bit.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Right now, all errors in gzip are panic()'s, although that is something
that will need addressing in due course.

Given that I'm shuffling Daniel's series a little in this code area
anyway, I'll slot this patch in ahead which I think will be the least
disruptive overall.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 09 01:22:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 01:22:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719065.1121673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4sU4-00068y-6d; Thu, 09 May 2024 01:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719065.1121673; Thu, 09 May 2024 01:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4sU4-00068r-3U; Thu, 09 May 2024 01:22:08 +0000
Received: by outflank-mailman (input) for mailman id 719065;
 Thu, 09 May 2024 01:22:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4sU2-00068h-6r; Thu, 09 May 2024 01:22:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4sU2-0004Sn-2a; Thu, 09 May 2024 01:22:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4sU1-0005nh-Q9; Thu, 09 May 2024 01:22:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4sU1-0006Rc-Pl; Thu, 09 May 2024 01:22:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QpUapBb4I385qdfmryEMZqHXSWZrGWiGmwKlIZrnJWU=; b=4WIUHUkBbPMX100x3Fctaf9i2d
	ADjIDd3LTw8ssr/Xj7ygDvCcAyHyd4IbHYIeOqYvuHp0wO81++9hp625BjH7HoM8+4TavLCGeDtOi
	vqNebIE893JhjK4n2UtNURQqgE8FfyIf4dRN9TJ5+9FHd7fNu3+qURH4AMJCumsja7SY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185947-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185947: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6d7ddd805123e03db0a16c7dbc8509366432a630
X-Osstest-Versions-That:
    linux=dccb07f2914cdab2ac3a5b6c98406f765acab803
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 01:22:05 +0000

flight 185947 linux-linus real [real]
flight 185948 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185947/
http://logs.test-lab.xenproject.org/osstest/logs/185948/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail pass in 185948-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 185948-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185948 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185948 never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 185948 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 185948 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185930
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185930
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185930
 test-armhf-armhf-xl-credit2   8 xen-boot                     fail  like 185930
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185930
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185930
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185930
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6d7ddd805123e03db0a16c7dbc8509366432a630
baseline version:
 linux                dccb07f2914cdab2ac3a5b6c98406f765acab803

Last test of basis   185930  2024-05-06 23:44:00 Z    2 days
Testing same since   185947  2024-05-08 17:40:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Arnd Bergmann <arnd@arndb.de>
  Bjorn Andersson <andersson@kernel.org>
  Bjorn Helgaas <bhelgaas@google.com>
  Conor Dooley <conor.dooley@microchip.com>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Samuel Holland <samuel.holland@sifive.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   dccb07f2914cd..6d7ddd805123e  6d7ddd805123e03db0a16c7dbc8509366432a630 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu May 09 07:35:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 07:35:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719100.1121684 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yJS-0002UQ-Qh; Thu, 09 May 2024 07:35:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719100.1121684; Thu, 09 May 2024 07:35:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yJS-0002UJ-M8; Thu, 09 May 2024 07:35:34 +0000
Received: by outflank-mailman (input) for mailman id 719100;
 Thu, 09 May 2024 07:35:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4yJR-0002UD-Fi
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 07:35:33 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b6a6f9cb-0dd6-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 09:35:31 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41b79451153so4297515e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 00:35:31 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccfe1358sm14607715e9.45.2024.05.09.00.35.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 00:35:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6a6f9cb-0dd6-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715240130; x=1715844930; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=kjjrkBdl8ySQSSZfW6M9y7F7JJ7hmW6Y1b6Li6t3Dzg=;
        b=Dg6cx1Y6BEhOCFC2YZr79wsIqnoyr//TNb/gOPEBteyLtHv6Nwj7JJBAbaqC9CkxYs
         GBgVqZiYcXxbI7wePMJXOi4z6l++MWHheoTb4IdKltCv63QhVpb9ALKg5BHq9Gpj0b5i
         fQAW2eG3ZE0JG1PWnDlE8Qzpm8TKGKVizVt7Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715240130; x=1715844930;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kjjrkBdl8ySQSSZfW6M9y7F7JJ7hmW6Y1b6Li6t3Dzg=;
        b=saj0Y1DafTlLI36hRhxafUI+5wM00GYGoKAFVnC0F5wQIutg+kt1pzS0LqAUcY9ryX
         JgwESYfKUCzxYBY9m69iicCyGKGap72ARviKYMUqnmIKYLOOlrmUiI69IRLeS5I++x5r
         Rzl+x6k7Sg6uYHZdCPA74vTfQdH72ZUUZs20mF7C9hijnJ+lw9Rvu3DYCuxmyl7bfX8T
         XeB1RUT+iGnNKrUR3nWXigeSWfa1UtZQkttrBSoqwn/soWNXkjILVa1svCDeGyLoxHEn
         4HSAxdJJ46o1ghC8UCxDv/UCJvh9wukPl0mixtCGEBHXFRGMySRZ1Rf30Sf5EOKX+bLl
         Fe/g==
X-Gm-Message-State: AOJu0YwP0nKFCWSsCpk/5eHPAkBlPX55HjTuu+BfELw3oqGhbAmtHwRJ
	yKhOjcmvE1LckGKXfn6/gubF6Rni+M3bW5nUGY3hK8vyxxPG4Hxwh9BEDX4vn3w=
X-Google-Smtp-Source: AGHT+IGHt+rp8xmgZw4sK/jaDjn2hrEhM/lH4cRlJvDoNZxnGGyfa40TooLd197IomPkfCLQPs2L0w==
X-Received: by 2002:a05:600c:4744:b0:41c:8123:f8a5 with SMTP id 5b1f17b1804b1-41f71bca3famr35272395e9.23.1715240130445;
        Thu, 09 May 2024 00:35:30 -0700 (PDT)
Date: Thu, 9 May 2024 09:35:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v7 2/6] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <Zjx8wf3m6AlQFbLT@macbook>
References: <cover.a422810661b997e9edf5f8bf5258a4422d9fa3ea.1715085837.git-series.marmarek@invisiblethingslab.com>
 <da180c8419882dc068512b69c6c7ad3ff3343fe0.1715085837.git-series.marmarek@invisiblethingslab.com>
 <ZjujzCnAfHye_eIp@macbook>
 <Zjv0Ea3hQyzFwpmc@mail-itl>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <Zjv0Ea3hQyzFwpmc@mail-itl>

On Wed, May 08, 2024 at 11:52:17PM +0200, Marek Marczykowski-Górecki wrote:
> On Wed, May 08, 2024 at 06:09:48PM +0200, Roger Pau Monné wrote:
> > On Tue, May 07, 2024 at 02:44:02PM +0200, Marek Marczykowski-Górecki wrote:
> > > Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> > > on the same page as MSI-X table. Device model (especially one in
> > > stubdomain) cannot really handle those, as direct writes to that page is
> > > refused (page is on the mmio_ro_ranges list). Instead, extend
> > > msixtbl_mmio_ops to handle such accesses too.
> > > 
> > > Doing this, requires correlating read/write location with guest
> > > MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
> > > it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> > > for PV would need to be done separately.
> > > 
> > > This will be also used to read Pending Bit Array, if it lives on the same
> > > page, making QEMU not needing /dev/mem access at all (especially helpful
> > > with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> > > map it to the guest directly.
> > > If PBA lives on the same page, discard writes and log a message.
> > > Technically, writes outside of PBA could be allowed, but at this moment
> > > the precise location of PBA isn't saved, and also no known device abuses
> > > the spec in this way (at least yet).
> > > 
> > > To access those registers, msixtbl_mmio_ops need the relevant page
> > > mapped. MSI handling already has infrastructure for that, using fixmap,
> > > so try to map first/last page of the MSI-X table (if necessary) and save
> > > their fixmap indexes. Note that msix_get_fixmap() does reference
> > > counting and reuses existing mapping, so just call it directly, even if
> > > the page was mapped before. Also, it uses a specific range of fixmap
> > > indexes which doesn't include 0, so use 0 as default ("not mapped")
> > > value - which simplifies code a bit.
> > > 
> > > Based on assumption that all MSI-X page accesses are handled by Xen, do
> > > not forward adjacent accesses to other hypothetical ioreq servers, even
> > > if the access wasn't handled for some reason (failure to map pages etc).
> > > Relevant places log a message about that already.
> > > 
> > > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > 
> > Thanks, just a couple of minor comments, I think the only relevant one
> > is that you can drop ADJACENT_DONT_HANDLE unless there's something
> > I'm missing.  The rest are mostly cosmetic, but if you have to respin
> > and agree with them might be worth addressing.
> > 
> > Sorry for giving this feedback so late in the process, I should have
> > attempted to review earlier versions.
> > 
> > > ---
> > > Changes in v7:
> > > - simplify logic based on assumption that all access to MSI-X pages are
> > >   handled by Xen (Roger)
> > > - move calling adjacent_handle() into adjacent_{read,write}() (Roger)
> > > - move range check into msixtbl_addr_to_desc() (Roger)
> > > - fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
> > > - no longer distinguish between unhandled write due to PBA nearby and
> > >   other reasons
> > > - add missing break after ASSERT_UNREACHABLE (Jan)
> > > Changes in v6:
> > > - use MSIX_CHECK_WARN macro
> > > - extend assert on fixmap_idx
> > > - add break in default label, after ASSERT_UNREACHABLE(), and move
> > >   setting default there
> > > - style fixes
> > > Changes in v5:
> > > - style fixes
> > > - include GCC version in the commit message
> > > - warn only once (per domain, per device) about failed adjacent access
> > > Changes in v4:
> > > - drop same_page parameter of msixtbl_find_entry(), distinguish two
> > >   cases in relevant callers
> > > - rename adj_access_table_idx to adj_access_idx
> > > - code style fixes
> > > - drop alignment check in adjacent_{read,write}() - all callers already
> > >   have it earlier
> > > - delay mapping first/last MSI-X pages until preparing device for a
> > >   passthrough
> > > v3:
> > >  - merge handling into msixtbl_mmio_ops
> > >  - extend commit message
> > > v2:
> > >  - adjust commit message
> > >  - pass struct domain to msixtbl_page_handler_get_hwaddr()
> > >  - reduce local variables used only once
> > >  - log a warning if write is forbidden if MSI-X and PBA lives on the same
> > >    page
> > >  - do not passthrough unaligned accesses
> > >  - handle accesses both before and after MSI-X table
> > > ---
> > >  xen/arch/x86/hvm/vmsi.c        | 205 ++++++++++++++++++++++++++++++++--
> > >  xen/arch/x86/include/asm/msi.h |   5 +-
> > >  xen/arch/x86/msi.c             |  42 +++++++-
> > >  3 files changed, 242 insertions(+), 10 deletions(-)
> > > 
> > > diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> > > index 999917983789..f7b7b4998b5e 100644
> > > --- a/xen/arch/x86/hvm/vmsi.c
> > > +++ b/xen/arch/x86/hvm/vmsi.c
> > > @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
> > >      return d->arch.hvm.msixtbl_list.next;
> > >  }
> > >  
> > > +/*
> > > + * Lookup an msixtbl_entry on the same page as given addr. It's up to the
> > > + * caller to check if address is strictly part of the table - if relevant.
> > > + */
> > >  static struct msixtbl_entry *msixtbl_find_entry(
> > >      struct vcpu *v, unsigned long addr)
> > >  {
> > > @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
> > >      struct domain *d = v->domain;
> > >  
> > >      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> > > -        if ( addr >= entry->gtable &&
> > > -             addr < entry->gtable + entry->table_len )
> > > +        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
> > > +             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
> > >              return entry;
> > >  
> > >      return NULL;
> > > @@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
> > >      if ( !entry || !entry->pdev )
> > >          return NULL;
> > >  
> > > +    if ( addr < entry->gtable ||
> > > +         addr >= entry->gtable + entry->table_len )
> > > +        return NULL;
> > > +
> > >      nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
> > >  
> > >      list_for_each_entry( desc, &entry->pdev->msi_list, list )
> > > @@ -213,6 +221,152 @@ static struct msi_desc *msixtbl_addr_to_desc(
> > >      return NULL;
> > >  }
> > >  
> > > +/*
> > > + * Returns:
> > > + *  - ADJACENT_DONT_HANDLE if no handling should be done
> > > + *  - a fixmap idx to use for handling
> > > + */
> > > +#define ADJACENT_DONT_HANDLE UINT_MAX
> > 
> > Isn't it fine to just return 0 to signal that the access is not
> > handled?
> > 
> > fixmap index 0 is reserved anyway (see FIX_RESERVED), so could be used
> > for this purpose and then you don't need to introduce
> > ADJACENT_DONT_HANDLE?
> 
> It was this way before in v2 and you asked me to not use 0 for this
> purpose...

Sorry, I think I didn't realize fixmap idx 0 was reserved, and hence
can be used to signal no idx.

> > > +
> > > +    if ( !msix->adj_access_idx[adj_type] )
> > > +    {
> > > +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> > > +                             adjacent_not_initialized) )
> > > +            gprintk(XENLOG_WARNING,
> > > +                    "Page for adjacent(%d) MSI-X table access not initialized for %pp (addr %#lx, gtable %#lx\n",
> > 
> > Do you really need to log an error here?  There's an error already
> > printed in msix_capability_init() if the adjacent pages can't be
> > mapped.
> 
> IMO it's better to keep this message, otherwise it might be pretty hard
> to debug not working device - a message buried somewhere on startup
> might be hard to correlate with an issue much later.

Would you mind starting the entry with the SBDF then?

"%pp: MSI-X adjacent memory not mapped, dropping access to %#lx\n"

Or similar.

> > > +    fixmap_idx = adjacent_handle(entry, address, false);
> > > +
> > > +    if ( fixmap_idx == ADJACENT_DONT_HANDLE )
> > > +    {
> > > +        *pval = ~0UL;
> > > +        return X86EMUL_OKAY;
> > > +    }
> > 
> > FWIW, I find it safer to unconditionally init *pval = ~0UL at the
> > start of the function, and then the return here and in the default
> > switch statement case can avoid setting it.  It's less easy to return
> > without the variable being set.
> 
> It was this way in v5, but Jan asked me to move it to only relevant
> branch.

Hm, I see, we had this discussion with Jan in the past.  I'm fine this
way if you prefer, but I think it's less robust.

> > > @@ -374,16 +550,25 @@ static bool cf_check msixtbl_range(
> > >  {
> > >      struct vcpu *curr = current;
> > >      unsigned long addr = r->addr;
> > > -    const struct msi_desc *desc;
> > > +    const struct msixtbl_entry *entry;
> > > +    bool ret = false;
> > >  
> > >      ASSERT(r->type == IOREQ_TYPE_COPY);
> > >  
> > >      rcu_read_lock(&msixtbl_rcu_lock);
> > > -    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
> > > +    entry = msixtbl_find_entry(curr, addr);
> > > +    if ( entry )
> > > +    {
> > > +        if ( addr < entry->gtable || addr >= entry->gtable + entry->table_len )
> > > +            /* Possibly handle adjacent access. */
> > > +            ret = true;
> > > +        else
> > > +            ret = msixtbl_addr_to_desc(entry, addr) != NULL;
> > > +    }
> > 
> > You could probably put all this into a single condition:
> > 
> > if ( entry &&
> >       /* Adjacent access. */
> >      (addr < entry->gtable || addr >= entry->gtable + entry->table_len ||
> >       /* Otherwise check if there's a matching msi_desc. */
> >       msixtbl_addr_to_desc(entry, addr)) )
> >     ret = true;
> > 
> > That's IMO easier to read by setting ret once only.
> 
> Is multi-line "if" mixed with comments really easier to follow?

It is for me, because ret gets set in a single place, it's a single
branch to analyze and reduces indentation.

Ultimately it's a question of taste, so would leave that up to you as
the author of the code.  I also dislike the 'ret =
msixtbl_addr_to_desc(entry, addr) != NULL' expression, but again it's
a question of taste.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 07:55:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 07:55:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719107.1121692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yct-0005Vl-Bs; Thu, 09 May 2024 07:55:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719107.1121692; Thu, 09 May 2024 07:55:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yct-0005Ve-9A; Thu, 09 May 2024 07:55:39 +0000
Received: by outflank-mailman (input) for mailman id 719107;
 Thu, 09 May 2024 07:55:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4ycr-0005VU-V4
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 07:55:37 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 853647db-0dd9-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 09:55:36 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b782405d5so6157095e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 00:55:36 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f88110ff8sm50555485e9.38.2024.05.09.00.55.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 00:55:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 853647db-0dd9-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715241336; x=1715846136; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=GK4WIT53jOeN92OklH7WM88IbyAPFWX3LSqQzOen3ms=;
        b=PUOwdVqSPIcqquBsYd63tShDzyeIsTN1kkNGvALY5KAxfYY6u37DXJ4WEF6/3KiWYz
         6D38knZumWvK66WmmQ8OLUQrw66ft2HVTs7NeNSHsnpigLHa3EZ6HUlxZrLanKV89H/m
         RW9XiDT1xc9IrIi45S1s/FmqF6DIRI27PGubk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715241336; x=1715846136;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GK4WIT53jOeN92OklH7WM88IbyAPFWX3LSqQzOen3ms=;
        b=IFzJ5qi1Q5xMLw2nKo1PmZlZ4x4LzDswCCm5lYOnLUCwTqdI4Ik8YwgDhliYAgT+/x
         RDK9jGrv1xT9rahX12+WtoLamOBaGRjNLqEzm+dQnm9FfXYkVVeejubQxd2ZnJzBJaBK
         7RV2n2Eaz5qSBDeuzWBUTnVumTUbY2NPX4TH27JkOJNifony1chnghJ4o+bvlUfAOPdw
         G/a0lFrpmmAiuQLQ5VR2WTQwxg39YVhhzbRf/SZ+Qo+KCdabEzAur7RiEzbjUQYdchYE
         pNmaZkWRJ/SIcYO8yNwlU/Q7r9SmyHYwcEcqZLyGua2uN45gnXaHxyBorg5CAhuC3k7J
         ZQIg==
X-Gm-Message-State: AOJu0Yy/ZtpUSmWv3omSsCmGK4yTX+/ohGvLGgQBu/zlK8L/XIMIp8Ih
	VhHHRuXpJ/TpnGStlDMJENM+KPx7K/mvUWZYw2E12cpMwGssx2ZeoAkcG7RSRaU=
X-Google-Smtp-Source: AGHT+IFHFSgC9nPYF7uyKYTij2oDA1KnFJ/pye2r/J/WlFTqEAI8yfyz7Zi4WXZBuafsLDBgngXPUg==
X-Received: by 2002:a05:600c:1548:b0:41b:edf4:4071 with SMTP id 5b1f17b1804b1-41f71accb44mr42967215e9.35.1715241336086;
        Thu, 09 May 2024 00:55:36 -0700 (PDT)
Date: Thu, 9 May 2024 09:55:35 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: xen-devel@lists.xenproject.org, Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Message-ID: <ZjyBd7Yn5Od3qPF9@macbook>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240423082532.776623-4-luca.fancellu@arm.com>

On Tue, Apr 23, 2024 at 09:25:28AM +0100, Luca Fancellu wrote:
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
> 
> This commits implements a new function p2m_put_superpage to handle superpages,
> specifically for helping put extra references for foreign superpages.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v1:
>  - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
> ---
>  xen/arch/arm/mmu/p2m.c | 58 +++++++++++++++++++++++++++++++-----------
>  1 file changed, 43 insertions(+), 15 deletions(-)
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index 41fcca011cf4..479a80fbd4cf 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>      return rc;
>  }
>  
> -/*
> - * Put any references on the single 4K page referenced by pte.
> - * TODO: Handle superpages, for now we only take special references for leaf
> - * pages (specifically foreign ones, which can't be super mapped today).
> - */
> -static void p2m_put_l3_page(const lpae_t pte)
> +/* Put any references on the single 4K page referenced by mfn. */
> +static void p2m_put_l3_page(mfn_t mfn, unsigned type)
                                          ^ p2m_type_t?

Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 08:14:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 08:14:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719117.1121702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yui-0000fE-5D; Thu, 09 May 2024 08:14:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719117.1121702; Thu, 09 May 2024 08:14:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4yui-0000f7-2e; Thu, 09 May 2024 08:14:04 +0000
Received: by outflank-mailman (input) for mailman id 719117;
 Thu, 09 May 2024 08:14:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4yug-0000f1-Mu
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 08:14:02 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 172fc42b-0ddc-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 10:14:00 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-34e7a35d5d4so429858f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 01:14:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b79bc8esm1019480f8f.15.2024.05.09.01.13.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 01:13:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 172fc42b-0ddc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715242440; x=1715847240; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=r4QhY4syQdA3dcGAQvCztiEB0K7cQzMPfGby1P/xNkU=;
        b=OsaDBFqvccmBQm0BGon2p7BbgRFDW2f4qtVr6znVNa7mHOu8csSdurqYlFughayFFb
         17IRc2tljSw05rNAvHaKHQEqQgqEv4EGKeUhnlj7PDB0vXpCkbvsOGTumR1XcfyNCrO1
         7nWgIckY84DwXL1A8gQSokhnyN2JvTa5hznEc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715242440; x=1715847240;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=r4QhY4syQdA3dcGAQvCztiEB0K7cQzMPfGby1P/xNkU=;
        b=SYjv+suXEnnqc/iQPDLY+8kve7fmqgFpcbqPVd3dN1Cz5E5EwSkI1Ct3kIRj3/WG+w
         3At72FmwRedv8NSGUSFgSTMFxiiqtc7jTrwhthTenAKrU6ZFWiOhohz4/WqIiJd8qNh5
         9xlVPz3zhxOrY7q86YkctV/wFcmj3B9FYueLmxD2Zv9agXURzooNHD4L5eMr1AZ4QSTO
         9FUVUTTHgCxkvioC0Wg1NY6xo2eZ3oVhEFDfnPhJRvFp1Urbfj9G32AYk7zQjRwyH2J4
         IW8vMGyxpcHwfCojuZ7cTwZPc0cYGXJTFogUXhQn7MKuvhbJVDjce2UQD/56+vDf/aaC
         NZ/Q==
X-Forwarded-Encrypted: i=1; AJvYcCUpl1QKjCWcn35QX9g5EbyWSLpIQYG2TcNPhq7vgJH4GyuIcUp5EZV9Osqfd4ZpwZf5b7a5iN95JJ7fpHxtMtRFzM6tz5Z4IuqpcsHRba4=
X-Gm-Message-State: AOJu0YzUDzDFGz/N/Y6hwMLwvWlX/HGh8TaROF+XIeIb6/13dYmv/Ck9
	raGO3XDwERWFz+O8zdtouZr7uS8Rav+Spvks4paJZxQi0D6LvmIsZYGlhYmTh4o=
X-Google-Smtp-Source: AGHT+IH9NgWVp/YFWIQh9Wm7+y8a6hgfrygCsKrSZzRjOAsN9z809Lccl5RlrdnEQEVGoBLj1TMT1w==
X-Received: by 2002:a05:6000:1741:b0:34a:3f3d:bb22 with SMTP id ffacd0b85a97d-34fca2439dcmr3363188f8f.26.1715242439940;
        Thu, 09 May 2024 01:13:59 -0700 (PDT)
Date: Thu, 9 May 2024 10:13:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Message-ID: <ZjyFxrvHJ04ZlBGg@macbook>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>

On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
> Hi,
> 
> CC-ing Roger as he is working on adding support for the foreign mapping on
> x86. Although, I am not expecting any implication as only 4KB mapping should
> be supported.

I don't think we have plans on x86 to support foreign mappings with
order != 0 ATM.

We would need a new interface to allow creating such mappings, and
it's also not clear to me how the domain that creates such mappings
can identify super-pages on the remote domain.  IOW: the mapping
domain could request a super-page in the foreign domain gfn space,
but that could end up being a range of lower order mappings.

Also the interactions with the remote domain would need to be audited,
as the remote domain shattering the superpage would need to be
replicated in the mapping side in order to account for the changes.

> On 08/05/2024 22:05, Julien Grall wrote:
> > On 07/05/2024 14:30, Luca Fancellu wrote:
> > > > On 7 May 2024, at 14:20, Julien Grall <julien@xen.org> wrote:
> > > > 
> > > > Hi Luca,
> > > > 
> > > > On 23/04/2024 09:25, Luca Fancellu wrote:
> > > > > From: Penny Zheng <Penny.Zheng@arm.com>
> > > > > But today, p2m_put_l3_page could not handle superpages.
> > > > 
> > > > This was done on purpose. Xen is not preemptible and therefore
> > > > we need to be cautious how much work is done within the p2m
> > > > code.
> > > > 
> > > > With the below proposal, for 1GB mapping, we may end up to call
> > > > put_page() up to 512 * 512 = 262144 times. put_page() can free
> > > > memory. This could be a very long operation.
> > > > 
> > > > Have you benchmark how long it would take?
> > > 
> > > I did not, since its purpose was unclear to me and was not commented
> > > in the last serie from Penny.
> > 
> > Honestly, I can't remember why it wasn't commented.
> 
> I skimmed through the code to check what we currently do for preemption.
> 
> {decrease, increase}_reservation() will allow to handle max_order() mapping
> at the time. On a default configuration, the max would be 4MB.
> 
> relinquish_p2m_mapping() is preempting every 512 iterations. One iteration
> is either a 4KB/2MB/1GB mapping.
> 
> relinquish_memory() is checking for preemption after every page.
> 
> So I think, it would be ok to allow 2MB mapping for static shared memory but
> not 1GB. relinquish_p2m_mapping() would also needs to be updated to take
> into account the larger foreign mapping.

FWIW, relinquish_p2m_mapping() likely does more than what's strictly
needed, as you could just remove foreign mappings while leaving other
entries as-is?  The drain of the p2m pool and release of domain pages
should take care of dropping references to the RAM domain memory?

> I would consider to check for preemption if 't' is p2m_map_foreign and the
> order is above 9 (i.e. 2MB).

How can those mappings be removed?  Is it possible for the guest to
modify such foreign super-pages?  Not sure all paths will be easy to
audit for preemption if it's more than relinquish_p2m_mapping() that
you need to adjust.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 08:23:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 08:23:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719124.1121717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4z3X-0002BI-01; Thu, 09 May 2024 08:23:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719124.1121717; Thu, 09 May 2024 08:23:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4z3W-0002BB-To; Thu, 09 May 2024 08:23:10 +0000
Received: by outflank-mailman (input) for mailman id 719124;
 Thu, 09 May 2024 08:23:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s4z3V-0002B5-F5
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 08:23:09 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5cebbd77-0ddd-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 10:23:07 +0200 (CEST)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-34da04e44a2so397075f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 01:23:07 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502baacff4sm1037126f8f.79.2024.05.09.01.23.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 01:23:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cebbd77-0ddd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715242986; x=1715847786; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Rp9uqeEeeTpVuGLE1TK2mKhExUqdNcRzK4GLCWB/SbA=;
        b=d1MRtwtzKcDjdcC5FGUX1FUdgCyGkD2HI3eBnlsws2ZAHyz5kyCwS65ZcSE+SSGlr6
         K/Ygs0DB6YehC3Qz5r7OnxZ8Gxv/F9neLO2QLp5IJ6syriGZWJ7u3EcxBX4eYhEZabAT
         L6GH25lFdidMICsVVsjfKN3ndGeKvXiMHHj2I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715242986; x=1715847786;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Rp9uqeEeeTpVuGLE1TK2mKhExUqdNcRzK4GLCWB/SbA=;
        b=J9jlCw41QAJCZkNSdlTyxA1UgdXcp64s9Ay/6CdVRV30v5wOb1clCfWudeYfJxg/C5
         dmPCra9Ho/F+zYeaTcII3EJC0tZoImF/0WF87GOGNMEi3RlCIuxOt7Xtw7eZbqI4lH7o
         urG8QZGJOsiNbLL/SWYNJ7EBRTiONtY6hvAf7TutWcuzXoMa8dSb3195wfWffNlXZP18
         W/CcFGAyuiLB1cfDDdO1Q+kHLGrmowHcsZjG0TfciQN1swxbWZq7AiSUlg3ZAvLE7nSr
         DsHjwwRXH+c1fnF5CT61QoSH++/nPW+M3/dbOPcv+XnoSdkmct2seSFYoTvWRMlsimgw
         uMbg==
X-Gm-Message-State: AOJu0YwYg/oCbvu4WAryUgSZoxfhrJqlQh0UsQh0R7zzcBHgi9i0VeV/
	6v8UNi0J23C0PhDkRXZBjPMVOLGHBQBaXZ15hqGd5LZK+hEJ6IQo/OMJv55VoK4=
X-Google-Smtp-Source: AGHT+IHy4GzO3XZd4HD9UcdrSM0dak1uGWem4KVfsBnprBVTeTpNjY5oiBcLtmKhLn5kTTgKev+C2w==
X-Received: by 2002:adf:e68e:0:b0:34d:95e2:c5cb with SMTP id ffacd0b85a97d-34fca14b375mr3287726f8f.3.1715242986453;
        Thu, 09 May 2024 01:23:06 -0700 (PDT)
Date: Thu, 9 May 2024 10:23:05 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Petr =?utf-8?B?QmVuZcWh?= <w1benny@gmail.com>
Subject: Re: [PATCH for-4.19 v2 2/3] xen/x86: enable altp2m at create domain
 domctl
Message-ID: <ZjyH6f-diiqLO-0D@macbook>
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-3-roger.pau@citrix.com>
 <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com>

On Wed, May 08, 2024 at 08:38:07PM +0100, Andrew Cooper wrote:
> On 08/05/2024 12:23 pm, Roger Pau Monne wrote:
> > Enabling it using an HVM param is fragile, and complicates the logic when
> > deciding whether options that interact with altp2m can also be enabled.
> >
> > Leave the HVM param value for consumption by the guest, but prevent it from
> > being set.  Enabling is now done using the misc_flags field in
> > xen_arch_domainconfig.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> > Changes since v1:
> >  - New in this version.
> 
> Ha.  So this is actually work that Petr has been wanting to do.
> 
> Petr has a series hoping to make it into 4.19 (x86: Make MAX_ALTP2M
> configurable), which just missed out on this side of things.
> 
> altp2m is not architecture specific at all, and there's even support for
> ARM out on the mailing list.  Therefore, the altp2m mode wants to be
> common, just like the new MAX_ALTP2M setting already is.

Initially I had it as a set of XEN_DOMCTL_CDF_* flags, but it wasn't
clear to me whether the modes could be shared between arches.

> Both fields can reasonably share uint32_t, but could you work with Petr
> to make both halfs of this land cleanly.

I'm happy for Petr to pick this patch as part of the series if he
feels like.

I assume the plan would be to add an XEN_DOMCTL_CDF_altp2m flag, and
then a new field to signal the mode.

> 
> As to the HVMPARAM, I'd really quite like to delete it.  It was always a
> bodge, and there's a full set of HVMOP_altp2m_* for a guest to use.

I've assumed we must keep HVM_PARAM_ALTP2M for backwards
compatibility.

> > diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> > index 20e83cf38bbd..dff790060605 100644
> > --- a/xen/arch/x86/domain.c
> > +++ b/xen/arch/x86/domain.c
> > @@ -708,13 +711,33 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> >          }
> >      }
> >  
> > -    if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED )
> > +    if ( config->arch.misc_flags & ~XEN_X86_MISC_FLAGS_ALL )
> >      {
> >          dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n",
> >                  config->arch.misc_flags);
> >          return -EINVAL;
> >      }
> >  
> > +    if ( altp2m && (altp2m & (altp2m - 1)) )
> > +    {
> > +        dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
> > +                config->flags);
> > +        return -EINVAL;
> 
> I think this would be clearer to follow by having a 2 bit field called
> altp2m_mode and check for <= 2.

Don't we need 3 bits, for mixed, external and limited modes?

We could do with 2 bits if we signal altp2m enabled in a different
field, and then introduce a field to just contain the mode.

FWIW, the check should be `if ( altp2m & (altp2m - 1) )`.  I had
updated this, but seems like I missed to re-generate the patches.

> > +    }
> > +
> > +    if ( altp2m && nested_virt )
> > +    {
> > +        dprintk(XENLOG_INFO,
> > +                "Nested virt and altp2m are mutually incompatible\n");
> 
> There's nothing inherently incompatible.  I think it's more that noone
> had any interest in trying to make it work in combination with nested p2ms.
> 
> I'd phrase it as "not supported", rather than incompatible.

"Nested virt and altp2m are not supported together\n"

> > +        return -EINVAL;
> > +    }
> > +
> > +    if ( altp2m && !hap )
> > +    {
> > +        dprintk(XENLOG_INFO, "altp2m requires HAP\n");
> > +        return -EINVAL;
> > +    }
> 
> altp2m ought to work fine with shadow.  It's only if you want VMFUNC/#VE
> acceleration that you depend on EPT.
> 
> Again, I'd phrase this as "not supported".

"altp2m is only supported with HAP\n"

To avoid the double negation of "not supported without HAP" wording.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 08:40:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 08:40:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719132.1121726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4zK8-0005WR-Du; Thu, 09 May 2024 08:40:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719132.1121726; Thu, 09 May 2024 08:40:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4zK8-0005WK-BD; Thu, 09 May 2024 08:40:20 +0000
Received: by outflank-mailman (input) for mailman id 719132;
 Thu, 09 May 2024 08:40:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4zK7-0005WA-Bb; Thu, 09 May 2024 08:40:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4zK7-0005YF-90; Thu, 09 May 2024 08:40:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s4zK6-0006xl-TU; Thu, 09 May 2024 08:40:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s4zK6-0003B7-T2; Thu, 09 May 2024 08:40:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cwRx+01diU6CcfVgVvr0M98FCF2MqXAG2vnUrOkBYLw=; b=Qi3LnRuuaChiQaDkp6ZAcmaYjI
	FcW9wEHj6l05T8hcc/HCuME7AmatGcLLuVPbrnIzWTKZVyGcbwlunCt3KlmB04j9Hiv1ujI+/FZdo
	lbnmwzZ3DyWA16i7g5n0Umz8/8KzbIrOIbVGAMiMVaWdzletzYC2MPR7jN1XuuZ4pMhw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185952-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185952: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=25996a3441f4f40fcf457c4ba39a1e88c4f75bdd
X-Osstest-Versions-That:
    ovmf=b82c9631da39ca5a1f0702185a46fea60446dd0a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 08:40:18 +0000

flight 185952 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185952/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 25996a3441f4f40fcf457c4ba39a1e88c4f75bdd
baseline version:
 ovmf                 b82c9631da39ca5a1f0702185a46fea60446dd0a

Last test of basis   185946  2024-05-08 11:45:03 Z    0 days
Testing same since   185952  2024-05-09 07:13:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Praveen Sankar N <praveensankarn@ami.com>
  praveensankarn <praveensankarn@ami.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   b82c9631da..25996a3441  25996a3441f4f40fcf457c4ba39a1e88c4f75bdd -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 09 08:59:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 08:59:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719141.1121736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4zcC-0007sT-TW; Thu, 09 May 2024 08:59:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719141.1121736; Thu, 09 May 2024 08:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s4zcC-0007sM-Ql; Thu, 09 May 2024 08:59:00 +0000
Received: by outflank-mailman (input) for mailman id 719141;
 Thu, 09 May 2024 08:59:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cZF+=MM=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s4zcB-0007sG-Ka
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 08:58:59 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on2060e.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::60e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ecf55b8-0de2-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 10:58:57 +0200 (CEST)
Received: from AS9PR06CA0595.eurprd06.prod.outlook.com (2603:10a6:20b:486::29)
 by PA4PR08MB5917.eurprd08.prod.outlook.com (2603:10a6:102:f1::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.45; Thu, 9 May
 2024 08:58:55 +0000
Received: from AMS1EPF00000048.eurprd04.prod.outlook.com
 (2603:10a6:20b:486:cafe::72) by AS9PR06CA0595.outlook.office365.com
 (2603:10a6:20b:486::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Thu, 9 May 2024 08:58:55 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS1EPF00000048.mail.protection.outlook.com (10.167.16.132) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Thu, 9 May 2024 08:58:54 +0000
Received: ("Tessian outbound daa456608199:v315");
 Thu, 09 May 2024 08:58:54 +0000
Received: from c1bf2658ad95.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 2948FFA0-EB45-491A-8867-C9AFF8573DA3.1; 
 Thu, 09 May 2024 08:58:47 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c1bf2658ad95.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 09 May 2024 08:58:47 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB9PR08MB6714.eurprd08.prod.outlook.com (2603:10a6:10:2a4::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Thu, 9 May
 2024 08:58:45 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.045; Thu, 9 May 2024
 08:58:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ecf55b8-0de2-11ef-909c-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=LFfsvtQB6lv+VK3l6n8InjlKp+ftF9jLJdEGPoy+eNoJ8J0BrLH3YvWYVoMMOPJNOf3KdLSCjaaVh9tJT56xDh3cqhVsRmKTZWpx16bhEH9ZVLOpZujdWC7JA3gB9QNQ31hJ4a4SsCuWfG+tmxHQjnMEmmW0Lq8TOLQxJIZ6wgCBzp52M468bd2VcjRAumq+/NPQY4rKDxN3Adhp8EILfGBrDMdqhDUsqDE290/JKamWiAa4S6kDUMGSFbVtRwC1/TlA9TDLsE5LrhLnhOyTOz0+q5IQOURbP7PmGoMrITJ6/TmUzglRfpu9bG2tlPZMMYQpFSO7YrlhOK6T0WjEhQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GB07rgyqgbV5QuSTYi/imjCNAQ3g6YJl1ZO3tXKI8eM=;
 b=gqwiznNXFuFklCz1vFmSgXdt3K02x8TtnG40oe42woL67kb6xac5uk0ANI14QsZSvljWlagzxvyEWy3qKpewhzEmbPCxqHfgGtHpmpzMDDrVfg8YlzzQMxVl+BKzSMMI1tB7JvX4BuDmQlICz1AZMC9T+kq7o2YnwbQNb5TspHTBeD1jVvLX1uy+gUhX/x7c7wvffwd2gZZbviFrhntr6t25QweeEBANPEz6bIaqMKnFyFPIo06pn/sw343F7nbuYDMnI73mFuy8to1DdP0tgQW7CuQvRLqdI//5KJzPhVPZVW/ebr8Cruqw2pFLNls4CgoafKZbwKouT+BLRw2jzw==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GB07rgyqgbV5QuSTYi/imjCNAQ3g6YJl1ZO3tXKI8eM=;
 b=jDn1hZPh6j6VVSSK+nU73AcJ6ywXn1k1fjEkHOgLtniW7tmYiPZwyWWWHHvOWuRn+/Ud+U5uQtkPO8asWoWN0st0DQq7oV5WG8KptVNvzvi4uNoLSl12sC+xZC/50m6hD30NryMTr68XibCiWaqakLJKpIKSkgvbgaOIvbY/mu0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: ba81e519cf24e559
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=i6sMbQzMVc4q7BXVYFduQOHyQ9b4TvB4mLy/m3nhQvK4uU8ru0onu8JnRvBTvflRee5LtGJ374p6QGwmTdo/fF3+lIVDNpmk+tNqQZpmtUqId0BZrnQB4eflSi2g7gAtq6DMKTqV+gswum9HoYg9DFghs00+rey+Ju+qxhcmJobOehDtdK34keLy5sFeU8gL0QqN9HLcApAKi/64xT2Y37urhJbwK7JU+OlAePoyIBa+adDLzGR16+lmuvfZJWVr8JI10ulJMFwy9h0bFIoJTAI8xeNALTwtMRCn8D0E0UzesUSYmwzCuq7L6izxs76omKjcriK2d7NP7HetEGRdrA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GB07rgyqgbV5QuSTYi/imjCNAQ3g6YJl1ZO3tXKI8eM=;
 b=NdO7RFQUGy8GswiZyV3KAfDhATat9UntPm7ib1dQ3CIv3TBtFLpPZdiRdsWCWQ9x0pIcnqqlJLjvHu+FRgLohGO5M+Ae3mE9gGV+yQYSX2JyxwlBDZz62diVp4OdiL69Y9a3HtF9Q/P4C50+acctoxe0oHMCZyszrngwIjFs1rx56hDFjpuw7UDyky3PZn2LGxj5wvELwroQ4/M10y3lQqhyKL33HHnvf4LTx9qT91kG2Ja3V88/Fgng/Pz/TfKue5ofeQKe5DR6D1JQ6Bae8emeIYhqxKtUlrz+x8mf5eCeAmb4+BTGFDc581QeS1He47yyKhFjZqWfFDutAVpBkQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GB07rgyqgbV5QuSTYi/imjCNAQ3g6YJl1ZO3tXKI8eM=;
 b=jDn1hZPh6j6VVSSK+nU73AcJ6ywXn1k1fjEkHOgLtniW7tmYiPZwyWWWHHvOWuRn+/Ud+U5uQtkPO8asWoWN0st0DQq7oV5WG8KptVNvzvi4uNoLSl12sC+xZC/50m6hD30NryMTr68XibCiWaqakLJKpIKSkgvbgaOIvbY/mu0=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 4/7] xen/arm: Parse xen,shared-mem when host phys address
 is not provided
Thread-Topic: [PATCH 4/7] xen/arm: Parse xen,shared-mem when host phys address
 is not provided
Thread-Index: AQHalVfmskMuPI2TlEqCln/x5OWH6LGNVmgAgAAV/wCAAUcPgA==
Date: Thu, 9 May 2024 08:58:45 +0000
Message-ID: <E3AC4A71-1C0E-48A7-B2FA-09EC8B5F86FD@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-5-luca.fancellu@arm.com>
 <59b02ed2-e543-4bfd-946e-6a82919c582d@amd.com>
 <A9708869-7739-4306-8527-1A0978D542FF@arm.com>
In-Reply-To: <A9708869-7739-4306-8527-1A0978D542FF@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB9PR08MB6714:EE_|AMS1EPF00000048:EE_|PA4PR08MB5917:EE_
X-MS-Office365-Filtering-Correlation-Id: e39114c9-dd45-4bf1-f394-08dc70064171
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?TG03TEUwL1pCbHBWdE5mWnhTVTZxOVhvdzJMWFlxSkVRMTNXQzJkNSt1VGYx?=
 =?utf-8?B?K0Uyak1DeERxNFdBTzBkMmNraXY1N3hUL1Z4N3ZsQUU5K3p2U09ZWFl4UHBy?=
 =?utf-8?B?cXdkRmlrYTJpdVpoV2VCaWUrNkVtc2RvcVlYMGRkSU9TOENuVklMOHVuWHM5?=
 =?utf-8?B?SUtsZ05Td0FEMVM4dG9KaTRlWEx1TUxWTk5Nd0dKNlVjL2J6U2VRU0ZUYkRt?=
 =?utf-8?B?elNodUJ1dmJGUWNmK0xOMTR5aUhtancvSWhFMHRvL0c4bFBPY1IyYnlQc1Fa?=
 =?utf-8?B?aXBNTm44Zjd0elNtWktjRDR4Wm5PRWcvbUw3UTlXOUJob3VkeWhOZStMb1kw?=
 =?utf-8?B?T3YzSlBpN1VoV2VEOXgyMUlhWWoraE5YTWZZTEt2Tks0ZEhDY0NyUU80QThF?=
 =?utf-8?B?aDlrK1lpUVBLUDg2MmxkdUwwWUlDR0pUQy9Sem5mWGRLZkNWelZjN3JKSWNQ?=
 =?utf-8?B?VExDOWlqcWVmYXIwZWU3UVR1YVJvejZ3dUZhMURnK2VzL3dhUTNkcHk4S0g2?=
 =?utf-8?B?Z1VaZjdqUk9zQXcxWmVXSjd6RzdjMk1MZkNiV3AzK2lwa3REWWVqWFcrQTBu?=
 =?utf-8?B?aDRQVEJiWmQ1Z0ZNNm5vZHdpSi9ybnhOeU1aQndBM2hzZzRiZFR4aDV5OVFy?=
 =?utf-8?B?d0lDZEhzTk1uOEc0b0FPZmdSZ2Vmc3hJR2RQLzFWMDlNbWdERnN0Q2IxV2oz?=
 =?utf-8?B?VUl5TTI1ZS9ZV0ZnOTBBRXBrajFMQVBPbEFJLzFGRXNoOGUvd3dDUXl5V3pY?=
 =?utf-8?B?aTYwcW1UV1RWSTllMUVlN0lrc3dBWVpRVjBQRjdBTEFGR3RlNGpWUHZ4Y2dw?=
 =?utf-8?B?dWNzMnNORzZLYzA3cCtlQVE2RGZWSG5FY2kwUyt0QU9GaTFXN2wwdnZUVzMy?=
 =?utf-8?B?ekdPY2lQVUpjdDViZi9MbUZWU2VGclQyUUczT1JRMzUvR1hSYlJDSHo2eEJB?=
 =?utf-8?B?RHNvZjVQcDBrcDRhb1YvZnhEWkxISzVmMXlxZDc4OUZtSmIzNE1GMkRYTDBh?=
 =?utf-8?B?c0Z2NVFhQWh3d1kvd0Uvc3ZsNnNzck52c2tlSGpoM09zME42b0N4VWhkQm9M?=
 =?utf-8?B?b3hadUdXSGw5ZkRhWE5qcXlYQmRYOFdjNVR3dmJQTmpMNEhCQmQ4UXFjTldP?=
 =?utf-8?B?R3IvZU5iNUIvVE1hRjVydktIUjlwTFpqbURITWxsVUdzTVZSb3E1VlhmTk02?=
 =?utf-8?B?L2s2WXhueGx6V0Z0VDdKQmdmd3ZsNjR2Qyt2WTV0NUYrdG9yNC9yM2pQVkFt?=
 =?utf-8?B?UkhDNmhTRkVzaUFrVFdxQ3A5RENsOWl3QzVwaDByd1hrelpEWmJUd1NaS2da?=
 =?utf-8?B?R1JSWHN3MTV4SFN3bCt6QkFkSlFhMEc2QS9VNVUzeCtQQzNGdXI4TTBTUERW?=
 =?utf-8?B?OTFxOWNPVXZxTm1zVSt0aUI1bXkwRGZMaE1KYlNNb2JkeVV2bmNScUdCMUVh?=
 =?utf-8?B?ellJZVRlSm1ZRzErYVdaWWpMVTY0YndVNEJBWUdCQUl0M00yUWNiTERjT05S?=
 =?utf-8?B?dDhuWHRvUW5xS3o1SHhub3poSzc4b0dvYlR5MEJZQndra1RjQ1VPczVrMjEw?=
 =?utf-8?B?dks1cjEwMkFBZTI2NzVtT2xrdDIwNmZ1eEFzd0hOS1pzQVduZ0UwVlB2Wkx1?=
 =?utf-8?B?TXV0SFNWK05sR1h4RmlXM012QkkxOVYrNkdESmlJWEZwMWpRa1FIOEZkS3NR?=
 =?utf-8?B?Q3RaSEdjQS9LbVg2YkRSNXAzaGpYdmxaM1ZYemx3VjExTHR1bXVtQzM1RnZH?=
 =?utf-8?B?L0JHWktnUE9NUGE1cVBiS1Zwa3gyNWIxdzRtOHBEeHd5dXkxbDF2WXhjQXJN?=
 =?utf-8?B?TVc0bFF2RzhjS3ZCd2R0Zz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <13318C2E59F5A94A9B959424F9C096B8@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6714
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS1EPF00000048.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	253d6509-e540-4263-3e14-08dc70063bd8
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|35042699013|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?MnB4ckhIZDBRd2FzYzRscTlqOHF3dmduN05xNlRaT2haMmZPdHMxUUhwNlpN?=
 =?utf-8?B?S0xYd01hYndseXVoUXR1ZGg2NWpWMzNLU2ZtK2pZdVBGa2dUSHp2blVVdWRr?=
 =?utf-8?B?a1VZcmJXTkF1eU9EUzVKYitKNkRNNWpBekFwUFhESWladnhHd1I5SUhhdG1G?=
 =?utf-8?B?RWM4MmpBM01WSkVDTjZzTURXTnhNaGVvSWZCQ2locXMyWXFWY0trdG4yQmlx?=
 =?utf-8?B?QWF3RmpIb3VjdkdZMzkzNDJyZ0xkR0dGMWJVOHh5NUhSQ01tQldXa0YzZmhD?=
 =?utf-8?B?V2VhTkNyTjNrdm9ZY3ZjV0lyVDZlS3hHRHN0S2hiTEFsaUhQMlFPQXBoZlNv?=
 =?utf-8?B?TnI4eHhLN2QvR0VpbUpTWkU2TjRocDdJM1lCT0g2Sys5QWlHKytXUktMRzN2?=
 =?utf-8?B?dUtMRTQ2K05ndW5aOEJxRXJmOURpYlJyTEVnZTVVZVJuc3FXOEgwR05JZEpj?=
 =?utf-8?B?c0ZJamFxbW5tY3EyUGJWZFR0LzBKU1NrVWNxOUpZUDVQVmNFSEdtWGRVSEdm?=
 =?utf-8?B?VVJaN3BKN1MvclhKc095NGtIWUFtbE9UWUVDVVMwZWFnWjZuaGgxdzdqZWRS?=
 =?utf-8?B?TDlLYjdvQ1Z1ZC9UV2lBMGFOYXVzNUxPdEhKSlR6cTAxRWFpN012MVFCS0Jp?=
 =?utf-8?B?NzV0WmZGUjk0VlE3OGNBb0hQa1d0aTJvV2Rka2VWVnlrcEhjVDI2Ky81cThs?=
 =?utf-8?B?MG1KOGNlTnoyN3BzUHBaK2d4RWFqc3JublU3ZDVOaURpUVZhUE93OVAzelJS?=
 =?utf-8?B?MFFMVnB0YlFGVjFvSUh0WHlldjJobnc0N012OU5scE9xMHVrNytMOTIwM1JE?=
 =?utf-8?B?aFZiaGVCcGVjYmdCSE01dnJpaVMwODZUS2xESlg4NnBobXFrRkNtbGtPQWQ1?=
 =?utf-8?B?UGpBbVB2N0NRRzhSTE11OU4zRFpESEZOR3hXenkzdFZPUjNhSE05UFFuOW1r?=
 =?utf-8?B?THJIeXh6UDY3K1dCakJ2MmdJck9XTHFIcEdLbThjOVNaMXlCR3Q2dURwL2lq?=
 =?utf-8?B?WnB6UEFaZXVpZEh4WjBZNzd4L2VDRm5EVUlZUVY0TDZ5L01oU2d2WmIyRGwr?=
 =?utf-8?B?dnlFV3ZRQTJCdTM3a290cC9jcGRpeWx6R0ZPSnVOcitxTTlBUmcwVWt2K2c1?=
 =?utf-8?B?TGxUeThvYm1MOG4rODA2TkZhOFBReDdYVzNmcUMyMEN2ZXNkTFozQzY4YXpM?=
 =?utf-8?B?V2NvVGd0MkZDT01HSkVrdXhFWnRTTEdCU3MrVlhXb0VCY3N5U2FKYWY5TWh6?=
 =?utf-8?B?Y0tEeUJXUjNNS1RzeXhVMUJWUTRpZnVjNUhNTi81TE1oTnNhZzZCS3pWMHk0?=
 =?utf-8?B?YUt4QUxUbFZSRVo5bk9hVjg4SXpMYTZ1bHhtSmUyRmVuNkRFMFg3YjNnVktZ?=
 =?utf-8?B?UjZ0dHczL2Jjc0RpYkFLOHhkQ0JRZXZFaVEyYjJzeHNZRWtnTE4zOFRWSXlI?=
 =?utf-8?B?Tkk4MWN3QU1HZnJmb1JVVTJUYzE1c0IzTzJBV3RJcGRaYnVTcG9ZNWRrSFZs?=
 =?utf-8?B?UHF4clE1K0ZIUm1jNHVmeGNrazYwa1dISERwa2dGaGZVTFlLbm90S1BOOGlr?=
 =?utf-8?B?WWdaN3E3NVZiVlduOU9vZDJIUW1pdklOUkVBanZNckN0OUtOU0lPZ0JyVTF6?=
 =?utf-8?B?Y2hoWHBvbkJCbnlVWFNzSGduYjBBT1Y3SE55Zkh2WmNKcG1VKzVRZ1Z1TjVy?=
 =?utf-8?B?Qlg4ZmthMW0xd0M1bE13NXliWUtVc2Jkd1FzN3h2TlI0SW9IVXdDWFJ3cnlj?=
 =?utf-8?B?SVQxejVudnduZVdKTVVzcVJXeTlUdGNnS1JPd09zUHlPdnUxMlZ6bDdTcDZj?=
 =?utf-8?Q?w54CV5VevVo0Dqt3kOKtJ3wrDvaR150Pxt+MM=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(35042699013)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 08:58:54.9244
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e39114c9-dd45-4bf1-f394-08dc70064171
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS1EPF00000048.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5917

SGkgTWljaGFsLA0KDQo+PiANCj4+PiArICAgICAgICBpZiAoIHNobV9pZF9tYXRjaCApDQo+Pj4g
ICAgICAgIHsNCj4+PiAtICAgICAgICAgICAgaWYgKCBzdHJuY21wKHNobV9pZCwgc2htZW1fZXh0
cmFbaV0uc2htX2lkLA0KPj4+IC0gICAgICAgICAgICAgICAgICAgICAgICAgTUFYX1NITV9JRF9M
RU5HVEgpID09IDAgICkNCj4+PiArICAgICAgICAgICAgLyoNCj4+PiArICAgICAgICAgICAgICog
UmVnaW9ucyBoYXZlIHNhbWUgc2htX2lkIChjYXNlcyk6DQo+Pj4gKyAgICAgICAgICAgICAqIDEp
IHBoeXNpY2FsIGhvc3QgYWRkcmVzcyBpcyBzdXBwbGllZDoNCj4+PiArICAgICAgICAgICAgICog
ICAgLSBPSzogICBwYWRkciBpcyBlcXVhbCBhbmQgc2l6ZSBpcyBlcXVhbCAoc2FtZSByZWdpb24p
DQo+Pj4gKyAgICAgICAgICAgICAqICAgIC0gRmFpbDogcGFkZHIgZG9lc24ndCBtYXRjaCBvciBz
aXplIGRvZXNuJ3QgbWF0Y2ggKHRoZXJlDQo+Pj4gKyAgICAgICAgICAgICAqICAgICAgICAgICAg
Y2Fubm90IGV4aXN0cyB0d28gc2htZW0gcmVnaW9ucyB3aXRoIHNhbWUgc2htX2lkKQ0KPj4+ICsg
ICAgICAgICAgICAgKiAyKSBwaHlzaWNhbCBob3N0IGFkZHJlc3MgaXMgTk9UIHN1cHBsaWVkOg0K
Pj4+ICsgICAgICAgICAgICAgKiAgICAtIE9LOiAgIHNpemUgaXMgZXF1YWwgKHNhbWUgcmVnaW9u
KQ0KPj4+ICsgICAgICAgICAgICAgKiAgICAtIEZhaWw6IHNpemUgaXMgbm90IGVxdWFsIChzYW1l
IHNobV9pZCBtdXN0IGlkZW50aWZ5IG9ubHkgb25lDQo+Pj4gKyAgICAgICAgICAgICAqICAgICAg
ICAgICAgcmVnaW9uLCB0aGVyZSBjYW4ndCBiZSB0d28gZGlmZmVyZW50IHJlZ2lvbnMgd2l0aCBz
YW1lDQo+Pj4gKyAgICAgICAgICAgICAqICAgICAgICAgICAgc2htX2lkKQ0KPj4+ICsgICAgICAg
ICAgICAgKi8NCj4+PiArICAgICAgICAgICAgYm9vbCBzdGFydF9tYXRjaCA9IHBhZGRyX2Fzc2ln
bmVkID8gKHBhZGRyID09IG1lbS0+YmFua1tpXS5zdGFydCkgOg0KPj4+ICsgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cnVlOw0KPj4+ICsNCj4+PiArICAg
ICAgICAgICAgaWYgKCBzdGFydF9tYXRjaCAmJiBzaXplID09IG1lbS0+YmFua1tpXS5zaXplICkN
Cj4+PiAgICAgICAgICAgICAgICBicmVhazsNCj4+PiAgICAgICAgICAgIGVsc2UNCj4+PiAgICAg
ICAgICAgIHsNCj4+PiAtICAgICAgICAgICAgICAgIHByaW50aygiZmR0OiB4ZW4sc2htLWlkICVz
IGRvZXMgbm90IG1hdGNoIGZvciBhbGwgdGhlIG5vZGVzIHVzaW5nIHRoZSBzYW1lIHJlZ2lvbi5c
biIsDQo+Pj4gKyAgICAgICAgICAgICAgICBwcmludGsoImZkdDogZGlmZmVyZW50IHNoYXJlZCBt
ZW1vcnkgcmVnaW9uIGNvdWxkIG5vdCBzaGFyZSB0aGUgc2FtZSBzaG0gSUQgJXNcbiIsDQo+Pj4g
ICAgICAgICAgICAgICAgICAgICAgIHNobV9pZCk7DQo+Pj4gICAgICAgICAgICAgICAgcmV0dXJu
IC1FSU5WQUw7DQo+Pj4gICAgICAgICAgICB9DQo+Pj4gICAgICAgIH0NCj4+PiAtICAgICAgICBl
bHNlIGlmICggc3RybmNtcChzaG1faWQsIHNobWVtX2V4dHJhW2ldLnNobV9pZCwNCj4+PiAtICAg
ICAgICAgICAgICAgICAgICAgICAgICBNQVhfU0hNX0lEX0xFTkdUSCkgIT0gMCApDQo+Pj4gLSAg
ICAgICAgICAgIGNvbnRpbnVlOw0KPj4+ICAgICAgICBlbHNlDQo+Pj4gICAgICAgIHsNCj4+IFRo
ZXJlIGlzIG5vIG5lZWQgZm9yIHRoaXMgZWxzZSBhbmQgZW50aXJlIGJsb2NrIGdpdmVuIHRoYXQg
dGhlIGJsb2NrIHdpdGhpbiBpZiBlaXRoZXIgY2FsbHMgYnJlYWsgb3IgcmV0dXJuDQo+IA0KPiBU
aGVyZSB3YXMgYSBNSVNSQSBkaXNjdXNzaW9uIGFib3V0IGVsc2UgYXQgdGhlIGVuZCBvZiBpZiAu
Li4gZWxzZSBpZiAuLi4gKFIxNS43KSBhbmQgSSBkb27igJl0IHJlbWVtYmVyDQo+IHRoZSBvdXRj
b21lDQoNClNvcnJ5IEkgd2FzIG1pc3JlYWRpbmcgdGhlIGNvZGUgaGVyZSwgc3VyZSBJ4oCZbGwg
cmVtb3ZlIHRoZSBlbHNlLg0KDQpDaGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu May 09 09:51:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 09:51:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719148.1121746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s50QX-0007n0-IU; Thu, 09 May 2024 09:51:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719148.1121746; Thu, 09 May 2024 09:51:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s50QX-0007mt-Fq; Thu, 09 May 2024 09:51:01 +0000
Received: by outflank-mailman (input) for mailman id 719148;
 Thu, 09 May 2024 09:51:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s50QW-0007mn-1b
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 09:51:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s50QV-0006mv-KT; Thu, 09 May 2024 09:50:59 +0000
Received: from [15.248.2.239] (helo=[10.24.67.31])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s50QV-0005CT-9s; Thu, 09 May 2024 09:50:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Dbv5TjkeWmyORyLfXsOMt6XuaLjwJnv/029oy+C3QXQ=; b=eWHgfLsBmAaXDogX8myHRQDeRZ
	oH2QjOhP4C+GbdfSbsj/5w2elTG3BrfZlMV36vHFCpW/bkR1RmJ6cY6I7TMow1s8ijqksA419p0Oj
	NYhrfEeo1c1rrISV0ZYpXKJRdrLuuwqCAE+BoKfW6kT4nXgKIPDnGnAjHnvElWSwNz7Q=;
Message-ID: <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org>
Date: Thu, 9 May 2024 10:50:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
 <Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org> <ZjyFxrvHJ04ZlBGg@macbook>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ZjyFxrvHJ04ZlBGg@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 09/05/2024 09:13, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
>> Hi,
>>
>> CC-ing Roger as he is working on adding support for the foreign mapping on
>> x86. Although, I am not expecting any implication as only 4KB mapping should
>> be supported.
> 
> I don't think we have plans on x86 to support foreign mappings with
> order != 0 ATM.
> 
> We would need a new interface to allow creating such mappings, and
> it's also not clear to me how the domain that creates such mappings
> can identify super-pages on the remote domain.  IOW: the mapping
> domain could request a super-page in the foreign domain gfn space,
> but that could end up being a range of lower order mappings.

I agree with this. But ...

> 
> Also the interactions with the remote domain would need to be audited,
> as the remote domain shattering the superpage would need to be
> replicated in the mapping side in order to account for the changes.

... I don't understand this one. How is this different from today's 
where a domain can foreign map a 2MB which may be using a superpage in 
the remote domain?

> 
>> On 08/05/2024 22:05, Julien Grall wrote:
>>> On 07/05/2024 14:30, Luca Fancellu wrote:
>>>>> On 7 May 2024, at 14:20, Julien Grall <julien@xen.org> wrote:
>>>>>
>>>>> Hi Luca,
>>>>>
>>>>> On 23/04/2024 09:25, Luca Fancellu wrote:
>>>>>> From: Penny Zheng <Penny.Zheng@arm.com>
>>>>>> But today, p2m_put_l3_page could not handle superpages.
>>>>>
>>>>> This was done on purpose. Xen is not preemptible and therefore
>>>>> we need to be cautious how much work is done within the p2m
>>>>> code.
>>>>>
>>>>> With the below proposal, for 1GB mapping, we may end up to call
>>>>> put_page() up to 512 * 512 = 262144 times. put_page() can free
>>>>> memory. This could be a very long operation.
>>>>>
>>>>> Have you benchmark how long it would take?
>>>>
>>>> I did not, since its purpose was unclear to me and was not commented
>>>> in the last serie from Penny.
>>>
>>> Honestly, I can't remember why it wasn't commented.
>>
>> I skimmed through the code to check what we currently do for preemption.
>>
>> {decrease, increase}_reservation() will allow to handle max_order() mapping
>> at the time. On a default configuration, the max would be 4MB.
>>
>> relinquish_p2m_mapping() is preempting every 512 iterations. One iteration
>> is either a 4KB/2MB/1GB mapping.
>>
>> relinquish_memory() is checking for preemption after every page.
>>
>> So I think, it would be ok to allow 2MB mapping for static shared memory but
>> not 1GB. relinquish_p2m_mapping() would also needs to be updated to take
>> into account the larger foreign mapping.
> 
> FWIW, relinquish_p2m_mapping() likely does more than what's strictly
> needed, as you could just remove foreign mappings while leaving other
> entries as-is?  The drain of the p2m pool and release of domain pages
> should take care of dropping references to the RAM domain memory?
I believe the code was written in a way we could easily introduce 
reference for all the mappings. This has been discussed a few times in 
the past but we never implemented it.

> 
>> I would consider to check for preemption if 't' is p2m_map_foreign and the
>> order is above 9 (i.e. 2MB).
> 
> How can those mappings be removed?

 From any p2m_* functions. On Arm we don't (yet) care about which 
mapping is replaced.

>  Is it possible for the guest to
> modify such foreign super-pages?

Yes.

>  Not sure all paths will be easy to
> audit for preemption if it's more than relinquish_p2m_mapping() that
> you need to adjust.

I thought about it yesterday. But I came to the conclusion that if we 
have any concern about removing 1GB foreign superpage then we would 
already have the problem today as a domain can map contiguously 1GB 
worth of foreign mapping using small pages.

I went through the various code paths and, I believe, the only concern 
would be if an admin decides to change the values from max_order() using 
the command line option "memop-max-order".

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 09 10:09:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 10:09:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719157.1121757 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s50ih-0001lG-1q; Thu, 09 May 2024 10:09:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719157.1121757; Thu, 09 May 2024 10:09:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s50ig-0001l9-Tl; Thu, 09 May 2024 10:09:46 +0000
Received: by outflank-mailman (input) for mailman id 719157;
 Thu, 09 May 2024 10:09:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s50if-0001kz-78; Thu, 09 May 2024 10:09:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s50if-0007A2-4Y; Thu, 09 May 2024 10:09:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s50ie-0000je-P4; Thu, 09 May 2024 10:09:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s50ie-0005qn-Oc; Thu, 09 May 2024 10:09:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ZdTMnEYo2u3yWp2d7VNMFNDn5SUhyO+k9b7R5pV1BxE=; b=tWHtmMUGEbWqMY255Qo6C5Q2Ww
	Fi0mt0CNZhK76AACjayiJSt8K0JgM3sbBmWMzzlpKv+S0xN1kKtLuAYO61rgrFGmlH+aYhnLLlY9G
	37B2UWHH+W/p3OGOW6DZdb1NWMFsNHEAu1ntsoguS/RYfTLBPZ2m/LqiSE7xCccnWG/M=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185949-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185949: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=45db3ab70092637967967bfd8e6144017638563c
X-Osstest-Versions-That:
    linux=6d7ddd805123e03db0a16c7dbc8509366432a630
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 10:09:44 +0000

flight 185949 linux-linus real [real]
flight 185953 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185949/
http://logs.test-lab.xenproject.org/osstest/logs/185953/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 185953-retest
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 185953-retest
 test-armhf-armhf-xl-arndale   8 xen-boot            fail pass in 185953-retest
 test-armhf-armhf-examine      8 reboot              fail pass in 185953-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185947

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185953 like 185947
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 185953 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 185953 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185953 never pass
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 185953 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 185953 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185947
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185947
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185947
 test-armhf-armhf-xl-credit2   8 xen-boot                     fail  like 185947
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185947
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185947
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                45db3ab70092637967967bfd8e6144017638563c
baseline version:
 linux                6d7ddd805123e03db0a16c7dbc8509366432a630

Last test of basis   185947  2024-05-08 17:40:50 Z    0 days
Testing same since   185949  2024-05-09 01:43:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Amir Goldstein <amir73il@gmail.com>
  Brian Foster <bfoster@redhat.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Linus Torvalds <torvalds@linux-foundation.org>
  Miklos Szeredi <mszeredi@redhat.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Reed Riley <reed@riley.engineer>
  Steve French <stfrench@microsoft.com>
  Yuezhang Mo <Yuezhang.Mo@sony.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6d7ddd805123..45db3ab70092  45db3ab70092637967967bfd8e6144017638563c -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu May 09 10:41:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 10:41:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719175.1121767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51DJ-0007As-GA; Thu, 09 May 2024 10:41:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719175.1121767; Thu, 09 May 2024 10:41:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51DJ-0007Al-Cl; Thu, 09 May 2024 10:41:25 +0000
Received: by outflank-mailman (input) for mailman id 719175;
 Thu, 09 May 2024 10:41:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VSaB=MM=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s51DH-0007Af-PZ
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 10:41:23 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id acd2f3ea-0df0-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 12:41:21 +0200 (CEST)
Received: from LAPTOP-EFA9O91E.localdomain
 (host-79-60-221-62.business.telecomitalia.it [79.60.221.62])
 by support.bugseng.com (Postfix) with ESMTPSA id 8AE014EE0737;
 Thu,  9 May 2024 12:41:20 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acd2f3ea-0df0-11ef-b4bb-af5377834399
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [XEN PATCH v2] arm/mem_access: add conditional build of mem_access.c
Date: Thu,  9 May 2024 12:39:53 +0200
Message-Id: <ad49a2006a6f19c2db1ff5eabb9ffd666693c4c5.1715250761.git.alessandro.zucchelli@bugseng.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In order to comply to MISRA C:2012 Rule 8.4 for ARM asm/mem_access.h in
the case where MEM_ACCESS=n stubs are needed to allow the conditional
compilation of the users of this header.

Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
---
Changes from v1:
Reverted preprocessor conditional changes to xen/mem_access.h;
added conditional build for xen/mem_access.c;
provided stubs for asm/mem_access.h functions.
---
 xen/arch/arm/Makefile                 | 2 +-
 xen/arch/arm/include/asm/mem_access.h | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 7b1350e2ef..45dc29ea53 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -37,7 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
 obj-y += irq.o
 obj-y += kernel.init.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o
-obj-y += mem_access.o
+obj-$(CONFIG_MEM_ACCESS) += mem_access.o
 obj-y += mm.o
 obj-y += monitor.o
 obj-y += p2m.o
diff --git a/xen/arch/arm/include/asm/mem_access.h b/xen/arch/arm/include/asm/mem_access.h
index 35ed0ad154..2f73172e39 100644
--- a/xen/arch/arm/include/asm/mem_access.h
+++ b/xen/arch/arm/include/asm/mem_access.h
@@ -17,6 +17,7 @@
 #ifndef _ASM_ARM_MEM_ACCESS_H
 #define _ASM_ARM_MEM_ACCESS_H
 
+#include <xen/types.h>
 static inline
 bool p2m_mem_access_emulate_check(struct vcpu *v,
                                   const struct vm_event_st *rsp)
@@ -35,12 +36,20 @@ static inline bool p2m_mem_access_sanity_check(struct domain *d)
  * Send mem event based on the access. Boolean return value indicates if trap
  * needs to be injected into guest.
  */
+#ifdef CONFIG_MEM_ACCESS
 bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec);
 
 struct page_info*
 p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
                                   const struct vcpu *v);
+#else
 
+static inline bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec) {return false;}
+
+static inline struct page_info*
+p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
+                                  const struct vcpu *v) {return NULL;}
+#endif /*CONFIG_MEM_ACCESS*/
 #endif /* _ASM_ARM_MEM_ACCESS_H */
 
 /*
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 10:52:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 10:52:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719180.1121776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51OG-0000Ut-E7; Thu, 09 May 2024 10:52:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719180.1121776; Thu, 09 May 2024 10:52:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51OG-0000Um-BU; Thu, 09 May 2024 10:52:44 +0000
Received: by outflank-mailman (input) for mailman id 719180;
 Thu, 09 May 2024 10:52:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s51OE-0000Ug-Ch
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 10:52:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s51OD-00085E-JC; Thu, 09 May 2024 10:52:41 +0000
Received: from [15.248.2.239] (helo=[10.24.67.31])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s51OD-0000hj-4f; Thu, 09 May 2024 10:52:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=r3vDx8nObzjwmzGlXXp26n113bpq967ebdxZgc3vgXA=; b=RNzf0JrLFwoWriP95zSnVFnD35
	+9nzBQNB/qMXzxJdBjFnazez4j1gHKRZN7PUxRhRxOSPbPvO2WbqV84Wq+EH/lzylZzNyURZUphDB
	mtpoAFCPVoZptZHCQ3eiEKf4KNU7dxuaKoKoZSk8vkua8E5PjPMPE99TLe4147zEk5IY=;
Message-ID: <4c63493b-3743-4d20-a5e2-5f1b3038a9c8@xen.org>
Date: Thu, 9 May 2024 11:52:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] arm/mem_access: add conditional build of
 mem_access.c
Content-Language: en-GB
To: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>
References: <ad49a2006a6f19c2db1ff5eabb9ffd666693c4c5.1715250761.git.alessandro.zucchelli@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ad49a2006a6f19c2db1ff5eabb9ffd666693c4c5.1715250761.git.alessandro.zucchelli@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 09/05/2024 11:39, Alessandro Zucchelli wrote:
> In order to comply to MISRA C:2012 Rule 8.4 for ARM asm/mem_access.h in
> the case where MEM_ACCESS=n stubs are needed to allow the conditional
> compilation of the users of this header.

I think you need to update the commit message given ...

> 
> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
> ---
> Changes from v1:
> Reverted preprocessor conditional changes to xen/mem_access.h;
> added conditional build for xen/mem_access.c;
> provided stubs for asm/mem_access.h functions.
> ---
>   xen/arch/arm/Makefile                 | 2 +-
>   xen/arch/arm/include/asm/mem_access.h | 9 +++++++++
>   2 files changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 7b1350e2ef..45dc29ea53 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -37,7 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
>   obj-y += irq.o
>   obj-y += kernel.init.o
>   obj-$(CONFIG_LIVEPATCH) += livepatch.o
> -obj-y += mem_access.o
> +obj-$(CONFIG_MEM_ACCESS) += mem_access.o

... this not only adding stub.

>   obj-y += mm.o
>   obj-y += monitor.o
>   obj-y += p2m.o
> diff --git a/xen/arch/arm/include/asm/mem_access.h b/xen/arch/arm/include/asm/mem_access.h
> index 35ed0ad154..2f73172e39 100644
> --- a/xen/arch/arm/include/asm/mem_access.h
> +++ b/xen/arch/arm/include/asm/mem_access.h
> @@ -17,6 +17,7 @@
>   #ifndef _ASM_ARM_MEM_ACCESS_H
>   #define _ASM_ARM_MEM_ACCESS_H
>   
> +#include <xen/types.h>

Can you explain why this is needed?

Style: Newline here please.

>   static inline
>   bool p2m_mem_access_emulate_check(struct vcpu *v,
>                                     const struct vm_event_st *rsp)
> @@ -35,12 +36,20 @@ static inline bool p2m_mem_access_sanity_check(struct domain *d)
>    * Send mem event based on the access. Boolean return value indicates if trap
>    * needs to be injected into guest.
>    */
> +#ifdef CONFIG_MEM_ACCESS
>   bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec);
>   
>   struct page_info*
>   p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>                                     const struct vcpu *v);
> +#else
>   
> +static inline bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec) {return false;}

Coding style: The line is well over the 80 characters limit. Also we 
don't tend to provide the implementation of the stub in the single line 
if it is more than {}. So "{return false;}" should look like:

{
    return false;
}

> +
> +static inline struct page_info*
> +p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
> +                                  const struct vcpu *v) {return NULL;}

Ditto.

> +#endif /*CONFIG_MEM_ACCESS*/
>   #endif /* _ASM_ARM_MEM_ACCESS_H */
>   
>   /*

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 09 11:04:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 11:04:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719184.1121788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51Zd-0002N7-GQ; Thu, 09 May 2024 11:04:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719184.1121788; Thu, 09 May 2024 11:04:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51Zd-0002N0-CM; Thu, 09 May 2024 11:04:29 +0000
Received: by outflank-mailman (input) for mailman id 719184;
 Thu, 09 May 2024 11:04:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aro1=MM=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s51Zc-0002Mr-7n
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 11:04:28 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e67b2933-0df3-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 13:04:26 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so162603666b.3
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 04:04:26 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b0195dsm61403666b.184.2024.05.09.04.04.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 04:04:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e67b2933-0df3-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715252666; x=1715857466; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PMufEVeBYg1pSZzp2+OPQIs/m8kDLgfByhJaHU++gLs=;
        b=fVS48UGmqJScjFMYbcxre81Bp81D6Bgu+vO/lMrlIOgzDjGb1qtjax0pZDEVeh+OxU
         jaAf51ILCUYXq11hi0wbDvM8yeR5HzKIWS64J91XPcLtHwXmodF9W4bpVRaaeT/VJSQw
         sbORLzEqN2gJ+6nRpr1FGRa6lakz7UUPxWPcE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715252666; x=1715857466;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PMufEVeBYg1pSZzp2+OPQIs/m8kDLgfByhJaHU++gLs=;
        b=CNB108plyVeRmcLlj0k6NDpWohPqNjETpPtC4ess09TJlmMFBbrC5VFCSWuzasarcL
         C58CeGtJQdB8e2A+vXLhAefeE2G1CvtA69+3JDeNM95FcGOzmtOdU4z1v8/irZnR/Z4p
         vDbOuUOoe7BYxd4R2IbOz741rXKb70KB/uGAkAQBZxbd7lyMHy1HovioxihMGHeZh7gE
         ktAqHZLkCWH5OhG+4TAtdNYfwpcXvRs+LYO4TIBpJTm7E7U7IBp0yXGE1hHcUtIfsqAS
         iVMuXNyYjRPabPjr2f7B/X5eGvibdDzgtBHAfJ3bzN+Yo0Btd2dmU6IHU5TfwH96eABP
         SDSA==
X-Forwarded-Encrypted: i=1; AJvYcCXwi1bru/3HwJ2xZyhidIZuUQ/8gRc0hIpRIAmysRhxW7A+/xnHjvtofQ2lkvlz8gBEE9UhgxyM8fKb3DK7APoTCYxKjnJR0qKtuM1WwRw=
X-Gm-Message-State: AOJu0Yyx24yZGM3Tk1C8mT4dVzyJwJmqbbaulzx+rb3PZcoRgD1dOJNt
	P6N/D5XfaCyzmvYFqaAVkOe9an+QCyhOWY3gLavHX3kRvvuKcYM8VZq9XdK8awZhapkYSGX2bbJ
	W
X-Google-Smtp-Source: AGHT+IG6grUV5/lS2w3bMTtoCAulDTTROye5EolaEoqZa76P2VjxCyKwO58J3r0yHOUXyg8bKxKEgQ==
X-Received: by 2002:a17:906:4fc3:b0:a59:bfd3:2b27 with SMTP id a640c23a62f3a-a59fb9f45dfmr377339866b.70.1715252666270;
        Thu, 09 May 2024 04:04:26 -0700 (PDT)
Message-ID: <d187b452-c847-4681-b6a0-5eaa6cd9b2e7@cloud.com>
Date: Thu, 9 May 2024 12:04:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/8] tools/hvmloader: Wake APs with hypercalls and not
 with INIT+SIPI+SIPI
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
 <45753c70-eef4-4ca7-b7f7-73b452310d4a@citrix.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <45753c70-eef4-4ca7-b7f7-73b452310d4a@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/05/2024 20:13, Andrew Cooper wrote:
> On 08/05/2024 1:39 pm, Alejandro Vallejo wrote:
>> Removes a needless assembly entry point and simplifies the codebase by allowing
>> hvmloader to wake APs it doesn't know the APIC ID of.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> 
> This is basically independent of the rest of the series, and it would be
> good to pull it in separately.  A few notes.
> 
> The commit message ought to note that this has a side effect of removing
> an LMSW instruction which has fastpath at all on AMD CPUs, and requires
> full emulation, and it gets rid of 13 vLAPIC emulations when 3
> hypercalls would do.
> 
> The point being that this is borderline backport material, although it
> does depend on the 32 vCPU bugfix.
> 
>> ---
>> v2:
>>   * New patch. Replaces adding cpu policy to hvmloader in v1.
>> ---
>>  tools/firmware/hvmloader/smp.c | 111 +++++++++++++--------------------
>>  1 file changed, 44 insertions(+), 67 deletions(-)
>>
>> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
>> index 082b17f13818..a668f15d7e1f 100644
>> --- a/tools/firmware/hvmloader/smp.c
>> +++ b/tools/firmware/hvmloader/smp.c
>> @@ -22,88 +22,68 @@
>>  #include "util.h"
>>  #include "config.h"
>>  #include "apic_regs.h"
>> +#include "hypercall.h"
>>  
>> -#define AP_BOOT_EIP 0x1000
>> -extern char ap_boot_start[], ap_boot_end[];
>> +#include <xen/asm/x86-defns.h>
>> +#include <xen/hvm/hvm_vcpu.h>
>> +
>> +#include <xen/vcpu.h>
>>  
>>  static int ap_callin, ap_cpuid;
>>  
>> -asm (
>> -    "    .text                       \n"
>> -    "    .code16                     \n"
>> -    "ap_boot_start: .code16          \n"
>> -    "    mov   %cs,%ax               \n"
>> -    "    mov   %ax,%ds               \n"
>> -    "    lgdt  gdt_desr-ap_boot_start\n"
>> -    "    xor   %ax, %ax              \n"
>> -    "    inc   %ax                   \n"
>> -    "    lmsw  %ax                   \n"
>> -    "    ljmpl $0x08,$1f             \n"
>> -    "gdt_desr:                       \n"
>> -    "    .word gdt_end - gdt - 1     \n"
>> -    "    .long gdt                   \n"
>> -    "ap_boot_end: .code32            \n"
>> -    "1:  mov   $0x10,%eax            \n"
>> -    "    mov   %eax,%ds              \n"
>> -    "    mov   %eax,%es              \n"
>> -    "    mov   %eax,%ss              \n"
>> -    "    movl  $stack_top,%esp       \n"
>> -    "    movl  %esp,%ebp             \n"
>> -    "    call  ap_start              \n"
>> -    "1:  hlt                         \n"
>> -    "    jmp  1b                     \n"
>> -    "                                \n"
>> -    "    .align 8                    \n"
>> -    "gdt:                            \n"
>> -    "    .quad 0x0000000000000000    \n"
>> -    "    .quad 0x00cf9a000000ffff    \n" /* 0x08: Flat code segment */
>> -    "    .quad 0x00cf92000000ffff    \n" /* 0x10: Flat data segment */
>> -    "gdt_end:                        \n"
>> -    "                                \n"
>> -    "    .bss                        \n"
>> -    "    .align    8                 \n"
>> -    "stack:                          \n"
>> -    "    .skip    0x4000             \n"
>> -    "stack_top:                      \n"
>> -    "    .text                       \n"
>> -    );
>> -
>> -void ap_start(void); /* non-static avoids unused-function compiler warning */
>> -/*static*/ void ap_start(void)
>> +static void ap_start(void)
>>  {
>>      printf(" - CPU%d ... ", ap_cpuid);
>>      cacheattr_init();
>>      printf("done.\n");
>> +
>> +    if ( !ap_cpuid )
>> +        return;
>> +
>>      wmb();
>>      ap_callin = 1;
> 
> /* After this point, the BSP will shut us down. */
> 
>> -}
>>  
>> -static void lapic_wait_ready(void)
>> -{
>> -    while ( lapic_read(APIC_ICR) & APIC_ICR_BUSY )
>> -        cpu_relax();
>> +    while ( 1 )
> 
> For this, we tend to favour "for ( ;; )".
> 
>> +        asm volatile ( "hlt" );
>>  }
>>  
>>  static void boot_cpu(unsigned int cpu)
>>  {
>> -    unsigned int icr2 = SET_APIC_DEST_FIELD(LAPIC_ID(cpu));
>> +    static uint8_t ap_stack[4 * PAGE_SIZE] __attribute__ ((aligned (16)));
> 
> I know you're just copying what was there, but 4 pages is stupidly large
> for something that needs about 4 stack slots.
> 
> 4K is absolutely plenty.
> 
>> +    static struct vcpu_hvm_context ap;
>>  
>>      /* Initialise shared variables. */
>>      ap_cpuid = cpu;
>> -    ap_callin = 0;
>>      wmb();
>>  
>> -    /* Wake up the secondary processor: INIT-SIPI-SIPI... */
>> -    lapic_wait_ready();
>> -    lapic_write(APIC_ICR2, icr2);
>> -    lapic_write(APIC_ICR, APIC_DM_INIT);
>> -    lapic_wait_ready();
>> -    lapic_write(APIC_ICR2, icr2);
>> -    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
>> -    lapic_wait_ready();
>> -    lapic_write(APIC_ICR2, icr2);
>> -    lapic_write(APIC_ICR, APIC_DM_STARTUP | (AP_BOOT_EIP >> 12));
>> -    lapic_wait_ready();
>> +    /* Wake up the secondary processor */
>> +    ap = (struct vcpu_hvm_context) {
>> +        .mode = VCPU_HVM_MODE_32B,
>> +        .cpu_regs.x86_32 = {
>> +            .eip = (uint32_t)ap_start,
>> +            .esp = (uint32_t)ap_stack + ARRAY_SIZE(ap_stack),
> 
> Not that it really matters, but these want to be unsigned long casts.
> 
>> +
>> +            /* Protected mode with MMU off */
>> +            .cr0 = X86_CR0_PE,
>> +
>> +            /* Prepopulate the GDT */
> 
> /* 32bit Flat Mode */
> 
> This isn't the GDT; it's the segment registers, but "Flat Mode" is the
> more meaningful term to use.
> 
> 
> I'm happy to fix all on commit.
> 
> ~Andrew

All sound ok to me.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 09 11:05:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 11:05:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719186.1121798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51aE-0002o6-On; Thu, 09 May 2024 11:05:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719186.1121798; Thu, 09 May 2024 11:05:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51aE-0002nz-K8; Thu, 09 May 2024 11:05:06 +0000
Received: by outflank-mailman (input) for mailman id 719186;
 Thu, 09 May 2024 11:05:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w/8E=MM=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s51aD-0002mg-Kt
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 11:05:05 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb70cf7f-0df3-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 13:05:03 +0200 (CEST)
Received: from DM5PR07CA0117.namprd07.prod.outlook.com (2603:10b6:4:ae::46) by
 DS7PR12MB8371.namprd12.prod.outlook.com (2603:10b6:8:e9::18) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7544.42; Thu, 9 May 2024 11:04:59 +0000
Received: from DS1PEPF0001708E.namprd03.prod.outlook.com
 (2603:10b6:4:ae:cafe::ee) by DM5PR07CA0117.outlook.office365.com
 (2603:10b6:4:ae::46) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48 via Frontend
 Transport; Thu, 9 May 2024 11:04:59 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS1PEPF0001708E.mail.protection.outlook.com (10.167.17.134) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Thu, 9 May 2024 11:04:58 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 9 May
 2024 06:04:58 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 9 May 2024 06:04:57 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb70cf7f-0df3-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oFWHu18ARKRHYNQFn+mHirwiJ+wKSZ0wIpItQgFn13fSnwlCJu97CqQWAk+zB5MTVYgInY/G1Mi2Bfz+FyN/yhP2ILIOqsGs+AFWep/yfZdN+SUakvZubMPki1tXMBYzFcGCQJa7yHYD/RqmMOxS0+cBo00mkTgf9Tc0cmFupA9swvkuBTUA4lDBQ7W8Lujuaigk3hrD8iySuSeAzQtTLyWEHqRzHxI3/I+pJd3nhjcnV6ktGZnpQb9atLYzj1B8khkLLpKyHRgvhtEvDJKzf2hSlK5Im+UlohRJFzYd0ZwuKczPsE5ScS0G7ErOE/U50EdGcXbAXylLWWBoCvQejQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=58tk2iO5nZkSEk4pdwHDQp+LLGrAotAPa3v3zDIQrG8=;
 b=iL0506ofxRA9GYOmnIuvKsxYAvbQ+RFRR2DIpAayWBalCpXyTQI2JQIEqCSrGa2K3v28Mo4I9eajdeyc+1NJf4PgHX5olfG1cyJpQORw8EI9EuCp+UbPo6sDXqoe0SC90SVLjh4Rmf5a5HWurV1JiTOS1Ob6E8HtvHZ3FkNRpHxnaHK76djaKBkkO/c19/9eAevKazRRo6fhsBnt2Y2C1gPbXsLWXvVWi2eWMcNouwSokCafPKn9oNSWMuIliQ/Kl8SNVgOGgmFjQEctKgF/8MNFSZQlsvwHQ8aK81q31xrI1RlNvjxViSPuSONNVVDbzJIqatExztAPYbNLSEbIcw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=58tk2iO5nZkSEk4pdwHDQp+LLGrAotAPa3v3zDIQrG8=;
 b=aWRjUSjo1hzuVT+Z5dt0K35n/stdWE2L4R6Ea/fs7NT+5hhMquGM5n5FaS8i+nAvFS9fg70o6zvOUhOrKXIHJX/ZYY4B5JYErqojE5BmS3Sj4RqXBoQ14QD0eiw2pzURU9f9kyEF1VsrfqmjIXIh1y4JMFnhT7L19tD765KHfwE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <43e6fb55-bab0-4d61-9e18-9aa215157341@amd.com>
Date: Thu, 9 May 2024 13:04:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/7] xen/arm: Rework heap page allocation outside
 allocate_bank_memory
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-6-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-6-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF0001708E:EE_|DS7PR12MB8371:EE_
X-MS-Office365-Filtering-Correlation-Id: b55f75bc-336c-4243-5634-08dc7017dde6
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZGhydFd0RlNBc2V0SmtaYTJjTG9hdU1OYmp1T0wzcTVPWEVQZnB1aFR2WDE3?=
 =?utf-8?B?TEo4VXhHb0tlR0F0VENmU1dEczFubHo0Tm9XdnVmdmVETmlRRnZ3RFZQM0h4?=
 =?utf-8?B?WGJkcnNxcFBaaWZmM3FQdEtRallFd1l3czQwejRwZ1I5a1ZzZEpHN0ZUZndZ?=
 =?utf-8?B?TWFmdGpQZm04WFZ2WmFhSE9KMTY4OTYrSVB2WmVFT2dLeWFRazdDSGRyNU5V?=
 =?utf-8?B?UWw3Z0FKdVJHb1pKU09UeG1NK1lBNjdEaGg5Vk5aZW1TbHJzNE5QQXY2c2I4?=
 =?utf-8?B?YzQ5Sm5vNE15N2E3ZDUvWUUwVkZUR1ZWL3RpK3lNaklMd1FNV0RJNWJEcEJE?=
 =?utf-8?B?Qng0TjNCOHU2S3hJSUlPdUoxWVlNRDQxT3R6M1gzQTB1M2c1aFNDbW5IS1g3?=
 =?utf-8?B?SWtzYWFMSXJ1aTlwSjJZK3lpcmltZXpsdlo0WVo5TnRnUWlGUmdwK1k1Qmlw?=
 =?utf-8?B?a0ZFMGQwa2JRMmRKa3IvMEFOdkNDVnVpMFJEY0tmbmplS2Nzc1lQWUdqMHQ1?=
 =?utf-8?B?Wjg3djJDT1JEbFNjTWE1Sy94WXlrdTRBbFprVHVNMmV0UnFpb0UzT1dvaWsy?=
 =?utf-8?B?dnc2WTZyOWlxV004ZmhZMlMrUkdQNCtKZVBUcW8xOXpRVVU2V3dYWEZrYndm?=
 =?utf-8?B?ZEZ2d3JXR080cUJQcHJDRDNJZHl4WTE3T0ovOUJ1ZnpSNzBHNTFWcUlWMjJC?=
 =?utf-8?B?NVBYUXRHQ0pjTW1zcmppN3UxQjFsenJWSDI5aGp4MkM3d2JXUmQ0YzBPRDlH?=
 =?utf-8?B?OWJsV3dFaVdpWC9HcDRZOStlS0ZFWXhpSDNiaVFmTjBYWUMrajdzR0M2c1FX?=
 =?utf-8?B?WUd1aXg3OEdzZWVieFVMbkVnS1JwaUNVbGtqNktLT2RGNys3bFdYSXR4Tk9j?=
 =?utf-8?B?d2xjWHJCTzFDdFlMTG9hemUrV2Zid09INCsrVGJ0UnVPM0RKVWwyc0NLbktm?=
 =?utf-8?B?TkFBRVBpVkk0alAyRmlRSTRYSTNOQUtYWk9EajNjeEJYQ3dDQXJSYkZVVnMr?=
 =?utf-8?B?My80QytjQUFjcUVIOEwwbHZWUmxsS3F4NHlBZjlxRXhSWDd2MjBkRHNyUkVn?=
 =?utf-8?B?MU41elZsbVIxVjlEc0o5Y2lIZmpLbHgrbVVlU3NLK014U1NBM09oWVVwWWlG?=
 =?utf-8?B?cDQrNjA2Sm9DWjJNR21hc1UvdEEzdVJWSVJCdVZabUZqY05tcURRaGhOQUV2?=
 =?utf-8?B?NkpuNEk0anhVbUF2a29XOUR3ME51U0JXTU9rUEFsQ2Jqa2ZHYnd4U05vWHpM?=
 =?utf-8?B?ZjREbGJPZ3lxUnU4STRtdUgwN0t1ZjJUekZibTU4VUh6dEtkVFZXM2pwWGps?=
 =?utf-8?B?ZnZITjZ2dHplVmFrNjFySldsRzNoWk9vV0psRjN3R3ZGNDdIRC9XS01iRWtJ?=
 =?utf-8?B?R0krWnBZT0dCMUswY2s3dDJacldLMTltQ3NYRnhnUVVLZFZyUlBLdUhkWUZi?=
 =?utf-8?B?TzhRSHpNTnBRUERqdkFMUENiUllTT3JEaWQ2c3lTMzRMbnRqaGZ5M2VOR2NX?=
 =?utf-8?B?Z3orUG9nZ2VERkZxeHFCRUVtbFZIbjdFUmROS0xJVTZ4bWpJajdKRlIvZzJD?=
 =?utf-8?B?OFVnV0lDQXdKNG53R0Y5azRZNWpROU1kU3Fwcm9qZ0xrNlMzSloxQnN6VzZQ?=
 =?utf-8?B?MnUvNEEwQ3cvNCtqdGpXNXRKbnZnVElrb2EyNnFqMVh6bUNFMk1ZaXg3eUdG?=
 =?utf-8?B?VUtRS2h4YnQySmhTRTg5b3hSZU9SWjM5MmxEM0kvNU02QTV1WXlkRW1GSy9h?=
 =?utf-8?B?SFN1UDNGd2NRTllOMlRQTFhFTTBQdk1tcXhyN2VicHMwbSt5OGI1Nk1ZSzdu?=
 =?utf-8?B?VjU3Yi9WdHFOS2QrVXVZOVpMSVJmRE4vUlNNWnZVL3hEK2hxaGdTaERtbDVt?=
 =?utf-8?Q?AkCCdnd30S3Qj?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 11:04:58.8249
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b55f75bc-336c-4243-5634-08dc7017dde6
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF0001708E.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8371

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> The function allocate_bank_memory allocates pages from the heap and
> map them to the guest using guest_physmap_add_page.
s/map/maps

> 
> As a preparation work to support static shared memory bank when the
> host physical address is not provided, Xen needs to allocate memory
> from the heap, so rework allocate_bank_memory moving out the page
> allocation in a new function called allocate_domheap_memory.
> 
> The function allocate_domheap_memory takes a callback function and
> a pointer to some extra information passed to the callback and this
> function will be called for every page allocated, until a defined
for every region given that you pass sgfn and order

> size is reached.
> 
> In order to keep allocate_bank_memory functionality, the callback
> passed to allocate_domheap_memory is a wrapper for
> guest_physmap_add_page.
> 
> Let allocate_domheap_memory be externally visible, in order to use
> it in the future from the static shared memory module.
> 
> Take the opportunity to change the signature of allocate_bank_memory
> and remove the 'struct domain' parameter, which can be retrieved from
> 'struct kernel_info'.
> 
> No functional changes is intended.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  xen/arch/arm/dom0less-build.c           |  4 +-
>  xen/arch/arm/domain_build.c             | 77 +++++++++++++++++--------
>  xen/arch/arm/include/asm/domain_build.h |  9 ++-
>  3 files changed, 62 insertions(+), 28 deletions(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242f4..20ddf6f8f250 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -60,12 +60,12 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
> 
>      mem->nr_banks = 0;
>      bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem);
> -    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
> +    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
>                                 bank_size) )
>          goto fail;
> 
>      bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem);
> -    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
> +    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
>                                 bank_size) )
>          goto fail;
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 0784e4c5e315..148db06b8ca3 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -417,26 +417,13 @@ static void __init allocate_memory_11(struct domain *d,
>  }
> 
>  #ifdef CONFIG_DOM0LESS_BOOT
> -bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
> -                                 gfn_t sgfn, paddr_t tot_size)
> +bool __init allocate_domheap_memory(struct domain *d, paddr_t tot_size,
> +                                    alloc_domheap_mem_cb cb, void *extra)
>  {
> -    struct membanks *mem = kernel_info_get_mem(kinfo);
> -    int res;
> +    unsigned int max_order = UINT_MAX;
>      struct page_info *pg;
You can limit the visibility of these variables by moving them into loop

> -    struct membank *bank;
> -    unsigned int max_order = ~0;
> 
> -    /*
> -     * allocate_bank_memory can be called with a tot_size of zero for
> -     * the second memory bank. It is not an error and we can safely
> -     * avoid creating a zero-size memory bank.
> -     */
> -    if ( tot_size == 0 )
> -        return true;
> -
> -    bank = &mem->bank[mem->nr_banks];
> -    bank->start = gfn_to_gaddr(sgfn);
> -    bank->size = tot_size;
> +    BUG_ON(!cb);
No need for that

> 
>      while ( tot_size > 0 )
>      {
> @@ -463,17 +450,61 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
>              continue;
>          }
> 
> -        res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
> -        if ( res )
> -        {
> -            dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
> +        if ( cb(d, pg, order, extra) )
>              return false;
> -        }
> 
> -        sgfn = gfn_add(sgfn, 1UL << order);
>          tot_size -= (1ULL << (PAGE_SHIFT + order));
>      }
> 
> +    return true;
> +}
> +
> +static int __init guest_map_pages(struct domain *d, struct page_info *pg,
Does it make sense to return int if it is not taken into account by the user?

> +                                  unsigned int order, void *extra)
> +{
> +    gfn_t *sgfn = (gfn_t *)extra;
> +    int res;
> +
> +    BUG_ON(!sgfn);
> +    res = guest_physmap_add_page(d, *sgfn, page_to_mfn(pg), order);
> +    if ( res )
> +    {
> +        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
> +        return res;
> +    }
> +
> +    *sgfn = gfn_add(*sgfn, 1UL << order);
> +
> +    return 0;
> +}
> +
> +bool __init allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
> +                                 paddr_t tot_size)
> +{
> +    struct membanks *mem = kernel_info_get_mem(kinfo);
> +    struct domain *d = kinfo->d;
> +    struct membank *bank;
> +
> +    /*
> +     * allocate_bank_memory can be called with a tot_size of zero for
> +     * the second memory bank. It is not an error and we can safely
> +     * avoid creating a zero-size memory bank.
> +     */
> +    if ( tot_size == 0 )
> +        return true;
> +
> +    bank = &mem->bank[mem->nr_banks];
> +    bank->start = gfn_to_gaddr(sgfn);
> +    bank->size = tot_size;
> +
> +    /*
> +     * Allocate pages from the heap until tot_size and map them to the guest
until tot_size is 0

> +     * using guest_map_pages, passing the starting gfn as extra parameter for
> +     * the map operation.
> +     */
> +    if ( !allocate_domheap_memory(d, tot_size, guest_map_pages, &sgfn) )
> +        return false;
> +
>      mem->nr_banks++;
>      kinfo->unassigned_mem -= bank->size;
> 
> diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
> index 45936212ca21..9eeb5839f6ed 100644
> --- a/xen/arch/arm/include/asm/domain_build.h
> +++ b/xen/arch/arm/include/asm/domain_build.h
> @@ -5,9 +5,12 @@
>  #include <asm/kernel.h>
> 
>  typedef __be32 gic_interrupt_t[3];
> -
> -bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
> -                          gfn_t sgfn, paddr_t tot_size);
> +typedef int (*alloc_domheap_mem_cb)(struct domain *d, struct page_info *pg,
> +                                    unsigned int order, void *extra);
> +bool allocate_domheap_memory(struct domain *d, paddr_t tot_size,
> +                             alloc_domheap_mem_cb cb, void *extra);
> +bool allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
> +                          paddr_t tot_size);
>  int construct_domain(struct domain *d, struct kernel_info *kinfo);
>  int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit);
>  int make_chosen_node(const struct kernel_info *kinfo);
> --
> 2.34.1
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Thu May 09 11:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 11:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719197.1121807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51wg-0006bT-Kd; Thu, 09 May 2024 11:28:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719197.1121807; Thu, 09 May 2024 11:28:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s51wg-0006bM-Hr; Thu, 09 May 2024 11:28:18 +0000
Received: by outflank-mailman (input) for mailman id 719197;
 Thu, 09 May 2024 11:28:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s51wf-0006bG-Nl
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 11:28:17 +0000
Received: from mail-vs1-xe33.google.com (mail-vs1-xe33.google.com
 [2607:f8b0:4864:20::e33])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39d2cdd5-0df7-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 13:28:15 +0200 (CEST)
Received: by mail-vs1-xe33.google.com with SMTP id
 ada2fe7eead31-47eee2a2a87so253854137.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 04:28:15 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df56d1915sm7028931cf.87.2024.05.09.04.28.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 04:28:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39d2cdd5-0df7-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715254094; x=1715858894; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=pOCjym16+RdPYo3c/L1pOeEY5AawlMiKptW3c5QhqG4=;
        b=QdqKImOzgiMUhsN2++MhctZJbT1nTQzEg4WjcQTX+UqEV9a5D8xYSy02+CGC+ZvQZG
         3FZ95NWSIEAJNEHcxw4amYnHjJLVX7jMYj0dsUnHvTjTcbtjpiBa08amqpunSsZ5ua7d
         JHeLs7QIWHNElL0Z3+k8gyJVpfSqYOFC+n6+c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715254094; x=1715858894;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pOCjym16+RdPYo3c/L1pOeEY5AawlMiKptW3c5QhqG4=;
        b=BaAfv7xJspjsKC+CWcRtKoVhiVYH82G4fUQF3BXuB4iKOL4cOU+qaUW7weEgGK4UwE
         7KYdZxXnezc0nFc0G7RFnQyXqJd7Z9ZeMeuAk3HvC1l7WpG4oxjEp+zaV/e+t62p98Sa
         kMRLsVeT3UpiQfj3Ya3RLhYxS0a7rAlNjGTdd1fLSJYw7SN6s55KiQmidF/KKRdkRa3N
         fIMVOQF2oR94yymq8K7cYMGggRrx+d7lQs0ZPQCgMTl3NypkOw1B2vvrhjTbYLFPNmlf
         Z/8vGER4UFCoNhq+EfYtMuCXr0RDVWk+ijhkzAiNtfRTDrJgl/AAYpwW2fmdblmHorpl
         UHYw==
X-Forwarded-Encrypted: i=1; AJvYcCW3VufR1wW+jstqTEdkEP4Vf76PppRhOoFQL2k6Z5A+V85L34TGY2NomnwG8e80qsVzYPWZSterPMWlbB4WPuSLV66esLrf08owsHdGa2s=
X-Gm-Message-State: AOJu0YxsMbo/MCXqonXrTIvMQchM7KMeQ1w6cCOwBichjXG7wogyb8qo
	lAFepdGtQQ3/yxjV/P38/0Sc4HQFa6LxSQ2lGI7L+fpNHd2cfxyyMChn3YrSbBw=
X-Google-Smtp-Source: AGHT+IEgPvSMBPtpv9kf4y2Uy8xaL+az55dduRrPZtkFAC8981a2BkpkRsshy4r9k4ZXa2J3cl8Geg==
X-Received: by 2002:a67:f405:0:b0:47c:1251:cf25 with SMTP id ada2fe7eead31-47f3c2dff19mr5223145137.12.1715254094428;
        Thu, 09 May 2024 04:28:14 -0700 (PDT)
Date: Thu, 9 May 2024 13:28:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Message-ID: <ZjyzTCOSeNvPq2Np@macbook>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>
 <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org>

On Thu, May 09, 2024 at 10:50:56AM +0100, Julien Grall wrote:
> 
> 
> On 09/05/2024 09:13, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
> > Also the interactions with the remote domain would need to be audited,
> > as the remote domain shattering the superpage would need to be
> > replicated in the mapping side in order to account for the changes.
> 
> ... I don't understand this one. How is this different from today's where a
> domain can foreign map a 2MB which may be using a superpage in the remote
> domain?

Hm, right, I was wrong with that I think, as long as proper references
as taken for the superpage entries it should be fine.

> >  Not sure all paths will be easy to
> > audit for preemption if it's more than relinquish_p2m_mapping() that
> > you need to adjust.
> 
> I thought about it yesterday. But I came to the conclusion that if we have
> any concern about removing 1GB foreign superpage then we would already have
> the problem today as a domain can map contiguously 1GB worth of foreign
> mapping using small pages.

Yeah, but in that case addition or removal is done in 4K chunks, and
hence we can preempt during the operation.

OTOH for 1GB given the code here the page could be freed in one go,
without a chance of preempting the operation.

Maybe you have to shatter superpages into 4K entries and then remove
them individually, as to allow for preemption to be possible by
calling put_page() for each 4K chunk?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 11:55:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 11:55:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719203.1121817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52Me-0002th-GM; Thu, 09 May 2024 11:55:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719203.1121817; Thu, 09 May 2024 11:55:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52Me-0002ta-DP; Thu, 09 May 2024 11:55:08 +0000
Received: by outflank-mailman (input) for mailman id 719203;
 Thu, 09 May 2024 11:55:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s52Mc-0002tQ-M5; Thu, 09 May 2024 11:55:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s52Mc-0000gM-GN; Thu, 09 May 2024 11:55:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s52Mc-0003Mc-3D; Thu, 09 May 2024 11:55:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s52Mc-0006G8-2l; Thu, 09 May 2024 11:55:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Uxv+GBF4m+k4HOeRiV8H2in+o9E6Vna+EEyKizIGiX8=; b=6MJ8YXmNS00tQ7E8M5OpjqAW7J
	ASocXYst7nRzqL+vdrLxPHEFLvFbRUtSqYagCoglYJW1pOCiBmWjtD6xH24VgBYOYjJbYq4/VN908
	VM3+enC9Yk+Q+lG5JTh5eNmjHZEeM/uA2Q9vo05s1l0pnLsnLfVBzOoAJH+JnnmjFRdM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185954-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185954: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e8e8afee990ae74a4d2797ddae1b58a4e03912b5
X-Osstest-Versions-That:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 11:55:06 +0000

flight 185954 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185954/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  e8e8afee990ae74a4d2797ddae1b58a4e03912b5
baseline version:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b

Last test of basis   185939  2024-05-07 17:03:58 Z    1 days
Testing same since   185954  2024-05-09 09:04:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a2330b51df..e8e8afee99  e8e8afee990ae74a4d2797ddae1b58a4e03912b5 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 09 12:04:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 12:04:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719222.1121827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52VV-0004rk-OU; Thu, 09 May 2024 12:04:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719222.1121827; Thu, 09 May 2024 12:04:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52VV-0004rd-L3; Thu, 09 May 2024 12:04:17 +0000
Received: by outflank-mailman (input) for mailman id 719222;
 Thu, 09 May 2024 12:04:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y+Ct=MM=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s52VV-0004rW-1t
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:04:17 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4122a765-0dfc-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 14:04:14 +0200 (CEST)
Received: from nico.bugseng.com.homenet.telecomitalia.it
 (host-79-60-221-62.business.telecomitalia.it [79.60.221.62])
 by support.bugseng.com (Postfix) with ESMTPSA id C863E4EE0737;
 Thu,  9 May 2024 14:04:11 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4122a765-0dfc-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH] automation/eclair_analysis: tag MISRA C Rule 8.2 as clean.
Date: Thu,  9 May 2024 14:04:07 +0200
Message-Id: <8d29601a2aa1a371daadc024752b7a24f200fab4.1715256189.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Tag the rule as clean, as there are no more violations in the codebase
since e8e8afee990a ("svm: Fix MISRA 8.2 violation").

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index bdf94ed99676..a014500a4118 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 12:07:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 12:07:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719228.1121837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52Y6-0005eb-4q; Thu, 09 May 2024 12:06:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719228.1121837; Thu, 09 May 2024 12:06:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52Y6-0005eU-1B; Thu, 09 May 2024 12:06:58 +0000
Received: by outflank-mailman (input) for mailman id 719228;
 Thu, 09 May 2024 12:06:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1XlW=MM=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s52Y5-0005eM-IH
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:06:57 +0000
Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com
 [2607:f8b0:4864:20::f2b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a12b58ca-0dfc-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 14:06:56 +0200 (CEST)
Received: by mail-qv1-xf2b.google.com with SMTP id
 6a1803df08f44-69b59c10720so3498266d6.3
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 05:06:56 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f194bf9sm6275516d6.58.2024.05.09.05.06.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 05:06:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a12b58ca-0dfc-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715256415; x=1715861215; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=95kgWf4VLs7wrXV+tckodQzFphNwJklB7/vg9y61p/o=;
        b=fsfCiOl4KvsngiAYQY41IA/wjfWa0vJa+wxz4/8Yhjoi64w7IoaLdQn3THQc09qAD2
         zxEqy3n2qFSECTt8ZRkArIurntNu0D9r1OPOCljg7Dya0BWd4z4yKZKGiMKliD1sUNro
         7EguiCc5sZTk6s73zQNOPRDu5UnSLYndjnmfY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715256415; x=1715861215;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=95kgWf4VLs7wrXV+tckodQzFphNwJklB7/vg9y61p/o=;
        b=jG5i+WnF9BO6mDVIEtNXouuQCDlmzA3M6Bs0yTn3jaHtJcWB6Ir9qpmbZU8foFq787
         p1i1dBKUWg4JGYbj/fa5lTcTlNio/cO6tCAlcGk/xxgNRKC7uWixqiECmB83bXtVPG8R
         9rtXfr/7vjXO/wJngECKT6ECUgfzVuKzJs+tNL5/5lp1/sfkGOEBpzA86gifeKPzf7Ft
         WLNmVo6H6/NAULp5P52b0N2q+jZwj2LHyjf2IJppH/DfG4YFscIlF5O8RTkmk9tbPn84
         C3oe+MKuP4ef23wqMxEe+vpFYj91Ms09gmwTd3MuE9LvWLStA000+sx+aVsnEedC+s4L
         jOzw==
X-Forwarded-Encrypted: i=1; AJvYcCV9oCnP+ChLcV5HcCo+BVTs3R7v8CpIQhcIi/QqeyZFDcEM4HDbe/8dIZbOiKqRbSPV8Nkz7+ot7IiR8EgN42upkxqK7Zow2S70u3Eb5dE=
X-Gm-Message-State: AOJu0Yy3Z+4wVJQg1bUCXklyo4x4jdEmxwhk3zgbs0aFq1Dl8Lcqq/H7
	XFSAX6XKUHTJmKjhi+kxowiGbxAA01GMFkid0g5fI1fLuwDDXEkKpBDyU3xB12Q=
X-Google-Smtp-Source: AGHT+IGvV8YgQbPefaLWLHQDt/RnPz7je16Rw9Lc66leVr0Su0cm2wRAk5lDUfVi+JsgCZ2bLwaG8Q==
X-Received: by 2002:ad4:5d4c:0:b0:696:a982:92a8 with SMTP id 6a1803df08f44-6a15143c76cmr69637136d6.3.1715256415353;
        Thu, 09 May 2024 05:06:55 -0700 (PDT)
Message-ID: <faea922f-05c9-4c5c-9503-ace9ad960a77@citrix.com>
Date: Thu, 9 May 2024 13:06:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair_analysis: tag MISRA C Rule 8.2 as
 clean.
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>
References: <8d29601a2aa1a371daadc024752b7a24f200fab4.1715256189.git.nicola.vetrini@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <8d29601a2aa1a371daadc024752b7a24f200fab4.1715256189.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09/05/2024 1:04 pm, Nicola Vetrini wrote:
> Tag the rule as clean, as there are no more violations in the codebase
> since e8e8afee990a ("svm: Fix MISRA 8.2 violation").
>
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

I'll include this shortly.


From xen-devel-bounces@lists.xenproject.org Thu May 09 12:12:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 12:12:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719232.1121847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52d2-00076g-M2; Thu, 09 May 2024 12:12:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719232.1121847; Thu, 09 May 2024 12:12:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52d2-00076Z-JL; Thu, 09 May 2024 12:12:04 +0000
Received: by outflank-mailman (input) for mailman id 719232;
 Thu, 09 May 2024 12:12:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s52d0-00076T-W5
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:12:03 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s52d0-0000zu-Mw; Thu, 09 May 2024 12:12:02 +0000
Received: from [15.248.2.239] (helo=[10.24.67.31])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s52d0-0005lf-B8; Thu, 09 May 2024 12:12:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=rkPAf+otm3SjNAG5343agvv9UQeGDPXADvf0hs0GFd4=; b=DZzWHSLAi9fENydgKe3YbzY4us
	NpgY/vE9Lc1InQanWHOOC6cJ/E1AU+cAWNeKQSDGgxhK7WH07rucuUVdo6nQm96CJqNGbVnM1ZEAJ
	wDMYx01CLEa0X8hahkSsttRr+WWM3sIimFRUiCv5OyOztKVL0vaXiPGytw75wKve7mkc=;
Message-ID: <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org>
Date: Thu, 9 May 2024 13:12:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
 <Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org> <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org> <ZjyzTCOSeNvPq2Np@macbook>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ZjyzTCOSeNvPq2Np@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 09/05/2024 12:28, Roger Pau Monné wrote:
> On Thu, May 09, 2024 at 10:50:56AM +0100, Julien Grall wrote:
>>
>>
>> On 09/05/2024 09:13, Roger Pau Monné wrote:
>>> On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
>>> Also the interactions with the remote domain would need to be audited,
>>> as the remote domain shattering the superpage would need to be
>>> replicated in the mapping side in order to account for the changes.
>>
>> ... I don't understand this one. How is this different from today's where a
>> domain can foreign map a 2MB which may be using a superpage in the remote
>> domain?
> 
> Hm, right, I was wrong with that I think, as long as proper references
> as taken for the superpage entries it should be fine.
> 
>>>   Not sure all paths will be easy to
>>> audit for preemption if it's more than relinquish_p2m_mapping() that
>>> you need to adjust.
>>
>> I thought about it yesterday. But I came to the conclusion that if we have
>> any concern about removing 1GB foreign superpage then we would already have
>> the problem today as a domain can map contiguously 1GB worth of foreign
>> mapping using small pages.
> 
> Yeah, but in that case addition or removal is done in 4K chunks, and
> hence we can preempt during the operation.

I am not entirely sure how that would work. From my understand, today, 
most of the users of the P2M code expects the operation to complete in 
one go and if preemption is needed then the caller is responsible to 
handle it by breaking up the happy.

With your suggestion, it sounds like you want to rework how the 
preemption today and push it to the P2M code. This would mean we would 
need to modify all the callers to check for -EERESTART (or similar) and 
also tell them how many pages were handled so the call can be restarted 
where it stopped. Is it what you had in mind?

I don't expect the work to be trivial, so I wonder if this is really 
worth it to try to change the way we preempt.

> 
> OTOH for 1GB given the code here the page could be freed in one go,
> without a chance of preempting the operation.
> 
> Maybe you have to shatter superpages into 4K entries and then remove
> them individually, as to allow for preemption to be possible by
> calling put_page() for each 4K chunk?
This would require to allocate some pages from the P2M pool for the 
tables. As the pool may be exhausted, it could be problematic when 
relinquishing the resources.

It may be possible to find a way to have memory available by removing 
other mappings first. But it feels a bit hackish and I would rather 
prefer if we avoid allocating any memory when relinquishing.

So I think we want to allow at most 2MB superpages for foreign mapping.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 09 12:13:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 12:13:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719235.1121857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52e5-0007bj-Uk; Thu, 09 May 2024 12:13:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719235.1121857; Thu, 09 May 2024 12:13:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s52e5-0007bc-Rq; Thu, 09 May 2024 12:13:09 +0000
Received: by outflank-mailman (input) for mailman id 719235;
 Thu, 09 May 2024 12:13:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s52e4-0007bS-KR
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:13:08 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7dc32c36-0dfd-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 14:13:06 +0200 (CEST)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-792bd1f5b38so63816185a.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 05:13:06 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2fc57asm60858485a.81.2024.05.09.05.13.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 05:13:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7dc32c36-0dfd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715256785; x=1715861585; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=32AosGZ+UPa7S5J79mBww78Vop5lui3LlGLrxy9W2Qk=;
        b=YuscnrYsPOJgAlsNH8UHh/f4RMPH90AAF8alDpknJUBRzVui6TQWB/DfeHibTCjF8K
         P7CdjoL9/XUzLmZHXZ2HV8jJX8StJ0WBaxbtNPfKDepvtx88B7ia8BockC+RKgX5Uwh8
         4BvfS8d4mWDDUOKs6oueqRlA1jJOcGFOVPPZk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715256785; x=1715861585;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=32AosGZ+UPa7S5J79mBww78Vop5lui3LlGLrxy9W2Qk=;
        b=Vmf3VoxEeahQ3IfFvFHuSCc406e1rh9ExhKM7aH41HgHR/cTWNExJwK29CrbxSEA3n
         00EmdI4UzoB2M0i3k576EMu+cZtYyb+294QPT/hGC2OorFz5UbBncSWpDcj3yCLJ3YnB
         R4YtlreOCZI1iJgDrg/XeoM81K85JxcG4YRAK8JA36iHDkQB0YVpRHYYzgnByTC1qEqj
         DrHfCckeIUWfu8eA7ykjJfLN0g5db2IV6UB33Tdxhuoo6JEt9bSVsUT8SWiAqzE5564X
         aD6Su3J+fjfhsILUimNDm5Iw0KOMsCuWh3ikjG3LxD4RsyEspD1diBpafaJo87m5WK0I
         SjwQ==
X-Forwarded-Encrypted: i=1; AJvYcCUDDy5w+yVKegefgWbVy92WTxdkWdxibmeu1caW7NQgehGWWTPIOiKPqBHEV9mK2bP7dkvSmoy8Hfj2Z+eVqGrwO5Gk1y0icBEakk04tJM=
X-Gm-Message-State: AOJu0YwrfWtLguBzMFScBmPUcB4WRF91tdA4REN07hWRM8KlgtWA/fD0
	eyPCvjKb1xTmtA8gud16V3wmzl0XeHWqwc+uWa7rB0TVf0JOq1vhmV0JYXBdCIY=
X-Google-Smtp-Source: AGHT+IGJMWgEDYfhIAYWPQrq5m4Syq+n7eW/SSKwm3mtf34ktSsfdRL9gPKl8tYvWrFm7g0LRSEL1w==
X-Received: by 2002:ae9:f70e:0:b0:792:c13a:1a80 with SMTP id af79cd13be357-792c13a1b3bmr81648985a.44.1715256785284;
        Thu, 09 May 2024 05:13:05 -0700 (PDT)
Date: Thu, 9 May 2024 14:13:02 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy
 defined names
Message-ID: <Zjy9zmA1kjgZt9s8@macbook>
References: <20240502114922.94288-1-roger.pau@citrix.com>
 <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com>

On Thu, May 02, 2024 at 03:16:54PM +0200, Jan Beulich wrote:
> On 02.05.2024 13:49, Roger Pau Monne wrote:
> > Like it was done recently for libxl, switch to using the auto-generated feature
> > names by the processing of cpufeatureset.h, this allows removing the open-coded
> > feature names, and unifies the feature naming with libxl and the hypervisor.
> > 
> > Introduce a newly auto-generated array that contains the feature names indexed
> > at featureset bit position, otherwise using the existing INIT_FEATURE_NAMES
> > would require iterating over the array elements until a match with the expected
> > bit position is found.
> > 
> > Note that leaf names need to be kept, as the current auto-generated data
> > doesn't contain the leaf names.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Oleksii, now that Jan has provided a Reviewed-by, can you provide a
release-ack for this to go in?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 12:58:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 12:58:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719243.1121866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s53La-0005dm-Uc; Thu, 09 May 2024 12:58:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719243.1121866; Thu, 09 May 2024 12:58:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s53La-0005df-S8; Thu, 09 May 2024 12:58:06 +0000
Received: by outflank-mailman (input) for mailman id 719243;
 Thu, 09 May 2024 12:58:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0Rdn=MM=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s53LZ-0005dZ-Fa
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:58:05 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c6069408-0e03-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 14:58:04 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41ebcf01013so4839075e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 05:58:04 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f882085c5sm60337195e9.40.2024.05.09.05.58.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 05:58:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6069408-0e03-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715259484; x=1715864284; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=h5e99J8ljbVH9DUR5p4im2Mm3YPtAJnvq7zveHHzS+M=;
        b=ESE/NQcCW2xHx2iLvKrww0JVw1U9QJ3B1zBUj8scpwEaRd1JrHfLfkaZoZt9MtKCjI
         tHFdPUkgLDJNkBMQVokGjc9PRmmWD5zoUz6+McSiRi50pP6OOJv9u67WiF3fA4FAfB/C
         s4kYkuqAkc1fzdkGGfOoprrs+/0iExby/P1zM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715259484; x=1715864284;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=h5e99J8ljbVH9DUR5p4im2Mm3YPtAJnvq7zveHHzS+M=;
        b=FT2qfL2qB4bvcFZtJDhyuuKTOAvPWMnYEcmY3kVsIXoJSSwyqFsuO7tQ6s0AfGLktk
         3smsJxOKBTWgTLbQF7oveNM2I9V/qMHCw/sHcLf+Zb1uKuFx29aZyEn+Clr+wOM1szLY
         lllbWVd+/H/h8KpREvw+llFDFewj1MaPNCCWfimVgDkctocVB0Wgg+fkrwaSVxoYLewN
         AgHIYxF1wOacVLJ+8lqL1NfeG7uqxcXLG752g7N3i3GE8bbX4c5bAobEk2kjcLJkjdf6
         SxFhrUC64gKKquPu0C58RcFTow6pOcSSocAAZCkaH11xvgrWHgJKJCu+yAOSqBun21hR
         hwOQ==
X-Forwarded-Encrypted: i=1; AJvYcCUNaMV1ce/69NB1xVsjhCapTn0kqAPVUidmVkMiBsiwiM2ZB0dUjJoTX2olErt6EHEDdXuaWXPtVurm9UWYbTa9V1fIk58epyosEpM8ZTc=
X-Gm-Message-State: AOJu0YxXxzrFsYCSna41NGiVny0eUpjHjOivcTEAV99z4Ni8aAqlF6SY
	mtWXNzqpOD5iTNzFXQgsdA6Da6AdcfGktztsAd2mu1LeceIXdZVKMRjNDbO2h44=
X-Google-Smtp-Source: AGHT+IEl4HwV7vax8ufWt5nC2P6TE77OFQE//tV2ZP13qmuSURXdM61V8wgDrPDp+ColIL3d0wC6RQ==
X-Received: by 2002:a05:600c:a03:b0:41c:13f6:1ee1 with SMTP id 5b1f17b1804b1-41fbc9326bcmr23489705e9.4.1715259483672;
        Thu, 09 May 2024 05:58:03 -0700 (PDT)
Date: Thu, 9 May 2024 14:58:02 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Message-ID: <ZjzIWsoRmgmWkgGh@macbook>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>
 <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org>
 <ZjyzTCOSeNvPq2Np@macbook>
 <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org>

On Thu, May 09, 2024 at 01:12:00PM +0100, Julien Grall wrote:
> Hi,
> 
> On 09/05/2024 12:28, Roger Pau Monné wrote:
> > On Thu, May 09, 2024 at 10:50:56AM +0100, Julien Grall wrote:
> > > 
> > > 
> > > On 09/05/2024 09:13, Roger Pau Monné wrote:
> > > > On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
> > > > Also the interactions with the remote domain would need to be audited,
> > > > as the remote domain shattering the superpage would need to be
> > > > replicated in the mapping side in order to account for the changes.
> > > 
> > > ... I don't understand this one. How is this different from today's where a
> > > domain can foreign map a 2MB which may be using a superpage in the remote
> > > domain?
> > 
> > Hm, right, I was wrong with that I think, as long as proper references
> > as taken for the superpage entries it should be fine.
> > 
> > > >   Not sure all paths will be easy to
> > > > audit for preemption if it's more than relinquish_p2m_mapping() that
> > > > you need to adjust.
> > > 
> > > I thought about it yesterday. But I came to the conclusion that if we have
> > > any concern about removing 1GB foreign superpage then we would already have
> > > the problem today as a domain can map contiguously 1GB worth of foreign
> > > mapping using small pages.
> > 
> > Yeah, but in that case addition or removal is done in 4K chunks, and
> > hence we can preempt during the operation.
> 
> I am not entirely sure how that would work. From my understand, today, most
> of the users of the P2M code expects the operation to complete in one go and
> if preemption is needed then the caller is responsible to handle it by
> breaking up the happy.
> 
> With your suggestion, it sounds like you want to rework how the preemption
> today and push it to the P2M code. This would mean we would need to modify
> all the callers to check for -EERESTART (or similar) and also tell them how
> many pages were handled so the call can be restarted where it stopped. Is it
> what you had in mind?

TBH, I didn't have a specific location in mind about where to do the
split.

One solution that could simplify it is allowing foreign entries to
only be removed by specific functions, so that the split required in
order to do the removal can be handled by the caller knowing it's
dealing with a foreign map superpage.

But that would require the superpage to be shattered, and hence will
require creating lower levle leaf entries in order to do the
shattering and the removal in 4K chunks.

> I don't expect the work to be trivial, so I wonder if this is really worth
> it to try to change the way we preempt.
> 
> > 
> > OTOH for 1GB given the code here the page could be freed in one go,
> > without a chance of preempting the operation.
> > 
> > Maybe you have to shatter superpages into 4K entries and then remove
> > them individually, as to allow for preemption to be possible by
> > calling put_page() for each 4K chunk?
> This would require to allocate some pages from the P2M pool for the tables.
> As the pool may be exhausted, it could be problematic when relinquishing the
> resources.

Indeed, it's not ideal.

> It may be possible to find a way to have memory available by removing other
> mappings first. But it feels a bit hackish and I would rather prefer if we
> avoid allocating any memory when relinquishing.

Maybe it could be helpful to provide a function to put a superpage,
that internally calls free_domheap_pages() with the appropriate order
so that freeing a superpage only takes a single free_domheap_pages()
call.  That could reduce some of the contention around the heap_lock
and d->page_alloc_lock locks.

Note also that a foreign unmap resulting in a page free is also not
the common case, as that should only happen when the foreign domain
has been destroyed, or the page ballooned out, so to benchmark the
worst case some effort will be needed in order to model this
scenario.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 09 13:48:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 13:48:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719252.1121876 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s548Y-0004bF-Ig; Thu, 09 May 2024 13:48:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719252.1121876; Thu, 09 May 2024 13:48:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s548Y-0004b8-G0; Thu, 09 May 2024 13:48:42 +0000
Received: by outflank-mailman (input) for mailman id 719252;
 Thu, 09 May 2024 13:48:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s548X-0004ay-2n; Thu, 09 May 2024 13:48:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s548W-0002kA-RW; Thu, 09 May 2024 13:48:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s548W-00065g-7O; Thu, 09 May 2024 13:48:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s548W-0004KG-6q; Thu, 09 May 2024 13:48:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6uo7KfADfKetbZZmdaXp5wt3f4Eh9R5heypdgXiZNe0=; b=tAjVgNl+7aRUdufrnw+IfzhOp1
	TLJc4pFIc30rL8amtlg4lGtyukxn2pw2OX3ZLZtCccQkfzfw/gTDsWq1bqWi5w7/SJNFa7TxHM3dJ
	6APgVZI0KqjepprrXRXga2Z1k+OZW5SiC1aDKZBeVpMHg7nU0Lt++wYnTIgcwp8eATbw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185950-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185950: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-examine:examine-iommu:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
X-Osstest-Versions-That:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 13:48:40 +0000

flight 185950 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185950/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1   8 xen-boot         fail in 185942 pass in 185950
 test-armhf-armhf-examine     13 examine-iommu              fail pass in 185942
 test-armhf-armhf-xl-multivcpu  8 xen-boot                  fail pass in 185942

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 185942 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 185942 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185942
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185942
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185942
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185942
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185942
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185942
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b
baseline version:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b

Last test of basis   185950  2024-05-09 01:51:55 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu May 09 13:59:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 13:59:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719259.1121887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54J8-0006Nj-CT; Thu, 09 May 2024 13:59:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719259.1121887; Thu, 09 May 2024 13:59:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54J8-0006Nc-9l; Thu, 09 May 2024 13:59:38 +0000
Received: by outflank-mailman (input) for mailman id 719259;
 Thu, 09 May 2024 13:59:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XcCC=MM=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1s54J6-0006NW-OO
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 13:59:36 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5e1ac562-0e0c-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 15:59:35 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51f45104ef0so930027e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 06:59:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e1ac562-0e0c-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715263175; x=1715867975; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wRI7kDDz7pt7rS72ptF2zkQYN6Tx7uA4BFgiTe1GCsI=;
        b=PQnKJi2dgbDS1bzPzSJVzKySl6ghqaRaGFbuirzSYAtmbxx51d2BL3GdZ9QHKn8by3
         WxF/o0Gwycp3LVjUAT5GL+rL8fDSOMfUa2o0Kx4YU9TG/LZqSShYwXpwAVU5R/mtADWm
         SLwro4SeJC0WiCyiqkmZxMzjh3bH6xAYC9gKY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715263175; x=1715867975;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wRI7kDDz7pt7rS72ptF2zkQYN6Tx7uA4BFgiTe1GCsI=;
        b=ktvYoQZ+XZazTlco66hI8D8X6JLqtGErwFMmd7vR57yW9wuSmzflftK9qvpB3V+Nc3
         WzYeJ91lP+G61eW1LD64Hq3NAJrvikbhlJvQ1p4XJmHpUMrVMb4XZ29tfqgspEElG3Ne
         HHaiBiSvILSQ71p+McDJlficTS6VUIn7FFnFUf02HnRry22zaqs5gCz5ThPHK1eUW49y
         F6Rd6wX+48evYt+u2fWhyK4YobEuRy2z8PHKLmFe+yWzFrk/6lglFwirG4pKMNBIJJyI
         FbcCjWho5lWMkpMKvnBZfYJDhQd8SEF1GK/qPOK2z7UcY3+4adwpeA//nIkP51zFHSbX
         z0vQ==
X-Forwarded-Encrypted: i=1; AJvYcCU1RmasclNKIpCo5kIM5miAX7LPLIdP+tjJVyHnyg1Au1gZHEqiFrLy5SVhTEkfaojq3KlaHid4s8vmRtYZ1BwGWYs0PYky5hpQDpMU3tY=
X-Gm-Message-State: AOJu0Yxr+v/x+RI0e7RgDkf1QICVfSKI2WMEDEYykIcjRuv43uqPy+8O
	LJN++9lAcf54z0Iayo1xeF0bsre4qMJMXjy3dlQDjBISAdADk+iRJvr3G42PWdOq9UFsCJyAui4
	l7k2mhELQn9AiJNNfyOH1tm4rkprQSpTavgKCkIztoqRePnOV
X-Google-Smtp-Source: AGHT+IHv9Cc+M8D6PNEzgk/W+q+FXbIztSrIoNciqYfzONSRoNGpgChKXhQDUDq4nrHZ/DwVoy2/88seia7tl5z1hS4=
X-Received: by 2002:a19:9102:0:b0:51e:41a1:4d5 with SMTP id
 2adb3069b0e04-5217c277400mr3182992e87.9.1715263174717; Thu, 09 May 2024
 06:59:34 -0700 (PDT)
MIME-Version: 1.0
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-4-fouad.hilly@cloud.com> <cc1f42bf-7629-41fa-99b5-f77f15a371be@suse.com>
In-Reply-To: <cc1f42bf-7629-41fa-99b5-f77f15a371be@suse.com>
From: Fouad Hilly <fouad.hilly@cloud.com>
Date: Thu, 9 May 2024 14:59:20 +0100
Message-ID: <CAJKAvHbdcq2yf9Om22HY+CkKvBTrLiApHtEqxzhs5yUZKLAmDw@mail.gmail.com>
Subject: Re: [PATCH v3 3/5] x86: Add usage() to print out usage message
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 9:20=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 30.04.2024 14:47, Fouad Hilly wrote:
> > Refactor xen-ucode tool by adding usage() to handle usage\help messages=
.
> > As we add more command options this will keep help\usage messages in a =
common block.
> > Only generic error message is printed to stderr. usage and show_curr_cp=
u are printed to stdout.
> >
> > Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> > ---
> > [v3]
> > 1- Reverted usage message.
> > 2- Utilized usage().
> >
> > [v2]
> > 1- Improved message description.
> > 2- Fixed formatting and indentation.
> > 3- Error message to print to stderr.
>
> Already on v2 I pointed out that this change wasn't really done. And now,
> in v3, ...
>
In V4 all printing within ext_err path will be printed to stder

> >  tools/misc/xen-ucode.c | 12 +++++++++---
> >  1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
> > index c6ae6498d659..005bf85b6551 100644
> > --- a/tools/misc/xen-ucode.c
> > +++ b/tools/misc/xen-ucode.c
> > @@ -17,6 +17,13 @@ static xc_interface *xch;
> >  static const char intel_id[] =3D "GenuineIntel";
> >  static const char   amd_id[] =3D "AuthenticAMD";
> >
> > +static void usage(const char *name)
> > +{
> > +    printf("%s: Xen microcode updating tool\n"
> > +           "Usage: %s [<microcode file> | show-cpu-info]\n",
> > +           name, name);
> > +}
>
> ... you're still unconditionally printing to stdout, when ...
>
> >  static void show_curr_cpu(FILE *f)
> >  {
> >      int ret;
> > @@ -89,9 +96,8 @@ int main(int argc, char *argv[])
> >      if ( argc < 2 )
> >      {
> >          fprintf(stderr,
> > -                "xen-ucode: Xen microcode updating tool\n"
> > -                "Usage: %s [<microcode file> | show-cpu-info]\n", argv=
[0]);
> > -        show_curr_cpu(stderr);
> > +                "%s: unable to process command line arguments\n", argv=
[0]);
> > +        usage(argv[0]);
> >          exit(2);
> >      }
>
> ... originally the usage message (correctly) went to stderr in this case.
> It ought to go to stdout only in response to something like a --help
> command line option.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Thu May 09 14:00:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 14:00:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719260.1121897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54Jh-0007l1-KC; Thu, 09 May 2024 14:00:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719260.1121897; Thu, 09 May 2024 14:00:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54Jh-0007ku-HB; Thu, 09 May 2024 14:00:13 +0000
Received: by outflank-mailman (input) for mailman id 719260;
 Thu, 09 May 2024 14:00:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XcCC=MM=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1s54Jg-0007ka-6a
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 14:00:12 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 739a1ff7-0e0c-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 16:00:11 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2df848f9325so10328371fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 07:00:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 739a1ff7-0e0c-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715263211; x=1715868011; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Bd3FuNnGsiYvjztUaoUz5XSWfmfGI0oEndzpmDo13hE=;
        b=IwFs7r2e2bj/2YbzNzp2NUYcnrBp7KjYYHHzOzZi8QTk6K3O1mU7X0aSo8gCwJ2a9O
         vJRXInNuJ9xSxB2jFzBuksPRfxLobu/tgiM3vDsRJze6OMuX31jNmvPvie0yfaVS4Hw9
         uJ3N/CQI+5oo6pHQ6e9Gf3fwOA3fz9G44E1I8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715263211; x=1715868011;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Bd3FuNnGsiYvjztUaoUz5XSWfmfGI0oEndzpmDo13hE=;
        b=T5V6Tkyij5nuUsNKSetwpJEStoXiwaaq/DAcuBwZmz5TArFmoQQ+1TvLWPNsCNBTz9
         RAht8FPAuSWQzavxtoeH9cFAjpeN5B7IVEJydYXS0dFOU8D8gEId59yTt+vcxGuq8B/N
         EGSmFTj0EtibaJTAKPlYt4PqmognopaR6+ifvw3arwENcJsl/zujmV1C/E9o50Hn7BuP
         Zwi016E/nQGalusHOPY38eGIz4VEa5f1GVQ9fF01OKx1d+1lyvVAAvTmgH2vC9MSLQar
         ZS8MbpPfpHKquy3uRoAoSIRf1vVQxBBwYIbyWMoNptgTsFCbdKVX5SXqzNchzTMKYr4L
         v6SQ==
X-Forwarded-Encrypted: i=1; AJvYcCU7XQ0GiC5GSvWaM0F/gX4dcNGrEMYRZuvORaPmCwX3JX+V06RgAb4Ozk0N8Nr7hcCqA8KpmrXVrw8ODk/+85ZmIEn15fJva743+ab07Rg=
X-Gm-Message-State: AOJu0YxssCxqPyazo9Ag1JoF7mzHrq2SD7Wud3OTvfVd2271iA7K+IWp
	wvzXagtZndIEel4r4vXsnVJd1671qernsJ6MMyRLDt6C+TkveXLmInwzSe7ed9sT0KCVjqe0XHv
	G0EmTUVo7i2s4tcxGeQqrVYC0eus40hi9XRC2GJ+jPZbYkaTO
X-Google-Smtp-Source: AGHT+IFWqgqLCLWK2w7REBEM7Opbk4mCjoW+OJ2iFy3xZ/0i3PtOOGiLMHrhtQroC2KfzqjgBkRW2cic9+n220IiL8U=
X-Received: by 2002:a2e:97d7:0:b0:2e1:afa6:9291 with SMTP id
 38308e7fff4ca-2e446d7a622mr46734161fa.8.1715263210998; Thu, 09 May 2024
 07:00:10 -0700 (PDT)
MIME-Version: 1.0
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-5-fouad.hilly@cloud.com> <fc0c86f5-837a-45e8-807e-6861063be85f@suse.com>
In-Reply-To: <fc0c86f5-837a-45e8-807e-6861063be85f@suse.com>
From: Fouad Hilly <fouad.hilly@cloud.com>
Date: Thu, 9 May 2024 14:59:58 +0100
Message-ID: <CAJKAvHYcnmzEXptMOUTBny09jczbWnxk9QJEMhScYEi-z+Nhsw@mail.gmail.com>
Subject: Re: [PATCH v3 4/5] x86: Use getopt to handle command line args
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 9:21=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 30.04.2024 14:47, Fouad Hilly wrote:
> > Use getopt_long() to handle command line arguments.
> > Introduce ext_err for common exit with errors.
> >
> > Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
>
> Nit: Neither subject nor description make clear ...
>
> >  tools/misc/xen-ucode.c | 45 +++++++++++++++++++++++++++++++-----------
> >  1 file changed, 33 insertions(+), 12 deletions(-)
>
> ... what tool is actually being touched.

Noted and will be fixed in V4
>
> Jan


From xen-devel-bounces@lists.xenproject.org Thu May 09 14:00:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 14:00:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719261.1121906 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54Ji-00080E-RB; Thu, 09 May 2024 14:00:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719261.1121906; Thu, 09 May 2024 14:00:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54Ji-000807-OG; Thu, 09 May 2024 14:00:14 +0000
Received: by outflank-mailman (input) for mailman id 719261;
 Thu, 09 May 2024 14:00:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VSaB=MM=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s54Jg-0007kd-NR
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 14:00:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 728e4f6c-0e0c-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 16:00:09 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 320A14EE0737;
 Thu,  9 May 2024 16:00:09 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 728e4f6c-0e0c-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Thu, 09 May 2024 16:00:09 +0200
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, Stefano
 Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu
 <ppircalabu@bitdefender.com>
Subject: Re: [XEN PATCH v2] arm/mem_access: add conditional build of
 mem_access.c
Reply-To: alessandro.zucchelli@bugseng.com
Mail-Reply-To: alessandro.zucchelli@bugseng.com
In-Reply-To: <4c63493b-3743-4d20-a5e2-5f1b3038a9c8@xen.org>
References: <ad49a2006a6f19c2db1ff5eabb9ffd666693c4c5.1715250761.git.alessandro.zucchelli@bugseng.com>
 <4c63493b-3743-4d20-a5e2-5f1b3038a9c8@xen.org>
Message-ID: <10ae9ea84866cc9678ac056c829c30ea@bugseng.com>
X-Sender: alessandro.zucchelli@bugseng.com
Organization: BUGSENG Srl
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-09 12:52, Julien Grall wrote:
> Hi,
> 
> On 09/05/2024 11:39, Alessandro Zucchelli wrote:
>> In order to comply to MISRA C:2012 Rule 8.4 for ARM asm/mem_access.h 
>> in
>> the case where MEM_ACCESS=n stubs are needed to allow the conditional
>> compilation of the users of this header.
> 
> I think you need to update the commit message given ...
> 
>> 
>> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
>> ---
>> Changes from v1:
>> Reverted preprocessor conditional changes to xen/mem_access.h;
>> added conditional build for xen/mem_access.c;
>> provided stubs for asm/mem_access.h functions.
>> ---
>>   xen/arch/arm/Makefile                 | 2 +-
>>   xen/arch/arm/include/asm/mem_access.h | 9 +++++++++
>>   2 files changed, 10 insertions(+), 1 deletion(-)
>> 
>> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
>> index 7b1350e2ef..45dc29ea53 100644
>> --- a/xen/arch/arm/Makefile
>> +++ b/xen/arch/arm/Makefile
>> @@ -37,7 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
>>   obj-y += irq.o
>>   obj-y += kernel.init.o
>>   obj-$(CONFIG_LIVEPATCH) += livepatch.o
>> -obj-y += mem_access.o
>> +obj-$(CONFIG_MEM_ACCESS) += mem_access.o
> 
> ... this not only adding stub.
> 
>>   obj-y += mm.o
>>   obj-y += monitor.o
>>   obj-y += p2m.o
>> diff --git a/xen/arch/arm/include/asm/mem_access.h 
>> b/xen/arch/arm/include/asm/mem_access.h
>> index 35ed0ad154..2f73172e39 100644
>> --- a/xen/arch/arm/include/asm/mem_access.h
>> +++ b/xen/arch/arm/include/asm/mem_access.h
>> @@ -17,6 +17,7 @@
>>   #ifndef _ASM_ARM_MEM_ACCESS_H
>>   #define _ASM_ARM_MEM_ACCESS_H
>>   +#include <xen/types.h>
> 
> Can you explain why this is needed?

Without the inclusion of xen/types header NULL would be undefined.

> Style: Newline here please.
> 
>>   static inline
>>   bool p2m_mem_access_emulate_check(struct vcpu *v,
>>                                     const struct vm_event_st *rsp)
>> @@ -35,12 +36,20 @@ static inline bool 
>> p2m_mem_access_sanity_check(struct domain *d)
>>    * Send mem event based on the access. Boolean return value 
>> indicates if trap
>>    * needs to be injected into guest.
>>    */
>> +#ifdef CONFIG_MEM_ACCESS
>>   bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct 
>> npfec npfec);
>>     struct page_info*
>>   p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>>                                     const struct vcpu *v);
>> +#else
>>   +static inline bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, 
>> const struct npfec npfec) {return false;}
> 
> Coding style: The line is well over the 80 characters limit. Also we 
> don't tend to provide the implementation of the stub in the single line 
> if it is more than {}. So "{return false;}" should look like:
> 
> {
>    return false;
> }
> 
>> +
>> +static inline struct page_info*
>> +p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>> +                                  const struct vcpu *v) {return 
>> NULL;}
> 
> Ditto.
> 
>> +#endif /*CONFIG_MEM_ACCESS*/
>>   #endif /* _ASM_ARM_MEM_ACCESS_H */
>>     /*

Thanks for the feedback, I will soon provide the new version of the 
patch with the requested stylistic changes and a clearer description.
-- 
Alessandro Zucchelli, B.Sc.

Software Engineer, BUGSENG (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu May 09 14:15:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 14:15:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719271.1121916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54YX-0002Gd-8j; Thu, 09 May 2024 14:15:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719271.1121916; Thu, 09 May 2024 14:15:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54YX-0002GW-5l; Thu, 09 May 2024 14:15:33 +0000
Received: by outflank-mailman (input) for mailman id 719271;
 Thu, 09 May 2024 14:15:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XcCC=MM=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1s54YW-0002GQ-18
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 14:15:32 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 97d0bd2f-0e0e-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 16:15:31 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51fea3031c3so1235561e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 07:15:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97d0bd2f-0e0e-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715264130; x=1715868930; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VUygLG8Ses1pT2iZk/dTdvQs7OObxOc8EGqow86X2rY=;
        b=VP1yNcSK4LMF8Wf2fbhjQCTON7rnLSy2Vb4c2hEMCNcqVHF4vOLAUVliEblz8HiEW8
         3sUw88Fb2AOoPCc8Tw9i8ddXlLXFpjjUT5eFzaqpcYvNl0VaasT6tV/Q+GGGYImCDWYR
         NlMb14hIoVlv6tc7hp1WQ+Gn2rd+neSJSOZto=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715264130; x=1715868930;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VUygLG8Ses1pT2iZk/dTdvQs7OObxOc8EGqow86X2rY=;
        b=goeof3RmIL6kmWZ5MQcUzoASx3yaHTB6D66kOPqhFpOxKLAbM74z5MdHnSAS6NN9qK
         bLAYDRtStI5Yw5WCu5raR+ITQwe9luKLSKkADHiw+OBqxrRglOCg271McvgKXGl8TvMk
         Z6fRimWBSjUbJFtMmzoWPQBJu7U0O8pvUVpV+upe1/7AijDDpKzBwlYYIVWuxvTPeU8q
         CTWioMD0yOejHL8sy0Uqh5HQyZv665FZal71ESLNaiHyjq+vPyRktEfUOE0SbBbdtqcJ
         dNm/OkwoQoc2CcYJ7REBcdwizXjYQiKPiVR5vvyqcYjHK7XmiRoqlFyYgL2xQXFX0Szg
         gi2w==
X-Forwarded-Encrypted: i=1; AJvYcCVB38tEqIAvxuX36QEUjoiHwHDpjjBgiddsaJODK2zLQtDtP5Wdhyn9DZUXZtqq09mW3vO0fo59skjRsK48vQKDAigRAKsgU/bvmY8GqGI=
X-Gm-Message-State: AOJu0Yz3AgnOZdvA/UWF74MswgTH0ICbShmqZRNhQU7p0rTA/hWHrTwm
	0I7RhHASOe1sowU3zyw4cNSE0PH/crdrIuL0c9TbiHWyIdq7KiZW575dqgHu0Uq2Rp3G7Y8pEX5
	ok5gFYdS4CwGqW7vHJRTdxNEWYrDmBPShl1/U/Q==
X-Google-Smtp-Source: AGHT+IG6frklNvQith32cz7dXyVi0Wv1QJMUyFzHocct1cqIqBr1sJtZGXIwaQg+/Mq5eQejckewhWuldyAt1Fss0BM=
X-Received: by 2002:a19:5f03:0:b0:521:e065:c6c6 with SMTP id
 2adb3069b0e04-521e065c97dmr2446474e87.53.1715264130570; Thu, 09 May 2024
 07:15:30 -0700 (PDT)
MIME-Version: 1.0
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-3-fouad.hilly@cloud.com> <ec48e32a-30e5-45ab-8f11-7d3d6ce6122e@suse.com>
In-Reply-To: <ec48e32a-30e5-45ab-8f11-7d3d6ce6122e@suse.com>
From: Fouad Hilly <fouad.hilly@cloud.com>
Date: Thu, 9 May 2024 15:15:17 +0100
Message-ID: <CAJKAvHajhOy2SpYZk1G6MtiK1weL6cuD+bp8puY=o9XGkt0TAw@mail.gmail.com>
Subject: Re: [PATCH v3 2/5] x86: Refactor microcode_update() hypercall with flags
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 10:14=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 30.04.2024 14:47, Fouad Hilly wrote:
> > @@ -633,12 +637,12 @@ static long cf_check microcode_update_helper(void=
 *data)
> >                                    microcode_cache);
> >
> >          if ( result !=3D NEW_UCODE &&
> > -             !(opt_ucode_allow_same && result =3D=3D SAME_UCODE) )
> > +             !((opt_ucode_allow_same || ucode_force_flag) && result =
=3D=3D SAME_UCODE) )
>
> Why would "force" not also allow a downgrade?

It should be allowed. Will be fixed in v4

>
> > @@ -708,11 +712,15 @@ static long cf_check microcode_update_helper(void=
 *data)
> >      return ret;
> >  }
> >
> > -int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long l=
en)
> > +int microcode_update(XEN_GUEST_HANDLE(const_void) buf,
> > +                     unsigned long len, unsigned int flags)
> >  {
> >      int ret;
> >      struct ucode_buf *buffer;
> >
> > +    if ( flags > 1 )
>
> How is one to connect this literal 1 with what is really meant here? Also
> would be nice if this check fit with other similar checks we do, i.e.
>
>     if ( flags & ~XENPF_UCODE_FLAG_FORCE_SET )
Will be done in v4

>
> > +        return -EINVAL;
> > +
> >      if ( len !=3D (uint32_t)len )
> >          return -E2BIG;
>
> As an aside: Isn't this dead code, with the respective hypercall interfac=
e
> struct fields (now) both being uint32_t?

Will be cleaned up in v4.
>
> > --- a/xen/arch/x86/platform_hypercall.c
> > +++ b/xen/arch/x86/platform_hypercall.c
> > @@ -311,7 +311,17 @@ ret_t do_platform_op(
> >
> >          guest_from_compat_handle(data, op->u.microcode.data);
> >
> > -        ret =3D microcode_update(data, op->u.microcode.length);
> > +        ret =3D microcode_update(data, op->u.microcode.length, 0);
> > +        break;
> > +    }
> > +
> > +    case XENPF_microcode_update2:
> > +    {
> > +        XEN_GUEST_HANDLE(const_void) data;
> > +
> > +        guest_from_compat_handle(data, op->u.microcode2.data);
> > +
> > +        ret =3D microcode_update(data, op->u.microcode2.length, op->u.=
microcode2.flags);
>
> Nit (style): Overlong line.
>
> > --- a/xen/include/public/platform.h
> > +++ b/xen/include/public/platform.h
> > @@ -624,6 +624,19 @@ struct xenpf_ucode_revision {
> >  typedef struct xenpf_ucode_revision xenpf_ucode_revision_t;
> >  DEFINE_XEN_GUEST_HANDLE(xenpf_ucode_revision_t);
> >
> > +/* Hypercall to microcode_update with flags */
> > +#define XENPF_microcode_update2    66
> > +struct xenpf_microcode_update2 {
> > +    /* IN variables. */
> > +    uint32_t flags;                   /* Flags to be passed with ucode=
. */
> > +/* Force to skip microcode version check when set */
> > +#define XENPF_UCODE_FLAG_FORCE_SET     1
>
> Nit: What is "SET" in the identifier intended to mean?
"SET" to be removed in v4.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Thu May 09 14:31:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 14:31:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719277.1121927 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54nq-00054I-Hl; Thu, 09 May 2024 14:31:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719277.1121927; Thu, 09 May 2024 14:31:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54nq-00054B-Dv; Thu, 09 May 2024 14:31:22 +0000
Received: by outflank-mailman (input) for mailman id 719277;
 Thu, 09 May 2024 14:31:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XcCC=MM=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1s54np-000545-2n
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 14:31:21 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd68208b-0e10-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 16:31:20 +0200 (CEST)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso18281331fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 07:31:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd68208b-0e10-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715265079; x=1715869879; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zfecLCj4ppaiJFUOJSfuq5jz2MPsvW1TMhLtuPxJ2Ew=;
        b=fuAnQ2fhkZBxr20XPig6OJMzwus8r137XHUKZgTUgrONMg03reI1lgUopHE9KmzMxX
         hfikVBTBR1C9b3GBk0fS1suxNzB7y0VXjkM5Np7gXjMorGwmP8UZ2wfWMjcj59+6GxKg
         sDZqqV+7nNyCp4ViRm+KyTlAL6jf97nlOPjXU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715265079; x=1715869879;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zfecLCj4ppaiJFUOJSfuq5jz2MPsvW1TMhLtuPxJ2Ew=;
        b=Y7vWanZm01tH5n7uct5Sckldb768BEi3A51cfcnxYgM/XjxcHJgIPo6NW5WrkvfJ01
         SsDNjREtWlxM7CMlk3Fj0ua2Z19PzCudD+tuQQVoo+YsPBhk4buv15YnKKSU3Ni/JaXi
         GFqhSCyUEyR97bA7H9p8HcV+1foptEShz8gyGB9GhRXW4YyZL3yt38qPS2YpG1jfoa5n
         PMMw8W6vCwAAr4yLsMRoCazZ7iLEbwEZCWtVhDw8y7QZ61jzAo8LGxHH0hHFgxNbihuX
         /g7fLbVkjim2IEt8kRBWZc80hv4GV5uEXyIPj2caXS8Agy0dUrGISMKZkA7c1SYBa4Hu
         aoSw==
X-Forwarded-Encrypted: i=1; AJvYcCVJoQE+PTEM53S/gK/F4OcDYiRFc/6F+BeBLVj/KgcdqsnFLp7WPjLSQO6jON/iZPe6ANmHwDsLHjOB9I+DhtbGjWSLvX8XaKIF+sv6lf8=
X-Gm-Message-State: AOJu0YyeS40o/uJAeAN6xAni0tj5F4NL7bYcSfEfCm7cS2Wzc/YwVY0N
	mBQpoc7cWC1R7DLGwBQ7akxSzKPnpTfX+8H1hrYkO+xXRJ7TxxmJQ5D2XHS410Jkdtdgy/6taqX
	wKG9pE7UhIy5FwrQXv2fw4opk04Mo6xNtIFGEZQ==
X-Google-Smtp-Source: AGHT+IHw8USn1pwtfDYRiGrAV9UlikvZVFNUltLg1Z0kyZTfGor8u+pHXtigfdaRATjwOlfrdJZO+AA3REyDvpOGgNk=
X-Received: by 2002:a05:6512:488d:b0:51f:4165:930b with SMTP id
 2adb3069b0e04-5217cd4963emr4404267e87.49.1715265079540; Thu, 09 May 2024
 07:31:19 -0700 (PDT)
MIME-Version: 1.0
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-6-fouad.hilly@cloud.com> <8b433feb-467c-4569-9426-cc245fc9da50@suse.com>
In-Reply-To: <8b433feb-467c-4569-9426-cc245fc9da50@suse.com>
From: Fouad Hilly <fouad.hilly@cloud.com>
Date: Thu, 9 May 2024 15:31:06 +0100
Message-ID: <CAJKAvHZfOVnNR4uv5d8X53o=m3YPCQSPSU0UaYhnq6dKPgbDuQ@mail.gmail.com>
Subject: Re: [PATCH v3 5/5] Add --force option to xen-ucode to override
 microcode version check
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 10:39=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 30.04.2024 14:47, Fouad Hilly wrote:
> > @@ -21,10 +23,11 @@ static const char   amd_id[] =3D "AuthenticAMD";
> >  static void usage(const char *name)
> >  {
> >      printf("%s: Xen microcode updating tool\n"
> > -           "Usage: %s [<microcode file> | Options]\n"
> > +           "Usage: %s [microcode file] [options]\n"
> >             "options:\n"
> >             "  -h, --help            display this help and exit\n"
> > -           "  -s, --show-cpu-info   show CPU information and exit\n",
> > +           "  -s, --show-cpu-info   show CPU information and exit\n"
> > +           "  -f, --force           force to skip micorocde version ch=
eck\n",
>
> Besides (nit) the typo, isn't "version check" too broad here? Elsewhere i=
n the
> series you mention Intel's "min_rev" addition. _That_ kind of version che=
ck
> probably shouldn't be permitted to skip? In which case maybe better "cert=
ain
> version checking" or some such?
I will remove "min_rev" mention from this patch series. The main
reason for this change is to allow developers to downgrade microcode.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Thu May 09 14:33:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 14:33:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719279.1121937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54q7-0005bo-Tt; Thu, 09 May 2024 14:33:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719279.1121937; Thu, 09 May 2024 14:33:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s54q7-0005bh-Po; Thu, 09 May 2024 14:33:43 +0000
Received: by outflank-mailman (input) for mailman id 719279;
 Thu, 09 May 2024 14:33:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XcCC=MM=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1s54q6-0005bZ-6m
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 14:33:42 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 20d8d714-0e11-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 16:33:40 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e0a0cc5e83so13348111fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 07:33:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20d8d714-0e11-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715265219; x=1715870019; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jrUqNBLetUqfVaCu7pawLWrJ5P482gx0aXgdZdZotRw=;
        b=iAnF1gxlVBZiWQUif7XoTbzIY1+bEGi9eL+3VE5zuaHjMsQUafb3Y3lXDp4UBwBo+O
         l0fvw41DWOBjzritWEm4TYrfZOKlV2il1kF4722yO9qV9vWMjYl9McQKSElZmPADWlTO
         7wSmvqfPNakZ8WXsVaVOa45FZbmU2Zo8cKAtA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715265219; x=1715870019;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jrUqNBLetUqfVaCu7pawLWrJ5P482gx0aXgdZdZotRw=;
        b=Tv5KAoenP4VmyiXnyA5BrfAvvIPvuUTBrt3OzQWDyvhlDe1njALg94JtYeKh8kZIn1
         k0anmB/WYYBK610MJPDs9oKxgoaddGzd11+RXmiZeWpc+XzfxDnruBCNeAFSVM8AaqOb
         hP4LCWwkV32aQWA3V6V4KvKxAXmPTik8Fueaz/6X8Os2gUQ9FzysJZkV50xuT9JPr1p4
         qzmYolsVNneRKYKQoixdG+UCZlLJu6DkF9WeOCuiumMAZpgLTPU7ckq6AlyDNqiPz28P
         PRBFY0GaY7NI2vgBJil+sYVE/pGD71mpIKyAgKhZuCFxmWF63cenk6r6SErE25+g4php
         uNgA==
X-Forwarded-Encrypted: i=1; AJvYcCVBvWv5iSKE9AUQHf3H3b7xahAq3lJu4+RaO3u7Z4odnI9lXlsOm1P9V88VnIK7eX6w8elr4jvRh7QqsrkTfrYGFfGZ2ZmM0qjeX/s7glQ=
X-Gm-Message-State: AOJu0Yxebf/MVeGputBh6xsZXXg6tY8m49J6ycAZwNMJKsRErE5qRyyV
	3FWYdsWZMVlJHrkbZFuMEGIdsnfZTucZBbHYJT06SPhBQCp7Plu6NAYTk6T97pEdPijgUlJMP90
	sL7zoesfo7SVw+fNYibQGydxxO4thCIwWVRBybrK0sDB+kMq3
X-Google-Smtp-Source: AGHT+IHsmFbMpBJfx/+eeZpK5D5Jqck6dZ/kKj+SADUNgv1bRMcf45pXvsZF5mZpsUaRYfixpdXGEVcerFZytiEaKiY=
X-Received: by 2002:a05:651c:150:b0:2e1:a767:a80a with SMTP id
 38308e7fff4ca-2e447baa79fmr39335821fa.38.1715265219623; Thu, 09 May 2024
 07:33:39 -0700 (PDT)
MIME-Version: 1.0
References: <20240430124709.865183-1-fouad.hilly@cloud.com>
 <20240430124709.865183-2-fouad.hilly@cloud.com> <1f16e73f-a5a9-4816-8054-81ad0c186030@suse.com>
In-Reply-To: <1f16e73f-a5a9-4816-8054-81ad0c186030@suse.com>
From: Fouad Hilly <fouad.hilly@cloud.com>
Date: Thu, 9 May 2024 15:33:26 +0100
Message-ID: <CAJKAvHZBmxWCW_rUR9FEY2CkJL8CMw6ByKASukaxE2ubtFY4-g@mail.gmail.com>
Subject: Re: [PATCH v3 1/5] x86: Update x86 low level version check of microcode
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 9:45=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 30.04.2024 14:47, Fouad Hilly wrote:
> > Update microcode version check at Intel and AMD Level by:
> > Preventing the low level code from sending errors if the microcode
> > version is not a newer version. this is required to allow developers to=
 do
> > ucode loading testing, including the introduction of Intel "min rev" fi=
eld,
> > which requires an override mechanism for newer version checks.
>
> Won't "min rev" checking, for being Intel-only, require quite the opposit=
e,
> i.e. leaving more of the checking to vendor specific code?

The checking of the microcode signature and if it is applicable to the
CPU, will be at vendor code level i.e Intel\AMD.
"min_rev" mention to be removed. This change is for microcode
downgrade capability.

>
> > Even though
> > the check for newer is removed at this level, it still exists at higher
> > common level, where by default only newer version will be processed.
> > The option to override version check, will be added as part of this pat=
ch
> > series.
>
> Please avoid wording like "this patch", "this commit", and all the more
> "this patch series". Especially this last one will become completely
> meaningless once part of a commit message in the tree.
>

Noted and will be fixed in V4

> > --- a/xen/arch/x86/cpu/microcode/amd.c
> > +++ b/xen/arch/x86/cpu/microcode/amd.c
> > @@ -384,11 +384,10 @@ static struct microcode_patch *cf_check cpu_reque=
st_microcode(
> >              }
> >
> >              /*
> > -             * If the new ucode covers current CPU, compare ucodes and=
 store the
> > -             * one with higher revision.
> > +             * If the microcode covers current CPU, then store its
> > +             * revision.
> >               */
>
> Nit: This can become a single line comment now, can't it? (Again then in =
Intel
> code.)
>
> > --- a/xen/arch/x86/cpu/microcode/intel.c
> > +++ b/xen/arch/x86/cpu/microcode/intel.c
> > @@ -294,8 +294,7 @@ static int cf_check apply_microcode(const struct mi=
crocode_patch *patch)
> >
> >      result =3D microcode_update_match(patch);
> >
> > -    if ( result !=3D NEW_UCODE &&
> > -         !(opt_ucode_allow_same && result =3D=3D SAME_UCODE) )
> > +    if ( result =3D=3D MIS_UCODE )
> >          return -EINVAL;
>
> I continue to be in trouble with this change, despite the v3 adjustment
> you make: If this is needed here, why would a similar change not be neede=
d
> for AMD?

Will be fixed in V4

Fouad


>
> Plus the original question remains: Is this actually valid to be changed
> right here? IOW despite the somewhat improved patch description I'm still
> having difficulty identifying which vendor-independent check this is
> redundant with. As opposed to the AMD change further up and ...
>
> > @@ -355,11 +354,10 @@ static struct microcode_patch *cf_check cpu_reque=
st_microcode(
> >              break;
> >
> >          /*
> > -         * If the new update covers current CPU, compare updates and s=
tore the
> > -         * one with higher revision.
> > +         * If the microcode covers current CPU, then store its
> > +         * revision.
> >           */
> > -        if ( (microcode_update_match(mc) !=3D MIS_UCODE) &&
> > -             (!saved || compare_revisions(saved->rev, mc->rev) =3D=3D =
NEW_UCODE) )
> > +        if ( (microcode_update_match(mc) !=3D MIS_UCODE) && !saved )
> >              saved =3D mc;
>
> ... this one, where I can see that they are about caching of ucode blobs,
> which looks to be dealt with by cache maintenance logic in
> microcode_update_helper() and microcode_update_cache().
>
> Jan


From xen-devel-bounces@lists.xenproject.org Thu May 09 15:13:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 15:13:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719285.1121946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55Sh-0002uk-Jn; Thu, 09 May 2024 15:13:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719285.1121946; Thu, 09 May 2024 15:13:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55Sh-0002ud-Gj; Thu, 09 May 2024 15:13:35 +0000
Received: by outflank-mailman (input) for mailman id 719285;
 Thu, 09 May 2024 15:13:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s55Sf-0002tc-MZ; Thu, 09 May 2024 15:13:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s55Sf-0004Dw-Is; Thu, 09 May 2024 15:13:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s55Sf-0007yf-6p; Thu, 09 May 2024 15:13:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s55Sf-0001MR-6J; Thu, 09 May 2024 15:13:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=S2kKGMgc1RfK4BWbf6Q1Xamm3ks7eZ0I+/diHy+NRBk=; b=IZR1FE9kjrt9L3W9utRi9ccd6l
	wWKG8LuDVDuBDtK6yjyIg+xzQsz++6UqoD9dcUG1rN5clKpYZFh0bXkvoBRpwwz9WnU2Pp2fLmFic
	4sWe4u4A6mNUB/+x//T6CLYZ+XrgcIPOhQMqdfxpxFpiRC7Gp353k482QHWAOS9WIgks=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185951-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185951: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-arm64-pvops:kernel-build:fail:regression
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=a47e73d6e76c85b0897e3cccaf0a58ed3c367664
X-Osstest-Versions-That:
    libvirt=df9ffb025646d649f981ca25600225bc8cf794d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 15:13:33 +0000

flight 185951 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185951/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvops             6 kernel-build             fail REGR. vs. 185943

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185943
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              a47e73d6e76c85b0897e3cccaf0a58ed3c367664
baseline version:
 libvirt              df9ffb025646d649f981ca25600225bc8cf794d1

Last test of basis   185943  2024-05-08 04:22:17 Z    1 days
Testing same since   185951  2024-05-09 04:20:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            fail    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit a47e73d6e76c85b0897e3cccaf0a58ed3c367664
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 14:35:39 2024 +0100

    src/node_device: don't overwrite error messages
    
    The nodedev code unhelpfully reports
    
      couldn't convert node device def to mdevctl JSON
    
    which hides the actual error message
    
      No JSON parser implementation is available
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 479333af2c9d234ec1639a0a976e27afb1e26604
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 14:35:04 2024 +0100

    tests: don't run mdevctl test if lacking YAJL
    
    The mdev code requires YAJL in order to convert from node dev XML to
    mdev's config format.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 7817c3f89cc9f887bea269ebd3f34641cd0a052a
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 14:26:55 2024 +0100

    test: drop bogus check for YAJL from libxl test/mock
    
    The libxlmock.c conditionalizes on WITH_YAJL, but this mock is
    used from other tests which only conditionalize on WITH_LIBXL.
    The libxl code does not have any dependancy on YAJL, so the
    bogus condition can be removed from the mock and also from
    libxlxml2domconfigtest.c
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 971305e86f26269d3211ff41f4931e66c6f0b716
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 14:17:36 2024 +0100

    tests: always build securityselinuxhelper if libselinux is present
    
    The securityselinuxhelper build is conditionalized on the SELinux
    security driver feature. It is also needed, however, by viridentitytest
    whenever libselinux is present.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit c8c5170b74ae9af6f2ade66be310e58c90170f0f
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 14:05:12 2024 +0100

    test: conditionalize 'virsh-auth' on test driver
    
    The 'virsh-auth' test is mistakenly conditionalized on the libvirtd
    daemon build, however, it just uses the 'test:///default' driver
    URI, so does not require a daemon.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 08bfb18736a0dc50068090e1a7a024b7a920f789
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 13:23:39 2024 +0100

    tests: build driver modules before virdrivermoduletest
    
    The virdrivermoduletest will attempt to dlopen() each driver module,
    so they must be build before the test can run.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit a76ffee3657a449b7be06b9e89b33bd2246cc8e8
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 13:16:26 2024 +0100

    tests: build 'virsh' before running virsh-auth test
    
    The 'virsh-auth' test needs to be able to invoke the 'virsh' binary
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 0dc278dd020680d1a490d7d88ccf2c7be82b5c05
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 13:10:59 2024 +0100

    src: ensure augeas test file is generated before running test
    
    We fail to express an ordering between the custom target that
    generates the combined augeas test input file, and the meson
    test command.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit bdfe46ed6e7c11790f7234be46a4136e06e482ff
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 11:38:07 2024 +0100

    meson: record which other options are a pre-requisite
    
    Several meson options cannot be enabled, without first enabling another
    option. This adds a small comment prior to an option to record its
    mandatory dependencies.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 67ac2cd5908a686ab8f83703d5167caf5310a9f3
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 10:40:15 2024 +0100

    tests: fix tests when test driver is disabled
    
    Various tests try to open a connection to 'test:///default' and
    must be skipped when the test driver is disabled.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit 20acd5b1beddaa77c9d0da1917940eeceb98e470
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Tue May 7 10:38:59 2024 +0100

    scripts/rpcgen: skip tests if tirpc is not present
    
    This skips building tests which rely on tirpc when it is not
    present.
    
    Reviewed-by: Peter Krempa <pkrempa@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit ae6a89884aeea9f6e334b8b6951d1aa6c362bdad
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Wed May 8 13:03:20 2024 +0100

    tests: fix two off-by-1 errors in read-big-pipe test
    
    When testPipeFeeder copies the XML document into the padded buffer, it
    tells virStrcpy that 'xmlsize' bytes are available. This is under
    reporting size by 1 byte, and as a result it fails to copy the trailing
    '\n' replacing it with '\0'. The return value of virStrcpy wasn't
    checked, but was reporting this truncation.
    
    When testPipeFeeder then sends the padded buffer down the pipe, it asks
    to send 'emptyspace + xmlsize + 1' bytes, which means it sends the data,
    as well as the trailing '\0' terminator.
    
    Both bugs combined mean it is sending '\0\0' as the last bytes, instead
    of '\n' which was intended. When virFileReadAll reads data from the
    pipe, it ends up adding another '\0' resulting in a very NUL terminated
    string ('\0\0\0'). This is all harmless, but should be fixed regardless.
    
    Reviewed-by: Andrea Bolognani <abologna@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

commit e1c32120ce6eddb72272b4717dd8384075b32c2f
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Wed May 8 11:50:09 2024 +0100

    tests: fix hang in virshtest 'read-big-pipe' case
    
    The virshtest program testPipeFeeder method is doing this:
    
      mkfifo("test.fifo", 0600) ;
    
      int fd = open("test.fifo", O_RDWR);
    
      char buf[...];
      memset(buf, 'a', sizeof(buf));
      write(fd, buf, sizeof(buf)) == sizeof(buf));
      close(fd);
    
    while the the 'virsh' child process then ends up doing:
    
      fd = open("test.fifo", O_RDONLY);
      read(fd, buf, sizeof(buf)) == sizeof(buf));
      close(fd);
    
    The 'virsh' code hangs on open() on at least ppc64 and some other
    arches. It can be provoked to hang even on x86 by reducing the size of
    the buffer. It can be prevented from hanging on ppc64 by increasing the
    size of the buffer.
    
    What is happening is a result of differing page sizes, altering the
    overall pipe capacity size, since pipes on linux default to 16 pages
    in size and thus have architecture specific capacity when measured
    in bytes.
    
     * On x86, testPipeFeeder opens R+W, tries to write 140kb and
       write() blocks because the pipe is full. This gives time for
       virsh to start up, and it can open the pipe for O_RDONLY
       since testPipeFeeder still has it open for write. Everything
       works as intended.
    
     * On ppc64,  testPipeFeeder opens R+W, tries to write 140kb
       and write() succeeds because the larger 64kb page size
       resulted in greater buffer capacity for the pipe. It thus
       quickly closes the pipe, removing the writer, and triggering
       discard of all the unread data. Now virsh starts up, tries
       to open the pipe for O_RDONLY and blocks waiting for a new
       writer to open it, which will never happen. Meson kills
       the test after 30 seconds.
    
       NB, every now & then, it will not block because virsh starts
       up quickly enough that testPipeFeeder has not yet closed the
       write end of the pipe, giving the illusion of correctness.
    
    The key flaw here is that it should not have been using O_RDWR
    in testPipeFeeder. Synchronization is required such that both
    virsh and testPipeFeeder have their respective ends of the pipe
    open before any data is sent. This is trivially arranged by
    using O_WRONLY in testPipeFeeder.
    
    Reviewed-by: Andrea Bolognani <abologna@redhat.com>
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>


From xen-devel-bounces@lists.xenproject.org Thu May 09 15:32:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 15:32:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719296.1121960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55kb-0005x5-8v; Thu, 09 May 2024 15:32:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719296.1121960; Thu, 09 May 2024 15:32:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55kb-0005wy-4q; Thu, 09 May 2024 15:32:05 +0000
Received: by outflank-mailman (input) for mailman id 719296;
 Thu, 09 May 2024 15:32:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fDZs=MM=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s55kZ-0005ws-2h
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 15:32:03 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060b.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 465fe7db-0e19-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 17:32:00 +0200 (CEST)
Received: from MN2PR18CA0017.namprd18.prod.outlook.com (2603:10b6:208:23c::22)
 by CYXPR12MB9317.namprd12.prod.outlook.com (2603:10b6:930:e2::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Thu, 9 May
 2024 15:31:55 +0000
Received: from MN1PEPF0000ECD5.namprd02.prod.outlook.com
 (2603:10b6:208:23c:cafe::94) by MN2PR18CA0017.outlook.office365.com
 (2603:10b6:208:23c::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42 via Frontend
 Transport; Thu, 9 May 2024 15:31:55 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MN1PEPF0000ECD5.mail.protection.outlook.com (10.167.242.133) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Thu, 9 May 2024 15:31:55 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 9 May
 2024 10:31:54 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 9 May 2024 10:31:52 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 465fe7db-0e19-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kqSlTqQJhWSTeBw7QJsoO6jJH8hKkMiRYkwT0Xg2ExS6iqGYNvMq1BD2JPMsC1t2vE/c4QFTRDbjPnZirTeC/po+h4mdjO1QCy2etNJmCQH/Bq23wcdkQ0UNKpigmQrx/9lWk7xETOcfJLc4b1bdQGdzmIftRg6RLThS6ANDuV+Y8gjXiDUbFBLYfFPjF327u5nttn28oCoht/uoBnyPhth0XOLBMN10XjorLMn0D+2SOi7kDavCOuXDDFQzVDcnWWginjpUIhFuI7hJN0XVuKTZ5vu95JQgvbQ6IYaSckeUa1O0q8lxyVA+zshivTMIxL5p5MZl3bIZTs/lM3lRMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=YcU+yZxf3EXhz/qPnftjxilBRj0xGQiRpjMhQXqU3TI=;
 b=HDrA7w7HWysUdHBlN+z2c4Sbf4a7wGn/5AG4OyQN6i17dD0rlzDPj/uZuC6GTYwriIfuO40IJZ7Wa+R04ayeT4qX3RQJA5A2UBaMQOCvQxHRIPO9Iv0UDd+GrSpKXjw/BgheWFy8DdRj+JvhbyYPe+8QFWqg7jALQcyvJs9lCWF7HGnbBgpClPxSn6MQ3bmGB2K4ORazrzVKnN8c+wwb2zpU/zE/vtVfcYIKJ6DYcj3rHezw49eC0qEfBilNfsMI0+6dm+dUODrAwdCdCSHh5IW/UJkfn9BAkN4sv5b1nXRZfk8/k8hRFmZytpeepqIQRZLq32tFYOj9k1bpnUU9IQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YcU+yZxf3EXhz/qPnftjxilBRj0xGQiRpjMhQXqU3TI=;
 b=1J79elJrvgl6DqgxtaEi431ii8FmkX34ohm+9iREOyivnTDUjd4WLnBcn/Hy1N0HCTG7ZRzMg4aP/BMa0Rr4iuAeuWhlwwYh+n2BxbS+gV0WB5iZ3Wix2JA1Y03iuHPKrWfgKrmmWbf4t9cSV3mV0LJLgqRKzjP9L1DH+vOx5H8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
Date: Thu, 9 May 2024 23:31:51 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
 <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD5:EE_|CYXPR12MB9317:EE_
X-MS-Office365-Filtering-Correlation-Id: 2bbe64f3-168a-48eb-42b1-08dc703d2863
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?aXI1TGZrSC85VklyOU9tbVc2cGZxaEo5TkMzMk5ybVh4SmVPVEJjYUVuYmIy?=
 =?utf-8?B?cmRHUnplaG1RcXpMbXNHcEs3c1o2Qi9KZjhCZ2xSNEF6VW1mRmhmT0ZGUUt3?=
 =?utf-8?B?Smt1ZUlacXJ2Y2ttQVA2ZTJGclE3Y1Zsd01ZMHNFUzAzSUg2VnJpWkhIV3Vy?=
 =?utf-8?B?QTJIY0gvaTliTTNJdGZaUHh3Q2RucGJTQVhVUnpMZVkzZ2VaK2FmQkpnL0tR?=
 =?utf-8?B?ZStQVlAzMHIvT3lHbjRTOExFOW5RWjJUNEQ4VnhQMWhWSEVnRC9qUDFWTTBT?=
 =?utf-8?B?RlU5WnIvVnAwUU5ITjZrbVpkUW55T2dLamttdUhET3UxWlE4czJ6WkRpOGFr?=
 =?utf-8?B?dmVTVHJ0R04zMWhHczBrRHlzVGM2Q1duVFI4VktQbFRnaVV5T0NCU2VCSmlG?=
 =?utf-8?B?MzFuNDhwK0JuTG9qS2owbVBuWXB1a2FrVHJTK3duRDNDaVQ5OVFPL0laWWpH?=
 =?utf-8?B?ZTc3dkN1ZEpwaGRKeXpYYm5BSkh4SHJqalNNZm1VMWxtaEZHSUdFUHpoQTBj?=
 =?utf-8?B?U1NaMVhER1M5VEZMQmF2Z0M5OVhBRTJaVXJuVEJOdis2SURtcU1yd2hoZFJi?=
 =?utf-8?B?YndpTzBoanZZZFBHV3ZsaDFsemdvTmdRNjQzSXplaHNpdmFMT0tlbCtCRVQx?=
 =?utf-8?B?SUgzYnIybDMwTUJSY1hUaUNwQXQxMG9CZG9RYnpxQWFDbzRadDNiWUdlL2V2?=
 =?utf-8?B?NmhaRk9ENXROTkhqWjZiYnVLVVV6TldZUGpWbnkzWjUxWHpNakxscFBYWU8v?=
 =?utf-8?B?Y041Rjg3NUR1ZktCZ3ppTU1pYmd4MVE0ZmduVC9CRmtJNEs1djQ1ZEZGVVh3?=
 =?utf-8?B?NG5KSWJ0emJrQXk2dTA0ZGFoRjBmVEVEYlFwa1hvMWVWWE5BNUpLRDBHWG0v?=
 =?utf-8?B?VVFEZGNnRnA1cmNoMkc5dkppK09RYnZxTS9qSUhqQ05oZGV6TUNOZG1vZWZP?=
 =?utf-8?B?c2tBNkNqQWRycmpDYmEwdFRBaVNVNGpyNVVQTE52SXNaNlNFWlBIRitKeTd1?=
 =?utf-8?B?STM3VEJ3Q2poUlpZaS9DV1drd3MvRWNSQ3N0UlBDSDZoVjVtNys1NDJuYnFO?=
 =?utf-8?B?RWVWbWFDdlNVUVJZZ1ovUU5wcXhjUm9jdm1IY1h0Z1ZyMEw5LzRZYVlRUGRv?=
 =?utf-8?B?WitVeDU4QUEvQ1M4d0tHRGNxQmxZN1huVmM5VHJlK1lHRFdzWk0wMDVMamJn?=
 =?utf-8?B?N2w5WGtKL2hPWjNvQ2YrT0VhNWZ1aWJZRXFWUU83aXJyclJ2ZUJWdW5CNlNn?=
 =?utf-8?B?eklhREFod1RReDVoblQ3dEdSNnpCWkhTTU45eFQ0WVlScG53elBobTF3M1ZE?=
 =?utf-8?B?amxXY1VuZ1VKSzcyVm84SXlEdmpuc1ZQR0tuTEVWM0MzckV0cmViOE9CN2RM?=
 =?utf-8?B?d0hKY2NyS2xHNmJGM1BZSlhsNmd5K0l4NDg2TWNFRDE2M2VNelFHOVM0clcy?=
 =?utf-8?B?WS9TU0ZPNHRsTVRsWmlBZUtzWmlIbDR0dFNkZjQrcUY2OEoyeUNvUWhJVzFV?=
 =?utf-8?B?UTVsckNNNm5NY2xrR1BKVUZvVUdTeXRGeU1kcVoyZ3lDRnBmVzhuUFpzeDJZ?=
 =?utf-8?B?a3RzNWFwSWIwOUk5VndlalJOb1lYeE5oS005RHBxVHRlbEZETHFlRXpPWHJk?=
 =?utf-8?B?YmRROHlRSlBZMC9jMTV2TTliR1RHVjBqVXVrblkxeXBwUFpYNlF1Ym8wemcr?=
 =?utf-8?B?TktQMzB4cHZVQU9EU3dqTDJYdHM3K2lZSTdYVlBmZ3FrU2JDandPMExlMDZo?=
 =?utf-8?B?UXUvbGZpUzB6ZEtlNTJabUl3aTVJRDYvN0lxM2kzYVJ6SC90a2VleHRVYnNY?=
 =?utf-8?B?VzA4TlVtNDlQWkxOMjJ4WFBSL0JpeHhwenZEQWwvSWRWTm5NUWNDM1VJZEpt?=
 =?utf-8?Q?M8dzwU6Krfb3u?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 15:31:55.2208
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2bbe64f3-168a-48eb-42b1-08dc703d2863
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD5.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR12MB9317

Hi Julien,

On 5/9/2024 4:46 AM, Julien Grall wrote:
> Hi Henry,
> [...]
>
>>>> we have 3 possible states which can be read from LR for this case : 
>>>> active, pending, pending and active.
>>>> - I don't think we can do anything about the active state, so we 
>>>> should return -EBUSY and reject the whole operation of removing the 
>>>> IRQ from running guest, and user can always retry this operation.
>>>
>>> This would mean a malicious/buggy guest would be able to prevent a 
>>> device to be de-assigned. This is not a good idea in particular when 
>>> the domain is dying.
>>>
>>> That said, I think you can handle this case. The LR has a bit to 
>>> indicate whether the pIRQ needs to be EOIed. You can clear it and 
>>> this would prevent the guest to touch the pIRQ. There might be other 
>>> clean-up to do in the vGIC datastructure.
>>
>> I probably misunderstood this sentence, do you mean the EOI bit in 
>> the pINTID field? I think this bit is only available when the HW bit 
>> of LR is 0, but in our case the HW is supposed to be 1 (as indicated 
>> as your previous comment). Would you mind clarifying a bit more? Thanks!
>
> You are right, ICH_LR.HW will be 1 for physical IRQ routed to a guest. 
> What I was trying to explain is this bit could be cleared (with 
> ICH_LR.pINTD adjusted).

Thank you for all the discussions. Based on that, would below diff make 
sense to you? I did a test of the dynamic dtbo adding/removing with a 
ethernet device with this patch applied. Test steps are:
(1) Use xl dt-overlay to add the ethernet device to Xen device tree and 
assign it to dom0.
(2) Create a domU.
(3) Use xl dt-overlay to de-assign the device from dom0 and assign it to 
domU.
(4) Destroy the domU.

The ethernet device is functional in the domain respectively when it is 
attached to a domain and I don't see errors when I destroy domU. But 
honestly I think the case we talked about is a quite unusual case so I 
am not sure if it was hit during my test.

```
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index a775f886ed..d3f9cd2299 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,16 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, 
unsigned int virq,
      ASSERT(virq < vgic_num_irqs(d));
      ASSERT(!is_lpi(virq));

-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-#ifndef CONFIG_OVERLAY_DTB
-    if ( d->creation_finished )
-        return -EBUSY;
-#endif
-
      ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
      if ( ret )
          return ret;
@@ -169,20 +159,40 @@ int gic_remove_irq_from_guest(struct domain *d, 
unsigned int virq,
      ASSERT(test_bit(_IRQ_GUEST, &desc->status));
      ASSERT(!is_lpi(virq));

-    /*
-     * Removing an interrupt while the domain is running may have
-     * undesirable effect on the vGIC emulation.
-     */
-#ifndef CONFIG_OVERLAY_DTB
-    if ( !d->is_dying )
-        return -EBUSY;
-#endif
-
      desc->handler->shutdown(desc);

      /* EOI the IRQ if it has not been done by the guest */
      if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
+    {
+        /*
+         * Handle the LR where the physical interrupt is de-assigned 
from the
+         * guest before it was EOIed
+         */
+        struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], virq);
+        struct vgic_irq_rank *rank = vgic_rank_irq(v_target, virq);
+        struct pending_irq *p = irq_to_pending(v_target, virq);
+        unsigned long flags;
+
+        spin_lock_irqsave(&v_target->arch.vgic.lock, flags);
+        /* LR allocated for the IRQ */
+        if ( test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) &&
+             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+        {
+            gic_hw_ops->clear_lr(p->lr);
+            clear_bit(p->lr, &v_target->arch.lr_mask);
+
+            clear_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
+            clear_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
+            p->lr = GIC_INVALID_LR;
+        }
+        spin_unlock_irqrestore(&v_target->arch.vgic.lock, flags);
+
+        vgic_lock_rank(v_target, rank, flags);
+        vgic_disable_irqs(v_target, (~rank->ienable) & rank->ienable, 
rank->index);
+        vgic_unlock_rank(v_target, rank, flags);
+
          gic_hw_ops->deactivate_irq(desc);
+    }
      clear_bit(_IRQ_INPROGRESS, &desc->status);

      ret = vgic_connect_hw_irq(d, NULL, virq, desc, false);
```

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Thu May 09 15:36:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 15:36:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719300.1121969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55op-0006oQ-OO; Thu, 09 May 2024 15:36:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719300.1121969; Thu, 09 May 2024 15:36:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s55op-0006oJ-LL; Thu, 09 May 2024 15:36:27 +0000
Received: by outflank-mailman (input) for mailman id 719300;
 Thu, 09 May 2024 15:36:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1XlW=MM=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s55oo-0006oD-L8
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 15:36:26 +0000
Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com
 [2607:f8b0:4864:20::82e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e45141c8-0e19-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 17:36:24 +0200 (CEST)
Received: by mail-qt1-x82e.google.com with SMTP id
 d75a77b69052e-434d0e3f213so5241671cf.3
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 08:36:24 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54f5e05sm9406931cf.32.2024.05.09.08.36.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 08:36:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e45141c8-0e19-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715268983; x=1715873783; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=2rLhyY7+jFslnqo1KWFK3KBVrrUmsZ2Qd4R799lPfSc=;
        b=NfC4ZJ9heSHMnlwRb/LIBb5dqtBmhiceGrJV8ICS3MMRyqQ8ILDGyfPj1siENCy52O
         rPKTg8StJ5mLsifqYTGe3VZOPJooM4b0AF83PG2Ti8doBbCBpqUScFrc7mgaMkdxUvsz
         22IZRIWHUPHoN+EEK2oF2WYZDNmLpDvDrzgsI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715268983; x=1715873783;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2rLhyY7+jFslnqo1KWFK3KBVrrUmsZ2Qd4R799lPfSc=;
        b=F6RExw2vMvPrZMLaosPT98zXGVjgfhGIe9G+yIKOhVfRb/Mkx8SFMxQrW9o1oDAkkk
         ElJDNlveAfjbvnRLu590kMQJZjGOvikyvrfIg5/VIoM/e/eBqDvxJ1mTmVWsrNrsct2a
         CYHNdIOhGHuh32xz2XipxkHzKLTcHleD1fS00dEEAEJDSMDZ6L85qNFZR22rC27xAQ20
         R4Lxwp/q+aR5WXmzqdh0syHhufcf2uHpPE9hO5Ufse+E237A1xbcOwW2O0Ggc+dhkRdT
         3IiyFFCTRIUJ/oClOdo3e8omNT5nEkeZQ0EE8LXnyl8xMs0+E9RwrKkD/Lk72E+PJKjq
         kpAw==
X-Forwarded-Encrypted: i=1; AJvYcCXT7dqgXBTbIB6VlO2uxWZgYB8KvMKzwLr27ZTNrhgkOKrIWeMVtTX1JbRNBNf4J9UYYBOiP88ooSVAWspMvjkvXvTSRuB/eVTBpNAaytg=
X-Gm-Message-State: AOJu0YzN8tRRUcIBRFnfFB4qIilKXZ6qcoRUUTC6sPzLZqd0vQiXyNVH
	GB9fdJ6jw53GFwfxkKLaF9BWUg0retvmun7CSBelbIlkw9pcQ1IyGGJOBdB6leg=
X-Google-Smtp-Source: AGHT+IHgHOG4xtfch0vyDazppqDtgJeZdoSuZImSsyYfeQercExIaT1iKeCHt1AmD3iS0Klbc7vMZA==
X-Received: by 2002:ac8:5d8f:0:b0:43a:ffa5:3f24 with SMTP id d75a77b69052e-43dbf74d77amr60079061cf.58.1715268983385;
        Thu, 09 May 2024 08:36:23 -0700 (PDT)
Message-ID: <5bf22886-08a1-459d-939e-008f2c1836ea@citrix.com>
Date: Thu, 9 May 2024 16:36:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 0/8] Clean up of gzip decompressor
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
References: <20240424163422.23276-1-dpsmith@apertussolutions.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240424163422.23276-1-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 24/04/2024 5:34 pm, Daniel P. Smith wrote:
> An issue ran into by hyperlaunch was the need to use the gzip decompressor
> multiple times. The current implementation fails when reused due to tainting of
> decompressor state from a previous usage. This series seeks to colocate the
> gzip unit files under a single directory similar to the other decompression
> algorithms.  To enable the refactoring of the state tracking, the code is then
> cleaned up in line with Xen coding style.
>
> Changes in v3:
> - patch "xen/gzip: Drop huffman code table tracking" was merged
> - patch "xen/gzip: Remove custom memory allocator" was merged
> - patch "xen/gzip: Drop unused define checks" was merged
> - move of the decompressor state into struct was broken up to ease review
> - expanded macros that were either only used once or obsfucated the logic
> - adjusted variable types as needed to be more appropriate for their usage
>
> Changes in v2:
> - patch "xen/gzip: Colocate gunzip code files" was merged
> - dropped ifdef chunks that are never enabled
> - addressed formatting changes missed in v1
> - replaced custom memory allocator with xalloc_bytes()
> - renamed gzip_data to gzip_state
> - moved gzip_state to being dynamically allocated
> - changed crc table to the explicit size of uint32_t 
> - instead of moving huffman tracking into state, dropped altogether
>
>
> Daniel P. Smith (8):
>   gzip: clean up comments and fix code alignment
>   gzip: refactor and expand macros
>   gzip: refactor the gunzip window into common state
>   gzip: move window pointer into gunzip state
>   gzip: move input buffer handling into gunzip state
>   gzip: move output buffer into gunzip state
>   gzip: move bitbuffer into gunzip state
>   gzip: move crc state into gunzip state

In hindsight my suggestion that lead to "refactor and expand macros"
wasn't great.

We want to keep the underrun labels for a future when the error handling
isn't panic().  After that, expanding flush_window() is better folded
into "refactor the gunzip window into common state" to reduce the churn.

As that was my fault, and unpicking it is reasonably hard, and we're on
a tight deadline for 4.19 now, I've gone ahead and unpicked this.

Also I've rebased over Jan's patch addressing the memory leaks reported
by Coverity, as the two sets of fixes collide, along with various other
minor notes.

I've conferred with Daniel who is happy with the aformentioned changes.

This area of code still has a lot of work needing doing on it, but
removing the use of global state is a concrete improvement in the status
quo.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 09 15:54:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 15:54:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719305.1121980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s565o-0001KR-4C; Thu, 09 May 2024 15:54:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719305.1121980; Thu, 09 May 2024 15:54:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s565o-0001KK-15; Thu, 09 May 2024 15:54:00 +0000
Received: by outflank-mailman (input) for mailman id 719305;
 Thu, 09 May 2024 15:53:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1XlW=MM=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s565m-0001KE-OG
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 15:53:58 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5748768a-0e1c-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 17:53:56 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a5a1054cf61so255500766b.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 08:53:56 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f78sm86358066b.133.2024.05.09.08.53.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 08:53:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5748768a-0e1c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715270035; x=1715874835; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=AFqPM6h4b+zncfhX+ydefZM27ejofMemYZ4BF17oo1w=;
        b=EoqXlwxjWryS9LYa+4qVhOxSDrxiAl1AR6J5hPbM5uzcdRQ/fA+LMGGCzkEiK8//dc
         wlhkME8zCXk0dc3e558xDniqJ8rlxZ78+6EfwX6FGr0Dea88VzGdgxZfrnHiw5CsmKcO
         m7bn+CEBE5iGGBcIl5kFd6mavq5yHaJGHZleI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715270035; x=1715874835;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AFqPM6h4b+zncfhX+ydefZM27ejofMemYZ4BF17oo1w=;
        b=NowCZlVNXFZ1QiDnP6zkpxlxymtdvLwUlbPWhIby6DFWgqooTYluGX4TM48hqV3Ikp
         PBXZCJkpXRMUF8qisVtiK93h2m0PoTrdJkAxHLF/PAIk0vuK/oDGPWVXNP4F+/tHAIaL
         jjDmxInQT9YCutHcmIlvlyWPfC3tuMyYgzbJmO1awsGQov1QQtB2ZvCFePXde3oigU9I
         HbqGfT2E8K5A/3PU+tb9jbdJTZCgpV/3Vu3FEHXcEZwX8WZ4peqrW/gHyts+HytU1YDX
         CsyNTvYe1WzV+Q9Bk/z3LDeCVccp7uiClN5DJlXbCy2RI+y1Eqm/QoOJv3uYoDZ+WGZ/
         loZQ==
X-Forwarded-Encrypted: i=1; AJvYcCUjIS6FOpDHXVGlMvSoTNZAhHtTG10oVhgcUzC8X1RsfKkORWS47BWiDsXnR3SeSFACLLrVn6y2utD2LJ0NGtPlDW8w2YlRS+VOQnYdiX0=
X-Gm-Message-State: AOJu0YyvlmerDRa/BmJNpfT5XvCIdYZDnk2r0aDhcglCRvj9XPwwhSZd
	tNarER+n/oBPMmuLjcH+gR0W4GNaureAzNYGP4oQDE4i/S3ciw0GIsxSOsr14B8=
X-Google-Smtp-Source: AGHT+IF1co9OQ3ixr+I28h9DoIxwf0x7gUYgeTs3rNBjEe1X13EmUATNhQdN/agxNB7nvl/G0emcDw==
X-Received: by 2002:a17:906:3e0d:b0:a5a:1077:334 with SMTP id a640c23a62f3a-a5a2d57a39amr2153066b.26.1715270035338;
        Thu, 09 May 2024 08:53:55 -0700 (PDT)
Message-ID: <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
Date: Thu, 9 May 2024 16:53:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH v2 0/5] Add bridge VLAN support
To: Leigh Brown <leigh@solinno.co.uk>, xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Juergen Gross <jgross@suse.com>, Jason Andryuk <jandryuk@gmail.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240508213824.55824-1-leigh@solinno.co.uk>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/05/2024 10:38 pm, Leigh Brown wrote:
> Hello all,
>
> I realised over the weekend that there is a valid use case for providing
> a VIF to a domain that has access to multiple VLANs, e.g. a router. Yes,
> you can create a VIF per VLAN, but if you start having several VLANs (as
> I do), it would be nicer to create a single interface that has access to
> all the relevant VLANs (e.g. enX0.10, enX0.20, etc.).
>
> So, version 2 changes the name and type of the parameter from an integer
> called `vid' to a string called `vlan'. The vlan parameter is then
> parsed by the vif-bridge script (actually, the functions called by it in
> xen-network-common.sh).
>
> As it quite a common practice to allocate VLANs in round numbers, I also
> implemented the ability to specify contiguous or non-contiguous ranges.
> You can specify whether a VLAN is tagged or untagged, and which VLAN is
> the PVID (only one PVID is allowed).  For example,
>
> vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20-29' ]
>
> will setup the VIF so that 10 is the PVID and VLAN IDs 20 through 29
> are permitted with tags. Another example:
>
> vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=1p/10+10x9' ]
>
> will setup the bridge to set 1 as the PVID and permit access with
> tags for VLAN IDs 10, 20, 30, 40, 50, 60, 70, 80 and 90.
>
> This patch set enables this capability as follows:
>
> 1. Adds `vlan' as a new member of the libxl_device_nic structure;
> 2. Adds support to read and write the vlan parameter from the xenstore;
> 3. Adds `vlan' as a new keyword for the vif configuration option;
> 4. Adds support to assign the bridge VLANs in the Linux hotplug scripts;
> 5. Updated xl-network-configuration(5) manpage and example configs.
>
> Original blurb below:
>
> For many years I have been configuring VLANs on my Linux Dom0 by
> creating VLAN interfaces for each VLAN I wanted to connect a domain
> to and then a corresponding bridge. So I would tend to have things
> like:
>
> enp0s0    -> br0     -> vif1, vif2
> enp0s0.10 -> br0vl10 -> vif3, vif4
> enp0s0.20 -> br0vl20 -> vif5
> dummy0    -> br1     -> vif6
>
> I recently discovered that iproute2 supports creating bridge VLANs that
> allows you to assign a VLAN to each of the interfaces associated to a
> bridge. This allows a greatly simplified configuration where a single
> bridge can support all the domains, and the iproute2 bridge command can
> assign each VIF to the required VLAN.  This looks like this:
>
> # bridge vlan
> port              vlan-id  
> enp0s0            1 PVID Egress Untagged
>                   10
>                   20
> br0               1 PVID Egress Untagged
> vif1.0            1 PVID Egress Untagged
> vif2.0            1 PVID Egress Untagged
> vif3.0            10 PVID Egress Untagged
> vif4.0            10 PVID Egress Untagged
> vif5.0            20 PVID Egress Untagged
> vif6.0            30 PVID Egress Untagged
>
> This patch set enables this capability as follows:
>
> 1. Adds `vid' as a new member of the libxl_device_nic structure;
> 2. Adds support to read and write vid from the xenstore;
> 3. Adds `vid' as a new keyword for the vif configuration option;
> 4. Adds support for assign the bridge VLAN in the Linux hotplug scripts.
>
> I don't believe NetBSD or FreeBSD support this capability, but if they
> do please point me in the direction of some documentation and/or examples.
>
> NB: I'm not very familiar with Xen code base so may have missed
> something important, although I have tested it and it is working well
> for me.
>
> Cheers,
>
> Leigh.
>
>
> Leigh Brown (5):
>   tools/libs/light: Add vlan field to libxl_device_nic
>   tools/xl: add vlan keyword to vif option
>   tools/hotplug/Linux: Add bridge VLAN support
>   docs/man: document VIF vlan keyword
>   tools/examples: Example Linux bridge VLAN config
>
>  docs/man/xl-network-configuration.5.pod.in    |  38 ++++++
>  tools/examples/linux-bridge-vlan/README       |  68 +++++++++++
>  tools/examples/linux-bridge-vlan/br0.netdev   |   7 ++
>  tools/examples/linux-bridge-vlan/br0.network  |   8 ++
>  .../examples/linux-bridge-vlan/enp0s0.network |  16 +++
>  tools/hotplug/Linux/xen-network-common.sh     | 111 ++++++++++++++++++
>  tools/libs/light/libxl_nic.c                  |  10 ++
>  tools/libs/light/libxl_types.idl              |   1 +
>  tools/xl/xl_parse.c                           |   2 +
>  9 files changed, 261 insertions(+)
>  create mode 100644 tools/examples/linux-bridge-vlan/README
>  create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
>  create mode 100644 tools/examples/linux-bridge-vlan/br0.network
>  create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network
>

This is past the last-post date, so Oleksii will need to decide whether
he's happy to make an exception for it.

Anthony is OoO for a month now, so if this is to get in for 4.19, then
it will need reviewing by others.  I've CC'd a few plausible candidates...

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 09 16:07:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 16:07:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719314.1121991 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s56IL-00043C-9z; Thu, 09 May 2024 16:06:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719314.1121991; Thu, 09 May 2024 16:06:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s56IL-000435-5m; Thu, 09 May 2024 16:06:57 +0000
Received: by outflank-mailman (input) for mailman id 719314;
 Thu, 09 May 2024 16:06:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cFQq=MM=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s56IK-00042p-8o
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 16:06:56 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2681e829-0e1e-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 18:06:53 +0200 (CEST)
Received: from PH8PR05CA0015.namprd05.prod.outlook.com (2603:10b6:510:2cc::16)
 by SA1PR12MB8598.namprd12.prod.outlook.com (2603:10b6:806:253::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48; Thu, 9 May
 2024 16:06:47 +0000
Received: from CY4PEPF0000E9DB.namprd05.prod.outlook.com
 (2603:10b6:510:2cc:cafe::6f) by PH8PR05CA0015.outlook.office365.com
 (2603:10b6:510:2cc::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46 via Frontend
 Transport; Thu, 9 May 2024 16:06:46 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000E9DB.mail.protection.outlook.com (10.167.241.74) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Thu, 9 May 2024 16:06:46 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 9 May
 2024 11:06:45 -0500
Received: from [172.30.155.127] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 9 May 2024 11:06:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2681e829-0e1e-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kb2hlkTxefASncpU0TUBK6DYhPZbsvfJI7B/C19vAW+T0+RqofiuwipXXc73HR9FPmZcSN+Jwg9kSPI3Jp56uC3nJWGDnLk3VT9+hvMJZEy0Aj/aSrGPgdHupKAu5KPpqRV6mDItCWgZXxutlJUTgnsaBXfzNYsDE3rzVQ6GoGqpCKDN78Vx978djq2UObZVexD7qqb+q4VYq15vY9r0ddCVZZWV4jaiL1GzBOAtpC1EzQBhfXohfFdWzluIXfW6Q+MsIad3L9PWvYUMuZwiREA/WOnvaQ5B/aaL28DuTwydN+u2/C8mmWmNHCJ+lq6CJXIAY8ICzzXut91xSgC+yw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RTmKuPIJV+UNMuSZ/dUUUWQVTWGzNtqp+DZ+XxePY04=;
 b=C2CQWStWXzeoTFLPvObV3U81M9NZL5LfSvigIN2rvU1KINrCcdrIwqwyLF85uZQVEdAIHBq17eV5lW1pqtAhkWk2Uf7QRS/Cfg+iJhwk2JSEi7SR69EzKjSlpkP4vOf/ZwhRXbK7zPMzvf3Ef303KULftXZt8yb3VcE+pa7jLXZ41IpiY8lt3/nlVT2ctJb5mLBCZk3xZf4Y4mEPVCnLnc4dd8PSdom1RFIGNeO9GxmWUSA4V3yIbAdk6ea+N0hti90IkMvs5QPMiBaOiX6XxROpkn6kMtfA4h6cPkljBgeHOWhAbbdwdEwOdCui3h0enmU2EC4LT1hkIH8XGeto4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RTmKuPIJV+UNMuSZ/dUUUWQVTWGzNtqp+DZ+XxePY04=;
 b=CSmfBOo0qpgARRTxgYzkZPTHRnilCEqeAGt682lqTp5CWuynSosdfRftGnhrnO/i41+uT/NLAiaACoSRVQP25jdOInlkwiulNkI1IxO3SnQ0sJsvY6byZJtuviB9U5YZS9sU/3VftmWX1h+CDyq2WnLF/cUT+1Qe614AoQ1nu0s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <c53b9134-4127-4c3c-a7a8-1630df7c3eb1@amd.com>
Date: Thu, 9 May 2024 12:06:40 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] x86: detect PIC aliasing on ports other than
 0x[2A][01]
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <0c45155a-2beb-4e69-bca3-cdf42ba22f2b@suse.com>
 <c29ced52-6e1e-4997-81ab-8882df2d38a7@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <c29ced52-6e1e-4997-81ab-8882df2d38a7@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DB:EE_|SA1PR12MB8598:EE_
X-MS-Office365-Filtering-Correlation-Id: 88dca06f-20cd-49a3-bbd4-08dc704206ec
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?LzBxU09Bc2EzU2tEVkljNXRUSTZ1NWs1ZkdROGpFbmh4SVBSUGJFRW02WXZW?=
 =?utf-8?B?UUZ6aUhxVU1RcjFrYncyRGRJM3c1NWhCYi9PSE9oSU1SL1Q4YlBtVkdMUURJ?=
 =?utf-8?B?c3Z0dUFVeFYrd240dVBza21ESzZNUW1VS0tBeHQ4YUE5L1FtU09YVUdPalVK?=
 =?utf-8?B?TVM4YndybnRVZGhRSmtSUnFrTVpENUJGU0ZobnRNeUZwbm81VXFJNDF5US9O?=
 =?utf-8?B?Y2ZMTUwxdjRTRCtpL3FvMnRSdmcvUG1IeTBGYldRVUFCTzFNUC96T1JEUGZl?=
 =?utf-8?B?cGpJdFVGUXlzamJkVDRVWUYrNG11aHc1WGR0L3c1ZlhmYUc5cFhqUkxiUGl3?=
 =?utf-8?B?S0NpTi9MQ1FvdDZ3QnhmN1hiQVdXY244RFlIeEFuSXh0SDUwSlZRN0NxT2dF?=
 =?utf-8?B?eGdtNldOT2M2L3NoZmxSaE9WcnozcmVwWGdEMXpxTzcvTzRyL2w3SzBsWkNo?=
 =?utf-8?B?KzAvK0dDNDlWZUVUbzV2TXFGcFpIdmZRL2s2b1djMW0zSHVhcm10ZWgvMVlF?=
 =?utf-8?B?dGVIRFBrc0Q0UjJTd0U4S1U2UUNYYis5S2dIN2IrUVp5cXNqQXFZTDJIcEZ0?=
 =?utf-8?B?empPSSsrNFoxeW0zdWJSUE5SU1l1RXFRbXl0MUhBTjNsTXNrZ2ZERTJHZlZY?=
 =?utf-8?B?N2RoT2Z0NXB0WjdvZnJwSUxtY29rRUhyNUVMSFNKczAxbm91NjRxY0N2OWto?=
 =?utf-8?B?V3EwK3JMbWh6V0dKWjRmTkx5cG5zYklBTStNd1I1Vk4xcHJhV1JpQWN6SDlm?=
 =?utf-8?B?SFBRNkZzeFlEbmVDT1k1RmlraVQ3MGU1T1RPRE0vUnNlSlpzditXNVNDSDF3?=
 =?utf-8?B?U2xyc3hpajB5WXhhS09ZUmNVRGJuYTlNSkg5Q3hQeTByR2t1TWgxNERabXFm?=
 =?utf-8?B?bTRySE54QTQyYUVkaGpiZzc0OWplMHlScC9nYmRwclJzWXlTZStvajE4VWNC?=
 =?utf-8?B?ejVyQ2tvek1qc0xUeUg4U3NiMmoxSzMwVnJHVUVpSy8wbXVEaDk1TC9PZ2pM?=
 =?utf-8?B?YjY2RHA3d2tHdXJUZ1hqN0F4UkJwMjBXRTJZVnRLWVFVMU9YTHIwaTFYN25l?=
 =?utf-8?B?eVdDakpGRm9LQkZxR25xcVRKVlh6Umg1ZjQ4UzAxYXZmQTRmSkNSOGlpbCto?=
 =?utf-8?B?dlBVUVRyWXJhbGI2MkVHbk81Uk53MmZwQzh6ZVhFSzV3b28vd3NtZ3hkVTAx?=
 =?utf-8?B?NjUvVjZ4M3lkMFl0M3FaNGpGRzJjWEhJWm8wSUxocklrWCtJTUZSenloN3Jz?=
 =?utf-8?B?Mnc0ZUxna1JmcENDZUpGdHhEZkU2ekxOb1R1bjFrUHVsWk03UFlLTnVnd2Zh?=
 =?utf-8?B?WDFqRWgrV0QydkpoZTlBR2d0MEkrQndXN1M2Z0dKQjBhcjlKYVZQd3NmVVU1?=
 =?utf-8?B?cVB0Q1dpTENlY05Qckw1ZlFBdW1PcGVzRlRGVkowRFRDdDcxR3NsRXR4bXpG?=
 =?utf-8?B?dUJBTnJQaUZDSjcvMloxZEpkdHpHUGhMY0puNUZFMU1DakJTcFloWWNhRGgw?=
 =?utf-8?B?cUtjVlV0eHRGc1hqa2dKc0k5a2kyQjRObGtRemtteU9OdVJmdEt2R0I0QllL?=
 =?utf-8?B?c01JaHdlYVVSVVNvbGw0SW1HTFRERWx2WkdjS1VJSFFjZTNXczFLN1BiN0RM?=
 =?utf-8?B?Tk4veTAzTTdSQlJBT0F6U1VJeUpHRnhYdzlucHNJZFJITGZlL0FDeXpkR3dP?=
 =?utf-8?B?cWlEU2ZVQ2NidUpNc3Z0V3JvUVlnZngweE5ENUNKUDcvV2JmME9KMm1jOEZY?=
 =?utf-8?B?aCtYVnpUbWFFbGJTRUtGS3ROSFpXbzM3TnBvQ2llQXNKb1BYeWJ0MmYwVGpT?=
 =?utf-8?B?U0hYK1lIOStsNi9nRkp4ZENCL25ybTllbDY2SUEvYmxnakFIdEZlVzhoNkxC?=
 =?utf-8?Q?bsPGFHjIVXzAt?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 16:06:46.5004
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 88dca06f-20cd-49a3-bbd4-08dc704206ec
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8598

On 2023-12-18 09:48, Jan Beulich wrote:
> ... in order to also deny Dom0 access through the alias ports. Without
> this it is only giving the impression of denying access to both PICs.
> Unlike for CMOS/RTC, do detection very early, to avoid disturbing normal
> operation later on.
> 
> Like for CMOS/RTC a fundamental assumption of the probing is that reads
> from the probed alias port won't have side effects in case it does not
> alias the respective PIC's one.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Code-wise
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

I appreciate what Jan is trying to achieve.  I also share Roger's 
concern about reading IO ports since we don't know what is there.

I also don't have a good alternative to suggest...

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Thu May 09 16:10:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 16:10:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719316.1121999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s56MB-0005UW-No; Thu, 09 May 2024 16:10:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719316.1121999; Thu, 09 May 2024 16:10:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s56MB-0005UP-L5; Thu, 09 May 2024 16:10:55 +0000
Received: by outflank-mailman (input) for mailman id 719316;
 Thu, 09 May 2024 16:10:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y6XG=MM=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s56M9-0005UJ-Nq
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 16:10:54 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b43ed473-0e1e-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 18:10:51 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 5B25F80058;
 Thu,  9 May 2024 17:10:50 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b43ed473-0e1e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715271050;
	bh=28qez7AsmLm1bdQt/hM8A9aHukwMHE/I7EpLIEWCf68=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=VyKqEEl5+vnojZHjPS+4VCT7PCG3WTcg4cofqNAijZDHokz5sMoMiTJIjwwC3GYDc
	 3sPRSmQuLffMmTrqtj+U9xEtLCZKafA9UvxW3TVBj998jFJVTVhVZSgS9rOUL5F7ie
	 ssWLgKhAktPJDofIwe8DMxhYPgOcIXu57JaGLXpI=
MIME-Version: 1.0
Date: Thu, 09 May 2024 17:10:50 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Juergen Gross <jgross@suse.com>, Jason Andryuk
 <jandryuk@gmail.com>
Subject: Re: [RFC PATCH v2 0/5] Add bridge VLAN support
In-Reply-To: <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
 <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
Message-ID: <2d89f70c625c3abd24bd57f2ee0d2f94@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Andrew,

On 2024-05-09 16:53, Andrew Cooper wrote:
> On 08/05/2024 10:38 pm, Leigh Brown wrote:
>> Hello all,
>> 
>> I realised over the weekend that there is a valid use case for 
>> providing
>> a VIF to a domain that has access to multiple VLANs, e.g. a router. 
>> Yes,
>> you can create a VIF per VLAN, but if you start having several VLANs 
>> (as
>> I do), it would be nicer to create a single interface that has access 
>> to
>> all the relevant VLANs (e.g. enX0.10, enX0.20, etc.).
>> 
>> So, version 2 changes the name and type of the parameter from an 
>> integer
>> called `vid' to a string called `vlan'. The vlan parameter is then
>> parsed by the vif-bridge script (actually, the functions called by it 
>> in
>> xen-network-common.sh).
>> 
>> As it quite a common practice to allocate VLANs in round numbers, I 
>> also
>> implemented the ability to specify contiguous or non-contiguous 
>> ranges.
>> You can specify whether a VLAN is tagged or untagged, and which VLAN 
>> is
>> the PVID (only one PVID is allowed).  For example,
>> 
>> vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20-29' ]
>> 
>> will setup the VIF so that 10 is the PVID and VLAN IDs 20 through 29
>> are permitted with tags. Another example:
>> 
>> vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=1p/10+10x9' ]
>> 
>> will setup the bridge to set 1 as the PVID and permit access with
>> tags for VLAN IDs 10, 20, 30, 40, 50, 60, 70, 80 and 90.
>> 
>> This patch set enables this capability as follows:
>> 
>> 1. Adds `vlan' as a new member of the libxl_device_nic structure;
>> 2. Adds support to read and write the vlan parameter from the 
>> xenstore;
>> 3. Adds `vlan' as a new keyword for the vif configuration option;
>> 4. Adds support to assign the bridge VLANs in the Linux hotplug 
>> scripts;
>> 5. Updated xl-network-configuration(5) manpage and example configs.
>> 
>> Original blurb below:
>> 
>> For many years I have been configuring VLANs on my Linux Dom0 by
>> creating VLAN interfaces for each VLAN I wanted to connect a domain
>> to and then a corresponding bridge. So I would tend to have things
>> like:
>> 
>> enp0s0    -> br0     -> vif1, vif2
>> enp0s0.10 -> br0vl10 -> vif3, vif4
>> enp0s0.20 -> br0vl20 -> vif5
>> dummy0    -> br1     -> vif6
>> 
>> I recently discovered that iproute2 supports creating bridge VLANs 
>> that
>> allows you to assign a VLAN to each of the interfaces associated to a
>> bridge. This allows a greatly simplified configuration where a single
>> bridge can support all the domains, and the iproute2 bridge command 
>> can
>> assign each VIF to the required VLAN.  This looks like this:
>> 
>> # bridge vlan
>> port              vlan-id
>> enp0s0            1 PVID Egress Untagged
>>                   10
>>                   20
>> br0               1 PVID Egress Untagged
>> vif1.0            1 PVID Egress Untagged
>> vif2.0            1 PVID Egress Untagged
>> vif3.0            10 PVID Egress Untagged
>> vif4.0            10 PVID Egress Untagged
>> vif5.0            20 PVID Egress Untagged
>> vif6.0            30 PVID Egress Untagged
>> 
>> This patch set enables this capability as follows:
>> 
>> 1. Adds `vid' as a new member of the libxl_device_nic structure;
>> 2. Adds support to read and write vid from the xenstore;
>> 3. Adds `vid' as a new keyword for the vif configuration option;
>> 4. Adds support for assign the bridge VLAN in the Linux hotplug 
>> scripts.
>> 
>> I don't believe NetBSD or FreeBSD support this capability, but if they
>> do please point me in the direction of some documentation and/or 
>> examples.
>> 
>> NB: I'm not very familiar with Xen code base so may have missed
>> something important, although I have tested it and it is working well
>> for me.
>> 
>> Cheers,
>> 
>> Leigh.
>> 
>> 
>> Leigh Brown (5):
>>   tools/libs/light: Add vlan field to libxl_device_nic
>>   tools/xl: add vlan keyword to vif option
>>   tools/hotplug/Linux: Add bridge VLAN support
>>   docs/man: document VIF vlan keyword
>>   tools/examples: Example Linux bridge VLAN config
>> 
>>  docs/man/xl-network-configuration.5.pod.in    |  38 ++++++
>>  tools/examples/linux-bridge-vlan/README       |  68 +++++++++++
>>  tools/examples/linux-bridge-vlan/br0.netdev   |   7 ++
>>  tools/examples/linux-bridge-vlan/br0.network  |   8 ++
>>  .../examples/linux-bridge-vlan/enp0s0.network |  16 +++
>>  tools/hotplug/Linux/xen-network-common.sh     | 111 
>> ++++++++++++++++++
>>  tools/libs/light/libxl_nic.c                  |  10 ++
>>  tools/libs/light/libxl_types.idl              |   1 +
>>  tools/xl/xl_parse.c                           |   2 +
>>  9 files changed, 261 insertions(+)
>>  create mode 100644 tools/examples/linux-bridge-vlan/README
>>  create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
>>  create mode 100644 tools/examples/linux-bridge-vlan/br0.network
>>  create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network
>> 
> 
> This is past the last-post date, so Oleksii will need to decide whether
> he's happy to make an exception for it.

 From my own perspective, I know this is an enhancement and am more than
happy to maintain it locally for the time being, so no problem if it has
to wait.  Still happy for any feedback though :-)

> Anthony is OoO for a month now, so if this is to get in for 4.19, then
> it will need reviewing by others.  I've CC'd a few plausible 
> candidates...
> 
> ~Andrew

Regards,

Leigh,


From xen-devel-bounces@lists.xenproject.org Thu May 09 17:30:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:30:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719347.1122116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57af-0000bv-OL; Thu, 09 May 2024 17:29:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719347.1122116; Thu, 09 May 2024 17:29:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57af-0000bm-LW; Thu, 09 May 2024 17:29:57 +0000
Received: by outflank-mailman (input) for mailman id 719347;
 Thu, 09 May 2024 17:29:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QLvV=MM=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s57af-0008Hw-66
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:29:57 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c0f9c272-0e29-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 19:29:56 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51f17ac14daso1451647e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:29:56 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f39d2cb0sm364149e87.276.2024.05.09.10.29.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:29:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0f9c272-0e29-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715275796; x=1715880596; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jPCYivsqUnFKto+ateYrmLlKD4VaCAkHPo9leQ/G7oM=;
        b=cdxGtIkoQ2NtQdDstJoE1QMd6RABv6PUMUNpPzqgkafNHpYnkHWPndoFdUG+8RZgUb
         FPh1O0VyQMF31JHf1lAekHquOxoiRdYpdNebtq1YcdNxNWaoIUSJ8QxVOIw+UCJxX1GL
         zUC/Krxo8ASngS/wOFDLk+1Vf/OFR0R1blPebHCPCTB5Yqsr3wEWqO9xu2GBbB4rZDSQ
         4yjitCxpo1wXDjk9rkkMc4yKA2Po2+Ezv1qJSxvIHgXWwtSaO5jPcUrRED0hQDOZi6GD
         dt5zxmVpjckQxKAeznh6cmfenlRiY/4UBNXbi+mdehGpxUhLrh+q9JRxneWaezjpwhRo
         4H1w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715275796; x=1715880596;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jPCYivsqUnFKto+ateYrmLlKD4VaCAkHPo9leQ/G7oM=;
        b=jimZS/YeLK1nKdORtedOOKqSOnxIq7M2oFrd6OeDW5cHnKBIYDCv45KXM7HxS1uWdW
         r4/Gzughpl0H9quEUWZrIAMQeqqjf8Px9KWLBc24Cjy8bA3uuuBOdNWyTSVR9wjlmxr+
         lGRsZvVJAdZfAtyBVhYSYfdaRD9gtJ8U7aA+SzQ1BZY6bCb83zEFxwhuMvqwigNxk/y2
         Qqbm5MlYH0zyQmfxDrTEsAhu5CTfa8/q3+YuOrD67M5o/T1Hv+QEVfZ5woUQ2KFirzaK
         xQdyRWpTu4mpXZqgv4ERsyZZWa5iv/0m537bjJr1ZtxF6GPCDhT55nbN0S23igz0aFJj
         oXFw==
X-Forwarded-Encrypted: i=1; AJvYcCX5FfBL6c2Sd0T9V9J8KxPNp7+dWrt//o3qMM59lSZEC2oQI1eWK8rJlxv9KoR6IxxoduplMFny15K7hdvUKc18S6k1e++lzttb5WeTs6Y=
X-Gm-Message-State: AOJu0YzcHwJJqcQU++seGNKTeBJBMqdQ6BehbZInhkwO5Q0Lp3gMWnD3
	P3iEUmbKgYxveHOxnQ2Z4JkLJ5I2Ug9hlYWpZLs/7k3Q9kQGSkrb
X-Google-Smtp-Source: AGHT+IGXpW00WomSWOz6NY89WyIw3oecHI+P6P9NdIdqTkVjlwXFHCkPAdJcU5KxXWJWBW7z9vO7OA==
X-Received: by 2002:a19:700b:0:b0:521:44ce:2999 with SMTP id 2adb3069b0e04-52210074a29mr116899e87.47.1715275795997;
        Thu, 09 May 2024 10:29:55 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()
Date: Fri,  3 May 2024 03:44:47 +0200
Message-Id: <20240503014449.1046238-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 hw/xen/xen-mapcache.c         | 16 +++++++++++-----
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c              |  9 +++++----
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..26bc38a9e3 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
                              void *vaddr,
                              hwaddr size,
                              hwaddr address_index,
-                             bool dummy)
+                             bool dummy,
+                             ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
     xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
+                                       ram_addr_t ram_offset,
                                        uint8_t lock, bool dma, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
     if (!entry) {
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
-        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                         ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
                 !test_bits(address_offset >> XC_PAGE_SHIFT,
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
-            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                             ram_offset);
         }
     }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
                        hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
     uint8_t *p;
 
     mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+                               lock, dma, is_write);
     mapcache_unlock(mapcache);
     return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false);
+                     cache_size, address_index, false, new_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
                         void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
                                      hwaddr phys_addr,
                                      hwaddr size,
+                                     ram_addr_t ram_addr_offset,
                                      uint8_t lock,
                                      bool dma,
                                      bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
          */
         if (xen_mr_is_memory(block->mr)) {
             return xen_map_cache(block->mr, block->offset + addr,
-                                 len, lock, lock,
-                                 is_write);
+                                 len, block->offset,
+                                 lock, lock, is_write);
         }
 
         block->host = xen_map_cache(block->mr, block->offset,
-                                    block->max_length, 1,
-                                    lock, is_write);
+                                    block->max_length,
+                                    block->offset,
+                                    1, lock, is_write);
     }
 
     return ramblock_ptr(block, addr);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 17:30:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:30:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719344.1122085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57aZ-0008I9-09; Thu, 09 May 2024 17:29:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719344.1122085; Thu, 09 May 2024 17:29:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57aY-0008I2-Tq; Thu, 09 May 2024 17:29:50 +0000
Received: by outflank-mailman (input) for mailman id 719344;
 Thu, 09 May 2024 17:29:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QLvV=MM=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s57aY-0008Hw-4e
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:29:50 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bc40b60b-0e29-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 19:29:48 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51f74fa2a82so1374036e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:29:48 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f35ad6aasm380187e87.46.2024.05.09.10.29.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:29:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc40b60b-0e29-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715275788; x=1715880588; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=nOC6rTy44r+zghYsUwaCP7+YZqjTSf/d95MwiNxlDoJXH9vgGoxpo5hgdo+Tmq6+l6
         JgkSr8cNn46KPLfimy5+ODBpVqnHQizYzUcATkVE0vYPKZ7BJGrfT/+my+5h/Ib9ACNu
         0i/lcHl7Bzfyky/C28moiEiMCw8bLaoOu3xmW3b/AVZx4M5NMxJEk1F2c5WyI7CnCeWb
         bHHrCAuW7KnECoEPk1+abmqvUzR+e+ujXuyIPzxopXnb5q8dnL4ANZqMsaOz/APO5p0r
         3tQI3DDL4NdFfzVnk8jnxgGYkyMhT7jlmVxXi6upup02Qwqmt6UsogPvp0enTqbnrKHi
         7wlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715275788; x=1715880588;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=kZXTZRptPhbtXGuYtgsaSYPekwv+IX0jRBz1exW14ikP3Lm98j0usIUyUmwl2KyieF
         PLOeuxsc2G5rvFe6svIPABdNTpcaDrX9pnNIrRYVTLlUSSDEAetB72GZhM41IUrFgGi9
         +Jl68M45h1zDbp29bxD1c8H68uN81zfgkKh0zuwt+IZ3EnLdP2HkMHyqenx8WcJSjRh1
         rF9lBamefTUZb75A5e3JjVpvyRPWLD8Sm+2IguXerBsDPM9ripVrb5nKPWI6z4SlGI8g
         jEedoIf+rm9V26EJicNMARszEzE4iLZD0jVGzJlqXmnMlHIVK414tMZXRiFFYxuL6pUP
         F+1g==
X-Forwarded-Encrypted: i=1; AJvYcCXaJAHJmrv9127a45R6TD5XBdx8wwWF+DEvACUrQpoeaMRUPd5Wdqs+Mo5mq73UeDYf/ZivhUec47YaaozwzNh+CNX8zo0WNHMm0gllcyY=
X-Gm-Message-State: AOJu0YziXWMLglbrRB+HaU9EOncJVr3OBNvlV75HtYWcqJ8VdljqqoHf
	n4FKoBGK6KDuzS69NVRaeAczsLHy2zXgVbbmZsZXcY872UXQdEH6
X-Google-Smtp-Source: AGHT+IHeG8OvDdp1Hd81Fpds95oiK0KakPxNqX9S4bKF8EhIHxH47+4gyRALZxXJUz32rN+uwgkC3w==
X-Received: by 2002:a19:4358:0:b0:521:b42f:2674 with SMTP id 2adb3069b0e04-52210277c35mr145762e87.63.1715275788076;
        Thu, 09 May 2024 10:29:48 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
Date: Fri,  3 May 2024 03:44:42 +0200
Message-Id: <20240503014449.1046238-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 54 ++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE     (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
     /* For most cases (>99.9%), the page address is the same. */
     MapCacheEntry *last_entry;
     unsigned long max_mcache_size;
-    unsigned int mcache_bucket_shift;
+    unsigned int bucket_shift;
+    unsigned long bucket_size;
 
     phys_offset_to_gaddr_t phys_offset_to_gaddr;
     QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
                                            void *opaque,
+                                           unsigned int bucket_shift,
                                            unsigned long max_size)
 {
     unsigned long size;
     MapCache *mc;
 
+    assert(bucket_shift >= XC_PAGE_SHIFT);
+
     mc = g_new0(MapCache, 1);
 
     mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
     QTAILQ_INIT(&mc->locked_entries);
 
+    mc->bucket_shift = bucket_shift;
+    mc->bucket_size = 1UL << bucket_shift;
     mc->max_mcache_size = max_size;
 
     mc->nr_buckets =
         (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
-         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+          (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+         (bucket_shift - XC_PAGE_SHIFT));
 
     size = mc->nr_buckets * sizeof(MapCacheEntry);
     size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
 {
     struct rlimit rlimit_as;
     unsigned long max_mcache_size;
+    unsigned int bucket_shift;
+
+    if (HOST_LONG_BITS == 32) {
+        bucket_shift = 16;
+    } else {
+        bucket_shift = 20;
+    }
 
     if (geteuid() == 0) {
         rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
         }
     }
 
-    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+    mapcache = xen_map_cache_init_single(f, opaque,
+                                         bucket_shift,
+                                         max_mcache_size);
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
     entry->valid_mapping = NULL;
 
     for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
     }
 
     /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
     bool dummy = false;
 
 tryagain:
-    address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = phys_addr >> mc->bucket_shift;
+    address_offset = phys_addr & (mc->bucket_size - 1);
 
     trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
         return mc->last_entry->vaddr_base + address_offset;
     }
 
-    /* size is always a multiple of MCACHE_BUCKET_SIZE */
+    /* size is always a multiple of mc->bucket_size */
     if (size) {
         cache_size = size + address_offset;
-        if (cache_size % MCACHE_BUCKET_SIZE) {
-            cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+        if (cache_size % mc->bucket_size) {
+            cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
         }
     } else {
-        cache_size = MCACHE_BUCKET_SIZE;
+        cache_size = mc->bucket_size;
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
         trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
         raddr = RAM_ADDR_INVALID;
     } else {
-        raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+        raddr = (reventry->paddr_index << mc->bucket_shift) +
              ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
     }
     mapcache_unlock(mc);
@@ -585,8 +597,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     hwaddr address_index, address_offset;
     hwaddr test_bit_size, cache_size = size;
 
-    address_index  = old_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = old_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = old_phys_addr >> mc->bucket_shift;
+    address_offset = old_phys_addr & (mc->bucket_size - 1);
 
     assert(size);
     /* test_bit_size is always a multiple of XC_PAGE_SIZE */
@@ -595,8 +607,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         test_bit_size += XC_PAGE_SIZE - (test_bit_size % XC_PAGE_SIZE);
     }
     cache_size = size + address_offset;
-    if (cache_size % MCACHE_BUCKET_SIZE) {
-        cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+    if (cache_size % mc->bucket_size) {
+        cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -609,8 +621,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
-    address_index  = new_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = new_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = new_phys_addr >> mc->bucket_shift;
+    address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 17:30:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:30:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719346.1122106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57ad-0000L4-I0; Thu, 09 May 2024 17:29:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719346.1122106; Thu, 09 May 2024 17:29:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57ad-0000Kx-EK; Thu, 09 May 2024 17:29:55 +0000
Received: by outflank-mailman (input) for mailman id 719346;
 Thu, 09 May 2024 17:29:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QLvV=MM=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s57ab-0008WC-Gv
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:29:53 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id be241a25-0e29-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 19:29:51 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e34e85ebf4so10422421fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:29:51 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0ef229csm2555471fa.66.2024.05.09.10.29.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:29:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be241a25-0e29-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715275791; x=1715880591; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8CWrR8s0gYDS+czdwwl88oUb1Zx/cHo9QTV6MX+PdaQ=;
        b=PuaZxYbh4dl/k+kRVrw4kzEzL7R/4qBmasKmm76uPOIncuiut2RFZf7lflsADVIVsB
         SaTosd8Vsn7VFTZk/e8aVWYApbSRPAQozer5yS2XcDPdEl1vsk3w7PrqFIR/qEnJZoCv
         yaouowfMbdNOajLJsZXiDslCnGw5yn3mEjMLOBvNq7glRAVtg8w659yNHGWvR1aPqXiy
         Uio6i1I3cwzCsXscFobRyHqk9sO19RH/qK3FhrhBJUqnG7oX80tH76TUDTg7jNz06X5n
         BbcXxkNkngr4U237Omw0/9ng0oprG6TsLszr8PKqVei5tdlubq4RI1dqWegwyQ8SfkxI
         8yJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715275791; x=1715880591;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8CWrR8s0gYDS+czdwwl88oUb1Zx/cHo9QTV6MX+PdaQ=;
        b=fhZYJ55ZiLTSSXCsvMjp4V9Jm1AB1xj9xofq/lH3ROLOWHnHU9sY5SoTa9LQOBj/MN
         xjjmJmyHC6OFujz+yJfIFPTeZOWPfHKj/mOMorLXDze/ZCLJSEQcuxt8tD4tfvzrdiRY
         UsqurTc6wsX80rhyACDipanFagin67DaScQ97jEqFvXtlBmYBcyTHcAI/sJi3+l3ho2C
         2XIl5CXwhkiKUXDNTeq2kFFoiNDtK1GbjK8SPBOvS1JJRxa8SQetE27LIQJxrPsPLdst
         Qhl8ozjSp6WakvYAIizMb478SzsNQmP9+Sbhp02OANTZDefpeTy2W+KZK4vLmRcYeRrt
         raZw==
X-Forwarded-Encrypted: i=1; AJvYcCW4brdFwiJHJtaoWyYV1Ul6i3CprM5vMhm1s3AkAnDBU9Od8K3XEIc7OoCHJKhSXcLMrZxjFjSsdb+4cI7y1L8WXXBiuXWpW1tX/RJHhas=
X-Gm-Message-State: AOJu0Ywl9BeN31qCpPz/bNiExqHt+gHc9R17/AMMKMOGoxLKL1MlNpsG
	niEB8b0lgAIOO7MH0KENycxnTFfwBd1CjsvD2fzhS9xkwF31Ou6k
X-Google-Smtp-Source: AGHT+IEqPcOqKJgEtzLBhZogEkbqeGe1tWfxm6fA+Z6yvRB5yQwN2WVk6iss6MyCzgHI/1kn9iQ+GQ==
X-Received: by 2002:a2e:be13:0:b0:2de:25e2:6187 with SMTP id 38308e7fff4ca-2e51fe57b06mr778381fa.23.1715275791414;
        Thu, 09 May 2024 10:29:51 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 3/8] xen: Add xen_mr_is_memory()
Date: Fri,  3 May 2024 03:44:44 +0200
Message-Id: <20240503014449.1046238-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
---
 hw/xen/xen-hvm-common.c | 10 ++++++++--
 include/sysemu/xen.h    |  8 ++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 1627da7398..c94f1990c5 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    return mr == &xen_memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
                    Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
         return;
     }
 
-    if (mr == &xen_memory) {
+    if (xen_mr_is_memory(mr)) {
         return;
     }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
     XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-    if (section->mr == &xen_memory) {
+    if (xen_mr_is_memory(section->mr)) {
         return;
     } else {
         if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
     g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 17:30:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:30:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719345.1122096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57ac-00004u-8G; Thu, 09 May 2024 17:29:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719345.1122096; Thu, 09 May 2024 17:29:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57ac-0008WT-4H; Thu, 09 May 2024 17:29:54 +0000
Received: by outflank-mailman (input) for mailman id 719345;
 Thu, 09 May 2024 17:29:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QLvV=MM=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s57aa-0008WC-Rl
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:29:52 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd582202-0e29-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 19:29:50 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51fa75d54a4so1073217e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:29:50 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d3445sm369206e87.176.2024.05.09.10.29.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:29:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd582202-0e29-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715275790; x=1715880590; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=aQPuzg28DBNabnU97KSBvxy03U0QOnNHPs9bhMEg2LFL8rJPouWJh6MAzqyKrOp9Lf
         GrTp+x9/GXvS64i31INkqD0VZlzgAIQwhe6PFIhI+me2XDBuY+OhsYGeUkIRTtSm7Wp5
         lY8klhHUDcgKguAnAEVqRsX4ql28WBaV+RBBpwZSNo/ZThHxTzajDgnSYHeRP/xmZSoA
         7HpBEEwTPk/uAQ6+CRZnxw+bv9+1CQr9cu/BHLHX650ZDSAARYSY476MT9+H4x2K6W/X
         8M6BmDuLwLRGcWavejxeK4uCrolFD512rCinkyKaW0tCukFKhOkO7vCZkJHaR7TJ3hCu
         1qWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715275790; x=1715880590;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=lMSWSLQDsZfoDjB49splZp0NBvGMKit9cl3qq2UFV6ikLit41LPJR0Gpph7qKhaLfO
         06Dm4BDKPsxo8aXE1o9SKE7TT3skYVpR8jgxPr3udyto9m2hDUSeIgFb3va1WSGccz+9
         rzA3TX5aXnNMcMUTFynAvP+iiaQaLEiChOYhsokuhhtkAYuCQev2tHy32wDx38E4vvFH
         /n0qq0RJn2rs+4YAb468vfoPLfiBCtFlsh030FPvttqnjWAy+0xi66/56LLB58xRWHFu
         FMdJKjsX7HKScStaEKw4iBKTOj0ixfBzo3UBFjQ89v3hulbfh8ucu2WThzUuglm8nWdf
         uglw==
X-Forwarded-Encrypted: i=1; AJvYcCX05g2umvrbV3n8QEfHLA+McmTfGPwXzgkHEtZk37AipmeefB9OXiec5bR78xU5DQZWGlCSyLfy9mk7fDPeDc3jAVrmm00WMHGwAJeQvhc=
X-Gm-Message-State: AOJu0YxK+fdHAtMTwOUpswQLocbnq/I42955vMrDuOvqsWzmNgNDQJK5
	y2jbtBA45THYCQMxQReMwAxHev+SfcJlJu3rU4eBz2FlZDq5l/Bq
X-Google-Smtp-Source: AGHT+IE3429+Ctid+D4EtDpgneRZV++HVd+uIfkCNO9kCmImtP+ysIxcQpHU7yrc/kJeI1cp4rrEpg==
X-Received: by 2002:a05:6512:4027:b0:51d:866d:8cf2 with SMTP id 2adb3069b0e04-5220e27a334mr125318e87.6.1715275790078;
        Thu, 09 May 2024 10:29:50 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 2/8] xen: mapcache: Unmap first entries in buckets
Date: Fri,  3 May 2024 03:44:43 +0200
Message-Id: <20240503014449.1046238-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
         return;
     }
     entry->lock--;
-    if (entry->lock > 0 || pentry == NULL) {
+    if (entry->lock > 0) {
         return;
     }
 
-    pentry->next = entry->next;
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
     if (munmap(entry->vaddr_base, entry->size) != 0) {
         perror("unmap fails");
         exit(-1);
     }
+
     g_free(entry->valid_mapping);
-    g_free(entry);
+    if (pentry) {
+        pentry->next = entry->next;
+        g_free(entry);
+    } else {
+        memset(entry, 0, sizeof *entry);
+    }
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 17:30:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:30:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719348.1122126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57ak-0000vi-0f; Thu, 09 May 2024 17:30:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719348.1122126; Thu, 09 May 2024 17:30:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57aj-0000vO-Tx; Thu, 09 May 2024 17:30:01 +0000
Received: by outflank-mailman (input) for mailman id 719348;
 Thu, 09 May 2024 17:30:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QLvV=MM=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s57ai-0008WC-CG
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:30:00 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c1f8af77-0e29-11ef-b4bb-af5377834399;
 Thu, 09 May 2024 19:29:58 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51ef64d051bso1304131e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:29:58 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f39d33ccsm368544e87.293.2024.05.09.10.29.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:29:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1f8af77-0e29-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715275798; x=1715880598; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PjxS9Se2SHC9m/lZ6uD/ahAlHp9nRH2Q1MtnYySD9v0=;
        b=W9d0mG0BaWt7TCuX8g8QYV4JmqXip+11ZS62UaPKXkT24DoWqgcSzTBO0lXDBW/WET
         qOxbcabEohqXs3xt8wZzyGmIU+NCvWNO2z7WaTMxSPaIZ2l01Oy+eTq4wlbplkTdTKFz
         ejkI2/arJ7xEU5oo6zad/Qd7pyiJT0D2J8mqE3TPMOdKmQLleHccnoeQV8+aA+Wo5XFv
         3AhEyo+7W6HNCFzk4b1rgfB1VfY/YfPKH9Tmlh69pubBkB9JpXQHoh09QjsLQzR/ZGSg
         m+NYnq9icsBk2Jsd9CisOTlcartj2CRhtN76HaM2O5uLvux1NTdy5f9r4mia1JpaiwAq
         GGmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715275798; x=1715880598;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PjxS9Se2SHC9m/lZ6uD/ahAlHp9nRH2Q1MtnYySD9v0=;
        b=utICSxUibsX27aY5TtuuqaQG40aWZzSOELz2iTDctbTX/kitklFTrOXqdLdxHwjZmU
         CjpK+8exoYRiTFbRuxlpqMS2VPzUOcJkV6PntU+OczTEbtsIx/uY78+CmQE4NIY6xBcR
         EDLU/Zsh52p9Ytvc52aBPiVZuALSIixNWnL02TnpBfls7fRGr16Zybv1tMPCtyZI3zjX
         T9GEA6giKxVkAwFZuT0HYol2BMPEya3D9fw1r+dakFw7XAB8S4HNF/rq4hMHtHxG3fnM
         cTSfoe/E4pYwyp8zWTkPsv+F1ijMFYwdz3b0uD/EJbvuQkGEsn4qXSLsedi8EJ9FnGIm
         vosw==
X-Forwarded-Encrypted: i=1; AJvYcCUQhc5UN/G5/Q7Wwwhk21k/LLSDt3tbIpTNg/3xM9sGmHZoH/wfHhJEmcYBlYH2mC11y5eJm3qK2/TnknC+3dMGxXDoevim7jzyJ8yfnC0=
X-Gm-Message-State: AOJu0YyvNj4aYe9cyOSpKnUcWiU4btFynbcb9dBDSCxuEP8UWzLuZH7N
	dBz3OvPpQ6DqqGD9z2AS3q2qtWEhaNHj9e0AOdINYyiDRolf4qrZ
X-Google-Smtp-Source: AGHT+IGRcD9VdYxUVdFG81QPXGUJhDXoOaVerBAJyHXQpyPwC5eKutgVITtAb83XqX4DbciXEEMQrg==
X-Received: by 2002:a05:6512:3091:b0:51e:e703:d11c with SMTP id 2adb3069b0e04-5220fa71891mr166903e87.12.1715275797539;
        Thu, 09 May 2024 10:29:57 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 7/8] xen: mapcache: Add support for grant mappings
Date: Fri,  3 May 2024 03:44:48 +0200
Message-Id: <20240503014449.1046238-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 hw/xen/xen-hvm-common.c         |  12 ++-
 hw/xen/xen-mapcache.c           | 163 ++++++++++++++++++++++++++------
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h            |   7 ++
 4 files changed, 152 insertions(+), 33 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index c94f1990c5..7a1e2ce4b3 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-    return mr == &xen_memory;
+    return mr == &xen_memory || mr == &xen_grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    return mr == &xen_grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 26bc38a9e3..25041ab02d 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include <sys/resource.h>
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include <xenevtchn.h>
+#include <xengnttab.h>
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
     unsigned long *valid_mapping;
     uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
     uint8_t flags;
     hwaddr size;
     struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     unsigned long max_mcache_size;
     unsigned int bucket_shift;
 
+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
+    if (xen_region_gnttabdev == NULL) {
+        error_report("mapcache: Failed to open gnttab device");
+        exit(EXIT_FAILURE);
+    }
+
     if (HOST_LONG_BITS == 32) {
         bucket_shift = 16;
     } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     mapcache = xen_map_cache_init_single(f, opaque,
                                          bucket_shift,
                                          max_mcache_size);
+
+    /*
+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+     * map anything beyond the number of pages granted to us.
+     */
+    mapcache_grants = xen_map_cache_init_single(f, opaque,
+                                                XC_PAGE_SHIFT,
+                                                max_mcache_size);
+
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
                              hwaddr size,
                              hwaddr address_index,
                              bool dummy,
+                             bool grant,
+                             bool is_write,
                              ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
-    xen_pfn_t *pfns;
+    uint32_t *refs = NULL;
+    xen_pfn_t *pfns = NULL;
     int *err;
     unsigned int i;
     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
     trace_xen_remap_bucket(address_index);
 
-    pfns = g_new0(xen_pfn_t, nb_pfn);
+    if (grant) {
+        refs = g_new0(uint32_t, nb_pfn);
+    } else {
+        pfns = g_new0(xen_pfn_t, nb_pfn);
+    }
     err = g_new0(int, nb_pfn);
 
     if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
     g_free(entry->valid_mapping);
     entry->valid_mapping = NULL;
 
-    for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+    if (grant) {
+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+        for (i = 0; i < nb_pfn; i++) {
+            refs[i] = grant_base + i;
+        }
+    } else {
+        for (i = 0; i < nb_pfn; i++) {
+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+        }
     }
 
-    /*
-     * If the caller has requested the mapping at a specific address use
-     * MAP_FIXED to make sure it's honored.
-     */
+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
+
     if (!dummy) {
-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
-                                           PROT_READ | PROT_WRITE,
-                                           vaddr ? MAP_FIXED : 0,
-                                           nb_pfn, pfns, err);
+        if (grant) {
+            int prot = PROT_READ;
+
+            if (is_write) {
+                prot |= PROT_WRITE;
+            }
+
+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
+            assert(vaddr == NULL);
+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
+                                                         nb_pfn,
+                                                         xen_domid, refs,
+                                                         prot);
+        } else {
+            /*
+             * If the caller has requested the mapping at a specific address use
+             * MAP_FIXED to make sure it's honored.
+             *
+             * We don't yet support upgrading mappings from RO to RW, to handle
+             * models using ordinary address_space_rw(), foreign mappings ignore
+             * is_write and are always mapped RW.
+             */
+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
+                                               PROT_READ | PROT_WRITE,
+                                               vaddr ? MAP_FIXED : 0,
+                                               nb_pfn, pfns, err);
+        }
         if (vaddr_base == NULL) {
-            perror("xenforeignmemory_map2");
+            perror(grant ? "xengnttab_map_domain_grant_refs"
+                           : "xenforeignmemory_map2");
             exit(-1);
         }
     } else {
@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,
         }
     }
 
+    g_free(refs);
     g_free(pfns);
     g_free(err);
 }
@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
                                        ram_addr_t ram_offset,
-                                       uint8_t lock, bool dma, bool is_write)
+                                       uint8_t lock, bool dma,
+                                       bool grant, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
                   *free_entry = NULL, *free_pentry = NULL;
@@ -340,7 +400,7 @@ tryagain:
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                         ram_offset);
+                         grant, is_write, ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
@@ -348,7 +408,7 @@ tryagain:
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                             ram_offset);
+                             grant, is_write, ram_offset);
         }
     }
 
@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
+    bool grant = xen_mr_is_grants(mr);
+    MapCache *mc = grant ? mapcache_grants : mapcache;
     uint8_t *p;
 
-    mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
-                               lock, dma, is_write);
-    mapcache_unlock(mapcache);
+    if (grant) {
+        /*
+         * Grants are only supported via address_space_map(). Anything
+         * else is considered a user/guest error.
+         *
+         * QEMU generally doesn't expect these mappings to ever fail, so
+         * if this happens we report an error message and abort().
+         */
+        if (!lock) {
+            error_report("Trying access a grant reference without mapping it.");
+            abort();
+        }
+    }
+
+    mapcache_lock(mc);
+    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
+                               lock, dma, grant, is_write);
+    mapcache_unlock(mc);
     return p;
 }
 
@@ -449,7 +525,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 {
-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    ram_addr_t addr;
+
+    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    if (addr == RAM_ADDR_INVALID) {
+        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
+    }
+
+    return addr;
 }
 
 static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
@@ -460,6 +543,7 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     hwaddr paddr_index;
     hwaddr size;
     int found = 0;
+    int rc;
 
     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
         if (reventry->vaddr_req == buffer) {
@@ -502,7 +586,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     }
 
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
-    if (munmap(entry->vaddr_base, entry->size) != 0) {
+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
+        rc = xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
+                             entry->size >> mc->bucket_shift);
+    } else {
+        rc = munmap(entry->vaddr_base, entry->size);
+    }
+
+    if (rc) {
         perror("unmap fails");
         exit(-1);
     }
@@ -521,14 +612,24 @@ typedef struct XenMapCacheData {
     uint8_t *buffer;
 } XenMapCacheData;
 
+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
+{
+    mapcache_lock(mc);
+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
+    mapcache_unlock(mc);
+}
+
+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
+{
+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
+}
+
 static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
     XenMapCacheData *data = opaque;
 
-    mapcache_lock(mapcache);
-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
-    mapcache_unlock(mapcache);
-
+    xen_invalidate_map_cache_entry_all(data->buffer);
     aio_co_wake(data->co);
 }
 
@@ -543,9 +644,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
                                 xen_invalidate_map_cache_entry_bh, &data);
         qemu_coroutine_yield();
     } else {
-        mapcache_lock(mapcache);
-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
-        mapcache_unlock(mapcache);
+        xen_invalidate_map_cache_entry_all(buffer);
     }
 }
 
@@ -597,6 +696,7 @@ void xen_invalidate_map_cache(void)
     bdrv_drain_all();
 
     xen_invalidate_map_cache_single(mapcache);
+    xen_invalidate_map_cache_single(mapcache_grants);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
@@ -632,13 +732,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) == 0);
+
     address_index  = new_phys_addr >> mc->bucket_shift;
     address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false, new_phys_addr);
+                     cache_size, address_index, false,
+                     false, false, new_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 65a51aac2e..3d796235dc 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -16,6 +16,7 @@
 #include <xen/hvm/ioreq.h>
 
 extern MemoryRegion xen_memory;
+extern MemoryRegion xen_grants;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
 
@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
     do { } while (0)
 #endif
 
+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
+
 static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
 {
     return shared_page->vcpu_ioreq[i].vp_eport;
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index dc72f83bcb..19dccf4d71 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
 bool xen_mr_is_memory(MemoryRegion *mr);
+bool xen_mr_is_grants(MemoryRegion *mr);
 
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
     return false;
 }
 
+static inline bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 09 17:51:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 17:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719366.1122135 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57v4-0006YK-Rm; Thu, 09 May 2024 17:51:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719366.1122135; Thu, 09 May 2024 17:51:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s57v4-0006YC-PJ; Thu, 09 May 2024 17:51:02 +0000
Received: by outflank-mailman (input) for mailman id 719366;
 Thu, 09 May 2024 17:51:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1XlW=MM=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s57v3-0006Y5-7c
 for xen-devel@lists.xenproject.org; Thu, 09 May 2024 17:51:01 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b1f47f4d-0e2c-11ef-909c-e314d9c70b13;
 Thu, 09 May 2024 19:51:00 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so290850366b.3
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 10:51:00 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b177fcsm96102066b.202.2024.05.09.10.50.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 May 2024 10:50:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1f47f4d-0e2c-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715277059; x=1715881859; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0pNAOzdrTXcAGJ1Ot2q3kGu92saKCNjFwk0wxdWZLiI=;
        b=Cd51WdbHqU5ZJ/jdqExPsi3tWEGu4dv91kx567rqg9I20f+Vgnx9RQOBA1HGhFABwu
         vU/U2UySXcIWTe49ijE+Ptz3kf9ohwLty45jUGpahW3ADELRK4U+ZhwSpmWHQZff53UL
         JEACzStF/7sOaKqrPrYJefRb6hgndf4E4LLcs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715277059; x=1715881859;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0pNAOzdrTXcAGJ1Ot2q3kGu92saKCNjFwk0wxdWZLiI=;
        b=cPQ2Z+ld6NAyne/Y0CanJ9f495v0SgPrivZqAxzeGV/WXN0kk57QWTghiIrp8yqLuM
         jW6Wicimd0xHGALuQJqXoro+5MIy/b4vrBp3kWJGgR8BER2m2DUraFRjvIZmCamWHbgO
         9atKGXi5FKgDQsr88EZvWsh/MoTL0pY3jgWonGbhQcppt3POsa5/7S0BOtJzFGo2nllR
         8+b7iqd2Z5YfjxaJRPoDF8LB1KBcEQ+suh1me3e7fbLjvBdjINKsfaBruqg6g8PMwmlO
         bIiX+XYOcu3ZrKbsPoaG0gQdzwUKbEvOqjHKpApisrzkHQ756XUpXkcEurlNrLJGUP9C
         KGeg==
X-Gm-Message-State: AOJu0YyG4kTOFkUOtkEEZlkLszX8B8PpzLllEc0kFkBCTInLJJOwjqFr
	Bb+9eR4TUV1OQdWUanNZUm45VSJl0LVqbTuOyK5Vr6iliTngNq+EhLm+VRG0EncSdJlFBcJ6QV3
	n
X-Google-Smtp-Source: AGHT+IH5MZCYaDFzD4zge9TBvPnXZMBYvVb3Z8Al0dhmacUqE8oCSW6JhmyPzMctMLDfgx5W4ubDPA==
X-Received: by 2002:a17:907:36c:b0:a59:c6fd:5160 with SMTP id a640c23a62f3a-a5a2d6bc140mr22992166b.76.1715277059271;
        Thu, 09 May 2024 10:50:59 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Alejandro Vallejo <alejandro.vallejo@cloud.com>
Subject: [PATCH 4.5/8] tools/hvmloader: Further simplify SMP setup
Date: Thu,  9 May 2024 18:50:57 +0100
Message-Id: <20240509175057.1921538-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
References: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Now that we're using hypercalls to start APs, we can replace the 'ap_cpuid'
global with a regular function parameter.  This requires telling the compiler
that we'd like the parameter in a register rather than on the stack.

While adjusting, rename to cpu_setup().  It's always been used on the BSP,
making the name ap_start() specifically misleading.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Alejandro Vallejo <alejandro.vallejo@cloud.com>

This is a trick I found for XTF, not that I've completed the SMP support yet.

I realise it's cheating slightly WRT 4.19, but it came out of the middle of a
series targetted for 4.19.
---
 tools/firmware/hvmloader/smp.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
index 6ebf0b60faab..5d46eee1c5f4 100644
--- a/tools/firmware/hvmloader/smp.c
+++ b/tools/firmware/hvmloader/smp.c
@@ -29,15 +29,15 @@
 
 #include <xen/vcpu.h>
 
-static int ap_callin, ap_cpuid;
+static int ap_callin;
 
-static void ap_start(void)
+static void __attribute__((regparm(1))) cpu_setup(unsigned int cpu)
 {
-    printf(" - CPU%d ... ", ap_cpuid);
+    printf(" - CPU%d ... ", cpu);
     cacheattr_init();
     printf("done.\n");
 
-    if ( !ap_cpuid ) /* Used on the BSP too */
+    if ( !cpu ) /* Used on the BSP too */
         return;
 
     wmb();
@@ -55,7 +55,6 @@ static void boot_cpu(unsigned int cpu)
     static struct vcpu_hvm_context ap;
 
     /* Initialise shared variables. */
-    ap_cpuid = cpu;
     ap_callin = 0;
     wmb();
 
@@ -63,9 +62,11 @@ static void boot_cpu(unsigned int cpu)
     ap = (struct vcpu_hvm_context) {
         .mode = VCPU_HVM_MODE_32B,
         .cpu_regs.x86_32 = {
-            .eip = (unsigned long)ap_start,
+            .eip = (unsigned long)cpu_setup,
             .esp = (unsigned long)ap_stack + ARRAY_SIZE(ap_stack),
 
+            .eax = cpu,
+
             /* Protected Mode, no paging. */
             .cr0 = X86_CR0_PE,
 
@@ -105,7 +106,7 @@ void smp_initialise(void)
     unsigned int i, nr_cpus = hvm_info->nr_vcpus;
 
     printf("Multiprocessor initialisation:\n");
-    ap_start();
+    cpu_setup(0);
     for ( i = 1; i < nr_cpus; i++ )
         boot_cpu(i);
 }

base-commit: 53959cb8309919fc2f157a1c99e0af2ce280cb84
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 09 21:43:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 21:43:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719395.1122152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5BXW-0002nh-QK; Thu, 09 May 2024 21:42:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719395.1122152; Thu, 09 May 2024 21:42:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5BXW-0002na-Ks; Thu, 09 May 2024 21:42:58 +0000
Received: by outflank-mailman (input) for mailman id 719395;
 Thu, 09 May 2024 21:42:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5BXV-0002nP-Ij; Thu, 09 May 2024 21:42:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5BXV-0003K9-H3; Thu, 09 May 2024 21:42:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5BXV-0007ne-1Y; Thu, 09 May 2024 21:42:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5BXV-0004q1-0f; Thu, 09 May 2024 21:42:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uNe2POFsIvVRF2Lo05pNNK8tfjdruGlMCxYU4SvMZ5Q=; b=Sqtw0qYG6kMRpp9pWABmmdxuAc
	aiSxGeG/g2dT0sjNlo8cKip6c2nhGRWjJIyk/OZ5iymmKfK767jvEHH6yZioyRxfhLFpjCMQLnlRz
	ghxr5u+KEfWwm9Q0kJBF+o+87JI+KSqIZqKQCyqx03b0z4SexYSxZXKYf0VtGIXPqIwk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185958-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185958: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=53959cb8309919fc2f157a1c99e0af2ce280cb84
X-Osstest-Versions-That:
    xen=e8e8afee990ae74a4d2797ddae1b58a4e03912b5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 21:42:57 +0000

flight 185958 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185958/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  53959cb8309919fc2f157a1c99e0af2ce280cb84
baseline version:
 xen                  e8e8afee990ae74a4d2797ddae1b58a4e03912b5

Last test of basis   185954  2024-05-09 09:04:07 Z    0 days
Testing same since   185958  2024-05-09 18:02:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Federico Serafini <federico.serafini@bugseng.com>
  Jan Beulich <jbeulich@suse.com>
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e8e8afee99..53959cb830  53959cb8309919fc2f157a1c99e0af2ce280cb84 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 09 22:20:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 May 2024 22:20:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719405.1122160 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5C7N-0007Fw-Fu; Thu, 09 May 2024 22:20:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719405.1122160; Thu, 09 May 2024 22:20:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5C7N-0007FC-BJ; Thu, 09 May 2024 22:20:01 +0000
Received: by outflank-mailman (input) for mailman id 719405;
 Thu, 09 May 2024 22:20:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5C7M-0007CV-6Q; Thu, 09 May 2024 22:20:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5C7M-0004CQ-2j; Thu, 09 May 2024 22:20:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5C7L-0000WQ-OD; Thu, 09 May 2024 22:19:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5C7L-0006PW-Nc; Thu, 09 May 2024 22:19:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fQo4xmTfmvXak6ZjcEvW5K1nviEV8Rq5t0Y2P92YAA4=; b=EDfFLawGRTBSPiGZOcmkLI3Mjc
	xfqZbSHkM8g4eCzIBd++/y1lLcrrvgZo1pnkYVTPQPk4kXP27Dg3cr1C6+AJ043XqsidD5tNxJnnJ
	fQdNtsv1cBhzKwTuFf5TFCh8OKkHr6SG/+Vog70hGAysulH0wBDlxb+oA/5TmdDNW2v4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185955-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185955: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e8e8afee990ae74a4d2797ddae1b58a4e03912b5
X-Osstest-Versions-That:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 May 2024 22:19:59 +0000

flight 185955 xen-unstable real [real]
flight 185959 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185955/
http://logs.test-lab.xenproject.org/osstest/logs/185959/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt    18 guest-start/debian.repeat fail REGR. vs. 185950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185950
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185950
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  e8e8afee990ae74a4d2797ddae1b58a4e03912b5
baseline version:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b

Last test of basis   185950  2024-05-09 01:51:55 Z    0 days
Testing same since   185955  2024-05-09 14:10:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e8e8afee990ae74a4d2797ddae1b58a4e03912b5
Author: George Dunlap <george.dunlap@cloud.com>
Date:   Thu Apr 25 09:49:42 2024 +0100

    svm: Fix MISRA 8.2 violation
    
    Misra 8.2 requires named parameters in prototypes.  Use the name from
    the implementaiton.
    
    Fixes: 0d19d3aab0 ("svm/nestedsvm: Introduce nested capabilities bit")
    Reported-by: Andrew Cooper <andrew.cooper@cloud.com>
    Reported-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
    Signed-off-by: George Dunlap <george.dunlap@cloud.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Fri May 10 02:16:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 02:16:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719441.1122189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5FoI-0002mo-9w; Fri, 10 May 2024 02:16:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719441.1122189; Fri, 10 May 2024 02:16:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5FoI-0002mh-6M; Fri, 10 May 2024 02:16:34 +0000
Received: by outflank-mailman (input) for mailman id 719441;
 Fri, 10 May 2024 02:16:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5FoH-0002mX-7k; Fri, 10 May 2024 02:16:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5FoH-0001LN-5b; Fri, 10 May 2024 02:16:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5FoG-0001Tf-Mm; Fri, 10 May 2024 02:16:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5FoG-0006qG-MH; Fri, 10 May 2024 02:16:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FQqjTeyRJGY0RlZKxFLZcbOV+2LKBFBn2nYtoFUdWCk=; b=Q7krp8p0DkdSQTPTwVOwDP+7hr
	sUxUF+KtI7UPS8cl6b1Sp5KlZDECng3td3kcaXbudP8pNPeODBmJGBgqKKfsEAiCNC2n6f3uHePjN
	ay3pkmnMmL/v1MyRiWLFJ8SvDHfSgKPLs14Byf3baAKK04424htggbKfqCVQL0TpxzpU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185957-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185957: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8c3b7565f81e030ef448378acd1b35dabb493e3b
X-Osstest-Versions-That:
    linux=45db3ab70092637967967bfd8e6144017638563c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 02:16:32 +0000

flight 185957 linux-linus real [real]
flight 185962 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185957/
http://logs.test-lab.xenproject.org/osstest/logs/185962/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185962-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185949
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185949
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185949
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185949
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 185949
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185949
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185949
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                8c3b7565f81e030ef448378acd1b35dabb493e3b
baseline version:
 linux                45db3ab70092637967967bfd8e6144017638563c

Last test of basis   185949  2024-05-09 01:43:03 Z    1 days
Testing same since   185957  2024-05-09 16:10:43 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Andrey Ryabinin <ryabinin.a.a@gmail.com>
  Antony Antony <antony.antony@secunet.com>
  Boy Wu <boy.wu@mediatek.com>
  Boy.Wu <boy.wu@mediatek.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Daniel Golle <daniel@makrotopia.org>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  Dimitri Fedrau <dima.fedrau@gmail.com>
  Donald Hunter <donald.hunter@gmail.com>
  Doug Brown <doug@schmorgal.com>
  Douglas Anderson <dianders@chromium.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Eric Dumazet <edumazet@google.com>
  Felix Fietkau <nbd@nbd.name>
  Geliang Tang <geliang@kernel.org>
  Gregor Herburger <gregor.herburger@ew.tq-group.com>
  Gregory Detal <gregory.detal@gmail.com>
  Ido Schimmel <idosch@nvidia.com>
  Jakub Kicinski <kuba@kernel.org>
  Jian Shen <shenjian15@huawei.com>
  Jijie Shao <shaojijie@huawei.com>
  Johan Hovold <johan+linaro@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
  Lukasz Majewski <lukma@denx.de>
  Marek Vasut <marex@denx.de>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Neal Cardwell <ncardwell@google.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Paolo Abeni <pabeni@redhat.com>
  Paul Davey <paul.davey@alliedtelesis.co.nz>
  Peiyang Wang <wangpeiyang1@huawei.com>
  Potnuri Bharat Teja <bharat@chelsio.com>
  Roded Zats <rzats@paloaltonetworks.com>
  Ronald Wahl <ronald.wahl@raritan.com> # KS8851 SPI
  Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
  Rémi Denis-Courmont <courmisch@gmail.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Steffen Bätz <steffen@innosonix.de>
  Steffen Klassert <steffen.klassert@secunet.com>
  Sungwoo Kim <iam@sung-woo.kim>
  Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
  Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
  Vincent Duvert <vincent.ldev@duvert.net>
  Wen Gu <guwen@linux.alibaba.com>
  Yonglong Liu <liuyonglong@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   45db3ab70092..8c3b7565f81e  8c3b7565f81e030ef448378acd1b35dabb493e3b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:00:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:00:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719452.1122200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5GUM-00087i-Gv; Fri, 10 May 2024 03:00:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719452.1122200; Fri, 10 May 2024 03:00:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5GUM-00087D-CE; Fri, 10 May 2024 03:00:02 +0000
Received: by outflank-mailman (input) for mailman id 719452;
 Fri, 10 May 2024 03:00:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5GUL-0007wn-A1; Fri, 10 May 2024 03:00:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5GUL-00020l-51; Fri, 10 May 2024 03:00:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5GUK-0002jn-PQ; Fri, 10 May 2024 03:00:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5GUK-00074H-Ox; Fri, 10 May 2024 03:00:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eDXRAGAY1EUNLjgTogj+K3rybJjqDZlARxSJva2FpHc=; b=NyyPk+vbrz0jrKpD3QS3rostc6
	fhfLA7e8JUDiqYReTwbkII9jbnqHntZUQgABzx2vNFNEpDV54sN2P4kMxQpa1c6o/kQjUr1ewfRXz
	SzAT7xi+XwPtWuZeaKjPlXytpxZzTtP03u7IwKGfWU0laTkorVKLhI0tS5SI2iWXyOzw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185963-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185963: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3c0b84420f098c5a88281a6be370c2766184c99b
X-Osstest-Versions-That:
    ovmf=25996a3441f4f40fcf457c4ba39a1e88c4f75bdd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 03:00:00 +0000

flight 185963 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185963/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3c0b84420f098c5a88281a6be370c2766184c99b
baseline version:
 ovmf                 25996a3441f4f40fcf457c4ba39a1e88c4f75bdd

Last test of basis   185952  2024-05-09 07:13:15 Z    0 days
Testing same since   185963  2024-05-10 01:11:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abdul Lateef Attar <AbdulLateef.Attar@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   25996a3441..3c0b84420f  3c0b84420f098c5a88281a6be370c2766184c99b -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719466.1122246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM1-0007gN-CV; Fri, 10 May 2024 03:55:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719466.1122246; Fri, 10 May 2024 03:55:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM1-0007gE-91; Fri, 10 May 2024 03:55:29 +0000
Received: by outflank-mailman (input) for mailman id 719466;
 Fri, 10 May 2024 03:55:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HLz-00077u-8E
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:27 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 219c5a54-0e81-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 05:55:26 +0200 (CEST)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailfout.west.internal (Postfix) with ESMTP id 3CC8E1C001A1;
 Thu,  9 May 2024 23:55:24 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute4.internal (MEProxy); Thu, 09 May 2024 23:55:24 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:22 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 219c5a54-0e81-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715313323; x=1715399723; bh=hZ6VYL08E0
	Yxr0n/W2twXlHVhmqK+fKxAH0QY/uH+Wo=; b=tZJkovbMemvjdPp4zK3A6sZR2J
	V20Npezq6UOnjrEZwu8Ea7YNkKGVs+gC7ZFUIHO/nAgNkDQqM8578BfAREpADbSY
	yDz8f2iE7q1qTvaWC85+WQ2IDUlijA5nPdSprsEf9Xge8MgzBxX3BYPSeI2enXsZ
	CSoJ5HyzAErAHaiZ0C7wXnREeZFdrQUD8HkKwAdSDnix6nb/6ZUSiQesuydDAil8
	ixHvInYbL0ZlocZD7x8Z7asNxu5CTp4tkFOtCMLWrU9fCBj0w4WnxpHQ5u3ZZouB
	s83j+4KBJiZr+a4Vk8fiudkndm/kgfDK+7ILr8bTV0ncoZdeMrSqJya9qTdw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313323; x=
	1715399723; bh=hZ6VYL08E0Yxr0n/W2twXlHVhmqK+fKxAH0QY/uH+Wo=; b=N
	xyNYU8a2ezLe1UavSUvGnDz6vY3qcnI8kTm6L/haH/l0LPNELbDzOkQgzCMr4DN4
	kWFwyxRnwwniafQcY+GTjN2NQwpTbCJmrG5jk6GDUFvFv4Za/Jf48PU9iEEEYi/Z
	/wJtCS3dtat9QzD0yeEeTrCm1MmDKG90SQfjwTkecA1SLWu+VA3mhgfdaFOIhhn1
	EyHFZOQAKVeCw9a0tQvgiIUJrQjbR92UuEihYYLo7+OOMHF6pwFQv3TRkcAIZr7X
	2kjuHcdsHnDo/UW7+VtdtU3mZrYrfaQ1099V4QOS01l6rwTgeqY4vWqFZ4VTNazw
	DaowRZWDA2FfUHI2cAiUA==
X-ME-Sender: <xms:q5o9ZnyflwpGN4ZWgRVOrxPAYb3KorQeQqFoAo_FsBQgSeAjiBP1HQ>
    <xme:q5o9ZvRrHpInK7KrguMwCdVOXV3ANExIDHRmrD4rASUUIvwauYRPxI6gpvMhKXV9u
    6WWUntKOMt6sw>
X-ME-Received: <xmr:q5o9ZhWHQaBHe5G05gs0Ecw7Nnzx0UHXAnRsuU1Z05gd6HeSO5Z3_b-QedXc2Oxgfhhi_2cT7gEycc5fMzvTdNiUpj1nKHBi4nrF2-u05BpF4vSPLIA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:q5o9ZhjOgh1Ebe-VpyUJCudImQ8b-49dsx89RPClbkIhNeiexxZQkg>
    <xmx:q5o9ZpAUPyA_H6lVfTxS7cg41TXOkqxPspawR7fq6fC9nVyoiCUMHQ>
    <xmx:q5o9ZqK9-7y8Ht06pFi5fLD-qce5966Bijl4UPqxgmdwXzdO_bHeEg>
    <xmx:q5o9ZoDbhGMG0e8ZaV1-A8SUD2jcwmyCrgIeCGt_OnkGoDxQrQD9ew>
    <xmx:q5o9ZoN1TaNdrj9YVccu3jHjQ6YNWjLmhy9cvo-V_wliIEyzlptuq03p>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH v8 3/6] automation: prevent QEMU access to /dev/mem in PCI passthrough tests
Date: Fri, 10 May 2024 05:53:23 +0200
Message-ID: <38e018c5e94020ba37b5571088033cc6b648a8c9.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

/dev/mem access doesn't work in dom0 in lockdown and in stubdomain.
Simulate this environment with removing /dev/mem device node. Full test
for lockdown and stubdomain will come later, when all requirements will
be in place.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
This can be applied only after QEMU change is committed. Otherwise the
test will fail.
---
 automation/scripts/qubes-x86-64.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index d81ed7b931cf..7eabc1bd6ad4 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -163,6 +163,8 @@ ifconfig eth0 up
 ifconfig xenbr0 up
 ifconfig xenbr0 192.168.0.1
 
+# ensure QEMU wont have access /dev/mem
+rm -f /dev/mem
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
 xl create /etc/xen/domU.cfg
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719468.1122266 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM4-0008Dq-Ul; Fri, 10 May 2024 03:55:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719468.1122266; Fri, 10 May 2024 03:55:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM4-0008Dh-RW; Fri, 10 May 2024 03:55:32 +0000
Received: by outflank-mailman (input) for mailman id 719468;
 Fri, 10 May 2024 03:55:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HM3-00077u-Ju
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:31 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 249be738-0e81-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 05:55:31 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id 142351C001A1;
 Thu,  9 May 2024 23:55:29 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute6.internal (MEProxy); Thu, 09 May 2024 23:55:29 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:26 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 249be738-0e81-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to; s=fm3; t=1715313328; x=1715399728; bh=EAaNxhwX2Xjwv3rkR/3PC
	xXHxt3VIaaNGAHnw+G6qes=; b=Qp7ozR5kZrbKidnRDsTgV8GbjXNKxBB2IpDSq
	Z9fRzlWYWe+eWLTw5P+L6RRM6pj3P4kn2nqTKxfeYmTsnh5RzaNdDRB8j0ZSJgU7
	gyL6G2eB8JZevecWSI97v/XZeBTqtgmhukzEtPxWpPSR4FyzN+w6jVGz3DUQv+KS
	/nEdyUyZ3O015MTCv/eJkHnFyA+kYT/AsjWnKgguIy1ryJ7d0eJrT384jxQl1y2u
	/Nyf81h2PYKHr1ge0FruIflCBT6XVRNplZt+UFNDddZHEr+w5h11azBzgnBn3PQe
	1+/18U9xVysPRVW0W2tN13qDJn/5jBjdl9Capez6fzs/RChbQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:feedback-id:feedback-id:from:from
	:in-reply-to:in-reply-to:message-id:mime-version:references
	:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313328; x=
	1715399728; bh=EAaNxhwX2Xjwv3rkR/3PCxXHxt3VIaaNGAHnw+G6qes=; b=h
	ofNBWsoWiALZp8ATMlJ+MUvf/A9O9oANXMI6bBH6B9uOOcFgYgAqAleEJr3QZG8i
	p6OHDKy3tUq9WnUnW3MHfQaiGg3ljQfphjEFF/lKb42Bnux/cw7CniSeBxXEk3pZ
	7uyFhzJGXLaN9We22Ij5TDjts5oqU16lOmmxl0U0vLFLYu7Ur/sgnIqHaAfJw3zx
	qfKz4FIZk6xaWjOGdv5vDLtPOlOd2iWrN8CI00PnNZJyj/9FJMdAphuGZ7bbbPHq
	cNg5MP2AkYQoNiBRrg6QOckV/Xq2bGqojol/Lu9TWRzCzbwJxRNOdWSqc/ELn+i8
	Zo9NwdHDM/roGaBEYcuPw==
X-ME-Sender: <xms:sJo9Zj733zRYvk3IIeIGiS1WnVj07tm4NLR9plP-B6ZRKuXm7sCfdA>
    <xme:sJo9Zo4VnLQLiNAkMRXQRGeiUhPuBaUME9sTiImZrrl9umqj1hgYs-HJAMsAbGieb
    7xzbXr-Ki0vaQ>
X-ME-Received: <xmr:sJo9ZqcZGxy-Ttc51yOQ1pg2floNxn2YIrfgTyaLe9dB3R1rtlWezHW4TP4Ge6PKyK3rwJQUqo8zuBHVxvqNDJ1z7LuAI1imWudagu_74gfzkvT9Q_k>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgeet
    tdeigefhkefhfeegvdeuiedvtdffkeeltdeivdfgteelueeuhfetleevueegnecuffhomh
    grihhnpeigvghnrdhorhhgpdhgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigv
    pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg
    hlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:sJo9ZkJibQ1MSGzLsV045RxpFSdbFhD1g-TTMn3i529M89qd-D6KLQ>
    <xmx:sJo9ZnLm9QoMFJxnPfIyRsraf7Crz7061FdGeTfrSxzNTsb-HTlcpw>
    <xmx:sJo9ZtwmvGVppswgPxkchQesCyWS1dXD9VPFKWZwNvWvDHePeh6u8A>
    <xmx:sJo9ZjLFc2nkL993h7Tejgyi6i8y4ITrGlI7pXAxkPCuOchzrbgFvw>
    <xmx:sJo9Zv-nUSBf1i4lybesNcSeZnxIdz4zpIpUTU6i4ey998iV1vReD_Ax>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v8 5/6] [DO NOT APPLY] switch to qemu fork
Date: Fri, 10 May 2024 05:53:25 +0200
Message-ID: <2ec0e3416a21087e3855b56113cc1a80fc5dea2f.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This makes tests to use patched QEMU, to actually test the new behavior.
---
 Config.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Config.mk b/Config.mk
index a962f095ca16..5e220a1284e4 100644
--- a/Config.mk
+++ b/Config.mk
@@ -220,8 +220,8 @@ endif
 OVMF_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/ovmf.git
 OVMF_UPSTREAM_REVISION ?= ba91d0292e593df8528b66f99c1b0b14fadc8e16
 
-QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
-QEMU_UPSTREAM_REVISION ?= master
+QEMU_UPSTREAM_URL ?= https://github.com/marmarek/qemu
+QEMU_UPSTREAM_REVISION ?= origin/msix
 
 MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
 MINIOS_UPSTREAM_REVISION ?= b6a5b4d72b88e5c4faed01f5a44505de022860fc
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719465.1122232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM0-0007Gc-6V; Fri, 10 May 2024 03:55:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719465.1122232; Fri, 10 May 2024 03:55:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM0-0007G8-0T; Fri, 10 May 2024 03:55:28 +0000
Received: by outflank-mailman (input) for mailman id 719465;
 Fri, 10 May 2024 03:55:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HLy-00077u-DM
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:26 +0000
Received: from wfhigh5-smtp.messagingengine.com
 (wfhigh5-smtp.messagingengine.com [64.147.123.156])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ef14a55-0e81-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 05:55:23 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfhigh.west.internal (Postfix) with ESMTP id AD0A2180012B;
 Thu,  9 May 2024 23:55:19 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute5.internal (MEProxy); Thu, 09 May 2024 23:55:20 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:17 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ef14a55-0e81-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715313319; x=1715399719; bh=lLrdK6DtjD
	VUONkQTnGTL9QCNQzxUIg1GxNz3rQOZI8=; b=AUsaaCW7QioLA42ri0O7skyWj/
	fWkqCxJvUCjByVjhqLy1jHO5ixhSRIVXrFeV0wBHI16tzkoCvIcuKraxCTS4tMw5
	e9d1ehUovmDMvAyOLJ7fA0xjqDebh05vwdgHCF7BC86xuCsebnLOR7s4ui2hbwNK
	sZTAWsN8VWQdTb0PBSxEVwBPJEyP+RdHHPyi0D/sTmTeCNTVktLkDUmFvXYqQymq
	IjnpQj3/IhSGnpcoUCo4YNIqDsEOJtwLPwsY3SOwypRjpntJuanosgbVdTxL13Ct
	1m04c3ySEr21k3EYxZLRFdMQTE7bm7QwS72grZJNFrVfwn92ov5z2Bb9VzaQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313319; x=
	1715399719; bh=lLrdK6DtjDVUONkQTnGTL9QCNQzxUIg1GxNz3rQOZI8=; b=i
	fW+UsNFVioTzCLTlh9JVlL7RgOAtNEk+43D/Q10Mdfc3BoD0TEzLnajDpqQXoAQS
	c4Sk1Aa9AeZgdFCai3YlOTGNarum/uoDWLp1g36TK8vd8vqoLcHiC6mwsPd7IKhW
	UBra08s1zfNw/RLkUjX1Huz6W6VEfso2AerCpAR4ebxq5izlpJNRkKrpoUSdsgwg
	KU9ILfN5hBbxQF7q21sCf1ft3FQEXTIih5rARn9hJAgXnF3rn66rgLyoAL+90onI
	hN48UjJwCI+t9nCkLZllWfag3LpTnpgn3Iw6bK9w8E9JVUNTBBtt0hHRqfEofszJ
	rSyBGm+praXsuY1B3wTJg==
X-ME-Sender: <xms:p5o9ZlpplyMQXpPYa-bWL8uEoz1waabI6960Gf-64ioN0aYTizf0gQ>
    <xme:p5o9ZnrTDKOCt9-0XU2VoSZ_EKnUl2utTTopwTC8YlQ8PTqgRbx8rbPUiaEK3EFan
    7grGE5v1FUFSg>
X-ME-Received: <xmr:p5o9ZiP_TapxrJ1__ElR7APA_QPlnBed2rpBLUS0HCLbKhr9XxX9KRUaMG6cfXrP77k1yXaUoZ17DIvpml2Jef0mb7UK1m19nEF_wt5Z31v12PieF34>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:p5o9Zg6AlmmswgKPDllOUrAkcQehgJ-SgQJYl_xQlAtgIQn7UYhALA>
    <xmx:p5o9Zk7eTJkHVkTtksw096GpJJjpSZOCg3VFZgcDymp0_ijaehNDgg>
    <xmx:p5o9ZohXQ2ppe4oqP_PIcSC8uMrElgFgpdwU6OHKM2eEqXsSTW8O6g>
    <xmx:p5o9Zm4GTtNpezbxXsxlbdwFSkgOIvLFigfgJVrB8z_m06xbP78jmg>
    <xmx:p5o9Zv205Ediz1ugEFg8_sGSeEttm_Mw1E2kumTY5cqgdgZdSDcrk9TB>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v8 1/6] x86/msi: Extend per-domain/device warning mechanism
Date: Fri, 10 May 2024 05:53:21 +0200
Message-ID: <0ff02aa7d6f78106a69158794d51c0b32a75431d.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The arch_msix struct had a single "warned" field with a domid for which
warning was issued. Upcoming patch will need similar mechanism for few
more warnings, so change it to save a bit field of issued warnings.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v6:
- add MSIX_CHECK_WARN macro (Jan)
- drop struct name from warned_kind union (Jan)

New in v5
---
 xen/arch/x86/include/asm/msi.h | 17 ++++++++++++++++-
 xen/arch/x86/msi.c             |  5 +----
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
index 997ccb87be0c..bcfdfd35345d 100644
--- a/xen/arch/x86/include/asm/msi.h
+++ b/xen/arch/x86/include/asm/msi.h
@@ -208,6 +208,15 @@ struct msg_address {
                                        PCI_MSIX_ENTRY_SIZE + \
                                        (~PCI_MSIX_BIRMASK & (PAGE_SIZE - 1)))
 
+#define MSIX_CHECK_WARN(msix, domid, which) ({ \
+    if ( (msix)->warned_domid != (domid) ) \
+    { \
+        (msix)->warned_domid = (domid); \
+        (msix)->warned_kind.all = 0; \
+    } \
+    (msix)->warned_kind.which ? false : ((msix)->warned_kind.which = true); \
+})
+
 struct arch_msix {
     unsigned int nr_entries, used_entries;
     struct {
@@ -217,7 +226,13 @@ struct arch_msix {
     int table_idx[MAX_MSIX_TABLE_PAGES];
     spinlock_t table_lock;
     bool host_maskall, guest_maskall;
-    domid_t warned;
+    domid_t warned_domid;
+    union {
+        uint8_t all;
+        struct {
+            bool maskall                   : 1;
+        };
+    } warned_kind;
 };
 
 void early_msi_init(void);
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index e721aaf5c001..42c793426da3 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -364,13 +364,10 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)
             domid_t domid = pdev->domain->domain_id;
 
             maskall = true;
-            if ( pdev->msix->warned != domid )
-            {
-                pdev->msix->warned = domid;
+            if ( MSIX_CHECK_WARN(pdev->msix, domid, maskall) )
                 printk(XENLOG_G_WARNING
                        "cannot mask IRQ %d: masking MSI-X on Dom%d's %pp\n",
                        desc->irq, domid, &pdev->sbdf);
-            }
         }
         pdev->msix->host_maskall = maskall;
         if ( maskall || pdev->msix->guest_maskall )
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719467.1122256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM3-0007xb-O6; Fri, 10 May 2024 03:55:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719467.1122256; Fri, 10 May 2024 03:55:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM3-0007xP-K6; Fri, 10 May 2024 03:55:31 +0000
Received: by outflank-mailman (input) for mailman id 719467;
 Fri, 10 May 2024 03:55:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HM2-0006yN-Cb
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:30 +0000
Received: from wfhigh5-smtp.messagingengine.com
 (wfhigh5-smtp.messagingengine.com [64.147.123.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 22db5715-0e81-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 05:55:28 +0200 (CEST)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailfhigh.west.internal (Postfix) with ESMTP id 6113B180012B;
 Thu,  9 May 2024 23:55:26 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Thu, 09 May 2024 23:55:26 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:24 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22db5715-0e81-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715313326; x=1715399726; bh=Uac2qx6Vbh
	R5d6LSLBww4OHHHB4xRISCtst2qrKvDik=; b=w2D+lAYsyqL5Yt1/KjA/K7+087
	GPEhsECSdBsTWU+T6243GImObLIxzRLcMOXQQ/B8Nu3PA5/YO+WVaAEYuVhIZpGI
	oz03idPiaded75Qoq4amyWkIVRxK33kgWehEvT5iqkJRLk3N89p35dl8zxclygfm
	HwSJMejJ5mZz/d2GM11n5kKZPmf8MY8q9THz2tRiA5oenJL32OVxtG41Lw0QvByO
	wIopligZP5mqgcuLKe2llvcBdWQi9lSjRSZlcOLZH33vzNu9PZrjpg+K6MS+/5DV
	1NjsW2ZvA9qHgxmWkb+Gd+ikFgEMuE5DzwnxpPtRCNJ3BysWdDcjl1SrDALw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313326; x=
	1715399726; bh=Uac2qx6VbhR5d6LSLBww4OHHHB4xRISCtst2qrKvDik=; b=J
	YxxpGxc/MIMaxzkLte1Vm9jls+CbjD2naXZS9tjilKFNpa4CCX3Oo1mBueodpsjt
	UpmbGl5nMrpyPOLE+A4R6807/CURiamk117zOza2c8nsH7DfziTrwuXZbAL+XKtn
	E84YZGgeckPlYV4Xs10aVNYYKsal6AJN8KpSJ1rrUVRcVxWZssB73scJhhKdfTJi
	BxJAls+C3CPymQ1+cknSd7egKW9rC0TZYMPwzzVmOZgcBGM+iAIjKhXAsNpwuxPJ
	bYC7fsVkjxajJ+GYhaoHIc9v/YnLXLIxprgHm0qH1OpSa3TjuB2ZYFQbHviuvrS2
	+tsn/LBNY6RdldMfQ2H3g==
X-ME-Sender: <xms:rZo9ZtCvOQSxi4CRoMRRWyg4CfzoXvAgflmpAx9x8G9kt0P-T_Plzg>
    <xme:rZo9ZrjXOlxb3RoZThYzym255HIH83xo6LF65rgZ6_ny23KoXaO0_0C0f4s652q9V
    PrwMw0tiDzQfQ>
X-ME-Received: <xmr:rZo9ZomB3Yefn2PPJVhrcy-LrIUQubD1Ct-52pAyxM65-vofryMQCdh9IiiDoEbfLyVh5lSqCAo37naIIpZVFEz6bro0gNymA0j6zL4658ggUqLiEk0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeff
    iedtgeevffetledvgefhhfevgffhfeekleehueejjeegvddvgfffjeeutddvleenucffoh
    hmrghinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr
    rghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhngh
    hslhgrsgdrtghomh
X-ME-Proxy: <xmx:rZo9Znyp4FhIS8fm9Hvba_amqn864PoFElIJ1yD90RyBILCMuOw6yA>
    <xmx:rZo9ZiRWgbJQqPOu8i7Jx58MoGUT6TnKLB_DXszcBFpqIZtQop8yOw>
    <xmx:rZo9ZqahjFZJj0QXUpb-8Zn4M6zqbyf79eCCAxg6vmol3yZcwa7FEQ>
    <xmx:rZo9ZjSJwZ49puO4IVArPRa7eWZRLO2SYfQ2FRQ7zxzhnUV9Q1Krdg>
    <xmx:rpo9ZgdWLd1x-75kyQW9RGaTg0wrPpJfbBdI8mrYvnuDgjyAlB1UQwUG>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH v8 4/6] automation: switch to a wifi card on ADL system
Date: Fri, 10 May 2024 05:53:24 +0200
Message-ID: <ac5f7b07e47edf6f14d50935d7fd01fdd6a6e900.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Switch to a wifi card that has registers on a MSI-X page. This tests the
"x86/hvm: Allow writes to registers on the same page as MSI-X table"
feature. Switch it only for HVM test, because MSI-X adjacent write is
not supported on PV.

This requires also including drivers and firmware in system for tests.
Remove firmware unrelated to the test, to not increase initrd size too
much (all firmware takes over 100MB compressed).
And finally adjusts test script to handle not only eth0 as a test device,
but also wlan0 and connect it to the wifi network.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
This needs two new gitlab variables: WIFI_HW2_SSID and WIFI_HW2_PSK. I'll
provide them in private.

This change requires rebuilding test containers.

This can be applied only after QEMU change is committed. Otherwise the
test will fail.
---
 automation/gitlab-ci/test.yaml                      | 4 ++++
 automation/scripts/qubes-x86-64.sh                  | 7 +++++++
 automation/tests-artifacts/alpine/3.18.dockerfile   | 7 +++++++
 automation/tests-artifacts/kernel/6.1.19.dockerfile | 2 ++
 4 files changed, 20 insertions(+)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index ad249fa0a5d9..6803cae116b5 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -193,6 +193,10 @@ adl-pci-pv-x86-64-gcc-debug:
 
 adl-pci-hvm-x86-64-gcc-debug:
   extends: .adl-x86-64
+  variables:
+    PCIDEV: "00:14.3"
+    WIFI_SSID: "$WIFI_HW2_SSID"
+    WIFI_PSK: "$WIFI_HW2_PSK"
   script:
     - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE}
   needs:
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 7eabc1bd6ad4..60498ef1e89a 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -94,6 +94,13 @@ on_reboot = "destroy"
     domU_check="
 set -x -e
 interface=eth0
+if [ -e /sys/class/net/wlan0 ]; then
+    interface=wlan0
+    set +x
+    wpa_passphrase "$WIFI_SSID" "$WIFI_PSK" > /etc/wpa_supplicant.conf
+    set -x
+    wpa_supplicant -B -iwlan0 -c /etc/wpa_supplicant.conf
+fi
 ip link set \"\$interface\" up
 timeout 30s udhcpc -i \"\$interface\"
 pingip=\$(ip -o -4 r show default|cut -f 3 -d ' ')
diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile
index 9cde6c9ad4da..c323e266c7da 100644
--- a/automation/tests-artifacts/alpine/3.18.dockerfile
+++ b/automation/tests-artifacts/alpine/3.18.dockerfile
@@ -34,6 +34,13 @@ RUN \
   apk add udev && \
   apk add pciutils && \
   apk add libelf && \
+  apk add wpa_supplicant && \
+  # Select firmware for hardware tests
+  apk add linux-firmware-other && \
+  mkdir /lib/firmware-preserve && \
+  mv /lib/firmware/iwlwifi-so-a0-gf-a0* /lib/firmware-preserve/ && \
+  rm -rf /lib/firmware && \
+  mv /lib/firmware-preserve /lib/firmware && \
   \
   # Xen
   cd / && \
diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile
index 3a4096780d20..84ed5dff23ae 100644
--- a/automation/tests-artifacts/kernel/6.1.19.dockerfile
+++ b/automation/tests-artifacts/kernel/6.1.19.dockerfile
@@ -32,6 +32,8 @@ RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSI
     make xen.config && \
     scripts/config --enable BRIDGE && \
     scripts/config --enable IGC && \
+    scripts/config --enable IWLWIFI && \
+    scripts/config --enable IWLMVM && \
     cp .config .config.orig && \
     cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
     make -j$(nproc) bzImage && \
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719463.1122215 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HLx-0006yc-H8; Fri, 10 May 2024 03:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719463.1122215; Fri, 10 May 2024 03:55:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HLx-0006yV-Dk; Fri, 10 May 2024 03:55:25 +0000
Received: by outflank-mailman (input) for mailman id 719463;
 Fri, 10 May 2024 03:55:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HLw-0006yN-Fi
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:24 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1df50f2c-0e81-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 05:55:21 +0200 (CEST)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailfout.west.internal (Postfix) with ESMTP id 764D11C001B1;
 Thu,  9 May 2024 23:55:17 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Thu, 09 May 2024 23:55:17 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:16 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1df50f2c-0e81-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1715313317; x=1715399717; bh=2cGOtahSU32QEdVU8CxuJ6IEc/Czt0JO
	m9V0st6+axo=; b=D9FuxEDAlhmohLcKzMJg7jWMEMbid7zMrH2ZNRq39eIJAW3z
	CtHYWWSpEostCUJ9Rz3yNFiVk6YwyzIbwlJeuAvuD4h+OS8waupPAGR6tnl4EKPi
	1fVLh63ZUipGl3usvTTtwcxLoyzWEctnQBaG3E5yFVoUvzAG2mkCNzOqyQhQ+HvR
	2ahpl/fKhQiaKxCL3QqIX7aFWCTK7koAo74yCHLCGM8azXU1vMiXiPYJf32e1M38
	tRzqGiJ2G1+h3deXMWBwesjRWnhjDRNY8N9bGNyAxPQJyLWlL4z5mV+4zpcjyKil
	I4eUVgn+M4kbtM17GLoJL9aRcKCHseIaN/MQ3A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm3; t=1715313317; x=1715399717; bh=2cGOtahSU32QE
	dVU8CxuJ6IEc/Czt0JOm9V0st6+axo=; b=GL7pym3bXc8rA90S8NfKkCOwDhtpD
	wVfeB0RrlaQLHJcmhL2X8en64eU1PQLnKwxWINmrnDfS6Vd2EVv+8RNVp9P9kbCN
	U4drKYoRz6T9BpEwDvHPT+JbdaRfND/0XUdTqHL3q/OSyQ/zC1ugTjMvLun6wvHO
	6BBv0rW8hV7zfmB9WdOEfq1j7guV3Z+3Ve1dQ/TZEINNtyzUUL2umOv5r+l6c/nK
	MOQdTLUSW6S4lX6GMpNRgQtepwAgVhrzkrMFdGwyIYVPoyhh4J6OLOU5uPg1k+I8
	Ske7SlYlalP+ZYb1s3pyv6DiOlNUKdQhVnR9xqIomc0j0M2BBiZjm6r1Q==
X-ME-Sender: <xms:pJo9ZsKpjxu8xXhFIjLUQMoBZ7Mru37kt9XOZxJgdiTKk6LZhNHBFA>
    <xme:pJo9ZsKvfdFYmGuOsOajiP_tWJkJ885kCmnhOswx4SUhKf6tYt64qarEOOYp6mLkA
    MERHh4Vo0i4Hg>
X-ME-Received: <xmr:pJo9Zst5pFhrNGPY8XYsaNW8f4Ic1vGq27Jk9k5w3deZKHr1aU-UD8_rIL6pD-PyYl7_9WzPpXL-s4mNCijnu-1AEyf79YoiUPeXYrTsU7BFBtVI9KM>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggt
    khhiuceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
    eqnecuggftrfgrthhtvghrnhephfefvdektdeivdfgieevteffveefgfdtheevtedtveej
    udffveethefhveeghfehnecuffhomhgrihhnpehgihhthhhusgdrtghomhdpghhithhlrg
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:pJo9Zpapexknt5IIg6lc4SpypoHk4mRDaZAcg9muSqh-is0Pm4pyZQ>
    <xmx:pJo9Zja9j5oh6rlhdehuwPvIl5amje2YEOUx0gHi14AI9zrOenrVSQ>
    <xmx:pJo9ZlCw4LWeNADGbQp4tpHhS9Bd0I9MX7-fjBevBdbiDU4R45q-WQ>
    <xmx:pJo9ZpaDcKtTlQMeaG41Ebrxxb0jEeigj-2H5EUTA7QLAUQkCOlCqg>
    <xmx:pZo9ZqwRSt_EycSF_2rvHulam6qVfNxKsa11ko3MN7R-t7YSjNpkq2at>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH v8 0/6] MSI-X support with qemu in stubdomain, and other related changes
Date: Fri, 10 May 2024 05:53:20 +0200
Message-ID: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This series includes changes to make MSI-X working with Linux stubdomain and
especially Intel Wifi 6 AX210 card. This takes care of remaining reasons for
QEMU to access /dev/mem, but also the Intel Wifi card violating spec by putting
some registers on the same page as the MSI-X table.
Besides the stubdomain case (of which I care more), this is also necessary for
PCI-passthrough to work with lockdown enabled in dom0 (when QEMU runs in dom0).

See individual patches for details.

This series include also tests for MSI-X using new approach (by preventing QEMU
access to /dev/mem). But for it to work, it needs QEMU change that
makes use of the changes introduced here. It can be seen at
https://github.com/marmarek/qemu/commits/msix

Here is the pipeline that used the QEMU fork above:
https://gitlab.com/xen-project/people/marmarek/xen/-/pipelines/1285354093
(the build failures are due to issues with fetching or building newer QEMU
 discussed on Matrix)

v7:
 - "x86/msi: passthrough all MSI-X vector ctrl writes to device model" is already applied

Marek Marczykowski-Górecki (6):
  x86/msi: Extend per-domain/device warning mechanism
  x86/hvm: Allow access to registers on the same page as MSI-X table
  automation: prevent QEMU access to /dev/mem in PCI passthrough tests
  automation: switch to a wifi card on ADL system
  [DO NOT APPLY] switch to qemu fork
  [DO NOT APPLY] switch to alternative artifact repo

 Config.mk                                           |   4 +-
 automation/gitlab-ci/build.yaml                     |   4 +-
 automation/gitlab-ci/test.yaml                      |   4 +-
 automation/scripts/qubes-x86-64.sh                  |   9 +-
 automation/tests-artifacts/alpine/3.18.dockerfile   |   7 +-
 automation/tests-artifacts/kernel/6.1.19.dockerfile |   2 +-
 xen/arch/x86/hvm/vmsi.c                             | 208 ++++++++++++-
 xen/arch/x86/include/asm/msi.h                      |  22 +-
 xen/arch/x86/msi.c                                  |  47 ++-
 9 files changed, 288 insertions(+), 19 deletions(-)

base-commit: ebab808eb1bb8f24c7d0dd41b956e48cb1824b81
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719464.1122226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HLz-0007DH-Sh; Fri, 10 May 2024 03:55:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719464.1122226; Fri, 10 May 2024 03:55:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HLz-0007DA-Oy; Fri, 10 May 2024 03:55:27 +0000
Received: by outflank-mailman (input) for mailman id 719464;
 Fri, 10 May 2024 03:55:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HLy-0006yN-8r
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:26 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 20539327-0e81-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 05:55:23 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id 0D6CA1C000F6;
 Thu,  9 May 2024 23:55:21 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute6.internal (MEProxy); Thu, 09 May 2024 23:55:22 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:20 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20539327-0e81-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715313321; x=1715399721; bh=ACyP2w5iOq
	1oYIJ7NbzixfPI/5xHvfYdC3cDB7RY4rE=; b=ZhtWLV2P0ka6pWzFXQar6Ja25N
	I8aE2yjzrwm2rnF9QfPW8DQyCRfp0eQxRFGfZVXJaExNMPzSeOqof4UsHXf71+Ua
	phU42xLaQu++LzXLvbkx/ePMjyNnobqy3USIXpWUZ2gVd4u4GhTOxfPKPWyNqMIl
	GEn2MZjB9q0q7yqZkE0pCtr1Pg6H8JzOypwfIxc4gHt3vsPyDYu+HItcNYnCwhVl
	8TL2Bn2F+Lpvp54wLc7+jwouqWbDn9Wmq+xXR/bygUDP3qkt+AP4IiCO+tZ6nYyd
	qCQ2GB0541+xtEV3e6YoS+fA2BcBb2XOT7wq1MLnipKefEZ0zppW6UeCROdw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313321; x=
	1715399721; bh=ACyP2w5iOq1oYIJ7NbzixfPI/5xHvfYdC3cDB7RY4rE=; b=k
	fXBhdGpv2EahviQsFjLNyISaKyOrVVNdnMsqjOm4nX6/ZdqFsR8S0fSnwOLNKOZM
	t8CejMYEh8zyF+Pr8prWhddDPu0PHxRfh59aBsGwK8um99xeHjnoab8s1rwBn2RN
	akcF8xJ3bq1f36zhLO/XMuaoZBy5kLLImEDE2/xd5gQ+gozDmPcboVJwxksgquxS
	ZPjLqj0qKdwJDe2sB/LiOPf38v6K7Y8ObBcc3s34qnUS19NtleUEoF9jzkcR4a+o
	Tl6GZ1lh+8WQTlEXoWzwjynpzu+5rOAVTd/MnZUCns9vzmXO0c3p99sCvdm7e4QK
	/cKJoxxdLTZ9/JjbZnvYQ==
X-ME-Sender: <xms:qZo9ZoZ-hSBpmCsyPWLh_0MmEBYiu03pWuKK2XglFPLHhP-sFdHxIQ>
    <xme:qZo9ZjZfAeicguXXRnOQVRYxHr7wzcNhCL1-KE1P_HN_92IHpK-6GI0BmgDOs9fQv
    qrrZVTmC72isA>
X-ME-Received: <xmr:qZo9Zi9mSJWobilgRThEQa__HosIzrAcklUFddhUYFFiU86L5UL2hz-0XCu9--jd0i0D57ia7-0dDUlEkdxAMEJYCroxtyWQfQbUl2_EXFBakeQG_eM>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefh
    iefhgfehgfekteeigfdtiedvgfevhffhfedvudeltedvgfegtdekteeffeegheenucffoh
    hmrghinhepmhhsihigthgslhgplhhishhtrdhnvgigthenucevlhhushhtvghrufhiiigv
    pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg
    hlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:qZo9Zioo2dV-TOp_6gkP2xBuZ5q8z_FkuuhC8M-Bbgpvks9qZRoxUw>
    <xmx:qZo9Zjqah9ZSmpRjFVOoNxsQWcdySEjqNWldPcnyaE41aFj2zdM1Mw>
    <xmx:qZo9ZgRjqyYBtDTpPszO5UxNXnso1OG5OGDGp8ITn55Ck2Cc6Cag2g>
    <xmx:qZo9ZjpSPKS1B4SioQPdhj8eI_Bzf0EEELmkCT_lglb_LPlEX3VsRg>
    <xmx:qZo9ZkkuLOBA52Ut98IRkjXLdTwuwi5N6uTGcP8NzYCO-ou3eVyySfxK>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v8 2/6] x86/hvm: Allow access to registers on the same page as MSI-X table
Date: Fri, 10 May 2024 05:53:22 +0200
Message-ID: <fbe01c945d75409406ac0b02bc17d44f57a39ccf.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
on the same page as MSI-X table. Device model (especially one in
stubdomain) cannot really handle those, as direct writes to that page is
refused (page is on the mmio_ro_ranges list). Instead, extend
msixtbl_mmio_ops to handle such accesses too.

Doing this, requires correlating read/write location with guest
MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
for PV would need to be done separately.

This will be also used to read Pending Bit Array, if it lives on the same
page, making QEMU not needing /dev/mem access at all (especially helpful
with lockdown enabled in dom0). If PBA lives on another page, QEMU will
map it to the guest directly.
If PBA lives on the same page, discard writes and log a message.
Technically, writes outside of PBA could be allowed, but at this moment
the precise location of PBA isn't saved, and also no known device abuses
the spec in this way (at least yet).

To access those registers, msixtbl_mmio_ops need the relevant page
mapped. MSI handling already has infrastructure for that, using fixmap,
so try to map first/last page of the MSI-X table (if necessary) and save
their fixmap indexes. Note that msix_get_fixmap() does reference
counting and reuses existing mapping, so just call it directly, even if
the page was mapped before. Also, it uses a specific range of fixmap
indexes which doesn't include 0, so use 0 as default ("not mapped")
value - which simplifies code a bit.

Based on assumption that all MSI-X page accesses are handled by Xen, do
not forward adjacent accesses to other hypothetical ioreq servers, even
if the access wasn't handled for some reason (failure to map pages etc).
Relevant places log a message about that already.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v8:
- rename adjacent_handle to get_adjacent_idx
- put SBDF at the start of error messages
- use 0 for ADJACENT_DONT_HANDLE (it's FIX_RESERVED)
- merge conditions in msixtbl_range into one "if"
- add assert for address alignment
- change back to setting pval to ~0UL at the start of adjacent_read
Changes in v7:
- simplify logic based on assumption that all access to MSI-X pages are
  handled by Xen (Roger)
- move calling adjacent_handle() into adjacent_{read,write}() (Roger)
- move range check into msixtbl_addr_to_desc() (Roger)
- fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
- no longer distinguish between unhandled write due to PBA nearby and
  other reasons
- add missing break after ASSERT_UNREACHABLE (Jan)
Changes in v6:
- use MSIX_CHECK_WARN macro
- extend assert on fixmap_idx
- add break in default label, after ASSERT_UNREACHABLE(), and move
  setting default there
- style fixes
Changes in v5:
- style fixes
- include GCC version in the commit message
- warn only once (per domain, per device) about failed adjacent access
Changes in v4:
- drop same_page parameter of msixtbl_find_entry(), distinguish two
  cases in relevant callers
- rename adj_access_table_idx to adj_access_idx
- code style fixes
- drop alignment check in adjacent_{read,write}() - all callers already
  have it earlier
- delay mapping first/last MSI-X pages until preparing device for a
  passthrough
v3:
 - merge handling into msixtbl_mmio_ops
 - extend commit message
v2:
 - adjust commit message
 - pass struct domain to msixtbl_page_handler_get_hwaddr()
 - reduce local variables used only once
 - log a warning if write is forbidden if MSI-X and PBA lives on the same
   page
 - do not passthrough unaligned accesses
 - handle accesses both before and after MSI-X table
---
 xen/arch/x86/hvm/vmsi.c        | 208 ++++++++++++++++++++++++++++++++--
 xen/arch/x86/include/asm/msi.h |   5 +-
 xen/arch/x86/msi.c             |  42 +++++++-
 3 files changed, 245 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 999917983789..d506d6adaaf6 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
     return d->arch.hvm.msixtbl_list.next;
 }
 
+/*
+ * Lookup an msixtbl_entry on the same page as given addr. It's up to the
+ * caller to check if address is strictly part of the table - if relevant.
+ */
 static struct msixtbl_entry *msixtbl_find_entry(
     struct vcpu *v, unsigned long addr)
 {
@@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
     struct domain *d = v->domain;
 
     list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
-        if ( addr >= entry->gtable &&
-             addr < entry->gtable + entry->table_len )
+        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
+             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
             return entry;
 
     return NULL;
@@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
     if ( !entry || !entry->pdev )
         return NULL;
 
+    if ( addr < entry->gtable ||
+         addr >= entry->gtable + entry->table_len )
+        return NULL;
+
     nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
 
     list_for_each_entry( desc, &entry->pdev->msi_list, list )
@@ -213,6 +221,157 @@ static struct msi_desc *msixtbl_addr_to_desc(
     return NULL;
 }
 
+/*
+ * Returns:
+ *  - 0 (FIX_RESERVED) if no handling should be done
+ *  - a fixmap idx to use for handling
+ */
+static unsigned int get_adjacent_idx(
+    const struct msixtbl_entry *entry, unsigned long addr, bool write)
+{
+    unsigned int adj_type;
+    struct arch_msix *msix;
+
+    if ( !entry || !entry->pdev )
+    {
+        ASSERT_UNREACHABLE();
+        return 0;
+    }
+
+    if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable )
+        adj_type = ADJ_IDX_FIRST;
+    else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) &&
+              addr >= entry->gtable + entry->table_len )
+        adj_type = ADJ_IDX_LAST;
+    else
+    {
+        /* All callers should already do equivalent range checking. */
+        ASSERT_UNREACHABLE();
+        return 0;
+    }
+
+    msix = entry->pdev->msix;
+    if ( !msix )
+    {
+        ASSERT_UNREACHABLE();
+        return 0;
+    }
+
+    if ( !msix->adj_access_idx[adj_type] )
+    {
+        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
+                             adjacent_not_initialized) )
+            gprintk(XENLOG_WARNING,
+                    "%pp: Page for adjacent(%d) MSI-X table access not initialized (addr %#lx, gtable %#lx))\n",
+                    &entry->pdev->sbdf, adj_type, addr, entry->gtable);
+        return 0;
+    }
+
+    /* If PBA lives on the same page too, discard writes. */
+    if ( write &&
+         ((adj_type == ADJ_IDX_LAST &&
+           msix->table.last == msix->pba.first) ||
+          (adj_type == ADJ_IDX_FIRST &&
+           msix->table.first == msix->pba.last)) )
+    {
+        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
+                             adjacent_pba) )
+            gprintk(XENLOG_WARNING,
+                    "%pp: MSI-X table and PBA share a page, "
+                    "discard write to adjacent memory (%#lx)\n",
+                    &entry->pdev->sbdf, addr);
+        return 0;
+    }
+
+    return msix->adj_access_idx[adj_type];
+}
+
+static int adjacent_read(
+    const struct msixtbl_entry *entry,
+    paddr_t address, unsigned int len, uint64_t *pval)
+{
+    const void __iomem *hwaddr;
+    unsigned int fixmap_idx;
+
+    ASSERT(IS_ALIGNED(address, len));
+
+    *pval = ~0UL;
+
+    fixmap_idx = get_adjacent_idx(entry, address, false);
+
+    if ( !fixmap_idx )
+        return X86EMUL_OKAY;
+
+    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
+
+    switch ( len )
+    {
+    case 1:
+        *pval = readb(hwaddr);
+        break;
+
+    case 2:
+        *pval = readw(hwaddr);
+        break;
+
+    case 4:
+        *pval = readl(hwaddr);
+        break;
+
+    case 8:
+        *pval = readq(hwaddr);
+        break;
+
+    default:
+        ASSERT_UNREACHABLE();
+        break;
+    }
+
+    return X86EMUL_OKAY;
+}
+
+static int adjacent_write(
+    const struct msixtbl_entry *entry,
+    paddr_t address, unsigned int len, uint64_t val)
+{
+    void __iomem *hwaddr;
+    unsigned int fixmap_idx;
+
+    ASSERT(IS_ALIGNED(address, len));
+
+    fixmap_idx = get_adjacent_idx(entry, address, true);
+
+    if ( !fixmap_idx )
+        return X86EMUL_OKAY;
+
+    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
+
+    switch ( len )
+    {
+    case 1:
+        writeb(val, hwaddr);
+        break;
+
+    case 2:
+        writew(val, hwaddr);
+        break;
+
+    case 4:
+        writel(val, hwaddr);
+        break;
+
+    case 8:
+        writeq(val, hwaddr);
+        break;
+
+    default:
+        ASSERT_UNREACHABLE();
+        break;
+    }
+
+    return X86EMUL_OKAY;
+}
+
 static int cf_check msixtbl_read(
     const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
     uint64_t *pval)
@@ -222,7 +381,7 @@ static int cf_check msixtbl_read(
     unsigned int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
-    if ( (len != 4 && len != 8) || (address & (len - 1)) )
+    if ( !IS_ALIGNED(address, len) )
         return r;
 
     rcu_read_lock(&msixtbl_rcu_lock);
@@ -230,6 +389,17 @@ static int cf_check msixtbl_read(
     entry = msixtbl_find_entry(current, address);
     if ( !entry )
         goto out;
+
+    if ( address < entry->gtable ||
+         address >= entry->gtable + entry->table_len )
+    {
+        r = adjacent_read(entry, address, len, pval);
+        goto out;
+    }
+
+    if ( len != 4 && len != 8 )
+        goto out;
+
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
 
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
@@ -291,6 +461,17 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
     entry = msixtbl_find_entry(v, address);
     if ( !entry )
         goto out;
+
+    if ( address < entry->gtable ||
+         address >= entry->gtable + entry->table_len )
+    {
+        r = adjacent_write(entry, address, len, val);
+        goto out;
+    }
+
+    if ( len != 4 && len != 8 )
+        goto out;
+
     nr_entry = array_index_nospec(((address - entry->gtable) /
                                    PCI_MSIX_ENTRY_SIZE),
                                   MAX_MSIX_TABLE_ENTRIES);
@@ -356,8 +537,8 @@ static int cf_check _msixtbl_write(
     const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
     uint64_t val)
 {
-    /* Ignore invalid length or unaligned writes. */
-    if ( (len != 4 && len != 8) || !IS_ALIGNED(address, len) )
+    /* Ignore unaligned writes. */
+    if ( !IS_ALIGNED(address, len) )
         return X86EMUL_OKAY;
 
     /*
@@ -374,16 +555,23 @@ static bool cf_check msixtbl_range(
 {
     struct vcpu *curr = current;
     unsigned long addr = r->addr;
-    const struct msi_desc *desc;
+    const struct msixtbl_entry *entry;
+    bool ret = false;
 
     ASSERT(r->type == IOREQ_TYPE_COPY);
 
     rcu_read_lock(&msixtbl_rcu_lock);
-    desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr);
+    entry = msixtbl_find_entry(curr, addr);
+    if ( entry &&
+          /* Adjacent access. */
+         (addr < entry->gtable || addr >= entry->gtable + entry->table_len ||
+          /* Otherwise check if there is a matching msi_desc. */
+          msixtbl_addr_to_desc(entry, addr)) )
+        ret = true;
     rcu_read_unlock(&msixtbl_rcu_lock);
 
-    if ( desc )
-        return 1;
+    if ( ret )
+        return ret;
 
     if ( r->state == STATE_IOREQ_READY && r->dir == IOREQ_WRITE )
     {
@@ -429,7 +617,7 @@ static bool cf_check msixtbl_range(
         }
     }
 
-    return 0;
+    return false;
 }
 
 static const struct hvm_io_ops msixtbl_mmio_ops = {
diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
index bcfdfd35345d..923b730d48b8 100644
--- a/xen/arch/x86/include/asm/msi.h
+++ b/xen/arch/x86/include/asm/msi.h
@@ -224,6 +224,9 @@ struct arch_msix {
     } table, pba;
     int table_refcnt[MAX_MSIX_TABLE_PAGES];
     int table_idx[MAX_MSIX_TABLE_PAGES];
+#define ADJ_IDX_FIRST 0
+#define ADJ_IDX_LAST  1
+    unsigned int adj_access_idx[2];
     spinlock_t table_lock;
     bool host_maskall, guest_maskall;
     domid_t warned_domid;
@@ -231,6 +234,8 @@ struct arch_msix {
         uint8_t all;
         struct {
             bool maskall                   : 1;
+            bool adjacent_not_initialized  : 1;
+            bool adjacent_pba              : 1;
         };
     } warned_kind;
 };
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 42c793426da3..4f9249f11b2f 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -913,6 +913,37 @@ static int msix_capability_init(struct pci_dev *dev,
         list_add_tail(&entry->list, &dev->msi_list);
         *desc = entry;
     }
+    else
+    {
+        /*
+         * If the MSI-X table doesn't start at the page boundary, map the first page for
+         * passthrough accesses.
+         */
+        if ( PAGE_OFFSET(table_paddr) )
+        {
+            int idx = msix_get_fixmap(msix, table_paddr, table_paddr);
+
+            if ( idx > 0 )
+                msix->adj_access_idx[ADJ_IDX_FIRST] = idx;
+            else
+                gprintk(XENLOG_ERR, "%pp: Failed to map first MSI-X table page: %d\n", &dev->sbdf, idx);
+        }
+        /*
+         * If the MSI-X table doesn't end on the page boundary, map the last page
+         * for passthrough accesses.
+         */
+        if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE) )
+        {
+            uint64_t entry_paddr = table_paddr +
+                (msix->nr_entries - 1) * PCI_MSIX_ENTRY_SIZE;
+            int idx = msix_get_fixmap(msix, table_paddr, entry_paddr);
+
+            if ( idx > 0 )
+                msix->adj_access_idx[ADJ_IDX_LAST] = idx;
+            else
+                gprintk(XENLOG_ERR, "%pp: Failed to map last MSI-X table page: %d\n", &dev->sbdf, idx);
+        }
+    }
 
     if ( !msix->used_entries )
     {
@@ -1079,6 +1110,17 @@ static void _pci_cleanup_msix(struct arch_msix *msix)
         msix->table.first = 0;
         msix->table.last = 0;
 
+        if ( msix->adj_access_idx[ADJ_IDX_FIRST] )
+        {
+            msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_FIRST]);
+            msix->adj_access_idx[ADJ_IDX_FIRST] = 0;
+        }
+        if ( msix->adj_access_idx[ADJ_IDX_LAST] )
+        {
+            msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_LAST]);
+            msix->adj_access_idx[ADJ_IDX_LAST] = 0;
+        }
+
         if ( rangeset_remove_range(mmio_ro_ranges, msix->pba.first,
                                    msix->pba.last) )
             WARN();
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 03:55:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 03:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719469.1122276 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM8-00006V-81; Fri, 10 May 2024 03:55:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719469.1122276; Fri, 10 May 2024 03:55:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5HM8-000066-3R; Fri, 10 May 2024 03:55:36 +0000
Received: by outflank-mailman (input) for mailman id 719469;
 Fri, 10 May 2024 03:55:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XugN=MN=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5HM6-0006yN-P0
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 03:55:34 +0000
Received: from wfhigh5-smtp.messagingengine.com
 (wfhigh5-smtp.messagingengine.com [64.147.123.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 25dbe0f9-0e81-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 05:55:33 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfhigh.west.internal (Postfix) with ESMTP id 6A5C0180012B;
 Thu,  9 May 2024 23:55:31 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute6.internal (MEProxy); Thu, 09 May 2024 23:55:31 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 9 May 2024 23:55:29 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25dbe0f9-0e81-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to; s=fm3; t=1715313331; x=1715399731; bh=1Lf7PaioW5WU8mR8fRuXT
	r4i1fep7cAUzdBzJdvpuFY=; b=Dx8+i/kOF4Zs6YKMhB+Fe5ATpeKljbzgQLUAR
	Kf/qKXT+bTYYXGysc1XX1i2WCOrDi4j6S24cO+bSNj6cRyXEmBWg/lI2B36u+jC0
	xBaBX56vUA2sHqDIN42f5itz8HjMsWEIpceBAG8sK0o2PmW6+J4ByDIfsgHNM78j
	YFzExFm+U+CEzn9JpjwnLN46TCyV9ZBaeN8wOvv0vb5gIljMEoqyr5x3rJ6kStxo
	0M6EppiuwLM/HGxq+NilZ4QNmjBf238F4YEvtnDQFvnwO97d1Qi8s3dfnZMqAGRl
	mHwdiAkRHThgQXwSrovUCARbuzvlaIJmPrAteazZ4IKFY7sOg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:feedback-id:feedback-id:from:from
	:in-reply-to:in-reply-to:message-id:mime-version:references
	:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715313331; x=
	1715399731; bh=1Lf7PaioW5WU8mR8fRuXTr4i1fep7cAUzdBzJdvpuFY=; b=Z
	y5HDmtp1u2xJD7C4roZ3NTJK60dDW1l9yU8n29DZikmQ4SdFOT6LeU2RVKgzpo7J
	f+zWQ4z8ke85+XBLJiezFaZDzpYfLnvhyAEQ8VgwBBneu9BRrZYK4meVUhTpCGAG
	aSmy886aCbSwWVypKTc+Jthc0sdOc796ZRoDy6tSQ9O0rCs9klWUl/stQGKiPgnq
	xS5bff2TRK5M9mPQRV3pSd0CSwGjStXOSSgso6CZVopCs0VSMhGVSS8nwh8nyvuA
	1+w55fYgKJWLK1yoVw8ArIfCnCJsz5108EtHCNf2dKm/6SXIDJPAwxPacmFhfg+a
	6/KchQm0bTIr07YkQ/iCA==
X-ME-Sender: <xms:spo9ZuWlXC8MNBDSKFXYM8hNQACVH3hsIMuwPAzKpOfV7N7iMt23NA>
    <xme:spo9Zql8kDRgWS0qbGFN3GCYkp8hrWgPDugYXlg-IfUVjrMvtF2C8d_SI8qvA2x9i
    D2NzbIATzQyVA>
X-ME-Received: <xmr:spo9ZiZCsaJUSVtWy517Df6IiGgQnSjPXgafyzMf2HZf89vE9XAMHEQyJFQPWu3WLMgSRrFpkh5LIaTJrRSShoZeLuchk7fU863ha-gZuh0Gy9XUSXI>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefhedgudefucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueet
    hefgieehheeftdeuleduhffgieelueejhffggfeiieevtefhfeffffeftdehnecuffhomh
    grihhnpehgihhtlhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:spo9ZlW60ro5wM7ILDLAm-FCu2xapcTtOQMdOOUPJk8U2WnJS4OzFQ>
    <xmx:spo9Zol4wZdSn6jvMOcpabkvEU6d_KNoLuavnympKNqCgPYdU04Zew>
    <xmx:spo9ZqcE3tyN5W17j6yStnnq2fF_-18h_Y8tJb5XjF4s6JTkgnUEDA>
    <xmx:spo9ZqGWRxErf3PuOy5xtQco5J5r2_x7w9li09TmkdmNSru1fyxVaQ>
    <xmx:s5o9Zii4TagOLC6NF8afDL_JGFaX0WElSnqpnMWMmQXUJJndm-mIG_ZY>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v8 6/6] [DO NOT APPLY] switch to alternative artifact repo
Date: Fri, 10 May 2024 05:53:26 +0200
Message-ID: <1fdd3d09ab9ec27825d88e65f42f0d5b9b3cd109.1715313192.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

For testing, switch to my containers registry that includes containers
rebuilt with changes in this series.
---
 automation/gitlab-ci/build.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5b4..0d7e311417d8 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -320,7 +320,7 @@ qemu-system-ppc64-8.1.0-ppc64-export:
 
 alpine-3.18-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/alpine:3.18
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
   artifacts:
@@ -331,7 +331,7 @@ alpine-3.18-rootfs-export:
 
 kernel-6.1.19-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/kernel:6.1.19
   script:
     - mkdir binaries && cp /bzImage binaries/bzImage
   artifacts:
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Fri May 10 05:03:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 05:03:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719500.1122286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5IP8-0005Gb-VE; Fri, 10 May 2024 05:02:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719500.1122286; Fri, 10 May 2024 05:02:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5IP8-0005GU-SP; Fri, 10 May 2024 05:02:46 +0000
Received: by outflank-mailman (input) for mailman id 719500;
 Fri, 10 May 2024 05:02:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5IP6-0005GI-Tz; Fri, 10 May 2024 05:02:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5IP6-0004sW-OI; Fri, 10 May 2024 05:02:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5IP6-0007fA-AW; Fri, 10 May 2024 05:02:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5IP6-0002ff-A7; Fri, 10 May 2024 05:02:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iZOkaFOaCSnS2yMUxTdO93LKsHGS+5P3lKx23NlJy0s=; b=fsIQMpnn44bOcj7hOXMwpL4Zgf
	JPeAhwXpsltuowaNgAzRTkcP5bfG/LkKBnsG5SiVIh32pbD+AgS9Wp1xNEjMkKXFkAwa2r+shMOZm
	JOE1tqn6bmI29Z6eaWvAQU7EqTQ//EiexyWiyV9jXjjou9Zkww8Erd4TqBTq/TwoByAo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185964-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185964: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b0082b908391b29b7c4dd5e6c389ebd6481926f8
X-Osstest-Versions-That:
    xen=53959cb8309919fc2f157a1c99e0af2ce280cb84
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 05:02:44 +0000

flight 185964 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185964/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b0082b908391b29b7c4dd5e6c389ebd6481926f8
baseline version:
 xen                  53959cb8309919fc2f157a1c99e0af2ce280cb84

Last test of basis   185958  2024-05-09 18:02:09 Z    0 days
Testing same since   185964  2024-05-10 02:02:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
  Federico Serafini <federico.serafini@bugseng.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   53959cb830..b0082b9083  b0082b908391b29b7c4dd5e6c389ebd6481926f8 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 10 06:16:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 06:16:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719515.1122296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5JY0-0007Qm-5l; Fri, 10 May 2024 06:16:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719515.1122296; Fri, 10 May 2024 06:16:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5JY0-0007Qf-2R; Fri, 10 May 2024 06:16:00 +0000
Received: by outflank-mailman (input) for mailman id 719515;
 Fri, 10 May 2024 06:15:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5JXy-0007QS-C1; Fri, 10 May 2024 06:15:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5JXy-00068z-79; Fri, 10 May 2024 06:15:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5JXx-0001ga-TQ; Fri, 10 May 2024 06:15:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5JXx-0001WV-T0; Fri, 10 May 2024 06:15:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Gi8zRcE570U/c1bOTb/5k9KWiA72FRj4xEyYIz3Z+Zs=; b=adA6oxKEUJKaFjbsGi/V0XiRMt
	cljuv5QJSnZJEomuNUdvSFGVfH8icHmnJqmBzMbStWAvA0foiedLhYAJDCnP4HXt/trqE+U75EkTR
	meW9Q7MeM95Zm5CX3tJrj7zEGpry6gKjg6cdCPCMYLRwlE6MVzDF8zyfRwTaN63NkTOk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185967-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 185967: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=4b6ee06a090d956f80b4a92fb9bf03098a372f39
X-Osstest-Versions-That:
    ovmf=3c0b84420f098c5a88281a6be370c2766184c99b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 06:15:57 +0000

flight 185967 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185967/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 4b6ee06a090d956f80b4a92fb9bf03098a372f39
baseline version:
 ovmf                 3c0b84420f098c5a88281a6be370c2766184c99b

Last test of basis   185963  2024-05-10 01:11:14 Z    0 days
Testing same since   185967  2024-05-10 04:11:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Hongbin1 Zhang <hongbin1.zhang@intel.com>
  Ray Ni <ray.ni@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3c0b84420f..4b6ee06a09  4b6ee06a090d956f80b4a92fb9bf03098a372f39 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 10 06:36:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 06:36:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719524.1122306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5JrJ-0001zY-Tz; Fri, 10 May 2024 06:35:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719524.1122306; Fri, 10 May 2024 06:35:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5JrJ-0001zR-R4; Fri, 10 May 2024 06:35:57 +0000
Received: by outflank-mailman (input) for mailman id 719524;
 Fri, 10 May 2024 06:35:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5JrH-0001zL-PT
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 06:35:55 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d4195a5-0e97-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 08:35:54 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a5a157a1cd1so398883266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 23:35:54 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17fb4sm150003366b.222.2024.05.09.23.35.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 23:35:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d4195a5-0e97-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715322954; x=1715927754; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lxCNtcupovxNdLpa4liNmJ+Aj9ToML7xaMOdcpDzR+Y=;
        b=SF7zVFeIM3i73fgLW6lkyCCl9lxuD99rZsqcGpmQs5NuoDtScuZ1DsaB5NyvM/817L
         liDGX/BoUnaoLy9FmgKaKXi3v5TcYwYVb5sBl4xABsuUphZbIgheN0gtkzEFfPVbMYyS
         x3zfMvkB7GuOHh/18/PHEK0p6JOQvy1BYVxoY6Qzuh4gC24I5GGNesJ6QXeZV5hjNywM
         WH3+VLHCBXXGmnENiUbruoasocZYQU3QMOjO9fBFqQvAp0BOAxdS3aG+kmFxrOWt4IbZ
         NKHxYK5T5RsieX/EQkr9Wwwc6IfS2WIgzUtvwnQ21woK74BGxstq7IEjP+KToeygLqMF
         SAww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715322954; x=1715927754;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lxCNtcupovxNdLpa4liNmJ+Aj9ToML7xaMOdcpDzR+Y=;
        b=pj+9ARsH0a4zGKok4ePKmxdel2A725+WyXYq6ZWUzrYwAW6eNmH5ZMYkD6N33r7Mds
         j+mByR6RIfQeAM8428WSNavK4Qpr3UOrOpYjn+SEE4/36zwEVhhI7wVGflg03SjSk9/9
         RDOdSjmZH5rRA0rmy5VAq9T5GymXUW2HO3/h4QxGykvwbubSCabWWjh1laIdHWGwkr7I
         HgJgzv8r63V4SgSpLvck9XWkCDhbJXkHkRCAVN2pp7TmhEoLTsIvljNWRTHZf3sm4oZx
         BPmBiXpnGs5HzRA+uO4IBC9Ln15BCZmm7AiSJPnqhWKNaogg/U9V4cP/7hvhiEg1Aiuu
         yWsQ==
X-Forwarded-Encrypted: i=1; AJvYcCWZNfDE1DSVkI3BeTrQoeLynL0A/uD8g3UhMbxead8a+tOissvLYxz52JnTMWBz3C8krZDPfeHvlKSVtOdT9xeZLT52vodtsAigvXCXbu4=
X-Gm-Message-State: AOJu0YyYMFJnogqopOh+PCsJMwn86dlMwWGeAFchtAuytchr0eROnQ/G
	8UR2uxatsQMdEtB/9G7UQBOQxSVV89ylmdgHqUtnUvoB9XgBOrXm5d2AyxtgSMA=
X-Google-Smtp-Source: AGHT+IEsYqtRcknYkEI9dadmuP1mmqjZ/X8PJ/ypoow+HFuD3UV69F9PwuJeDUZtGN2n3mzgj3MI8A==
X-Received: by 2002:a17:906:e297:b0:a59:afdd:590d with SMTP id a640c23a62f3a-a5a2d66a335mr85754966b.51.1715322953917;
        Thu, 09 May 2024 23:35:53 -0700 (PDT)
Message-ID: <7bc9a67e-a73a-4c12-9e35-01cb8db27fe1@suse.com>
Date: Fri, 10 May 2024 08:35:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/x86: add extra pages to unpopulated-alloc if
 available
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org,
 linux-kernel@vger.kernel.org
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>
References: <20240429155053.72509-1-roger.pau@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240429155053.72509-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 29.04.24 17:50, Roger Pau Monne wrote:
> Commit 262fc47ac174 ('xen/balloon: don't use PV mode extra memory for zone
> device allocations') removed the addition of the extra memory ranges to the
> unpopulated range allocator, using those only for the balloon driver.
> 
> This forces the unpopulated allocator to attach hotplug ranges even when spare
> memory (as part of the extra memory ranges) is available.  Furthermore, on PVH
> domains it defeats the purpose of commit 38620fc4e893 ('x86/xen: attempt to
> inflate the memory balloon on PVH'), as extra memory ranges would only be
> used to map foreign memory if the kernel is built without XEN_UNPOPULATED_ALLOC
> support.
> 
> Fix this by adding a helpers that adds the extra memory ranges to the list of
> unpopulated pages, and zeroes the ranges so they are not also consumed by the
> balloon driver.
> 
> This should have been part of 38620fc4e893, hence the fixes tag.
> 
> Note the current logic relies on unpopulated_init() (and hence
> arch_xen_unpopulated_init()) always being called ahead of balloon_init(), so
> that the extra memory regions are consumed by arch_xen_unpopulated_init().
> 
> Fixes: 38620fc4e893 ('x86/xen: attempt to inflate the memory balloon on PVH')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen



From xen-devel-bounces@lists.xenproject.org Fri May 10 06:46:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 06:46:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719530.1122316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5K1L-0003nG-Tp; Fri, 10 May 2024 06:46:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719530.1122316; Fri, 10 May 2024 06:46:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5K1L-0003n9-RE; Fri, 10 May 2024 06:46:19 +0000
Received: by outflank-mailman (input) for mailman id 719530;
 Fri, 10 May 2024 06:46:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5K1L-0003n3-5M
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 06:46:19 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00c71e91-0e99-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 08:46:17 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51ab4ee9df8so1799590e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 09 May 2024 23:46:17 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bebb660sm1526751a12.36.2024.05.09.23.46.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 May 2024 23:46:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00c71e91-0e99-11ef-909c-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715323577; x=1715928377; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SXiDu9fHyqu4n99SnTsmyeycGZXOyLvOeHSQ/rJF36g=;
        b=emQlZ/CQwZxSDNXqe0MI/11X+BjQak2a+PAdDfs/5wXFAXG5zUmn1rGKB6ULHcLGni
         rRM7XYS04E74bgBnCnxoKDXAtJ8bkceUDC5U8IU0zMSAZ/b1ws9pIEkqo1Z0O7ckwibD
         B2TTsd+6MrRcQPnv0JHSdFVbNpkZnMbd3DM2h7Ib3V4fjaCwD9B0RKBVCqNjhJelliaL
         +CXjTjXGi52fzjRp8g0FGoqTHAdwHBYWA1xcPNuOw2mMtnh5chcsqx6vtVn22s6H4SZo
         ttwgZjnNRsHBXFaFl9G2QZmvVTPtWk2GvulC/4BiOybB+qgdIfxvYT9kyRGk4TEI3Ivf
         +vig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715323577; x=1715928377;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SXiDu9fHyqu4n99SnTsmyeycGZXOyLvOeHSQ/rJF36g=;
        b=sOn77v/uQoE4Aj5ZjrlU+UMJJkOC5iqSMvyWzgR+CLjBEFt+afswGLcDwM+LdlhJci
         hlSx0TppH7VsLTGmfd9z3iPxtdZvQriL3gMPWL1o+Q6lW+unAG5v2QdYpLg2aNIyV+kl
         Bh4U5a7wjjtELeRreUT4aGdgVeRkpt3NEg2dLUj1seSCgfmA7SQ5BQEvtApOrRhmGMat
         UrCEqHL1XIRbqHB6i8CGnAzodjfbhbBauxbAQn2tiqrrkf2TRmgMQgpu/6PhOIRbPwqr
         gtQysADZbedydKqIPpUonn/pCshF2EbUJmwrFdDr9vOgD6mh33LRUv8TSYjg6xQpwWH5
         IUFA==
X-Gm-Message-State: AOJu0YyA84EjC38fmxra9tPtKSDAu4x8DPx400fcVXIOUpolA5hbSJlQ
	OVr/0rJfEnrBtqLK9541JhStJo7qwQVH9YBEBYh/XUJnYSBPrItyAB8tbuzq7zU=
X-Google-Smtp-Source: AGHT+IFqK/cSOVqRKoyq3BVFcNHdx6Uad4d+cjVO+0R0aeAeSJBQPqsxX5qu4tubXu8bA2FhNXUVRg==
X-Received: by 2002:ac2:4437:0:b0:519:2d60:d71b with SMTP id 2adb3069b0e04-5220fb748e2mr967415e87.22.1715323577232;
        Thu, 09 May 2024 23:46:17 -0700 (PDT)
Message-ID: <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
Date: Fri, 10 May 2024 08:46:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
To: Jiqian Chen <Jiqian.Chen@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org,
 Huang Rui <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240419033616.607889-4-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 19.04.24 05:36, Jiqian Chen wrote:
> In PVH dom0, it uses the linux local interrupt mechanism,
> when it allocs irq for a gsi, it is dynamic, and follow
> the principle of applying first, distributing first. And
> the irq number is alloced from small to large, but the
> applying gsi number is not, may gsi 38 comes before gsi 28,
> it causes the irq number is not equal with the gsi number.
> And when passthrough a device, QEMU will use device's gsi
> number to do pirq mapping, but the gsi number is got from
> file /sys/bus/pci/devices/<sbdf>/irq, irq!= gsi, so it will
> fail when mapping.
> And in current linux codes, there is no method to translate
> irq to gsi for userspace.
> 
> For above purpose, record the relationship of gsi and irq
> when PVH dom0 do acpi_register_gsi_ioapic for devices and
> adds a new syscall into privcmd to let userspace can get
> that translation when they have a need.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>   arch/x86/include/asm/apic.h      |  8 +++++++
>   arch/x86/include/asm/xen/pci.h   |  5 ++++
>   arch/x86/kernel/acpi/boot.c      |  2 +-
>   arch/x86/pci/xen.c               | 21 +++++++++++++++++
>   drivers/xen/events/events_base.c | 39 ++++++++++++++++++++++++++++++++
>   drivers/xen/privcmd.c            | 19 ++++++++++++++++
>   include/uapi/xen/privcmd.h       |  7 ++++++
>   include/xen/events.h             |  5 ++++
>   8 files changed, 105 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
> index 9d159b771dc8..dd4139250895 100644
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -169,6 +169,9 @@ extern bool apic_needs_pit(void);
>   
>   extern void apic_send_IPI_allbutself(unsigned int vector);
>   
> +extern int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
> +				    int trigger, int polarity);
> +
>   #else /* !CONFIG_X86_LOCAL_APIC */
>   static inline void lapic_shutdown(void) { }
>   #define local_apic_timer_c2_ok		1
> @@ -183,6 +186,11 @@ static inline void apic_intr_mode_init(void) { }
>   static inline void lapic_assign_system_vectors(void) { }
>   static inline void lapic_assign_legacy_vector(unsigned int i, bool r) { }
>   static inline bool apic_needs_pit(void) { return true; }
> +static inline int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
> +				    int trigger, int polarity)
> +{
> +	return (int)gsi;
> +}
>   #endif /* !CONFIG_X86_LOCAL_APIC */
>   
>   #ifdef CONFIG_X86_X2APIC
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
> index 9015b888edd6..aa8ded61fc2d 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -5,6 +5,7 @@
>   #if defined(CONFIG_PCI_XEN)
>   extern int __init pci_xen_init(void);
>   extern int __init pci_xen_hvm_init(void);
> +extern int __init pci_xen_pvh_init(void);
>   #define pci_xen 1
>   #else
>   #define pci_xen 0
> @@ -13,6 +14,10 @@ static inline int pci_xen_hvm_init(void)
>   {
>   	return -1;
>   }
> +static inline int pci_xen_pvh_init(void)
> +{
> +	return -1;
> +}
>   #endif
>   #ifdef CONFIG_XEN_PV_DOM0
>   int __init pci_xen_initial_domain(void);
> diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
> index 85a3ce2a3666..72c73458c083 100644
> --- a/arch/x86/kernel/acpi/boot.c
> +++ b/arch/x86/kernel/acpi/boot.c
> @@ -749,7 +749,7 @@ static int acpi_register_gsi_pic(struct device *dev, u32 gsi,
>   }
>   
>   #ifdef CONFIG_X86_LOCAL_APIC
> -static int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
> +int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
>   				    int trigger, int polarity)
>   {
>   	int irq = gsi;
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 652cd53e77f6..f056ab5c0a06 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -114,6 +114,21 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi,
>   				 false /* no mapping of GSI to PIRQ */);
>   }
>   
> +static int acpi_register_gsi_xen_pvh(struct device *dev, u32 gsi,
> +				    int trigger, int polarity)
> +{
> +	int irq;
> +
> +	irq = acpi_register_gsi_ioapic(dev, gsi, trigger, polarity);
> +	if (irq < 0)
> +		return irq;
> +
> +	if (xen_pvh_add_gsi_irq_map(gsi, irq) == -EEXIST)
> +		printk(KERN_INFO "Already map the GSI :%u and IRQ: %d\n", gsi, irq);
> +
> +	return irq;
> +}
> +
>   #ifdef CONFIG_XEN_PV_DOM0
>   static int xen_register_gsi(u32 gsi, int triggering, int polarity)
>   {
> @@ -558,6 +573,12 @@ int __init pci_xen_hvm_init(void)
>   	return 0;
>   }
>   
> +int __init pci_xen_pvh_init(void)
> +{
> +	__acpi_register_gsi = acpi_register_gsi_xen_pvh;

No support for unregistering the gsi again?

> +	return 0;
> +}
> +
>   #ifdef CONFIG_XEN_PV_DOM0
>   int __init pci_xen_initial_domain(void)
>   {
> diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
> index 27553673e46b..80d4f7faac64 100644
> --- a/drivers/xen/events/events_base.c
> +++ b/drivers/xen/events/events_base.c
> @@ -953,6 +953,43 @@ int xen_irq_from_gsi(unsigned gsi)
>   }
>   EXPORT_SYMBOL_GPL(xen_irq_from_gsi);
>   
> +int xen_gsi_from_irq(unsigned irq)
> +{
> +	struct irq_info *info;
> +
> +	list_for_each_entry(info, &xen_irq_list_head, list) {
> +		if (info->type != IRQT_PIRQ)
> +			continue;
> +
> +		if (info->irq == irq)
> +			return info->u.pirq.gsi;
> +	}
> +
> +	return -1;
> +}
> +EXPORT_SYMBOL_GPL(xen_gsi_from_irq);
> +
> +int xen_pvh_add_gsi_irq_map(unsigned gsi, unsigned irq)
> +{
> +	int tmp_irq;
> +	struct irq_info *info;
> +
> +	tmp_irq = xen_irq_from_gsi(gsi);
> +	if (tmp_irq != -1)
> +		return -EEXIST;
> +
> +	info = kzalloc(sizeof(*info), GFP_KERNEL);
> +	if (info == NULL)
> +		panic("Unable to allocate metadata for GSI%d\n", gsi);

Please don't kill the system here, just return -ENOMEM.

> +
> +	info->type = IRQT_PIRQ;
> +	info->irq = irq;
> +	info->u.pirq.gsi = gsi;
> +	list_add_tail(&info->list, &xen_irq_list_head);

I think you need some kind of locking to protect changing of the list against
concurrent accesses.

> +
> +	return 0;
> +}
> +
>   static void __unbind_from_irq(struct irq_info *info, unsigned int irq)
>   {
>   	evtchn_port_t evtchn;
> @@ -2295,6 +2332,8 @@ void __init xen_init_IRQ(void)
>   	xen_init_setup_upcall_vector();
>   	xen_alloc_callback_vector();
>   
> +	if (xen_pvh_domain())
> +		pci_xen_pvh_init();
>   
>   	if (xen_hvm_domain()) {
>   		native_init_IRQ();
> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> index 67dfa4778864..11feed529e1d 100644
> --- a/drivers/xen/privcmd.c
> +++ b/drivers/xen/privcmd.c
> @@ -842,6 +842,21 @@ static long privcmd_ioctl_mmap_resource(struct file *file,
>   	return rc;
>   }
>   
> +static long privcmd_ioctl_gsi_from_irq(struct file *file, void __user *udata)
> +{
> +	struct privcmd_gsi_from_irq kdata;
> +
> +	if (copy_from_user(&kdata, udata, sizeof(kdata)))
> +		return -EFAULT;
> +
> +	kdata.gsi = xen_gsi_from_irq(kdata.irq);
> +
> +	if (copy_to_user(udata, &kdata, sizeof(kdata)))
> +		return -EFAULT;
> +
> +	return 0;

Shouldn't you return an error if xen_gsi_from_irq() returned -1?


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 10 06:48:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 06:48:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719533.1122325 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5K3Z-0004Jz-AZ; Fri, 10 May 2024 06:48:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719533.1122325; Fri, 10 May 2024 06:48:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5K3Z-0004Js-7Y; Fri, 10 May 2024 06:48:37 +0000
Received: by outflank-mailman (input) for mailman id 719533;
 Fri, 10 May 2024 06:48:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5K3X-0004Jk-Qv
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 06:48:35 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5196ba1e-0e99-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 08:48:33 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C8EB15C8EC;
 Fri, 10 May 2024 06:48:32 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 97B361386E;
 Fri, 10 May 2024 06:48:32 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id nhkWI0DDPWZFBgAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 10 May 2024 06:48:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5196ba1e-0e99-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715323713; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=uEgMya1+RT7iEjZo0wLYBmwjGpXueiVyU2F+/IhSBsg=;
	b=ejjaDLLKulO3269xFgojakqmlYXkeLXd/8PItFRRBeOybwaTkvqMhWUzqvxG6IgB5zYbXl
	lhuyavq09z49+spvgEq1tYVcxERnhomHARFymNhmfVbEnj/rhlF2DwuiZqaQ3kVbKmzfmX
	rdf+4tOi+c6lVFnqKv6zDS7wulIDQyQ=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=nAsqXj1F
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715323712; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=uEgMya1+RT7iEjZo0wLYBmwjGpXueiVyU2F+/IhSBsg=;
	b=nAsqXj1FfMANvHFRGPkK8vjFW5mdh0s/D1wDja+UJF+C9UONqA7jZ/xmoy6pnZGW4xEXKH
	2Vhf0SA17j8L+hB7xP+xvnxCuTy7ctWYGDsT+A9P6EkFHBT8S1tvmfP9fUZmT7gJbM6MHG
	FN3px2vDASmqFQDVVrD1Va9ToBn5rKE=
Message-ID: <0c1cb3d0-5931-450a-ac7c-7f6171d5ab9a@suse.com>
Date: Fri, 10 May 2024 08:48:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1 1/1] xen/xenbus: Use *-y instead of *-objs in Makefile
To: Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
References: <20240508152658.1445809-1-andriy.shevchenko@linux.intel.com>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240508152658.1445809-1-andriy.shevchenko@linux.intel.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------XVUPab31CzwcWFB0Zc6iX48J"
X-Spam-Flag: NO
X-Spam-Score: -7.40
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: C8EB15C8EC
X-Spam-Level: 
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-7.40 / 50.00];
	BAYES_HAM(-3.00)[100.00%];
	SIGNED_PGP(-2.00)[];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	MIME_BASE64_TEXT_BOGUS(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_BASE64_TEXT(0.10)[];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MX_GOOD(-0.01)[];
	XM_UA_NO_VERSION(0.01)[];
	TO_DN_SOME(0.00)[];
	SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	ARC_NA(0.00)[];
	FROM_HAS_DN(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	RCPT_COUNT_FIVE(0.00)[5];
	MID_RHS_MATCH_FROM(0.00)[];
	DKIM_TRACE(0.00)[suse.com:+];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	HAS_ATTACHMENT(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[intel.com:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------XVUPab31CzwcWFB0Zc6iX48J
Content-Type: multipart/mixed; boundary="------------uMya3f1c4AAUHhW0t6Lr1DEW";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Message-ID: <0c1cb3d0-5931-450a-ac7c-7f6171d5ab9a@suse.com>
Subject: Re: [PATCH v1 1/1] xen/xenbus: Use *-y instead of *-objs in Makefile
References: <20240508152658.1445809-1-andriy.shevchenko@linux.intel.com>
In-Reply-To: <20240508152658.1445809-1-andriy.shevchenko@linux.intel.com>

--------------uMya3f1c4AAUHhW0t6Lr1DEW
Content-Type: multipart/mixed; boundary="------------UaG8iQnktprvcDJU1dUIBzjF"

--------------UaG8iQnktprvcDJU1dUIBzjF
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMDguMDUuMjQgMTc6MjYsIEFuZHkgU2hldmNoZW5rbyB3cm90ZToNCj4gKi1vYmpzIHN1
ZmZpeCBpcyByZXNlcnZlZCByYXRoZXIgZm9yICh1c2VyLXNwYWNlKSBob3N0IHByb2dyYW1z
IHdoaWxlDQo+IHVzdWFsbHkgKi15IHN1ZmZpeCBpcyB1c2VkIGZvciBrZXJuZWwgZHJpdmVy
cyAoYWx0aG91Z2ggKi1vYmpzIHdvcmtzDQo+IGZvciB0aGF0IHB1cnBvc2UgZm9yIG5vdyku
DQo+IA0KPiBMZXQncyBjb3JyZWN0IHRoZSBvbGQgdXNhZ2VzIG9mICotb2JqcyBpbiBNYWtl
ZmlsZXMuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBBbmR5IFNoZXZjaGVua28gPGFuZHJpeS5z
aGV2Y2hlbmtvQGxpbnV4LmludGVsLmNvbT4NCg0KQWNrZWQtYnk6IEp1ZXJnZW4gR3Jvc3Mg
PGpncm9zc0BzdXNlLmNvbT4NCg0KDQpKdWVyZ2VuDQoNCg==
--------------UaG8iQnktprvcDJU1dUIBzjF
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------UaG8iQnktprvcDJU1dUIBzjF--

--------------uMya3f1c4AAUHhW0t6Lr1DEW--

--------------XVUPab31CzwcWFB0Zc6iX48J
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmY9w0AFAwAAAAAACgkQsN6d1ii/Ey+w
fwf+JlyryXgxIHm63o4c7G+tB662b+z0puAWOF10bzMWP7Llu1gGBRYYn89BX4Aajrr7E3z6mM09
/CvVfChbMDeVML2eOn/8FhsxRKjoMU6SVfK84C73UNaGc84nEl7jvRtaOfKAdVWgwAMvhqODzRtb
+X/m/Ir/9aQKO+88Ryl71523pn+hm88DDHIYMAknkF8UyC8IAehvrTlRFULXW7xG2Vd/6QqhHTC9
EfvhTLY84xBJfOWRMC35zfsa4UuB1wfvrpI3cyoJRIexX1ahbcbxGtK8ABpsBHfMOn36fzkOMzfe
mjMfHG8ccKlSHexVMlYqWqMXrnCJaPoQWKqMlinL6Q==
=Jw5o
-----END PGP SIGNATURE-----

--------------XVUPab31CzwcWFB0Zc6iX48J--


From xen-devel-bounces@lists.xenproject.org Fri May 10 07:38:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 07:38:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719542.1122336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5KpD-0003QB-2r; Fri, 10 May 2024 07:37:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719542.1122336; Fri, 10 May 2024 07:37:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5KpC-0003Q4-Uw; Fri, 10 May 2024 07:37:50 +0000
Received: by outflank-mailman (input) for mailman id 719542;
 Fri, 10 May 2024 07:37:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+rhO=MN=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s5KpB-0003Py-PD
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 07:37:49 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31c8121d-0ea0-11ef-909c-e314d9c70b13;
 Fri, 10 May 2024 09:37:47 +0200 (CEST)
Received: from CH0PR04CA0109.namprd04.prod.outlook.com (2603:10b6:610:75::24)
 by MW3PR12MB4361.namprd12.prod.outlook.com (2603:10b6:303:5a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 07:37:44 +0000
Received: from CH1PEPF0000AD7F.namprd04.prod.outlook.com
 (2603:10b6:610:75:cafe::ad) by CH0PR04CA0109.outlook.office365.com
 (2603:10b6:610:75::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.51 via Frontend
 Transport; Fri, 10 May 2024 07:37:44 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD7F.mail.protection.outlook.com (10.167.244.88) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7452.22 via Frontend Transport; Fri, 10 May 2024 07:37:43 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 02:37:43 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 02:37:43 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 02:37:41 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31c8121d-0ea0-11ef-909c-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V2FZHqx4tTnnsdo/8XVzfXTK9kKlHFuPIu7Kr8fqCkjGtOsyi64vW0oKyiw35ZpZBMNq/vm0/PPKeWYneE8AgSy2LlrUEwp2EZlsGbKM8wm6uvexiCKlMeNYliQJk4r0bwvtmCvQSosKFMBH8OajS3kHkhw68LywUbuwHXNjANXvFzT+r1HxLXA3W+WrU9JJ/MY9JbikcpKvoD8xgNfu9PkJj4E3jYkU7dSkIAXQB0e25ccdrHkCkgu+UdUYGwixYxiJ5VHWNdBH4qqfqvCqnzGnbaTZ/BzsTZyhOgQjJeRbItdnm3/CLjbnvceH5cwOd6ky3hT7ZfJdyAH2kENHgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=48cBJhgTwMMujLUdyxPhBeMKf9PaNp9uYn1TxzAT8P0=;
 b=fUkH0pkyiOveG8XlZuWC60EfR1MNAhrOvPulapdFxH6NCVsMSoVYPkr6b1s0eV0p7g8TwbDp8+gxlbdtM1x0VxEqPik9yWyTAvEbDGlE5byF+/CRfJFxz/r32GZfqscB44JimP388MsUjbYlg92rDZZBsJFoOQNHK1m0n+pTAvtwWUhbhK1Ui1hCYsg8UPEjdTg7C4p5UVFfpI5CwrM8EzszzDlfqMGxxlVbtjlEIbmKh7TvGEsFPJytOwI0QQjNtN1ChgSGm0nbykhjfXp8hqapRhbV06yqVX2hTUxxpWK/PS3FSMSfJd5LFVGWnER7DxGleHLlReYPGxza7Tbn8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=48cBJhgTwMMujLUdyxPhBeMKf9PaNp9uYn1TxzAT8P0=;
 b=jz3pXWG9Tv0ud6UQrpGQ8uUYv2DRHO+C6nW+mwWqQzH8eIMbdzamPIvmq3pWw0P/CsqArG/vHGVaqUT71DE+mmk9J1aJAwdqvliiKzXrcDCcN/cIZC80OvAZkwrjGlXDAI6Tqpgmc3/FtDwOHmILSGGGiVvizquh8VgfaKg9QU4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <5d3ead96-5079-4fa3-b5fd-4d9803c251b6@amd.com>
Date: Fri, 10 May 2024 09:37:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] xen/arm/dom0less-build: Alloc magic pages for
 Dom0less DomUs from hypervisor
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
References: <20240426031455.579637-1-xin.wang2@amd.com>
 <20240426031455.579637-2-xin.wang2@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240426031455.579637-2-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7F:EE_|MW3PR12MB4361:EE_
X-MS-Office365-Filtering-Correlation-Id: 4ac4e60a-3a36-4a8c-91ff-08dc70c4147e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?aUQweVRaRG5SNEFjY0ZZbmROMHMreVFkQmdUN3phdmF6bWNrMmM5TW5sanlN?=
 =?utf-8?B?Q1ptSUdwdHJNcUNRL0lHTUtmSmVvb2dSVXN1cTN5dnpWZktLNEwyeU5MMm1X?=
 =?utf-8?B?YTVIdUpLbUM5dHNGbFFQdzRzdHdNRTFuMEZPUlErY1FMeGZHdjBFdUtFRkJp?=
 =?utf-8?B?QXFNczFzZExKOHJwekJrRXVxbGZpSC84R09KNnRENStlTVlIb0xrRit5cFpQ?=
 =?utf-8?B?SVlaMDRWcng4OEhOYXpvSS9admZGOWJGVUdqRGRuak84M2xQVkxxb1FCeTJY?=
 =?utf-8?B?cXZjMGxjOUhNdVJsRzl6eHFjeHpLWXh4MUJRQ1dNWDZkTGZmcEgvYmNxRTRN?=
 =?utf-8?B?NVVrVEpFRERUNGhqbEF2bnVPTmZVbE56d2dwUStjeGlXblY2aXpQbkRkSkd0?=
 =?utf-8?B?WHpYR0NMWTNLZXNHV0pTTm1OaHBzenZ3N0c1S3grVVpaL09SSGxLY1hkWlNv?=
 =?utf-8?B?UVE4a01KU256azBlV3o5cHBrWE96RUQyMnNQY2RJOG1vQU42Q2IxMS9HM0lM?=
 =?utf-8?B?YXNyVldxK3cxUlpTRlEva0IyM09vcER2bWpRcXAyaExTVWFITC9YOGxxR1Fh?=
 =?utf-8?B?aG9TcEd2eW1UdWJiM2dpRi9xWHorcTJhOU9HVmZocG9LQkJYVzI0Tyt4UkRo?=
 =?utf-8?B?Mmt1bU5KcUcxRnBhL2VzZWc5U0lFYmc5NWRkQ3dzd2c5OXZ6TVJPY0x5em1H?=
 =?utf-8?B?UnhtZ1crN2Rjd2YxbjFNRGpMN1ozUmFrMS9NV1lsUGtseFd2Qll2L2psSHJL?=
 =?utf-8?B?MEFTc05LUUE4cWpzRGthS2dFTXMwTnlYdld6VHJYOFMwWVpPWVZMQ1dIZTd5?=
 =?utf-8?B?L09EUEczem9QMDZXcy9HM2lNNHJoM0hwTFpZN1FLRDBlSTdBRkNramd2UnBq?=
 =?utf-8?B?VXFKTDNwMG9GL0JxRXdkOGJvU0hTYWVRT09rajBMOVlSdytJUzZrS2Y3bGdB?=
 =?utf-8?B?U0VRamFrZDMyUGNrWnFKOENDQVAzQnpVcVp0SjZsOThsZ1c2TlpxczJTeEZx?=
 =?utf-8?B?V0JLYUdtcldIdmlmVnl1cHI4cVhKU1NyRzBYcmIrb2ZFeFprd0FIRmpESHpn?=
 =?utf-8?B?eWs5dW5hNmlJbldGZEdJY0xnWVVnM1RRUnlwZUJjNkx2RmFud1creHlkbUox?=
 =?utf-8?B?bG01L24zbSs5ZTlkb1RDaVV2c3dpcVBGOUVVSzZpVUk4VWZjbWlLQjlBOW80?=
 =?utf-8?B?Sk5JOStucmQwZEV6ZXBhMHdaNDlnd0ZiNnZ4aC9nSm1HbStVbHBUeUQzajJP?=
 =?utf-8?B?YnlBNzNuUzlVdXA1QVBEajJvMFAvc2ZYdWJjdEhMMEdWODYyQk1VS2lyTmFG?=
 =?utf-8?B?d2pVaEhIalZYSkQrVm1pWFJjYjcrSFQvcEdpYUhRTDc3OUhHZWEwakZvYms5?=
 =?utf-8?B?eThFMzZGcnpKTit2TnZ6aCsxQ0pncm5wWTBEZ0pjdU1CMEJvZEpxdXppUlRD?=
 =?utf-8?B?dXhIOGZ6S0pLS09zeTlKMTVHcURmZU1lanRxWjl4ZXFHaUIvTm1BZ1AvU1Yw?=
 =?utf-8?B?aXA0Qlcxc1FXakt5ZVpxbE5nUk9Fb3ZjS0RGV2lwdWV0cUNvUkh1Um1BNUdx?=
 =?utf-8?B?aExUVVhSS2pMRXplNkdtZWd6NWEvUHJLZmx2eXZSTHVhWnV0QVZSSEx0VVc3?=
 =?utf-8?B?cXd6RFJTMlFOSHM3RG5FeFFhYi9wUDlMNm1SdEttMC9leUFZZ3FVUnJYeUhy?=
 =?utf-8?B?YmNmWXVpT2lpbWF1d1Mrc2dlRTd1MTdqbytYclR4WldLNnJxdHY3NmhiVE1x?=
 =?utf-8?B?dGZvSXhGbjB5L1EwUjF5SlF6YW1FdG52VHIrbk04Y21NeVF6aXhPS3d5SCt5?=
 =?utf-8?B?MmlMZnpMdXNwYUdNZ3o4c2h2elZkZE1hd0tBQ3k1cnJ2am1CL0wvdThYMDlo?=
 =?utf-8?Q?X1s3zTDuC/0+v?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 07:37:43.8882
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4ac4e60a-3a36-4a8c-91ff-08dc70c4147e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD7F.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4361

Hi Henry,

On 26/04/2024 05:14, Henry Wang wrote:
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> This is because currently the magic pages for Dom0less DomUs are
> populated by the init-dom0less app through populate_physmap(), and
> populate_physmap() automatically assumes gfn == mfn for 1:1 direct
> mapped domains. This cannot be true for the magic pages that are
> allocated later from the init-dom0less application executed in Dom0.
> For domain using statically allocated memory but not 1:1 direct-mapped,
> similar error "failed to retrieve a reserved page" can be seen as the
> reserved memory list is empty at that time.
> 
> To solve above issue, this commit allocates the magic pages for
> Dom0less DomUs at the domain construction time. The base address/PFN
> of the magic page region will be noted and communicated to the
> init-dom0less application in Dom0.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>  tools/libs/guest/xg_dom_arm.c |  1 -
>  xen/arch/arm/dom0less-build.c | 22 ++++++++++++++++++++++
>  xen/include/public/arch-arm.h |  1 +
>  3 files changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
> index 2fd8ee7ad4..8cc7f27dbb 100644
> --- a/tools/libs/guest/xg_dom_arm.c
> +++ b/tools/libs/guest/xg_dom_arm.c
> @@ -25,7 +25,6 @@
>  
>  #include "xg_private.h"
>  
> -#define NR_MAGIC_PAGES 4
Moving only this macro to arch-arm.h while leaving the offsets does not make much sense to me.
I think they all should be moved. This would also allow init-dom0less.h not to re-define XENSTORE_PFN_OFFSET.

>  #define CONSOLE_PFN_OFFSET 0
>  #define XENSTORE_PFN_OFFSET 1
>  #define MEMACCESS_PFN_OFFSET 2
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index fb63ec6fd1..40dc85c759 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -834,11 +834,33 @@ static int __init construct_domU(struct domain *d,
>  
>      if ( kinfo.dom0less_feature & DOM0LESS_XENSTORE )
>      {
> +        struct page_info *magic_pg;
> +        mfn_t mfn;
> +        gfn_t gfn;
> +
>          ASSERT(hardware_domain);
>          rc = alloc_xenstore_evtchn(d);
>          if ( rc < 0 )
>              return rc;
>          d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
> +
> +        d->max_pages += NR_MAGIC_PAGES;
> +        magic_pg = alloc_domheap_pages(d, get_order_from_pages(NR_MAGIC_PAGES), 0);
80 char exceeded

> +        if ( magic_pg == NULL )
> +            return -ENOMEM;
> +
> +        mfn = page_to_mfn(magic_pg);
> +        if ( !is_domain_direct_mapped(d) )
> +            gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
> +        else
> +            gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
> +
> +        rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
> +        if ( rc )
> +        {
> +            free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
> +            return rc;
> +        }
Please create a function alloc_magic_pages to encapsulate the above block.

>      }
>  
>      return rc;
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index e167e14f8d..f24e7bbe37 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -475,6 +475,7 @@ typedef uint64_t xen_callback_t;
>  
>  #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>  #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
> +#define NR_MAGIC_PAGES 4
>  
>  #define GUEST_RAM_BANKS   2
>  

~Michal



From xen-devel-bounces@lists.xenproject.org Fri May 10 07:42:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 07:42:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719546.1122346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Ktz-0004rr-JQ; Fri, 10 May 2024 07:42:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719546.1122346; Fri, 10 May 2024 07:42:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Ktz-0004rk-GA; Fri, 10 May 2024 07:42:47 +0000
Received: by outflank-mailman (input) for mailman id 719546;
 Fri, 10 May 2024 07:42:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lVkm=MN=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5Kty-0004re-0Y
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 07:42:46 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e1fe63a7-0ea0-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 09:42:43 +0200 (CEST)
Received: from CH0PR03CA0409.namprd03.prod.outlook.com (2603:10b6:610:11b::17)
 by SJ1PR12MB6076.namprd12.prod.outlook.com (2603:10b6:a03:45d::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48; Fri, 10 May
 2024 07:42:39 +0000
Received: from CH2PEPF00000140.namprd02.prod.outlook.com
 (2603:10b6:610:11b:cafe::31) by CH0PR03CA0409.outlook.office365.com
 (2603:10b6:610:11b::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48 via Frontend
 Transport; Fri, 10 May 2024 07:42:39 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH2PEPF00000140.mail.protection.outlook.com (10.167.244.72) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Fri, 10 May 2024 07:42:39 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 02:42:35 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 02:42:35 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 02:42:33 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1fe63a7-0ea0-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LDkTKADkdvAUiTzzWSCT4qHLQFP/jJblZ4fQQgkJ6gsmk9zMtTt3EX9o0LszOM98YguF3JjeiKsL3GOa/Ood+dba0ybMXyXokTKYtNWBnWB7N8UEz4TBCOCznd0Bef4W+RYo8sRb7LE0Lb3mmkMy8v8g+VVytJuwlFq3RCSfTpDYkR2NxCYGISmS8NL9bMAI8Gn/jV6GW983BvI1sjArO/7WrJvVTSlWup4Z9yRBaL8jx4wuWAuYmkJ9yo5wrsDcvn2/O2kSudagxNU5z+UtS7gRcWYFqJ/hDPnhvt7AK4rbGI2njLJTkI4gdEB9Oadatjiwq4idVzA4DBjQB1M+5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=aZeDQT3YDD/Cq5x2elGP1TIeC8Q6qpFMEIdHpnkqlAk=;
 b=YXnWVPSpJDNzb3t/aB/izFrEsIBBYOB3jgNYFYnHt4ud8CJmTyKa1NmL2YOrMFEArsMjtzeuqi+J3KgnKSIbqFTVomi6EMfGTLGwu5nY2eeXVh9L47pDOEakPxTmlGidDUdVWklRs6iJPEKq9Sg8cnZZW/IQePO9IIQi+VCvL9T0Q/xKrW3z9w61VOGxdXra4gXkLdcKcRRfQXDwA2spuj26rpT/OJ8Q1ZJXvsHfEiB4OVrGLd4TjAGLaHJcB+63lNSAcJpRiUUv4B4wwZOIO3nwK5ULM4VVygMW7ZJdVtTmkQz7IEz+U90q/LhC2z2fpoB/ix/Zjg+DtbJFD/7VdQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aZeDQT3YDD/Cq5x2elGP1TIeC8Q6qpFMEIdHpnkqlAk=;
 b=d34UITI3og7MF5HesAMyLoQSbI01UX21yAVDIEm78naO9oaIN8jputJzqgL/B17Lrv7uthFsVIMFjvHuP+4ZeWNZ8HNLWzLz8fKNd3WhcebEXE6aF6jIyQH1qKy0RycHANxsEUsoIPCZ+7UXTBKA1hlqGGp1oplqZS6UD+0jwkI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <d357beeb-6874-4262-8137-8b4145fd64c4@amd.com>
Date: Fri, 10 May 2024 15:42:32 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] xen/arm/dom0less-build: Alloc magic pages for
 Dom0less DomUs from hypervisor
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
References: <20240426031455.579637-1-xin.wang2@amd.com>
 <20240426031455.579637-2-xin.wang2@amd.com>
 <5d3ead96-5079-4fa3-b5fd-4d9803c251b6@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <5d3ead96-5079-4fa3-b5fd-4d9803c251b6@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000140:EE_|SJ1PR12MB6076:EE_
X-MS-Office365-Filtering-Correlation-Id: a4127491-8a4e-4148-de40-08dc70c4c471
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RWxtWVlYMHgxNlROR1BVMi9sRCtRRHRqRDEvb0wyb0c1eUJWbWJyODlmWlhW?=
 =?utf-8?B?Ui9nSU1Gb3pzdkZnWXExNWFtbjBUVjM2c0NjeEdwekZwekFhRllHaDZUelNl?=
 =?utf-8?B?d0JxNHpMaEd6K21STjRrYlBpdVFFeUwwbkZSNHF2OXVBZHJQWlNZQm9VZWc1?=
 =?utf-8?B?TmM4OE5ZYUdieVNTZjNjVkpoSHFtWm9SMWthZGRlUTAwbzRObWp4c28ycmRn?=
 =?utf-8?B?c1Izb01kUGlrNm10dnhUNkx3b1JXMDUwa2ZSUlpIMmZxeGZlTjJSMnJlZDg5?=
 =?utf-8?B?b2FIK2I0dXF2QzdRb0FlRmtDUXp1RE0ycjhHTFcxb0VQdmZEYkxsTTNzcm5m?=
 =?utf-8?B?UUxNTmtBaXRjai93ZlFXSHBmU2R5dDhOVkJvK1pVdTBaYjBPWXVJbFBHbW1W?=
 =?utf-8?B?Uk1BdThqT3hHTkVmS1NySjJ0Wi9RZkFWeFFnTEtMTkdsWlNycUpkd1JmUk1U?=
 =?utf-8?B?QUJqTUhuRjJHMkRCVkNQM0NsWVovZzlpZ0k1aE0xQVdCc25mYWcwQkJoY0Rt?=
 =?utf-8?B?eC9OdDdYQ3BFeDNETWJlOU4zTGxSelVCQXdzcXY0WUN3TEMwVDBjUk9BYkRX?=
 =?utf-8?B?eGZZYWJSc0ExYUZlR2RER01KTXZTdTk3RGhuaFZrckU2V0UyS25sNWZaamQx?=
 =?utf-8?B?MHN1KytmOHZFZEVWRXc2NEx5TjdBYzlIZWJVWUlyVkJkVFhwRS9RazRSMGF2?=
 =?utf-8?B?Z25UdFVSTHE2TURXVXBJTGd2ckFIeng2SXUyekZkYzBhK0ZoZyt5NFVDQVVw?=
 =?utf-8?B?MWNpYVhHcjFzS1ZURFhzd1pLNCtNY2JLLy9XWkMxRmdBM0pSWDRZODRPMGph?=
 =?utf-8?B?R1hObjdYZng3TCtFOStSc1hocHlsbTFoWTJ2UXpmVENCcE1ybmUzTkRqWEY1?=
 =?utf-8?B?SU0rRUV5OGM5aUpRNVNvSXFiWFpUQmVQSFhWSWZaV0QrS253WHc5WEsrc2Nh?=
 =?utf-8?B?dmR4N2NRZTNNMlFoOTNadTRhRFhCOXNBSGtqdnFCLzVCdFM2cHdyRHozYnRu?=
 =?utf-8?B?Qkw2WWpEUHJzOTduYzNyU3ZCU3dHTlgrYURRU3FRUzdNNGwrbTY5MFhCMnBl?=
 =?utf-8?B?Q25TVmlaQWN5VGJvVFJjNFJBcEdTOXV2SlFqMkM3ZzhVN1drcnlScWFnWXRZ?=
 =?utf-8?B?UWVCSGtEa2ZKa0pmRWI3c1d2bkZxRGR4SnNKdDNEWWkrMWhFUWhHa3JVbGEw?=
 =?utf-8?B?OE9VRVN4K1BhRVF4dmJReWJhMm04NkdxeGVMV3BlT3pRRG56UFRvNmFJRDRw?=
 =?utf-8?B?Q0FvVjFEUFhVMnZRMlZYeTRQTmpUTndjTHljZHdZcklOejhPWWtySlIwTHpr?=
 =?utf-8?B?STVoVlR6VlVTZVhiQW9PVUczOWd4eFZaUWhNOFhhdnZxTnF2NEhjY2tLQS9o?=
 =?utf-8?B?cFNsbmJLNHNWd2o3bkdjM292N1dQV0dCalhPQVM4OVYrSW1WYVhjTGpvQkpn?=
 =?utf-8?B?QkpQMGNWSzNDT2NWMVZ2dmlpRy9mbE9Ya294Um43RFVMYU5TNEx5Q1lXNGVD?=
 =?utf-8?B?WGlwbXZ3Z1R3WjJKZFlZbHQyb3U1REIxNWN6SU5hSGp3c0N3cGxlZlV3TThR?=
 =?utf-8?B?YWlOejNHUW90eUdiZEc2NnFkTWtsd3RyMS94akpFZ2ZYTTIxS0FQTXEwVWRL?=
 =?utf-8?B?WjZrWnFJWTkvRS9HVXNGY3Y5NzRUWXIveXBwVXF0RU1nUWdrRkFXYnJYd0FJ?=
 =?utf-8?B?WklNODYzVjhuTnUweWwzVkFUK3FJdFpXVTZCeUZ0dnBOaG5pa0haeUt4aFM4?=
 =?utf-8?B?djQyY2ZFTU4xdU52K25yM2FEL2NLRUZIbU0zeDAvTy96YzdkejRpUEhSK3Y0?=
 =?utf-8?B?eXJsbWJKTDZuN2tKOUp3cTIraW5qOXhXMVNMZjFuYUs5czMveXR1Sjh0aVJ1?=
 =?utf-8?Q?Yj0nHrrHxya59?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 07:42:39.0840
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a4127491-8a4e-4148-de40-08dc70c4c471
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000140.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6076

Hi Michal,

Thanks very much for taking a look!

On 5/10/2024 3:37 PM, Michal Orzel wrote:
> Hi Henry,
>
> On 26/04/2024 05:14, Henry Wang wrote:
>> There are use cases (for example using the PV driver) in Dom0less
>> setup that require Dom0less DomUs start immediately with Dom0, but
>> initialize XenStore later after Dom0's successful boot and call to
>> the init-dom0less application.
>>
>> An error message can seen from the init-dom0less application on
>> 1:1 direct-mapped domains:
>> ```
>> Allocating magic pages
>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>> Error on alloc magic pages
>> ```
>> This is because currently the magic pages for Dom0less DomUs are
>> populated by the init-dom0less app through populate_physmap(), and
>> populate_physmap() automatically assumes gfn == mfn for 1:1 direct
>> mapped domains. This cannot be true for the magic pages that are
>> allocated later from the init-dom0less application executed in Dom0.
>> For domain using statically allocated memory but not 1:1 direct-mapped,
>> similar error "failed to retrieve a reserved page" can be seen as the
>> reserved memory list is empty at that time.
>>
>> To solve above issue, this commit allocates the magic pages for
>> Dom0less DomUs at the domain construction time. The base address/PFN
>> of the magic page region will be noted and communicated to the
>> init-dom0less application in Dom0.
>>
>> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
>> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>>   tools/libs/guest/xg_dom_arm.c |  1 -
>>   xen/arch/arm/dom0less-build.c | 22 ++++++++++++++++++++++
>>   xen/include/public/arch-arm.h |  1 +
>>   3 files changed, 23 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
>> index 2fd8ee7ad4..8cc7f27dbb 100644
>> --- a/tools/libs/guest/xg_dom_arm.c
>> +++ b/tools/libs/guest/xg_dom_arm.c
>> @@ -25,7 +25,6 @@
>>   
>>   #include "xg_private.h"
>>   
>> -#define NR_MAGIC_PAGES 4
> Moving only this macro to arch-arm.h while leaving the offsets does not make much sense to me.
> I think they all should be moved. This would also allow init-dom0less.h not to re-define XENSTORE_PFN_OFFSET.

Sounds good. Will do in v2.

>>   #define CONSOLE_PFN_OFFSET 0
>>   #define XENSTORE_PFN_OFFSET 1
>>   #define MEMACCESS_PFN_OFFSET 2
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>> index fb63ec6fd1..40dc85c759 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -834,11 +834,33 @@ static int __init construct_domU(struct domain *d,
>>   
>>       if ( kinfo.dom0less_feature & DOM0LESS_XENSTORE )
>>       {
>> +        struct page_info *magic_pg;
>> +        mfn_t mfn;
>> +        gfn_t gfn;
>> +
>>           ASSERT(hardware_domain);
>>           rc = alloc_xenstore_evtchn(d);
>>           if ( rc < 0 )
>>               return rc;
>>           d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
>> +
>> +        d->max_pages += NR_MAGIC_PAGES;
>> +        magic_pg = alloc_domheap_pages(d, get_order_from_pages(NR_MAGIC_PAGES), 0);
> 80 char exceeded

Ooops, I am sorry. Will fix in v2.

>> +        if ( magic_pg == NULL )
>> +            return -ENOMEM;
>> +
>> +        mfn = page_to_mfn(magic_pg);
>> +        if ( !is_domain_direct_mapped(d) )
>> +            gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
>> +        else
>> +            gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
>> +
>> +        rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
>> +        if ( rc )
>> +        {
>> +            free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
>> +            return rc;
>> +        }
> Please create a function alloc_magic_pages to encapsulate the above block.

Sure. Will do.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Fri May 10 07:48:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 07:48:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719550.1122355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5KzZ-0005yg-6o; Fri, 10 May 2024 07:48:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719550.1122355; Fri, 10 May 2024 07:48:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5KzZ-0005yZ-46; Fri, 10 May 2024 07:48:33 +0000
Received: by outflank-mailman (input) for mailman id 719550;
 Fri, 10 May 2024 07:48:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5KzY-0005yT-Io
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 07:48:32 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b205ae97-0ea1-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 09:48:31 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 53F4161170;
 Fri, 10 May 2024 07:48:30 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E754913A44;
 Fri, 10 May 2024 07:48:29 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id bEe3Nk3RPWbhFwAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 10 May 2024 07:48:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b205ae97-0ea1-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715327310; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=OegCOaYo2ZMoCAcon3ZgxTonFDo/ULoFAXa2mqPDvU8=;
	b=mCpBEAvrYi4ed7oOhfELeIGhqjHno4X1j+Ev95BGTQLyQDoZDV87aCDznsyrezYd15EavY
	MY9gpbhdbUR7FOIT0A3vEkqps5LtOAOQAU2LsXAL9BDWscarb0VD7B0KbJ+ow+nyZSiyNY
	mUciW46qpReBmJhWOq0DMpf3hx+ShC8=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=mCpBEAvr
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715327310; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=OegCOaYo2ZMoCAcon3ZgxTonFDo/ULoFAXa2mqPDvU8=;
	b=mCpBEAvrYi4ed7oOhfELeIGhqjHno4X1j+Ev95BGTQLyQDoZDV87aCDznsyrezYd15EavY
	MY9gpbhdbUR7FOIT0A3vEkqps5LtOAOQAU2LsXAL9BDWscarb0VD7B0KbJ+ow+nyZSiyNY
	mUciW46qpReBmJhWOq0DMpf3hx+ShC8=
Message-ID: <9ebdcc0b-3775-4c5e-9ffb-f45d048281dd@suse.com>
Date: Fri, 10 May 2024 09:48:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 2/3] xen/pvh: Setup gsi for passthrough
 device
To: Jiqian Chen <Jiqian.Chen@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org,
 Huang Rui <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-3-Jiqian.Chen@amd.com>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240419033616.607889-3-Jiqian.Chen@amd.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------Knae4IyoLuapUbvuflU12aH9"
X-Spam-Level: 
X-Spamd-Result: default: False [-8.40 / 50.00];
	BAYES_HAM(-3.00)[100.00%];
	SIGNED_PGP(-2.00)[];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	MX_GOOD(-0.01)[];
	XM_UA_NO_VERSION(0.01)[];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	ARC_NA(0.00)[];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	FROM_HAS_DN(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	TO_DN_SOME(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCPT_COUNT_SEVEN(0.00)[10];
	DKIM_TRACE(0.00)[suse.com:+];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	HAS_ATTACHMENT(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: 53F4161170
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Flag: NO
X-Spam-Score: -8.40

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------Knae4IyoLuapUbvuflU12aH9
Content-Type: multipart/mixed; boundary="------------O3s0i0TBCQzGkMudaipiCF2z";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jiqian Chen <Jiqian.Chen@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org,
 Huang Rui <Ray.Huang@amd.com>
Message-ID: <9ebdcc0b-3775-4c5e-9ffb-f45d048281dd@suse.com>
Subject: Re: [RFC KERNEL PATCH v6 2/3] xen/pvh: Setup gsi for passthrough
 device
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-3-Jiqian.Chen@amd.com>
In-Reply-To: <20240419033616.607889-3-Jiqian.Chen@amd.com>

--------------O3s0i0TBCQzGkMudaipiCF2z
Content-Type: multipart/mixed; boundary="------------SncFPlxTg8DkoLMqUFXsOL99"

--------------SncFPlxTg8DkoLMqUFXsOL99
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTkuMDQuMjQgMDU6MzYsIEppcWlhbiBDaGVuIHdyb3RlOg0KPiBJbiBQVkggZG9tMCwg
dGhlIGdzaXMgZG9uJ3QgZ2V0IHJlZ2lzdGVyZWQsIGJ1dCB0aGUgZ3NpIG9mDQo+IGEgcGFz
c3Rocm91Z2ggZGV2aWNlIG11c3QgYmUgY29uZmlndXJlZCBmb3IgaXQgdG8gYmUgYWJsZSB0
byBiZQ0KPiBtYXBwZWQgaW50byBhIGRvbVUuDQo+IA0KPiBXaGVuIGFzc2lnbiBhIGRldmlj
ZSB0byBwYXNzdGhyb3VnaCwgcHJvYWN0aXZlbHkgc2V0dXAgdGhlIGdzaQ0KPiBvZiB0aGUg
ZGV2aWNlIGR1cmluZyB0aGF0IHByb2Nlc3MuDQo+IA0KPiBDby1kZXZlbG9wZWQtYnk6IEh1
YW5nIFJ1aSA8cmF5Lmh1YW5nQGFtZC5jb20+DQo+IFNpZ25lZC1vZmYtYnk6IEppcWlhbiBD
aGVuIDxKaXFpYW4uQ2hlbkBhbWQuY29tPg0KPiBSZXZpZXdlZC1ieTogU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KDQpUaGlzIHBhdGNoIGlzIGJyZWFr
aW5nIHRoZSBidWlsZC4NCg0KT24gQXJtIEkgZ2V0Og0KDQpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gL2hvbWUvZ3Jvc3Mva29yZy9zcmMvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0
dWIuYzoyMzowOg0KL2hvbWUvZ3Jvc3Mva29yZy9zcmMvaW5jbHVkZS94ZW4vYWNwaS5oOiBJ
biBmdW5jdGlvbiAneGVuX2FjcGlfc2xlZXBfcmVnaXN0ZXInOg0KL2hvbWUvZ3Jvc3Mva29y
Zy9zcmMvaW5jbHVkZS94ZW4vYWNwaS5oOjY3OjM6IGVycm9yOiAnYWNwaV9zdXNwZW5kX2xv
d2xldmVsJyANCnVuZGVjbGFyZWQgKGZpcnN0IHVzZSBpbiB0aGlzIGZ1bmN0aW9uKTsgZGlk
IHlvdSBtZWFuICd4ZW5fYWNwaV9zdXNwZW5kX2xvd2xldmVsJz8NCiAgICBhY3BpX3N1c3Bl
bmRfbG93bGV2ZWwgPSB4ZW5fYWNwaV9zdXNwZW5kX2xvd2xldmVsOw0KICAgIF5+fn5+fn5+
fn5+fn5+fn5+fn5+fg0KICAgIHhlbl9hY3BpX3N1c3BlbmRfbG93bGV2ZWwNCi9ob21lL2dy
b3NzL2tvcmcvc3JjL2luY2x1ZGUveGVuL2FjcGkuaDo2NzozOiBub3RlOiBlYWNoIHVuZGVj
bGFyZWQgaWRlbnRpZmllciANCmlzIHJlcG9ydGVkIG9ubHkgb25jZSBmb3IgZWFjaCBmdW5j
dGlvbiBpdCBhcHBlYXJzIGluDQptYWtlWzZdOiAqKiogWy9ob21lL2dyb3NzL2tvcmcvc3Jj
L3NjcmlwdHMvTWFrZWZpbGUuYnVpbGQ6MjQ0OiANCmRyaXZlcnMveGVuL3hlbi1wY2liYWNr
L3BjaV9zdHViLm9dIEVycm9yIDENCm1ha2VbNV06ICoqKiBbL2hvbWUvZ3Jvc3Mva29yZy9z
cmMvc2NyaXB0cy9NYWtlZmlsZS5idWlsZDo0ODU6IA0KZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2tdIEVycm9yIDINCm1ha2VbNF06ICoqKiBbL2hvbWUvZ3Jvc3Mva29yZy9zcmMvc2NyaXB0
cy9NYWtlZmlsZS5idWlsZDo0ODU6IGRyaXZlcnMveGVuXSBFcnJvciAyDQoNCkFkZGl0aW9u
YWxseSBJJ20gc2VlaW5nIHRoaXMgd2FybmluZyBvbiB4ODZfNjQ6DQoNCi9ob21lL2dyb3Nz
L2tvcmcvc3JjL2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW5fcHZoLmM6OTc6NTogd2FybmluZzog
bm8gcHJldmlvdXMgDQpwcm90b3R5cGUgZm9yIOKAmHhlbl9wdmhfcGFzc3Rocm91Z2hfZ3Np
4oCZIFstV21pc3NpbmctcHJvdG90eXBlc10NCiAgaW50IHhlbl9wdmhfcGFzc3Rocm91Z2hf
Z3NpKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQoNCg0KSnVlcmdlbg0K
--------------SncFPlxTg8DkoLMqUFXsOL99
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------SncFPlxTg8DkoLMqUFXsOL99--

--------------O3s0i0TBCQzGkMudaipiCF2z--

--------------Knae4IyoLuapUbvuflU12aH9
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmY90U0FAwAAAAAACgkQsN6d1ii/Ey+H
2wf+PSgCmVegUuGJiEjtMEC4NGK5ZCd5/kTaohRVA1KDMrhKcLFgL5BnL6VjenRhYw0R18hcvloB
cRyQTUsx+1AKilmAulFK0GqxigUyGzN/wkBQvklWgaIQAkbeZ1RAyblB+wCRW2z5mp3zIue2k878
RLu6y39cURX53WvVDgvCUAIg+AjXNpkesMCBgxoWetlrR3GsjRj+g7RYRGcVG+rHQU8e7TCgyF6z
BvcrbDDSxyNkzyXYJQ2n58G1CGZ+cLRo2X+BWNiOOa/29nvZF8bRRji7RaDnJIgNd4kdAOVbYqSB
IFYKk7iULEiSyt/KZLOuYz37P4cgiaqqYXbw6+FCKA==
=eS2b
-----END PGP SIGNATURE-----

--------------Knae4IyoLuapUbvuflU12aH9--


From xen-devel-bounces@lists.xenproject.org Fri May 10 08:06:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 08:06:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719564.1122366 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LGd-0001hm-2j; Fri, 10 May 2024 08:06:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719564.1122366; Fri, 10 May 2024 08:06:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LGc-0001hf-Ut; Fri, 10 May 2024 08:06:10 +0000
Received: by outflank-mailman (input) for mailman id 719564;
 Fri, 10 May 2024 08:06:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5LGU-0001dp-Do
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 08:06:02 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 23cfc18e-0ea4-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 10:06:01 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so420486166b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 01:06:01 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c323887sm1565300a12.89.2024.05.10.01.06.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 01:06:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23cfc18e-0ea4-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715328360; x=1715933160; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KjUmBYZRcJLqI6YjjRBHpniyNELY4eAJBJMOmGd2QSI=;
        b=FQVCUruW5XjuZhzc9HBuCv8fpBfUU/7vioAvsqoTV7EVPBa0pEd/sj0lTG8i7eBX1I
         t+6mv2Yk6xeX3NHzAj8KVellYx3bmcoj5UvVVHIQWxVR5rmlIIMzTfYttVo69r1G9Aym
         RQt/ix2J47o4vyTIBioTKiVL6hz4kCShytd/w0pKXifjLGEJFEvY8UQp8T3t4CiialNV
         UK93REUmJSsszibM1ZVSi8kdllWxaE2k+x2a1IzqdC0tEyxU4cn8Q0GcVbd1DhywAe7m
         NqMD9x+G5UoXmxQ3++1Sigxegoa19cYkV62fp2tNI5Io0Tne9RRzlunBwPEGONPjD06l
         +0Jw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715328360; x=1715933160;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KjUmBYZRcJLqI6YjjRBHpniyNELY4eAJBJMOmGd2QSI=;
        b=VrfZhBls5BINFPzkKH5OtXfVEAPTonjpMqMDEHhKcD6uVTFUFZgEj9fMT6/KrNCzwu
         KMqariFcd2RvVcOIovWkQNX9W7kqXc2j+D0UoXwK/2m4S4Vsna/E1qdwKB1KHKy3DjMX
         7O6csXUyjgdKnHyYpQZQENS1azXRmtqNYFluS/BsXMaxrNaWomvwINQMtfzp6jdxbtxF
         U3CyUDr6D2nuON7gM5RvDY7/AH7hZdWpFPg/nRgdcmEj1iqAUPAmylIu2aksXkkd+Mga
         TZyzCrBnGI9URfA7DaJ2bNy9HSYwsPJE4/H6UJHmbtqYMZu3ccoPK5QJhx35oKF5/gHX
         NlgQ==
X-Forwarded-Encrypted: i=1; AJvYcCVrK6kAvZ+aPtaXrPiMoROcs0HuxTrIXb85JepNP/1mqr1uo77RubydXyV7IP5DOkE2wCEAXDkjyUhfltcBQCB7Q7eIgDEKeUkSQ2Qbzjc=
X-Gm-Message-State: AOJu0YxHfl21ab5LldQ6eUFvHX8Hqtpj2gc7U0NNvqNWIrtphHGkQCV+
	S6q+RquVhQA5NKnE9oNJz4pExNA0TfRQeUWQUpLopMoPXlOPtmrTq/Hec7882dQ=
X-Google-Smtp-Source: AGHT+IGob/d3TfUjBbghLynpjknZOHPeymBsHMDPCDMq2sreJOIGOkoi5rR8z3BhZwdrEdxSurL+dw==
X-Received: by 2002:a50:8a8c:0:b0:572:46db:8fa with SMTP id 4fb4d7f45d1cf-5734d67ea8emr1323463a12.33.1715328360588;
        Fri, 10 May 2024 01:06:00 -0700 (PDT)
Message-ID: <c978ac2b-0529-4bd8-a493-55e806740808@suse.com>
Date: Fri, 10 May 2024 10:05:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] libxl: Fix handling XenStore errors in device creation
To: Demi Marie Obenour <demi@invisiblethingslab.com>,
 xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
References: <84a6e6376ab9f5aafac8f33bf772d73fcfd0d818.1713998669.git.demi@invisiblethingslab.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <84a6e6376ab9f5aafac8f33bf772d73fcfd0d818.1713998669.git.demi@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 27.04.24 04:17, Demi Marie Obenour wrote:
> If xenstored runs out of memory it is possible for it to fail operations
> that should succeed.  libxl wasn't robust against this, and could fail
> to ensure that the TTY path of a non-initial console was created and
> read-only for guests.  This doesn't qualify for an XSA because guests
> should not be able to run xenstored out of memory, but it still needs to
> be fixed.
> 
> Add the missing error checks to ensure that all errors are properly
> handled and that at no point can a guest make the TTY path of its
> frontend directory writable.
> 
> Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>

Apart from one nit below:

Reviewed-by: Juergen Gross <jgross@suse.com>

> ---
>   tools/libs/light/libxl_console.c | 10 ++---
>   tools/libs/light/libxl_device.c  | 72 ++++++++++++++++++++------------
>   tools/libs/light/libxl_xshelp.c  | 13 ++++--
>   3 files changed, 59 insertions(+), 36 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_console.c b/tools/libs/light/libxl_console.c
> index cd7412a3272a2faf4b9dab0ef4dd077e55472546..adf82aa844a4f4989111bfc8a94af18ad8e114f1 100644
> --- a/tools/libs/light/libxl_console.c
> +++ b/tools/libs/light/libxl_console.c
> @@ -351,11 +351,10 @@ int libxl__device_console_add(libxl__gc *gc, uint32_t domid,
>           flexarray_append(front, "protocol");
>           flexarray_append(front, LIBXL_XENCONSOLE_PROTOCOL);
>       }
> -    libxl__device_generic_add(gc, XBT_NULL, device,
> -                              libxl__xs_kvs_of_flexarray(gc, back),
> -                              libxl__xs_kvs_of_flexarray(gc, front),
> -                              libxl__xs_kvs_of_flexarray(gc, ro_front));
> -    rc = 0;
> +    rc = libxl__device_generic_add(gc, XBT_NULL, device,
> +                                   libxl__xs_kvs_of_flexarray(gc, back),
> +                                   libxl__xs_kvs_of_flexarray(gc, front),
> +                                   libxl__xs_kvs_of_flexarray(gc, ro_front));
>   out:
>       return rc;
>   }
> @@ -665,6 +664,7 @@ int libxl_device_channel_getinfo(libxl_ctx *ctx, uint32_t domid,
>                 */
>                if (!val) val = "/NO-SUCH-PATH";
>                channelinfo->u.pty.path = strdup(val);
> +             if (channelinfo->u.pty.path == NULL) abort();

Even with the bad example 2 lines up, please put the "abort();" into a
line of its own.


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 10 08:07:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 08:07:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719567.1122376 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LID-0002uL-DV; Fri, 10 May 2024 08:07:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719567.1122376; Fri, 10 May 2024 08:07:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LID-0002uD-AJ; Fri, 10 May 2024 08:07:49 +0000
Received: by outflank-mailman (input) for mailman id 719567;
 Fri, 10 May 2024 08:07:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zKRT=MN=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s5LIB-0002td-CO
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 08:07:47 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 62169674-0ea4-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 10:07:45 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51ff65b1e14so1860252e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 01:07:45 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f35ba746sm601833e87.104.2024.05.10.01.07.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 01:07:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62169674-0ea4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715328465; x=1715933265; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=yt+2vWS4MhJW/ZahGWzw0UqXhZCmCY00qcl6efnl6W8=;
        b=BrB19D/LMJnvsIhkpMz953U913aKx32hlrYpA0UwajYCOwboZbkyrrFqJ/ktSG+Bt7
         u+7jBnAQSGYH66d9WNk08EhUd0XsnhLoncyfEfj6Q1OeBaE2qpAJr6b7ovNq8dJb7gK7
         JPwvUfDtOxlflphQ9tbxNOSC04VW7qmLPIAFY7i9ZwFiYF0WazYjRt8XylZGH2wIKgQk
         KiV/fUK9W5tjz0fDit2b/96HGxiC6W/RuHv413Z5Yq3n/fyNyMhhtip9ijq00bqo/T4P
         taPkSEOsD7HCMs8N2fUTOhD6QXpfEJ5K1BKyi+KhVpY90GkPDom0s4AOwTE+BXQQj1A9
         ytEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715328465; x=1715933265;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yt+2vWS4MhJW/ZahGWzw0UqXhZCmCY00qcl6efnl6W8=;
        b=sBNoxJDRwIyJcyuBENkDa7bzgh7NyHLbDRpjyGPLvJJFt0OdtnSLFX1veduHXtSDcY
         329ADFLyQEmgr3gmNfSJW9G2xdQWRoZB0gxyDlNTPzKiEyjaTF47S3rJZR8IELpkdDTO
         XuDmW7Pr/JrMCNZERXrGNNF2zdspMYlrv/7d9djSNVuOxBSCfQlnju3SUZcDjPH5VOah
         Z66rxMZgfEufvpJGceo7SUxyKWE0gWfPnvBaHc7MiI+AJLsgKV8VEp5q/zQOdAP03FUf
         m8ecWQjbvLcq2epLKjrlVYkAucr94bsoIdrsg2Y4z1ScTf07MuDXDiZP3i1qikLsD6z5
         rAMA==
X-Gm-Message-State: AOJu0YxMaNPX5HVbtikZlsInd1qhCsDQH2eFHQU/X7ssJioynHgDmO/J
	NR/fEk4Cs8TfoYLDKROElmF94+DdymYxocHsd5pn7Kxl4TFEo3B5H1s+4ZVx
X-Google-Smtp-Source: AGHT+IHQdRxDmI1bON/ZGGjaBKA/weZ6303htgZ2XJ+ksSbF6GmId73RmTiFoKy/trTXz5tQyGMjSQ==
X-Received: by 2002:a05:6512:38d1:b0:51f:1853:25f with SMTP id 2adb3069b0e04-5220fc79467mr1187814e87.19.1715328464474;
        Fri, 10 May 2024 01:07:44 -0700 (PDT)
Message-ID: <b2360598c0753717463d5c206fc2020e09d0fdf7.camel@gmail.com>
Subject: Feature freeze date for Xen 4.19 is 17.05.2024
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Fri, 10 May 2024 10:07:43 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hi all,

I would like to remind that the feature freeze date for Xen 4.19 is May
17, 2024. If you want your features to be included for the release,
please make sure they are committed by May 17, 2024.

Have a nice day!

Best regards,
 Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 10 08:09:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 08:09:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719569.1122386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LK1-0003TP-NF; Fri, 10 May 2024 08:09:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719569.1122386; Fri, 10 May 2024 08:09:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5LK1-0003TI-KX; Fri, 10 May 2024 08:09:41 +0000
Received: by outflank-mailman (input) for mailman id 719569;
 Fri, 10 May 2024 08:09:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5LK1-0003TA-1M
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 08:09:41 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a65ebd29-0ea4-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 10:09:40 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51fdc9af005so2762397e87.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 01:09:40 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7dd4sm156394666b.139.2024.05.10.01.09.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 01:09:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a65ebd29-0ea4-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715328580; x=1715933380; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=NOA3rf3iEwXBosJZswL8rY8kkufkCURmYw7VrB7WzZY=;
        b=E7JK0ADc93nphX7/RhJGXumlGh9KmCzErcsLagVR0EoFzkle7Eu6UUyZvILto7Mv0u
         6kIQUAZ20lRPlxgH+UXSjVA3K99XxQPayEBSbfwrJ1D8Wy0YLK6WM1px6FE37mj5KteR
         fFtuss7t6fYEr79XsOG6SF5ro2fnT3vfLUooSfvV41SGHLmYVp1WqXTp+q7j7Anw+lnv
         kDkC6aFYMMggI8PT3PwuKWmUzuQJNNq9010K1E+hephsF7QFQUZtKV6Typ0KIGRgseGM
         njeUxKYO44uZhpZSwek2mQoAAx69uKt9SZPn/dVNE/YMghRYBur7xIoEu7CiMAxTMENw
         UNIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715328580; x=1715933380;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=NOA3rf3iEwXBosJZswL8rY8kkufkCURmYw7VrB7WzZY=;
        b=fit5F5MmJHrI1KXC3fLjAOpmxWYzKglwY4F/9rpkUXrvSxtIIjTxjILet7eOG5Yp4k
         5FUmgrzBp1lz062Zmjh1JCCVfAqINFHhnlNmiPUfsCpvg4JoOGmrU3DXYaj1O8rvuiYk
         wYyMw3YeJNWIs/iSzk7FVGEY/4q7I/bULMqxDeH4Ql49YLOwkiwA1jD+HJK91BzqHc+U
         HhsJLJLWGQ+Iq0OMByyuabSSQdOE58jrv7VgOW3iOKF40Fnr7VS3o0nsOEZqItN61PIH
         BqddMUwDpvc1tg+a5cfqA2ewTzFRvlIHWyyx4qPko7Xs14IjyVkp/HbJGPOGOw2b+yfj
         lQHg==
X-Gm-Message-State: AOJu0YwR5qT93C/dxg3czEPdAajfJoO1eaC1x8/bfLtP/XMPExtmKLRs
	gKl8gWcnP3ia2N5lJGt8xyK2B4q3tTgg6v1eqnqtDGHKVQ7bVmiIl2xKyE0ygZ4=
X-Google-Smtp-Source: AGHT+IHtDaPtq3+zWdvt/+QCSVMqgZzNN1SJ3jMP1mQYF2RqhDLEuqTWeGEoaIonyooUIL8R+/1DAg==
X-Received: by 2002:a05:6512:3f20:b0:51d:4383:9e59 with SMTP id 2adb3069b0e04-5220f86c902mr1439410e87.0.1715328579679;
        Fri, 10 May 2024 01:09:39 -0700 (PDT)
Message-ID: <817f6c43-101e-4e9c-9ab7-f99c960d3499@suse.com>
Date: Fri, 10 May 2024 10:09:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/5] xen: sync elfnote.h from xen tree
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-2-jason.andryuk@amd.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240410194850.39994-2-jason.andryuk@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 10.04.24 21:48, Jason Andryuk wrote:
> Sync Xen's elfnote.h header from xen.git to pull in the
> XEN_ELFNOTE_PHYS32_RELOC define.
> 
> xen commit dfc9fab00378 ("x86/PVH: Support relocatable dom0 kernels")
> 
> This is a copy except for the removal of the emacs editor config at the
> end of the file.
> 
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen



From xen-devel-bounces@lists.xenproject.org Fri May 10 08:42:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 08:42:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719590.1122396 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Lpw-00013r-7T; Fri, 10 May 2024 08:42:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719590.1122396; Fri, 10 May 2024 08:42:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Lpw-00013k-2s; Fri, 10 May 2024 08:42:40 +0000
Received: by outflank-mailman (input) for mailman id 719590;
 Fri, 10 May 2024 08:42:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=asbK=MN=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s5Lpv-00013e-HI
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 08:42:39 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3f8109b1-0ea9-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 10:42:36 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by PH0PR12MB5645.namprd12.prod.outlook.com (2603:10b6:510:140::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.49; Fri, 10 May
 2024 08:42:32 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.047; Fri, 10 May 2024
 08:42:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f8109b1-0ea9-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Q31kmu1URS2/jPZHUjbpUAL3YXK/GnC3dv4ylrh2jH8mOeC44gy4kk3gJkK6uDmC+qgrH9yxyA8XoHnzqrLPPSb30QQJtFbkKH+FxigWQIkfj5wZRUDix7OO2/JAbi5IjELuQB0cujLsdTCIGj1AioR+f3TcwLZmuKmfQbjl/+ADgAm6ofmc8xR/44wywqIOGheldKZJbaG+W7CF5BrDiGSRDGGdYT1Dld2mdZW+IW5N44OyNbhXucxvfulnLPWkZEFdqd2+T2CflhVUQLUxjmnBoP9UKsVrih0px3gEJFBBZIwWk7OuEzf3Siz6rsJn7UEaPJXOn5j7s1nl73hW/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2/oSdE+eyGm1cwEYkGUybu0Kw1gWDAc+wgB0BtK8ZJk=;
 b=L2NrbiuO8tDg0qdINKci52V6h+R5dtoXzEY2k/bgYyAYpB51af6kz/fynA3l1u6i6CWHWKNgwehYh1ZnL1wkUF7+vxfdM9QIzGS79iWr3nh/gM9PgGYmipuKk+5q7e/a6fyg1hBhOk6ILYaPlGRbAvkUYzrRz25dVmvU4PyOdUe6LZYEu7o0fAqZO9aFHmZj9G9EcM3GDOccrZj+pderr94Ey0z/kQjXYpLmLzDAbS6O68bodz3Tptj+GFuRk8R/99vXhcCkYPErk1QztdG/8Fd5yYfrar7+Wlb0L3CJumJtiQkFbGF4Ps5ycuEHMmN2YGMAfvOZXWVgFQUo9qMb0A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2/oSdE+eyGm1cwEYkGUybu0Kw1gWDAc+wgB0BtK8ZJk=;
 b=iFJOcjhrry+aMzvhWHjcAajEqriBVEnHMAxWIDYspp8xK6zMMa3BsOORof8KIvjGUmT/5hK4kyoHbuyOWRXs+lG62YYv8YfHx/ZvJZZDDB7mW/Di8kFYWjFyn4NjskUt/OXKIRXP8z3/UG+Q2K1a6wtgfiyVrgl3GTqorgX3waY=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Juergen Gross <jgross@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>, "Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 2/3] xen/pvh: Setup gsi for passthrough
 device
Thread-Topic: [RFC KERNEL PATCH v6 2/3] xen/pvh: Setup gsi for passthrough
 device
Thread-Index: AQHakgrNAUhcae4lxEaXCyVvErnSM7GQOM+AgACN+IA=
Date: Fri, 10 May 2024 08:42:32 +0000
Message-ID:
 <BL1PR12MB58492D7614F25F29548F1731E7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-3-Jiqian.Chen@amd.com>
 <9ebdcc0b-3775-4c5e-9ffb-f45d048281dd@suse.com>
In-Reply-To: <9ebdcc0b-3775-4c5e-9ffb-f45d048281dd@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: IA0PR12MB8931.namprd12.prod.outlook.com
 (15.20.7587.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|PH0PR12MB5645:EE_
x-ms-office365-filtering-correlation-id: c9de64e8-a863-4023-a75a-08dc70cd2203
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?Uk5nMDR4aGpxeTVPWG9QNnl3cW5mVnRqeVRjM0Fqa3NOdkZKb0p1RVpvN20y?=
 =?utf-8?B?RTFJaXRqWjhKcm1FUTNLdEYwVnN3SGpaeFBjemhvVVNsRHlMLzJwMmxJQWt2?=
 =?utf-8?B?alF2KzJBeVZ3ZnJ5U0l4RllhRG1kOG5qMzMxZi92YmIrWDRZbnR3U08wNzIx?=
 =?utf-8?B?Q0pBNzJJei9RL2ZsMnBIa0w4MUVwZ0MwVmFXWlpna3hmT0VUMFhMcEg1ZjJ4?=
 =?utf-8?B?eXg3OTdzd3BKaUNUMEhRYkUwUnBBaW0xNUxHM0N3elhnNjNpOCtFS0dOM2pL?=
 =?utf-8?B?M0NEM3dIYTI4OVNYNTMzNS9ORU1zUDBNb3doM3JvNm9rSlRhaXNhTTMvN2Nj?=
 =?utf-8?B?d2NhSm8vZTlibzBFbXk5amkyRm1FSFMvaG1BUGM0SDRzL3U2WHFnbHo1Rkw4?=
 =?utf-8?B?cEVPYjBRMHkzejJCZ3VzRm8wZGZBSnF6dEhleUZ4b2JwK2tKdFR3bVh3QnNy?=
 =?utf-8?B?LzdNOSttTDdRMEx3TGtvQkJ3azJJTXBxVmZjYzRGdnJoVmkzczBiellxY0tU?=
 =?utf-8?B?TkdHK0pkclZrWkUrblBWNFRKd0ZqaXgzQ2QzNkRCSmN4bzdtcEVSNE5YOW5N?=
 =?utf-8?B?d0lYM2RsSHNQOHE4UkljQzAwWWNJRk15SzBJNlVyNC9PSEl4OWJEaWVodGxB?=
 =?utf-8?B?Z24rMUtkNzlFWDFoTlN1RUFVbFQzVk1UK2V3eno1UGQ1RC9lSTVhMllHbTFU?=
 =?utf-8?B?MldESmJqRUZ0cVlmdzJabGxWZjErNmZwQ1RiR216MzlNZENLdUZpRDFKY0hV?=
 =?utf-8?B?OGlXM0JDaTBpcUFuSmZvVzJtc0QzQTVpaTY5ZXd1dXpsZ01rcU5xM2RIam50?=
 =?utf-8?B?QmZCdVhRSzRPVnVrSnJOZCtLWURCWGF2SmdZazRBa015aVZVSTZ5clIxRE9r?=
 =?utf-8?B?Z2lwT09kUHk0TWovcWNmZzN6ZmhsWWNuT01YTDZZQzFwc0sxRmhLcCtYZVJq?=
 =?utf-8?B?SkNpTU51ekM5SnE4eHlTVHlJK0NSNXJiQ2pUVG5BdGpHVTFGNlU3amFtZDdD?=
 =?utf-8?B?VEZxT3lxekpLUElSRjRRVE1yVWdPdVIxSExqV0h0OVFuTXFvYXpPdlcrc0Rh?=
 =?utf-8?B?UDRlbTRSTWFDczJrbWZqclFabkNieVAzeU5GcmhvSlBQOXFkQ01LcTR4ZkUw?=
 =?utf-8?B?blhnZWV0WDZpeVAvMERkWUlHUmtRZkIrNEZpb3VVdWZjbjNER0ZqSndyNUJP?=
 =?utf-8?B?RTBxUEc5ZktKTm9JRThZMTZHSEJtRG9WWEswaTF2blpvOFpvcUxFK1lWRW9O?=
 =?utf-8?B?blFHWkdmbDFnby9oSGMxbHpFQTJFWGxpbURwU2l0NmNoRG5POGVsQVhTL0tL?=
 =?utf-8?B?UDZjTEI2NkdEOW00QmdnQldUalJuLzNnZ296bXlyZnI4bkhtL09OdVVvc0ZD?=
 =?utf-8?B?UHd6eTdFTHFvbGNKWTR5eWVTemhENEFCdlhhelpYZG9yaUZlR01FVDFxbWlI?=
 =?utf-8?B?cytzZ0g0OVZORnpWbmdWU0lsR1FiTTV4aEFBbW1Nck10YW1nTFI4NzlYZ084?=
 =?utf-8?B?M0JuVmhSZm41MDFrMUZrWmVyMm1GOWxOMEZjUHNVendZTjhTSnFySnJFWmV2?=
 =?utf-8?B?UW5BVnJnVGVWOTN1T1U5eVc4dzkyTWtUMWtRaURMOVlxcmt1QURqMjRNNmhU?=
 =?utf-8?B?a2taQmdCK2RuU1ozL3BGMlVac0xNSTFXQmZUOEU3MTlTV2VhUWlEd0Jhc3ll?=
 =?utf-8?B?RFZQdkRzSHg4SkVUVEg1TTNST1d2L2dCNGs0cWpESTJ2MWdFRzBTN3ZmMzhM?=
 =?utf-8?B?aXZxYVRGZmN6WHlZYVhydkRxWHM3d1VGaitxRU1VSzNoM2RSUFZGQW1HMkxS?=
 =?utf-8?B?VEFMK2FkYlZlYVdTM0M5QT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?M054S0RXNFNJWXZJVHJiaStuc2JOQWs4Z2s0RmExV3hDcTJlSFJZYytBZk1q?=
 =?utf-8?B?cnVrK3pRb0RKTkpKdmNtSzJTQ0lnOEtWWU5DUFVpT3pwTE92d3Bvci9iT3dE?=
 =?utf-8?B?SGl0WTBEUWlTY0k0d0tVTUwvSEVWNnY0VGlhYXl6REVOS3Q5RkJxeG9sMGZI?=
 =?utf-8?B?eG14b3JJNlhETXhyRXpZNlJycVdtTVVOeml2VGl6UFpBMTNhdTV4T3ZvcTFl?=
 =?utf-8?B?bFdUcHVzc2NmVC9oOUY4Y1VXN3Q0YWJ4K2lPWStTMXZqUERJZDBIcUo2QTBz?=
 =?utf-8?B?VU9IZVZFbXYyTUc4dTNINERWeEVHYUtDQU04dGxzcHZqSHE1RHg5OXNwYXNZ?=
 =?utf-8?B?OVk5cmY3SEg3TW1kalp0YkhTa2dBWmxnbXJFQmExaVhOU1ZJMkFvYiszUlVB?=
 =?utf-8?B?eFhTdUh5STVwTnpCMkM5MVFiZzBxWXdvT0NPcjdoMVd3T2sxYWFSUXpDMGhB?=
 =?utf-8?B?TUIwN3NldUtlMVRyRVFwblorcW8wbk5nS1U1STNjNThteGNGdmVuTmppdXor?=
 =?utf-8?B?U0poQmNTaWxzSW9SRWczV3VDZElya2MrQlVKT01VWVprbHVUOHFodGdJNVVG?=
 =?utf-8?B?WkE0QzlRd0ZiMWtEM0lTWTJsQnJ4ZXpWNU11Z3hkbDlqK1lLR280ZFFleGxJ?=
 =?utf-8?B?bTZTaUlIL0tHYlczMHlUcWVteUVkSlNmS1dHZWlnNVNUZGFJWnZYWUJwa0Fo?=
 =?utf-8?B?K0Q0OVpaeVJJVEpKL0tNaVkzN2lDWnhxVm1hRXlIN1Vpd1pOdG5yOXM3TnlB?=
 =?utf-8?B?ZmRscTVNeTVFM21NN0JwUWN6b09RQSsvNXRUeWlsNC94Z2xEZG1BY0dKdHFW?=
 =?utf-8?B?bHFxR3MxUlErVVU2eEZKZEMyZHdqSFNjZ3dvc2hweFpjcm5UenhFWGhmdDdX?=
 =?utf-8?B?VWZUWjhnU3BzK1RoMU56N3k5cGZHVElsZFFmYVF4TG1rSlhBcFkydjFkcm5P?=
 =?utf-8?B?WVVYbDFDV2Q3OUZzYXlOM0orZjQxdE04WWlYdVBoOC9maUJBYzBZRkNqUitV?=
 =?utf-8?B?dXpPcXlpR04xNFJURys5YUVsOENaQjIyelZ0Mml5NzdpYmZLcUlYYTFaNVV2?=
 =?utf-8?B?T3N1dHFQVE9PZzR1dVh3MzNrK0xldHRBYlNWOElIcG1VTlZ2dWxnTFUyZDQw?=
 =?utf-8?B?Vm1JaVFrZmExdXphRG9NQUpNMTU1Z29KV29XT2xLUDFTT1dtN21BNVJmMzFs?=
 =?utf-8?B?YkpEU2JRemZ3aXZiWE4yWjFXc1hsM1ByUVpsU29YMGl0cGF3aW9lTTdyUnAw?=
 =?utf-8?B?UFRiZFdYMkFzSlVKcEhJL3VmY1RabFg5N010cGYwZC94aFRQL2h1alNyNzJB?=
 =?utf-8?B?RHRGTmZLc0IwdFE5anljQ2ppTk9zbWdIemdMOCtQcERHYkRaOHZkSmRKN2xV?=
 =?utf-8?B?R3pNa2E5d3RaMXlxOG5YbkljbjJPOEwwcFdiOTJHVEJXVS90RUV1OFQwekl0?=
 =?utf-8?B?VDZuZ1N4M2RxSkx0M2xFYmNadzdTQ3lxZmsyOUZINGhzQzdwT1U5YmsrbGcy?=
 =?utf-8?B?d2c4SVNYU1VoM01JZ1pUQVdiQ1pNTFZHazdLeERFaXBkT0JmY0hNbjlqRUc3?=
 =?utf-8?B?R3BhY3FibmErOEY5NS9sVVpJc0tyeGtBOEliUWgxekI4cXBNNExHQ0FrdzAw?=
 =?utf-8?B?SHRhdGZ6eGFTRlZVbHJlMzczNjBUNjZ2aXNzNFNFSmd3N0ViL1JvQmgxakUz?=
 =?utf-8?B?OGJUSVNwWmZNZmgwbXZWZ3lwaFlZK3huVElWSjNUaEh2VE4rWnh4SFJnQXcz?=
 =?utf-8?B?aVpQTGdCRzdXRyt5eFBsWHVubEpVUGREWm1pOXIvdjB5dWZ4OStXUEhpcXlV?=
 =?utf-8?B?T2dvbFVzVHpsM2V1ZkpoNlVlckVncE9vdERGd3ZicVFRNlIxMWFzSUNCaDQ2?=
 =?utf-8?B?Z1cxRFVETUhIbisyZVMxdkxEQ3o2Z2tjdHlzSS84WUlrc3ZQcTRTdjBFVytm?=
 =?utf-8?B?M21yYzBMZzVYdUZEVlBPNkNFejllYjNWTWVHTnJnQVhhM3F6QXE5Mjh3OUVD?=
 =?utf-8?B?eWpLWUhvUDN1Zkx2RHgrTU5mZlY1OGxoOEtPalpDUExabnJkeTY3S1hBZk00?=
 =?utf-8?B?dWJVdXRHVTRtVlp2bno3a3luUnRkbHh4OVUwalNUeCtjSGJ3S0tkMjVFUGJZ?=
 =?utf-8?Q?7mc8=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <96DB0031C092EC429883C1D1457488BB@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c9de64e8-a863-4023-a75a-08dc70cd2203
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2024 08:42:32.0869
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SuKyDpcFC+SeD9vUrBF4sbNaFTRVhRrONAO4YME57KxihCikCsK328gME3dxd/NjQs1mXcEyWhnCf5dkseIrvQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB5645

SGksDQoNCk9uIDIwMjQvNS8xMCAxNTo0OCwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMTku
MDQuMjQgMDU6MzYsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4gSW4gUFZIIGRvbTAsIHRoZSBnc2lz
IGRvbid0IGdldCByZWdpc3RlcmVkLCBidXQgdGhlIGdzaSBvZg0KPj4gYSBwYXNzdGhyb3VnaCBk
ZXZpY2UgbXVzdCBiZSBjb25maWd1cmVkIGZvciBpdCB0byBiZSBhYmxlIHRvIGJlDQo+PiBtYXBw
ZWQgaW50byBhIGRvbVUuDQo+Pg0KPj4gV2hlbiBhc3NpZ24gYSBkZXZpY2UgdG8gcGFzc3Rocm91
Z2gsIHByb2FjdGl2ZWx5IHNldHVwIHRoZSBnc2kNCj4+IG9mIHRoZSBkZXZpY2UgZHVyaW5nIHRo
YXQgcHJvY2Vzcy4NCj4+DQo+PiBDby1kZXZlbG9wZWQtYnk6IEh1YW5nIFJ1aSA8cmF5Lmh1YW5n
QGFtZC5jb20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKaXFpYW4gQ2hlbiA8SmlxaWFuLkNoZW5AYW1k
LmNvbT4NCj4+IFJldmlld2VkLWJ5OiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtl
cm5lbC5vcmc+DQo+IA0KPiBUaGlzIHBhdGNoIGlzIGJyZWFraW5nIHRoZSBidWlsZC4NCj4gDQo+
IE9uIEFybSBJIGdldDoNCj4gDQo+IEluIGZpbGUgaW5jbHVkZWQgZnJvbSAvaG9tZS9ncm9zcy9r
b3JnL3NyYy9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5jOjIzOjA6DQo+IC9ob21l
L2dyb3NzL2tvcmcvc3JjL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVuY3Rpb24gJ3hlbl9hY3Bp
X3NsZWVwX3JlZ2lzdGVyJzoNCj4gL2hvbWUvZ3Jvc3Mva29yZy9zcmMvaW5jbHVkZS94ZW4vYWNw
aS5oOjY3OjM6IGVycm9yOiAnYWNwaV9zdXNwZW5kX2xvd2xldmVsJyB1bmRlY2xhcmVkIChmaXJz
dCB1c2UgaW4gdGhpcyBmdW5jdGlvbik7IGRpZCB5b3UgbWVhbiAneGVuX2FjcGlfc3VzcGVuZF9s
b3dsZXZlbCc/DQo+IMKgwqAgYWNwaV9zdXNwZW5kX2xvd2xldmVsID0geGVuX2FjcGlfc3VzcGVu
ZF9sb3dsZXZlbDsNCj4gwqDCoCBefn5+fn5+fn5+fn5+fn5+fn5+fn4NCj4gwqDCoCB4ZW5fYWNw
aV9zdXNwZW5kX2xvd2xldmVsDQo+IC9ob21lL2dyb3NzL2tvcmcvc3JjL2luY2x1ZGUveGVuL2Fj
cGkuaDo2NzozOiBub3RlOiBlYWNoIHVuZGVjbGFyZWQgaWRlbnRpZmllciBpcyByZXBvcnRlZCBv
bmx5IG9uY2UgZm9yIGVhY2ggZnVuY3Rpb24gaXQgYXBwZWFycyBpbg0KPiBtYWtlWzZdOiAqKiog
Wy9ob21lL2dyb3NzL2tvcmcvc3JjL3NjcmlwdHMvTWFrZWZpbGUuYnVpbGQ6MjQ0OiBkcml2ZXJz
L3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5vXSBFcnJvciAxDQo+IG1ha2VbNV06ICoqKiBbL2hv
bWUvZ3Jvc3Mva29yZy9zcmMvc2NyaXB0cy9NYWtlZmlsZS5idWlsZDo0ODU6IGRyaXZlcnMveGVu
L3hlbi1wY2liYWNrXSBFcnJvciAyDQo+IG1ha2VbNF06ICoqKiBbL2hvbWUvZ3Jvc3Mva29yZy9z
cmMvc2NyaXB0cy9NYWtlZmlsZS5idWlsZDo0ODU6IGRyaXZlcnMveGVuXSBFcnJvciAyDQpUaGFu
a3MgZm9yIHRlc3Rpbmcgb24gQXJtLCBpdCBzZWVtcyBJIHNob3VsZCB1c2UgbWFjcm8gIkNPTkZJ
R19YODYiIHRvIGlzb2xhdGUgdGhlIG1vZGlmaWNhdGlvbnMgdG8gdGhpcyBmaWxlLg0KDQo+IA0K
PiBBZGRpdGlvbmFsbHkgSSdtIHNlZWluZyB0aGlzIHdhcm5pbmcgb24geDg2XzY0Og0KPiANCj4g
L2hvbWUvZ3Jvc3Mva29yZy9zcmMvYXJjaC94ODYveGVuL2VubGlnaHRlbl9wdmguYzo5Nzo1OiB3
YXJuaW5nOiBubyBwcmV2aW91cyBwcm90b3R5cGUgZm9yIOKAmHhlbl9wdmhfcGFzc3Rocm91Z2hf
Z3Np4oCZIFstV21pc3NpbmctcHJvdG90eXBlc10NCj4gwqBpbnQgeGVuX3B2aF9wYXNzdGhyb3Vn
aF9nc2koc3RydWN0IHBjaV9kZXYgKmRldikNCkkgdGhpbmsgSSBuZWVkIHRvIGFkZCAiICNpbmNs
dWRlIDx4ZW4vYWNwaS5oPiAiIGluIGZpbGUgZW5saWdodGVuX3B2aC5jLg0KDQo+IA0KPiANCj4g
SnVlcmdlbg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 10 08:54:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 08:54:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719597.1122405 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5M1F-0003QA-8D; Fri, 10 May 2024 08:54:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719597.1122405; Fri, 10 May 2024 08:54:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5M1F-0003Q3-5b; Fri, 10 May 2024 08:54:21 +0000
Received: by outflank-mailman (input) for mailman id 719597;
 Fri, 10 May 2024 08:54:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5M1D-0003Px-Gq
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 08:54:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5M1D-0000xZ-1K; Fri, 10 May 2024 08:54:19 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5M1C-0003Uo-Oq; Fri, 10 May 2024 08:54:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=eD6xefNzE/+8FNvv8rtChVIbgkpHLFLLx+ZqnzygP1A=; b=rUV6fF4wk08LAK/GBmXyOl+rDI
	h/wcnjQWWaWFghF+PSqG6OveOoatt95fUlZVLoUn+cX6RCP6JhHZsIwHDN/ZiFqqkbNNLc8OrruY3
	tJS+kBg+NBP3toF1uBYfdoTOjhaoXzXScz6PJlUDMqZcnIGX4ROO1AvAsXEuUOLd8UNA=;
Message-ID: <1ad2c873-219c-4d96-82d6-e1bc58216f2d@xen.org>
Date: Fri, 10 May 2024 09:54:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
 <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
 <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 09/05/2024 16:31, Henry Wang wrote:
> On 5/9/2024 4:46 AM, Julien Grall wrote:
>> Hi Henry,
>> [...]
>>
>>>>> we have 3 possible states which can be read from LR for this case : 
>>>>> active, pending, pending and active.
>>>>> - I don't think we can do anything about the active state, so we 
>>>>> should return -EBUSY and reject the whole operation of removing the 
>>>>> IRQ from running guest, and user can always retry this operation.
>>>>
>>>> This would mean a malicious/buggy guest would be able to prevent a 
>>>> device to be de-assigned. This is not a good idea in particular when 
>>>> the domain is dying.
>>>>
>>>> That said, I think you can handle this case. The LR has a bit to 
>>>> indicate whether the pIRQ needs to be EOIed. You can clear it and 
>>>> this would prevent the guest to touch the pIRQ. There might be other 
>>>> clean-up to do in the vGIC datastructure.
>>>
>>> I probably misunderstood this sentence, do you mean the EOI bit in 
>>> the pINTID field? I think this bit is only available when the HW bit 
>>> of LR is 0, but in our case the HW is supposed to be 1 (as indicated 
>>> as your previous comment). Would you mind clarifying a bit more? Thanks!
>>
>> You are right, ICH_LR.HW will be 1 for physical IRQ routed to a guest. 
>> What I was trying to explain is this bit could be cleared (with 
>> ICH_LR.pINTD adjusted).
> 
> Thank you for all the discussions. Based on that, would below diff make 
> sense to you? I did a test of the dynamic dtbo adding/removing with a 
> ethernet device with this patch applied. Test steps are:
> (1) Use xl dt-overlay to add the ethernet device to Xen device tree and 
> assign it to dom0.
> (2) Create a domU.
> (3) Use xl dt-overlay to de-assign the device from dom0 and assign it to 
> domU.
> (4) Destroy the domU.
> 
> The ethernet device is functional in the domain respectively when it is 
> attached to a domain and I don't see errors when I destroy domU. But 
> honestly I think the case we talked about is a quite unusual case so I 
> am not sure if it was hit during my test.

Correct, they are not errors that will happen in normal operations. You 
will want to tweak Linux or use XTF to trigger them.

We want to test the state active, pending and both together before the 
physical interrupt is routed and also removed.

> 
> ```
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index a775f886ed..d3f9cd2299 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -135,16 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, 
> unsigned int virq,
>       ASSERT(virq < vgic_num_irqs(d));
>       ASSERT(!is_lpi(virq));
> 
> -    /*
> -     * When routing an IRQ to guest, the virtual state is not synced
> -     * back to the physical IRQ. To prevent get unsync, restrict the
> -     * routing to when the Domain is been created.
> -     */
> -#ifndef CONFIG_OVERLAY_DTB
> -    if ( d->creation_finished )
> -        return -EBUSY;
> -#endif
> -
>       ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);

This is checking if the interrupt is already enabled. Do we also need to 
check for active/pending?

>       if ( ret )
>           return ret;
> @@ -169,20 +159,40 @@ int gic_remove_irq_from_guest(struct domain *d, 
> unsigned int virq,
>       ASSERT(test_bit(_IRQ_GUEST, &desc->status));
>       ASSERT(!is_lpi(virq));
> 
> -    /*
> -     * Removing an interrupt while the domain is running may have
> -     * undesirable effect on the vGIC emulation.
> -     */
> -#ifndef CONFIG_OVERLAY_DTB
> -    if ( !d->is_dying )
> -        return -EBUSY;
> -#endif
> -
>       desc->handler->shutdown(desc);
> 
>       /* EOI the IRQ if it has not been done by the guest */
>       if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
> +    {

I assume this is just a PoC state, but I just want to point out that 
this will not work with the new vGIC (some of the functions doesn't 
exist there).

> +        /*
> +         * Handle the LR where the physical interrupt is de-assigned 
> from the
> +         * guest before it was EOIed
> +         */
> +        struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], virq);

This will return a vCPU from the current affinity. This may not be where 
the interrupt was injected. From a brief look, I can't tell whether we 
have an easy way to know where the interrupt was injected (other than 
the pending_irq is in the list lr_queue/inflight)

> +        struct vgic_irq_rank *rank = vgic_rank_irq(v_target, virq);
> +        struct pending_irq *p = irq_to_pending(v_target, virq);
> +        unsigned long flags;
> +
> +        spin_lock_irqsave(&v_target->arch.vgic.lock, flags);
> +        /* LR allocated for the IRQ */
> +        if ( test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) &&
> +             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> +        {
> +            gic_hw_ops->clear_lr(p->lr);

This works on the current pCPU LR. However, the vCPU may not run on this 
pCPU.

> +            clear_bit(p->lr, &v_target->arch.lr_mask);
> +
> +            clear_bit(GIC_IRQ_GUEST_VISIBLE, &p->status);
> +            clear_bit(GIC_IRQ_GUEST_ACTIVE, &p->status);
> +            p->lr = GIC_INVALID_LR;

You also need to remove 'p' from the various list (e.g. inflight/lr_queue).

But as I wrote previously, I think it would be much easier if we simply 
clear the HW bit in the LR. So we don't have to try to mess up with the 
vGIC internal state which is quite complex.

I think it could be done as part of vgic_connect_hw_irq(). But I haven't 
fully investigate it.

> +        }
> +        spin_unlock_irqrestore(&v_target->arch.vgic.lock, flags);
> +
> +        vgic_lock_rank(v_target, rank, flags);
> +        vgic_disable_irqs(v_target, (~rank->ienable) & rank->ienable, 
> rank->index);
> +        vgic_unlock_rank(v_target, rank, flags);

Why do you need to call vgic_disable_irqs()?

> +
>           gic_hw_ops->deactivate_irq(desc);
> +    }
>       clear_bit(_IRQ_INPROGRESS, &desc->status);
> 
>       ret = vgic_connect_hw_irq(d, NULL, virq, desc, false);
> ```
> 
> Kind regards,
> Henry
> 
>>
>> Cheers,
>>
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:06:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:06:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719605.1122415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MCq-0005l9-8K; Fri, 10 May 2024 09:06:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719605.1122415; Fri, 10 May 2024 09:06:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MCq-0005l2-5X; Fri, 10 May 2024 09:06:20 +0000
Received: by outflank-mailman (input) for mailman id 719605;
 Fri, 10 May 2024 09:06:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=asbK=MN=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s5MCo-0005kw-Am
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:06:18 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8e1644b2-0eac-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:06:16 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DS0PR12MB8415.namprd12.prod.outlook.com (2603:10b6:8:fc::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:06:12 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.047; Fri, 10 May 2024
 09:06:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e1644b2-0eac-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d+VV6Hy203q9meAN235RWuBOsYlGzi6iH4TZw+rmqjKU0/V0q7Me/LIJ3Uzj73jnLYCbAKTUpGb+iCIK0XcPozmsHhfQvTEfIjNWWa2oBgGXEYIPmSmeuHFRL9md4IJTQ4Rmj0GLdGxEMn8SAaRUzx1iFntHmUkXI6Zel3SQNZlxzDV7aELCGKsx+r8dXKzBvPQtPkIP5paZNDZh2NkP3cdiEb2ppAZoMzy4Wxv+747lhNpNjn4r7a+KH67C+L13QlIRGiyTF7ZfYIqzRid5HFgIhda7pcgHvrLGkIdtYMSjy0EUp6Qi8psKiKjZxisCrB/8hTRErn8wnXCywGe3Tw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=oa9hriqXaOY72HZvfj7FW9X7aeh/P+SaChcnyOu+c9A=;
 b=gTKgmN1fXiZHtQWW3FUO3o2JLb0O8udeBJgwTXjvedzbJYIbfE6kytIf3y5flkgE8XSeTkVJMJNHwMkIdxc2IDqNw08VKZ9kKXj4U6DOCl2tadHV2Hqvc7+7wUdU5UNODedA7xJdyq6wAOQKvQ7k755h6Mz6IeNubSBvrOZ0G0/i4hU6t+ky/UB+orwn/dfIzR4QQW7CPBPwubuo+V4OuxnsK3esfJsrGq8sLTaSrdVRknLh1RJcBvEzonlwi4aH+plOJJ00jj/F9ScfyH2KJKpurJd2MB/GNYtBT15Nx0nJ1krZDqwPDfxGS8hNEPpNeIInRz/Pw5ATyER2epfErA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oa9hriqXaOY72HZvfj7FW9X7aeh/P+SaChcnyOu+c9A=;
 b=waUAb3U2rpomN5OwuKEHEFO/NIWyxt0VRtjE6CdIbNLetqJUeSXQ9PuYi8plyNr0OGkZ2VJJ/xBlJYEIc/j4EARlxGatAnFk+lUIZ8If2EeIwDMsq1purirDifVX0jb1n6Wqf4unXuzP1WRbMmkEMNtB3dVynd944GFeuQzZUAQ=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>, "Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
Thread-Topic: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get
 gsi from irq
Thread-Index: AQHakgrPZQMiHHEzE0Gs/TUOGsHh17GQJ2yAgACnAgA=
Date: Fri, 10 May 2024 09:06:12 +0000
Message-ID:
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
In-Reply-To: <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: IA0PR12MB8931.namprd12.prod.outlook.com
 (15.20.7587.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DS0PR12MB8415:EE_
x-ms-office365-filtering-correlation-id: 3a96e41d-2728-4dd3-4c73-08dc70d0707c
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?T0pUMDYxV24rVk5MMVU4SnJHem4zQ3FjY0c5TmdaRkdLMzFoalZkNGllWWcy?=
 =?utf-8?B?YlVXRTdaM3BKK3kwb1FKamJ1MjFGb0tIRGJKQ1k1RGRGbTU5RktaaW1ldFE0?=
 =?utf-8?B?WGl2UVZOcVpIU2NNWGRrYUJKbDhrajM5UmY4K29QNGI3ZnY0VTY1RGdBQ0U3?=
 =?utf-8?B?UDA3QXp3Q0JOb2dUcmJBVTlpMWN4ZFZDSUdNbitFMFJNWVd5RDI2Ui9sYlBI?=
 =?utf-8?B?WFUybGVteUVReklqNWUrSG90MHBSWmdQZjVvYWhnQkNRbFA0d05QOC9uT0dZ?=
 =?utf-8?B?WnNLNTNzUXRLeFdWQ1VLTmhtTUloQ1cyaUZlRG1qQWJ4Ull4U3BVYlhIb1ph?=
 =?utf-8?B?QXNDaUg1ZXQ5Z2VxeGlwT2w0SlF3eVdwK0hEaFNPbWhWSjZ6endlK3dZY2JR?=
 =?utf-8?B?M2FlK3dZbDBxN1BCaXVkU1RRL2VBYTJlK1BHTlBkenB0Q0IwRlIvaU43OHM2?=
 =?utf-8?B?a0hBc3FUTzUzZFI3ajFNeWd4ZUpraVlWMWNxUmV3K2I4Z0l3ZyticjlkR2xq?=
 =?utf-8?B?UHFkbmw2WXM3MUZRNXdXWDY2SHA1bWVybEtJUzNEaURUZXdPMUNyMTY4Y2dV?=
 =?utf-8?B?UzcwQUx4OEprRkk5aVdMb1FTTG0rZTVtRVlQNzhSV0JubC9tZFIwMFV4YUpD?=
 =?utf-8?B?TzhlbWFkL0tWcTIwbTE4cTlVNU5QTDVkQWR0YjlKTW4zaThEandRQUxWeldt?=
 =?utf-8?B?NmVza28vZHVSSWoyR2VIeDh0QUl6TWJMRWJsNnBWenZpdkJBbER1TXgreXI5?=
 =?utf-8?B?ckZhbk9Gdkd0V1dJejdmZVFLZkxyclNZM25BQVA2Z0V0b1RuRE1kdjY1QWVx?=
 =?utf-8?B?TEFXZlRLN2lGd2FOQ0VsVUl0STZFM0hJNDRjWHExeDV5MG42WUJUWmw2Rndv?=
 =?utf-8?B?RHA5MzJodmtBM3FQdzhrbmZPbU53TU5nUVFQZmVpdnRiTS9xS2FwdnhsV1Rs?=
 =?utf-8?B?ZlVBb3dEQ1doOEFMWHNJSFg1NG9kekJNanlheHVoYnlzb0graVI5ZndSNlZS?=
 =?utf-8?B?U1RtQTBQdkt0OFhxeFdqYkhKUCsxNzFUcENuVktyY2hDTVJHMlhiQ2I1OUxx?=
 =?utf-8?B?WXdjM0IwNjJwcmd0QzNjcnlTaDlTS1JYMjczb2cxcWovRjRydXAyRE8zL0Za?=
 =?utf-8?B?ZjdZei9MVHo2MWFrQWs0ZEdpcC9NeVhlQW54TXlLV0U0TXdyTmowVXhNNEEx?=
 =?utf-8?B?VzRpU214dVFyWFNNaGowaHc4WFV3UkEyUGFPWEJFSi9oUkJDYVQvVktGVzFZ?=
 =?utf-8?B?REJSUFNqc3J1d1JSblFydC9YdGNFalhRT2NqMXZidlV1NEwrK3luaENxMkNl?=
 =?utf-8?B?UE9QUy9nQmRMY0V2R0Yvd053MTd0VEIvbWNxQW5hdUw3SXhjc3BFb2YyYjc0?=
 =?utf-8?B?M3c0MEQ0R3NXYXhNdWo3bVl5RzJ0MDBiemFXM3ZKNFcrbFpBM2dUaXdKTlI5?=
 =?utf-8?B?SnQ5TEJQb0dJZ1VYRWxWTzNGczcrNlVFd0VETnlhbSsxd1FNVlhLck1ualNr?=
 =?utf-8?B?OVNwNjZHN2p1NEJtaEovNFIrZzlFQ3FlYXFIbVkrTk5pa3h6NmNxc3dSdEtk?=
 =?utf-8?B?Y05mY3VtRFFHRzljcTlDd0llK2FTMVJGOFdtWm93VTBQdWlmUURyQTFKbHhW?=
 =?utf-8?B?TmlSVlgrOE52VEFWVDlzeEpnMDVDMGMvZVdRTHRiSWlEUXdHc01mVjl1eDJW?=
 =?utf-8?B?Q280OFBhOXVwR3dERW1vbmwvSnVmVmhKK2pwTXNubFhmOUUrdW96UEV4QmRG?=
 =?utf-8?B?QWdSVnJ1ZDYvQkQyVzNjOU9TM21ZUXRvVVFJZjA4WGZIV0E2cDJ6aXl4Z1R5?=
 =?utf-8?B?Rkcxcm5hWlJiRFpXNDk4UT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?Y0FSQTZRNmZoeFZpeXJDallnNWJ5cTlVdm41clRmTXN6T0ZBWUs0QXNlQjFI?=
 =?utf-8?B?ZXhwRkFLbXBKa0RVMWZqcVZOV1M3SzJwSjVJeitiWlRpOUQrNWZqVDY5Yi90?=
 =?utf-8?B?MWhGSFNkOWlSUTNZcWkrZzc4S1hCOEJyL3VKd2N0UmZqNzNzclY3VHByMjNk?=
 =?utf-8?B?eWU3bVByNnZEdFBaYUZyQmtxcU05Q0tEeVltcDV6Z0VFOG5vaUt2T080bTBi?=
 =?utf-8?B?S21vd29KMTZQYmxWa0RnNW1DdTdTVjFma3RBWWxVU082Q1JzK0ZFMFBmczRo?=
 =?utf-8?B?VVB6YjdYTkpjb29PMVFka3VLaHZKZjdRMFdNd1E3TCtoTWVZQk8xcCtkcVNL?=
 =?utf-8?B?Z1V6NEVwZmFlNk8vNnhvN0FGM3l3MjJjVjRBTUNMZWlwSEt6WUFzNmdyQWVP?=
 =?utf-8?B?OFYzMDhTN01jMzJwcUZLdjJ1Slo4eXllRlZmcmx6QklhVkVoRDVNT01ZTzFI?=
 =?utf-8?B?NXVndzZ0SDk5UDZmV2lLL2hyWEkvRHdZY2Z0d2k3MDNkVTJZU1NiTkRPTmJZ?=
 =?utf-8?B?dE1hakdVellaVi85Qk50VE55TkZBakpsdERXUnAxUmtPZThwM0hRa1czaWhO?=
 =?utf-8?B?aHhuNU53VzlMcjJrYnllZ05KajhyY2k1WmM5WWEzNlJZZDJLTFIzSlNMd0RM?=
 =?utf-8?B?dDdrODlSZ1hIMEhFMlFCaEcvdU1LWndzVGtOM2t5TzdmRkRSNXVhTjVhamJT?=
 =?utf-8?B?ZWszbjZ3SzF0RGNINWcwcit0b09HajJ3VEZWMnNiRE0rWXhMZ1VtbmMrWGhm?=
 =?utf-8?B?UkJuOHRyZ0h1MGtINjVCRkt4WTA3cEF0TStnNmpLeEc2U2FYazJZVDN6dW1w?=
 =?utf-8?B?ZGEwaHFZa2N4SnZrTjk3aWhwK2RvR1dqZGlhaTdSV0FCME1KeTlKWmphQjZT?=
 =?utf-8?B?ZW0zdm12NXlRU1AxNnVkdWVoV3ZFclN0MkVzb3VMS1dRMGU2YnRrNnZsbk82?=
 =?utf-8?B?MHJFbUxNRXYrbml1SUc0QTdmdWJ4R0EvbnIzS1NoY2tQaVk4VWFCWjVOS1l5?=
 =?utf-8?B?cGJST2gvTkthaUhUREdNcnZPaGw0MmEzeTdNVTB2TlJyTFU2eDd0bzVnVVY1?=
 =?utf-8?B?UTBBTlJYZG04MFc1WURsQWxnRmhtaUV0QUNNbDZ1K0pwdFVzMzJpcUEwVkJk?=
 =?utf-8?B?Y2QwZW9JK3NuTVJzMzhWSTliaWo5emxhSlFHUDc4VDRZay9GcmVqRy9zK2RD?=
 =?utf-8?B?ZzZ1cGpsa3pOdmZOa25IS0dtTGtjbDdXcVlLM0FVYU50ai9MaUZNZ051MmVY?=
 =?utf-8?B?dngwQjZFOExMU0xzQWdBclRVaGdxUm44WGVxRTIzYitoS3MwOExNMWtUMUFQ?=
 =?utf-8?B?cC81Mldabno4OXVMUGJEVEplbmpvWE9oWjNpK2R6VE1YTWFxcG1rZzVuSTZE?=
 =?utf-8?B?RTlpdStPYkV3dW9zL1ErVmlSRXo2S2hjQzV2V2VMelpjV1RHV2E0MHFiSUcv?=
 =?utf-8?B?V0lUMzA2dit0dHdOS2hJcG9tRHBhbFNJL2EzcnJSTC8vOVF3WS9QZWsvN0Jt?=
 =?utf-8?B?NXhWUkdOd3k0RDdQOC85RGl2U0pXWWVFVWJuME1jdEpqSTcrT2RkUXhUcExx?=
 =?utf-8?B?dWdGQmZld2ZMUjJpekZLRDZ1SjBMNnNTMnVqZ0EreThZeEkyZDhEUXlEb3Jy?=
 =?utf-8?B?WGdwZyswd3dXaTRTTitwRmlKTi9NSDBaTGIxazh1elNQOCttT3RUMEtxQVJI?=
 =?utf-8?B?NEFJQU1QUmJpcWVvWS9IT0JPcmUxZmtUakxPLzdmUmt2SWxuM3dnVkNFVjVE?=
 =?utf-8?B?dnMxL25rdWJzQmtZTmduVVFkTmU4RnFDaUxUU0JSLzlGTXRBQlV0MFJ0cGVq?=
 =?utf-8?B?aHpQbXFKOVlxUWNodjMzVUViZXp4UWVBa2NZdjJkdW5BZU40WUt4bXhlNmRo?=
 =?utf-8?B?bGE4aFFFdWFuSWwyTitlMzh3K2luVTNmV2g3Wk04TGZpNGpOTmJUdHY5dldl?=
 =?utf-8?B?ZnlLTVl5YTI4OFUyQldrbEtKaU1FKyswWXkzZVBzSHJlUEovK2VDVnZyUWx1?=
 =?utf-8?B?VC9kcWIyU2ZKclNST0hEVU9ZNzVKVWxtR3liNzNCZ29WSGFyelJ3RnIxZlZw?=
 =?utf-8?B?RHgvNldRcVFoc3M1bUNOM1JzcFVjaSsydTBwQ2dkOXdpcEZOQVczbXVqT2RT?=
 =?utf-8?Q?1dn8=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D9CE6193184B554DA02855F94EDE5874@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3a96e41d-2728-4dd3-4c73-08dc70d0707c
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2024 09:06:12.1866
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: t5cnJPiXY7OZRDYDxMbAv23v04mTbBARh9V2+VKlHEB2LeyfNa771srtkUJoEKfndmFKuophmOUnpMWZ6sWHEQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8415

SGksDQoNCk9uIDIwMjQvNS8xMCAxNDo0NiwgSsO8cmdlbiBHcm/DnyB3cm90ZToNCj4gT24gMTku
MDQuMjQgMDU6MzYsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4gSW4gUFZIIGRvbTAsIGl0IHVzZXMg
dGhlIGxpbnV4IGxvY2FsIGludGVycnVwdCBtZWNoYW5pc20sDQo+PiB3aGVuIGl0IGFsbG9jcyBp
cnEgZm9yIGEgZ3NpLCBpdCBpcyBkeW5hbWljLCBhbmQgZm9sbG93DQo+PiB0aGUgcHJpbmNpcGxl
IG9mIGFwcGx5aW5nIGZpcnN0LCBkaXN0cmlidXRpbmcgZmlyc3QuIEFuZA0KPj4gdGhlIGlycSBu
dW1iZXIgaXMgYWxsb2NlZCBmcm9tIHNtYWxsIHRvIGxhcmdlLCBidXQgdGhlDQo+PiBhcHBseWlu
ZyBnc2kgbnVtYmVyIGlzIG5vdCwgbWF5IGdzaSAzOCBjb21lcyBiZWZvcmUgZ3NpIDI4LA0KPj4g
aXQgY2F1c2VzIHRoZSBpcnEgbnVtYmVyIGlzIG5vdCBlcXVhbCB3aXRoIHRoZSBnc2kgbnVtYmVy
Lg0KPj4gQW5kIHdoZW4gcGFzc3Rocm91Z2ggYSBkZXZpY2UsIFFFTVUgd2lsbCB1c2UgZGV2aWNl
J3MgZ3NpDQo+PiBudW1iZXIgdG8gZG8gcGlycSBtYXBwaW5nLCBidXQgdGhlIGdzaSBudW1iZXIg
aXMgZ290IGZyb20NCj4+IGZpbGUgL3N5cy9idXMvcGNpL2RldmljZXMvPHNiZGY+L2lycSwgaXJx
IT0gZ3NpLCBzbyBpdCB3aWxsDQo+PiBmYWlsIHdoZW4gbWFwcGluZy4NCj4+IEFuZCBpbiBjdXJy
ZW50IGxpbnV4IGNvZGVzLCB0aGVyZSBpcyBubyBtZXRob2QgdG8gdHJhbnNsYXRlDQo+PiBpcnEg
dG8gZ3NpIGZvciB1c2Vyc3BhY2UuDQo+Pg0KPj4gRm9yIGFib3ZlIHB1cnBvc2UsIHJlY29yZCB0
aGUgcmVsYXRpb25zaGlwIG9mIGdzaSBhbmQgaXJxDQo+PiB3aGVuIFBWSCBkb20wIGRvIGFjcGlf
cmVnaXN0ZXJfZ3NpX2lvYXBpYyBmb3IgZGV2aWNlcyBhbmQNCj4+IGFkZHMgYSBuZXcgc3lzY2Fs
bCBpbnRvIHByaXZjbWQgdG8gbGV0IHVzZXJzcGFjZSBjYW4gZ2V0DQo+PiB0aGF0IHRyYW5zbGF0
aW9uIHdoZW4gdGhleSBoYXZlIGEgbmVlZC4NCj4+DQo+PiBDby1kZXZlbG9wZWQtYnk6IEh1YW5n
IFJ1aSA8cmF5Lmh1YW5nQGFtZC5jb20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKaXFpYW4gQ2hlbiA8
SmlxaWFuLkNoZW5AYW1kLmNvbT4NCj4+IC0tLQ0KPj4gwqAgYXJjaC94ODYvaW5jbHVkZS9hc20v
YXBpYy5owqDCoMKgwqDCoCB8wqAgOCArKysrKysrDQo+PiDCoCBhcmNoL3g4Ni9pbmNsdWRlL2Fz
bS94ZW4vcGNpLmjCoMKgIHzCoCA1ICsrKysNCj4+IMKgIGFyY2gveDg2L2tlcm5lbC9hY3BpL2Jv
b3QuY8KgwqDCoMKgwqAgfMKgIDIgKy0NCj4+IMKgIGFyY2gveDg2L3BjaS94ZW4uY8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqAgfCAyMSArKysrKysrKysrKysrKysrKw0KPj4gwqAgZHJpdmVy
cy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMgfCAzOSArKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKw0KPj4gwqAgZHJpdmVycy94ZW4vcHJpdmNtZC5jwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCB8IDE5ICsrKysrKysrKysrKysrKysNCj4+IMKgIGluY2x1ZGUvdWFwaS94ZW4vcHJpdmNtZC5o
wqDCoMKgwqDCoMKgIHzCoCA3ICsrKysrKw0KPj4gwqAgaW5jbHVkZS94ZW4vZXZlbnRzLmjCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgIDUgKysrKw0KPj4gwqAgOCBmaWxlcyBjaGFuZ2VkLCAx
MDUgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS9hcmNo
L3g4Ni9pbmNsdWRlL2FzbS9hcGljLmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9hcGljLmgNCj4+
IGluZGV4IDlkMTU5Yjc3MWRjOC4uZGQ0MTM5MjUwODk1IDEwMDY0NA0KPj4gLS0tIGEvYXJjaC94
ODYvaW5jbHVkZS9hc20vYXBpYy5oDQo+PiArKysgYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9hcGlj
LmgNCj4+IEBAIC0xNjksNiArMTY5LDkgQEAgZXh0ZXJuIGJvb2wgYXBpY19uZWVkc19waXQodm9p
ZCk7DQo+PiDCoCDCoCBleHRlcm4gdm9pZCBhcGljX3NlbmRfSVBJX2FsbGJ1dHNlbGYodW5zaWdu
ZWQgaW50IHZlY3Rvcik7DQo+PiDCoCArZXh0ZXJuIGludCBhY3BpX3JlZ2lzdGVyX2dzaV9pb2Fw
aWMoc3RydWN0IGRldmljZSAqZGV2LCB1MzIgZ3NpLA0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIGludCB0cmlnZ2VyLCBpbnQgcG9sYXJpdHkpOw0KPj4gKw0KPj4g
wqAgI2Vsc2UgLyogIUNPTkZJR19YODZfTE9DQUxfQVBJQyAqLw0KPj4gwqAgc3RhdGljIGlubGlu
ZSB2b2lkIGxhcGljX3NodXRkb3duKHZvaWQpIHsgfQ0KPj4gwqAgI2RlZmluZSBsb2NhbF9hcGlj
X3RpbWVyX2MyX29rwqDCoMKgwqDCoMKgwqAgMQ0KPj4gQEAgLTE4Myw2ICsxODYsMTEgQEAgc3Rh
dGljIGlubGluZSB2b2lkIGFwaWNfaW50cl9tb2RlX2luaXQodm9pZCkgeyB9DQo+PiDCoCBzdGF0
aWMgaW5saW5lIHZvaWQgbGFwaWNfYXNzaWduX3N5c3RlbV92ZWN0b3JzKHZvaWQpIHsgfQ0KPj4g
wqAgc3RhdGljIGlubGluZSB2b2lkIGxhcGljX2Fzc2lnbl9sZWdhY3lfdmVjdG9yKHVuc2lnbmVk
IGludCBpLCBib29sIHIpIHsgfQ0KPj4gwqAgc3RhdGljIGlubGluZSBib29sIGFwaWNfbmVlZHNf
cGl0KHZvaWQpIHsgcmV0dXJuIHRydWU7IH0NCj4+ICtzdGF0aWMgaW5saW5lIGludCBhY3BpX3Jl
Z2lzdGVyX2dzaV9pb2FwaWMoc3RydWN0IGRldmljZSAqZGV2LCB1MzIgZ3NpLA0KPj4gK8KgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGludCB0cmlnZ2VyLCBpbnQgcG9sYXJp
dHkpDQo+PiArew0KPj4gK8KgwqDCoCByZXR1cm4gKGludClnc2k7DQo+PiArfQ0KPj4gwqAgI2Vu
ZGlmIC8qICFDT05GSUdfWDg2X0xPQ0FMX0FQSUMgKi8NCj4+IMKgIMKgICNpZmRlZiBDT05GSUdf
WDg2X1gyQVBJQw0KPj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2ku
aCBiL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaA0KPj4gaW5kZXggOTAxNWI4ODhlZGQ2
Li5hYThkZWQ2MWZjMmQgMTAwNjQ0DQo+PiAtLS0gYS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4v
cGNpLmgNCj4+ICsrKyBiL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaA0KPj4gQEAgLTUs
NiArNSw3IEBADQo+PiDCoCAjaWYgZGVmaW5lZChDT05GSUdfUENJX1hFTikNCj4+IMKgIGV4dGVy
biBpbnQgX19pbml0IHBjaV94ZW5faW5pdCh2b2lkKTsNCj4+IMKgIGV4dGVybiBpbnQgX19pbml0
IHBjaV94ZW5faHZtX2luaXQodm9pZCk7DQo+PiArZXh0ZXJuIGludCBfX2luaXQgcGNpX3hlbl9w
dmhfaW5pdCh2b2lkKTsNCj4+IMKgICNkZWZpbmUgcGNpX3hlbiAxDQo+PiDCoCAjZWxzZQ0KPj4g
wqAgI2RlZmluZSBwY2lfeGVuIDANCj4+IEBAIC0xMyw2ICsxNCwxMCBAQCBzdGF0aWMgaW5saW5l
IGludCBwY2lfeGVuX2h2bV9pbml0KHZvaWQpDQo+PiDCoCB7DQo+PiDCoMKgwqDCoMKgIHJldHVy
biAtMTsNCj4+IMKgIH0NCj4+ICtzdGF0aWMgaW5saW5lIGludCBwY2lfeGVuX3B2aF9pbml0KHZv
aWQpDQo+PiArew0KPj4gK8KgwqDCoCByZXR1cm4gLTE7DQo+PiArfQ0KPj4gwqAgI2VuZGlmDQo+
PiDCoCAjaWZkZWYgQ09ORklHX1hFTl9QVl9ET00wDQo+PiDCoCBpbnQgX19pbml0IHBjaV94ZW5f
aW5pdGlhbF9kb21haW4odm9pZCk7DQo+PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYva2VybmVsL2Fj
cGkvYm9vdC5jIGIvYXJjaC94ODYva2VybmVsL2FjcGkvYm9vdC5jDQo+PiBpbmRleCA4NWEzY2Uy
YTM2NjYuLjcyYzczNDU4YzA4MyAxMDA2NDQNCj4+IC0tLSBhL2FyY2gveDg2L2tlcm5lbC9hY3Bp
L2Jvb3QuYw0KPj4gKysrIGIvYXJjaC94ODYva2VybmVsL2FjcGkvYm9vdC5jDQo+PiBAQCAtNzQ5
LDcgKzc0OSw3IEBAIHN0YXRpYyBpbnQgYWNwaV9yZWdpc3Rlcl9nc2lfcGljKHN0cnVjdCBkZXZp
Y2UgKmRldiwgdTMyIGdzaSwNCj4+IMKgIH0NCj4+IMKgIMKgICNpZmRlZiBDT05GSUdfWDg2X0xP
Q0FMX0FQSUMNCj4+IC1zdGF0aWMgaW50IGFjcGlfcmVnaXN0ZXJfZ3NpX2lvYXBpYyhzdHJ1Y3Qg
ZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+PiAraW50IGFjcGlfcmVnaXN0ZXJfZ3NpX2lvYXBpYyhz
dHJ1Y3QgZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgaW50IHRyaWdnZXIsIGludCBwb2xhcml0eSkNCj4+IMKgIHsNCj4+
IMKgwqDCoMKgwqAgaW50IGlycSA9IGdzaTsNCj4+IGRpZmYgLS1naXQgYS9hcmNoL3g4Ni9wY2kv
eGVuLmMgYi9hcmNoL3g4Ni9wY2kveGVuLmMNCj4+IGluZGV4IDY1MmNkNTNlNzdmNi4uZjA1NmFi
NWMwYTA2IDEwMDY0NA0KPj4gLS0tIGEvYXJjaC94ODYvcGNpL3hlbi5jDQo+PiArKysgYi9hcmNo
L3g4Ni9wY2kveGVuLmMNCj4+IEBAIC0xMTQsNiArMTE0LDIxIEBAIHN0YXRpYyBpbnQgYWNwaV9y
ZWdpc3Rlcl9nc2lfeGVuX2h2bShzdHJ1Y3QgZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+PiDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZmFsc2UgLyogbm8gbWFwcGluZyBvZiBH
U0kgdG8gUElSUSAqLyk7DQo+PiDCoCB9DQo+PiDCoCArc3RhdGljIGludCBhY3BpX3JlZ2lzdGVy
X2dzaV94ZW5fcHZoKHN0cnVjdCBkZXZpY2UgKmRldiwgdTMyIGdzaSwNCj4+ICvCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpbnQgdHJpZ2dlciwgaW50IHBvbGFyaXR5KQ0K
Pj4gK3sNCj4+ICvCoMKgwqAgaW50IGlycTsNCj4+ICsNCj4+ICvCoMKgwqAgaXJxID0gYWNwaV9y
ZWdpc3Rlcl9nc2lfaW9hcGljKGRldiwgZ3NpLCB0cmlnZ2VyLCBwb2xhcml0eSk7DQo+PiArwqDC
oMKgIGlmIChpcnEgPCAwKQ0KPj4gK8KgwqDCoMKgwqDCoMKgIHJldHVybiBpcnE7DQo+PiArDQo+
PiArwqDCoMKgIGlmICh4ZW5fcHZoX2FkZF9nc2lfaXJxX21hcChnc2ksIGlycSkgPT0gLUVFWElT
VCkNCj4+ICvCoMKgwqDCoMKgwqDCoCBwcmludGsoS0VSTl9JTkZPICJBbHJlYWR5IG1hcCB0aGUg
R1NJIDoldSBhbmQgSVJROiAlZFxuIiwgZ3NpLCBpcnEpOw0KPj4gKw0KPj4gK8KgwqDCoCByZXR1
cm4gaXJxOw0KPj4gK30NCj4+ICsNCj4+IMKgICNpZmRlZiBDT05GSUdfWEVOX1BWX0RPTTANCj4+
IMKgIHN0YXRpYyBpbnQgeGVuX3JlZ2lzdGVyX2dzaSh1MzIgZ3NpLCBpbnQgdHJpZ2dlcmluZywg
aW50IHBvbGFyaXR5KQ0KPj4gwqAgew0KPj4gQEAgLTU1OCw2ICs1NzMsMTIgQEAgaW50IF9faW5p
dCBwY2lfeGVuX2h2bV9pbml0KHZvaWQpDQo+PiDCoMKgwqDCoMKgIHJldHVybiAwOw0KPj4gwqAg
fQ0KPj4gwqAgK2ludCBfX2luaXQgcGNpX3hlbl9wdmhfaW5pdCh2b2lkKQ0KPj4gK3sNCj4+ICvC
oMKgwqAgX19hY3BpX3JlZ2lzdGVyX2dzaSA9IGFjcGlfcmVnaXN0ZXJfZ3NpX3hlbl9wdmg7DQo+
IA0KPiBObyBzdXBwb3J0IGZvciB1bnJlZ2lzdGVyaW5nIHRoZSBnc2kgYWdhaW4/DQpfX2FjcGlf
dW5yZWdpc3Rlcl9nc2kgaXMgc2V0IGluIGZ1bmN0aW9uIGFjcGlfc2V0X2lycV9tb2RlbF9pb2Fw
aWMuDQpNYXliZSBJIG5lZWQgdG8gdXNlIGEgbmV3IGZ1bmN0aW9uIHRvIGNhbGwgYWNwaV91bnJl
Z2lzdGVyX2dzaV9pb2FwaWMgYW5kIHJlbW92ZSB0aGUgbWFwcGluZyBvZiBpcnEgYW5kIGdzaSBm
cm9tIHhlbl9pcnFfbGlzdF9oZWFkID8NCg0KPiANCj4+ICvCoMKgwqAgcmV0dXJuIDA7DQo+PiAr
fQ0KPj4gKw0KPj4gwqAgI2lmZGVmIENPTkZJR19YRU5fUFZfRE9NMA0KPj4gwqAgaW50IF9faW5p
dCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpDQo+PiDCoCB7DQo+PiBkaWZmIC0tZ2l0IGEv
ZHJpdmVycy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMgYi9kcml2ZXJzL3hlbi9ldmVudHMvZXZl
bnRzX2Jhc2UuYw0KPj4gaW5kZXggMjc1NTM2NzNlNDZiLi44MGQ0ZjdmYWFjNjQgMTAwNjQ0DQo+
PiAtLS0gYS9kcml2ZXJzL3hlbi9ldmVudHMvZXZlbnRzX2Jhc2UuYw0KPj4gKysrIGIvZHJpdmVy
cy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMNCj4+IEBAIC05NTMsNiArOTUzLDQzIEBAIGludCB4
ZW5faXJxX2Zyb21fZ3NpKHVuc2lnbmVkIGdzaSkNCj4+IMKgIH0NCj4+IMKgIEVYUE9SVF9TWU1C
T0xfR1BMKHhlbl9pcnFfZnJvbV9nc2kpOw0KPj4gwqAgK2ludCB4ZW5fZ3NpX2Zyb21faXJxKHVu
c2lnbmVkIGlycSkNCj4+ICt7DQo+PiArwqDCoMKgIHN0cnVjdCBpcnFfaW5mbyAqaW5mbzsNCj4+
ICsNCj4+ICvCoMKgwqAgbGlzdF9mb3JfZWFjaF9lbnRyeShpbmZvLCAmeGVuX2lycV9saXN0X2hl
YWQsIGxpc3QpIHsNCj4+ICvCoMKgwqDCoMKgwqDCoCBpZiAoaW5mby0+dHlwZSAhPSBJUlFUX1BJ
UlEpDQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb250aW51ZTsNCj4+ICsNCj4+ICvCoMKg
wqDCoMKgwqDCoCBpZiAoaW5mby0+aXJxID09IGlycSkNCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIHJldHVybiBpbmZvLT51LnBpcnEuZ3NpOw0KPj4gK8KgwqDCoCB9DQo+PiArDQo+PiArwqDC
oMKgIHJldHVybiAtMTsNCj4+ICt9DQo+PiArRVhQT1JUX1NZTUJPTF9HUEwoeGVuX2dzaV9mcm9t
X2lycSk7DQo+PiArDQo+PiAraW50IHhlbl9wdmhfYWRkX2dzaV9pcnFfbWFwKHVuc2lnbmVkIGdz
aSwgdW5zaWduZWQgaXJxKQ0KPj4gK3sNCj4+ICvCoMKgwqAgaW50IHRtcF9pcnE7DQo+PiArwqDC
oMKgIHN0cnVjdCBpcnFfaW5mbyAqaW5mbzsNCj4+ICsNCj4+ICvCoMKgwqAgdG1wX2lycSA9IHhl
bl9pcnFfZnJvbV9nc2koZ3NpKTsNCj4+ICvCoMKgwqAgaWYgKHRtcF9pcnEgIT0gLTEpDQo+PiAr
wqDCoMKgwqDCoMKgwqAgcmV0dXJuIC1FRVhJU1Q7DQo+PiArDQo+PiArwqDCoMKgIGluZm8gPSBr
emFsbG9jKHNpemVvZigqaW5mbyksIEdGUF9LRVJORUwpOw0KPj4gK8KgwqDCoCBpZiAoaW5mbyA9
PSBOVUxMKQ0KPj4gK8KgwqDCoMKgwqDCoMKgIHBhbmljKCJVbmFibGUgdG8gYWxsb2NhdGUgbWV0
YWRhdGEgZm9yIEdTSSVkXG4iLCBnc2kpOw0KPiANCj4gUGxlYXNlIGRvbid0IGtpbGwgdGhlIHN5
c3RlbSBoZXJlLCBqdXN0IHJldHVybiAtRU5PTUVNLg0KV2lsbCBjaGFuZ2UgaW4gbmV4dCB2ZXJz
aW9uLg0KDQo+IA0KPj4gKw0KPj4gK8KgwqDCoCBpbmZvLT50eXBlID0gSVJRVF9QSVJROw0KSSBh
bSBjb25zaWRlcmluZyB3aGV0aGVyIEkgbmVlZCB0byB1c2UgYSBuZXcgdHlwZShsaWtlIElSUVRf
R1NJKSBoZXJlIHRvIGRpc3Rpbmd1aXNoIHdpdGggSVJRVF9QSVJRLCBiZWNhdXNlIGZ1bmN0aW9u
IHJlc3RvcmVfcGlycXMgd2lsbCBwcm9jZXNzIGFsbCBJUlFUX1BJUlEuDQoNCj4+ICvCoMKgwqAg
aW5mby0+aXJxID0gaXJxOw0KPj4gK8KgwqDCoCBpbmZvLT51LnBpcnEuZ3NpID0gZ3NpOw0KPj4g
K8KgwqDCoCBsaXN0X2FkZF90YWlsKCZpbmZvLT5saXN0LCAmeGVuX2lycV9saXN0X2hlYWQpOw0K
PiANCj4gSSB0aGluayB5b3UgbmVlZCBzb21lIGtpbmQgb2YgbG9ja2luZyB0byBwcm90ZWN0IGNo
YW5naW5nIG9mIHRoZSBsaXN0IGFnYWluc3QNCj4gY29uY3VycmVudCBhY2Nlc3Nlcy4NCk9LLCB3
aWxsIGFkZCBhIGxvY2sgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+IA0KPj4gKw0KPj4gK8KgwqDCoCBy
ZXR1cm4gMDsNCj4+ICt9DQo+PiArDQo+PiDCoCBzdGF0aWMgdm9pZCBfX3VuYmluZF9mcm9tX2ly
cShzdHJ1Y3QgaXJxX2luZm8gKmluZm8sIHVuc2lnbmVkIGludCBpcnEpDQo+PiDCoCB7DQo+PiDC
oMKgwqDCoMKgIGV2dGNobl9wb3J0X3QgZXZ0Y2huOw0KPj4gQEAgLTIyOTUsNiArMjMzMiw4IEBA
IHZvaWQgX19pbml0IHhlbl9pbml0X0lSUSh2b2lkKQ0KPj4gwqDCoMKgwqDCoCB4ZW5faW5pdF9z
ZXR1cF91cGNhbGxfdmVjdG9yKCk7DQo+PiDCoMKgwqDCoMKgIHhlbl9hbGxvY19jYWxsYmFja192
ZWN0b3IoKTsNCj4+IMKgICvCoMKgwqAgaWYgKHhlbl9wdmhfZG9tYWluKCkpDQo+PiArwqDCoMKg
wqDCoMKgwqAgcGNpX3hlbl9wdmhfaW5pdCgpOw0KPj4gwqAgwqDCoMKgwqDCoCBpZiAoeGVuX2h2
bV9kb21haW4oKSkgew0KPj4gwqDCoMKgwqDCoMKgwqDCoMKgIG5hdGl2ZV9pbml0X0lSUSgpOw0K
Pj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3ByaXZjbWQuYyBiL2RyaXZlcnMveGVuL3ByaXZj
bWQuYw0KPj4gaW5kZXggNjdkZmE0Nzc4ODY0Li4xMWZlZWQ1MjllMWQgMTAwNjQ0DQo+PiAtLS0g
YS9kcml2ZXJzL3hlbi9wcml2Y21kLmMNCj4+ICsrKyBiL2RyaXZlcnMveGVuL3ByaXZjbWQuYw0K
Pj4gQEAgLTg0Miw2ICs4NDIsMjEgQEAgc3RhdGljIGxvbmcgcHJpdmNtZF9pb2N0bF9tbWFwX3Jl
c291cmNlKHN0cnVjdCBmaWxlICpmaWxlLA0KPj4gwqDCoMKgwqDCoCByZXR1cm4gcmM7DQo+PiDC
oCB9DQo+PiDCoCArc3RhdGljIGxvbmcgcHJpdmNtZF9pb2N0bF9nc2lfZnJvbV9pcnEoc3RydWN0
IGZpbGUgKmZpbGUsIHZvaWQgX191c2VyICp1ZGF0YSkNCj4+ICt7DQo+PiArwqDCoMKgIHN0cnVj
dCBwcml2Y21kX2dzaV9mcm9tX2lycSBrZGF0YTsNCj4+ICsNCj4+ICvCoMKgwqAgaWYgKGNvcHlf
ZnJvbV91c2VyKCZrZGF0YSwgdWRhdGEsIHNpemVvZihrZGF0YSkpKQ0KPj4gK8KgwqDCoMKgwqDC
oMKgIHJldHVybiAtRUZBVUxUOw0KPj4gKw0KPj4gK8KgwqDCoCBrZGF0YS5nc2kgPSB4ZW5fZ3Np
X2Zyb21faXJxKGtkYXRhLmlycSk7DQo+PiArDQo+PiArwqDCoMKgIGlmIChjb3B5X3RvX3VzZXIo
dWRhdGEsICZrZGF0YSwgc2l6ZW9mKGtkYXRhKSkpDQo+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJu
IC1FRkFVTFQ7DQo+PiArDQo+PiArwqDCoMKgIHJldHVybiAwOw0KPiANCj4gU2hvdWxkbid0IHlv
dSByZXR1cm4gYW4gZXJyb3IgaWYgeGVuX2dzaV9mcm9tX2lycSgpIHJldHVybmVkIC0xPw0KT2gs
IHdpbGwgY2hhbmdlIGluIG5leHQgdmVyc2lvbi4NCg0KPiANCj4gDQo+IEp1ZXJnZW4NCg0KLS0g
DQpCZXN0IHJlZ2FyZHMsDQpKaXFpYW4gQ2hlbi4NCg==


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719609.1122427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MGJ-0006Np-Rk; Fri, 10 May 2024 09:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719609.1122427; Fri, 10 May 2024 09:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MGJ-0006Ni-Mg; Fri, 10 May 2024 09:09:55 +0000
Received: by outflank-mailman (input) for mailman id 719609;
 Fri, 10 May 2024 09:09:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=McEs=MN=dingwall.me.uk=james@srs-se1.protection.inumbo.net>)
 id 1s5MGI-0006Nc-DF
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:09:54 +0000
Received: from smarthost01a.sbp.mail.zen.net.uk
 (smarthost01a.sbp.mail.zen.net.uk [212.23.1.1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0ef2156c-0ead-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 11:09:51 +0200 (CEST)
Received: from [217.155.64.189] (helo=mail0.xen.dingwall.me.uk)
 by smarthost01a.sbp.mail.zen.net.uk with esmtpsa (TLS1.0) tls
 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (Exim 4.95)
 (envelope-from <james@dingwall.me.uk>) id 1s5MGE-00Fd5h-Ia
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:09:51 +0000
Received: from localhost (localhost [IPv6:::1])
 by mail0.xen.dingwall.me.uk (Postfix) with ESMTP id 0F2B89D6EDB
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 10:09:51 +0100 (BST)
Received: from mail0.xen.dingwall.me.uk ([127.0.0.1])
 by localhost (mail0.xen.dingwall.me.uk [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id DHeB11bR55EC for <xen-devel@lists.xenproject.org>;
 Fri, 10 May 2024 10:09:50 +0100 (BST)
Received: from ghoul.dingwall.me.uk (ghoul.dingwall.me.uk
 [IPv6:2a02:8010:698e:302::c0a8:1c8])
 by dingwall.me.uk (Postfix) with ESMTP id DDE9A9D6ED8
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 10:09:50 +0100 (BST)
Received: by ghoul.dingwall.me.uk (Postfix, from userid 1000)
 id D13885AE; Fri, 10 May 2024 10:09:50 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ef2156c-0ead-11ef-b4bb-af5377834399
X-Virus-Scanned: Debian amavisd-new at dingwall.me.uk
Date: Fri, 10 May 2024 10:09:50 +0100
From: James Dingwall <james-xen@dingwall.me.uk>
To: xen-devel@lists.xenproject.org
Subject: Domain Birth Time
Message-ID: <Zj3kXlHJzlqaFpMw@dingwall.me.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Originating-smarthost01a-IP: [217.155.64.189]
Feedback-ID: 217.155.64.189

Hi,

We've added a feature to Xen 4.15 such that `xl uptime -b` reports the birth
time of the domain (i.e. a value preserved across migrations).  If this would
be of wider interest I can try porting this to a more recent release and
submitting it for review.

Regards,
James


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:17:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:17:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719616.1122439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MNh-0008NG-Hx; Fri, 10 May 2024 09:17:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719616.1122439; Fri, 10 May 2024 09:17:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MNh-0008N9-EW; Fri, 10 May 2024 09:17:33 +0000
Received: by outflank-mailman (input) for mailman id 719616;
 Fri, 10 May 2024 09:17:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+rhO=MN=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s5MNg-0008N3-SP
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:17:32 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1f3f8b27-0eae-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:17:29 +0200 (CEST)
Received: from MN2PR06CA0027.namprd06.prod.outlook.com (2603:10b6:208:23d::32)
 by CH3PR12MB7642.namprd12.prod.outlook.com (2603:10b6:610:14a::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.49; Fri, 10 May
 2024 09:17:25 +0000
Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com
 (2603:10b6:208:23d:cafe::5a) by MN2PR06CA0027.outlook.office365.com
 (2603:10b6:208:23d::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48 via Frontend
 Transport; Fri, 10 May 2024 09:17:25 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Fri, 10 May 2024 09:17:25 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 04:17:24 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 04:17:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f3f8b27-0eae-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AHTSkPXUlMFH8qve5zWU2dBDrvX11MwR2iGINpbw4HPzOwiIQnt6E+BH6cXIKEzDF57WSvyE0HBJdnEwLWmueH/Om+25w08SBSuVvBlHNIJIeNH8v4wlwgacrFNq352zuje4GvtUkTW7vTvXyEBjL0Pp8fIRuOXfsqg/v0PqC6USc1npWE7SLr+/pJf2SWGIBekTzi+VjcotKGqfaV7e7K9X+yxJQdpLPCj3bu3Y9bHUtuw7Mcx6HK30SxoJFynjU9+08IyGJ4zZJvlcmlIY/tt08hhvPHBPwMsYJyHtUFAgYjjVIs8nmh8rD7VzjbK6ioH7D9mVfvz4Lz3D3LYlSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Lj+U8YqWUH/kMLeIlkE57KNm9IA8QVsOtwHc4YUjnbc=;
 b=Q9EvgGhsJ+koIWWVzLR7f2WA3SywTeZwOyzb5vYC2UeKGN7PouJaxwGv4Ai6EiR07u513OkDsRVaz4SN0duV5S6dlCRyzhwVDamI9u9tdVK/0q5K89SC5YWDOhdUHqNGP5AGv+5mJ0WABNLKhiiZCOg/k5AiCkmcuqZJ1T8WLkCaNohrRZn95yCfHeaZpDLMy+U4tNMnOTTedt8wsbCUIUP02NCQyI7EMFBSmDcT9KRW+o/dDXKhO7R9AbUMsgx9wHbHUohbdl8mfSHY/OQcllvy1uIUB92W0E8RCzVpyrz/AFs7YT8TRPqKb6bQIV0nO0hKVrjBXqT/vVfWSBJvLQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lj+U8YqWUH/kMLeIlkE57KNm9IA8QVsOtwHc4YUjnbc=;
 b=VDCBChkBDReMc5EPL1qYR4ZdGCafCYau2bONPGTmflSLtcIm5c3wSUCkJSHHDQ+2Hc5Ih7bGPwsXdh/qGqZIWuxepJMmOfjSb8zfFkxYdXA4U1TvbOAJlJaKHG9MyfOJc0+ioebtZdd/fYetAWObkZi4eq9DdICyf0VSW938Ugw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <cb496741-dd76-4536-818a-cbfdbd496b0d@amd.com>
Date: Fri, 10 May 2024 11:17:23 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/7] xen/arm: Implement the logic for static shared memory
 from Xen heap
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-7-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-7-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|CH3PR12MB7642:EE_
X-MS-Office365-Filtering-Correlation-Id: 4cce3d4c-d0f3-4ac5-2667-08dc70d201ab
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?S0k0Y1JPa2ZrVk4yc1BueVFVVm9zaDZhUjMyc3ZvY1dUZml1SlExY3ZObzR1?=
 =?utf-8?B?VjB4ejNnWFVpVDBwQXZPV01LNWZHMVpIa2R2Nk96SWRHbEdQMzRTZGtNdzBW?=
 =?utf-8?B?QXdnTVlabllDdUlYTzdMN0F1VHF5TmIyWWljQTFScW5WNTA1WG1MVVZMWWFS?=
 =?utf-8?B?Z2swRTF1dE5kUm9aZTB1VWk0eDlhdm5acXF2dUI2SmM0KytielRpM1NFZVhB?=
 =?utf-8?B?d2l0UVo5R2pIUnpGQ2kxcDQwZ0ZnV0pIM2pkbmlET3IwSjdnSmlZTkdGWERG?=
 =?utf-8?B?ajMwTC9hWkM1TE1zNWQ4K1Z4bEtDNzJ3ejJQRWh1bDZGc05zbERFTkhvSW5Z?=
 =?utf-8?B?U0hIajVBU0pTd2RYYWovaXNDZWRrTzVYZkdSK1c0akpBaEc2SnNNRDQ2NkRa?=
 =?utf-8?B?ODhCVm11bEdkeVdxMFVhN3BVOTJpOHBuNnBLY2hsWFY3QzhCRGxwd3ArejFD?=
 =?utf-8?B?cUl6R25scDZRY1RkVG8yeWFwTXUrK1QyNk1Wa1RPTUN3SVRIKzk5U0FMc1Ux?=
 =?utf-8?B?TkdTcFNHbGw1RzdkcW1ZZHNlaHlXMjd4aGFDM1NWMDVObWxKZWxEQVZKM2du?=
 =?utf-8?B?ZHlCL2NwWkw5MnlPcnArcG1sZC82Uy95S1dYRThHVllnakZyb3Q3bGFDc3B1?=
 =?utf-8?B?VzQ1UHJBOC9FMUtQOG5lTnpTWW45aUFKRWptSUY3U3RvOE4zZnFuVUxmaFNV?=
 =?utf-8?B?YXRtMlZUcm1VaTNtRVM1Rk1zOGhNZUFwMlJhM25pcHJIcEJsNmhXSkRCblpx?=
 =?utf-8?B?UTQwRWY0RFNhajFuK0tpUm5aUmVsZDVzd3ZHTlBPNFNRUHJxMTBsS0pXbkNB?=
 =?utf-8?B?NjVjTnRTNlNieCtGSUQvUTczSGlIWjJHellSY3pCKzZTeXNydU5KTnluTUVk?=
 =?utf-8?B?VHh1KzQrUDIvaTFhVm1VY2trYTFycnRnekxVUGVIQis4SmU3dGhmQzNhTUFi?=
 =?utf-8?B?U3FISyt4cExnSlMwNUwrUjZxYnorQnJXRGNMbzQxejNsUWpDTzR0OHU0aDFB?=
 =?utf-8?B?SzJsbGRVRENDbDA3YTVSTVliMzdzYW40U1ZFYWowY0RjaWY3bW1kNnZhRG9M?=
 =?utf-8?B?RENVdmdWbXovdTMwRUU1NERQY3JHU2VyNDRtMUJ0aTlLaFJPZ1UwUEN6U3h0?=
 =?utf-8?B?TGJFR1o4cEFKUE9aZUVYd2VacS9ScTJmWlh6bVRRNCs4TUNOdjNGQXRSLzVN?=
 =?utf-8?B?Sy9Yb1ZJNU8wdVR0NURxaWhURVhtRmdxQ0lxTHNiNnlwK2FGV3B0aWdCK2Z1?=
 =?utf-8?B?MWFJdGkvRC9odmlMMFU5VE5kakV4VmU4N0FTS2lUVlVrQjR6c2xEbFNvb2Ji?=
 =?utf-8?B?dWtXMXIzdE85NkIxcWQ0L2JLWXZuVXF4Wks0TDc1TjZsUXFGbkNnVzdXckwx?=
 =?utf-8?B?Z2xlcjdPZ0UwOC85NnZJbWhDMXc1eEdQTUhMZnZzTUlUNG9LRExQQUd2TE9m?=
 =?utf-8?B?MDNEYkxGUXZYNktBOFVoRHdXZm9SYUFVb2tMQ1k3ck03VmdIMm52MFVJMUV4?=
 =?utf-8?B?SW9pNG9paDNPZmYxWmhuYVV1Ylg3QXp0Z1FJaHlyTTlnZzZtVUIvZ3c0bGZP?=
 =?utf-8?B?S0RoclVSL3VmOGp1NGp2WWFWQzZVbitDeU5OazkxeDhzZGo4RlIxY2IyUHZW?=
 =?utf-8?B?MTBUMGozRm9UWVBxQUZuNElIdGhYVUxMNmg3dVVvalZkUXYyQ0VwQ08reXBH?=
 =?utf-8?B?ZG16K1pDYmNhbE01clZuZXp2ZUNReFFYdXN4YnJoVE9lbFVxSWU0a1p1NjJC?=
 =?utf-8?B?cWlna0J3TjBzUDI4WUEyd3JkWldjckt5QU1qUDRyUWw2MXErL0lEbkd4S2Fk?=
 =?utf-8?Q?LSKUY8hZ+NzhYmu4EkPVPxbswS1aJ+mcUzMQw=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 09:17:25.2735
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4cce3d4c-d0f3-4ac5-2667-08dc70d201ab
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000F0E5.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7642

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> This commit implements the logic to have the static shared memory banks
> from the Xen heap instead of having the host physical address passed from
> the user.
> 
> When the host physical address is not supplied, the physical memory is
> taken from the Xen heap using allocate_domheap_memory, the allocation
> needs to occur at the first handled DT node and the allocated banks
> need to be saved somewhere, so introduce the 'shm_heap_banks' static
> global variable of type 'struct meminfo' that will hold the banks
> allocated from the heap, its field .shmem_extra will be used to point
> to the bootinfo shared memory banks .shmem_extra space, so that there
> is not further allocation of memory and every bank in shm_heap_banks
> can be safely identified by the shm_id to reconstruct its traceability
> and if it was allocated or not.
> 
> A search into 'shm_heap_banks' will reveal if the banks were allocated
> or not, in case the host address is not passed, and the callback given
> to allocate_domheap_memory will store the banks in the structure and
> map them to the current domain, to do that, some changes to
> acquire_shared_memory_bank are made to let it differentiate if the bank
> is from the heap and if it is, then assign_pages is called for every
> bank.
> 
> When the bank is already allocated, for every bank allocated with the
> corresponding shm_id, handle_shared_mem_bank is called and the mapping
> are done.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

I tested this patch and it resulted in assertion:
Assertion 's <= e' failed at common/rangeset.c:189

I checked and in find_unallocated_memory(), given that start is ~0UL (host address not provided),
start + size would overflow. Did you test this patch?

~Michal


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:25:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:25:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719624.1122449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MVj-0001vq-BZ; Fri, 10 May 2024 09:25:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719624.1122449; Fri, 10 May 2024 09:25:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MVj-0001vj-73; Fri, 10 May 2024 09:25:51 +0000
Received: by outflank-mailman (input) for mailman id 719624;
 Fri, 10 May 2024 09:25:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jEWd=MN=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s5MVh-0001vd-SI
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:25:50 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2611::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 48ede01a-0eaf-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:25:48 +0200 (CEST)
Received: from AS9PR06CA0620.eurprd06.prod.outlook.com (2603:10a6:20b:46e::18)
 by GVXPR08MB10810.eurprd08.prod.outlook.com (2603:10a6:150:150::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:25:45 +0000
Received: from AMS0EPF000001AE.eurprd05.prod.outlook.com
 (2603:10a6:20b:46e:cafe::ee) by AS9PR06CA0620.outlook.office365.com
 (2603:10a6:20b:46e::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47 via Frontend
 Transport; Fri, 10 May 2024 09:25:45 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF000001AE.mail.protection.outlook.com (10.167.16.154) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Fri, 10 May 2024 09:25:44 +0000
Received: ("Tessian outbound b7675f20d34d:v315");
 Fri, 10 May 2024 09:25:43 +0000
Received: from bd6c78b1c7e9.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D20B5C61-11BA-4B6A-8DB6-9DD13D7A4B1C.1; 
 Fri, 10 May 2024 09:25:37 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bd6c78b1c7e9.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 May 2024 09:25:37 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB5PR08MB10161.eurprd08.prod.outlook.com (2603:10a6:10:4a5::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:25:35 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.048; Fri, 10 May 2024
 09:25:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 48ede01a-0eaf-11ef-909d-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=WoMIFSpLToePcC2S1YLX7Y0jR9ZScLxKUeeb3pLRyb8EOCbdZpLGQZuyu+ZD9X/IeEuHKbCjUi3ZE9hIFAPYYaBLMH3ptQ+p9DPVhbNgNaKNlOdqoo2twd413SzPeUJHBfPit0pcSk+aG+q/jFqLUFBkHSjdv/vaSgSoOqhf/A99B5JRVCo73o67Z1OraUqRmevMJ/+jasl0/eTsGgohGnPi1S+R/iQDZp9vMqn6EOed3jHwUw7b7v/y6CQySn7zeL0adqe8Spjk4M6rJNBSJ+6gAyQBxCLIgDgyXd2ky3C7atoaBDCTooOXmh3F43fpCB+Z1O3rvHHeNpYWt08kmw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Y/DxX1DNCNf2QgaMJTIwoc53PkU1J8qu6qS4WJS5Qoc=;
 b=KmcaGVym2dWaxKn6RvG7dJCfozNjFJ2mzhUGLAv+T4UglQsNHzG9Do6f9Cv0FUiXpcI0souZnYGNJrdJIZq+YceKRW3Ht2m7UO+/N+7DrljcDd+J8HW9q1PLtMw53YlbOavg87PVRONsRhk+2eyfK0epOthg2gJ4tQv5BAxyLSCf8cV7tGpoUOJvNXH+m0I9NlRpsNZ2sW7FiDQ01LotNRodzVgjJQ3MHmtDD6WGGGb8idhbT5ixRe51dt217DQnDPGIbPSzyEi3keHZLJOyKYxjrCxzB6EhtAwuEEIYdNipDYquMZtOW7y+96KtnQtTDTfHvLCl3K3n5d7LGloovA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y/DxX1DNCNf2QgaMJTIwoc53PkU1J8qu6qS4WJS5Qoc=;
 b=GwW2n91rRrLiwn6VHYvlzb2LcltrXgP6Z+jp+4CV6PdHsQf9nMholO+RowUE6AkVgYTmV9ajP6UaLM3Rf5EJ55NcOuUiQcAqr9r7NAzvSjT6+A/c9ieFSQbENoHx+9j4x3Et3RxQ0lGE4f5xSIb+mTK4gI5TsDPi9F1ZVrm/D4M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 49ca7593cfc0f2c3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SYy5fgiSAn7b695MtJPysFdGHjeJoa3Pjj2sfNH6JJSj88d/0lEoHQmXW9FYBFvrx3VuGqJQ96WzVNe4JH3uTytrsJgbLyPzzTDlOpAMW6aqC0O+oI9X8wSuryTpt9a7rkCh4/U9NZixYE4EWuHkaMAY8qLpLfHryCKnd/F/6AObAjmCIMoF4NbAau9RyuO3Kkditgl9YaeDbrSSOeHuIH9vU7jAWrsZjOWqZ7f9736PFs0EBJbZoRVb/Le6H/PJCp8l6Hh52rjR5zPPfeN2i5Wnm+QW5Jd05O5nf8PpLVra/G+CppagaX569kkP5ocgt6TItjpcU6l0t3IjWirC9Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Y/DxX1DNCNf2QgaMJTIwoc53PkU1J8qu6qS4WJS5Qoc=;
 b=dtDRAHL+uEka9C91mG8YN7j4t7DOQWBZygfpVu20BduWmJXn52kiMPd4W1EWvAUf1sOth1KeURjhKjNtcvko89y4rpGixqPVh0OCG+7xtJPvKK0kbVqyM6k2koQ4A4fJJAyFs8mQk5cawU0QrvKc/gbjxTrJ2B2g0/vODeuKaIbnCPURBPkCplSSNcbdODsc7oUkTdEgnQYKyxSeGwgjg1vfKId2//fHarJJMAgrfsv4KYrtc0ZkgG5U+nQTIDx0VefqKEoxOM6uUA8k1NhkVgm7w+6C5XeN4aeLdQ4Af1za9UJYkQAM2nZtC2lqKUQipXDFnhmUQI/fPRtDRLXSVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y/DxX1DNCNf2QgaMJTIwoc53PkU1J8qu6qS4WJS5Qoc=;
 b=GwW2n91rRrLiwn6VHYvlzb2LcltrXgP6Z+jp+4CV6PdHsQf9nMholO+RowUE6AkVgYTmV9ajP6UaLM3Rf5EJ55NcOuUiQcAqr9r7NAzvSjT6+A/c9ieFSQbENoHx+9j4x3Et3RxQ0lGE4f5xSIb+mTK4gI5TsDPi9F1ZVrm/D4M=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 6/7] xen/arm: Implement the logic for static shared memory
 from Xen heap
Thread-Topic: [PATCH 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Index: AQHalVfty94kOt4AK0ecWyByn7T1+rGQSwuAgAACP4A=
Date: Fri, 10 May 2024 09:25:35 +0000
Message-ID: <10212FF3-3CCD-4634-9890-4CDCA11A5F88@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-7-luca.fancellu@arm.com>
 <cb496741-dd76-4536-818a-cbfdbd496b0d@amd.com>
In-Reply-To: <cb496741-dd76-4536-818a-cbfdbd496b0d@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB5PR08MB10161:EE_|AMS0EPF000001AE:EE_|GVXPR08MB10810:EE_
X-MS-Office365-Filtering-Correlation-Id: 9bea7074-8234-45b7-586e-08dc70d32b15
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?d01LSjFUcE14RHVwYnEyb1VRd05kQmlFSDJIdHNrdVJrY08raWlUbng1NDJ3?=
 =?utf-8?B?em9rUCtWbnFBbHNoRjFYZk5hNFFCZGsyUUc1RUovazN4YzNha1Z0WGJrYlhK?=
 =?utf-8?B?amZVREdwMUhnMWZLUXR1M0hYMUJVeHhweGFyZGJEUjliVVN0WUZVM2hWcU1Y?=
 =?utf-8?B?Wm1GM29xVGNOc2JTY2txVjBiaFZ2Uy91SlZ0Z0pPdmFJUk41VzJ3MzZVekdX?=
 =?utf-8?B?K2pDMVNkRHF4bGgvNnhkUWthVHRiUkV1c3NvbUtPZS9wRjdsYVV2UmpMSFly?=
 =?utf-8?B?dG1Iek53VVhUenBLL0VNaXRPeC9GVHd3TEErYnJNVUd6K0dpd25yb3lSOFBs?=
 =?utf-8?B?bHZwWTRsSXFTRTZITWhJVm10Rzd0SXNCTzBCd00wbjhkam9VM093amJMOUhy?=
 =?utf-8?B?MkVaUUxFM1c2OHhpWlJyeDZHTll5aE4vTnhGR1IxNlpnVzkxU1BrRisyc0dI?=
 =?utf-8?B?c3pGckljSDI0Y3RZUTlWVHRiMGRFN0IwWlU4aitwM1ZQb0ZXNUd1S0JrMTRD?=
 =?utf-8?B?Zk5ENkhOY21WU2gwUlZHVDhMMzgxc09vd0QrSDZlclliNGIxRkF2MFRUSVRa?=
 =?utf-8?B?bE5KNTlrcXBkaFZTMEJPQXhlaEdUdWVNQ2ZWeEhQU1RKT1cyczczdlNCNERI?=
 =?utf-8?B?aGJSSFNqaXR1VzRSMjY0dTJFc05wb0pRSk13U21lb21ZL0lONlFKM2gzMHJR?=
 =?utf-8?B?UGlhRGRFZDQwY2V6RmJPcGNvVDdhVjB2M2YvWTJ1WUxUWmI2UTMwZ2tKMlRM?=
 =?utf-8?B?OGxzL1VtWk5kZjQ0Y0tGUG1kRlZ0YU41dGNESVljRGZ5c3hSSFNQb0tBSnF5?=
 =?utf-8?B?cGRpSG9la1FZN0hpanYyV0VFbk9LcGdNM2NnK0xFWjhQcG5uUHBWTm9GVm1k?=
 =?utf-8?B?ZGxSNGJKYW9xY0xpQzRCYTRnYU9vcUhjZDhhVUU3WlBON3ZoSEdrOFFvT0U0?=
 =?utf-8?B?eXpVUUxaLzNyM2ZJazRhem0vd2xaVXZFQXpka3cyblp2THRFeS9QdUgvSndN?=
 =?utf-8?B?VldvUFZRd3hIdENHZkhaek5LYTNuQkkxTVY2b1VXVE5mUkQyWGVHZDd6VTR4?=
 =?utf-8?B?eWFpQlEya2l0cXdySU40SmtCckFKMUFRSHVXSDhVMm5udEo0cllITGxaeVNL?=
 =?utf-8?B?NmZzRlhKNnA0WDV4SUVKbnAxbzI3VlZpSGdxZVk3NDA4dXA1ZmRWdmNYTEdT?=
 =?utf-8?B?ZXJDNHRNZ01sWVBsTkgzMEhzVzlGR0QweU5vNy9YZnJBQWF2cjNHMUdTS3lS?=
 =?utf-8?B?VFZuWDcxRzZ0d1ZIYktoUTNnTmNRMjVwT1pyQnYzNlpvVkZtNmhEUTAwSFZS?=
 =?utf-8?B?cWptVi9QZnl1c3RzRGJoOXVSQytrTCtOYnRXSkQxQ3BEUVpXWHpxYXdFQ0lH?=
 =?utf-8?B?YnB2R2ZlMmZrS1dpQzl2Ry9URVhUa0NncElhZkhBVDJYaVNTOHdBa0xrSCt5?=
 =?utf-8?B?ZTltVHFoNkpFbmZROFdVOC9OcGlGTzFCVlhMVktvTkdLWjNydGlkVnN4cHpI?=
 =?utf-8?B?RXY3ZXhOL0YrSnZNNFhIZzNFSnBzaFBaUFhpUTlhRC9oTVpLbzluRVYyNnZQ?=
 =?utf-8?B?OEQ1YTFPeUVWTGRFV0xMdVpnd3FQR3lQZlZIYStBVlNGeVV0bmk5a0d3R1dE?=
 =?utf-8?B?V3RWclg2cE9jazF6NDBhdHpTUXFTY0lLNk1qalhSaWNoZTRhbkMvUndBOHp4?=
 =?utf-8?B?T01wNHFDUklyOTZ2Q3ZTaUE3enc5N3FUeEY5ejdDYmtiSDc1MDQwbnFwMDdK?=
 =?utf-8?B?MUcyWDUyR3A3MHBtK015WFlLYU5nNjl5Wi9ybnpDMC9HSFovZ2EvbXZWT3No?=
 =?utf-8?B?dFUrMFlwYkg2K1hlbjBqZz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <49A7C1245BA31D4BA7C050717371F13A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR08MB10161
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF000001AE.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	891d4236-6d1e-4340-995a-08dc70d3261b
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|35042699013|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NFgwN1JsTnduOVdrUU5aV3pWSjdsZTJBM3ZKSEMzNm10ODVId2E1MmxYT1gr?=
 =?utf-8?B?SzBYcnJYN2IrcTZvaW41Q2hTMFZZdlJVdFQ3Z29QbWNuQ1pXYVZPRmFYb1Fs?=
 =?utf-8?B?SHJQWko1S3dGSC91ZnlOVFFtVWh4MEtkMFliRWpMdE9nU1RFU0FZSWtMZVdp?=
 =?utf-8?B?T254OVc4VnZqRDMrMFpnSUJvc2xGWk9FeDloT0Q0Y2xpajFnUFEwRW80VG5R?=
 =?utf-8?B?NXMwUmdkQTlSeDg4K2JwcWZxWGYya0lHL1FDTFpLM0NwWDYwaW43ZEZnb3ZF?=
 =?utf-8?B?eUNyTGJPaDE2SVd5bm5Nd085ei9YL3NWMGJCK2lqZGM4V0dhSlVwY2xIYW1k?=
 =?utf-8?B?RUJjQXI2cmV3MFFnQW93TGVQSFRleVVDZVVKNFdNbUw2WkwzZmNiZDJ6ZUNn?=
 =?utf-8?B?MFRCclYzYS83Z0l1MFdUVXlhak8yWDlSNHBrMWEvak9YSTBsVHFqSzZxOXZu?=
 =?utf-8?B?b0NMNmFmMG9CMWdoY2R6bnkyNU1uTTBtMWYwSERWV09FS2J4cTdjeFRTaE0w?=
 =?utf-8?B?MlFNaU55NFI5OWIzNm50cTcyY2dHNzlINXA0UTNmTWExQWdBVTRTK0tPM1Ny?=
 =?utf-8?B?YTNGMi95TlNOeUdTdjdkZ1JtcGZ5VjFoSHg1T2FISjVTb282NGxmcGxoUkZN?=
 =?utf-8?B?TTE4U3JtTzZmNXA3YmFUQXoxdEw3U3dJMnFRK28vN0pVUXdYVjZ5MHRMdzBt?=
 =?utf-8?B?Ri9jUGIxWkVxanBQRjRCaEJpZjdaR1NRWklvd29McFo0MUs1bkV1RmlmV2w0?=
 =?utf-8?B?TElrQit6citxZFdZV3pEN0RZaExyajNuRURGM3orUDhIbzNVSnpzSmdWZ2Ji?=
 =?utf-8?B?WElsTHROTXJvQ2NSZDRVZGQxN2ZZais2U3JoSGkwSUNRWnY5RmN0cHFaVno2?=
 =?utf-8?B?b0c0OFREeSsxSnJFeW4zZGlFTzV3U3U5MzUzTVQ0Y2Y4NTZwZ0lUbTFRS3pn?=
 =?utf-8?B?NXFsOHpwMDlxeGtJOHJGRHo0VHcrUysrcm5HR2xNaGc1TEZVVnVhUGJoUEgr?=
 =?utf-8?B?TU9FRGU3eXByRXRidmhiVjVwb3ZWYkxoTG9zczgySVRkTlZUNGxCY1lRV3ZO?=
 =?utf-8?B?WG1mT0ZyZjBjdm9acVBGekk2bi92ZGxtOUhtbWZaaFc5VEM2M3RkYkNKVDVG?=
 =?utf-8?B?cmlEMGd0cWJ6UldaTzdUYi9KNDF1bXpNeGc2WlVNdkVVV0VNelViUDFIaGZz?=
 =?utf-8?B?WmtxR3gwYTQvbXVnT0RJNk5Va0x4bHNZVkJNWjExQTJtTlhqWm91dnJodXll?=
 =?utf-8?B?Ujh6NCtSQTJTVXlQQ3lWN3VFWnVGNWRHdkdJdnJOZmNBTTkyeGFnNGJCZEt4?=
 =?utf-8?B?RWVXZGFyLzFxWktXdldLQlh6U2hGZ2RuclZMU0owZUZvcGxuNHg0R0NvYVI2?=
 =?utf-8?B?blZzcDNOZUF3RHE2UHJPb21Qckw3endCTjdxSFNvYW9kVHg5TjYvcVRSUE40?=
 =?utf-8?B?V0RXTGZvd1VIWVRydFZKbVlwaFFEMWw4dEsrRm5waUw1aWpubVdaWk5hVFNw?=
 =?utf-8?B?UU9rcGp5M2Jjb0VFMldlWmtjWHNVQ21uV255ditwS3VYQW9abG9FT1hnU1BG?=
 =?utf-8?B?dWhEWVV0Qkp4c2xGSGJPWXJWNjUxYUd4NTNibGNwV3pLR1Bvbm4zRjEwMGUw?=
 =?utf-8?B?YkQ0NzEyZnFHNURqb0YyRHVHWXpJcjI4dE9FSnZJc0tzMG1sc1lxVFBjclY4?=
 =?utf-8?B?b3FXM2o2d1lhdS83M1d1Yy9WSmVLZTN1NDBXVE5vS3d4YmhWdXIxaGxCeUxH?=
 =?utf-8?B?eU0rZDM4RlJvelROeExZOE84cFZ1VXNTUTRsVERFazNOQ2dXUUhUWU5YWkEr?=
 =?utf-8?B?UW9aY0liS2NNMi9YdnQ5emJDdkN5UXJSVnErWEFzWmxFU0ltNE11cGUxekVw?=
 =?utf-8?Q?Ep5DgRy3iFBq7?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(35042699013)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 09:25:44.2216
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9bea7074-8234-45b7-586e-08dc70d32b15
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF000001AE.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB10810

DQoNCj4gT24gMTAgTWF5IDIwMjQsIGF0IDEwOjE3LCBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnpl
bEBhbWQuY29tPiB3cm90ZToNCj4gDQo+IEhpIEx1Y2EsDQo+IA0KPiBPbiAyMy8wNC8yMDI0IDEw
OjI1LCBMdWNhIEZhbmNlbGx1IHdyb3RlOg0KPj4gDQo+PiANCj4+IFRoaXMgY29tbWl0IGltcGxl
bWVudHMgdGhlIGxvZ2ljIHRvIGhhdmUgdGhlIHN0YXRpYyBzaGFyZWQgbWVtb3J5IGJhbmtzDQo+
PiBmcm9tIHRoZSBYZW4gaGVhcCBpbnN0ZWFkIG9mIGhhdmluZyB0aGUgaG9zdCBwaHlzaWNhbCBh
ZGRyZXNzIHBhc3NlZCBmcm9tDQo+PiB0aGUgdXNlci4NCj4+IA0KPj4gV2hlbiB0aGUgaG9zdCBw
aHlzaWNhbCBhZGRyZXNzIGlzIG5vdCBzdXBwbGllZCwgdGhlIHBoeXNpY2FsIG1lbW9yeSBpcw0K
Pj4gdGFrZW4gZnJvbSB0aGUgWGVuIGhlYXAgdXNpbmcgYWxsb2NhdGVfZG9taGVhcF9tZW1vcnks
IHRoZSBhbGxvY2F0aW9uDQo+PiBuZWVkcyB0byBvY2N1ciBhdCB0aGUgZmlyc3QgaGFuZGxlZCBE
VCBub2RlIGFuZCB0aGUgYWxsb2NhdGVkIGJhbmtzDQo+PiBuZWVkIHRvIGJlIHNhdmVkIHNvbWV3
aGVyZSwgc28gaW50cm9kdWNlIHRoZSAnc2htX2hlYXBfYmFua3MnIHN0YXRpYw0KPj4gZ2xvYmFs
IHZhcmlhYmxlIG9mIHR5cGUgJ3N0cnVjdCBtZW1pbmZvJyB0aGF0IHdpbGwgaG9sZCB0aGUgYmFu
a3MNCj4+IGFsbG9jYXRlZCBmcm9tIHRoZSBoZWFwLCBpdHMgZmllbGQgLnNobWVtX2V4dHJhIHdp
bGwgYmUgdXNlZCB0byBwb2ludA0KPj4gdG8gdGhlIGJvb3RpbmZvIHNoYXJlZCBtZW1vcnkgYmFu
a3MgLnNobWVtX2V4dHJhIHNwYWNlLCBzbyB0aGF0IHRoZXJlDQo+PiBpcyBub3QgZnVydGhlciBh
bGxvY2F0aW9uIG9mIG1lbW9yeSBhbmQgZXZlcnkgYmFuayBpbiBzaG1faGVhcF9iYW5rcw0KPj4g
Y2FuIGJlIHNhZmVseSBpZGVudGlmaWVkIGJ5IHRoZSBzaG1faWQgdG8gcmVjb25zdHJ1Y3QgaXRz
IHRyYWNlYWJpbGl0eQ0KPj4gYW5kIGlmIGl0IHdhcyBhbGxvY2F0ZWQgb3Igbm90Lg0KPj4gDQo+
PiBBIHNlYXJjaCBpbnRvICdzaG1faGVhcF9iYW5rcycgd2lsbCByZXZlYWwgaWYgdGhlIGJhbmtz
IHdlcmUgYWxsb2NhdGVkDQo+PiBvciBub3QsIGluIGNhc2UgdGhlIGhvc3QgYWRkcmVzcyBpcyBu
b3QgcGFzc2VkLCBhbmQgdGhlIGNhbGxiYWNrIGdpdmVuDQo+PiB0byBhbGxvY2F0ZV9kb21oZWFw
X21lbW9yeSB3aWxsIHN0b3JlIHRoZSBiYW5rcyBpbiB0aGUgc3RydWN0dXJlIGFuZA0KPj4gbWFw
IHRoZW0gdG8gdGhlIGN1cnJlbnQgZG9tYWluLCB0byBkbyB0aGF0LCBzb21lIGNoYW5nZXMgdG8N
Cj4+IGFjcXVpcmVfc2hhcmVkX21lbW9yeV9iYW5rIGFyZSBtYWRlIHRvIGxldCBpdCBkaWZmZXJl
bnRpYXRlIGlmIHRoZSBiYW5rDQo+PiBpcyBmcm9tIHRoZSBoZWFwIGFuZCBpZiBpdCBpcywgdGhl
biBhc3NpZ25fcGFnZXMgaXMgY2FsbGVkIGZvciBldmVyeQ0KPj4gYmFuay4NCj4+IA0KPj4gV2hl
biB0aGUgYmFuayBpcyBhbHJlYWR5IGFsbG9jYXRlZCwgZm9yIGV2ZXJ5IGJhbmsgYWxsb2NhdGVk
IHdpdGggdGhlDQo+PiBjb3JyZXNwb25kaW5nIHNobV9pZCwgaGFuZGxlX3NoYXJlZF9tZW1fYmFu
ayBpcyBjYWxsZWQgYW5kIHRoZSBtYXBwaW5nDQo+PiBhcmUgZG9uZS4NCj4+IA0KPj4gU2lnbmVk
LW9mZi1ieTogTHVjYSBGYW5jZWxsdSA8bHVjYS5mYW5jZWxsdUBhcm0uY29tPg0KPiANCj4gSSB0
ZXN0ZWQgdGhpcyBwYXRjaCBhbmQgaXQgcmVzdWx0ZWQgaW4gYXNzZXJ0aW9uOg0KPiBBc3NlcnRp
b24gJ3MgPD0gZScgZmFpbGVkIGF0IGNvbW1vbi9yYW5nZXNldC5jOjE4OQ0KPiANCj4gSSBjaGVj
a2VkIGFuZCBpbiBmaW5kX3VuYWxsb2NhdGVkX21lbW9yeSgpLCBnaXZlbiB0aGF0IHN0YXJ0IGlz
IH4wVUwgKGhvc3QgYWRkcmVzcyBub3QgcHJvdmlkZWQpLA0KPiBzdGFydCArIHNpemUgd291bGQg
b3ZlcmZsb3cuIERpZCB5b3UgdGVzdCB0aGlzIHBhdGNoPw0KDQpIaSBNaWNoYWwsDQoNCk1tbSBJ
4oCZbSB0ZXN0aW5nIHdpdGggYSBkb20wbGVzcyBzZXR1cCwgd2l0aG91dCBkb20wLCBJIHRoaW5r
IEkgbWlzc2VkIHRoYXQgcGFydCBiZWNhdXNlIG15IGd1ZXN0cyBkb2VzbuKAmXQgaGF2ZQ0KdGhl
IGh5cGVydmlzb3Igbm9kZSAoZW5oYW5jZWQpLCBzb3JyeSBhYm91dCB0aGF0LCBJ4oCZbGwgdGVz
dCB1c2luZyB5b3VyIHNldHVwLCBjYW4geW91IGNvbmZpcm0gd2hhdCBhcmUgeW91IHVzaW5nPw0K
DQoNCj4gDQo+IH5NaWNoYWwNCg0K


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:32:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:32:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719629.1122459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5McW-0003Rk-4H; Fri, 10 May 2024 09:32:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719629.1122459; Fri, 10 May 2024 09:32:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5McW-0003Rd-1R; Fri, 10 May 2024 09:32:52 +0000
Received: by outflank-mailman (input) for mailman id 719629;
 Fri, 10 May 2024 09:32:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+rhO=MN=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s5McV-0003RX-82
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:32:51 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4202ca60-0eb0-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:32:49 +0200 (CEST)
Received: from SJ0PR05CA0123.namprd05.prod.outlook.com (2603:10b6:a03:33d::8)
 by CY5PR12MB6227.namprd12.prod.outlook.com (2603:10b6:930:21::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.49; Fri, 10 May
 2024 09:32:43 +0000
Received: from SJ1PEPF00001CE6.namprd03.prod.outlook.com
 (2603:10b6:a03:33d:cafe::40) by SJ0PR05CA0123.outlook.office365.com
 (2603:10b6:a03:33d::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.43 via Frontend
 Transport; Fri, 10 May 2024 09:32:43 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ1PEPF00001CE6.mail.protection.outlook.com (10.167.242.22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Fri, 10 May 2024 09:32:42 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 04:32:42 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 04:32:41 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4202ca60-0eb0-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XD3ZOpXdOtqvfHbVGCn/e94EmWIgNx76MxFtbgv7HNDvCuQwzid46w5PgOb1f0sKiST9JxFZBiaRmWe0HeO8bLsx49EdN+/OqPjz8lx1nf1qglE6cNXFtGotzt0vJ6r5uQ1/220U433ycOKlADvlH8eZRhVwOqfiHN7UhWorkd0jLw7TVJjpLT8/5PZ9710ZFm2ySwe8z40ciCPazhgv03BjtaoFppOZQYggoyPe6GTMTLf8SNYrXQNFWNHNqTXFTd/6uuiE2QlN4yWFfC2hu1XuHrkzfwDkx5GAabCOgoEIU1uuoT1wLZ0c35eOnwI+tjTEsLMyQ8M57WFpNNchIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=y7G/nZPMHErP/m19JHaKGEtBUlctLBSZKr2vW07IE5A=;
 b=Q6GuujIQX0MFPoPfAFs+DY5hGb8yLrfrZAQSA3CdVnWkROVIM6QxSNQwumN5vC4upC4ffDiNmPR9yT7Q9/J+66y9cHO+D1QgSZwPRC3PRfUpePsVDyd+ito70mKrNe8p8UaFhP8DeFE3eSMEMTPIsuaodb3J9SyHYQLX5V7tkOH+H7Shr52wPJpX2wrlKmzlDWxf7gnSPwPLkYTMGHyzU0MHxTG4ULD5JCbfnsiux4FAtbDto1n+De9ICYXT17yDaNVIuasWgN1dxTb3RhNPMHPcmf/sCFe0uc7icMJncPGQj4MYJeBVZhyIkZxjqaB75+/wM1QCIp5oesXli8VNww==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=y7G/nZPMHErP/m19JHaKGEtBUlctLBSZKr2vW07IE5A=;
 b=5GYGbcHMlc3ZYiLDoDg8oq+hXdxk87P6I2gOyAs50WD2rTjxD2U8dH+fiq34lRi2ms+cF0HVy/D5qE/b4DmnvwOsLL0mi4wBio9ZYk/nvk632Y5bV5sR1FDILnr3A/AmfzMJKJ7xICWE9GzAuJSzt0gBXPCtQjNStqrI11vn2us=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e025553b-4b3f-4591-b996-2e31eb081bba@amd.com>
Date: Fri, 10 May 2024 11:32:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/7] xen/arm: Implement the logic for static shared memory
 from Xen heap
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-7-luca.fancellu@arm.com>
 <cb496741-dd76-4536-818a-cbfdbd496b0d@amd.com>
 <10212FF3-3CCD-4634-9890-4CDCA11A5F88@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <10212FF3-3CCD-4634-9890-4CDCA11A5F88@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE6:EE_|CY5PR12MB6227:EE_
X-MS-Office365-Filtering-Correlation-Id: 1ea9ea1a-4f40-4b64-f882-08dc70d424b4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QWVUT2pqQlZxNldZNFY2S2RYR1dWY0VRQ2dnWGM3STV4OWY1Mk5HY3hiMzBE?=
 =?utf-8?B?L1FYVjd1NUcyZDBMUDBKSjF2aFh4Q1NWZE94Mk02aldSQmNpR3lrdW5icFhD?=
 =?utf-8?B?QkxSWXNlSS9LU3IzT09ldFlyWDhoUTF5Q1EzTTk1NVBlOVNVSno0aDNYZ1lY?=
 =?utf-8?B?emRJY2REd0ZKbXRsR3dJTnpZUDlxSEFlazBCT2t4SlZ4dEk4QkJrYUloYjF2?=
 =?utf-8?B?MkQ0dTNBdmc3Szc1TkhGcjJFb2trUjczOTROb1Noby9IOHpNTEtrNDZobURw?=
 =?utf-8?B?QWd0OStWNVY5VXFTdjJQaGJZenZxK2E0blk4TFY2ZlZ2dlpPL1ZscDh5Um52?=
 =?utf-8?B?QjExeWg4OEhMa3RpekJ2azRtdDZkWmtaMU9ZTjFWZ0RLUHdtaXB1U21sRnQr?=
 =?utf-8?B?THpQaUZKZHFzRWtJL0E1R1FTWldCM1ZGRUdtZVJHdFZUamFPOFFFM1hTMm1I?=
 =?utf-8?B?YlVBWVpEdGFTT2FiTnRmMTRONXJDd0NFbktZUk9nODRpTkhHODhZajJGQUls?=
 =?utf-8?B?MWVaUmJnUytrL3R1bVpGZ3VJY0NrMy9ISXNqN2l5NmNUZ3N5MFlBT3dXTmZT?=
 =?utf-8?B?NmhxMkN1WFlFc1hROGRGcjNvQ0NzR2h2dWkwcWZvWGNBVkNRVkVSMTNMNTd6?=
 =?utf-8?B?Z3BQNDRFM0Jld0dqRkNKZ0dNQSt2ODJiaTJHcU9uZmNoU2hMV0tLd3ExRlB2?=
 =?utf-8?B?QmFqMVZYdUljVFJtM3p5R1ZDTENMRFNNSVVnc3NBTGZSekZoaUFKSFZ2S1NM?=
 =?utf-8?B?WC9iWWxtKzN2bUhLK2tkb3lWNERpVmd2M2hMNEp2SHRUUXJBUjRmS0l1c3hJ?=
 =?utf-8?B?bzV3Y0cvQlB0S0dqdjAyU2dXMHc4VzI5QlA1d0FMUWtSd1BpelVSWXpqbm5S?=
 =?utf-8?B?Y2FVbFUyUk1GTklMSTlXTEY4ZHFXREFvZEYzbXFuQnVsK3dsaEY4d2N1RWU1?=
 =?utf-8?B?dWRWMTBES1dOTUJUODM5KzNjeWx0MWRkTlZjc05TUmo4Q1FVMXJhWFZvc3Rs?=
 =?utf-8?B?VjZndWcyZktMWGQ3Nk9UVXE1VGQ2eExVQjhaMldLSStLMDhiUXJHbHY3ZTdM?=
 =?utf-8?B?L3hzSGdWYXhXa1psb1I4dWJMS1FTZHIrNWpOcWVaV3ZtMmxkSGphLzcvNzRa?=
 =?utf-8?B?SnhZVHVjMGh4NHBkaGdINmpIeW51eEpBeHJXZXRjelpUQ3liSGQrbDROdGlZ?=
 =?utf-8?B?ejNpaGxZb3Z1UDVFM08wTDdCczJiWFpwN25uZWxldmF5WFZqUTRkbjhEU0cw?=
 =?utf-8?B?NlF1bFVucVRaWDVsblB4OVZqcDFOMXRnaEY1bFZheWJkZUtRbG84cmlBNTVJ?=
 =?utf-8?B?NVVwQW1leWRDQzNaUFpDVnQ3emVNVjk4MUwvVllIL3ZZcDdQOEk1ekJjRzRy?=
 =?utf-8?B?U2l0TlFUbDg4cU9JMERxNlBCL3VUeHBKQ1JyU0FWSGMrRmM1bTB5R0xkNHo5?=
 =?utf-8?B?TFpqbHJjditjNU9wVUsxTEVVS0k5cnZhcWkvSytKUys5ZFBKZU9vdDI0OHYy?=
 =?utf-8?B?bWd5U3NLOVFYZXQrdzJ5SC9kQ3kza25pa00vTnF5bkxCZlFUN0RVWUpPSkpj?=
 =?utf-8?B?M3hzaE4vRGVWUHdTelJWd1RDb0JDZVMzQzdTUGtNU3BYUTFaeTVQemsrS012?=
 =?utf-8?B?WDJkbWhISUtVL3lMYmpJMlBockY1eU5SVlpEdEsrMThpVEFQeFlCR2xnNU5n?=
 =?utf-8?B?S2M2d3NUQXRxRTdxaFRMYzJyK0VIYlZweTFhNEdlVUZnWkxhVUVwSWt5OHFL?=
 =?utf-8?B?NmpndjVtYS9ZZWl4aHBHWGV3cjJSV2d3VzFEOTZxdzRiYWZ3YUczcFpGS2t3?=
 =?utf-8?B?azRzRTIvd0pnRnQrM3NoRSs0R0FKQ2JscTJXSVYxd3dFQTMyYkxWTmZ2MnVD?=
 =?utf-8?Q?7xt6UujoIBHpP?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 09:32:42.9735
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ea9ea1a-4f40-4b64-f882-08dc70d424b4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE6.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6227

Hi Luca,

On 10/05/2024 11:25, Luca Fancellu wrote:
> 
> 
>> On 10 May 2024, at 10:17, Michal Orzel <michal.orzel@amd.com> wrote:
>>
>> Hi Luca,
>>
>> On 23/04/2024 10:25, Luca Fancellu wrote:
>>>
>>>
>>> This commit implements the logic to have the static shared memory banks
>>> from the Xen heap instead of having the host physical address passed from
>>> the user.
>>>
>>> When the host physical address is not supplied, the physical memory is
>>> taken from the Xen heap using allocate_domheap_memory, the allocation
>>> needs to occur at the first handled DT node and the allocated banks
>>> need to be saved somewhere, so introduce the 'shm_heap_banks' static
>>> global variable of type 'struct meminfo' that will hold the banks
>>> allocated from the heap, its field .shmem_extra will be used to point
>>> to the bootinfo shared memory banks .shmem_extra space, so that there
>>> is not further allocation of memory and every bank in shm_heap_banks
>>> can be safely identified by the shm_id to reconstruct its traceability
>>> and if it was allocated or not.
>>>
>>> A search into 'shm_heap_banks' will reveal if the banks were allocated
>>> or not, in case the host address is not passed, and the callback given
>>> to allocate_domheap_memory will store the banks in the structure and
>>> map them to the current domain, to do that, some changes to
>>> acquire_shared_memory_bank are made to let it differentiate if the bank
>>> is from the heap and if it is, then assign_pages is called for every
>>> bank.
>>>
>>> When the bank is already allocated, for every bank allocated with the
>>> corresponding shm_id, handle_shared_mem_bank is called and the mapping
>>> are done.
>>>
>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>
>> I tested this patch and it resulted in assertion:
>> Assertion 's <= e' failed at common/rangeset.c:189
>>
>> I checked and in find_unallocated_memory(), given that start is ~0UL (host address not provided),
>> start + size would overflow. Did you test this patch?
> 
> Hi Michal,
> 
> Mmm I’m testing with a dom0less setup, without dom0, I think I missed that part because my guests doesn’t have
> the hypervisor node (enhanced), sorry about that, I’ll test using your setup, can you confirm what are you using?
I have these Qemu tests (with and without SMMU):
 - shmem between domU1 and domU2 with/without host address provided (owner domIO or domU1)
 - shmem between dom0 and domU1 with/without host address provided (owner domIO or dom0)

BTW. What was the conclusion about the issue if shmem region clashes with RAM allocated for 1:1 domU e.g. dom0.
Accidentally, I end up with a configuration where Xen allocated for dom0 a region of RAM clashing with my shmem region.

~Michal


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:33:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:33:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719633.1122468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MdM-00040X-Bi; Fri, 10 May 2024 09:33:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719633.1122468; Fri, 10 May 2024 09:33:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MdM-00040Q-94; Fri, 10 May 2024 09:33:44 +0000
Received: by outflank-mailman (input) for mailman id 719633;
 Fri, 10 May 2024 09:33:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+rhO=MN=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s5MdK-0003RX-HQ
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:33:42 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 62c9e70e-0eb0-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:33:41 +0200 (CEST)
Received: from CH2PR07CA0042.namprd07.prod.outlook.com (2603:10b6:610:5b::16)
 by CH3PR12MB7740.namprd12.prod.outlook.com (2603:10b6:610:145::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:33:37 +0000
Received: from CH1PEPF0000AD7A.namprd04.prod.outlook.com
 (2603:10b6:610:5b:cafe::35) by CH2PR07CA0042.outlook.office365.com
 (2603:10b6:610:5b::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34 via Frontend
 Transport; Fri, 10 May 2024 09:33:37 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000AD7A.mail.protection.outlook.com (10.167.244.59) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7452.22 via Frontend Transport; Fri, 10 May 2024 09:33:37 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 04:33:37 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 04:33:36 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62c9e70e-0eb0-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iISezZRuFnH9yMgST7aFWYfhQIl/9DX36HZLNO5CWHDf5KThLMyt04hiYJsG5HOfV2/z3eGzMtC7dSZwddw/9HHz8AvZ5ss+jgzXHeEHFHmF7uWRH+/U3ypA0Jt39C973mm6c5vqbkj95WmQwhg3hsX7FV34gPJx2xNgFSi4pYnrFACVIuiXCzQFzvcbustUoW46J9NdMfmTtX2toOs4Uu25hM3GbzUPoC4DWXQ7+DVZBva1O+NmBZBl3Xv05UcuOgZ8fFWRdd+kW00OlxPcEU6wSvYfFwtqr2mmBjhVJsF1e/wF9125IuxN8nF0FF42t+kDHRDqQ+V/XYZGRf6tqg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WKYMYY4d6k3hmrRfYGm44xNye4xJQrwmgggn56WTo2I=;
 b=ELha2lsBz/O58JLtNpXKraN/rUDulA4y787kvKfX9m1sTg4cc8G8u1GRT0hCU5OrH9UshdLC7EomJYpwqYTFYhxHMWMqsFfZ/U2gDXRvkyuRhm8mIjictquTgvQUK+q2lO0h608QSe8MwqM5ZqYWxfLLsB57Mc1yMg4mH1sUn9/AGw77rAykgZIckiDeb0z6EYaQ97IHqmi3JOa/gzBKydDBuOLhFCpYRtVUFBQkNY1MSZPsrcDtu0i8Ei1NpNQxiqFzVxUzs/WePmr3my8T1bDpWMaK4XBmqsQ2u8u/NGIiRoQuAsNya/skVZtP1MXNrdRkTkrOnpJrbsXoqmoVAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WKYMYY4d6k3hmrRfYGm44xNye4xJQrwmgggn56WTo2I=;
 b=wsGcHW9ePn7awRXKIzi9evm9wTOscJzsjJCXLCy9MfUTFrc1josZV7/ZjQ12hsMwqXaZV8YKKERdpV1A4L+ugZdivabAqeHymbgPdiWc6TU1lAtM+2CKaNQg38W3gnjmeeOQfoKH18WmgGbQosny6hcXFT6clH4vmSIV+bT12xg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <ff63a9e3-2816-4312-8e95-7892ecffedd6@amd.com>
Date: Fri, 10 May 2024 11:33:35 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 7/7] xen/docs: Describe static shared memory when host
 address is not provided
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Penny Zheng <Penny.Zheng@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-8-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240423082532.776623-8-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7A:EE_|CH3PR12MB7740:EE_
X-MS-Office365-Filtering-Correlation-Id: 522f5f2f-e954-41ee-1a08-08dc70d44552
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?WVAwd25YQkJiWENEdEJxNXJzUEpJMktjaEd6OStvaEVoTWV2TGlzL3lxUVc3?=
 =?utf-8?B?bUlZL0YvdHp3cWxJZHJyRjVNWXhvdkZGSWh0Z0MrNHl1VWdpVXFWMEJPL2dG?=
 =?utf-8?B?cUc2aTQ0dmF4cHAzVk5HMWdBVWwybnZuUDVFRDh3VjJqME92blF3dHo3OWFt?=
 =?utf-8?B?azBUK3JLajN5bStjN2pZZXVtODVUYStkY2NqWjBMMFlDY2JNVzNTNjc0bU8x?=
 =?utf-8?B?N3FaWHJENXhxVlFrcVByZVl0Y2xrNEFSaUwrYWJuN2YwODhLOSt2K05CWkw3?=
 =?utf-8?B?U3NyT2xwUXNHOHJvaVQ0bXhZTDZqY2I0N3E1NCtFdEZHOGgrT1ZDNG9IdXpy?=
 =?utf-8?B?WDlFeVkxTEl2VlBPY3J2eE5Nd29GeDFVbWllRFhwR0drM1NUTUhCNXluS05X?=
 =?utf-8?B?cnJ6M05BbmJZOWxyTHN4MHRwNmVCZWRWQlJQcHJmbTYrNkZGRFZkVXErT1pJ?=
 =?utf-8?B?MHAwSlVtTlBYRERMTndKMFcyVnJENXlsMVMzQXZQREdaRGlFZitjU2taWGdn?=
 =?utf-8?B?V3BOaEJUM3FJVUlvR0ZaYWE2MFBIZnNjVUN0YzdTbmdYVE1jajJvVzFUY2o4?=
 =?utf-8?B?R2hzVXRuTEtvOTk1VjVHRXgwMWNCT0VpRVJPeXEyRGgrSWhKL2VoTm9EMHVV?=
 =?utf-8?B?RUxqQjRnaWhtOWdNUkpCcEtrdytuVlIreVZheWxwQnJ5SkMyQ0NiS3NJK2hi?=
 =?utf-8?B?VlZ2SVRpdlRjZXNiWGZlSnVtSWI2b3A0T3N6TCtJN3kzNkpOb0g1YzVlMHZT?=
 =?utf-8?B?YXVpR2xHcWJ3TWdPQjFBK0JZTHM4RHg4VFhpKzh5d01NVitMeS9iNEx4cjVt?=
 =?utf-8?B?cmRYaG5id2dVNS81c2pHNEhTbkJDZk12c3ljWW14U3R0ZXQrQlBPT1ZkM2NE?=
 =?utf-8?B?Qnl2QlRDZ0tma09BT2c3NTVtSkNYdzlhczdraVdZdnpHSUZuYUFtK0tPOElN?=
 =?utf-8?B?djNmTWhIdGxha2RiQWxCRTJNVnBHOFZqZ3N2ak1jbkFFTGd0dEo3dENVbENY?=
 =?utf-8?B?cFFvUjN0WTc5VWVzVWIwa2pnRmp1eDZVSTFoTWw4Y1plTjkrQ1RKQ1FudDFX?=
 =?utf-8?B?Qys3SnprZHI1Mm9vNlB4WjFjcUFlUVpqV1pTNEZhWlpGNElKR2E5ZFFDSEx3?=
 =?utf-8?B?MW9oVVZXREh5Y2pKQ0o1TDM5Ly8vT3BQdmQxclpMUFhkODl3RTYrellWNzdN?=
 =?utf-8?B?c2wvNS9WajNuQUR6Vm1JdDRCeWJSV2lJcTRzM3Q1ZFNTUnYxNFI2aWttdEZZ?=
 =?utf-8?B?UEdWakJyTzZJUkV3Q01OcG5nNmNyNUt1Yzlad2hzcC81a1cxSHhQaEx6enpu?=
 =?utf-8?B?bko2dmx0TlF4YU8yc3V5R1MwcGJMSXl2UXlJQ000dUIyTW5DZnJieitEbktQ?=
 =?utf-8?B?cmRDSXAyeGZ0MjRWZVJ4MTMyL2JzT1czc1MxWlZrcGZSWmo5V2JucE9SL3cr?=
 =?utf-8?B?TjBvN2ZocDRmNzY3eGQ2ekFUV2F1eGJVbjFuQXNsZ3p5aUZCSXdQUjMzR25Z?=
 =?utf-8?B?ejZOTGFJMU1YcDBEVUpjbC9FWW5sS3h5eUtNNExSR01Mdk9wVk1sLzZ6RDF4?=
 =?utf-8?B?aWRVZ1Vkck1veHU3VEFJeWRsV3dLSFJncS9ORFJYclRDU2FpeUozelFEYlhX?=
 =?utf-8?B?MDdCc3B1VEpaMDEvQ01vVDhsNGNTZG5ZcDVQclhzVzFETFRjSXBqeVp4bXFG?=
 =?utf-8?B?QVhQaWFrOUJJTmJLemZvbFlrdmNQTk5Eakh4eXRyQTVKVVRSYnI0Q1pEY0Mx?=
 =?utf-8?B?b0I5TGlUbElmNHZ1Vm5PekhOckVPVENPc3pIVU9DWTc0N0ZGWm5tTGlkVVRW?=
 =?utf-8?B?YXJSY09oRXZtV1VnQ2xkRzZWOHpIWEhKWW9OUjFqelNUVDZybE83aXFxM2JW?=
 =?utf-8?Q?5iz2x0V5W2Dmc?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 09:33:37.7528
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 522f5f2f-e954-41ee-1a08-08dc70d44552
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD7A.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7740

Hi Luca,

On 23/04/2024 10:25, Luca Fancellu wrote:
> 
> 
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> This commit describe the new scenario where host address is not provided
> in "xen,shared-mem" property and a new example is added to the page to
> explain in details.
> 
> Take the occasion to fix some typos in the page.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Fri May 10 09:37:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:37:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719640.1122479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MhE-00056m-Sk; Fri, 10 May 2024 09:37:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719640.1122479; Fri, 10 May 2024 09:37:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5MhE-00056f-PM; Fri, 10 May 2024 09:37:44 +0000
Received: by outflank-mailman (input) for mailman id 719640;
 Fri, 10 May 2024 09:37:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jEWd=MN=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s5MhD-00056Z-Ed
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:37:43 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2607::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f214db81-0eb0-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 11:37:41 +0200 (CEST)
Received: from AS9PR05CA0323.eurprd05.prod.outlook.com (2603:10a6:20b:491::13)
 by AS8PR08MB10362.eurprd08.prod.outlook.com (2603:10a6:20b:56c::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:37:36 +0000
Received: from AM1PEPF000252DF.eurprd07.prod.outlook.com
 (2603:10a6:20b:491:cafe::7d) by AS9PR05CA0323.outlook.office365.com
 (2603:10a6:20b:491::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47 via Frontend
 Transport; Fri, 10 May 2024 09:37:36 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM1PEPF000252DF.mail.protection.outlook.com (10.167.16.57) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7544.18
 via Frontend Transport; Fri, 10 May 2024 09:37:35 +0000
Received: ("Tessian outbound e46bb127ed3d:v315");
 Fri, 10 May 2024 09:37:35 +0000
Received: from bf7312ff9561.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 71FCD132-B67C-4DF5-B031-02AF3BCFCDAC.1; 
 Fri, 10 May 2024 09:37:25 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bf7312ff9561.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 May 2024 09:37:25 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS2PR08MB8805.eurprd08.prod.outlook.com (2603:10a6:20b:5f5::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47; Fri, 10 May
 2024 09:37:22 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.048; Fri, 10 May 2024
 09:37:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f214db81-0eb0-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=Kit/bHzcEwllSfMx0RgNvhrQhguidy3OgLBVNJLKTlZzQV1S46lEBlIPMQ6UlePciX8tQ7XHRaWuFziVJfTOgw832GRRPNR/sWpz4jjRF1MBiWY4Ph5ZsIPWMNSQKEqW0WU5qMIUqk/95E5ekzdSRGza1c8hgdkOx2/CadO4L8ve0p6AkUX2Kt5/lEYGbTolInqCvi4EFmvDXAC9TaMoKqX8ZzdX813Kce7V+1zjhOOHiwxSxXOZniLmQQzQbNVCqFGSxYZFzZJTa/YnKfiDEtdU1upxpkNoHoSzN8VpmtPyULBD5q31g44nTlFZqwdGhVKzXMXYZsByuUz/lYi49Q==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Lge1AzD5rhGXz01NwK80rM038779ps/C04JAL6lzDtc=;
 b=dj/Ltiesp5JN6UneCGYMlqFE5er2V+WMTdJzglncUua+Wg2JidPveLchop+gKbPqxnoaHl4/39+AoCqQWyHxe0YuXVXkVuYAxHg9bZeH4RbW37sMLXrQ9OdD9uL10dT7oCA9dT45IpxW670Q156QkM8Q6Nt2T8vFJXFB5It0Lkxy2vRu2l8EKbADi5DmasPiAjXYmxlSBEGf5D3rnU/MLmVX6TbLwBMFTm3ykqceFH4BPjSxjso0WJ14NuvmeVhA8CzZn5DIUVtGfHFDmYJKbqDRtetLlDwg2bAi4ReXeJ0EwHYNU0d6QA4q1pUetGQdzcP6fEl3xgPef95sF2OJ2w==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lge1AzD5rhGXz01NwK80rM038779ps/C04JAL6lzDtc=;
 b=CTjkJ+tQMjCV8DcozNTMdljDybZYkujn0UrZzN/RNgxCLWnLsSnMebURwAZDu2rhnvL+FoCJ6VDCPMYxeLVwuY29FrSDpw5S5VlHc/yZsw8FS4lgyoB8xQUEEbl3yjAyIdDUUGhlRu2elPhrGj8FERhwnyxU003erbq7h9eUY+E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2c6a67df5bb7d963
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gh5KAypQsCbuxvqKESV45TZmNuvvvSux8bZfNFr1Y3v+4jk/eDFGQ5FKEbwPsY1PJwL2IXcYdMuL3TapAUy7mKNq2BTFvtSVr60wIhOiUR3HcT2pOiRHiZfdKnsyW09Q/jsq4TAC7vhafGCr+qND7qRkKHR8ai9JAOHX4OMOHBJprYL+UfmcjCg4e3RvnUBQN80hDmmdeBqGW1YiJsI5Oqm1qj+W0JyZS0RIhJJhy5BI5o5weNCAwmyjHlBwyuPLnKeWzVwew7p35/BlgT2H9PtaKFLXsNaNY+kF4v9jlq8Nf+ztpbFOQNPC7zQfj3GlzVDP+iwtXvTajdGHmIXJtw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Lge1AzD5rhGXz01NwK80rM038779ps/C04JAL6lzDtc=;
 b=koSIgZjFWGDV6hQ74V+jyQ+wqOPJkW7shC27fHj+o8R/RA1onSYqGJgG5KYWMYZgo4aqbbteeJ1jVzIiBcC+ZB+udzVMCgKGe6IDo4fL6+qvxddlMZR5VXvvcrYYa6PgHWPK13Pt4srb3eNzqrpiXwIwgQ/yhoSZeZimb7h6NDbiRS3Bdc4Gl71RDZpNm4ssQONav9EiydWkqs8qh8gjQpC75W5gmO+EGCUlldohndnI40bCKJAtCxqEdNvfVRcjPQnW1LP1v+cb8+SLq7xErQP75OtQkuhLDeGyLxzcWHmEqXphOY8EAl/nh9YtVI92VoByz6R72dOQ3oOV6bT/1g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lge1AzD5rhGXz01NwK80rM038779ps/C04JAL6lzDtc=;
 b=CTjkJ+tQMjCV8DcozNTMdljDybZYkujn0UrZzN/RNgxCLWnLsSnMebURwAZDu2rhnvL+FoCJ6VDCPMYxeLVwuY29FrSDpw5S5VlHc/yZsw8FS4lgyoB8xQUEEbl3yjAyIdDUUGhlRu2elPhrGj8FERhwnyxU003erbq7h9eUY+E=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 6/7] xen/arm: Implement the logic for static shared memory
 from Xen heap
Thread-Topic: [PATCH 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Index: AQHalVfty94kOt4AK0ecWyByn7T1+rGQSwuAgAACP4CAAAIGAIAAAUOA
Date: Fri, 10 May 2024 09:37:22 +0000
Message-ID: <210C1AE6-013C-47FE-9543-689A89A4FF8C@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-7-luca.fancellu@arm.com>
 <cb496741-dd76-4536-818a-cbfdbd496b0d@amd.com>
 <10212FF3-3CCD-4634-9890-4CDCA11A5F88@arm.com>
 <e025553b-4b3f-4591-b996-2e31eb081bba@amd.com>
In-Reply-To: <e025553b-4b3f-4591-b996-2e31eb081bba@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS2PR08MB8805:EE_|AM1PEPF000252DF:EE_|AS8PR08MB10362:EE_
X-MS-Office365-Filtering-Correlation-Id: 2b8548b2-8402-4deb-aea7-08dc70d4d349
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?TzI5OXF0ZFd3K0RMT0MvWTNjSHVjc1NFZ1VxU09yN3VKancrUWV4dlBsbzRp?=
 =?utf-8?B?a1dZYmhjK0tweEo4UGtUNU94QjZ5ZG1pcFZvRXNDTWY4UlJreFFRUmMwN0Zj?=
 =?utf-8?B?aVFXdlJpaEQ5YStadXlLNXd2NnBUTENkUTZ4S2QxdWtHKzhvNW5vQUxveWdp?=
 =?utf-8?B?NzZhZHRZZUZJK3JzR2ZTbkhIZ2FWN1F4ZzQ5M0gzS3d4Skk5em9CY2g2YVFt?=
 =?utf-8?B?N21Ic2tNZGwzV1RMYk5WK1NxRndRRHlYdG9ldDgzYm9IOE5CYnlUWlllaWhh?=
 =?utf-8?B?SUhQTUR5dW1xcldtaGs4eWlLYTZPTVY2ajFvK0JtRk92MUQ0T1dsd282NUtC?=
 =?utf-8?B?Qkh4Q1R4VGs0VHVWVU1lamZxZ0p6NzNheHV3bzMvKzhkR1JVZnFJdGFHQzc1?=
 =?utf-8?B?eEVSalFYOXNSaGthZS9TMFlQTnFaQ2NOYUV6NGxnVGt4b1Z1Y1hQTEM5UXZV?=
 =?utf-8?B?U2NxalpHQWk1ekZrTkJCTHZPbUVWL21DSWtOaVNWckRSOVFwSTk5OCtYaTB5?=
 =?utf-8?B?MmVSZStFZ25GeFpTNTF3WUJiTndubHZTcUZtOG1mcWhkR1hlQWpxTWhrNDc5?=
 =?utf-8?B?QlE3ejZqdzVRWU1jdzVNaG5ESUFNV3JxTGp4OCtRSEEyMnR3b1lsZElrM2xu?=
 =?utf-8?B?SlFDYUVFSVZjSmszcDdCYTRONkZVTkZMaTlFQjBmSWNzV09FVTFHV0NBTloy?=
 =?utf-8?B?aXd3NXFYMUFCdEhyRDFtTE01MW4rTWFQOTk1R2t5cXlCcjgrbXQ0MXNkbCsv?=
 =?utf-8?B?cFBIdmVKOHQyVHJ2VmRhQVEwM0dvSEZWVm1PVTdpNmdNTnU3emxhbWZseERm?=
 =?utf-8?B?NXhyL0hKQVdmRzZ4NXhINXEwSHpyRDNSZ3c0eW5PbFdpQVFUNnNndHVla29a?=
 =?utf-8?B?Y1JvVUk5Mk96eDV1YVMranVEcjlEN3ZEN215aktGTlN1VzJiUjVpRmthUElQ?=
 =?utf-8?B?bTNSY0lyTGZleVJrUGtOUU1xVndHUnp3MklpclhmNFdUMXlxa3V3NzB1Wmgr?=
 =?utf-8?B?UGREZlhXZWZMbnVnV1BiMUIzbUhySC9Sdm41b0NUanhFSEt6Ujh3K1lINTFk?=
 =?utf-8?B?eDdOcmdKN0dmSEtJSEZOM0ZmNTdvc01nckRvOERZbkg0MFB2NmRpU0ZMOXoy?=
 =?utf-8?B?MFg4eVFUeEk2Y09GR3I5cWIrZnRwa0NqTEQ2YWVpcUk3SHYzN01VUEc5YzZr?=
 =?utf-8?B?NUh5Sk1qdk1tTFdkeEMvMHJNYnlmd1NVR1NYZ2VzazdzRHZuY3JNT3NMV012?=
 =?utf-8?B?Qm5SdmtGYmVibUN3b0xTVWdQWDM5eVlDOVU5UWxVSlZBOG9MemZSN2xYdEov?=
 =?utf-8?B?MnJ0TENGLzBRenhTbkhpaHdUMitRUHh0RUFvN21pcU4rcUhiK050Vzd4bEZC?=
 =?utf-8?B?WHhVYkFPdVQ0RTlzdDNNQWduQ1VwRlJveExKQTNxRnNRdzJ0Q01OQml1UTd3?=
 =?utf-8?B?aGN4czl4aWZ3VThsZ3hCN0pYZmIrbmt5ZUpic2lnMC9NYms1blhwR3JyMmZp?=
 =?utf-8?B?UlJlckcyMklNZGczaU84d1E5VXdIaHREVlcrMUxoMkdlLzh6NHYyNlh2UjFV?=
 =?utf-8?B?RFJwZkpoTE5Ja1NBZDdBWGNCMWZ3TWo4ckNNQjNqVWRZWUdUc3NVTm5JNlUv?=
 =?utf-8?B?bVI2dTBMQzNtMk81YWVheDJQVDN3a2h0VzZ0U3ltMGRTajR0WEtGL2FsK2I5?=
 =?utf-8?B?YU9PUncyT0dhK0ZjUlZscTkzSEgzSGFRWjA1NDA2WDZrK3JYKzJPT21hNmx5?=
 =?utf-8?B?OWhBdjUvNHI1TVZtYVRsUWVsbm5ibHlFQnVkRzJ4RmJtenVGamRjMVBiZ0NZ?=
 =?utf-8?B?clBiK01ucWIxSkhCSGRXUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <8A3361E72D9C4D49AB361A833CBE5FAD@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB8805
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM1PEPF000252DF.eurprd07.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	eb315c8b-d956-436b-d04c-08dc70d4cb25
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZUkydUh4ZVFuRGExNkljb1UrWEduZFNpYTJ4Q0plQlpQZ29RMHhRSGpIaGdC?=
 =?utf-8?B?L2YyWjdUdjc4NkUrNkl4SWtNTTFRRUU2TGtWNURLcjNUWmc1VzJ0M0FzZHU1?=
 =?utf-8?B?ekZseGhCZXZvRkxvUUhiTGl1UnNPa3ZGOGk4eUcxQjJTUXRvS0MxazRnZ2VI?=
 =?utf-8?B?VUxkeGEra3ZTcjN1YTdYUUplb3g2bUZCWFA2aEJJT3lreXpKcm5tcU9VYkJ5?=
 =?utf-8?B?YnNidGttSHVCQ1kzckRZRS9JMjZWNGsxSmtpMTVOZ1Z1WlFUTncybkozci9D?=
 =?utf-8?B?YkZSSXhBWG83M1RoelBPeXhySHI3L2xzQ29ZaDlJVmlTcFIzRm5LZmJpWjZt?=
 =?utf-8?B?aFg3SHN6a3NPZ0R6Q0E3dTBJTkNjeE5VOUlVUU9UdkhvYjRzdW44aWJvY0ow?=
 =?utf-8?B?ODZOTlNZUTl5T28zZHUyS0VGazRRWElaOStXVHJGYUxiS3hyTW9heFJ4eFlX?=
 =?utf-8?B?UUs3SGJsVnRkQWc1Nk5xV1dPN25LSW1qdCtZc0tmclRXdDB2eEdNbloxUytP?=
 =?utf-8?B?TU9vRTBVR0dnNEZrR2J3RkVpZDdTQWY2Ujh1UGJhL3UybU01K2NDcGJQZXda?=
 =?utf-8?B?TzRlek5EVDFvWnB0aFovMWg3OFRjaFFjYzBROHM4U09nRjlEcTh6ZDRmWkh2?=
 =?utf-8?B?RVlOV3ZzUFJvYnpBQms5Vk53SUwrRXUwTXVQSllyenQzVFRGRW04VlBlcC9t?=
 =?utf-8?B?THdUT1RVQTlCSXZUR1FsVEU4NUVHKzVnUjVaNzhQSThlaDlCQms2eXduVDVH?=
 =?utf-8?B?RXVDTGlRQTRaaTBwMWpuWmRMOEpodE43R3pDZTQwS0Q5YTZXVlBSbTJiT3Za?=
 =?utf-8?B?ZHRBcmZLUVB6K1hOUmRCSHdMWVEvVlhrSU93UHdJbjBZbGk5NXZSV2F0YW9W?=
 =?utf-8?B?MWQ2TUJydGZoeG51QTBaNDVaOFhqbFkzKzJvTHBCWVhhcVE0UXBkaWM5dVly?=
 =?utf-8?B?VmpKWVYvM01kV0JQQWhMbmVGQ3RscDZVa1RjeVJHVW5wYktlTVlEdGZ4LytP?=
 =?utf-8?B?RmNTSVNCM1VIRDZzZi9NSVpBQjNQbjdzZGdUUmtjR0xWTzJseWc0eHR1YXZm?=
 =?utf-8?B?YWVYcExqWHp3Z0xEZGh4OGU0RXdGcDRCSzhGVmxPazhUbTVwekg4cjVHT0lv?=
 =?utf-8?B?dkxCUmJKNWdMWG00dU4xd095Z0d1VEh0MXZ2MENyL2oyaTMvV3NNR2tvY3V5?=
 =?utf-8?B?N0xPVlkvZGUzZWxpZGdRT2FIc0d2Mk00bzJuWVRkUWtuT3RZYVhlMGNteXRD?=
 =?utf-8?B?dUlkS0ZJTWI5K0JtZW5tZEhIQm9sazVhTitzWlRON3QvSCtubUpzNmhZUUdO?=
 =?utf-8?B?MS9wdm9uMENsTjNxOVd4K1hRenJSdlZQN0Vxd1BlTG9xUFBOM09ka29OZWdN?=
 =?utf-8?B?Y1F6eXRKeU0wM21NVGNva0Jjd2NZRTlFQWNqR3loT1R6cGVBQ004WkxOczE4?=
 =?utf-8?B?VWVOOHcwQkVCai9odHJrMWJORHFmeGd4bWVhcW0rYmh4d1BnV016b3ZiSUFO?=
 =?utf-8?B?Yk9LWlkybWYzTmp1ajQwWXJYZk1ON093ckk1eTlURmo4Mk5VVmZ4Ui9GcWsr?=
 =?utf-8?B?czdBK1JQb2dWekFXbnJ1Vno2ZVNtQTFNVnF1a0c3M2tldlhGckQ0SGJORkds?=
 =?utf-8?B?ZVhCK1dSU0JpcGllUjROQXpwa2ptdmlvYVljSi9odDlFS3FHNXlMbWxZelIw?=
 =?utf-8?B?d0VWSDZWOFppbzRPU1FkTWhKUE9lS1BXemhhR0N3bTFrODlPMUx0ck8zaDlw?=
 =?utf-8?B?a3NjK3NzNm53Q2lhWkdmaUtpcGRnL0RkL0NOU1RZOWxlNzFYRUNkUVJpQ2hG?=
 =?utf-8?B?VlM5ZG5HMlp0UFVaRUdoVlNLMVRQc2szcUE3RUc2NDVBL3ZxbmNESzdJVnZ4?=
 =?utf-8?Q?kNcgGk3nz5ALj?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 09:37:35.9341
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b8548b2-8402-4deb-aea7-08dc70d4d349
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM1PEPF000252DF.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB10362

DQoNCj4gT24gMTAgTWF5IDIwMjQsIGF0IDEwOjMyLCBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnpl
bEBhbWQuY29tPiB3cm90ZToNCj4gDQo+IEhpIEx1Y2EsDQo+IA0KPiBPbiAxMC8wNS8yMDI0IDEx
OjI1LCBMdWNhIEZhbmNlbGx1IHdyb3RlOg0KPj4gDQo+PiANCj4+PiBPbiAxMCBNYXkgMjAyNCwg
YXQgMTA6MTcsIE1pY2hhbCBPcnplbCA8bWljaGFsLm9yemVsQGFtZC5jb20+IHdyb3RlOg0KPj4+
IA0KPj4+IEhpIEx1Y2EsDQo+Pj4gDQo+Pj4gT24gMjMvMDQvMjAyNCAxMDoyNSwgTHVjYSBGYW5j
ZWxsdSB3cm90ZToNCj4+Pj4gDQo+Pj4+IA0KPj4+PiBUaGlzIGNvbW1pdCBpbXBsZW1lbnRzIHRo
ZSBsb2dpYyB0byBoYXZlIHRoZSBzdGF0aWMgc2hhcmVkIG1lbW9yeSBiYW5rcw0KPj4+PiBmcm9t
IHRoZSBYZW4gaGVhcCBpbnN0ZWFkIG9mIGhhdmluZyB0aGUgaG9zdCBwaHlzaWNhbCBhZGRyZXNz
IHBhc3NlZCBmcm9tDQo+Pj4+IHRoZSB1c2VyLg0KPj4+PiANCj4+Pj4gV2hlbiB0aGUgaG9zdCBw
aHlzaWNhbCBhZGRyZXNzIGlzIG5vdCBzdXBwbGllZCwgdGhlIHBoeXNpY2FsIG1lbW9yeSBpcw0K
Pj4+PiB0YWtlbiBmcm9tIHRoZSBYZW4gaGVhcCB1c2luZyBhbGxvY2F0ZV9kb21oZWFwX21lbW9y
eSwgdGhlIGFsbG9jYXRpb24NCj4+Pj4gbmVlZHMgdG8gb2NjdXIgYXQgdGhlIGZpcnN0IGhhbmRs
ZWQgRFQgbm9kZSBhbmQgdGhlIGFsbG9jYXRlZCBiYW5rcw0KPj4+PiBuZWVkIHRvIGJlIHNhdmVk
IHNvbWV3aGVyZSwgc28gaW50cm9kdWNlIHRoZSAnc2htX2hlYXBfYmFua3MnIHN0YXRpYw0KPj4+
PiBnbG9iYWwgdmFyaWFibGUgb2YgdHlwZSAnc3RydWN0IG1lbWluZm8nIHRoYXQgd2lsbCBob2xk
IHRoZSBiYW5rcw0KPj4+PiBhbGxvY2F0ZWQgZnJvbSB0aGUgaGVhcCwgaXRzIGZpZWxkIC5zaG1l
bV9leHRyYSB3aWxsIGJlIHVzZWQgdG8gcG9pbnQNCj4+Pj4gdG8gdGhlIGJvb3RpbmZvIHNoYXJl
ZCBtZW1vcnkgYmFua3MgLnNobWVtX2V4dHJhIHNwYWNlLCBzbyB0aGF0IHRoZXJlDQo+Pj4+IGlz
IG5vdCBmdXJ0aGVyIGFsbG9jYXRpb24gb2YgbWVtb3J5IGFuZCBldmVyeSBiYW5rIGluIHNobV9o
ZWFwX2JhbmtzDQo+Pj4+IGNhbiBiZSBzYWZlbHkgaWRlbnRpZmllZCBieSB0aGUgc2htX2lkIHRv
IHJlY29uc3RydWN0IGl0cyB0cmFjZWFiaWxpdHkNCj4+Pj4gYW5kIGlmIGl0IHdhcyBhbGxvY2F0
ZWQgb3Igbm90Lg0KPj4+PiANCj4+Pj4gQSBzZWFyY2ggaW50byAnc2htX2hlYXBfYmFua3MnIHdp
bGwgcmV2ZWFsIGlmIHRoZSBiYW5rcyB3ZXJlIGFsbG9jYXRlZA0KPj4+PiBvciBub3QsIGluIGNh
c2UgdGhlIGhvc3QgYWRkcmVzcyBpcyBub3QgcGFzc2VkLCBhbmQgdGhlIGNhbGxiYWNrIGdpdmVu
DQo+Pj4+IHRvIGFsbG9jYXRlX2RvbWhlYXBfbWVtb3J5IHdpbGwgc3RvcmUgdGhlIGJhbmtzIGlu
IHRoZSBzdHJ1Y3R1cmUgYW5kDQo+Pj4+IG1hcCB0aGVtIHRvIHRoZSBjdXJyZW50IGRvbWFpbiwg
dG8gZG8gdGhhdCwgc29tZSBjaGFuZ2VzIHRvDQo+Pj4+IGFjcXVpcmVfc2hhcmVkX21lbW9yeV9i
YW5rIGFyZSBtYWRlIHRvIGxldCBpdCBkaWZmZXJlbnRpYXRlIGlmIHRoZSBiYW5rDQo+Pj4+IGlz
IGZyb20gdGhlIGhlYXAgYW5kIGlmIGl0IGlzLCB0aGVuIGFzc2lnbl9wYWdlcyBpcyBjYWxsZWQg
Zm9yIGV2ZXJ5DQo+Pj4+IGJhbmsuDQo+Pj4+IA0KPj4+PiBXaGVuIHRoZSBiYW5rIGlzIGFscmVh
ZHkgYWxsb2NhdGVkLCBmb3IgZXZlcnkgYmFuayBhbGxvY2F0ZWQgd2l0aCB0aGUNCj4+Pj4gY29y
cmVzcG9uZGluZyBzaG1faWQsIGhhbmRsZV9zaGFyZWRfbWVtX2JhbmsgaXMgY2FsbGVkIGFuZCB0
aGUgbWFwcGluZw0KPj4+PiBhcmUgZG9uZS4NCj4+Pj4gDQo+Pj4+IFNpZ25lZC1vZmYtYnk6IEx1
Y2EgRmFuY2VsbHUgPGx1Y2EuZmFuY2VsbHVAYXJtLmNvbT4NCj4+PiANCj4+PiBJIHRlc3RlZCB0
aGlzIHBhdGNoIGFuZCBpdCByZXN1bHRlZCBpbiBhc3NlcnRpb246DQo+Pj4gQXNzZXJ0aW9uICdz
IDw9IGUnIGZhaWxlZCBhdCBjb21tb24vcmFuZ2VzZXQuYzoxODkNCj4+PiANCj4+PiBJIGNoZWNr
ZWQgYW5kIGluIGZpbmRfdW5hbGxvY2F0ZWRfbWVtb3J5KCksIGdpdmVuIHRoYXQgc3RhcnQgaXMg
fjBVTCAoaG9zdCBhZGRyZXNzIG5vdCBwcm92aWRlZCksDQo+Pj4gc3RhcnQgKyBzaXplIHdvdWxk
IG92ZXJmbG93LiBEaWQgeW91IHRlc3QgdGhpcyBwYXRjaD8NCj4+IA0KPj4gSGkgTWljaGFsLA0K
Pj4gDQo+PiBNbW0gSeKAmW0gdGVzdGluZyB3aXRoIGEgZG9tMGxlc3Mgc2V0dXAsIHdpdGhvdXQg
ZG9tMCwgSSB0aGluayBJIG1pc3NlZCB0aGF0IHBhcnQgYmVjYXVzZSBteSBndWVzdHMgZG9lc27i
gJl0IGhhdmUNCj4+IHRoZSBoeXBlcnZpc29yIG5vZGUgKGVuaGFuY2VkKSwgc29ycnkgYWJvdXQg
dGhhdCwgSeKAmWxsIHRlc3QgdXNpbmcgeW91ciBzZXR1cCwgY2FuIHlvdSBjb25maXJtIHdoYXQg
YXJlIHlvdSB1c2luZz8NCj4gSSBoYXZlIHRoZXNlIFFlbXUgdGVzdHMgKHdpdGggYW5kIHdpdGhv
dXQgU01NVSk6DQo+IC0gc2htZW0gYmV0d2VlbiBkb21VMSBhbmQgZG9tVTIgd2l0aC93aXRob3V0
IGhvc3QgYWRkcmVzcyBwcm92aWRlZCAob3duZXIgZG9tSU8gb3IgZG9tVTEpDQoNCk9rLCBJIHRl
c3RlZCB0aGlzIG9uZSwgYnV0IHdpdGhvdXQgZW5oYW5jZWQgZG9tVXMNCg0KPiAtIHNobWVtIGJl
dHdlZW4gZG9tMCBhbmQgZG9tVTEgd2l0aC93aXRob3V0IGhvc3QgYWRkcmVzcyBwcm92aWRlZCAo
b3duZXIgZG9tSU8gb3IgZG9tMCkNCg0KSeKAmW0gbWlzc2luZyB0aGlzIG9uZSwgSeKAmWxsIGNo
ZWNrIGV2ZXJ5d2hlcmUgd2hlcmUgYm9vdGluZm9fZ2V0X3NobWVtKCkgaXMgdXNlZCwgc29ycnkg
Zm9yIHRoZSBvdmVyc2lnaHQgDQoNCj4gDQo+IEJUVy4gV2hhdCB3YXMgdGhlIGNvbmNsdXNpb24g
YWJvdXQgdGhlIGlzc3VlIGlmIHNobWVtIHJlZ2lvbiBjbGFzaGVzIHdpdGggUkFNIGFsbG9jYXRl
ZCBmb3IgMToxIGRvbVUgZS5nLiBkb20wLg0KPiBBY2NpZGVudGFsbHksIEkgZW5kIHVwIHdpdGgg
YSBjb25maWd1cmF0aW9uIHdoZXJlIFhlbiBhbGxvY2F0ZWQgZm9yIGRvbTAgYSByZWdpb24gb2Yg
UkFNIGNsYXNoaW5nIHdpdGggbXkgc2htZW0gcmVnaW9uLg0KDQpTbyB0aGUgY29uY2x1c2lvbiBp
cyB0aGF0IHdlIHNob3VsZCBub3QgaGF2ZSB0aGlzIGNvbmZpZ3VyYXRpb24sIGJ1dCBhdCB0aGUg
bW9tZW50IHRoZXJlIGlzIGEgdGVjaCBkZWJ0IGJlY2F1c2UgdG8gZW5mb3JjZSB0aGUNCmNoZWNr
IHdlIHNob3VsZCBkbyBzb21lIHdvcmsgYXJvdW5kIHRoZSBwMm0gYXMgSnVsaWVuIHN1Z2dlc3Rl
ZCwgYnV0IGl04oCZcyBub3QgdHJpdmlhbCBiZWNhdXNlIHNlZW1zIHRoYXQgc29tZSBoeXBlciBj
YWxscw0KYXJlIGN1cnJlbnRseSByZWxheWluZyBvbiBvdmVyd3JpdGluZyB0aGUgbWFwcGluZ3Mu
DQoNCg0KPiANCj4gfk1pY2hhbA0KDQo=


From xen-devel-bounces@lists.xenproject.org Fri May 10 09:54:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 09:54:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719648.1122489 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Mwn-0008Uy-9l; Fri, 10 May 2024 09:53:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719648.1122489; Fri, 10 May 2024 09:53:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Mwn-0008Ur-6m; Fri, 10 May 2024 09:53:49 +0000
Received: by outflank-mailman (input) for mailman id 719648;
 Fri, 10 May 2024 09:53:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5Mwl-0008TE-Rb
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 09:53:47 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31a79322-0eb3-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 11:53:46 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a157a1cd1so445915066b.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 02:53:46 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01968sm164814366b.166.2024.05.10.02.53.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 02:53:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31a79322-0eb3-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715334826; x=1715939626; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IgDr/j3I45aVQZVCtriHh098OsnX9VLTCcgYdnyh5r0=;
        b=G1oOVFy0LWVpm7QSBPsjlwoIHh/vZMdmF7oy4kPcfSniWTs30q8mRraGW4ZfigEmTZ
         XQFsGNmof1tp1NngQu+T1p27SuBm7k4dcbMogotIheAvOoDgYcr5Z1GZr6pUZNEuxiD0
         mL2AH8Dg0Eue6tbMvayYK6Egk2xuLgD/j8dnWiWaDQ3xxHsdfjiqqzvxkvu0DUlA5lg/
         xkFqR0AbpOaYcJidiErgHPMjZLv63BAVUWOfrXaJk1yTCLoOWF8e2cuBTWtla93tp73o
         8LEAKDz0adnMlDhY6a9VpnMP9LTl9HXjlfRofYejYnv2tiN2xjFsQ9vjItBNqlInJlbC
         KUAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715334826; x=1715939626;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IgDr/j3I45aVQZVCtriHh098OsnX9VLTCcgYdnyh5r0=;
        b=ZXbY/hEoY/zANWyFOnfMu8+MbdEjacf6vmJ46kPSnYdDfcU55Jvg5Stc7MUSakZs9x
         E3NBs6PD4CpvlDoGCb/Appi47SR1+IybSKq+Vp5COkDb/RQmEP6KpLkfX2U+PqLOmWNs
         3ffzke03+Q9PnVDu2TQbAVkrCnLPTHGNL6TGVEwcRTOAjuO54e2xgKnFg1IQOJnGX+3h
         jXYlvfoBPG0YDBk+feKfGXyyins+CscKy37nfsfK766dIbzoZMjK4usZnhRWXQAmaQX2
         CS7oEQo2L9C+t0aiwZhFqhetProGV7ATkrjrTJ8+wXobITDRrLKJWceXSFq0mrK9GFby
         CD5A==
X-Forwarded-Encrypted: i=1; AJvYcCW5zK7Qld1EbqpSamQgCkEH7AAUDcJhFi/IdiPAnSPdqCXfHygFFSGPnptDPYTrsCxXoJzF8lNE6iD/QBDYiCbJj/l/EahsmipJ9jTQoIg=
X-Gm-Message-State: AOJu0Yw5hhrAnH6LFjARDOPVP3CFg6g7NE24Vz30ktycLsagxGXdgLz5
	OopnJY0qgcoXtBqgNTW/Ww+ApHO7rpM8qhKc76b5+rv8GCayGH7pA4k40Bldh0k=
X-Google-Smtp-Source: AGHT+IEKz31JvWqjAEmRomlfNgaF8EqP2HQK8r73XnIs3tomW69Tn3zJ4arHD6nbkZkJnphFIwRQPg==
X-Received: by 2002:a17:906:134d:b0:a59:9eab:162b with SMTP id a640c23a62f3a-a5a2d5d01abmr136503366b.35.1715334826351;
        Fri, 10 May 2024 02:53:46 -0700 (PDT)
Message-ID: <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
Date: Fri, 10 May 2024 11:53:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
 "Huang, Ray" <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10.05.24 11:06, Chen, Jiqian wrote:
> Hi,
> 
> On 2024/5/10 14:46, Jürgen Groß wrote:
>> On 19.04.24 05:36, Jiqian Chen wrote:
>>> +
>>> +    info->type = IRQT_PIRQ;
> I am considering whether I need to use a new type(like IRQT_GSI) here to distinguish with IRQT_PIRQ, because function restore_pirqs will process all IRQT_PIRQ.

restore_pirqs() already considers gsi == 0 to be not GSI related. Isn't this
enough?


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:10:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:10:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719657.1122499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NCL-0002rS-NF; Fri, 10 May 2024 10:09:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719657.1122499; Fri, 10 May 2024 10:09:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NCL-0002rL-Jd; Fri, 10 May 2024 10:09:53 +0000
Received: by outflank-mailman (input) for mailman id 719657;
 Fri, 10 May 2024 10:07:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hQS+=MN=amazon.es=prvs=8538b48ee=nsaenz@srs-se1.protection.inumbo.net>)
 id 1s5N9w-0002mv-LX
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:07:24 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 167ba843-0eb5-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:07:21 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2024 10:07:15 +0000
Received: from EX19MTAEUC002.ant.amazon.com [10.0.10.100:54770]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.14.131:2525]
 with esmtp (Farcaster)
 id 0c298036-b7a6-49e7-b4ba-4e0b449ad4e5; Fri, 10 May 2024 10:07:14 +0000 (UTC)
Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by
 EX19MTAEUC002.ant.amazon.com (10.252.51.245) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Fri, 10 May 2024 10:07:13 +0000
Received: from localhost (10.13.235.138) by EX19D004EUC001.ant.amazon.com
 (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Fri, 10 May
 2024 10:07:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 167ba843-0eb5-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715335641; x=1746871641;
  h=mime-version:content-transfer-encoding:date:message-id:
   subject:from:to:cc:references:in-reply-to;
  bh=ODj1UcfgLsilkOJ93AaBzpY+nLn2txKitmtJHEr2EDY=;
  b=uG+dDPkwkAFq226GFq4iDLlUiLv6H1U46ZqcdZ+yh4ACgeE8BgHoHHiZ
   UjXwZV7FpLhhGn9wWNjkR2NsgYyUz2ekzRhobbSdavUiLpP2MNV5qrSk7
   zYUiNi5hHzYJX9gTly4lQsCmQnZV9yvn6maEL+fdU4Q56AiR0LBX00vNc
   U=;
X-IronPort-AV: E=Sophos;i="6.08,150,1712620800"; 
   d="scan'208";a="88359429"
X-Farcaster-Flow-ID: 0c298036-b7a6-49e7-b4ba-4e0b449ad4e5
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="UTF-8"
Date: Fri, 10 May 2024 10:07:00 +0000
Message-ID: <D15VQ97L5M8J.1TDNQE6KLW6JO@amazon.com>
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
From: Nicolas Saenz Julienne <nsaenz@amazon.com>
To: Sean Christopherson <seanjc@google.com>,
	=?utf-8?q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
CC: Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, "Kees
 Cook" <keescook@chromium.org>, Paolo Bonzini <pbonzini@redhat.com>, "Thomas
 Gleixner" <tglx@linutronix.de>, Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>, Rick P Edgecombe
	<rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, Angelina Vu
	<angelinavu@linux.microsoft.com>, Anna Trikalinou
	<atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>,
	"Forrest Yuan Yu" <yuanyu@google.com>, James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>, John Andersen
	<john.s.andersen@intel.com>, "Madhavan T . Venkataraman"
	<madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?utf-8?q?Mihai_Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?utf-8?q?Nicu=C8=99or_C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>, Thara
 Gopinath <tgopinath@microsoft.com>, "Trilok Soni" <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, Yu Zhang
	<yu.c.zhang@linux.intel.com>, =?utf-8?q?=C8=98tefan_=C8=98icleru?=
	<ssicleru@bitdefender.com>, <dev@lists.cloudhypervisor.org>,
	<kvm@vger.kernel.org>, <linux-hardening@vger.kernel.org>,
	<linux-hyperv@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-security-module@vger.kernel.org>, <qemu-devel@nongnu.org>,
	<virtualization@lists.linux-foundation.org>, <x86@kernel.org>,
	<xen-devel@lists.xenproject.org>
X-Mailer: aerc 0.16.0-127-gec0f4a50cf77
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net> <ZjTuqV-AxQQRWwUW@google.com>
 <20240506.ohwe7eewu0oB@digikod.net> <ZjmFPZd5q_hEBdBz@google.com>
 <20240507.ieghomae0UoC@digikod.net> <ZjpTxt-Bxia3bRwB@google.com>
In-Reply-To: <ZjpTxt-Bxia3bRwB@google.com>
X-Originating-IP: [10.13.235.138]
X-ClientProxiedBy: EX19D036UWC002.ant.amazon.com (10.13.139.242) To
 EX19D004EUC001.ant.amazon.com (10.252.51.190)

On Tue May 7, 2024 at 4:16 PM UTC, Sean Christopherson wrote:
> > If yes, that would indeed require a *lot* of work for something we're n=
ot
> > sure will be accepted later on.
>
> Yes and no.  The AWS folks are pursuing VSM support in KVM+QEMU, and SVSM=
 support
> is trending toward the paired VM+vCPU model.  IMO, it's entirely feasible=
 to
> design KVM support such that much of the development load can be shared b=
etween
> the projects.  And having 2+ use cases for a feature (set) makes it _much=
_ more
> likely that the feature(s) will be accepted.

Since Sean mentioned our VSM efforts, a small update. We were able to
validate the concept of one KVM VM per VTL as discussed in LPC. Right
now only for single CPU guests, but are in the late stages of bringing
up MP support. The resulting KVM code is small, and most will be
uncontroversial (I hope). If other obligations allow it, we plan on
having something suitable for review in the coming months.

Our implementation aims to implement all the VSM spec necessary to run
with Microsoft Credential Guard. But note that some aspects necessary
for HVCI are not covered, especially the ones that depend on MBEC
support, or some categories of secure intercepts.

Development happens
https://github.com/vianpl/{linux,qemu,kvm-unit-tests} and the vsm-next
branch, but I'd advice against looking into it until we add some order
to the rework. Regardless, feel free to get in touch.

Nicolas


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:13:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:13:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719665.1122508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NGI-0004P4-63; Fri, 10 May 2024 10:13:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719665.1122508; Fri, 10 May 2024 10:13:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NGI-0004Ox-3H; Fri, 10 May 2024 10:13:58 +0000
Received: by outflank-mailman (input) for mailman id 719665;
 Fri, 10 May 2024 10:13:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=asbK=MN=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s5NGG-0004Gh-D3
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:13:56 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0184163c-0eb6-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:13:55 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DS0PR12MB6392.namprd12.prod.outlook.com (2603:10b6:8:cc::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.52; Fri, 10 May
 2024 10:13:50 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.047; Fri, 10 May 2024
 10:13:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0184163c-0eb6-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SWnSJgCYVW65VrBuxTED7HnZOEdGgnvU7RmLEjpiBvIsDqOXix1Sgx3gUlfUHWQKIdDKlUfLrdWa5GuvvzyBPa+v12RceFE7uTxzv5REkS0IRdNy5qQk6u1XH1MCyMRq/rszg5xL9gWkOTqdcH8iVWSgL6cZJWR91yXgNbsLNeDLjP+Z5iTP+0mIwCoORaEBdw6GBdwwgrTws+XFhO97Hvd1jHwyfZmgo7fTFvofBm145c7p8UKT4rboQFx/rVb+oFy09KJq78wvIQPPz3yZ20bWSzkQkTXeZOtxUHIhlqk80IbEQxRHtTCL1GDwfRFyRWsRT161JZ/VaAHXp9XNMQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fE1xWl7iYKuUfNNEOxMllWLYvD3ACCLq5DbAXgM4d+I=;
 b=lwPMk7g4shJq9CPQiwPkK1dE6WE2vbshSZt+37w1UkJ2W/7qdDIBifkQ1Phrb7FFi2Ks0r9w1Wh34y5kKFXVp1regreM/Uqig/vyFBW5reS8U1CCrQV46z5/3kH0JOsrQMktKctfQk40R0nLeSUXeMnyAtCEw9tuD+oUiL8P7StZtV8c+aj9nWrQOXr4vQEdFEHKW11EcI2WGshzDuhhayFOuFy6qO6U0y/jk7UBXtSSUvMLfj4ZT3YWiFroGhhoGA0vqO8CteEFW4npIHhqX7oDiVW4u6F0MVoUDXKVLtMInfDsriDpahWNZm/dxYlWdsuA31Bnr3hz52kgHpJbVw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fE1xWl7iYKuUfNNEOxMllWLYvD3ACCLq5DbAXgM4d+I=;
 b=JJQplDlxl4HJwgip2oV0P1hxgboYf9S2Al4MMx8Seo1f5uefDjfbz2ixoKMjVK+5l2UTrvxSE7S18L3R4Xv1Zx4TijDp21BoE59l7D4L7rVMnbQDTtG6+Gt8lNfOxqiXGsI8O69kI5BEGCP4I0VBEy2hMWmaUgmgLc24lgZyLuc=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>, "Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
Thread-Topic: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get
 gsi from irq
Thread-Index: AQHakgrPZQMiHHEzE0Gs/TUOGsHh17GQJ2yAgACnAgD//41hgIAAiXAA
Date: Fri, 10 May 2024 10:13:50 +0000
Message-ID:
 <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
In-Reply-To: <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: IA0PR12MB7579.namprd12.prod.outlook.com
 (15.20.7587.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DS0PR12MB6392:EE_
x-ms-office365-filtering-correlation-id: eb5a51df-e362-4be3-b4c9-08dc70d9e382
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?QlRVTCtCRi9tYk5zSm9OU3dGNnNqSUl0NzRGOVF3Q0s2b1pBZkhoZUNuMWxZ?=
 =?utf-8?B?bkYxeHRNQ0lpNUpNa2JGUS9VQkZySWNkbDk0Rk8xQm5URUs0dUFrd1NzamNK?=
 =?utf-8?B?d3JZUlFCbVZHOU1zbmZCeGtMZ1Yyck5BVDljcmVKaGJRVmFqTWxUenJYRmtm?=
 =?utf-8?B?bGdnaFZ4NmVjODM5NlFiYnVaMm1tWXRkQVJObXljeUp1WnVTREwxZVdYRzl2?=
 =?utf-8?B?TGZtNk9xeGJBeWFSQTFHSXdWSE82TnJJM1NIelFXS3pucnBtdVd5R3U5L0tG?=
 =?utf-8?B?QVdQZ1FFelA1QStOdVIwY0U4aGt3Y1p0cGg2Q2V2NUNFRit2N2dJTXhDSzE4?=
 =?utf-8?B?TWdhemUwdkRZekI1SmJxVG1EbDBySkxtWVhnajd3U0toTDVHc1hYaEZ5L1NZ?=
 =?utf-8?B?emxyWWlOTHEwTmxJdEJ2b2REWkptQWhucG9Rd0VHMDd3aUVWTG5VRXE1WXdJ?=
 =?utf-8?B?MGZUdUdjTnFsOHh1akx4QWNLQ3ROemMvN0RZNFVwcjNkRzZiL0RGNmlmZ2w3?=
 =?utf-8?B?UVIwUWRWbnRpUGk5My94V0p6WUFEM1kvUllsUHUvYTRyZzZQenlTY1pzQ05s?=
 =?utf-8?B?bk1GZWJicDR5bjV4NCtlZmNmNEdCLzltYm5MRXV0VmhBcUFPQ1pEU1hKS2FZ?=
 =?utf-8?B?eCs2RnN6MEFzcjlZRlF4bzd1WUpkcmZtTThCNlUwYUtMNlVqRDNkQUVvOGVo?=
 =?utf-8?B?cUYzTVEwa2p5eG1zcjBLOGozdHVqazBlcnZJNUh6R0FpZ2taR25xUktFYUJG?=
 =?utf-8?B?TnpIbFIvL0lOeEQydkJZTDlZblZuTko5UE1oZ3l4RnFuWkFVQnZaSW1Wa1Ba?=
 =?utf-8?B?Q2pWVlVkMlNycExxNTRybklGUkNqZnp3SVBuVjlaV3c1S2hlU0RYR05nZCtt?=
 =?utf-8?B?UktxakhYOWRnWG4wQmxmc2pSTXJFQktURVQ1dmhsOEhDdU5hQlhFWHRNdEsx?=
 =?utf-8?B?RWFYOStycS9jaTlYQnNmUEpkMGtOa2tBME5TOGtrREZTMFNxZUx0c25nelhY?=
 =?utf-8?B?Y0NRN2lCNk14UnNJWjc0Q0xCbTV3SngxRHBrR0trVmM1R05xNHpmMHJLZFFt?=
 =?utf-8?B?bDhwNGJnSCtPcDMxeFcxaVJTZnhXLzJwbGQvZURhQ1BuY21mbzRoSU9IQUVT?=
 =?utf-8?B?K2lhV2hoemV5b3dkR2RQV2lrd2UvRHI0L3ZVaGhhbnplWWlZWjNGMnMwNnRn?=
 =?utf-8?B?WGp3Vm0wV2x6bzViMUsyNXZwQTZ5NXZvN0pybXh1TnpPUXpmNERSMkUrbWpO?=
 =?utf-8?B?NUE2TDhoa3V4QzBHTXF2UFdGeXdWSndBWU1EZkdXWkxkQThOaEI5cHdCUm9V?=
 =?utf-8?B?WXI5THZ0UHRUd0RrVHVMV2RyelgrVk5HMUJIclQ0MzlXMUQ1NDdmTTdLZkNF?=
 =?utf-8?B?ditraTlRejQrellEVnd2QmVOUzFGTzgxajBQSndqYk5FTnFxa2tOYXh1N3Zr?=
 =?utf-8?B?NG1oUG11SmVhWkp4WTNEUTN3b3B0OWpBQ0JJSE5aQ2Q3dHU5Y2szTy9MZmJB?=
 =?utf-8?B?VExsNVRhSEx3b3JqdmprWUk0Zitqa21ScEorMmNsOWxOemhMcFFqbzRiRjdT?=
 =?utf-8?B?RHN1OXhVejh4VXNqWjdPcFdUQlhsbnczUzMyMEw2SjFtQUovTXFSUWpEVndB?=
 =?utf-8?B?amFoaUJWMk9tV3ViQVpEUGRJTVo2OVU0ZmxmVWFWSHdQeUhsWnFzZm83L050?=
 =?utf-8?B?YlQ2ZUJvQmtjWXlhVnFKUWpIVFB6Yy9RbThSUG9QeGhZOW9LN2t2OTNWUmFN?=
 =?utf-8?B?NVlhcFRrcUpaN1ErMWhZQW9yRUg3M1JyQitGdUJkckt6SkZveW1jRjRSU2ZF?=
 =?utf-8?B?OXk1c2FHeGkzeVJETlJwdz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?R2hUMHdOZDlqeThONkpRN0ZucXVJcXBLZ1BVM1BZeXA0RU4xbWJOMElOZXQ0?=
 =?utf-8?B?MVhGMkJIVTQzSDBJcnlxK3lDeHpoMXhPd3lJdU1remRlcHB3ZWhhVlF4QnBY?=
 =?utf-8?B?ODFsV290ZUtTblFjQ1M3eEJFeDROYk1kUE9WVkZTaWJCYUJKMXpXZk9DbVVY?=
 =?utf-8?B?a2FTN3hrSG8zQWlnS3ZlVDlrN3haRVM0ZC9rcDhnbkNZTTEwMGdZNmtjRGh5?=
 =?utf-8?B?cElWbkxmdFBIK05PMEZPcFBCK3pORFIvTTc2cndRZTZ3Q3RQelVxbE9ZbUdP?=
 =?utf-8?B?OGZQTkpTbTZwZkFzc3VVZm0xeGNSeXJQY0hmd3RWTXZvemJRNFVJamtLMkN2?=
 =?utf-8?B?YjdwMWFuTGY4cE1NeHZTbTY2Y1hZbjZiQ3NTMm1LZklWZ3pJcHJ0V0FzSk1B?=
 =?utf-8?B?dk9HRW1CakNVK2VPaDZQMkJTbkkyQW1WMThZL01JR3pZZHZQdmxsNTIvbDV5?=
 =?utf-8?B?N0xsc3BwdW1HdkRsaXREOHYrbWRwakZGSFNZbXowTHptQnhQeXFnOGJVTERT?=
 =?utf-8?B?OTVIYThWd1pVMTR2clI0c2FoaThCdUIxSFRjOXlRQUlNc3VlQ0thVXQvYXFL?=
 =?utf-8?B?RWRGUmV0dWY0bUNOaFNCUlJueGU5TTRaNmJIY0QwM1J5QnN4bDhYSTF3NlZW?=
 =?utf-8?B?UEc1WVlYRVNxZVhKVWJZdis1aDhBUjJ6R255T2dxUDZzanFWTDl6MHpZSTlI?=
 =?utf-8?B?RWhKc00xb1E5cDBqNEtXdEVqYm0ySWdXemdYU2dXZ0ZxT1cxbCtBTW0rTWJi?=
 =?utf-8?B?dXFZZHFhbnpDcmwrejM0Z2x4OVVzRlo5WWZmbUxTQzhxRGZOMFVkSWYzQ3Z2?=
 =?utf-8?B?L2xiMGd6VWRNZlhUUEtKd1dIQzZGeGpwOWVNVnhkNXZ5dnRvQXVzcjZuYUNB?=
 =?utf-8?B?bTJvVUlaU0pLOTlpdUNReHlZVjN6NDVYcUl2dTVXV0l1OEpUcWRMZ0lmMzF4?=
 =?utf-8?B?VmtHQTBXbVVvVTlRV25pclhjeDl6aHc4MVo4bEpmb0dFSnFtMXQxb0VsTmFu?=
 =?utf-8?B?M25VbEdKdGNRcUE1TkQyZm11QXlzM3FCQkNRSXNldUxCNXQweW1oTFpFQkZW?=
 =?utf-8?B?Nk54R25tRDFGYkxmdDFNVDlmeitiVGdKVFlSUkl3d1JPdUpDKzI4SEtkWGdQ?=
 =?utf-8?B?elJTbXcvTWErUC9JNHM3VFAwaW1aQWxFcE1Zd3M0Q2FrWUEzVTM2MGZMVjdT?=
 =?utf-8?B?dnNHaEFWcnZuc0NZTm1mRjNVZTVScUJ5eC9WMTNjZ2VTdkJ4dGEwTkFOTmNz?=
 =?utf-8?B?L2pzaXdHSnozYWZCTXZ3QmFBYXRYUmRXbU02aklyczVOZGtod3ZmNTB0amZn?=
 =?utf-8?B?UjVtbHdKTkJxRVpuaXNmYWd4VXpNQllPWXJYSzVRUXlzNXlvSjFZampsdDBI?=
 =?utf-8?B?d1ZXUjlnaWkvUS9iclk2OGkwVjRBaCtMVWhpelZmc1RYQXVsRjNnQXJFek50?=
 =?utf-8?B?VEJKZldnbW1NWU9PYTFLNW5LUGZhVGF5T1VqYThGV1N0SS84Q2ZUcEoxNWRt?=
 =?utf-8?B?djFMd0R1ZDh1c1BLN01MTnI3Wno4ZTJ6WUdwSlE0WEpydEhjeWw1QjJrdXJk?=
 =?utf-8?B?Um5sYUtqNTV5eHpsQkJwZW94cDJQTERYenBDZ0QvdFdCOHB5NDdETjFvWTQv?=
 =?utf-8?B?UTJtYm01cmkwNm1qczJleUxOM1loR3VEQmFWTVRiMXlOdmpObWlZaGthd1Zx?=
 =?utf-8?B?MFd3RmY2L1JNYXE2U1d6RVUzWUptQ21LQVpRMjdLQll1aUEyS2g3VjV3allk?=
 =?utf-8?B?SUkwaEhIa0IxT2ZldmlWS2xzdlY0MGJOa3ZKbmdUUFBpd0tlaTNncnZDOTA4?=
 =?utf-8?B?S2tqc1RNOVhBOVhBbjFuVXFoUmJLSHRhbVBwZlM1SVdxWEFzendCNjlpWU9n?=
 =?utf-8?B?WERETXNnbjlRbzEyK3kyRnNQd2Q1MkgzYlVhWEptMlpiWUhqRCt0S1N0TDFN?=
 =?utf-8?B?Nks2VnFyT3k5NzBkbG16ZkJhZTEzeXlVam5xck14TDFxTWtIMXJIN3oxeE40?=
 =?utf-8?B?S2p5RVBDZ01RUnFaTkNVZk1vV3JvT3Jqdk1pRmFmdk1nZ0lPU1Q2bSt2amMx?=
 =?utf-8?B?OG16WmFqTGRHbDh4WjV0VVByV3FMTElKcGxDa1hpOUNQbGdMMSsxYTExckFy?=
 =?utf-8?Q?3RzY=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C27481B130639B41BBF51FF7DFAF3F19@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eb5a51df-e362-4be3-b4c9-08dc70d9e382
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2024 10:13:50.6549
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: bXAkf5tAeRZ8Z3Vw+wUSJl9wUDmqgCndMNtFq0tI3UbzpZyftp0tDuN8DS+I0frQcmFixGOl4qtK7oHek4R6Tw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6392

T24gMjAyNC81LzEwIDE3OjUzLCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPiBPbiAxMC4wNS4yNCAx
MTowNiwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gSGksDQo+Pg0KPj4gT24gMjAyNC81LzEwIDE0
OjQ2LCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPj4+IE9uIDE5LjA0LjI0IDA1OjM2LCBKaXFpYW4g
Q2hlbiB3cm90ZToNCj4+Pj4gKw0KPj4+PiArwqDCoMKgIGluZm8tPnR5cGUgPSBJUlFUX1BJUlE7
DQo+PiBJIGFtIGNvbnNpZGVyaW5nIHdoZXRoZXIgSSBuZWVkIHRvIHVzZSBhIG5ldyB0eXBlKGxp
a2UgSVJRVF9HU0kpIGhlcmUgdG8gZGlzdGluZ3Vpc2ggd2l0aCBJUlFUX1BJUlEsIGJlY2F1c2Ug
ZnVuY3Rpb24gcmVzdG9yZV9waXJxcyB3aWxsIHByb2Nlc3MgYWxsIElSUVRfUElSUS4NCj4gDQo+
IHJlc3RvcmVfcGlycXMoKSBhbHJlYWR5IGNvbnNpZGVycyBnc2kgPT0gMCB0byBiZSBub3QgR1NJ
IHJlbGF0ZWQuIElzbid0IHRoaXMNCj4gZW5vdWdoPw0KTm8sIGl0IGlzIG5vdCBlbm91Z2guDQp4
ZW5fcHZoX2FkZF9nc2lfaXJxX21hcCBhZGRzIHRoZSBtYXBwaW5nIG9mIGdzaSBhbmQgaXJxLCBi
dXQgdGhlIHZhbHVlIG9mIGdzaSBpcyBub3QgMCwNCm9uY2UgcmVzdG9yZV9waXJxcyBpcyBjYWxs
ZWQsIGl0IHdpbGwgZG8gUEhZU0RFVk9QX21hcF9waXJxIGZvciB0aGF0IGdzaSwgYnV0IGluIHB2
aCBkb20wLCB3ZSBzaG91bGRuJ3QgZG8gUEhZU0RFVk9QX21hcF9waXJxLg0KDQo+IA0KPiANCj4g
SnVlcmdlbg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:21:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:21:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719671.1122518 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NNt-0006Wf-TR; Fri, 10 May 2024 10:21:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719671.1122518; Fri, 10 May 2024 10:21:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NNt-0006WY-Qr; Fri, 10 May 2024 10:21:49 +0000
Received: by outflank-mailman (input) for mailman id 719671;
 Fri, 10 May 2024 10:21:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5NNs-0006WS-BA
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:21:48 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1b03186c-0eb7-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:21:47 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso521845466b.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:21:47 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d342sm168537766b.6.2024.05.10.03.21.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 03:21:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b03186c-0eb7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715336506; x=1715941306; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gB3+mueVJC6rTSgQyDulrWqT4OQyPBK4CkJ9nqNOnDg=;
        b=ZqV+DatO0NVL4iTgcFtyb8OHCm8NseP1ZdFl+xC5+S1ZsCZsfe9n2s3K5hDVCLY2dN
         w0/BEoILo1Cxv3I4ErqCAp8bWwV5CU3jPxSX5BXzt0ZaRQPTQMdOEdaRDXVYNuMLnEby
         KyofvvNUgcBldppshgk+hXaNmGIzJp2a3BYm7YjyqWppwapqy0Z1KVX2SPDw0ypbiItv
         wCQMWm0zOHIk+M2RbM1a694FdbpWyqCy6cnLpS1FQLOtRpjKQkzLlBr7JAvkFuhbOX9V
         cPCMZFld8Yzpu4g3G6nw+LHhXEGPgCO6jjyp/A84qe7jJBTfsCgJRs2jCYp455YGhuC5
         egSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715336506; x=1715941306;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gB3+mueVJC6rTSgQyDulrWqT4OQyPBK4CkJ9nqNOnDg=;
        b=pvbBw4Iz7AZ1RPMc5sLgZT5GefSp0uqJ/wBpDA3C0AnI+GX9mUD5V6dyx2b6trMabR
         C5IA9rERsU0n1Ym82HWqkdy8skjK1B/dQr+bKAAJxX9vKUjStvWpYQVJcv7zI2r3rh69
         codce9oka077nDycfGThG8KOYZrJvlBGT7OtJRAQUP30t5c+HjRBN8+FTJP4A5VE0b/1
         GKhiGC90SPKzlBLMElEDMYWSJ0aE8J78TarucqjEM1ujVOKt4dKihg4JmbXbuPmZ/i+w
         BNzhj2wmMMejDAN8LdPWznOPzYZBR0+lATdriqPR9hi/ZdYcyiVU4UDg2zNTODnPbzQB
         ZDgg==
X-Forwarded-Encrypted: i=1; AJvYcCUkqeazVFndQTf0QY2CnVDJiUh15anydoh5YF30E5n3dMGlkKfUd6fo55WgkgS0KUgIoBPGz6/N2XQ4caBEZo1CO/U3YvmsRmUoYz4R/hc=
X-Gm-Message-State: AOJu0YzCwwi4gNcS02NhbSVa81I01IdShtbosZXgpZBah+5jMwDv8CAr
	1eu/VbOdT+TE0ZGy1dk1NTt95Xmn05rkeG3PKwcdVGlryBVDIt4McSbGIKJr6xU=
X-Google-Smtp-Source: AGHT+IGm0ShzWqxUTz6WdBSaEzgp1Qnpb/3O6/pweXuPBaR9LwjAUlSsib161KD3DVm9rCH0vGd4qA==
X-Received: by 2002:a17:906:cc84:b0:a59:c28a:d350 with SMTP id a640c23a62f3a-a5a2d55ab1fmr161431566b.24.1715336506363;
        Fri, 10 May 2024 03:21:46 -0700 (PDT)
Message-ID: <d0b5e7d5-3503-49be-9fa3-4b79c62059ca@suse.com>
Date: Fri, 10 May 2024 12:21:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
 "Huang, Ray" <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
 <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10.05.24 12:13, Chen, Jiqian wrote:
> On 2024/5/10 17:53, Jürgen Groß wrote:
>> On 10.05.24 11:06, Chen, Jiqian wrote:
>>> Hi,
>>>
>>> On 2024/5/10 14:46, Jürgen Groß wrote:
>>>> On 19.04.24 05:36, Jiqian Chen wrote:
>>>>> +
>>>>> +    info->type = IRQT_PIRQ;
>>> I am considering whether I need to use a new type(like IRQT_GSI) here to distinguish with IRQT_PIRQ, because function restore_pirqs will process all IRQT_PIRQ.
>>
>> restore_pirqs() already considers gsi == 0 to be not GSI related. Isn't this
>> enough?
> No, it is not enough.
> xen_pvh_add_gsi_irq_map adds the mapping of gsi and irq, but the value of gsi is not 0,
> once restore_pirqs is called, it will do PHYSDEVOP_map_pirq for that gsi, but in pvh dom0, we shouldn't do PHYSDEVOP_map_pirq.

Okay, then add a new flag to info->u.pirq.flags for that purpose?


Juergen



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:25:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:25:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719682.1122529 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NRG-0007mg-To; Fri, 10 May 2024 10:25:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719682.1122529; Fri, 10 May 2024 10:25:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NRG-0007mZ-Pa; Fri, 10 May 2024 10:25:18 +0000
Received: by outflank-mailman (input) for mailman id 719682;
 Fri, 10 May 2024 10:24:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IhR9=MN=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s5NQL-0007LN-0z
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:24:22 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 765e6878-0eb7-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:24:20 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59a64db066so469948066b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:24:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 765e6878-0eb7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715336659; x=1715941459; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=fEGPsBZydelNzF6GCnmCbxAZEoOEsL0Jb2ytxlR6SrA=;
        b=Lqs8OS3CAKdYxVtsSHGAHDdbsHhRUG+GKvNRDbDVzjdhjqcSgHwljZ9BRx4M85kEW9
         jmpac+k9WUmx1h96pzlswCDPZMf+0R4R10ggjdJOtvLm8bhG/UVdq9JY/tyeicyXSttj
         uEQYUfrqOkhGRvN68Z5oMjxMINCh1PqAUc5k8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715336659; x=1715941459;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fEGPsBZydelNzF6GCnmCbxAZEoOEsL0Jb2ytxlR6SrA=;
        b=aBpvOvwGRtisl0ZhADj93INq7/YQzEFMw5K2fhC7GjqEj90h8yYpeP0EPEkbvos4I2
         /N2Nvr5ybIEZEBK6neGVBWTYtUxvlOPYejBI/mFvV3lZorY3XYc+wBZAirOdSsYPJcxf
         pGdF+vK94yPOaP34UbU+OINr61RhUgWVWDitN0pMjsypHgSBkhkS9rlMBREuMYy8mv8L
         pe2AdgpkjgAKld+cwyisczsFfc2wB8tA8HrxEQ1wTNxkIH+MpE/AbvBCwlwR6J9hRkub
         7lGJW1sNbPKP3jz421mgkCnyoBTbBMeYhSiAlxUXqnWmgBSJZDET1/Uh7RNWFED3xOAW
         gPjA==
X-Gm-Message-State: AOJu0Yx2VU/Oxuea8/JRLRcCrXdPZ7P6yL01tY6a7WUkGgwNLZISe8Dh
	sbSU5Ezp9vNsL2l4Dg4UBdO1rwmbwwPnM7UVMEfl4WCDnWLgi5EjuyMQTJLiFjeMx+w7ArcLvkf
	cxA41eUFfDALDSNjgx3o+idRR3org6v/xf0xYJHah4+ZQwEL3yBI=
X-Google-Smtp-Source: AGHT+IHgM7tiDCLDAJVcr14t3xlizbxKETtb+kmHdnLb2rlrOJ6JiH+c5ElraLA8AceWy1xDWSUL43zySew8Ndad2AE=
X-Received: by 2002:a50:d791:0:b0:572:707f:1a99 with SMTP id
 4fb4d7f45d1cf-5734d5d3747mr1375630a12.25.1715336658046; Fri, 10 May 2024
 03:24:18 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Fri, 10 May 2024 11:23:42 +0100
Message-ID: <CAO-mL=wJoCGJHr5bPbU7L85njPLGDbG7eWfxA_yMqMJ5ytEjgA@mail.gmail.com>
Subject: Xen workshop slides (in German)
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Content-Type: multipart/mixed; boundary="0000000000002c93af061816f0de"

--0000000000002c93af061816f0de
Content-Type: multipart/alternative; boundary="0000000000002c93ab061816f0dc"

--0000000000002c93ab061816f0dc
Content-Type: text/plain; charset="UTF-8"

Hi all,

Juergen recently gave an introductory Xen workshop in Germany.

The workshop was a technical introduction to Xen and was created to educate
a company considering using us for aeronautic related concepts.

For all our German community members, you might find this interesting.
The slides are attached.

Many thanks,
Kelly Choi

Community Manager
Xen Project

--0000000000002c93ab061816f0dc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,=C2=A0<div><br></div><div>Juergen recently gave an =
introductory Xen workshop in Germany.=C2=A0</div><div><br></div><div>The wo=
rkshop was a technical introduction to Xen and was created to educate a com=
pany considering using us for aeronautic related concepts.=C2=A0</div><div>=
<br></div><div>For all our German community members, you might find this=C2=
=A0interesting.=C2=A0</div><div>The slides are attached.=C2=A0</div><div>=
=C2=A0<br clear=3D"all"><div><div dir=3D"ltr" class=3D"gmail_signature" dat=
a-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Many thanks,</div><di=
v>Kelly Choi</div><div><br></div><div><div style=3D"color:rgb(136,136,136)"=
>Community Manager</div><div style=3D"color:rgb(136,136,136)">Xen Project=
=C2=A0<br></div></div></div></div></div></div></div>

--0000000000002c93ab061816f0dc--
--0000000000002c93af061816f0de
Content-Type: application/pdf; name="2024-Xen-Workshop.pdf"
Content-Disposition: attachment; filename="2024-Xen-Workshop.pdf"
Content-Transfer-Encoding: base64
Content-ID: <f_lw0j11xh0>
X-Attachment-Id: f_lw0j11xh0

JVBERi0xLjcKJcOkw7zDtsOfCjIgMCBvYmoKPDwvTGVuZ3RoIDMgMCBSL0ZpbHRlci9GbGF0ZURl
Y29kZT4+CnN0cmVhbQp4nLVV3YsaMRB/z18xzwcXZya7+YBFcF0tVzioPaEP5R7KtfaDs0U5uH+/
ycTNuVatLZwLMZlkZ34fE0VN8KxGk+3T99Wnhydob6dqAwiokT04Cjo4ggprjRXD9ov6cAU/FUF6
tl8V5i0Ha5XPch94hD6QUqU17s3KoW9qdaVmsegCpCw4Lrt9tY0EsbxGJdXDWo1u1hV0v2ChRu+a
ZnQ7vekAx+O2KynPMtlA4dIuFcdTlXfAnjVZWH6G0ZzAxNmqQUJGgzEBWnToMeAkztrx8oeaLdVC
WLxgoIxhmL+ypIkYDGsXSn6qNKKJNT42OB1Tgx269DVLQ6yZvqo0zHPcxoFQQkR58375NoPYJL62
/8SFKZ+48DsctfPaMZ3GQSwAXupFtiHVEzCBBE4QfFSnuJyx5PaRiE9xPKgWTUMdAsPzX7C+fxP7
65o1E8EaiG2/eIS7P/TmY3pHRlXtwJBGEzJNBhdZclZb1BOqZnzNO5XrwnnAJnfgzkejfc0n8jbo
S08cvOZj752BE4qqXuBM0rRNg7SFoO2OKTxMLALXlPSVJ+sYOw9t1NEYHTu4X2cphzA9aWv8Rez2
xfa19tG5k+yknefSu4ldbl1pHs6Ud+2125b2rsuOLTMn2zIN5QrU2S8584+XgVz1SrYcZN7z5YK+
P+/X4FcL3cWGXSJIvGJnnaQkQPYnW9XmG9TfpapYNRW9xOMuDbNiVZVjR0Q7qP5/olnWAWkoWv6X
+Q1Py3U5CmVuZHN0cmVhbQplbmRvYmoKCjMgMCBvYmoKNTYyCmVuZG9iagoKNCAwIG9iago8PC9U
eXBlL1hPYmplY3QvU3VidHlwZS9JbWFnZS9XaWR0aCAxMDI0IC9IZWlnaHQgNjgyIC9CaXRzUGVy
Q29tcG9uZW50IDggL0NvbG9yU3BhY2UvRGV2aWNlUkdCL0ZpbHRlci9EQ1REZWNvZGUvTGVuZ3Ro
IDY5OTcwPj4Kc3RyZWFtCv/Y/+AAEEpGSUYAAQEBAEgASAAA/9sAQwADAgIDAgIDAwMDBAMDBAUI
BQUEBAUKBwcGCAwKDAwLCgsLDQ4SEA0OEQ4LCxAWEBETFBUVFQwPFxgWFBgSFBUU/9sAQwEDBAQF
BAUJBQUJFA0LDRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU
FBQU/8IAEQgCqgQAAwEiAAIRAQMRAf/EAB0AAAEFAQEBAQAAAAAAAAAAAAABAgMEBQYHCAn/xAAb
AQACAwEBAQAAAAAAAAAAAAAAAgEDBAUGB//aAAwDAQACEAMQAAAB8hcx/KlzmLMPEAFAURwwijgF
R0w5yKK4AhQVpAcI0eDMcqSqqgAioAoEKIMCACq1ZV40hnNEUVECRrmggJAox0jgASOSIIYZK8WM
RiSshGBIsShIjUBI1jYRqMB6xKE0kEoszmuUB4DEkYETJ2QRJIwGI9gyA0AQB7WgqjECRiIM5rGQ
70YweRrEiZGxjRI0aS4aDOGoCujWanjAh7WoK9I0aJUjaEzYiSVIgHNYxh0asdUFGb0IF5l6gA4C
YFUFBxIA6RBXMoqKyq5j4BUWRQJV4igAAATAAAKQCKSIK4Vo4Aa5sMAAiKKrEe2RrJEUa4c8IryV
ghtVhq1SSiXTJUY5eSiBdKKsXEpsGuR02TNtlVkJdfnyi356FspsyMkKQCYc0SJGPRSNkkbEaOaD
WuaDBGDPIRXekcaXzNYFr2oQoyRC1rVR86oD1qiMCRY5kcJVovppJXvoeQK6vVoA1EkURpL2CDKi
DADQBwwCyKd2SrymiWR0kayPZYFmUSIlWYiWVwsRKrpEsyhEsoEZI50jWQEYSOCMkUiJXjKwepDC
RAEc4GEiA1JAIiUgibMSsbZmqREqM0RKKsRKOMJhipQ1MuXycy5iTumjotbVeSgjNoSZ9hVsMCIi
jWs8vSsjlubPmWNW5laMYdCxXuNlRs4qwE4NXSZArMliHRgLYxqxARtamhUHRe0meRXWwTVC6R4V
2TRpsgJXW4oEmHSFs4sxyudm0zKr811DL28PXXWdAm2b02bZRLbXrWkRIkkbXoDEcjMiPnUryXZ6
VpWLT6G6VzjNUhIsw1XqysSUmI3SDLG57xIHyrMQrOrJA6ZzrAth0lV1lWiqttZWo64rlJ15WKDr
zpigug4M1+goZ7tFWbNXUCMs1FDKNQGy01FJym6zRss1AMt2oTOa7Se8YeR1+c08HzHonLJv5hLM
CaYldLI+43UjNWXXkmjnqHS5M3YMVyvN7HijW9XN3Ez3b6aNmWqaALQboRsZ9bQoI9SGSqugjaRc
2ORKoia9a7EsMsKpIj9FA17HVBW1aIWuji8ma98rEVHRj0kqlo+DLdYnqXKivhdHzt5lop02SSNQ
v2M6zSlhkblVQYBPVmDRtUp6EtlSPO19KL3XuXXFz66a3nStJ90aKS31laDro8U1uPlaTrj7Epre
c0Un3ZGXPfoveMx+grrnv0FmM995zlB9wZa62HBWdYAgfISMVzAeqNmJBoS4a2R7EZBI6BYaZYnM
r1jeD5I1cWrcgY57k+y5qu7hMnpMDLrgsNsBd1c3SfNpTVrD1VMbYyEbn69+vVdAk7WssbmHqSnQ
aWDcenXTMRk0YKMCvboRV0uIWjWjiRJbHJDnGkZEzOrsVba5iXLptzh4021313DG1ZL60pGpsJVj
mb1jL0KSarNTpeS3hmhNXHbXsGgaGQAFfGBOsCqWY4wFkikC5LXnpSRsjlSFZWB6o+mvP6dx9BXi
+UmyaBRaJoJnsk0zLBdd2OPG1JiPlNyXClddxcVWjZdiK6bZiDptpjNZdhMUaNp2G5l2jGUNdMgk
148tg2uuPJK6rssY1UzFDQbQQLzqEgXZKb2La0lC66ijxfjoRMLz+pkV2c3i9DkZtFWZ0oPtQ2Hp
tTVSFTOsZ9NdOOaOqYlRXee3RsNboSUQnQhz41a/HQAuJWCZn152i0V20k1eGpM2J6tiAilrkVWC
aZWxBoIWJpZcD5ubsZemC3Su2Uvq34Cqto0tyvRTz+/B/LWdjzG5KauFGgAgKCAoIoAKjwSRkili
zVtVpOrlEQkaHaJEzi9mwtYYspAkxO2EYkbG2UkbGMSOhJWda40WnUiV0DOVo0DPV10HUHul59OV
0neye5BzpniN8rwgLSuU1tvlacttWiB1p7FR10Ckt5Ca800xEBdInPS+xppsvxuufBpQXGZn7dJW
5zK6jPpbnJdR1JTsWpoKEenVIyc/UyM1NaNzc6RsfHa8k1V0TeWB8XEc4NXdadC1ZbbkK0d2kq16
76GiXwxyXvfs07GZZqk9QaNFL5n0a2jkaV8Rmevn3KumqnPX1tDVrX0PZy3cF7PyPB4NfuzeJ6bW
ravE906cR4T6zzWijyxt2lpzghICoCuYoOc1VJCMFszU1Q1JMuSE13Zsyx2zGJx+3KRjRIjEkkRi
MrkYSr0ajEhGSr1YhLxgyqIrw5WPdHyQyTE01ea2uzYpz2LakrutW0+uSWH1XMtlaz5LTqiwXnU3
yXFqqFkgSGuSUXBplBJL8dNoXo6scliGCIJK7I4I6dmOCkTsQRyMJKc9MTPx9PEpoYkZUPYjWHOa
srJPXkrLE1exU00kclayLGxWdRnz9C1KduLVdAPY7yz01gtwRoD7FXQQvWoZ8FsUcQ46KXXFqfS3
G9zl6kPy79E/M+/Na7/hOrQ9E57yiCxJNblDTRIxo66MdKwk11cPDR4DB5AgpAiqA0cgIqOB9yvf
ROqdVfyunOQDEpEBKRExIRjRIRoRKkZMSEY6ykYEpG5olWJXWdYnzEsld4WZKb3W6+i8LrqLmi4t
JZi+tJQuvz1adJ2cpGkuY8NF2YoaS5gTprlhGmmYBox0mE3G0wiwyuxSdleELUdZhFhsDCZqj6gt
LI0MtKmIwaHoxCZHRKROsbq0sTZ7onRkzEWNCOhGxerwDy9o5mbHeVTLbegteuPJexpw6GRJobWf
RbTkqz6a7n0H5H9G4d3jPQ7nRRp8v8R9U8d6uCJULaRzQAAAACaEDuOWpvqlyorVtbI0AaTLhrgG
ukBk8tyIjuumhJ307HO6cohAogwoEABMADAAyiooDmumByEj1YEPfGrEixukkdEosyxKEroXSSpG
CyrCrE6wAWCECwtdQnWuBYdXGLDYkCVYQJUhFJEjQJISGBtYpLXZMxGXVdmOGv1oK8LHn2qbK1Fb
YDRJkfGEzrAqw9g1hyIoIx5MjmuiHy13qujPQt5UsQTNRM4syXPJer3M2iDM02XGHbu7j0ezdDoe
aY+0/wAp9C8g10YLBduNBUAAAAABQQc0B7QLAiIiscyZBAZVaBNcr6SpLYesIPa9lqT0pedu0Fph
dYSmCX5aVkmUHDNHANHjDBysrHKRAo5gFARRZgc1zCgArmgrxFBRCRXMUHDSRysIV6xuGcNbKyOj
WGeMQVw1ozxiSPa1sEkSoFSrowQmTV1qhVnklR67EcLZZ7Go45g1pGjZlQQFEUYVJBRXCw1rmyCo
2RzRCXz1iI0Js11SaFijcom/draFJXsWS/Mer+fehZujtc7n4OvZ5xy+vkbMKuYEKgAAAAAAAAAA
A9WPgEGyOaAAIFnUxbqLvuzrK02lZKxhPoJTo02UGj3GVEdNu9i6ma7RkgnS4AAUAEUFHNewAACk
iKBDlQkUAhXMVhw1wKICuEJFBIFEJFAAABw0BwNAQAEABoAIoDWvQK8F+AXKoaufFefHNHZSxFbZ
LUGs6IqDDXNYAcEkkcqKqDIgQJEa5syigDgIhVaBauZs9Mbt7BvZ11q9FjJ6zzHuVSjuea+Ies+Q
9HFEBbSAAAAAAAAAAAAAArkQAEAVAFQAV8YFu1mOVOhvc/qwnNqNm5RFBXxywWtXL08j6NivbouG
yIOwHONcAqg8Zo4lRryRAWRBSYFAAcEADSoCwAAAACjADgBQVBQEa9AYjkBBUkBUgaObLILHEFch
JhpW6hXSguVnogjWO4ECZQAYFUEV5Cj2hDo3OCNsjSQAgAAaJMqgoEkb4i7ehvYXg9F4H6Uo1bPC
RYF2ny7mLlPpc0HBDUVAAAAAAAAAAAAAVzXAI8UjJCSNJEkYPAYSKBp0LaIlTZp1X5xPHbWsiSKS
6FC/lnTvZ+hzmeyRsWRtc3faOa5lV8blZyoooAwoigA6QAIAAUQkUEBQCABhVaqjlRrDxAFAAQaA
IAqCADQByoSytJVUbXVjKhPJMZ9TZnenk4Ovw3XKW+Q1Bb6E022oJSNFRhZI5FiUc6IhjlgeYwSZ
ewAAAAAJ680G5fqafJt1vofl/RqOjyHi/ofhXWz8qx7NWR6NIBHNmAAAAAAAAAAAAFkbIsqqLEgB
Akj1Gzz2iurJovdMuW+otarer4upm171S7MySOeUsW4LOUv38+9zlnY9K7q7J4+nc0crAo4YUAFQ
AAkURZVw0kcNCHAAA2RyscQogCqiA4QJUQBRAhRBgQaAggEjZFkilrRMVSWu6I9dN40vpHreftTr
b3zBic/V9f8Aj/kOw1Pk+V7T49piuliNSpWu1xKiWUdYHORkRUGrbFIhMLpHS0RYFWs2aNmaipMp
YgvqbfRc/wC0cPT6Di9LyGXreK+c+7eMdrmZiB0MoogKigIAAAAAAAAAAqKD3xuWXgQCgD5IJlNG
7RvRXbVq3VALMUmRv5nVgrXY5WnJIPXJZgmpS3coWsqXnQT5Zak623V0urpaktyK2YmyI7MbIwEU
AFRWVw0BwNkcAAAQ1zXAASAAKIAogCggKisYcxWrCgpL3sfW7ILDKlox3B0r+9eIfYetG+CdZ5rh
3VCfSrzu6W/c2YaHgf1P873aOXZJGlkUNqOSBJmwteOWu+YWMdJGtQHy15AlajFhkDmW2q5swN1o
NTJG79UeVem8TtTcD1lKNC+K+tfL+7HQz/S+F62DMVC6tQQAAAAAABUUEAAVHgqtVZkEIFVqgs0L
1nTvZtxK9Bay3U2CmrD6fW5XF6uQ6Rl1bHKMqTR36kSzLPirS6uphmjJsSZLcifVs7KcWDoc/orz
1PYy9OyukjXeMkQhByMI16SrRwAAwAEDXADXNAc10gqEAAABINcNDWuASRJEZki/TRHzJH9neU1J
4MzWycZ1301xGj1W8qvanoebT5ZN6PUtzc++W9q5x4L7x5AbvLknZTMCWCJrx241M+noUXy12zN0
LGDZZyxhMjBwRjnA7Qh2cqP7DnPcOVf6Lta3ld3RrJR3MGnzjz70PzVqtHzXa570HNaqS6KoxzQQ
AAAAAFBQaKgD2vAUeqg9IGorJZzogm7azJ0TSbRa9dpKETT7BBeXzHoefxOyy9WPkC/j7MWlqYWz
nTUsVpeYam5zupz79xcopv0FxmdOrZz6NTpVSZxHdcqAOAoIKSMHjDB4DEeAwVGRBVYaKAgpIgqA
CoAKgAqSCixD/eML1Nqeq3OH7+9eM1bVaXr8P6FYleMsanmdOjR8vwe0413oG/EdLnQ1pG6YxfMP
SOM1W+HpIYbGO0WI9KLZ6UPNqfqld83mE3RRTi5+p1FWU5c2KraayzqFZZgLmrlaGKd76f5b3PD0
vIaU3TNso163FqSfOXpPnF9GGgvc5qPVYlrHsmAVAAAFFBw1IkQJhz0WBysSFkSNsj0aksoig5zZ
FJHFisqw6EYdptcLb5PQ7RmP01GjFwu9osnndzcy78FmTOlozaNjHiR9+Hnq92npEw7WmdZaU0vM
NRokWNyTIrHQKqLEjgkGuUGD0Bg8aI1eEMSVGI2zNYjSRAYPaKgrhmCpKgRkfXr8Vz5+437FfRHP
X7nNQ+5mcRjJZV4P0zyDn6s/2vwzok53q/nHmXGaLfq658+6GWPasu4zvHz90ncfS+Y5HC7zmYO5
2fA8Kt/plvyqSvu+d4BMkfQeNzu1anD8Z9CYcR80Y326jHwXmfoVxLr8Xdl6HeyaPXNnw/veb0/R
+b0eN6WXx/l/TPKMt2BldnyOvPkrOzblQchETQdUUAQUAHNBBQEek0EavZCta5rMgAAKADgJorCE
9hljO7ElaGdCy1emj6hx3oPnNtmtoR49nKZ27l9CnnanQ52jBUFcVwRXEvupzujta3Yo2Ia2+GYl
j2osSvgeNK+JyzI5iqOBQACQFaEUco0egNR4xGSAQpKjRGkjWGtegsdezTVOr+lPnf6/ujQxqPNW
Ra4Trb43k/stLjEibj+YsZbOSuWcbHTJfwpR9DYydZLet0daDs0+u8DzONdmj5zpqeLn+X1e24/H
0K9iv002c7Bv86yPsV3mmxNRJr6izxzYT1zovnkvu+mOE67jcfTZ7do691OFyDvOLW4vAzpynHzN
LN35rbI7kTmIhbS1HtaEeANRUAciAqE4EivqRrZnQlKO1WssYAzCooDmuB1ivaqLksElEuhdBJX7
nF9J5IzpIrfktcebczwo5tyPsrnJNJ0tGXR6Ku9XPuu17a42WnOudJYglppqz7XsRqMrHsIiaSCQ
mV8UiNIMUl4iqKADhriVEUAAAUkRFSYajwI2yIEVO1Rev6iyu88bmznuboUqU63pPN7ax7t0Hz1u
3L11Sl1qHD8n9G8FS3jtu/USZNKho1XelbvIT9jHq8b3XL20SXIdLClXgPS8Gt/HbvY2Ez8dl+sY
jtxNnpUp2ZfS1x3K9ZluC02n6BTr7TGye9x9qrn95yfUx8P4/wDQdDPo+caktHXjz6c8OzOxzB0B
AFAAY5rQCAAqg67HYqSNwQjnMeiw071Ox4ALbQAFEUJJ60qFx9RalsRxuD0jvMjp/n+mvk6fLV6Y
LfMu7VHUU9BcOfm1scz19HUyY+no0x5tymZ681W0mVta7EWUnIux3yItzjJGvA6OQiSSKRWe5qoy
ua4lVaqigA4a4lRAFQAFABFASF9OVZWjz5zfZHnHf1rdnz7h+mcEtUVqvcpHWYH0lnawbdb93q+d
3EjcwOgmLuTZt5GciuZli1PTK/NbvXqucJ1DasHnmN6RwGcWfLs13ejeb6uRqKcO7UptymaUbLn1
dSrK7/1h4z67GvkuozulstrZeDwpL/C+x521MGNkWnMyCeK2tgoyoioADWhzQBByATxKpbWs6EsP
gkhJhZVqhq3IRs1JI7tIAAqKDlY4HPiVVlfE+D6K14LHyfZjcZ1fIdt8plk9Bm6fX5zr+DTz/Eek
8lobi97nT0NnR1MsrNC7k6NFFxFfTdTbYS1yQV5ja5uxoXiPEr43LMj41iXiCs5zFByoLLhAHjXA
ARKoAKhCEefNmFEGfJX0YvqjpvGPVr9/nnD9hwOWdAiEjTI7VTw2YrtMtsxspNHSwb0NvV6OxF3I
P1c9arHceN7+xO0zdqm9cNu/miZUfc5Npz8ezFVVAuhkq5maGjJzlXR6dm0eqv8AHR0NbpuK4HM+
v5r6149oSHkdbhNmWSSOPZmvUGpIKhKgDQ0ABFAFQAex0DXBEunqPI0LFCdKrFR1cSuxUt0AAAoA
Dga4VRXMUX6mGV/kmzn+Y3ud9I1Va69fNp93570GJNzmreDlXBzNCp6GKrnM0PZv5dnOmxPmz5i0
6FVsnrrXhp2RO23PFL1V0akSOhcNKsbllz2OJc5rq2UAlXNAcNdEgqBHVsUxaOZfzHyRMe27H6N7
V4D7w3S4Hzf1LzPLfempXYrlv5lgL8tK5W80kSVNcmrT5iW3RfW93F6CjZPA2akltG/Wr0Fr9bse
O6jv6vB5F6Drrq2eehz0dQYYX3ugzOHtjt/Ufn76qd3/ADl798etq9QuXpuXs4zivcJdCfPXEdZy
va5awzQ21DVSVa5riBoMAAIqKAAA5rlAAlrlfIyRUiCIaKgr2Zi2pUKK6NqDHl35a5wDo7NbctN6
N0lNuxS3ee8C2LhbnPdwryVZupRYZBTZLGVXpaarzs+ayqdGvi9JJGK62qc1ee5PQkRLULWFstil
bsusKx1tzI1qsth1OcLckMg71HDDkWuXDXK6qig4FiUFQGVLkUxlUdutFGLX2Kb5D6J+afarNO55
T7h5Bn35V+COc1yxE2B2xkakNcSCWpyelerJ7CSVSZexxgZkhM9DobtRbKtmKNksyMz7G7/n5aNe
fQp5sLt6h53QvPPqvdZCx0ODPRLQ2dndFx2S3vPnbI4brY6iPh6OBUakq4AhUFJYPQhEcMNBwNHK
og5xLCWaGgnu6KLju6SVY5KXrJ4fl7vS6FV3L3+okh+bs7zZXGp78CPn793oeVun6OrpMubWRuHk
8t5v6n5xrnmlzq/Tr1adOvcr67W6Sy+rMiXNjn+iytpM07HHu49nQ5OymGYlfMwmUKy2EsUdGj7G
U5qbvLboX5Ls8E5e57HEuEFl6scrOAWXKhEqg0BrmkMimbJTp6teEwux5uvbR9CcF2OVj6nCq23d
lgp28906DPsyIXLKFTVNOnPDas9OSpoMO21zJk1qSRHX1M9prVLlRa2se2EjkjJWJGudoPZPH/oi
rRu0KnV6reDta2c08TytKK6vjKO5qvTz3OWauzMrUfZWNHKNexxKq1QGSNAc61BWl1tSpufv9T0t
F3nlj0eur8dc3I7ko2JJ4eIloJMlrDrh1LObjI3I+bbYvSaHF3st3pV3zqHmbPW5PI5GSt3HhFvr
8D1ri86vU9TPu1tDxMkbYQEqOrHSOgbpZwkd5Y4fT5JrZ1etYl1c5bqtNcpRNiTLtRFwjdYR59+n
N7LlO5Oi/LXnL5XRqS9WIsyugesyq1Vd4EA1zSURUmGI6MJo5RXysvayXy+k9f457XTq81ob3ODR
TxyW0WNzPuVTLTy9wRNHOs57LeXHJD1dupvO2RS0aarBRuWro5tPX8Fk85Zo5uZ1RqOkCSMlOv8A
p7C6rW7KD9a9PIqPruFXf8v8X6pUpfp+F0PG9dMYO24mgAOagKCgo10SqDoNDqNTqsejmNPVfFk3
SYu1h0Ox9HlLVMmpmaqNeTDcF7OzatlWhQoU2jYblugtV4YnWy6m4jSlpa1MvlsSWUYCuM0RtdE8
jEjtd7BJlz0miB8s1CU4L9VisixaJlkrilkhWEndDYSJNGvs41aai4U5yluYfRZLudZufUsULa7L
D2PhyOVMlUCxthLs9OxdolWMWXsjjSucjVpexzbLpwRHo42xjNRm/QvzzraM3tXDeoed8zs5sds1
47cMMyRPbpyoTsp2Umrs156bZZ6erZZWp2QMeC/WK+17DxO/cntNPl+n008pw/vM7R8p9p7X21Ma
ngHb8Fou7q/h5dFl70fzFjT0/gWj4RZEKyLu58COR4GigAoIOCRoEL1PMetZ7+yfmU8uuXOx8i2n
r9TzjSy2dbzBgwMgz49ubXTFGjTqV7QsNq9qIuVY23OuDU6SgJzSaNJRbubILupjtdplrLTEkSo0
wxyR2OgjbCe1SsVLckryZ0krPjaK9exHfZCDbWkdC5VtXKN+hru3j38NnSyUNPhpSyOrrGXz2v1/
J94tX8rQ0bb09axQ1xg/l5s2vfz78y2qEtl1xI1RUQBJJopX3PY9NGqUVEtzsXaxXzZszZtOT0/p
PHvfef1/P6m9gyrXvmelsRcSH17NtLIWxqPcu07kWvgTER9SOntSvOZfb46V5N+vTdN/qfO4HT37
v/HbGxPJuk470HnbYb3k/e3J9bfJOP4v3udcqNdKCMCBzSYEAAUAABAA3O45R2a/oKGBXU0qlNjJ
bfQA0469hGrw30sqots07a5NHIu1z0uvgbNRoNbHprjz7VGqc+jdpUNBFJDeNa5bLLAwqrc1CRY3
hMTZmuzZoXQWlrCJMV5hJWSCTWgtwXPGPRhblORJ29TE2OfboWa9fLd3F6j0HnuVzPE+n8Vsjj7N
2r3ttlazETTkylz4L9MGSG0uoPFNpyomHHtUZupuVl3UeIXXTIgllDI2s1q8uw+xYjO05KWH988x
158O3BtOlvzSl/QqtpV7kCPkVdbNFsvotVpMixWEo2xXXorHJjN1NKhfBtfW0En2Pb5LoNdPiPVe
LPS7tsrl9uU8krM1evz855OTRR72SBWuaERUAABUAB7LUE8Mr6iC67UWqma0TV4VfQosOlrvrmzH
BG4lZzHubLErNt7HJ6FK9QmC96NWtUbVK05Iq3jjkjsZtpt9L8probaXkcrNJM2/neNb78xi19ul
cmZHbqahZ60otxY30Q2GVHeulhrxC56kWNrF0shssp2sz+i7eRf87zTh+h5JwoaMe3Rz1TXw+osk
tGa6nQsZ97NGrsY2tmNiSm4RudbhSym67dq6OUzp0bTycG5lb7KFO5Xsas5y21DxXU9M8ybU3pU3
nVCm7tuZzbF1ep3HIdBm1aOO+xz9FOC9C9VGHXr3pTimjeutXuV2ShSu5dyaff8AAfRNdvsXzh7F
87xZ3uJPw9VmBBpcZty4bpq/RyCJGyKxyNCAoCPYAAAAE+5oOqzZ0dynOcu51ik04I4mmpTnrvaN
c2LUjkjeY2SNl2CuZhUapNJXAuvoyIlsgnRUHSQ01yvaz7OeY9NuZs62K7DQqzZbdCSq7KpTsQ21
Z9TTg2RnOkbdEssMtaqDlZySCEViG+ybepJoGPMuau5zabUnRO8lPlvCex+W9XRJZw9/XMGX1Nep
PP4eo5ftvLczpHTa0OdsZk6R3O3M9e5YzNDEaGvkaPPu2oI4H6Gdz+vz/d01Ynx7HYqPetRIWSSK
OuJLHVcLf38Ttadd/p49rNt5LhPZ+HSuLoPLdvNfs19is7YmfsZ2nJBVuU3rqZmlRsq6f638l3S+
HhN/zujRdv4/mttXpPjHacXrz3c2aPRngJ4bUaikwgKCCoACgt6lq1t1OPRKsstdkbVyTVHrNyOF
FEY9XIyRjNFG+B7BBr2uGgOGuWWjmkOVAmaw3SpK1m/Jmeo99WLsWwy5poJ3T5r4lkVHaquWI2zE
VQMtNlM2rp1NKVZB1sOkLVRXS5Gq01I7I7Da84nXP6T0/h+9jy/Rmt5z3mKlOI9Dp1XfPsfrfnqb
9BaepzE5vivReL7S89LFL3CSeOznrl0quhiqtXq0/OS4uZVjRuQc/X379WrXl6WlGSRDtayvNMsN
eAzz1WstpW3BqRo2/QOY7bD1dnQpTFjsHQ5+yvksDez6Ug6jloct3aZ3OV713qlKa3PX6XlfoiLe
o+fPov5R0HuXlWdzaFqpp1Xrz8XQpas7InsvRIrTCIAHQHADXAMVALOvBYx3Vopaz51YkdiSEShY
WJYiZYmizQxxO6xiPaAAKig4BZAUC3JtZ7o9aa5zNNOG7lBXyZ8vfRo2pJKXa94ksJEiRVcDCRFi
GJ9dqWVpYtNELlSwuaOfsV0yR6gtXN5/T5Ma8ZLtS4roR3r0Hsfz7dsx/Wdn5973Hk7DlbGL5++t
br6nMuxea7Tk9Jxxqxd2mvYllqV16tLlS1DDRjQ/LjyOppty4su27pLPP3KX1oKcA9itWhemdIJX
qmkhmWuzv5vWU7dTqKPQ5eosdupYZvOdBn3Z+Tu7+mjctkdzz/J05mbendOOp97UvqvfRvlvqtr1
Pmro+amcqz1fL6KL9BlCuOk4PouE0VLLX19NVTNUsrQHTAoqy0VkhJHZI2mWocN1Otdq2U1opoNC
itWSQYgSRowlzQaQAAFAcCgr9FXpamlrc7Vn6Vp2HQ2BmM6yZENDpY7FdI9dfYKpy9YAABI4Y+By
LGjV6tipdlYj220tJAHaeRYKunnwLtVNzOsRV9LNpatWxsmLQi0JVdZeJBddYpS5s89rZKfRukw+
mxYM3i+95K7PwsetCtNVt6Gpq0U1PRpjoyUNnSqUrNXfdCOLiSWsqLZSJYFCQFehXEl2vpq+z2fP
95j6Wxtsv5NVOjr19NeMzUj0JQsNrVOvnvcPxvyEjudy3bT832NYrb0FHo1eJcj1V66jX5XrOzeP
B+4jtUnleBer9XFPDayZVHBYgCwCtJBo6YTZyOroda9ytiuzYbNPTnjrvj0QDXSKg1hAAAAAAAUC
aToKbK+7b0OVur2CpjunoV87TSY8uZ0skTBNlCoAdqinI3IqAAICq0gkjWJZhrzMuzRrMspA6Z0P
XWUJLVeassPjeWx1bVeSoyaG0dJHIq2bEF6vK2wk6ZOs67gL3DbsOTiqaskEb0v5TI5K6EFGXP6W
6CjJT2dJlSSO29qvHBVVRBxEEkblHzw3EWxtZ/W0bNjv8TtMPSlmkjeIa6V7VsZ78RWZj5dwXql1
eNxW8pzFjmdlXbe+eNe2O0FTTwQ8a7nM9PqPAc7T5jdk09HGyUnJu59jdlSq5ro4a8Ea5swAgDmu
DU2+b1sdmjWdHlM2ndz91LGqlzNAYQAAAAAAJgj0ZumzXVegvaODZSgs89msTMgh1ZpYJ4rkoZmn
ma6YgLqwAO1A4vQaKjqA0ARIHRvCIiVSI3zy1TTWdAiVyMw8mlWuvSvGVBr04XLr3qzxDaZfWl9i
WQ5zJhtCaFzJXladGtSsNFxyS05KtHUq304OVv4PZ05cc0uzdXLjVmlHagtGo5WhiK4GuFB97Pv1
G73XKd7j6XV9Hj6Vem5WM91qY8nLumvhZeI6W/YPM/YKnz/Le38vzaMnv8X2KuTQwqt5xW9Psunp
Hg2R2l1GfU82fVOXB1HA7s0KubpzMBGhVaoAjwaioD5bFmt8q7DBKbi5VhEkpWY5SmyaF7gHMMHt
BAUEdJrJNPYuamPRW3a1nPdrx5mdFi803HvzXrlDTeJUkak5+Vr496VANVAo4O1WS3wdtKPWSqMg
1Ecy01IAzyxHa5YfLTMaSIjQssROsSvbYTaNfbtpWTXlnn8xn9Nh0LiVtKsaKlyKwU6D4JDJBHNR
1V24K7cd9mzmyq/QW+dtYo2oaC1lbG1s3o2ZJJD0bpGRxs6VRL0fI2eEYliRYprbfC1+gqd/n02e
xOix9eOWVulamdcw3inzGjzwmPgXsPTm9w7jzPoOfu5jjdfr6bOj0eK7GnTgs06ltTvIKqb8fovK
59aF6Ghp8DYmQ1zd2JRkskIEwAArmuBr26Kts1JmYrsyls52mqtZq3HqnSezTGPFrUbCoSK8xk0k
Fa1o7FNudq3rGa+GdlVHkqMpzD8pmVpqcQ2NFejcoPzXX6tOjK2KaJppRXOaEkfIk9xp52/5TSi6
r+amFX3KF9eZVu09i0pJnXaIxYi8K8tgrZGw0LUS9dToua6Bs3QyU5jk0MbWzs+fKr7EKb8l2gJo
hLJFNCnpZ+2qjDJV3VzS01lbaUK1d20/myX36udOOsE0Nz145IrmHtcyy2K01Ndt8dhKlkLlLanq
mN6LzuzJflnbRRraOfcuVz3RcsHN850fLaaMmjoNsp1Om5X3fnbM650Pm72cl3uFh0aqzMep0MPQ
0KeBWulhUbPRxR17bHWuIOjXNCAAAAAAH9FSt5bHOY+tm52nUdceVrdFetZy7eZFrrXuGta53kuV
rNS7e5jbWbZbpS5iOhFK0Mzr+RK52dNFuoJYySwyJkDmoroABJNHYSVSVCO46Hnt7yWrcmpzcWiD
M0M3TTmQ6FzcZZ0bKtnJ5fUc10bamhQv3kkSwK0KwP0Lc2+cvlXTWMHSr49x8lwx0q+1Vz7MiK5T
xbWVpMy5n5jM/pIlRtTpJpTZly2mapPCsUY54n0OtN1Kh1vT0q6OSyO+wJs5cnLbmWS2iJYdPUyd
5i+wYt9/fg08m1YJ6euqvTkzKrKXOXOatShzVzI1ZieOw6db6x5/p8PpMr5m7Gzn8D1PxLXnz+sq
23p4SHZh2ZOdu1ptNSVJarI1r4ramgTAAAAC2Yd+qZISPLD2Q1bDQhpscSETQPfCBKrtyqc13Sw0
vza2M7RX0Gry+rnu24I3V2KraIJiXKNyV2T1762iDIAAAArmuCSxBOpIorJ3GplX/IbNWTOs4s07
pNChKulPdrmGHTjo18fy/d8p1n5wtVOrJUnpXy18clpPar2Kc9/Xx9Gnl9BfxLevHo0K8Od4c6xm
YOlVy5cbquUGR9QeLNMRyrHCWHpMtVJl+ItNjM1cr9Rv4HV04c7neywbI8+i3s9+jDYdI9heh7em
7f73K2ef0LLqcF6acOfG0WMKfna5yeV0uR10IyO1bTLIx0z6N6h8/wDp3G6Nnm+gxat3G8K7X7PK
sV7JXGTU2OWvpjgt5mvNLErHS1Vs1RgBkAAAeF7QigxvNXhjszkKsucQGYAAeOg2um5rpcOnWzb+
Qj87h6uZtzTa2fr1vclSCp481cu1LLY3OrK6ttRBSYRQBBQEUcDrEVhSR4rJ2Nyjd8nff0s/U51d
+7Vl5y6EuasRow1Ka3R89oYvVszaM9Tt3R1nmlxw0i1PSmqw6E2WqZtmTn47Y6Cbm7iValMgzac/
M0KPTapFah2I2eCxA1k6qJKllEjissR27FTfyvodRn7WfBFidFkNm5bP3s+zZlt0Nm3oP9Qr9Jm6
7ZmtsHV5o2Wnz23y1iZ+XUwb6DHkYySW68pElyp6xRfNSsbvK63Dcn0XDb82ps4+ZZTq8nZ53Tm0
syObRnSrKOsTVkZZak0IAEwAAWqttSWAjQcxBxRFBGPawWo92l4E35cl/L6E9B00KVSs8RxtNFWn
p5F/Pbepx0YmvWG66XjCYAAVFQFQAUQBXNcEtmGypIo506W7nWvMatnQw7mBNlmYuerVly5K4vUY
6VtiZLqPV0JWmra7CRR7Y4p4ZSN0CXY7DYWCypAtjWrlC3npvxtiorp0rFDe0iMt2rVmsKthYfYz
LXbcbCwWX3KyToM/dzJqX6s8c+HF1MOKKdWWdNmz3Mu4vp2yzQabq0E8GlGNjrVj+a2+apON5fou
U6ONZa9iVsWqPs1MwX/ROH5/Tu4Njknu886Stna8PL9Pz/oFlOFwvS8zpqszQRXU2atio0NmiVoG
gQAAAASoQKgQMRRhUQBXNcpo7+FvYtWrJWZntbm3s26rPoXMzXQiIXJYnqSVtLWdDMCorwgqAAAA
AAAAAKjgntUralhrGTV//8QAMxAAAQQCAQMDAwMFAAEFAQAAAAECAwQREgUGEBMgITAUMUAVIjIH
IyRBUBYlMzQ1Qib/2gAIAQEAAQUCMmf+Rn41FFF9OfXkz+dnv/v5c+jbsv8A23CjhfhX0p2T8DPb
JntkyZMmflz6M9tjJkz2yZ/7aiqOcZNjc2NjY3Nzc2Nzc2NhFE/DyZM/OvbPbPfPbJkz/wBfBgwY
MdsCoPHqOebnkNzc3NzcV5ubm5ubm4ijXDV+Bfh2M/Hn0IYMD0FU2NvwNPRjtjvgwYMd8GDBgwYM
GDBgwYMGpgwYMGpgwYMGDHpx2x63EhKo94rzyHkNzyG5sbCuNzc3NxrxrhqjVE9ai+lRfVgwai+n
HfAgidnoSGwjxHH3+TQbGaGDBgwYMdsejH5WDBgwampqampqampqamg9CVCYlf77GxsZEEQ1NRyC
+lijBqDUMGpqYNTAo7vkVwq98GDU0NTHZ3ox3wYEE7SE3fI1w34MCR5GxCMNDH4uO2DBgwampqaG
hoaGhoaGhoaGhoamhoaGhoaGhoaHjHxksRYjJmY9CINQawSM8Q6MkYKnojIiJo1hoaGhoKgoovfP
oRDU1NTUwYMd0QwYMGDBjsnZ6E3oRRq+nHZoxqDWp3z6cGDBgwYMGDBg1NTQ0NDQ0NDQ0NDQ0NDQ
8ZoaGh4zQ8Z4zxnjPGeM8Z4zxniPEeI8R4jxHjPEeMdCSwFiAsVx7MKvZEGNGMGxHiHxkrB6eiIh
aQsGMNDUwKPHjhymfUghkyZMmRVM9k9Ge6jFEJMYselrjY2MmRVMjXDXm55jym55DQ0NDU1NDQ1N
DQ0NDQ0NDQ0PGaGh4zxnjPGeI8Z4zQ8Z4zxnjPGeM0NDQ0NDQ0NUNTBg1NUNDQ0NDQ0HxliMtRFm
Pu1CNCMZ2eTNJU98GO0JERuGvNzc3HOHvHvHO747L6Njc3NzcRxsbGwjjc3Nzb3Rez1GPElHykr8
+rY2NzY37Io1RPToaGhoaGhqaGpqY7YQwY7ampoaIampgx6MmTJk2Njc3Njc3Nzc3NzY3Nzc3Nzc
3Njc3Hu9pyyWRwiDUIxgxxsPUmUk+/dhGox4kh5jzCzDph0g55sZE9DnGe2wrzynkNxq9lNjbsps
Nd7t7SG55h0uflQQT0YMmTJsbGxsbmxubm5ubm5ubm5ubm55DyHkPIeU8h5TynlPKeU8p5TynmPM
eY8p5TynkPIeQ8huI83Nzc8h5DyiykzydSyKgiDUGjVGuNx7yVw/0NGqI88p5jzCzHmPIbGwiiGw
rhzzOV7OUe4z2jGIaD0HDPcaK0VoxPdv2J1/AQaJ6P8Ae5ubm5ubnkPIeQ8h5TynlPMeY8x5jzHm
PMeY8x5jzHmPMeY8x5DyHkPIbm55Dc3Nzc3NzY2Nu6enIrhXD1JSZg5hjs1o1vZR48X0IIvdRVM9
89kNjcc8V4zu8Xsn3jYNaYHj0GDezj/cPH2Zm0+kuSvMsdC8lvb6U5KrJLXkgd8qDRonbHbc3Nzc
3Nzc3Nzc3Nzc3PIeQ8p5jzHmPMeY8x5jym5uZ9WpoaGgjDQ0PGaHjNDQSM0PGaGh4zQ0FYOYSMJY
h8J4hsYkZoaDmD0Hi+hBDPowamgkZoKKorhz+zO7+7EGNGijyTsjjchqz214ToWd80lavCrrH05u
shNZVhHTZZfzfS1OyX+HsUfmRRrzc2/KQQRBEMCIIhgwampqYMGDBgx2wYETvj0qKPaPYPYaGgjD
U1HoSD17r3RRF74NRGmhqKPHqPd2aNURTI5fcQYg1PbIqjh/2ON4+XlLfG/01psr8lfh4itFyivr
cJyVl/JcvcrsRlxI6/KdUea1HZbBT5HrxI3WeZhu0LEaRv8AjybG4jxHm/5CCCCd0+LJnvk2EUyb
GxsbGwrhyijhw5pqamO8hKpKvwJ2ROyJ3cPJDHdqmwrzPZhH2coruz/txfCWOWscHxVbhG9S8u6H
jJbk1lsU8sUlKGe1LZbRgZzHVCNgffe61c6it24XP27QS4LESRO+ZFGmTJntkz689s/EhkRRHG5s
bGxsbmxubmxsbG5ubGxuI83Nzc3Nzc3FcbGwqii9smRVHqSqSL607NEUTtubj3j3DvTkz2T3Vgzs
5xntxfGv5S7z0/8A47BwMsd2r1VT4+au9vifVmg3t9V/4jbsss916ul9LW7itVF74+JjBrDJkyZM
mfXkz8mTJsbm5ubmxubm5ubm5ubm5ubm55DyHlPKeQ8h5Dc3FebGxsZNjY2Mj1JR/wAKONhHm4rz
cc/Iq99RWCp6I2DGiIPUVfdgh0JQbWpdaWZLljpqaSnSv8pUvvvaNsfDGur/APGussxtinMfCjRr
BjBGGe2TJkz2yZMmTPpz8OTJkybGxsbGxsbGxsbm5ubm5ubm5ubm5ubm5sbGxuK88hubmxk2HuJF
Hr8OTY3NzyG5uZ7Ia5NMjojxKK1UETKsZ7RtFJVHqRHA8K/mLNx8XD8NS42HkaUdqrQh6nuUrkjn
bu+LODOV+FEGsEYNb2z+Nn0Z9GTYyZNjY2MmxsbGxsbG5sbGTY2NjY2MmRXCvFkPIeQ3Nzce4kUf
6sme2TJsZF9LRoiGBzR6DIxqDE9nrgcPblWNwdH01bxXJ1fDVn5K0yvyXMvFVXfK34WjGjGdk7Io
i/8AIz8GTJntns5RyjlFeeQ8hueQc8c4X4M/EjhrxrzcUVMq1BvZ54jw/u43jFv2a0n6bVvdRusc
jf6vghfy08dqx+MwYNTsnZFEURTJubGf+Jkz6s9sme6j0HtHNFN/fc3FebGTPxJ68mRJRJTyjXZE
GIYPGJF7MhQ6b4jxt5zlUhitcT/gczUmjl/HY4jcI7snbYRxuK83NxrhF/6KoOaOYOYPYL+WwYRi
dtkKDPNPc5dKEfmgY/muulWO1yMtv8hBjxjxHjV7bG5ueQV4knux4xRP+krBzR7R6fKgnytUYox4
xxuPmOnY0ZBNRtc9yHIwfp1GxJ5ZfyUUa8ZKMeZFebG5ubdo3DFG/wDTcPHi/lIpuRvGvHSleJ92
1V6eZxtJ6RcPB1NzWWvXLvy0dgjeMf64xg0T/oqOHDx35jBO3QdJJua5zlI22JoJZXcxbWS3+a1R
jxU9TBgwT/oKo5RyijvcVByfDj0YMGPjQjYI00OiYYqPGX2/X8vyXKvrFqZZ5/w8GDHbHfBoaKeM
YwcwVvdOzSMYN/56jlHKKvbUc0ewVhqYNTUwY9SGOy/Ez7xDUOH42Tkr7KH01fjOL+lg6tubSOzn
8jA1okYkR4TxGg9BzBzeydmjBgz/AJ7hyi9kQ1PC55+lWntm4a5DG6LB4zQ0NRU9SL2X44/vCNQ/
p3B4V5pzYjket6lReY5n9Qmc7b8lBjRGCMNDU1FQcg5PfsgiDBgzsov/ACcdlHC9kQ4bgrfOWOK/
p3xnFpnjuNVOXpkt2nIzqHozjuZTkeNl421qaioKg5DBj4MGpj1RfyhKFb6y1QrtonVL33KnKcD9
NG9MO/JaMGiehRyDkF7NGjRg3uv/AClHDuyIU6klyxwnDVulOJl56KWCbmJZU/UZ0fJy9t5xnO2a
0vWHj5Gv3cKhgwY9WDBgVPVChGdG8d4XMjc5/Ly+VOo7aOr2aMtb8powb2XvkUcgrTUQaINUaJ3w
amP+Qo5DQ1P6XcN5rnU95K5y1x7139m/eGh5XV+Pj0XjUfx80fil7KL8q+hqZI2nDUH8legrRxv5
idIy3XSaSDj6sbec5GrY5C2rHzfkNUYNE9GfUgg0ao3vg0NDxmn/ABlFaYEacJG3pvpfqK3MlqZV
eNwVqyPbUrposSFdP8fqCPTkvRjs4XvkyZ7bCuFXvqRRjWHS1P8ATafDVHXW31Z5oayeTn3rSowV
HzPZ0y5kV6JIZvx0+7BHG4shueQkhx3waGvZrRrBrBjBsYkR4RIBID6cfCPjFT/i47dLcUvMc11I
qack98tqLh1mgXg0akdIZEjWtT9zP49VNf8AWmDBgwKOF7ZM9smTYz3YRx5GMwdN8P8ArHKP4XyJ
5IqFOzZS4SSPgh5q1NfdcbJxrn8899eeVZpfx0Ua43NxXivPKPhyS1hYdO2ezRjRrBGETBkQkJ4h
sYkaCtJEJR//AAMDWZOlelYOOqX+D4rko+V/pvoy7x1njpFOhaacdwXI3I4yjwtnm7kNSpXS7PTq
SrWSxDlUc2NquZ7r1hC/btg1NRzR6DkNDQ19ODBgijImDWnRXDSaciiUqHJXFZDwMbmRX6y24bkU
nHnN8l9Y5bbWw9v9fjIojjYVxsbGmR0RLCSxYM4MkZGN7QoRoN7ZNxXkjiRw789qHAdCQtir06/H
IuWEMnjllvMQsxN5Cp/4NwnnYr+Wjl6boVjlOoHSxP5SSza42Nbc0dKGtducQ+J01X2iZ/d6zrPY
nbBg1FQegrDQ0FhHwiswampqYEaRIMKteSy7pv8AwWWubW1PHWfKxKyNa6VKzuYuNuRci/8Au98f
C78HUVpqMVOz2ksRPBgdlqwuIlGuNiN4yY84tkWyfUi2B846TP8AwOhOGbPZsy3X3On7tma1Lr49
4YmLBTuySwo9300j4JLlXgGcxZkkdyl2SzJ05w+08lotqs1uW2rC5mtXi/fN1dXWeHtWpy23zV3w
SMpTSRQ9HcjPSl6G5OOjxPRHJcscjw8/F3H0pYTwj4R7BWGDHdjiNffprhpIIrNdEoM4/wCpezat
DyHOpXdz1lUp1OcR7eW97XZPmTvn4EEMGpoRWVQZPsm2TTYkr5J6Z4VYRqojzcSQSwLZPqz6k855
M/8AC4WotajLiODgKn01Wd7Wskr7TwcfBm3cg41l27IwucqnguPXetG2xNUsxMdy3WlLgWwf1TSW
xU6oo3FdyUE54WVL/Oo+6zXC8PwFzm39G9JJwlPleg15nmoODpV6y6tFVrxVjrx1eJp3bl/p+ly9
iborjn05ugKX0dr+lK+Gx/TexGtno3kIXr07yObPH2Kb/dBHnTVf6vluP5Z1U47nWNltNZWiu2NK
1lLD+a6ksvtTMa6EuPV7vy0GmOyiSDLWpWtbrG3KeEmgJYB8epk3NjdRz1N1GuGuEURf+Bkm6pq8
ZxnTfUKdRz4JmKqwVGwNudS1qssvMOjfLTs8nPy/S3ISNnk1dQe/9Qrcq/i7fMQzcktapNNNUoS1
hjJ4Cpb+piu6+XiegbnJ2eN46p09RudTVoIYesmMrWOvnNk5HrxbT2dcywz3evJbTGdW2mQ8X1nY
py1v6iJI/wD87iV8PW1SZJ+p61qxBPVfBT+kkW3wVOy650Hx9lvL/wBP42v6b4GPgWP5lL1vgaax
zTbWU8b5peqLCUTwrO98bULyt2RBRPlT5GDPR5VFeqnHyL5av2RhNH7SMJIyWPBnvqadkGqJ6s/g
47Y9SjlP38pxX9P+Pio8DLLrHa5X6M5CGXn4b8VyGGk+CtJw12qlPnuuvpuW6n+mkFY+OW1JpFDZ
lY+HkZFZJIqyRRo9IoXMZh7ZeGfNNx/NzzxzywSWXcjVWBJGrnXtDC6ZZmOidn2Y5c+RRJXH1SsG
c/aYyt1JarEPXr421uvWPZS5hnN3f6gciiNg0fR4Gz9HRuW8RXOU+gqc1zr+WlkseNsliR4pC0lZ
j5UTtg1FT4Wjey94YVkXj6OroIcNT2H+6TMHD2ZJIDXHfBqaiDRO2PzFHnRvC2eW5RslVY0tVWvZ
PxfHxupxyyVuRkay3xNDlG9Q8XJTZLD4ZbOLDJXpGbeFZMxu2agjPKtPJx8LLdSKqsNm5zUvGkl5
bYyxollFnWzxi4kyxThP/d5r/wCSvo8SuPG4VjjxOEapwXG/o/Bc3T/VOQ4Wrxv0s9aOtE9Vmn68
5Vsddnsit8xNhj/97kip4/ja30YHIL60GDTJkyRx7vo0kRK9fCxM9nNJVH/uJGGDQfDkdHp21NRW
GBBBPzVHnTbE43pbk+QkSO5zcjHu5WxIkXMSskrdU2IqlXqKN8tm3Byc01OxXmZwMnlvcPYoq+DV
ZGatRJFV86RlXTSm9WorGIcrC+WhSYx8aUUV30yj62zOS4r3dA5q0lfDJdZJYd9M5RlJyr+nqQ8U
riHjmRNkoRyL+mtJqbEZwnC/WXrTvqV4igtmrap6cWnJz/S2rEtevJx9rkpuTh+llWTRknt8zUyu
Oyd3DvgQRxkz246ts+tXwjIsGdUdMhL7jhVyiiGo+LI+HvqOZ3z+a5R6lpiUuL5iaSRLj8SMzsxR
E94ZnwPqcgsTmcs9jON5X6ZiPjbS5jgWNWeusZ9GhXhZ5Wt8L+OsorX2tFmy4+gWObzfTEN+J6TW
Yya5G5JIY3kcMaJNxTkg0jaiyMav1LBl9jR/ItefW6OfypHY+rnq8rHNV5fyV+D45WRUbHHawX1a
8+lWZkyU+n6d+3+pX5nIiPyovysTX1OH/Ahk278ZXIovb7FibA6yqKybdFTI9g9dRkmROz0Hs7IK
g5PznKPeMcizctMk9+7XR68hC5is9hv7Ux+7dSJyqMXZyTL5K3KyNts5WOZ9msj2PpPqp7n3WGVz
X+ZXJxVhko+wnmXV7b0D4nz3pERbjxl9T9Qexf8Ay6J1CW4rh1l551UWZx5nCzOFkU6ZjW3zFOOp
wtLm9ncjwcSzLb5hbatfHXi5v+oUbDkuQtcrO32Jl9l+VDPdDHd6fLQh9ms9pvYsyErytNh0f7ke
wsx+zpVifBY2N/Zzx7jI3s5BUEMGOy/jqo9w94+Q4udLk99qqczWV93w/tTAzKD2fs1XLRj9l39/
dEhs6oiluskiezj3ZI9XHEySfUXJmSzMn93avLdJlptnjZI3R1nKfSuc65wr61eKq55+nK9H8Y9h
9C4+jVB9JwtdT+nlCKBzosWeOg+rme1aNdkKynUsr5asD6nj5N8ai/dfv7jk+XJsIvZi5Nezx33+
OpHhP9WV9rP3VBEwtN2zdSdnteiwRyKxWWco6UfKI4Z2VDBgTtj8h7h7x7xXH9Pbr38Gsrduq5Y2
8u2REV6ZGfuVqo5/8DXUwZ1IRj1xBKrB70Vtyq1SRMO//VaXwu4+pHar2qElJWZVXLo5Y2ysk4xW
DaauW3FI9lasg+ujBiMw+q0lrIeFMPoJIdOt/wDH6EFhl3iuLXNzn7kbFuclambzc74p5HxW0mWL
dXbOZ95JGoKu3z5GuEEcKPHfJC32f9rbib79qD8KiZSaMvQkjVa5r8HkNhpGonbBgT1p+Erh7x7x
7u39Pr6si4SVs/P9azOnurNo5qoy1Ev+OxybKwRD7uxqjGCplzHqMmQa5EJ4cPm1xyVlYV6e5GSM
u8l5G13xTDqPmZWasU8kOWq9Yl82wmGqrfKzVWjZlaqQ+Rs6Kxel6yz2OP46Bz+p5mVadZUpJyd/
9e5X6+Kgc85vIX+RnbTrdl/CRRrzY3Hu+WMl+1pxMZEUrv1fC9FZM8su9rKe+OyCEbhHGeydlX8d
R6kij3C9uhfflem7TYud6kjfNcXLqDGo6Sv/AHWV/wC5Why9iP8AdP2p/E99mmUQ+4yRMP8A3pP/
AD5Bf8rjZlhltzKpUtrC+hzaMbauMxT5OOZttR9zRfLu1lxYTy+Y+7q0qRtuTRvd0zB+ncBFX/T+
At9QRxczy9+zLUrcakDbjJNqtCTy9ScetNfxsmfXg1EiPEeFRtVyn0L1GKTv9p1yTdmjStZ/bLMT
TEzs+jIxw14jzY2MiqZ9OTJn53DyQenfpGz9Nz/Tq/8A9J1J/wDNh92RSMbXZ/YmT+zyDv7NiN27
mfdiZhxs5r8jf5fzRcuSPEkdn90tld5qzP3v/ln9yyOaRTPmiZYdDLXufUxcgz91a2pN+8rSK2R9
b9t7kJGP6cqTc5zGI5bPUPOQN4HhYvrOTkh+rgsTI0iptc2Ph4qqdS8q3leRaOUX8JGmpj0IxVEh
U+mUSooymolQ+nErjKmy1uNV5Fw41+CxISKTfc1PsLNqSXMElpVPIZz6ciOEeIpkVRF7J3yZMifM
o9B7BzBWCtKFh1S9FfdxfJdSVfDcsJ4pHp/jW1xPacjCz+5HNxyj/wC09I0cTJoKv7mIqCZyiezW
uxyVlK0Ssw1jfZBw4rSKx1xv7qU6skmw+P2Yv1Aywvlh5CNYeWVHz9Ftl4fhHtvRcP1bDYnqcJ09
9DE+OKCtVrIszpoK0fWPWLubVyiKKov4KCIaHjU8Cjag2mMpjaYygJSG00HVkPCMrFWj71aaIMgQ
lpqT1nkqK0ncI8R4ryRxI8298jXmTB4zTsgnbPdvdyiuNhFE+dRzRzBzB7B6FG4svFcvWndBybHK
PakjXtVeNf8A5NeDNipZVXtryt5CGllUdMqyxO0a33Ge57bK9GNs+8v83s/kjR6e7k9k+78OjxhY
bieGZRZsCvHTvxxdB3KWqFN12Xlep7S8vI2DkbfHctTms8x01I05XlU4eHmeXsci+Rdlen4ODUbC
pHWGVBtQSsfRiVhlYZXRBIENMGEPYkGMyV6xBEhD7CPP5EkSF6qmLjFYu55RZh8o+TJsZ7MUhYNh
JKo+NWqIaqaGhgQyZHOHL75GDfwMCtHNHsHsOmrKZpwyWal+BHNh/ayqxWs41f8AHpy/5yf5RQ/t
2JnLXtcjC9xAqWWMeujMG2pK9FbH/lL4tBFRGtapL/L/AEbL2+wrvbsp0Bx8U5e5NtbieJng5jiO
arW+Lj4joz9PodTcyvFs6ltrfe9iue+P+7NTSCFfv8eBGHiVRlZVGUlEpDawysMhEiGwniFbgzhU
eeUWYfZPMeYhf7wyphllEVt1EPrkGTIqPkLb/bkMKSrhfKK8cvdDIxcLWI/t/qZEUcnuZ9Cp2UUc
IMG/hYHsHsIZnVLENmGG3ytVEsakb1W/Bj6hWa80z+1csp+6xixTpy+YkhWF0FhqkUTHCvfiaFMe
8o9qK1WYGNRhZ7OPsZFP/wAuP9YVzuO4t/F8Vy3ES8hxnSVTwnUss0N+91BKknKwq7qTm+Xitvkm
SVknCrSpTTunX40Gx5I62RlUgp5IOOFoIOpI0WHCuwgiiOFfhHzDpT6nB9XkfZHzn1J9QR3D9Swf
qgnKn6mpx/UT2kfLNmbauFmXYmF7KYMd0K02iwzIqOeSPHuMmTc3NjPdw8RBg38JqD2j0JUOBuJZ
4jl/82K4xEljcr7US+Odif8AqvJLiGxHmDjHpJUfUXw0Ztmss6vkia1fpkRWww5ZC56W2H2UmRcI
/wDc8/0gp/pe3SnH/qHNScW+OvbpV+Ip105VYblW1L1BdgRXzXorLp41YvQ/B1LFXrnnIOQtfJVr
LM6GmR1RlUrV8EUOGujLPsPVCR/vub4SWYfKPlHTHnHWB859SLYUbOokyiZcMapgwRzSRH1ivR8g
9cimO2O2pqY7R2VYJcyLYyLKeU8p5RJBJRJO7x3Zg1fTkyZM/EwcPJjpy4lbkKsn1NfkoFF/cmN+
RlzFaliZmvJ9RDTZJ9ZBMsNixCtedbMVlkkDEjbUgmghqxm+Em/e5zR3s6vh7mcNWniudJvQtcba
rKZ7exjK/wBOeH/T+KucklG029dv3pvPfYz9PqP57inui4zgbCxU+E8snL8rWo8c77/GxMrxVDWJ
tNEPEjTKEUiEcqYmsoiWrRJbHWBJh05JJlXSD5hXqe5oosSiswY7MaRMGtMGDB9jPox2RBGGhoKn
wINQjjEYObgeo5w1Rg1RPQpk2EE+Fg4kJhF9+luVZbhu1vNHLDrLuqS3E/szZfFTfhyqkF2zD5Ur
WUkEigst2fGOjjajHoIzYVovusmdo3Kx/H8ijiCz5VcsbyfgadxLPRLHFzpu5UHMcx/SnBy8zytu
NvHQc31La5WfpWFtdIOd8XKcg2pzfIObWsQTcUyFOquo4NO2PiU4XjluTNhRjXL+2aXAtjA217tt
+0k2S1N7vmPKeUWQV49cmmRlcbXPAeEfCOjwYGkbzcdIY7L6kEEMCoKg7vkz2QaMaRMGxixE0A+N
UUY4Yo3vjI5ovZqiGTJsZ9LB48n7cdcWham8dmhNEsT3s0R7/wCzqsc1rEMvIQK99adfJIn0/IWI
CL98C5Nv2p/CZqqPZ7PjHt1b5FYlbl5IErcij2svs8cFj2SZsx+jUb9jjI6XAxdTc29K3Qnh46W0
2ehx/Q1Z3K1eoul5ePqcb+oN4TqGN/E8S5vuxPd37fjRNl4WD6Or9USW0J7fu+yJZwrLY+37TTZH
vNzyqeRTKqNTJHGMhEiNBzSRpIwx23PMLNkyZHdl7Z7IIJ2UcO9TBiDEIPcY00HxE0BNCrBqjFGi
DRqDkHoKNURxkVxkz6WjyQm79J3Y2VOVg8Z4mwRNZsx8K21sotypFYSZl2HR8jfPX45sitz5Ys5Y
2NHDXI9Nk2X7vao+NNXMXV0S5SR6Iy9I1IOYlYyDlkRKFlLJykUVLp1/LyX5qCXLNzrbmZPF/Tzm
G8e+SWvco9a9as3muTTpn2R6off4+Mg8kzrI65gfbHzCynlPMp5lMqPQ1NezSMhGGOzxxIOF9OfQ
vdqjRFNjYUUx6WfdgwhXBFKNXJpkfBks1vaWNWOYMGjRqijh5kRRDBgwInoaPJCYRBCpMtaxD/mw
WONl2b7Cq/Nd66zxeNVcluGtXfCp/EY7BjDmftJlQmsqjoeQTOWyCwpq9iOHx6H+8DsodCcdNyFn
rRs8FHjOEscg6bkZunK72SSu6Z8cdrrvql9+Tt/r5KqpDC+yOsHkVT3Pc9z3GidtRw8RxE4heMeI
8c8co9R44Xvjvn1INUybme2DAqemNwwiPsRzYWt+9HQk8Ptah/dphYxDYR5v7OeO7MaNYeM8Zr6W
/ZxITII0Ro04Tk31nXY/qYXwpDK1vliezzuwkrIa0jEWMewkYfYZLqefBJMqq9R3uQ2nwrDfZIjW
o5Zo9z6XA6LI6up04yGtxPMcAs9NkueZh4Di7a9ScczjYOLgb9LalSSdrcmoxhn42/dZTZVGpkbE
JCLEPb2QQyK4e/u1feKQZMeY8grx7x6iiimBjPXqaGvfJkb3d6WESkSn+l+/G/w1Jme1xn7nRGMG
4rzym5v2azJFENiPEOjHtF9CDh5IgjRGmO3Acsqt5DjtH6KpGxXthrYZhI0bpIOJPs5ojfZy+zlH
Jk0PGOTAyw9gzlMDbrJRjmkPi3s1rP6Xx3UMkFWzx1vknurYXmuQv3ZOoVTi+nisiavXd+mBPu/H
xRsymh4xjPdjDQehJ3ybCqL6GvwNmPMeY8puZyOF7MaMb6UQawSM8Y6MVvoYf6F7L6I3ELhHn8nU
ExGiky/tsrl+pIwf7CvNxHCKMIkIkGJ2ePNBIjwi1zTA4ePQx6PsUuXZfr/TaubPDXZc6hd5Jr89
lacz4XV7LZieE27OFYaGMDu2Bw5dVq2Hq/hGOvclf8ccPN9Rtfyv1DGpep8fxpDc+t5vrS79Xyuv
s1fZP2q9+zvijjWRyQ6njFQQYvtt7SKSKL8WTc3NzY27oNG+hBiDGjWGg5g9o5BezTPfBgwYNRsS
jEVBHlf3kq/wUszate/d49CaIeiooijVGKRuGOGyeyyDnH3EYMiGwj4SWMkQcL64rskDFldqRtyV
4RK64+sfC5k0VpHwuYae+g9pjHZRxMSKcdH+7ozxVIOoIbNmGLiG8dJU6b+tWXp6aFlCkjr/ADrk
dykf7Wt+7nfJx9RErO9hyjlMjVNh7x6/NkRTInbA0b3RBrRowTs4cPHJ2QT04MELMrBWEroLRQqc
d++GorWvhXHJMXTyaSMdsmMjosk9YliVi5EUY8ZIMmPqDziPGKMIkEb7SITEw4X1K4V5sN91rRFa
Aih9r9PKP3hkock15JWZI17VYqj0Ffg29nPHj/vxdZ8rqvBRVeKfztLk+ajhdyXJdVcrYuUrL/0b
pitL+mcEr3TSuazRzFz8dRm8z7SavkyOeKpkRxsOd6V+RBBGjWGvdBGiNEaNEUyKvZWjmip2T11v
5w/xYMahTYmYmZR9fJfpKqX6asdDJqsXuaEkJZgHpq4Qa83EfkYMGDCN+BJh83tNKSvyovfIrhXD
nivMldNlpwleEaz2ezZOTojsxuqcq+EhmjttlgVivTI6P3c0VB4xm8vQvHLEO69r3uTZaXluN6Yu
y1ZeUftxk0rpWcvyazcSxcI+VXIyTVXvRV+H7kTVibupuKpkyZM+pVF+ONmRkI2E0FHr2a0awRDH
q1FaOaK31KOGP1WndRWpZaMuJmnMilZ3tgmhR6cjxuxboLE6vJq6P9zXsLEZcZq4TsgwYgwab4PK
fUD7I+bb05FcK8V455kYhRhKceEhb2eW/dLsPvjBDO6F0HMI9sj2PHOQeOJDpnhn8xddfr8LxbJY
KvOclDY47hrvNUKcFyWSSDKvbek2rMbsjv3KrBfihbtI8XvkyZ9GTIrjPxxxbEMA2E0Hj1JXjGjW
iNMGO+O+BRyDjHZqDWHjFjFb2RcHncglpUWnzDoncVyzJkim2QlYil2gyQtccrHVcoKTtL8ZgRBr
RrRjRjRqGcDpB0w6webIjjPZVFcOeOebmTA1hAzLqcZXQYbj5CzMWZvfGw5gqCPe0dPI1WXTzI4e
px6MqVusP8fovpn6ety3VfNP6g5NtTVk396Gwx8ddWKrdHNajsGRkSyD0a34a0WqOHfBkyK4Vfiw
RQZIYBkeDUeuCWQkePEaNaInfHfHocL3YMYJGPhJIRzPRkock6q/iudZI2G4j0dITKhbTIxMOUlQ
uN2R0WDQa0RoxBhtgdIPlHzjrJHKMeI83HPHPFeK/wBDGlaIrMIeyvJnk71UfErlq1VctuskaMrb
iwNakrEJYzZzTpTj4b8/EcZHRk5q/BZv8xx6QXq9PCsw56xvmOab9HX4fjYYalvkPPP/ALjYsr7E
qIvqXsz3cKOHC+rYz8bIlcRViOAZGYHyYJZh8orxxqI3tj1KKZM+hhENaK0fGSRjmDmCpjvBYfAt
DqFWFfnmSI6+16TT7DGK5UqqpNXVpaaOYeM8YjRDIrx7yWQkeq9mPwMlGvPILIK8z6GfeuzK1oSt
ARwnjHsJGC1sjKZHW1LFbcsPSFNXzCV0FpoovF+ReN4VIrDZtYuCuSX+fv8AKR24OHataOdWRzed
9aGOvJyEvU9lUP8ATUySL9PD8NdnkkwKKOHfgI3YirEVcZCIwX2JJsE0xLKK/JkVfiUeO747MUie
MkMjh7R6D2jmmhqaDWDMtI7cjCrZdI6mzYYzBMxMXKyD2fu0NDXsqjnj3j3Du6KJIeQ3Pv6EGIVG
FOHJWhGsFYPYLGeISI1Qmbllqu98rK6xt8Y72JJJK1Tgqj6fH82zHHcrInDcXxtdZ3z1sQw8XLMW
a0h09pBX5t/l5V/stZn7rUnkk9X++1BntgeO7L87GK5Ya5FCMiMDn4JZiWUkkHOz8rhw71I7AyUS
Q3F9xw5BUMGBGiNNTUo+z6WMNJ/tbdgemVRvZRR6j1HjxfjY0hZlacZThIWYNRR6Ghgc4fNg8u6v
hRqXJsH1yorbCPOOqqlb6jwxRLa5c6o4t8F3gKD3PuX1hKFqP6PqGZLDpK68Rx08vluTN945fFX9
Cr2z3RquWFmkQ8eKL80cSuWCsRVxI8Cu1HzD5h8g9xIvzKOF74MGOyDXCdnDhfQ1DUwR+y07SsIb
yYmsIqWX5VTPZw4eo9Ryj/Rj1sQYwrRFGEqw4RjOzu8r8E02FksbOpw4bafhLs2XTyI1Olkjs8jx
M80VRkNi3Y6itrWo82klvkKNKSrFd2/ULVjwHDcwlG6/nJL9lXK5709pfb4qnsMd2ePcOd80Nfda
9Qir4NMD3ak0w6TIrlFJB/3+VRTBqampgwamBEE7KOQd3aMG9moQCvFnUe/PbXs5R7iRw93u5wvb
HwsI0ytOEowkLOzxzzyD5SeYt2ffiKyzvdhjOSs6pbmHbzP6e4RYoGTJ4P1qGvFzd+O9Vpccq25a
+sNiu2W/Zr6nHUVc5v74hXi/FG/UjeZHEiC/I1iuIa2SvWIYRGYJn4SeYfMIvZw9B6fPgwYNTUwa
mDHfBgcOTtgYwawRpqIhG7BuOU+43sqDyTJKOMHjFYKnwIMKrcrTYUmYRn2JHE8g6wSWye3kV6zT
cXX8Ne0/DeSly9WOnl4vp7aTDfo1fJHFf4inSurSlv8AVNuzFUo3HydQ81wvCNedQ/o9ZL999gWy
kNP4k91VmD7EbxrzYeO+OKDYirEMJCwZ7JLLhLNgmm90XIwwKg9pL+AiGpqY9LGDIjxDmD2Dmmoj
SNojTHdHCyCzDFG9lQewkQnQ192sNRRe+PShEmVpxFKIrtwiKK8keWHk0ioss5NYydPVPqrH/tM5
KyjW3ZTgaT5Jaes1qG+29y0vVUNpvT1Br4uF53wnVXN/r1+GhL05wcfHyysg45bk3KxU+LY+RZBP
t8MFdXj2D2mRrzbsqC/A1mxDWI4RkQ1DdEFsE1n2nsCvy6MZ9hSRxIvq1NDU0NDQ07ogjDHqY0jj
GRHhHxkjR6GpgjG9nexsZFcZGPGSCPQyPJiVgrfQ/wBGDUwYGsKVXZalUrQYRidnuJHll5YeWJcE
kx0UiOr33owv2d3zIsz+KrJT4+LWrDy704jiKEtfiuP5BsnG8fw/JPgda5BL1C7Dd4+3S6kkrJbk
TjaznK9TIvwNbsvsxrx7RUx2YIKKKnowaqR11UhrjIRrD7D5B0w+cmnHvyMIxFNx8o+XIvowNaNY
eI8R4jxmg5opqNaY7r3QjQhYRxixkrCVBzRWmBg1eyjjc3NhrhsgkwkwsqD35JFHKZNxzxzuydsG
pqaEMOzqFYq1RkODUchKTuwTSk7yx7q7+XR0/jj5i5+6RVkdw/Buet69NPdqwxUnW7Dntg6et2a/
JzyXbteurj6Z6Lds27C9L2qtBOVvfqF3u74KkS7L2e0kb2aMTJqg9iD24MdkQRhFERQDYRGCvRB8
mRR6krxzs9mDVNx0o5+fTgwRoMYJEeEWIVgrR6GojPQnoQhICMcSkiCtFYaDWiNMCjx5k3EkNzzH
1B9SLMOeorhXGwq+honb7duPg2dx9PCRRGhoPQkLJYJnE3uaHFzfTSTOW47huEZCzqK+vH8d04xL
EzbsaOjvTs5F/PWOW5u1WjeJ/Zvcrzr7k/1ko6Rys+NEyrf2N7KhKgoxRhkVw/u0jT3haRsMYR7x
XdnD1JnehHivFX4IiFBjDQeweg8cvoVRXje7u0LiB4x455Io5uTQWM8RoamB48e4z3dkV6nlU8qi
Sm/ZzFHNX0IIN7YIYlkdxFD2hhwjGGo/7PJ1wlknJh41hDCr38Jw7KVd8yK7qeFeVdTWtx9HnWyV
qVGZrb9av5ZUe+kK96qh7KP90emqfFWi1QQz2kHjVGPNxVHvM9kI3e8Lhjx7x7xOzlJnj3bejPqQ
TvCQDBR5KPEZkSI8Q9mCR+Bnu5vZRe0TiKQZMeTJ9zxnjFYKw1FHqPeSPHvNxHdlHoO7IgyMZCfT
D6xJDqvZEGINQRBrcnE8dkrVtGxRmuo5R7h6k8hM8sy+0j/f7jUOmq7ZuRv30YLyT3qknjexPqLn
NcpI3la2zI0qpWguTOnlhbsyRmjqtXdFw4kxln83/wAvXBH5HuXPbJsbj1Hdsm6m6jWq9WQnh9vs
MX90Txkgrs9kFUkeTO+NBO2CEhGPNh6jzxZVlcSuPgLERMwjQTs9TPZijHjHkajEEYaD2jxyjnkk
g+YfKPebDXiPNxw4/wBsaRMIYTwez4PazCK33wNYNaI0RDjaflfQqoxrGjPYc8eo4mfhLExYmLEv
ZO3SNZJpOUpy+WpUk+ryjr9RkPHMdyMdvleHr7pyrk8DkjZGlV7IfZXy2Vkb5dGPcjhn8l+/qa3Z
zGJC3JsOeeQ3Ff6WR7rBWPpiSEnZgyRvI3iO7K72fIPd7L7ip8TfRGRuEeeQ+4jMjIBkIkJJD7WI
SxGaY7vXugwYQkYgryRw9xI8fISTD5hZDPfYQwOQ1GEJUbkSH2khLMJNF76jWjWmCGLyP4qnqkSY
RHHlPIOeOcWX+1mUs2B78qgnbo+yjLD7MDIuOk/Uq+F4yv1TemWpxiNa6zyUtZ7bazk1dVLcr/pG
f3HSyeSRe0X29dRv73u91cK8e71oVmkDDxk7MJa7RkYw+xI8e827Od8SCd2DBFGjGDGIMYInZ/2s
oTt95OyqOXu0aMI3jJTzHmHPHqSKSqSuHGBEEaa9m91QYRfep94/s9CywniPGa9mN2XjaBWh0a1B
xk3FmJbBZnyXJfaaTLvuNTvxTnsvcvi9Xa6GvQ5fkvp4rVybkLtbFd0XGyWSRlak5Zo1LdjzrEmP
Q32i9SJlURI2q8c429eo3713EDzzIiWJsll3aNpEgn2e4mkFeIoqi/E30NGjEI2jGiDTY3HvJ3kx
L2evoaNUa43PMfUn1B5hXj1JR6GpgTvgQ1MDhjSFhWYQoPaTtJmCsNDBxPHrI6tWRiMZ2Ucg/wBi
zPgfZJrJZn2X/wDTe9Ws+5NYpQ8bByL0mj5970qc6qfV8Nwk91tTio4J+qL/AI3zSthdY5BZkZg3
w7vJ7M9UP3V6iuM+uKPY8AsB/AZY1PqSWYkd7tGDDcleSvz2yZ+NondowYRjTc8h5DyD5SSQlePc
KOURvdFNjc3PIeU8gx5kePUd2cN76iCCmhHGV4SvCMZgeTEjRzTTJT43d1Gpo1rD7G5sOUmUtk7y
eYVc9m9morl4et9BUtI+abweRnUHKJx0+ZeQOZ5V9CDiLViulqKSa05yvenuq+ya+8jNHEabPk/n
6mfx+BjMrBEMhyLASwis17Od2QY4a8dISPF+ZondqjFGPGSHmPMNlPKLMPmJJR8go9THZe2xubm5
ubCKMURw9w9wrjJgRoiDWmhoaiMyMiIoSCMhb2kUmcPcYyvH8WryvRRgyPXs/tsbj19riltxI/33
E7RxvlXgOk5nyJbSvDVYxkHH8vJftLtf5CjHHx8LXOs2WQ5i5OFanFkbsIxqPGYc/b3GO1/BhaQt
GYHO9nko4f3apubjl+dgnZVEGjBvdOz/ALPHi/f0uF9P+kGiDyQUQT1oNISIi7TEo4p+89T+LOzh
ez+0n2tlsk/m0T7f76Iai1LP7eKl9rfPqreFsqrOkeP/APqed/8Ar+P+/F//AB+fcruUIv5OG/8A
sfCnwoQkYnZSQlF9K/gN+4on3//EACoRAAICAgICAgICAwEAAwAAAAABAhESIQMQIDEwQRNRBCJA
YXEyM4Gx/9oACAEDAQE/AdFFFDKKGiihoo0UUUUUYlFFFCXVedd0q7vqyyyyy9nsxsUSl1YmWZmZ
l9ll6LPfd9WX02WWWWWNllljZZkWizJGSMjJGSM1Y5immWV+ivBoscqHM/IfkZmZMyaMmKTIpiVd
ZF6JyoybMmRbZ9DRsXwZGRmZmYpjmZjmfkZ+RmbHyM/Iz8jM2fkZmxzZkzJmTMmWxtjZEgrEjE9F
oTRQ0ckRplFCQoihZgLjMB6HIgrMSaKKIquqKQ4ldUV3RRRRQ0yitGymUzEcWUymYsxMDAxMTExM
TExHEUSDr10yUiyMhPqexpFGJiRgKKKEhjWxo40P0NbKEhdV20UbN9IZgfjMDAwHxn4zAcDExMTF
GA4mJRRRQ19CRRRRQkRsXoZJFC0J6ExkkYiiKIqL8KMSKoY0JFD+FvvEoooxMTExMTHZiYmJiYmJ
RjZiOLMShxMTExMRIQ2MoaLEy7FExMem+k/GhkhS2LY4y/Xx2UUUUUUUUUUNFFFGu33RRRXVdoox
RQ2PQmWKQ5GRkX4WWLqystEf4yb2z8cYtRRXUoR+vOiur8L7vxfVFFFGJRRRQ0UUUV4fQ0UV0nRY
mX0tdNCQxDZFW6Rxwio2vZyNp0RkkrsfJrRk+r+Bllll9X1fVl+FFd14V3XhfTGkUUNFdUYldZCk
ult9Nn8eL9s3Fto5Hb35Sae/Jssvu/J/4bbEyxvxortldoQ5URk5ypF/SOSX6fyMuuoyssyE/wDJ
aGtfLRiRRJ/olJn8PjpfkZPSH8jH0tGRbIsX+TRJdL5WrPxtukNRpRX0ckrdfM4j8IiZfxV8tdMa
K+WKOKG7/RVIfv55Ia7TSMhMv/FrpvptfbLX0yum+n0vKLON4Kv2csvmbMi/2NEl4RF89eKXTG/s
UXyb+j8MEh8cBIoaMR9IssXfFDKRjk6OeKT18tkuqExqyhrpMUjIv/CQ2MdvSElGBKVI/Las+12y
T7aKEWWcfF+PjybOGUVFpnM/VEotbfyyG+pQa9FloY0JWYslojJ2Lquq7r4UOVGX76gm3ZyS+kNO
raFH8j19D6oaJItGij10zhinLYm+X29Eai6P5D+ycr+VmJiNWTgSTRZZDqUbFAS86+GUrdLq/wBm
iEXJ0tI5OSPEsYDmyNJtIT0IQyVMYpsUhssoxXDHH22ZRiqQk47Zzu466r4a8F1HkoTUiXHZPjob
ISFIv4qKK8lBuOQjb0hQX2RnFInLYtUkiUL9GNKjL9FSZ+N/bHCvsVJMXEvaQuJJ7Px/7MH9HBxt
zVk+ODi5EI/sk8kTX9SvmQxs4ZUjNEkmT4/0RiJf4TG/60KKLa0NqXscU/Q41pCET90cjaTxFHkz
UrE39la2JfRHSo5XocteyMrRw8TUXI5OZNYUQSStlkhU15tlifihkVbIqiTIzKTJRPRZZYyyy/jZ
xq0N76xLkLfsoSpWTTdk45IhDH7Jt1SISm/ZDNsV2S9Em60fwuKXK8n6RNuEXGPoi1I0/ZJqxvzb
LLIvxssghE31Dk/ZdkjJ2J2Jlj+RslIi60iXsRXWOxUNMaQ0vslFqeiNxk0yrVoUf0L0N9RVQxf/
AEnL+qiRgq0STobH54lDIvzifRLpPZGVEpH2LpPtPq/gkxyON3Gya30vViPsTdntD0OSFBS2OBDW
mLTMmpUKSboisno/q/6xWx8VvL6G9VRySSdLpv4XEUfOIx9Ms9jFIUixD7sXkybGzg3BnItioWhO
y9CE6JL7E0vZ/wCfRbbJRbVoUH7PfsSXshDHf7OLLK37bG3WI4r7OSs3Q/lrrFs/Gz8IhjGM2RY0
MTZkRnYmMTF0vFko2SgfxtNxJq1Y0Vsui0JfYqJetD9laE6ZkinZKLaojCUmkvZi40l7MqV1sg4y
Wmcsq0hl/BRTMWKDFAxRiiMSMBosyGzRoojplkqHRoypkJpjaI/DQ4n/AMcsj2hofpH1ZSsi0tGm
PRQ1oooTL2cPtuyUkk6MpOKr2N6bZjaJyT9eeJGBiVQkaRkkZFkJI/IhzR69C6rtoslsqyhqhM9k
WLu/KjkWj+PNSjRNb6ati/2QGiEa9kou7RVFv7F6KRRk08V9f/pbXsjybVkouRyzpYrzUBREiQ+m
xsscjNmY+R9Lt9X1RRiOI4lEHuhdNimJmQ5oT6Ryejg5MZf6J7VoZ62JraIvQloqzZTu2Sjfo31F
1v7IwdWxpO2xpSj/AFfonN8apbK8uNW7ZaHIjMnKyzJDlfTkWMRj0u2PpPtFWSgS0RIlkpmQuQfI
KVsgIRyEYnA7jiycaKR9CjexEUM0xopJ0zBVRHj3VkP9mKbaLxdItv35p0qMhyMhSJSE2+mxsTGL
uheDQvBMe0TeyLIy0TlolI9ljbONWQXSY0JCeLsb/LG0NChezGij0NlCl+z+rdmNk07yiZSa2Xpv
9CVsSK8nJDnocxSbLZtiVdNEkJj9WIiiiihorpooruLpjnok7IiZJ2YmNFDIOmRmJ2JjH1Cb43of
MvpDnKXtnG2j+r9Di0SXVO9kmQurOKH9U7/6PTOW8CqG/Llk1oux9RXSF20YjRFEekihrqvCyXof
7HySoUv2RSaGqKENFfY0zaItkGIYxjYnZxxsSGiPImqkOGrQ1T0Ssf6GpaSHqkiaxevs5Wm0v0Nr
yS2csdiiUV2l5YWRhRVCQl00NeFljHB/RPjkvoTIycWRkpIaFoZY2bsgiAmWNkpGT+zjONUhsbE7
ZGUo+iU0z7IRt2Rlb/8Ascnuh7JbdmhrxWkcgkUUUJeUYMUBqiUhLwofc3RGVkVZVdcnAnuJuOmQ
Y5oyJcg5dRgRgJ0KRY2Mo4okVSGhojGnY6eyrHwscXFYjilAukLekcjp4oX7G78aJIrqivFRbIcY
o0SlRKQ35SfSRONkYuyKGjExMV9koKtDgZPjJTsU/oi7ONecYnFEsk+qErHitDlSFbeT9E/7Sr6R
SbIKm2S3bJOlXjBbGP4IxshEtIlMk/NseyhRMRREu2TdGbscEzk4ySknojZx2yLYn4MRxqxaRfVF
3pCdE50JOqY3vZW7HaejKSVMQ3fjFjZb84xIxG6JSvqXm0YmJRRQ5GQ2TlRORFiZLZLjTPxkI0JF
eHsRxIb6uhzOP9jOOCrKQ3uySog/bYpP6Ryf19vfko6Ghi8UrIxEqMiUhdS7sssyE/GTJ8tEeW2K
RLZJCFKy7GJCF3ZZkcSyYo0h9SZI4v8AyiSvQqr/AIO1slJuZP3bHPCP+z/vilbGNDK7sURRLLJS
oTvpuhu+2NjkKRF+M0chHRB2UOI4Di1sVpiVmLKF0xjTYotujh48UN9yKs4XSP7SdIr8cCVyaaJS
pmUpO7GvKOix9ooSIofTG7EX4MkNjZEj4zOX2XRCQnrqhqxxEIkNuyMhlbMTg4vtjddsb64eO42R
hWzllVR/YlUbGm3YqWxs+vBL7Miy+1Ex6iy+mPxZ/okxvqJFl+E2ciHE41siX02afUWNaJ2mRlsj
TKIQtiVIfVkn3wu4UaxovOeX0Syu2TaSH6GP0vB6VFC8IjH7F02WN+L6kUMTExdIZMxsfEKFMQ2W
U+mRG6RyPYpbITOO5+iMcUN9SMmPfXHDJ0Rhqjkn7ivYpxjLFI5ZO7ZbY76fhfhGNmJVDfSLG782
x2MY+kRQihkhI+iXddMfUiZ9nErZxQUVofSJEuoLOWzjio3Q/Z6i5nAsvZz/APuj6H6F8MBjH2/i
/8QAMxEAAgIBBAEEAgEDAgUFAAAAAAECEQMQEiExIAQTMEEiUXEUMmEjsQUkQEKhgcHR4fH/2gAI
AQIBAT8Bsb0vnRMssvwsvSyyy9bLL860Wta0UVpRRRXhWlFaUUUUUUUJC86FpXwLkssssvwWlFFD
VFFFFDRRRRRRRSKH4pWbTaPgsTsorzss3FlllllliYhMXwUNFFMSEitEihlsRQ1r1pWqY2XokS0i
XpJ2NmJ2UNCL1boeRDnYitKEiihIopFG02m02m02m02m02mw2m02igbRxrSyrK0T1SKK0oobGxMX
QmPvwnIXJNHp3zq0Vo0Siz22LHXh0IWiQkJISEhHGteFaUUJC0kh96JCGJllCZY2WIkzsaEhLRiL
0mRXI8dkMe3wooY15cCQtE2WWWJ6Xpa0v4Fo0OIo/ZWkmXotOShcF0N2KNkhCJOkOXJjdlaSTl0Y
4c2bX5sZQ0Wy2WWyy2JiZeiRyJMSKo2lIoS8b0ihUNRRKiyxk1YhaWWbhyLsjGyqJIitJsa5ID4V
snl2r8VZj9RklFz+zfwqG+V+yOeVpSXAnfPk0VpRQhIrStELRCFwJllllllm43Flm43DkNjLobJd
FaskWIirI8aNWJUSdIfLNpurlnqM0lKKfTMSi42nySg3KkiOKpcjhFu2tKp38EiiiiiihIorVaJl
llllljZZZZZZZZuLLGy9KGtHE2GwUa03UKSek5EVbGuDLy0ie3LGO76IUv7Xx4shBx4vjyelPWvG
y/hssvW/Cyyy9FrRQl4NEkxWmbuB8kFwZJbY8mONfk1ZjhH7XyooaKK8r8bL0sWll62X4PRUxIor
4WkxwRJUJCMst72Iwy5S+l8rQtNhsNiJKn4L478X5diF5PzlEqj6tmKMlcn9mOKS/wA/NXgyY2J+
F/FfyRYvlnLky5FSj+y3u4L+K/Cx0J0WMmTkzHLkWtf9AnpeiWihN8pH5LtG43CejFfk+ib5FBZf
yf0zAqTv5bEUbWWRYyTJkXTE/G/FfE2Xok2b1i4Xf7Jeqm3wR9XkT64G0+V09EbvCyxaSdIzSo3+
xDcv3wejyPJG5fKyIi9E6LJEkPgjloWRMv4l8DZvoUzHLtk5SlJ30RXJHELmFfrwWlDQkJDMkz3f
dzbYqz1kJycXFnpIunuIzUnS+RoSEVRHJ+yy2TlRKZPJR7xjnZFifGllll6X8LZGDn/Bk9NLuDsV
xdSQkowpkI/cuhxhJ1F8n9qpkV+L0vSLLWlaWZJ0ZJN8Ltmxen5S5MzeSHJ/w7hOLMUa+SjoTLYm
yMiNSMqokmZYtkMNmLC0RgyvCy/jb9tXVsxy3Rsd/TFz2ZMigr+xRnne+b/9BOlwRn7joUe1qr+j
bIdp0zfQp2bizKmenxvNP3Okuv8AJtlObb6G45Vtj9Hoo7ZtSXPztikSwjTgRnXQ57lyOB7SZD06
FFIoor4L83KMpuK7EtqpDSXLHP8ARmi12dQ47Mcd8LlLn/A8zg+7ISU+UbOeei4L8T34rpH9RGuT
+oi+z34NG6DVo4KR6mljf+SGTLjlHEn/APhOVcoUdkrMTuVl/KxsRxFWes9THdwRzJkJWhP96Qfi
0V8jMUWrl+xzGt67IxljTJZJNNEXu4Y4MpuSRhTj/BOaj+LRJRdlEVbonHa6P4H2Ubn+yfqVuUZf
yYPTu1lvkncpUirasgqQ7T82xMXhLoYkeu9Tt/FGXI8jMaafJjyNClxpGRvE71ooaK8rL0vTK6dG
3gtx6Fmf2boZHt6ZkwU90RST7JzrhGPJFJGfpNEmkzgap8GSElTZFiu+SrZ6maxQoxR91xlkXLV/
/A4uNPo3P/t7Yk+ER685Mooj4NG0SPWZt0z08NzJenW20KVOjB+SGqH0MixTQmtX8SFyycN3LKpD
G6Eqe4lmm/7S3XJ0yM66Fk3YxyUo0SQmZMk2lGQnSHITIflNyXfX/wBkMb37vonk/LkhLnoUebF5
1ohd+eSVyPRRLPUx2ys9LkpjV8jgTQixTIvRlfFFcD+hNsYyQ1wcivplUyKaVjdF3yPH9krfIuVY
ytpjjKH5ydLkh6l7WmuWyMb5bMUHJWyhRS+BrRX5pXI9Kq09TDcjDjaZDhaSjZQ0JEPjRFc6N8xI
2tOB8lFP7IXFuyS3K0c0NvsSTFN9Mk66LrkcfsX5S/j/AHM0Y+13wkY0v7mb2+F9mOLjBJ9i+Xcj
ehzHlo/qDHFXyen7EjbZDCu6HD9aWmNDQoi4E/j/AMikTf42hUpVpQ6qx91pJWqIQ4okLkXDG7Y1
fBTN+2P5GNqSbfS/3JQv8b/EyxywdbeD0yrkRXwWjcbhyY5C50nKkTyNsiyOEx4drtCx2iMBJIaM
1pcDytMhk3LkdlsVitC78H8C5/FjtPRfYkimqf6H+0NIVKIyKbKYkJ0Lsy/SqzHB1BPuhwip7X0u
R1u2w6oUtvBCLS58m6HIlkFKxDKY4tiVaZUz25ChJDxxuzbTI8CLLJRU1RP0rT4IYmjYzYzosXg/
NiM6ad/sg7R9j6Rym2mTklHckQbatibGubIukPG3yhp2Vpixpxc39/7CSbqPCRLE22xZFG3/AJPT
40rlJcvzc7Y2TfJjFwWfQkUKKHFDir4IwTLEJaNjkRkIooZknQs1MxzUhoQyUxSZuTHJI3kZXoyP
ZkjuRHh0J8plLr9l2rX0Onyb5SboSaXOlkM1cMi0++jZCXKPYb7fBLKo/jDvouUIpL7YpOM05rlm
LHDJ+VVzwXz5ZnSpCVFWTiQi0UzYyqNzfBFD4EyvsTZtEhaSQ0JUJ6UTMsLRNOL5IZ6ZhyrItJyp
Em7IOx6x70ZDsbM6pqaItMfB07RJ/SFWm7RxE2j3GjFkqNtGV7nUfsnlltj90JKk32ceb5dm02o2
IpITETRFCRMiLSitLQzaNFieklZPsnjUuzLBxkelyNMjkTJcjxiVDesF9lDEdm21TEvalTLHKuFp
ZYkI22ODGuDDKO1Qn2z28alUXwjanKMV9kntXA35M4RdsS0aELSqYmSEhEpC0lKj3CMxOxooTLLM
iT6GnVmSScnZgjzZGVMxtSHEkhljke+0R9SyMrQkJaZMayLkWF/bFCMekZYqS4Pyj2RdiEJCRk5/
H9mXJ+TW3roVNcppGCnltLpHYl9+V3p9iZd6LwoaGqLHpKRLkcSJGRdj0sscq4M0krRkUt3B6HKm
tsjJHaQz0zFk9xEoksY40TJMgmYU65ELT70m6LslEeNrmJHJzTE1ojHFSuUv4HT3NmJ+6lfS6PTx
2QbfbIppeU3SIuiyhLVeUpJDmJ2SlQ5FjLFIjIsbLESVmT07kj+lcSfpE1a4YputuQl+Mj0c+Rkm
ZJEuRYrMXpzbS0+hISG0iatiibScK5NsZdkYVwU0TfFIlFRx8d10LHFJKTsSpJEKUaQi/rxnK5CE
iiihLyyZEuiWS2RdsxxG78GJEdKGKVCkmIi0uGZsN8wMsWicHd0enbTPcHMlyRx2YsJ7aXQ00IUS
q0bHJEZCJ8qhpwdIi2uxeqiuGQ2zluZbc+hRux8PdLpGCNx3y7ZJ/SEkl4N0hERaUV4yko9mT1H0
hzbIQcjHioSL1ekdLLokyxMUjeObMitWx9myMh2iyMbIQIrRqylq2ZJCTIIssyuuipNEMTlMk1jj
tj2Rklj4N0klRnlcVF9shwkiPLb8csqVCZEXnOaRkyWU2yGMxwXheqEWbix6oY0T6JZJRlSMeUck
yKsxxQkJFl6slKhu2JF0WPhWf3PkhjT5+i1ubj0LG1b/AGKTlwiLT7NsHJtf4GhKvHJHmxIhXnPJ
+iUirIY0uWKiHwWORuLFyVRYmSkRdmUcLZspG5ohPkwzsujeJ+EiTFEoSsUETpuiuTLKTeyJCDSr
quCP+pdfxbM0dtY8ff8Aj/yOMFbbZi/JLil5OfNHZsKrxlJIlNsbs22Qx/ZI6Mesp0LKj3ULJZuJ
SLExF0tJsUhshLkaUkPHRKJLELG0YbRyKLEqGzcbjJMUrZFjdiQkZHTYpUNU1+zjp8/+4oxjDaY2
l+MWRxvJLqkv/PlJ0rEiLJCnyJllksg2UIhBMfCKFCxKtLM0qPd5IZLItm4XJXAhDGyTsuhzFIhO
xUz20z2Uf0yFj2iFWjLobpE57nRGOl8kWWZ47mkuyEY4VchP3sv6QpbFJftEIWuzbGKpIT8pu3Wi
fJ2hxdkdH0SEUJEVS0rV6ep5Y07MZ7iij3uTC77JiRRLoscLJqhvkgmzHBkI6cCZKRuLrksbGzNP
6RCNs6L5EiOmf1Dx5aJZfc4fRgTdyrhDnunRu4pDtqkRVH34ZJ7VQhRsUORFEppCmLlEkVpFfYvJ
dGRWOKQ5V0SkbnZ6fIJ2QWkmJkWZRK2YsaZGKQ3R2XRu+jIxTFP6Iy+i7Jy2o7dkVQ0zaQWubH/z
G76Enu4Q/wDThtE047UjGm5fwR5Yhdvwf5OxREvCZFEeiT0jGyhKvJE0ZGO7GKLZghRBcC4R2NG0
SZkTIx5MfAmMscjcZJCYnQpHuJDk5uyKGyDHFMXGmSe1cEp7XwY4ulOXRLFKcN7fZijHakvopLhC
rRday6EvDJPajey0yMSuCSErYlS+BUSROCFjQ4IhjRigkRGI2o2o2okLsgjobHJlkmN63RGW7siI
kJ8ketMs3CPBu+5dkIqhf6k/bfR6mbj0YFWP+T7I/Y/hydiEQ0aEvhR//8QARhAAAQMCAwUDCQYE
BAUFAQAAAQACAxESBCExEyIyQVEQQmEUICMwUFJgcYEFM0BikaEkscHRQ3KC8BVTorLhNGOSoNLx
/9oACAEBAAY/Av8A684LYjaequbEGN/MUQNnTrcrTAX/AJmZhUexzD4j46pFC+T/AChRS4shjddk
mMaGl3Si5NRdTJfdVRdPG0/RF8WHDHflyRc9vo/eHxsyCAXPd+yuxT3Sv+doTYMI1gsy+Se/ygbS
n6J0+IkJruqKR0m9XLNNe/ReTw5Z0JV8hGifHEyufEjE7iOq3eH40a2Ju5XOTkFSNg0zdzUz4ZRG
+iNZCXFNFxpXMKN1C2H3kHPIfIMx801gC2/NbLaFrPDttOdVkaj4yjgYDnr4BYeHDNp18Aryaqsp
oR0KJYat5K6VbDDx2+KBe+q4rvO1zWfxi/FvG8/+SNkZ2daXLNjnt8E6N7Cz/MKJ7Y+D1TSm3Gy1
Oaw3N5fF9t1kY4nLZxjhbQAIGdgNc95UyAGWiMjKB9OJqr8dtLWUuzu6ol7juq1sTmt95SR1Kz+O
2x1tHMprA3daKJzKHYtOS2OzIy1TpWc/jzbOG85CJgue40RJiF3Xmrn0t+PGjlzTY7hnkvKiau1r
qjBh4/8AUVv/AB46QjeTpA07FjlW22iLvjyOFnE91qjjrc52pKDRq7P6rY+8ifjwSyMujjH7prGU
HQLbyPLjp4KQEZtNPj2rhvk5qaSM5MyqtiRQhOe7X48jjYPE+AVngi9xBc7eOWqcyyrgdVn8PZAn
5KrcPKR/kKvfhZWt6lh/GSTPbW7ROfwsATog8kgd0KrG0b48/h3Y4WK88zyb80yTGu8ql6O4a/JW
xQxNHO0Klw/RUe5rh4p0mEpFP7zU+CVtrm/iY4q0uKbHHW0J0RNAU2TPNEfDkcELDJK80a1qNG+k
pdK7m4rEzytkYSKRdESH0VRIcl94VW8ubzUeKYN5up/E+WSt14AnPIoE0d2qjhj+9eKAIXDI/Dkv
2i7hh3GjxKiaauk1t5URa3Jp7vTsoh0WeikiTme6fw8UDeZzPQJsYItYEI4zkUxvTVB9jbupUjWU
tZu1Ttnp8N4YWVkpc+33ivS3Xa73RXHUrNGvFyWa1RUm7bX8O6dzd5/NOccmpzByX0TnsO9aqEVc
StpqrRl8M4fD9ytz/kFFFG8sd7lMinB5JKqd2irTs0VEVUjL8NFATRnE5OhZ90zIlFkR3k5sfE05
ovKbGMwQmucK9FZZ4pz3an4Ugxc0ZmxUke9GaGlfBFk+Fi3P+Xk4foq4Ccvm/wCTNTRWYiB8LvzD
smxUsJjfiNJDqW8qJztu7FT6tk4RH9FewUZqZncK2DsTtMRda4jhajCTtbaVc0hbWKjwAK05IXBG
pKKY88P4YYgZOkyHyWyad46p4Yi543nkuRY021TrxmP5JuW6FQH6fCcU/wBqXmR4q3DNrp4pjcNh
YWNrTdZonkn0dNA3MKSVmGufymndxDw5o0cHSOAth2lqbFjMG2QVzikkBopH+Ty7MZUbKXN/uvJY
4B5Mx4btGC21o5UK2uKlNmltaArZRjybDsq0ge6oo8MxzrjkKZyFNM2IijoP/T4WKvhm5RlgdSvf
yz8RRRl8gLfeplVcQKtTandr+FpHG5/+UJu6bGM06KV79BoEN3VAUVTojsxdyTmUpQ/CYxmIjMkb
DSJvV3X6KS3HRRgft4KZmKYbmColByfU5J11LUwl91dKuqtts/TMcM3tNR+uirEWyFuulf1Vr3a5
nmm7V5fJKc3dVNHjX7wN0Yaa/RbEE7NoudTkEyeWhjpWRrfHSNMMQEUbK8IoG58xr9UyrLnRtqQ6
gt6VPJNaxxkhDqNuNDp3T0Ub3EWuHIFDMBPfXh7bIoy93giyRpY4agoyNjcWDvUXlWysjpUXc15S
6Kn5OacREYmt5vCOHlG+OnNVfG5n+YerhbHFdLq9SC0VpnanWDdCbRNjA9I7QKr32t1+ac2ic7r8
J4KLhjEIzLLW/rzWJfHqa7tK0b9D+6LiANqb93LL5clvP2YPOqadla0985uKufGHvByu3qfqr5Hg
E6NLuJROMsWHY4VdHJxUUzcGyzD3b+0NzinbxLepUjZLqSvbHujxU8Ti1rZJ/wDEdftKDlllp/Zb
K10r27xv5f8AlVkwrKDQ2jLNPxArDOBUusMrvpnlRWX2X0Dmnicf/IzXopTNByryUrWGgp2ObhYi
+mp5IuxLQ7Ev18EZy8Q4bQ9SmwMgbs2oNy8Ai3I+CJya0KTHSNa+Q6eC2WVvOi2DIg3xWzjjbd1X
o5TtF9+PqEQyPaN95U8lkr8lbLE5jvl2xNObW7xT2RMa4nvFOw7qyPrvOW4A1rz+ionSvra3hUbC
d1o4UaKvsLX2jgn2B8roAaV5f1Rw7MG1kcO892lldPr2NoPC6ui95x1e7MlHDRB0k3DutyafFXSO
g+08QDRrY4+D5dU3EbIiWV2UYOlO84cltbxNJq4NyCzoM9AsOxoP3ld35f8A9U5pIWvdtWsY2sZq
Mj/vonyxNcWk3Wk3FWticXdKJpLqUyqrq3c8lCXgB1lteqszzGivcNjhbuI60WyiAYwZk9USNfFG
tpcOaa67LoE0t5LaCv6ojP8AVWX0VXGqF5oFxANWo/VUDhRVdRHTNaNTjsGfTJfw7izwKnnxj23H
IHoF5P8AZ8gaxvHN/Rqyq/q5PFMmqhrYE1jG75V70c1QexB7PhIJvw52Rz5clE4MLZ5yXSk94g0/
RdH8hqiJW3O7scJuc76K+OSbCDk176A/NuqMg28EWhlu2m08arbyudAGig2L/SP/ALLyqrIIdHXt
36/PmhHhJS+JoANoyJUM8VjHT1kfEBwn5qOeOnojdRXMe7ZvNpDTTxFf1TqXXnPOgFeadXebrVNJ
0JtCa2mbuiBr6I/9JRe7OmhTaPb+ie2SS5wW840VAfMoAqHt1WqyKt2hVb1mDVUcbUIYjvfyCiw8
X3JNC4d5MjwuH2WXFSizZUp1h4tU6R40zUeQYwfC3o93Csznkdw06fNCGN1rG5CzIBMJbLDJKwuD
tcliJ4MRJt3i25+bvoh6Vv2nM7Njr7LW+Klmnl2VlWsidHk36o4iTDDDlwDhiCdflRYYYcS+TN/x
HOvDifBO2rRtPlRRtYHbZg3xXJB4OR1yUkbs8O/IKjrSOvCCjnu/VOceHJxyU1wrRnXh6KNkgsNM
1JA9273U0R3NGlyqTvLVUoiaKnYfO07dOxlwsxWMzd4MWEaRu5n6JjHNZtB1TpMtnyW7k1MwjeN+
Z8B20HZReP4+ntLAivpJmmWx3DmdT9FIX4lxr7rdSnlkzq8LcuSA2lzWGuake6oLhlRNivqx3cUM
LxsXtto+E6fRCMfx27QOduub/ReTyvG09x4CzfZJXTWqLMQ0+JbmrX5xu0QY8bRnJwQFzq0p9EGh
+0k08E6tbiaUTWZgAVTc69D1UgbGHBfmCq1UVEXALRVAVxC0WnbmFl2aJr3xPfhWHfc0ZBSS3C1j
aADRrQpvtGXd9xvgmzMdbJqjHKSpHQ/eUyqtpJWWQ6uTYu8qDn7Dqh7Rw8XpBHDE2PdA1tVQH+94
NWUhIbpVZ0dVN5OCyybXOqbI3jYdVLUDaS97onSSFskgNtHZ5IySOcx794RuG6Qo7wYLzc53G2n9
EThX7UEnllTszqxWsrU948k0A6LEPY+lukbuaEo56x9E1oktuCcRotFqsiFyVezaUy83Va9kcd1L
jSqf9mYW5kbTUupm9MijrtcU8Ny5BR4SlSU3acDUBG3JUonSzvp89XKWci0OOQ6D2IPaLKnK4JwE
rmNZuhOF8kt1AXpjiW7wyCbl9U4W511RzLq6fNAn6tVPeNa9FQ/7KZtTc2M6dUDE4Rwvfw1Tw7+G
kbu3R6JtQGGlw5tcqWVy7yJAt8FVxqt3JWHloml4vc1idaKNWiJYclTMFZnsqFs7TtKUWvbr5kEP
M1oosJC1pfT0mVXFRgAbOLOjVtpMuikZk2Nv7oz4ktbG3PPQKmBh2n53ZBGXESF7v5exR7SknY4V
lhEtzhumvJSXnYyUOmlOVFYyBsJZH31m01rxBA7xrk5U5sTqZ0NVQdKqo93JV4qZqpy5qwGrTvFd
W8NqY2Pfa7WNbSEnpZzRy7CU23lqEZGCios1SlKI0C0QCElfM07NFopvtB7S+SNwYxtOqxWOazLR
p5J19E5ycU6Jz/R81m35KrPhdzn2vbG0sLZDk4BGItsF3pJO9meScfS2aVdqpBeWBubQVKCdReKJ
vM8QXz/qm+63VfLeTv1TuXRUrbVXoEUuFd5NDaba7iQkaLTo4I+Kqi8Girdm5E6hV7vZlmqhaK05
t7aFZdu6MyppZGVlL6Uccgn0lcGg5qJ0eUddVYSLaKzCMtjP+IoI5SSK1otKNA/VPz0VezL4U+0s
KaW7MS73c3hUqIue+Vu0e5kfMfNTOpS11UH8YdulMbWrXM/RSM0kiTK8JFqB/NQtT6ilX0d4IeO6
PFqaSfA+CA6hZbvRcO8FvblFmeLWvVZJrOabE0/NU1W9oqNOaqFZ2GoWY7MlRUVVRSz23tgbllWr
zwouxFDBhxbvc5DxFYfB/Z9dpKc3KGujBmrcxCDWnVMi4qpz2nhVjeNwy+GJ20vJw7rY/eNQpJXs
ve97msc3m6ixAJ8SuWSw+VMtVJXjcXA0VP8AEbyVa/efsUJKejI3gmjujLLkVQ0ub+4TR+bVfM0W
h8V1WzP+lyqSqV+iJXiql9zlUFb5yQlat52aJ7K9lVRUPZNipCWh+bf6Kk/HxmvVTF++YdxjR15q
K7c2pANOSDxrRPeDnyW9W4lMe51bsvYmi09RT2XhqDelrCD0uFKpvfaPux1NNVNV9BXPxKmjDVAS
TdeKKSId7eam00eE0dxykaaNkY6ufRUIzrmzqOSaBz0+fQroD/0lZ8913gVnrouQKbTibyVQaHmi
iepQzTufZrkqIVyoqErcVpW6rCrwqKKAVIJq89Ao2htMHgv+5Yia7dZ+5RfJnnchdo05IRt1WbaJ
2KxBADRXe7qLo/uWZMr7B07NFotPaeHmZk5jw4L+DaJ5n1ia5w6ZI3NyjyJ8VIX7l4yDVhxlk9Yd
135SVHK0brHUUNchpVNceF+4URId08L/AHSFU/du4/7oj9f7pxGfIjqFnxcJP8iq6FVWmas77lUn
eWqp5ly1VezJVVHFVasZ9pbrTJux3DiUMUURMk2bqLCwF52l28zkvSszKo/cror+OiOIxDmxxs95
eT4cOiwtc66v/H6eZotPa2GxoedtENkAG6EZf2/VQxzG6RjNpMfzFCY6A5NUtRYyyqY6nAao0qbs
06M/eDeH0Q5Pt/RwUbshdukeKlgceHQqyQASNH/yCoeW4f6I114Xj+q/fsr+idebiiaKgz8+06qv
bqrHSbOMZvkPIL7Nw8cZZg2+9rYOagGEfZA9xYaN5BXz5W6FPwr5QGsNq8pOIE+GGg6KOyjnP0Xp
patGjRp7Ky9bl7ElwcryI3+kY3qQsVfJdNM5vPlSqkL+FuQW0lF3dCdE8kyDIN8FTnp8lLF7zaBS
e8DQfonMP3b+XQqGSuptcmyMpdF/Jc891391tO8zJ3yQezP8zVpRqL9GjqnyObaDoqHRZNVfP18y
fEYkegYa5jWicW98Haye43oFEzDYa2auTjrRULbXy7rKquJNb+aw2BZM+SMuBMVe6sORGYg0c+y0
K5x/A6LRaeZp67X2pHMzJzHXBHE12jHQ1Y5viFTutbcnV0ZmEHEhlzQsUIybA/dW7Wmv6qRrqhkh
yVRxO8O8myEnqtm87zdHaXBbSLJ3eZ1RJrHX3kHRk2+BVmV6rLJX5oHgiCBKoM/V0GZKhw7oz5Qa
ukqcqaqPBxztdXfks5DmqsZbFHuArC4SZ+2c994ceQCbgiCSeEq6fOgBqmsYN1mVyayIH5rymY0c
t4/L8BotFp5+vqdVqtVqqPPZr7S2VrS+IbPIZ0ur/dRTsZQTk/QBVp6MJu0e0AD9Fiw11mep1Vc8
mZKJ/h/VVuPvIs902oPiOozatmW3oRzMs+eids3lo6NOSrtHb3itdqeQqt/lyTenZmtPUwNLzG1p
vLgK6IYOvpcQ70kreTa1TtiwRuIta8ZuTIsMxuHDNXTDiT5MbKJHxijbeSMrxRrVNKGue5xyVe6j
isRQ0OVdE3D4Q1hj1d1PrtPxWTis/W6+xdnI+zDzbrypIrSG4aK0Xc6uRGgbmmOpYxvM9VOQTaWd
OJQSX7zhm5NDqtjqWnwrohcfykqRjXlnVbB/M5ITRilNWreyPQokvLWJpBOfiha2juqDf1WWnaAh
VlCroX18Ct+I0WfmnESRvZPOc7hTd5IvkZI5zn2saDl8084XDQNa3d20huT8PFjWxzMyc6Nqb9ny
ODpab73dVJo7DkUDmqYACjTTNXyj0Q1UmHwMoo/d3T+vrgTzWnqtVr8BVU00pbtKelDu8/u/zV54
XkhbJzvRjMAKOQUD+E/lCgI3i1xbkngO42Ap0fDfog/rqmkcTdCrZDQt6o6V8FQ+ljbkD0QfS5Cg
p0V/Psr2VQDisis6FVMYqqxSEI7l48FR7S0+KhDWOdCx1ZHDKgUuKD3ueyOm+aryP7NiL5WDaF9U
2T7QxZdieLZ6AFY6SNgMVdFHi3yGFoOcdcj80yCB9gUwjntjHFQo4TAPFuj3s9cD3QqDt17dVr8E
MlAupq3qsPs5AWx4drskHjeec81IDvX605FSe8N0H5rBwu4NR4qKXul1fqmyCvu/NCMnlkVcR6J+
vzW0jyPTqrzl4JraZEZJo5jJFqHm0qq3UAQAO8VUGoC3qJgla2nUp0MTNm10hqQNVjWQvrtTsx8l
isViX0c4Wxt5lTfaBLWm3vKYl3ESS5OmgJL6p0gaNs4ZOrwp0bsU57p9c9fXZLPiOvwjjMO9/pH2
mOPrrX+iFB3QPqrH/eahUz36n/UoSw+khrVqq3JzTmOigD8pg9Nm/wB1WQyd4p21Jqw5LRBze7ks
uSqO0dlOynJcStrRqFxQAUxmztbWrdVJK95EbTk3wUbsOywa5jkhhGPq533mWngtnLdbrkjI94EF
KlzsqJ+E+yMT6HR8jP6FUe8u+frrjwt7NfhCOUcTDctpUESUc1vQK+u+7Na06u8VtGUbIwbzeSPK
amY6psjBb4dCq0Of7FWk1ZTtqPqgW6dlVkqPVR2FaKi07HyyH+Fi4/HwWJdhnkYYut2Y0aFuCgTo
tsPKCKMa3UIvl+aulNuWS/4fh5P4VnHaeJ3/AI/AeJ+ExDlQ1pcqscLXbu6iwG5o0z0VO8DxHvLa
DjZ+5W8OPn4qlRafOy8zwWeSqFTzMNDHcxz21e52lSnMhoWFmfim/ZhkZh4YW3P5VKa+SUMn6l2o
VQd3kU7ESHdA5qRzBa0nLtP4LT4Pdhp5GiIM3LuqqOHov/cGngs8n80ANAs6AI2Or6vIreWq1Q2j
S9tOFptu+qw0uFBitjbKGnl4IjEs37cmtU/2lIz799yZJ5Vsp49G9VDDin3U0AChwv8AiTa/17CS
q/Ag/HyPxFrHMKdQg8yopJDuPOqcIhu6LeeqgqjhvLJZ+qyKpVYeA8JcLvkrCbIw2i/4XhaRtJpN
PrQLDYHDTsmgYM3c02fFPzO82ijnkb6O/JvggO4xgAHZT1tB7XCHafxsjWnJ4oVbdu9PMyW/mF4r
w9VVGUUGIndZV/CGpjInBgjbe415J8uIeNriZNy7WnVSzxSjDw22s8fFTR0EjQOMKNo5ZqemgNET
6686u9q6ecR7BHZouiDJFVnqWRxtL5HmgAWGwzHtuY01k+fEnYXekw8I3j3cl/xAObNEHWhgPAEz
7PghtDZeIfyUcRxAOIkFKVzUmPoNo7JtUXu3nE1K6etbXhGqp7a0VVT2HUdlCvFZee7HlvD6OM/9
xWIwOzEELA4NkpVxWI8lwQweFDDRzhnI5FlkYaNZCsRidrR+0Gz8VmKu5yE6rC4cNoxuvZ4LNZD1
fz9qVCoVqtfNOSyCofYtQqPVR5tOGJm89/QJuIiif5LGNmLu74lYqPCO208+UMjDwuPJRYAYgYeK
yskj0MJhoBiD3pa80wF+7rRNYRQK3oeyg9YPa+q1WZQz8zRVCp7Gy7M+3A/ZcZbHJO0PmsGYZTr4
rEx9WUoEybFkts3g1SPq4Rt3WtqtM1G0Nzaq804LTzKepuPP20OibVy19j08+ebE12OHZfQd53dC
GKnt8pmN2QzAUX2eQ6WEDaPPKvRTmotca7o08ETStqJOiEcdtUICWmStTRSY/GG2JujU8xx2MOgV
SqUyVrfUivtzIoBxWq1WvsXNUHbogGirjkEMCJT5LDSfEOGrncmrETyP0GefJT4iWUiFrXPdU8ky
FtbnPL5HO/YJ0zwKHS5Ppm5xTiaXErTU1UGFB3GNrTtAHEfYOnsfVarP2KezPtM7GuvJsjc33kwS
G6eYh7yUYmG3aBxNeSZg43NfNiPSSuHdHur6rj3WhSSlpd0VXC1oKklOfIKf50VFXl6tzvgEfix6
mirTsz7IoLTaBWR3Sv8Aui2srTk/cHvHosRJjBnwsjGVGq9zDG2mdy3aJ2HFt/O5cDWkDMqOEMbl
7vNYeM5TSG4sUrzzcex3j6qg1TR8SUCqfMc/ESBkMLb81K2QW7Y1tOdP9hOe8Vja/wBFTT5phw7T
e6axhI16qKB2LfiyNbuRQEIO1HNO2r6yA5lODJuLxW1fDtuixONmO8G0azkAq9UFT1RP4TT2hr+I
Hqbz2U7TisQ8Q4Zzmhx71K5/ROeyrmOFsZ6hPkkxGHjjYLbSbf2WHG0BmeC7Dt4WtHvUV7MO4xNG
UvvlXl4hAGrisS8b9HKtE553WtFVO4afF2q1WqDeqCKKtCZeE3Ahokc82lp6LejJdTZxs/qsKDMJ
nOn3gMzapJRCW4FgDAHjuoyRRhjTkWu7vyTrXu2EJoxh/msRJiJYoIb+IuTBHifKJOkeasY3Zx9F
s2s3ncTvicepyRedAqdhW0omx3gBvdrqtwbODBVfdyLuSxEjNoaMeRI3kVFjZXGR8I4j4rESkbWJ
pNM9VHLA2aKFgtz4a1TsS+11/eB3leWG1CKM7x6oxV2uIGvgs/V15fDdfV17Cg0K7nT90YopxPih
Gf8AU45kpzIt+fFPtqOinixMOlGf53lBj9lFhXguMnT8qkH+G/UIRYfZhzSKR+8o34pt7nD7tx0T
2OhGy5NbnmhJf/FS73yVTmfV0Cp+A09br7dAQWnqnNVFkhJILRqo8JEQxpdSP/8ARQdSszt27mVE
yTZ+U0OzYRT6lYfE4ycQ4WL0j3PyqVK+R7tiX1aHf2W4Pkm6h4NQVSV7nv8Aecc1iJsY0PsGQKkm
DbGnRvT1l1MvhwesFuZQfM3eXo42uDzszUp2JkudZXZjqVNjH/dYZljPF3NYfHPcJ5ZmE2HRoTWY
7E3QtdwMyZXki+Uh0l2WzzaoQ7K0VK9F6KMaU5qpdU9SqV9Zkmj4aogaesDRqU18g9J2YfDR903P
PTJRW3O1bTxUeDirWZ1bU4mK9sMdLap0pp1TrWgu/NyT5JHFznansCp09Zefp8JaepB9QfNaXaNz
VoKsZqclNlvSsq0/JYXBxtyhF0jurlMGtulYyyM+4nQR5vl4nIVd804g7qoU4BXE2tCdTsCPqPl8
NgoetmeeSIAJVp4hu/VTsiy2TdmD4oySUvA3nlYqXJrC5xaeqrbvNzWR+au0DRl4lOldk2mauJQY
Buqiy9SAFQfDQCHrpIzzzUj3HhT8cRs7iWinh0UkjqOurUO6lYWAuzlFzlc9l3RRwR7kAGacOOqb
WiYyu5X9U3oq9rj6ivT4RHqaKtPXxFnVOw7q4aAkXyeCjc11I27ouyoFHURyPfTcT5JCHPf05DwV
jI75ac9Am3OAB1cUNg/buB4hoVPJJuEDJqY0aNC18w+Pn0VB8NVK08/Pz2xxirisKyBl8p1d1P8A
ZZ0AhA4zRpcdKrC4AEPqBITzopHRndDRGPnTNSyRiuzCpK+6TmhgIqNY3N9OaiAJq0ZojZhePmtb
59enw1U+voBUpr9mfKHZfqnYku9HF6FlPnvH9f5KPacOUj/lyCtLWyYqXjcBwt6KGNvECXvTcBhy
Yy2lzmnUqWdj/TO0c5SSyG99bnORefMoewI/EFzgtPXboLvkvKMQdlBHnc0qSPCMc+aeXZQvk/6n
fRFrsmxilzvdHNYvFubbgBuRt605qTESd52SdjZnUbTcZ1Kc851NSr2De0UbZDbNK6tvh2UCJcaA
IuOQVeyvxAEPUlHzcWaZ7SNUGX8TyUP5cM+ngp6Gnom6fNYe3Ld5KV3erqsH9UU3/OpamvmO9pDs
/8QAKRABAAICAgIBBAIDAQEBAAAAAQARITEQQVFhcSCBkaGxwTDR8OHxQP/aAAgBAQABPyEZ78bl
zvH0V6lQhDg4qdzqp6lTuVOofrlRZ3LqdS4ueLly0ly8RcS88bub4rj7SpRuVHnecRy5cvMtON5i
xYtxYvOY4HiuaxE/MS4zvjzxuXiGRIRw3HMudzCXNsXHF/mXbP0i+pfpjW0vzL7udsvEX8wYIOJf
Uvi63FzLxfUamIsGty88VziOo7Yv4i304jvzK4OCEeNzqHAcENzrjRMcErnXPr6e+a4989x47lRi
1L4HrhYsWI4o5iajfBqeMxcRyTPsi+NxbZqCXuPzCVqFVNypVRqoxoj+446xGPbO6jRL+zLomOIM
fGVfmLExJG8fzj5xZealy5vPGM8Vi6xGfaG1qef8QZZHbE1L6uYOJvG8rUvMr6uVWLu+HMqVbiVD
GP3xr6Pvwc9cHmHFzfBZLZuDSMc8VxUrMqVKlZlSpWM8PGPMrPv6F8xhiWQeLi4jecZpH2lPozD7
T5xtGMrvEBVy+aEDMNQ0fxx3vhnmJiM1LIxf13B2s1GovjjU+e+Ky4x9z81P9Q/qUStefiWS/Dc7
hu5V7hxGx6nfMv3iVrcv3mD7/ccfvFqYzWp55ixfcuX8y56uMqMHHAnDMVDk1O74J6l+IS5eZfHc
uXjMXj/scXPtMs6lZ4B31KlXKTMzKlVKlSpUqe/P8YXlR4+HxKxgXGfl3PnznjGd2cxlxiftHH5h
5sJx7gX/AFBqa1DMunUXPqfDLzidzxjXmaZYXHVksWLmpd/MWvEcIyuYN4nUuvmbLlRPUcME3PEp
+paXLZlYbhKWbcytc5jhxusVL9R9YmY8f9nlePXDKVhb+5TuU8FZU3ldSn6OTV3KeJN+uOMHLpN+
K+O/0+MJfiF+n018RAJRUrOoEr4iBMYkoleoFEqVMJrNsId8/wArjJ5synyj+0q7j5x/+Y57IOpY
3E82RUJpI1MzZMxsrufbuPiNs6/zMYsRTEYtuI+NvX8zcpejErMruuIxahh1DEiMTMpGLO5W6lYr
uFX1DT6nriYZXHTNtz3cQk1XcrGSJ6lfmVcSVEzKlH3gKQT1wnWpXuZzGscSa1KE41KIBfqYzEq5
iVMTEDgqYhU7vjErvh+Oal3ipUqdyuOvvjrxtPhLMtznapZ45MWnEwsSDKIU3LXFS8asd+8LH+oo
vfCXxmPAwdG5jKj8EtzWZh1DGW8TLi22RvD5JlL2RT9ytzcsdfEahC5mZwhA6nwhUZWO4LifiVnH
cDBHzlN4ibeJjqFhzEhxLXUbHgYyiorCV5iWSpdmoxs1PtBvU8fE9E88KYruVLxOoSoQbJ3643Nw
1KzKgblTxMymU7geYFy8CD43o47c9rm0IxdQjaZz4wxmXU+M1nxnqS3jhtqbanoSxD1nxnwmFmyL
TiM3EquriHF/U7amDMfVRValJUqMHGlPBuVgupg4TCqmmpk3UwNQ5isfUeYsX3H8Ytx/E+8vYYQg
Ecoyb6+IymZUzwNVGszISY8azHbAuY2GluPNbBq2WMXZFzUpTMb9/iFSj8zxJ9zKIoZYlm5UDOpV
wxhx374+sMdcdYTZmmofZwMpkQpxPWawk9Z8YY6hJOfU+MPXies3uoes+E9Saz4T4TGfCPpPhH0n
wh6T4Q1xD0g71LOpaMWklFtMxUPgZWe5nl3XH2JMDUwxQZS+YmYmqhVlwZgUQKIBmAqUqJhCUlTM
RMrG8u4GIx3waYwIG5W5W8ROod+NMXxGMuKXuX7m8GyNR8LK+0uTeZs+I5eRpnRAVMGBuJrcP3Kw
CssAzAhsYi5RdsDqVqHpNPmHCQLh8IcFoKFISQW4+FQ9Yd6hJ6z4cdp8ISSY6h6wz1PhxrAysrKS
lalPEpPREEr4lfEB4hCl6lPEBASkRDrUHNEKmiUt1E3KqaLnrzCYgMRCoYFMpU95rUrM2IgCFRMB
HCIj8LgO5RMEyXcc6zKhNeJp7j+OBJ8o8Pc7lO3MJ1GGGKi7v4nYvgwQvCLK57O4Vb6gVSwW9/SN
MKcSDDJHD3Lbj3uKsu7WGPm5WNSsxf1KQEBASkpKSkpKQMAhIPqB8QDxAQEpwUgIGVlEoJZcrKys
rKeZSuFJ84RQmv0Bhw3wOA4CCCNJmRkpmEbuDLHZhmMdOZRUYvgtzPKVExcqYJNcrCY4+0+cLzMG
+G3uIWJlZeIuItTJ7gZeIxXth4S76grq5hi4jo+I0YQN/wDkdHzH4Zlm2YrJeIwJu5jCd4q/4Bl5
4Jm3HfUMMExcQ3P58TjPpc4cKTaBuEkme4EgYGBlbgfM+U+UTG2mfKU8zYzPlPlPlGMmrjTvhbzD
2hyGXrcvLvfG+Crc33xxmMy7mJzLCX3LZxiZ3Er1w2ihNEYTuWDMlnt3NQlDMplZuCI0MsZ64puN
+42lr9S+PEQMxMo7g80GL/1ym4l/Ev5gtw3WJbH0lMwYr6gxGTiX84KszqZ4ttxf8mzh06i8w99y
sHuV6iFf3Ejy+k8Z8o474/KPkw9p8uB7Q9oaZh7Q958584e8PePW58orzHLcOly/mfKay8c4vzxV
BsJ6TaClo2ioLBZfxLZbC7lxcbqNHSDcZZluFYESJATEaFRMvdw31mOF1Hx1DePEemZ8wVisbHuL
up1cvuXqA/iVlDuMe2XMsmag/Ef2ywN1Db5lMN4EzQE1Gow7CVuYqfcWZipSFWxOm/HAYkAGrqv2
SmDHpv8AUT1QsW34YkOdjCSpUf8AAcVDXUEVVNesSoeuJVcHjOey8a48ePG/E+UI++b7nzm8J+UL
cbLBsFhcLlMpZao5QTBXrixhPw5yrqBh3dT4z4THqbzyjDc5y+8SzqVOo5dxfE13A+Izg1KfiDc2
9RDxGrmI6Y/nMPfcW43/AMTepfwwVO4J6lfU0zKjK83XFkxBt7msM3KNRbiZgC1PXKjUFdR4rjiQ
3uVYlOruGXq0Kp+YY+oqW/CzHldUYqUKoDrxA6O8S3b35hY+8CGUJky/1EyAhqy539ZDko1cEh1A
7YYQjCxyTuMzwkSVmJKlSpX4lSokqd8Z4CBAhxDPXMcPo/zZpOkOEk9YGGOpvqUeOIeMCUVEz7So
kT1CVDcuuXT0yt6gfE9UPAj6THGX/UrY5Yt3n7cau4bj3jzL+4FM6hJwgViFIcZmGZTcscTfESY4
1uW0oi+YbZnMSmCu5fqLMx8pdKRYqd9DtfUqrBai+wQZ+imqQC59HpV4isPAOjMQCrEbvxH4qncp
xGWO4XujLOY2utDpH5EbUd3Xq/4bly4VfvmCxmEefMeoKyv3Fjy1Llkv8S+b9z7yw3z95QyjzKqV
AlQnEyvUKgErCqhRLzLxMXB9wZYMGEk3L4ntxhiIipSIiPELhfuKU8Sw1HLUpA+5QG4pGBBO503L
p3MeZZU21K9yofMZUPmY9wD/AJnfAJQwT1zPGMpUDO6mHcIuNxvuLDrcOZqUzFKs3BCARUTV3CAq
ZW3yzU0RcxEz4nmKi7kP8xuC7B38wGQtrrpLjV9eai9KXsIQVDSYXEGZadw8EOGYJDYv4+iuKle5
U7lHmM1MwY5FZ8xn3YTcfeYsv+Zflly+rl5qXKy5rLly5cuDcNy4R5hrDviI2gHuA881PPAgg9p8
oAlPMpxI98+fF56fMQdzxMco5T3xw07l1xZrmPctu5fcG5eGXeJe8wfEHE+2CXnU/wDiePEeYJUo
MTW4P/M9kV/EQ8xI6zBSHHcxoahojqZI31A8SyceQaNsp0Lh1o1+WEM/te5cTcQp/ITZpuW6jiIp
nNwcESjGtRmyfPUsbRWPX1Jc0auVII+HipRGalSuK3KjO6gXEepp/bM9c9+e5d9y5cuLx1l3Ll5l
2wZeIMGEuCwkkpCCnAV8T6DJB5h7Q4x/R63NNx9uLKYy8ZhwjDLlxYffjX5m/f4ghM/mWy9y/wBw
bnUMMYhpK2VCZ9zTg+cjKuaRvEr464upZmqV1Kz+pnRX5nWtQ+Ka2+Gj8zW2F1/+Sj7xjc6hYnQ+
ZclQ4464VW3f1GpocwCiouxz+ZSLXLyQOKSoB1KJU9Suon5l09EatTF4+YTc787Nzvz/AD43Lgy5
4wZcGoOfUGmDLzBzPWH/AMwn5wj5Q9plPlD24/KbbhGByCJabcmm+Rhy3LeY2j2j7SvuNXcZfaNo
+8eFvi3xcWXjBLz8QYMvvqYzwWbT1YyvGeHeXN4IxmjqNkJzHqmJKm4cfEW5XfzEp7+ISyYwfo9x
MBKwt8QoijA2seFwqKoEsmbjItv+MXaGR+YcbY7g8dy5Udq56H8Srr9SiVCR9y9y5fUs8S5cuW+O
L6grLgzu5i4cDLwQf/vG8S5cJuDLhUm3EnCftxMeJ9LzKbTy5/abTDiy8aqVtRn5cjbgeK2dy86j
LzLxiEn5y8ZI8Zwu4bg8Hu4i9SjOuoms9wMswzPw0wZllfMQjxKpFsu6XlDVG7n+o5Tf0/qXBsXV
xy0r7/y7TAV+Yuoy4y5cvUpcq6gHUDE74nnjvhzYSyaYalYmOMe+anfJXBPU6g8EuXNQZdS7lwcx
ZqXLxLl8bzLly5cvEuY898VuUzbKZV3Csr5lbu4l7nTcCMuYjFGLLPMv5hU3rqKZi+JROolPB+uL
7zCuaIHmASMqH5pTXzBkIGKhtqFi4XtBJYbQ0RcLUUVQRDBYHmJtT1q4cNLzTh4qV9ZLvjFQaYI5
jFjL4uDCXqGCR1iZ6jWIeIeMwzcTW4glIF3CmVykqV53x1DjE+3B5lfQTXF54uXLK4uXLzLly+pf
Em5cv5i4qXxveZrxXxLxO0tm6JDXn1FHaHuxn3Vx/nMVi+5cW5fFkWXOpp6lblRq54iy5iwZKoTl
iOpZKjzUzkMYXbqdyb1QK8ZC+iYuBA2R1LpY8DzmZMgUV1wT7x/yDUX5l8986Mwl9wmfErMZhT/y
VO4UN8blD24LzMMysY4qb+ZUr8wlZleea+OSBxn6D6PvO51OuuLl/RcuXLly52xl3Lxw5JfLJZoJ
6eAxvz+44ma3HXD5nzGd8/MIEr/mEXTxiyZlzfJBzFi8TNhMSrUMEonj03AaG1rdSrWblaVMqGsv
/UMRGd1/+A/wOmUPiH57huLgrVx1EFZge5U7jnPfUeLKb74XxjjXFTXzN9Ssw47lY4NZnUud446+
nU75xMSy+fvzvi/cxz9+P5hwxJdAqBwpGpiP3+Z1F8x+Y/QQ4lfmKZ/uXHNTEeppnc7gSqZioMH/
AOylM5g9bngiTuALmG9UXawDxoDi2D0JSBcA8oXX/wCE3w/XWylgMFYHpnUO4U7+eKk3mWdwxLv6
lsUJ39W4SuT6PMOH/swl54vjv6bx9XfGJf09TcxHUq5iuYMOYxjqLGmP0kIUfEXMv3L89y++4xlT
qdS5cGdZCnHSWswOZe/WH3YVtzkymFs0y+URKaDdYPMowoXX/wCE8/4rjTnu5TiWrB8z44IFsNUc
5lV8ysXzXm4nFO4QJWWVKuViVc1AlcViViVKzxrh5qVKlT7xJURiSp8ypWeK4eNXH7inaY/MGcRM
y24xcx+isSsSoQx6l+LjbGdypVyqlSuWCku2FqFqJZmN1TRrAtS41TZU2ngHQ9R0wrLviuH/APBU
SVxX1Usuq2OMyc0w+Jvie9TSFqJGBzU1N64r3PtzWM8feZ+8q5nisyu/qOK4DipT9Gue/oz1Ncft
4XZcDePFziN+Z1LjxUDEqE1Nysy8vGa+0+874cMtjMyuDbMRGqKweeoZIKUxggQsYSnY2Xv4hSUn
TwGY+pX+cl2uevxHOVKl4Jg0Hf8AEQMxf+Iw+JSbhdzxIDipaz5jXjapWPtE4+J9obm4fRXcON89
Q4zN8fxxmZhLxDU61x9vouPCx4ZcrM1HRMrw56m+o8BLcRruK8aizjpxa8XLhcIb6mPUTEuffMYs
vEuevouZiCgzFr1EDoNtYCEJspY+ITi6/YzaCLpqWNtbCXMy/wDOfM8HUNZxAlDKzfVy61LJjn2o
dvMsuoV9y2WOvjgCuofzCuiDuWvm+0fMeLmYfMJ546+jNz553zfOn6G5cvHFy4sv5l45v9R8RZc2
wNxYuF4YCvGJ+IdbfC4LB3Y39Qw66EIlxEeNB1c6Ylm5Xoieor1KgNwuPFxH6juNrzX1aoMFzDDL
Qwt3X/ssNKCn9yh2irKfmX3Yl2USmNS5cuN/5y4uSbf9zqaL/qeobTxLCeqYo2PUKRoTOy2+ElMN
rUNpcdgnT3NOM4eazOyZrm+L5Po747l8XL4Jc+OL5vE7mu+FixZfA5ojijlWzPLSCp1gtWo7t6NH
xdvvAC3dIqEwA9RY4TQnazEN/JLWyr0nkn7c+vhbr3NZ1DGoLUtYudR/fF2FpclPDh1ybFTQjn01
KdEIaAMfEQJifOYIqq5H1AMyDL5P8xDMzNdTST/rn2xKimcWuLDLNHuZk/lgwk1MAgs640wRS7jq
HMqt8VmVmdSvc6mpkj9T9Fk7ncuWc3OpR9FzuXNGJ8MvgLSABc14r3HLxnxQJlZm+Rs3R0evBG1i
a8A8feZiF6Y5aF27jdoB0MIqxwYEI4g8PmIDGOJ2ueOJuNepWg3KzU74TcqakPWGfU+OJUd8epVz
vo+81KgDaBSHXmWnJxeKCAmoLCOYMdoFmvLHUrZTXjfJy/5DjqXLOmdHKzuY8TU09QdeI9ysN1Ut
Zc+Sbah4v8zprEXUfCYmcfbEv7h3CErip881Pnkq5c743qX9F9TO4cahmdTzhneeHk3DU0R1D6mW
ChCuwur5Sy/Y/mG1zaDTXrzLxM7PAlrH2gsZZZnF+UEgV5XubJCNMdbaEWM94aleokSLrzLlxJVf
MNT+OFjxFuXuEtkqDEMjALaDaxnMpdarqCgW1jrzABvJHU1ttB218w6Wtnq+4qDfoiU//gGuLuDi
faDLpg5x+Zl8Zj8zCOpmpfmXZP2l4jnqJjqUZxcr8zriufpiuHxAvqHrD0mfUb9RR1ETqVEzK3ib
eD44/f0bYbl+ubsjL4u46+JfN8s6l8EMTrUZvxHrCvUzIwHLvb/R9pZT4W9tS/sE/UCGmYrYByiK
KqMIwpVpSCaPzPlA+YxIlxLmuSJvEGI6/wDkuX5m8znjLzKQuPcCFplLJ6JQAHazIeI2HeXB4AeF
9Rd3vbohO6MPxFvk/GqA3kWo1Cxs/wDx3dxZcvyQTqO4RuVYhjK7m8d0l9S42OYPFin3gUyyWMyT
0T0z1Qx1L5rxMuoASm44ypWJWYhXNSp3PPFSon1Y419H3+kIE2YgHaV7leEfvK8x7zTMqt2Gsz/r
7wm4OipfHlhRtilyephqlSPiHEqPHcMooQ0MSFxi0YB7galArqAxWQPmVGWa3Ew4eL9xa46y4T84
+0Z3wLZ34MYNZs9HX3hE6wnTWUhdjkCdvmCVXMHSR+WymIK6qFxu2jqVdgGV7fcTC0t4qV/hr6K1
KlSqlSuLmfeIGPELG8wp3KjLDXcyNwptzUbaWErUWC08cGDU7qlqSysSuYdQbycYqh5IDMErMqJm
VKxxUqV3NsCdRjznivPPnj1Opu59uKlQOKOgV8GZe9fVxkAtNVb8y1CFDXoB3+2pW2paoX0T+48t
BC8FN06YaYfbloU9Bd1t9wXVQ7R6tmMk6vaV0Pb6gm/BsLvN7iuBUCByuK9Hcru4LtS6ZniF9RWN
WrJhKKNVBMDZCuokrPXE4aPiYXEyxSsWfeKM83Kt3CWS7Fa8SnqeAhwbwnj/AJbBJcIu3OY0CtVZ
KUS0fcWoslDURSp4xuFMkKj+5mcor2i8XWnD/g3iJXNSqgVvh1Fl8f8Abhz/AKuWpGYor7xVx8OI
Caltyy1qPfub7juHUECDZDCU8RA3Ekw7h+YWcwnuLLKnUqViJKiQJUqdYlSpRK4qfv6O5qdxlcVi
Vwrgh1UqhXtC79YjRlohtgtLfuwOAjF2DdVuAnE2CPbatD1EMBoabz3VMWD6KXQb8/eWLXGvB9GF
fMO4Ktw1DIIS3yfIDdD2vluAxuRUNKL9eiaRMdgdBevcoVeX3vKirvwfqHmrVGQQAGM6U2THQlig
8J4xSlNy2pPBURrBpEwSpvPOpkLLzh8xDLzK1dYhPUQf7j9ILshoTSZlzNmUhKQyjsssjU0ZSHpL
sBEm4OTz6hkpRcRPRM3HQQDYi3kqEl86x+slF1uajwEKJouYuLfcWLL9/QXcC+5pN0afMUFMTHUB
HEBvEe6EwDMh44cFQ0lbue6Fe4VbheZ8v3LjcWty3L+rzKlcVKajKn8zud541x9p8fR3xcrM/iBK
eoSOHtjX8viMrOCLdvh8OazUXlNYCooVYB58seRFKVahmmDMbeqHK5l4pwFPswLHuiDW3jbteNLG
VQb3Ro+suCHeGfao7q8fBG5rD9Fot4KNbuUnl7VBnHctAI7GkZNaym/vUyurZsGDQYGdi3AZiANV
oBwlKCZobrGGMHjcrKGRyKSkz7jVjArB7z0xFIrkR3G6u9SrgZhGl7QXRE6FQck2oUGBM7EoWEdT
DxSx/wDM+praJ3n0S2jop+kGBj6JkZGVX3Kes9zqhnrUN/7iQaaJQS8AK3QS/cA1avav6j6BNQzc
XJFy+WCkWXs1Lji0P8szIGSOPSCAUYx3B5R28VDmae7j9XUvxFi3zjvhxGbly/oJm98T1jp/Uaai
pbj5hhbuMXwSq4lzgg/EMl8RnK27lRK+45bqNnfxBvcHXMu5dblwcy+Bx9BwzUqJmVKlYiSllSuP
UrMqViVO75uhZYDtjHaNsG3+c6lsRXQ+QYKe2rivVSAiwVlt8b3CwKPA+ZYCRaYl6oF0bU8S1ycL
09V0+0b9Bft4B7hdwhPmqttRmB9VocDnPWjUvWyraxxLDyLYr+PJNoKHSiOyjsmdYZlUYjdi3R3a
7u8eiBbStvILgwpi4kibbMcpFKYq9XMGbsdMC1bTZBQq1sYHCWfuNodjU+EmcfeLQuzsHRCmZoCx
9SpTQBN15lNKjhImq4ZWZWka2iopcKt0m9hz4PiNMPq5b/8AYBt1q5n5vcun3zSfiYOgGcbGNnoY
uHpbwLT545CbcMqxKqD83da/dRvZimAMX+bgzp2sCy5ITqrPfshPgqTB4QhGKqg4vuXDAYDWyU1A
gbiWxs39eT6AXPF1j+ov1d8VDaQXVdQmoSOaZegSLlVEPUFGibYqgzEAhmK8xKUoI5Yj3NE15mMn
UxUH3Btgkvg746lcUanc+WVEJRA1KgEqVUaicY4SdY4a/MKlthVgbTs9dSsTBwCssNKp6xNMKWUV
1LC1cjRPn3Gpm53U3l/rUInLovSOlF9ysvSC7ucjw6l2Wk4MFo0U/UbOroKDeHv8EfCFs7StMMot
UIHCut5rwghyT9AJm0oaMQqeBtxb32wRnoyH/wAmRS6B7r/7UbQBfpH18RCFijVDP3uFC6Hsbgi0
Wszgz7mJeYqz1NIrdUTBNcuSxDnR1BqGxVlMPACmLtUWgOtkBpPnuDlF6iVfojcdAKeaQ3mFebj0
ih3DaF1i4CBlM4GKay9i36mUB57JWkFBo+57/iIdJYul15HywM5tqZV7YwOBgXYdQY8IPE1AGEMV
7gsvVAdRUIoIrxYdYhnduLZjEd8vN1xqXNsJWWVEzGP1lEGdwEoqXGjEeidzK4l24hJqbQTMkNvE
W9EGMFYlxT1Fdykj/MGPqDxeYMHOocTjrmuKlc1jUqVK41xqVKzO5U1lMaHhO4F7Vfk+0sEok4Qj
4AV8y2yLgtnrENoCnRa1aYI0GiqsG8NH3mBcoBsry8JjuwXI+3A/czsl7hfT1fBMYwkWm2l+xGom
k6yxjSnddMNBN9ZEPVZp/mYZSGLIlgwbF9aZk4JpsNabln+g6sHH5uHxaKGwQ2Pi3MFBKQDrEwsp
k0l3LhFR7YAHQocqmvGBNSlItgg5VJ1Cob3FXcVvUyRMoGjMVAbS8q1QnevmGcn5hkUGC2KIaOUY
e9wjuPGL33LgiLVMBtWJTV4Mkcp8/wBQHM/8Ha+2EYzFXKLUixCIcFPn1NjVaG4z00p1LwGr/iXv
OWImSBmai5itx+u/HNZcB/M+MctSuP0VyHU6wYh9o5fAoBqALUIoZq1ypUnhAOpvQlnnioy9o0Y6
rnKjNwZcHMHMJfU8QmuK4rPFe5Wb4qVElX8SvUTMTxEpjDjhW8JkKf3OvzKywosMLq9GI/AYtgbd
v8QZKhagvNqlgArmKy7ZWOQotxaPux1KC6QMuwt95IdXegNATpMjlsLk9y8kk64y7718zJhQ+J+P
3EGNUVx6+zVQnuTqFjZd+oJsUBV6F3i4QGrZDBV1nPj8y4uZyYNsfdbf2hjD2nZtuDIa21vESejI
aZdLeysKjJgpbQdliFLKSbhKEuCqVRDcCoH5iM6SfiVtIk1qjtVXxPEZbWyPsevVzWrUA6VHVdUm
P4lTjsDxFYs+ZmSQQ6P/AGKpe4So1W5e7apm7QRQKzHvJbUvOJvHPmPJzguBqVnEDHjjh+sTFGYO
SLEccRl/OOQ3BTUAsQAYghKbqIsO5WqEL4Y3h9cxVArHO0xda53w8DBhBgy9HA3O/p65qvjiuGJE
iSocSkJ5qI5LfJpR6YkUw60384iZ621vyiNNEO8yUIK1LqWtvJb8y9jRnsKxo49QlbZArO6ETfcf
VlwM06R9QCzqRuB23gp83ABLpoBOnGuo9C5lO/ZGA2aw68Ww0BSBLHw/mKpdxyP9suuIwBvy/mNF
tBCxH/UfQGFq3TxKiws8kIj4G5ci5gEGIyUHEVoTFQr8S/z3CKYN3lKtM9G4oWYmCkZULlDkInKS
lAa2+C2W6dYq02Pi4r4Hp10JctZQuu49JTGepmosadPUzxm2dR2ChlI5rT2h3Xufeoy+sxM+uK64
eLjzhpjjolV8wwO/5iAG4BgpfpN8GIy7hbmJHOLeRlBS5okI6ibIjFxZwIw6mGn1DiNjVwDqMXRA
pgH5mRwVTDEJGMTMIPuGIMGDBnf139V/jhlT7TDLxDxiIjBOhQZ86z+ZRiQRVAaH8Q2rWkaubIQR
dMRpc36h4BYW6MVhyvfxLauMXZzZ4lYdM7HkPzBAMyrVDzW5UbkLCi09iypopkMApURzvcZR0qW6
fV9ylL4NN5+0UVQUDVKzUoQBFqdQONwo0cNPzL4mYfsRIgrK8kEjacyyykg4QPcyH3sxwLDVQpmP
BHeUFoEyuBmOVEM2Ym4TEIQgLsBe4QuGw3UZNqVXG1f6iYghdkc/og3RM2agJbYND+IZSgy/1Lo7
m3RDufmTwESxAfxEJA2+oxbldQj5uX6lzv6twM4m93N/iZww35goV64GIG/rI67l5ofGpdm5fuVC
qADEcZRTgR8XUl7OoOMZ1EKgwtkNgTWeuU2wYZgsGOTgYMHEHgYJ9Rzcvj1xvqPcZTufBAUBEroL
LhtQsDAezs2j4YQBUMsClaPxGQMabD3AlUw5DlgcgFrdHmVqoCwdwNRyteggNaHyPwgiS9L7dwpO
MF3CGGSZ6D2eJXQpoZKt2GHUPBbQPRy6+8PAVytjyU9QudK92Diuda8TIFy8VsYujaMnZLKMleai
BzWpdjeLYysPEzNFdysLQuJVaBl4rmCok0gWUMrDv3xgDuO7WPdsEl4XyrEfe1Z9g37ZUxNHkqj+
5VWpkaxBiIp8oNAFKoDtjbhWq/wHiKWGhwDwHUWMFAOfia5iy/Uc8faMuL9NTMLwcy+4x9Tz/Mr9
RyM2xKfpIS8wZbXVQbl3DMUolVZnaYjeYtayoswLFszJFCENxMeyWG5i7ixNktGXs1mZiE13DTcI
QhPvLzfBxb54eT5muO4c98eDc90UybMkp/UXRJS8TQlZBYwBTtIyCho1d/3AoqWjrv1DmIqjzKZi
ZAexgArFS9vcQTiCAYy+ZYq2m/s6hIrSQ+Q3LKzUoHULaGFLvuUm70LPUNwA5uq3XiLlKbais6/M
yIAjQ7uNzyMU0d67hxyRflO5SAro8y1BpitKrKfLGyW4IQ3YgQlW9zIAOyUtjUaA/iZYGviKFq/E
V5H8SlYoVppKGQANdmIuqbWBXcpC2V/olxqNBX2jVvd35dwqSWx5rawgGE7FV+Y2jRVDVWxlMx5Z
tlLuKbgZjwy36DU1CX7qD8w8mpj3Aen9ztxt3MmO4PcOefpJeOLgwcwyMoYlRWo7W5fK5lSRse5e
ipZXuE8yutzNmZgjLuLENXMczhVhiRzjmO4QVhxcu5cGEMcku5ct+i4uPEonujbIzNuVuHKD2CTB
fC8WwNF7CiIRSVLl4uDlZSblMfSysJupfgyS0tNsj1SVuFFKdy1siindeoVTagA9DuOYY0q2o7gu
gAseUZtJeitFQmisWNrF+BrU1bmK0phde4/esZ9y63vn/wAjWpBL+YWAcnwy5Ob9QmO0paX3LwRs
dDy7JbFidwMtXqBKSBWBFUD7xzYPiohqoDR8dyrBUoA2xSygAU1Z7htY4GnzNuPN9koMGzKPsB2a
9HXzLSKJ6xq44o4h2qZwUxv/AFGQYv8AUY5Q1M5a8cXcvnM28aiw+gdwLCfOVJeJVjpYrf8ADcut
QZUJhFAxWpWZRuHXDuJhloxiOMxL5g/My7iqm5rLlWdRk84aYsWKTcIIRwcy5euLl8eifPHcvE8z
q5eJ1FlRMDPdLmrmLllpiVigAeaR+0s8jRaA0/ym1yoOszMISxMF6gkdRSxaC3KJ+4SxipPLpl1L
XxDUdLDoTQiuYZYDASoIINnSUNdID7RxHYuFs9XfcKWhYbffr5gkLDAxMH4TWPD/AFNeAdeGWVE2
Y4LJaI3mkcEr8CtNwAJTqIgZCXAZvMa4C4UWVryRISy/UNyXBnLCmk6ZaDiFW3UvAmWpkPQBXy3P
hEF7B5/9hMCjZry/uWpcpOsQ0btR9lzDDA4EwHb98QHAOg69wt1RH7YXh/mLTXcecLE/wn1dcVs6
pRrzHgO/ormpXFerg1DQRUpcpfcVLEkvlCw2+oKS5QVpE1iNjHHRmFiZghfgkfUzy8fxF4SGoOJ4
zB7gwm2dQ1ji4cVzjhzCzfngdu4lStRLts0A+Kt9DAZl5yo/BVwSAK0ncaBc51lpm4+9DtmQwUBi
s4id8CvslQAAoXolZBmE58KkYk2iu7az8S0OM0m7O4CTor3TGK+0yG0BEeE1FtJXreGv/Zad1HGK
1/5Np6v4vfkgdwHI7Y0C0KD18w0BdFRMiCImfCOomQTCS0vwXMZDwZpckTJW4UsCWBdnqND3Zb53
uB0D5laEuXGAje3SPLeo9quWw2znyxu0J3Gy+ba+0YjbxA3U3JIbdvAa+JnfYG7WXlKoPrc6rxKH
LE7vEveea98V/hrE7jCHmHj1HGL9FSoXgnFRvH3lpdqJ0QDajDKm3L7G21FWLNRJTeSUWMGnMuHM
s4jv+YmWacQrEEg+ZiIW4tyZ4N3K9wIkuuc43wNQc/4D8R495ncVXEzE9RKQ2bebat1coUGyGBaz
7QMxWNE7n3IVoVul6GYQqDuqz9oXWmPWzdRqjVqnogR7dp4vT+SMMu5GFbIQWF2/mHmW0HKy6UBq
t1NZo8nuojdQfCd/1+JYoqtZ8P8A7CwqlR4rxHZES7dPf2Yz3rFu/mEpGDTXXuViA4IHQ8rK2Ce5
FkOUP0Il6nuWEWvMosAJU1ON5jPT3FcEjxhYWTGovKgeZi3MOnbOo42OEFCnglFygWlZsAfJAqRG
l7VthIBQ/BuWAdKCAc32R1qrYA8sANCr7Hn7sWZlrEdkMk75+3NY5Po64qN4gr1Fx4pmpI7cG6lz
Us5J5iWNRcWkxBFJAp/cmlLRzLy7j0hUAwasbRjncd7gVziV1jXfGtwSFXEYzcvZeWx4+eDcJ3HU
cXE+ZWEuG+Di88k74xxiGyW3LGDPVxOVfy60+esQWY1sAth7rPudAwEb7/eWqbXneop70TZis/eL
WFLBrPUJcs27piywTTu2WP5NRDtYV+Mf7PmLQcOOya+cRyixoGkTAjlREq/oaB8HUcKHjTEwMdDI
Ybt8kuwVXh0vZDTRK09j4YhY2dikZgsNd/M7qMB4nkKhDw+IC/cobgkvDEBgjDwMF1crRWTqVdxc
oHRrTMjX7StG67j5LJvNmQSGazZllfBgV0bV/wC7iaCdeLVhr1/cqFHIoS9mCh36h66dO7gIFtcA
/wBscANrcnS+D1EXqHcz4jzmXcCLHXOj6zU61Ah88Zhgi5qLOR+JZ1PTD6J6f1LDUO48CCME3qpc
0WhIuCzEMGIC6IZU1BmyVO8zNviOtwvNzMxtG8y1/Uqu4XcQbGjUCmax4gzWLcMRQSXRBmaFu51x
whxX4ncxDk46eO6hvxMjwhPRPLBANEtbB+BQN0dr0XlhtMEF4/3E4moLyoY/coZsfljMFgU9vBho
/wC1NwksdFMf6l7K2tV0MfiORpoTR7/JLsRvT0P/AKXFV9GNdn5IArsrVoLkfr8wOOAT9mkVbuzC
TI9MUu0vV1sjHJ/tN9AivqYqQGr6jqwNPsQq6uOfCWx5uogm2OcNR4TwlbRx3xXMkw35QGIiDYX9
wAJiaJbV4ti+sqj3D7YwRbzG3tlvDB8Pi4RstSD+GH5lW7kLrG19Qg3XCofaXmsMUDpjgmdvFXL4
NRMwI+OvpD1KzczYJNRHUdq4WMQDxO+iC3RHOpk6nUkr6JXGHim0ZUFwKGoQIAkIJRDRikwzuYsp
5qe6BnMy5tcL1BmeoLmA0VHRRmUhIGYXwwTqWeoekWYjTEM44yzhEp7mkCHU/HNQgcH2463w+p5z
EzFNkGeuHY7Q0JsvrAfiVBUO7dj7H9RymtF8+okM4V1trBCFM27yN/iDTZInrLuJmgMi1m/9zeCj
2dJlf4lcQuvSCCQAO/X41ALRM1t6Y/P5jhLGGOfUVW2JO+2PJAsg1jsPCTIW2NrqI1rZMI/6gSIS
Z5e4+GHzLJQ+CBPnMKslFMAKNR1mFu7jOdoqzMuMojoJuKFqJs2TOO38XiFhWxdP/ke3SXGrcXGU
hzs5Vl3RBWxq/vGAB92eUnb4lRG18Su+3r34I7tF+8NxmuXdTFVMTuBbD1jMNEnys8p7jjmMsIdV
KdkxanZBlqoXWVkeKgDshShww73G3cKipAxtRBauYNIaRGHUG2ZdWK0tfgjvcV7i7n3jp/1DGekj
GoRyiVpBXBBIGoGtwlz5/MO7gENGJeJvMeeIxCEJfFw9/S8PDmMbk7JuxEPr0q9MrgbBowVZ1j+Y
PeDq/KWyltglL3esQmAEXGrGo7OJSem7YwtCaZy0H+ViFmVV7cj+P7hgoMolKH/n8wEwUKjQ4v8A
iABDjs+QSu4Wtx8ISJSWBpgRej4FmYXrS3LZZGwdfqBqRa7Z3JEO4Je5oDrudK8wKKgpSWG+oFzU
ZKYCNmpcBBrKAMrEiqCFNhY/BLV5o7LY+cV+ZnUDYKzLwHFVlOD5T8Qdy5XB5jRsyOmj/wCSoUKg
VfxABk3SIEPnLq5fUQwOgn88Bi+DXH3i5hH4hDmLXDGrRMpiGz+kuBgmIUyQsRB08ZeYe6GRZm4t
4YI0tSjAz2/uDr1DyYQ7hTR+0csftG7hqx+0eDTeC/mGjUXHNwCsLXXF0zUcZfEBvMzxpB1BRcGt
8YXD3nzxMO4Z5hlueMsjkgKgvqVJBUWINw+ghUPqc8eY/EtJgm/EpICsV6wCvhsHxKOtSHsUX+IO
LVymFdX8RiBADVAbr5j1Ysh6P+4IWksSWiHR3mZ4ykfVD/bFI2FDFZo/VStC3sG6P/kAi2Kfmoqo
YXd5qGpLp3RgSNzPu2QOy7KWzcxjRkiWCAA6mcxjqXauPgg0iZdVEmLnMaf6w1qHyTJDVlwC9aZF
sYPdSphXm94Hi7/mVK4zC/Hc2ra4b91KAig0DuO1uUp+4gAd0yHVxyrs6cyuTSmqsqy4q2oo/wBX
0W8dc9w1HcJqODufKhlYlrogVsgapa3glXBFrDMIF7/HGG2mbIWcymGd3ETcp7JbpjO4Nu246PM8
BYZueZHChXULxZlvctueK+4ljHeB/wBcYIr3Cx/uatshHNPmF31M25hNpbqP3CIabg3L9xe4g/3B
zHnhIS5cZD94SXl8dwl1iMY8aTIhJiQaagq9GT94+8cYLZZVg/hnz2DzWiEA7XfaBIo6dn5PvAVK
D0dJ+iAAILV6H9oqgFHeaTH8Qr3TK8WYb/EvQ7Hd0/PiKHaxZM+5iwLHj+JvUQLofUAYB3qw7QO6
vqoFpfNiPeihKq37y2ctiaZO4LE3ZLsRm6KLsIWNWHwxu1LK9S1qXKswWFrbwfwbuH3Jjg0p6JR2
m1R5oK+PtCfBtwi/diqSXmLOXyyvmwhnO1qXtas9p/MRLapY2komu55tiXsub7hDLiOH6Q9z4hU7
ikRRXnAbYEagTggkLqE2IqLh25l3ctdxjsmGHaop7m7Oo53BRBSM8xpHAzvWp5YGJQEZrHries13
Hq24+mL7jIeZfup/8uDhjjVvcVuoOdwcZgt5Z+IOq3xVZjL3HAiG4pwLZFPbHDhY45wyj5/iX7iy
7Jc8x3wWJnceGOEKI2J1KXoMAowj1k/aNkKQpV00wNoqH/rjRnYg0av3cfQmVxQjcQ8tC9FJj70S
/nY2zT3+4aaVmJuzH7xAALkEyn/kFdOXjcqamQi17lm5R4ELWjuiZUijT1HyMENL7ZVhrqGkoruE
tQYQiPXohWKniUwfTUZCo2FT7YvZMQBdwqAdgqD1q/he/MQSEWgL3XmVxyWE+NxrlyI9lg6NR6uE
9qMr7ZjYUqrgsecSixmbcB8QVlhqD5c9fMC9U6B9D49wW7jabjTi/PFRhwEzwZKDZuBAKCiCeEBt
sJ/0g5DEpmPdjMh7zcEO4e0v7m6L3g3NUB6hjqPepjYD8QlB1qHD3m0uEu55xz/cWWxWWyvWoSob
neb45BTiZuuDjuPUyzMxAoxBepicR3IR14gpsgcKuGZog50Q0y6eF3CWWKsuF7/c3HXcyJgal5qw
3Kohbqp2RCv9Xdrl+9sIiwra67jr7AtUE1cfCCAXN6P6WK8MFAr7vhJUuAqvDDUbxJkPRyf3M++y
mb8MMMEoJg8ouuot6iUc6pSDBKnKoRSklmkzqVbZCwHqUgKoI6luGYmtO5mBT2SrHWrhaolq9saw
eyXM3cuXEC7HdEEGK8IKbOv4zH/VUMhSrP3AGL2bDUa+GVhkMRVisHazP1wDI/aKy1CoyxfujNRU
kIV2nKp4lSpsltAlElR3DTxUuXcC4s08OILXQSmbmcPUTpcQg5Zc/EsWl3G3SoLZZvL6zLpjUj7J
izcO6xWLDepeGIQaiQ1AhFzNGtzIwp2+oANsOb0fzGUsdfed2Ny/Uu3cbN5cbgpLwJ4gzGxlpL9w
VxLanT98LjGVvzCqNsSwlt0Zlgh3FHMWJxmOC54ZhjwzDKq5zI/95g3CdTcGJgy81ahhWH/VLUyN
eFqfeBeFTDeRbFYzMSsH2gDiyCqsZA9Q/FC1GOvncfFqU/7hIgZUOPA5lWqW5rof7jEhhpypgJZB
ZxUWoAWxCX3jIMWHYXfmIdxLSMdutViAQae4ZXuKxV4gfgw1Zp4jQEYolYMnOdw8pH3liW26Jg69
5spgZYCeTQDAfiKRShkAbYI1iX0Gh4zB7BLTRuPIlgp7N6lqll0qeEz9yAR55XFs6ncVzXPF4+rv
mj2N/LE84lHEMjmXu9++IVtQB3LQzFs3Fa3FOalgjhP7izK3v4g1CYhjGg4d+IrirhjZruF1dy2D
IuY6uomJf/2M7VCLF6nn1LFxhqf+Q2+YsxEZ6g4n2jwGItp4LrOhg95jjlxEtiKcR5IniuWPESYI
Zz1wrLjF3B+JXAhvjTh3mY0TvhmiEi+6jrI05ugd+4Cu9tnqUY3Lhd9sQyGxNYg195lQMrMCfUxQ
ivgf7jC1osO7uYjjsreJdKLKYB6FwhfVLrxAZHfiW7bJb3iK0FdyqJd+I1XvccAEUZw8yyBUztqJ
zLjUoS7CttMD+WGtAb7eGjpYALd2tEaK1isNWf7Yvbltbar5YDgEOsFcg8D6/lG33KiGVymEYcm5
c7+gkGAtzKO4jdZjqxsxB1K+EGLIQyrrEcMEwLQizUYZak0eZiMwklnDeMeYtkdRauN6xBy2tRc6
jniWy8R1Ud9VHxUeZTDvHt5jXUG0xiakW9SvoiV1PLUIKRo7xBC6xyLxCEzBDtZK1iXl7R1qYSuZ
xZgiblZIjWItal3rEa9Ro6gQg1iXUUZbhs9zDqN4lTB/8hxLiaoFKp9S/qKq/AXLywROClgC7qv4
SqtMiJX/AAQIcIyc/KVrvlqHZTUz5jSr5lALZdrhGVty9YF1+4OBUdMpTX7lkMMUU1AtXL1E6xFQ
LUG0UdmP9EJRWFbt5gXOGW3QXf3ggILl5MbYhCtrzTa57XqtwKaqPBeImoscqcMyJ0RuxZcNfTfI
sRqoYp+Ir1LAxMWoR4SpieucIalRLPmLbxSJTNWYYbjeXSyeWNe58Zvgg26J5jhX9y87hknmUuty
zr7xWWWHUCDGsLrPlOt3EIB1EtqdzV3Nl/qKmDV3NMGGhlqL3MiHaZUs6jPSXPiZNWwnKXZw9GJg
J8JhZkhqUy+HiKCXkzSrBD1gVWY/JSsMkvPcZWwfuWLRa6FYBHDZrF23HvFGb7m6iMqxKAr4lLru
Ew7ibjZXFhSK8zIxA1kgnBERSjq4jAfeaYnqMyhisCk1bAwW18wYKso5Yte7I0dNyYHvPiLjylS6
FwH2hm4H+NiGzGVJ95cxuDyoz+1Eq31DoVUs+ka3O4TeVNHBmOH60uqDc1CoSoQK9wwgl6h/MZcu
TAjxx4GmI7mHcG9zKZdxt/8AY3uOJxcbwHB6IsWXThg6l8R+0v6jnc9MY6qJFxBitvqDh1Lm8Rhl
Y1KcV1M+JirMoqYoNJBmMh3zDeMXjUs/BMW4ZRpjuZS0LgrBqIVO9QWzKXdQv1GCXnCaM28QzAuV
cFYjkbGMIM01Yn+4JbJlvRBiNCjYQA4Ih/siCynYYIFePi9yloPuY7VxooPvEsrqeHEKV1iTUaRu
NYC6YtkC9iQTs00sDl/UAi1D7aPxO8LTZDIHTLRdK0kDAxQpFA2qepXgUXdDRGMx/OB+7C1jcW1t
w0KXHs1WIiZfor6iAXCFV9iMnGX6naYdGvExuZe8S2feLGMeQhIrhjuC8sPaZRVI67ht8QQRUGmK
ty8w4xLQZZwU8TA4lGIv6m8Iu7hjLY3REu8+pd3Lz4wSNRLgam/Ixwhr4pcIyZlog2S0jJYZJnRi
DnjtnXMxcIMQTaY6lncq8uZd1ct6g1qU3jEreOvNTqfPiM7cIMsYUkNicEu9xnqJiZgN+o08TIKI
yhnyhfLcSHA74F7hPiLOIqI/W4uci9ELB2h23LHUHQfKMtsBkrZ/D+olyRXhptfeOywhbt7giLWw
Hggu8E/YqJsCIzUIt9cKvJuLf1VGyXnxAXINS0/3DOUzTqfkiq+fMbgITuPUYnNc3LSFdTPmD7mu
Nyq+YYnHpHBHevmZ4niCq+YzuVixbmV4g/6oJK/Edbjg4Jm/UDPe4Ea8cNUOrDim5CvUsDUI4hNI
BVBqQsDCuUdS/rERVmofbK5ozNOZTWYe0zxcxy1uHMthdIdQ4Y6YrjncNYhjheaNqliO5ZWJlVQc
EiukFdKxnePVy9xqMB1CtsUjOo8ArG5nQ3BhAAtVmRI585I+dfEvdlTu6B58y4gOTBoqZn5LU11+
2YGVJpod/wCvvPSYi7Vo/eftHNUke13NjfrAYDURvJKr/ClQ622uD1YAwEV2xNy9zN4VcU2xbZV7
leokxjF4xxfg+jqC8bjGem5SEKXFojuG243U1cWHnjxJjq42y68SwmdKgI8RncMQ+ZeJdRlLgK1A
MRBF4KjqOoQgYtjcWIwuaxLhxLhxMbTBzHPSviFY28buG0hqWxL8OtwlZhZllF41NRxhTNuBe5mp
g3hbwanbUEg1EzCbLJB4aQAVSPMkrkWTTSoMYMZh9zGMB7lzRe9pXHsoa+8zaCBQKUDXcRhcvMnO
NEdgRGU/6lNJiQVtv+YTGJthYeM206WqP7hupcMGulxe0aibBL+jT9BFV5mc7yjZazS7l8ceJWax
d9TLmVEruL1PfHLjL5vN83iBH6y7qYNRoVDTKyGWrPmWE8ErqaupUMVLsmYl8WzjgqZVbh8kLU1c
Bh9pfMRUQEr1DyUnmE6SqKZ9xkRhIgUiASCEQhaIwbJUErMwuJtxMgGobjh8kObgvM0yhOsI4dxp
tmHcZOYr5jKl1Ek90q8T7cyQv4jqEsSz5lNiBRMVUQXKMMNSFMXjLUUzFO3uGLkR3mJNRN+I93Fh
UNc/6WOSGPLN2e1u5XaGZHeeSFUAF35C+1m5oFpfp5iNEtOpE0Cy13EAVUGp6IqQMRze4KM1c7l4
nfJqVwRhnm+F7l52Reo/GOeYeNEHO5cHLLx95Uxl+m4QncyxFqYdSghSCGXBBqp5CeiasQznnKuV
PGVKxGTLRqCZ+YH8xVMRNVM+o+WMdQw5uO8YZ3U/eX4RDA1fxBaQDMaS4C2QlaRlg9xHaGyNnErF
qDvuOlRE/jgs66nmJhMTHKbbm2bMxbl3FYfFRMLG7XcWx+cC9Sus7ldYlIRgRqQwczEyhESLqIXZ
DrFwyk1LRmoLjeOhL7/UzFPuFZvySlS6NFVQMrdyhkdLt61KK3M9rvG4BnyMAoA1dblPVZC5X5UL
haizGdJnzFjeYneiOUVfPBuUzF+ZmPvluw04j5iukixWXUFg+4MrUed39NQJU1BuhiosZT1xwbgm
CYjrGquZVWJgmPUrwSpjAlcaxGLcS5hvqOYbMQXLpjgU4m6iUddxEcxVItu4Vl/rR8wE2eYGQP3g
pHEG5Cm8QfENcSzgSC1X7+gmgTDMLN8cvMsaFl22W9w2OO+R6xlzh/cPEBbgaiPUa9SsIWiDRxKD
LmIGpeBiYeUxskmcbicBc2oVBFW1DfYxgHQRrS9bYYyoyilRZ0X/AFLkIRljS++ftCyrlZV3SU66
rxmBLQbCPgFjA7mIxUWTuFHBaFr4+fRCD2Z8oe/cs1G4TGDcRYwws3GVC5VQyxxhBixQlUUGCX7g
b1cDMesRaly5cvPqdrzGV+up+oC6iWopsldYgniVDUMueFmXf3Fe2HpmN4gT3JWJUqBuVcNEb1Gv
/kXVdxzKxAz/AKlxgwSwm7EvGVriU/MSP1z7lNQGpbG+I8gMBFW5gA/MFIRcIfdwpauNk7lvzCTG
NiGpic0t+ozVy83bErmp8n2i1Gd5cbzG7vEMZhmHi4LjLTVMJiOF1FhGqOxusC9Q8ggi8qiDZLF9
QhTuHw16QA7WXWgMoeLDYdfMBku8GCP+JaBVWvA/7xDZXiIXB6iPAUBuY034XggX/AuftLkLYNuX
zMfVgO1wfoi0UKiu9y7B5/EW1WBcSoF/EMfEC+5r7zqMUC6DLNYZiQDB5jwa4uL9ZLlxtIqlkoIr
LqGdQkkdxb3E85jT7y5ZUMTuHX0EHEeLiijllY9M6ynhpdw0lUlKhZ5SzrEVNoQsvFM+IUb/ADDY
L+JQFIB1GKZyURRCfeHhNYQlSudUxfEZ8Rce45/qMEWxs7izTDUGZ1NcwksmJpxKSHM+ob1K+IQ6
gBCqSrg1cJw3LMQecwcCzTyv2P5gd0C3VaF+P3FZukdAvEOqRtU193/bnY7doxLj8eWUbA4EKOIG
YRcZs7/5gUNAH2ICpqXhFjNwXTRiEvMXM13B4W4eEuXTgjjBFn1FrFx4+YzuHC/4sQlVWfqYckAr
ECECcG4mW49qssy5f0XN5mp/EvzU1NnUUWZlAlXKiQx+Zuxisy83G2I+8GOE7nxm+sT1VC1CDbEV
YxrJAJCDlF/iHdUIdRCH1nj4ZX2xXvh3xH9QrGeCa4Mf7gU6lCZJVtVPRKIqXpiaFQzNIYLExqMA
gHcFU1O2EYtYhTLE7K3xKdd9GMDpNrDTygUfBSCUWynQT2zAy1li+g+1QhoqxrcFqw2BFgQDNi5a
FKx2TrEKXejo3/JG2lz+YAENwAUXpFthmJ4lSzB1LuNOK7hAFWgmL6Qye4kOJSM7jj/lxIxK6xKN
krkIdQesT3R2wjZtl8fozGXmE+I6l43LlxxX/wCx7gZqA+Jn3CerKzGjE1Hib42zqGMCoYzBarML
QxlzSGF6iKKRlTHaOz4lbnuIiu+pWuCZ4pX+4m8R8ypV7h/OBbO5nNTJ1njLMvEjTMo1K5qUEcRR
qKBBk2LiAgrFr8y79S6L7ji3MeQRUu3RX3/qESXeVMltZqmfc1F7vAYZEFgycK/W40S8V9j5i4wb
9TEjnB7YzdbW+0ymQsLkYcPG2BaD7xg9q8RX3Tpt1vi5bB647l8ZphLu7qeCKx4qMfEyeIr5j646
/wAIKxAsxNEAbqBOoQmRzw2e+LsnR9fVTHHxCfuXuesOeJtPhD4TXnM5WQ/iVuHHCI4z1DcTcsTw
hTBsmJENMOUpJ5cWaXHEwpEDdzNsmQ3PZHnEq4VYGpUS6/qV7zPiDi4PUCGV0wmIK4SgIBUVEpdy
t5YAbh05mUXKOtdXBS1iGgZlUZRANQqA6aqt8EfmDuwjbEaMvxcpTUxrHRtv8y9ViFWZXlnuEy4x
35W/mdgqVFWZAS1empYzZdVKLUkuokhkr8QxXzGe8EVquWdQLYG9Tfzw8js6lajjzIsQ24ac/wCT
rMS4YgYxBCqhmAxN2ZcsvZWILK41G/rqffM6jPfF4nUTAsJ+HHWUhJIQLMS8XBXCmNQbEWUl8R2l
cq7lE3BqLeAqVxmVXE2ufES3mXYXI9yuVRiVRuVWIzudT/iPJca1KtZplU1KkLiVSuZIxMVWDBaq
BiU13L5cS4w3lVlodFyav1fUapYwp2U81j9RnQ7VKvav+6mNkp0ds/Ny9GMqxujq25QSIWincK6O
iRRn5XEbdAOd+8SsANlT8hX7gqXwtvzMf28gXLzmbeLjf0hAJr0+4DNdWyw3GxuK73BEz/hBYtGm
UBRBHMECIkDUzYXLUXHfmDXiVqHUD4g6PoqVRw/edTuffn7xJRL4WlI+MqMrMBuK8HwlPU/NPyRj
NxYoTgg1Ku5WYZipzDxcynUxwGb5nrMbL5miUL8wkOfcQ/ECb4iZlWsCV5hQikRukbDXB1zHN8wL
HSMXuUUiiFtREA8SKA3NzuGncuYk4tKKLu/R/LCIhKAKLo+ilgZmJKcQNRWSarpdA9+4DfHWsgYA
8T3XSsPzBgqq7B1V9w6xfG2oUCz7ICDhA0vczFMeIdrjje/qCO+MzsGvcIlBiFZo8wt3N9w3MPce
KlSuXR/qAIpb8TXiUVBNwLuEG5XtLrzzUQajKzCB8Q7881AgX5lvEtFS0sblmYRM+oZjv9z5oevx
wkSJnMD3LJekStRXiKPMuT1CrDkgI1mJe/vDKM3TFtlVTpjImIqDDLLlTDEXEXEX4Yre8wtgXK1/
UtcxYTbruOCMSkMSmxKQJcQJjg1uZWUd5lXcUrdwlbjAvUPhbZdoAw9rX7iUDQXe4PQv6iDtcHoV
/wDPmALxqC0lpfQYxG6uat+A9sR90IniZwXMBB15gX7ssg/qGXZIrPylmlDQ1aLEUbVgYYYH+D3E
HYtShkADqBV/UzPGGorPUJUNw2ZEh9SpWJXEZwMZzBNRV1BCUG2U4Ih3KbzFe4jczJ9zHjQGYBmG
dXiNXioDxslstFeIw1jU1KOocQsyjNTAvgMwhE9Spt74zUmOY9Si5llDN03gqUkVr3MyKrY06jbU
OkUNz2yruednmQi5JvmX1K+o0mCIuIN1C+PmBC/U7Q9D7QtKEG2CACYyCOuNM0ri7peKRLQLWG86
ZSg7JWxa9EpTBZeiBheFI0mU/qEhXSi0Mq+pYCVGK9qeCXqC+da+4aWpJwaH2qAWhnB2xUBWDhEm
YhZYr7wvq72n1DNNE0PEQoxOmG4MD9JF6qLbGFqun3Hbc2TySq8QMwgYH1BeoUZIzc+xm8qZ9Qnq
A9QXqAdQg4lcY6yzOyi4qYFuphUwyoVqUdxG3Lx7m4fECE4y/cvrEuNRirqV9TdKY2ZVmBFpi4jo
jmJmOI87lLIdTWLEyupnZc5mKW8cDJFw0cLDcWEMqYdd/mNe4e8EdyibjJhirjXGNMVsjsy/eCma
g33Esdt3PGISDecx7A33M2IABUwKqOEpIbuPLEW1MSeIrws3FvFLgVZsAS5M8gk1MEUUJnHeJayo
K2s5rqjEtr7BflfnEI2qobdAnRcdEsmFHQ8fMJetU7F6uEb3rbj5OnRn2ZvY+IWjg79z1LliPF3t
+tCBa9EoDiioZYExswsNMXUR/wAzEhTI+ZtjZUEvygeJiMRMqzEZdwxMjcr7maOYeYNf9uNQR/Md
ecQG/EDMDw5iTrLagVEwIZEFy2B6iVHEMJc1MjhqIwQaZmJVUOtwEm8JZjx2CrqYSr8wUyhA6j7y
zuK3iIZ76hHNxCAM90Cu4l61Oh5gHJiU3Nt18wmPqNiuAu4SCBKUQRAQwlIgUyuX0VrDSsYrKXUs
YTdpRARKLV6i4vBoIyBToMB8v+3Kg9DM2azByNAHKb/mWXMsoWHb4uYmFtYwL1AGmMGi815gJTbG
BQ8ShoXBcAIETtJlxERR/wALUmXtHeSbe4SokrUpcpSVM14+iD6g+SPuuIVQgGXkuY7YPqU3DLly
6nua6l3xW/pcx6/U66mJuXOk1hxNWdpvHcxaiqjTmkXxTAJqOPP/ALBlDuU1mYN5g8bgVhdyRxqG
GoVRFw1BHcGoWZu/3KXuFe4I5NQ3BzBNOZfWZbWZeGIWNYmKG0SpWY/DhlcRgGWXUPmEYHDg9ykl
kpFm7MKlcKxcJhz4WAAdoFSKI7QEoHdx7qc1WPh8uIuiHjfn+WUm4C1ntCIdrnoL1MZQFj3Fw2aC
tkuMtsJVdMFh5asILI6XudRsCVLNXKNWrj9bCXRksFxY1mHxN2Wj4itlwY3xLcAWrY00iLzFArNO
Zf3GMc3qGVm545YxPX+Ej14lSoeLneftFrxKMRsbmNl2K1S7qYtSk1ABxM61K2axfcpI3YMyyqWE
vrjummpVFh8Slghx7szBlxLY5bnRj7TEXHDFRDNvvKIuqB5hINQPSH0j5xGFiHrLDU6Knq41UMTG
0qjDcB1DrG4ql8Si8y2y4GZdq3mHMNTfSAJkidJFqwUQHYv8B/MrQVSODt/9ixVLjiLlh7AQGQuL
jMewX8xgL2qB1cB1c0vKR0ArL5i/fD3KCobpmRSl/UAwqvMON6mafrTYFom3S7fMfdqMUMXf2l7u
473Fv6HDxK6xChqVdSq4hVjrx4ioNt9yqL8XPMzHe5ji+PruG8TD3BUnQzqGqlVTHLy3j73Up6hV
dSrSYHEyuOExiL4h9Q3CHhuRTWVhVUJuZqlVx7m7MuY61FrKbhY6gj/cduIv5m3EEOoazBaSyYhZ
VMTLhxKJPTgoYZxiCEipTQgnhfHBMFzMblF5lCkSV1h4dakghpt1XqWZHUli35ar7QJzXGbsxjvM
X208Ng0faJGWaARZsycKrvMTr8szaKFAl4IV3+EFCmnUZahmgIRLuagPQI/Xc31x8yyCn6pYxZf0
m2ZCFRiBSE0eWO5e9y9webik46mVtjZj7lA+4uf8XOrhglRGItR9QXUvZ4UA64zBKMDmXKEuaIcF
cQ6huA0weIKzCEzCJWtxN7gtwWX9ym8zZTLKwnPCjFUyhAKiFahjqAgqCoR2ZeQGyErMXBCAIgCP
hRXFSkiQi7uXO4VZYFUMroMAWZbZg3qDDEIgCOyHXcGXAlpQ2vxePmI1Lg0HIB899wMpUdbPPojr
CrGgOvAiHQbFoVbO2OEVnqnc9h5gg8IRtZ4Ab5ZZUphpl04YtmY7mQzlX1uQ2w/9x8z3yyNvMX6h
eIKFwCswsZjCouYCuYFu/wBQvEFqY8ATuXLO+4dUx2/4tIHUAuVF5il6QqJihCIqGFcKe5fccVLF
lJNsBCG4q3w+yFdMK9zTLGzdzI3L+5eTJ9TKxp4jnDkgURDxK3D6IW6mMFtEurCSxJpmGmYNS65l
YZio9pe6MsNnj4goIoIahvhdY5wxwyg02xUz9oqwIYIXLok/Ey9RXh+T5RdalxrF8mRfxEEhXoDv
72wVrU3Qsh9oKd7HBAOYrVu3oiVQhTlOaZnkYCGkR77qar2cviXq9+Y4WX3Nx5j2/We+o84jN6je
XL4eGKoN8fiPsICow3FPeIqOYyswZuE8kdR6XKRiPcvM1i36b+k9ywy7udVF3FdRZOLLIgJUhjN9
xqbnuht8GZjoirU80ZdTPAVuWO4y57uKXcL4lvcPnFhl9zJ6uOe4K74m9kpX0xtUL9oSquHH/ZhZ
jqTIYgASj7mkI3CVgRToN+IgjALFQAizSZQZhWZW9RxbZc4fxL9rClmHiDBDmaANwqvfQFfCuiME
2RygKD3aCuGKBCqFJ7u2BCOIgnr5VFrFRK7XX4hGVVV2N/qX0fOKfOZfPMh37iO7dxeSvM7TVxSQ
26iYC6leZWHVxjQ2X9aoxVjvHJLQhGIWJqufcwwgeI/oRUUYlVHKrGXAvcGpTN+SK3/JqEE1gY9x
7/qU1BurmqV9zA3HxZg3NJi3PdMW5f3Hc6YZyoY/Mw7h7RPnUXccpkQ63MMqGmNU3TNeGpu5I3MR
hl6ZmTMMPMcNTyjfGZNTIQypgmjjBYDZF1EqNjRACDFVHBErG+phG8vWYNJKLzM7zLPxxH7ysKeB
cbFBgVuPBE/m+z0O5eQeEoEtX7/MDCMUb2qM5kSj2/6hTlQLltTI3e73LtoapX7hIvbVe2NVuXQF
vb1Kd8zPcYUEXFLWV4JK8Yjlz9d1XDHgZfBuFeoQH9wiAwBuAqod5iL8RYSlmmGPtLHriv8AIRV6
9zHH8zB/uHbTN5tw7RXzDfF4O/H+KOnjs+IcdpvDU8R7j18x1nSaE14f1m83+80IaYQ3DR88eJ0m
xOnzNJoR4bPHcZzAa50+je+HfFpO3P6ofwj2DsV7ZlYdcYeZosaP268S9bzlQopfwT/7B0z/AJPi
fpM/TS6KmMvqHc0fed/mfvkYQ19feOngjHf0bzr9p3m/2jud/jjozfkhNfof8ZrgOpp8Zon/2gAM
AwEAAgADAAAAEKbsH7+Vlwv+/q/63vINzHs1+ev+oqw17zHfTVvv/JdhPEfepLK3pTDpzk+JHorT
PB/wn8R+VKlP9/8A7777vNTy/Vub+5q47WT/AEtp9/d7dpp0Ojuf+9404N8NxZ18yGwMM6Z1O3P3
f7f7GT2+q6+MRX4PPx/eL4/89Na38d+sedrKlM/fy4sd3f8AF13embj/AI635f6Fm5SnuxedWrr6
x33yVTyO7lG7c35j27OL92ebD9R/h3q2z7P3/XyxTv3eT6+TexWz93a2iWaqFmGPdbPX+5X0WU+b
LR3z7X++ndNB0cGZ9xzby0O+3PzzxoLJpueezbbz14pfSzyW5n28Z+G2m39ewj+ZZK2A/wBsze+z
443t8R9tc2d8wrw8888sO11E39+Qacv8f29p9pV807VtsVo/2816j7JkEKQXfyVVf1efWtUxF4iR
orpU88sW/n9Ve8+p/s/eN8bOZ74+p7tf300+GMoO80d90+UVheBWd7399PQ8TQ8ywpse488OeOW6
vN9+95iD/wCve++r6F1/HXbXPbfIX+/71n/qXOmvPj29rPrEz9hMvLPPLsf2zy07NIc8efPtfaXn
pOn164tHsnvImMrOGOvaP8uH4TuNA1v/ADW2Dzz6jzzzzzSzvfLLjazb75XjD7W/s96jy3X+897v
acrf9f8AsV92y5b72/RrXfMqP9/mc9s888888ckc81fa8YcQ+++9t9//AE/rCMf/AH//APPH/jf/
AG13M6Z6lOBff1/w88FjkNvPPPPPPPJPPPLFputvEuHOo+s3Rz/8gPvvvODTWbSydveNWPPwjv7y
2ww3/wD2qF23DTzzzzzyg7zjDzw/DL97lSmZX33sMPPvby5TDiTzgyic3N79Dlev7Q79/wD/AJfO
bPLNPPPPPPM//wDvYpdDT6yflPfp777088MdrLr/AO+8k4O9dooWt8iu/vR5k/Q/JdtNc8488888
8n//AO9MNvv/AKbinc716Zb00cMOsMPPPu1q+jv+HrVDFxKgz+1yLWm512k/zzzzzzzjfPe9Nfz+
a19x8Ax/+X4xvMEBNetMMNCzHjgW+tWtvf3Makd/svH+e9snzTzzzjyz/O5+ur3XPLz7FNbeOMc/
/wDx/NNPLLLLfm8+zPMeHe9nt9v4m+yRSq/4ctU88408cfls8Scct64C28YlWM5b/jXzxQoU414/
+VKqSPojj9zEytU6W81+v6m1ULc8808k/U884cO6Il9/d26MNov83vH/APe+8QfpGTXHtp7P4yNj
sER1eU1roN9Od3c9IPLOCqC/PKOnV9+nn7rbNfmjfj8j9/y9/wD2vF8gYnn1Xx29+zZR1xzz52D/
AMaHvz/084+grz8880n7LZqF64do/d16e/Upz/Lzjfvc+Y/fftc4SO35n/NuxbK2LHJ6/rEc02U8
FY884Y3ztVPretd3qd9p6m9wsT/LDjn3+qF9GHHoyl5Q06J7fasa9WAoTA88s4j6tzc884mvP0eu
3cU82XMU9do67q/vLLeXXLxjvR6auncmP9W/MTyplzUrqc880qDo+n8tYX8sp5Kc5nuGn9U6Y5dN
5dL/ABy/2v2fb9TJNsReL0XuyD8RZKpdwp+kONi+JZmHtN97qdlUvy6xEmpNG386SOfWP0566z3K
6cTfuI9zvXlP6rxteZx2Gu9+1chPgvDYfOxVfyPtOv5XOOvfeBf/AD2R2ncNeu80imyALxMGi+bQ
izwH8hGLjyzggfqnY27ot6Q2ZT467xnTlan/AFQdH8U+pL5g/wDvdu+IIvVJHWJz+uufIgz3LX0G
MMHEpPu+f7ObdHKk5lqPtLf8Q+FzO/0sMcmPLv43sfFFdNS/CZfvOUODWx5BXPLPNPPG33bQm7Nb
bNaJlf3vVlEePfefD/b2GfdwbXd7ss9PvJ19BKamvwKMZGjzBTyPPPOKMzBG/wCvz2yvkK9Xyzrj
bfxyR2rmE973XjTjW7hnpWbd9A/TV+/yyn5za/NTzTzwz39rNf3vhYK78wbD/TXtdrXr4wDwupn2
PHFIyx3PPrb7eyXw3z2Lfb//AEzI8888Nx8pLV+1kML2T3/1MtdaR2fEG98rGs+IetVmtaYYkaun
V841ey0/VpGxyfhe8wA82p/38Fd888DH1vreiz5Z7xoqc/0m/rukg8KMfW808c0tLv0+f6MsPCT9
YyZx4fMV06V3/wB3PPIhC1n98H//AMv6bYD2/nrijJYQndS6brJfX9/zyt53J7QljV3ua2AfRhDq
/AqTzzzyMIRg2Rz/AP7xK21qflwS0DY5U17/ALovUW8xL5b4vdv2P3qKNY/fTFPKGbT6vPPPOCNH
W2/fPPix+fTN+zJ1kTHj5d8HWd9ud1+5nrpNfZQ3l+0Ak/V/LOPFdPLtuNPLeFibyp8fKPTvpvNI
fFmdMbOKd+9arKe3XzfuS+m43sAbFgoZU6+PPKJt/EdK/wDFzOl4vnxsfTE10a8ooaVp3i8d4zOR
+v8Ab3+r0h83o/t4msN92myjAs8887q30Z8W/wB79NyHvHPIalZhnFu+7H6NPSgZHNHFDU5N8fTj
K/PvPv8AzCautjZXzzxVzaYww9th7nflLzzygpzpR19zbfnt4env8t3pzfoz1a865Wd3Vbvvnke1
VW8zzzj4+/zyR5w5z+iUXzzzjaXdzPS6VyFjefyEXqrxda4gbQ6ECrE+oabwIzDpu5DXzyr5LZQW
feo2rkUzzzzyjLrgLv5lS/zWpfj42aCDFj3zZ67zVeX/AB+pjgBJGhh4c884+4cqM3Xgcn9s8888
84ml/8QAIREAAgIABwEBAQAAAAAAAAAAAREAUBAgITAxYHBAQVH/2gAIAQMBAT8Q8qAAAIAACAOB
AgJRdRUA3SSIHnIPralVSqcGmAYgb1PW4qAECEBU3kXyhYw9QCAAB93kaACkQGoSAJLdio8AAMAE
H0AAAAABEcFI5EPO6ACAACACEYGghIAAABoApDgAAABxAg6CYOAECT3K9EQCOgWCmAAA+A9CBAAA
ACg3ekIAAYwQeO/YoAgIDb+AAbMAMJoDzAGT6IEAAcLPQpGjiWhASCmTuUBKiA8EYDVvDMDF80Dk
AIBhe7YgMgQQLWEEKaeMKhqHIED08hIRCTaIDIFRVQiEAVEsA4FCCZoSOKnEQUgAIluCAQAAECIQ
AVEoACgwAmIAEgSxAAIk9/CQMAsBAIoCQACLAAgNaDC/AAAC0CC4A7EAMAAQANIyFYkFGAya0xID
aJY3uAAAAAuAQIZ4Igcq9AYJpMAIWE0wEBABQQ3yYEpAAFgogLUnPQAGCLmUyRumYxvIASgAArur
S9cYABAYcBBavQKG4gAIym4WAKAIICAEItwAIJILQBAAZUklZBBCnMbAEAbTF2CIpACgATSIAJAG
hqbvCgAoM0gzrzAKQAZY0IQKBGgIQAgAiIAZKFJd19uSrpQdBUNCDSUBMkXQC7eRcj3YNWcKTkGC
I1engAPLjSLB5GQKhCAACQBibX5IwTYOfOFqYQBGeAy1AMkE/wApCA0QQnZYlg7nXyIABACAWvSq
AgBpwaQSHEAFBUNvBEGDEe/okgQQgAyd9BAQVq4ikw+edCAAEAyoAJ1oAMuOAUVnBNawnL6BogAg
QIhAwAxBgIoHIQgYHikOBoOxMY0QsQUHSOW4AoPV/Ee2Awt5fqn/xAAqEQABAgYCAAUEAwAAAAAA
AAABABEQICExQFAw8EFRYZHRcYHB4WCAof/aAAgBAgEBPxD+2IAAAAAAAAAAAYIiEAAAAAAAAQjS
AA7+zIgAAEgQJAAAAAAAEAa+AARAoA5WpUKIAAAACAIAgU7ULIB8LhHoF3YIwIACIBoABUmkMUNL
YDUHwI3YAAQNDAAc68EvIUwUDKQH3IgAAAAplmt4/gGQAAAAoIhHfEGEAUEBATv02IAgCAsEGCBV
a1mAIEwWaUB4Q2tBluAJggQqXmAgEV2Pg3AlUy3sDgwKPKgAYhYRDiPH0AMIc434AKwFDAYwedOS
AQdQZQgIMkXQHcJejYgAKAA57JTOQABCi4Kc9AEjkjeR+cfIyAYjYoC8kkYAcjpg1ooFQxBnDuBm
YNCFyIrCB7iW4QvmfeeEEpABo4JbjosoAoQGoBICMLIAoYIgBf0mJNfxHYAODgyZOwOoHFchAhHS
IBPQu4QdfRAUKm9AABXJQuIwAMQF4AziBgEDegL5BboAF1kX7ZsNxhrCQgAUhTJCG3sG4JtSQcsu
WUIAIghCVIIAlat0+VWCgeYo9MBrVWYHxW0YhQwAAUgIiH/aWPgQMZ9DLWg0tKDjsCjuACUoFiEw
kAQCdAQhuC3O/wCruA+YCQZIgjywCVgp27NiB4YbfhZaCDTjABoCAALMtLgkgBNBDYoeSBJAsDD0
0FQbauAcQAATA8ZwGoB1hhgDEPCx4G98DyzAoIijHlYoIFOtAAg2VKAAgxUbesgQgAAG1NWqSHgQ
ACrbDV1wm0Qs2qAkBjyA1CYtWwmUGGHJUQKihPuj8KpQB4HnAuPhgALlPHgDC+O9yaxXpAdxIhKY
euoh4CAA6RIDEhBAMqDCkjpjF6t2AFOiodB30T4QUWRAEQioQlKQNqAaatoEBL9oH39ACE0Rqwuy
78+iNFgJ8MGBjjhAjMxUrvAAPUAtMMrwiEQQSTCQfNxkQJCw3S2JeXhoZjW4BCEkyHSXaoCXr9Jl
8OFSGKWmmD2iK26GqAPnqcSao5BkVAQcLgICWEDlEF0S2TAcSGzVUufUgCEymqkiewKpiA4/mQPC
KDgCAFR+CAV+QwKU7N3xKErQKOUCK42cwGAXIFBjuCC/OCqyDF4HYJgUJ4ALRqCr05kBonYoB1SZ
kiCsq2eIB6gibMgl3XIBUEYkKnmiWpPSjjKRT6SCngYV0lRBccPITgamANBOz9UQCLAYWwRK8cDQ
M9AvAlB0YGwjJSqgopZAhCgkULDORIjlFhwjE6DEMhwuJSVKxe3t90YZ86L7ED2REeRNB+pM6XFg
IAr8JL//xAApEAABAQcEAQUBAQEAAAAAAAABEQAQIDAxQFAhQWBxUWFwgZGh8MHR/9oACAEBAAE/
EOQ4y/n/AF/9/wB7sR+v95/+3572/XI72uwn/wD2u+3++nthz34//wD7/wD3f3W75N7pTPm+8X5c
Pk2znObhzRuacS/92709f/5S0fufo5j0bp7UKVv4P/8AXz/NhnyjiczoqFysoe2/053+efn0/wAK
4d51Cy//ADMs1eaSTvLz9r9+7a9GUsTpI7umM3r+6xbAuIGEoXaEk9uKyEdyZ1y72zI8Nm3hts76
W16rfT5Zp8lY+9y21ePrQObNXlP/ALuVgWRtNrTkK74ntC1ezdn5e2cQUr3u5Ub/ANJ/7ZTFm5U9
hYLbPNV38/I5CeiBhrjJf+7mkg5VG/v/AOG2eBXZu3vOSyO6uoNw6j88fcUQDFijs/cGhA6frsW9
qt4hVVtr6lBey1a880nzsAg1lRVv13azLINNrWq0KrF4kjDvnDU/72WEU2K1rv8AsqSEQmXqekU8
9+26YA7JbMWY4rtzHihL1HZSdcSIYAwTSc5iQkJEhH6FV3XcW90zJOiObo+poVfu6Ln6SCjL36cV
uXCTI6fJICLNYjNgdEsVZRRx0kuPR38a8fp25c9QwiQVdqH989nOS9vfv33tVWKrITn9mcYJ9G6N
0vWBv9u2kVOw6r53TJOZSEcsXqIymK5eD7nqs2zU78l7VLMRetY1xhOVI60Xg+XwQzr1vodCzXT3
gkqatsIqFJ8TdI/bAJiKWn19tLKG6uMW5FIjhViM2KPHDtftn87AWvj8X9fPVbkCiEsQ+36Z2z22
GrXnHOotkyPj2ErZezut0BnRs5DLGkGUiBEtc/n9pzkLMGfUr8dupz7p7jvmkk4W069geOX+jT0x
Rm9v/wBPCf79oEBXqSFkNI+R/VWDdV1vXxC7bDsjc3ZKRdjNa6L/AKtLGys9ci7K9yev3hY2tr/9
DbwuwS5AhDajesT6NSpN9Kc5yln6IsS+bvrYm12KXPYxzKlOpiGbyQYdflsxkr568dn+qE7ttx29
7mpqvgr5Sdk5qulxGBHB52yFVqbQlisOQ/mx7CxU6W/3ui1rOktF6kPTKymZJNgs0ase+25jC9PS
L85sn+/S8aQt2wMe21Y1ee/fic2mAwNCk39hOIJHwxTqAb6CZETsl/MLqE75/v8A9XqnpeQ4mwey
nnu3TCcvG525e1MQ/AZHHc21Q78suq7duKTiKsGB+3XZmgmHkAI67m+HdyQ6lSfft7+Wu3bpk6SJ
AZVPo9ueeQ8X44W8bnHblzAVsWbaMfsRoZ7Dy2KHLsZUNuBVqMhj5Nv/ABoE+3p5bw/776lvR16O
+jdG6N0hlukFfTH0k1FQ/eViZ/NcINGxxrTM+YswNUp/wdz8sBWyz/7wugdPz3//AP8A9/0tbSqp
LMrvqXce4tiaB29nucIPeAnVV6U+gvce2+OY8dr/ANcxURChLK/cv8/q7LLcfmTZj04YPunnx1JU
zZLtjcQUsfd+F+zuUirtousfbZkzc0u68J9dd/jH/wDH0YY13/8AF+jqe/O3z1bsN5H3jhDYSx8K
TPbKiiYF08S38zkY+eRy9bO/f+OSqVoeb0cn2Lk7e64mhxpy9V0A4k1sf9v93mzCk/779J/wr48B
GUMzpO2sFFliSHWjLHothGNe+c3cZfJ7/wD/AP8A99/9b88//wCX/wDdi+5i9VfDaZPd+7WGlbTp
8n6sGlMr/tDu+dJOyJT/AL1txWNFkFld58yz0rN3Uadb3kynhm7PyGxWoNWVZ33s/P8A2uC8IWZ7
XsvWXb//ANSCre6MfIsl44Op3fK2v9kbqcdOL+bPnGAtU/f7r/8Al2bzNl971h0+QxKsmzsOu5kN
VBrGHw/99fXdT9spkHr7y24N/wAN3uOCpXOMz/8A9/m7gcQY21aXA/Tnt6LwcQCGSfB+/P3taBq3
tvZ5OMfT1q2erSi53Cs3kUS4eqWD/v5jqpF3m7IF+9/3OZ8f7OyHYyRUIIDaQLdXeYOGLeCQ34fd
VcOzcbqFvuvjNCz7lDfhpn9ZPbPbxNr11/aQu7VEarHyYfvdLz+utuO7LOOe6P8Azsfh/wBWfKXL
o/8AcZCnOEks9zf10p18vxvL9d//AP73y9mmfuG/sSS2938E0HyuhsxKN97Ntst+69b6t1lPdJrl
r91cD/3z/qxBJgwiqAoDDn3W+7/XWP2rOVxYe+wd2yuo/kgmqVoZozzOPEklWp8wTe/H+Y6tz6JV
NXvdEXE8Mx/3vX7DypkMxIzUKBE9IElrvzd4+rGs3V1jpD1/XoxdlusdauERaO8h9jtcvt7fg03p
n+cvzG01YXHFP/y7tohTo4Cn7aD7n4Cwtf8A9/T3oh8u/wCsfs40ITKCCxPl6b31ZRtzjvOCWfFj
VZmKzpB7YYK4RJGW3tanbY/f/KTtpaRiVUm/PtYeKy61yru5eSWonQRndrj823M+3sSKjcKqejLm
Z0Zg374Kx7kf/wB8U35RzUpFK/8A/wAzrP8A/dAwZ3QDbSt8x5GUFIChAdOQQ2DRMIsin2jzcMDz
hURnGmwe9a1AjvkhPtD0e4A3ZxwW370/2KDtnSFSUrENrzGhMH12v0cZz+jxAXSIeoz2Mka72OLS
fcyxzLnbt33w+lnltg4V5mGSkQGvwW476ZPzXl2wR4uzvlCBuNN55f58X3M70FMpWHEandBiAOmO
BSaYL7IAhKFHyLaTDRGxmWSWj/XCrhiy67yECXWwhkGkezqi29eQBnoh9yeH5ccO/HFXIgkalfi5
f9eT85p2tk/3vr9E25kuz4o+953mh7RfX/ZdpsXaFL+nOEo1hZYxJn9BQ2met3fG9JbeirKXM1e+
hi8eWtn/AET67KcakjPjxyHb9kovf/8A/v8AzwJg4MD64ynXvWnkd/hckPkbxIm16HCqobTMBmr/
AJM01mspRqawbnmEmipPd5lV8wPfU3cv2tld+vQ4hBP/AEGJA/v26VAfROxs3pmGnNklU3k/b9ST
v+D4OKwXMXgyAxiOc0CzScrA0DbCg5/U0gpbsnu/qYP1TOOP6efNG7fAYDaHXjRzqWyIUfyRW52g
hFIv1xlT/U6z3+VCUHFyFKOU7P8Asa5jN2/px+9ssWWuxJIJnlELaE4QVMIBEbefCG9Rj8IHEMwy
kgPvttoxzXtTxx3anGY8KvU9n/jfNeih3eOvt3CviwsxgnCrWaHNV0pL2unz/EaYPY/tRkJ7pSNP
F/7FZarfU1fqPpgu/nAYqsBbRNW+b74O9zOfTw2ZtrZGOr95yxMGEtbe+7G2H6cc3oCxZa3Ap/8A
04KKf1lNYQo4IQYaJPdqK4v+vzr9t+9/3q401VvpIIgbJAAFKgpUqlvRzQFKvRuQ6+tKjbP9+hbo
NlOorhAhUSiA6eFanTcfjYc1/wBG6nlI3nx5XVlH/wCdSosSlO+HI8v5NmvpfDWC/Nm7z69UyOPk
Y9w62PvDdW7xReo73atcqNpOfYvXD/qJnJga2H96MMNlh9eaYNAFbVIIiG/vuwKv/wA/z/fdYGYg
ByF8+5RoH8F2rLUpop8L4pAXynUgTnHFvnxLGG/JflHR7+7TMqvu/wAj64z0Rv8AKp/htL/P+X5n
Xadestab3Tv6bLl0frc/p+b33GYWYpkWN6js2o9ZHWCBTfNSUufxZSXfWnI4knnmrlZ0z1U2Vmed
Kyy8U+Me7N2woQP/AO3/APt9PbDbamD3zi6mzjuurrQ19+iLww436cSeSARe0bf4s6Xee993czgd
iknXDvr/AJeH4aft9wvbVIk18mawRrNWksl/1kFwXuxci+VYOWAaQZhvdT+k74nfgKXi/u9VhC61
Qlo+9A5t5K7uj/8AsRXitXymk3QRr9Smdmt3/f8A9MfWoCCe8vAS/wALHzyc1csXgFfFD9STzb9O
6LzF/Jnz/ZHbh9dw8/8Ar3smfz6d3nof677aADkbfJ97N6rYBb5YfN0LG0cvecxkhI9mVEpTkfI7
AxrOT7ZaZNUntdxv51sO9E6/7P5DPqewZWf+dbNdmm6RV2FtAmiFEW/e/wD770Em7e6Fi75K1G67
+f8Af1JDe3vYvvf6a5tuLm01twt+L33eQHfn5rumY8Mu79bxHyQ/XeXQM+Rac1VpiO3A/mi/askU
vc5OMj/9NQXHf5S2o+qeG9eFvZfFeB0Ns0cy/eV/sNPFp7+u9xpsmQDjDeGt8c36mpTvTt4kG605
rX5i+33I0d5Gv/u8ty7Wh09v/N3N7fnnKp9L7YrLyjjgU3Zydtjq7hsz3t1ynPP+4U2zu22zUXNx
6ga+SoGTzOM8DgdFrsZ/9lum1MjnHbaFTVl3dS+HYyWQcqTqIarS/wD99viNx94t+bPKcKKuBp7/
AH//ADrkVkp8W1zo7p7F7Zz1VTloe+cK3uM/lDrOLcd9oX4vXMcTHddJnM6y/rwe9XFtzdaBydxE
DozdP63/ABHDAHmyVm41b7UldWxJua+MpBGya8SUbRWaafRv3X69NiXvuvyvGQ9f37+/v9mEPVCi
jfEejfvX/PH2OcFK3yPzzXMTFusLQjHbXQdoE31vOzlsPmv4k2G7vyXpYK7/AHbvb6Y5v8s6cO6q
/kRTS19cdvGACe96uPNdZv8AIKLL1b9In5XgmsfJIyc8qf5CD2edq1lsv088sHvrXkLz/TxVz/vD
5N3Yv45C1iqqhP8A2/nJksw9w6/75L4adgVnqo1vtmc6/eKNuuvPL2/Xld2vhe+Ut/j9WJ+ySFPA
q/nLsnFmmh9Ov/rFrzDfbKO0p8+Ozn4JGBF4YitAy4vBdDJWG+L80vtflE7iS7lOBXO/66u9RE3R
Pr2nZOCTget7bFMrlb9VwHj/AEuaEv8ALv8AZHMU0Olc86oaiml/M1L/AMWz6X7/AGXBekv/AJzZ
yO9bvfmfDzcuUNr4fGtL0zW94R+s9/1N848f8A5GLx4yGV4KE/3p3bW3xpLmXWAvIf8AQuBjwnac
DcKOUVLu7bt2bZWzSW9vtnTQbVVwrTUhDvOMSrjy3vBUZ1uTECJoa1lGMgZPwz/GMsElFzWsN0y5
ZCGCb5xZSHmJzqo0vXeDj4s3+nX+P+Tem0Xnyx3XomLedLFfeb2PJv8A23aM4Zt1fmiTVseXWYdN
9tGPU3Y8hOeGUiUzJrtDSJ/9U56VfbHU/lXC/wBS86CTf52il1x7YwhGK6+5dvpa9cVNoyQhJ+8J
aacvd+0fF6ZGxicbeCf5Ov1uvcEy2v7MtlZ28XpUZ93T3Whrf9gKUW23Llh78dO1/icHU1/9HZNo
Xbfd8jSs1q65H/dyCkv2CQcZQTbNkxM+pfyXBe+0xPkMTPVSs09jpLvOBbHmo/8Ami2strSIHBwZ
NaonOpir1zf8hQ6+lTB1cOjxySrfZn31UfWf0h/z4hR/dkT+v8zzbfywfL45v3Af8+Dz3Q+dmsN8
u83q9+vCB9xbSGdbtnlJ7/3p5fzIlvRCnSWrJQJtj0Lbm+uy2GqtjHwsUmPwRrNt2q7f5Q7/AAvQ
eGmDfFYk9YKtgdlTB7Sp7tM7LRzVLpwY2j294snoaLOMN/37FHhrXPIn/suyb2meZPju9m8tQy+W
ZXV6bLp+P3trNMtl2YC7eMR7fHxcp/NT/F+cTisKe6hlU0Prgu5jbbrevt3v/UpSL1/yIdwOcDs4
Kl8ed/m0rY9VDMBoC2IoFE5szfXECGZnptgLwRLfMmRIDeDSWQbQ9Epd+roMp0KLZhWFZibWO/u6
tVZ/cd+9mC3/ALPU9zQuvlR9wSPPexyZ1fBXFjYo3vt+9Gze4K3h6687Lb/T3YMkvaeUg5uyN1a8
QamMunev14PtqJy7taWESxeRpMb/AC6DrHi65YPkzt3RIE9RMYIfyO4xySCjhgtrcj16bER2t1lr
uOQFXeqYSP3eLmYHQT4CjaW/76a/dO/t+8Iwsugt166pr3DT/R3+hOe227f52y89q9s7g+t6zXTx
zC3v8ev7cwMo7r88HzYlF5hFzta+08NWJ+Wja/wl7BTFKVfYoUQyjfSHiFb+WaELxY194g77yJ5M
GoK4ab2GLdTl5Qzu6nSxeXjYM1hJDao9+7gmAjs0LM1iFRrWdo6mavgPNj25L1u9GjLruewTQp7b
y+4qbctn5De903f5n+iR3XDZNa31tV9mcj958zN6isQ4aJ/j/aG3oqP3N3XuP/3GpEn/AHU+2Xw8
Q963gKRNy0CV1ywEb300fqGAy8Xf6LKnxqh3dUh8SpyzMO6e3K2p5GyU+m2Igx9Rn42i1LNmBRpH
plb45KkV9Ef47adZzzaeTum1PAyl/rtnPez3uXX57yf9TVs30xz1/rx/vXMm1Ult3/Mbel2LxJj4
qh+1/c18vW/nFpVmH5FggMJI0/mcVRwn/FM3t7ijZ+55fxeBoPu+XkZGLngRxe3mFAmv4ro+UAvj
smfyKIQXa+2WbdEhh31Q1JtNe7ajJJNpzJQkGn2xrmhv2d7fdZpDb0Pm68he8/ahnxLtbsNb3BRz
BFUWruD91PXWPrtd+96PR8tr6vgcv5U2iJbzL9WDXGgwgkpXWuIhr9iAP3WPrYhBHARgNaAYXChn
6u4VMHB+8CMOrk5XPWCaV/ge2071oE+OaEw3qPlveJAPhYA1LVusuZShSiD6ovOgStyYtIu6x39p
/wCOFz5dA/6o71/999+fjI1x31E6Yp7KdTV+AHuk7Km0cVW5hm9K+zkMzXdOWqxTLnvDBHIalKjI
0PQ5tjHQttP9/d73RF7OQ7X/AD+pHUcmOyK+uxOnV3KR3RY1I2/TYtgt+/fzRR/CO81Y9Qz9uv8A
9nxYrW+94/d9DOGGgLKN8I4bSj/oJhdj9coiyB4Z5gHs3vMtGfUe763koML/AGQbAHMTpeUKU5c4
6i+JfsTMXXtVSQoCZeTK0lb7WO17peS9cRtr0C8bt/0tl6bZPESyt5PKMw7Gc/meyXs2ZGquKTMD
SRF2pFZX7XlPZeTb1rMj0OMTwUgYFZc/qI+4KHnq5ht1bu2VXF4Gl7hgzzg65XRdp9TbLpKK/nMi
iK/2f3MoueUOegRqh3hyiJqmG51uVOl95iVNfVUrLmev7n924c2g93+9L9fYgauuM+nt9bbW2a4O
R498un/U/kOMxwUVZ5TPL88zhhsnkvMt+IDOCydiNX+nob3m/ob2OcLNDykSybd4aBt4pWXHqFTy
x20xdS7yKT2Xbmj37tRbzEOnIaxsf3bHLjhiGniB1Y1+8/INnt5G9nvYO6CZms/hSy3t4hS3VVw3
prPKr24j9r/tFD+d0p0DPYe3g5TTwSz/AMdKqHcfT1cejNk6b0qSJ2RG9UG/3PTprjNiZ1CdeiHZ
WELzXOrD9xb/AEleid3xfanuccVcIdtMy/MKfHwHDxujRImAZgbEfnPaxhIMt05Jzlb04Snv9uRu
ECu8zb9+yTu+lVcGBnTkrql3Lgz/ACip3z3U0iTf7t7ivXND36L1gZ6HCFWtXgoACYns6i+0TW8G
sC63oXr/AOw7cz14TJP9GsX/AElViuNSI1X1zB89FVZq4/Tb83aUjbS7pnZkPuneddFY3mWNTaOu
KiqANiMTT+WdnyidqdcDLK5i44G2SVPPVfaGOFuzzYdU5GEapq38Zcx01uRzpUM79BZ8N8fQ4x3Z
wGNIjn2PfCttoV1nuJmr3zdi0WlQQlcC1opfWx2dSkDzt8nLxUQgZtsyxs/ZbDf5mSSL3/8AOk6R
MEVmZF2sNaYd9kKuLRkxMtcvCBJkHScqXZuZuxMXIgMwARiUrhQZ7MjsPe2ZP5EQKYGS9Z3KXvtN
06pwUESmm+VoQg88qdosOE2jBodmzUDGXrYQt9Wu54XC+HnpwDtjV+JC0tHag3Wj+iy/ZS7z/Ut9
0O/nYo5mHL0GbNKTQ60VQoDXihKLvKuKEBJuEMcu92XkacE6c0MIkb0uamP+yXb/ANb7XO+A5s08
5RrqgtuIflX6K0USkAq5GC/Y3U+oZROVABRlq/K4tp8nq5lljRYA/wDuLiv/AHzMHnS7kOWqfXeC
7osefdgJjFX8bNm/hjFIzbO3UgTFX79OT9W73hSAKyA7/wBn3ocmV0eSt9bXZWeBX7roz8RXuNU0
x7vkP64Ue/iWFzs4i/wN16/XfI1Wyw+oKhqyxdd3e3354+CaZKUnwwBz+/gsNy2BV/i/m8celFeD
GG53/wD5PjUvhkVPtj7a+8R/KFGLr/KOIbcXMjr6F+v8L1eep/r9X/2nzPJzDiTp5bH6rvzpo/04
urtxG153WuCg+pReoarA0zYAM/1G0u4NZMmaGq6f5vrMUt2Ps5jT2m+jK15dWbj37lmVdOxUy3iw
vt5mK5kaMEHmhT1nP986ZlxBMptiPOMpUdTfN0fhOJRa3oE293a2BxJJjMVs1uXSZ/Jl80eZDrK6
3su3A6T3e+GtWMctPdlrHMnX/wDzeNW4+vwiiXlUPtvPXQ2oi/Vjg2r3u7GCXFmfoWy2apf9TH3q
1j6vVO5NygUvCA91MPobjDMK43eySUIDHc/0JSX/AMmu3DVr8z2SPzygkbs3zdjans3q2sYz+zr1
253rUiYx58tfUcaf3Drq/wBXKDLMS/co2q2s11+d2zhpwfUOvib66Dzwq1QBiEGA5Octx30akheS
TF1ZdOYxwG5JcazD2fqt1+1zmMJlwJJ9bpc1/wCXG0ppTtUIXpVW+dLYbtFF862gmnH1BDMf8/3J
fd+n/oNbdg9OzWHjCzhSOf8AB4BLDt8GbCYl1voS9LuRhxmeXS3nAelWP9DBF3GI/M7ogSfpd+um
iwcVbb/f246rg8VDHVtW4lfVtHL8l41vvX1PvMTCWtZe78743b1BgeNy/avmHvYfyJlA9gqPMKkQ
SBLNR9EwPYiq8ZyNhD3k0rWoGSMYIqTK4a94uf8Aryc77l3A5ByC6TcoYOu+13teqtNCRow2LdvR
15pYlwo7wdrW5dtnoDpsCL5/63B+czUmgEz3K/bPRNQPydx81PlfiF9wNcr0osvIyV3TQc7/AN2p
60zTpgAfLKyFqUhkBk2lh3noU3JebM7PH92vsLfgsI3xhpUE7aIY+ymBWna5WIljNxkN/wBvEtoJ
lKxTrEPvjq3yrwKiabEjbbfpg8OnUZ69R0i94+VJt1Zr33iECHSUjFRW4GxlgRJ+8yCj5f2uarq4
pBt/HmOEruWMg5mtd5+kfc/ZuWk16Of9bdp4zFr6mZ+xjauxte3K710E0WZNgQRs5PfctCN2HLL7
WyV72SOrgtlyiy6Q8lRY6KshcpVwbZFawViMa7yvpte/FspOd2nhvQmQNO1vliwZZCw6xlGjRpY+
MbumTuCnES4lFtM44mwcmMzL0XDrgVlfYx6pTHDWr6pjJ45jlx1gGg/U3HZmVRUSmIFJihhHv5/J
PebrCFsBR1J5I31jHCSOXJuXMTdobivNI4vvvRTLzSwZNTZ/i0lLS6U5C82vlKX7W83S9hyr65dY
1IZLUB195+Nd6PdjK8t6/wDBxdwCAXOOuuGm4GcTco418JuGmUeoUVofl3JrENal3b06gbPpXdEV
nENKxtHVUEvDkqEnLbZNac/YQ7n395k5NUXrHPlJ/wB20Zstgv8AjfuodjDaIV6MokL8F3LkiznI
rTJdVtgGt6FUdH2tdjoN2caQb4FMZz6PWZouqw/2EYxeuJPxmF7WNiVuXgvfdCL/AN9EbufZ5e29
K/ChNMwuC1OmHbNT2d7eIDe9os81HFBpLcxZl5jzZA73Qg9ILrWKOSb89g3AbEICTr9m2L7YdYl/
CTaIoCMAnFXb3dPXeM//AD9fW5ycI9efVewpz55OyI7qQmTp8luypqWDuvQXJIG/86PXXnxM9e35
7vWMWPxdDTAs8KnCnYs8/EuYRQIqIR4Tti16cgmNJdS7MePxBOsj+5O7FbGvaZcaPrtTR77u3YvL
vVRaHtON2vojB2rPHBdXDIi9381s7zN5Lgm6/IabvtdYbsaAl44oSxscxJJrql7CLF6ezKy0YEWg
RN2bwzCOF9TGiq2s5deL+67AtJmYRqYttMbMQvN7Nl1HZdZWsbxb0RvvORl79w3nm4/9W7p5oRyi
qOXDt8FvT/xJgC9MXPsbdjpF26LaP1vkYvI9HrD32R01BD9hCY1YKYvpzPjf+145a95sIxerDKvO
fbuq2qeNdg2VUJX46y/llXsddjAELs1amH0waO7IGVhneYJHc95IjBYE97b6qmgPUNoWkcOj+zZq
KQ6lWunndtwCpK/G+jP9asHrAO5fQWKDZq1LfzvI9QdrwbSHdTCv3pXI+v8A/9kKZW5kc3RyZWFt
CmVuZG9iagoKMTYgMCBvYmoKPDwvTGVuZ3RoIDE3IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+Pgpz
dHJlYW0KeJylltuK2zAQhu/9FLpeiKLR6AjGEOcALV3oQqAXpRdtt+mBbUvDQunbdyTZsdKVRc0S
ELKsjOb/PPPbggP73aw358evp/cfH1l/u21+McEEF9IxC557C0wJzYWS7PypeXPDfjTAwu/8uRHp
lmXfm7RXjgsPbFwIocK1yGaXTV+a002zp0PzSDLbEC5W9EcB8Y/Z9DrG+nXbrm+3L3ZMdF2/G1TQ
j7Lsjw0Kxb1EhkpxYRw73rP1gS4lhUN2PL1tBXQraIXsaMAwqDDoMJju3fFlsz82dzHV9asPD+Nh
UDqMztBCMek9R+HTWYpZLpylo8JJx295uP7n/Z8xoCwFtKQaDAVU3FDgIXnQ3MeIlLyNyU95O+E7
bMUmLfVxHmbbbiVbsYvjPo6HeXFYE2cMFwjLxKmaOAMcpS6JS08miog5gwhSogiApPAAADJJJU2O
ZoDxvsqQPIWUYmjhwITo4RZFNJfwxGuWjq7RocKyWi6jY2p0pKMONAU6kFSldIMscHFqohbw9ICv
OMBm2BtwgUxY0+YBx4QMB07/4owr1zhnMdkKJqAOURaXYXIVTEAdYqf2zjH1MVszAUpiHT39KCGr
LJNkDxtSiYSyGnEAgprI/CcHX+NAzUSpL+MARaMbQVA3KedLILYh3dl6uPRQWpzsI68FmZmjy8lm
7vm8qoGasQJ1lwezEFfNWYHaS8K890zeCbuoEzYXpc9zEaiZLLWHQbtQZ81kqT08lnQOmUeh6Smm
IfOVtOJKvhqLY15izSmp8lG7hRJrTkmFbzSWJO6npsapuB0cOu0GQaHSYSODCyQrCK2QeAwI5lXW
jA6RPlUWfglAzeioXtGWvgQkjBkPzrUb3wNPXv1JIca2lHIwg4yDLr0Wip17x/4C9ixAnAplbmRz
dHJlYW0KZW5kb2JqCgoxNyAwIG9iago2NzMKZW5kb2JqCgo1MCAwIG9iago8PC9MZW5ndGggNTEg
MCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nFVP0WrCQBB836+YZ6GX2TPxchACRi20
IFQ46IP4IGpsoQoGwd/3ztSSsg83szszt0ujuEk27a7f7XZ3RbOcyQUEDW0Jp954p8hZGOYW3UE+
RziLIlV3FPYjh5P0Wvts/ODZSFGJc4D+RF/SjmQRPx0m2YEgkZdopD6MA/g/I/uoqmw5e5uDdd3M
f6+IFbdsglhGoy+hpRpfFAh7ZK8Fcme8V4R2XVFpa604ZgzmpCeOk/SU9Jwm0NSb8C6LIKvH0ivc
AWioQ0kKZW5kc3RyZWFtCmVuZG9iagoKNTEgMCBvYmoKMjAxCmVuZG9iagoKNTUgMCBvYmoKPDwv
TGVuZ3RoIDU2IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVk9tq20AQhu/3KfY6
4PXM7GoPIBYiW4aWBhoQ5KL0ok3iHkhbagKlb9+ZXclVTCwwgmF3NDOe7/cvMKj/qPX14fnb/tP9
s+5uNuq3Bg0GKOqAyaSA2kFjwJE+PKq7K/1ToZbn8EVBfRX0D1VraUo86Skho+QOs9Ox6KvaX6me
f3Q+iWYFcllxI2BpnB1fzli/b9v1zebNVkPO3Xak4Ie37AZFHoxNqK1zBnzUw4Ne76y2xOOsHvYf
Wgh5hS3YzMFJiJBK5jpP6a6eNxK2EvpSsMsfh7eqH9RtIVm/+/w07YKv7cIrNOA0pWg8s5RVnI4G
mWbYt4B5+D4f1/16+DsNpNcGBhYFPQ90hryb2DAWLtxlTLKvZQDEvKIWSS4+k7BYece5WKlIWDl0
fOU6EjG4pHREaSYrkUeVlJOKRk4bOflaW4aUt2eVsUvKeGdsEy9Txi0p48V6eKIMhIrqLmKz7dR1
lq1ZYrNgkFe6iM0vsVFjQkinbMhu7RGOfy+O7ibxK7ZzI3QVnu1O4nY7cwBfacQvBupLc61Hi67K
NMn3X9Cz0oQFaTA1JgZ7mTRxQRpMaBrrT6QhV20+qmPrhwFHbgEd5YlVKze5BFMeRaSmVr0AvdX/
AC4UKVIKZW5kc3RyZWFtCmVuZG9iagoKNTYgMCBvYmoKNDc2CmVuZG9iagoKNzQgMCBvYmoKPDwv
TGVuZ3RoIDc1IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJytWFmr20YUfvev0HPA
unNmHzACWbahpYEGLvQh9KFtmi6kLb0ESv99zzIzGsuybgwlxFfSzJz1O9uoHrp/dk/jy+ffPv7w
0+fu+Hba/d2pTvVKxy5A6lOAzirXK6u7l593373p/txBR/9eftkpWQrdHzvZq8uHT135QKToXTVP
ddOvu49vdmdk2lLSzQZ62eNBBXywebym8fTt4fD0dvrq1KlhOJ6yFvgPpTw+7yAAamA6Y22vfOye
P3RPF3zVSM50zx/fH+AyQDyo0wAHdQQATQ+efib6oe8Qh31+VEb28S5LP/wUtRn2+nrBVRq+7jOy
+v3z17vz8+4dW+Dpmx8/FR1gTQcU3SnbGYU2gKyC7UKvYkANDgqG599bcse/PvxbCOo1goEI+U4n
30OCYhNwfWKK7w/ao7Kkz4V/+VmzPkbUklXSSwcV6U+clVp4wJne+wU333JDBTQbiGiawaCFhItX
gb/JSsx/kxhSZwuP7BC2tXgoqTP+l70XULxHDllxhyyN/AIgR5jtJN4jimDAFj7gxIO2HgbPJCY5
3Yg1QSBCvjA+5UXWQ3aL0FliYZ4F4hdGk1AX5BX8kIQIz8SC3jN28WyIvS1QubI1awVQjMhKj2BB
gAlHkZ4XaAuzg4leORBOVdVsAtvI2+jHBBg8YodJGDaKTnAmKkreWj83x5bWmAaHgtyRYoILKuLE
qHH2ScgYZlMLb62GkA4aGNdaDrvMPtOKpEHrPJM9fKNJAc9rPvGq99at+IRyR0kvcZaBIe5ZSUSu
4M4KeGLe15iq4p0Pa3tPGg26ty5ci9ME/0GnRTqZs5PZyE4aU6qHx5KT3UpOOvYhmrXkNAocOVso
QmzOQZjJdc1NxyEHXE7o6J+ayiBeHW+TORPC76fWD1IBKAWRaSlyJYwc50Q+jiEllQHZaooX9AA6
VZ/kZF4Gh6yjPtP6hZ3O/C2khkv+tULOuYNRUmKEoqt1RGQ8zxgwzRGRVRswGoNmSyRj7tcjt+Vx
rEfBpsdc7jdcDlghktar9WgwnpxY3YVib5TRsMol9k53EExvki1Sp16DYan1htRxtbVQ2FIYbIiw
xTB6jiToE5U5SrbHYW/JLXtDXmXg8QvDJD9yb4GY5mCva44ejVt8LSdcICuTEzHsndBGsO7ZOPQL
/JvkTVaUbtiduawNVTi15HEtoKEsuYfCIxJ9KJLJt5R32/kMhlwRP4s18ufFRiq2iOU5ZveQ92cG
E6+Kkqcsl2OgZ0WFygqn2PA7VRtlnVl9Uc5yIThVxcoGfdgKjrSBMhsxu8bHUAbrHWyBmfW9C2kF
ZiZ3q8VjJoiy46x3NqHY9jRDJFuu+sXl/YKSnD7n3Y0lid0SXq76E+zSx48gXFOdN7G+o18iNPBm
12OXd4s4v2RbRC4onWE9tWD4n3DYRnRjmy8NGAxLlw2js8QZ1K7wFO7nHLCF373Wu0DHmB6sXstQ
F84lTczr+mgSe2gcoN2wnlKoiPGza+RiL8niDZtLg9eb0NxmxC25vbGdpVyrW/5MljOdYkdVqSyV
1NlTF87Sx0q2QcmMPNHGM6Fml2/S57SVLGB1tMvZAlRvvXswW6yOdtXlKmKZ81/icixKxUgcKbIy
NuF2Uz9mN7srN3PLemoi4jp+ZhDX+DFttK1ExFUU5Y3mlUC6Vwrgyn+3UG3L3b1sZQuN5QlRbK4p
X1L83M0XMeMxT8Jz4mVuV65pCvn92dt5xFLU12jwLRpwuOGgR9kvKmXB96U6asmGIqVuzE6tco6L
8R5zb7GtS1tQ1OmVCgtbE0cEjBP9WPsJWyNHCL2JtxNRmThMbvS5reKOX+51pO2OJUdLax3bKYVH
DxjLRqyOZW7moQXSPLSYMrSUK6hmaJmfS6dfZxe+Xcp9/ViHHovzIOTTVj6YOurEdsaJyjUDBKFL
5JeOmxeNq1cR/I320pgBY517FE/0DaE8MpXhqLGGWYwx1BnfL2TZQS7hiLk2H2SJllMbzUw4XLFB
ADnOCtEqz0RwNmk23eJWz2xe1W3NRgawVwsPYnNrNtKhD6vQ9DdjYHE561BGY9OAR1C2ORcX98mV
jsnDJJumTI9y2solUT0kaJIJl65QtKFawJ21GJ0ulIypyHACmmbwbemI2Bm7i7nXNpdy+WpSkCvf
x9mLXOCqdeQCrEy/wGKYBfrdqvffdf8BPTn/mwplbmRzdHJlYW0KZW5kb2JqCgo3NSAwIG9iagox
NjM5CmVuZG9iagoKMTA2IDAgb2JqCjw8L0xlbmd0aCAxMDcgMCBSL0ZpbHRlci9GbGF0ZURlY29k
ZT4+CnN0cmVhbQp4nKVWW6vkNgx+z6/I88JkLNlOHAgDk8tASxe6MLAPpQ9tt6cXti09LJT+++pi
O55zMtkdykDGsSVF+iR9smmg/qc6np8//fb0w0+f6vHtVP1dm9o0BkPdQd/0HdTO+MY4rJ9/rt6/
qf+soObf8y+V0aOu/qNSWUwbH+u0wab43RSrLPRr9fSmWuijpSUsBPjlQIoGRLFY3to4fjsMx7fT
V3NtTqdxjlHQj7wcrxV0QBHY2jrXmDbU1w/18UKvSOZsfX36boDLCcJg5hMMZgQA5EXLj4kfvA/h
dIhLY1VOpBw/ZBXQng54e+CzjTbLWT39/vp1tVyrd4LA8ZsfP6YYYCsGct0bV2MfmpYwkBBcHRog
FK5Pg4HT9ffS3PjXh3+TQdwy2BGY0JJB12DrEiYQBA904E44wJkDAuDIrdcoLAVIIVgGhDYWMLw1
k7RZGCZzUg16F8SgJzhEKfCmDSzJ+mQLGZuoDYH+LB8xhGwJeeV4TwyIOgGa1NmgDxw55+UsHqmr
dCR+XqIfbFfiEWskte85DNhF/3OObrHzFhrKxC14PoFHMWG2y3bE/Y7fL7zqBVT+OH9Nju7Wgt2r
hdY11ofHasHt1ULLTQovasGAZEJrQWMRdM85E24jPYqxbCJL4oz+Nr1W8xJA1EcIuIOD33Q7NB5r
tAxHn2AA07gQBAfcwaHdJApDBGGBTGIDvstAuEwUCFKZMXHa77OZAI0zyOF6wsBYrXbP1c4ND/LC
TylETj+SyCLillRVsc0rT0atrpVUYI42QHXuI9XtIGW6xrXdg0iFXaSIhwHcBlIxdApqMtF5CWpk
rLSUjABCNcNvipl3uaGjjoAEBpyGHp8F3tCzFaWn+VapwNYXa6o841hN1lN0J59vI77H2P19zInW
GofuQcxhe45F0IFMGxqur0GHUbCzQtFSm1qpL0KZtGK/JAsk21L/H5QA7BDX0stI4udYzTkduJTV
K1hjdMbFtpli2yxi0MQPa3fQIHZ8NuZEvyqg9Yt2pw9gb4yCC00fHqRO2JujQJZb/5I77ahTJw2W
gv7COnSKwbdy5Eqqtpx5WwIvCFbY1Oq9BVeCVVk5kB3K4UEoGrrBTiffZRP61TkvdYC77N9dlYWW
eywOm+MsNgrdGME+SuOwOc9yo0DbdGGLx9l78nWWivWRrGee+36IDZHInavclQ0ExB5S45iqUTXs
RTCnIc9GRnA4wZT6URtAek9NCWxzbEmtbidTIR2LLUciXkcuG5hig6LhCKyXuxiJnWW/1WTFOVO2
fPuaPJ1ZY6Wuc0HcjLc84odCdA0xs4a/fznK4Bu6dfut0YAocAsZWFq7gZAi8FquWL1Bw8o12xMk
042n4PJ4TRJC19lrxrF4s/Erbp3T4MqpreBbmh/jhjGvlaFjh0z7WCc2cdb/mSSwc9Hp2sa2D0+S
3YtOR3+wNUg2+2Mp+4NIB1YBK5Qj+ENCNKJpivBtKvU5QZQmVgRnFCDPZUb1Gzc2rGaomObrLEqm
smjRYbjo1HdaXiM/XeEWLHIzkLKzXbGf2zjfFSZrH+ig5Gp0Zlmd+Hwf+dBg12+10ReWfvZ5pIrm
ZjqXt0yp0ILx4nonOavUZxIE4bbQ39X/AbpGiyEKZW5kc3RyZWFtCmVuZG9iagoKMTA3IDAgb2Jq
CjEyMTUKZW5kb2JqCgoxMzkgMCBvYmoKPDwvTGVuZ3RoIDE0MCAwIFIvRmlsdGVyL0ZsYXRlRGVj
b2RlPj4Kc3RyZWFtCniclVddq+M2EH33r/DzwvXVjEayDCaQrwstXehCoA+lD223tx9sW3pZKP33
Hc2MLDvrOISAk8iSPOfMmTOy66D9t3nev33+/fXHnz+3h/fH5p/Wta5zmNoehm7ooSUXOkfYvv3S
fPeu/auBNn/efm2c3urbPxudi2XgU1sG8lb5v5v9mib91ry+a8780PlOOJuQ/zzxQgeycPZzucfz
t+P4/P741al1u93hZCj4w1EeLg2S70KC1hN1Lqb28rF9fvGtR97Ot5fX70fX755gdH7HF8qXhH73
hKM7AE5jIV+O+RLzRW7Yih8uXzfnS/NB0Dx/89OnEg+sxcNhBEctDkOHwWs41PZdGHqOZnSwu/wx
3+7w98f/yoa4tmHPxEDkDUMHQAUf9IwvCT5/3AU/4imjxOCSP7m0wxH3uyeqwwaZIYF3p3wLEpAN
3oTotyDG2LnUPwaRtiBG6AYa1iDihIAUHEfuR40fUGC5s6ZZ758z0LOsOMmkwe5S/ZXHsefZusTj
i2x6xqwD4D9h9GfvygTdjrXDF5spA2W259kE/jRbccijFmvcPcXR+9tchy2uWc8D4mNcxy2uMXV9
8itcKzZViurD+I0QQMpE1MMEypRk+tIpMv8MTr6/pPiQVxZ+AubwmWQWYtKcQHB7rsKyfUhjJVPj
Uj0DZHblKUCOvJMZHqdnaCRgVX2T8n6DcuAK7sOD8k4blANXcIS0QnmmwY/KFuwz4OiMemMmw1OK
rXi5DljppQxkkZDiQ4VvC1SgolU/6H5cMsQel0eqWEsOnK3liBIHgAClqFQZMyeRx5L+DlPQZ5si
QQfAup+UKORsTSlZp6qPXSD4gqoR00T+1ULmlpO1WBgXsg4SvBLjVc8vghjrEEnE8j/mtoFyl4QI
+R50nk7Zz53zqitB7AbX3wTC8Qw7y6t4FXo4WW5yGo1xy2Zl3C1tUBKArt6eqoqh7mX4qGjUKFVc
slJRkz1tvgNKx/RT/Zo6UEF788+TsVafoXWbFvLVCWwP0cBViDPlIYvnriRC31Fc638zCKUVKHvQ
i7tkEzC4czJklfYGmncWszJtkaXajjO0uDTANQLViYywmTWuPXSRz8GqU/54a1RWXiWWUBKz1UyG
LWdj7yf/oLPB6umrJAepw2HN2rzaMYmJ1IYhRKBTHx+q7IU8Oa8RH+xqeZCwRCReZso+TYz6XJRT
W3GlAJQ3PfHJJsFrb19UgBVzHiI+RE4tnctbNTA9gSbF+iRnK5xOItp19BRwa3c7NFAu6ljyal3t
UGun2Ps1Eo7HX+EuAd2pHD5OI656UAagjxDIheVbCHxPMkjaUzKf1ejJIrfuX/OZpcuev3HKhK2T
NDsoDPSgWLdO0j2/GoR4pw0X5zUjC7ljLjqyWVtRCUOsRlBPPCBOqiLcmx0on8XIQ/VBrZXSh1Ue
0rtj7QKTI1mThTQ7IVH2u1Jn1T2Wx6hDMTTKBwdNctjIzdYrAL97ORwezM3WK4BPXVp7AUA/dRDQ
bqTnwJR9RbzRlHZaak9J8r158bEUEGMfZuVdTHre/FT9koBiYnXvO142icWUAGfrl/bCohY3S0os
PZWmilOjKHq4/VJB47UC+ESmvThOrQyvtrnjGEOXYO3dYP5srX1uyVG/axQEs8tCWB/a/wFLfZLM
CmVuZHN0cmVhbQplbmRvYmoKCjE0MCAwIG9iagoxMjQxCmVuZG9iagoKMTY3IDAgb2JqCjw8L0xl
bmd0aCAxNjggMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJVX24rkNhB991f4eWE0
qipJlqEx9BUSspCFhjyEPCTZTC5sEjIshPx9SlWSLHvavTM0GLctlVTnnDolWwP9v93j/vnz708/
/vy5P7w/dv/0trfGYuwHGM04QO+sN9Zh//xL9927/q8O+vR7/rWz+mro/+x0LJYHn/ryIIVK/21z
Vwf91j296868aBsJmwHpzwNPtCATm9tljMdvd7vH98evTr2dpsMpZ8E/3uXh2oEbDULoyTljQ+yv
H/vHC/WEHI7669P3OwvTA+zswZ6nB9yBnfjPiS8AltL9Jb0GAJxkVLSjjN+nv6TP5P5YJqZAZZ6N
QDLfpRdyiXaQN6RPfrh+3Z2v3QcB4/Gbnz6VdOBWOpyFt67HMZrAcEg2ro8GGJDrU8rl+kcb7vD3
x/9KQLwVcGBcGR4cncHgCjwQBRrcT7jDE/qJ0n75cgDiHGkHkTMiwSOhxslKjpgy4rcgDw/14TGN
Paa7U7rkUDoqJjRQIAwaMi1HPg0MUw7DU3gnCWLkWbwCpqFYX6ewwDGGdH0oS+je8u7LUHmIJGTL
tChk51mbdNA9OoIz5OPb6HD36AipZGBNx4XBkS1TyVlBJQWDbsFLDU2atcDMBLbwxznx5YYQyUSP
yx2FuqNEkFAMXtkTSvZpgRGHraBkgeW7meaom2IJSJZVE5VPrI8lu0ubCpAok/ejKS8lXFRJdqZ+
IWQs41pZhDWgeku2LFtwnsXPPkKqLxF/3eEGIoV3Gk0MwwoQO+jOXcUgryHLuVJbEDMCo1ZHTgku
k29zbypLZon9AdYFcIW05BvnynEiGHHIqJMVLw9xu3r8veqx3C/CG80s3KsebhAB12amOBQ/KrxQ
VlJGVJIP1bJUJDpkFg+sHCaVlZBNKzOp5XWmwaXQrpmUhAnSJOhCVPFl4RTF7euyM/NuKJxQZqcU
w7GItgqyvHFTq88cJ1ZuvWSNx2QvKqkxk96GOzX7uC9iiN64Fxr2ExWncGEWcqtVRk5aY96RshTm
6jlNWfOakcyYNVpN7rKemK/73IgLjV4btBK07YDknRmiW6S1NEA+EQiVkO1P7kYVkZrrQ7YZ3gnl
V5vm6AIZN9ItGNkaWSnLophrbLhTY+C98aW/v7bG4p0aY44M4pplRPFFdUe5Z1lR04e0SNYF1EhZ
WrRvBFuJdpX9Q2k2c/M7rKqBi7Y5AYRpZZF32B5GE9b5zXR7BozG1NzuE78Z3w1oBnJb+OXWV6qt
JE9BHKZmr7VDi0NXUX/yuXxWIjaxobbJU2nNGelXVjMBn7ZpvdFz2lKKMhdk0ww0dNOzT3pEWJ5C
Fk12bqb85CCRhLazoroZtcn9hccQ5Rf1CNnoJstkeWLabGDjveLi4wyENx7/4ObnSgZ9DCbS+lwE
x+aEvZ8BSF5ZXHTTLdXKfWlvbQkuAFyYKSxbW0uw8hQbCrbY3i61GIyP4yLZ0FTCWPUarZu9m7TI
turLR3bNDQRreYUaGmvGy5P0jRNeUJxLJ3nxddHgWIfytyDklv6lQhuCCbzxFedUKLzxOdNU/dbn
zMJCS7PFRablm+LF19Mc/lX+fNHaWvvR+dXfbCrRbFZnPUC/OGr52r6bvIbsmY0RlUVCqZAv4u9Z
jYjrY8tlxn+r5hoHUU3Vr1ZafU9+6P8HyUXHrQplbmRzdHJlYW0KZW5kb2JqCgoxNjggMCBvYmoK
MTI0MQplbmRvYmoKCjE4OSAwIG9iago8PC9MZW5ndGggMTkwIDAgUi9GaWx0ZXIvRmxhdGVEZWNv
ZGU+PgpzdHJlYW0KeJztWk1vJMcNvfev6LOBrS2S9QkIA6ykVRADBuJEQA5BDsHE6ziQAmThwH8/
j+yq7p5Wa2xh7Dkku4vVDtlTLBbfI4tVLe9o/Gl4/+Hzjz98+tvxx/H2m7vh36MfvfNcxkzV1Uxj
8NH5wOPn74Y/fzX+a6BR/37+fvDTozw+D9N3uSuexq5QUyr71af5S/8YPn01fMSka0u8+oIK7zDQ
kw1cfTy18f4PNzfvv7n7/f3oD4fb+7YK/IWXt48DheqY0ighOJ/K+Pj38f2DjMIwJ+Pjp7/ceDq8
oxt/6z8e3vEN+QOEe/wg8qKfH/QxEfHBvlV8te9/UFEmnX2+6wPVUB/nC4mND/rAfhSf7YlMmr8+
fj18fBy+tWC8f/jh+/98/q6viKYVeZdyCUKSOSKWZS1U6X90wRKKSz6ORNFRrYgpxeQC1t81T2PX
cCXHnqGJwbuc4qIZumYZtbXcou8xQCrDZknwRmryk3cQUvMy1PGPvzNYfsK/r0Ggf17Z0T8hvIhb
6n/G1wUNImcXKY5VbJLnQUiDnrqCa0ZSyFgR/SSzWIoDyY+zHIuLEuHjrIiuIJe6tSgu5jD22Zp4
NKYSL89jdrXkeXQXJ+vHoSum6Z/mLzTvurXm+2Ztx63iabgM1v+n0HVetTgxZnxd+PV4VaLT2PfB
RZz4OnvXRCyued+f97W20SdLP75x6dulXFoM/ncCM3HCSvm8L/HeviTEMCRjii771LclCq7WaNsS
h4Po3kQ3oZ7dJKRvEtXC66lq8OM6+NRC723v5gI4YkF9DS5jk0ftzRjgw6xB7W0azlh+CpociAAM
Lpqha5ZRW8sX8uK6jr4ELuwBx4lgbOSkLOMd4Hy0PiIc3oUbige+CeksfnHGj2sgyilkhczXMgVN
habHtqZ1JOK7RXfKCpilh0V41rSwqAbflazFLQq5IHXRDLNmHrW1fGleX9XRl/il/cQDVEgEZLGD
23uZ9wGg8b3ix4YlWjX9eWu4iur9PRXDN7Qn5wDOHeBf2oAECS7gI8XqCAx+HoIXuBdmzdPYNVzY
CZgMTcISEKVZM3TNatTG8oUAX9fRt3VxQeC72ezFI3hkUsqrpO+apXgEZDda/VXx6JrVqI3li4N4
TUdfZknZy5JA4UyJswyhB2xQ8nCW+vWt1I8+TbgnLdkaixwnQk0KLKopGPt3zcEKRmp8apqha+ZB
G7MXInY9J99G+ei1SclrJqG6hrre+bpmYVKvrguTlgrcR20tXxzAazr6kvK0e1UQtMcr9RzvlfK0
pfxilnbNameZcWhF68h+d8NpZkkT6+x9AL+5V/DFYe6lQGvRrXFdoLuGA4N2dg5C8Y2UF83QNcuo
jeFLO4UrurlDB9lt9IBbIpCrJk3HDhsBNjLYJGtjIHqJg59oCFATyT6aAl2BtQ5nAQ1zgVzhWTqc
GkNr6C2Khie4KWEVKJHqmHm95KZZBapomqzjNClWY07NXorm1ZzcwTLuNn0hOhZ6HctQDDLD05p2
pNpZ4NJbMxGhArC05rhWr7DuerpmiQrn4sDqVVi6ZjVqY/nSU9dVHd1BMO9moxRHqJdYE7aFPQjp
1jrzuwP3BPzoH7STv3u1Xu+2Pqx3wyj/hO3AJ96ZyX8w82k1iTIntkqAA+BZ6tQ3U0c3tbxuswn9
Bfs1Il2zQgQHnHySUV2zQmRj+VLqXNXRnZuW/VcApaINLmeoE2qDUKyC0xnO8O4eT+A6xfybcYbn
jT/YdWIEWWx6QEC1uBTBV2t97Cq9ayhZI7MK30tNSPb/E8L5wjoW122VMrW67LEN6w2MJ5zbkwJa
cVQ7kdcjGqN2mKGlmIM5EHw1l8AMLieaDer9dvplFOwmTikDI5L0rmHWgEyU6rr+vNT8TBS6rSqO
8rSjkcsoRzrddCPB4lgtLYqTIc31neIa2yFtWbU+U8dXmk3hbHGYF992xraK50XhXbSFzkfBHQ27
WHha+rzeFzt4TM6XNTaCM3usfKI53Z8nJ3eOtwi/IM91Zo9yN532JxgWzebo2pH/pWchMwWmo4mo
wZJCtApEveelON031WyXyJa4iR2OZl08QhFdidEUBTinCjTHkpE+QCYjZjo4u6C3vlrTQrXRRV9G
FT1A1BF1JwcEpAAWdKVNPFqVqBiodUnjUKP1QhA5seUPTzOHClcsdzTCOh4r0jTxdhketYz6ajWm
iUcUqGBL6M9J76+DDtYXusxazk2koLfhzMpPm7xWfB2slbSMVpoLr6xjG87LdHqmKWYsax8OjujF
e3bY+9V21FqBhSPT9dwWEb1UFAG/FgXMErXdFYwMzCYGuAQEC3jG+jYwmyhRzHrFUvB1XdGo17La
MOdqb/1msTg/Od4ViHXUyQXNCY8A0l4pBGQv0kJxBX9gvCkAO3BlwREO7WOCg3pAZHADta6JMM4Y
x/NjgmNgRwJ1gDej5kaUCoixWliAkMeDhDTwZHjqXQHyKdkWAraULk1kgfPzY4SrFHtv0JjmQU/M
TMWIiLmwTcN0VNZqC1ZsJg5KYmU3dfFoqarw9ueRJk8QeGQKBVAnaAgmkgN6QGxyQCUQRAhzZEKE
8yKy0fG4KETPMxBRLrCMjHwGxwX7EFajeYMQqHF2Gn+EIumXJ5bioE96VdtFZIQ3v5sCq2dzBWaU
OGRnvdPMP25LwYU9zpfC8qWwfCksXwrL/LL358oDFEhUVaQ0GdG2V9nGKoKkYk/196yO447Fk5qg
RyrBWlhlqmLnqQSmIRdKwZIAVIFhiNrn7Ro8rQPaimcUEFPgDDaqGBBRTKD2Ienr84yqmHbtbVIf
LS14GqZ36EWy0db7aT4Ek6lMVQecQbe/a/E04Z9NgWOcKjx4pqK+0NVFggIqxmjlIJSwb/E0ybXt
BrTRkjzjP+2wPeKlwGtJYH1nrjmf0yuLBnCCwt0S+9kUmrtAJ2rek9ZSpR3BX5kOB4oNvbLk02TW
IGKtKH5Q2JsEiByyJokeb1QUTV71cB/lotcI3DJYaaPFtFgOatsOzsdoKVn09aveqaN8YTbWvNkz
uM7aZ/vFOp/DnLXY9lgsiaOIZbzupRBfo80mVZ+tDpRAqvBFDyYorMgYJHbR3+fCjhBwToFYqRNb
j8rfjv8FaS8/MwplbmRzdHJlYW0KZW5kb2JqCgoxOTAgMCBvYmoKMjMzNQplbmRvYmoKCjIwOCAw
IG9iago8PC9MZW5ndGggMjA5IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJydV1ur
4zYQfvev0PPC0dGMLrYhGOzYgZYudCGwD6UPbbenF7YtPSyU/vtKM9bNcRxOCTiyIo1mvvnmG0VJ
EP80z+Prl99efvjpi5jen5u/hRJKKuxEC73sWxBGWakMitefm4/vxJ8NiPB5/aVR/FMr/mh4LcaJ
zyJOBFPhXRWjtOjX5uVds/hDS0tYLAgvT36jAtpYDGsbz9+eTs/vz1/NQg3DNK9R+I/3cro2iEZ2
4IQ2RirXiesn8XzRQqM3p8X15bsTmuEJTmoe/GMZnvAEKgx1eEzFOCwAgI5WX8ITTJgPD+hpNqxD
S+uG769fN8u1+UBBPn/z4+foJuy56b2zygjsO+l8mOSlEZ0EH+j15aRguP5empv++vRvNIh7BluP
lw8beyPRmRg2dBQyTN5V9OFxcBgChxA3Bs8ZDT+Jgz4BhpHxI0UjDt+GoQ6TU5iJL3NcCz1PnNNm
ejXJAo4BqpnxyraCG8qlneHhAYcTH4ILrdOt3xOXzHzanF3XfIhtPWpAIYzk4VS+0HnsuWX36pNG
/44z2uT6BJrP6sDwe17tPcYTW18nU/L3k9Ian9h+kxTURL45ehLCwQI4rfOBFwbJlKCG6YFIuU6u
oc4xd+ShB3mOAVP+12PIgF5zEamRU2MSXrgA4aBSalMe1OJndWVrHiqw0eDMGC3305xpFDdBS5w5
B3eXdJqJMenW/+DhuVty+qjk/Dcq87aSM0clp3vZuXabXYKZqynlgXIELWVqh0xjzowHLvG0wjon
67YO19ctUxlaW+/WG8x19mVDRuhWETAFp2gTh4HEjKlQCqIkWONy+S+7FPbn6CgFY9hj1rLe0PBB
efn+04LeJOCY21prldGGPYQYwHGoQGaniZ68igjtknCOKbw5QbdqToQuSW+tR1Vlp3SQAKWKhWAx
C+148/OOu9k98gdjkIRPLO3V4laCHkIPfS+t7bbcN4XIz1W7ieLaJQ6SB7fnkvJQpzVcSZMpYaBS
WZmXVRxJiGid5vGBSNgDkQDXS2f120TCHYgEOCu1/77pyy7C4wskU97WCpnCzbWHF7gMFlkkA11y
49oimnuF7lPVm0T2LAIFjyuSwo1EQ5flyd+FoBT+VeJvzT9qeEXW8S7xrLXSX1ArPF0iHqGpufuF
gFVPfnh3HvDYWAkO9syxJllWr1zjdx0Eo6RHpzaZS6Mv23otHmtX2KK2xM5KybdZhXYqo2yVvMqM
gRqsPWN188Ez18h6Hz6oK9NHKeUrF/S6vMRdNlJeKG+tq7z7qCbbo5oEK9G6t9Vkd1SToGQP22uZ
vhRNc6NbzH1XX3TMHp93LzbcBphPxh2U5u0t7OaOdJd/2qE0bhNdojSM3gi7krVXv5nkxrUStb0D
Ifbb5p6InvvP/7yMmi6xyTKrH/WoXsn29h/RjqKOpBr6gucDpSqvDarNLOc+DIxkvmIV1Y7nh6lT
3mPTVh5vtG39TwbrzW4i/xkoUrs4t961ntYtfMEZ76YTlXReq3ag8smsyuuD+A960IU8CmVuZHN0
cmVhbQplbmRvYmoKCjIwOSAwIG9iagoxMTQwCmVuZG9iagoKMjI3IDAgb2JqCjw8L0xlbmd0aCAy
MjggMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nG1QXUvDQBB831+xzwUvs5eLdwsh
0PQDFAoWAj4UH0SbKqhgKPj3vW2sRpGFY2Z3dubu4IQ/qJgPx+f+/uHI7WZB7wyGg08cRZ1G4YDK
IXge9nQ74zcSthoOhHEU+ZVGrT83XvjcMCvjmKBv0RP1M1rl0KmTnwiMXORFyGlxAn97FDd1XWwW
V0tG07TLr1fkyrdsO9LoQirZI7mYH9Q9crGuOESnmlm/q8UjYt5ILSVUAnyGaI0HO6o8DVgbFJQZ
yqgwqHKJ0Nx117TqaPsn2GtwURJLVTkp9b/keLIac5KZKtKPnX3Olj8BQcFZhAplbmRzdHJlYW0K
ZW5kb2JqCgoyMjggMCBvYmoKMjM5CmVuZG9iagoKMjMyIDAgb2JqCjw8L0xlbmd0aCAyMzMgMCBS
L0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJVU22rcMBB911foObDaGY1uBiNY27vQ0kAD
hjyUPiRNtxfSli6B0r/vSI63jrtWMQahkcfH54zmDCiUv8R2d3r6crz78CSb61b8lCBBgQ7SY6Uq
j9KAVWC0PH0Ut1fyu0CZntMnAcMrL7+JIVePB49yPEhQKYbJ7pz0WRyvxJ5/OkXSk4QUbPhDwPzh
ZPsSY/u2rrfX7atOQoxN96yCH2bZ9AItKR8kGaPABdk/yO2BJGlGI9kf39WAcYM1NLCPG10jRA46
XhCB0v6QXiOijjkrIOUDk8K8BPAZgSYnGcBysju/Yjw3YkMb3/evxb4XN7kG2zf3j6MKvKSC2Vsw
UldBOa5CVmFkUMh16I9JQ/91Ctf8ePg9AupLgJ7LiY4BjdLOjGXBkEuid1HXutM2UuLOS4PExKnG
wMJTvMyfSvydUWTDOv6mxN+l1sI5f0pXyeXm+wy59Dnmra+pjdZPxOl0Z4tibEkMgULmsEqMK4nR
VnlfzcRglmHSYo1bpuoLVLGyKnhaRzUUqGKFypIrUKVmqP4i36rE16FytLLP8aL/R8LWK272GWHw
YwdQao/k/9wqucmHTm//IwNLfkXtFYWV/Y4lw6ImBXbe8FMdbFF8nlA6GTWPn2X6JbtWpNCsbHAs
2TUEFWDe4PqAecamC8Ds3b9TJ8+cwb3pRLdsX/JpDueELkncnY1cmEpYcrINqoKV7sCSky3jun/c
cYhY1XmMsrJBVDcY5gXtG/kHl2euEQplbmRzdHJlYW0KZW5kb2JqCgoyMzMgMCBvYmoKNTYzCmVu
ZG9iagoKMjYzIDAgb2JqCjw8L0xlbmd0aCAyNjQgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0
cmVhbQp4nJ2V22rcMBCG7/0Uug5Eq5mRZBmMYY/Q0kADhlyUXrRNtwfSloZA6dt3RlrZjrNRcFnw
WgeP5/s189toUH+q1fr+4dvxw6cHtbnaVr+VUUYbDKqGRjc1KGucNhbV/efq5kL9rEDJ7/5LZdJS
rX5UaS/miTuVJySUjM3kbtj0tTpeVHt+6TQSTjbI4JIfNBAfnNw+jrF627arq+2rnTJdt9mdKPjH
WW76CgNpS16Rtdr4oPpbtTqQIuRwpPrjuxbX3SW0uItX10FrSC4bIJkxOx5AkFuwcb5737+u9n11
HdNfvfl4lxOAcwnwe52xCpugPQPE91sVNDBCf2wNdP33abjNr9u/OSCeC1izEuA5oNXobQaCkGAs
7jhbTha6SxQUbMGZmD9inLI8xVSUt+xk7Hlstt1pGCOkBZKFDaD8xSfxeX4q8XuryYVl/LbE76Wq
4Cm/G9KOAHsRw0xQKB0xylFCSy7ho+BnTRg3CkVFXFfCJaOBs1yE60u46HRdN3PcAwNJntBEPAjp
8G3Gk5NnnkPnBgEy8enAB2IaiLElE0sjlY2NfTDT0cnbMMuWpH5JsLogGDROh5qWCRYKgkED2pGf
C5Z7gMYOEVFOJY5uXIjKsTnk8rAZe+yTF4mbErEH7WmhI8BZk8vIrtZsC3Nkigc89PWU+3+q43n7
K/kfYK0pLOx/KBkgIGnjnhjApIExHW0kX3dQy1V8PJjD0CSTSkidlE43FnU0iSRR+kaYx20lypIo
SzNln5eoZJENabALLQNKFhmCDuaJZdjR/IBGj5wUBPnINR76YfhU0ES2FMcWaEsO6YJuzMJ+h5JD
Oo7r5/0O++xOzCqEuGXP9OOh77mjZv419IWjZiijsedPRph34V6K49EnNH5IQlqel9p6EDIaLUxq
NH+gQGqNZlV0rf4B2Jwg9AplbmRzdHJlYW0KZW5kb2JqCgoyNjQgMCBvYmoKNjk1CmVuZG9iagoK
Mjk0IDAgb2JqCjw8L0xlbmd0aCAyOTUgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4
nJVXW6vkNgx+z6/w88L4WJLtOBACk2QGWrrQhYF9KH3o7vb0wralh4XSf1/JVi5zTk66YSCT2Ios
6ZM+Kc6C+ad6OD99+e3xp49fTP92qP42zjjrMJkaGtvUYLwL1nk0Tz9X79+YPysw8nv6pXJlqzZ/
VEUWp4XPZloQVfLsVnez0K/V45vqwoeuNeFKQB5O/KKD/OLq9l7Hw/dt+/B2+GY0ruv6Ub3gH1vZ
3yqsk/W+NuS9dTGZ2yfzcCVDyOrI3B5/aJG6E7YudtBC6k58BZc6SC0NXQgtjrKNgbcdycV3P96+
rS636l22/+G7D58nC2DLAj44OG+wSTayB9kAb5IF9uH22Drobr+v1fV/ffp3UohbCmsOBURW6C1G
P3kEqXhzhWtHTYt19irlq++oBez4rhcP8iLVHnnZ0+vu0ObpyQY0GJOt+V+9AWd9Stkd3HHHbykE
x8gQsMpoqWlmf/yMEGVs0Hl37U6B8elO1LoRfHHOjY5cL6sh+8YSHhyvkz7lKwY+pMkyIj/wT/fx
LEGJrneBwnRLOFAOllfFckRajkcnh5f7ojYbiKryIiqyeeNiCHpJL4ySXDjAhRrVyG9mWaQdNMIO
Gh5s7ZqDaMRdNKixFOoNNEq9wLhEI0enPIVuDk3KoUm67WcBhpIrmOE7FzBWoKxApLZgQQKLhtKX
DbzkirxKMBkrzFgNHVsuj2c3ZGFSYQWXj8zLQwYEyS0HIeb7FdBsCGJYAL7PE7Fe1eRCcouI78Wo
PRTrHRQ5yDHWB1FMuygCWgJ/EEX2LgIqPjN0jFRQ9qA5Zbmqet4ghVBqCguUq+iRpneB9aoZwCek
1+PU7FApJLJIB6kUNtuDcikwAql5waVnpkzIeUbXhVBzD0AhVPatn9iUvfpfQoXNBlHQh0CW0lFG
hc0OMcEPAbjzblGqP5ciKfWIE8Iz1oEpVLZ81AXKpQ04F9GzapVypJI5sLBkySFGu59zR8ixEKG+
Xgoq0++SJaW6sV9eKuUuxTliAKc13osle/UGO00MkDsOHaVN2O1igGSbdJQ3S2OjhkPhVxV31WYm
sUMhHVoH183N46tZ9WVtL0SYW2TPp00atM3OkV5InBRVv8aGC8VLX9N5Ss4dJy/1X3SMzNBFot8d
pTbbnRIATz6NRPVQ/W+2O63/xtkQ49YoJVl43jFzh88jj6nBH82uXT6PyH0etuq5KRmgdRYzE/dT
Cx3nNpyLq9AxJ07/tfjNUw1kSVR9nqdm0eQyOXAfrleJxsmYAb+u0n1OH5rSWjXrDH5XG6XR0NSU
p9lseJZUdKfizux1cktBMDuX+slt2sNd6tN9ODL3yGjAZDQsCbCNi4+2iVs8O00psJpEtYZdLjZ0
qGRW3J3mlNkazlsJ7dxiL8W1PPKQ1tQ+A262Uh050GJ9lABxs5POBMgff/g6/73EmK/MCOvZWieN
cllTnWfJMmcFuuCwFzbPH3w4sPy4Pgkvwm7UlryVjwDN5LtZ8f5DYE7nuMjqmJi51J2XUec5Du/M
fwiFVQ4KZW5kc3RyZWFtCmVuZG9iagoKMjk1IDAgb2JqCjExNDYKZW5kb2JqCgozMzQgMCBvYmoK
PDwvTGVuZ3RoIDMzNSAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVdbq+M2EH73
r9DzQnw0M5ItQzDEcQwtXehCYB9KH7q7Pb2wbelhofTfd0YjyUqOk204HCNLo7l88814Ylsw/zRP
h5cvvz3/9PGLmd4em7+NNba1GEwPQzv0YJz1rXVoXn5u3r8xfzZg5O/ll8bqUW/+aFQW88ZnkzdE
lbzbalWEfm2e3zQnNlprwkpAXnZ80UK8WC0vdTx9v98/vT1+Mxs7jtOcouA/9nI6NwR9SzQYcq61
XTDnT+ZpIUPI6sicn3/YQzfuYA9u5Ifnh+vGH8/fNqdz8y46+PTdh8/ZBGyZYM3eOoNDaDt2MVpw
JrTATp6f9xbG8++1uumvT/9mhbilsOdYoWOFrsXOZZchZHdxb2d5TJZG2ttFPIdxxztBnkAcDHEw
+ooHlsUZvcjGCxMQX4k6IERZVhVl7aEIxdOkAiaWwr2qrSzZohNCMuZwFndCEotnPp9aXOWTQKcO
geSAT1D034Sf7sHfuZZ8eAx+dw/+TmgLV/DTEr1keOgySusqGOCUIdRtiZv6BNGJASImnQQ+pqgV
lElQQKcpVkVWFHWKi0AUEmqoSq0oY6Ei35U8eSdrPIq1aKfKXqJBKMxBWw5hUM2RF8gqyOaMZ1PK
nHI5U6hs25P6HUqKk7N5u2R4G3ka2tD1V8gzLkJCe8yIKHpqlOg2a/w91ljuVd2DRdvdYw03pw6v
i9b2TpB0gjv2awFxTFUZxPC8hhcB9ZcgcuiYKMY0WWI+ltGzyiPSyos1Mc5WphZYYitIRPE0jLki
j5nRKUNU3VPLh7GwJm1HNrmLrOZtJX5h5toxOF1LIie3I0yGaaVexc6vkQSCb901R6SsYmVkT5ak
clZv83ZXRZ3Nwqx+3ORRf4dH4H3rc6/+vzwKd3jETbNFfFUDMQ+v2kFpzlzTvgpOE0a+rLVYMsC7
1ETY1mGt1KAlTAWs9XoiUix1liRN9FGbeibTXMTdbSyHe1gitIT4GJaw+fXPYELXDsFfFyVEoqTY
sfDiUAVQgj7Fz6zWjCtnK0jhmr9z5vcr+Au6WhHpK1Heuwr4bGKq77A3tBI2XBbNRcPlVGPRsfZN
vNBbf9ArX6fKvHhDxI3r9mC0ORn1ofVoBuBCpZxOsK0LIeYT7+VzczICy1McgQlDC1jS6cosx/Uh
CIrz5POSJDbPvTa2zVnWEHckcgkxjn08E/EWC0/55lHOWQqts5h0nNDagwild7GnKtHLdtIyJwOO
Efb6BfV7vQKDPF00LR98SuukULu6Ux9YIR5pUXcc/6OmGlJqYu1rFLUJCuvxAfizqYGISLbj4PIp
+b2T3c3BS7PLY5cjeDS7m4NXzm7Hw3kfNtILR0WHCbnTwpWi4iFNcZlsp5Ey/iyT8Jb0+bReRhe+
lmNJYfxYRHRTWlLBXfJmlSCZ6uIYVbPD9jrVFlfUFieE7Wqm7oG+Obco6MQ/akJ4FPTNuSWDTvyj
ivxWTWH0VEnCa8fjhk6qjN8pxrmk0mGgNa4Ub0wQYzLlDkMRSZeQpISIvULhnfkPHesq9wplbmRz
dHJlYW0KZW5kb2JqCgozMzUgMCBvYmoKMTE0NgplbmRvYmoKCjM2NiAwIG9iago8PC9MZW5ndGgg
MzY3IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVVduK3DAMfc9X+HlhMpJs5wIh
MM5MoKULXQj0ofSh7XZ6YdvSZaH07ytbsePM7syyBBJbto+OpGMFSlR/i+3u/uH78ePnB+Wuh+KP
AgUlUKNqbMu2RmXAlmBI3X8p3l2pXwUq/9x/LUCWavWzkL0UDXcqGjyUn0M2Spu+Fcer4sBOcyTK
NvjJhg8ChoPZcI2xfdt12+vh1V5B37v9HAU/zNJNhYaqtK1V2pgSqkZNt2o7aqWJ4bSaju87rPoN
dmh6fll+aRfmTf9hel0cpuIm0Ny++XQXHeFTjhjfglHUNmXFRIMfo5oSmep07AD76UcO537f/ouA
9BRgzRFjxYCmpMpE4tgE0qbtyXOmDlwgDpUfD73uQAertv6jvWHv9za8TZaaEB/5oeFlshFFNuo4
YrSIEDbvF3DvEvuNgFFHBxgFfvYWgJO36P2AwGYMODbRDftlSQDkRKiCGNawuAuOPTkOgudNQF8C
AkoonBXtjeQd5uyzkUSgg9OUqTm6mU9SwtMVqg1XuT2pEFrtayRkhpQ/CVTr8+rSl9TFXwLzMnWZ
S+rSbdlU9Ql3MiE/WU3MUsoBTbT4UkSxUKwHuJD1OYM+aNGpSVURae57ygtOwdkuGd1cwXpNgN2N
awIxszmBi1JFkVErNCEJUdZHcDiv60xdZq0cv0PDDPuMPLgd1qgfpZjWFxQO3g0s8fc2c8FxE4Xc
hVh0GNMgss0iDQLG+erQ+vwYzC5Gg22MRiJsJK2OBt8EuSnG2lcJU+6YW527JGV7QcrYsPKgfpmU
qwtSxsaW5pGSUzmjUihX5mlDWDQa2uBKuLbugnTId0wtreVM7znByrqQKLheettipQONOPbW19Vj
1qlc2q6aa96bnCjepCat43Qp/dm7Rd5CewbPtOiCELPfyZD+Hs8pHStbav6uSyC3jRJx0d0uKksi
PBHRjfoPbqfeRwplbmRzdHJlYW0KZW5kb2JqCgozNjcgMCBvYmoKNzE1CmVuZG9iagoKMzgyIDAg
b2JqCjw8L0xlbmd0aCAzODMgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJ1WW6vc
NhB+96/Qc+BoNaOLZVgM6/VZaGmggYU+lDykTU7akrb0ECj9952LZHsdH5MNBiNrbprvm5HHWTD/
NofT8+ffn979+tkMr8/NP8YZZx1m00JnuxZMcNG6gOb5Q/PTK/NXA4af54+NU1Fr/mxUF+vGJ1M3
2BV/u8VqUvqteXrVPFLQpSdcKPDHAxk6EMPF8tbH4cfj8fD6/N1oXN8PY8mCHjrlcG2w89ZjZ3wI
1qVsru/N4eKNR3LnzfXp56Nr+wc4Ot/TK/ArAwAv5OXOvMr92+v3zeO1eSNnPvzwy6caFbaiUrDo
gsEu20SnlqDBZAt07uvT0UF//WPpbvj7/X/VIW45bCl9SOQwWEyhZgFZMwB36pGP6o8u0QqQVnDq
H+oyOmQFzwqnqkDZ0osFqBBkNnCR5UGsAjvzbhb6gYVZHQ199CLFIqUwIhhFiXEDEXqOLR5zP4UZ
NEZgXXgZXr8HbwrWx3wfvGEP3sSVCit4YZAMbjOio7sJUgFS05rhz7xDFoPAN1bolaVxAlkcLnxd
1i4rV0JN3cRLAVCJHRhXCR0rCeFlTOMept5ZICjuwjTtYYrRtm23xvTCxaOVMWhZTJAs4ZzL9qId
SakzChgnwVkraC7gVMutQMWo40jmC6MB/BQza9RB+cLHuYgV9XGC2c+kDKpwqifYYNSvGdU+gcJ2
mipiXUOr+GqTS+kVs6pxrg6k3ibKt6mgizPh+vZAFL/z0StCpzmeYrLsddXHOd/uizTGfsvtulk0
Fz8xUHISKD0ThmdtGelBYc1PbVS4uLPDJh+LJvO1vJbna79QC1Pjh+nYKsmlBuec1vzo3bdPEeRo
Q/vtDPmXu77d6XqI0cb6X/nars87XU/3kEVcJ+JTudAqfpeSz1h/BbqdbrtKCRyn9Ol39WKW3eah
so1ogG+i+WoDZ0POkiXuZAmbYwU4Gic8DUNI40XyU55hGiswSNvSgSMlRGTpyrvgZbWUQ8CIkWW1
wTxXpEhc4O84jylSCqrDPrgK5TpjndE96o6WpKzFmm61G3uV1ehu0H1u8GpR9ZdnwkG6RM3F8EwK
uk7yCyJMYtHZq0XYnJqUJp6ZWn8vS5tTU2WpSzQDbnAEZ+EgaI7cTty2NAFoww0uFdbGmQdAWsWy
1jeIXSpjjXwpx7NcOS5WXrAPN8jSXqAav/Ho937isDkZKYQt2OThXgg3J6MKYeostnmrzlEQCoye
lITk47kcBKrLDEGBcT+tzeFE0+LLKud709ocTmpaAS36uFUaUs/enSgJfsbSd3h79Dfmf9g+4dUK
ZW5kc3RyZWFtCmVuZG9iagoKMzgzIDAgb2JqCjk3NQplbmRvYmoKCjQxNCAwIG9iago8PC9MZW5n
dGggNDE1IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJy1VsmKI0cQvddX5HmgUxmR
O4gCVUkCGw94QDAH48PY4/bC2MbNgPHfOyP3qi6VrcPQoK7cYnkR8SIEB/b3cDi9fP71+cOPn9n0
dh7+YoIJLtAxC557C0wJzYVC9vLT8P4N+2MARn8vPw8iHVn2+5DuYtn4xMoGiaK16L7qpV+G5zfD
JSjtJWF3gRZP4aGA+LD7XMo4fHs8Ht7OX52ZGMfpnL0If8HK6Tag19wbzaRSXBjHbh/Z4SqZxCBO
stvzd0dhxyc4CjmGHxV+wNEaAGghpnDw/e3r4XIb3kV7D9/88KlohC2NQZEWiqF33ASLo0LFHIdg
8+35KGC8/daLm/78+E8RiFsCbXAdTBCoOBpVPAAXrUcYn5CsR7JeBrvjehImLMQ8xjNJG44OQNDi
3G2b9O9CngvaP5MU122093BUnkChN6CjUpzp3nW0eGwqAekr2hS1okpmgUqidiCVe5AaxaV2j0Gq
9iA1lJmwghRsQBIzklJXCHR1aiIQEp5mjI7nO6BBlUikCwnrbk1S4iXQytTthIxew53UUUQE1jNV
v5oJ8X0IiRlfq0xnYuMpXjt/6BXqdLWZkVPlbrz0Xryk4BBgfiheZi9eqLm1frMEIAKEzWo4jWDp
l6rZFX/mMSdqytFThSo+cS2wPl1wubxkByv4tDHXx3KFuO0u54KEHMXXtyPo4UK3I+MbU0zKMjZt
cQBtO1o0V8dSbd8Pnt0JHgTidFY+Fjy3E7yAKNfSrIInp+RJqjfoCqx4hqKiCL4HI0RXBl9tZTFV
QxjL4Noun0uJbIYLL5HMSpFMJVcKgQUxqUecq5YcVNJwrnnXCktcpa/aWtF1NGCqx5kMChGnY1yy
d9RRGTnC0Kih81HknK0R346EBS4driLRchPO1diOkVoSArEiVIrQ1XfyRO6Qhd/LNwVc2QfJAjZ7
fnFTWg5yzRbCk0unNdPiXdCAxhmpluJMJZ/IMinhVAp9CpPMmu7LdZ5rVHfMRJ87wLXlds8wuttO
O66QeNe6k2siefy6e6k5CWgRnBLrqS6nl3ppTHrV474cPf5HIoPh3ul1hE9LKRGoS6ntVSlBLuw7
/nb9wiWE5RJ2jKK8Svw0qa5IqPVgRr9hjqfERUU36hH7qQk7rDvKyv06TFqahq7cIXLMOhb4X3UI
e4Ors2G69g/W4d7g6pBbtWabMBJiGTiwzkSJUbtuF1An9+MUlkATNbVcDV2ohnT7skxE0iBt6+lz
LoCSlZ2kxfjaxoAyKqyrKrelzaqCnGSnKr+2n9zGRH11bZp6R2JXLPmWheMsr9CX6yK879i/AXQA
VgplbmRzdHJlYW0KZW5kb2JqCgo0MTUgMCBvYmoKOTg4CmVuZG9iagoKNDM5IDAgb2JqCjw8L0xl
bmd0aCA0NDAgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nO1bUYvjNhB+96/w88Fq
NSPLkiAE4iQLLRz0INCHow+lvb22tIUuB/37tTRey/VGY2ft3XP2zEIyduxPlr/5ZqSRVgrI/81u
dw9ffr//+ZcvefV+n/2Ty1wKiTY34IQzkBdSC1lg/vAp+/Fd/ncGuf97+JxJ+snkf2V0LT6e+DN/
POGh/LHsWO1Fv2X377Jj3WgXCTsX+IOb+kYJ4caO+X+M2x82m9v3++8Oudxuq0PTi/qvfsrqlKHT
wpU6V0UhZGnz06/57Z3KFdZwKj/df9xIs72BjVTb+qOoP8D6YwDwB7Kqf/jp9H12PGUfwvPG9uBc
e1BKAdrUDaCwylB7JndCQkHNwfYGNxK3oU1vhnborPZmGU0TTONNHUy7mn0zvCgXzF1gLJj7FGvI
sgYoSoV91nwjpz8IrX+jFrr29cSNNd2HSPcx0n0Xn5d6UUbTROZD3+xq6vh2nDd3way8mSRasURL
FAWek+ejU7WoPQAjheMAQK5KfiFRA/jwjMGULPcFxz06FIiK4R5YByiFqhWfRtlFmQfFg0rKvM4x
3gz9Um2/IFwQ8g8EF4JvKSSclTmE/Ag+jDYCS3KvWe5tnfjBTuU+iUIkepah0yUuDizmvS/BnMp9
yXJvQFgY1D0f/i0Iw2JVa/j/OjHfsNyXIErpxnHPhgBrBPgQkAQMIR2CD1N0v/MmtEkBdv2z3gTq
7T6ZKiTGNxNvI5lQhrni/DBV95blXoMoZDGP7lNY67Dva+nesdwXIMDNqfsk4Kr719c9nK35tOSr
eq7mBoU/NNhLonRG9zS1x+izDKFXzNeyyGcLcIhSmJHCHwr6CSiavUHVSp5Mqu14twAq/hzjBcek
W4BpYx0EhDDpQxnfzFvKJRNjPrBVPAQptNXzcJ/CosBNkT08MnZm8dUYmpFuU62r09niDdH8MuSz
lT2UUqjBCf447hNQ4GIePsQcgG3kagYAabFj7CeJHYteZlgOW4uini3sgau/TTkx36dRItddD1DR
A9Z8/6L5nq3sgaElyzmEn8Qi3mHXKl+uE75XUj5b2oPSCVMOKn9EEGimeUlADCghaVMQH/aAxQhw
McqnxBeVP7RmC2xtD7QTuhyp/BFBwKHQHCx0FnhC8MdQ71fRt5nQgMExmD0AdMHQqjGaNoKSSTGJ
xh3BOcuYluh5D/FazwQ8OStbE/3zIvUNeylssvKbsW5U/sA6LrDFPSicUJpbyH0e+SnYkeTDIfJ1
iO32hwbVm88WU8lnq3ugXP19btfFNPJTsOcn/A35Z+X+ZOV2Ocwsn3xkq3uAVrhifuWnYDtj/mbE
Esi3vfhNyn/eVOHp4OKKQ8NU8tnqHoAVpphf+UnYIUKp1EcusQ9mJy+cHyLGFBG2elDGvmbGZySf
re+BtEKPrfFcQH4KtlmjwVbutDWnimpmBnw0r/mmMsBU8tn6nrNCqZGbOMZzn0QNo2+wMbzHEn+I
9GjbC5rab8cMCt9FhOO424p0glh6cXgq9Wx9z1pRZ+e5qU+hDjO7bvWZk3q2umeMcDhyC0ffTPJv
wr8ppKCbp46r7bTAc4i5gOZ/Lq3WJ2O4xVRkFlXeQba2Vxph4JIdHOOpT0E3InctyeTRVZQ+Ud+p
uexi71WbLJgRAVVXrt43JlPPVva0ERouV/1AwE+hDgb8dY/XrAGfresVRqiL9muOV30KenrAp7Xc
qxf1y6uereopI0CO3L1xIfUp6NcI+NFNrtk3nkf9h/w/K9Ry7wplbmRzdHJlYW0KZW5kb2JqCgo0
NDAgMCBvYmoKMTMwNwplbmRvYmoKCjQ3NyAwIG9iago8PC9MZW5ndGggNDc4IDAgUi9GaWx0ZXIv
RmxhdGVEZWNvZGU+PgpzdHJlYW0KeJylV1ur4zYQfvev8PNCFM3oYhmCIXZiaOlCFwJ9KH1ouz29
sG3pYaH033ek0c1e26dnS8CxpdFoLt98GkkB7d/N+fr88den73/82I5vp+avVrZSSHRtB73oO2i1
NEJqbJ9/ar550/7RQOt/zz83kqe69veGZTENfGjTgFflv2X1loV+aZ7eNHfatNaElYD/ONFCCWFh
9brUcf76cjm/nb64tXIYxlv0gn5k5fhoEMkYcK3SWkjr2sf79jyrViGpU+3j6dsLzsMJLjBLh2o4
4UWqAS5yhBv9AYQvnNQ0KEujPOmn5NU/WHb47vFlc38074JH569++JBsgi2byBQjdYu9E5Z8Cibp
1gkgrx5PFwnD47da3fjn+3+SQtxS2FFwwJJCLdDq5CN57f2TIGcA7xngoLzJ9JgGjA9XZjQ9WJKF
RnD+k/yXJBl19CEs/hW7IBoUUDSiVvBa5N3HZVayktBpc169mPPitDBKgcufEy+qxckcHZ4jDaNG
XGtTPAGjl/ITkLx0wbjkLMkbelsY4gcovcg2oMk2WK/zNnRqoVF7UfSL4Mo7DFAUZlRsZ6vTlPF+
lS1WpjiANm8/DauIzQzBLR/iUlJjgqWcyzBSy3mLIU3UoleOUZCALnoWtQymUkKTVBceCbOa92tA
HdUA/aPUr6sBfVQDqhfOduuocmXblNiIclc80xHhNBfAfst4hgI8ZYdTiMICrbDUwhhK0VEds0aE
FdeJB3SJZYbnyAnn2vkchUkTz2helmAdLAajbR7mUg0Yjxh5YRMvgxPYVOZg2OK43S4CzBECJBG0
fSUL2iME0PFgcc2C28FN7lc1ZXPYq/jZVBLMVHJIiCglCq5inwCymBWcPJPMXGwZOViKEgzFkfcJ
jGyr+jslNcUYvAcO1kzHo66hfMvGMz25zLmhvpn5FrnfAM/SpWMSA2eEXlcbq7UJV+RfjqdS+yjp
DlACxghjX8kT7gAlZJRAXFsOU6wS+pVTIkR15rxAri+OoC5QwOywSzBJ9eNDHD5r4kGZ2HadAuWq
qlKsuYJRMG2DnKviZ0RBID4VIRUPc3BY0h5cu7OlKyMIZ2pBdBVL+UVcLgWcsZBewIsC6t3Ump5j
JFae4K42ACt67JfqbFY3clDY0OuKnFaaLIheqz3D+hB2NZfTsRCAzRzCDUPETCIC7pQw5Y8jm4Me
E3fNikalcuiZakqFvhANjSBo771oTNTb9lUk+MDZ09VZKujdRPUMK+gB60RpBmspGZubzBuH5FaS
W5pNWrDLBv0RG0gQkJr5/8oGsHk/iLjsrXAKVt7+vzLMR0xsnOUiLsuuIeTY0KELFfFHkZF7kcwu
y469tAdhebWhMkxSEXp5w+XJtjox8hVIpY0rQwlJmTtoomfc8yHFJVDxnsoshKFvWtjCfQOp3r84
Hd2crK9+88r0H92cLArj3Gb61eogHhesB33VByRAFIbnu0mgXYWeiQN8qIFKHfqnKDiFsGWeUFe/
bC2XOGadvhzokL4YfSplvdwA573+ftFn7HEE6aU0dIuofRb5GqOEQdgJf32lw3hjyfaWDotPtT3i
PD6ONAqtPmkV7z7CddmnO+iqeXnX/guDSL6YCmVuZHN0cmVhbQplbmRvYmoKCjQ3OCAwIG9iagox
MjAzCmVuZG9iagoKNTAyIDAgb2JqCjw8L0xlbmd0aCA1MDMgMCBSL0ZpbHRlci9GbGF0ZURlY29k
ZT4+CnN0cmVhbQp4nK1W24rcOBB991foOTDqukiyBcbQ7gskJLCBhn1Y9mFzmWyWTEKGLEv+fksq
uVsOdvcEQoPbsuTSqVN1jgwWzX/NZvv47eP9X2+/mfHVrvlqwIAF6kyL0cYWjQNvwZF5fN/8/sx8
btCk3+OHBnSqNQ+NrqXpwSczPUih0hiqu/Oiv5v7Z81BNq0jUbUgDe7kRcD8YnU7j7H5re83r3bP
9waGYdyXLOQnKMdTQyRgsDPsnIXQmdM7szmyYZJwbE73f/R0HO6wxyN0xMMd9cAD9jDiXv4Q84h2
vBs4yFOdTFOwTRddO/x5etEcTs3rnNHm+PHDv4/vJ1g4wXKEloXMBI68tw7IUIc2eJmyFJx5+9Bs
nj94SW7/xZRgL998miLRUoIu2FbIZuis95qeM8FSTNn1gMPpnwray/HLu+9TPF6KF9i2QpdwjDSx
he7MFneZABDwtCvUEW4Hgl6ud14oEZa6cucQ0x2CsJmo3aUx9zJ/Zmy+O3bOBpptH+rtKVMf6QAe
YtnGrQUjDja6K8loACTJxq9jUkaoi5a7bh2TBiNCB9uCLK6mSa3N1a9iLiAjymy5zJ2HPUblFTV8
l2aVdYLM+tp2UUTQuWs5+FRJiZwrlnc7agrS4mv0eqG3i6tZJIBT8y0y2rL0fljIXiuScazmFNEi
u3mQeUrHJOBLMTx6rRP/INdKYe6KwiiIfD3/nMT8usTItzaSW0jfHYp8fBGTd2mXVGzSchcZjRCA
Ck15Nt9rM5d3JOMiQvbKx6jD8gpAyC9tk1pvakB8CiMu9at0p0RQpgl2oFa6h4PuosaqeyabFXAj
R94O3OtShmIMuVQXE1n0m/TCxXM4e871bin4KVjn4xJ+7fYg+7iy8WFVTRAsYDDEKOfQOVpnUYw7
NYRbaXs5Tay/iqIqW11qn88j4XVieKtOICWV9W6q582SP6XEgrGVw3oBXJggSRTWkoqkMvtuXVLh
iqQwdtZNEniqpNp1SWF0NsYlSWHQNpoIDFklqcvKE3JqgNq2RV9lLhuHJK+DynuFZ5XcWDrblz6/
4sUFaRvlxF86k7DN1ef5CVpAjkMl2dSqLFIrLZGrqzO16Ma5CEtme/WNW8dxASt/k/HNPwayo6Z0
E5cuaXiCyv4s7YumamGnu8JoIbsoeFEByezyvBjeTbiutW1cPFWXdaPILs1xE1mlL876mnGpIbJl
aTtc3DqP6Sk6xPSFGvwa6XJCKunaF+tepZYzi/bTp+QPIVsvrRuuIKw7IJydOVEgJ0NRSG7K9GV4
kwgMlpl/0TeL8CZB3Txq/c3CPLOe1+Z/XKPS0gplbmRzdHJlYW0KZW5kb2JqCgo1MDMgMCBvYmoK
OTg2CmVuZG9iagoKNTA3IDAgb2JqCjw8L1R5cGUvWE9iamVjdC9TdWJ0eXBlL0ltYWdlL1dpZHRo
IDgxMC9IZWlnaHQgNDkxL0JpdHNQZXJDb21wb25lbnQgOC9MZW5ndGggNTE5IDAgUgovRmlsdGVy
L0ZsYXRlRGVjb2RlL0NvbG9yU3BhY2UvRGV2aWNlUkdCCj4+CnN0cmVhbQp4nO29CXxV1bn3TyAj
CchMDiEhM2OYEgQSxoRJZUhABpkSBgMoCYNAEJMqoIIERKBKDwpqb2siXmxREsSqYDUqLe99+w91
6HBfjXWqb+8N2nr7trbyf8hTFot9ztk5yTlnr73P/n0/v08+2fvss/fa8++s9axnXb4MAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEFgaGxuLiooKAHChpqZG9eUJAACt
pL6+XvVDFCiG7A2ZHNVXonvefPNNBwDu2LNnj+rLEwAAWgk9wVQ/RIF6yOSovhLdw+5r8MgBd/1g
BQSxlmzNd8B9AQCsDLuvkTcVz7+nBrKh+g0bb373dWPu0ENvVkAQiwwY3BcAwNKw+8pdeF/Jsf8L
2VCDRt0C9wVZS3BfAACrA/dlc8F9QZYT3BcAwOrAfdlccF+Q5QT3BQCwOnBfNhfcF2Q5wX0BAKwO
3JfNBfcFWU5wXwAAqwP3ZXPBfUGWE9wXAMDqwH3ZXHBfkOUE9wUAsDpwXzYX3BdkOcF9AQCsDtyX
zQX3BVlOcF8AAKsD92VzwX1BlhPcFwDA6sB92VxwX5DlBPcFALA6cF82F9wXZDnBfQEArA7cl80F
9wVZTnBfwKK0aUJ1KYApgPuyueC+IMsJ7st42rjQrl272NjY/Pz8119/XXXpvGL79u2enI9m19q2
bdutW7ebb775Zz/7mX/LAPcFBHBfNhfcl/GSn/PKC9NSPXR6s/Lyw30Zj6tt+Oqrr15++eW+ffuS
V6murlZVMO8ZMWKEvvsSk9988w09eQYPHhwSEvLjH//Yj2WA+wICuC+bC+7LeCl3L74I7sueeLIN
58+fp/np6elizq9+9atp06bFxMRER0ePGTPm3Llzmq/s378/Li4uIiIiKyuLvt69e3daw1//+ldP
G5Ln6K/8v/7rv9asWRMfHx8ZGZmUlHTvvff+4x//oPl/+MMf5Is2JyfHm727cOECzUxLS+PJv/zl
LzSZmJhIZejfv394eDjPr6urmzBhApUnKioqOzv71VdflVdy8ODB3r17087eeOON77zzjmZD+jtL
vP7666NHj6Y1Jycn7969+7vvvvPmOHsqqpHU19fXNUHOvPIqRUVFBRJ02Ti8QCxfUVFBK6EV0mob
GxuN3yn/AvdlcwWT+9I2jUiEtA2JbB/evXeXQTnpc9ZN3V1zl0IDo9y9+CK4L3viyX3xi57sAU/+
7ne/69Chw9ChQ3//+99/9NFH5K/o1U82Rix/8uRJWn7s2LFffvnl+++/n5eX17ZtW5ojfIWOIWl2
5fSCpiWffPLJP//5z6tWraL/77nnnmZ3wdNH33zzDc0U1uXvf/87TZJXXLJkCRWA7eLFixfJ6WVm
ZtIc8njk68LCwn75y1/yV06dOkVfGTdu3KeffvrBBx/QzrbIfb377rtk2+bPn//1119v2bKF5u/b
t8+b4+y2qP6FnVVtbS3bqtLS0hYZKr9A26It0tbJ6fl9Bw0A7svmson70kBObNG2maoMjHL34ovg
vuyJJ+vy2muv0fxRo0bx5PLly2lShEudPn2aJufOnSuWnz59Os05c+YMT/7kJz/x3pA0u/KYmBia
Q66J/v/kk09ocsKECc3ugqePfvGLX9DMjIwMebGQkJATJ06IOWRvaOaLL74oF2nBggU8OWPGDJp8
+eWXefKnP/1pi9zXsmXL6H8qBv3/xRdfkBMbOXIkf9TsoXAtaiuoqqqSnZX35iotLS2/icLCwj1X
qampefMqLfVL4ou8KlozOV55i+Q/qZymfZG5Be7LQir+/u/lF59f1mlP98Ws2XubEvOg3L34Irgv
e+LqE77++msyUcnJyVFRUWfPnuWZiYmJtNilS5d48k9/+hNNpqSkiG/16tWL5nz55Zc8+fnnn3tv
SJpdeVJSEs3Ztm0bG7Bmd8HTR3/961/ffvvtoUOHkoEhf6hZTBSe6NOnD8354x//yJP0D00mJCTI
O0vllD/1cmfF7oj9lWn2ULgWtRWQyXFrrnJzc9lclZeXsx1qnafyHdoolYHKI9swp9NpiXZJuC8L
Ce6r1a5m/9ltO06ULto284ZuHcQyKUMSlJgH5e7FF8F92ZM27oiIiFiyZMnFixfFYuHh4a6LRUZG
igVCQ0NpDodjEd9++633hqTZlb/66qtdunRp09QSOmvWrNOnT7tdjzd7d8MNN0yePPmVV15xXUwU
XhRJszuisTIsLKzVO0vQXtD///znP10L3OyhcC1qK5BrqwgzW5qGhoaqqiphw9LT00tLS2mm6nLp
0Tr35fZO1EG5b/FGKZkzTF5auC/fXU3poaXXHlbtw10X2HXqrpmrc1OH9Ynp1L5daNuIqHBHcvcJ
80be++xaT+t85Ny2hVtnZIxJ79SjY2h4O1ptbGK3cXNGbPu31S0t54bHisLCQ8WnS8pnyZ9Wvrxl
zrqp/W5M7tg1JjSsHZWwT/9es9bkkSNy3coDL2wQ6+mblcQz73h44aCcpnKGtaOVDM8bsOnxFW4L
ueFw0YipGZ173kB71KFzdNqwPgs233Lg9XvgvuwJn255zubNm0NCQt544w15JvmxNlIIvSsa9/W3
v/1Ns2bXDYk5za6cuHTp0uOPPy4irPbu3auzZm8+0l+MXRDZKp7kgCsqp7yzwj7xp17uLBEdHU3/
0yFyLUmzh8LLPQo+6JVRUlIiqsLM7MHgvlhrnH8Ii4g2eWnhvnx3X2RgxDJkKjSf3rFvYfuOUW6v
4fDIsBUP3Oq6wq1PFXeL6+z2KyFtQ6YsHXPwjXIvy3nPj1bLWycTKH9acmBxhy7RbjdEbmqjc5lm
K3tfLhMLkEmjYozJz3RbyEV3z9B8d8qSnJAQNxtKyui940Spp/LDfQUxfLrlOV9//XXPnj0HDRpE
pkLMTE1NpcV03nf0FVrgv//7v3lS9EYUC4Q0XXkiCJ/b1HiBZlcu8/zzz9PCdJ3o7II3H+kvxo2D
mpZHmsmTPXr0kHf2008/9X5nxf66bT1s9lB4uUfBCh0Zej6kpaWZ2YPBfbFuKfk385cW7st390Ue
RiwzMDtN/qjimTvCIkLbeCY0vF35M2vkr5T/eE1kezdNADKTFmV7U877f7q+c88bxPxxs7Pkr2w4
XERb19kK2bbvVd8pf+WRc9vEp7GJ3easm+Jxv8La0b6LL85ZN1VnQ4Ny0t2WH+4ruOHTrZl59OhR
mvnQQw+JOatXr27T1OuQJ997773evXvTTLHA1KlXri4RJ1ZVVaVZMzcdCj/z4osvigWaXXl+fn7X
rl0/+eSTy1c7Y9LaxKcaq9Ps3nmzGAfGv/DCCzxZW1tLk0VFRTw5ZcqVm05ko33uuee831li5cqV
ba6G1n/++efh4eHDhw/nj5o9FF7uUXDT2Ngoe7CKigpTNZ767r6UexK/qH/ObebfI7ivVruvAz8v
f/DFjct2zBFxX23btd1y7HZ5mRunDRZf7z8ypaLqjodfu5uW6ZXSQ8zPnjlM/kp8X4f8lbInb3/4
1bvve65kauHYkLb/qjyiR/7mJ1bol3PPmS3yVoaM70cFlnzUPXL1GpVh27+t3vuzsq1PFQ8Z10/M
H3S9mTz4Rrn4KPqGqMjoCPJgd+5ftO/VrQ+8sOGm5ePkYkyYN5K/tbvmLlpSzO8zoFfpoaX0FSrh
7bvm9Yjv2uZ64L5sAp9uzUxyMpmZmdHR0aJi4cMPP+zUqVN8fPxvf/vbzz77bOLEifStRx55RHyl
urq6TVPGLU7CMG7cuHbt2slr5l6Emzdv/vOf//wf//EftAYOf/Jm5WVlV+p7165dS9br8ccfbyMZ
ISIhIYHmvP322/SpN3vnzUGgXWjfvn1WVtbHH39MB2HUqFFUWvJC/OkzzzxDy0+ePJm8E82cMGEC
R4J5s7OXm1JmkeMiC/fVV19t2rSJ5u/atcvL4+zlHtkB2YOlp6c7nU7VJfoXBruvFY+8J74YP2A8
zVn16IdDJt3evmOPtu1Cozv1lBde+8Qfp615Iu3Ggo7d+oRFRIeGt+/QtXfS0Km5RfvvcH7iZuX7
39WsnDRr43H6SkznXm1Dw2n9aTfmz6t42dO+aNCsn4o6buGDCQMntr+hB60tMqZrj6Rh2XO/R6bI
LyURunVbTd/Rc2ln24VGRHXsHtcvZ2Lhvjuf+ALuy3v3pU8XR6c7Hl6o+Xra8ESaT0YlNKzd9569
Vo+09pFr1WXdenUW88nJiPnknfaf3SavLXfBKPHpiKkZOu6LvkibFnNSBseThZOXX/q9fPHp4HF9
NcVOHZogPtVEmskb6tg1Ztep67KcUanEp3GpPXnm/E03i5lde3Uijyd/5YGTG2I6tdeUH+7LDvDp
dp1PdyjNnzlzppjz61//mibJG5DTGDhw4LFjxzRfefDBB2NjY7kmh7yQZs1ffPHFnDlzunTpEhMT
Q77l/fffj4uLa3M1/El/5d9++y0ZMHJZERER9Hf9+vXkasSnzz//fO/evUNDQ+XmSP2982ax8+fP
k61q30Rubu4777yj2VnaHKeWfeutt7jhVYRy6e8scebMmWHDhtHXk5OTaVVyFL3+ofByj+wDeWMR
D5aXl2eGFGEGuy8yMOKLPZMz73z889iUEW7XtmTX+W4Jg9p4oGO3hFu31WpX/ti1lZMvWnv0y0ET
ily/GxLSNm/5gZa6r/zNz5NFdLsYGaq595z2vSSszFvWtXEXc+NIvbFo7688FQ/uy3v3lZwRv2bv
bQdev8fLd/2+V7aK74aGtxPzc2YOF/Nv23KL5lsVVXdwYDwZtn4jkj2V8+Ab5ZmTBorJ2MRuD9Vu
0qwqa/K1G6HkwGLNp6srF4hPpxdP9LShOeumaL5YcnCJ+FR0QJAr027dMM31aNxy+wR5tXBfwEf4
QlJdCmAL6IUi+kVWVlaqLYzB7uuOI5+KL3aN6z9mwU7Nm5EXW/rQL6M6dNO+Na8nNLy9pu7ojiOf
iU+79Oo79rYHPH23bWj44gff8d593bqtpl1ohM6SkdGdyS76WBLSuIUP6mwlaeh1ATlwX61zX4wj
qfvWp4r1V/jIuXsefvVuOVC/jeQ35LbC8h+v8aaEruWcOH+k+D88MmzHiVLX5bvHdxHLaOqvDjXV
R4lPM8ake9qQJipM80Xy+//aVu9r23LbZ3PL0ZVujwbcF2gdfCGpLgWwEU6nkxsis7KyFFaCGey+
1h79UnwxpnMvEnmwWXc9t/pww+rDHy1+4G1ezJF27ZXUJa7fjHXPrHr0Q1qmYPNPeiQOFR91jk29
8/HP3a48MqZLeFQHcj608jU/+HjF/ndvnHVdT/mhk1d5uUe0iRt6JIlPB45bsuj+N6m0C+47mzz8
FjE/ccgUH0uy8uBvaEkxv2fS8NllJ+krqx79P7eU/LBTbEqb64H70nE1mo8OvnEl7qv00NJhE/uL
Zdp3jNK4HfIbZIfi+zqiOkS67fQnr1nun7jvla2tc18aVtzvplslFUbnKzI94rt62pCmYfRQUyyc
637JnQj2vlzmWpg9LyHjBPAnfCGpLgWwFw0NDSKXrKpIMOOj7uXvRneKvf3gbzULkMUSC8R0idNE
Va167MOO3RPFAjevfdrzynuuPPCB/Gnf0XPFp93iB3m5R1NuPyw+Sh52s+bTXumjxafkynwpyYQl
lWJmx259yODJX1m+/9ea+kC4L+/dl6wJ8655+zH5mWL+9OKJIlReB7F823ZtxUw5SN4X99Wpe4fK
l7dolpc3pE/7DpEtOiCuC7Rtd+0IuN2pR87d0+xq4b6A9/CFpLoUwI6w/yGKioqM7w5pfMYJef6Y
+dtdVz5w3LVwlLEL7nddYPyi3WKBftnzPa187G0P6Pi6sMgYL91X+qg54qP8zc9rPp2xvkp8Onr2
Pb6URK5JG79ol+uOjyq4W6eccF/ywdFZ7L7nSsRiXRydeObKB+a2cQf5sXah15kfsZ6omGuN0W6r
ibwpJ6188uIcOQP/+FtHaJZv73XdF/k0H91XRJRU9/UzNzv1UO2mZlcL9wUAsAT0iuFWyLy8PIMN
mFr3dduO111X3rX3ALHAwh0/d12AZooFuvTq62nlciAWa/n+X1/7OCTES/fVqWey+EhThaVZZ9LQ
ab6URG7fXLjzDdcdn3/vqzrlhPuSD47OYvvPXkuEFRr2ryh6uedgfN/YOx5euOvUXRyZL6dukNfc
I+Fa+oW7f7iqde6LY6uK7pt97XJoG6LJUNGzz7U6T7dp7Vt9QFwX6NbrWmoLtzu18QfLvDzOcF8A
APNDpotD8dPT040MA1PrvlY99qHryiPaX8s5qWl9Y8kdJ2lhTyu/48inmi+ufeKP3pRKpzz6dIpN
8aUkYZEx+kem+Pv/qVNOuC/54Ogsdu+za8ViMZ3a80y5ImvH8+vk5ct/vMbtmuWMDXM3arsHVjxz
pc9j9A1RXRydUof1abacfbOuee/e6bFyl0w5EdldR5YH1H3JyVRvXT/V9SuTF+d4eZzhvgAAloAM
GOejMNKAqY37Wnv0S9cF2ra7lm+cXIrrAnc+8YVYgBZuUcE8LaDzRbk8+kREd/KlJCFtryUzd7/j
j3/uy5GH+2LlLbwWqpd21ReRUxIz5ZwPB98o7z/yuv4OIhnXigduFTN7xHfd9+p1gfdyZ8ZhE/s3
W87yZ9bIZZhdMll8tHzntbZvTeqwQ03Z+8Ojwrr16pw0qPfYgqxmN6S/gJzovmuvTpogtO89e6cm
tz/cFwAgODDYgKl1X24XIA8jFlj1qJsqoFWP/h+xQGR050C7L7k8+shWsBUlCYu8Noqf20q/2w/9
rtnVwn0xrp/ueWnz5idWjLplqLzY4ntm8qdy6x4t88ALG8hlbThcxC2S4ZFh4lOayV858Po9cu73
tOGJm59Y+fBrV3LdT1qULbpM0j/iK/rlnLJ0jJgfERUu+mPShuREECNvGnz3D1ftP7ttd+2movtm
y+lPNbVV+gfE7QK04/JwS30G9Fr3/Su57h86vZm21al7hzbXA/cFAAgajDRgJnRf3eKvJZ9ccO9r
rgvI4U/dErztuthq99XZkSY+ck1r3+rddF1A7svpNuDt1m21vhx5W7kvb+h3Y7IYAnvasnGeFusW
13nh1ulisn2HyD79e/G3NjqXhYU3UzWad9toL13ivle2dnF0Eh9ljL2W1v4u53JN8L8rKUMSHjl3
jzcb0l/g5hXjdbYyesYwedL7pLVwXwAA88MGLCsrK9BB+CZ0X3Ja+OxbK1wXoJnX3lC5y/3uvjTt
of2y54uP5pWfCZz7kpOpjlv4oOtXMm9e58uRh/sShIS0GXnzEHlAn70/K5MHbRR069W5/MdrHqrd
pOl4KL647tFC1xohpm27kJuWjxMGzxtTVLx7nvzp7Q/OFR+VHlrSoXN0Gw8MGd/PNVWFzoZ0Fjjw
83I5pE2mb1YSWUR5vO8WdfaE+wIAmJ/CwkJ6whQUFAR0KyZ0X7PLTooFojv1vP3Q7+RPVx74oP0N
13KMz7n7lN/d1/L9v5Y/uumOo+KjftnzNCvM3/x8WER0x+6JsSkjZCvYipLIie47duujaXVduvsX
clh+K468zd1X23Zto2+IShwYl7tglNss9+Qrblk5vldKj7Dw0MjoCPpn+u0TRAzYXc7l/UemkBmj
+aNuGaL54q3rp/Ybkdyxa0xoWDv+Lm1FHizSe1Mkx7136tFRtjdkEeesmyI2FB4V1j2+C9nIdd9f
2ooN6S+wfMcc2lBMp/a0oRu6daB9X1I+izOAde55rR/KAyc3wH0BAIIJ0QsyoA8ZE7qvkispTK+N
UNwtfmD+phOc8n3Gumc6O669m+L65bR05Z4WCA2/FjwzOG/lqsc+XLH/Xf7ozie+kHNB9MtZsHDH
z+848untB387ddWRyJhrkT9yhVUrSkJbDA2/lj69Kdf9C7Tjxd//PW0ourO2ZgbuC7Kn4L4AAAGl
vr6eE7E2NDQEaBPmdF+Fe/5Xs+M8tu/Yo6jyf/vLfZHVcd2E+HRu+Utt24W5LiBDjlEe9qh1JRmZ
X6aziYHjFsuTZAvhviAbCu4LABBo2B0Frv3RnO6rpKmhrXuC++ATokfikKUPXWjFyj0tMGnF9123
Ii9QsOWnUR27eypPSuZ0TUNh60qy9ok/9sue18Yd8QPGrz7cIA/27TYnGNwXFPSC+wIAGEBmZiY9
aqqqqgKxctO6L7Yi01Y/kTpiZoeu8aHhUVfCq7r1SR85+6Y7jrlNFOaL+yJNXLq3c2xq23a0qQ6d
HWkDxy/VLLD68Edjb3sgfuCE9jf0aBsaTuXp1DO5f85ts8te8G9Jpq15grYS1aEbbSW6U2yfjDwy
h5wBrEPX3uJbmvg0uC/IJoL7AgAYQE1NTeCqv1rnvqCgEdwXZDnBfQEAjIGrvwKR/gvuy+aC+4Is
J7gvAIAxOJ1OetqUl5f7fc1wXzYX3BdkOcF9AQCMoaGhgZ42eXl5fl8z3JfNBfcFWU5wXwAAw0hL
S6MHjt9T38N92VxwX5DlBPcFADAMTn3v97ck3JfNBfcFWU5wXwAAw2Cb5Pe8E3BfNhfcF2Q5wX0B
AAyDbZLfHzhwXzYX3BdkOcF9AQAMA3VfUCAE9wVZTnBfAADDQNwXFAjBfUGWE9wXAMAw8vLyAjHc
NtyXzQX3BVlOcF8AAGPgfF+ZmZl+XzPcl80F9wVZTnBfAABjYI9UUlISoDXDfdlWcF+Q5QT3BQAw
hqysrEA0O16G+7K94L4gywnuCwBgAFVVVfSoyc/PD8TK4b5sLrgvyHKC+wJeUncVp9NZWVlZXl6+
YMGCXIlx48ZluYPmi2XoK6WlpZs2baKViBWq3jMQcBobG9PT0wP3foT7srngviDLCe4LyDQ0NJAd
qq6uJn/F5opemrHX4/Af8mppQ2TVaKPkzagAVIxANFEBJZBXp9NdWFgYoPXDfdlccF+Q5QT3ZWca
Gxu5Lmvx4sXjxo3z3mL1GzQ8e+JMobnLy5asfUBfedMXy1+hNXhpzMiSzZ49+95776Wi+n10ZmAA
/OpJS0sL3OmD+7K54L4gywnuy1aw3aqsrMzNze3bt6+O0UpMTiOPdMu8NWScSrc/dd9jr5AOPPf+
j37+F/9q91PneeXFZYdoW7RF2i5t3ZMlS01NzcnJ2bRpE8yYJaBzxMH2ZPIDtxW4L5sL7guynOC+
gp6Ghobq6up169YNHz7ck90SRotcENkhv1us1okKU1Z5gi3Z8JET3JqxgQMHLl68+OjRo2imNCcF
BQWOwGSZkIH7srngviDLCe4rKGlsbKytrV2+fHn//v1dHRfXa3Gllnm8lpcVZVTmucvLqPxuq8Vm
z559/Phx1ImZBA73ys3NDfQZgfuyueC+IMsJ7iuYqK+vr6ysHDFihKvj6jdo+C3z1pB1CUTroVoz
RvuliSKjfR82bFhxcTEdENXnxL5wiom0tDQDzgLcl80F9wVZTnBfQQBXc/Xp00fjuIaPnDB3eVmQ
OS5Pon10dWJ0QHr37j1jxozjx4+rPkv2ghwXp5ioqakxYHNwXzYX3BdkOcF9WRc2XQkJCRrTlTd9
sU0cl44TKy47RMdBPixxcXFTp059+umnVZ+34EdYr6qqKmO2CPdlc8F9QZYT3JfloBvZtaaLGxbL
Kk8odz6m0pHaT7hCTO5ESTZsxowZyPIaIERi1UBH2svAfdlccF+Q5QT3ZRXopbZ//36OopdN19zl
ZdaKnFclsqZ0rOR2yaSkpPXr1yNE34/QwczLyzPYel2G+7K94L4gywnuy/zU1tbOmDFDNl2JyWm3
zFtz32OvKLc0VhTZMLk2jA5sZmbm4cOHVZ9nyyOslwGdHDXAfdlccF+Q5QT3ZVro/bVjxw5NZRea
F/0lbpSUM1fEx8cXFhaiKqx1KLRel+G+bC+4L8hygvsyIQ0NDcuXL5ezRvQbNLy47BAZBuWmJfjE
IfqiRZKOObmIt956S/VVYCXUWq/LcF+2F9wXZDnBfZkKukl5vEW5AyNaGI1RWeUJuSosMzPzBz/4
georwgIot16X4b5sL7gvyHKC+zIJ1dXVPBKQqOxasvYBO2eNUCU65nJUWHJy8saNG1VfHebFDNbr
MtyX7QX3BVlOcF/KId8lB3eR7yrd/pRyE2JzHan9RG6OjIuLgwdzxSTW6zLcl+0F9wVZTnBfCjl6
9Gh6errwXdkTZyKi3mwiJwwP5hbzWK/LcF+2F9wXZDnBfSmBbka5vot8F4K7zCzZg8XHx+/cuVP1
FaSYhoYG81ivy3BfthfcF2Q5wX0ZDN2GOTk58F1WFJ0pEZbfp08f28bki4GE8vPzzWC9LsN92V5w
X5DlBPdlGA0NDXLSVPgui4rO2vCRE/gkZmRknDlzRvWVZSjCehmczV4fuC+bC+4LspzgvgygsbFx
06ZNcXFxIq4e8V1Wl9wWmZeXd/HiRdVXmRFUVVWZ0HpdhvuyveC+IMsJ7ivQHD16NDU1Ff0Zg09H
aj9ZsvaBxKQrJ7dXr7iioiKTNMMFCLJefBmb8HEB92VzwX1BlhPcV+Cor68XIV70jqY3NZLVB5/o
nN4yb40IBvvhD3+o+roLCJWVlbyP5MFUl8UNcF82F9wXZDnBfQWI8vJyEeKVN30x8qYGt+j8ioD8
MWPGBFklWGlpKe1XWlqaOa3XZbgv2wvuC7Kc4L78zunTpzMyMkRTI0Lr7aPS7U9xkvy4uLitW7eq
vhL9ANnIoqIitl719fWqi+MRuC+bC+4LspzgvvwIvapWrFghqryWrH1AuR+ADJbcEDls2LC6ujrV
V2XrEflUMzMzzWy9LsN92V5wX5DlBPflL06fPi2i67MnzkRTo50lZ6UoLS1VfW22BrJbWVlZ5smn
qg/cl80F9wVZTnBffmHNmjUiur647JDytz9kBi1Z+wAbsIEDBzY0NKi+SFtATU0NZ5YoLCw0v/W6
DPdle8F9QZYT3JeP1NfXiygvVHlBGu1+6jxXgsXFxd13332qr1avcDqdfD2Xl5erLou3wH3ZXHBf
kOUE9+UL9957L+dQRZUXpKO5y8vYz0yYMMHkVUncvdG0mSU8Afdlc8F9QZYT3FfroHfopEmT+D01
fOSE3U+dV/6Kh8ys+x57hXPjx8fHHz9+XPX16wa6pAsKCrh7o2nfYp6A+7K54L4gywnuqxXU1dUl
JyejYyPUIsndIdesWaP6Kr6O+vp67t6Ym5tr8u6NboH7srms7r7auNC2XUjHrjGDx/Vd9/2lyn1C
s7p5xXgqs/4y9/xo9ejpQ7v26hQa3i6mU/uUwfFLymcd+Hm58sKrEtxXS6msrPxXgH1yGnJ5QS3V
lZxgTYMT5eTkmKQVkl4NHGOfn59vkiK1FLgvmys43Jc8Z8+ZLXfuX9QjoWtI25Bl22crtwr66tO/
l777KrpvdmhYu559uhU/NL/y5S2k+Ztvjmwfnjlp4CPn7lFefiWC+2oRM2bMEK2NCLCHWicRih8f
H6+8okmM3mi2gbNbBNyXzRV87utfL+gjy2l+j/iuYs7Wp4r7j0yJiAoPjwpLGRxfekhbMza7dEqn
7h1Cw9sl9HPQ12M6tac1PPzq3W63opmjv/LdtZvGFGR27tExLDy0a69ONy0fd+D1e3Y8v06uskvO
iHfdi/Ifr6HydOrRcXfNXfL84t3z6Cvz7rpJzNlwuChtWB/aelhEaFJG75IDi8VH+17ZSgt3ib3h
nh+t7puVRIWMviFqWO6AB1/cqFM8T4dXzBGrXf9YoSOpO5WzV0oPOg5UNvKKVAzao3ufXavZI08H
au/Pyq6szdGJFohN7EaG0y/uq6Ghoa45LPrD2Rto9/v168fvqbnLy5S/wSFLS26F3L17t6qruqKi
gsvgdDpVlcEvwH3ZXMHqvvhtTh6AJ79XfWdk+/DeaT2/9+yd2/+9hPwVvd83P7FS42dShiSQJyHP
Qy4lpG0IzTn4Rnmz7qvZlQ8Z148WXrRt5t6Xy3JmDaf/py4do19+VvbMYRqXJTRgVOr04on8/90/
XEXGKb6vgwpApo5sT7vQtpufWMGfkpWilYRHhqUO60MzK1/ekrtgFM0ZNrF/K4on5ojVjpiasevU
XeRdaZJsGK2NvCJPDhydKn9X50A9cm4bLU+Ol9ZGn7Lp9SSHUeTl5RVcpaioqLKJ6upqS9g2KqHo
21hWeUL5uxsKDhWXHeK7Y/bs2QbfAvIQQjU1NUZuOhDAfdlcweq+Sg4uofmJA+N4ctQtQ2nyzv2L
eHLN3ttk+0EalJ1Gc+54eCFP3v7gXHnN+u6r2ZVHRIXTnH2vbKX/d/5kHU2mDevjjfvqFteZPiU3
omNFSORYaLFVD82XC5A5aaCmtOQwefKBFzbQZPuOUa0onuth2ehcRv+T4+LJu44sF5PtO0TK39U/
UPR/SEibFQ/cqr+zXPHlFnom5/tAbm5uKxwaGbPS0lJyZabyYyLQC30bIb+LrigeGjIzM9Owa14M
IWTy0Ru9B+7L5go+91X58hYyUd16dQ6LCC09tIRndnF0osX2vLSZJ9kbkLcR37qhWweaIxrjHji5
wXv31ezKu/a6ssCUpWPY4XjjHllh4aH06f6z2/QNSZfYG+TC0z802bnnDZqtiObLg2+Us9VpRfFc
DwsdcLFOYu/LZa6b8OZA8dfFXjRrwIyJ+2poaHjzKlVVVXuaKCwsJLdGrx631WUVFRW1tbUKnVhJ
SQkXJnvizCO1nyh/WVtFS0seFNp++FVZystmNtF1xWFgiYmJBngh2gTH2FtiCCEvgfuyuYLDfWkI
DW83YmrG3T9cJRYLDWvnuhh5G7FA23ZtaY6Id3rk3D3eu69mV15yYHH7jlFtmlpCM8b2XbP3Nm/d
V8QV96XfDCcKoCm8HDqlX/4WFU/nsOhPNnugeFLshUncV7PQe6GmpoYtGf0ql51YUVFRdXW1wS8L
EWN/y7w1yt/R1lKz9ZxJKek5ubNIc5dvJYdWtvd5O9crijCwhISEgLYD0q8etl70syJorNdluC/b
Kzjclzwnb+HokJA26x8rvO69H95O38Zo3Nf+s9ta4L6aWzlpz0ubbyub3jcrib+Yf+ckb9xXzz7d
6FPZRuq4L9EFkmOoqFRemqgWFc8n96V7oPQPhWndlwYyY06nk5yYxobV1tYGetP0YhLjB5Vuf0r5
29lyatZ9eWL4qImTZiwhP7Zux9N282MiJf7Ro0cDcVVXVlby+s15v/sC3JfNFXzuq/LlLR26RDuS
u5MJETO79+5Ci20/UeppPfQVWuCh2k08KTok8mTIlQD8f0XgH5JinLxcuayVD1yJKOvYNcYbyzH+
1hH06fTbJ7h+NK1obO6CUfw/Nx1qWh5ppvfuy1Px9He8pe5L/0AFh/sSkBei3+yyDcvKyiJjFqAf
77Ra7t6YmJSKjF6tU6vdlwauIuPmS+U7ZYDI6vOOb9myxb9XtUWHEPISuC+bK/jcF2nh1hk0c9aa
PDFnTH5mm6ZufTx5z49Wd+rRkWaKBfqPTKEFRJxY0X2z5TVzw5ywN6semi9/2uzKB4/rG31D1M6f
rDt0tTOmiHjX2BuN7ju+NjwyrH2HSI1jKXvy9rCI0GnLxvHkqFuG0EqKr4+6H3nzEC/dl07x9He8
pe5L/0AFmfsSsA2TQ/rpneLfO+7ixYucxz4xOc1udS9+VHHZoSVrH9Bo7vKy7IkzSTzmTisYPmri
3OVby/Y+r3wHAycy/JyOdcWKFa27hrkfsZgMvhh7V+C+bK6gdF9kZuL7OsKjwoRpue+5kqgOkZ17
dKyouuP+n65PG55I35qzbor4yrLtV+xWckY8fVr+zJrUoQlt24WINXOnwryFo/e+XLbl2O30dY7I
8nLlkxZl05xxc0aQt7mtbLpsjTr3vBIwv9G5jD5yu4O3Pzg3LDyUFlu2Y86elzbvrt1UeG9Bhy7R
w3IHiMo9KjCZtIR+jh0nSmmXEwfGUfHI23jpvnSKp7/jLXVf+gcqWN2XgF4iJSUlIjasoKDA0313
6dIl71crMksgmaphToPEVq1FriwpJX3SjCXrdjwdlKdJdIRcsmRJK+4O8lrp6en0O4LvlKysLEdT
jH1DQ0Mr1mYJ4L5srqB0X6QNjxXR/Iwx6WLOtn9bTZP09m8X2taR1H3R3TM0X5mxamLHrjGhYe3i
+8aSHZLX/MALG4ZO6N++Y1REVHi/EcnlP17TqfuVPpKiN6L+yh85dw85HHJQoeHt6O+EeSO5Y+Ch
ppa+Tj06tm3XVjT2uep71XfmzBzOIw1xktKi+2ZrRhq668jyK9lWI8NI6ZmJdzmX6x8leY5O8fR3
vKXuS/9ABb37YuhHvdPpFP0l6aVTXV2tWYazinmzNtl6oXujQpEfK93+FPsxLyvElpY8GGQ2TBiw
KVOmtKiFna52PiwjRox46aWXrD6EkJfAfdlcVndfgZP3ZgAyWJZ2X4KqqirhweiXvrBbly5d6tu3
L81s1oCdOHEC1suc4sqxvOmL2Y3o2zBaMmhOn8hE4f2IkOKC5wx1/NfSQwh5CdyXzQX35UlwX6ZV
cLgvRg4JIw9GF7zo5EXo9JQk6/Wv2rPpi4Pm3R2UOvDc++zE9G0YN0oqL63vaqkB4ws+9ir0/8yZ
M/15j5kVuC+bC+7Lk+C+TKtgcl8MXedcD0Zvn969e4saABEJo0FYLyT1spDIlpRVnqBTphMqlpSS
Pnf5Vqu3SHpvwD7++GPecVHxxf8cOXIkUDebaYD7srngvjwJ7su0Cj73xVRVVXFMvtwE42rA6urq
YL2srt1PnZ+7vEynXXLSjCWWzhzipQHjcRv5Upfdl8OLlnerA/dlc8F9QZZTsLovTT2A+Kdv376i
F6QIs4f1Cg5xbZgnDzZ81ETrNkc2a8D4d0SsBO81/eIoLS2VE1AEJXBfNhfcF2Q5Bav74tySmpcR
/500aRIZsIsXL4owe+XvVsiPIqNSuv0pT/0lk1LSl5Y8aMXoPmHAxo0b52rAOKOXTEFBQXV1dYsy
rlgXuC+bC+4LspyC0n2Jii9PjB07tk+fPg70cAxq7X7qvKeqsP4ZmVasBxMGbPXq1fIFL7JMOJr6
mxw5coRuAVV3nxLgvmwuuC/IcgpK93W5qet9XV1dbW1tZWVlaWlpQUEB55xkuBIM1ssOOvDc+0vW
PuA2KsyKHoyuWO5oIAwYZ5lIT0+vqKhw26/EDsB92VxwX5DlFKzuyxP0eho1ahS7rySMJWQnlW5/
ym0HSct5MJGIlX5WXG6K+DJg3HmTA/dlc8F9QZaT3dwXx4ONHj167ty5jqYooLLKE8rfp5Bh8uTB
cnJnWciKT5qxhMocHx+v+n4yC3BfNhfcF2Q52cp98SM6OTm5urr6woULxcXF/OalN7Ly9ylkpDx5
sLnLt5q8MZqKN33+HaIB3SZ3brPAfdlccF+Q5WQf91VVVUV7mpKSwtaL2bVrFwyYbeU2HiwpJd20
DZFXou5HTeRfEJWVlXQx0/90Yau+t9QD92VzwX1BlpNN3Fd9fT2/W8luXbgeYcDmLt+q/PUKGSzy
M+TBLNEQKazX6NGjz549K1+6dHmrvsMUA/dlc8F9QZaTHdxXQ0NDeno67WZ5efkFdxw8eDA5+Uo1
wvT5dyh/yULG68Bz77vND1Zcdkh52VhkBZNS0mXrxdAl7WhKqUoXuer7TCVwXzYX3BdkOQW9+2ps
bORElAsXLnRrvZjq6moYMJurrPKEazBYTu4s5SNFCutF17BsvRiaSR/RRa76VlMJ3JfNBfcFWU5B
775EJ0cd6yUZsGRH05A0Jg+9hgIktw2R5HwUVoKVbn+Ki+Hp5wP5Mbq8HVcTUNgTuC+bC+4LspyC
231xpD15qlOnTjXrvghajF9kMGB21u6nznNKeZlJM5YYf0kI6+Wp0Vxct/zDwbYR+HBfNhfcF2Q5
BbH7EpH2x44d88Z6aWoSyIApb3KCFMptJZiRofhzl2/l7bp2FXGFLnI7R+DDfdlccF+Q5RSs7qux
sVE/0t4bA2bw2xYym+577BXXSDBjWiE5qVdycsrBgwe9vG43bNjgaBrn0XUM7qAH7svmgvuCLKdg
dV8FBQW0X7NmzWqp9RJwMDMMmM11pPaTucvLNAaMrFHgWiFpzTm5sxxSWmDvoQuevkgXv+r7z2jg
vmwuuC/IcgpK9+V0OmmnMjIyXDuItc6AYTQim4suAE1e1gA1TMtJvVpqvS40VdvSZU9fp1tA9V1o
KHBfNhfcF2Q5BZ/7EuFerXh5uYLRiCCWayi+3235lU1cn0+1FXAAWHp6uq0CwOC+bC64L8hyCj73
xdm9NmzY4Lv1YjAaEcQ6UvtJ3vTFmlZIf10VIqnXpEmTfKyz5QAwW2UAg/uyueC+IMspyNwXp/6m
95e/rBeD0YggoeKyQxoDtrTkQR/XWVZ5QuRT9csVy91G6HZQfUcaBNyXzQX3BVlOweS++ApPSUnx
MrtXi8BoRJDQfY+9ogkD8+WqEEm9iouL/XW50i3AY3Cb9n3kX+C+bC64L8hyChr3JVJMeJMcqXVg
NCJIaPdT5zXJKFqXjlXUpPn9uuUKW7op7JCAAu7L5oL7giynoHFf3OboS4oJb8BoRJDQlf6J18fh
t/Sq4KRegfvJwAko7ND+CPdlc8F9QZZTcLgv0eboY7iyN2A0IkiILoDsiTNbYcBoGZFP1S+dc91C
t4NN2h/hvmwuuC/IcgoC99XY2JiVlRXQNkfXlxpGI4KEbpm3pkUGTCT1akU+1ZbCdcJ0g6i+TQML
3JfNBfcFWU5B4L74wev3fo76YDQiSJb3BkzOp2pAVS1Bt4bV7/FmgfuyueC+IMvJ6u6Lc6sGqJ9j
s2A0IkjIGwMmknoZZr0uSP0fgzj/KtyXzQX3BVlOVndffs+t2lIwGhEkpG/AhPWia8Yw68UEff5V
uC+bC+4Lspws7b6qqqq4GsHIF5krGI0IEtIYsJzcWTxfJPXyVz7VlsIN5XTLqL5rAwLcl80F9wVZ
TtZ1XyLB17Fjx5S8zmQwGhEkpDFg0+ffIaxXeXm5qktUjP8YlOm/4L5sLrgvyHKyrvvi562qmgRX
MBoRJKQxYLGxsQ4D++R6gtN/WfFmbxa4L5sL7guynCzqvhoaGhQG23vCbqMR7X7q/PbDr5KWljwo
ROYzJ3cWi/6XP+KFbdJDQTMe94oVK1RfntfC7+n2UX0H+xm4L5sL7guynCzqvoqKitQG23siWEcj
OlL7CRmn4rJDtF9kqzh63Bf6Z2SyPaN10pqV76Df9cjxdxMSk+VdNkMTOYff0+2j+g72M3BfNhfc
F2Q5WdF98ZWckZFhcMcxLwma0YgOPPf+uh1Pk90ip+TWQeU3UVhYuEeipqbmzetxOp38UXl5OX8l
LS3NdW10uGhbtMUgSGDLSb1iY2Pj4uLEDqakBDCtvZfQLUM3jpnfU60D7svmgvuCLCcrui+DM9u3
AuuORkSlLdv7vKvjIr9EronsE10q9CTxS9MVrYesGq3Q1Y/R1qkMVBLlB6QV2v3UeZFP9dixY2zF
mcGDByv/ycABikGWfQLuy+aC+4IsJ8u5L84yYXBm+1ZgrdGIyHSt2/H0pBlLNI6rpKSEDrgxYUK0
FdoWbTEz85rxS0pJ5wox5YfIe+vFzbJ0ibLR4s6GcoWh6mvzX9nvgyn7BNyXzQX3BVlOlnNfXPFl
hhCaZrHEaERc06WxB06nU21edHJiVIbCwkKNDbvvsVeUHzG9g1l5QuRTla8E0R+WUZh3gmFDGEyD
P8J92VxwX5DlZC33ZZWKLxlzjkZ04Ln3l5Y8KDcvktWpqalRfYa1NDY20kmXbRiVubjskAnbc0VS
r+LiYk+XgUB5AFiQVX/BfdlccF+Q5WQt92Whii8ZU41GRL5LruzKzMx0Op3mT0FANozKmZuby8VO
TE6jvTBPky4ZQi6Yp3BEURHKpKSkqA0AC7LqL7gvmwvuC7KcLOS+rFjxJTDDaEQa31VSUmLah5UO
VGYqudgLM3gwcVT1e4KIzrDM5MmTDbv83BJM1V9wXzYX3BdkOVnIfVm04kugcDQi2XelpaXR6bb6
cDMNDQ20FympqbxTc5dvVdIWSRvlA5uc7FU2CVMFgAVT9Rfcl80F9wVZTlZxX5au+BIYPxpR8Pku
GdoXfu1yW+TSkgeN9GCc1KvJeiV7H8clB4ApH6whaKq/4L5sLrgvyHKyivuyesWXwMjRiMiNBKvv
kmloaBBtkezBjLReo0ePbmkElxwAprb9MWiqv+C+bC64L8hysoT7qqmpCYKKL4EBoxHd99groj9j
EPsuGdmDkS8KaA9TkdSrFdaLLwDztD9y9ZdpX1teAvdlc8F9QZaTJdxXQUFBsyHN1iJwoxHR2uYu
38qv9fz8fPN3ZvQv9JQbO3Ys736AKsGE9Vq4cGGr+y3yeItmaH/k1nC6xVSfOp+A+7K54L4gy8n8
7qu+vt7RNKqjqtdTgAjEaESiyistLc3pdKo+dcoQwWB+rwQTSb00+VRbgXnaH3nkR7X5dX0E7svm
gvuCLCfzuy9uTgqmii+BH0cjEp3v7Fnl5Qp5Cb9Xggnr5Ze2QvO0P9KmHU0ZSFSftNYD92VzwX1B
lpPJ3Re5CEdTnzLlYxMHCL+MRiSGdU5LSzNhvnqFyJVgPlYwivZcP/4Q0LQ/qrrIabvcDm5d0w73
ZXPBfUGWk8ndF/8qp5eUkreSYfgyGlFZ5YnE5LQrr57cXDtE17eU+vr6kSNH+uhvRVKvgwcP+vfU
c6ufv1ozWw37QLrdVJ+uVgL3ZXPBfUGWk8ndV3r6lfBmtTmRjKF1oxGJnBKWbjYKNGRKV61aRUcp
MSm1paluj9R+kpM7y9HCpF7ewzkfBKrGf6RbjLZOt5vqc9VK4L5sLrgvyHIys/viDKuzZs1S8j4y
HtEO5Y1DIFcwacYSXj4IsmUagGiF9D7VrZzUK3C+iHM+KA+/pxvNutcS3JfNBfcFWU5mdl+caMLv
bT1mxsvRiA48974I9LJ0VzWDqampiY+Pp+OWkzur2TAwEU3XuqRe3sP1TgJVFzxtl7ZeWFio+iy1
BrgvmwvuC7KcTOu+ON4++BJNNEuzoxGJfFP5+fkI9GopZFbJsjYbhy8O8qRJkwwIhpfD7wcPHqwq
/N66sfdwXzYX3BdkOZnWffHjtLi4WMlrSC3Hjh3zlAxfuAIEerUasqzDhw/XMWBllSdEPlVjzrjo
dcio6mZCt5s5nwbNAvdlc8F9QZaTad0XD+xoh3h7t7gdjQjWy1/oGDCR1Mtg5y/qPB3qsk9wG6gV
h32E+7K54L4gy8mc7ivIBnZsHZrRiGC9/ItbA1Zcdoj9j5LsvnL2CVXVX9wFwHJZ4+C+bC64L8hy
Mqf7CuL89i1CToYP6+V3NAZMDBag6sIzQ/UXl8Fylxncl80F9wVZTuZ0X5zmS2F++2effVbVpjUc
PXq0V69e/Ea23DvR/JABGzp0qKMp05qjKZ+qqoxbjFz9pWTsIbrpHBZM/AX3ZXPBfUGWkwndFzc7
KkzzdejQofz8fFVblyEnkJKSEhsbO3LkSOvmITc5ZMD69evHhufRRx9Ve8bl6q/BgwcrKQMn/rJW
4yPcl80F9wVZTiZ0X2qbHZ999tnU1NSNGzcq2brM2bNns7OzUeVlAGTApk6dSi6XDrjyEUXl6i8l
d4EVGx/hvmwuuC/IcjKh+1LY7MjWi7ZuBve1aNEiR1NSL9UnxC7QoaYDTodd7XlXXv1lxcZHuC+b
C+4LspzM5r4UNjueO3cuJyeH33rK3Re/gtPS0pBP1TDoUHMiVuXdPeTqLyWp7y3X+Aj3ZXPBfUGW
k9ncV2FhoZLXn2y9lLuv6upqroIz7cMkWOGHJB18tbH35eXl4lJUMvKj5Rof4b5sLrgvyHIym/tS
1exI7ziHhEL3JcK9zHNSbAW/x9UGgGlS3xufc9hyjY9wXzYX3BdkOZnKffElanyzI0dYmcR9IdxL
OWYIAFu4cKG4GpWMt8WNj6Z9nWmA+7K54L4gy8lU7osbXAxOc7Rq1SqHC6rc18GDBxHupRwRAKYk
5oqprq4WV6OSzKviZlR9NrwC7svmgvuCLCdTua+8vDyD21l2797tar1UuS96w9J71szPEPvAT0tV
CecZHvSHMT4Sku0f3ZKqT4VXwH3ZXHBfkOVkHvfV0NBAJcnIyDDs/eLJehFPPvmkYcUQFBcX06YL
CwtVnwpwBe4AoqTVj5FTT2RnZxtfAO56STem6lPRPHBfNhfcF2Q5mcd9VVVVGfmyE6m9mNgmFLov
rmpAm6N5INfB7Y8K+z+qjb3n2DO6MVWfiuaB+7K54L4gy8k87ourGoyJtHFrvdS6L+506XQ6VZ8H
cA06HQ5FOR8Yrg5ljB/2kaMQLVEZC/dlc8F9QZaTedwX55ow4J1y7tw5t7VeCt0XtzHl5uaqPglA
C50UJWFXjBx7b3zjo4XyTsB92VxwX5DlZBL3VV9fT8WYNGlSoF8omqyqZnBfCLY3M8rD7+W898Y3
PnLkP92eqs9DM8B92VxwX5DlZBL3xU08BgR9abKqunow493Xhg0bHJbKK243eNh3Ok2GXRIycuOj
8V0AeOvmbxCH+7K54L4gy8kk7otfcAYEfZ07d2737t2cTF4Hw9yXqPiyRM8ye8K9cVVVf8mNj8YP
us2hX+b/aQD3ZXPBfUGWk0ncV1ZWlsPYAYaeffbZefPmyfVdStwXKr4sgdrqL7nx0eAOmBz6Rben
6jPQDHBfNhfcF2Q5mcF9GZ/pi6Hf9a69HZlz584ZUABUfFkFtdVfans+WiLrF9yXzQX3BVlOZnBf
NTU1VIaFCxca/FqRX2qrV69etGgReyGHIV0vL6Diy1IorP46duyYuFAN6JmigbN+0U2q+gzoAfdl
c8F9QZaTGdwXjyhnfKf+wYMHi5ca9yYTUWEGbB0VX9ZCVH8ZcG24IqddNXjTnA7F5AM+wn3ZXHBf
kOVkBveVn59vfEAL2S3xOjO+0fPC1ZcaKr4sBFd/Kcn9NWvWLHG5Gjz2N4f9002q+vDrAfdlc8F9
QZaTGdyXwl/0jPGNnheu1ryZP5MSEPAj1PiOhwTXDzPG553g7ao+/HrAfdlccF+Q5aTcfXGDjvHR
LAorEy5crU9AcnvLwanvjR/5Ua6qNd7+jR492uRN5HBfNhfcF2Q5KXdfqkLuRYC9w9hMFwxHMps/
iSXQYFhaYFcUJr3ny9W0r7bLcF+2F9wXZDkpd1/82DS4K5mcwZJ+1xu5aYa9X2Njo6rDDloHnTJV
sfdyF12DY8+4c67yrIA6wH3ZXHBfkOWk3H0VFhZSAY4dO2b824QxPocA5w+nHVd1zIEv8BVrfGu1
HKlocOUbp7ww8xUL92VzwX1BlpNy92V8lvsLV8cOZgw2fheuhpxVVVWpOubAF+jE0emjk2jwZSNX
2BocJ8lRZ3l5eaqPvUfgvmwuuC/IclLuvmjrycnJRr5KLlztw6Vk0zx0S1pamqoDDnyHTp+ScEGF
Wb9406oPvEfgvmwuuC/IclLrvviaNPiHvJw53PgaDKT5CgJUJf6S62wN7nfJmzbt2w3uy+aC+4Is
JzO4L4M7PMqpk4wP+rLEuC1AH1UddeV4RYO9n8m7PcJ92VxwX5DlpNZ9KenwKPcdMz52mpOsorej
peGej8bn3eL+GozB3s/k3R7hvmwuuC/IcjKD+zL4V7zC5hskWQ0aOO2qwXm35JyrBrfXc4s53Bdk
TsF9QZaTWvfFIzwa3OtQzrNq5HYvXG30NPmAxcAbxKk0+BJySBi5XY6WNO1oj3BfNhfcF2Q5mcF9
GVmBwF0OGePzrHKuCQR9BQEc+mV8rw255tbITpdc7Qb3BZlTcF+Q5aTWfan6Ca8kcuYCUtwHEaqS
3svjkxpcacwbVX3g3QP3ZXPBfUGWk3L3lZGRYeQbRGGWew76Mm3tAWgpSkbcVtjtkQeaVH3U3QP3
ZXPBfUGWk0L3pSTZl/zyMrjDI0cKmTZuGbQUJX1G5PGGDP75YOaUX3BfNhfcF2Q52c19KWy4YeOH
AYaCBh5yyGALpDBXMNwXZFrBfUGWk3L3ZfB4wXLQssHpAsz88gKtQMnPB7nbiMGbNnPCVbgvmwvu
C7KcFLovp9NpfNUBJzs1Ptr/AvKsBh0ceJ+dnW3whSQuYINj/rnylm5b1QfeDXBfNhfcF2Q5KXRf
ShLdizeXkqG9Mbh2kKHExqsaa9vM6e7hvmwuuC/IclLuvozMV6mw1QYdHoMSTlinZMBr492XmbuN
wH3ZXHBfkOWk0H2VlJQYHPouRywb7L54hD7aZeOPMwgcxl/DF9QNlcW3jzlHaoD7srngviDLSaH7
Mn6YIdl9GRztb+ZWG9BqlLSey+7L+NvHnPW3cF82F9wXZDnZ1n0Z/MZk94UxhoIMHm9IybUE9yUD
92VzwX1BlhPclzEUFxeb+eEAWoeSrCmq0t3DfUGmFdwXZDnZyn1x2LAS94VkX0GJ8vEajLyM4b4g
0wruC7KclLsvw94dF5QO8gj3FZTYyn1xl2G4L8iEgvuCLCfbui+Dh+eD+wpKbOW+LjTlrIP7gkwo
uC/IclLovvLy8hS6LyVZApDoPsjgdPcGuy+FA23DfUHmFNwXZDkpdF/8+jDy3aHQffEwQ8YfZBBo
HIYPNqSw8whvVPUhdwPcl80F9wVZTnBfxmDa1xbwEeMvY7gvV+C+bC64L8hygvsyBtO+toCPwH2Z
AbgvmwvuC7Kc4L6MgUdGNv4gg0BDpzUjI8PIawnuyxW4L5sL7guynBS6r8zMTPu4L466N/4gg0Dj
MDzqXq37ottW9SF3A9yXzQX3BVlOts04UV5ebuSmkXEiKFGScQJ9Hl2B+7K54L4gy8m27gvZVoHv
IN+XSYD7srngviDLSbn7qq6uNuzdAfcF/Iut3BdGGoJMK7gvyHJS7r5sMs7jwoULzfxwAK2DH6p0
co28llSFL8J9QaYV3BdkOdnKfSkMV+Y3pmkfDqB18ENVybUE9yUD92VzwX1BlpNt3desWbMM2+6F
q29MJccZBA5+6cN9KQfuy+aC+4IsJ4Xuq7CwUKH7MjhW5+DBg46mjpbGH2cQOLgtm06ukdcSxxCq
cl8lJSWqj7ob4L5sLrgvyHJS6L74gblr1y7DXh8XrqaLNN59mbneALQa4+tvL1zvvs6ePWvYdjnT
hTnrb+G+bC64L8hyUu6+lCTrJgYPHmzkdnnT6enpxh9nEDjohDqMzZpCpKSkiMvYyO2aufUc7svm
gvuCLCfbui/jX5oZGRkOpLsPLhyGDzN0QbqGDd40uy+n06n6qLsB7svmgvuCLCeF7osvyOLiYiPf
IHKrjZHbvYCUX0GHkmRfZ8+eVdV6TreqaS9guC+bC+4LspyUuy+D3yCqIpYvXH15VVVVGX+oQSCg
U2n8zweF/XbN/PMB7svmgvuCLCe7uS+5t77BXdW4f5w5w2ZAK+A3vsEDhirMWQf3BZlWcF+Q5aTQ
fTU2NtKms7OzjXyDKEx3j26PQYaSDo8KR8uiW5U2Sret6gPvBrgvmwvuC7KcFLqvy01BywbHX8lV
BwYPEHPhary0kkMN/I6S6EEeskpJ07mZr164L5sL7guynNS6r9zcXIexOYtOnTqlKmj5grnbbkCL
4MepwZFXF64PXKSL2bDtcrQ/3bCqD7x74L5sLrgvyHJS676UtN2Il1dKSoqR272A0K8gQknQ1wV1
KVNM3m4O92VzwX1BlpNa96VkoJbRo0eL95eR1W5EdXW1mV9hwHv4hwOdUCOvH4U1t2ZOdH8Z7sv2
gvuCLCe17ktJwlWFkTMXriYqV3K0gR+hk5icnGzwxcOjhSqJWjRzovvLcF+2F9wXZDmpdV81NTUO
wzMmKez2SMyaNcvMTwngDaqCvuQOjwY3evJvFtNet3BfNhfcF2Q5qXVfSlJ+KcxXeeGq96O/Sg44
8AviJBp88bB1V1Jta/IOI3BfNhfcF2Q5qXVfl5tacJQMeK0q8J5Dd7KyslQdcOA7PLi2kV0OGXl8
bYNDFukmNXOLOdyXzQX3BVlOyt1XZmam8a8SOfDe+HcoVyPU1NSoOubAF7i53PhaU+6ywdAFbOSm
Od0E3aqqj71H4L5sLrgvyHJS7r6UJJ2QA+8N7nF54Wr3sZKSElXHHPgCnTg6fXQSDb5s5HhFJYNL
mrmvLtyXzQX3BVlOyt2XkrxJ7H+UvMguNNUkpKSkpKenm3PQFqADD49Fp8/g2toLSgeIN3+eOrgv
mwvuC7KclLsvJd0e5UYcgweaZLjyraqqStVhB62DTplDxRhVF9TlWb1w9XI1c1s53JfNBfcFWU7K
3Vd9fb1DxaA/GRkZqgKYjx07Nnny5NjYWNOO2wI8UVBQoKS1Ws70pWp4I7pVVR9+j8B92VxwX5Dl
pNx9XTbBaMXGxPCcOnWqvLyc+44xZMAaGhoUHnnQIuhk0Vkj327A1aKhuLhYXDaqhjdSffj1gPuy
ueC+IMvJDO6Lx9o2uO+hHPoV6IakQ4cOcecCZuCQ/ut3rqa/9P/8+fMVHnnQIjje3vgMvUR2dra4
fpQMb2TmkPvLcF+2F9wXZDmZwX0p6UTGneiZACUco9fWqlWrRI6mpJTE2UtnHD5ZWft+Nal4SyGq
vywEV3wZFm+/aNGiJ598UlxI4lo1vubNEl104b5sLrgvyHIyg/viSGbj+x7KWb/8WJ9Ab+fdu3dP
njxZrHzclJxtD284fv4o+y4WTZIfo0/XrVun8OADLzG44ouvnCFDhlRUVNx7772G1dO6wo2eJu8h
Avdlc8F9QZaTGdwX1yoY3/fQ77E0ZOEWLVokKrsGDum/+M55T7/2qGy6ZHH1V3x8PFJPmBw6QZzf
3rAOGo7riW3C+CriC1cbPU1eQwv3ZXPBfUGWkxncF5GWlmbkq42RB3z0xftRsTXh9DfNmbTTebcn
0yVExoyX3759u9rjD/Thl7th9U5PPvmkwx1kwLZs2XLu3DljinHhagM93Z6qz0AzwH3ZXHBfkOVk
EvdVWHilFsjgHJIXrs870Yqwf7fh9DqVXa6avXQGqr9Mjqj4MqxjiCf3JVi8eLGICgso/AuFbk/V
J6EZfHFfbVwIaduu/Q09kofdPLvsBeW+AvJGcF+Q5WQS9+V0Oh0qepO1rvGRw+lFZReH0+97Zqf3
pkuu/uLor7vuukvtKQCeMLji68L17osbHN1Czj/QJaFbkjZEt6fqk9AMvrsvec6qR//PrLue6xyb
GhLSdurqx5VbC6hZwX1BlpNJ3BdfnMbnXG1p0ntNOP2InOGu4fQtFTo/mhnR1dHIjCga9yWQrRcV
qaamJtAl4Tyrpn2pCfzrvljzys/Q/E6xKfLMBfed7ZORFxYZHRYR7UgbObvspPjo9oO/zZi4LKZL
XLuwyI7d+tw4a/OdT3zBH60+3ECriozuPPee0937DG4bGh7dKTZ77vc0W7x1W01cvxxac2h4lCP1
xvzNz8tf79A1ftH9dfEDxtPWI2O6pI2YtfLA+81uutliB4fgviDLySTu67KinKsXrm989NTzURNO
n5SSqB9O3yKReePcX0h9b0I4ub3BtbKudV8a90WX4rPPPmtASXhzqk9C8wTCfa0+/BHNJyMk5izZ
dT4sMqZbwqClu39RuOc/eiQOJR81/3uv8KfJw2+h5SctP7jqsQ8HTSik/7Omb+CPyAtdadAMadu7
/9jFD76z8sAHvfuNoTnTVj8hVr5wx8/JO/VIHEIrX7b3/yOP1LZd2LyKn4mvh4a3j+ubTXNWPfrh
0ClraE7qiJnNbrrZYgeH4L4gy8k87ktV6Jfc+KjJesHh9HKiSy/D6VuqbQ9v4DesmYfSsyE8CKnx
Kbbc1n3JFV/GWC+rBH1dDoz7Ktj8E5ofm5Il5gwYu5Dm5G/6d56cuaFatkBhkdE0ufpwA/2/bF89
Tcb1y9Fs5aY7jsnfTRiUKxbolz2P5kxf92N5gbQbC+Sv31Lybzy5Yv+7XJnmzab1ix0cgvuCLCfz
uC8O/TI+65fc+CjSrh46dGjRokVifivC6VuqcVNyaEMJCQkIvzcJItje+F8Eru5Ltl67d+82phj8
w8T8QV+X/R/39eGsjcc7dk8MDY8q2PJTMb9jtwRasvj7/8mTKw/+hiZv6JF09dM+VyqdblnPLsjt
Vpbtu/gv+/TIe9yYKBag/2kOzb9+gd7y12mLPLn26JdN3QNCvNm0frGDQ3BfkOVkHvelKuvXhesb
H8l0ybkjWh1O31IdPlnJW1y9erXqUwGuUF5e7lAxpPUF3T6PhlmvCxbJ9MX4t88j0S40ol/2vIU7
fi4v2TY03M2SYZH8af7m5yOjO3NjZfKwm2ZuqHbdyh1HPuPJO458yo2JmpWLeK07H/+cJmmm/HXX
FXqzaf1iB4fgviDLyTzui8jMzHQYPuAjwe9ZUdXg8FM4fUslwu/r6+tVnwq7Q6fAYXiwvaCiosKt
9aKfBoaVgYc3oltS9anwCv/WfQ2ftrZNSMicrS9qliRLRkuucf7B06qKv/+fuUX7e/cfx+scM3+7
ZivCXNFKaDIsMlrrvh7/vORf9uwzNoHeuC/9TTdb7CAQ3BdkOZnKfbELMjKbN4fTJyUlye+4B4+W
G2m6ZJHrczQFGqH9USF08PPy8hyKBtQmNm7cqNZ6Xbg6vCPdkqrPhlf4veWxfcceXeP6i6oq1g09
kmjJosr/3ew6b177FC0Z3amnZiuiYXH5/l/TZMdufcQC3HqoaXkUCzTrvnQ27X2xrSu4L8hyMpX7
4iBnA9p6eChGOZx+QEY/8X/xlkJV7uvwyUpO/7VgwQLVZ8O+lJaWOlTkPxG4ui/jW+TpNqTtWqUb
iN+j7vOWHaCZmqQQgyYUtWnqWsiTi+6vi+nci2byZPKwmyNjuizbV19ytb+kiIoXW5le+iOenLG+
iiaThk4VC/Qfc5vrAv1zbvNUSHmO/qb1ix0cgvuCLCdTua/GxkZu7gncO+XYsWOacHryWk+/9qgI
uyKSU5NUuS/S+p2ruf3xiSeeUH1C7AiP+U4XocHjXslo3BdZLyNHF2I4v4rqs+Etfndfa49+2SNx
SFhEtFxlVLjnf0W0vyGmS9ySXeeXP/xrzhox9rYH+NPMm0tpcnDeSvI/uUX7Ze/EWwlp2y5h4ET6
7sqDv+mZnCl7LdLiB94ODW/fI3Fo0d5f0UZjU7JCw6PIKXnjvvQ3rV/s4BDcF2Q5mcp9Xb6ad+Lg
wYP+fZWcOnWK3mj64fTc6sdse3iDQgPG/R979+6NADCDoQOuqp+jjOy+yAUZb73oBnQozTVRW1tb
KVFXV6ffFh+IjBNz7j7VVD01TZ65cOcbNIfMTNt2YV3i+uUtPyA+uvPxz8kFdejau11oBP0dOnnV
qsc+1LivW7fV9kgaRgtEd3a4+p955Wfi+uWQByP17j92bvlLOoWU5+hvWr/YwSG4L8hyMpv74poH
P47qsnv3bnkoRrJY63eudhtOz0m3mBvHZCp0XyL/KgLAjER5uJdAuC/DsqpqoBuQtk43o/FngbwW
G2BX6Ow4nU63fTDNP8q2J48H+UVwX5DlZDb35a/Gx+rq6lWrVsnZ6WcvnXH4ZKW+7eGYK8aYRBOe
JALAZsyYofqc2AXl4V4Cdl+qrNeFq82OBjt/slXsfrnajR5K9M4iB0j/lJSUpKWliXuzqKhI856F
+7K54L4gy8ls7uuyb42PruH046bkeN+MyDkfmPFTxyh0X7VSXRwygBkA97dNTk5WGO4lYPdlZGov
GSXNjqLiMT8/31OGsfr6etmGZWVl0Y8s/gjuy+aC+4IsJxO6r9Y1PnI4vajsEuH0LW3yM0/1V62U
AQwR+AFFRNp7GujTYOhKVmW9LihqduTxNMlcNbsk+TQqG+cGZA/mdDq3b98O92VnwX1BlpMJ3VeL
Gh9PnTpVXl6uCaf3ZShGU1V/kW6aMwkGLKCw9Wp1dWsgOHTokMKtG9/syKlmyFC1aKN04uSQTrpH
zOy+oIAK7guynEzovi571/hIbygvw+ktXf1F5eHOmPRyOX36tOozE2zwU9FhbI5fM6Ok2TErK6vV
r076VklJSZ8+feC+7Cy4L8hyMqf70ml8PHXqVCvC6Vsks1V/CQMWFxdXV1en+uQEDyK/BKyXgJOs
GtnsyBVf9EvKl5XQEwzuy86C+4IsJ3O6L4Jja0UItKdw+kAMxWi26i8YsEAgrJcf05tYHbrL6IDQ
rWfkiSgpKfHd75k/6h4KqOC+IMvJtO6Ln8nl5eWu4fSL75zX0nD6lkqu/ho4ZIBy9wUD5l/oYQjr
5Qp3/DR4bEc+ET6GmcF92VxwX5DlZFr3VV9fHxsb26tXL+GCbpozyZdw+pZaHbn6a/3O1crdV+31
MWDf//73VZ8iqyLC7GG9NHDvFSNHWOC3ko/Njpf94b4W3V83YOyijt36tAuNiOrQzZE2cvLKR9c+
8UflvgLyRnBfkOVkWvdFiMouv4TTt1Q83iKTnJpkfAGaNWCVlZWqT5H1ENYLsV4ajh07dsXA5OYa
eTrYNfn+/PHRfU1ddaRtaHhnR9r00h+tevRD0sSle8MiY9JHzr7z8c+VWwuoWcF9QZaTCd3Xxx9/
/Oyzz9I/OTlXhjtUOOQiD/fDzF46Q7n1EqLCcKm8yY8EBBUVFbBenlCS5ot7N/v+0vTFfS164K12
oRExnXutPPgbef4tJf/Wpk2bCUv2yDNv3VYT1y8nLCI6NDzKkXpj/ubnef7qww20cIeu8Yvur4sf
MD4sMjoypkvaiFkrD7zPC9x+8LcZE5fFdIlrFxbZsVufG2dtvvOJL8RqPQ3jKFY7Z+uLXeL6UTm7
xQ9ccN9ZKht5xStlSBu59KFfavaIFuiTkUdloHLSArPLTl4t5Ee0to7dEmiBzo50MpzKXRPcF2Rb
mcR9Xbx48ciRI8uWLRsxYgSVh/6nmcePH3coHXJx3zM7HRL+7VnpLwM2depUjAXZLHSIeCChlJQU
8+T1Mg8i3t7ga8kvQV+XfXNfA8ctcXVZLPIwo2ffIyYX7vg5eaceiUOW7v7Fsr3/Hxmbtu3C5lX8
jD4iK0UrCQ1vH9c3m+asevTDoVPW0JzUETP5u8nDb6HJScsPrnrsw0ETCun/rOkbmnVfYrV9R89d
eeCDsQvup0myYbQ28oo82WfwJPmLS3adD4uM6ZYwiApZuOc/eiQOJZc1/3uv0Ed3HPmMlo/q0K1f
9jz6dI3zD8pdE9wXZFspdF91dXV79+6dPXt23759ZZNTVFQklklOTnYo7XU4bkqOKJgZsk/IEkMR
0UsTcfg61NfX8yg2dDmZJJu92eB4e+OrUh1NSVZ9X48v7uuGHknkSciNNLskmRZacvq6H/PkzA3V
NJl2Y4Hsl24p+TeeXLH/XZqMjO7Mk2GR0VyXRf8v21dPk3H9cpp1X+Kfufecpv/JcfHkvPIzYjIi
upP8xQFjF9LM/E3/LhdSmMArXw4JufnOJ5X7JbgvyOYy3n3R648cl+N6YptwNI0bcunSJbHwfffd
p7bV7+nXHpXLaZLwe6HDJyu5eTQuLg5hYG6pqqriCpZJkyaZYQxHc8Lx9p4GWAwQ/gq5v+yb+2oX
FkmW5I4jnza7ZIeu8bTkikfe+5e/euS9pmbB3rJfEs2Xa49+yVaHJzt263OlvuuW9WzANGrWfa16
9MNr66TJxz503cTVDSXQvOLv/ydPskMjh3ldIa+2hwaNLOG+AHDFSPd18eJF3mjsVfh/nqmpw2ls
bCRfQfMDnWJCR3L2ieTUJIUlcavj54+KCjq0Qspwa6O4xhDr5Qk6Mg4/uaAWwT0g/JLgwhf3FRoe
RYbEm2a4tqHh3BrIk3c+/jlNiugpHQdFyt/8fGR056ZmxKjkYTfN3FDdIvflaUnXL3IhNZDDlJeX
Q86CQyZ3X/S7Jh8Ad9TU1Bh5KVZWVsqOS7wf3dbezJ8/nz4iC6TQ3sjh92Zrf2QJi5icnHzmzBkj
z6Y54dZGcY0xaHZ0CycxNv7N5a8Oj5d9c1+dHWlkSBbu+Lm37utqL0gOo2oXGuGN+yIVf/8/c4v2
9+4/juePmb89EO6LyqNjJl2XDw6Z3H0BYBIaGxu5MUiu+CooKHC7MP1q4Eonhd5GE35vtvZH1uGT
lZyMgliwYIFtK8Fox8nGC0vvaMoUt/jOefTP4MGD0fioQUmiCYYzKvvljemL+xqct5IMyajZ21w/
GjHjrqFT1ohJbj3UtDzSzGYdlEY3r32K5kd36nltyZAQmrP26Jc8KeK7XFfSrPviMLaiyv/tdmfh
vgCwLTU1NWS9NJUSNEcO99KgPPUEiV/fpm1/FBKVYH369Nm3b5+RZ9YM0NWVlZUlrq6klESRoZfb
Z/Pz81X7HXMxefJkh+GJJhg6F2ZwX0sfuhAa3j4iupPGsSy477XQ8KgbZ24Sc/qPuY2sy/TSH/Hk
jPVVNNk/5zZPxkaekzzs5siYLsv21ZdczfwgAvJJ3CgpfN30dT9utfsaNKGoTVPnSp5cdH9dTOde
NBPuCwDb0tjYWFRUxK/FwsLCnTuvVSjpd9njkEW1I/5Yov2RJULxiYyMjLfeesuwU6yQhoaGgoIC
ucpr9tIZco5cMX5BeXm5astjFk6dOuXwU6/DVsAdUf1SSetjttWb1z7dLiyyQ9fe01Y/Ufz9/7z9
4G+nFP+gfcceaSNm3XHkM7HY4gfeJp/WI3Fo0d5fkVWLTckie0b2xhv3lXlzKf0/OG8lWa/cov2y
bSu52pty+LS1qx77cMG9r/XuN4aj0Vrhvgr3/K+I9jfEdIlbsuv88od/TauiBcbe9gDcFwD2hKu8
HE3pEUSYGT9+vemsRy8I5dVflmh/FKLiJSU3DZbU60qr7kcffRTgM6wM16ZGMp9us5SIM4gAMIYz
rKpK98fnwi+r8n2kIfIqA8cv5ZGGOEnp1FVHXEcamld+Jq5fDnkwUu/+Y+eWv6RjhOQ5dz7+ORkw
Mni0fvo7dPIq7rfIWrH/3ZTMGZHRncMio+MHTlj0wFvRnR1tmnpittR9kRbufCNp6DTyYG3bhXWJ
65e3/ID+8kEguC8AXOFKCVHlJf/UvXjxoqdwLw0vvfSS8uqv2uvbHx1KE5F5o+Pnj8oFDj4Pxr4r
NTVVbmos3lKoMywUDyCFALALVyu+jM+wKvBjtRtG2ba54L4A0OB0OrnKix6zPvasNEP1lxhmUQSA
mWT8Rx09/dqjcs7Y4PBgrfBdQjfNmUTLT548WbX9UYzaiq/6+nqH/9JcwH3ZXHBfAAjkKq/y8nLf
f1+bpPrr8MlKDh8yfwCYpthicCJHryupRy0aD0bXVUVFRVJSUit8F0uE8G3YsEG1A1KG8oovP6Za
vQz3ZXvBfQHAyFVefrwjzFD9VSsN8cMozEXWUj392qPXPJjDMWDAgI0bN/rr7ASa6urqefPmyfFd
LfVdshflNRw7dky1D1IDd3VUOMAruy9/jW0E92VzwX0B4PcqLxmTVH/VWi0AzNWDkWmRu3COGzfu
xIkTfjxTfqS+vr60tDQxMVHOUjIiZzh5YF+afTkALCUlxYYBYJzjS1VXR4Z+oPnR/sF92VxwX8Dm
BKjKS4Zzf5mhukkTAHb4ZKXyIrVUO513cxDUvxoke/UaO3bs4cOHA3HiWgqZrnXr1vXv319T2UW+
11/J1rga0IYBYApzfAn8mOj+MtyX7QX3ZQyXLl3STw8FjKehoYHTRwSiykuGg3XNEO5OHkAOABs4
ZIDyIrV6R9bvXC2bSWLIkCFFRUUGV4jRySUDP3PmzISEBE0+XnKJfm9xFn0obBUAZoaKr8tX/RKd
bj+uDe7LtoL7MgayXiNGjNBJjQ4Mhn5Ec5VXbm6uAdf/9OnTaVuL75yn3LdoMoDdOCbTogasVrJh
cm0YM2DAgIKCAnpR+j1Qn0w73c6bNm2aPHmyq+MaOKT/7KUzdjrvDtxRFX0oDh48qNoWGcTgwYOV
V3wR9BvNj69LuC+bC+7LGKqrqx3epegEgUZOX0+PU2M2Sq/s3r3jaItmGO5HE4F/062TlRfJd5Hb
Ic9D/lZTIfavZtbk5IyMDPJjGzdudDZBDopOis75qmvipZdeuv/++0tLS3NycmgNcpOizLgpOWQC
DWvJ5TOYkpJy6tQp1c4o4LDn8VdPQ1/w4zBDl+G+bC+4L2Mg38VP6Y8//lh1WWxNfX19VlaWo6nf
usGXPY/PaxKrI4ZWZOYUzlReJP9q3zM7aR9vmjPJrRnTEOtCs1+h1c5eOoMcl6rOCxwAlp2drdoc
BZazZ8+SyTTJSwruC/Kj4L6MQXSpmz17tuqy2BfussS/o41PGURbjI+Pd5ims6GcxsFhgpwYAdXh
k5U7nXeTH2NLNm5KjjeuLCklkZYk0Vfoi+y1zFB7WSsFgBUXF6u2SAGE9s7hvyQPPsLuS6fKtEXA
fdlccF/GwNHd/Ju6trZWdXFsBzmf0tJSfqUqzBe0e/duhzmyT7A0DiS4DVjwKegDwJSnV9XA1eb+
Whvcl80F92UMooHjSpsFwu+NRfRtNL610RVOvmqG7BO1LoMQwYBZTjuddzuaAsCCcgxu5elVNfA9
4q+1wX3ZXHBfBnDx4kVhvfhvRUWF6kLZhfr6etG30V9NBr5QV1fHTVpma8CCAbOoOIludnZ2kKVg
PXjwoMMEWSZk4L4gPwruywD4hSu7L4IsmepyBT8irURJSYlJGi8uXw2/N89gizBgVhefvkWLFql2
TH7DVMH2Av+6QbgvmwvuywC4w6PwXfwPwu8DDVkv5YFebiEfmJycTAXb6bxb+YubpcnCCgNmLYnT
t2vXLtW+yT9wfLtJgu0Zum0dfk18we4L2By4r4DC8d6u/dmrq6tVFy1oEdZLeYZGtxw/ftxhjuz3
QiKEW7B+x2rlpYK8VDAFgHFme/ME2zM8xLYf3RetkPbRuNc8MB+ZmZmmusiDD5FuQpCenk4zjxw5
orpowQlbL3qy1dTUqC6LR6ZNm+YwWaItVwNmquJB+uIcblYPAKPCc2Z7s928fndfAIBAk95EQROx
sbH33HOP6hIFMxUVFWy96uvrVZdFD/rJExd3Jfu9edofa5sM2MAh/TUGzDwVdJC+xk25Mp67pQPA
OMFXYWGh6htUC7svExYMAOAJkd8e92+gEbVeJrdezFNPPeUwWftjrbsgfKuPBWkf0Wni2svy8nLV
Nqo1iDZHM3RP1lBTU+MwXxApAMAbOG4zKytLdUGCE348OiwVxzhp0pXhoU0y/JD8EudaFNmAmSRF
BqQvMYq65QLARD9Hc8ZqcpA83Fegqaurw3h8IBBwmCUC7fyOyOtlzke3J+hKSEy8Ulmxfudq5S9u
jTRDESWnJpmqkRTyJA4AGzx4sLUCwLifo2mbBuC+jKGgoADDwYBA4N9xWgFDHoYHASkvL1ddlhbz
0ksvOcyUf1WWxoARq8qKlJcKalZcdUlPG9WWylt27dpl8i5gcF8GUF1dTQe5srJSdUFAEIJbOBAU
FRWZ+Vdzsyxfvpxb95S/tV217eENGgM2fuoYhIGZXHSCuPeEJQLATp06ZcLcqhr40W3mEgYB/CO6
oKBAdUFAEMKxSdb1CSaED6nZsgO1CCr5gAEDaC8W3zlP+YvbVfue2anJRDFwyIDDJyuVFwzSEZ0g
PlnHjh1Tba+aITs72/wV12i2CDRc8UX07dtXdVlAENLQ0IDAez9CvoXDvcyWHail1NfXmzABhZBr
R8jk1CSkYzW51u9c7WhKwWrmADBOMZGbm6v6FmwGuK+AcunSJTJdjqtjwSDwHgQCBN77EW4OMNWI
JK3mscce4wAw01YruYaBjZ86xoThapDQTXOudKqdPHmyapPlHg73skSKGHZfJkyFERyI8fjYfWEg
GBAI8BvKX4iKr6B5JC5evNhh7vxa2x7eoGmFTE5NwqCQppUIANuwYYNqq6WF3rAc7mWJimsOSVJd
iuBEU/FFVFRUqC4UCEIQeO8vgqniS8ABYGYe4ufwyUpNK6SjKWWZaR2jzWXOADAxopBVnoR8DFWX
IjgRFV/iLwLvQSBA4L2/yMvLoyNp/jaLFiFGIDJ5hdLiO+dpDNiVSrD9G5UXDHIVd1w1VQAYNwFY
aNhEuK8A8fHHH/OxFRVfONQgQCDw3i/wYczMzFRdEP/DGcCIfc/sVP7i1hEVTzMoJEeCmTZuzc7i
mL3s7GzVtusKGzZscJg7u5cGHqbEQl7RQpSWloqILwHNuXjxouqigSAEgfe+43Q6g6/ZUcBV8WaO
wGcdP3/UtRKMWLJ2PhoiTSXRa7W4uFit9bJQpL2Ah+iF+/I7ly5d4thdVxB4DwIBAu99p7y83GGR
eN3WMX36dEdTZi3z25idzrtdK8HQEGk2kZPnHhMHDx5UZb1EpL21RgSD+wo0dXV1SUlJsbGxK1as
KCgoIEtWWlqqulAgCEHgve/YwcHm5FwZL8bMXSCFqIQ8vKBrQ6TJ209tJREAdurUKYXWy3KPPg7W
tVyxLQQfYfOnfQNWB4H3vmOH9DuNjY39+vVzmLsLpKynX3uURxiEBzOtVAWAnT17lnPaWzFaAL+X
Aw1dFXSEnU6n6oKAIAeB977Dr3XVpQg4ZMB69eplIQNW66EhEh7MPOIAsEWLFhnpvth6WbTxjt0X
vEHgCLLMjcDMIPDeR2zivi43RURwDor1O1crf3F7KW6I1ORlhQcziZ5+7VE+Nbt27TLGepHT43Yl
iz7xCgsLgz7OQSFodgRGYoewpYBiH/dFnDlzhvfX5EnANNLxYDeOyURMvkLtdN7NAWDV1dXGWC/6
vWndmg08rgMKmh2BkSCQwEds5b4uX81BYTkDVttU0+I6QCQzcMiAVWVF5u9TEJTiVCHZ2dkBTcEq
rJeF8ku4womdVZciOOFcamgJAoaBwHsfsZv7umxlA1ar68EcTVFtJs9sFpQKdABYcXFxEFivy7Z8
2hhGUA4YB8wMAu99hAPnVJfCaIQBs6hXIQ/mqS1SVIXRMsrLaRMFNADMillV3cKVM4hKChA8fLnV
LxJgLRB47wt2yDjhlrvvvhKxQy9N68auHz9/dNvDG9z2i2Q4KgwtkgaIriJHAALA2Ho5giIfMlKt
Bo6qqiocW2A8iOT0BTv3Qpo3b551myA1r363+cEE46eOWb9jNWrDAipOkzt48GB/BYAJ62WthPae
YIeAGN1AUFBQEDTXCbAQyCHjC3z0bHvbrl+/PjgMWO3V5kidqjBulFyydv79R7YpL21Qij0w/R6E
9XIFPaQCBFcqZmZmqi4IsB0IvPcFHmXbzo/EYDJgrMMnK2cvneEpKkyuEFtVVkRODE2T/hIdST7s
5eXlvlivjRs3Bpn1umzvavaAgoovoAoO5kxPT1ddEEvCv5ts3lOGY8CCyYCxdjrv9saGOZoixOYU
zty2f+PDVfcrL7alxQFgRKsDwERyiSCI9ZLhdBM2DDENKKj4AmrJzc3Ffd062Lui06joBbmqrEj5
GzwQlmDxnfP0GyVlBg4ZMH7qmCVr56/fsZr8GALGWqT1O1c7WhsAFhx5vdzCl5bqUgQbqPgCauEc
v7gCWwf9bnKg0+jly4899hi/ICw0FmRLRT6KvMFNcyZ5UyHmClkyEh0f8qgstmeylO+jGURHmA7X
5MmTYb0Y2iMHOuX5G1R8AeVwbxqbN5+1GsRjCE6cOCEMWNBHQx0+WUlObPbSGd7XiXmD8v0yg+ji
4aO6YcMGb3zX2bNnefjsoLRel/GIDgyo+ALK4ZyreXl5qgtiSRB4L3PmzJlevXo5mkKhgt6AyW5h
p/Pu4i2FN82ZxJnb4b58FJlbPiDHjh3Tt16nTp1i65Wbmxus4RPl5eXwCf6FDS0qvoBykHO11SAL
ooaLFy8mJl5pmEtOTbJoMny/mAfhx8ZNyfHekikvuXnEAWApKSk6AWDV1dW0AFuvIH58cch9UFbr
qYKT2wdZ1wxgRbj5DJdi67DneEM60Htw5MiRjivJ8JOCrCOkjyJXtu+ZnSQyZq4in6a8hKaSfgDY
rl272HoFfZMcN6qqLkXwwA0W+MkMzABfjeXl5aoLYkngXd3Cwyg4grQjJGSAjp8/ytWGrgFgIqlX
0Df6c0pGWAV/Qb8N09PTHQjWBeaA+9Qg9Kt1ICbWE025WGMd9ojDhwKhwycruYfpwYMHRYy9MPZ2
CIXioC8MR+Iv+HjicQ3MA0K/Wg13W0DWL7eIjpA3jslE5iuoFdr28AYOADt16lR1dTXH2GdmZtok
DoojlIK1Q4HB8LMaxxOYCjSf+QJnrLXJ66ClfPTRR8nJyRwGttN5t/K3OWQ5zV46g66fjIwMDvTK
z8+3ye9EbpWgx4vqggQJnGUi6FurgbVA6Jcv4OjpQ+9K7rdFLFk7H62QUIskAsBiY2NtdZdxM5mt
djlwcP/0tLQ0m1h3YBUQ+uULaHz0BjEi5I1jMm2bjAJqnTgBha1qmEV8OJrJ/AK34dohVhBYDgya
4wscCYymW33OnDnD2cCSUpLW71it/J0OWUJL1s7niq958+apvoSNg7vzFBYWqi5IMMDNE2jDBeYE
Az76Ano+es9NN93E9Rg33ToZrZCQjg6frLxxTCY3GNELtKCgYMSIEevWrTt9+vSlS5dUX8iBhetq
kBjBd5BlApgc+AdfoBucjh7d46g89IYf/OAHbMAGDhmw75mdyt/ykAm1qqyILxIxitDHH3/Mr1Fm
0qRJe/fufeutt1Rfzv5nz549SPPlLzh8DrWIwLQgeMlHUHnYIj766KPBgwfzaxQJwSBZT7/26Pip
Y9zmU62urna4Y9myZUeOHLl48aKq69mPiLoa+wS5BQ5kmQCWAJkTfIH71MC+tojVq/8VTZ2cinwU
0BWt37E6KSVJJ6lXUVERh4EJZBtWV1dn/GXsX3gH0QzhF/hgIssEMDnIq+wjbF8Re98iLl682L9/
f351jp86BklZbat9z+zkKC9Os+CpEf/SpUty+yPDHqyiosLYi9f/8NBCSIzgF5BlAlgFjCnmIxw7
V1BQoLog1mPHjh2iEgzdIe2m4+ePcsdGjvJqtvq9rq5ONl38d8KECVYPxW9oaGBjiV9wfoHTDCIa
BFgCfqapLoWF4TGbEGPQCj766KORI0fyFYicYPbRTufdZLnppKempnpf8V5RUSHaH/maGTx4sKU7
tYmkxEiv6hf4t3BmZqbqggDgFRhyyEe4sxJiNlrNY489Fh8fL6Lx0RAZxJKj6+nJ06LfLJcuXRID
KHC8Jf9TWVkZuIszoJSWljqQk8p/IGUHsBYYNMdH6Acsqr98hI4hZ691NOVlXVVWhB6RQSbyXWSt
+RQPGzasda/Iixcv8hq4rZ+fXY6mMTss13WIrVdmZiYilPwCV3whigZYCOSd8B1Uf/kFereKhsjk
1KRt+zcq9wyQ7yIjTXY6KSWxqakxhW4WX/xGZWVlenr6xx9/zJNkurjnC820UO8htl70q81yptGc
IL0qsCjIO+EjqP7yIydOnOCD6WhKzYqsFJbW+h2rOcRLv1dji3DN8cV9t4mioiLzVyXBevkd5KoF
FoWfXUiQ4guo/vIvO3bsiIuL41fq+KljkB7fctq2fyOZZz6DdF8E+ofJm2++yQPXpqenmzaKlZwh
Z6OC9fIjyFULrAtdtBw7obogFkZUf+EJ4C/okN5+++0iyhoezCqSfdf06dMNawyiC4b7EJmzEoye
DNxrwFNSWdA68MsXWBr+2YiGM1/ghwByf/kXeocuXbpUeLAbx2QiHsycOn7+6Podq5X4Lpmamhr+
HZSenl5dXW18AdxSVVXF9TO5ublms4WWBhVfwOrwkIUWCls1J2xiTdvwYV00Hoxe8fBg5hHH1Yv4
LlW+S75aRCUY/RpS+6NStDaia3kg4H6viPgC1oWT3qPx0Uf4MKIDaYDgtkgRD0ave+SmUKunX3t0
ydr53J+RWLhwoXk6nYlIMEdTTjAlNU7kDbhmJi0tDT/KAgFyfIEgAL32/ALnrUIXhsBBr9EdO3Yk
JCTwizUpJWlO4UzkyTdY9x/ZJvKmEqtWrTLho4MuFY4H4N9ERjZE0rZEPlh/dfYEGpDjCwQHaHz0
C5w/DT7WAJ566qlBgwZpQsJQFRZQcSOjCO5KSkoie2PyS52KJ3L5kiMKaB0UuSx6hArfRdtFtUzg
QMUXCA7Q+OgvEH5vJCdOnJg6daoY+8/RNFwRekf6Xfcf2SaS1XP0uLXGMqZ3tPBg9JTzez0YebzS
0lJuZ+Tjg6bGgIKKLxBMoPHRX3DACSoSDaOxsXHr1q1JSUlyZP76HasxaqSPOnyycsna+SKivk+f
PqtWrbJu/zJ6ZYtgsKysLLpDfWwTrKurq6iokAegLCkpQW2MAdDPWzra1voJAIAnOO0qPIPv0OPX
0dTnHVbWYOhtOHXq1F69eom34fipY2DDWioOpxctjER2dja96YIjfqmmpkbUgzmakoNVV1d7uWu0
GF1j9JCUa7ocTVm8fBxECXgPP2DpmKsuCAD+AWlX/Qi3P+JgKoFegpWVlcOHD5dbJG8ck0k2DPH5
OqKDs6qsiA6UOGh0DB955JGg/BFBj7uSkhIxrBXfreSp6MohM1bXRKVEQUGBbLdEZBc5MetWBloU
ziiCii8QTCDtqh/hATTR/1EhZMPWrVs3YMAA2YYNHDJgydr5sGGs4+ePckyXXNPVv3//zZs328FU
0BVSU1OjsWE60BOSHBfd1EFTE2g5uGcTnS8cfxBMcOMjsgL6BXp58SMdcSDKoSf2vffe62rDbrp1
sj0rxGiXacdp92VrMXToUDKrdjBdbqGLhJzYHne82YTqAoIrcPd8/KoFQQb/rEC+UH/BHXPS09Px
M80kcKOkJjbM0ZS7Neid2MNV97PjElH0RHx8/KRJk44ePYoab2B+6P7l6xZPVBB8cHsZukv7C/6l
hgAwE1JXV3f77bdrKsRkJ0Z2Rbll8kXHzx+lXVhVViRnRmUyMjLWrFmD2xxYC4ypDYIYrq4pLCxU
XZAggX6jsaEtLS1VXRbgHjpHp0+fJic2dOhQjRPjBkpyL+Rhtu3faHI/dvhkJRWS7ZZcwSVilui1
dfz4cdQbAIvCGVZt2zgOght6MnO0Eh7R/qKhoYEPKbJ5WIK6urqtW7fm5OQkJCS4mjFHU/fJm26d
TCbn/iPbyI8paaykjXK9FnstuaOiIDExccKECZs2bULMEggCkGEVBD0YdcjvcDYPB3pJW5DTp0+T
GaNnPteMufVjwpWREWJjRiJ3JKtFOcfEt9bvWM1rm1M4k1butlJLQCWkclJpYbdA8MFZbdFcDoIY
zmWH2Hv/wj/c8PSwOg0NDcKP5eTkpKWlxcXF6ViyQEBb7N+/P219/vz5O3bsOHHiBGLmQXCDDKvA
JnDiL/yC9i+iCyTiFoIPOqd0v/zgBz8gY7ZgwQLyZmPHjk2T4HZMfZ+WnJwslh80aFBubi6thywW
rZPW/MMf/hBXDrAnyLAKbAJi7wME99mBAQMAAC9BhlVgH0TsPVo0/A6H1eF3HAAAeAMyrAJbgbz3
gUPEgMGAAQCADo2NjTzCJqoCgE3gyl667FUXJDgRBqyiokJ1WQAAwKQgwyqwIQh0DCg1NTXcvFtQ
UIB4BgAAcIUzrKILGLAVSD0RaOrr67l7aXp6OjJRAACADDKsAttClz2qvwJKY2OjiMMvKipCbAMA
ADDIsApsC//0wCDRgUa0QhKVlZVoiAQA2BxkWAU2B5lXjYEcF/cz5YZIeDAAgJ0pKChAywuwM1z9
RTeC6oLYgoaGBu7swB6soqICbZEAALvBY+OmpaWpLggAKkH1l8HQk0cEg3E8WG1trepCAQCAQSDD
KgCXUf2liIaGhvLychEPlpWVVVFRgSGKAADBDYYWAkCA6i9V0POH3C93PoUNAwAEPaj4AkCAzo/K
4aowtsGyDUOjJAAgaOCKLwwtBIAgNzcXPVDMQH19vcaGcWyY0+lEhRgAwNJwxReGFgJAgNT3ZoN+
G5LjEn0kRU9JODEAgBVBxRcAbuHoI3qzqy4I0FJTU1NeXs71k7ITKygoqKysrKurQ/wqAMDkoOIL
ALdw9Re90/EqNy10atw6MQ7bKy0thRkDAJgQVHwBoAN+m1gLMsx79uwpLCwUmSvkoP2ioiIyY7W1
tXjcAQDUwi8X+uWouiAAmJHGxkZ+jyP7hOUgi1VTU0NmLD8/39WMcUq3iooKrhyDHwMAGAZyfAHQ
LE6nE9knggB63ImaMddmSuHHRGMlwvgBAAGiqKgIOb4AaBZ+WSP8Psggf1VVVcWVY578GLlujuSv
rq5G/BgAwHc4ohgVXwA0C8LvbQL5MW6sLCwslPPtu/asFFVkaLI0LW2aUF0KALTQAwQ/5wHwEo6Q
LCoqUl0QYCjcXknPyfLycvJjmqSvmhAyT2nH2rjQrl272NhYWuHrr79u/E6pRXMo2rZt261bt5tv
vvlnP/tZIDbk33UC4CM8kAo9SVQXBABr0NjYyG9e/GAB5MdEFZnbJkvNEO2uNuCrr756+eWX+/bt
S96jurra2OIrRnM0vvnmGzqegwcPDgkJ+fGPfxy4DQGgHHqPpKenoxsXAC1CtD+isQlo4BAyrh+j
i4T+yp96sgHnz5+n+XRFyTN/9atfTZs2LSYmJjo6esyYMefOnRMf/dd//deaNWvi4+MjIyOTkpLu
vffef/zjH/wReRhaVZcuXd5+++3hw4eHh4f36tXroYce0myxrq5uwoQJtOaoqKjs7OxXX31V/nqf
Pn3ee++9SZMm0da7du06b968L7/8stlNN1tsDW6PxoULF2hmWloaT/7lL3+hycTERFpt//79aXf0
y88cPHiwd+/eERERN9544zvvvKPZkOt2NXNef/310aNH05qTk5N379793XffebN3nooKgCv0k42e
D/SrTXVBALAY9Hp1rdkAQIPGn3tyX/zipte9mPO73/2uQ4cOQ4cO/f3vf//RRx9lZWXR25ycCX9K
Fx4t/+STT/75z39etWoV/X/PPffwR+SFuBUvNzf3N7/5zf/9v/934sSJNOfZZ58VK7948SJ5p8zM
TFr5H/7wh5ycnLCwsF/+8pfi6+3btx8/fjzN+frrrzdu3Ehz5s6d2+ymmy22BrdHg+2fsC5///vf
abJ79+5Lliyhdf71r3/VLz9x6tQp+sq4ceM+/fTTDz74IC8vr0Xu69133yXbNn/+fNr3LVu20Px9
+/Z5s3duiwqAK/QbjevG8fsdgJaC9kfQCjy5r9dee43mjxo1SsxZvnw5zRERUKdPn5YtUExMDE2S
UaH/P/nkE5qcMGGCZiv//u//Ln936tSpYgGyBzTnxRdflBdYsGCB/PWTJ0/y5BdffMGVad5sWr/Y
3hyNX/ziFzQzIyNDXiwkJOTEiRNeln/GjBk0+fLLL/PkT3/60xa5r2XLltH/VAzed3JiI0eO9HLv
XIsKgCv0iwBZJgBoNeL3C/JBAS9xfe9//fXXZ86cSU5OjoqKOnv2rJifmJhIS166dIkn//SnP9Fk
SkoKTyYlJdHktm3b2AW53cqnn37Kk3/84x+5MVEsQP/THJovL5CQkCB/nbbIk9999x2bCm82rV9s
/aPx17/+9e233x46dCht6yc/+YlmMdH02Wz5e/XqJZefP/XeffEOil1o0d65FhUADdzmmJubq7og
AFgYvo+ysrKQgAJ4Qxt3RERELFmy5OLFi/KS4eHhrktGRkbyp6+++mqXLl3aNDVWzpo16/Tp065b
+fvf/86Tf/vb39o0NSZqVi7itb799ts2UmOfvjnR37R+sZs9GjfccMPkyZNfeeUV18Xk6DL98oeF
hbl+6r37ov2i///5z3+6FrjZvXMtKgAyDQ0NHGyP3+wA+AgHVyMBBfAG1/f+5s2bQ0JC3njjDc2S
ZMloSZ3YoUuXLj3++OMiqGnv3r2arQgPQCuhyZiYGLEAuwiyJTzJAUu0RU+F1MzR2XSzxdZZrfeL
6Zc/NDRUtk/8qffuKzo6mv4ny+pakmb3zss9AraFE3xhSEcAfEcEgKERHzSL69v566+/7tmz56BB
g0RVFZOamkpLehOU+/zzz9OSdAVqtiIa5j7//HOaTEpKEgtw45qm5U4s0Kz70tm098XWWW2zi+mX
v0ePHjT53//93zz56aefatZAdpcmRU9GbkAUC/AuuG09bHbv4L6ADqLNEW0lAPiF+vp6Hru5pqZG
dVmAqXH7dj569CjN1CSFWL16dZumroU8+d577/Xu3Ztm8mR+fn7Xrl0/+eSTy1f7S4qoeLGVF154
gSfpsqTJGTNmiAU4sFwsUFtbS5Oi/lbffelvWr/Y3hwNbxbTL/+UKVNoUmSvfe655zRr4JZTYd5e
fPFFeYGVK1e2uRpaT8Y1PDx8+PDhXu4d3BfwBOcpQpsjAP6Fsxanp6fjzgI6uH07f/fdd5mZmdHR
0XKlyocfftipU6f4+Pjf/va3n332GWeNeOSRR/jTsrIymly7di35n8cff1z2HryVdu3akQmh7/7p
T38aOXKk7FWIDz74oH379llZWR9//DFtdNSoUVFRUeQlPBVSnqO/af1ie3M0vFlMv/zPPPMMLT95
8mTyTjRzwoQJHAkmvs5dJjdv3vznP//5P/7jP6iQHOvFn9ItTI6Ljt5XX321adMmmr9r1y4v9w7u
C7ilsbGRLlf0kQcgEPAIRHl5eahVBp7w9Ham38U0f+bMmfLMX//61zSHXvdkHgYOHHjs2DHx0bff
fksuKCEhISIigv6uX7+ejIS8FXJfb7311ogRI2iBuLg4V/9z/vx5siXtm8jNzX3nnXd0CinP0d+0
frG9PBreLKZTfuLBBx+km5FKSK88Og49e/ZsI4VyffHFF3PmzOnSpUtMTAyZtPfff58OkbzAmTNn
hg0bRl9PTk6mVclR9Pp7B/cF3EK/UFxzLwMA/AVH4MOAAbXAAwBgHjjcKy0tDe8FAAIE3Vw82B8Z
MNVlAfYF7gsAk8BBKWS9EJQCQEARBqy0tFR1WYBNgfsCwAyQ4+LsXuTBVJellZjzYeKpVOfPnx8+
fHhoaKic3hDYh4aGBu4CCQMGlGDOByYAtkIkVhXJiNroEtDCbN++vXWbMOfDxFOpxo8f36apv7Mm
gQ+wDyIHBQwYAADYjcbGRh7MsaSkRMxU6GRGjBhhB/fVr18/E5YWGAwMGAAA2BBhvQoLC+X5+k7m
yy+/DAsLW7ZsmWb+7bffTvM5/e+vfvWradOmxcTEREdHjxkz5ty5c7zMN99806ZpLNf33ntv0qRJ
tEDXrl3nzZtH3/rDH/4g17Dl5OS43frrr78+evToqKio5OTk3bt3i1zE3IGaVjtlypToJubOnSsG
USWOHTuWlZVFX+zQocOQIUMeffRRHmZCp0g6H4nVetpTT0eyqqrKU11iXV3dhAkTaD1UyOzs7Fdf
fVV8xBkLExMTaXP9+/cXo5UBqwMDBgAAtkJYL9ec9s3WIxUUFHTs2FEeyur//b//17lz59mzZ9P/
v/vd78jhDB069Pe///1HH31EnofcwoULF+ijf/zjH22axnIdP378L3/5y6+//nrjxo00h5ySN5t+
9913IyIi5s+fT1/csmULLblv3z7xRfpo3LhxtFranQ0bNtCcJUuW8Kd79+6lSXKMf2pi8eLFNEnm
Tb9IzZZWZ0/1d8d1/sWLFyMjIzMzM2lVZETJfJKVpY3ypzwMWffu3WmPaAEvh0gDlkA2YOhuDAAA
QYyO9brshfviYReqq6vFnBMnTtCcU6dO0f/Lly9vc3UUBuL06dOu/urkyZM8+cUXX7SRxqHQ3zQP
HvGLX/yCv0h2a+TIkfIXqWA8+dlnn9Fkjx49eHLMmDFkoj799FOeJKdEn6ampjZbJP3S6u+pzu64
zufUyqL8vKoFCxbIXwkJCaHj7OngAOsiDBjygAEAQLBCj3od63VZN+qeF/j2229jY2OnT58uvsKj
cnO+38TERFry0qVL/BGPT5qSkiKvXLQJfvfdd+wr5E89lZwHThVrdi2zGDWVq63atm3rdj3//Oc/
5U91iqRfWv091dkd1/l9+vRp4zImbEJCguYrbsd1BUGAnAcMWV8AACDIEMkl8vPzPf3Kbrbui9iy
ZUtoaCibAVpPREREWVkZfxQeHu5q2yIjIz2tXJ6jv2keaYvjtZotszyHzNjBgwdHjRrVrVu3sLAw
Hr9eZ6Nijv5q9fdUZ3dc5/OqxHAV5G9pUo7v4q/I41mAIIPuI86ET3fom2++qbo4AAAA/AM90tl6
yT0cXfHGfb3//vu0DFka+v/IkSP0/wcffMAfkROjSU+xSb64r+jo6DbSSFver/auu+5q0zSmKpnP
b775hmvGfHdf+nuqszue3BeZLp7kQC9af7OrAkEGjwVJVFZWqi4LAAAAX+Fs9s1ar8tev+hHjx7N
YVfjxo3Lzs4W81NTU+nrDQ0NXq7ce/fFa3bb+qa/2q5du9L/n3zyCU9+/vnnfnFf+nuqszuu87lR
VdPySDObXRUIPsStWlRUhDAwAACwLvQ7mp/nTqez2YW9fNE//vjjISEhdXV19PeJJ54Q81evXk1f
f/LJJ3nyvffe6927N830tHJ5DrcJijwSGlauXNnmapQ7Oajw8PDhw4d7s9oOHTrQ///zP//Dk488
8gh/yhtqtfvS31O3X/c0nzsUvPDCCzxZW1tLk/TybXZVICgRcfhZWVlohQQAAMtBv51LS0vZenk5
kJCXL/qvv/66ffv2GRkZ0dHR9L+Y/+GHH3bq1Ck+Pv63v/3tZ599NnHiRFobGR5PK5fnJCQk0P9v
v/32X/7yF9ct0iuJHNeUKVO++uqrTZs20ZK7du3yZrXTpk2j/w8cOEAG7Nlnn508eXLPnj1pzm9+
8xv97+qvVn9PdY6k6/wPPviADia9aj/++OOGhoZRo0ZFRUWRnWt2VSBYoTu3sLCQ79yKigrVxQEA
AOAt9B7n7o0tGj67jS7ykkuXLqU5mmStxK9//euZM2eSMwkLCxs4cOCxY8c0K3fdHP///PPP9+7d
OzQ0lMrstmxnzpwZNmxYREREcnLygw8+KKLQm7VJ5LhiYmK6d+++bNkyeq/t2bOHTGOHDh30v6u/
Wv09dft1nfnnz5+fMGFC+yZyc3Pfeecdb1YFghun0ymSUaASDAAAzI+IsfeUWQIAYH7odxMno+BK
MNzLAABgWugns5cx9gAA8yMqwegnlZzoGAAAgBmgn8ZFRUUtCvQCAJifhoYGEQmWlZVVU1OjukQA
AACuIFobMzMzESUCQPBB97VoiCwoKMBtDgAAaqmoqOBnMv1ARnAIAEFMVVUV/cLi+z0vLw9tkQAA
YDxy30a0NgJgE2QPlpWVVVlZiZ9dAABgDE6nU/RtxPi8ANiNmpoaHiBSZMivra1VXSgAAAhayGtx
lRdRXl6uujgAAGXQ06CkpIT7RXLXyIqKCvwcAwAA/yIGDzJzlZecI/T8+fPDhw8PDQ1t37795cCk
D0VKUmBzGhsbq6qq5KqwrKwssmGoDQMAAB9588036YnKj9Y9e/aoLs51bN++XfY/sh0aP348/f/6
66///e9/v+w/pyRvEe4LAKahocHpdIrekVwbVlRURDN1xoUHAADgCj02xaCN9PPWhE/RESNGePI/
/fr1C4Q10tkiAIBtmFwbxhVi9CSprq42bbU5AACYhMrKSo6uT0tLo8dpgLbyzTffkJnp06fPe++9
N2nSpJiYmK5du86bN+/LL7/kBerq6iZMmBAdHR0VFZWdnf3qq6/y/D/84Q/ycJA5OTmXr1ZGVVVV
uQ4Wqamnev3110ePHk3rTE5O3r1793fffSeX6tixY/S+oE87dOgwZMiQRx999J///KfrFjXr9FRU
b3YTgCCjsbGxpqamvLxcdJOU68To8UL3C7pMAgCAgJ6ZoqmRHp4BfUL+4x//IFvSvn378ePH//KX
v/z66683btxIc+bOnUufXrx4MTIykp7ev//978n8kOEJCwujxcTXNf5HntT56N13342IiJg/fz5t
bsuWLTR/3759Ysm9e/fSnGXLlv2picWLF9MkOTT9TegXVX83AQhuGhoa6DcRPUzkpklRLcZmrLa2
FjVjAAB78uabbxYUFIimRmMehuxhTp48yZNffPEFTXbp0oX+X7JkCf3/4osv8kenT5+myQULFmi+
63ZS5yNyVvT/L37xC94cObGRI0eKJceMGUM26dNPP+XJjz76iBZOTU3V34SXRXW7mwDYB/o1R8+Z
PXv2FBYWaqrFRDZX9mPV1dV1dXWqywsAAAFEDvGiR6KRY7qxLfnTn/7Ek9999x1NhoSE0P99+vSh
///4xz/yR/QPTSYkJGi+63ZS56OkpCT6/9KlS94U75///Cct3LZtW/1NeFlUt7sJgG0RZqykpEQT
LSY3VtKvQtmSockSAGB1ZN+VlpZmfK9GjU2S54SHh9M///jHP3j+t99+S5M009N3vXRfUVFR9D/Z
Krfloc0dPHhw1KhR3bp1CwsLI4PkabXy/y0tqts5AAB6IrEfKy8vJz8mUoq5rSUjV0aWzOl0kiVD
wyUAwBK4+i4lvyibdV/kZHj+3//+d5qMiIjw9F0v3Vd0dDT9/7e//c1tee666y76dPLkyfQw/+ab
bzhky0v35X1R3c4BALiFbkbZkrnGj8mBZGTJKioqOLAfbZcAAPNgEt/F6NgSbiLUNOfRTE/f9dJ9
paam0v+e+ht27dqVPv3kk0948vPPP/fGfbW0qG7nAAC8h2vJqqqqOIrMU8OlaLsUlsyEyXMAAMGN
HFdPvivQXRq9QceWcHj8Cy+8wPNra2tpsqioSCzJzYIiX4SX7mvlypX0/89+9rPLTeYqPDx8+PDh
YskOHTrQp//zP//Dk4888gh/l7cib1FeZ7NFhfsCwAA4kKympkbEknlquxQNl4glAwAEDnrCyL5L
bX2XjI4t+eCDD9q3b5+VlfXxxx/Tz9VRo0ZFRUW99957YsmEhARa8u233/7LX/5y2Wv3VV9fT45r
ypQpX3311aZNm2j+rl27xJLTpk2jOQcOHCAD9uyzz06ePLlnz5405ze/+Y1mi/I6my0q3BcAChGW
TKeWjKvISktLkQEDAOAjZLGcTqfI35WZmVlVVWUS38Xo25Lz589PmDChfRO5ubnvvPOOvOTzzz/f
u3fv0NBQ2rXLXrsv4syZM8OGDYuIiEhOTn7wwQdFtDzx4YcfkuOKiYnp3r37smXL6FjREzs6OrpD
hw6aLWrWqV9UuC8ATAU3XNLjkWPJ3Ka/QBUZAKCl0INFBHc5mkbHJt+lulAAAGBe6uvr5VbLZqvI
0NcSAMDQTzP6jUa/18Szgh4jeD4AAEAr4EAyp9NJlsybKjJOf4EqMgDsQ21trVzZRU8Jeg7gIQAA
AP7FmyoyQiSJpYczcl8AEGTQc6CiooJHxGYKCwvpx5rqcgEAgF3wsopME9gPSwaA5WDTJcLpObIL
lV0AAGASRJJY/dwXsGQAmB9X00U/ssrLy5E/EAAAzI+X6chcR7fEQx4Ag+FAevplJDcvkumi+xf3
IwAAWB1hyfTTXziuDqXEFWVwZQAEgvr6eqfTKfdeFM2L6MMIAADBjWi45FgynfB+4cq4rkyMcYlA
FAC8hB0X3UFyNVdaWlpJSYnZsqQCAAAwnjeb8NKViUZMToghaszwEx4AuhHojtA4Lq7mKi8vR+9F
AAAAzcKujPtdclBZs8ZM9mbcminsGaN6nwDwJ3RJ0+VdUVEhBl7UOK6amhpUcwEAAPAL3IhZVVUl
15jpRJe5wulk5cZN+DRgcshHce0W/bLQBHExdBfQ7YA6LgAAAMbD3kzUm8lVZ7m5ud47NIYj0OS2
Tvg0YAB0adXW1rLXomtP05go7FZ5eTn9DIHjAgAAYAmESePumUxhYWGrfZpo9CQqKirYp/FQTQy6
cwK3eGO0HE2pIbh2i65YXEsAAACCG3rTvXkV4dNEfZo30WiuNNv0iXCdIIOuIuGyCD77noxWWloa
XVd0jbHXQtUWAAAA4Am3Po2TnjGestHqI6rUEKVmcurr6/mM8Dmi80VnTc4nD6MFAAAAKET4NLnp
U65Sa51VE7VqGrcmBn5CM2jrEM7K6XTK5kqnCksmNzeXw7ToLHOkFtKkAAAAAGZGWDXR01OOUmtd
66eMHLSmCV0T1LlD9YFpDdwIKKiurpZ3Uz4I+tVWGjguS1RkkUnjU6Z6dwEAAAAQcESHAk3Fmsaw
taJngffIfUX1EZnZWgQnudLHbVqG1qFxVqL+ikAtIgAAAABajWzbXJ2bxrzJ+MvkBBphogTcFMiI
2ioGHR8AAAAAYDnkbgje4+r6OBzde1AfBQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIKY
/x8x586lCmVuZHN0cmVhbQplbmRvYmoKCjUxOSAwIG9iagozNjE1NwplbmRvYmoKCjUyMSAwIG9i
ago8PC9MZW5ndGggNTIyIDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVVktr3DAQ
vvtX6ByIVzOSZRmMYW2voaWBBhZ6KD20TdMHaUtDoPTfV5rRa52N27CHlUbSPL5vHpY1iN/Vbn//
8PX2/ccHMV5N1S8hhawlWtFCV3ctCC2bWmoU95+qNxfiRwXC/+4/V5KPWvG94rsYBXciCrwqv5fF
Kl36Ut1eVAdntNSExQW/uXQPJdDDYnmqY/e673dX04tZyGEY5xCF+zkvx2OlUNXWtkJpXUtjxfFG
7BYlFDp1Shxv3/awDGB7OQ/QyxGAFqCHd8eX1eFYXZOTu1cf7qIZOGfGaW+kFtjZ2jg3yYoWtgbn
6PG2lzAcv5Xqxp83f6JCPKewdfGCcQp1jUZHt8EmlzvvsiKXh0v0G/RuO4n1e0C/9Ptwrty5HH2Q
dC5bluQXrMH66PlFlDoVzXDJD6HHhdakaY4aoKM1Nlmb8st8YU9C44UHkNnNRnX+Dnk7DcFeuKJk
9kM1Q1SqV36TUROdD3pHksiDlyzxnfKXTyJjZRQYHRGC0GjD4pQE58lptSO4W5FDYQDBhinyElI8
MCQU81gEEYx7Z7Q5Ex+BHpDko70/Ih6nBDfFuk9QYoFbVImamGz4dXwYeFGlW6cskqkCZVSJVbAp
z/yGIQ1548/JGDYnzij1dJWprSpz/yj186pMb1WZ6mpr2hWRqBnaORcRg7JEjHTCa0pppRe313Rd
+Zd6zhm2j+E7pDEhTaxaSlWSe1spjw9DmamUM6oN6kyUlDlxhhNmbB2LykWotltJw5LCrWAkFFnu
MmCfZrTZYlS6kWGe2TfNFqNuRhhc9029T60QGq4/DmdhLDn8QMTjdGeYURYFAir1wTEAtELPQ6VC
v1yhFYuRPKHawNQGEhk2M5jrsMk5pZO5sq/ytXnI4ZwSWHTmIhuoV4ycDGleuAtPctpucAptU7tP
gedxajc4hRZqZXE9C1s3kVrnK6glTykuMLOu3FU7yxQyQLooGpy8GIdcPgXr/5ouqfWte0dROLGk
y7KivhzTIeTEo3SA/0nEWOt5vsFIhqacL+Q9TmBSWwqN4JC7+PYEdFVUI64bJ0/zXDwhMcvEyoPr
3CdLMaVz72KgoCuB4IvzqblV1a75PjfT4ocRtL1yk7QJgtQyc+eWIyeG4a9EP6+Zy8dWcuM+qZ9r
8RdVDXvYCmVuZHN0cmVhbQplbmRvYmoKCjUyMiAwIG9iago4NzAKZW5kb2JqCgo1NDAgMCBvYmoK
PDwvTGVuZ3RoIDU0MSAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicbVBNSwNBDL3n
V+RccPZlPjo7sCx0ty0oFCwseCgeRLtVUMGl4N93Yrd2FQlkXpKXl2RghD+pWAzHl/7h8cjNpqUP
BsPAlhwlmRSFPYKBtzzs6W7G7ySsNhwIp1LkNzpx7TnxyueESmmMCfohPVM/o1UeOlWyE4IGV7kR
8t04gb81ituqKjbt9ZJR181yvCJb3rLpyAcTrWOL0sR8UPfExTqwjyalHPW7ShISnKCWSkp1Xt0C
jT6SKwqQpJWAeGHkTC7CKnTwWI90+Pq+u6FVR9s/mziZ5xPmLCEYcemfVSCj3kVCf2jLXwWeWo4K
ZW5kc3RyZWFtCmVuZG9iagoKNTQxIDAgb2JqCjIzOAplbmRvYmoKCjU0NSAwIG9iago8PC9MZW5n
dGggNTQ2IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVk9tq3EAMhu/nKXQd2LE0
5wFjWO8BWhpowJCL0os2yfZA2tIlUPr2lcaHepeNYTEIaeaXPP/nMWqCP6paH1++HT49vEB7u1G/
AQE1mgSRss6RwKHX6Awcn9T9DfxUBPIcvyjstyL8UL3WjAvPMC7IKKlxlk2ir+pwo3b80vkkMxNI
seJGpNI4S09nVO/rurrdvNkCNk27HVzww6dsO0WWxTmDdU5jSNA9QrW3YA2Ps9AdPtQUmxXVuG04
OA6USh0kJcql2EskN0nWvdxILgHbQV/W/wvRTlki26zMfC9hHPc/dm/VrlN3hUj17vPz6IkueWIr
Hh2YnHRgJsWSg6SJqXSHGqnpvs/Htb8e/44DzaWBkeFS4IFOm+BGRpQKH+OI5OTs0MhZbY1eQilb
Ob+XrFAzu7LHJtmpd0F0SZo9NqsxHyUcvG174lRb+zoFu0QhOG19uo6CW6IQ5LrSCYXapGnaWVNg
sb/YxFerMNoKDLHYIzMCxUzw7LQmtAbUtqdsapNfp+KXqFjUxGauohKWqBivY8xXUrnQNFA5uSKb
ZvBMqYBach0XXFP2OkV7neu04JoyaW/DmQG7n33THaEcfD383OymJd9/1bP/+g7+AVbwMOsKZW5k
c3RyZWFtCmVuZG9iagoKNTQ2IDAgb2JqCjQ3MAplbmRvYmoKCjU2NCAwIG9iago8PC9MZW5ndGgg
NTY1IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJztWttuHMcRfZ+vmGcDanVV3wGC
gCiJQQwYiBMCeQjyEGwsxwEZIIID/37OqZmemV0O1ybW3odEEkRu1WxX1+XUpXvknYw/DW/fff7x
h09/O/w43n3zfvj36EfvvNaxSHOtyBh9cj7q+Pm74c9fjf8aZOTfz98PfnpUxqdh+q52xuPYGRRF
2m8+LV/6x/Dpq+EjNt1K0s0XSLzBQi+2cPPxWMbbP9zcvP3m/e8/jP729u7DbAX+Qsu7h0GKON/a
GGJ0Ptfx4e/j2/swBoW4MD58+suNxts3ciPi863c+MgfiT+MvJs+V/1oH4SPJUV7VpPnSl+3j/Aj
3N2+0RupfBjC7V8fvh4+PgzfmrVv73/4/j+fv+sqy6Syd7nUGCQUTXBW3RIt9D+0KMTqsk+jSHIC
s54GSdlFyQvncewcbeLUKzgpeldyWjlD56yrTiXP7vVYEJpCZs3QJrTsJ+1A5FnL2MY//s78/hP+
fQ2E/PPKiv4J7oXfcv8zvkzQiVpckjS2YJs8DUHo9NwZ2gpQH8YG7+ewkLU6oPiw0Km6FBJ0XBjJ
VSRLl5aCSyWOfbeZPBgURdfnqbhWy7K6k5P0w9AZ0/aPyxdm7bq0WfcT2w6njMfhsrD+P7mu42r2
k2LHl4lfD1c1Ofq+L67BBd8W7WYSxs3a9+fd1nn1kemHV5p+asqlxeB/xzETJqyUL41H9xpPEIWg
MObkis+970h0raXed8KNFzSM2M42idCbRDP3eml0fto6X2bXe2vOWhGOVFFfoyvo4qi9BQt8XDio
vTNHC8zPkckBD0Dgyhk6Z111KvlCXFxX0eeBi3uB0ywQNmomynQncGj37O8YGyIbvt5gHDgXv7TE
T1sUKTkWhsy3OjmNxMxHW2MdSfhuZadsCHPobgm6cGa3kIPvhsLiloK4GNrKGRbOsupU8qV5fVVF
n8cv7yceQoVEQBY7qL2Xee8QNP3A+KnF0gf7Oc1tgXz/gfNbtCjbk3MBLj3Av3QAiSG6iI+SmhMg
+GmIPkC9uHAex87Rqi4AyeBkmAAvLZyhczarTiRfGODrKvq6KS4G6G4ye/GIHpmUyybpO2ctHhHZ
LaKb4tE5m1Unki924jUVfZ4ldS9LosQzJc4yRO7RoML9Wei310I/+TzFPbNk0xclTYCaGDBqZij6
dyvRCkae8TRzhs5ZFp2IvTBi11PydZBPnkNK2SIJ1TW2befrnBVJvbquSForcF91KvliB15T0eeQ
l927gMgZr7ZzuCfk5RTyq1jZFcvJsuDQitFR/W7DmcUKE+vsfYC+elbw1WHvtUCz6La0LdCdo1EB
OzsHofgmKStn6Jx11YngSyeFK6q5A4ewO+ghblkArpaZjj1sgrCJhS0UDgaBl0L4iYEANVHsozEw
FdjocDagcSmQm3jWHk760AZ686LFE9gMceOoEJpT1a3JM2fjqMo02fppYmzWHIu9NJpXU3Inlml3
6IvJaZCXYxmrhcziaUM7Uu1s4PJrMxGuQmBli3FWr7idejpn9YqW6oDqjVs6Z7PqRPKlp66rKroT
wbKbjaE6Qb2ETWgLeyGUO5vM399qT8CP/p6T/PsX6/Xu6KO8/EX5F7QDn3VnJ//OxOfNJkROmisB
DoBnodNeDR02tbIdswXzhfptRDpnExEccMpRRnXOJiInki+FzlUV3blp2b/jrw1jcD0DndjmEAar
4HIGM7rb4wVYl1R+M8zo0vijXScmgMW2RwikVZcT8Gqjj12ld45kG2Q27nvOidl+P8Kdz6TDuC6r
1mnUVY82zBsYLzi3Zwa04ah2RG9XzIjaQQZLsUZTIPpmKgEZWo84J1Hvt9PPvWA3cYQMhITMu4aF
AzBJbtv685zzM17oslpwUqaOJq6gHHG76UZCg1NKWhlHS2bVd4prmg9pq9V8RsU3nJPCOfthMX7u
jLMVTyvDu2SGLkfBHY66VHUyfbH3WQdP2fm6jU3AmT01PeIc9+dJyZ3jLdwfkOfc2aPcTaf9KQwr
5+To2iP/S89CJgpIxxDRoiVFYBVIvOeVNN03tWKXyJa4WR2OZp08gJFcTckYFXHODdEca0H6IDIF
PuPi4iJvfVnTYrPVlS+jKg8QbUTdKREOqQgLptKZPFiVaFjIukQ/tGSzEEjNavmj086xQRXLHXqY
62ER08TbZXhiGfXNasxMHlCgopnQnwvvryMX842tKsu5kRJ5G65KfNrmreHrQG3I62rCPOhGOtpw
WbfjmaaasMI5HBjhxXtx6P2UnVgrYDgynee2BO/lygj4LRmArEDZnaHIwGJkhEqIYAXOlG8Di5Eh
BZPeYAq+TotGXstyYC7N3votZHV+Urwz4OvEzQOGEx0RSHulEJG9SAvGFfiB8JmBsCOuGnCEw/iY
oSAPiApsoNbNJIQr1unyWKAY0JEBHcRbUXMTSgXI1MwtiJDHg4w08GLx5F0B8ilbCwFaaqcmsED5
5THcVau9N5iR5gFP7CzVgIi90KYhOhG1HMGq7aSRICa6pZMHS1WGtz9PMmkCxyNTJAI6kS6YQI7Q
I8RGR1SCAA9hD75AR4wXUg2Oh5UReJ4BiXIBMwryGRgP6EOwhnkDF1C4Ovofrsj88oRSHPSFV7Wd
REZ403tmwHo1VSCGwBE76x1n/uG0FFw443wpLF8Ky5fC8qWwLC97f648gIFEJSPnSQjHXqJNSQKk
wZ7yP1Idxh2JRzWBR6oAW5S0tGDnqQykIRdqhUkIVIVgkJzzdgUe1wGO4gUFxBg4g40kIzyKDSgf
FF+fF1TFvCvvJPUx0gKncXqHXkMx2Ho/7QdnqtSp6gAzmPZ3JR4n/JMxcIwjwwNnJPlCl0YCAiRT
snIQa9yXeJzkHLsR2mRJXvCLE7aHvxh4lgTlO3PmfMkvGI3ABRTuObGfjMHcRXQS815YSwk7gb5h
OhwwNvKCycfJTCfCVhQ/MOxNAkiNhUnC4w3JwOSlhvtRrrxG0DmDCRsW02o5yLEdmE/JUrLy9Svv
1FG+sJsyb/YEbrOWuEZUS1yyFm1PgyVxCsEynr0U5EuwOUnVJ6sDNQoZvvJggsKKjEFiV/5/LnSE
iHMKyCYd2Dwqfzv+F+IOOG4KZW5kc3RyZWFtCmVuZG9iagoKNTY1IDAgb2JqCjIzMTgKZW5kb2Jq
Cgo1ODMgMCBvYmoKPDwvTGVuZ3RoIDU4NCAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFt
Cnic7Vpdq9zIEX3Xr9DzguWurv6EywVf2zdkYSGbvZCHkIcwWW822IGYDfv3c05JLWk0momHCcNC
bOPrW0fT1aX67upxg/S/dq/ffP7l5w9/PfzSP333tvtX73o3OF/6LHWoWfrg4uCC7z//2P3pm/6f
nfT8+/mnzo2Pcv+pGz/rG/CxbwBZkXar3+YP/b378E33HpuuOfnVB0i8wkIntnD16zGP1394eHj9
3dvfv+vd4+PTu+kt8BdSPr10PiZsHnsNYXCp9C9/618/a68e7LR/+fDnB0mPr+TBvXuUByn2a8Kv
zs8/npzOv/ORyAjEx7+8fNu9f+m+txd5/fzzT//+/GOTRkZp3JByCSqaIYQbypqo2v5QWA1lSC72
4tMQtUAfAtmDpBn52DdEnR9ioDpicENOcUG6hiyrtpwnzbnBO60ePEuCNFqTG6UDkSYpQ+3/+DtT
6a/49y2M/487C/oD1Au9pfanP09QiT4PUcDBBdvlU6dCracZ8TXDpRW0H9QtdK3YuvaHrgGpYp1C
hhnIg5S+8UthqEJHGjecyIM5GuSfn8c81JLb4kaNvA9dA6bdP86faOI1dk387QseTl75Vuv+/2mw
OdmkLZ9Df5743zpZTUOBrG19DbBNmUWcyEUl7Xl752n1sQoWnXyxCk6Qj91vxYl+KxoancQS/Vxx
/F7FUfFgpX2m/LEVHIEz12gFx4dHfXCCEhLqxRKirYR8afaTUofoQy8WOpWZOesQ64IgM0+IL2UQ
sehEPQ0xLkjXkNWqDecbk8x9BT01XNgznOQwhIANsXGKabacwHJilkPJf6UPEuxnfPQPIV00YGwG
rL4GkZxChmLE1TJqjcSEo+oxs0R8FqoUZXcgpheHUuhnhHqZkCRDRolFaVUZgtYF6WZkXrXlfGuV
uKugpwZM+5EHmyOYITU8R/dC7w2M5t89vgoPHrb0D+jk+POJP0WJu3fsAoNZ2Z5cMnC+NkID3DKg
wRbUrxAD9BaCH8oK+dg3xFdvuQxISdaWz0jXkNWqDecbDXxfQa9r8gLcZ8wVicFE2RRHjhWCXSbE
Ix+oU5NNB1kjXUNWqzacb1biPQU9jZKyFyVIcUONTHN+8HIuSOQZRUqfL3p/vdb7I3aBusz0vphT
oV7nWGcEbzYhdKGQi2UN1GYXFqRryLJqy/lGw91X0Ou8P6ITqpLXTlX5akfuMSGLU7VEuzjVkozb
qi3nm5V4T0FPvV92BwIRHV9Fe3be/eNUDC6e7uXq0u7qkCvTXkTtU2jDl4xuVGcEDc6ELC2P4t1r
Wrc8DVlWbTnfWtrvKuiO2Xa7avKNsBCOYEPys9mgfzCk2TSP9dwFKbDOq4gKzroeWOMvmvLqNttD
fu/okGjx0cZAQ6GgEsYZgYYmxPRhp3JfkLLTCukaslq14XyjKe8r6I4pd/tsH3VwOfceHV6sZRWC
80juyeZwb90z7ejfbuy38I/7/KGvigwCjiXFHf7ujUvuPXm7yxVO0tW+wYPjUcmHmjaluiGrk02I
x0jXkNWqDedbfeOugu74Rt61HaxktoNviKQd2+HMDNdQhrheco3d3sfjbJ6Kmmto0ZtcY+5+gg2G
IlK/7QRNSy04QeLkI5wu2IR0RuKgoyablk4RxEf1UO0Pp8x5yG+sHFU0Gg4ljV0Gtis58uiF3uSY
Pl4zjXd2fMDroMGbDBpNKqTxEtIa2dq3zYpOFWFjFE55JA3Z2RyoIYoakOs6lZwiER1WrWc1MfPi
kMYqFWtORpVA4R18spEyXsjj+L5Gjlc1XcwZLeH8SZv5hOoSs4nhcY6lYDiyJDlCjlfNuvjikQ3O
UC77vkLMwksQwXkgaeTkKlAFkCHbWCzwvRKaljKTByulJUYC49QD/wdS8HSEmB8iFxc0qqU/4Iwe
sU7NDTwdpnCohl2cDA5sBYc166smGvzRn1X6AxHOxyre14+OlHjUBm8/7l8g9AG+CEu5kaWI2MyG
rYOjFxXLKJymNPpgoSgxL58QTuaC0WgY4G1Tz0jG0NDBfBSvNn5Ci/mjphUHOoH69R4KDax29TgC
FeMYkWN8wBGVo0XECEhsEBlPmQByae9jhtEKLaIsW43kMZaTwRlQFNJkZIDX0qJo9HkBFaBjnogR
1eSOxhZOXCElj8o4ySF599XZlUcj2f2ipTwsQLJiyeKpeLkSxrFpwC6p9jggVmf6b4CYu3sEFfrn
PsMjPGT3cBdNjQR3Dxfx82PBrnAYkMGyZLKeC2RVUwwTZQgcbaZgpuXJNfM/ugZDoFGj60D4+TFE
LGCax0Ch67kC3mjrpJpvJgtH8K70ZDzQYltFemZCvYnSSHCHFmnk9jzKKAsMnbE6gEvoscfk+HAA
mNkATn2V0Rb7gvTM1NZI6NR4Nxq+rvw0EgVehJMMngA1IeQyyawmudK0PVpidePciv6KAglNzRSk
BXmYAVg/miSIyNF3XAHv42xw2KaHW8euX5PN12TzNdl8TTb7yWZs0/9bygCA0DUmjGqQGSqkxyWS
gGE+kMpg2GF4nCfGm5+KIz0AOg/TRkr2nKEPQ/F0WqFid4bhcVZgr53x/mOagu1BsWdjmkojyXvD
AseIu/y2OQCNoFT7zbpnxC5p58YdHR25WBZinkx5n+dx1H8yQGADAJx1gUICt7fUMUdEGB1k8brP
8DjQwTDyJqwQcJV5g193qWb5yED2dlwDKe7Mawf296EF9ycDGL4lWMLxQZhb4XgcFujYVfMUGM+8
8nE8U4vZjv0IC37jhaQPRvJkQFLp99ilnjF04UzNtyim51TYoxAo5jl4gDcwlCS0iRACGRk6exyP
Ipe+DcPmMEcuBwlqgVxYAhD2rLHFnfWdTbh+slxQgoVzQM4TKBFVlKQwYFEaQvEk/azGzRWzO3/M
3F5fJpzxYkvWp4NNeW8Tsi+4ePZy9b0WTrFxNR7HUQwJoh4NzBuyDMyDMhhXSNeQ1Zh9w/nWK5m7
CnrlvRbHBai+y+RVK11zQSjbiHAAU10y2SjtCukasqzacr5ZifcUdCco/JcGhbKt4tdNMpox2bsS
nm57xYb+3iLkzeXgaPPitmPgzMraBh2iDTlmhI2CK+v7uh1kmfY0wef1rIV2cSJoYOzWCM2iRlMU
y2U8Qo4WTZOInStItIzHI57Avv5oxHNyvTiPNVaXHaXdddB3qpPJe3ZfAK41ftdvEq5vgDWoo+nZ
SaMxmJGuIfOi7Sve6sX3EnLHg8PGibZuyxqGjUWtzzv9KkoonH/y6wvTV1Hguxe9Nl6b0hXunOye
Hp4W2Acpe8EVwi9s+um+H0qwywNlH7tGuoasVm0433phdVdBr/w+akIWtECaMyWO3LJCeHVWp/v+
limVZ4U10jVktWrD+WYl3lPQnYBIlwMioL11/kIe9/zCgjyjufGb/P19/x+lvXvQCmVuZHN0cmVh
bQplbmRvYmoKCjU4NCAwIG9iagoyNTU1CmVuZG9iagoKNjAzIDAgb2JqCjw8L0xlbmd0aCA2MDQg
MCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJVWXavcNhB996/Qc+B6NTOSLMNiWHv3
QkoCCSzkofShbXrbhnyQSyDk33ekscay7+6mlwWvPZbOaM4cHdm2YL43u8Pjt38ffv/zmxlfT81X
Y41tLUbTQd/2HRhnfWsdmse/mncvzOcGTPo9/t1YedWZT42MxRL4aEogQaVnW93poH+ahxfNiZPW
SFgNSA93PNFCnljdrjF2b/b73evp5dHYYRiPcxX841WO5wZjbCOiIedaG6I5vze7ezKEDEfm/PDr
HsJwB3twA188X1zgi53ShVIs5tdx+O38S3M6N2/zqnev/vhY8sKlvJzOW2ewj23gdee0zsQWeOXn
h72F4fyhhhu/vP9RAPESYMcEQGBA12JwpQ6IuQZ06AdMC6Y99PbIfzakwJQCh+EOq2e+4J5sDsZ8
PaRomsszNehLkFlhBmzmwWoaO8o4PHAEj8wUVu8gUZfRmLnMLr8aB1gyYpURvIRh5IEkowFyhGR2
zodaVg7gvFao4cYKVRCiZuU0uR6ri6F8J1OkGsoDE4EQBaC81f5f7kvnuLf9pi+CAfOqU0rFlPWV
peEJ+vQW1zwhD6lbU0+xmSXUVsDcaLQVhu9YatyOQnAe7hYkIUB6BPNsKsz1GiW6Ln+6JX/+R+ue
J393S/7UtzF0W5onEZhb1J6FnKotDAWtS5h0ykQWiC/aqpvuVBdb3RfqjorSq56mEtuCJHFREdda
C4KJp3ojSw2nNNBW2aUe6vmPDioP3c8+JnbTAp2YQRDJz/t+1kHenif0QtBMgmL8ROrs7x3Q1oJg
FtjMey6LvJJESjXpDqYNNcU8jkuvcm0kg6hugi+g6y5d8Cx2KFCHItX8Txxq5UtBzZWkmlN+wom4
2Z1aSFVP7tjiq2kHVp5UaWyS6NXt5W9sL4i8G2z3vO0VbmwviL51T3ZXGMqGWgqEUSJOPXnc9rkY
21Ra5pT9acFBRfEapDjMBxbT5tX8Muq9qAGSGmCrhrLx4mK603o9dA9zsuKG07qwqqzKMg/FMuQC
y+GZsZ+cvpULiTMs9I23ut3d6rb3rQ/PNNN4q9seWsRtu4ukq0qlhI0RVqS6vuqXdqFWwVVP4fy8
Br9eS1BTCcJ7AsYuAXcpU5+7c92oyHb8xXmlPuxnedxna7Ru3e/Kehe9jhfqLwfBytXrY3X5YCpn
0OLloZxMEGfcoGZQfbBU5bvwfz5DgIC/jbfeLGpcDn8x6EqSpWkrXb41/wF/t6A/CmVuZHN0cmVh
bQplbmRvYmoKCjYwNCAwIG9iago5MjgKZW5kb2JqCgo2MjIgMCBvYmoKPDwvTGVuZ3RoIDYyMyAw
IFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVTbitswEH33V+h5IYpmNJJlMIY4F2jp
QhcMfSh9aLtNL2xbGhZK/74zI8txQjYQAs5IMzqjc87YzoL5Wy1Xh+fv+4+fn01/v67+GGecdZhM
DY1tajDkgnWE5vClendnflVg5Hf4Wrmcqs3PKtdi2XgyZUOgZO1m0VT0rdrfVVtuOkfCWYEsFnzQ
gR6chacYy7dtu7xfv9oY13X9ZmTBP75lP1QYkyUC44msi8kMj2a588Yjw3kz7N+3fuc2HbSQugW0
bitPcLKxAuI/10NwXgLqPgyvq+1QPejFl28+PZXWcKk1dwyODDbJRr66diaTLPDlh33roBt+zOH6
34//CiBeAqxZA4gMSBYjFSqQjjRQaAgLZuCZQbfgHeJwZIHCgldJElRLqJso0VoiwWARVIOS7bk+
JLmwKNRLEgQgw7gwFSr2Ti6wkmXUxi+r5q+pFsn6kG5Tja6pFmUK4Uw1aEhZQS/PHI+0ao29cBBe
kDLpUc6cyJWbU11zajPpOkKURJ+rIHYjGDsTs/SlTgtQXUgZSE7OHNFZBanFU/WRfC+7pAW1d0ef
kGArKMrL+5dtCdds8c4Cq3mTLfGaLRhsXTdntlCeXBRaq6IhJFx7zFSKRJSmbFDKONmgb8Lome7q
TIpcKhyrrioGlmh6I5pJaz3nQ/YI89S73cw69lEMA3+bq2HmSMbJ1PS1mSas+Hq13WTgZWH5Kxnx
/CtR7of5fn42vPNZySkq35M888clHMf8yKxIK9RJ2pzP2IP5D3dUZbwKZW5kc3RyZWFtCmVuZG9i
agoKNjIzIDAgb2JqCjU4MAplbmRvYmoKCjYzOCAwIG9iago8PC9MZW5ndGggNjM5IDAgUi9GaWx0
ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJxdT9uKwjAQfZ+vOM/CpjNpaxsohdZWcEFQCPgg+7Co
dRdWwSL4+yapLlUGJucylwwrwY2iqr/+dt+7K+rljC5gsGKdIxOjTCZIOFWcaPQH2kxwJoGP/kg8
WBlONNTqp/CHp+BHec4j9F/0Q92EWrd0PEmPCjz5cI0soXEEX2dEq6KIlrNFAy7Lunlc4cL9srYU
T5V2uyUXZdIUdo9oniLJlDEC220LaSQvpRB2iU14ZMras8rjONgcv3kxJzz3XAKUh1p+2U9qLa3D
bWvcAVGHS3YKZW5kc3RyZWFtCmVuZG9iagoKNjM5IDAgb2JqCjIxMgplbmRvYmoKCjY0MyAwIG9i
ago8PC9MZW5ndGggNjQ0IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVlNuK1EAQ
hu/7Kfp6YXvq0KdACGxmIiguuBDwQrxQ1/HAqrgsiG9vVScZs+Ns4xAIXZXqyv9VdTU4tL/M5ur+
4cv+3YcH219vzU8LFhxQtgkb1yS0HoIDT/b+o3l9Yb8btPrcfzIwfUr2m5liaXHc2cWhqdSG1eoQ
9NnsL8wgP11nolWAGpeyEbBsXC0f59i8atvN9fb5zkLX9buZQh5R2Y8GQ3bog2XvHcRsx1u7ecaW
SdKxHfdvWuy7S2whdthiLsvdYYkBWAwYigHdHAe+W4WRhKnNhy8ZuWzwfz2QSuo55u34wgyjuSkV
2Lx8f7cw4CkGkR7AW2qyi1KDguCtcEkVxn0L2I1f1+n6H7e/l4R0KmGSYmKUhN5R9EtNMJd6kIdB
UFnFUgtBVkhqbtXc6curSU9TcI0ieschn0fhaxRRjxf+Q0FBpbJKLaKHrrRw8fXIU9PE0cuKS3jU
11aRsTSW/gc31HAZHIrKs3BjDZeCS6k5xr0SobSjMPVN+WAQmqVtBSk/jZAqCNgElxOfh5ArCNig
CxyPEVgrLrqLai+q5x5UZDc12RFd5DPHBU9eIovukJzMzJFuTMhTyRfNOu+qnHp1wHSueD5H/nAC
tTE1OKxdBkjJcT5zjLB2GyCxg3A8RxgnvSKVp0tvHqXio61OjhyupI3j6fA9ArqxfwBTWGj+CmVu
ZHN0cmVhbQplbmRvYmoKCjY0NCAwIG9iago1MTkKZW5kb2JqCgo2NjggMCBvYmoKPDwvTGVuZ3Ro
IDY2OSAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVZdq+Q2DH3Pr/DzwvVYkp04
MARuJjPQ0oUuDPSh9KG729sPti29LJT++8qWP5LcJDAMeJLYlqVzjmQZDerf5vT8+vX3l58/fVXj
+0vzjzLKaINeddDrvgNljdPGonr9pfnhnfqrARV+r782RqY69WcjazF/+KLyh2AqvJvZU1n0W/Py
rrnyoXNLOFsQXp54o4G4cfa4tHH6/nw+vb98MykzDOOUouAfezneG/RG90CKrNWm9er+WZ1u/Ips
jtT95ccz2uEJzuYaRjADP1IYHA+A4ekShikMNgw4/HT/trnemw/R/9N3H79kD2DLAz7YGauw97rl
CKIDVnkNHMP95WxguP8xNzf+/fm/bBC3DHYMBbRs0GpsbY4IfIwG2gFDBDyMwVs/PCGHaK4cGqUJ
x08cGoXQ4mJ+GmVlnIurwJftppN9NsyJwWd+x4AJumoCSJAKm8HKp5mVKVsFiO/xq+yp/rb5IFlE
wXy2Jduiz1P4kmISO8/RL/Mmlqn4J55HIxw9xhPy2VNGI85IpFiNk7iQACj8b/PSWea2X/GCEXST
YQMvR944YCoxZdQLWfHMDBWOYfeNRlm34wUS6wx3vEjAtGJ4z0JL2vX20ESlWniBBByVaC4r3Ivz
sJBW0dtcrhSPWOuF9hOPjhKP/9HYxxLPHiUe9dq33ZrgdoAuhRJjvw6xnmyElwvOYj4lZdydZZtk
yFtiDRK9XvLHlBaiJ5F1MAcu0UFzoIFSklJNWDtUAmiRlbJI6I0ryNTJqFzO8LJ+kUT2DWsLRTyH
1zYDgt0+p26TAq8dixtA2x4zpUD8RpFTPOC03TIIhu8FAoXGa2twTWpQLfkQUhdRDZ4jRoiYkjaF
D0skcwE63h3Ga82P66wSxAELMS5BvJBFTS0xvkI/Wkuq6KHnL5n4JE0uPbQi9hF/N6r5TjnJCIP3
Gq1f31fuLXh2I2lmddnVyMN7vIGSE1GZLtf8ZGBx3ayvCUhpMC0SkeemBQdp9zWrdy7ueFv1dFCe
un0pg+Ni0tGDUvZHUgbX6s7TCmjmFVziUYiVMpTUFwTiC2xFinjhHdQVZKneRjZXsIyDk+tz2VYc
wdIfwIIhHfsHYYHNFrDggk4TvLnPICATZQ5nGxIlpoEN7ttbGFyNOcGUs3EPppqvIhtRZOo3ZjqU
9F5UUk5WzAVepqXKiyxt+brQX9zMt09xAoUrTo4L81S6C0l8OfSIGNhsZIWZvtW9tY8Ss9nIZmJ6
0t6tG1mmggtO7HWuMlA7p0Hki7Vf7UtHUgvpI0pN+OcCsag+s9ppSxmuxWHcboryIbtNUfV0dlNL
PQsVqnaKvPRWuuPSH6yaxSwhjDdr7hPXPH9Q/wP+0ApMCmVuZHN0cmVhbQplbmRvYmoKCjY2OSAw
IG9iagoxMDIyCmVuZG9iagoKNjk0IDAgb2JqCjw8L0xlbmd0aCA2OTUgMCBSL0ZpbHRlci9GbGF0
ZURlY29kZT4+CnN0cmVhbQp4nJVXXavcNhB996/Qc2C1mhnJH7AYrne9kNBAAwt5CH1okt62IWnJ
JVD676PRWB/29XrvckFXlqXxzDlnRrNGg/qv2j88/fj78fdPP9Tw9lh9V0YZbbBVDXS6a0BZ47Sx
qJ7+qN6/Uv9UoPjv6c/KyKtGfatkL8aFryousCl+NsUsbfqrenxVjf6jpSUsNvDDzh80EA4W07mN
/a+Hw/7t8fVJmb4fTlMU/s97OVwqbI3ugBRZq03dqstntT/7R/TmSF0ePxzQ9js4mJFHML2fEg/O
D4A8O/Jw4sHygP1vlzfVeKneBf/3v3z8Gj2ANQ/8h52xCrtW1z6C4IBVrQYfw+XxYKC/fCnNDf9+
/j8axDWDjYcCam/QaqxtjAjaKRoz+iiIo0COgjgK4iiQ1/xsMG2/8w9nHw11vGx52aZDYRb24NnU
vDJw4DwDF2wABAulOflo2P0gloJNjJZGxrOzEGAeeLRipO2hOdCxp4ZR9rZd4IPP49GHExznF2YM
i0TXCaAtAmqrybX3EWC3CKhZ17AgAGrxN8Ij0W+yckqgo58h6w0pYDMIoHa+vxbY+RgInnEqaPvj
owd0RxMpg3gSjjm/k0yxk6kFm7fll+y2vBmDVoLv8kGiQkPZx0I5EugpiQCjJqAtxRPiOWZFhbPk
+mhgkFTEpKer1Lst6slo8IzdRX29RT063TTds9zDE7SMJyQIfMIEnM1Z8LfzBCu4F8QHVsCZ5F+A
GgMkYYSxD1qShMNoTzjJgBWfFwaOM3lkgAPpbpb32FyHuFlFpNUOFXSNtsZGhIG07ShAjBsQt2sG
wfhKTf6e8fWN4Fl9O3tNexC4HkMr4U3QplwKHOSEW2AbQB3StpuKi5lim7hIrp8KlNAX9NtOhCah
y9FaUjcaWaZFyAi37p+XQeKemHuq2Uqi5wpsjdVR6rkodaIWjFzHjHezyi8a4QoQgUEI3+ZziyNZ
gFPdulURMn5S/0L84VYw6bR8yJkzpMPwkPMEE5iD3MlJxbfUvV07ug1hW5/prbtT2LDaiySKLOq6
cwuO7JjBKaQoNMh1UEfVluFu1ODM0jCXHybSZxIPl0GqPqK7VHlEfbt8HUW9QJfSwS5ulytV77YU
uFcQHtcTevL5Vi4gameXN/TtPJuy2TtBUV+FB+LuWpNzv/JgtWUU6XWcyPW9ylttGSMgHfjOeVkc
NoX3sppU9H/zq7yokljU1PghUV1swCP/WRgYy+uiRyq6U5IuadY3ztrT3tFckzSv/ebsm2BKyTNz
sk1pNxV5yMvPm1l4gSR9A9EhLhhYdgEFark2z2+J6x0CuRhoqcZ8llq21oQ+FeVo6BVpXG1Xp57x
yo8IoL7sPIp71MaTJSPbWTK7BQoDsVN6cd47o5v8Wy+CvAFqUaWyQ8N2eVyDPpXIIpem7oGLpLQd
ocAUm1/yk21eQd6pn32oaaQKZW5kc3RyZWFtCmVuZG9iagoKNjk1IDAgb2JqCjEwODMKZW5kb2Jq
Cgo3MjAgMCBvYmoKPDwvTGVuZ3RoIDcyMSAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFt
CniclVfZiuQ2FH33V+h5oF26WryAMZSrXJCQgQwUzEPIQ5JJZ2GSkGYg5O9zF0mWVNXuFA1uL9Ld
zrnnqnQL6p/mcHz58tvzDz99Ucv7U/O30kq32gyqh7Ede1BO+1Y7o15+bj6+U382oOjv5ZdGy6de
/dHIWhNffFbxBZmiZ53dpUW/Ns/vmhWd5pZMtoAennCjBt6Y3ZY2Dt9O0+H96auz0vO8nEMW+IdR
LtfGDLodwSrrXKu7QV0/qcMFHw2as+r6/N1k3PwEk17pCnrGW0sXjxcwdHeiy5kuji5m/v76dbNe
mw8c/+GbHz/HCOBeBOjYa6fMOLQdZsABODW0gDlcnycN8/X33Nzy16d/o0Fzz2CPpYAODbrWdC5m
BANnY7v5yUxGz2bSix4oKdfTg50tJRXvBl7m8RZoraH8YMA08dtFd/TvRIvPeIGB8l5xoS12L7Tx
Qi4s3cpbrI+VbfpI1oFfyifai044KrHsweNbipVcxFXs11K9i3DBzmGfXsQumBQSR3uq8jNs0Kfv
N8Fj4K+iaffQ7Fxr/fAYmm4PzY6aBCo04TLDGIpRpczZvppXAkWwk8Je7ChVLUrr0l6E3RTPNkHp
klMGD3i7Ty839AKewpVFLCx3KGeRchIb2y8pxyxZZxBu/k/K8arjHCvkSr6HrBLY90GwYzt0fd1S
wUrM88YoBbZyYlmxXMliislaq2M3ULcBLWTrXE+p2SmlJc9dhn30FaMwUmU4ckPZCj2XAlqyCiXk
NsfBHaEBKXZ5zLuXI6NwuMDk1GROJTkoqHRK/Z7vjEryJho4R3qU7lfROBfGuDJZEpUgxETxs2f+
cC+YQug2zt1mz7x1N2BIE3aRdqkqwaO1r+uL39EXGJCJun9MX7odfYHBt65mtkFMukAxlxfJpDLH
8tzrxsf1OIkuBCysT3htJINzSTKxnffORrvgt5xQkRsphxskozDxm21A1uTamLElJpnEQwNMmVMv
cUdW1INNJt6bvIfOtxb/V7wf5tBW7JB1tkCM0ZB+znTpUkjSplAxpWGDbVPMCqwkuTxO5QorUye2
3Yi+nUCx8JkDai3IYjpGIdnEKEW7iPw7ka3zhvC5hvHOcMqGnbHJdd7LcgLZqFRo9JvAON9CV8/o
agziyPYm8ctEdZAgt9nt4hiMbBOapzFqgwWbT5ucvCmhgmP1gDR0eDW41GYFu+0nt+wJVb8nVOBb
47vHhGrYEyqgA/tYK5XAucjEKysRKnYMB9+tr40rDq0+nQ22tt5QCTMwo0PGS5MGasYjdLOwPTnT
BCGSgprVogT0qedl/eztlHNP7FqIzSVfYcdrANucqFVYN2EMpM5G34Kw71N51G1/89vhkTQtqU2e
C/S2Zz3kKLO1UoIkQHn3VQfJDLkj/yYx+Yc1OyLKwaJk7Af1H4EXNDAKZW5kc3RyZWFtCmVuZG9i
agoKNzIxIDAgb2JqCjEwNjYKZW5kb2JqCgo3MzkgMCBvYmoKPDwvTGVuZ3RoIDc0MCAwIFIvRmls
dGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVbfa9wwDH7PX+HnQn2WZCc2HIFL7gobK6xwsIex
h23d7Qfdxkph7L+fLceOk+WyHoEQK5IsfZ8kW0kQv6vN7vHp6+n9xyfR3fbVL6GEkgqtaMBJ14DQ
ykilUTx+qt5ciR8ViPA8fq5U/NWI71XUxSR4EEkQXIW1Kr6y0pfqdFUd/KalJywUwuLaGypgw+Jz
6mPzervd3PYv9kK1bbcfsvCPj7I7VuiDMZoEaS1VbcXxXmxu/BK9OxLH09st6vYatmha2CoKr314
Hdpr3ILKwg4oSIYFmWCj6rDo/QsAMHzr8ML23fFldThWd5zg5tWHhxQiLIXoIzNKC3RW1j5FjlAL
K8EneTxtFbTHb6W77uf9n+QQlxw2HiuovUMtsdYpZbCcLtQthhw5F9oqG9JC7fNPEv53CFmpLAvp
MxRe0PkvYnV21fsXQMDDY4ABAwoYJNfsDsywaKILPe6882sMkBcRDLuxpQVW7oL6KOP9ogegwciH
Q9F1+I0xRm8dQh4M+pz7mB/7hR0Ho1J+2JwnkRYxt9KgwNpIa13iEEhqR0wirpColxyC8gVL4F2i
bJybsYicIQMeky2o6wMCJRU5SYpY6izJtU8T7j0wSdANNAW8VMMg0tRHtBlhH4ukUCriMaNlPasF
bjGIFHNX5cyIzvNhVvggLTXqC/moV/kgkERm3lU3raGUemoOGpuDuaJ/GFFFzQ8d4oJk11KBI1hW
Gfy4Fub0lmjxtrvstk8VkRrvLIzNCowKpasvRNGuoQjOSdvMixo5QU5G57jrscb3AQU9LRplJlj9
t5IGq4AwuDinAkY6aozTMDkNe44dkRhxkU7M2/JWuxaaYZSolYJ155GGBjy45kKoYfHoy1jXVvoz
c16x/YikLzCG1pY46YjQBOSL6nRW8ZcUaZ4IB+xyLxVjCfshfBYxz5GGbM9tQmok9VkGOduyKzOP
Z+DVVqKBGbzU86aHsGUHNkSS3VM+MdUh1t3aeIO1SwOAlaTryy4NsHZrAPBpqfl5Q3VxPg6za2wP
BtZlOPt8QDzjhpELiOtj/D8txTQwyWSlrijKpOCRtGmX3IesSwtxFkMEkytfttwKKp+kkaCykoIy
32jOVMaApCNpzfz6RTeRfhXvObvxHMw3knTVonyUjNGahbM9d2u6D6VT/vm3u8HSpkm2qJAkdUbc
RKSGIZ2mwYSPdFxlpG/I5eh0Ef6k/u/EX8cQvXMKZW5kc3RyZWFtCmVuZG9iagoKNzQwIDAgb2Jq
CjkxNgplbmRvYmoKCjc2NSAwIG9iago8PC9MZW5ndGggNzY2IDAgUi9GaWx0ZXIvRmxhdGVEZWNv
ZGU+PgpzdHJlYW0KeJy1V12r4zYQffev0PNCHM2MJNsQDHGcQEsXuhDYh9KHbre3H2xbelko/feV
ZvRl38S3KSyBYCvSaObMmTMT3YL6u9kfnz//+vTDj5/V9PbU/KW00q3GXnUwtEMHymjbaoPq+afm
/Rv1RwMqfJ5/brT81KnfG9mLaeGTSgvBVHjX1VPe9Evz9KY5+0trS1htCC87f1ADH6welzb23x4O
+7enr2alx3GaYxT+472crg32tkVERca02vXq+lHtL6QIvTlS16fvDngcd3DAmb/tCAdN4WvS57Ci
XXg5pS/ox++vXzfna/OOvd9/8+FTuh9u3e+vtdooHPrWef/5eqP6FnwE16eDhvH6W21u+vPjP8kg
3jLYeSDAeYOmRWdSPNCnWNCH4sOgEAZxGD4ECt5j+CIfgu7HHYZQfFBzWDZhgw1PPcfMj7Mcp/B8
CaFDOCVW+9F23ndA/yMwfnryL8QvvGlpi/K9EHxkuBltinsnoOQN+wWGVxlxcZejkDviChKHMfNS
7Z25nyPaypEzLdn+sRyZrRy5QHxY5QjcmBAK8cVYXiYOSMfQjGRCNvKOCXUGO0YuaB8T2pJYPmK6
DHMGHOyNZGDhzAiHBW3wQLpcI9k5ChN25Zjwir1iB2WpJoE2hRkBh5o0KwZRzcvV2ZB6ipTg1DOO
Oeu3s0FD27tulY3kOseZKcY4LoAVZvqlu9SyW9TSXhLdg+XvtqjlNdDh/yv/OpOp1Eth76IeCA4p
p5Rr0pRtnZAqrkB3IC+RejgwYYW1J4Yt3oVjnVgRD8YYhqIvbJpqMhaVKHdXInHK9ywkQBib6gt0
yq/QdmWOBU4W54pp24QC31rMmk+Zji6XyJy9pCRvWUAnZjMWR/HMG3CWfecY1KlEjpViZ/8DXgK7
lTMQU35PXVKJ45IadJ/f3Qa/wdrWpm70X/ndb/AbLPi2vQaXXJQWLNKSFc8j30sWY99K4OCFN0ZV
PRYB4/ZiY7aS3JzrQrjQkH8wC/bGZFjJqa7uXFz9QD8uy4kiwUAPk8iS8ODMV+HwOjkJ/JBEa3YO
sYOm5HucbSWumFl6LBLNpdbXaki2ArrEVunzGXu6iIRUpcmVcKO34HCvEaYmeV53ow2iDltE1dCC
e7DHw83JMgI9uLandY/Hi5TWVHAWjlYSNd2Uupccq0Y0OdDnTZjEW3K0MF4OSCs+VtfkNCcxvKGY
1RBYZKua63JpJOfutvdI2lcp27nWdW6NJMXaWAQpGlnQysQSnY9hHpccKvIeHaKls8LkaSEmc2Xu
tdkStv4AGNd23YMDAGz9ATDYGvpC87/JqZvuDt8RLB5RadXjhGAVdhU1ZfL0BqbUsirWL6bJuyPZ
gqZdEfe7g8haafN8k8aPFzVn892FKXFaf2XKxBa74eaQOVelsqjoqBaV0CYiT1BGMcyNJoVIXabl
XBXz4o+FyEPEHE4LdaDc/aNoU1eAwFNJcRwAfd1YLhIisxqE36l/ATGMieoKZW5kc3RyZWFtCmVu
ZG9iagoKNzY2IDAgb2JqCjExMTUKZW5kb2JqCgo3OTAgMCBvYmoKPDwvTGVuZ3RoIDc5MSAwIFIv
RmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVfbaiQ3EH3vr9CzwRpV6dLdMDRMzwUSYohh
IA8hD9n1epPFu0uMIeTvo7tKck/PDoaxRpejKtU5VTWCA/u32+xe3/5+/vPjG5sf9t0/TDDBBQ6s
h5GPPTAlNBcK2eun7rc79q0D5v5eP3ciLPXsaxf2Ypp4YWnCQbnvgozypr+657vuaC+lSEg2uC/3
9qAAf5AMa4zNr9vt5mH/04GJaZoP0Qv7Z62cz50E4CMqJpXiwgzs/MQ2J8kkWjjJzs+/b1FO97gV
szATbIWyHwAwTH+cf+6O5+7RG7n55cNLugaWrrHoWiiG48CNNdPfotjAwRp6ft4KmM5fKNz8/em/
BIhLgL31F4wFVByNSmbD0JosncloTXYz1mzrwuAXtVs62CUh0yY/Ah03yLxBuZF0KHEt4jt0GAK0
FNN9wgYke5MN0l2ZYOd0qdwWRAt28Ij+iaMNWO2CDC8w4PgTuwl69+mOlEuDGXkWqAEFM9zhYATW
y/TyHO3lKPTKRnJso6CwXOgf8ug8E+QVvFkwAjbmzC5O+4k6InbBphSr8CAecVTgcWb3qYA4XHx7
F9+wO0THoYgTAXfkiCHwpkScyoo54ozOBMxWh0kzIZ08Wq9ljAX6oUhrYZcK778QFFU915U4aMlH
rds4oCdooGkYy8vylWvyRZsk9G3qVWvqtZmwB3m7eiWVkm7Jeo9UuFSxDSZscY8nYbKePEGGGEOZ
I+/iFK70W/f5eMgKfugSAC4llwX1zRfSCElODVtxZ2fwgLrQD2QQ/wDqx9gB48i1HprXToy2F54m
iB6qlAQAiFeYvdqFpYsk0iskAjNyo+VtLDIrLAKjubT/a7/kkJ7fyY5GsnAFxqCxvETdTRxLp5aS
5w9Q1ctd7NqkV8Eliqg+TdrLgWbOfR7hAtsvmYF7ixLk31Qg2V+ni9IcDLTiPJaEF7z09NU+AWvy
PrmYWD3JzKRwmJKqVKq2OFcVt3IiLsp+4QTJwuvPcir5v7xpqt9VyW4CtmwTfVi3/UB0lGPvy0BK
IokWVwOBtp9zGql1e8ljGSeykK85vVYN+hUhj5Yft/Zyw4qOR8H7d62cOoanuyW+tPiXBwglOKup
ImHTDJIargIJTdUM2qqB5VxL2aE10KTZ3NU5VZSCsVQbSFuBpGM8NPlMNQaYnMl2qSu97KTubbAs
KWHJEF8G14lp/73LD8kJ2rxFohPvm5427co9LVY9LX3Ro8/roZlBE5uZleqbr1Hx0dIjQvOIaorC
B+3zWMW0tX66VJDFwMmcDD1c61tbAkjiz51N1fKmOPt0koxZj5MWXOm2PqYkV7EYVQySKbLJuRn3
XhJkbdk22kTFvsWdJhQPrdUhy8IjzFWYc1/vOnVFtNqKvlFyAW1nCefgNGmaTkipzxWm6suriEOk
afk1cy0AUnAp3inF1AHIPwyXSnv1U7PJ14/sf7DNltkKZW5kc3RyZWFtCmVuZG9iagoKNzkxIDAg
b2JqCjEwODcKZW5kb2JqCgo4MDkgMCBvYmoKPDwvTGVuZ3RoIDgxMCAwIFIvRmlsdGVyL0ZsYXRl
RGVjb2RlPj4Kc3RyZWFtCniclVfbiuQ2EH33V+h5Yd0qlSTb0Bjal4aELGShIQ8hD9ndTC5sEjIs
hPx9pNKt5LZ7dhiYbstSqS7nnKqWLYh/m9Pl+cvvTz9//CKmd3Pzj5BCtlL1ooOhHToQWppWaiWe
f2l+eCP+akD4v+dfGxledeLPJuxVaeGzSAvelH+W7Fve9Fvz9KZZ3aXckmIb/MNbd1ACHWRfaxun
78/n07v5m0XIcZyWGIX7c15OtwYB2kFpgVq30vbi9kmcrihQOXMobk8/nhWOb9VZTtKOcJba/QOA
fvzp9m2z3pr35OTpuw+f0zWwd42zbqQWauhb69ykW7ToW3CO3p7OEsbbH9zc9Pen/5JBtWewc/GC
dQZ1q6xObkNPLsN1NHiWOHq/0fgPHNE77/75R1Dh0b2gb70PEdDHRos6L7LoowEA2tyH9+G8t+5W
Vm9AegPLGO93j+lIPIC1oeSe8p5lp32qw3a/6NxNiysZjL71YMgv5W2oIR+CHGi4bapt0UptKJjW
2dbiFtQgWZn3099pV8Jhk35UZJTfb0sWljE5qHO8LN97SVAX8im9C/6Hisx+53yMRnyERveppH4d
GvUjNOLQ9rbbpENZqsmS0OCcr9Hog4QY9PJ18IsF3eyucO+y57AFR9BHD11myAM4IlKSO+HFtdwc
+MJAPEXHdUb8eg93m/ALF9qsEiqIKAEIlx0gIIeP2W54AZZOYTvAbR1WvDK6kM2SwKIDRzFrxp6Q
7ksozliKhzHnC8NmSo4qpbapENGEyvFlOZEXJlvpICtoTkrcgHhMAvOABNA7zMrudSSwD0gAvWn1
HQc0CSsVlWoeZHIOSTedu46iABIxm6kyl5zdydkRN9SsrmSi0uaCRjIKfZHYDFEm4M6p3ucg4BsP
HSuyzXRW5R2YQRapMVRgYZLIARiNkYXi3KUcKJR+iQpgTYvuc1OOTepUTp1XjZA9IkjFkugkT92S
Usk5Hw5NcCdjdm+3vFZtNzIipyQxbmHlJ72wmx6xpqCYwhHRu3RXzDzznzVfsrQZFVI1VN35fW80
+Qwy4FQqFaX9hfpo04KFTX0CGEIG0SUkASyrfM5FiKDAOrhYa3zweAab8QSbwFhNYvqu3NJ0lxOq
uznszjRKRXAEGKReUasaDpwEyd3QhR7oWfdIz8C0ytjX6Vn/SM9AtgNsZxzo6glw2TCUyQ2lEiKJ
kTWG3OrWrHIewxiBZWvcqRltGe4mGv1CshIIIw98La/ULXxKC/zVWbNhd1ILO8s4sc/eA6f8rJB8
j5eum0vJC83gO1GgS4FcUehMLPgqcRtk290N/0fahh6TWhbH4Eq+ciWgKsbOGipYRpOlRGDKaJ81
f6+zpNDdt7mEHgPVMVy87xJ1N6mno5oySSThmCzDA7JY2fZmeB1XYPd3ZKyIsa1RalsRleenmMC1
mr2IGdiVruLDx+6uubAWEKHLBHrJkxKHWuFlGsTS/BHkrUj2tfx68j2a/eQK1BnCyJYlNm5C1hj3
h3KcKIzoOHVYRUwh61BOx19JJsLLmahq+l78DygBlWkKZW5kc3RyZWFtCmVuZG9iagoKODEwIDAg
b2JqCjExNTYKZW5kb2JqCgo4MzEgMCBvYmoKPDwvTGVuZ3RoIDgzMiAwIFIvRmlsdGVyL0ZsYXRl
RGVjb2RlPj4Kc3RyZWFtCnicnVbbattAEH3XV+xzIfLOzEpagRFYtgwtDTRg6EPpQ9s0vZC2NARK
/76zs1c5ikKKQZZWs3M5c+asdA3qT7XZ3d1/u/nw6V6Nl/vqt9JK1xqt6qCv+w6U0U2tDaq7z9Xb
F+pnBcr97r5U2r/q1I/K22JcuFVxwblyz7q4S0Zfq5sX1cRBS09YGLiHC96oQTYWt3Mfmzfb7eZy
//Kg9DCMh1AF/zjL8VSh6WrdWkXGyP/pWm2OpAjZHanTzbstdMMFbIHcVdPAF8MXAG1xlDfarbXx
hb+I3d7Z2eH96VU1naorKWfz+uNtTAiWEuI8Gm0U9rZuuSDJxyhbA5d0utlqGE7fS3fjr+u/0SEu
OewYGWjZoamxNbFAsKE4IE6WXNrIRUmRdrjALReHrjZ0tcn7YOTN9+6tHaI1WH1IBo27k2Vve3CL
Ji92AcrgFffiCJvoHHpxBjuxPqRIE6dDzoRXxxQaUcycJ5d/DDrqEdB5b9ILDtTxqvX9wrAK/eMd
orUOtaamxj6vQ2atQ62bAzjrEBqPH2X8jtQvdETurDCyJ8FyAbnCDrc0JZ9IAoXAbksYs1ec9C51
s0l3o7c7Bu5ITJMa3kauEJHO5CookQsYfZJpiw8XeSSbSnumnM+LUw/bfOrgzaVyFzSWCxKvyfF0
MZvLHaG+tm13PjMJXkliGiDMyTlgZWIiGUUuuDAtuONnPAhjIyQUwvjkxwAHzropzhKZ54Eo1BtH
pikSfZT2zRrtNat3+0xhatdoz3Ld4rkwFdyk1uEYAIDGA5opil1R7eT1ZIo6ATQEOjClEs4Fk0sN
631X5MG4cMaphHENMHn8niOZkeRLkun1zjUFHuGM9+0n55zED3XQw+TV0N3zFfO+pZrXyQ+2qc1D
7hcYkSDjciHn1risGxhmwlKiRTNmRrTIi/ETB4wfihHyMeIlywt5Hq8SLVijTpDSeBYU4b2Gkuzi
PSvB5tMYhs2LqI6US5ZPoN02NfH/2RT8z/R2K9MLxE195vDaleFlmPhbi87PrHwkBxZMs+8A5hCD
1aXmSknUZXIG0PLBN5ZztnQahDij37NLZ0g4j5I+5i+IYzH4LiTOx8n0XllxpqxZtvMYnH/fkBDb
hsTRfQXiUbLPw45ezYWPDZOMvItZT6/UP09zfQoKZW5kc3RyZWFtCmVuZG9iagoKODMyIDAgb2Jq
Cjg2OAplbmRvYmoKCjg1MCAwIG9iago8PC9MZW5ndGggODUxIDAgUi9GaWx0ZXIvRmxhdGVEZWNv
ZGU+PgpzdHJlYW0KeJyVVdtq3DAQffdX6Dmw2rlZksEY1l4vtDTQwEIfSh/aptsLaUtDoPTvO5Lv
bnabYJBlaTw6Z84ZGyya39l2d//w9fT+44Opr5vslwEDFigYj4UtPBqB3IKQuf+UvbkyPzI08br/
nEG35c33rIulYeHODAsxVXyG2WwM+pKdrrJWD51nollAfNjoi4Dpxdl0mWP7uiy3182LvYGqqvc9
C70UZX3MyLMVIMMiFlwwx1uzPbBh0nRsjqe3JbpqgyVwhSVymu51Cu24Sg03FbsS6vSow7vjy6w9
ZjeJwPbVh7sBAj4GQU/OQQwVwTqlkBCICRaVxPFUAlbHb/N09c/bP0NCeiyh11qg04RiyclACUOi
I0VFJZIOUEe0odpQB51LaOJynCGDiwsS55hCUiDu4kgQ92pdyr3igxrz85z5EmfH1vnwPM5yibOL
XsIVZ5VQoUskd4gyUmSV9wIuWaZI7GVOL6X9kT+t4nmoxMh/5S9B67lYAvMdMD0CCBiFPLRwQAHW
mSBBq/c9MhZw0H3GPdZIemfIx8Isz8mJbL4+ZyxA4pJg93p7Bb5YUZyJ1WSMtjNCOK9sfklZimLy
85R1l5QlsSJuRYxd70fqVFDdSNIYmxRDL2Ukizk1WsbEUnqVO1k1sOg8EXA/SotFrELqiFS9NiaE
oUF6O/ipsn2+OqWA5BOu5n2DA86p7fzSg0uP0VM9JkGsJ1mWaPCYOolpp5YQdc9uVfxJS39BSyzU
W/TMLg0XtMQQbNA/wH+1VLv3PYg511M16yRaL+XqC/VkSf5t5UmcM5XWXrVhTWCqNO2wQad9SrFz
tZ9JD3m8X4X1y3emFBHAvLA35i/RZqMnCmVuZHN0cmVhbQplbmRvYmoKCjg1MSAwIG9iago2MzkK
ZW5kb2JqCgo4NjkgMCBvYmoKPDwvTGVuZ3RoIDg3MCAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4K
c3RyZWFtCniclVZdq9w2EH33r9Bz4Ho1M5IsgzGs7V1ISSCBhTyUPrS9vW1DPsglEPLvOxpZH+vu
Gi4GIcuj0cyZozPWLagfzeH4/P3fp9///K6mt3PzTWmlW41eddC3fQfKaNtqg+r5r+bDK/WlARWe
578bHT916nMTbTEtfFJpIbgK77qaZaN/mqdXzYkPrT1hZRBeHnijBtlYTa99HN4Nw+Ht/HpRehyn
Zc2CH45yujTQdS05o8iYVjuvLo/qcCZFyO5IXZ5+HcCNDzBoGmEAkunCU33KqzjTPJIb9CSvcfBo
ggEuYQQvI0QnFmeYZMGMaYDxt8svzenSvJekD2/++JTChlthc7RWG4W9bx2nLVEb5VvgxC9Pg4bx
8rF2N319/Jkc4i2HHeMHjh2aFhmOFQbwEYJOmxFDlA886pEGWHjQc17UXj7NnA2vi/FxxYo/YtiB
YTZpF+ZxT/ABwVaHXeC1DZupGLhxtYoneQFWDjqGEUMk7FKWZLN4A2vCIaslpoCi92gri5ijNFUO
tR+a5Fjs7leHboLpW4sKnW2971NxgFrTk1QHd6pjbrJUMzsJ2CW2Xd9vy3MK6bicI12/ThFSH/Gj
hPcdJJmM80ptWTLRI4Ya7IJKfpSS8VFWHMinJRfufB9EuwMimdageSGIbhdEgpbIbkDU3RYacsIH
ymlByT1iGu0Qxc7kdAu7YxXmRGOm+JLhnMK9x3ykzAR/PMktQnvtq9wUiUa8kq4uXz8mJsNUMxo6
oBiuKVn4spVPZF2jGFGMuDbkxfqYGifmFKxCtiYk34jul7rbKbXGtncvrLTfqzT0feu7TaHRsIKt
JU11rFh8lhTARiWbcFkvxVRfCkq15jYAQ/RXEeI0rpigrdUOhySPhRm0i1a/o/3guI068zLth5tN
cBV/YMFyuMWLhN6ZjZAz+R8bV4KcA2P6/LkISLlaOGVwMM9MBmij9uKxahTHjGfqLFKUJVdhKk2J
Q7FjKVzRR7PTcfdaLlD85XkZ7Hs9F4Ie+a0ehTuJtVRU0lKoBjZLDpQGlnFYr/dZvk5XrLxCCpcI
dobkOtDQdvruOtAuBsqlhBPryxEAnDYQngmPwJeZGyfgEWcOKCGzQdTyv1d3J39a00jCUrIqNIot
SCT1FH9L6kbuNsY28av8INzJNxVGs3z0ftsoMBMaM47HKrqi43iWay5Ujlq5/j4x7iYzV/5D9A0l
eK/+Aw2GgLgKZW5kc3RyZWFtCmVuZG9iagoKODcwIDAgb2JqCjkxMwplbmRvYmoKCjg5NSAwIG9i
ago8PC9MZW5ndGggODk2IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJytVlmL2zAQ
fvev0HNhlTlkWYJgiJ0EWrrQhUAfSh/abtODbUuXhdJ/39FlO8YJGygBYSuj0XyHNAaN6k+12jw+
fTt++PSkutu++q1AgQZyqkGvfYPKQK3BkHr8XL19oX5WqMLv8UsF6a9G/ahSLJWJB1UmQqrwDpOn
IehrdXxR7WTTaSaaBISXG1kIGBdOHk9zrN6s16vb/uVWQdt224xCflJld6jQOQn2io3RYJ063KvV
nhWTpGN1OL5bo21vcA3c4ho5Pm7lEXbDLPXct2zX0MXXNDiowwIThpq7EEw9dmkiDti+P7yqdofq
LuJcvf74UCrFpUqlwBqMIu+0FaSxUKOcRsF6OK4B28P3abru1/3fkpCWEjZCGVpJaDRZU5CjS6gb
MC2FKm9khJbXuJUB+mGSmvMAeHE/p2tSZGvtnC/1I2vjOQKgCwDMonYgmjFKStKN9zMEZKiWYkUO
qd1DqB43oXB5lGEfVLIZDaZJDJAFt+B0MbIOjyGBaJtmGnnNMaLnvq2HAHRpj1kO2oVdRX0I8130
RuS0kSV12jaRHEchWHLQkIOmxdC0mGDHsnXERTAs2IRQm0JjSCwqLiKKycyInEthmZ5AitgbM3WO
+bzQ9QWh2WhD5kqh7UWhGTVzPRMaMFHTtRkMugDGCQujehFkdPAuujmLMCEuK0wcSeiKPaIM2fHh
oCOWPzblPMAFgpoLBAFpb6/kx13iB73Xrvkv9PBzOEoWG60kBO0jNxQOThzSYctWznsPB8OUqOz9
fAaLX20h+JID/XmCsUHhtL6SYVzuE4Vi67Q0mPldUw7VACXz2iHNMJ7jVIrlotAkkRB2vk0s9omM
3UCs90rsi41iwM5We4PzTmHTEUiocKr4eLmdSMsTlGf5kJ65xGgyL+bwYjz0kedkm/HKjIzOL+SY
GBuWts5FuHQzDoacVH1yoT/LkLjY/HL3RrCa6/q67o2L3S+3b5RbxHo3N2Qyz3DkcNJnfLBZPzvw
0s1Hd9LpsR47BEzceFoNCTBP/rScJpUjyYFpA4RGWtZGrqOCdZbDs+Z5jukXyXZmiXwB0dDcBh+N
9daNkEtBc9qP/Y/6QIptafwGKBYzp9LeqX8VMmhBCmVuZHN0cmVhbQplbmRvYmoKCjg5NiAwIG9i
ago4MzcKZW5kb2JqCgo5MjQgMCBvYmoKPDwvTGVuZ3RoIDkyNSAwIFIvRmlsdGVyL0ZsYXRlRGVj
b2RlPj4Kc3RyZWFtCnicbZBNS8RADIbv+RU5Lzh9MzPd6UAp2C9QWHCh4GHxIGpVUMGy4N83Y7da
RXJI3nw8mQyM8Adl59Pxeby9O3K9a+idwTCwBQeJJgZhj9zAW54e6HrDbyScbHokzKXArzT32iXx
wksioZLGKvpueqJxQ50uXZPsqiGJMx2EfA2uwt+M7Koss11z0TKqqm5PV6jpK+tBmcYWji0KE/Sg
4Z6zPmcfTIyqxkMpnQic1JWUiNJIjoAIm2SYnYPXrJNefSuFplQulepmuKRuoP2fvXa7NcGz5LkR
F//Ziz6BJDEgJ9wPLP3Mnj8BOetY2wplbmRzdHJlYW0KZW5kb2JqCgo5MjUgMCBvYmoKMjM3CmVu
ZG9iagoKOTI5IDAgb2JqCjw8L0xlbmd0aCA5MzAgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0
cmVhbQp4nL2US4vcMAzH7/4UPi+MR/I7EAJ5zEJLF7oQ2EPpoe12+mDb0mGh9NtXkidpZpgG5lIC
RlZkWb+/LYNB/Utt28Pzl/27D8+6u+vVTw0aDNisE1amSqg9BAPe6sNH9XCjvyvU/B0+KSi/kv6m
SqydHE96cnAqnsPCmoM+q/2N2tGmy0x2EcCTDS0ElIUL8zTH9nVdb+/6F4OGpumGIwV9VGU3KrRg
ELx23huIWY+PenvrtLOUzulx/6bGvtlgjb7BGnjAFga2Lds8QFccrthuCoRcfosdiqODOC9bBrrT
PSBDYs8U83Z8qXajuhdFtq/eP01MeImJUAIh2SqbSJoIkteZOBMR1YDN+HWZrvvx+HtKaC8lTCQu
RkrojY1+0giz6AOJqrRcpaO6m42trScSy5SOBdkVWFdj/jeIWwOJ3riQrwPxayCRbxyegWBHFRPI
wGX3bIWCFBJvxcfRMgXwMMzIQilTsW4FVRyIrEaZr5xhWEN3dD2p4qvQ4xq6DSal6hy9lxMTIGyl
6ploOsbINs7/0BcsK4cuN9xOx1w82Mn4V59Z07WLkFbUwCqYnNx1auQVNbBCE1z8H2rYltrA8kNh
5VpJz3QoDS4ZciMvAHvZWpGoWpMooonuyqbHi0/jpFFIhjr/XKO0uPNz/x8V6ec22km3HGHlYeOA
S512Qnuv/wDL4WxFCmVuZHN0cmVhbQplbmRvYmoKCjkzMCAwIG9iago1MjAKZW5kb2JqCgo5NTEg
MCBvYmoKPDwvTGVuZ3RoIDk1MiAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVbb
qts6EH33V+i5EEUzo5shGOI4G3pooYVAH0ofejn7XOhp6aZQ+vdHGl0sezuBYDCyPBqtNWtmJCVB
/Or2x6ef/zx+/PxTjK9P3Q+hhJIKvXDQy96B0MpIpVE8/dm9eyG+dSDi8/RXp9IvJ/7rki2Wia+i
TERX8Vs1o2r0d/f4ojuHTVtP2BjEj11YqIAXNsOlj/2bw2H/+vRyEmoYximzCE9AOV46tF6S14K0
lsp6cfki9g8kCIM7EpfH9wflhh0cFA3hpePLo4Y4AIxfozoPO0z/wQ8fLn9050v3lrHvX336WnaH
rd3DpkZpgb2XNqDnzbXwEgL+y+NBwXD5t3U3fv/yuzjELYcuhAFscKglWl3YgM9MAkqMLCiwiKhp
DN9owjd4AOYxDkw2jGycNsmQKWOkTIlyNgqMIftK/9gKckTYtonJEispK501S7CmgIUTO+nZtUaF
EGVAdo0Y8fF+xFZ22FG1jUKoyIuyLc22lGczRBfR8kbVMRtjdpyNF1CuS0y3JLZakvH3SaxvSWxj
1cBKYvIJe3idEuUAmVPXFuVbdZJwD4PJpqw+9OnnFCfOJZopa6aiOXg2Qnc9HGYTvZcGBRJIq22J
BpDUPXE48EY47JZDUKFwCQRi8GzcOh4ucZlT3mAd4jFmycT539CnhiWvG69zdDc4KiXB450c/S2O
0DupelxxTIIyVC5SVj7os6araiywKM81lSqbmZ6WJY1nbgI8nV7EroMBKXaZuohJ7WKxNqWP3twd
T8EMNRf0xMnUJGVpOUkF0zSq5L0KhScwiXAWsG7FNKboK5YyHJkl70B0Xcv+upYQTgkH7k4tYfPE
qWJaIy2uE5a5JFbnVGZrGcFWuR6454YY0aq47TycqvWNzgWbp1PmTk6ioXu5bx5PlTtpCZbWiexz
QnIDPkbMNRolEdp8NrUj0ZwYJUlqkeuifusYT8V1jB8f8Lb2PN0s3jr25urCc7oVzCZ+jj9rc2zK
xAVjHSHoJlfHhHWuLkpIZjVv5SxsnjlZuHAPct7fK9zmoVOFUyht79fC2XrqjLUBzEdRaqUwR0JN
7Y3Dr+4dzw6u5bXCs5IcV1TLLVNGrBoRjGkGa5dvmlINdbb1DE2dQ+nWOZ0Ssb3R7EpXyQjCFLgD
hbYGNUMS3NDqrlx/Skg9SqOfXdUgta6mtZtnVc0cpxlYSm+drjCLlHkr/gffXZFgCmVuZHN0cmVh
bQplbmRvYmoKCjk1MiAwIG9iago5MDgKZW5kb2JqCgo5ODAgMCBvYmoKPDwvTGVuZ3RoIDk4MSAw
IFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicxVfLquQ2EN37K7QeaLdKD1sGY7DbNiRk
IAMNswhZJJncPJgk5DIQ8vdRqVSSrNvTPXcVGvyQ5FLVqVNH1bIF8U9znp8//fb0w0+fxPL20vwt
pJCtVE70MLRDD8JI20qjxPPPzfs34s8GBP6ef2kkTfXij4bWKh74KHgATeG7LJ7Sol+bpzfN5jct
LaliAb6c/IcSwofF49HG+dtxPL+9fLUKOU3LGqPwP+/lcm0UyLZ3RmhjWtk5cf0gzrsWWnlzWlyf
vhthmU4wgppglCteLnixeHGT7UYJcvYvIPESFhi85PXhaZ9OapSaBwHo2UzfX79utmvzLgR7/ubH
j+wu3HLXe2mlEWpwbefDDd4a4VrwAV+fvC/T9ffS3PLXh3/ZoLplsPe4QecNmlZ1hsMHR6Hvk9Uj
mMm7vshu0giDvziJNx9BeA43wPhgwDvgjEJ0oB+1RwuAbPiLRvj8kyP4/Ddqgw2HVx4mU2yCoFeI
lN9FZxP+grvHjVccDnMW58IrfqYRZdUHi2E7sD4OhfkACkDLPElhbXLzk4fxmT0kN9BydHPNIeEX
HFXYRFve6aWxLoUrSycSG25nqTc+00OVJWW9BbVGJNAcoYSQ541hNMvnyabvkc3flTSvI5u5RzY9
tK7rqzDMFvIRQTIEEqYjhBSzdyEyhizMMQExb4GaTFNIcxkLQoEIEqlL/OIC5xwHetmwItFU7npI
HDNsorCrLTOKti0KYZlySSDjDSQW0dxW+KPGQ6HlkYLemiLVafQhcbzk9qArxLWbYqEFCpFXrnA/
Vj6J1gG1SO5Qdz4/hyCjnymVBeAJzhuYx+AUyqvaA3pok5RjYOtUM662R4ko+RIVik13nNNj3rhI
I6I2fWSS/wrVgJVnQc3vUfOTBpFb9mEOYBhaa12VA1IDnUSLfOoNum9qfdJR3NhhgnIwkGlsoCh4
l5C3gW2lQmk1nfzErnfdTV5UDpPKkEsB0Jmdo1S4Ws+o7iiXc7WLWbwCf1Z17B3VgW5oO6tfpzrd
HdWBzrba36sjrveuByXhBGcOLHTaM+QbqfUR/AtVjcWQcxUdFEIfjrSaWCFrlPs5USoKV6rDvOue
rZZJUHNaUTI9BqH1F3L9JhtrRB4S3dgWOqiB/t+kO+7WFf1CtNelT1IhFzMkwXPRO4R2KCNBc0py
s8Ene4hlP54FybJKMTApCreK8ar1odgfQq986yvrkzVLXuwREJtAVj5Eombekm+vbMcYPI26RGf6
7lLQqkB2TsBoonfQCXNrw7JPSfnivFB7FSfdC/2bM3SVN3e66/6O9Ayev69trt0d5Rn8X4wXvTXq
bzh4U6+Qir5ofi5VrFREYFlqeHGV4dsiQVmt4KP6WnNSykyDnuALCJ+VIbBLrqlubeUCWgC0wNQ5
djGnYwCppAvhov8Jh/Nv4kJxuedaj9AsD0vH315qlitSUwtF4XLHfRQlbc//YoK3+tBA8VFpWP0Y
apY7XZQZ+5+VkbUyNkAHgr8T/wE7TmQzCmVuZHN0cmVhbQplbmRvYmoKCjk4MSAwIG9iagoxMTMz
CmVuZG9iagoKOTk5IDAgb2JqCjw8L0xlbmd0aCAxMDAwIDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+
PgpzdHJlYW0KeJyVVlur4zYQfvev0PPCUTQzkixDMNhxAi1d6EJgH0ofurs9vbBt6WGh9N93dJed
xNtDwLGl0Wjmm28uSoL4pztML19+e/7p4xcxvz11fwsllFToRA+DHHoQWhmpNIqXn7v3b8SfHQj/
e/mlU3GrF390URbzwmeRF7wq/62atyL0a/f8pjvzpa0mbAT8xxMfVBAONq9rHYfvj8fD29M3i1Dj
OC/JC/6xlfO1A9dLi0aQ1lJZJ66fxOFCgpDVkbg+/3CE0/gER9AjHJV/wKQW/47+3T/UrKx/h7Be
BZVTxv+RX3XjE/IKzEFZOBa0nPzDRIkfr99252v3Lvh9+O7D52w53LOcDTZKCxx6OWC0WwsngV2/
Ph8VjNffW23zX5/+zfrwnr6eEQTL+rREqzMQ4AIICthSZCOV8w6wT+zNHMwGCK7RSB6J4CdqdfZ7
Kkuxr3w4CCbxqGYa88kgYOJx8ICiR4nSrh7zESAGuFnhZ38jYxLI5FGFZDV6nGltKMwshf52OgYH
o0jcM7dWU5YLelUTsjWWxgzSbsHsI5h8DNlkhWrCQV0QGEIE4ptzwO7HpUfZ9/02LpcMB/s6BRu9
0Sr4jtHTpYA8t+BTImeQMQ3UD2lIezTUKGGA1/FQ7/GQnDRkNv7SpQnKY/yHgcsSrXV8DX5YYMGJ
3wDsw2CQ4oBuNTfRoCFHI/KRymdmYGYbjbeBiKwKOyGg0ctTOXPmjKJGIColDDln6okA0WMjbDmP
WR0M5eYlbkSQ90ieAwU9g+22gTLx5lwATGPIQ4KZHYL5Qg36lYXO7hAMHEqu0Bu7cY7Y4SanMiYp
p6gpE6GENIUGIuxZ1LXlUec8gxXCXg8lGWqhaotWTV7chZeOpO5K6jUPCm+w8oYa39dlFQtbWGYp
MqYQMnSDr+Rlht6itAZeQZmCtou8DjlxWiVD6zRsbbspeasmAjG44VrOYIZmSkHLtjTZ6r9DskX9
dlx1h9wYSrLXdsPrfWqeq7tRpwGjluzzXV1lN/bWRMkoGgxGv1i77v+NBpduPWw7C6Ub96KB51A3
dGyRs24ImhlU23iBegVCCexlNLVNuTIzlDgRPa4b/V7dAJTmtY3J7dUN5SSa7YAEfZkyQha5ttXy
1rJKwDzFJBpjIC88bBLGeXPrTBMz0frHdC9JCDapzp1jb0zYsrzN9UhUW8gZbxiaj5gMcyTfVDIh
Rk01KEy1+5y2g9S6UlXKJB/2Gew4qtt4VENiQqyLxaZuJUemYoDdyFCEP0zvMTRTUVp4mkZFVUJy
uWm9ZdZt67AdnzLYtqq5DUszbdeB9VRjVoe4gPGlloBgIuWkuhkGsWR7Owyu0u2d+A9jDQjiCmVu
ZHN0cmVhbQplbmRvYmoKCjEwMDAgMCBvYmoKMTAxNAplbmRvYmoKCjEwMTggMCBvYmoKPDwvTGVu
Z3RoIDEwMTkgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nK1W22rcMBB991foORDt
jG6WwRjs9S60NNDAQh9KH9qm2wtpS0Og9O87o5u9m7UhEAyybh7NOXNmZJAo/lab/uHx+/Hj50cx
3GyrPwIESFBe1NjIpkZhwEowSjx8qd5diV8VCn4evlYQl2rxs4p7VZ64F3mCTfEYZr2y6Vt1vKp2
dOjckppt4ME1fQgYPpx1T21s3rbt5mb7ahTQdcOYUNBDXg6HCo2T1gttjATnxeFObPZaaEXWtDgc
37e47a6xRdNhC9xgDyP3Ffe5gQEc9zHMTxvBg+WX5lnfXSuaUT0bU2NoyyoMqHkmGvDTeQMNPhxe
V7tDdRvI2Lz5dJ/h4CU4BMOCEarx0hEdAY4RXiIRcji2gN3hx9zc8PvuXzaoLhmsiVd0ZNBI5Uzm
B33kxnWKIVAzBMSMkjBSMxI8zWs6wCNomqARLB7HjbDluZF2E0wGjhhZCgR44BXY8QrwV+EbbXly
jMfmSbBlPKSvd2Ubn0hx0q2u427c5q0mepg8SbbYe2xihDWvqNYUv2hJx6glyNgHyMFB1YXPqecK
I3NuSiQvM1wbilJzzvCWKKNTbaAz9LPbmnu0FJ1L4wJjH6kO6IOPkQ0V2chOjTPikoVFvek1vdFb
gXme3sya3nQjvasvs7GESsdc1JmTIC1TiIoiU2PRWooP2sRZ0miO9kK8lCPX9KmHLnuoITMfdBTT
OvG9YE+rRrrGL0Bek31f9Bs8TzCy9jPsSfuYitmZ/oPKF8Nu18IOjVT+mWXGrYWdSrdT52UGMEg2
RXyIhTeOc9HBxoSA4sCtmRcSm0prlsU8YbZZFyvhRiexPvWsjp5RdaLLz1F1sziqkdod9XuwEN5U
wjQo6FUDe3XOyERwvUIwUlHQvn4ewX6FYOS3fcm0iixr2yWSaa+dJFpKcNJxXYoiT+pgMEVrF2/P
BT+SPRdTK1fbEsTYU7EeLmq5WaPagETzzBKGF/8pMtfaSg8vWsMi2dN9RxKLM4EDl4t7sjvdCdMF
OcXi7J7OmaVhupHjbV0O2Uf6zWlMZ8f68hMA5bpCa1zZSzVqnO6sczBcwGw9T/fwD/Qk13UmZP1G
RaSUtfo0ZWOKqp7C4FRDQbeUpAZJtKpGTtk90I8ctZy2StXzFCZXsxTOJARG+vMTp/KVfoLm2J8U
IZzwaXv6VzUP3om0b8V/WKCG0QplbmRzdHJlYW0KZW5kb2JqCgoxMDE5IDAgb2JqCjg5MAplbmRv
YmoKCjEwNDAgMCBvYmoKPDwvTGVuZ3RoIDEwNDEgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0
cmVhbQp4nJVUyYrcMBC9+yt0Hmh1VWmxDI2h3QskZCADhjmEHJJMOguThDQDIX+f0uqlF9IYZJVc
Kr9X70kgUfypluvjy7fDh08vorvfVL8FCJBATtTYyKZGocFI0CSOn6vHO/GzQuGf45cK4qda/Khi
LuWFZ5EXfCkfw2hWkr5Wh7tqxz8dV6JRgg8WvBEwbBxNpzWWb1er5f3m1VZA23bbxIIfRtn1FSkr
DSmhtJZgneifxHLPIXE5JfrDuxXW7QJXsG15ID8oP2geEGHT5k87n4VQEjpw2IU1Kjkh27Tv+9fV
rq8eAr/lm4/PGSGeQ8jADGhBTS0bivi0cBKZYn9YAbb993G17tfT31yPztWruVNouZ6WZHUmjC6S
7RguebjKwyUPl2euXfB0D9YH2i/rsowNs9IYpl18pw+Gy/ArLxXaMwVIycaaKaA6AuK2sr0sKjC4
pR0YIgaUCU/rKIWndQoxlk55ZRiPC9IFUIFPlwLFAY7wK9OmVRbT5O3M/KJ+6pp+lqTS9jYB9TUB
jZPWNTOeRIGAlwedZ4OeJSUxN56FTaIEV/JUF1GjbN6+OBMZ6tiIIS12Z1vakiwS96ZMXIfFkORt
hRSbOezqBi1cUSRYMO11AedYiZjnmUSQ/peZ4qBRAG5qbm8MAn2bgcRat7jU2EaCU9Oun7r0gjez
ZMpJTTjbvIc1auxAM9a5GQZvmWveAr7E/HV1i7fsNW8BSZfNWryl9v/VqoY7pdW0xmmnaM0jor3Y
M4L6tM5wnveqiTbLrlQlnJze5L9yxpPpimnZfyadEpXMNPhZhdNEZigwOzGzn44uyOG+aWJNFTeH
D+eumQzMjOpObpoH8Q/X96PBCmVuZHN0cmVhbQplbmRvYmoKCjEwNDEgMCBvYmoKNjQwCmVuZG9i
agoKMTA1NiAwIG9iago8PC9MZW5ndGggMTA1NyAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3Ry
ZWFtCnicVU9diwIxDHzPr5hnwW5S2m0Ly8KtH3AHgkLBB/FBvNtTUMFF8O/bruexEmgymck0YSW4
U/HR3Y7tbn9Ds5jQFQxWrD2cBBWcwLBVbDS6H1qPcCFBju6X+Ek5nOmp1a/GCa9GtsqYB9W/6EDt
iGbp06GTHggyGKdBln5wUL57FMuqKhaTzym4rpvp3xUp0pZNJCmtMt5BvKhgLeI3irmFcSoEQWw3
lWgu2WtTSyVWmpRY58dwkJKNuB5Kz/tMBPb1Nn7RLNKqP2GFBzEuRkYKZW5kc3RyZWFtCmVuZG9i
agoKMTA1NyAwIG9iagoyMDgKZW5kb2JqCgoxMDYxIDAgb2JqCjw8L0xlbmd0aCAxMDYyIDAgUi9G
aWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVk9uK3DAMhu/9FLpeGEeSHduBEJhMNtDShS4E
elF60XY7PbBt6bBQ+vaVnAOZYTcwBDyyLMn6/pHREvw1xf709P348fMTtHcH8wcQ0CIniFTZKhJ4
LC16htMX8+4GfhkC/U5fDY5HEX6aMZZnxyPMDi2le1xZS9A3c7wxt3LpuhKvAnSzk0SknLgyz2sU
b+u6uDu86gCbpu0mCvmky3Yw7CrLFMB5bzEkGB6g6B04lnIOhuP7GqnZUY37RpaDLNTpwrrs0ak3
YXaVmCjkWPUSYdDDbsz7MLw2t4O5z0zFm0+Pc1f0XFfSTIkeuEo2CFVuykOyJFzDUVsafqzLtb8f
/s0F+bmCUeQRSq685eBnSkqZkFryDdeyOGm72YmZCdSJrfbPagU5xuxjtdJFoF+cGEWBHPsitdui
Dt66Ml1H7beogw4YXVCzH1mFa8Kc9iNDJugWMKcCpfxv6zG7JTV7p1RNkv2L2OUWtkNL0u1V2GEL
m0sbY3WJzdqpy/06vh5YAc9o9VynP09+t4zQlNZT35Tyqy/HHRoXRerVVftmvrndGpe4oRtVpU3R
Xadb2tCNKrKlC5e69awzovItmvSrAZhEIcww41vK70TfjnjP2O7hP7BJLBUKZW5kc3RyZWFtCmVu
ZG9iagoKMTA2MiAwIG9iago0NzMKZW5kb2JqCgoxMDgwIDAgb2JqCjw8L0xlbmd0aCAxMDgxIDAg
Ui9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJzFV12r5DYMfc+v8PPCzViy4yQQApPJDLR0
oQsD+1D60O7u7Qe7W3pZKP33leXYVjyZmXvLwjKQSRRHlnSkI1nXoP6pdvunL388/vLui5peH6q/
lVa61tipFvq6b0FZ3dTaonr6UL19pT5XoPzv6bdKh1et+lSFtRgFH1UUeFX+WYu7tOj36vFVdaRN
pSYUC/zDA32ogT8Ut2sdux+HYff68N2s9DhO8+IF/cjK6VyhdjWiMtbW2nXq/F7tTkYZJG1GnR9/
GmAaH2AAO8YLgEkSPWn0f85fWMCPnVzCl063XqJNkPx8/r46nqs37OHuh18/Rhthy0ayrdFWYd/V
jnxkG63qaiAvz4+DhvH8p1Q3/fX+36gQtxS2FCxwpNDW6Gx0GrrFYbIbvfGGnB0fcNCH+EyO0qWh
R/LEePe7ZQGtnb24yV+FV21Y6VV6CQy4pxjg7C9+Mb2mNxNwZLwK6EKovXRs7GC00Mbq2bzGsi0s
hsOiJUQ3WncY43eAwYZo/EqZCcqi2HT+1rE4rJ79fadPDKRfgsfwHr0hewa2G5uWkCBHGmwXgxPK
29FvLSHYF9HPivXRx0GnuAEnmXcsx0ofWYAHQwi0SwBMCsDK98XhxfegJ+FrBitAMxlhRgTSJ4S4
eU7YGcS0P6bYmyLIGIJsBPT6xH/GLJbO6fOM2jqnCBArcsnDpY8UuZRcwZY7cDSm7pumLIbN4E0s
9sijSHaQlZENpnynyyyT3QNrVskuaklm1lWeMLd4AonLmpfRhL1FE8TXLZgiMoVbhmt4KZW1Q3MK
4ZzJYY5FFqo0oz9ncpE8Il6WdbGVX5xaXLFiP1E35QYJkAAO5MTNTHJIAFKxdz6eAfJUFeT4hQs5
f/A6ms0NNKHr684j+BI43Q04oWtq2/5vNF3KbheDgvPK71UQRQ2ETyeOGDEMJME9rvoqCZRZZsEq
UeVC6b4hWc6EAKzbciS6n3kIJes+i2nANbWh/wKBk55CM+plMsVGGHfwK2RHhNa3HBEY2bJl8m3w
5yTIU/K9qLBlHX8+md5vvl+aAQo7uHzCWmTe5dLZp4lBYmzc+BAtcNlsXPm5tIDrJdPeKhlDGf7C
Oam7VTEGaKD81gR4CEptnjnmiKwInsv9eo6zkmhGTIGl8MpUhcdntNW0MyUHlskBedrAWFbHMGgY
IweEoujv1A/ouofNwYlmB+G7KWMJ2WEUk9/VJOtvJFlHORGb4nOTDDaPIItbratdW7ICuGBpOfBe
zuimGLxFs1t8D1lyP+WuH05unU6sq9v2hUUHt04nFmtrLg4n2/GIPHVa8WSoFfb94hARCoUZtivn
iPW8m48x66JJgRSz7pVK0mmSCDtCHlFEgykR5ClclHYi6RX/X55sIM3MdxiHTzVbdHNnUsYa24vy
EwHCA7uZuwr06XC3z9MYhy0RCscK3HBxkmHe2Y8FIJt+44m9nyLHhX2NQLroKG/UfycVqXQKZW5k
c3RyZWFtCmVuZG9iagoKMTA4MSAwIG9iagoxMTA2CmVuZG9iagoKMTEwNSAwIG9iago8PC9MZW5n
dGggMTEwNiAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVfbiuQ2EH33V+h5YTSq
KsuWwRjafYGELGShYR+WPCTZTC5sEjIshPx9JFXp4m63p5sBj9uWylV1qk4dGQ3q3+Z59/r195cf
f/6q5vf75h9llNEGneph0EMPqjVWmxbV6y/Nx3fqrwZU+Hv9tTH8qld/NrwW04MvKj0IpsJvU93l
Rb81L++ao/9obQmrBeHHk99oIG6sbpc2nr8fx+f3+28OykzTfJAo/J/3cj43iE6THRS1rTadU+fP
6vlEitCbI3V++TRiOz3BCGC6CUYz+xtnKNwewqUNl/ATXFhmHNL0hGNczI/8Bnn/w/nb5nhuPsS4
nr/76UvyDNY88w5Z0yoyoJ3/Hx1rVa+N671fo4Hp/Edtbv7783/JIK4Z7H2KoFM4dBoGSJGC1UO0
+Gk0fQyBclwOAMJNvJh9CCzE5MNz/gEeJxyp5+cE/N9nJVx3/j318XafN4BHJqXLzJM8vdoHVD2Y
J/IZXLUuH6ZsqGOvu2wUD+yybKNoTeJjj/YRoTZFDDtZiuEH8voIJfCLstRXga2xR8beFuj9aogr
tqCnDeixJw1t9xj07Rb0Xa8t0Rr0EBOBnIOYdLmPmbWLMAP2AQI8oE1g0gI2cB7r+LBsgSG+iWat
z96JS0Ba6QnHYsojxSmO6ZO37WW2qyfIC3lTjUJpvrjFf/hQgKkQlHogW7w5guFlVUUIqnWLFGjX
U25B24ThIuV4it3W5aA7sDEhUjX8UEI6+vvr6oorucgxeE7G+2NR0OSKHrxdU9rMLVqQgm32I6LV
ZpyqtMm1LejE6MkwRdjoEjd7CxUCYS0NtJvaEM2ijvySyKpYnJGkx0+ZkgBwISiPC9XrEgSlgt7E
gay21q3gQC7a9rVS4uSCitaPhnAfHDmWr+cqYu+wXfpts4WatGL+MNq6hp6bYgG9J7abrGG3WANQ
W6DHWKPbYg3T6eEmZ0iM+1Ijpyrowg7IXYY3qLOUQFu3KD/q5CNV291RPwnCUG4Q+cpIydd9f0UZ
Ul+R5aKhXd0tV5RxHxPeRLLfQNJXl3b2wdHvNpCE3mn0cu0ayzcGYe6Gt2dhGeycVGnjw5SV0CUa
hRhuolGRYMS9xlkGRlEGtalVwVHLHEgaR+xwr/OK0rfe8xwFlzAJ/XERkxTNviZB0Y1T6fSsCO6b
HdChxoT/Ei66BADmQilHaUGpUrvWZmEEUzWhc/H6BHhRdQq+VnJti4yGrRKmQaMZHithWFXqKSlE
2g6rAxU5F2lCHLIQr3n4Qg0ueakqRa5NoKxDBTyvCqmYvOIlxqGoZgBm/DlU/Inmi8/UkoqrvtYs
tw8MWycGv9+PugdTvnViANPqgdZmp4yAU6aEWrq/yf05x1d6XwTdep5ZTdZ5Zh0UChfnPF7pkrDX
VPo9ud6S6M4fzgZ8MNdbEr3vNPZr0/bGiKoZspPscUhQMxSnekELXJ285OLgF0RNrVoe6yi4Zllm
VSdDUhQmb96VD7iku5OkAzvSfmpNrTuF5xcK9pg5W2opR5IlWlak7H56F0+Q5h4y7ozGYW10XiYv
i/OlYC7nVBCespz7/UXLLGYeA17OHot6ttfIVZvlBCa5NNXJhqopW5r0kGbTCaQ4CjQGkwaI55Qi
zWQErcX1QP2skgYknqydfgMk35Veft/sniIJD7n4LwbbB/U/cn8XygplbmRzdHJlYW0KZW5kb2Jq
CgoxMTA2IDAgb2JqCjEyNTQKZW5kb2JqCgoxMTMzIDAgb2JqCjw8L0xlbmd0aCAxMTM0IDAgUi9G
aWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJzlWU1vGzcQve+v4DlAaHL4DQgCLDsqGiBA0xjo
oeihcJM0hVOgRor8/b7hklxqtVLsWtahjZGV+HZndmYeOTOklNTi63Bxef/l04dfb7+IzZur4S+h
hJKKogg6yRS0sMpJZUncvx9+eiH+HLTgv/uPgxpvBfF5GJ+lCtyJCrAqHqvuW3vo9+HDi+EVXtpr
ou4BHryEoNJZsPu6q+Pih9Xq4s3V99dCrdeb6+IF/mDl5mbQ5GSMQRhrpfJR3PwmLrZGGII6I24+
/LwiMtrQ+iWtVFRmrVfqmi+WLzzUcf0SXyOZ/IwvEK20bvdZVrssCDTSlkX0du3ciq7M1dr4ldp0
yi+rcqB2/cvN6+HVzfA2x+Ni++nj3/fvq1N6dEpJH6KFoYEcwhn7QTL1H/tsnJY2eZFwI2lEVSeS
wTXgTlTAUJTkGHFaSTIdMlSkCc3UluDjIWUSaeejhyUmeTVahoEvFtokfvwus/IV/19j/vxxRiPf
IaxKWhuUi6CQ4+XxQclqHTiS+PD5LgePkoGCJHyQeAJ2UdASDlWAnIGEFd5K8noaQqkhcdsAUtKF
BBMroL1MTlRtGmBMor6tDG+F0U4ilhUAo5IQy1G4joru26EC5e137YliXdVWbd/17XYO3A1PovR/
Fbo6r3w/r0Ib8byKnoJ1Kp10Xjm4E3wLBtCE7Fy1lyHcK+YXoDpXpGfO3z7S+bkzT0wF/6nQjPMi
p/FWlmipLFHwUpkIhRJVrRQlHXNB0n5NXCeISwtKxPZogTC1QHRl4PCA52K00vBKcpiy1nPElYX3
E3InKmKwfmIu3cYbGXSHDBXppGaanzozzmroPnF2kTiP+ZWQShJnljCjTrm1WWnL1Dnrj/LmRu3c
T6mdug291nbOFmDyVZsobeCsZVPE9EkTMlSkCc3UnqpuP7uR+3T4JTqMihKlW+OtBklglw26BBF0
TUwKui1cNtpgYYEiZIw8PkpReOzSsgYOIE3wvEL6+DyY4GRSVAF4WgCenYhzF56GtPA0oZnaJ3J4
PiP3OYxLHFrUbKViXlLIrYdXlDlOV6p02bqmqk3kI7cBcDVqdIahAnC1Asjk3E2zq7zA/YQMDalC
M7Un4uP5jdznQy/umUzE7oi450K12ytPYIErU1lQKu9+xiW2GRcVjTc8E+eOb250TYII0Ywy6ySR
7cNRkC4eSOyKfB+PikzxaFJzzaei7SyGLlC32FdYFbA5ZBtC7oNn1Pm8VeXWQl3xJXM4Mph3wEbn
Kx1nzRxmjTiRxz4YBemCYZI0fSjG8RSIJjHXeirGnt3IBbYWmwkDbZCHrJXam4ezpSPlswbsXvPh
hMn58Thr7iBrOgUZAnUBqUgXEHS+Ltg+JBVpQZmk5ppPVbLOYugCe8u9RyK8EWlSoeSYeSuoN5m0
MSW+QhOvC3cjl5wd6bok0ePEhcPEeZJ6ZyZXZIqHZp9Dn3saMsWjSc01n4q4sxi6QNxiw8H1zSm8
MaD9j3s9/CVzkkna6su8vEphy12IHk/7Mu74oc14qodb5kF0pj06c5113Et5Seid0VbrILWxDUFf
XZGQEK1UjizyiUdFhoZMUjPNT92RndXQha30YrtC2kqfJ4fmY5f5JmBLzBDXN1yvsBT92J+4lk3H
r5txZ/AgEqkey2YzECdKnAOwEbFoiyOfaFbEByQf3pi+G5rR7Wn2VuedbQzSKxiNus171bvc2hEy
Uo/sSpUDuwo7J02M+c0cD4G+T49bsuilRlwmYFei6FmakNFh2bme54JMPJNK0ruO5gpMLE8yM72n
mo5nMHNhMtJsDsxnZZIq5RzmzN6ktOOktFPbpVc2p5b++q+n536HVn3iYugVdbGqSBcrbnYo9sGq
SItWJzXTfCJSz2PoAq32OK1Go97Y3O5B0XMVfdrv1pr5vC1wffKtSBeYYGXKO/cWhopMgZmkZppP
xeBZDF1g0H9zYQY+GuaDY3werv7jd7/HZVuPXPPXpf3+FqHtMInqTwb8CymlWGLMEa4/RUW2ejpQ
sVIZ3mA2hLx0+eT93UP1zZtG1AfWh20HcRk20vvx0ACxQcbrkV2p6VeiB7phrJYeX8mrvMdBC4i6
E40BAUnG/NPeY9zgWeHAPeszniutyWSGmgksDOXfejx35zU39BKPdmF+Brpr/uRQz8vbQ51weeaQ
FE+dt+If6vqDqwplbmRzdHJlYW0KZW5kb2JqCgoxMTM0IDAgb2JqCjE2MzYKZW5kb2JqCgoxMTUy
IDAgb2JqCjw8L0xlbmd0aCAxMTUzIDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJy9
W9uOXbcNfT9fsZ8DeFv3C3AwwIxnXDRogboZoA9GHtqTOE0wbhEjRdC/71qUqL3PNR6fdBrUHnEk
iiIpcpHaNrOdft28vv30y48f/r77Zbr785vNz5OZzGxcmbKtc812CibOJrjp0/ebv301/WtjJ/73
6YeNab/K08dNm+uU8DQpgaw4NqufxqR/bj58tXnApmtObjWBg1dYaKwsXP24z+P1X7bb139+88f7
ydzc3N33U+A/SHn3uLEuzqXkyYcwm1Smx++m12/95B3Y+enxw/utc956d/PKbU0x/sZuzT3/CPyD
Q1tuXuHH4rzMSZ3kttaO33OtjbIQ1OLecol9exPj1r3xb2582pq7FfNbZQ5quPn28evNw+Pmnejj
9Z/+8aQnsqdOhINEEyZX02yr1QPZOFcc8/HD1t97f/P405rj3b+/+6/ydKd4ZmjXprM8m5IgLvQU
eVRRhaWKcOrCv+5ESXKkYJ3owdZ+xBuqhz+5lcIs+Yk2ZUIbevnLG0wfSjktaS5zsOWEpKs97278
VpVOEz3IvqYbiRslnWaDGLybtBk/njeMv2SYkOYY8wnDPFw0TLhkmDM8cVwrriYav21KlLOLYkmF
tehtzjR1FHtnknkwb0Ujq8O6fP648aRoZY5ucjbPIfvj05p08bTp5GU1uKTenmX6fuuL2DSIz7n1
FXVqVBfEFd8sLgp69wA5amlMVhTncEW37dauNCfaOeuJKqytdT5pGWEkYqgw5m3f1a/c/qzW83mt
W/hFgl8ca/32otbLJa2fY4rb73ntxX9SvyfrwLfnX46qk4jK2MerlfSi9zverfCbeo1uLuGUE6xN
d98tfz+Cc1d5+0VZflal+4tKrxeU7vycT+n88r22p/OSnvI00y/3dElDI74WaJ9W66qX674Xi7vt
1m56QTv2Ukaqdi71OCRvQ7ysn0sZ6QzPz1NPu889vMcx8WHfG+iyGvA7G5/XcEAYGF/JoCWPCwq6
lBkSzuRPJIYQLivoUmY4w/P91mZJDL7ft66TdkPCOmenRSdHSaG5ULtED9ZoHt2fuK+Mtz/+8J9P
3w/Ze+pAZMm4zNZnFwEHy3pQvf6PR4Oi5pjyBNzmHFGhd2UutirhaVKCdxmQjpgwFTN7GxfKRilj
0QHbDh7NjGtQnY2pJEjiazJNMgxSlzDU6a9/EOX/iv9/Dfz70wsK+Q30uqei8wMoz+O+1BwRc+zs
oqdgHtmp1EGBZJ3irZsjJHyaAvwTAXuhbJSyWnXA+UoFvqyg36hzjlrBnsQfPtU5VOJgB+mXsFNa
2okSSZi1Y0iX3b7nblZLZu3VqDsyqpDluJ2wnBY+YU2iv0AYb/MgbJQwlhww/Z18+v8t4gljnIQl
Idk5OdaVsPChLdwtzODuXWwQH3/cIdTd00AFJuL4soGqxqXPvVoxOVzKAL8yc7YB+gkx4thuUOCf
nUL/NKauNDQoQ0XLqkPO19rxRQU9tqY7iXeiL7MDmuPVOjbn+mr5y5ZzVi0XxuXyGfk1TbGgepUD
A5JDd0rAeTvBRjuHwqAQM1Jl8Atlo5Sx6IDttWZ5MSFPmOQkxIqMkBn7eaQb64ZJwtIaaS0PVIwN
cAP5vCJgunFm69/cAMOyILjt6NFftptXu7lsYinOR7aIXC1dnVSmzSnIb6k/X8Oc4W7O1Nm2iNQp
qAxSYn785nPZSW7s7FBcAhUJO6JL8C9zrlVyB5QZ9yn7i5426gSfeYifJ1QxPqTFE6b3E6xpUd6e
/OtbzPuu2Tmz5oA1ABG6u0ALSuHh313NfokRyn7c7d+Bfc8LC/eROcjcEgi2diKSLvSGASJMcCJP
RvQfY1wOGaNSmUty9Htn4aZloWyUslpzwPXKG/xyQp64weG5t+cgmaJcKB6oqCAfrzp8q7vubl5F
FO684WyHBlTvJkrlCLq5jHdcVLwDFbZNkbrnxABfcccclBVLmqtdKFBPpzjgCoa7hCBGVfTxRsfL
igOmVxr0pUQ8Yc50nTlhQgndzvnZuHrCnL24e4PK90EM+JZ/otznz3dsdx6VsYt0+TrpEhv0qF+c
yViUT0gHx7oz92wjRzaEfiPjl+eKk4yfAzKpC4jaUNPHhVLKnKJ9ZuoYq2F1ghjyS1gdZIccmIoS
7jgOvkfZX6W5QzR17IzeAhiBI2InC/uPKwo9Kk7Jm1Yd4i47oIHkKqog+GJC7i9x2sGhyxyCBSXN
ls0BB++lBCkgPIMDCkvD+iZ5oIU07UDxqJKwRyLs4wzsGTnDzr64KVkE7ljGGHtY8nSrGUEiy+AA
AFmrW+9hyXMRAukcul2ERLKNeX0KuA3i5HJO6LrGlR7GuGlqt6LA3rVMCwcULHHsUJDu5QgqQfG4
3HkIWOKcoM9xAtz94rOcoJ8R3kPHHjro3jS01Me0RNejzlA9Kwe1xLKH2kqlUFuqkGrtcYrhDu2Q
w1u6Dg78aXdEoT+eCE31ysvPay4+CAy1j/O3/n6/zbV/z705QvaKG5wjDrFMufQaYwcFCVQpcA9c
NcHEdMW0UDaDsqw64Hw1NnhJQY+t5u2V+CCxKcVWFJDb8ky6Ctle2of3JhigY8ZtB3AgXeRLsdu7
58v1ZZCTiuMVjeAenBrA1UG5DtGCfWEMQbnlhn0rbrZSGvuIWI+QxSTASwtIEZCpGWtACanh6hgQ
yGqbxYKPs9jGqvLOl6zvs+IcS3tPtKXILM4HIc7Zxz4JUSInoXnTWNW5MvaBZSl9Q9SPXtgjBEeZ
hRIqC/MgUbvNCjgJSB77yhxEIuGNSB51ToZSyR3rvUwqLQqB4HKD9pFNaYImnL1EKisiWYRGCE4n
QaPC3crLPieF2QtvBLaQ+yycVdijcnCC0eCpriOyVHUWo2xDZbhLnJVZaeB8vswl9B2BziUrgOar
CI97V1OzapX2FGcxTXmhxdB4RRyjCMXaviOjrPCHwVKQWYVhmIR2uTGJ5U4lDQysbIjgnYQ5zFX6
hgjwkjx8QAKIMqu/JHv+rbxwMaOQTAmKVSm6n3N2zwQ0x7eFjuu7I5nWfIDKJQ1iTnRRw9Vq1WVA
I15Qm1FC2wMmr6UOChscRjSjFKoVhZBIEX1axhE6gnS7zaCk9lg5OOA6ZmunsUcf7yZx2VwG5Umu
iBfQ09fAwXkJBk8d664LReVSDiq37tHHu6PTnwnbz26KHNQBkbtYiRHwiv2mmn+Qp7/L8fnZdWWo
uH+FnQm4h3TuWcHWQAor2HY/SEumvZlWud2kADNO/CDH90ARqji1BfQprk1CyIHHW4AfV+OY5BJp
wDmpyizcFMfGlW3AWmbhrvD5AD6ais5ygRTE1qSzIA7gFlteyWeZhWhXSeHltmOWi6QV+eqCs5zE
c9RN8mrbZjmgSpT/CGDRxD4L9x8UzM61z2KPibSgTQNQGNctfIlvmzoLoRg0FiySvSrUa0hZAhhp
CJKT5f2rOks6sgjwOfVZrDwLm4Y4hWwIQmSzB/FdWzukOXlk1+QSgExLJSVDNTqLCiCtIprqLAc1
Wbh3LM8NO4JlqAyL6xS0qyWvNY6vNq0fAkUh+tO/kqjncNXlsBMQPJNtD+vGiX8WtkjdoJAjfBB5
SCnshDLT0cUY4ceY0R0OsNsMChM1pBocuinGHn28AwUrRJ+N8rQJ2c+FLTZdg0xGRD946lh3XShd
rsGhyz326OPd0enPhJ14bTcpSK1rUYIEZ0/GndvLcefZDRCgTaCQIEigGEknqPICE2dob+CSJtm5
ZXqQ/gzmWAbIhgNiDwIR5Z6vLZ8Xl2QW0IxxQjGlJ1xiX98SOoXjrNhQFINH78BGVIKJyAZBpmRB
TRZYAheWFISGPqu2dg2SdTCSvR3vD9ctrWz2BvmpIu7l3OCJ8/I5Hgm2duEd4hxRGlUfRXh+4chu
C+Oj7cI7RDN+SIU4WosIz3ISCiDFhS68RFSupPOK8NjJZK5DZsxdeIjoiWxAM16EJ+YwXBelv9Nm
sajkSj74yaTUHrdxiOK68NCvFN70ztJmlYaikTt86MLDnqhDQCNYch1ES1/e8dMBRb5O4hVpfFDq
IJoYCdoOxg4QjfgBWlGcxlRJFE079WxFFG1pa9Csgmhp3cC8IXbhcW0zTQ03yNKgIYgmPnBwFY33
EVEX0pDUgRWibqKlEb3iQNrtOw1n9a2VAIINBb6+9fxCZJ3k6QG5QV6MoAKITOZG8lKDvvhtkjeS
ohiLOYKEhuM4p4UNPmuw9PkosNoiJbBDlxRhAg0J6sRFK1mhdpH2dRL1tFkMRVwZ5RI1qB0C1+Gg
yQ6onQNXYucqmDYT5Tqh2LhAbc8miXESRhrUToXrYHotXnJqONQgI4bcobarrdHudMPKTjwyRWhl
STHSoUFcRZ3S8X/p6RkIplYxM6A3QgoJLrk+KcjpLeAKUW8D2pahHyE/d8z+zJRH3cM5eaMYtvj5
cpaQRMflI3mWjM1CSz5sPlz0W0AbykbCzFwUeiHFrNEJhNntc2elUIE+MmBHMcYY87sHGHW3UYqo
mI+UnQNdgQ/ifQsdEmUj4NKDO+VJik5WL2NJ4GZ1YdnHY9OFomJ1DkPsvoeOd4dHP5Ptrmxn85oD
YEMsBIPkTma7h8vZ7tkdbMZAwMh1fYkYmEKrOLUYZ2EuqYaCxR4CU6gjabVZTJx2VKatj5BiY2U1
uiHiScMStJahEO5aTR2XOAlN2GRbQdujm4d8raDNrozoZlMceZKzeIrGa0RTeH8WSujBFDeYIYml
sfEjuLUD8oW69PDWDsgPGDQkhX5AhOFqe3hrBwTFaL8h9wPiTrX0OroGUJqGb/YNUuMVW9Dlm15q
GR54q8/y/YQIYC1+ExGlxstp2M2pHZH1h+gdtWs7IqBFTCNwWekbePkXFy10pdwq+xKf+8o9avTR
qKIreUZs8pO0SEo2vQ7P68Jee1mX401mp95LFo819YDcKEhGsLgclJ+tI+vy3ZOakCYR0mluFzcz
L0uaTJUL/EiItQYJN6iPJPflVoHDMJ4QjbmvYAWM51iRI8/VxLqfzdU8xjvJehUGW2bYFmUHB9Ma
1cse8hFFWaSg29a0SMmGmK3rYyC5syuo54ytXzr0MMZNU7sVhS/keVIGQGUMkbqBZ+u5ygZdBFgI
lfciom81wzhEv0e75ZiMHFDIUEO37VBUH+82Q5U6Q1WtHNQYyx5qLpWiW1OFVHMvx1CH6Occ/tK1
cOBRuyPK6bcKf+VbBXXFe8CPO9L+U0WIl54qgjl679bvDoWZXDxAG+DYMihP06Cw8SRNwgDoLdFF
KZtBWVYdcP6dPr18GUGPjRaufKrwsbXYkZ/CyZeK00/f5pb/7qWPzzx8hy94rdgTjYC2B/i88qgv
fPgOX9QdXD8ps1yq+rnU3vdTPaO/u7BEsIIS5HuGpN8FnXvDZskinRbCeO64UFAWIRNzzEyj3cME
R6uFvR1E0MgGb2JtRNDHIAseyOOo5jgjSs2YYitw2A0qSV5OWZihDkBZIj2aFFrhxLHhHvzHU+zV
9DH2YDNu9Xsn8W5ZT89abwCkntj9UxGkVxgXEYmZgORXhyDwYDdSj0nw4FZqGOOmqN2KUgT6Dw69
xTL2kBgrW3QhiFRKWYQkzMNCPYVvtf1uOSYn8oG4KYGNEuw7lNTHVFJXo85QNet6NcSyg5pKRVBT
qohq7HEG9QY95PCWroQDf9qdDihXfhuVQi8z2DeI9iAPhL088G76H4f7G2kKZW5kc3RyZWFtCmVu
ZG9iagoKMTE1MyAwIG9iago0MjY1CmVuZG9iagoKMTE5OCAwIG9iago8PC9MZW5ndGggMTE5OSAw
IFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicvVtbj1y5cX7vX9HPBnREsngFBgNodAli
2EDWK8APgh+cXq9jQ0rghQPD/z7fV8XL6Z6e1s6OoxV2Zk41D1ms61dFttv88R+H129++vtffvzj
6e/Hh9++Pfzt6I5uc6Eei29bK/4YXdpcDMef/nT4/a+O/33wR/776c8HZx+V45eDjQ2D8Pk4CJyK
z2731xz0X4cff3V4j0X3M4XdAD68wovO64u7P8/neP0fd3evf/v2398d3f39w7u+C/wDlw8fDz6k
rdZylBg3l+vx4w/H1x/kKAHTyfHjj5/uQhAv4f5VuHPVyb2/c+/4I/IHH329f4U/axAdkzsp3Hk/
P+e7PumLoNbwga/4D/cp3YW38vZe8p172E3+ZkwOarz/w8dfH95/PHyn8nj9m//8PHbkr+0IG0ku
HkPLm29+bMinrWGbH3+8k3ci9x//up/x4X9++OeYM1ybs0C6Pj8556c7X3RH4kQFw02uLdwH/pC7
sUGK471KyK2x/kHFSPHZh+6D/pQp7PS0IOSWIHLcRK7I4f1NOcRbcrg+5adrm+gSAfc0Ev9miSAa
TXbyivYUlG7v1fuEsV7FS7rOgPEq0Ox0aB0z2gtTTNeZj3lLqVzhXirnHMxx5gDTvDNrtU0FMPik
FtItLfi45ZYeqyGmm2rIt9TwxJyfzuXl/bRC28Nyyg8+hrQ0Io+F6d6qoerDh+ngRjVlJfgzZrlu
3MMD5Ot68XCuFsK13WSzjTDN5c2aF6Zgi4ZlR+E9NqvOqCPOeDObM5nU+FZVuwx0OqzZYBzy+jr3
pW0+tmvcJzVpE8J775RDyRYUhmh70Ng7OxcVfVfDppuCPHOor7GV/SYlX3NVRN+610+2XxZ6TVgq
pS4JFYPcsP5yw/p9aFvy8Yr1x5vWX29Y/1NzDj+WWFcs2ScltZAHUpbo7bNg9ryitg4Ne/uQMrKg
Ko9akTb0A3k9KZ52QzytbDldCdHxw03p+KsZvYvniTktn5PrcM/ky01kVf+lBdagJvdm78LI8WmY
iw40C+kZfh/aVXKR3mUBRkZogAP8/JgAp6rlqlOZWn3SqKZL6I9bGvC34EKsWLBeUYHcVsEtuPDE
nOcqUCefaUetLj2OvPDOt0vIoRs09JWnKfoR2M4F8OEvf/7fn/40+e1IwW251AhgV0IC/Kz7hybj
P24neQGaBKyFt6UKFCpBNp/G8+djf5aASNOIQIsLW2thUQ6DMt+5mLRDVbdhYy34lGsGH9KyM77w
kDt/CLG/+zcV9z/w/6+Btv/6zVj8HjI9E8/TDxAc5t3gB8eWtlSFXElSnN8JYKsTxOWtJhBiLFup
bRIOg7BeOZ/0hYL7Vix+P4xx1iL+KryUXLcGaBYqoMDO7+tIpEJ/gK2nmG+beYdirMbc3opD2Vyr
u512wtppiVsUVk8Z+/CwmkE4DMJ85WLSf5EV/3+zeEUZV0FmRBSDoFEZbnEVhl0X4Q3UEN6FZGUN
fjx4hnhhVIr6fFtBZcShn+tOqSRFMo0Vd4J4YmpbljoIMM1OoGmKtC4fV8uiHAZlvnQx7Ut1+M2Y
vKLFq2ApJUa4pC71WI17l5KvaKwNjcXpVEhwucRjClsK3G1yBeZXBwGMd4KPaauewSA1yjYsymFQ
1kvn075UJd+MyccqCVcBWvJhQ9Q0ABtXwRNXy8VaKQ/I7oqmkNRfKVoK7k7e3nsXCdffzCrslt6C
H3oLxaUKCJfYegqtdnFSmL7kqJ9SfrA0pDxouXFvGok6BZVz0yjy/c+djnmwvxwcFJWjTidbCTo/
UBLiGmCeIIXsCOevfD4ME/iZW/jbMWxVYl52cPx0RFryRa7/+gPG/WBaBs3XjS93W8Ffg8Kdf/fC
2WdsGJNPr3753D0RrLlnquDcnkjP+pOAvJuHCaa8laSBqhSND4OCSDUo4M83dQshUgyLcpiU+dbF
xC913m/I5hX3Dc91ncsMWrdCHIrf+87GztHD/at0J+re7LFGVGQuuUyHT6yxbnq2DJADIdqiOeJP
oFVf44YiAwLLwLUOUGpQkG86JYggExENZ/hfcDvKYVB2b13M/ELFfltGr6g2vky12SFkOWAzXzbJ
VzTbO6FvfWQ3LGnzLLF5yb8fQI2PKrXFXHohcyh3HKw6OL/lUK5wBxt7cO9QDJIb+QpWC/lx5u8p
k9v32atjIh7VMil0zE7JCOa5ac4MiPt+UQ6TMt+6nPnFAeRbMnrFzspLQ0gyQJ/0pSuqFGu7sEEH
gAB9Ymi+0qe+0Gl9Pl+/LDFRcOCfbRZEYJN/8ovysrQXEQhQhgbBpF29AFeLYLPHLFtEJgiRvzWP
ZA8o45USmiXImB04K0qzCoxg3XngEGyi2ZhUMCYopYgOyZsDnichie+DsDVJSmP1wlGYUkQpXlIf
BRYTglkEK5J1lIPfFqUUMZwVgTMVCoGWUB180YLbVSVI7AsCp6ZiNB9VypCu57EDomGLsY8KGyxD
aUXhTcR415pSUuxSEBQoISlNoi4odfNBlOI775IxCFJuqNd1PZSmxBuklCR9EJSDtUlLlr0lMOuS
IGksR4hXlea1eRMBi9koI8UVeSbaVPOKsje4TNeteu4TdL5lDlx7hID9WwNwqic+CioRoN8h9AT8
bs40NyhOywHqNlIc0J7n2W9izCrKg4M7nCB6yCDogs4lbatUtT2YjD6nranfsBTMx9NRNSom89o4
AkvSyPAcuAR1IWU+YwnhnH43IgGOyJoB+qnsDaw1eOwb6uJCuJ28uIRqwPh+G6FucW0zYEXZiWE+
m6BOO0raApTc34dXAebP6eF2BUn9dJgMwF0r0u1kEBpMcIK5hRyh46Jb6JvMEKDnG10IDOVwlymm
/nw6TEGOEUPQY4ahirXGUNbgoutyMDmUvbYxzKHvc1pLF8KFPZ0eUWiPV5JLe2FykQhcqtrM5awz
cCfvz5ve59lDel37pH9EyIf1HLRTNYRMijeJ4Rl7w2L0evqLMMHjuWgEPKlHeWieMK7VqkLNAQCk
iskMJleqx3PQewvUCqpTIRTUfrbqsUavz4HFK0N2lPlMzWNOVgFjBAKdMtUnYJArZb8EBFZhKpMJ
mJ9IWEzCPKHU/TaQh4LLa6MMd7ITxHw2UZ12FIar45ygVexf5hJaGjdzc2MiAC4DLQ8eAyqc2OYm
gktmjGOXmBQ5bYggAM2ksIQ0nk+HIcY1wsQ83jc17Kbvihrrdz0O7rqid+wPUxgbnKZi+7+wpdMj
ynXfkGe3PS59o8Dbs6pyf45W7dTypnc8u2yMCDWRSkJ0lBY1abE+hZwCE23HHkhkFbgk8MqODBST
MJ4Ut0CMEJaGqCF8B2JQFSRZMKZQiSEMKAAcE2k2AWikQw+EJw9IQxLY7jCm0viC2+JAMcA9icAY
sCHZTBheeDuhbrUvF+3GE0lem3C0m9zyqJ9sEDfPQQTdBk8ywC+hOjbsBtSJ2pIjzZkQIqNw09sQ
oUykE5hlPBJ9samAgwIHYcPeL6RDaOWxZRMCXIMyAMF3GcAMXOEYiLWWjnTUnwiTxkRskdPGsWOT
AXySF4VIcV0GcGRetKIfEXF+MejD1ARPGtDxOUDnDOh+UZCbuWZghzkqzsm2AGyh6BnWJTo2mLNr
IIzDm12zDgquYVE+HyeFcDUqoBqQbVAWiFtvXcz8Lzq6+jaMXokv8rL4IvDKQG3BUkq51hu63kFw
b1Ddjecn+gfywuYG+7MK0uCuLr60fSDpNliAF0Q6tCMor90vjOLV/amiGi0FFoWJ+A0PYobIWbNs
KNZQBnrHFBFBMzCvNwZMToBYWJIm3Vw0kQdWa3zDaekfGdA0sWMpLsF+r5T5jCU85/S7EQjQYHLO
gMAS9Lh4roGQAVYXF45dwLy4RCCose23wTPVtU08aRAbYpjPJqjTjpL0moS9z5gTJMz5NXQ5Rbqd
AwZBQdk3OGTk9HMHjKziNdP2PWpEdhxvMtCWB9YdUhrPVITJcY7ocp4zdE3s1ui66jyYIieDXdO7
LXRbmJucttJFcGFNp0eUJ+BCfiFcQKZRe0Jwz+ECLaSbaOEXdYj2ndSmbYR+NnP1sOa7a+/QaDEk
E58PAoEs7LO/9JTHumjOk1H2mMdOSrGLL5B1y3AnoJisphy0lPJAMEztNEVkQ6JcwJWozuHM4RI1
nhGEUReCEc9+S2HRJXSTZD18ZmKhG/GUCiilZMEzU1uZzyekCM7p14ha1V3mDDXr5Y/dGth+inVx
AZzvGHk6l1K9CmztQypxxdwoHwVocQhiPZuoTotSPOJ+O44JSlSsNVeAEoNa/OShVO0ATh4LEldr
axeVKM2W6PtE3RLhdFMOnAlampLqz5RUl+Uc0WU9Z+jamGtMfXUupj47l1Pjcx/TJvpOp810QVxa
1el62v0Ffcuz1AY4H9XuRLu250Xvu5uO+uxqm0LQDjF2rl1AiEBba4h1MqCqXtb3SivWSiMc7KjX
lzGqQHhBaWzhcVQzlMJrZa1DTITJqoAWWD5oGcGYzgaf79d3vtdIVaKF6GColws1C+u5d2QZ6rMY
iHeGVvlXNQyfwhjV9ISYtOa10EcCaKxb8ImMviOSAEYqrWQbBYCRrY4Y4SYGpgQlpaZ4NvBsLmv5
4XyvEZAQeAxPWrRGJ9FuLEqpuUyo7djfw8JBsSGr5pSLUvIoSoDatcfCy2E+dmgfUz2vlFA/N2fV
U+/S9iKBqSyvSqI0oxWrEiI8hVUzdoVstvqvhJqBSmyjS8v0GNhEHl3a3ssFDaXbaPiyog5Zy4vR
Fo65acUYqoqLNRb76kyhLs8WM3kMEBt9zhqV1WWlUHOjXc0KnLSWrF0NDrXShBbLKhk9OwPMwE7F
hSiZKk/jnPr4KD+tK8kuQO41aqxacrg0BgXDIaD18oxfKGDiZ5HUe74sirXLxo6HpZnM7CpKKaGL
K1dtPBBHe2tX93tLpKR+Zs/+blYKv7XDMRZUA2wAwa6PYcPCK62Yv5bUW6pJw56NQpVaRWmoT3RU
tSoDlsI2kY5iR4llo/DWjVapCJ26wTyL4soQRFFhRq9SQGjXeo29h9HaR3jXMwE24UWlgBCW6APs
4+QuK4T8plKu2ojiKDqgtYe9e25xyYSARKOVcM6hxy6JQSlBXZYhv/RoU7SGv3zr2dc2GJu02ZXo
iKoohMXkNWWHEQMQrkrNSmtVefO83aZ5vIxTDEJTxQ9Vm7pfNFZlZtbEk9sRTixaMb87OyxgyIiG
S2q/OMFolTSJ+xF84ec1GZ6Z5gPv9Jq/ZJw0BuaarJQ2wj0Rre+AoYVe7dhN/aRZeQQwnQq7NjEw
frU0ccU4XqmtKa3UeQjDOifX3XEOLJDH6kyrXu2V1lWJQOAnQxA8a+GhebGLEtYwkWzpOQ9BRK93
1zy7CCYIHoxFjuKmx4lVtOqrMOFbaEJIC15Tf5ktGjhCIBzAtk0QPERw3iDDEERkluMohkDpZ2S8
7VHYgl9RrsHsiD2SCYJdxmoYSYYgeCiXo3Z3vR9RrnKu3u/dRzkinWyCgAXpbS1ioSEIdvaTIapg
goBt8GomKW0IgtahuAvbtmNAtmi0G0yBjIZd3BS+YdcmB2rdx9lEtkGFzVSSqomBYU8RGfc8QhMs
iRZO16/jrFWKFal+yAHK0642aM3kQEXRvnvpaqOSKoHFa+xhLveSGXvucnhGt6qXqp7NwRq7Z+uF
a+pKMyTBitNbOlWvP34+Xr6161f1e0yEQXr/w/OgRK8wQ0/JlUlBWBoUZkO930dHafTCQTlMynzr
cubVr/olZZqeZifHU8fx206wv+0G9DCclXbjdceqp9QGALX6A9BoA/d4XsHOShNzBIZUAnh6rhtg
smkOIC1YzPPmP9HS48B/ajmRF91qx3+NAQGhAolxj//w8lZr7vgvM76Bwl7BCJ+Ojq23Cgb+Y+uc
FL87U66ZvLM93sOnnsuwCzzwZj+dJg2RuodPHtJ7no9OZEcoHZQWLJH3A0UkY8VII3wGpXhVIYMn
kZHH2DTb24hS8C4vo+hm41oASkmRMtBfVNDrJelZYm+CI5F72SNJzgveAVVyHP30xquLBLUzltEk
yZdAytJb84UhVlidjegpmjxBktCP6jRhiF8n+SNSgtYP6REnNT6IwXUbBYmwUmVGs/Kjo0HedI3z
FMNrceKhyprHAT1b+qSEvKKi3sAlmtUxSdfhMxtLNoYAACQGptqDojAAhqQ4zwbRQovSxLBm8Xan
k0l8NOdLUF8gLVhdwaCobIpuagRFUUZFL6ZZUCzKJsJk7SosLJnIqtcSmKMaITAJMion5hxl1Y2b
jZU2WpXiR0SvosWNZ51lkLTSRrNSXMgTIkamUjhoNJTPPMEUAmBUc2erNjti9fP8oVF3DBlZrUJH
NeZSrzRvwQHxnoU+Y8SIDY1GysgS9ViHg2ikVSlxxIZGI+Wb8GjlqvUGKFFejw2JfQRekUSQyhob
EmElvzLIbNBjQ3KiBZ33rCXsYi4jB3l3WnnaqKSXAUgTDQ68o8UFndmADaKRNqVpxZq80ws8Hh+k
HhsS12E25/GNxgbe2Sbe8I63TZ99VoM8xsqP4TpbhcWTKbE4aawxCGeugDH2Tf7Lt75yKSVQIlUN
PZnZTQrAGWxmRl3aXLXmrN6Khd3w9OWk0Zt1LLVPXoiPg8o9arxgCmAo8jxp0psQHOiqKiu5fBxw
3fOsSzSjs/c6nk/aouDJ+RwBCfva1gT8EkTM+yVc/wrp4IG24MLiEfaijrp24Yjn8tonbYwhbshh
PpukTjtK0RPWOQMr2+TXGhYkrZmlTPDkz9fFpPDEsq1tsLYTa5j1jYoV3lMQeu8pLFH159MsfeaA
Luo5gelirTCUNXgYyhw8DnXPTQx7GLuc9tKlcGFR1zty0b2wc+6KfkGFEY2djPPW+YdbHbnob54s
Aaxo14MAwjMcLILT02/JvSRgR4cNYd7nYTCCBAPvGuANgv6qMkTklsxvJSUVYYKIeYincYn1Gjuv
mhVVoJ5tUx6gE1gz/cE0Jdm93PGM+ZMdS88RSONV/JqBHfra1gLRrtlNBmIHMZ2/aG2u3QYIgXxd
WyQCSjsRzGeV0WlHgNIRhuYEvPOk3wbrS7DdpBdEJg+8sxT84jHxWx5h7QHAovHG1dplytb0HlLo
t/GnnPoz5dQlOUeYoOcEXQ9riaGpwcRQ5GByqHptYxjD2Og0li6Ic2t6whle+I0BSkaTEdtENV44
w/kXkb87/h+gc1K6CmVuZHN0cmVhbQplbmRvYmoKCjExOTkgMCBvYmoKNTM5NgplbmRvYmoKCjEy
NDAgMCBvYmoKPDwvTGVuZ3RoIDEyNDEgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4
nJ1X24rkNhB991f4eaE1qipdwRj6Mg0JWchCQx5CHrK7mVzYJGRYCPn7lK7WeNXu6aXBbcmSXHXO
qYulgPHf4WH//Pn3p58/fB4Pb4/DP6McpZDoRgteeAujklpIhePzL8MPb8a/BhjD7/nXQaZHdvxz
SGuxTHway0Q4Koxlc1cX/TY8vRke+aXtSdgsCIMdb5QQNza3L894+H6aHt4evzmNcp4Pp+wF/9jK
w2UA1MI5O5JSQho3Xj6OD2caCfk4Gi9PP06IBITzDifpJM0wyVO4qHAJQ3Dzjm8dUlxj8hROAPV5
2As6buRZh+ewBc6z1hMe6TiTmeShOXxfDudZNf90+XZ4vAzvIh4P373/VDyCnkfsiJZqJAkCNCaH
1GiF9pb9mSTMlz/a4w5/f/yvHIi9Ay1DC2ZErwWAKgiBZYRcRmjGiSBcEI/BRzvviCGSOrpO0vEz
9mynJtDBKweHuMLIxwjeWSbw8g60132mrolOsKtoURhmPrsMPAr2ss+44bPqykKyHAhGNFZY8zU+
x0ky0Wfk6UQ5LEAonnThnpWiI0LboPCSCsoVY7UU2mHHWtLhlenF8jDzS8qRQbFf2kZ5KfHi+OQq
HXqDDnQch/pOOswmHUgC0XY8LNhFtx6TR2y+imOdHY2DY7lGDtLCU9Zp9jYgQy4hkMIbdeArcuUy
LbfpkEZI3dOOirZmE/L7KvhBFmGgG5viTLaoLIopKaospJOom5Sl6Dpb9jpb4EhohXey5bbYAj7Z
+B4CX4KblJ5zaUoYASad+UoLFqgWQeOUEFoEnVJMEnLdsIrXG9SBAaGhq7Mc5llXbCnEGJcmvauE
bKQw8wH7YIB8qT7cDCu/QRR5zsd38gT94lfcJSXAUE+qWX9Grgk4x+moQY0J/FP2HXOcuMxSPkM3
4NzCH4zwrqeciGU6j5qXYh6V/31510bx7FbPhLFXQll3L8bd6llccl4Ygq+OhXOMhZifuB4A5jxH
SwBUlS81iGxqTV4EzLoC3KCCa6rSPWmUdBl1TT79b2kaNkq34pZQmXsB3yzdirtA766GMFvTlIjX
ZZYSBmmhiZkgU9Kc0ladWyXCCt+r10XHtdhUSjcB7hZjMloY47l/M8IuicMJkLcbQuiWY2ZbgGPz
PQoyS0tYW6NdzLXhmqsUWAYlii7mSAoQpujN3c8OKqAQV4YLQC5oOJXttNkcQrfAkefY42yChoT3
VWQklKfbIuuWOMWfDcANNRop/Mp/sNw/UfIhaiLF4C5qK/vMfb1eUAiLIWRRDPktfUVkCCRWxLZy
WbdeVL/5a4ZPv9Nx7JaM6jh6oUivXKfUYgUKcUUhTY0C0vfRPvdeOK3EkTe5nMIpSYCTHlXxaMvC
bQAmHeNwjgIqkF+JvuqD9AIdrHxoDEoG6GJAybO72IthuzKwVHsxyPxle6+yht0KVFjj2BdemntZ
6xah4jFYK7hhWQdsriJFdZgJYOlDw1XsYQK+KVx7YY4qAXXqBW8TApTOWZDNC24QBtpyKsM1YbQy
O+lqYSw/us5DtzBVHjCoxN7LQ7c2VUfQ8L9bR4+bM2AReQzwNJHkmsSJhxmnkijzesCUTKmSFBuC
XUZgQZvbgteEB4TvF0VrIzdyGIdeUAj5YIhMGriOer9aZdRtYNrdC3q3WhV/rBLarpP1q4oVHlNi
A6xaPacvxVrDKhUpOJpc5m4jrbndxHUZlZAI3ddOpI5bdcfzV7VlhiXPQltCDjGH5iN7ReXd+D/g
u1iCCmVuZHN0cmVhbQplbmRvYmoKCjEyNDEgMCBvYmoKMTMxNQplbmRvYmoKCjEyODkgMCBvYmoK
PDwvTGVuZ3RoIDEyOTAgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJVW24rcOBB9
91foOTBuVeliG4yh3e6GhA1soCEPIQ/ZnZ1NQi5kCCz791FV6Wb3TE+GBrcsS6WqOqdOSbeg/mt2
+/ufn+4+/P1Tza8PzQ+llW419qqDoR06UFa7VltU9/80b1+obw0o+t3/22j51KmvjazFNPFFpQky
Re+6GuVFH5u7F80xHFpbwmoBvdyEjRp4YzVc29j9OY6714eXi9LTNC8xivALXs7nBs0QIvDKWNtq
36vzrdqdjDIYzBl1vns34mm6gRFnfqLu5V37CUboaUhPHAFAhyk9g6UvSGN6hNn351fN8dy84Xh2
f/z1JXkED3kUHHHaKhz61oeI2CGr+hZCTOe7UcN0/lybm7/f/p8M4kMGu5Aa8MGgbdHbFCH0HB10
eplw1HYyo+ZA9AnoBRw6+nCgD2aiADlY4DU80VOMvAdPIT/hD/nlgCYv0jPlYMn7Zb1h03zwrF05
2tCw+MOnuHiE2PTkWh+9kMQPkm9D+Y42KvuGN4hl+zgU5hoU3rbG9c+Dwl6DwhPjYQOF8RynlqxE
akkQRyGSqXIqowIbzBVKYX8OXu8nSRVIRiR7UFLOmeLspB2ScAbTyPMUfvOEq02M7J7PZjciksXD
QwYh+p/2I78bOWyWwArqLvsx156Cn8gRnoDJZc4wIwh58oUXZ5AfTn6o+N53m+SjxXLuxmfxJaaQ
vGC2mXCcGVcxkCsSdZVzV7GQZ4jJpmYyVPVkzOMcddc4qoOo+mfKhb/G0aCiHi/kYq4IE0ufiXoi
RFLlw1JXaSy9DBiWhBBLjoXYaZTIKAewlarAmVuLAA5dpGBF3AJR/CSz7sI2qzpE9tqynuGn/XhI
8ZaK48ijzyvDl4IIXENwTRBXmXiKudC71l4QV+caLzx9JFzTlRk8kH8uMT0l9CZ5aFM9Zq0NuTah
G3R0RimTfVXzzo5GX6Y85U/K2OkqeOkG3UrPQrp4Y3ZLzvDZ1VM0v6yJwgIRe5f1pQBTPwPHTMVM
R/ZjEDMFmuTYE1h415rwvwHjGMLz2a2NbPdFUHGTGl+ke5EoSg9cQfiA9CYblW5W5SYnST8Ak1N6
yIsxQ1k0d1VPqbPsi1LpLHTcbS8wL+b6UlGVTyfSi8RBlsNKMi57W4GrqPLv4mRdC37bakvv4aCk
56IUUqJaRdm+ohrFd02kuysiDeBadP55It1fEelw52wHGLYsfJ74pGsfdZ8hJz5fwGoclnwhS+19
q7VF7AsdC3thjvBmWZmr8tsXgjzdRmBILSFuf/TOl3dgdcVIk5tdqyJ6iluDbruLC3UJWfyuykYo
ZWvykii7CfKRtWqvKPZG/QI/Ou8SCmVuZHN0cmVhbQplbmRvYmoKCjEyOTAgMCBvYmoKMTAwMApl
bmRvYmoKCjEzMDggMCBvYmoKPDwvTGVuZ3RoIDEzMDkgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+
CnN0cmVhbQp4nFWPUUsDMRCE3/dXzHOhuZ1cYho4At61BYWChYAP4oPYngoqeBT8++auVk4Wwuzk
28lGDfEt1fVweuufnk9od518QaFG7QqB0cRAOPVGncVwlPsFPoUYa3gRPV8FfMiZtRfjHRdjjBp7
nak/6FX6hWzKo/MkOwPGZlkGldPgTP7PqO6aptp1N2toSu369xelypZtFtbORBvBFU30HvmAauvh
gomRyP1Dw6A2sdErenbsNGpdTqfbYpJaF8m0LMBE1ekx38omy37afo8f1JRFLgplbmRzdHJlYW0K
ZW5kb2JqCgoxMzA5IDAgb2JqCjIwOQplbmRvYmoKCjEzMTMgMCBvYmoKPDwvTGVuZ3RoIDEzMTQg
MCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nKVUW6vTQBB+z6/Y5wPdzmVvgbDQtBUU
D3gg4IP4oB7rhaNiOSD+e3c22SQtbaBIYNjLzGS+b74d0Kj+VOvN8fnb4cOnZ9Xeb6vfChRooKA8
1rr2qAxYDYbU8XP19k79rFDJd/xSQX/l1Y+q96Vy8KTKgaSSPcxWo9PX6nBX7dNP55lo5iCbVQoE
zIGz5WmO9ZumWd9vX+4UxNjuBhTpS1W2XUWpGKfYGA0uqO5RrV+wYkrJWHWHdw26uMIGOCazSQZJ
VpPZiTHjNiCLP5rxOICfMpj4vntV7bvqIWNbv/74VKrDS9WlsiwYRXXQLqHL5RkVNCZ83aEBjN33
ebr21+PfkpAuJfSJJnQpodHkTMGLIWOlTaSGdmQjS73JtMgJITcYEiLZQ4grElJo8JCQxAkL/myo
yREJKg1nOQScLLdyuB1vJAPagR6WQFrkiJc4ckazDbdxZJY4ciJLPOMIfCo1V5lKx4ys7bHtY4Zx
tXi7VDyDxvTPm4p3S8WT1d7XZ8XnvmHWI7bcSrc5Q8gtYhKbuk8FWW69KXD7HuVt39LNeCiNQzuI
Q/goohhCrpLiF0jB2urg+TZSwgIpWKO27C6R0jdVAOGkY5ppkudiJogn8MuzCL0ERBP9bCivZNI9
T0kM7BFGSnejoAYryYY+cB/OY46rfNZLfDrUjm+cInhxahZCrddplPw/ob33NDdOWM1qyu/NZs2O
A2UQGQY5QXHE4mjcwqRdGrVIXnO4cYzg0qxFYg32fI7gfnxbGcNmeHCl0UJIr6cw3ZiJsHEMncB8
UP8AeP2wawplbmRzdHJlYW0KZW5kb2JqCgoxMzE0IDAgb2JqCjYwMgplbmRvYmoKCjEzMzggMCBv
YmoKPDwvTGVuZ3RoIDEzMzkgMCBSL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nJ1X24rk
NhB991foeWHUVSXJlsA0+NINCVnIQkMeQh422UwubBIyLIT8fUpXy263s7M0eCxZKlWdOnVKAxLF
P81pePn02/P7nz6J8e3U/C1AgASyokMnXYdCg5GgSbz83Hz3RvzZoPC/l18aiJ868UcT11Ke+Cjy
hDflx1C9lUW/Ns9vmgsfWluiaoEfPPFGwLCxel3bOH3b96e301ezgPN5nFMU/GMvx1uDZKUjobSW
0Fpx+yBOVyUUsTUlbs/f9zScn7CnOTzNGXtQ/jGi8jMw8wCtf0Ud5sFiG76EZYP/TP5tefgtoMvQ
Quv/TOVB+RA00Uja8MPt6+Zya94FWE7f/PgxB4Z7gXFABrQgZ2XLwITAtLASGZrbcw94vv1emxv/
+vBvNkh7BjtGGFs2qCW1OiOFNqNEDBIDpLzvKgDEjisGh4FRAZjzE/UMDqUVfgtjo3zE4UF92MGh
UpqLW4xyZcFUVl0QyhKCbDAdTt1juNRudFYaEtQaaa3LaKGS2qkAFx3ApXeJBcwohWySZOfcBi+c
MESpK4RCaK2foDAbIQh4DQXWAxqYg7iUlpr0K+NqD+NSKJUym7igYzdzXOhTw4lvQ7oiqx963x14
DyRd+0rn7ZHz6Jy03V1OwGRKoiH/DqGy6ZIpPQQ+Jj4nrvsQfbHGaCMdNXqyhmoIj5js7TLmcYIq
jctSNNqDpg7wcgdFjy0rbKtfV/S4q4+p6pELo6V7xL6YxRkDDEIXDJggmxFxr5vkdY82KKsAGisu
VqAmUzaKJWVtSFgPxYFl7Z1gJBNTDEKvz5iyDwqyj/573joXF6fV6uhQPI+CK1VZ1IxKFRLn7JLz
/VRoI3WntqkIfptVQA/sIIJ02q4NddFQjxdAMDAzhAo1bQlTNZ2jroPITSKLw2cT8KjtIHDFOruJ
Wl1jYnVWGM57RmDVbhINsHSa2DzmnK+UB7UuyEiBJV+V9apqdeljY2QvumrI365ns+LwsDCQyt7S
9DzHcnUosz5wqRuoOm2i2F4LzhWx6sKYa21e6Ph/rLNOdnrb9dEup6F7WAc1oj4f2OHoz1ej91hV
4YYqCcrr3cq+L6GvxAPTlSBdJlKRR9nBDYY5Icndx1ep3ctBYrVxEo19Jal3LwcJVGPYLm5bqIts
XUkTp2xMl0yVHiFEmuhaaVsAwMR81DKTGHKpJDLxOafoYf6JW2XHgda+ZrFgmSDwMtGxeF1Rg6IO
WThgwBkUf7vwT8MAM48Rrjyv/P04zvm9Bbn1oa1pA7o7AHFTZGf3tyVY+bIDeHczWRqTjTBsO1HV
PoqSROinAv2qM6RuRxPgZzY7TP8Z5Dus355rPM3ltmKX9KVrwr0e5TpbKmNF7HfiPznk/v8KZW5k
c3RyZWFtCmVuZG9iagoKMTMzOSAwIG9iagoxMDI5CmVuZG9iagoKMTM2NyAwIG9iago8PC9MZW5n
dGggMTM2OCAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVVti9QwEP7eX5HPwmUz
eWlSKIVNtwuKBx4U/CB+UM/1hVPxOBD/vTOTNputu9WlME0mk5lnZp4kSoL4VW22j09fDu8+PIl4
21c/hRJKKh2Eh0Y2HoRVTiqrxePH6vUz8b0CQd/jp0qlJS++VclWz4oHMSvIFc1VMcpGn6vDs2rA
oKUnXRjQ5AY3KuCNxfDUx+ZV225u++c7obou7qYs8EOUcax042RTO2GslaoOYrwXm70RRqM7I8bD
m1b57gZaZToUFgWAiqqm2TAtvB1fVMNY3THezcv3D3NEOBcRAzllhW6CrBExB7QiSEDM46FV0I1f
S3fxx/3v2aE+59Bj6lCjQyt1becMIDB6iAhQE3LTmgYFQHeD8x0p9XEeSOqIWqNpaBvM0NSsZYU2
JB05Mz0NTSAXQDXgfeAhmph3kUTvPpkl9wPDABIqgsMq4qCndcbIm12e7sg24NhM+9EexZ57kJMA
aowxZIF6TRqXzIFSYA3Oc4/O185brH+zrN2W3VCVEPiZ0DatmKmcjDSmn73MCrPGCvxrZa9jhV1j
hWlkqP0iM23VAKqAy/i3JR3AU5+Nb5ntaDKwpe7BJ2t7tI5AvWI6pdZN7cZy6S3O9U67vBb56BS9
JwGBuqgKKmaEjI9YkRrRz9RILdGZHYmmCVZqHA/D0Su0lvEmOXNmzmaZNcFkwlEaDJ2TYBQYAC1T
mCKriXYJaPw36/Ai9GCWrAspzu5Y31NiMVCHECFhCfOZ4VMFJWe5KReJ6FaICKGRoP11RKxXiAjB
SbvkIfRg/+KSLZhREAq5pGaWntT8QpEh1NJ7dxLYp8AtDGDUFiLgWVMGr5A5p8Xj4I2EhYuMPfPq
ilZkdjOz/zcTq7VsdLOSCWYB0Ot9kcmy2X6t2Q5vHaOva3ZYa7bFW6dxF96idLGnK+F4cfa5OnwZ
HFdAZZu0PMwbMfVCve8cPy5TPdPp5EonDzE/B+cOxp34Ay3M8bAKZW5kc3RyZWFtCmVuZG9iagoK
MTM2OCAwIG9iago3MzkKZW5kb2JqCgoxMzg2IDAgb2JqCjw8L0xlbmd0aCAxMzg3IDAgUi9GaWx0
ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJy1Vl2r5DYMfc+v8PPCzViSHTsQApPMDLR0oQsDfSh9
2HZ7+8G2pZeF0n9fS44/ks2k3IclkNixLR/rHMnSLah/mtP55dNvz+9/+qSmt3Pzt9JKtxq9ctC3
vQNltG21QfXyc/PdG/VnA4qfl18aHYec+qOJczH9+KjSDzbFfV218qRfm+c3zTVsWlvCagJ3nsJC
DbKwaq5tnL4dhtPb+auL0uM4XZZThCegnO4NGKPImFZ3Xt0/qNONFGEwRer+/P0A3fgEAzh5TzTx
F2l8wkF7Qv6iHcMQ6At/THhpeVFueX3OP2SS1R7EkO64i9X0H+5fN9d7804Ofvrmx48JOuxBD6it
Ngp733bh6ILeKN9COPz9edAw3n+vzU1/ffg3GcQ9gy74ELpg0LTYmeQO8MkVyEBxgLOgv3DfjLS0
eERP4ozoH3lfeNyMqU8unnZZSTIVZn0FEyeSWMfFJoa+ieYsj8vCM0/ikTAcLOq0G9QYxAiY1A/L
BWJYNEXMAgcntiA8mp6bgW4aMBIrNFteA0KUEwHEgWQgwi7YZC9feNz3rzOBo37j34yFChbcYgk7
Ou3wFlVJfgQ70DyCT3j66JkpdK0LEtC3wgi7FBYdZm/E+SZ6+6H+6Eh/4YvavE5/5kh/1Le+c1v/
RK1g1lJCTul8hX2vr/zR+Zhy4IWmlSDBjysp26wNoOQhmXIN1qj2XFGUyfT7QlYEwmiRbeJ6OrrH
vra7rvGtRYUArekxuRoo9Eh8jQe+7nbzng45j0Ch9q3RuHE2CyerhtgXO5JZIjDGHlAK4ezaPnog
LPdwyTRBr+fsyQ1NppjrYqKgHO9LOoivQljRQwyYSs9b6Rd8hbaHsZr8A963aPxWjFdJh5wTGG3O
CylhXHNekbCGmWfJEN44apfBOsbDGlpumjqnVbGe8lO0U9KF7B5zxiyWohFegKbYwuEAc1orq0Lk
8s6C3NUGYi42MhzbRI+F7B4LGWwIcUevFLI/EjLYrnWevoSQ5/Kn5Atp5cyThlnzMdY5/lEv12Wd
reJetQRZA0eCWsTxmaDW7MoamNDIVQhREBvyoKLwmLz+gDzklNG/kjzYL78Se2hbgu2dGC67iWyu
LkoSSQRWnaLwvTKhHu42ZkrWySzkem/FAq4rhRx/Wjwed6jDNViQCWjARRofcJxp+r9kFCpb+Kwy
W7CIMoFuEX1KNpgqGmDgkFNEzEIVcjMdyQGOKlBnW7T96woAOKpAw6fHnTtpZqbmfBVIqrVL5be6
FXwtEr64iUmT1kpAS/AZLCWcVKzLbV9qunN17xNLaFVorgsSXJURpYaNebqqjleVxZx3uO7gLBKP
F0I6m5Tipc5dvNHl5HLVG1Lfqf8AN5b/2wplbmRzdHJlYW0KZW5kb2JqCgoxMzg3IDAgb2JqCjEw
MjcKZW5kb2JqCgoxNDEyIDAgb2JqCjw8L0xlbmd0aCAxNDEzIDAgUi9GaWx0ZXIvRmxhdGVEZWNv
ZGU+PgpzdHJlYW0KeJyVVt1r2zAQf/dfoedCHN2dZMsQDHGcwMYKKwT2MPawrcs+6DZWCmP//U5f
ZzlpvbUF15ZOJ/0+7lRdg/pdrbf3D19P7z8+qOF6V/1SWulao1MtdHXXgjLa1tqguv9UvblSPypQ
/vf+c6XjVKu+VzEW88CdygM+lf/WxZsEfalOV9WeNy0zYRHgP1a8UENYWLzOc6xfbzbr692LUem+
H8aEgn/5lMOxAq1ra6wiY2rdOHW8VesDKUJOR+p4eruBpl/BRlPPjy0/AP3b9Bj9w8inQ50j45pB
O3D9CuVzlKmdPCguNXrvNwMtOX0w+B8XJuBy4bvjy2p/rG4CW+tXH+4yXngML8O02ijsXN0wXwGu
Ua4GZux42mjoj9/KdMPP2z85IT6WsGXioeGEpsbGZP7ABe4QA3DT0wZtj547YhBhMHDCZJEnizwS
HhwCD34GHI9Q5CvP7vzbrg9UUs5ggjz7PtBGfjP0tGHYB9Ju4OLXZZ54Ep456EaWG0kPtsgVh4xH
gp59nmwknw9jC6TD8VQUPRwsIh7CnmEl82zzztCRlTM7P7mPSKg4iqj8OPutYQW7c/a3fruRmafE
2AAU6GQ6TCQ3gsSInB+2z9JovyyEgTWNDEcJbYG0PLLQkSDTmd4R6SS1FTHN00amJSPzX9TmeUY2
S0amrnZNe0YlHHorHp0hHgWBnWtWWNQVc97rkW8UMxxCMI4hMXaJf08jdWIKmhszpgyeIZn2nObC
iWo12VnR3ZiTRGFTaCOl6TTGgVCVkzayaxEqpYp7dIGMeHRItSXMwFa8wORlN/zD0XxJtEDnMuwF
9KygC0ao9LVl/jLqqUilvHK5M5R2QywFQELopqqVMFrotXbBouDYUbp9nkWbBYuCs7W5cGjL3Nrg
l6hh0oAy3rIMo1ECLZON7AQ1Nc0xcrlkp7ygtJObeBuLcsCZZ0yye9kobGiHU1DRfKVZ6j4fzIi0
k5IXFii7mF/PnexJEdslEa2pnXtmn3FLIlqoEc9VpMMc8RNF0nX8bxfNc7QxB9eIBs3EAcEWHAKM
2PD7TlvSMGgDBvgaII6Rk8+TE9ranWeXExaSzgjGeE3I8KFPtRRjzXTRWjFmql3rPI0rWOxBuf1Q
8m268CXrtBXO7Jas7GeI+9BZ3S83IUBuQgbONfrfO+xG/QV+znA4CmVuZHN0cmVhbQplbmRvYmoK
CjE0MTMgMCBvYmoKODgyCmVuZG9iagoKMTQzMSAwIG9iago8PC9MZW5ndGggMTQzMiAwIFIvRmls
dGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicxVXbitwwDH3PV/i5sB5JvgZCYDIXaOlCFwb6UPrQ
dju9sG3pslD695XlOMmkm9ndvpQBjyUrx0fSsQ0a1a9qtb69+3J89+FOdZeb6qcCBRooqoC1rgMq
C06DJXX7sXr9TH2vUKXf7acK8lJQ36ocS8Vxo4ojQSUbJrMh6HN1fFbteNMpEk0CknHBHwLKh5Pp
KcbqVdOsLjfPtwrattv2WfCPWXaHCtFpT04ZazX4qA7XarU3yhDDGXU4vmnQtxfYgGl5WPOAlGbj
sE2DHcyYI0evmB3EjIPJh850AhrBF0SM2ZPCqIRZ3749vKh2h+pKirF6+f6mpIP3pcNZOLCK6qBr
yslYFTVyPQ7HBrA9fJ2idT+ufxc8ug8vcFnRM57V5G2pDsZcmcBpUkrTpDR5FhN3cGxjckqeKG4e
95ysSdHlE4lGJyCIYmBfaw7ZJIDIQUMFTqkxHx2iPeUWMrcGd4Dg0IPFiBYNcyqZn6IYj7qeo4wZ
0sDYZcYLZEztdc1FPkcmwvoRhBwLMsyRBkKTElKuOhcJkikLm6GSsjIpM0uXl2ruhnGt843NUYsJ
lcYH0iGEGQuGoCb3XrZxw6zre+2HJlNZY4FjISfL3WjLemHEo+kR+6jJBnb5QJhzB8KSxhqfdiLs
uRNhonbGzQuzn4i5L8XAnZUuydFe/JL+2MlOOohOopYkT3ysjT3dvKiMgiiLY1hte1yzvWOdpf81
n73BBoMdK3JJfzYGjfM9Rv3tTT0Iy7Z/68zEoXOSlpQjnnY6q4Qef7pKzTHwaxNnlLi1ozxEk7u0
HUw0uS37sNcsHJVydc2VnDzjZTbccVOB97efnK6c+rYov7/Q/l3U7oyoMXpt/ROveX9G1BhJR3hI
1AtNqmtujjnFGG/ApDxWZweEhAa3gMjvtGiVwIAF5PmW5zuei1oN9DZvb3BZr95rmu/7eL1CyB2R
Dsf+FbJZACmUNkTSwgflc167yO+MwXpObxTt+IiWJ5P+i/iu1B/veSdUCmVuZHN0cmVhbQplbmRv
YmoKCjE0MzIgMCBvYmoKNzU4CmVuZG9iagoKMTQ0NyAwIG9iago8PC9MZW5ndGggMTQ0OCAwIFIv
RmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCniclVbfi9w2EH73X6Hng9NqZiRbhmVh7b2FlgZ6
sNCHkockl0tSrgk9AqX/fUca/bKz53As2PJIGs18882nNRrUv93u+Pz9y+O7D9/V9Gbu/lFGGW3Q
qwFGPQ6grHHaWFTPH7s/btTXDlT4PX/qjEwN6u9O1mI2PKlsCK7Ct2lGZdHn7vGmu+NDW0/YLAgf
t7zRQNzYDJc+dr/v97s38y8nZQ6H6ZSy4B9HOV06dE7D4BRZq03v1eVB7c6kCNkdqcvjn3u4O8Ae
LD8MhtHRnMJ4Dg8KBn+45aEXc1znzRBtJJa3l1+7u0t3H/PZ/fb+KUcE1yLiQJyxCkeve84oBmSV
18A5XR73Bg6Xv1p307eH/7JDvOZwYGigZ4dWY29zhuBjdmbgKDFESZwJQIibk2XLJAnyiBPEAAOF
GQowYHjw6JS3yqcsxOIPwjf74KlJJkkOINNHI8mKWVwnr2KkQTbaqxYJFVNAk0zEEzCUy5xlIviU
nMwkB4Sj+IEp9hTbMbz6Jk5wcT5lIBV2eQNP+lrU62APlgs2rsDGcwwjQxgR7sMxbsslOK9HGpc+
B/HJZWEwj2BhMggIBCdDBtkyAfIbOOpMl6VXsqjd2muJVKJ0h1X5IizUwJLnbQGL2wH26Jqy3IEJ
r0Iz45afUrZzRKLFRLBecKiu55Jl80lW/7QkDnWPtC6JTSeU09vkgA65HZqjFy1Tdqy6hujlvqet
vkfUAwyv63u71ffgNflV2tTXpOfcF+BTgrETzhWLiHAUs6ZpWpyoZcq18kZjRAgcTfzCGeamR33B
ea57QmC1XeKaIjHL5qYCfVGZHLHUVXRn5THLipQseUySlET/hfiqLApWx9LTVPSFsixyjC8ywW0w
AbzXlq+mVzGh32ACeG71H6iQhT2pr5AgaHRNXyysv3P5+KnKV/wXQl/x3xT6gDLOzh5uqXAnCG+s
Yr+4enAWnvV1QfDnXHBlfzwX965vbgvJQa4RVpF+WePAKSY3tFLjy2emdmyRSottHYKe9OjXOlQ5
k9RkrO236iUKZ8VHygHjqvZaFIiiZWyovSFJwxYRif8bDf51RPRbRMRRDxbWEEAkFRzrnUuDDUSx
VHgnd0NkzzGSYyk66XIWPHLN0wUUzdbXhYnHmWmplIlp11iGmWWFYVQZ1jYRM2yB9L36H0NrcX0K
ZW5kc3RyZWFtCmVuZG9iagoKMTQ0OCAwIG9iago4ODMKZW5kb2JqCgoxNDY2IDAgb2JqCjw8L0xl
bmd0aCAxNDY3IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJxVT9GKAjEMfM9XzLNw
3aS73VooBXdV8EBQKNyD+CDqegd3By6Cv2+rp+wRCJPJZJKwElypmPSXr263v6BZtnQGgxXrMaw4
5aygYqO40uiP9DHCLwly9CfiR8vihx5a/SS+8SSyVa55gF6iT+pGNEtLh056IMjFWxpkuQ8O4H+P
YuV9sWwXU3AIzfTvixTpyiaS2FrVabmMRTljEA8o5gaVVc4JYrfx2rDjeRDPOqdK18GWXoStNInQ
VsoEDZfShm18p1mk9f3yNW7GWUR1CmVuZHN0cmVhbQplbmRvYmoKCjE0NjcgMCBvYmoKMjA3CmVu
ZG9iagoKMTQ3MSAwIG9iago8PC9MZW5ndGggMTQ3MiAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlPj4K
c3RyZWFtCniclZJfS8MwFMXf8ynu82DpvTdpmkEJrOsGigMHBR/EB3XWP0zFMRC/vbntOurQygik
9yQ3J+dXgprgUyXT7e65vr3fQbGcqQ9AQI3sIaOJnmQEFlONlmH7oK5G8KYIZGwfFbZbGbyqtpe7
hQ10C2IlGnvVoelJ1SM1j5f2nbjXIGIcDyI1B3vlT4/kMs+T5eysBAyhKPcUccSURaXYeo2GwFir
0Xmo1pAsDBiOdgaq+jqneaAcyziRCWPO0Yj2tAhpmpMVUdCUURqwVSyfZofDTXWu5pVaNTDJxd2m
i0O/xYkpUrTAE69dxGnSWPCaIlBV50iheunbFe/rr86QfzPM4n8hFw2tZmc7PPINGrdALnBOXvJT
o73M/QUXjGDRHt8I3J9cZojLWW1SfxqXHeJy8nboiAuzmJIlpdnT4FTKBpSl+g8iHYIwqCnefRKE
G4LgVGfZ5AiCJG3LIeHb3CyyqcoD4NEbW8E3y/TQjwplbmRzdHJlYW0KZW5kb2JqCgoxNDcyIDAg
b2JqCjM2NgplbmRvYmoKCjE0ODcgMCBvYmoKPDwvTGVuZ3RoIDE0ODggMCBSL0ZpbHRlci9GbGF0
ZURlY29kZT4+CnN0cmVhbQp4nFVQ0UrEMBB836+Y5wPT3TRpslAK7d0JCgceBHw4fBC1KqhgOfD3
TXp3UllIdiazk91lI/ihqp+O7+Pj0xHDbk3fYLBhGxFEjQaBY2/YWUwvdL/CFwlKTK/Ep6eATzpp
7YX4wIUoVgXzIvsTvdG4om3+dOlkF4ICrnIhy1y4SP97VHdtW+3WNxtw1w2b8xQ5cpdDosYajQ0k
ilHvkZ5RXXu4YFQFaTy0NooX7qS1Lh/ipOfIyn0BNdt85QUo1zIURjhII3FWsmZ4ltTdQ7qlbaL9
PNQev0UbSfoKZW5kc3RyZWFtCmVuZG9iagoKMTQ4OCAwIG9iagoyMTkKZW5kb2JqCgoxNDkyIDAg
b2JqCjw8L0xlbmd0aCAxNDkzIDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJyVkm9L
w0AMxt/fp8jrwa5J7tpe4Sis2wTFgYOCL8QX6px/mIpjIH57k6sdnYyBFK65p7mn+SWHluDLZJPt
7mV997CDZjE1n4CAFjlASZWtSgKPuUXPsH001yN4NwT6bJ8Mdp9KeDNdLvfCBnpBrXSPg2if9GzW
IzOXnw6deJCgm7EcREoHB+GhR3YVY7aYns8A67qZ/VLII1U2raGysJWrwHlvsQjQriA7c+BY7By0
65vIRFxTpFCPZZ1pSFTpBqeyQVVwXo85EmrodAnM6DV1QvrChnIs0tGU34l+n+7r2/bCzFuzTMzZ
5f2mr5qOVS3F5uiBq2ALoU5FewiWhLtdR6S6fR3aNR+r796QjxmW0j4qxNBbLnzfBQqpA66pOXIu
izRBGRQWGyndRaFyUeBC0srUlqSfQHKnkApvXR7+h+RPIRV6u+gPEpYyB9YJsk4w1cvRlWl2aZbc
i5hrlETuyF3XDqcDlyhdgOYQdwk/Q+O0lwplbmRzdHJlYW0KZW5kb2JqCgoxNDkzIDAgb2JqCjM3
MwplbmRvYmoKCjE1MDUgMCBvYmoKPDwvTGVuZ3RoIDE1MDYgMCBSL0ZpbHRlci9GbGF0ZURlY29k
ZT4+CnN0cmVhbQp4nNVUS2vcMBC++1foHIhWMyNZMhjDetcLLQ00YOih9NA23T5IWxoCpf++MyMp
8YbdhdBTMdiah+bxfTN2FszvZrW+u/+6f//x3oxXm+aXccZZh8lE6GwXwXgXrPNo7j41by7MjwaM
PHefG5dN0Xxvsi9Wxa2pCgklslucHpy+NPuLZuKky0i4cBDhki860IuL42GM1eu+X11tXmyNG4Zx
W7rgh6sc5wZDtDF4Q95b1yYz35jVjgwhhyMz79/2NA6X0GMYoIckRwA3OmLRtaILLrko+qzzw7v5
ZTPNzbWWv3r14bYWAMcK4LzBeYNdsi03oPm9SRa4hXnfOxjmb8tw48+bPzUgPgS00LqYPAFFFoKD
LsssRAy+E0VJGBkpaDmht9j62jAkbRbcQNzgcInlzX2zImEUgUD6JDVQGLB3W3nt+EO9z+rIMJDA
QHKXraOA5PnkoxhIdJPo1D/fAnFnVJE/J8IX9Y5BV4PmwD5INixazAlOEkDnCGi9pZCeR4D/FwJa
GXv4zwhwMQcQf8Wedjr7bc0ZxKyEkBCCvZbDm3OSlXCOFWJkGaxnsdIeC1hRx2Bj7J6gjqgg8Cor
5F7hAK1+VKy0CdzQZqDY6/pXJPMvASsmGdRsH3M4pwwqwBPjUPkruSboWFVYGP3CVO9VulJmH9Yl
A2bUy7BgzpjBzp5OKMytBK8Lsqllax2Q/OOIVQYP66Uz2xTP8AZdsCnS83hLZ3iDDmyg9ilvaxm2
bW4gYw6k0wqpjnSGGRUQae+AIQFTp7eA/AhO4UAW4PhciM90Yi6oXD87FxTrYGldZSoe/my5ns1Q
uQ11l2jxX1DLOlNbL0+HlF2bv+2GsD8KZW5kc3RyZWFtCmVuZG9iagoKMTUwNiAwIG9iago2MzkK
ZW5kb2JqCgoxNTI0IDAgb2JqCjw8L0xlbmd0aCAxNTI1IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGU+
PgpzdHJlYW0KeJztWlur5DYSfu9f4edAe6SSbNlwaOjbgYQNbOBAHpZ9CEkmWdgEdhjYvx9JpbpI
LXvOzIHkJTSYliVLpbp8dZHMaIf/H96dP3z8z/sffvw4XL69Hv43mMGMBpYh2HVcgx28mUbjYfjw
8+H7r4bfD3ZIvw+/HAx2heG3A44FevHfgV6kqVLbqH886NfD+68O97iongnUgNQ4xg+NzR+qv/Uc
7/759PTu2+vXt8GcTpdb2UX8RSovLwdw8+jneXDej2ZehpefhnfPbnAQp3PDy/t/PZlwOtonczvF
x/VE/3x8uNgDT2axQG/MlB4LWOPiH0gta9K/Wx55Of375ZvD/eXwXd6akGZ7pPl5DJFNsNjRg0PK
YAjjunokLFOzJOrsc1opUQAmtzNFc3p5y21DxNk1txf8NI63eXe5C7bIgz3yAoywTo/kAchyN+bN
LO20vL3L8ob7M/fydiJNR6I0s9SndmasyXt0uT1TGwKPj0vT/PlvntQAt2mSI4stvczsRM55fDA7
F2YnrCchuv7eMP1In6J/wo+4fRZ6gNpwqcUFk9A/8/dxVHxci2LF5z39TTrgcP2wJUnHkhztbMLi
nXUhNiZjV2zHRoDJr+lFI+nZxY+WjqRXUbQbSapIMiSiLD5SvyVJKE6C551C/n4Wzi/0fVFU4ayr
OYfzI1Mn1pz8aSZKS2pqxgOPJyFU40EMh4RAkiIhZBmk9TZtyO/Z0BTBxz8y1mZq1lrFikmxiphW
ZZd6vPHMOFcetclNwkhRMdzowoLB8TdRcWakMlHUU1zffykiIX24lTTIXhqTBp4f6UNIcF+GeEwv
sGJeav65PL/LgJ8/UhCzCL88fy+QR5vYUIlpTyWiP1ptB/WdE+MwNc9wecM6AIZ51sCUbndh2vMe
FU+znWodKojJsJT3bBmWBLaRXhCZGDZOwm6SORr/peZxoQ94P0qGsj7231im5EGoDQw+BZYd9Uez
r2C68HdlG/FsEy613SzjZ6Yv/yXfUPd71jHSiWp93E+cdUNT5j1NcdMYlo4DtvcTw6Rhzoh15fjB
igPM/WpnLmG3Qc6Hk4izdkji0EE0DSXpeWewsuaJJMmEyTo1ZyZUog12hD12wDzOU9gNl5yAiWHj
JWtR4LGx/LK3vA3j1DPblZnpMqon4bis5u5KzEHhCPQWn1NQi/sFqsRnocTOombi81BYxda4Daym
jqMxh+1A44vasvCKGWRluYtZTSw8xT1fw0DWIKtgYZX9iau5NtHRwusr5RCYKmbLUblDH/zM+5H1
nEhf+FWUW1zNTZTvjBaS+q849Z6BrnsqYZbRG+hopBWeT6eiGBixZnLdLAbiiIeF5nu9J/KMFXST
D6rcr4LGDOUaKosV1+HBIu5VRbzuRBF5M77ItF1PAEK5mu1wxgpU3nr76UfUngGJvFiloxCkbXk/
Er6ouFTiPHQVxCrOEApAMcDJflwdLihAzeGcPbNOb6uT7eaqRZ/suo4QOoCf9clg+HNmm1CAfkbD
Tu1zrV8qvJt5P66krltE7mWtdo0Zht+H4UwE+AbYpE2ucWv9vbTUxqy5a3M3lqnZzOoKeVkdlI6y
zpi50Rlg949ZqG9CQpDvPzNcUzJSOOvFhkDaK7NvYpvS3zc67wR323DI1zip/IYOpzgF0DbWFSf2
e8aESrzslQPTd2v8lMr1sl9R4VrrJw3zD7P8AlQdTJPwroSnEi6q8Pci/XMHg3bCY7cZ+Fu3p7nB
jsvaUV0dH32CtB2j3ctD7QxjmHuJaBCklKUM7xqECw+1G9ayNrH7Qqlv1jbs9Ibihp3cOLsOXn1G
cQMNZie/uJKClq1daDzI1q0AVBtlU3rJgHNjhZ2YPkyqVH5x4fVyV55PRfFtzu/EieBUKqdtABC3
niMVBJS0lDf4OJbiiM+DnlM7/839TpzQK/CfArkHAMv7zVYfekEKJUBE7yfScbuXZdmYj09m/VNN
gwycAZhrCjtGEN5iBDGVdEsHev42gn0jyNmcz/tH/S8vjyUbcXn/z9zGWu1fagQ036tSd0wRL9Kv
Cm2e+r1EVVT421LSvQzaxgQ+qudrq6FLLcTSz0qn0gMqHx0tF43YCYllin/97JR92yj30kNrw2ih
43ly+oBKiOlCyaOTkuUoI9Hrs7/3E+8f/X8hNVNKRucnVpLMCj8z/6LSbB397GYiJlHbAUVda+Go
Gu0NK/6hZrXGA9bHtpCMpOZdqeoFQYPtRblbm9rLXNZlXEIv/Loyey2wevUj2dfsmYCtbn/Rnh/G
S4WjqSiBKv81mNo9D9so13XP86RwrLLhM++/e172UPGSQrCuRjSFUvw0Ax9ifoFT+8lTInVYQkB5
JPhXhW1H5lR0qkc/bfJIMazyQc15HdXcqe3aakpb3dhEEoA3uPdlHYPvAOtneHd1rDMLJ2+ILswZ
qQuR0rPmGfreBxmvcih9bELeFr2xRAOkblXIVaKPzVqk9tZnkRSc5CXRg5p2Z0eB6/va+znRNJ0z
cu0QpKaAkqVjpi3J7uWEixlnO+/ibLeitYOr0FaRfW0j3YoiAdtRhMwuZdPbw17KGZNdv3Q8iFNo
OdXWJMc8upaojsW4ruManSsyyOhhWeewzqIiPI5gClrYej1V91B1JOEkos+2Cb8lTY0Jupvt2wJ0
c5IHg7WcwiH5UkYyfKrVlmpLwA6seBIwv469lumlGxlH4ax9SCDaAN62CYO6DHIVyJEyj8rCZLw6
JWyd+ZX51V5ZwPkbyMDxD8czbFj6+OJa8wP5feVYbi94uBK/Cn1y8i9lSH2yL8752tArZUSryoYN
PQJpKnYGleBNtJ+iL3KT4Cr8FHqxYCCxaXxuWcpeQj65EVzHmSlI9OxW+4caolkPdw4UBD5qyha5
e8eS3o/WdBC8TWqU4NSVh/aW0cSKRfc2WDEXxkE5s1D1YnU8LdcHQI63yVppfiqSE+K3x+868m7r
p7KfbTBc3gCGzo9r6PiOv6sVYqyb1QqsRuRCRK7TFWPMiePziRPdElRX4EAlB4rvVCSN1iMXD9S1
pO34YPfwdBqXqePwXpWco10pmbTnFgrwts+B5HJCe3tRZVPqdqJhwGuzvYerS8rhqTIhw1Yb4+qr
Ruoq0yeynZ2YeLv4AUH4I9dg1LUX4Uc3O2t15uFE4h9f853EbolhXkbj7BDhYJzWme7l2qWcdMZJ
gQ7n4n+XGODyhiCRBU/5Jd36Si9dfBmNxKWt52b8d8E+9xw/cakXksXTPNZFrtB48hgPk9ipLBDo
eygzpY9dEApCmcJhV1kk5KkS6yClszT/3eRrDNcTkX5BqVRc/G74A163xeoKZW5kc3RyZWFtCmVu
ZG9iagoKMTUyNSAwIG9iagoyNDg2CmVuZG9iagoKMTU3NSAwIG9iago8PC9MZW5ndGggMTU3NiAw
IFIvRmlsdGVyL0ZsYXRlRGVjb2RlL0xlbmd0aDEgMzA3MDg+PgpzdHJlYW0KeJzdvQt8VNW1OLzX
3ufM+3HmmUxeM8PkPZOZIZN3mOQAyTAhPAIkkARCEkhIwishCSgPJSggBDRREUVQaEUrvgiIilVL
Wr1aWy3canuv17Zwb623rVK4vdZ/FZJ8e5+ZkIDW+/99/+/3+37fN+Gcsx9rr732a+211n7Q272x
FWlQHyJIXLmuuev6h+fGEELvIQTGlZt6HVdmPTqFui8hhK2rutrWpQc+/gtC5O8Iyfm2tZtXfe96
3yBCapqk0dHe2txylsM5CK120IC8dhpwYvQuOfXXUn9y+7re2z9Jm9FA/dsozva1nSubR4PtIkJr
PqHxL69rvr1rQLaOILT2CPU71jeva839b0Lj1r6OkELs6uzpPYgyKX2911l8V3drV97yn5YgtDGB
0vcrGgb0j/001Cljfkw4XiZXKDVanV4wGE1mizUm1hYXn5CYZHdOcSWnpKalZ2S6PVlen39qdiAn
Ny+/oLCoeFqwpFScPmNmWXloVrhiduWcufPmVy1YuKi6ZvGS2jr0/50f/x7/HrqD34EsaLP0vunH
FSEzug2hsc+Zb+I9uuT/WSoUkc8Z9AY6iY7dFLUH3Unfz90Udg69iZ6VXIfRvd+B9lX0TNR1AB1C
9/xDuNXobornOM1/4tdEQzejR2jOZ9EPaEeZAgGa65po7Mfo3W9HBf8O76IH0NMU8gH0Cn0fpj1v
K/4regAvROvxv5Ad6C60l5bxKHSgAQrfhI7DUrSchkZ+y1Er6rwFaT8aRE+iLXQU3vjxO8b+G2mv
/4BSvpfiOYg60IZJKZ6Gr9iH2CntL6CXpLAd45HyMFmNX8Z45EHquR+10acZPqJ03kumozLeACcQ
EsvramuqFy1cUDV/3tw5lbMrwrNC5WUzZ0wXS0uC04qLCgvy83Kn+n3eLE96WmpKsmuK0x5rNgh6
nVatUirkMp4jGJCn3BVqcgylNg1xqa5wOIv5Xc00oHlSQNOQgwaFboYZcjRJYI6bIUUKueoWSDEC
Kd6ABMExDU3L8jjKXY6h98tcjrNQv6CWuu8tc9U5hi5L7rmSm0uVPFrqcTppCkd5bHuZYwiaHOVD
oU3t/eVNZRTfKbVqpmtmqyrLg06p1NSppq6hdFfXKUgvAcmB08uLTmGk0LJsh0hKeXPLUNWC2vKy
eKezLstTMaRzlUlRaKaEckg2c0guoXR0MNLRPscpz3D//rMCWtHk1rS4WpqX1Q6RZpq2n5T3998z
ZHAPZbjKhjK2fBJLS9465HGVlQ+5GdbKhTfyqZzIEob4FMHl6P8bosVxXf785pDmaIgsRfgbYs4Q
rd7+/pDLEepv6m8+O9a3wuUQXP2nNJr+rnJaw6iqlqY6O/bDffFDof11Q0JTOxRFCxtaWDlkWrC0
dginhBztzTSE/it1OQvinYa6cZiqfxSNaEXQ6qB16nSygu87K6IV1DPUt6A24negFfGnkehz1w3h
JhYzPB5jqWExfeMxN5I3uWhrVi6q7R/iUipaXOW0jvc1D/WtoP1pNWsKlzCk+zLe6eo3GhyFvjoJ
1kGpqmjpcAzxqbRaaKrJCWhPYUn6Bcmj+zLyuRxPM0g1GB2FLoqG4Sl3lTdF/21qj6UIHFmeobA7
0vTVtUNiGXWIzdE2Kj/l99EUzU20iTrKpOYb8rm6hsyuGTfak5FV3rGoVkoSTTZknjmEmlZGUw35
ystYzo7y/qayCAkMl2tB7asoMHbpVI4j/sUAykF1ZQzYOpP2q9Ty/tqWVUP2pvgWOtJWOWrjnUNi
HW3gOldtax3raLSGMi7R7JxSjkN4ZnVt5SJX5YL62oIoIZEIho5LKb8Fjas2PoKGdrkhRYrCUYvj
SR0FFGiAI0QdrhnT6HtInqKgj0ArXAplXXXGNEctxKNxaErGUIajvLUsCsf8NyHlWXeaGR7HJmNe
imdmON5Z54z8sjyYRjuiGdMUClap4fEokkI5AQ3DFI0UxOoylvV5R62r1VXnancMiVW1rGyseqRa
jlaGVOfRtqq+yTepsmg1ISeNHvewyhwKueMnV+7QLMl/wxu+JbpiPNrRr3BVLupnyF1RhIhSXjGE
WBcWCwzx0uhn49kVaqaDmI5oaTz3nxJFNpbb2bDtd1W09LsW1U6ToCkHuSN+C8vLiCqhsnpGlocy
sxmnXLBnwSkR9iyqr31VoCLVnura0xjwzKYZdaeSaVztqw46V0ihmIWyQOZxMA/DtJB6FBJ8/Ksi
Qn1SLCcFSP6VZwFJYYrxMEArz+JImDAehmkYFwkTpTD2o60U207rmPLvckcLa59tde39TXWsjyMr
rRH6D4bAVUJrx1VyCrBMM6Rytc4YUrtmsPBSFl4aCZexcDntGWCFLM+WfqHc9bfYLDZZYlRGXy18
DZWA5ch7CpBv2mk5p7icfUrG/2baaYKpE50iLJhnwaflMuX1aaeBhQcMTkOK0+Asw47RZHhktJ2v
+frZMu59CS9HZ/GFVOaSIzMYxV+swpvwbkxWmTaZdptIB2yGvUA6zJvNe82kR3a3DLfKYBu/n8er
edhCRQJciOrorE82kl0E55HFZBUh9RyEOaiRwyw5mDABM7LIUmS5MiKTwaeyL2U4js/ki3ii5OFP
/Fc8lvFaLReHMlERIkoEf0JfUboEuUPulxOHHORyq4WkkFxCZAQ+JV8STE5y5zjMVVmHrNhvbbIO
WoetV628zwqAG80m0yYtaHnCISMqzS4NNBT6ApezoaFhg68h0LBhgyHga9jAHMbCoC/AHJK7kP6b
6m+45eckTuKCgBK8JE1H5MTJDXxv5M7vv41LP8J5Iy8IiVY9YF1Mov4M1sOR0RZ+x9fbOZy+cGYW
z3vLFqaPTqVlWTy6hFzhKtFUVI4+EsNbpvZPxZvku+W4tQRqNK0aXF+0uginkjyCU42Q4QRljC1m
c8zeGE6WaE3clLg7kVP6QmL2FL8Otusu6rBuVp8sxJ8dGxYXxCSEeX7aLH0cqOIcs8RZ+PwsQLMc
swZnDc3iqi7OguFZMH8W9M06NgvrZ/lm4QuzrjIXKDL0U/Kr7IJ+epXFqqzKlUGqjGoj8bTOGhou
l17OZv9gOa22Dd3d3ezrW95wOeBzX44TftOwITs77vJUP1o+UVUwxYvT8pMgkF2Cg+DSERcNyM0p
wYFsa0yAfvMDOmIxmGWmHC92TdFhizkJM2Cc3DC4StS9ZNzaEmwJpWJzcU1XuO3BBre7+XBnzwkv
JoTDzwLGcNEztaotr3zldLtdXFGW17Ywe3RJ6qwV0+IqF0ypvH3xCxmVRa7y/vfvuevC/XM7mm0l
+elE6Z5WkXb9n37/B/L2hu+t8vvbvte18eiKTG/L45FxtWTscy6Wm4cSUApaI3rrXatduD5pdRKu
Ia10VFUolfGzRHsiDCZCYlpfCpplN4DBnzacdiGNpLEWMCW5wgoFj6pSUnhHlVXgq3RWVEor0Fjo
u2wo9IGb9rvL2VKlxV3O9tFaa0C0tqTC5+XHsHrCBqmaknAipHmB1Y0czK6K7qqN99oeNwRXHVp7
9dqcnUMte17p9P1QP3hP1srqIg7+V81AW+HycFbW0gofJEHcIx/sLK49/Mstsf3PPpY4e/sKpghQ
zQWRv9DypaB1r6IpY5fEZLk6nBISqxAcRWO0AtL6kFSaS2nccBro06AvDaSCOUwxYZsmNKwFpBW0
fu0l7VUtr9DaZLTTmPRaqqXR0UU7CdB+seFytpt1lGgpp/rdTsN4KxtczJkWSCKWQAlhpbQYSGFM
Tk2p2FaRfoa16zNUA8YkrmTh6lD9XdVp+IX5q6fHZ9XcsWDkXrJoSuVMv5z3FBabfXNyEz3L7m8Z
8UXabvHY5+QK1RztKIC2iktqprROwfXZq7NxIVQAzlOFVFjJ2bjN3F6Ok8mtcjbmOFNIzEQDxitG
bMztc8yy007flTuYi+25MJYLw7mXcrHNYkRqX5VCQClVhDWmNBZKGQthbckKGe3+kZ6fogPWhoJT
6uheyMk1BAy0fWNcXpIL0V6eG60Pkpv95Jb3fwz3bT2ejQGk8j9Huzge+beEkqbyWesqUlNnrwnN
aBLtL7TXgxlicV79CpXbl6mEJ66Z0sLT3EpVij83Drq6jrX5vW1P3d5zdIXbu+qJG/25h7Z3NpqO
vidmbw7uDeLNmr0ajNOV2rCKj+OxO1ZpCPMJlgSckpIUEr3KzoLtBQMFpGBmn3mWhTW92ZIQtlhK
Z9kJEP/M4Zn42EyYKfUJ2tmdC9KthQuUyrhAoxl85gEzNpv1VXGCN1CFopXFOKyB9f/Sy+BmfPcG
32D1lu2jQ8Dd4Ka1x09JZRyiFMZrR57GuoiVsgkrGx0WVrO081BukiaxFTllIWYrPP7E8QV3P73k
vxOKlhTnVJekyl5XFbQdXv/eLzKL9Um6KTNTAxXeWCJLLF+20bV4R03mP824rT630fzcwTV75yVh
rnjm8qJ4fdrMgEFcM8/92qlRb9UCjnQpFPH5C/Jyqosd95Su6M2t48CQXV9R28TGkZfOk246T6rx
beLYMg4qOCjmIFUNlWgpwhmoEOEyXI1xKs7DWElsJIOQBSooV0FABTKVVYUbFDBHAW5FsQIrFTYF
XiiDmTIgqbI8GZbLYmRY87EW3tLCs1oY1MJOLTRpoVoLfi0ksyEIX2jhggRwTAsHtNClhRYtiFrI
0YKgBU4L713SwodaGNLCcS30aaFXC1VaKNOCQwtmhqVau1N7RvuJlue0OdoW7QHtW9ovtLIjE25e
rYV4baV2q/ZZ7cdafp/2Ne1nWsJpzRS+TMutrqaQO7UHxCMUzVsU0RdaRbFEnloi7wb9lLytEnn/
+8QX3kK8mWbYqyWTiK6WyPxCy9FkGFGGhGUKlRw4TsnTP4LUwrD6qhqrWT8V4pLCJ9UwqIbtakBq
YIEvqfVhNSZnx66KGQJxEHyJXCXYTk6Sc+Q84Ui3HHLkZfJqOeHkKuAUPCFRCSKmkPJrHx39BbQb
swHvbigIUHHB5zPGFGYHGhppVLf0mzQdUl4x/ovGbGDvKOA/Bp7qp+GNt8ohTBRRgkvJRBH6j6sd
rRut+gDHjho+gD1wxwejSViN20cewb/BT478C84YWTGSiNDYGHp87HM+Q3bamIpUCBnktC7YxGd7
GUMcn4rh7NjomaQE3pwC7txcxB7a16U0tK/b0B4xWBfTEYPrSAfBdbgD43KB9WlLvSgaq4zHjMRv
HDReNZIB41HjSSMxxveJ8SDGD8dfiL8Uz8WztkjWGcKK+vl8I9/JkwH+KI/P84B4gRd5IudNMqJt
RCY2YwbYTEIr6DJlGdJE0j3BY3kmS0S4qsRMmVOH8ce9Fx5bdq5w1b5FCx9YX/qjhmP/Yi65+91+
suP6wJqHlrs9TYc7Scv1++89v2cG5Y1Z/Ne4XnZakqFdoonsQvJdHPD3o0YOOFGpCXOIti8TFn2X
f0OzhtyAxUkfeOWJkQ+e5L8+S3+Mx+6kPDYxOu88ICo7pm6eije7wH6WTq+xclV4d9LBJFwRXxeP
K7g6Du+Gg8Bq+pJYSiPRLPGYAxy5fZm8KYSMgtFP649TGIdyoTQXvjEVpVQlCkaksfC+KhzhrtKc
GxHLmESWLYkW0pwb6ZoNIDinpKa5aP3IXSVEqiqZXCYPlFDRzEhuEb64xNHurU8E6ARE4AU2F50B
+uOkOeqNGxNSxVppQsJTRn5WtzKuwD+Fo3JVOI27PFqXlG+xWdvrRz8f/Q9pPlr11O29j6+MzkeS
/IH30PnIjkrEdMHit2CLxamxh4aZDVxAfnQJXUW8AtnSTVYqbhgFuZ5JFqWlgffdUcmCCexT/YGb
pQhrjCUiKVkM90VECFoAzuQuqiq0pquN/qSSJflxpGTKrBlFMTHFJYXmkqXFiXLyJM8XrNy7YIQt
HVD1BUEJbUeCOsUcwki6ME7VVcQp0CX+Ko8v8jDED/P4KA9dfB+P9bydx1d5oOGSBB4/JSU8n4ex
SPQwf4G/xFMQoIhYOYANbveG6LBnBeq+zAoTMPSf4d/7Oof1p72UjqBEhxz1iFoiDyHECZyfIwqO
ZZFpiQ1znEI5poRLSriohCHlsBIfVUKXsk+J7UpASrgqRSgZuIFRpAQazus5C1pEKaGji1IyQUd3
N5ue3ctZZ5/qN9E+TihFe8+cOcM7nnvu60tc0bW3JUMykyfIz2j7paGN4oLNAmyOgZUpsJKAI2S3
K0LHaJ7KDCoVm8DkqoqzO7Y7BhwXHZzDESc4FF2KPsUFxSUqHSsERZPkHaYBciovUynZngENKCIs
lNKXcJny2MuGwB2+DbE08HJEGbtJTpb0CY6JkDHySOuDKV7smNu0Q/+yclrbgebtpzuzk6fXtnUX
Lb2vTdS+quvumNsmxuMpDUc2lLSv1czctrxw8cPv377uB3fUBGKyl2wq09WvDrQdiZSV9geO0HZQ
o2ti9TYMWxSwSQ6LlauUu5UHldw+dBhhpu1uRqRXtVN1QEXKVAAqlTpTAUShFHpZZdB5qFrdqz6g
Jux1Rv2h+hP1F2qZXw2RqamD8gC1nIQiLXyVtitn15ZqMXs1ase0nF4bcW7X8oVacdHicJO2T3tM
O6y9oOUvssk04uciUrkYjWTSuVKOQa7iFHoe0XaXRlFpTCHV4Wizu1nPk/iFz90oTWW0ygsCjd2G
QkMgqveiBtogDU6QU5E9YGDzDPGPPrDzzBn4+IPRCvgF/GXd6Hb+vevNWDvqG3lYqrP7mH5B68zJ
9AsFY/dUv1BT/UIDRzVjGqxx9SHXsOuC65KLG3aB3gV9LnCN6xcJsaFhGyCbYPPbLtmu2niFLQ7Z
1BZkrOIFidsFSr9dv4BxVcI1WdGQZgU5BJNC1S0l7bvmJr5k8NeGJD3jDFUxgOzIn5sdU9C6v3rE
h18oby93eatvrxy5i39v9E7njII0uVQm+9hVnMl7qDRLtYp0HXToNuv26ki6Fjq0m7V7tWQfnS8c
VJxey23jjnDPcRz1acKd1u1WbNVorUQIKRUDkUnOQac5Ts73xYJeVqUpVYFKqTdFtIrS96W5poGV
LxC4HJPN2BxyN1CRuWFDwwZJq8g1uHID+QFLwOKKisQ4M6Om4F/v2Jl7+09/GiiNm5qoUGv/hn95
91//evdIzbxShSzSl3fRcfsZV0T1vs1i+UECcc5MZ5GT2HQh0aceUONzahhQH1WPqYk6rQ9CF5Ov
JGOULCT7k68mc4rkoYiGO5R2NQ2PpUFXVCfUMp2QKoBWk0WD9OMKIKNXYm2Xx6ftgDRvw03TdVQf
GjcMGKDou3RArmhk4bgWiHuuP3+rFjjYiv+ZlZO2GPmKllPAS8V8Se87BJCHQgjvRgcRLtLP1uNH
9dCh36zfqye5pJzgh6hW00ZuI/cQohO0hjDHRMIS6qB8VIn1guAWtgqYE8yRV5lQLewUDghvCR8K
io8FmPDz8QJwAigEghmKMTVeinEmVhvjjdKr0rjUuM942Phz48dGxZgR3jJ+aMTHjLDTeMCIm4xQ
Zqw2YocROKPZiN+5NAHAAlgkA5SNO1ikLJ5FwscMFA4zTLCU4YFI+MPfyDXyIRTu1vwufZOe8Wy5
tskEMCjFP8oxEh7JVlwZyViWP5kEWakRviPPm2i6NRJXGcFnBCYlYbke65VAmVqACopMdG6cLDeP
y9M3/Msny9UTovYk0Iif9Vsq1i+PeLqZXM+sgpTDuGkOG6SR2LDB6YKAKYnElJB8UwC3/mr0tuG/
yE1mg0xmMlsUX56jvVW0lpaVUqV5RqkV/+SGfYJLpHOnDk1BvWLxZsdeB+5N2JmAN1l3W/Fm414j
Pqh5SoM5jVmD1cp4JVbz8TzmsBljOZUTB/WgT+7zJ0OypJRT3ftiMthm2RWgMFephKQoG5HmzQb3
hnE5cEKvAAFLY01wUlYp/4bk9/VfNpzeOh3+cOcrGwveSKtcW1beOS/DM7ejpLxrXiZOGv1k9M9l
+z8YwP7Q/l/uv/P4irSMlce33vnkivS0FU9FZTuZi5avGP/kVZQ5dulFhTrsYHLuGHVMKaZSjNYb
+sj3lQ+/7IMMX51vr4/IfPCU72Xfr32f+ri9PtjkgzofyHxWX8hH5D6bJvS2FmRaqzZP+6n2S2Zv
uhaEd4MfBf8UJK8F4VAQ9gWhI7g5iJcGoSII7mBxEH8VhM+C8FEQfh6ENyaAgIJkBAuDOD4IyiD8
7LPgtSDuCO4NHgq+Gnw3yNPouRMQESQsK3wjozuCQHOoDC4Nrgly9iBwLIvPgvhk8FwQ0/jtwZui
1UF4dIyhEcfgYhAompMMzeEg3s6IWRPE84NQHIRkCZTmdgPoMMM1EMQtQagMQilDC/qgPYgjQFuD
+4LPBl8Lcp1S+khWq18LMmKIlAdIOQDFT4tyjSW6wsrxc0YrtAQPsCIyUgktwhcswbPBj4OEJloT
hBwpkT4Iha/RwGtBciwIvSxJpGwkkh3Li8YdZ8AseGuQo4guBAE3BQeDx4LDQY7m7g+CLwhINAVB
MSW3Kl2Imgt9EXuhNF+wKSPSRxtvDMiJITlpQH8zdFLMrdGNN0VP1q0nEkckichgl0IlI20hm3W/
w2BpjQl8i7BBEJ2J8guXTXe9ODF9xRZUNotbBxJI7LSqFnHhbXOST49DfZdRc8WaCdNmBM5dfeei
kXspD3kaIb6V6uIaqo23iKETsbA1Fp6LgfgYd0xxzNYY7oQA8XTGKqZzFrdVD4cIbMaA6kWfFrSS
Nm6uRzJBJsqIXGZqlBNzo8wY5RpUGmygTEMSqCZ4hhlTwcNpcGZzVqPEM5i7BPOtq89eu3/kv+CX
T4Dp7c7hhQd+sXX0v6Co843+efj80Oh/v9TA71hwYvT6mYGf3xW8dip834eMR1D6uVFmN0NzX5IT
OUEqxstSqJCoUml5ZT1Tt0RE5MihBUXjdqpU8UquEYiyERnHzegbLktCkqQJuiW7C9OgnBZn9Hma
y7r+AMm+/gvyML/jyOi0R0ctR1jehVQGepmrRPnoDbGm17vTizst2y0DFrLGCil5kBkPlhzgsQVj
dVJ8Ek6ucLlQmMqlfhMeNB0zDZmIqbBPXaESbUmUWk94fmJjInYkQmJT4XAh7iuEQkk7S8sMlxaC
UAgmD59R5UDJMEiFKJyc7BB0VXyTukuN+9SgVvOUWzNlR7gc/dCiANPYN1DfxMrK+CKBJI6zpQLa
JDqJj4+vrjBjaRoVpXLyWOeUe6lAJWNcPSaJJy8Xdz3Rsfzh7rnGozGDfUXNoTTvwo2h6X1t4gc/
e/GDhO8r/WU13i297rlrp7vrayoLnOCec9sCd6LYMce+ZIGQNt0/tTTTbjJklq+ae+DwnfvMmYUu
/exKT2FaoqC2uXwzaiPy5VQqd71M5S4lqMV5j+HnMd6HQbIQPSD/vhz3ynfKcUi+WL5KTtKZdoS3
yoCTmWXJsuOyM7IPZZ/I5HKZnKihGJYCUYGYlh8G0WAOS7Y6MX1ZS3hYDWfUcEwNB9RAK7BXDU1q
qFaDqIYcdZm6Rb1TzUnAjoqFYYcazJKprzQSc1zNcepkCUwCejFQEpaAjfa08CWqjCG1Q11F24aT
S8FawRyWkSpOD/IqxqoibIpqTY0NbndEgHBPMBK3z82U50BEdG/cMD5+ArngzHVawGmZih8aeZAU
jKzGr+0lqfv2Xv+3fVK9JVLBYBr/MxSPvieqicqkCqhmqjitik2bqxWacJxeAJ1gEyDEIwxGbE/0
JbJ+tz1xIPFoolyfWEqdJxPPJV5MvJIoL26kLhyJI4ni4pZwopjmCTsS/YlNieSkBETERNBTLNhU
paGNVmWT6ZkYVRpgRYyYJ6QCMu5Iwxh7ZOI7LR9QfkfVj/E1vKjBJRECFug488gj1uJVCxzlcYYs
Y3ogUf0BeeV6BXnl7i3FrZVumWwv4a0Z09Ka70ZE4gF/pjzASrVELzosrj0c+2wsftABuxzwYBZs
zNqVhbck9yc/mkx4tUWdoiYybMWpmDxrhKNGWGPcSqVJYkzQ1seIVEyPicmg/M0+xTcFn5wCU/x9
CRkTLM6Y3rg9ARISMuIpr8sY53USEymUON74Shyzf1w2FNLyR8ZalPnpIvyOMnumSSYRyEmNMkDT
BC/k/lx93+vtI29htPFs30znzNaZNXfXekf/cuTA6DmYXt0bdiyYumxH1egR6KnYWpcN9655uNHD
70ir3lFf3F4T1KuK6m/DM7pXjM5wBheP/Gjm8mkJo1zstJYIv+RLJX75qNj2LMBjAP2KRxW4Hx5l
Nq5VGO6WPyjHLXJ4DD2P8Bq0Fe1DpA7BYhUcVkEyM0isVnytwArVoPKYEksGCWYBV2vlmNQzq4PI
ETkzcWOOGbkRZ5RkamNMtH4ke1tBoNDXUBjYYIhqchHzQEOUI002EXALR94/e+4c/sG/jzyN6d/+
kU/4HSMl+CcjR67/Hkk2aGlNit9qTEUyyQbtQJ9JNmiCZxhSme70szOhMpI8yQaNURb3Jq7nt0o2
sR+fJrsQG8Mma5h9xbu1QpjZbkUawIk6qsIxK24xB1kcqLl4DheOcPAZBx9z8C4Hz3KvcfgQB/dy
AFyfxx+O59wc1jOIaxw+x53nLnLkWQ6OcrCV28fhTg6WcjBfWuCxcz4OU9hSju054DjRaovmiHxM
Q5gslrgbJ0klkhXZ7b78G/rcaknm3oxYksfnRx9tbx4ViOnMIoRZEzloIzVxfdwx7iqn4MjELAmE
RCdGqbHoxBExJtJJ8Bz+Kb/jWnzUlqWk/WguxSugz8SXZuvqdat1pJyr4Vo5sl8PRfp6/Wr9Fj13
N4Fcwpa6NxKuF+1EWIlgP8BWABlAHGRCEZ2KAf4EXwG2oBSUi4gMwafoS8qcmLYcp8/UF+kJUejh
j/q/UxUth2rH2CEAFgTwC03CoDAsXBV44RtbOPA5jKniV2XEfmOTcdA4bLxq5Kmyp8dksxKUwHpl
A1X1buzboGWl1R1g78k7N27as8H0NEiCGGtMXn4J5EOA6N4e+fG7sFufpNFpNTpNogF2vsvvuO7I
qnNlpKVkuGr95BLro/uorPBfUh81yyJ99HJ0nSTVIK2T/PeZzPRk003rJNNp2z1B61iJtoiuFq6X
28mRdeQOci8hSwlkkEJSQUi34i4Frla00BHJjARKlTasJIIc5NJkpDGE5QpOj+20MkSNPhyxEeoo
l5MTrpEnWNHIakKajeh0xJYnAtIrau5vaMiA8TmHe+La46T++hXy5+tPkb0D3OIj+65Jetob9LWN
0kkonSFSr0fAdp4HBVrIC4g0oS463qQOVoWGEC9HJ5np6hg/RJmDtExTJXmG+au8wsEP0g9hlu8X
C4Jh6Zvll74v00EJ0pJZKbjp2NjQLVm26GigHTXSS984x/bHSP2zbexzivUgykKvinftdsNqN8xI
WZiC+VhLbE0sWRwDi03AGy3GGiPZounX4DpNhwavJrAaQ0VKXQrOTYSt2n1aLKpApUpeZhedTrTd
PmDHdl+fw9fk6/MRy9KT6BzCDgS0yMjZeJXOFlP4uMYMk9DI+9WiGg+qL6kxlcy4G5IZ27LRwDTq
Bkk4fq+B+uOEyxHbXEOUE0Z/phxjXnLUGk0mq9b5ubRFDMzQlZyK2+pOgez5u94abHG9HFfRvre2
741twRl3/Xj7ov0bFieOLsU1vu2Pv77m9OiXp+rwO0+C6rVW7+Ktc/NyaqY5qw5c2L79g0M1CVn5
CaPHRuP8S2akrnuTNR+rwwbKV/9O67AU/Ye4/mAJlJXAU8WwOw92ToWH0+GEE9TOeKfbedjJ1SWe
SMT7DLBPDgcxSBaGnUXQlAcdFthkgMz6DDanDpnANL1PWa8QBVNYocipR3bBLtqJ3G4SKKu93bTH
9IiJFJsgh3VlHw26LeeenIdzSFEOmHJ4X2NnJtRlQmUmyDIhM5nTNTYpYaESyuiA5k1sQLOtRezd
wIRfabVKWuW5If1GhV9pOp5c1dGp2ZFMZWAyIQPzudGNM5NbgIrB3N/LBz8+OPrV6L+lv6orWvnA
qpr7VhWWdj/eVHzbuqZQ+oLBt7rv/mHf3JjXdbmLty5asWuBq3TtfVXTd2xqm+OGXXUH1wXPvpCS
Xz89OXFa44zyxQWpVq3dXbRgTahlYFlmxsLNVc5AVV6Ca9oCX+mCvGSjnkZWd0s2n52jS7hEbi5K
RUXogCh0FGwuwB2ZmzPx7uSDVCdgYp5JrgpX2OvsuEJeJ8e7yUHCVqUnVgfTIG1a39QEfQgJguCn
3JNTCEPToHQadE0bnIbt02BsGgxPuzQNJ3iqpghWvT5ekVfF/2+sDrpvKBOpaS6q3n5za843jUTp
DYe6el/w8mxZUDLUvkDnIMLZxIWtpV2HGtLfiC1eMXva6vletkhYubI4Fk/ZeuFgTW0LdviLE0fr
eFlauDhTSZIDRXE5FT5L1f3v72g5srZgStOJe9jGlaL1R6PzlXw35VFebolYsdkDGw2wlaoUuAJD
r2anBoc0sCWpPwlXJNUldSSRjfZddjzLvsTeZif3ZcHSrDVZW7PIDgFahF4B1wgADspDtZIxijru
RrARQRmqRi2I5NEpTgdbdFCpW6pboyM6QUnlbl2mrkhHlDr4k+4rWn5tijZXS2RaYGYorNNa45Iy
k4qSiDIJ/pT0FRUP7Sn2XDuR2eFT+5eU6STJ/6fZMm1rGvRad1qxVZ6WJrcSfVxWZlZRFp07s+CP
WX/Pwlkfe+G8F17zwkkvHPbCgBe2eqHTC0u9MN8LeL53wHvSS7yiLSHs8Pq9WO8FpZcX4A/C3wT8
svC28GuBEEGhL9Dfrt+jf0R/Vi/T6EVxzJYa1t/mf9j/L36S5w/5F/tJjD/Nj2V+yPe3+W/z/8D/
iv8d/3/6/5dfkeoHuT/Gj999h0L/p5/c7n/E/7T/rJ/r8EO6v8BPp0sbA4E/+uEjPzzt/6kfH/JD
vx9q/e1+XMFQgsIf68f/6Yef+uEHEV+6P+zf4+cPvROB2yNh5SsYTlD6bX78r/4/+vHP/fCo/xn/
D/1knx/8w9u2hwv9kOkHmqPKD1/54c9Spj/zw1k/7PUf8p9gBAIlrcg/21/vJxl+iPODxg/tI374
3A+/9cN7fhDH3vDDs3444geKd5sf1vhhmR8q/TDND24/JPhB7YfrfvjMD7/xA6Xi9XF4dK8ftvth
nR8a/TDXDz5/qR8n+kHvB5rDFSmH836g+E/64TE/DDDYO/x4qQRd7IcsP8T7QeuHgmt+uOyHj/3w
vh9e88PzfjjsB4p+q4S+0r/UjwslcmwSOV9J5PxWIidC/mMS+XdI5DdI5Af9wBLY/YAb/dv9R/3n
/Bf9Y34ZopXONrJQ/SWL6EmaqLdute6jHc+h1IWtoGOiVcDdYAiUxgQamCmuccJ89k0b263WtEkG
tcZvB/+m9c19A2r5TQa5CXDJPMd06sbITtOGm4TriNQXQ8U+H52ZDeOy3zcdN8QiqrXfQrTTRbyQ
lpomk+tATiSDflRQZEZ9Zt23xrC52xTg//OTL9Q2lUaj1ahj1V9+Mtr8zojBrtaq9YJcp9fL/vbK
32R6vU4u6EGITdR/9Q7Zntruyy8syvevSr2+g4qXO0q3TS3KKZ+ZUDItP4asu/5gTH5xScLMUHn7
5hyync0Tr1GmdydXJOk3eWIykR+kwuQgh0WuirvEloUHqfCPOZEKVRx/hENHqAxTetlHJRMmWU0s
29PntTfffJOsOX/++kPnz0tzUCU5ixujNsM6scjgUFN9hb3MInvZDmrlzCHXHFQ93MeEIzvCChTz
sPZo/Ml4rIo5okVIe0JmJrSffJLtE6RFysu/oY1y+XK2m3rdbHkEciN7esHCdJsJNzysMiWat2RN
rR35q8qcZN6clV1Hzj6Qs7hyhmP6zulvTrhupXXNq0gYu/QiJU2aD7XUYX1YezBW+TA6KGNysZ6G
yPrUoFfb1VihNrGwWBpmKo2F2HHiY3neeEIdexPxDQE3Iz/bPXJZGLksUR/ZHgO5rtwAP8mNG1Um
RvPU2p+MvqQ0J5q3Uueb3NXccaLvz50gP6qvOyR9vU1MVNeLigF0FJ1EFxGHtH0i2+PGFu4vaTmt
ZPfUMcF+QgvHjVVyGJJfkmO9HBRyuXJi23qhtBkqYqXNju6IkjadOg1OSfl20odrHpFR3fvrc/je
kR6qdj+Hq6PydTuVrzdT2TANPS/OZ8vXeKN2lxZvTdmXglenwrbk/cl4dTKsTmAb9esJZCauTsS7
YyAzZnUM5hUWBZYMo/zSqrimOHwy7lwcdsSBPg7ipgiS1VOuDQtChiMD5rvA5UKNdg7pBT3260V9
l75PP6y/oKeDRNVoMY3v8WBvaGA23AATrzdcvrHt4IZ9ARpgshgdExWhcyOCNsekPBJ7+6t3iOU7
Xtu48J51i51HUrsePrfp2dGx5xcvPQno+L+Dd9ZL5rJVe7mvI8LzI9XueWumz5u/p6WQSc5HJQF7
6Plp2UtDmbSelsIFPB930XHoFM0Ii7gLE6oNvXYUzgP2UZkH+aRWYPK/iapZS+ELuHDsmFTH/OgS
cp2OYTs+LxY/SOBBDIcEOIjgXuExAd+LHkN4S2J/4qOJpCMRHkuCJIGO5gdMsNsE3Saq36wy4QeM
QIxMmE6mUQKKVdA/Q5JdOGSH3Xaos0PIDjY7UDFDYTcaJECDzAkyZ6ozzxlyrnJucu52PuV82fm2
81Pnl07NO+yNnayFxj76U/gtJ7BIvPPmJLJ/mF7mtNKokHMxjWIRkWD1w1844ZITfuL8pROfccIx
J9zlfMCJe53Q5IQZzoVOnOMEhxOw0+jEnzi/cGIJ9LjzjBNLkC3OXieWAJOdOU783XCLGU6QAK0M
J7RJoL9mBIAEe5ARAN8OPA4rPkWhKalDrPgHnLjJ2eXEZc5qJ3Y4/U7MOc1OfMl51Ynfcn7oxN8N
l08LHwWDKBBEQSCK6BvxGDkZgionV+Xscw46h52cz0nVUMGJ5bSlkSPJoNdU8fHS1kc6gRkiy9rR
SfUbU+nEjPsPZtpvXx+TvG7JvDu+zN0Q2b0a9MX6DJIhIbLb1D1pKDrZwEtNy2XrDXmlMHnxe1nq
/BW3z5tS5DD5DfP3BAyji4Y/UdntsZjEJCapPvzRisc6izn5PYRs2uHmckeeia+vDyvV06sWJuHV
dMxsH63Fj/PvISuaIWbdo4V7lFBrhloMhljKG3n2EmSCIOuTYZn6r8ws4aBjU4jnEbMrUNX8vYaC
bGl1XlLAo6dKxjeXbM+s39/8wvK9tW537d7lLzTvr8/E5n2jf/5NR8dvPxvdt2/0c+r6zZ9H9kvz
5LhdWodi0EZx0QkMD8bAYeFZAatIHMkkhNdYNCkaQjVjva1PtAH9Z5bXs81sooIoZI12s88839xo
3m7m9ebz5jEzkZtFJZ1izXJTo5LImS3kcmSj3QZplT5b4uMT21qj9maXIbJAHzE153F/Dm5+efPo
inN4wR0/3FYyfPz46C64+8nD5KNlRzeWjXzM7wh2Pta8e9/Ihw9IfOjs6NewA31M59B5ryJu7NIr
dEpU0TlTMjFZKT3yPs2gBouaKs2QhgxqjmmwRjIzpWaENUzC0MifRofVyDfyCbNZMvnCLc2SKZaJ
mRF2jM+GHz+duyQyBX4csUOgsc9xobR/MSDGbUZ7qbYTS5WuDAxYQFjADtyHOYwo9zrDoE1SSxa8
31AgnQOwQADg8OOjHWb+0tcO1i7NdO76Fzp3mejstVacvTQF4lJAmQILnWBxgtwJ1fFgiYelMWCL
gVUGaNdQPi7qzWDO6HNk9GXgpKUnVedU2KECvcquwipbo55zNfKm8d0SDd+28Ck1B5ZxN8040SXQ
SZMS/y+NZ0avP/nC6Ncv1C47DfyJp4E/tezN6dtf33rnj7aXTt/+xtad57YW43eeHP3rcPvElLPq
9dEvn9h+4UDVuE1n8aEPaW1o6HhwS+MhLGr3aGGPEpaYYQkdD2fH/vgiGxL0e0YaFYz7C3bZAB0Z
6K8DalALiHEPNipGJo8KGN/5O67bY/e3DAth/wgbFr/5nA2Lz37LhsXovsi+lWo6LtJo/aegDLRX
LNuYvCsZb3TtcuH6uNVxuN4C9botun4dqddu0fZrSZFstgwrwyrVUMZwBs44mJpqig8loBAyOUyi
adA0ZOIlSS0m1RM2mZyNqgHaJGlHUufEQ3xso5Nj8kGDb+TtBkMhk26pkMtW5TcEIoag8TVQViwZ
LZY8clREFjkpkoSZaZ1LSymZv3B+aRqcxGkl86rmTkvJnLtu5v7jurPqmZtf3NJ9eqs4MvAmp5i1
tmbmtKKyRfnl7VWlhYWh2uLiZaWO/Vs01Q91TS9oO3DtsXelKxVgbITdSkHn9kxySkz+QwwUZczO
wFsy+jMezSC5QrmANwq7hIcEkpcYSsR5iZDIWsdKe31hQkUCLkyABLa1TNqIhiS7Lx2TedqQFmuZ
L0B90mY1EJhLV6HDzAxhCOu01sQEOSBXugtqXWCVu1zMVJCRKWSyKqzwZYcrMiEnE1Iz4atMeDvz
00x8PBMOZsLmTMjLDGWuyiS2TPgiE15mUTszD2TiVZmbMnGhlMQsWeYUmYJe2rCm1Nfp2c44TqV/
2/Op50sPOe6Bgx7Y7IFVHqj2QJ4n5ME2D3zhgU898JYHXvbAIQ/s9kCvBFLoAbMn2YNlHvjZVyzp
yx6GiOuIJlV6bB5MU77qgcWeVZ7dHkJTuFkioEk+8cCvx7F+3wMHJMTdHmhh0JDjKfPgKeOwh770
wE88v/TgMx54ygM7PbCJUdjiwTMYKFg9qR7MeeA/PH/14A898LYHaFkekCBXeTZ58HhpkhkscKxM
4q+ipTotATP6DnpImafag/PG8+34kuGED8cLR3o9O1l0iBaHJDMQqwd/wYrwqQcf8Bz3YFqGDqkA
ZSw2z4NvFPMpigHvlYoITYyGZJoVKTjuecvzoecLD9cnVWulB/zRar0mJTsmVc3WSI20eEi8B65K
lfdzVlU7PQc8ZzxcqYcKHB7BgxVypjulU44xgx1ogSlykCdkEL3ela4xhLNon5K+VgCri+gsiNkA
3A1u9mHG2OXRsye3KuiN3y5kfIv8cYsMcovtYDLAtx2AmbQd72Zwd0EgagDY0C1JS4Go8LLBHflr
YP/Y34Z/rOmDpOlP8nAH//l5hUGhUipVCpPi9IXRfz79ilwnlysUSoUge+vHb8gF6lYo5Hr5uSH8
w/iqVI8vy5O60D4ymysaccbMdKSkpSbbRQv+zxFb3IzEKS7qmxmHL46v7d0vrQ91vIowWyGJDWMs
U7CRXCTXhBUKNfATS9d+NXCNVwD0UAqdsB2OwkmqhFwEhQLEmKQwAI8a5YS/WaiYvDHGR/VykPbE
UIHCAm3EeP0v58ifuD+MfPH4yD/xO45QmmIQIn+mfE2Nfy2GD2DYjeFexWMKvFkBd8kekOFNMpAU
l80I8lW3qXCCCrZwYOKAxMLtsAceAaqF3SN/WE5kChXIOU6pFHjGTIrHT2FlqAvVmFObaQ7qT9Vf
qslbajiofkr9sprsVINMnaoOqVepd6tZ2NsUQqmQTmfF2sdPZ6mVBJSkkFApjDLLPrH34p/Cm6QD
Y4ul42x50qEyq3RsTLLO/lILw1o4zQ7NHdAe15J/BPzOl1r4RAu/lg6ivSydNJNRlrxYu1t7UPuU
9m3tr7WfapUHqQNL2vprrwyHdzJEq7SbtIQiS9XmaTFF9DBzsMCntC9TaEaE8lPpzN4mlik7qUYm
Z/zNfDdJeZKWyGbHVIkKvm2CmggtikPaj7T4W8vyaylX8hZDwKgJabn8VRI90r5Jif684IxwoRam
aEE6wou/YPXEjBHkDDtjN6g9piW94+cMI4fxHFqQkk4xxoaPSefsBG2VtkvLoGVyFcfJgWCFTI+w
ZXzXgAEC0i6ZSaO/MaJyfHM77jeD3N92dG4Cyk27dzc7dre8IcIHogfvCgqko3eRzQk3nZabfEyO
OEd/O/rxT2DH6P3vgA40747eD7vh9dEy7MG60aXw5MgXI79E47oA307HqgEloV+KJSe00G9+1PyM
mRxIhI2JuxLxCwgeQ7AN7Ud4NqpHqxEhxwC64S54gFbJSgARmCCbAkCEs2Nd4hJDRZfQJwwKpFpo
EfAMAbBLCAh0xhdsxnq1GiGD3yAamgyDhmMGmUF0DDqOOYhtknFIwI0+W6Ot0zZg42w2FNt4wzjE
hn9D5OQ1VSl8DYFCJjNJ5zhYXUQUusYo05Y0DSeMaxmSSA9k0maW/aMPjFacww/f/uqdM9Kq71oK
g3/3VN8+Z7QY3l94+9wUXDHyCr8jv/3g8pl3rZ0njHyPfC4uL7WP/D0jvCKyv0Pa5yytnVuj+zu+
phExopZgF9vgQcUzBZq0ci6dG6FpBqjcOxP9Wjz4UAB2lT5Uinf5HvLhXd6HvHjTdNhUAIvi7o57
MI4sS1ubti2N7dorssL9OWDJScnB1hmpMzbNkPQ0HAzH5oXtjliw20EZS5VRJXFTedpPaxYfMwwZ
sKG8LzOE3ILb7yYKd1c5lJbPL8cf2uGf7PCyHT42wM8M8KoB7AY+uWoqclt0kfU0idMWRpWDBonX
jm8fiimUbNHCZWFcTL2hP6Mbqt2UVB+w9co8KrRy7OxS7uRNppGlS9ogHFUuOG4gY8me5VUH8rFt
5ux5KctW5C1oXJDnnFo01eksrS8sW1ORdqb09hfWrzrckesMFAWc+QsbF+ZD2oyemqm22OM6m0m9
eH7B3JLCihUVgYoif06pO6UkK841d0vtdcKRTc935Xmr1pUE68tzff6E1OJCsaY70hb//zzvKZ3Z
5s6wu03gQyrrKuFd5UfKr5TkNSVUKOuUm5V7lVwxO7JlU+IvlXBI+a4S74v4K5QdSu6dj5R/UuKf
K+FlJWTQBB00wSElH68EmRJsygwJxyHlCYpV/ieKGH+shBNKOKiEQgqLs5QAaiU8vEa5VblP+azy
NeVnymtKebWShrqVxYyOa0p8XAnFykoKQpKVsE95mIL9nIbz25WA5ysbldivBL0S2s4rLyrxEHOz
0AEld1UJR5UnlSyc61JCoxJE6cyfXVlKATqVR2nEFaUcKSH/ihL6xAbloPKCknQqoUoJPunM4AUl
nFTCoBI6lduVWFA6lKKySslFjhmeYwiblGwTGVeqBIdEBhU/OB3UYxHLdV3yY/IhdkNLnxxLlgZ9
TEJY7qBKP2I7V0CSUyiPaniPiWYQFyvMHfkku/GmHd03WP8NMS96suOGb1zunDRJRECZqEM5Wq7T
gi/8aDSB28394Vo894cj0f1P9MPpadur0HLR4WDnJJVKxXo8yAHngwGqeAEnI8w4geVYkl/iBGu4
jK/mW3haRiJQH0dkMgX0otjIQbSA2xeI7FaICfga2HlFaVNIINdp4HNT2BasI9A2+hOY+xQsOcRN
+/0zf7gWeyiy14XSoqH6dDr6njh3Iwcb43bF4S1Cv4BbU2BxCmQ465wdTtLhggQX2CywMX5XPJbF
Q1rielEhpmaGRQUMKKja1mdcb+pN3ZmKTanMLoalzXD2pNQwUqTeZ4RlxrXGbUaiMsYZsVHbEyuH
1Aj1hUzDbmgwSvp1gCrbES7mjmPLSOP8KvIN5JbwN1/GIHEomdxpaQs8+P2jffOTyxqL8hpn++Rn
lTN6n1jTcXzDtEBN15Zt6xbH4ovbN754/7ZtexZPW1piT5pWV2yYs7u1KHvF4PJZfb1r21pXdRQe
urHvTYZp+3iwR9Q8kgEP20EjUKFDQ9vixcjS/VUxngakU4mEqdKZDmcKfRkT6IsqyklUy3mRhkhf
GpgkLfXTCK0mOTnesywzGeUg/CEC6dQmh0CBvPu80OuFYi/83AtnvKD2wvlnvZDjBYcXzF7KL+AL
L1zwwlteGGKgO73HvaTJC9VeECU4wQucFx6+ypK/5f3ES44xsANeXOWFMi/4WXSyF1MslxjIh148
6IWdXuhiqcu8LV4SySmSTSSDt7xcE4uu9uII+jaGMYKfr4pgLPMSszeCYaeX4f3Cq2Apv/CSfQyC
pe71cvniok+kwrEUESw8LSQDx695gSXGlYwA2r7XvHA8UoY+L2DRW+Xt8pJSVgkOL06KX4YSxAQs
T5BZpAtBBCOte0siqUymkmQySTBKaiMbE4ZAVAqJSBs3RvXy6KD9xgGOb9MLpYiovCj8RnJkX544
iCX9GtjT4DTk5OXn5UuaHbjYrUypadaYJJAOf0LESA0BA78YY0x0eq1dP3pg9+iATKvXyw1UBMP4
mWtwm9xs1BMiWMwK6PobeS6w2hPwB7LdzWnXRTKsT8/yxeQWFuT72tKuV/M7rvvMpTOKBWHajBIz
+WdpeY2NPGZzpXqUBgmoTrK6iiq1ISwwu6v2fzK8Gm8YXrVHdHr50+iIZHxltsmo+TUwws5Q3mSA
NbD9ozeMsG+ePzJuhT2Pj7CFX4isKaOPEUFe0UEOIjSIcGRTn3TIfhAdo4SzbBEcRmh8LZntzZNW
kT9mKeuoXBZHeVUGCqIF6Jy4c2P6rnS80bnLiSUxeGPCrgS8MXZXLN4S0x+Dt5j6TXiLBrYo+hV4
i7xfjreQfoI34l0Y15S3luOaytZKXD999XScs96Sud6uT7bPt4DFbrGr0HpRX2IvwfYSX8nREjJY
AiWL+sLrL6hAlayfubGwcK5vY5x87kbeduM+pULKwSZOmRqYJsCu7ooIYRMC2I3v+MkJaXk6lx2Z
uCFzYbklidx6G8KtV1NxcYUbX9627aXbCn3zW/OKG0qdhV1Pres50ZnnLG0IBtvneH4XX9JSMWtF
aYK1aFVVTVu+wZVQ1rN4fme5wxHqXrCwqywR9tY/vL6kZN3DdXNvX5Kr5HQlS1YXz757ZWHRyp2z
i1YvCWo4Ve6S2/Gc3NpSl6u0NjezNuz1hmtHngg0VmRlzW7Omb5mbmbm3HVSv2un7WOj7ZOMCtBJ
0b/fAlus/VZcJ++Q43qymmwhpBq3YFztanH1ukj1lJYpvVNIblJ5Et6VDdmMq2aptOGdcZASlxtX
HrcxjrPGgaXTbKbN4Us5moIHUyClqM+XuF5QOVRYOq4TS6cYVWLmRocjJ7bHZjlkwRa9PKcn0jCB
BqlpsiXb7c33XDH17hsiMWU3uTlGdhnFxDF+S8T7jd1ntuKeZzq3nt0SLL/rtU3hO9vmxTyXsHXB
7Nurs6ae7mk60hl8JTm8OjS1ZUEgvXL1jOlt4VR4f/Wp7bOWnwI4/jok/Kgpaeb6KvvSBaF9v9y/
tLFk4w+6KjYtykqcvnrOvHtairJqtrA69dH5OV+SFdg6UNWjejgkgz0yYAZevEmAWCuwE5sHrcTK
ixpLmF+qWqPaqiIqBXTGyOwYuvAljJkw4cdV1MkrqAjUo7BaQS2TQSw7a2GMXEInbYgNsFMWDT53
oGHDeKeNiDKUcdEhriNy9jjJhmdG2vGuN94eHcSC2aIYfZg3mc0y+CuUjv4YSveTl6/PuY/cxicm
p2hGPlfExcfJKQ+YF+0fxehfxYcmxuLGol1FeGNgVwBv9O3yRfYCbEnpT8FLDWsMOC4DLArY5N3t
xbRBKtIgdX2ebSpTPXGaLc2mMjnWT51K+4fd5DMdNZFBE5iCfer1V1TADrTn2Xrj4oR7U2FZ6trU
balElRqXilNdPR650LNDDYvUK9U9amJWA68GNeszkQ2g4zKJjx3MGD87LvUcg6TiN9DP5Q039oJO
jOxJx6GiqhV2Q25kU2jqpJWB8UHO2Uo3v9hz94u9RcofKtyz187ec7i8bXNg1YrA+qXFu+6+7UHN
S+qqrY/XbXpmbWBKuHNezZ0LM2BX8yMd+dPX7K0wFCybkbx757zGXOMRS/7yig13b+nUNfQvzSpu
3TO3ZO3iEoFTFtd2sX5UReveFV236hKr6lI6UnBkKZ0sjl8VjyXVijPCJsNuA2YXCWC1BtQK2Czf
K8ebyG6qKWOQ03oeNB8zY3NGX9J6Vr+2Hr3cFR1s0iriNw/6Noyf25s8uuAba1eca+6uM6vaTt9d
WbnzzOrWUzvnvJI+b0N4Tu/89Iz53RWzuue78Y9/PvrZs7NnPwOW934FMU/NnPnU6J9+9fTFXfkF
uy7+4Pu/u6e4+J7fITz2i9El0TtLYiAgPiyP1QhhebpCHSZcyKLRIGQRLA6LaOHkFkusbdAGpZF7
HvDHNvGXvwpfsF2y4QG2olplw3qbXbJzjNn4Qdsx6SIIbj4Dh2OltpO2c7bzNu6KDYZsw7YLNlJq
m0+hicMGA1IUoXCdNPoCy2PAhpm95ChNRrHZGPOqnr8wfNEGDPOQjfhsLD1LE7SJuQXhLlsfpW7I
xjEi8JgNbKIrNUzppQRQH4tmuV6y8XYbWPVClTJ+/EaVy5HLXTa42dkPqS3cGya0mRsSkBTFrluR
DowJTORGbjfbvGWIbsXNjR7rMwT2nrHkNYSnTE/QJmtTpyaoItex1JesnpfF8fswZ3bP9HJPouid
kle5eWgKykNPirfV+Fp9uN692o0LHRUOnKcMUYUHbNL9nZyMt/Kb+N08p7SGRJd6e8zFGBxT0Jc1
S09ARRwFYgE+XwCowFEwWDBUwFVdLIDhAphfAH0FxwqwvsBXgC8UXGUuUFjjDYI6vUomJGbfuO6N
3Y1360WRky7Ji3ZTnl2RZ5S4fmB8txizgjB5zqUjcEtfhU9+MfxP785sWxC0SRdAPo956WoiHFe6
sA2EjMo1M8pXBBPjg80htvvYDFb6FwcJsb6QzxbMSyOGwd3X8uCVuAJbQmKgcmoMLtl0rCkj0PG9
9WuPtQdSmk/ctH/LhN56FamoFJemcYelqwORg7rYxW4ViKiFRGdYzZQPZrXGGWoApqfQQOlep1wa
8ah0BDgVwLQMsSsiDgjkEhU4keAXRKFLGBYuCDJBtIBoGbZcsFyycBHBWmcIq+TLolsK5AqQAhPd
YQMoiVqkDjVSRPZrsufGbjAfu5wgIJlLGqWtjm43sGUcJ0xIw6lpXmCGE65oRMcOJz/zO/w1oTzm
OW5oqj9jqet6Lb/jWnjq1MyVWeTIuFzbPHqHtPbO5IvvixvWmreZ95tJeyy0J0LrFFgig5qpUJ8G
9Vmw1rTNtN9E4kz1JhyXXp+OLZ4aD47Lq8/DijywZNdkY3k2aLVWPp4tz0fEiqUn48/FY0c86OPt
8TjeUdRXhDMbHVxOI281qRoNVCwtDTA+1yAt00dW6amgJ10aOG5ji6irkWs0pVN6k+8K9OLvXL2f
e/8Huy25ZYsCCWJM6tKpi/uaQ/GPXzla9Z1r+Q/9rx9uNIr1hfEKfrdSG2jsX6Lo+TGQR/7xyj5C
BKxUx/2zJI/LkRotFoPygxyH1AeRwq9oUvQpBhXHFBcUVxUKFiDSoEHpeqSrbMGJHPSxsysC22YG
ysMcUiCfJDiw7QuXpYPsbOqc6ndGj5axR5b+k+uvDF9/iYry7Mdss9L9vtL9bxC9/23tuG3WwjPb
rC2Gk99km8Woic5jxmgf+JHYcSINnlDDQ5onNfgZAxwyQK9rpwv3OnY6DjjIFnu//VE72RLfH/9o
PNkaty/ucBypT1+dHjEfCrFU2MxTQApa7zOVmvB80zkTZhsD/CaRbQyQm4rUnSqVb71DWkIfFuMd
qeFEU05PY2xnLI6N5TN7psh1PTE3JIfxEzoRcTPgu8z0RarEBKS9JDfOTI+/x2fFG2en8yffS/vN
CdI4a+cbmxbctSpsfS7htqUVm2v82FLRsDa/6fDa4tLbT3Z+9te3UirWhKa3h9Ncofby7LZFufi3
r4/+4fXlzlDX/PilNeX9F/b7Zgfiy7edXLNuaOuM0eMn5/WvKvbWbKmctaUuMCW0JmL/cIx9Ttql
/+tim6jfi+EuHraqoEXWK8MyxlJmUMaAdA4d7tMN6i7pruo4nY4Y6kWmSzZpSKcGNDF9lH8TRww4
YsSYqpimGE7XqGcH50UOaD8zodJsZqViY2mDb8PEsVcmdiJ3ZADxdPqJilHMlsZmJHj32Y9dr6m8
4ryM9ttH/wveL1jzqPn0sH16w7SEgX5SfuS6q/X7PSW0DAslu/8O5EEviQlslWy3ltTgViwpIrjG
3erGNZmtmTj17Ni/inW0QFTIlKnAlnwoGe9N/igZkzInyFwOGuNSM/HSAj7LUQsepAqity/ZnrLe
6XComULIxKC4nvR0IbnXKRN6etR3q3GbGqyTBMqohgiRs/SB6MmXiJronupv3NC4oVu64MotCZPw
DUFy4opWucvktDhJRCfhEsU7X9va+WT3TN3L6vTy1jBV6zyZVGrKmjM9N2bIR5wjd8f7B1d2PL1J
hJ+vHtoeylm6JWRJryx2uau3zJ++br5HSEgx4y8PjU5PyRU3fj/S/i3oDDeLe5zyhVmi9wM17FLD
bzjo50AJz8jYlimMP8JQgPdQB+KeV/5aDvnye+RYzr8olVfSIditlIGIDa8hqkM4pcOg+OiDo7Xw
9IPwNG4arYbnHoDnRqsfYPl2SLZItuetTMzaou5X4w4qmK+XyVCVBgSNQ4M16kbGKxxIRE2oD/EI
aXq1Mj523Jy7gW3mjfAemqlMhS2unDicT7PmNKtP9816vWrPmdYRNXmCu/L90V+N/vPoj8+cgHIo
BO+DUdsfgje5bjyH8kW3mIQR+qEe7FAK85lpli19n4SLwIOY6g4Du5BR2n471W/IdVq4bnhzL7su
b+zfRzu4/tHPKA6XGCPgJtyHj+ELmGeOq9IW3h+CZOKQujuhae3cidGOu+6i+beiR7kw9wySodWv
Ip72TYPRGibp9IXYay9is/u/igHq3g0gQ7Q1qnAXFc4dNHGpbLtsQEZkfhrSx+4UQ+x0cCd3lDvP
8Ryy0RYJvN/wfgPEMiOxtEIp2a5MuWBRgqWV/P76U6QeF78P33909IHR+6N1chNNMJkmmUSTjN02
GKWJByRzyKpkXTJORhhNMt7PdXF9HOFK0XY0QCtFj0txJz6Kz2MeM5oYRYEAoykQOcko0ZSfq4Rc
RlM9pen3jx6idb/+0dHl77PTkvg0HpH6yRqxci7fwK/jSSW3lFvDkRmKhQo8U75IjpFMQ7SNHOgp
wyEOqv1irPOpSlVYpVSc057XYkHr0GJt5LZEOo/xOoJK32uI3Kb6XuC9ht++1xB4b0O3MSZ69Tqk
xPAxcjWk5afk8yQAPwVF1uinr9528Mhtb4z+MQvUmj1cfdvOiq/DgMdQ+O8Vu9dARqRPITB8dvCT
t7Mb9dP+huyR/8vn/JYT/zT+/80wjUW+W3Yasf/oB0cDaTp5yeg8NPPG/1kD6OZfPf4clfGLEcf1
oMX0WUKfPeNuGu7FhXSGfQc9zh58D8qi4TvxM2gPvIP66bOXTsBL6NNPw++jsHb63UX9coZDRuFo
uqcpnqdpWCF9plKYROaXwmgeFBfDyfxK+t3H/R5NpzBvyApRGw1rYPnJ70VKGvYaha1kj4TzHdRO
3UuZAEC/2xk+CnOW0UvTNtMwDU1bzb8zNkJxsvxjWDqpbL9H97Hy0K+X+o/Qp43SOp7+Neqvo087
fXz0mUefKnhn7BcsLcPB8CsQWKP11kRxO+h3IfccaqHfDvogrmfs36m/lT0kETlpXXvQ7+A5nID7
iZkM8wn0L4//Pv+2bJnsGdlf5NPkpxVzFC8rY5X7Vf+mnq25rl2mfVQX1n2iX67/Qlgu3C58bog1
DBl+ZkwytptUpg/NXdY/xlhj1sa8Fxsb+1zsz+Ja4p6J18XPi98U/3lCXsLZpH+zJzuGnPOc7VO8
Ux51PeL6XXJ7yk9Tjak5aVzaC+nm9F9mhDMVmc+5c9z/5knw5Hn+mDU764/eZO8q7x7vU96rvgLf
YLTH1KOFdNDNptIWFfiRj/oReZwfpmGspyXA4hv9qulGH6OMg/oibkwlxM6om6A4tCnq5ijMYNTN
07F4NOqWUffzUbccbUGvRd0KZIbcqFuJdFARdaspDUtu/K9jXuiJurWoE74XdetQCRU3CQJOSX3D
eF7UDSiJ6KJujHTEE3UTlEOKom6OwqyPunmUQPZH3TLqfjrqlqMvyFtRtwKlc2ejbiVK4C5F3WpU
wF2PujVoGZ8TdWvR7/jBqFuHtsnWz+zs2tzd0dbe60hfmeHI9vvzHQtbWxzh5l6Po2L9Sq9j+tq1
Dgmgx9Hd2tPavam1xeuYUzGjfOH06or58xwdPY5mR293c0vruubuNY7OVTenn9OxorW7ubejc71j
UfP6noWtbRvXNndP71nZur6ltduR5bgF4Bbv4tbuHuae6vXne3MmIm8B/R+IoJS3dfT0tnbTwI71
jhrvIq+jqrm3dX2vo3l9i6P6RsL5q1Z1rGyVAle2dvc2U+DO3nZK5+qN3R09LR0rWW493hvkz+zs
7uqMUtTbuqnVMbe5t7e1p3N9e29vV5HPd9ttt3mbo8ArKax3Zec633fF9W7uam1p7eloW08L7m3v
Xbd2DiVofQ8lfKOUI6VmcpWFOtfThlkbgfE4elpbHQx9D8W/qrWFktbV3bm6dWWvt7O7zXdbx5oO
XwRfx/o23wQahiWaz/9Zasr/O1EX2oy6qXzUhtpRLxV/0tFKlEG/2chP//KpayGdpVvoN4yaKYSH
uirQegrlpa7paC39c0zC0CP5Wum3lX43SWkZ5ByaagYqp9imo2rqno/m0dAOCb6ZPr0UupnCtqJ1
9NuN1tCwTrTqO/OfQ9OvkPJhMR0Ufj2NXUR96ylelq4NbaT0MXzTachKGrJeyqObwmVJVH0Xhu+O
XSzF9NwIn0opYjXmRTnfmvK7sf6f1USkztskLL0S7ghkh4S7hkIskqCqpJSsFnql3NZLUNXfkuN8
muMqmp7V2QTkSgl3L/VHMHdSd3u0PlfTuu6WKGiR0o2XrYfm/M3aZ32vm/a+zlvqiFG3ScpzrhTe
K/UlFtcu+bpQEZ1pfOg26c9LYW7GvDKK1yu51lHI/7vpeunI6JLqsVVq5TYKG2lxr4RzHe1Zc6I1
tF7q76yGNk4qY6Ru/lEvC0nfyIhZexMe1rLsy9KOU98TpX+VlE+k1rrou5PWe6tU214ptE0qYwdt
ww7qmkwfa7G2aNit1IzTcnN5/t/Mm0Tl1zR0EH3L75RS/BGwlR+79D4KnHgfDI/AyRFAI6Cafw0c
1+BvVen2v4bS7f8VyrRfDbntjVe2X8H6K/OvNF4ZuHLyCq/+wydJ9t//R8iu/w8Q/yNktf/7pZD9
/KWLl65cIuKlQF7oUijW/tvgxZrfBUnNRSA1vyFjdv2v7L/C0kv8WWx86PxP4I3hafYfV6XaX/9R
un3sVag623W27yyRDiOeNWaH7K+UvjL/lc5Xtr9y9JWTr8i7Th87PXSa6E/D4Esw9BLoXwKF/sXS
F6+8SPqGBofw0NDw0IUh4jtZehIfe37oeTz8/IXnse+50ufw0Wdh+JkLz+D5JwZOYN+JzhPnToyd
4I4cTrZXHYbOg3DuIBwMJdofOhBj335g4MDYAeK/X7wf990PXQN9A3hwAIYHLgzg+fsb93fuJ7tD
Y/aju2Dn3VPtvT2l9h5ags710+zrQ7n2OIitsQVia+QBUiOjZW6icY30WRaaal9aH7bX068p21jD
0zrhsklNJwE9KSX4yoKxBVhckFsQEhekpIfOi9VVUBFy2MMU5yz6nAzBxdCVEO4LgTXbUmMAfY2Q
ra+h2mQNILDb9aX6Rv12PafX+/Tz9Z36Af1F/ZheXkrDrugJFRX7rFRFPguDp6oXud2VZ+VjCyuH
5FVLh2DPUMoi9hYX1A/J9gyhmvqltacA7qvbde+9aEZi5VD2otqhpsS6yqEW6hCZo486hMRTVjSj
rrend6O03QIiDtTrdvf0MBe70wdFtmKA5AJ3D42mYD29PdTTuxH1uHt6oaeHDuReGt4Dy6m7p4cF
9wBNQZ8edwQ9xUARL6cI6Ks3grqnh8L30PQ9sctpv/6/ANepd8IKZW5kc3RyZWFtCmVuZG9iagoK
MTU3NiAwIG9iagoyMDg1NQplbmRvYmoKCjE1NzcgMCBvYmoKPDwvVHlwZS9Gb250RGVzY3JpcHRv
ci9Gb250TmFtZS9EQUFBQUErTGliZXJhdGlvblNhbnMKL0ZsYWdzIDQKL0ZvbnRCQm94Wy0yMDMg
LTMwMyAxMDUwIDkxMV0vSXRhbGljQW5nbGUgMAovQXNjZW50IDkwNQovRGVzY2VudCAtMjExCi9D
YXBIZWlnaHQgOTEwCi9TdGVtViA4MAovRm9udEZpbGUyIDE1NzUgMCBSCj4+CmVuZG9iagoKMTU3
OCAwIG9iago8PC9MZW5ndGggNTk1L0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nF3Uza6b
MBAF4D1PwfJ2cQX22CGRokj5Q8qiP2puH4CAkyI1gAhZ5O3LmeO2UheJDmCPP8bI2f50OHXtlH0b
+/ocpvTads0YHv1zrEN6Cbe2S4xNm7ae4pX+1/dqSLJ57vn1mML91F379TrJvs/PHtP4St+2TX8J
n5Ls69iEse1u6duP/Xm+Pj+H4Ve4h25K82SzSZtwnet8roYv1T1kOuv91MyP2+n1Pk/5N+DjNYTU
6rUhpe6b8BiqOoxVdwvJOs836bosN0nomv+eLReccrnWP6txHmrmoXnuzGbOVvOiQBZmj+yYj8ie
2SEvmHVuodkvkZeabY680nzYI2853iLvNBea97yvYw7MK+QjsyCXzLs5m5wZaxn6C3gM/QXMhv5i
gRz9JXL0H5DpdzqXfqdz6S+wrqG/2CJHv9ak38Fv6Hdqo/+oNel3eC9Dv8dcS3+J+5Z+j/qWfod3
t/Q77IWl38Nj6S/QW0u/1zH0i9aB3+ZG66+Y8b6Wfoe+2ejXmvRbHUN/oQb6rY6Pfqwr9FvUF/ot
eivRr5l+izpCv+j96IdZYv/RE6G/wFrC/nv0Vtj/UsfQf9A69Hvsl9Dv0XOhX/BNCv2ideh3eEdH
v2C/XPTj23b0CwyOfsG6LvpR39Ev6JujX9ATR/8C/Xfx+4HB0e+0Pv1e7+90X6zWp3+v9aMffXPx
+9Ga9Ave18f+w+/pF/TTW+41bJ5+j1756Md++eiHx9M/fxY4KOKJgCMDZ9qfoyitn+M4H0N68On5
g5On7cLfs3HoB8zS32+vLjWTCmVuZHN0cmVhbQplbmRvYmoKCjE1NzkgMCBvYmoKPDwvVHlwZS9G
b250L1N1YnR5cGUvVHJ1ZVR5cGUvQmFzZUZvbnQvREFBQUFBK0xpYmVyYXRpb25TYW5zCi9GaXJz
dENoYXIgMAovTGFzdENoYXIgODYKL1dpZHRoc1swIDY2NiA1NTYgNTU2IDU1NiA1NTYgNTU2IDY2
NiAyNzcgNzIyIDU1NiAzMzMgMjIyIDIyMiA1MDAgNTAwCjU1NiAyNzcgNTU2IDUwMCA1NTYgODMz
IDcyMiA2NjYgNTAwIDUwMCAyNzcgNjY2IDY2NiA1NTYgNTU2IDY2Ngo1NTYgNjEwIDI3NyA3Nzcg
NjY2IDU1NiA5NDMgMjc3IDMzMyAzMzMgNzIyIDcyMiAzMzMgNTAwIDMzMyA2NjYKMjc3IDI3NyA3
NzcgMzMzIDU1NiA3MjIgNjEwIDUwMCA3MjIgNTU2IDYxMCA3MjIgNjEwIDU1NiA1NTYgODMzCjI3
NyAyNzcgNTU2IDU1NiA1NTYgNTU2IDU1NiAyMjIgNTU2IDcyMiA3NzcgMTAwMCA2NjYgNTU2IDUw
MCA1NTYKNTgzIDU1NiA1NTYgNTU2IDU4MyA1ODMgMzg5IF0KL0ZvbnREZXNjcmlwdG9yIDE1Nzcg
MCBSCi9Ub1VuaWNvZGUgMTU3OCAwIFIKPj4KZW5kb2JqCgoxNTgwIDAgb2JqCjw8L0xlbmd0aCAx
NTgxIDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGUvTGVuZ3RoMSAxMTY0MD4+CnN0cmVhbQp4nOV6C3Rb
1ZXoOfejr62PJVm2ZfteWbYlW7ZkS7FjJ/5cfyR/E8sfBdn5WI4lf3BsOZYcmhBIoNBQk5SQhvAJ
D0JfP0AJkQMthqaQlk6hM6SBMvR1tZSkBQqdEkJnoO3rI/Lb50oOSYbpvPXmrTVr1rvWuXefffbZ
Z5/9O/tKjs3Nh1Ea2otoJIxOj8wONvnXI4ReQQhnjO6I8YvTb7cBfB4hauvY7Pi0zf2rDxFiWhGS
suPbdo793Q1f7UVI2Y2QXj8RHgn9hLrRhRAXBB7VE4A4eulxCfTvg37hxHTsC19lvyaF/ikyvi0y
OqIQvlYP/Y9Jf3rkC7N/pL9HI8SXQZ+fGZkOz21/6G0Agb8sNBuJxs4gawIhW5yMz86FZ+8ffzIf
+q+CTI8CDsMfudIAlJA+RTOsRCqTK5Rp6Rpthk5vyDRmZaP/by72FZSPbmI9yIDWi/erLmYN0qMb
EFr+gPQ+uyeuW/7z/0spZOIdZ+Ni9Ef0FqbQm5hBT6CP0F/QD9Bx9Do6eyU1tmIOY/QR0LyDPkZv
opc/nyvwS8NFIvgaegu9h36MDn4uXQJ9Hdb8DboD34a+iw/jIRTFevQroH4cm9BD4P3zzDHmRWzG
7+AP0XG8FldSR6gZrEK/pW65mlfin5c/BE4f4oNw/z2+j0jO3kJnozjVgPZR36D86B9A5iB1gJpG
y+gVvAbW34j2oHtTDKLopmvEK0a3o6MofDUebJVGrVv+F/RddBr9PfoluhXtR0fQMeqLlHv5Y1j/
IkiAqGxswIrUlOdX5koeo2epU5Ts0v3obvhbB38hFMKvov+B+cQ0SHca3YNeRTtxOh5lFrENW9A5
0M80+hrahg6h76DH0S/wAMTPA+gwbsWe5TJ0NBlWmEa/Yx9hb4X4OoYmUSurxY8hJHgGA/6B/r5e
X8/6dd1dnR3tbV5Pa0tzk9DYUF+3dk1tzerqqsoKp6O8zGYtLiq0FJi5LL1Wo1alKxVymVTCMjSF
UZnH4g3y8eJgnCm2tLeXk75lBBAjVyCCcR5Q3qtp4nxQJOOvphSAcuwaSiFJKVymxBq+DtWVl/Ee
Cx8/02rhl/BQbwDgA62WQT5+QYTXiTBTLHbSoWM2wwzekzXRysdxkPfEvTsmFjzBVuC3qFS0WFrC
ivIytKhQAqgEKG6zzIKmG7AIUDbPmkUKydLJsnG6yDMSivt6A55Wk9k8WF7WEVdZWsUh1CKyjEta
4lKRJT9JREd38otlpxf2L2nQ1qA9LWQJjWwKxOkRmLtAexYW9sW19niJpTVesuudLNh5OF5mafXE
7YRrV9/ldbo+WxLH2SKNhV/4BMF2LBc+uBozksJIijSfIAJ6Qb0LC14L710ILowsLe/dauE1loXF
tLSFWQ9oGPkCMGtp+bk7TXHv/sG4JjiB16Q26+3riut6NwbiVJGXnxgBDHwaLeYak1k7uELj+7eG
ESgC1AE6NZvJxu9cEtBW6MT39gaSfR5tNZ1EgtM+GKeCZOT0yojBT0b2roxcnh60gDW7+gMLcaao
I2TxgI7vHInv3Qr+dD0xhUUTV/3JZLYsZGj5WuegSMuDVB2hST7OFoNaYNaVE8BTyJQFjdhR/Sn5
uGCCBYq1GXytBdgQPh6LJ5j67JjIAgZ8eVm83Z40/UAgLrQCIIykbORZrHDCjJEgmGiyVTRf3GmZ
jestzZftScTyTPYHxCmpaXF9SxwFR1Oz4k5PK1mZ9ywEW5MiEF6W3sCzyL18fnEVb3rKjVahwVZC
nNkCflXsWQiExuJc0BSCSBvjAyZzXBgEAw9aAuFB4migoZLzsJxZXDFOtQwEuvotXb1DgZqUIMkB
wo4p8lzDxhIwJdmAy8VlRTI+QJnoQSDUAIL3AmBproN7XFokg6YBhYtY4qrNdXwAcvgKNYgRL+E9
4dYUHelfxZQl7tTSvsJNQrrAp6XdZB40J6/yMgqG+dTCMENGlNq+MkQXQSYAHAVsRBTRZRbxeT5g
CVsGLRN8XPAFyN6IekQtp5Qh6jxlq4GrelcoC9SEzDC80iHKjHvtpiuVG28T+5e77dcMd6wM8wsy
S1f/AmFuSTGEk6qoI46ICws1WpMY/SSeLd4RCGKIaDGeFxYFgcTyBAnbBUtHaMHSH6gTqSGD3GTa
RdbKQF24a6C5vAySWfOiBd/RuyjgO/qHAs9qoDy7YyBwksJUS7B5cLEQxgLP8nBWiFiKYAmSdHjS
IZz6oCMT6U3PCgjtFUcZESH2R5cwEnGyFRxGo0tUEqdZwVGAY5I4QcSRC6yUNQE6hvzt4UPEPrsH
JxaCg8THUSZoBD44ji0NoB1LwyKmJGlxhSXcHFdamgm+keAbk3gJwUvBM3AmLi/btaDxWD7JKifH
I4WgLqZCrB+qaSlyLGLkrDspZRwXXIsS9s26kzQFIFqkCZol6JNSifPTupOY4N1as7bIrDW3Unyi
EN+XmGD9f/12K3OGHLuEOXMGagIlel7IamcCDHU9wtfLcFiCwxTGQWUa6pYuLR8TmgikoORSWiZR
SjAlZ2gZi1gFQJQUYBpp+mBgI4v0LItkmj7Ab6RlepqWyeVKlqWVS8unhSyFql2pTKcxo5AyEpmc
ZmmKVtIoAzUaXY1Gt7EWO7Vupx0afPZp7PvsP9qXJT40+06fTjbZaVnqjuz2ygq8mVxmOb78oe9N
vJZ4NfEy/ofEFlyDpViCqxMD+A18IaGjXqJuTvThJy/deqkW9Bpa/oCZZNZDLVuCAkLGmBWPFeCx
NJydk6PxHivCRURqTqlqLyqyF2rbeLaCpdjs7Extb2FhJter0GT6UCZqvNB4IaPWeUFb68TOzRdc
ANvh4XRXViCQjS1wUFWrGii3K5/Kw2Z9PkDVq1kVTReoKKnZQVNz8z9cWDd4+IWxvd/bXXepgKvf
VN85p8fyjLbYt6L23kYbhV9hrPW9es+BN+46+o9fXDN47Nw+Weu8v7Kx2eiYuK6WXsxrDHmbtjab
xXIK7QO7+pgeZEZrBS2Tp8+jzF6ELEaF96wSi6ZQydPalUamz6TRqpVQvze6GhsvuLAoONkLEd2s
XVVsARkN4EKrHJTVnE8b3A2Y7MOgpWvWbXEP1BUk3sM0ReE0imFopvaJGwJ7+qzU1IHbyvujnkun
6HZLZ0tVelqVIGTunq2demDrpd6U370CfufABqF3ksZjajxmwBt4zJdXlFNOfaOeKuFwiaPWQRXr
qnVUq2XAQrUWDBRQLeZ+M2UwQw2vh6gYgAqUVWaYMroyNmZMZbDgheXW8iUsF9IcSO9wILOmD8vx
ElYIaVaz3mo185o+rVwrInJ4fU4On6Hp0+foyRSbLkOv02V4rRusVEfOYA5llefQOo1DcPgctMNR
obPmyAU1Wicn+jOkq1C33JzD8/pyB62jsTbjMz+udda63XZwY+wEXW7OMNZu3nyVN1/9WHHizy6z
hXbQ1mKrRKqipbSFNuvysTHTWL26Aa/WuZMdVwO1Wmem//oXgVWp1VKtNkMrVatVbPNfXsf0ICsB
o0jkMn3a6Iuj6TqZHIKWkbADmMZK3aq1jfne7nXe/Ma1bl3iE9ZzqcO6qaTCUWizXOfA/Ynj5ddZ
bEXVvs1W6hkx/2yCONkOcaJDFtQlGCfNeCwHjxnxuBajtvP6j/TUMT3WE71kQrDo9UW5bbyiQkEp
sn1qDedjkzFCgkOMje0XkpGxGetVlOWK8GDBzSwFEoM+0w3bY7Y33PLSl2/74S2NdXteWjj00o1V
iV9mV/nrdk9HphuGak1U/u6f3tPfd/jsTTvPHBkYuOfMruCdQ6XxF36waB/aL8bBj8HPakFuDlU9
xeTg7KXl80ImeL5Go+A4c6bcq8ik+3I1GrUcIgACAAwmuj9kH5DQvhk78DUBYHQ30G5XppGpdY43
DOzx2xPvp2kyJFgtUaUpacZQOxBbX7OtlPbJlK3zDwYSndT3a2LjfVmZq+sbcms3CRaFFOQ6DnJ9
Fd6hleh2wTmmgLdS3Ie6lQq9UqmYkmEF7pNJZV1yhV6uOKfACqVMJsdy+QnlC0pKjF6FXE0SqZT2
MqSrgy1BSgULy2VqFikZWoxoF8S0mFPtF1wkrW52Oze7tbWb9xGnw5uJ99k1PxIjHUthi2atmD6D
iXM7Iaqz8a3YkbgN34//aSBxPfvKp0/i04mhS9uIXneCP5D8shbtEkoka7DcnG2mqo1e4wYjLaFw
aXtJiQt1Ggz1rk5O2a5QHOPiHMURUQtAVM5lYJ2+wtlSzJU6S6nS0kKNxsdqlDzsTkl8BeTUXCB3
eGTUkg2A6Bc0K0kWvOdK57GuzqeJu1QREznoFV8ySh206EsA5tOMr2n309Hxb940YPhzWmlDf1VF
f31BpT/a1HrbhFAX+3YkcN8XejX/U1pY5S0NhUq6xuu77t7uwXXrd1/nzPfM9BaV1+QrlKbKotJK
zqhWl7ZH/Ot2+svNbTPrc6zufKW7rqgsz6DW2Lt2gH7uIvkX7MuJ+ZfVsxS4msKck+k9a8TGlfxr
zEJGcDxNH62B+HA1uhuvzr94JdeK3rfiiGYt7FuKvw65dust6/jE+6ri1ip3f52Z2IyCN1566+5Z
Yfa+wKU+ahpOhSIxE3+ffSUxxTfVOtJJPO9PXMdsA/sVoxrUIWROZOEpHQ6pcCgda9oqK9dYJWZv
AREzA8QsKFabTTllPkMmu8qnTIoKn0bRry4LSwwing+0KPOKbSz6TH2muINiKxaPPmIo8VjBD1IS
ODNMbX1DjuGDI5UVW+8Jlw/1tmRB5ZF4D9woz9rQWejoqsqbDFq6Gm3MGmnpqmpDdZfTMHDkzM4b
f3pkQF/eUaWwuqqzE1uYP/fcWHDprakD/QWlQwdGl5YKN9wt5oCx5Q/o/yV+R1UnKNVqVjuUlmaE
SoVsTw8FCasaVtPyYcRLKiSURAI5rtEl+t8FskGyPWiVFeQUtxblg0Wqq7TmKtEK1KrRh2KdamzR
DzV2bEi8TvXXzz+mr9n238L2HbEdM9S33vt059TxnU0ILS+jfRAzdexDGcVIhZBWgtQgnVFIp3Ch
tpjCrK24QFeE7VVViDRyhhN6Zg1k3CbIqpyJo7LTvXvS7kqj0tKKkPesBVtWHMmSLenLy9Qb0oCp
eJC7RUciZ88FsQjB2hXVE/eBgICqo8qiwmLu1eJj+tKmcnt3DQ9BT9M48TFmaZqteXRiy94eC7Pm
0iONk12lxI2o1k+XCrpWDvQbZ4XIfQHqhFijYjPzIv0E+5BYo/KChr4dIentPgYzLD7AoHtAMFJf
gEDEX3CV22CGRv3k7Uvz55kXfwUX+OXuxHpmiOlCVagNbRCMU248lY1DHK6u9BarVLkdEpYV2nLJ
to2w7dwy9VpfsVkDRmvx6YwGeS/KJQeNm+wc7qKHup1uLXTdF1LVGL5sR9BGHV5dfVkPK3nDDcBq
swobiIl1K5GXPJKoidGjEU86zlZvaSrtAoVhqvKJqeGvbHFUjh4JOYZ8LVkszbAUVlBM657vhqz1
nRZHd3XetmFLV70tsd6+bkowN3XntUR6c+tz8vMmt6w/8OOdN/39wfUGR4fbsNptkxbeOvDpb3ac
iK2h3xrb5zOXXPelLU8+lt+zj/iED3xigh2CanWzUFMtw4Z8bLMVKfQbDQa0kS+qKDpWREOpmjck
KHxw+lpHbDY+a1g9a8CNBmwwqBl+mNWJKkrlVufKoSzaRSt6OmipSAWp04FBJxnJHGRkUydhUg2r
qyACmIni8E2H+g+8ttDS/OWf3R26N9KdlfjQtEVoDzXkfunLxS1VVmXiQ4p/lm+vt00+h/H938L4
xFD16MJAdWhkbfTbs+f+WWvMlv/hl2Kc7oW9PQA5sxEdEaqr6rHC0GkYMtCsyWDymPwmhm0wNFAK
OoemWKfBSdV2ZsHhIsviuCwZXYw6kRZrwTGe0uW1k6egUarbtdqmYu9Z6znrRSttXTkordpajmYL
+suR1aBKlSZQv4vnZLI+IQ9jrdPp1MCZk8ptm3HqvCm2VpHjphpOGwZchhGTmUTMcEn/gdTHwKHD
MA+UDR8aa7ulIauuoT67t9PVFehymV21LvPmwPi2xHt1Nz5389yj01Vm11pAdwY6XbimY1egUm/8
qdKgVXhqHa1razuGmx2eNa6qBlvT+j0HPlXSFyOLNzZXb9zVWu1vrS4v0edWrm7ybwf94VZ8P30H
dQDiT/0dGqHn8ClEUpiTBJvZgFspIPj44+T75HziOjoIujYiE9RzTrFWMyGvLkunzuQyqcxMhUbH
6yp0tE6XhxQacKWs/mxk8GWqVX2sSSzmxHJOrG6had2aCy44ldmVtF+VCjCSKYtgeTrYvfebgyV9
3U1ZZQu99s5qLvEQtZucWOnTD4ac0jS1/GdKo7ZioIWee/7Fs2dBxrLEu8w3QMZRtAuXCLqtm7B+
E0btmnaqtW2gLdRGN5Nq7htg5mATXtXU2jTQRCNBI1Cmuqk6amotnqrGoRLcff2m66miERxuxxEB
hzlshFOgvTRjTQbl8yHPttdLwM66zPaj8O4W4t1r20M8V9weshHQRkCbIp6DcwSoa/9RgX+swIcV
X1c8raDHFThHkaMYqv2ZgHwoCGpHuz0/G+rp6VSzhje2r9o+/obZZjVvF8DlzKJDArDdvN3c2fDG
MOp8w/9Y6r0RHI28KoAGRS26xLdIgCEktWSQOJ8dNGy3k89mIL0cp/aUZyYv+1UVdR1UD5Da6FRq
Y5KpbbWbVtGGy+ewmNuYlf5q4+Wwz8HVMAFnsKk3wBWKBpr5Rsn6ufb8VdZMjCVqx9qO0sYNVUZj
WVPv1pqiGlu2hDXvG5WplRJZeoaiuL4syz169+aeL20bqMmrnX4otHZssLe4zT4SubljzytfWTfw
yO+PfPHFvY2J9aur3T2r86SGoryiplW2dFomZah/OXjxiaCmqK6MUqlVtLWz1tI0dWdn95E75v0V
OY7G4nsTbGlvR73BaFIx3Qsv7rj511/fUjfzwPd+MX/kT/GR7NI15pGi1ipu+6u4+Luv4VHMHNrx
cuIvO8Jbhg4cH6jc0lHuGt7n162qrvyvd95i/HMI5D/jSnC8UkFLH0GIfEeF9qI4+ggxCER9BlwW
4aPkrZSUMuJbl1jk/xybcWXiLOx5Bvbsh/cjHnUKZZP5eDIHa0wmdW5uQWabhiVfd9CszqcY1mB4
Y9IIJp+JMpmykt95gIMmC/XUCeJ2p77wuPL7juS5ITFoUl95ULO3/GCv4L3tBzdO/vcdHarEu2nB
wPaJX/u2pUM0te18Qu87dGb3vtfv7q4Zua1b1T/63GJiIRxK7/ryZCOxkQfq6ltB3lo08yyyL59/
Sq4lwXVeqAOrnK09V0vVdiCU7mznHDjiwBoHdjjWZqe1X0xfTqd86QfTj6XT6VW9toJscNReTqNT
O9OTr0rEiG4IOrKbZAm+GRRGXjY2V1bY7VdU4BbxVdC9UutmGldK3uSIeBjgT+T6gqzsklzNuxje
w18jNyq90O11BK7TURn2mnZn/cAq4zuEyr2WOtQ9Vp9TOXjzukvP0f1g7wopW1JVo7e2VXPBkLPD
lVO26WD4kjNJ98gjl76V/O4Oa//gLhhuGlbXfYK45G+hZ3c99ndX/Kq4njkDGRQjGXGN5E9t5He8
S/df+TPmtT9rMmfI940oRL5FgmsTvEUfRzvhnWY/GgPcPmxGuyHl7cWtaB6VQf/naIZYBq4y9ATO
whN4ghKoB6n36Xb6F4yNeYj5LfNXto+NSRjJnDRDapPukp2UK+Rl8tsUipQEOehGkvPBmymkQU40
hBDtoI1iH6FcvOGynMHLMmMIv2AKpsDnIymYhveuaAom3xMdSsEsVN7fTMGkzn86BUvRLvRSCpYh
PW5KwXKkwv4UrAQZRi//F4AD70nB6SiCj6dgFWqgcmF1zMihd5oaSsEY5dM5KZhCMromBdOogRZS
MINs9M0pmEW59DdTsAQV0z9MwVL0Mf1+CpYhG/NmCpajXJZJwUpUw/IpOA1tYgMpOB29xT6dglVo
t+RQS2R259zk+ESMt42W8K6KitV8XzjEt4/EyviOmVEH37RtGy8SRPm5cDQ8tyMccvDdHc2evqaB
jp71/GSUH+FjcyOh8PTI3BQfGbt6fvfk1vDcSGwyMsOvi8xEmiPbQk3R0fBMKDzHl/PXjPJk+PNw
G8JzUYKodFSsdqz6jIIQlF8z6d8RCHYxPhmNhecAOTnD+x39Dt43EgvPxPiRmRA/cHliz9jY5GhY
RI6G52IjQByJTYDY18/PTUZDk6Nktajj8m5aInOzkaQM/bHwjjC/biQWC0cjMxOx2Owap/OGG25w
jKSIR4HWMRqZdv6tsdjO2XAoHJ0cn4HdOyZi09u6QaCZKAg+L64I0lypQW9kBoy0LUlTxkfDYZ6w
jwL/sXAIRJudi1wfHo05InPjzhsmpyadSX6TM+POz9gQLql1/mOzUQtE4ixkjDk0icbRBIrB+WKD
Oq4Eni5UAX+rAepDYcgxPGpHI0BRBlAH5JFR5ACoCW2DP/4KDlGxF4ZnGJ47xLmEshtmNUP26YM5
AwD3IHKaTYr0I9BiQD0CtGE0Dc85NAW4COSxv7V+N8zfKq5DRiaBfgZG14nPCKwWAdlCsF4UqMOA
C4m0PCoX5flbc/nLs/9P6TaINNHLFJUgH9GfA636XB4rHMr/nZX+YxpK2mJc5BITeScpJ0XefqDo
F6l84kyio5i42oxINfA5K/bAimMwn2j0M8pRkXcM+knOEYAnUtq+Hk6fOVGCkDhvZW9RWPlf24b4
5Bx4ZeQqPfSL0u0Q11wn4mOij5GxCbE3i9bAaeREN4h/DqC5mvNoiq9DhKaB8v92XgwiZlbUY1i0
9zjQJm3vEHlOgzW7UxqaEeOAaGj+ij0mdfNv+aBXfCYjadtVfIhlyZPMXZE+mpJ/TFwnqbVZuEdA
72FR2w4ROy7ucRJsOAnQlfIRi42ncNdKsyLL1fv5z1ybTlVMFnTmX1VEcAk5mkv4j9467iPvMtd4
MXKR6vlg+APK+YfGP1DPYTuqx6VP+/+p/vf+Z3EpLjlZx/HfwyWkQMElz+Dfv5/Dzb6PydtD9vvy
NG/8Pbznd3f97uHf0WffxcK7fKH34jt4CduE2rc1XPBt/NvzOdxvztu4c/Vv+X9dT/tPvIXfpJe5
i28sv0E9/PqJ16mfvFzHCS/rcrw/ejGHE17UZXmXsPnkD+u4JcwL8tN1nPoFHHwBWEqEvOdtnPN5
/P1TOZz6FHdqz6m7TjHPPZvD9Tyz5xlK/TxmEQftYWgnMCsomVefwuqn8EnaBbxMTz3eznFNRpyL
nNAo1AP3YWgRaCegvQDtLLSL0JahyZCAcwVDntUbP17HPQnNdzx4/Pzxj44zwhLOP6nReZ9dPo3z
BEOa2vvtx2H48eDj5x//6HHmMdejfhDTiu6C9jC0E9gqFDLoUc2j/KOzjzLHHrRxvoeDD1MPHc3i
+Ad9D1J7jx48+tFRuqJJhQth2ULkg0aDeBZQ+x7xfgJbhDyseWDvAwcfoA/ej++718YF78WRI2eP
nDtCHzyC9zaZsAYIe+C+BxqFKsT+C+J9Fu4HReyrYl+dojqBNUImrb6n8R5KfRirDzsPNx7ec/jh
wxcPS5ZwhpC338p99ZCVOwjPiwewej+3nyI35/6H9zPqU1gLGtciivyKs6DO8Pbsw8O3gqlyBN0N
ZdyOqI2bj2ZzMWhRXwb3fXhDzIQA5aCszjxZxu15Abp3QXsYGg37NpzMzgXr64UKbxk363VxEWgz
4KQ5OMuf7c7yS920XwLes3W4kBuBFoQ27MvmZp/DCqjhFd+htngLuZ4lrBOy8SZvNrdxyMUNeXM4
nSvDz2Laz7ho/xKWCWYTz6lp3A9S9UHr9eVz6735XLfXwe3pwl0wcwmrnvLaODB1mjClVHk7vbnc
xY7lDsrXgTu8Sq6xvaedavdmcm1eHaf27vFe9C57mdyQyZ/pMvi1WO3XuNR+CiM/diE/p25UD6v3
qBm12qnuUUfUd6nPqZfV0kbAXVTTUOz3ILw3E7OgvIOLA/12e9eSdLmvKy73bYzjO+JF/eQu9A7F
JXfEkX9oY2AR468M3n7gAGrO64q7+gPxYN5gVzwEgECAvQBo8hYzUfNgNBaNzdvFC8fs9pg9Cnc7
ssdEVFTEEwAl4ZW+OISS3WgUhoEPQcai0WgsNj8/jwmCdOZhDCCAgZKwTtJBi2IYsotomA6phXBG
hKGIJj0yYhc5kfniWihKqKJZ/xv7dIHECmVuZHN0cmVhbQplbmRvYmoKCjE1ODEgMCBvYmoKNzY2
OQplbmRvYmoKCjE1ODIgMCBvYmoKPDwvVHlwZS9Gb250RGVzY3JpcHRvci9Gb250TmFtZS9CQUFB
QUErTGliZXJhdGlvbk1vbm8tQm9sZAovRmxhZ3MgNQovRm9udEJCb3hbLTI2IC0zMDAgNjE0IDgz
M10vSXRhbGljQW5nbGUgMAovQXNjZW50IDgzMgovRGVzY2VudCAtMzAwCi9DYXBIZWlnaHQgODMy
Ci9TdGVtViA4MAovRm9udEZpbGUyIDE1ODAgMCBSCj4+CmVuZG9iagoKMTU4MyAwIG9iago8PC9M
ZW5ndGggMzMzL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nF2STW+DMAyG7/yKHLtDBUk/
6CSE1EKROOxDY/sBNDEd0ghRoAf+/WK726QdQI+d186bOHFRl7Xt5/jVj7qBWXS9NR6m8eY1iAtc
extJJUyv53tEfz20LopDbbNMMwy17cYsi+K3sDbNfhGroxkv8BDFL96A7+1VrD6KJsTNzbkvGMDO
IonyXBjoQp+n1j23A8RUta5NWO7nZR1K/gTviwOhKJZsRY8GJtdq8K29QpQlSS6yqsojsObfmtpw
yaXTn60PUhmkSbI75IEV8X6HvGE+I2+JVYK8Y32KvGdNhZwSpwr5wPkT8iPnN8hHztNeJ85Tz4J4
e0QuiasC+cx62qtiDbJMiEvcV7J/VSLf/WMfyf632F+y/xTPJdm/wnPJu3/0Jtn/vqRLu98OXh/O
92csQt+8DyOhR0CzwCn0Fn7fiRsdVtH3Da9XoVQKZW5kc3RyZWFtCmVuZG9iagoKMTU4NCAwIG9i
ago8PC9UeXBlL0ZvbnQvU3VidHlwZS9UcnVlVHlwZS9CYXNlRm9udC9CQUFBQUErTGliZXJhdGlv
bk1vbm8tQm9sZAovRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDIzCi9XaWR0aHNbMCA2MDAgNjAwIDYw
MCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAw
IDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIF0KL0ZvbnREZXNjcmlwdG9yIDE1ODIgMCBSCi9Ub1Vu
aWNvZGUgMTU4MyAwIFIKPj4KZW5kb2JqCgoxNTg1IDAgb2JqCjw8L0xlbmd0aCAxNTg2IDAgUi9G
aWx0ZXIvRmxhdGVEZWNvZGUvTGVuZ3RoMSA1MzIwPj4Kc3RyZWFtCnic3ViLU1vXmT/nXEkXEAiE
XoCE0AMkXhJXukjiIV4CCQECAwJBzMtYFggbBAZsY8fZ4Cy2qePE3abjOJ2MJ3aTdjdNs7XjZGmS
abJej71tk6aT3ewkzqPuJpt2Z7fJxLPtxJP4st+9EuQxs//A3juc+52jc75zvt/3PCwvHoihDLSK
KGSIzk0u3FNnNyCEnkEI50YPLhvWiQm6+FVoRNOzh6dujGTcC/1rCGWvx2OTexzGy40IyUPwuzsO
A+9yL1LQX4N+cXxueWUwPcH3/xb6U7Pz0cnzu3OHEMrN4ftzkysLK+gwkLkr0BgSk3Ox2z0PfQH9
cwiRnIX5pWUWbWwipOTX8+fC8PJPJpASvk8okVhCp6VnSDOzZNk58lyFUqXW5OUXaHWF+iKD0WQu
LrFYS8vKKypt9irGgf6/PSayjggCJWEneRq0SCP7JYyqvJdpEfqT85JE/J73MkWARJcofljMD1+m
JfhL72XMj7Nyo9zKys0mnPPJa6+Rp+9GTGSKR3oamk2yiNIRUnjkrMroYuV04neXufhzPybR7Lu/
YBgiz+LPQJBr8zZx4NdhrgIModrNOtVKidlElGrW6a62mE2uk4cOHz9++NCOvaNje/eOjd537fTD
rz6E634wlzg3z/PAqAN4OPE/IzN0NBKVUkNb7ZTZJFGqnR4NMLO4qhsx69QAS48b/5HJoKpEIqm+
2T42NfnifQVrB2ZOOpd2DPooLBERfF3cqisoo+WKvDLN3PcWhqzR9WhdvMNDabLhvF7Yi4XzGlEV
7KZiVWZXagcCO7iq7cRsoq0ghp6iXXB8GaXiRcFvjzw8EQuOqe07ve1BsbKmonu3cmHA0W5UEDbY
wPiNCqqlIoarNpnO4eEajdNpkBkUBWWaCrtda6zWXunPVBWY5WmWGjiDD/0jKcE/FXSGFEaV8Prw
h1yh8PdT9gQrYHsBG/F10pKax1Jm/u/CZEZoY3dmF2nhbmEDdwvw+2jzU/wAOoKkCJU41SplUiA3
ZosYpkjvcBwp1RdZSgxFVoSxZfM8vkkeQ9kgPSsjKqUea8x2zE//UbXJUlVlNVbXBQLkseb+J4Pt
T/U3Rd4cHPiXpJ5AP8QPeirkT6PiAVMXYh5B1sVrmpYLtNxsPmksDpr9A/0fDHtsXVOR9wcG8Eat
RqMd7/qUYf6rpqWP+xPD4GFexv2bt/FfyFGkQhbekvUAtwxUYKd4jahNMqLUE2cjAcVgzDLB8vy0
bGt/PRMsA8LSPxXxajMytPURrw4+5GiFb6TPVheLN1S0jPQCMdNo94wO7BocdbtHBxnHwKggx2nA
wAcYgAlrzG6etZXVY8DtqnuFsRrdDa3+3e9dCP4o3DD+m/6BNwRduDf/QsLkAThnEbDwCKKaU0AD
DhrWRSt4EHijdWPju0OnSFNi4vipYUdUS5764VhvYCr28BSZV9x9lglFToxFV6vK/pDBnW1LRM/F
9iTx5TeKQYQF7QN/j8BXaPRYwFkOR6VSOMtTprqF+YGs4mpjjazGq63OzM6sm8N0VoGimmlapKT6
PHel/xeBAN4oziw261vylBrW8wcGa8TqfLMiu5X7kMFakaawVFXLvbOll2m+IU+AX2fx2k5FAbPH
xaqO/+oSN/38o6udeINb4eefP3JEOP9eWCMGXWbwcWNrCb3318a/52YvP0lMO+9e4SNHiJ/7/OZ3
cQKtg22jEpY2Y2rfQsf650kcuoFPAN9A+m07AyOlLS5eYBmhVamzDD5tLt9p0zPqbI8oLV2nKGqq
16v+iXvw/kN4Y8pcYlMoc0tFokypTMsW5t0SBCMoBP7vBRvW8FmlRPOVBkH9KqNexDpLjHbM70NK
NPevHHpgLTF3lHDjOENb1m7byX2I5/DoYFVXmWraFlyfjJ76q550M1FYnSUF79ZNmKrtcP7o5mdE
BDHGvx3PhHAGvibR8AYOm1p5fzG7q60ePcUm7ZtYkydJhblXqjLENlGG3uesb6MV8uI8p06fa6ov
ORQK7qBIhtQmkdioTEObU1nhMSnt5da3pnrbHOmyjDSMX6c9RnWFWKW1KMuNMnVOZrpYhAmWKNkS
f0cFqTB5IYTp7QUSuUWn1maJcdrRmg59ep44if8Y+OSfyZzg53xsSsLDu6XF6uHjetLdrcdwhHvb
7xmZV9uH6539DYrxUG843B5sLiBzzt/olgZNrS1lqnx72WBby06HTPAjxeYE5IvrSM1nLtAum8Ie
IDFvKVaj3N5EolgnDUt7Vh8c2tdOznG3BgbWFtsC0yN9PTN4Ixg5NRpdG74Kqr3z1EA03h3YJ5gt
b2D/DXYoQ7lA8R6S5OwyuhQU8MdnxsbGuD+e4j7hXrtFyFGuyzHFMFP1+CdcFz6SxMAIdlKLN1Ce
YP1KUImc97NUvrCTBmw2npwa7B1+MHyiJ2xvMcggxnGfrd2DMxjwipqAsbbe9LskLzUcqg94yQSJ
zVoMnFRmCl71zfBjj/f9NvRz5qUuvMFwz+FuBpbjdQErMcQcG1mD9cZkJt7GCoxFcHtasZ1nJeLv
Ef/eiRPHdlma7IrsE/3jj8S7OiejHZ1xcm8698HO4xP3fEdenJu7gyEGMXcmOBP/bjwunG8WZLVC
ntElNaIyF2Ilb5ZC8rUKsrrYS+znnxOpqrwocL9iNdxnl+dQfj9peZS5KirU29Ryl73OLmfLKq4y
As/8zSgZAZkV35AZC3rwUOb8V9ovXva/VM/9foR7/fyOVxp56X+JPSC9lGG+g2d5HjdwJn4C+/kY
wfO48c472F8L41oAJwS8tclxjUfgTWuEUGilhY/Hai6/3rS6Wv9y44WNlt92fP/77b9sunLRB/us
rjLMxRcY5uxZhnnhIuAc3vwMvwf2YkB2yAguQLSR8li3Q4MesqSMoqGP+cggEQqCRhK2KCtb252W
NFI72hOLd4Oxl4o1lhDbNGjEDeJxZ3VnwJ2lc9hlLJGavfEgCR3s3bGsa1jq1tZOdHcplPiH40M9
Pu+QJNchYBYBv/sAzlGNUBNOuoCdEiKFCyKC4A8ySTax8sUQ2AEt5Cw4ZUSxq7Nhp1ylzRbrS1TS
ZpPL70rPyc1SpCvE0ozcArbH5p/pKfMfHK4IFNuqOlu0tnylBGdgRl3cDJKIRGkiKaElmaUm01Bw
hMkJNw/GVcn6jgF/deOXIRaX8xUDJB6lpAK7ACGwChlOYlSPnQC+YIUCOG78sxPhqbWWxsZgkYpk
D3tH7//rYTxTS3a1+vZM+gNjvLYfjY4xVLbeXV6f7zk2PHpfz0Od4+PtgcndsC+FEoDFHaEuMKIK
0LNQogjWnxRb2FextR9gkyvQPPlWUcPs43OMjXXNvBivPd3i66x19fRmr4X6V8LYm7Oj1h+N+isG
8jXDFb5EwscM4pLuSkdPe01tM/d2r7M6FKp2CrLH4Qy3yX1Ju8AqoRqkNV9zQqhRKD5NYz3eClgy
gm8rLDZfkDeMup09UzMpw7CG2LZ+A3dVPCYYRmYhYydHWSw1eWeCpPtQ744lnRcso24cLEPFjY5H
un3eYYnCAXYxALt/BFjo+VwJDuT2NOJkdCwEt0qeh5ZhWmXEzzHc/p/vMeePMWX3uEOJ0EFT/t9N
44KzZ91PxMwNFl/AwdKtI/vCXpf1QpgA70ng/T8Q69OEWA8lplFlnsSP3Kh/g3vTSeYcd68R7zjM
KwFAWiAWaYVI5HElXQ7ysuBtFiudjEcl516t/bfeZ+qX9ENvtbiO1a0PTpJDWdz+vyEMs+Y6bGGY
ppXKlxhSlCbkmRWSTt7g/RtrsBW/X8mtVpI3FF++rEzdLSZIeCtXYDC9ZBT/quyCqkvzNasj4YEB
7tZjpH3f0Olje5YaiGimp29kOtC2yNsbzTBXh9eio6ciwbeC0/sC3fEon6thIwlZTda0qVsALegx
WXKlUlJ0X4a+zdU0rAp0mspzRDdv3uTW9+3Dz9SWVDGanGp5TrFO62Y4qG9wSPDlAuDbDXgVCnj9
n7UyXXBWX9pu8ocjt4bctp5o383IBFnJ5L5Q5xckq2WPT6iWiSEtdScIAmapOwE2e6B28rD4pZn5
4ObMfAcO3rlzR5g3hoNfvzvQZr7K+gkUWWhz70IHaUnOw8i3+SnZRfr52j5ZDeVslSCsUihdLNYk
wHxCII2me88ce+Txg33OLJFUbMFSRqopHd19KLOI6XUunCo5eAWn/bhS6kjT0cG+tGZTw5m4sTXo
MSdjSWAzhs9ATakRrMgoV8rEkF88Qrx1yY18bTmEazAt01mbrM5haWuT1aOVEe6Lm3ypuV+uthdZ
8+S5jmy1jinM+0bdib+EfJAl2DAGhYHSMBjyAXySu4J1DJfAHdxtBm9Uck+OPwyF4IiwzoNGSBUx
Iih7bESThYmW0+GPlNwRfGLk1sf2jeeEORGYo+PnKLKw1YZD/B1NhU9wR4ju9/9h/4crW7U7Rtd+
/Z8nX5jI9v4ZSUXCFf19+63mry7sYM0xwt8m0gCL5APrqDHuHTS6PQl/65aP4UpugmrchTqQF/o+
dAF9hC2gr/3oNHLDyDTI/zzUzCGw5THItyGImGo47yzKRzcA6TCKIAYlUBziyCT48RjwioJ9XgDK
hwKw2gMv/5SjZrQbnUM34f0E6sU87Ma78A/wFfwxqSQBMkr2knPkV1Q5NUydop6h3qQ+EZlEIVFC
dER0UfSxmBHPip8V35YYJNWScUlC8oTkNcnndCZtoCP0A/R5+mf0tZSETtiXQkmr2PrfTjNEWSJQ
MvTsNg6l25hgsOTSFE2ArkrRFODjTtEiGB9O0WKUgyZStATG52EmFqVDrxcdTNEY9rqeognQ/5qi
KdSG/j1Fi5AM8E7SYmTA1SlaAuMDrfMLhxdnpuPLhtJomaHD121onV9cmF+cXJ6ZTxgctbVMJTQO
e+v8gcWZ2GJ/bPrA7ORiYD6xHJhfnI4ZnHbGUGdI/QoUa6uxORmnIzUSiS0u8YwYxmFnGHZrImoF
aRbQYbSIZkD/cbQMOaoUtFoG3w7Qajd8+TmLMItvJ2HGDFAJGHegWngZVJmiHIA7P/eAwC0GbT+0
09CfhXWLoCl+3bLwXYTxGPBwwhoGvnWpfb5amxxjkQ3VwJ8TZjlhh2/OiQjt0vaJGHgdAkcGVn6b
I5Xynivw27e9A57/BfcqKQ4KZW5kc3RyZWFtCmVuZG9iagoKMTU4NiAwIG9iagozODUzCmVuZG9i
agoKMTU4NyAwIG9iago8PC9UeXBlL0ZvbnREZXNjcmlwdG9yL0ZvbnROYW1lL0hBQUFBQStDb3Vy
aWVyCi9GbGFncyA1Ci9Gb250QkJveFstNDcgLTI4OCA2ODMgODQxXS9JdGFsaWNBbmdsZSAwCi9B
c2NlbnQgODQwCi9EZXNjZW50IC0yODgKL0NhcEhlaWdodCA4NDAKL1N0ZW1WIDgwCi9Gb250Rmls
ZTIgMTU4NSAwIFIKPj4KZW5kb2JqCgoxNTg4IDAgb2JqCjw8L0xlbmd0aCA0MzgvRmlsdGVyL0Zs
YXRlRGVjb2RlPj4Kc3RyZWFtCnicXZPBbqMwEIbvPIWP3UMFtlPcSBFSShIph22rTfcBCDgpUgPI
IYe8/fqf392V9gD6bM+MP8w4r/eb/dDP+XsY24Of1akfuuCv4y20Xh39uR8ybVTXt3Maybu9NFOW
x9zD/Tr7y344jatVlv+Ka9c53NXDuhuP/keWv4XOh344q4ff9SGOD7dp+vIXP8yqyKpKdf4U6/xs
ptfm4nPJetx3cbmf748x5V/Ax33yyshYU6UdO3+dmtaHZjj7bFUUlVrtdlXmh+6/tcWSKcdT+9mE
GKpjaFGUdRXZkHdgS7bgBVmDn4RNAS6F7QbsOG/Az+QteEl+Bq9ZR+ZfmCt1asYswRvGLMBbstTf
kRGjC+bCR9PfwF/T38FBJ/8nMP0dvkXT38o8/R0cNP0t9tX0dxJDf1eCk7/Up7+TePo7caO/gbOm
v8O3a/pbxJjkDx+Tzt+B6W8Rb+hvsa+hv8W+JvlLPP3LF3A6f/w7k/xxPob+pdRJ/lIf/qbQODdT
k+Fs6L8Qt3T+azD9Hdgmf+Ra+tutNFvqKrQd7sV3O6v2FkJsZbk80sPo3n7wf+/XNE7IkucP1q3e
HQplbmRzdHJlYW0KZW5kb2JqCgoxNTg5IDAgb2JqCjw8L1R5cGUvRm9udC9TdWJ0eXBlL1RydWVU
eXBlL0Jhc2VGb250L0hBQUFBQStDb3VyaWVyCi9GaXJzdENoYXIgMAovTGFzdENoYXIgNDkKL1dp
ZHRoc1swIDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5
OSA1OTkgNTk5CjU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5
IDU5OSA1OTkgNTk5IDU5OQo1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkg
NTk5IDU5OSA1OTkgNTk5IDU5OSA1OTkKNTk5IDU5OSBdCi9Gb250RGVzY3JpcHRvciAxNTg3IDAg
UgovVG9Vbmljb2RlIDE1ODggMCBSCj4+CmVuZG9iagoKMTU5MCAwIG9iago8PC9MZW5ndGggMTU5
MSAwIFIvRmlsdGVyL0ZsYXRlRGVjb2RlL0xlbmd0aDEgMTQ2ODA+PgpzdHJlYW0KeJzdegt0E9e1
6NlnZiSNZFkjWx5/sUYW/sq2bAsbGxs8YFs2MeA/sR2MbGyDzceWbUHCJwUSEoITCknz4TZpoLe5
aZo2QSYkmLa30L6m35CQNjS3N23wbWmTtqFw70vavATLd5+RDCZNc99696311nqSZmafffbZ55x9
9tmfMwqMbu0nUWQP4Yjau6XHH331928RQl4hBGJ6twWU/WsvliI8RQi9Zb1/w5Ysz1t/JoTPJUQv
bNi8fX1ozYUaQkwLCFGyBvp7+n7GrUDYtQZ5lAwgYlfoLj2WH8by/IEtgTsuJpmfwfIklk9uHu7t
ufTPG58iJHczltds6bnDv0GfwmP5R1hWhnq29I9v+tzrWH6XEEOff3gs8CbJmSGkeAWr94/2+09I
f72MZWzP/QFxgF/2iUJQx8qU4wWDaDRFmaMtkjUm1hYnxyckJiWnzEu1K4405/z0jKzsHFcu+f/0
I7wivELuFPaSOLJdu9/04RcRG7mdkJn3WOnGPXTr/91RGLQ7JEI6+YD8aU7Fd8nPyTdJkLw2lxoy
IZutHsSQS+R98oO/xxX52WGFBl4kr5OXyYt/h46Sr8E0+RdIRD0/hRDDVZK3oAvH8yzitpKDcA22
g4McA0mrLUTe0cB/Cq/FMEOmcHQPkynyMFSTKWGMS8SKf6Evkye4vfQc+SmOeRU9iLgZ8iZ5BQqg
hoyRk+RpjcEY9ndwLkeOkH8kR8jdN7DC86FvC3vpS8Q68xfyEvm2JoHdZJx0X290Ff4Mh3FPJoIB
Ztf0O7OV+jpuI32J0ukvYOFBsgGvHvglUh/kln5iOs+GhkMDIJAv4Ah+A03kEHJ5PnQ69BRZS47T
C6SN/AeOu1qwwtcIUWs62ttaW5qbGhtWrVxRf8vyulpvTXXVsqVq5ZLFFeWLykoXlhQXFrjz83Kz
MjPS5zvTHPYEm1WyRJtNRtGg1wk8R4Hk1ji93UowozvIZzjr6vJY2dmDiJ45iO6ggijvzTRBpVsj
U26mVJFy/Sco1TClep0SJKWCVOTlKjVOJXiu2qlMQmdTO8IHq50dSvCyBq/UYD5DK5ix4HBgC6Um
YaBaCUK3UhP0bhsYr+muRn4TJmOVs6rfmJdLJowmBE0IBbOc/gnIWgIaQLNqFk1QYjCzboNcek1P
X7Cxqb2mOtnh6MjLXR6MdlZrVaRKYxnUVQX1GktlkA2d3K9M5J4df2BSIuu6XVF9zr6eNe1Brgfb
jnM14+P7g1ZXMNtZHczecSkBZ94fzHVW1wRdjGt98/V+6m90CUEhXXIq4x8QnI7z8ns3Y3oiGF26
9AFhoBfFOz7udSre8e7xnsmZPeuciuQcn4iKGvfXoIRJYzu2mpz55v3JQe8DHUGpewAWRSbrba4P
xjbd1h6k6V5loAcx+Kt0OkqTHdaOWZrGv1dNUBAoDpSpw8Emfv+kStZhIbinqT1cVsi65BNEdbs6
grSb1ZydrYlrYzV7ZmuuN+924mrWt7SPB/n05X3OGpTx/T3BPetQnzaypXBKwei/JDuc4zFWpczd
odEqOKrlfYNKUMhAsWCruQ1QU1iTcUkrRP8l/LicjB1kWGOUMieyYXxqnDXdkd+2gQRkoOTlButc
4aVvbQ+q1QioPZE1qpkocGOLnm5cosFqbfmCbqc/aHMuu76ebFg1gy3tWpNIs6CtKki6eyOtgu6a
atazUjPeXR0eAuPlbGo/TTwzUxMLlOQXPGQB6ahmxHIV6lVGzXh73/qgvTu5D3faeqU92RFUO3CB
O5zt/R1M0VBC2VPYnUPrMUirWtvrW5z1TZ3tpZGBhCsYOz695hNsnO3JYTaockFDukFpp8lcBxJK
iFC8CDiXVeA9qE834CWhwDUsU9VlFUo7JJNZahxGMFup6a+O0LHyTUwFpk5VdbPcdKyIfKrqkh0d
jvAnL5ditRLpGFsYmFDrZqu4dLQEiKPIRkMxWSYwnVfanf3ODueAElQb29ncmHg0KUeEock8slat
N5XmCAvFRBxYPVtgwgx6XclzhRus1crXi3WfqF4+W62MG5z1LeOMuTPCkODIlwcJU2G11Jqs7X62
n53eHtzEuKO1/Tw+oapsLw+wbTvuXN437mxpr9Co0YLcmbyD9RVD6qG+dVleLhqzZRNOuK9pQoX7
WjrbT6O3VO5rbT9BgVZ1L+uYmI917acV9BUaljIsQ7KCwgqMUzMWDBp98mmVkD1aLa8htHLvJBAN
Z5jFAemdpGGcNIujiOPDOFXDsQ+uUsIAyhjtd43Sx9ZnV8fAeHcH03Eio0TwB0FwLkHpOJdMANVF
BY3O/mVBk3MZw1cyfGUYr2N4PWoGyJCXu2NcqnF+kJDHvCYl1XjrE9rQeetJ/gQQd8UJPZ92uWhC
J/yq4gRHESQTHEMLDH1Cr3NeqzgBDO+xOqzpDqujmiqh+XAkNCC0ffT1av4cCy5IDHr/aozTTOTz
any8DuIpJBDQ80YOwGQQRUknCJMzV1VXUARREAViktwmn+mQibOYKk3DpqOm10wzJt1FE0gmxVRg
4kyUY/QxFs7OUc6oB57nDALHYU+VRZWemLJEt8fjgQR3V2mRu6trZLTUU4ZGa6TU6sGftcwa7yks
6Orqcohw/cdHh57E74/g3dBSKIUy/JaEWuECLZx+jf6OPhGKhv+Y7ptORjk1zrzH5/GrSCrJJA0v
JXqTktK90uTMWTUz01UnZbuzG7J92cPZR7OF+dZawdo8f75sbzZKchOREiExkcg4SneXdLnyckyZ
+zK4Lxd1SW+7LnvclwsLhLR8WrxgCfUUpdJ54LClIlSyUIjmuLRoqnfkA71l67f2eVsOTq7f82Kg
ZPqWhJLVi29ZawMxZunmL4+56hemUXjKMGSrOXjh0ONv3L2o49jF/YbqrW2Flcvi8wduLeMm5lX2
ee++G9dlP8bKfXwDcZKe08Q+M6VmGMhK4p1yXnVSZ/rh9GPp1K89gulX04Ur6ZAY5T1vBjNO9YX5
WXXsqUYbourMibrGeVKsxYyROMq/8rIH3C5XF05rZFR6uwtnRVwO64J86sQ5xFk1MNORysV5lgCb
Z5yVW2HLv6XY01LhCL0DHKUQRXFJ+bJv3L5uf1sG9VUO1ufktYzVTH+bq3PeUlVsjipWVXmXX/Uf
aZ9uIjMzZOfMe8ItugsxGQRjbKue6OHzqHvxqplCkpCBAVpqimBLB1dxMWGXppdaG9TLRFKvFsd2
Ttmu2qgtWU1uTO5O5vzJe5KpklyQfCz5bDKv77To4IpuRkcVXYGO6mJ1nNlHYknl5UqcbFfX6OXr
c9VWECepLd2SMBhN6c/3/epoB2TmNgdq67bfWgCZa595x1Z5z/nD3Mi1pzv2tmRmr953G9d47ckH
37i/KrwfLcJH9Iu6C9p+zFbjyT0+DjhOfw8vwEHehxNQxag6QtyurqLLRahGHjfrH4o9cQ684MLL
01u/J3z0Fn7C/Paj3h7CzGk+esiVqi299jVMMSgp8ZeAbMzznreBLbK07KnGIHdbVkOKIsURa1xR
g1FAxfVcrqzEW2SBi0akt0dYnzhDJ84702ljWroEtFnr9Dp9ZImFBRna4of1mT9UuP5Lmwq6V9cl
sVSWhq5wAuYm+OE9J7YOPNqTH3pr84irdVlWltrq2hSgjp2vPdaKKlISV+LJMjj3tvAXQuuUxUmp
yf7ulkde2T55Iq3t0JaND7SmTYT1moZQr+eRtaeJZebdk6KRrIhjU0rBKcXF2Xfbj9rpeTu47T77
cTsXNc97HptF5s6eqplJNrExKkbSW5hWz5kz2hOrx+1iSu2Zo8mcp0iOj8sHZ5ouznpojhLH5ZQ1
licWmpMXZgxu5Sqcq2rLzFHl3pq48q6lTlH3J5347FenLzN9PIS3Z3B9OFKjFoD3DAU/hYvohQRV
aBToMSEo0N0CdAtgF+CKAHsQcVY4L/BAtDECDg0/TA9H2Kowo3wIEoVXPlqgrf8N/nrSoJaA4KVn
dODXwUUd6ERVbBTpMTEo0t0idItgF+GKCHsQcVY8L/KChY8jLdgTqvzNPaHyFRbEFjviINwfbuFE
fvq11z7m+EUf/wDnxezlQrSXClmopkpeiyXZ25ACKWlyrRDbYJQko6QmQ3JyQgOzjEyLKz1uidlF
tqH+xirGC2EhSxHDSJv3fneP6t333Z2DX9m2PDr0u6ju9pGBXzduNkOSsXb7N2yND53btf/nD64o
7dm3Irql95sTofH+PnP9gcHKsMz5rSgT5pvmpxBIArjVuMFIBSMYDWBk/imDlBAveZTwIjomE9MO
CbXDZDK7zaDnvOd54COqw8/qGL8T7gcKSmxiHdUbAfR6g0UgfMQ+lrnL0At5XF2zGsX0qdTj6WK+
qUzzTOibQI/idFiZb+KWhS5uZ2KFuyA/tA/+Af7YGtoovHLteTgb6pzejPM4gD52seZjV6oeA+lE
Z2kyN5q7zX4zt8d82EwLzKo5aD5v5vVcJ+ELeBVdJk+79XpR4PiI49S80dpZwz3C1tVhvfHl5ekv
hn4JmXQ9XmumnxL2omss/Gg3010pdCs/wC9Ck+gkp9H6zq8MRp2biLFUBsEDQZs7GPcLMhETZY0J
krKCwvRYB6B1cKZlFsu4iiWVAA5a9xFkhFqbbvV9sX9FcZua5MmOQQec/e/c8o8PvtzcYf6RISYu
Z30B79ZsGfZ3iG9CW1ZCuk6TNPReKei9OAVvs4at1F8KeSlW7/kYiImsUIxm2NBnxWiGTbZYkz/T
sBGX64Ztw33OnFfGXN91k2FbQiOWzbd6eZIO0ASE3hE4sFKBo8yybXi0xx16yz+c07I0O2tpc05J
a1kqTdt5/tE2W97yEiGruDwx1MP/seWu+fqsBaVxm9a0Pnxux+kXnMy2HWx1um77vOa3DKizG1HW
OoxAJnhD5YQQEbPeHTSgmAV+VszgYF9aDBYU7UgoGLoTsn/L3frxD7A1kDrcl3a0k+XkyGmyaOZd
1SiSlQ6JWUvK4itWLPLac7zZdiY2b6JSZ8+2ZxvjvJp8G5DF4sbFULA4uJiqi8G9GExeY1Gc4G6a
/7Mc8OeAPcedQ3Ny5ktSk8BCN2oyaaLWgh+8d+EzpqwMd3pX2H2OsKLb7ZIuSyh81xzhL0zlNG/K
DG4+d90W6PM5ZgtQ+vGpHG9fuuvk2Ian72yN+2tUzpKW4oKWxWmFbWNLq/cNqBWBrw+3H7mjSfpf
+vnF3py+vuz6DYvrHxypgYpVu251p9YMNaXnlaYaTcmF6TmF9niLJaduuG3l9rY8R+3QqqRMT6rJ
U5GeOy/OIrnqt2n2lNm1L6BdYzo/X7XZakl6d7o/nSbVGuMbLJK9gU0WozycH6rTHDc5O3xNeXTX
decLNeOvHtj3kwPeqvteHX/o1bsXh/71c3fs2udUO0uW9CxLo6m7Xn2kpfnh1+7cfu7R1tZHzu34
7vPBMz0PdLpcnQ+w8XjQBjyINkBPbPQ3akMsRa/azJm5tbxg43GX24DooJmYyVq9zqbXoSlQY+U6
vT5eT016+YfymzI9IsO9MujleDlTvlW+XRYek78qn5LfkXmZqcDMT87VGWT4EaN9R+bC1LfLkBVu
Q/+KdTKckuFpGR6T4Q4ZalnxHZneJx+R6QbkSEtliJdBJ8ORv8jwhnxJpi/L8JIMX5C/ItN9MvTL
W2XaKkOVDPPlBTKN04jfl+F3jBxelF+W6VMyPKL13ScHZFolt8h0gQzpMsgyUBn+56cy3irDehmQ
c80NzrwMA7+TP5ApckbqF2Qgx2R4SIaAvE+m62RolKFIXibT+Rpv9U3kflWG38jwffkNmZ6QAYdy
WIa7WAtoltfJtFqGEtYBSNpYLsnvy/QXjB7+UT4h04dl2CbfK9M+Rg4L5GqZZshg0yZZ+h+MHpD6
ezKc1KS4j5HjoPsYmVemMTIAYdyQ6zE5yPgE5IdlrpHxYNy4fVg8KX8f5y/4ZahmLdlIsJEhyJ/l
z2NIogfqsyViJC/Exurmmv+Ysi7P5SI3RrRru3zMC41obt43euOztmvOZ+Smz3Ua398Qdn064Q2y
m2rWdrk0hPQLhKxl+137Xd+Xzp6VyHf3CwmRAromDr/MPeZzmdGcnnPwW+6cfufO0C/R7q6hZPqh
qHirCcBojY+6Hx6BgdARYe9Hu7m3nLXlGZTLWFTrDG0ikbiL78MYwM7yICGSBxm9U+JVkYqOw45j
DurXHkHHVYdwxQFJsvd8PMRHfEr8bB4Un0DixTi71MhJLI6p9FT+bR4EsynPTQmRliLo4alIwhN6
d046lEg5Cty6SLLTTNdeT4b+WXgltElZWpZvDucziTN/pg8KpSSetKoemsXseFSUD6N6mzHaiwIS
hAbOxw1zHKdm5NYdw2TCLDTojDqDQWexhoMvtMpWj8SyiS7XSATGXNXjRmsspGUUW53FnoWeOE+c
02qTWQAWFw3w1K57DzzWHjx3rqIyKSdpQSBm/wH6ue+EQt+ZfrWh3qB73moNj2832ss16HmcZI2a
b9f8jM3kvRgN0SqKMfrTMk7wXkmbSaNpmogxsEpjqaZsi4silhupZlc4+WKS1kR8U+qFnsGBDsMZ
DZrltcIRW/5yTbYYTnEchN4HgeOE0udu77mvLYNfNP3lWfnS6muTafWzyeZOvzp8pJ0eD+cXuifR
b5aRZ06TrJmrWn6BLnJKbWOQUuYlxJznvZT7fi7NLW8t31f+Rvmlcr68HB4v/1b5W+VcazlgwV0O
1F4OpByC5efLp8q5o+XQXb6nnGIFZtrErJgLWMCm5dmxLOH26BoTHRkcy7Xzwsk2iiDew5YrnJmw
feQL7yOmcFYM6DSRfEb6LcdructNKqm3clHWzMo8l9eTclNCvuDxzbseS9LNq2rylXbsXOkM/YGR
5dR4UujfT9K3j1V1LkwIp+phKlfDUNX0cc2Hsrh1BH1WFGbh7adJwsyUJs2ESQxHCnEbWhQj3kox
ErEkYNF8G0k+nAwFmKVTWyfBXFzVcQZdrE/P2Xy6mIi7Xcs0QvO4OHUtiqC4/LjNing5RsBJoo4w
BRFGNk5+/OD0DyD0FMT8YCj0ZtUdX+1//aOOx4Yr6U+DoQ9OrhH2Nj8b+ujkzuBIybXaJTtf0sZc
gXq8B/1+MakmB14qTBCjtOW/qqbjMPUS3rgE0URWlBSZvGlns89n0+zsNO8ZLzR4Qa7VzEV2XEJd
fHxFraCapTphYbPRmFJpl93ybvkQulrZ2pQiZVY2FLm1syJMsLS0yBo+LmInRS42wVe0U6SiIi1S
cs3mSJVQPLuM6bMrq20I3KsYLGVk4laI1UdzcWwD4yaBA6t2rs6vCjzR/oe4rPIM58KsRCH0VpQ6
8k/D/U8OLdLHOlOU1MSsrLzUjf1GXenxnxzOa1o8v7a8pH1xms3VsmNV991N6cAvLG8oiot2ludF
125d7S7qPewLbcuoyI7THUELww/09/upSKnJuaxsZX1+/TpPJGfRaTlLs1ps4TCZ0nOiUVN71cwJ
YqdCClDcBp9PGBYOCReFGUHAtFcQRN4HnOgj4fVGqWDgiBaLnaO9atVO+rTkN84RuQ7wrdd+Tq9O
S9xqYe+l0JOXQgcvYf/VhHA/RHskkka1iNbq9LXhIx5Ox+nAYDCpJjhrOm+iu00YP9uhAXzAg45r
5C2gb2Tbj60LM5XM/Lzq1nruCoeunmJwsIzYEVdNU0JmLnv6HXrLD+kzP/3x9JpzzBY6UZGeR5+T
TG49TYxoOuYxtT8zD/bMw1TYZwGLJRG8VIfbroAC1bJOKbaOxjZFobdqSBQswFJ+DxsCqsRNtrpr
JGwGHcxah2PNyNFENJ0Hnjj4ZejdJ59MrdtxW0FXekxebKZnXtQFbte1fdyu74wsGVyVa9S9yAuS
Uqi0fYfpe2toFd+J9s5DaphnmR+9wHsRYwRmtN21/lrQeQVhSe3ZlPMptADz+hxLWcN8h7S0ITY+
TmwiOh1OsrLIzUJ+7QdsuTxaTKylWhgXswg/PRU0fayAhSXXzXVk8B58LnREQxzzk7GzpiyswLR1
01fGasyQaOltLGopdwDQwokdGx5Z5y7a8KXNm79eILAjJjBSvvruMyNZaktOScuiVJaELcsKrXKt
3KQ6lt6SUjXUlLIoISVpcG3Tgz/efudPH2pYsyFuoSdLP/+u1mv/tu14YBH39vr7mtNy2u9be/yE
o2WccJr+bkT9jcEJZpE+VZacsbf5zMNmajY7UzpJjpJTgPmPdjKc4qxLcHaGDw+lNJ+UAL6E4YRD
CdyVBEhIcMajAXPOGjBNoUcvl2kq7epCQ8YweItkEMyQ4a626fSpAJhCsHKJJhUGoVnbuPqRHwxP
V9J77vwf99bcc9/XXwjdu/32UBCa7xxpzl25oSI0DnsX9S3PgYc2/UNPnrDX1X5P5x3/NM92HC74
GkNfuTUklq7It4U2x7obSWSfntT26dNqHBBoxmhyrcloM5mM1XpYjiEW+r4X0E7PeX6oJqMdNChM
q0kCu2tnKZJ4/RAlPnyIwnY71XOdCl/AU57qGQ/VjG2NoAfDnOC0zOpxWT3hbR45MHGXhg9MmPkj
rq6u/YLkMrik7889O+HbQ7rntVOLndA+/TS9n+6/Zzok7J3eS3dPf+Paz5l+s7hEOxdFi67m2IV0
70UMLJl+Oxb6F4K79mz0+WhagIGKneQ0JMRKhQ2CKJPwBryu0jf0eW7AMZupehZkLMjQIhI5HmY1
WGIKzK2gAs6z8PiO9Y/0uAvXf2lz8brW6gQKAIk6Gvpt1tIWV0lL2bxf57QuyzZkLyiJG7yt9dFz
23e++kirnF/nMWYWlSTC/R8rDTvTqHPTwRZHTucDvaHD81c/GD73fRkX8Xe8kZ37wW51htM/iiZE
4qmJbxS7RWoR7SJlZ3xTItcg+kRKxAKRvnpRvCLOiNwx7QCQqxSHRSqJikinRDgqHhcpa7tH5LTW
V0RgDGikuYQMuMc/g8EhkbHgtO5Fddv2uuutD4mc1v3AcfGM+JrI+bGPw6yXSpHOiIBtfchot8gp
jOqMyDrhDovHRGoX3axvKA3H7Wp+YZ3G3y4CI6KHcRS0UmxQi7Ebjg1SEbmj4V7Y2SfribeIoCdP
VKKl3w1H4QwIoMppdSA8wRP3ZRZkwdpIsjI66grnLpG0hcGuSAYzFz8aboAxyfenv19YEMvUEq+X
QQlNgUKvQlLo99MSOzXVYmTbzJs0XTujLThNOPaSC/cTGNgGAipJ6Ao4gqHGC9pfwtyoeOwNV6mH
7YF48IANnH8O/SZb+OAjUzjmjtbe8SzCvfvNlwzaPtRNRk7EKYuxPDc2qJ4zGE06E4iU5wxgRBnq
EeAi7+k4g0hEk3TYBNruNRnNbPfqBCH8Lm5eA3eUo9xVM0yZocF81ExBb+R43iBYuPCxp6cs0R3v
YSfH7L2cx8OOQFGiYSiGvZOT9mOOhxc1nDUwwIBJHrC97TBpL+mStbNQz7VQ6AI8ENoP7nd+D4X4
PAT3he6gbmoKPQnrpj+cfhXnfT9O/htQiHJMUaO5R8ORBBF4eBzFdtkaA2XayxJ2RH4/OJAN+7PW
zAxZgLbgDmE8JoMkfcjeI2WSN7Ei8SWO1lgzOBTZmyeX1wrZc94kUSLxx+njwrh2rt6r3iKJoKA+
sZPzKVFg2twtMuW7KuqW3vwOZ5iAhbhJJfGR3eQK0RHVVVFHVEc23pLmaa92MH8cufF2Zy36hTlv
eL73y+mtv+CPh9/waHaafwrttEDS1Fhe36gHLhJCYZeRaInZT4ySwjPHyAjt4vvC3o97L4V1RajD
9hL5yosWCfVOW1mLpnssDIcENMua9hm1J0boiDgbA0djoDsG3DEARER7IkluySdRv3RMCkpTEi9J
BVK3dBYBTjtXdFTSYbqbHqXH6Rl6hc5Qg4XaschRUQQL5SCGGVb0ecyslrJ4ZrQUi1aPh+2rsLl3
ubIB1y0V4uX4koVLYCE4uGWhC9N/hQWQFp1kijIZTcbkaLa0wt5rt+etzczPyc7L8Lm5A9o6a7mo
ts7xhEi4zpCk7ZfFKMNnUAYi8Z8mhrA7M7BBa2F9FhOKwiC9dtZviUmq4/W8HojpkAmotjMSzYl1
BhYvUmDBPeh5n8BRg4/NKhIwuro8Lt9IJFZFpxaeUjbMBo38Mx8/Qj+abub+Mq3jnK9zP37n3DVP
eD/j6CGI4+PIoJrQLYAiFAiq4Bf2aG+C9FPCVYEK2gYtXFB3VAAigFUrp9vi67hOnCtRCIeR71ly
nkwR/ioBpiJTyBC09+eaL0OjddmFhsul6RtTFVQUdnqijQEv658ufbtvj89S8QGxh/8H+dqOr718
4y94oVW6J3UXtLNqOvsnQ0L0S0KrSNWNvx1+4n97efQ9Ui2sJjH8GGnkf0v20zLUyB+SneyizxIL
w3HzNPwhdml0Y+QQ1h/Ap8Tq8WnAqw7hRsR78HkI2ybic7ce20ZoK7CfAzwh1VjnxLpWhtfqEEbc
y8jfhnTR+LwfnwsQx/gfwHbRjBdei7EO14PkkofJh7AWv3fAj+Aq7aNn6VUul9vBvc2n8qv5N/gP
hSPClH6XfspQYHhYXCQ+bCw13mF8CbX0SFRU1K6o98zfM/8xen30XdFPRE9ZiGWb5S7LzySbJqE8
4sXFWUl4ZmnQWuAa8jt0X0Yck2wKrL4ux+7rMgViwVIYpthyOAJzxE7GIjBPbOShCCzg3n86AutI
LDkZgfVkB/lhBDYQGyyNwCKJhrYIbMIx9F7/h3I+7I7AZjIMz0XgaLKEpjAV40UsnaWdERhIKpcU
gTHR40ojMEeWcGoE5kkW97kILJAU7ukIrCMZ3PcisJ68z70bgQ0ki/9VBBZJisBHYBMpFZQIHEXW
CO0R2EzeFk5G4GiyS/dQ1bB/++jghoGAktWbrRQVFCxUmvv7lLqeQK6yfKg3X1m6ebOiEYwpo/1j
/aPb+vvylRXLl9U0L21d3rBKGRxTepTAaE9f/5ae0U3K8Pqb268YXNc/2hMYHB5SWnqGxpYNb+5b
OtbbP9TXP6rkKZ+oVVj1p+FW94+OMURhfsHC/AU3KBhB3ica/RcDwllsGBwL9I8icnBIactvyVca
ewL9QwGlZ6hPab3esGH9+sHefg3Z2z8a6EHi4cAADnvj1tHBsb7BXtbbWP712VQNj/qHI8MK9G/r
V1b2BAL9Y8NDA4GAf5Hbffvtt+f3RIh7kTa/d3iL+7PqAtv9/X39Y4MbhnD2+QOBLZtX4ICGxnDg
W7UecTRzJegdHsJF2hymyVXG+vsVxn4M+a/v78Oh+UeHN/b3BvKHRze4bx/cNOgO8xsc2uC+wYZx
ifTz32uNtm+Y+Ml2MkoGyQYyQAJojLNIL8nGZxGa4QKyEKFm0k/68FlHepAiF6HlZAip8hFaSjbj
V5nDYUwr9eOzH5/btLaMcgW2WoYZezO2aUW4gbA38IMafQ9eAaTuQdp+sgWfo2QT4obJ+s/sfwW2
X6f1w2oGkX4Ia1uwNIR8l2F5M7ZcinAvUg1p3EeRIk8bz2e1Va63/t+lW63RjF2nKMTxMfnlkwWf
ymOWQ95/0dN/T0LhtdigcQlovMOUgxrvNqRo0agatZZMRgGttyGNqvVTemzAHtdjeybRG5S9Gu8A
lsOchxEeiEh7I9mq6ccYUrJ2s3Mbw57/dm2YTo6iVg5/QlpsdNu0Pldq+ICmY6xuQCv5ySL0Rm5y
u/bNR5qbOfdG+OZr0Bak/D9tF8Ad49fk2K+t9wakDa99vsZzC67mioiEhrR9wCS0dc4cw7L5ezro
1Z7hnbT5Jj5sZdmTtZ0d/Vhk/Ou1fsJS8+N9GOXer0k7X8Nu0OY4iGs4iNDc8bEV2xDBfXI0s2O5
eT7/L/vmwsHFTDgT+pvPGdKIKTMQt3Y/DrxaC+en4cw0SNMw/DGoH8OeDw5/cOwD7t+vFtvdV49e
pb4r4L7iuzJ85eiVi1eE319S7L+7tNj+m6lM+79NLbZfXPzrtrcXc22/noTUExV291ITpLI/mOBd
wUvFi5s5C6lqVmKK91fcjJ28Bf/KV9jf+FmK/ec/y7B3v3749bOvc+wRRGDqdRb+vvB64jwvPk++
bjR7LZMgqxY4850Mu/qt7KVe9Vtpmd5JcKjOlxbbySRMnjLaySkgp5RT6qnuU/5TAnscPnX+1NVT
wiQoqrkO6V7sfpEee/H8i9rZbPSLpmiv5YTvBJ3gwmNOJJV4NeDFkUN4Bxx5opqVke21H3cfrzx+
9DhvOQ7q8WjZS57zP7fnOW7quavP0a8/W2x/tjHDfhqSIQmnj8NJegksXwPLM/BtiIdYUkHsEKfu
b6ywP/l4pv1LeD2B157H4Yg3y370seOP0Ue9xXbLw/aH6RcOZ9gfejDDfugBk/3zD2TYLQftB6nv
4PDB3QdnDvLqwdh4r+UBUB8wWbyWA/YD9N57LHbfPVByl/cuug0HsRWvAF5jeGX7IdkPnB/e98Mv
/L/30wE/dPiBpUYBPwp1eKjOPuQtsidBQluiJ6FN7+HadLg6Pdi221dk9+FzbWedfY03035b5x32
Tm+hPbYopk0Aro0v4tqGObBwlRz1tYDakpXrVVtS0/AWm+BtbsqyNzWk2BvxSmzIbqAdDYMNdBJi
1Gxvun25N9Fe53XYa3HSH3pRCCAXxbVZwdImFVnaKJA2IDP2SbCeSBbxIamL8Smx90dSMvt7pz+Z
t1sqLT7LbgtvsbgtDZZhyyHLRcuMRR/GXrGwPN1HYI8MAkzC4YnWFperflI/01wf1DfeFoT7gukt
7K42dQZ19wVJW+dt7RMAn++45+BBsmxefbCopT3YPa+jPtiHgMqAPQhI8yZksqxjLDAW2DoWCCe1
LghDZBYxNraVYbUDzlkSDT02FggESLjJmGuMuMZcga1aC0CQjEVajzFyxi3yA3bH8lZXQGPFCMcC
jMbFoEhnREMyNtoHexhLwL3+n8Ruw5EKZW5kc3RyZWFtCmVuZG9iagoKMTU5MSAwIG9iago5NjY1
CmVuZG9iagoKMTU5MiAwIG9iago8PC9UeXBlL0ZvbnREZXNjcmlwdG9yL0ZvbnROYW1lL0ZBQUFB
QStMaWJlcmF0aW9uU2Fucy1Cb2xkCi9GbGFncyA0Ci9Gb250QkJveFstMTg0IC0zMDMgMTA2MiAx
MDM0XS9JdGFsaWNBbmdsZSAwCi9Bc2NlbnQgOTA1Ci9EZXNjZW50IC0yMTEKL0NhcEhlaWdodCAx
MDMzCi9TdGVtViA4MAovRm9udEZpbGUyIDE1OTAgMCBSCj4+CmVuZG9iagoKMTU5MyAwIG9iago8
PC9MZW5ndGggNDAyL0ZpbHRlci9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nF2STW6DMBBG95zCy3QR
gR0DjYSQEggSi/6otAcgMKRIxSBDFty+nhnaSl2AnvHM8DCfn5V5afrFf7VjU8Eiut60FubxbhsQ
V7j1xpNKtH2zbCu6N0M9eb7rrdZ5gaE03Zgknv/m9ubFrmJ3ascrPHj+i23B9uYmdh9Z5dbVfZq+
YACziMBLU9FC5+Y81dNzPYBPXfuyddv9su5dy1/B+zqBULSWrNKMLcxT3YCtzQ28JAhSkRRF6oFp
/+3preXaNZ+1daXSlQZB+Jg6VsRRiHxgviBrYhUgh8R5hhxxjUKOiWPiR35ONUfmI/KJ+YB8Zj4j
Z8SaHHKeQ/UXZnpvwRw5lgEzzpHsHxXI7K8l8uaPMyX7xxqZ/WP8Rrn558jsr6lm86c57K+pnv3j
E/LmTz7sH8XI7K/xHCT7a/xGyf4Rzlfsf8A5iv0V+iv2j9FZsb/C81fsf6Fe9td4tor9Q3RQ7F/Q
c/YPaf7mf6QwbH8dY4G5/YmbaO7WuqhRuCljmK7ewG/+p3HCLrq+AbHNy4MKZW5kc3RyZWFtCmVu
ZG9iagoKMTU5NCAwIG9iago8PC9UeXBlL0ZvbnQvU3VidHlwZS9UcnVlVHlwZS9CYXNlRm9udC9G
QUFBQUErTGliZXJhdGlvblNhbnMtQm9sZAovRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDQxCi9XaWR0
aHNbMCA2NjYgNTU2IDYxMCAyNzcgNzIyIDYxMCAzODkgMjc3IDI3NyA1NTYgNTU2IDcyMiA1NTYg
NjEwIDU1Ngo1NTYgNjEwIDcyMiA2MTAgMzMzIDYxMCA4ODkgNzIyIDU1NiA2NjYgNTAwIDMzMyA2
MTAgNzIyIDcyMiA2MTAKMzMzIDI3NyA1NTYgMjc3IDU1NiA2MTAgNjY2IDYxMCA2MTAgMjc3IF0K
L0ZvbnREZXNjcmlwdG9yIDE1OTIgMCBSCi9Ub1VuaWNvZGUgMTU5MyAwIFIKPj4KZW5kb2JqCgox
NTk1IDAgb2JqCjw8L0xlbmd0aCAxNTk2IDAgUi9GaWx0ZXIvRmxhdGVEZWNvZGUvTGVuZ3RoMSAy
Mjg5Nj4+CnN0cmVhbQp4nN28CXhT17UovNc+50g6OhptWZYt2ZIsD5ps2ZJHDLZAtpFjwAM22IAn
sA1msrENCSEpJiEQTOZSkpQ00HTMVERCGpLcW2ibNk2b3NA2zW2avsJtczvchpLXm6S3Dbbf2kcS
GELa+/73f9/7vifrSHtYe+211957DXsveWJs+yDRkEnCkfC6Lf2j/tpQLiHkNUIgZd2OCcepjj+V
Yfo8IdQ/NLp+izv0zp8I4TWEKIX1m3cO/eiW5s8SIpkIqf1gw2D/wE/grwjfpEIc5Ruw4N2ZB5WY
D2M+d8OWiZsyDVu+h/kBzP9g88i6/lsnOm4hZIkf80Nb+m8a/aLyP3nMH8K8Y2v/lsGaN4qtmD9J
iOqW0ZHxicNgnyGko5jVj44Njp6qfXwC8y1IkwPLAP/YC+kDBctTjhcUSpWoljRand5gTEk1pZnT
LRmZVltWtsOZ48rNyy9we7w+f2FRoLiE/L/2El4TfkVuFfaQNLJT/rzqxc8jJnIjIbPvsdyVz5mV
//9SoYp/nST/TI6TY+QtcoJ8gXydHCb7yZ3kM1jy1BV6wUFOk++SJzHzT+QIuftTxrUHUsgLiG2M
PEMeJ4fIw7iGPw1uI7mffAN7X02WkAkyAG/DHiw7hb1+jkzBIPkIVJADIXKB/AF7/grS9A55g7yK
6Sriuxof/Bu8Sh5A2jfh5/P4eYSV0j+TKfoA2Urf4vZgHwdwRD1Y/K9yky/Daszdhj2zVw8ZJCPX
EHknjvIr5OYrI5j5d2HP7H8S7aWvkdvl2sNkmGyb0+Ixuoh9cXYcOyHPyWV7kpXKKLeRfpPSadyZ
OPL1+PTD20jl3dxC8nVYDvXwIPkN2UnuohuR0r3ks6ROMMLj4fquzo725W2tLc3Lli5puqExurih
vi6yaGG4tmbB/Op5VZUV5WUlxYGiQr+7ID8v15XjtFtMRoNep5XUokqpEHiOAvHXuxr6HLH8vhif
74pGC1ne1Y8F/XMK+mIOLGq4Gibm6JPBHFdDhhFy6BrIcBwyfBkSDI75ZH6h31HvcsRer3M5TsGq
1k5M313n6nLELsjppXKaz5czWsw4ndjCUW/ZUOeIQZ+jPtawY8NUfV8d4jshqSOuyKC60E9OqCVM
SpiKuV2jJ8BdA3KCuuvnnaBEpWXdxri8+v6BWEtrZ32d1ensKvQ3xnSuOrmKRGSUMUUkppRROoYZ
6eSg44T/zNRdpwxkbZ9PM+Aa6F/TGeP6se0UVz81tT9m9MU8rrqY5+Z3LTjywZjfVVcf8zGsTW2X
+2m60iXEhDyDyzH1IcHhuC68d3VJf6JEkWf4kLBkA7J3aqrB5WiY6pvqPzU7udblMLimTmg0U6P1
yGHS0omtTs2+eNAaa7irK2bo2wDzEoNtaGuKpbau7ozRvAbHhn4swXety1lpdRq7kjAtn1ZNkBHI
DuSp08kGfvBUmKzFTGyytTOed5C11mdIOODritE+VnMmWZPWwWomkzWXm/e5cDablndOxfi8xgFX
PfL4YH9sci2up41sKlyGmO4jq9M1lWJ0VAW6ZFgHUtU4MOyICfnIFmw1twGuFNZkyiBndB/Fvy5Y
sYN8Y4qjyoVoGJ56V31f4r1jgwUROAr9sagvPvXtnbFwHSbC/Yk5qj9RHMAW/X04RcN18vTFAq7R
mMm16PJ8MrLqh5d3yk0SzWKmSIz0rUu0igXq61jPjvqpvro4CQyXq7XzBRKaPX+i1GF9NkRKSVcd
AzZHcF3l1091DgzF7H3WAdxpQ45OqzMW7sIJ7nJ1DnaxhYYc8pzH7pxyjzEaae9sWu5qal3VWZkg
JF7B0PF59degcXVa42hwycVUeSpHJ7VyXQhowAJHAyZci+bjZ0yZp8LHgAyXS9lSXTTf0QlWkoRG
MmIeR/1gXQKO5a9CKrDlFIkmsSlYFvFEolZnlzP+KvRTrHYkOsYWKsbUaLKKy0NJgGUU0chFjJcW
tuYdna5BV5drgyMWbulkY2PskbmcYIbM88RctV+Vm8MsZBNxYnUyw5gZa/BZ5zI3tljOX85Gr6lu
TFY7plSupuVTDLkrgZAg5Y0xwpZwuNJolXc/28+uhn7cxLij5f08dSIcZnt5A9u2U67GgSnX8s75
MjRKkFutN7O+UkgTNLUvKvSjMFt0wgV3tp4Iw53LV3W+YEAz6872zmco0Ejfoq4TuVjX+YKDkLBc
SlkpK2QZB8swTG2YUcnw1hfChEzKtbxcIOfXnQIil6mSZUDWnaLxMkOyjGIZHy8Ly2XshbNk2YA8
Rvld7xhg83NL14apvi62xokZOYJviIGrBrnjqjkBVKGJqV2Di2KSaxErr2XltfFyBStX4soAMxT6
b54y1Ls+tBTK6pzU4ceA0IFWsZIUnQASmP+Mkk+9EDyhEH45/xmOYpKc4FixwIqfUSpMl+Y/A6w8
ZHQa85xGZx11zOTCQzMbhI6/PVnHv86MB7R7iPAh2mEGcmd4fhMaJLRO366nDcxwbdOL+qUGYjIY
CEAbJ3I9FEyUgiRaRZ+4SeRFkRrwBQZDCj01+344I0CbaS8dQd4a9HqOcs1ir0hFtNxJbSi0DZ9A
CALdlcYQfqRULQh077cYfPt9Lye+Soqhe9s2pwtCqdmQbk4vr6iBCgjRr3x/5iff+b5SIyiEVHNq
IJPWnRX2XHIUdrk8BXkeV1eAO8945MQBfSi8JvNo6rlJ8Sz2fWr2TLgjEIw2i2AXA+JR8bh4WpwV
FbQBAO1waFM28YKJFwQDPn08TPJnecqH1eYorxD0Sj6NLAfS2dX5vgjnRTgrwhkRYiIcE2FMfvlI
7YXaUC0Oy7dtLBQI+Obv9/kMLxshZAlc6N7m24ZjqggpITXEgT3nZM7Msp+c/KngeOqpv53n5338
fXkOcpD4i/wypH88fMM+BawU4bgTep0jTuoMu31RZzjVjB9qHYoFl1klqkRoE5vUkkktSQZ81Ixc
tUoy4xS12AwGvVpMQ5KRtFBt7YWU9Cok7gpp+8/IxPl8SHBJcXc3FIErR0eVnCu1hqsI6ThlqIYL
BbNpOv0XYyAzPLQ0KG09+ZhSI6m5JzhRLfK8fcGipkIp02pRck8IqhvGDtRMv8bVuBoWVZvN82uq
TDWrq7PQzaE4tu7Z9+ik4CdZ5Fi4XXCI+mhWltGsVduNAWOzsdd43HjaqDDG7GfsVDKkmvV6M06J
Wq82NynVJqVSjW9JvVsJdiUozZmp6latkjNodTqtPrOFM8dHiRMQMIZSqgwXgik41u5tuM5Iw/PZ
brVBqcxW+wzzkQHzfYbofsHgM5Bv7xdkNpwBS6A70G0MGVOqAiXFvjwdMiK/rLS8ItVVEcJJCyld
nA7STOZQsLwCzuR1Fnf0q4XP3Jq5e/imwA8yf3Arr/IWZxZm8aJ+qN3RmEHvvf3Pf759epdpntFb
pFTKc7ti9gJ/A3o1BZAfNq/J2pxFa7KXZdOAvlbfrN+tv1cvCKdmz4cfkoxRIkAuLaV1dILupUJG
DuRrdmj2abgMDZB83I6QUQA8vhFrLhAeMiWdHvdok6Q1SZIW39paCc5JUCs1S29I5yReCutTopLk
IRzu3qbcAlNubgG+C2ZzQZ9rzw3kcrnhbFc0N5xijubm6jM5BycRZYvDnJam02oLiD7O3mAwwNaR
saoqvpR83WMyf93uAkNurvsa/iYXv697WxJMa5Ck64Ox9Xfl5SP4AUmW19AK3DS4HF2lmHTpuOTk
wNHPqNTtveFtaxYadpz8qlIrKeEJhSjyXE505XDk0Oe0qt38vOldBa3ZzZ2h7ttbac+lp3OaIsVK
wV9VbQosKbM9MJW7Kp/ejTKjaPY9vhD3XpgsJ/eHu/kSU0luCSelg5RiTfGlcJsW71p8cDG3K/9g
PnU3eDwm20JFqEHvAIdjXsN9CphVAFGAQtHRuNgUVqdFTYtardZlYpF+1A0BN7jdrpYiQ3jBshZO
b8b9eCEYrL2A0q8b15yxKnABOXKBScPubaz0QgryGEtLipETgMy4siTl8RfRstIavpbWQFlpvrxp
C+JbNc2kUM5ZqkqdkGbKpjIPeXtWxNHdwYPC1bR69IZl21vcgdYtu/bW/l4pKDf2lHREQoZnUhdt
PrK5a6qnRFLdICgKvHq7bubSK+4Vy5sqnWn+hf6Vt60pExVKcekSvUtX1l5tDy7fXNW6o70sWwNc
bkaRafVKfUEklFWz/gaPf9nmhfaVBTP7zEWmvFyeg8ew71RHxbIKx4JA1vzO9T62L67I6tFw0woO
/V3oE2BSOCvQZgHsQkA4KhwXTguzgoLi8iVAgWsiYMJtwLRRH4FJcpbQXkTFpB9+cSQh9IK1TBhv
GwvGRV5ywzPVIgu8EITACY6TM+eE1/5Wii2F2fe4t/gmUk2eCBfx80zzBuZxpVUwUAmj5dBeBpLT
6vQ5OV/WpizUbKgBbysDEk0LRu1pTL1ERE00zZ5mV3saifeYl/Z5z3vf93Je7wKpcUR9XE0D6lp1
s5pTh/WmqDotKBS25jq893mp15trMLQKkiSwpREwMDnWbWCLANj2kRfGNpbFgRguGORlAcnNAiYd
ZSuioCKbYxNdVorpIg4XCJWFtzIu1tk6SM/muIN1tz4z0n3XYCT1s/pAfW9teNNSX379ugULt3eV
Ld75xVVtdw4t0nzJfPP6su4GT359//yq7T3zYUHH7naPY8Hq6kp3rc+cVhQNuat8ealpwYbecNt4
1Glb0F+f0dxgyJ3n8VR4nakpxYsH4/ZKPe6rAtxXeSRE9oStE/xenu4qOVhCd/kO+qjXAnkZoMgC
TaPEWFiOuk2SHA1veKDYA82eNzznPBzxgMdTFuIawmYwM+1nliQFaQmFFEUtVoOixWhmov8C2zC4
lZBlYxeC8b00hpvJl9xFslDRgTIhU9KTGymlBuJbR95GwPgnbylcDxzcsa9yTVNV6lRGdNPhDQOP
bqnqePD1m7Y+trmMU4iKo5TmLlpTWbyxvyUd/mKrst1xT1bJwtzawq5oYdWWRwcfBfrNdfM23L8y
DTdjxpTJZVzUX5uddcPutSSp73kzv5QUkPtfIDko/TslfTRDDxJv5Wl+AbQVzFnpAQLN5Cg5h+ub
eDI1Wo0W2rRNOr1Jp9cb8NGx5a/TQL4+U2DC25Bm0mt1BZd3Q3oQrYBPMwJQVV4uYsZXt8+ZekW4
JCwCxiDZJDDLYoa+/o/FL73jsvCdfoxbfh3hOz0eXyf8zEq+H3kxj7SR34RvuqFqVdXGKi7PBRvN
sFcJVYsaF9EdZdBRMFhwRwHnzQG1LdPmtXENqbqycmgrb9J5TDqdB9+egA5GdWd0tFkHOl17cTtM
toOiMSy0CH0CJwjVi4uzIZuto2xsypHigD61wcNlNLXkGhRsERpxHysUYk1LRprYQqxJ24ItMJ/M
wdpQnI1jsqzu3ibvz7iSW+72GHS65Z9UcoYzZ2TG4ovI2za+ZUFei2jdzpHs5RWZ8opMDyXsMFR+
kFi1c9anDuDu7qneCs2XjNsHolubywwCGsYKaqtt2xxt/kxnQNIEL/5L1VBrtZYKep2GPknVOr3q
i2k33Hx85tLwUPnIumazSpW7sDNUM9gy3yxpZlbmLequ9C5pzVty4/LM+Zl2u3vJPFdF//6Wki21
IABvLVtSnFFT4eZE3/zGgkvf2/T4jhoue+fuooFHR71djsi6RY68rs9tq94cn9Ms3PulKNs9pJY8
HB5aFxwP3h7kvNIqaaPEpXF5HBUWn1kAxQugecEbC84t4MgCWLBgYWBh7cLmhVxuicGcnY1moLkp
PcuUnpVlwCc9nJ0TTWdLPT07i6Sb0wy2klyelLdozd4WYrgyV2yWZGv8qsWOth7aeZa4bEAdwOQo
JPSmbFgUuJihgXxnJgcoFWzBpzNuKy4r0nLcFvT3Lbe0+0WNtfOWY+t2fd7BCTqtSJ/m1FqD8sOT
ywUteih8blPHWlV4fG1TempObVd5zUBrjTm9h18WXHN7a/DG2nt+dNuCDT2Z1aV5PLIzWsBfmOnK
Kk01pwTqCtMO5nc/etPa5vC6SE7e6ocndnbsukWWGbNvosxQoB0pcrqw9wAFM4V8ZblySLlPyXMq
FahUCoqWs0K5BKgJAAlHo0RC4ToZdqH7s1KCcqlBonkSKCSzRAukxdJ66WvSK9LvpL9ISiaEn03L
iMrCWIdWY6UESnxzIHKsaFZnjXLAoa41WqOocLVRoqAtgh6ULSTtLxI8L8HXJHhQghslWCHBYgkq
JCiQIF1GgwD/Ogdmvwy2XoKVMphbApUEPzwlwUMS3CTBBgksEvyXBD+X4AcSnEq2WS/jRVy/k+AV
GddiuQ/s4OE3JfiuBCcl+JIEn5XgdgnGJVgnwXIJIhKUSuCSwCQBL8EHErwrwVz4QzL8xCfgSQo+
ErwvwXkp/GM4K8EZCWISHJPgAQkmJRiTYK0EbRIskqBYAocE8RbDf2ZN4CcSfEeCZyR4TO7jNrmP
vmSDoAQ5Mk1UpunXMk0vyzR9We4gDo8dtEtQl4RPuRr+TBL/NfCL5DHkJvFX/TnZAAlKDIH2jo31
Mge2p/ua17ZrXmPXvK7ToPfvtei5Bu8nkcxpipsYDeSgvH/ZBmZwzFFlnjYTt/vlc4K47iop7sVa
VwU4K5xKcCr9Vi5r+mtWrtJ66d/o16wHuPyDBy794mBc50bQxvsD2iMBMhl2HCiAgzmwnYNh8YBI
89PL06lgS7OhnMzUUUWjwJZ8GepkQbBHiQOIo89x3vG+g3c4SgobDGG1PmrIa8kcYU5ApoGplxhT
MKRFYhYJUhm345K2/baENRJXAGjUX2WGlFfUCAmDjUvIdzTj4WeVqxrKUx+xtY7cs3L9g/0BpWiu
71g3vKWsa39PmXCCs4VuKKnfurImTVSvv9k5v6O0pOnG9sKakYdWuXsKR768fYnPWNS+fXFG4Q1l
2e5V9w8Xb8zffVfC9khD2ewiXw5rVDaLjYpGEEVQMi90JbrnkgbaNE1anQmdbOZoB7TQrL1Xe1R7
TstrtXkZJCuTrMyCzCxoy2qyWU02m9VmsOInE87WLBvJ0EkuXtBmWjlNmqlFuCyb00OycL5shsjq
Ec2zqv3CmetaIz6mLq/YHa6rLBOQXUPYfcXmeG6OWfIsmvdNCr1OTbk9c0yOq8wS4bWZzzgXVbiV
CXv1j/zjyJcCsoB8IxzekbEvg27Mgw79oH67nqvyN/qpVwKvGtRcJuflOEMgx5INbdlNOemmnJx0
fKfrc+w5gZyRnNM5Qg5bQbnpWdGcnNrqxQIzOgR7dpolneQEDBwpbdEa3C3EfB21xVaMb5tsU9S4
0w05OTX/wKaAbiG5jhirZOMh6TfLxkVBQnklzAcFW133z1VjRxwCVYkqaDv5oUqrVsBTklrIbWqf
q8MWDLYuSE/vSWMqrPCWhApTF4WKRPjSx6kF0fk+0dRZOkeBLVx7lQID0opMzhH2oBW78rmYcEag
bJM9m++NypvNbXFFawUwCMCcPa7nivlLwmiVEXT+OJKyeUvcvWN8Cl5eKcxq9aEwYG5dK6ScnvmT
sOdvu1mf9djnf+CcZpHbXiDq2Z+HbxCN0QwObDZ6XH9aT/Vh3ON6K4FMUHA0E6ANmqjSRJVKA1Xy
Sv48BcqmjiozFWktGkK4VqtCT+Hy4VqICauEu8bEVZIknCBD/HzJ1y2fLXUTH+ACZsdK8VlKbHY9
hJTwzAsnX3go86F0Tf7i0RWOSEammJ5pEfNDTpMq9aeZ3POXGrnnb7+5emiJT6E4QJFBZu+Cgv7b
47YWyjU+iPZzBvpZfeGINQAT3r1easqBXVkH0V21QCAFJNROApDFxZnQnPlG5rlMjmRCZmZZX9lo
GS1YrGUSTWtvSTP4W1Rxp4qtxG2Jg4krhzSQMJdwvaHpGj96UF6xSy+bSjXABzW67E33xQb6HhlZ
AKIma+iuE8Mbv7il8sWchWuqyjb1t1i1Un5kdXn5cE+zVSfRJ0uHq/e+OFFWve1LGwID5Z95bkdl
5eYj65ZurndkLLlromKdt20i6rQsvWf7/IGED6VownEv5HpfIL7Z88/iVLqYDJuVdNEqI3DhhdC2
cI4fRciiYHEJtJU0BUOmYChkwEcMwl+DsC8Iw8GdQRoMRg5HYDgC7RHIjzREqBiBFZF9kW9G3or8
NvJRRCFGPBH61wj8NgIPR16N0B0R6IpAeQRMkdwIVURAadPp42dyBqPJYDQa2MPkoqG6xWvIs+nC
ECo22pREbMkxmPUlQb1hoSEYd9X+EIG3I/D9CHwzAl+NwKEI7IsA9rBWpqcuAsEI5EbAHAEagQ8i
8G4E3orAyxE4GYEvR+CBCOyV4cOzA4xsaJApi7dA0n54TRPsAgd7WwQmIjCU7AIbOCKQEgE+Ah8l
4b+f7OJwsot4B3WJoV8FHKcEKR+bA1bK+MnoQMjPfyTz700Z71dl0CEZrlwGiuN6WebC5f6QBdAi
Y8LhvC939bLc1V6Z/PZklRSBjR9Hwn1wPgJnZZiYDDYpgw3IYEjvebn7OIaDcqEvggYCY2q81TGZ
/bvkqmLkt0OunndRrj8dgbj90nut+XJ9G2nbFfPo042o/32T66qX7OYHQoHEcb8xJG/guQ4Q9Fxp
KqOMe0NzNa4xBaosgas0cC/KLXYiwH3qiUD6Vfr5Chw9dAsvKoUlK6rWt83X7Xh27jFBdnhJd/XO
e3SOuuWD4f33o+p+5ipgerOpuNinb2oOdd/ROv3la48PbtwWbK91JTV6Eja4Zm/r9BevnK+sQLmv
gewXiIiyAdWMyGSDJcsRvU88JsZEbiWFlQq0mx+TnpE4dHvOPpsfYO7P2fBDCCRIadJ2iROkQfal
UOerG9Qr1EPqfWqFUoniVIPyhKrR4eIUolJcpqYmNeYItJEmjWTSSCBxao3BrgGNRm04p76onlVz
AfVR9XH1aTWvVutqdc26XlTRlFMo5MP5K3dQ8bsnim4NL6r1CoFwPK+5fLTJjuLx7xOnObJ/eyta
UJVsXiuNqJTiNexiTeV7eb/qjCphNBCmLkG2q50Qt6QqICQC/fHMB1+PMsvp2ddg+FfTf4IW+OiW
mX3Ca5c2fQvqZwLTDyJvawkRfoB6PJV8HPYpU9an0BfJD8kvUK9b7O7oF8jTqJMsOmO0EWGLUXWf
VL+sflPNqRn70xDiAfVjajqo3q6+Q80xnlI147nIlDy0qSX1KVCF0YtCF5ZIRmgzsnxeSqopJSU1
xZCamtqcgo6ZIYUeTzmdQkdTJlNoSkoasHs8BpmjEk0qUTTgowqjMatKMYLayIlcqkSIUpWwIFBV
B4NojwZCV3ExboPiBpljn165mGTWqC/x1+0EVIRx5oF8SZkaAu6d6QsKpUKveu4JML5JzZg2qJ97
auYi/2JRsae74FKrsOfjhb5iz2CI+xIaKHFevoq8lMhnwtGVagAWFYj+YQ8RTQRHgQ8hWjTZOHmR
9MxdJDg4Xi1y6MeplEqFwHG8dNlCYkd+VQFmnISuXSbyGENzDRVA08mZ6pTXgZMNY3r3NOw+jYbU
BJ2deHHmT/Tu6XFhz/RTtJ1ZVWx/DcNZegsdxTnPC5sDuO7RTjKwOMeXjsIbQAOAk8fcHmDXa4g+
Ff2zYTt8YIezx44h3LzZ94SwcJhUk++FD2yaB7kVExV0tByqQ7C6EPblwEDORA4dsE3YaLUFqtNg
IhXqdO06WqdqV9Gd3AGObuIgb01ubtVqQ2DVaOZkJs3MFEYMcNYAdkPAUGu418AbDAsUa9D0NkaF
vQFYEYDGANxkgCYDBAzE22fPy7PzUi8JK1oUVKHABV0bZF6bfBCP2ydu8Fy+nkEzrvuC4UIwcTlz
1Uv4u8fw3Jxj+DQTTzfc9t07Ww0ntJU9Uz1rH9kyL6f93s2dRybqeo++vnXnc7uX6J/Wl7dvX7bh
3i63p31P58qvTC6d+aEpsPqOlXk1axZkW2sHm0LRqlC2pXrlLauGP9vj8zdvXRgqRsnoql0ZKl8c
9Fgs87r2rvuXX7G5GkI7qYyfR9zcvWGNzpJqi345FR52A8EN+WxGXpSw7WdKSYsOkR3kMOHas17O
ejPr3Sw+i+3Yt6x50fIs2Kc9rP2qlivVDmgntFwVBXBgBRSnmKNT8ATQalgN1JMPvoLqgiMFnM2h
NUQX2MBtAw6N6ixnVpMSTCg4bdCmVWqbzTaT2WzD/V7gLFjiJia3m+igzWA2NOl1Jr1et1L5NeXz
Sq7SGXV2Ou90ft0pVJqj5k7znWberHQ6lWZO7zYo3Gb3CveQmy9373Dvcx92826393EvHPZClbfR
2+Xd6eXNXvirF97y/tZLhzF/wMuVexu81MtcDnWaJfqAF/Z6IZ5Fs1H0VnmpUs8iGWb17FJU7y7I
sjlxJ7OjNk40h0VdVGnWGWwFnF6PNqVWl/ZNL3xZ7nSfF3Z4YcgLDV4o9cJHXnjBC4/LHcTL2+Wq
ci+IXnj9Ay/81gsvewHbfzXZftgLK+TGJi8oZABE8bCMYsILA0kUCJDvRSeAIcLh/cEL73jh+0lU
cToQj8cLKV7gvXDkbRnRkzKueDXiavMChL1Q7IVcucOPvXBSHssh1lt4d6I/rERa35VpRQxHvKxR
abIRYh/+oxd+5IWXktXYplquxjZnZZxIe50MTZK43kyiO+iFXTI1Di+KEBmi6rwX3vDC1SZW9zWG
0dX21eWq3usaTJ9mWV0HiawbQuyooqrK14MqNRBXEpU9TOr7khjkU51KtLQWoM+Hza+OYvlETEsv
0xyywpU/4wmf08UVQUF+gaxJwMWFUlE+JCJeUrMhmeT9P84D3HGQkmJw6WZeObvmGTdQSvV6rVWC
jm/7OU5v0GVpYVmMvmha5g6ECv35bfbpG/h5005NxYIqba4re1E6/d10RvlALqbrsui5uO84Mvue
gqAcZr7jWLjVFzwYpD6cu7y9eTQ3C3Za4GA6HDRDl3nYTKv1sEOCvWrYK8IKcUikpRSEVc2B3gAN
BMpyVulTAqiLwyZLNKWgV5ua2Uv4pCfZLcvQC1df9nfHD+Bx7DrKXz4Ko7npn/AlmUjlnrjvVw83
g6jOvufHM//5hcPnP9eE6bt+DLrDt7/24NpMUVW56cjava8fXpchKudt/rxwuOPzb06WDBd//i/f
6Fn5hV/cXtRXcvij472fv+GuH076VhZs+uKmsqa7X73Vuyofk+UyP2atMyvhI/lOOo3cHV62Vwsm
CxomgXRg0UJ6A7NH5Js39PJ0ev0bunO6izpOFy4ojOrC2bnR+FWcMozmj9LI6wx6vdTCWxMBRPGD
HlTMLETFx8Jv4jdHZjdOn878KVEUF1B9EjTXWAhRTn784qg8fr4DTudJ5/QrOfWZ6dqgqXq+8Xcn
L2gKFq1jwUV/rRCFO6mifQ3/lY/DNRuXFRIyO0tKZ9/jQ8KulHyC+s6oJHZQ47jTw1qOthvzObq/
c4UpnAe+sjLCHnmNHOC/S98WdskxVSPhZQrlXoGn3F6QWICYcDf5I/8xT0/zb/DneO5JHo7ysIs/
yNMRHlbz0MxDNQ92PsDjeuVr+eP8aZ7nw+aMKM+YxBMWTDLn8DjETn678ZMZL/HAKSc+VHQ97Zr+
qeu4i//uKXwxvTaKNv5a1GsGcji8cAgLDCjArxOy1nQlZI2KDo0+Kses9ckxa46UcEoLmpC8HLyW
TcMef7SYnfmg6KciE/YGvQh6jqKsT5PPe1AuBEI+Fl7gq+y+HMXm+2QYm6/bGd/R6CDhJoaPf/bN
lwY5NNSolO10amDyZdyeYXNtXW1aWu2iWjP9Dg4BByasQXtQSTaF64jCoKDnFXBaAZOKYwoaUPSi
ndLLj/C0GZlNeQVP2Rihh3ImynEGfNAWoxzhewWOQtLWrb0QlJk618yVz1oDsuBDWtFzYyfo3KoP
vvXBjC2b35fN//vHVv7fH3mEwOxdMyv59chnO3kzXL4awMtBPgErAfmsHBmebcleaicmu50YoC3F
ktJkNJiMRkO1pcmy2sJZLHYjY25Br3HEuNvI9RrBaFTZDS32PjtlH5N2zm53OpxhJ21WgYqp4tT0
zKheZVcFVJyKTYXKnm1h5WnptqjeYrdQiyE7xagnhjRNi2BNxN/hzHT7ZFmNYro7IaCrAtvGPi3O
MH526ozf/sX3VS3MnbIPK4ZGb+9w1mfbjT5j7/0N1refH1XoNCKoHQWeVGh7pfeR0Rqlch8nTN4c
4Iumn3C0L1+cYq5d3OSim2V58jryroRfJt+D/igsdWlghTQk7ZCYN/TzZ9FGcjMbayMu3NwC4B0m
R66Dy1l87bVofslQyY6SfSV8CadO36eGdDU6Sk2SxoRuKL41dikg1Uqn5ZAvhcTkkMSVpHPE25Jl
QDZp9GrJWj7nsDlUeyUgoFsWQWG3xiBJ4U+KIOjeP+fmVBbgTHUJ7L7UdJ3jgOSFaup1bk697JyZ
V0mKtmf/qtKIAnyNCgpeviydf+U0OvcT96amDT3UVVqTNdMlKAqi1V5RlVsYNLOj5jn3qdy/Xv/e
lM4eQTnxe3lPmciJ8A50Sw8oQKWwKNyKTsUGxU3of6dSTkBZoeXWMAcLHxbR1Ua0pEepQKtVIWoz
tB7tMLv+4HcpwaesVr6k5JRK8xH+Sf4lnuPDoXIUaChbAjxQOw/42WtKTdXKbpkS3bK4Sxb3yXAv
9jIuykt07tpEX+xKKm41dDs52TEToYgrYNcdTj6vaXrP+2e5lrchd2a3Sq9WAk3P9Vtm/kz18MjM
ADv65tWeZQt9PO9f2OyecaNcaUGBWUPeQRluDkv0cPxIHZAz8fizkmIW7Nji+67vnXdkmc/80kbk
mQ4tg83hyM1pU2l0ZQYIq+9LPZZKA6mQmmrVS5qMDHZV1HPlqkjLvFKt1KsPi7hT9ZqMXh2n1VwR
RCyQ4tqDqcTNT9w6gEQ8jouNOtUZjN9aOLn2iRdua2j+7E/3znRuPA3Gcrj9zLcK28YbZr4k7Ckf
uH/V1uM7F06/g05q9SNHFq6Zb0uM4Q55DFZyR7jrDhMcFGGvCqYsIFmsFp9ll+WgRdiVfjCdtisH
lHuVnLC6Ng1q05rTetNOp/FpaVlKa7qV6q3WHp0WVb52zhCtODB9rzYxsgtzTtsMvlvZtpF3TOLk
nMgxSMl4Ax3vcgLnTHiKPqC/P/TRibW1Y4+8+OOtMwSMZWD77P98srtxMjb0Z2FP8yO/e+iOX8Zu
7/QJez62dH7lwuGbvjFWweaoC3X5WrTdstB2WxNeOBHaG6L7C2GX+6Cb3pkDknW1dZOVW52xKYPy
ErSrB9SUR4Nt5N58yM8vKxnRp9jRYONSsse1GZ5x3B7xYXzSVEuGSSgoi0WUIxDRKBO4pG2Wf+XS
kv6y9d7hBZLRvfcrL/Usv2d9NVWoVdaWvtEdtzy3ZHJTu92ovXFzMiEcrhiL7ax9ePmPZv5ypGb+
Td/c5axddIP/np8+vK4q/W9/KN/z1hc77qs+8sjlRPzsAedV8QTOq5drCWdXu5vcq92cO5yWEZVH
7i7G6THZcm2lNs6WiGmwMSErSsYotaXYqIZpI605K7pCM6TZofm+htfItwYspEEDWgPWrNZu0u7S
/kjL3ybCroKDBXSld72X3qaGCXXilp2JcdrJbeCoilgIVVqAc+NWUGiW5bhNOTluLbTZcmzLUrVM
ChzJeTKHluc05KzI4XJzQJmTnkPFnBwrEzKKpR6vyePxegxer9fqgTc973roLs9BD63ztHuox+Pn
/aX+On+7n5/wH/Kf9HOSf7WfvuuHl/xwyA9N/k1+yvtNfpqaAW0ZSzOtpkyr1YBPJluomR43rwCN
ItWW6uasXI5G68UtreUy2cp9xw8/8sOTfjjih4PhWT/s8sOAH9r9UOeHXD/80Q8v++Gk3M9eP5T6
gccWT87JY/WX5fQEo4SVmPzwgYz2pSQunx8kPzyMFL8pN7gv2SDeT7EfHHIzXm4Z7zCOdDSJolQm
Jw5zGc/JT6BCAOKH4fdloJflirnt4y1jfqDH5KZ9fmjxQ1ju34CJPGyMbc/KMFg/4ofeZD1WVWHd
cT/M9RqTVmvv9R3Tf+h3zqm6NkYifqQfCqAhzLxRY6in+5rDSkge5l99kC+r7RRAtSKjvJ4rmnjJ
oaSJA000Z9FqYqF/FeUVSX/0inPKXfODDCfwF06qzWpUAZnmoHn6l7e8oNAplCqVqDKqTj4Bxvf+
mZMEhTIt1ejSUA+QmYvcsdBGf6i4NOTvy7u0hDuZ3VtYVlVZERhyX+oT9lwK6CMud7Hf7Vqczr3K
zkZn30b5fZidjYItLCkduYVRpQHtQmDHVpgBto31WJDBzh7z/FGzbDy+H16E6QM8bAJQU1HJqRSS
AqjIcyqBCGpMURYuxaHdihVNAjEJAlFBG5Yv4VQmjlNJBoGhCYUlkARJQP3K85xoaBZ3i/S0CGI4
Jz86Kk6KVC+y35OwH7gIoqid1QLH2lk4FhzFgZpX8oKoUqEJgHa5xEyA+Nl9KBAyyqZX3H9A7ssH
CSzGt7uycv/c6TLs17F7ZPmDHd8brhzi44RtI+hC+pwiuERmIOCb+93Mj6c/+Cm9Z+apV0AH7/x0
+uswSnumH6P/QR+cfpsWTK+ZdiV+86MqQt4W89Zn9xnAyQRlVl7UyY4CF6TZonudh9C15HY4od0J
4LDnRZuhFyg40AB/A86xlNEU3QHQBKthF3BVAD4AawD0bHaYlc5EqiQao58jXyHPod1xavb3z6Zn
s8PG34dnRW3U4cCpG3YA1+mARhZGgxYJ+jJOj3OJCCZRBIfsyTVbHCaLxYGzFfAEeoqJqbg44WX0
xL2Mr4rwoAj7RFgsrhRp1NPpoQ974CYPrPeA21PpoekeeMjydQuK6HTLYstKy3qLYBE9HtHCGYsN
K4qHimlxcVAMwoHg48EXgm8H/xAUguHCYHRXEKqCjcGuIOeRv4aDB4KvBv8aVCqCIIpGyDKy2AFj
ccApOlIcAc7IeSwELHrgDCmI7uMg/CEIbwfh1SC8EITHg/Aw6wMQ7abwLMMN8bvnuSCJ+iB0BaEp
CfPqXDxPBmEnu6xmEI0yhCcIGUFAon4bhO8H4XAQhoKwQi6PF74pl39Vrtolt21MtkLsDyMJfwzC
O0H4URBekjs4IhOyUyakW4auDkJhEKxyAxzYhavhcWAHk/CrZcIR3ifDS0lGIPzrMvzjDH/40UQP
w8mhVsskYQstFrI2SNM5uY/TQTgu03QwyZzLfQTm9HHxauB7ZeC5dMxeTfTRJMIRGUCPvKQX5T4R
ATtI/O/I8bFrAT8B/Y/ucq8H8ulHl7JGQF0gf8XPJXu6exiQb9tY/DDi8omk3PiTGuA65xSyN+JL
nE0yZbDNFz+qZOeT3NXnkwkVwMknlBxmmBEIIeEz5xYDx3HAUUtGWiBlpvvcPd9rFhSUQlaWxpqZ
IcHR55sozyAAsvxlGfC1V7ndqcVlFRnzw7ULSrfmX9qDWmBP7a7gvNL6SNaC+RXp3JZLnzVXVi/I
ijTUb9hZyiXungZn3xP+De3eQvL78D135MBBCW7Oh1s4OAhwK8BO7QEt3aiZ0lAWMU47smCVbaON
rkqHNGee8w4n5/VAm6eHFJoIokARai8sJGmrTOGU9KjJZCfQR86gdU0C9tXsNxu9ak6vDqipmp1r
qr29gcLewt2FXGFhbmavvjcN0tL0fG6vhwipCScnePk3HGhFz/kVB4sAxDJ8o79d5C40EFL0SX87
frUUdxcS4eL4lXf5FwvludfeKTH/upYFQApbK29/9J/HHvufR5tvfAcqv3376X3Lzd/WV66+rfPh
C48tLx95YmL1gzetzg7NrHbQjp8Ut9fkrnzkF3v/Cap/s7t8zU2LygPLa/OGX5o93P/Y9kXZhZUZ
M8eS9rYwnLijbBlVTCruU3AONagVaoXIjpuoVlCJPM9+kXh11JY+SlS9dgGahd3CceGiwB8XzglU
EES+Fzixl1xxnhivto19imMo//qlWw7vcirRHZafWjDyGdmXvtrJZWdf+nUvd3u2sOeRmfmfn0l7
JOm/3ok069E4/6fwjexHdHRX2sE0uiJlKIWutID821Yd6EhP/MJpUnef7piOG9BN6GgL+/UApw+n
ZKALK8Vd3tSwpIumpmZaKJeeft2rWB3ttVhIei/z+Hn95VA1pvLlH6gErzs6XzcLDWXj72aRrclf
KjEVLy+BuPvvTHhgIjCHOB/9Yd76nzOLt6EnTAc2PLWr/uc/Bc3M6HOnPndopmOQNk4/L+wJ9R1Y
8U+/zJr+Kj3/4IHRm6b/S45xnFnJD8q/O1lAfvxcHrum1zONLGGiwzZo227jrCxfrTVGrwp2DOTa
0q3QZm2yGUw2mwHfhjO2s7bzNm7EdtpGW2wQsNXaqM1WW8tsCdSftfEASFs0Gf1os1rTzfprox+P
1cLYJyIgfb74lWsyCtKAHV4vClJ3RkiENKDUIr5PhkGGZNs1Nfkri/9OIKSg02n4p4WczD+d7EDz
T1BcG81/3UjIjAUVfpXxvn0fl8PzmaVpGeZPi4QEsg3l109QfvnIF8Pu/S44qIMpLezkQBZag+47
3HTIBGXeem+Hl6tSNipp+hozY2adxRY1m31ragmcZmJqVP4XSaTQuSqA8yTLJ1uvsTm9N52mpxv5
Yl/Y1+Kb9J3xnfUpfL783qSIuio+ufvKjw99yd9FJB/2q6nLvzmcI3fgsuCpEJaV3PyFV/cc+h8P
RMsH7nvurZv2fGeqPfWkvqb/jpUPvPtoW+n6zw+tOnzrunxobbytv2rN0X/ddcv500c2V1f33bKo
IrCy3rPpn2emH+55dDxc0jpcmZQ3X8S9qyH/wmI2z4d342LMUHlUjaoDKj5LAxJ66SSLUEEqk+iV
SBwWe9PDYm80Em6UNgUnLolH5Ui4iDUGFocTQIFO1WrdpwdUyFE3ClHNsW2s4a/axunXCbqJi6jK
UCAecGPwqVi5YImvzzkxN+y6b26sDTcz4/4WO8o6fedd9OEPp1NoIX1sz/Q5Yc/00j00OD116UP5
/0aB8Y+73z1u7dXP/5DY4/+z6I2bH/9e8v/qsBsrtORfw72tYlor/g+BCFHWzCwjkcv/m2fgmv+y
FKTvkTphBTkBrxAnfYLk0CrSzY+TFfgUsTIeJwHT9fjk4MNjPcHvLJ7Mvol1kUR5PZa34sPgIgrE
g3U5wiuklj2Idxj7mIfpIUUVGYFXZq0IV4rlBxBulP8NOYX47uLHZ18XVsweQTwtCC+3RbguxFeL
5W8r7yYnsP3gnLp6/N7G8jgWP3mA/AnWwC/pU9wG7ht8J/9bYbNihWKFkqiOibliTN2n/rykke7U
HtbV6f5D36AfMthSvCnPpOakzk+9M/XPpvmmX6Q9Ys42l5sfSlel7824KTPftjdrftbh7Jzsd+0T
9r/FeUqCpB15vIbw+GkgAbIKebJU+S3chIzzNliR+E9fhPQlWrBPPebiaUoUZCSR5oiTbE+keWIm
hxNpgejIU4m0AmftxURaSW4mbyTSKmKCxkRaJDrcrPG0hDRsuvzfxopgfyKtRd4/l0jrSA3Nxd6B
FzF3hq5NpIFkczmJNCUaLpxIc2QRtySR5omfuyeRFoiNezGRVhAv91YirSQf8CSRVhE3/0EiLRKb
YEukJVIpVCbSGrJGGE2kteRX8t0rS+vILYonIiOjO8eG12+YcLjXeRzB4uIKR9vggCPaP+F3NG5d
V+RYuHmzQwYYd4wNjg+O7RgcKHIsaVxU37awvbF5mWN43NHvmBjrHxjc0j+2yTEydHX7JcNrB8f6
J4ZHtjqW928db5zo3zy8buH4usGtA4NjjkLHNfWOOMD1S1cMjo2zopKi4oqi0iswDKTwEw3/AWE4
mvXD4xODY1g4vNXRUbS8yNHSPzG4dcLRv3XA0X65YfPQ0PC6Qblw3eDYRD8Cj0xsQOI3bh8bHh8Y
Xsd6Gy+6PKbIyNjoSIKwicEdg46l/RMTg+MjWzdMTIzOCwRuvPHGov4E8DqELVo3siXw9+omdo4O
DgyOD6/fiuMv2jCxZfMSJGjrOBK+Xe4RqZnLx4aRrThZm+Mwfsf44KCDoR9H/EODA0ja6NjIxsF1
E0UjY+sDNw5vGg7E8Q1vXR+4goZhSfTzf9YaZeQIKtKdZIwMk/VkA5kgDuIm64gHv4OkGP8qMNVG
BlGGOkiU9COEH1ONZCtCFWFqIdmMf445GMbl3CB+D+L3Drktg1yCrRaResS2ECVJI2kmy7B0WIbv
x2cCofsRdpBswe8xsgnLRsjQ3+1/CbZfK/fDaoYRfivWLsfcVsTbiGX9SN0wwi7E/DqE3Cr3MIZQ
hTJNf6+94yoM/zuwK2S48ctQJUgr42URKb0uniSWwv9Gj/9nHIvPzXoZy4SMOw45LOPuQIjlMlSL
3JLxa0LubasM1X6dHpuxxyGZ3sE5kOtk3GwsccwjmN6Q4PxGlP1jMgUDcrvk2FD7Xmee2Bodw1U6
cg3HGHU75D6XyuUT8ppjdRvk3CiZh1oqQG6U/4oQ5mrM6xJ4i+TUFoT8/9puAnfQqMzHQXnO1yNs
fP6LZJxbcD6XJDi0Vd4XjEPb54wxzptPW48N8nd8Z22+Cg+bWfbN2iapH0/QPyT3E+faKH6OIN8H
ZW4XyaXr5TEO4xwOY2oufWzG1ifKrqUmScvV4/m/2Tc6AbMf4FOAVsR1XqdJCyhRwQfkz+PAhz8H
Z6fh9DQYpmHkYwh/DB+2uO1/bnDb32/w2XdfPHqRBi6OXNx98fjFNy4K5D3De+H3Wt4bfW/yPYX4
7+9m23/z6wa7/tcQ/nWD2f5v5xvsp8+/cf7ceS58PlTecL7BYv8fC851/GoB13EOuI5fcrN2/c9A
/7PZn9HZn8HRN+GnP5lvP/0d+HZLvr3vW6PfmvwWFz7Vd2r0lPwzY/FUSrBB/3zt8/QUpDxz0mU/
BfrwoueCdv2ztc9efJYT+2KjMXpf7FgsFuMmn77vaXrs6djTdPeTcOyJ2BM08PjI41T/ePPjRx8/
9zgvHTvqs4ePisYG8iIYiAEMz9B54YVaTLfgQ0kMP8/gcxYfbhZTYbsjt+HRI7n2L+DzCD4tR+Ch
VVH7g4dz7WcPnz9MGY0rD2uNDbs/B72HRg69cejcIV5/yH5o96F7D80eEj77wHx7+IH0rIbwA6Km
QX8/9N5/9P7j95++/+L9s/crwvfb8hqO3Ru7l5659+y95+/l7rm7wV58d/huOnk3jHwLNDg/59kn
0qIJF+iMDY6p4il6x94G+54ts/ZJ5OUb289tv7idu7gdJsZr7ePIxLOnQBeugbGGMvs2fMKj+f4G
x2jxKB3B3FZ8MsHSkRGydChDXIcCUXxtC3i2wGZM9WHzXnyaFoqgIquBuRBH8BOIA1TPlpQ3OFhI
fUm+t6FnVdC+pqHEvhqZsQq/U4MpHQJOLx/kOkY40HO1HD3cBbG2M21n29hMPttWWNrAuPVQG7L0
YutsKw23llU2hFvz3A1vtIBjmSfQoFpmz2kQl2YspdGlnUt/vvT3S/9rqfDQUrAsyS1ssCzJcjQ8
tOTrS2hTQ4W9scFhj+JgFuNzvAHONVxsoJMNYA6mdRhB32EI6jvQBexAL9Bu19fqe/W79bxeH9A3
60f09+rP6Wf1ylosu6jn0AKfNIMAp+C+E+3Lfb6mU8rZtqaYsmV1DO6M5S1nn+HWVTHFnTHSsWp1
5wmAe7ruuPtusiirKRZc3hnry+pqig1gIswSk5gwZJ0wk0VdE+MT29mVjA/iCeIb9/nk5IRPTsI4
GWcVIFeyOvYx7osXTbCy8XgpYGIc5d2EL/5mpeOs0Edk8PHtPZj1kZ7xCRhHrNh1vGBc7sXHYEmC
HBm1r2cc0UMPKwWZNPbCtuOW/wWKRKDiCmVuZHN0cmVhbQplbmRvYmoKCjE1OTYgMCBvYmoKMTQ4
MzAKZW5kb2JqCgoxNTk3IDAgb2JqCjw8L1R5cGUvRm9udERlc2NyaXB0b3IvRm9udE5hbWUvR0FB
QUFBK0xpYmVyYXRpb25TYW5zLUl0YWxpYwovRmxhZ3MgNjgKL0ZvbnRCQm94Wy0yNzEgLTMwMyAx
MDYyIDEwMTVdL0l0YWxpY0FuZ2xlIC0zMAovQXNjZW50IDkwNQovRGVzY2VudCAtMjExCi9DYXBI
ZWlnaHQgMTAxNAovU3RlbVYgODAKL0ZvbnRGaWxlMiAxNTk1IDAgUgo+PgplbmRvYmoKCjE1OTgg
MCBvYmoKPDwvTGVuZ3RoIDQzNS9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJxdk8uO4jAQ
Rff5Ci97Fq3EhZPQEorEU2IxDw09HxASQ0dqksiEBX8/vnUzM9IsQMflquLYlNPtcXfsuyn9EYbm
5Cdz6fo2+PvwCI03Z3/t+sSKabtmmlf63dzqMUlj7el5n/zt2F+G1SpJf8a9+xSe5mXdDmf/JUm/
h9aHrr+al1/bU1yfHuP46W++n0yWVJVp/SX2+VqP3+qbT7Xq9djG7W56vsaSfwnvz9Eb0bWlSjO0
/j7WjQ91f/XJKssqszocqsT37X97bsmS86X5qENMtTE1y/KiiizKxRt4oVwK2JEdOCfn4IL5FlyS
t+AlcxbgN8Y1f03egzfkErxVlgy8Y1x/d88+a/CBcfS0GXkJpn8BN0t/B39L/wLnsvQvDmD6Fzvw
7L8B09/BzdLfaX/6i+bTP1cH+pfwt/R32n/2h7Ol/17d6F/CR+ifIy70L+Es9C9xD0J/h7sV+gvc
hP655tDfaR/6LzQ++8NZ6J/jLDL7w1no75Rnf/xHQv8c9y/z/asD/R3OvqC/2+iAzZOEUcNb+DPC
pnmEEMdXH4zOLSa26/3fNzUOI6r08xtAEdvOCmVuZHN0cmVhbQplbmRvYmoKCjE1OTkgMCBvYmoK
PDwvVHlwZS9Gb250L1N1YnR5cGUvVHJ1ZVR5cGUvQmFzZUZvbnQvR0FBQUFBK0xpYmVyYXRpb25T
YW5zLUl0YWxpYwovRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDQ4Ci9XaWR0aHNbMCA2NjYgMjIyIDMz
MyAyNzcgNTU2IDU1NiAyMjIgNTAwIDU1NiA1NTYgNTU2IDI3NyA1NTYgNTAwIDUwMAo1NTYgNTU2
IDI3NyAyNzcgNTU2IDgzMyA1MDAgNzIyIDcyMiAzMzMgNjY2IDcyMiA3NzcgMjIyIDU1NiA1MDAK
NjEwIDUwMCA1NTYgNjY2IDI3NyA2NjYgNzIyIDU1NiA4MzMgNjY2IDk0MyA3NzcgNjY2IDcyMiA1
NTYgNzIyCjY2NiBdCi9Gb250RGVzY3JpcHRvciAxNTk3IDAgUgovVG9Vbmljb2RlIDE1OTggMCBS
Cj4+CmVuZG9iagoKMTYwMCAwIG9iago8PC9MZW5ndGggMTYwMSAwIFIvRmlsdGVyL0ZsYXRlRGVj
b2RlL0xlbmd0aDEgMjEzNTY+PgpzdHJlYW0KeJzVvAlYW9e5KLqGvTULbUkIBAIkIcQkQICYJ20D
koXBRsZgMxgDNqMHwAwek5g4HnFinMYnSRs3djO0GZxadpIGJ2nsNjk9aZo0OW3PfR3SE3c4uX1N
3Pj0pD2nTRDvX1vCU0773vfu+773XaG997/WXuO//nktMTUxPYA0aAZRJG7a1jf+jXcuPoEQehsh
bNi0Y8r2ykZTGsBXECJVg+ND2zI9v/gDQlwSQnJ+aOvuwbbFH3sRUkOZ8veGB/r6f0T3FCBU/ym0
UTIMGY8sPCNDyJcD6bThbVO7Hky8xtJBSD+8dWxTX5HjpxTSv2PpbX27xt+iH0PaPwNp22jftoGq
eP6nkD6DkEIcH5ucakUZYYRWXWHvxycGxj/5/VcOIdQMSW4z3DD8sY8GQBlLE8rxMrlCqVJrtDE6
gzHWFBdvTki0JCWnWG32VEeaMz0jMyvblZOb584vKPQUFZeUlpVXVFZV13jFZbV19T7/8kDDCvS/
64d/m38b3cn7kAn1SvdbPlwFikU7EVr8mKVu3MPr/r8dhSLyeAF9Gz2DzqA3AToYfXUE3YWeRJdv
KX4JfR89i+5Fr6JH0H1/p9mL0M4dEnQS9fztYvgpNIZ2oSeg33ugvVfQBjyLKepFU+huNA99B7nz
3OvhJvQRvoBex0q0F7vIgzCGB/9Gg19CT6EtcH8J7o+wDPJH9CVSg0bJk9SHjsIMe0kTZL8Ofa9C
T+AutAE4bARGgQAavqWtdLoCHUZ7AZq++Q2/n3iQevFTGPFRNAcjGUHbURdqib6+QIBL0H3UCrP5
JnpRyju2VFf2NB0nrxLFwpfR/fC3Ev76UT++G51GT4WHw6fQI9iHfehE+M8oHu3hfWQl0iz+gX+Y
HEOjqAltRH70O1TP60VfR3tb65qW1cHmVSubGlc0BJb7ffV1tctEb011VWVFeVlpSXFBvjsvNycz
I92Z5ki1W82xekEXo1WrlAq5jOcowSjH5/D32kLpvSEu3REI5LK0ow8y+m7K6A3ZIMt/a5mQrVcq
Zru1pAglB28rKUZKitdLYsFWhapyc2w+hy30Tr3DNo87V7cDfF+9o8MWuirBKyWYS5cSWkjY7VDD
5jMP19tCuNfmC/l3DM/6euuhvfNqVZ2jbkCVm4POq9QAqgEKZTrGz+PMGiwBJNNXcZ4ghZZ1G6JO
X19/KLi63Vdvsds7cnMaQjGOeukVqpOaDMnqQnKpSdsIGzo6Zjufc3n23nkBbex1afod/X3r20O0
D+rOUt/s7OGQ3hXKctSHsvb81gwzHwjlOOp9IRdrtbHlej+NN7rEId4pOGyzf0IwHcfVj2/N6Yvm
yJzCnxAD/YDe2Vm/w+af7Z3tm1+c2eiwCY7Z8xrN7LgPMIyC7VBrfvHlY5aQ/96OkNA7jCuik/W3
NIaMq7vaQ8Tptw33QQ58vQ57mcWu71gqE/xbrxEgAtABOLXb2cSPzYtoIyRCM6vbI2kb2mi5gES3
qyNEetmby0tvTG3szczSm+vVex2wmo1r2mdDnLOh3+EDHB/rC81sBHrazJbCIYRi/myxO2YNelu5
u0Mqa4NRNfSP2EJ8OqAFat1cASiFVZkVpETMnyOPqxboIF1vsJU7oBnWjs/h641+dwyboQFbbk4o
4IosfWt7SKwHQOyLrpHvfL4bavT1whKN1EvLF3I7xkOxjtrr68mG5RtZ0y5ViVYLxdaFUO+maK2Q
21fPerb5ZnvrI0NgbTlWt19EnsUr54tsluc9qAh11LPCcXVAV+m+2fb+wZC119IPnDZoa7fYQ2IH
LHCHo32ggxEaYCjrCnRnl3oMkbrW9sY1jsbVne1l0YFEXrDmOKfvtmYc7ZZIM0ByIYVTYWsnFtoB
BQXIsPkBcNRWwT0kdyrgEgDhUi4j1doqWzu2oKXSMIxQls03UB8tx9K3NMozcqoLLLUmY0lopy5g
sXfYI5/cHAKvbdGOoYaCITWw9Io6QRJAHoFmpCyGSzOjeVu7Y8DR4Ri2hcRgO5sbQ4+E5SgyJJxH
16r1ltRNyAI0ITu8XkowZIb8LsvNyA0tl9LXk4HbXjcsvbbNKhyNa2ZZ445ogwhG3hBCjITFMr1F
4n7Gzw5/HzAxcLTEz7PnRZHx8jBj21lHQ/+sY017lVQaJMidlj2sLwNqxI2ttbk5IMxqzzvwkdXn
RXxkTWf7RQHMriOt7RcIJnW9tR3n0+Bd+0UbQqKUS1guy2QJG0uwllogoZDKWy6KCM1IbzkpQ0pv
msdIylMs5WG0aZ5E8oSlPAJ5XCRPlPLYB1bJPAw4Bvnts/Wz9bmjY3i2t4PROIoDjMAXh7CjBrDj
qDmPiUwTUjkGakNqRy3L97J8byRfxvLlQBk4Dufm7JkVfI4/mXMl9Y7q4dbPt4GVLEd55zFyV12Q
c0lXC8/L+PerLlACIDpPWTbPsi/IZcmfV13ALN+jt+uddr29ntjCafjh8DDf9tdn67l3mGGCmhc/
5ob5TgTjEHcXVeNtubgxuSuZbBFwrwZP1RyoIVPVWKwOVpNDVbi+qrWK7Cg9VEqmMg5kEM4+ZT9g
p61qfFSO98jxMYJ3E5zdmZWFOpERG43WZcrOfYo5BVEUdSKrYCXWIiPv7k3Lzk7jYnp4URlUEqWS
NyKvx90tXIW7cNVQXo7d3Vf/R/dVvaEcgEL31YJ81H3zB0fusTHEkZqWUZpCPYU1pLgoj2Tk8cVF
NcRTmELi5XnUkRpDTLEAp/DccM2+1w/9PvxZ2kVNfuuetsDOtvxlmw/WFw5uWF2dGrjrbP/U49vr
Y19Vu5b31we31iYXd+32lY4PddSm44m2ueGK+W86SzuXpVkqOr3+9jKHXpucUdw4UNd7dF1mVsvu
oN0TLEmyl6/KrWossBlirNnlLaMMv+AbcOvBAlYij5ioUPAcx2MM7gSPeV6llgfBMOWC1IS8Vwu9
MGFc7t4OM9Z73DD3gnyn3WQvhqUrtpMr+I7Ps/HXwx/gfzl+/DgXOh6hizmoPyC1r0WimCqXsx7U
KzSa673E6FRr1Ah60t3U0019AX6/2F1GscdE/s9ol9/B//Hmm9ApTTn+o8uXI/3OLn7IJUC/VuRB
W8TG+tTW1P5U2ljYVbilkFbgzZgUqzAvN8mn5QflnIpL5PZwsxxn9GejoAGfMJwxEIOh2LYcybAs
0WRA7qBaITiDiMYBLVxlXy+jAkYB2z2eGxTgjMGOVKIX7IVx8Z6i9KL0Yr1Hz5bYkUeLMQBLpMDW
nm4oeGz3W6/hI7sfLyAYv4AJwWcx2IQLv0qu7q3zbWlwOgObfWKv1/bNkR7gOjMpWbuRzy7MVeLH
PzNmBKpcSi6tsCIRbx0/M1yQP/LUrqmv9mbnDT4u4eAg8I6fW4VsyIVOiAP9abif4oasjqyRLNqQ
1pFGppMPJpOdSYeTyK6YIzFkj3ZWS+pJK+knNGO51apueFf1gWpRRZFKUPWqqEqVm46WI9MVEzGh
YHo6bw8mCnxQ+ED1iYrMqLBKJcRJnMJYhaHmKkNRIcPRbWyCGXdEcFEaHwN8kEeW+CIZY8CM3J5H
cW160/YVpd0rCjWPCuLmh4cnv3VXXfOhc733vDjhCTvzBtZWUhJQFK3ciP+zbW6oPCGnOi03t6vB
HTj+L/ed+sn+staHfjJjxj9KXrFv44a7mmySPPkG4CSNa0Rl6KDYvtWMd8cfjSdb4/HmEjxZcE8B
WV6wroCkleLsZKyiiZTwNpONZDZkZKCAyWStUDeoVO4AsvZax60zVs7qNvG5wVRb5olMkpmZKghB
Xq3mJSRI0qL7urjwuCVx8QVpERESIBlukRK0uKjkhpCIiAjKpVWOPz6y4aGJlYbTmuLGjeUVmwJZ
nvbdy8W7NlX/5PvNs1sbYh6T59W15v3JtWrrsmVHtzfg4qY9bXmWmqEma+myVLU+a1l+YXV2ilGf
uax/5QNfT/Zu8sXmltk1D+WVp1v0QrZ/G9ANvga3X4Dny2R5mmiQ8fRBhjqkVHCP8DoMDp17iUFh
YoUF+XagcQoXvvbRRx9hV/hfqI7dWaXtgO8dIL+z0BNibnkG9ievTR5MpoNJuIRimzXfShgqyRkr
tlptYnJawGZTqdUhIKb5xcuiT6kJqFSuHSCmCOrKN4mmoIkikwAE6E9cm0icPYk2WyInqGyqfKDQ
cRWfr5pRvae6ouJUqgahQyB8jwDS+2o3rMV2aVW6YTW2X+2GtXBBJkgWF3s1EV2U26gSpESRoQbf
LK9xRWbrPeu9A80VptNJLVOPDN73zuFl4oG3ZgceHluZ8JSuqKGvqmNfSzp5+p4f3BvIXzVYssx/
cEQcmMfk0a9jfK6rtHt37bK89vrs+sP/xGQVRY+BjJzhfSgO2VEe2ik2D+TizWl40Ib3kmOETKkP
qE+qqVyNxwy40dAFkilL2xkvamMD8fEy0GSpODU1PymrE8kE2RlZSMbJDJk9SUlZlh45zeqRGZA3
yo7l1/mRJV1X9eXd3depUaJEEK4wVxOA8hSKi9JdWM/QYQSqBNAOIDfTevzV4XAmPjU9P1Nnr+uv
b7unPS/8h4ce+uBXqycC9tWe7v3Bz7Gq4c4uD75j6ERXNu9Lb75jbdnA6soYZdm67SRzS1e411oe
XPiZt708MZwaX7ZBCkKgNMDDU4AHBXpQHC6S1ctaZXS9HFfKG+VEKU+QZ8npGh4X8fU8OPe8wNt4
quHlihjMkRjezWFOhjt1xEvmyGlyibxLZM2kh+yDJCeAOUZUShkVURD1Iupl5IzlHMWAHK9HklAe
T4SigTrKv9dd6MJmYeWHQNw9G7p7JAShbqx36LFHiT3kjVfDSe/gHXjqHRJYmCcBsrDwVbIJYbwK
v0fvIuOwqsmijlIeIQFIF73sxqDwAO+wApIqw8V2E15F8vF7Z84wWQ0f7m2YuxzFogmx1kgoj1uo
lm7g+FiOp1wsBk3UAkp0g1wWK5dpOUGO5fK4fdwcRzg5Jj2xRmOnFjdqsRb4lUMwr0Kvx9PdXe72
gNCBboHUoevDrsOuN4TLlwX0ncO8OZoADqZ26mBTy6MZMVRO7ZxqduGdo2/g8M/wnxb+XZsUr+c4
IT5B+xB+F98V3s/7/voKfTl9lb9Yoynxr3KGOwCjocWP6c+lGFrjSwPCtHBQoOr5xd+JqVp9wGhU
oE69XhE/o8RKJVIIYHMZe2IpWFlLBhZTGjBOGKYkKSU0IVdUQqZHBGRJMdP+9OeegQc3Pvu24xVV
dkVD1vBI+Pv4T/iJ/scnay5ctordVclzs1R5Ci0uoosgf6z8XkM6rAPSy5AeEJ3wLYLT9ekEzy/+
9YXszDSjE7uKixG7JDqU6oD8cyKfmLMvdS6V7LUfsxOVPdGebaflSTghxq9WgyxDaTNpJC1BFrTG
xZo0SMdw7r3qiRoGVz1LQkW/pPdvsgUcUbWnx9vii9u84lBDxotM/z+DCSU0saZls79zf2sGV7HQ
0rx5mSW37c7VZPLz51Ib6/LlfE55Zay7qTg5Z/39/eSfmcxu5l6nd/B7JZldIFrpQYTkBwXgCR6f
4GBgqAedQ5cQj8S4lAAIcFd34dXCqGJi1OgBywpMqp89vvDjJ7nX5+ETwcUfgS5NYENYkV907ZId
kREuMTaRcAmxCcRqtccp/SpBsKrGVPtUcyB042hLkiDolGDGRTHhucNtFiShE8EGzsOS2QusxFSd
J4XGe2pgZePiOZN7qKb10PqCFzSCQfasXBejInxSTdtUS9nWbPqgQl0/fao9zJFvl00NtZjNNXX1
yTX9/nSVXLIzw+1cCuj1AuRDp8SaPfmz+WTQi32aNs2AhjZWdFVsqaDFdswnm5KZ2cOp4hPj98TP
xnNuf2FqMAafiDkTQ2JiEpfL/PzLi9cQD+onU2cK8HzV8lAiRonvJZLETF1pMNUqLAvqTCZlENlk
+TIikyELLHzEP5C+koYBxVIW8RBusQ+7cUTZY0YP1dhxs/HDLEZQPZ4YatLHyoxRI9EUJR4y1Hl0
U7V2Xr99fXnXMgcxVq4ZDQyeWO9y9Tw8uv0buRSkDX6WkdBP7PnNA8V1G8WUFG9fbfFgsCDcnr58
Y1VC4+rUxl1rv5vZVOGoPfTW4Xvevm/FSF9CTWkmVbqqGjI+/95v/o2+u/3RTTm5m766ffrUhgxX
7yNAA88DDSQADThRtZg+njqTCo5VrORacU4/QhkJGj/SzmiJlnGCYNRpQQLc4AS29JH52/VLU7qx
+CZp8UHl6Gm/vTKYXzm0MueFG3xgXta2uXbNzLpc8lJwqNrMuGDhPrrmdi44MbDgjtArszeaJXvj
iLhCQbApxZkynXIwhWsz4IwMlfOcCZtMsbGo87LzPSfZ55xzEuTETqerJHlH8qHkbyVzyV35KlEV
jFi9RJXRk5lp080vXhNjQIyZe2DldZyth4msiD/oihq6EaOiWzJ1ozIssurO/8aUiPCAYJBsDSbS
uJVpPVNHVx363n6xZv+bR/sfGms0P5t+bKzzrmbH02fz19QVaMO/wB8cNJfkpoy8jPGXvwG2RGdR
x25f2ehu3+F/2h/+NPx7fXyCEgclfbIf8NAAa6ZDDtQhVvenYn/q2lTSn4T9SWuTyBYzbjB3mMkW
A24wdBgIWn5OwEgQhF6BCoIzcTkCC8wU1ArWIDNnI5I54u3cbL8uWa/FbCYGehvFcg11d704se38
nfULv/v0D992rdwsLhttznU3j5QvGwvmkpTjP77XV3/vj+dwPE4I/z784V1PbMzI2vTE3rue3JiZ
sfHrbD2PL/6GmwM/rg59W8y6o+LeCjJddrCMDNTiwWW43tvqnfIe8HIDbjyQN513MI/STDAZPUzr
OAEot+KSgDW9uqSaVAfMojloPmE+Y+asSrNZaaXZKKAHK/N5Q7z0FLOUMQG93pftd4kGUwC5sEtf
baV8WrAAuUy9ynHljJIqlTERjMASA4Ub4plN5ZLMKRdLud1MELAvU16uqLeDlrSYG0e0WFEeBzzP
Fd/MEBH/B2wvDhDIcXNZ6w73BE+WksTaFauc6zeWrO5ZXWIvqCiw272d5fVbGjJe8O765ujgIyPF
dk+Fx17a0tNSijNqd7QXJZif0CUaVD3NlS3e8uUb/J6Givwir8tZk5voWLmn/XPK0Z1nx0vygttq
KtfVFRUU2jKrK8TWMYSJMbyOXAR8UxQvquizwPk8mEog4LySdcR0BSgxYvx6eJ18+r+OSTz3FpTC
oPfVQGllMhmO4eQgk5Ux4/CO69Qpvco55Wkl16zsUe4D8F3lB0qZwMwAjZZT9MgNVN0D/Ul+f+F1
G+yNbhfYWma3sPK3hcBIRnD99VH3/y1aGn4DV33+fVwVfoP3nfrsP06d4rQR/t8EY+kHui9Hxy+i
7MX3nlfqArb5xffEDqU2kFoOqk+bF7jkfte96Ka97nE3scKKed202d3j3ueec3OC2+bOd1O3uzJB
E0BaQUuuabFO69WSd7UfaIkABlaxLJiQmkmZqHNHZB0Iu3iPpE+7t2+fiLq827dvj1jdEiX8beEX
F39dBjoihWQmwLDG4rLZi9Jjn2bS8B/YjWjTPP689nVGYnCVBdzVrUXxT7FStSvJN5mN4Om+Z/XC
15akY1ZxWWzG8hJrb7+7oTAxIiMj5Z56auE+JiNmQEYcAVwVomXoSdE7m4b3pMymkJHq3dVHq6mz
pLiEGG3ARbL4uHiSrMxVVispn2vKJYhiSvP9Fk1zWU8ZKSsz1aUvz5hf/C8x93IGSFnvcgRStny1
xeLJ0aVb04k7fS59MZ2mp9uDOYInaECr5XKNgbnJYH0ws4+5YPqlgAEkoprTw0zXqIcisZKzsKTU
xCIsgD9JdaYzzenFS3wkz5AQCuozzinpVzno0tg4bqUuJSa1Lt3TkGemsqT69okVj55pnnm89WNL
aWtZ4eoKp+xVVWn/Q1ve/H72ws/X7Fmd+VrpWEdZxxhHxxUKS+nqkqLWStvZB7ccXWXFXGXdhgqL
LqPOoxe3rHK9cj6cF1wt8/ZOFXdwWJff1dTSD3TYFPXp5GiFWCCXXBTub3gn75IPiEzyUBRKxPXw
EafkJlb4XsQXYaznYLrCRP7xtbCPK+Ke/mwd9/SpU4zuP47G2dTonJgzKN8hPySn/cop5QEl3YuP
YdKAO/AIpg2KDgXZgvaiY4g2oA5EVExUZqr1AaxSKIV6ZSsLeKLT6nPqS2qK1IKawKXWyqkfeJxj
0tLqDEjPOIv0FPWgFjkiV2E5Vuh4xEWVf3k5yAkwAFwgBGEuHncZc7FgaT3lwBAuyZ+SJKMdy2Fa
Hsmxol8Kf+nACy/gX/w43IB/iP9jY3iMf/vzPqINuxcekmJazFf+MuBVDzLpoYsocfHa8zq0Uphf
vCI6dHrUdBRjNaDDFqMPIL2g12NOwIJAd6gPwUw6DWzAJSZzYJ9hDpSeARsMyXG0E3ECd4YLceDi
kB5r3L444o0bizsd90kcFxeHTD3XHaqoBiy8EddyXe3WM0otdJd5XMybdiGXHTvYQi050pgWZUV8
Z/rt8Jf+cAn/etfzOyszWvd34RP/FX4vfA2b12z3WYl34Tu8z9M3u65m71CTbuFZ+s4H4ezMuk5J
rr22eI3+hW+FWbeLJZvNe8yzZmrOBIUlk8EceKU/8QR4mJgkigZnIFFUxwRQIk7UBOPjFCiImWmo
k9hNYFKWqSyXRwpPsVl4Io6Wsdhj9DAZC9wj2SWmCIfpX3umYhcOhYNtfZltzT5znH/dYMljj9Gz
x3Fi+MPjC1PNTXKVhj8qE2JUR0+S09J4fbBOh8GHEtBJsXYthxtiOmKIRdelI0oddtI2SmQ0jpJM
PIwJwi06pa5RQLGCAC50C1XSRoJjCaZEScaFGYEIbN1SVDEBQTAQNjmiI7o2Ja5X4mxlBZAswiYk
ubvdXkZzLKwD1pjk5DIf97BZYA/goKjhYgdnF1RxfA0tBW/+B/8Sbn4LaxSGWL1MZow1KXDs98Hx
EuO89V6TyVvrjSPfjdLeANCeBsWjPtE3TQ4SskeYFch4/Ew8iY1Pi++Pp/t0czqYqEu3RUeL1LhI
hSspRp1IewYMZG1CrBSlITJjj5zGLsVmlmyrW0KDgACQdJGAjDwSfwEbixsIPvzLe8PHcemj//H1
tstlE2endz/2kx+S0FPhxW928L6Vj/317OHLu8s/+w+sYutggjHPwzpk4xZRmEo5kEL8KWtTSFpK
UQpJZlyzwmgOnEjGA5JXRNOSi5KJLDkumSTZ4MXpJLwlaW/SsSSanJSbVJ1E1Ul4r+WYhVhssAgx
NmD9xJjsmIoYqo7BWtaclTWnxU5tsXZae1DLYakZjLNxBd6DKWJlxqGaCbyJYkSpDOEY3CLECY26
mFidLkaLW5Likprk2li5Fkgh2ZHcJMexcpwsByrJcmQ1ZqPYbOSQx1FdbPZU9oFsmp2do8selONC
OSbypKS4rGyB6nSOuOm4g3EkTotRjEQaQBx6D2hpRu893d1u7JYCIvHRiMj1cMhh820E08NUT/Qb
sedcSze7g+bhjPQMGQh4OXVQjxF0eklpDS41AohBbTJ/zuihf317ThGrVCvhG6s6+YOXLnRTtVqh
hpT88Hf2y4xKtUqh0dCBEP4/LD6b0+lMsy63hnOBBr9mbutdn5yb5WiIxU+F18Uud2TlpXT2rE0g
PddjZk9Ltteb4nqlEqtlWEk7KMnCmMPAQIA2hZpTy4hciRUqXiWDLMYucoJbsEKuwBuIPJYQOa/C
LWqO59QbVHysilerWEyVEHCAYImUnEIS9WZVUoDjtApRbQkokJLKOKzgDCpYVq/H65FsYYn1ACoH
S3h7hP+idwmnSzCLQB1ml+KyInqXmJORv5JpAof09ZBBvPKd8Er8/jvh2fCxd/D74ZXv0HqCF+5Y
8JLqhdfJd8gBCQ9XAQ9NgAcepYtxHCennW7kZXE1Fn4D+deDJFZjFloksgTKCWTd1UvkDO/7bN0p
plueD6/jarkgykRVaECs76/E/sq1lWSLCze4OlxElh6XviP9UDpXZMeVCrybHqXgN/Vm4aysGk+K
3o8MMwZiyAumpAlmnd5SFlTxkc0iL9wkjVF4O5Nfj/47gHC+6BTLb3eoajPXPzha2r+2Lp5n0l4K
GJ2D2VEuwRvsrxl9cH3my+bKvobK4ZW5zsDm+oaNlQkkde97D7bF5vgLiTWvIjncwdPMFZXZSuos
qkosaiqIDz7wwzv7T20pc/adPTz51b7s8tHH/reNg6HnYcy1YAfZUBHI6No9ZJaQek+rp99DGx1d
ji0OWsHj4iTMW03WaetBKwcraMd2e4lLFutHxhkjMaYHE1MEI9Ka8oMy+vdXEAtSaNLB1Lyjhkrz
ksllcg9z9L/oD9f+4NtTz+QTQin+JpvjC2wRufynp19/4+UUb09t7eaGzMyGkWXeXtFOUsO/DX+4
dmNiaZ6dU7qqAhnc1XBHcnlcYvxIT/ij8K/GvzZUkDv09B07H+3JYDt+WPL5y6Q4zVfFlv443C/g
3SlHU0h/Iq43tZpIo75LT6YVBxVEvVylSlrutTRb5iynLecs/LuWDyyLFtprwRZLhnE5W/1xC0YW
wdJrGbfMWC5bZBYmAZIsKQGLxRy0x4EbbFXvU8+pqVqETLWaF1gQ19Udsedd3duXtBozmG7segGT
G29sgeMb+ClZ2gncv/a+F3o2fmXMr70szOxYO73cal0+3bb1uP41Zf3Eo/1HL+0sISmP/3SmuHDd
jvqYrpHSkUf6+0+PVmzepqm7Y0P52ofegdHXAi8fBdq1gj3ckmLWGQIDhmnDQQNtSO5IJpRprlxa
TYmLVBKCApqGaXQQkXQ0iMhmgSl1Km0fIcE8Yt5tPgrGlgCekNmqUOhTrIIVcPGSSh+wWu0GPYsN
qfSiWhfQ7zdjM9NwWq0QMAtCil6nCfIWKVIEasfVDfKxzMMkZBnTQ4ZyF+gkZrRIwQLQKYAcl12y
ujKKWYiwxIuv2ylGD/mOM7hx16rUCpsxX998LN8Qzr4M0t2WEk9ofIpV8/lrG786VsnJD1O64+4C
zr1wl6V5ZY1SXd3YnEzulngarw2vo3uBP0woU4wTZNf0Ot7KE56PR9eQGqsFJGO+/tLSgf8FYnmJ
VaPBSaLPaj/a+9r6wx0uV8fh9a/1Hm3PIobZTz/81Y7tv/jwsxMn/vJvP9u+41cffirJEBY7lPbh
vWLGeMxMDOG0sdop7QEtp/cbDHaepPiRdcZKrFa9yRzkYQSIbclc9XquM1yEdowSb93uKkv+9MUX
ZZSaxZYRcc3+TveL1+OI/NvhPYUDeTdihDdHESMybgOMbxvQiRpdFMeXc5g5TqQc78ZHMd2hwAnM
X1IRpZwqZKBaiZKjCh6BsgTLVM6BfYpbFCCmFI1KEqtUMksWijXyKJZHMigl5zgKWl9Qj6tngFHU
Wp6nchU4UgpeRyOOkodpTWkjkm1Hgua8xWj9f6Avcbc9oiulL30ifCn87e/i0+HJf8I5OPv74Un8
BH41XE9ySEy4Cz+58OnCjxBe/GwxjfvnxQOIopgXKEKvYMScb9CLcvAvee5/vN/bK/lbQ1GbV4W0
qE0sHpHtlh2V0X0seqPAY1rZCXKGwEenGWObDYibhJlprBq3ZkzDaTRgjsiweWmLzRMJkVxdgiXf
vjvi1GJgC5MS2+nTLyxMkL3/9Gr4RFiDP8Xe8Hew91665/Mjx2ndQqNEw9L+NHbB2BNFTXRvmuMj
+9LRXenInjS+9tHSPvRJdv4d5mFAHWI50mO9XhgjCoUVYx324mbcg+fwadClcoxjwBcHm57MgAIi
xljFpFJJY2QCnkRmKUjl8Sx1s535UNslp4pNxVNs1wPzFjPfiRkXX8UvGXvvPF67Yu6O3sSv4777
6ULm8iLrydTq1sLPmu6X9n6lMxpSvLoarUbfEu+azjyYSXZoDmnILnKEkF2KIwqyM/lwMtmdejSV
7DIcMZDNfrxZxANBPLACD1ThgRKcJliLRk3W0WwT/KnQqFiDUc3lmvdqaE3NmsDoFdU1FWFnOUgk
pp0mCHWT5eUrdyQeSiTuyUT5bv4oT1ZO8glsq9wrBbKvljPvvbt7KeprYBadcFU60PCF805LYWAm
IYojksKz5P6aUujtivD2rQ3OXz794p13vLiz3N08UFLZ7bWXj3992+TTYyV2b3d19fBK179aavob
/Ju8SXEVg8G1g6V6R1L95NrmMZ/N5p9YvWasPhnvab1/a3X1lvvXLJ9sLVJyMTWt/cX1d/aWV/Te
VV/U31qt4VTFrduJx91S5XBUtbidLb7cPF/Lwiu5HctzcgKd7sqBFdlZjUMRORkEOTktyck00Yiu
qdXxev6aEPHdBMRHhaRnaXcDL1kAUQuOTme3H15/uXe2PTu7fbb38vrD7dkk9sRnH/5CkoyzRyVp
+bN/+wujzQmggQ4p5lotJs1iXI5HQABRTARC+IMUl9BBugOMPHCerr3EKtAEjCIDKCvsZotV5mEq
JB50xQQtvvD5u7H8lb9Gzrs8CnSfC3Qfg4pFgR+VyTRoTA1CViNoiIZpdHWsOaBBmkmtjDdLYScs
uc7S/iubmExFTI6iRFLKNilye5/d3/Rq8FCob8FK57iXZ8M/D78b/t7pR3ElLsRpB++X+pyFPhsl
mb/hIlKBdiwB1ZmM/cTLznvJSEqK3k28ZEyKfvFEjHUGEMFET/Q2e3xQw06cJfM65tGDHogeOwPi
62baEgRlJG7BNGY0Hnb9SJ0pckQjGdtNpP6F8F+xLLf/9ERyMLjcZAg0B4y1G0Ur/oie/byVnj2e
MDDXlU1lCu5+KuNpbMmGRtXxiE5gfv5GyafqFCvloio2IJcjFeCfHXHS8srOiFeh6NHxXn4fP8dz
Y3A7zV/iP+Dho+R6MFVGHQ6Gxu0TNw5CMRlf7nYx/yMa0YbrMfrQQiY5tdBPMe87Fe78SthzCsZx
t4RHH8pBX76IUhZ/JxrBaklPKUkh2WaAsrLLs9kaXhMT1PqAVmA3TVpaUs76bCQmJAUQykthb7Uq
PWpKsaxHSUISSWKhu8zcAHuKSnNKIElmMiWnsZQmRh9I02qzk2lS9PgAs1mWongwfrbT5fJc1YNk
8LhgCrAgLKxdUlpSuuQB51Fwh+PiUzBT0FEfOOoQ66kUydbrY+wxTxz9uixGJygMRmYBn7l6URnP
fn6kUZuVF39F+z2bczwFpcW5/enhFfhbuix3gbm0Kr8od8Tz+SneF46PX5meU+B2OVcl4N//9RWJ
X2n4H8iLkk4oFzOUKAHWR4D5KEkCaCcG4S+Th+hX3GgM7YNC6BR+hEPCn6WFWbjKApJgnRnt0NDT
WBn+PrT24MLmaMynR+KfBNQl1gzGY+kwUoOuQzcCKrxTCvfoJHTrsE5niZV3ssMNZxQhBaeQ9cSK
Sn0gNlZu7FFSuRTWlQ7eXCcHQGvkCE709I0jGu5ZOnjD9VTu/tbe8P+8hN++8+U7ai4/8cSfwy27
m9Pp+dZHdgUWwryvYvM/bLjz7oWrrhWDS7SbLdHuOrFK3amwIuYHs0nz7DaHLiEOIS34x+McvhH3
JD1BOT4jD8mJXK68cX5EkvzS3mZhZCdbUqx2KV4rHfrasvDnS5eI6hIZW7if9y28RUoi64EaYBz3
wDiUaFpMP6rAIwq8TrFTQbIU5YoGBeUUsQqiwC1gO22QK2LlCiqXgsq62ABitJisjAlwcoTAxlK7
JQVNwZjq4SlR9LDwuLRXhKX4jdsN1HndaFoK8WXhYsxC5djO3fPZ92jhQjz97uc/pH1zXOKpY5/9
GkX2Fj/mqmGMCWiDWD9gmjYdNNEB3bTuIFvYmfgT8e/FX4m/Fs974/FlgMm++Ll4Eh9v0XbOqU6r
zqnouyocUahmNTUsbQlLm0BRxye6WXpdP0b83qgLzFVX7Hxxz9Q3JysuJVSPBJkSew3//uh395YV
T79ymG77/Lm+2fbMzI4TQ9Qv2Qp3RPe2jSgDudBusaXEhv36tXri167VkgZ5h5w0sNiPkG3LJmJ2
MJugLJydlZ2VAhZBbDCWoFgcG5ubMhYZdHMmbs7sySSZWVmOhEmd3BHR/xKJvnPDfYuqfpjNdRdO
ikzevKWNDbeHKpzFHhPXvPLgC4NDF+5pbDzwwuaB8weaXspctT3QNNWcmdU80bB8otkVbv+IPP56
+D+fWL78cax6/W2sOS2Kp8Ofvn36J3cVFd/149Nf+dFMWdnMj1L/8peIbD4BdFUFOEhHh8X107KD
MlJuw1OWAxYiGTM7TYdNZGfc4TgyosJbBNwhYD4Rp6cnj1oVXgUZU1wCslOgTMOo0Yh608bTZtJo
Wtq7zIJkIt1g0E6a5ThtydQDR5Ytp0c6PRU9zXG10H2L/eMpruFv3X+KbOLK7aYTOScef/Jga8by
vsqSbn+OfF4pjj22eeQb0zVrDz9xasZGfjzd98T9B+4+uKJqXbklpaqjUt90aKAiv3u2Y8PXTuyf
udv/JTbnKfQCt457FHi6QXSqI3tJSvyMKAOtqQIbVaOFDDkfky/HcsQ9p+SfZ0GLwu91uxDbN5Js
1Z4N3Z5EyLttK0kKgpFzD4Tb8VMP4KdIb7gVn/0SPhtu/VIE32yPMKKDjosbRhS7FUcVdES+W35U
TlfEd8YTn7HNSFJFUEepqWjU5EyzW8dsNpVowsh02fSeiZpMeerRayp8k/npnLTZEifHMnFmppA2
aZcJk+zUbMLSGTApxOKR9gKXrE8pilRYyOxs1/WTml88OCstgYNtFdPISQuuUbzrlb1jT07UxXxL
nekbCICxmJMNNJjbtKw4PuSm9oWDyfknNo08tUPEr/Y/u9df1LXHb8psrHS4gtMNVUNNLiHJGUt+
en94nbNY3H7qJpx0okr0obhpuvxgOdlUjsXyYDnZVXSkiGwtwo1FXUVkyn3ATWR5O/IO5dGdWYez
yFYn7nTiTQbcr8VtWsxwSRgmiRRKlM56liQWjHptPTZis6FRo2jE7GcKTCBWmcyBRGOiUVWdPpqR
oR5l6Pxk6YxKScJkYqKQk57umMyRAzZF9Qk1YbdrzOmMYLZ7eyQwo186CLp0Jhl428VeetzbQbXf
btr/N4eU2ekF4sLFEaynO1JlN5G9ZO9zjd7dF6b2vzBVoXxZ4VqxreHII76h3Z7BjZ7RrsqD9+x8
QPOiOrj30Y6dT2/1OBvGVrTc2erCE13395cs23K0QV+2vjbtrt31ncVxpxJKO7ybd+4Zi+me7cot
6727oXRorajjlFXrNkfO+d8Ha+GFtUhDZehJcecexayC7JHPysmQbaftsI0OJ+5KPJJIfaVtpcRX
0FZAVuR25pLGjK4MYpTUGDWiUa9zznnaec7JSYeCnM7kCvWoSuUevZJ8LZm8m4xRspBMkosmjWbR
GBcwm2OyJ1PlMZM88+rJdQRHadfliVLuDYnh+eK+zk2nwEpvPgP2BUHKef0Hvr1z9f7BQNzZpJ1d
K3a1FWBTQ/eWst5HtlZ6d50b+/jf33A2bPEvGw5kOPzDvsKhNcXk0vnwH1/qtvvHmy1dbfUH3rjb
vcJjqd/z9MDw07tqwy+EGu7ZVJm7ZtJfO7muMLVuAPD4EOAxEMXjs+Kde+gsJVLwdJjuokcoHXLs
dBx20KHUnamHUyPorCtYczM6kyw5lioLNY3GxqLRc04sOjFyMoRSp7Mi2T16QnUmIgCYN8qpsifH
bFiw2RipF5kndXPsYJZOXhRVQtGz84xMb7I/tv/N8/NfPFwVSX4BnYHKyWfG9s7vqfbtf2VH4K6h
VfFnk/auXrGrNbfgwmTvV8eqXkoLbPYX9K/2ZDZurl02FEjHr/U/e6e/++yfHj6PhfmelLrRoLVr
te/QP97d1VMzcWZL3Xgw175s47LA3RvLc1umInLzu6CnqiX7Z7mYS4hMoVBjPrr3x/V48T48h6kX
j+HTYNmIpqwAxjzqkVP+upHYfcNriJwYlpwFkNomfDd5YmH9JbqXezpseHThN+AzoJttPyPaJaYZ
1iMz2J5oRn9C/57+ip7z6vFlgMiYfp9+Tk/1epO0958LboNKvn5cgW/YrfdivAeDE4r1WEnvUeNp
NR5UYzVSGK5vsrJ7ZGFcUVPH42LGIb5u/OP0jDzMLB5av/BfHEf5Z/6VCOzXP2e5za7c9I6Mz0/y
vs+OZ+ZlbPTQXslsBDqcwu3cOvqxdI61SSyVEyWHsIAJ5p+zcvu409w57hLHc6IhLsCh5/JxEPfi
ccwF4TaDQ/g9DI6jdNIahuZ2ScarR/gheItGZgzCNUV/83ky/Q1tP3kyjE6eZHh7APCWw1XAWrWL
4MnEydPla+WcHLeAU9ook8fK5ETWK8PgKypklKpZmOgSpuB0YBkNcjosDyLTDZO0e+kU2M0mqWfJ
GjU9QJ5e2EebFtaSd4/S9GNHP//5Men3SuF1XCK3QuK/QXF5Y25X7pZcWl/QWtBfQCuScDG4hmmm
tOm0g2mcypZo22ObtXEGv5MPGvEJ4xkjMRor3MvZTwGKggnWOIMlO8iDSeMtvLq0d/GFo6iYccuN
E6eOiCcn7ccQvWDAt58/1dcONlebCTtLcI55ci9KBw4SvKsHlr312uvEkLliZFl9X3VyUlWvr3J4
VS63Ii7P504oL3BQZWb9xrrPSvBLiRWJKUmeVcWJOAGb4EpeuLLz0Q2ZnpGvjW4+vbkkY9OzgIvp
8Dpaz78Ndnky8omZ08aDRiL9GMdsVpkSVyQnIz87YGRKsaoS11hQfNCsE6Jhdmkz/frv8q6fweSX
FFhx1FhY+g0ZrV++49TarJamZea4qkBzZllfICs8QhrffDNp20M92XKNTnkvr1HJk6p76ug/Sj8v
I2gQZOUgMLMexrdbXMvOjZKiZLwj4VACKUqI/KSoMh7vNh41kkojXitgn4Bvs9FRV48BGyS/kZ1C
sZq7giossEOXPTrOEvUlIqJP8hO3f/FHhrdb4Yiy3Si2dOycwKD/wKVd+y7PiPX7L93xn1jx/Ucf
PnXq0S8/fpI8/Vz4r+c7NjyP+W88h8nZteEnw89gcN/Dfwn/Ef4+xmbpf4Rg/UfX3ljX16Or+hOy
Rv4/xbt7nv7Hpf+hsPgZUOw/wyqxf15BopmY/W+FhS/f9N8bMLr1g7l3UD1qBh96Ds2ig+gbLJqL
toPsSsOrAAqhi+gibkZ/hPfPQ/5+dJwY0VtoE5pBTehjKPca8sHdBJxyFUpchGs/qsVrAdqw+Bka
gvZOooM4iCbQo9DDY+huzH5h8xh4pMfRHeA/TEFLM+g+9BDI6Mcg9QCUmkbgPION+z7W4AB+i7SS
D6kG/h7lUrkD3Bme8E38IH+Z/1gmyv4gr5e/qchR/FYRVtYrf646orqsLlKHNArNhOZt7asxYszG
mN/r6nXfFQLCT/Qa/Xr9T/UfG0oMQcMp47DxUePbsbLYwdhnTKJpq+mNOEVcZ9yF+KL4b8Z/bN6R
UJVwBayV71rSLBssZ5LSk3Yk/U7CYCNqAYmYBxcBpnajTnAHc2m8lEYoCWa/hOfe6zjHSAepCExA
mo5FYYoS0Y4ozEGZE1GYRzHodBSWAfxcFJajPeiVKKwAD7I4CitRDG6IwmoYw7rr/1kmD09GYS0a
w1+LwjGohgjQO+aUkLpMVkVhjFJoTBQmKIbmRGGKimhFFOagzGgU5lESvTcKywB+KgrL0af0jSgM
Hh43H4WVKIm7EoXVqIz7PApr0Hq+KApr0b/yJ6JwDLpDNlo3Nr57YmRoeMqWuSnLVpifX2prGei3
BfqmcmwNo5vybMu2brVJBSZtEwOTAxM7BvrzbE0Ntb6WZa0NzatsI5O2PtvURF//wLa+iS22scFb
6zeNbByY6JsaGRu1rRwbHWsZGJre2jexbHLTwGj/wIQt13ZbgduSawcmJhlckJdfmld04+VtRf9v
BgEjHxqZnBqYgMyRUVtb3po8W7BvamB0ytY32m9rvV6xeXBwZNOAlLlpYGKqDwqPTQ3DODdPT4xM
9o9sYr1N5l0fft3YxPhYZAxrpgZ2DNhW9k1NDUyOjQ5PTY1XuN07d+7M64sW3gRl8zaNbXP/vXdT
u8cH+gcmR4ZGYeJ5w1PbtjbBgEYnYeDTUo8wmptR5h8bhYXZGimTY5scGLCx5ieh/cGBfhja+MTY
5oFNU3ljE0PunSNbRtyR9kZGh9w3mmGtRPv5X6uN6oD7xtFukEkjaAgNg9SxoUyQaVnwLET58FcK
UAsaQP3wDKA+KJEDUAMahVJ5AC1DW+HPdlMLk1JqAJ4D8Nwh1WUlm6BWLcjIFqjTCnAzYr8AHpHK
98E1BaX7oOwA2gbPCbQF8sZA/v29/pug/kapH/aG/Z+eUXi7UnqOSfWGQIZuldpbBj1tgpxRqY8J
KJcrjervtfD3366V3kxezy+AETGM5aGi/7bm32/1fw0TEZwPSa1MSW1HSo5IbbdBiTVSqaBUk2Fh
SuptVCrV+t/02Aw9DkJ9hrMbJTdJbU9BOtLyGMDDUXxuBlxPSCPol+otzW0Sev4i9hntTQD1jd2C
hzXS6HZIfa6U8qckWmLvhqXUOKoATeNGO6W/PChza8ubou3mSdA2KPn/tt4UcMa4hMcBaZWHoGxk
xfOkNrcBZTVFMTQq0TvD0PRNc4zg5m9RmV96Rjhm6y3tsJVlT1Z3afST0fEPSv1EsDYO9zHA+4CE
7Twpd0ia4wis4QhAN4+PrdhQNO/20SyN5db5/P/ZN40YDmEHUOft1hp8xH4l5Rbwn4KZ1j/6M63/
7s+2XvMXW4/94ZE/PPsHuuWTY5+QVz7BT3yCLZ90fbLlE8p9XP8xUX3kX7T+z9+mWz/8bbX1336b
Yv3Nr/1W9a+x+Gt/nPVXV/zWV6784MovrlDxiqfEf8Vvtr6KY1ENNqA2bBQ11bTtg+pftv1r9ftt
ryxT4jgYVivc++Eii5dx3POaOL9tHsddwAS9CpkCe4FNF96vts5jvbiJLlqFX9p+Kf4y+MuZX4Z+
KbO9H3x/5v3Q+5zuffyeyWPteXPszX1v0p7Xx17f9zq99F38nWC6dfw1bHst/7XLr9Hx12ZeI7pX
ra8S96veV8dePffqB6/yF59Lt9rm8+eD8+PzM/O8dBZq3pjlF17CtpeCL828FHqJm3kx9CLRPe99
/pPn6TzWiq5nAtaZ0IkQCYUuh94LUfc57zly5rnQc+Tyc+89R9xnvWfJ6Wfx5Wfee4Ys02EdKsQx
IEIQ3AW4bHCNwwVGGhZEEw6e6j01fop++cF068P+dGv+Q+JDhO2ZPBiX5GfjyX4wRu//B8uiVXfy
3MlLJ6l4MrnAL56Ms8BNo/PrHnA/4H1g3wOfPMDrXsYasMw0oo186b506/1rFq0fnMD5J7D1hPsE
GTux7wRBc8KcbY6ydm1z5iS/7Xj+cdJ8X899Y/fR/Hux7l7rve57qXivYPQLl7AaBqtG+XBRWCD1
hXib/yIDxKAQ6z+2P916dEWV9cjhauvhA1XWQysWracPYuGA7UD+AZp/D963H4v7lRr/JCzDGNDW
KFyJ2NyW4DG3yT20TQYL2gvveuC6uHgFyy9Y0/0SIFqNSf4NnQHren+BtQuenfA0FhraeEzbuELa
No8VL1qqrDqKL4K3ab5QbBXn4RGf6Z/HKtEJDbYELdZPVi+uJuLq4jK/uNqZ6X83iD9owk3+ZGuj
P2ANzmOL2IlXAMobYGABuJbDdc6PP/B/4iczfhxXaGrTY12bUKhrIxhoGGGrVefV9ej26Tidzq1r
1o3p5nQf6BZ1ci/kfaKjYIw3I3wmDvN4Hp8437rG5Wqcly+2NIaUwa4QPhJyrmF3cXVnSHYkhNo6
u9rPY3y84+B996Ha5MZQ4Zr2UG9yR2OoHwCRATMACMnn41Btx+TU5NS0i31wFJhELtfUFDylhPQG
LuRa+mCWwK7JqanJaA7UgNSUa1q6uyYnpYpYqo2gA2h6EiQqVJhyTYK9zx5Qg3UINTGIM6gDZSd7
JqWepT7guWFS6hYDgKDA1GRkIJPmng0I/V8XDYmuCmVuZHN0cmVhbQplbmRvYmoKCjE2MDEgMCBv
YmoKMTQ5MzcKZW5kb2JqCgoxNjAyIDAgb2JqCjw8L1R5cGUvRm9udERlc2NyaXB0b3IvRm9udE5h
bWUvQ0FBQUFBK0xpYmVyYXRpb25Nb25vCi9GbGFncyA1Ci9Gb250QkJveFstMjQgLTMwMCA2MDgg
ODMzXS9JdGFsaWNBbmdsZSAwCi9Bc2NlbnQgODMyCi9EZXNjZW50IC0zMDAKL0NhcEhlaWdodCA4
MzIKL1N0ZW1WIDgwCi9Gb250RmlsZTIgMTYwMCAwIFIKPj4KZW5kb2JqCgoxNjAzIDAgb2JqCjw8
L0xlbmd0aCA1MzMvRmlsdGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicXZTNjpswFIX3PAXL6WIE
+Bo8I0WRMslEyqI/aqYPQMBJkRpAhCzy9uXcc9tKXQR9wPX15xObbHvYHfpuzr5NQ3OMc3ru+naK
t+E+NTE9xUvXJ4VL266Z7U6vzbUek2wZe3zc5ng99OdhtUqy78u72zw90qdNO5zipyT7OrVx6vpL
+vRje1zuj/dx/BWvsZ/TPFmv0zaelz6f6/FLfY2Zjno+tMvrbn48L0P+FXw8xpg6vS+o0gxtvI11
E6e6v8RklefrdLXfr5PYt/+9C45DTufmZz0tpcVSmuelrBd2ytUWLORXsCd7cEkuwZVy0LFBWTbg
F2WXg1+VvdZsyA78xnm1/5b1O/CONQX4naw998p7uBU550Wfgv5VAJv/O5j+Xp+b/x5M/52y+WtP
+nv4FPSv4FPQv4JPQf8SORT0r97A9C/Vh/5Ba+gfKjD9PZ4784ebs/xfwPT38HH0D8jQ0T8gc2f5
Yy5H/wrZOvoH+DvLX3uavzL9g7Lljxwc/UXd6O+QoaO/YO2O/oKxQn+HuYT+Dn2E/oJ1ie0frF3o
7+Ev9PfIVujvsF6hf4n1Cv29zkX/EntALH/tQ3/RevNHJmL++F/E/LXe/JGVp7/A39Nf4Oktf2Xb
P8jB01/g7Okf4ONt/yAfT/9Kn5s/D6CdNBxFfCv+HPG0uU/Tcrz1g6LnGie66+Pfb844jBilv997
+xRFCmVuZHN0cmVhbQplbmRvYmoKCjE2MDQgMCBvYmoKPDwvVHlwZS9Gb250L1N1YnR5cGUvVHJ1
ZVR5cGUvQmFzZUZvbnQvQ0FBQUFBK0xpYmVyYXRpb25Nb25vCi9GaXJzdENoYXIgMAovTGFzdENo
YXIgNzIKL1dpZHRoc1swIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2
MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYw
MCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAw
IDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAg
NjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2
MDAgNjAwIDYwMCA2MDAgNjAwIF0KL0ZvbnREZXNjcmlwdG9yIDE2MDIgMCBSCi9Ub1VuaWNvZGUg
MTYwMyAwIFIKPj4KZW5kb2JqCgoxNjA1IDAgb2JqCjw8L0xlbmd0aCAxNjA2IDAgUi9GaWx0ZXIv
RmxhdGVEZWNvZGUvTGVuZ3RoMSAzNDA0Pj4Kc3RyZWFtCnic5VbdbyNXFT/XX9l6873pksplucPs
tls8TmyHXYiUdr1JnI2TZmPspBqXCDqxJ/Z07Rl3xk6TQGmRoBSrrVSBUKVW3WSLhBBquQlIrHja
qvBU9gFUZRECFaRFINR9oBIPQJrl3Otxkt3mPyDO3Ps7v3s+7jn3Y6Zm13Voh+fAC4l8Rav2EeIB
gN8AkN78co0uHH/Zj/jPyKWXqsXK+6e3/gbgieKjF8urS5ff+OmrAD584GZJ1wrvrK3IAP6vo3y2
hMTU7rUAyj9H+WSpUluh8Mt2lH+PcnvZymtvwAJC/01sjlS0lWqf9wRB+UOUqalV9HPh8zcAAmgT
uFW1nFoffOM2mu7w8aqtV/vDH70F0HE/ygP4EPzxP9QnAS574P/+7/vwCnwX3oIUXIEcDMLnQYE4
PAFfAhnGYRQkeAd+Db+Fd+GH8Dz8AL4Jr8E6MPgRJOBZ+BZ5He7zbvvP+X8MX/b3MFAYHJtmn0ur
bGo5x0A+188CYfXhnOCeydH3GTk20B9hRKF/YO3hCPMo0xk1KeekCPMqRj9libQqsUQuwnwKN5Vk
aU39U+h6LoR66sehW7mQLDF/WGUTyzkxkMuhP7/SsfB4hAWUzc+SFzA6fWFhIcQA3bQpmycFldij
jii9PXR4MMLuUegzPMiv0A1l3lMpmTLfA1MM0mpDb2iUgy+GJCkXaggp05R4wGBzdt2hbgk9HlXo
70Q67QodZG3hBZXSC/KE9iRVaWGx6YLrdfDIGJo26IXGhCY3aEMW4WTunCVQE/PjBEvoXECbThHp
4e1+SQrR7QaWAY1SOJt5d26SUOtSZLrtBpepOp0NSYzk1AYmlJIbMm2kGrLGDZomvIuwbr4MvTjv
Hp4AB713JdDgnaw9+cTBTLjpMQWTaHyHl22qIDfaGE2rI6FrONKn/AwSJDE6SqavdkMeRMuV51Xe
ZlR5EWcvj4awI/IoVj6RUbeAwlh+dItQgh2jeXaffn8r1r0KQxbrgk2E71oP7k3wFPzzeDO1wcAm
gcGRrTZf1634ZsD/x5EtrwchbHo57ef0VlvgwZ2RLcL5oR6p55TUI4176O5J8upuyT//n5+M+64D
vyE+wPvlWd86dMAp2EImzNqut3rCOgeZb5sdvY7/m10kDNGYRB98oPsLZyX6qePdbQFvZPefl9fX
L5Mu0vHmlStvbqx7TqxvbGx8fHNjg8+b3N4h7/ksz1dw3r14igaZF736Bjf9wplXOiP5fM5/XyLv
fU/cVUsnri6HHv/HV7tG/gWfOSIO7LVCzy9ah/f2Dp8tVgHvR2hdZvyGfHF3ggztnXFy15n3ej6E
8cACfOCzuQc88QtgCy0vdLp+PCh74Dg39hL35uyEt1u+yI09vwSCKBHXqo38xcVe5P/uYh/ij1zs
hy6y4+IABD1HUZP47kGDtz33uphAn/9dF3ug03/DxV7k/+piH+J/u9gPJwIdLg5AXyB8Ov8QjUej
wzRbN+mMkbctZ9Wp6RWHpsz8QHBuMplJ0vHZZJZenJ2jSTWVnaNNm1iMTtXLhm7Si9qiXgumM8mx
5DgqjkQe2bfIzo+NJZPj+zazZWPZ0G06qZXLFrdKzSSFyVwmlX40SZuEqz5EZ7RaydAcVHccvVzR
TDNYLRnCAPtYS/EMnS5pZb1AJy3HXA267FnM6JJOp7WiZjqXjOCablsKtUxdobWnEdVKto54yarb
2BrLiB1jBRt9WTcVqhvFUk2hpsENnqrrTs2wkK6bBd128paNbN6qrtpNNVsvGlg5Wy9wFSMej51R
qGYWyzxaCfklYVG21nSzqLcmHqUXLAt16JhlVy1b4yGC3Dw6vpfcMJ3TLukOpn1+UeeDE9GhqChB
Ew+7ivEo5msW6WRdm63qZna1smiVM3qxXtbsfWIfPYZpYDgai8YHYnG6PwCn8TZ6CC+aOETxN4wo
C3UwsZ8BA8dssMCBVXxqoEMFe4ovRhNHBiAIczAJScjgQ/H6mcU+i+giojnsk6Ciblbgg3Fi+KMw
hXHKGEMX0S6CBouIa+g1LTyO4TPuehyBCDxyaIwszKMm1+Xah8WZFVGWRSQb5UmMVMaftRcrhbkm
D0SZE1waHhXsQY07vQ+JKmnop4TeNVGbSdE7GKuM1dIwNxOjVIXGfoSmHPuExzOIpnGMz1CHgvDH
62/iCgTv0j3rrtEl1ORWGhRFPAcZA7XXRL4WXmUUWxMljmrwtMvxWdvINvklZOsoNzGvV5N3EK+4
SBesKSQd+SJ6qAnJRKkV4Sn0o4v9Yoi4nOM7qiDm42AOlojLeY6rmJt9hzc+WkSmuedsUYmWFwOz
j4tKcYbnWxS1amXU1F86EIOv9JqYd1FU6s4qRhFdQA3L9YOvXGFbFa22l0VwL3oU99knV25Y7BtN
rIbjrvZ5saNblhOoNyTitXbBQX74Lo9xoTkj6loU+6CO3mdxXjyTLNasgt4tnHNGVIufJQ1nfJjG
Ydxj7mo0s6OiEnE80zFs6aEW4v3VfMd+DfoO+3C+Sm5/m5GXYJodSaubhLyc25zgX4SsGz92+zII
nst9Gr/cFtQc6wsD/A8vwrHMCmVuZHN0cmVhbQplbmRvYmoKCjE2MDYgMCBvYmoKMTkyMgplbmRv
YmoKCjE2MDcgMCBvYmoKPDwvVHlwZS9Gb250RGVzY3JpcHRvci9Gb250TmFtZS9FQUFBQUErT3Bl
blN5bWJvbAovRmxhZ3MgNAovRm9udEJCb3hbLTE3OSAtMzEyIDEwODMgOTI3XS9JdGFsaWNBbmds
ZSAwCi9Bc2NlbnQgOTI2Ci9EZXNjZW50IC0zMTIKL0NhcEhlaWdodCA5MjYKL1N0ZW1WIDgwCi9G
b250RmlsZTIgMTYwNSAwIFIKPj4KZW5kb2JqCgoxNjA4IDAgb2JqCjw8L0xlbmd0aCAyMzAvRmls
dGVyL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnicXZBBa8QgEIXv/gqPu4dFY+lNAiXLQg7blqb9AUYn
qdCoTMwh/76ju22hB8XHe9/wHNH15z74LF4x2gEyn3xwCGvc0AIfYfaBNYo7b/Nd1dsuJjFB7LCv
GZY+TFFrJt7IWzPu/PDk4ghHJl7QAfow88NHN5AetpS+YIGQuWRtyx1MNOdq0rNZQFTq1Duyfd5P
hPwF3vcEXFXd3KrY6GBNxgKaMAPTUrZcXy4tg+D+eepGjJP9NEjJhpLqsaOslqq8ZfNQuXuiTChf
/GnG7YZIreoeap1SxAf4XVWKqVD1fAMs5m/qCmVuZHN0cmVhbQplbmRvYmoKCjE2MDkgMCBvYmoK
PDwvVHlwZS9Gb250L1N1YnR5cGUvVHJ1ZVR5cGUvQmFzZUZvbnQvRUFBQUFBK09wZW5TeW1ib2wK
L0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyCi9XaWR0aHNbMCA3OTQgNTU1IF0KL0ZvbnREZXNjcmlw
dG9yIDE2MDcgMCBSCi9Ub1VuaWNvZGUgMTYwOCAwIFIKPj4KZW5kb2JqCgoxNjEwIDAgb2JqCjw8
L0YxIDE1ODQgMCBSL0YyIDE2MDQgMCBSL0YzIDE1NzkgMCBSL0Y0IDE2MDkgMCBSL0Y1IDE1OTQg
MCBSL0Y2IDE1OTkgMCBSL0Y3IDE1ODkgMCBSCj4+CmVuZG9iagoKMTYxMSAwIG9iago8PAovRm9u
dCAxNjEwIDAgUgovWE9iamVjdDw8L0ltNCA0IDAgUi9JbTUwNyA1MDcgMCBSPj4KL1Byb2NTZXRb
L1BERi9UZXh0L0ltYWdlQy9JbWFnZUkvSW1hZ2VCXQo+PgplbmRvYmoKCjEgMCBvYmoKPDwvVHlw
ZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcy
MCA0MDUuMDcwODY2MTQxNzMyXS9Bbm5vdHNbCjE1NjIgMCBSIDE1NjMgMCBSIDE1NjQgMCBSIDE1
NjUgMCBSIF0KL1RhYnMvUwovU3RydWN0UGFyZW50cyAwCi9Db250ZW50cyAyIDAgUj4+CmVuZG9i
agoKMTUgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAw
IFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDEKL0Nv
bnRlbnRzIDE2IDAgUj4+CmVuZG9iagoKNDkgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0
IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMy
XS9TdHJ1Y3RQYXJlbnRzIDIKL0NvbnRlbnRzIDUwIDAgUj4+CmVuZG9iagoKNTQgMCBvYmoKPDwv
VHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAw
IDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDMKL0NvbnRlbnRzIDU1IDAgUj4+
CmVuZG9iagoKNzMgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMg
MTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRz
IDQKL0NvbnRlbnRzIDc0IDAgUj4+CmVuZG9iagoKMTA1IDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJl
bnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2
NjE0MTczMl0vU3RydWN0UGFyZW50cyA1Ci9Db250ZW50cyAxMDYgMCBSPj4KZW5kb2JqCgoxMzgg
MCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVk
aWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDYKL0NvbnRlbnRz
IDEzOSAwIFI+PgplbmRvYmoKCjE2NiAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBS
L1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0
cnVjdFBhcmVudHMgNwovQ29udGVudHMgMTY3IDAgUj4+CmVuZG9iagoKMTg4IDAgb2JqCjw8L1R5
cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3
MjAgNDA1LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA4Ci9Db250ZW50cyAxODkgMCBSPj4K
ZW5kb2JqCgoyMDcgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMg
MTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRz
IDkKL0NvbnRlbnRzIDIwOCAwIFI+PgplbmRvYmoKCjIyNiAwIG9iago8PC9UeXBlL1BhZ2UvUGFy
ZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4
NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTAKL0NvbnRlbnRzIDIyNyAwIFI+PgplbmRvYmoKCjIz
MSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9N
ZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTEKL0NvbnRl
bnRzIDIzMiAwIFI+PgplbmRvYmoKCjI2MiAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQg
MCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJd
L1N0cnVjdFBhcmVudHMgMTIKL0NvbnRlbnRzIDI2MyAwIFI+PgplbmRvYmoKCjI5MyAwIG9iago8
PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFsw
IDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTMKL0NvbnRlbnRzIDI5NCAw
IFI+PgplbmRvYmoKCjMzMyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291
cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBh
cmVudHMgMTQKL0NvbnRlbnRzIDMzNCAwIFI+PgplbmRvYmoKCjM2NSAwIG9iago8PC9UeXBlL1Bh
Z2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQw
NS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTUKL0NvbnRlbnRzIDM2NiAwIFI+PgplbmRv
YmoKCjM4MSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjEx
IDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTYK
L0NvbnRlbnRzIDM4MiAwIFI+PgplbmRvYmoKCjQxMyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50
IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYx
NDE3MzJdL1N0cnVjdFBhcmVudHMgMTcKL0NvbnRlbnRzIDQxNCAwIFI+PgplbmRvYmoKCjQzOCAw
IG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRp
YUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMTgKL0NvbnRlbnRz
IDQzOSAwIFI+PgplbmRvYmoKCjQ3NiAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBS
L1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0
cnVjdFBhcmVudHMgMTkKL0NvbnRlbnRzIDQ3NyAwIFI+PgplbmRvYmoKCjUwMSAwIG9iago8PC9U
eXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAg
NzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjAKL0NvbnRlbnRzIDUwMiAwIFI+
PgplbmRvYmoKCjUyMCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNl
cyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVu
dHMgMjEKL0NvbnRlbnRzIDUyMSAwIFI+PgplbmRvYmoKCjUzOSAwIG9iago8PC9UeXBlL1BhZ2Uv
UGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4w
NzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjIKL0NvbnRlbnRzIDU0MCAwIFI+PgplbmRvYmoK
CjU0NCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAg
Ui9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjMKL0Nv
bnRlbnRzIDU0NSAwIFI+PgplbmRvYmoKCjU2MyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1
NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3
MzJdL1N0cnVjdFBhcmVudHMgMjQKL0NvbnRlbnRzIDU2NCAwIFI+PgplbmRvYmoKCjU4MiAwIG9i
ago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJv
eFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjUKL0NvbnRlbnRzIDU4
MyAwIFI+PgplbmRvYmoKCjYwMiAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jl
c291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVj
dFBhcmVudHMgMjYKL0NvbnRlbnRzIDYwMyAwIFI+PgplbmRvYmoKCjYyMSAwIG9iago8PC9UeXBl
L1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIw
IDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjcKL0NvbnRlbnRzIDYyMiAwIFI+Pgpl
bmRvYmoKCjYzNyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAx
NjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMg
MjgKL0NvbnRlbnRzIDYzOCAwIFI+PgplbmRvYmoKCjY0MiAwIG9iago8PC9UeXBlL1BhZ2UvUGFy
ZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4
NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMjkKL0NvbnRlbnRzIDY0MyAwIFI+PgplbmRvYmoKCjY2
NyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9N
ZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzAKL0NvbnRl
bnRzIDY2OCAwIFI+PgplbmRvYmoKCjY5MyAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQg
MCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJd
L1N0cnVjdFBhcmVudHMgMzEKL0NvbnRlbnRzIDY5NCAwIFI+PgplbmRvYmoKCjcxOSAwIG9iago8
PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFsw
IDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzIKL0NvbnRlbnRzIDcyMCAw
IFI+PgplbmRvYmoKCjczOCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291
cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBh
cmVudHMgMzMKL0NvbnRlbnRzIDczOSAwIFI+PgplbmRvYmoKCjc2NCAwIG9iago8PC9UeXBlL1Bh
Z2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQw
NS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzQKL0NvbnRlbnRzIDc2NSAwIFI+PgplbmRv
YmoKCjc4OSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjEx
IDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzUK
L0NvbnRlbnRzIDc5MCAwIFI+PgplbmRvYmoKCjgwOCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50
IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYx
NDE3MzJdL1N0cnVjdFBhcmVudHMgMzYKL0NvbnRlbnRzIDgwOSAwIFI+PgplbmRvYmoKCjgzMCAw
IG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRp
YUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzcKL0NvbnRlbnRz
IDgzMSAwIFI+PgplbmRvYmoKCjg0OSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBS
L1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0
cnVjdFBhcmVudHMgMzgKL0NvbnRlbnRzIDg1MCAwIFI+PgplbmRvYmoKCjg2OCAwIG9iago8PC9U
eXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAg
NzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgMzkKL0NvbnRlbnRzIDg2OSAwIFI+
PgplbmRvYmoKCjg5NCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNl
cyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVu
dHMgNDAKL0NvbnRlbnRzIDg5NSAwIFI+PgplbmRvYmoKCjkyMyAwIG9iago8PC9UeXBlL1BhZ2Uv
UGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4w
NzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNDEKL0NvbnRlbnRzIDkyNCAwIFI+PgplbmRvYmoK
CjkyOCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAg
Ui9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNDIKL0Nv
bnRlbnRzIDkyOSAwIFI+PgplbmRvYmoKCjk1MCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1
NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3
MzJdL1N0cnVjdFBhcmVudHMgNDMKL0NvbnRlbnRzIDk1MSAwIFI+PgplbmRvYmoKCjk3OSAwIG9i
ago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJv
eFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNDQKL0NvbnRlbnRzIDk4
MCAwIFI+PgplbmRvYmoKCjk5OCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jl
c291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVj
dFBhcmVudHMgNDUKL0NvbnRlbnRzIDk5OSAwIFI+PgplbmRvYmoKCjEwMTcgMCBvYmoKPDwvVHlw
ZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcy
MCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDQ2Ci9Db250ZW50cyAxMDE4IDAgUj4+
CmVuZG9iagoKMTAzOSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNl
cyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVu
dHMgNDcKL0NvbnRlbnRzIDEwNDAgMCBSPj4KZW5kb2JqCgoxMDU1IDAgb2JqCjw8L1R5cGUvUGFn
ZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1
LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA0OAovQ29udGVudHMgMTA1NiAwIFI+PgplbmRv
YmoKCjEwNjAgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYx
MSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDQ5
Ci9Db250ZW50cyAxMDYxIDAgUj4+CmVuZG9iagoKMTA3OSAwIG9iago8PC9UeXBlL1BhZ2UvUGFy
ZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4
NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNTAKL0NvbnRlbnRzIDEwODAgMCBSPj4KZW5kb2JqCgox
MTA0IDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBS
L01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA1MQovQ29u
dGVudHMgMTEwNSAwIFI+PgplbmRvYmoKCjExMzIgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAx
NTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQx
NzMyXS9TdHJ1Y3RQYXJlbnRzIDUyCi9Db250ZW50cyAxMTMzIDAgUj4+CmVuZG9iagoKMTE1MSAw
IG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRp
YUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNTMKL0NvbnRlbnRz
IDExNTIgMCBSPj4KZW5kb2JqCgoxMTk3IDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAw
IFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0v
U3RydWN0UGFyZW50cyA1NAovQ29udGVudHMgMTE5OCAwIFI+PgplbmRvYmoKCjEyMzkgMCBvYmoK
PDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hb
MCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDU1Ci9Db250ZW50cyAxMjQw
IDAgUj4+CmVuZG9iagoKMTI4OCAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jl
c291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVj
dFBhcmVudHMgNTYKL0NvbnRlbnRzIDEyODkgMCBSPj4KZW5kb2JqCgoxMzA3IDAgb2JqCjw8L1R5
cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3
MjAgNDA1LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA1NwovQ29udGVudHMgMTMwOCAwIFI+
PgplbmRvYmoKCjEzMTIgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJj
ZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJl
bnRzIDU4Ci9Db250ZW50cyAxMzEzIDAgUj4+CmVuZG9iagoKMTMzNyAwIG9iago8PC9UeXBlL1Bh
Z2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQw
NS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNTkKL0NvbnRlbnRzIDEzMzggMCBSPj4KZW5k
b2JqCgoxMzY2IDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2
MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA2
MAovQ29udGVudHMgMTM2NyAwIFI+PgplbmRvYmoKCjEzODUgMCBvYmoKPDwvVHlwZS9QYWdlL1Bh
cmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcw
ODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDYxCi9Db250ZW50cyAxMzg2IDAgUj4+CmVuZG9iagoK
MTQxMSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAg
Ui9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNjIKL0Nv
bnRlbnRzIDE0MTIgMCBSPj4KZW5kb2JqCgoxNDMwIDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQg
MTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0
MTczMl0vU3RydWN0UGFyZW50cyA2MwovQ29udGVudHMgMTQzMSAwIFI+PgplbmRvYmoKCjE0NDYg
MCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9SZXNvdXJjZXMgMTYxMSAwIFIvTWVk
aWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1Y3RQYXJlbnRzIDY0Ci9Db250ZW50
cyAxNDQ3IDAgUj4+CmVuZG9iagoKMTQ2NSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDE1NzQg
MCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAgNzIwIDQwNS4wNzA4NjYxNDE3MzJd
L1N0cnVjdFBhcmVudHMgNjUKL0NvbnRlbnRzIDE0NjYgMCBSPj4KZW5kb2JqCgoxNDcwIDAgb2Jq
Cjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94
WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0vU3RydWN0UGFyZW50cyA2NgovQ29udGVudHMgMTQ3
MSAwIFI+PgplbmRvYmoKCjE0ODYgMCBvYmoKPDwvVHlwZS9QYWdlL1BhcmVudCAxNTc0IDAgUi9S
ZXNvdXJjZXMgMTYxMSAwIFIvTWVkaWFCb3hbMCAwIDcyMCA0MDUuMDcwODY2MTQxNzMyXS9TdHJ1
Y3RQYXJlbnRzIDY3Ci9Db250ZW50cyAxNDg3IDAgUj4+CmVuZG9iagoKMTQ5MSAwIG9iago8PC9U
eXBlL1BhZ2UvUGFyZW50IDE1NzQgMCBSL1Jlc291cmNlcyAxNjExIDAgUi9NZWRpYUJveFswIDAg
NzIwIDQwNS4wNzA4NjYxNDE3MzJdL1N0cnVjdFBhcmVudHMgNjgKL0NvbnRlbnRzIDE0OTIgMCBS
Pj4KZW5kb2JqCgoxNTA0IDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIvUmVzb3Vy
Y2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0vQW5ub3RzWwox
NTY2IDAgUiAxNTY3IDAgUiBdCi9UYWJzL1MKL1N0cnVjdFBhcmVudHMgNjkKL0NvbnRlbnRzIDE1
MDUgMCBSPj4KZW5kb2JqCgoxNTIzIDAgb2JqCjw8L1R5cGUvUGFnZS9QYXJlbnQgMTU3NCAwIFIv
UmVzb3VyY2VzIDE2MTEgMCBSL01lZGlhQm94WzAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMl0vQW5u
b3RzWwoxNTY4IDAgUiAxNTY5IDAgUiAxNTcwIDAgUiAxNTcxIDAgUiAxNTcyIDAgUiAxNTczIDAg
UiBdCi9UYWJzL1MKL1N0cnVjdFBhcmVudHMgNzAKL0NvbnRlbnRzIDE1MjQgMCBSPj4KZW5kb2Jq
CgoxNjEyIDAgb2JqCjw8L0NvdW50IDcxL0ZpcnN0IDE2MTMgMCBSL0xhc3QgMTY4MyAwIFIKPj4K
ZW5kb2JqCgoxNjEzIDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMwMDZDMDA2OTAwNjQw
MDY1MDAyMDAwMzE+Ci9EZXN0WzEgMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIv
TmV4dCAxNjE0IDAgUj4+CmVuZG9iagoKMTYxNCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYw
MDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyPgovRGVzdFsxNSAwIFIvWFlaIDAgNDA1LjA3IDBd
L1BhcmVudCAxNjEyIDAgUi9QcmV2IDE2MTMgMCBSL05leHQgMTYxNSAwIFI+PgplbmRvYmoKCjE2
MTUgMCBvYmoKPDwvQ291bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAz
Mz4KL0Rlc3RbNDkgMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjE0
IDAgUi9OZXh0IDE2MTYgMCBSPj4KZW5kb2JqCgoxNjE2IDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8
RkVGRjAwNTMwMDZDMDA2OTAwNjQwMDY1MDAyMDAwMzQ+Ci9EZXN0WzU0IDAgUi9YWVogMCA0MDUu
MDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYxNSAwIFIvTmV4dCAxNjE3IDAgUj4+CmVuZG9i
agoKMTYxNyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAw
MjAwMDM1PgovRGVzdFs3MyAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2
IDE2MTYgMCBSL05leHQgMTYxOCAwIFI+PgplbmRvYmoKCjE2MTggMCBvYmoKPDwvQ291bnQgMC9U
aXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNj4KL0Rlc3RbMTA1IDAgUi9YWVog
MCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYxNyAwIFIvTmV4dCAxNjE5IDAgUj4+
CmVuZG9iagoKMTYxOSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0
MDA2NTAwMjAwMDM3PgovRGVzdFsxMzggMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAw
IFIvUHJldiAxNjE4IDAgUi9OZXh0IDE2MjAgMCBSPj4KZW5kb2JqCgoxNjIwIDAgb2JqCjw8L0Nv
dW50IDAvVGl0bGU8RkVGRjAwNTMwMDZDMDA2OTAwNjQwMDY1MDAyMDAwMzg+Ci9EZXN0WzE2NiAw
IFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2IDE2MTkgMCBSL05leHQgMTYy
MSAwIFI+PgplbmRvYmoKCjE2MjEgMCBvYmoKPDwvQ291bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMw
MDY5MDA2NDAwNjUwMDIwMDAzOT4KL0Rlc3RbMTg4IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50
IDE2MTIgMCBSL1ByZXYgMTYyMCAwIFIvTmV4dCAxNjIyIDAgUj4+CmVuZG9iagoKMTYyMiAwIG9i
ago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzMD4K
L0Rlc3RbMjA3IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYyMSAw
IFIvTmV4dCAxNjIzIDAgUj4+CmVuZG9iagoKMTYyMyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZF
RkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzMT4KL0Rlc3RbMjI2IDAgUi9YWVogMCA0
MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYyMiAwIFIvTmV4dCAxNjI0IDAgUj4+CmVu
ZG9iagoKMTYyNCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2
NTAwMjAwMDMxMDAzMj4KL0Rlc3RbMjMxIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIg
MCBSL1ByZXYgMTYyMyAwIFIvTmV4dCAxNjI1IDAgUj4+CmVuZG9iagoKMTYyNSAwIG9iago8PC9D
b3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzMz4KL0Rlc3Rb
MjYyIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYyNCAwIFIvTmV4
dCAxNjI2IDAgUj4+CmVuZG9iagoKMTYyNiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUz
MDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzND4KL0Rlc3RbMjkzIDAgUi9YWVogMCA0MDUuMDcg
MF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYyNSAwIFIvTmV4dCAxNjI3IDAgUj4+CmVuZG9iagoK
MTYyNyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAw
MDMxMDAzNT4KL0Rlc3RbMzMzIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1By
ZXYgMTYyNiAwIFIvTmV4dCAxNjI4IDAgUj4+CmVuZG9iagoKMTYyOCAwIG9iago8PC9Db3VudCAw
L1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzNj4KL0Rlc3RbMzY1IDAg
Ui9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYyNyAwIFIvTmV4dCAxNjI5
IDAgUj4+CmVuZG9iagoKMTYyOSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAw
NjkwMDY0MDA2NTAwMjAwMDMxMDAzNz4KL0Rlc3RbMzgxIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFy
ZW50IDE2MTIgMCBSL1ByZXYgMTYyOCAwIFIvTmV4dCAxNjMwIDAgUj4+CmVuZG9iagoKMTYzMCAw
IG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAz
OD4KL0Rlc3RbNDEzIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYy
OSAwIFIvTmV4dCAxNjMxIDAgUj4+CmVuZG9iagoKMTYzMSAwIG9iago8PC9Db3VudCAwL1RpdGxl
PEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMxMDAzOT4KL0Rlc3RbNDM4IDAgUi9YWVog
MCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzMCAwIFIvTmV4dCAxNjMyIDAgUj4+
CmVuZG9iagoKMTYzMiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0
MDA2NTAwMjAwMDMyMDAzMD4KL0Rlc3RbNDc2IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2
MTIgMCBSL1ByZXYgMTYzMSAwIFIvTmV4dCAxNjMzIDAgUj4+CmVuZG9iagoKMTYzMyAwIG9iago8
PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzMT4KL0Rl
c3RbNTAxIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzMiAwIFIv
TmV4dCAxNjM0IDAgUj4+CmVuZG9iagoKMTYzNCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYw
MDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzMj4KL0Rlc3RbNTIwIDAgUi9YWVogMCA0MDUu
MDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzMyAwIFIvTmV4dCAxNjM1IDAgUj4+CmVuZG9i
agoKMTYzNSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAw
MjAwMDMyMDAzMz4KL0Rlc3RbNTM5IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBS
L1ByZXYgMTYzNCAwIFIvTmV4dCAxNjM2IDAgUj4+CmVuZG9iagoKMTYzNiAwIG9iago8PC9Db3Vu
dCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzND4KL0Rlc3RbNTQ0
IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzNSAwIFIvTmV4dCAx
NjM3IDAgUj4+CmVuZG9iagoKMTYzNyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2
QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzNT4KL0Rlc3RbNTYzIDAgUi9YWVogMCA0MDUuMDcgMF0v
UGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzNiAwIFIvTmV4dCAxNjM4IDAgUj4+CmVuZG9iagoKMTYz
OCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMy
MDAzNj4KL0Rlc3RbNTgyIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYg
MTYzNyAwIFIvTmV4dCAxNjM5IDAgUj4+CmVuZG9iagoKMTYzOSAwIG9iago8PC9Db3VudCAwL1Rp
dGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzNz4KL0Rlc3RbNjAyIDAgUi9Y
WVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTYzOCAwIFIvTmV4dCAxNjQwIDAg
Uj4+CmVuZG9iagoKMTY0MCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkw
MDY0MDA2NTAwMjAwMDMyMDAzOD4KL0Rlc3RbNjIxIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50
IDE2MTIgMCBSL1ByZXYgMTYzOSAwIFIvTmV4dCAxNjQxIDAgUj4+CmVuZG9iagoKMTY0MSAwIG9i
ago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMyMDAzOT4K
L0Rlc3RbNjM3IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0MCAw
IFIvTmV4dCAxNjQyIDAgUj4+CmVuZG9iagoKMTY0MiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZF
RkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAzMD4KL0Rlc3RbNjQyIDAgUi9YWVogMCA0
MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0MSAwIFIvTmV4dCAxNjQzIDAgUj4+CmVu
ZG9iagoKMTY0MyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2
NTAwMjAwMDMzMDAzMT4KL0Rlc3RbNjY3IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIg
MCBSL1ByZXYgMTY0MiAwIFIvTmV4dCAxNjQ0IDAgUj4+CmVuZG9iagoKMTY0NCAwIG9iago8PC9D
b3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAzMj4KL0Rlc3Rb
NjkzIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0MyAwIFIvTmV4
dCAxNjQ1IDAgUj4+CmVuZG9iagoKMTY0NSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUz
MDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAzMz4KL0Rlc3RbNzE5IDAgUi9YWVogMCA0MDUuMDcg
MF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0NCAwIFIvTmV4dCAxNjQ2IDAgUj4+CmVuZG9iagoK
MTY0NiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAw
MDMzMDAzND4KL0Rlc3RbNzM4IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1By
ZXYgMTY0NSAwIFIvTmV4dCAxNjQ3IDAgUj4+CmVuZG9iagoKMTY0NyAwIG9iago8PC9Db3VudCAw
L1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAzNT4KL0Rlc3RbNzY0IDAg
Ui9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0NiAwIFIvTmV4dCAxNjQ4
IDAgUj4+CmVuZG9iagoKMTY0OCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAw
NjkwMDY0MDA2NTAwMjAwMDMzMDAzNj4KL0Rlc3RbNzg5IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFy
ZW50IDE2MTIgMCBSL1ByZXYgMTY0NyAwIFIvTmV4dCAxNjQ5IDAgUj4+CmVuZG9iagoKMTY0OSAw
IG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAz
Nz4KL0Rlc3RbODA4IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0
OCAwIFIvTmV4dCAxNjUwIDAgUj4+CmVuZG9iagoKMTY1MCAwIG9iago8PC9Db3VudCAwL1RpdGxl
PEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDMzMDAzOD4KL0Rlc3RbODMwIDAgUi9YWVog
MCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY0OSAwIFIvTmV4dCAxNjUxIDAgUj4+
CmVuZG9iagoKMTY1MSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0
MDA2NTAwMjAwMDMzMDAzOT4KL0Rlc3RbODQ5IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2
MTIgMCBSL1ByZXYgMTY1MCAwIFIvTmV4dCAxNjUyIDAgUj4+CmVuZG9iagoKMTY1MiAwIG9iago8
PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM0MDAzMD4KL0Rl
c3RbODY4IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY1MSAwIFIv
TmV4dCAxNjUzIDAgUj4+CmVuZG9iagoKMTY1MyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYw
MDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM0MDAzMT4KL0Rlc3RbODk0IDAgUi9YWVogMCA0MDUu
MDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY1MiAwIFIvTmV4dCAxNjU0IDAgUj4+CmVuZG9i
agoKMTY1NCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAw
MjAwMDM0MDAzMj4KL0Rlc3RbOTIzIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBS
L1ByZXYgMTY1MyAwIFIvTmV4dCAxNjU1IDAgUj4+CmVuZG9iagoKMTY1NSAwIG9iago8PC9Db3Vu
dCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM0MDAzMz4KL0Rlc3RbOTI4
IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY1NCAwIFIvTmV4dCAx
NjU2IDAgUj4+CmVuZG9iagoKMTY1NiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2
QzAwNjkwMDY0MDA2NTAwMjAwMDM0MDAzND4KL0Rlc3RbOTUwIDAgUi9YWVogMCA0MDUuMDcgMF0v
UGFyZW50IDE2MTIgMCBSL1ByZXYgMTY1NSAwIFIvTmV4dCAxNjU3IDAgUj4+CmVuZG9iagoKMTY1
NyAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM0
MDAzNT4KL0Rlc3RbOTc5IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYg
MTY1NiAwIFIvTmV4dCAxNjU4IDAgUj4+CmVuZG9iagoKMTY1OCAwIG9iago8PC9Db3VudCAwL1Rp
dGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM0MDAzNj4KL0Rlc3RbOTk4IDAgUi9Y
WVogMCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY1NyAwIFIvTmV4dCAxNjU5IDAg
Uj4+CmVuZG9iagoKMTY1OSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkw
MDY0MDA2NTAwMjAwMDM0MDAzNz4KL0Rlc3RbMTAxNyAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVu
dCAxNjEyIDAgUi9QcmV2IDE2NTggMCBSL05leHQgMTY2MCAwIFI+PgplbmRvYmoKCjE2NjAgMCBv
YmoKPDwvQ291bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNDAwMzg+
Ci9EZXN0WzEwMzkgMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjU5
IDAgUi9OZXh0IDE2NjEgMCBSPj4KZW5kb2JqCgoxNjYxIDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8
RkVGRjAwNTMwMDZDMDA2OTAwNjQwMDY1MDAyMDAwMzQwMDM5PgovRGVzdFsxMDU1IDAgUi9YWVog
MCA0MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY2MCAwIFIvTmV4dCAxNjYyIDAgUj4+
CmVuZG9iagoKMTY2MiAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0
MDA2NTAwMjAwMDM1MDAzMD4KL0Rlc3RbMTA2MCAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAx
NjEyIDAgUi9QcmV2IDE2NjEgMCBSL05leHQgMTY2MyAwIFI+PgplbmRvYmoKCjE2NjMgMCBvYmoK
PDwvQ291bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNTAwMzE+Ci9E
ZXN0WzEwNzkgMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjYyIDAg
Ui9OZXh0IDE2NjQgMCBSPj4KZW5kb2JqCgoxNjY0IDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVG
RjAwNTMwMDZDMDA2OTAwNjQwMDY1MDAyMDAwMzUwMDMyPgovRGVzdFsxMTA0IDAgUi9YWVogMCA0
MDUuMDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY2MyAwIFIvTmV4dCAxNjY1IDAgUj4+CmVu
ZG9iagoKMTY2NSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2
NTAwMjAwMDM1MDAzMz4KL0Rlc3RbMTEzMiAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEy
IDAgUi9QcmV2IDE2NjQgMCBSL05leHQgMTY2NiAwIFI+PgplbmRvYmoKCjE2NjYgMCBvYmoKPDwv
Q291bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNTAwMzQ+Ci9EZXN0
WzExNTEgMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjY1IDAgUi9O
ZXh0IDE2NjcgMCBSPj4KZW5kb2JqCgoxNjY3IDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAw
NTMwMDZDMDA2OTAwNjQwMDY1MDAyMDAwMzUwMDM1PgovRGVzdFsxMTk3IDAgUi9YWVogMCA0MDUu
MDcgMF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY2NiAwIFIvTmV4dCAxNjY4IDAgUj4+CmVuZG9i
agoKMTY2OCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAw
MjAwMDM1MDAzNj4KL0Rlc3RbMTIzOSAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAg
Ui9QcmV2IDE2NjcgMCBSL05leHQgMTY2OSAwIFI+PgplbmRvYmoKCjE2NjkgMCBvYmoKPDwvQ291
bnQgMC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNTAwMzc+Ci9EZXN0WzEy
ODggMCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjY4IDAgUi9OZXh0
IDE2NzAgMCBSPj4KZW5kb2JqCgoxNjcwIDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMw
MDZDMDA2OTAwNjQwMDY1MDAyMDAwMzUwMDM4PgovRGVzdFsxMzA3IDAgUi9YWVogMCA0MDUuMDcg
MF0vUGFyZW50IDE2MTIgMCBSL1ByZXYgMTY2OSAwIFIvTmV4dCAxNjcxIDAgUj4+CmVuZG9iagoK
MTY3MSAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAw
MDM1MDAzOT4KL0Rlc3RbMTMxMiAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9Q
cmV2IDE2NzAgMCBSL05leHQgMTY3MiAwIFI+PgplbmRvYmoKCjE2NzIgMCBvYmoKPDwvQ291bnQg
MC9UaXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNjAwMzA+Ci9EZXN0WzEzMzcg
MCBSL1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjcxIDAgUi9OZXh0IDE2
NzMgMCBSPj4KZW5kb2JqCgoxNjczIDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMwMDZD
MDA2OTAwNjQwMDY1MDAyMDAwMzYwMDMxPgovRGVzdFsxMzY2IDAgUi9YWVogMCA0MDUuMDcgMF0v
UGFyZW50IDE2MTIgMCBSL1ByZXYgMTY3MiAwIFIvTmV4dCAxNjc0IDAgUj4+CmVuZG9iagoKMTY3
NCAwIG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM2
MDAzMj4KL0Rlc3RbMTM4NSAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2
IDE2NzMgMCBSL05leHQgMTY3NSAwIFI+PgplbmRvYmoKCjE2NzUgMCBvYmoKPDwvQ291bnQgMC9U
aXRsZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNjAwMzM+Ci9EZXN0WzE0MTEgMCBS
L1hZWiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjc0IDAgUi9OZXh0IDE2NzYg
MCBSPj4KZW5kb2JqCgoxNjc2IDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMwMDZDMDA2
OTAwNjQwMDY1MDAyMDAwMzYwMDM0PgovRGVzdFsxNDMwIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFy
ZW50IDE2MTIgMCBSL1ByZXYgMTY3NSAwIFIvTmV4dCAxNjc3IDAgUj4+CmVuZG9iagoKMTY3NyAw
IG9iago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM2MDAz
NT4KL0Rlc3RbMTQ0NiAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2IDE2
NzYgMCBSL05leHQgMTY3OCAwIFI+PgplbmRvYmoKCjE2NzggMCBvYmoKPDwvQ291bnQgMC9UaXRs
ZTxGRUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNjAwMzY+Ci9EZXN0WzE0NjUgMCBSL1hZ
WiAwIDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjc3IDAgUi9OZXh0IDE2NzkgMCBS
Pj4KZW5kb2JqCgoxNjc5IDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMwMDZDMDA2OTAw
NjQwMDY1MDAyMDAwMzYwMDM3PgovRGVzdFsxNDcwIDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50
IDE2MTIgMCBSL1ByZXYgMTY3OCAwIFIvTmV4dCAxNjgwIDAgUj4+CmVuZG9iagoKMTY4MCAwIG9i
ago8PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM2MDAzOD4K
L0Rlc3RbMTQ4NiAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2IDE2Nzkg
MCBSL05leHQgMTY4MSAwIFI+PgplbmRvYmoKCjE2ODEgMCBvYmoKPDwvQ291bnQgMC9UaXRsZTxG
RUZGMDA1MzAwNkMwMDY5MDA2NDAwNjUwMDIwMDAzNjAwMzk+Ci9EZXN0WzE0OTEgMCBSL1hZWiAw
IDQwNS4wNyAwXS9QYXJlbnQgMTYxMiAwIFIvUHJldiAxNjgwIDAgUi9OZXh0IDE2ODIgMCBSPj4K
ZW5kb2JqCgoxNjgyIDAgb2JqCjw8L0NvdW50IDAvVGl0bGU8RkVGRjAwNTMwMDZDMDA2OTAwNjQw
MDY1MDAyMDAwMzcwMDMwPgovRGVzdFsxNTA0IDAgUi9YWVogMCA0MDUuMDcgMF0vUGFyZW50IDE2
MTIgMCBSL1ByZXYgMTY4MSAwIFIvTmV4dCAxNjgzIDAgUj4+CmVuZG9iagoKMTY4MyAwIG9iago8
PC9Db3VudCAwL1RpdGxlPEZFRkYwMDUzMDA2QzAwNjkwMDY0MDA2NTAwMjAwMDM3MDAzMT4KL0Rl
c3RbMTUyMyAwIFIvWFlaIDAgNDA1LjA3IDBdL1BhcmVudCAxNjEyIDAgUi9QcmV2IDE2ODIgMCBS
Pj4KZW5kb2JqCgo2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDUgMCBSCi9QZyAx
IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNjg1IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQv
QmxvY2sKPj4KZW5kb2JqCgo1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQg
MCBSCi9QZyAxIDAgUgovQSAxNjg1IDAgUgovS1s2IDAgUiBdCj4+CmVuZG9iagoKOCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA3IDAgUgovUGcgMSAwIFIKPj4KZW5kb2JqCgo5IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDcgMCBSCi9QZyAxIDAgUgo+PgplbmRvYmoK
CjEwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDcgMCBSCi9QZyAxIDAgUgovS1sx
IF0KPj4KZW5kb2JqCgoxMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA3IDAgUgov
UGcgMSAwIFIKPj4KZW5kb2JqCgo3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1Ag
MTY4NCAwIFIKL1BnIDEgMCBSCi9LWzggMCBSIDkgMCBSIDEwIDAgUiAxMSAwIFIgXQo+PgplbmRv
YmoKCjEzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEyIDAgUgovUGcgMSAwIFIK
L0tbMiBdCj4+CmVuZG9iagoKMTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTIg
MCBSCi9QZyAxIDAgUgo+PgplbmRvYmoKCjEyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9E
aXYKL1AgMTY4NCAwIFIKL1BnIDEgMCBSCi9LWzEzIDAgUiAxNCAwIFIgXQo+PgplbmRvYmoKCjE5
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE4IDAgUgovUGcgMTUgMCBSCi9LWzAg
XQo+PgplbmRvYmoKCjE2ODYgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+Pgpl
bmRvYmoKCjE4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAx
NSAwIFIKL0EgMTY4NiAwIFIKL0tbMTkgMCBSIF0KPj4KZW5kb2JqCgoyMyAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTGJsCi9QIDIyIDAgUgovUGcgMTUgMCBSCi9LWzEgXQo+PgplbmRvYmoK
CjI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAyMiAwIFIKL1BnIDE1IDAg
UgovS1syIF0KPj4KZW5kb2JqCgoyMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1Ag
MjEgMCBSCi9QZyAxNSAwIFIKL0tbMjMgMCBSIDI0IDAgUiBdCj4+CmVuZG9iagoKMjYgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAyNSAwIFIKL1BnIDE1IDAgUgovS1szIF0KPj4K
ZW5kb2JqCgoyNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjUgMCBSCi9Q
ZyAxNSAwIFIKL0tbNCBdCj4+CmVuZG9iagoKMjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xJCi9QIDIxIDAgUgovUGcgMTUgMCBSCi9LWzI2IDAgUiAyNyAwIFIgXQo+PgplbmRvYmoKCjI5
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjggMCBSCi9QZyAxNSAwIFIKL0tb
NSBdCj4+CmVuZG9iagoKMzAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDI4
IDAgUgovUGcgMTUgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjI4IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAyMSAwIFIKL1BnIDE1IDAgUgovS1syOSAwIFIgMzAgMCBSIF0KPj4KZW5k
b2JqCgozMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDMxIDAgUgovUGcgMTUg
MCBSCi9LWzcgXQo+PgplbmRvYmoKCjMzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCAzMSAwIFIKL1BnIDE1IDAgUgovS1s4IF0KPj4KZW5kb2JqCgozMSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgMjEgMCBSCi9QZyAxNSAwIFIKL0tbMzIgMCBSIDMzIDAgUiBd
Cj4+CmVuZG9iagoKMzUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAzNCAwIFIK
L1BnIDE1IDAgUgovS1s5IF0KPj4KZW5kb2JqCgozNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEJvZHkKL1AgMzQgMCBSCi9QZyAxNSAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjM0IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMSAwIFIKL1BnIDE1IDAgUgovS1szNSAwIFIg
MzYgMCBSIF0KPj4KZW5kb2JqCgozOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDM3IDAgUgovUGcgMTUgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgozOSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzcgMCBSCi9QZyAxNSAwIFIKL0tbMTIgXQo+PgplbmRvYmoK
CjM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMSAwIFIKL1BnIDE1IDAgUgov
S1szOCAwIFIgMzkgMCBSIF0KPj4KZW5kb2JqCgo0MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTGJsCi9QIDQwIDAgUgovUGcgMTUgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgo0MiAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDAgMCBSCi9QZyAxNSAwIFIKL0tbMTQgXQo+
PgplbmRvYmoKCjQwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMSAwIFIKL1Bn
IDE1IDAgUgovS1s0MSAwIFIgNDIgMCBSIF0KPj4KZW5kb2JqCgo0NCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDQzIDAgUgovUGcgMTUgMCBSCi9LWzE1IF0KPj4KZW5kb2JqCgo0
NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDMgMCBSCi9QZyAxNSAwIFIK
L0tbMTYgXQo+PgplbmRvYmoKCjQzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAy
MSAwIFIKL1BnIDE1IDAgUgovS1s0NCAwIFIgNDUgMCBSIF0KPj4KZW5kb2JqCgo0NyAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDQ2IDAgUgovUGcgMTUgMCBSCi9LWzE3IF0KPj4K
ZW5kb2JqCgo0OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDYgMCBSCi9Q
ZyAxNSAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjQ2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MSQovUCAyMSAwIFIKL1BnIDE1IDAgUgovS1s0NyAwIFIgNDggMCBSIF0KPj4KZW5kb2JqCgoy
MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAyMCAwIFIKL1BnIDE1IDAgUgovS1sy
MiAwIFIgMjUgMCBSIDI4IDAgUiAzMSAwIFIgMzQgMCBSIDM3IDAgUiA0MCAwIFIgNDMgMCBSIDQ2
IDAgUiBdCj4+CmVuZG9iagoKMjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAx
Njg0IDAgUgovUGcgMTUgMCBSCi9LWzIxIDAgUiBdCj4+CmVuZG9iagoKNTMgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTIgMCBSCi9QZyA0OSAwIFIKL0tbMCBdCj4+CmVuZG9iagoK
MTY4NyAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKNTIgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDQ5IDAgUgovQSAxNjg3
IDAgUgovS1s1MyAwIFIgXQo+PgplbmRvYmoKCjU4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9QCi9QIDU3IDAgUgovUGcgNTQgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE2ODggMCBvYmoKPDwv
Ty9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjU3IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA1NCAwIFIKL0EgMTY4OCAwIFIKL0tbNTggMCBS
IF0KPj4KZW5kb2JqCgo2MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDYxIDAg
UgovUGcgNTQgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjYzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MQm9keQovUCA2MSAwIFIKL1BnIDU0IDAgUgovS1syIF0KPj4KZW5kb2JqCgo2MSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjAgMCBSCi9QZyA1NCAwIFIKL0tbNjIgMCBS
IDYzIDAgUiBdCj4+CmVuZG9iagoKNjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAov
UCA2NCAwIFIKL1BnIDU0IDAgUgovS1szIF0KPj4KZW5kb2JqCgo2NiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgNjQgMCBSCi9QZyA1NCAwIFIKL0tbNCBdCj4+CmVuZG9iagoK
NjQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDYwIDAgUgovUGcgNTQgMCBSCi9L
WzY1IDAgUiA2NiAwIFIgXQo+PgplbmRvYmoKCjY4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgNjcgMCBSCi9QZyA1NCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKNjkgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY3IDAgUgovUGcgNTQgMCBSCi9LWzYgXQo+Pgpl
bmRvYmoKCjY3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2MCAwIFIKL1BnIDU0
IDAgUgovS1s2OCAwIFIgNjkgMCBSIF0KPj4KZW5kb2JqCgo3MSAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDcwIDAgUgovUGcgNTQgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjcyIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA3MCAwIFIKL1BnIDU0IDAgUgovS1s4
IF0KPj4KZW5kb2JqCgo3MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjAgMCBS
Ci9QZyA1NCAwIFIKL0tbNzEgMCBSIDcyIDAgUiBdCj4+CmVuZG9iagoKNjAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgNTkgMCBSCi9QZyA1NCAwIFIKL0tbNjEgMCBSIDY0IDAgUiA2
NyAwIFIgNzAgMCBSIF0KPj4KZW5kb2JqCgo1OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
RGl2Ci9QIDE2ODQgMCBSCi9QZyA1NCAwIFIKL0tbNjAgMCBSIF0KPj4KZW5kb2JqCgo3NyAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA3NiAwIFIKL1BnIDczIDAgUgovS1swIF0KPj4K
ZW5kb2JqCgoxNjg5IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2Jq
Cgo3NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNzMgMCBS
Ci9BIDE2ODkgMCBSCi9LWzc3IDAgUiBdCj4+CmVuZG9iagoKODEgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xibAovUCA4MCAwIFIKL1BnIDczIDAgUgovS1sxIF0KPj4KZW5kb2JqCgo4MiAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODAgMCBSCi9QZyA3MyAwIFIKL0tb
MiBdCj4+CmVuZG9iagoKODAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDc5IDAg
UgovUGcgNzMgMCBSCi9LWzgxIDAgUiA4MiAwIFIgXQo+PgplbmRvYmoKCjg0IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgODMgMCBSCi9QZyA3MyAwIFIKL0tbMyBdCj4+CmVuZG9i
agoKODUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDgzIDAgUgovUGcgNzMg
MCBSCi9LWzQgXQo+PgplbmRvYmoKCjgzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCA3OSAwIFIKL1BnIDczIDAgUgovS1s4NCAwIFIgODUgMCBSIF0KPj4KZW5kb2JqCgo4NyAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDg2IDAgUgovUGcgNzMgMCBSCi9LWzUgXQo+
PgplbmRvYmoKCjkxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgOTAgMCBSCi9Q
ZyA3MyAwIFIKL0tbNyBdCj4+CmVuZG9iagoKOTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDkwIDAgUgovUGcgNzMgMCBSCi9LWzggXQo+PgplbmRvYmoKCjkwIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA4OSAwIFIKL1BnIDczIDAgUgovS1s5MSAwIFIgOTIg
MCBSIF0KPj4KZW5kb2JqCgo5NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDkz
IDAgUgovUGcgNzMgMCBSCi9LWzkgXQo+PgplbmRvYmoKCjk1IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MQm9keQovUCA5MyAwIFIKL1BnIDczIDAgUgovS1sxMCBdCj4+CmVuZG9iagoKOTMg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg5IDAgUgovUGcgNzMgMCBSCi9LWzk0
IDAgUiA5NSAwIFIgXQo+PgplbmRvYmoKCjk3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgOTYgMCBSCi9QZyA3MyAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjk4IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5NiAwIFIKL1BnIDczIDAgUgovS1sxMiBdCj4+CmVu
ZG9iagoKOTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg5IDAgUgovUGcgNzMg
MCBSCi9LWzk3IDAgUiA5OCAwIFIgXQo+PgplbmRvYmoKCjg5IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MCi9QIDg4IDAgUgovUGcgNzMgMCBSCi9LWzkwIDAgUiA5MyAwIFIgOTYgMCBSIF0K
Pj4KZW5kb2JqCgo4OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODYgMCBS
Ci9QZyA3MyAwIFIKL0tbNiA4OSAwIFIgXQo+PgplbmRvYmoKCjg2IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MSQovUCA3OSAwIFIKL1BnIDczIDAgUgovS1s4NyAwIFIgODggMCBSIF0KPj4K
ZW5kb2JqCgoxMDAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA5OSAwIFIKL1Bn
IDczIDAgUgovS1sxMyBdCj4+CmVuZG9iagoKMTAxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MQm9keQovUCA5OSAwIFIKL1BnIDczIDAgUgovS1sxNCBdCj4+CmVuZG9iagoKOTkgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDc5IDAgUgovUGcgNzMgMCBSCi9LWzEwMCAwIFIg
MTAxIDAgUiBdCj4+CmVuZG9iagoKMTAzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwK
L1AgMTAyIDAgUgovUGcgNzMgMCBSCi9LWzE1IF0KPj4KZW5kb2JqCgoxMDQgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEwMiAwIFIKL1BnIDczIDAgUgovS1sxNiBdCj4+CmVu
ZG9iagoKMTAyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3OSAwIFIKL1BnIDcz
IDAgUgovS1sxMDMgMCBSIDEwNCAwIFIgXQo+PgplbmRvYmoKCjc5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MCi9QIDc4IDAgUgovUGcgNzMgMCBSCi9LWzgwIDAgUiA4MyAwIFIgODYgMCBS
IDk5IDAgUiAxMDIgMCBSIF0KPj4KZW5kb2JqCgo3OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA3MyAwIFIKL0tbNzkgMCBSIF0KPj4KZW5kb2JqCgoxMDkg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTA4IDAgUgovUGcgMTA1IDAgUgovS1sw
IF0KPj4KZW5kb2JqCgoxNjkwIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4K
ZW5kb2JqCgoxMDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1Bn
IDEwNSAwIFIKL0EgMTY5MCAwIFIKL0tbMTA5IDAgUiBdCj4+CmVuZG9iagoKMTEzIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTEyIDAgUgovUGcgMTA1IDAgUgovS1sxIF0KPj4K
ZW5kb2JqCgoxMTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDExMiAwIFIK
L1BnIDEwNSAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTEyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MSQovUCAxMTEgMCBSCi9QZyAxMDUgMCBSCi9LWzExMyAwIFIgMTE0IDAgUiBdCj4+CmVu
ZG9iagoKMTE2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTE1IDAgUgovUGcg
MTA1IDAgUgovS1szIF0KPj4KZW5kb2JqCgoxMjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAxMTkgMCBSCi9QZyAxMDUgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjEyMSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTE5IDAgUgovUGcgMTA1IDAgUgovS1s2IF0K
Pj4KZW5kb2JqCgoxMTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDExOCAwIFIK
L1BnIDEwNSAwIFIKL0tbMTIwIDAgUiAxMjEgMCBSIF0KPj4KZW5kb2JqCgoxMjMgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjIgMCBSCi9QZyAxMDUgMCBSCi9LWzcgXQo+Pgpl
bmRvYmoKCjEyNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTIyIDAgUgov
UGcgMTA1IDAgUgovS1s4IF0KPj4KZW5kb2JqCgoxMjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xJCi9QIDExOCAwIFIKL1BnIDEwNSAwIFIKL0tbMTIzIDAgUiAxMjQgMCBSIF0KPj4KZW5k
b2JqCgoxMjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjUgMCBSCi9QZyAx
MDUgMCBSCi9LWzkgXQo+PgplbmRvYmoKCjEyNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgMTI1IDAgUgovUGcgMTA1IDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTI1IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMTggMCBSCi9QZyAxMDUgMCBSCi9LWzEyNiAw
IFIgMTI3IDAgUiBdCj4+CmVuZG9iagoKMTE4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
Ci9QIDExNyAwIFIKL1BnIDEwNSAwIFIKL0tbMTE5IDAgUiAxMjIgMCBSIDEyNSAwIFIgXQo+Pgpl
bmRvYmoKCjExNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTE1IDAgUgov
UGcgMTA1IDAgUgovS1s0IDExOCAwIFIgXQo+PgplbmRvYmoKCjExNSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTExIDAgUgovUGcgMTA1IDAgUgovS1sxMTYgMCBSIDExNyAwIFIg
XQo+PgplbmRvYmoKCjEyOSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEyOCAw
IFIKL1BnIDEwNSAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjEzMyAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDEzMiAwIFIKL1BnIDEwNSAwIFIKL0tbMTMgXQo+PgplbmRvYmoKCjEz
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTMyIDAgUgovUGcgMTA1IDAg
UgovS1sxNCBdCj4+CmVuZG9iagoKMTMyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCAxMzEgMCBSCi9QZyAxMDUgMCBSCi9LWzEzMyAwIFIgMTM0IDAgUiBdCj4+CmVuZG9iagoKMTM2
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTM1IDAgUgovUGcgMTA1IDAgUgov
S1sxNSBdCj4+CmVuZG9iagoKMTM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCAxMzUgMCBSCi9QZyAxMDUgMCBSCi9LWzE2IF0KPj4KZW5kb2JqCgoxMzUgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzMSAwIFIKL1BnIDEwNSAwIFIKL0tbMTM2IDAgUiAxMzcg
MCBSIF0KPj4KZW5kb2JqCgoxMzEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTMw
IDAgUgovUGcgMTA1IDAgUgovS1sxMzIgMCBSIDEzNSAwIFIgXQo+PgplbmRvYmoKCjEzMCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTI4IDAgUgovUGcgMTA1IDAgUgovS1sx
MiAxMzEgMCBSIF0KPj4KZW5kb2JqCgoxMjggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJ
Ci9QIDExMSAwIFIKL1BnIDEwNSAwIFIKL0tbMTI5IDAgUiAxMzAgMCBSIF0KPj4KZW5kb2JqCgox
MTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTEwIDAgUgovUGcgMTA1IDAgUgov
S1sxMTIgMCBSIDExNSAwIFIgMTI4IDAgUiBdCj4+CmVuZG9iagoKMTEwIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDEwNSAwIFIKL0tbMTExIDAgUiBdCj4+
CmVuZG9iagoKMTQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE0MSAwIFIKL1Bn
IDEzOCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTY5MSAwIG9iago8PC9PL0xheW91dAovUGxhY2Vt
ZW50L0Jsb2NrCj4+CmVuZG9iagoKMTQxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9Q
IDE2ODQgMCBSCi9QZyAxMzggMCBSCi9BIDE2OTEgMCBSCi9LWzE0MiAwIFIgXQo+PgplbmRvYmoK
CjE0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE0NSAwIFIKL1BnIDEzOCAw
IFIKL0tbMSBdCj4+CmVuZG9iagoKMTQ3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCAxNDUgMCBSCi9QZyAxMzggMCBSCi9LWzIgXQo+PgplbmRvYmoKCjE0NSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQ0IDAgUgovUGcgMTM4IDAgUgovS1sxNDYgMCBSIDE0
NyAwIFIgXQo+PgplbmRvYmoKCjE0OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDE0OCAwIFIKL1BnIDEzOCAwIFIKL0tbMyBdCj4+CmVuZG9iagoKMTUwIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAxNDggMCBSCi9QZyAxMzggMCBSCi9LWzQgXQo+PgplbmRv
YmoKCjE0OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQ0IDAgUgovUGcgMTM4
IDAgUgovS1sxNDkgMCBSIDE1MCAwIFIgXQo+PgplbmRvYmoKCjE1MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDE1MSAwIFIKL1BnIDEzOCAwIFIKL0tbNSBdCj4+CmVuZG9iagoK
MTUzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxNTEgMCBSCi9QZyAxMzgg
MCBSCi9LWzYgXQo+PgplbmRvYmoKCjE1MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkK
L1AgMTQ0IDAgUgovUGcgMTM4IDAgUgovS1sxNTIgMCBSIDE1MyAwIFIgXQo+PgplbmRvYmoKCjE1
NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE1NCAwIFIKL1BnIDEzOCAwIFIK
L0tbNyBdCj4+CmVuZG9iagoKMTU2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCAxNTQgMCBSCi9QZyAxMzggMCBSCi9LWzggXQo+PgplbmRvYmoKCjE1NCAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQ0IDAgUgovUGcgMTM4IDAgUgovS1sxNTUgMCBSIDE1NiAw
IFIgXQo+PgplbmRvYmoKCjE1OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE1
NyAwIFIKL1BnIDEzOCAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMTU5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCAxNTcgMCBSCi9QZyAxMzggMCBSCi9LWzEwIF0KPj4KZW5kb2Jq
CgoxNTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDE0NCAwIFIKL1BnIDEzOCAw
IFIKL0tbMTU4IDAgUiAxNTkgMCBSIF0KPj4KZW5kb2JqCgoxNjEgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xibAovUCAxNjAgMCBSCi9QZyAxMzggMCBSCi9LWzExIF0KPj4KZW5kb2JqCgox
NjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDE2MCAwIFIKL1BnIDEzOCAw
IFIKL0tbMTIgXQo+PgplbmRvYmoKCjE2MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkK
L1AgMTQ0IDAgUgovUGcgMTM4IDAgUgovS1sxNjEgMCBSIDE2MiAwIFIgXQo+PgplbmRvYmoKCjE2
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE2MyAwIFIKL1BnIDEzOCAwIFIK
L0tbMTMgXQo+PgplbmRvYmoKCjE2NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgMTYzIDAgUgovUGcgMTM4IDAgUgovS1sxNCBdCj4+CmVuZG9iagoKMTYzIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxNDQgMCBSCi9QZyAxMzggMCBSCi9LWzE2NCAwIFIgMTY1
IDAgUiBdCj4+CmVuZG9iagoKMTQ0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDE0
MyAwIFIKL1BnIDEzOCAwIFIKL0tbMTQ1IDAgUiAxNDggMCBSIDE1MSAwIFIgMTU0IDAgUiAxNTcg
MCBSIDE2MCAwIFIgMTYzIDAgUiBdCj4+CmVuZG9iagoKMTQzIDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDEzOCAwIFIKL0tbMTQ0IDAgUiBdCj4+CmVuZG9i
agoKMTcwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE2OSAwIFIKL1BnIDE2NiAw
IFIKL0tbMCBdCj4+CmVuZG9iagoKMTY5MiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Js
b2NrCj4+CmVuZG9iagoKMTY5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQg
MCBSCi9QZyAxNjYgMCBSCi9BIDE2OTIgMCBSCi9LWzE3MCAwIFIgXQo+PgplbmRvYmoKCjE3NCAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE3MyAwIFIKL1BnIDE2NiAwIFIKL0tb
MSBdCj4+CmVuZG9iagoKMTc1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAx
NzMgMCBSCi9QZyAxNjYgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjE3MyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTcyIDAgUgovUGcgMTY2IDAgUgovS1sxNzQgMCBSIDE3NSAwIFIg
XQo+PgplbmRvYmoKCjE3NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE3NiAw
IFIKL1BnIDE2NiAwIFIKL0tbMyBdCj4+CmVuZG9iagoKMTc4IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MQm9keQovUCAxNzYgMCBSCi9QZyAxNjYgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjE3
NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTcyIDAgUgovUGcgMTY2IDAgUgov
S1sxNzcgMCBSIDE3OCAwIFIgXQo+PgplbmRvYmoKCjE4MCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDE3OSAwIFIKL1BnIDE2NiAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMTgxIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxNzkgMCBSCi9QZyAxNjYgMCBSCi9L
WzYgXQo+PgplbmRvYmoKCjE3OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTcy
IDAgUgovUGcgMTY2IDAgUgovS1sxODAgMCBSIDE4MSAwIFIgXQo+PgplbmRvYmoKCjE4MyAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE4MiAwIFIKL1BnIDE2NiAwIFIKL0tbNyBd
Cj4+CmVuZG9iagoKMTg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxODIg
MCBSCi9QZyAxNjYgMCBSCi9LWzggXQo+PgplbmRvYmoKCjE4MiAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEkKL1AgMTcyIDAgUgovUGcgMTY2IDAgUgovS1sxODMgMCBSIDE4NCAwIFIgXQo+
PgplbmRvYmoKCjE4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE4NSAwIFIK
L1BnIDE2NiAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMTg3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MQm9keQovUCAxODUgMCBSCi9QZyAxNjYgMCBSCi9LWzEwIF0KPj4KZW5kb2JqCgoxODUg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDE3MiAwIFIKL1BnIDE2NiAwIFIKL0tb
MTg2IDAgUiAxODcgMCBSIF0KPj4KZW5kb2JqCgoxNzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0wKL1AgMTcxIDAgUgovUGcgMTY2IDAgUgovS1sxNzMgMCBSIDE3NiAwIFIgMTc5IDAgUiAx
ODIgMCBSIDE4NSAwIFIgXQo+PgplbmRvYmoKCjE3MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyAxNjYgMCBSCi9LWzE3MiAwIFIgXQo+PgplbmRvYmoKCjE5
MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxOTEgMCBSCi9QZyAxODggMCBSCi9L
WzAgXQo+PgplbmRvYmoKCjE2OTMgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+
PgplbmRvYmoKCjE5MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgov
UGcgMTg4IDAgUgovQSAxNjkzIDAgUgovS1sxOTIgMCBSIF0KPj4KZW5kb2JqCgoxOTQgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcgMTg4IDAgUgovS1syIF0KPj4K
ZW5kb2JqCgoxOTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcg
MTg4IDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxOTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L1AKL1AgMTkzIDAgUgovUGcgMTg4IDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxOTcgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcgMTg4IDAgUgovS1s4IF0KPj4KZW5k
b2JqCgoxOTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcgMTg4
IDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTk5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDE5MyAwIFIKL1BnIDE4OCAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjIwMCAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxOTMgMCBSCi9QZyAxODggMCBSCi9LWzEzIF0KPj4KZW5k
b2JqCgoyMDEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcgMTg4
IDAgUgovS1sxNSBdCj4+CmVuZG9iagoKMjAyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDE5MyAwIFIKL1BnIDE4OCAwIFIKL0tbMTcgXQo+PgplbmRvYmoKCjIwMyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxOTMgMCBSCi9QZyAxODggMCBSCi9LWzE4IF0KPj4KZW5k
b2JqCgoyMDQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTkzIDAgUgovUGcgMTg4
IDAgUgovS1syMCBdCj4+CmVuZG9iagoKMjA1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDE5MyAwIFIKL1BnIDE4OCAwIFIKL0tbMjEgXQo+PgplbmRvYmoKCjIwNiAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxOTMgMCBSCi9QZyAxODggMCBSCj4+CmVuZG9iagoKMTY5
NCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCi9CQm94WzEzMC4zMzcgNTMuNDg5
IDU0MC44NSAyOTEuMjU5XQo+PgplbmRvYmoKCjE5MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvRmlndXJlCi9QIDE2ODQgMCBSCi9QZyAxODggMCBSCi9BIDE2OTQgMCBSCi9LWzEgMTk0IDAg
UiAzIDE5NSAwIFIgNSAxOTYgMCBSIDcgMTk3IDAgUiA5IDE5OCAwIFIgMTk5IDAgUiAxMiAyMDAg
MCBSIDE0IDIwMSAwIFIgMTYgMjAyIDAgUiAyMDMgMCBSIDE5IDIwNCAwIFIgMjA1IDAgUiAyMiAy
MDYgMCBSIF0KPj4KZW5kb2JqCgoyMTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
MjEwIDAgUgovUGcgMjA3IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNjk1IDAgb2JqCjw8L08vTGF5
b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoyMTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDIwNyAwIFIKL0EgMTY5NSAwIFIKL0tbMjExIDAgUiBd
Cj4+CmVuZG9iagoKMjE1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjE0IDAg
UgovUGcgMjA3IDAgUgovS1sxIF0KPj4KZW5kb2JqCgoyMTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xCb2R5Ci9QIDIxNCAwIFIKL1BnIDIwNyAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMjE0
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMTMgMCBSCi9QZyAyMDcgMCBSCi9L
WzIxNSAwIFIgMjE2IDAgUiBdCj4+CmVuZG9iagoKMjE4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MYmwKL1AgMjE3IDAgUgovUGcgMjA3IDAgUgovS1szIF0KPj4KZW5kb2JqCgoyMTkgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDIxNyAwIFIKL1BnIDIwNyAwIFIKL0tb
NCBdCj4+CmVuZG9iagoKMjE3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMTMg
MCBSCi9QZyAyMDcgMCBSCi9LWzIxOCAwIFIgMjE5IDAgUiBdCj4+CmVuZG9iagoKMjIxIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjIwIDAgUgovUGcgMjA3IDAgUgovS1s1IF0K
Pj4KZW5kb2JqCgoyMjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDIyMCAw
IFIKL1BnIDIwNyAwIFIKL0tbNiBdCj4+CmVuZG9iagoKMjIwIDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAyMTMgMCBSCi9QZyAyMDcgMCBSCi9LWzIyMSAwIFIgMjIyIDAgUiBdCj4+
CmVuZG9iagoKMjI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjIzIDAgUgov
UGcgMjA3IDAgUgovS1s3IF0KPj4KZW5kb2JqCgoyMjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xCb2R5Ci9QIDIyMyAwIFIKL1BnIDIwNyAwIFIKL0tbOCBdCj4+CmVuZG9iagoKMjIzIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyMTMgMCBSCi9QZyAyMDcgMCBSCi9LWzIy
NCAwIFIgMjI1IDAgUiBdCj4+CmVuZG9iagoKMjEzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MCi9QIDIxMiAwIFIKL1BnIDIwNyAwIFIKL0tbMjE0IDAgUiAyMTcgMCBSIDIyMCAwIFIgMjIz
IDAgUiBdCj4+CmVuZG9iagoKMjEyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1Ag
MTY4NCAwIFIKL1BnIDIwNyAwIFIKL0tbMjEzIDAgUiBdCj4+CmVuZG9iagoKMjMwIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDIyOSAwIFIKL1BnIDIyNiAwIFIKL0tbMCBdCj4+CmVu
ZG9iagoKMTY5NiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoK
MjI5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAyMjYgMCBS
Ci9BIDE2OTYgMCBSCi9LWzIzMCAwIFIgXQo+PgplbmRvYmoKCjIzNSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCAyMzQgMCBSCi9QZyAyMzEgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE2
OTcgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjIzNCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgMjMxIDAgUgovQSAxNjk3
IDAgUgovS1syMzUgMCBSIF0KPj4KZW5kb2JqCgoyMzkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCAyMzggMCBSCi9QZyAyMzEgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjI0MCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjM4IDAgUgovUGcgMjMxIDAgUgovS1sy
IF0KPj4KZW5kb2JqCgoyMzggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDIzNyAw
IFIKL1BnIDIzMSAwIFIKL0tbMjM5IDAgUiAyNDAgMCBSIF0KPj4KZW5kb2JqCgoyNDIgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAyNDEgMCBSCi9QZyAyMzEgMCBSCi9LWzMgXQo+
PgplbmRvYmoKCjI0MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjQxIDAg
UgovUGcgMjMxIDAgUgovS1s0IF0KPj4KZW5kb2JqCgoyNDEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDIzNyAwIFIKL1BnIDIzMSAwIFIKL0tbMjQyIDAgUiAyNDMgMCBSIF0KPj4K
ZW5kb2JqCgoyNDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAyNDQgMCBSCi9Q
ZyAyMzEgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjI0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEJvZHkKL1AgMjQ0IDAgUgovUGcgMjMxIDAgUgovS1s2IF0KPj4KZW5kb2JqCgoyNDQgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDIzNyAwIFIKL1BnIDIzMSAwIFIKL0tbMjQ1
IDAgUiAyNDYgMCBSIF0KPj4KZW5kb2JqCgoyNDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAyNDcgMCBSCi9QZyAyMzEgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjI0OSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjQ3IDAgUgovUGcgMjMxIDAgUgovS1s4IF0K
Pj4KZW5kb2JqCgoyNDcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDIzNyAwIFIK
L1BnIDIzMSAwIFIKL0tbMjQ4IDAgUiAyNDkgMCBSIF0KPj4KZW5kb2JqCgoyNTEgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAyNTAgMCBSCi9QZyAyMzEgMCBSCi9LWzkgXQo+Pgpl
bmRvYmoKCjI1MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjUwIDAgUgov
UGcgMjMxIDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMjUwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MSQovUCAyMzcgMCBSCi9QZyAyMzEgMCBSCi9LWzI1MSAwIFIgMjUyIDAgUiBdCj4+CmVu
ZG9iagoKMjU0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjUzIDAgUgovUGcg
MjMxIDAgUgovS1sxMSBdCj4+CmVuZG9iagoKMjU1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MQm9keQovUCAyNTMgMCBSCi9QZyAyMzEgMCBSCi9LWzEyIF0KPj4KZW5kb2JqCgoyNTMgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDIzNyAwIFIKL1BnIDIzMSAwIFIKL0tbMjU0
IDAgUiAyNTUgMCBSIF0KPj4KZW5kb2JqCgoyNTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAyNTYgMCBSCi9QZyAyMzEgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgoyNTggMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDI1NiAwIFIKL1BnIDIzMSAwIFIKL0tbMTQg
XQo+PgplbmRvYmoKCjI1NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMjM3IDAg
UgovUGcgMjMxIDAgUgovS1syNTcgMCBSIDI1OCAwIFIgXQo+PgplbmRvYmoKCjI2MCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDI1OSAwIFIKL1BnIDIzMSAwIFIKL0tbMTUgXQo+
PgplbmRvYmoKCjI2MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjU5IDAg
UgovUGcgMjMxIDAgUgovS1sxNiBdCj4+CmVuZG9iagoKMjU5IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAyMzcgMCBSCi9QZyAyMzEgMCBSCi9LWzI2MCAwIFIgMjYxIDAgUiBdCj4+
CmVuZG9iagoKMjM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDIzNiAwIFIKL1Bn
IDIzMSAwIFIKL0tbMjM4IDAgUiAyNDEgMCBSIDI0NCAwIFIgMjQ3IDAgUiAyNTAgMCBSIDI1MyAw
IFIgMjU2IDAgUiAyNTkgMCBSIF0KPj4KZW5kb2JqCgoyMzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMjMxIDAgUgovS1syMzcgMCBSIF0KPj4KZW5kb2Jq
CgoyNjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMjY1IDAgUgovUGcgMjYyIDAg
UgovS1swIF0KPj4KZW5kb2JqCgoxNjk4IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxv
Y2sKPj4KZW5kb2JqCgoyNjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAw
IFIKL1BnIDI2MiAwIFIKL0EgMTY5OCAwIFIKL0tbMjY2IDAgUiBdCj4+CmVuZG9iagoKMjcwIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjY5IDAgUgovUGcgMjYyIDAgUgovS1sx
IF0KPj4KZW5kb2JqCgoyNzEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDI2
OSAwIFIKL1BnIDI2MiAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMjY5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MSQovUCAyNjggMCBSCi9QZyAyNjIgMCBSCi9LWzI3MCAwIFIgMjcxIDAgUiBd
Cj4+CmVuZG9iagoKMjczIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjcyIDAg
UgovUGcgMjYyIDAgUgovS1szIF0KPj4KZW5kb2JqCgoyNzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xCb2R5Ci9QIDI3MiAwIFIKL1BnIDI2MiAwIFIKL0tbNCBdCj4+CmVuZG9iagoKMjcy
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyNjggMCBSCi9QZyAyNjIgMCBSCi9L
WzI3MyAwIFIgMjc0IDAgUiBdCj4+CmVuZG9iagoKMjc2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MYmwKL1AgMjc1IDAgUgovUGcgMjYyIDAgUgovS1s1IF0KPj4KZW5kb2JqCgoyNzcgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDI3NSAwIFIKL1BnIDI2MiAwIFIKL0tb
NiBdCj4+CmVuZG9iagoKMjc1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyNjgg
MCBSCi9QZyAyNjIgMCBSCi9LWzI3NiAwIFIgMjc3IDAgUiBdCj4+CmVuZG9iagoKMjc5IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjc4IDAgUgovUGcgMjYyIDAgUgovS1s3IF0K
Pj4KZW5kb2JqCgoyODAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDI3OCAw
IFIKL1BnIDI2MiAwIFIKL0tbOCBdCj4+CmVuZG9iagoKMjc4IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAyNjggMCBSCi9QZyAyNjIgMCBSCi9LWzI3OSAwIFIgMjgwIDAgUiBdCj4+
CmVuZG9iagoKMjgyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjgxIDAgUgov
UGcgMjYyIDAgUgovS1s5IF0KPj4KZW5kb2JqCgoyODMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xCb2R5Ci9QIDI4MSAwIFIKL1BnIDI2MiAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjI4MSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMjY4IDAgUgovUGcgMjYyIDAgUgovS1sy
ODIgMCBSIDI4MyAwIFIgXQo+PgplbmRvYmoKCjI4NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTGJsCi9QIDI4NCAwIFIKL1BnIDI2MiAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjI4NiAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMjg0IDAgUgovUGcgMjYyIDAgUgovS1sx
MiBdCj4+CmVuZG9iagoKMjg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAyNjgg
MCBSCi9QZyAyNjIgMCBSCi9LWzI4NSAwIFIgMjg2IDAgUiBdCj4+CmVuZG9iagoKMjg4IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMjg3IDAgUgovUGcgMjYyIDAgUgovS1sxMyBd
Cj4+CmVuZG9iagoKMjg5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAyODcg
MCBSCi9QZyAyNjIgMCBSCi9LWzE0IF0KPj4KZW5kb2JqCgoyODcgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xJCi9QIDI2OCAwIFIKL1BnIDI2MiAwIFIKL0tbMjg4IDAgUiAyODkgMCBSIF0K
Pj4KZW5kb2JqCgoyOTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAyOTAgMCBS
Ci9QZyAyNjIgMCBSCi9LWzE1IF0KPj4KZW5kb2JqCgoyOTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xCb2R5Ci9QIDI5MCAwIFIKL1BnIDI2MiAwIFIKL0tbMTYgXQo+PgplbmRvYmoKCjI5
MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMjY4IDAgUgovUGcgMjYyIDAgUgov
S1syOTEgMCBSIDI5MiAwIFIgXQo+PgplbmRvYmoKCjI2OCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTAovUCAyNjcgMCBSCi9QZyAyNjIgMCBSCi9LWzI2OSAwIFIgMjcyIDAgUiAyNzUgMCBS
IDI3OCAwIFIgMjgxIDAgUiAyODQgMCBSIDI4NyAwIFIgMjkwIDAgUiBdCj4+CmVuZG9iagoKMjY3
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDI2MiAwIFIK
L0tbMjY4IDAgUiBdCj4+CmVuZG9iagoKMjk3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDI5NiAwIFIKL1BnIDI5MyAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTY5OSAwIG9iago8PC9P
L0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKMjk2IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAyOTMgMCBSCi9BIDE2OTkgMCBSCi9LWzI5NyAw
IFIgXQo+PgplbmRvYmoKCjMwMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDMw
MCAwIFIKL1BnIDI5MyAwIFIKL0tbMSBdCj4+CmVuZG9iagoKMzA1IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgMzA0IDAgUgovUGcgMjkzIDAgUgovS1szIF0KPj4KZW5kb2JqCgoz
MDYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDMwNCAwIFIKL1BnIDI5MyAw
IFIKL0tbNCBdCj4+CmVuZG9iagoKMzA0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCAzMDMgMCBSCi9QZyAyOTMgMCBSCi9LWzMwNSAwIFIgMzA2IDAgUiBdCj4+CmVuZG9iagoKMzA4
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMzA3IDAgUgovUGcgMjkzIDAgUgov
S1s1IF0KPj4KZW5kb2JqCgozMDkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDMwNyAwIFIKL1BnIDI5MyAwIFIKL0tbNiBdCj4+CmVuZG9iagoKMzA3IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MSQovUCAzMDMgMCBSCi9QZyAyOTMgMCBSCi9LWzMwOCAwIFIgMzA5IDAg
UiBdCj4+CmVuZG9iagoKMzExIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMzEw
IDAgUgovUGcgMjkzIDAgUgovS1s3IF0KPj4KZW5kb2JqCgozMTIgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDMxMCAwIFIKL1BnIDI5MyAwIFIKL0tbOCBdCj4+CmVuZG9iagoK
MzEwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAzMDMgMCBSCi9QZyAyOTMgMCBS
Ci9LWzMxMSAwIFIgMzEyIDAgUiBdCj4+CmVuZG9iagoKMzAzIDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MCi9QIDMwMiAwIFIKL1BnIDI5MyAwIFIKL0tbMzA0IDAgUiAzMDcgMCBSIDMxMCAw
IFIgXQo+PgplbmRvYmoKCjMwMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MzAwIDAgUgovUGcgMjkzIDAgUgovS1syIDMwMyAwIFIgXQo+PgplbmRvYmoKCjMwMCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMjk5IDAgUgovUGcgMjkzIDAgUgovS1szMDEgMCBS
IDMwMiAwIFIgXQo+PgplbmRvYmoKCjMxNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDMxMyAwIFIKL1BnIDI5MyAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMzE4IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMzE3IDAgUgovUGcgMjkzIDAgUgovS1sxMSBdCj4+CmVu
ZG9iagoKMzE5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzMTcgMCBSCi9Q
ZyAyOTMgMCBSCi9LWzEyIF0KPj4KZW5kb2JqCgozMTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xJCi9QIDMxNiAwIFIKL1BnIDI5MyAwIFIKL0tbMzE4IDAgUiAzMTkgMCBSIF0KPj4KZW5k
b2JqCgozMjEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAzMjAgMCBSCi9QZyAy
OTMgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgozMjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDMyMCAwIFIKL1BnIDI5MyAwIFIKL0tbMTQgXQo+PgplbmRvYmoKCjMyMCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzE2IDAgUgovUGcgMjkzIDAgUgovS1szMjEg
MCBSIDMyMiAwIFIgXQo+PgplbmRvYmoKCjMxNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TAovUCAzMTUgMCBSCi9QZyAyOTMgMCBSCi9LWzMxNyAwIFIgMzIwIDAgUiBdCj4+CmVuZG9iagoK
MzE1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzMTMgMCBSCi9QZyAyOTMg
MCBSCi9LWzEwIDMxNiAwIFIgXQo+PgplbmRvYmoKCjMxMyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEkKL1AgMjk5IDAgUgovUGcgMjkzIDAgUgovS1szMTQgMCBSIDMxNSAwIFIgXQo+Pgpl
bmRvYmoKCjMyNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDMyMyAwIFIKL1Bn
IDI5MyAwIFIKL0tbMTUgXQo+PgplbmRvYmoKCjMyOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTGJsCi9QIDMyNyAwIFIKL1BnIDI5MyAwIFIKL0tbMTcgXQo+PgplbmRvYmoKCjMyOSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzI3IDAgUgovUGcgMjkzIDAgUgovS1sx
OCBdCj4+CmVuZG9iagoKMzI3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAzMjYg
MCBSCi9QZyAyOTMgMCBSCi9LWzMyOCAwIFIgMzI5IDAgUiBdCj4+CmVuZG9iagoKMzMxIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMzMwIDAgUgovUGcgMjkzIDAgUgovS1sxOSBd
Cj4+CmVuZG9iagoKMzMyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzMzAg
MCBSCi9QZyAyOTMgMCBSCi9LWzIwIF0KPj4KZW5kb2JqCgozMzAgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xJCi9QIDMyNiAwIFIKL1BnIDI5MyAwIFIKL0tbMzMxIDAgUiAzMzIgMCBSIF0K
Pj4KZW5kb2JqCgozMjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMzI1IDAgUgov
UGcgMjkzIDAgUgovS1szMjcgMCBSIDMzMCAwIFIgXQo+PgplbmRvYmoKCjMyNSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzIzIDAgUgovUGcgMjkzIDAgUgovS1sxNiAzMjYg
MCBSIF0KPj4KZW5kb2JqCgozMjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDI5
OSAwIFIKL1BnIDI5MyAwIFIKL0tbMzI0IDAgUiAzMjUgMCBSIF0KPj4KZW5kb2JqCgoyOTkgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMjk4IDAgUgovUGcgMjkzIDAgUgovS1szMDAg
MCBSIDMxMyAwIFIgMzIzIDAgUiBdCj4+CmVuZG9iagoKMjk4IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDI5MyAwIFIKL0tbMjk5IDAgUiBdCj4+CmVuZG9i
agoKMzM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDMzNiAwIFIKL1BnIDMzMyAw
IFIKL0tbMCBdCj4+CmVuZG9iagoKMTcwMCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Js
b2NrCj4+CmVuZG9iagoKMzM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQg
MCBSCi9QZyAzMzMgMCBSCi9BIDE3MDAgMCBSCi9LWzMzNyAwIFIgXQo+PgplbmRvYmoKCjM0MSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM0MCAwIFIKL1BnIDMzMyAwIFIKL0tb
MSBdCj4+CmVuZG9iagoKMzQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAz
NDAgMCBSCi9QZyAzMzMgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjM0MCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMzM5IDAgUgovUGcgMzMzIDAgUgovS1szNDEgMCBSIDM0MiAwIFIg
XQo+PgplbmRvYmoKCjM0NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM0MyAw
IFIKL1BnIDMzMyAwIFIKL0tbMyBdCj4+CmVuZG9iagoKMzQ1IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MQm9keQovUCAzNDMgMCBSCi9QZyAzMzMgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjM0
MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzM5IDAgUgovUGcgMzMzIDAgUgov
S1szNDQgMCBSIDM0NSAwIFIgXQo+PgplbmRvYmoKCjM0NyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDM0NiAwIFIKL1BnIDMzMyAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMzQ4IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzNDYgMCBSCi9QZyAzMzMgMCBSCi9L
WzYgXQo+PgplbmRvYmoKCjM0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzM5
IDAgUgovUGcgMzMzIDAgUgovS1szNDcgMCBSIDM0OCAwIFIgXQo+PgplbmRvYmoKCjM1MCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM0OSAwIFIKL1BnIDMzMyAwIFIKL0tbNyBd
Cj4+CmVuZG9iagoKMzUxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzNDkg
MCBSCi9QZyAzMzMgMCBSCi9LWzggXQo+PgplbmRvYmoKCjM0OSAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEkKL1AgMzM5IDAgUgovUGcgMzMzIDAgUgovS1szNTAgMCBSIDM1MSAwIFIgXQo+
PgplbmRvYmoKCjM1MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM1MiAwIFIK
L1BnIDMzMyAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMzU3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MYmwKL1AgMzU2IDAgUgovUGcgMzMzIDAgUgovS1sxMSBdCj4+CmVuZG9iagoKMzU4IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzNTYgMCBSCi9QZyAzMzMgMCBSCi9L
WzEyIF0KPj4KZW5kb2JqCgozNTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDM1
NSAwIFIKL1BnIDMzMyAwIFIKL0tbMzU3IDAgUiAzNTggMCBSIF0KPj4KZW5kb2JqCgozNjAgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAzNTkgMCBSCi9QZyAzMzMgMCBSCi9LWzEz
IF0KPj4KZW5kb2JqCgozNjEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDM1
OSAwIFIKL1BnIDMzMyAwIFIKL0tbMTQgXQo+PgplbmRvYmoKCjM1OSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMzU1IDAgUgovUGcgMzMzIDAgUgovS1szNjAgMCBSIDM2MSAwIFIg
XQo+PgplbmRvYmoKCjM2MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM2MiAw
IFIKL1BnIDMzMyAwIFIKL0tbMTUgXQo+PgplbmRvYmoKCjM2NCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEJvZHkKL1AgMzYyIDAgUgovUGcgMzMzIDAgUgovS1sxNiBdCj4+CmVuZG9iagoK
MzYyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAzNTUgMCBSCi9QZyAzMzMgMCBS
Ci9LWzM2MyAwIFIgMzY0IDAgUiBdCj4+CmVuZG9iagoKMzU1IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MCi9QIDM1NCAwIFIKL1BnIDMzMyAwIFIKL0tbMzU2IDAgUiAzNTkgMCBSIDM2MiAw
IFIgXQo+PgplbmRvYmoKCjM1NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MzUyIDAgUgovUGcgMzMzIDAgUgovS1sxMCAzNTUgMCBSIF0KPj4KZW5kb2JqCgozNTIgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDMzOSAwIFIKL1BnIDMzMyAwIFIKL0tbMzUzIDAg
UiAzNTQgMCBSIF0KPj4KZW5kb2JqCgozMzkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wK
L1AgMzM4IDAgUgovUGcgMzMzIDAgUgovS1szNDAgMCBSIDM0MyAwIFIgMzQ2IDAgUiAzNDkgMCBS
IDM1MiAwIFIgXQo+PgplbmRvYmoKCjMzOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2
Ci9QIDE2ODQgMCBSCi9QZyAzMzMgMCBSCi9LWzMzOSAwIFIgXQo+PgplbmRvYmoKCjM2OSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAzNjggMCBSCi9QZyAzNjUgMCBSCi9LWzAgXQo+
PgplbmRvYmoKCjE3MDEgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRv
YmoKCjM2OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgMzY1
IDAgUgovQSAxNzAxIDAgUgovS1szNjkgMCBSIF0KPj4KZW5kb2JqCgozNzMgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAzNzIgMCBSCi9QZyAzNjUgMCBSCi9LWzEgXQo+PgplbmRv
YmoKCjM3NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzcyIDAgUgovUGcg
MzY1IDAgUgovS1syIF0KPj4KZW5kb2JqCgozNzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xJCi9QIDM3MSAwIFIKL1BnIDM2NSAwIFIKL0tbMzczIDAgUiAzNzQgMCBSIF0KPj4KZW5kb2Jq
CgozNzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAzNzUgMCBSCi9QZyAzNjUg
MCBSCi9LWzMgXQo+PgplbmRvYmoKCjM3NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJv
ZHkKL1AgMzc1IDAgUgovUGcgMzY1IDAgUgovS1s0IF0KPj4KZW5kb2JqCgozNzUgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDM3MSAwIFIKL1BnIDM2NSAwIFIKL0tbMzc2IDAgUiAz
NzcgMCBSIF0KPj4KZW5kb2JqCgozNzkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAov
UCAzNzggMCBSCi9QZyAzNjUgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjM4MCAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzc4IDAgUgovUGcgMzY1IDAgUgovS1s2IF0KPj4KZW5k
b2JqCgozNzggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDM3MSAwIFIKL1BnIDM2
NSAwIFIKL0tbMzc5IDAgUiAzODAgMCBSIF0KPj4KZW5kb2JqCgozNzEgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0wKL1AgMzcwIDAgUgovUGcgMzY1IDAgUgovS1szNzIgMCBSIDM3NSAwIFIg
Mzc4IDAgUiBdCj4+CmVuZG9iagoKMzcwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYK
L1AgMTY4NCAwIFIKL1BnIDM2NSAwIFIKL0tbMzcxIDAgUiBdCj4+CmVuZG9iagoKMzg1IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDM4NCAwIFIKL1BnIDM4MSAwIFIKL0tbMCBdCj4+
CmVuZG9iagoKMTcwMiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9i
agoKMzg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAzODEg
MCBSCi9BIDE3MDIgMCBSCi9LWzM4NSAwIFIgXQo+PgplbmRvYmoKCjM4OSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM4OCAwIFIKL1BnIDM4MSAwIFIKL0tbMSBdCj4+CmVuZG9i
agoKMzkwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAzODggMCBSCi9QZyAz
ODEgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjM4OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgMzg3IDAgUgovUGcgMzgxIDAgUgovS1szODkgMCBSIDM5MCAwIFIgXQo+PgplbmRvYmoK
CjM5MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDM5MSAwIFIKL1BnIDM4MSAw
IFIKL0tbMyBdCj4+CmVuZG9iagoKMzkzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCAzOTEgMCBSCi9QZyAzODEgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjM5MSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzg3IDAgUgovUGcgMzgxIDAgUgovS1szOTIgMCBSIDM5
MyAwIFIgXQo+PgplbmRvYmoKCjM5NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDM5NCAwIFIKL1BnIDM4MSAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMzk2IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAzOTQgMCBSCi9QZyAzODEgMCBSCi9LWzYgXQo+PgplbmRv
YmoKCjM5NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzg3IDAgUgovUGcgMzgx
IDAgUgovS1szOTUgMCBSIDM5NiAwIFIgXQo+PgplbmRvYmoKCjM5OCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDM5NyAwIFIKL1BnIDM4MSAwIFIKL0tbNyBdCj4+CmVuZG9iagoK
NDAyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNDAxIDAgUgovUGcgMzgxIDAg
UgovS1s5IF0KPj4KZW5kb2JqCgo0MDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDQwMSAwIFIKL1BnIDM4MSAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjQwMSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNDAwIDAgUgovUGcgMzgxIDAgUgovS1s0MDIgMCBSIDQw
MyAwIFIgXQo+PgplbmRvYmoKCjQwNSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDQwNCAwIFIKL1BnIDM4MSAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjQwNiAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDA0IDAgUgovUGcgMzgxIDAgUgovS1sxMiBdCj4+CmVu
ZG9iagoKNDA0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA0MDAgMCBSCi9QZyAz
ODEgMCBSCi9LWzQwNSAwIFIgNDA2IDAgUiBdCj4+CmVuZG9iagoKNDA4IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MYmwKL1AgNDA3IDAgUgovUGcgMzgxIDAgUgovS1sxMyBdCj4+CmVuZG9i
agoKNDA5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA0MDcgMCBSCi9QZyAz
ODEgMCBSCi9LWzE0IF0KPj4KZW5kb2JqCgo0MDcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xJCi9QIDQwMCAwIFIKL1BnIDM4MSAwIFIKL0tbNDA4IDAgUiA0MDkgMCBSIF0KPj4KZW5kb2Jq
Cgo0MTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA0MTAgMCBSCi9QZyAzODEg
MCBSCi9LWzE1IF0KPj4KZW5kb2JqCgo0MTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDQxMCAwIFIKL1BnIDM4MSAwIFIKL0tbMTYgXQo+PgplbmRvYmoKCjQxMCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNDAwIDAgUgovUGcgMzgxIDAgUgovS1s0MTEgMCBS
IDQxMiAwIFIgXQo+PgplbmRvYmoKCjQwMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAov
UCAzOTkgMCBSCi9QZyAzODEgMCBSCi9LWzQwMSAwIFIgNDA0IDAgUiA0MDcgMCBSIDQxMCAwIFIg
XQo+PgplbmRvYmoKCjM5OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMzk3
IDAgUgovUGcgMzgxIDAgUgovS1s4IDQwMCAwIFIgXQo+PgplbmRvYmoKCjM5NyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMzg3IDAgUgovUGcgMzgxIDAgUgovS1szOTggMCBSIDM5
OSAwIFIgXQo+PgplbmRvYmoKCjM4NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAz
ODYgMCBSCi9QZyAzODEgMCBSCi9LWzM4OCAwIFIgMzkxIDAgUiAzOTQgMCBSIDM5NyAwIFIgXQo+
PgplbmRvYmoKCjM4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBS
Ci9QZyAzODEgMCBSCi9LWzM4NyAwIFIgXQo+PgplbmRvYmoKCjQxNyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCA0MTYgMCBSCi9QZyA0MTMgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3
MDMgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjQxNiAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNDEzIDAgUgovQSAxNzAz
IDAgUgovS1s0MTcgMCBSIF0KPj4KZW5kb2JqCgo0MjEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCA0MjAgMCBSCi9QZyA0MTMgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjQyMiAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDIwIDAgUgovUGcgNDEzIDAgUgovS1sy
IF0KPj4KZW5kb2JqCgo0MjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDQxOSAw
IFIKL1BnIDQxMyAwIFIKL0tbNDIxIDAgUiA0MjIgMCBSIF0KPj4KZW5kb2JqCgo0MjQgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA0MjMgMCBSCi9QZyA0MTMgMCBSCi9LWzMgXQo+
PgplbmRvYmoKCjQyNSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDIzIDAg
UgovUGcgNDEzIDAgUgovS1s0IF0KPj4KZW5kb2JqCgo0MjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDQxOSAwIFIKL1BnIDQxMyAwIFIKL0tbNDI0IDAgUiA0MjUgMCBSIF0KPj4K
ZW5kb2JqCgo0MjcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA0MjYgMCBSCi9Q
ZyA0MTMgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjQyOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEJvZHkKL1AgNDI2IDAgUgovUGcgNDEzIDAgUgovS1s2IF0KPj4KZW5kb2JqCgo0MjYgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDQxOSAwIFIKL1BnIDQxMyAwIFIKL0tbNDI3
IDAgUiA0MjggMCBSIF0KPj4KZW5kb2JqCgo0MzAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCA0MjkgMCBSCi9QZyA0MTMgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjQzMSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDI5IDAgUgovUGcgNDEzIDAgUgovS1s4IF0K
Pj4KZW5kb2JqCgo0MjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDQxOSAwIFIK
L1BnIDQxMyAwIFIKL0tbNDMwIDAgUiA0MzEgMCBSIF0KPj4KZW5kb2JqCgo0MzMgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA0MzIgMCBSCi9QZyA0MTMgMCBSCi9LWzkgXQo+Pgpl
bmRvYmoKCjQzNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNDMyIDAgUgov
UGcgNDEzIDAgUgovS1sxMCBdCj4+CmVuZG9iagoKNDMyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MSQovUCA0MTkgMCBSCi9QZyA0MTMgMCBSCi9LWzQzMyAwIFIgNDM0IDAgUiBdCj4+CmVu
ZG9iagoKNDM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNDM1IDAgUgovUGcg
NDEzIDAgUgovS1sxMSBdCj4+CmVuZG9iagoKNDM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MQm9keQovUCA0MzUgMCBSCi9QZyA0MTMgMCBSCi9LWzEyIF0KPj4KZW5kb2JqCgo0MzUgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDQxOSAwIFIKL1BnIDQxMyAwIFIKL0tbNDM2
IDAgUiA0MzcgMCBSIF0KPj4KZW5kb2JqCgo0MTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0wKL1AgNDE4IDAgUgovUGcgNDEzIDAgUgovS1s0MjAgMCBSIDQyMyAwIFIgNDI2IDAgUiA0Mjkg
MCBSIDQzMiAwIFIgNDM1IDAgUiBdCj4+CmVuZG9iagoKNDE4IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDQxMyAwIFIKL0tbNDE5IDAgUiBdCj4+CmVuZG9i
agoKNDQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0MSAwIFIKL1BnIDQzOCAw
IFIKL0tbMCBdCj4+CmVuZG9iagoKMTcwNCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Js
b2NrCj4+CmVuZG9iagoKNDQxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQg
MCBSCi9QZyA0MzggMCBSCi9BIDE3MDQgMCBSCi9LWzQ0MiAwIFIgXQo+PgplbmRvYmoKCjQ0MyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA0MzggMCBSCj4+
CmVuZG9iagoKNDQ0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIK
L1BnIDQzOCAwIFIKPj4KZW5kb2JqCgo0NDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0Rp
dgovUCAxNjg0IDAgUgovUGcgNDM4IDAgUgo+PgplbmRvYmoKCjQ0NyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzEgXQo+PgplbmRvYmoKCjQ0
OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9L
WzIgXQo+PgplbmRvYmoKCjQ0OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYg
MCBSCi9QZyA0MzggMCBSCi9LWzMgXQo+PgplbmRvYmoKCjQ1MCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzQgXQo+PgplbmRvYmoKCjQ1MSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzUg
XQo+PgplbmRvYmoKCjQ1MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBS
Ci9QZyA0MzggMCBSCi9LWzYgXQo+PgplbmRvYmoKCjQ1MyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzcgXQo+PgplbmRvYmoKCjQ1NCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzggXQo+
PgplbmRvYmoKCjQ1NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA0NDYgMCBSCi9Q
ZyA0MzggMCBSCi9LWzkgXQo+PgplbmRvYmoKCjQ1NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzEwIF0KPj4KZW5kb2JqCgo0NTcgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1sxMSBdCj4+
CmVuZG9iagoKNDU4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMTIgXQo+PgplbmRvYmoKCjQ1OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgo0NjAgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1sxNCBdCj4+
CmVuZG9iagoKNDYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMTUgXQo+PgplbmRvYmoKCjQ2MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzE2IF0KPj4KZW5kb2JqCgo0NjMgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1sxNyBdCj4+
CmVuZG9iagoKNDY0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjQ2NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzE5IF0KPj4KZW5kb2JqCgo0NjYgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1syMCBdCj4+
CmVuZG9iagoKNDY3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMjEgXQo+PgplbmRvYmoKCjQ2OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzIyIF0KPj4KZW5kb2JqCgo0NjkgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1syMyBdCj4+
CmVuZG9iagoKNDcwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMjQgXQo+PgplbmRvYmoKCjQ3MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzI1IF0KPj4KZW5kb2JqCgo0NzIgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1syNiBdCj4+
CmVuZG9iagoKNDczIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDQ0NiAwIFIKL1Bn
IDQzOCAwIFIKL0tbMjcgXQo+PgplbmRvYmoKCjQ3NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCA0NDYgMCBSCi9QZyA0MzggMCBSCi9LWzI4IF0KPj4KZW5kb2JqCgo0NzUgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNDQ2IDAgUgovUGcgNDM4IDAgUgovS1syOSBdCj4+
CmVuZG9iagoKNDQ2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIK
L1BnIDQzOCAwIFIKL0tbNDQ3IDAgUiA0NDggMCBSIDQ0OSAwIFIgNDUwIDAgUiA0NTEgMCBSIDQ1
MiAwIFIgNDUzIDAgUiA0NTQgMCBSIDQ1NSAwIFIgNDU2IDAgUiA0NTcgMCBSIDQ1OCAwIFIgNDU5
IDAgUiA0NjAgMCBSIDQ2MSAwIFIgNDYyIDAgUgo0NjMgMCBSIDQ2NCAwIFIgNDY1IDAgUiA0NjYg
MCBSIDQ2NyAwIFIgNDY4IDAgUiA0NjkgMCBSIDQ3MCAwIFIgNDcxIDAgUiA0NzIgMCBSIDQ3MyAw
IFIgNDc0IDAgUiA0NzUgMCBSIF0KPj4KZW5kb2JqCgo0ODAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgNDc5IDAgUgovUGcgNDc2IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzA1IDAg
b2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo0NzkgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDQ3NiAwIFIKL0EgMTcwNSAwIFIK
L0tbNDgwIDAgUiBdCj4+CmVuZG9iagoKNDg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgNDgzIDAgUgovUGcgNDc2IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo0ODUgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDQ4MyAwIFIKL1BnIDQ3NiAwIFIKL0tbMiBdCj4+
CmVuZG9iagoKNDgzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA0ODIgMCBSCi9Q
ZyA0NzYgMCBSCi9LWzQ4NCAwIFIgNDg1IDAgUiBdCj4+CmVuZG9iagoKNDg3IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNDg2IDAgUgovUGcgNDc2IDAgUgovS1szIF0KPj4KZW5k
b2JqCgo0ODggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDQ4NiAwIFIKL1Bn
IDQ3NiAwIFIKL0tbNCBdCj4+CmVuZG9iagoKNDg2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MSQovUCA0ODIgMCBSCi9QZyA0NzYgMCBSCi9LWzQ4NyAwIFIgNDg4IDAgUiBdCj4+CmVuZG9i
agoKNDkwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNDg5IDAgUgovUGcgNDc2
IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo0OTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDQ4OSAwIFIKL1BnIDQ3NiAwIFIKL0tbNiBdCj4+CmVuZG9iagoKNDg5IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA0ODIgMCBSCi9QZyA0NzYgMCBSCi9LWzQ5MCAwIFIg
NDkxIDAgUiBdCj4+CmVuZG9iagoKNDkzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwK
L1AgNDkyIDAgUgovUGcgNDc2IDAgUgovS1s3IF0KPj4KZW5kb2JqCgo0OTQgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDQ5MiAwIFIKL1BnIDQ3NiAwIFIKL0tbOCBdCj4+CmVu
ZG9iagoKNDkyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA0ODIgMCBSCi9QZyA0
NzYgMCBSCi9LWzQ5MyAwIFIgNDk0IDAgUiBdCj4+CmVuZG9iagoKNDk2IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MYmwKL1AgNDk1IDAgUgovUGcgNDc2IDAgUgovS1s5IF0KPj4KZW5kb2Jq
Cgo0OTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDQ5NSAwIFIKL1BnIDQ3
NiAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjQ5NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgNDgyIDAgUgovUGcgNDc2IDAgUgovS1s0OTYgMCBSIDQ5NyAwIFIgXQo+PgplbmRvYmoK
CjQ5OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDQ5OCAwIFIKL1BnIDQ3NiAw
IFIKL0tbMTEgXQo+PgplbmRvYmoKCjUwMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJv
ZHkKL1AgNDk4IDAgUgovUGcgNDc2IDAgUgovS1sxMiBdCj4+CmVuZG9iagoKNDk4IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA0ODIgMCBSCi9QZyA0NzYgMCBSCi9LWzQ5OSAwIFIg
NTAwIDAgUiBdCj4+CmVuZG9iagoKNDgyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9Q
IDQ4MSAwIFIKL1BnIDQ3NiAwIFIKL0tbNDgzIDAgUiA0ODYgMCBSIDQ4OSAwIFIgNDkyIDAgUiA0
OTUgMCBSIDQ5OCAwIFIgXQo+PgplbmRvYmoKCjQ4MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA0NzYgMCBSCi9LWzQ4MiAwIFIgXQo+PgplbmRvYmoKCjUw
NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1MDQgMCBSCi9QZyA1MDEgMCBSCi9L
WzAgXQo+PgplbmRvYmoKCjE3MDYgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+
PgplbmRvYmoKCjUwNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgov
UGcgNTAxIDAgUgovQSAxNzA2IDAgUgovS1s1MDUgMCBSIF0KPj4KZW5kb2JqCgoxNzA3IDAgb2Jq
Cjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKL0JCb3hbMjgxLjY1IDQyLjIwNyA3MDMuMDIg
Mjk3LjYzN10KPj4KZW5kb2JqCgo1MDYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0ZpZ3Vy
ZQovUCAxNjg0IDAgUgovUGcgNTAxIDAgUgovQSAxNzA3IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo1
MTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA1MTAgMCBSCi9QZyA1MDEgMCBS
Ci9LWzIgXQo+PgplbmRvYmoKCjUxMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgNTEwIDAgUgovUGcgNTAxIDAgUgovS1szIF0KPj4KZW5kb2JqCgo1MTAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDUwOSAwIFIKL1BnIDUwMSAwIFIKL0tbNTExIDAgUiA1MTIg
MCBSIF0KPj4KZW5kb2JqCgo1MTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA1
MTMgMCBSCi9QZyA1MDEgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjUxNSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgNTEzIDAgUgovUGcgNTAxIDAgUgovS1s1IF0KPj4KZW5kb2Jq
Cgo1MTMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDUwOSAwIFIKL1BnIDUwMSAw
IFIKL0tbNTE0IDAgUiA1MTUgMCBSIF0KPj4KZW5kb2JqCgo1MTcgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xibAovUCA1MTYgMCBSCi9QZyA1MDEgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjUx
OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNTE2IDAgUgovUGcgNTAxIDAg
UgovS1s3IF0KPj4KZW5kb2JqCgo1MTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDUwOSAwIFIKL1BnIDUwMSAwIFIKL0tbNTE3IDAgUiA1MTggMCBSIF0KPj4KZW5kb2JqCgo1MDkg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgNTA4IDAgUgovUGcgNTAxIDAgUgovS1s1
MTAgMCBSIDUxMyAwIFIgNTE2IDAgUiBdCj4+CmVuZG9iagoKNTA4IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDUwMSAwIFIKL0tbNTA5IDAgUiBdCj4+CmVu
ZG9iagoKNTI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDUyMyAwIFIKL1BnIDUy
MCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTcwOCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50
L0Jsb2NrCj4+CmVuZG9iagoKNTIzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2
ODQgMCBSCi9QZyA1MjAgMCBSCi9BIDE3MDggMCBSCi9LWzUyNCAwIFIgXQo+PgplbmRvYmoKCjUy
OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDUyNyAwIFIKL1BnIDUyMCAwIFIK
L0tbMSBdCj4+CmVuZG9iagoKNTI5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCA1MjcgMCBSCi9QZyA1MjAgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjUyNyAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgNTI2IDAgUgovUGcgNTIwIDAgUgovS1s1MjggMCBSIDUyOSAw
IFIgXQo+PgplbmRvYmoKCjUzMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDUz
MCAwIFIKL1BnIDUyMCAwIFIKL0tbMyBdCj4+CmVuZG9iagoKNTMyIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCA1MzAgMCBSCi9QZyA1MjAgMCBSCi9LWzQgXQo+PgplbmRvYmoK
CjUzMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNTI2IDAgUgovUGcgNTIwIDAg
UgovS1s1MzEgMCBSIDUzMiAwIFIgXQo+PgplbmRvYmoKCjUzNCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDUzMyAwIFIKL1BnIDUyMCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKNTM1
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA1MzMgMCBSCi9QZyA1MjAgMCBS
Ci9LWzYgXQo+PgplbmRvYmoKCjUzMyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1Ag
NTI2IDAgUgovUGcgNTIwIDAgUgovS1s1MzQgMCBSIDUzNSAwIFIgXQo+PgplbmRvYmoKCjUzNyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDUzNiAwIFIKL1BnIDUyMCAwIFIKL0tb
NyBdCj4+CmVuZG9iagoKNTM4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA1
MzYgMCBSCi9QZyA1MjAgMCBSCi9LWzggXQo+PgplbmRvYmoKCjUzNiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgNTI2IDAgUgovUGcgNTIwIDAgUgovS1s1MzcgMCBSIDUzOCAwIFIg
XQo+PgplbmRvYmoKCjUyNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA1MjUgMCBS
Ci9QZyA1MjAgMCBSCi9LWzUyNyAwIFIgNTMwIDAgUiA1MzMgMCBSIDUzNiAwIFIgXQo+PgplbmRv
YmoKCjUyNSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA1
MjAgMCBSCi9LWzUyNiAwIFIgXQo+PgplbmRvYmoKCjU0MyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvUAovUCA1NDIgMCBSCi9QZyA1MzkgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MDkgMCBv
YmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjU0MiAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNTM5IDAgUgovQSAxNzA5IDAgUgov
S1s1NDMgMCBSIF0KPj4KZW5kb2JqCgo1NDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AK
L1AgNTQ3IDAgUgovUGcgNTQ0IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzEwIDAgb2JqCjw8L08v
TGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo1NDcgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDU0NCAwIFIKL0EgMTcxMCAwIFIKL0tbNTQ4IDAg
UiBdCj4+CmVuZG9iagoKNTUyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNTUx
IDAgUgovUGcgNTQ0IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo1NTMgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDU1MSAwIFIKL1BnIDU0NCAwIFIKL0tbMiBdCj4+CmVuZG9iagoK
NTUxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA1NTAgMCBSCi9QZyA1NDQgMCBS
Ci9LWzU1MiAwIFIgNTUzIDAgUiBdCj4+CmVuZG9iagoKNTU1IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MYmwKL1AgNTU0IDAgUgovUGcgNTQ0IDAgUgovS1szIF0KPj4KZW5kb2JqCgo1NTYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDU1NCAwIFIKL1BnIDU0NCAwIFIK
L0tbNCBdCj4+CmVuZG9iagoKNTU0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA1
NTAgMCBSCi9QZyA1NDQgMCBSCi9LWzU1NSAwIFIgNTU2IDAgUiBdCj4+CmVuZG9iagoKNTU4IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNTU3IDAgUgovUGcgNTQ0IDAgUgovS1s1
IF0KPj4KZW5kb2JqCgo1NTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDU1
NyAwIFIKL1BnIDU0NCAwIFIKL0tbNiBdCj4+CmVuZG9iagoKNTU3IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MSQovUCA1NTAgMCBSCi9QZyA1NDQgMCBSCi9LWzU1OCAwIFIgNTU5IDAgUiBd
Cj4+CmVuZG9iagoKNTYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNTYwIDAg
UgovUGcgNTQ0IDAgUgovS1s3IF0KPj4KZW5kb2JqCgo1NjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xCb2R5Ci9QIDU2MCAwIFIKL1BnIDU0NCAwIFIKL0tbOCBdCj4+CmVuZG9iagoKNTYw
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA1NTAgMCBSCi9QZyA1NDQgMCBSCi9L
WzU2MSAwIFIgNTYyIDAgUiBdCj4+CmVuZG9iagoKNTUwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MCi9QIDU0OSAwIFIKL1BnIDU0NCAwIFIKL0tbNTUxIDAgUiA1NTQgMCBSIDU1NyAwIFIg
NTYwIDAgUiBdCj4+CmVuZG9iagoKNTQ5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYK
L1AgMTY4NCAwIFIKL1BnIDU0NCAwIFIKL0tbNTUwIDAgUiBdCj4+CmVuZG9iagoKNTY3IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDU2NiAwIFIKL1BnIDU2MyAwIFIKL0tbMCBdCj4+
CmVuZG9iagoKMTcxMSAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9i
agoKNTY2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA1NjMg
MCBSCi9BIDE3MTEgMCBSCi9LWzU2NyAwIFIgXQo+PgplbmRvYmoKCjU2OSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBSCi9LWzIgXQo+PgplbmRvYmoK
CjU3MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBS
Ci9LWzQgXQo+PgplbmRvYmoKCjU3MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1
NjggMCBSCi9QZyA1NjMgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjU3MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBSCi9LWzggXQo+PgplbmRvYmoKCjU3
MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBSCi9L
WzEwIF0KPj4KZW5kb2JqCgo1NzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTY4
IDAgUgovUGcgNTYzIDAgUgovS1sxMSBdCj4+CmVuZG9iagoKNTc1IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDU2OCAwIFIKL1BnIDU2MyAwIFIKL0tbMTMgXQo+PgplbmRvYmoKCjU3
NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBSCi9L
WzE1IF0KPj4KZW5kb2JqCgo1NzcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTY4
IDAgUgovUGcgNTYzIDAgUgovS1sxNyBdCj4+CmVuZG9iagoKNTc4IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDU2OCAwIFIKL1BnIDU2MyAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjU3
OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1NjggMCBSCi9QZyA1NjMgMCBSCi9L
WzIwIF0KPj4KZW5kb2JqCgo1ODAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTY4
IDAgUgovUGcgNTYzIDAgUgovS1syMSBdCj4+CmVuZG9iagoKNTgxIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDU2OCAwIFIKL1BnIDU2MyAwIFIKPj4KZW5kb2JqCgoxNzEyIDAgb2Jq
Cjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKL0JCb3hbMTMwLjMzNyA1My40ODkgNTQwLjg1
IDI5MS4yNTldCj4+CmVuZG9iagoKNTY4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9GaWd1
cmUKL1AgMTY4NCAwIFIKL1BnIDU2MyAwIFIKL0EgMTcxMiAwIFIKL0tbMSA1NjkgMCBSIDMgNTcw
IDAgUiA1IDU3MSAwIFIgNyA1NzIgMCBSIDkgNTczIDAgUiA1NzQgMCBSIDEyIDU3NSAwIFIgMTQg
NTc2IDAgUiAxNiA1NzcgMCBSIDU3OCAwIFIgMTkgNTc5IDAgUiA1ODAgMCBSIDIyIDU4MSAwIFIg
XQo+PgplbmRvYmoKCjU4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1ODUgMCBS
Ci9QZyA1ODIgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MTMgMCBvYmoKPDwvTy9MYXlvdXQKL1Bs
YWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjU4NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
SAovUCAxNjg0IDAgUgovUGcgNTgyIDAgUgovQSAxNzEzIDAgUgovS1s1ODYgMCBSIF0KPj4KZW5k
b2JqCgo1ODggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTg3IDAgUgovUGcgNTgy
IDAgUgovS1syIF0KPj4KZW5kb2JqCgo1ODkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AK
L1AgNTg3IDAgUgovUGcgNTgyIDAgUgovS1s0IF0KPj4KZW5kb2JqCgo1OTAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTg3IDAgUgovUGcgNTgyIDAgUgovS1s2IF0KPj4KZW5kb2Jq
Cgo1OTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNTg3IDAgUgovUGcgNTgyIDAg
UgovS1s4IF0KPj4KZW5kb2JqCgo1OTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
NTg3IDAgUgovUGcgNTgyIDAgUgovS1sxMCBdCj4+CmVuZG9iagoKNTkzIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9QCi9QIDU4NyAwIFIKL1BnIDU4MiAwIFIKL0tbMTIgXQo+PgplbmRvYmoK
CjU5NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1ODcgMCBSCi9QZyA1ODIgMCBS
Ci9LWzE0IF0KPj4KZW5kb2JqCgo1OTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
NTg3IDAgUgovUGcgNTgyIDAgUgovS1sxNSBdCj4+CmVuZG9iagoKNTk2IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9QCi9QIDU4NyAwIFIKL1BnIDU4MiAwIFIKL0tbMTcgXQo+PgplbmRvYmoK
CjU5NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1ODcgMCBSCi9QZyA1ODIgMCBS
Ci9LWzE4IF0KPj4KZW5kb2JqCgo1OTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
NTg3IDAgUgovUGcgNTgyIDAgUgovS1syMCBdCj4+CmVuZG9iagoKNTk5IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9QCi9QIDU4NyAwIFIKL1BnIDU4MiAwIFIKL0tbMjIgXQo+PgplbmRvYmoK
CjYwMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA1ODcgMCBSCi9QZyA1ODIgMCBS
Ci9LWzI0IF0KPj4KZW5kb2JqCgo2MDEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
NTg3IDAgUgovUGcgNTgyIDAgUgovS1syNiBdCj4+CmVuZG9iagoKMTcxNCAwIG9iago8PC9PL0xh
eW91dAovUGxhY2VtZW50L0Jsb2NrCi9CQm94WzEzMC4zMzcgNjQuODI3IDU0MC44NSAzMDIuNTk3
XQo+PgplbmRvYmoKCjU4NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRmlndXJlCi9QIDE2
ODQgMCBSCi9QZyA1ODIgMCBSCi9BIDE3MTQgMCBSCi9LWzEgNTg4IDAgUiAzIDU4OSAwIFIgNSA1
OTAgMCBSIDcgNTkxIDAgUiA5IDU5MiAwIFIgMTEgNTkzIDAgUiAxMyA1OTQgMCBSIDU5NSAwIFIg
MTYgNTk2IDAgUiA1OTcgMCBSIDE5IDU5OCAwIFIgMjEgNTk5IDAgUiAyMyA2MDAgMCBSIDI1IDYw
MSAwIFIgXQo+PgplbmRvYmoKCjYwNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA2
MDUgMCBSCi9QZyA2MDIgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MTUgMCBvYmoKPDwvTy9MYXlv
dXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjYwNSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNjAyIDAgUgovQSAxNzE1IDAgUgovS1s2MDYgMCBSIF0K
Pj4KZW5kb2JqCgo2MTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA2MDkgMCBS
Ci9QZyA2MDIgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjYxMSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEJvZHkKL1AgNjA5IDAgUgovUGcgNjAyIDAgUgovS1syIF0KPj4KZW5kb2JqCgo2MDkg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDYwOCAwIFIKL1BnIDYwMiAwIFIKL0tb
NjEwIDAgUiA2MTEgMCBSIF0KPj4KZW5kb2JqCgo2MTMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCA2MTIgMCBSCi9QZyA2MDIgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjYxNCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNjEyIDAgUgovUGcgNjAyIDAgUgovS1s0
IF0KPj4KZW5kb2JqCgo2MTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDYwOCAw
IFIKL1BnIDYwMiAwIFIKL0tbNjEzIDAgUiA2MTQgMCBSIF0KPj4KZW5kb2JqCgo2MTYgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA2MTUgMCBSCi9QZyA2MDIgMCBSCi9LWzUgXQo+
PgplbmRvYmoKCjYxNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNjE1IDAg
UgovUGcgNjAyIDAgUgovS1s2IF0KPj4KZW5kb2JqCgo2MTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDYwOCAwIFIKL1BnIDYwMiAwIFIKL0tbNjE2IDAgUiA2MTcgMCBSIF0KPj4K
ZW5kb2JqCgo2MTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA2MTggMCBSCi9Q
ZyA2MDIgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjYyMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEJvZHkKL1AgNjE4IDAgUgovUGcgNjAyIDAgUgovS1s4IF0KPj4KZW5kb2JqCgo2MTggMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDYwOCAwIFIKL1BnIDYwMiAwIFIKL0tbNjE5
IDAgUiA2MjAgMCBSIF0KPj4KZW5kb2JqCgo2MDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0wKL1AgNjA3IDAgUgovUGcgNjAyIDAgUgovS1s2MDkgMCBSIDYxMiAwIFIgNjE1IDAgUiA2MTgg
MCBSIF0KPj4KZW5kb2JqCgo2MDcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAx
Njg0IDAgUgovUGcgNjAyIDAgUgovS1s2MDggMCBSIF0KPj4KZW5kb2JqCgo2MjUgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNjI0IDAgUgovUGcgNjIxIDAgUgovS1swIF0KPj4KZW5k
b2JqCgoxNzE2IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo2
MjQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDYyMSAwIFIK
L0EgMTcxNiAwIFIKL0tbNjI1IDAgUiBdCj4+CmVuZG9iagoKNjI5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgNjI4IDAgUgovUGcgNjIxIDAgUgovS1sxIF0KPj4KZW5kb2JqCgo2
MzAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDYyOCAwIFIKL1BnIDYyMSAw
IFIKL0tbMiBdCj4+CmVuZG9iagoKNjI4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCA2MjcgMCBSCi9QZyA2MjEgMCBSCi9LWzYyOSAwIFIgNjMwIDAgUiBdCj4+CmVuZG9iagoKNjMy
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNjMxIDAgUgovUGcgNjIxIDAgUgov
S1szIF0KPj4KZW5kb2JqCgo2MzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDYzMSAwIFIKL1BnIDYyMSAwIFIKL0tbNCBdCj4+CmVuZG9iagoKNjMxIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MSQovUCA2MjcgMCBSCi9QZyA2MjEgMCBSCi9LWzYzMiAwIFIgNjMzIDAg
UiBdCj4+CmVuZG9iagoKNjM1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNjM0
IDAgUgovUGcgNjIxIDAgUgovS1s1IF0KPj4KZW5kb2JqCgo2MzYgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDYzNCAwIFIKL1BnIDYyMSAwIFIKL0tbNiBdCj4+CmVuZG9iagoK
NjM0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2MjcgMCBSCi9QZyA2MjEgMCBS
Ci9LWzYzNSAwIFIgNjM2IDAgUiBdCj4+CmVuZG9iagoKNjI3IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MCi9QIDYyNiAwIFIKL1BnIDYyMSAwIFIKL0tbNjI4IDAgUiA2MzEgMCBSIDYzNCAw
IFIgXQo+PgplbmRvYmoKCjYyNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2
ODQgMCBSCi9QZyA2MjEgMCBSCi9LWzYyNyAwIFIgXQo+PgplbmRvYmoKCjY0MSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvUAovUCA2NDAgMCBSCi9QZyA2MzcgMCBSCi9LWzAgXQo+PgplbmRv
YmoKCjE3MTcgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjY0
MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNjM3IDAgUgov
QSAxNzE3IDAgUgovS1s2NDEgMCBSIF0KPj4KZW5kb2JqCgo2NDYgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL1AKL1AgNjQ1IDAgUgovUGcgNjQyIDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzE4
IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo2NDUgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDY0MiAwIFIKL0EgMTcxOCAw
IFIKL0tbNjQ2IDAgUiBdCj4+CmVuZG9iagoKNjUwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgNjQ5IDAgUgovUGcgNjQyIDAgUgovS1sxIF0KPj4KZW5kb2JqCgo2NTEgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY0OSAwIFIKL1BnIDY0MiAwIFIKL0tbMiBd
Cj4+CmVuZG9iagoKNjQ5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2NDggMCBS
Ci9QZyA2NDIgMCBSCi9LWzY1MCAwIFIgNjUxIDAgUiBdCj4+CmVuZG9iagoKNjUzIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNjUyIDAgUgovUGcgNjQyIDAgUgovS1szIF0KPj4K
ZW5kb2JqCgo2NTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY1MiAwIFIK
L1BnIDY0MiAwIFIKL0tbNCBdCj4+CmVuZG9iagoKNjUyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MSQovUCA2NDggMCBSCi9QZyA2NDIgMCBSCi9LWzY1MyAwIFIgNjU0IDAgUiBdCj4+CmVu
ZG9iagoKNjU2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNjU1IDAgUgovUGcg
NjQyIDAgUgovS1s1IF0KPj4KZW5kb2JqCgo2NTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDY1NSAwIFIKL1BnIDY0MiAwIFIKL0tbNiBdCj4+CmVuZG9iagoKNjU1IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2NDggMCBSCi9QZyA2NDIgMCBSCi9LWzY1NiAw
IFIgNjU3IDAgUiBdCj4+CmVuZG9iagoKNjU5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgNjU4IDAgUgovUGcgNjQyIDAgUgovS1s3IF0KPj4KZW5kb2JqCgo2NjAgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY1OCAwIFIKL1BnIDY0MiAwIFIKL0tbOCBdCj4+
CmVuZG9iagoKNjU4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2NDggMCBSCi9Q
ZyA2NDIgMCBSCi9LWzY1OSAwIFIgNjYwIDAgUiBdCj4+CmVuZG9iagoKNjYyIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNjYxIDAgUgovUGcgNjQyIDAgUgovS1s5IF0KPj4KZW5k
b2JqCgo2NjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY2MSAwIFIKL1Bn
IDY0MiAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjY2MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEkKL1AgNjQ4IDAgUgovUGcgNjQyIDAgUgovS1s2NjIgMCBSIDY2MyAwIFIgXQo+PgplbmRv
YmoKCjY2NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDY2NCAwIFIKL1BnIDY0
MiAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjY2NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgNjY0IDAgUgovUGcgNjQyIDAgUgovS1sxMiBdCj4+CmVuZG9iagoKNjY0IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA2NDggMCBSCi9QZyA2NDIgMCBSCi9LWzY2NSAw
IFIgNjY2IDAgUiBdCj4+CmVuZG9iagoKNjQ4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
Ci9QIDY0NyAwIFIKL1BnIDY0MiAwIFIKL0tbNjQ5IDAgUiA2NTIgMCBSIDY1NSAwIFIgNjU4IDAg
UiA2NjEgMCBSIDY2NCAwIFIgXQo+PgplbmRvYmoKCjY0NyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA2NDIgMCBSCi9LWzY0OCAwIFIgXQo+PgplbmRvYmoK
CjY3MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA2NzAgMCBSCi9QZyA2NjcgMCBS
Ci9LWzAgXQo+PgplbmRvYmoKCjE3MTkgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9j
awo+PgplbmRvYmoKCjY3MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAg
UgovUGcgNjY3IDAgUgovQSAxNzE5IDAgUgovS1s2NzEgMCBSIF0KPj4KZW5kb2JqCgo2NzUgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA2NzQgMCBSCi9QZyA2NjcgMCBSCi9LWzEg
XQo+PgplbmRvYmoKCjY3NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNjc0
IDAgUgovUGcgNjY3IDAgUgovS1syIF0KPj4KZW5kb2JqCgo2NzQgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xJCi9QIDY3MyAwIFIKL1BnIDY2NyAwIFIKL0tbNjc1IDAgUiA2NzYgMCBSIF0K
Pj4KZW5kb2JqCgo2NzggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA2NzcgMCBS
Ci9QZyA2NjcgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjY4MiAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDY4MSAwIFIKL1BnIDY2NyAwIFIKL0tbNSBdCj4+CmVuZG9iagoKNjgzIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA2ODEgMCBSCi9QZyA2NjcgMCBSCi9L
WzYgXQo+PgplbmRvYmoKCjY4MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjgw
IDAgUgovUGcgNjY3IDAgUgovS1s2ODIgMCBSIDY4MyAwIFIgXQo+PgplbmRvYmoKCjY4NSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDY4NCAwIFIKL1BnIDY2NyAwIFIKL0tbNyBd
Cj4+CmVuZG9iagoKNjg2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA2ODQg
MCBSCi9QZyA2NjcgMCBSCi9LWzggXQo+PgplbmRvYmoKCjY4NCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEkKL1AgNjgwIDAgUgovUGcgNjY3IDAgUgovS1s2ODUgMCBSIDY4NiAwIFIgXQo+
PgplbmRvYmoKCjY4OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDY4NyAwIFIK
L1BnIDY2NyAwIFIKL0tbOSBdCj4+CmVuZG9iagoKNjg5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MQm9keQovUCA2ODcgMCBSCi9QZyA2NjcgMCBSCi9LWzEwIF0KPj4KZW5kb2JqCgo2ODcg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDY4MCAwIFIKL1BnIDY2NyAwIFIKL0tb
Njg4IDAgUiA2ODkgMCBSIF0KPj4KZW5kb2JqCgo2OTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCA2OTAgMCBSCi9QZyA2NjcgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgo2OTIgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDY5MCAwIFIKL1BnIDY2NyAwIFIKL0tb
MTIgXQo+PgplbmRvYmoKCjY5MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjgw
IDAgUgovUGcgNjY3IDAgUgovS1s2OTEgMCBSIDY5MiAwIFIgXQo+PgplbmRvYmoKCjY4MCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA2NzkgMCBSCi9QZyA2NjcgMCBSCi9LWzY4MSAw
IFIgNjg0IDAgUiA2ODcgMCBSIDY5MCAwIFIgXQo+PgplbmRvYmoKCjY3OSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNjc3IDAgUgovUGcgNjY3IDAgUgovS1s0IDY4MCAwIFIg
XQo+PgplbmRvYmoKCjY3NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjczIDAg
UgovUGcgNjY3IDAgUgovS1s2NzggMCBSIDY3OSAwIFIgXQo+PgplbmRvYmoKCjY3MyAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA2NzIgMCBSCi9QZyA2NjcgMCBSCi9LWzY3NCAwIFIg
Njc3IDAgUiBdCj4+CmVuZG9iagoKNjcyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYK
L1AgMTY4NCAwIFIKL1BnIDY2NyAwIFIKL0tbNjczIDAgUiBdCj4+CmVuZG9iagoKNjk3IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDY5NiAwIFIKL1BnIDY5MyAwIFIKL0tbMCBdCj4+
CmVuZG9iagoKMTcyMCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9i
agoKNjk2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA2OTMg
MCBSCi9BIDE3MjAgMCBSCi9LWzY5NyAwIFIgXQo+PgplbmRvYmoKCjcwMSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTGJsCi9QIDcwMCAwIFIKL1BnIDY5MyAwIFIKL0tbMSBdCj4+CmVuZG9i
agoKNzAyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA3MDAgMCBSCi9QZyA2
OTMgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjcwMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgNjk5IDAgUgovUGcgNjkzIDAgUgovS1s3MDEgMCBSIDcwMiAwIFIgXQo+PgplbmRvYmoK
CjcwNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDcwMyAwIFIKL1BnIDY5MyAw
IFIKL0tbMyBdCj4+CmVuZG9iagoKNzA1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCA3MDMgMCBSCi9QZyA2OTMgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjcwMyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjk5IDAgUgovUGcgNjkzIDAgUgovS1s3MDQgMCBSIDcw
NSAwIFIgXQo+PgplbmRvYmoKCjcwNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDcwNiAwIFIKL1BnIDY5MyAwIFIKL0tbNSBdCj4+CmVuZG9iagoKNzExIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MYmwKL1AgNzEwIDAgUgovUGcgNjkzIDAgUgovS1s3IF0KPj4KZW5kb2Jq
Cgo3MTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDcxMCAwIFIKL1BnIDY5
MyAwIFIKL0tbOCBdCj4+CmVuZG9iagoKNzEwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
SQovUCA3MDkgMCBSCi9QZyA2OTMgMCBSCi9LWzcxMSAwIFIgNzEyIDAgUiBdCj4+CmVuZG9iagoK
NzE0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzEzIDAgUgovUGcgNjkzIDAg
UgovS1s5IF0KPj4KZW5kb2JqCgo3MTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDcxMyAwIFIKL1BnIDY5MyAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjcxMyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNzA5IDAgUgovUGcgNjkzIDAgUgovS1s3MTQgMCBSIDcx
NSAwIFIgXQo+PgplbmRvYmoKCjcxNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDcxNiAwIFIKL1BnIDY5MyAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjcxOCAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNzE2IDAgUgovUGcgNjkzIDAgUgovS1sxMiBdCj4+CmVu
ZG9iagoKNzE2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3MDkgMCBSCi9QZyA2
OTMgMCBSCi9LWzcxNyAwIFIgNzE4IDAgUiBdCj4+CmVuZG9iagoKNzA5IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MCi9QIDcwOCAwIFIKL1BnIDY5MyAwIFIKL0tbNzEwIDAgUiA3MTMgMCBS
IDcxNiAwIFIgXQo+PgplbmRvYmoKCjcwOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJv
ZHkKL1AgNzA2IDAgUgovUGcgNjkzIDAgUgovS1s2IDcwOSAwIFIgXQo+PgplbmRvYmoKCjcwNiAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNjk5IDAgUgovUGcgNjkzIDAgUgovS1s3
MDcgMCBSIDcwOCAwIFIgXQo+PgplbmRvYmoKCjY5OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTAovUCA2OTggMCBSCi9QZyA2OTMgMCBSCi9LWzcwMCAwIFIgNzAzIDAgUiA3MDYgMCBSIF0K
Pj4KZW5kb2JqCgo2OTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAg
UgovUGcgNjkzIDAgUgovS1s2OTkgMCBSIF0KPj4KZW5kb2JqCgo3MjMgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL1AKL1AgNzIyIDAgUgovUGcgNzE5IDAgUgovS1swIF0KPj4KZW5kb2JqCgox
NzIxIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo3MjIgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDcxOSAwIFIKL0EgMTcy
MSAwIFIKL0tbNzIzIDAgUiBdCj4+CmVuZG9iagoKNzI3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MYmwKL1AgNzI2IDAgUgovUGcgNzE5IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo3MjggMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDcyNiAwIFIKL1BnIDcxOSAwIFIKL0tb
MiBdCj4+CmVuZG9iagoKNzI2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3MjUg
MCBSCi9QZyA3MTkgMCBSCi9LWzcyNyAwIFIgNzI4IDAgUiBdCj4+CmVuZG9iagoKNzMwIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzI5IDAgUgovUGcgNzE5IDAgUgovS1szIF0K
Pj4KZW5kb2JqCgo3MzEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDcyOSAw
IFIKL1BnIDcxOSAwIFIKL0tbNCBdCj4+CmVuZG9iagoKNzI5IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCA3MjUgMCBSCi9QZyA3MTkgMCBSCi9LWzczMCAwIFIgNzMxIDAgUiBdCj4+
CmVuZG9iagoKNzMzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzMyIDAgUgov
UGcgNzE5IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo3MzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xCb2R5Ci9QIDczMiAwIFIKL1BnIDcxOSAwIFIKL0tbNiBdCj4+CmVuZG9iagoKNzMyIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3MjUgMCBSCi9QZyA3MTkgMCBSCi9LWzcz
MyAwIFIgNzM0IDAgUiBdCj4+CmVuZG9iagoKNzM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgNzM1IDAgUgovUGcgNzE5IDAgUgovS1s3IF0KPj4KZW5kb2JqCgo3MzcgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDczNSAwIFIKL1BnIDcxOSAwIFIKL0tbOCBd
Cj4+CmVuZG9iagoKNzM1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3MjUgMCBS
Ci9QZyA3MTkgMCBSCi9LWzczNiAwIFIgNzM3IDAgUiBdCj4+CmVuZG9iagoKNzI1IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDcyNCAwIFIKL1BnIDcxOSAwIFIKL0tbNzI2IDAgUiA3
MjkgMCBSIDczMiAwIFIgNzM1IDAgUiBdCj4+CmVuZG9iagoKNzI0IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDcxOSAwIFIKL0tbNzI1IDAgUiBdCj4+CmVu
ZG9iagoKNzQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDc0MSAwIFIKL1BnIDcz
OCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTcyMiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50
L0Jsb2NrCj4+CmVuZG9iagoKNzQxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2
ODQgMCBSCi9QZyA3MzggMCBSCi9BIDE3MjIgMCBSCi9LWzc0MiAwIFIgXQo+PgplbmRvYmoKCjc0
NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDc0NSAwIFIKL1BnIDczOCAwIFIK
L0tbMSBdCj4+CmVuZG9iagoKNzUwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
NzQ5IDAgUgovUGcgNzM4IDAgUgovS1szIF0KPj4KZW5kb2JqCgo3NTEgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDc0OSAwIFIKL1BnIDczOCAwIFIKL0tbNCBdCj4+CmVuZG9i
agoKNzQ5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3NDggMCBSCi9QZyA3Mzgg
MCBSCi9LWzc1MCAwIFIgNzUxIDAgUiBdCj4+CmVuZG9iagoKNzUzIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgNzUyIDAgUgovUGcgNzM4IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo3
NTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDc1MiAwIFIKL1BnIDczOCAw
IFIKL0tbNiBdCj4+CmVuZG9iagoKNzUyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCA3NDggMCBSCi9QZyA3MzggMCBSCi9LWzc1MyAwIFIgNzU0IDAgUiBdCj4+CmVuZG9iagoKNzU2
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzU1IDAgUgovUGcgNzM4IDAgUgov
S1s3IF0KPj4KZW5kb2JqCgo3NTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDc1NSAwIFIKL1BnIDczOCAwIFIKL0tbOCBdCj4+CmVuZG9iagoKNzU1IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MSQovUCA3NDggMCBSCi9QZyA3MzggMCBSCi9LWzc1NiAwIFIgNzU3IDAg
UiBdCj4+CmVuZG9iagoKNzU5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzU4
IDAgUgovUGcgNzM4IDAgUgovS1s5IF0KPj4KZW5kb2JqCgo3NjAgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDc1OCAwIFIKL1BnIDczOCAwIFIKL0tbMTAgXQo+PgplbmRvYmoK
Cjc1OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNzQ4IDAgUgovUGcgNzM4IDAg
UgovS1s3NTkgMCBSIDc2MCAwIFIgXQo+PgplbmRvYmoKCjc0OCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTAovUCA3NDcgMCBSCi9QZyA3MzggMCBSCi9LWzc0OSAwIFIgNzUyIDAgUiA3NTUg
MCBSIDc1OCAwIFIgXQo+PgplbmRvYmoKCjc0NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgNzQ1IDAgUgovUGcgNzM4IDAgUgovS1syIDc0OCAwIFIgXQo+PgplbmRvYmoKCjc0
NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNzQ0IDAgUgovUGcgNzM4IDAgUgov
S1s3NDYgMCBSIDc0NyAwIFIgXQo+PgplbmRvYmoKCjc2MiAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDc2MSAwIFIKL1BnIDczOCAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjc2MyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNzYxIDAgUgovUGcgNzM4IDAgUgov
S1sxMiBdCj4+CmVuZG9iagoKNzYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3
NDQgMCBSCi9QZyA3MzggMCBSCi9LWzc2MiAwIFIgNzYzIDAgUiBdCj4+CmVuZG9iagoKNzQ0IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDc0MyAwIFIKL1BnIDczOCAwIFIKL0tbNzQ1
IDAgUiA3NjEgMCBSIF0KPj4KZW5kb2JqCgo3NDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0RpdgovUCAxNjg0IDAgUgovUGcgNzM4IDAgUgovS1s3NDQgMCBSIF0KPj4KZW5kb2JqCgo3Njgg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgNzY3IDAgUgovUGcgNzY0IDAgUgovS1sw
IF0KPj4KZW5kb2JqCgoxNzIzIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4K
ZW5kb2JqCgo3NjcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1Bn
IDc2NCAwIFIKL0EgMTcyMyAwIFIKL0tbNzY4IDAgUiBdCj4+CmVuZG9iagoKNzcyIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzcxIDAgUgovUGcgNzY0IDAgUgovS1sxIF0KPj4K
ZW5kb2JqCgo3NzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDc3MSAwIFIK
L1BnIDc2NCAwIFIKL0tbMiBdCj4+CmVuZG9iagoKNzcxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MSQovUCA3NzAgMCBSCi9QZyA3NjQgMCBSCi9LWzc3MiAwIFIgNzczIDAgUiBdCj4+CmVu
ZG9iagoKNzc1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzc0IDAgUgovUGcg
NzY0IDAgUgovS1szIF0KPj4KZW5kb2JqCgo3NzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDc3NCAwIFIKL1BnIDc2NCAwIFIKL0tbNCBdCj4+CmVuZG9iagoKNzc0IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3NzAgMCBSCi9QZyA3NjQgMCBSCi9LWzc3NSAw
IFIgNzc2IDAgUiBdCj4+CmVuZG9iagoKNzc4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgNzc3IDAgUgovUGcgNzY0IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo3NzkgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDc3NyAwIFIKL1BnIDc2NCAwIFIKL0tbNiBdCj4+
CmVuZG9iagoKNzc3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3NzAgMCBSCi9Q
ZyA3NjQgMCBSCi9LWzc3OCAwIFIgNzc5IDAgUiBdCj4+CmVuZG9iagoKNzgxIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzgwIDAgUgovUGcgNzY0IDAgUgovS1s3IF0KPj4KZW5k
b2JqCgo3ODIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDc4MCAwIFIKL1Bn
IDc2NCAwIFIKL0tbOCBdCj4+CmVuZG9iagoKNzgwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MSQovUCA3NzAgMCBSCi9QZyA3NjQgMCBSCi9LWzc4MSAwIFIgNzgyIDAgUiBdCj4+CmVuZG9i
agoKNzg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgNzgzIDAgUgovUGcgNzY0
IDAgUgovS1s5IF0KPj4KZW5kb2JqCgo3ODUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDc4MyAwIFIKL1BnIDc2NCAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjc4MyAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgNzcwIDAgUgovUGcgNzY0IDAgUgovS1s3ODQgMCBS
IDc4NSAwIFIgXQo+PgplbmRvYmoKCjc4NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDc4NiAwIFIKL1BnIDc2NCAwIFIKL0tbMTEgXQo+PgplbmRvYmoKCjc4OCAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNzg2IDAgUgovUGcgNzY0IDAgUgovS1sxMiBdCj4+
CmVuZG9iagoKNzg2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA3NzAgMCBSCi9Q
ZyA3NjQgMCBSCi9LWzc4NyAwIFIgNzg4IDAgUiBdCj4+CmVuZG9iagoKNzcwIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MCi9QIDc2OSAwIFIKL1BnIDc2NCAwIFIKL0tbNzcxIDAgUiA3NzQg
MCBSIDc3NyAwIFIgNzgwIDAgUiA3ODMgMCBSIDc4NiAwIFIgXQo+PgplbmRvYmoKCjc2OSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA3NjQgMCBSCi9LWzc3
MCAwIFIgXQo+PgplbmRvYmoKCjc5MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA3
OTIgMCBSCi9QZyA3ODkgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MjQgMCBvYmoKPDwvTy9MYXlv
dXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjc5MiAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgNzg5IDAgUgovQSAxNzI0IDAgUgovS1s3OTMgMCBSIF0K
Pj4KZW5kb2JqCgo3OTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA3OTYgMCBS
Ci9QZyA3ODkgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjc5OCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEJvZHkKL1AgNzk2IDAgUgovUGcgNzg5IDAgUgovS1syIF0KPj4KZW5kb2JqCgo3OTYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDc5NSAwIFIKL1BnIDc4OSAwIFIKL0tb
Nzk3IDAgUiA3OTggMCBSIF0KPj4KZW5kb2JqCgo4MDAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCA3OTkgMCBSCi9QZyA3ODkgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjgwMSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgNzk5IDAgUgovUGcgNzg5IDAgUgovS1s0
IF0KPj4KZW5kb2JqCgo3OTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDc5NSAw
IFIKL1BnIDc4OSAwIFIKL0tbODAwIDAgUiA4MDEgMCBSIF0KPj4KZW5kb2JqCgo4MDMgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4MDIgMCBSCi9QZyA3ODkgMCBSCi9LWzUgXQo+
PgplbmRvYmoKCjgwNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODAyIDAg
UgovUGcgNzg5IDAgUgovS1s2IF0KPj4KZW5kb2JqCgo4MDIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDc5NSAwIFIKL1BnIDc4OSAwIFIKL0tbODAzIDAgUiA4MDQgMCBSIF0KPj4K
ZW5kb2JqCgo4MDYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4MDUgMCBSCi9Q
ZyA3ODkgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjgwNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTEJvZHkKL1AgODA1IDAgUgovUGcgNzg5IDAgUgovS1s4IF0KPj4KZW5kb2JqCgo4MDUgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDc5NSAwIFIKL1BnIDc4OSAwIFIKL0tbODA2
IDAgUiA4MDcgMCBSIF0KPj4KZW5kb2JqCgo3OTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0wKL1AgNzk0IDAgUgovUGcgNzg5IDAgUgovS1s3OTYgMCBSIDc5OSAwIFIgODAyIDAgUiA4MDUg
MCBSIF0KPj4KZW5kb2JqCgo3OTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAx
Njg0IDAgUgovUGcgNzg5IDAgUgovS1s3OTUgMCBSIF0KPj4KZW5kb2JqCgo4MTIgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgODExIDAgUgovUGcgODA4IDAgUgovS1swIF0KPj4KZW5k
b2JqCgoxNzI1IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo4
MTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDgwOCAwIFIK
L0EgMTcyNSAwIFIKL0tbODEyIDAgUiBdCj4+CmVuZG9iagoKODE2IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgODE1IDAgUgovUGcgODA4IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo4
MTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDgxNSAwIFIKL1BnIDgwOCAw
IFIKL0tbMiBdCj4+CmVuZG9iagoKODE1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQov
UCA4MTQgMCBSCi9QZyA4MDggMCBSCi9LWzgxNiAwIFIgODE3IDAgUiBdCj4+CmVuZG9iagoKODE5
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgODE4IDAgUgovUGcgODA4IDAgUgov
S1szIF0KPj4KZW5kb2JqCgo4MjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDgxOCAwIFIKL1BnIDgwOCAwIFIKL0tbNCBdCj4+CmVuZG9iagoKODE4IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MSQovUCA4MTQgMCBSCi9QZyA4MDggMCBSCi9LWzgxOSAwIFIgODIwIDAg
UiBdCj4+CmVuZG9iagoKODIyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgODIx
IDAgUgovUGcgODA4IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo4MjMgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDgyMSAwIFIKL1BnIDgwOCAwIFIKL0tbNiBdCj4+CmVuZG9iagoK
ODIxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA4MTQgMCBSCi9QZyA4MDggMCBS
Ci9LWzgyMiAwIFIgODIzIDAgUiBdCj4+CmVuZG9iagoKODI1IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MYmwKL1AgODI0IDAgUgovUGcgODA4IDAgUgovS1s3IF0KPj4KZW5kb2JqCgo4MjYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDgyNCAwIFIKL1BnIDgwOCAwIFIK
L0tbOCBdCj4+CmVuZG9iagoKODI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA4
MTQgMCBSCi9QZyA4MDggMCBSCi9LWzgyNSAwIFIgODI2IDAgUiBdCj4+CmVuZG9iagoKODI4IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgODI3IDAgUgovUGcgODA4IDAgUgovS1s5
IF0KPj4KZW5kb2JqCgo4MjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDgy
NyAwIFIKL1BnIDgwOCAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjgyNyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgODE0IDAgUgovUGcgODA4IDAgUgovS1s4MjggMCBSIDgyOSAwIFIg
XQo+PgplbmRvYmoKCjgxNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA4MTMgMCBS
Ci9QZyA4MDggMCBSCi9LWzgxNSAwIFIgODE4IDAgUiA4MjEgMCBSIDgyNCAwIFIgODI3IDAgUiBd
Cj4+CmVuZG9iagoKODEzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAw
IFIKL1BnIDgwOCAwIFIKL0tbODE0IDAgUiBdCj4+CmVuZG9iagoKODM0IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9QCi9QIDgzMyAwIFIKL1BnIDgzMCAwIFIKL0tbMCBdCj4+CmVuZG9iagoK
MTcyNiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKODMzIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA4MzAgMCBSCi9BIDE3
MjYgMCBSCi9LWzgzNCAwIFIgXQo+PgplbmRvYmoKCjgzOCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDgzNyAwIFIKL1BnIDgzMCAwIFIKL0tbMSBdCj4+CmVuZG9iagoKODM5IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA4MzcgMCBSCi9QZyA4MzAgMCBSCi9L
WzIgXQo+PgplbmRvYmoKCjgzNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgODM2
IDAgUgovUGcgODMwIDAgUgovS1s4MzggMCBSIDgzOSAwIFIgXQo+PgplbmRvYmoKCjg0MSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDg0MCAwIFIKL1BnIDgzMCAwIFIKL0tbMyBd
Cj4+CmVuZG9iagoKODQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA4NDAg
MCBSCi9QZyA4MzAgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjg0MCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEkKL1AgODM2IDAgUgovUGcgODMwIDAgUgovS1s4NDEgMCBSIDg0MiAwIFIgXQo+
PgplbmRvYmoKCjg0NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDg0MyAwIFIK
L1BnIDgzMCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKODQ1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MQm9keQovUCA4NDMgMCBSCi9QZyA4MzAgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjg0MyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgODM2IDAgUgovUGcgODMwIDAgUgovS1s4
NDQgMCBSIDg0NSAwIFIgXQo+PgplbmRvYmoKCjg0NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTGJsCi9QIDg0NiAwIFIKL1BnIDgzMCAwIFIKL0tbNyBdCj4+CmVuZG9iagoKODQ4IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA4NDYgMCBSCi9QZyA4MzAgMCBSCi9LWzgg
XQo+PgplbmRvYmoKCjg0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgODM2IDAg
UgovUGcgODMwIDAgUgovS1s4NDcgMCBSIDg0OCAwIFIgXQo+PgplbmRvYmoKCjgzNiAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA4MzUgMCBSCi9QZyA4MzAgMCBSCi9LWzgzNyAwIFIg
ODQwIDAgUiA4NDMgMCBSIDg0NiAwIFIgXQo+PgplbmRvYmoKCjgzNSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA4MzAgMCBSCi9LWzgzNiAwIFIgXQo+Pgpl
bmRvYmoKCjg1MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCA4NTIgMCBSCi9QZyA4
NDkgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MjcgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVu
dC9CbG9jawo+PgplbmRvYmoKCjg1MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAx
Njg0IDAgUgovUGcgODQ5IDAgUgovQSAxNzI3IDAgUgovS1s4NTMgMCBSIF0KPj4KZW5kb2JqCgo4
NTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4NTYgMCBSCi9QZyA4NDkgMCBS
Ci9LWzEgXQo+PgplbmRvYmoKCjg1OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgODU2IDAgUgovUGcgODQ5IDAgUgovS1syIF0KPj4KZW5kb2JqCgo4NTYgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg1NSAwIFIKL1BnIDg0OSAwIFIKL0tbODU3IDAgUiA4NTgg
MCBSIF0KPj4KZW5kb2JqCgo4NjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4
NTkgMCBSCi9QZyA4NDkgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjg2MSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgODU5IDAgUgovUGcgODQ5IDAgUgovS1s0IF0KPj4KZW5kb2Jq
Cgo4NTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg1NSAwIFIKL1BnIDg0OSAw
IFIKL0tbODYwIDAgUiA4NjEgMCBSIF0KPj4KZW5kb2JqCgo4NjMgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xibAovUCA4NjIgMCBSCi9QZyA4NDkgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjg2
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODYyIDAgUgovUGcgODQ5IDAg
UgovS1s2IF0KPj4KZW5kb2JqCgo4NjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDg1NSAwIFIKL1BnIDg0OSAwIFIKL0tbODYzIDAgUiA4NjQgMCBSIF0KPj4KZW5kb2JqCgo4NjYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4NjUgMCBSCi9QZyA4NDkgMCBSCi9L
WzcgXQo+PgplbmRvYmoKCjg2NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
ODY1IDAgUgovUGcgODQ5IDAgUgovS1s4IF0KPj4KZW5kb2JqCgo4NjUgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xJCi9QIDg1NSAwIFIKL1BnIDg0OSAwIFIKL0tbODY2IDAgUiA4NjcgMCBS
IF0KPj4KZW5kb2JqCgo4NTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgODU0IDAg
UgovUGcgODQ5IDAgUgovS1s4NTYgMCBSIDg1OSAwIFIgODYyIDAgUiA4NjUgMCBSIF0KPj4KZW5k
b2JqCgo4NTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcg
ODQ5IDAgUgovS1s4NTUgMCBSIF0KPj4KZW5kb2JqCgo4NzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgODcxIDAgUgovUGcgODY4IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzI4IDAg
b2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo4NzEgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDg2OCAwIFIKL0EgMTcyOCAwIFIK
L0tbODcyIDAgUiBdCj4+CmVuZG9iagoKODc2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgODc1IDAgUgovUGcgODY4IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo4ODAgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4NzkgMCBSCi9QZyA4NjggMCBSCi9LWzMgXQo+Pgpl
bmRvYmoKCjg4MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODc5IDAgUgov
UGcgODY4IDAgUgovS1s0IF0KPj4KZW5kb2JqCgo4NzkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xJCi9QIDg3OCAwIFIKL1BnIDg2OCAwIFIKL0tbODgwIDAgUiA4ODEgMCBSIF0KPj4KZW5k
b2JqCgo4ODMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA4ODIgMCBSCi9QZyA4
NjggMCBSCi9LWzUgXQo+PgplbmRvYmoKCjg4NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgODgyIDAgUgovUGcgODY4IDAgUgovS1s2IF0KPj4KZW5kb2JqCgo4ODIgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg3OCAwIFIKL1BnIDg2OCAwIFIKL0tbODgzIDAg
UiA4ODQgMCBSIF0KPj4KZW5kb2JqCgo4ODYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xi
bAovUCA4ODUgMCBSCi9QZyA4NjggMCBSCi9LWzcgXQo+PgplbmRvYmoKCjg4NyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODg1IDAgUgovUGcgODY4IDAgUgovS1s4IF0KPj4K
ZW5kb2JqCgo4ODUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDg3OCAwIFIKL1Bn
IDg2OCAwIFIKL0tbODg2IDAgUiA4ODcgMCBSIF0KPj4KZW5kb2JqCgo4NzggMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgODc3IDAgUgovUGcgODY4IDAgUgovS1s4NzkgMCBSIDg4MiAw
IFIgODg1IDAgUiBdCj4+CmVuZG9iagoKODc3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
Qm9keQovUCA4NzUgMCBSCi9QZyA4NjggMCBSCi9LWzIgODc4IDAgUiBdCj4+CmVuZG9iagoKODc1
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA4NzQgMCBSCi9QZyA4NjggMCBSCi9L
Wzg3NiAwIFIgODc3IDAgUiBdCj4+CmVuZG9iagoKODg5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MYmwKL1AgODg4IDAgUgovUGcgODY4IDAgUgovS1s5IF0KPj4KZW5kb2JqCgo4OTAgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDg4OCAwIFIKL1BnIDg2OCAwIFIKL0tb
MTAgXQo+PgplbmRvYmoKCjg4OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgODc0
IDAgUgovUGcgODY4IDAgUgovS1s4ODkgMCBSIDg5MCAwIFIgXQo+PgplbmRvYmoKCjg5MiAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDg5MSAwIFIKL1BnIDg2OCAwIFIKL0tbMTEg
XQo+PgplbmRvYmoKCjg5MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgODkx
IDAgUgovUGcgODY4IDAgUgovS1sxMiBdCj4+CmVuZG9iagoKODkxIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MSQovUCA4NzQgMCBSCi9QZyA4NjggMCBSCi9LWzg5MiAwIFIgODkzIDAgUiBd
Cj4+CmVuZG9iagoKODc0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDg3MyAwIFIK
L1BnIDg2OCAwIFIKL0tbODc1IDAgUiA4ODggMCBSIDg5MSAwIFIgXQo+PgplbmRvYmoKCjg3MyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA4NjggMCBSCi9L
Wzg3NCAwIFIgXQo+PgplbmRvYmoKCjg5OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAov
UCA4OTcgMCBSCi9QZyA4OTQgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MjkgMCBvYmoKPDwvTy9M
YXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjg5NyAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgODk0IDAgUgovQSAxNzI5IDAgUgovS1s4OTggMCBS
IF0KPj4KZW5kb2JqCgo5MDIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA5MDEg
MCBSCi9QZyA4OTQgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjkwNiAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDkwNSAwIFIKL1BnIDg5NCAwIFIKL0tbMyBdCj4+CmVuZG9iagoKOTA3
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5MDUgMCBSCi9QZyA4OTQgMCBS
Ci9LWzQgXQo+PgplbmRvYmoKCjkwNSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1Ag
OTA0IDAgUgovUGcgODk0IDAgUgovS1s5MDYgMCBSIDkwNyAwIFIgXQo+PgplbmRvYmoKCjkwOSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDkwOCAwIFIKL1BnIDg5NCAwIFIKL0tb
NSBdCj4+CmVuZG9iagoKOTEwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5
MDggMCBSCi9QZyA4OTQgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjkwOCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgOTA0IDAgUgovUGcgODk0IDAgUgovS1s5MDkgMCBSIDkxMCAwIFIg
XQo+PgplbmRvYmoKCjkxMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDkxMSAw
IFIKL1BnIDg5NCAwIFIKL0tbNyBdCj4+CmVuZG9iagoKOTEzIDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MQm9keQovUCA5MTEgMCBSCi9QZyA4OTQgMCBSCi9LWzggXQo+PgplbmRvYmoKCjkx
MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgOTA0IDAgUgovUGcgODk0IDAgUgov
S1s5MTIgMCBSIDkxMyAwIFIgXQo+PgplbmRvYmoKCjkxNSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDkxNCAwIFIKL1BnIDg5NCAwIFIKL0tbOSBdCj4+CmVuZG9iagoKOTE2IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5MTQgMCBSCi9QZyA4OTQgMCBSCi9L
WzEwIF0KPj4KZW5kb2JqCgo5MTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDkw
NCAwIFIKL1BnIDg5NCAwIFIKL0tbOTE1IDAgUiA5MTYgMCBSIF0KPj4KZW5kb2JqCgo5MTggMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCA5MTcgMCBSCi9QZyA4OTQgMCBSCi9LWzEx
IF0KPj4KZW5kb2JqCgo5MTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDkx
NyAwIFIKL1BnIDg5NCAwIFIKL0tbMTIgXQo+PgplbmRvYmoKCjkxNyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgOTA0IDAgUgovUGcgODk0IDAgUgovS1s5MTggMCBSIDkxOSAwIFIg
XQo+PgplbmRvYmoKCjkwNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA5MDMgMCBS
Ci9QZyA4OTQgMCBSCi9LWzkwNSAwIFIgOTA4IDAgUiA5MTEgMCBSIDkxNCAwIFIgOTE3IDAgUiBd
Cj4+CmVuZG9iagoKOTAzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5MDEg
MCBSCi9QZyA4OTQgMCBSCi9LWzIgOTA0IDAgUiBdCj4+CmVuZG9iagoKOTAxIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCA5MDAgMCBSCi9QZyA4OTQgMCBSCi9LWzkwMiAwIFIgOTAz
IDAgUiBdCj4+CmVuZG9iagoKOTIxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
OTIwIDAgUgovUGcgODk0IDAgUgovS1sxMyBdCj4+CmVuZG9iagoKOTIyIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCA5MjAgMCBSCi9QZyA4OTQgMCBSCi9LWzE0IF0KPj4KZW5k
b2JqCgo5MjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDkwMCAwIFIKL1BnIDg5
NCAwIFIKL0tbOTIxIDAgUiA5MjIgMCBSIF0KPj4KZW5kb2JqCgo5MDAgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0wKL1AgODk5IDAgUgovUGcgODk0IDAgUgovS1s5MDEgMCBSIDkyMCAwIFIg
XQo+PgplbmRvYmoKCjg5OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQg
MCBSCi9QZyA4OTQgMCBSCi9LWzkwMCAwIFIgXQo+PgplbmRvYmoKCjkyNyAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvUAovUCA5MjYgMCBSCi9QZyA5MjMgMCBSCi9LWzAgXQo+PgplbmRvYmoK
CjE3MzAgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjkyNiAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgOTIzIDAgUgovQSAx
NzMwIDAgUgovS1s5MjcgMCBSIF0KPj4KZW5kb2JqCgo5MzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgOTMxIDAgUgovUGcgOTI4IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzMxIDAg
b2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgo5MzEgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDkyOCAwIFIKL0EgMTczMSAwIFIK
L0tbOTMyIDAgUiBdCj4+CmVuZG9iagoKOTM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgOTM1IDAgUgovUGcgOTI4IDAgUgovS1sxIF0KPj4KZW5kb2JqCgo5MzcgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDkzNSAwIFIKL1BnIDkyOCAwIFIKL0tbMiBdCj4+
CmVuZG9iagoKOTM1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA5MzQgMCBSCi9Q
ZyA5MjggMCBSCi9LWzkzNiAwIFIgOTM3IDAgUiBdCj4+CmVuZG9iagoKOTM5IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgOTM4IDAgUgovUGcgOTI4IDAgUgovS1szIF0KPj4KZW5k
b2JqCgo5NDAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDkzOCAwIFIKL1Bn
IDkyOCAwIFIKL0tbNCBdCj4+CmVuZG9iagoKOTM4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MSQovUCA5MzQgMCBSCi9QZyA5MjggMCBSCi9LWzkzOSAwIFIgOTQwIDAgUiBdCj4+CmVuZG9i
agoKOTQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgOTQxIDAgUgovUGcgOTI4
IDAgUgovS1s1IF0KPj4KZW5kb2JqCgo5NDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDk0MSAwIFIKL1BnIDkyOCAwIFIKL0tbNiBdCj4+CmVuZG9iagoKOTQxIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA5MzQgMCBSCi9QZyA5MjggMCBSCi9LWzk0MiAwIFIg
OTQzIDAgUiBdCj4+CmVuZG9iagoKOTQ1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwK
L1AgOTQ0IDAgUgovUGcgOTI4IDAgUgovS1s3IF0KPj4KZW5kb2JqCgo5NDYgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDk0NCAwIFIKL1BnIDkyOCAwIFIKL0tbOCBdCj4+CmVu
ZG9iagoKOTQ0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA5MzQgMCBSCi9QZyA5
MjggMCBSCi9LWzk0NSAwIFIgOTQ2IDAgUiBdCj4+CmVuZG9iagoKOTQ4IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MYmwKL1AgOTQ3IDAgUgovUGcgOTI4IDAgUgovS1s5IF0KPj4KZW5kb2Jq
Cgo5NDkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDk0NyAwIFIKL1BnIDky
OCAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjk0NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgOTM0IDAgUgovUGcgOTI4IDAgUgovS1s5NDggMCBSIDk0OSAwIFIgXQo+PgplbmRvYmoK
CjkzNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA5MzMgMCBSCi9QZyA5MjggMCBS
Ci9LWzkzNSAwIFIgOTM4IDAgUiA5NDEgMCBSIDk0NCAwIFIgOTQ3IDAgUiBdCj4+CmVuZG9iagoK
OTMzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDkyOCAw
IFIKL0tbOTM0IDAgUiBdCj4+CmVuZG9iagoKOTU0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9QCi9QIDk1MyAwIFIKL1BnIDk1MCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTczMiAwIG9iago8
PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKOTUzIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA5NTAgMCBSCi9BIDE3MzIgMCBSCi9LWzk1
NCAwIFIgXQo+PgplbmRvYmoKCjk1OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDk1NyAwIFIKL1BnIDk1MCAwIFIKL0tbMSBdCj4+CmVuZG9iagoKOTU5IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCA5NTcgMCBSCi9QZyA5NTAgMCBSCi9LWzIgXQo+PgplbmRv
YmoKCjk1NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgOTU2IDAgUgovUGcgOTUw
IDAgUgovS1s5NTggMCBSIDk1OSAwIFIgXQo+PgplbmRvYmoKCjk2MSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDk2MCAwIFIKL1BnIDk1MCAwIFIKL0tbMyBdCj4+CmVuZG9iagoK
OTY1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgOTY0IDAgUgovUGcgOTUwIDAg
UgovS1s1IF0KPj4KZW5kb2JqCgo5NjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDk2NCAwIFIKL1BnIDk1MCAwIFIKL0tbNiBdCj4+CmVuZG9iagoKOTY0IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCA5NjMgMCBSCi9QZyA5NTAgMCBSCi9LWzk2NSAwIFIgOTY2
IDAgUiBdCj4+CmVuZG9iagoKOTY4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
OTY3IDAgUgovUGcgOTUwIDAgUgovS1s3IF0KPj4KZW5kb2JqCgo5NjkgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDk2NyAwIFIKL1BnIDk1MCAwIFIKL0tbOCBdCj4+CmVuZG9i
agoKOTY3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCA5NjMgMCBSCi9QZyA5NTAg
MCBSCi9LWzk2OCAwIFIgOTY5IDAgUiBdCj4+CmVuZG9iagoKOTcxIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgOTcwIDAgUgovUGcgOTUwIDAgUgovS1s5IF0KPj4KZW5kb2JqCgo5
NzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDk3MCAwIFIKL1BnIDk1MCAw
IFIKL0tbMTAgXQo+PgplbmRvYmoKCjk3MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkK
L1AgOTYzIDAgUgovUGcgOTUwIDAgUgovS1s5NzEgMCBSIDk3MiAwIFIgXQo+PgplbmRvYmoKCjk3
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDk3MyAwIFIKL1BnIDk1MCAwIFIK
L0tbMTEgXQo+PgplbmRvYmoKCjk3NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgOTczIDAgUgovUGcgOTUwIDAgUgovS1sxMiBdCj4+CmVuZG9iagoKOTczIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCA5NjMgMCBSCi9QZyA5NTAgMCBSCi9LWzk3NCAwIFIgOTc1
IDAgUiBdCj4+CmVuZG9iagoKOTc3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
OTc2IDAgUgovUGcgOTUwIDAgUgovS1sxMyBdCj4+CmVuZG9iagoKOTc4IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCA5NzYgMCBSCi9QZyA5NTAgMCBSCi9LWzE0IF0KPj4KZW5k
b2JqCgo5NzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDk2MyAwIFIKL1BnIDk1
MCAwIFIKL0tbOTc3IDAgUiA5NzggMCBSIF0KPj4KZW5kb2JqCgo5NjMgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0wKL1AgOTYyIDAgUgovUGcgOTUwIDAgUgovS1s5NjQgMCBSIDk2NyAwIFIg
OTcwIDAgUiA5NzMgMCBSIDk3NiAwIFIgXQo+PgplbmRvYmoKCjk2MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgOTYwIDAgUgovUGcgOTUwIDAgUgovS1s0IDk2MyAwIFIgXQo+
PgplbmRvYmoKCjk2MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgOTU2IDAgUgov
UGcgOTUwIDAgUgovS1s5NjEgMCBSIDk2MiAwIFIgXQo+PgplbmRvYmoKCjk1NiAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTAovUCA5NTUgMCBSCi9QZyA5NTAgMCBSCi9LWzk1NyAwIFIgOTYw
IDAgUiBdCj4+CmVuZG9iagoKOTU1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1Ag
MTY4NCAwIFIKL1BnIDk1MCAwIFIKL0tbOTU2IDAgUiBdCj4+CmVuZG9iagoKOTgzIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDk4MiAwIFIKL1BnIDk3OSAwIFIKL0tbMCBdCj4+CmVu
ZG9iagoKMTczMyAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoK
OTgyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyA5NzkgMCBS
Ci9BIDE3MzMgMCBSCi9LWzk4MyAwIFIgXQo+PgplbmRvYmoKCjk4NyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDk4NiAwIFIKL1BnIDk3OSAwIFIKL0tbMSBdCj4+CmVuZG9iagoK
OTg4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5ODYgMCBSCi9QZyA5Nzkg
MCBSCi9LWzIgXQo+PgplbmRvYmoKCjk4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkK
L1AgOTg1IDAgUgovUGcgOTc5IDAgUgovS1s5ODcgMCBSIDk4OCAwIFIgXQo+PgplbmRvYmoKCjk5
MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDk4OSAwIFIKL1BnIDk3OSAwIFIK
L0tbMyBdCj4+CmVuZG9iagoKOTkxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCA5ODkgMCBSCi9QZyA5NzkgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjk4OSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgOTg1IDAgUgovUGcgOTc5IDAgUgovS1s5OTAgMCBSIDk5MSAw
IFIgXQo+PgplbmRvYmoKCjk5MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDk5
MiAwIFIKL1BnIDk3OSAwIFIKL0tbNSBdCj4+CmVuZG9iagoKOTk0IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCA5OTIgMCBSCi9QZyA5NzkgMCBSCi9LWzYgXQo+PgplbmRvYmoK
Cjk5MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgOTg1IDAgUgovUGcgOTc5IDAg
UgovS1s5OTMgMCBSIDk5NCAwIFIgXQo+PgplbmRvYmoKCjk5NiAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDk5NSAwIFIKL1BnIDk3OSAwIFIKL0tbNyBdCj4+CmVuZG9iagoKOTk3
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCA5OTUgMCBSCi9QZyA5NzkgMCBS
Ci9LWzggXQo+PgplbmRvYmoKCjk5NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1Ag
OTg1IDAgUgovUGcgOTc5IDAgUgovS1s5OTYgMCBSIDk5NyAwIFIgXQo+PgplbmRvYmoKCjk4NSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCA5ODQgMCBSCi9QZyA5NzkgMCBSCi9LWzk4
NiAwIFIgOTg5IDAgUiA5OTIgMCBSIDk5NSAwIFIgXQo+PgplbmRvYmoKCjk4NCAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyA5NzkgMCBSCi9LWzk4NSAwIFIg
XQo+PgplbmRvYmoKCjEwMDIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTAwMSAw
IFIKL1BnIDk5OCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTczNCAwIG9iago8PC9PL0xheW91dAov
UGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKMTAwMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvSAovUCAxNjg0IDAgUgovUGcgOTk4IDAgUgovQSAxNzM0IDAgUgovS1sxMDAyIDAgUiBdCj4+
CmVuZG9iagoKMTAwNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEwMDUgMCBS
Ci9QZyA5OTggMCBSCi9LWzEgXQo+PgplbmRvYmoKCjEwMDcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xCb2R5Ci9QIDEwMDUgMCBSCi9QZyA5OTggMCBSCi9LWzIgXQo+PgplbmRvYmoKCjEw
MDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwMDQgMCBSCi9QZyA5OTggMCBS
Ci9LWzEwMDYgMCBSIDEwMDcgMCBSIF0KPj4KZW5kb2JqCgoxMDA5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MYmwKL1AgMTAwOCAwIFIKL1BnIDk5OCAwIFIKL0tbMyBdCj4+CmVuZG9iagoK
MTAxMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTAwOCAwIFIKL1BnIDk5
OCAwIFIKL0tbNCBdCj4+CmVuZG9iagoKMTAwOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgMTAwNCAwIFIKL1BnIDk5OCAwIFIKL0tbMTAwOSAwIFIgMTAxMCAwIFIgXQo+PgplbmRv
YmoKCjEwMTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMDExIDAgUgovUGcg
OTk4IDAgUgovS1s1IF0KPj4KZW5kb2JqCgoxMDEzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MQm9keQovUCAxMDExIDAgUgovUGcgOTk4IDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxMDExIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDA0IDAgUgovUGcgOTk4IDAgUgovS1sx
MDEyIDAgUiAxMDEzIDAgUiBdCj4+CmVuZG9iagoKMTAxNSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDEwMTQgMCBSCi9QZyA5OTggMCBSCi9LWzcgXQo+PgplbmRvYmoKCjEwMTYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEwMTQgMCBSCi9QZyA5OTggMCBS
Ci9LWzggXQo+PgplbmRvYmoKCjEwMTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDEwMDQgMCBSCi9QZyA5OTggMCBSCi9LWzEwMTUgMCBSIDEwMTYgMCBSIF0KPj4KZW5kb2JqCgox
MDA0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDEwMDMgMCBSCi9QZyA5OTggMCBS
Ci9LWzEwMDUgMCBSIDEwMDggMCBSIDEwMTEgMCBSIDEwMTQgMCBSIF0KPj4KZW5kb2JqCgoxMDAz
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDk5OCAwIFIK
L0tbMTAwNCAwIFIgXQo+PgplbmRvYmoKCjEwMjEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L1AKL1AgMTAyMCAwIFIKL1BnIDEwMTcgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MzUgMCBvYmoK
PDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjEwMjAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDEwMTcgMCBSCi9BIDE3MzUgMCBSCi9L
WzEwMjEgMCBSIF0KPj4KZW5kb2JqCgoxMDI1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgMTAyNCAwIFIKL1BnIDEwMTcgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjEwMjYgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEwMjQgMCBSCi9QZyAxMDE3IDAgUgovS1sy
IF0KPj4KZW5kb2JqCgoxMDI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDIz
IDAgUgovUGcgMTAxNyAwIFIKL0tbMTAyNSAwIFIgMTAyNiAwIFIgXQo+PgplbmRvYmoKCjEwMjgg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMDI3IDAgUgovUGcgMTAxNyAwIFIK
L0tbMyBdCj4+CmVuZG9iagoKMTAyOSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgMTAyNyAwIFIKL1BnIDEwMTcgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjEwMjcgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwMjMgMCBSCi9QZyAxMDE3IDAgUgovS1sxMDI4IDAg
UiAxMDI5IDAgUiBdCj4+CmVuZG9iagoKMTAzMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TGJsCi9QIDEwMzAgMCBSCi9QZyAxMDE3IDAgUgovS1s1IF0KPj4KZW5kb2JqCgoxMDMyIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMDMwIDAgUgovUGcgMTAxNyAwIFIKL0tb
NiBdCj4+CmVuZG9iagoKMTAzMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTAy
MyAwIFIKL1BnIDEwMTcgMCBSCi9LWzEwMzEgMCBSIDEwMzIgMCBSIF0KPj4KZW5kb2JqCgoxMDM0
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTAzMyAwIFIKL1BnIDEwMTcgMCBS
Ci9LWzcgXQo+PgplbmRvYmoKCjEwMzUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDEwMzMgMCBSCi9QZyAxMDE3IDAgUgovS1s4IF0KPj4KZW5kb2JqCgoxMDMzIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDIzIDAgUgovUGcgMTAxNyAwIFIKL0tbMTAzNCAw
IFIgMTAzNSAwIFIgXQo+PgplbmRvYmoKCjEwMzcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAxMDM2IDAgUgovUGcgMTAxNyAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMTAzOCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTAzNiAwIFIKL1BnIDEwMTcgMCBSCi9L
WzEwIF0KPj4KZW5kb2JqCgoxMDM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAx
MDIzIDAgUgovUGcgMTAxNyAwIFIKL0tbMTAzNyAwIFIgMTAzOCAwIFIgXQo+PgplbmRvYmoKCjEw
MjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTAyMiAwIFIKL1BnIDEwMTcgMCBS
Ci9LWzEwMjQgMCBSIDEwMjcgMCBSIDEwMzAgMCBSIDEwMzMgMCBSIDEwMzYgMCBSIF0KPj4KZW5k
b2JqCgoxMDIyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1Bn
IDEwMTcgMCBSCi9LWzEwMjMgMCBSIF0KPj4KZW5kb2JqCgoxMDQzIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDEwNDIgMCBSCi9QZyAxMDM5IDAgUgovS1swIF0KPj4KZW5kb2JqCgox
NzM2IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxMDQyIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxMDM5IDAgUgovQSAx
NzM2IDAgUgovS1sxMDQzIDAgUiBdCj4+CmVuZG9iagoKMTA0NyAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTGJsCi9QIDEwNDYgMCBSCi9QZyAxMDM5IDAgUgovS1sxIF0KPj4KZW5kb2JqCgox
MDQ4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMDQ2IDAgUgovUGcgMTAz
OSAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTA0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEkKL1AgMTA0NSAwIFIKL1BnIDEwMzkgMCBSCi9LWzEwNDcgMCBSIDEwNDggMCBSIF0KPj4KZW5k
b2JqCgoxMDUwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTA0OSAwIFIKL1Bn
IDEwMzkgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjEwNTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xCb2R5Ci9QIDEwNDkgMCBSCi9QZyAxMDM5IDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxMDQ5
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDQ1IDAgUgovUGcgMTAzOSAwIFIK
L0tbMTA1MCAwIFIgMTA1MSAwIFIgXQo+PgplbmRvYmoKCjEwNTMgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xibAovUCAxMDUyIDAgUgovUGcgMTAzOSAwIFIKL0tbNSBdCj4+CmVuZG9iagoK
MTA1NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTA1MiAwIFIKL1BnIDEw
MzkgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjEwNTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xJCi9QIDEwNDUgMCBSCi9QZyAxMDM5IDAgUgovS1sxMDUzIDAgUiAxMDU0IDAgUiBdCj4+CmVu
ZG9iagoKMTA0NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMDQ0IDAgUgovUGcg
MTAzOSAwIFIKL0tbMTA0NiAwIFIgMTA0OSAwIFIgMTA1MiAwIFIgXQo+PgplbmRvYmoKCjEwNDQg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMTAzOSAwIFIK
L0tbMTA0NSAwIFIgXQo+PgplbmRvYmoKCjEwNTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L1AKL1AgMTA1OCAwIFIKL1BnIDEwNTUgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3MzcgMCBvYmoK
PDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjEwNTggMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDEwNTUgMCBSCi9BIDE3MzcgMCBSCi9L
WzEwNTkgMCBSIF0KPj4KZW5kb2JqCgoxMDY0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDEwNjMgMCBSCi9QZyAxMDYwIDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzM4IDAgb2JqCjw8
L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxMDYzIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxMDYwIDAgUgovQSAxNzM4IDAgUgovS1sx
MDY0IDAgUiBdCj4+CmVuZG9iagoKMTA2OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDEwNjcgMCBSCi9QZyAxMDYwIDAgUgovS1sxIF0KPj4KZW5kb2JqCgoxMDY5IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMDY3IDAgUgovUGcgMTA2MCAwIFIKL0tbMiBd
Cj4+CmVuZG9iagoKMTA2NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTA2NiAw
IFIKL1BnIDEwNjAgMCBSCi9LWzEwNjggMCBSIDEwNjkgMCBSIF0KPj4KZW5kb2JqCgoxMDcxIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTA3MCAwIFIKL1BnIDEwNjAgMCBSCi9L
WzMgXQo+PgplbmRvYmoKCjEwNzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDEwNzAgMCBSCi9QZyAxMDYwIDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxMDcwIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDY2IDAgUgovUGcgMTA2MCAwIFIKL0tbMTA3MSAwIFIg
MTA3MiAwIFIgXQo+PgplbmRvYmoKCjEwNzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xi
bAovUCAxMDczIDAgUgovUGcgMTA2MCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMTA3NSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTA3MyAwIFIKL1BnIDEwNjAgMCBSCi9LWzYg
XQo+PgplbmRvYmoKCjEwNzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwNjYg
MCBSCi9QZyAxMDYwIDAgUgovS1sxMDc0IDAgUiAxMDc1IDAgUiBdCj4+CmVuZG9iagoKMTA3NyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEwNzYgMCBSCi9QZyAxMDYwIDAgUgov
S1s3IF0KPj4KZW5kb2JqCgoxMDc4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCAxMDc2IDAgUgovUGcgMTA2MCAwIFIKL0tbOCBdCj4+CmVuZG9iagoKMTA3NiAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTA2NiAwIFIKL1BnIDEwNjAgMCBSCi9LWzEwNzcgMCBS
IDEwNzggMCBSIF0KPj4KZW5kb2JqCgoxMDY2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
Ci9QIDEwNjUgMCBSCi9QZyAxMDYwIDAgUgovS1sxMDY3IDAgUiAxMDcwIDAgUiAxMDczIDAgUiAx
MDc2IDAgUiBdCj4+CmVuZG9iagoKMTA2NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2
Ci9QIDE2ODQgMCBSCi9QZyAxMDYwIDAgUgovS1sxMDY2IDAgUiBdCj4+CmVuZG9iagoKMTA4MyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMDgyIDAgUgovUGcgMTA3OSAwIFIKL0tb
MCBdCj4+CmVuZG9iagoKMTczOSAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+
CmVuZG9iagoKMTA4MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgov
UGcgMTA3OSAwIFIKL0EgMTczOSAwIFIKL0tbMTA4MyAwIFIgXQo+PgplbmRvYmoKCjEwODcgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMDg2IDAgUgovUGcgMTA3OSAwIFIKL0tb
MSBdCj4+CmVuZG9iagoKMTA4OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MTA4NiAwIFIKL1BnIDEwNzkgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjEwODYgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwODUgMCBSCi9QZyAxMDc5IDAgUgovS1sxMDg3IDAgUiAx
MDg4IDAgUiBdCj4+CmVuZG9iagoKMTA5MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDEwODkgMCBSCi9QZyAxMDc5IDAgUgovS1szIF0KPj4KZW5kb2JqCgoxMDkxIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMDg5IDAgUgovUGcgMTA3OSAwIFIKL0tbNCBd
Cj4+CmVuZG9iagoKMTA4OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTA4NSAw
IFIKL1BnIDEwNzkgMCBSCi9LWzEwOTAgMCBSIDEwOTEgMCBSIF0KPj4KZW5kb2JqCgoxMDkzIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTA5MiAwIFIKL1BnIDEwNzkgMCBSCi9L
WzUgXQo+PgplbmRvYmoKCjEwOTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDEwOTIgMCBSCi9QZyAxMDc5IDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxMDkyIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMDg1IDAgUgovUGcgMTA3OSAwIFIKL0tbMTA5MyAwIFIg
MTA5NCAwIFIgXQo+PgplbmRvYmoKCjEwOTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xi
bAovUCAxMDk1IDAgUgovUGcgMTA3OSAwIFIKL0tbNyBdCj4+CmVuZG9iagoKMTA5NyAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTA5NSAwIFIKL1BnIDEwNzkgMCBSCi9LWzgg
XQo+PgplbmRvYmoKCjEwOTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwODUg
MCBSCi9QZyAxMDc5IDAgUgovS1sxMDk2IDAgUiAxMDk3IDAgUiBdCj4+CmVuZG9iagoKMTA5OSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEwOTggMCBSCi9QZyAxMDc5IDAgUgov
S1s5IF0KPj4KZW5kb2JqCgoxMTAwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCAxMDk4IDAgUgovUGcgMTA3OSAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjEwOTggMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEwODUgMCBSCi9QZyAxMDc5IDAgUgovS1sxMDk5IDAg
UiAxMTAwIDAgUiBdCj4+CmVuZG9iagoKMTEwMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TGJsCi9QIDExMDEgMCBSCi9QZyAxMDc5IDAgUgovS1sxMSBdCj4+CmVuZG9iagoKMTEwMyAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTEwMSAwIFIKL1BnIDEwNzkgMCBSCi9L
WzEyIF0KPj4KZW5kb2JqCgoxMTAxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAx
MDg1IDAgUgovUGcgMTA3OSAwIFIKL0tbMTEwMiAwIFIgMTEwMyAwIFIgXQo+PgplbmRvYmoKCjEw
ODUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTA4NCAwIFIKL1BnIDEwNzkgMCBS
Ci9LWzEwODYgMCBSIDEwODkgMCBSIDEwOTIgMCBSIDEwOTUgMCBSIDEwOTggMCBSIDExMDEgMCBS
IF0KPj4KZW5kb2JqCgoxMDg0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4
NCAwIFIKL1BnIDEwNzkgMCBSCi9LWzEwODUgMCBSIF0KPj4KZW5kb2JqCgoxMTA4IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExMDcgMCBSCi9QZyAxMTA0IDAgUgovS1swIF0KPj4K
ZW5kb2JqCgoxNzQwIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2Jq
CgoxMTA3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxMTA0
IDAgUgovQSAxNzQwIDAgUgovS1sxMTA4IDAgUiBdCj4+CmVuZG9iagoKMTExMiAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDExMTEgMCBSCi9QZyAxMTA0IDAgUgovS1sxIF0KPj4K
ZW5kb2JqCgoxMTEzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMTExIDAg
UgovUGcgMTEwNCAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTExMSAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEkKL1AgMTExMCAwIFIKL1BnIDExMDQgMCBSCi9LWzExMTIgMCBSIDExMTMgMCBS
IF0KPj4KZW5kb2JqCgoxMTE1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTEx
NCAwIFIKL1BnIDExMDQgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjExMTYgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDExMTQgMCBSCi9QZyAxMTA0IDAgUgovS1s0IF0KPj4KZW5k
b2JqCgoxMTE0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMTEwIDAgUgovUGcg
MTEwNCAwIFIKL0tbMTExNSAwIFIgMTExNiAwIFIgXQo+PgplbmRvYmoKCjExMTggMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMTE3IDAgUgovUGcgMTEwNCAwIFIKL0tbNSBdCj4+
CmVuZG9iagoKMTExOSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTExNyAw
IFIKL1BnIDExMDQgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjExMTcgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xJCi9QIDExMTAgMCBSCi9QZyAxMTA0IDAgUgovS1sxMTE4IDAgUiAxMTE5IDAg
UiBdCj4+CmVuZG9iagoKMTEyMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEx
MjAgMCBSCi9QZyAxMTA0IDAgUgovS1s3IF0KPj4KZW5kb2JqCgoxMTIyIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAxMTIwIDAgUgovUGcgMTEwNCAwIFIKL0tbOCBdCj4+CmVu
ZG9iagoKMTEyMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTExMCAwIFIKL1Bn
IDExMDQgMCBSCi9LWzExMjEgMCBSIDExMjIgMCBSIF0KPj4KZW5kb2JqCgoxMTI0IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTEyMyAwIFIKL1BnIDExMDQgMCBSCi9LWzkgXQo+
PgplbmRvYmoKCjExMjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDExMjMg
MCBSCi9QZyAxMTA0IDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTEyMyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTExMCAwIFIKL1BnIDExMDQgMCBSCi9LWzExMjQgMCBSIDExMjUg
MCBSIF0KPj4KZW5kb2JqCgoxMTI3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
MTEyNiAwIFIKL1BnIDExMDQgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgoxMTI4IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMTI2IDAgUgovUGcgMTEwNCAwIFIKL0tbMTIgXQo+
PgplbmRvYmoKCjExMjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDExMTAgMCBS
Ci9QZyAxMTA0IDAgUgovS1sxMTI3IDAgUiAxMTI4IDAgUiBdCj4+CmVuZG9iagoKMTEzMCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDExMjkgMCBSCi9QZyAxMTA0IDAgUgovS1sx
MyBdCj4+CmVuZG9iagoKMTEzMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MTEyOSAwIFIKL1BnIDExMDQgMCBSCi9LWzE0IF0KPj4KZW5kb2JqCgoxMTI5IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMTEwIDAgUgovUGcgMTEwNCAwIFIKL0tbMTEzMCAwIFIg
MTEzMSAwIFIgXQo+PgplbmRvYmoKCjExMTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wK
L1AgMTEwOSAwIFIKL1BnIDExMDQgMCBSCi9LWzExMTEgMCBSIDExMTQgMCBSIDExMTcgMCBSIDEx
MjAgMCBSIDExMjMgMCBSIDExMjYgMCBSIDExMjkgMCBSIF0KPj4KZW5kb2JqCgoxMTA5IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDExMDQgMCBSCi9LWzEx
MTAgMCBSIF0KPj4KZW5kb2JqCgoxMTM2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9Q
IDExMzUgMCBSCi9QZyAxMTMyIDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzQxIDAgb2JqCjw8L08v
TGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxMTM1IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxMTMyIDAgUgovQSAxNzQxIDAgUgovS1sxMTM2
IDAgUiBdCj4+CmVuZG9iagoKMTEzOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAx
MTM3IDAgUgovUGcgMTEzMiAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTEzOSAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvUAovUCAxMTM3IDAgUgovUGcgMTEzMiAwIFIKL0tbNCBdCj4+CmVuZG9i
agoKMTE0MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTM3IDAgUgovUGcgMTEz
MiAwIFIKL0tbNiBdCj4+CmVuZG9iagoKMTE0MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
UAovUCAxMTM3IDAgUgovUGcgMTEzMiAwIFIKL0tbOCBdCj4+CmVuZG9iagoKMTE0MiAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTM3IDAgUgovUGcgMTEzMiAwIFIKL0tbMTAgXQo+
PgplbmRvYmoKCjExNDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTEzNyAwIFIK
L1BnIDExMzIgMCBSCi9LWzEyIF0KPj4KZW5kb2JqCgoxMTQ0IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9QCi9QIDExMzcgMCBSCi9QZyAxMTMyIDAgUgovS1sxNCBdCj4+CmVuZG9iagoKMTE0
NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTM3IDAgUgovUGcgMTEzMiAwIFIK
L0tbMTYgXQo+PgplbmRvYmoKCjExNDYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
MTEzNyAwIFIKL1BnIDExMzIgMCBSCi9LWzE4IF0KPj4KZW5kb2JqCgoxMTQ3IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9QCi9QIDExMzcgMCBSCi9QZyAxMTMyIDAgUgovS1syMCBdCj4+CmVu
ZG9iagoKMTE0OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTM3IDAgUgovUGcg
MTEzMiAwIFIKL0tbMjIgXQo+PgplbmRvYmoKCjExNDkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL1AKL1AgMTEzNyAwIFIKL1BnIDExMzIgMCBSCi9LWzI0IF0KPj4KZW5kb2JqCgoxMTUwIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExMzcgMCBSCi9QZyAxMTMyIDAgUgovS1sy
NiBdCj4+CmVuZG9iagoKMTc0MiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCi9C
Qm94WzE5Mi42OTkgMTMuODA0IDUxMC4zMjEgMzI4LjMwOF0KPj4KZW5kb2JqCgoxMTM3IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9GaWd1cmUKL1AgMTY4NCAwIFIKL1BnIDExMzIgMCBSCi9B
IDE3NDIgMCBSCi9LWzEgMTEzOCAwIFIgMyAxMTM5IDAgUiA1IDExNDAgMCBSIDcgMTE0MSAwIFIg
OSAxMTQyIDAgUiAxMSAxMTQzIDAgUiAxMyAxMTQ0IDAgUiAxNSAxMTQ1IDAgUgoxNyAxMTQ2IDAg
UiAxOSAxMTQ3IDAgUiAyMSAxMTQ4IDAgUiAyMyAxMTQ5IDAgUiAyNSAxMTUwIDAgUiAyNyBdCj4+
CmVuZG9iagoKMTE1NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTU0IDAgUgov
UGcgMTE1MSAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTc0MyAwIG9iago8PC9PL0xheW91dAovUGxh
Y2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKMTE1NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
SAovUCAxNjg0IDAgUgovUGcgMTE1MSAwIFIKL0EgMTc0MyAwIFIKL0tbMTE1NSAwIFIgXQo+Pgpl
bmRvYmoKCjExNTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMTU4IDAgUgov
UGcgMTE1MSAwIFIKL0tbMSBdCj4+CmVuZG9iagoKMTE2MCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEJvZHkKL1AgMTE1OCAwIFIKL1BnIDExNTEgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjEx
NTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDExNTcgMCBSCi9QZyAxMTUxIDAg
UgovS1sxMTU5IDAgUiAxMTYwIDAgUiBdCj4+CmVuZG9iagoKMTE2MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTGJsCi9QIDExNjEgMCBSCi9QZyAxMTUxIDAgUgovS1szIF0KPj4KZW5kb2Jq
CgoxMTY2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTE2NSAwIFIKL1BnIDEx
NTEgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjExNjcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDExNjUgMCBSCi9QZyAxMTUxIDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxMTY1IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMTY0IDAgUgovUGcgMTE1MSAwIFIKL0tb
MTE2NiAwIFIgMTE2NyAwIFIgXQo+PgplbmRvYmoKCjExNjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xibAovUCAxMTY4IDAgUgovUGcgMTE1MSAwIFIKL0tbNyBdCj4+CmVuZG9iagoKMTE3
MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTE2OCAwIFIKL1BnIDExNTEg
MCBSCi9LWzggXQo+PgplbmRvYmoKCjExNjggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJ
Ci9QIDExNjQgMCBSCi9QZyAxMTUxIDAgUgovS1sxMTY5IDAgUiAxMTcwIDAgUiBdCj4+CmVuZG9i
agoKMTE3MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDExNzEgMCBSCi9QZyAx
MTUxIDAgUgovS1s5IF0KPj4KZW5kb2JqCgoxMTczIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MQm9keQovUCAxMTcxIDAgUgovUGcgMTE1MSAwIFIKL0tbMTAgXQo+PgplbmRvYmoKCjExNzEg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDExNjQgMCBSCi9QZyAxMTUxIDAgUgov
S1sxMTcyIDAgUiAxMTczIDAgUiBdCj4+CmVuZG9iagoKMTE2NCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTAovUCAxMTYzIDAgUgovUGcgMTE1MSAwIFIKL0tbMTE2NSAwIFIgMTE2OCAwIFIg
MTE3MSAwIFIgXQo+PgplbmRvYmoKCjExNjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDExNjEgMCBSCi9QZyAxMTUxIDAgUgovS1s0IDExNjQgMCBSIF0KPj4KZW5kb2JqCgox
MTYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMTU3IDAgUgovUGcgMTE1MSAw
IFIKL0tbMTE2MiAwIFIgMTE2MyAwIFIgXQo+PgplbmRvYmoKCjExNzUgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xibAovUCAxMTc0IDAgUgovUGcgMTE1MSAwIFIKL0tbMTEgXQo+PgplbmRv
YmoKCjExNzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDExNzQgMCBSCi9Q
ZyAxMTUxIDAgUgovS1sxMiBdCj4+CmVuZG9iagoKMTE3NCAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEkKL1AgMTE1NyAwIFIKL1BnIDExNTEgMCBSCi9LWzExNzUgMCBSIDExNzYgMCBSIF0K
Pj4KZW5kb2JqCgoxMTc4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTE3NyAw
IFIKL1BnIDExNTEgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgoxMTc5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCAxMTc3IDAgUgovUGcgMTE1MSAwIFIKL0tbMTQgXQo+PgplbmRv
YmoKCjExNzcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDExNTcgMCBSCi9QZyAx
MTUxIDAgUgovS1sxMTc4IDAgUiAxMTc5IDAgUiBdCj4+CmVuZG9iagoKMTE1NyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMTU2IDAgUgovUGcgMTE1MSAwIFIKL0tbMTE1OCAwIFIg
MTE2MSAwIFIgMTE3NCAwIFIgMTE3NyAwIFIgXQo+PgplbmRvYmoKCjExNTYgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMTE1MSAwIFIKL0tbMTE1NyAwIFIg
XQo+PgplbmRvYmoKCjExODEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTE4MCAw
IFIKL1BnIDExNTEgMCBSCi9LWzE2IF0KPj4KZW5kb2JqCgoxMTgyIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDExODAgMCBSCi9QZyAxMTUxIDAgUgovS1sxOCBdCj4+CmVuZG9iagoK
MTE4MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTgwIDAgUgovUGcgMTE1MSAw
IFIKL0tbMjAgXQo+PgplbmRvYmoKCjExODQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AK
L1AgMTE4MCAwIFIKL1BnIDExNTEgMCBSCi9LWzIyIF0KPj4KZW5kb2JqCgoxMTg1IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExODAgMCBSCi9QZyAxMTUxIDAgUgovS1syNCBdCj4+
CmVuZG9iagoKMTE4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMTgwIDAgUgov
UGcgMTE1MSAwIFIKL0tbMjYgXQo+PgplbmRvYmoKCjExODcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgMTE4MCAwIFIKL1BnIDExNTEgMCBSCi9LWzI3IF0KPj4KZW5kb2JqCgoxMTg4
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExODAgMCBSCi9QZyAxMTUxIDAgUgov
S1syOSBdCj4+CmVuZG9iagoKMTE4OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAx
MTgwIDAgUgovUGcgMTE1MSAwIFIKL0tbMzEgXQo+PgplbmRvYmoKCjExOTAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTE4MCAwIFIKL1BnIDExNTEgMCBSCi9LWzMzIF0KPj4KZW5k
b2JqCgoxMTkxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExODAgMCBSCi9QZyAx
MTUxIDAgUgovS1szNSBdCj4+CmVuZG9iagoKMTE5MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCAxMTgwIDAgUgovUGcgMTE1MSAwIFIKL0tbMzcgXQo+PgplbmRvYmoKCjExOTMgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTE4MCAwIFIKL1BnIDExNTEgMCBSCi9LWzM5
IF0KPj4KZW5kb2JqCgoxMTk0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDExODAg
MCBSCi9QZyAxMTUxIDAgUgovS1s0MSBdCj4+CmVuZG9iagoKMTE5NSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCAxMTgwIDAgUgovUGcgMTE1MSAwIFIKL0tbNDIgXQo+PgplbmRvYmoK
CjExOTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTE4MCAwIFIKL1BnIDExNTEg
MCBSCi9LWzQ0IF0KPj4KZW5kb2JqCgoxNzQ0IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQv
QmxvY2sKL0JCb3hbMzI4Ljc2MiAyNS4xNDIgNjgwLjQgMzI3LjExN10KPj4KZW5kb2JqCgoxMTgw
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9GaWd1cmUKL1AgMTY4NCAwIFIKL1BnIDExNTEg
MCBSCi9BIDE3NDQgMCBSCi9LWzE1IDExODEgMCBSIDE3IDExODIgMCBSIDE5IDExODMgMCBSIDIx
IDExODQgMCBSIDIzIDExODUgMCBSIDI1IDExODYgMCBSIDExODcgMCBSIDI4IDExODggMCBSIDMw
IDExODkgMCBSIDMyIDExOTAgMCBSIDM0IDExOTEgMCBSIDM2IDExOTIgMCBSIDM4IDExOTMgMCBS
IDQwIDExOTQgMCBSIDExOTUgMCBSIDQzIDExOTYgMCBSIF0KPj4KZW5kb2JqCgoxMjAxIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEyMDAgMCBSCi9QZyAxMTk3IDAgUgovS1swIF0K
Pj4KZW5kb2JqCgoxNzQ1IDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5k
b2JqCgoxMjAwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAx
MTk3IDAgUgovQSAxNzQ1IDAgUgovS1sxMjAxIDAgUiBdCj4+CmVuZG9iagoKMTIwNSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEyMDQgMCBSCi9QZyAxMTk3IDAgUgovS1sxIF0K
Pj4KZW5kb2JqCgoxMjA2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMjA0
IDAgUgovUGcgMTE5NyAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTIwNCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTIwMyAwIFIKL1BnIDExOTcgMCBSCi9LWzEyMDUgMCBSIDEyMDYg
MCBSIF0KPj4KZW5kb2JqCgoxMjA4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
MTIwNyAwIFIKL1BnIDExOTcgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjEyMDkgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEyMDcgMCBSCi9QZyAxMTk3IDAgUgovS1s0IF0KPj4K
ZW5kb2JqCgoxMjA3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMjAzIDAgUgov
UGcgMTE5NyAwIFIKL0tbMTIwOCAwIFIgMTIwOSAwIFIgXQo+PgplbmRvYmoKCjEyMTEgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjEwIDAgUgovUGcgMTE5NyAwIFIKL0tbNSBd
Cj4+CmVuZG9iagoKMTIxMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTIx
MCAwIFIKL1BnIDExOTcgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjEyMTAgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xJCi9QIDEyMDMgMCBSCi9QZyAxMTk3IDAgUgovS1sxMjExIDAgUiAxMjEy
IDAgUiBdCj4+CmVuZG9iagoKMTIxNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9Q
IDEyMTMgMCBSCi9QZyAxMTk3IDAgUgovS1s3IF0KPj4KZW5kb2JqCgoxMjE1IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMjEzIDAgUgovUGcgMTE5NyAwIFIKL0tbOCBdCj4+
CmVuZG9iagoKMTIxMyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTIwMyAwIFIK
L1BnIDExOTcgMCBSCi9LWzEyMTQgMCBSIDEyMTUgMCBSIF0KPj4KZW5kb2JqCgoxMjE3IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTIxNiAwIFIKL1BnIDExOTcgMCBSCi9LWzkg
XQo+PgplbmRvYmoKCjEyMTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEy
MTYgMCBSCi9QZyAxMTk3IDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTIxNiAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgMTIwMyAwIFIKL1BnIDExOTcgMCBSCi9LWzEyMTcgMCBSIDEy
MTggMCBSIF0KPj4KZW5kb2JqCgoxMjIwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwK
L1AgMTIxOSAwIFIKL1BnIDExOTcgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgoxMjIxIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMjE5IDAgUgovUGcgMTE5NyAwIFIKL0tbMTIg
XQo+PgplbmRvYmoKCjEyMTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEyMDMg
MCBSCi9QZyAxMTk3IDAgUgovS1sxMjIwIDAgUiAxMjIxIDAgUiBdCj4+CmVuZG9iagoKMTIwMyAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMjAyIDAgUgovUGcgMTE5NyAwIFIKL0tb
MTIwNCAwIFIgMTIwNyAwIFIgMTIxMCAwIFIgMTIxMyAwIFIgMTIxNiAwIFIgMTIxOSAwIFIgXQo+
PgplbmRvYmoKCjEyMDIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAg
UgovUGcgMTE5NyAwIFIKL0tbMTIwMyAwIFIgXQo+PgplbmRvYmoKCjEyMjMgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTIyMiAwIFIKL1BnIDExOTcgMCBSCi9LWzE0IF0KPj4KZW5k
b2JqCgoxMjI0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEyMjIgMCBSCi9QZyAx
MTk3IDAgUgovS1sxNiBdCj4+CmVuZG9iagoKMTIyNSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCAxMjIyIDAgUgovUGcgMTE5NyAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjEyMjYgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTIyMiAwIFIKL1BnIDExOTcgMCBSCi9LWzIw
IF0KPj4KZW5kb2JqCgoxMjI3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEyMjIg
MCBSCi9QZyAxMTk3IDAgUgovS1syMiBdCj4+CmVuZG9iagoKMTIyOCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCAxMjIyIDAgUgovUGcgMTE5NyAwIFIKL0tbMjQgXQo+PgplbmRvYmoK
CjEyMjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTIyMiAwIFIKL1BnIDExOTcg
MCBSCi9LWzI1IF0KPj4KZW5kb2JqCgoxMjMwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDEyMjIgMCBSCi9QZyAxMTk3IDAgUgovS1syNyBdCj4+CmVuZG9iagoKMTIzMSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMjIyIDAgUgovUGcgMTE5NyAwIFIKL0tbMjkgXQo+
PgplbmRvYmoKCjEyMzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTIyMiAwIFIK
L1BnIDExOTcgMCBSCi9LWzMxIF0KPj4KZW5kb2JqCgoxMjMzIDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9QCi9QIDEyMjIgMCBSCi9QZyAxMTk3IDAgUgovS1szMyBdCj4+CmVuZG9iagoKMTIz
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMjIyIDAgUgovUGcgMTE5NyAwIFIK
L0tbMzQgXQo+PgplbmRvYmoKCjEyMzUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
MTIyMiAwIFIKL1BnIDExOTcgMCBSCi9LWzM2IF0KPj4KZW5kb2JqCgoxMjM2IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9QCi9QIDEyMjIgMCBSCi9QZyAxMTk3IDAgUgovS1szOCBdCj4+CmVu
ZG9iagoKMTIzNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxMjIyIDAgUgovUGcg
MTE5NyAwIFIKL0tbNDAgXQo+PgplbmRvYmoKCjEyMzggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL1AKL1AgMTIyMiAwIFIKL1BnIDExOTcgMCBSCi9LWzQyIF0KPj4KZW5kb2JqCgoxNzQ2IDAg
b2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKL0JCb3hbMzIzLjA5MyAxOS40NzMgNzAz
LjA3NyAzMjEuNDQ4XQo+PgplbmRvYmoKCjEyMjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0ZpZ3VyZQovUCAxNjg0IDAgUgovUGcgMTE5NyAwIFIKL0EgMTc0NiAwIFIKL0tbMTMgMTIyMyAw
IFIgMTUgMTIyNCAwIFIgMTcgMTIyNSAwIFIgMTkgMTIyNiAwIFIgMjEgMTIyNyAwIFIgMjMgMTIy
OCAwIFIgMTIyOSAwIFIgMjYgMTIzMCAwIFIgMjggMTIzMSAwIFIgMzAgMTIzMiAwIFIgMzIgMTIz
MyAwIFIgMTIzNCAwIFIgMzUgMTIzNSAwIFIgMzcgMTIzNiAwIFIgMzkgMTIzNyAwIFIgNDEgMTIz
OCAwIFIgXQo+PgplbmRvYmoKCjEyNDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
MTI0MiAwIFIKL1BnIDEyMzkgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3NDcgMCBvYmoKPDwvTy9M
YXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjEyNDIgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDEyMzkgMCBSCi9BIDE3NDcgMCBSCi9LWzEyNDMg
MCBSIF0KPj4KZW5kb2JqCgoxMjQ3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
MTI0NiAwIFIKL1BnIDEyMzkgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjEyNTEgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjUwIDAgUgovUGcgMTIzOSAwIFIKL0tbMyBdCj4+CmVu
ZG9iagoKMTI1MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTI1MCAwIFIK
L1BnIDEyMzkgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjEyNTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDEyNDkgMCBSCi9QZyAxMjM5IDAgUgovS1sxMjUxIDAgUiAxMjUyIDAgUiBd
Cj4+CmVuZG9iagoKMTI1NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEyNTMg
MCBSCi9QZyAxMjM5IDAgUgovS1s1IF0KPj4KZW5kb2JqCgoxMjU1IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCAxMjUzIDAgUgovUGcgMTIzOSAwIFIKL0tbNiBdCj4+CmVuZG9i
agoKMTI1MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTI0OSAwIFIKL1BnIDEy
MzkgMCBSCi9LWzEyNTQgMCBSIDEyNTUgMCBSIF0KPj4KZW5kb2JqCgoxMjU3IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTI1NiAwIFIKL1BnIDEyMzkgMCBSCi9LWzcgXQo+Pgpl
bmRvYmoKCjEyNTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEyNTYgMCBS
Ci9QZyAxMjM5IDAgUgovS1s4IF0KPj4KZW5kb2JqCgoxMjU2IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAxMjQ5IDAgUgovUGcgMTIzOSAwIFIKL0tbMTI1NyAwIFIgMTI1OCAwIFIg
XQo+PgplbmRvYmoKCjEyNjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjU5
IDAgUgovUGcgMTIzOSAwIFIKL0tbOSBdCj4+CmVuZG9iagoKMTI2MSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTI1OSAwIFIKL1BnIDEyMzkgMCBSCi9LWzEwIF0KPj4KZW5k
b2JqCgoxMjU5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMjQ5IDAgUgovUGcg
MTIzOSAwIFIKL0tbMTI2MCAwIFIgMTI2MSAwIFIgXQo+PgplbmRvYmoKCjEyNjMgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjYyIDAgUgovUGcgMTIzOSAwIFIKL0tbMTEgXQo+
PgplbmRvYmoKCjEyNjQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEyNjIg
MCBSCi9QZyAxMjM5IDAgUgovS1sxMiBdCj4+CmVuZG9iagoKMTI2MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTI0OSAwIFIKL1BnIDEyMzkgMCBSCi9LWzEyNjMgMCBSIDEyNjQg
MCBSIF0KPj4KZW5kb2JqCgoxMjY2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
MTI2NSAwIFIKL1BnIDEyMzkgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgoxMjY3IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMjY1IDAgUgovUGcgMTIzOSAwIFIKL0tbMTQgXQo+
PgplbmRvYmoKCjEyNjUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEyNDkgMCBS
Ci9QZyAxMjM5IDAgUgovS1sxMjY2IDAgUiAxMjY3IDAgUiBdCj4+CmVuZG9iagoKMTI0OSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMjQ4IDAgUgovUGcgMTIzOSAwIFIKL0tbMTI1
MCAwIFIgMTI1MyAwIFIgMTI1NiAwIFIgMTI1OSAwIFIgMTI2MiAwIFIgMTI2NSAwIFIgXQo+Pgpl
bmRvYmoKCjEyNDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEyNDYgMCBS
Ci9QZyAxMjM5IDAgUgovS1syIDEyNDkgMCBSIF0KPj4KZW5kb2JqCgoxMjQ2IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMjQ1IDAgUgovUGcgMTIzOSAwIFIKL0tbMTI0NyAwIFIg
MTI0OCAwIFIgXQo+PgplbmRvYmoKCjEyNDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wK
L1AgMTI0NCAwIFIKL1BnIDEyMzkgMCBSCi9LWzEyNDYgMCBSIF0KPj4KZW5kb2JqCgoxMjQ0IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDEyMzkgMCBSCi9L
WzEyNDUgMCBSIF0KPj4KZW5kb2JqCgoxMjcwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgMTI2OSAwIFIKL1BnIDEyMzkgMCBSCi9LWzE1IF0KPj4KZW5kb2JqCgoxMjc0IDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTI3MyAwIFIKL1BnIDEyMzkgMCBSCi9LWzE3
IF0KPj4KZW5kb2JqCgoxMjc1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAx
MjczIDAgUgovUGcgMTIzOSAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjEyNzMgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEyNzIgMCBSCi9QZyAxMjM5IDAgUgovS1sxMjc0IDAgUiAx
Mjc1IDAgUiBdCj4+CmVuZG9iagoKMTI3NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDEyNzYgMCBSCi9QZyAxMjM5IDAgUgovS1sxOSBdCj4+CmVuZG9iagoKMTI3OCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTI3NiAwIFIKL1BnIDEyMzkgMCBSCi9LWzIw
IF0KPj4KZW5kb2JqCgoxMjc2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMjcy
IDAgUgovUGcgMTIzOSAwIFIKL0tbMTI3NyAwIFIgMTI3OCAwIFIgXQo+PgplbmRvYmoKCjEyODAg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMjc5IDAgUgovUGcgMTIzOSAwIFIK
L0tbMjEgXQo+PgplbmRvYmoKCjEyODEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDEyNzkgMCBSCi9QZyAxMjM5IDAgUgovS1syMiBdCj4+CmVuZG9iagoKMTI3OSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTI3MiAwIFIKL1BnIDEyMzkgMCBSCi9LWzEyODAg
MCBSIDEyODEgMCBSIF0KPj4KZW5kb2JqCgoxMjgzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgMTI4MiAwIFIKL1BnIDEyMzkgMCBSCi9LWzIzIF0KPj4KZW5kb2JqCgoxMjg0IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMjgyIDAgUgovUGcgMTIzOSAwIFIK
L0tbMjQgXQo+PgplbmRvYmoKCjEyODIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDEyNzIgMCBSCi9QZyAxMjM5IDAgUgovS1sxMjgzIDAgUiAxMjg0IDAgUiBdCj4+CmVuZG9iagoK
MTI4NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEyODUgMCBSCi9QZyAxMjM5
IDAgUgovS1syNSBdCj4+CmVuZG9iagoKMTI4NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgMTI4NSAwIFIKL1BnIDEyMzkgMCBSCi9LWzI2IF0KPj4KZW5kb2JqCgoxMjg1IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMjcyIDAgUgovUGcgMTIzOSAwIFIKL0tb
MTI4NiAwIFIgMTI4NyAwIFIgXQo+PgplbmRvYmoKCjEyNzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0wKL1AgMTI3MSAwIFIKL1BnIDEyMzkgMCBSCi9LWzEyNzMgMCBSIDEyNzYgMCBSIDEy
NzkgMCBSIDEyODIgMCBSIDEyODUgMCBSIF0KPj4KZW5kb2JqCgoxMjcxIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAxMjY5IDAgUgovUGcgMTIzOSAwIFIKL0tbMTYgMTI3MiAw
IFIgXQo+PgplbmRvYmoKCjEyNjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEy
NjggMCBSCi9QZyAxMjM5IDAgUgovS1sxMjcwIDAgUiAxMjcxIDAgUiBdCj4+CmVuZG9iagoKMTI2
OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyAxMjM5IDAg
UgovS1sxMjY5IDAgUiBdCj4+CmVuZG9iagoKMTI5MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCAxMjkxIDAgUgovUGcgMTI4OCAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTc0OCAwIG9i
ago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKMTI5MSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgMTI4OCAwIFIKL0EgMTc0OCAwIFIK
L0tbMTI5MiAwIFIgXQo+PgplbmRvYmoKCjEyOTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAxMjk1IDAgUgovUGcgMTI4OCAwIFIKL0tbMSBdCj4+CmVuZG9iagoKMTI5NyAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTI5NSAwIFIKL1BnIDEyODggMCBSCi9L
WzIgXQo+PgplbmRvYmoKCjEyOTUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEy
OTQgMCBSCi9QZyAxMjg4IDAgUgovS1sxMjk2IDAgUiAxMjk3IDAgUiBdCj4+CmVuZG9iagoKMTI5
OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEyOTggMCBSCi9QZyAxMjg4IDAg
UgovS1szIF0KPj4KZW5kb2JqCgoxMzAwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCAxMjk4IDAgUgovUGcgMTI4OCAwIFIKL0tbNCBdCj4+CmVuZG9iagoKMTI5OCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTI5NCAwIFIKL1BnIDEyODggMCBSCi9LWzEyOTkg
MCBSIDEzMDAgMCBSIF0KPj4KZW5kb2JqCgoxMzAyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgMTMwMSAwIFIKL1BnIDEyODggMCBSCi9LWzUgXQo+PgplbmRvYmoKCjEzMDMgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzMDEgMCBSCi9QZyAxMjg4IDAgUgov
S1s2IF0KPj4KZW5kb2JqCgoxMzAxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAx
Mjk0IDAgUgovUGcgMTI4OCAwIFIKL0tbMTMwMiAwIFIgMTMwMyAwIFIgXQo+PgplbmRvYmoKCjEz
MDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMzA0IDAgUgovUGcgMTI4OCAw
IFIKL0tbNyBdCj4+CmVuZG9iagoKMTMwNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJv
ZHkKL1AgMTMwNCAwIFIKL1BnIDEyODggMCBSCi9LWzggXQo+PgplbmRvYmoKCjEzMDQgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEyOTQgMCBSCi9QZyAxMjg4IDAgUgovS1sxMzA1
IDAgUiAxMzA2IDAgUiBdCj4+CmVuZG9iagoKMTI5NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTAovUCAxMjkzIDAgUgovUGcgMTI4OCAwIFIKL0tbMTI5NSAwIFIgMTI5OCAwIFIgMTMwMSAw
IFIgMTMwNCAwIFIgXQo+PgplbmRvYmoKCjEyOTMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0RpdgovUCAxNjg0IDAgUgovUGcgMTI4OCAwIFIKL0tbMTI5NCAwIFIgXQo+PgplbmRvYmoKCjEz
MTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTMxMCAwIFIKL1BnIDEzMDcgMCBS
Ci9LWzAgXQo+PgplbmRvYmoKCjE3NDkgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9j
awo+PgplbmRvYmoKCjEzMTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAw
IFIKL1BnIDEzMDcgMCBSCi9BIDE3NDkgMCBSCi9LWzEzMTEgMCBSIF0KPj4KZW5kb2JqCgoxMzE2
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEzMTUgMCBSCi9QZyAxMzEyIDAgUgov
S1swIF0KPj4KZW5kb2JqCgoxNzUwIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sK
Pj4KZW5kb2JqCgoxMzE1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBS
Ci9QZyAxMzEyIDAgUgovQSAxNzUwIDAgUgovS1sxMzE2IDAgUiBdCj4+CmVuZG9iagoKMTMyMCAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEzMTkgMCBSCi9QZyAxMzEyIDAgUgov
S1sxIF0KPj4KZW5kb2JqCgoxMzIxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQov
UCAxMzE5IDAgUgovUGcgMTMxMiAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTMxOSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTMxOCAwIFIKL1BnIDEzMTIgMCBSCi9LWzEzMjAgMCBS
IDEzMjEgMCBSIF0KPj4KZW5kb2JqCgoxMzIzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
YmwKL1AgMTMyMiAwIFIKL1BnIDEzMTIgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjEzMjQgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzMjIgMCBSCi9QZyAxMzEyIDAgUgovS1s0
IF0KPj4KZW5kb2JqCgoxMzIyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzE4
IDAgUgovUGcgMTMxMiAwIFIKL0tbMTMyMyAwIFIgMTMyNCAwIFIgXQo+PgplbmRvYmoKCjEzMjYg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMzI1IDAgUgovUGcgMTMxMiAwIFIK
L0tbNSBdCj4+CmVuZG9iagoKMTMyNyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkK
L1AgMTMyNSAwIFIKL1BnIDEzMTIgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjEzMjUgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzMTggMCBSCi9QZyAxMzEyIDAgUgovS1sxMzI2IDAg
UiAxMzI3IDAgUiBdCj4+CmVuZG9iagoKMTMyOSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TGJsCi9QIDEzMjggMCBSCi9QZyAxMzEyIDAgUgovS1s3IF0KPj4KZW5kb2JqCgoxMzMwIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMzI4IDAgUgovUGcgMTMxMiAwIFIKL0tb
OCBdCj4+CmVuZG9iagoKMTMyOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTMx
OCAwIFIKL1BnIDEzMTIgMCBSCi9LWzEzMjkgMCBSIDEzMzAgMCBSIF0KPj4KZW5kb2JqCgoxMzMy
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTMzMSAwIFIKL1BnIDEzMTIgMCBS
Ci9LWzkgXQo+PgplbmRvYmoKCjEzMzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDEzMzEgMCBSCi9QZyAxMzEyIDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTMzMSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTMxOCAwIFIKL1BnIDEzMTIgMCBSCi9LWzEzMzIg
MCBSIDEzMzMgMCBSIF0KPj4KZW5kb2JqCgoxMzM1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MYmwKL1AgMTMzNCAwIFIKL1BnIDEzMTIgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgoxMzM2IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMzM0IDAgUgovUGcgMTMxMiAwIFIK
L0tbMTIgXQo+PgplbmRvYmoKCjEzMzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDEzMTggMCBSCi9QZyAxMzEyIDAgUgovS1sxMzM1IDAgUiAxMzM2IDAgUiBdCj4+CmVuZG9iagoK
MTMxOCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMzE3IDAgUgovUGcgMTMxMiAw
IFIKL0tbMTMxOSAwIFIgMTMyMiAwIFIgMTMyNSAwIFIgMTMyOCAwIFIgMTMzMSAwIFIgMTMzNCAw
IFIgXQo+PgplbmRvYmoKCjEzMTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAx
Njg0IDAgUgovUGcgMTMxMiAwIFIKL0tbMTMxOCAwIFIgXQo+PgplbmRvYmoKCjEzNDEgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTM0MCAwIFIKL1BnIDEzMzcgMCBSCi9LWzAgXQo+
PgplbmRvYmoKCjE3NTEgMCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRv
YmoKCjEzNDAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDEz
MzcgMCBSCi9BIDE3NTEgMCBSCi9LWzEzNDEgMCBSIF0KPj4KZW5kb2JqCgoxMzQ1IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTM0NCAwIFIKL1BnIDEzMzcgMCBSCi9LWzEgXQo+
PgplbmRvYmoKCjEzNDkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMzQ4IDAg
UgovUGcgMTMzNyAwIFIKL0tbMyBdCj4+CmVuZG9iagoKMTM1MCAwIG9iago8PC9UeXBlL1N0cnVj
dEVsZW0KL1MvTEJvZHkKL1AgMTM0OCAwIFIKL1BnIDEzMzcgMCBSCi9LWzQgXQo+PgplbmRvYmoK
CjEzNDggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzNDcgMCBSCi9QZyAxMzM3
IDAgUgovS1sxMzQ5IDAgUiAxMzUwIDAgUiBdCj4+CmVuZG9iagoKMTM1MiAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEzNTEgMCBSCi9QZyAxMzM3IDAgUgovS1s1IF0KPj4KZW5k
b2JqCgoxMzUzIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMzUxIDAgUgov
UGcgMTMzNyAwIFIKL0tbNiBdCj4+CmVuZG9iagoKMTM1MSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTEkKL1AgMTM0NyAwIFIKL1BnIDEzMzcgMCBSCi9LWzEzNTIgMCBSIDEzNTMgMCBSIF0K
Pj4KZW5kb2JqCgoxMzU1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTM1NCAw
IFIKL1BnIDEzMzcgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjEzNTYgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzNTQgMCBSCi9QZyAxMzM3IDAgUgovS1s4IF0KPj4KZW5kb2Jq
CgoxMzU0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzQ3IDAgUgovUGcgMTMz
NyAwIFIKL0tbMTM1NSAwIFIgMTM1NiAwIFIgXQo+PgplbmRvYmoKCjEzNDcgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTM0NiAwIFIKL1BnIDEzMzcgMCBSCi9LWzEzNDggMCBSIDEz
NTEgMCBSIDEzNTQgMCBSIF0KPj4KZW5kb2JqCgoxMzQ2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9MQm9keQovUCAxMzQ0IDAgUgovUGcgMTMzNyAwIFIKL0tbMiAxMzQ3IDAgUiBdCj4+CmVu
ZG9iagoKMTM0NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTM0MyAwIFIKL1Bn
IDEzMzcgMCBSCi9LWzEzNDUgMCBSIDEzNDYgMCBSIF0KPj4KZW5kb2JqCgoxMzU4IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTM1NyAwIFIKL1BnIDEzMzcgMCBSCi9LWzkgXQo+
PgplbmRvYmoKCjEzNTkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzNTcg
MCBSCi9QZyAxMzM3IDAgUgovS1sxMCBdCj4+CmVuZG9iagoKMTM1NyAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEkKL1AgMTM0MyAwIFIKL1BnIDEzMzcgMCBSCi9LWzEzNTggMCBSIDEzNTkg
MCBSIF0KPj4KZW5kb2JqCgoxMzYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1Ag
MTM2MCAwIFIKL1BnIDEzMzcgMCBSCi9LWzExIF0KPj4KZW5kb2JqCgoxMzYyIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMzYwIDAgUgovUGcgMTMzNyAwIFIKL0tbMTIgXQo+
PgplbmRvYmoKCjEzNjAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzNDMgMCBS
Ci9QZyAxMzM3IDAgUgovS1sxMzYxIDAgUiAxMzYyIDAgUiBdCj4+CmVuZG9iagoKMTM2NCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEzNjMgMCBSCi9QZyAxMzM3IDAgUgovS1sx
MyBdCj4+CmVuZG9iagoKMTM2NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MTM2MyAwIFIKL1BnIDEzMzcgMCBSCi9LWzE0IF0KPj4KZW5kb2JqCgoxMzYzIDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzQzIDAgUgovUGcgMTMzNyAwIFIKL0tbMTM2NCAwIFIg
MTM2NSAwIFIgXQo+PgplbmRvYmoKCjEzNDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wK
L1AgMTM0MiAwIFIKL1BnIDEzMzcgMCBSCi9LWzEzNDQgMCBSIDEzNTcgMCBSIDEzNjAgMCBSIDEz
NjMgMCBSIF0KPj4KZW5kb2JqCgoxMzQyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYK
L1AgMTY4NCAwIFIKL1BnIDEzMzcgMCBSCi9LWzEzNDMgMCBSIF0KPj4KZW5kb2JqCgoxMzcwIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDEzNjkgMCBSCi9QZyAxMzY2IDAgUgovS1sw
IF0KPj4KZW5kb2JqCgoxNzUyIDAgb2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4K
ZW5kb2JqCgoxMzY5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9Q
ZyAxMzY2IDAgUgovQSAxNzUyIDAgUgovS1sxMzcwIDAgUiBdCj4+CmVuZG9iagoKMTM3NCAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEzNzMgMCBSCi9QZyAxMzY2IDAgUgovS1sx
IF0KPj4KZW5kb2JqCgoxMzc1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAx
MzczIDAgUgovUGcgMTM2NiAwIFIKL0tbMiBdCj4+CmVuZG9iagoKMTM3MyAwIG9iago8PC9UeXBl
L1N0cnVjdEVsZW0KL1MvTEkKL1AgMTM3MiAwIFIKL1BnIDEzNjYgMCBSCi9LWzEzNzQgMCBSIDEz
NzUgMCBSIF0KPj4KZW5kb2JqCgoxMzc3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwK
L1AgMTM3NiAwIFIKL1BnIDEzNjYgMCBSCi9LWzMgXQo+PgplbmRvYmoKCjEzNzggMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzNzYgMCBSCi9QZyAxMzY2IDAgUgovS1s0IF0K
Pj4KZW5kb2JqCgoxMzc2IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzcyIDAg
UgovUGcgMTM2NiAwIFIKL0tbMTM3NyAwIFIgMTM3OCAwIFIgXQo+PgplbmRvYmoKCjEzODAgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxMzc5IDAgUgovUGcgMTM2NiAwIFIKL0tb
NSBdCj4+CmVuZG9iagoKMTM4MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1Ag
MTM3OSAwIFIKL1BnIDEzNjYgMCBSCi9LWzYgXQo+PgplbmRvYmoKCjEzNzkgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzNzIgMCBSCi9QZyAxMzY2IDAgUgovS1sxMzgwIDAgUiAx
MzgxIDAgUiBdCj4+CmVuZG9iagoKMTM4MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJs
Ci9QIDEzODIgMCBSCi9QZyAxMzY2IDAgUgovS1s3IF0KPj4KZW5kb2JqCgoxMzg0IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxMzgyIDAgUgovUGcgMTM2NiAwIFIKL0tbOCBd
Cj4+CmVuZG9iagoKMTM4MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTM3MiAw
IFIKL1BnIDEzNjYgMCBSCi9LWzEzODMgMCBSIDEzODQgMCBSIF0KPj4KZW5kb2JqCgoxMzcyIDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MCi9QIDEzNzEgMCBSCi9QZyAxMzY2IDAgUgovS1sx
MzczIDAgUiAxMzc2IDAgUiAxMzc5IDAgUiAxMzgyIDAgUiBdCj4+CmVuZG9iagoKMTM3MSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyAxMzY2IDAgUgovS1sx
MzcyIDAgUiBdCj4+CmVuZG9iagoKMTM4OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAov
UCAxMzg4IDAgUgovUGcgMTM4NSAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTc1MyAwIG9iago8PC9P
L0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoKMTM4OCAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgMTM4NSAwIFIKL0EgMTc1MyAwIFIKL0tbMTM4
OSAwIFIgXQo+PgplbmRvYmoKCjEzOTMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAov
UCAxMzkyIDAgUgovUGcgMTM4NSAwIFIKL0tbMSBdCj4+CmVuZG9iagoKMTM5NCAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTM5MiAwIFIKL1BnIDEzODUgMCBSCi9LWzIgXQo+
PgplbmRvYmoKCjEzOTIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzOTEgMCBS
Ci9QZyAxMzg1IDAgUgovS1sxMzkzIDAgUiAxMzk0IDAgUiBdCj4+CmVuZG9iagoKMTM5NiAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDEzOTUgMCBSCi9QZyAxMzg1IDAgUgovS1sz
IF0KPj4KZW5kb2JqCgoxNDAwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTM5
OSAwIFIKL1BnIDEzODUgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjE0MDEgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzOTkgMCBSCi9QZyAxMzg1IDAgUgovS1s2IF0KPj4KZW5k
b2JqCgoxMzk5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzk4IDAgUgovUGcg
MTM4NSAwIFIKL0tbMTQwMCAwIFIgMTQwMSAwIFIgXQo+PgplbmRvYmoKCjE0MDMgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDAyIDAgUgovUGcgMTM4NSAwIFIKL0tbNyBdCj4+
CmVuZG9iagoKMTQwNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQwMiAw
IFIKL1BnIDEzODUgMCBSCi9LWzggXQo+PgplbmRvYmoKCjE0MDIgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xJCi9QIDEzOTggMCBSCi9QZyAxMzg1IDAgUgovS1sxNDAzIDAgUiAxNDA0IDAg
UiBdCj4+CmVuZG9iagoKMTQwNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE0
MDUgMCBSCi9QZyAxMzg1IDAgUgovS1s5IF0KPj4KZW5kb2JqCgoxNDA3IDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAxNDA1IDAgUgovUGcgMTM4NSAwIFIKL0tbMTAgXQo+Pgpl
bmRvYmoKCjE0MDUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDEzOTggMCBSCi9Q
ZyAxMzg1IDAgUgovS1sxNDA2IDAgUiAxNDA3IDAgUiBdCj4+CmVuZG9iagoKMTM5OCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxMzk3IDAgUgovUGcgMTM4NSAwIFIKL0tbMTM5OSAw
IFIgMTQwMiAwIFIgMTQwNSAwIFIgXQo+PgplbmRvYmoKCjEzOTcgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDEzOTUgMCBSCi9QZyAxMzg1IDAgUgovS1s0IDEzOTggMCBSIF0K
Pj4KZW5kb2JqCgoxMzk1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxMzkxIDAg
UgovUGcgMTM4NSAwIFIKL0tbMTM5NiAwIFIgMTM5NyAwIFIgXQo+PgplbmRvYmoKCjE0MDkgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDA4IDAgUgovUGcgMTM4NSAwIFIKL0tb
MTEgXQo+PgplbmRvYmoKCjE0MTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9Q
IDE0MDggMCBSCi9QZyAxMzg1IDAgUgovS1sxMiBdCj4+CmVuZG9iagoKMTQwOCAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTM5MSAwIFIKL1BnIDEzODUgMCBSCi9LWzE0MDkgMCBS
IDE0MTAgMCBSIF0KPj4KZW5kb2JqCgoxMzkxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9M
Ci9QIDEzOTAgMCBSCi9QZyAxMzg1IDAgUgovS1sxMzkyIDAgUiAxMzk1IDAgUiAxNDA4IDAgUiBd
Cj4+CmVuZG9iagoKMTM5MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvRGl2Ci9QIDE2ODQg
MCBSCi9QZyAxMzg1IDAgUgovS1sxMzkxIDAgUiBdCj4+CmVuZG9iagoKMTQxNSAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNDE0IDAgUgovUGcgMTQxMSAwIFIKL0tbMCBdCj4+CmVu
ZG9iagoKMTc1NCAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2NrCj4+CmVuZG9iagoK
MTQxNCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAgUgovUGcgMTQxMSAw
IFIKL0EgMTc1NCAwIFIKL0tbMTQxNSAwIFIgXQo+PgplbmRvYmoKCjE0MTkgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDE4IDAgUgovUGcgMTQxMSAwIFIKL0tbMSBdCj4+CmVu
ZG9iagoKMTQyMCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQxOCAwIFIK
L1BnIDE0MTEgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjE0MTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDE0MTcgMCBSCi9QZyAxNDExIDAgUgovS1sxNDE5IDAgUiAxNDIwIDAgUiBd
Cj4+CmVuZG9iagoKMTQyMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE0MjEg
MCBSCi9QZyAxNDExIDAgUgovS1szIF0KPj4KZW5kb2JqCgoxNDIzIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCAxNDIxIDAgUgovUGcgMTQxMSAwIFIKL0tbNCBdCj4+CmVuZG9i
agoKMTQyMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQxNyAwIFIKL1BnIDE0
MTEgMCBSCi9LWzE0MjIgMCBSIDE0MjMgMCBSIF0KPj4KZW5kb2JqCgoxNDI1IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTQyNCAwIFIKL1BnIDE0MTEgMCBSCi9LWzUgXQo+Pgpl
bmRvYmoKCjE0MjYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDE0MjQgMCBS
Ci9QZyAxNDExIDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxNDI0IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9MSQovUCAxNDE3IDAgUgovUGcgMTQxMSAwIFIKL0tbMTQyNSAwIFIgMTQyNiAwIFIg
XQo+PgplbmRvYmoKCjE0MjggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDI3
IDAgUgovUGcgMTQxMSAwIFIKL0tbNyBdCj4+CmVuZG9iagoKMTQyOSAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQyNyAwIFIKL1BnIDE0MTEgMCBSCi9LWzggXQo+PgplbmRv
YmoKCjE0MjcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDE0MTcgMCBSCi9QZyAx
NDExIDAgUgovS1sxNDI4IDAgUiAxNDI5IDAgUiBdCj4+CmVuZG9iagoKMTQxNyAwIG9iago8PC9U
eXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxNDE2IDAgUgovUGcgMTQxMSAwIFIKL0tbMTQxOCAwIFIg
MTQyMSAwIFIgMTQyNCAwIFIgMTQyNyAwIFIgXQo+PgplbmRvYmoKCjE0MTYgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMTQxMSAwIFIKL0tbMTQxNyAwIFIg
XQo+PgplbmRvYmoKCjE0MzQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTQzMyAw
IFIKL1BnIDE0MzAgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3NTUgMCBvYmoKPDwvTy9MYXlvdXQK
L1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjE0MzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0gKL1AgMTY4NCAwIFIKL1BnIDE0MzAgMCBSCi9BIDE3NTUgMCBSCi9LWzE0MzQgMCBSIF0K
Pj4KZW5kb2JqCgoxNDM4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTQzNyAw
IFIKL1BnIDE0MzAgMCBSCi9LWzEgXQo+PgplbmRvYmoKCjE0MzkgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0xCb2R5Ci9QIDE0MzcgMCBSCi9QZyAxNDMwIDAgUgovS1syIF0KPj4KZW5kb2Jq
CgoxNDM3IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxNDM2IDAgUgovUGcgMTQz
MCAwIFIKL0tbMTQzOCAwIFIgMTQzOSAwIFIgXQo+PgplbmRvYmoKCjE0NDEgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDQwIDAgUgovUGcgMTQzMCAwIFIKL0tbMyBdCj4+CmVu
ZG9iagoKMTQ0MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQ0MCAwIFIK
L1BnIDE0MzAgMCBSCi9LWzQgXQo+PgplbmRvYmoKCjE0NDAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xJCi9QIDE0MzYgMCBSCi9QZyAxNDMwIDAgUgovS1sxNDQxIDAgUiAxNDQyIDAgUiBd
Cj4+CmVuZG9iagoKMTQ0NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE0NDMg
MCBSCi9QZyAxNDMwIDAgUgovS1s1IF0KPj4KZW5kb2JqCgoxNDQ1IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MQm9keQovUCAxNDQzIDAgUgovUGcgMTQzMCAwIFIKL0tbNiBdCj4+CmVuZG9i
agoKMTQ0MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQzNiAwIFIKL1BnIDE0
MzAgMCBSCi9LWzE0NDQgMCBSIDE0NDUgMCBSIF0KPj4KZW5kb2JqCgoxNDM2IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9MCi9QIDE0MzUgMCBSCi9QZyAxNDMwIDAgUgovS1sxNDM3IDAgUiAx
NDQwIDAgUiAxNDQzIDAgUiBdCj4+CmVuZG9iagoKMTQzNSAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvRGl2Ci9QIDE2ODQgMCBSCi9QZyAxNDMwIDAgUgovS1sxNDM2IDAgUiBdCj4+CmVuZG9i
agoKMTQ1MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNDQ5IDAgUgovUGcgMTQ0
NiAwIFIKL0tbMCBdCj4+CmVuZG9iagoKMTc1NiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50
L0Jsb2NrCj4+CmVuZG9iagoKMTQ0OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAx
Njg0IDAgUgovUGcgMTQ0NiAwIFIKL0EgMTc1NiAwIFIKL0tbMTQ1MCAwIFIgXQo+PgplbmRvYmoK
CjE0NTQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xibAovUCAxNDUzIDAgUgovUGcgMTQ0
NiAwIFIKL0tbMSBdCj4+CmVuZG9iagoKMTQ1NSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TEJvZHkKL1AgMTQ1MyAwIFIKL1BnIDE0NDYgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjE0NTMgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDE0NTIgMCBSCi9QZyAxNDQ2IDAgUgovS1sx
NDU0IDAgUiAxNDU1IDAgUiBdCj4+CmVuZG9iagoKMTQ1NyAwIG9iago8PC9UeXBlL1N0cnVjdEVs
ZW0KL1MvTGJsCi9QIDE0NTYgMCBSCi9QZyAxNDQ2IDAgUgovS1szIF0KPj4KZW5kb2JqCgoxNDU4
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxNDU2IDAgUgovUGcgMTQ0NiAw
IFIKL0tbNCBdCj4+CmVuZG9iagoKMTQ1NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkK
L1AgMTQ1MiAwIFIKL1BnIDE0NDYgMCBSCi9LWzE0NTcgMCBSIDE0NTggMCBSIF0KPj4KZW5kb2Jq
CgoxNDYwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTQ1OSAwIFIKL1BnIDE0
NDYgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjE0NjEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xCb2R5Ci9QIDE0NTkgMCBSCi9QZyAxNDQ2IDAgUgovS1s2IF0KPj4KZW5kb2JqCgoxNDU5IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxNDUyIDAgUgovUGcgMTQ0NiAwIFIKL0tb
MTQ2MCAwIFIgMTQ2MSAwIFIgXQo+PgplbmRvYmoKCjE0NjMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL0xibAovUCAxNDYyIDAgUgovUGcgMTQ0NiAwIFIKL0tbNyBdCj4+CmVuZG9iagoKMTQ2
NCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQ2MiAwIFIKL1BnIDE0NDYg
MCBSCi9LWzggXQo+PgplbmRvYmoKCjE0NjIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJ
Ci9QIDE0NTIgMCBSCi9QZyAxNDQ2IDAgUgovS1sxNDYzIDAgUiAxNDY0IDAgUiBdCj4+CmVuZG9i
agoKMTQ1MiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxNDUxIDAgUgovUGcgMTQ0
NiAwIFIKL0tbMTQ1MyAwIFIgMTQ1NiAwIFIgMTQ1OSAwIFIgMTQ2MiAwIFIgXQo+PgplbmRvYmoK
CjE0NTEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMTQ0
NiAwIFIKL0tbMTQ1MiAwIFIgXQo+PgplbmRvYmoKCjE0NjkgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgMTQ2OCAwIFIKL1BnIDE0NjUgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3NTcg
MCBvYmoKPDwvTy9MYXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjE0NjggMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDE0NjUgMCBSCi9BIDE3NTcg
MCBSCi9LWzE0NjkgMCBSIF0KPj4KZW5kb2JqCgoxNDc0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxl
bQovUy9QCi9QIDE0NzMgMCBSCi9QZyAxNDcwIDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzU4IDAg
b2JqCjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxNDczIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxNDcwIDAgUgovQSAxNzU4IDAg
UgovS1sxNDc0IDAgUiBdCj4+CmVuZG9iagoKMTQ3OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvTGJsCi9QIDE0NzcgMCBSCi9QZyAxNDcwIDAgUgovS1sxIF0KPj4KZW5kb2JqCgoxNDc5IDAg
b2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxNDc3IDAgUgovUGcgMTQ3MCAwIFIK
L0tbMiBdCj4+CmVuZG9iagoKMTQ3NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1Ag
MTQ3NiAwIFIKL1BnIDE0NzAgMCBSCi9LWzE0NzggMCBSIDE0NzkgMCBSIF0KPj4KZW5kb2JqCgox
NDgxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTQ4MCAwIFIKL1BnIDE0NzAg
MCBSCi9LWzMgXQo+PgplbmRvYmoKCjE0ODIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xC
b2R5Ci9QIDE0ODAgMCBSCi9QZyAxNDcwIDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxNDgwIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxNDc2IDAgUgovUGcgMTQ3MCAwIFIKL0tbMTQ4
MSAwIFIgMTQ4MiAwIFIgXQo+PgplbmRvYmoKCjE0ODQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL0xibAovUCAxNDgzIDAgUgovUGcgMTQ3MCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMTQ4NSAw
IG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTQ4MyAwIFIKL1BnIDE0NzAgMCBS
Ci9LWzYgXQo+PgplbmRvYmoKCjE0ODMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9Q
IDE0NzYgMCBSCi9QZyAxNDcwIDAgUgovS1sxNDg0IDAgUiAxNDg1IDAgUiBdCj4+CmVuZG9iagoK
MTQ3NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTAovUCAxNDc1IDAgUgovUGcgMTQ3MCAw
IFIKL0tbMTQ3NyAwIFIgMTQ4MCAwIFIgMTQ4MyAwIFIgXQo+PgplbmRvYmoKCjE0NzUgMCBvYmoK
PDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0RpdgovUCAxNjg0IDAgUgovUGcgMTQ3MCAwIFIKL0tbMTQ3
NiAwIFIgXQo+PgplbmRvYmoKCjE0OTAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1Ag
MTQ4OSAwIFIKL1BnIDE0ODYgMCBSCi9LWzAgXQo+PgplbmRvYmoKCjE3NTkgMCBvYmoKPDwvTy9M
YXlvdXQKL1BsYWNlbWVudC9CbG9jawo+PgplbmRvYmoKCjE0ODkgMCBvYmoKPDwvVHlwZS9TdHJ1
Y3RFbGVtCi9TL0gKL1AgMTY4NCAwIFIKL1BnIDE0ODYgMCBSCi9BIDE3NTkgMCBSCi9LWzE0OTAg
MCBSIF0KPj4KZW5kb2JqCgoxNDk1IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE0
OTQgMCBSCi9QZyAxNDkxIDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzYwIDAgb2JqCjw8L08vTGF5
b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxNDk0IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxNDkxIDAgUgovQSAxNzYwIDAgUgovS1sxNDk1IDAg
UiBdCj4+CmVuZG9iagoKMTQ5OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE0
OTggMCBSCi9QZyAxNDkxIDAgUgovS1sxIF0KPj4KZW5kb2JqCgoxNTAwIDAgb2JqCjw8L1R5cGUv
U3RydWN0RWxlbQovUy9MQm9keQovUCAxNDk4IDAgUgovUGcgMTQ5MSAwIFIKL0tbMiBdCj4+CmVu
ZG9iagoKMTQ5OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTQ5NyAwIFIKL1Bn
IDE0OTEgMCBSCi9LWzE0OTkgMCBSIDE1MDAgMCBSIF0KPj4KZW5kb2JqCgoxNTAyIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTUwMSAwIFIKL1BnIDE0OTEgMCBSCi9LWzMgXQo+
PgplbmRvYmoKCjE1MDMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5Ci9QIDE1MDEg
MCBSCi9QZyAxNDkxIDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxNTAxIDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9MSQovUCAxNDk3IDAgUgovUGcgMTQ5MSAwIFIKL0tbMTUwMiAwIFIgMTUwMyAw
IFIgXQo+PgplbmRvYmoKCjE0OTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTQ5
NiAwIFIKL1BnIDE0OTEgMCBSCi9LWzE0OTggMCBSIDE1MDEgMCBSIF0KPj4KZW5kb2JqCgoxNDk2
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAwIFIKL1BnIDE0OTEgMCBS
Ci9LWzE0OTcgMCBSIF0KPj4KZW5kb2JqCgoxNTA4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9QCi9QIDE1MDcgMCBSCi9QZyAxNTA0IDAgUgovS1swIF0KPj4KZW5kb2JqCgoxNzYxIDAgb2Jq
Cjw8L08vTGF5b3V0Ci9QbGFjZW1lbnQvQmxvY2sKPj4KZW5kb2JqCgoxNTA3IDAgb2JqCjw8L1R5
cGUvU3RydWN0RWxlbQovUy9ICi9QIDE2ODQgMCBSCi9QZyAxNTA0IDAgUgovQSAxNzYxIDAgUgov
S1sxNTA4IDAgUiBdCj4+CmVuZG9iagoKMTUxMiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
TGJsCi9QIDE1MTEgMCBSCi9QZyAxNTA0IDAgUgovS1sxIF0KPj4KZW5kb2JqCgoxNTEzIDAgb2Jq
Cjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9keQovUCAxNTExIDAgUgovUGcgMTUwNCAwIFIKL0tb
MiBdCj4+CmVuZG9iagoKMTUxMSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTUx
MCAwIFIKL1BnIDE1MDQgMCBSCi9LWzE1MTIgMCBSIDE1MTMgMCBSIF0KPj4KZW5kb2JqCgoxNTE1
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MYmwKL1AgMTUxNCAwIFIKL1BnIDE1MDQgMCBS
Ci9LWzMgXQo+PgplbmRvYmoKCjE1MTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xCb2R5
Ci9QIDE1MTQgMCBSCi9QZyAxNTA0IDAgUgovS1s0IF0KPj4KZW5kb2JqCgoxNTE0IDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAxNTEwIDAgUgovUGcgMTUwNCAwIFIKL0tbMTUxNSAw
IFIgMTUxNiAwIFIgXQo+PgplbmRvYmoKCjE1MTggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9T
L0xibAovUCAxNTE3IDAgUgovUGcgMTUwNCAwIFIKL0tbNSBdCj4+CmVuZG9iagoKMTUxOSAwIG9i
ago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEJvZHkKL1AgMTUxNyAwIFIKL1BnIDE1MDQgMCBSCi9L
WzYgXQo+PgplbmRvYmoKCjE1MTcgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL0xJCi9QIDE1
MTAgMCBSCi9QZyAxNTA0IDAgUgovS1sxNTE4IDAgUiAxNTE5IDAgUiBdCj4+CmVuZG9iagoKMTUy
MSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvTGJsCi9QIDE1MjAgMCBSCi9QZyAxNTA0IDAg
UgovS1s3IF0KPj4KZW5kb2JqCgoxNTIyIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MQm9k
eQovUCAxNTIwIDAgUgovUGcgMTUwNCAwIFIKL0tbOCBdCj4+CmVuZG9iagoKMTUyMCAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvTEkKL1AgMTUxMCAwIFIKL1BnIDE1MDQgMCBSCi9LWzE1MjEg
MCBSIDE1MjIgMCBSIF0KPj4KZW5kb2JqCgoxNTEwIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQov
Uy9MCi9QIDE1MDkgMCBSCi9QZyAxNTA0IDAgUgovS1sxNTExIDAgUiAxNTE0IDAgUiAxNTE3IDAg
UiAxNTIwIDAgUiBdCj4+CmVuZG9iagoKMTUwOSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1Mv
RGl2Ci9QIDE2ODQgMCBSCi9QZyAxNTA0IDAgUgovS1sxNTEwIDAgUiBdCj4+CmVuZG9iagoKMTUy
NyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNTI2IDAgUgovUGcgMTUyMyAwIFIK
L0tbMCBdCj4+CmVuZG9iagoKMTc2MiAwIG9iago8PC9PL0xheW91dAovUGxhY2VtZW50L0Jsb2Nr
Cj4+CmVuZG9iagoKMTUyNiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvSAovUCAxNjg0IDAg
UgovUGcgMTUyMyAwIFIKL0EgMTc2MiAwIFIKL0tbMTUyNyAwIFIgXQo+PgplbmRvYmoKCjE1Mjkg
MCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9L
WzEgXQo+PgplbmRvYmoKCjE1MzAgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUy
OCAwIFIKL1BnIDE1MjMgMCBSCi9LWzIgXQo+PgplbmRvYmoKCjE1MzEgMCBvYmoKPDwvVHlwZS9T
dHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzMgXQo+PgplbmRvYmoK
CjE1MzIgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMg
MCBSCi9LWzQgXQo+PgplbmRvYmoKCjE1MzMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AK
L1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzUgXQo+PgplbmRvYmoKCjE1MzQgMCBvYmoKPDwv
VHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzYgXQo+Pgpl
bmRvYmoKCjE1MzUgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1Bn
IDE1MjMgMCBSCi9LWzcgXQo+PgplbmRvYmoKCjE1MzYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVt
Ci9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzggXQo+PgplbmRvYmoKCjE1MzcgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzkg
XQo+PgplbmRvYmoKCjE1MzggMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAw
IFIKL1BnIDE1MjMgMCBSCi9LWzEwIF0KPj4KZW5kb2JqCgoxNTM5IDAgb2JqCjw8L1R5cGUvU3Ry
dWN0RWxlbQovUy9QCi9QIDE1MjggMCBSCi9QZyAxNTIzIDAgUgovS1sxMSBdCj4+CmVuZG9iagoK
MTU0MCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNTI4IDAgUgovUGcgMTUyMyAw
IFIKL0tbMTIgXQo+PgplbmRvYmoKCjE1NDEgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AK
L1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzEzIF0KPj4KZW5kb2JqCgoxNTQyIDAgb2JqCjw8
L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE1MjggMCBSCi9QZyAxNTIzIDAgUgovS1sxNCBdCj4+
CmVuZG9iagoKMTU0MyAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNTI4IDAgUgov
UGcgMTUyMyAwIFIKL0tbMTUgXQo+PgplbmRvYmoKCjE1NDQgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RF
bGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzE2IF0KPj4KZW5kb2JqCgoxNTQ1
IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE1MjggMCBSCi9QZyAxNTIzIDAgUgov
S1sxNyBdCj4+CmVuZG9iagoKMTU0NiAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAx
NTI4IDAgUgovUGcgMTUyMyAwIFIKL0tbMTggXQo+PgplbmRvYmoKCjE1NDcgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzE5IF0KPj4KZW5k
b2JqCgoxNTQ4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE1MjggMCBSCi9QZyAx
NTIzIDAgUgovS1syMCBdCj4+CmVuZG9iagoKMTU0OSAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0K
L1MvUAovUCAxNTI4IDAgUgovUGcgMTUyMyAwIFIKL0tbMjEgXQo+PgplbmRvYmoKCjE1NTAgMCBv
YmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMgMCBSCi9LWzIy
IF0KPj4KZW5kb2JqCgoxNTUxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9QCi9QIDE1Mjgg
MCBSCi9QZyAxNTIzIDAgUgovS1syMyBdCj4+CmVuZG9iagoKMTU1MiAwIG9iago8PC9UeXBlL1N0
cnVjdEVsZW0KL1MvUAovUCAxNTI4IDAgUgovUGcgMTUyMyAwIFIKL0tbMjQgXQo+PgplbmRvYmoK
CjE1NTMgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIKL1BnIDE1MjMg
MCBSCi9LWzI1IF0KPj4KZW5kb2JqCgoxNTU0IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9Q
Ci9QIDE1MjggMCBSCi9QZyAxNTIzIDAgUgovS1syNiBdCj4+CmVuZG9iagoKMTU1NSAwIG9iago8
PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNTI4IDAgUgovUGcgMTUyMyAwIFIKL0tbMjcgXQo+
PgplbmRvYmoKCjE1NTYgMCBvYmoKPDwvVHlwZS9TdHJ1Y3RFbGVtCi9TL1AKL1AgMTUyOCAwIFIK
L1BnIDE1MjMgMCBSCi9LWzI4IF0KPj4KZW5kb2JqCgoxNTU3IDAgb2JqCjw8L1R5cGUvU3RydWN0
RWxlbQovUy9QCi9QIDE1MjggMCBSCi9QZyAxNTIzIDAgUgovS1syOSBdCj4+CmVuZG9iagoKMTU1
OCAwIG9iago8PC9UeXBlL1N0cnVjdEVsZW0KL1MvUAovUCAxNTI4IDAgUgovUGcgMTUyMyAwIFIK
Pj4KZW5kb2JqCgoxNTI4IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAw
IFIKL1BnIDE1MjMgMCBSCi9LWzE1MjkgMCBSIDE1MzAgMCBSIDE1MzEgMCBSIDE1MzIgMCBSIDE1
MzMgMCBSIDE1MzQgMCBSIDE1MzUgMCBSIDE1MzYgMCBSIDE1MzcgMCBSIDE1MzggMCBSIDE1Mzkg
MCBSIDE1NDAgMCBSIDE1NDEgMCBSIDE1NDIgMCBSIDE1NDMgMCBSIDE1NDQgMCBSCjE1NDUgMCBS
IDE1NDYgMCBSIDE1NDcgMCBSIDE1NDggMCBSIDE1NDkgMCBSIDE1NTAgMCBSIDE1NTEgMCBSIDE1
NTIgMCBSIDE1NTMgMCBSIDE1NTQgMCBSIDE1NTUgMCBSIDE1NTYgMCBSIDE1NTcgMCBSIDE1NTgg
MCBSIF0KPj4KZW5kb2JqCgoxNTYxIDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9MSQovUCAx
NTYwIDAgUgovUGcgMTUyMyAwIFIKL0tbMzAgXQo+PgplbmRvYmoKCjE1NjAgMCBvYmoKPDwvVHlw
ZS9TdHJ1Y3RFbGVtCi9TL0wKL1AgMTU1OSAwIFIKL1BnIDE1MjMgMCBSCi9LWzE1NjEgMCBSIF0K
Pj4KZW5kb2JqCgoxNTU5IDAgb2JqCjw8L1R5cGUvU3RydWN0RWxlbQovUy9EaXYKL1AgMTY4NCAw
IFIKL1BnIDE1MjMgMCBSCi9LWzE1NjAgMCBSIF0KPj4KZW5kb2JqCgoxNjg0IDAgb2JqCjw8L1R5
cGUvU3RydWN0VHJlZVJvb3QKL1BhcmVudFRyZWUgMTc2MyAwIFIKL0tbNSAwIFIgNyAwIFIgMTIg
MCBSIDE4IDAgUiAyMCAwIFIgNTIgMCBSIDU3IDAgUiA1OSAwIFIgNzYgMCBSIDc4IDAgUiAxMDgg
MCBSIDExMCAwIFIgMTQxIDAgUiAxNDMgMCBSIDE2OSAwIFIgMTcxIDAgUgoxOTEgMCBSIDE5MyAw
IFIgMjEwIDAgUiAyMTIgMCBSIDIyOSAwIFIgMjM0IDAgUiAyMzYgMCBSIDI2NSAwIFIgMjY3IDAg
UiAyOTYgMCBSIDI5OCAwIFIgMzM2IDAgUiAzMzggMCBSIDM2OCAwIFIgMzcwIDAgUiAzODQgMCBS
CjM4NiAwIFIgNDE2IDAgUiA0MTggMCBSIDQ0MSAwIFIgNDQzIDAgUiA0NDQgMCBSIDQ0NSAwIFIg
NDQ2IDAgUiA0NzkgMCBSIDQ4MSAwIFIgNTA0IDAgUiA1MDYgMCBSIDUwOCAwIFIgNTIzIDAgUiA1
MjUgMCBSIDU0MiAwIFIKNTQ3IDAgUiA1NDkgMCBSIDU2NiAwIFIgNTY4IDAgUiA1ODUgMCBSIDU4
NyAwIFIgNjA1IDAgUiA2MDcgMCBSIDYyNCAwIFIgNjI2IDAgUiA2NDAgMCBSIDY0NSAwIFIgNjQ3
IDAgUiA2NzAgMCBSIDY3MiAwIFIgNjk2IDAgUgo2OTggMCBSIDcyMiAwIFIgNzI0IDAgUiA3NDEg
MCBSIDc0MyAwIFIgNzY3IDAgUiA3NjkgMCBSIDc5MiAwIFIgNzk0IDAgUiA4MTEgMCBSIDgxMyAw
IFIgODMzIDAgUiA4MzUgMCBSIDg1MiAwIFIgODU0IDAgUiA4NzEgMCBSCjg3MyAwIFIgODk3IDAg
UiA4OTkgMCBSIDkyNiAwIFIgOTMxIDAgUiA5MzMgMCBSIDk1MyAwIFIgOTU1IDAgUiA5ODIgMCBS
IDk4NCAwIFIgMTAwMSAwIFIgMTAwMyAwIFIgMTAyMCAwIFIgMTAyMiAwIFIgMTA0MiAwIFIgMTA0
NCAwIFIKMTA1OCAwIFIgMTA2MyAwIFIgMTA2NSAwIFIgMTA4MiAwIFIgMTA4NCAwIFIgMTEwNyAw
IFIgMTEwOSAwIFIgMTEzNSAwIFIgMTEzNyAwIFIgMTE1NCAwIFIgMTE1NiAwIFIgMTE4MCAwIFIg
MTIwMCAwIFIgMTIwMiAwIFIgMTIyMiAwIFIgMTI0MiAwIFIKMTI0NCAwIFIgMTI2OCAwIFIgMTI5
MSAwIFIgMTI5MyAwIFIgMTMxMCAwIFIgMTMxNSAwIFIgMTMxNyAwIFIgMTM0MCAwIFIgMTM0MiAw
IFIgMTM2OSAwIFIgMTM3MSAwIFIgMTM4OCAwIFIgMTM5MCAwIFIgMTQxNCAwIFIgMTQxNiAwIFIg
MTQzMyAwIFIKMTQzNSAwIFIgMTQ0OSAwIFIgMTQ1MSAwIFIgMTQ2OCAwIFIgMTQ3MyAwIFIgMTQ3
NSAwIFIgMTQ4OSAwIFIgMTQ5NCAwIFIgMTQ5NiAwIFIgMTUwNyAwIFIgMTUwOSAwIFIgMTUyNiAw
IFIgMTUyOCAwIFIgMTU1OSAwIFIgXQo+PgplbmRvYmoKCjE3NjMgMCBvYmoKPDwvTnVtc1sKMCBb
IDYgMCBSIDEwIDAgUiAxMyAwIFIgXQoxIFsgMTkgMCBSIDIzIDAgUiAyNCAwIFIgMjYgMCBSIDI3
IDAgUiAyOSAwIFIgMzAgMCBSIDMyIDAgUiAzMyAwIFIgMzUgMCBSCjM2IDAgUiAzOCAwIFIgMzkg
MCBSIDQxIDAgUiA0MiAwIFIgNDQgMCBSIDQ1IDAgUiA0NyAwIFIgNDggMCBSIF0KMiBbIDUzIDAg
UiBdCjMgWyA1OCAwIFIgNjIgMCBSIDYzIDAgUiA2NSAwIFIgNjYgMCBSIDY4IDAgUiA2OSAwIFIg
NzEgMCBSIDcyIDAgUiBdCjQgWyA3NyAwIFIgODEgMCBSIDgyIDAgUiA4NCAwIFIgODUgMCBSIDg3
IDAgUiA4OCAwIFIgOTEgMCBSIDkyIDAgUiA5NCAwIFIKOTUgMCBSIDk3IDAgUiA5OCAwIFIgMTAw
IDAgUiAxMDEgMCBSIDEwMyAwIFIgMTA0IDAgUiBdCjUgWyAxMDkgMCBSIDExMyAwIFIgMTE0IDAg
UiAxMTYgMCBSIDExNyAwIFIgMTIwIDAgUiAxMjEgMCBSIDEyMyAwIFIgMTI0IDAgUiAxMjYgMCBS
CjEyNyAwIFIgMTI5IDAgUiAxMzAgMCBSIDEzMyAwIFIgMTM0IDAgUiAxMzYgMCBSIDEzNyAwIFIg
XQo2IFsgMTQyIDAgUiAxNDYgMCBSIDE0NyAwIFIgMTQ5IDAgUiAxNTAgMCBSIDE1MiAwIFIgMTUz
IDAgUiAxNTUgMCBSIDE1NiAwIFIgMTU4IDAgUgoxNTkgMCBSIDE2MSAwIFIgMTYyIDAgUiAxNjQg
MCBSIDE2NSAwIFIgXQo3IFsgMTcwIDAgUiAxNzQgMCBSIDE3NSAwIFIgMTc3IDAgUiAxNzggMCBS
IDE4MCAwIFIgMTgxIDAgUiAxODMgMCBSIDE4NCAwIFIgMTg2IDAgUgoxODcgMCBSIF0KOCBbIDE5
MiAwIFIgMTkzIDAgUiAxOTQgMCBSIDE5MyAwIFIgMTk1IDAgUiAxOTMgMCBSIDE5NiAwIFIgMTkz
IDAgUiAxOTcgMCBSIDE5MyAwIFIKMTk4IDAgUiAxOTkgMCBSIDE5MyAwIFIgMjAwIDAgUiAxOTMg
MCBSIDIwMSAwIFIgMTkzIDAgUiAyMDIgMCBSIDIwMyAwIFIgMTkzIDAgUgoyMDQgMCBSIDIwNSAw
IFIgMTkzIDAgUiBdCjkgWyAyMTEgMCBSIDIxNSAwIFIgMjE2IDAgUiAyMTggMCBSIDIxOSAwIFIg
MjIxIDAgUiAyMjIgMCBSIDIyNCAwIFIgMjI1IDAgUiBdCjEwIFsgMjMwIDAgUiBdCjExIFsgMjM1
IDAgUiAyMzkgMCBSIDI0MCAwIFIgMjQyIDAgUiAyNDMgMCBSIDI0NSAwIFIgMjQ2IDAgUiAyNDgg
MCBSIDI0OSAwIFIgMjUxIDAgUgoyNTIgMCBSIDI1NCAwIFIgMjU1IDAgUiAyNTcgMCBSIDI1OCAw
IFIgMjYwIDAgUiAyNjEgMCBSIF0KMTIgWyAyNjYgMCBSIDI3MCAwIFIgMjcxIDAgUiAyNzMgMCBS
IDI3NCAwIFIgMjc2IDAgUiAyNzcgMCBSIDI3OSAwIFIgMjgwIDAgUiAyODIgMCBSCjI4MyAwIFIg
Mjg1IDAgUiAyODYgMCBSIDI4OCAwIFIgMjg5IDAgUiAyOTEgMCBSIDI5MiAwIFIgXQoxMyBbIDI5
NyAwIFIgMzAxIDAgUiAzMDIgMCBSIDMwNSAwIFIgMzA2IDAgUiAzMDggMCBSIDMwOSAwIFIgMzEx
IDAgUiAzMTIgMCBSIDMxNCAwIFIKMzE1IDAgUiAzMTggMCBSIDMxOSAwIFIgMzIxIDAgUiAzMjIg
MCBSIDMyNCAwIFIgMzI1IDAgUiAzMjggMCBSIDMyOSAwIFIgMzMxIDAgUgozMzIgMCBSIF0KMTQg
WyAzMzcgMCBSIDM0MSAwIFIgMzQyIDAgUiAzNDQgMCBSIDM0NSAwIFIgMzQ3IDAgUiAzNDggMCBS
IDM1MCAwIFIgMzUxIDAgUiAzNTMgMCBSCjM1NCAwIFIgMzU3IDAgUiAzNTggMCBSIDM2MCAwIFIg
MzYxIDAgUiAzNjMgMCBSIDM2NCAwIFIgXQoxNSBbIDM2OSAwIFIgMzczIDAgUiAzNzQgMCBSIDM3
NiAwIFIgMzc3IDAgUiAzNzkgMCBSIDM4MCAwIFIgXQoxNiBbIDM4NSAwIFIgMzg5IDAgUiAzOTAg
MCBSIDM5MiAwIFIgMzkzIDAgUiAzOTUgMCBSIDM5NiAwIFIgMzk4IDAgUiAzOTkgMCBSIDQwMiAw
IFIKNDAzIDAgUiA0MDUgMCBSIDQwNiAwIFIgNDA4IDAgUiA0MDkgMCBSIDQxMSAwIFIgNDEyIDAg
UiBdCjE3IFsgNDE3IDAgUiA0MjEgMCBSIDQyMiAwIFIgNDI0IDAgUiA0MjUgMCBSIDQyNyAwIFIg
NDI4IDAgUiA0MzAgMCBSIDQzMSAwIFIgNDMzIDAgUgo0MzQgMCBSIDQzNiAwIFIgNDM3IDAgUiBd
CjE4IFsgNDQyIDAgUiA0NDcgMCBSIDQ0OCAwIFIgNDQ5IDAgUiA0NTAgMCBSIDQ1MSAwIFIgNDUy
IDAgUiA0NTMgMCBSIDQ1NCAwIFIgNDU1IDAgUgo0NTYgMCBSIDQ1NyAwIFIgNDU4IDAgUiA0NTkg
MCBSIDQ2MCAwIFIgNDYxIDAgUiA0NjIgMCBSIDQ2MyAwIFIgNDY0IDAgUiA0NjUgMCBSCjQ2NiAw
IFIgNDY3IDAgUiA0NjggMCBSIDQ2OSAwIFIgNDcwIDAgUiA0NzEgMCBSIDQ3MiAwIFIgNDczIDAg
UiA0NzQgMCBSIDQ3NSAwIFIKXQoxOSBbIDQ4MCAwIFIgNDg0IDAgUiA0ODUgMCBSIDQ4NyAwIFIg
NDg4IDAgUiA0OTAgMCBSIDQ5MSAwIFIgNDkzIDAgUiA0OTQgMCBSIDQ5NiAwIFIKNDk3IDAgUiA0
OTkgMCBSIDUwMCAwIFIgXQoyMCBbIDUwNSAwIFIgNTA2IDAgUiA1MTEgMCBSIDUxMiAwIFIgNTE0
IDAgUiA1MTUgMCBSIDUxNyAwIFIgNTE4IDAgUiBdCjIxIFsgNTI0IDAgUiA1MjggMCBSIDUyOSAw
IFIgNTMxIDAgUiA1MzIgMCBSIDUzNCAwIFIgNTM1IDAgUiA1MzcgMCBSIDUzOCAwIFIgXQoyMiBb
IDU0MyAwIFIgXQoyMyBbIDU0OCAwIFIgNTUyIDAgUiA1NTMgMCBSIDU1NSAwIFIgNTU2IDAgUiA1
NTggMCBSIDU1OSAwIFIgNTYxIDAgUiA1NjIgMCBSIF0KMjQgWyA1NjcgMCBSIDU2OCAwIFIgNTY5
IDAgUiA1NjggMCBSIDU3MCAwIFIgNTY4IDAgUiA1NzEgMCBSIDU2OCAwIFIgNTcyIDAgUiA1Njgg
MCBSCjU3MyAwIFIgNTc0IDAgUiA1NjggMCBSIDU3NSAwIFIgNTY4IDAgUiA1NzYgMCBSIDU2OCAw
IFIgNTc3IDAgUiA1NzggMCBSIDU2OCAwIFIKNTc5IDAgUiA1ODAgMCBSIDU2OCAwIFIgXQoyNSBb
IDU4NiAwIFIgNTg3IDAgUiA1ODggMCBSIDU4NyAwIFIgNTg5IDAgUiA1ODcgMCBSIDU5MCAwIFIg
NTg3IDAgUiA1OTEgMCBSIDU4NyAwIFIKNTkyIDAgUiA1ODcgMCBSIDU5MyAwIFIgNTg3IDAgUiA1
OTQgMCBSIDU5NSAwIFIgNTg3IDAgUiA1OTYgMCBSIDU5NyAwIFIgNTg3IDAgUgo1OTggMCBSIDU4
NyAwIFIgNTk5IDAgUiA1ODcgMCBSIDYwMCAwIFIgNTg3IDAgUiA2MDEgMCBSIF0KMjYgWyA2MDYg
MCBSIDYxMCAwIFIgNjExIDAgUiA2MTMgMCBSIDYxNCAwIFIgNjE2IDAgUiA2MTcgMCBSIDYxOSAw
IFIgNjIwIDAgUiBdCjI3IFsgNjI1IDAgUiA2MjkgMCBSIDYzMCAwIFIgNjMyIDAgUiA2MzMgMCBS
IDYzNSAwIFIgNjM2IDAgUiBdCjI4IFsgNjQxIDAgUiBdCjI5IFsgNjQ2IDAgUiA2NTAgMCBSIDY1
MSAwIFIgNjUzIDAgUiA2NTQgMCBSIDY1NiAwIFIgNjU3IDAgUiA2NTkgMCBSIDY2MCAwIFIgNjYy
IDAgUgo2NjMgMCBSIDY2NSAwIFIgNjY2IDAgUiBdCjMwIFsgNjcxIDAgUiA2NzUgMCBSIDY3NiAw
IFIgNjc4IDAgUiA2NzkgMCBSIDY4MiAwIFIgNjgzIDAgUiA2ODUgMCBSIDY4NiAwIFIgNjg4IDAg
Ugo2ODkgMCBSIDY5MSAwIFIgNjkyIDAgUiBdCjMxIFsgNjk3IDAgUiA3MDEgMCBSIDcwMiAwIFIg
NzA0IDAgUiA3MDUgMCBSIDcwNyAwIFIgNzA4IDAgUiA3MTEgMCBSIDcxMiAwIFIgNzE0IDAgUgo3
MTUgMCBSIDcxNyAwIFIgNzE4IDAgUiBdCjMyIFsgNzIzIDAgUiA3MjcgMCBSIDcyOCAwIFIgNzMw
IDAgUiA3MzEgMCBSIDczMyAwIFIgNzM0IDAgUiA3MzYgMCBSIDczNyAwIFIgXQozMyBbIDc0MiAw
IFIgNzQ2IDAgUiA3NDcgMCBSIDc1MCAwIFIgNzUxIDAgUiA3NTMgMCBSIDc1NCAwIFIgNzU2IDAg
UiA3NTcgMCBSIDc1OSAwIFIKNzYwIDAgUiA3NjIgMCBSIDc2MyAwIFIgXQozNCBbIDc2OCAwIFIg
NzcyIDAgUiA3NzMgMCBSIDc3NSAwIFIgNzc2IDAgUiA3NzggMCBSIDc3OSAwIFIgNzgxIDAgUiA3
ODIgMCBSIDc4NCAwIFIKNzg1IDAgUiA3ODcgMCBSIDc4OCAwIFIgXQozNSBbIDc5MyAwIFIgNzk3
IDAgUiA3OTggMCBSIDgwMCAwIFIgODAxIDAgUiA4MDMgMCBSIDgwNCAwIFIgODA2IDAgUiA4MDcg
MCBSIF0KMzYgWyA4MTIgMCBSIDgxNiAwIFIgODE3IDAgUiA4MTkgMCBSIDgyMCAwIFIgODIyIDAg
UiA4MjMgMCBSIDgyNSAwIFIgODI2IDAgUiA4MjggMCBSCjgyOSAwIFIgXQozNyBbIDgzNCAwIFIg
ODM4IDAgUiA4MzkgMCBSIDg0MSAwIFIgODQyIDAgUiA4NDQgMCBSIDg0NSAwIFIgODQ3IDAgUiA4
NDggMCBSIF0KMzggWyA4NTMgMCBSIDg1NyAwIFIgODU4IDAgUiA4NjAgMCBSIDg2MSAwIFIgODYz
IDAgUiA4NjQgMCBSIDg2NiAwIFIgODY3IDAgUiBdCjM5IFsgODcyIDAgUiA4NzYgMCBSIDg3NyAw
IFIgODgwIDAgUiA4ODEgMCBSIDg4MyAwIFIgODg0IDAgUiA4ODYgMCBSIDg4NyAwIFIgODg5IDAg
Ugo4OTAgMCBSIDg5MiAwIFIgODkzIDAgUiBdCjQwIFsgODk4IDAgUiA5MDIgMCBSIDkwMyAwIFIg
OTA2IDAgUiA5MDcgMCBSIDkwOSAwIFIgOTEwIDAgUiA5MTIgMCBSIDkxMyAwIFIgOTE1IDAgUgo5
MTYgMCBSIDkxOCAwIFIgOTE5IDAgUiA5MjEgMCBSIDkyMiAwIFIgXQo0MSBbIDkyNyAwIFIgXQo0
MiBbIDkzMiAwIFIgOTM2IDAgUiA5MzcgMCBSIDkzOSAwIFIgOTQwIDAgUiA5NDIgMCBSIDk0MyAw
IFIgOTQ1IDAgUiA5NDYgMCBSIDk0OCAwIFIKOTQ5IDAgUiBdCjQzIFsgOTU0IDAgUiA5NTggMCBS
IDk1OSAwIFIgOTYxIDAgUiA5NjIgMCBSIDk2NSAwIFIgOTY2IDAgUiA5NjggMCBSIDk2OSAwIFIg
OTcxIDAgUgo5NzIgMCBSIDk3NCAwIFIgOTc1IDAgUiA5NzcgMCBSIDk3OCAwIFIgXQo0NCBbIDk4
MyAwIFIgOTg3IDAgUiA5ODggMCBSIDk5MCAwIFIgOTkxIDAgUiA5OTMgMCBSIDk5NCAwIFIgOTk2
IDAgUiA5OTcgMCBSIF0KNDUgWyAxMDAyIDAgUiAxMDA2IDAgUiAxMDA3IDAgUiAxMDA5IDAgUiAx
MDEwIDAgUiAxMDEyIDAgUiAxMDEzIDAgUiAxMDE1IDAgUiAxMDE2IDAgUiBdCjQ2IFsgMTAyMSAw
IFIgMTAyNSAwIFIgMTAyNiAwIFIgMTAyOCAwIFIgMTAyOSAwIFIgMTAzMSAwIFIgMTAzMiAwIFIg
MTAzNCAwIFIgMTAzNSAwIFIgMTAzNyAwIFIKMTAzOCAwIFIgXQo0NyBbIDEwNDMgMCBSIDEwNDcg
MCBSIDEwNDggMCBSIDEwNTAgMCBSIDEwNTEgMCBSIDEwNTMgMCBSIDEwNTQgMCBSIF0KNDggWyAx
MDU5IDAgUiBdCjQ5IFsgMTA2NCAwIFIgMTA2OCAwIFIgMTA2OSAwIFIgMTA3MSAwIFIgMTA3MiAw
IFIgMTA3NCAwIFIgMTA3NSAwIFIgMTA3NyAwIFIgMTA3OCAwIFIgXQo1MCBbIDEwODMgMCBSIDEw
ODcgMCBSIDEwODggMCBSIDEwOTAgMCBSIDEwOTEgMCBSIDEwOTMgMCBSIDEwOTQgMCBSIDEwOTYg
MCBSIDEwOTcgMCBSIDEwOTkgMCBSCjExMDAgMCBSIDExMDIgMCBSIDExMDMgMCBSIF0KNTEgWyAx
MTA4IDAgUiAxMTEyIDAgUiAxMTEzIDAgUiAxMTE1IDAgUiAxMTE2IDAgUiAxMTE4IDAgUiAxMTE5
IDAgUiAxMTIxIDAgUiAxMTIyIDAgUiAxMTI0IDAgUgoxMTI1IDAgUiAxMTI3IDAgUiAxMTI4IDAg
UiAxMTMwIDAgUiAxMTMxIDAgUiBdCjUyIFsgMTEzNiAwIFIgMTEzNyAwIFIgMTEzOCAwIFIgMTEz
NyAwIFIgMTEzOSAwIFIgMTEzNyAwIFIgMTE0MCAwIFIgMTEzNyAwIFIgMTE0MSAwIFIgMTEzNyAw
IFIKMTE0MiAwIFIgMTEzNyAwIFIgMTE0MyAwIFIgMTEzNyAwIFIgMTE0NCAwIFIgMTEzNyAwIFIg
MTE0NSAwIFIgMTEzNyAwIFIgMTE0NiAwIFIgMTEzNyAwIFIKMTE0NyAwIFIgMTEzNyAwIFIgMTE0
OCAwIFIgMTEzNyAwIFIgMTE0OSAwIFIgMTEzNyAwIFIgMTE1MCAwIFIgMTEzNyAwIFIgXQo1MyBb
IDExNTUgMCBSIDExNTkgMCBSIDExNjAgMCBSIDExNjIgMCBSIDExNjMgMCBSIDExNjYgMCBSIDEx
NjcgMCBSIDExNjkgMCBSIDExNzAgMCBSIDExNzIgMCBSCjExNzMgMCBSIDExNzUgMCBSIDExNzYg
MCBSIDExNzggMCBSIDExNzkgMCBSIDExODAgMCBSIDExODEgMCBSIDExODAgMCBSIDExODIgMCBS
IDExODAgMCBSCjExODMgMCBSIDExODAgMCBSIDExODQgMCBSIDExODAgMCBSIDExODUgMCBSIDEx
ODAgMCBSIDExODYgMCBSIDExODcgMCBSIDExODAgMCBSIDExODggMCBSCjExODAgMCBSIDExODkg
MCBSIDExODAgMCBSIDExOTAgMCBSIDExODAgMCBSIDExOTEgMCBSIDExODAgMCBSIDExOTIgMCBS
IDExODAgMCBSIDExOTMgMCBSCjExODAgMCBSIDExOTQgMCBSIDExOTUgMCBSIDExODAgMCBSIDEx
OTYgMCBSIF0KNTQgWyAxMjAxIDAgUiAxMjA1IDAgUiAxMjA2IDAgUiAxMjA4IDAgUiAxMjA5IDAg
UiAxMjExIDAgUiAxMjEyIDAgUiAxMjE0IDAgUiAxMjE1IDAgUiAxMjE3IDAgUgoxMjE4IDAgUiAx
MjIwIDAgUiAxMjIxIDAgUiAxMjIyIDAgUiAxMjIzIDAgUiAxMjIyIDAgUiAxMjI0IDAgUiAxMjIy
IDAgUiAxMjI1IDAgUiAxMjIyIDAgUgoxMjI2IDAgUiAxMjIyIDAgUiAxMjI3IDAgUiAxMjIyIDAg
UiAxMjI4IDAgUiAxMjI5IDAgUiAxMjIyIDAgUiAxMjMwIDAgUiAxMjIyIDAgUiAxMjMxIDAgUgox
MjIyIDAgUiAxMjMyIDAgUiAxMjIyIDAgUiAxMjMzIDAgUiAxMjM0IDAgUiAxMjIyIDAgUiAxMjM1
IDAgUiAxMjIyIDAgUiAxMjM2IDAgUiAxMjIyIDAgUgoxMjM3IDAgUiAxMjIyIDAgUiAxMjM4IDAg
UiBdCjU1IFsgMTI0MyAwIFIgMTI0NyAwIFIgMTI0OCAwIFIgMTI1MSAwIFIgMTI1MiAwIFIgMTI1
NCAwIFIgMTI1NSAwIFIgMTI1NyAwIFIgMTI1OCAwIFIgMTI2MCAwIFIKMTI2MSAwIFIgMTI2MyAw
IFIgMTI2NCAwIFIgMTI2NiAwIFIgMTI2NyAwIFIgMTI3MCAwIFIgMTI3MSAwIFIgMTI3NCAwIFIg
MTI3NSAwIFIgMTI3NyAwIFIKMTI3OCAwIFIgMTI4MCAwIFIgMTI4MSAwIFIgMTI4MyAwIFIgMTI4
NCAwIFIgMTI4NiAwIFIgMTI4NyAwIFIgXQo1NiBbIDEyOTIgMCBSIDEyOTYgMCBSIDEyOTcgMCBS
IDEyOTkgMCBSIDEzMDAgMCBSIDEzMDIgMCBSIDEzMDMgMCBSIDEzMDUgMCBSIDEzMDYgMCBSIF0K
NTcgWyAxMzExIDAgUiBdCjU4IFsgMTMxNiAwIFIgMTMyMCAwIFIgMTMyMSAwIFIgMTMyMyAwIFIg
MTMyNCAwIFIgMTMyNiAwIFIgMTMyNyAwIFIgMTMyOSAwIFIgMTMzMCAwIFIgMTMzMiAwIFIKMTMz
MyAwIFIgMTMzNSAwIFIgMTMzNiAwIFIgXQo1OSBbIDEzNDEgMCBSIDEzNDUgMCBSIDEzNDYgMCBS
IDEzNDkgMCBSIDEzNTAgMCBSIDEzNTIgMCBSIDEzNTMgMCBSIDEzNTUgMCBSIDEzNTYgMCBSIDEz
NTggMCBSCjEzNTkgMCBSIDEzNjEgMCBSIDEzNjIgMCBSIDEzNjQgMCBSIDEzNjUgMCBSIF0KNjAg
WyAxMzcwIDAgUiAxMzc0IDAgUiAxMzc1IDAgUiAxMzc3IDAgUiAxMzc4IDAgUiAxMzgwIDAgUiAx
MzgxIDAgUiAxMzgzIDAgUiAxMzg0IDAgUiBdCjYxIFsgMTM4OSAwIFIgMTM5MyAwIFIgMTM5NCAw
IFIgMTM5NiAwIFIgMTM5NyAwIFIgMTQwMCAwIFIgMTQwMSAwIFIgMTQwMyAwIFIgMTQwNCAwIFIg
MTQwNiAwIFIKMTQwNyAwIFIgMTQwOSAwIFIgMTQxMCAwIFIgXQo2MiBbIDE0MTUgMCBSIDE0MTkg
MCBSIDE0MjAgMCBSIDE0MjIgMCBSIDE0MjMgMCBSIDE0MjUgMCBSIDE0MjYgMCBSIDE0MjggMCBS
IDE0MjkgMCBSIF0KNjMgWyAxNDM0IDAgUiAxNDM4IDAgUiAxNDM5IDAgUiAxNDQxIDAgUiAxNDQy
IDAgUiAxNDQ0IDAgUiAxNDQ1IDAgUiBdCjY0IFsgMTQ1MCAwIFIgMTQ1NCAwIFIgMTQ1NSAwIFIg
MTQ1NyAwIFIgMTQ1OCAwIFIgMTQ2MCAwIFIgMTQ2MSAwIFIgMTQ2MyAwIFIgMTQ2NCAwIFIgXQo2
NSBbIDE0NjkgMCBSIF0KNjYgWyAxNDc0IDAgUiAxNDc4IDAgUiAxNDc5IDAgUiAxNDgxIDAgUiAx
NDgyIDAgUiAxNDg0IDAgUiAxNDg1IDAgUiBdCjY3IFsgMTQ5MCAwIFIgXQo2OCBbIDE0OTUgMCBS
IDE0OTkgMCBSIDE1MDAgMCBSIDE1MDIgMCBSIDE1MDMgMCBSIF0KNjkgWyAxNTA4IDAgUiAxNTEy
IDAgUiAxNTEzIDAgUiAxNTE1IDAgUiAxNTE2IDAgUiAxNTE4IDAgUiAxNTE5IDAgUiAxNTIxIDAg
UiAxNTIyIDAgUiBdCjcwIFsgMTUyNyAwIFIgMTUyOSAwIFIgMTUzMCAwIFIgMTUzMSAwIFIgMTUz
MiAwIFIgMTUzMyAwIFIgMTUzNCAwIFIgMTUzNSAwIFIgMTUzNiAwIFIgMTUzNyAwIFIKMTUzOCAw
IFIgMTUzOSAwIFIgMTU0MCAwIFIgMTU0MSAwIFIgMTU0MiAwIFIgMTU0MyAwIFIgMTU0NCAwIFIg
MTU0NSAwIFIgMTU0NiAwIFIgMTU0NyAwIFIKMTU0OCAwIFIgMTU0OSAwIFIgMTU1MCAwIFIgMTU1
MSAwIFIgMTU1MiAwIFIgMTU1MyAwIFIgMTU1NCAwIFIgMTU1NSAwIFIgMTU1NiAwIFIgMTU1NyAw
IFIKMTU2MSAwIFIgXQpdPj4KZW5kb2JqCgoxNTc0IDAgb2JqCjw8L1R5cGUvUGFnZXMKL1Jlc291
cmNlcyAxNjExIDAgUgovTWVkaWFCb3hbIDAgMCA3MjAgNDA1LjA3MDg2NjE0MTczMiBdCi9LaWRz
WyAxIDAgUiAxNSAwIFIgNDkgMCBSIDU0IDAgUiA3MyAwIFIgMTA1IDAgUiAxMzggMCBSIDE2NiAw
IFIgMTg4IDAgUiAyMDcgMCBSIDIyNiAwIFIgMjMxIDAgUiAyNjIgMCBSIDI5MyAwIFIgMzMzIDAg
UiAzNjUgMCBSCjM4MSAwIFIgNDEzIDAgUiA0MzggMCBSIDQ3NiAwIFIgNTAxIDAgUiA1MjAgMCBS
IDUzOSAwIFIgNTQ0IDAgUiA1NjMgMCBSIDU4MiAwIFIgNjAyIDAgUiA2MjEgMCBSIDYzNyAwIFIg
NjQyIDAgUiA2NjcgMCBSIDY5MyAwIFIKNzE5IDAgUiA3MzggMCBSIDc2NCAwIFIgNzg5IDAgUiA4
MDggMCBSIDgzMCAwIFIgODQ5IDAgUiA4NjggMCBSIDg5NCAwIFIgOTIzIDAgUiA5MjggMCBSIDk1
MCAwIFIgOTc5IDAgUiA5OTggMCBSIDEwMTcgMCBSIDEwMzkgMCBSCjEwNTUgMCBSIDEwNjAgMCBS
IDEwNzkgMCBSIDExMDQgMCBSIDExMzIgMCBSIDExNTEgMCBSIDExOTcgMCBSIDEyMzkgMCBSIDEy
ODggMCBSIDEzMDcgMCBSIDEzMTIgMCBSIDEzMzcgMCBSIDEzNjYgMCBSIDEzODUgMCBSIDE0MTEg
MCBSIDE0MzAgMCBSCjE0NDYgMCBSIDE0NjUgMCBSIDE0NzAgMCBSIDE0ODYgMCBSIDE0OTEgMCBS
IDE1MDQgMCBSIDE1MjMgMCBSIF0KL0NvdW50IDcxPj4KZW5kb2JqCgoxNTYyIDAgb2JqCjw8L1R5
cGUvQW5ub3QvU3VidHlwZS9MaW5rL0JvcmRlclswIDAgMF0vUmVjdFs1NzguMTc2IDI5Ljc5MSA3
MDUuMjk2IDQyLjk0NF0vQ29udGVudHM8RkVGRjAwNkEwMDY3MDA3MjAwNkYwMDczMDA3MzAwNDAw
MDczMDA3NTAwNzMwMDY1MDAyRTAwNjMwMDZGMDA2RD4vQTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VS
SShtYWlsdG86amdyb3NzQHN1c2UuY29tKT4+Cj4+CmVuZG9iagoKMTU2MyAwIG9iago8PC9UeXBl
L0Fubm90L1N1YnR5cGUvTGluay9Cb3JkZXJbMCAwIDBdL1JlY3RbNDcuNzU3IDI5LjY3OCA4MS45
NTcgMzUuNzE2XS9Db250ZW50czxGRUZGMDA0MzAwNDMwMDIwMDA0MjAwNTkwMDJEMDA1MzAwNDE+
L0E8PC9UeXBlL0FjdGlvbi9TL1VSSS9VUkkoaHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xp
Y2Vuc2VzL2J5LXNhLzIuMC8pPj4KPj4KZW5kb2JqCgoxNTY0IDAgb2JqCjw8L1R5cGUvQW5ub3Qv
U3VidHlwZS9MaW5rL0JvcmRlclswIDAgMF0vUmVjdFsxNzMuNzU3IDI5LjY3OCAyMDcuOTU3IDM1
LjcxNl0vQ29udGVudHM8RkVGRjAwNDMwMDQzMDAyMDAwNDIwMDU5MDAyRDAwNTMwMDQxPi9BPDwv
VHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0dHBzOi8vY3JlYXRpdmVjb21tb25zLm9yZy9saWNlbnNl
cy9ieS1zYS8yLjAvKT4+Cj4+CmVuZG9iagoKMTU2NSAwIG9iago8PC9UeXBlL0Fubm90L1N1YnR5
cGUvTGluay9Cb3JkZXJbMCAwIDBdL1JlY3RbMjExLjU0MyAyOS42NzggMjc1LjE2NiAzNi4xMTNd
L0NvbnRlbnRzPEZFRkYwMDU0MDA2RjAwNzIwMDZCMDA2OTAwNkMwMDY0MDAyMDAwNTIwMDY1MDA3
NDAwNzYwMDY1MDA2NDAwNzQ+L0E8PC9UeXBlL0FjdGlvbi9TL1VSSS9VUkkoaHR0cHM6Ly93d3cu
ZmxpY2tyLmNvbS9waG90b3MvdG9ya2lsZHIvKT4+Cj4+CmVuZG9iagoKMTU2NiAwIG9iago8PC9U
eXBlL0Fubm90L1N1YnR5cGUvTGluay9Cb3JkZXJbMCAwIDBdL1JlY3RbNzEuMjU2IDI5MC40OTQg
NDAzLjI5MiAzMDcuMzAzXS9Db250ZW50czxGRUZGMDA2ODAwNzQwMDc0MDA3MDAwNzMwMDNBMDAy
RjAwMkYwMDc3MDA2OTAwNkIwMDY5MDAyRTAwNzgwMDY1MDA2RTAwNzAwMDcyMDA2RjAwNkEwMDY1
MDA2MzAwNzQwMDJFMDA2RjAwNzIwMDY3MDAyRjAwNzcwMDY5MDA2QjAwNjkwMDJGMDA0RDAwNjEw
MDY5MDA2RTAwNUYwMDUwMDA2MTAwNjcwMDY1Pi9BPDwvVHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0
dHBzOi8vd2lraS54ZW5wcm9qZWN0Lm9yZy93aWtpL01haW5fUGFnZSk+Pgo+PgplbmRvYmoKCjE1
NjcgMCBvYmoKPDwvVHlwZS9Bbm5vdC9TdWJ0eXBlL0xpbmsvQm9yZGVyWzAgMCAwXS9SZWN0Wzcx
LjI1NiAyNTYuMjUxIDQ1MC40ODkgMjczLjA2XS9Db250ZW50czxGRUZGMDA2ODAwNzQwMDc0MDA3
MDAwNzMwMDNBMDAyRjAwMkYwMDc3MDA2OTAwNkIwMDY5MDAyRTAwNzgwMDY1MDA2RTAwNzAwMDcy
MDA2RjAwNkEwMDY1MDA2MzAwNzQwMDJFMDA2RjAwNzIwMDY3MDAyRjAwNzcwMDY5MDA2QjAwNjkw
MDJGMDA1ODAwNjUwMDZFMDA1RjAwNEQwMDYxMDA2RTAwNUYwMDUwMDA2MTAwNjcwMDY1MDA3Mz4v
QTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VSSShodHRwczovL3dpa2kueGVucHJvamVjdC5vcmcvd2lr
aS9YZW5fTWFuX1BhZ2VzKT4+Cj4+CmVuZG9iagoKMTU2OCAwIG9iago8PC9UeXBlL0Fubm90L1N1
YnR5cGUvTGluay9Cb3JkZXJbMCAwIDBdL1JlY3RbNDcuNDczIDI2MS40NjcgMjkwLjUzIDI2OC45
NV0vQ29udGVudHM8RkVGRjAwNjgwMDc0MDA3NDAwNzAwMDczMDAzQTAwMkYwMDJGMDA2NzAwNjkw
MDc0MDA2ODAwNzUwMDYyMDAyRTAwNjMwMDZGMDA2RDAwMkYwMDU4MDA2OTAwNkMwMDY5MDA2RTAw
NzgwMDJGMDA3ODAwNjUwMDZFMDAyRjAwNzQwMDcyMDA2NTAwNjUwMDJGMDA3ODAwNkMwMDZFMDA3
ODAwNUYwMDcyMDA2NTAwNjIwMDYxMDA3MzAwNjUwMDVGMDAzNDAwMkUwMDMxMDAzNz4vQTw8L1R5
cGUvQWN0aW9uL1MvVVJJL1VSSShodHRwczovL2dpdGh1Yi5jb20vWGlsaW54L3hlbi90cmVlL3hs
bnhfcmViYXNlXzQuMTcpPj4KPj4KZW5kb2JqCgoxNTY5IDAgb2JqCjw8L1R5cGUvQW5ub3QvU3Vi
dHlwZS9MaW5rL0JvcmRlclswIDAgMF0vUmVjdFs0Ny40NzMgMTUxLjkzNiA0NjcuNjY3IDE1OS40
MTldL0NvbnRlbnRzPEZFRkYwMDY4MDA3NDAwNzQwMDcwMDA3MzAwM0EwMDJGMDAyRjAwNzgwMDY5
MDA2QzAwNjkwMDZFMDA3ODAwMkQwMDc3MDA2OTAwNkIwMDY5MDAyRTAwNjEwMDc0MDA2QzAwNjEw
MDczMDA3MzAwNjkwMDYxMDA2RTAwMkUwMDZFMDA2NTAwNzQwMDJGMDA3NzAwNjkwMDZCMDA2OTAw
MkYwMDczMDA3MDAwNjEwMDYzMDA2NTAwNzMwMDJGMDA0MTAwMkYwMDcwMDA2MTAwNjcwMDY1MDA3
MzAwMkYwMDMyMDAzNzAwMzAwMDM5MDAzOTAwMzQwMDM2MDAzMzAwMzYwMDM5MDAyRjAwNDIwMDc1
MDA2OTAwNkMwMDY0MDA2OTAwNkUwMDY3MDAyQjAwNTgwMDY1MDA2RTAwMkIwMDQ4MDA3OTAwNzAw
MDY1MDA3MjAwNzYwMDY5MDA3MzAwNkYwMDcyPi9BPDwvVHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0
dHBzOi8veGlsaW54LXdpa2kuYXRsYXNzaWFuLm5ldC93aWtpL3NwYWNlcy9BL3BhZ2VzLzI3MDk5
NDYzNjkvQnVpbGRpbmcrWGVuK0h5cGVydmlzb3IpPj4KPj4KZW5kb2JqCgoxNTcwIDAgb2JqCjw8
L1R5cGUvQW5ub3QvU3VidHlwZS9MaW5rL0JvcmRlclswIDAgMF0vUmVjdFs0Ny40NzMgMTMzLjY4
MSA1NTkuMzM5IDE0MS4xNjRdL0NvbnRlbnRzPEZFRkYwMDY4MDA3NDAwNzQwMDcwMDA3MzAwM0Ew
MDJGMDAyRjAwNzgwMDY5MDA2QzAwNjkwMDZFMDA3ODAwMkQwMDc3MDA2OTAwNkIwMDY5MDAyRTAw
NjEwMDc0MDA2QzAwNjEwMDczMDA3MzAwNjkwMDYxMDA2RTAwMkUwMDZFMDA2NTAwNzQwMDJGMDA3
NzAwNjkwMDZCMDA2OTAwMkYwMDczMDA3MDAwNjEwMDYzMDA2NTAwNzMwMDJGMDA0MTAwMkYwMDcw
MDA2MTAwNjcwMDY1MDA3MzAwMkYwMDMxMDAzNjAwMzkwMDM2MDAzMTAwMzMwMDM3MDAzODAwMzMw
MDM4MDAyRjAwNDIwMDc1MDA2OTAwNkMwMDY0MDA2OTAwNkUwMDY3MDAyQjAwNTgwMDY1MDA2RTAw
MkIwMDQ4MDA3OTAwNzAwMDY1MDA3MjAwNzYwMDY5MDA3MzAwNkYwMDcyMDAyQjAwNzQwMDY4MDA3
MjAwNkYwMDc1MDA2NzAwNjgwMDJCMDA1OTAwNkYwMDYzMDA3NDAwNkYwMDJCMDA0NjAwNkMwMDZG
MDA3Nz4vQTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VSSShodHRwczovL3hpbGlueC13aWtpLmF0bGFz
c2lhbi5uZXQvd2lraS9zcGFjZXMvQS9wYWdlcy8xNjk2MTM3ODM4L0J1aWxkaW5nK1hlbitIeXBl
cnZpc29yK3Rocm91Z2grWW9jdG8rRmxvdyk+Pgo+PgplbmRvYmoKCjE1NzEgMCBvYmoKPDwvVHlw
ZS9Bbm5vdC9TdWJ0eXBlL0xpbmsvQm9yZGVyWzAgMCAwXS9SZWN0WzQ3LjQ3MyA4OC4wNDMgMzc2
LjkzIDk1LjUyNl0vQ29udGVudHM8RkVGRjAwNjgwMDc0MDA3NDAwNzAwMDczMDAzQTAwMkYwMDJG
MDA2NDAwNkYwMDYzMDA3MzAwMkUwMDdBMDA2NTAwNzAwMDY4MDA3OTAwNzIwMDcwMDA3MjAwNkYw
MDZBMDA2NTAwNjMwMDc0MDAyRTAwNkYwMDcyMDA2NzAwMkYwMDZDMDA2MTAwNzQwMDY1MDA3MzAw
NzQwMDJGMDA2MjAwNkYwMDYxMDA3MjAwNjQwMDczMDAyRjAwNzgwMDY1MDA2RTAwMkYwMDc4MDA2
NTAwNkUwMDc2MDA2RDAwMkYwMDY0MDA2RjAwNjMwMDJGMDA2OTAwNkUwMDY0MDA2NTAwNzgwMDJF
MDA2ODAwNzQwMDZEMDA2Qz4vQTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VSSShodHRwczovL2RvY3Mu
emVwaHlycHJvamVjdC5vcmcvbGF0ZXN0L2JvYXJkcy94ZW4veGVudm0vZG9jL2luZGV4Lmh0bWwp
Pj4KPj4KZW5kb2JqCgoxNTcyIDAgb2JqCjw8L1R5cGUvQW5ub3QvU3VidHlwZS9MaW5rL0JvcmRl
clswIDAgMF0vUmVjdFs0Ny40NzMgNjAuNjYxIDY3OC42MjEgNjguMTQ0XS9Db250ZW50czxGRUZG
MDA2ODAwNzQwMDc0MDA3MDAwNzMwMDNBMDAyRjAwMkYwMDc3MDA3NzAwNzcwMDJFMDA3MzAwNkMw
MDY5MDA2NDAwNjUwMDczMDA2ODAwNjEwMDcyMDA2NTAwMkUwMDZFMDA2NTAwNzQwMDJGMDA1MzAw
NzQwMDY1MDA2NjAwNjEwMDZFMDA2RjAwNTMwMDc0MDA2MTAwNjIwMDY1MDA2QzAwNkMwMDY5MDA2
RTAwNjkwMDJGMDA3MzAwNzQwMDYxMDA3NDAwNjkwMDYzMDAyRDAwNzAwMDYxMDA3MjAwNzQwMDY5
MDA3NDAwNjkwMDZGMDA2RTAwNjkwMDZFMDA2NzAwMkQwMDc3MDA2OTAwNzQwMDY4MDAyRDAwNzgw
MDY1MDA2RTAwMkQwMDZDMDA2OTAwNkUwMDc1MDA3ODAwNzIwMDc0MDAyRDAwNjEwMDZFMDA2NDAw
MkQwMDdBMDA2NTAwNzAwMDY4MDA3OTAwNzIwMDJEMDA2MTAwMkQwMDYzMDA2RjAwNkUwMDYzMDA3
MjAwNjUwMDc0MDA2NTAwMkQwMDY1MDA2RTAwNjQwMDc0MDA2RjAwNjUwMDZFMDA2NDAwMkQwMDY1
MDA3ODAwNjEwMDZEMDA3MDAwNkMwMDY1MDAyRDAwNjUwMDZDMDA2MzAwMkQwMDZFMDA2MTAwMkQw
MDMyMDAzMDAwMzIwMDMyPi9BPDwvVHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0dHBzOi8vd3d3LnNs
aWRlc2hhcmUubmV0L1N0ZWZhbm9TdGFiZWxsaW5pL3N0YXRpYy1wYXJ0aXRpb25pbmctd2l0aC14
ZW4tbGludXhydC1hbmQtemVwaHlyLWEtY29uY3JldGUtZW5kdG9lbmQtZXhhbXBsZS1lbGMtbmEt
MjAyMik+Pgo+PgplbmRvYmoKCjE1NzMgMCBvYmoKPDwvVHlwZS9Bbm5vdC9TdWJ0eXBlL0xpbmsv
Qm9yZGVyWzAgMCAwXS9SZWN0WzQ3LjQ3MyAzMy4yNzggNDE0Ljk3IDQwLjc2MV0vQ29udGVudHM8
RkVGRjAwNjgwMDc0MDA3NDAwNzAwMDczMDAzQTAwMkYwMDJGMDA3ODAwNjkwMDZDMDA2OTAwNkUw
MDc4MDAyRDAwNzcwMDY5MDA2QjAwNjkwMDJFMDA2MTAwNzQwMDZDMDA2MTAwNzMwMDczMDA2OTAw
NjEwMDZFMDAyRTAwNkUwMDY1MDA3NDAwMkYwMDc3MDA2OTAwNkIwMDY5MDAyRjAwNzMwMDcwMDA2
MTAwNjMwMDY1MDA3MzAwMkYwMDQxMDAyRjAwNzAwMDYxMDA2NzAwNjUwMDczMDAyRjAwMzEwMDM4
MDAzODAwMzQwMDMyMDAzNTAwMzMwMDMwMDAyRjAwNTgwMDY1MDA2RTAwMkIwMDQ4MDA3OTAwNzAw
MDY1MDA3MjAwNzYwMDY5MDA3MzAwNkYwMDcyPi9BPDwvVHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0
dHBzOi8veGlsaW54LXdpa2kuYXRsYXNzaWFuLm5ldC93aWtpL3NwYWNlcy9BL3BhZ2VzLzE4ODQy
NTMwL1hlbitIeXBlcnZpc29yKT4+Cj4+CmVuZG9iagoKMTc2NCAwIG9iago8PC9UeXBlL0NhdGFs
b2cvUGFnZXMgMTU3NCAwIFIKL1BhZ2VNb2RlL1VzZU91dGxpbmVzCi9PcGVuQWN0aW9uWzEgMCBS
IC9YWVogbnVsbCBudWxsIDBdCi9PdXRsaW5lcyAxNjEyIDAgUgovU3RydWN0VHJlZVJvb3QgMTY4
NCAwIFIKL0xhbmcoZGUtREUpCi9NYXJrSW5mbzw8L01hcmtlZCB0cnVlPj4KPj4KZW5kb2JqCgox
NzY1IDAgb2JqCjw8L0NyZWF0b3I8RkVGRjAwNDkwMDZEMDA3MDAwNzIwMDY1MDA3MzAwNzM+Ci9Q
cm9kdWNlcjxGRUZGMDA0QzAwNjkwMDYyMDA3MjAwNjUwMDRGMDA2NjAwNjYwMDY5MDA2MzAwNjUw
MDIwMDAzNzAwMkUwMDM2PgovQ3JlYXRpb25EYXRlKEQ6MjAyNDAzMjYwNzEyNTYrMDEnMDAnKT4+
CmVuZG9iagoKeHJlZgowIDE3NjYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMjY4MDU3IDAwMDAw
IG4gCjAwMDAwMDAwMTkgMDAwMDAgbiAKMDAwMDAwMDY1MiAwMDAwMCBuIAowMDAwMDAwNjcyIDAw
MDAwIG4gCjAwMDAyODg4MzYgMDAwMDAgbiAKMDAwMDI4ODcxNiAwMDAwMCBuIAowMDAwMjg5MTgx
IDAwMDAwIG4gCjAwMDAyODg5MjQgMDAwMDAgbiAKMDAwMDI4ODk4NiAwMDAwMCBuIAowMDAwMjg5
MDQ4IDAwMDAwIG4gCjAwMDAyODkxMTggMDAwMDAgbiAKMDAwMDI4OTQxNCAwMDAwMCBuIAowMDAw
Mjg5Mjc5IDAwMDAwIG4gCjAwMDAyODkzNTAgMDAwMDAgbiAKMDAwMDI2ODI0NSAwMDAwMCBuIAow
MDAwMDcwODAzIDAwMDAwIG4gCjAwMDAwNzE1NDkgMDAwMDAgbiAKMDAwMDI4OTYyNCAwMDAwMCBu
IAowMDAwMjg5NTAxIDAwMDAwIG4gCjAwMDAyOTE5NzIgMDAwMDAgbiAKMDAwMDI5MTgzOSAwMDAw
MCBuIAowMDAwMjg5ODY1IDAwMDAwIG4gCjAwMDAyODk3MTUgMDAwMDAgbiAKMDAwMDI4OTc4OSAw
MDAwMCBuIAowMDAwMjkwMTAwIDAwMDAwIG4gCjAwMDAyODk5NTAgMDAwMDAgbiAKMDAwMDI5MDAy
NCAwMDAwMCBuIAowMDAwMjkwMzM1IDAwMDAwIG4gCjAwMDAyOTAxODUgMDAwMDAgbiAKMDAwMDI5
MDI1OSAwMDAwMCBuIAowMDAwMjkwNTcwIDAwMDAwIG4gCjAwMDAyOTA0MjAgMDAwMDAgbiAKMDAw
MDI5MDQ5NCAwMDAwMCBuIAowMDAwMjkwODA2IDAwMDAwIG4gCjAwMDAyOTA2NTUgMDAwMDAgbiAK
MDAwMDI5MDcyOSAwMDAwMCBuIAowMDAwMjkxMDQzIDAwMDAwIG4gCjAwMDAyOTA4OTEgMDAwMDAg
biAKMDAwMDI5MDk2NiAwMDAwMCBuIAowMDAwMjkxMjgwIDAwMDAwIG4gCjAwMDAyOTExMjggMDAw
MDAgbiAKMDAwMDI5MTIwMyAwMDAwMCBuIAowMDAwMjkxNTE3IDAwMDAwIG4gCjAwMDAyOTEzNjUg
MDAwMDAgbiAKMDAwMDI5MTQ0MCAwMDAwMCBuIAowMDAwMjkxNzU0IDAwMDAwIG4gCjAwMDAyOTE2
MDIgMDAwMDAgbiAKMDAwMDI5MTY3NyAwMDAwMCBuIAowMDAwMjY4MzgwIDAwMDAwIG4gCjAwMDAw
NzE1NzAgMDAwMDAgbiAKMDAwMDA3MTg0NCAwMDAwMCBuIAowMDAwMjkyMTc2IDAwMDAwIG4gCjAw
MDAyOTIwNTMgMDAwMDAgbiAKMDAwMDI2ODUxNSAwMDAwMCBuIAowMDAwMDcxODY1IDAwMDAwIG4g
CjAwMDAwNzI0MTQgMDAwMDAgbiAKMDAwMDI5MjM5MCAwMDAwMCBuIAowMDAwMjkyMjY3IDAwMDAw
IG4gCjAwMDAyOTM1MTkgMDAwMDAgbiAKMDAwMDI5MzQyMSAwMDAwMCBuIAowMDAwMjkyNjMxIDAw
MDAwIG4gCjAwMDAyOTI0ODEgMDAwMDAgbiAKMDAwMDI5MjU1NSAwMDAwMCBuIAowMDAwMjkyODY2
IDAwMDAwIG4gCjAwMDAyOTI3MTYgMDAwMDAgbiAKMDAwMDI5Mjc5MCAwMDAwMCBuIAowMDAwMjkz
MTAxIDAwMDAwIG4gCjAwMDAyOTI5NTEgMDAwMDAgbiAKMDAwMDI5MzAyNSAwMDAwMCBuIAowMDAw
MjkzMzM2IDAwMDAwIG4gCjAwMDAyOTMxODYgMDAwMDAgbiAKMDAwMDI5MzI2MCAwMDAwMCBuIAow
MDAwMjY4NjUwIDAwMDAwIG4gCjAwMDAwNzI0MzUgMDAwMDAgbiAKMDAwMDA3NDE0NyAwMDAwMCBu
IAowMDAwMjkzNzIzIDAwMDAwIG4gCjAwMDAyOTM2MDAgMDAwMDAgbiAKMDAwMDI5NTkxNiAwMDAw
MCBuIAowMDAwMjk1ODEwIDAwMDAwIG4gCjAwMDAyOTM5NjQgMDAwMDAgbiAKMDAwMDI5MzgxNCAw
MDAwMCBuIAowMDAwMjkzODg4IDAwMDAwIG4gCjAwMDAyOTQxOTkgMDAwMDAgbiAKMDAwMDI5NDA0
OSAwMDAwMCBuIAowMDAwMjk0MTIzIDAwMDAwIG4gCjAwMDAyOTUyNDAgMDAwMDAgbiAKMDAwMDI5
NDI4NCAwMDAwMCBuIAowMDAwMjk1MTU3IDAwMDAwIG4gCjAwMDAyOTUwNjYgMDAwMDAgbiAKMDAw
MDI5NDUwOCAwMDAwMCBuIAowMDAwMjk0MzU4IDAwMDAwIG4gCjAwMDAyOTQ0MzIgMDAwMDAgbiAK
MDAwMDI5NDc0NCAwMDAwMCBuIAowMDAwMjk0NTkzIDAwMDAwIG4gCjAwMDAyOTQ2NjcgMDAwMDAg
biAKMDAwMDI5NDk4MSAwMDAwMCBuIAowMDAwMjk0ODI5IDAwMDAwIG4gCjAwMDAyOTQ5MDQgMDAw
MDAgbiAKMDAwMDI5NTQ3OSAwMDAwMCBuIAowMDAwMjk1MzI1IDAwMDAwIG4gCjAwMDAyOTU0MDEg
MDAwMDAgbiAKMDAwMDI5NTcyMiAwMDAwMCBuIAowMDAwMjk1NTY2IDAwMDAwIG4gCjAwMDAyOTU2
NDMgMDAwMDAgbiAKMDAwMDI2ODc4NSAwMDAwMCBuIAowMDAwMDc0MTY5IDAwMDAwIG4gCjAwMDAw
NzU0NTkgMDAwMDAgbiAKMDAwMDI5NjEyMyAwMDAwMCBuIAowMDAwMjk1OTk3IDAwMDAwIG4gCjAw
MDAyOTg0OTEgMDAwMDAgbiAKMDAwMDI5ODM5NCAwMDAwMCBuIAowMDAwMjk2MzczIDAwMDAwIG4g
CjAwMDAyOTYyMTcgMDAwMDAgbiAKMDAwMDI5NjI5NCAwMDAwMCBuIAowMDAwMjk3NDYzIDAwMDAw
IG4gCjAwMDAyOTY0NjMgMDAwMDAgbiAKMDAwMDI5NzM3NiAwMDAwMCBuIAowMDAwMjk3Mjc5IDAw
MDAwIG4gCjAwMDAyOTY2OTYgMDAwMDAgbiAKMDAwMDI5NjU0MCAwMDAwMCBuIAowMDAwMjk2NjE3
IDAwMDAwIG4gCjAwMDAyOTY5NDIgMDAwMDAgbiAKMDAwMDI5Njc4NiAwMDAwMCBuIAowMDAwMjk2
ODYzIDAwMDAwIG4gCjAwMDAyOTcxODkgMDAwMDAgbiAKMDAwMDI5NzAzMiAwMDAwMCBuIAowMDAw
Mjk3MTA5IDAwMDAwIG4gCjAwMDAyOTgzMDQgMDAwMDAgbiAKMDAwMDI5NzU1MyAwMDAwMCBuIAow
MDAwMjk4MjE2IDAwMDAwIG4gCjAwMDAyOTgxMjcgMDAwMDAgbiAKMDAwMDI5Nzc4OSAwMDAwMCBu
IAowMDAwMjk3NjMxIDAwMDAwIG4gCjAwMDAyOTc3MDkgMDAwMDAgbiAKMDAwMDI5ODAzNyAwMDAw
MCBuIAowMDAwMjk3ODc5IDAwMDAwIG4gCjAwMDAyOTc5NTcgMDAwMDAgbiAKMDAwMDI2ODkyMiAw
MDAwMCBuIAowMDAwMDc1NDgyIDAwMDAwIG4gCjAwMDAwNzY3OTggMDAwMDAgbiAKMDAwMDI5ODcw
MSAwMDAwMCBuIAowMDAwMjk4NTc1IDAwMDAwIG4gCjAwMDAzMDA2NTEgMDAwMDAgbiAKMDAwMDMw
MDUyMiAwMDAwMCBuIAowMDAwMjk4OTUxIDAwMDAwIG4gCjAwMDAyOTg3OTUgMDAwMDAgbiAKMDAw
MDI5ODg3MiAwMDAwMCBuIAowMDAwMjk5MTk3IDAwMDAwIG4gCjAwMDAyOTkwNDEgMDAwMDAgbiAK
MDAwMDI5OTExOCAwMDAwMCBuIAowMDAwMjk5NDQzIDAwMDAwIG4gCjAwMDAyOTkyODcgMDAwMDAg
biAKMDAwMDI5OTM2NCAwMDAwMCBuIAowMDAwMjk5Njg5IDAwMDAwIG4gCjAwMDAyOTk1MzMgMDAw
MDAgbiAKMDAwMDI5OTYxMCAwMDAwMCBuIAowMDAwMjk5OTM2IDAwMDAwIG4gCjAwMDAyOTk3Nzkg
MDAwMDAgbiAKMDAwMDI5OTg1NiAwMDAwMCBuIAowMDAwMzAwMTg0IDAwMDAwIG4gCjAwMDAzMDAw
MjYgMDAwMDAgbiAKMDAwMDMwMDEwNCAwMDAwMCBuIAowMDAwMzAwNDMyIDAwMDAwIG4gCjAwMDAz
MDAyNzQgMDAwMDAgbiAKMDAwMDMwMDM1MiAwMDAwMCBuIAowMDAwMjY5MDU5IDAwMDAwIG4gCjAw
MDAwNzY4MjEgMDAwMDAgbiAKMDAwMDA3ODEzNyAwMDAwMCBuIAowMDAwMzAwODYxIDAwMDAwIG4g
CjAwMDAzMDA3MzUgMDAwMDAgbiAKMDAwMDMwMjI5OSAwMDAwMCBuIAowMDAwMzAyMTg2IDAwMDAw
IG4gCjAwMDAzMDExMTEgMDAwMDAgbiAKMDAwMDMwMDk1NSAwMDAwMCBuIAowMDAwMzAxMDMyIDAw
MDAwIG4gCjAwMDAzMDEzNTcgMDAwMDAgbiAKMDAwMDMwMTIwMSAwMDAwMCBuIAowMDAwMzAxMjc4
IDAwMDAwIG4gCjAwMDAzMDE2MDMgMDAwMDAgbiAKMDAwMDMwMTQ0NyAwMDAwMCBuIAowMDAwMzAx
NTI0IDAwMDAwIG4gCjAwMDAzMDE4NDkgMDAwMDAgbiAKMDAwMDMwMTY5MyAwMDAwMCBuIAowMDAw
MzAxNzcwIDAwMDAwIG4gCjAwMDAzMDIwOTYgMDAwMDAgbiAKMDAwMDMwMTkzOSAwMDAwMCBuIAow
MDAwMzAyMDE2IDAwMDAwIG4gCjAwMDAyNjkxOTYgMDAwMDAgbiAKMDAwMDA3ODE2MCAwMDAwMCBu
IAowMDAwMDgwNTcwIDAwMDAwIG4gCjAwMDAzMDI1MDkgMDAwMDAgbiAKMDAwMDMwMjM4MyAwMDAw
MCBuIAowMDAwMzAzNjY3IDAwMDAwIG4gCjAwMDAzMDI2MDMgMDAwMDAgbiAKMDAwMDMwMjY3OCAw
MDAwMCBuIAowMDAwMzAyNzUzIDAwMDAwIG4gCjAwMDAzMDI4MjggMDAwMDAgbiAKMDAwMDMwMjkw
MyAwMDAwMCBuIAowMDAwMzAyOTc5IDAwMDAwIG4gCjAwMDAzMDMwNTUgMDAwMDAgbiAKMDAwMDMw
MzEzMSAwMDAwMCBuIAowMDAwMzAzMjA3IDAwMDAwIG4gCjAwMDAzMDMyODMgMDAwMDAgbiAKMDAw
MDMwMzM1OSAwMDAwMCBuIAowMDAwMzAzNDM1IDAwMDAwIG4gCjAwMDAzMDM1MTEgMDAwMDAgbiAK
MDAwMDI2OTMzMyAwMDAwMCBuIAowMDAwMDgwNTkzIDAwMDAwIG4gCjAwMDAwODE4MDggMDAwMDAg
biAKMDAwMDMwNDAxMyAwMDAwMCBuIAowMDAwMzAzODg3IDAwMDAwIG4gCjAwMDAzMDUxOTYgMDAw
MDAgbiAKMDAwMDMwNTA5MSAwMDAwMCBuIAowMDAwMzA0MjYzIDAwMDAwIG4gCjAwMDAzMDQxMDcg
MDAwMDAgbiAKMDAwMDMwNDE4NCAwMDAwMCBuIAowMDAwMzA0NTA5IDAwMDAwIG4gCjAwMDAzMDQz
NTMgMDAwMDAgbiAKMDAwMDMwNDQzMCAwMDAwMCBuIAowMDAwMzA0NzU1IDAwMDAwIG4gCjAwMDAz
MDQ1OTkgMDAwMDAgbiAKMDAwMDMwNDY3NiAwMDAwMCBuIAowMDAwMzA1MDAxIDAwMDAwIG4gCjAw
MDAzMDQ4NDUgMDAwMDAgbiAKMDAwMDMwNDkyMiAwMDAwMCBuIAowMDAwMjY5NDcwIDAwMDAwIG4g
CjAwMDAwODE4MzEgMDAwMDAgbiAKMDAwMDA4MjE0NSAwMDAwMCBuIAowMDAwMzA1NDA2IDAwMDAw
IG4gCjAwMDAzMDUyODAgMDAwMDAgbiAKMDAwMDI2OTYwOCAwMDAwMCBuIAowMDAwMDgyMTY3IDAw
MDAwIG4gCjAwMDAwODI4MDUgMDAwMDAgbiAKMDAwMDMwNTYyNiAwMDAwMCBuIAowMDAwMzA1NTAw
IDAwMDAwIG4gCjAwMDAzMDc4MzIgMDAwMDAgbiAKMDAwMDMwNzY5NSAwMDAwMCBuIAowMDAwMzA1
ODc2IDAwMDAwIG4gCjAwMDAzMDU3MjAgMDAwMDAgbiAKMDAwMDMwNTc5NyAwMDAwMCBuIAowMDAw
MzA2MTIyIDAwMDAwIG4gCjAwMDAzMDU5NjYgMDAwMDAgbiAKMDAwMDMwNjA0MyAwMDAwMCBuIAow
MDAwMzA2MzY4IDAwMDAwIG4gCjAwMDAzMDYyMTIgMDAwMDAgbiAKMDAwMDMwNjI4OSAwMDAwMCBu
IAowMDAwMzA2NjE0IDAwMDAwIG4gCjAwMDAzMDY0NTggMDAwMDAgbiAKMDAwMDMwNjUzNSAwMDAw
MCBuIAowMDAwMzA2ODYxIDAwMDAwIG4gCjAwMDAzMDY3MDQgMDAwMDAgbiAKMDAwMDMwNjc4MSAw
MDAwMCBuIAowMDAwMzA3MTA5IDAwMDAwIG4gCjAwMDAzMDY5NTEgMDAwMDAgbiAKMDAwMDMwNzAy
OSAwMDAwMCBuIAowMDAwMzA3MzU3IDAwMDAwIG4gCjAwMDAzMDcxOTkgMDAwMDAgbiAKMDAwMDMw
NzI3NyAwMDAwMCBuIAowMDAwMzA3NjA1IDAwMDAwIG4gCjAwMDAzMDc0NDcgMDAwMDAgbiAKMDAw
MDMwNzUyNSAwMDAwMCBuIAowMDAwMjY5NzQ2IDAwMDAwIG4gCjAwMDAwODI4MjcgMDAwMDAgbiAK
MDAwMDA4MzU5NyAwMDAwMCBuIAowMDAwMzA4MDQyIDAwMDAwIG4gCjAwMDAzMDc5MTYgMDAwMDAg
biAKMDAwMDMxMDI0OCAwMDAwMCBuIAowMDAwMzEwMTExIDAwMDAwIG4gCjAwMDAzMDgyOTIgMDAw
MDAgbiAKMDAwMDMwODEzNiAwMDAwMCBuIAowMDAwMzA4MjEzIDAwMDAwIG4gCjAwMDAzMDg1Mzgg
MDAwMDAgbiAKMDAwMDMwODM4MiAwMDAwMCBuIAowMDAwMzA4NDU5IDAwMDAwIG4gCjAwMDAzMDg3
ODQgMDAwMDAgbiAKMDAwMDMwODYyOCAwMDAwMCBuIAowMDAwMzA4NzA1IDAwMDAwIG4gCjAwMDAz
MDkwMzAgMDAwMDAgbiAKMDAwMDMwODg3NCAwMDAwMCBuIAowMDAwMzA4OTUxIDAwMDAwIG4gCjAw
MDAzMDkyNzcgMDAwMDAgbiAKMDAwMDMwOTEyMCAwMDAwMCBuIAowMDAwMzA5MTk3IDAwMDAwIG4g
CjAwMDAzMDk1MjUgMDAwMDAgbiAKMDAwMDMwOTM2NyAwMDAwMCBuIAowMDAwMzA5NDQ1IDAwMDAw
IG4gCjAwMDAzMDk3NzMgMDAwMDAgbiAKMDAwMDMwOTYxNSAwMDAwMCBuIAowMDAwMzA5NjkzIDAw
MDAwIG4gCjAwMDAzMTAwMjEgMDAwMDAgbiAKMDAwMDMwOTg2MyAwMDAwMCBuIAowMDAwMzA5OTQx
IDAwMDAwIG4gCjAwMDAyNjk4ODQgMDAwMDAgbiAKMDAwMDA4MzYxOSAwMDAwMCBuIAowMDAwMDg0
ODQwIDAwMDAwIG4gCjAwMDAzMTA0NTggMDAwMDAgbiAKMDAwMDMxMDMzMiAwMDAwMCBuIAowMDAw
MzEzNDE5IDAwMDAwIG4gCjAwMDAzMTMzMjIgMDAwMDAgbiAKMDAwMDMxMTU1MSAwMDAwMCBuIAow
MDAwMzEwNTUyIDAwMDAwIG4gCjAwMDAzMTE0NjQgMDAwMDAgbiAKMDAwMDMxMTM2NyAwMDAwMCBu
IAowMDAwMzEwNzg1IDAwMDAwIG4gCjAwMDAzMTA2MjkgMDAwMDAgbiAKMDAwMDMxMDcwNiAwMDAw
MCBuIAowMDAwMzExMDMxIDAwMDAwIG4gCjAwMDAzMTA4NzUgMDAwMDAgbiAKMDAwMDMxMDk1MiAw
MDAwMCBuIAowMDAwMzExMjc3IDAwMDAwIG4gCjAwMDAzMTExMjEgMDAwMDAgbiAKMDAwMDMxMTE5
OCAwMDAwMCBuIAowMDAwMzEyMzkxIDAwMDAwIG4gCjAwMDAzMTE2NDEgMDAwMDAgbiAKMDAwMDMx
MjMwMyAwMDAwMCBuIAowMDAwMzEyMjE0IDAwMDAwIG4gCjAwMDAzMTE4NzYgMDAwMDAgbiAKMDAw
MDMxMTcxOCAwMDAwMCBuIAowMDAwMzExNzk2IDAwMDAwIG4gCjAwMDAzMTIxMjQgMDAwMDAgbiAK
MDAwMDMxMTk2NiAwMDAwMCBuIAowMDAwMzEyMDQ0IDAwMDAwIG4gCjAwMDAzMTMyMzIgMDAwMDAg
biAKMDAwMDMxMjQ4MSAwMDAwMCBuIAowMDAwMzEzMTQ0IDAwMDAwIG4gCjAwMDAzMTMwNTUgMDAw
MDAgbiAKMDAwMDMxMjcxNyAwMDAwMCBuIAowMDAwMzEyNTU5IDAwMDAwIG4gCjAwMDAzMTI2Mzcg
MDAwMDAgbiAKMDAwMDMxMjk2NSAwMDAwMCBuIAowMDAwMzEyODA3IDAwMDAwIG4gCjAwMDAzMTI4
ODUgMDAwMDAgbiAKMDAwMDI3MDAyMiAwMDAwMCBuIAowMDAwMDg0ODYzIDAwMDAwIG4gCjAwMDAw
ODYwODQgMDAwMDAgbiAKMDAwMDMxMzYyOSAwMDAwMCBuIAowMDAwMzEzNTAzIDAwMDAwIG4gCjAw
MDAzMTU5MTYgMDAwMDAgbiAKMDAwMDMxNTgwMyAwMDAwMCBuIAowMDAwMzEzODc5IDAwMDAwIG4g
CjAwMDAzMTM3MjMgMDAwMDAgbiAKMDAwMDMxMzgwMCAwMDAwMCBuIAowMDAwMzE0MTI1IDAwMDAw
IG4gCjAwMDAzMTM5NjkgMDAwMDAgbiAKMDAwMDMxNDA0NiAwMDAwMCBuIAowMDAwMzE0MzcxIDAw
MDAwIG4gCjAwMDAzMTQyMTUgMDAwMDAgbiAKMDAwMDMxNDI5MiAwMDAwMCBuIAowMDAwMzE0NjE3
IDAwMDAwIG4gCjAwMDAzMTQ0NjEgMDAwMDAgbiAKMDAwMDMxNDUzOCAwMDAwMCBuIAowMDAwMzE1
NzEzIDAwMDAwIG4gCjAwMDAzMTQ3MDcgMDAwMDAgbiAKMDAwMDMxNTYyNSAwMDAwMCBuIAowMDAw
MzE1NTI4IDAwMDAwIG4gCjAwMDAzMTQ5NDIgMDAwMDAgbiAKMDAwMDMxNDc4NCAwMDAwMCBuIAow
MDAwMzE0ODYyIDAwMDAwIG4gCjAwMDAzMTUxOTAgMDAwMDAgbiAKMDAwMDMxNTAzMiAwMDAwMCBu
IAowMDAwMzE1MTEwIDAwMDAwIG4gCjAwMDAzMTU0MzggMDAwMDAgbiAKMDAwMDMxNTI4MCAwMDAw
MCBuIAowMDAwMzE1MzU4IDAwMDAwIG4gCjAwMDAyNzAxNjAgMDAwMDAgbiAKMDAwMDA4NjEwNyAw
MDAwMCBuIAowMDAwMDg2ODk3IDAwMDAwIG4gCjAwMDAzMTYxMjYgMDAwMDAgbiAKMDAwMDMxNjAw
MCAwMDAwMCBuIAowMDAwMzE3MDU1IDAwMDAwIG4gCjAwMDAzMTY5NTggMDAwMDAgbiAKMDAwMDMx
NjM3NiAwMDAwMCBuIAowMDAwMzE2MjIwIDAwMDAwIG4gCjAwMDAzMTYyOTcgMDAwMDAgbiAKMDAw
MDMxNjYyMiAwMDAwMCBuIAowMDAwMzE2NDY2IDAwMDAwIG4gCjAwMDAzMTY1NDMgMDAwMDAgbiAK
MDAwMDMxNjg2OCAwMDAwMCBuIAowMDAwMzE2NzEyIDAwMDAwIG4gCjAwMDAzMTY3ODkgMDAwMDAg
biAKMDAwMDI3MDI5OCAwMDAwMCBuIAowMDAwMDg2OTE5IDAwMDAwIG4gCjAwMDAwODc5NjkgMDAw
MDAgbiAKMDAwMDMxNzI2NSAwMDAwMCBuIAowMDAwMzE3MTM5IDAwMDAwIG4gCjAwMDAzMTk1NTIg
MDAwMDAgbiAKMDAwMDMxOTQ0NyAwMDAwMCBuIAowMDAwMzE3NTE1IDAwMDAwIG4gCjAwMDAzMTcz
NTkgMDAwMDAgbiAKMDAwMDMxNzQzNiAwMDAwMCBuIAowMDAwMzE3NzYxIDAwMDAwIG4gCjAwMDAz
MTc2MDUgMDAwMDAgbiAKMDAwMDMxNzY4MiAwMDAwMCBuIAowMDAwMzE4MDA3IDAwMDAwIG4gCjAw
MDAzMTc4NTEgMDAwMDAgbiAKMDAwMDMxNzkyOCAwMDAwMCBuIAowMDAwMzE5MzU3IDAwMDAwIG4g
CjAwMDAzMTgwOTcgMDAwMDAgbiAKMDAwMDMxOTI3MCAwMDAwMCBuIAowMDAwMzE5MTY1IDAwMDAw
IG4gCjAwMDAzMTgzMzEgMDAwMDAgbiAKMDAwMDMxODE3NCAwMDAwMCBuIAowMDAwMzE4MjUxIDAw
MDAwIG4gCjAwMDAzMTg1NzkgMDAwMDAgbiAKMDAwMDMxODQyMSAwMDAwMCBuIAowMDAwMzE4NDk5
IDAwMDAwIG4gCjAwMDAzMTg4MjcgMDAwMDAgbiAKMDAwMDMxODY2OSAwMDAwMCBuIAowMDAwMzE4
NzQ3IDAwMDAwIG4gCjAwMDAzMTkwNzUgMDAwMDAgbiAKMDAwMDMxODkxNyAwMDAwMCBuIAowMDAw
MzE4OTk1IDAwMDAwIG4gCjAwMDAyNzA0MzYgMDAwMDAgbiAKMDAwMDA4Nzk5MSAwMDAwMCBuIAow
MDAwMDg5MDU0IDAwMDAwIG4gCjAwMDAzMTk3NjIgMDAwMDAgbiAKMDAwMDMxOTYzNiAwMDAwMCBu
IAowMDAwMzIxNDU2IDAwMDAwIG4gCjAwMDAzMjEzMzUgMDAwMDAgbiAKMDAwMDMyMDAxMiAwMDAw
MCBuIAowMDAwMzE5ODU2IDAwMDAwIG4gCjAwMDAzMTk5MzMgMDAwMDAgbiAKMDAwMDMyMDI1OCAw
MDAwMCBuIAowMDAwMzIwMTAyIDAwMDAwIG4gCjAwMDAzMjAxNzkgMDAwMDAgbiAKMDAwMDMyMDUw
NCAwMDAwMCBuIAowMDAwMzIwMzQ4IDAwMDAwIG4gCjAwMDAzMjA0MjUgMDAwMDAgbiAKMDAwMDMy
MDc1MCAwMDAwMCBuIAowMDAwMzIwNTk0IDAwMDAwIG4gCjAwMDAzMjA2NzEgMDAwMDAgbiAKMDAw
MDMyMDk5NyAwMDAwMCBuIAowMDAwMzIwODQwIDAwMDAwIG4gCjAwMDAzMjA5MTcgMDAwMDAgbiAK
MDAwMDMyMTI0NSAwMDAwMCBuIAowMDAwMzIxMDg3IDAwMDAwIG4gCjAwMDAzMjExNjUgMDAwMDAg
biAKMDAwMDI3MDU3NCAwMDAwMCBuIAowMDAwMDg5MDc2IDAwMDAwIG4gCjAwMDAwOTA0NTggMDAw
MDAgbiAKMDAwMDMyMTY2NiAwMDAwMCBuIAowMDAwMzIxNTQwIDAwMDAwIG4gCjAwMDAzMjE3NjAg
MDAwMDAgbiAKMDAwMDMyMTgzMSAwMDAwMCBuIAowMDAwMzIxOTAyIDAwMDAwIG4gCjAwMDAzMjQx
NjggMDAwMDAgbiAKMDAwMDMyMTk3MyAwMDAwMCBuIAowMDAwMzIyMDQ4IDAwMDAwIG4gCjAwMDAz
MjIxMjMgMDAwMDAgbiAKMDAwMDMyMjE5OCAwMDAwMCBuIAowMDAwMzIyMjczIDAwMDAwIG4gCjAw
MDAzMjIzNDggMDAwMDAgbiAKMDAwMDMyMjQyMyAwMDAwMCBuIAowMDAwMzIyNDk4IDAwMDAwIG4g
CjAwMDAzMjI1NzMgMDAwMDAgbiAKMDAwMDMyMjY0OCAwMDAwMCBuIAowMDAwMzIyNzI0IDAwMDAw
IG4gCjAwMDAzMjI4MDAgMDAwMDAgbiAKMDAwMDMyMjg3NiAwMDAwMCBuIAowMDAwMzIyOTUyIDAw
MDAwIG4gCjAwMDAzMjMwMjggMDAwMDAgbiAKMDAwMDMyMzEwNCAwMDAwMCBuIAowMDAwMzIzMTgw
IDAwMDAwIG4gCjAwMDAzMjMyNTYgMDAwMDAgbiAKMDAwMDMyMzMzMiAwMDAwMCBuIAowMDAwMzIz
NDA4IDAwMDAwIG4gCjAwMDAzMjM0ODQgMDAwMDAgbiAKMDAwMDMyMzU2MCAwMDAwMCBuIAowMDAw
MzIzNjM2IDAwMDAwIG4gCjAwMDAzMjM3MTIgMDAwMDAgbiAKMDAwMDMyMzc4OCAwMDAwMCBuIAow
MDAwMzIzODY0IDAwMDAwIG4gCjAwMDAzMjM5NDAgMDAwMDAgbiAKMDAwMDMyNDAxNiAwMDAwMCBu
IAowMDAwMzI0MDkyIDAwMDAwIG4gCjAwMDAyNzA3MTIgMDAwMDAgbiAKMDAwMDA5MDQ4MSAwMDAw
MCBuIAowMDAwMDkxNzU5IDAwMDAwIG4gCjAwMDAzMjQ2MDIgMDAwMDAgbiAKMDAwMDMyNDQ3NiAw
MDAwMCBuIAowMDAwMzI2Mjk2IDAwMDAwIG4gCjAwMDAzMjYxNzUgMDAwMDAgbiAKMDAwMDMyNDg1
MiAwMDAwMCBuIAowMDAwMzI0Njk2IDAwMDAwIG4gCjAwMDAzMjQ3NzMgMDAwMDAgbiAKMDAwMDMy
NTA5OCAwMDAwMCBuIAowMDAwMzI0OTQyIDAwMDAwIG4gCjAwMDAzMjUwMTkgMDAwMDAgbiAKMDAw
MDMyNTM0NCAwMDAwMCBuIAowMDAwMzI1MTg4IDAwMDAwIG4gCjAwMDAzMjUyNjUgMDAwMDAgbiAK
MDAwMDMyNTU5MCAwMDAwMCBuIAowMDAwMzI1NDM0IDAwMDAwIG4gCjAwMDAzMjU1MTEgMDAwMDAg
biAKMDAwMDMyNTgzNyAwMDAwMCBuIAowMDAwMzI1NjgwIDAwMDAwIG4gCjAwMDAzMjU3NTcgMDAw
MDAgbiAKMDAwMDMyNjA4NSAwMDAwMCBuIAowMDAwMzI1OTI3IDAwMDAwIG4gCjAwMDAzMjYwMDUg
MDAwMDAgbiAKMDAwMDI3MDg1MCAwMDAwMCBuIAowMDAwMDkxNzgyIDAwMDAwIG4gCjAwMDAwOTI4
NDMgMDAwMDAgbiAKMDAwMDMyNjUwNiAwMDAwMCBuIAowMDAwMzI2MzgwIDAwMDAwIG4gCjAwMDAz
MjY2ODcgMDAwMDAgbiAKMDAwMDA5Mjg2NSAwMDAwMCBuIAowMDAwMzI3NjE1IDAwMDAwIG4gCjAw
MDAzMjc1MTggMDAwMDAgbiAKMDAwMDMyNjkzNiAwMDAwMCBuIAowMDAwMzI2NzgwIDAwMDAwIG4g
CjAwMDAzMjY4NTcgMDAwMDAgbiAKMDAwMDMyNzE4MiAwMDAwMCBuIAowMDAwMzI3MDI2IDAwMDAw
IG4gCjAwMDAzMjcxMDMgMDAwMDAgbiAKMDAwMDMyNzQyOCAwMDAwMCBuIAowMDAwMzI3MjcyIDAw
MDAwIG4gCjAwMDAzMjczNDkgMDAwMDAgbiAKMDAwMDEyOTE4NyAwMDAwMCBuIAowMDAwMjcwOTg4
IDAwMDAwIG4gCjAwMDAxMjkyMTEgMDAwMDAgbiAKMDAwMDEzMDE1NiAwMDAwMCBuIAowMDAwMzI3
ODI1IDAwMDAwIG4gCjAwMDAzMjc2OTkgMDAwMDAgbiAKMDAwMDMyOTAwOCAwMDAwMCBuIAowMDAw
MzI4OTAzIDAwMDAwIG4gCjAwMDAzMjgwNzUgMDAwMDAgbiAKMDAwMDMyNzkxOSAwMDAwMCBuIAow
MDAwMzI3OTk2IDAwMDAwIG4gCjAwMDAzMjgzMjEgMDAwMDAgbiAKMDAwMDMyODE2NSAwMDAwMCBu
IAowMDAwMzI4MjQyIDAwMDAwIG4gCjAwMDAzMjg1NjcgMDAwMDAgbiAKMDAwMDMyODQxMSAwMDAw
MCBuIAowMDAwMzI4NDg4IDAwMDAwIG4gCjAwMDAzMjg4MTMgMDAwMDAgbiAKMDAwMDMyODY1NyAw
MDAwMCBuIAowMDAwMzI4NzM0IDAwMDAwIG4gCjAwMDAyNzExMjYgMDAwMDAgbiAKMDAwMDEzMDE3
OCAwMDAwMCBuIAowMDAwMTMwNDkxIDAwMDAwIG4gCjAwMDAzMjkyMTggMDAwMDAgbiAKMDAwMDMy
OTA5MiAwMDAwMCBuIAowMDAwMjcxMjY0IDAwMDAwIG4gCjAwMDAxMzA1MTMgMDAwMDAgbiAKMDAw
MDEzMTA1OCAwMDAwMCBuIAowMDAwMzI5NDM4IDAwMDAwIG4gCjAwMDAzMjkzMTIgMDAwMDAgbiAK
MDAwMDMzMDYyMSAwMDAwMCBuIAowMDAwMzMwNTE2IDAwMDAwIG4gCjAwMDAzMjk2ODggMDAwMDAg
biAKMDAwMDMyOTUzMiAwMDAwMCBuIAowMDAwMzI5NjA5IDAwMDAwIG4gCjAwMDAzMjk5MzQgMDAw
MDAgbiAKMDAwMDMyOTc3OCAwMDAwMCBuIAowMDAwMzI5ODU1IDAwMDAwIG4gCjAwMDAzMzAxODAg
MDAwMDAgbiAKMDAwMDMzMDAyNCAwMDAwMCBuIAowMDAwMzMwMTAxIDAwMDAwIG4gCjAwMDAzMzA0
MjYgMDAwMDAgbiAKMDAwMDMzMDI3MCAwMDAwMCBuIAowMDAwMzMwMzQ3IDAwMDAwIG4gCjAwMDAy
NzE0MDIgMDAwMDAgbiAKMDAwMDEzMTA4MCAwMDAwMCBuIAowMDAwMTMzNDczIDAwMDAwIG4gCjAw
MDAzMzA4MzEgMDAwMDAgbiAKMDAwMDMzMDcwNSAwMDAwMCBuIAowMDAwMzMxOTg5IDAwMDAwIG4g
CjAwMDAzMzA5MjUgMDAwMDAgbiAKMDAwMDMzMTAwMCAwMDAwMCBuIAowMDAwMzMxMDc1IDAwMDAw
IG4gCjAwMDAzMzExNTAgMDAwMDAgbiAKMDAwMDMzMTIyNSAwMDAwMCBuIAowMDAwMzMxMzAxIDAw
MDAwIG4gCjAwMDAzMzEzNzcgMDAwMDAgbiAKMDAwMDMzMTQ1MyAwMDAwMCBuIAowMDAwMzMxNTI5
IDAwMDAwIG4gCjAwMDAzMzE2MDUgMDAwMDAgbiAKMDAwMDMzMTY4MSAwMDAwMCBuIAowMDAwMzMx
NzU3IDAwMDAwIG4gCjAwMDAzMzE4MzMgMDAwMDAgbiAKMDAwMDI3MTU0MCAwMDAwMCBuIAowMDAw
MTMzNDk2IDAwMDAwIG4gCjAwMDAxMzYxMjYgMDAwMDAgbiAKMDAwMDMzMjMzNSAwMDAwMCBuIAow
MDAwMzMyMjA5IDAwMDAwIG4gCjAwMDAzMzM1NzcgMDAwMDAgbiAKMDAwMDMzMjQyOSAwMDAwMCBu
IAowMDAwMzMyNTA0IDAwMDAwIG4gCjAwMDAzMzI1NzkgMDAwMDAgbiAKMDAwMDMzMjY1NCAwMDAw
MCBuIAowMDAwMzMyNzI5IDAwMDAwIG4gCjAwMDAzMzI4MDUgMDAwMDAgbiAKMDAwMDMzMjg4MSAw
MDAwMCBuIAowMDAwMzMyOTU3IDAwMDAwIG4gCjAwMDAzMzMwMzMgMDAwMDAgbiAKMDAwMDMzMzEw
OSAwMDAwMCBuIAowMDAwMzMzMTg1IDAwMDAwIG4gCjAwMDAzMzMyNjEgMDAwMDAgbiAKMDAwMDMz
MzMzNyAwMDAwMCBuIAowMDAwMzMzNDEzIDAwMDAwIG4gCjAwMDAyNzE2NzggMDAwMDAgbiAKMDAw
MDEzNjE0OSAwMDAwMCBuIAowMDAwMTM3MTUyIDAwMDAwIG4gCjAwMDAzMzM5MzcgMDAwMDAgbiAK
MDAwMDMzMzgxMSAwMDAwMCBuIAowMDAwMzM1MTIwIDAwMDAwIG4gCjAwMDAzMzUwMTUgMDAwMDAg
biAKMDAwMDMzNDE4NyAwMDAwMCBuIAowMDAwMzM0MDMxIDAwMDAwIG4gCjAwMDAzMzQxMDggMDAw
MDAgbiAKMDAwMDMzNDQzMyAwMDAwMCBuIAowMDAwMzM0Mjc3IDAwMDAwIG4gCjAwMDAzMzQzNTQg
MDAwMDAgbiAKMDAwMDMzNDY3OSAwMDAwMCBuIAowMDAwMzM0NTIzIDAwMDAwIG4gCjAwMDAzMzQ2
MDAgMDAwMDAgbiAKMDAwMDMzNDkyNSAwMDAwMCBuIAowMDAwMzM0NzY5IDAwMDAwIG4gCjAwMDAz
MzQ4NDYgMDAwMDAgbiAKMDAwMDI3MTgxNiAwMDAwMCBuIAowMDAwMTM3MTc0IDAwMDAwIG4gCjAw
MDAxMzc4MjkgMDAwMDAgbiAKMDAwMDMzNTMzMCAwMDAwMCBuIAowMDAwMzM1MjA0IDAwMDAwIG4g
CjAwMDAzMzYyNTkgMDAwMDAgbiAKMDAwMDMzNjE2MiAwMDAwMCBuIAowMDAwMzM1NTgwIDAwMDAw
IG4gCjAwMDAzMzU0MjQgMDAwMDAgbiAKMDAwMDMzNTUwMSAwMDAwMCBuIAowMDAwMzM1ODI2IDAw
MDAwIG4gCjAwMDAzMzU2NzAgMDAwMDAgbiAKMDAwMDMzNTc0NyAwMDAwMCBuIAowMDAwMzM2MDcy
IDAwMDAwIG4gCjAwMDAzMzU5MTYgMDAwMDAgbiAKMDAwMDMzNTk5MyAwMDAwMCBuIAowMDAwMjcx
OTU0IDAwMDAwIG4gCjAwMDAxMzc4NTEgMDAwMDAgbiAKMDAwMDEzODEzOCAwMDAwMCBuIAowMDAw
MzM2NDY5IDAwMDAwIG4gCjAwMDAzMzYzNDMgMDAwMDAgbiAKMDAwMDI3MjA5MiAwMDAwMCBuIAow
MDAwMTM4MTYwIDAwMDAwIG4gCjAwMDAxMzg3NTQgMDAwMDAgbiAKMDAwMDMzNjY4OSAwMDAwMCBu
IAowMDAwMzM2NTYzIDAwMDAwIG4gCjAwMDAzMzgzODMgMDAwMDAgbiAKMDAwMDMzODI2MiAwMDAw
MCBuIAowMDAwMzM2OTM5IDAwMDAwIG4gCjAwMDAzMzY3ODMgMDAwMDAgbiAKMDAwMDMzNjg2MCAw
MDAwMCBuIAowMDAwMzM3MTg1IDAwMDAwIG4gCjAwMDAzMzcwMjkgMDAwMDAgbiAKMDAwMDMzNzEw
NiAwMDAwMCBuIAowMDAwMzM3NDMxIDAwMDAwIG4gCjAwMDAzMzcyNzUgMDAwMDAgbiAKMDAwMDMz
NzM1MiAwMDAwMCBuIAowMDAwMzM3Njc3IDAwMDAwIG4gCjAwMDAzMzc1MjEgMDAwMDAgbiAKMDAw
MDMzNzU5OCAwMDAwMCBuIAowMDAwMzM3OTI0IDAwMDAwIG4gCjAwMDAzMzc3NjcgMDAwMDAgbiAK
MDAwMDMzNzg0NCAwMDAwMCBuIAowMDAwMzM4MTcyIDAwMDAwIG4gCjAwMDAzMzgwMTQgMDAwMDAg
biAKMDAwMDMzODA5MiAwMDAwMCBuIAowMDAwMjcyMjMwIDAwMDAwIG4gCjAwMDAxMzg3NzYgMDAw
MDAgbiAKMDAwMDEzOTg3MyAwMDAwMCBuIAowMDAwMzM4NTkzIDAwMDAwIG4gCjAwMDAzMzg0Njcg
MDAwMDAgbiAKMDAwMDM0MDM2OCAwMDAwMCBuIAowMDAwMzQwMjc5IDAwMDAwIG4gCjAwMDAzMzg4
NDMgMDAwMDAgbiAKMDAwMDMzODY4NyAwMDAwMCBuIAowMDAwMzM4NzY0IDAwMDAwIG4gCjAwMDAz
NDAxODkgMDAwMDAgbiAKMDAwMDMzODkzMyAwMDAwMCBuIAowMDAwMzQwMTAyIDAwMDAwIG4gCjAw
MDAzMzk5OTcgMDAwMDAgbiAKMDAwMDMzOTE2NiAwMDAwMCBuIAowMDAwMzM5MDEwIDAwMDAwIG4g
CjAwMDAzMzkwODcgMDAwMDAgbiAKMDAwMDMzOTQxMiAwMDAwMCBuIAowMDAwMzM5MjU2IDAwMDAw
IG4gCjAwMDAzMzkzMzMgMDAwMDAgbiAKMDAwMDMzOTY1OSAwMDAwMCBuIAowMDAwMzM5NTAyIDAw
MDAwIG4gCjAwMDAzMzk1NzkgMDAwMDAgbiAKMDAwMDMzOTkwNyAwMDAwMCBuIAowMDAwMzM5NzQ5
IDAwMDAwIG4gCjAwMDAzMzk4MjcgMDAwMDAgbiAKMDAwMDI3MjM2OCAwMDAwMCBuIAowMDAwMTM5
ODk2IDAwMDAwIG4gCjAwMDAxNDEwNTQgMDAwMDAgbiAKMDAwMDM0MDU3OCAwMDAwMCBuIAowMDAw
MzQwNDUyIDAwMDAwIG4gCjAwMDAzNDIzNTMgMDAwMDAgbiAKMDAwMDM0MjI1NiAwMDAwMCBuIAow
MDAwMzQwODI4IDAwMDAwIG4gCjAwMDAzNDA2NzIgMDAwMDAgbiAKMDAwMDM0MDc0OSAwMDAwMCBu
IAowMDAwMzQxMDc0IDAwMDAwIG4gCjAwMDAzNDA5MTggMDAwMDAgbiAKMDAwMDM0MDk5NSAwMDAw
MCBuIAowMDAwMzQyMTY2IDAwMDAwIG4gCjAwMDAzNDExNjQgMDAwMDAgbiAKMDAwMDM0MjA3OSAw
MDAwMCBuIAowMDAwMzQxOTgyIDAwMDAwIG4gCjAwMDAzNDEzOTcgMDAwMDAgbiAKMDAwMDM0MTI0
MSAwMDAwMCBuIAowMDAwMzQxMzE4IDAwMDAwIG4gCjAwMDAzNDE2NDQgMDAwMDAgbiAKMDAwMDM0
MTQ4NyAwMDAwMCBuIAowMDAwMzQxNTY0IDAwMDAwIG4gCjAwMDAzNDE4OTIgMDAwMDAgbiAKMDAw
MDM0MTczNCAwMDAwMCBuIAowMDAwMzQxODEyIDAwMDAwIG4gCjAwMDAyNzI1MDYgMDAwMDAgbiAK
MDAwMDE0MTA3NyAwMDAwMCBuIAowMDAwMTQyMjE4IDAwMDAwIG4gCjAwMDAzNDI1NjMgMDAwMDAg
biAKMDAwMDM0MjQzNyAwMDAwMCBuIAowMDAwMzQzNzQ2IDAwMDAwIG4gCjAwMDAzNDM2NDEgMDAw
MDAgbiAKMDAwMDM0MjgxMyAwMDAwMCBuIAowMDAwMzQyNjU3IDAwMDAwIG4gCjAwMDAzNDI3MzQg
MDAwMDAgbiAKMDAwMDM0MzA1OSAwMDAwMCBuIAowMDAwMzQyOTAzIDAwMDAwIG4gCjAwMDAzNDI5
ODAgMDAwMDAgbiAKMDAwMDM0MzMwNSAwMDAwMCBuIAowMDAwMzQzMTQ5IDAwMDAwIG4gCjAwMDAz
NDMyMjYgMDAwMDAgbiAKMDAwMDM0MzU1MSAwMDAwMCBuIAowMDAwMzQzMzk1IDAwMDAwIG4gCjAw
MDAzNDM0NzIgMDAwMDAgbiAKMDAwMDI3MjY0NCAwMDAwMCBuIAowMDAwMTQyMjQxIDAwMDAwIG4g
CjAwMDAxNDMyMzIgMDAwMDAgbiAKMDAwMDM0Mzk1NiAwMDAwMCBuIAowMDAwMzQzODMwIDAwMDAw
IG4gCjAwMDAzNDU3MzEgMDAwMDAgbiAKMDAwMDM0NTY0MiAwMDAwMCBuIAowMDAwMzQ1MzA0IDAw
MDAwIG4gCjAwMDAzNDQwNTAgMDAwMDAgbiAKMDAwMDM0NTIxNyAwMDAwMCBuIAowMDAwMzQ1MTEy
IDAwMDAwIG4gCjAwMDAzNDQyODMgMDAwMDAgbiAKMDAwMDM0NDEyNyAwMDAwMCBuIAowMDAwMzQ0
MjA0IDAwMDAwIG4gCjAwMDAzNDQ1MjkgMDAwMDAgbiAKMDAwMDM0NDM3MyAwMDAwMCBuIAowMDAw
MzQ0NDUwIDAwMDAwIG4gCjAwMDAzNDQ3NzUgMDAwMDAgbiAKMDAwMDM0NDYxOSAwMDAwMCBuIAow
MDAwMzQ0Njk2IDAwMDAwIG4gCjAwMDAzNDUwMjIgMDAwMDAgbiAKMDAwMDM0NDg2NSAwMDAwMCBu
IAowMDAwMzQ0OTQyIDAwMDAwIG4gCjAwMDAzNDU1NTIgMDAwMDAgbiAKMDAwMDM0NTM5NCAwMDAw
MCBuIAowMDAwMzQ1NDcyIDAwMDAwIG4gCjAwMDAyNzI3ODIgMDAwMDAgbiAKMDAwMDE0MzI1NCAw
MDAwMCBuIAowMDAwMTQ0NDQ0IDAwMDAwIG4gCjAwMDAzNDU5NDEgMDAwMDAgbiAKMDAwMDM0NTgx
NSAwMDAwMCBuIAowMDAwMzQ3NjM1IDAwMDAwIG4gCjAwMDAzNDc1MTQgMDAwMDAgbiAKMDAwMDM0
NjE5MSAwMDAwMCBuIAowMDAwMzQ2MDM1IDAwMDAwIG4gCjAwMDAzNDYxMTIgMDAwMDAgbiAKMDAw
MDM0NjQzNyAwMDAwMCBuIAowMDAwMzQ2MjgxIDAwMDAwIG4gCjAwMDAzNDYzNTggMDAwMDAgbiAK
MDAwMDM0NjY4MyAwMDAwMCBuIAowMDAwMzQ2NTI3IDAwMDAwIG4gCjAwMDAzNDY2MDQgMDAwMDAg
biAKMDAwMDM0NjkyOSAwMDAwMCBuIAowMDAwMzQ2NzczIDAwMDAwIG4gCjAwMDAzNDY4NTAgMDAw
MDAgbiAKMDAwMDM0NzE3NiAwMDAwMCBuIAowMDAwMzQ3MDE5IDAwMDAwIG4gCjAwMDAzNDcwOTYg
MDAwMDAgbiAKMDAwMDM0NzQyNCAwMDAwMCBuIAowMDAwMzQ3MjY2IDAwMDAwIG4gCjAwMDAzNDcz
NDQgMDAwMDAgbiAKMDAwMDI3MjkyMCAwMDAwMCBuIAowMDAwMTQ0NDY3IDAwMDAwIG4gCjAwMDAx
NDU2MjkgMDAwMDAgbiAKMDAwMDM0Nzg0NSAwMDAwMCBuIAowMDAwMzQ3NzE5IDAwMDAwIG4gCjAw
MDAzNDkwMjggMDAwMDAgbiAKMDAwMDM0ODkyMyAwMDAwMCBuIAowMDAwMzQ4MDk1IDAwMDAwIG4g
CjAwMDAzNDc5MzkgMDAwMDAgbiAKMDAwMDM0ODAxNiAwMDAwMCBuIAowMDAwMzQ4MzQxIDAwMDAw
IG4gCjAwMDAzNDgxODUgMDAwMDAgbiAKMDAwMDM0ODI2MiAwMDAwMCBuIAowMDAwMzQ4NTg3IDAw
MDAwIG4gCjAwMDAzNDg0MzEgMDAwMDAgbiAKMDAwMDM0ODUwOCAwMDAwMCBuIAowMDAwMzQ4ODMz
IDAwMDAwIG4gCjAwMDAzNDg2NzcgMDAwMDAgbiAKMDAwMDM0ODc1NCAwMDAwMCBuIAowMDAwMjcz
MDU4IDAwMDAwIG4gCjAwMDAxNDU2NTIgMDAwMDAgbiAKMDAwMDE0Njg4MyAwMDAwMCBuIAowMDAw
MzQ5MjM4IDAwMDAwIG4gCjAwMDAzNDkxMTIgMDAwMDAgbiAKMDAwMDM1MDY3NiAwMDAwMCBuIAow
MDAwMzUwNTYzIDAwMDAwIG4gCjAwMDAzNDk0ODggMDAwMDAgbiAKMDAwMDM0OTMzMiAwMDAwMCBu
IAowMDAwMzQ5NDA5IDAwMDAwIG4gCjAwMDAzNDk3MzQgMDAwMDAgbiAKMDAwMDM0OTU3OCAwMDAw
MCBuIAowMDAwMzQ5NjU1IDAwMDAwIG4gCjAwMDAzNDk5ODAgMDAwMDAgbiAKMDAwMDM0OTgyNCAw
MDAwMCBuIAowMDAwMzQ5OTAxIDAwMDAwIG4gCjAwMDAzNTAyMjYgMDAwMDAgbiAKMDAwMDM1MDA3
MCAwMDAwMCBuIAowMDAwMzUwMTQ3IDAwMDAwIG4gCjAwMDAzNTA0NzMgMDAwMDAgbiAKMDAwMDM1
MDMxNiAwMDAwMCBuIAowMDAwMzUwMzkzIDAwMDAwIG4gCjAwMDAyNzMxOTYgMDAwMDAgbiAKMDAw
MDE0NjkwNiAwMDAwMCBuIAowMDAwMTQ3ODQ5IDAwMDAwIG4gCjAwMDAzNTA4ODYgMDAwMDAgbiAK
MDAwMDM1MDc2MCAwMDAwMCBuIAowMDAwMzUyMDY5IDAwMDAwIG4gCjAwMDAzNTE5NjQgMDAwMDAg
biAKMDAwMDM1MTEzNiAwMDAwMCBuIAowMDAwMzUwOTgwIDAwMDAwIG4gCjAwMDAzNTEwNTcgMDAw
MDAgbiAKMDAwMDM1MTM4MiAwMDAwMCBuIAowMDAwMzUxMjI2IDAwMDAwIG4gCjAwMDAzNTEzMDMg
MDAwMDAgbiAKMDAwMDM1MTYyOCAwMDAwMCBuIAowMDAwMzUxNDcyIDAwMDAwIG4gCjAwMDAzNTE1
NDkgMDAwMDAgbiAKMDAwMDM1MTg3NCAwMDAwMCBuIAowMDAwMzUxNzE4IDAwMDAwIG4gCjAwMDAz
NTE3OTUgMDAwMDAgbiAKMDAwMDI3MzMzNCAwMDAwMCBuIAowMDAwMTQ3ODcxIDAwMDAwIG4gCjAw
MDAxNDg1ODUgMDAwMDAgbiAKMDAwMDM1MjI3OSAwMDAwMCBuIAowMDAwMzUyMTUzIDAwMDAwIG4g
CjAwMDAzNTM0NjIgMDAwMDAgbiAKMDAwMDM1MzM1NyAwMDAwMCBuIAowMDAwMzUyNTI5IDAwMDAw
IG4gCjAwMDAzNTIzNzMgMDAwMDAgbiAKMDAwMDM1MjQ1MCAwMDAwMCBuIAowMDAwMzUyNzc1IDAw
MDAwIG4gCjAwMDAzNTI2MTkgMDAwMDAgbiAKMDAwMDM1MjY5NiAwMDAwMCBuIAowMDAwMzUzMDIx
IDAwMDAwIG4gCjAwMDAzNTI4NjUgMDAwMDAgbiAKMDAwMDM1Mjk0MiAwMDAwMCBuIAowMDAwMzUz
MjY3IDAwMDAwIG4gCjAwMDAzNTMxMTEgMDAwMDAgbiAKMDAwMDM1MzE4OCAwMDAwMCBuIAowMDAw
MjczNDcyIDAwMDAwIG4gCjAwMDAxNDg2MDcgMDAwMDAgbiAKMDAwMDE0OTU5NSAwMDAwMCBuIAow
MDAwMzUzNjcyIDAwMDAwIG4gCjAwMDAzNTM1NDYgMDAwMDAgbiAKMDAwMDM1NTQ0NyAwMDAwMCBu
IAowMDAwMzU1MzUwIDAwMDAwIG4gCjAwMDAzNTQ3NjUgMDAwMDAgbiAKMDAwMDM1Mzc2NiAwMDAw
MCBuIAowMDAwMzU0Njc4IDAwMDAwIG4gCjAwMDAzNTQ1ODEgMDAwMDAgbiAKMDAwMDM1Mzk5OSAw
MDAwMCBuIAowMDAwMzUzODQzIDAwMDAwIG4gCjAwMDAzNTM5MjAgMDAwMDAgbiAKMDAwMDM1NDI0
NSAwMDAwMCBuIAowMDAwMzU0MDg5IDAwMDAwIG4gCjAwMDAzNTQxNjYgMDAwMDAgbiAKMDAwMDM1
NDQ5MSAwMDAwMCBuIAowMDAwMzU0MzM1IDAwMDAwIG4gCjAwMDAzNTQ0MTIgMDAwMDAgbiAKMDAw
MDM1NTAxMiAwMDAwMCBuIAowMDAwMzU0ODU1IDAwMDAwIG4gCjAwMDAzNTQ5MzIgMDAwMDAgbiAK
MDAwMDM1NTI2MCAwMDAwMCBuIAowMDAwMzU1MTAyIDAwMDAwIG4gCjAwMDAzNTUxODAgMDAwMDAg
biAKMDAwMDI3MzYxMCAwMDAwMCBuIAowMDAwMTQ5NjE3IDAwMDAwIG4gCjAwMDAxNTA1MjkgMDAw
MDAgbiAKMDAwMDM1NTY1NyAwMDAwMCBuIAowMDAwMzU1NTMxIDAwMDAwIG4gCjAwMDAzNTc2ODgg
MDAwMDAgbiAKMDAwMDM1NzU5OSAwMDAwMCBuIAowMDAwMzU3MjYxIDAwMDAwIG4gCjAwMDAzNTU3
NTEgMDAwMDAgbiAKMDAwMDM1NzE3NCAwMDAwMCBuIAowMDAwMzU3MDYxIDAwMDAwIG4gCjAwMDAz
NTU5ODQgMDAwMDAgbiAKMDAwMDM1NTgyOCAwMDAwMCBuIAowMDAwMzU1OTA1IDAwMDAwIG4gCjAw
MDAzNTYyMzAgMDAwMDAgbiAKMDAwMDM1NjA3NCAwMDAwMCBuIAowMDAwMzU2MTUxIDAwMDAwIG4g
CjAwMDAzNTY0NzYgMDAwMDAgbiAKMDAwMDM1NjMyMCAwMDAwMCBuIAowMDAwMzU2Mzk3IDAwMDAw
IG4gCjAwMDAzNTY3MjMgMDAwMDAgbiAKMDAwMDM1NjU2NiAwMDAwMCBuIAowMDAwMzU2NjQzIDAw
MDAwIG4gCjAwMDAzNTY5NzEgMDAwMDAgbiAKMDAwMDM1NjgxMyAwMDAwMCBuIAowMDAwMzU2ODkx
IDAwMDAwIG4gCjAwMDAzNTc1MDkgMDAwMDAgbiAKMDAwMDM1NzM1MSAwMDAwMCBuIAowMDAwMzU3
NDI5IDAwMDAwIG4gCjAwMDAyNzM3NDggMDAwMDAgbiAKMDAwMDE1MDU1MSAwMDAwMCBuIAowMDAw
MTUwODYzIDAwMDAwIG4gCjAwMDAzNTc4OTggMDAwMDAgbiAKMDAwMDM1Nzc3MiAwMDAwMCBuIAow
MDAwMjczODg2IDAwMDAwIG4gCjAwMDAxNTA4ODUgMDAwMDAgbiAKMDAwMDE1MTQ4MCAwMDAwMCBu
IAowMDAwMzU4MTE4IDAwMDAwIG4gCjAwMDAzNTc5OTIgMDAwMDAgbiAKMDAwMDM1OTU1NiAwMDAw
MCBuIAowMDAwMzU5NDQzIDAwMDAwIG4gCjAwMDAzNTgzNjggMDAwMDAgbiAKMDAwMDM1ODIxMiAw
MDAwMCBuIAowMDAwMzU4Mjg5IDAwMDAwIG4gCjAwMDAzNTg2MTQgMDAwMDAgbiAKMDAwMDM1ODQ1
OCAwMDAwMCBuIAowMDAwMzU4NTM1IDAwMDAwIG4gCjAwMDAzNTg4NjAgMDAwMDAgbiAKMDAwMDM1
ODcwNCAwMDAwMCBuIAowMDAwMzU4NzgxIDAwMDAwIG4gCjAwMDAzNTkxMDYgMDAwMDAgbiAKMDAw
MDM1ODk1MCAwMDAwMCBuIAowMDAwMzU5MDI3IDAwMDAwIG4gCjAwMDAzNTkzNTMgMDAwMDAgbiAK
MDAwMDM1OTE5NiAwMDAwMCBuIAowMDAwMzU5MjczIDAwMDAwIG4gCjAwMDAyNzQwMjQgMDAwMDAg
biAKMDAwMDE1MTUwMiAwMDAwMCBuIAowMDAwMTUyNDg1IDAwMDAwIG4gCjAwMDAzNTk3NjYgMDAw
MDAgbiAKMDAwMDM1OTY0MCAwMDAwMCBuIAowMDAwMzYxNzk3IDAwMDAwIG4gCjAwMDAzNjE3MDgg
MDAwMDAgbiAKMDAwMDM2MDAxNiAwMDAwMCBuIAowMDAwMzU5ODYwIDAwMDAwIG4gCjAwMDAzNTk5
MzcgMDAwMDAgbiAKMDAwMDM2MTYxOCAwMDAwMCBuIAowMDAwMzYwMTA2IDAwMDAwIG4gCjAwMDAz
NjE1MzEgMDAwMDAgbiAKMDAwMDM2MTQxOCAwMDAwMCBuIAowMDAwMzYwMzM5IDAwMDAwIG4gCjAw
MDAzNjAxODMgMDAwMDAgbiAKMDAwMDM2MDI2MCAwMDAwMCBuIAowMDAwMzYwNTg1IDAwMDAwIG4g
CjAwMDAzNjA0MjkgMDAwMDAgbiAKMDAwMDM2MDUwNiAwMDAwMCBuIAowMDAwMzYwODMyIDAwMDAw
IG4gCjAwMDAzNjA2NzUgMDAwMDAgbiAKMDAwMDM2MDc1MiAwMDAwMCBuIAowMDAwMzYxMDgwIDAw
MDAwIG4gCjAwMDAzNjA5MjIgMDAwMDAgbiAKMDAwMDM2MTAwMCAwMDAwMCBuIAowMDAwMzYxMzI4
IDAwMDAwIG4gCjAwMDAzNjExNzAgMDAwMDAgbiAKMDAwMDM2MTI0OCAwMDAwMCBuIAowMDAwMjc0
MTYyIDAwMDAwIG4gCjAwMDAxNTI1MDcgMDAwMDAgbiAKMDAwMDE1MzcxNSAwMDAwMCBuIAowMDAw
MzYyMDA3IDAwMDAwIG4gCjAwMDAzNjE4ODEgMDAwMDAgbiAKMDAwMDM2MzE5MCAwMDAwMCBuIAow
MDAwMzYzMDg1IDAwMDAwIG4gCjAwMDAzNjIyNTcgMDAwMDAgbiAKMDAwMDM2MjEwMSAwMDAwMCBu
IAowMDAwMzYyMTc4IDAwMDAwIG4gCjAwMDAzNjI1MDMgMDAwMDAgbiAKMDAwMDM2MjM0NyAwMDAw
MCBuIAowMDAwMzYyNDI0IDAwMDAwIG4gCjAwMDAzNjI3NDkgMDAwMDAgbiAKMDAwMDM2MjU5MyAw
MDAwMCBuIAowMDAwMzYyNjcwIDAwMDAwIG4gCjAwMDAzNjI5OTUgMDAwMDAgbiAKMDAwMDM2Mjgz
OSAwMDAwMCBuIAowMDAwMzYyOTE2IDAwMDAwIG4gCjAwMDAyNzQzMDAgMDAwMDAgbiAKMDAwMDE1
MzczOCAwMDAwMCBuIAowMDAwMTU0ODI4IDAwMDAwIG4gCjAwMDAzNjM0MDIgMDAwMDAgbiAKMDAw
MDM2MzI3NCAwMDAwMCBuIAowMDAwMzY0NjI1IDAwMDAwIG4gCjAwMDAzNjQ1MTQgMDAwMDAgbiAK
MDAwMDM2MzY1OCAwMDAwMCBuIAowMDAwMzYzNDk4IDAwMDAwIG4gCjAwMDAzNjM1NzcgMDAwMDAg
biAKMDAwMDM2MzkxMiAwMDAwMCBuIAowMDAwMzYzNzUyIDAwMDAwIG4gCjAwMDAzNjM4MzEgMDAw
MDAgbiAKMDAwMDM2NDE2NiAwMDAwMCBuIAowMDAwMzY0MDA2IDAwMDAwIG4gCjAwMDAzNjQwODUg
MDAwMDAgbiAKMDAwMDM2NDQyMCAwMDAwMCBuIAowMDAwMzY0MjYwIDAwMDAwIG4gCjAwMDAzNjQz
MzkgMDAwMDAgbiAKMDAwMDI3NDQzOCAwMDAwMCBuIAowMDAwMTU0ODUyIDAwMDAwIG4gCjAwMDAx
NTU4MTkgMDAwMDAgbiAKMDAwMDM2NDg0MCAwMDAwMCBuIAowMDAwMzY0NzExIDAwMDAwIG4gCjAw
MDAzNjYzNDQgMDAwMDAgbiAKMDAwMDM2NjIyMyAwMDAwMCBuIAowMDAwMzY1MDk5IDAwMDAwIG4g
CjAwMDAzNjQ5MzcgMDAwMDAgbiAKMDAwMDM2NTAxNyAwMDAwMCBuIAowMDAwMzY1MzU2IDAwMDAw
IG4gCjAwMDAzNjUxOTQgMDAwMDAgbiAKMDAwMDM2NTI3NCAwMDAwMCBuIAowMDAwMzY1NjEzIDAw
MDAwIG4gCjAwMDAzNjU0NTEgMDAwMDAgbiAKMDAwMDM2NTUzMSAwMDAwMCBuIAowMDAwMzY1ODcw
IDAwMDAwIG4gCjAwMDAzNjU3MDggMDAwMDAgbiAKMDAwMDM2NTc4OCAwMDAwMCBuIAowMDAwMzY2
MTI4IDAwMDAwIG4gCjAwMDAzNjU5NjUgMDAwMDAgbiAKMDAwMDM2NjA0NSAwMDAwMCBuIAowMDAw
Mjc0NTc4IDAwMDAwIG4gCjAwMDAxNTU4NDIgMDAwMDAgbiAKMDAwMDE1NjU1OSAwMDAwMCBuIAow
MDAwMzY2NTYwIDAwMDAwIG4gCjAwMDAzNjY0MzEgMDAwMDAgbiAKMDAwMDM2NzUzMSAwMDAwMCBu
IAowMDAwMzY3NDI4IDAwMDAwIG4gCjAwMDAzNjY4MTkgMDAwMDAgbiAKMDAwMDM2NjY1NyAwMDAw
MCBuIAowMDAwMzY2NzM3IDAwMDAwIG4gCjAwMDAzNjcwNzYgMDAwMDAgbiAKMDAwMDM2NjkxNCAw
MDAwMCBuIAowMDAwMzY2OTk0IDAwMDAwIG4gCjAwMDAzNjczMzMgMDAwMDAgbiAKMDAwMDM2NzE3
MSAwMDAwMCBuIAowMDAwMzY3MjUxIDAwMDAwIG4gCjAwMDAyNzQ3MTggMDAwMDAgbiAKMDAwMDE1
NjU4MiAwMDAwMCBuIAowMDAwMTU2ODY3IDAwMDAwIG4gCjAwMDAzNjc3NDcgMDAwMDAgbiAKMDAw
MDM2NzYxOCAwMDAwMCBuIAowMDAwMjc0ODU4IDAwMDAwIG4gCjAwMDAxNTY4OTAgMDAwMDAgbiAK
MDAwMDE1NzQ0MCAwMDAwMCBuIAowMDAwMzY3OTczIDAwMDAwIG4gCjAwMDAzNjc4NDQgMDAwMDAg
biAKMDAwMDM2OTIxMCAwMDAwMCBuIAowMDAwMzY5MDk4IDAwMDAwIG4gCjAwMDAzNjgyMzIgMDAw
MDAgbiAKMDAwMDM2ODA3MCAwMDAwMCBuIAowMDAwMzY4MTUwIDAwMDAwIG4gCjAwMDAzNjg0ODkg
MDAwMDAgbiAKMDAwMDM2ODMyNyAwMDAwMCBuIAowMDAwMzY4NDA3IDAwMDAwIG4gCjAwMDAzNjg3
NDYgMDAwMDAgbiAKMDAwMDM2ODU4NCAwMDAwMCBuIAowMDAwMzY4NjY0IDAwMDAwIG4gCjAwMDAz
NjkwMDMgMDAwMDAgbiAKMDAwMDM2ODg0MSAwMDAwMCBuIAowMDAwMzY4OTIxIDAwMDAwIG4gCjAw
MDAyNzQ5OTggMDAwMDAgbiAKMDAwMDE1NzQ2MyAwMDAwMCBuIAowMDAwMTU4NjQ2IDAwMDAwIG4g
CjAwMDAzNjk0MjYgMDAwMDAgbiAKMDAwMDM2OTI5NyAwMDAwMCBuIAowMDAwMzcxMTk4IDAwMDAw
IG4gCjAwMDAzNzEwNjggMDAwMDAgbiAKMDAwMDM2OTY4NSAwMDAwMCBuIAowMDAwMzY5NTIzIDAw
MDAwIG4gCjAwMDAzNjk2MDMgMDAwMDAgbiAKMDAwMDM2OTk0MiAwMDAwMCBuIAowMDAwMzY5Nzgw
IDAwMDAwIG4gCjAwMDAzNjk4NjAgMDAwMDAgbiAKMDAwMDM3MDE5OSAwMDAwMCBuIAowMDAwMzcw
MDM3IDAwMDAwIG4gCjAwMDAzNzAxMTcgMDAwMDAgbiAKMDAwMDM3MDQ1NiAwMDAwMCBuIAowMDAw
MzcwMjk0IDAwMDAwIG4gCjAwMDAzNzAzNzQgMDAwMDAgbiAKMDAwMDM3MDcxNCAwMDAwMCBuIAow
MDAwMzcwNTUxIDAwMDAwIG4gCjAwMDAzNzA2MzEgMDAwMDAgbiAKMDAwMDM3MDk3MyAwMDAwMCBu
IAowMDAwMzcwODA5IDAwMDAwIG4gCjAwMDAzNzA4OTAgMDAwMDAgbiAKMDAwMDI3NTEzOCAwMDAw
MCBuIAowMDAwMTU4NjcwIDAwMDAwIG4gCjAwMDAxNjAwMDEgMDAwMDAgbiAKMDAwMDM3MTQxNCAw
MDAwMCBuIAowMDAwMzcxMjg1IDAwMDAwIG4gCjAwMDAzNzM0NTQgMDAwMDAgbiAKMDAwMDM3MzMx
NSAwMDAwMCBuIAowMDAwMzcxNjczIDAwMDAwIG4gCjAwMDAzNzE1MTEgMDAwMDAgbiAKMDAwMDM3
MTU5MSAwMDAwMCBuIAowMDAwMzcxOTMwIDAwMDAwIG4gCjAwMDAzNzE3NjggMDAwMDAgbiAKMDAw
MDM3MTg0OCAwMDAwMCBuIAowMDAwMzcyMTg3IDAwMDAwIG4gCjAwMDAzNzIwMjUgMDAwMDAgbiAK
MDAwMDM3MjEwNSAwMDAwMCBuIAowMDAwMzcyNDQ0IDAwMDAwIG4gCjAwMDAzNzIyODIgMDAwMDAg
biAKMDAwMDM3MjM2MiAwMDAwMCBuIAowMDAwMzcyNzAyIDAwMDAwIG4gCjAwMDAzNzI1MzkgMDAw
MDAgbiAKMDAwMDM3MjYxOSAwMDAwMCBuIAowMDAwMzcyOTYxIDAwMDAwIG4gCjAwMDAzNzI3OTcg
MDAwMDAgbiAKMDAwMDM3Mjg3OCAwMDAwMCBuIAowMDAwMzczMjIwIDAwMDAwIG4gCjAwMDAzNzMw
NTYgMDAwMDAgbiAKMDAwMDM3MzEzNyAwMDAwMCBuIAowMDAwMjc1Mjc4IDAwMDAwIG4gCjAwMDAx
NjAwMjUgMDAwMDAgbiAKMDAwMDE2MTczOCAwMDAwMCBuIAowMDAwMzczNjcwIDAwMDAwIG4gCjAw
MDAzNzM1NDEgMDAwMDAgbiAKMDAwMDM3NDg3OSAwMDAwMCBuIAowMDAwMzczNzY3IDAwMDAwIG4g
CjAwMDAzNzM4NDUgMDAwMDAgbiAKMDAwMDM3MzkyMyAwMDAwMCBuIAowMDAwMzc0MDAxIDAwMDAw
IG4gCjAwMDAzNzQwNzkgMDAwMDAgbiAKMDAwMDM3NDE1OCAwMDAwMCBuIAowMDAwMzc0MjM3IDAw
MDAwIG4gCjAwMDAzNzQzMTYgMDAwMDAgbiAKMDAwMDM3NDM5NSAwMDAwMCBuIAowMDAwMzc0NDc0
IDAwMDAwIG4gCjAwMDAzNzQ1NTMgMDAwMDAgbiAKMDAwMDM3NDYzMiAwMDAwMCBuIAowMDAwMzc0
NzExIDAwMDAwIG4gCjAwMDAyNzU0MTggMDAwMDAgbiAKMDAwMDE2MTc2MiAwMDAwMCBuIAowMDAw
MTY2MTA0IDAwMDAwIG4gCjAwMDAzNzUyNTUgMDAwMDAgbiAKMDAwMDM3NTEyNiAwMDAwMCBuIAow
MDAwMzc3MzgwIDAwMDAwIG4gCjAwMDAzNzcyNjggMDAwMDAgbiAKMDAwMDM3NTUxNCAwMDAwMCBu
IAowMDAwMzc1MzUyIDAwMDAwIG4gCjAwMDAzNzU0MzIgMDAwMDAgbiAKMDAwMDM3NjY1NSAwMDAw
MCBuIAowMDAwMzc1NjA5IDAwMDAwIG4gCjAwMDAzNzY1NjQgMDAwMDAgbiAKMDAwMDM3NjQ2MSAw
MDAwMCBuIAowMDAwMzc1ODUxIDAwMDAwIG4gCjAwMDAzNzU2ODkgMDAwMDAgbiAKMDAwMDM3NTc2
OSAwMDAwMCBuIAowMDAwMzc2MTA4IDAwMDAwIG4gCjAwMDAzNzU5NDYgMDAwMDAgbiAKMDAwMDM3
NjAyNiAwMDAwMCBuIAowMDAwMzc2MzY2IDAwMDAwIG4gCjAwMDAzNzYyMDMgMDAwMDAgbiAKMDAw
MDM3NjI4MyAwMDAwMCBuIAowMDAwMzc2OTE0IDAwMDAwIG4gCjAwMDAzNzY3NTAgMDAwMDAgbiAK
MDAwMDM3NjgzMSAwMDAwMCBuIAowMDAwMzc3MTczIDAwMDAwIG4gCjAwMDAzNzcwMDkgMDAwMDAg
biAKMDAwMDM3NzA5MCAwMDAwMCBuIAowMDAwMzc4ODE4IDAwMDAwIG4gCjAwMDAzNzc0NjcgMDAw
MDAgbiAKMDAwMDM3NzU0NiAwMDAwMCBuIAowMDAwMzc3NjI1IDAwMDAwIG4gCjAwMDAzNzc3MDQg
MDAwMDAgbiAKMDAwMDM3Nzc4MyAwMDAwMCBuIAowMDAwMzc3ODYyIDAwMDAwIG4gCjAwMDAzNzc5
NDEgMDAwMDAgbiAKMDAwMDM3ODAyMCAwMDAwMCBuIAowMDAwMzc4MDk5IDAwMDAwIG4gCjAwMDAz
NzgxNzggMDAwMDAgbiAKMDAwMDM3ODI1NyAwMDAwMCBuIAowMDAwMzc4MzM2IDAwMDAwIG4gCjAw
MDAzNzg0MTUgMDAwMDAgbiAKMDAwMDM3ODQ5NCAwMDAwMCBuIAowMDAwMzc4NTczIDAwMDAwIG4g
CjAwMDAzNzg2NTIgMDAwMDAgbiAKMDAwMDI3NTU1OCAwMDAwMCBuIAowMDAwMTY2MTI4IDAwMDAw
IG4gCjAwMDAxNzE2MDEgMDAwMDAgbiAKMDAwMDM3OTIyNiAwMDAwMCBuIAowMDAwMzc5MDk3IDAw
MDAwIG4gCjAwMDAzODA5OTggMDAwMDAgbiAKMDAwMDM4MDg2OCAwMDAwMCBuIAowMDAwMzc5NDg1
IDAwMDAwIG4gCjAwMDAzNzkzMjMgMDAwMDAgbiAKMDAwMDM3OTQwMyAwMDAwMCBuIAowMDAwMzc5
NzQyIDAwMDAwIG4gCjAwMDAzNzk1ODAgMDAwMDAgbiAKMDAwMDM3OTY2MCAwMDAwMCBuIAowMDAw
Mzc5OTk5IDAwMDAwIG4gCjAwMDAzNzk4MzcgMDAwMDAgbiAKMDAwMDM3OTkxNyAwMDAwMCBuIAow
MDAwMzgwMjU2IDAwMDAwIG4gCjAwMDAzODAwOTQgMDAwMDAgbiAKMDAwMDM4MDE3NCAwMDAwMCBu
IAowMDAwMzgwNTE0IDAwMDAwIG4gCjAwMDAzODAzNTEgMDAwMDAgbiAKMDAwMDM4MDQzMSAwMDAw
MCBuIAowMDAwMzgwNzczIDAwMDAwIG4gCjAwMDAzODA2MDkgMDAwMDAgbiAKMDAwMDM4MDY5MCAw
MDAwMCBuIAowMDAwMzgyNDM4IDAwMDAwIG4gCjAwMDAzODEwODUgMDAwMDAgbiAKMDAwMDM4MTE2
NCAwMDAwMCBuIAowMDAwMzgxMjQzIDAwMDAwIG4gCjAwMDAzODEzMjIgMDAwMDAgbiAKMDAwMDM4
MTQwMSAwMDAwMCBuIAowMDAwMzgxNDgwIDAwMDAwIG4gCjAwMDAzODE1NTkgMDAwMDAgbiAKMDAw
MDM4MTYzOCAwMDAwMCBuIAowMDAwMzgxNzE3IDAwMDAwIG4gCjAwMDAzODE3OTYgMDAwMDAgbiAK
MDAwMDM4MTg3NSAwMDAwMCBuIAowMDAwMzgxOTU0IDAwMDAwIG4gCjAwMDAzODIwMzMgMDAwMDAg
biAKMDAwMDM4MjExMiAwMDAwMCBuIAowMDAwMzgyMTkxIDAwMDAwIG4gCjAwMDAzODIyNzAgMDAw
MDAgbiAKMDAwMDI3NTY5OCAwMDAwMCBuIAowMDAwMTcxNjI1IDAwMDAwIG4gCjAwMDAxNzMwMTcg
MDAwMDAgbiAKMDAwMDM4Mjg0NiAwMDAwMCBuIAowMDAwMzgyNzE3IDAwMDAwIG4gCjAwMDAzODQ5
NzEgMDAwMDAgbiAKMDAwMDM4NDg4NiAwMDAwMCBuIAowMDAwMzg0NzkxIDAwMDAwIG4gCjAwMDAz
ODI5NDMgMDAwMDAgbiAKMDAwMDM4NDcwMCAwMDAwMCBuIAowMDAwMzg0NTcwIDAwMDAwIG4gCjAw
MDAzODMxODUgMDAwMDAgbiAKMDAwMDM4MzAyMyAwMDAwMCBuIAowMDAwMzgzMTAzIDAwMDAwIG4g
CjAwMDAzODM0NDIgMDAwMDAgbiAKMDAwMDM4MzI4MCAwMDAwMCBuIAowMDAwMzgzMzYwIDAwMDAw
IG4gCjAwMDAzODM2OTkgMDAwMDAgbiAKMDAwMDM4MzUzNyAwMDAwMCBuIAowMDAwMzgzNjE3IDAw
MDAwIG4gCjAwMDAzODM5NTcgMDAwMDAgbiAKMDAwMDM4Mzc5NCAwMDAwMCBuIAowMDAwMzgzODc0
IDAwMDAwIG4gCjAwMDAzODQyMTYgMDAwMDAgbiAKMDAwMDM4NDA1MiAwMDAwMCBuIAowMDAwMzg0
MTMzIDAwMDAwIG4gCjAwMDAzODQ0NzUgMDAwMDAgbiAKMDAwMDM4NDMxMSAwMDAwMCBuIAowMDAw
Mzg0MzkyIDAwMDAwIG4gCjAwMDAzODY3NDIgMDAwMDAgbiAKMDAwMDM4NjY0NyAwMDAwMCBuIAow
MDAwMzg1MDU4IDAwMDAwIG4gCjAwMDAzODY1NTUgMDAwMDAgbiAKMDAwMDM4NjQzNCAwMDAwMCBu
IAowMDAwMzg1MzAzIDAwMDAwIG4gCjAwMDAzODUxMzkgMDAwMDAgbiAKMDAwMDM4NTIyMCAwMDAw
MCBuIAowMDAwMzg1NTYyIDAwMDAwIG4gCjAwMDAzODUzOTggMDAwMDAgbiAKMDAwMDM4NTQ3OSAw
MDAwMCBuIAowMDAwMzg1ODIxIDAwMDAwIG4gCjAwMDAzODU2NTcgMDAwMDAgbiAKMDAwMDM4NTcz
OCAwMDAwMCBuIAowMDAwMzg2MDgwIDAwMDAwIG4gCjAwMDAzODU5MTYgMDAwMDAgbiAKMDAwMDM4
NTk5NyAwMDAwMCBuIAowMDAwMzg2MzM5IDAwMDAwIG4gCjAwMDAzODYxNzUgMDAwMDAgbiAKMDAw
MDM4NjI1NiAwMDAwMCBuIAowMDAwMjc1ODM4IDAwMDAwIG4gCjAwMDAxNzMwNDEgMDAwMDAgbiAK
MDAwMDE3NDExOCAwMDAwMCBuIAowMDAwMzg2OTU4IDAwMDAwIG4gCjAwMDAzODY4MjkgMDAwMDAg
biAKMDAwMDM4ODE5NSAwMDAwMCBuIAowMDAwMzg4MDgzIDAwMDAwIG4gCjAwMDAzODcyMTcgMDAw
MDAgbiAKMDAwMDM4NzA1NSAwMDAwMCBuIAowMDAwMzg3MTM1IDAwMDAwIG4gCjAwMDAzODc0NzQg
MDAwMDAgbiAKMDAwMDM4NzMxMiAwMDAwMCBuIAowMDAwMzg3MzkyIDAwMDAwIG4gCjAwMDAzODc3
MzEgMDAwMDAgbiAKMDAwMDM4NzU2OSAwMDAwMCBuIAowMDAwMzg3NjQ5IDAwMDAwIG4gCjAwMDAz
ODc5ODggMDAwMDAgbiAKMDAwMDM4NzgyNiAwMDAwMCBuIAowMDAwMzg3OTA2IDAwMDAwIG4gCjAw
MDAyNzU5NzggMDAwMDAgbiAKMDAwMDE3NDE0MiAwMDAwMCBuIAowMDAwMTc0NDI4IDAwMDAwIG4g
CjAwMDAzODg0MTEgMDAwMDAgbiAKMDAwMDM4ODI4MiAwMDAwMCBuIAowMDAwMjc2MTE4IDAwMDAw
IG4gCjAwMDAxNzQ0NTEgMDAwMDAgbiAKMDAwMDE3NTEzMCAwMDAwMCBuIAowMDAwMzg4NjM3IDAw
MDAwIG4gCjAwMDAzODg1MDggMDAwMDAgbiAKMDAwMDM5MDQwOSAwMDAwMCBuIAowMDAwMzkwMjc5
IDAwMDAwIG4gCjAwMDAzODg4OTYgMDAwMDAgbiAKMDAwMDM4ODczNCAwMDAwMCBuIAowMDAwMzg4
ODE0IDAwMDAwIG4gCjAwMDAzODkxNTMgMDAwMDAgbiAKMDAwMDM4ODk5MSAwMDAwMCBuIAowMDAw
Mzg5MDcxIDAwMDAwIG4gCjAwMDAzODk0MTAgMDAwMDAgbiAKMDAwMDM4OTI0OCAwMDAwMCBuIAow
MDAwMzg5MzI4IDAwMDAwIG4gCjAwMDAzODk2NjcgMDAwMDAgbiAKMDAwMDM4OTUwNSAwMDAwMCBu
IAowMDAwMzg5NTg1IDAwMDAwIG4gCjAwMDAzODk5MjUgMDAwMDAgbiAKMDAwMDM4OTc2MiAwMDAw
MCBuIAowMDAwMzg5ODQyIDAwMDAwIG4gCjAwMDAzOTAxODQgMDAwMDAgbiAKMDAwMDM5MDAyMCAw
MDAwMCBuIAowMDAwMzkwMTAxIDAwMDAwIG4gCjAwMDAyNzYyNTggMDAwMDAgbiAKMDAwMDE3NTE1
MyAwMDAwMCBuIAowMDAwMTc2MjU5IDAwMDAwIG4gCjAwMDAzOTA2MjUgMDAwMDAgbiAKMDAwMDM5
MDQ5NiAwMDAwMCBuIAowMDAwMzkyNzUwIDAwMDAwIG4gCjAwMDAzOTI2MzggMDAwMDAgbiAKMDAw
MDM5MTc2NyAwMDAwMCBuIAowMDAwMzkwNzIyIDAwMDAwIG4gCjAwMDAzOTE2NzYgMDAwMDAgbiAK
MDAwMDM5MTU3MyAwMDAwMCBuIAowMDAwMzkwOTY0IDAwMDAwIG4gCjAwMDAzOTA4MDIgMDAwMDAg
biAKMDAwMDM5MDg4MiAwMDAwMCBuIAowMDAwMzkxMjIxIDAwMDAwIG4gCjAwMDAzOTEwNTkgMDAw
MDAgbiAKMDAwMDM5MTEzOSAwMDAwMCBuIAowMDAwMzkxNDc4IDAwMDAwIG4gCjAwMDAzOTEzMTYg
MDAwMDAgbiAKMDAwMDM5MTM5NiAwMDAwMCBuIAowMDAwMzkyMDI1IDAwMDAwIG4gCjAwMDAzOTE4
NjIgMDAwMDAgbiAKMDAwMDM5MTk0MiAwMDAwMCBuIAowMDAwMzkyMjg0IDAwMDAwIG4gCjAwMDAz
OTIxMjAgMDAwMDAgbiAKMDAwMDM5MjIwMSAwMDAwMCBuIAowMDAwMzkyNTQzIDAwMDAwIG4gCjAw
MDAzOTIzNzkgMDAwMDAgbiAKMDAwMDM5MjQ2MCAwMDAwMCBuIAowMDAwMjc2Mzk4IDAwMDAwIG4g
CjAwMDAxNzYyODMgMDAwMDAgbiAKMDAwMDE3NzA5OSAwMDAwMCBuIAowMDAwMzkyOTY2IDAwMDAw
IG4gCjAwMDAzOTI4MzcgMDAwMDAgbiAKMDAwMDM5NDIwMyAwMDAwMCBuIAowMDAwMzk0MDkxIDAw
MDAwIG4gCjAwMDAzOTMyMjUgMDAwMDAgbiAKMDAwMDM5MzA2MyAwMDAwMCBuIAowMDAwMzkzMTQz
IDAwMDAwIG4gCjAwMDAzOTM0ODIgMDAwMDAgbiAKMDAwMDM5MzMyMCAwMDAwMCBuIAowMDAwMzkz
NDAwIDAwMDAwIG4gCjAwMDAzOTM3MzkgMDAwMDAgbiAKMDAwMDM5MzU3NyAwMDAwMCBuIAowMDAw
MzkzNjU3IDAwMDAwIG4gCjAwMDAzOTM5OTYgMDAwMDAgbiAKMDAwMDM5MzgzNCAwMDAwMCBuIAow
MDAwMzkzOTE0IDAwMDAwIG4gCjAwMDAyNzY1MzggMDAwMDAgbiAKMDAwMDE3NzEyMiAwMDAwMCBu
IAowMDAwMTc4MjI2IDAwMDAwIG4gCjAwMDAzOTQ0MTkgMDAwMDAgbiAKMDAwMDM5NDI5MCAwMDAw
MCBuIAowMDAwMzk2Mjc2IDAwMDAwIG4gCjAwMDAzOTYxNzMgMDAwMDAgbiAKMDAwMDM5NDY3OCAw
MDAwMCBuIAowMDAwMzk0NTE2IDAwMDAwIG4gCjAwMDAzOTQ1OTYgMDAwMDAgbiAKMDAwMDM5NTgx
OSAwMDAwMCBuIAowMDAwMzk0NzczIDAwMDAwIG4gCjAwMDAzOTU3MjggMDAwMDAgbiAKMDAwMDM5
NTYyNSAwMDAwMCBuIAowMDAwMzk1MDE1IDAwMDAwIG4gCjAwMDAzOTQ4NTMgMDAwMDAgbiAKMDAw
MDM5NDkzMyAwMDAwMCBuIAowMDAwMzk1MjcyIDAwMDAwIG4gCjAwMDAzOTUxMTAgMDAwMDAgbiAK
MDAwMDM5NTE5MCAwMDAwMCBuIAowMDAwMzk1NTMwIDAwMDAwIG4gCjAwMDAzOTUzNjcgMDAwMDAg
biAKMDAwMDM5NTQ0NyAwMDAwMCBuIAowMDAwMzk2MDc4IDAwMDAwIG4gCjAwMDAzOTU5MTQgMDAw
MDAgbiAKMDAwMDM5NTk5NSAwMDAwMCBuIAowMDAwMjc2Njc4IDAwMDAwIG4gCjAwMDAxNzgyNTAg
MDAwMDAgbiAKMDAwMDE3OTIwOSAwMDAwMCBuIAowMDAwMzk2NDkyIDAwMDAwIG4gCjAwMDAzOTYz
NjMgMDAwMDAgbiAKMDAwMDM5NzcyOSAwMDAwMCBuIAowMDAwMzk3NjE3IDAwMDAwIG4gCjAwMDAz
OTY3NTEgMDAwMDAgbiAKMDAwMDM5NjU4OSAwMDAwMCBuIAowMDAwMzk2NjY5IDAwMDAwIG4gCjAw
MDAzOTcwMDggMDAwMDAgbiAKMDAwMDM5Njg0NiAwMDAwMCBuIAowMDAwMzk2OTI2IDAwMDAwIG4g
CjAwMDAzOTcyNjUgMDAwMDAgbiAKMDAwMDM5NzEwMyAwMDAwMCBuIAowMDAwMzk3MTgzIDAwMDAw
IG4gCjAwMDAzOTc1MjIgMDAwMDAgbiAKMDAwMDM5NzM2MCAwMDAwMCBuIAowMDAwMzk3NDQwIDAw
MDAwIG4gCjAwMDAyNzY4MTggMDAwMDAgbiAKMDAwMDE3OTIzMiAwMDAwMCBuIAowMDAwMTgwMDY3
IDAwMDAwIG4gCjAwMDAzOTc5NDUgMDAwMDAgbiAKMDAwMDM5NzgxNiAwMDAwMCBuIAowMDAwMzk4
OTE2IDAwMDAwIG4gCjAwMDAzOTg4MTMgMDAwMDAgbiAKMDAwMDM5ODIwNCAwMDAwMCBuIAowMDAw
Mzk4MDQyIDAwMDAwIG4gCjAwMDAzOTgxMjIgMDAwMDAgbiAKMDAwMDM5ODQ2MSAwMDAwMCBuIAow
MDAwMzk4Mjk5IDAwMDAwIG4gCjAwMDAzOTgzNzkgMDAwMDAgbiAKMDAwMDM5ODcxOCAwMDAwMCBu
IAowMDAwMzk4NTU2IDAwMDAwIG4gCjAwMDAzOTg2MzYgMDAwMDAgbiAKMDAwMDI3Njk1OCAwMDAw
MCBuIAowMDAwMTgwMDkwIDAwMDAwIG4gCjAwMDAxODEwNTAgMDAwMDAgbiAKMDAwMDM5OTEzMiAw
MDAwMCBuIAowMDAwMzk5MDAzIDAwMDAwIG4gCjAwMDA0MDAzNjkgMDAwMDAgbiAKMDAwMDQwMDI1
NyAwMDAwMCBuIAowMDAwMzk5MzkxIDAwMDAwIG4gCjAwMDAzOTkyMjkgMDAwMDAgbiAKMDAwMDM5
OTMwOSAwMDAwMCBuIAowMDAwMzk5NjQ4IDAwMDAwIG4gCjAwMDAzOTk0ODYgMDAwMDAgbiAKMDAw
MDM5OTU2NiAwMDAwMCBuIAowMDAwMzk5OTA1IDAwMDAwIG4gCjAwMDAzOTk3NDMgMDAwMDAgbiAK
MDAwMDM5OTgyMyAwMDAwMCBuIAowMDAwNDAwMTYyIDAwMDAwIG4gCjAwMDA0MDAwMDAgMDAwMDAg
biAKMDAwMDQwMDA4MCAwMDAwMCBuIAowMDAwMjc3MDk4IDAwMDAwIG4gCjAwMDAxODEwNzMgMDAw
MDAgbiAKMDAwMDE4MTM1NyAwMDAwMCBuIAowMDAwNDAwNTg1IDAwMDAwIG4gCjAwMDA0MDA0NTYg
MDAwMDAgbiAKMDAwMDI3NzIzOCAwMDAwMCBuIAowMDAwMTgxMzgwIDAwMDAwIG4gCjAwMDAxODE4
MjMgMDAwMDAgbiAKMDAwMDQwMDgxMSAwMDAwMCBuIAowMDAwNDAwNjgyIDAwMDAwIG4gCjAwMDA0
MDE3ODIgMDAwMDAgbiAKMDAwMDQwMTY3OSAwMDAwMCBuIAowMDAwNDAxMDcwIDAwMDAwIG4gCjAw
MDA0MDA5MDggMDAwMDAgbiAKMDAwMDQwMDk4OCAwMDAwMCBuIAowMDAwNDAxMzI3IDAwMDAwIG4g
CjAwMDA0MDExNjUgMDAwMDAgbiAKMDAwMDQwMTI0NSAwMDAwMCBuIAowMDAwNDAxNTg0IDAwMDAw
IG4gCjAwMDA0MDE0MjIgMDAwMDAgbiAKMDAwMDQwMTUwMiAwMDAwMCBuIAowMDAwMjc3Mzc4IDAw
MDAwIG4gCjAwMDAxODE4NDYgMDAwMDAgbiAKMDAwMDE4MjE0MiAwMDAwMCBuIAowMDAwNDAxOTk4
IDAwMDAwIG4gCjAwMDA0MDE4NjkgMDAwMDAgbiAKMDAwMDI3NzUxOCAwMDAwMCBuIAowMDAwMTgy
MTY1IDAwMDAwIG4gCjAwMDAxODI2MTUgMDAwMDAgbiAKMDAwMDQwMjIyNCAwMDAwMCBuIAowMDAw
NDAyMDk1IDAwMDAwIG4gCjAwMDA0MDI5MjkgMDAwMDAgbiAKMDAwMDQwMjgzNSAwMDAwMCBuIAow
MDAwNDAyNDgzIDAwMDAwIG4gCjAwMDA0MDIzMjEgMDAwMDAgbiAKMDAwMDQwMjQwMSAwMDAwMCBu
IAowMDAwNDAyNzQwIDAwMDAwIG4gCjAwMDA0MDI1NzggMDAwMDAgbiAKMDAwMDQwMjY1OCAwMDAw
MCBuIAowMDAwMjc3NjU4IDAwMDAwIG4gCjAwMDAxODI2MzggMDAwMDAgbiAKMDAwMDE4MzM1NCAw
MDAwMCBuIAowMDAwNDAzMTQ1IDAwMDAwIG4gCjAwMDA0MDMwMTYgMDAwMDAgbiAKMDAwMDQwNDM4
MiAwMDAwMCBuIAowMDAwNDA0MjcwIDAwMDAwIG4gCjAwMDA0MDM0MDQgMDAwMDAgbiAKMDAwMDQw
MzI0MiAwMDAwMCBuIAowMDAwNDAzMzIyIDAwMDAwIG4gCjAwMDA0MDM2NjEgMDAwMDAgbiAKMDAw
MDQwMzQ5OSAwMDAwMCBuIAowMDAwNDAzNTc5IDAwMDAwIG4gCjAwMDA0MDM5MTggMDAwMDAgbiAK
MDAwMDQwMzc1NiAwMDAwMCBuIAowMDAwNDAzODM2IDAwMDAwIG4gCjAwMDA0MDQxNzUgMDAwMDAg
biAKMDAwMDQwNDAxMyAwMDAwMCBuIAowMDAwNDA0MDkzIDAwMDAwIG4gCjAwMDAyNzc4MzUgMDAw
MDAgbiAKMDAwMDE4MzM3NyAwMDAwMCBuIAowMDAwMTg1OTQwIDAwMDAwIG4gCjAwMDA0MDQ1OTgg
MDAwMDAgbiAKMDAwMDQwNDQ2OSAwMDAwMCBuIAowMDAwNDA3MDQ4IDAwMDAwIG4gCjAwMDA0MDQ2
OTUgMDAwMDAgbiAKMDAwMDQwNDc3MyAwMDAwMCBuIAowMDAwNDA0ODUxIDAwMDAwIG4gCjAwMDA0
MDQ5MjkgMDAwMDAgbiAKMDAwMDQwNTAwNyAwMDAwMCBuIAowMDAwNDA1MDg1IDAwMDAwIG4gCjAw
MDA0MDUxNjMgMDAwMDAgbiAKMDAwMDQwNTI0MSAwMDAwMCBuIAowMDAwNDA1MzE5IDAwMDAwIG4g
CjAwMDA0MDUzOTcgMDAwMDAgbiAKMDAwMDQwNTQ3NiAwMDAwMCBuIAowMDAwNDA1NTU1IDAwMDAw
IG4gCjAwMDA0MDU2MzQgMDAwMDAgbiAKMDAwMDQwNTcxMyAwMDAwMCBuIAowMDAwNDA1NzkyIDAw
MDAwIG4gCjAwMDA0MDU4NzEgMDAwMDAgbiAKMDAwMDQwNTk1MCAwMDAwMCBuIAowMDAwNDA2MDI5
IDAwMDAwIG4gCjAwMDA0MDYxMDggMDAwMDAgbiAKMDAwMDQwNjE4NyAwMDAwMCBuIAowMDAwNDA2
MjY2IDAwMDAwIG4gCjAwMDA0MDYzNDUgMDAwMDAgbiAKMDAwMDQwNjQyNCAwMDAwMCBuIAowMDAw
NDA2NTAzIDAwMDAwIG4gCjAwMDA0MDY1ODIgMDAwMDAgbiAKMDAwMDQwNjY2MSAwMDAwMCBuIAow
MDAwNDA2NzQwIDAwMDAwIG4gCjAwMDA0MDY4MTkgMDAwMDAgbiAKMDAwMDQwNjg5OCAwMDAwMCBu
IAowMDAwNDA2OTc3IDAwMDAwIG4gCjAwMDA0MDc1NjEgMDAwMDAgbiAKMDAwMDQwNzQ3NiAwMDAw
MCBuIAowMDAwNDA3Mzk2IDAwMDAwIG4gCjAwMDA0MTc1NjYgMDAwMDAgbiAKMDAwMDQxNzc5MiAw
MDAwMCBuIAowMDAwNDE4MDEzIDAwMDAwIG4gCjAwMDA0MTgyMzYgMDAwMDAgbiAKMDAwMDQxODQ3
OSAwMDAwMCBuIAowMDAwNDE4ODM0IDAwMDAwIG4gCjAwMDA0MTkyMDggMDAwMDAgbiAKMDAwMDQx
OTYwNiAwMDAwMCBuIAowMDAwNDIwMTkxIDAwMDAwIG4gCjAwMDA0MjA4NzEgMDAwMDAgbiAKMDAw
MDQyMTM1OCAwMDAwMCBuIAowMDAwNDIyMTYxIDAwMDAwIG4gCjAwMDA0MTY4NjcgMDAwMDAgbiAK
MDAwMDE4NTk2NCAwMDAwMCBuIAowMDAwMjA2OTEwIDAwMDAwIG4gCjAwMDAyMDY5MzUgMDAwMDAg
biAKMDAwMDIwNzEzNSAwMDAwMCBuIAowMDAwMjA3ODAyIDAwMDAwIG4gCjAwMDAyMDgzMTMgMDAw
MDAgbiAKMDAwMDIxNjA3MyAwMDAwMCBuIAowMDAwMjE2MDk3IDAwMDAwIG4gCjAwMDAyMTYzMDAg
MDAwMDAgbiAKMDAwMDIxNjcwNSAwMDAwMCBuIAowMDAwMjE2OTY4IDAwMDAwIG4gCjAwMDAyMjA5
MTEgMDAwMDAgbiAKMDAwMDIyMDkzNSAwMDAwMCBuIAowMDAwMjIxMTI2IDAwMDAwIG4gCjAwMDAy
MjE2MzYgMDAwMDAgbiAKMDAwMDIyMTk5MSAwMDAwMCBuIAowMDAwMjMxNzQ3IDAwMDAwIG4gCjAw
MDAyMzE3NzEgMDAwMDAgbiAKMDAwMDIzMTk3OCAwMDAwMCBuIAowMDAwMjMyNDUyIDAwMDAwIG4g
CjAwMDAyMzI3ODcgMDAwMDAgbiAKMDAwMDI0NzcwOCAwMDAwMCBuIAowMDAwMjQ3NzMzIDAwMDAw
IG4gCjAwMDAyNDc5NDUgMDAwMDAgbiAKMDAwMDI0ODQ1MiAwMDAwMCBuIAowMDAwMjQ4ODE3IDAw
MDAwIG4gCjAwMDAyNjM4NDUgMDAwMDAgbiAKMDAwMDI2Mzg3MCAwMDAwMCBuIAowMDAwMjY0MDY4
IDAwMDAwIG4gCjAwMDAyNjQ2NzMgMDAwMDAgbiAKMDAwMDI2NTEyNyAwMDAwMCBuIAowMDAwMjY3
MTM5IDAwMDAwIG4gCjAwMDAyNjcxNjMgMDAwMDAgbiAKMDAwMDI2NzM1OSAwMDAwMCBuIAowMDAw
MjY3NjYxIDAwMDAwIG4gCjAwMDAyNjc4MzAgMDAwMDAgbiAKMDAwMDI2NzkzOSAwMDAwMCBuIAow
MDAwMjc4MDQ4IDAwMDAwIG4gCjAwMDAyNzgxMTEgMDAwMDAgbiAKMDAwMDI3ODI0MSAwMDAwMCBu
IAowMDAwMjc4Mzg2IDAwMDAwIG4gCjAwMDAyNzg1MzEgMDAwMDAgbiAKMDAwMDI3ODY3NiAwMDAw
MCBuIAowMDAwMjc4ODIxIDAwMDAwIG4gCjAwMDAyNzg5NjcgMDAwMDAgbiAKMDAwMDI3OTExMyAw
MDAwMCBuIAowMDAwMjc5MjU5IDAwMDAwIG4gCjAwMDAyNzk0MDUgMDAwMDAgbiAKMDAwMDI3OTU1
NSAwMDAwMCBuIAowMDAwMjc5NzA1IDAwMDAwIG4gCjAwMDAyNzk4NTUgMDAwMDAgbiAKMDAwMDI4
MDAwNSAwMDAwMCBuIAowMDAwMjgwMTU1IDAwMDAwIG4gCjAwMDAyODAzMDUgMDAwMDAgbiAKMDAw
MDI4MDQ1NSAwMDAwMCBuIAowMDAwMjgwNjA1IDAwMDAwIG4gCjAwMDAyODA3NTUgMDAwMDAgbiAK
MDAwMDI4MDkwNSAwMDAwMCBuIAowMDAwMjgxMDU1IDAwMDAwIG4gCjAwMDAyODEyMDUgMDAwMDAg
biAKMDAwMDI4MTM1NSAwMDAwMCBuIAowMDAwMjgxNTA1IDAwMDAwIG4gCjAwMDAyODE2NTUgMDAw
MDAgbiAKMDAwMDI4MTgwNSAwMDAwMCBuIAowMDAwMjgxOTU1IDAwMDAwIG4gCjAwMDAyODIxMDUg
MDAwMDAgbiAKMDAwMDI4MjI1NSAwMDAwMCBuIAowMDAwMjgyNDA1IDAwMDAwIG4gCjAwMDAyODI1
NTUgMDAwMDAgbiAKMDAwMDI4MjcwNSAwMDAwMCBuIAowMDAwMjgyODU1IDAwMDAwIG4gCjAwMDAy
ODMwMDUgMDAwMDAgbiAKMDAwMDI4MzE1NSAwMDAwMCBuIAowMDAwMjgzMzA1IDAwMDAwIG4gCjAw
MDAyODM0NTUgMDAwMDAgbiAKMDAwMDI4MzYwNSAwMDAwMCBuIAowMDAwMjgzNzU1IDAwMDAwIG4g
CjAwMDAyODM5MDUgMDAwMDAgbiAKMDAwMDI4NDA1NSAwMDAwMCBuIAowMDAwMjg0MjA1IDAwMDAw
IG4gCjAwMDAyODQzNTUgMDAwMDAgbiAKMDAwMDI4NDUwNSAwMDAwMCBuIAowMDAwMjg0NjU1IDAw
MDAwIG4gCjAwMDAyODQ4MDUgMDAwMDAgbiAKMDAwMDI4NDk1NSAwMDAwMCBuIAowMDAwMjg1MTA2
IDAwMDAwIG4gCjAwMDAyODUyNTcgMDAwMDAgbiAKMDAwMDI4NTQwOCAwMDAwMCBuIAowMDAwMjg1
NTU5IDAwMDAwIG4gCjAwMDAyODU3MTAgMDAwMDAgbiAKMDAwMDI4NTg2MSAwMDAwMCBuIAowMDAw
Mjg2MDEyIDAwMDAwIG4gCjAwMDAyODYxNjMgMDAwMDAgbiAKMDAwMDI4NjMxNCAwMDAwMCBuIAow
MDAwMjg2NDY1IDAwMDAwIG4gCjAwMDAyODY2MTYgMDAwMDAgbiAKMDAwMDI4Njc2NyAwMDAwMCBu
IAowMDAwMjg2OTE4IDAwMDAwIG4gCjAwMDAyODcwNjkgMDAwMDAgbiAKMDAwMDI4NzIyMCAwMDAw
MCBuIAowMDAwMjg3MzcxIDAwMDAwIG4gCjAwMDAyODc1MjIgMDAwMDAgbiAKMDAwMDI4NzY3MyAw
MDAwMCBuIAowMDAwMjg3ODI0IDAwMDAwIG4gCjAwMDAyODc5NzUgMDAwMDAgbiAKMDAwMDI4ODEy
NiAwMDAwMCBuIAowMDAwMjg4Mjc3IDAwMDAwIG4gCjAwMDAyODg0MjggMDAwMDAgbiAKMDAwMDI4
ODU3OSAwMDAwMCBuIAowMDAwNDA3NjQ4IDAwMDAwIG4gCjAwMDAyODg3ODUgMDAwMDAgbiAKMDAw
MDI4OTU3MyAwMDAwMCBuIAowMDAwMjkyMTI1IDAwMDAwIG4gCjAwMDAyOTIzMzkgMDAwMDAgbiAK
MDAwMDI5MzY3MiAwMDAwMCBuIAowMDAwMjk2MDcyIDAwMDAwIG4gCjAwMDAyOTg2NTAgMDAwMDAg
biAKMDAwMDMwMDgxMCAwMDAwMCBuIAowMDAwMzAyNDU4IDAwMDAwIG4gCjAwMDAzMDM1NzkgMDAw
MDAgbiAKMDAwMDMwMzk2MiAwMDAwMCBuIAowMDAwMzA1MzU1IDAwMDAwIG4gCjAwMDAzMDU1NzUg
MDAwMDAgbiAKMDAwMDMwNzk5MSAwMDAwMCBuIAowMDAwMzEwNDA3IDAwMDAwIG4gCjAwMDAzMTM1
NzggMDAwMDAgbiAKMDAwMDMxNjA3NSAwMDAwMCBuIAowMDAwMzE3MjE0IDAwMDAwIG4gCjAwMDAz
MTk3MTEgMDAwMDAgbiAKMDAwMDMyMTYxNSAwMDAwMCBuIAowMDAwMzI0NTUxIDAwMDAwIG4gCjAw
MDAzMjY0NTUgMDAwMDAgbiAKMDAwMDMyNjYwMCAwMDAwMCBuIAowMDAwMzI3Nzc0IDAwMDAwIG4g
CjAwMDAzMjkxNjcgMDAwMDAgbiAKMDAwMDMyOTM4NyAwMDAwMCBuIAowMDAwMzMwNzgwIDAwMDAw
IG4gCjAwMDAzMzE5MDEgMDAwMDAgbiAKMDAwMDMzMjI4NCAwMDAwMCBuIAowMDAwMzMzNDg5IDAw
MDAwIG4gCjAwMDAzMzM4ODYgMDAwMDAgbiAKMDAwMDMzNTI3OSAwMDAwMCBuIAowMDAwMzM2NDE4
IDAwMDAwIG4gCjAwMDAzMzY2MzggMDAwMDAgbiAKMDAwMDMzODU0MiAwMDAwMCBuIAowMDAwMzQw
NTI3IDAwMDAwIG4gCjAwMDAzNDI1MTIgMDAwMDAgbiAKMDAwMDM0MzkwNSAwMDAwMCBuIAowMDAw
MzQ1ODkwIDAwMDAwIG4gCjAwMDAzNDc3OTQgMDAwMDAgbiAKMDAwMDM0OTE4NyAwMDAwMCBuIAow
MDAwMzUwODM1IDAwMDAwIG4gCjAwMDAzNTIyMjggMDAwMDAgbiAKMDAwMDM1MzYyMSAwMDAwMCBu
IAowMDAwMzU1NjA2IDAwMDAwIG4gCjAwMDAzNTc4NDcgMDAwMDAgbiAKMDAwMDM1ODA2NyAwMDAw
MCBuIAowMDAwMzU5NzE1IDAwMDAwIG4gCjAwMDAzNjE5NTYgMDAwMDAgbiAKMDAwMDM2MzM1MSAw
MDAwMCBuIAowMDAwMzY0Nzg5IDAwMDAwIG4gCjAwMDAzNjY1MDkgMDAwMDAgbiAKMDAwMDM2NzY5
NiAwMDAwMCBuIAowMDAwMzY3OTIyIDAwMDAwIG4gCjAwMDAzNjkzNzUgMDAwMDAgbiAKMDAwMDM3
MTM2MyAwMDAwMCBuIAowMDAwMzczNjE5IDAwMDAwIG4gCjAwMDAzNzQ3OTAgMDAwMDAgbiAKMDAw
MDM3NTIwNCAwMDAwMCBuIAowMDAwMzc4NzMxIDAwMDAwIG4gCjAwMDAzNzkxNzUgMDAwMDAgbiAK
MDAwMDM4MjM0OSAwMDAwMCBuIAowMDAwMzgyNzk1IDAwMDAwIG4gCjAwMDAzODY5MDcgMDAwMDAg
biAKMDAwMDM4ODM2MCAwMDAwMCBuIAowMDAwMzg4NTg2IDAwMDAwIG4gCjAwMDAzOTA1NzQgMDAw
MDAgbiAKMDAwMDM5MjkxNSAwMDAwMCBuIAowMDAwMzk0MzY4IDAwMDAwIG4gCjAwMDAzOTY0NDEg
MDAwMDAgbiAKMDAwMDM5Nzg5NCAwMDAwMCBuIAowMDAwMzk5MDgxIDAwMDAwIG4gCjAwMDA0MDA1
MzQgMDAwMDAgbiAKMDAwMDQwMDc2MCAwMDAwMCBuIAowMDAwNDAxOTQ3IDAwMDAwIG4gCjAwMDA0
MDIxNzMgMDAwMDAgbiAKMDAwMDQwMzA5NCAwMDAwMCBuIAowMDAwNDA0NTQ3IDAwMDAwIG4gCjAw
MDA0MDg4OTUgMDAwMDAgbiAKMDAwMDQyMjY4OCAwMDAwMCBuIAowMDAwNDIyODgyIDAwMDAwIG4g
CnRyYWlsZXIKPDwvU2l6ZSAxNzY2L1Jvb3QgMTc2NCAwIFIKL0luZm8gMTc2NSAwIFIKL0lEIFsg
PDAxNENFNDk1MkI4QzM2OThFQ0UzNzRENDgxQjBBRkI0Pgo8MDE0Q0U0OTUyQjhDMzY5OEVDRTM3
NEQ0ODFCMEFGQjQ+IF0KL0RvY0NoZWNrc3VtIC9GRDg4MDRCNEZFQkRBMzA5NzQ5RkZBM0Q2QUJE
MTgyRAo+PgpzdGFydHhyZWYKNDIzMDYzCiUlRU9GCg==
--0000000000002c93af061816f0de--


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:29:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:29:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719702.1122551 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NVF-0000L6-Cl; Fri, 10 May 2024 10:29:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719702.1122551; Fri, 10 May 2024 10:29:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NVF-0000JA-6t; Fri, 10 May 2024 10:29:25 +0000
Received: by outflank-mailman (input) for mailman id 719702;
 Fri, 10 May 2024 10:29:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IhR9=MN=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s5NVE-0000AU-8Z
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:29:24 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 29355476-0eb8-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:29:20 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59a64db066so471395366b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:29:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29355476-0eb8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715336960; x=1715941760; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:in-reply-to:references:mime-version
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DCce+NW2yUN6XuQdr+dHXWXXVx5hWNLro/e1lSco0rA=;
        b=kthvIuOkuGTbiHRYAe0avT11p6bVmNbMTb6s95QQmC8bEu47dLh7aGRhyFMw/SRB2M
         4xQw/Tlf9Nq0p+wWsudCxmiqNV/06UL5FdLlnAGo13iHJ3hBgj0b5uRU26aHH47f+7aX
         tIjbm54pK+Pj2K77Qgrb/ICgTMBsNXLGylqj4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715336960; x=1715941760;
        h=to:subject:message-id:date:from:in-reply-to:references:mime-version
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DCce+NW2yUN6XuQdr+dHXWXXVx5hWNLro/e1lSco0rA=;
        b=tryLnSmnFxmRyL/zaahx3Z/7sPii9sEMKyvB5CBEpkkOV63+EvySvAGvFMgP+1w4Md
         uu9tQJ31imK62Qa0qzbJSo1stWz82/jzKyH0Dsl5KXYKdBYApDBsfEViFqPdoyyZqhLp
         3dS6kXfT/VGZv8XubdafZK9tNObu7teV7Y6THl8tHY5ahajATE8edUVvEn9XdDTSWgaj
         LOOwl7tix2pSjVVz3IIa5a90wi6Nbfjmg4m5d5jBmPEXui/lg9PDlQe9wiZRAWLxGB6n
         gHwufZyw0leI4ZW6fzvoWn+Tsvfh4PMcgxfO0LtqRuKWCGS647EKodBkpPfM2WfwRD0d
         LDVg==
X-Gm-Message-State: AOJu0Yyasv6FotCmAroIzWaNfSO7hdVX2MKVsBErxr1uy6isjimk5Trj
	fm5GvO/zqmAvkIHaJuCXr71A8I6AdkCWPS2GHcupta5DTH7Lh6aqMjljjkRv2K8KK8dLWmwyZNP
	j8PhJt9E5EHS557H+t0UYyIt5OoRFpBiF2TM3cTmle/JjtOKAMbU=
X-Google-Smtp-Source: AGHT+IE/L3ppUUb+U5bYZDALwX47F0hhzNpsAETPKEpjV/xTDTaEKci+/zOpqJ7NHYQG4EexfZz17DHDsQ6vryW6z2w=
X-Received: by 2002:a17:906:5691:b0:a59:bbf0:88fa with SMTP id
 a640c23a62f3a-a5a2d57b1ddmr134681266b.28.1715336959589; Fri, 10 May 2024
 03:29:19 -0700 (PDT)
MIME-Version: 1.0
References: <p2-q466qgwwlqfn4m-qx47948p-a01-2x6n-0011N000017qoUMQAY@cvent-planner.com>
 <LV8PR84MB3654E4FE6C656979B4A47A78A6172@LV8PR84MB3654.NAMPRD84.PROD.OUTLOOK.COM>
 <CAO-mL=wM8siytp7hQezY8mm1AyL0tJc0Y-Za8wt1ebhAjbiOyQ@mail.gmail.com>
In-Reply-To: <CAO-mL=wM8siytp7hQezY8mm1AyL0tJc0Y-Za8wt1ebhAjbiOyQ@mail.gmail.com>
From: Kelly Choi <kelly.choi@cloud.com>
Date: Fri, 10 May 2024 11:28:43 +0100
Message-ID: <CAO-mL=wsbpmeyz7ZoD29TBPRpbxR_ZqKQPkQsO6FNduuzFQKcA@mail.gmail.com>
Subject: Fwd: [ANNOUNCE] Xen Project Summit 2024 Design Sessions
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000259fa40618170227"

--000000000000259fa40618170227
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

*Our design sessions are now open for Xen Summit! *

If you've attended Xen Summit before, you might be familiar with the
process.

For anyone who hasn't done so before, please follow the instructions below,
using the link to create an account
<https://design-sessions.xenproject.org/>.
Once you've created your account, you'll be asked to verify using the code
below. That's it, you're in!

We aim to have design sessions with a virtual element (using Jitsi/Zoom).
This is free for the community to join in and listen, and will be hosted by
an attendee in the session.

If you're going to be an *in-person attendee*, you can propose a session
and vote on sessions you would like to see discussed.
If you're going to be a *virtual attendee*, you can vote on sessions you
would like to see discussed. You can still propose a session, but please
note there will be no professional audio or visual equipment in person for
these sessions.

The final schedule will be allocated and arranged by the highest-voted
sessions.

Virtual links for the community to join Xen Summit design sessions will be
shared closer to the event.

If you have any questions, please let me know.

For in-person tickets, click here
<https://events.linuxfoundation.org/xen-project-summit/register/>.

------------------------------




We look forward to holding the Design Sessions at the upcoming Xen Project
Summit. The design sessions will be on Wednesday, 5 June, and Thursday, 6
June 2024.

We encourage everyone to submit a Design Session, the verification code is:
=E2=80=9C*LFXEN24*=E2=80=9D.

*SUBMIT A DESIGN SESSION* <https://design-sessions.xenproject.org/>



The process involves the following steps:

   - Anyone interested can propose
   <https://design-sessions.xenproject.org/discussion/new> a topic.
   - All participants review the list of sessions
   <https://design-sessions.xenproject.org/list/discussion>, indicating
   their interest in attending each one.
   - The session scheduler optimizes the schedule
   <https://design-sessions.xenproject.org/schedule> to accommodate as many
   preferences as possible.

Participants can also propose long-form talks by adding [TALK] to the
session title.

For suggested topics, sample Design Session submissions, and more tips
check out the Xen Design Session page
<https://events.linuxfoundation.org/xen-project-summit/program/design-sessi=
ons/>
for more information.


Best Regards,
Xen Project Events Team


------------------------------

--000000000000259fa40618170227
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"ltr"><div><div dir=
=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div =
dir=3D"ltr"><div><b>Our design sessions are now open for Xen Summit!=C2=A0<=
/b><br></div><div><br></div><div>If you&#39;ve attended Xen Summit before, =
you might be familiar with the process.=C2=A0</div><div><br></div><div>For =
anyone who hasn&#39;t done so before, please follow the instructions below,=
 using the=C2=A0<a href=3D"https://design-sessions.xenproject.org/" target=
=3D"_blank">link to create an account</a>.=C2=A0</div><div>Once you&#39;ve =
created your account, you&#39;ll be asked to verify using the code below. T=
hat&#39;s it, you&#39;re in!</div><div><br></div><div>We aim to have design=
 sessions with a virtual element (using Jitsi/Zoom). This is free for the c=
ommunity to join in and listen, and will be hosted by an attendee in the se=
ssion.=C2=A0<br></div><div><br></div><div>If you&#39;re going to be an <b><=
i>in-person attendee</i></b>, you can propose a session and vote on session=
s you would like to see discussed.=C2=A0</div><div>If you&#39;re going to b=
e a <b><i>virtual attendee</i></b>, you can vote on sessions you would like=
 to see discussed. You can still propose a session, but please note there w=
ill be no professional audio or visual equipment in person for these sessio=
ns.=C2=A0</div><div><br></div><div>The final schedule will be allocated and=
 arranged by the highest-voted sessions.=C2=A0</div><div><br></div><div><fo=
nt color=3D"#ff0000">Virtual links for the community to join Xen Summit des=
ign sessions will be shared closer to the event.=C2=A0=C2=A0</font><br></di=
v><div><br></div><div>If you=C2=A0have any questions,=C2=A0please let me kn=
ow.</div><div><p class=3D"MsoNormal">For in-person tickets, <a href=3D"http=
s://events.linuxfoundation.org/xen-project-summit/register/" target=3D"_bla=
nk">click here</a>.=C2=A0</p><p class=3D"MsoNormal"><u></u></p><table borde=
r=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:9=
49.612px"><tbody><tr><td style=3D"padding:0.75pt"><div class=3D"MsoNormal" =
align=3D"center" style=3D"text-align:center"><hr size=3D"1" width=3D"100%" =
noshade align=3D"center" style=3D"color:rgb(160,160,160)"></div><p class=3D=
"MsoNormal"></p></td></tr></tbody></table></div><div><br></div></div></div>=
</div><div class=3D"gmail_quote"><div><div lang=3D"EN-US" link=3D"#0099E0" =
vlink=3D"#0099E0" style=3D"word-wrap:break-word" id=3D"m_616053144227370237=
0m_-2685003709366885060body"><div><div id=3D"m_6160531442273702370m_-268500=
3709366885060mail-editor-reference-message-container"><div>
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;float:left">
<tbody>
<tr>
<td style=3D"background:#eff0f2;padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;float:left">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;background:white">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td valign=3D"top" style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><img bo=
rder=3D"0" width=3D"699" style=3D"width:7.2812in" id=3D"m_61605314422737023=
70m_-2685003709366885060_x0000_i1028" src=3D"https://custom.cvent.com/64779=
B2AC74D4D2BBFF4459DE93A5253/pix/d770244914e047019777029a32e20c53.png?d=3D69=
9"></p>
</td>
</tr>
</tbody>
</table>
</div>
<p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><span s=
tyle=3D"display:none"><u></u>=C2=A0<u></u></span></p>
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in"></td>
</tr>
</tbody>
</table>
</div>
<p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><span s=
tyle=3D"font-size:10.0pt;font-family:&quot;Times New Roman&quot;,serif"><u>=
</u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
<tr>
<td style=3D"padding:0in 0in 0in 0in;word-break:break-word">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;background:white;float:left">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in;float:left">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;float:left">
<tbody>
<tr>
<td valign=3D"top" style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100.0%;word-break:break-word">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:11.25pt 11.25pt 0in 11.25pt">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in">
<p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:black">We look forw=
ard to holding the Design Sessions at the upcoming Xen Project Summit. The =
design sessions will be on Wednesday, 5 June, and
 Thursday, 6 June 2024.<br>
<br>
We encourage everyone to submit a Design Session, the </span><span style=3D=
"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#212121">v=
erification code</span><span style=3D"font-size:10.5pt;font-family:&quot;Ar=
ial&quot;,sans-serif;color:black"> is: =E2=80=9C</span><b><span style=3D"fo=
nt-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#212121">LFXE=
N24</span></b><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;=
,sans-serif;color:black">=E2=80=9D.<u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td valign=3D"top" style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100.0%;float:left">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;border-radius:15px;border-collapse:separate!important">
<tbody>
<tr>
<td style=3D"padding:11.25pt 11.25pt 11.25pt 11.25pt">
<div align=3D"center">
<table border=3D"1" cellspacing=3D"0" cellpadding=3D"0" style=3D"background=
:#72be44;border:solid #72be44 4.5pt">
<tbody>
<tr>
<td style=3D"border:none;padding:3.75pt 15.0pt 3.75pt 15.0pt">
<p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><span s=
tyle=3D"color:black"><a href=3D"https://design-sessions.xenproject.org/" ta=
rget=3D"_blank"><b><span style=3D"font-size:13.0pt;font-family:&quot;Arial&=
quot;,sans-serif;color:white;text-decoration:none">SUBMIT
 A DESIGN SESSION</span></b></a></span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><span s=
tyle=3D"font-size:1.0pt">=C2=A0</span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td valign=3D"top" style=3D"padding:0in 0in 0in 0in;word-break:break-word">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100.0%;float:left">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in;float:left">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;word-break:break-word">
<tbody>
<tr>
<td style=3D"padding:0in 11.25pt 11.25pt 11.25pt">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in">
<p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#212529">The proces=
s involves the following steps:</span><span style=3D"font-size:10.5pt;font-=
family:&quot;Arial&quot;,sans-serif;color:black"><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in;word-break:break-word">
<ul type=3D"disc">
<li class=3D"MsoNormal" style=3D"color:black;line-height:13.5pt">
<span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;co=
lor:#212529">Anyone interested can=C2=A0</span><a href=3D"https://design-se=
ssions.xenproject.org/discussion/new" target=3D"_blank"><span style=3D"font=
-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#0d6efd">propos=
e</span></a><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,s=
ans-serif;color:#212529">=C2=A0a
 topic.</span><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;=
,sans-serif"><u></u><u></u></span></li><li class=3D"MsoNormal" style=3D"col=
or:black;line-height:13.5pt">
<span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;co=
lor:#212529">All participants review the=C2=A0</span><a href=3D"https://des=
ign-sessions.xenproject.org/list/discussion" target=3D"_blank"><span style=
=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#0d6efd=
">list
 of sessions</span></a><span style=3D"font-size:10.5pt;font-family:&quot;Ar=
ial&quot;,sans-serif;color:#212529">, indicating their interest in attendin=
g each one.
</span><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-s=
erif"><u></u><u></u></span></li><li class=3D"MsoNormal" style=3D"color:blac=
k;line-height:13.5pt">
<span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;co=
lor:#212529">The session scheduler optimizes the=C2=A0</span><a href=3D"htt=
ps://design-sessions.xenproject.org/schedule" target=3D"_blank"><span style=
=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#0d6efd=
">schedule</span></a><span style=3D"font-size:10.5pt;font-family:&quot;Aria=
l&quot;,sans-serif;color:#212529">=C2=A0to
 accommodate as many preferences as possible.</span><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif"><u></u><u></u></span></l=
i></ul>
</td>
</tr>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in;word-break:break-word">
<p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:#212529">Participan=
ts can also propose long-form talks by adding [TALK] to the session title.<=
/span><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,sans-se=
rif;color:black"><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in;word-break:break-word">
<p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:black">For suggeste=
d topics, sample Design Session submissions, and more tips check out the
</span><a href=3D"https://events.linuxfoundation.org/xen-project-summit/pro=
gram/design-sessions/" target=3D"_blank"><span style=3D"font-size:10.5pt;fo=
nt-family:&quot;Arial&quot;,sans-serif;color:#1155cc">Xen Design Session pa=
ge</span></a><span style=3D"font-size:10.5pt;font-family:&quot;Arial&quot;,=
sans-serif;color:black">
 for more information.<u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style=3D"padding:3.75pt 0in 3.75pt 0in;word-break:break-word">
<p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-siz=
e:10.5pt;font-family:&quot;Arial&quot;,sans-serif;color:black"><br>
Best Regards,<br>
Xen Project Events Team<u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;background:#62ab30;float:left">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<div align=3D"center">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%;float:left">
<tbody>
<tr>
<td valign=3D"top" style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:0in 0in 0in 0in">
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:11.25pt 11.25pt 11.25pt 11.25pt"></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" styl=
e=3D"width:100.0%">
<tbody>
<tr>
<td style=3D"padding:.75pt .75pt .75pt .75pt">
<div class=3D"MsoNormal" align=3D"center" style=3D"text-align:center">
<hr size=3D"1" width=3D"100%" noshade style=3D"color:#a0a0a0" align=3D"cent=
er">
</div>
<p class=3D"MsoNormal"><br></p>
</td>
</tr>
</tbody>
</table>
<p class=3D"MsoNormal"><img border=3D"0" id=3D"m_6160531442273702370m_-2685=
003709366885060_x0000_i1025" src=3D"https://www.cvent.com/api/email/dispatc=
h/v1/open/q466qgwwlqfn4m/qx47948p/transparent.gif"></p>
</div>
</div>
</div>
</div>

</div></div></div>
</div></div>

--000000000000259fa40618170227--


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:32:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:32:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719730.1122577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NYX-00039G-Q9; Fri, 10 May 2024 10:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719730.1122577; Fri, 10 May 2024 10:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NYX-000399-Md; Fri, 10 May 2024 10:32:49 +0000
Received: by outflank-mailman (input) for mailman id 719730;
 Fri, 10 May 2024 10:32:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=asbK=MN=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s5NYV-00036i-UT
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:32:48 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2408::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a32f425a-0eb8-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:32:45 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DS0PR12MB6629.namprd12.prod.outlook.com (2603:10b6:8:d3::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48; Fri, 10 May
 2024 10:32:42 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.047; Fri, 10 May 2024
 10:32:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a32f425a-0eb8-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=APD3gvOqEcQJQ6279n9k7BN7Y1z7b/0qm6JzmuP5jKOt2J4uE80wAB2xg2U/F8Y8IdCgIHewANU8H55nUHOgnM8efEDlYJH+oeD6jx+vdP4BRqvLiXBsEPOsC2QJ8kyX5naHOG2rcLiV9wdvO6Uvu1OxlXAsjKFptsIyd/3Q/4grHe3XcqBn4Fxy1gFiVXKq0Hh9zM3f3p4cRnlUl3Xg3FM8g3nNjkUU2GYaxXgIUcDQtCw4Hle2Xseaqtcc7fvG5GR7HdWZV56paTYgHwoEw0nakR2V+zxk8BDAVo+sikkuk24cYZnMWFghxe8++qFC0M04tVDsHdGOH/j0VozX8w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=3DlvffFaJWIcmHhxIE29kRj/ZR0JyrabAJyzSjN9rTQ=;
 b=EvZDVBLUoLq6z9SwcW9iIww5N6Fr/oSpsSeVPXkj2/5uakikqucRno+e8lKU/X3wvmIp3ZF2ObfC5K8OaBppMNlQaR2Gd6UyWeihiF8SzyuYQl/2SBp833tCZKlwS84WEqLl/08lUJ2gBP/M1KRTSrFloosPnASCSr8ssiHT3W0vc/ACmIQHyx+NRVHBdHdzk4OlpezDf3iLgBzgwiPltuWlgARQv6/ZZbrvZep6JZfbRezGp820bBdGI1isR1f5pd2HV4PTQ0n2FAt9m8tq3wF4Bn2AhC2N1GEJA1k8kOWkf3D6jTEJmuZkd1I5p3MY2juZX0auhr3P/JLzm+1TJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3DlvffFaJWIcmHhxIE29kRj/ZR0JyrabAJyzSjN9rTQ=;
 b=t7SMxhd4BL7jJRgEIwWORoLKiMfeX6Nglgyrc4bvpkcPa6/YmMrra757kmsMSuqANKrQ2pUY0/S7vY2JHc4QUx1YwnKj+ssmxJb0NFjHwmLv/zgYws02hOfn0v7aKHbpIiM39c/PZqll0O13xwscS+Y6ANQ7T3QQ1xpSXZBegY4=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>, "Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
Thread-Topic: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get
 gsi from irq
Thread-Index:
 AQHakgrPZQMiHHEzE0Gs/TUOGsHh17GQJ2yAgACnAgD//41hgIAAiXAA//9+YoCAAIdwAA==
Date: Fri, 10 May 2024 10:32:41 +0000
Message-ID:
 <BL1PR12MB5849F1DE8B4A3538C79CE5D3E7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
 <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <d0b5e7d5-3503-49be-9fa3-4b79c62059ca@suse.com>
In-Reply-To: <d0b5e7d5-3503-49be-9fa3-4b79c62059ca@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: IA0PR12MB7579.namprd12.prod.outlook.com
 (15.20.7587.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DS0PR12MB6629:EE_
x-ms-office365-filtering-correlation-id: 349365c0-84e0-43e3-6bc4-08dc70dc85b6
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|1800799015|366007|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?SVdyVTlKbCtTWHNsTTZyUDRzUXRaSjBmYXJqVHMwc0VweG5mSDVnV05ZVUpz?=
 =?utf-8?B?YW9jSTRKS3l4WVdRWmllbGVsRTNleXEyUjFvY2g0UUhRSStQb1ZFbFZ1eHlh?=
 =?utf-8?B?SVFycktjWkpZV2pGeVRMWWR6VTVFQk1QSjhvajNHQTREWDB3S21uNDZiQW9P?=
 =?utf-8?B?Mys4ajJlMmZPdnYwZDhlZkpVZnhKUGNMckEzSkhBK2V6WTlsa1dZbk5xOEZJ?=
 =?utf-8?B?ek1JZHlnYmsxNVBzb2hmVHdCZDcrRnBnQTh1WjBqekNCZXc5d0c0OGVJaWtO?=
 =?utf-8?B?N0ZvVWFjVmh6QjVqT2pIMEtmUjZuM3huSnVlRmN3SW83Z28ra0FSNVAvSjVv?=
 =?utf-8?B?OHJ1ZlQ2OUw3ejl1YTVtdElybWp5TlRmaG9CNzZRQlhkTkxVempsc2Q4WmR3?=
 =?utf-8?B?SVJQWTZqdkNPQ1BLSTJVWlIyMVVCTi9iSm1zaFdxUVpkMFNTZVIwdU1LVDFw?=
 =?utf-8?B?YjJOZ29tUVU4MmlXak9YUW5FanJMMVN6OHQwRVYvMXlleXIrTGRGOXRQUlE1?=
 =?utf-8?B?eG9VYmJqZk92T3Z6Lzk0RVZMc3dFNFQ3ZmtWemNvVWVweUdNK1BpeUh0K0sw?=
 =?utf-8?B?bGZmcFI4bUhPSnl4M0VMSzV4ajBTTERMVlJwOXUxWjl5RXpjTEpvN3o5YTlC?=
 =?utf-8?B?bGtFYzUwSldmOW01eTM5UFgyaTlkN29OYVpVMDdTci9kSzRJODhXTytNTDU4?=
 =?utf-8?B?VUJMSmNrZjRINUQ2cnpmTjhJSDJLMTdnNjk2SXg5QnAwT3FjSHFwNXRmS2FR?=
 =?utf-8?B?dVNjZG50THlCVGV5S1h6UWNRTzBtZmVGeUgxNkpjUWc4K3lKdGd5ZGN6eUJK?=
 =?utf-8?B?NWNNZExxTk1nYWJlZ2N4WE5IR3ZKK3p6R3Y4c2FzVE1VV0tFMW9GSXlTODdi?=
 =?utf-8?B?ZU9RY3JyU2piaE1zRlRETXJvM0Q1UzN4UFdMSHpJYVZIUlZGVjVMUUFuMTNo?=
 =?utf-8?B?OWJxUWUzaSt3aGhBTGpRSnpoUyt2bE10M3ZkcnRQeGhJMmF3WHZJTnBCOThu?=
 =?utf-8?B?UElSSkxpU2JQK1V5YTRwSjMwdTdndXRKSzR4REt3SmF6Yk13UHkvdksvNXlz?=
 =?utf-8?B?YXpOUEFsb3kvMlNKR0dTSlJVRHdHSVhKT2RCaXdGMVd3Tk9RUXNhcUVDZ0hP?=
 =?utf-8?B?blhSck5QbFJUaitjVUZ4cFFKdklveW40Z1FQbXBRY3VrQTAyVW1YTUlvR29m?=
 =?utf-8?B?TksyRDQwK1JzVUFmNGU5NXFqYVhRdm5OMnIxWXFYTWhQUDRhTU1SOXBKUE56?=
 =?utf-8?B?V1FwQ0VPSVYwQ3FzQlRjamcrMWNQNEFmZ09iT2VkQlpCOHRCRGdkTFlmaHh5?=
 =?utf-8?B?U2lXU3RLNzJYSWtVQTk4dTAxWElTRUxXWFNpVnlZdlpXRTNuVWVkbTNaSklR?=
 =?utf-8?B?UXBnbkU0SHRnUFNJTGo2MEdFSUhlOFRaZU9kUVdUUzlPQUNMaUl2N1gxM292?=
 =?utf-8?B?TmJTdStjbHpNTXJrYlRwWkswN2tMcCtkY3N3UXU4SzYvbGFLQU5ZOG1tdDVt?=
 =?utf-8?B?K00vb05RZTlySTJ4eFVaaE52bmxnWnNiN2NOdldVVnErMmxNR0NnUWxRMWxT?=
 =?utf-8?B?aml2M2ZwTjM4VjkyQkM4amdFQlZDdm16dkVZRDZKT2ZwSFFKaG1oSDVaUEww?=
 =?utf-8?B?MFlnWkN5N2F1UnhRTUhIRE11eHlEMGhHMEJZekdjSnlJYW1mcXl0cnh2b1Vu?=
 =?utf-8?B?dStIU1NYa1Vndm5xU2t3K084RjdJUGExdFl0WWtzUE5CbUw5VlRxaE1pb3FG?=
 =?utf-8?B?Snc3MUZRNW9OUlY0OUU3RDVpQVF5YlhjSEpqYkFBR1RGeERzd1RoV3IrRkhV?=
 =?utf-8?B?OHhPbENhN2VKbzdUWWJtZz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?L3MxTCtvbTdZaHI2cFNwRXVvM1lQVmh0UG9FaHpDSHdwcHJ2K2JGUUphcXJ4?=
 =?utf-8?B?K2t3WVhibTBoaDJSejViT0oyb3NsdnhGSjIyZDVxT28wVCtReUdJUGJ6V0E1?=
 =?utf-8?B?d0FQak81S0xQdldldWhRZ0h4Wk1Mc0dBWVBra2xXaE9aUTBSdlYwbWtQZjBs?=
 =?utf-8?B?allkNXFXc1o5RTh4SVIxcTJlMzJBNXBQKytzZzh5S2JucWwxby9UazByNFR2?=
 =?utf-8?B?UDM3Qktrb3NzNnF2UHJUdzFjOC9NalZwQ2RSa3g3QUp1VmVmNERKU2YwMTZ2?=
 =?utf-8?B?cmVXMHJtOGtJS3p4c015NFZKU1hNczZUNDk2L05EeHJ6VGRhUnovNjVoR1Vw?=
 =?utf-8?B?Y3RrOFJzZzZqOWVGRzRaY09kQnV6T2RTL0dmRExHeWd6azBvKzVZSHNrQzN6?=
 =?utf-8?B?czJLQlVqRjhxdUEycU51bGJrUENyWDFKMjRSbUN3N29qakhQdldiZEJaVEdo?=
 =?utf-8?B?MXVTellxTjBkOEMwM1Z4NzFHMHNYREo2b1NMZkt4SUx2dkZBQnZNUnREak1w?=
 =?utf-8?B?aTZiTURWSGpybmlLYm5EeW4ycWtGMDdKWEZqQVJQdHNWNzlJRVluamtySisr?=
 =?utf-8?B?L2RpNTZ4TjU2ZmxhWnFtd0l5WFVhdS9HMDh5aHRTQnhrTE9ycktUUS9CaFVt?=
 =?utf-8?B?eU5rckZxK3U3UWg2L1J5ZVRTQ3o3OHdOWjZaUHBTZkJIZ1BrelB5WkVVRU8w?=
 =?utf-8?B?dHVHNVliWFV5dmd2NFpFd2hKSzA5ZVg2WDZJbmV1dTU3dVFxb1RzQWJrVlFD?=
 =?utf-8?B?LzhaVTlSTUI4ZmdJd1QwRHdtdzh6ZUZkVkVIQjRYSGdOSTd2NFJzaGUyMXhX?=
 =?utf-8?B?TjRlUjJKa1IvcmdBZXViUzd2cEpqWE1TTlk4RWRMWVlkL3htRTQxTm4wUTU5?=
 =?utf-8?B?djRCRlJEUEtQSldsTEo5Nk54bU9WM1JSUmdCM244aGhUN1VvaC9IUDVScFlF?=
 =?utf-8?B?SGN2cE5WVk53SFhZQndTeXZqRmtLUVlneHd3eGFtWURIL0ZFSFdPZ2p4Y254?=
 =?utf-8?B?NVIrWDNUcVVrcnZBV0JMNmp2TWpBZjZacWpyVDV6Q0JZb2U3L3M0dWdHbGFZ?=
 =?utf-8?B?OGJ3TUpvOUpQZFBjR2k5Y1dpbGdSc1daOVBNME5JMHl0MEdma09SVHRtSW5p?=
 =?utf-8?B?aWxjODNqck1mOFdiUEJmYWhhdUtwNUxVeXJnSVFrT2hXYjZSZlUxNkRMcUl5?=
 =?utf-8?B?OTk5WmdJODlEUmZrbFpFdGxmaW50eSt5SFZsZmhkeTk3U1VjSVdiWlZpK1Fi?=
 =?utf-8?B?TjRYemVRckNQR0RZVmw1bTEzK2RYZHM5dUlFcmJRaGJrMUlJWnZtWE5zRGdP?=
 =?utf-8?B?b3k5WldKd3NFQXNFUHBhQ0UzcVFRVEpuUk42OEN5eVJDRlVFdys5ZDd3eDdo?=
 =?utf-8?B?aXViK2xsR1l1eFI5eXpJU3BYQ0tRNVhtTmpRL2JqRGsrWkFUbXRMZUtXRWwr?=
 =?utf-8?B?OVNmbnBEazY3cm4vMkZFaVJ6TXVWL3VyNEIvQkNlZnViQWRLV1ZqUEsxQjRa?=
 =?utf-8?B?dUZwbWZ1QnNBSDA2SFRrV3ZMZVF1cW9VQkhWQ21RK2RidGtCcW5qNjRCR3hH?=
 =?utf-8?B?cFpLdkJRSmhhY0Q4dmI4TVhnMlFYUUR2NSs4QTlVRG51M1FuT3p6VXVUWEM2?=
 =?utf-8?B?eVZ1cktMWjdMMlk5Y25HdnVGdWFYZHdqUng2Q2F4V2plVXA4TFFCaXVBTU1V?=
 =?utf-8?B?c0UzMERpLzJTbmVnZDVPbFBWRU9saDhqUWN6T2dlbzNaejQyYUg1UUJFaXQ1?=
 =?utf-8?B?MnJzQXVMVnRIbitoT3FydGtWczhURG1zdnFDWU5Wa0gvNm42d2IzR3R5QXVW?=
 =?utf-8?B?MCsvSjdhZVI5Z2MraWhXSHpBa0FGTG5jR3JGNTRFckRkY0VTaWxqd00vRHRP?=
 =?utf-8?B?WnVkK2pLRTEzRzFBWFBySDFsbmwzaFR3OEZoYlJ2MCswR2RjQmlHOFZzd0U4?=
 =?utf-8?B?Z2dJSHJpOUlGejlUSmd6MlEwSlR5aEYzUU8xUVN0c1ZZVW1uZVZLcHVQbk1h?=
 =?utf-8?B?UkpaazZOQ3Q3OThzU1p0QWNrWFVyVmx0eW93dkRMQjJQOU91ZjNndjJQM0pl?=
 =?utf-8?B?S2JaRnd5bVc3RGdoN05wNmh1TUsyNHNMOFBTeHJvK0JnVU9VWEI4TkdaWjhL?=
 =?utf-8?Q?Eros=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D4484D65BF67E54B92A7D46EA85A2853@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 349365c0-84e0-43e3-6bc4-08dc70dc85b6
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2024 10:32:41.7907
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fe9jDEy0gxptDI8M10sIrAqA+uKHMDogMvsOl62gzxqSSFAwk0zx/Q/DmdCBudXrh57TmkQsoR/zBFhtO0lq8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6629

T24gMjAyNC81LzEwIDE4OjIxLCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPiBPbiAxMC4wNS4yNCAx
MjoxMywgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzEwIDE3OjUzLCBKw7xyZ2Vu
IEdyb8OfIHdyb3RlOg0KPj4+IE9uIDEwLjA1LjI0IDExOjA2LCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IEhpLA0KPj4+Pg0KPj4+PiBPbiAyMDI0LzUvMTAgMTQ6NDYsIErDvHJnZW4gR3Jvw58g
d3JvdGU6DQo+Pj4+PiBPbiAxOS4wNC4yNCAwNTozNiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+Pj4+
Pj4gKw0KPj4+Pj4+ICvCoMKgwqAgaW5mby0+dHlwZSA9IElSUVRfUElSUTsNCj4+Pj4gSSBhbSBj
b25zaWRlcmluZyB3aGV0aGVyIEkgbmVlZCB0byB1c2UgYSBuZXcgdHlwZShsaWtlIElSUVRfR1NJ
KSBoZXJlIHRvIGRpc3Rpbmd1aXNoIHdpdGggSVJRVF9QSVJRLCBiZWNhdXNlIGZ1bmN0aW9uIHJl
c3RvcmVfcGlycXMgd2lsbCBwcm9jZXNzIGFsbCBJUlFUX1BJUlEuDQo+Pj4NCj4+PiByZXN0b3Jl
X3BpcnFzKCkgYWxyZWFkeSBjb25zaWRlcnMgZ3NpID09IDAgdG8gYmUgbm90IEdTSSByZWxhdGVk
LiBJc24ndCB0aGlzDQo+Pj4gZW5vdWdoPw0KPj4gTm8sIGl0IGlzIG5vdCBlbm91Z2guDQo+PiB4
ZW5fcHZoX2FkZF9nc2lfaXJxX21hcCBhZGRzIHRoZSBtYXBwaW5nIG9mIGdzaSBhbmQgaXJxLCBi
dXQgdGhlIHZhbHVlIG9mIGdzaSBpcyBub3QgMCwNCj4+IG9uY2UgcmVzdG9yZV9waXJxcyBpcyBj
YWxsZWQsIGl0IHdpbGwgZG8gUEhZU0RFVk9QX21hcF9waXJxIGZvciB0aGF0IGdzaSwgYnV0IGlu
IHB2aCBkb20wLCB3ZSBzaG91bGRuJ3QgZG8gUEhZU0RFVk9QX21hcF9waXJxLg0KPiANCj4gT2th
eSwgdGhlbiBhZGQgYSBuZXcgZmxhZyB0byBpbmZvLT51LnBpcnEuZmxhZ3MgZm9yIHRoYXQgcHVy
cG9zZT8NCkkgZmVlbCBsaWtlIGFkZGluZyAibmV3IGZsYWcgdG8gaW5mby0+dS5waXJxLmZsYWdz
IiBpcyBub3QgYXMgZ29vZCBhcyBhZGRpbmcgIiBuZXcgdHlwZSB0byBpbmZvLT50eXBlIi4NCkJl
Y2F1c2UgaW4gcmVzdG9yZV9waXJxcywgaXQgY29uc2lkZXJzICIgaW5mby0+dHlwZSAhPSBJUlFU
X1BJUlEiLCBpZiBhZGRpbmcgIiBuZXcgZmxhZyB0byBpbmZvLT51LnBpcnEuZmxhZ3MiLCB3ZSBu
ZWVkIHRvIGFkZCBhIG5ldyBjb25kaXRpb24gaW4gcmVzdG9yZV9waXJxcy4NCkFuZCBhY3R1YWxs
eSB0aGlzIG1hcHBpbmcoZ3NpIGFuZCBpcnEgb2YgcHZoKSBkb2Vzbid0IGhhdmUgcGlycSwgc28g
aXQgaXMgbm90IHN1aXRhYmxlIHRvIGFkZCB0byB1LnBpcnEuZmxhZ3MuDQoNCj4gDQo+IA0KPiBK
dWVyZ2VuDQo+IA0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719759.1122587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NoS-0007MW-6v; Fri, 10 May 2024 10:49:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719759.1122587; Fri, 10 May 2024 10:49:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NoS-0007MP-3o; Fri, 10 May 2024 10:49:16 +0000
Received: by outflank-mailman (input) for mailman id 719759;
 Fri, 10 May 2024 10:49:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5NoR-0007MJ-1i
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:15 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id efdb43ed-0eba-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:49:13 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-41ebcf01013so10491975e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:13 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f87b2648fsm94463915e9.7.2024.05.10.03.49.10
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efdb43ed-0eba-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338152; x=1715942952; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=cZjM77xwKEbJpKmLT3A1NDbHI9hBidp1PDA51BLQSpY=;
        b=J11ViC51vwH2yn9ELK9ECH17l3hKCb7zXO8G4fNIU9qAgdCyafYNGipd68RAfSYE2G
         HtQktOI+6n02NO3nL27VzZ6p9mH8e5Ps7q3RP/Lw5hUf1kGiq308ni4HlOlNKl1hrZOk
         HzoLKUUDuSKLgc0a/BOBnAuEA2uJLJwm3gp0xL4vQqaaaaK8D4ygI1/adpB7AXTwWl/R
         Jqoqy2Ynuh3YPi/51eYcWxX9n73YlCtTuukcS4PicSRJadCpqfn2rAJVDAaQ2aFe7UmT
         8HM5xDMySvtIzI40CE/20psNZIprnCms9O/YqPCh1rpxrCXU4DjDmLloWMxnJqKUdY6U
         F3OQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338152; x=1715942952;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=cZjM77xwKEbJpKmLT3A1NDbHI9hBidp1PDA51BLQSpY=;
        b=j0qXMTk53Ers1XoQwsFZl3EEoGyhK/upCz3QLsuqMCOhVG1tGxX0WpL4YyEwB8CuDw
         FJFALx19ZnTp/7fY0WCNzuoRYmjmTUAQiMZBRNruFaclCpvVxAcSCC/n9pJ3gBJPuTOv
         1DXqqLwh+4TB9cIa++B4bLF9v9qcYSAOGo7Jd0CMRWr+n2vROSwFXr7PRZVhAeL5KMHB
         VyAUnG2GdkV7wU+P5khuDRuNYoVQZ0bA2kamrkT54Lq1jlY2IEu7AREB8kp0RuyrlyYq
         CdGPS2tBMqxSWSgIw6S3Zye1jQucPQ2OZdbtyF4vTHUVSNkH5TmntGavqi6CkFUiOztm
         Sy9Q==
X-Forwarded-Encrypted: i=1; AJvYcCXzGhbpZ3cXAKEIv6XBcnvv3In0h9DSPMMADg1ljgR8C4gJXTa5YC+XITjsBpEgg1BYm0Tn3OGV5QWvyUsS7tvnI35JlcAnLp4h1dF35mQ=
X-Gm-Message-State: AOJu0YxD8pEqhsXvmBAi/lVZeu2wlNw5oNw+02EhgNDEuTMsr+jGGEV3
	fnI3AMUo4e1quXhEWvgUVwSrh8WV7a5oxQGNR7SuHQwsGmiy8T2zuyCqhbY/bKw=
X-Google-Smtp-Source: AGHT+IH46uVuTTAQ+BBftsxQmGw9P4+zW2TPR3t6Wdb1Buyly/dSLdeEPF5bm3XgUtGobYA/S4G4qg==
X-Received: by 2002:a05:600c:450b:b0:41a:7c1d:3326 with SMTP id 5b1f17b1804b1-41feaa30479mr20356075e9.8.1715338151876;
        Fri, 10 May 2024 03:49:11 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Philippe =?unknown-8bit?q?Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 0/7] hw/xen: Simplify legacy backends handling
Date: Fri, 10 May 2024 12:49:01 +0200
Message-ID: <20240510104908.76908-1-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit

Respin of Paolo's Xen patches from
https://lore.kernel.org/qemu-devel/20240509170044.190795-1-pbonzini@redhat.com/
rebased on one of my cleanup branches making backend
structures const. Treat xenfb as other backends.

Paolo Bonzini (2):
  hw/xen: initialize legacy backends from xen_bus_init()
  hw/xen: register legacy backends via xen_backend_init

Philippe Mathieu-Daudé (5):
  hw/xen: Remove declarations left over in 'xen-legacy-backend.h'
  hw/xen: Constify XenLegacyDevice::XenDevOps
  hw/xen: Constify xenstore_be::XenDevOps
  hw/xen: Make XenDevOps structures const
  hw/xen: Register framebuffer backend via xen_backend_init()

 include/hw/xen/xen-legacy-backend.h | 15 +--------------
 include/hw/xen/xen_pvdev.h          |  3 +--
 hw/9pfs/xen-9p-backend.c            |  8 +++++++-
 hw/display/xenfb.c                  | 15 +++++++++++++--
 hw/i386/pc.c                        |  1 -
 hw/usb/xen-usb.c                    | 14 ++++----------
 hw/xen/xen-bus.c                    |  4 ++++
 hw/xen/xen-hvm-common.c             |  2 --
 hw/xen/xen-legacy-backend.c         | 24 ++++--------------------
 hw/xenpv/xen_machine_pv.c           |  7 +------
 10 files changed, 35 insertions(+), 58 deletions(-)

-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719760.1122597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NoX-0007bV-DI; Fri, 10 May 2024 10:49:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719760.1122597; Fri, 10 May 2024 10:49:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5NoX-0007bM-A4; Fri, 10 May 2024 10:49:21 +0000
Received: by outflank-mailman (input) for mailman id 719760;
 Fri, 10 May 2024 10:49:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5NoV-0007MJ-Mp
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:19 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f34551cd-0eba-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:49:18 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41ffad2426eso3650765e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:18 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b79bdbcsm4196331f8f.23.2024.05.10.03.49.16
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f34551cd-0eba-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338157; x=1715942957; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XVbCv81rAzGflBvcSrzxqDvQbbVCQ+aUGbh/Oa+Rhcw=;
        b=WGWLDirjkrVHyU6JwVlrULpLZGf7hJrkCfjZuocw28RRNUxenNRpApohRTL7gTxWKN
         zVyQUWf75wErhf0SJBNR7rSwd5RzUzJKGPW+jckrVVU+sPkGTmjWSHqKc66lebQfNccZ
         r3vn9jJ9CPqdrFup+wNc76W57DgSv85ifbk1U94JiW7AK+U40ncM5EyXl3oXc8uuYdPb
         yi3/QRHd1d4KMagHgwzzb0iYpztAetFyfBtkGHKDMyQ3IS96K7PIOhU5bPAenVom09rL
         vdKmjAL5XRYL0n7srcbXowJfhiwl2EIA3aHlZ36wtBOKTCubgW4hFC8Kx+7K1moBqN4n
         xRhw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338157; x=1715942957;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XVbCv81rAzGflBvcSrzxqDvQbbVCQ+aUGbh/Oa+Rhcw=;
        b=cYNt/sziYvK/qsyFYgulknaUDrmEUiPltyIZs4ltqg1m8HM49a4KcPCcpzuJKWYD7J
         5pe0CzKBqycSPm5Fn7KF4ArAbZQhoQkHWABHYoUAeP1At7KTJJbcx4bcoGBr7QMsUk0v
         SAB9nEsoCASPJ2JI1o4s9etlYAEdNslWkAkZRLB8dtDdaGZGOuW/dYsq6fcsHqLrobIx
         rfD0BG78pj8At8GjREl0jIuyZnuR7Gmu+MbndQ7wk+1eUV8UI0XxBAjbITNA3DdejX4w
         GfhjkUAVBEkr2RshVSVW6aEFZmJHNzM2GaTP1O02g3poHW1055rbOzWRU7UnbbGHfvw2
         GQTA==
X-Forwarded-Encrypted: i=1; AJvYcCWoebHz5D7K6R8nRYMfnShiZ5lIvOJ/fX0QRbDC4ywIYjOKxixST0C4Yld2Av/Fmgsz0kgi922sh/Yg5Ih2rrZMEKGLwU3ZsVI9ikOKiVE=
X-Gm-Message-State: AOJu0YyMqT8xVY1nP1o7dkiF3DRW/F40c2XVPfDovlEC+LW9INZLWj9B
	7qMyaibAg25Cje/20EEygK3KoN2V3IHkkzclxdPeNm5xrP9maGBIx5JlESvHxRc=
X-Google-Smtp-Source: AGHT+IHdk5RMerXt6xlL7OKXtzLIPevwNSQsVWTfglLFy+cVjQ0DDHmvBq710DZ/D55SqgKGH3hL1w==
X-Received: by 2002:a05:600c:3508:b0:41b:f116:8868 with SMTP id 5b1f17b1804b1-41feaa38a79mr24859125e9.12.1715338157651;
        Fri, 10 May 2024 03:49:17 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 1/7] hw/xen: Remove declarations left over in 'xen-legacy-backend.h'
Date: Fri, 10 May 2024 12:49:02 +0200
Message-ID: <20240510104908.76908-2-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

'xen_blkdev_ops' was removed in commit 19f87870ba ("xen: remove
the legacy 'xen_disk' backend"), 'xen_netdev_ops' in commit
25967ff69f ("hw/xen: update Xen PV NIC to XenDevice model") and
'xen_console_ops' in commit 9b77374690 ("hw/xen: update Xen
console to XenDevice model"). Remove them.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen-legacy-backend.h | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/include/hw/xen/xen-legacy-backend.h b/include/hw/xen/xen-legacy-backend.h
index 2cca174778..eb22633caa 100644
--- a/include/hw/xen/xen-legacy-backend.h
+++ b/include/hw/xen/xen-legacy-backend.h
@@ -67,14 +67,11 @@ static inline void xen_be_unmap_grant_ref(struct XenLegacyDevice *xendev,
 }
 
 /* actual backend drivers */
-extern struct XenDevOps xen_console_ops;      /* xen_console.c     */
 extern struct XenDevOps xen_kbdmouse_ops;     /* xen_framebuffer.c */
 extern struct XenDevOps xen_framebuffer_ops;  /* xen_framebuffer.c */
-extern struct XenDevOps xen_blkdev_ops;       /* xen_disk.c        */
 #ifdef CONFIG_VIRTFS
 extern struct XenDevOps xen_9pfs_ops;       /* xen-9p-backend.c        */
 #endif
-extern struct XenDevOps xen_netdev_ops;       /* xen_nic.c         */
 #ifdef CONFIG_USB_LIBUSB
 extern struct XenDevOps xen_usb_ops;          /* xen-usb.c         */
 #endif
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719761.1122607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Noc-0007tk-Je; Fri, 10 May 2024 10:49:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719761.1122607; Fri, 10 May 2024 10:49:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Noc-0007tb-Gl; Fri, 10 May 2024 10:49:26 +0000
Received: by outflank-mailman (input) for mailman id 719761;
 Fri, 10 May 2024 10:49:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Nob-0007sQ-HF
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:25 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f6e23306-0eba-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:49:24 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e538a264e0so7789451fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:24 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b896a06sm4231043f8f.27.2024.05.10.03.49.21
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6e23306-0eba-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338164; x=1715942964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WG6hHvxHnsfDLBmoixzqYQgPTY2J5n6T9oUHauE6pdQ=;
        b=YiearKB/mIu9+rSlpfPyjFaEzdcPkvOzunu9UYVmdCLQSVOVENujUBdRDGAaWJdiNU
         T9kl73sB+LhujSJqZnAIwX5OaxEU60uDQEJf0/icpor8BA3WnOydayMRW7QZABX/SQ6b
         HTUZkHC/fy/+UfX7cZUKpWfbrEmq6n1BVrk7VQ4f2F4BBdN2mYwBAViwute58nB86MGu
         gjqT6fMHEyhOxoFTc3Zp8ffyZ9FIuiPC9SsWYsK/I+hZyj5klTsQGsiKwuybUz8LLbAj
         Xmb4UTeKpcMq0KMqe0ZRDcxQS0a+qxGdgLs7uGDdZNC5l8ATmuj6wu5rGS29js84GEd6
         3TVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338164; x=1715942964;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=WG6hHvxHnsfDLBmoixzqYQgPTY2J5n6T9oUHauE6pdQ=;
        b=HLjMrzKBUnMEUunN2jxrrpmVQ1ArCK9QWfZ7/fj4eFg13YU48iq2Tqkw7SUcd77/SX
         x+KLMHuTeQd5e098V47q0k2DEHgCU36FUnuwKYUYQL6cJunw1KJ2q2JmWAHyus1IsEpX
         Vn7Bwf11ZKcl+2viSO+xkOeowIZZqWcsjZFAxF2mLQF7DcMBF56pbQB/ZhPn7S6yS77o
         iqcEQqGFbX4JHB2lom6M6z4+NpFbHyXG1iwhT9pKCyWUQ23zEc+bn8vAmK4rhwl2M5r0
         KN2rpEiXZrjWpmRAIIr/+4MqO7mf8Oq+AApQiCF8DCy5HcbpHC6xgNwxeu6a7BivYnad
         PS+w==
X-Forwarded-Encrypted: i=1; AJvYcCWCjjjDelFk7bmSUnjP8WU1O6hxqIV/VB+4gcNK3T/ZRXlFNfJlBlqqO8URc7pCo7tJXDt/ZQqvPLSW9Z5Cfz2PStia6RiXfejONCORuL0=
X-Gm-Message-State: AOJu0YyxJseW1pOQW7u7our+eIX7Ir1lNTWp/PW8sY0xktFvpHSoiBLK
	uivlWO02hf1EncsENlOHsjkPmg7Wk5HDHZ2AmFkliiFBmt1m7y2+1m3f5Xcrz5A=
X-Google-Smtp-Source: AGHT+IFmnZ+GsRWrg13UBzYocLVh19CoDwULJS52v4mItPwgRYVUMX2l1wSejvKjHmjEdqsBZaZvSw==
X-Received: by 2002:a2e:859:0:b0:2e5:4171:1808 with SMTP id 38308e7fff4ca-2e541711898mr7342581fa.51.1715338163501;
        Fri, 10 May 2024 03:49:23 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 2/7] hw/xen: Constify XenLegacyDevice::XenDevOps
Date: Fri, 10 May 2024 12:49:03 +0200
Message-ID: <20240510104908.76908-3-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XenDevOps @ops is not updated, mark it const.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen_pvdev.h  | 2 +-
 hw/xen/xen-legacy-backend.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/hw/xen/xen_pvdev.h b/include/hw/xen/xen_pvdev.h
index ddad4b9f36..fd9a33e6ab 100644
--- a/include/hw/xen/xen_pvdev.h
+++ b/include/hw/xen/xen_pvdev.h
@@ -53,7 +53,7 @@ struct XenLegacyDevice {
     xenevtchn_handle   *evtchndev;
     xengnttab_handle   *gnttabdev;
 
-    struct XenDevOps   *ops;
+    const struct XenDevOps *ops;
     QTAILQ_ENTRY(XenLegacyDevice) next;
 };
 
diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index 124dd5f3d6..35f2b57560 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -170,7 +170,7 @@ int xen_be_copy_grant_refs(struct XenLegacyDevice *xendev,
  */
 static struct XenLegacyDevice *xen_be_get_xendev(const char *type, int dom,
                                                  int dev,
-                                                 struct XenDevOps *ops)
+                                                 const struct XenDevOps *ops)
 {
     struct XenLegacyDevice *xendev;
 
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719762.1122618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Noi-0008Hj-Va; Fri, 10 May 2024 10:49:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719762.1122618; Fri, 10 May 2024 10:49:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Noi-0008HV-Pn; Fri, 10 May 2024 10:49:32 +0000
Received: by outflank-mailman (input) for mailman id 719762;
 Fri, 10 May 2024 10:49:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Noh-0007MJ-Ou
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:31 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa5aefe3-0eba-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:49:30 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b2119da94so12276475e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:30 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccce2580sm58674215e9.18.2024.05.10.03.49.27
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa5aefe3-0eba-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338169; x=1715942969; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lLydyXWVq+dvuttt9ieifXl3jWWZU+LQZOiqeB7g+WE=;
        b=YaRAKhpNMVRrQIrZ80PwU3rbfxaLvgjW4SiRcIGQ5kf+8oDfoMO0sE58LCe0UAJh8g
         vOTvD8tgz/lyDNFL35xVPyA8ouWEd/26fAz/qKGTZM19TAJPAsNMt7JKLCSIKns4rr97
         P0RfzXUVyfxRut6PSgeNsbke62evzwW8I0OPKYJZZHmc/95QU/pZgJ0Up/EvXjya5Zu/
         ylHPWAE3nW3d8DfeueGEgrPGLdKlH2tg0wcMI1TbJyfLEangrQbPCohD93Fj19YCZaLW
         GynwdBVylaaCF1Sz008eYMc5ckEJHQOKlMgBZ4VnOmpWatISSIJFMJhmeUuOPCFwYIuH
         4szQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338169; x=1715942969;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=lLydyXWVq+dvuttt9ieifXl3jWWZU+LQZOiqeB7g+WE=;
        b=Zw7VqXMCkpzpOCnp3EBcdcLBRabCUvZzRIYL2g0jV7a8pkgdFe4/jTdozp4AflDF8t
         EMQJ3gkTnxhE5MEFkfJgNjqU5n5V4D8dGHstL6LdLLSo+6xbnVRrKLAWuquQs4D47seu
         H8yEDJWlLUiJ8xr/8lhPicFgV9RZbQ9xqJ4O8rfFnacUtG/AEePrGo0hgHJ1jQKYnyNu
         jlLB7JFxqP+Bf2p22FWP7joSjmIYb4UbOyzIuDzLuJYgV6YtiT0H1S7lHjsCOFFYqGYx
         YgZxCTl4f2biJ/pAQR0qJFzbf0uFbTVTruq5Tu8u4+Z9U5jIBRO3NK07ObnyPEc7rCeH
         mrIQ==
X-Forwarded-Encrypted: i=1; AJvYcCWVfoDiqpTpK5mYY3IA2uX7nOKNz8ErAE5LQJwATfoGtfuOo374PKQziA8qBQNNwPeilzDqlEEf09q4OUV7e/hhRLWDayeWhSkrqpeKw+4=
X-Gm-Message-State: AOJu0YwQx1Tf71UtFrdGG54pWHGNibT+PjF6xjxfr6z6Gykw6gC8NICD
	h5Kei48tIAVkRrVapvQf0t5apQk1HOr4M+prhrmbyWYyDq0lAuQq8eMgRY6Udw8=
X-Google-Smtp-Source: AGHT+IHGIbDdIaTxFsBrR2IWNVmIfQPPJ+675dXTXWbJ7moWgoCOFX7c+yR+LoYYFbbyaZxuWDMrRA==
X-Received: by 2002:a05:600c:3b26:b0:419:f241:633c with SMTP id 5b1f17b1804b1-41feaa2f3b9mr18911745e9.4.1715338169611;
        Fri, 10 May 2024 03:49:29 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 3/7] hw/xen: Constify xenstore_be::XenDevOps
Date: Fri, 10 May 2024 12:49:04 +0200
Message-ID: <20240510104908.76908-4-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XenDevOps @ops is not updated, mark it const.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen-legacy-backend.h | 2 +-
 hw/xen/xen-legacy-backend.c         | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/hw/xen/xen-legacy-backend.h b/include/hw/xen/xen-legacy-backend.h
index eb22633caa..2b2e43c7e7 100644
--- a/include/hw/xen/xen-legacy-backend.h
+++ b/include/hw/xen/xen-legacy-backend.h
@@ -40,7 +40,7 @@ void xen_be_check_state(struct XenLegacyDevice *xendev);
 
 /* xen backend driver bits */
 void xen_be_init(void);
-int xen_be_register(const char *type, struct XenDevOps *ops);
+int xen_be_register(const char *type, const struct XenDevOps *ops);
 int xen_be_set_state(struct XenLegacyDevice *xendev, enum xenbus_state state);
 int xen_be_bind_evtchn(struct XenLegacyDevice *xendev);
 void xen_be_set_max_grant_refs(struct XenLegacyDevice *xendev,
diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index 35f2b57560..4dc3a561cf 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -520,7 +520,7 @@ void xen_be_check_state(struct XenLegacyDevice *xendev)
 struct xenstore_be {
     const char *type;
     int dom;
-    struct XenDevOps *ops;
+    const struct XenDevOps *ops;
 };
 
 static void xenstore_update_be(void *opaque, const char *watch)
@@ -557,7 +557,7 @@ static void xenstore_update_be(void *opaque, const char *watch)
     }
 }
 
-static int xenstore_scan(const char *type, int dom, struct XenDevOps *ops)
+static int xenstore_scan(const char *type, int dom, const struct XenDevOps *ops)
 {
     struct XenLegacyDevice *xendev;
     char path[XEN_BUFSIZE];
@@ -632,7 +632,7 @@ void xen_be_init(void)
 #endif
 }
 
-int xen_be_register(const char *type, struct XenDevOps *ops)
+int xen_be_register(const char *type, const struct XenDevOps *ops)
 {
     char path[50];
     int rc;
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719764.1122627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Nop-0000Ld-4x; Fri, 10 May 2024 10:49:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719764.1122627; Fri, 10 May 2024 10:49:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Nop-0000LW-1i; Fri, 10 May 2024 10:49:39 +0000
Received: by outflank-mailman (input) for mailman id 719764;
 Fri, 10 May 2024 10:49:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Non-0007sQ-Pf
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:37 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id feb9777f-0eba-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:49:37 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e271acb015so23950491fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:37 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fe004eae9sm42086695e9.1.2024.05.10.03.49.35
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: feb9777f-0eba-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338177; x=1715942977; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4TpUZ8CKDgQBKN9dqD4YnU9Z+7ud1E8NXp6jYp8jowk=;
        b=e7LX8ldC7IbLCg3cad21oVjET7xzBXL/XwYHj5BzLtk6xvTaqAVvBkCUqwCyc5g8CC
         54s9Lhi24PifFYcFAYkyjqvtLWMwhln4Pm+CUrzhaD2TtC9IXkBkgyLnJemN4I9UZBUz
         cPk0A6PNR0C4XawrVtPmoJHs+6LMtpjofob5pEO+HuVw2qYd758w62VI4srl0xlK+TUB
         2LB6BfptbrTCF85lDEiRDUBPEltcjLTVumaBdPCxq2e2tTsmsO1YYxaN92QqhFjyjTYN
         QoB8mNYrzPslogtzzOZAxoGVssrVzgV/XdD8xKg4/t2oJNTf1jhRtIFRUFhIRDXkz5LN
         5q9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338177; x=1715942977;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4TpUZ8CKDgQBKN9dqD4YnU9Z+7ud1E8NXp6jYp8jowk=;
        b=Qdhs6LcdfYlkpxjmz3qNNjSWCnYjfdZoi5UHFKeJKUOQpJ02jFFoBPWoX5kzHx3KnZ
         PCzyJwZtlweHWFM0SwI0+aqMbudBSMrP7610K83ubUbOHS0prqswAST6I5i5/8fKu4ch
         byO8GfKxhUFPW2rLomuNg5s34AMHEi0rMfbazJIq2L0zhPRB5DjmUBye43h0upOkb/d/
         0IP7o23CnpYPqdJ4UZQK2sJwZspyMbbTBWTBpeu7JJtTtHUKO/Y/YerducwdKp9KPbTu
         AHU56Rvs9rCCT0wugLmmkQegEzm043OnqQYBznhMkWRUCnOiuZjnCP3JNbfOxBQCoxgG
         4m4Q==
X-Forwarded-Encrypted: i=1; AJvYcCUvXjuvf9NLUrNDH9XvqOf7zN34lCRIPK6quNDa9ZlnsVv7GuVxFJE2rRXKwPGmB6vEkGLs/JxAhusaJ+ykQjYOcxT4+a/GgoNxT5O80VA=
X-Gm-Message-State: AOJu0YwprW/sBpaPMrKjUE8wIvqaaNxaB1hkean4sHEy5GsPo75Plw0m
	SedqL/gg5bmgxsWEU+WUDIP4wj6M4zlzAaDf88c0OX+sBWEdvrlLVY1KxodKiLY=
X-Google-Smtp-Source: AGHT+IF8RYJPL89SDGvqPQidQK0KvxqcH+BiwQtlAacHw890jlHQhQtuBmxCLpPl4I/7qfyZFdFshA==
X-Received: by 2002:a2e:93cf:0:b0:2e4:9606:6b86 with SMTP id 38308e7fff4ca-2e51fd42c6dmr14616001fa.8.1715338176994;
        Fri, 10 May 2024 03:49:36 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 4/7] hw/xen: Make XenDevOps structures const
Date: Fri, 10 May 2024 12:49:05 +0200
Message-ID: <20240510104908.76908-5-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Keep XenDevOps structures in .rodata.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen-legacy-backend.h | 8 ++++----
 hw/9pfs/xen-9p-backend.c            | 2 +-
 hw/display/xenfb.c                  | 4 ++--
 hw/usb/xen-usb.c                    | 4 ++--
 4 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/hw/xen/xen-legacy-backend.h b/include/hw/xen/xen-legacy-backend.h
index 2b2e43c7e7..cfa43a984a 100644
--- a/include/hw/xen/xen-legacy-backend.h
+++ b/include/hw/xen/xen-legacy-backend.h
@@ -67,13 +67,13 @@ static inline void xen_be_unmap_grant_ref(struct XenLegacyDevice *xendev,
 }
 
 /* actual backend drivers */
-extern struct XenDevOps xen_kbdmouse_ops;     /* xen_framebuffer.c */
-extern struct XenDevOps xen_framebuffer_ops;  /* xen_framebuffer.c */
+extern const struct XenDevOps xen_kbdmouse_ops;     /* xen_framebuffer.c */
+extern const struct XenDevOps xen_framebuffer_ops;  /* xen_framebuffer.c */
 #ifdef CONFIG_VIRTFS
-extern struct XenDevOps xen_9pfs_ops;       /* xen-9p-backend.c        */
+extern const struct XenDevOps xen_9pfs_ops;         /* xen-9p-backend.c  */
 #endif
 #ifdef CONFIG_USB_LIBUSB
-extern struct XenDevOps xen_usb_ops;          /* xen-usb.c         */
+extern const struct XenDevOps xen_usb_ops;          /* xen-usb.c         */
 #endif
 
 /* configuration (aka xenbus setup) */
diff --git a/hw/9pfs/xen-9p-backend.c b/hw/9pfs/xen-9p-backend.c
index 4aa9c8c736..b1780eb819 100644
--- a/hw/9pfs/xen-9p-backend.c
+++ b/hw/9pfs/xen-9p-backend.c
@@ -513,7 +513,7 @@ static void xen_9pfs_alloc(struct XenLegacyDevice *xendev)
     xenstore_write_be_int(xendev, "max-ring-page-order", MAX_RING_ORDER);
 }
 
-struct XenDevOps xen_9pfs_ops = {
+const struct XenDevOps xen_9pfs_ops = {
     .size       = sizeof(Xen9pfsDev),
     .flags      = DEVOPS_FLAG_NEED_GNTDEV,
     .alloc      = xen_9pfs_alloc,
diff --git a/hw/display/xenfb.c b/hw/display/xenfb.c
index b2130a0d70..dd83d0f5a5 100644
--- a/hw/display/xenfb.c
+++ b/hw/display/xenfb.c
@@ -972,7 +972,7 @@ static void fb_event(struct XenLegacyDevice *xendev)
 
 /* -------------------------------------------------------------------- */
 
-struct XenDevOps xen_kbdmouse_ops = {
+const struct XenDevOps xen_kbdmouse_ops = {
     .size       = sizeof(struct XenInput),
     .init       = input_init,
     .initialise = input_initialise,
@@ -981,7 +981,7 @@ struct XenDevOps xen_kbdmouse_ops = {
     .event      = input_event,
 };
 
-struct XenDevOps xen_framebuffer_ops = {
+const struct XenDevOps xen_framebuffer_ops = {
     .size       = sizeof(struct XenFB),
     .init       = fb_init,
     .initialise = fb_initialise,
diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c
index 09ec326aea..95a901e81f 100644
--- a/hw/usb/xen-usb.c
+++ b/hw/usb/xen-usb.c
@@ -1083,7 +1083,7 @@ static void usbback_event(struct XenLegacyDevice *xendev)
     qemu_bh_schedule(usbif->bh);
 }
 
-struct XenDevOps xen_usb_ops = {
+const struct XenDevOps xen_usb_ops = {
     .size            = sizeof(struct usbback_info),
     .flags           = DEVOPS_FLAG_NEED_GNTDEV,
     .init            = usbback_init,
@@ -1102,7 +1102,7 @@ static int usbback_not_supported(void)
     return -EINVAL;
 }
 
-struct XenDevOps xen_usb_ops = {
+const struct XenDevOps xen_usb_ops = {
     .backend_register = usbback_not_supported,
 };
 
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719766.1122636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Now-0000uj-Ft; Fri, 10 May 2024 10:49:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719766.1122636; Fri, 10 May 2024 10:49:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Now-0000ta-CY; Fri, 10 May 2024 10:49:46 +0000
Received: by outflank-mailman (input) for mailman id 719766;
 Fri, 10 May 2024 10:49:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Nou-0007MJ-UD
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:44 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0242e75e-0ebb-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:49:43 +0200 (CEST)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41b79450f78so11940995e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:43 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502baacf52sm4278278f8f.87.2024.05.10.03.49.41
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0242e75e-0ebb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338183; x=1715942983; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=f6xsQqOmikZFwXbyDk6ZpqqheWDHvbw4l/2JG+MVzRA=;
        b=ejok603IdcxQfgqps6+fPbxbUnH+knmZrcZUrCtY6LGZMmtqAJa4d/W2sH9QL7+u9H
         60vtIG3HiCPKSHNFRGaOvdRes/FxcvZffoJuIPAHjdIq1aP0PHVEmDvYLFU+njJFHaSG
         NtM/T0rtxiO1ALRpMV1PMjMJ1wxC3SrvbH96Fou6P2g9BDpX4zpxUD07TpgdDQo8py+9
         BULthWTn3dym+ysKlHx+zmAc7nEY3QXojN5GduB3kjFgBXR/xEIeMQKhU+u1YFXz8vA+
         cYsbdHILhkBgeUYVNMYG2dG4JzWc0POSCSj8l1EJ5WGa9mqom4g1Cyy4GclAFOMM2EZD
         SAUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338183; x=1715942983;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=f6xsQqOmikZFwXbyDk6ZpqqheWDHvbw4l/2JG+MVzRA=;
        b=cd+HwZPpZbOLioH0yayylsGj7bcpX3bdbPh71Ye6DY5at+egpwQh5tpU4ZywhP/1N1
         5a6cfPbbLA4ixgl/6rO/ILcVgP9jZpOm/G8qUxVHrrmKuZrVGJVTdc2ldq7Bl9q39Jfg
         /QLAhgfGSCCT9GmYtpeW7m3YU3CICw3W211d5D0zZ8t9LdGmnx+8flaReglkXmnTOEkO
         ZmsgkKMvLGsLyLyOwTBuUspXG3uAtJn5v3CMZMLj/dPvshxjiL2cZFjyWCuDvIlMGgf/
         ECxL/KbB1+aroUZp14spkMoycVr0S/3jir9OeSpQV805qwtxuTNNaNmevrRj9b2D5GwR
         QLDg==
X-Forwarded-Encrypted: i=1; AJvYcCUhyKXxKfOlfvG04RnPd1HKHR8VVOVW8Qvh9evrPWBOZY0ZGZffNxR0UfHlCYwk0Gz1rT+8QMYff5zz8fqKJt5EIhFY5QU/q1/ZHDwxOBk=
X-Gm-Message-State: AOJu0YyfeC0UG/sPPAf8YwsIpaheXXbGyJ1WkB/YbQanLk0wBL5QJywP
	pPdMjK2mi3dFaWQBqqM/+CEbeCu9Ft1eiRO56Q3m7tggRY6V6xQ6SbZzbo+AIHs=
X-Google-Smtp-Source: AGHT+IH3RnmZXXuxt2IBSPxIon61ivul84ejF6Ul9PK/puKcX3RJUgwhud+k6udV+bWCsrG6K2uZGg==
X-Received: by 2002:a05:600c:490a:b0:41f:dcdd:5631 with SMTP id 5b1f17b1804b1-41feaa30cfamr23776095e9.13.1715338182860;
        Fri, 10 May 2024 03:49:42 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 5/7] hw/xen: initialize legacy backends from xen_bus_init()
Date: Fri, 10 May 2024 12:49:06 +0200
Message-ID: <20240510104908.76908-6-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Paolo Bonzini <pbonzini@redhat.com>

Prepare for moving the calls to xen_be_register() under the
control of xen_bus_init(), using the normal xen_backend_init()
method that is used by the "modern" backends.

This requires the xenstore global variable to be initialized,
which is done by xen_be_init().  To ensure that everything is
ready at the time the xen_backend_init() functions are called,
remove the xen_be_init() function from all the boards and
place it directly in xen_bus_init().

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240509170044.190795-7-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/i386/pc.c              | 1 -
 hw/xen/xen-bus.c          | 4 ++++
 hw/xen/xen-hvm-common.c   | 2 --
 hw/xenpv/xen_machine_pv.c | 5 +----
 4 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 505ea750f4..19f21953b4 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1250,7 +1250,6 @@ void pc_basic_device_init(struct PCMachineState *pcms,
             pci_create_simple(pcms->pcibus, -1, "xen-platform");
         }
         xen_bus_init();
-        xen_be_init();
     }
 #endif
 
diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index fb82cc33e4..95b207ac8b 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -13,6 +13,7 @@
 #include "hw/sysbus.h"
 #include "hw/xen/xen.h"
 #include "hw/xen/xen-backend.h"
+#include "hw/xen/xen-legacy-backend.h" /* xen_be_init() */
 #include "hw/xen/xen-bus.h"
 #include "hw/xen/xen-bus-helper.h"
 #include "monitor/monitor.h"
@@ -329,6 +330,9 @@ static void xen_bus_realize(BusState *bus, Error **errp)
         goto fail;
     }
 
+    /* Initialize legacy backend core & drivers */
+    xen_be_init();
+
     if (xs_node_scanf(xenbus->xsh, XBT_NULL, "", /* domain root node */
                       "domid", NULL, "%u", &domid) == 1) {
         xenbus->backend_id = domid;
diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 1627da7398..2d1b032121 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -872,8 +872,6 @@ void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
 
     xen_bus_init();
 
-    xen_be_init();
-
     return;
 
 err:
diff --git a/hw/xenpv/xen_machine_pv.c b/hw/xenpv/xen_machine_pv.c
index 1130d1a147..b500ce0989 100644
--- a/hw/xenpv/xen_machine_pv.c
+++ b/hw/xenpv/xen_machine_pv.c
@@ -34,8 +34,7 @@ static void xen_init_pv(MachineState *machine)
 {
     setup_xen_backend_ops();
 
-    /* Initialize backend core & drivers */
-    xen_be_init();
+    xen_bus_init();
 
     switch (xen_mode) {
     case XEN_ATTACH:
@@ -60,8 +59,6 @@ static void xen_init_pv(MachineState *machine)
         vga_interface_created = true;
     }
 
-    xen_bus_init();
-
     /* config cleanup hook */
     atexit(xen_config_cleanup);
 }
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719771.1122647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Np1-0001Mg-Nc; Fri, 10 May 2024 10:49:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719771.1122647; Fri, 10 May 2024 10:49:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Np1-0001MZ-Kc; Fri, 10 May 2024 10:49:51 +0000
Received: by outflank-mailman (input) for mailman id 719771;
 Fri, 10 May 2024 10:49:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Np1-0007MJ-3Y
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:51 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05c4b305-0ebb-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 12:49:49 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e538a264f7so7492571fa.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:49 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccce2449sm60037515e9.16.2024.05.10.03.49.47
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05c4b305-0ebb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338188; x=1715942988; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=59LwZUfkdqE1X1uqHmA8fJxVEaLTRUaAXUjy1rLp7lY=;
        b=Sta0JE2ikyWuGJHzkCb628DYKvp0Hcdfrg5T9n6zjT579Subv1/7vE9N54eiw8h6PJ
         SMW7sjJtq7yJUl4Egptv95DlR2b/AV53StOH9In8rf4KVZBbJnQbsW8iXUGOIxsCaLsq
         YWU28a0PikPuueXwzhw1HAqzk8+aL7xvLB8BHXs9wRSgZz1cq0iFbqhlnVH7VRP/WN15
         2sxauccTT+fGUtgCoGXU9L8aIuKW/vnLucfq/JO1+CbvID0OQlZNnfeh+M0LhzolLO0l
         TKrodb0R3Xbjt9B6c9Nv4JWSfEn0xHPzNlih9scZOp1iu+cky1Wl1T5amot0kX1/bSKw
         o1qg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338188; x=1715942988;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=59LwZUfkdqE1X1uqHmA8fJxVEaLTRUaAXUjy1rLp7lY=;
        b=lhvTLEaUya6QhYtFFnGOFraOc36qBeFg7TkavMWz8KhaC74kZuEp64mSz/Vfnk2bWo
         LGpC4P/saFpyWmLrELOKCfHKWXdFRBi/YVqgh6z021idPd9KwAWQfnzPyzpxe6Orzjw0
         aaHMD/6txJSG0WDt7g7JuOOq4rm8zL6OyjoVv0HVxM6u5rsI7SFEwI7aIs/dlid+DToM
         8OIVyxNtQ5n0Owzvvry/KdgEZcA1WwHPILQv7Y+gWtQERz9cWms82wnFRGZr9xsZ0R66
         kf+eGTXDirTAy4/iEJJpCkrdEjJTN8ZPNth2jepzxu9SHIeAJ8gS+07fJs4ik15AUaSC
         iGww==
X-Forwarded-Encrypted: i=1; AJvYcCWSH0d1X77BWvtgRG/VWXEWQRHOckbI7qJHcnQ3rW/q14EYNkcxT9DabnIDsHiy4NHJZkcKs6wQLEAARNnhyI0O0Ut4eVAFwhVmDlYqu3k=
X-Gm-Message-State: AOJu0YwblU9j3oBhSkACnZGNsjuArUtEpACfPpBFcVkGjBXMSH8DpCAP
	jpP69SLwWpxwXL0cLKxxw6JP3631Po7llnHNzD/r24XXP99XwUM0UXOjl5QGNNg=
X-Google-Smtp-Source: AGHT+IHXwM/S059R6UkOB6MRtl349Xb619zHzrqPuRu/IZXAzDzUInG0zBbE08W5hL9UPuwtPHGoJA==
X-Received: by 2002:a2e:9e14:0:b0:2e1:c97b:6f1f with SMTP id 38308e7fff4ca-2e5203a795emr13764691fa.51.1715338188613;
        Fri, 10 May 2024 03:49:48 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 6/7] hw/xen: register legacy backends via xen_backend_init
Date: Fri, 10 May 2024 12:49:07 +0200
Message-ID: <20240510104908.76908-7-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Paolo Bonzini <pbonzini@redhat.com>

It is okay to register legacy backends in the middle of xen_bus_init().
All that the registration does is record the existence of the backend
in xenstore.

This makes it possible to remove them from the build without introducing
undefined symbols in xen_be_init().  It also removes the need for the
backend_register callback, whose only purpose is to avoid registering
nonfunctional backends.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20240509170044.190795-8-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen-legacy-backend.h | 11 ++---------
 include/hw/xen/xen_pvdev.h          |  1 -
 hw/9pfs/xen-9p-backend.c            |  8 +++++++-
 hw/display/xenfb.c                  |  8 +++++++-
 hw/usb/xen-usb.c                    | 14 ++++----------
 hw/xen/xen-legacy-backend.c         | 16 ----------------
 6 files changed, 20 insertions(+), 38 deletions(-)

diff --git a/include/hw/xen/xen-legacy-backend.h b/include/hw/xen/xen-legacy-backend.h
index cfa43a984a..e55a14057f 100644
--- a/include/hw/xen/xen-legacy-backend.h
+++ b/include/hw/xen/xen-legacy-backend.h
@@ -66,15 +66,8 @@ static inline void xen_be_unmap_grant_ref(struct XenLegacyDevice *xendev,
     return xen_be_unmap_grant_refs(xendev, ptr, &ref, 1);
 }
 
-/* actual backend drivers */
-extern const struct XenDevOps xen_kbdmouse_ops;     /* xen_framebuffer.c */
-extern const struct XenDevOps xen_framebuffer_ops;  /* xen_framebuffer.c */
-#ifdef CONFIG_VIRTFS
-extern const struct XenDevOps xen_9pfs_ops;         /* xen-9p-backend.c  */
-#endif
-#ifdef CONFIG_USB_LIBUSB
-extern const struct XenDevOps xen_usb_ops;          /* xen-usb.c         */
-#endif
+/* backend drivers not included in all machines */
+extern const struct XenDevOps xen_framebuffer_ops;  /* xenfb.c */
 
 /* configuration (aka xenbus setup) */
 void xen_config_cleanup(void);
diff --git a/include/hw/xen/xen_pvdev.h b/include/hw/xen/xen_pvdev.h
index fd9a33e6ab..0c98444047 100644
--- a/include/hw/xen/xen_pvdev.h
+++ b/include/hw/xen/xen_pvdev.h
@@ -29,7 +29,6 @@ struct XenDevOps {
                                  const char *node);
     void      (*frontend_changed)(struct XenLegacyDevice *xendev,
                                   const char *node);
-    int       (*backend_register)(void);
 };
 
 struct XenLegacyDevice {
diff --git a/hw/9pfs/xen-9p-backend.c b/hw/9pfs/xen-9p-backend.c
index b1780eb819..79359d911a 100644
--- a/hw/9pfs/xen-9p-backend.c
+++ b/hw/9pfs/xen-9p-backend.c
@@ -513,7 +513,7 @@ static void xen_9pfs_alloc(struct XenLegacyDevice *xendev)
     xenstore_write_be_int(xendev, "max-ring-page-order", MAX_RING_ORDER);
 }
 
-const struct XenDevOps xen_9pfs_ops = {
+static const struct XenDevOps xen_9pfs_ops = {
     .size       = sizeof(Xen9pfsDev),
     .flags      = DEVOPS_FLAG_NEED_GNTDEV,
     .alloc      = xen_9pfs_alloc,
@@ -522,3 +522,9 @@ const struct XenDevOps xen_9pfs_ops = {
     .disconnect = xen_9pfs_disconnect,
     .free       = xen_9pfs_free,
 };
+
+static void xen_9pfs_register_backend(void)
+{
+    xen_be_register("9pfs", &xen_9pfs_ops);
+}
+xen_backend_init(xen_9pfs_register_backend);
diff --git a/hw/display/xenfb.c b/hw/display/xenfb.c
index dd83d0f5a5..b6d370bdf6 100644
--- a/hw/display/xenfb.c
+++ b/hw/display/xenfb.c
@@ -972,7 +972,7 @@ static void fb_event(struct XenLegacyDevice *xendev)
 
 /* -------------------------------------------------------------------- */
 
-const struct XenDevOps xen_kbdmouse_ops = {
+static const struct XenDevOps xen_kbdmouse_ops = {
     .size       = sizeof(struct XenInput),
     .init       = input_init,
     .initialise = input_initialise,
@@ -995,3 +995,9 @@ static const GraphicHwOps xenfb_ops = {
     .gfx_update  = xenfb_update,
     .ui_info     = xenfb_ui_info,
 };
+
+static void xen_vkbd_register_backend(void)
+{
+    xen_be_register("vkbd", &xen_kbdmouse_ops);
+}
+xen_backend_init(xen_vkbd_register_backend);
diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c
index 95a901e81f..13901625c0 100644
--- a/hw/usb/xen-usb.c
+++ b/hw/usb/xen-usb.c
@@ -1083,7 +1083,7 @@ static void usbback_event(struct XenLegacyDevice *xendev)
     qemu_bh_schedule(usbif->bh);
 }
 
-const struct XenDevOps xen_usb_ops = {
+static const struct XenDevOps xen_usb_ops = {
     .size            = sizeof(struct usbback_info),
     .flags           = DEVOPS_FLAG_NEED_GNTDEV,
     .init            = usbback_init,
@@ -1095,15 +1095,9 @@ const struct XenDevOps xen_usb_ops = {
     .event           = usbback_event,
 };
 
-#else /* USBIF_SHORT_NOT_OK */
-
-static int usbback_not_supported(void)
+static void xen_usb_register_backend(void)
 {
-    return -EINVAL;
+    xen_be_register("qusb", &xen_usb_ops);
 }
-
-const struct XenDevOps xen_usb_ops = {
-    .backend_register = usbback_not_supported,
-};
-
+xen_backend_init(xen_usb_register_backend);
 #endif
diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index 4dc3a561cf..5514184f9c 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -622,27 +622,11 @@ void xen_be_init(void)
     qbus_set_bus_hotplug_handler(xen_sysbus);
 
     xen_set_dynamic_sysbus();
-
-    xen_be_register("vkbd", &xen_kbdmouse_ops);
-#ifdef CONFIG_VIRTFS
-    xen_be_register("9pfs", &xen_9pfs_ops);
-#endif
-#ifdef CONFIG_USB_LIBUSB
-    xen_be_register("qusb", &xen_usb_ops);
-#endif
 }
 
 int xen_be_register(const char *type, const struct XenDevOps *ops)
 {
     char path[50];
-    int rc;
-
-    if (ops->backend_register) {
-        rc = ops->backend_register();
-        if (rc) {
-            return rc;
-        }
-    }
 
     snprintf(path, sizeof(path), "device-model/%u/backends/%s", xen_domid,
              type);
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:49:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:49:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719773.1122657 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Np6-0001k6-Uo; Fri, 10 May 2024 10:49:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719773.1122657; Fri, 10 May 2024 10:49:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Np6-0001jx-Rj; Fri, 10 May 2024 10:49:56 +0000
Received: by outflank-mailman (input) for mailman id 719773;
 Fri, 10 May 2024 10:49:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Np5-0007sQ-FV
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:49:55 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 092f1925-0ebb-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:49:54 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-350513d2c6aso461525f8f.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:49:54 -0700 (PDT)
Received: from m1x-phil.lan (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbbfecsm4215282f8f.105.2024.05.10.03.49.52
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 10 May 2024 03:49:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 092f1925-0ebb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338194; x=1715942994; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4MMieI22tvUHRez0ASZq1ieLRFlQN49cydbbEGQRCXw=;
        b=IvOZucE8hH8TOcDSfuzDd82e9OeYpCLk4a8VDxP+3ybBISDaV33EnKG1lmBj4cmWV4
         u92uqVsh+MASZuM6GUrXRh9SJM4dyDiod5bFjPcMfoTL82szKrNn08kUYaiy/B+Iq4NO
         s+h/4FaPmdvQfPQ06JgKeRN8FpC94Li6fB0U7bWpYcsNOgqjuDyw49EIPC2zBAJdV7Yl
         1ezIzr952wrqZNv5qP9FsH14r0tauQf+h8nH0MAk8OI24xxgLQQ46mjZe/FMKWtY0IXh
         n6G60PaIVvefTSGhLF+f4/JoNmuybSvKnMZ1qoybm4l5bIe0JvwPoxdsTfqhGdzA5Dfx
         6WDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338194; x=1715942994;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4MMieI22tvUHRez0ASZq1ieLRFlQN49cydbbEGQRCXw=;
        b=Gz7Z5ESAqEJWgPL8QbejG/75CdDwGlxSDET06Meov0CAb0G837HeQrbMU6AcIuddoh
         0ibHDreKTsdJa49JEXDJ/dOA50hg5Td2nUToVcbBQuAb2/lXUzb6CMQ2AfmBjPRw1yD9
         1qOs+iKEnVuoOayAWXZbiXPqqWc3Q7N/VPex9DjRhuDA73zFaOQ7d1S3DtNR9WEayL2K
         DdtyIYWgxvmmQtackt25IfTVDAOaiNvd4UMwJ1fOuEXDgW1EpnV4jyMDQHS4yWLaUmfS
         fpPWX8hNxC4+V/pt1nj3acESuDPBlVuhf5ZwhP+6dxcwcVeEakZwp886Rv1VozER3dH+
         3BKQ==
X-Forwarded-Encrypted: i=1; AJvYcCX2Ajy/OYd20dAqtzDhnnA4mnlKkYIM95pJb4EhL5T+w668AOXLXs1eFWRzhhEli8Txa0pXlNcBTz67urgRWMvqJNRB1177p4TxFwO72IQ=
X-Gm-Message-State: AOJu0Yy5oJ5QvTkviDG0hqTG98Bcu/fstJe40a1lCtU+Iz5fIGI219It
	ww7qibY365sc+ui1TlcXGYESk09WdZiSmQPA87vt3SGQNco36clewFwGOV3nEig=
X-Google-Smtp-Source: AGHT+IEewNU5yU5UDos2eDjrq2/6gYromkISrbJeR/i31/cfP+aw0nvt1M9WGWO7AigSVj0qKxcq5A==
X-Received: by 2002:adf:ef09:0:b0:34c:fd72:f308 with SMTP id ffacd0b85a97d-3504aa69de0mr1652917f8f.71.1715338194433;
        Fri, 10 May 2024 03:49:54 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
To: qemu-devel@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eduardo Habkost <eduardo@habkost.net>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>
Subject: [PATCH v2 7/7] hw/xen: Register framebuffer backend via xen_backend_init()
Date: Fri, 10 May 2024 12:49:08 +0200
Message-ID: <20240510104908.76908-8-philmd@linaro.org>
X-Mailer: git-send-email 2.41.0
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
References: <20240510104908.76908-1-philmd@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Align the framebuffer backend with the other legacy ones,
register it via xen_backend_init() when '-vga xenfb' is
used. It is safe because MODULE_INIT_XEN_BACKEND is called
in xen_bus_realize(), long after CLI processing initialized
the vga_interface_type variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/xen/xen-legacy-backend.h | 3 ---
 hw/display/xenfb.c                  | 9 +++++++--
 hw/xenpv/xen_machine_pv.c           | 2 --
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/hw/xen/xen-legacy-backend.h b/include/hw/xen/xen-legacy-backend.h
index e55a14057f..943732b8d1 100644
--- a/include/hw/xen/xen-legacy-backend.h
+++ b/include/hw/xen/xen-legacy-backend.h
@@ -66,9 +66,6 @@ static inline void xen_be_unmap_grant_ref(struct XenLegacyDevice *xendev,
     return xen_be_unmap_grant_refs(xendev, ptr, &ref, 1);
 }
 
-/* backend drivers not included in all machines */
-extern const struct XenDevOps xen_framebuffer_ops;  /* xenfb.c */
-
 /* configuration (aka xenbus setup) */
 void xen_config_cleanup(void);
 int xen_config_dev_vfb(int vdev, const char *type);
diff --git a/hw/display/xenfb.c b/hw/display/xenfb.c
index b6d370bdf6..ff442ced1a 100644
--- a/hw/display/xenfb.c
+++ b/hw/display/xenfb.c
@@ -29,6 +29,7 @@
 
 #include "ui/input.h"
 #include "ui/console.h"
+#include "sysemu/sysemu.h"
 #include "hw/xen/xen-legacy-backend.h"
 
 #include "hw/xen/interface/io/fbif.h"
@@ -996,8 +997,12 @@ static const GraphicHwOps xenfb_ops = {
     .ui_info     = xenfb_ui_info,
 };
 
-static void xen_vkbd_register_backend(void)
+static void xen_ui_register_backend(void)
 {
     xen_be_register("vkbd", &xen_kbdmouse_ops);
+
+    if (vga_interface_type == VGA_XENFB) {
+        xen_be_register("vfb", &xen_framebuffer_ops);
+    }
 }
-xen_backend_init(xen_vkbd_register_backend);
+xen_backend_init(xen_ui_register_backend);
diff --git a/hw/xenpv/xen_machine_pv.c b/hw/xenpv/xen_machine_pv.c
index b500ce0989..24395f42cb 100644
--- a/hw/xenpv/xen_machine_pv.c
+++ b/hw/xenpv/xen_machine_pv.c
@@ -50,8 +50,6 @@ static void xen_init_pv(MachineState *machine)
         break;
     }
 
-    xen_be_register("vfb", &xen_framebuffer_ops);
-
     /* configure framebuffer */
     if (vga_interface_type == VGA_XENFB) {
         xen_config_dev_vfb(0, "vnc");
-- 
2.41.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 10:52:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 10:52:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719780.1122666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Nrr-00044K-Bs; Fri, 10 May 2024 10:52:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719780.1122666; Fri, 10 May 2024 10:52:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Nrr-00044D-9E; Fri, 10 May 2024 10:52:47 +0000
Received: by outflank-mailman (input) for mailman id 719780;
 Fri, 10 May 2024 10:52:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+Xy6=MN=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1s5Nrq-000442-2j
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 10:52:46 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ef39b32-0ebb-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 12:52:45 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41b21ed19f5so13128435e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 03:52:45 -0700 (PDT)
Received: from [192.168.69.100] (sev93-h02-176-184-17-152.dsl.sta.abo.bbox.fr.
 [176.184.17.152]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f87d200c6sm93588085e9.23.2024.05.10.03.52.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 03:52:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ef39b32-0ebb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715338365; x=1715943165; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=f7sIFTUuD+KOnWU1gakeh0NFMU6Pja/uxomvMgU9Skw=;
        b=X9TJJv7tV4BtLmuH4ldVqzUyE6DGrh3VTu25Fqm8M410QPXs3orVZBF7IMkAlA8fj1
         m6s5GXIn1e6vrQ9k05S1PQCTETQfj2Q9bnLYKmNNi/o4ArbeGOL95A23o62RFYURBhm9
         yVzyD/z9sWiNPN577W5lnjfw9D2vdGcin/nafzJXuCjog3EM3+VUmOKfhT5v0kJCAfLI
         ju1AkygOrt/oMOI8eLgddMGFabFZXjCPRQnfSUoErJv0rJ5f++NlLYl4/kAGCsDqwZHF
         jmnDtvxYNQNG4PmYjoKaCje+M3SLH+9Q8r4X4971VKUFhXO8Q4DRj0DbXMi8MN6sV2wE
         e5mw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715338365; x=1715943165;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=f7sIFTUuD+KOnWU1gakeh0NFMU6Pja/uxomvMgU9Skw=;
        b=rPvI0SwXiwjrC/PTYkdWZBQ1UQ1So/3+SoRDXtNJXbt/66dtcDaLvXbm/GBoS30I+9
         kUtksBE6Y9PcJCIY/PffyjbYtQSlZBm0VAScLt+fs2iLSKm+1AGRKOpm9NsHoEVpfE6F
         5qzXLOmxZLH16+xwz8h6sWSu67pqZyD8w05Va0koSTPd1DjRHOUevCeSWOLWBy81b2AG
         QZRqqA8W7rS1OMA0ZRI/PS/mLEgXokMFpj4VfOH+FGhsWeZ1L0rMg/crKdUrFxAana7z
         QxJOyXg3sNjdCx/p4TOo4bkUaNVoBfalnoj/nu/jggHc0mVhE+I6qOAl03QvQEDYRAjS
         8sRw==
X-Forwarded-Encrypted: i=1; AJvYcCUKbDHF/stjaQ/BL798tndfq3nj0ByaP/5O89Thyo58BmjnD3Bw38/8+qgRy0tQQigPT9x+8SLOfwuWiO7p7ptm+F2RkhsU5tb/Fyzp8GA=
X-Gm-Message-State: AOJu0Yxpp3cTke7Trgu2DuX4LFRVpXIERLojZUdk3FEeaZHvGYlVgOvR
	ftaGL0HM2IofOOeyscPj6ZRmZJq2g6IVybjawr/VS8sX12JWerOH7GmxrJxFswA=
X-Google-Smtp-Source: AGHT+IHuuJHQzBXjTwqPYG6Xft9Ercp9HGQcE/jtvUNQuTmVyJXtzh7vd29oX2VK0Fk1fhEyp1ZyvQ==
X-Received: by 2002:a05:600c:1e13:b0:41b:3e4e:bd99 with SMTP id 5b1f17b1804b1-41feaa390c4mr19037495e9.12.1715338365250;
        Fri, 10 May 2024 03:52:45 -0700 (PDT)
Message-ID: <84c89274-f8a9-4f86-84e9-40a962dc4f77@linaro.org>
Date: Fri, 10 May 2024 12:52:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/7] hw/xen: register legacy backends via
 xen_backend_init
To: qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-7-philmd@linaro.org>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240510104908.76908-7-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/5/24 12:49, Philippe Mathieu-Daudé wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
> 
> It is okay to register legacy backends in the middle of xen_bus_init().
> All that the registration does is record the existence of the backend
> in xenstore.
> 
> This makes it possible to remove them from the build without introducing
> undefined symbols in xen_be_init().  It also removes the need for the
> backend_register callback, whose only purpose is to avoid registering
> nonfunctional backends.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-ID: <20240509170044.190795-8-pbonzini@redhat.com>

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 11 ++---------
>   include/hw/xen/xen_pvdev.h          |  1 -
>   hw/9pfs/xen-9p-backend.c            |  8 +++++++-
>   hw/display/xenfb.c                  |  8 +++++++-
>   hw/usb/xen-usb.c                    | 14 ++++----------
>   hw/xen/xen-legacy-backend.c         | 16 ----------------
>   6 files changed, 20 insertions(+), 38 deletions(-)




From xen-devel-bounces@lists.xenproject.org Fri May 10 11:27:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 11:27:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719805.1122678 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5OPf-0005Rm-30; Fri, 10 May 2024 11:27:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719805.1122678; Fri, 10 May 2024 11:27:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5OPe-0005Rf-Uh; Fri, 10 May 2024 11:27:42 +0000
Received: by outflank-mailman (input) for mailman id 719805;
 Fri, 10 May 2024 11:27:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5OPd-0005QZ-6c
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 11:27:41 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e966a06-0ec0-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 13:27:38 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so425129366b.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 04:27:38 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea651asm1706341a12.11.2024.05.10.04.27.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 04:27:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e966a06-0ec0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715340458; x=1715945258; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zNXwSOWeBMueOtABnUo69wqjco2rwqNtJ/nGjlKDC3g=;
        b=LYsY7xMpdOo5RJNnFRf67hMbFY8du/3v+bJKMGXsNT4OxZBRDdH1M14g5mDLizG6+3
         39bcWn4NUsoTBmUxBEy21XiZLiYownv6fh0SlL16+sbpJwWG97mtrZBTaI53nvlwxwMn
         m6oyuanbxq3q9K8D6NxrSviKF/k1JpP3sUa2Wy2SzvHbyNyR2H7Vcg0enofpxBneSejw
         paVeCU1rINFaJ+9RkNTRLBcRKjiS6cpXbKIhGvGaSkEI2O3K7PGnNO/2Ih1Wkm0UNzDg
         cQOhsn8A+K8dyVwn0tyn6ZpdgMVlc7xzRHfMym94/5EaN2XpIQuOLuiljo8ydmvLl/CR
         VPuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715340458; x=1715945258;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zNXwSOWeBMueOtABnUo69wqjco2rwqNtJ/nGjlKDC3g=;
        b=nO1gpvTxEZXErCZroIAhcMMQgKt+SWJgfROpMXekP4hW5b9EHK0paa2vveMvka36LD
         SlMH8QIB+V6z1WAvllLSGkOE8sWIukFhPNDJvw9yMGSQJl7ZuQy2Vy2HlMRu6zXl3GfV
         G8lIrXqwnXTh6n40jsSNFouvmY1UgxgnraoSmOwKeEb4BV7APX9WH5fut+D6JrfoClh5
         +UA5cm1A0zomuhUjLt9iCh+nI750kxkxMAQeSvLaiPoos/vNrHy5oecKu+Uia8zYHzpm
         nx1dII+Me0954KvOzR11wp2YshJKRzX81SIY/y9YEfWLquIoUB0WK+D3Tz1RzH2mdLX2
         0LeA==
X-Forwarded-Encrypted: i=1; AJvYcCUMXGYvbHsC1zXLL3QCCGIDu+3nYmcaH3RI6u9oVxLDaMoF0ekAyDYGczHDkw5MoNcdad3nRgN8An8cBQ1Yr03mZElseJzWBZ1sTdOwLEQ=
X-Gm-Message-State: AOJu0YzgUEGMHZbUQUdkGasw7KnkE1OX27lCCNsVKJ1BB2QODkegec+j
	VtWSSEL/GmwdNV/JtxjtIoy07m2VAXJtEyRFhTBiCBKaCo5qJuSqEtmjsVDvnZ8=
X-Google-Smtp-Source: AGHT+IGWH9NtKldOHxG5Jn85QZLrAtp/ZGMY4ac6yPg5kx3Hs4lPh9R3ZW28bnofErwGNFhMNfHAYA==
X-Received: by 2002:a50:d583:0:b0:572:637b:c7e1 with SMTP id 4fb4d7f45d1cf-5734d5f48b1mr2421443a12.21.1715340458312;
        Fri, 10 May 2024 04:27:38 -0700 (PDT)
Message-ID: <0aac68ac-cf40-4c3d-ac02-95b9a37aaa11@suse.com>
Date: Fri, 10 May 2024 13:27:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
 <rafael@kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
 "Huang, Ray" <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
 <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <d0b5e7d5-3503-49be-9fa3-4b79c62059ca@suse.com>
 <BL1PR12MB5849F1DE8B4A3538C79CE5D3E7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <BL1PR12MB5849F1DE8B4A3538C79CE5D3E7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10.05.24 12:32, Chen, Jiqian wrote:
> On 2024/5/10 18:21, Jürgen Groß wrote:
>> On 10.05.24 12:13, Chen, Jiqian wrote:
>>> On 2024/5/10 17:53, Jürgen Groß wrote:
>>>> On 10.05.24 11:06, Chen, Jiqian wrote:
>>>>> Hi,
>>>>>
>>>>> On 2024/5/10 14:46, Jürgen Groß wrote:
>>>>>> On 19.04.24 05:36, Jiqian Chen wrote:
>>>>>>> +
>>>>>>> +    info->type = IRQT_PIRQ;
>>>>> I am considering whether I need to use a new type(like IRQT_GSI) here to distinguish with IRQT_PIRQ, because function restore_pirqs will process all IRQT_PIRQ.
>>>>
>>>> restore_pirqs() already considers gsi == 0 to be not GSI related. Isn't this
>>>> enough?
>>> No, it is not enough.
>>> xen_pvh_add_gsi_irq_map adds the mapping of gsi and irq, but the value of gsi is not 0,
>>> once restore_pirqs is called, it will do PHYSDEVOP_map_pirq for that gsi, but in pvh dom0, we shouldn't do PHYSDEVOP_map_pirq.
>>
>> Okay, then add a new flag to info->u.pirq.flags for that purpose?
> I feel like adding "new flag to info->u.pirq.flags" is not as good as adding " new type to info->type".
> Because in restore_pirqs, it considers " info->type != IRQT_PIRQ", if adding " new flag to info->u.pirq.flags", we need to add a new condition in restore_pirqs.
> And actually this mapping(gsi and irq of pvh) doesn't have pirq, so it is not suitable to add to u.pirq.flags.

Does this mean there is no other IRQT_PIRQ related activity relevant for those
GSIs/IRQs? In that case I agree to add IRQT_GSI.


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 10 11:31:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 11:31:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719812.1122686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5OTB-0007VH-G6; Fri, 10 May 2024 11:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719812.1122686; Fri, 10 May 2024 11:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5OTB-0007VA-DL; Fri, 10 May 2024 11:31:21 +0000
Received: by outflank-mailman (input) for mailman id 719812;
 Fri, 10 May 2024 11:31:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/KIo=MN=gmail.com=rob.townley@srs-se1.protection.inumbo.net>)
 id 1s5OTB-0007V4-4R
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 11:31:21 +0000
Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com
 [2607:f8b0:4864:20::112a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d23bc2b9-0ec0-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 13:31:20 +0200 (CEST)
Received: by mail-yw1-x112a.google.com with SMTP id
 00721157ae682-61b4cbb8834so18273337b3.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 04:31:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d23bc2b9-0ec0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715340679; x=1715945479; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=COlgQ18n1H1PLCXLn6irjwUuzTBnZ5prI1L/oP0ZVF0=;
        b=T4zPQQCTBY/n4qddlzMNBWKuPb+lC1qPrlBgOMJuUW35LmY1uD554cYEDtw7uZAMC7
         qiW5N3dwMpQqKnA4OrMJPdcpPqUTrMDFgtju0BBPJGdGy9w14fKFE8rMsuB2J/ue48cP
         OvJI8R+iuiC8o6hIH57MalLi84LrfsgO8gf5LkgbTIOYOhMmBnerH8f87iG0BoRr3jqM
         dgrr2/PrgyrEMa0z3ibKKsRD0jc7p/wsML2eeP9yqpZO5TxLONW6+JvQFxWmeYssjzDb
         oM5X88h7nlQoFdqlGAzOb65I6HDj4dez/9bnueuHjL+wKzuo9no+6PnFKxtty7D5EuBX
         ozdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715340679; x=1715945479;
        h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=COlgQ18n1H1PLCXLn6irjwUuzTBnZ5prI1L/oP0ZVF0=;
        b=AvNW555FPp6YtlfsQpBeN5MOvpsQevACjrQ25ZZo3BUdHwv+zgK12GInCbYOIMf+Sw
         49x7SEaDuP1qbE82Vak6aLeWTT27gWhqsZbkb8725F8YnuSxCaCHGuUk0uqQts3yeuG/
         l8sqIA+8VexVkF5fmxmQnIb5+OgeJSvLfbS8mhd+zteE6ufX/LOJqwQuKL1Illt0J4tS
         CDU92nCMj/PPA/Rw8rgLthb8TDXdipUC5uRwsJegV1EcOycLE4qyzhXzn700RBwABSBY
         E0VXbQ+Ol4UwO2DGPLdW+eAlO0HNuCQjGCe4gGTXMLAHxPezlD1QqrNngE4N0HDrquA1
         aYMQ==
X-Gm-Message-State: AOJu0Yy8h9KGm2VsNAmpffyTlgbau8P+Ihl3J/gGlR7SWvmpAPxzcv5q
	ZTSH8w5Na/GE2KxkpX8xnbIiblDUN9s4ioic2bJWP1OuCqx6W/o+TWKR23Ly1krM/L01CQZdVQF
	GhwdP0sfFY2wYb/mnFM0RfYp6gvBTzA==
X-Google-Smtp-Source: AGHT+IGSwq5uGfBHUD9Y4mLh4ft6n74c/JL5173FK9cj0MifOk2Xwk3V/d5ASh938nUbqp1MC3Hg4eTWBsXSn5mzhxM=
X-Received: by 2002:a81:49ca:0:b0:61a:e4f3:555c with SMTP id
 00721157ae682-622aff9027fmr25243147b3.8.1715340679030; Fri, 10 May 2024
 04:31:19 -0700 (PDT)
MIME-Version: 1.0
References: <Zj3kXlHJzlqaFpMw@dingwall.me.uk>
In-Reply-To: <Zj3kXlHJzlqaFpMw@dingwall.me.uk>
Reply-To: Rob.Townley@gmail.com
From: Rob Townley <rob.townley@gmail.com>
Date: Fri, 10 May 2024 06:31:08 -0500
Message-ID: <CA+VdTb9CCBOfq3orob5XxD6VXuy7CXAKizgry+MMm=FkJs1wJw@mail.gmail.com>
Subject: Re: Domain Birth Time
To: James Dingwall <james-xen@dingwall.me.uk>
Cc: xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000d7b319061817df65"

--000000000000d7b319061817df65
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 10, 2024 at 4:10=E2=80=AFAM James Dingwall <james-xen@dingwall.=
me.uk>
wrote:

> Hi,
>
> We've added a feature to Xen 4.15 such that `xl uptime -b` reports the
> birth
> time of the domain (i.e. a value preserved across migrations).  If this
> would
> be of wider interest I can try porting this to a more recent release and
> submitting it for review.
>
> Regards,
> James
>
> Awesome!  birth date time and last modification are fundamental to about
all data usefulness.

--000000000000d7b319061817df65
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div>On Fri, May 10, 2024 at 4:10=E2=80=AFAM James Dingwall &lt;<a href=3D"=
mailto:james-xen@dingwall.me.uk">james-xen@dingwall.me.uk</a>&gt; wrote:<br=
></div><div><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<b=
r>
<br>
We&#39;ve added a feature to Xen 4.15 such that `xl uptime -b` reports the =
birth<br>
time of the domain (i.e. a value preserved across migrations).=C2=A0 If thi=
s would<br>
be of wider interest I can try porting this to a more recent release and<br=
>
submitting it for review.<br>
<br>
Regards,<br>
James<br>
<br>
</blockquote></div></div>Awesome! =C2=A0birth date time and last modificati=
on are fundamental to about all data usefulness.=C2=A0<div dir=3D"auto"><br=
></div>

--000000000000d7b319061817df65--


From xen-devel-bounces@lists.xenproject.org Fri May 10 12:29:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 12:29:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719852.1122696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5PN1-0004EG-Ua; Fri, 10 May 2024 12:29:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719852.1122696; Fri, 10 May 2024 12:29:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5PN1-0004E9-S2; Fri, 10 May 2024 12:29:03 +0000
Received: by outflank-mailman (input) for mailman id 719852;
 Fri, 10 May 2024 12:29:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+D8H=MN=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s5PN0-0004E3-VM
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 12:29:03 +0000
Received: from mail-vs1-xe32.google.com (mail-vs1-xe32.google.com
 [2607:f8b0:4864:20::e32])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e0d64e2b-0ec8-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 14:29:00 +0200 (CEST)
Received: by mail-vs1-xe32.google.com with SMTP id
 ada2fe7eead31-47ef11b1a31so824057137.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 05:29:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df5513017sm20748731cf.46.2024.05.10.05.28.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 05:28:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0d64e2b-0ec8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715344139; x=1715948939; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=zVZagDSfufW5sKWfE9/POvo7VWOMtlKaVnyuvAnZeFo=;
        b=kenX9gKtQR7SMshhzldQhqU86xLBIVMtz+OzeUtVDqWZlw3MVLdAth2JW4RDO4KCym
         AqgYxKuTmYyKOQJcVMRlK7wrOB9N24pW1VprSWQ2cZSI4nuuaHyNZI5hA3HctgmpeXWG
         c/xjfNtIhvuoUBqFyY9p9OrqHCcgggztkbVug=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715344139; x=1715948939;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zVZagDSfufW5sKWfE9/POvo7VWOMtlKaVnyuvAnZeFo=;
        b=eu+t2xuaAQYuxQjlFLASDNTAkDjCkb3ulm1hA8QqxMBzbFnSOgx/Bv408+DJCn/GgJ
         uKSJpIs9Hknel6BON5Jm5FJMvhRd7Jrp7JPq5WZWGktyGpMuWgUykdpesAgr2dqdXSsh
         6eHbbgwfhNdRwyf0edaBJNsOAmYWg6TkmKNTjZVQq5EflGTYFn71bgchw26h5BkFkiM2
         8dIp/2YjIEe7hQRCoxu+zsnv4LJkuIQa3E368COfiQskOFym8HZ3Oko0AnT8wOhxnZB4
         Qx2UG1QydpVlp19/EAqrgEeFBRngbr8rhOfHZXtXMwfZuzXJQGM5OWa+GreaaOwzbt3Z
         nozA==
X-Gm-Message-State: AOJu0YxIg2Nf5WQZGfPzlrhrYJK2F7ZXLHIH1mOidscTuA9AumL3EyG1
	hL8eJYuAVA9XWnlzxslpUlhwo8elKsSEirHodG0ZDgFpN6v8c07J4aMU3XLy0iwocPDGgmGIuOH
	7
X-Google-Smtp-Source: AGHT+IEGu544bcBvZtqjfCZMb4mtH8Ju2ZiAI5Zv/cCa6ulyaanyEFIeYF1ckbJvq8OXer+Rx5xXZQ==
X-Received: by 2002:a05:6102:41a8:b0:47b:6fb1:5299 with SMTP id ada2fe7eead31-48077e03423mr2609268137.14.1715344137600;
        Fri, 10 May 2024 05:28:57 -0700 (PDT)
Date: Fri, 10 May 2024 14:28:55 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: Re: [PATCH for-4.19 v2 3/3] xen/x86: remove foreign mappings from
 the p2m on teardown
Message-ID: <Zj4TB9iXpwFkQoLE@macbook>
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-4-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240508112323.38946-4-roger.pau@citrix.com>

On Wed, May 08, 2024 at 01:23:23PM +0200, Roger Pau Monne wrote:
> Iterate over the p2m up to the maximum recorded gfn and remove any foreign
> mappings, in order to drop the underlying page references and thus don't keep
> extra page references if a domain is destroyed while still having foreign
> mappings on it's p2m.
> 
> The logic is similar to the one used on Arm.
> 
> Note that foreign mappings cannot be created by guests that have altp2m or
> nested HVM enabled, as p2ms different than the host one are not currently
> scrubbed when destroyed in order to drop references to any foreign maps.
> 
> It's unclear whether the right solution is to take an extra reference when
> foreign maps are added to p2ms different than the host one, or just rely on the
> host p2m already having a reference.  The mapping being removed from the host
> p2m should cause it to be dropped on all domain p2ms.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - Use existing p2m max_mapped_pfn field.
>  - Prevent creating foreign mappings by guests that have altp2m or nestedhvm
>    enabled.
> ---
>  CHANGELOG.md                   |  1 +
>  xen/arch/x86/domain.c          |  8 +++-
>  xen/arch/x86/include/asm/p2m.h | 26 +++++++------
>  xen/arch/x86/mm/p2m-basic.c    | 17 +++++++++
>  xen/arch/x86/mm/p2m.c          | 68 ++++++++++++++++++++++++++++++++--
>  5 files changed, 103 insertions(+), 17 deletions(-)
> 
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 8041cfb7d243..09bdb9b97578 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
>     - HVM PIRQs are disabled by default.
>     - Reduce IOMMU setup time for hardware domain.
>   - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
> + - Allow HVM/PVH domains to map foreign pages.
>  
>  ### Added
>   - On x86:
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index dff790060605..1cb3ccddab00 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -718,7 +718,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }
>  
> -    if ( altp2m && (altp2m & (altp2m - 1)) )
> +    if ( altp2m & (altp2m - 1) )
>      {
>          dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
>                  config->flags);
> @@ -2387,6 +2387,7 @@ int domain_relinquish_resources(struct domain *d)
>          enum {
>              PROG_iommu_pagetables = 1,
>              PROG_shared,
> +            PROG_mappings,
>              PROG_paging,
>              PROG_vcpu_pagetables,
>              PROG_xen,
> @@ -2435,6 +2436,11 @@ int domain_relinquish_resources(struct domain *d)
>          }
>  #endif
>  
> +    PROGRESS(mappings):
> +        ret = relinquish_p2m_mapping(d);
> +        if ( ret )
> +            return ret;
> +
>      PROGRESS(paging):
>  
>          /* Tear down paging-assistance stuff. */
> diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
> index 107b9f260848..c1478ffc3647 100644
> --- a/xen/arch/x86/include/asm/p2m.h
> +++ b/xen/arch/x86/include/asm/p2m.h
> @@ -383,6 +383,8 @@ struct p2m_domain {
>  
>      /* Number of foreign mappings. */
>      unsigned long      nr_foreign;
> +    /* Cursor for iterating over the p2m on teardown. */
> +    unsigned long      teardown_gfn;
>  #endif /* CONFIG_HVM */
>  };
>  
> @@ -395,16 +397,7 @@ struct p2m_domain {
>  #endif
>  #include <xen/p2m-common.h>
>  
> -static inline bool arch_acquire_resource_check(struct domain *d)
> -{
> -    /*
> -     * FIXME: Until foreign pages inserted into the P2M are properly
> -     * reference counted, it is unsafe to allow mapping of
> -     * resource pages unless the caller is the hardware domain
> -     * (see set_foreign_p2m_entry()).
> -     */
> -    return !paging_mode_translate(d) || is_hardware_domain(d);

This must be:

    return is_pv_domain(d) ||
           (d->arch.hvm.params[HVM_PARAM_ALTP2M] == XEN_ALTP2M_disabled &&
            !nestedhvm_enabled(d));

Sorry.


From xen-devel-bounces@lists.xenproject.org Fri May 10 12:32:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 12:32:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719856.1122706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5PQS-0005eG-Cu; Fri, 10 May 2024 12:32:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719856.1122706; Fri, 10 May 2024 12:32:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5PQS-0005e9-9g; Fri, 10 May 2024 12:32:36 +0000
Received: by outflank-mailman (input) for mailman id 719856;
 Fri, 10 May 2024 12:32:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=oMhx=MN=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s5PQQ-0005e3-RC
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 12:32:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5f767244-0ec9-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 14:32:32 +0200 (CEST)
Received: from LAPTOP-EFA9O91E.localdomain
 (host-87-7-28-64.retail.telecomitalia.it [87.7.28.64])
 by support.bugseng.com (Postfix) with ESMTPSA id AFE114EE073C;
 Fri, 10 May 2024 14:32:31 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f767244-0ec9-11ef-b4bb-af5377834399
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [XEN PATCH v3] arm/mem_access: add conditional build of mem_access.c
Date: Fri, 10 May 2024 14:32:11 +0200
Message-Id: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
changes are done:
revert preprocessor conditional changes to xen/mem_access.h which
had it build unconditionally, add conditional build for xen/mem_access.c
as well and provide stubs in asm/mem_access.h for the users of this
header.

Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
---
Changes from v2:
Stylistic changes to code aimed to respect xen's coding guidelines.
---
Changes from v1:
Reverted preprocessor conditional changes to xen/mem_access.h;
added conditional build for xen/mem_access.c;
provided stubs for asm/mem_access.h functions
---
 xen/arch/arm/Makefile                 |  2 +-
 xen/arch/arm/include/asm/mem_access.h | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 7b1350e2ef..45dc29ea53 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -37,7 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
 obj-y += irq.o
 obj-y += kernel.init.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o
-obj-y += mem_access.o
+obj-$(CONFIG_MEM_ACCESS) += mem_access.o
 obj-y += mm.o
 obj-y += monitor.o
 obj-y += p2m.o
diff --git a/xen/arch/arm/include/asm/mem_access.h b/xen/arch/arm/include/asm/mem_access.h
index 35ed0ad154..abac8032fc 100644
--- a/xen/arch/arm/include/asm/mem_access.h
+++ b/xen/arch/arm/include/asm/mem_access.h
@@ -17,6 +17,8 @@
 #ifndef _ASM_ARM_MEM_ACCESS_H
 #define _ASM_ARM_MEM_ACCESS_H
 
+#include <xen/types.h>
+
 static inline
 bool p2m_mem_access_emulate_check(struct vcpu *v,
                                   const struct vm_event_st *rsp)
@@ -35,12 +37,28 @@ static inline bool p2m_mem_access_sanity_check(struct domain *d)
  * Send mem event based on the access. Boolean return value indicates if trap
  * needs to be injected into guest.
  */
+#ifdef CONFIG_MEM_ACCESS
 bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec);
 
 struct page_info*
 p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
                                   const struct vcpu *v);
+#else
+
+static inline bool
+p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec)
+{
+    return false;
+}
+
+static inline struct page_info*
+p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
+                                  const struct vcpu *v)
+{
+    return NULL;
+}
 
+#endif /*CONFIG_MEM_ACCESS*/
 #endif /* _ASM_ARM_MEM_ACCESS_H */
 
 /*
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 10 12:50:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 12:50:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719864.1122716 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Pi4-0001av-Qk; Fri, 10 May 2024 12:50:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719864.1122716; Fri, 10 May 2024 12:50:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Pi4-0001ao-O6; Fri, 10 May 2024 12:50:48 +0000
Received: by outflank-mailman (input) for mailman id 719864;
 Fri, 10 May 2024 12:50:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+D8H=MN=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s5Pi3-0001ad-SA
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 12:50:47 +0000
Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com
 [2607:f8b0:4864:20::f31])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eb6c1511-0ecb-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 14:50:46 +0200 (CEST)
Received: by mail-qv1-xf31.google.com with SMTP id
 6a1803df08f44-6a073f10e8eso8098416d6.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 05:50:46 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f195018sm17144226d6.74.2024.05.10.05.50.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 05:50:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb6c1511-0ecb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715345445; x=1715950245; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=mdAQL6LWUhcHd+2TKy01L2FyMVtjyGGs4TMnOgqrMqU=;
        b=IcRehdOAu2vk84PoQP+HmqLn5Yoao4CQBH1GB6Yu49QvM/OKvros10XRrZr9r1HRGO
         WzIEJO/RQxD7kX4fVm3zjADXznOnGFfFbyUjwgS+7kHAwe8kUDZgWoEg8MWwR0ELrjqt
         W2NH6nMctJ/KiaABQUEYaSMbRxnVKECNC+XgM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715345445; x=1715950245;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mdAQL6LWUhcHd+2TKy01L2FyMVtjyGGs4TMnOgqrMqU=;
        b=i2FOaTyAG7EQ7t76MCKC5VUycCBDORccWQVTGr+x3nw9iofhv9n/b24E9m3xiDJoLa
         0mwePhYHFbafp9CtZn27qELu1s3zL1ZB9RWQqDVxWeEWtSeMc3XEevkAjqQn6+TFWHOl
         9SATQwQXCVQW80IWxrWpSN05yesutgcbLTjb18RlWNOAuHoHUKtIpQ92uZ/EIGBqUzFm
         0bo48RrYHGWHtziDrEpV/k1IxeJC7PZwzsI8zxLi8b0q9POjFGkYMNlwflOE4PxgmFE8
         bmwyvMopuISyIPNILc8prx2OHOTL9j8ajgGHVVNfRc9K58SCEH7o79AC+s0sRJBZZsY1
         GxDg==
X-Gm-Message-State: AOJu0YxkwZtdq9uZuBoyimmFyQaOU3uo4lJmMkcsjQcVfC1/AqGrdMQk
	CsDjWBmjI1IyfqDQE/c9gzwOwlT0mMPPHHoplSvYmN6e2SmNhtlGeiZNOR27a+TMkbln0Sk4ni7
	q
X-Google-Smtp-Source: AGHT+IEYrRMn6RljjSD/DkKmMC+boUrUg/gIAaIUrqGnt1xLnLi86R/xTvCv6GEu4L6bi+EBeV+EuA==
X-Received: by 2002:a05:6214:3906:b0:6a0:ab13:e058 with SMTP id 6a1803df08f44-6a168160125mr25512216d6.16.1715345444981;
        Fri, 10 May 2024 05:50:44 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	"Oleksii K ." <oleksii.kurochko@gmail.com>,
	Arthur Borsboom <arthurborsboom@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH for-4.19] libxl: fix population of the online vCPU bitmap for PVH
Date: Fri, 10 May 2024 14:49:13 +0200
Message-ID: <20240510124913.49945-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

libxl passes some information to libacpi to create the ACPI table for a PVH
guest, and among that information it's a bitmap of which vCPUs are online
which can be less than the maximum number of vCPUs assigned to the domain.

While the population of the bitmap is done correctly for HVM based on the
number of online vCPUs, for PVH the population of the bitmap is done based on
the number of maximum vCPUs allowed.  This leads to all local APIC entries in
the MADT being set as enabled, which contradicts the data in xenstore if vCPUs
is different than maximum vCPUs.

Fix by copying the internal libxl bitmap that's populated based on the vCPUs
parameter.

Reported-by: Arthur Borsboom <arthurborsboom@gmail.com>
Link: https://gitlab.com/libvirt/libvirt/-/issues/399
Reported-by: Leigh Brown <leigh@solinno.co.uk>
Fixes: 14c0d328da2b ('libxl/acpi: Build ACPI tables for HVMlite guests')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Note that the setup of hvm_info_table could be shared between PVH and HVM, as
the fields are very limited, see hvm_build_set_params() for the HVM side.
However this late in the release it's safer to just adjust the PVH path.

Also note the checksum is not provided when hvm_info_table is built for PVH.
This is fine so far because such checksum is only consumed by hvmloader and not
libacpi itself.

It's a bugfix, so should be considered for 4.19.
---
 tools/libs/light/libxl_x86_acpi.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 620f3c700c3e..5cf261bd6794 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -89,7 +89,7 @@ static int init_acpi_config(libxl__gc *gc,
     uint32_t domid = dom->guest_domid;
     xc_domaininfo_t info;
     struct hvm_info_table *hvminfo;
-    int i, r, rc;
+    int r, rc;
 
     config->dsdt_anycpu = config->dsdt_15cpu = dsdt_pvh;
     config->dsdt_anycpu_len = config->dsdt_15cpu_len = dsdt_pvh_len;
@@ -138,8 +138,8 @@ static int init_acpi_config(libxl__gc *gc,
         hvminfo->nr_vcpus = info.max_vcpu_id + 1;
     }
 
-    for (i = 0; i < hvminfo->nr_vcpus; i++)
-        hvminfo->vcpu_online[i / 8] |= 1 << (i & 7);
+    memcpy(hvminfo->vcpu_online, b_info->avail_vcpus.map,
+           b_info->avail_vcpus.size);
 
     config->hvminfo = hvminfo;
 
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 10 13:15:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 13:15:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719873.1122726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Q6L-0006LQ-M5; Fri, 10 May 2024 13:15:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719873.1122726; Fri, 10 May 2024 13:15:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Q6L-0006LJ-JU; Fri, 10 May 2024 13:15:53 +0000
Received: by outflank-mailman (input) for mailman id 719873;
 Fri, 10 May 2024 13:15:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=E7ZJ=MN=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s5Q6J-0006Gl-Po
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 13:15:52 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6b325ff0-0ecf-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 15:15:49 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id A54C280057;
 Fri, 10 May 2024 14:15:48 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b325ff0-0ecf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715346948;
	bh=siIHVV1CqvBrMRJJb5ymjMh0hJqBbTR8oSZSgRT6bkI=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=kgT68tdgdiC1XsBtR9pZKYg2EU/AXaez7k0hqI7+9In+aPpj32HJkBz1/byujPaoi
	 +62eoNouNZFuyJR9huZsi/KwG+zH454efElBgnHAqBQrzSUgXj3uj1gdIHLSZl44Ts
	 o7uYJZwyN12LGURkUKHCD4DQkQg7IgXwyjwKH/Os=
MIME-Version: 1.0
Date: Fri, 10 May 2024 14:15:48 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Roger Pau Monne <roger.pau@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Oleksii K ." <oleksii.kurochko@gmail.com>, Arthur Borsboom
 <arthurborsboom@gmail.com>
Subject: Re: [PATCH for-4.19] libxl: fix population of the online vCPU bitmap
 for PVH
In-Reply-To: <20240510124913.49945-1-roger.pau@citrix.com>
References: <20240510124913.49945-1-roger.pau@citrix.com>
Message-ID: <c285e23db3b50dbc10a90abc979dc0dc@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Roger,

Thanks for responding and fixing this so quickly.

On 2024-05-10 13:49, Roger Pau Monne wrote:
> libxl passes some information to libacpi to create the ACPI table for a 
> PVH
> guest, and among that information it's a bitmap of which vCPUs are 
> online
> which can be less than the maximum number of vCPUs assigned to the 
> domain.
> 
> While the population of the bitmap is done correctly for HVM based on 
> the
> number of online vCPUs, for PVH the population of the bitmap is done 
> based on
> the number of maximum vCPUs allowed.  This leads to all local APIC 
> entries in
> the MADT being set as enabled, which contradicts the data in xenstore 
> if vCPUs
> is different than maximum vCPUs.
> 
> Fix by copying the internal libxl bitmap that's populated based on the 
> vCPUs
> parameter.
> 
> Reported-by: Arthur Borsboom <arthurborsboom@gmail.com>
> Link: https://gitlab.com/libvirt/libvirt/-/issues/399
> Reported-by: Leigh Brown <leigh@solinno.co.uk>
> Fixes: 14c0d328da2b ('libxl/acpi: Build ACPI tables for HVMlite 
> guests')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Note that the setup of hvm_info_table could be shared between PVH and 
> HVM, as
> the fields are very limited, see hvm_build_set_params() for the HVM 
> side.
> However this late in the release it's safer to just adjust the PVH 
> path.
> 
> Also note the checksum is not provided when hvm_info_table is built for 
> PVH.
> This is fine so far because such checksum is only consumed by hvmloader 
> and not
> libacpi itself.
> 
> It's a bugfix, so should be considered for 4.19.
> ---
>  tools/libs/light/libxl_x86_acpi.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_x86_acpi.c 
> b/tools/libs/light/libxl_x86_acpi.c
> index 620f3c700c3e..5cf261bd6794 100644
> --- a/tools/libs/light/libxl_x86_acpi.c
> +++ b/tools/libs/light/libxl_x86_acpi.c
> @@ -89,7 +89,7 @@ static int init_acpi_config(libxl__gc *gc,
>      uint32_t domid = dom->guest_domid;
>      xc_domaininfo_t info;
>      struct hvm_info_table *hvminfo;
> -    int i, r, rc;
> +    int r, rc;
> 
>      config->dsdt_anycpu = config->dsdt_15cpu = dsdt_pvh;
>      config->dsdt_anycpu_len = config->dsdt_15cpu_len = dsdt_pvh_len;
> @@ -138,8 +138,8 @@ static int init_acpi_config(libxl__gc *gc,
>          hvminfo->nr_vcpus = info.max_vcpu_id + 1;
>      }
> 
> -    for (i = 0; i < hvminfo->nr_vcpus; i++)
> -        hvminfo->vcpu_online[i / 8] |= 1 << (i & 7);
> +    memcpy(hvminfo->vcpu_online, b_info->avail_vcpus.map,
> +           b_info->avail_vcpus.size);
> 
>      config->hvminfo = hvminfo;

Tested-by: Leigh Brown <leigh@solinno.co.uk>

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Fri May 10 13:54:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 13:54:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719884.1122737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5QhE-0004Nd-Gx; Fri, 10 May 2024 13:54:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719884.1122737; Fri, 10 May 2024 13:54:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5QhE-0004NW-DI; Fri, 10 May 2024 13:54:00 +0000
Received: by outflank-mailman (input) for mailman id 719884;
 Fri, 10 May 2024 13:53:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tag6=MN=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s5QhD-0004NA-PQ
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 13:53:59 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id becd9878-0ed4-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 15:53:57 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-573137ba8d7so5677960a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 06:53:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: becd9878-0ed4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715349236; x=1715954036; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2k5X2b8fVQEMzVfPhF0Bw+AAwKodrKYcFlsJMkzPLzQ=;
        b=baFJFWf172MlBnFqhUncqOsXXQ3r6pVOVgbjZC2JPFpvCriT1B4mTkAKFhUFMvZcML
         Exg0O3T90G6SWCSHYWhAtLMDIZ5omu1LYV0HF4iKDUNNKqvt+On27ZIPmlkKji24YQmx
         7axXNYLU5M0pee1/P0l5uaBKROiGRTA5uKCqEXziqOT8vUvUy385dAvYuWAhEx1U29oJ
         SmTlCrMj4amLy1EIekI0XAWfL0s+WlF/6tuhC8gxXHAeXQnYhcMBuOXT/Uo9MHULJ1hY
         52gYASKnDAk6fyhHlgHjkp661W7HyZT58Xul9byGhjkZK+f/aME2+yY7ade9TFO0CgFi
         cTRQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715349236; x=1715954036;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2k5X2b8fVQEMzVfPhF0Bw+AAwKodrKYcFlsJMkzPLzQ=;
        b=L6pW157RTuBow0OQNlPGrgn2Hs8smw47cozV7erZXsEUtOIQdzuA1M1pDt/7P0B6mv
         CU7+0F/t0oJTLTlRjgpvL6K035a8MeeBYbieSRXMnDKy4mvrKP/QJEcVc+ViGxhF5CoM
         MYO9yOnEXEQjm627svtSmtwoMtU+OgBwJrCU/2ZKgUNUqVIjrX0UO94ZsHQVDJAkZIVN
         gLRfQm4pwKbWLwwt+d54n1weZH7GTMaHhS3oVDi51xwfUTvnLqzBYDh+yPjBqs9kUlqU
         EHH4QC7VRh1LU7xG9HPM8qjPRLKzrh95IPPfTAynpGWOGwKo+Xf9Z7fPQBIRmgy4fmMR
         jJWg==
X-Forwarded-Encrypted: i=1; AJvYcCXkyO+0vVInh7hFbJNA41HMFsL7G4nh1VuuYUpcyain1efVAUEn0cAsQBJNskkf0o8Nc98eSzE2fm7IvSiA6ByB7o2EZ/a9t8UoS+av1s4=
X-Gm-Message-State: AOJu0YwS5xpTD5BycX/zQX773GsDT/NMJWJSSTbgtxfj8cyATmp6xZrb
	6hKeDW4F3yvTKIXENV+gHf9ENXlN+cLc7kwFFvu4h3oEAww+MrkyemJxEpQ46kw2HiIHrqy9/HC
	BJOMH91v2sWElM4HtBgC0PeJsEU4=
X-Google-Smtp-Source: AGHT+IECqtPzKLt+Qs2LGB+mYMQbUATaTPIssKDGvU8AIjnQg1zjSG41jBZnLiN3PKaIy8n1hpzRoXWF7VFT3jat9vM=
X-Received: by 2002:a05:6402:2071:b0:572:aaaf:c349 with SMTP id
 4fb4d7f45d1cf-573327883b9mr4778566a12.11.1715349236261; Fri, 10 May 2024
 06:53:56 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
 <2d89f70c625c3abd24bd57f2ee0d2f94@solinno.co.uk>
In-Reply-To: <2d89f70c625c3abd24bd57f2ee0d2f94@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Fri, 10 May 2024 09:53:44 -0400
Message-ID: <CAKf6xpvJupdPatge_FLApUeKHMFCUOpzrENX8kpHd-BSpSU3=Q@mail.gmail.com>
Subject: Re: [RFC PATCH v2 0/5] Add bridge VLAN support
To: Leigh Brown <leigh@solinno.co.uk>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org, 
	Oleksii Kurochko <oleksii.kurochko@gmail.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Juergen Gross <jgross@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 9, 2024 at 12:10=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> w=
rote:
>
> Hi Andrew,
>
> On 2024-05-09 16:53, Andrew Cooper wrote:
> > On 08/05/2024 10:38 pm, Leigh Brown wrote:
> > This is past the last-post date, so Oleksii will need to decide whether
> > he's happy to make an exception for it.
>
>  From my own perspective, I know this is an enhancement and am more than
> happy to maintain it locally for the time being, so no problem if it has
> to wait.  Still happy for any feedback though :-)
>
> > Anthony is OoO for a month now, so if this is to get in for 4.19, then
> > it will need reviewing by others.  I've CC'd a few plausible
> > candidates...

I'll try to review them in the next few days.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Fri May 10 14:16:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 14:16:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719896.1122746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5R3D-0008In-7i; Fri, 10 May 2024 14:16:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719896.1122746; Fri, 10 May 2024 14:16:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5R3D-0008Ig-4n; Fri, 10 May 2024 14:16:43 +0000
Received: by outflank-mailman (input) for mailman id 719896;
 Fri, 10 May 2024 14:16:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YvKo=MN=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s5R3B-0008IT-SM
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 14:16:41 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ebc649d1-0ed7-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 16:16:41 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C01136723C;
 Fri, 10 May 2024 14:16:39 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7AC0A1386E;
 Fri, 10 May 2024 14:16:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id blCCHEcsPmZXFQAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 10 May 2024 14:16:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebc649d1-0ed7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715350599; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=prPdvlgPTP2uaA20U/fjxR/6B0ZmtRktXaxuzxFsYp8=;
	b=CKsjsZVJUZ/SAcbYDZ/1BP7B+D5P2qzcOFzQKNRjwLCynmDtHeoCVkiY7m8NPupa+Fb5n8
	UtydbJe6Uedmsj/f8SC24FXeGILWKOU/cHJ0yyKc5AlrjbA8LAjVIJ0MUlYoseoWai4wg+
	UZF5B8GjnAeN5vqOmvH0JWMD/4OoVA0=
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=CKsjsZVJ
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715350599; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=prPdvlgPTP2uaA20U/fjxR/6B0ZmtRktXaxuzxFsYp8=;
	b=CKsjsZVJUZ/SAcbYDZ/1BP7B+D5P2qzcOFzQKNRjwLCynmDtHeoCVkiY7m8NPupa+Fb5n8
	UtydbJe6Uedmsj/f8SC24FXeGILWKOU/cHJ0yyKc5AlrjbA8LAjVIJ0MUlYoseoWai4wg+
	UZF5B8GjnAeN5vqOmvH0JWMD/4OoVA0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v7] xen: allow up to 16383 cpus
Date: Fri, 10 May 2024 16:16:36 +0200
Message-Id: <20240510141636.20543-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Rspamd-Action: no action
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Level: **
X-Spamd-Result: default: False [2.29 / 50.00];
	SEM_URIBL(3.50)[changelog.md:url];
	BAYES_HAM(-3.00)[100.00%];
	SUSPICIOUS_RECIPS(1.50)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	MID_CONTAINS_FROM(1.00)[];
	R_MISSING_CHARSET(0.50)[];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	BAD_REP_POLICIES(0.10)[];
	MIME_GOOD(-0.10)[text/plain];
	MX_GOOD(-0.01)[];
	R_DKIM_ALLOW(0.00)[suse.com:s=susede1];
	MIME_TRACE(0.00)[0:+];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	ARC_NA(0.00)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	FREEMAIL_CC(0.00)[suse.com,gmail.com,xenproject.org,citrix.com,xen.org,kernel.org];
	DKIM_TRACE(0.00)[suse.com:+];
	TO_DN_SOME(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	FROM_HAS_DN(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim,suse.com:email];
	RCPT_COUNT_SEVEN(0.00)[9];
	DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	TAGGED_RCPT(0.00)[];
	FREEMAIL_ENVRCPT(0.00)[gmail.com]
X-Spam-Score: 2.29
X-Spam-Flag: NO
X-Rspamd-Queue-Id: C01136723C
X-Spamd-Bar: ++

With lock handling now allowing up to 16384 cpus (spinlocks can handle
65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit for
the number of cpus to be configured to 16383.

The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.

Add a support limit of physical CPUs to SUPPORT.md (4096 on x86, 128
on ARM).

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
V5:
- new patch (Jan Beulich)
V7:
- add SUPPORT.md and CHANGELOG.md entries
---
 CHANGELOG.md     | 2 ++
 SUPPORT.md       | 5 +++++
 xen/arch/Kconfig | 2 +-
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8041cfb7d2..c43c45d8d4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    - HVM PIRQs are disabled by default.
    - Reduce IOMMU setup time for hardware domain.
  - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
+ - Increase the maximum number of CPUs Xen can be built for from 4095 to
+   16383.
 
 ### Added
  - On x86:
diff --git a/SUPPORT.md b/SUPPORT.md
index e10d46d924..d5d60c62ec 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -61,6 +61,11 @@ For the Cortex A77 r0p0 - r1p0, see Errata 1508412.
 
     Status, x86: Supported
 
+### Physical CPUs
+
+    Status, x86: Supported up to 4096
+    Status, ARM: Supported up to 128
+
 ### Host ACPI (via Domain 0)
 
     Status, x86 PV: Supported
diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
index 67ba38f32f..308ce129a8 100644
--- a/xen/arch/Kconfig
+++ b/xen/arch/Kconfig
@@ -6,7 +6,7 @@ config PHYS_ADDR_T_32
 
 config NR_CPUS
 	int "Maximum number of CPUs"
-	range 1 4095
+	range 1 16383
 	default "256" if X86
 	default "8" if ARM && RCAR3
 	default "4" if ARM && QEMU
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri May 10 14:23:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 14:23:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719911.1122784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5R9V-0001lb-9G; Fri, 10 May 2024 14:23:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719911.1122784; Fri, 10 May 2024 14:23:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5R9V-0001lT-6Y; Fri, 10 May 2024 14:23:13 +0000
Received: by outflank-mailman (input) for mailman id 719911;
 Fri, 10 May 2024 14:23:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5R9U-0001lB-RQ; Fri, 10 May 2024 14:23:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5R9U-00071I-Pr; Fri, 10 May 2024 14:23:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5R9U-0006E7-Cs; Fri, 10 May 2024 14:23:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5R9U-0005Ia-CK; Fri, 10 May 2024 14:23:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=G4Sc+dxXv9GdycVd6uYaiJ6ZspTVRCYDceSupl33qPk=; b=4iU/BAV7BDx4qrb3R7nmhO17cM
	wDbWXe3v4QZE8hq1Sc1XTiZfwfrP5d//6Cqh3qZ+NEZcQ6G8I+764kJDhIeIplRmYvZ/ptUkhlkAK
	y+yZ8IqH71oZd+8Gr54xnmuHP/akWNbho/NsGmI1CVI0aie81rBXoiarIhPKLzvTClqc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185961-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185961: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=53959cb8309919fc2f157a1c99e0af2ce280cb84
X-Osstest-Versions-That:
    xen=a2330b51df267e20e66bbba6c5bf08f0570ed58b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 14:23:12 +0000

flight 185961 xen-unstable real [real]
flight 185969 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185961/
http://logs.test-lab.xenproject.org/osstest/logs/185969/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl           8 xen-boot            fail pass in 185969-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 185969 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 185969 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185950
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185950
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  53959cb8309919fc2f157a1c99e0af2ce280cb84
baseline version:
 xen                  a2330b51df267e20e66bbba6c5bf08f0570ed58b

Last test of basis   185950  2024-05-09 01:51:55 Z    1 days
Failing since        185955  2024-05-09 14:10:38 Z    1 days    2 attempts
Testing same since   185961  2024-05-09 22:36:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Federico Serafini <federico.serafini@bugseng.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a2330b51df..53959cb830  53959cb8309919fc2f157a1c99e0af2ce280cb84 -> master


From xen-devel-bounces@lists.xenproject.org Fri May 10 14:50:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 14:50:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719933.1122847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Ra0-0007oq-00; Fri, 10 May 2024 14:50:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719933.1122847; Fri, 10 May 2024 14:50:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5RZz-0007oj-T0; Fri, 10 May 2024 14:50:35 +0000
Received: by outflank-mailman (input) for mailman id 719933;
 Fri, 10 May 2024 14:50:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+D8H=MN=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s5RZy-0007oZ-Vz
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 14:50:35 +0000
Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com
 [2607:f8b0:4864:20::72c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a701a6e4-0edc-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 16:50:33 +0200 (CEST)
Received: by mail-qk1-x72c.google.com with SMTP id
 af79cd13be357-792b934de39so165529585a.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 07:50:33 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf27519esm187109685a.1.2024.05.10.07.50.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 07:50:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a701a6e4-0edc-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715352632; x=1715957432; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=rcf9dyx3znNaO7fh0N6bAZdnABe5MNxooRIJMetNFKg=;
        b=TAHqfM6m7YO/pf9v9kKAnb4Tm6zfjkpyoucjb4w34nmbk/eSdmQtUre/db1oxgZCkx
         4S6B3yP/VFEez45+bKBga1iXFbA2W78DHYxrCZSx6OfLSH81w1g/VA/aHpxmY/1olpbb
         8pizSaJ1OeKTN4XoTdwaNGu4ipNNjBaXRqN44=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715352632; x=1715957432;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rcf9dyx3znNaO7fh0N6bAZdnABe5MNxooRIJMetNFKg=;
        b=rF4qautRH6sqkkqzBQIS35PgP4L22vVbgtLh3LDZcFX01h0WcGmTvL7Nul4bY8OaBN
         Z/YArmECCvXSLlxPAGmcD99pmTlU5XGnBDSvcnx+sIijBMm+fC/8okAq370g7A8N0sBs
         D80nENSa5dVOXfPBDHbf1JCACW+x7C66fW1tw3AnWtdI1oq9jjIkLIj7BDQqOYQVWRji
         lulwl6MdjTyts9YEhEOuyhBOdxb9PcGuUL2mqTXlq9YPdFZOI95oMQZqi62AVFxwH62F
         Qe9vWPqVyuCdSrl+4lrWnOsg1AGyTXlUUQomvjA9DguR+ooCgtY3QhB5QxntJyTan797
         96hQ==
X-Gm-Message-State: AOJu0Yyfnjt9PtLrE6A7s1/aoW/N1RPQZm9NEBrJQlf07uLoTw3d07HU
	VAKZeCEHqo1pNgv/KfIShzV0mCUz50SstZR7wVgEonXZRcjJzWv8qJPgOzZKgdZAERM7q+J2lla
	4
X-Google-Smtp-Source: AGHT+IE9xdJTA0JhX1yaBpXCpcH+cOXsY5DXrU7ZmmhvxrZTsP5FSAbdwRTwJjbGRrHPSCNKZaWUlQ==
X-Received: by 2002:a05:620a:4958:b0:792:bada:c0a5 with SMTP id af79cd13be357-792c7574a97mr293716885a.16.1715352632197;
        Fri, 10 May 2024 07:50:32 -0700 (PDT)
Date: Fri, 10 May 2024 16:50:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v8 2/6] x86/hvm: Allow access to registers on the same
 page as MSI-X table
Message-ID: <Zj40NvNqlJWFjcCb@macbook>
References: <cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com>
 <fbe01c945d75409406ac0b02bc17d44f57a39ccf.1715313192.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <fbe01c945d75409406ac0b02bc17d44f57a39ccf.1715313192.git-series.marmarek@invisiblethingslab.com>

On Fri, May 10, 2024 at 05:53:22AM +0200, Marek Marczykowski-Górecki wrote:
> Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers
> on the same page as MSI-X table. Device model (especially one in
> stubdomain) cannot really handle those, as direct writes to that page is
> refused (page is on the mmio_ro_ranges list). Instead, extend
> msixtbl_mmio_ops to handle such accesses too.
> 
> Doing this, requires correlating read/write location with guest
> MSI-X table address. Since QEMU doesn't map MSI-X table to the guest,
> it requires msixtbl_entry->gtable, which is HVM-only. Similar feature
> for PV would need to be done separately.
> 
> This will be also used to read Pending Bit Array, if it lives on the same
> page, making QEMU not needing /dev/mem access at all (especially helpful
> with lockdown enabled in dom0). If PBA lives on another page, QEMU will
> map it to the guest directly.
> If PBA lives on the same page, discard writes and log a message.
> Technically, writes outside of PBA could be allowed, but at this moment
> the precise location of PBA isn't saved, and also no known device abuses
> the spec in this way (at least yet).
> 
> To access those registers, msixtbl_mmio_ops need the relevant page
> mapped. MSI handling already has infrastructure for that, using fixmap,
> so try to map first/last page of the MSI-X table (if necessary) and save
> their fixmap indexes. Note that msix_get_fixmap() does reference
> counting and reuses existing mapping, so just call it directly, even if
> the page was mapped before. Also, it uses a specific range of fixmap
> indexes which doesn't include 0, so use 0 as default ("not mapped")
> value - which simplifies code a bit.
> 
> Based on assumption that all MSI-X page accesses are handled by Xen, do
> not forward adjacent accesses to other hypothetical ioreq servers, even
> if the access wasn't handled for some reason (failure to map pages etc).
> Relevant places log a message about that already.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Just one typo in a logged message, and one comment re the return type
of  adjacent_{read,write}().

> ---
> Changes in v8:
> - rename adjacent_handle to get_adjacent_idx
> - put SBDF at the start of error messages
> - use 0 for ADJACENT_DONT_HANDLE (it's FIX_RESERVED)
> - merge conditions in msixtbl_range into one "if"
> - add assert for address alignment
> - change back to setting pval to ~0UL at the start of adjacent_read
> Changes in v7:
> - simplify logic based on assumption that all access to MSI-X pages are
>   handled by Xen (Roger)
> - move calling adjacent_handle() into adjacent_{read,write}() (Roger)
> - move range check into msixtbl_addr_to_desc() (Roger)
> - fix off-by-one when initializing adj_access_idx[ADJ_IDX_LAST] (Roger)
> - no longer distinguish between unhandled write due to PBA nearby and
>   other reasons
> - add missing break after ASSERT_UNREACHABLE (Jan)
> Changes in v6:
> - use MSIX_CHECK_WARN macro
> - extend assert on fixmap_idx
> - add break in default label, after ASSERT_UNREACHABLE(), and move
>   setting default there
> - style fixes
> Changes in v5:
> - style fixes
> - include GCC version in the commit message
> - warn only once (per domain, per device) about failed adjacent access
> Changes in v4:
> - drop same_page parameter of msixtbl_find_entry(), distinguish two
>   cases in relevant callers
> - rename adj_access_table_idx to adj_access_idx
> - code style fixes
> - drop alignment check in adjacent_{read,write}() - all callers already
>   have it earlier
> - delay mapping first/last MSI-X pages until preparing device for a
>   passthrough
> v3:
>  - merge handling into msixtbl_mmio_ops
>  - extend commit message
> v2:
>  - adjust commit message
>  - pass struct domain to msixtbl_page_handler_get_hwaddr()
>  - reduce local variables used only once
>  - log a warning if write is forbidden if MSI-X and PBA lives on the same
>    page
>  - do not passthrough unaligned accesses
>  - handle accesses both before and after MSI-X table
> ---
>  xen/arch/x86/hvm/vmsi.c        | 208 ++++++++++++++++++++++++++++++++--
>  xen/arch/x86/include/asm/msi.h |   5 +-
>  xen/arch/x86/msi.c             |  42 +++++++-
>  3 files changed, 245 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> index 999917983789..d506d6adaaf6 100644
> --- a/xen/arch/x86/hvm/vmsi.c
> +++ b/xen/arch/x86/hvm/vmsi.c
> @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d)
>      return d->arch.hvm.msixtbl_list.next;
>  }
>  
> +/*
> + * Lookup an msixtbl_entry on the same page as given addr. It's up to the
> + * caller to check if address is strictly part of the table - if relevant.
> + */
>  static struct msixtbl_entry *msixtbl_find_entry(
>      struct vcpu *v, unsigned long addr)
>  {
> @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry(
>      struct domain *d = v->domain;
>  
>      list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list )
> -        if ( addr >= entry->gtable &&
> -             addr < entry->gtable + entry->table_len )
> +        if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) &&
> +             PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) )
>              return entry;
>  
>      return NULL;
> @@ -203,6 +207,10 @@ static struct msi_desc *msixtbl_addr_to_desc(
>      if ( !entry || !entry->pdev )
>          return NULL;
>  
> +    if ( addr < entry->gtable ||
> +         addr >= entry->gtable + entry->table_len )
> +        return NULL;
> +
>      nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
>  
>      list_for_each_entry( desc, &entry->pdev->msi_list, list )
> @@ -213,6 +221,157 @@ static struct msi_desc *msixtbl_addr_to_desc(
>      return NULL;
>  }
>  
> +/*
> + * Returns:
> + *  - 0 (FIX_RESERVED) if no handling should be done
> + *  - a fixmap idx to use for handling
> + */
> +static unsigned int get_adjacent_idx(
> +    const struct msixtbl_entry *entry, unsigned long addr, bool write)
> +{
> +    unsigned int adj_type;
> +    struct arch_msix *msix;
> +
> +    if ( !entry || !entry->pdev )
> +    {
> +        ASSERT_UNREACHABLE();
> +        return 0;
> +    }
> +
> +    if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable )
> +        adj_type = ADJ_IDX_FIRST;
> +    else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) &&
> +              addr >= entry->gtable + entry->table_len )
> +        adj_type = ADJ_IDX_LAST;
> +    else
> +    {
> +        /* All callers should already do equivalent range checking. */
> +        ASSERT_UNREACHABLE();
> +        return 0;
> +    }
> +
> +    msix = entry->pdev->msix;
> +    if ( !msix )
> +    {
> +        ASSERT_UNREACHABLE();
> +        return 0;
> +    }
> +
> +    if ( !msix->adj_access_idx[adj_type] )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_not_initialized) )
> +            gprintk(XENLOG_WARNING,
> +                    "%pp: Page for adjacent(%d) MSI-X table access not initialized (addr %#lx, gtable %#lx))\n",
                                                                                                             ^ extra )
> +                    &entry->pdev->sbdf, adj_type, addr, entry->gtable);
> +        return 0;
> +    }
> +
> +    /* If PBA lives on the same page too, discard writes. */
> +    if ( write &&
> +         ((adj_type == ADJ_IDX_LAST &&
> +           msix->table.last == msix->pba.first) ||
> +          (adj_type == ADJ_IDX_FIRST &&
> +           msix->table.first == msix->pba.last)) )
> +    {
> +        if ( MSIX_CHECK_WARN(msix, entry->pdev->domain->domain_id,
> +                             adjacent_pba) )
> +            gprintk(XENLOG_WARNING,
> +                    "%pp: MSI-X table and PBA share a page, "
> +                    "discard write to adjacent memory (%#lx)\n",
> +                    &entry->pdev->sbdf, addr);
> +        return 0;
> +    }
> +
> +    return msix->adj_access_idx[adj_type];
> +}
> +
> +static int adjacent_read(
> +    const struct msixtbl_entry *entry,
> +    paddr_t address, unsigned int len, uint64_t *pval)
> +{
> +    const void __iomem *hwaddr;
> +    unsigned int fixmap_idx;
> +
> +    ASSERT(IS_ALIGNED(address, len));
> +
> +    *pval = ~0UL;
> +
> +    fixmap_idx = get_adjacent_idx(entry, address, false);
> +
> +    if ( !fixmap_idx )
> +        return X86EMUL_OKAY;
> +
> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        *pval = readb(hwaddr);
> +        break;
> +
> +    case 2:
> +        *pval = readw(hwaddr);
> +        break;
> +
> +    case 4:
> +        *pval = readl(hwaddr);
> +        break;
> +
> +    case 8:
> +        *pval = readq(hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +        break;
> +    }
> +
> +    return X86EMUL_OKAY;
> +}
> +
> +static int adjacent_write(
> +    const struct msixtbl_entry *entry,
> +    paddr_t address, unsigned int len, uint64_t val)
> +{
> +    void __iomem *hwaddr;
> +    unsigned int fixmap_idx;
> +
> +    ASSERT(IS_ALIGNED(address, len));
> +
> +    fixmap_idx = get_adjacent_idx(entry, address, true);
> +
> +    if ( !fixmap_idx )
> +        return X86EMUL_OKAY;
> +
> +    hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address);
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        writeb(val, hwaddr);
> +        break;
> +
> +    case 2:
> +        writew(val, hwaddr);
> +        break;
> +
> +    case 4:
> +        writel(val, hwaddr);
> +        break;
> +
> +    case 8:
> +        writeq(val, hwaddr);
> +        break;
> +
> +    default:
> +        ASSERT_UNREACHABLE();
> +        break;
> +    }
> +
> +    return X86EMUL_OKAY;

Since adjacent_{read,write}() unconditionally return X86EMUL_OKAY now
they could as well just return void.

I'm fine with leaving it like this, unless the committer doesn't mind
adjusting on commit.  I should have requested earlier to make the
function return void.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 10 15:21:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 15:21:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719947.1122856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5S3I-0004SH-CI; Fri, 10 May 2024 15:20:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719947.1122856; Fri, 10 May 2024 15:20:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5S3I-0004SA-9k; Fri, 10 May 2024 15:20:52 +0000
Received: by outflank-mailman (input) for mailman id 719947;
 Fri, 10 May 2024 15:20:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5S3G-0004S4-Bz
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 15:20:50 +0000
Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com
 [2607:f8b0:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e08bab29-0ee0-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 17:20:48 +0200 (CEST)
Received: by mail-oi1-x233.google.com with SMTP id
 5614622812f47-3c98b66c5f9so1260214b6e.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 08:20:48 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1cccf3sm18098816d6.86.2024.05.10.08.20.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 08:20:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e08bab29-0ee0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715354447; x=1715959247; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=2HZxSem/Mq8C06oiBy6tz2P6HbSGkkWucCT4/70iKno=;
        b=rUcyGXKgnbnEcF/w3wFzMlrcPU6CxJragL5RA82Z4oo5BUpzF8y0gsu+IZa77KMsrD
         gZeWyumtQg+dNeyWKf1BYMW0Pk7HQs6y6PzcS/N8CrKTXbTPJtrAsUYHgMG6Lff0InYS
         GQic9w1D9Oy9M8sAzKaW5pFDsaixDi+9aH3Qc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715354447; x=1715959247;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2HZxSem/Mq8C06oiBy6tz2P6HbSGkkWucCT4/70iKno=;
        b=I79QuIdmJDANeo6FrRwvZ2+lNVx9msGa151fCAEIahST06ASFBFqyyzJxaCMjY206o
         UZna/97XeRnDDZGqzHvBmiXSVvuTncTWouWkDJCQx08RdOaDHSrNqACyK2NG0n3QLqP/
         oxzHU0XI4xmcDi/ZjNIUTHM7S5O22HZ+wasRWi7hH4k0qH5SEkoPQfebxpb0mM3A3330
         mwqvWAo1sDB9z9d3r0EAuJIhY4ynreBC9gd0LP2G5pTr5RUJb8EqECeo70T9UT+FpML9
         4igNn56kRkLcM09HkODPljvSkbqREneAhUpuVcZTEuu5UszeAeg4mmOw2TJDJUEVjJUS
         o1kQ==
X-Forwarded-Encrypted: i=1; AJvYcCV9F2vCeT6/28nvgV4kGpbJoQLyI6gyu1n0C7bDK7mAgXmppdHw/eK8lkJq7vWgw+W4/6cqKn54NeqKMuaDuZu0qrUtoVO8zp5lJY8Yp+Q=
X-Gm-Message-State: AOJu0YxU57vGFnQPsNbD6Y5nUZs9kr6OIbLqpDemzGdI+qbyHzKxRXPz
	zr73E/xzGOjQAQo8dWEcKJ6VIHH5V/n1qm2VXjQSq5C2MmSymu9nru8mzy7oFhQ=
X-Google-Smtp-Source: AGHT+IGztjJm6j3+zuVyHiPtl7OVt1FKtDVfdX+zCJuHrN+GPKlsAPa8r+pNKroFQwicOWH9nCsIXw==
X-Received: by 2002:a05:6808:2208:b0:3c7:4976:7953 with SMTP id 5614622812f47-3c99702a800mr3613347b6e.3.1715354446880;
        Fri, 10 May 2024 08:20:46 -0700 (PDT)
Message-ID: <df58677c-d9a3-43a7-b5b4-85565ad73b12@citrix.com>
Date: Fri, 10 May 2024 16:20:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] libxl: fix population of the online vCPU bitmap
 for PVH
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Oleksii K ." <oleksii.kurochko@gmail.com>,
 Arthur Borsboom <arthurborsboom@gmail.com>, Leigh Brown <leigh@solinno.co.uk>
References: <20240510124913.49945-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240510124913.49945-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 10/05/2024 1:49 pm, Roger Pau Monne wrote:
> libxl passes some information to libacpi to create the ACPI table for a PVH
> guest, and among that information it's a bitmap of which vCPUs are online
> which can be less than the maximum number of vCPUs assigned to the domain.
>
> While the population of the bitmap is done correctly for HVM based on the
> number of online vCPUs, for PVH the population of the bitmap is done based on
> the number of maximum vCPUs allowed.  This leads to all local APIC entries in
> the MADT being set as enabled, which contradicts the data in xenstore if vCPUs
> is different than maximum vCPUs.
>
> Fix by copying the internal libxl bitmap that's populated based on the vCPUs
> parameter.
>
> Reported-by: Arthur Borsboom <arthurborsboom@gmail.com>
> Link: https://gitlab.com/libvirt/libvirt/-/issues/399
> Reported-by: Leigh Brown <leigh@solinno.co.uk>
> Fixes: 14c0d328da2b ('libxl/acpi: Build ACPI tables for HVMlite guests')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Note that the setup of hvm_info_table could be shared between PVH and HVM, as
> the fields are very limited, see hvm_build_set_params() for the HVM side.
> However this late in the release it's safer to just adjust the PVH path.
>
> Also note the checksum is not provided when hvm_info_table is built for PVH.
> This is fine so far because such checksum is only consumed by hvmloader and not
> libacpi itself.
>
> It's a bugfix, so should be considered for 4.19.

Bugfixes are still completely fair game at this point.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Fri May 10 15:46:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 15:46:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719955.1122867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5SSE-0008O0-8w; Fri, 10 May 2024 15:46:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719955.1122867; Fri, 10 May 2024 15:46:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5SSE-0008Nt-5n; Fri, 10 May 2024 15:46:38 +0000
Received: by outflank-mailman (input) for mailman id 719955;
 Fri, 10 May 2024 15:46:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5SSC-0008Nj-Nn; Fri, 10 May 2024 15:46:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5SSC-0008PK-Gs; Fri, 10 May 2024 15:46:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5SSC-0000bz-3q; Fri, 10 May 2024 15:46:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5SSC-0000gw-3T; Fri, 10 May 2024 15:46:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MQOLVX/88mfhc+Ur2lg/TgWNoaOOSmsT7srg53wqkV8=; b=mx1hE96tqx3A5hfIZQAPl0qMa0
	4ecvbp3fVZtC4XK8pkIvQPquAh62uYGFfby3b470JZvMJjs7bosezaYWk7ILjGH2/KRXq47W4dQVi
	JEFNqY48g0yrLL7dI+UiMq39ZK6es8EKl51zx6KCm9BZ8a9ZKHCtV1DMgdE6UaQ0lXhY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185968-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185968: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=dda10ac8acd0b5ffa03f9659c5678c2bddd9eed4
X-Osstest-Versions-That:
    libvirt=df9ffb025646d649f981ca25600225bc8cf794d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 15:46:36 +0000

flight 185968 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185968/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185943
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              dda10ac8acd0b5ffa03f9659c5678c2bddd9eed4
baseline version:
 libvirt              df9ffb025646d649f981ca25600225bc8cf794d1

Last test of basis   185943  2024-05-08 04:22:17 Z    2 days
Failing since        185951  2024-05-09 04:20:39 Z    1 days    2 attempts
Testing same since   185968  2024-05-10 04:18:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>
  Jiri Denemark <jdenemar@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   df9ffb0256..dda10ac8ac  dda10ac8acd0b5ffa03f9659c5678c2bddd9eed4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 10 16:27:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 16:27:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719974.1122877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5T5E-0006Uc-8D; Fri, 10 May 2024 16:26:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719974.1122877; Fri, 10 May 2024 16:26:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5T5E-0006UV-4l; Fri, 10 May 2024 16:26:56 +0000
Received: by outflank-mailman (input) for mailman id 719974;
 Fri, 10 May 2024 16:26:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ou8D=MN=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s5T5C-0006UP-U7
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 16:26:54 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1c9620a1-0eea-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 18:26:53 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59c5c9c6aeso550311466b.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 09:26:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c9620a1-0eea-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715358413; x=1715963213; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4YencXVp0PeCkXZse6MjETaVRXLmeumikfkf3wsxusE=;
        b=T6elLqeco0wJrbZwySQe+LgwFSvdGzGuy/H+kZbJRr+6YzOWH3dY9C/n5VrctMiwlH
         c3wZiuzITiLmHS20mmAmruT1ebRp93x1/7i511SRmdWWMkN2ooSNhtTSDtAIc3XMYNiM
         l1HzoGcmJW5/j4Nq9fmmcZDTtmj5nQDtKDjGVhKcH1RI/bedYLNbjBP1WWHWDq8AO8CW
         FN1yASRwQKBoKDsX3aDTWb1FD5du4VapXmGPcClTwAqVb+sjHtJNv7FtNxnNy4gIp1zY
         ujlGvCHU8iQG+CsrSEEGd1MQ4b2pdQOrII5VidHXCux+42cIMq2OcqUXKJxHSWiT+w/1
         c1eA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715358413; x=1715963213;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4YencXVp0PeCkXZse6MjETaVRXLmeumikfkf3wsxusE=;
        b=qBTkPK5a3a+u2W4vxHqbUkJ+Ci19tNICBwIzgRDbCBTNepiJri4byFLTh3xhtOzb8O
         xUwRn/ElywJhs5lFr2bppt4RrAazGiFp4pl1UU93iS91k8QgEIiSXTUkNcAYIcapeHRC
         nYVth4EPAWl3oz3zkSmGcL5XRC9K+mQwFLGwBQNayH+1HUvFAW0A9ANguiGYXDipZiyq
         Tq64NHmGQVIr2+tAQsvKzk5ockxIwks24/hPVL49kluRZ2m7Oba+XsiOdfzQLCkiHoTN
         pl7igWADNh2o/AFKsjV6kfVjoyc3B1QHTivmk68ooONHE6lt41Ej/oAoTDXYshy4ZLhG
         Ft1A==
X-Forwarded-Encrypted: i=1; AJvYcCUAJewOBU8NZc6h/swdHZTWwQ3MwD6z+pNgMtyaK66vjQgFJxehq1Oy+k84f7D9YrOoX5rqxudVkpwSSLBPlahhOYfBFLmGOeC4A34Td2o=
X-Gm-Message-State: AOJu0YxF5fvPzU2/CIyfI+kSvr0HorbLGAl9QW4JdE/ikdIlW18cguL3
	JpXIP3XNtEh94vxvOx/tf4M1Cqzi9blZU6FEfRivrPWGBa3a2iVn/vlYuh/oKv4QbwNPJbqpBLh
	ozE4C6g6fv36JJB57fmNuRPFJCdw=
X-Google-Smtp-Source: AGHT+IEmHnQyOSa++p0nsgPYr2jcAt9R09TE2w6ukXEp5RjofEYAQIxUz/WCMwpZPg22FtGWT1WlAAldAetkPX1L3dI=
X-Received: by 2002:a50:d51a:0:b0:56d:c722:93a3 with SMTP id
 4fb4d7f45d1cf-5734d5cccb2mr2330861a12.21.1715358413003; Fri, 10 May 2024
 09:26:53 -0700 (PDT)
MIME-Version: 1.0
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
 <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
 <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org> <CAJy5ezpNQP5q0NVaGdXtt5BbJSzCMfTfFt-0RRgVq3+DqMbKgA@mail.gmail.com>
 <35c8fbfb-8b09-4aae-8c98-402dcd65ca2e@xen.org>
In-Reply-To: <35c8fbfb-8b09-4aae-8c98-402dcd65ca2e@xen.org>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Fri, 10 May 2024 18:26:41 +0200
Message-ID: <CAJy5ezq7=jH2bzBqGG2qQMc4TJu5bw6wHAx+_RW63EaZv2wXVg@mail.gmail.com>
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol annotations
To: Julien Grall <julien@xen.org>, jbeulich@suse.com
Cc: Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org, 
	bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
	andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 7, 2024 at 7:37=E2=80=AFPM Julien Grall <julien@xen.org> wrote:
>
>
>
> On 07/05/2024 17:55, Edgar E. Iglesias wrote:
> > On Tue, May 7, 2024 at 11:57=E2=80=AFAM Julien Grall <julien@xen.org> w=
rote:
> > Hi Julien,
>
> Hi Edgar,
>
> >
> > The reason I choose FUNC for the start of the symbol is because these
> > symbols contain
> > executable code (not only a table of pointers to code somewhere else)
> > and the ELF spec
> > says that STT_FUNC means the symbol contains functions or other executa=
ble
> > code (not only callable functions IIUC):
> >
> > "STT_FUNC The symbol is associated with a function or other executable =
code."
> > https://refspecs.linuxbase.org/elf/elf.pdf
> > (Symbol Table 1-20).
>
> Thanks for the pointer. I originally did intend to suggest the change,
> but then I saw the use of LABEL in x86 (such as svm_stgi_label). There
> are a few others example with LABEL_LOCAL.
>
> AFAICT, this is also executable code which the only difference that it
> is not meant to be called by someone else. Furthermore, LABEL is using
> DO_CODE_ALIGN(...) for the alignment which imply that it is intended to
> be used by executable code. So I thought the only difference was whether
> the label was intended to be used as a function.
>

Thanks Julien, yes, good points.

> >
> > I think using LABEL instead of GLOBAL for the _end labels of these
> > code sequences makes sense.
> > I'm happy to change the _start labels to LABEL too if you guys feel
> > that's better.
>
> I have to admit I am little confused with the difference between LABEL
> vs FUNC. I think I will need some guidance from Jan (he introduced
> linkage.h).
>

Jan, do you have any guidance on this specific case using FUNC , LABEL
(or something else)?

Cheers,
Edgar


From xen-devel-bounces@lists.xenproject.org Fri May 10 17:40:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 17:40:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719992.1122887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5UES-00086f-DQ; Fri, 10 May 2024 17:40:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719992.1122887; Fri, 10 May 2024 17:40:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5UES-00086Y-AL; Fri, 10 May 2024 17:40:32 +0000
Received: by outflank-mailman (input) for mailman id 719992;
 Fri, 10 May 2024 17:40:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ycSx=MN=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s5UEQ-00086S-BY
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 17:40:30 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6318a868-0ef4-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 19:40:28 +0200 (CEST)
Received: from DS7PR03CA0297.namprd03.prod.outlook.com (2603:10b6:5:3ad::32)
 by DS7PR12MB6045.namprd12.prod.outlook.com (2603:10b6:8:86::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Fri, 10 May
 2024 17:40:21 +0000
Received: from DS3PEPF0000C37E.namprd04.prod.outlook.com
 (2603:10b6:5:3ad:cafe::9a) by DS7PR03CA0297.outlook.office365.com
 (2603:10b6:5:3ad::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48 via Frontend
 Transport; Fri, 10 May 2024 17:40:21 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF0000C37E.mail.protection.outlook.com (10.167.23.8) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Fri, 10 May 2024 17:40:21 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 12:40:19 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 12:40:19 -0500
Received: from [172.30.155.127] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 12:40:18 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6318a868-0ef4-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FvafahyBO87WAMtN8IBUGdj/ssg7lvYqbzM1vaatAHu12M6gz3T7G4q8PEROremfS+l79p7DxJa4jyHmcFC9uD8V85sHI4FkI2wOsJzKngdxJ4t3I3fkQfzht+OVtfJzmqslC7Pd7FQUKjCNw8Gi2iTHAh783sYA9HbuiH/2Lh8scGTpIHlYnQMGwO5sobBOugUUgOTeVoIp8EUBJIgr6EFAUQPpGBRpa8VHgkMzS3cyUK7/iVChJzjsecPG7Ff09v5JvzikUocdGP63gNgfT3NIwpccwuWQlgoQJx58dSoE+rftrP2li666g5S7EQ5S480PESNWMrl0AyZkYA7b/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=a11P9yYW/fZFQXXStAFZjMiY+7Mw3PQbrY/J83w7YV0=;
 b=gpgX+f3r8zMhgh2lPatmGJ5KjkSXuVwyKeozmxjS821YR4c2ECdfzGBxRc8Pp7saWhVMuFRpPYX3j+PX9eJRjOylvWL9+PGJu+NiMxfKHYLyV6j0uaknvOUnhSsngeuAbu9gJb7AIXW1ouy0YDomN6GxfOglWRiCBbszpekl8O6L7IXHtOJilRiBoiakvgmI1yGDo8vxpiddaPON2rsyF18qduSYNAeVis0/5ulDTzcOLAZId3WpiEAiLxGvxl2iSMALtiwh7DXlc14gPGcA3XzTk+tD8LQR484dqYvRGqZZ4m95cSyKbMvblugVGEshzBD2QfbPTShoFmSt6L8J4A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=a11P9yYW/fZFQXXStAFZjMiY+7Mw3PQbrY/J83w7YV0=;
 b=zmxEAJvlmWDt18xN7RrdlhyHR2ygpf09S+QvrmIV1vVl/UYo1VndasxocjxwAn4RJp7Js1HCCvaNRo5ttUwyw10kNq6J8c0JjeAvUMdujGyxQunMzlUQo4t6KxwSGAl7pvgLSc0sysjRMpFnIQfHqjKJsUXM7W7c84dHDKDM/fw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <0cfaeb1f-947a-4e45-9f69-a0e3f8143e2a@amd.com>
Date: Fri, 10 May 2024 13:40:18 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/3] x86: detect PIT aliasing on ports other than
 0x4[0-3]
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <0c45155a-2beb-4e69-bca3-cdf42ba22f2b@suse.com>
 <039b9ceb-4862-4e26-a344-e47fc04bd979@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <039b9ceb-4862-4e26-a344-e47fc04bd979@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37E:EE_|DS7PR12MB6045:EE_
X-MS-Office365-Filtering-Correlation-Id: 5687ec14-f0c0-4af0-3991-08dc711843fb
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?OFVzUXZiZTFnMUFXSVpkcks0TGJWVk9zVXJ5SmQ0NmsyQlBPN1hxcVhIbFJk?=
 =?utf-8?B?N21vNm9IanFrQVdrc2hxTFhteS9SeVRjUkVDR0k0S2VKMEJPa09Ea0s2VmNp?=
 =?utf-8?B?TFBBWkcrd0JLYXcwMkY1aFRNUkRMeHdKd3dkc0phcTJ6R0RiZWVFaDBDbDZj?=
 =?utf-8?B?N2lia3dDMHAvOWloVjRENVFXMnJpTlV1U3MrVUdXMWFqRlUzeFEvVTkyaVZV?=
 =?utf-8?B?d21vQTNPY2xXclE1dE14eFQ5QVRMZ21kb01WR2x4dFdURzkyS3VmckQ2Nyt5?=
 =?utf-8?B?cE03Yk1XdjY2cXg5aUNiZlBRRXI0UW1NWjg2TVpkZ2M5aTB1dG1oenNUZW5R?=
 =?utf-8?B?eFBkUU4vVzc5Qy9JWXcxeU5LNEY5MkZFdnREZEhzZ0JEOHhUNFZuaGhlaXZI?=
 =?utf-8?B?SWtrZ01ZS2hERy9idEZ2YlV1UVRCRHJnSWsyMi8wdXpUaTFiSTRBSFhZVjgz?=
 =?utf-8?B?Zy9jUWxTS2pOK0Z5elhPWTU4RE9WcnFlb29sT0JPSVAvN25rUHJBQTlnZEdr?=
 =?utf-8?B?dGlvVGorWDdCV0RHZEkyMDgrRjRmZFp3TUxsUFQ3aGpsRWcwYlVRVXA0Y2Rv?=
 =?utf-8?B?eHpqTjF6STF1WWpSeTVLdDk2UlA4YkoyZ2VpTVo5c3Z5cjNRNmFhNGxOMWtG?=
 =?utf-8?B?a1BLRzVFZDdEbGVRYkdkZ01KN2RnbFFLVjdLRHdYSkE4bW5BcDFQZStxNG42?=
 =?utf-8?B?ZXhacWZlVjVSNGFTcVhWR09JY2FyOVJFbDdNeHk2amRsejVQL3YxcFV2M0Fv?=
 =?utf-8?B?emtVdzRrTlR1RFhGclhPQXFrWi9oUWtoMU5Ra1RzN2NMYmdWMVdQQmxLdWdH?=
 =?utf-8?B?blBBcjN1T2VidURhZnRINjR2dFJEajhwMDB1NjRkbktSYmVoZVlOcDlPeHFX?=
 =?utf-8?B?amNEZWQvMURid2wxcmdwakx3MmJSaDBQQjVsRndvQTNOdUpHVGU0T3BJak9I?=
 =?utf-8?B?MDBqQXZORXJ2aUdZNTZzZzNvWGxmL1U0ZFo1akNtSzdzVXU3SDJncHhhWHZS?=
 =?utf-8?B?dTJ6OGltYks3OXpzSUpYTGFIMFd2Vk1hUzZDOHNGTE1SVXE1S0YvNjN2Sjkv?=
 =?utf-8?B?Z3QzSkQ4d2J4K1FhdytGVjFhOXFHSHdlWlkrYkU2b2VpbmdpeGZwVFF1bE5i?=
 =?utf-8?B?dTFydnl0V0xxelNxUkM5c2oraGlDbXBEMkt0YXEwbUcrNmhCNW54dTgzbkZ5?=
 =?utf-8?B?aG5qS3NYT05kdG51Z0M4YkNjWFFpTklRV204SlhaRjdRd0h1V1F4TmVqMTZV?=
 =?utf-8?B?UnBtbk9mL0phU3VBcGRudkdrbjhjTTlrRDMrTjRDekptOVpmd2ttT1Q1Z2ZJ?=
 =?utf-8?B?VFFXMHJOWHRUcThyUVZvUkFoZkNWc2h6aTk4T0JQSGF2TVdGVjFnZ3hHTW1W?=
 =?utf-8?B?dnpxTlZEMlk1Nk4vUi82YkZkZ3pvUVc2OTI4MS8yY29NalY2RG5PdVMzYWNQ?=
 =?utf-8?B?bkJ1NmdwWUxFaWZhMVdrQnpteTR6ZCtBM3gwK3ZQREtyTlJiU0RmWGtxWU5P?=
 =?utf-8?B?WE9vYnBMdkZhR2RGaFdIazZCTDRjV1o0NTI2YWF2eENUN0tlNWdKUE9wWUxJ?=
 =?utf-8?B?NnFaN0JpTHFSR294MjZFanFNek1zbkZtN2ZnNzdtcUthSGx0N09xUXU4eWtt?=
 =?utf-8?B?RVFkaTBVZzdxR1J6VDlQeDVBci9rOXNtc0hMNVhta09VT3BJejM1SDRIdEdl?=
 =?utf-8?B?d0pHdElMT0Y4UVFNcnArSkJzb3ZUVVE4bkhlQXNKaGczWVZyQTJMTjNTN3Rn?=
 =?utf-8?B?M3hWQXRjVUhoS1BtWW12VVcrZkF2bVpxVWtHQkhVY3RDMllVQkdPYnJ4ZmdR?=
 =?utf-8?B?WW1EUkN3YlUvSHEwV3V3Sm5WbW5XaTcvd0QxNHAvZG5ibXJMOFBzRDQvK0lm?=
 =?utf-8?Q?jsc9TBpQ/hanv?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 17:40:21.0803
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5687ec14-f0c0-4af0-3991-08dc711843fb
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF0000C37E.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6045

On 2023-12-18 09:48, Jan Beulich wrote:
> ... in order to also deny Dom0 access through the alias ports. Without
> this it is only giving the impression of denying access to PIT. Unlike
> for CMOS/RTC, do detection pretty early, to avoid disturbing normal
> operation later on (even if typically we won't use much of the PIT).
> 
> Like for CMOS/RTC a fundamental assumption of the probing is that reads
> from the probed alias port won't have side effects (beyond such that PIT
> reads have anyway) in case it does not alias the PIT's.
> 
> At to the port 0x61 accesses: Unlike other accesses we do, this masks

s/At/As/

> off the top four bits (in addition to the bottom two ones), following
> Intel chipset documentation saying that these (read-only) bits should
> only be written with zero.

This might be more useful in a comment next to `& 0x0c`.

> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> If Xen was running on top of another instance of itself (in HVM mode,
> not PVH, i.e. not as a shim), prior to 14f42af3f52d ('x86/vPIT: account
> for "counter stopped" time') I'm afraid our vPIT logic would not have
> allowed the "Try to further make sure ..." check to pass in the Xen
> running on top: We don't respect the gate bit being clear when handling
> counter reads. (There are more unhandled [and unmentioned as being so]
> aspects of PIT behavior though, yet it's unclear in how far addressing
> at least some of them would be useful.)
> ---
> v2: Use new command line option. Re-base over changes to earlier
>      patches. Use ISOLATE_LSB().
> 
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -495,7 +495,11 @@ int __init dom0_setup_permissions(struct
>       rc |= ioports_deny_access(d, 0x4D0, 0x4D1);
>   
>       /* Interval Timer (PIT). */
> -    rc |= ioports_deny_access(d, 0x40, 0x43);
> +    for ( offs = 0, i = ISOLATE_LSB(pit_alias_mask) ?: 4;
> +          offs <= pit_alias_mask; offs += i )
> +        if ( !(offs & ~pit_alias_mask) )
> +            rc |= ioports_deny_access(d, 0x40 + offs, 0x43 + offs);
> +
>       /* PIT Channel 2 / PC Speaker Control. */
>       rc |= ioports_deny_access(d, 0x61, 0x61);
>   
> --- a/xen/arch/x86/include/asm/setup.h
> +++ b/xen/arch/x86/include/asm/setup.h
> @@ -47,6 +47,7 @@ extern unsigned long highmem_start;
>   #endif
>   
>   extern unsigned int i8259A_alias_mask;
> +extern unsigned int pit_alias_mask;
>   
>   extern int8_t opt_smt;
>   extern int8_t opt_probe_port_aliases;
> --- a/xen/arch/x86/time.c
> +++ b/xen/arch/x86/time.c
> @@ -425,6 +425,72 @@ static struct platform_timesource __init
>       .resume = resume_pit,
>   };
>   
> +unsigned int __initdata pit_alias_mask;
> +
> +static void __init probe_pit_alias(void)
> +{
> +    unsigned int mask = 0x1c;
> +    uint8_t val = 0;
> +
> +    if ( !opt_probe_port_aliases )
> +        return;
> +
> +    /*
> +     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
> +     * count is loaded independent of counting being / becoming enabled.  Thus
> +     * we have a 16-bit value fully under our control, to write and then check
> +     * whether we can also read it back unaltered.
> +     */
> +
> +    /* Turn off speaker output and disable channel 2 counting. */
> +    outb(inb(0x61) & 0x0c, 0x61);
> +
> +    outb((2 << 6) | (3 << 4) | (0 << 1), PIT_MODE); /* Mode 0, LSB/MSB. */

Channel 2, Lobyte/Hibyte, 0b000 Mode 0, (Binary)

#define PIT_MODE_CH2 (2 << 6)
#define PIT_MODE0_16BIT ((3 << 4) | (0 << 1))

outb(PIT_MODE_CH2 | PIT_MODE0_16BIT, PIT_MODE);

> +
> +    do {
> +        uint8_t val2;
> +        unsigned int offs;
> +
> +        outb(val, PIT_CH2);
> +        outb(val ^ 0xff, PIT_CH2);
> +
> +        /* Wait for the Null Count bit to clear. */
> +        do {
> +            /* Latch status. */
> +            outb((3 << 6) | (1 << 5) | (1 << 3), PIT_MODE);

Read-back, Latch status,  read back timer channel 2

> +
> +            /* Try to make sure we're actually having a PIT here. */
> +            val2 = inb(PIT_CH2);
> +            if ( (val2 & ~(3 << 6)) != ((3 << 4) | (0 << 1)) )

if ( (val2 & PIT_RB_MASK) != PIT_MODE0_16BIT )

I think particularly a define for PIT_MODE0_16BIT would be helpful to 
show what is expected to be the same.

> +                return;
> +        } while ( val2 & (1 << 6) );

I think Roger might have mentioned on an earlier version - would it make 
sense to have a counter to prevent looping forever?

Also, FYI, I tested the series.  My test machine didn't show any aliasing.

Thanks,
Jason

> +
> +        /*
> +         * Try to further make sure we're actually having a PIT here.
> +         *
> +         * NB: Deliberately |, not ||, as we always want both reads.
> +         */
> +        val2 = inb(PIT_CH2);
> +        if ( (val2 ^ val) | (inb(PIT_CH2) ^ val ^ 0xff) )
> +            return;
> +
> +        for ( offs = ISOLATE_LSB(mask); offs <= mask; offs <<= 1 )
> +        {
> +            if ( !(mask & offs) )
> +                continue;
> +            val2 = inb(PIT_CH2 + offs);
> +            if ( (val2 ^ val) | (inb(PIT_CH2 + offs) ^ val ^ 0xff) )
> +                mask &= ~offs;
> +        }
> +    } while ( mask && (val += 0x0b) );  /* Arbitrary uneven number. */
> +
> +    if ( mask )
> +    {
> +        dprintk(XENLOG_INFO, "PIT aliasing mask: %02x\n", mask);
> +        pit_alias_mask = mask;
> +    }
> +}
> +
>   /************************************************************
>    * PLATFORM TIMER 2: HIGH PRECISION EVENT TIMER (HPET)
>    */
> @@ -2414,6 +2480,8 @@ void __init early_time_init(void)
>       }
>   
>       preinit_pit();
> +    probe_pit_alias();
> +
>       tmp = init_platform_timer();
>       plt_tsc.frequency = tmp;
>   
> 
> 
> 


From xen-devel-bounces@lists.xenproject.org Fri May 10 17:55:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 17:55:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.719998.1122898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5USn-0001lc-Q4; Fri, 10 May 2024 17:55:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 719998.1122898; Fri, 10 May 2024 17:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5USn-0001lV-Kp; Fri, 10 May 2024 17:55:21 +0000
Received: by outflank-mailman (input) for mailman id 719998;
 Fri, 10 May 2024 17:55:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5USm-0001lL-VS; Fri, 10 May 2024 17:55:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5USm-0003Aw-PN; Fri, 10 May 2024 17:55:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5USm-0006X1-9P; Fri, 10 May 2024 17:55:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5USm-0000he-8k; Fri, 10 May 2024 17:55:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lqhKSYtMSA8mAp9MJhZbEYUeKmK4x9XkrNwlweA4/uE=; b=jetzFp3HPWUkG6CyuYtCG981F0
	Uyl+EFCm00uCjvvhyd9GjdflH34hpQICqWTkpH/9sSPUQ3G7nDxJmVRqDrYeCSU8QpIDEKjnpFVsE
	3lx/8yic/cDZk8Lhv9uD51RS4I26S+w821eSGJdzXD/4c6KkpfoK8GHYUBINeyL6FXyA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185965-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185965: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=448b3fe5a0eab5b625a7e15c67c7972169e47ff8
X-Osstest-Versions-That:
    linux=8c3b7565f81e030ef448378acd1b35dabb493e3b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 17:55:20 +0000

flight 185965 linux-linus real [real]
flight 185971 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185965/
http://logs.test-lab.xenproject.org/osstest/logs/185971/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 185971-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 185971-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185971 like 185957
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185971 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185971 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185971 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185957
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185957
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185957
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 185957
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185957
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185957
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                448b3fe5a0eab5b625a7e15c67c7972169e47ff8
baseline version:
 linux                8c3b7565f81e030ef448378acd1b35dabb493e3b

Last test of basis   185957  2024-05-09 16:10:43 Z    1 days
Testing same since   185965  2024-05-10 02:20:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aleksa Savic <savicaleksa83@gmail.com>
  Guenter Roeck <linux@roeck-us.net>
  Lakshmi Yadlapati <lakshmiy@us.ibm.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marius Zachmann <mail@mariuszachmann.de>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   8c3b7565f81e..448b3fe5a0ea  448b3fe5a0eab5b625a7e15c67c7972169e47ff8 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri May 10 18:01:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:01:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720003.1122907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5UYC-0003IW-CN; Fri, 10 May 2024 18:00:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720003.1122907; Fri, 10 May 2024 18:00:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5UYC-0003IP-8m; Fri, 10 May 2024 18:00:56 +0000
Received: by outflank-mailman (input) for mailman id 720003;
 Fri, 10 May 2024 18:00:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5UYA-0003IJ-B4
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:00:54 +0000
Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com
 [2607:f8b0:4864:20::f2d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3d98279b-0ef7-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 20:00:53 +0200 (CEST)
Received: by mail-qv1-xf2d.google.com with SMTP id
 6a1803df08f44-69b514d3cf4so24132406d6.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 11:00:53 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1851f7sm19714496d6.47.2024.05.10.11.00.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 11:00:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d98279b-0ef7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715364052; x=1715968852; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zgiiMiwvtXwsEu/I1HVq6MxgkcX1ePwW93sRwxJLpfY=;
        b=PVTMYga2LVTsoOlLYm2sxeO0D+nbkd3nFisWR08HSZsUOR7HkgWzk+/CzNRo2FJ/Vh
         FWbqU/Kt39IFahBgd5rl48GFTpOqg1cn6jg+jOgbmZR9FKRMZ74VgiDG/yGowrvpv0x5
         r9E/xazQjJUALPDnqJP54yhs3zJBjvMb9a9kA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715364052; x=1715968852;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zgiiMiwvtXwsEu/I1HVq6MxgkcX1ePwW93sRwxJLpfY=;
        b=d49XGxttP9xivIfZJKEH6gwR9HQNzA3uucC/Ell3+8wpDJLd8w3HyqacmC3sOBBCol
         7GxgFYr7A4qiq14KNotVlyezE8God/es3xQA1oTHDisrzCVvVWtiGHTPR8USBtIyRvC0
         E0vI2NO3DuFMIKLYEsI4D/SciSNTqa+8xvYVY/sKuHsoxS+UZI791xdFUoNSmUNWzXpg
         4D9rphMZzDxeCBY9Sx0lcvEnpKS849qD9N6BcO1TveByWIF92fbebSM6BBoJDtV7Escr
         vtXo2/dDUZn1+U/De+FSjhREo7J7Yyig9yTMMznCy0CUGHGwilsvhwTOv0phPZk3Spm8
         3cVw==
X-Forwarded-Encrypted: i=1; AJvYcCXVae2qt+zv4SfsKcxC27TybcMjg7l5YBgg4Eqrpcxbzz+yUjgAub3Slah2N6X5eVO7mkX6tR/qBFCr+WK0bcrcS5p/+XueV9vIF8v/YiA=
X-Gm-Message-State: AOJu0YyVkf60K7YirBs5AQKMjO4voHqRoZT5TLJvyxkYhNc1VhWOKpKB
	64NY9fW/nAWBIQYDkqSyKj5p/7T9P46XfL9+87XaV4dYb8EVMPv6C719HYwcI7E=
X-Google-Smtp-Source: AGHT+IF/PQU707pGUrM+QiaobJ/qGWLuci8Nwf9AvYKnR0BxkXkAjiyfTe5Tbehaxr7qoMKFjMw9Ng==
X-Received: by 2002:a05:6214:5c44:b0:6a0:d20e:5406 with SMTP id 6a1803df08f44-6a16814f5b6mr32649476d6.17.1715364051980;
        Fri, 10 May 2024 11:00:51 -0700 (PDT)
Message-ID: <7384a44d-0eb8-4033-98b6-ddb7fd9a8131@citrix.com>
Date: Fri, 10 May 2024 19:00:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] libxl: Fix handling XenStore errors in device creation
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Demi Marie Obenour <demi@invisiblethingslab.com>,
 xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
References: <84a6e6376ab9f5aafac8f33bf772d73fcfd0d818.1713998669.git.demi@invisiblethingslab.com>
 <c978ac2b-0529-4bd8-a493-55e806740808@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c978ac2b-0529-4bd8-a493-55e806740808@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 10/05/2024 9:05 am, Jürgen Groß wrote:
> On 27.04.24 04:17, Demi Marie Obenour wrote:
>> If xenstored runs out of memory it is possible for it to fail operations
>> that should succeed.  libxl wasn't robust against this, and could fail
>> to ensure that the TTY path of a non-initial console was created and
>> read-only for guests.  This doesn't qualify for an XSA because guests
>> should not be able to run xenstored out of memory, but it still needs to
>> be fixed.
>>
>> Add the missing error checks to ensure that all errors are properly
>> handled and that at no point can a guest make the TTY path of its
>> frontend directory writable.
>>
>> Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>
> Apart from one nit below:
>
> Reviewed-by: Juergen Gross <jgross@suse.com>
>
>> ---
>>   tools/libs/light/libxl_console.c | 10 ++---
>>   tools/libs/light/libxl_device.c  | 72 ++++++++++++++++++++------------
>>   tools/libs/light/libxl_xshelp.c  | 13 ++++--
>>   3 files changed, 59 insertions(+), 36 deletions(-)
>>
>> diff --git a/tools/libs/light/libxl_console.c
>> b/tools/libs/light/libxl_console.c
>> index
>> cd7412a3272a2faf4b9dab0ef4dd077e55472546..adf82aa844a4f4989111bfc8a94af18ad8e114f1
>> 100644
>> --- a/tools/libs/light/libxl_console.c
>> +++ b/tools/libs/light/libxl_console.c
>> @@ -351,11 +351,10 @@ int libxl__device_console_add(libxl__gc *gc,
>> uint32_t domid,
>>           flexarray_append(front, "protocol");
>>           flexarray_append(front, LIBXL_XENCONSOLE_PROTOCOL);
>>       }
>> -    libxl__device_generic_add(gc, XBT_NULL, device,
>> -                              libxl__xs_kvs_of_flexarray(gc, back),
>> -                              libxl__xs_kvs_of_flexarray(gc, front),
>> -                              libxl__xs_kvs_of_flexarray(gc,
>> ro_front));
>> -    rc = 0;
>> +    rc = libxl__device_generic_add(gc, XBT_NULL, device,
>> +                                   libxl__xs_kvs_of_flexarray(gc,
>> back),
>> +                                   libxl__xs_kvs_of_flexarray(gc,
>> front),
>> +                                   libxl__xs_kvs_of_flexarray(gc,
>> ro_front));
>>   out:
>>       return rc;
>>   }
>> @@ -665,6 +664,7 @@ int libxl_device_channel_getinfo(libxl_ctx *ctx,
>> uint32_t domid,
>>                 */
>>                if (!val) val = "/NO-SUCH-PATH";
>>                channelinfo->u.pty.path = strdup(val);
>> +             if (channelinfo->u.pty.path == NULL) abort();
>
> Even with the bad example 2 lines up, please put the "abort();" into a
> line of its own.

I've fixed this on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 10 18:03:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:03:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720009.1122917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Uas-0003uG-PT; Fri, 10 May 2024 18:03:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720009.1122917; Fri, 10 May 2024 18:03:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Uas-0003u9-Lt; Fri, 10 May 2024 18:03:42 +0000
Received: by outflank-mailman (input) for mailman id 720009;
 Fri, 10 May 2024 18:03:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bUyD=MN=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s5Uas-0003u3-6Y
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:03:42 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1da003c-0ef7-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 20:03:41 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 4E77A4EE073C;
 Fri, 10 May 2024 20:03:39 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1da003c-0ef7-11ef-909d-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH] automation/eclair_analysis: tag MISRA C Rule 1.1 as clean
Date: Fri, 10 May 2024 20:03:36 +0200
Message-Id: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Tag the rule as clean, as there are no more violations in the codebase since
93c27d54dd23 ("xen/arm: Fix MISRA regression on R1.1,
flexible array member not at the end").

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index 1e13258fa9ae..a14bda5033df 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 10 18:08:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:08:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720013.1122927 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Ufr-0004i6-Fj; Fri, 10 May 2024 18:08:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720013.1122927; Fri, 10 May 2024 18:08:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Ufr-0004hz-Bs; Fri, 10 May 2024 18:08:51 +0000
Received: by outflank-mailman (input) for mailman id 720013;
 Fri, 10 May 2024 18:08:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5Ufq-0004ht-0j
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:08:50 +0000
Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com
 [2607:f8b0:4864:20::835])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5933aa6a-0ef8-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 20:08:49 +0200 (CEST)
Received: by mail-qt1-x835.google.com with SMTP id
 d75a77b69052e-43dfc9b9519so5518601cf.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 11:08:49 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54ef4c9sm23779851cf.30.2024.05.10.11.08.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 11:08:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5933aa6a-0ef8-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715364528; x=1715969328; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=l+I+R/j4eBA09bagd+hxXCmFCakGyqWsUzW4oMAalz4=;
        b=duWUnLTHGgXvVJFCpRxYnh0dEeXRbGTG42Dq2m43pMeLo39sWiDXAJfSRmecnLK7Sn
         Uba38V+HYYfoSmjP9/l6IYex87vFzGmgN87fNF5JuBBWpIMbrLiG/YeQe/4drQ822gTb
         ZulK+IGDLagdFXQtzHw0IOEMqMrX5FpwL3//0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715364528; x=1715969328;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=l+I+R/j4eBA09bagd+hxXCmFCakGyqWsUzW4oMAalz4=;
        b=pZFONkzYHsa6yAmYoWdmHol2AYaT/G9ZuXszpt4b3pPiYMCUXrBUxrBJfDx4ExRoRg
         hROzj8H38d3wHvYBDyHpu3dxF237iO2o+Fl2RM9ofOG/gn77MFARad67LiPj4Rq2gABt
         nnV889gOGZc9PBOFiqDiGNSoZ31sNwbOyGkfC9pq4SUJQyU8Vbw9t0WLl1DM+cWNauEl
         JlmM27xTNqIbKq4sBaCemC82+V8xuz/M1qk0WoNiotokPz6hUvbgUAhfWCtXbcgPpbJ1
         In24K/JJEnewnE8WsVVEjg7akAaHZSv1fHwq/aLf4L1NxokiaNux6xKtaBRuJ8S/Fccs
         Y/0w==
X-Forwarded-Encrypted: i=1; AJvYcCX74q0QtVTZKKpcI+fT+nchjPqpiKTOT3z/2fEOpbkHNrc/asgFnUHBcn97VVUQvYFxpjwoDZzwF3Z8awYx8TsvJ9nQHg/t7Hpf8Nmq7vs=
X-Gm-Message-State: AOJu0Yy50glscuFyeI/UYoiuxsCZvcE5Mr/1iV46OFNhCMRCXLAMFzeZ
	9LMmuRufOosCjoMRyj6Au47elqHB1YeH6EH3RAhWxHnxxsHB+5qSYAZAiGU1ALg=
X-Google-Smtp-Source: AGHT+IGj5eETBxJMZWh2mgld18FPj1J0+3x8QQaE8gYja8305UO+fgOjMWdpIxHbDYgqJ7YvnNxhjw==
X-Received: by 2002:ac8:5a49:0:b0:43b:b5e:cb3f with SMTP id d75a77b69052e-43dfdb482abmr37570661cf.47.1715364527818;
        Fri, 10 May 2024 11:08:47 -0700 (PDT)
Message-ID: <82599ec2-ac3a-41b7-84a0-9eb7c41b59fa@citrix.com>
Date: Fri, 10 May 2024 19:08:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy
 defined names
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240502114922.94288-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240502114922.94288-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/05/2024 12:49 pm, Roger Pau Monne wrote:
> diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
> index 07aed92684b5..3e6a66080a4f 100644
> --- a/tools/include/xen-tools/common-macros.h
> +++ b/tools/include/xen-tools/common-macros.h
> @@ -83,6 +83,10 @@
>  #define __packed __attribute__((__packed__))
>  #endif
>  
> +#ifndef __maybe_unused
> +# define __maybe_unused __attribute__((__unused__))
> +#endif
> +
>  #define container_of(ptr, type, member) ({              \
>      typeof(((type *)0)->member) *mptr__ = (ptr);        \
>      (type *)((char *)mptr__ - offsetof(type, member));  \

This hunk needs a matching:

diff --git a/tools/tests/x86_emulator/x86-emulate.h
b/tools/tests/x86_emulator/x86-emulate.h
index 8f8accfe3e70..a702c9faf207 100644
--- a/tools/tests/x86_emulator/x86-emulate.h
+++ b/tools/tests/x86_emulator/x86-emulate.h
@@ -50,7 +50,6 @@
 #define this_cpu(var) per_cpu_##var
 
 #define __init
-#define __maybe_unused __attribute__((__unused__))
 
 #define likely(x)   __builtin_expect(!!(x), true)
 #define unlikely(x) __builtin_expect(!!(x), false)

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 10 18:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720022.1122937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5V90-0000YS-KT; Fri, 10 May 2024 18:38:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720022.1122937; Fri, 10 May 2024 18:38:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5V90-0000YL-Ha; Fri, 10 May 2024 18:38:58 +0000
Received: by outflank-mailman (input) for mailman id 720022;
 Fri, 10 May 2024 18:38:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5V8z-0000YE-1A
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:38:57 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8d25d419-0efc-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 20:38:54 +0200 (CEST)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-6a0ffaa079dso29548776d6.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 11:38:54 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179685sm19594636d6.14.2024.05.10.11.38.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 11:38:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d25d419-0efc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715366333; x=1715971133; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=+uCd3k4S2h4kpULClLQ4KsyX2o+rNejT1whJjLp1Eiw=;
        b=DBnEvClUgOaFdAkVHtfH46M9JOF961ovhYHn9HG0cKJ4Z8B832x+mHCJrPWvlpDaqW
         DFsOKjPTRMcCTtQ1Qb8krkr0MCrk/zs6uWw4IqADIp40phS7F9pOME2oWl5xGySqnG+r
         GlhOK0E2GOmTStnObUcBRc1wVacOOjMOURKBs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715366333; x=1715971133;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+uCd3k4S2h4kpULClLQ4KsyX2o+rNejT1whJjLp1Eiw=;
        b=XONt1chxW1tXsfPSuVpasafFiGgDi0RsuVVwYz0VKSq5LbZoUCPd29lLayI44j06fR
         fSDrYIP8VWUi42Ddm/9jjiqoW/1phBcWCPQvsrfHG8h3TDXAicsK7fkUCysTmDitiiWy
         6cOKSokmyiy5lOn6xU1a9z7b1vgOx/SRAyGMu7GmRwubQ94Y/+kfSdpZtdVlzyswKJNf
         p4gs1doUTc0aAHoO6eO1ffiMdwPsFGxfjoS5dyGgYwHMzDGsbaqDDFeC6mDuasubMOmk
         9sHOIukLG2W8tO9/Lz+9dFssm+As3j+kHNPL0Uw3kMfrc4fZQcb/HOP3b5raitYCqTtx
         USRQ==
X-Forwarded-Encrypted: i=1; AJvYcCVklzvBDFOqdJukZQvTgnH3A/q+JZrW9O3+pc13JJwoE7E+pYW6Gjvs+GhukGVwwbDagfj4wCORvhTMr60a6E50m8SIJj1AsQvd5st60s0=
X-Gm-Message-State: AOJu0YxON5vtdQL0nWycAGlUTSW6mfCuYICFvB7nYgvZnMvSNLKYvBOe
	o9jCeXDsiwX11JW5wl3yOMnP7wmbB3oISEMKlFXzBd1I9pfF0NLdvHwi/NriuAVNRU9oFrFXPf6
	f
X-Google-Smtp-Source: AGHT+IHvtYCYaHND0Sb2jL0BiO4LZ56Fc9N75yLNdjwfGZJLm2q/954mLway+/Lw02tQVW2k7ATT0A==
X-Received: by 2002:a05:6214:4412:b0:696:4086:5e1 with SMTP id 6a1803df08f44-6a16790406fmr53215156d6.2.1715366332964;
        Fri, 10 May 2024 11:38:52 -0700 (PDT)
Message-ID: <fc278245-4666-4820-80cf-a0d39ff965fe@citrix.com>
Date: Fri, 10 May 2024 19:38:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair_analysis: tag MISRA C Rule 1.1 as
 clean
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>
References: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10/05/2024 7:03 pm, Nicola Vetrini wrote:
> Tag the rule as clean, as there are no more violations in the codebase since
> 93c27d54dd23 ("xen/arm: Fix MISRA regression on R1.1,
> flexible array member not at the end").
>
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>



From xen-devel-bounces@lists.xenproject.org Fri May 10 18:45:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:45:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720025.1122946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5VFO-0002KC-Ai; Fri, 10 May 2024 18:45:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720025.1122946; Fri, 10 May 2024 18:45:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5VFO-0002K5-7u; Fri, 10 May 2024 18:45:34 +0000
Received: by outflank-mailman (input) for mailman id 720025;
 Fri, 10 May 2024 18:45:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5VFM-0002Jz-GH
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:45:32 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 79b3457d-0efd-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 20:45:30 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2db13ca0363so36704481fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 11:45:30 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c699sm211206566b.53.2024.05.10.11.45.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 11:45:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79b3457d-0efd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715366729; x=1715971529; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SKpaY5WU7UHJhmw5XT4Z3i9Z6X0mzfzZ8gQ458pu6SA=;
        b=PuX9oRmlt7MSiIPKJlLi1dB2WK9S2HkfZ+/n1V73Dl85xv49OsZ7btWxgLYxNDkTmC
         Ap7Q5qyaH34ZkUYrOdWJB4rojlUNuBLMZ22Tbli2r1kisLpJ1UyqnFxqm3GFP+q46z+u
         Sv86g9nReKnlR8ZnhmN1k8ucA5VHK/0EdbrC0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715366729; x=1715971529;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SKpaY5WU7UHJhmw5XT4Z3i9Z6X0mzfzZ8gQ458pu6SA=;
        b=oIWGf1usXkBCebN0UyeG8VFhKis4CerqK0a4uioJrd1pSSJeCH7D+Oax/w0TQYuOyA
         deh/Z4J+Q+Ek1J5tU6OFVfi1lNjCO2kZZpzMQ/UaIXcerz7DtYTtpO8IlEmkYLW5sjuX
         msmjmc6R+VEDu5TinCsRoXO2ABOgEGCes5Uq7a+O617+iJ5ql9JP20TIsjLCcL9x/lbE
         3xNJDKb6pDeSFpDNvW+IIu+jOUcTX6XU6HEikLeucnda8qiZmUsyUVy8xhqplhrqiKHu
         M1c9FcFjnHyo/APIgY3uf5rhreip82PSJwv782uqdMqL8J6bXviBQ1hmz/aHOQHeGfvO
         KyyA==
X-Gm-Message-State: AOJu0YzIrt8I8rxAEO9OcTXYs+M2lv4mSF3bOOXMUT1aE4bKN5bFhGhk
	bv0MezBp6vr5JmbC1pZxuXeuB6byIjnKf7O3LVbEGbIfixhWYnpRIiw/jv+bgRIehuJ+0oYAVwP
	T
X-Google-Smtp-Source: AGHT+IFqcW7azKlpdKNF1stCeXPIMc8sFgQJijxbq8hsukoFN3xRc9RG79AaxqKQplbx5DtHevvd/g==
X-Received: by 2002:ac2:51a1:0:b0:521:f39d:1a8 with SMTP id 2adb3069b0e04-5220fc7cd5cmr2531479e87.8.1715366729521;
        Fri, 10 May 2024 11:45:29 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>
Subject: [PATCH] xen/build: Use -Wflex-array-member-not-at-end when available
Date: Fri, 10 May 2024 19:45:28 +0100
Message-Id: <20240510184528.2272560-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com>
References: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This option is new in GCC-14, and maps to MISRA Rule 1.1.  The codebase is
clean to it, and Eclair is blocking.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>

I really ought to have posted this before we regressed the tree, but oh well.
Better late than never.
---
 xen/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/Makefile b/xen/Makefile
index b6c39fc6aefb..71f0cb5071b3 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -396,6 +396,7 @@ CFLAGS += -nostdinc -fno-builtin -fno-common
 CFLAGS += -Werror -Wredundant-decls -Wwrite-strings -Wno-pointer-arith
 CFLAGS += -Wdeclaration-after-statement
 $(call cc-option-add,CFLAGS,CC,-Wvla)
+$(call cc-option-add,CFLAGS,CC,-Wflex-array-member-not-at-end)
 CFLAGS += -pipe -D__XEN__ -include $(srctree)/include/xen/config.h
 CFLAGS-$(CONFIG_DEBUG_INFO) += -g
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 18:48:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 18:48:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720027.1122957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5VIZ-0002uA-QQ; Fri, 10 May 2024 18:48:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720027.1122957; Fri, 10 May 2024 18:48:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5VIZ-0002u3-MZ; Fri, 10 May 2024 18:48:51 +0000
Received: by outflank-mailman (input) for mailman id 720027;
 Fri, 10 May 2024 18:48:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pcT1=MN=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s5VIY-0002tt-R8
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 18:48:50 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id efc8d4c2-0efd-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 20:48:49 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B514B61EC0;
 Fri, 10 May 2024 18:48:47 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7CDD2C113CC;
 Fri, 10 May 2024 18:48:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efc8d4c2-0efd-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715366927;
	bh=F6NpiZV0HywBr+Nx9rFTyJ0MlO0/atyKjAYoXr0moP8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hO14C2wjG/pVWHkcahozrgDkwUC5oE64vuam6LWzpX1aOJ+2G98qnj7uGiE6Q+fR2
	 3TiFcvLXdgbsYTQz+NjDRAhi4bMPl78j7bD7fiQlrLXF1HF4Z4J/RxfYZfV6AiSUAu
	 xLr2Ot+4SfHbWnv9Xj/8vINgKLp8T6j3vukDIikjKQG4xE7pUBOskjdLsrsryJyreB
	 L5LGOCPBqeBv0EBVX88w31wdej++FflD5ulhdv3HQhOLt5PYmefjwSBLDR1E6374EX
	 JLngOw34HY5BCjhoBD8sOy3FBmcRGHdwmEapbSRFu2kSlDLqOFL1VJoghlug8YHlVz
	 ra49czcoFY04A==
Date: Fri, 10 May 2024 11:48:45 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>
Subject: Re: [PATCH] xen/build: Use -Wflex-array-member-not-at-end when
 available
In-Reply-To: <20240510184528.2272560-1-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405101148390.2544314@ubuntu-linux-20-04-desktop>
References: <c19bc75727d18399f73a19dae991660c608a4a84.1715352180.git.nicola.vetrini@bugseng.com> <20240510184528.2272560-1-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 10 May 2024, Andrew Cooper wrote:
> This option is new in GCC-14, and maps to MISRA Rule 1.1.  The codebase is
> clean to it, and Eclair is blocking.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
> CC: Simone Ballarin <simone.ballarin@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> 
> I really ought to have posted this before we regressed the tree, but oh well.
> Better late than never.
> ---
>  xen/Makefile | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/xen/Makefile b/xen/Makefile
> index b6c39fc6aefb..71f0cb5071b3 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -396,6 +396,7 @@ CFLAGS += -nostdinc -fno-builtin -fno-common
>  CFLAGS += -Werror -Wredundant-decls -Wwrite-strings -Wno-pointer-arith
>  CFLAGS += -Wdeclaration-after-statement
>  $(call cc-option-add,CFLAGS,CC,-Wvla)
> +$(call cc-option-add,CFLAGS,CC,-Wflex-array-member-not-at-end)
>  CFLAGS += -pipe -D__XEN__ -include $(srctree)/include/xen/config.h
>  CFLAGS-$(CONFIG_DEBUG_INFO) += -g
>  
> -- 
> 2.30.2
> 


From xen-devel-bounces@lists.xenproject.org Fri May 10 19:26:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 19:26:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720035.1122967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Vt7-0000Ul-JU; Fri, 10 May 2024 19:26:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720035.1122967; Fri, 10 May 2024 19:26:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Vt7-0000Ue-F3; Fri, 10 May 2024 19:26:37 +0000
Received: by outflank-mailman (input) for mailman id 720035;
 Fri, 10 May 2024 19:26:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5Vt6-0000UW-Au
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 19:26:36 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35efeb63-0f03-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 21:26:33 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59a64db066so632204866b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 12:26:33 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfcccsm2126414a12.40.2024.05.10.12.26.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 12:26:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35efeb63-0f03-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715369193; x=1715973993; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=C1zO85bs5eljBEee9OS77+rFx14ErRnjC85N7aXgk4I=;
        b=sJrgcjIa1trWeMP1i2RvNfRdWslT3uhJG3W57pmVRrTXvwbCo7j9tg1rZAhpDpg77Q
         Gf2fNSZHSsFmLfktPBGmMpfWs7lnYyqPD8tU+5FhZVU9Bgb5m4YdwPaQNCJNyegzjZhU
         dDOXJTVNVBiKuVx+6TbNwtUb/Qp5P9iHB2P3U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715369193; x=1715973993;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=C1zO85bs5eljBEee9OS77+rFx14ErRnjC85N7aXgk4I=;
        b=ozpiAXZsTGmXhwOBFDb9px8h0Okj2G18RhAmX2yNRp1/F70kvahWVZgDwMe/FVf8mD
         CrUXldDXuyk/7JkIZSVD2Y1enwuK1VlUixNdJ53NwpzIJQk+jDEn3ov6Y10KpxnqT3j0
         vuHgiWlS4hJXBrZxhMhsgr6iRrgkwl3/0fC6hRS5iF8Dp5zH8+dS8RAXSC6ahfEeHg7t
         ScJTRD6FKbjiEcvY2+oenZ8OAWWauuu62baF89f8nZjK2UFOGRIUTgT6pqv5S1GgB5gD
         X8KDqpeNxti0opDNNZJLHgQkNRLE26C27HXZHtSAn+xyxsJqfo+YvEuLBk3jPRzX+rdb
         /Rug==
X-Gm-Message-State: AOJu0Yzt+dlrBNASSE3SJZ8g3qaNihKcwQ5niJSUMbUQh10jgQ/7CKdS
	3eb5NnbnubOsI7fqWqsN5KxR9fKaIZMp3Cz6SvydLy9amg5nVgjyrFQOKNa/Fv3SkyhlDSSrx3a
	P
X-Google-Smtp-Source: AGHT+IE+6E5bagkzJq1ZWGqkrY22ZhIBGiY1ImI4WyxLzIQ7QwP/vbDCvTIW315SL3+aX6aLBUfbeA==
X-Received: by 2002:a50:ee86:0:b0:573:4f61:ca9c with SMTP id 4fb4d7f45d1cf-5734f61cafemr2001581a12.4.1715369192998;
        Fri, 10 May 2024 12:26:32 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>
Subject: [PATCH] x86/io: Don't cast away constness in read{b..q}()
Date: Fri, 10 May 2024 20:26:31 +0100
Message-Id: <20240510192631.2296870-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Addresses various MISRA R11.8 violations.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
---
 xen/arch/x86/include/asm/io.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/include/asm/io.h b/xen/arch/x86/include/asm/io.h
index 9b19d2d389ee..1cb4217cff39 100644
--- a/xen/arch/x86/include/asm/io.h
+++ b/xen/arch/x86/include/asm/io.h
@@ -4,10 +4,10 @@
 #include <xen/vmap.h>
 #include <xen/types.h>
 
-#define readb(x) (*(volatile uint8_t  *)(x))
-#define readw(x) (*(volatile uint16_t *)(x))
-#define readl(x) (*(volatile uint32_t *)(x))
-#define readq(x) (*(volatile uint64_t *)(x))
+#define readb(x) (*(const volatile uint8_t  *)(x))
+#define readw(x) (*(const volatile uint16_t *)(x))
+#define readl(x) (*(const volatile uint32_t *)(x))
+#define readq(x) (*(const volatile uint64_t *)(x))
 #define writeb(d,x) (*(volatile uint8_t  *)(x) = (d))
 #define writew(d,x) (*(volatile uint16_t *)(x) = (d))
 #define writel(d,x) (*(volatile uint32_t *)(x) = (d))

base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 20:59:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 20:59:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720046.1122977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XKl-000414-UD; Fri, 10 May 2024 20:59:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720046.1122977; Fri, 10 May 2024 20:59:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XKl-00040x-RJ; Fri, 10 May 2024 20:59:15 +0000
Received: by outflank-mailman (input) for mailman id 720046;
 Fri, 10 May 2024 20:59:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5XKk-00040r-Vn
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 20:59:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XKk-0006dm-3K; Fri, 10 May 2024 20:59:14 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XKj-0007u2-Qx; Fri, 10 May 2024 20:59:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=DqBTFidJT/35FJRBdx2zdosdhl55KUQFS99G2blc5WY=; b=r6eAUzTucXye3FtYeeqwwUkRN7
	DrXCus8xZekIHjL4UErTRk/yywWfiwDZMBfFGC9tiECEsKbig+h8pi7y+9paacdUrQ5TWPS9FAfNa
	PV4weSCv574d/yNw5QVt3TNca6jWDLqnWHtjnd8cDjd/LLZWNy+SvkUK+Myo06k2XMq8=;
Message-ID: <d659e73f-762a-48c9-8936-94bbc771218c@xen.org>
Date: Fri, 10 May 2024 21:59:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of
 mem_access.c
Content-Language: en-GB
To: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 10/05/2024 13:32, Alessandro Zucchelli wrote:
> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
> changes are done:
> revert preprocessor conditional changes to xen/mem_access.h which
> had it build unconditionally, add conditional build for xen/mem_access.c

I am afraid, I don't understand this one as you don't seem to modify 
xen/mem_access.h. Is this meant to be part of the changelog?

You also don't seem to mention the change in Makefile. This is the one I 
was asking for in the previous version. So what about:

"xen/arm: mem_access: Conditionally compile mem_access.c

Commit 634cfc8beb ("Make MEM_ACCESS configurable") intended to make 
MEM_ACCESS configurable on Arm to reduce the code size when the user 
doesn't need it.

However, this didn't cover the arch specific code. None of the code in 
arm/mem_access.c is necessary when MEM_ACCESS=n, so it can be compiled 
out. This will require to provide some stub for functions called by the 
common code.

This is also fixing violation of the MISRA C:2012 Rule 8.4 reported by 
ECLAIR.
"

The patch itself loks good so once we agree on the commit message, then 
I am happy to update it on commit.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:06:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:06:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720052.1122987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XRh-0005ms-Lh; Fri, 10 May 2024 21:06:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720052.1122987; Fri, 10 May 2024 21:06:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XRh-0005ml-HB; Fri, 10 May 2024 21:06:25 +0000
Received: by outflank-mailman (input) for mailman id 720052;
 Fri, 10 May 2024 21:06:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5XRg-0005mb-E1; Fri, 10 May 2024 21:06:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5XRg-0006o5-9W; Fri, 10 May 2024 21:06:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5XRf-0005Cq-V4; Fri, 10 May 2024 21:06:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5XRf-0001kX-Ub; Fri, 10 May 2024 21:06:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FjZsQBSYw3UiodHHyZ4Ei8OYoZ32jsHz6fw0rKcRvaA=; b=wEYtkS1TrFpruYdv0Efw0XW913
	OFhcP+kApoJear+CBIkcimP1v9/31mg099QQXUbtS4GRB6RNWafqDF1E23UIm1sGnKPbuThWx35gv
	PtR9OXAl297GLCur6b7KKs4zMEIlMT4x4tZny9scfUl5ibuioxLC3rOwGV2OiIlh0i1s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185970-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185970: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b0082b908391b29b7c4dd5e6c389ebd6481926f8
X-Osstest-Versions-That:
    xen=53959cb8309919fc2f157a1c99e0af2ce280cb84
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 May 2024 21:06:23 +0000

flight 185970 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185970/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185961
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185961
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185961
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185961
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185961
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185961
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b0082b908391b29b7c4dd5e6c389ebd6481926f8
baseline version:
 xen                  53959cb8309919fc2f157a1c99e0af2ce280cb84

Last test of basis   185961  2024-05-09 22:36:58 Z    0 days
Testing same since   185970  2024-05-10 14:25:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
  Federico Serafini <federico.serafini@bugseng.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   53959cb830..b0082b9083  b0082b908391b29b7c4dd5e6c389ebd6481926f8 -> master


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:08:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:08:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720057.1122996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XTd-0006Ja-Vk; Fri, 10 May 2024 21:08:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720057.1122996; Fri, 10 May 2024 21:08:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XTd-0006JT-Sd; Fri, 10 May 2024 21:08:25 +0000
Received: by outflank-mailman (input) for mailman id 720057;
 Fri, 10 May 2024 21:08:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5XTc-0006JN-Js
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:08:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XTc-0006pk-FO; Fri, 10 May 2024 21:08:24 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XTc-000081-9l; Fri, 10 May 2024 21:08:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=xe7wyW4aNB7A0bXarsn4WH1tWm4wudSxNwAnothZjLs=; b=D0zKra2Wtajqag0ssKcd2/O4WW
	dStJY84wIA57XO7ocHqL7ChDergAI/bZOs2t5337VQROrb6blmZKYZwWkaClsZa0vob8t3996j6uT
	t5QlnZpPg4JE7cKupebDpYrIV7CDxCx47Ynw6Hn/AO0qaOveznBptA4OUP/KYmqvpdxw=;
Message-ID: <9650300c-9947-4d1d-87d5-aaa6102bfabf@xen.org>
Date: Fri, 10 May 2024 22:08:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7] xen: allow up to 16383 cpus
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Community Manager <community.manager@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240510141636.20543-1-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240510141636.20543-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 10/05/2024 15:16, Juergen Gross wrote:
> With lock handling now allowing up to 16384 cpus (spinlocks can handle
> 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit for
> the number of cpus to be configured to 16383.
> 
> The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
> QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
> 
> Add a support limit of physical CPUs to SUPPORT.md (4096 on x86, 128
> on ARM).
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

I am a bit surprised that this is kept given you added SUPPORT.md. I'd 
like Jan (or other x86 maintainers) to confirm they are happy with the 
proposed x86 security supported limit the SUPPORT.md.

For Arm (only):

Acked-by: Julien Grall <jgrall@amazon.com>

> ---
> V5:
> - new patch (Jan Beulich)
> V7:
> - add SUPPORT.md and CHANGELOG.md entries
> ---
>   CHANGELOG.md     | 2 ++
>   SUPPORT.md       | 5 +++++
>   xen/arch/Kconfig | 2 +-
>   3 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 8041cfb7d2..c43c45d8d4 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -14,6 +14,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
>      - HVM PIRQs are disabled by default.
>      - Reduce IOMMU setup time for hardware domain.
>    - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
> + - Increase the maximum number of CPUs Xen can be built for from 4095 to
> +   16383.
>   
>   ### Added
>    - On x86:
> diff --git a/SUPPORT.md b/SUPPORT.md
> index e10d46d924..d5d60c62ec 100644
> --- a/SUPPORT.md
> +++ b/SUPPORT.md
> @@ -61,6 +61,11 @@ For the Cortex A77 r0p0 - r1p0, see Errata 1508412.
>   
>       Status, x86: Supported
>   
> +### Physical CPUs
> +
> +    Status, x86: Supported up to 4096
> +    Status, ARM: Supported up to 128
> +
>   ### Host ACPI (via Domain 0)
>   
>       Status, x86 PV: Supported
> diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
> index 67ba38f32f..308ce129a8 100644
> --- a/xen/arch/Kconfig
> +++ b/xen/arch/Kconfig
> @@ -6,7 +6,7 @@ config PHYS_ADDR_T_32
>   
>   config NR_CPUS
>   	int "Maximum number of CPUs"
> -	range 1 4095
> +	range 1 16383
>   	default "256" if X86
>   	default "8" if ARM && RCAR3
>   	default "4" if ARM && QEMU

Cheers,


-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:11:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:11:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720063.1123019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XWc-0007sx-Jc; Fri, 10 May 2024 21:11:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720063.1123019; Fri, 10 May 2024 21:11:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XWc-0007sq-GE; Fri, 10 May 2024 21:11:30 +0000
Received: by outflank-mailman (input) for mailman id 720063;
 Fri, 10 May 2024 21:11:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5XWa-0007sM-VW
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:11:29 +0000
Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com
 [2607:f8b0:4864:20::72d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dd19c30c-0f11-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 23:11:27 +0200 (CEST)
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-78ecd752a7cso170143485a.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 14:11:27 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf31171esm214186085a.116.2024.05.10.14.11.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 14:11:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd19c30c-0f11-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715375486; x=1715980286; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=CfTRCP8kjMYc4ed/WJ8lC8rhANgJZgIfNM6JbRDL/G8=;
        b=tqd86d7KU+jUcAjJFsYBjaYKoIPMRr+HYA5jBflS10TQpWM6FAeL6pzy4OYpqfH9bw
         Nrynocq///E+2146+gmsTXJIkLVppXixUAFW64PnF1+5JBHTw2iGw5tikkHegLdC17lJ
         Kf5cwhVVbW0mjm7Ij24MyItFNhOHECb2NRwnY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715375486; x=1715980286;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CfTRCP8kjMYc4ed/WJ8lC8rhANgJZgIfNM6JbRDL/G8=;
        b=IF4JNtsP075HgOJjY+KjMwCVJdqakmCTLRDfwaYxf1QsABLHbaUce7JborU2SrBxvp
         KzmKrM/eikQt4EBu9QupxX2myStRe1WYRsAqM3QV/BNZqmdzWi4X7mDEHnWkQ7vpd+eT
         cUXBXZ99FqS1CFd7XQdksU+lHruFXwFiI5Rlu3PloLhB7aaJ4a8Dq0S1yokxS+VTu4ja
         gmRGLnd0rCQMBVe4DGTagamA2ARrD8EgzWLCG7WR0RVrE/cjO+FkTK5eKxkoUsHmrepR
         RU1SVKDo+Nv42HBEReRCKPK8XM6SWjVvIiYNKaAlwY8zh/4d44pUxCni8lqDIUCyO+FA
         Fs/A==
X-Forwarded-Encrypted: i=1; AJvYcCVz7XT6Mff1zw9Zb7/G1gpxr5S6+/fS8Ii08VIm+TO0gRLtlq4/bi+f4OF4ZJSYhhtRMrCJRQDQqshvBOOT2EC26sQ5rxGuTuUDjycUuqw=
X-Gm-Message-State: AOJu0YyDu21Uo1mIeBE0Cs+3eVXLVbFp1alofiIdIbuIBF0iCGTyCmwV
	CiaHHgKezbmuMJAL5cdHP0CUkz5oNvj8hyFriaBQzGUZ3Q7DMiGyKqsbrXedsGk=
X-Google-Smtp-Source: AGHT+IEgIzkrMVTTP0mKdRizot2RI2Tmzb+sPPsSxvmM6DYh7QQdGMFH2m4xGTONIkSV0ZSWlGXoXQ==
X-Received: by 2002:a05:620a:148c:b0:78e:db73:913b with SMTP id af79cd13be357-792c75900famr432670985a.28.1715375486535;
        Fri, 10 May 2024 14:11:26 -0700 (PDT)
Message-ID: <78fcd0d2-00fc-41ea-a531-7edbb8201dfc@citrix.com>
Date: Fri, 10 May 2024 22:11:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7] xen: allow up to 16383 cpus
To: Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
 xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Community Manager <community.manager@xenproject.org>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240510141636.20543-1-jgross@suse.com>
 <9650300c-9947-4d1d-87d5-aaa6102bfabf@xen.org>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <9650300c-9947-4d1d-87d5-aaa6102bfabf@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10/05/2024 10:08 pm, Julien Grall wrote:
> Hi Juergen,
>
> On 10/05/2024 15:16, Juergen Gross wrote:
>> With lock handling now allowing up to 16384 cpus (spinlocks can handle
>> 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit for
>> the number of cpus to be configured to 16383.
>>
>> The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
>> QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
>>
>> Add a support limit of physical CPUs to SUPPORT.md (4096 on x86, 128
>> on ARM).
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>
> I am a bit surprised that this is kept given you added SUPPORT.md. I'd
> like Jan (or other x86 maintainers) to confirm they are happy with the
> proposed x86 security supported limit the SUPPORT.md.
>
> For Arm (only):
>
> Acked-by: Julien Grall <jgrall@amazon.com>

XenServer is currently at 2k and we're waiting on this series to move to 4k.

I'm happy with this being the security statement for now.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:17:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:17:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720068.1123032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Xbx-0000XO-9h; Fri, 10 May 2024 21:17:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720068.1123032; Fri, 10 May 2024 21:17:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Xbx-0000XH-6u; Fri, 10 May 2024 21:17:01 +0000
Received: by outflank-mailman (input) for mailman id 720068;
 Fri, 10 May 2024 21:17:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5Xbw-0000XB-CD
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:17:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5Xbw-00071S-6T; Fri, 10 May 2024 21:17:00 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5Xbw-0000ho-08; Fri, 10 May 2024 21:17:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=7y6JqEcIu4UOJtCE5pOaee4TRfCG53EmNkxLng7PBZk=; b=TjqUKmN5VduPmwEIuXb0GOz6g5
	LZ8urIjDliGYcA3KVtXHVvoeKbvEr4INiAQlSTXX4vYITEi+756c349AcIxziOKMRyT35K72XHd6Y
	rMw89woAYAVmGfgHAZC4PkFz33OMfjY97myCNMQ8PuWAZ/piS6hnKvTONIg/H6ErKV8A=;
Message-ID: <83f234f2-d43e-4141-b447-bc53a368d81f@xen.org>
Date: Fri, 10 May 2024 22:16:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7] xen: allow up to 16383 cpus
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>, Juergen Gross
 <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Community Manager <community.manager@xenproject.org>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240510141636.20543-1-jgross@suse.com>
 <9650300c-9947-4d1d-87d5-aaa6102bfabf@xen.org>
 <78fcd0d2-00fc-41ea-a531-7edbb8201dfc@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <78fcd0d2-00fc-41ea-a531-7edbb8201dfc@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 10/05/2024 22:11, Andrew Cooper wrote:
> On 10/05/2024 10:08 pm, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 10/05/2024 15:16, Juergen Gross wrote:
>>> With lock handling now allowing up to 16384 cpus (spinlocks can handle
>>> 65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit for
>>> the number of cpus to be configured to 16383.
>>>
>>> The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
>>> QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
>>>
>>> Add a support limit of physical CPUs to SUPPORT.md (4096 on x86, 128
>>> on ARM).
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>
>> I am a bit surprised that this is kept given you added SUPPORT.md. I'd
>> like Jan (or other x86 maintainers) to confirm they are happy with the
>> proposed x86 security supported limit the SUPPORT.md.
>>
>> For Arm (only):
>>
>> Acked-by: Julien Grall <jgrall@amazon.com>
> 
> XenServer is currently at 2k and we're waiting on this series to move to 4k.
> 
> I'm happy with this being the security statement for now.

Thanks for confirming :). Committed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:21:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:21:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720073.1123043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Xgb-0001zC-Qn; Fri, 10 May 2024 21:21:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720073.1123043; Fri, 10 May 2024 21:21:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Xgb-0001z5-Nr; Fri, 10 May 2024 21:21:49 +0000
Received: by outflank-mailman (input) for mailman id 720073;
 Fri, 10 May 2024 21:21:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pcT1=MN=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s5Xga-0001yj-9E
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:21:48 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4cc9fbc9-0f13-11ef-b4bb-af5377834399;
 Fri, 10 May 2024 23:21:46 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id C2586CE1E9B;
 Fri, 10 May 2024 21:21:42 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 087C0C113CC;
 Fri, 10 May 2024 21:21:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cc9fbc9-0f13-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715376102;
	bh=NXPBefajJWp5diqqNfsdQGCulnXOO8URjp6YDf+9Xh4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jqzmA9ObewJi5LbCMrINWHOmSP9EJEu/Fy+xByuAZet3fSj8sJ4sNdOSpqlJ4fRZy
	 zBOkIw1DoWbGH4ya4jEFOIs39W27x55u0YdA9QEBfS7s/9nJjx/bIcegaAJLNNHKrN
	 UA+KFm00Wj/TWzaADOBhaMLu25peYimZbTgtgjN0jLNv++OZz4vMjWBSLhlh8Qosba
	 OreZsC/gQ/P9Mo76SIK2ti7QcUns2KM0C5NfL7NwWXCDJ9YysS4MoFF6dRDg8yfBVo
	 Yt1qSAJeGU7oXvw7wc+MCc8O6chtb+2Ho9pRum+k6jKoyXAvL7qE1Ek8X5affP15Yq
	 6XVjF/SvG8iog==
Date: Fri, 10 May 2024 14:21:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Jan Beulich <JBeulich@suse.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>
Subject: Re: [PATCH] x86/io: Don't cast away constness in read{b..q}()
In-Reply-To: <20240510192631.2296870-1-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405101421310.2544314@ubuntu-linux-20-04-desktop>
References: <20240510192631.2296870-1-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-2057329677-1715376102=:2544314"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2057329677-1715376102=:2544314
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 10 May 2024, Andrew Cooper wrote:
> Addresses various MISRA R11.8 violations.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> ---
>  xen/arch/x86/include/asm/io.h | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/x86/include/asm/io.h b/xen/arch/x86/include/asm/io.h
> index 9b19d2d389ee..1cb4217cff39 100644
> --- a/xen/arch/x86/include/asm/io.h
> +++ b/xen/arch/x86/include/asm/io.h
> @@ -4,10 +4,10 @@
>  #include <xen/vmap.h>
>  #include <xen/types.h>
>  
> -#define readb(x) (*(volatile uint8_t  *)(x))
> -#define readw(x) (*(volatile uint16_t *)(x))
> -#define readl(x) (*(volatile uint32_t *)(x))
> -#define readq(x) (*(volatile uint64_t *)(x))
> +#define readb(x) (*(const volatile uint8_t  *)(x))
> +#define readw(x) (*(const volatile uint16_t *)(x))
> +#define readl(x) (*(const volatile uint32_t *)(x))
> +#define readq(x) (*(const volatile uint64_t *)(x))
>  #define writeb(d,x) (*(volatile uint8_t  *)(x) = (d))
>  #define writew(d,x) (*(volatile uint16_t *)(x) = (d))
>  #define writel(d,x) (*(volatile uint32_t *)(x) = (d))
> 
> base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
> -- 
> 2.30.2
> 
--8323329-2057329677-1715376102=:2544314--


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:38:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:38:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720078.1123056 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XwD-0004bV-62; Fri, 10 May 2024 21:37:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720078.1123056; Fri, 10 May 2024 21:37:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5XwD-0004bO-3A; Fri, 10 May 2024 21:37:57 +0000
Received: by outflank-mailman (input) for mailman id 720078;
 Fri, 10 May 2024 21:37:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5XwC-0004bI-1l
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:37:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XwB-0007VM-Kc; Fri, 10 May 2024 21:37:55 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5XwB-0002CX-Ck; Fri, 10 May 2024 21:37:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=U1d9YXDaA9GQHUJ6ut9tZLYFSAMZ/8EBZ7ZZK6wdh2U=; b=2A3whaY+/dBpHNthoRv9gI9ITV
	kDGBA4L5Mg1f1TeQGkHYyu8mwcu2ZV4nK0GVtuCGR8P2JQqMpIMOw4u4c3Gjlza8qfLuQNLkXc4YT
	JfK83fga0kfh8v/1v+Mx1pddUAj8QgU8R7VWUkLjbHC1QYq3s0VuJGDs7XXZ43QhkuMw=;
Message-ID: <e1cf9c7c-581d-4bec-acaf-654d40b7ac07@xen.org>
Date: Fri, 10 May 2024 22:37:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
 <Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org> <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org> <ZjyzTCOSeNvPq2Np@macbook>
 <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org> <ZjzIWsoRmgmWkgGh@macbook>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ZjzIWsoRmgmWkgGh@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Roger,

On 09/05/2024 13:58, Roger Pau Monné wrote:
> On Thu, May 09, 2024 at 01:12:00PM +0100, Julien Grall wrote:
>> Hi,
>>
>> On 09/05/2024 12:28, Roger Pau Monné wrote:
>>> On Thu, May 09, 2024 at 10:50:56AM +0100, Julien Grall wrote:
>>>>
>>>>
>>>> On 09/05/2024 09:13, Roger Pau Monné wrote:
>>>>> On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote:
>>>>> Also the interactions with the remote domain would need to be audited,
>>>>> as the remote domain shattering the superpage would need to be
>>>>> replicated in the mapping side in order to account for the changes.
>>>>
>>>> ... I don't understand this one. How is this different from today's where a
>>>> domain can foreign map a 2MB which may be using a superpage in the remote
>>>> domain?
>>>
>>> Hm, right, I was wrong with that I think, as long as proper references
>>> as taken for the superpage entries it should be fine.
>>>
>>>>>    Not sure all paths will be easy to
>>>>> audit for preemption if it's more than relinquish_p2m_mapping() that
>>>>> you need to adjust.
>>>>
>>>> I thought about it yesterday. But I came to the conclusion that if we have
>>>> any concern about removing 1GB foreign superpage then we would already have
>>>> the problem today as a domain can map contiguously 1GB worth of foreign
>>>> mapping using small pages.
>>>
>>> Yeah, but in that case addition or removal is done in 4K chunks, and
>>> hence we can preempt during the operation.
>>
>> I am not entirely sure how that would work. From my understand, today, most
>> of the users of the P2M code expects the operation to complete in one go and
>> if preemption is needed then the caller is responsible to handle it by
>> breaking up the happy.
>>
>> With your suggestion, it sounds like you want to rework how the preemption
>> today and push it to the P2M code. This would mean we would need to modify
>> all the callers to check for -EERESTART (or similar) and also tell them how
>> many pages were handled so the call can be restarted where it stopped. Is it
>> what you had in mind?
> 
> TBH, I didn't have a specific location in mind about where to do the
> split.
> 
> One solution that could simplify it is allowing foreign entries to
> only be removed by specific functions, so that the split required in
> order to do the removal can be handled by the caller knowing it's
> dealing with a foreign map superpage.

That would work. It would require quite a bit of work though.

> But that would require the superpage to be shattered, and hence will
> require creating lower levle leaf entries in order to do the
> shattering and the removal in 4K chunks.
> 
>> I don't expect the work to be trivial, so I wonder if this is really worth
>> it to try to change the way we preempt.
>>
>>>
>>> OTOH for 1GB given the code here the page could be freed in one go,
>>> without a chance of preempting the operation.
>>>
>>> Maybe you have to shatter superpages into 4K entries and then remove
>>> them individually, as to allow for preemption to be possible by
>>> calling put_page() for each 4K chunk?
>> This would require to allocate some pages from the P2M pool for the tables.
>> As the pool may be exhausted, it could be problematic when relinquishing the
>> resources.
> 
> Indeed, it's not ideal.
> 
>> It may be possible to find a way to have memory available by removing other
>> mappings first. But it feels a bit hackish and I would rather prefer if we
>> avoid allocating any memory when relinquishing.
> 
> Maybe it could be helpful to provide a function to put a superpage,
> that internally calls free_domheap_pages() with the appropriate order
> so that freeing a superpage only takes a single free_domheap_pages()
> call. 

Today, free_domheap_page() is only called when the last reference is 
removed. I don't thinkt here is any guarantee that all the references 
will dropped at the same time.

 >  That could reduce some of the contention around the heap_lock
 > and d->page_alloc_lock locks.

 From previous experience (when Hongyan and I worked on optimizing 
init_heap_pages() for Live-Update), the lock is actually not the biggest 
problem. The issue is adding the pages back to the heap (which may 
requiring merging). So as long as the pages are not freed contiguously, 
we may not gain anything.

Anyway, it sound like someone needs to spend some time investigating 
this issue.

> 
> Note also that a foreign unmap resulting in a page free is also not
> the common case, as that should only happen when the foreign domain
> has been destroyed, or the page ballooned out, so to benchmark the
> worst case some effort will be needed in order to model this
> scenario.

Good callout. It may be easier to reproduce it with some XTF tests. 
Unfortunately, I don't have the bandwith to look at it. Maybe Luca can?

Otherwise, we may have to accept not supporting 1GB superpage for the 
time being for shared memory. But I am not actually sure this is a big 
problem?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 10 21:48:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 21:48:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720083.1123066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Y6N-0006my-29; Fri, 10 May 2024 21:48:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720083.1123066; Fri, 10 May 2024 21:48:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Y6M-0006mr-VZ; Fri, 10 May 2024 21:48:26 +0000
Received: by outflank-mailman (input) for mailman id 720083;
 Fri, 10 May 2024 21:48:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5Y6L-0006mj-QZ
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 21:48:25 +0000
Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com
 [2607:f8b0:4864:20::b33])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06600dea-0f17-11ef-909d-e314d9c70b13;
 Fri, 10 May 2024 23:48:24 +0200 (CEST)
Received: by mail-yb1-xb33.google.com with SMTP id
 3f1490d57ef6-de46b113a5dso2393446276.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 14:48:24 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1cd425sm20746426d6.79.2024.05.10.14.48.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 14:48:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06600dea-0f17-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715377703; x=1715982503; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=WMjaW/BZOFli0+JU+OoX6QUE6PumScRfCzonCuH3W0g=;
        b=hKySHGg4OfSZtMJp+oDsV5Coaz7wnWqt7l0DryN75i73y/N6IJk8Jo/OATUFBbOUqQ
         lQynGFAclHcfzU9HUGW5ARZr6AXk2hPphxPPC/0XMBTIjrjO95UpwIvF2bIDruuMgizd
         1XgzWcPNfefBNnJSccH+uXeIiRn5xzSvMZsnE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715377703; x=1715982503;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WMjaW/BZOFli0+JU+OoX6QUE6PumScRfCzonCuH3W0g=;
        b=DnWwUjCjafYB1ulT5jCRX6/M+ftcVGxtOrHRJjZ0fuOKt7VvxXH6d+tihypHDvKk/e
         ynQYfIry+hbK0ryPa2ckd3zMRMTbuIpTlwZS1DQSoi5rGLhwaVC2fqrO+UJVSymEJZKB
         1rvqRpQc6UU5TpUP8l5bEjQvRIZpRpRZtiheEHM7B/WLbBrdp8UcnkZrPqd7yCXwFiQW
         Ww2IhrcMa8TlepW4s6AMqui7XhE5PnL9e1HYJRftGmFDfl0JhVTZZAgx5/7avrP2s9js
         bBbuo0oCmoWEG4OgeZMRitez3LZ/9deLtdstOw71YGFjShvUbVWj9fOcKOMIoPMfCUR9
         WFSw==
X-Forwarded-Encrypted: i=1; AJvYcCWBYiy6V7SXXvwYQFXazQqvhNHSBqLE3FFvsLryzqfw+Jgp/1+xGwGAhO61EoNH++gUsr1vVTdhgY3A0I86oTf77zlREhpiRjPSAxbK0lE=
X-Gm-Message-State: AOJu0YxsuIHOes+zthhNiu4J0yLeN+AWsDgfn5a29xc7caxDlC1Upcwz
	F8Vt6GyDx8A7zDOpWYWMdyPpIelos6s+njxORrzkzNeNerLoRm3yVYMbOL/8Hxg=
X-Google-Smtp-Source: AGHT+IHfvHOMLj/MMiZqozdHMF1GUSKXoW8DYK5MvtedC6h+inkQixXhrXDnWcey4TmyOIbpoJmajA==
X-Received: by 2002:a05:6902:2583:b0:de5:8816:2d1c with SMTP id 3f1490d57ef6-dee4f39cc39mr3872740276.65.1715377703200;
        Fri, 10 May 2024 14:48:23 -0700 (PDT)
Message-ID: <cf0cd2ef-58df-476f-89c5-c843a85f71f7@citrix.com>
Date: Fri, 10 May 2024 22:48:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] docs/misra: add R21.6 R21.9 R21.10 R21.14 R21.15
 R21.16
To: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <alpine.DEB.2.22.394.2404261435010.3940@ubuntu-linux-20-04-desktop>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <alpine.DEB.2.22.394.2404261435010.3940@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26/04/2024 10:36 pm, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
>
> Changes in v3:
> - add explanation in footnote
> - remove comment from 21.14, 21.15, 21.16
>
>  docs/misra/rules.rst | 42 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 42 insertions(+)
>
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index b7b447e152..5ba7394f05 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -652,12 +652,48 @@ maintainers if you want to suggest a change.
>         declared
>       - See comment for Rule 21.1
>  
> +   * - `Rule 21.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_06.c>`_
> +     - Required
> +     - The Standard Library input/output routines shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
> +   * - `Rule 21.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_09.c>`_
> +     - Required
> +     - The library functions bsearch and qsort of <stdlib.h> shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
> +   * - `Rule 21.10 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_10.c>`_
> +     - Required
> +     - The Standard Library time and date routines shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
>     * - `Rule 21.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_13.c>`_
>       - Mandatory
>       - Any value passed to a function in <ctype.h> shall be representable as an
>         unsigned char or be the value EOF
>       -
>  
> +   * - `Rule 21.14 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_14.c>`_
> +     - Required
> +     - The Standard Library function memcmp shall not be used to compare
> +       null terminated strings
> +     -
> +
> +   * - `Rule 21.15 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_15.c>`_
> +     - Required
> +     - The pointer arguments to the Standard Library functions memcpy,
> +       memmove and memcmp shall be pointers to qualified or unqualified
> +       versions of compatible types
> +     -
> +
> +   * - `Rule 21.16 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_16.c>`_
> +     - Required
> +     - The pointer arguments to the Standard Library function memcmp
> +       shall point to either a pointer type, an essentially signed type,
> +       an essentially unsigned type, an essentially Boolean type or an
> +       essentially enum type
> +     - void* arguments are allowed
> +
>     * - `Rule 21.17 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_17.c>`_
>       - Mandatory
>       - Use of the string handling functions from <string.h> shall not result in
> @@ -712,3 +748,9 @@ maintainers if you want to suggest a change.
>       - The value of a pointer to a FILE shall not be used after the associated
>         stream has been closed
>       -
> +
> +
> +.. [1] Xen implements itself a few functions with names that match the
> +       corresponding function names of the Standard Library for developers'
> +       convenience. These functions are part of the Xen code and subject to
> +       analysis.

I discovered the hard way (c/s 1357f1fa588f1) that numbered footnotes
are an issue when the document gets more complicated.

I'd suggest using #xen-stdlib as a named footnote instead of [1].

Can fix on commit.

~Andrew




From xen-devel-bounces@lists.xenproject.org Fri May 10 22:15:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:15:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720087.1123076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YVo-0003iJ-WB; Fri, 10 May 2024 22:14:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720087.1123076; Fri, 10 May 2024 22:14:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YVo-0003iC-TY; Fri, 10 May 2024 22:14:44 +0000
Received: by outflank-mailman (input) for mailman id 720087;
 Fri, 10 May 2024 22:14:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aEqs=MN=invisiblethingslab.com=demi@srs-se1.protection.inumbo.net>)
 id 1s5YVn-0003i4-4t
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:14:43 +0000
Received: from wfout6-smtp.messagingengine.com
 (wfout6-smtp.messagingengine.com [64.147.123.149])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b01fd2dd-0f1a-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 00:14:38 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id 6CD921C000E1;
 Fri, 10 May 2024 18:14:35 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute6.internal (MEProxy); Fri, 10 May 2024 18:14:35 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 10 May 2024 18:14:34 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b01fd2dd-0f1a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715379275;
	 x=1715465675; bh=NiJLpQcs0yme5KQLluRGxF758s0CI5seD6WPvuBOpgg=; b=
	tdKrJoJrLQZtCetaZOlXrAMZ0iPo6YDpni1RrcHzLlwkpC1ZoJYmK5Z6LgDQkan/
	a9I3V5+nAtHvTfGGOMhAcY860fi+G4G2nXwzWqyQjfoigWhZgPdaUbwyLtoLN1Yy
	ArF86Y6wsU84+GYY2R5QScqHKVJY0orHTdYazxke0NxMFmsm3erh40Soe7tGu/97
	ibulekWAGQeI/BjvvD2+PTFydbVU9KX8yOOyj0YcDIgfgUzate+YQ4ZorbWlruMS
	kTKHXt4UBi3sVYWp8pMVKvCF/05LieMvwU7ZgdnemIylmDD9Ld2NkbEJxWD5t9gq
	KqpNbDo8qOA33SDoTpMsSw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715379275; x=1715465675; bh=NiJLpQcs0yme5KQLluRGxF758s0C
	I5seD6WPvuBOpgg=; b=KG+NwuBHGdoJ0adLFYEqqJkuNVvS3yO4gLp64T5TfEb5
	eD6jOKicRl1q46fO6WeyoO1l+n3yUAtl7ofdusFRC+HqTsMukeaQ3/0lwqbUJghl
	DP9vvpIlTOyzJROerWAG03sb+ENisTy91HJ5HQhhtYXkIPbG8geTFWR2FJIBgFdS
	pDdugSzDTI+2Wp/t/btQtx2CuxsEOlj5SGxac5kQZL1yIOcEICVZMCtV03hcrazE
	E6xSI3BtR3auMZwPhU6an4o46/VB2Cx/5LbH6yiYc4au0V90jndUy+8uIv/ybA3T
	xe6u2/RMMFB1Vk7qtMSJd9HGvMOsHOkP11juyq9TSA==
X-ME-Sender: <xms:Spw-Zn7EWC_YbqGfixjoNchGbQJxIVfVBrHgygRoH6r_xl-H0qlp5A>
    <xme:Spw-Zs75cploQw-3w9Np8PUJqaRVCA9Wfuwe0x3YoRloKc0A04melaPHcvz-LW9ge
    v43M2fu--dl4Xg>
X-ME-Received: <xmr:Spw-ZufIq4da1KNYzmniiLObCIHxcTilP_77wztRYUaFK2y8w3FUOFCJ9s3MId0wmLakMQJRmmKItyd6IMLX2RpwID88jkT7AZHjd7y5703k2M05>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefledgtdejucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepffgvmhhi
    ucforghrihgvucfqsggvnhhouhhruceouggvmhhisehinhhvihhsihgslhgvthhhihhngh
    hslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepvdejteegkefhteduhffgteffgeff
    gfduvdfghfffieefieekkedtheegteehffelnecuvehluhhsthgvrhfuihiivgeptdenuc
    frrghrrghmpehmrghilhhfrhhomhepuggvmhhisehinhhvihhsihgslhgvthhhihhnghhs
    lhgrsgdrtghomh
X-ME-Proxy: <xmx:Spw-ZoKAe-aA8HpesJIJR7QuN6Jfw-8TqcK_Tpt-PU4rRsEBcoSR-Q>
    <xmx:Spw-ZrJHJohV40lDquEX40rphKL57PvqVoEgOUe5b9OwJuFczyNNNQ>
    <xmx:Spw-ZhwPtVE6qsIQplZI0xlyECW6yPxvVURy-TKyStMCCP1GSTftzA>
    <xmx:Spw-ZnIoGB613FLEkzdEOZlVVXN5ZYEr9nzS84DjMT6H0UUeqv6Z8g>
    <xmx:S5w-ZoGTp912U1_hrfll_4KBgG34apJGPCYHbSP35w5ZlOvOO1lVMMLv>
Feedback-ID: iac594737:Fastmail
Date: Fri, 10 May 2024 18:14:20 -0400
From: Demi Marie Obenour <demi@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: Re: [PATCH] libxl: Fix handling XenStore errors in device creation
Message-ID: <Zj6cSdiyu31BoFkE@itl-email>
References: <84a6e6376ab9f5aafac8f33bf772d73fcfd0d818.1713998669.git.demi@invisiblethingslab.com>
 <c978ac2b-0529-4bd8-a493-55e806740808@suse.com>
 <7384a44d-0eb8-4033-98b6-ddb7fd9a8131@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="zHTcFoPsAYLYMqmE"
Content-Disposition: inline
In-Reply-To: <7384a44d-0eb8-4033-98b6-ddb7fd9a8131@citrix.com>


--zHTcFoPsAYLYMqmE
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Fri, 10 May 2024 18:14:20 -0400
From: Demi Marie Obenour <demi@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: Re: [PATCH] libxl: Fix handling XenStore errors in device creation

On Fri, May 10, 2024 at 07:00:49PM +0100, Andrew Cooper wrote:
> On 10/05/2024 9:05 am, J=C3=BCrgen Gro=C3=9F wrote:
> > On 27.04.24 04:17, Demi Marie Obenour wrote:
> >> If xenstored runs out of memory it is possible for it to fail operatio=
ns
> >> that should succeed.=C2=A0 libxl wasn't robust against this, and could=
 fail
> >> to ensure that the TTY path of a non-initial console was created and
> >> read-only for guests.=C2=A0 This doesn't qualify for an XSA because gu=
ests
> >> should not be able to run xenstored out of memory, but it still needs =
to
> >> be fixed.
> >>
> >> Add the missing error checks to ensure that all errors are properly
> >> handled and that at no point can a guest make the TTY path of its
> >> frontend directory writable.
> >>
> >> Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> >
> > Apart from one nit below:
> >
> > Reviewed-by: Juergen Gross <jgross@suse.com>
> >
> >> ---
> >> =C2=A0 tools/libs/light/libxl_console.c | 10 ++---
> >> =C2=A0 tools/libs/light/libxl_device.c=C2=A0 | 72 ++++++++++++++++++++=
------------
> >> =C2=A0 tools/libs/light/libxl_xshelp.c=C2=A0 | 13 ++++--
> >> =C2=A0 3 files changed, 59 insertions(+), 36 deletions(-)
> >>
> >> diff --git a/tools/libs/light/libxl_console.c
> >> b/tools/libs/light/libxl_console.c
> >> index
> >> cd7412a3272a2faf4b9dab0ef4dd077e55472546..adf82aa844a4f4989111bfc8a94a=
f18ad8e114f1
> >> 100644
> >> --- a/tools/libs/light/libxl_console.c
> >> +++ b/tools/libs/light/libxl_console.c
> >> @@ -351,11 +351,10 @@ int libxl__device_console_add(libxl__gc *gc,
> >> uint32_t domid,
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flexarray_appen=
d(front, "protocol");
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flexarray_appen=
d(front, LIBXL_XENCONSOLE_PROTOCOL);
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }
> >> -=C2=A0=C2=A0=C2=A0 libxl__device_generic_add(gc, XBT_NULL, device,
> >> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_of_flexarray(gc, back),
> >> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_of_flexarray(gc, front),
> >> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_of_flexarray(gc,
> >> ro_front));
> >> -=C2=A0=C2=A0=C2=A0 rc =3D 0;
> >> +=C2=A0=C2=A0=C2=A0 rc =3D libxl__device_generic_add(gc, XBT_NULL, dev=
ice,
> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_=
of_flexarray(gc,
> >> back),
> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_=
of_flexarray(gc,
> >> front),
> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 libxl__xs_kvs_=
of_flexarray(gc,
> >> ro_front));
> >> =C2=A0 out:
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return rc;
> >> =C2=A0 }
> >> @@ -665,6 +664,7 @@ int libxl_device_channel_getinfo(libxl_ctx *ctx,
> >> uint32_t domid,
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 */
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 if (!val) val =3D "/NO-SUCH-PATH";
> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 channelinfo->u.pty.path =3D strdup(val);
> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 if (channelinfo->u.pty.path =3D=3D NULL) abort();
> >
> > Even with the bad example 2 lines up, please put the "abort();" into a
> > line of its own.
>=20
> I've fixed this on commit.
>=20
> ~Andrew

Thank you.

Should this be backported to stable braches?  It's not a security
vulnerability from a Xen upstream PoV, but "running Xenstore out of
memory" should be denial of service only, not a potential privilege
escalation.  This is especially true if Xenstore is in dom0, where there
might be other processes that could eat up lots of memory.
--=20
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

--zHTcFoPsAYLYMqmE
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEdodNnxM2uiJZBxxxsoi1X/+cIsEFAmY+nEgACgkQsoi1X/+c
IsH6ww/+JgKGiVdt6utiKU47BftXQFvoWqOq1a2VOJlf57u4unbSC7s5eANu2g6c
QQoHDHwhdIKMhtOsd5a1JOrkZsG6D8aUWKXp6x/z0IehK7TTZr6twRL50+v6BqY6
/dmW6rdanwO3W7PqAKw93sY6jT61Phi+bPE+cSgIPtvSiMxFpW+ZGUi2FmjSRAmK
eEuQMh1XlztIKOATspWwWWu7Y9xXU+Bi9mAtSz6R+41ju/Ai0n8BrspNZN/drYmR
gDpp6nvoxfX+VR2lNPWZPIia85D29BTlp7VPp8kFPgPpgBSM5X9urEfquGyJf8fi
fcyAHr+vQHYnTE5wsXJEIdtp0MZvP4M2qSDWm0fPpXWQwn8IqI4ujLS3vZ4VPNXe
aYhtHWBGIoHTxJm6s+bqctEslBnuFmAv9ArAR4TKJuPu4cYIEPAgZeONkRjXHjE7
KSjnFAtI6XL1YnIK0VTjTefzuAwKNLnWHR6jM7HuQDeGQR0P9Oe2zn5EfRpq7wSI
tb+4ZAPXU5zTxIbNxb4KMHL9hqJE88FkETS8hF9jD/fCYR32f6ZxyfUT0GDdzgmk
l1AL/C0O75CFOkh7Oeg/jkLrlmwvYILD2evm/kpTeNyAseEl+pqIzEXDynrtq/Ec
YrmssBTT4mFqlsRuH9ongZu9O6dFT9PLaYsxfL5Z+OvX63Y//2s=
=msjs
-----END PGP SIGNATURE-----

--zHTcFoPsAYLYMqmE--


From xen-devel-bounces@lists.xenproject.org Fri May 10 22:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720094.1123091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuO-0008A2-22; Fri, 10 May 2024 22:40:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720094.1123091; Fri, 10 May 2024 22:40:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuN-00089K-UU; Fri, 10 May 2024 22:40:07 +0000
Received: by outflank-mailman (input) for mailman id 720094;
 Fri, 10 May 2024 22:40:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YuN-000872-BN
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:40:07 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f3a152e-0f1e-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 00:40:05 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59a8f0d941so604152466b.2
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:40:05 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm233915866b.14.2024.05.10.15.40.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 15:40:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f3a152e-0f1e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715380805; x=1715985605; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+3wFptMesC7scMbASW9uUHmAUxFS1IgS9Pg8ty+kfrw=;
        b=EpTAo49FsvSsODgxrPqrr9TrNKTloxjagvc3oiBngOaPPcakoQaQ9eM15/aGp3XLvI
         lURNMpM/bg0eElR/1juMUb8M+nvwc7+ecRglNqaWy6pwaaL2cSvY0z3IYTOTameYCRk1
         Kst9OBeIcYid6LPIi9Kez30+hupCgQBxPqx0c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715380805; x=1715985605;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+3wFptMesC7scMbASW9uUHmAUxFS1IgS9Pg8ty+kfrw=;
        b=od3iLsbvYRnPepxP5HMVC1wTer8cJND7pHsL/3/n25FWl5i0mRpmZou2oa9kdJQRTa
         ZXztr9XU1w1N8zQ8GBtQFIaE+gWbVufy2LoZryg27sHxFHs3t/VqVnkxOxKrkw1LuIsw
         LQEk3gQFlpx7FAX64tsnmEcL5oyoCu1NYQKGSbqJolJoZXgWzxVJG8UyYu0WAfV+L/mz
         lFuSr1jlMVRwPnn2YttkavI8OoeNYZKYN3FgOWQPoJLhSoMWUfYyBFfF0rk8m2M02Jli
         H2UFvchyyDakBWNnjrnWk0INCVYEoVtyCa6binhfhF5COKMOuDvUuZmQLujD0nugR8fx
         h5Zw==
X-Gm-Message-State: AOJu0Yw6cLT3lcJapXexiLc8qfEzGY42oX86xx2O0PgZVlPfrU42E8T+
	vtcVYP8O194xiaGMeyvMVbQPLCi93i1SieAOgdFzpIytw0n7M0tr08PZEuTsNDuWLtXhZlJaPRa
	u
X-Google-Smtp-Source: AGHT+IH0oLMFKMzpEMCVInvb1E6s+F94kooOglSc0Ol6TYVo3iWnR3bdBI+xVyJPk4/L2nL4aRjWeA==
X-Received: by 2002:a17:906:22cc:b0:a59:c319:f1e3 with SMTP id a640c23a62f3a-a5a2d53ae7emr246124866b.12.1715380805100;
        Fri, 10 May 2024 15:40:05 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 1/4] x86/gen-cpuid: Minor cleanup
Date: Fri, 10 May 2024 23:39:59 +0100
Message-Id: <20240510224002.2324578-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Rename INIT_FEATURE_NAMES to INIT_FEATURE_NAME_TO_VAL as we're about to gain a
inverse mapping of the same thing.

Use dict.items() unconditionally.  iteritems() is a marginal perf optimsiation
for Python2 only, and simply not worth the effort on a script this small.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * New
---
 tools/libs/light/libxl_cpuid.c | 2 +-
 xen/arch/x86/cpu-policy.c      | 2 +-
 xen/tools/gen-cpuid.py         | 9 ++-------
 3 files changed, 4 insertions(+), 9 deletions(-)

diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index ce4f3c7095ba..063fe86eb72f 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -296,7 +296,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *policy, const char* str)
 
         {NULL, 0, NA, CPUID_REG_INV, 0, 0}
     };
-    static const struct feature_name features[] = INIT_FEATURE_NAMES;
+    static const struct feature_name features[] = INIT_FEATURE_NAME_TO_VAL;
     /*
      * NB: if we switch to using a cpu_policy derived object instead of a
      * libxl_cpuid_policy_list we could get rid of the featureset -> cpuid leaf
diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 99871b8e0e05..b96f4ee55cc4 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -43,7 +43,7 @@ static const uint32_t deep_features[] = INIT_DEEP_FEATURES;
 static const struct feature_name {
     const char *name;
     unsigned int bit;
-} feature_names[] __initconstrel = INIT_FEATURE_NAMES;
+} feature_names[] __initconstrel = INIT_FEATURE_NAME_TO_VAL;
 
 /*
  * Parse a list of cpuid feature names -> bool, calling the callback for any
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 380b9d973a67..79d7f5c8e1c9 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -459,15 +459,10 @@ def write_results(state):
     state.output.write(
 """}
 
-#define INIT_FEATURE_NAMES { \\
+#define INIT_FEATURE_NAME_TO_VAL { \\
 """)
 
-    try:
-        _tmp = state.values.iteritems()
-    except AttributeError:
-        _tmp = state.values.items()
-
-    for name, bit in sorted(_tmp):
+    for name, bit in sorted(state.values.items()):
         state.output.write(
             '    { "%s", %sU },\\\n' % (name, bit)
             )
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 22:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720096.1123116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuQ-0000OP-Em; Fri, 10 May 2024 22:40:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720096.1123116; Fri, 10 May 2024 22:40:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuQ-0000OI-Bw; Fri, 10 May 2024 22:40:10 +0000
Received: by outflank-mailman (input) for mailman id 720096;
 Fri, 10 May 2024 22:40:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YuP-000872-Bd
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:40:09 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 40260fd6-0f1e-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 00:40:07 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so559624066b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:40:07 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm233915866b.14.2024.05.10.15.40.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 15:40:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40260fd6-0f1e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715380806; x=1715985606; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Phw85PeYT8ClsRvMd2JBZsNjZhs23kHM7gjroo/IJSY=;
        b=eKxt9k9HpPpDkLmGwthZfYlmFQzLunWL9YafRWJ+NEwTtUXDdYkMg7S86W80ZQKm/r
         1ZSfFQMDZzPw7X6u+wQ0AcdRSguAXAsq33oQugU6zSYvZTHtVvDXSe1tFjLWyAh03fyg
         6L5KdP0T2rY+0QnmYWzXI9QO5A34nLSv+/iR0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715380806; x=1715985606;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Phw85PeYT8ClsRvMd2JBZsNjZhs23kHM7gjroo/IJSY=;
        b=utwtBdbmsfHhqI7ES8U5hzn0ERN0d/ISdjDv+ikcLwKG7T4NWd4JoyABk9yl5oGcNh
         tWohC3IE2itRUgKfBDPdS6Uq/lLT9clwgwjgwGtj15cJR0kX7pvBSCK8qLWo27k9a0mc
         Z6LbA9UzSBURBSzSmlPL/nTVNLxklrKM/LXjeyXaOhkTsMhaht1dfiMb2xU8b1e8QiAT
         4t8na+IysUga5NCKRhPaUlSeS/KsiZvk8g1+WN1UC0yya/FmCQjAyEfM9sCihxX7el9s
         bxi2fxbSRqqwLSYqawOn3FzvG5A21mrvwPQnEtpDudfFnIfsobnrS6sP8fpkYHAKymKx
         otvA==
X-Gm-Message-State: AOJu0YykJ1zS3nijXdp/UwsLH7Ks+6VUB+ANQ5K3N7M976MQTMnbUFvS
	Jc/IB5kVby7eOYMknVhYf0xONeVWcf6l07JfLXJuKzrSMAJd3EQtgXmSG6yznDf2+vLot9xQxWZ
	i
X-Google-Smtp-Source: AGHT+IHf4xSJp8NCGGmvT9XjSTZCP+6zYExEd3Ow9Gm2SOkX5JRmY26CWDUdpoU0non11Cvmx80fkQ==
X-Received: by 2002:a17:907:77c3:b0:a59:9db6:535d with SMTP id a640c23a62f3a-a5a2d67862amr251200766b.65.1715380806572;
        Fri, 10 May 2024 15:40:06 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>
Subject: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature names
Date: Fri, 10 May 2024 23:40:01 +0100
Message-Id: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Roger Pau Monné <roger.pau@citrix.com>

Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
are needed.

In xen-cpuid.c, have the compiler pad both leaf_info[] and feature_names[] if
necessary.  This avoids needing complicated cross-checks.

As dump_leaf() rendered missing names as numbers, always dump leaves even if
we don't have the leaf name.  This conversion was argumably missed in commit
59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

Differences in names are:

 sysenter    -> sep
 tm          -> tm1
 ds-cpl      -> dscpl
 est         -> eist
 sse41       -> sse4-1
 sse42       -> sse4-2
 movebe      -> movbe
 tsc-dl      -> tsc-deadline
 rdrnd       -> rdrand
 hyper       -> hypervisor
 mmx+        -> mmext
 fxsr+       -> ffxsr
 pg1g        -> page1gb
 3dnow+      -> 3dnowext
 cmp         -> cmp-legacy
 cr8d        -> cr8-legacy
 lzcnt       -> abm
 msse        -> misalignsse
 3dnowpf     -> 3dnowprefetch
 nodeid      -> nodeid-msr
 dbx         -> dbext
 tsc-adj     -> tsc-adjust
 fdp-exn     -> fdp-excp-only
 deffp       -> no-fpu-sel
 <24>        -> bld
 ppin        -> amd-ppin
 lfence+     -> lfence-dispatch
 ppin        -> intel-ppin
 energy-ctrl -> energy-filtering

Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
one.  Several of the + names would be nice to keep as were, but doing so isn't
nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
but we intentionally don't list them, so I'm not worried.

Thoughts?

v3:
 * Rework somewhat.
 * Insert aliases of common_1d.
---
 tools/misc/xen-cpuid.c | 15 ++++++---------
 xen/tools/gen-cpuid.py | 21 +++++++++++++++++++++
 2 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 6ee835b22949..2f34694e9c57 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -11,6 +11,7 @@
 #include <xenguest.h>
 
 #include <xen-tools/common-macros.h>
+#include <xen/lib/x86/cpuid-autogen.h>
 
 static uint32_t nr_features;
 
@@ -268,7 +269,7 @@ static const struct {
     const char *name;
     const char *abbr;
     const char *const *strs;
-} leaf_info[] = {
+} leaf_info[FEATURESET_NR_ENTRIES] = {
     { "CPUID 0x00000001.edx",        "1d", str_1d },
     { "CPUID 0x00000001.ecx",        "1c", str_1c },
     { "CPUID 0x80000001.edx",       "e1d", str_e1d },
@@ -291,6 +292,9 @@ static const struct {
 
 #define COL_ALIGN "24"
 
+static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
+    INIT_FEATURE_VAL_TO_NAME;
+
 static const char *const fs_names[] = {
     [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
     [XEN_SYSCTL_cpu_featureset_host]    = "Host",
@@ -304,12 +308,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
 {
     unsigned i;
 
-    if ( !strs )
-    {
-        printf(" ???");
-        return;
-    }
-
     for ( i = 0; i < 32; ++i )
         if ( leaf & (1u << i) )
         {
@@ -338,8 +336,7 @@ static void decode_featureset(const uint32_t *features,
     for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
     {
         printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
-        if ( leaf_info[i].name )
-            dump_leaf(features[i], leaf_info[i].strs);
+        dump_leaf(features[i], &feature_names[i * 32]);
         printf("\n");
     }
 }
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 79d7f5c8e1c9..d0bb2e4a229f 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -470,6 +470,27 @@ def write_results(state):
     state.output.write(
 """}
 
+""")
+
+    state.output.write(
+"""
+#define INIT_FEATURE_VAL_TO_NAME { \\
+""")
+
+    for name, bit in sorted(state.values.items()):
+        state.output.write(
+            '    [%s] = "%s",\\\n' % (bit, name)
+            )
+
+        # Add the other alias for 1d/e1d common bits
+        if bit in state.common_1d:
+            state.output.write(
+                '    [%s] = "%s",\\\n' % (64 + bit, name)
+            )
+
+    state.output.write(
+"""}
+
 """)
 
     for idx, text in enumerate(state.bitfields):
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 22:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720095.1123106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuP-00008Q-6Y; Fri, 10 May 2024 22:40:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720095.1123106; Fri, 10 May 2024 22:40:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuP-00008J-40; Fri, 10 May 2024 22:40:09 +0000
Received: by outflank-mailman (input) for mailman id 720095;
 Fri, 10 May 2024 22:40:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YuO-000872-BY
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:40:08 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3fad9b8d-0f1e-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 00:40:06 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-572baf393ddso5879082a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:40:06 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm233915866b.14.2024.05.10.15.40.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 15:40:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3fad9b8d-0f1e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715380806; x=1715985606; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=r/L62oz/vhjY7sgRvmqLoVnO9fQ3LM/iOAHiRUewZCg=;
        b=ERuTKkERbxwCLDKkYxFCkSm8wi6CGoMth3NP6SDedM3m8QOLyDGg8Osm8+a2YWwIon
         67FJ/F/kpVlvA2lhJSmpeKPecfsvhFfuu9soISFCUVzWmgTlbRpEG0c0f1Bct94ivBbD
         FbgNyB8WloExS0PvzPN4M/fyQvzpmhG0XK0aM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715380806; x=1715985606;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=r/L62oz/vhjY7sgRvmqLoVnO9fQ3LM/iOAHiRUewZCg=;
        b=AIBgRj+GqloqEhR6LdbrjucPTJD15/d7wfMVY6633BmqgaqLsJOEy+89ZTjKsT5Afg
         fuihdT/w01D74r/A+xEBK0FB8f7S4jrfwE2FbqeTnV1OSdkn4uHHJqcdPSUYfIRxxDBy
         bWsBDwwQLEVZs5bT7DrAdZLJbdmTn5+Iw4ULD4xFSyECNI5AnbU9R8itxZ43GSGYqBYw
         rpfZ3B/QjKuCYR0IoakaV2dfDhNsC2YDda0/c/eg1ZcCAkHivnZzGcmfZUC+SlH9/tYO
         6yZcTWicSVvndLOZFLaiO6ZgyC1VuqK+AB4Kgvw2pS30hONKFGt612xUuHvS8v/w4tDA
         07Ag==
X-Gm-Message-State: AOJu0YwxQx3gNODqR3fRjituKynuoJEOBRMdsz0Pc1eG3as4aWXJ8HK+
	I1agSNJC6iDxEHfejf0XlW8HARxiQQFekIoWdRgCcjhBHoUMpYIXqplBi6mG42kT3J8pQKtC5iM
	q
X-Google-Smtp-Source: AGHT+IG8gv5M6eQdPRaW5RagjFQLPIwyOi6pGwQ/53qXmVlAW9nj+CYDAo9G7aWsdcVk84ybzUMJUQ==
X-Received: by 2002:a17:906:2bc4:b0:a59:ac10:9be5 with SMTP id a640c23a62f3a-a5a2d292b1dmr351577666b.27.1715380805809;
        Fri, 10 May 2024 15:40:05 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>
Subject: [PATCH 2/4] tools/xen-cpuid: Rename decodes[] to leaf_info[]
Date: Fri, 10 May 2024 23:40:00 +0100
Message-Id: <20240510224002.2324578-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Roger Pau Monné <roger.pau@citrix.com>

Split out of subsequent patch to aid legibility.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * New (split out)
---
 tools/misc/xen-cpuid.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 8893547bebce..6ee835b22949 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -268,8 +268,7 @@ static const struct {
     const char *name;
     const char *abbr;
     const char *const *strs;
-} decodes[] =
-{
+} leaf_info[] = {
     { "CPUID 0x00000001.edx",        "1d", str_1d },
     { "CPUID 0x00000001.ecx",        "1c", str_1c },
     { "CPUID 0x80000001.edx",       "e1d", str_e1d },
@@ -336,11 +335,11 @@ static void decode_featureset(const uint32_t *features,
     if ( !detail )
         return;
 
-    for ( i = 0; i < length && i < ARRAY_SIZE(decodes); ++i )
+    for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
     {
-        printf("  [%02u] %-"COL_ALIGN"s", i, decodes[i].name ?: "<UNKNOWN>");
-        if ( decodes[i].name )
-            dump_leaf(features[i], decodes[i].strs);
+        printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
+        if ( leaf_info[i].name )
+            dump_leaf(features[i], leaf_info[i].strs);
         printf("\n");
     }
 }
@@ -355,8 +354,8 @@ static void dump_info(xc_interface *xch, bool detail)
     if ( !detail )
     {
         printf("       %"COL_ALIGN"s ", "KEY");
-        for ( i = 0; i < ARRAY_SIZE(decodes); ++i )
-            printf("%-8s ", decodes[i].abbr ?: "???");
+        for ( i = 0; i < ARRAY_SIZE(leaf_info); ++i )
+            printf("%-8s ", leaf_info[i].abbr ?: "???");
         printf("\n");
     }
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 22:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720097.1123127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuS-0000fd-Nh; Fri, 10 May 2024 22:40:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720097.1123127; Fri, 10 May 2024 22:40:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuS-0000fY-Js; Fri, 10 May 2024 22:40:12 +0000
Received: by outflank-mailman (input) for mailman id 720097;
 Fri, 10 May 2024 22:40:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YuQ-0000RL-UH
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:40:10 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40e7ea54-0f1e-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 00:40:08 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a64db066so661797066b.3
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:40:08 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm233915866b.14.2024.05.10.15.40.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 15:40:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40e7ea54-0f1e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715380807; x=1715985607; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NZZNtujjN9pYr6HT0e0A2/j3Mym7RChiLwN22jfYFXQ=;
        b=dMcczuGdoQatw4rz5HetC7jd1Ll80NHUcG8fkCjdn2wstbs3XH4089I61pI+j1yc+x
         9byQh0qXw3hHgrwZaJq9ChMZ7nkNmYZD/FIits5YxxClNmWb+dfmewkP3Ar8Ts3q6FBW
         CmVJ/pAyDNoLlipGv4auyRxSRz5svuz9779Tg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715380807; x=1715985607;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=NZZNtujjN9pYr6HT0e0A2/j3Mym7RChiLwN22jfYFXQ=;
        b=q/rBjAmNnQf+6nmPw0mH8EBedFvannOaxG9OwL68IJXTQCuxoDXpV4eunR4q4bIcvh
         CAE9fz4K9Knb4w1adgbaJxXFOZRpNZYE3VhhPQhUkpE32C6zAcaCkrc74SWM9+vyWxpA
         v/WYMKWrNrW9kuUCuG5oIwnOx+QYcti1+Uvu8WetWiYR+LPPcbfW5NRN1Ugl+oLSheRd
         cyGpXFaWCkd/tq3PCNfJUSWXmqSsx/7AoHa/6Wnt30dp9Maba6uOHAOHOP4fVoGPpmcT
         MJ3oqnSyx9gBoAT0xinOiLGO+HXVudOqVVNe6M6dqyZOVjVWy4EJlzyAy4YXRaqzqv6t
         +xgA==
X-Gm-Message-State: AOJu0YywnCw8MX1NLb0HfgZAyMXE4TAb9gyFbKA6zw0FgHUOveAbw7+k
	kZWQBDRFhl9eSGD+DYURm7TYtD+L8tS4PkbtrlxsZal6hvGNbu1jRHABwXaDrm1xBa9rom+F5+Q
	M
X-Google-Smtp-Source: AGHT+IFXXDmaCK9fx4msP1oV+MH9H9mEiBmmVYHwccM8SXOciaaLj3tBQWdnSEKuMH8C/e8C2XT1iw==
X-Received: by 2002:a17:906:1d16:b0:a59:dd91:3417 with SMTP id a640c23a62f3a-a5a2d53b132mr250376266b.5.1715380807400;
        Fri, 10 May 2024 15:40:07 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>
Subject: [PATCH 4/4] tools/xen-cpuid: Drop old names
Date: Fri, 10 May 2024 23:40:02 +0100
Message-Id: <20240510224002.2324578-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Roger Pau Monné <roger.pau@citrix.com>

Not used any more.  Split out of previous patch to aid legibility.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * New (split out)
---
 tools/misc/xen-cpuid.c | 287 +++--------------------------------------
 1 file changed, 18 insertions(+), 269 deletions(-)

diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 2f34694e9c57..2cb5322ed9aa 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -15,279 +15,28 @@
 
 static uint32_t nr_features;
 
-static const char *const str_1d[32] =
-{
-    [ 0] = "fpu",  [ 1] = "vme",
-    [ 2] = "de",   [ 3] = "pse",
-    [ 4] = "tsc",  [ 5] = "msr",
-    [ 6] = "pae",  [ 7] = "mce",
-    [ 8] = "cx8",  [ 9] = "apic",
-    /* [10] */     [11] = "sysenter",
-    [12] = "mtrr", [13] = "pge",
-    [14] = "mca",  [15] = "cmov",
-    [16] = "pat",  [17] = "pse36",
-    [18] = "psn",  [19] = "clflush",
-    /* [20] */     [21] = "ds",
-    [22] = "acpi", [23] = "mmx",
-    [24] = "fxsr", [25] = "sse",
-    [26] = "sse2", [27] = "ss",
-    [28] = "htt",  [29] = "tm",
-    [30] = "ia64", [31] = "pbe",
-};
-
-static const char *const str_1c[32] =
-{
-    [ 0] = "sse3",    [ 1] = "pclmulqdq",
-    [ 2] = "dtes64",  [ 3] = "monitor",
-    [ 4] = "ds-cpl",  [ 5] = "vmx",
-    [ 6] = "smx",     [ 7] = "est",
-    [ 8] = "tm2",     [ 9] = "ssse3",
-    [10] = "cntx-id", [11] = "sdgb",
-    [12] = "fma",     [13] = "cx16",
-    [14] = "xtpr",    [15] = "pdcm",
-    /* [16] */        [17] = "pcid",
-    [18] = "dca",     [19] = "sse41",
-    [20] = "sse42",   [21] = "x2apic",
-    [22] = "movebe",  [23] = "popcnt",
-    [24] = "tsc-dl",  [25] = "aesni",
-    [26] = "xsave",   [27] = "osxsave",
-    [28] = "avx",     [29] = "f16c",
-    [30] = "rdrnd",   [31] = "hyper",
-};
-
-static const char *const str_e1d[32] =
-{
-    [ 0] = "fpu",    [ 1] = "vme",
-    [ 2] = "de",     [ 3] = "pse",
-    [ 4] = "tsc",    [ 5] = "msr",
-    [ 6] = "pae",    [ 7] = "mce",
-    [ 8] = "cx8",    [ 9] = "apic",
-    /* [10] */       [11] = "syscall",
-    [12] = "mtrr",   [13] = "pge",
-    [14] = "mca",    [15] = "cmov",
-    [16] = "fcmov",  [17] = "pse36",
-    /* [18] */       [19] = "mp",
-    [20] = "nx",     /* [21] */
-    [22] = "mmx+",   [23] = "mmx",
-    [24] = "fxsr",   [25] = "fxsr+",
-    [26] = "pg1g",   [27] = "rdtscp",
-    /* [28] */       [29] = "lm",
-    [30] = "3dnow+", [31] = "3dnow",
-};
-
-static const char *const str_e1c[32] =
-{
-    [ 0] = "lahf-lm",    [ 1] = "cmp",
-    [ 2] = "svm",        [ 3] = "extapic",
-    [ 4] = "cr8d",       [ 5] = "lzcnt",
-    [ 6] = "sse4a",      [ 7] = "msse",
-    [ 8] = "3dnowpf",    [ 9] = "osvw",
-    [10] = "ibs",        [11] = "xop",
-    [12] = "skinit",     [13] = "wdt",
-    /* [14] */           [15] = "lwp",
-    [16] = "fma4",       [17] = "tce",
-    /* [18] */           [19] = "nodeid",
-    /* [20] */           [21] = "tbm",
-    [22] = "topoext",    [23] = "perfctr-core",
-    [24] = "perfctr-nb", /* [25] */
-    [26] = "dbx",        [27] = "perftsc",
-    [28] = "pcx-l2i",    [29] = "monitorx",
-    [30] = "addr-msk-ext",
-};
-
-static const char *const str_7b0[32] =
-{
-    [ 0] = "fsgsbase", [ 1] = "tsc-adj",
-    [ 2] = "sgx",      [ 3] = "bmi1",
-    [ 4] = "hle",      [ 5] = "avx2",
-    [ 6] = "fdp-exn",  [ 7] = "smep",
-    [ 8] = "bmi2",     [ 9] = "erms",
-    [10] = "invpcid",  [11] = "rtm",
-    [12] = "pqm",      [13] = "depfpp",
-    [14] = "mpx",      [15] = "pqe",
-    [16] = "avx512f",  [17] = "avx512dq",
-    [18] = "rdseed",   [19] = "adx",
-    [20] = "smap",     [21] = "avx512-ifma",
-    [22] = "pcommit",  [23] = "clflushopt",
-    [24] = "clwb",     [25] = "proc-trace",
-    [26] = "avx512pf", [27] = "avx512er",
-    [28] = "avx512cd", [29] = "sha",
-    [30] = "avx512bw", [31] = "avx512vl",
-};
-
-static const char *const str_Da1[32] =
-{
-    [ 0] = "xsaveopt", [ 1] = "xsavec",
-    [ 2] = "xgetbv1",  [ 3] = "xsaves",
-};
-
-static const char *const str_7c0[32] =
-{
-    [ 0] = "prefetchwt1",      [ 1] = "avx512-vbmi",
-    [ 2] = "umip",             [ 3] = "pku",
-    [ 4] = "ospke",            [ 5] = "waitpkg",
-    [ 6] = "avx512-vbmi2",     [ 7] = "cet-ss",
-    [ 8] = "gfni",             [ 9] = "vaes",
-    [10] = "vpclmulqdq",       [11] = "avx512-vnni",
-    [12] = "avx512-bitalg",
-    [14] = "avx512-vpopcntdq",
-
-    [22] = "rdpid",
-    /* 24 */                   [25] = "cldemote",
-    /* 26 */                   [27] = "movdiri",
-    [28] = "movdir64b",        [29] = "enqcmd",
-    [30] = "sgx-lc",           [31] = "pks",
-};
-
-static const char *const str_e7d[32] =
-{
-    /* 6 */                    [ 7] = "hw-pstate",
-    [ 8] = "itsc",             [ 9] = "cpb",
-    [10] = "efro",
-};
-
-static const char *const str_e8b[32] =
-{
-    [ 0] = "clzero",
-    [ 2] = "rstr-fp-err-ptrs",
-
-    /* [ 8] */                 [ 9] = "wbnoinvd",
-
-    [12] = "ibpb",
-    [14] = "ibrs",             [15] = "amd-stibp",
-    [16] = "ibrs-always",      [17] = "stibp-always",
-    [18] = "ibrs-fast",        [19] = "ibrs-same-mode",
-
-    [20] = "no-lmsl",
-    /* [22] */                 [23] = "ppin",
-    [24] = "amd-ssbd",         [25] = "virt-ssbd",
-    [26] = "ssb-no",
-    [28] = "psfd",             [29] = "btc-no",
-    [30] = "ibpb-ret",
-};
-
-static const char *const str_7d0[32] =
-{
-    [ 2] = "avx512-4vnniw", [ 3] = "avx512-4fmaps",
-    [ 4] = "fsrm",
-
-    [ 8] = "avx512-vp2intersect", [ 9] = "srbds-ctrl",
-    [10] = "md-clear",            [11] = "rtm-always-abort",
-    /* 12 */                [13] = "tsx-force-abort",
-    [14] = "serialize",     [15] = "hybrid",
-    [16] = "tsxldtrk",
-    [18] = "pconfig",
-    [20] = "cet-ibt",
-    /* 22 */                [23] = "avx512-fp16",
-
-    [26] = "ibrsb",         [27] = "stibp",
-    [28] = "l1d-flush",     [29] = "arch-caps",
-    [30] = "core-caps",     [31] = "ssbd",
-};
-
-static const char *const str_7a1[32] =
-{
-    [ 0] = "sha512",        [ 1] = "sm3",
-    [ 2] = "sm4",
-    [ 4] = "avx-vnni",      [ 5] = "avx512-bf16",
-
-    [10] = "fzrm",          [11] = "fsrs",
-    [12] = "fsrcs",
-
-    /* 18 */                [19] = "wrmsrns",
-
-    /* 22 */                [23] = "avx-ifma",
-};
-
-static const char *const str_e21a[32] =
-{
-    [ 0] = "no-nest-bp",    [ 1] = "fs-gs-ns",
-    [ 2] = "lfence+",
-    [ 6] = "nscb",
-    [ 8] = "auto-ibrs",
-    [10] = "amd-fsrs",      [11] = "amd-fsrc",
-
-    /* 16 */                [17] = "cpuid-user-dis",
-    [18] = "epsf",          [19] = "fsrsc",
-    [20] = "amd-prefetchi",
-
-    /* 26 */                [27] = "sbpb",
-    [28] = "ibpb-brtype",   [29] = "srso-no",
-};
-
-static const char *const str_7b1[32] =
-{
-    [ 0] = "ppin",
-};
-
-static const char *const str_7c1[32] =
-{
-};
-
-static const char *const str_7d1[32] =
-{
-    [ 4] = "avx-vnni-int8",       [ 5] = "avx-ne-convert",
-
-    [10] = "avx-vnni-int16",
-
-    [14] = "prefetchi",
-
-    [18] = "cet-sss",
-};
-
-static const char *const str_7d2[32] =
-{
-    [ 0] = "intel-psfd",    [ 1] = "ipred-ctrl",
-    [ 2] = "rrsba-ctrl",    [ 3] = "ddp-ctrl",
-    [ 4] = "bhi-ctrl",      [ 5] = "mcdt-no",
-};
-
-static const char *const str_m10Al[32] =
-{
-    [ 0] = "rdcl-no",             [ 1] = "eibrs",
-    [ 2] = "rsba",                [ 3] = "skip-l1dfl",
-    [ 4] = "intel-ssb-no",        [ 5] = "mds-no",
-    [ 6] = "if-pschange-mc-no",   [ 7] = "tsx-ctrl",
-    [ 8] = "taa-no",              [ 9] = "mcu-ctrl",
-    [10] = "misc-pkg-ctrl",       [11] = "energy-ctrl",
-    [12] = "doitm",               [13] = "sbdr-ssdp-no",
-    [14] = "fbsdp-no",            [15] = "psdp-no",
-    /* 16 */                      [17] = "fb-clear",
-    [18] = "fb-clear-ctrl",       [19] = "rrsba",
-    [20] = "bhi-no",              [21] = "xapic-status",
-    /* 22 */                      [23] = "ovrclk-status",
-    [24] = "pbrsb-no",            [25] = "gds-ctrl",
-    [26] = "gds-no",              [27] = "rfds-no",
-    [28] = "rfds-clear",
-};
-
-static const char *const str_m10Ah[32] =
-{
-};
-
 static const struct {
     const char *name;
     const char *abbr;
-    const char *const *strs;
 } leaf_info[FEATURESET_NR_ENTRIES] = {
-    { "CPUID 0x00000001.edx",        "1d", str_1d },
-    { "CPUID 0x00000001.ecx",        "1c", str_1c },
-    { "CPUID 0x80000001.edx",       "e1d", str_e1d },
-    { "CPUID 0x80000001.ecx",       "e1c", str_e1c },
-    { "CPUID 0x0000000d:1.eax",     "Da1", str_Da1 },
-    { "CPUID 0x00000007:0.ebx",     "7b0", str_7b0 },
-    { "CPUID 0x00000007:0.ecx",     "7c0", str_7c0 },
-    { "CPUID 0x80000007.edx",       "e7d", str_e7d },
-    { "CPUID 0x80000008.ebx",       "e8b", str_e8b },
-    { "CPUID 0x00000007:0.edx",     "7d0", str_7d0 },
-    { "CPUID 0x00000007:1.eax",     "7a1", str_7a1 },
-    { "CPUID 0x80000021.eax",      "e21a", str_e21a },
-    { "CPUID 0x00000007:1.ebx",     "7b1", str_7b1 },
-    { "CPUID 0x00000007:2.edx",     "7d2", str_7d2 },
-    { "CPUID 0x00000007:1.ecx",     "7c1", str_7c1 },
-    { "CPUID 0x00000007:1.edx",     "7d1", str_7d1 },
-    { "MSR_ARCH_CAPS.lo",         "m10Al", str_m10Al },
-    { "MSR_ARCH_CAPS.hi",         "m10Ah", str_m10Ah },
+    { "CPUID 0x00000001.edx",        "1d" },
+    { "CPUID 0x00000001.ecx",        "1c" },
+    { "CPUID 0x80000001.edx",       "e1d" },
+    { "CPUID 0x80000001.ecx",       "e1c" },
+    { "CPUID 0x0000000d:1.eax",     "Da1" },
+    { "CPUID 0x00000007:0.ebx",     "7b0" },
+    { "CPUID 0x00000007:0.ecx",     "7c0" },
+    { "CPUID 0x80000007.edx",       "e7d" },
+    { "CPUID 0x80000008.ebx",       "e8b" },
+    { "CPUID 0x00000007:0.edx",     "7d0" },
+    { "CPUID 0x00000007:1.eax",     "7a1" },
+    { "CPUID 0x80000021.eax",      "e21a" },
+    { "CPUID 0x00000007:1.ebx",     "7b1" },
+    { "CPUID 0x00000007:2.edx",     "7d2" },
+    { "CPUID 0x00000007:1.ecx",     "7c1" },
+    { "CPUID 0x00000007:1.edx",     "7d1" },
+    { "MSR_ARCH_CAPS.lo",         "m10Al" },
+    { "MSR_ARCH_CAPS.hi",         "m10Ah" },
 };
 
 #define COL_ALIGN "24"
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 22:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720093.1123086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuN-00087K-Qf; Fri, 10 May 2024 22:40:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720093.1123086; Fri, 10 May 2024 22:40:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YuN-00087D-Ny; Fri, 10 May 2024 22:40:07 +0000
Received: by outflank-mailman (input) for mailman id 720093;
 Fri, 10 May 2024 22:40:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YuM-000872-TH
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:40:06 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f19a453-0f1e-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 00:40:05 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a2d0d8644so418110666b.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:40:05 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm233915866b.14.2024.05.10.15.40.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 May 2024 15:40:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f19a453-0f1e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715380805; x=1715985605; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=oVrYVxNngnuJ5RDy2Fy0Qi7znwDB3526BZ29njRt168=;
        b=cknb/XomjDIKWLt7V/gkk3Xol8QW3mKuKn5GBqV4eqHsBYBcS6NEyvXF5xRSM5PVQg
         80QnHol4raT8Qa4IeDoOFY7fU9d+LIp+FUanrvWIlnhGaxwakyObWtXdyFs9o4yKJuq6
         JawwtWKBl0eOsTThOfmHbl6NehNrY5nW7QxMg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715380805; x=1715985605;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=oVrYVxNngnuJ5RDy2Fy0Qi7znwDB3526BZ29njRt168=;
        b=czYL7ob1vdwuPoTrRkypLnSAEwiQsM/ofv9X5Ib96YvYrU8JgCun1Ndv4wqG7ymvDV
         NfR2Vy7QBD08ce2nxv3eUG4hjrHmYWOhZ+62wPo28+VkQf+H3RHSc9Eq/kYL4tQXOcyC
         2OC/hOAeZqKkCECkDnLsnWGSv8DHv5j73HtKgFx8WNiePx6sBYML5c/atnLlIFrUaX/K
         F4XnSXCAAgFf3CQB+/0JEzwISawOigxP14F32qaKGlm+zU1LP4xEWfNxu/tu5kevrtMf
         MnrEPRUBDp9zk2UkvF1hbvZh+rpyWwjxENubXigOcaktt3TmxduYpzXaJ/UIgZo7PHKT
         d4AA==
X-Gm-Message-State: AOJu0YxOuyIf4hBCb7muMmECk8lnjvXPyqTplfZWV0ivEr6EOHUPjsbd
	BxG68fkB6Gh6lzM4tw1ch/K1sRQJubQHBOpbwvos59B0/dsUEEc0i75/JBRSWNBYGH45o6na8A6
	B
X-Google-Smtp-Source: AGHT+IEqKVwst80V0QidpfCtbT9KEbGXMQWPcjZlg/AJ7Y7/GxJ2igY6pXaxqdNYEE6qkBlnqVZQ0w==
X-Received: by 2002:a17:907:d25:b0:a5a:2aed:ca2b with SMTP id a640c23a62f3a-a5a2d5c9fdcmr324910766b.28.1715380804614;
        Fri, 10 May 2024 15:40:04 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v3 0/4] x86/xen-cpuid: Use automagically generated names
Date: Fri, 10 May 2024 23:39:58 +0100
Message-Id: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

I may have got a bit carried away tweaking Roger's v2.

The major bugfix is getting both alias of common_1d, which matters when using
xen-cpuid on AMD hardware.

Andrew Cooper (1):
  x86/gen-cpuid: Minor cleanup

Roger Pau Monné (3):
  tools/xen-cpuid: Rename decodes[] to leaf_info[]
  tools/xen-cpuid: Use automatically generated feature names
  tools/xen-cpuid: Drop old names

 tools/libs/light/libxl_cpuid.c |   2 +-
 tools/misc/xen-cpuid.c         | 311 +++------------------------------
 xen/arch/x86/cpu-policy.c      |   2 +-
 xen/tools/gen-cpuid.py         |  30 +++-
 4 files changed, 53 insertions(+), 292 deletions(-)


base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 10 22:45:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 22:45:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720112.1123136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YzV-00039H-IB; Fri, 10 May 2024 22:45:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720112.1123136; Fri, 10 May 2024 22:45:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5YzV-00039A-Fc; Fri, 10 May 2024 22:45:25 +0000
Received: by outflank-mailman (input) for mailman id 720112;
 Fri, 10 May 2024 22:45:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i6G=MN=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5YzU-000392-A7
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 22:45:24 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fbc5d350-0f1e-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 00:45:22 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41a72f3a20dso17708135e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 May 2024 15:45:22 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccce24efsm78524385e9.19.2024.05.10.15.45.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 May 2024 15:45:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbc5d350-0f1e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715381121; x=1715985921; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Sn6WsIBQzi2paWx2AV+3jgJbQsaBnBIozeWKELO4BZU=;
        b=BFn+FueisxjxAw4xnWWTs+rHKR1eo/lTAl+lpsxSAi7zyM0BTbD286Q/YtPL+mDAKx
         p+JUU+9zaJp8lvajZ6X2l4HXqnE+cdSPjCwKt71ayorLkk2CaxzkDFmGD2E3y2tURkjP
         HbyZCXOykr3qzYfx+lhZLw0rcqtxMZoTnVeEs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715381121; x=1715985921;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Sn6WsIBQzi2paWx2AV+3jgJbQsaBnBIozeWKELO4BZU=;
        b=PaQaw+h3465u/OMRpvDTCKwLc9Zr2m2tKN8/sYbxnQyYj6Ls80E/aulbDVdGs9eeJg
         f6tMFNv+9ZqrF3vnixsdNJ2yvyaIzOEWySbxGVPmjWToeZu84ywFYBPxaPcLnxFGzD1x
         202YQ1OT+dEgvSArGxo10lW+sG2t5/LMEG/9ciCOtVk27+rLkpicxbh1lI8MY1qZLD4Y
         PXIBig4t9DkWXFeac5gMq5qALQl0d6EAfCufj7eJeg90/2K+fv9CsGhSWyDvhrIQ4h7B
         9IX9VYSRjzCQF6kFzNWMLtJzSbmynn5YxAcoggErISIf7LoE39AyEYjh4zWORwc8Trg9
         2ZgA==
X-Forwarded-Encrypted: i=1; AJvYcCXhi6zWTODcXNdvwNY1PxHlZ7/4sD6ktntuDZ+cr9dUCi0JhoHF0okt2WOPTRTllu+CqzHUZnEEbBfgJUxFfdxo9jP2//qM8UrKgv9/g+I=
X-Gm-Message-State: AOJu0Yxpz0sCmHbkjcMspFxYQbX4yqX1DNPtGYOf6v7X/4oO/ZpOXR3X
	o6btXu9dKPNTx1v9AZsXN8gvq+eI7BOx19GNzGlU8pdrxR8KBNHPeP0vayt9X+c=
X-Google-Smtp-Source: AGHT+IHLax7T17bbvizcPdXomnhVe6GKAZmA6+qk9CH8eom9zoZKay3NOCDtz1jUPjv8wq4BkaIyRw==
X-Received: by 2002:a05:600c:1f93:b0:417:d4f5:6501 with SMTP id 5b1f17b1804b1-41feac5a3b8mr29511265e9.27.1715381121468;
        Fri, 10 May 2024 15:45:21 -0700 (PDT)
Message-ID: <be534982-5b4f-4f90-8484-faf09e44a618@citrix.com>
Date: Fri, 10 May 2024 23:45:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] docs/misra: add R21.6 R21.9 R21.10 R21.14 R21.15
 R21.16
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <alpine.DEB.2.22.394.2404261435010.3940@ubuntu-linux-20-04-desktop>
 <cf0cd2ef-58df-476f-89c5-c843a85f71f7@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <cf0cd2ef-58df-476f-89c5-c843a85f71f7@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 10/05/2024 10:48 pm, Andrew Cooper wrote:
> On 26/04/2024 10:36 pm, Stefano Stabellini wrote:
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>> ---
>>
>> Changes in v3:
>> - add explanation in footnote
>> - remove comment from 21.14, 21.15, 21.16
>>
>>  docs/misra/rules.rst | 42 ++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 42 insertions(+)
>>
>> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
>> index b7b447e152..5ba7394f05 100644
>> --- a/docs/misra/rules.rst
>> +++ b/docs/misra/rules.rst
>> @@ -652,12 +652,48 @@ maintainers if you want to suggest a change.
>>         declared
>>       - See comment for Rule 21.1
>>  
>> +   * - `Rule 21.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_06.c>`_
>> +     - Required
>> +     - The Standard Library input/output routines shall not be used
>> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
>> +
>> +   * - `Rule 21.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_09.c>`_
>> +     - Required
>> +     - The library functions bsearch and qsort of <stdlib.h> shall not be used
>> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
>> +
>> +   * - `Rule 21.10 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_10.c>`_
>> +     - Required
>> +     - The Standard Library time and date routines shall not be used
>> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
>> +
>>     * - `Rule 21.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_13.c>`_
>>       - Mandatory
>>       - Any value passed to a function in <ctype.h> shall be representable as an
>>         unsigned char or be the value EOF
>>       -
>>  
>> +   * - `Rule 21.14 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_14.c>`_
>> +     - Required
>> +     - The Standard Library function memcmp shall not be used to compare
>> +       null terminated strings
>> +     -
>> +
>> +   * - `Rule 21.15 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_15.c>`_
>> +     - Required
>> +     - The pointer arguments to the Standard Library functions memcpy,
>> +       memmove and memcmp shall be pointers to qualified or unqualified
>> +       versions of compatible types
>> +     -
>> +
>> +   * - `Rule 21.16 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_16.c>`_
>> +     - Required
>> +     - The pointer arguments to the Standard Library function memcmp
>> +       shall point to either a pointer type, an essentially signed type,
>> +       an essentially unsigned type, an essentially Boolean type or an
>> +       essentially enum type
>> +     - void* arguments are allowed
>> +
>>     * - `Rule 21.17 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_17.c>`_
>>       - Mandatory
>>       - Use of the string handling functions from <string.h> shall not result in
>> @@ -712,3 +748,9 @@ maintainers if you want to suggest a change.
>>       - The value of a pointer to a FILE shall not be used after the associated
>>         stream has been closed
>>       -
>> +
>> +
>> +.. [1] Xen implements itself a few functions with names that match the
>> +       corresponding function names of the Standard Library for developers'
>> +       convenience. These functions are part of the Xen code and subject to
>> +       analysis.
> I discovered the hard way (c/s 1357f1fa588f1) that numbered footnotes
> are an issue when the document gets more complicated.
>
> I'd suggest using #xen-stdlib as a named footnote instead of [1].
>
> Can fix on commit.

And a ".. rubric:: Footnotes" is wanted too.  Fixed on commit, after
conferring with Stefano.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 10 23:33:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 23:33:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720125.1123146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5ZjO-0002jX-PH; Fri, 10 May 2024 23:32:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720125.1123146; Fri, 10 May 2024 23:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5ZjO-0002jQ-MC; Fri, 10 May 2024 23:32:50 +0000
Received: by outflank-mailman (input) for mailman id 720125;
 Fri, 10 May 2024 23:32:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pcT1=MN=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s5ZjN-0002jK-Na
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 23:32:49 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9a99d379-0f25-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 01:32:47 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 3FF69CE1F15;
 Fri, 10 May 2024 23:32:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8014BC113CC;
 Fri, 10 May 2024 23:32:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a99d379-0f25-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715383962;
	bh=VFyqvmvkheA5VgmNYOdDhu9yAtABDX1w7nVHdXCv7do=;
	h=Date:From:To:cc:Subject:From;
	b=t0Tg5HnuM3T+kbZiaFHtDFvoWj/6KOkQldZ4+kmkGkGpbVBB+H1zBCT7a7neFJPL8
	 84uu18y860jBla1wL0V/SvxdaPc1/8xY4p19meNFzrPxYlGweEAyAWH2ltuGMKym75
	 d/+JTBvIDliPHxHxLXd22Ctcpegq3gpk9Cl3NlQkOF+F8qK80WssdBUbeQXZPQdce/
	 ECIdop3OdvyjjXRC+XI/7lsLhUGdJHr8ZX90hUxvqU3a62qUeQLAVHCOnRZFJmfmFA
	 GoTpqoSm+Xqv9gZ7GVNK0rQAz+kbe+9TWzBPlOO/m1LsWXYbej8zAp+ajczhfg0wQ5
	 y/vA6z061yhjQ==
Date: Fri, 10 May 2024 16:32:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, 
    julien@xen.org, michal.orzel@amd.com, bertrand.marquis@arm.com, 
    roger.pau@citrix.com
Subject: [PATCH] docs/misra: replace R13.1 with R13.2
Message-ID: <alpine.DEB.2.22.394.2405101625470.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

After looking closely at the R13.1 violations and potential fixes or
deviations, upon further reflection and discussions, we realized that it
is R13.2, limited to initializers list, that we want to comply with.

Link: https://marc.info/?l=xen-devel&m=170751643325805
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index d3b70fdf04..c02dea9af8 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -449,10 +449,11 @@ maintainers if you want to suggest a change.
        parameter declared as "array of type"
      -
 
-   * - `Rule 13.1 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_13_01_1.c>`_
+   * - `Rule 13.2 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_13_02.c>`_
      - Required
-     - Initializer lists shall not contain persistent side effects
-     -
+     - The value of an expression and its persistent side-effects shall
+       be the same under all permitted evaluation orders
+     - Limit the scope of this rule to initializer lists
 
    * - `Rule 13.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_13_06.c>`_
      - Required


From xen-devel-bounces@lists.xenproject.org Fri May 10 23:37:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 May 2024 23:37:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720128.1123157 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Znj-0003sV-Au; Fri, 10 May 2024 23:37:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720128.1123157; Fri, 10 May 2024 23:37:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5Znj-0003sO-6b; Fri, 10 May 2024 23:37:19 +0000
Received: by outflank-mailman (input) for mailman id 720128;
 Fri, 10 May 2024 23:37:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pcT1=MN=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s5Zni-0003s2-7O
 for xen-devel@lists.xenproject.org; Fri, 10 May 2024 23:37:18 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3b5c99a0-0f26-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 01:37:17 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 66609CE1F22;
 Fri, 10 May 2024 23:37:14 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B93DC113CC;
 Fri, 10 May 2024 23:37:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b5c99a0-0f26-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715384233;
	bh=jywT3wBiVd/k0saOAsMgRhZQlshpedfhcqSBOO7XSo8=;
	h=Date:From:To:cc:Subject:From;
	b=K/7EKbMzmOGrUOGebLZVEW498lySajGEkl+yNbCmMK/vuyxSynTkDIJjS8Qln7rFZ
	 ppXY/I64SRXaq8+qqVo32M+B0bqA+glspr0mkTlG9IjH1laOQKDGdYJgEK7vf8ARME
	 QTUY4DatJ4I+xWPhyqLgFHmqfMrK41BQphKWahZwfskngOzIN9kE8IdOYQRaBnG16F
	 kLUrfKgsTH3uGoiPWAkyhqclpVgKRM3VoxZIBIEt0bHSrur7P7n9vTyZflscgCd3Yn
	 InOrxZ/ax+n9SoYkg6mHaZBKgiOSuV1Nfughyu9cZAdm8PrIwZ6Y1C3/WMsaGtiohV
	 v6vEqvnbKmYBQ==
Date: Fri, 10 May 2024 16:37:11 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, michal.orzel@amd.com
Subject: [PATCH] fix Rule 10.2 violation
Message-ID: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Change opt_conswitch to char to fix a violation of Rule 10.2.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 2c363d9c1d..3a3a97bcbe 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -49,7 +49,7 @@ string_param("console", opt_console);
 /* Char 1: CTRL+<char1> is used to switch console input between Xen and DOM0 */
 /* Char 2: If this character is 'x', then do not auto-switch to DOM0 when it */
 /*         boots. Any other value, or omitting the char, enables auto-switch */
-static unsigned char __read_mostly opt_conswitch[3] = "a";
+static char __read_mostly opt_conswitch[3] = "a";
 string_runtime_param("conswitch", opt_conswitch);
 
 /* sync_console: force synchronous console output (useful for debugging). */


From xen-devel-bounces@lists.xenproject.org Sat May 11 00:56:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 00:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720146.1123231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2P-00087j-9w; Sat, 11 May 2024 00:56:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720146.1123231; Sat, 11 May 2024 00:56:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2P-00086q-58; Sat, 11 May 2024 00:56:33 +0000
Received: by outflank-mailman (input) for mailman id 720146;
 Sat, 11 May 2024 00:56:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5b2N-0007p5-CT
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 00:56:31 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4cb18fb0-0f31-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 02:56:29 +0200 (CEST)
Received: from PH8PR07CA0047.namprd07.prod.outlook.com (2603:10b6:510:2cf::10)
 by DM6PR12MB4218.namprd12.prod.outlook.com (2603:10b6:5:21b::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54; Sat, 11 May
 2024 00:56:22 +0000
Received: from CY4PEPF0000E9D6.namprd05.prod.outlook.com
 (2603:10b6:510:2cf:cafe::cb) by PH8PR07CA0047.outlook.office365.com
 (2603:10b6:510:2cf::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54 via Frontend
 Transport; Sat, 11 May 2024 00:56:21 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D6.mail.protection.outlook.com (10.167.241.69) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 00:56:20 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:19 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:19 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 19:56:16 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cb18fb0-0f31-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=To37Un+0ohRQslENSwZeF92zxH4tKGvRGUqkAK7QMpuiaE3ww8wTNQdl6gAYuUGNoRrHyvaRF9eaoigbFEes0iKz7VKzje1ujcd/ZGhQ/E33K3+2ZtUdignTBE2BlObDZ/8j1u0gn5fGCPg7hzPoj9RuRdjHfgwlj9h8zxXtrGEvOS6DDfXOGOTXk0DKY1bQnVWrReUKHKeTTJSQhjVgLyzl0F8M/A5D2Lz4arRsuWSwg/P41tdKayL4zBnd0e4/vN+G97p40aX6ai2bWw9H9VXoua8vIS3wiGWq+q98vvs77OU/X4sVLKQtdzVvPjIH75Z2ejmp9eu84Fq+5BLrZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Lk6GKSsCklZm9WOlSrI/19kZkT/LFQhP7ioFjW6GXdg=;
 b=ho15/4FRrUErZvFa9XSuYtv8qMWNnIgRZea+0B6lWGNZCm4AEOxAVez8He5LpGhOmCe/H6OCvY4e+I2w4noUA+8PQzN5SMQcue2/fxOtx6bH08jaH/tChSqNVd9xBAJdOKkvyr3cCbQtlonGgoqI+3n0REhc7JODKw6lL48MuCv7is/f9vkO/vRhhA91MCW8SK9hU6dFuwiFDR/qfWHPl6kHW5YWit7y8YpULPFua04sNnsPeAx6vFpKN5uRFUlit9FsOD5/4kKbXVBzAvwXJ5jZf7IAvPOiTj0vjjDoOzq9lbVRRKgMtPJhuVi4lNLaNrTcfxvC0Jq1IzL9nQuaHg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lk6GKSsCklZm9WOlSrI/19kZkT/LFQhP7ioFjW6GXdg=;
 b=qOt+r6YCYQKvqUsUzxef8ks0l12kYD4YzyMIDC2fMTcmn+umm3BUXrHppPcelQdDW81/o0crCaRHaFvfmPLDQHhu8VHWc/AkAOM9fOemuktnujdw18zqcBQp4nftz8aKntowp7VkjXmAWamf7INQjFhn5BKJ1qwB/p1g4p1rnK8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Alec Kwapis <alec.kwapis@medtronic.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic pages for Dom0less DomUs
Date: Sat, 11 May 2024 08:56:08 +0800
Message-ID: <20240511005611.83125-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240511005611.83125-1-xin.wang2@amd.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D6:EE_|DM6PR12MB4218:EE_
X-MS-Office365-Filtering-Correlation-Id: 10690689-d50c-4993-d9fb-08dc71552bfc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?2fTIgIGKbmc+axpa+4GH5Zzw4dlsjAIBeaxogAJgMC3vgjw3xXEuLG1n7fqt?=
 =?us-ascii?Q?LYPDA+5RnKsKTtD8RQOQdydaTCyCpDLwFBmbVKNKwkd9gaYeVyH0pilXBuyi?=
 =?us-ascii?Q?NFPkXtdbk6N0uWnNaax4MElZa5tZfcdnvK9geWlOf2iNuz0WWyAK7uUbADYu?=
 =?us-ascii?Q?EAxnzTZTO8X3NwAw1WKnD3/cdzYAqsXhIpFTgDEtjNEd7WNXQFfqT35enNtG?=
 =?us-ascii?Q?dgDYQpKUjO9VkHYcZ13lvggRhD0TV0DzevrQ8U2aQbhL8mFMMCeCCPJuyDgF?=
 =?us-ascii?Q?TbLO/Wwy7K7hj9MG/v81q49VHVKI1texQOdhBzYWvSpguyGrZORyIeH/TvxB?=
 =?us-ascii?Q?HFTozrZFgfumV1Bu52d9H236rZPy7StC38KSLOZavxJ0F5b9wQWrqSza6EWp?=
 =?us-ascii?Q?bYXbY9tsUYSBw9KJvss6ER+FN8xh2ZHi3S3b6VHiUHIfhrklR7IkL61l32dk?=
 =?us-ascii?Q?iJDRJadLnKTDDhNS7pzT/yqDHWu/NCCpeERS9DAr2T8igho440qrrsePmM4S?=
 =?us-ascii?Q?BLwwwUPyFP2UuL62kzahUkqxEye87KK7KOBc+L7wLM4LIDRfCkhVzHqb0JEQ?=
 =?us-ascii?Q?RUNjSDDecgNVoNiMwhSKU+09AAFeBx/YO8eEi1ReGuRou+olqK5YfV6UhbmK?=
 =?us-ascii?Q?DnFSzY4kF17/S/eUjtsZUjxgh7Fj4e4GccURuifpvcGZsYvF+4T4tOVcTx+x?=
 =?us-ascii?Q?QuIBltcPySl4kOs+XfYfdF/mwGUuntpK27h5RWt8Fx58HjDaCKC+LvMRcFPl?=
 =?us-ascii?Q?7bIA1HMNDvifXZOXEsXLnWrMNi3YAr3SSQbc3Xg1ss4LwDElg8BmPpaE8TCz?=
 =?us-ascii?Q?L6XvZzswrsjgUm3RBWFt5hKLS4N+Hyhx46eqIiYeTabtOK50dIwGXCTEdmpQ?=
 =?us-ascii?Q?SW810yJO77RvowIAdAtcaC5QZpVMjlIkoS8Ng4F7jWnFAlsxIQqaqzKFcjl9?=
 =?us-ascii?Q?hMDYnGHamHK7w/qG/HLrN12Wo49lBh8gWj4tLsGMn3YPM/ybT2XGkWhgpcMv?=
 =?us-ascii?Q?sIgme9AYghW87YBLxAYjDHs9mHoIILBIrKPBe6WouqtJp4oBov7WMtGAXCwd?=
 =?us-ascii?Q?7zcSjilQTul0UKQu7gHXBnSyC6CRqKjTzYvmJy3WU7fHzUGDy6uBZsLxSVTm?=
 =?us-ascii?Q?ufJ55aW+fRCdChfhCZQdf5hL6KtV1MCeKBAAce2UCnK9H/8vLcCdmRpzrEW3?=
 =?us-ascii?Q?W/GhBKUe0A8NZygCnKDO2pMyviDPYGnvVfUt2o5aXPtnENUHtwAKgp7VXqRr?=
 =?us-ascii?Q?o/UjSgqPJhm56No8joZ12S0E5SBN7Yv/Fllcl48Q0z5xcE2z46SfgZkFc2eh?=
 =?us-ascii?Q?XectRlpo2mUzEH7+z4nBRApbnhPtEmWsUByBNubAC7gmhm570OtqI06MNp0h?=
 =?us-ascii?Q?WPfSCXszj1mdGKMx01mA/Zj2yEcC?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 00:56:20.2980
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 10690689-d50c-4993-d9fb-08dc71552bfc
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D6.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4218

There are use cases (for example using the PV driver) in Dom0less
setup that require Dom0less DomUs start immediately with Dom0, but
initialize XenStore later after Dom0's successful boot and call to
the init-dom0less application.

An error message can seen from the init-dom0less application on
1:1 direct-mapped domains:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

The "magic page" is a terminology used in the toolstack as reserved
pages for the VM to have access to virtual platform capabilities.
Currently the magic pages for Dom0less DomUs are populated by the
init-dom0less app through populate_physmap(), and populate_physmap()
automatically assumes gfn == mfn for 1:1 direct mapped domains. This
cannot be true for the magic pages that are allocated later from the
init-dom0less application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list is
empty at that time.

To solve above issue, this commit allocates hypervisor reserved pages
(currently used as the magic pages) for Arm Dom0less DomUs at the
domain construction time. The base address/PFN of the region will be
noted and communicated to the init-dom0less application in Dom0.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Reword the commit msg to explain what is "magic page" and use generic
  terminology "hypervisor reserved pages" in commit msg. (Daniel)
- Also move the offset definition of magic pages. (Michal)
- Extract the magic page allocation logic to a function. (Michal)
---
 tools/libs/guest/xg_dom_arm.c |  6 ------
 xen/arch/arm/dom0less-build.c | 32 ++++++++++++++++++++++++++++++++
 xen/include/public/arch-arm.h |  6 ++++++
 3 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
index 2fd8ee7ad4..8c579d7576 100644
--- a/tools/libs/guest/xg_dom_arm.c
+++ b/tools/libs/guest/xg_dom_arm.c
@@ -25,12 +25,6 @@
 
 #include "xg_private.h"
 
-#define NR_MAGIC_PAGES 4
-#define CONSOLE_PFN_OFFSET 0
-#define XENSTORE_PFN_OFFSET 1
-#define MEMACCESS_PFN_OFFSET 2
-#define VUART_PFN_OFFSET 3
-
 #define LPAE_SHIFT 9
 
 #define PFN_4K_SHIFT  (0)
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..4b96ddd9ce 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -739,6 +739,34 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
     return 0;
 }
 
+static int __init alloc_magic_pages(struct domain *d)
+{
+    struct page_info *magic_pg;
+    mfn_t mfn;
+    gfn_t gfn;
+    int rc;
+
+    d->max_pages += NR_MAGIC_PAGES;
+    magic_pg = alloc_domheap_pages(d, get_order_from_pages(NR_MAGIC_PAGES), 0);
+    if ( magic_pg == NULL )
+        return -ENOMEM;
+
+    mfn = page_to_mfn(magic_pg);
+    if ( !is_domain_direct_mapped(d) )
+        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
+    else
+        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
+
+    rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
+    if ( rc )
+    {
+        free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
+        return rc;
+    }
+
+    return 0;
+}
+
 static int __init construct_domU(struct domain *d,
                                  const struct dt_device_node *node)
 {
@@ -840,6 +868,10 @@ static int __init construct_domU(struct domain *d,
         if ( rc < 0 )
             return rc;
         d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
+
+        rc = alloc_magic_pages(d);
+        if ( rc < 0 )
+            return rc;
     }
 
     return rc;
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 289af81bd6..186520d01f 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
 #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
 #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
 
+#define NR_MAGIC_PAGES 4
+#define CONSOLE_PFN_OFFSET 0
+#define XENSTORE_PFN_OFFSET 1
+#define MEMACCESS_PFN_OFFSET 2
+#define VUART_PFN_OFFSET 3
+
 #define GUEST_RAM_BANKS   2
 
 /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 11 00:56:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 00:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720144.1123216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2N-0007pW-LS; Sat, 11 May 2024 00:56:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720144.1123216; Sat, 11 May 2024 00:56:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2N-0007pP-ID; Sat, 11 May 2024 00:56:31 +0000
Received: by outflank-mailman (input) for mailman id 720144;
 Sat, 11 May 2024 00:56:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5b2M-0007ah-BE
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 00:56:30 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4c0c8333-0f31-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 02:56:28 +0200 (CEST)
Received: from PH8P222CA0004.NAMP222.PROD.OUTLOOK.COM (2603:10b6:510:2d7::34)
 by DM4PR12MB5772.namprd12.prod.outlook.com (2603:10b6:8:63::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.49; Sat, 11 May
 2024 00:56:25 +0000
Received: from CY4PEPF0000E9D5.namprd05.prod.outlook.com
 (2603:10b6:510:2d7:cafe::2f) by PH8P222CA0004.outlook.office365.com
 (2603:10b6:510:2d7::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54 via Frontend
 Transport; Sat, 11 May 2024 00:56:24 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D5.mail.protection.outlook.com (10.167.241.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 00:56:24 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:24 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:23 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 19:56:22 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c0c8333-0f31-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OpPUu7rbvrb9YeD0Nu9zccpeG6gNCszERa4y0u9Gum8d+cSe8vR1tqI5L8Fo/aQE+EnlwQv1KKLfsmVArumYWjUJKeuC4/nRj03+JHFhCGihAY3xQn7UJX4Y6+y+l4Cv4RoBZopmYo1lZ3vKNIMpvS/x5IxZsTIMd2URAmR1IRk4l204//HLwJO3ktOHr5nOqBozmKSCjDeZ3tBH9Ng4SlI4ChBzy0Obuk8puIw1hGeT8qru5MStyFh9J5cZYpe3IBeDeZjQPviuEYT9+VggxaeAHdvM3cUE470QEUPIXB5958e004BY+fz5PI1Om+SXZI8oZObGOXIZp6d765KeYg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=L0sVjO8e/hTL6PfGKicWPBSowcKLwyxV2IFDm4hHQWE=;
 b=X27UtIozaBMJ0TTME8eUva2ZwMW1Fs1Ywm2Ar4GkqbxIY3vR29fo0tnZNH7AVX3eHqy7ZWxHSqjNk6UrboIWxQxvzIHvXLHnmInKIU3uWZY+3n5K7yBlZND12Fjf2BDhIxNxy5ztW7Wnlah5/JON46or+n7+tc6agbkCqhHvC4tJOhD3ddlQNMw+PwIhlpa96N9dx0M10ZDAXbAiFGxexmrV8JjVuTAfYi66h/61FLqrJDZjgpx2t+EYuot2w64+tZPvi2J7r/hfPXPrERI9ECb2rS98IhgDenw/3U0V36MTogI17u/srJP5JyHmRQ7puEt7nnFjFj16hynddfBzaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L0sVjO8e/hTL6PfGKicWPBSowcKLwyxV2IFDm4hHQWE=;
 b=WAvsK2mtMhjRbraEDoH6ipsMdT12LguyMwV0mRGl3WB4ritF6TX5VMapc5OvBZVoG6/40+AM2vZ4yqRy15NBk0MKjd3j/CYLJ7CdEF4JrQfkS7KBogzGaOWpKsWxBHMqcXdTbfpSmbpp1H/bnti/oKmDcEnmrc6dR1oYRiCA0aU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Alec Kwapis <alec.kwapis@medtronic.com>
Subject: [PATCH v2 3/4] tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
Date: Sat, 11 May 2024 08:56:10 +0800
Message-ID: <20240511005611.83125-4-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240511005611.83125-1-xin.wang2@amd.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D5:EE_|DM4PR12MB5772:EE_
X-MS-Office365-Filtering-Correlation-Id: eb9aa798-a039-41f7-ba48-08dc71552e9d
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?W/Odr8W15mADA2aUJJx+XWOBlnokmdylAOH5p0E6ZsywRUpM1QTf/cVXQZ35?=
 =?us-ascii?Q?ZGMeaZC524E9GbCHRxFl7ENRxuY2mqLZvnaEhCN9C54L6DJUJxPLUCdCboWH?=
 =?us-ascii?Q?PA/oCEZ9JIJBJ6Qzo2CKFihbzAXGaRAJErGcUf7Qj2w2j3vO4mbmWr2Jh1Eu?=
 =?us-ascii?Q?tQzssdkqqToYRpETOmdISd4j0YwRIlOxENjdsGU/J5j3Y7imt7I5aaNDxqOh?=
 =?us-ascii?Q?LVCSvoFYh0Rq3rdcLgpvqNpxVWUSAMx93oI8b9tbdCEoVXmdz1EXElei/fN9?=
 =?us-ascii?Q?Fqq/Tywz+t1q/l0AshZqVezndZxXiNlDOlBxa6mbswQEr3iPngivFDU4lej9?=
 =?us-ascii?Q?24ML8832HS/URwIl1REeB57EINi8tkuKpt5yfhVwquEvqrKNb+euOABocC36?=
 =?us-ascii?Q?ws6Pix2fAsJmM20j2/ntG6o1YC4MAwprT2hSYEbbusZTE3AhS8ssPWyMwENp?=
 =?us-ascii?Q?AIm2cUnXjsrlXtgsEFl7bAa9IAb+B+DpTs+1eHKN0tq2fEINoHkYzXwudwtb?=
 =?us-ascii?Q?Nk/8pzqYBogv0TLmFE1AYKrpVlSPQ6dUwF173NKPli3Ghi2CvEetzsSf2PPQ?=
 =?us-ascii?Q?GXNksdI0wWyo8YvRC55VTv5hSBoBZ1cyCsaJyAi4N3zDYSnX2O6FPFqv48Rt?=
 =?us-ascii?Q?6W3agX1oP9Hlja3UfvYy0uaS7vu1jrVLjwQy6sLSDecYz/PC1oCBZdN88FXM?=
 =?us-ascii?Q?ysFBAnuMLa/F9TTSrVkbpKaukINN48cOQByhfAGZs7GZae6hu+JyhuDGEZRl?=
 =?us-ascii?Q?FKc8NtWgoX3DvpqP4XeiyvUABju6Hs/akVZbjCV+TT1LsS3vZAwo+anoFWa7?=
 =?us-ascii?Q?dM/vpf5IL0SBlUEHJZu5IWiMeruhRUjRw854vYryGW8nAXN7nYTToSgxZZhY?=
 =?us-ascii?Q?X3zQeA/+BPu8JGeXmaRd8/MaJ0TcdMiQ6K2wBOMCabAVAxOt2LU2QMwbGirc?=
 =?us-ascii?Q?GYiaregbRulLLqGpmVOV1LEfOZLxu5DWpW0lLn299n1InfcJgfAg3jZFl6jz?=
 =?us-ascii?Q?mzKL6NlGPVqLqh9AUeV4sOh6lTcXwZXc0Vi3gFtg6JzuzHOVLbqUTcl6mdJA?=
 =?us-ascii?Q?9Sf296JVVZpmY/dsCtSBF0IPnXPTR+VlTaCdi7lko1i9QaeZeapwRcAxaZSi?=
 =?us-ascii?Q?iXR3dOMbkG7x3ah6qY0lNB6gPfZIW31GIOhKkxUNjcPzH8P2ZnwPi0wqtr3/?=
 =?us-ascii?Q?EF4aBnpxPS9qcaL12yxMvlSy0dTSZNjOTASqenbj3y+iIUIdjmddIxq/cDwz?=
 =?us-ascii?Q?cPviSGUhnIdIgpobIJNgMGSjK0reYWWUVTP56ZgAHg20awFi3Ld74ZT5LLD0?=
 =?us-ascii?Q?/jjAyMHrzoHCboyhWrJy3uB6pXt8v14QG+RqeTG7yCdxb5G1H8yMoqdPwyqJ?=
 =?us-ascii?Q?xCq12X+/JG5YjsNXZMcFW9lIZAaP?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 00:56:24.6921
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: eb9aa798-a039-41f7-ba48-08dc71552e9d
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D5.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5772

Currently the GUEST_MAGIC_BASE in the init-dom0less application is
hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
DomUs.

Since the guest magic region is now allocated from the hypervisor,
instead of hardcoding the guest magic pages region, use
xc_hvm_param_get() to get the guest magic region PFN, and based on
that the XenStore PFN can be calculated. Also, we don't need to set
the max mem anymore, so drop the call to xc_domain_setmaxmem(). Rename
the alloc_xs_page() to get_xs_page() to reflect the changes.

Take the opportunity to do some coding style improvements when possible.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Update HVMOP keys name.
---
 tools/helpers/init-dom0less.c | 40 +++++++++++++++--------------------
 1 file changed, 17 insertions(+), 23 deletions(-)

diff --git a/tools/helpers/init-dom0less.c b/tools/helpers/init-dom0less.c
index fee93459c4..04039a2a66 100644
--- a/tools/helpers/init-dom0less.c
+++ b/tools/helpers/init-dom0less.c
@@ -19,24 +19,20 @@
 #define XENSTORE_PFN_OFFSET 1
 #define STR_MAX_LENGTH 128
 
-static int alloc_xs_page(struct xc_interface_core *xch,
-                         libxl_dominfo *info,
-                         uint64_t *xenstore_pfn)
+static int get_xs_page(struct xc_interface_core *xch, libxl_dominfo *info,
+                       uint64_t *xenstore_pfn)
 {
     int rc;
-    const xen_pfn_t base = GUEST_MAGIC_BASE >> XC_PAGE_SHIFT;
-    xen_pfn_t p2m = (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET;
+    xen_pfn_t magic_base_pfn;
 
-    rc = xc_domain_setmaxmem(xch, info->domid,
-                             info->max_memkb + (XC_PAGE_SIZE/1024));
-    if (rc < 0)
-        return rc;
-
-    rc = xc_domain_populate_physmap_exact(xch, info->domid, 1, 0, 0, &p2m);
-    if (rc < 0)
-        return rc;
+    rc = xc_hvm_param_get(xch, info->domid, HVM_PARAM_HV_RSRV_BASE_PFN,
+                          &magic_base_pfn);
+    if (rc < 0) {
+        printf("Failed to get HVM_PARAM_HV_RSRV_BASE_PFN\n");
+        return 1;
+    }
 
-    *xenstore_pfn = base + XENSTORE_PFN_OFFSET;
+    *xenstore_pfn = magic_base_pfn + XENSTORE_PFN_OFFSET;
     rc = xc_clear_domain_page(xch, info->domid, *xenstore_pfn);
     if (rc < 0)
         return rc;
@@ -100,6 +96,7 @@ static bool do_xs_write_vm(struct xs_handle *xsh, xs_transaction_t t,
  */
 static int create_xenstore(struct xs_handle *xsh,
                            libxl_dominfo *info, libxl_uuid uuid,
+                           xen_pfn_t xenstore_pfn,
                            evtchn_port_t xenstore_port)
 {
     domid_t domid;
@@ -145,8 +142,7 @@ static int create_xenstore(struct xs_handle *xsh,
     rc = snprintf(target_memkb_str, STR_MAX_LENGTH, "%"PRIu64, info->current_memkb);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
-    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%lld",
-                  (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET);
+    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%"PRIu_xen_pfn, xenstore_pfn);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
     rc = snprintf(xenstore_port_str, STR_MAX_LENGTH, "%u", xenstore_port);
@@ -245,9 +241,9 @@ static int init_domain(struct xs_handle *xsh,
     if (!xenstore_evtchn)
         return 0;
 
-    /* Alloc xenstore page */
-    if (alloc_xs_page(xch, info, &xenstore_pfn) != 0) {
-        printf("Error on alloc magic pages\n");
+    /* Get xenstore page */
+    if (get_xs_page(xch, info, &xenstore_pfn) != 0) {
+        printf("Error on getting xenstore page\n");
         return 1;
     }
 
@@ -278,13 +274,11 @@ static int init_domain(struct xs_handle *xsh,
     if (rc < 0)
         return rc;
 
-    rc = create_xenstore(xsh, info, uuid, xenstore_evtchn);
+    rc = create_xenstore(xsh, info, uuid, xenstore_pfn, xenstore_evtchn);
     if (rc)
         err(1, "writing to xenstore");
 
-    rc = xs_introduce_domain(xsh, info->domid,
-            (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET,
-            xenstore_evtchn);
+    rc = xs_introduce_domain(xsh, info->domid, xenstore_pfn, xenstore_evtchn);
     if (!rc)
         err(1, "xs_introduce_domain");
     return 0;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 11 00:56:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 00:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720143.1123207 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2J-0007au-Da; Sat, 11 May 2024 00:56:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720143.1123207; Sat, 11 May 2024 00:56:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2J-0007an-AV; Sat, 11 May 2024 00:56:27 +0000
Received: by outflank-mailman (input) for mailman id 720143;
 Sat, 11 May 2024 00:56:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5b2I-0007ah-KU
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 00:56:26 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 48c17f15-0f31-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 02:56:22 +0200 (CEST)
Received: from PH8P222CA0027.NAMP222.PROD.OUTLOOK.COM (2603:10b6:510:2d7::28)
 by MW4PR12MB6852.namprd12.prod.outlook.com (2603:10b6:303:207::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.51; Sat, 11 May
 2024 00:56:18 +0000
Received: from CY4PEPF0000E9D5.namprd05.prod.outlook.com
 (2603:10b6:510:2d7:cafe::4e) by PH8P222CA0027.outlook.office365.com
 (2603:10b6:510:2d7::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.51 via Frontend
 Transport; Sat, 11 May 2024 00:56:18 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D5.mail.protection.outlook.com (10.167.241.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 00:56:17 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:16 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:16 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 19:56:13 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 48c17f15-0f31-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xq0vkuxLdWo/Sc9lYOLJ6B2ZllopHfG61h6VmuHNFUpETa/NJxVuIBY46PYhE8sASSoV8fx6Js9MDZ58kAIWlqEjaql481/EWo2u2uNdxhLmKcVYChiG3tNWJo6GKyHqvusuNTC0YQdkR9+/O5sCEsdeq+y2WLAaPwub/fRoRGKdiLYI+81iV4bIf/QoUMH/zFDAV2Z9l6hwFY5FKinTOBcOVLgxCr1D/1PWD5vWTRy0oRPNaUmR7m+SY1Z+oM8wBjhRf5fplDk2KdjAM8s6RTsjBk8YgX6Q98ZDN8CUsFzoFsNVfm28rhQ8vkwytJVOC6hVCUfLxkotMfjbB5P7AA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=yo+gmdMoRH1k9Ot7fqaAxmEZs90haIiaP3UwHtp3dho=;
 b=B8mrNgr6UNA4cCLtx+fDXsFYtiGtJtwVsAcJwU1YY8oshEOXeImokMe5j+TIHZgDRy0jXlOJq7BpS83bVzWoxjJtjNyJEG3qVQ+UguXRA78+f3gI4+v66Q+Yr7WzVVYT9km/91fq9ST6FGQnWKxv8Ed+UA3h8TDMHsAF80J7csmQFlXW/xJOk2rWEEh4RnE6WaECsoI9ZWaspRwcHmXny2VLiDH9apEzOqXVvVTMV3g7JO5lxKK988JS+Od+YfypwLt1w+e3loGKtf8tU8595gMk79fhJNZhDs4tli+yv74woD9sL5YdfxK1aoF6mmuX5I3+JoUQAZ+KiNAFpjLdMw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yo+gmdMoRH1k9Ot7fqaAxmEZs90haIiaP3UwHtp3dho=;
 b=dApnrCrvPlJx6S4cad82eHv2l7zgVda/owLMBVBuLKdcNaicASk4uHzVyAGp6qqjUbY+e27Ea4ruwUgr+vB8I5jGQGPDS1b8hyGFxohx+YP+4L4cjn+vUSKxS0EuhtE8zEBDWdpNLhXBIMvffZ+Gwuv+rYXzP8iTAfObkpALpnQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/4] Guest magic region allocation for 11 Dom0less domUs - Take two
Date: Sat, 11 May 2024 08:56:07 +0800
Message-ID: <20240511005611.83125-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D5:EE_|MW4PR12MB6852:EE_
X-MS-Office365-Filtering-Correlation-Id: 543b58ec-639f-4d09-ed50-08dc71552a41
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|36860700004|1800799015|7416005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?8HJAi+UQMJWSpWcWeyUfkoXQq8ER/v4PMkUoAEksdV/pN/s9Mw8LsbBSmXz1?=
 =?us-ascii?Q?ZyXacuVXPKTA8lKwR21p5SVheC/yWQoCCKHejAh1d7CFAIDT+jG77KG6CIhG?=
 =?us-ascii?Q?skyG1sbDuaTWM2eO6tc5D17Mp38lkvmHlpHHTjPMMFVBjn23iPkN+8UVPWeT?=
 =?us-ascii?Q?3qQZYahRMKAACdiKMNi07jLz1cFLFqksbY9Wk4oz/ZoFBHTzIFFD1NQOg/mV?=
 =?us-ascii?Q?TB2t5bBjZtLTI6fvQ0kmcBWQ+Qjm/E/BjZyLXF7pMuZNQvOSxsQdzw85PKuX?=
 =?us-ascii?Q?angmdMjWn1U4GIDO4o+4wNBJoGtdRBdJItUIXjX4IiPi048jpsfTKn+MX/5Y?=
 =?us-ascii?Q?0mzv6ZH+jP8Gi9oZxFoP+BOS4NrComKQ2Xf0HMmnNFRKtkcgw9og8BQxqFI2?=
 =?us-ascii?Q?rtLQKfw0Sa6GQ2sm9RXZLhV4umylz3Mr1VpXPqEbSLBt8wLRgIUmx31vdkGW?=
 =?us-ascii?Q?N+MWf47CrbR/LNwRgJ+0qLdOeUM1++ArFSQARC5JQPDBv/8lEGKe5ymNPNL/?=
 =?us-ascii?Q?Ml27EI/bux37eOzL4mIzMz2Xlvps7aRjM2AROy6k8zWDyO7LRzZEwCdGcJYf?=
 =?us-ascii?Q?PjYS/YdG4tIgwXO4GtNuL3yDQZmRBva4buCMEDbHDZgnYbhi5Gn5cRzJFf6j?=
 =?us-ascii?Q?/amfJBkzdkaN4hlsbhjarBexv4PP+V6MIN70A7lsyRHblYO/y1vuabV+TbCO?=
 =?us-ascii?Q?OWAiDU4EfXxMIvFJgqw10PUZDcvdkDKMF/p8Yrwfdn1D5FpsVuyQXbJWpz1m?=
 =?us-ascii?Q?LEdwDtrHO/OZAF00C4u+XwbZp2yLGIXbQp+BOg2kiV8N31uIMacUijCplPgq?=
 =?us-ascii?Q?c23Ywyt5JN59wK4aeIFCP9oJl11UiOOXygHvzkjRPNBJRwY6I6YHjGRTbXzv?=
 =?us-ascii?Q?2309HZM27BNoONcs03OxKFXVqnuyx+J/YkOLeKrnNkLHhi3gh2WNvl0grnFy?=
 =?us-ascii?Q?vNYKbrmZ7GLCibDaf3v5YjakB8ONagOktOmx6qlq17hNnzbF3kAf23WC6+H0?=
 =?us-ascii?Q?s9/4ecRK6oXi6qLTgloOwy44VJlkiPkJQcASD2YqoyfQgnD6BuvyQczq9uPn?=
 =?us-ascii?Q?hMnP6zyHy5kljNJjkibkhskpE3O5uDqOqGJ3C86aXmBUqgSNqgfNacqKQNbu?=
 =?us-ascii?Q?6M9hA3UlmmMxrbH0/z6gtun6cEGRpzPV72FaxV4lguDwxcGjCo8IJ4n4qvM0?=
 =?us-ascii?Q?It2aErI/spOD1ZrznBCCjcXSJN+5SDF6v4RqXDgmatNc4O294Wv26jb1z4HX?=
 =?us-ascii?Q?vVvqt4nIG+OMruda8qqM1S5L6mJxHtnHTu/d8hUiBJz1vwkkJoHYKXHfpP2D?=
 =?us-ascii?Q?cio20KcBTpC9opXs/JhakyWK1hkPu2Gf2yGCqLHD68hDlA=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(36860700004)(1800799015)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 00:56:17.3796
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 543b58ec-639f-4d09-ed50-08dc71552a41
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D5.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6852

Hi all,

This series is trying to fix the reported guest magic region allocation
issue for 11 Dom0less domUs, an error message can seen from the
init-dom0less application on 1:1 direct-mapped Dom0less DomUs:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

This is because populate_physmap() automatically assumes gfn == mfn
for direct mapped domains. This cannot be true for the magic pages
that are allocated later for 1:1 Dom0less DomUs from the init-dom0less
helper application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list
is empty at that time.

In [1] I've tried to fix this issue by the domctl approach, and
discussions in [2] and [3] indicates that a domctl is not really
necessary, as we can simplify the issue to "allocate the Dom0less
guest magic regions at the Dom0less DomU build time and pass the
region base PFN to init-dom0less application". Therefore, the first
patch in this series will allocate magic pages for Dom0less DomUs,
the second patch will store the allocated region base PFN to HVMOP
params like HVM_PARAM_CALLBACK_IRQ, and the third patch uses the
HVMOP to get the stored guest magic region base PFN to avoid hardcoding
GUEST_MAGIC_BASE. The last patch will update documentation.

Gitlab CI for this series can be found in [4].

[1] https://lore.kernel.org/xen-devel/20240409045357.236802-1-xin.wang2@amd.com/
[2] https://lore.kernel.org/xen-devel/c7857223-eab8-409a-b618-6ec70f6165aa@apertussolutions.com/
[3] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2404251508470.3940@ubuntu-linux-20-04-desktop/
[4] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1285727622

Henry Wang (4):
  xen/arm: Alloc hypervisor reserved pages as magic pages for Dom0less
    DomUs
  xen/arm: Add new HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} keys in HVMOP
  tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
  docs/features/dom0less: Update the late XenStore init protocol

 docs/features/dom0less.pandoc   |  8 ++++---
 tools/helpers/init-dom0less.c   | 40 ++++++++++++++-------------------
 tools/libs/guest/xg_dom_arm.c   |  6 -----
 xen/arch/arm/dom0less-build.c   | 35 +++++++++++++++++++++++++++++
 xen/arch/arm/hvm.c              |  2 ++
 xen/include/public/arch-arm.h   |  6 +++++
 xen/include/public/hvm/params.h | 11 ++++++++-
 7 files changed, 75 insertions(+), 33 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 11 00:56:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 00:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720145.1123227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2P-00084W-15; Sat, 11 May 2024 00:56:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720145.1123227; Sat, 11 May 2024 00:56:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2O-00084N-Tt; Sat, 11 May 2024 00:56:32 +0000
Received: by outflank-mailman (input) for mailman id 720145;
 Sat, 11 May 2024 00:56:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5b2N-0007p5-3Y
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 00:56:31 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4c44d00c-0f31-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 02:56:29 +0200 (CEST)
Received: from SJ0PR05CA0136.namprd05.prod.outlook.com (2603:10b6:a03:33d::21)
 by BL3PR12MB6452.namprd12.prod.outlook.com (2603:10b6:208:3bb::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.49; Sat, 11 May
 2024 00:56:24 +0000
Received: from SJ5PEPF000001C8.namprd05.prod.outlook.com
 (2603:10b6:a03:33d:cafe::e7) by SJ0PR05CA0136.outlook.office365.com
 (2603:10b6:a03:33d::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.18 via Frontend
 Transport; Sat, 11 May 2024 00:56:23 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ5PEPF000001C8.mail.protection.outlook.com (10.167.242.36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 00:56:23 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:22 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:22 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 19:56:19 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c44d00c-0f31-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=M/GTqvrUh3lG7vVOODAiuRg5gsaArpYy5zp/D0mu5yZ1/RudkWsk9y5k/zKjMRuiVxcZmSdgpXQy1CP4vWIFnVGUAWMMDEbPxzeEJU9R9xk8XvG5R7zMxY96NOMJeI3BZzVECwJ90oQhCLJIWPFlvoKMt+EW5/QQCGQqhO1OsrKISo2mmoq2V1DfMcA+0WneXI6603qySECDsXZdR2lUTd+c3VbXgKRktzWrQc8SX/FVPE8brgVhvVThSxh6u5U3hVhS0NZX1G8H54k4WvBF5PHdqwvczry/eionr/HapQOTA1zJZ0SVndvbT0deQ/14iiXGi/t3hktVwywb+agAXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8pDtPG/gwtKzswtOqvdxKDHnF7RDE7VlIypy2jSq22o=;
 b=B5lhSGdWmNKbjWDV7srQNFt4QaBPG/CT6c0Jweo5eJrSV/TJoeetYN9TzBDserlSrme+IfrdElCy72UPml+8iHA8Ox0sHBLtX1bvIfjmfuStkuty5daS4daEoJGRsYITSydE/T7zX5gUE/FylwSiskfgj9UH1fiR/3Va+S+Bmx/ih7HSLpOOg5OvJfHiy2wfutNY9/MgEM2vlIf3yEofoSoKVsE6bnHoO8mZ5NgNGH0d2ES1w2hKXXYk0nuctveeXpVBwYn1GZPPf1Mk+A8CFGM4r55FoVCPYB+Jh/VJ2S1e5yeYWTDg/HOi0Dr0CfgIQFlkEmVv9s4VGHOE3YGl6A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8pDtPG/gwtKzswtOqvdxKDHnF7RDE7VlIypy2jSq22o=;
 b=Wfc0LLInNkZZiaLcaMQ4zg6Dh+CViHAkWJo87ZadO0j0kYBfrHMTGuPCqjm33V8nGegM4rhnyp+lVRTnVtmdIYiQbS5Uv0wcyObCYJi8qciSffhj+CC6hTHvsbkjYn5Irat/1IBmcir36FfQlggVH+aZ7294FaQx8QW0EN7uOGE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Alec Kwapis <alec.kwapis@medtronic.com>
Subject: [PATCH v2 2/4] xen/arm: Add new HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} keys in HVMOP
Date: Sat, 11 May 2024 08:56:09 +0800
Message-ID: <20240511005611.83125-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240511005611.83125-1-xin.wang2@amd.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001C8:EE_|BL3PR12MB6452:EE_
X-MS-Office365-Filtering-Correlation-Id: 74eccd9c-6c6b-419a-0958-08dc71552dcd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?kg+ZdbpbSZavsTg+6Plfd2hHqc2S9v5hb4o/i4O/w9JNQtZEAEUipThT+/Ll?=
 =?us-ascii?Q?XFqH04Z3YJaQhY4aNs1da2EsVH2VIh/ESpYRrRgGLt9rQPW65/NZ5Juq/G7O?=
 =?us-ascii?Q?Xx/qfej/cmRnLXL2H8696v+RSAQV7J5b/CJanexpwb+s+N2XXf+vVbwmV1/S?=
 =?us-ascii?Q?B8j2fsArazyY5INhq2KA+XX9kkDK34dHkGEo7Tq269ljoOTOIzgBt2y1KFza?=
 =?us-ascii?Q?zoeCpcMkJkJaqKq27fASF3t51jlJzHi03UhgN0zseN1pWtzrMwwFqP2TL2NB?=
 =?us-ascii?Q?493tS7dRKvhdB54sSR6mP+Yt8cgHujg7IliDWFR0zY3tVYAQ6QXHhk1WUXmx?=
 =?us-ascii?Q?fw+CYe4yD8Z/uvNgiieD/cJ9vnLZ6pIkTeJleeDK5E/MVkLAKpm0GpffMDGq?=
 =?us-ascii?Q?00tRoIjaE4wahjsRET/75Ywvs5VfchY9MrJ8pu3zxfHQU/gF0prUQqrCmHYi?=
 =?us-ascii?Q?ZI9oUohf9d1WR94hZjfANJPbqt2vHz/pfUYKzwGRFP+bNNyV4nyHPdv+YiNn?=
 =?us-ascii?Q?omXXoHkZnu5cInLXjv7T1n7L619+uOGcYUClMlRRtSHJfNayO2AWFoXMI+EB?=
 =?us-ascii?Q?x2U7WuYANssyAqwDH03TAwBdSTbhP8IcJtVXujkzbpcMWqeTEgxnCzA7ZRGf?=
 =?us-ascii?Q?+ubVUM/yffbkZBU8XFthtzvmMguY/GL6I+G8eXj79sJQyCvDN8NVF8Fwci4G?=
 =?us-ascii?Q?EZy5Af4lEsPF/S0nj25k5JlPYTLBkkM/pnWhUjLwiv331WetljMC43SPTZJ6?=
 =?us-ascii?Q?ZnFMclvda41bA1OCuNy41NPRN8/CQUHTuxNsiOWXLJPOZxY5rS+KT9aQizWI?=
 =?us-ascii?Q?gJuKUDq62u8qzYVWlbgqaQZr5v3UUn3ZfSNErCUpTrqPFER2xDsUOxyyhLLq?=
 =?us-ascii?Q?SXZ2cJzgmwkcte7f2QTr4Xxpo5YdoNSoAx1P1w2+o/fUs7ITzP/nbjq51e9z?=
 =?us-ascii?Q?KsSqFG6DTwyt7qq2X8QhZHdJ2nhAnYv8Js/pldAyVEKzPbtQPBHG0txU8eMt?=
 =?us-ascii?Q?SiBjIJS4ZxVFVQBOg3ojNkDzCWXsMfhpGzutD4OA1GsejvEo/GAxMzJVirkR?=
 =?us-ascii?Q?vQa0Qt0NpeQpEHWX58wOmny3qZvrhFSjKpjwt3wKMjKJvg6nP0O+40/K4E8A?=
 =?us-ascii?Q?RsUhZaFnvG4lVZNaDqPxa0sSm1EM6UA/pArbP46Vma+g+FD87N6L1AWlkXi1?=
 =?us-ascii?Q?fLPUrEQgYXH+ouiJ5+4lVowGxoMkebemM+k+Bc+EYUHNB5pMz1k8IMCTWwst?=
 =?us-ascii?Q?KRRBXUt5e9GAz1tozlv1WGyHmzsKr/MDMlCYEodmfGjuqf+D8nPOQ7hrXgWj?=
 =?us-ascii?Q?IuLeYTwb6Iue24FTpHmQXBzCex6HyVtrjvANn8AEuMRWvv7jBWUdPEXj12l8?=
 =?us-ascii?Q?q+UMv4w=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 00:56:23.2932
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 74eccd9c-6c6b-419a-0958-08dc71552dcd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001C8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6452

For use cases such as Dom0less PV drivers, a mechanism to communicate
Dom0less DomU's static data with the runtime control plane (Dom0) is
needed. Since on Arm HVMOP is already the existing approach to address
such use cases (for example the allocation of HVM_PARAM_CALLBACK_IRQ),
add new HVMOP keys HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} for storing the
hypervisor reserved pages region base PFN and size.

Currently, the hypervisor reserved pages region is used as the Arm
Dom0less DomU guest magic pages region. Therefore protect the HVMOP
keys with "#if defined(__arm__) || defined(__aarch64__)". The values
will be set at Dom0less DomU construction time after Dom0less DomU's
magic pages region has been allocated.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Rename the HVMOP keys to HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE}. (Daniel)
- Add comment on top of HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} to describe
  its usage. Protect them with #ifdef. (Daniel, Jan)
---
 xen/arch/arm/dom0less-build.c   |  3 +++
 xen/arch/arm/hvm.c              |  2 ++
 xen/include/public/hvm/params.h | 11 ++++++++++-
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 4b96ddd9ce..5bb53ebb47 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -764,6 +764,9 @@ static int __init alloc_magic_pages(struct domain *d)
         return rc;
     }
 
+    d->arch.hvm.params[HVM_PARAM_HV_RSRV_BASE_PFN] = gfn_x(gfn);
+    d->arch.hvm.params[HVM_PARAM_HV_RSRV_SIZE] = NR_MAGIC_PAGES;
+
     return 0;
 }
 
diff --git a/xen/arch/arm/hvm.c b/xen/arch/arm/hvm.c
index 0989309fea..949d804f8b 100644
--- a/xen/arch/arm/hvm.c
+++ b/xen/arch/arm/hvm.c
@@ -55,6 +55,8 @@ static int hvm_allow_get_param(const struct domain *d, unsigned int param)
     case HVM_PARAM_STORE_EVTCHN:
     case HVM_PARAM_CONSOLE_PFN:
     case HVM_PARAM_CONSOLE_EVTCHN:
+    case HVM_PARAM_HV_RSRV_BASE_PFN:
+    case HVM_PARAM_HV_RSRV_SIZE:
         return 0;
 
         /*
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index a22b4ed45d..337f5b0bf8 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -296,6 +296,15 @@
 #define XEN_HVM_MCA_CAP_LMCE   (xen_mk_ullong(1) << 0)
 #define XEN_HVM_MCA_CAP_MASK   XEN_HVM_MCA_CAP_LMCE
 
-#define HVM_NR_PARAMS 39
+/*
+ * Base PFN and number of pages of the hypervisor reserved pages region.
+ * Currently only used on Arm for Dom0less DomUs as guest magic pages region.
+ */
+#if defined(__arm__) || defined(__aarch64__)
+#define HVM_PARAM_HV_RSRV_BASE_PFN 39
+#define HVM_PARAM_HV_RSRV_SIZE 40
+#endif
+
+#define HVM_NR_PARAMS 41
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 11 00:56:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 00:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720147.1123247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2S-00009U-Ih; Sat, 11 May 2024 00:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720147.1123247; Sat, 11 May 2024 00:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5b2S-00009L-FX; Sat, 11 May 2024 00:56:36 +0000
Received: by outflank-mailman (input) for mailman id 720147;
 Sat, 11 May 2024 00:56:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5b2R-0007p5-Bz
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 00:56:35 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4fc6ea7f-0f31-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 02:56:34 +0200 (CEST)
Received: from SJ0PR05CA0145.namprd05.prod.outlook.com (2603:10b6:a03:33d::30)
 by PH7PR12MB7282.namprd12.prod.outlook.com (2603:10b6:510:209::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.51; Sat, 11 May
 2024 00:56:27 +0000
Received: from SJ5PEPF000001CD.namprd05.prod.outlook.com
 (2603:10b6:a03:33d:cafe::95) by SJ0PR05CA0145.outlook.office365.com
 (2603:10b6:a03:33d::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.13 via Frontend
 Transport; Sat, 11 May 2024 00:56:27 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ5PEPF000001CD.mail.protection.outlook.com (10.167.242.42) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 00:56:26 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 10 May
 2024 19:56:25 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 10 May 2024 19:56:24 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fc6ea7f-0f31-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f3Sxtlr7qzg+ofD7hAOzn8/acXan2BrrW+i0wY7aGB4z2jCyPnAo8+fGKtddzITFU+iHnCvYXEtXpZsdwHjB3KOK40XsMRMJHGVOIWO/TIdhvrsAxtZZYZ0rIsIAqbQC++1MiwrVh+v8Wvs6OrSIn1AmpWY4yACMQruCJ9KbTeOX6vEzpSytFCYBmLXXdRHKn8y9MduPA/JZvjA2+kgg/3UKqcdG5NzzD7lsIEJO00o3eJPvsLQgbRY/3FKEZpQHx4Po9tcJY1ALkfXgdILDAYJhaJQYkLduro95Xn2Ta+vIQNnrAExCERuzGJevpt4oOrseZqoWSYNbe4+YAcrsvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=+g1Sl34/9TCXkY67P68n4Fomec7VFpUiwGKx0eZZ7xY=;
 b=MsnwKKF3OPBOKJ7brfJpSkEMUCJlEAdHCm7Wg5PILGLPbzgVNvd9veEms0tZ7RSyDjNurUHlQQ8kXiqHJ0O4mR9lsxCD6ZgkN6ZkNbQnHno6B/f8ZD2/zZRQrwDV/TbBjZmNGwD6ylHsdn/F7okvYwjEgUOgr1SZYigyiNuw1LdsNTPatiBCdegrwFRa5UDjkbd5mcTp3DLaHjJdQ35sfPU9cumAZteqNUoU5fcGdPmgkPEIjhj8LVk7bLMBnSgVwBTKBj7pJzj5Jg0capCLI6O5CbVp3IPuXI6DPsP9cXWOBmBkhN2Wz3BK3sZx5/uqg0vvhAXVwBSl46g8lvcRwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+g1Sl34/9TCXkY67P68n4Fomec7VFpUiwGKx0eZZ7xY=;
 b=XPf8OZjnV0QsIPQ3m40wszIuwTDYYNdRazqqP3pjZ8kq/HDrzCJc79LPiSWNt0bapni7e6/eGFj4uvUG1rsqwTL/ZwqHwC0z8OSRE2jSqUpDX9/gIqP2k4EmJB2dmpZEwi/MFO9sjxreJNoK6Ch8JVbFK0JpT0owUN4ULaCid4o=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v2 4/4] docs/features/dom0less: Update the late XenStore init protocol
Date: Sat, 11 May 2024 08:56:11 +0800
Message-ID: <20240511005611.83125-5-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240511005611.83125-1-xin.wang2@amd.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001CD:EE_|PH7PR12MB7282:EE_
X-MS-Office365-Filtering-Correlation-Id: d4e71e78-47a3-48ec-c3b6-08dc71552fdd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?iL5IOhV4p7cUoNSGwwrJPaPMZsYsgELpNwSHCUdF4oR08an2u887BiiX7zG+?=
 =?us-ascii?Q?wiIV0t3ZBw9D1zzH6Mup4ehOCdXoGwAQq4W4CAcaYHM57rv77WeG6dV/pZ5u?=
 =?us-ascii?Q?dRhj56g7NhrJujfw3OSA85KlwdX66f1KGEA7d3UPSDxNKrvqAUTYEknaoxg+?=
 =?us-ascii?Q?XH3pIFOD2bFE5Jf9C/HCX+KUAiOF2FmEv7Q8zaOphowPU+fRaQfUxDoY91Hz?=
 =?us-ascii?Q?1nTLiNCqPl7uVXP+OLE5vLSYeTr48CviwEiXmc+YQ4U4E4nLWqGnluCRib+a?=
 =?us-ascii?Q?w7Gw7A1zQvJof8JTyZpclC/gUyzUcZZ7c5+PZV1ZKGoJCTWtVzLw0MCAMfpN?=
 =?us-ascii?Q?8GPenZhb7rJnHMLrj9pMyqDPVDjYnhoz0g9TKhxIMhFEsBh1zkejFq59nc+0?=
 =?us-ascii?Q?4fxEzyxkTf63ulGRrZUdb9gP61pQ99Yn1xSKwkpq3IT/l74Urx2ZYo/Op8C6?=
 =?us-ascii?Q?wIOLQOcY3PwHdhY/2NfSjr+IBbV8oX01M/erVA1AOXF0zhyexLu5GauP/Lqq?=
 =?us-ascii?Q?K3qSj6CF707UcwFuCXklm0uXNICeerOFwn2hXmmFRTBHahsMAwZKZ0CLFtYT?=
 =?us-ascii?Q?HBhUPQ2EfUiTDkyO/y5WzCO4K9oXKuLUPbrNoitToqaZgavq0Vjww9TP49+T?=
 =?us-ascii?Q?qjigrUR/kybx7WhFlRy8gQrdFXA5Kuf/AQaQB2JuHI9lELoBl3IxQflXjJI4?=
 =?us-ascii?Q?6CknW3jGsw/74ivZsooFOwZMTnw6AWjcDoec64/8hIvLr4A6LRupJ+Aayw0j?=
 =?us-ascii?Q?jRIvMDgtWvJzq+ks1Iv/2Fv8y6DlprZglKcwc/CFGcS3RtR5mVDhK8rw0wEt?=
 =?us-ascii?Q?V1FeaGdRdNGHH/qZlYWk9o3iCIEFicd6YTolmAa6YIc6UPMDLdtWuIy5jGbz?=
 =?us-ascii?Q?+yhikmBTfu0hl5dWevJjycMAJIdtr6T2QNiUxwF9nb6+Oky1vIpRUj9erWW0?=
 =?us-ascii?Q?G7buIeeBjjfLgME79EK5KwDOzPe5wuYRw1WYdLrC4xeoKaUUnEpGhgfkCnOc?=
 =?us-ascii?Q?F9tbE3mc+Shgy0OdxL+SsJBIXnHkpY9+dPtrJfaPlIPo699PIPrvL3ong02b?=
 =?us-ascii?Q?u2sq2CIkrHLZ4upqyNHMnWWqifYWmQgalrLoPj9DhzzkdAIbz7CqYfw3Cy5y?=
 =?us-ascii?Q?iVFQjPm9fEc/1QOyC+L7baILcjsHcanXpmpqWixnDL8GV4toymZwt7B0tx96?=
 =?us-ascii?Q?9JKH/X44XoDRLAX9ata+Or1CH9dB871YByHUUGcbaD/YL/j8dotuUDxMyMcu?=
 =?us-ascii?Q?P36WJcWJQFnnnIXQYv14C8LYhauJEq1E7hMq799gDQ/wp1vuMols8/DchXr4?=
 =?us-ascii?Q?FiU5evfjwJlgUeT9t/J6SukBbnmjQRST1Lh/wsEUmpUb8H6Wed99EdebPkgH?=
 =?us-ascii?Q?4bGuYStkNoInVnWfCEqJID9ze1at?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 00:56:26.7739
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d4e71e78-47a3-48ec-c3b6-08dc71552fdd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001CD.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7282

With the new allocation strategy of Dom0less DomUs magic page
region, update the documentation of the late XenStore init
protocol accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch.
---
 docs/features/dom0less.pandoc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/docs/features/dom0less.pandoc b/docs/features/dom0less.pandoc
index 725afa0558..137e6b618b 100644
--- a/docs/features/dom0less.pandoc
+++ b/docs/features/dom0less.pandoc
@@ -110,8 +110,9 @@ hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.
 The implementation works as follows:
 - Xen allocates the xenstore event channel for each dom0less domU that
   has the "xen,enhanced" property, and sets HVM_PARAM_STORE_EVTCHN
-- Xen does *not* allocate the xenstore page and sets HVM_PARAM_STORE_PFN
-  to ~0ULL (invalid)
+- Xen allocates the hypervisor reserved pages region (the xenstore page
+  is part of it) and sets HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} accordingly.
+  Xen sets HVM_PARAM_STORE_PFN to ~0ULL (invalid).
 - Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid
     - Old kernels will continue without xenstore support (Note: some old
       buggy kernels might crash because they don't check the validity of
@@ -121,7 +122,8 @@ The implementation works as follows:
       channel (HVM_PARAM_STORE_EVTCHN) before continuing with the
       initialization
 - Once dom0 is booted, init-dom0less is executed:
-    - it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN
+    - it gets the xenstore shared page from HVM_PARAM_HV_RSRV_BASE_PFN
+      and sets HVM_PARAM_STORE_PFN
     - it calls xs_introduce_domain
 - Xenstored notices the new domain, initializes interfaces as usual, and
   sends an event channel notification to the domain using the xenstore
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 11 01:14:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 01:14:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720171.1123256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5bJw-0003nt-4P; Sat, 11 May 2024 01:14:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720171.1123256; Sat, 11 May 2024 01:14:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5bJw-0003nm-1f; Sat, 11 May 2024 01:14:40 +0000
Received: by outflank-mailman (input) for mailman id 720171;
 Sat, 11 May 2024 01:14:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5bJu-0003nc-Rh; Sat, 11 May 2024 01:14:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5bJu-0002Sz-OP; Sat, 11 May 2024 01:14:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5bJu-0004RM-GK; Sat, 11 May 2024 01:14:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5bJu-0007bd-Fw; Sat, 11 May 2024 01:14:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Vuv0BxPwpqXpA604XuOWuik0LOV30z61pbHWkvu7jOg=; b=qcyxhwweTqHbppYC+lv1Fw1kSF
	+OV63fDLeOAicsQUEpZa/WoxV5jiXL9VdgYcSgMRncF01cMI2rEEmaqIvLyGpkE/ANZVAy/lc2cBm
	1AiEyJtBvoVVhKnk6z2thFbmBcOLKSzhD9ZljdRip4PbNMOjBsW+nZBDs58+y/ssJExg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185973-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185973: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=996576b965ccdf0de17aafa14282925e408e1200
X-Osstest-Versions-That:
    xen=b0082b908391b29b7c4dd5e6c389ebd6481926f8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 01:14:38 +0000

flight 185973 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185973/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  996576b965ccdf0de17aafa14282925e408e1200
baseline version:
 xen                  b0082b908391b29b7c4dd5e6c389ebd6481926f8

Last test of basis   185964  2024-05-10 02:02:10 Z    0 days
Testing same since   185973  2024-05-10 22:04:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b0082b9083..996576b965  996576b965ccdf0de17aafa14282925e408e1200 -> smoke


From xen-devel-bounces@lists.xenproject.org Sat May 11 02:16:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 02:16:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720180.1123267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5cHY-0004lJ-Gc; Sat, 11 May 2024 02:16:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720180.1123267; Sat, 11 May 2024 02:16:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5cHY-0004lC-DN; Sat, 11 May 2024 02:16:16 +0000
Received: by outflank-mailman (input) for mailman id 720180;
 Sat, 11 May 2024 02:16:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zEDt=MO=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s5cHX-0004l6-Ag
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 02:16:15 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6eafd7a7-0f3c-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 04:16:11 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by SN7PR12MB7449.namprd12.prod.outlook.com (2603:10b6:806:299::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.48; Sat, 11 May
 2024 02:16:07 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.047; Sat, 11 May 2024
 02:16:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6eafd7a7-0f3c-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V6zng0vRYgHAQc4tkRtV9jCJ8FzCWb6t9J3ErKhz7yYiLfMHMVXFIk3aHvAamBAR6Ra+DaWXhTimhOUlUR5zyUEVSXLVMAjRCdgc1o/PIshQO+Z5hNkmOX8AOR+wEA9ox4BAajtm+ZQCtuBNiLIppWrDep1Rvb7bt/qQth9fJWoSxG4l2RHBEC0ow7FC5Faf7MheMJkYU29ygRTUtE18suoQRX6KpskSjmIRHaj5JSkl74vQb+lnh2tJky9UangCrHEjL87U/0GQcuy+nwNzuMP6eIyI6xAY6eoK3hTBb22O99dh+Jrjs4ccKb7w8hEA5V/hVSCcGlbbU1l4OhzePg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=I0428Ijj/UEPn6TKWG5i49LAwCZg7A4VDslimAPxVGk=;
 b=LP5XWpBD4T0mCBG0xABBDSerDnCQx/3R+6c9T4bZr7dWoF4Htvq2MfmXI4yjXNwCCZKhp0WUTItMnVur5QfHO5fQB7fqS0TC3b7svQaOjOxZmRYqdfZuYKrRLRkY+wniLT3mr7CBlliJXSvtk43sEl2i7sUiCPlHI3bnGBzXGzYFJgIypE//2hx0ptlHxRO0rxzGqE1kmv1ffjGyR7Q/FUWqMcDVmNnViB/wXMRI7Pmqzpdg76mtjdynQbXyXSCK4U4y6iZojw8W0F2GOnKio6YsRIX9by1EtD8qTraP7a6xoqDy1osoXvkTz0tvjIigd0pLVh+EYULK3YS2hCAzzg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=I0428Ijj/UEPn6TKWG5i49LAwCZg7A4VDslimAPxVGk=;
 b=a6d26LlNhjmUOKtEQbzSqEg2AGx5hQfS5SGzJSWDdcHYXSVb8js2uTc0IypN09HQLbM4auI9HGedDPtw2zH0rXhJQJjC1p6whDJ/WALr683Y0Kh3ps4S9h89tOFXhoOSkhrVsk2tDVpF2mB2OLsD72p8ksR5SY+qr8NR6JKzcEg=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>, "Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
Thread-Topic: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get
 gsi from irq
Thread-Index:
 AQHakgrPZQMiHHEzE0Gs/TUOGsHh17GQJ2yAgACnAgD//41hgIAAiXAA//9+YoCAAIdwAP//iviAAC/BN4A=
Date: Sat, 11 May 2024 02:16:07 +0000
Message-ID:
 <BL1PR12MB5849B13B1BC7EAFA6207C57DE7E02@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <c30ebad2-1ad3-4b58-afaf-e6dc32c091fc@suse.com>
 <BL1PR12MB58491D2210091DF9607A354AE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <d0b5e7d5-3503-49be-9fa3-4b79c62059ca@suse.com>
 <BL1PR12MB5849F1DE8B4A3538C79CE5D3E7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0aac68ac-cf40-4c3d-ac02-95b9a37aaa11@suse.com>
In-Reply-To: <0aac68ac-cf40-4c3d-ac02-95b9a37aaa11@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5754.namprd12.prod.outlook.com
 (15.20.7587.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|SN7PR12MB7449:EE_
x-ms-office365-filtering-correlation-id: 5351b627-e5bb-41a0-c6d0-08dc7160512d
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|376005|366007|1800799015|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?Y2YzK2xJT3Ewd09lOTdadFRid1J4RlRoRXphRGk0UVlHbW1mbGNCaXJ2cXE0?=
 =?utf-8?B?VFNIdGtKZjdpTVBuTE8rSXBvRm5LWU43ZTZUZ1NIVGp5VDZSOU56SVhEVmFi?=
 =?utf-8?B?MndKWVI5YmMwd2xwZUNDY1B0S2ZDWlpqT0NSWU1OV1l5LzlETk9aV3ZYNW9u?=
 =?utf-8?B?ZHRheW14Yy9OdGFoT1hGVjJUVzExdFlJUHVMMzBjNDlaWFZTMjJsekNQejMv?=
 =?utf-8?B?MmhzRUdUVVdqT1JCSHJvZ0UwQ3hyOUNRMDFwTXlGODRTNjIrVWIyNEhWcFBG?=
 =?utf-8?B?TXZsTkJSTUhsd1dGMUVFUWJZMll6TktJd3FHTU00L25hL2ZyNnB4cGt3ekpo?=
 =?utf-8?B?SDk5Qmg1OVlQRUM0MFZ0QzVLZWo1N3JPbGhKU2tPdmZKd1hRYlhva3RBS1JH?=
 =?utf-8?B?WlVVaWswN2JIRFd0dkxHbm1ScXFxMDV0c01MMDh6UlY4SHQzZTJ2anUwcmtF?=
 =?utf-8?B?eWpPdGxlVDYrZk5zS0tzYUkxU0V5S2V5WjlBVEhYRVVxQ1hQNEpMWmZaOElZ?=
 =?utf-8?B?bTBzYXVjL1lNaVBGeHlyNVdyYStkc0hBY2FRdE05VkhQM2ZIV01Fa1pOblMv?=
 =?utf-8?B?ZjNMTzJGVkI0YVFXeHNUVWVEdnZMZlYxSVVPekVod0c3UnZWakhMWFdIMVFp?=
 =?utf-8?B?QXlDUHUxNzFDMEJLN1c5ZURVNTRCckxoRzBGQ0dkaHhMdXJFVVpDYVJFZlE2?=
 =?utf-8?B?dXZVQVdkMXdpYktPdGQ5MnBjcTFxckJLWW1aT0JFZ3JDYjVZY0JlSTF5cFp2?=
 =?utf-8?B?djZVMHB5NmlwYWpnejdiVkVTUjBrMlRWcTB3VTZLNkxMSGJyVjBmTDFacSs1?=
 =?utf-8?B?NnhaOHRjUDlUZW5tZlN1S01EN2RKc2JZQnNDOXRvOUQ0ZSt3TWZvYVRINEo0?=
 =?utf-8?B?dzhweG9FaUttanUzQ1Z5MzNkbDhYTFpOemZyb0ZwakpEYmtheDNVMHg1c3pE?=
 =?utf-8?B?dlRQd2Rlc2piTEpzazJJMEJrNnlTcjVvZGJpMDB3V2lGbXUxdzE4OHVMczhD?=
 =?utf-8?B?aGFTK1RvSWdYc0hCK3hFMFVOVDFjbktRaWY5RW1LSWtiaXBoOWZlaVhZazdM?=
 =?utf-8?B?alBlbkZjblc5Z1p1QUQ5VWI4bWxRbm9kdFU3L3pEQ1M1NVVROFVKbkRXbHpn?=
 =?utf-8?B?d3N3eEh1ejBiR3UwdUZHc1h6WDJROFlWUzBFbDRVQ0dRMzMxM0FMR2loNWln?=
 =?utf-8?B?UTVRdVJWWDZDMjJMdWhHcXR3VkJZYmozYUgySitFN2pCOUtlRDVqc05GMDh4?=
 =?utf-8?B?ZVBvbDFkR1RBenVZN0ZQd3pGRlpVZEdRbVRuQTFIak5TUlZWbVJBNngzb3Zl?=
 =?utf-8?B?Qnk1U0txcXZGd0UzRDFicmg5SHR2aWM3V2hoWk5PZG84Tnd1TXQ1L3BiMjVE?=
 =?utf-8?B?c0hoWHMrek9LZXNEb1lPNXh3eUgvdU1zcGpVMnM2cXNlM3ptTGJKWHI5SnFp?=
 =?utf-8?B?NHpSbDNrVG82Vmd4NENvN29SQzd0a0ZGZHduZm02YTMyQ1I3NSsvNWRZWnhX?=
 =?utf-8?B?VEg2TEhubG9TL2FoQlFoeXpRdnpKd01xTkt1Sm5YcHZkQTc3dStsTUpnS0lh?=
 =?utf-8?B?SG95MUlBc1Q5UzNFdTFIczRaL2VDUFJzU09MdCtiYkVDaWdXVEJLTmdxTzlr?=
 =?utf-8?B?ZzhWbzZPM1BHNGVqUVFJRjF3NFJ4dTZGNVdJS3paaURCUFRYamdQL1YxUUdE?=
 =?utf-8?B?Z2REbDU0N0RBZW5GN1NvNXprV2ZoUmlqa0NPZnZGRkZRR1lmR25BTzdmaEt6?=
 =?utf-8?B?WGtmV1RtY01HMmJzUWs5b0h2NkZYSzJPN0J4TTdOZ0hKVFY4VlBkck1yV1pD?=
 =?utf-8?B?cjZ4UWM5cTNOS2ZzeERZdz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?bmxXekNxVEJYK3BjY0hsYUgreXhUbGVtTmdLQ28rbjc1MGE1eWo3eFlSZnBT?=
 =?utf-8?B?TFUyM0ZVaTF4Skx3OXpCS3NHQkZnZS9PdUt4a282ZEZnMEh6a3g3bVpmcDZs?=
 =?utf-8?B?aW9Ndm1WQkFuT05Jelh1MklvdlVwRmM4OTBqN1NZU2dYOWV6RmpxU2FwTjdM?=
 =?utf-8?B?cUxXaVJKVDdLZHRNNlg2Nk5KdWR3TzBEaHJCaXdLS3BQNHUzOVY3amFFcW9i?=
 =?utf-8?B?WHd4NkprQTRsUXJDdWlFR09wWjdtZ0FJcmVNYzlwb0FPeFlCeEw0RHlCb2NP?=
 =?utf-8?B?VFNPZmsxRGFIL3pjRU9ERmQ5VDk3NFBPYUYrdTNrUFI4V3pBamY1QUZEMHF1?=
 =?utf-8?B?S3hNbGk3dDFlZDZxQ04rWDVCbWZjQ1RYOUZJTyt0RGVyM2Izb1B4ejRyekxC?=
 =?utf-8?B?MzNkRnJmU21HcnYyZDhmdGZTdE1HRGdDVHBEMC9sWU85Tml5eXBaNnJnb1hX?=
 =?utf-8?B?TWplVkx6MDY2RURmMDNzRjlTT1lmNmdsTzBTUE8vYXpUdXZWQldybHhRQVFP?=
 =?utf-8?B?NWlnTTRhUy9sMzZNOEtUb2oyeFZnNU9iUnJsSHdjb3NNaG1SRkRSTURPa3Ba?=
 =?utf-8?B?cjVRTmlrYzZUVGZqQitSZGhvQnRib0FPdVhKY1VSbHNCUXBSUEFaWGNvOGpQ?=
 =?utf-8?B?VE1hWXJxQ0FVL2h4S21wYnoxcWlaQStpN0ViVldWMnM0SThKSFZpMnZkUlFj?=
 =?utf-8?B?VHhjRzdkdHVnZGw1ejErTjhYK3AyMlEyVEpuWmpDLzYrTkRIYjRpTmxZTmpx?=
 =?utf-8?B?Z0I4YUx6bmNKU3RBU0JaV0pycVlJeHhoeHVwMSsrcHhFeU9abjZkdFE4ZmhE?=
 =?utf-8?B?NXZTKzh4QTk1WXlZdzZRSTVVLzZaaVZuYjVoNGJmalBXdU9zTjFTZUVsdEpS?=
 =?utf-8?B?QWl5a3BqcVVQVkVUVUp2alhWdnphb0JuNDYzVUgwajQvd1phU29Gd1dHcU4x?=
 =?utf-8?B?UWp1akdkRDNud1N1TEFaejhORlBrSzFEQk53UmpqeDNlc2loVTNjTFMzOVp6?=
 =?utf-8?B?UVVjSzFZWGxDTTBuYUZTQkdocHlERkhPcyt6NWhaMTNUZUtxM1Flb3NXRnVI?=
 =?utf-8?B?NWpyNk41dkdPRktrOE1UVjZieWVDbXJsd3YzWG8rRys4L2Npd0phL0dVYk5v?=
 =?utf-8?B?RlF0SjloNEpnVmhncm5XQWN5bVJFRWgwV3BBbWxOQnE5NTFRUWtvOE4vK3Jn?=
 =?utf-8?B?aGRuQ3BDMnNGTS80dStqcHJzSndlREd2Nyt1bERDRXdRVGp6VFAxV0FDQ05U?=
 =?utf-8?B?T2Rmcm9xQzJESEhSd3M1anB1SjR6TjVGSmlWaXNwTGlTTGcxNGVUbmFTZ09P?=
 =?utf-8?B?TUNKdmFCUDVROHM1R1IraDNnekpMcnA4NDRmQm5rODcxQzVaTFNqUzliYlFu?=
 =?utf-8?B?V2ZqMUw3OUJqQ21FK3lCZG9hYmFYWWxPNmM3VDEyaDJoSHo4bGVPa21RZHdP?=
 =?utf-8?B?TEJMS2k1djNNMDhEaXM0Nkwvd08wTDRLbDJZSTBlRGNjNmlMQUl1QW5PaWlM?=
 =?utf-8?B?RCtkK05IdFJHRUdLb1FmSFlRbTAvYmpSV3RJeE14cWJLcVhWQzdhWGVXWXph?=
 =?utf-8?B?aWdyTUpEUmwxU2tZRVVtUEZUUThPK3JaUktVTEtUV04vcGpsTGhpUXRSWVV0?=
 =?utf-8?B?KzRPSjBKVldmQjMycVVRQThPYXZqcHRGSktiR01DcXorcVV2TEZ4VFQ3dTRp?=
 =?utf-8?B?eGw5N2pJRWhQVkdvVDhDUUNlOHp3Mkhtb0FwZmZZWmIrdEM1WlgxMkNSMW15?=
 =?utf-8?B?ZFFKcUhURmNFeW9FNXRMKzV2Rzcvd0pzMUEyQWtzcEtod1ZwTnVHNEI3cUhN?=
 =?utf-8?B?aXJXZ1R6MHJwakVCb05xZEl3c0U4VzhzUS93eEJqV24yTlBXVW1Gc25RUW5H?=
 =?utf-8?B?SGh1L2lrNEFRWmdKcUd4cGlzVGc0T3JRcXJOL3UxM3BNanIwUTRNRVE1eVdl?=
 =?utf-8?B?YjR6RHE2bDNUUzYrZVRpSmVDcWVlZWhLaXBZdEZJa2FpMm52am1WK2VrWm41?=
 =?utf-8?B?MnVSbzZaL2tKZXBORm0zdDl3MmQ4RWZMOGJtTDhkcjNjazBSTkt6VmJZKzFa?=
 =?utf-8?B?OElDcEFSTDVKQU1PbFlsaFluU3lBSTRrOFkwZmc3MEJGZFYyQi9sbXlvdG1i?=
 =?utf-8?Q?wu4Y=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <CC6BE08C21267F45A1048F77C5C991F6@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5351b627-e5bb-41a0-c6d0-08dc7160512d
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 May 2024 02:16:07.1936
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: QnuotWuZ5u7AN8z5Bw/GmMZJsz1Qd2F05kxZpiwVBk9RbxH3Yh8bzAmUoLfC/KMm2DLW+DztYZDsJdk5bMCyMw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7449

T24gMjAyNC81LzEwIDE5OjI3LCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPiBPbiAxMC4wNS4yNCAx
MjozMiwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzEwIDE4OjIxLCBKw7xyZ2Vu
IEdyb8OfIHdyb3RlOg0KPj4+IE9uIDEwLjA1LjI0IDEyOjEzLCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IE9uIDIwMjQvNS8xMCAxNzo1MywgSsO8cmdlbiBHcm/DnyB3cm90ZToNCj4+Pj4+IE9u
IDEwLjA1LjI0IDExOjA2LCBDaGVuLCBKaXFpYW4gd3JvdGU6DQo+Pj4+Pj4gSGksDQo+Pj4+Pj4N
Cj4+Pj4+PiBPbiAyMDI0LzUvMTAgMTQ6NDYsIErDvHJnZW4gR3Jvw58gd3JvdGU6DQo+Pj4+Pj4+
IE9uIDE5LjA0LjI0IDA1OjM2LCBKaXFpYW4gQ2hlbiB3cm90ZToNCj4+Pj4+Pj4+ICsNCj4+Pj4+
Pj4+ICvCoMKgwqAgaW5mby0+dHlwZSA9IElSUVRfUElSUTsNCj4+Pj4+PiBJIGFtIGNvbnNpZGVy
aW5nIHdoZXRoZXIgSSBuZWVkIHRvIHVzZSBhIG5ldyB0eXBlKGxpa2UgSVJRVF9HU0kpIGhlcmUg
dG8gZGlzdGluZ3Vpc2ggd2l0aCBJUlFUX1BJUlEsIGJlY2F1c2UgZnVuY3Rpb24gcmVzdG9yZV9w
aXJxcyB3aWxsIHByb2Nlc3MgYWxsIElSUVRfUElSUS4NCj4+Pj4+DQo+Pj4+PiByZXN0b3JlX3Bp
cnFzKCkgYWxyZWFkeSBjb25zaWRlcnMgZ3NpID09IDAgdG8gYmUgbm90IEdTSSByZWxhdGVkLiBJ
c24ndCB0aGlzDQo+Pj4+PiBlbm91Z2g/DQo+Pj4+IE5vLCBpdCBpcyBub3QgZW5vdWdoLg0KPj4+
PiB4ZW5fcHZoX2FkZF9nc2lfaXJxX21hcCBhZGRzIHRoZSBtYXBwaW5nIG9mIGdzaSBhbmQgaXJx
LCBidXQgdGhlIHZhbHVlIG9mIGdzaSBpcyBub3QgMCwNCj4+Pj4gb25jZSByZXN0b3JlX3BpcnFz
IGlzIGNhbGxlZCwgaXQgd2lsbCBkbyBQSFlTREVWT1BfbWFwX3BpcnEgZm9yIHRoYXQgZ3NpLCBi
dXQgaW4gcHZoIGRvbTAsIHdlIHNob3VsZG4ndCBkbyBQSFlTREVWT1BfbWFwX3BpcnEuDQo+Pj4N
Cj4+PiBPa2F5LCB0aGVuIGFkZCBhIG5ldyBmbGFnIHRvIGluZm8tPnUucGlycS5mbGFncyBmb3Ig
dGhhdCBwdXJwb3NlPw0KPj4gSSBmZWVsIGxpa2UgYWRkaW5nICJuZXcgZmxhZyB0byBpbmZvLT51
LnBpcnEuZmxhZ3MiIGlzIG5vdCBhcyBnb29kIGFzIGFkZGluZyAiIG5ldyB0eXBlIHRvIGluZm8t
PnR5cGUiLg0KPj4gQmVjYXVzZSBpbiByZXN0b3JlX3BpcnFzLCBpdCBjb25zaWRlcnMgIiBpbmZv
LT50eXBlICE9IElSUVRfUElSUSIsIGlmIGFkZGluZyAiIG5ldyBmbGFnIHRvIGluZm8tPnUucGly
cS5mbGFncyIsIHdlIG5lZWQgdG8gYWRkIGEgbmV3IGNvbmRpdGlvbiBpbiByZXN0b3JlX3BpcnFz
Lg0KPj4gQW5kIGFjdHVhbGx5IHRoaXMgbWFwcGluZyhnc2kgYW5kIGlycSBvZiBwdmgpIGRvZXNu
J3QgaGF2ZSBwaXJxLCBzbyBpdCBpcyBub3Qgc3VpdGFibGUgdG8gYWRkIHRvIHUucGlycS5mbGFn
cy4NCj4gDQo+IERvZXMgdGhpcyBtZWFuIHRoZXJlIGlzIG5vIG90aGVyIElSUVRfUElSUSByZWxh
dGVkIGFjdGl2aXR5IHJlbGV2YW50IGZvciB0aG9zZSBHU0lzL0lSUXM/DQpZZXMsIEkgdGhpbmsg
c28uDQo+IEluIHRoYXQgY2FzZSBJIGFncmVlIHRvIGFkZCBJUlFUX0dTSS4NClRoYW5rIHlvdSEN
Cj4gDQo+IA0KPiBKdWVyZ2VuDQoNCi0tIA0KQmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Sat May 11 02:24:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 02:24:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720185.1123277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5cP6-0006Rg-8V; Sat, 11 May 2024 02:24:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720185.1123277; Sat, 11 May 2024 02:24:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5cP6-0006RZ-4y; Sat, 11 May 2024 02:24:04 +0000
Received: by outflank-mailman (input) for mailman id 720185;
 Sat, 11 May 2024 02:24:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5cP4-0006RO-LG; Sat, 11 May 2024 02:24:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5cP4-00046U-Da; Sat, 11 May 2024 02:24:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5cP3-000641-SH; Sat, 11 May 2024 02:24:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5cP3-0001RQ-Rq; Sat, 11 May 2024 02:24:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7dEWBD624PR1rQg3/IKsy0UEcQHMCIvPjFS3PuKGttw=; b=040kVxVvzPr934CdpIhp2YjfHQ
	H7Fk5m+CTBPlDw0JXSawzrljbkmixfPZ4x3ZxKIJPUEclTFs4qI/mWLLXuJyWC65IFlY9vd0pd5h0
	d4tDzT4iO5fhARzNcFOYkOgxgEowyvepv22gGlmzK1pk2NTMNAjX2CbHsePD5aK1HlzM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185972-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185972: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f4345f05c0dfc73c617e66f3b809edb8ddd41075
X-Osstest-Versions-That:
    linux=448b3fe5a0eab5b625a7e15c67c7972169e47ff8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 02:24:01 +0000

flight 185972 linux-linus real [real]
flight 185974 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185972/
http://logs.test-lab.xenproject.org/osstest/logs/185974/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-pvshim 22 guest-start/debian.repeat fail pass in 185974-retest
 test-armhf-armhf-examine      8 reboot              fail pass in 185974-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185965
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185965
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185965
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185965
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185965
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185965
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                f4345f05c0dfc73c617e66f3b809edb8ddd41075
baseline version:
 linux                448b3fe5a0eab5b625a7e15c67c7972169e47ff8

Last test of basis   185965  2024-05-10 02:20:02 Z    1 days
Testing same since   185972  2024-05-10 18:14:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alina Yu <alina_yu@richtek.com>
  Ben Wolsieffer <ben.wolsieffer@hefring.com>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Jason Gunthorpe <jgg@nvidia.com>
  Jens Axboe <axboe@kernel.dk>
  Jerry Snitselaar <jsnitsel@redhat.com>
  Joerg Roedel <jroedel@suse.de>
  Johan Hovold <johan+linaro@kernel.org>
  Keith Busch <kbusch@kernel.org>
  Levi Yun <ppbuk5246@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mark Brown <broonie@kernel.org>
  Maurizio Lombardi <mlombard@redhat.com>
  Ming Lei <ming.lei@redhat.com>
  Sagi Grimberg <sagi.grimberg@vastdata.com>
  Sagi Grimberg <sagi@grimberg.me>
  Sean Anderson <sean.anderson@linux.dev>
  Thomas Gleixner <tglx@linutronix.de>
  Vasant Hegde <vasant.hegde@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   448b3fe5a0ea..f4345f05c0df  f4345f05c0dfc73c617e66f3b809edb8ddd41075 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat May 11 04:10:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 04:10:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720203.1123287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5e3r-0003WT-PX; Sat, 11 May 2024 04:10:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720203.1123287; Sat, 11 May 2024 04:10:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5e3r-0003WM-MC; Sat, 11 May 2024 04:10:15 +0000
Received: by outflank-mailman (input) for mailman id 720203;
 Sat, 11 May 2024 04:10:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ElH0=MO=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1s5e3p-0003WG-Ty
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 04:10:13 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5b39f051-0f4c-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 06:10:10 +0200 (CEST)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.17.1) with ESMTPS id 44B49s3g016991
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Sat, 11 May 2024 00:10:00 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 44B49s11016990;
 Fri, 10 May 2024 21:09:54 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b39f051-0f4c-11ef-b4bb-af5377834399
Date: Fri, 10 May 2024 21:09:54 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>,
        Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
        Wei Liu <wl@xen.org>, Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Serious AMD-Vi(?) issue
Message-ID: <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
References: <Zf3aWXfCANR7zXj8@mattapan.m5p.com>
 <e9b1c9c4-523b-481b-946e-37c7c18ea1d2@suse.com>
 <ZgHwEGCsCLHiYU5J@mattapan.m5p.com>
 <ZgRXHQpamLIdu7dk@mattapan.m5p.com>
 <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Thu, Apr 18, 2024 at 09:33:31PM -0700, Elliott Mitchell wrote:
> 
> I suspect this is a case of there is some step which is missing from
> Xen's IOMMU handling.  Perhaps something which Linux does during an early
> DMA setup stage, but the current Xen implementation does lazily?
> Alternatively some flag setting or missing step?
> 
> I should be able to do another test approach in a few weeks, but I would
> love if something could be found sooner.

Turned out to be disturbingly easy to get the first entry when it
happened.  Didn't even need `dbench`, it simply showed once the OS was
fully loaded.  I did get some additional data points.

Appears this requires an AMD IOMMUv2.  A test system with known
functioning AMD IOMMUv1 didn't display the issue at all.

(XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr fffffffdf8000000 flags 0x8 I
(XEN) DDDD:bb:dd.f root @ 83b5f5 (3 levels) dfn=fffffffdf8000
(XEN)   L3[1f7] = 0 np

I find it surprising this required "iommu=debug" to get this level of
detail.  This amount of output seems more appropriate for "verbose".

I strongly prefer to provide snippets.  There is a fair bit of output,
I'm unsure which portion is most pertinent.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Sat May 11 05:26:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 05:26:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720212.1123296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5fFC-0004ZO-WA; Sat, 11 May 2024 05:26:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720212.1123296; Sat, 11 May 2024 05:26:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5fFC-0004ZH-TV; Sat, 11 May 2024 05:26:02 +0000
Received: by outflank-mailman (input) for mailman id 720212;
 Sat, 11 May 2024 05:26:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5fFB-0004Z7-0M; Sat, 11 May 2024 05:26:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5fFA-0007vO-Ux; Sat, 11 May 2024 05:26:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5fFA-0006WN-Lf; Sat, 11 May 2024 05:26:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5fFA-0001MF-L2; Sat, 11 May 2024 05:26:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=plZEvqEJ1GRAy8IlWhZfjPqdVxXBsPq06xCU+aHt4zk=; b=N6wMRI8vEnJzalKfd1EA0vZFZ/
	z8WevZa+8tFRfjRJe+8C4Rg+X1q1ySSUN6na5DQBEDjYt81EoEhDr3XA//DvK/y3wWsLU3VmKnxrB
	BB2NGnJIITzWazAM3GR7OUx5zaber7gjJ/zVerkJiF9/hnEng8sMnLh8SYkhoP9CqmWs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185976-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185976: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=996576b965ccdf0de17aafa14282925e408e1200
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 05:26:00 +0000

flight 185976 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185976/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  996576b965ccdf0de17aafa14282925e408e1200

Last test of basis   185973  2024-05-10 22:04:11 Z    0 days
Testing same since   185976  2024-05-11 02:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Leigh Brown <leigh@solinno.co.uk>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   996576b965..46aa3031ae  46aa3031ae89ac1771f4159972edab65710e7349 -> smoke


From xen-devel-bounces@lists.xenproject.org Sat May 11 07:30:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 07:30:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720227.1123307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5hBB-0002tJ-Fb; Sat, 11 May 2024 07:30:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720227.1123307; Sat, 11 May 2024 07:30:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5hBB-0002tC-Be; Sat, 11 May 2024 07:30:01 +0000
Received: by outflank-mailman (input) for mailman id 720227;
 Sat, 11 May 2024 07:30:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5hBA-0002t5-GR
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 07:30:00 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20605.outbound.protection.outlook.com
 [2a01:111:f400:7e88::605])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 43d4684c-0f68-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 09:29:58 +0200 (CEST)
Received: from CH0PR03CA0309.namprd03.prod.outlook.com (2603:10b6:610:118::30)
 by MN0PR12MB6001.namprd12.prod.outlook.com (2603:10b6:208:37d::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54; Sat, 11 May
 2024 07:29:51 +0000
Received: from DS3PEPF000099E2.namprd04.prod.outlook.com
 (2603:10b6:610:118:cafe::73) by CH0PR03CA0309.outlook.office365.com
 (2603:10b6:610:118::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54 via Frontend
 Transport; Sat, 11 May 2024 07:29:51 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF000099E2.mail.protection.outlook.com (10.167.17.201) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 07:29:50 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sat, 11 May
 2024 02:29:50 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sat, 11 May 2024 02:29:48 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43d4684c-0f68-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HD5PtHUD/UEHJ3cPC5NTNAN4rvPJ4RwTRc0j7wPcSfoD4Nk/p/a/DSe0vXPRUNN48+KQ9gOMnfSVv8CDetQv9Lf3cfSzyanVu8VM64EXVZFhPzbYSHVgBZVPr7HeYHWvCKoMoXLmn/gsHVyXcz6CIxYkGsAAZY8PDOEMEbKQuz/W4OTXmtB66ohtUEHJng6rCmxcE9jHxAYvTbUypdCd7ucV5qquhBE/OK/JrLxj8OEuqDLb+lZA98sPZ1NpHjiIhmgv046cIyPX6u6ccaqc8KD1hrDOqZTFV0ZQKBx7vLits3XaDNaE1vtGuXXDABUgCzWTKxMx7FSb7ZIR3IxZ8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hSIOxDBb6opTW+eXESXU5shvpcAEyhf3/skZqhC1BIw=;
 b=W8WKfdfdIJZLbYgLccE6OheeEZ5+5mjIX48pBZCaMSazC50LmHMxMvsVRnnOP+UQoy7m0ECPkvAILerTOT8SHfhoJAPr9E4NV3m4BO6X1gIvWkKPWcorlMTlCAaRZjQX6Re+XvcZTvoNIEiKhxkPosspN7L+3h0i16xncuDSJ9aGIsYuFsiaY/+SNXfUtCRvfRvWDBJIyzGxrZF838Fzm2PLJjWtuIfuJ9jhJ6+aIvzuVTmtvlf+m7/6U2Pc/hX9pHLP21q19LvRbL34mapjWfugOhDjCLZbQIN7daj9D1cxT+tTnntoCX1ku6vZwWWRw5gzTOv4byjdIhUrcf8gUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hSIOxDBb6opTW+eXESXU5shvpcAEyhf3/skZqhC1BIw=;
 b=EJo4M7KeTgOOGy2q26bxnGBCJhRoP0HU58ZZzdFClZ11QA3Lgey8CkCDZdgKEqd07O/7s9G9YUh3D5/4ITv+4K+9IIMNBnChdW3wFPQKw0Yu9CiTFC/ypysLf9mT98nmxBnv6lQEgt+jzwwmBzBUREKn9lDbmp+LtM31NdQjRG0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <1e9c6513-8e45-4202-a25e-bc342072d6cd@amd.com>
Date: Sat, 11 May 2024 15:29:47 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
 <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
 <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
 <1ad2c873-219c-4d96-82d6-e1bc58216f2d@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <1ad2c873-219c-4d96-82d6-e1bc58216f2d@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099E2:EE_|MN0PR12MB6001:EE_
X-MS-Office365-Filtering-Correlation-Id: 774ccb05-707a-4d05-b2f9-08dc718c24df
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?R0hKaHNXcGNXR0Z5Q2xnbDZFWWZrbSs4Q1V4OCtuQ0wwWUZaQTU4elY1aUVx?=
 =?utf-8?B?QkVGa2hEd0J4V3FTZHU0UXVNc01NWVhYbTNEWHVZa01aMnVIc0dNd3BxR3ZM?=
 =?utf-8?B?UFh1ck1DN21PdUlxT1MxMmdwTmJEMFNseE05OWdBWStKc3lLellvVFJXVTkr?=
 =?utf-8?B?emdKSHpZYTMwSTV4aVZQOVd0RFFkekRtL1dBbGt0MndRenprMVFLbnJNT09w?=
 =?utf-8?B?d3dDZ3RpRlBzeTZaem44b0xORWVsZHMyRGdDOStxejJkMG84OStudUxncCsx?=
 =?utf-8?B?OW85WDZ0VEVDeHJicFV1N1d1S1krMXRuRUJMY1k0Wk4veGJMTG9xTDVPSjRP?=
 =?utf-8?B?SEQyU2hRcjJQS1lmYjFjVWd5Z21vRkU4UGVUSTVheU1ueHozaVpsQVBYNWQr?=
 =?utf-8?B?c0p0Q1RCSVNwQWRQZThKOUZCQUNmbXRpdDVtbGpIMDNZWHNieUZydmJrK1lF?=
 =?utf-8?B?cVVEc09kU2RvamQ5K1JXSTVEZXNsM1RrbndsZU1tS0JZcGtDdHI5aXZneEov?=
 =?utf-8?B?S2ljbGRwejFJeE5jbktiQ29RVm54NTE2QW1ZQWoxUTZTcjZSRkRHV1NpK3Ax?=
 =?utf-8?B?ZDh5UjdiY2tkWlJpOU1QTVFnOUxKWEduZWJsSTdlb0huUWIySUhQREFpVmd0?=
 =?utf-8?B?TUc1WjVnUE4ya2NOMklFaEhEbytuelJZZlVKUy9yYkM4enVzUzJKOWJQL1Fl?=
 =?utf-8?B?ZnhYV0tmbHRaZUNsNGxzMEMwdGNsYUlVVHNheENjYnhnbi8xRFN4c09VYkZZ?=
 =?utf-8?B?U29SRmRUaUViNkFYUjFYWGZsMmZ4MDh5aTN1Y0FRQ05NeU5lMEIrL0hJdDBu?=
 =?utf-8?B?QkdxVEthVFFZWVA3VThoczBmNzRCZ0JGcDJPaVlhemJiVUpESFhYWXFxRmo2?=
 =?utf-8?B?SGNxV3Fka0d2TTFxMkY2c09wTzJnU3oyYUZaazRuc1FDUDM4OFpzMkZsMWx1?=
 =?utf-8?B?MjlVbURidFMwY3k1M29FdEg4MkRaUitiaDNlZkN0UzVLRks4Vm5WNGlCWGl3?=
 =?utf-8?B?OXRwQ1RpVG9RZmQzUzRkZWs0dlpTUmY1MkpLb3JNaVFmZU82MDNla244VlVT?=
 =?utf-8?B?WThrQU14SU9mZGwwSUk3bW85SlpnNkFEb1o4Wm1vRUxRWUwxQkNiaDJIOUFn?=
 =?utf-8?B?alhBMXN0UkQwQ3lWZUQrckx0K2lRbFdFTjJ1SDRyVG5aU29BZ0kyazFhMktI?=
 =?utf-8?B?bXBCQ1hUVjRMU2lRbkptOWwzUVdvY1Z3SHhoK1hRZ09yYTRXaWpsTFFOaEpl?=
 =?utf-8?B?ZzVwUS9UU1pqc3ZCTnB4VzgrWnNUNjRZNE10VlVCNXFJSm5vc2c2YUZ5Q29u?=
 =?utf-8?B?KzVaL3NJeEJTdkFlTUxWdTh2OHNiaWZzT0l0MjBqYVVZSFpTUzA3THhFZjhj?=
 =?utf-8?B?RUFjanpNYzZwclcwRGZLZFRqWHQ0SHNtWC9udFhwUkhZMndpZnRQNnB4Q3BX?=
 =?utf-8?B?NzVyUmdYTjY5MFpKcDZWaGNzaVRlNHNIWXN0UmN4V1lHS204c1VvRE1GRldp?=
 =?utf-8?B?cGNRR3dQK0Z0UXBvZ09RQ2pqcng2Z0pjQzdoOUl0bDF5YWFValNBSlBIRVNu?=
 =?utf-8?B?QUsyNEpOMDIzY1BYY3hDU25RaTY0ZS9CckFPRms3U0lKaW80Znh1OVVyTDBE?=
 =?utf-8?B?WnVaU2hhSVdTMUJ1QlpxYmVsSTlYeUZRQmhlRlhYaElWSkJXdk92UDZTTDIv?=
 =?utf-8?B?RXp2ckNFQTdvYjJBbXpXOXBReCtMK1MzZ2NZZHlxWnl6N080UlpuSDlpN0ZO?=
 =?utf-8?B?bUZIZGxGN0oveDdZeEdhZWZDQXIrU0ZmdFBPQzFRV3V0VE1tbXlVU25mdEtl?=
 =?utf-8?B?T1JiVnI1SVJKbjRrOVhBend2dXpiTUxPdGxjT2JTUVBSNGFlem9OY1hBRVhD?=
 =?utf-8?Q?psYvby3eRT5Zo?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 07:29:50.6986
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 774ccb05-707a-4d05-b2f9-08dc718c24df
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099E2.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6001

Hi Julien,

On 5/10/2024 4:54 PM, Julien Grall wrote:
> Hi,
>
> On 09/05/2024 16:31, Henry Wang wrote:
>> On 5/9/2024 4:46 AM, Julien Grall wrote:
>>> Hi Henry,
>>> [...]
>>> ```
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index a775f886ed..d3f9cd2299 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -135,16 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, 
>>> unsigned int virq,
>>>       ASSERT(virq < vgic_num_irqs(d));
>>>       ASSERT(!is_lpi(virq));
>>>
>>> -    /*
>>> -     * When routing an IRQ to guest, the virtual state is not synced
>>> -     * back to the physical IRQ. To prevent get unsync, restrict the
>>> -     * routing to when the Domain is been created.
>>> -     */
>>> -#ifndef CONFIG_OVERLAY_DTB
>>> -    if ( d->creation_finished )
>>> -        return -EBUSY;
>>> -#endif
>>> -
>>>       ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
>
> This is checking if the interrupt is already enabled. Do we also need 
> to check for active/pending?

Thank you for raising this! I assume you meant this?
@@ -444,7 +444,9 @@ int vgic_connect_hw_irq(struct domain *d, struct 
vcpu *v, unsigned int virq,
      {
          /* The VIRQ should not be already enabled by the guest */
          if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
              p->desc = desc;
          else
              ret = -EBUSY;

I think adding the check for active/pending check at the time of routing 
the IRQ makes sense, so I will add them (both for old and new vGIC 
implementation).

>>       if ( ret )
>>           return ret;
>> @@ -169,20 +159,40 @@ int gic_remove_irq_from_guest(struct domain *d, 
>> unsigned int virq,
>>       ASSERT(test_bit(_IRQ_GUEST, &desc->status));
>>       ASSERT(!is_lpi(virq));
>>
>> -    /*
>> -     * Removing an interrupt while the domain is running may have
>> -     * undesirable effect on the vGIC emulation.
>> -     */
>> -#ifndef CONFIG_OVERLAY_DTB
>> -    if ( !d->is_dying )
>> -        return -EBUSY;
>> -#endif
>> -
>>       desc->handler->shutdown(desc);
>>
>>       /* EOI the IRQ if it has not been done by the guest */
>>       if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
>> +    {
>
> I assume this is just a PoC state, but I just want to point out that 
> this will not work with the new vGIC (some of the functions doesn't 
> exist there).

Thank you. Yes currently we can discuss for the old vGIC implementation. 
After we reach the final conclusion I will do the changes for both old 
and new vGIC.

>> +        /*
>> +         * Handle the LR where the physical interrupt is de-assigned 
>> from the
>> +         * guest before it was EOIed
>> +         */
>> +        struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], virq);
>
> This will return a vCPU from the current affinity. This may not be 
> where the interrupt was injected. From a brief look, I can't tell 
> whether we have an easy way to know where the interrupt was injected 
> (other than the pending_irq is in the list lr_queue/inflight)

I doubt if we need to handle more than this - I think if the pending_irq 
is not in the lr_queue/inflight list, it would not belong to the corner 
case we are talking about (?).

>> +        }
>> + spin_unlock_irqrestore(&v_target->arch.vgic.lock, flags);
>> +
>> +        vgic_lock_rank(v_target, rank, flags);
>> +        vgic_disable_irqs(v_target, (~rank->ienable) & 
>> rank->ienable, rank->index);
>> +        vgic_unlock_rank(v_target, rank, flags);
>
> Why do you need to call vgic_disable_irqs()?

I will drop this part.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Sat May 11 08:22:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 08:22:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720239.1123316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5i0J-0002f5-ON; Sat, 11 May 2024 08:22:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720239.1123316; Sat, 11 May 2024 08:22:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5i0J-0002ey-Ll; Sat, 11 May 2024 08:22:51 +0000
Received: by outflank-mailman (input) for mailman id 720239;
 Sat, 11 May 2024 08:22:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5i0I-0002es-Ma
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 08:22:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5i0I-00037b-Bs; Sat, 11 May 2024 08:22:50 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5i0I-0004cu-2P; Sat, 11 May 2024 08:22:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=w7+1Ax8sDLaEv0ZjTTR00m0X/003BDMxt3SFxsvnG1E=; b=GdJvMwFeO6pA6GpggeOCDw0iYF
	orfv7BsPjDQzyUu6vKJlhGdWYm8+hDpE6YWRKB+RBEB2gkWu0Cn/baNTvzqfxTFaBGuZRcwALrwf+
	VkbbUAHmWHPKiYmg4id+HHto7E4+6iEynggqOyoI/7YaN49ptqfwnMB3ta8ds7HLqptc=;
Message-ID: <f2580cb8-c391-43c7-9696-200f9633dd4a@xen.org>
Date: Sat, 11 May 2024 09:22:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
 <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
 <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
 <1ad2c873-219c-4d96-82d6-e1bc58216f2d@xen.org>
 <1e9c6513-8e45-4202-a25e-bc342072d6cd@amd.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <1e9c6513-8e45-4202-a25e-bc342072d6cd@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 11/05/2024 08:29, Henry Wang wrote:
>>> +        /*
>>> +         * Handle the LR where the physical interrupt is de-assigned 
>>> from the
>>> +         * guest before it was EOIed
>>> +         */
>>> +        struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], virq);
>>
>> This will return a vCPU from the current affinity. This may not be 
>> where the interrupt was injected. From a brief look, I can't tell 
>> whether we have an easy way to know where the interrupt was injected 
>> (other than the pending_irq is in the list lr_queue/inflight)
> 
> I doubt if we need to handle more than this - I think if the pending_irq 
> is not in the lr_queue/inflight list, it would not belong to the corner 
> case we are talking about (?).

I didn't suggest we would need to handle the case where the pending_irq 
is not any of the queues. I was pointing out that I think we don't 
directly store the vCPU ID where we injected the IRQ. Instead, the 
pending_irq is just in list, so we will possibly need to store the vCPU 
ID for convenience.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 08:35:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 08:35:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720245.1123327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5iCi-0004gI-SZ; Sat, 11 May 2024 08:35:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720245.1123327; Sat, 11 May 2024 08:35:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5iCi-0004gB-PQ; Sat, 11 May 2024 08:35:40 +0000
Received: by outflank-mailman (input) for mailman id 720245;
 Sat, 11 May 2024 08:35:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5iCh-0004g5-T5
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 08:35:39 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 70d687c8-0f71-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 10:35:38 +0200 (CEST)
Received: from PH7PR13CA0001.namprd13.prod.outlook.com (2603:10b6:510:174::7)
 by DM4PR12MB8523.namprd12.prod.outlook.com (2603:10b6:8:18e::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.51; Sat, 11 May
 2024 08:35:33 +0000
Received: from CY4PEPF0000E9D0.namprd03.prod.outlook.com
 (2603:10b6:510:174:cafe::f2) by PH7PR13CA0001.outlook.office365.com
 (2603:10b6:510:174::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.20 via Frontend
 Transport; Sat, 11 May 2024 08:35:32 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000E9D0.mail.protection.outlook.com (10.167.241.135) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 08:35:31 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sat, 11 May
 2024 03:35:31 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sat, 11 May
 2024 03:35:31 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sat, 11 May 2024 03:35:28 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70d687c8-0f71-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GAAXROwKecQIuKBmSX7OxN5ruENGQa8qwQAlakNxbBw3IN1olielxQOfcAoB++3CxALl4W0NvHd+GG8yB63KvwVTEOxSZbHc+h0YcH/mJaOiPXVDPYu5/Nq7+5qAPw7USd8EluV1BnNFkHoo5j/iPlsHcTXlcAOEqASQbtHXSX1YMc2FRw9tT9JO4R9+Rb5erSH/a9qAgg30H6TFpl8hbxRW3nokHEd0NUVpLBWJ/UIq/NVp0YU12/izMVjCVLCaAfOqzaRhhFC/tD7kT8UVTz2BPBeO1tNp57DdMAAt2jUNhA+wmKo1uG1a2+fiKmJ4bpJlBSPyM0M9Eu3Wb10nXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JDpgzmes3wIY3sYaKYUQcA9ltTQSQkPL7jlEx+Brflk=;
 b=BWNL/PVaWdgtsgCGFToLPtMqb8tiEbsj8GjLMgyggbEF0i3aCMqJcUXqyJMr3UjCdjAZPGMTqAWbcarow0GA6UqHsOetJUz+oM2fGRjewph84zChDuxRStzMNwPZRriOa/MsfdR1PflPWQglRhzbOsgvmBfQsxW6cTlNvFRR6ZPPExdzRuuG2nUvKAXqITW6kmdO/nQVtZkQLmwHHCADYzHxQRxhgayyEOyoRTbTmvOGBdjJ+gFiSeVr6RjDxar5I1uGc0eRfprmI1fvX8tc5nXJv1rGkKwJykx0SXm6eBFB8m2CBzfPRShEydAIf/hZGQn61py1iOx6O7n7FsnoUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JDpgzmes3wIY3sYaKYUQcA9ltTQSQkPL7jlEx+Brflk=;
 b=IyW2SPVDZ0OWCMMNBK56Xe2SWg+2wjIrexsxINbqVYnt9tVL5EA3h/qcHcUU0xO7iB86f+UNwOGmV07OA1+UjiynO2JPoHX5ivsvmRa7HlbBuedxgHMsgCLAJU092TrUH54AJSgC8Ng2ZaLxx8aAK7BDNmJ0VYJo2BkZ5Z7KJpw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <e8106388-9916-4d1f-acee-f6cbdbf5f364@amd.com>
Date: Sat, 11 May 2024 16:35:27 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/15] xen/arm/gic: Enable interrupt assignment to running
 VM
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>
References: <20240424033449.168398-1-xin.wang2@amd.com>
 <20240424033449.168398-3-xin.wang2@amd.com>
 <670d1033-2443-4245-8ff1-cb7099a7c793@xen.org>
 <70c5a593-9ac5-485b-be81-b789562249d8@amd.com>
 <5ba38b7b-16b1-489b-ad55-083504690bbd@xen.org>
 <1a29ae76-a902-4a24-8fd1-ac26ef228971@amd.com>
 <365e1b12-6b20-40ed-8ca9-beb643c907ab@xen.org>
 <145e4192-3113-415f-876c-e67145d1d5c9@amd.com>
 <f30f7599-6bcd-4b8d-bd1f-6afde18c14c8@xen.org>
 <8957ab21-796f-4e15-a89e-d040e6f7b5ca@amd.com>
 <995ce39d-6a04-4bdc-bfe8-bcef3da24cc2@xen.org>
 <ff5e2477-ed5a-40cd-a592-5ceb3bc8f1f1@amd.com>
 <1ad2c873-219c-4d96-82d6-e1bc58216f2d@xen.org>
 <1e9c6513-8e45-4202-a25e-bc342072d6cd@amd.com>
 <f2580cb8-c391-43c7-9696-200f9633dd4a@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <f2580cb8-c391-43c7-9696-200f9633dd4a@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D0:EE_|DM4PR12MB8523:EE_
X-MS-Office365-Filtering-Correlation-Id: 2fabbb50-767b-4427-7b85-08dc71955214
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?OGVVTk1vbGx3Wnd5TmhYVmdZeng2NlNhMzBDYStrS0lWdEcyWXNvb0kxY3Ur?=
 =?utf-8?B?ZmNnbXptWHc5dDh6UHgwckFEMDBJQ3hKN2l6ZmM2ZEo1dGlUY1NpVWZaMDR1?=
 =?utf-8?B?STQ2VzlDU3NlRHNYSnpwclZ4QTNRZVFjTUVZeGtPbVVibzY3dGU0K0s4Wi94?=
 =?utf-8?B?cFg0OUtPWXBuNVBKQndqYmtHNE04NTRRMTdFV3FvOVdFZUR3cFVKRS9MN1NI?=
 =?utf-8?B?SzVCMjdzbXBHUmpCNW5jWUlxb2t5RzVpUnV5WlFCOXJFVFF1REV3NmozMVc0?=
 =?utf-8?B?VG43NlY5a2xXbkI2b3ZNMGhWZHdNTUEwbEp5a3FuR1lYNnFlUGdOcEtmdXZv?=
 =?utf-8?B?QkdnVXZ0ZWV2cnNRdFJTL1lYSi9HSnRLVHE5cTJmUkZHa3pINGREWEduMlRh?=
 =?utf-8?B?M0ZWN2hvbEloMU1wa2JlRGVVdGx6dzlMVURpSUwvY043WFBjc2Y0c3RHMlEy?=
 =?utf-8?B?T2dHSUVSU0NXYkUrMUd6dmR2TGdkRDBFcHJQcHhRUHErTTYzU0lzWGdPUWdN?=
 =?utf-8?B?THdyb0R6akc4QVJpKzVQeHhYTmpiejNZemcybngyeTZrU010QWhNSU81SXFo?=
 =?utf-8?B?c3NsUEswbjJLU1ZmZ0JRNXR3ZmVpOFc3Wm1PVVNGZE4wK0FGYzZTTnk0c0ZR?=
 =?utf-8?B?QjY0T2VkU2lvRnQ0WWNjU1ZOQWVWUGFPMVQ5V0N1czczeWtEWUJLcHQ2Njdj?=
 =?utf-8?B?ZnJyMUp5bUx1NDJmeE56VXJTbXV5eldyRWkwQ0lmZ2lleHlFaTliQ2tnQmFT?=
 =?utf-8?B?WlBxeGFDR084SXpTR3A4T250N29EWjFiNkFESVNZNmdTMTZvOWJFNUYwTGNE?=
 =?utf-8?B?dHpHQ3pPOUNtWWI2a2xNV3pxN3JJdVQ3YVVsZkZ6cEhIQ3REdzBaS0h1QmRr?=
 =?utf-8?B?MUVDZ29ldHZBaG04MVMvaGJEeEVSY0NPMTUycTcrdEFvZkNSekVFU2x4Q3dP?=
 =?utf-8?B?cWlzOXR5ZmpPbklNREZZc21jNWFlY0pjc200QWJuUzIzbXRlZm1lK2pwK09n?=
 =?utf-8?B?Vm8vZHR6N2pJK3BERGtIeUpWTDdOOTRnUjdPZ3lsZFFKdjlDQ00yd2FmVHZC?=
 =?utf-8?B?ZXlkNUEySys1SUQyREc0aE9lOExZMWx3Q3dhUGJERkx4ZklkTGRUbTRWN0Vl?=
 =?utf-8?B?VmxGZGN6eE1KTC9NQXZKSUc3M0ltZHBLNEtVR0w3SnpaWHpnS1JnczZCeUxx?=
 =?utf-8?B?cSttVHRmQnpRZ3N5ejY5emI5U1hOaVUxVnRzWDBDSmovN3F6MVhrRW1OdGZy?=
 =?utf-8?B?R3MzNTNQeTlBK3oyTStUWHRvRU9pLzd5UWtnQlZ1eHVZVFI2WEE0bGNnTklJ?=
 =?utf-8?B?cS9ZZ0llYU1VVEE1YnZ2R29UaDFVU1U2L3VJYlptbC9uVm5rcE1pdDVvSjJ6?=
 =?utf-8?B?WVlnVTVEYlhNWmN6bXd5cEdNS2sxMGxHaysrN3d4d0ltaFpKY2JCM1pvMzJN?=
 =?utf-8?B?YUZrNlN4QkZ4eXF2MHprRWxxbWM3RHdWcENYUWk3VnRVTS9iOFkzNFRqTXd6?=
 =?utf-8?B?Rkp0ODhiQ3hMVGdrL2g5UWVUaUluL1FoVGtpWCtUaWNSZUtMNkhPaWNXZ1Fk?=
 =?utf-8?B?Sm1zL2VPZWVCTHBjK3J1aFZGSEdacVVweEdBWFhNSzRJK3Y4aUFwa2NWUC9Y?=
 =?utf-8?B?Vmk4ZXpnelc0cjJwUkdzSVZLY1RMU2tXNXVtK25yWkd2OXZHbXUvTnNLKzRQ?=
 =?utf-8?B?bmdlR2g4WTdFSTdGc2ZzallzM0ZDbkhjMU1vdkZFU3NkcnhiT0F3VlpaaUxL?=
 =?utf-8?B?bGZEZXZ1eTVFTmVvR2xvT2laSWw4Ri8vbmhSQVpMVjdPRTFxWXdVMmMxRDdR?=
 =?utf-8?B?TUp0c2JUUkF4TVUya3dXc2Y5eE83UUc1YlhmQndXMkZydHJqU0ltTTRRUGFC?=
 =?utf-8?Q?GWpdJPSLrfvJa?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 08:35:31.9813
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2fabbb50-767b-4427-7b85-08dc71955214
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D0.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8523

Hi Julien,

On 5/11/2024 4:22 PM, Julien Grall wrote:
> Hi Henry,
>
> On 11/05/2024 08:29, Henry Wang wrote:
>>>> +        /*
>>>> +         * Handle the LR where the physical interrupt is 
>>>> de-assigned from the
>>>> +         * guest before it was EOIed
>>>> +         */
>>>> +        struct vcpu *v_target = vgic_get_target_vcpu(d->vcpu[0], 
>>>> virq);
>>>
>>> This will return a vCPU from the current affinity. This may not be 
>>> where the interrupt was injected. From a brief look, I can't tell 
>>> whether we have an easy way to know where the interrupt was injected 
>>> (other than the pending_irq is in the list lr_queue/inflight)
>>
>> I doubt if we need to handle more than this - I think if the 
>> pending_irq is not in the lr_queue/inflight list, it would not belong 
>> to the corner case we are talking about (?).
>
> I didn't suggest we would need to handle the case where the 
> pending_irq is not any of the queues. I was pointing out that I think 
> we don't directly store the vCPU ID where we injected the IRQ. 
> Instead, the pending_irq is just in list, so we will possibly need to 
> store the vCPU ID for convenience.

Sorry for misunderstanding. Yeah you are definitely correct. Also thank 
you so much for the suggestion! Before seeing this suggestion, I was 
struggling in finding the correct vCPU by "for_each_vcpus" and 
comparison... but now I realized your suggestion is way more clever :)

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Sat May 11 08:46:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 08:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720250.1123336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5iMw-0006Zw-RV; Sat, 11 May 2024 08:46:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720250.1123336; Sat, 11 May 2024 08:46:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5iMw-0006Zp-Oa; Sat, 11 May 2024 08:46:14 +0000
Received: by outflank-mailman (input) for mailman id 720250;
 Sat, 11 May 2024 08:46:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5iMv-0006Zj-Rv
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 08:46:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5iMv-0003lX-0q; Sat, 11 May 2024 08:46:13 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5iMu-0005mi-PM; Sat, 11 May 2024 08:46:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=XORHB6a8m1colt+QcM1Vrsw78pratXVMJJzEBgQqzjk=; b=4cPPsVHZJsyz3Hq62YaF0HfMKr
	mMJ5iMG7P5ZyLSV29jtX2mpkEO+bT1PS81xgnXX8iVlVD+vqMyd5YafFbAGViQvwhLPd9Txbw49qf
	aeZ+K6TLA7X9MGJmDJv0JKA3jWV8Mi0ytljVkjMES9meiUHB/FUzzuxOJzuilXeNP6wE=;
Message-ID: <37ee9f1f-0d62-4d88-9507-2906c3149bba@xen.org>
Date: Sat, 11 May 2024 09:46:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic
 pages for Dom0less DomUs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-2-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240511005611.83125-2-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 11/05/2024 01:56, Henry Wang wrote:
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> The "magic page" is a terminology used in the toolstack as reserved
> pages for the VM to have access to virtual platform capabilities.
> Currently the magic pages for Dom0less DomUs are populated by the
> init-dom0less app through populate_physmap(), and populate_physmap()
> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> cannot be true for the magic pages that are allocated later from the
> init-dom0less application executed in Dom0. For domain using statically
> allocated memory but not 1:1 direct-mapped, similar error "failed to
> retrieve a reserved page" can be seen as the reserved memory list is
> empty at that time.
> 
> To solve above issue, this commit allocates hypervisor reserved pages
> (currently used as the magic pages) for Arm Dom0less DomUs at the
> domain construction time. The base address/PFN of the region will be
> noted and communicated to the init-dom0less application in Dom0.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - Reword the commit msg to explain what is "magic page" and use generic
>    terminology "hypervisor reserved pages" in commit msg. (Daniel)
> - Also move the offset definition of magic pages. (Michal)
> - Extract the magic page allocation logic to a function. (Michal)
> ---
>   tools/libs/guest/xg_dom_arm.c |  6 ------
>   xen/arch/arm/dom0less-build.c | 32 ++++++++++++++++++++++++++++++++
>   xen/include/public/arch-arm.h |  6 ++++++
>   3 files changed, 38 insertions(+), 6 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
> index 2fd8ee7ad4..8c579d7576 100644
> --- a/tools/libs/guest/xg_dom_arm.c
> +++ b/tools/libs/guest/xg_dom_arm.c
> @@ -25,12 +25,6 @@
>   
>   #include "xg_private.h"
>   
> -#define NR_MAGIC_PAGES 4
> -#define CONSOLE_PFN_OFFSET 0
> -#define XENSTORE_PFN_OFFSET 1
> -#define MEMACCESS_PFN_OFFSET 2
> -#define VUART_PFN_OFFSET 3
> -
>   #define LPAE_SHIFT 9
>   
>   #define PFN_4K_SHIFT  (0)
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..4b96ddd9ce 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -739,6 +739,34 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
>       return 0;
>   }
>   
> +static int __init alloc_magic_pages(struct domain *d)
> +{
> +    struct page_info *magic_pg;
> +    mfn_t mfn;
> +    gfn_t gfn;
> +    int rc;
> +
> +    d->max_pages += NR_MAGIC_PAGES;

Here you bump d->max_mages by NR_MAGIC_PAGES but...

> +    magic_pg = alloc_domheap_pages(d, get_order_from_pages(NR_MAGIC_PAGES), 0);

... here you will allocate using a power-of-two. Which may end up to 
fail as there is nothing guaranteeing that NR_MAGIC_PAGES is suitably 
aligned.

For now NR_MAGIC_PAGES seems suitably aligned, so it BUILD_BUG_ON() 
woudl be ok.


> +    if ( magic_pg == NULL )
> +        return -ENOMEM;
> +
> +    mfn = page_to_mfn(magic_pg);
> +    if ( !is_domain_direct_mapped(d) )
> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
> +    else
> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));

Allocating the magic pages contiguously is only necessary for direct 
mapped domain. For the other it might be preferable to allocate page by 
page. That said, NR_MAGIC_PAGES is not big enough. So it would be okay.

> +
> +    rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
> +    if ( rc )
> +    {
> +        free_domheap_pages(magic_pg, get_order_from_pages(NR_MAGIC_PAGES));
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
>   static int __init construct_domU(struct domain *d,
>                                    const struct dt_device_node *node)
>   {
> @@ -840,6 +868,10 @@ static int __init construct_domU(struct domain *d,
>           if ( rc < 0 )
>               return rc;
>           d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
> +
> +        rc = alloc_magic_pages(d);
> +        if ( rc < 0 )
> +            return rc;

This will only be allocated xenstore is enabled. But I don't think some 
of the magic pages really require xenstore to work. In the future we may 
need some more fine graine choice (see my comment in patch #2 as well).

>       }
>   
>       return rc;
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 289af81bd6..186520d01f 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>   
> +#define NR_MAGIC_PAGES 4
This name and the other below are far too generic to be part of the 
shared header. For NR_MAGIC_PAGES, can you explain why GUEST_MAGIC_SIZE 
cannot be used? Is it because it is "too" big?

For the rest below...

> +#define CONSOLE_PFN_OFFSET 0
> +#define XENSTORE_PFN_OFFSET 1
> +#define MEMACCESS_PFN_OFFSET 2
> +#define VUART_PFN_OFFSET 3

... the order we allocate the magic pages is purely an internal decision 
of the toolstack. For the rest of the system, they need to access 
HVM_PARAM_*. So it doesn't feel like they should be part of the shared 
headers.

If there is a strong reason to include them, then I think they all 
should be prefixed with GUEST_MAGIC_*.

Cheers,

> +
>   #define GUEST_RAM_BANKS   2
>   
>   /*

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 09:03:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 09:03:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720254.1123347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5idD-000126-6s; Sat, 11 May 2024 09:03:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720254.1123347; Sat, 11 May 2024 09:03:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5idD-00011z-2r; Sat, 11 May 2024 09:03:03 +0000
Received: by outflank-mailman (input) for mailman id 720254;
 Sat, 11 May 2024 09:03:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0W/u=MO=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s5idB-00011t-5f
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 09:03:01 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 429dd4bc-0f75-11ef-b4bb-af5377834399;
 Sat, 11 May 2024 11:02:59 +0200 (CEST)
Received: from BLAPR03CA0098.namprd03.prod.outlook.com (2603:10b6:208:32a::13)
 by IA0PR12MB8279.namprd12.prod.outlook.com (2603:10b6:208:40c::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54; Sat, 11 May
 2024 09:02:52 +0000
Received: from BL02EPF0001A106.namprd05.prod.outlook.com
 (2603:10b6:208:32a:cafe::f8) by BLAPR03CA0098.outlook.office365.com
 (2603:10b6:208:32a::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.47 via Frontend
 Transport; Sat, 11 May 2024 09:02:52 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL02EPF0001A106.mail.protection.outlook.com (10.167.241.139) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7544.18 via Frontend Transport; Sat, 11 May 2024 09:02:51 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sat, 11 May
 2024 04:02:51 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sat, 11 May 2024 04:02:48 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 429dd4bc-0f75-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wao6OVIL52RH8r4V3L5jZIeYjDMTGLC6H7Eg66E3t2mEMkY0WbmYBHeKS4WeIa7bCd8BPas+8pfoizW1thvHqpwr72RAOfahve/+tWwZwvEJpvp3n4sN9Q1uPvfF13HjYjuja2O95dRZGv8dgQRZ25acv1KQ3PRXZv8DdiQ1uq9mu8t/hWDY8qPs+mgqpNArgYogBxwy3vcDdsmmlEP7ghye2HT/Kls9TUVcHOJzGuSfoNd5usbVhRz7pc7CW+kgYwZ3bQYyyFENZpZtYC8MhPL5npwqZISsCqKHzjG0OmBNgRcRwhFiJIRqSqn/loupv345DMsokypQqkw0M/b3Ag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5nZ4TEgqhjZedag6+bEADIVI/j54bFMNXQKxIOp3gcs=;
 b=lNJwp8SgN/bhYCCdi9wDvT2Yoyuk9DzeP1STEONW5Ous8QzabLABNuOVate6qQvjQmPcZ07BWrz/0qNRUDzYew1I/7jbKdwsDQGD9uMG0ymIG81PYGTY2ozNWERkgB/Y/3i+KGdGoNLt4FaGcfzfjoBRxWnnzk7Z+7ay2UQXizLr1SfcM9LwHkIUfEtMyOPYaqjVJxINzSy6DDbUafBLIasm1oNa6HV0Bby7KVsaoJvg2U+FScRh8avdVEsfsuDzbW1n4VH7MZ2zSPalcsWvS2s2Nv8wsoe9LZkTvbd/XIp4iWSdyuiPaNPaFn8En7cPEvbol/R2NfqK23qyJiYLXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5nZ4TEgqhjZedag6+bEADIVI/j54bFMNXQKxIOp3gcs=;
 b=BLNe0b51ao0YTXs3HzFxmkthQ94Gb0VjokCYKdU0MVtA4SiGTV1LnREMxCjGLeW03E4xTJUMrLFtdF1sRsIEL5csTDuQjzbi2bRhXbA7vmLLVQHwmwVwJTdxbUCFgAi9gYZurf77U9DiJNDB+FD7olH1zHeTIumg+CM9ChpnY6Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <448a31b5-db24-4dbb-a159-624676466536@amd.com>
Date: Sat, 11 May 2024 17:02:48 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic
 pages for Dom0less DomUs
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-2-xin.wang2@amd.com>
 <37ee9f1f-0d62-4d88-9507-2906c3149bba@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <37ee9f1f-0d62-4d88-9507-2906c3149bba@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A106:EE_|IA0PR12MB8279:EE_
X-MS-Office365-Filtering-Correlation-Id: 586c2e65-6e77-4963-6876-08dc71992384
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eVUxbk9Ob2tNU0VBT3BZSVZWTDQ0NEkrelRMVXhSN3Rlc081Qnd0dWxrb01D?=
 =?utf-8?B?RjlpTStLNmM5d2doYk5XbFc4TWZQTXNlVzhueTBHeVkvMFpsTjRVTExva25T?=
 =?utf-8?B?MFd0UlB5eU1sanVnaGc4Tm1MdHprYU1LU095SWhVMEFzVjJXU1ZEUnB4SVBh?=
 =?utf-8?B?V2tDQ1Q4SFVjamxYRVJyTk8xNThLajRxWG5ueW81ZGs0K2MvbUF6T2VXS0pS?=
 =?utf-8?B?MVF5NmxzRlVGYXBVK0N6OHNYUVlYenM5Q21HYU1nMkhlOXY3UmQ5dkhxVDMx?=
 =?utf-8?B?ZUZRWEN3OWFKSm1rbHNUQ0pEbnEwSWZtblZBSitxd1dCYUNnbkFTRnIycUlT?=
 =?utf-8?B?eDhqT3pvVHBBQ2kycjh0ZWRBeWRrdm1pU3VPcXFBejZvTlI1N3IxRkhMNDM0?=
 =?utf-8?B?M0JGblFWak1lY1dDaEFORE9zY1pNRlgvK0x3akJneWpPdFpqK2lWaElLdVAz?=
 =?utf-8?B?ai96UEM3SENNbWNFSU5kdk5qQllRQXc5bElrRzRjWExWUEI5cU00ckltUWh0?=
 =?utf-8?B?OHFvLzlGc29EbkltbXFmNE1vdnY3bEJ5QnRNVjZZNkJYakoySU9SdXRBYnh0?=
 =?utf-8?B?RkZIbHFxQXNMOXprNUZTZXdmZXBNVFc0MFd1N3l0M3dEUGozUVZBYnhKN3Rt?=
 =?utf-8?B?ZkRDOU5MMHU1ZjFldE1VcDY4Z0JJRjdRbExoSzF1eTI4cUdxVlNhNTJ5NmdS?=
 =?utf-8?B?VWhlZS9YaFNFN1R6OXBFMEdjOFJndjcwaFVwdHAreEZtc1hvRjFpZUlCSUJD?=
 =?utf-8?B?c0VnbHFzTmtydDlraVJMUWcwbUxnRkhBM29mQ1IxZEd3eU9tVHpqOGwvenJO?=
 =?utf-8?B?ejVKTnhUdkYwU2hFZysrUmF5VWNWR2tpRVRqS0lNd1d4bnQvOTRDbDJyTFU1?=
 =?utf-8?B?Ly9MbUc2UWpGZWJ0cUIwYkxlc3MvaDMyWVo5TzIvUmViRm8yVzBKNWptNnd5?=
 =?utf-8?B?SVlXMlNsN0pBcWZJOSt3YndzWmxUb1ppbGlZY2Rwc1FyMVBtUEVxY3R0a1hm?=
 =?utf-8?B?dEpRaGwxRHpxSFQyZENwbkxJcytnNjhraDNQcTYzR2kzU045dDYrcFdiZ2dE?=
 =?utf-8?B?VFB0QUY5N2hFYWp6Zk1OWStxV0pqMGxXcU1MUk50OXRMYm9mWndtU2JmUThH?=
 =?utf-8?B?aE82RW5UOGFINlhkM05QUTNzcWdMRlpMbGptWE9PWHBmTnlybEFtdkN3RGlF?=
 =?utf-8?B?ekp2Q2dqeXErZFRraXRJMU9rS2FKVktHcGtwNXd6MG9ZMWsxZVdxcE9wYVAy?=
 =?utf-8?B?TXJzSFNXODFtVHNsaFl3d04yZk55U2VWa2s5Mk9aT3JtbXpqRmpodU5mMzN3?=
 =?utf-8?B?RTFGazNWbU9mWnBpY3REWkZhUVBCdXJnSWZHUTBlM083Uk11djBpVXVRQnBY?=
 =?utf-8?B?ZUp2aXBYcWEyMDYweTdRTjhiMS9nMW1jeml0L3l4Q2RDRHBuemFISWRiSFhp?=
 =?utf-8?B?elQ5b1NZdCtSK0Jmd2kvVW5MbEpaSC9mdWdvOVp1ZUxlNVVuRHU2OWVyZkVF?=
 =?utf-8?B?bnhwMHhRNGZuS2NORDgvV3duWFE0TVFyOFNMQlo2UUdLbGNwQU14ampJVnF4?=
 =?utf-8?B?NGFsamd4eU9UTVU3cjFIckhWa1NiNkJqVGxQOXJZbjZ0Z0FnL1dtb1pFNVNU?=
 =?utf-8?B?ZU1iTnF2STJsZVp0WDF4cFZwSkJ6Y3p4ekREUWR5bks3OTJXSitKeTIwb3Ez?=
 =?utf-8?B?azMvUkw1Qi9McUpMZ1RweUJGcjZpTkZKbjU5ZHJ3TDE3RVJmMS9NV3Rva0I4?=
 =?utf-8?Q?4Awl4dpJXTdJck8rX0D/j/fATiwKLMtrv0xcEuV?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2024 09:02:51.9114
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 586c2e65-6e77-4963-6876-08dc71992384
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A106.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8279

Hi Julien,

On 5/11/2024 4:46 PM, Julien Grall wrote:
> Hi Henry,
>
> On 11/05/2024 01:56, Henry Wang wrote:
>> There are use cases (for example using the PV driver) in Dom0less
>> setup that require Dom0less DomUs start immediately with Dom0, but
>> initialize XenStore later after Dom0's successful boot and call to
>> the init-dom0less application.
>>
>> An error message can seen from the init-dom0less application on
>> 1:1 direct-mapped domains:
>> ```
>> Allocating magic pages
>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>> Error on alloc magic pages
>> ```
>>
>> The "magic page" is a terminology used in the toolstack as reserved
>> pages for the VM to have access to virtual platform capabilities.
>> Currently the magic pages for Dom0less DomUs are populated by the
>> init-dom0less app through populate_physmap(), and populate_physmap()
>> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
>> cannot be true for the magic pages that are allocated later from the
>> init-dom0less application executed in Dom0. For domain using statically
>> allocated memory but not 1:1 direct-mapped, similar error "failed to
>> retrieve a reserved page" can be seen as the reserved memory list is
>> empty at that time.
>>
>> To solve above issue, this commit allocates hypervisor reserved pages
>> (currently used as the magic pages) for Arm Dom0less DomUs at the
>> domain construction time. The base address/PFN of the region will be
>> noted and communicated to the init-dom0less application in Dom0.
>>
>> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
>> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>> v2:
>> - Reword the commit msg to explain what is "magic page" and use generic
>>    terminology "hypervisor reserved pages" in commit msg. (Daniel)
>> - Also move the offset definition of magic pages. (Michal)
>> - Extract the magic page allocation logic to a function. (Michal)
>> ---
>>   tools/libs/guest/xg_dom_arm.c |  6 ------
>>   xen/arch/arm/dom0less-build.c | 32 ++++++++++++++++++++++++++++++++
>>   xen/include/public/arch-arm.h |  6 ++++++
>>   3 files changed, 38 insertions(+), 6 deletions(-)
>>
>> diff --git a/tools/libs/guest/xg_dom_arm.c 
>> b/tools/libs/guest/xg_dom_arm.c
>> index 2fd8ee7ad4..8c579d7576 100644
>> --- a/tools/libs/guest/xg_dom_arm.c
>> +++ b/tools/libs/guest/xg_dom_arm.c
>> @@ -25,12 +25,6 @@
>>     #include "xg_private.h"
>>   -#define NR_MAGIC_PAGES 4
>> -#define CONSOLE_PFN_OFFSET 0
>> -#define XENSTORE_PFN_OFFSET 1
>> -#define MEMACCESS_PFN_OFFSET 2
>> -#define VUART_PFN_OFFSET 3
>> -
>>   #define LPAE_SHIFT 9
>>     #define PFN_4K_SHIFT  (0)
>> diff --git a/xen/arch/arm/dom0less-build.c 
>> b/xen/arch/arm/dom0less-build.c
>> index 74f053c242..4b96ddd9ce 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -739,6 +739,34 @@ static int __init alloc_xenstore_evtchn(struct 
>> domain *d)
>>       return 0;
>>   }
>>   +static int __init alloc_magic_pages(struct domain *d)
>> +{
>> +    struct page_info *magic_pg;
>> +    mfn_t mfn;
>> +    gfn_t gfn;
>> +    int rc;
>> +
>> +    d->max_pages += NR_MAGIC_PAGES;
>
> Here you bump d->max_mages by NR_MAGIC_PAGES but...
>
>> +    magic_pg = alloc_domheap_pages(d, 
>> get_order_from_pages(NR_MAGIC_PAGES), 0);
>
> ... here you will allocate using a power-of-two. Which may end up to 
> fail as there is nothing guaranteeing that NR_MAGIC_PAGES is suitably 
> aligned.
>
> For now NR_MAGIC_PAGES seems suitably aligned, so it BUILD_BUG_ON() 
> woudl be ok.

Great catch! I will add BUILD_BUG_ON(NR_MAGIC_PAGES & (NR_MAGIC_PAGES - 1));
Thanks.

>> +    if ( magic_pg == NULL )
>> +        return -ENOMEM;
>> +
>> +    mfn = page_to_mfn(magic_pg);
>> +    if ( !is_domain_direct_mapped(d) )
>> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
>> +    else
>> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
>
> Allocating the magic pages contiguously is only necessary for direct 
> mapped domain. For the other it might be preferable to allocate page 
> by page. That said, NR_MAGIC_PAGES is not big enough. So it would be 
> okay.
>
>> +
>> +    rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
>> +    if ( rc )
>> +    {
>> +        free_domheap_pages(magic_pg, 
>> get_order_from_pages(NR_MAGIC_PAGES));
>> +        return rc;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>>   static int __init construct_domU(struct domain *d,
>>                                    const struct dt_device_node *node)
>>   {
>> @@ -840,6 +868,10 @@ static int __init construct_domU(struct domain *d,
>>           if ( rc < 0 )
>>               return rc;
>>           d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
>> +
>> +        rc = alloc_magic_pages(d);
>> +        if ( rc < 0 )
>> +            return rc;
>
> This will only be allocated xenstore is enabled. But I don't think 
> some of the magic pages really require xenstore to work. In the future 
> we may need some more fine graine choice (see my comment in patch #2 
> as well).

Sorry, but it seems that by the time that I am writing this reply, I 
didn't get the email for patch #2 comment. I will reply both together 
when I see it.

>>       }
>>         return rc;
>> diff --git a/xen/include/public/arch-arm.h 
>> b/xen/include/public/arch-arm.h
>> index 289af81bd6..186520d01f 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>>   +#define NR_MAGIC_PAGES 4
> This name and the other below are far too generic to be part of the 
> shared header. For NR_MAGIC_PAGES, can you explain why 
> GUEST_MAGIC_SIZE cannot be used? Is it because it is "too" big?

Yes, I don't really want to allocate whole 16MB when we merely need 4 pages.

> For the rest below...
>
>> +#define CONSOLE_PFN_OFFSET 0
>> +#define XENSTORE_PFN_OFFSET 1
>> +#define MEMACCESS_PFN_OFFSET 2
>> +#define VUART_PFN_OFFSET 3
>
> ... the order we allocate the magic pages is purely an internal 
> decision of the toolstack. For the rest of the system, they need to 
> access HVM_PARAM_*. So it doesn't feel like they should be part of the 
> shared headers.
>
> If there is a strong reason to include them, then I think they all 
> should be prefixed with GUEST_MAGIC_*.

One of the benefits as Michal pointed out in comments for v1 [1] would 
be this would also allow init-dom0less.h not to re-define 
XENSTORE_PFN_OFFSET. I will rename them as suggested if both of you are 
ok with moving them to the arch header.

[1] 
https://lore.kernel.org/xen-devel/5d3ead96-5079-4fa3-b5fd-4d9803c251b6@amd.com/

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Sat May 11 09:17:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 09:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720260.1123356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5irC-0003S5-AZ; Sat, 11 May 2024 09:17:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720260.1123356; Sat, 11 May 2024 09:17:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5irC-0003Ry-7m; Sat, 11 May 2024 09:17:30 +0000
Received: by outflank-mailman (input) for mailman id 720260;
 Sat, 11 May 2024 09:17:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5irA-0003Rs-TG
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 09:17:28 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5irA-0004Zh-Bi; Sat, 11 May 2024 09:17:28 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5irA-0007KW-3z; Sat, 11 May 2024 09:17:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=hCwP4b4DAVi72fRcSDnZAzkDAx2URX7FOKYLm+be2gg=; b=lExUW1Taky0DUW8YR6Z4Wakh3w
	84CdvvWnDaBIB8YzjqtxkWZSNJrfP7buJ6LvkdFtmR2g4lFPSCpu37BIXmWHOW2GjT78y0gyFmJ/A
	p3yuMv9PBVEwCaPgHd0UiYMjlRnSPdmbdW77fxuhMJkRczLiXeqg1lsgxKurXAMVe/hE=;
Message-ID: <e84a2ac3-f09b-40a0-848b-4d2932a61fe7@xen.org>
Date: Sat, 11 May 2024 10:17:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] xen/arm: Add new HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE}
 keys in HVMOP
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-3-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240511005611.83125-3-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 11/05/2024 01:56, Henry Wang wrote:
> For use cases such as Dom0less PV drivers, a mechanism to communicate
> Dom0less DomU's static data with the runtime control plane (Dom0) is
> needed. Since on Arm HVMOP is already the existing approach to address
> such use cases (for example the allocation of HVM_PARAM_CALLBACK_IRQ),
> add new HVMOP keys HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} for storing the
> hypervisor reserved pages region base PFN and size.
> 
> Currently, the hypervisor reserved pages region is used as the Arm
> Dom0less DomU guest magic pages region. Therefore protect the HVMOP
> keys with "#if defined(__arm__) || defined(__aarch64__)". The values
> will be set at Dom0less DomU construction time after Dom0less DomU's
> magic pages region has been allocated.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - Rename the HVMOP keys to HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE}. (Daniel)
> - Add comment on top of HVM_PARAM_HV_RSRV_{BASE_PFN,SIZE} to describe
>    its usage. Protect them with #ifdef. (Daniel, Jan)
> ---
>   xen/arch/arm/dom0less-build.c   |  3 +++
>   xen/arch/arm/hvm.c              |  2 ++
>   xen/include/public/hvm/params.h | 11 ++++++++++-
>   3 files changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 4b96ddd9ce..5bb53ebb47 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -764,6 +764,9 @@ static int __init alloc_magic_pages(struct domain *d)
>           return rc;
>       }
>   
> +    d->arch.hvm.params[HVM_PARAM_HV_RSRV_BASE_PFN] = gfn_x(gfn);
> +    d->arch.hvm.params[HVM_PARAM_HV_RSRV_SIZE] = NR_MAGIC_PAGES;
> +
>       return 0;
>   }
>   
> diff --git a/xen/arch/arm/hvm.c b/xen/arch/arm/hvm.c
> index 0989309fea..949d804f8b 100644
> --- a/xen/arch/arm/hvm.c
> +++ b/xen/arch/arm/hvm.c
> @@ -55,6 +55,8 @@ static int hvm_allow_get_param(const struct domain *d, unsigned int param)
>       case HVM_PARAM_STORE_EVTCHN:
>       case HVM_PARAM_CONSOLE_PFN:
>       case HVM_PARAM_CONSOLE_EVTCHN:
> +    case HVM_PARAM_HV_RSRV_BASE_PFN:
> +    case HVM_PARAM_HV_RSRV_SIZE:

We should not allow the guest to read HVM_PARAM_HV_*. So this would need 
to be handled like HVM_PARAM_HV_*.

But I have some reservation with using HVM params. It means we are 
setting in stone how we communicate/allocate the reserved pages.

I can see a few issues that may bite back in the future.

We always pre-allocate the reserved pages. But most of the feature may 
not be used which ends up to be a waste of memory. For instance, you 
only seem to one page for dom0less, but allocate 4. I understand that 
libxenguest is doing the same today, however we have the flexibility to 
change it by just not allocating the page.

Similarly, we expect the hypervisor to provide enough reserved pages for 
the toolstack to work. Yet, I believe some of the features (e.g. 
memaccess) doesn't need to be known in advance.

The number of pages are tiny at the moment (4) on Xen. But given the 
name, I could see someone asking for setting aside more reserved spaces 
and a single range may start to be a problem for direct mapped domain 
(unless we don't need to map the magic pages 1:1).

Overall, if we want to make it part of the stable ABI. Then it would be 
better to provide a (large) reserved space that will be allocated on 
demand (IOW there is no associated physical memory). If it doesn't work 
for direct mapped domain, then another solution is to have a toolstack 
hypercall to allocate a GFN (they don't need to be contiguous) and the 
backing memory.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 09:33:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 09:33:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720267.1123367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5j6i-0006OG-OW; Sat, 11 May 2024 09:33:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720267.1123367; Sat, 11 May 2024 09:33:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5j6i-0006O9-LX; Sat, 11 May 2024 09:33:32 +0000
Received: by outflank-mailman (input) for mailman id 720267;
 Sat, 11 May 2024 09:33:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5j6h-0006O3-Gj
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 09:33:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5j6g-0004vM-TR; Sat, 11 May 2024 09:33:30 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5j6g-00086J-L3; Sat, 11 May 2024 09:33:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6hULbS/6eE4qhzO2a1XGxmjFpDfPiPHz9CqnqPs+1Lw=; b=mG9hbwje9rxKrj65nNUXBMeIQR
	nwNcJfYQQa+yyztjubkTHvH2yF0Wk7g0NBK4lJakJyZ8obIHaEhgAAYFllLJFDEaPXaCqyPkKi8nk
	SoKKk+m58Ed2+IpnJ0rZlk08vK8O1JAIn1/s0IZWsMAfSSooBm/siSmPmqn3O4VNoqXU=;
Message-ID: <b3a62eb5-d39c-4799-a3c7-707991fc8200@xen.org>
Date: Sat, 11 May 2024 10:33:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic
 pages for Dom0less DomUs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-2-xin.wang2@amd.com>
 <37ee9f1f-0d62-4d88-9507-2906c3149bba@xen.org>
 <448a31b5-db24-4dbb-a159-624676466536@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <448a31b5-db24-4dbb-a159-624676466536@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 11/05/2024 10:02, Henry Wang wrote:
>>> +
>>> +    rc = guest_physmap_add_pages(d, gfn, mfn, NR_MAGIC_PAGES);
>>> +    if ( rc )
>>> +    {
>>> +        free_domheap_pages(magic_pg, 
>>> get_order_from_pages(NR_MAGIC_PAGES));
>>> +        return rc;
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>   static int __init construct_domU(struct domain *d,
>>>                                    const struct dt_device_node *node)
>>>   {
>>> @@ -840,6 +868,10 @@ static int __init construct_domU(struct domain *d,
>>>           if ( rc < 0 )
>>>               return rc;
>>>           d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
>>> +
>>> +        rc = alloc_magic_pages(d);
>>> +        if ( rc < 0 )
>>> +            return rc;
>>
>> This will only be allocated xenstore is enabled. But I don't think 
>> some of the magic pages really require xenstore to work. In the future 
>> we may need some more fine graine choice (see my comment in patch #2 
>> as well).
> 
> Sorry, but it seems that by the time that I am writing this reply, I 
> didn't get the email for patch #2 comment. I will reply both together 
> when I see it.

That was expected, I knew what I wanted to mention about patch #2 but 
the e-mail was not fully written. You should have it in your inbox now.

> 
>>>       }
>>>         return rc;
>>> diff --git a/xen/include/public/arch-arm.h 
>>> b/xen/include/public/arch-arm.h
>>> index 289af81bd6..186520d01f 100644
>>> --- a/xen/include/public/arch-arm.h
>>> +++ b/xen/include/public/arch-arm.h
>>> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>>>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>>>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>>>   +#define NR_MAGIC_PAGES 4
>> This name and the other below are far too generic to be part of the 
>> shared header. For NR_MAGIC_PAGES, can you explain why 
>> GUEST_MAGIC_SIZE cannot be used? Is it because it is "too" big?
> 
> Yes, I don't really want to allocate whole 16MB when we merely need 4 
> pages.
What about updating GUEST_MAGIC_SIZE to 0x4000? It doesn't make any 
sense to have two define which have pretty much the same meaning.

Then on the toolstack side, you can check that NR_MAGIC_PAGES is smaller 
or equal to GUEST_MAGIC_SIZE.

> 
>> For the rest below...
>>
>>> +#define CONSOLE_PFN_OFFSET 0
>>> +#define XENSTORE_PFN_OFFSET 1
>>> +#define MEMACCESS_PFN_OFFSET 2
>>> +#define VUART_PFN_OFFSET 3
>>
>> ... the order we allocate the magic pages is purely an internal 
>> decision of the toolstack. For the rest of the system, they need to 
>> access HVM_PARAM_*. So it doesn't feel like they should be part of the 
>> shared headers.
>>
>> If there is a strong reason to include them, then I think they all 
>> should be prefixed with GUEST_MAGIC_*.
> 
> One of the benefits as Michal pointed out in comments for v1 [1] would 
> be this would also allow init-dom0less.h not to re-define 
> XENSTORE_PFN_OFFSET.

At the moment, yes they have the same values. But with series, 
XENSTORE_PFN_OFFSET should technically be 0 in init-dom0less.

This is because Xen may only allocate one page (you don't need 4) for 
the reserved area. So...

> I will rename them as suggested if both of you are 
> ok with moving them to the arch header.


... I don't think they should be shared.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 09:46:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 09:46:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720272.1123377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5jJL-0000Q0-Rq; Sat, 11 May 2024 09:46:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720272.1123377; Sat, 11 May 2024 09:46:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5jJL-0000Pt-PA; Sat, 11 May 2024 09:46:35 +0000
Received: by outflank-mailman (input) for mailman id 720272;
 Sat, 11 May 2024 09:46:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5jJK-0000Pn-9F
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 09:46:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5jJI-0005Ft-Dq; Sat, 11 May 2024 09:46:32 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5jJI-0000M1-8D; Sat, 11 May 2024 09:46:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=og76OEDatqruWuco+k6x3qQ+7PlUJSmh337By1tf1J4=; b=x1yfeCO31fdHDr1rLG4xBTK4yR
	tCiyLGRZ9skv1XyMmCcQBQaJavg/9pVFdrxhA0P/TEshcCHpCIYYZkoSfr7Sx7zytHK614D4zI+7U
	YZResUk4Y9n5FNxDmxZEiur2F0hFmOijW+QqsNYHnbPRoHhApKAF4UvqVajT1QbzUdtA=;
Message-ID: <d33ea00d-890d-45cc-9583-64c953abd70f@xen.org>
Date: Sat, 11 May 2024 10:46:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] tools/init-dom0less: Avoid hardcoding
 GUEST_MAGIC_BASE
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>,
 Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-4-xin.wang2@amd.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240511005611.83125-4-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 11/05/2024 01:56, Henry Wang wrote:
> Currently the GUEST_MAGIC_BASE in the init-dom0less application is
> hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
> DomUs.
> 
> Since the guest magic region is now allocated from the hypervisor,
> instead of hardcoding the guest magic pages region, use
> xc_hvm_param_get() to get the guest magic region PFN, and based on
> that the XenStore PFN can be calculated. Also, we don't need to set
> the max mem anymore, so drop the call to xc_domain_setmaxmem(). Rename
> the alloc_xs_page() to get_xs_page() to reflect the changes.
> 
> Take the opportunity to do some coding style improvements when possible.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - Update HVMOP keys name.
> ---
>   tools/helpers/init-dom0less.c | 40 +++++++++++++++--------------------
>   1 file changed, 17 insertions(+), 23 deletions(-)
> 
> diff --git a/tools/helpers/init-dom0less.c b/tools/helpers/init-dom0less.c
> index fee93459c4..04039a2a66 100644
> --- a/tools/helpers/init-dom0less.c
> +++ b/tools/helpers/init-dom0less.c
> @@ -19,24 +19,20 @@
>   #define XENSTORE_PFN_OFFSET 1

Based on previous discussion, I think this wants to be 0 so the 
hypervisor only need to allocate one page (rather than 2 with the 
current offset).

>   #define STR_MAX_LENGTH 128
>   
> -static int alloc_xs_page(struct xc_interface_core *xch,
> -                         libxl_dominfo *info,
> -                         uint64_t *xenstore_pfn)
> +static int get_xs_page(struct xc_interface_core *xch, libxl_dominfo *info,
> +                       uint64_t *xenstore_pfn)
>   {
>       int rc;
> -    const xen_pfn_t base = GUEST_MAGIC_BASE >> XC_PAGE_SHIFT;
> -    xen_pfn_t p2m = (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET;
> +    xen_pfn_t magic_base_pfn;
>   
> -    rc = xc_domain_setmaxmem(xch, info->domid,
> -                             info->max_memkb + (XC_PAGE_SIZE/1024));
> -    if (rc < 0)
> -        return rc;
> -
> -    rc = xc_domain_populate_physmap_exact(xch, info->domid, 1, 0, 0, &p2m);
> -    if (rc < 0)
> -        return rc;
> +    rc = xc_hvm_param_get(xch, info->domid, HVM_PARAM_HV_RSRV_BASE_PFN,
> +                          &magic_base_pfn);
> +    if (rc < 0) {
> +        printf("Failed to get HVM_PARAM_HV_RSRV_BASE_PFN\n");
> +        return 1;
> +    }
>   

If we go down the route of using HVM_PARAM... then we should avoid to 
rely on the hypervisor to provide a large enough region. So we should 
check the offset against the size.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 10:10:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 10:10:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720278.1123387 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5jgk-0004pp-JL; Sat, 11 May 2024 10:10:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720278.1123387; Sat, 11 May 2024 10:10:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5jgk-0004pi-Gk; Sat, 11 May 2024 10:10:46 +0000
Received: by outflank-mailman (input) for mailman id 720278;
 Sat, 11 May 2024 10:10:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5jgj-0004pY-7p; Sat, 11 May 2024 10:10:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5jgj-0005u3-5w; Sat, 11 May 2024 10:10:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5jgi-0008U4-Of; Sat, 11 May 2024 10:10:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5jgi-00078r-OC; Sat, 11 May 2024 10:10:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=RVHS5+wFSk4lY/c4QOS1dJ13w/bOdXekiZ5lrE8CSWY=; b=5TjeyeyG07pAFIFvKRVw66P6kr
	RazRlnf5yq6Dk4TevCgbCYhzt/GiOVOOKZwrSUerfF9dFQqlEOnwaucNcivAWn9XbsfkQWKyRb51A
	3aLkIH1YGioaQu9ITrmSHqf3hqJmYYyA0osg9I6D4kr/IhoxQaWrdb7bhifwTaL9C3Pc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185975-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185975: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qcow2:guest-start.2:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=996576b965ccdf0de17aafa14282925e408e1200
X-Osstest-Versions-That:
    xen=b0082b908391b29b7c4dd5e6c389ebd6481926f8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 10:10:44 +0000

flight 185975 xen-unstable real [real]
flight 185979 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185975/
http://logs.test-lab.xenproject.org/osstest/logs/185979/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qcow2    22 guest-start.2       fail pass in 185979-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185979-retest
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 185979-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 185979 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 185979 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185970
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185970
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185970
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185970
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185970
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185970
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  996576b965ccdf0de17aafa14282925e408e1200
baseline version:
 xen                  b0082b908391b29b7c4dd5e6c389ebd6481926f8

Last test of basis   185970  2024-05-10 14:25:40 Z    0 days
Testing same since   185975  2024-05-11 01:37:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    fail    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b0082b9083..996576b965  996576b965ccdf0de17aafa14282925e408e1200 -> master


From xen-devel-bounces@lists.xenproject.org Sat May 11 11:03:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 11:03:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720294.1123401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5kVV-0003ic-Df; Sat, 11 May 2024 11:03:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720294.1123401; Sat, 11 May 2024 11:03:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5kVV-0003iV-Ax; Sat, 11 May 2024 11:03:13 +0000
Received: by outflank-mailman (input) for mailman id 720294;
 Sat, 11 May 2024 11:03:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s5kVU-0003iP-M2
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 11:03:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5kVU-0006lh-5y; Sat, 11 May 2024 11:03:12 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s5kVT-0005G7-SP; Sat, 11 May 2024 11:03:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=IfBIzTlB8gbtJsjuqohges46El9qHcXdYkkiuX1+SAQ=; b=rPYELTDoM39nOWQP+IeMdB12Mh
	Z7/uaGKjwVY2brh8JJTh/BbYalWV1htbJwvyqF0srGBBs9ycE4/ZKrz6XbdnF/9r/54QM4+Rbv7c5
	eSI5eCBLJ5awYREegm8WX1VXUueBLiB+QlM/iqQ63ABZAEYs4tyf8ZLXUWR14yzaGijE=;
Message-ID: <686ba256-f8bf-47e7-872f-d277bf7df0aa@xen.org>
Date: Sat, 11 May 2024 12:03:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic
 pages for Dom0less DomUs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-2-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240511005611.83125-2-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 11/05/2024 01:56, Henry Wang wrote:
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> The "magic page" is a terminology used in the toolstack as reserved
> pages for the VM to have access to virtual platform capabilities.
> Currently the magic pages for Dom0less DomUs are populated by the
> init-dom0less app through populate_physmap(), and populate_physmap()
> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> cannot be true for the magic pages that are allocated later from the
> init-dom0less application executed in Dom0. For domain using statically
> allocated memory but not 1:1 direct-mapped, similar error "failed to
> retrieve a reserved page" can be seen as the reserved memory list is
> empty at that time.
> 
> To solve above issue, this commit allocates hypervisor reserved pages
> (currently used as the magic pages) for Arm Dom0less DomUs at the
> domain construction time. The base address/PFN of the region will be
> noted and communicated to the init-dom0less application in Dom0.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - Reword the commit msg to explain what is "magic page" and use generic
>    terminology "hypervisor reserved pages" in commit msg. (Daniel)
> - Also move the offset definition of magic pages. (Michal)
> - Extract the magic page allocation logic to a function. (Michal)
> ---
>   tools/libs/guest/xg_dom_arm.c |  6 ------
>   xen/arch/arm/dom0less-build.c | 32 ++++++++++++++++++++++++++++++++
>   xen/include/public/arch-arm.h |  6 ++++++
>   3 files changed, 38 insertions(+), 6 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
> index 2fd8ee7ad4..8c579d7576 100644
> --- a/tools/libs/guest/xg_dom_arm.c
> +++ b/tools/libs/guest/xg_dom_arm.c
> @@ -25,12 +25,6 @@
>   
>   #include "xg_private.h"
>   
> -#define NR_MAGIC_PAGES 4
> -#define CONSOLE_PFN_OFFSET 0
> -#define XENSTORE_PFN_OFFSET 1
> -#define MEMACCESS_PFN_OFFSET 2
> -#define VUART_PFN_OFFSET 3
> -
>   #define LPAE_SHIFT 9
>   
>   #define PFN_4K_SHIFT  (0)
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..4b96ddd9ce 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -739,6 +739,34 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
>       return 0;
>   }
>   
> +static int __init alloc_magic_pages(struct domain *d)
> +{
> +    struct page_info *magic_pg;
> +    mfn_t mfn;
> +    gfn_t gfn;
> +    int rc;
> +
> +    d->max_pages += NR_MAGIC_PAGES;
> +    magic_pg = alloc_domheap_pages(d, get_order_from_pages(NR_MAGIC_PAGES), 0);
> +    if ( magic_pg == NULL )
> +        return -ENOMEM;
> +
> +    mfn = page_to_mfn(magic_pg);
> +    if ( !is_domain_direct_mapped(d) )
> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
> +    else
> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));

Summarizing the discussion we had on Matrix. Regions like the extend 
area and shared memory may not be direct mapped. So unfortunately, I 
think it is possible that the GFN could clash with one of those.

At least in the shared memory case, the user can provide the address. 
But as you use the domheap allocator, the address returned could easily 
change if you tweak your setup.

I am not entirely sure what's the best solution. We could ask the user 
to provide the information for reserved region. But it feels like we are 
exposing a bit too much to the user.

So possibly we would want to use the same approach as extended regions. 
Once we processed all the mappings, find some space for the hypervisor 
regions.

Any other suggestions?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat May 11 13:52:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 13:52:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720322.1123412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5n9C-0008Lq-Fq; Sat, 11 May 2024 13:52:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720322.1123412; Sat, 11 May 2024 13:52:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5n9C-0008Lj-Ap; Sat, 11 May 2024 13:52:22 +0000
Received: by outflank-mailman (input) for mailman id 720322;
 Sat, 11 May 2024 13:52:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5n9B-0008LZ-8n; Sat, 11 May 2024 13:52:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5n9B-0001La-6c; Sat, 11 May 2024 13:52:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5n9A-0008B4-Ua; Sat, 11 May 2024 13:52:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5n9A-0004hU-U3; Sat, 11 May 2024 13:52:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cj33h/br3WZImBMrWYmYaoSP/L8W52HWOVVztleTWXs=; b=pvOSCKtistG+fGcwpIqgBfbhEc
	BV10QMlvuQz94seKAcm9OIy9aceCELzCQMB/l5lNii2s+RzxpO9BlZIW43aI4Hev/iSG7avn/Qz/B
	8noUzDXeoPFV4rf3sBiKwmbfdDegAXbRQKIzspnxvdDQjDqbrgd5cY5YVLkbAMBp/+94=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185978-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185978: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=d4528bb9dbf21464e68beb9175a38aaf6484536e
X-Osstest-Versions-That:
    libvirt=dda10ac8acd0b5ffa03f9659c5678c2bddd9eed4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 13:52:20 +0000

flight 185978 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185978/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185968
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              d4528bb9dbf21464e68beb9175a38aaf6484536e
baseline version:
 libvirt              dda10ac8acd0b5ffa03f9659c5678c2bddd9eed4

Last test of basis   185968  2024-05-10 04:18:49 Z    1 days
Testing same since   185978  2024-05-11 04:20:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andi Chandler <andi@gowling.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   dda10ac8ac..d4528bb9db  d4528bb9dbf21464e68beb9175a38aaf6484536e -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat May 11 15:17:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 15:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720360.1123421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5oSv-0002FZ-Bt; Sat, 11 May 2024 15:16:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720360.1123421; Sat, 11 May 2024 15:16:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5oSv-0002FS-8P; Sat, 11 May 2024 15:16:49 +0000
Received: by outflank-mailman (input) for mailman id 720360;
 Sat, 11 May 2024 15:16:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HmPl=MO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5oSt-0002FM-TC
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 15:16:48 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a67c631-0fa9-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 17:16:45 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so743386166b.3
 for <xen-devel@lists.xenproject.org>; Sat, 11 May 2024 08:16:45 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17d02sm339118966b.218.2024.05.11.08.16.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 11 May 2024 08:16:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a67c631-0fa9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715440604; x=1716045404; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=6+FCfeWQ07lM7xrLNFgOCKq9dA+XCwsh9U//qQFZcMQ=;
        b=mcuNhmSQUooHdiSCRTh9Ja+AqviraRnw9chP3/+z/3AWrq0bNv+3SlYoM1mlPiYZDk
         8V3uwMqPa+o5f6SW6y5X3ckVcJoPZqilpFpAADf2fJ5jHKj2dCvz9VEHnWeEi4HMn22k
         6Yd5JVC53lM7xLpVCKbe6/ghk/SbFxsCOsprs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715440604; x=1716045404;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=6+FCfeWQ07lM7xrLNFgOCKq9dA+XCwsh9U//qQFZcMQ=;
        b=YDsfprfLav55VBCmjrR9gqRfep/j5riTQR5ce4s5mmJPxrH7MDGdybxqo0Vbb5DsXZ
         fzY3dYMUKhwl/VojN8Ebu4LyYku/rAqXeu427V3CeJwPe9pbt8Kjn9nRw53AriluL3xp
         R54dV/gZkOH66gSX3Ksyr2Aog5Zy4o6PziTuhhsEmixYdhWy2vbQIxCJ+RnaRqKaYM0P
         3sfDfghaq3wV3fDZ2UxuIVWgK6H71IpCEEIgfH5i8+EG6tfzxoDBp0HB7wMHEjrrUsdt
         bV79A3DIJorRWl6aGVhKh7Q//j7tWi41B52CTZ7N/5zv/PE90+x8EOqFrtkJ9QMh+rr3
         mggw==
X-Gm-Message-State: AOJu0YwddhQVYHTJeC669LAgcDOZ5jUiuyI31PaENRUkwjgTkDi3nTqL
	l51MLJM7UTbvVi77glo1dTeqHN923Y4pVyeYS4YcWgRiMssFGHTMPXnBOpMNDn4W4Cex+9gF6SD
	P
X-Google-Smtp-Source: AGHT+IGx9DnLgKwuPXmVFOSL0hPxvOCxs/s47ciBuRQIn3mVlhytg8mLXWwUuTJfe+wIgrdwq73x4A==
X-Received: by 2002:a17:906:c285:b0:a59:cd18:92fc with SMTP id a640c23a62f3a-a5a2d672f17mr401276866b.62.1715440604184;
        Sat, 11 May 2024 08:16:44 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] x86/pv: Sanity check bytes_per_rep in rep_outs()
Date: Sat, 11 May 2024 16:16:42 +0100
Message-Id: <20240511151642.2476555-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

I was playing with GCC-14, and gave the new static analyser (-fanalyze) a
try.  One of the more serious things it came back with was this:

  In file included from ./arch/x86/include/asm/mm.h:10,
                   from ./include/xen/mm.h:233,
                   from ./include/xen/domain_page.h:12,
                   from arch/x86/pv/emul-priv-op.c:10:
  In function '__copy_from_guest_pv',
      inlined from 'rep_outs' at arch/x86/pv/emul-priv-op.c:718:20:
  ./arch/x86/include/asm/uaccess.h:174:9: warning: stack-based buffer overflow [CWE-121] [-Wanalyzer-out-of-bounds]
    174 |         __asm__ __volatile__(                                           \
        |         ^~~~~~~
  ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
    229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
        |             ^~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
    236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
        |     ^~~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
    308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
        |             ^~~~~~~~~~~~~~
    'rep_outs': events 1-3
      |
      |arch/x86/pv/emul-priv-op.c:674:21:
      |  674 | static int cf_check rep_outs(
      |      |                     ^~~~~~~~
      |      |                     |
      |      |                     (1) entry to 'rep_outs'
      |......
      |  688 |     if ( !guest_io_okay(port, bytes_per_rep, curr, ctxt->regs) )
      |      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |           |
      |      |           (3) calling 'guest_io_okay' from 'rep_outs'
      |......
      |  710 |         unsigned int data = 0;
      |      |                      ~~~~
      |      |                      |
      |      |                      (2) capacity: 4 bytes
      |
      +--> 'guest_io_okay': events 4-5
             |
             |  159 | static bool guest_io_okay(unsigned int port, unsigned int bytes,
             |      |             ^~~~~~~~~~~~~
             |      |             |
             |      |             (4) entry to 'guest_io_okay'
             |......
             |  165 |     if ( iopl_ok(v, regs) )
             |      |          ~~~~~~~~~~~~~~~~
             |      |          |
             |      |          (5) calling 'iopl_ok' from 'guest_io_okay'
             |
             +--> 'iopl_ok': event 6
                    |
                    |  148 | static bool iopl_ok(const struct vcpu *v, const struct cpu_user_regs *regs)
                    |      |             ^~~~~~~
                    |      |             |
                    |      |             (6) entry to 'iopl_ok'
                    |
                  'iopl_ok': event 7
                    |
                    |./include/xen/bug.h:141:13:
                    |  141 |     do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
                    |      |             ^
                    |      |             |
                    |      |             (7) following 'false' branch...
  arch/x86/pv/emul-priv-op.c:153:5: note: in expansion of macro 'ASSERT'
                    |  153 |     ASSERT((v->arch.pv.iopl & ~X86_EFLAGS_IOPL) == 0);
                    |      |     ^~~~~~
                    |
                  'iopl_ok': event 8
                    |
                    |./include/xen/bug.h:124:31:
                    |  124 | #define assert_failed(msg) do {                                 \
                    |      |                               ^
                    |      |                               |
                    |      |                               (8) ...to here
  ./include/xen/bug.h:141:32: note: in expansion of macro 'assert_failed'
                    |  141 |     do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
                    |      |                                ^~~~~~~~~~~~~
  arch/x86/pv/emul-priv-op.c:153:5: note: in expansion of macro 'ASSERT'
                    |  153 |     ASSERT((v->arch.pv.iopl & ~X86_EFLAGS_IOPL) == 0);
                    |      |     ^~~~~~
                    |
             <------+
             |
           'guest_io_okay': event 9
             |
             |  165 |     if ( iopl_ok(v, regs) )
             |      |          ^~~~~~~~~~~~~~~~
             |      |          |
             |      |          (9) returning to 'guest_io_okay' from 'iopl_ok'
             |
      <------+
      |
    'rep_outs': events 10-13
      |
      |  688 |     if ( !guest_io_okay(port, bytes_per_rep, curr, ctxt->regs) )
      |      |        ~  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |        |  |
      |      |        |  (10) returning to 'rep_outs' from 'guest_io_okay'
      |      |        (11) following 'true' branch...
      |......
      |  691 |     rc = read_segment(seg, &sreg, ctxt);
      |      |     ~~   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |     |    |
      |      |     |    (13) calling 'read_segment' from 'rep_outs'
      |      |     (12) ...to here
      |
      +--> 'read_segment': events 14-18
             |
             |  493 | static int cf_check read_segment(
             |      |                     ^~~~~~~~~~~~
             |      |                     |
             |      |                     (14) entry to 'read_segment'
             |......
             |  510 |     if ( ctxt->addr_size < 64 )
             |      |        ~
             |      |        |
             |      |        (15) following 'false' branch...
             |......
             |  535 |         switch ( seg )
             |      |         ~~~~~~
             |      |         |
             |      |         (16) ...to here
             |......
             |  538 |             if ( !is_x86_user_segment(seg) )
             |      |                ~
             |      |                |
             |      |                (17) following 'false' branch (when 'seg <= 5')...
             |  539 |                 return X86EMUL_UNHANDLEABLE;
             |  540 |             reg->base = 0;
             |      |             ~~~
             |      |             |
             |      |             (18) ...to here
             |
      <------+
      |
    'rep_outs': events 19-30
      |
      |  691 |     rc = read_segment(seg, &sreg, ctxt);
      |      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |          |
      |      |          (19) returning to 'rep_outs' from 'read_segment'
      |  692 |     if ( rc != X86EMUL_OKAY )
      |      |        ~
      |      |        |
      |      |        (20) following 'false' branch (when 'rc == 0')...
      |......
      |  695 |     if ( !sreg.p )
      |      |     ~~ ~
      |      |     |  |
      |      |     |  (22) following 'false' branch...
      |      |     (21) ...to here
      |  696 |         return X86EMUL_UNHANDLEABLE;
      |  697 |     if ( !sreg.s ||
      |      |     ~~ ~ ~~~~~~~~~~
      |      |     |  |         |
      |      |     |  |         (26) following 'false' branch...
      |      |     |  (24) following 'false' branch...
      |      |     (23) ...to here
      |  698 |          ((sreg.type & (_SEGMENT_CODE >> 8)) &&
      |      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |                |
      |      |                (25) ...to here
      |  699 |           !(sreg.type & (_SEGMENT_WR >> 8))) )
      |      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |......
      |  706 |     poc->bpmatch = check_guest_io_breakpoint(curr, port, bytes_per_rep);
      |      |     ~~~
      |      |     |
      |      |     (27) ...to here
      |  707 |
      |  708 |     while ( *reps < goal )
      |      |             ~~~~~~~~~~~~
      |      |                   |
      |      |                   (28) following 'true' branch...
      |  709 |     {
      |  710 |         unsigned int data = 0;
      |      |         ~~~~~~~~
      |      |         |
      |      |         (29) ...to here
      |......
      |  713 |         rc = pv_emul_virt_to_linear(sreg.base, offset, bytes_per_rep,
      |      |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |              |
      |      |              (30) calling 'pv_emul_virt_to_linear' from 'rep_outs'
      |  714 |                                     sreg.limit, seg, ctxt, &addr);
      |      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |
      +--> 'pv_emul_virt_to_linear': events 31-33
             |
             |  580 | static int pv_emul_virt_to_linear(unsigned long base, unsigned long offset,
             |      |            ^~~~~~~~~~~~~~~~~~~~~~
             |      |            |
             |      |            (31) entry to 'pv_emul_virt_to_linear'
             |......
             |  596 |     else if ( !__addr_ok(*addr) )
             |      |             ~
             |      |             |
             |      |             (32) following 'false' branch...
             |......
             |  603 |     return rc;
             |      |     ~~~~~~
             |      |     |
             |      |     (33) ...to here
             |
      <------+
      |
    'rep_outs': events 34-37
      |
      |  713 |         rc = pv_emul_virt_to_linear(sreg.base, offset, bytes_per_rep,
      |      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |      |              |
      |      |              (34) returning to 'rep_outs' from 'pv_emul_virt_to_linear'
      |  714 |                                     sreg.limit, seg, ctxt, &addr);
      |      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |  715 |         if ( rc != X86EMUL_OKAY )
      |      |            ~
      |      |            |
      |      |            (35) following 'false' branch (when 'rc == 0')...
      |......
      |  718 |         if ( (rc = __copy_from_guest_pv(&data, (void __user *)addr,
      |      |         ~~         ~
      |      |         |          |
      |      |         |          (37) inlined call to '__copy_from_guest_pv' from 'rep_outs'
      |      |         (36) ...to here
      |
      +--> '__copy_from_guest_pv': events 38-41
             |
             |./arch/x86/include/asm/uaccess.h:294:8:
             |  294 |     if (__builtin_constant_p(n)) {
             |      |        ^
             |      |        |
             |      |        (38) following 'true' branch...
             |  295 |         unsigned long ret;
             |      |         ~~~~~~~~
             |      |         |
             |      |         (39) ...to here
             |  296 |
             |  297 |         switch (n) {
             |      |         ~~~~~~
             |      |         |
             |      |         (40) following 'case 8:' branch...
             |......
             |  307 |         case 8:
             |      |         ~~~~
             |      |         |
             |      |         (41) ...to here
             |
           '__copy_from_guest_pv': event 42
             |
             |  174 |         __asm__ __volatile__(                                           \
             |      |         ^~~~~~~
             |      |         |
             |      |         (42) out-of-bounds write from byte 4 till byte 7 but 'data' ends at byte 4
  ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
             |  229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
             |      |             ^~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
             |  236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
             |      |     ^~~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
             |  308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
             |      |             ^~~~~~~~~~~~~~
             |
  ./arch/x86/include/asm/uaccess.h:174:9: note: write of 4 bytes to beyond the end of 'data'
    174 |         __asm__ __volatile__(                                           \
        |         ^~~~~~~
  ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
    229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
        |             ^~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
    236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
        |     ^~~~~~~~~~~~~~~
  ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
    308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
        |             ^~~~~~~~~~~~~~

    ┌──────────────────────────────────────────────────────────────────────┐
    │                    write of 'uint64_t' (8 bytes)                     │
    └──────────────────────────────────────────────────────────────────────┘
                     │                                   │
                     │                                   │
                     v                                   v
    ┌──────────────────────────────────┐┌──────────────────────────────────┐
    │  'data' (type: 'unsigned int')   ││        after valid range         │
    └──────────────────────────────────┘└──────────────────────────────────┘
    ├────────────────┬─────────────────┤├────────────────┬─────────────────┤
                     │                                   │
            ╭────────┴────────╮              ╭───────────┴──────────╮
            │capacity: 4 bytes│              │⚠️  overflow of 4 bytes│
            ╰─────────────────╯              ╰──────────────────────╯

What is happening is that it's seen that __copy_from_guest_pv() has a case for
8, and it doesn't have any information about bytes_per_rep which is an input
to the function.

Architecturally, there's no 64-bit variant of OUTS in x86.  Leaving an
assertion to this effect is enough to satisfy the analyser.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

A little RFC, but the analyser is right - if rep_outs() were to be called with
anything more than 4, the stack would be clobbered, and no developer is
finding that bug the easy way...
---
 xen/arch/x86/pv/emul-priv-op.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index f101510a1bab..1460ecc187be 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -685,6 +685,8 @@ static int cf_check rep_outs(
 
     *reps = 0;
 
+    ASSERT(bytes_per_rep <= 4); /* i.e. 'data' being 4 bytes is fine. */
+
     if ( !guest_io_okay(port, bytes_per_rep, curr, ctxt->regs) )
         return X86EMUL_UNHANDLEABLE;
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Sat May 11 16:00:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 16:00:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720371.1123430 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5p8a-0007Y4-Au; Sat, 11 May 2024 15:59:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720371.1123430; Sat, 11 May 2024 15:59:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5p8a-0007Xx-8H; Sat, 11 May 2024 15:59:52 +0000
Received: by outflank-mailman (input) for mailman id 720371;
 Sat, 11 May 2024 15:59:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5p8Z-0007Xn-59; Sat, 11 May 2024 15:59:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5p8Z-000451-0A; Sat, 11 May 2024 15:59:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5p8Y-0003ta-Lk; Sat, 11 May 2024 15:59:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5p8Y-000670-LK; Sat, 11 May 2024 15:59:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=w0ShHxPWbXZ5p8Nsr3yfGgC4DjRP18xsgYNQTQqu13s=; b=nuOZbefnEpY5aKJccmxqjsVnMY
	2CTQL9Q7M9bn0xYd2jc10vPkyZnRMdwWqrNULfVHNHu3nIaSaQJNJJ2lAFgLu9Jb9pYMGMYN5HbGD
	1OLH0hHXJcuO+7+xl5gPLLGfs/R3evFTqqiyrIuIixamm2X+r1fbNudBsdIKIT4EMoX0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185977-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185977: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt-vhd:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=cf87f46fd34d6c19283d9625a7822f20d90b64a4
X-Osstest-Versions-That:
    linux=f4345f05c0dfc73c617e66f3b809edb8ddd41075
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 15:59:50 +0000

flight 185977 linux-linus real [real]
flight 185981 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185977/
http://logs.test-lab.xenproject.org/osstest/logs/185981/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-qcow2     8 xen-boot                 fail REGR. vs. 185972

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-vhd 10 host-ping-check-xen fail pass in 185981-retest
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 185981-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185981 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185981 never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check fail in 185981 never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check fail in 185981 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185972
 test-armhf-armhf-examine      8 reboot                       fail  like 185972
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185972
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185972
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                cf87f46fd34d6c19283d9625a7822f20d90b64a4
baseline version:
 linux                f4345f05c0dfc73c617e66f3b809edb8ddd41075

Last test of basis   185972  2024-05-10 18:14:02 Z    0 days
Testing same since   185977  2024-05-11 02:27:16 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Paul E. McKenney" <paulmck@kernel.org>
  Agustin Gutierrez <agustin.gutierrez@amd.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Potapenko <glider@google.com>
  Andi Shyti <andi.shyti@linux.intel.com>
  Andrew Morton <akpm@linux-foundation.org>
  Animesh Manna <animesh.manna@intel.com>
  Barry Song <v-songbaohua@oppo.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
  Christoph Hellwig <hch@lst.de>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
  Dave Airlie <airlied@redhat.com>
  David Hildenbrand <david@redhat.com>
  Douglas Anderson <dianders@chromium.org>
  Gnattu OC <gnattuoc@me.com>
  Hailong.Liu <hailong.liu@oppo.com>
  Hariom Panthi <hariom1.p@samsung.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jerome Brunet <jbrunet@baylibre.com>
  John Garry <john.g.garry@oracle.com>
  Kai Vehmanen <kai.vehmanen@intel.com>
  Karthikeyan Ramasubramanian <kramasub@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kent Gibson <warthog618@gmail.com>
  Krzysztof Gibala <krzysztof.gibala@intel.com>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  Lijo Lazar <lijo.lazar@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lucas De Marchi <lucas.demarchi@intel.com>
  Luis Chamberlain <mcgrof@kernel.org>
  Maninder Singh <maninder1.s@samsung.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Marius Fleischer <fleischermarius@gmail.com>
  Matthew Brost <matthew.brost@intel.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Michel Dänzer <mdaenzer@redhat.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD
  Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
  Nicholas Susanto <nicholas.susanto@amd.com>
  Oscar Salvador <osalvador@suse.de>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Peter Xu <peterx@redhat.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Tiezhu Yang <yangtiezhu@loongson.cn>
  Tom Chung <chiahsuan.chung@amd.com>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Zhongqiu Han <quic_zhonhan@quicinc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1257 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat May 11 19:02:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 19:02:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720421.1123481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5rzX-0005pO-9i; Sat, 11 May 2024 19:02:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720421.1123481; Sat, 11 May 2024 19:02:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5rzX-0005pH-6F; Sat, 11 May 2024 19:02:43 +0000
Received: by outflank-mailman (input) for mailman id 720421;
 Sat, 11 May 2024 19:02:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5rzV-0005p6-GG; Sat, 11 May 2024 19:02:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5rzV-0007hm-Ee; Sat, 11 May 2024 19:02:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5rzV-0002FW-5V; Sat, 11 May 2024 19:02:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5rzV-00006H-56; Sat, 11 May 2024 19:02:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eTEq43yG/74r+rwqORR9Mk27d1kgKlMc5zpL8o2Ktbs=; b=Rw3Jnvk7bf0B+Z4nqiUQeLV1tm
	zJj2kPc5rwhRXa8TkC2RssIdP+RgMTYJ+GY8U6T74wnv8+cqVU8GhGuH0jm0/xWFo1dPcsHRCfir8
	qmDZHIODYJ6sbzdVYTduSvBKl5HICjWgonzL0qA3ji4yg1AKJ/T4fHPz8rDaYDSqMEFc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185980-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185980: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=996576b965ccdf0de17aafa14282925e408e1200
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 19:02:41 +0000

flight 185980 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185980/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185975
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185975
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185975
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185975
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185975
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185975
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  996576b965ccdf0de17aafa14282925e408e1200

Last test of basis   185975  2024-05-11 01:37:06 Z    0 days
Testing same since   185980  2024-05-11 10:13:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Leigh Brown <leigh@solinno.co.uk>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   996576b965..46aa3031ae  46aa3031ae89ac1771f4159972edab65710e7349 -> master


From xen-devel-bounces@lists.xenproject.org Sat May 11 19:42:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 19:42:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720430.1123491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5scF-0002ME-5d; Sat, 11 May 2024 19:42:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720430.1123491; Sat, 11 May 2024 19:42:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5scF-0002M7-2D; Sat, 11 May 2024 19:42:43 +0000
Received: by outflank-mailman (input) for mailman id 720430;
 Sat, 11 May 2024 19:42:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HmPl=MO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s5scD-0002M1-7Z
 for xen-devel@lists.xenproject.org; Sat, 11 May 2024 19:42:41 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a02c1680-0fce-11ef-909d-e314d9c70b13;
 Sat, 11 May 2024 21:42:39 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso145039066b.2
 for <xen-devel@lists.xenproject.org>; Sat, 11 May 2024 12:42:39 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c34fed1sm3751139a12.96.2024.05.11.12.42.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 11 May 2024 12:42:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a02c1680-0fce-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715456559; x=1716061359; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=oqZtZryQor/ppvAR+iR/fgku6YlEtXdshM2WNruV4V4=;
        b=XMhne4QkVm/0Y3MqcyGvhGbDfR9Gk/UwRfhwNbBwtTaGPNmvkSvFV5VmbACad8Imh0
         1DjRYUqPgDd4F67hPkQwlZBHtjDSmvY9EPMevaY9puHKC/uILNaugSskUWcfRb8BDnBv
         3lzga4bge2+ejXLc0fOjmgaYeje25upOfmsX0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715456559; x=1716061359;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=oqZtZryQor/ppvAR+iR/fgku6YlEtXdshM2WNruV4V4=;
        b=OjuLNBg0Lu0IJEsS62r13hR4pACtpuVYcxRQr/Ht+na8iRhC1sJme6jZ0Q3FuglZSP
         pF8nBr7cUhr9DyakrICrV3aZZwTznYtZ3dhm8asq3gWt5jRVjC1SBQgQM8gsuRvs2T+5
         kxU/0gc5MeyT6zh3reVnYCAU7lFWszVfUT2lyxid3oWVhlL8rmG55Sf0UEFrNeBnXIyr
         Q3MnLhABzTcXAVdVQiWcz1ycSWs8nNCxPSiSb4CYTN69TqPyLFFtcW72+clhoGzhg3Bj
         TjVcCph/g/UVXP5xyyqIXEKQxOLEp9oNuodxv3ade+zTohjIH1Ef1Y0xoy9FyU/8VSwu
         eKyw==
X-Gm-Message-State: AOJu0YzG0P5deCgf3kFWqTXCmClbBEax8OW5BX8jYvImkvA+Dpee+t6O
	R+t+UYfPLNw8is7HK/knu1aFT95iIOeiq1CG2RstYDK5UvWwUaSOMRiAr4kE0fDmFjPou2QH8U1
	K
X-Google-Smtp-Source: AGHT+IHvW56HBIUibeOV4rBI6uPxDVDXUpS2j7ihhlwFhZHdcW6XWB5QCaydJX8oFiqgVuGb8rcdrw==
X-Received: by 2002:a50:d4c8:0:b0:572:719f:b430 with SMTP id 4fb4d7f45d1cf-5734d6f0080mr5347809a12.32.1715456558918;
        Sat, 11 May 2024 12:42:38 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>
Subject: [PATCH] x86/cpufreq: Rename cpuid variable/parameters to cpu
Date: Sat, 11 May 2024 20:42:37 +0100
Message-Id: <20240511194237.2596271-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Various functions have a parameter or local variable called cpuid, but this
triggers a MISRA R5.3 violation because we also have a function called cpuid()
which wraps the real CPUID instruction.

In all these cases, it's a Xen cpu index, which is far more commonly named
just cpu in our code.

While adjusting these, fix a couple of other issues:

 * cpufreq_cpu_init() is on the end of a hypercall (with in-memory parameters,
   even), making EFAULT the wrong error to use.  Use EOPNOTSUPP instead.

 * check_est_cpu() is wrong to tie EIST to just Intel, and nowhere else using
   EIST makes this restriction.  Just check the feature itself, which is more
   succinctly done after being folded into its single caller.

 * In powernow_cpufreq_update(), replace an opencoded cpu_online().

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>

cpu needs to stay signed for now in set_px_pminfo(), because of get_cpu_id().
This can be cleaned up by making better use of BAD_APICID, but that's a much
more involved change.
---
 xen/arch/x86/acpi/cpu_idle.c              | 14 ++++----
 xen/arch/x86/acpi/cpufreq/cpufreq.c       | 24 +++----------
 xen/arch/x86/acpi/cpufreq/hwp.c           |  6 ++--
 xen/arch/x86/acpi/cpufreq/powernow.c      |  6 ++--
 xen/drivers/cpufreq/cpufreq.c             | 18 +++++-----
 xen/drivers/cpufreq/utility.c             | 43 +++++++++++------------
 xen/include/acpi/cpufreq/cpufreq.h        |  6 ++--
 xen/include/acpi/cpufreq/processor_perf.h |  8 ++---
 xen/include/xen/pmstat.h                  |  6 ++--
 9 files changed, 57 insertions(+), 74 deletions(-)

diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index cfce4cc0408f..c8db1aa9913a 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -1498,14 +1498,14 @@ static void amd_cpuidle_init(struct acpi_processor_power *power)
         vendor_override = -1;
 }
 
-uint32_t pmstat_get_cx_nr(uint32_t cpuid)
+uint32_t pmstat_get_cx_nr(unsigned int cpu)
 {
-    return processor_powers[cpuid] ? processor_powers[cpuid]->count : 0;
+    return processor_powers[cpu] ? processor_powers[cpu]->count : 0;
 }
 
-int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
+int pmstat_get_cx_stat(unsigned int cpu, struct pm_cx_stat *stat)
 {
-    struct acpi_processor_power *power = processor_powers[cpuid];
+    struct acpi_processor_power *power = processor_powers[cpu];
     uint64_t idle_usage = 0, idle_res = 0;
     uint64_t last_state_update_tick, current_stime, current_tick;
     uint64_t usage[ACPI_PROCESSOR_MAX_POWER] = { 0 };
@@ -1522,7 +1522,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
         return 0;
     }
 
-    stat->idle_time = get_cpu_idle_time(cpuid);
+    stat->idle_time = get_cpu_idle_time(cpu);
     nr = min(stat->nr, power->count);
 
     /* mimic the stat when detail info hasn't been registered by dom0 */
@@ -1572,7 +1572,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
             idle_res += res[i];
         }
 
-        get_hw_residencies(cpuid, &hw_res);
+        get_hw_residencies(cpu, &hw_res);
 
 #define PUT_xC(what, n) do { \
         if ( stat->nr_##what >= n && \
@@ -1613,7 +1613,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
     return 0;
 }
 
-int pmstat_reset_cx_stat(uint32_t cpuid)
+int pmstat_reset_cx_stat(unsigned int cpu)
 {
     return 0;
 }
diff --git a/xen/arch/x86/acpi/cpufreq/cpufreq.c b/xen/arch/x86/acpi/cpufreq/cpufreq.c
index 2b6ef99678ae..a341f2f02063 100644
--- a/xen/arch/x86/acpi/cpufreq/cpufreq.c
+++ b/xen/arch/x86/acpi/cpufreq/cpufreq.c
@@ -55,17 +55,6 @@ struct acpi_cpufreq_data *cpufreq_drv_data[NR_CPUS];
 static bool __read_mostly acpi_pstate_strict;
 boolean_param("acpi_pstate_strict", acpi_pstate_strict);
 
-static int check_est_cpu(unsigned int cpuid)
-{
-    struct cpuinfo_x86 *cpu = &cpu_data[cpuid];
-
-    if (cpu->x86_vendor != X86_VENDOR_INTEL ||
-        !cpu_has(cpu, X86_FEATURE_EIST))
-        return 0;
-
-    return 1;
-}
-
 static unsigned extract_io(u32 value, struct acpi_cpufreq_data *data)
 {
     struct processor_performance *perf;
@@ -530,7 +519,7 @@ static int cf_check acpi_cpufreq_cpu_init(struct cpufreq_policy *policy)
         if (cpufreq_verbose)
             printk("xen_pminfo: @acpi_cpufreq_cpu_init,"
                    "HARDWARE addr space\n");
-        if (!check_est_cpu(cpu)) {
+        if (!cpu_has(c, X86_FEATURE_EIST)) {
             result = -ENODEV;
             goto err_unreg;
         }
@@ -690,15 +679,12 @@ static int __init cf_check cpufreq_driver_late_init(void)
 }
 __initcall(cpufreq_driver_late_init);
 
-int cpufreq_cpu_init(unsigned int cpuid)
+int cpufreq_cpu_init(unsigned int cpu)
 {
-    int ret;
-
     /* Currently we only handle Intel, AMD and Hygon processor */
     if ( boot_cpu_data.x86_vendor &
          (X86_VENDOR_INTEL | X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-        ret = cpufreq_add_cpu(cpuid);
-    else
-        ret = -EFAULT;
-    return ret;
+        return cpufreq_add_cpu(cpu);
+
+    return -EOPNOTSUPP;
 }
diff --git a/xen/arch/x86/acpi/cpufreq/hwp.c b/xen/arch/x86/acpi/cpufreq/hwp.c
index e61212803e71..59b57a4cef86 100644
--- a/xen/arch/x86/acpi/cpufreq/hwp.c
+++ b/xen/arch/x86/acpi/cpufreq/hwp.c
@@ -506,11 +506,11 @@ static void cf_check hwp_set_misc_turbo(void *info)
     }
 }
 
-static int cf_check hwp_cpufreq_update(int cpuid, struct cpufreq_policy *policy)
+static int cf_check hwp_cpufreq_update(unsigned int cpu, struct cpufreq_policy *policy)
 {
-    on_selected_cpus(cpumask_of(cpuid), hwp_set_misc_turbo, policy, 1);
+    on_selected_cpus(cpumask_of(cpu), hwp_set_misc_turbo, policy, 1);
 
-    return per_cpu(hwp_drv_data, cpuid)->ret;
+    return per_cpu(hwp_drv_data, cpu)->ret;
 }
 
 static const struct cpufreq_driver __initconst_cf_clobber
diff --git a/xen/arch/x86/acpi/cpufreq/powernow.c b/xen/arch/x86/acpi/cpufreq/powernow.c
index 8a27ee82a5b0..69364e185562 100644
--- a/xen/arch/x86/acpi/cpufreq/powernow.c
+++ b/xen/arch/x86/acpi/cpufreq/powernow.c
@@ -68,12 +68,12 @@ static void cf_check update_cpb(void *data)
 }
 
 static int cf_check powernow_cpufreq_update(
-    int cpuid, struct cpufreq_policy *policy)
+    unsigned int cpu, struct cpufreq_policy *policy)
 {
-    if (!cpumask_test_cpu(cpuid, &cpu_online_map))
+    if ( !cpu_online(cpu) )
         return -EINVAL;
 
-    on_selected_cpus(cpumask_of(cpuid), update_cpb, policy, 1);
+    on_selected_cpus(cpumask_of(cpu), update_cpb, policy, 1);
 
     return 0;
 }
diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
index 36c800f4fa39..a74593b70577 100644
--- a/xen/drivers/cpufreq/cpufreq.c
+++ b/xen/drivers/cpufreq/cpufreq.c
@@ -459,21 +459,21 @@ static void print_PPC(unsigned int platform_limit)
 
 int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
 {
-    int ret=0, cpuid;
+    int ret=0, cpu;
     struct processor_pminfo *pmpt;
     struct processor_performance *pxpt;
 
-    cpuid = get_cpu_id(acpi_id);
-    if ( cpuid < 0 || !perf )
+    cpu = get_cpu_id(acpi_id);
+    if ( cpu < 0 || !perf )
     {
         ret = -EINVAL;
         goto out;
     }
     if ( cpufreq_verbose )
-        printk("Set CPU acpi_id(%d) cpuid(%d) Px State info:\n",
+        printk("Set CPU acpi_id(%d) cpu(%d) Px State info:\n",
                acpi_id, cpuid);
 
-    pmpt = processor_pminfo[cpuid];
+    pmpt = processor_pminfo[cpu];
     if ( !pmpt )
     {
         pmpt = xzalloc(struct processor_pminfo);
@@ -482,11 +482,11 @@ int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
             ret = -ENOMEM;
             goto out;
         }
-        processor_pminfo[cpuid] = pmpt;
+        processor_pminfo[cpu] = pmpt;
     }
     pxpt = &pmpt->perf;
     pmpt->acpi_id = acpi_id;
-    pmpt->id = cpuid;
+    pmpt->id = cpu;
 
     if ( perf->flags & XEN_PX_PCT )
     {
@@ -564,7 +564,7 @@ int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
 
         if ( pxpt->init == XEN_PX_INIT )
         {
-            ret = cpufreq_limit_change(cpuid);
+            ret = cpufreq_limit_change(cpu);
             goto out;
         }
     }
@@ -573,7 +573,7 @@ int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
     {
         pxpt->init = XEN_PX_INIT;
 
-        ret = cpufreq_cpu_init(cpuid);
+        ret = cpufreq_cpu_init(cpu);
         goto out;
     }
 
diff --git a/xen/drivers/cpufreq/utility.c b/xen/drivers/cpufreq/utility.c
index dbf8985969b2..6d2413b32a55 100644
--- a/xen/drivers/cpufreq/utility.c
+++ b/xen/drivers/cpufreq/utility.c
@@ -86,13 +86,12 @@ void cpufreq_statistic_update(unsigned int cpu, uint8_t from, uint8_t to)
     spin_unlock(cpufreq_statistic_lock);
 }
 
-int cpufreq_statistic_init(unsigned int cpuid)
+int cpufreq_statistic_init(unsigned int cpu)
 {
     uint32_t i, count;
     struct pm_px *pxpt;
-    const struct processor_pminfo *pmpt = processor_pminfo[cpuid];
-    spinlock_t *cpufreq_statistic_lock = 
-                          &per_cpu(cpufreq_statistic_lock, cpuid);
+    const struct processor_pminfo *pmpt = processor_pminfo[cpu];
+    spinlock_t *cpufreq_statistic_lock = &per_cpu(cpufreq_statistic_lock, cpu);
 
     spin_lock_init(cpufreq_statistic_lock);
 
@@ -101,7 +100,7 @@ int cpufreq_statistic_init(unsigned int cpuid)
 
     spin_lock(cpufreq_statistic_lock);
 
-    pxpt = per_cpu(cpufreq_statistic_data, cpuid);
+    pxpt = per_cpu(cpufreq_statistic_data, cpu);
     if ( pxpt ) {
         spin_unlock(cpufreq_statistic_lock);
         return 0;
@@ -114,7 +113,7 @@ int cpufreq_statistic_init(unsigned int cpuid)
         spin_unlock(cpufreq_statistic_lock);
         return -ENOMEM;
     }
-    per_cpu(cpufreq_statistic_data, cpuid) = pxpt;
+    per_cpu(cpufreq_statistic_data, cpu) = pxpt;
 
     pxpt->u.trans_pt = xzalloc_array(uint64_t, count * count);
     if (!pxpt->u.trans_pt) {
@@ -138,22 +137,21 @@ int cpufreq_statistic_init(unsigned int cpuid)
         pxpt->u.pt[i].freq = pmpt->perf.states[i].core_frequency;
 
     pxpt->prev_state_wall = NOW();
-    pxpt->prev_idle_wall = get_cpu_idle_time(cpuid);
+    pxpt->prev_idle_wall = get_cpu_idle_time(cpu);
 
     spin_unlock(cpufreq_statistic_lock);
 
     return 0;
 }
 
-void cpufreq_statistic_exit(unsigned int cpuid)
+void cpufreq_statistic_exit(unsigned int cpu)
 {
     struct pm_px *pxpt;
-    spinlock_t *cpufreq_statistic_lock = 
-               &per_cpu(cpufreq_statistic_lock, cpuid);
+    spinlock_t *cpufreq_statistic_lock = &per_cpu(cpufreq_statistic_lock, cpu);
 
     spin_lock(cpufreq_statistic_lock);
 
-    pxpt = per_cpu(cpufreq_statistic_data, cpuid);
+    pxpt = per_cpu(cpufreq_statistic_data, cpu);
     if (!pxpt) {
         spin_unlock(cpufreq_statistic_lock);
         return;
@@ -162,22 +160,21 @@ void cpufreq_statistic_exit(unsigned int cpuid)
     xfree(pxpt->u.trans_pt);
     xfree(pxpt->u.pt);
     xfree(pxpt);
-    per_cpu(cpufreq_statistic_data, cpuid) = NULL;
+    per_cpu(cpufreq_statistic_data, cpu) = NULL;
 
     spin_unlock(cpufreq_statistic_lock);
 }
 
-void cpufreq_statistic_reset(unsigned int cpuid)
+void cpufreq_statistic_reset(unsigned int cpu)
 {
     uint32_t i, j, count;
     struct pm_px *pxpt;
-    const struct processor_pminfo *pmpt = processor_pminfo[cpuid];
-    spinlock_t *cpufreq_statistic_lock = 
-               &per_cpu(cpufreq_statistic_lock, cpuid);
+    const struct processor_pminfo *pmpt = processor_pminfo[cpu];
+    spinlock_t *cpufreq_statistic_lock = &per_cpu(cpufreq_statistic_lock, cpu);
 
     spin_lock(cpufreq_statistic_lock);
 
-    pxpt = per_cpu(cpufreq_statistic_data, cpuid);
+    pxpt = per_cpu(cpufreq_statistic_data, cpu);
     if ( !pmpt || !pxpt || !pxpt->u.pt || !pxpt->u.trans_pt ) {
         spin_unlock(cpufreq_statistic_lock);
         return;
@@ -194,7 +191,7 @@ void cpufreq_statistic_reset(unsigned int cpuid)
     }
 
     pxpt->prev_state_wall = NOW();
-    pxpt->prev_idle_wall = get_cpu_idle_time(cpuid);
+    pxpt->prev_idle_wall = get_cpu_idle_time(cpu);
 
     spin_unlock(cpufreq_statistic_lock);
 }
@@ -389,7 +386,7 @@ int cpufreq_driver_getavg(unsigned int cpu, unsigned int flag)
     return policy->cur;
 }
 
-int cpufreq_update_turbo(int cpuid, int new_state)
+int cpufreq_update_turbo(unsigned int cpu, int new_state)
 {
     struct cpufreq_policy *policy;
     int curr_state;
@@ -399,7 +396,7 @@ int cpufreq_update_turbo(int cpuid, int new_state)
         new_state != CPUFREQ_TURBO_DISABLED)
         return -EINVAL;
 
-    policy = per_cpu(cpufreq_cpu_policy, cpuid);
+    policy = per_cpu(cpufreq_cpu_policy, cpu);
     if (!policy)
         return -EACCES;
 
@@ -413,7 +410,7 @@ int cpufreq_update_turbo(int cpuid, int new_state)
     policy->turbo = new_state;
     if (cpufreq_driver.update)
     {
-        ret = alternative_call(cpufreq_driver.update, cpuid, policy);
+        ret = alternative_call(cpufreq_driver.update, cpu, policy);
         if (ret)
             policy->turbo = curr_state;
     }
@@ -422,11 +419,11 @@ int cpufreq_update_turbo(int cpuid, int new_state)
 }
 
 
-int cpufreq_get_turbo_status(int cpuid)
+int cpufreq_get_turbo_status(unsigned int cpu)
 {
     struct cpufreq_policy *policy;
 
-    policy = per_cpu(cpufreq_cpu_policy, cpuid);
+    policy = per_cpu(cpufreq_cpu_policy, cpu);
     return policy && policy->turbo == CPUFREQ_TURBO_ENABLED;
 }
 
diff --git a/xen/include/acpi/cpufreq/cpufreq.h b/xen/include/acpi/cpufreq/cpufreq.h
index b4685a908599..443427153bb6 100644
--- a/xen/include/acpi/cpufreq/cpufreq.h
+++ b/xen/include/acpi/cpufreq/cpufreq.h
@@ -145,8 +145,8 @@ extern int cpufreq_driver_getavg(unsigned int cpu, unsigned int flag);
 #define CPUFREQ_TURBO_UNSUPPORTED   0
 #define CPUFREQ_TURBO_ENABLED       1
 
-extern int cpufreq_update_turbo(int cpuid, int new_state);
-extern int cpufreq_get_turbo_status(int cpuid);
+int cpufreq_update_turbo(unsigned int cpu, int new_state);
+int cpufreq_get_turbo_status(unsigned int cpu);
 
 static inline int
 __cpufreq_governor(struct cpufreq_policy *policy, unsigned int event)
@@ -167,7 +167,7 @@ struct cpufreq_driver {
     int    (*init)(struct cpufreq_policy *policy);
     int    (*verify)(struct cpufreq_policy *policy);
     int    (*setpolicy)(struct cpufreq_policy *policy);
-    int    (*update)(int cpuid, struct cpufreq_policy *policy);
+    int    (*update)(unsigned int cpu, struct cpufreq_policy *policy);
     int    (*target)(struct cpufreq_policy *policy,
                      unsigned int target_freq,
                      unsigned int relation);
diff --git a/xen/include/acpi/cpufreq/processor_perf.h b/xen/include/acpi/cpufreq/processor_perf.h
index 7cd240061947..5f48aceadb21 100644
--- a/xen/include/acpi/cpufreq/processor_perf.h
+++ b/xen/include/acpi/cpufreq/processor_perf.h
@@ -12,9 +12,9 @@ unsigned int powernow_register_driver(void);
 unsigned int get_measured_perf(unsigned int cpu, unsigned int flag);
 void cpufreq_residency_update(unsigned int cpu, uint8_t state);
 void cpufreq_statistic_update(unsigned int cpu, uint8_t from, uint8_t to);
-int  cpufreq_statistic_init(unsigned int cpuid);
-void cpufreq_statistic_exit(unsigned int cpuid);
-void cpufreq_statistic_reset(unsigned int cpuid);
+int  cpufreq_statistic_init(unsigned int cpu);
+void cpufreq_statistic_exit(unsigned int cpu);
+void cpufreq_statistic_reset(unsigned int cpu);
 
 int  cpufreq_limit_change(unsigned int cpu);
 
@@ -59,5 +59,5 @@ struct pm_px {
 
 DECLARE_PER_CPU(struct pm_px *, cpufreq_statistic_data);
 
-int cpufreq_cpu_init(unsigned int cpuid);
+int cpufreq_cpu_init(unsigned int cpu);
 #endif /* __XEN_PROCESSOR_PM_H__ */
diff --git a/xen/include/xen/pmstat.h b/xen/include/xen/pmstat.h
index 43b826ad4d37..e6ab1423a9be 100644
--- a/xen/include/xen/pmstat.h
+++ b/xen/include/xen/pmstat.h
@@ -7,9 +7,9 @@
 
 int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf);
 long set_cx_pminfo(uint32_t acpi_id, struct xen_processor_power *power);
-uint32_t pmstat_get_cx_nr(uint32_t cpuid);
-int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat);
-int pmstat_reset_cx_stat(uint32_t cpuid);
+uint32_t pmstat_get_cx_nr(unsigned int cpu);
+int pmstat_get_cx_stat(unsigned int cpu, struct pm_cx_stat *stat);
+int pmstat_reset_cx_stat(unsigned int cpu);
 
 int do_get_pm_info(struct xen_sysctl_get_pmstat *op);
 int do_pm_op(struct xen_sysctl_pm_op *op);
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Sat May 11 23:43:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 May 2024 23:43:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720459.1123501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5wMn-0005Br-AN; Sat, 11 May 2024 23:43:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720459.1123501; Sat, 11 May 2024 23:43:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5wMn-0005Bk-7b; Sat, 11 May 2024 23:43:01 +0000
Received: by outflank-mailman (input) for mailman id 720459;
 Sat, 11 May 2024 23:43:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5wMm-0005Ba-7A; Sat, 11 May 2024 23:43:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5wMm-0004DG-4A; Sat, 11 May 2024 23:43:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s5wMl-0001jw-OJ; Sat, 11 May 2024 23:42:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s5wMl-0002kX-Ny; Sat, 11 May 2024 23:42:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+DOMwn3x6x03zQpCVn7RTdzhvjC4LJy0hIZ9px2COsA=; b=YhdzsGJFBi/jAjtuPP2HEDsEk4
	4Kwu8RC4wIJum+OQPz2neGLKxiqXjOjbUxt8n+1UPPukbHsQTXxfFOuy1nzPTQmKiNPi1KCli+AoQ
	LZM/X7xdFOHCARVOkHzfBWN6Q6xt77AfQcJcV6L4zQOaNvldqxma0YUE9LYilu/uxDBU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185982-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185982: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-vhd:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-pvshim:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=cf87f46fd34d6c19283d9625a7822f20d90b64a4
X-Osstest-Versions-That:
    linux=f4345f05c0dfc73c617e66f3b809edb8ddd41075
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 May 2024 23:42:59 +0000

flight 185982 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185982/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-vhd 10 host-ping-check-xen fail in 185977 pass in 185982
 test-armhf-armhf-xl-qcow2     8 xen-boot         fail in 185977 pass in 185982
 test-armhf-armhf-xl-credit2   8 xen-boot         fail in 185977 pass in 185982
 test-amd64-amd64-xl-pvshim   20 guest-localmigrate/x10     fail pass in 185977
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail pass in 185977
 test-armhf-armhf-xl-multivcpu  8 xen-boot                  fail pass in 185977
 test-armhf-armhf-libvirt      8 xen-boot                   fail pass in 185977

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185977 like 185972
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 185977 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 185977 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185977 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185972
 test-armhf-armhf-examine      8 reboot                       fail  like 185972
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185972
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185972
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                cf87f46fd34d6c19283d9625a7822f20d90b64a4
baseline version:
 linux                f4345f05c0dfc73c617e66f3b809edb8ddd41075

Last test of basis   185972  2024-05-10 18:14:02 Z    1 days
Testing same since   185977  2024-05-11 02:27:16 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Paul E. McKenney" <paulmck@kernel.org>
  Agustin Gutierrez <agustin.gutierrez@amd.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Potapenko <glider@google.com>
  Andi Shyti <andi.shyti@linux.intel.com>
  Andrew Morton <akpm@linux-foundation.org>
  Animesh Manna <animesh.manna@intel.com>
  Barry Song <v-songbaohua@oppo.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Chaitanya Kumar Borah <chaitanya.kumar.borah@intel.com>
  Christoph Hellwig <hch@lst.de>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
  Dave Airlie <airlied@redhat.com>
  David Hildenbrand <david@redhat.com>
  Douglas Anderson <dianders@chromium.org>
  Gnattu OC <gnattuoc@me.com>
  Hailong.Liu <hailong.liu@oppo.com>
  Hariom Panthi <hariom1.p@samsung.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jerome Brunet <jbrunet@baylibre.com>
  John Garry <john.g.garry@oracle.com>
  Kai Vehmanen <kai.vehmanen@intel.com>
  Karthikeyan Ramasubramanian <kramasub@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kent Gibson <warthog618@gmail.com>
  Krzysztof Gibala <krzysztof.gibala@intel.com>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  Lijo Lazar <lijo.lazar@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lucas De Marchi <lucas.demarchi@intel.com>
  Luis Chamberlain <mcgrof@kernel.org>
  Maninder Singh <maninder1.s@samsung.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Marius Fleischer <fleischermarius@gmail.com>
  Matthew Brost <matthew.brost@intel.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Michel Dänzer <mdaenzer@redhat.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD
  Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
  Nicholas Susanto <nicholas.susanto@amd.com>
  Oscar Salvador <osalvador@suse.de>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Peter Xu <peterx@redhat.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Tiezhu Yang <yangtiezhu@loongson.cn>
  Tom Chung <chiahsuan.chung@amd.com>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Zhongqiu Han <quic_zhonhan@quicinc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   f4345f05c0df..cf87f46fd34d  cf87f46fd34d6c19283d9625a7822f20d90b64a4 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun May 12 02:16:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 02:16:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720484.1123511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5ylO-0005dM-EG; Sun, 12 May 2024 02:16:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720484.1123511; Sun, 12 May 2024 02:16:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s5ylO-0005dF-AI; Sun, 12 May 2024 02:16:34 +0000
Received: by outflank-mailman (input) for mailman id 720484;
 Sun, 12 May 2024 02:16:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UWbV=MP=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s5ylN-0005d9-Au
 for xen-devel@lists.xenproject.org; Sun, 12 May 2024 02:16:33 +0000
Received: from wfhigh2-smtp.messagingengine.com
 (wfhigh2-smtp.messagingengine.com [64.147.123.153])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a2ad942b-1005-11ef-909d-e314d9c70b13;
 Sun, 12 May 2024 04:16:30 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfhigh.west.internal (Postfix) with ESMTP id 110D51800154
 for <xen-devel@lists.xenproject.org>; Sat, 11 May 2024 22:16:25 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Sat, 11 May 2024 22:16:25 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA for
 <xen-devel@lists.xenproject.org>; Sat, 11 May 2024 22:16:23 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2ad942b-1005-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:content-type:content-type:date:date
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to; s=fm3; t=1715480184; x=1715566584; bh=BGbSvIgpus
	qPh4LxVAhAAPKBQ9harTdkfHymlMQv7vQ=; b=2ZY0YASdmxee2YuWIcHO1h5wc8
	rJla1BjcSJYgGmPTZZ4kAtMjFfWtCWLkkdyvKTjk/N63HlLfsgf2R2aMKVsrKk0v
	R6TO2YYBCo2alpezppT74DQY7WNKH2OhkKxsGbqHu3Ytyrj5ew2ke1F6NP4Nj66T
	iBNgH9aC0yFmyIjYQkCrA8VytH6fRXNHdE4t/mJKRLGzQPU60ngLi+xMnMJycJ6v
	zIUsQIbsWVkMQ00I/+v5BwVqdv10KI/lihNplGYpTH7JKR6PGr58qsV4y9IUjc5c
	G2bS2cOseDDisA+H0XSr3U/vnh2cva/XfpNTB3kl1rKcRo0PtHH+BarMKyXg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:message-id
	:mime-version:reply-to:subject:subject:to:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
	1715480184; x=1715566584; bh=BGbSvIgpusqPh4LxVAhAAPKBQ9harTdkfHy
	mlMQv7vQ=; b=Z9HfXRMBoXxjvxXXiAyUXB6IjmKEAiQQZKPfGV+jW2Fw0HgmZkP
	6jJ2e1sxoxjqu/7vd518nQolxzRQDMXGm1xLNeIeyQE5x82o5kR4s04oL2+lM9R4
	bA5lfZWdXNZfn2KVzZbicsLbJ4u3eyDAmHc8vZUk+qeJJfLL0IdtIQYjrcb7UrYL
	CgKDh+zjAPYjVrLhLZ7KL4SvydEyFBi+93gIGC6X1ik5+vpHSrUKl3nYnPtLqoeF
	RxmK/1r/vOVnDVO+EIlhlUZmYHdZVLt/SU+v+abmIDKp6RTNO+m9c/TyDPrBAkQZ
	Hmyei65zTxhEwweOjnEOpa7cRieKsBcCRiQ==
X-ME-Sender: <xms:eCZAZoo9dQu48THlg7L5Mq8QFMb3gDMpoKba-YQhtSxZQxFjIiZoRg>
    <xme:eCZAZupLY-WUee0w6b6FM5nAt46PI6r2YiYo5MJUKUpTSVmGd73JJuCJwk0L2TB8s
    aB8N_dX0I4KxQ>
X-ME-Received: <xmr:eCZAZtPmk8DhW3zo7h6wxyJP5KXxE-acpm4KeOk0OnwXNESfT1aG1H-W0Y7UpJIsV5ZFBFC9pt4uP7Aad0VP3DFVFDB15Gm2Xw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeguddgheeiucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesghdtreertd
    dtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhi
    uceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqne
    cuggftrfgrthhtvghrnhepteduteeiudevkeegvefhtdekhfelgffhhedukedvvdeuuddv
    jeehvddtieehudfgnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvg
    hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhn
    vhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:eCZAZv5FwulNRyhWrZQJ0YMCKB-YM8rYX-1BZjN88XKBPov6YqGHVA>
    <xmx:eCZAZn7FBRmfoHTdjQcHyEG7sgjKZaLeTknVFmEB6P9Z_mYknwOLCQ>
    <xmx:eCZAZvjm8quJUoskxxkahEEWJJXN7PhEDbtFUCMfpZrWrmGC9RNFmg>
    <xmx:eCZAZh65LOHE51jhuSgmX-L9X5TcNjyAzd2kJrbaIy_VIB5JmT8Wzg>
    <xmx:eCZAZkjrOUGPpLTdPYRJQBksJnQfPTZFT9UraI0iNusxKJyIC3UHFL8N>
Feedback-ID: i1568416f:Fastmail
Date: Sun, 12 May 2024 04:16:21 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel <xen-devel@lists.xenproject.org>
Subject: Linux HVM fails to start with PANIC: early exception 0x00 IP
 0010:clear_page_orig+0x12/0x40 error 0
Message-ID: <ZkAmdiHI-DfUMiOV@mail-itl>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="J6SyACfGTN/1W6hE"
Content-Disposition: inline


--J6SyACfGTN/1W6hE
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sun, 12 May 2024 04:16:21 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel <xen-devel@lists.xenproject.org>
Subject: Linux HVM fails to start with PANIC: early exception 0x00 IP
 0010:clear_page_orig+0x12/0x40 error 0

Hi,

I've got a report[1] that after some update Linux HVM fails to start with t=
he
error as in the subject. It looks to be caused by some change between
Xen 4.17.3 and 4.17.4. Here the failure is on Linux 6.6.25 (both dom0
and domU), but the 6.1.62 that worked with older Xen before, now fails
too. The full error (logged via earlyprintk=3Dxen) is:

    [    0.009500] Using GB pages for direct mapping
    PANIC: early exception 0x00 IP 10:ffffffffb01c32e2 error 0 cr2 0xffffa0=
8649801000
    [    0.009606] CPU: 0 PID: 0 Comm: swapper Not tainted 6.6.25-1.qubes.f=
c37.x86_64 #1
    [    0.009665] Hardware name: Xen HVM domU, BIOS 4.17.4 04/26/2024
    [    0.009710] RIP: 0010:clear_page_orig+0x12/0x40
    [    0.009766] Code: 84 00 00 00 00 00 66 90 90 90 90 90 90 90 90 90 90=
 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 b9 40 00 00 00 0f 1f 44 00 00 ff c9=
 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 47
    [    0.009862] RSP: 0000:ffffffffb0e03d58 EFLAGS: 00010016 ORIG_RAX: 00=
00000000000000
    [    0.009915] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000=
00000003f
    [    0.009967] RDX: 0000000000009801 RSI: 0000000000000000 RDI: ffffa08=
649801000
    [    0.010015] RBP: 0000000000000001 R08: 0000000000000001 R09: 6b7f283=
562d74b16
    [    0.010063] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000=
000000001
    [    0.010112] R13: 0000000000000000 R14: ffffffffb0e22a08 R15: ffffa08=
640000000
    [    0.010161] FS:  0000000000000000(0000) GS:ffffffffb16ea000(0000) kn=
lGS:0000000000000000
    [    0.010214] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [    0.010257] CR2: ffffa08649801000 CR3: 0000000008e80000 CR4: 0000000=
0000000b0
    [    0.010310] Call Trace:
    [    0.010341]  <TASK>
    [    0.010372]  ? early_fixup_exception+0xf7/0x190
    [    0.010416]  ? early_idt_handler_common+0x2f/0x3a
    [    0.010460]  ? clear_page_orig+0x12/0x40
    [    0.010501]  ? alloc_low_pages+0xeb/0x150
    [    0.010541]  ? __kernel_physical_mapping_init+0x1d2/0x630
    [    0.010588]  ? init_memory_mapping+0x83/0x160
    [    0.010631]  ? init_mem_mapping+0x9a/0x460
    [    0.010669]  ? memblock_reserve+0x6d/0xf0
    [    0.010709]  ? setup_arch+0x796/0xf90
    [    0.010748]  ? start_kernel+0x63/0x420
    [    0.010787]  ? x86_64_start_reservations+0x18/0x30
    [    0.010828]  ? x86_64_start_kernel+0x96/0xa0
    [    0.010868]  ? secondary_startup_64_no_verify+0x18f/0x19b
    [    0.010918]  </TASK>

I'm pretty sure the exception 0 is misleading here, I don't see how it
could be #DE.

More logs (including full hypervisor log) are attached to the linked
issue.

This is on HP 240 g7, and my educated guess is it's Intel Celeron N4020
CPU. I cannot reproduce the issue on different hardware.

PVH domains seems to work.

Any ideas what could have happened here?

[1] https://github.com/QubesOS/qubes-issues/issues/9217

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--J6SyACfGTN/1W6hE
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZAJnYACgkQ24/THMrX
1yxdpwf/bGl6gTGNA66alI8MrlzcQQm4jb9JZFQRY9yKJITI7/RbhJOv13tr4TaC
pv9udiaMzYkflGkUEC/9yyUq2EGV9G+6yK2GQ7e9gZYL7642pBiabx5OjNkLk8eo
yOsKUPe6HAYF0BBEURLL8PMrBua/+LN5qMI4XfrenmPzUdwhOcJ8DO1hfU/ueO68
Rjt9zSG4N1L8TRmXy0zxc1uxiIZpz5nzCGCLRJNbo1X36CZ7wQd2VW0mIakysSJk
/j49P1F24zNWD9R5mxM9NZFwVwvxQhpqjkeR2ieXRea/rDuNBvkVxPHpWzpUsrAe
chKFaq+3pOZCC30Hztg79uZ97EKUoA==
=9pf0
-----END PGP SIGNATURE-----

--J6SyACfGTN/1W6hE--


From xen-devel-bounces@lists.xenproject.org Sun May 12 09:01:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 09:01:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720527.1123521 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s654W-00037r-Mk; Sun, 12 May 2024 09:00:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720527.1123521; Sun, 12 May 2024 09:00:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s654W-00037k-JV; Sun, 12 May 2024 09:00:44 +0000
Received: by outflank-mailman (input) for mailman id 720527;
 Sun, 12 May 2024 09:00:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s654V-00037Z-Hp; Sun, 12 May 2024 09:00:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s654V-0006PI-E6; Sun, 12 May 2024 09:00:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s654U-0001YM-V4; Sun, 12 May 2024 09:00:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s654U-0002Gw-Ub; Sun, 12 May 2024 09:00:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=la1bUyE7I1L5SZOm/48/SVie3dLFT1CciItqF3ua11M=; b=xask7QRTANBvl8vl/FjkUU4LDU
	kAo0VfcBrYjit58VYJp10N2oMuWJhTT5FHa2Z9IVDsKNkK/QzXsGgQjGCS5SnYF53yxru9mfWSJJm
	77CqsbxvBPStsAi0ycqCvArgO+7K1BQmOcHeCBrckoPovIDYg3VxJeP+dLou+QBVcqOA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185983-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185983: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 May 2024 09:00:42 +0000

flight 185983 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185983/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-raw       8 xen-boot                   fail pass in 185980
 test-armhf-armhf-libvirt      8 xen-boot                   fail pass in 185980

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185980 blocked in 185983
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185980 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 185980 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 185980 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185980
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185980
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185980
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185980
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185980
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   185983  2024-05-12 01:55:09 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun May 12 13:52:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 13:52:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720573.1123531 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s69c8-0002hh-Os; Sun, 12 May 2024 13:51:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720573.1123531; Sun, 12 May 2024 13:51:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s69c8-0002ha-M1; Sun, 12 May 2024 13:51:44 +0000
Received: by outflank-mailman (input) for mailman id 720573;
 Sun, 12 May 2024 13:51:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=or6S=MP=bugseng.com=alessandro.zucchelli@srs-se1.protection.inumbo.net>)
 id 1s69c8-0002hU-5X
 for xen-devel@lists.xenproject.org; Sun, 12 May 2024 13:51:44 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c2e84fd6-1066-11ef-909d-e314d9c70b13;
 Sun, 12 May 2024 15:51:43 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 1CADD4EE073E;
 Sun, 12 May 2024 15:51:41 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2e84fd6-1066-11ef-909d-e314d9c70b13
MIME-Version: 1.0
Date: Sun, 12 May 2024 15:51:41 +0200
From: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, Stefano
 Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu
 <ppircalabu@bitdefender.com>
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of
 mem_access.c
Reply-To: alessandro.zucchelli@bugseng.com
Mail-Reply-To: alessandro.zucchelli@bugseng.com
In-Reply-To: <d659e73f-762a-48c9-8936-94bbc771218c@xen.org>
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
 <d659e73f-762a-48c9-8936-94bbc771218c@xen.org>
Message-ID: <0857b27d0e832a281e0bddb87d51d16a@bugseng.com>
X-Sender: alessandro.zucchelli@bugseng.com
Organization: BUGSENG Srl
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-10 22:59, Julien Grall wrote:
> Hi,
> 
> On 10/05/2024 13:32, Alessandro Zucchelli wrote:
>> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
>> changes are done:
>> revert preprocessor conditional changes to xen/mem_access.h which
>> had it build unconditionally, add conditional build for 
>> xen/mem_access.c
> 
> I am afraid, I don't understand this one as you don't seem to modify 
> xen/mem_access.h. Is this meant to be part of the changelog?
> 
> You also don't seem to mention the change in Makefile. This is the one 
> I was asking for in the previous version. So what about:
> 
> "xen/arm: mem_access: Conditionally compile mem_access.c
> 
> Commit 634cfc8beb ("Make MEM_ACCESS configurable") intended to make 
> MEM_ACCESS configurable on Arm to reduce the code size when the user 
> doesn't need it.
> 
> However, this didn't cover the arch specific code. None of the code in 
> arm/mem_access.c is necessary when MEM_ACCESS=n, so it can be compiled 
> out. This will require to provide some stub for functions called by the 
> common code.
> 
> This is also fixing violation of the MISRA C:2012 Rule 8.4 reported by 
> ECLAIR.
> "
> 
> The patch itself loks good so once we agree on the commit message, then 
> I am happy to update it on commit.

Hi,
Thanks for the feedback,
> 
> I am afraid, I don't understand this one as you don't seem to modify 
> xen/mem_access.h. Is this meant to be part of the changelog?
> 
you are right, this should be part of the changelog as it referes to the 
revert of a previous patch's changes.
I approve of the commit message you provided.

Cheers,

-- 
Alessandro Zucchelli, B.Sc.

Software Engineer, BUGSENG (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Sun May 12 14:48:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 14:48:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720582.1123541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6AUk-0000eJ-Pq; Sun, 12 May 2024 14:48:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720582.1123541; Sun, 12 May 2024 14:48:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6AUk-0000eC-Ma; Sun, 12 May 2024 14:48:10 +0000
Received: by outflank-mailman (input) for mailman id 720582;
 Sun, 12 May 2024 14:48:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZUm/=MP=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6AUj-0000e6-OU
 for xen-devel@lists.xenproject.org; Sun, 12 May 2024 14:48:09 +0000
Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com
 [2607:f8b0:4864:20::835])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a549f764-106e-11ef-909d-e314d9c70b13;
 Sun, 12 May 2024 16:48:08 +0200 (CEST)
Received: by mail-qt1-x835.google.com with SMTP id
 d75a77b69052e-43cff222427so17277071cf.3
 for <xen-devel@lists.xenproject.org>; Sun, 12 May 2024 07:48:08 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54da2d3sm44613731cf.27.2024.05.12.07.48.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 12 May 2024 07:48:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a549f764-106e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715525287; x=1716130087; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=lOImqBUuSz91DharMd8kZr0C/2tEUjAPixNxKnsR+u4=;
        b=ohXKn84XJ6J/53DLDVBIPswoDcI9i14hMbLvJgAg/PXvFdhfOXFvMA5HQvF7KbRrsG
         IAmZOqZd2/4UlyBobE0o3XgeH+56yTONbpSbo1VsX6zATA0TQvGIC204pjUf8oiTS98A
         HhlRaBxXCnLdWLMaqaqdiC/RBCsAav/7rW1ps=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715525287; x=1716130087;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=lOImqBUuSz91DharMd8kZr0C/2tEUjAPixNxKnsR+u4=;
        b=HYiTtfAld2HrHVWhEDLYp34UUEpluXCHo8kmF5mMM27R0Ah6DIgSxKrMRSRzGC4TXz
         MELcFQR0mfA3J559F77NM3p5VvT0xhV5NC1sKplL4t1JkHEP19wP1FQBfTK/+p41oC2/
         jx9XqNKp+7rvRAOsZefYIBC/7hpNQDMPcPM1FAxoUPVgJQ880X4dlCsycVJ/eiy3Sdhv
         fbCizbrz7sPn7mwuyrzuLhN42SgYDsJQG+jJ0u/5jBkgp4fI/9rgVg8nxrn18WO1uLAm
         Yk5NmwMjRwuI8BW9jWOAIZTX9MtEabp1duTJ5e22DIwM+DDuEfkbqIhVln6NCx5m+d5l
         7gxA==
X-Forwarded-Encrypted: i=1; AJvYcCU5tmmW4M+qW83pkUAI1z8gb+FkUFkBdCOmqCZjAmYsygm9fnNX4m2ca61ZK5ClMtKNscMM5alCwTA0nTGQPyhqup8OHpwgIaZ4OIylOUc=
X-Gm-Message-State: AOJu0YwgOsyt0w4oCbpk7LDAYSZ78Tw49vkfkYNMa3VuJZDbzNDRy0Ds
	NNXVMzDFCJRJNGFijoAJLyStbbYrTvc5OlJPHHylwfbc8qynSiSNq4tuYm7TvR1yWQ83NSwlnge
	u
X-Google-Smtp-Source: AGHT+IFASHmDzLJCUOJgho0HwTJCmjxbHz5ePKcnzkVo6AhZ/gu7NnO4zccBUPzWweeP8Welpl+Esg==
X-Received: by 2002:a05:622a:447:b0:43a:bec1:37c with SMTP id d75a77b69052e-43dfdd4b54bmr114718271cf.66.1715525287211;
        Sun, 12 May 2024 07:48:07 -0700 (PDT)
Message-ID: <653ba4f5-dc1e-48b9-9833-35b47bfc5fc4@citrix.com>
Date: Sun, 12 May 2024 15:48:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Linux HVM fails to start with PANIC: early exception 0x00 IP
 0010:clear_page_orig+0x12/0x40 error 0
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel <xen-devel@lists.xenproject.org>
References: <ZkAmdiHI-DfUMiOV@mail-itl>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkAmdiHI-DfUMiOV@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12/05/2024 3:16 am, Marek Marczykowski-Górecki wrote:
> Hi,
>
> I've got a report[1] that after some update Linux HVM fails to start with the
> error as in the subject. It looks to be caused by some change between
> Xen 4.17.3 and 4.17.4. Here the failure is on Linux 6.6.25 (both dom0
> and domU), but the 6.1.62 that worked with older Xen before, now fails
> too. The full error (logged via earlyprintk=xen) is:
>
>     [    0.009500] Using GB pages for direct mapping
>     PANIC: early exception 0x00 IP 10:ffffffffb01c32e2 error 0 cr2 0xffffa08649801000
>     [    0.009606] CPU: 0 PID: 0 Comm: swapper Not tainted 6.6.25-1.qubes.fc37.x86_64 #1
>     [    0.009665] Hardware name: Xen HVM domU, BIOS 4.17.4 04/26/2024
>     [    0.009710] RIP: 0010:clear_page_orig+0x12/0x40
>     [    0.009766] Code: 84 00 00 00 00 00 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 b9 40 00 00 00 0f 1f 44 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 47
>     [    0.009862] RSP: 0000:ffffffffb0e03d58 EFLAGS: 00010016 ORIG_RAX: 0000000000000000
>     [    0.009915] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000003f
>     [    0.009967] RDX: 0000000000009801 RSI: 0000000000000000 RDI: ffffa08649801000
>     [    0.010015] RBP: 0000000000000001 R08: 0000000000000001 R09: 6b7f283562d74b16
>     [    0.010063] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
>     [    0.010112] R13: 0000000000000000 R14: ffffffffb0e22a08 R15: ffffa08640000000
>     [    0.010161] FS:  0000000000000000(0000) GS:ffffffffb16ea000(0000) knlGS:0000000000000000
>     [    0.010214] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>     [    0.010257] CR2: ffffa08649801000 CR3: 0000000008e80000 CR4: 00000000000000b0
>     [    0.010310] Call Trace:
>     [    0.010341]  <TASK>
>     [    0.010372]  ? early_fixup_exception+0xf7/0x190
>     [    0.010416]  ? early_idt_handler_common+0x2f/0x3a
>     [    0.010460]  ? clear_page_orig+0x12/0x40
>     [    0.010501]  ? alloc_low_pages+0xeb/0x150
>     [    0.010541]  ? __kernel_physical_mapping_init+0x1d2/0x630
>     [    0.010588]  ? init_memory_mapping+0x83/0x160
>     [    0.010631]  ? init_mem_mapping+0x9a/0x460
>     [    0.010669]  ? memblock_reserve+0x6d/0xf0
>     [    0.010709]  ? setup_arch+0x796/0xf90
>     [    0.010748]  ? start_kernel+0x63/0x420
>     [    0.010787]  ? x86_64_start_reservations+0x18/0x30
>     [    0.010828]  ? x86_64_start_kernel+0x96/0xa0
>     [    0.010868]  ? secondary_startup_64_no_verify+0x18f/0x19b
>     [    0.010918]  </TASK>
>
> I'm pretty sure the exception 0 is misleading here, I don't see how it
> could be #DE.
>
> More logs (including full hypervisor log) are attached to the linked
> issue.
>
> This is on HP 240 g7, and my educated guess is it's Intel Celeron N4020
> CPU. I cannot reproduce the issue on different hardware.
>
> PVH domains seems to work.
>
> Any ideas what could have happened here?

Yes.

Revert the microcode back to revision 0x38 for now.

~Andrew


From xen-devel-bounces@lists.xenproject.org Sun May 12 15:46:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 15:46:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720590.1123550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6BOY-0008CH-T6; Sun, 12 May 2024 15:45:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720590.1123550; Sun, 12 May 2024 15:45:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6BOY-0008CA-QS; Sun, 12 May 2024 15:45:50 +0000
Received: by outflank-mailman (input) for mailman id 720590;
 Sun, 12 May 2024 15:45:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fAR6=MP=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s6BOX-0008C2-Ho
 for xen-devel@lists.xenproject.org; Sun, 12 May 2024 15:45:49 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b3623ce7-1076-11ef-b4bb-af5377834399;
 Sun, 12 May 2024 17:45:47 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a599eedc8eeso870232366b.1
 for <xen-devel@lists.xenproject.org>; Sun, 12 May 2024 08:45:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3623ce7-1076-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715528747; x=1716133547; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VKmmFPcTG26A6s5ZXSLIEmZvIAaQiyyqS4DPU6yGGTc=;
        b=HM2I2aJvNQMlNn3RaymTxQW3dmbO3Kw1AugKiuZFY11D5pVnmNu+5d/g9f2ZcGAk0u
         kBKJiUl9PGhrA8b1ha9xqbYDBdzDeYRVKDYE9v2hOjkl/zXMcrpB6qgbLIE1pM+mNyv3
         pbYEMqFvMwSddWNb1fdZs5AboTthyqwRRogKN3teUGYT6Wu+Cplzec4JybtyvLOQq7+l
         ijP37g1dgIDrGFf/EzZLR8VnngXvQrHlwyqSUeu1/OvwKF1MHM66HC+BlrQL9WwLIX5W
         EKCNVJt7rDt7Wzazu3X26AUsY+BbF054TCq3OVVr3L2wt4fVtoJExUwvDJqZ17U+12eo
         QJZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715528747; x=1716133547;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VKmmFPcTG26A6s5ZXSLIEmZvIAaQiyyqS4DPU6yGGTc=;
        b=BLulC396m1BfFwJFLOm9zD8km54zt5Ou44Haw9CiDIGWmTmOrwk2FdcxO0z8X9jHyZ
         S9l0r598zoITRX+ffWkuWzkkn3an08MEX9guWlMgtyFSLh2NCXPqxvJs11eSMZk+uUj9
         LGAkHLlpQHngJ0BefjkVgFr5LLCZ102hGm65HNPTZ4ndiQ7sTU61D4UaP/7cOqhDyMep
         MIM0bkkD4f6ymKFTK4ApR92tFOyEp7zy2Djbujb+zgyBQZVC1O2kJm4J/KEof2mhHfJO
         LitvSIvZiLQ1rYmL5EsxG0tCqELgO1CVZyHmmdq22YiuTXlhTlH0B9CsUT+bQnE/pJf5
         D1Ag==
X-Gm-Message-State: AOJu0YzWKUPWzPjRJkHeewUEn0Nd8pT0UxDBa3IwdNZcxAZ055wDPmB/
	Sf7fgPV6seIts/N3FDKaM8MiLFdTQV/HGmUSuWjVMWupT6beclq3edW+c+PB1j8Koh8na+qwMhN
	f2H6C7FD9oGMDjEuGPnq6Ad37Y5g=
X-Google-Smtp-Source: AGHT+IF6oBvJFCg8ByuGE3oCtb80sRB3w6NPDaCr2YObwkOUZFXc3DeGOBWmK+O5JR44ShgLDmRHmP8JG4lcFERthv8=
X-Received: by 2002:a50:bb05:0:b0:572:5f28:1f25 with SMTP id
 4fb4d7f45d1cf-5734d5c1692mr5465543a12.7.1715528746560; Sun, 12 May 2024
 08:45:46 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <20240508213824.55824-2-leigh@solinno.co.uk>
In-Reply-To: <20240508213824.55824-2-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 12 May 2024 11:45:34 -0400
Message-ID: <CAKf6xptmD79GaPrEB5-Z4p1zr-0atQUoE=1UJEv9MOBg9z-=Fw@mail.gmail.com>
Subject: Re: [RFC PATCH v2 1/5] tools/libs/light: Add vlan field to libxl_device_nic
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
	anthony.perard@citrix.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 7:43=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> wr=
ote:
>
> Add `vlan' string field to libxl_device_nic, to allow a VLAN
> configuration to be specified for the VIF when adding it to the
> bridge device.
>
> Update libxl_nic.c to read and write the vlan field from the
> xenstore.
>
> This provides the capability for supported operating systems (e.g.
> Linux) to perform VLAN filtering on bridge ports.  The Xen
> hotplug scripts need to be updated to read this information from
> the xenstore and perform the required configuration.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

I think vlan is a better name than vid for the field.

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Sun May 12 15:46:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 15:46:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720591.1123560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6BOl-0008TX-4D; Sun, 12 May 2024 15:46:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720591.1123560; Sun, 12 May 2024 15:46:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6BOl-0008TQ-1g; Sun, 12 May 2024 15:46:03 +0000
Received: by outflank-mailman (input) for mailman id 720591;
 Sun, 12 May 2024 15:46:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fAR6=MP=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s6BOk-0008C2-0o
 for xen-devel@lists.xenproject.org; Sun, 12 May 2024 15:46:02 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bb27bd4f-1076-11ef-b4bb-af5377834399;
 Sun, 12 May 2024 17:46:00 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so113881566b.0
 for <xen-devel@lists.xenproject.org>; Sun, 12 May 2024 08:46:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb27bd4f-1076-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715528760; x=1716133560; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XbTbRtgUiP72GkT6/YqDaFY9yg+GfdxSuJV/XCZAyTA=;
        b=TqwJgGYqwG8IN6HuamjODh4ggvI3eNaDWLfTBXdYCeaoTalTY2v+FW0RHBdkrUTOMS
         /hIVVtzDEHiGHIf9F96WfwRIOVaOmBUGwEEH4o/0MLxfgZVRPkCfDmqxBqrbMG53mRii
         xrl9gH1b6tSY8IvS0NpD6+9o2jK59UoEmiEFQGhnoIHS15aGulX5yFSKac1Iki55Xu94
         1Dz+5dNfTjFsC2JEldW+nXEI75jErqbxO9XSNn1wf0yPeCvIZwTGPjH6zFU3hofmMVBN
         l0J7gF9FZjO1jC7vCkBG4iS+Kr0Utxtd4x/fxHtT92lmwB/iMH6GYc862wzRjohKcyd0
         k2VQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715528760; x=1716133560;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XbTbRtgUiP72GkT6/YqDaFY9yg+GfdxSuJV/XCZAyTA=;
        b=bV23iVFv1ojvRTsFp5MGOndUK3EwvYWVJA6AbkSzDSAZGMf11P2YA9PoVcTiCr4h8A
         4jVLHXV1rHS7Qa3WBTc/b1u3sI1QR+027sZ/8bomwsDEI+N3EiWR+9/vTMCobGWafllq
         g9Gjb5D7wmhbCbZtRiUcBxo8AWJbKSUI9jblPvL8XbM6ZqyKFAIT0l5HL+y0MFtw71Sa
         sDRRgZCmaw3cteParCkgTj/PsFJr1t8V2D8exktEueUzkX7fH9iru91MWvx79RHO2Ml2
         OD4aKPC7U2NScEYiDGy4zbEd/aPSZrXKG//qfvgU3JlqMNZiRE9PcYtN7ogDLoyHYYn1
         gSvA==
X-Gm-Message-State: AOJu0YyIiLYBrpy3FYwX1bsqBktz+rbSLdASoQ+Zb/n34k6M5x53Q19t
	8ffYjPvawp9W+0qHjPESbSxU7heiqvfFGevTu2tkCvzzdQTua5OjdtiW5OJiBMU7z1uIGFIHIwN
	/dM5a4TPtAiOLuEOFMzft3tbfs0zs/Q==
X-Google-Smtp-Source: AGHT+IFv/3YnW79HbfRjo6N82zFBOYCQARPGHsoAE1HtdGuBbT+Aj88sIRFCrVpH4cUu2Z0W1w6rsjMfQb4ilHtDfQE=
X-Received: by 2002:a50:8a95:0:b0:566:2f24:b063 with SMTP id
 4fb4d7f45d1cf-5734d5e4ba9mr5451332a12.23.1715528759881; Sun, 12 May 2024
 08:45:59 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <20240508213824.55824-3-leigh@solinno.co.uk>
In-Reply-To: <20240508213824.55824-3-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 12 May 2024 11:45:48 -0400
Message-ID: <CAKf6xpty-wAU36L4KAqbEB=BHQEF=gGs=dBOhB_BYkvOrBEA2w@mail.gmail.com>
Subject: Re: [RFC PATCH v2 2/5] tools/xl: add vlan keyword to vif option
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
	anthony.perard@citrix.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 6:13=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> wr=
ote:
>
> Update parse_nic_config() to support a new `vlan' keyword. This
> keyword specifies the VLAN configuration to assign to the VIF when
> attaching it to the bridge port, on operating systems that support
> the capability (e.g. Linux). The vlan keyword will allow one or
> more VLANs to be configured on the VIF when adding it to the bridge
> port. This will be done by the vif-bridge script and functions.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Sun May 12 23:33:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 May 2024 23:33:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720653.1123570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Igt-0003mN-JM; Sun, 12 May 2024 23:33:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720653.1123570; Sun, 12 May 2024 23:33:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Igt-0003mF-GM; Sun, 12 May 2024 23:33:15 +0000
Received: by outflank-mailman (input) for mailman id 720653;
 Sun, 12 May 2024 23:33:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6Igs-0003m5-Go; Sun, 12 May 2024 23:33:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6Igs-0005gH-Bv; Sun, 12 May 2024 23:33:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6Igr-0007rT-Vr; Sun, 12 May 2024 23:33:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6Igr-0001q6-VC; Sun, 12 May 2024 23:33:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LuwCgcVHHGYDHk7FFmeHqZuzPKirqq/IQ1Tjrytsn2c=; b=K8khiEf50lQRBfpFiX0Xhzedwz
	uHHjorSwakJa/Ib65SuoAcwwf+j3jB8UJ8s5rZ9uXlUpRtJ4QwD8hYuPBCYdtPk30yg7cC9yxdksF
	kieSX91ziVBGVwWU3reW1UTd53kPnFBLwJ9nx2YRKuAj6BWPerUEuTgOUyv00WOsdJhg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185984-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185984: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:guest-start/freebsd.repeat:fail:regression
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:host-ping-check-native:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ba16c1cf11c9f264b5455cb7d57267b39925409a
X-Osstest-Versions-That:
    linux=cf87f46fd34d6c19283d9625a7822f20d90b64a4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 May 2024 23:33:13 +0000

flight 185984 linux-linus real [real]
flight 185985 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185984/
http://logs.test-lab.xenproject.org/osstest/logs/185985/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-qemuu-freebsd12-amd64 21 guest-start/freebsd.repeat fail in 185985 REGR. vs. 185982
 test-armhf-armhf-xl           8 xen-boot       fail in 185985 REGR. vs. 185982

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 19 guest-localmigrate/x10 fail pass in 185985-retest
 test-armhf-armhf-xl        6 host-ping-check-native fail pass in 185985-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 185982

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 185977
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185977
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185982
 test-armhf-armhf-xl-multivcpu  8 xen-boot                     fail like 185982
 test-armhf-armhf-examine      8 reboot                       fail  like 185982
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185982
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                ba16c1cf11c9f264b5455cb7d57267b39925409a
baseline version:
 linux                cf87f46fd34d6c19283d9625a7822f20d90b64a4

Last test of basis   185982  2024-05-11 16:13:39 Z    1 days
Testing same since   185984  2024-05-12 16:44:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Borislav Petkov (AMD) <bp@alien8.de>
  Linus Torvalds <torvalds@linux-foundation.org>
  Serge Semin <fancer.lancer@gmail.com>
  Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
  Thomas Gleixner <tglx@linutronix.de>
  Yuezhang Mo <Yuezhang.Mo@sony.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit ba16c1cf11c9f264b5455cb7d57267b39925409a
Merge: 775a0eca3357 591c946675d8
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Sun May 12 09:09:27 2024 -0700

    Merge tag 'edac_urgent_for_v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras
    
    Pull EDAC fix from Borislav Petkov:
    
     - Fix a race condition when clearing error count bits and toggling the
       error interrupt throug the same register, in synopsys_edac
    
    * tag 'edac_urgent_for_v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras:
      EDAC/synopsys: Fix ECC status and IRQ control race condition

commit 775a0eca3357d79311c0225458f8fe90791a8857
Merge: cf87f46fd34d 5754ace3c319
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Sun May 12 08:54:28 2024 -0700

    Merge tag 'x86_urgent_for_v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
    
    Pull x86 fixes from Borislav Petkov:
    
     - Add a new PCI ID which belongs to a new AMD CPU family 0x1a
    
     - Ensure that that last level cache ID is set in all cases, in the AMD
       CPU topology parsing code, in order to prevent invalid scheduling
       domain CPU masks
    
    * tag 'x86_urgent_for_v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
      x86/topology/amd: Ensure that LLC ID is initialized
      x86/amd_nb: Add new PCI IDs for AMD family 0x1a

commit 5754ace3c3199c162dcee1f3f87a538c46d1c832
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed May 8 21:53:47 2024 +0200

    x86/topology/amd: Ensure that LLC ID is initialized
    
    The original topology evaluation code initialized cpu_data::topo::llc_id
    with the die ID initialy and then eventually overwrite it with information
    gathered from a CPUID leaf.
    
    The conversion analysis failed to spot that particular detail and omitted
    this initial assignment under the assumption that each topology evaluation
    path will set it up. That assumption is mostly correct, but turns out to be
    wrong in case that the CPUID leaf 0x80000006 does not provide a LLC ID.
    
    In that case, LLC ID is invalid and as a consequence the setup of the
    scheduling domain CPU masks is incorrect which subsequently causes the
    scheduler core to complain about it during CPU hotplug:
    
      BUG: arch topology borken
           the CLS domain not a subset of the MC domain
    
    Cure it by reusing legacy_set_llc() and assigning the die ID if the LLC ID
    is invalid after all possible parsers have been tried.
    
    Fixes: f7fb3b2dd92c ("x86/cpu: Provide an AMD/HYGON specific topology parser")
    Reported-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
    Tested-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
    Link: https://lore.kernel.org/r/PUZPR04MB63168AC442C12627E827368581292@PUZPR04MB6316.apcprd04.prod.outlook.com

commit 0e640f0a47d8426eab1fb9c03f0af898dfe810b8
Author: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
Date:   Fri May 10 16:48:28 2024 +0530

    x86/amd_nb: Add new PCI IDs for AMD family 0x1a
    
    Add the new PCI Device IDs to the MISC IDs list to support new
    generation of AMD 1Ah family 70h Models of processors.
    
      [ bp: Massage commit message. ]
    
    Signed-off-by: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
    Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
    Link: https://lore.kernel.org/r/20240510111829.969501-1-Shyam-sundar.S-k@amd.com

commit 591c946675d88dcc0ae9ff54be9d5caaee8ce1e3
Author: Serge Semin <fancer.lancer@gmail.com>
Date:   Thu Feb 22 21:12:46 2024 +0300

    EDAC/synopsys: Fix ECC status and IRQ control race condition
    
    The race condition around the ECCCLR register access happens in the IRQ
    disable method called in the device remove() procedure and in the ECC IRQ
    handler:
    
      1. Enable IRQ:
         a. ECCCLR = EN_CE | EN_UE
      2. Disable IRQ:
         a. ECCCLR = 0
      3. IRQ handler:
         a. ECCCLR = CLR_CE | CLR_CE_CNT | CLR_CE | CLR_CE_CNT
         b. ECCCLR = 0
         c. ECCCLR = EN_CE | EN_UE
    
    So if the IRQ disabling procedure is called concurrently with the IRQ
    handler method the IRQ might be actually left enabled due to the
    statement 3c.
    
    The root cause of the problem is that ECCCLR register (which since
    v3.10a has been called as ECCCTL) has intermixed ECC status data clear
    flags and the IRQ enable/disable flags. Thus the IRQ disabling (clear EN
    flags) and handling (write 1 to clear ECC status data) procedures must
    be serialised around the ECCCTL register modification to prevent the
    race.
    
    So fix the problem described above by adding the spin-lock around the
    ECCCLR modifications and preventing the IRQ-handler from modifying the
    IRQs enable flags (there is no point in disabling the IRQ and then
    re-enabling it again within a single IRQ handler call, see the
    statements 3a/3b and 3c above).
    
    Fixes: f7824ded4149 ("EDAC/synopsys: Add support for version 3 of the Synopsys EDAC DDR")
    Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
    Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
    Link: https://lore.kernel.org/r/20240222181324.28242-2-fancer.lancer@gmail.com


From xen-devel-bounces@lists.xenproject.org Mon May 13 03:42:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 03:42:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720672.1123582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6MZv-0007I3-Oq; Mon, 13 May 2024 03:42:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720672.1123582; Mon, 13 May 2024 03:42:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6MZv-0007Hw-Jw; Mon, 13 May 2024 03:42:19 +0000
Received: by outflank-mailman (input) for mailman id 720672;
 Mon, 13 May 2024 03:42:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=A6re=MQ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s6MZu-0007Hq-Do
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 03:42:18 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ca150923-10da-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 05:42:16 +0200 (CEST)
Received: from CH0PR03CA0211.namprd03.prod.outlook.com (2603:10b6:610:e7::6)
 by SA3PR12MB8023.namprd12.prod.outlook.com (2603:10b6:806:320::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Mon, 13 May
 2024 03:42:13 +0000
Received: from DS2PEPF00003448.namprd04.prod.outlook.com
 (2603:10b6:610:e7:cafe::29) by CH0PR03CA0211.outlook.office365.com
 (2603:10b6:610:e7::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Mon, 13 May 2024 03:42:13 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS2PEPF00003448.mail.protection.outlook.com (10.167.17.75) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Mon, 13 May 2024 03:42:12 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 12 May
 2024 22:42:05 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 12 May
 2024 22:42:05 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 12 May 2024 22:42:02 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca150923-10da-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CMySr3I2YBydBi7ozd2MjjHNtOfQm4n81gJYRRfWHZF6KKacl+UvjrZVk+xY1X11e1DbHCOVxO4we+Vs/lg5mlnuv5/UzK+8bciInx7X+IZxzDLfHKkogdvwOEbIbra94/HcbnzHKpMRzUFsv8YswdzSLo9tsF1mPn/bwEu1Gz/VpWWjaDPiKLzhtP6vVadYzM5dcF2560dpHk86RnVKnj31pNgIJYIoKI19LUs76InpaO5Hgz08+FyzikO5Pz0bWimcFyi4vudEuFqro4ZijH3FLzHTyFZEcq8dYfybmw6ec6YaMph7QTslTysbQGPNscqqL/ZfOgUnWQN3jk5NZg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=gpVhZr3pJ/2KnxJu5+W/8YpRxSFTY/qNwTc7kC3HGOs=;
 b=V68Ml4Rgdg99TL+hEgzoH5DjxbvYJ5bFSdw+VSiJfaL7x+Bm60Z1ijaMyeivU2SG+D0w+UK+3x2tc25WBmqu4KIsB5VtrS5Xjv0ggeME4EJg/jJZqLn8MjpS1ONURhnd0MXX8TW2Ao/+CXUFy5b92WoEn6+X/UPS20qAqHo4LbOrMvF/hw3e8oC8vN+9u9NJ2yvvZu59/OXto/Ns5HRbZCORtD8NAPgIXxSBHV4dAgqKtE9sHzt9GEJj8QulYw+z6ledn8cfSKxGgbF6sVjnmmi4fHST/mRLUlJldVU1VXtGd3cAApyqP6wM/0qPBhY6CtM7LIZ5E0pLHsFkFj7JYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gpVhZr3pJ/2KnxJu5+W/8YpRxSFTY/qNwTc7kC3HGOs=;
 b=fRE/IA9cVNzuU/vfhBs4PQc2Rzf67hxx76RyUXTxuC5FA8JxFmQYozvPl1EDwAVQivLjuukdK47s/mGgNKgaVUXmAcaq+C2em+bY9LMHbyNEezV1OW/CeF4OO2FzYEqXvcamra8Kcg8QCpbaY0i2RyMiRU0ScFjjAKqV21owZ4s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <fad20165-ad8a-4bc4-be5f-e95061a57a2d@amd.com>
Date: Mon, 13 May 2024 11:42:01 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] xen/arm: Alloc hypervisor reserved pages as magic
 pages for Dom0less DomUs
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
References: <20240511005611.83125-1-xin.wang2@amd.com>
 <20240511005611.83125-2-xin.wang2@amd.com>
 <686ba256-f8bf-47e7-872f-d277bf7df0aa@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <686ba256-f8bf-47e7-872f-d277bf7df0aa@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS2PEPF00003448:EE_|SA3PR12MB8023:EE_
X-MS-Office365-Filtering-Correlation-Id: ccc9c830-ffa8-4b42-517f-08dc72feaccf
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VlU3S1FCRDdoY0E1Vlp0RldtT0ZQOGxHaE9lNFpJNHJBTXp1ZFVTa1FZbjhZ?=
 =?utf-8?B?L3hrOHZTYXdyT1hoQWRJQk5NZGhoSXBtR21QelhOVEx6ZHhaakpDb3dlcG01?=
 =?utf-8?B?OHBFbHA5RzRJUGlsZjU4OUJUVXNQeWFISVM1aE9hYzk5N1htRWI5Q01YeVBn?=
 =?utf-8?B?ZWFOMkhJTm80OENPb3BkSHZkSC9ZTVpSSjlRUGxTYWRabmc0bWMwM21MbXBQ?=
 =?utf-8?B?VUJCL3Uyckl5WkpNeUpsSzc3ckNId3B3d3RCMXVuSTNWbEZmMmVrK0ZrVzZT?=
 =?utf-8?B?YXFzSU5RSStwZXhyb0N0a2djWVZSVGN1d0xSV29IMzNzeWR6SENXTGZOdnpP?=
 =?utf-8?B?emxzNGJTRElhUk9JMmNOQnp2Qm5IWXZEMHd4RWNWVjFGREJ3TWZXMjRleTFp?=
 =?utf-8?B?NHNkaUdFM3QzQmFraXk1Qk5KdkZQM09vdVJKRG5qMCtWV1BYYWhML1R5a21U?=
 =?utf-8?B?c0RPVlBIbE12bENuL3NXb3N3QVN2OWFRejNTems5SnZTVnI4M1VnSHpQTTMr?=
 =?utf-8?B?bjR3QTNHc3hhNWwwYlFRT09WMEFua3V4aE56amVEV0w1ZExWVW5Nb201Q1Fs?=
 =?utf-8?B?VVM4QXdRRHlDSW5FOGdXcnZTTVBPRWFKVUllTUVvazlPT3B3czkyZTV2RERF?=
 =?utf-8?B?ZlByUXMrWWF2d0N1dDZzWW5TenhNNDRmM0NaVGpiSVc3cFJvdnlpd0Y0Qk1Q?=
 =?utf-8?B?WlVzbGhYOEJrSzB6SVhwWklhamg2UU50dU9Gc0VJUGZZZExYS1Y4M0NhR3lt?=
 =?utf-8?B?RHo5MzBaMG1rWGFmQ2UwRG9IeXUvcGF1dTVsZklURjdkdzhGNk5GeVdkOEtz?=
 =?utf-8?B?bFFhR09mV1RvYWNENUxzd08wWFVxSExtb1h1Rm1GK3BPeUR4a2VObkpjRWkw?=
 =?utf-8?B?R0VYa1dzS0gxTGFJcThCT2MyQ1d4TkxJL0kzY3AzZkRiRGc0Rkt5bjVTN0R3?=
 =?utf-8?B?bnN6dnA4L25QUEMyLzBYeHFobm1aMWh0dHRqM2d0bXRTeWE0N3IxcjVsbThK?=
 =?utf-8?B?VzZ0RllVQ05aUlpxR004dDRNMXdLWkk2Y3ZETGFENmVkVjhEbUl5OEJ3Y0FH?=
 =?utf-8?B?S2hDc1pSUFVrMXZ5SW9RQWZRVWlwdFFVOUt5VVZYVG04Q0lySzlTRnA0cDlu?=
 =?utf-8?B?YnloU1ZPZElBSEVNaldCNUpUU1ljd1VMeGRGR1poc0tuLzYyY1lORU8xQlVF?=
 =?utf-8?B?c0VPdE40YlR5UEtjTWxrSi9iL1BwT2JOVXpjZ0tjNDBGUFBGVTA4SFV1Zmhx?=
 =?utf-8?B?cHhURm42alpwVGRGUFNtNlphZHpFUWhjZDk5ckMrMVVMVVpUQ2NwaW41Zjkw?=
 =?utf-8?B?Z2Q1ZDhvNE43VllHcWhUdDlzdFFiTkh5cDNLM2NyaER5T0pzL1E4d3BROGZX?=
 =?utf-8?B?Uk5XWmNpR2JFdW13QTk1MVdrYWg1QVZ4Z0xjRW5UMjJiM1VMUngrN1VUSVJs?=
 =?utf-8?B?WVRTeEFBd2p3cGZVdHIwVkdZdTN4am1YY1Q3Q2VFakVMUjdIWFlxMWZSdktK?=
 =?utf-8?B?OWtQQnNiSUlobnRYaEpRcFdWclVqbTJpT2F3M2l2TFIwcXQ2VTVEWjBPRFhG?=
 =?utf-8?B?V2JSWnVQNWNzOWpUT2w5QXF6VythZG9YM1ZZcmRCSGxYTkdqblNoS3pnMDNK?=
 =?utf-8?B?MG0rY1VQRW1ERjR3U1pLS1VVQ0xzQ3J4NWNUU21YVVpCNk5aNXZCS3dXa1NS?=
 =?utf-8?B?NlErLzdJTEl2eDVqcjV0OU0zc3NDcGhHMnQ4UXZZSW0xUmE0QkozclZtTDlq?=
 =?utf-8?B?NXRUdG9HRTRYTjNFaFhkb09XTmQrSzVPZUl3SklDQ1RzV3hqejdQMUFXTzV5?=
 =?utf-8?B?dCt5SWRkK2h1VUNlSnRZeGUvNHFkZ004bnppSXliQ3NXN2JBcnQ3TmwyTnB6?=
 =?utf-8?Q?K0s2J7Ien6ZaO?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2024 03:42:12.5514
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ccc9c830-ffa8-4b42-517f-08dc72feaccf
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF00003448.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB8023

Hi Julien,

On 5/11/2024 7:03 PM, Julien Grall wrote:
> Hi Henry,
>
> On 11/05/2024 01:56, Henry Wang wrote:
>>   +static int __init alloc_magic_pages(struct domain *d)
>> +{
>> +    struct page_info *magic_pg;
>> +    mfn_t mfn;
>> +    gfn_t gfn;
>> +    int rc;
>> +
>> +    d->max_pages += NR_MAGIC_PAGES;
>> +    magic_pg = alloc_domheap_pages(d, 
>> get_order_from_pages(NR_MAGIC_PAGES), 0);
>> +    if ( magic_pg == NULL )
>> +        return -ENOMEM;
>> +
>> +    mfn = page_to_mfn(magic_pg);
>> +    if ( !is_domain_direct_mapped(d) )
>> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE);
>> +    else
>> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
>
> Summarizing the discussion we had on Matrix. Regions like the extend 
> area and shared memory may not be direct mapped. So unfortunately, I 
> think it is possible that the GFN could clash with one of those.
>
> At least in the shared memory case, the user can provide the address. 
> But as you use the domheap allocator, the address returned could 
> easily change if you tweak your setup.
>
> I am not entirely sure what's the best solution. We could ask the user 
> to provide the information for reserved region. But it feels like we 
> are exposing a bit too much to the user.
>
> So possibly we would want to use the same approach as extended 
> regions. Once we processed all the mappings, find some space for the 
> hypervisor regions.

One thing that I noticed when I re-visit the extended region finding 
code from the hypervisor side is:
When the domain is direct-mapped, when we find extended region for the 
domain, we either use find_unallocated_memory() or find_memory_holes(). 
It looks like the removal of shared memory regions in both functions 
uses the paddr parsed from the device tree to remove the regions, which 
indicates there is an assumption that when a domain is direct-mapped, 
the shared memory should also be direct-mapped. I might be wrong, but 
otherwise I don't think the extended region finding logic will carve out 
the correct shared memory region gpaddr range for guests.

So I think we are missing the documentation (and the corresponding 
checking when we parse the device tree) for above assumption for the 
static shared memory, i.e., when the domain is direct-mapped, the static 
shared memory should also be direct-mapped, and user should make sure 
this is satisfied in the device tree otherwise Xen should complain.

If we add this assumption and related checking code, I think your 
concern of clashing with static shared memory can be addressed. Do you 
agree?

Kind regards,
Henry

>
> Any other suggestions?
>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Mon May 13 07:31:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 07:31:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720690.1123591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Q9s-0000gz-Cq; Mon, 13 May 2024 07:31:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720690.1123591; Mon, 13 May 2024 07:31:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Q9s-0000gs-92; Mon, 13 May 2024 07:31:40 +0000
Received: by outflank-mailman (input) for mailman id 720690;
 Mon, 13 May 2024 07:31:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6Q9q-0000gk-Rb
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 07:31:38 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d3ff7a35-10fa-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 09:31:35 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59e4136010so1011377466b.3
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 00:31:35 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01652sm556288066b.167.2024.05.13.00.31.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 00:31:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3ff7a35-10fa-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715585495; x=1716190295; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=5FyUfslIyVZYh3axe0IevBunTjWFWu02v2aKCjG/MwU=;
        b=dAeWKWfB8ipRGbZbyogvYhPKfcIhH7MAog0GAliXVFKh+qoIpZy1TbcT5LZk+9+nPV
         3WyLETXlONfI7VR6OEwnFYQdhCNQ7EdmmZ2Z120MCr9vYZuPYjAznDijn3HXvcRHqPO5
         GDpe/phNyjQj3dK1KKCdlkxTAgE7D0kLzEiRE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715585495; x=1716190295;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=5FyUfslIyVZYh3axe0IevBunTjWFWu02v2aKCjG/MwU=;
        b=w//juKejhiVnhWiLoY2MMClJi1SxpnWm6+qisJyjfsaix43lLKiDrKgE5tHDUPoLfM
         XFMx1D6usRwBQhMCfu8CNiNWcHqfTy5Vo2Chu2aoZ17Bc1zO9d6UR3BHyq/ZXO4Dg4MM
         WJGOHKS4Viiizi+YIjJx0mh4xD4FFyuqrttjZBx1KbWDgKjc0jNo6y4eSGUcuMifjgPz
         yirhr4pPR7p2qk6a2LUcHdHuI+zw/C1l4/91MHDyxHg4a3od2vaa60S7++40yDP6rhXS
         lxjMv1cmeDQ8E/2xU8eYH21ZWUb3JqAmHaB/Sz7iJlvbbA5X6wJ+/V2eJfGn9ocQR8E1
         YR2A==
X-Gm-Message-State: AOJu0YwD2ejxmE/ttIotrfgcVwG3qCwhY368KA1DLZbzMcCWOjxB+229
	3E4RFVRarTx1zM3yho4E6e/hw7RrjqQZk6+HD80dJFHwxSowxTlB+tTci199ddFkZ3CaARcC5aZ
	C
X-Google-Smtp-Source: AGHT+IHXzUCG2m9XGa+75xFOgk22cGczkjqwtB5niAVmdOsCIPoC3etJh6sRI3KLWRmfkRADSZpV8w==
X-Received: by 2002:a17:906:15cd:b0:a59:c9ce:3386 with SMTP id a640c23a62f3a-a5a2d5cb0c1mr526162066b.33.1715585494324;
        Mon, 13 May 2024 00:31:34 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 v3] tools/xen-cpuid: switch to use cpu-policy defined names
Date: Mon, 13 May 2024 09:31:02 +0200
Message-ID: <20240513073102.56233-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Like it was done recently for libxl, switch to using the auto-generated feature
names by the processing of cpufeatureset.h, this allows removing the open-coded
feature names, and unifies the feature naming with libxl and the hypervisor.

Introduce a newly auto-generated array that contains the feature names indexed
at featureset bit position, otherwise using the existing INIT_FEATURE_NAMES
would require iterating over the array elements until a match with the expected
bit position is found.

Note that leaf names need to be kept, as the current auto-generated data
doesn't contain the leaf names.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes since v2:
 - Remove __maybe_unused definition from the emulator harness.

Changes since v1:
 - Modify gen-cpuid.py to generate an array of strings with the feature names.
 - Introduce and use __maybe_unused.
---
 tools/include/xen-tools/common-macros.h |   4 +
 tools/misc/xen-cpuid.c                  | 320 +++---------------------
 tools/tests/x86_emulator/x86-emulate.h  |   1 -
 xen/tools/gen-cpuid.py                  |  26 ++
 4 files changed, 68 insertions(+), 283 deletions(-)

diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
index 60912225cb7a..560528dbc638 100644
--- a/tools/include/xen-tools/common-macros.h
+++ b/tools/include/xen-tools/common-macros.h
@@ -83,6 +83,10 @@
 #define __packed __attribute__((__packed__))
 #endif
 
+#ifndef __maybe_unused
+# define __maybe_unused __attribute__((__unused__))
+#endif
+
 #define container_of(ptr, type, member) ({              \
     typeof(((type *)0)->member) *mptr__ = (ptr);        \
     (type *)((char *)mptr__ - offsetof(type, member));  \
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 8893547bebce..2a1ac0ee8326 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -12,282 +12,33 @@
 
 #include <xen-tools/common-macros.h>
 
-static uint32_t nr_features;
-
-static const char *const str_1d[32] =
-{
-    [ 0] = "fpu",  [ 1] = "vme",
-    [ 2] = "de",   [ 3] = "pse",
-    [ 4] = "tsc",  [ 5] = "msr",
-    [ 6] = "pae",  [ 7] = "mce",
-    [ 8] = "cx8",  [ 9] = "apic",
-    /* [10] */     [11] = "sysenter",
-    [12] = "mtrr", [13] = "pge",
-    [14] = "mca",  [15] = "cmov",
-    [16] = "pat",  [17] = "pse36",
-    [18] = "psn",  [19] = "clflush",
-    /* [20] */     [21] = "ds",
-    [22] = "acpi", [23] = "mmx",
-    [24] = "fxsr", [25] = "sse",
-    [26] = "sse2", [27] = "ss",
-    [28] = "htt",  [29] = "tm",
-    [30] = "ia64", [31] = "pbe",
-};
-
-static const char *const str_1c[32] =
-{
-    [ 0] = "sse3",    [ 1] = "pclmulqdq",
-    [ 2] = "dtes64",  [ 3] = "monitor",
-    [ 4] = "ds-cpl",  [ 5] = "vmx",
-    [ 6] = "smx",     [ 7] = "est",
-    [ 8] = "tm2",     [ 9] = "ssse3",
-    [10] = "cntx-id", [11] = "sdgb",
-    [12] = "fma",     [13] = "cx16",
-    [14] = "xtpr",    [15] = "pdcm",
-    /* [16] */        [17] = "pcid",
-    [18] = "dca",     [19] = "sse41",
-    [20] = "sse42",   [21] = "x2apic",
-    [22] = "movebe",  [23] = "popcnt",
-    [24] = "tsc-dl",  [25] = "aesni",
-    [26] = "xsave",   [27] = "osxsave",
-    [28] = "avx",     [29] = "f16c",
-    [30] = "rdrnd",   [31] = "hyper",
-};
-
-static const char *const str_e1d[32] =
-{
-    [ 0] = "fpu",    [ 1] = "vme",
-    [ 2] = "de",     [ 3] = "pse",
-    [ 4] = "tsc",    [ 5] = "msr",
-    [ 6] = "pae",    [ 7] = "mce",
-    [ 8] = "cx8",    [ 9] = "apic",
-    /* [10] */       [11] = "syscall",
-    [12] = "mtrr",   [13] = "pge",
-    [14] = "mca",    [15] = "cmov",
-    [16] = "fcmov",  [17] = "pse36",
-    /* [18] */       [19] = "mp",
-    [20] = "nx",     /* [21] */
-    [22] = "mmx+",   [23] = "mmx",
-    [24] = "fxsr",   [25] = "fxsr+",
-    [26] = "pg1g",   [27] = "rdtscp",
-    /* [28] */       [29] = "lm",
-    [30] = "3dnow+", [31] = "3dnow",
-};
-
-static const char *const str_e1c[32] =
-{
-    [ 0] = "lahf-lm",    [ 1] = "cmp",
-    [ 2] = "svm",        [ 3] = "extapic",
-    [ 4] = "cr8d",       [ 5] = "lzcnt",
-    [ 6] = "sse4a",      [ 7] = "msse",
-    [ 8] = "3dnowpf",    [ 9] = "osvw",
-    [10] = "ibs",        [11] = "xop",
-    [12] = "skinit",     [13] = "wdt",
-    /* [14] */           [15] = "lwp",
-    [16] = "fma4",       [17] = "tce",
-    /* [18] */           [19] = "nodeid",
-    /* [20] */           [21] = "tbm",
-    [22] = "topoext",    [23] = "perfctr-core",
-    [24] = "perfctr-nb", /* [25] */
-    [26] = "dbx",        [27] = "perftsc",
-    [28] = "pcx-l2i",    [29] = "monitorx",
-    [30] = "addr-msk-ext",
-};
-
-static const char *const str_7b0[32] =
-{
-    [ 0] = "fsgsbase", [ 1] = "tsc-adj",
-    [ 2] = "sgx",      [ 3] = "bmi1",
-    [ 4] = "hle",      [ 5] = "avx2",
-    [ 6] = "fdp-exn",  [ 7] = "smep",
-    [ 8] = "bmi2",     [ 9] = "erms",
-    [10] = "invpcid",  [11] = "rtm",
-    [12] = "pqm",      [13] = "depfpp",
-    [14] = "mpx",      [15] = "pqe",
-    [16] = "avx512f",  [17] = "avx512dq",
-    [18] = "rdseed",   [19] = "adx",
-    [20] = "smap",     [21] = "avx512-ifma",
-    [22] = "pcommit",  [23] = "clflushopt",
-    [24] = "clwb",     [25] = "proc-trace",
-    [26] = "avx512pf", [27] = "avx512er",
-    [28] = "avx512cd", [29] = "sha",
-    [30] = "avx512bw", [31] = "avx512vl",
-};
-
-static const char *const str_Da1[32] =
-{
-    [ 0] = "xsaveopt", [ 1] = "xsavec",
-    [ 2] = "xgetbv1",  [ 3] = "xsaves",
-};
-
-static const char *const str_7c0[32] =
-{
-    [ 0] = "prefetchwt1",      [ 1] = "avx512-vbmi",
-    [ 2] = "umip",             [ 3] = "pku",
-    [ 4] = "ospke",            [ 5] = "waitpkg",
-    [ 6] = "avx512-vbmi2",     [ 7] = "cet-ss",
-    [ 8] = "gfni",             [ 9] = "vaes",
-    [10] = "vpclmulqdq",       [11] = "avx512-vnni",
-    [12] = "avx512-bitalg",
-    [14] = "avx512-vpopcntdq",
-
-    [22] = "rdpid",
-    /* 24 */                   [25] = "cldemote",
-    /* 26 */                   [27] = "movdiri",
-    [28] = "movdir64b",        [29] = "enqcmd",
-    [30] = "sgx-lc",           [31] = "pks",
-};
-
-static const char *const str_e7d[32] =
-{
-    /* 6 */                    [ 7] = "hw-pstate",
-    [ 8] = "itsc",             [ 9] = "cpb",
-    [10] = "efro",
-};
-
-static const char *const str_e8b[32] =
-{
-    [ 0] = "clzero",
-    [ 2] = "rstr-fp-err-ptrs",
-
-    /* [ 8] */                 [ 9] = "wbnoinvd",
-
-    [12] = "ibpb",
-    [14] = "ibrs",             [15] = "amd-stibp",
-    [16] = "ibrs-always",      [17] = "stibp-always",
-    [18] = "ibrs-fast",        [19] = "ibrs-same-mode",
-
-    [20] = "no-lmsl",
-    /* [22] */                 [23] = "ppin",
-    [24] = "amd-ssbd",         [25] = "virt-ssbd",
-    [26] = "ssb-no",
-    [28] = "psfd",             [29] = "btc-no",
-    [30] = "ibpb-ret",
-};
-
-static const char *const str_7d0[32] =
-{
-    [ 2] = "avx512-4vnniw", [ 3] = "avx512-4fmaps",
-    [ 4] = "fsrm",
-
-    [ 8] = "avx512-vp2intersect", [ 9] = "srbds-ctrl",
-    [10] = "md-clear",            [11] = "rtm-always-abort",
-    /* 12 */                [13] = "tsx-force-abort",
-    [14] = "serialize",     [15] = "hybrid",
-    [16] = "tsxldtrk",
-    [18] = "pconfig",
-    [20] = "cet-ibt",
-    /* 22 */                [23] = "avx512-fp16",
-
-    [26] = "ibrsb",         [27] = "stibp",
-    [28] = "l1d-flush",     [29] = "arch-caps",
-    [30] = "core-caps",     [31] = "ssbd",
-};
-
-static const char *const str_7a1[32] =
-{
-    [ 0] = "sha512",        [ 1] = "sm3",
-    [ 2] = "sm4",
-    [ 4] = "avx-vnni",      [ 5] = "avx512-bf16",
-
-    [10] = "fzrm",          [11] = "fsrs",
-    [12] = "fsrcs",
-
-    /* 18 */                [19] = "wrmsrns",
+#include <xen/lib/x86/cpu-policy.h>
 
-    /* 22 */                [23] = "avx-ifma",
-};
-
-static const char *const str_e21a[32] =
-{
-    [ 0] = "no-nest-bp",    [ 1] = "fs-gs-ns",
-    [ 2] = "lfence+",
-    [ 6] = "nscb",
-    [ 8] = "auto-ibrs",
-    [10] = "amd-fsrs",      [11] = "amd-fsrc",
-
-    /* 16 */                [17] = "cpuid-user-dis",
-    [18] = "epsf",          [19] = "fsrsc",
-    [20] = "amd-prefetchi",
-
-    /* 26 */                [27] = "sbpb",
-    [28] = "ibpb-brtype",   [29] = "srso-no",
-};
-
-static const char *const str_7b1[32] =
-{
-    [ 0] = "ppin",
-};
-
-static const char *const str_7c1[32] =
-{
-};
-
-static const char *const str_7d1[32] =
-{
-    [ 4] = "avx-vnni-int8",       [ 5] = "avx-ne-convert",
-
-    [10] = "avx-vnni-int16",
-
-    [14] = "prefetchi",
-
-    [18] = "cet-sss",
-};
-
-static const char *const str_7d2[32] =
-{
-    [ 0] = "intel-psfd",    [ 1] = "ipred-ctrl",
-    [ 2] = "rrsba-ctrl",    [ 3] = "ddp-ctrl",
-    [ 4] = "bhi-ctrl",      [ 5] = "mcdt-no",
-};
-
-static const char *const str_m10Al[32] =
-{
-    [ 0] = "rdcl-no",             [ 1] = "eibrs",
-    [ 2] = "rsba",                [ 3] = "skip-l1dfl",
-    [ 4] = "intel-ssb-no",        [ 5] = "mds-no",
-    [ 6] = "if-pschange-mc-no",   [ 7] = "tsx-ctrl",
-    [ 8] = "taa-no",              [ 9] = "mcu-ctrl",
-    [10] = "misc-pkg-ctrl",       [11] = "energy-ctrl",
-    [12] = "doitm",               [13] = "sbdr-ssdp-no",
-    [14] = "fbsdp-no",            [15] = "psdp-no",
-    /* 16 */                      [17] = "fb-clear",
-    [18] = "fb-clear-ctrl",       [19] = "rrsba",
-    [20] = "bhi-no",              [21] = "xapic-status",
-    /* 22 */                      [23] = "ovrclk-status",
-    [24] = "pbrsb-no",            [25] = "gds-ctrl",
-    [26] = "gds-no",              [27] = "rfds-no",
-    [28] = "rfds-clear",
-};
-
-static const char *const str_m10Ah[32] =
-{
-};
+static uint32_t nr_features;
 
 static const struct {
     const char *name;
     const char *abbr;
-    const char *const *strs;
-} decodes[] =
+} leaf_names[] =
 {
-    { "CPUID 0x00000001.edx",        "1d", str_1d },
-    { "CPUID 0x00000001.ecx",        "1c", str_1c },
-    { "CPUID 0x80000001.edx",       "e1d", str_e1d },
-    { "CPUID 0x80000001.ecx",       "e1c", str_e1c },
-    { "CPUID 0x0000000d:1.eax",     "Da1", str_Da1 },
-    { "CPUID 0x00000007:0.ebx",     "7b0", str_7b0 },
-    { "CPUID 0x00000007:0.ecx",     "7c0", str_7c0 },
-    { "CPUID 0x80000007.edx",       "e7d", str_e7d },
-    { "CPUID 0x80000008.ebx",       "e8b", str_e8b },
-    { "CPUID 0x00000007:0.edx",     "7d0", str_7d0 },
-    { "CPUID 0x00000007:1.eax",     "7a1", str_7a1 },
-    { "CPUID 0x80000021.eax",      "e21a", str_e21a },
-    { "CPUID 0x00000007:1.ebx",     "7b1", str_7b1 },
-    { "CPUID 0x00000007:2.edx",     "7d2", str_7d2 },
-    { "CPUID 0x00000007:1.ecx",     "7c1", str_7c1 },
-    { "CPUID 0x00000007:1.edx",     "7d1", str_7d1 },
-    { "MSR_ARCH_CAPS.lo",         "m10Al", str_m10Al },
-    { "MSR_ARCH_CAPS.hi",         "m10Ah", str_m10Ah },
+    { "CPUID 0x00000001.edx",        "1d" },
+    { "CPUID 0x00000001.ecx",        "1c" },
+    { "CPUID 0x80000001.edx",       "e1d" },
+    { "CPUID 0x80000001.ecx",       "e1c" },
+    { "CPUID 0x0000000d:1.eax",     "Da1" },
+    { "CPUID 0x00000007:0.ebx",     "7b0" },
+    { "CPUID 0x00000007:0.ecx",     "7c0" },
+    { "CPUID 0x80000007.edx",       "e7d" },
+    { "CPUID 0x80000008.ebx",       "e8b" },
+    { "CPUID 0x00000007:0.edx",     "7d0" },
+    { "CPUID 0x00000007:1.eax",     "7a1" },
+    { "CPUID 0x80000021.eax",      "e21a" },
+    { "CPUID 0x00000007:1.ebx",     "7b1" },
+    { "CPUID 0x00000007:2.edx",     "7d2" },
+    { "CPUID 0x00000007:1.ecx",     "7c1" },
+    { "CPUID 0x00000007:1.edx",     "7d1" },
+    { "MSR_ARCH_CAPS.lo",         "m10Al" },
+    { "MSR_ARCH_CAPS.hi",         "m10Ah" },
 };
 
 #define COL_ALIGN "24"
@@ -305,12 +56,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
 {
     unsigned i;
 
-    if ( !strs )
-    {
-        printf(" ???");
-        return;
-    }
-
     for ( i = 0; i < 32; ++i )
         if ( leaf & (1u << i) )
         {
@@ -326,6 +71,7 @@ static void decode_featureset(const uint32_t *features,
                               const char *name,
                               bool detail)
 {
+    static const uint32_t __maybe_unused known_features[] = INIT_KNOWN_FEATURES;
     unsigned int i;
 
     printf("%-"COL_ALIGN"s        ", name);
@@ -336,11 +82,21 @@ static void decode_featureset(const uint32_t *features,
     if ( !detail )
         return;
 
-    for ( i = 0; i < length && i < ARRAY_SIZE(decodes); ++i )
+    /* Ensure leaf names stay in sync with the policy leaf count. */
+    BUILD_BUG_ON(ARRAY_SIZE(known_features) != ARRAY_SIZE(leaf_names));
+
+    for ( i = 0; i < length && i < ARRAY_SIZE(leaf_names); ++i )
     {
-        printf("  [%02u] %-"COL_ALIGN"s", i, decodes[i].name ?: "<UNKNOWN>");
-        if ( decodes[i].name )
-            dump_leaf(features[i], decodes[i].strs);
+        static const char *const feature_names[] = INIT_FEATURE_NAME_ARRAY;
+
+        /* Ensure feature names stays in sync with number of feature words. */
+        BUILD_BUG_ON(ARRAY_SIZE(feature_names) !=
+                     ARRAY_SIZE(leaf_names) * sizeof(*features) * 8);
+
+        printf("  [%02u] %-"COL_ALIGN"s", i, leaf_names[i].name ?: "<UNKNOWN>");
+
+        /* Attempt to print features anyway even if the leaf name is unknown. */
+        dump_leaf(features[i], &feature_names[i * 32]);
         printf("\n");
     }
 }
@@ -355,8 +111,8 @@ static void dump_info(xc_interface *xch, bool detail)
     if ( !detail )
     {
         printf("       %"COL_ALIGN"s ", "KEY");
-        for ( i = 0; i < ARRAY_SIZE(decodes); ++i )
-            printf("%-8s ", decodes[i].abbr ?: "???");
+        for ( i = 0; i < ARRAY_SIZE(leaf_names); ++i )
+            printf("%-8s ", leaf_names[i].abbr ?: "???");
         printf("\n");
     }
 
diff --git a/tools/tests/x86_emulator/x86-emulate.h b/tools/tests/x86_emulator/x86-emulate.h
index 8f8accfe3e70..a702c9faf207 100644
--- a/tools/tests/x86_emulator/x86-emulate.h
+++ b/tools/tests/x86_emulator/x86-emulate.h
@@ -50,7 +50,6 @@
 #define this_cpu(var) per_cpu_##var
 
 #define __init
-#define __maybe_unused __attribute__((__unused__))
 
 #define likely(x)   __builtin_expect(!!(x), true)
 #define unlikely(x) __builtin_expect(!!(x), false)
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 380b9d973a67..c0caa3bdebef 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -475,6 +475,32 @@ def write_results(state):
     state.output.write(
 """}
 
+""")
+
+    state.output.write(
+"""
+#define INIT_FEATURE_NAME_ARRAY { \\
+""")
+
+    try:
+        _tmp = state.names.iteritems()
+    except AttributeError:
+        _tmp = state.names.items()
+
+    for bit, name in sorted(_tmp):
+        state.output.write(
+            '    [%s] = "%s",\\\n' % (bit, name.lower().replace("_", "-"))
+            )
+
+    # Ensure array size matches featureset size.
+    if not state.names.get(state.nr_entries * 32 - 1, ""):
+        state.output.write(
+            '    [%s] = NULL,\\\n' % (state.nr_entries * 32 - 1)
+            )
+
+    state.output.write(
+"""}
+
 """)
 
     for idx, text in enumerate(state.bitfields):
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Mon May 13 07:47:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 07:47:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720696.1123601 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6QPR-0002f8-Rz; Mon, 13 May 2024 07:47:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720696.1123601; Mon, 13 May 2024 07:47:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6QPR-0002f1-Od; Mon, 13 May 2024 07:47:45 +0000
Received: by outflank-mailman (input) for mailman id 720696;
 Mon, 13 May 2024 07:47:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F+8V=MQ=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s6QPP-0002ev-TV
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 07:47:44 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 123423bf-10fd-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 09:47:40 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by SA1PR12MB6773.namprd12.prod.outlook.com (2603:10b6:806:258::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Mon, 13 May
 2024 07:47:34 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7544.052; Mon, 13 May 2024
 07:47:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 123423bf-10fd-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jVHcVOvrByKPubIYVfFSrPICZ0cUrGTOhA/jOWISEhQS5D1ja3UI54LA07trYzyxMNXmPddnVjqneF2wyaiydxjc7uLPzTD3351MnBIUuLJ/I7MFiQI4fUyX3MSylE3nM0fVbgrFhEbc73IJNUdDlDsGJygnxY5t32TxTucLB5iOA2AGS5xw0z11hIhmPW73So2RJSy7vUWR5kaJ9bUnwlYlNAAV7YmkVVsBrgkWN4IRho3jSgJLRsXVm0qEbbtV73weZiOorUQm6iswMY96uSEIw8RpU7ByoA2ZPMuK13JR5o/zftJ/cQxiMQZ4ToQH3PzZqlpK0ZP4m6C7xr6M+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dKAZdFH5MH33AhCYF2jQ+kz+OeMz3jv3JyZyEsNwQyQ=;
 b=jo2itbIAwdJK6EMr2Ye7FWgcC+Rhai1lpKqV2lMcn484VRLVZ6XDNr5DtENEfYysWCXTMqRef+7OjuShJyAbuXBBqoSNvVhQIOuhx5jHfKwC/TtgcvxJsTX5WlXSlnoGW1uhTvJfjDjPPSRfLvXGKkyZwhVF6CAJry5KCtF2mrtoeJjhNVpri9Dw1+7nJ/rBcPc57KI6R5HUTc6dZiNBdJ7E2JbPUrmB+i7rd4Y31rOdd/ee0cnS3PgNhzL9Yj/u+MCycSFFn0XgOF4ldYJgJDKSBaYFrxLfsTWWoHcCRshHMRDURsVNY4oKoN0EipP7Z9/eEbRGOblBP/UaMEKaSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dKAZdFH5MH33AhCYF2jQ+kz+OeMz3jv3JyZyEsNwQyQ=;
 b=n6AshVJDdslCqDm3OVUOY/UksMlGm32EUa7dgyFTg79jXcvjhAmZWw+PjUegJqenoYLMCZy573jqaYiug1d5IO7ZagZ/q9/hRkLBm1sL+YHfgtJZq36UeO8Z8TK6bK9ojtpD0yCNfQe7c89Z0E1lwY59Zp7f3CWjZCumDQcgVcI=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>
CC: Bjorn Helgaas <bhelgaas@google.com>, "Rafael J . Wysocki"
	<rafael@kernel.org>, =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?=
	<roger.pau@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "linux-pci@vger.kernel.org"
	<linux-pci@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "linux-acpi@vger.kernel.org"
	<linux-acpi@vger.kernel.org>, "Huang, Ray" <Ray.Huang@amd.com>, "Chen,
 Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
Thread-Topic: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get
 gsi from irq
Thread-Index: AQHakgrPZQMiHHEzE0Gs/TUOGsHh17GQJ2yAgACnAgCABKJfgA==
Date: Mon, 13 May 2024 07:47:34 +0000
Message-ID:
 <BL1PR12MB5849496A5B3148D162787961E7E22@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
In-Reply-To:
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7544.049)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|SA1PR12MB6773:EE_
x-ms-office365-filtering-correlation-id: 1705d6a4-68c6-433c-87ac-08dc7320f3da
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|376005|366007|1800799015|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?WjFldDkvSE5KWjlOVmFOSWVWK2U5eE4vakdhcmYyQ2NYdVRsc0Y5SEJTb0hw?=
 =?utf-8?B?Y2VyQVN1eUp2djI4Y3lhQjlRajgxMnAvZ2o3UW9vUmRSZ2RseVF5S0dJL2Vi?=
 =?utf-8?B?NXlBRGFWTGxNVjhOQkloZ1A3UFBYY3ppTHVad2Z6ckRlTVBHNkNvS3NsOUdG?=
 =?utf-8?B?Y2RUV1FsWVFpNWR0OUIrY0JvTCt3bDVrRnhWaGo0ZTNIK2MxVUFyQ0grVWcx?=
 =?utf-8?B?WTFlR3NFSTdlbHNsazFaeUZrNHVnWFJhR0tTWkQyRm9LbjNpcnZXUXhzUEdO?=
 =?utf-8?B?NkovN2J2K09ZVXpqS1BXNkxFQ1RzNnM2d3p1SjJHQlpIMXhJK0FzTU1HTXk4?=
 =?utf-8?B?ZXFYVk9WVkdoeHFTOGgrMXpSS21tSXoyVHEycEJ1OTIwVk0rc3JleklKRTUr?=
 =?utf-8?B?U2kwMUdoQ1NmNTB6MlhybEJtVjVHSEtSUFVaSE8veE14eTRzK2hPTTV1c1Vl?=
 =?utf-8?B?WmVEaDVtSGxRQnI1SGZsTGZMNzR1eTA0VmJUcjRTbFNJUFpqM1ducXFsVmxY?=
 =?utf-8?B?MnpmZm13TzZxREhSZERJVnRJaExMZ09kanhTUWhodGtacDVDU0tYZTFld1pr?=
 =?utf-8?B?THJmcnk1d3dWaTZvTVQ2RjNEbTAxcEYwa1JHY1l3UHdXWVdWcHArQzBtZ2ZQ?=
 =?utf-8?B?Q0o1S3MxYXdkZGJhMHdTYVFjY1ZXUHoxamJiTUFYcGxLbllhK3ZwSEthR2NZ?=
 =?utf-8?B?M2h0bnorVFBLNUtWMXZpSFd0VU1Bblh0ZWhZRWJWUGllZ0h5RTFuV1V3OWlY?=
 =?utf-8?B?Z3VVQUhBZFhYYVVpbHNhL0VmUlFNNlA4eFlMc01CKzdiOFNJc1NHQktwRURN?=
 =?utf-8?B?eGlZbTRVV3ZDdFRvYmN4WE5Ub0tpUmpXVzZGTDJBWVBtdC9pb1RXQzRrWnN4?=
 =?utf-8?B?bkp5a09GbkhLbGNLeTFLM1JTSDlWMVRlOEFhWXh5akhXbSsyak1zMEpoYzJo?=
 =?utf-8?B?ZWpzdUoyODBqYXlVVElaWWV0M0hyaER4NjR0bkJTWVJNcUxLUnFNdVl6L0Ix?=
 =?utf-8?B?TXdYNHY5QVFGK1dnamI5a21mdkFkeTl2RU5CUXMyc3AxZ3RpWGUzdGNtSmRk?=
 =?utf-8?B?UllQNk14Qzd0ak02OWVzV09ldmNmMVFjTUM0ajlFR2NBMkRGTG5TREVwOXFj?=
 =?utf-8?B?blZCUlVsZ1J5OGIvWGtCOXluMVdycGZSNjYwWXc2c0Y3aWdhRjZGSmd2Y2Zj?=
 =?utf-8?B?REw4M2pjZjhReGZSRTZXc2szUWJrZmZJbTlUeW1Nd294Si9EQUlFUS9nVlR0?=
 =?utf-8?B?ZEdlQ0Zya054YTYwelcySDhjNXBCWWU2RHRJVzhUejVhQi9KMllMSVVzN2w4?=
 =?utf-8?B?a2RETC9QTDN1ZkNjbXk4VGkzemY2UDdhQzY0L2RQdDZBcnB2QmtNU1owaThU?=
 =?utf-8?B?aHBvcTlzKzJCR3ZWalo3Y1J0T2FBV0wvclJmM25SM1hQZWtNekVZZEN2QVJV?=
 =?utf-8?B?RmZyRUhqb1pldGhLNW9QamZjSmExa09TenlwYVRlNjRvazVscVR4WVBwczN0?=
 =?utf-8?B?VytkVFdCMTAxQW1uNjdUdlN2RnhuZ1pkb0pzS1UrYU1ydkdNUDdBMEJjOHNa?=
 =?utf-8?B?VGtJUmZBeHBTVGNheFZJMVpOVW1CSGZqc3Irb0JFNDM3UEVkUEtiZHBITWxG?=
 =?utf-8?B?Zkg1dmU5VUNudVRpSmdITzBFS04zT0MwUHpiR0NtMUpRV3JQRjZFWmNPWDkz?=
 =?utf-8?B?MGNRa0tmSEt0SUlrMEx3Nzdjc3RyTVgzN3Q1T2JHd2tvTWNacFVUQ1RuVmNG?=
 =?utf-8?B?alNRMVAwM0NNZzNpaDRBci9ZTjRXRGE5OEVEaDdnOFVRVnBiMk5uR1FVa0NP?=
 =?utf-8?B?M0VhT2lvSHIwd3I5T1huUT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?TDA4VTFWdzBlOG1uNk5hVXgxMFhBbDVVWFp5VGMreStCYmNTYStKelk2V2Jr?=
 =?utf-8?B?Y3hrRFNLUzhFNE9FVnNsTlUyR2VzYnRJS244M3RaejZuaDdMZlFpempBR1Y3?=
 =?utf-8?B?R2NUU05DT3dvMDZDZDNNVHo5RG80c1FTLzVVR3Fvb1VkUW1XQlBITkJTeGNC?=
 =?utf-8?B?ZlhyOXZtNnY3K1hLNDhWZEhPNzZKSVIxR3JIV1REaWNabzBFa1FDMzJKbzBa?=
 =?utf-8?B?ZHBFWWFXeGJoOHgxc01SSXBYVk12ZEU4NlIxSHIwTkhBRTRFK1BjeHI0REdj?=
 =?utf-8?B?bHlxdFRKSTZOeTNrTkhjUEx3SFNOWFh5NHE0aUVCaW1BMERDTzJYcW00amdj?=
 =?utf-8?B?cWFwRmozK3Arem5ucWhGYW5IazI5U1lZWjNkR0VHZkV5S2hBajRqaEdROGE1?=
 =?utf-8?B?UFF2UDNhYWorWWRGRmROM0p0c1ducXJMUDdaZHZRSmt4ZTl1dkdYZnB3YStB?=
 =?utf-8?B?d3l6UGx1T3dLTGhMN0cyY2NXREUyUHBVUEVDem9kVE1MV0taaGo3VWpkOS9B?=
 =?utf-8?B?SSt1S21kNlEzL1hmTnh2N3Q2b0JuSFc0N3FDa3p2cHV4YWNoWnVQeEliUWJr?=
 =?utf-8?B?ellnT0tqUmY2Rk10R3didmNwOVFwVVF5MEprTldLRGFKNkhGc3BlemNvY1cr?=
 =?utf-8?B?YmFRMFQySG5DSFNidHJicE1KZFdRWGNKOHhUZmY2REN2U0RhTW82RDdCWWFR?=
 =?utf-8?B?OHdFdGtZYTdCQVFjdmVLUWZaV2Z5WjdHWXFrbVMyUXMvRHk4SnZUdHJEa3l3?=
 =?utf-8?B?ZlNYNGgvb0hsM1dFZzM5NFJWZXV2R0V3em94OWRSbUIwaE9naWV6dytJb3Vo?=
 =?utf-8?B?bmNZc0VackZQUmRleWhiYS82TGxpSGpjVzJrOGNxZ2VTUFBRblZmZWpxZDZj?=
 =?utf-8?B?cE5DSDZVbUlwVnF5NXJYNW5zY1VVZHJveVRITGlTeHZ2SXFOR004WElWM0ty?=
 =?utf-8?B?VXdJeThFVndSMk9pdkp1TFJZa2F3Um9SVDR1U2JWZThVQndGY0NvNkgybExo?=
 =?utf-8?B?OHNldktxMjA2Y0RmVUxyVTJTOXZod1dDdjR5TS9FZ0RQSmJaRzZPbS9NS3Za?=
 =?utf-8?B?MjVPU3NvODJHUzBEMkxvcmJxTklxbWhXaGVKTkZkdTRIR1VJdm4wM1dBMzFa?=
 =?utf-8?B?TGd6RXlKR3ArTnZUZ3ViNThZWmh3alUzRXpURHUzSmI4cllXMGRtSFE3OVM4?=
 =?utf-8?B?NjNndkJkbVlzSXdsaUY1bVNRb1N6Z2lydTBmRytXSFhNNVRsTENJMHRzVU5u?=
 =?utf-8?B?a29SNER2cVQyRXdtSkFZZDBtc0wvYWxDcHZyN2IxM0FRRFdjcC83Sk1qUkpW?=
 =?utf-8?B?NGxvNHJBczQ0M1R2L1dFWk14cnRQamlKdUlhbS9mbjZ6dTRoZHpLYnpyQktS?=
 =?utf-8?B?R1FZWlp6bkhGd2JhL3hZWVZKQ29rT0UwbkRwQUs4K0s3RU1SUVYwSUxxQTFX?=
 =?utf-8?B?TC9aS0EzSXoxd3dRQTVYUWFJQnhFUjF3MzBzL3lsbmJxM2pzS3JkTFd1VzEx?=
 =?utf-8?B?S3g3ZWYwNFRyUmFMbFpWdzhWdVhnbGk2ZTRPZ1JSOWZIMk16bzBibkJUZzJk?=
 =?utf-8?B?Z0ZnN0VRdzNhM0hVYVZtT2RVNVRjTGcvbFZ4VHFOWmloQ0EvN0FNeG5OMHpw?=
 =?utf-8?B?TVdNcjR1Z3lNTWpVbU1ubDlUVHUrN0xLTHZOc1dvaGs2L3d3cTFTNFc3dnls?=
 =?utf-8?B?S2F2bkg1cUZ4dFE0RlRXV0ZWay9lNFZkSkxoWFhIWitPM2k4Y0k3MStkKzI3?=
 =?utf-8?B?N0dTd1VXVm5LdWhpeXlzMy9KWjNiQjB6b216bXRKRWppMVR6TS9oVlY5QTUy?=
 =?utf-8?B?d1oyOS9DeVBEZmdtY3pabkZXODVPbm5mZGxocS92Wmk3UTlmMHJZS0tKSXd6?=
 =?utf-8?B?aWJaelhhS1FodmN4Y0c4NTFLV0VTUE1UU25RUkVjOExqanpYcWZxNmEzaVNW?=
 =?utf-8?B?WXhid0FHMzk2SkdVUml1a3VsRHp6K3VQeXhmYkJtRjJtcEJUeUtSWHJhWThK?=
 =?utf-8?B?aVpGMzJqcngwYzJwV29QNk1Ea0V5cEh0bnFRTTNnRldNL0cxcklkT215U3lF?=
 =?utf-8?B?S2tZL09tSkJyVkF1eFNtWU95SGdWd2xpRG1aNFdWSXJWZHUyTlZLK0IwTmVN?=
 =?utf-8?Q?3I8g=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <03006C5E38003B48906FB0CB42D3B4DF@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1705d6a4-68c6-433c-87ac-08dc7320f3da
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 May 2024 07:47:34.7069
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: JK3D2SNKURFIpBu8puvMt/EMzLiDfCbXP72V/duqL5a5NbnCHaZABBcWhYtMqDSUjtudlpcoAjTvJQPqWeOmMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6773

SGksDQpPbiAyMDI0LzUvMTAgMTc6MDYsIENoZW4sIEppcWlhbiB3cm90ZToNCj4gSGksDQo+IA0K
PiBPbiAyMDI0LzUvMTAgMTQ6NDYsIErDvHJnZW4gR3Jvw58gd3JvdGU6DQo+PiBPbiAxOS4wNC4y
NCAwNTozNiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+Pj4gSW4gUFZIIGRvbTAsIGl0IHVzZXMgdGhl
IGxpbnV4IGxvY2FsIGludGVycnVwdCBtZWNoYW5pc20sDQo+Pj4gd2hlbiBpdCBhbGxvY3MgaXJx
IGZvciBhIGdzaSwgaXQgaXMgZHluYW1pYywgYW5kIGZvbGxvdw0KPj4+IHRoZSBwcmluY2lwbGUg
b2YgYXBwbHlpbmcgZmlyc3QsIGRpc3RyaWJ1dGluZyBmaXJzdC4gQW5kDQo+Pj4gdGhlIGlycSBu
dW1iZXIgaXMgYWxsb2NlZCBmcm9tIHNtYWxsIHRvIGxhcmdlLCBidXQgdGhlDQo+Pj4gYXBwbHlp
bmcgZ3NpIG51bWJlciBpcyBub3QsIG1heSBnc2kgMzggY29tZXMgYmVmb3JlIGdzaSAyOCwNCj4+
PiBpdCBjYXVzZXMgdGhlIGlycSBudW1iZXIgaXMgbm90IGVxdWFsIHdpdGggdGhlIGdzaSBudW1i
ZXIuDQo+Pj4gQW5kIHdoZW4gcGFzc3Rocm91Z2ggYSBkZXZpY2UsIFFFTVUgd2lsbCB1c2UgZGV2
aWNlJ3MgZ3NpDQo+Pj4gbnVtYmVyIHRvIGRvIHBpcnEgbWFwcGluZywgYnV0IHRoZSBnc2kgbnVt
YmVyIGlzIGdvdCBmcm9tDQo+Pj4gZmlsZSAvc3lzL2J1cy9wY2kvZGV2aWNlcy88c2JkZj4vaXJx
LCBpcnEhPSBnc2ksIHNvIGl0IHdpbGwNCj4+PiBmYWlsIHdoZW4gbWFwcGluZy4NCj4+PiBBbmQg
aW4gY3VycmVudCBsaW51eCBjb2RlcywgdGhlcmUgaXMgbm8gbWV0aG9kIHRvIHRyYW5zbGF0ZQ0K
Pj4+IGlycSB0byBnc2kgZm9yIHVzZXJzcGFjZS4NCj4+Pg0KPj4+IEZvciBhYm92ZSBwdXJwb3Nl
LCByZWNvcmQgdGhlIHJlbGF0aW9uc2hpcCBvZiBnc2kgYW5kIGlycQ0KPj4+IHdoZW4gUFZIIGRv
bTAgZG8gYWNwaV9yZWdpc3Rlcl9nc2lfaW9hcGljIGZvciBkZXZpY2VzIGFuZA0KPj4+IGFkZHMg
YSBuZXcgc3lzY2FsbCBpbnRvIHByaXZjbWQgdG8gbGV0IHVzZXJzcGFjZSBjYW4gZ2V0DQo+Pj4g
dGhhdCB0cmFuc2xhdGlvbiB3aGVuIHRoZXkgaGF2ZSBhIG5lZWQuDQo+Pj4NCj4+PiBDby1kZXZl
bG9wZWQtYnk6IEh1YW5nIFJ1aSA8cmF5Lmh1YW5nQGFtZC5jb20+DQo+Pj4gU2lnbmVkLW9mZi1i
eTogSmlxaWFuIENoZW4gPEppcWlhbi5DaGVuQGFtZC5jb20+DQo+Pj4gLS0tDQo+Pj4gwqAgYXJj
aC94ODYvaW5jbHVkZS9hc20vYXBpYy5owqDCoMKgwqDCoCB8wqAgOCArKysrKysrDQo+Pj4gwqAg
YXJjaC94ODYvaW5jbHVkZS9hc20veGVuL3BjaS5owqDCoCB8wqAgNSArKysrDQo+Pj4gwqAgYXJj
aC94ODYva2VybmVsL2FjcGkvYm9vdC5jwqDCoMKgwqDCoCB8wqAgMiArLQ0KPj4+IMKgIGFyY2gv
eDg2L3BjaS94ZW4uY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfCAyMSArKysrKysrKysr
KysrKysrKw0KPj4+IMKgIGRyaXZlcnMveGVuL2V2ZW50cy9ldmVudHNfYmFzZS5jIHwgMzkgKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+PiDCoCBkcml2ZXJzL3hlbi9wcml2Y21k
LmPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgMTkgKysrKysrKysrKysrKysrKw0KPj4+IMKgIGlu
Y2x1ZGUvdWFwaS94ZW4vcHJpdmNtZC5owqDCoMKgwqDCoMKgIHzCoCA3ICsrKysrKw0KPj4+IMKg
IGluY2x1ZGUveGVuL2V2ZW50cy5owqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHzCoCA1ICsrKysN
Cj4+PiDCoCA4IGZpbGVzIGNoYW5nZWQsIDEwNSBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0p
DQo+Pj4NCj4+PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYvaW5jbHVkZS9hc20vYXBpYy5oIGIvYXJj
aC94ODYvaW5jbHVkZS9hc20vYXBpYy5oDQo+Pj4gaW5kZXggOWQxNTliNzcxZGM4Li5kZDQxMzky
NTA4OTUgMTAwNjQ0DQo+Pj4gLS0tIGEvYXJjaC94ODYvaW5jbHVkZS9hc20vYXBpYy5oDQo+Pj4g
KysrIGIvYXJjaC94ODYvaW5jbHVkZS9hc20vYXBpYy5oDQo+Pj4gQEAgLTE2OSw2ICsxNjksOSBA
QCBleHRlcm4gYm9vbCBhcGljX25lZWRzX3BpdCh2b2lkKTsNCj4+PiDCoCDCoCBleHRlcm4gdm9p
ZCBhcGljX3NlbmRfSVBJX2FsbGJ1dHNlbGYodW5zaWduZWQgaW50IHZlY3Rvcik7DQo+Pj4gwqAg
K2V4dGVybiBpbnQgYWNwaV9yZWdpc3Rlcl9nc2lfaW9hcGljKHN0cnVjdCBkZXZpY2UgKmRldiwg
dTMyIGdzaSwNCj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW50
IHRyaWdnZXIsIGludCBwb2xhcml0eSk7DQo+Pj4gKw0KPj4+IMKgICNlbHNlIC8qICFDT05GSUdf
WDg2X0xPQ0FMX0FQSUMgKi8NCj4+PiDCoCBzdGF0aWMgaW5saW5lIHZvaWQgbGFwaWNfc2h1dGRv
d24odm9pZCkgeyB9DQo+Pj4gwqAgI2RlZmluZSBsb2NhbF9hcGljX3RpbWVyX2MyX29rwqDCoMKg
wqDCoMKgwqAgMQ0KPj4+IEBAIC0xODMsNiArMTg2LDExIEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBh
cGljX2ludHJfbW9kZV9pbml0KHZvaWQpIHsgfQ0KPj4+IMKgIHN0YXRpYyBpbmxpbmUgdm9pZCBs
YXBpY19hc3NpZ25fc3lzdGVtX3ZlY3RvcnModm9pZCkgeyB9DQo+Pj4gwqAgc3RhdGljIGlubGlu
ZSB2b2lkIGxhcGljX2Fzc2lnbl9sZWdhY3lfdmVjdG9yKHVuc2lnbmVkIGludCBpLCBib29sIHIp
IHsgfQ0KPj4+IMKgIHN0YXRpYyBpbmxpbmUgYm9vbCBhcGljX25lZWRzX3BpdCh2b2lkKSB7IHJl
dHVybiB0cnVlOyB9DQo+Pj4gK3N0YXRpYyBpbmxpbmUgaW50IGFjcGlfcmVnaXN0ZXJfZ3NpX2lv
YXBpYyhzdHJ1Y3QgZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+Pj4gK8KgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIGludCB0cmlnZ2VyLCBpbnQgcG9sYXJpdHkpDQo+Pj4gK3sN
Cj4+PiArwqDCoMKgIHJldHVybiAoaW50KWdzaTsNCj4+PiArfQ0KPj4+IMKgICNlbmRpZiAvKiAh
Q09ORklHX1g4Nl9MT0NBTF9BUElDICovDQo+Pj4gwqAgwqAgI2lmZGVmIENPTkZJR19YODZfWDJB
UElDDQo+Pj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaCBiL2Fy
Y2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaA0KPj4+IGluZGV4IDkwMTViODg4ZWRkNi4uYWE4
ZGVkNjFmYzJkIDEwMDY0NA0KPj4+IC0tLSBhL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2ku
aA0KPj4+ICsrKyBiL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaA0KPj4+IEBAIC01LDYg
KzUsNyBAQA0KPj4+IMKgICNpZiBkZWZpbmVkKENPTkZJR19QQ0lfWEVOKQ0KPj4+IMKgIGV4dGVy
biBpbnQgX19pbml0IHBjaV94ZW5faW5pdCh2b2lkKTsNCj4+PiDCoCBleHRlcm4gaW50IF9faW5p
dCBwY2lfeGVuX2h2bV9pbml0KHZvaWQpOw0KPj4+ICtleHRlcm4gaW50IF9faW5pdCBwY2lfeGVu
X3B2aF9pbml0KHZvaWQpOw0KPj4+IMKgICNkZWZpbmUgcGNpX3hlbiAxDQo+Pj4gwqAgI2Vsc2UN
Cj4+PiDCoCAjZGVmaW5lIHBjaV94ZW4gMA0KPj4+IEBAIC0xMyw2ICsxNCwxMCBAQCBzdGF0aWMg
aW5saW5lIGludCBwY2lfeGVuX2h2bV9pbml0KHZvaWQpDQo+Pj4gwqAgew0KPj4+IMKgwqDCoMKg
wqAgcmV0dXJuIC0xOw0KPj4+IMKgIH0NCj4+PiArc3RhdGljIGlubGluZSBpbnQgcGNpX3hlbl9w
dmhfaW5pdCh2b2lkKQ0KPj4+ICt7DQo+Pj4gK8KgwqDCoCByZXR1cm4gLTE7DQo+Pj4gK30NCj4+
PiDCoCAjZW5kaWYNCj4+PiDCoCAjaWZkZWYgQ09ORklHX1hFTl9QVl9ET00wDQo+Pj4gwqAgaW50
IF9faW5pdCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpOw0KPj4+IGRpZmYgLS1naXQgYS9h
cmNoL3g4Ni9rZXJuZWwvYWNwaS9ib290LmMgYi9hcmNoL3g4Ni9rZXJuZWwvYWNwaS9ib290LmMN
Cj4+PiBpbmRleCA4NWEzY2UyYTM2NjYuLjcyYzczNDU4YzA4MyAxMDA2NDQNCj4+PiAtLS0gYS9h
cmNoL3g4Ni9rZXJuZWwvYWNwaS9ib290LmMNCj4+PiArKysgYi9hcmNoL3g4Ni9rZXJuZWwvYWNw
aS9ib290LmMNCj4+PiBAQCAtNzQ5LDcgKzc0OSw3IEBAIHN0YXRpYyBpbnQgYWNwaV9yZWdpc3Rl
cl9nc2lfcGljKHN0cnVjdCBkZXZpY2UgKmRldiwgdTMyIGdzaSwNCj4+PiDCoCB9DQo+Pj4gwqAg
wqAgI2lmZGVmIENPTkZJR19YODZfTE9DQUxfQVBJQw0KPj4+IC1zdGF0aWMgaW50IGFjcGlfcmVn
aXN0ZXJfZ3NpX2lvYXBpYyhzdHJ1Y3QgZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+Pj4gK2ludCBh
Y3BpX3JlZ2lzdGVyX2dzaV9pb2FwaWMoc3RydWN0IGRldmljZSAqZGV2LCB1MzIgZ3NpLA0KPj4+
IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpbnQgdHJpZ2dlciwg
aW50IHBvbGFyaXR5KQ0KPj4+IMKgIHsNCj4+PiDCoMKgwqDCoMKgIGludCBpcnEgPSBnc2k7DQo+
Pj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L3BjaS94ZW4uYyBiL2FyY2gveDg2L3BjaS94ZW4uYw0K
Pj4+IGluZGV4IDY1MmNkNTNlNzdmNi4uZjA1NmFiNWMwYTA2IDEwMDY0NA0KPj4+IC0tLSBhL2Fy
Y2gveDg2L3BjaS94ZW4uYw0KPj4+ICsrKyBiL2FyY2gveDg2L3BjaS94ZW4uYw0KPj4+IEBAIC0x
MTQsNiArMTE0LDIxIEBAIHN0YXRpYyBpbnQgYWNwaV9yZWdpc3Rlcl9nc2lfeGVuX2h2bShzdHJ1
Y3QgZGV2aWNlICpkZXYsIHUzMiBnc2ksDQo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIGZhbHNlIC8qIG5vIG1hcHBpbmcgb2YgR1NJIHRvIFBJUlEgKi8pOw0KPj4+IMKg
IH0NCj4+PiDCoCArc3RhdGljIGludCBhY3BpX3JlZ2lzdGVyX2dzaV94ZW5fcHZoKHN0cnVjdCBk
ZXZpY2UgKmRldiwgdTMyIGdzaSwNCj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgaW50IHRyaWdnZXIsIGludCBwb2xhcml0eSkNCj4+PiArew0KPj4+ICvCoMKgwqAg
aW50IGlycTsNCj4+PiArDQo+Pj4gK8KgwqDCoCBpcnEgPSBhY3BpX3JlZ2lzdGVyX2dzaV9pb2Fw
aWMoZGV2LCBnc2ksIHRyaWdnZXIsIHBvbGFyaXR5KTsNCj4+PiArwqDCoMKgIGlmIChpcnEgPCAw
KQ0KPj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gaXJxOw0KPj4+ICsNCj4+PiArwqDCoMKgIGlm
ICh4ZW5fcHZoX2FkZF9nc2lfaXJxX21hcChnc2ksIGlycSkgPT0gLUVFWElTVCkNCj4+PiArwqDC
oMKgwqDCoMKgwqAgcHJpbnRrKEtFUk5fSU5GTyAiQWxyZWFkeSBtYXAgdGhlIEdTSSA6JXUgYW5k
IElSUTogJWRcbiIsIGdzaSwgaXJxKTsNCj4+PiArDQo+Pj4gK8KgwqDCoCByZXR1cm4gaXJxOw0K
Pj4+ICt9DQo+Pj4gKw0KPj4+IMKgICNpZmRlZiBDT05GSUdfWEVOX1BWX0RPTTANCj4+PiDCoCBz
dGF0aWMgaW50IHhlbl9yZWdpc3Rlcl9nc2kodTMyIGdzaSwgaW50IHRyaWdnZXJpbmcsIGludCBw
b2xhcml0eSkNCj4+PiDCoCB7DQo+Pj4gQEAgLTU1OCw2ICs1NzMsMTIgQEAgaW50IF9faW5pdCBw
Y2lfeGVuX2h2bV9pbml0KHZvaWQpDQo+Pj4gwqDCoMKgwqDCoCByZXR1cm4gMDsNCj4+PiDCoCB9
DQo+Pj4gwqAgK2ludCBfX2luaXQgcGNpX3hlbl9wdmhfaW5pdCh2b2lkKQ0KPj4+ICt7DQo+Pj4g
K8KgwqDCoCBfX2FjcGlfcmVnaXN0ZXJfZ3NpID0gYWNwaV9yZWdpc3Rlcl9nc2lfeGVuX3B2aDsN
Cj4+DQo+PiBObyBzdXBwb3J0IGZvciB1bnJlZ2lzdGVyaW5nIHRoZSBnc2kgYWdhaW4/DQo+IF9f
YWNwaV91bnJlZ2lzdGVyX2dzaSBpcyBzZXQgaW4gZnVuY3Rpb24gYWNwaV9zZXRfaXJxX21vZGVs
X2lvYXBpYy4NCj4gTWF5YmUgSSBuZWVkIHRvIHVzZSBhIG5ldyBmdW5jdGlvbiB0byBjYWxsIGFj
cGlfdW5yZWdpc3Rlcl9nc2lfaW9hcGljIGFuZCByZW1vdmUgdGhlIG1hcHBpbmcgb2YgaXJxIGFu
ZCBnc2kgZnJvbSB4ZW5faXJxX2xpc3RfaGVhZCA/DQpXaGVuIEkgdHJpZWQgdG8gc3VwcG9ydCB1
bnJlZ2lzdGVyaW5nIHRoZSBnc2kgYW5kIHJlbW92aW5nIHRoZSBtYXBwaW5nIGR1cmluZyBkaXNh
YmxlIGRldmljZSwNCkkgZW5jb3VudGVyZWQgdGhhdCBhZnRlciBydW5uaW5nICJ4bCBwY2ktYXNz
aWduYWJsZS1hZGQgMDM6MDAuMCIsIGNhbGxzdGFjayBwY2lzdHViX2luaXRfZGV2aWNlLT54ZW5f
cGNpYmtfcmVzZXRfZGV2aWNlLT5wY2lfZGlzYWJsZV9kZXZpY2UtPnBjaWJpb3NfZGlzYWJsZV9k
ZXZpY2UtPmFjcGlfcGNpX2lycV9kaXNhYmxlLT5fX2FjcGlfdW5yZWdpc3Rlcl9nc2kNCnJlbW92
ZWQgdGhlIG1hcHBpbmcsIGFmdGVyIHRoYXQgd2hlbiB1c2VyIHNwYWNlIGNhbGxlZCB4ZW5fZ3Np
X2Zyb21faXJxIHRvIGdldCBnc2ksIGl0IGZhaWxlZC4NCg0KVG8gY292ZXIgYWJvdmUgY2FzZSwg
SSB3YW50IHRvIGNoYW5nZSB0aGUgaW1wbGVtZW50YXRpb24gb2YgeGVuX2dzaV9mcm9tX2lycSB0
byBwYXNzIHNiZGYgdG8gZ2V0IHRoZSBnc2kgaW5zdGVhZCBvZiBwYXNzaW5nIGlycSwNCkJlY2F1
c2UgdGhlIHNiZGYgYW5kIGdzaSBvZiBhIGRldmljZSBpcyB1bmlxdWUgYW5kIHdpaWwgbm90IGJl
IGNoYW5nZWQgZXZlbiBkZXZpY2UgaXMgZGlzYWJsZWQgb3IgcmUtZW5hYmxlZC4NCg0KRG8geW91
IHRoaW5rIHRoaXMga2luZCBvZiBjaGFuZ2UgaXMgYWNjZXB0YWJsZT8NCg0KPiANCj4+DQo+Pj4g
K8KgwqDCoCByZXR1cm4gMDsNCj4+PiArfQ0KPj4+ICsNCj4+DQo+PiBKdWVyZ2VuDQo+IA0KDQot
LSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Mon May 13 07:59:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 07:59:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720700.1123611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Qav-0004RB-S8; Mon, 13 May 2024 07:59:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720700.1123611; Mon, 13 May 2024 07:59:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Qav-0004R4-Oe; Mon, 13 May 2024 07:59:37 +0000
Received: by outflank-mailman (input) for mailman id 720700;
 Mon, 13 May 2024 07:59:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Vyab=MQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s6Qav-0004Qy-A3
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 07:59:37 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bd652c75-10fe-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 09:59:35 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so919831066b.3
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 00:59:35 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b18110sm561472766b.225.2024.05.13.00.59.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 13 May 2024 00:59:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd652c75-10fe-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715587175; x=1716191975; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=O5smVEjR/VVS0e2yObdThJN4tCbUbbAC8XLh8Tbaot0=;
        b=AzFdcbKB9nDw6Yeg9OkcOoF8Bm9w1PkkPiwNFeUsjnfcecyidgdwSYfgCDRIIo+dhc
         FkIbRiN3U7pC7hGplebff/bOvTlfNB6zubwsbYvvBiXoBHXPvMQhAkAn5QJ0+8KwZqSd
         tVLO40kE+BTD8FGH17G+Bzjc2xzeVVntw/yYeBA9V3KYtrakb5rdztyZ2LZSzAKR7+VS
         DE6wZ3Fh4kD7PTQWOTtQRgtj3m8TdjNHMCVves5cP//emsxLO70usNh95vCR2yxLi79z
         wYGxhcVwD+zXasgzpO/mY3zK+S2a7LXuN3jfYEFEheBgRfgYee92dneFWmwtU+x4VFoB
         05nw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715587175; x=1716191975;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=O5smVEjR/VVS0e2yObdThJN4tCbUbbAC8XLh8Tbaot0=;
        b=hM52k1pZmvBU/YhqoFw+Xg3OFUT7nSc9GzDmdgpBV9J3AJ7FzPNcix5C+03vZ3c6nv
         RR7UN5vIsrfumcURNTXzfl4VRfjuYlUwEgNHteDU0Bk6bBDesjXG+nQodMWl4YngWmgB
         7apL2Dq1cF1f5g0tJDwbt1lwB9Zl8EkbrJv5UtNLI2NJR7fUrxQIe43U9oWnD/mT4x8r
         AFjpKmr2BjBcdNBYALfe6d4TNAe6cr11ne78rVDo/LXBm9pGeBoBs2wqxdSmbxnutRfY
         DWmWcHsO+T+URpeVex4oOQcR9xS2xvRs88DU3Iwg/F0zX2qrX8X9jZ1l3ezsYqO4vLGK
         xxZA==
X-Forwarded-Encrypted: i=1; AJvYcCUbcXod3IaTGc2Qk5Gi5VckdKYmyz9mfBjzeOZLEV7pCY2PEBwRc1HnfRpY6/nReSicLT1HsScecsp9jXBv4HJt7hasEKzO+iTx3lJkiZo=
X-Gm-Message-State: AOJu0YzImLYbHqk0xBPHAtWp78QMOAi/nhAC/0IwzC1599+BuvqftJW4
	yRoi9QiiIe3Jp4k6xF/k82A6Cok3ZxDtaVsXXmhrrqLpTjhqckCUMHTuC7kUhzk=
X-Google-Smtp-Source: AGHT+IFPTr9tpemxyIWdG1g9bjPz3FBmkzUXW4eLPPKVth/o3VbHpxOnuWWpwypLDHy5PoSAIltx2A==
X-Received: by 2002:a17:906:f111:b0:a5a:28cc:ff85 with SMTP id a640c23a62f3a-a5a2d58545amr563789566b.28.1715587175252;
        Mon, 13 May 2024 00:59:35 -0700 (PDT)
Message-ID: <55bbd23b-d851-4274-9511-c5a7af938a07@suse.com>
Date: Mon, 13 May 2024 09:59:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC KERNEL PATCH v6 3/3] xen/privcmd: Add new syscall to get gsi
 from irq
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
 "Rafael J . Wysocki" <rafael@kernel.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
 "Huang, Ray" <Ray.Huang@amd.com>
References: <20240419033616.607889-1-Jiqian.Chen@amd.com>
 <20240419033616.607889-4-Jiqian.Chen@amd.com>
 <79666084-fc2f-4637-8f0b-3846285601b8@suse.com>
 <BL1PR12MB58493D17E23751A06FC931DDE7E72@BL1PR12MB5849.namprd12.prod.outlook.com>
 <BL1PR12MB5849496A5B3148D162787961E7E22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <BL1PR12MB5849496A5B3148D162787961E7E22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 13.05.24 09:47, Chen, Jiqian wrote:
> Hi,
> On 2024/5/10 17:06, Chen, Jiqian wrote:
>> Hi,
>>
>> On 2024/5/10 14:46, Jürgen Groß wrote:
>>> On 19.04.24 05:36, Jiqian Chen wrote:
>>>> In PVH dom0, it uses the linux local interrupt mechanism,
>>>> when it allocs irq for a gsi, it is dynamic, and follow
>>>> the principle of applying first, distributing first. And
>>>> the irq number is alloced from small to large, but the
>>>> applying gsi number is not, may gsi 38 comes before gsi 28,
>>>> it causes the irq number is not equal with the gsi number.
>>>> And when passthrough a device, QEMU will use device's gsi
>>>> number to do pirq mapping, but the gsi number is got from
>>>> file /sys/bus/pci/devices/<sbdf>/irq, irq!= gsi, so it will
>>>> fail when mapping.
>>>> And in current linux codes, there is no method to translate
>>>> irq to gsi for userspace.
>>>>
>>>> For above purpose, record the relationship of gsi and irq
>>>> when PVH dom0 do acpi_register_gsi_ioapic for devices and
>>>> adds a new syscall into privcmd to let userspace can get
>>>> that translation when they have a need.
>>>>
>>>> Co-developed-by: Huang Rui <ray.huang@amd.com>
>>>> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
>>>> ---
>>>>    arch/x86/include/asm/apic.h      |  8 +++++++
>>>>    arch/x86/include/asm/xen/pci.h   |  5 ++++
>>>>    arch/x86/kernel/acpi/boot.c      |  2 +-
>>>>    arch/x86/pci/xen.c               | 21 +++++++++++++++++
>>>>    drivers/xen/events/events_base.c | 39 ++++++++++++++++++++++++++++++++
>>>>    drivers/xen/privcmd.c            | 19 ++++++++++++++++
>>>>    include/uapi/xen/privcmd.h       |  7 ++++++
>>>>    include/xen/events.h             |  5 ++++
>>>>    8 files changed, 105 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
>>>> index 9d159b771dc8..dd4139250895 100644
>>>> --- a/arch/x86/include/asm/apic.h
>>>> +++ b/arch/x86/include/asm/apic.h
>>>> @@ -169,6 +169,9 @@ extern bool apic_needs_pit(void);
>>>>      extern void apic_send_IPI_allbutself(unsigned int vector);
>>>>    +extern int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
>>>> +                    int trigger, int polarity);
>>>> +
>>>>    #else /* !CONFIG_X86_LOCAL_APIC */
>>>>    static inline void lapic_shutdown(void) { }
>>>>    #define local_apic_timer_c2_ok        1
>>>> @@ -183,6 +186,11 @@ static inline void apic_intr_mode_init(void) { }
>>>>    static inline void lapic_assign_system_vectors(void) { }
>>>>    static inline void lapic_assign_legacy_vector(unsigned int i, bool r) { }
>>>>    static inline bool apic_needs_pit(void) { return true; }
>>>> +static inline int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
>>>> +                    int trigger, int polarity)
>>>> +{
>>>> +    return (int)gsi;
>>>> +}
>>>>    #endif /* !CONFIG_X86_LOCAL_APIC */
>>>>      #ifdef CONFIG_X86_X2APIC
>>>> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
>>>> index 9015b888edd6..aa8ded61fc2d 100644
>>>> --- a/arch/x86/include/asm/xen/pci.h
>>>> +++ b/arch/x86/include/asm/xen/pci.h
>>>> @@ -5,6 +5,7 @@
>>>>    #if defined(CONFIG_PCI_XEN)
>>>>    extern int __init pci_xen_init(void);
>>>>    extern int __init pci_xen_hvm_init(void);
>>>> +extern int __init pci_xen_pvh_init(void);
>>>>    #define pci_xen 1
>>>>    #else
>>>>    #define pci_xen 0
>>>> @@ -13,6 +14,10 @@ static inline int pci_xen_hvm_init(void)
>>>>    {
>>>>        return -1;
>>>>    }
>>>> +static inline int pci_xen_pvh_init(void)
>>>> +{
>>>> +    return -1;
>>>> +}
>>>>    #endif
>>>>    #ifdef CONFIG_XEN_PV_DOM0
>>>>    int __init pci_xen_initial_domain(void);
>>>> diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
>>>> index 85a3ce2a3666..72c73458c083 100644
>>>> --- a/arch/x86/kernel/acpi/boot.c
>>>> +++ b/arch/x86/kernel/acpi/boot.c
>>>> @@ -749,7 +749,7 @@ static int acpi_register_gsi_pic(struct device *dev, u32 gsi,
>>>>    }
>>>>      #ifdef CONFIG_X86_LOCAL_APIC
>>>> -static int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
>>>> +int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
>>>>                        int trigger, int polarity)
>>>>    {
>>>>        int irq = gsi;
>>>> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
>>>> index 652cd53e77f6..f056ab5c0a06 100644
>>>> --- a/arch/x86/pci/xen.c
>>>> +++ b/arch/x86/pci/xen.c
>>>> @@ -114,6 +114,21 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi,
>>>>                     false /* no mapping of GSI to PIRQ */);
>>>>    }
>>>>    +static int acpi_register_gsi_xen_pvh(struct device *dev, u32 gsi,
>>>> +                    int trigger, int polarity)
>>>> +{
>>>> +    int irq;
>>>> +
>>>> +    irq = acpi_register_gsi_ioapic(dev, gsi, trigger, polarity);
>>>> +    if (irq < 0)
>>>> +        return irq;
>>>> +
>>>> +    if (xen_pvh_add_gsi_irq_map(gsi, irq) == -EEXIST)
>>>> +        printk(KERN_INFO "Already map the GSI :%u and IRQ: %d\n", gsi, irq);
>>>> +
>>>> +    return irq;
>>>> +}
>>>> +
>>>>    #ifdef CONFIG_XEN_PV_DOM0
>>>>    static int xen_register_gsi(u32 gsi, int triggering, int polarity)
>>>>    {
>>>> @@ -558,6 +573,12 @@ int __init pci_xen_hvm_init(void)
>>>>        return 0;
>>>>    }
>>>>    +int __init pci_xen_pvh_init(void)
>>>> +{
>>>> +    __acpi_register_gsi = acpi_register_gsi_xen_pvh;
>>>
>>> No support for unregistering the gsi again?
>> __acpi_unregister_gsi is set in function acpi_set_irq_model_ioapic.
>> Maybe I need to use a new function to call acpi_unregister_gsi_ioapic and remove the mapping of irq and gsi from xen_irq_list_head ?
> When I tried to support unregistering the gsi and removing the mapping during disable device,
> I encountered that after running "xl pci-assignable-add 03:00.0", callstack pcistub_init_device->xen_pcibk_reset_device->pci_disable_device->pcibios_disable_device->acpi_pci_irq_disable->__acpi_unregister_gsi
> removed the mapping, after that when user space called xen_gsi_from_irq to get gsi, it failed.
> 
> To cover above case, I want to change the implementation of xen_gsi_from_irq to pass sbdf to get the gsi instead of passing irq,
> Because the sbdf and gsi of a device is unique and wiil not be changed even device is disabled or re-enabled.
> 
> Do you think this kind of change is acceptable?

Yes, I think so.


Juergen


From xen-devel-bounces@lists.xenproject.org Mon May 13 08:04:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 08:04:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720709.1123621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6QfM-0006XA-Vq; Mon, 13 May 2024 08:04:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720709.1123621; Mon, 13 May 2024 08:04:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6QfM-0006X3-Sh; Mon, 13 May 2024 08:04:12 +0000
Received: by outflank-mailman (input) for mailman id 720709;
 Mon, 13 May 2024 08:04:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6QfL-0006Wu-B0
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 08:04:11 +0000
Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com
 [2607:f8b0:4864:20::830])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6095d4f5-10ff-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 10:04:10 +0200 (CEST)
Received: by mail-qt1-x830.google.com with SMTP id
 d75a77b69052e-43df751b5b8so26694781cf.0
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 01:04:10 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df56ddd77sm52324351cf.92.2024.05.13.01.04.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 01:04:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6095d4f5-10ff-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715587449; x=1716192249; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=rjFDyNE0Bma0WbeZjIZz33kb6FYOzh5eHByKzczNbH4=;
        b=C+8zTSD79rYfcYq0Nbh9NGjlyzYY374gdf28eY34kNjBgDdVrv2l4gQl6NHHqvjBNI
         zLH1UYosCGhtDVgJQTx9JWtV7wuCi2x92Ytlkx/P78m/6v1hoRlUc4j/uk9CCP/7j1dy
         E5Xu6NM8jy37zi0DC2DZWxMbDDrnbxWGSXuE8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715587449; x=1716192249;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rjFDyNE0Bma0WbeZjIZz33kb6FYOzh5eHByKzczNbH4=;
        b=IaYSSt4Bm51VSDdhjuN/clam/JQ6AFfmAZZWFpgQOmbQjlpLbFnE7fDjbZC0IEHYFf
         1mu8mRIymuV2rZ1EPze4ea6kFSV+JHU34wOQk/xj2UUHeQmBwkF+1DZfMapjEIJt+CFW
         LRBZk7hAi/OgLEBAnMpUQC8q/OxTGJWFadD6k03EKLhRHMshS29w0U7ccLgyJl+GxrpC
         H0tnMK0WSXsnU4EcTNR+z87OQJ1eFuJny+LAdryeJIR1GNZP6V0N+WjLXtyy9NTtBG1s
         2PmmTeqNo7oUfWBrzbKeuaagfCqpLikqpxjcbzva5SvrZ966hBvN/ppvDAXrjWxbYvx8
         vBlg==
X-Forwarded-Encrypted: i=1; AJvYcCWVFbZJDWKSD6RpbRrCKR972awVgjFWWQ6vrJ8jO5vYsixIe6tPitamF8vmMffMRanoBirhIg7QSXNP15M51ehJ/ct5MPBKt0dkT9AswRM=
X-Gm-Message-State: AOJu0YyNY6EvCUIqpQ2PmZwGXWja76UTbnLBfvjko21vfMQZVNwtTG/S
	Mtw5uSngm+7HpwvlSDvvLDKmQovv6QHev0FvqDJQ/nQlY7ejDsHQVqTp7u9AKBM=
X-Google-Smtp-Source: AGHT+IHAHc9UBuFgcuXON2WzD3TXDJHqmOdVKpY4SAixdRSr0AxZCL07fL8VxqdyyDWm28HOU+4ByA==
X-Received: by 2002:ac8:7f10:0:b0:43b:9f:5ac8 with SMTP id d75a77b69052e-43dfce579ddmr140191811cf.31.1715587448887;
        Mon, 13 May 2024 01:04:08 -0700 (PDT)
Date: Mon, 13 May 2024 10:04:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Luca Fancellu <Luca.Fancellu@arm.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Message-ID: <ZkHJdk8b35CgcjJC@macbook>
References: <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org>
 <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org>
 <ZjyzTCOSeNvPq2Np@macbook>
 <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org>
 <ZjzIWsoRmgmWkgGh@macbook>
 <e1cf9c7c-581d-4bec-acaf-654d40b7ac07@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <e1cf9c7c-581d-4bec-acaf-654d40b7ac07@xen.org>

On Fri, May 10, 2024 at 10:37:53PM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 09/05/2024 13:58, Roger Pau Monné wrote:
> > On Thu, May 09, 2024 at 01:12:00PM +0100, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 09/05/2024 12:28, Roger Pau Monné wrote:
> > > > OTOH for 1GB given the code here the page could be freed in one go,
> > > > without a chance of preempting the operation.
> > > > 
> > > > Maybe you have to shatter superpages into 4K entries and then remove
> > > > them individually, as to allow for preemption to be possible by
> > > > calling put_page() for each 4K chunk?
> > > This would require to allocate some pages from the P2M pool for the tables.
> > > As the pool may be exhausted, it could be problematic when relinquishing the
> > > resources.
> > 
> > Indeed, it's not ideal.
> > 
> > > It may be possible to find a way to have memory available by removing other
> > > mappings first. But it feels a bit hackish and I would rather prefer if we
> > > avoid allocating any memory when relinquishing.
> > 
> > Maybe it could be helpful to provide a function to put a superpage,
> > that internally calls free_domheap_pages() with the appropriate order
> > so that freeing a superpage only takes a single free_domheap_pages()
> > call.
> 
> Today, free_domheap_page() is only called when the last reference is
> removed. I don't thinkt here is any guarantee that all the references will
> dropped at the same time.

I see, yes, we have no guarantee that removing the superpage entry in
the mapping domain will lead to either the whole superpage freed at
once, or not freed.  The source domain may have shattered the
super-page and hence freeing might need to be done at a smaller
granularity.

> >  That could reduce some of the contention around the heap_lock
> > and d->page_alloc_lock locks.
> 
> From previous experience (when Hongyan and I worked on optimizing
> init_heap_pages() for Live-Update), the lock is actually not the biggest
> problem. The issue is adding the pages back to the heap (which may requiring
> merging). So as long as the pages are not freed contiguously, we may not
> gain anything.

Would it help to defer the merging to the idle context, kind of
similar to what we do with scrubbing?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 13 08:24:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 08:24:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720742.1123647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Qye-0002Z0-C4; Mon, 13 May 2024 08:24:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720742.1123647; Mon, 13 May 2024 08:24:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Qye-0002Yt-8H; Mon, 13 May 2024 08:24:08 +0000
Received: by outflank-mailman (input) for mailman id 720742;
 Mon, 13 May 2024 08:24:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QNkj=MQ=epam.com=prvs=1863cc35df=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s6Qyc-0002Yl-86
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 08:24:06 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 282e210f-1102-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 10:24:04 +0200 (CEST)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44D7jQRZ003252;
 Mon, 13 May 2024 08:23:54 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3y21neveyu-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 May 2024 08:23:54 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by GV1PR03MB9871.eurprd03.prod.outlook.com (2603:10a6:150:89::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.42; Mon, 13 May
 2024 08:23:50 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%5]) with mapi id 15.20.7587.015; Mon, 13 May 2024
 08:23:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 282e210f-1102-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U9Kbs2Y6VmuCgB0Uq1WmxyJy//nTW4sS07dczJHoT5Ec5psaAa44xj22OaCmJ8pMiZdxhLNww0d/YbQ49mMWeMcVGcoMRXotRGg3a2/fORai64Flat5XcMs3F+ru+FSYZBbo9SyCd1eE1EknzJ9txBOpQpRO9wNf2eEfsIwuPFTKfMZ2OhSGpzMVZldKfaCEZDTMz6qH7cgYTeYYStbMTi9yc19s5uIshpWd5CXyJXMTZ1/EkcKYSe+XF6CvqOxL4F1PWI+5icietUbGhOPrNI30W+HbRbjQEkcxrHPunNw/CHn8nPctVMsKVTNZrGO5yvXseh77lEEu5ODaXHXtRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qEULfpCmMCJg0XTKwtUmzCcb2/gt8CF9ru/uB20YN4Q=;
 b=PlrtcsfBF72ckJFhZ5amE8frDual9Xylz+XSMzZZd06ykrwVpMmFqe7TvAZ849oTYPapWzThdbYuala6hS+pm1sZ1zscbN+Cl/Ihl/H36GVJtgeSAmPpbMOukaNi3zhXfZ7me7XMcAA8Hz4DKi5HzfVFHrFLiA9FziORC5H4DAKu2rYbc3a8c94N1Ap0wiafxJW5yW5wYJeWu3RFNQViGFx32OwS9lrz/K0cPVTBj8k34T4acBNKcRBYXPtC3P6YSj9y8kXIza5vVSy28Yj8RufiglIAdZyySw0t0+0beSk7X6OOYAXfPRE2dpLCrq3lrbksd8a+bUWUV0rW0/4x4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qEULfpCmMCJg0XTKwtUmzCcb2/gt8CF9ru/uB20YN4Q=;
 b=iznXe32d2GCGfVBt6+Kwmz6GhTUVe0QzHsLQ2Fo/hF69SwUkKJiQTRsp5CEs9m1paTCETC1bzfnxnnSrpcAHJSpx7wiEwJkvc6BYnhwrQhQkn4JcJ2Drr8B3OKqTIkf91xorb0zejk3kTdmAa5tBG1yVK1oJDCu4qP886uIoiRjhuesI1eiblLaV5VBpJa2D/I4t/fH9bg5MPf8KybUYbHyPOBECDdUli0tn4+y5WiqpZSr6bx3DrfvqDKP652v4kzTZhNazadsnueAim38K0J1M2QWTxJEH+ucQbxbPRwCRE0gQ4fsOIUCIk2OScGXEJHzOhj1bNCjzi3tcnl7quQ==
Message-ID: <f1ec85fb-a418-471c-8dc2-6321cd94d7e3@epam.com>
Date: Mon, 13 May 2024 11:23:48 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/5] x86/intel: move vmce_has_lmce() routine to
 header
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <e759fc187f2e3f23f42666b2a8461ae851591d50.1714640459.git.Sergiy_Kibrik@epam.com>
 <12d7600c-ccfa-430c-9c62-a2402b9620b8@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <12d7600c-ccfa-430c-9c62-a2402b9620b8@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0018.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::25) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|GV1PR03MB9871:EE_
X-MS-Office365-Filtering-Correlation-Id: 28894c40-ab8b-4a33-cb77-08dc7326049f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|366007|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?SEJSd3dMRnB5ZDUxdWI2SHl5NGl5aE03ekZPaWxkdHRrNXk4M1FJV0hlbmp2?=
 =?utf-8?B?ZmpEMnJzRUt2YUxTYXZPU0lUMFpaZS8wZG1lOTVJZWx6Rkx1V0pXV2MvRWxu?=
 =?utf-8?B?c0dJTGxlK2s4SGRQdmgwelZXTnJoSTBYcldWd3pmNDBYRDhXc252NnY1UHIy?=
 =?utf-8?B?WW1YSVcrSSs1M2F2QThGdnAvdHZGY1EwZXY4V0Z5Z0dnTW0vYkdndUhHRDUr?=
 =?utf-8?B?L0JabE1QUnFXbmpoaEZxUGJMV3YyMFJhdlFFemNpRlI4V2FkV3VYN3BxOXN4?=
 =?utf-8?B?dnVORlMwUFQwbTFLNUJ4d21kSjVBY0w1WklHVC9LQ2IvWHNCd0dUSnp6NnE1?=
 =?utf-8?B?dEJUcm1sRWVVb3Ezc2pwZnk0algxMHdGUFh2K2toNlkyR1JZYjRkYzRoTmUw?=
 =?utf-8?B?T21KbUx2OGJLZ2c5b1p5bEdzY1VocnJuSTA1SVQwZVRER3ZQYXdrNVlyVURZ?=
 =?utf-8?B?ZFY5LzVCRkpCMjVOUUQwRTRBbU56VnJyeVArenRSeXdjYVJQeXBjSHdPOTBt?=
 =?utf-8?B?Y0x3bENZK3UvL0FLS1dRMTc4cVpVUlBlaHFqRUNaUS9CZjFqYWZWNXAvK3oz?=
 =?utf-8?B?VTBiMGp5UUFlUFJrcG9sbDZCTENtSTdMclEycEVlcUxFN0w1QTJYb2g5VklD?=
 =?utf-8?B?empwMnA3RDlUOTNzczRkaHJSVkt5VUlxSktmTFlseE8vbTV2VzhDWVdrSUZD?=
 =?utf-8?B?L2tWNVpPU0FJenpQRDlVK1ZHRitrZXdScUtxTmZTcFNVL2NuUlN2WDJHa3JZ?=
 =?utf-8?B?dGtxQUpRRDVDTkhnNU5TSlYvTUFDUFY4cHBXZm0yN1l2SjJQRFltcWhod01J?=
 =?utf-8?B?Q0FMZ21qN3gzY1ZLL0YrdXJrTStQeTVFOEo2TkRRUTg3cm8zQlpNRFN2ODZw?=
 =?utf-8?B?UE9vbFYvalVQZHN3ZHdSSFMyK3d4TEN4K0FSZ21Odk0xdVVxSy9HNzF4M1Bp?=
 =?utf-8?B?R09Xa0xKYWJlRHBHM0RQb3NYWWtNMlhOMlhVMEZBWElyQlRMeHFyNld2VFlE?=
 =?utf-8?B?eEVCWTZCRnp3T1JkeThNZUtGQjlPYk8zWC9teElDdzV4UXFMRFVDWmpZblZj?=
 =?utf-8?B?cnNzTWtKYldLbHlOeFZHeEI4OCt2STlocnN6N2xxZmlsblRIa1d4VmNHa1h2?=
 =?utf-8?B?ZlBoUVdQc0pJTzdMWjlDSy8xTFlmOVlCR09RWEhJWGlZUk02Wis0bHJLcDFJ?=
 =?utf-8?B?b25BTUNvS0pxZTVpMHRwZnRlUTdUV2VLSHZEMmZ6cks2S1ZEREFwYy8zd3BH?=
 =?utf-8?B?V0ZmODBNWnhiT3M0b0lPb1lBbjF6NlFZNzM0U2lvd3kvbDlvM3hjL0NFYmpQ?=
 =?utf-8?B?ajVFUnpXMXZxTk9kMUd0YW45cGw1U1E2NC8yVDN5cjRrMHZqNy9EbUVVVzdI?=
 =?utf-8?B?cURrTmNEeEpwMldBVklNTDh0cDBzT0l0YlV2Z1hrNklUc0pMVXNONXVzdnlZ?=
 =?utf-8?B?aTdxdFhBZWtBYkZkMGVNNm14dzRTalQ0K0xGSWFJYjMwelVycHBhZFJuQnlS?=
 =?utf-8?B?MkdOK053MEhJeUdaTTIzMml5bUp5MHRPMmxicVVaanplMDd4TWtCWDJGRHlo?=
 =?utf-8?B?eS90MzExdStid3hsQzNIb0VzM2J2b2w5TTRYaDFJbER6d3dJbzZKZWZBMDJk?=
 =?utf-8?B?alI1ZjZwRG45S3V6WTN1YjQ1YVF3Q09TaUY2dWJWakw4T01ONVk0UTdtVzBN?=
 =?utf-8?B?ZWVkVEVzMW9QY0d2c3FzbUFvUytHenNmM2U0WmE4WlgyV21pTUJHS2Z3PT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?RTZ2Ty9lcXlnNHEvQWNVZ0hDcktNeklUUEFTcCs3emZTczRNUUtYUkpRZ216?=
 =?utf-8?B?TmtseGk1dTdnVnNLMDl0U1dRcmh4Sm9yT0FCOHd6NHZwMUlHZFdKM3BMMHlO?=
 =?utf-8?B?ZWFuWmlrVGl4TFM4STZCTnhONThCcFZYbjZFcEN1Z0RUcG44NHhxcnpjVWY0?=
 =?utf-8?B?cmozdVA0NlZ1WUU1czVyeGNlbDdTSHBQZmRxSE1XZmVxeU1NOVRDZVp2ZDMw?=
 =?utf-8?B?UkNtN0Z2Tk0xZWRkQ2NKRGg4dGtubDdEUE12aDhWSTAxc3ZjNUtYZlZCT3F6?=
 =?utf-8?B?S1JkK2VTb2JoN1UzZSt2dFZSSm9mN3JOckNKU25HQ3puc2hlRFhGOC80UlBK?=
 =?utf-8?B?U1JKY0M5bXVrL1NrMU41UDU4RjJqSUUxZzNIcW0rYmZjMEg2VUpLeVl4dTBz?=
 =?utf-8?B?eVpDVHpnbDV2L3JtQUZQTkhIVmNXbm5hR0ZZMTlBSmtzU2JvSHl2dnk4LzJx?=
 =?utf-8?B?VUNvVWcwM3h5Z3dHa2NnOGZFNjEwZmFqSU1sMy9HWjIyZXRGZ2t2VU4vR0Jl?=
 =?utf-8?B?amFlYWNBMUhaeEpvSkZIM0tnbzNJcEdLWTB6UnR4ZWxTeGUvODU4WU5tSXpL?=
 =?utf-8?B?TkdVeWpPcnVIb0gvM3prVGZPTTR0LzUweml5QWtVLzdodVMzcHh1cUswcnFK?=
 =?utf-8?B?clZWRDdLRG4zaHArZzF0bFhJN0REZDQ4YkJ1QWhPbUNvek9EQzhtOHBNNFR1?=
 =?utf-8?B?c1Jma0xjbllXajJMT1cvNGtmZ2MrODNoamp5SnNzK29tc1JVQ3YxUGVwTWNy?=
 =?utf-8?B?SVU2Um4zY2ZJdFBIK3dJMXgwQ2M2ekp6K0lQdG9hNm5USFg1WVd3Z0pzS1I2?=
 =?utf-8?B?WVhncFdXQ3JzdmpYckVsbk5XVmJPZWk2aktwRG9PM3NWRXNvUnJra0JEcUtl?=
 =?utf-8?B?WXFOSUloZisyZ0NyQkFtTWZWZ1A5ZVJlTi9lalpwRlg0RVFoTVMwRmVUU1A3?=
 =?utf-8?B?WUx2d0NMU1ZaMzBpemZBTVVaMzk2aHB1TmVpYWFqUDlpSEJqWHNkZUl2NUpR?=
 =?utf-8?B?bUtZZmRkMHNkWVlZM0p5N3gxYklrQkhleExucDEzUzM1cURvU2FiUEJLRENu?=
 =?utf-8?B?cVRRRlg0aXZXMUJRemJOdVd3T3hIeWpDTzN4Z2tPRXdhdXJITjVCbnRmU1E2?=
 =?utf-8?B?SllxMWlrd1p3a2ZVdS9WdEdxbXc2OWY1ZUs1dnlMM2M3L0toYjR2TXJLRExD?=
 =?utf-8?B?b2dNdlNnV1l2QkVBeDdlalZMZnEwRDMxbWFrRVAxejlpQ2tLTlF6ZmVWbFFk?=
 =?utf-8?B?SG9mQUgreXh2MVRXU2R5VFRkQ05FUHVKMDdyNXRKYnUrZzJNS0oyU2pWbHdM?=
 =?utf-8?B?cVBlZ3dLL2xmblRsMmlzQUJXdFg1eHdDZzdOZFBuaGszS1lzSXhHd3pHWG8z?=
 =?utf-8?B?TFk3NCtpVG9ML3NnY3Bob1NzU0ZOMmJZUTBRYXJ1emN1U0N5SCttNEh0NXpT?=
 =?utf-8?B?TEpVd0lnM1VxcEV2bjRNOHNBSStGcG5tMGJOalVrMU5abVRxMFV4NG5Pd09w?=
 =?utf-8?B?eU9KOTA4bFhMcnEyWnlJQjcyMXBmRUlWSkk1bmNUaHMzZXhuMlRZZjZUc3hM?=
 =?utf-8?B?cXNUaENwUU94NWFvVGVxWEpTbGZ2VzJFdzVVSUtEMzg5ai8waG9kcHZJY3ox?=
 =?utf-8?B?ZlZGZ2dwVEpNckdjNEpGaDNIdGZVaHh6VE4vWVB0SGFkRzNOZERVMUJFRnpu?=
 =?utf-8?B?MkV5ejJZQU5IaDQ4WmdpTU9tald4RTZHaHQ4dnhsU0FnN2w5R3VnUkwvVkxP?=
 =?utf-8?B?Tkk2NHEvU0VIQ25yN0hSZW0zMGM2L3MvRTV3T2dCc1JFUE55Z3o1NFRRaG5q?=
 =?utf-8?B?Vi94VWF6STJBTHV1cU1WT3I4TEZacUQzcW12L0NYSWRBbUtNblNycWw4dG42?=
 =?utf-8?B?TUVNNWFDZ1pwSmJIMVJrdUNhSE01L2xHRjV4R2IrdzY0bFNQajlTcUNtaXhV?=
 =?utf-8?B?VzAwSGFycUsyUUZyTTVBcXZQSmplMUQrNzcyRXdnWTRNb2JrTHB4YUJ2bnlH?=
 =?utf-8?B?WlcrTUlmbFM5ei9DWmJqQkV2TmZSdFNVeWFoNXZZR3N3Q25ZdXZHTzJQVUxL?=
 =?utf-8?B?WXorRVdTOFcwcit2OFBLR2lReUE0VVBLOHBFK2dEUDVYbXdDRWgvanZXNDJV?=
 =?utf-8?B?ZkowVjluMDNZNXdCUjZOMFdyeXIxcDkyZTI4bzJudmZQNlpNYUgwT0xCNmNL?=
 =?utf-8?B?dmc9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 28894c40-ab8b-4a33-cb77-08dc7326049f
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2024 08:23:50.5487
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /tL9HAZpvDwCg14+IodsfXGrDfQQOOolQ/4ahkZZ5dPrDrgf/YuzrF5MfA7SIXUYAyxiiy8g8FjFWodaPIKwNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR03MB9871
X-Proofpoint-GUID: 5QQ1J6aVwzro1-bxjw-54KRrWOR7K49j
X-Proofpoint-ORIG-GUID: 5QQ1J6aVwzro1-bxjw-54KRrWOR7K49j
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-13_05,2024-05-10_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999
 lowpriorityscore=0 spamscore=0 adultscore=0 clxscore=1011 impostorscore=0
 phishscore=0 priorityscore=1501 mlxscore=0 malwarescore=0 bulkscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405010000 definitions=main-2405130051

06.05.24 14:18, Jan Beulich:
> On 02.05.2024 11:14, Sergiy Kibrik wrote:
>> Moving this function out of mce_intel.c would make it possible to disable
>> build of Intel MCE code later on, because the function gets called from
>> common x86 code.
>>
>> Add internal check for CONFIG_INTEL option, as MCG_LMCE_P bit is currently
>> specific to Intel CPUs only.
> My previously voiced concern regarding this was not addressed. If ...

I misunderstood you comment to v1 patch.
I'll drop checks for CONFIG_INTEL, I see now that we don't really need one.

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Mon May 13 08:45:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 08:45:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720757.1123661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6RIw-0006JI-27; Mon, 13 May 2024 08:45:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720757.1123661; Mon, 13 May 2024 08:45:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6RIv-0006JB-VU; Mon, 13 May 2024 08:45:05 +0000
Received: by outflank-mailman (input) for mailman id 720757;
 Mon, 13 May 2024 08:45:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6RIu-0006J5-Sq
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 08:45:04 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16aefc01-1105-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 10:45:03 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-792b8bf806fso367419385a.0
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 01:45:03 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f17960csm41294326d6.25.2024.05.13.01.45.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 01:45:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16aefc01-1105-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715589902; x=1716194702; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=b0s0EUv5cog85v5AgtMuY3qUfBWjAdPaBPvTqMQMicw=;
        b=cSxHxxX68/0l3nocPI1YXpSjEWKdUd5N4MBHxYj9CPvq+jYsqjPOdclWxiyqlUoIzP
         0+WcicYWBirw+/+2xTstnyF56Se1cfoufzfjvuIH15gLi1o/JJIw9EHoCpHCOeGxd9WG
         PRRcEhtLPGx9lV4FhA4EKJcM+gai1Shkn4cyU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715589902; x=1716194702;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=b0s0EUv5cog85v5AgtMuY3qUfBWjAdPaBPvTqMQMicw=;
        b=V7Mwxif2KREm8GYwc5B2tppAk8nQ3OX1jeBYFgGJSlnXnmKGfJ99vXeo7Qg3PMUEC4
         dio12msyalt4DFPayLz4N/pohk0oGZCjcadjDDqpa2InK6PECJfcwEUmNNBdsozCi6v1
         Tt3RRK1mpozlPuQvV0pws8IcfkmBEW62mF4/7xkSOyAheC8enlcRHWXe1TCBHFVgT43n
         YbK5S0FkrjztEesMtYAdXVGNk5AY94zQmNRIn5AMLE553+X0z1bA1H/qApXG61gZYpI5
         kPluPCL2cYsAR8YvkvVWGiI2KLtSM185IV87PAekENNoiwh6Bx+SInLsE4DeNF2z2/o8
         91vw==
X-Forwarded-Encrypted: i=1; AJvYcCXaRuBb40ijZujyYlxNNSc2bmKjpyr9upnQz1QeD1+OLPedsNO89y1hTxQoejkccpho+dKQOqM96dlLhhl7Gg5aWcTLawqcuMlCCefbH8Q=
X-Gm-Message-State: AOJu0Yxeu0yHcJvjqNGY3IiDiIthhzliQX2cbv8/uOTAv16fPDnBMjbP
	G+fszl0Fiq7ZMv/MqWZxCNobae4LVFiNNRX4/kYrN4dqFCmrBk4bsxqazPWoAnQ=
X-Google-Smtp-Source: AGHT+IHqJuVytQ/+x1PNmWqL4v4BXle3O3RBSKqX4kVJRaxl9Tpmg+MluGlU+42vBK3g10nFo8jpgg==
X-Received: by 2002:a05:6214:4a8e:b0:6a0:d9fa:9a95 with SMTP id 6a1803df08f44-6a16822df5bmr133301956d6.44.1715589901913;
        Mon, 13 May 2024 01:45:01 -0700 (PDT)
Date: Mon, 13 May 2024 10:44:59 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Serious AMD-Vi(?) issue
Message-ID: <ZkHTC4RpUSpKj4wf@macbook>
References: <e9b1c9c4-523b-481b-946e-37c7c18ea1d2@suse.com>
 <ZgHwEGCsCLHiYU5J@mattapan.m5p.com>
 <ZgRXHQpamLIdu7dk@mattapan.m5p.com>
 <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
 <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>

On Fri, May 10, 2024 at 09:09:54PM -0700, Elliott Mitchell wrote:
> On Thu, Apr 18, 2024 at 09:33:31PM -0700, Elliott Mitchell wrote:
> > 
> > I suspect this is a case of there is some step which is missing from
> > Xen's IOMMU handling.  Perhaps something which Linux does during an early
> > DMA setup stage, but the current Xen implementation does lazily?
> > Alternatively some flag setting or missing step?
> > 
> > I should be able to do another test approach in a few weeks, but I would
> > love if something could be found sooner.
> 
> Turned out to be disturbingly easy to get the first entry when it
> happened.  Didn't even need `dbench`, it simply showed once the OS was
> fully loaded.  I did get some additional data points.
> 
> Appears this requires an AMD IOMMUv2.  A test system with known
> functioning AMD IOMMUv1 didn't display the issue at all.
> 
> (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr fffffffdf8000000 flags 0x8 I

I would expect the address field to contain more information about the
fault, but I'm not finding any information on the AMD-Vi specification
apart from that it contains the DVA, which makes no sense when the
fault is caused by an interrupt.

> (XEN) DDDD:bb:dd.f root @ 83b5f5 (3 levels) dfn=fffffffdf8000
> (XEN)   L3[1f7] = 0 np

Attempting to print the page table walk for an Interrupt remapping
fault is useless, we should likely avoid that when the I flag is set.

> 
> I find it surprising this required "iommu=debug" to get this level of
> detail.  This amount of output seems more appropriate for "verbose".

"verbose" should also print this information.

> 
> I strongly prefer to provide snippets.  There is a fair bit of output,
> I'm unsure which portion is most pertinent.

I've already voiced my concern that I think what yo uare doing is not
fair.  We are debugging this out of interest, and hence you refusing
to provide all information just hampers our ability to debug, and
makes us spend more time than required just thinking what snippets we
need to ask for.

I will ask again, what's there in the Xen or the Linux dmesgs that you
are so worried about leaking? Please provide an specific example.

Why do you mask the device SBDF in the above snippet?  I would really
like to understand what's so privacy relevant in a PCI SBDF number.

Does booting with `iommu=no-intremap` lead to any issues being
reported?

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 13 08:59:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 08:59:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720762.1123671 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6RX4-00014u-9A; Mon, 13 May 2024 08:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720762.1123671; Mon, 13 May 2024 08:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6RX4-00014n-6S; Mon, 13 May 2024 08:59:42 +0000
Received: by outflank-mailman (input) for mailman id 720762;
 Mon, 13 May 2024 08:59:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6RX3-00014h-77
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 08:59:41 +0000
Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com
 [2607:f8b0:4864:20::72f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e387fd0-1107-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 10:59:35 +0200 (CEST)
Received: by mail-qk1-x72f.google.com with SMTP id
 af79cd13be357-792b8d31702so276464885a.3
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 01:59:38 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf27f90asm436657885a.39.2024.05.13.01.59.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 01:59:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e387fd0-1107-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715590776; x=1716195576; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=W/+OcyPkAtT0pCVAr3+POiO/21XGiS+ysCNprHlUJrs=;
        b=lVWZNrnsP32ayrG5wksMAReJHDBQqxH/reDHTe44xkIFZTWiAAH88/PiuOymTZmt2c
         +DQNmVqh6yCMpYoXIzh3r0mWwXM9nt3Q1dOX4UUYsXoOZi3kuYNE5aerCDGDgCK7tV16
         usSPgycvNLUq7PMpmS/0qY++3SATfqEW3KX/A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715590776; x=1716195576;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=W/+OcyPkAtT0pCVAr3+POiO/21XGiS+ysCNprHlUJrs=;
        b=t6D6OcOEkoTmJjMCxitO6YWz5DhAFtMx/pHbRrIVfo3at8IMiO/X7jUi5tEdhQUOkp
         x5QCron4yCF0Qx8VRl42NY6y1FpHzmGGOpErtbYw/QGk/7oJJkEre/apcjkURxqYQPeT
         Jxtj4O4souvgj8XfAx6EUSC+BwPq5t6MhDc0pxJ8b2lUZfDvaVmUDfg05ViRj8A6XpSs
         GWckLNMhDkPO2pBcuVBvt1f3JGTUQlaXpkl/wBbSj0r+lhzMQ61PPw1K469XBiVP7uoH
         A5e91YK2X7Hj42SdtbGf3lbG6NLdbwcMMTLz5qC//aCQcScM/nuJvg52Q6nroI/9zqxF
         LiRw==
X-Gm-Message-State: AOJu0YxcXL7X9+FH5u7uceiSLydKZuxUdU3XD4pgpHUBE6auirq1WWpD
	wdkm74fPJafAvKTos05ccTHxCP/EJ93UXkBbxYnwUgxnQr8s+kLY5em0pLRT3n6dMFAlBJq94aa
	h
X-Google-Smtp-Source: AGHT+IExYVTD2wi9mnSJckW9cM+0MIQtI6P1ojDxQ0lXkV9q56dtHiqj1d99dMo5temt9ww+2LZFyQ==
X-Received: by 2002:a05:620a:820f:b0:792:daab:b209 with SMTP id af79cd13be357-792daabb44fmr358087485a.6.1715590776386;
        Mon, 13 May 2024 01:59:36 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when setting AP MTRRs
Date: Mon, 13 May 2024 10:59:25 +0200
Message-ID: <20240513085925.59324-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There's no point in forcing a system wide update of the MTRRs on all processors
when there are no changes to be propagated.  On AP startup it's only the AP
that needs to write the system wide MTRR values in order to match the rest of
the already online CPUs.

We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
on all the CPUs in the system.

While there adjust the comment to clarify why the system-wide resetting of the
MTRR registers is not needed for the purposes of mtrr_ap_init().

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
triggering, but it's also a good performance improvement when performing
cpu-online.

Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
the chance of the watchdog triggering due to the system-wide resetting of the
range.
---
 xen/arch/x86/cpu/mtrr/main.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/cpu/mtrr/main.c b/xen/arch/x86/cpu/mtrr/main.c
index 90b235f57e68..0a44ebbcb04f 100644
--- a/xen/arch/x86/cpu/mtrr/main.c
+++ b/xen/arch/x86/cpu/mtrr/main.c
@@ -573,14 +573,15 @@ void mtrr_ap_init(void)
 	if (!mtrr_if || hold_mtrr_updates_on_aps)
 		return;
 	/*
-	 * Ideally we should hold mtrr_mutex here to avoid mtrr entries changed,
-	 * but this routine will be called in cpu boot time, holding the lock
-	 * breaks it. This routine is called in two cases: 1.very earily time
-	 * of software resume, when there absolutely isn't mtrr entry changes;
-	 * 2.cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug lock to
-	 * prevent mtrr entry changes
+	 * hold_mtrr_updates_on_aps takes care of preventing unnecessary MTRR
+	 * updates when batch starting the CPUs (see
+	 * mtrr_aps_sync_{begin,end}()).
+	 *
+	 * Otherwise just apply the current system wide MTRR values to this AP.
+	 * Note this doesn't require synchronization with the other CPUs, as
+	 * there are strictly no modifications of the current MTRR values.
 	 */
-	set_mtrr(~0U, 0, 0, 0);
+	mtrr_set_all();
 }
 
 /**
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Mon May 13 09:12:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 09:12:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720774.1123682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Rie-0001eS-Sq; Mon, 13 May 2024 09:11:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720774.1123682; Mon, 13 May 2024 09:11:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Rie-0001eK-NW; Mon, 13 May 2024 09:11:40 +0000
Received: by outflank-mailman (input) for mailman id 720774;
 Mon, 13 May 2024 09:11:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QNkj=MQ=epam.com=prvs=1863cc35df=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s6Ric-0001eE-Ty
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 09:11:39 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cc434f10-1108-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 11:11:36 +0200 (CEST)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44D72ehT028825;
 Mon, 13 May 2024 09:11:32 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3y1yafvutg-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 May 2024 09:11:32 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by VI1PR03MB9987.eurprd03.prod.outlook.com (2603:10a6:800:1c7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.22; Mon, 13 May
 2024 09:11:28 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%5]) with mapi id 15.20.7587.015; Mon, 13 May 2024
 09:11:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc434f10-1108-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZXLEMzQeJQTp6DQq2/fnMquD0jg83NA7Jm/eXDemoCXf/cooAMHa0yqVwpg+FC0nfuQQ60ui6VQf3oIh0pyF4cKtKBh1/48YRtCXtvJxSucoq/37QfplYWP73cwYbJ5veVdcGFq5mdvKVlAZQ9nyBRiZfSqvyqAqBQq4uH8CgrXwjSHbbev2Wvr+sqUQYDAWrlwNcgVGMf1n8GNdPyvUh6rMG/uMrQfdtZy7FPs5PXqmz+1FuEryGxGgPsjYjCL6iU4t8spX3TVE19NgnBj9bLC8TzVQpiJTO0ZqNOpV2WigcWsvoVJWpz0vYh/Jj3Dr+My0kGIS6HxjLduKi2deeQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5KFEwE20wNiNRdhe7S74RnbY5GaK8qhweDQOMEVT7IQ=;
 b=Qw0d3jPlCiUa/E9Jm2esejCTtMVKERqBi1pDWijAkx/BocsGUYewPbwAfF8BXB/rJ8RaSUkzN8xa25vpZIROQuwws5AXhorAooNjxPud2Vj4/aovXDyYx9xkxueEmdd7fVmDPjt1xwHVCLbNXR4uWoqj9T1S1wCzsX0WSRsIIPUPJ8U5t86irqt6I2LcfmWjjcRxCCaDidqVo3a2E7VU0I8747tpw1OS7fT0t0B09Zh6IlE9S0mhC1s0B4OaK94rgrGFvRiY1Uvm6Id4Z7uTJBHmAhp4uFyNQzl7a1gASeypvyI9IcTSgNOOT1HjYD2WrlKHHbiKkpV4mFQPYgRA7Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5KFEwE20wNiNRdhe7S74RnbY5GaK8qhweDQOMEVT7IQ=;
 b=PI4uTEXs/r/w+15BNavFfS7zsa9qzQMmfSHgP+ITLOex1kKM/e75wND6Vr61AIDBYu9fMXsFRQ2QpLGGHnxSAYsYbD24YoID2YmwhG/xWZC+1HOL/AX5U4j+WuZRaUUPN8ez2cksQothVoOyWRRTqiF2lb13AFuHEypYEPs8V9p7QyEuJ+nTPj46shNZ01trM6rSltGJslchMcR1DX71Ju6g/R0lpwikZpR2P+vP+52XicWX18uoPXJFnmc1np/d5wtOxnrU+4SZd18YrA3YpAH53jDZEaEAkGQzpCego7fbvj0DdabjzQLitTj6F9pDe5DHlZa1dYKnzzoSpweRoA==
Message-ID: <63dc5c56-19e4-405d-a016-03175dd704bf@epam.com>
Date: Mon, 13 May 2024 12:11:26 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 5/5] x86/MCE: optional build of AMD/Intel MCE code
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <47d32ecff7b915bd23b6d13b76cedf4b39db71a2.1714640459.git.Sergiy_Kibrik@epam.com>
 <dc14d45e-b2b2-4414-a937-ae95115aa2cd@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <dc14d45e-b2b2-4414-a937-ae95115aa2cd@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0013.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::16) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|VI1PR03MB9987:EE_
X-MS-Office365-Filtering-Correlation-Id: 7116f5ca-ef1b-4240-b40d-08dc732cac06
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|366007|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?Yk92eGdPTlNyZldqOHYxSHQxK3FVcHdEMVg1OGNhOStpc2swUFhRdk95U0Ur?=
 =?utf-8?B?RFNvVDZhZHU4TEhwdkxqQUtCUFc2elE2SW9vdnJiZ3ZQR2tWNS9OL2hMWTk5?=
 =?utf-8?B?M0JJejZWdTZXOG9SckIyOWRxVHUxVnArQnNqUk9xcDBuMjhoU1pqK0luc1dF?=
 =?utf-8?B?TnJsTTR6WHdlTTFraW5TYnJmMWJDOHZjR1JLd3BieFhqc3ZKUzB5dVhZNDF5?=
 =?utf-8?B?dG9kdWlUQTZ4STVYbVd0RnM5eXNRdHg3NWNTakNiTi9NUzlVSmtnd1JucXEr?=
 =?utf-8?B?R0ZpbDJxSDdsdVQ1SEh5N3ZucXJuRlBnY0ZUQkQwMHZtS1NDQ3dPeXc5amRX?=
 =?utf-8?B?TmNIaHlCZGI2eGJBQy9HaGNuNmtveDBxRkU2Lzh5NTZ5cWd6OUNkZUkyTDVQ?=
 =?utf-8?B?aGJyalpKS0VQUnc4VUxzdk5jcnduOFJIdVdTSFluUGJFcGlRRjE5aHNLb0My?=
 =?utf-8?B?ZUEwWm83WnRocVRLTjFWR0pxZ3Q3K3JFd2dzdGlpRGpDOHpWVGhSaWE1SGtT?=
 =?utf-8?B?c0VGSkRwTkRWeVFTeDd0K0NYajdXTFJ0TUxoekFNdTJXNmt3REpmYnIxSGZX?=
 =?utf-8?B?eGdCSFREbkJtWWM4UFFmM0Zwa3c0RitXcGNRaGlRODRQRWliTThYbDUvTU81?=
 =?utf-8?B?Q1ZxMXhZMjF3aFZRaFYydVNKOTZ4d2k3TU5oYzVqZUtxTkp5bTlNUEFKT0F4?=
 =?utf-8?B?M1FaNE1Bb2U5VFVZbkZST1RuNVIxN1MzMkcxR2RQYTlnOEJYU3lVK3NDS21L?=
 =?utf-8?B?Q2dycG0walVsMzZjSFcwamVhb2JLODg0L05TdzBNN2d6djNVeXM3WDRpbDlM?=
 =?utf-8?B?OXF4ekxQVHJsc05ZQk5HVlZDeG5RTUlWNm9XdWxPTVloaE81UXpTZWRRbUgx?=
 =?utf-8?B?YzhYaXNJanJjU3RSTFFHTlluemRGUjg0bGpQRTdNWWpDZ3RxS1F2NkpDSzRu?=
 =?utf-8?B?YmtPb0VnVFk1U0hHY3NadjVNTTdTMkFib0tSS3NXMzJxd2FwZ1ZyVk83Y0t0?=
 =?utf-8?B?NXNoL3hSOUsvejBZZFN2TjhsQU1vendtdTFXVEY2TjZCcWJUVHBBbmszYTEy?=
 =?utf-8?B?clNrVVExa3ZYR3RwQloxQWNOZUxtZSt2UVd1bXB2cCtkVmVOTjhiOVZmcnQ1?=
 =?utf-8?B?YkQ0ZDFpdGc1TU9LTGRWY0V1QkdqU2h0VjhCS05Kc3o1SThzZVRQU0dRYXNy?=
 =?utf-8?B?QXlZY3g5aEt2SU81aFVTRWpmMzMvcEk1UFZ5UlRHa0lDcVVPUnJwcnU2dWx5?=
 =?utf-8?B?NzYzOStWdW5LRFMwVm5BV05hY0pQWEZURTlRVDYwMGg2dWY3VzlkdVRiaEY3?=
 =?utf-8?B?a0RhYThGWVJ1SHE0KzRia3Nqb1Q1VlZDd1ZTaG1PTXE2QlFtU3R5MDR6eXlB?=
 =?utf-8?B?OEZlaXdtYUVKRTdHaGlVc3NpV05NblBiVUJ3MGVsT0FzWWF2MFhQZE9jbWR3?=
 =?utf-8?B?L0thY01odFVvUFVnTi9qZGplOXhFVmFWYmhEWU1XdUtPb1JzV1hnM09VM2s5?=
 =?utf-8?B?MUpFVVdUdjNJY3h5Ukd6WWJpbU0rRnlnZTBRRSs5akRCemNhMGFXbWhvd2oz?=
 =?utf-8?B?WmpBWVBnV3cvMDF1TzUwclplYkc3SHgralMvV2NaRUFNT0xabjVKNUZrOVg4?=
 =?utf-8?B?Y2x6TnM0KzVMS29mNTFSUWVVUk5jbXp5cnVJczhTK2k1ZlpPRWY3V0ZZeENn?=
 =?utf-8?B?UXpLSU5pQ1k5VWhvdHNwaHJTRGp0R1NYYUczMzVOWWZsYnlLSEp0cmN3PT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?WmFsa3JMOEZGK0N2RE43WlZyUk9iUGVEWEl5S0U5b1NnVFJlZE0zenpEdlRB?=
 =?utf-8?B?bWFacTZtYWh0dkt0N0ZOOUt6Tks1cnlqVWtGTm1KbVUyYkZxUlJodEUyMzZz?=
 =?utf-8?B?Z1pEZ0RCY0EvTHovUll0YzdRQSt0blc5L2R6a0RhRER0VURtWEpJZmhUYnd1?=
 =?utf-8?B?OEhxTGJuOGgra0VrWEVwV3pRYTJFVDZMYVVpS041MzRSM3BHU2p2aDZNQllG?=
 =?utf-8?B?U0g3K044Vll1RGFkRldjd09oOFRCdkk0VEFnbTFVbDA2TW1ZQUJ5SEsySEo2?=
 =?utf-8?B?bDhvOVFzRkF0aVBzVWpmM1dVU2diMlR3eS9QSDdjMHhMVU51VFdTdnVhMUFW?=
 =?utf-8?B?alZhWUUzekZrTE9IRVo2ZmU5blJlMUZaUldSbFd6bmZUMHJtR2l5dlFDdGhM?=
 =?utf-8?B?STlocWNJc0QrR3B1ck5jbngwUVdqcnEyODlJeldKZElpaGlBT1VvYjBRNnBt?=
 =?utf-8?B?OGk3elN5dXZQbDM0a1ZUZ0g4VXJIRzI0YUhyc1RzTnZEUFNTV21vcGtPYkRO?=
 =?utf-8?B?T2Z3MEdpNmVCcU82ZFNtTWF1ZXZoeFhsR2ZlcEtyM2xrYWhpSlFWMDk0UVN3?=
 =?utf-8?B?RGFkamxUM3pyZlVvQlBrajhlODlPWjU1b3g5endWRzVKV2M2MC83T2lIeWpB?=
 =?utf-8?B?Y2dSb2ZVaEgwZlRmRDZ4M3Z3YlFoY0drQjZiN3VCeHl4WnBqV3N2VHBNMUdR?=
 =?utf-8?B?YmxRUHdSV3ZFaUQzQmVDVUFRU0piTkJvVlZrSXk2NzYxakNFK29RSzQ4K1E1?=
 =?utf-8?B?b1hpRDRnakM0M013U1FHN0UyV2FOS3h6VjAvRGxqeGtudEtYcE9TOVpwVHJw?=
 =?utf-8?B?YzFsWlY4OXdqZmxsTjd6Z0l0eTI5Yk9FelRwelVlZms5QktnUnFockV1ZEd6?=
 =?utf-8?B?Ym9qSGpNRUd4OWZQNnF3R2xqbWNxbjN6amVUb0F4Y25OQVhNb0hQb1JTT1hm?=
 =?utf-8?B?dUdKRWFydE4xdm5FV1hXUkJxZjVWeldJRlhrYzl3alBsYmk2cDAzVHgwYXZ4?=
 =?utf-8?B?UG5rUXBlQkV0L0JpemptbFFJaHlET01rYklDVkc2aGdFQk5FN2lmODFzVFR6?=
 =?utf-8?B?U0RlMndvRjVnZkFwOUJMeWJLOFRNVGpwSGxjVmZpalJsUGpIU1ZzQU5NVlEv?=
 =?utf-8?B?LysySlFmTndwd0JqNjgvR1Jmbk1YNExvQmhCbm85ZGRYQ2l0bmRNREMvRmZF?=
 =?utf-8?B?SGVxUUNyYktweEs3bGUycEZtMW5oRjVsTENtUFVxMzFPMnlvNFlVVU5yTzM3?=
 =?utf-8?B?Ylp4RGJ4TUZXSGdXc2FkcjQwNldRVWNxQSttSFg1Y1Z0QUFOL0RpQkppWUY0?=
 =?utf-8?B?SFRKdGtRWHBxWXNadGhlVkxycGJBdHJkYm1KUEozdkdzWldJTUk1RGJvOVRP?=
 =?utf-8?B?N2Y3c1JpVkZmZVhjYjBnTzFTNk5NbEd5UTdGZ2tqR0pqZ0dwUzFQTXFhbVdY?=
 =?utf-8?B?clpsN0drdWZZTEtNb1hRYmZIODQxQTdjY3pXVGZacjVHelFkaG0rMUVHUno2?=
 =?utf-8?B?MTZDY2hKQjB0emZiUm1ybjRjNVphazIyczJ5SDE3aW9qK3FQcDY1bmFiWGlv?=
 =?utf-8?B?b2RYU2g3UUhUMzJKNHdOUi9uR2dkV1RiK2ZNNmxuVGJlZzBvelZiKzFYQ1Bh?=
 =?utf-8?B?MnhidHczL2V6bitRRGo0bWh4Tm02b1RJMjdXVDRKNDQxR1hRaGpoOWZ2Q0Mx?=
 =?utf-8?B?eGNHQ2VsaERXYXdWNGVJcHQvQThFRUhRY3Y4clI2NW5vamZnYW5lM0dTbEJq?=
 =?utf-8?B?WkJhdENwdFk1djhseTl5V0RLYlJNdThESFpCa2pPOXlUM09OZmkySEcweHl2?=
 =?utf-8?B?NXNyQ1owelVSMGdWS3RiWFIxMC9BeHdNNWpDdUVobzkyZkFqYVQ0ZmdYZzl4?=
 =?utf-8?B?N1JHVnBNVDMrUkp5UktTbzJVNzRQaFFIMXBMZFRvZXlGVHJUL3h5N2lFclQr?=
 =?utf-8?B?amEyU1VFV2FOaXB1bDZtQXdqSW9uS3U5ZkZ3RS93YVZ3b1pnZnBkWFZRTmVH?=
 =?utf-8?B?NjNpeUZEa1hGQ3RXRGpaSnVXMzV4NStDS2swYkFaMTg0S0VucWFIZFVhSEls?=
 =?utf-8?B?ZFM5c3dIWVdpZForMnFyM3RCVmVmM3FUSG1BbEdJMURPaVd5YUxGVStqOGda?=
 =?utf-8?Q?2j5OM+2ioNhbAHMrGQ1mYmPyn?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7116f5ca-ef1b-4240-b40d-08dc732cac06
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2024 09:11:28.2716
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cIAEU22IHVa8Q+TH5M/cbNmx3MdDOsqxpRF3Vb8+lvKIB0xma5bs3/oL/l9jGiZ1RhltacCyph/bfqF+EOwDGA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB9987
X-Proofpoint-GUID: DqYP8XzZFZiD_2eIBeIXxnCxWqaHz0Gm
X-Proofpoint-ORIG-GUID: DqYP8XzZFZiD_2eIBeIXxnCxWqaHz0Gm
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-13_06,2024-05-10_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 mlxlogscore=999
 lowpriorityscore=0 malwarescore=0 clxscore=1015 adultscore=0 phishscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405010000 definitions=main-2405130056

06.05.24 14:32, Jan Beulich:
> On 02.05.2024 11:21, Sergiy Kibrik wrote:
>> Separate Intel/AMD-specific MCE code using CONFIG_{INTEL,AMD} config options.
>> Now we can avoid build of mcheck code if support for specific platform is
>> intentionally disabled by configuration.
>>
>> Add default return value to init_nonfatal_mce_checker() routine -- in case
>> of a build with both AMD and INTEL options are off (e.g. randconfig).
> 
> I'm afraid that, as before, I can't accept this as a justification for the
> addition. The addition likely is wanted, but perhaps in a separate up-front
> patch and explaining what's wrong when that's missing.

sure, I'll do separate patch for that.

> 
>> Also global Intel-specific variables lmce_support & cmci_support have to be
>> redefined if !INTEL, as they get checked in common code.
> 
> Them being checked in common code may have different resolution strategies.
> The justification here imo is that, right now, both variables are only ever
> written by mce_intel.c. As mentioned for vmce_has_lmce(), there's nothing
> fundamentally preventing MCG_CAP from having respective bits set on a non-
> Intel CPU.
> 

so could these global variables just be moved to common code then? Like 
arch/x86/cpu/mcheck/mce.c ?

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Mon May 13 09:36:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 09:36:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720778.1123692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6S6B-00053m-CV; Mon, 13 May 2024 09:35:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720778.1123692; Mon, 13 May 2024 09:35:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6S6B-00053f-90; Mon, 13 May 2024 09:35:59 +0000
Received: by outflank-mailman (input) for mailman id 720778;
 Mon, 13 May 2024 09:35:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6S69-00053Z-G2
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 09:35:57 +0000
Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 319cf100-110c-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 11:35:55 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2])
 by smtp-border-fw-6001.iad6.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 09:35:52 +0000
Received: from EX19MTAEUA001.ant.amazon.com [10.0.43.254:51335]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.3.195:2525]
 with esmtp (Farcaster)
 id f3d64704-d3dd-4a0e-82e9-75f995f45500; Mon, 13 May 2024 09:35:51 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUA001.ant.amazon.com (10.252.50.192) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 09:35:47 +0000
Received: from [10.95.164.111] (10.95.164.111) by
 EX19D018EUA002.ant.amazon.com (10.252.50.146) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 09:35:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 319cf100-110c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715592955; x=1747128955;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=FBRM1OdC9cKzr+j3pNvlhcc8bnkFfM6GEY8jvy5lak8=;
  b=ewEaTqah2jnVNhwHiR1vYQBtpuCqTL/iJ8/IhR8NwOLB9QV+JPv9QLR4
   oe5jMj6FSGFBZ+DqGzV3bmJ1Wyq85iWTdqrhi3NBGdtqg3q2JVoO17XAC
   n702jxDbp+O0dRslGnConY4elW0HvPw4m1qCSjabDAHUX0nIJi1kzsoYD
   E=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="396039285"
X-Farcaster-Flow-ID: f3d64704-d3dd-4a0e-82e9-75f995f45500
Message-ID: <a74001e4-7ae1-48f7-854d-2a8aeb2ff8da@amazon.com>
Date: Mon, 13 May 2024 10:35:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 12/27] x86/mapcache: Initialise the mapcache
 for the idle domain
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Wei Wang <wawei@amazon.de>, Julien Grall <jgrall@amazon.com>,
	<xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-13-eliasely@amazon.com>
 <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.95.164.111]
X-ClientProxiedBy: EX19D045UWA001.ant.amazon.com (10.13.139.83) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)



On 20/02/2024 10:51, Jan Beulich wrote:
> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -750,9 +750,16 @@ int arch_domain_create(struct domain *d,
>>   
>>       spin_lock_init(&d->arch.e820_lock);
>>   
>> +    if ( (rc = mapcache_domain_init(d)) != 0)
>> +    {
>> +        free_perdomain_mappings(d);
>> +        return rc;
>> +    }
>> +
>>       /* Minimal initialisation for the idle domain. */
>>       if ( unlikely(is_idle_domain(d)) )
>>       {
>> +        struct page_info *pg = d->arch.perdomain_l3_pg;
>>           static const struct arch_csw idle_csw = {
>>               .from = paravirt_ctxt_switch_from,
>>               .to   = paravirt_ctxt_switch_to,
>> @@ -763,6 +770,9 @@ int arch_domain_create(struct domain *d,
>>   
>>           d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>>   
>> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
>> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
>> +
>>           return 0;
>>       }
> 
> Why not add another call to mapcache_domain_init() right here, allowing
> a more specific panic() to be invoked in case of failure (compared to
> the BUG_ON() upon failure of creation of the idle domain as a whole)?
> Then the other mapcache_domain_init() call doesn't need moving a 2nd
> time in close succession.
> 

To be honest, I don't really like the idea of having twice the same call 
just for the benefit of having a panic() call in case of failure for the 
idle domain.

If you don't mind, I'd rather keep just a single call to 
mapcache_domain_init() as it is now.

Elias


From xen-devel-bounces@lists.xenproject.org Mon May 13 10:26:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 10:26:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720786.1123702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6St2-0004Fv-QE; Mon, 13 May 2024 10:26:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720786.1123702; Mon, 13 May 2024 10:26:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6St2-0004Fo-Nk; Mon, 13 May 2024 10:26:28 +0000
Received: by outflank-mailman (input) for mailman id 720786;
 Mon, 13 May 2024 10:26:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6St0-0004Fe-Pk; Mon, 13 May 2024 10:26:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6St0-0001fG-No; Mon, 13 May 2024 10:26:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6St0-0005y4-CV; Mon, 13 May 2024 10:26:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6St0-0007pj-C1; Mon, 13 May 2024 10:26:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EmBJBh+/C5xFsAwhSoL5TPBFck0aGMFoDuhUy20LURw=; b=GFjtoD8W9nb0ZR9jNZSA0Odpd2
	ZIqGQVee1lXkF/fGjbw7LX22XPUE+l0Ww6N41pbYvqlszfJZ+Y1ErBpC6IBWlleORAUJGHNOpyWv9
	+D0bSBg7P5jLWmcIvibIRGoSBkDGsSRdnN5anr6d1GAYwGe+h2Pv5W4wCwS+7bTPFgs0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185986-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185986: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6
X-Osstest-Versions-That:
    linux=cf87f46fd34d6c19283d9625a7822f20d90b64a4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 May 2024 10:26:26 +0000

flight 185986 linux-linus real [real]
flight 185989 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185986/
http://logs.test-lab.xenproject.org/osstest/logs/185989/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 185989-retest
 test-armhf-armhf-xl           8 xen-boot            fail pass in 185989-retest
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 185989-retest
 test-armhf-armhf-libvirt-vhd  8 xen-boot            fail pass in 185989-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 185989 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 185989 never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check fail in 185989 never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check fail in 185989 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 185989 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 185989 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185977
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185982
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185982
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6
baseline version:
 linux                cf87f46fd34d6c19283d9625a7822f20d90b64a4

Last test of basis   185982  2024-05-11 16:13:39 Z    1 days
Failing since        185984  2024-05-12 16:44:05 Z    0 days    2 attempts
Testing same since   185986  2024-05-12 23:41:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Borislav Petkov (AMD) <bp@alien8.de>
  Christian Borntraeger <borntraeger@linux.ibm.com>
  Jean-Philippe Brucker <jean-philippe@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mickaël Salaün <mic@digikod.net>
  Paolo Bonzini <pbonzini@redhat.com>
  Sean Christopherson <seanjc@google.com>
  Serge Semin <fancer.lancer@gmail.com>
  Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
  Thomas Gleixner <tglx@linutronix.de>
  Yuezhang Mo <Yuezhang.Mo@sony.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   cf87f46fd34d..a38297e3fb01  a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon May 13 10:51:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 10:51:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720795.1123712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TGy-0008U9-Rc; Mon, 13 May 2024 10:51:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720795.1123712; Mon, 13 May 2024 10:51:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TGy-0008U2-OD; Mon, 13 May 2024 10:51:12 +0000
Received: by outflank-mailman (input) for mailman id 720795;
 Mon, 13 May 2024 10:51:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TGy-0008Tw-Eq
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 10:51:12 +0000
Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com
 [52.119.213.154]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b4a05f2f-1116-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 12:51:09 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2])
 by smtp-border-fw-52004.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 10:51:07 +0000
Received: from EX19MTAEUC002.ant.amazon.com [10.0.10.100:45430]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.28.177:2525]
 with esmtp (Farcaster)
 id b5d6bb5e-23a1-4f32-8db6-42871781f0ea; Mon, 13 May 2024 10:51:06 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUC002.ant.amazon.com (10.252.51.245) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 10:51:05 +0000
Received: from [10.95.164.111] (10.95.164.111) by
 EX19D018EUA002.ant.amazon.com (10.252.50.146) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 10:51:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4a05f2f-1116-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715597469; x=1747133469;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=USEHK303lNucM4WrYY3diktgzvI+QaFCKPi/PxCaKFk=;
  b=pP60BMFKw5x+nwWB4mBXMYvB74gSchwNn9kYAFUb6O6SQLNM/UI2iTm8
   /b4nUPIBgVvxLWiM4zlDjYLA/eVSkXK3ztOCeoMohDwDchf2oL/AWO1Iv
   e3FBr+t70jIX1KlmSa2JgMWb0/nz8fussqNIIPq253H30hKsgUhnAeHVQ
   Q=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="204717771"
X-Farcaster-Flow-ID: b5d6bb5e-23a1-4f32-8db6-42871781f0ea
Message-ID: <20fb9bf9-b24a-4119-807b-c0a4fd6b7439@amazon.com>
Date: Mon, 13 May 2024 11:50:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 13/27] x86: Add a boot option to enable and
 disable the direct map
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Julien Grall
	<jgrall@amazon.com>, <xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-14-eliasely@amazon.com>
 <18795bde-bfd5-41ab-bef5-f74819bae956@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <18795bde-bfd5-41ab-bef5-f74819bae956@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.95.164.111]
X-ClientProxiedBy: EX19D031UWC002.ant.amazon.com (10.13.139.212) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)



On 20/02/2024 11:14, Jan Beulich wrote:
> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>> --- a/xen/arch/x86/Kconfig
>> +++ b/xen/arch/x86/Kconfig
>> @@ -29,6 +29,7 @@ config X86
>>   	select HAS_UBSAN
>>   	select HAS_VPCI if HVM
>>   	select NEEDS_LIBELF
>> +	select HAS_SECRET_HIDING
> 
> Please respect alphabetic sorting. As to "secret hiding" - personally I
> consider this too generic a term. This is about limiting the direct map. Why
> not name the option then accordingly?
> 

I think it is a fairly decent name, would you have any suggestion? 
Otherwise I will just stick to it.

>> --- a/xen/arch/x86/include/asm/mm.h
>> +++ b/xen/arch/x86/include/asm/mm.h
>> @@ -620,10 +620,18 @@ void write_32bit_pse_identmap(uint32_t *l2);
>>   /*
>>    * x86 maps part of physical memory via the directmap region.
>>    * Return whether the range of MFN falls in the directmap region.
>> + *
>> + * When boot command line sets directmap=no, we will not have a direct map at
>> + * all so this will always return false.
>>    */
> 
> As with the command line doc, please state the full truth.
> 
>>   static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>>   {
>> -    unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
>> +    unsigned long eva;
>> +
>> +    if ( !has_directmap() )
>> +        return false;
> 
> Hmm. The sole user of this function is init_node_heap(). Would it perhaps make
> sense to simply map the indicated number of pages then? init_node_heap() would
> fall back to xmalloc(), so the data will be in what's left of the directmap
> anyway.
> 

There will be more users of arch_mfns_in_directmap() in the following 
patches.

>> +    eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
> 
> Irrespective I don't see a need to replace the initializer by an assignment.

I guess it was to avoid the useless min() computation in case directmap 
is disabled. I can put it back to what it was.

> 
>> --- a/xen/common/Kconfig
>> +++ b/xen/common/Kconfig
>> @@ -83,6 +83,23 @@ config HAS_UBSAN
>>   config MEM_ACCESS_ALWAYS_ON
>>   	bool
>>   
>> +config HAS_SECRET_HIDING
>> +	bool
> 
> This again wants placing suitably among the other HAS_*.
> 
>> +config SECRET_HIDING
>> +    bool "Secret hiding"
>> +    depends on HAS_SECRET_HIDING
>> +    ---help---
>> +    The directmap contains mapping for most of the RAM which makes domain
>> +    memory easily accessible. While making the performance better, it also makes
>> +    the hypervisor more vulnerable to speculation attacks.
>> +
>> +    Enabling this feature will allow the user to decide whether the memory
>> +    is always mapped at boot or mapped only on demand (see the command line
>> +    option "directmap").
>> +
>> +    If unsure, say N.
> 
> Also as an alternative did you consider making this new setting merely
> control the default of opt_directmap? Otherwise the variable shouldn't exist
> at all when the Kconfig option is off, but rather be #define-d to "true" in
> that case.

I am not sure to understand why the option shouldn't exist at all when 
Kconfig option is off.

If SECRET_HIDING option is off, then opt_directmap must be 
unconditionally set to true. If SECRET_HIDING option is on, then 
opt_directmap value depends on the commandline option.

The corresponding wrapper, has_directmap(), will be used in multiple 
location in follow-up patch. I don't really see how you want to do.

>> --- a/xen/include/xen/mm.h
>> +++ b/xen/include/xen/mm.h
>> @@ -165,6 +165,13 @@ extern unsigned long max_page;
>>   extern unsigned long total_pages;
>>   extern paddr_t mem_hotplug;
>>   
>> +extern bool opt_directmap;
>> +
>> +static inline bool has_directmap(void)
>> +{
>> +    return opt_directmap;
>> +}
> 
> If opt_directmap isn't static, I see little point in having such a wrapper.
> If there are reasons, I think they want stating in the description.

I don't think there is a specific reason to be mentioned, if you really 
wish to, I can remove it.

> On the whole: Is the placement of this patch in the series an indication
> that as of here all directmap uses have gone away? If so, what's the rest of
> the series about? Alternatively isn't use of this option still problematic
> at this point of the series? Whichever way it is - this wants clarifying in
> the description.

This patch is not an indication that all directmap uses have been 
removed. We need to know in follow-up patch whether or not the option is 
enabled and so we have to introduce this patch here.

At this point in the series, the feature is not yet complete.


Elias



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720804.1123722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tae-00036Y-EO; Mon, 13 May 2024 11:11:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720804.1123722; Mon, 13 May 2024 11:11:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tae-00036R-BD; Mon, 13 May 2024 11:11:32 +0000
Received: by outflank-mailman (input) for mailman id 720804;
 Mon, 13 May 2024 11:11:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tad-00036L-OS
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:31 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8afc0f84-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:29 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:25 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.0.204:12984]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.23.176:2525]
 with esmtp (Farcaster)
 id 8b3f6da4-aae2-4c9f-8d43-3c06064c01e3; Mon, 13 May 2024 11:11:25 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:24 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:24 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8afc0f84-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598689; x=1747134689;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=FHUsw7Ke1dSe493VXWfS2kJQWXB8M4+Qm3akn5JUvwU=;
  b=iBDESHQUkfNCXzIQRI1X1h3sA0OlUTZeewGcgq+mO2n2A0vwY8QIR2Yn
   wmvvDz6xOqlXrZ3+89IJNj3edcb2GhAyE0idRwDXmXkihYYXxI0l2dAkP
   cH+84IYFCduc6x4P05bhy/moIiCGgWI3J45ZVThwM5DFWrwijlpOFmZnJ
   U=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88825749"
X-Farcaster-Flow-ID: 8b3f6da4-aae2-4c9f-8d43-3c06064c01e3
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>
Subject: [PATCH V3 00/19] Remove the directmap
Date: Mon, 13 May 2024 11:10:58 +0000
Message-ID: <20240513111117.68828-1-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Hi all,

A few years ago, Wei Liu implemented a PoC to remove the directmap
from Xen. The last version was sent by Hongyan Xia [1].

I will start with thanking both Wei and Hongyan for the initial work
to upstream the feature. A lot of patches already went in and this is
the last few patches missing to effectively enable the feature.

=== What is the directmap? ===

At the moment, on both arm64 and x86, most of the RAM is mapped
in Xen address space. This means that domain memory is easily
accessible in Xen.

=== Why do we want to remove the directmap? ===

(Summarizing my understanding of the previous discussion)

Speculation attacks (like Spectre SP1) rely on loading piece of memory
in the cache. If the region is not mapped then it can't be loaded.

So removing reducing the amount of memory mapped in Xen will also
reduce the surface attack.

=== What's the performance impact? ===

As the guest memory is not always mapped, then the cost of mapping
will increase. I haven't done the numbers with this new version, but
some measurement were provided in the previous version for x86.

=== Improvement possible ===

The known area to improve on x86 are:
   * Mapcache: There was a patch sent by Hongyan:
     https://lore.kernel.org/xen-devel/4058e92ce21627731c49b588a95809dc0affd83a.1581015491.git.hongyxia@amazon.com/
   * EPT: At the moment an guest page-tabel walk requires about 20 map/unmap.
     This will have an very high impact on the performance. We need to decide
     whether keep the EPT always mapped is a problem

The original series didn't have support for Arm64. But as there were
some interest, I have provided a PoC.

There are more extra work for Arm64:
   * The mapcache is quite simple. We would investigate the performance
   * The mapcache should be made compliant to the Arm Arm (this is now
     more critical).
   * We will likely have the same problem as for the EPT.
   * We have no support for merging table to a superpage, neither
     free empty page-tables. (See more below)

=== Implementation ===

The subject is probably a misnomer. The directmap is still present but
the RAM is not mapped by default. Instead, the region will still be used
to map pages allocate via alloc_xenheap_pages().

The advantage is the solution is simple (so IHMO good enough for been
merged as a tech preview). The disadvantage is the page allocator is not
trying to keep all the xenheap pages together. So we may end up to have
an increase of page table usage.

In the longer term, we should consider to remove the direct map
completely and switch to vmap(). The main problem with this approach
is it is frequent to use mfn_to_virt() in the code. So we would need
to cache the mapping (maybe in the struct page_info).

=== Why arm32 is not covered? ===

On Arm32, the domheap and xenheap is always separated. So by design
the guest memory is not mapped by default.

At this stage, it seems unnecessary to have to map/unmap xenheap pages
every time they are allocated.

=== Why not using a separate domheap and xenheap? ===

While a separate xenheap/domheap reduce the page-table usage (all
xenheap pages are contiguous and could be always mapped), it is also
currently less scalable because the split is fixed at boot time (XXX:
Can this be dynamic?).

=== Future of secret-free hypervisor ===

There are some information in an e-mail from Andrew a few years ago:

https://lore.kernel.org/xen-devel/e3219697-0759-39fc-2486-715cdec1ca9e@citrix.com/

Cheers,

[1] https://lore.kernel.org/xen-devel/cover.1588278317.git.hongyxia@amazon.com/

*** BLURB HERE ***

Elias El Yandouzi (3):
  xen/x86: Add build assertion for fixmap entries
  Rename mfn_to_virt() calls
  Rename maddr_to_virt() calls

Hongyan Xia (9):
  x86: Create per-domain mapping of guest_root_pt
  x86/pv: Rewrite how building PV dom0 handles domheap mappings
  x86/mapcache: Initialise the mapcache for the idle domain
  x86: Add a boot option to enable and disable the direct map
  x86/domain_page: Remove the fast paths when mfn is not in the
    directmap
  xen/page_alloc: Add a path for xenheap when there is no direct map
  x86/setup: Leave early boot slightly earlier
  x86/setup: vmap heap nodes when they are outside the direct map
  x86/setup: Do not create valid mappings when directmap=no

Julien Grall (5):
  xen/x86: Add support for the PMAP
  xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
  xen/arm64: mm: Use per-pCPU page-tables
  xen/arm64: Implement a mapcache for arm64
  xen/arm64: Allow the admin to enable/disable the directmap

Wei Liu (2):
  x86/pv: Domheap pages should be mapped while relocating initrd
  x86: Lift mapcache variable to the arch level

 docs/misc/xen-command-line.pandoc             | 12 +++
 xen/arch/arm/Kconfig                          |  2 +-
 xen/arch/arm/arm64/mmu/mm.c                   | 45 ++++++++-
 xen/arch/arm/domain_page.c                    | 50 +++++++++-
 xen/arch/arm/include/asm/arm32/mm.h           |  8 --
 xen/arch/arm/include/asm/arm64/mm.h           |  7 +-
 xen/arch/arm/include/asm/domain_page.h        | 13 +++
 xen/arch/arm/include/asm/mm.h                 |  9 ++
 xen/arch/arm/include/asm/mmu/layout.h         | 13 ++-
 xen/arch/arm/include/asm/mmu/mm.h             |  2 +
 xen/arch/arm/mm.c                             |  1 +
 xen/arch/arm/mmu/pt.c                         | 12 +--
 xen/arch/arm/mmu/setup.c                      | 27 ++---
 xen/arch/arm/mmu/smpboot.c                    | 30 ++----
 xen/arch/arm/setup.c                          |  2 +
 xen/arch/x86/Kconfig                          |  2 +
 xen/arch/x86/dmi_scan.c                       |  4 +-
 xen/arch/x86/domain.c                         | 12 ++-
 xen/arch/x86/domain_page.c                    | 74 +++++++++-----
 xen/arch/x86/hvm/dom0_build.c                 |  4 +-
 xen/arch/x86/include/asm/config.h             | 10 +-
 xen/arch/x86/include/asm/domain.h             | 13 +--
 xen/arch/x86/include/asm/fixmap.h             |  9 ++
 .../x86/include/asm/mach-default/bios_ebda.h  |  2 +-
 xen/arch/x86/include/asm/mm.h                 |  6 ++
 xen/arch/x86/include/asm/page.h               |  8 +-
 xen/arch/x86/include/asm/pmap.h               | 25 +++++
 xen/arch/x86/include/asm/x86_64/page.h        |  2 +-
 xen/arch/x86/mm.c                             | 18 +++-
 xen/arch/x86/mpparse.c                        |  2 +-
 xen/arch/x86/pv/dom0_build.c                  | 70 +++++++++----
 xen/arch/x86/pv/domain.c                      | 36 +++++++
 xen/arch/x86/setup.c                          | 98 ++++++++++++++++---
 xen/arch/x86/tboot.c                          |  2 +-
 xen/arch/x86/x86_64/asm-offsets.c             |  1 +
 xen/arch/x86/x86_64/entry.S                   |  8 ++
 xen/arch/x86/x86_64/mm.c                      | 26 +++--
 xen/common/Kconfig                            | 17 ++++
 xen/common/efi/boot.c                         | 23 +++--
 xen/common/page_alloc.c                       | 89 ++++++++++++++---
 xen/common/trace.c                            |  8 +-
 xen/include/xen/mm.h                          |  7 ++
 42 files changed, 630 insertions(+), 179 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/domain_page.h
 create mode 100644 xen/arch/x86/include/asm/pmap.h

-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720805.1123733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tao-0003Mx-Mi; Mon, 13 May 2024 11:11:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720805.1123733; Mon, 13 May 2024 11:11:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tao-0003Mq-IJ; Mon, 13 May 2024 11:11:42 +0000
Received: by outflank-mailman (input) for mailman id 720805;
 Mon, 13 May 2024 11:11:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tan-0003MC-MB
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:41 +0000
Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 91fc7d4b-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:40 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-6002.iad6.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:38 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:11740]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id 420086cb-3f7c-44b0-a1f7-b23bc039bef8; Mon, 13 May 2024 11:11:37 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:33 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:33 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91fc7d4b-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598701; x=1747134701;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=X8yfiUTton8c3NyUnbJj3YCfUh5a4EGoBhji2snIQBs=;
  b=rXKyaAgV1lvh63QuvEmKSNGg4+BmWl+/lllqKr0xgmXxbz3CAhP9LftO
   yGblmu4J4nZ6iTYP9ji5iug3plCzXcaL4c25mPFHzQKL7eVfDoI1w6zat
   wy2cUlo0VpdSv9QeVg84I+20RtAZDqJtgAGyU14KBaZ4G/orarAjm0+/2
   I=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="406374361"
X-Farcaster-Flow-ID: 420086cb-3f7c-44b0-a1f7-b23bc039bef8
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Wei Liu
	<wei.liu2@citrix.com>, Wei Wang <wawei@amazon.de>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 02/19] x86/pv: Domheap pages should be mapped while relocating initrd
Date: Mon, 13 May 2024 11:11:00 +0000
Message-ID: <20240513111117.68828-3-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Wei Liu <wei.liu2@citrix.com>

Xen shouldn't use domheap page as if they were xenheap pages. Map and
unmap pages accordingly.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Rename commit title
        * Rework the for loop copying the pages

    Changes in V2:
        * Get rid of mfn_to_virt
        * Don't open code copy_domain_page()

    Changes since Hongyan's version:
        * Add missing newline after the variable declaration

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index d8043fa58a..807296c280 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -618,18 +618,24 @@ int __init dom0_construct_pv(struct domain *d,
         if ( d->arch.physaddr_bitsize &&
              ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) )
         {
+            unsigned int nr_pages = 1UL << order;
+
             order = get_order_from_pages(count);
             page = alloc_domheap_pages(d, order, MEMF_no_scrub);
             if ( !page )
                 panic("Not enough RAM for domain 0 initrd\n");
+
             for ( count = -count; order--; )
                 if ( count & (1UL << order) )
                 {
                     free_domheap_pages(page, order);
                     page += 1UL << order;
+                    nr_pages -= 1UL << order;
                 }
-            memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start),
-                   initrd_len);
+
+            for ( ; nr_pages-- ; page++, mfn++ )
+                copy_domain_page(page_to_mfn(page), _mfn(mfn));
+
             mpt_alloc = (paddr_t)initrd->mod_start << PAGE_SHIFT;
             init_domheap_pages(mpt_alloc,
                                mpt_alloc + PAGE_ALIGN(initrd_len));
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720806.1123742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taq-0003d2-TL; Mon, 13 May 2024 11:11:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720806.1123742; Mon, 13 May 2024 11:11:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taq-0003cs-QA; Mon, 13 May 2024 11:11:44 +0000
Received: by outflank-mailman (input) for mailman id 720806;
 Mon, 13 May 2024 11:11:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tap-00036L-LY
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:43 +0000
Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com
 [207.171.190.10]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9272aed3-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:41 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-33001.sea14.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:34 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.44.209:25808]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.23.176:2525]
 with esmtp (Farcaster)
 id 2912e586-6345-4f18-afe7-466c4c067f3f; Mon, 13 May 2024 11:11:32 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:31 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:31 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9272aed3-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598702; x=1747134702;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=l8CclplVrvwG16aVcnUczikHaxmRbby77xL0adFNfQA=;
  b=IAE9KB0CWo9i/mgLaP9do3dwd/7oSSd55nljFX3D5dcVEw7gS5ZoURAO
   8FQtk8VFO3VVraM99AkTrJwCWVcTOk6IZ/x+O+anX2VYBH8FpOm8VU0mw
   3K2Xmym5RuRGtN/xB84AYCdq2H25RJa7HQLseVMpS2yiAJrH5gJ5CK8of
   o=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="344136745"
X-Farcaster-Flow-ID: 2912e586-6345-4f18-afe7-466c4c067f3f
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
Date: Mon, 13 May 2024 11:10:59 +0000
Message-ID: <20240513111117.68828-2-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Create a per-domain mapping of PV guest_root_pt as direct map is being
removed.

Note that we do not map and unmap root_pgt for now since it is still a
xenheap page.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Rename SHADOW_ROOT
        * Haven't addressed the potentially over-allocation issue as I don't get it

    Changes in V2:
        * Rework the shadow perdomain mapping solution in the follow-up patches

    Changes since Hongyan's version:
        * Remove the final dot in the commit title

diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
index ab7288cb36..5d710384df 100644
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -203,7 +203,7 @@ extern unsigned char boot_edid_info[128];
 /* Slot 260: per-domain mappings (including map cache). */
 #define PERDOMAIN_VIRT_START    (PML4_ADDR(260))
 #define PERDOMAIN_SLOT_MBYTES   (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER))
-#define PERDOMAIN_SLOTS         3
+#define PERDOMAIN_SLOTS         4
 #define PERDOMAIN_VIRT_SLOT(s)  (PERDOMAIN_VIRT_START + (s) * \
                                  (PERDOMAIN_SLOT_MBYTES << 20))
 /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */
@@ -317,6 +317,14 @@ extern unsigned long xen_phys_start;
 #define ARG_XLAT_START(v)        \
     (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
 
+/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */
+#define PV_ROOT_PT_MAPPING_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
+#define PV_ROOT_PT_MAPPING_ENTRIES      MAX_VIRT_CPUS
+
+/* The address of a particular VCPU's PV_ROOT_PT */
+#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \
+    (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))
+
 #define ELFSIZE 64
 
 #define ARCH_CRASH_SAVE_VMCOREINFO
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index f5daeb182b..8a97530607 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -272,6 +272,7 @@ struct time_scale {
 struct pv_domain
 {
     l1_pgentry_t **gdt_ldt_l1tab;
+    l1_pgentry_t **root_pt_l1tab;
 
     atomic_t nr_l4_pages;
 
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc73..efdf20f775 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -505,6 +505,13 @@ void share_xen_page_with_guest(struct page_info *page, struct domain *d,
     nrspin_unlock(&d->page_alloc_lock);
 }
 
+#define pv_root_pt_idx(v) \
+    ((v)->vcpu_id >> PAGETABLE_ORDER)
+
+#define pv_root_pt_pte(v) \
+    ((v)->domain->arch.pv.root_pt_l1tab[pv_root_pt_idx(v)] + \
+     ((v)->vcpu_id & (L1_PAGETABLE_ENTRIES - 1)))
+
 void make_cr3(struct vcpu *v, mfn_t mfn)
 {
     struct domain *d = v->domain;
@@ -524,6 +531,13 @@ void write_ptbase(struct vcpu *v)
 
     if ( is_pv_vcpu(v) && v->domain->arch.pv.xpti )
     {
+        mfn_t guest_root_pt = _mfn(MASK_EXTR(v->arch.cr3, PAGE_MASK));
+        l1_pgentry_t *pte = pv_root_pt_pte(v);
+
+        ASSERT(v == current);
+
+        l1e_write(pte, l1e_from_mfn(guest_root_pt, __PAGE_HYPERVISOR_RO));
+
         cpu_info->root_pgt_changed = true;
         cpu_info->pv_cr3 = __pa(this_cpu(root_pgt));
         if ( new_cr4 & X86_CR4_PCIDE )
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 2a445bb17b..1b025986f7 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -288,6 +288,21 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
                               1U << GDT_LDT_VCPU_SHIFT);
 }
 
+static int pv_create_root_pt_l1tab(struct vcpu *v)
+{
+    return create_perdomain_mapping(v->domain,
+                                    PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v),
+                                    1, v->domain->arch.pv.root_pt_l1tab,
+                                    NULL);
+}
+
+static void pv_destroy_root_pt_l1tab(struct vcpu *v)
+
+{
+    destroy_perdomain_mapping(v->domain,
+                              PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v), 1);
+}
+
 void pv_vcpu_destroy(struct vcpu *v)
 {
     if ( is_pv_32bit_vcpu(v) )
@@ -297,6 +312,7 @@ void pv_vcpu_destroy(struct vcpu *v)
     }
 
     pv_destroy_gdt_ldt_l1tab(v);
+    pv_destroy_root_pt_l1tab(v);
     XFREE(v->arch.pv.trap_ctxt);
 }
 
@@ -311,6 +327,13 @@ int pv_vcpu_initialise(struct vcpu *v)
     if ( rc )
         return rc;
 
+    if ( v->domain->arch.pv.xpti )
+    {
+        rc = pv_create_root_pt_l1tab(v);
+        if ( rc )
+            goto done;
+    }
+
     BUILD_BUG_ON(X86_NR_VECTORS * sizeof(*v->arch.pv.trap_ctxt) >
                  PAGE_SIZE);
     v->arch.pv.trap_ctxt = xzalloc_array(struct trap_info, X86_NR_VECTORS);
@@ -346,10 +369,12 @@ void pv_domain_destroy(struct domain *d)
 
     destroy_perdomain_mapping(d, GDT_LDT_VIRT_START,
                               GDT_LDT_MBYTES << (20 - PAGE_SHIFT));
+    destroy_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START, PV_ROOT_PT_MAPPING_ENTRIES);
 
     XFREE(d->arch.pv.cpuidmasks);
 
     FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab);
+    FREE_XENHEAP_PAGE(d->arch.pv.root_pt_l1tab);
 }
 
 void noreturn cf_check continue_pv_domain(void);
@@ -371,6 +396,12 @@ int pv_domain_initialise(struct domain *d)
         goto fail;
     clear_page(d->arch.pv.gdt_ldt_l1tab);
 
+    d->arch.pv.root_pt_l1tab =
+        alloc_xenheap_pages(0, MEMF_node(domain_to_node(d)));
+    if ( !d->arch.pv.root_pt_l1tab )
+        goto fail;
+    clear_page(d->arch.pv.root_pt_l1tab);
+
     if ( levelling_caps & ~LCAP_faulting &&
          (d->arch.pv.cpuidmasks = xmemdup(&cpuidmask_defaults)) == NULL )
         goto fail;
@@ -381,6 +412,11 @@ int pv_domain_initialise(struct domain *d)
     if ( rc )
         goto fail;
 
+    rc = create_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START,
+                                  PV_ROOT_PT_MAPPING_ENTRIES, NULL, NULL);
+    if ( rc )
+        goto fail;
+
     d->arch.ctxt_switch = &pv_csw;
 
     d->arch.pv.xpti = is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu;
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 630bdc3945..c1ae5013af 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -80,6 +80,7 @@ void __dummy__(void)
 
 #undef OFFSET_EF
 
+    OFFSET(VCPU_id, struct vcpu, vcpu_id);
     OFFSET(VCPU_processor, struct vcpu, processor);
     OFFSET(VCPU_domain, struct vcpu, domain);
     OFFSET(VCPU_vcpu_info, struct vcpu, vcpu_info_area.map);
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index df015589ce..c1377da7a5 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -162,7 +162,15 @@ FUNC_LOCAL(restore_all_guest)
         and   %rsi, %rdi
         and   %r9, %rsi
         add   %rcx, %rdi
+
+        /*
+         * The address in the vCPU cr3 is always mapped in the per-domain
+         * pv_root_pt virt area.
+         */
+        imul  $PAGE_SIZE, VCPU_id(%rbx), %esi
+        movabs $PV_ROOT_PT_MAPPING_VIRT_START, %rcx
         add   %rcx, %rsi
+
         mov   $ROOT_PAGETABLE_FIRST_XEN_SLOT, %ecx
         mov   root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rsi), %r8
         mov   %r8, root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rdi)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720807.1123752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tau-0003v2-9c; Mon, 13 May 2024 11:11:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720807.1123752; Mon, 13 May 2024 11:11:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tau-0003up-5Q; Mon, 13 May 2024 11:11:48 +0000
Received: by outflank-mailman (input) for mailman id 720807;
 Mon, 13 May 2024 11:11:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tas-0003MC-8y
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:46 +0000
Received: from smtp-fw-52003.amazon.com (smtp-fw-52003.amazon.com
 [52.119.213.152]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 950fd354-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:45 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52003.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:43 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.29.78:45873]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id 736196f1-3ae1-401f-9856-bbbafdb52cd2; Mon, 13 May 2024 11:11:42 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:36 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:36 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 950fd354-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598706; x=1747134706;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=asW3riQLlEfjH7OhZSjftd0GSOA/iaeDlEAWCyKlviw=;
  b=peW9GTg2yc7m5eRRxKAxWXd9GLGmCCT0K5GBFZHirpAeagQTRgYIjww6
   9AWZQ74UHVXv8QMqfBu38tt4eWf5+KSCHLXDLfrwsv5avDJFpezcykZC9
   zdOjr6Kavncv+CARdTWWOVfTg7VF5eCcXw87y87WUNDRAFZeD2iPhZ1xx
   Y=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="658281373"
X-Farcaster-Flow-ID: 736196f1-3ae1-401f-9856-bbbafdb52cd2
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Wei Liu
	<wei.liu2@citrix.com>, Wei Wang <wawei@amazon.de>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH V3 04/19] x86: Lift mapcache variable to the arch level
Date: Mon, 13 May 2024 11:11:02 +0000
Message-ID: <20240513111117.68828-5-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Wei Liu <wei.liu2@citrix.com>

It is going to be needed by HVM and idle domain as well, because without
the direct map, both need a mapcache to map pages.

This commit lifts the mapcache variable up and initialise it a bit earlier
for PV and HVM domains.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 20e83cf38b..507d704f16 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -851,6 +851,8 @@ int arch_domain_create(struct domain *d,
 
     psr_domain_init(d);
 
+    mapcache_domain_init(d);
+
     if ( is_hvm_domain(d) )
     {
         if ( (rc = hvm_domain_initialise(d, config)) != 0 )
@@ -858,8 +860,6 @@ int arch_domain_create(struct domain *d,
     }
     else if ( is_pv_domain(d) )
     {
-        mapcache_domain_init(d);
-
         if ( (rc = pv_domain_initialise(d)) != 0 )
             goto fail;
     }
diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index eac5e3304f..55e337aaf7 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
 #endif
 
     v = mapcache_current_vcpu();
-    if ( !v || !is_pv_vcpu(v) )
+    if ( !v )
         return mfn_to_virt(mfn_x(mfn));
 
-    dcache = &v->domain->arch.pv.mapcache;
-    vcache = &v->arch.pv.mapcache;
+    dcache = &v->domain->arch.mapcache;
+    vcache = &v->arch.mapcache;
     if ( !dcache->inuse )
         return mfn_to_virt(mfn_x(mfn));
 
@@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr)
     ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
 
     v = mapcache_current_vcpu();
-    ASSERT(v && is_pv_vcpu(v));
+    ASSERT(v);
 
-    dcache = &v->domain->arch.pv.mapcache;
+    dcache = &v->domain->arch.mapcache;
     ASSERT(dcache->inuse);
 
     idx = PFN_DOWN(va - MAPCACHE_VIRT_START);
     mfn = l1e_get_pfn(MAPCACHE_L1ENT(idx));
-    hashent = &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)];
+    hashent = &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)];
 
     local_irq_save(flags);
 
@@ -233,11 +233,9 @@ void unmap_domain_page(const void *ptr)
 
 int mapcache_domain_init(struct domain *d)
 {
-    struct mapcache_domain *dcache = &d->arch.pv.mapcache;
+    struct mapcache_domain *dcache = &d->arch.mapcache;
     unsigned int bitmap_pages;
 
-    ASSERT(is_pv_domain(d));
-
 #ifdef NDEBUG
     if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
         return 0;
@@ -261,12 +259,12 @@ int mapcache_domain_init(struct domain *d)
 int mapcache_vcpu_init(struct vcpu *v)
 {
     struct domain *d = v->domain;
-    struct mapcache_domain *dcache = &d->arch.pv.mapcache;
+    struct mapcache_domain *dcache = &d->arch.mapcache;
     unsigned long i;
     unsigned int ents = d->max_vcpus * MAPCACHE_VCPU_ENTRIES;
     unsigned int nr = PFN_UP(BITS_TO_LONGS(ents) * sizeof(long));
 
-    if ( !is_pv_vcpu(v) || !dcache->inuse )
+    if ( !dcache->inuse )
         return 0;
 
     if ( ents > dcache->entries )
@@ -293,7 +291,7 @@ int mapcache_vcpu_init(struct vcpu *v)
     BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES);
     for ( i = 0; i < MAPHASH_ENTRIES; i++ )
     {
-        struct vcpu_maphash_entry *hashent = &v->arch.pv.mapcache.hash[i];
+        struct vcpu_maphash_entry *hashent = &v->arch.mapcache.hash[i];
 
         hashent->mfn = ~0UL; /* never valid to map */
         hashent->idx = MAPHASHENT_NOTINUSE;
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index 8a97530607..7f0480d7a7 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -285,9 +285,6 @@ struct pv_domain
     /* Mitigate L1TF with shadow/crashing? */
     bool check_l1tf;
 
-    /* map_domain_page() mapping cache. */
-    struct mapcache_domain mapcache;
-
     struct cpuidmasks *cpuidmasks;
 };
 
@@ -326,6 +323,9 @@ struct arch_domain
 
     uint8_t scf; /* See SCF_DOM_MASK */
 
+    /* map_domain_page() mapping cache. */
+    struct mapcache_domain mapcache;
+
     union {
         struct pv_domain pv;
         struct hvm_domain hvm;
@@ -516,9 +516,6 @@ struct arch_domain
 
 struct pv_vcpu
 {
-    /* map_domain_page() mapping cache. */
-    struct mapcache_vcpu mapcache;
-
     unsigned int vgc_flags;
 
     struct trap_info *trap_ctxt;
@@ -618,6 +615,9 @@ struct arch_vcpu
 #define async_exception_state(t) async_exception_state[(t)-1]
     uint8_t async_exception_mask;
 
+    /* map_domain_page() mapping cache. */
+    struct mapcache_vcpu mapcache;
+
     /* Virtual Machine Extensions */
     union {
         struct pv_vcpu pv;
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720808.1123762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taw-0004D3-Gn; Mon, 13 May 2024 11:11:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720808.1123762; Mon, 13 May 2024 11:11:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taw-0004Cu-Dv; Mon, 13 May 2024 11:11:50 +0000
Received: by outflank-mailman (input) for mailman id 720808;
 Mon, 13 May 2024 11:11:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tav-0003MC-9W
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:49 +0000
Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com
 [52.119.213.150]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 96e9b605-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:48 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52002.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:46 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:62332]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 0d4825d0-82ef-4415-bce3-98ee92835dc2; Mon, 13 May 2024 11:11:45 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:42 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:42 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96e9b605-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598708; x=1747134708;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=ZJrhmMV/WsLJjJ2b/S8Feij3BU9bNlhqRg9vQL1/RK0=;
  b=Dr+pWSjzsuHSbbU8DvY6NanqzLZY6/Aqbhnb6fxlLgKy9em0bownbQFF
   XkVSkDz206yKhLB5H85kwIGsP6ZOIGB0GRncEKwN+ShJufpzGyKoPr10T
   ati6Uq51kcuJyIIdFIpk+EsmBBieiVoNwNyFEOCrFfINqOeuWQKo0NjFP
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="632473707"
X-Farcaster-Flow-ID: 0d4825d0-82ef-4415-bce3-98ee92835dc2
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 07/19] xen/x86: Add support for the PMAP
Date: Mon, 13 May 2024 11:11:05 +0000
Message-ID: <20240513111117.68828-8-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

PMAP will be used in a follow-up patch to bootstrap map domain
page infrastructure -- we need some way to map pages to setup the
mapcache without a direct map.

The functions pmap_{map, unmap} open code {set, clear}_fixmap to break
the loop.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    The PMAP infrastructure was upstream separately for Arm since
    Hongyan sent the secret-free hypervisor series. So this is a new
    patch to plumb the feature on x86.

    Changes in v2:
        * Declare PMAP entries earlier in fixed_addresses
        * Reword the commit message

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index b4ec0e582e..56feb0c564 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -27,6 +27,7 @@ config X86
 	select HAS_PCI
 	select HAS_PCI_MSI
 	select HAS_PIRQ
+	select HAS_PMAP
 	select HAS_SCHED_GRANULARITY
 	select HAS_SECRET_HIDING
 	select HAS_UBSAN
diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
index 516ec3fa6c..a7ac365fc6 100644
--- a/xen/arch/x86/include/asm/fixmap.h
+++ b/xen/arch/x86/include/asm/fixmap.h
@@ -21,6 +21,8 @@
 
 #include <xen/acpi.h>
 #include <xen/pfn.h>
+#include <xen/pmap.h>
+
 #include <asm/apicdef.h>
 #include <asm/msi.h>
 #include <acpi/apei.h>
@@ -53,6 +55,8 @@ enum fixed_addresses {
     FIX_PV_CONSOLE,
     FIX_XEN_SHARED_INFO,
 #endif /* CONFIG_XEN_GUEST */
+    FIX_PMAP_BEGIN,
+    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
     /* Everything else should go further down. */
     FIX_APIC_BASE,
     FIX_IO_APIC_BASE_0,
diff --git a/xen/arch/x86/include/asm/pmap.h b/xen/arch/x86/include/asm/pmap.h
new file mode 100644
index 0000000000..62746e191d
--- /dev/null
+++ b/xen/arch/x86/include/asm/pmap.h
@@ -0,0 +1,25 @@
+#ifndef __ASM_PMAP_H__
+#define __ASM_PMAP_H__
+
+#include <asm/fixmap.h>
+
+static inline void arch_pmap_map(unsigned int slot, mfn_t mfn)
+{
+    unsigned long linear = (unsigned long)fix_to_virt(slot);
+    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
+
+    ASSERT(!(l1e_get_flags(*pl1e) & _PAGE_PRESENT));
+
+    l1e_write_atomic(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR));
+}
+
+static inline void arch_pmap_unmap(unsigned int slot)
+{
+    unsigned long linear = (unsigned long)fix_to_virt(slot);
+    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
+
+    l1e_write_atomic(pl1e, l1e_empty());
+    flush_tlb_one_local(linear);
+}
+
+#endif /* __ASM_PMAP_H__ */
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720809.1123771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taz-0004YI-Ob; Mon, 13 May 2024 11:11:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720809.1123771; Mon, 13 May 2024 11:11:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Taz-0004Y3-LI; Mon, 13 May 2024 11:11:53 +0000
Received: by outflank-mailman (input) for mailman id 720809;
 Mon, 13 May 2024 11:11:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tay-00036L-Jf
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:52 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97a85db1-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:50 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:44 +0000
Received: from EX19MTAUEC001.ant.amazon.com [10.0.0.204:34895]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.20.152:2525]
 with esmtp (Farcaster)
 id 40ab5167-bece-4296-856c-e215ecf97d24; Mon, 13 May 2024 11:11:43 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEC001.ant.amazon.com (10.252.135.222) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:40 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:40 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97a85db1-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598711; x=1747134711;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=YiMscZoqlqA2XJ1v3gO3e8DXR6HvvHTgLBqJAfmepmI=;
  b=qVbWoC8u00vno4vtdVdhIe0vQsEqzRskkIkDSL+Ee7y37BHFVAndT+k/
   Os0HWQfRfGp0jWo/OV3WT03jsF6Uc0e58j+ibnY2RXJpOJJUrfDCrOMdf
   1hrWUp7VE1qub3RVSXzWloD+AqsUhTws6ZlPFHgm3eLz8qhR8uEhSpoUs
   8=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="294662337"
X-Farcaster-Flow-ID: 40ab5167-bece-4296-856c-e215ecf97d24
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 06/19] x86: Add a boot option to enable and disable the direct map
Date: Mon, 13 May 2024 11:11:04 +0000
Message-ID: <20240513111117.68828-7-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Also add a helper function to retrieve it. Change arch_mfns_in_direct_map
to check this option before returning.

This is added as a Kconfig option as well as a boot command line option.
While being generic, the Kconfig option is only usable for x86 at the moment.

Note that there remains some users of the directmap at this point. The option
is introduced now as it will be needed in follow-up patches.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in V2:
        * Introduce a Kconfig option
        * Reword the commit message
        * Make opt_directmap and helper generic

    Changes since Hongyan's version:
        * Reword the commit message
        * opt_directmap is only modified during boot so mark it as
          __ro_after_init

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index e760f3266e..743d343ffa 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
 hardware guarantees (with, where available and known to Xen, respective
 tweaks applied).
 
+### directmap (x86)
+> `= <boolean>`
+
+> Default: `true`
+
+Enable or disable the directmap region in Xen.
+
+By default, Xen creates the directmap region which maps physical memory
+in that region. Setting this to no will sparsely populate the directmap,
+blocking exploits that leak secrets via speculative memory access in the
+directmap.
+
 ### dma_bits
 > `= <integer>`
 
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 7e03e4bc55..b4ec0e582e 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -28,6 +28,7 @@ config X86
 	select HAS_PCI_MSI
 	select HAS_PIRQ
 	select HAS_SCHED_GRANULARITY
+	select HAS_SECRET_HIDING
 	select HAS_UBSAN
 	select HAS_VPCI if HVM
 	select NEEDS_LIBELF
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 98b66edaca..54d835f156 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -622,11 +622,17 @@ void write_32bit_pse_identmap(uint32_t *l2);
 /*
  * x86 maps part of physical memory via the directmap region.
  * Return whether the range of MFN falls in the directmap region.
+ *
+ * When boot command line sets directmap=no, the directmap will mostly be empty
+ * so this will always return false.
  */
 static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
 {
     unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
 
+    if ( !has_directmap() )
+        return false;
+
     return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
 }
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index f84e1cd79c..bd6b1184f5 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     if ( highmem_start )
         xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
 
+    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
+
     /*
      * Walk every RAM region and map it in its entirety (on x86/64, at least)
      * and notify it to the boot allocator.
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 565ceda741..856604068c 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -80,12 +80,29 @@ config HAS_PMAP
 config HAS_SCHED_GRANULARITY
 	bool
 
+config HAS_SECRET_HIDING
+	bool
+
 config HAS_UBSAN
 	bool
 
 config MEM_ACCESS_ALWAYS_ON
 	bool
 
+config SECRET_HIDING
+    bool "Secret hiding"
+    depends on HAS_SECRET_HIDING
+    help
+		The directmap contains mapping for most of the RAM which makes domain
+		memory easily accessible. While making the performance better, it also makes
+		the hypervisor more vulnerable to speculation attacks.
+
+		Enabling this feature will allow the user to decide whether the memory
+		is always mapped at boot or mapped only on demand (see the command line
+		option "directmap").
+
+		If unsure, say N.
+
 config MEM_ACCESS
 	def_bool MEM_ACCESS_ALWAYS_ON
 	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 7c1bdfc046..9b7e4721cd 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -174,6 +174,11 @@ paddr_t __ro_after_init mem_hotplug;
 static char __initdata opt_badpage[100] = "";
 string_param("badpage", opt_badpage);
 
+bool __ro_after_init opt_directmap = true;
+#ifdef CONFIG_HAS_SECRET_HIDING
+boolean_param("directmap", opt_directmap);
+#endif
+
 /*
  * no-bootscrub -> Free pages are not zeroed during boot.
  */
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 7561297a75..9d4f1f2d0d 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -167,6 +167,13 @@ extern unsigned long max_page;
 extern unsigned long total_pages;
 extern paddr_t mem_hotplug;
 
+extern bool opt_directmap;
+
+static inline bool has_directmap(void)
+{
+    return opt_directmap;
+}
+
 /*
  * Extra fault info types which are used to further describe
  * the source of an access violation.
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720811.1123782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb4-0004yx-31; Mon, 13 May 2024 11:11:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720811.1123782; Mon, 13 May 2024 11:11:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb3-0004yq-Vg; Mon, 13 May 2024 11:11:57 +0000
Received: by outflank-mailman (input) for mailman id 720811;
 Mon, 13 May 2024 11:11:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb2-00036L-Dh
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:56 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 99746a71-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:54 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:49 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:31352]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 27d3b449-45c7-420e-af1d-caf6889650d5; Mon, 13 May 2024 11:11:49 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:48 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:48 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99746a71-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598714; x=1747134714;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=FfhF1uGZGyTejdo4gJox/0kvgwcb9PzofNFjsbGWbZM=;
  b=A/GDk4m5tw5i08QZifo33Ex5KhH+zWL+SAEcium2mEUqNrQB0mRSX1u3
   1ru4Yz8o5aVk1tqx0l7aTVCp2p5zwScAq7ESz5LHXWUAyjZgPbZMQS6kg
   VoE9Nt6BxgxReTkN7UsoOHwXjsakAV41aISV9VQyD/ArowpEgqjDoJTke
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88736852"
X-Farcaster-Flow-ID: 27d3b449-45c7-420e-af1d-caf6889650d5
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 11/19] x86/setup: Leave early boot slightly earlier
Date: Mon, 13 May 2024 11:11:09 +0000
Message-ID: <20240513111117.68828-12-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When we do not have a direct map, memory for metadata of heap nodes in
init_node_heap() is allocated from xenheap, which needs to be mapped and
unmapped on demand. However, we cannot just take memory from the boot
allocator to create the PTEs while we are passing memory to the heap
allocator.

To solve this race, we leave early boot slightly sooner so that Xen PTE
pages are allocated from the heap instead of the boot allocator. We can
do this because the metadata for the 1st node is statically allocated,
and by the time we need memory to create mappings for the 2nd node, we
already have enough memory in the heap allocator in the 1st node.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index bd6b1184f5..f26c9799e4 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1751,6 +1751,22 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 
     numa_initmem_init(0, raw_max_page);
 
+    /*
+     * When we do not have a direct map, memory for metadata of heap nodes in
+     * init_node_heap() is allocated from xenheap, which needs to be mapped and
+     * unmapped on demand. However, we cannot just take memory from the boot
+     * allocator to create the PTEs while we are passing memory to the heap
+     * allocator during end_boot_allocator().
+     *
+     * To solve this race, we need to leave early boot before
+     * end_boot_allocator() so that Xen PTE pages are allocated from the heap
+     * instead of the boot allocator. We can do this because the metadata for
+     * the 1st node is statically allocated, and by the time we need memory to
+     * create mappings for the 2nd node, we already have enough memory in the
+     * heap allocator in the 1st node.
+     */
+    system_state = SYS_STATE_boot;
+
     if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) )
     {
         unsigned long lo = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
@@ -1782,8 +1798,6 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     else
         end_boot_allocator();
 
-    system_state = SYS_STATE_boot;
-
     bsp_stack = cpu_alloc_stack(0);
     if ( !bsp_stack )
         panic("No memory for BSP stack\n");
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:11:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:11:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720812.1123792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb5-0005Hk-FU; Mon, 13 May 2024 11:11:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720812.1123792; Mon, 13 May 2024 11:11:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb5-0005HA-9E; Mon, 13 May 2024 11:11:59 +0000
Received: by outflank-mailman (input) for mailman id 720812;
 Mon, 13 May 2024 11:11:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb3-00036L-Dw
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:57 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9aeed16a-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:54 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:54 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.0.204:23012]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.82.155:2525]
 with esmtp (Farcaster)
 id db499768-4150-42be-b874-549dc3061146; Mon, 13 May 2024 11:11:53 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:52 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:51 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9aeed16a-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598715; x=1747134715;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=roZUKjZFy3HRbWFvKqZPI52BDqwj8toyBUHm6ct47Gk=;
  b=NDcqzEFASk15KlCTwr8iJYGo6QEDkjCinUYtCaFMDf+yMI1+Tc8nJeYa
   ER9GvxjwQZajhjPbPHE/cQcRNEeGipMwPAAg+NUdKfrvrN7Gp470QCAYm
   zjYHWSLWDN+E3pv6c2e8dDSMzXDR9QLU+HJ4EbcAOvThN0uguXj9+JLKw
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="294662424"
X-Farcaster-Flow-ID: db499768-4150-42be-b874-549dc3061146
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 13/19] x86/setup: Do not create valid mappings when directmap=no
Date: Mon, 13 May 2024 11:11:11 +0000
Message-ID: <20240513111117.68828-14-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Create empty mappings in the second e820 pass. Also, destroy existing
direct map mappings created in the first pass.

To make xenheap pages visible in guests, it is necessary to create empty
L3 tables in the direct map even when directmap=no, since guest cr3s
copy idle domain's L4 entries, which means they will share mappings in
the direct map if we pre-populate idle domain's L4 entries and L3
tables. A helper is introduced for this.

Also, after the direct map is actually gone, we need to stop updating
the direct map in update_xen_mappings().

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index f26c9799e4..919347d8c2 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -978,6 +978,57 @@ static struct domain *__init create_dom0(const module_t *image,
 /* How much of the directmap is prebuilt at compile time. */
 #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
 
+/*
+ * This either populates a valid direct map, or allocates empty L3 tables and
+ * creates the L4 entries for virtual address between [start, end) in the
+ * direct map depending on has_directmap();
+ *
+ * When directmap=no, we still need to populate empty L3 tables in the
+ * direct map region. The reason is that on-demand xenheap mappings are
+ * created in the idle domain's page table but must be seen by
+ * everyone. Since all domains share the direct map L4 entries, they
+ * will share xenheap mappings if we pre-populate the L4 entries and L3
+ * tables in the direct map region for all RAM. We also rely on the fact
+ * that L3 tables are never freed.
+ */
+static void __init populate_directmap(uint64_t pstart, uint64_t pend,
+                                      unsigned int flags)
+{
+    unsigned long vstart = (unsigned long)__va(pstart);
+    unsigned long vend = (unsigned long)__va(pend);
+
+    if ( pstart >= pend )
+        return;
+
+    BUG_ON(vstart < DIRECTMAP_VIRT_START);
+    BUG_ON(vend > DIRECTMAP_VIRT_END);
+
+    if ( has_directmap() )
+        /* Populate valid direct map. */
+        BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart),
+                                PFN_DOWN(pend - pstart), flags));
+    else
+    {
+        /* Create empty L3 tables. */
+        unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1);
+
+        for ( ; vaddr < vend; vaddr += (1UL << L4_PAGETABLE_SHIFT) )
+        {
+            l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)];
+
+            if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
+            {
+                mfn_t mfn = alloc_boot_pages(1, 1);
+                void *v = map_domain_page(mfn);
+
+                clear_page(v);
+                UNMAP_DOMAIN_PAGE(v);
+                l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR));
+            }
+        }
+    }
+}
+
 void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 {
     const char *memmap_type = NULL, *loader, *cmdline = "";
@@ -1601,8 +1652,17 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         map_e = min_t(uint64_t, e,
                       ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT);
 
-        /* Pass mapped memory to allocator /before/ creating new mappings. */
+        /*
+         * Pass mapped memory to allocator /before/ creating new mappings.
+         * The direct map for the bottom 4GiB has been populated in the first
+         * e820 pass. In the second pass, we make sure those existing mappings
+         * are destroyed when directmap=no.
+         */
         init_boot_pages(s, min(map_s, e));
+        if ( !has_directmap() )
+            destroy_xen_mappings((unsigned long)__va(s),
+                                 (unsigned long)__va(min(map_s, e)));
+
         s = map_s;
         if ( s < map_e )
         {
@@ -1610,6 +1670,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
             map_s = (s + mask) & ~mask;
             map_e &= ~mask;
             init_boot_pages(map_s, map_e);
+            if ( !has_directmap() )
+                destroy_xen_mappings((unsigned long)__va(map_s),
+                                     (unsigned long)__va(map_e));
         }
 
         if ( map_s > map_e )
@@ -1623,8 +1686,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 
             if ( map_e < end )
             {
-                map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e),
-                                 PFN_DOWN(end - map_e), PAGE_HYPERVISOR);
+                populate_directmap(map_e, end, PAGE_HYPERVISOR);
                 init_boot_pages(map_e, end);
                 map_e = end;
             }
@@ -1633,13 +1695,11 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         {
             /* This range must not be passed to the boot allocator and
              * must also not be mapped with _PAGE_GLOBAL. */
-            map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e),
-                             PFN_DOWN(e - map_e), __PAGE_HYPERVISOR_RW);
+            populate_directmap(map_e, e, __PAGE_HYPERVISOR_RW);
         }
         if ( s < map_s )
         {
-            map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s),
-                             PFN_DOWN(map_s - s), PAGE_HYPERVISOR);
+            populate_directmap(s, map_s, PAGE_HYPERVISOR);
             init_boot_pages(s, map_s);
         }
     }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720813.1123798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb5-0005PR-VX; Mon, 13 May 2024 11:11:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720813.1123798; Mon, 13 May 2024 11:11:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb5-0005MO-P5; Mon, 13 May 2024 11:11:59 +0000
Received: by outflank-mailman (input) for mailman id 720813;
 Mon, 13 May 2024 11:11:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb4-00036L-J7
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:58 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9c2d2cf0-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:56 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:46 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.44.209:61768]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.82.155:2525]
 with esmtp (Farcaster)
 id c198cc80-af3a-4fc8-8dd3-4517e4aca5e4; Mon, 13 May 2024 11:11:46 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:39 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:38 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c2d2cf0-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598716; x=1747134716;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=TyQ1NOFs+JBNhEZ9nul6mULL97GoHUzPHp2fRNIcmm8=;
  b=G530L5nXYBuZsVfLoljrP8cOdAU7gMGf+W6vXAqf2ZvK8ZR6v92qoqjO
   yNaI5Q+nZzrtXjlThjlL+f+YojqFpGSCd/C2tJMdj48iZ/jouE25usPbP
   RHNQL66uA3eCifkHmnwvxo2++Jadt1JspDx7QEUIDm5VWZ8Gs80rCgYxx
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88736833"
X-Farcaster-Flow-ID: c198cc80-af3a-4fc8-8dd3-4517e4aca5e4
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Wei Wang <wawei@amazon.de>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 05/19] x86/mapcache: Initialise the mapcache for the idle domain
Date: Mon, 13 May 2024 11:11:03 +0000
Message-ID: <20240513111117.68828-6-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

In order to use the mapcache in the idle domain, we also have to
populate its page tables in the PERDOMAIN region, and we need to move
mapcache_domain_init() earlier in arch_domain_create().

Note, commit 'x86: lift mapcache variable to the arch level' has
initialised the mapcache for HVM domains. With this patch, PV, HVM,
idle domains now all initialise the mapcache.

Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

	Changes in V2:
          * Free resources if mapcache initialisation fails
          * Remove `is_idle_domain()` check from `create_perdomain_mappings()`

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 507d704f16..3303bdb53e 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -758,9 +758,16 @@ int arch_domain_create(struct domain *d,
 
     spin_lock_init(&d->arch.e820_lock);
 
+    if ( (rc = mapcache_domain_init(d)) != 0)
+    {
+        free_perdomain_mappings(d);
+        return rc;
+    }
+
     /* Minimal initialisation for the idle domain. */
     if ( unlikely(is_idle_domain(d)) )
     {
+        struct page_info *pg = d->arch.perdomain_l3_pg;
         static const struct arch_csw idle_csw = {
             .from = paravirt_ctxt_switch_from,
             .to   = paravirt_ctxt_switch_to,
@@ -771,6 +778,9 @@ int arch_domain_create(struct domain *d,
 
         d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
 
+        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
+            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
+
         return 0;
     }
 
@@ -851,8 +861,6 @@ int arch_domain_create(struct domain *d,
 
     psr_domain_init(d);
 
-    mapcache_domain_init(d);
-
     if ( is_hvm_domain(d) )
     {
         if ( (rc = hvm_domain_initialise(d, config)) != 0 )
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720814.1123810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb7-0005nM-Ai; Mon, 13 May 2024 11:12:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720814.1123810; Mon, 13 May 2024 11:12:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb7-0005mj-6k; Mon, 13 May 2024 11:12:01 +0000
Received: by outflank-mailman (input) for mailman id 720814;
 Mon, 13 May 2024 11:11:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb5-0003MC-9P
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:59 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9c794f31-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:58 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:48 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.29.78:34994]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id 1c94a46c-60d4-4506-be81-1115b85ee72d; Mon, 13 May 2024 11:11:47 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:47 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:46 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c794f31-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598719; x=1747134719;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=NU5YSGcrFgwo4jtZeOfCQS23Pg2dIdFW+HjZiUxxomg=;
  b=fS5mDXXPl03/RtWlL+jUwrLUG5JUGvj2gV0U1RfT9NdWe5QBCjf0Uld2
   95KFi2SyjdFDybWs5CsnU8R9IFpUUjfvDfKt6cpep6OUd5a8qoQRW85Ch
   +H+7Jh+4vRVZ2NmVy5y7VEJG0UrRqKPFlKQFz+RJkxxFpiQe2oHdTNMht
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="725407063"
X-Farcaster-Flow-ID: 1c94a46c-60d4-4506-be81-1115b85ee72d
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 10/19] xen/page_alloc: Add a path for xenheap when there is no direct map
Date: Mon, 13 May 2024 11:11:08 +0000
Message-ID: <20240513111117.68828-11-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When there is not an always-mapped direct map, xenheap allocations need
to be mapped and unmapped on-demand.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    I have left the call to map_pages_to_xen() and destroy_xen_mappings()
    in the split heap for now. I am not entirely convinced this is necessary
    because in that setup only the xenheap would be always mapped and
    this doesn't contain any guest memory (aside the grant-table).
    So map/unmapping for every allocation seems unnecessary.

    Changes in v2:
        * Fix remaining wrong indentation in alloc_xenheap_pages()

    Changes since Hongyan's version:
        * Rebase
        * Fix indentation in alloc_xenheap_pages()
        * Fix build for arm32

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 9b7e4721cd..dfb2c05322 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2242,6 +2242,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe)
 void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 {
     struct page_info *pg;
+    void *ret;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2250,17 +2251,36 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
     if ( unlikely(pg == NULL) )
         return NULL;
 
+    ret = page_to_virt(pg);
+
+    if ( !has_directmap() &&
+         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
+                          PAGE_HYPERVISOR) )
+    {
+        /* Failed to map xenheap pages. */
+        free_heap_pages(pg, order, false);
+        return NULL;
+    }
+
     return page_to_virt(pg);
 }
 
 
 void free_xenheap_pages(void *v, unsigned int order)
 {
+    unsigned long va = (unsigned long)v & PAGE_MASK;
+
     ASSERT_ALLOC_CONTEXT();
 
     if ( v == NULL )
         return;
 
+    if ( !has_directmap() &&
+         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
+        dprintk(XENLOG_WARNING,
+                "Error while destroying xenheap mappings at %p, order %u\n",
+                v, order);
+
     free_heap_pages(virt_to_page(v), order, false);
 }
 
@@ -2284,6 +2304,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 {
     struct page_info *pg;
     unsigned int i;
+    void *ret;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2296,16 +2317,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
     if ( unlikely(pg == NULL) )
         return NULL;
 
+    ret = page_to_virt(pg);
+
+    if ( !has_directmap() &&
+         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
+                          PAGE_HYPERVISOR) )
+    {
+        /* Failed to map xenheap pages. */
+        free_domheap_pages(pg, order);
+        return NULL;
+    }
+
     for ( i = 0; i < (1u << order); i++ )
         pg[i].count_info |= PGC_xen_heap;
 
-    return page_to_virt(pg);
+    return ret;
 }
 
 void free_xenheap_pages(void *v, unsigned int order)
 {
     struct page_info *pg;
     unsigned int i;
+    unsigned long va = (unsigned long)v & PAGE_MASK;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2317,6 +2350,12 @@ void free_xenheap_pages(void *v, unsigned int order)
     for ( i = 0; i < (1u << order); i++ )
         pg[i].count_info &= ~PGC_xen_heap;
 
+    if ( !has_directmap() &&
+         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
+        dprintk(XENLOG_WARNING,
+                "Error while destroying xenheap mappings at %p, order %u\n",
+                v, order);
+
     free_heap_pages(pg, order, true);
 }
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720815.1123816 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb7-0005tA-VU; Mon, 13 May 2024 11:12:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720815.1123816; Mon, 13 May 2024 11:12:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb7-0005ru-KZ; Mon, 13 May 2024 11:12:01 +0000
Received: by outflank-mailman (input) for mailman id 720815;
 Mon, 13 May 2024 11:12:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb5-00036L-S2
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:59 +0000
Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com
 [52.119.213.156]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9c93357e-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:57 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52005.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:56 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.44.209:62283]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 2d8b20f1-bc7a-499d-8d1b-4d37c9ae2d90; Mon, 13 May 2024 11:11:55 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:55 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:54 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c93357e-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598718; x=1747134718;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=7GEtDZOeCCFB+yvu/Susicjynil2ILWZXCBC3LXl2pY=;
  b=BbzZB8u8Q0GmX44bUM+kH2QLvR+L6Vo0AX2PobHh380pWf/sQOdAzaFO
   piwsIKuhWaps5PCi5qCcanC8c2lOmL4HvRDyGhV7grqqav4YL5GpD37CT
   +zwl/IFqrvZoLtzd6PthHFENEtn0cark3em7CgfRgtIYXMVCbjAGOJzBo
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="653743988"
X-Farcaster-Flow-ID: 2d8b20f1-bc7a-499d-8d1b-4d37c9ae2d90
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>
Subject: [PATCH V3 15/19] Rename maddr_to_virt() calls
Date: Mon, 13 May 2024 11:11:13 +0000
Message-ID: <20240513111117.68828-16-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Until directmap gets completely removed, we'd still need to
keep some calls to mmaddr_to_virt() for xenheap pages or when the
directmap is enabled.

Rename the macro to maddr_to_directmap_virt() to flag them and
prevent further use of maddr_to_virt().

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
index 81f80c053a..ac016f3a04 100644
--- a/xen/arch/x86/dmi_scan.c
+++ b/xen/arch/x86/dmi_scan.c
@@ -277,7 +277,7 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len)
 			return "SMBIOS";
 		}
 	} else {
-		char __iomem *p = maddr_to_virt(0xF0000), *q;
+		char __iomem *p = maddr_to_directmap_virt(0xF0000), *q;
 		union {
 			struct dmi_eps dmi;
 			struct smbios3_eps smbios3;
@@ -364,7 +364,7 @@ static int __init dmi_iterate(void (*decode)(const struct dmi_header *))
 	dmi.size = 0;
 	smbios3.length = 0;
 
-	p = maddr_to_virt(0xF0000);
+	p = maddr_to_directmap_virt(0xF0000);
 	for (q = p; q < p + 0x10000; q += 16) {
 		if (!dmi.size) {
 			memcpy_fromio(&dmi, q, sizeof(dmi));
diff --git a/xen/arch/x86/include/asm/mach-default/bios_ebda.h b/xen/arch/x86/include/asm/mach-default/bios_ebda.h
index 42de6b2a5b..8cfe53d1f2 100644
--- a/xen/arch/x86/include/asm/mach-default/bios_ebda.h
+++ b/xen/arch/x86/include/asm/mach-default/bios_ebda.h
@@ -7,7 +7,7 @@
  */
 static inline unsigned int get_bios_ebda(void)
 {
-	unsigned int address = *(unsigned short *)maddr_to_virt(0x40E);
+	unsigned int address = *(unsigned short *)maddr_to_directmap_virt(0x40E);
 	address <<= 4;
 	return address;	/* 0 means none */
 }
diff --git a/xen/arch/x86/include/asm/page.h b/xen/arch/x86/include/asm/page.h
index c6891b52d4..bf7bf08ba4 100644
--- a/xen/arch/x86/include/asm/page.h
+++ b/xen/arch/x86/include/asm/page.h
@@ -240,11 +240,11 @@ void copy_page_sse2(void *to, const void *from);
 
 /* Convert between Xen-heap virtual addresses and machine addresses. */
 #define __pa(x)             (virt_to_maddr(x))
-#define __va(x)             (maddr_to_virt(x))
+#define __va(x)             (maddr_to_directmap_virt(x))
 
 /* Convert between Xen-heap virtual addresses and machine frame numbers. */
 #define __virt_to_mfn(va)   (virt_to_maddr(va) >> PAGE_SHIFT)
-#define __mfn_to_virt(mfn)  (maddr_to_virt((paddr_t)(mfn) << PAGE_SHIFT))
+#define __mfn_to_virt(mfn)  (maddr_to_directmap_virt((paddr_t)(mfn) << PAGE_SHIFT))
 
 /* Convert between machine frame numbers and page-info structures. */
 #define mfn_to_page(mfn)    (frame_table + mfn_to_pdx(mfn))
@@ -270,7 +270,7 @@ void copy_page_sse2(void *to, const void *from);
 #define virt_to_mfn(va)     __virt_to_mfn(va)
 #define mfn_to_directmap_virt(mfn)    __mfn_to_virt(mfn)
 #define virt_to_maddr(va)   __virt_to_maddr((unsigned long)(va))
-#define maddr_to_virt(ma)   __maddr_to_virt((unsigned long)(ma))
+#define maddr_to_directmap_virt(ma)   __maddr_to_directmap_virt((unsigned long)(ma))
 #define maddr_to_page(ma)   __maddr_to_page(ma)
 #define page_to_maddr(pg)   __page_to_maddr(pg)
 #define virt_to_page(va)    __virt_to_page(va)
diff --git a/xen/arch/x86/include/asm/x86_64/page.h b/xen/arch/x86/include/asm/x86_64/page.h
index 19ca64d792..a95ebc088f 100644
--- a/xen/arch/x86/include/asm/x86_64/page.h
+++ b/xen/arch/x86/include/asm/x86_64/page.h
@@ -48,7 +48,7 @@ static inline unsigned long __virt_to_maddr(unsigned long va)
     return xen_phys_start + va - XEN_VIRT_START;
 }
 
-static inline void *__maddr_to_virt(unsigned long ma)
+static inline void *__maddr_to_directmap_virt(unsigned long ma)
 {
     /* Offset in the direct map, accounting for pdx compression */
     unsigned long va_offset = maddr_to_directmapoff(ma);
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449..69181b0abe 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -664,7 +664,7 @@ void __init get_smp_config (void)
 
 static int __init smp_scan_config (unsigned long base, unsigned long length)
 {
-	unsigned int *bp = maddr_to_virt(base);
+	unsigned int *bp = maddr_to_directmap_virt(base);
 	struct intel_mp_floating *mpf;
 
 	Dprintk("Scan SMP from %p for %ld bytes.\n", bp,length);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 39aed5845d..1b02e2b6d5 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1764,7 +1764,7 @@ void __init efi_init_memory(void)
                 if ( map_pages_to_xen((unsigned long)mfn_to_directmap_virt(smfn),
                                     _mfn(smfn), emfn - smfn, prot) == 0 )
                     desc->VirtualStart =
-                        (unsigned long)maddr_to_virt(desc->PhysicalStart);
+                        (unsigned long)maddr_to_directmap_virt(desc->PhysicalStart);
                 else
                     printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
                         smfn, emfn - 1);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720816.1123824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb9-0006BJ-08; Mon, 13 May 2024 11:12:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720816.1123824; Mon, 13 May 2024 11:12:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb8-00067E-JM; Mon, 13 May 2024 11:12:02 +0000
Received: by outflank-mailman (input) for mailman id 720816;
 Mon, 13 May 2024 11:12:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb6-0003MC-9X
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:00 +0000
Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com
 [52.119.213.154]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9d69b471-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:59 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2])
 by smtp-border-fw-52004.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:57 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.0.204:60869]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.42.97:2525]
 with esmtp (Farcaster)
 id 9f190847-4ba4-44c0-9b7e-9231c51b654a; Mon, 13 May 2024 11:11:56 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:56 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:56 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d69b471-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598719; x=1747134719;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=BMLad5szd5dSnCh0YLtH/2Rm7w1Vj1S/B6o9FzZUpnM=;
  b=iUsIjZJkOmxZC6Lw+aB3lFd2yz+vv5KSL6J3zXsbTSFppZoB3Fh6DtSN
   6cw0+ByuxiiYyBCuc+d9CdK/mbiVB8Mql0wQ+H7t9wClEH5KXckNAp5jD
   pnyLlghaVdXyjIYPjGoP9JvGBSyHASmQU/upN5sgBlP2MFMOFhs7oDJEf
   I=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="204721251"
X-Farcaster-Flow-ID: 9f190847-4ba4-44c0-9b7e-9231c51b654a
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 16/19] xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
Date: Mon, 13 May 2024 11:11:14 +0000
Message-ID: <20240513111117.68828-17-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

The arm32 version of init_secondary_pagetables() will soon be re-used
for arm64 as well where the root table starts at level 0 rather than level 1.

So rename 'first' to 'root'.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changelog in v2:
        * Rebase
        * Fix typo

diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c
index 4ffc8254a4..e29b6f34f2 100644
--- a/xen/arch/arm/mmu/smpboot.c
+++ b/xen/arch/arm/mmu/smpboot.c
@@ -109,32 +109,32 @@ int prepare_secondary_mm(int cpu)
 #else
 int prepare_secondary_mm(int cpu)
 {
-    lpae_t *first;
+    lpae_t *root = alloc_xenheap_page();
 
     first = alloc_xenheap_page(); /* root == first level on 32-bit 3-level trie */
 
-    if ( !first )
+    if ( !root )
     {
-        printk("CPU%u: Unable to allocate the first page-table\n", cpu);
+        printk("CPU%u: Unable to allocate the root page-table\n", cpu);
         return -ENOMEM;
     }
 
     /* Initialise root pagetable from root of boot tables */
-    memcpy(first, per_cpu(xen_pgtable, 0), PAGE_SIZE);
-    per_cpu(xen_pgtable, cpu) = first;
+    memcpy(root, per_cpu(xen_pgtable, 0), PAGE_SIZE);
+    per_cpu(xen_pgtable, cpu) = root;
 
     if ( !init_domheap_mappings(cpu) )
     {
         printk("CPU%u: Unable to prepare the domheap page-tables\n", cpu);
         per_cpu(xen_pgtable, cpu) = NULL;
-        free_xenheap_page(first);
+        free_xenheap_page(root);
         return -ENOMEM;
     }
 
     clear_boot_pagetables();
 
     /* Set init_ttbr for this CPU coming up */
-    set_init_ttbr(first);
+    set_init_ttbr(root);
 
     return 0;
 }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720817.1123830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb9-0006I9-J0; Mon, 13 May 2024 11:12:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720817.1123830; Mon, 13 May 2024 11:12:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tb9-0006Gr-4l; Mon, 13 May 2024 11:12:03 +0000
Received: by outflank-mailman (input) for mailman id 720817;
 Mon, 13 May 2024 11:12:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb6-00036L-PO
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:00 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9c47b21e-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:11:58 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:51 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.29.78:57211]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.60.166:2525]
 with esmtp (Farcaster)
 id 5ea62801-8630-4432-90a1-4613b44f2052; Mon, 13 May 2024 11:11:50 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:50 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:50 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c47b21e-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598718; x=1747134718;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=YYAC4ge8TxViu8X0NtP8ExO8g2vNW+lgMuOpIFb06t0=;
  b=ksW0gEW0jUTVSil7zZV51DaSQEkSrqItaZOvY4dY7ryJNXhPbjUd/AGm
   zQ7t6V6gByd5IyQiSF6rJ9FYaWJN+wuOOUW5RkzgprMwpbQGd1pIGrAmB
   venz/dH/jNnz1j4jgM54NL75LB7VzKHDevUk8gWm/Tu2ClSjvw1tjb+t0
   s=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88736865"
X-Farcaster-Flow-ID: 5ea62801-8630-4432-90a1-4613b44f2052
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 12/19] x86/setup: vmap heap nodes when they are outside the direct map
Date: Mon, 13 May 2024 11:11:10 +0000
Message-ID: <20240513111117.68828-13-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When we do not have a direct map, archs_mfn_in_direct_map() will always
return false, thus init_node_heap() will allocate xenheap pages from an
existing node for the metadata of a new node. This means that the
metadata of a new node is in a different node, slowing down heap
allocation.

Since we now have early vmap, vmap the metadata locally in the new node.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in v2:
        * vmap_contig_pages() was renamed to vmap_contig()
        * Fix indentation and coding style

    Changes from Hongyan's version:
        * arch_mfn_in_direct_map() was renamed to
          arch_mfns_in_direct_map()
        * Use vmap_contig_pages() rather than __vmap(...).
        * Add missing include (xen/vmap.h) so it compiles on Arm

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index dfb2c05322..3c0909f333 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -136,6 +136,7 @@
 #include <xen/sched.h>
 #include <xen/softirq.h>
 #include <xen/spinlock.h>
+#include <xen/vmap.h>
 
 #include <asm/flushtlb.h>
 #include <asm/page.h>
@@ -605,22 +606,44 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
         needed = 0;
     }
     else if ( *use_tail && nr >= needed &&
-              arch_mfns_in_directmap(mfn + nr - needed, needed) &&
               (!xenheap_bits ||
-               !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )
+              !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )
     {
-        _heap[node] = mfn_to_virt(mfn + nr - needed);
-        avail[node] = mfn_to_virt(mfn + nr - 1) +
-                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
+        {
+            _heap[node] = mfn_to_virt(mfn + nr - needed);
+            avail[node] = mfn_to_virt(mfn + nr - 1) +
+                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        }
+        else
+        {
+            mfn_t needed_start = _mfn(mfn + nr - needed);
+
+            _heap[node] = vmap_contig(needed_start, needed);
+            BUG_ON(!_heap[node]);
+            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
+                          sizeof(**avail) * NR_ZONES;
+        }
     }
     else if ( nr >= needed &&
-              arch_mfns_in_directmap(mfn, needed) &&
               (!xenheap_bits ||
-               !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
+              !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
     {
-        _heap[node] = mfn_to_virt(mfn);
-        avail[node] = mfn_to_virt(mfn + needed - 1) +
-                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        if ( arch_mfns_in_directmap(mfn, needed) )
+        {
+            _heap[node] = mfn_to_virt(mfn);
+            avail[node] = mfn_to_virt(mfn + needed - 1) +
+                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        }
+        else
+        {
+            mfn_t needed_start = _mfn(mfn);
+
+            _heap[node] = vmap_contig(needed_start, needed);
+            BUG_ON(!_heap[node]);
+            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
+                          sizeof(**avail) * NR_ZONES;
+        }
         *use_tail = false;
     }
     else if ( get_order_from_bytes(sizeof(**_heap)) ==
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720818.1123837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbA-0006Z7-Dd; Mon, 13 May 2024 11:12:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720818.1123837; Mon, 13 May 2024 11:12:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbA-0006Wr-1i; Mon, 13 May 2024 11:12:04 +0000
Received: by outflank-mailman (input) for mailman id 720818;
 Mon, 13 May 2024 11:12:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb7-0003MC-9V
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:01 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9de7f59c-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:11:59 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:59 +0000
Received: from EX19MTAUEC001.ant.amazon.com [10.0.0.204:16777]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.82.155:2525]
 with esmtp (Farcaster)
 id 8b74a7ba-70b1-4c07-888e-5d094449f3ea; Mon, 13 May 2024 11:11:58 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEC001.ant.amazon.com (10.252.135.222) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:58 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:57 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9de7f59c-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598720; x=1747134720;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=SVsOmDKOmFhMFa0LMfYH4rjGO+/XR9rqN+YrORlcGU8=;
  b=kgMEKMV8c0rW42gv6vooGXWGXQ2fZr7JVm5BWV002kOQwQptbKmFoBYL
   K8IzSxDf/BOUo9SuwMEHxNqKvwI8URqoXpe3RRrGBTnROiOV3fRoY+FwL
   +9rtAPXdZWllx/uZPKvr73tUvGDfHFRIFopftyfeJzy7a85xiyFq+Lb2R
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="725407113"
X-Farcaster-Flow-ID: 8b74a7ba-70b1-4c07-888e-5d094449f3ea
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 17/19] xen/arm64: mm: Use per-pCPU page-tables
Date: Mon, 13 May 2024 11:11:15 +0000
Message-ID: <20240513111117.68828-18-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

At the moment, on Arm64, every pCPU is sharing the same page-tables.

In a follow-up patch, we will allow the possibility to remove the
direct map and therefore it will be necessary to have a mapcache.

While we have plenty of spare virtual address space to reserve part
for each pCPU, it means that temporary mappings (e.g. guest memory)
could be accessible by every pCPU.

In order to increase our security posture, it would be better if
those mappings are only accessible by the pCPU doing the temporary
mapping.

In addition to that, a per-pCPU page-tables opens the way to have
per-domain mapping area.

Arm32 is already using per-pCPU page-tables so most of the code
can be re-used. Arm64 doesn't yet have support for the mapcache,
so a stub is provided (moved to its own header asm/domain_page.h).

Take the opportunity to fix a typo in a comment that is modified.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changelog since v1:
        * Rebase
        * Fix typoes

diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 293acb67e0..2ec1ffe1dc 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -76,6 +76,7 @@ static void __init prepare_runtime_identity_mapping(void)
     paddr_t id_addr = virt_to_maddr(_start);
     lpae_t pte;
     DECLARE_OFFSETS(id_offsets, id_addr);
+    lpae_t *root = this_cpu(xen_pgtable);
 
     if ( id_offsets[0] >= IDENTITY_MAPPING_AREA_NR_L0 )
         panic("Cannot handle ID mapping above %uTB\n",
@@ -86,7 +87,7 @@ static void __init prepare_runtime_identity_mapping(void)
     pte.pt.table = 1;
     pte.pt.xn = 0;
 
-    write_pte(&xen_pgtable[id_offsets[0]], pte);
+    write_pte(&root[id_offsets[0]], pte);
 
     /* Link second ID table */
     pte = pte_of_xenaddr((vaddr_t)xen_second_id);
diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c
index 3a43601623..ac2a6d0332 100644
--- a/xen/arch/arm/domain_page.c
+++ b/xen/arch/arm/domain_page.c
@@ -3,6 +3,8 @@
 #include <xen/pmap.h>
 #include <xen/vmap.h>
 
+#include <asm/domain_page.h>
+
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
diff --git a/xen/arch/arm/include/asm/arm32/mm.h b/xen/arch/arm/include/asm/arm32/mm.h
index 856f2dbec4..87a315db01 100644
--- a/xen/arch/arm/include/asm/arm32/mm.h
+++ b/xen/arch/arm/include/asm/arm32/mm.h
@@ -1,12 +1,6 @@
 #ifndef __ARM_ARM32_MM_H__
 #define __ARM_ARM32_MM_H__
 
-#include <xen/percpu.h>
-
-#include <asm/lpae.h>
-
-DECLARE_PER_CPU(lpae_t *, xen_pgtable);
-
 /*
  * Only a limited amount of RAM, called xenheap, is always mapped on ARM32.
  * For convenience always return false.
@@ -16,8 +10,6 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
     return false;
 }
 
-bool init_domheap_mappings(unsigned int cpu);
-
 static inline void arch_setup_page_tables(void)
 {
 }
diff --git a/xen/arch/arm/include/asm/domain_page.h b/xen/arch/arm/include/asm/domain_page.h
new file mode 100644
index 0000000000..e9f52685e2
--- /dev/null
+++ b/xen/arch/arm/include/asm/domain_page.h
@@ -0,0 +1,13 @@
+#ifndef __ASM_ARM_DOMAIN_PAGE_H__
+#define __ASM_ARM_DOMAIN_PAGE_H__
+
+#ifdef CONFIG_ARCH_MAP_DOMAIN_PAGE
+bool init_domheap_mappings(unsigned int cpu);
+#else
+static inline bool init_domheap_mappings(unsigned int cpu)
+{
+    return true;
+}
+#endif
+
+#endif /* __ASM_ARM_DOMAIN_PAGE_H__ */
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 2bca3f9e87..60e0122cba 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -2,6 +2,9 @@
 #define __ARCH_ARM_MM__
 
 #include <xen/kernel.h>
+#include <xen/percpu.h>
+
+#include <asm/lpae.h>
 #include <asm/page.h>
 #include <public/xen.h>
 #include <xen/pdx.h>
diff --git a/xen/arch/arm/include/asm/mmu/mm.h b/xen/arch/arm/include/asm/mmu/mm.h
index c5e03a66bf..c03c3a51e4 100644
--- a/xen/arch/arm/include/asm/mmu/mm.h
+++ b/xen/arch/arm/include/asm/mmu/mm.h
@@ -2,6 +2,8 @@
 #ifndef __ARM_MMU_MM_H__
 #define __ARM_MMU_MM_H__
 
+DECLARE_PER_CPU(lpae_t *, xen_pgtable);
+
 /* Non-boot CPUs use this to find the correct pagetables. */
 extern uint64_t init_ttbr;
 
diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c
index da28d669e7..1ed1a53ab1 100644
--- a/xen/arch/arm/mmu/pt.c
+++ b/xen/arch/arm/mmu/pt.c
@@ -607,9 +607,9 @@ static int xen_pt_update(unsigned long virt,
     unsigned long left = nr_mfns;
 
     /*
-     * For arm32, page-tables are different on each CPUs. Yet, they share
-     * some common mappings. It is assumed that only common mappings
-     * will be modified with this function.
+     * Page-tables are different on each CPU. Yet, they share some common
+     * mappings. It is assumed that only common mappings will be modified
+     * with this function.
      *
      * XXX: Add a check.
      */
diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c
index f4bb424c3c..7b981456e6 100644
--- a/xen/arch/arm/mmu/setup.c
+++ b/xen/arch/arm/mmu/setup.c
@@ -28,17 +28,15 @@
  * PCPUs.
  */
 
-#ifdef CONFIG_ARM_64
-DEFINE_PAGE_TABLE(xen_pgtable);
-static DEFINE_PAGE_TABLE(xen_first);
-#define THIS_CPU_PGTABLE xen_pgtable
-#else
 /* Per-CPU pagetable pages */
 /* xen_pgtable == root of the trie (zeroeth level on 64-bit, first on 32-bit) */
 DEFINE_PER_CPU(lpae_t *, xen_pgtable);
 #define THIS_CPU_PGTABLE this_cpu(xen_pgtable)
 /* Root of the trie for cpu0, other CPU's PTs are dynamically allocated */
 static DEFINE_PAGE_TABLE(cpu0_pgtable);
+
+#ifdef CONFIG_ARM_64
+static DEFINE_PAGE_TABLE(xen_first);
 #endif
 
 /* Common pagetable leaves */
@@ -228,19 +226,22 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
     lpae_t pte, *p;
     int i;
 
+    p = cpu0_pgtable;
+
     phys_offset = boot_phys_offset;
 
+    /* arch_setup_page_tables() may need to access the root page-tables. */
+    per_cpu(xen_pgtable, 0) = cpu0_pgtable;
+
     arch_setup_page_tables();
 
 #ifdef CONFIG_ARM_64
     pte = pte_of_xenaddr((uintptr_t)xen_first);
     pte.pt.table = 1;
     pte.pt.xn = 0;
-    xen_pgtable[zeroeth_table_offset(XEN_VIRT_START)] = pte;
+    p[zeroeth_table_offset(XEN_VIRT_START)] = pte;
 
-    p = (void *) xen_first;
-#else
-    p = (void *) cpu0_pgtable;
+    p = xen_first;
 #endif
 
     /* Map xen second level page-table */
@@ -283,19 +284,11 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
     pte.pt.table = 1;
     xen_second[second_table_offset(FIXMAP_ADDR(0))] = pte;
 
-#ifdef CONFIG_ARM_64
-    ttbr = (uintptr_t) xen_pgtable + phys_offset;
-#else
     ttbr = (uintptr_t) cpu0_pgtable + phys_offset;
-#endif
 
     switch_ttbr(ttbr);
 
     xen_pt_enforce_wnx();
-
-#ifdef CONFIG_ARM_32
-    per_cpu(xen_pgtable, 0) = cpu0_pgtable;
-#endif
 }
 
 void *__init arch_vmap_virt_end(void)
diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c
index e29b6f34f2..eb51d6aae9 100644
--- a/xen/arch/arm/mmu/smpboot.c
+++ b/xen/arch/arm/mmu/smpboot.c
@@ -7,6 +7,7 @@
 
 #include <xen/domain_page.h>
 
+#include <asm/domain_page.h>
 #include <asm/setup.h>
 
 /* Override macros from asm/page.h to make them work with mfn_t */
@@ -93,20 +94,6 @@ static void set_init_ttbr(lpae_t *root)
     unmap_domain_page(ptr);
 }
 
-#ifdef CONFIG_ARM_64
-int prepare_secondary_mm(int cpu)
-{
-    clear_boot_pagetables();
-
-    /*
-     * Set init_ttbr for this CPU coming up. All CPUs share a single setof
-     * pagetables, but rewrite it each time for consistency with 32 bit.
-     */
-    set_init_ttbr(xen_pgtable);
-
-    return 0;
-}
-#else
 int prepare_secondary_mm(int cpu)
 {
     lpae_t *root = alloc_xenheap_page();
@@ -138,7 +125,6 @@ int prepare_secondary_mm(int cpu)
 
     return 0;
 }
-#endif
 
 /*
  * Local variables:
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d242674381..d15987d6ea 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -42,6 +42,7 @@
 #include <asm/gic.h>
 #include <asm/cpuerrata.h>
 #include <asm/cpufeature.h>
+#include <asm/domain_page.h>
 #include <asm/platform.h>
 #include <asm/procinfo.h>
 #include <asm/setup.h>
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720820.1123841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbB-0006og-5i; Mon, 13 May 2024 11:12:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720820.1123841; Mon, 13 May 2024 11:12:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbA-0006iF-PL; Mon, 13 May 2024 11:12:04 +0000
Received: by outflank-mailman (input) for mailman id 720820;
 Mon, 13 May 2024 11:12:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Tb8-00036L-TQ
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:02 +0000
Received: from smtp-fw-9105.amazon.com (smtp-fw-9105.amazon.com
 [207.171.188.204]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9dea8290-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:12:00 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9105.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:51 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.29.78:3493]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.32.236:2525]
 with esmtp (Farcaster)
 id 0b46e58a-ee0e-4f27-b49c-9b2581b1cd16; Mon, 13 May 2024 11:11:39 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:35 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:34 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9dea8290-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598721; x=1747134721;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=awTpJTVuZnpLM2XGjDjoGafH08PnS/u8+TgC4HSOyp8=;
  b=QkR2tgYY3YLURtMFK+Xsy7vsrYjj8MQvOBX5Hvdvsf3y9lRTE0W0Qpcg
   aG0c+W7swVn1sPLAjZ8EC10LOimjrJU/C5KDHajyglVXngSjXM9P3eqPZ
   DIzQcLk2EsHG6ivyaFG8gZlUBhAVNB5Up1NDy9bzuY6xwJDrhYXoObjxg
   s=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="726023919"
X-Farcaster-Flow-ID: 0b46e58a-ee0e-4f27-b49c-9b2581b1cd16
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 03/19] x86/pv: Rewrite how building PV dom0 handles domheap mappings
Date: Mon, 13 May 2024 11:11:01 +0000
Message-ID: <20240513111117.68828-4-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Building a PV dom0 is allocating from the domheap but uses it like the
xenheap. Use the pages as they should be.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Fold following patch 'x86/pv: Map L4 page table for shim domain'

    Changes in V2:
        * Clarify the commit message
        * Break the patch in two parts

    Changes since Hongyan's version:
        * Rebase
        * Remove spurious newline

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index 807296c280..ac910b438a 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
     l3_pgentry_t *l3tab = NULL, *l3start = NULL;
     l2_pgentry_t *l2tab = NULL, *l2start = NULL;
     l1_pgentry_t *l1tab = NULL, *l1start = NULL;
+    mfn_t l4start_mfn = INVALID_MFN;
+    mfn_t l3start_mfn = INVALID_MFN;
+    mfn_t l2start_mfn = INVALID_MFN;
+    mfn_t l1start_mfn = INVALID_MFN;
 
     /*
      * This fully describes the memory layout of the initial domain. All
@@ -710,22 +714,32 @@ int __init dom0_construct_pv(struct domain *d,
         v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
     }
 
+#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
+do {                                                    \
+    unmap_domain_page(virt_var);                        \
+    mfn_var = maddr_to_mfn(maddr);                      \
+    maddr += PAGE_SIZE;                                 \
+    virt_var = map_domain_page(mfn_var);                \
+} while ( false )
+
     if ( !compat )
     {
         maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
-        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
+        l4tab = l4start;
         clear_page(l4tab);
-        init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)),
-                          d, INVALID_MFN, true);
-        v->arch.guest_table = pagetable_from_paddr(__pa(l4start));
+        init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true);
+        v->arch.guest_table = pagetable_from_mfn(l4start_mfn);
     }
     else
     {
         /* Monitor table already created by switch_compat(). */
-        l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table));
+        l4start_mfn = pagetable_get_mfn(v->arch.guest_table);
+        l4start = l4tab = map_domain_page(l4start_mfn);
         /* See public/xen.h on why the following is needed. */
         maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table;
         l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+        UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc);
     }
 
     l4tab += l4_table_offset(v_start);
@@ -735,14 +749,16 @@ int __init dom0_construct_pv(struct domain *d,
         if ( !((unsigned long)l1tab & (PAGE_SIZE-1)) )
         {
             maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l1_page_table;
-            l1start = l1tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+            UNMAP_MAP_AND_ADVANCE(l1start_mfn, l1start, mpt_alloc);
+            l1tab = l1start;
             clear_page(l1tab);
             if ( count == 0 )
                 l1tab += l1_table_offset(v_start);
             if ( !((unsigned long)l2tab & (PAGE_SIZE-1)) )
             {
                 maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
-                l2start = l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
+                l2tab = l2start;
                 clear_page(l2tab);
                 if ( count == 0 )
                     l2tab += l2_table_offset(v_start);
@@ -752,19 +768,19 @@ int __init dom0_construct_pv(struct domain *d,
                     {
                         maddr_to_page(mpt_alloc)->u.inuse.type_info =
                             PGT_l3_page_table;
-                        l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+                        UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc);
                     }
                     l3tab = l3start;
                     clear_page(l3tab);
                     if ( count == 0 )
                         l3tab += l3_table_offset(v_start);
-                    *l4tab = l4e_from_paddr(__pa(l3start), L4_PROT);
+                    *l4tab = l4e_from_mfn(l3start_mfn, L4_PROT);
                     l4tab++;
                 }
-                *l3tab = l3e_from_paddr(__pa(l2start), L3_PROT);
+                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
                 l3tab++;
             }
-            *l2tab = l2e_from_paddr(__pa(l1start), L2_PROT);
+            *l2tab = l2e_from_mfn(l1start_mfn, L2_PROT);
             l2tab++;
         }
         if ( count < initrd_pfn || count >= initrd_pfn + PFN_UP(initrd_len) )
@@ -783,27 +799,31 @@ int __init dom0_construct_pv(struct domain *d,
 
     if ( compat )
     {
-        l2_pgentry_t *l2t;
-
         /* Ensure the first four L3 entries are all populated. */
         for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
         {
             if ( !l3e_get_intpte(*l3tab) )
             {
                 maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
-                l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
-                clear_page(l2tab);
-                *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT);
+                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
+                clear_page(l2start);
+                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
             }
             if ( i == 3 )
                 l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
         }
 
-        l2t = map_l2t_from_l3e(l3start[3]);
-        init_xen_pae_l2_slots(l2t, d);
-        unmap_domain_page(l2t);
+        UNMAP_DOMAIN_PAGE(l2start);
+        l2start = map_l2t_from_l3e(l3start[3]);
+        init_xen_pae_l2_slots(l2start, d);
     }
 
+#undef UNMAP_MAP_AND_ADVANCE
+
+    UNMAP_DOMAIN_PAGE(l1start);
+    UNMAP_DOMAIN_PAGE(l2start);
+    UNMAP_DOMAIN_PAGE(l3start);
+
     /* Pages that are part of page tables must be read only. */
     mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages, &flush_flags);
 
@@ -976,6 +996,8 @@ int __init dom0_construct_pv(struct domain *d,
         pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start,
                           vphysmap_start, si);
 
+    UNMAP_DOMAIN_PAGE(l4start);
+
 #ifdef CONFIG_COMPAT
     if ( compat )
         xlat_start_info(si, pv_shim ? XLAT_start_info_console_domU
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720821.1123855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbD-0007Jn-3I; Mon, 13 May 2024 11:12:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720821.1123855; Mon, 13 May 2024 11:12:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbC-0007I6-MT; Mon, 13 May 2024 11:12:06 +0000
Received: by outflank-mailman (input) for mailman id 720821;
 Mon, 13 May 2024 11:12:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TbA-0003MC-B2
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:04 +0000
Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com
 [207.171.184.29]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9f3767fb-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:12:02 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-9102.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:55 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.29.78:56418]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id 2b21e6f7-2a58-484f-8d2a-a0460334d2b3; Mon, 13 May 2024 11:11:53 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:53 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:53 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f3767fb-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598723; x=1747134723;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=mKCZV9DzUavScUGAYIjbkqz34voMdE7B7ItlMBncNvQ=;
  b=FfUMR3FUAQYz29EJ9SvP5Rw00w8z4QDxALGWzwo9OmjSS89RiGydwsIk
   X66HYB3wa4pD4AsIUWKi1/+1q763DiuGMA/irfseHAkumpZXkYSV43WTY
   ZHxiagunKNRZUMWvN/Dh1Dq7kF2sOwMlh7tmtCg4SwHnfbOPagi9kpOA2
   k=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="418645273"
X-Farcaster-Flow-ID: 2b21e6f7-2a58-484f-8d2a-a0460334d2b3
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>
Subject: [PATCH V3 14/19] Rename mfn_to_virt() calls
Date: Mon, 13 May 2024 11:11:12 +0000
Message-ID: <20240513111117.68828-15-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Until directmap gets completely removed, we'd still need to
keep some calls to mfn_to_virt() for xenheap pages or when the
directmap is enabled.

Rename the macro to mfn_to_directmap_virt() to flag them and
prevent further use of mfn_to_virt().

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 48538b5337..2bca3f9e87 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -336,6 +336,7 @@ static inline uint64_t gvirt_to_maddr(vaddr_t va, paddr_t *pa,
  */
 #define virt_to_mfn(va)     __virt_to_mfn(va)
 #define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+#define mfn_to_directmap_virt(mfn) mfn_to_virt(mfn)
 
 /* Convert between Xen-heap virtual addresses and page-info structures. */
 static inline struct page_info *virt_to_page(const void *v)
diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index 89caefc8a2..62d6fee0f4 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -81,14 +81,14 @@ void *map_domain_page(mfn_t mfn)
 
 #ifdef NDEBUG
     if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
-        return mfn_to_virt(mfn_x(mfn));
+        return mfn_to_directmap_virt(mfn_x(mfn));
 #endif
 
     v = mapcache_current_vcpu();
     if ( !v || !v->domain->arch.mapcache.inuse )
     {
         if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
-            return mfn_to_virt(mfn_x(mfn));
+            return mfn_to_directmap_virt(mfn_x(mfn));
         else
         {
             BUG_ON(system_state >= SYS_STATE_smp_boot);
@@ -324,7 +324,7 @@ void *map_domain_page_global(mfn_t mfn)
 
 #ifdef NDEBUG
     if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) )
-        return mfn_to_virt(mfn_x(mfn));
+        return mfn_to_directmap_virt(mfn_x(mfn));
 #endif
 
     return vmap(&mfn, 1);
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index b0cb96c3bc..d1482ae2f7 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -439,7 +439,7 @@ static int __init pvh_populate_p2m(struct domain *d)
                                  d->arch.e820[i].addr + d->arch.e820[i].size);
             enum hvm_translation_result res =
                  hvm_copy_to_guest_phys(mfn_to_maddr(_mfn(addr)),
-                                        mfn_to_virt(addr),
+                                        mfn_to_directmap_virt(addr),
                                         end - d->arch.e820[i].addr,
                                         v);
 
@@ -725,7 +725,7 @@ static int __init pvh_load_kernel(struct domain *d, const module_t *image,
 
     if ( initrd != NULL )
     {
-        rc = hvm_copy_to_guest_phys(last_addr, mfn_to_virt(initrd->mod_start),
+        rc = hvm_copy_to_guest_phys(last_addr, mfn_to_directmap_virt(initrd->mod_start),
                                     initrd_len, v);
         if ( rc )
         {
diff --git a/xen/arch/x86/include/asm/page.h b/xen/arch/x86/include/asm/page.h
index 350d1fb110..c6891b52d4 100644
--- a/xen/arch/x86/include/asm/page.h
+++ b/xen/arch/x86/include/asm/page.h
@@ -268,7 +268,7 @@ void copy_page_sse2(void *to, const void *from);
  */
 #define mfn_valid(mfn)      __mfn_valid(mfn_x(mfn))
 #define virt_to_mfn(va)     __virt_to_mfn(va)
-#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+#define mfn_to_directmap_virt(mfn)    __mfn_to_virt(mfn)
 #define virt_to_maddr(va)   __virt_to_maddr((unsigned long)(va))
 #define maddr_to_virt(ma)   __maddr_to_virt((unsigned long)(ma))
 #define maddr_to_page(ma)   __maddr_to_page(ma)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index efdf20f775..337363cf17 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -318,8 +318,8 @@ void __init arch_init_memory(void)
         iostart_pfn = max_t(unsigned long, pfn, 1UL << (20 - PAGE_SHIFT));
         ioend_pfn = min(rstart_pfn, 16UL << (20 - PAGE_SHIFT));
         if ( iostart_pfn < ioend_pfn )
-            destroy_xen_mappings((unsigned long)mfn_to_virt(iostart_pfn),
-                                 (unsigned long)mfn_to_virt(ioend_pfn));
+            destroy_xen_mappings((unsigned long)mfn_to_directmap_virt(iostart_pfn),
+                                 (unsigned long)mfn_to_directmap_virt(ioend_pfn));
 
         /* Mark as I/O up to next RAM region. */
         for ( ; pfn < rstart_pfn; pfn++ )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 919347d8c2..e0671ab3c3 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -399,7 +399,7 @@ void *__init bootstrap_map(const module_t *mod)
     void *ret;
 
     if ( system_state != SYS_STATE_early_boot )
-        return mod ? mfn_to_virt(mod->mod_start) : NULL;
+        return mod ? mfn_to_directmap_virt(mod->mod_start) : NULL;
 
     if ( !mod )
     {
@@ -1708,7 +1708,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     {
         set_pdx_range(mod[i].mod_start,
                       mod[i].mod_start + PFN_UP(mod[i].mod_end));
-        map_pages_to_xen((unsigned long)mfn_to_virt(mod[i].mod_start),
+        map_pages_to_xen((unsigned long)mfn_to_directmap_virt(mod[i].mod_start),
                          _mfn(mod[i].mod_start),
                          PFN_UP(mod[i].mod_end), PAGE_HYPERVISOR);
     }
diff --git a/xen/arch/x86/tboot.c b/xen/arch/x86/tboot.c
index ba0700d2d5..58df01dfb9 100644
--- a/xen/arch/x86/tboot.c
+++ b/xen/arch/x86/tboot.c
@@ -253,7 +253,7 @@ static void tboot_gen_xenheap_integrity(const uint8_t key[TB_KEY_SIZE],
             continue; /* skip tboot and its page tables */
 
         if ( is_page_in_use(page) && is_special_page(page) )
-            vmac_update(mfn_to_virt(mfn), PAGE_SIZE, &ctx);
+            vmac_update(mfn_to_directmap_virt(mfn), PAGE_SIZE, &ctx);
     }
     *mac = vmac(NULL, 0, nonce, NULL, &ctx);
 
diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index b2a280fba3..1697760d82 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -1247,19 +1247,25 @@ int memory_add(unsigned long spfn, unsigned long epfn, unsigned int pxm)
     i = virt_to_mfn(HYPERVISOR_VIRT_END - 1) + 1;
     if ( spfn < i )
     {
-        ret = map_pages_to_xen((unsigned long)mfn_to_virt(spfn), _mfn(spfn),
-                               min(epfn, i) - spfn, PAGE_HYPERVISOR);
-        if ( ret )
-            goto destroy_directmap;
+        if ( arch_mfns_in_directmap(spfn, min(epfn, i) - spfn ) )
+        {
+            ret = map_pages_to_xen((unsigned long)mfn_to_directmap_virt(spfn), _mfn(spfn),
+                                min(epfn, i) - spfn, PAGE_HYPERVISOR);
+            if ( ret )
+                goto destroy_directmap;
+        }
     }
     if ( i < epfn )
     {
         if ( i < spfn )
             i = spfn;
-        ret = map_pages_to_xen((unsigned long)mfn_to_virt(i), _mfn(i),
-                               epfn - i, __PAGE_HYPERVISOR_RW);
-        if ( ret )
-            goto destroy_directmap;
+        if ( arch_mfns_in_directmap(spfn, epfn - i) )
+        {
+            ret = map_pages_to_xen((unsigned long)mfn_to_directmap_virt(i), _mfn(i),
+                                epfn - i, __PAGE_HYPERVISOR_RW);
+            if ( ret )
+                goto destroy_directmap;
+        }
     }
 
     old_node_start = node_start_pfn(node);
@@ -1348,8 +1354,8 @@ destroy_frametable:
     NODE_DATA(node)->node_start_pfn = old_node_start;
     NODE_DATA(node)->node_spanned_pages = old_node_span;
  destroy_directmap:
-    destroy_xen_mappings((unsigned long)mfn_to_virt(spfn),
-                         (unsigned long)mfn_to_virt(epfn));
+    destroy_xen_mappings((unsigned long)mfn_to_directmap_virt(spfn),
+                         (unsigned long)mfn_to_directmap_virt(epfn));
 
     return ret;
 }
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index efbec00af9..39aed5845d 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1599,7 +1599,7 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end,
     for ( ; mfn < end; mfn = next )
     {
         l4_pgentry_t l4e = efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)];
-        unsigned long va = (unsigned long)mfn_to_virt(mfn);
+        unsigned long va = (unsigned long)mfn_to_directmap_virt(mfn);
 
         if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) )
             UNMAP_DOMAIN_PAGE(l3dst);
@@ -1757,15 +1757,18 @@ void __init efi_init_memory(void)
         if ( pfn_to_pdx(emfn - 1) < (DIRECTMAP_SIZE >> PAGE_SHIFT) &&
              pdx_is_region_compressible(mem_base, mem_npages) )
         {
-            if ( (unsigned long)mfn_to_virt(emfn - 1) >= HYPERVISOR_VIRT_END )
-                prot &= ~_PAGE_GLOBAL;
-            if ( map_pages_to_xen((unsigned long)mfn_to_virt(smfn),
-                                  _mfn(smfn), emfn - smfn, prot) == 0 )
-                desc->VirtualStart =
-                    (unsigned long)maddr_to_virt(desc->PhysicalStart);
-            else
-                printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
-                       smfn, emfn - 1);
+            if ( arch_mfns_in_directmap(smfn, emfn - smfn) )
+            {
+                if ( (unsigned long)mfn_to_directmap_virt(emfn - 1) >= HYPERVISOR_VIRT_END )
+                    prot &= ~_PAGE_GLOBAL;
+                if ( map_pages_to_xen((unsigned long)mfn_to_directmap_virt(smfn),
+                                    _mfn(smfn), emfn - smfn, prot) == 0 )
+                    desc->VirtualStart =
+                        (unsigned long)maddr_to_virt(desc->PhysicalStart);
+                else
+                    printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
+                        smfn, emfn - 1);
+            }
         }
         else if ( !((desc->PhysicalStart + len - 1) >> (VADDR_BITS - 1)) &&
                   (extra = xmalloc(struct rt_extra)) != NULL )
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 3c0909f333..e1e98d4d59 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -611,8 +611,8 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
     {
         if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
         {
-            _heap[node] = mfn_to_virt(mfn + nr - needed);
-            avail[node] = mfn_to_virt(mfn + nr - 1) +
+            _heap[node] = mfn_to_directmap_virt(mfn + nr - needed);
+            avail[node] = mfn_to_directmap_virt(mfn + nr - 1) +
                           PAGE_SIZE - sizeof(**avail) * NR_ZONES;
         }
         else
@@ -631,8 +631,8 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
     {
         if ( arch_mfns_in_directmap(mfn, needed) )
         {
-            _heap[node] = mfn_to_virt(mfn);
-            avail[node] = mfn_to_virt(mfn + needed - 1) +
+            _heap[node] = mfn_to_directmap_virt(mfn);
+            avail[node] = mfn_to_directmap_virt(mfn + needed - 1) +
                           PAGE_SIZE - sizeof(**avail) * NR_ZONES;
         }
         else
diff --git a/xen/common/trace.c b/xen/common/trace.c
index 4e7b080e61..955509a0d8 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -229,7 +229,7 @@ static int alloc_trace_bufs(unsigned int pages)
         offset = t_info->mfn_offset[cpu];
 
         /* Initialize the buffer metadata */
-        per_cpu(t_bufs, cpu) = buf = mfn_to_virt(t_info_mfn_list[offset]);
+        per_cpu(t_bufs, cpu) = buf = mfn_to_directmap_virt(t_info_mfn_list[offset]);
         buf->cons = buf->prod = 0;
 
         printk(XENLOG_INFO "xentrace: p%d mfn %x offset %u\n",
@@ -268,7 +268,7 @@ out_dealloc:
             if ( !mfn )
                 break;
             ASSERT(!(mfn_to_page(_mfn(mfn))->count_info & PGC_allocated));
-            free_xenheap_pages(mfn_to_virt(mfn), 0);
+            free_xenheap_pages(mfn_to_directmap_virt(mfn), 0);
         }
     }
     free_xenheap_pages(t_info, get_order_from_pages(t_info_pages));
@@ -529,7 +529,7 @@ static unsigned char *next_record(const struct t_buf *buf, uint32_t *next,
     per_cpu_mfn_offset = t_info->mfn_offset[smp_processor_id()];
     mfn_list = (uint32_t *)t_info;
     mfn = mfn_list[per_cpu_mfn_offset + per_cpu_mfn_nr];
-    this_page = mfn_to_virt(mfn);
+    this_page = (unsigned char *)mfn_to_directmap_virt(mfn);
     if (per_cpu_mfn_nr + 1 >= opt_tbuf_size)
     {
         /* reached end of buffer? */
@@ -538,7 +538,7 @@ static unsigned char *next_record(const struct t_buf *buf, uint32_t *next,
     else
     {
         mfn = mfn_list[per_cpu_mfn_offset + per_cpu_mfn_nr + 1];
-        *next_page = mfn_to_virt(mfn);
+        *next_page = mfn_to_directmap_virt(mfn);
     }
     return this_page;
 }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:12:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:12:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720823.1123870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbF-0007iu-5f; Mon, 13 May 2024 11:12:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720823.1123870; Mon, 13 May 2024 11:12:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TbE-0007c5-3u; Mon, 13 May 2024 11:12:08 +0000
Received: by outflank-mailman (input) for mailman id 720823;
 Mon, 13 May 2024 11:12:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TbC-0003MC-BK
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:06 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9f6b2f0d-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:12:03 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:12:00 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.29.78:51808]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.61.78:2525]
 with esmtp (Farcaster)
 id 41b2d30f-ddb1-4efb-88b6-74e78337e70b; Mon, 13 May 2024 11:11:59 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:59 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:59 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f6b2f0d-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598723; x=1747134723;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=Xrly80bM9b4Iylo4MumX8GuIMjwgPtJngKOUTgszWKw=;
  b=NNSZ4aiOHDISjYhEmI2cPwno0y4heS2Sv78NGsBVFBfKshrdXdegk25/
   jFeoToBxM9IYr8oiVqy2jJy+CRdvZW61ffpElN8a1I17RqejfY/jHuRPP
   i0xRkm08MkGArh19vdAuFS303PKiqA1DMOnqTx0WmlnG70rJUDpZDiiu+
   s=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88736903"
X-Farcaster-Flow-ID: 41b2d30f-ddb1-4efb-88b6-74e78337e70b
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 18/19] xen/arm64: Implement a mapcache for arm64
Date: Mon, 13 May 2024 11:11:16 +0000
Message-ID: <20240513111117.68828-19-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

At the moment, on arm64, map_domain_page() is implemented using
virt_to_mfn(). Therefore it is relying on the directmap.

In a follow-up patch, we will allow the admin to remove the directmap.
Therefore we want to implement a mapcache.

Thanksfully there is already one for arm32. So select ARCH_ARM_DOMAIN_PAGE
and add the necessary boiler plate to support 64-bit:
    - The page-table start at level 0, so we need to allocate the level
      1 page-table
    - map_domain_page() should check if the page is in the directmap. If
      yes, then use virt_to_mfn() to limit the performance impact
      when the directmap is still enabled (this will be selectable
      on the command line).

Take the opportunity to replace first_table_offset(...) with offsets[...].

Note that, so far, arch_mfns_in_directmap() always return true on
arm64. So the mapcache is not yet used. This will change in a
follow-up patch.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    There are a few TODOs:
        - It is becoming more critical to fix the mapcache
          implementation (this is not compliant with the Arm Arm)
        - Evaluate the performance

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 21d03d9f44..0462960fc7 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -1,7 +1,6 @@
 config ARM_32
 	def_bool y
 	depends on "$(ARCH)" = "arm32"
-	select ARCH_MAP_DOMAIN_PAGE
 
 config ARM_64
 	def_bool y
diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 2ec1ffe1dc..826864d25d 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -4,6 +4,7 @@
 #include <xen/mm.h>
 #include <xen/pfn.h>
 
+#include <asm/domain_page.h>
 #include <asm/setup.h>
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
@@ -237,6 +238,14 @@ void __init setup_mm(void)
     setup_frametable_mappings(ram_start, ram_end);
     max_page = PFN_DOWN(ram_end);
 
+    /*
+     * The allocators may need to use map_domain_page() (such as for
+     * scrubbing pages). So we need to prepare the domheap area first.
+     */
+    if ( !init_domheap_mappings(smp_processor_id()) )
+        panic("CPU%u: Unable to prepare the domheap page-tables\n",
+              smp_processor_id());
+
     init_staticmem_pages();
     init_sharedmem_pages();
 }
diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c
index ac2a6d0332..0f6ba48892 100644
--- a/xen/arch/arm/domain_page.c
+++ b/xen/arch/arm/domain_page.c
@@ -1,4 +1,5 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/domain_page.h>
 #include <xen/mm.h>
 #include <xen/pmap.h>
 #include <xen/vmap.h>
@@ -8,6 +9,8 @@
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
+#undef mfn_to_virt
+#define mfn_to_virt(va) __mfn_to_virt(mfn_x(mfn))
 
 /* cpu0's domheap page tables */
 static DEFINE_PAGE_TABLES(cpu0_dommap, DOMHEAP_SECOND_PAGES);
@@ -31,13 +34,30 @@ bool init_domheap_mappings(unsigned int cpu)
 {
     unsigned int order = get_order_from_pages(DOMHEAP_SECOND_PAGES);
     lpae_t *root = per_cpu(xen_pgtable, cpu);
+    lpae_t *first;
     unsigned int i, first_idx;
     lpae_t *domheap;
     mfn_t mfn;
 
+    /* Convenience aliases */
+    DECLARE_OFFSETS(offsets, DOMHEAP_VIRT_START);
+
     ASSERT(root);
     ASSERT(!per_cpu(xen_dommap, cpu));
 
+    /*
+     * On Arm64, the root is at level 0. Therefore we need an extra step
+     * to allocate the first level page-table.
+     */
+#ifdef CONFIG_ARM_64
+    if ( create_xen_table(&root[offsets[0]]) )
+        return false;
+
+    first = xen_map_table(lpae_get_mfn(root[offsets[0]]));
+#else
+    first = root;
+#endif
+
     /*
      * The domheap for cpu0 is initialized before the heap is initialized.
      * So we need to use pre-allocated pages.
@@ -58,16 +78,20 @@ bool init_domheap_mappings(unsigned int cpu)
      * domheap mapping pages.
      */
     mfn = virt_to_mfn(domheap);
-    first_idx = first_table_offset(DOMHEAP_VIRT_START);
+    first_idx = offsets[1];
     for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ )
     {
         lpae_t pte = mfn_to_xen_entry(mfn_add(mfn, i), MT_NORMAL);
         pte.pt.table = 1;
-        write_pte(&root[first_idx + i], pte);
+        write_pte(&first[first_idx + i], pte);
     }
 
     per_cpu(xen_dommap, cpu) = domheap;
 
+#ifdef CONFIG_ARM_64
+    xen_unmap_table(first);
+#endif
+
     return true;
 }
 
@@ -91,6 +115,10 @@ void *map_domain_page(mfn_t mfn)
     lpae_t pte;
     int i, slot;
 
+    /* Bypass the mapcache if the page is in the directmap */
+    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
+        return mfn_to_virt(mfn);
+
     local_irq_save(flags);
 
     /* The map is laid out as an open-addressed hash table where each
@@ -153,13 +181,25 @@ void *map_domain_page(mfn_t mfn)
 /* Release a mapping taken with map_domain_page() */
 void unmap_domain_page(const void *ptr)
 {
+    unsigned long va = (unsigned long)ptr;
     unsigned long flags;
     lpae_t *map = this_cpu(xen_dommap);
-    int slot = ((unsigned long)ptr - DOMHEAP_VIRT_START) >> SECOND_SHIFT;
+    unsigned int slot;
+
+    /* Below we assume that the domheap area doesn't start at 0 */
+    BUILD_BUG_ON(DOMHEAP_VIRT_START == 0);
 
-    if ( !ptr )
+    /*
+     * map_domain_page() may not have mapped anything if the address
+     * is part of the directmap. So ignore anything outside of the
+     * domheap.
+     */
+    if ( (va < DOMHEAP_VIRT_START) ||
+         ((va - DOMHEAP_VIRT_START) >= DOMHEAP_VIRT_SIZE) )
         return;
 
+    slot = (va - DOMHEAP_VIRT_START) >> SECOND_SHIFT;
+
     local_irq_save(flags);
 
     ASSERT(slot >= 0 && slot < DOMHEAP_ENTRIES);
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 60e0122cba..610dfa0466 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -432,6 +432,11 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
     } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
 }
 
+/* Helpers to allocate, map and unmap a Xen page-table */
+int create_xen_table(lpae_t *entry);
+lpae_t *xen_map_table(mfn_t mfn);
+void xen_unmap_table(const lpae_t *table);
+
 #endif /*  __ARCH_ARM_MM__ */
 /*
  * Local variables:
diff --git a/xen/arch/arm/include/asm/mmu/layout.h b/xen/arch/arm/include/asm/mmu/layout.h
index a3b546465b..c549420e8b 100644
--- a/xen/arch/arm/include/asm/mmu/layout.h
+++ b/xen/arch/arm/include/asm/mmu/layout.h
@@ -35,9 +35,13 @@
  *
  *  32G -  64G   Frametable: 56 bytes per page for 2TB of RAM
  *
- * 0x00000a8000000000 - 0x00007fffffffffff (512GB+117TB, L0 slots [21..255])
+ * 0x00000a8000000000 - 0x00007f7fffffffff (117TB, L0 slots [21..254])
  *  Unused
  *
+ * 0x00007f8000000000 - 0x00007fffffffffff (512GB, L0 slot [255])
+ *  (Relative offsets)
+ *  0  -    2G    Domheap: on-demand-mapped
+ *
  * 0x0000800000000000 - 0x000084ffffffffff (5TB, L0 slots [256..265])
  *  1:1 mapping of RAM
  *
@@ -130,6 +134,13 @@
 #define FRAMETABLE_SIZE        GB(32)
 #define FRAMETABLE_NR          (FRAMETABLE_SIZE / sizeof(*frame_table))
 
+#define DOMHEAP_VIRT_START     SLOT0(255)
+#define DOMHEAP_VIRT_SIZE      GB(2)
+
+#define DOMHEAP_ENTRIES        1024 /* 1024 2MB mapping slots */
+/* Number of domheap pagetable pages required at the second level (2MB mappings) */
+#define DOMHEAP_SECOND_PAGES (DOMHEAP_VIRT_SIZE >> FIRST_SHIFT)
+
 #define DIRECTMAP_VIRT_START   SLOT0(256)
 #define DIRECTMAP_SIZE         (SLOT0_ENTRY_SIZE * (266 - 256))
 #define DIRECTMAP_VIRT_END     (DIRECTMAP_VIRT_START + DIRECTMAP_SIZE - 1)
diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c
index 1ed1a53ab1..da33c6c52e 100644
--- a/xen/arch/arm/mmu/pt.c
+++ b/xen/arch/arm/mmu/pt.c
@@ -33,7 +33,7 @@ mm_printk(const char *fmt, ...) {}
 #define HYP_PT_ROOT_LEVEL 1
 #endif
 
-static lpae_t *xen_map_table(mfn_t mfn)
+lpae_t *xen_map_table(mfn_t mfn)
 {
     /*
      * During early boot, map_domain_page() may be unusable. Use the
@@ -45,7 +45,7 @@ static lpae_t *xen_map_table(mfn_t mfn)
     return map_domain_page(mfn);
 }
 
-static void xen_unmap_table(const lpae_t *table)
+void xen_unmap_table(const lpae_t *table)
 {
     /*
      * During early boot, xen_map_table() will not use map_domain_page()
@@ -228,7 +228,7 @@ void *ioremap(paddr_t pa, size_t len)
     return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE);
 }
 
-static int create_xen_table(lpae_t *entry)
+int create_xen_table(lpae_t *entry)
 {
     mfn_t mfn;
     void *p;
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:22:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:22:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720864.1123891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tl4-0005Xt-QQ; Mon, 13 May 2024 11:22:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720864.1123891; Mon, 13 May 2024 11:22:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Tl4-0005Xm-Nq; Mon, 13 May 2024 11:22:18 +0000
Received: by outflank-mailman (input) for mailman id 720864;
 Mon, 13 May 2024 11:22:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TbE-0003MC-BJ
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:08 +0000
Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com
 [99.78.197.218]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1e76c72-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:12:06 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80007.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:45 +0000
Received: from EX19MTAUEC001.ant.amazon.com [10.0.0.204:26712]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.61.78:2525]
 with esmtp (Farcaster)
 id f0ad0804-eb5e-41e5-af36-c86e6fab8dc7; Mon, 13 May 2024 11:11:45 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEC001.ant.amazon.com (10.252.135.222) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:43 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:43 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1e76c72-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598726; x=1747134726;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=tb2fNfqN0eF/OdROl8BG6kceQk3KA1cAs0YAyUr8QAc=;
  b=Rfk0Z7QxZggpW5NqHdMQHN2oBmN61CUtCK9tg4A+hd6v+dWfCF6w8XZT
   hpTDHwEPWccFzOgYZH/iixgcv4U+xK8suMDUHCN+UYOMKdPr2EOFi00qV
   MpQpB+5X2NER53oF1GWwJbzpSA1Fj32J1JFOQW9bRsnAA7FnLW2KN8fV/
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="295769747"
X-Farcaster-Flow-ID: f0ad0804-eb5e-41e5-af36-c86e6fab8dc7
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>
Subject: [PATCH V3 08/19] xen/x86: Add build assertion for fixmap entries
Date: Mon, 13 May 2024 11:11:06 +0000
Message-ID: <20240513111117.68828-9-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The early fixed addresses must all fit into the static L1 table.
Introduce a build assertion to this end.

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

     Changes in v2:
         * New patch

diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
index a7ac365fc6..904bee0480 100644
--- a/xen/arch/x86/include/asm/fixmap.h
+++ b/xen/arch/x86/include/asm/fixmap.h
@@ -77,6 +77,11 @@ enum fixed_addresses {
 #define FIXADDR_SIZE  (__end_of_fixed_addresses << PAGE_SHIFT)
 #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
 
+static inline void fixaddr_build_assertion(void)
+{
+    BUILD_BUG_ON(FIX_PMAP_END > L1_PAGETABLE_ENTRIES - 1);
+}
+
 extern void __set_fixmap(
     enum fixed_addresses idx, unsigned long mfn, unsigned long flags);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:22:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:22:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720865.1123902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TlC-0005pM-1w; Mon, 13 May 2024 11:22:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720865.1123902; Mon, 13 May 2024 11:22:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TlB-0005pF-V4; Mon, 13 May 2024 11:22:25 +0000
Received: by outflank-mailman (input) for mailman id 720865;
 Mon, 13 May 2024 11:22:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TbD-0003MC-BJ
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:07 +0000
Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com
 [99.78.197.218]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0151583-1119-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 13:12:04 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80007.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:48 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:62332]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 8f2b529f-472e-4c5c-9ad3-66a0982e429a; Mon, 13 May 2024 11:11:47 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:45 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:11:45 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0151583-1119-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598725; x=1747134725;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=Xc4KQdAGUonE6oCgkT6a8PPvVb9mrXzRPtcULUVgVPo=;
  b=YfknjdWUU8mqIbVuFIkpr+b9VYAflza38Z8+VTcdAJu5+DVpsjFMsvsL
   ecaXDbStw95jlpG24sDGhJNlvZWID6NNEV0sTW8rB31fa13yvvxlJ9eb4
   NW6vzJDUiFPEf2mLW4t8Ez11B/s0XdNcZvBnDI8LdPMU78yLkGRUN2pfA
   0=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="295769775"
X-Farcaster-Flow-ID: 8f2b529f-472e-4c5c-9ad3-66a0982e429a
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH V3 09/19] x86/domain_page: Remove the fast paths when mfn is not in the directmap
Date: Mon, 13 May 2024 11:11:07 +0000
Message-ID: <20240513111117.68828-10-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When mfn is not in direct map, never use mfn_to_virt for any mappings.

We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with
arch_mfns_in_direct_map(mfn, 1) because these two are equivalent. The
extra comparison in arch_mfns_in_direct_map() looks different but because
DIRECTMAP_VIRT_END is always higher, it does not make any difference.

Lastly, domain_page_map_to_mfn() needs to gain to a special case for
the PMAP.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>

----

    Changes since Hongyan's version:
        * arch_mfn_in_direct_map() was renamed to arch_mfns_in_directmap()
        * add a special case for the PMAP in domain_page_map_to_mfn()

diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index 55e337aaf7..89caefc8a2 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -14,8 +14,10 @@
 #include <xen/sched.h>
 #include <xen/vmap.h>
 #include <asm/current.h>
+#include <asm/fixmap.h>
 #include <asm/flushtlb.h>
 #include <asm/hardirq.h>
+#include <asm/pmap.h>
 #include <asm/setup.h>
 
 static DEFINE_PER_CPU(struct vcpu *, override);
@@ -35,10 +37,11 @@ static inline struct vcpu *mapcache_current_vcpu(void)
     /*
      * When using efi runtime page tables, we have the equivalent of the idle
      * domain's page tables but current may point at another domain's VCPU.
-     * Return NULL as though current is not properly set up yet.
+     * Return the idle domains's vcpu on that core because the efi per-domain
+     * region (where the mapcache is) is in-sync with the idle domain.
      */
     if ( efi_rs_using_pgtables() )
-        return NULL;
+        return idle_vcpu[smp_processor_id()];
 
     /*
      * If guest_table is NULL, and we are running a paravirtualised guest,
@@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn)
     struct vcpu_maphash_entry *hashent;
 
 #ifdef NDEBUG
-    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
         return mfn_to_virt(mfn_x(mfn));
 #endif
 
     v = mapcache_current_vcpu();
-    if ( !v )
-        return mfn_to_virt(mfn_x(mfn));
+    if ( !v || !v->domain->arch.mapcache.inuse )
+    {
+        if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
+            return mfn_to_virt(mfn_x(mfn));
+        else
+        {
+            BUG_ON(system_state >= SYS_STATE_smp_boot);
+            return pmap_map(mfn);
+        }
+    }
 
     dcache = &v->domain->arch.mapcache;
     vcache = &v->arch.mapcache;
-    if ( !dcache->inuse )
-        return mfn_to_virt(mfn_x(mfn));
 
     perfc_incr(map_domain_page_count);
 
@@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr)
     if ( !va || va >= DIRECTMAP_VIRT_START )
         return;
 
+    if ( va >= FIXADDR_START && va < FIXADDR_TOP )
+    {
+        pmap_unmap((void *)ptr);
+        return;
+    }
+
     ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
 
     v = mapcache_current_vcpu();
@@ -237,7 +252,7 @@ int mapcache_domain_init(struct domain *d)
     unsigned int bitmap_pages;
 
 #ifdef NDEBUG
-    if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( !mem_hotplug && arch_mfn_in_directmap(0, max_page) )
         return 0;
 #endif
 
@@ -308,7 +323,7 @@ void *map_domain_page_global(mfn_t mfn)
             local_irq_is_enabled()));
 
 #ifdef NDEBUG
-    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) )
         return mfn_to_virt(mfn_x(mfn));
 #endif
 
@@ -335,6 +350,23 @@ mfn_t domain_page_map_to_mfn(const void *ptr)
     if ( va >= DIRECTMAP_VIRT_START )
         return _mfn(virt_to_mfn(ptr));
 
+    /*
+     * The fixmap is stealing the top-end of the VMAP. So the check for
+     * the PMAP *must* happen first.
+     *
+     * Also, the fixmap translate a slot to an address backwards. The
+     * logic will rely on it to avoid any complexity. So check at
+     * compile time this will always hold.
+    */
+    BUILD_BUG_ON(fix_to_virt(FIX_PMAP_BEGIN) < fix_to_virt(FIX_PMAP_END));
+
+    if ( ((unsigned long)fix_to_virt(FIX_PMAP_END) <= va) &&
+         ((va & PAGE_MASK) <= (unsigned long)fix_to_virt(FIX_PMAP_BEGIN)) )
+    {
+        BUG_ON(system_state >= SYS_STATE_smp_boot);
+        return l1e_get_mfn(l1_fixmap[l1_table_offset(va)]);
+    }
+
     if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END )
         return vmap_to_mfn(va);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 11:26:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 11:26:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720877.1123911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TpF-0007Rm-Hb; Mon, 13 May 2024 11:26:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720877.1123911; Mon, 13 May 2024 11:26:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6TpF-0007Rd-F4; Mon, 13 May 2024 11:26:37 +0000
Received: by outflank-mailman (input) for mailman id 720877;
 Mon, 13 May 2024 11:26:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6TbD-00036L-Nq
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:12:07 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a0c81764-1119-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 13:12:05 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:12:02 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.29.78:57259]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.23.176:2525]
 with esmtp (Farcaster)
 id 684ff723-13e2-42a2-ac85-d062567c045f; Mon, 13 May 2024 11:12:01 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:12:01 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 11:12:00 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0c81764-1119-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715598726; x=1747134726;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=PLKm6zqnTr1kh3lkUidFqFNcSg3ycGZChl4qwlqzWSo=;
  b=YSM4xdZdF84nXOdrn584UrbfNFjcohDsq+xehbnrH2U2df0AXd3Drz89
   hm6akw2T2MpHdhY4ochxRQTVWrJoYk+6srAcwIqz3DA+3sKALbeUnoKac
   OUhbbWOD6m6Mdg8Axk3/QzoesBegZthJWRiRe5grMeShy91ist0+3SM77
   8=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="725407129"
X-Farcaster-Flow-ID: 684ff723-13e2-42a2-ac85-d062567c045f
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 19/19] xen/arm64: Allow the admin to enable/disable the directmap
Date: Mon, 13 May 2024 11:11:17 +0000
Message-ID: <20240513111117.68828-20-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

Implement the same command line option as x86 to enable/disable the
directmap. By default this is kept enabled.

Also modify setup_directmap_mappings() to populate the L0 entries
related to the directmap area.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in v2:
        * Rely on the Kconfig option to enable Secret Hiding on Arm64
        * Use generic helper instead of arch_has_directmap()

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 743d343ffa..cccd5e4282 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -799,7 +799,7 @@ that enabling this option cannot guarantee anything beyond what underlying
 hardware guarantees (with, where available and known to Xen, respective
 tweaks applied).
 
-### directmap (x86)
+### directmap (arm64, x86)
 > `= <boolean>`
 
 > Default: `true`
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 0462960fc7..1cb495e334 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -7,6 +7,7 @@ config ARM_64
 	depends on !ARM_32
 	select 64BIT
 	select HAS_FAST_MULTIPLY
+	select HAS_SECRET_HIDING
 
 config ARM
 	def_bool y
diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 826864d25d..81115cce51 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -158,16 +158,27 @@ void __init switch_ttbr(uint64_t ttbr)
     update_identity_mapping(false);
 }
 
-/* Map the region in the directmap area. */
+/*
+ * This either populate a valid fdirect map, or allocates empty L1 tables
+ * and creates the L0 entries for the given region in the direct map
+ * depending on has_directmap().
+ *
+ * When directmap=no, we still need to populate empty L1 tables in the
+ * directmap region. The reason is that the root page-table (i.e. L0)
+ * is per-CPU and secondary CPUs will initialize their root page-table
+ * based on the pCPU0 one. So L0 entries will be shared if they are
+ * pre-populated. We also rely on the fact that L1 tables are never
+ * freed.
+ */
 static void __init setup_directmap_mappings(unsigned long base_mfn,
                                             unsigned long nr_mfns)
 {
+    unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1);
     int rc;
 
     /* First call sets the directmap physical and virtual offset. */
     if ( mfn_eq(directmap_mfn_start, INVALID_MFN) )
     {
-        unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1);
 
         directmap_mfn_start = _mfn(base_mfn);
         directmap_base_pdx = mfn_to_pdx(_mfn(base_mfn));
@@ -188,6 +199,24 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,
         panic("cannot add directmap mapping at %lx below heap start %lx\n",
               base_mfn, mfn_x(directmap_mfn_start));
 
+    if ( !has_directmap() )
+    {
+        vaddr_t vaddr = (vaddr_t)__mfn_to_virt(base_mfn);
+        lpae_t *root = this_cpu(xen_pgtable);
+        unsigned int i, slot;
+
+        slot = first_table_offset(vaddr);
+        nr_mfns += base_mfn - mfn_gb;
+        for ( i = 0; i < nr_mfns; i += BIT(XEN_PT_LEVEL_ORDER(0), UL), slot++ )
+        {
+            lpae_t *entry = &root[slot];
+
+            if ( !lpae_is_valid(*entry) && !create_xen_table(entry) )
+                panic("Unable to populate zeroeth slot %u\n", slot);
+        }
+        return;
+    }
+
     rc = map_pages_to_xen((vaddr_t)__mfn_to_virt(base_mfn),
                           _mfn(base_mfn), nr_mfns,
                           PAGE_HYPERVISOR_RW | _PAGE_BLOCK);
diff --git a/xen/arch/arm/include/asm/arm64/mm.h b/xen/arch/arm/include/asm/arm64/mm.h
index e0bd23a6ed..5888f29159 100644
--- a/xen/arch/arm/include/asm/arm64/mm.h
+++ b/xen/arch/arm/include/asm/arm64/mm.h
@@ -3,13 +3,10 @@
 
 extern DEFINE_PAGE_TABLE(xen_pgtable);
 
-/*
- * On ARM64, all the RAM is currently direct mapped in Xen.
- * Hence return always true.
- */
+/* On Arm64, the user can chose whether all the RAM is directmap. */
 static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
 {
-    return true;
+    return has_directmap();
 }
 
 void arch_setup_page_tables(void);
diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index def939172c..0f3ffab6ba 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -12,6 +12,7 @@
 #include <xen/grant_table.h>
 #include <xen/guest_access.h>
 #include <xen/mm.h>
+#include <xen/param.h>
 
 #include <xsm/xsm.h>
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d15987d6ea..6b06e2f4f5 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -778,6 +778,7 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
     cmdline_parse(cmdline);
 
     setup_mm();
+    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
 
     vm_init();
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 12:03:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 12:03:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720892.1123923 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6UOy-00043c-Ei; Mon, 13 May 2024 12:03:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720892.1123923; Mon, 13 May 2024 12:03:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6UOy-00043V-9P; Mon, 13 May 2024 12:03:32 +0000
Received: by outflank-mailman (input) for mailman id 720892;
 Mon, 13 May 2024 12:03:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6UOx-000426-GB; Mon, 13 May 2024 12:03:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6UOx-0003N3-CJ; Mon, 13 May 2024 12:03:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6UOw-0008KO-V7; Mon, 13 May 2024 12:03:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6UOw-0002iJ-Ub; Mon, 13 May 2024 12:03:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eZBzizlKE1uAM3XETxjTlTz/ZXc0UJ3voa8D1lxEg/c=; b=KyOgeZUk1nQLLquIH5EqM7c3aB
	tSBNLMpOEYCBqtsG47/JexwvCCdaLKaQAvxg1Ul7oA030z6LhGQtcqtK8miqgj7u5L4aR6R3dvA6d
	uMTWaU8qxEwGKSHB9xSTt9z0DJPV8i43a3aMl+FgFhoCE84F0skk1PY3aFVqy1TSuWHQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185987-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185987: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 May 2024 12:03:30 +0000

flight 185987 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185987/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185980
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185983
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185983
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185983
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185983
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185983
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   185987  2024-05-13 01:53:43 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon May 13 12:53:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 12:53:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720907.1123932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VAs-00043B-VS; Mon, 13 May 2024 12:53:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720907.1123932; Mon, 13 May 2024 12:53:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VAs-000434-Sy; Mon, 13 May 2024 12:53:02 +0000
Received: by outflank-mailman (input) for mailman id 720907;
 Mon, 13 May 2024 12:53:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6VAs-00042y-6Q
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 12:53:02 +0000
Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com
 [2607:f8b0:4864:20::72f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9fc0f48-1127-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 14:53:00 +0200 (CEST)
Received: by mail-qk1-x72f.google.com with SMTP id
 af79cd13be357-792bc48f6beso371909985a.2
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 05:52:59 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792e11de78esm83100185a.124.2024.05.13.05.52.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 05:52:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9fc0f48-1127-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715604779; x=1716209579; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vkm7VM+VxXvFHzvK10kDlhr1qIauXrJTB6ObfXDxdTo=;
        b=PGQhO2dFApbM+tAP8uGvAlgY6HDYKccuv6foc2hoXg8GHoEpSksIc6Fw8kRlyFEYph
         UaWxs7EW21V8HKClH3DK9B18OEzSSWPoO/mWTYLCqjhq4JMqcm4FYLG+5g3rJY+2OFLX
         cpHrApQZj7E4jcKll3HsU82CuyHQ+MzUuaSKA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715604779; x=1716209579;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vkm7VM+VxXvFHzvK10kDlhr1qIauXrJTB6ObfXDxdTo=;
        b=mqT/n3iXV/LSAwrTzK5riLGmThmN+eIAWANYfL9AmibfjUmZXtgOEzYHRan+yZf2N+
         saVvpqXPqhyZA78Pb/0hvcYsBkMDEm9AmZ/08Qqx7OtmT4Ta7KufywjeF+1IsoTFJ7ZA
         VlI9oO1n58AjyKCmAKJ4F3SsZhRGb7QqEM/MRKlZpVDVKmUrvWzY2+263/a0P3OXhdyZ
         j/3KbUNY+1u5jobOtVI1xREaj+oy4X8fGfy1qDDjabf0auESagtqONXRG1DGqtN6VOpb
         MCL9LwEPxADEDKsdWghW84sDuulAl627fvpRwDD+eTBv36oN07IVMLVWZo8ksxr6lSER
         j3VA==
X-Gm-Message-State: AOJu0YxhcSlqncXieqVLGrolBwm9s+xH0yjbr4rm8JtlWZ5v/uEOuCKH
	mvsvtl4mwxvynOPVnp7IDWAUzom0KQHz4hSWSVCfb89RFUJ6sayfIWy6BWE7ajc=
X-Google-Smtp-Source: AGHT+IGEAa3d899xeQBSaq+ipTcbnprejxB2w79KGwf2mn3KvoZcqqbNmymltEMz5zB5ZEFNbY30Dg==
X-Received: by 2002:a05:620a:995:b0:792:9652:5a1d with SMTP id af79cd13be357-792c75ff3c3mr1055850085a.66.1715604778712;
        Mon, 13 May 2024 05:52:58 -0700 (PDT)
Date: Mon, 13 May 2024 14:52:56 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Lukasz Hawrylko <lukasz@hawrylko.pl>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Mateusz =?utf-8?B?TcOzd2th?= <mateusz.mowka@intel.com>
Subject: Re: [PATCH V3 00/19] Remove the directmap
Message-ID: <ZkINKBdQW64znaec@macbook>
References: <20240513111117.68828-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com>

You seem to have forgotten to add the maintainers on Cc for the
patches.  Adding them here for reference.

Regards, Roger.

On Mon, May 13, 2024 at 11:10:58AM +0000, Elias El Yandouzi wrote:
> Hi all,
> 
> A few years ago, Wei Liu implemented a PoC to remove the directmap
> from Xen. The last version was sent by Hongyan Xia [1].
> 
> I will start with thanking both Wei and Hongyan for the initial work
> to upstream the feature. A lot of patches already went in and this is
> the last few patches missing to effectively enable the feature.
> 
> === What is the directmap? ===
> 
> At the moment, on both arm64 and x86, most of the RAM is mapped
> in Xen address space. This means that domain memory is easily
> accessible in Xen.
> 
> === Why do we want to remove the directmap? ===
> 
> (Summarizing my understanding of the previous discussion)
> 
> Speculation attacks (like Spectre SP1) rely on loading piece of memory
> in the cache. If the region is not mapped then it can't be loaded.
> 
> So removing reducing the amount of memory mapped in Xen will also
> reduce the surface attack.
> 
> === What's the performance impact? ===
> 
> As the guest memory is not always mapped, then the cost of mapping
> will increase. I haven't done the numbers with this new version, but
> some measurement were provided in the previous version for x86.
> 
> === Improvement possible ===
> 
> The known area to improve on x86 are:
>    * Mapcache: There was a patch sent by Hongyan:
>      https://lore.kernel.org/xen-devel/4058e92ce21627731c49b588a95809dc0affd83a.1581015491.git.hongyxia@amazon.com/
>    * EPT: At the moment an guest page-tabel walk requires about 20 map/unmap.
>      This will have an very high impact on the performance. We need to decide
>      whether keep the EPT always mapped is a problem
> 
> The original series didn't have support for Arm64. But as there were
> some interest, I have provided a PoC.
> 
> There are more extra work for Arm64:
>    * The mapcache is quite simple. We would investigate the performance
>    * The mapcache should be made compliant to the Arm Arm (this is now
>      more critical).
>    * We will likely have the same problem as for the EPT.
>    * We have no support for merging table to a superpage, neither
>      free empty page-tables. (See more below)
> 
> === Implementation ===
> 
> The subject is probably a misnomer. The directmap is still present but
> the RAM is not mapped by default. Instead, the region will still be used
> to map pages allocate via alloc_xenheap_pages().
> 
> The advantage is the solution is simple (so IHMO good enough for been
> merged as a tech preview). The disadvantage is the page allocator is not
> trying to keep all the xenheap pages together. So we may end up to have
> an increase of page table usage.
> 
> In the longer term, we should consider to remove the direct map
> completely and switch to vmap(). The main problem with this approach
> is it is frequent to use mfn_to_virt() in the code. So we would need
> to cache the mapping (maybe in the struct page_info).
> 
> === Why arm32 is not covered? ===
> 
> On Arm32, the domheap and xenheap is always separated. So by design
> the guest memory is not mapped by default.
> 
> At this stage, it seems unnecessary to have to map/unmap xenheap pages
> every time they are allocated.
> 
> === Why not using a separate domheap and xenheap? ===
> 
> While a separate xenheap/domheap reduce the page-table usage (all
> xenheap pages are contiguous and could be always mapped), it is also
> currently less scalable because the split is fixed at boot time (XXX:
> Can this be dynamic?).
> 
> === Future of secret-free hypervisor ===
> 
> There are some information in an e-mail from Andrew a few years ago:
> 
> https://lore.kernel.org/xen-devel/e3219697-0759-39fc-2486-715cdec1ca9e@citrix.com/
> 
> Cheers,
> 
> [1] https://lore.kernel.org/xen-devel/cover.1588278317.git.hongyxia@amazon.com/
> 
> *** BLURB HERE ***
> 
> Elias El Yandouzi (3):
>   xen/x86: Add build assertion for fixmap entries
>   Rename mfn_to_virt() calls
>   Rename maddr_to_virt() calls
> 
> Hongyan Xia (9):
>   x86: Create per-domain mapping of guest_root_pt
>   x86/pv: Rewrite how building PV dom0 handles domheap mappings
>   x86/mapcache: Initialise the mapcache for the idle domain
>   x86: Add a boot option to enable and disable the direct map
>   x86/domain_page: Remove the fast paths when mfn is not in the
>     directmap
>   xen/page_alloc: Add a path for xenheap when there is no direct map
>   x86/setup: Leave early boot slightly earlier
>   x86/setup: vmap heap nodes when they are outside the direct map
>   x86/setup: Do not create valid mappings when directmap=no
> 
> Julien Grall (5):
>   xen/x86: Add support for the PMAP
>   xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
>   xen/arm64: mm: Use per-pCPU page-tables
>   xen/arm64: Implement a mapcache for arm64
>   xen/arm64: Allow the admin to enable/disable the directmap
> 
> Wei Liu (2):
>   x86/pv: Domheap pages should be mapped while relocating initrd
>   x86: Lift mapcache variable to the arch level
> 
>  docs/misc/xen-command-line.pandoc             | 12 +++
>  xen/arch/arm/Kconfig                          |  2 +-
>  xen/arch/arm/arm64/mmu/mm.c                   | 45 ++++++++-
>  xen/arch/arm/domain_page.c                    | 50 +++++++++-
>  xen/arch/arm/include/asm/arm32/mm.h           |  8 --
>  xen/arch/arm/include/asm/arm64/mm.h           |  7 +-
>  xen/arch/arm/include/asm/domain_page.h        | 13 +++
>  xen/arch/arm/include/asm/mm.h                 |  9 ++
>  xen/arch/arm/include/asm/mmu/layout.h         | 13 ++-
>  xen/arch/arm/include/asm/mmu/mm.h             |  2 +
>  xen/arch/arm/mm.c                             |  1 +
>  xen/arch/arm/mmu/pt.c                         | 12 +--
>  xen/arch/arm/mmu/setup.c                      | 27 ++---
>  xen/arch/arm/mmu/smpboot.c                    | 30 ++----
>  xen/arch/arm/setup.c                          |  2 +
>  xen/arch/x86/Kconfig                          |  2 +
>  xen/arch/x86/dmi_scan.c                       |  4 +-
>  xen/arch/x86/domain.c                         | 12 ++-
>  xen/arch/x86/domain_page.c                    | 74 +++++++++-----
>  xen/arch/x86/hvm/dom0_build.c                 |  4 +-
>  xen/arch/x86/include/asm/config.h             | 10 +-
>  xen/arch/x86/include/asm/domain.h             | 13 +--
>  xen/arch/x86/include/asm/fixmap.h             |  9 ++
>  .../x86/include/asm/mach-default/bios_ebda.h  |  2 +-
>  xen/arch/x86/include/asm/mm.h                 |  6 ++
>  xen/arch/x86/include/asm/page.h               |  8 +-
>  xen/arch/x86/include/asm/pmap.h               | 25 +++++
>  xen/arch/x86/include/asm/x86_64/page.h        |  2 +-
>  xen/arch/x86/mm.c                             | 18 +++-
>  xen/arch/x86/mpparse.c                        |  2 +-
>  xen/arch/x86/pv/dom0_build.c                  | 70 +++++++++----
>  xen/arch/x86/pv/domain.c                      | 36 +++++++
>  xen/arch/x86/setup.c                          | 98 ++++++++++++++++---
>  xen/arch/x86/tboot.c                          |  2 +-
>  xen/arch/x86/x86_64/asm-offsets.c             |  1 +
>  xen/arch/x86/x86_64/entry.S                   |  8 ++
>  xen/arch/x86/x86_64/mm.c                      | 26 +++--
>  xen/common/Kconfig                            | 17 ++++
>  xen/common/efi/boot.c                         | 23 +++--
>  xen/common/page_alloc.c                       | 89 ++++++++++++++---
>  xen/common/trace.c                            |  8 +-
>  xen/include/xen/mm.h                          |  7 ++
>  42 files changed, 630 insertions(+), 179 deletions(-)
>  create mode 100644 xen/arch/arm/include/asm/domain_page.h
>  create mode 100644 xen/arch/x86/include/asm/pmap.h
> 
> -- 
> 2.40.1
> 
> 


From xen-devel-bounces@lists.xenproject.org Mon May 13 12:59:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 12:59:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720911.1123942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VH2-0005io-Hc; Mon, 13 May 2024 12:59:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720911.1123942; Mon, 13 May 2024 12:59:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VH2-0005ih-Cv; Mon, 13 May 2024 12:59:24 +0000
Received: by outflank-mailman (input) for mailman id 720911;
 Mon, 13 May 2024 12:59:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6VH1-0005iX-DV; Mon, 13 May 2024 12:59:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6VH1-0004I0-A4; Mon, 13 May 2024 12:59:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6VH1-00019Y-16; Mon, 13 May 2024 12:59:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6VH1-00027X-0k; Mon, 13 May 2024 12:59:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/FeSGAIpJt2CgFglZz5Vg9pJvtADc2epCkRAS6eup0o=; b=a0WQmtxP8nQ5qHGWrLnxZd8BfM
	xhjZHKhHthYikEs6BzTWHctKKIVC4/BbofYFnWTCLPHZGC9GnANNdxkjBHL+ouLRW5STNA26Yz45G
	c+pUfyLNjYGKgg9FSKim287tYabPB7VtC36eoy97RBq7A3aq/pcKzP0nxKCqi+PykScU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185988-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185988: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=9e59ba56c8a26156799a556fa79c9654a5d1acd4
X-Osstest-Versions-That:
    libvirt=d4528bb9dbf21464e68beb9175a38aaf6484536e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 May 2024 12:59:23 +0000

flight 185988 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185988/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185978
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              9e59ba56c8a26156799a556fa79c9654a5d1acd4
baseline version:
 libvirt              d4528bb9dbf21464e68beb9175a38aaf6484536e

Last test of basis   185978  2024-05-11 04:20:38 Z    2 days
Testing same since   185988  2024-05-13 04:20:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dr. David Alan Gilbert <dave@treblig.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   d4528bb9db..9e59ba56c8  9e59ba56c8a26156799a556fa79c9654a5d1acd4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720922.1123952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvG-0002ln-MJ; Mon, 13 May 2024 13:40:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720922.1123952; Mon, 13 May 2024 13:40:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvG-0002lg-J4; Mon, 13 May 2024 13:40:58 +0000
Received: by outflank-mailman (input) for mailman id 720922;
 Mon, 13 May 2024 13:40:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvE-0002lY-Ul
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:40:56 +0000
Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com
 [52.119.213.150]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6af23afd-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:40:54 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52002.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:40:52 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.29.78:34588]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.61.78:2525]
 with esmtp (Farcaster)
 id 1b0b867d-7d2c-4dca-98d9-9b5ab36c1217; Mon, 13 May 2024 13:40:51 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:51 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:50 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:40:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6af23afd-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607655; x=1747143655;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=FHUsw7Ke1dSe493VXWfS2kJQWXB8M4+Qm3akn5JUvwU=;
  b=gOoAfWS33izZk5ygFBRlQttJ1SodwtzVGyTkxq93AliKukRy06DIihd5
   fRABosNCrS9nTuZnOSNSwnpmdjX25qgU/MNIRCP+5P4p7fbBlZGZvWN/p
   m5JFHBauc0mMW03MjRZhi36aeQR54LuS2qmE23lolqvexmNyPlDQlgAfM
   4=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="632499832"
X-Farcaster-Flow-ID: 1b0b867d-7d2c-4dca-98d9-9b5ab36c1217
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>, Jan Beulich <jbeulich@suse.com>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Lukasz Hawrylko <lukasz@hawrylko.pl>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	=?UTF-8?q?Mateusz=20M=C3=B3wka?= <mateusz.mowka@intel.com>
Subject: [PATCH V3 (resend) 00/19] Remove the directmap
Date: Mon, 13 May 2024 13:40:27 +0000
Message-ID: <20240513134046.82605-1-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Hi all,

A few years ago, Wei Liu implemented a PoC to remove the directmap
from Xen. The last version was sent by Hongyan Xia [1].

I will start with thanking both Wei and Hongyan for the initial work
to upstream the feature. A lot of patches already went in and this is
the last few patches missing to effectively enable the feature.

=== What is the directmap? ===

At the moment, on both arm64 and x86, most of the RAM is mapped
in Xen address space. This means that domain memory is easily
accessible in Xen.

=== Why do we want to remove the directmap? ===

(Summarizing my understanding of the previous discussion)

Speculation attacks (like Spectre SP1) rely on loading piece of memory
in the cache. If the region is not mapped then it can't be loaded.

So removing reducing the amount of memory mapped in Xen will also
reduce the surface attack.

=== What's the performance impact? ===

As the guest memory is not always mapped, then the cost of mapping
will increase. I haven't done the numbers with this new version, but
some measurement were provided in the previous version for x86.

=== Improvement possible ===

The known area to improve on x86 are:
   * Mapcache: There was a patch sent by Hongyan:
     https://lore.kernel.org/xen-devel/4058e92ce21627731c49b588a95809dc0affd83a.1581015491.git.hongyxia@amazon.com/
   * EPT: At the moment an guest page-tabel walk requires about 20 map/unmap.
     This will have an very high impact on the performance. We need to decide
     whether keep the EPT always mapped is a problem

The original series didn't have support for Arm64. But as there were
some interest, I have provided a PoC.

There are more extra work for Arm64:
   * The mapcache is quite simple. We would investigate the performance
   * The mapcache should be made compliant to the Arm Arm (this is now
     more critical).
   * We will likely have the same problem as for the EPT.
   * We have no support for merging table to a superpage, neither
     free empty page-tables. (See more below)

=== Implementation ===

The subject is probably a misnomer. The directmap is still present but
the RAM is not mapped by default. Instead, the region will still be used
to map pages allocate via alloc_xenheap_pages().

The advantage is the solution is simple (so IHMO good enough for been
merged as a tech preview). The disadvantage is the page allocator is not
trying to keep all the xenheap pages together. So we may end up to have
an increase of page table usage.

In the longer term, we should consider to remove the direct map
completely and switch to vmap(). The main problem with this approach
is it is frequent to use mfn_to_virt() in the code. So we would need
to cache the mapping (maybe in the struct page_info).

=== Why arm32 is not covered? ===

On Arm32, the domheap and xenheap is always separated. So by design
the guest memory is not mapped by default.

At this stage, it seems unnecessary to have to map/unmap xenheap pages
every time they are allocated.

=== Why not using a separate domheap and xenheap? ===

While a separate xenheap/domheap reduce the page-table usage (all
xenheap pages are contiguous and could be always mapped), it is also
currently less scalable because the split is fixed at boot time (XXX:
Can this be dynamic?).

=== Future of secret-free hypervisor ===

There are some information in an e-mail from Andrew a few years ago:

https://lore.kernel.org/xen-devel/e3219697-0759-39fc-2486-715cdec1ca9e@citrix.com/

Cheers,

[1] https://lore.kernel.org/xen-devel/cover.1588278317.git.hongyxia@amazon.com/

*** BLURB HERE ***

Elias El Yandouzi (3):
  xen/x86: Add build assertion for fixmap entries
  Rename mfn_to_virt() calls
  Rename maddr_to_virt() calls

Hongyan Xia (9):
  x86: Create per-domain mapping of guest_root_pt
  x86/pv: Rewrite how building PV dom0 handles domheap mappings
  x86/mapcache: Initialise the mapcache for the idle domain
  x86: Add a boot option to enable and disable the direct map
  x86/domain_page: Remove the fast paths when mfn is not in the
    directmap
  xen/page_alloc: Add a path for xenheap when there is no direct map
  x86/setup: Leave early boot slightly earlier
  x86/setup: vmap heap nodes when they are outside the direct map
  x86/setup: Do not create valid mappings when directmap=no

Julien Grall (5):
  xen/x86: Add support for the PMAP
  xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
  xen/arm64: mm: Use per-pCPU page-tables
  xen/arm64: Implement a mapcache for arm64
  xen/arm64: Allow the admin to enable/disable the directmap

Wei Liu (2):
  x86/pv: Domheap pages should be mapped while relocating initrd
  x86: Lift mapcache variable to the arch level

 docs/misc/xen-command-line.pandoc             | 12 +++
 xen/arch/arm/Kconfig                          |  2 +-
 xen/arch/arm/arm64/mmu/mm.c                   | 45 ++++++++-
 xen/arch/arm/domain_page.c                    | 50 +++++++++-
 xen/arch/arm/include/asm/arm32/mm.h           |  8 --
 xen/arch/arm/include/asm/arm64/mm.h           |  7 +-
 xen/arch/arm/include/asm/domain_page.h        | 13 +++
 xen/arch/arm/include/asm/mm.h                 |  9 ++
 xen/arch/arm/include/asm/mmu/layout.h         | 13 ++-
 xen/arch/arm/include/asm/mmu/mm.h             |  2 +
 xen/arch/arm/mm.c                             |  1 +
 xen/arch/arm/mmu/pt.c                         | 12 +--
 xen/arch/arm/mmu/setup.c                      | 27 ++---
 xen/arch/arm/mmu/smpboot.c                    | 30 ++----
 xen/arch/arm/setup.c                          |  2 +
 xen/arch/x86/Kconfig                          |  2 +
 xen/arch/x86/dmi_scan.c                       |  4 +-
 xen/arch/x86/domain.c                         | 12 ++-
 xen/arch/x86/domain_page.c                    | 74 +++++++++-----
 xen/arch/x86/hvm/dom0_build.c                 |  4 +-
 xen/arch/x86/include/asm/config.h             | 10 +-
 xen/arch/x86/include/asm/domain.h             | 13 +--
 xen/arch/x86/include/asm/fixmap.h             |  9 ++
 .../x86/include/asm/mach-default/bios_ebda.h  |  2 +-
 xen/arch/x86/include/asm/mm.h                 |  6 ++
 xen/arch/x86/include/asm/page.h               |  8 +-
 xen/arch/x86/include/asm/pmap.h               | 25 +++++
 xen/arch/x86/include/asm/x86_64/page.h        |  2 +-
 xen/arch/x86/mm.c                             | 18 +++-
 xen/arch/x86/mpparse.c                        |  2 +-
 xen/arch/x86/pv/dom0_build.c                  | 70 +++++++++----
 xen/arch/x86/pv/domain.c                      | 36 +++++++
 xen/arch/x86/setup.c                          | 98 ++++++++++++++++---
 xen/arch/x86/tboot.c                          |  2 +-
 xen/arch/x86/x86_64/asm-offsets.c             |  1 +
 xen/arch/x86/x86_64/entry.S                   |  8 ++
 xen/arch/x86/x86_64/mm.c                      | 26 +++--
 xen/common/Kconfig                            | 17 ++++
 xen/common/efi/boot.c                         | 23 +++--
 xen/common/page_alloc.c                       | 89 ++++++++++++++---
 xen/common/trace.c                            |  8 +-
 xen/include/xen/mm.h                          |  7 ++
 42 files changed, 630 insertions(+), 179 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/domain_page.h
 create mode 100644 xen/arch/x86/include/asm/pmap.h

-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720924.1123972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvK-0003Fv-4y; Mon, 13 May 2024 13:41:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720924.1123972; Mon, 13 May 2024 13:41:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvK-0003Fm-0Q; Mon, 13 May 2024 13:41:02 +0000
Received: by outflank-mailman (input) for mailman id 720924;
 Mon, 13 May 2024 13:41:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvJ-0002lY-3x
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:01 +0000
Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6e11da8b-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:40:59 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-2101.iad2.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:40:57 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.0.204:28640]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.23.176:2525]
 with esmtp (Farcaster)
 id 8f053902-e51d-4042-8390-6b1417a00a31; Mon, 13 May 2024 13:40:56 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:55 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:55 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:40:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e11da8b-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607660; x=1747143660;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=X8yfiUTton8c3NyUnbJj3YCfUh5a4EGoBhji2snIQBs=;
  b=qmryHJ0I4tVzQwvtg9aHAIn9ubq1V8ZQ1PSQEVy2ZYDmKZxAuiCK5uZ4
   fobIhFtE0WKejzDTdZT1YJuij76Ik1qirXc0iiCy9DCsyCfzfcy1IcK9E
   Z5l8AmsR8FYT7dXcXIw9JKo+q6pLyK8zpYLbHLjvnkV6bKlq4/5Ah78co
   M=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="400810201"
X-Farcaster-Flow-ID: 8f053902-e51d-4042-8390-6b1417a00a31
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Wei Liu
	<wei.liu2@citrix.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Wang <wawei@amazon.de>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 (resend) 02/19] x86/pv: Domheap pages should be mapped while relocating initrd
Date: Mon, 13 May 2024 13:40:29 +0000
Message-ID: <20240513134046.82605-3-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Wei Liu <wei.liu2@citrix.com>

Xen shouldn't use domheap page as if they were xenheap pages. Map and
unmap pages accordingly.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Rename commit title
        * Rework the for loop copying the pages

    Changes in V2:
        * Get rid of mfn_to_virt
        * Don't open code copy_domain_page()

    Changes since Hongyan's version:
        * Add missing newline after the variable declaration

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index d8043fa58a..807296c280 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -618,18 +618,24 @@ int __init dom0_construct_pv(struct domain *d,
         if ( d->arch.physaddr_bitsize &&
              ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) )
         {
+            unsigned int nr_pages = 1UL << order;
+
             order = get_order_from_pages(count);
             page = alloc_domheap_pages(d, order, MEMF_no_scrub);
             if ( !page )
                 panic("Not enough RAM for domain 0 initrd\n");
+
             for ( count = -count; order--; )
                 if ( count & (1UL << order) )
                 {
                     free_domheap_pages(page, order);
                     page += 1UL << order;
+                    nr_pages -= 1UL << order;
                 }
-            memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start),
-                   initrd_len);
+
+            for ( ; nr_pages-- ; page++, mfn++ )
+                copy_domain_page(page_to_mfn(page), _mfn(mfn));
+
             mpt_alloc = (paddr_t)initrd->mod_start << PAGE_SHIFT;
             init_domheap_pages(mpt_alloc,
                                mpt_alloc + PAGE_ALIGN(initrd_len));
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720923.1123963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvI-00031N-Td; Mon, 13 May 2024 13:41:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720923.1123963; Mon, 13 May 2024 13:41:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvI-00031G-PI; Mon, 13 May 2024 13:41:00 +0000
Received: by outflank-mailman (input) for mailman id 720923;
 Mon, 13 May 2024 13:40:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvH-0002zP-RM
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:40:59 +0000
Received: from smtp-fw-52003.amazon.com (smtp-fw-52003.amazon.com
 [52.119.213.152]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d7b0301-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:40:58 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52003.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:40:56 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:58017]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.49.39:2525]
 with esmtp (Farcaster)
 id 021f1184-37dc-40b2-97a9-0457a47e1b12; Mon, 13 May 2024 13:40:55 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:53 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:53 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:40:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d7b0301-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607658; x=1747143658;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=l8CclplVrvwG16aVcnUczikHaxmRbby77xL0adFNfQA=;
  b=Qn190890JYvm4NbMVdH1dkWMFmj/Z1WGQ0VItQa6o2saT/EaZGAbZDcj
   IwbHS0FbdHQapKs5/MTOsxplqF0ZamxTW9C35L/FmW987CjnxtQlMIqYO
   m4zVh4VgXz5b3zRphzaRzNZvucQtmd4CGh8tlO7f8hBQui1fDAMBpRJD3
   8=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="658309119"
X-Farcaster-Flow-ID: 021f1184-37dc-40b2-97a9-0457a47e1b12
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 01/19] x86: Create per-domain mapping of guest_root_pt
Date: Mon, 13 May 2024 13:40:28 +0000
Message-ID: <20240513134046.82605-2-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Create a per-domain mapping of PV guest_root_pt as direct map is being
removed.

Note that we do not map and unmap root_pgt for now since it is still a
xenheap page.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Rename SHADOW_ROOT
        * Haven't addressed the potentially over-allocation issue as I don't get it

    Changes in V2:
        * Rework the shadow perdomain mapping solution in the follow-up patches

    Changes since Hongyan's version:
        * Remove the final dot in the commit title

diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
index ab7288cb36..5d710384df 100644
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -203,7 +203,7 @@ extern unsigned char boot_edid_info[128];
 /* Slot 260: per-domain mappings (including map cache). */
 #define PERDOMAIN_VIRT_START    (PML4_ADDR(260))
 #define PERDOMAIN_SLOT_MBYTES   (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER))
-#define PERDOMAIN_SLOTS         3
+#define PERDOMAIN_SLOTS         4
 #define PERDOMAIN_VIRT_SLOT(s)  (PERDOMAIN_VIRT_START + (s) * \
                                  (PERDOMAIN_SLOT_MBYTES << 20))
 /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */
@@ -317,6 +317,14 @@ extern unsigned long xen_phys_start;
 #define ARG_XLAT_START(v)        \
     (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
 
+/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */
+#define PV_ROOT_PT_MAPPING_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
+#define PV_ROOT_PT_MAPPING_ENTRIES      MAX_VIRT_CPUS
+
+/* The address of a particular VCPU's PV_ROOT_PT */
+#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \
+    (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))
+
 #define ELFSIZE 64
 
 #define ARCH_CRASH_SAVE_VMCOREINFO
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index f5daeb182b..8a97530607 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -272,6 +272,7 @@ struct time_scale {
 struct pv_domain
 {
     l1_pgentry_t **gdt_ldt_l1tab;
+    l1_pgentry_t **root_pt_l1tab;
 
     atomic_t nr_l4_pages;
 
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc73..efdf20f775 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -505,6 +505,13 @@ void share_xen_page_with_guest(struct page_info *page, struct domain *d,
     nrspin_unlock(&d->page_alloc_lock);
 }
 
+#define pv_root_pt_idx(v) \
+    ((v)->vcpu_id >> PAGETABLE_ORDER)
+
+#define pv_root_pt_pte(v) \
+    ((v)->domain->arch.pv.root_pt_l1tab[pv_root_pt_idx(v)] + \
+     ((v)->vcpu_id & (L1_PAGETABLE_ENTRIES - 1)))
+
 void make_cr3(struct vcpu *v, mfn_t mfn)
 {
     struct domain *d = v->domain;
@@ -524,6 +531,13 @@ void write_ptbase(struct vcpu *v)
 
     if ( is_pv_vcpu(v) && v->domain->arch.pv.xpti )
     {
+        mfn_t guest_root_pt = _mfn(MASK_EXTR(v->arch.cr3, PAGE_MASK));
+        l1_pgentry_t *pte = pv_root_pt_pte(v);
+
+        ASSERT(v == current);
+
+        l1e_write(pte, l1e_from_mfn(guest_root_pt, __PAGE_HYPERVISOR_RO));
+
         cpu_info->root_pgt_changed = true;
         cpu_info->pv_cr3 = __pa(this_cpu(root_pgt));
         if ( new_cr4 & X86_CR4_PCIDE )
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 2a445bb17b..1b025986f7 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -288,6 +288,21 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
                               1U << GDT_LDT_VCPU_SHIFT);
 }
 
+static int pv_create_root_pt_l1tab(struct vcpu *v)
+{
+    return create_perdomain_mapping(v->domain,
+                                    PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v),
+                                    1, v->domain->arch.pv.root_pt_l1tab,
+                                    NULL);
+}
+
+static void pv_destroy_root_pt_l1tab(struct vcpu *v)
+
+{
+    destroy_perdomain_mapping(v->domain,
+                              PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v), 1);
+}
+
 void pv_vcpu_destroy(struct vcpu *v)
 {
     if ( is_pv_32bit_vcpu(v) )
@@ -297,6 +312,7 @@ void pv_vcpu_destroy(struct vcpu *v)
     }
 
     pv_destroy_gdt_ldt_l1tab(v);
+    pv_destroy_root_pt_l1tab(v);
     XFREE(v->arch.pv.trap_ctxt);
 }
 
@@ -311,6 +327,13 @@ int pv_vcpu_initialise(struct vcpu *v)
     if ( rc )
         return rc;
 
+    if ( v->domain->arch.pv.xpti )
+    {
+        rc = pv_create_root_pt_l1tab(v);
+        if ( rc )
+            goto done;
+    }
+
     BUILD_BUG_ON(X86_NR_VECTORS * sizeof(*v->arch.pv.trap_ctxt) >
                  PAGE_SIZE);
     v->arch.pv.trap_ctxt = xzalloc_array(struct trap_info, X86_NR_VECTORS);
@@ -346,10 +369,12 @@ void pv_domain_destroy(struct domain *d)
 
     destroy_perdomain_mapping(d, GDT_LDT_VIRT_START,
                               GDT_LDT_MBYTES << (20 - PAGE_SHIFT));
+    destroy_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START, PV_ROOT_PT_MAPPING_ENTRIES);
 
     XFREE(d->arch.pv.cpuidmasks);
 
     FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab);
+    FREE_XENHEAP_PAGE(d->arch.pv.root_pt_l1tab);
 }
 
 void noreturn cf_check continue_pv_domain(void);
@@ -371,6 +396,12 @@ int pv_domain_initialise(struct domain *d)
         goto fail;
     clear_page(d->arch.pv.gdt_ldt_l1tab);
 
+    d->arch.pv.root_pt_l1tab =
+        alloc_xenheap_pages(0, MEMF_node(domain_to_node(d)));
+    if ( !d->arch.pv.root_pt_l1tab )
+        goto fail;
+    clear_page(d->arch.pv.root_pt_l1tab);
+
     if ( levelling_caps & ~LCAP_faulting &&
          (d->arch.pv.cpuidmasks = xmemdup(&cpuidmask_defaults)) == NULL )
         goto fail;
@@ -381,6 +412,11 @@ int pv_domain_initialise(struct domain *d)
     if ( rc )
         goto fail;
 
+    rc = create_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START,
+                                  PV_ROOT_PT_MAPPING_ENTRIES, NULL, NULL);
+    if ( rc )
+        goto fail;
+
     d->arch.ctxt_switch = &pv_csw;
 
     d->arch.pv.xpti = is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu;
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 630bdc3945..c1ae5013af 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -80,6 +80,7 @@ void __dummy__(void)
 
 #undef OFFSET_EF
 
+    OFFSET(VCPU_id, struct vcpu, vcpu_id);
     OFFSET(VCPU_processor, struct vcpu, processor);
     OFFSET(VCPU_domain, struct vcpu, domain);
     OFFSET(VCPU_vcpu_info, struct vcpu, vcpu_info_area.map);
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index df015589ce..c1377da7a5 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -162,7 +162,15 @@ FUNC_LOCAL(restore_all_guest)
         and   %rsi, %rdi
         and   %r9, %rsi
         add   %rcx, %rdi
+
+        /*
+         * The address in the vCPU cr3 is always mapped in the per-domain
+         * pv_root_pt virt area.
+         */
+        imul  $PAGE_SIZE, VCPU_id(%rbx), %esi
+        movabs $PV_ROOT_PT_MAPPING_VIRT_START, %rcx
         add   %rcx, %rsi
+
         mov   $ROOT_PAGETABLE_FIRST_XEN_SLOT, %ecx
         mov   root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rsi), %r8
         mov   %r8, root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rdi)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720925.1123982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvL-0003Uv-Bv; Mon, 13 May 2024 13:41:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720925.1123982; Mon, 13 May 2024 13:41:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvL-0003Um-6o; Mon, 13 May 2024 13:41:03 +0000
Received: by outflank-mailman (input) for mailman id 720925;
 Mon, 13 May 2024 13:41:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvK-0002zP-1C
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:02 +0000
Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6f18f246-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:01 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-2101.iad2.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:00 +0000
Received: from EX19MTAUEC001.ant.amazon.com [10.0.29.78:20285]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id da52c68b-ca18-439b-8681-3a40d2ff44ac; Mon, 13 May 2024 13:40:59 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEC001.ant.amazon.com (10.252.135.222) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:57 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:57 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:40:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f18f246-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607662; x=1747143662;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=awTpJTVuZnpLM2XGjDjoGafH08PnS/u8+TgC4HSOyp8=;
  b=ZG+b+vK76qC/Bvo2fhIShUYZ745mK6DDLVzyRYeUwV08oYOH0o03A4Nw
   qTgRtp/UvdBfI9AISgrJ5H/I/Z37D4Sxf0jGekLyqvupGMbwC23IwYhPV
   yXXLD4FF89SE65ilb67u+OILxkn7FDxuqYxoh/32RCoigPntJ83+cIAjU
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="400810215"
X-Farcaster-Flow-ID: da52c68b-ca18-439b-8681-3a40d2ff44ac
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 03/19] x86/pv: Rewrite how building PV dom0 handles domheap mappings
Date: Mon, 13 May 2024 13:40:30 +0000
Message-ID: <20240513134046.82605-4-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Building a PV dom0 is allocating from the domheap but uses it like the
xenheap. Use the pages as they should be.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----
    Changes in V3:
        * Fold following patch 'x86/pv: Map L4 page table for shim domain'

    Changes in V2:
        * Clarify the commit message
        * Break the patch in two parts

    Changes since Hongyan's version:
        * Rebase
        * Remove spurious newline

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index 807296c280..ac910b438a 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
     l3_pgentry_t *l3tab = NULL, *l3start = NULL;
     l2_pgentry_t *l2tab = NULL, *l2start = NULL;
     l1_pgentry_t *l1tab = NULL, *l1start = NULL;
+    mfn_t l4start_mfn = INVALID_MFN;
+    mfn_t l3start_mfn = INVALID_MFN;
+    mfn_t l2start_mfn = INVALID_MFN;
+    mfn_t l1start_mfn = INVALID_MFN;
 
     /*
      * This fully describes the memory layout of the initial domain. All
@@ -710,22 +714,32 @@ int __init dom0_construct_pv(struct domain *d,
         v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
     }
 
+#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
+do {                                                    \
+    unmap_domain_page(virt_var);                        \
+    mfn_var = maddr_to_mfn(maddr);                      \
+    maddr += PAGE_SIZE;                                 \
+    virt_var = map_domain_page(mfn_var);                \
+} while ( false )
+
     if ( !compat )
     {
         maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
-        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
+        l4tab = l4start;
         clear_page(l4tab);
-        init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)),
-                          d, INVALID_MFN, true);
-        v->arch.guest_table = pagetable_from_paddr(__pa(l4start));
+        init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true);
+        v->arch.guest_table = pagetable_from_mfn(l4start_mfn);
     }
     else
     {
         /* Monitor table already created by switch_compat(). */
-        l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table));
+        l4start_mfn = pagetable_get_mfn(v->arch.guest_table);
+        l4start = l4tab = map_domain_page(l4start_mfn);
         /* See public/xen.h on why the following is needed. */
         maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table;
         l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+        UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc);
     }
 
     l4tab += l4_table_offset(v_start);
@@ -735,14 +749,16 @@ int __init dom0_construct_pv(struct domain *d,
         if ( !((unsigned long)l1tab & (PAGE_SIZE-1)) )
         {
             maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l1_page_table;
-            l1start = l1tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+            UNMAP_MAP_AND_ADVANCE(l1start_mfn, l1start, mpt_alloc);
+            l1tab = l1start;
             clear_page(l1tab);
             if ( count == 0 )
                 l1tab += l1_table_offset(v_start);
             if ( !((unsigned long)l2tab & (PAGE_SIZE-1)) )
             {
                 maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
-                l2start = l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
+                l2tab = l2start;
                 clear_page(l2tab);
                 if ( count == 0 )
                     l2tab += l2_table_offset(v_start);
@@ -752,19 +768,19 @@ int __init dom0_construct_pv(struct domain *d,
                     {
                         maddr_to_page(mpt_alloc)->u.inuse.type_info =
                             PGT_l3_page_table;
-                        l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
+                        UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc);
                     }
                     l3tab = l3start;
                     clear_page(l3tab);
                     if ( count == 0 )
                         l3tab += l3_table_offset(v_start);
-                    *l4tab = l4e_from_paddr(__pa(l3start), L4_PROT);
+                    *l4tab = l4e_from_mfn(l3start_mfn, L4_PROT);
                     l4tab++;
                 }
-                *l3tab = l3e_from_paddr(__pa(l2start), L3_PROT);
+                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
                 l3tab++;
             }
-            *l2tab = l2e_from_paddr(__pa(l1start), L2_PROT);
+            *l2tab = l2e_from_mfn(l1start_mfn, L2_PROT);
             l2tab++;
         }
         if ( count < initrd_pfn || count >= initrd_pfn + PFN_UP(initrd_len) )
@@ -783,27 +799,31 @@ int __init dom0_construct_pv(struct domain *d,
 
     if ( compat )
     {
-        l2_pgentry_t *l2t;
-
         /* Ensure the first four L3 entries are all populated. */
         for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
         {
             if ( !l3e_get_intpte(*l3tab) )
             {
                 maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
-                l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
-                clear_page(l2tab);
-                *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT);
+                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
+                clear_page(l2start);
+                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
             }
             if ( i == 3 )
                 l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
         }
 
-        l2t = map_l2t_from_l3e(l3start[3]);
-        init_xen_pae_l2_slots(l2t, d);
-        unmap_domain_page(l2t);
+        UNMAP_DOMAIN_PAGE(l2start);
+        l2start = map_l2t_from_l3e(l3start[3]);
+        init_xen_pae_l2_slots(l2start, d);
     }
 
+#undef UNMAP_MAP_AND_ADVANCE
+
+    UNMAP_DOMAIN_PAGE(l1start);
+    UNMAP_DOMAIN_PAGE(l2start);
+    UNMAP_DOMAIN_PAGE(l3start);
+
     /* Pages that are part of page tables must be read only. */
     mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages, &flush_flags);
 
@@ -976,6 +996,8 @@ int __init dom0_construct_pv(struct domain *d,
         pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start,
                           vphysmap_start, si);
 
+    UNMAP_DOMAIN_PAGE(l4start);
+
 #ifdef CONFIG_COMPAT
     if ( compat )
         xlat_start_info(si, pv_shim ? XLAT_start_info_console_domU
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720926.1123992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvV-0003xk-NB; Mon, 13 May 2024 13:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720926.1123992; Mon, 13 May 2024 13:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvV-0003xZ-Jd; Mon, 13 May 2024 13:41:13 +0000
Received: by outflank-mailman (input) for mailman id 720926;
 Mon, 13 May 2024 13:41:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvT-0002lY-Ox
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:11 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73fb9111-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:09 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:01 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.0.204:46266]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.20.152:2525]
 with esmtp (Farcaster)
 id 6668fe64-0491-4100-8058-7f885684559c; Mon, 13 May 2024 13:41:00 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:00 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:40:59 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:40:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73fb9111-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607670; x=1747143670;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=asW3riQLlEfjH7OhZSjftd0GSOA/iaeDlEAWCyKlviw=;
  b=Rug3P68yJ6Q+GNOGcYGnr83BjCfyIS2Ym+1hCzZjP3WFm4jLY2clTzh8
   pHfkFAtWBGdwTyCNo63BYRm68IPdFj7GSf6EjefnY5aAvjSw9BHijawW7
   6x3VVlxtT6asllcwEcZ5Cj8ax7SZwmCbw5sbMJetXmliHkWRWOs65M9v7
   I=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="725436938"
X-Farcaster-Flow-ID: 6668fe64-0491-4100-8058-7f885684559c
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Wei Liu
	<wei.liu2@citrix.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Wang <wawei@amazon.de>, Hongyan Xia
	<hongyxia@amazon.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH V3 (resend) 04/19] x86: Lift mapcache variable to the arch level
Date: Mon, 13 May 2024 13:40:31 +0000
Message-ID: <20240513134046.82605-5-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Wei Liu <wei.liu2@citrix.com>

It is going to be needed by HVM and idle domain as well, because without
the direct map, both need a mapcache to map pages.

This commit lifts the mapcache variable up and initialise it a bit earlier
for PV and HVM domains.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 20e83cf38b..507d704f16 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -851,6 +851,8 @@ int arch_domain_create(struct domain *d,
 
     psr_domain_init(d);
 
+    mapcache_domain_init(d);
+
     if ( is_hvm_domain(d) )
     {
         if ( (rc = hvm_domain_initialise(d, config)) != 0 )
@@ -858,8 +860,6 @@ int arch_domain_create(struct domain *d,
     }
     else if ( is_pv_domain(d) )
     {
-        mapcache_domain_init(d);
-
         if ( (rc = pv_domain_initialise(d)) != 0 )
             goto fail;
     }
diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index eac5e3304f..55e337aaf7 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
 #endif
 
     v = mapcache_current_vcpu();
-    if ( !v || !is_pv_vcpu(v) )
+    if ( !v )
         return mfn_to_virt(mfn_x(mfn));
 
-    dcache = &v->domain->arch.pv.mapcache;
-    vcache = &v->arch.pv.mapcache;
+    dcache = &v->domain->arch.mapcache;
+    vcache = &v->arch.mapcache;
     if ( !dcache->inuse )
         return mfn_to_virt(mfn_x(mfn));
 
@@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr)
     ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
 
     v = mapcache_current_vcpu();
-    ASSERT(v && is_pv_vcpu(v));
+    ASSERT(v);
 
-    dcache = &v->domain->arch.pv.mapcache;
+    dcache = &v->domain->arch.mapcache;
     ASSERT(dcache->inuse);
 
     idx = PFN_DOWN(va - MAPCACHE_VIRT_START);
     mfn = l1e_get_pfn(MAPCACHE_L1ENT(idx));
-    hashent = &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)];
+    hashent = &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)];
 
     local_irq_save(flags);
 
@@ -233,11 +233,9 @@ void unmap_domain_page(const void *ptr)
 
 int mapcache_domain_init(struct domain *d)
 {
-    struct mapcache_domain *dcache = &d->arch.pv.mapcache;
+    struct mapcache_domain *dcache = &d->arch.mapcache;
     unsigned int bitmap_pages;
 
-    ASSERT(is_pv_domain(d));
-
 #ifdef NDEBUG
     if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
         return 0;
@@ -261,12 +259,12 @@ int mapcache_domain_init(struct domain *d)
 int mapcache_vcpu_init(struct vcpu *v)
 {
     struct domain *d = v->domain;
-    struct mapcache_domain *dcache = &d->arch.pv.mapcache;
+    struct mapcache_domain *dcache = &d->arch.mapcache;
     unsigned long i;
     unsigned int ents = d->max_vcpus * MAPCACHE_VCPU_ENTRIES;
     unsigned int nr = PFN_UP(BITS_TO_LONGS(ents) * sizeof(long));
 
-    if ( !is_pv_vcpu(v) || !dcache->inuse )
+    if ( !dcache->inuse )
         return 0;
 
     if ( ents > dcache->entries )
@@ -293,7 +291,7 @@ int mapcache_vcpu_init(struct vcpu *v)
     BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES);
     for ( i = 0; i < MAPHASH_ENTRIES; i++ )
     {
-        struct vcpu_maphash_entry *hashent = &v->arch.pv.mapcache.hash[i];
+        struct vcpu_maphash_entry *hashent = &v->arch.mapcache.hash[i];
 
         hashent->mfn = ~0UL; /* never valid to map */
         hashent->idx = MAPHASHENT_NOTINUSE;
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index 8a97530607..7f0480d7a7 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -285,9 +285,6 @@ struct pv_domain
     /* Mitigate L1TF with shadow/crashing? */
     bool check_l1tf;
 
-    /* map_domain_page() mapping cache. */
-    struct mapcache_domain mapcache;
-
     struct cpuidmasks *cpuidmasks;
 };
 
@@ -326,6 +323,9 @@ struct arch_domain
 
     uint8_t scf; /* See SCF_DOM_MASK */
 
+    /* map_domain_page() mapping cache. */
+    struct mapcache_domain mapcache;
+
     union {
         struct pv_domain pv;
         struct hvm_domain hvm;
@@ -516,9 +516,6 @@ struct arch_domain
 
 struct pv_vcpu
 {
-    /* map_domain_page() mapping cache. */
-    struct mapcache_vcpu mapcache;
-
     unsigned int vgc_flags;
 
     struct trap_info *trap_ctxt;
@@ -618,6 +615,9 @@ struct arch_vcpu
 #define async_exception_state(t) async_exception_state[(t)-1]
     uint8_t async_exception_mask;
 
+    /* map_domain_page() mapping cache. */
+    struct mapcache_vcpu mapcache;
+
     /* Virtual Machine Extensions */
     union {
         struct pv_vcpu pv;
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720927.1123998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvW-00041q-8W; Mon, 13 May 2024 13:41:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720927.1123998; Mon, 13 May 2024 13:41:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvV-00040w-V3; Mon, 13 May 2024 13:41:13 +0000
Received: by outflank-mailman (input) for mailman id 720927;
 Mon, 13 May 2024 13:41:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvU-0002lY-P7
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:12 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7461a6a4-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:10 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:06 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:8888]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.42.97:2525]
 with esmtp (Farcaster)
 id 7a338e33-2920-4928-86c0-716608c0b7ff; Mon, 13 May 2024 13:41:06 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:02 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:01 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7461a6a4-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607670; x=1747143670;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=TyQ1NOFs+JBNhEZ9nul6mULL97GoHUzPHp2fRNIcmm8=;
  b=HNwzFK4Nk+oVDM9J24x97/R7AEqYlxqBWwIL7yypUJW3j0WNpa2VFi5k
   sO5yIl6/aBtYDh3apEP75oCoR07ij6kpD9KGmDRFyA1xdo7Wh6NDWeRdS
   BmDKdUABdX69Alxlkw7TkU2ryzvzIAgvZ1MuKFDBMTX7k7HMA446kFnwG
   g=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88857716"
X-Farcaster-Flow-ID: 7a338e33-2920-4928-86c0-716608c0b7ff
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Wang <wawei@amazon.de>, Julien Grall
	<jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 (resend) 05/19] x86/mapcache: Initialise the mapcache for the idle domain
Date: Mon, 13 May 2024 13:40:32 +0000
Message-ID: <20240513134046.82605-6-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

In order to use the mapcache in the idle domain, we also have to
populate its page tables in the PERDOMAIN region, and we need to move
mapcache_domain_init() earlier in arch_domain_create().

Note, commit 'x86: lift mapcache variable to the arch level' has
initialised the mapcache for HVM domains. With this patch, PV, HVM,
idle domains now all initialise the mapcache.

Signed-off-by: Wei Wang <wawei@amazon.de>
Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

	Changes in V2:
          * Free resources if mapcache initialisation fails
          * Remove `is_idle_domain()` check from `create_perdomain_mappings()`

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 507d704f16..3303bdb53e 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -758,9 +758,16 @@ int arch_domain_create(struct domain *d,
 
     spin_lock_init(&d->arch.e820_lock);
 
+    if ( (rc = mapcache_domain_init(d)) != 0)
+    {
+        free_perdomain_mappings(d);
+        return rc;
+    }
+
     /* Minimal initialisation for the idle domain. */
     if ( unlikely(is_idle_domain(d)) )
     {
+        struct page_info *pg = d->arch.perdomain_l3_pg;
         static const struct arch_csw idle_csw = {
             .from = paravirt_ctxt_switch_from,
             .to   = paravirt_ctxt_switch_to,
@@ -771,6 +778,9 @@ int arch_domain_create(struct domain *d,
 
         d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
 
+        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
+            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
+
         return 0;
     }
 
@@ -851,8 +861,6 @@ int arch_domain_create(struct domain *d,
 
     psr_domain_init(d);
 
-    mapcache_domain_init(d);
-
     if ( is_hvm_domain(d) )
     {
         if ( (rc = hvm_domain_initialise(d, config)) != 0 )
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720928.1124003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvW-00049q-Kj; Mon, 13 May 2024 13:41:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720928.1124003; Mon, 13 May 2024 13:41:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvW-00047x-CI; Mon, 13 May 2024 13:41:14 +0000
Received: by outflank-mailman (input) for mailman id 720928;
 Mon, 13 May 2024 13:41:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvV-0002zP-4g
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:13 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 75519601-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:12 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:08 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.29.78:3835]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.82.155:2525]
 with esmtp (Farcaster)
 id a4c1d04a-7ebb-419c-a634-05a07b3358b1; Mon, 13 May 2024 13:41:07 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:06 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:06 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75519601-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607672; x=1747143672;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=ZJrhmMV/WsLJjJ2b/S8Feij3BU9bNlhqRg9vQL1/RK0=;
  b=Qktw8LKQR3PNzly8tz8uCw92FwhbmOZT3J/eSTpxJImBYjSe4gZ3WcwR
   xO1E1edyvCtkPUxjkpvUloDLxQasoSXDZyil3V8SOhDNahjtbbrXzjZVi
   Cw4C6/USj9tfWJvjO1Bo818ZABgM+c2RdMjNyhTekYlKUUlyCXcY1FRoe
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="294692541"
X-Farcaster-Flow-ID: a4c1d04a-7ebb-419c-a634-05a07b3358b1
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Elias El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Date: Mon, 13 May 2024 13:40:34 +0000
Message-ID: <20240513134046.82605-8-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

PMAP will be used in a follow-up patch to bootstrap map domain
page infrastructure -- we need some way to map pages to setup the
mapcache without a direct map.

The functions pmap_{map, unmap} open code {set, clear}_fixmap to break
the loop.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    The PMAP infrastructure was upstream separately for Arm since
    Hongyan sent the secret-free hypervisor series. So this is a new
    patch to plumb the feature on x86.

    Changes in v2:
        * Declare PMAP entries earlier in fixed_addresses
        * Reword the commit message

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index b4ec0e582e..56feb0c564 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -27,6 +27,7 @@ config X86
 	select HAS_PCI
 	select HAS_PCI_MSI
 	select HAS_PIRQ
+	select HAS_PMAP
 	select HAS_SCHED_GRANULARITY
 	select HAS_SECRET_HIDING
 	select HAS_UBSAN
diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
index 516ec3fa6c..a7ac365fc6 100644
--- a/xen/arch/x86/include/asm/fixmap.h
+++ b/xen/arch/x86/include/asm/fixmap.h
@@ -21,6 +21,8 @@
 
 #include <xen/acpi.h>
 #include <xen/pfn.h>
+#include <xen/pmap.h>
+
 #include <asm/apicdef.h>
 #include <asm/msi.h>
 #include <acpi/apei.h>
@@ -53,6 +55,8 @@ enum fixed_addresses {
     FIX_PV_CONSOLE,
     FIX_XEN_SHARED_INFO,
 #endif /* CONFIG_XEN_GUEST */
+    FIX_PMAP_BEGIN,
+    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
     /* Everything else should go further down. */
     FIX_APIC_BASE,
     FIX_IO_APIC_BASE_0,
diff --git a/xen/arch/x86/include/asm/pmap.h b/xen/arch/x86/include/asm/pmap.h
new file mode 100644
index 0000000000..62746e191d
--- /dev/null
+++ b/xen/arch/x86/include/asm/pmap.h
@@ -0,0 +1,25 @@
+#ifndef __ASM_PMAP_H__
+#define __ASM_PMAP_H__
+
+#include <asm/fixmap.h>
+
+static inline void arch_pmap_map(unsigned int slot, mfn_t mfn)
+{
+    unsigned long linear = (unsigned long)fix_to_virt(slot);
+    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
+
+    ASSERT(!(l1e_get_flags(*pl1e) & _PAGE_PRESENT));
+
+    l1e_write_atomic(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR));
+}
+
+static inline void arch_pmap_unmap(unsigned int slot)
+{
+    unsigned long linear = (unsigned long)fix_to_virt(slot);
+    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
+
+    l1e_write_atomic(pl1e, l1e_empty());
+    flush_tlb_one_local(linear);
+}
+
+#endif /* __ASM_PMAP_H__ */
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720929.1124022 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvY-0004lI-Rd; Mon, 13 May 2024 13:41:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720929.1124022; Mon, 13 May 2024 13:41:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvY-0004l2-Nd; Mon, 13 May 2024 13:41:16 +0000
Received: by outflank-mailman (input) for mailman id 720929;
 Mon, 13 May 2024 13:41:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvW-0002lY-G8
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:14 +0000
Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7607bc47-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:12 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-6002.iad6.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:10 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.29.78:59609]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id b0625eca-4095-429b-ae8c-57fb22cad0fb; Mon, 13 May 2024 13:41:09 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:08 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:07 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7607bc47-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607673; x=1747143673;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=tb2fNfqN0eF/OdROl8BG6kceQk3KA1cAs0YAyUr8QAc=;
  b=IObegGfxLKD8fddKWloBtG6mYxTR4BATPPEiPpqc0T8EhiOQ76oP2GIX
   XrRA1Ayh4dt/lVQ9j5fTRYbv54h0Na6E/j8zfl0dyXSfO9M4/0n5n8VtU
   SYV+5ZYwYNYpiSD28b7T78CKmFjDhdT8X6g3oA0Zu6epVeA0GcGUoLzxB
   I=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="406401335"
X-Farcaster-Flow-ID: b0625eca-4095-429b-ae8c-57fb22cad0fb
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>, Jan Beulich <jbeulich@suse.com>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>
Subject: [PATCH V3 (resend) 08/19] xen/x86: Add build assertion for fixmap entries
Date: Mon, 13 May 2024 13:40:35 +0000
Message-ID: <20240513134046.82605-9-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The early fixed addresses must all fit into the static L1 table.
Introduce a build assertion to this end.

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

     Changes in v2:
         * New patch

diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
index a7ac365fc6..904bee0480 100644
--- a/xen/arch/x86/include/asm/fixmap.h
+++ b/xen/arch/x86/include/asm/fixmap.h
@@ -77,6 +77,11 @@ enum fixed_addresses {
 #define FIXADDR_SIZE  (__end_of_fixed_addresses << PAGE_SHIFT)
 #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
 
+static inline void fixaddr_build_assertion(void)
+{
+    BUILD_BUG_ON(FIX_PMAP_END > L1_PAGETABLE_ENTRIES - 1);
+}
+
 extern void __set_fixmap(
     enum fixed_addresses idx, unsigned long mfn, unsigned long flags);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720930.1124027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvZ-0004pL-7p; Mon, 13 May 2024 13:41:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720930.1124027; Mon, 13 May 2024 13:41:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvZ-0004oI-0H; Mon, 13 May 2024 13:41:17 +0000
Received: by outflank-mailman (input) for mailman id 720930;
 Mon, 13 May 2024 13:41:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvW-0002lY-Sh
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:14 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7682f9fd-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:12 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:08 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.0.204:52542]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id 111a19b8-6e8a-4b34-aba9-a09724c45941; Mon, 13 May 2024 13:41:08 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:04 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:04 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7682f9fd-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607673; x=1747143673;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=YiMscZoqlqA2XJ1v3gO3e8DXR6HvvHTgLBqJAfmepmI=;
  b=HUQ7GfIFvo7e1Fn0AnP3cCiYzhtqHgQKuHy9nIbEYwr6LGx946yyNdm3
   hGOpSp8Flm1bJpw/bv10j3eE9kv3FzOrzP3OTl6UAcXR9OqANFNyQUsK+
   7FAd7SWddaEWKa1v6nH0oXbWhj50x7FMHB4OaPE89GaD5+4Xi9EeCFxAG
   o=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88857729"
X-Farcaster-Flow-ID: 111a19b8-6e8a-4b34-aba9-a09724c45941
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and disable the direct map
Date: Mon, 13 May 2024 13:40:33 +0000
Message-ID: <20240513134046.82605-7-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Also add a helper function to retrieve it. Change arch_mfns_in_direct_map
to check this option before returning.

This is added as a Kconfig option as well as a boot command line option.
While being generic, the Kconfig option is only usable for x86 at the moment.

Note that there remains some users of the directmap at this point. The option
is introduced now as it will be needed in follow-up patches.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in V2:
        * Introduce a Kconfig option
        * Reword the commit message
        * Make opt_directmap and helper generic

    Changes since Hongyan's version:
        * Reword the commit message
        * opt_directmap is only modified during boot so mark it as
          __ro_after_init

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index e760f3266e..743d343ffa 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
 hardware guarantees (with, where available and known to Xen, respective
 tweaks applied).
 
+### directmap (x86)
+> `= <boolean>`
+
+> Default: `true`
+
+Enable or disable the directmap region in Xen.
+
+By default, Xen creates the directmap region which maps physical memory
+in that region. Setting this to no will sparsely populate the directmap,
+blocking exploits that leak secrets via speculative memory access in the
+directmap.
+
 ### dma_bits
 > `= <integer>`
 
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 7e03e4bc55..b4ec0e582e 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -28,6 +28,7 @@ config X86
 	select HAS_PCI_MSI
 	select HAS_PIRQ
 	select HAS_SCHED_GRANULARITY
+	select HAS_SECRET_HIDING
 	select HAS_UBSAN
 	select HAS_VPCI if HVM
 	select NEEDS_LIBELF
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 98b66edaca..54d835f156 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -622,11 +622,17 @@ void write_32bit_pse_identmap(uint32_t *l2);
 /*
  * x86 maps part of physical memory via the directmap region.
  * Return whether the range of MFN falls in the directmap region.
+ *
+ * When boot command line sets directmap=no, the directmap will mostly be empty
+ * so this will always return false.
  */
 static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
 {
     unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
 
+    if ( !has_directmap() )
+        return false;
+
     return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
 }
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index f84e1cd79c..bd6b1184f5 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     if ( highmem_start )
         xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
 
+    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
+
     /*
      * Walk every RAM region and map it in its entirety (on x86/64, at least)
      * and notify it to the boot allocator.
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 565ceda741..856604068c 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -80,12 +80,29 @@ config HAS_PMAP
 config HAS_SCHED_GRANULARITY
 	bool
 
+config HAS_SECRET_HIDING
+	bool
+
 config HAS_UBSAN
 	bool
 
 config MEM_ACCESS_ALWAYS_ON
 	bool
 
+config SECRET_HIDING
+    bool "Secret hiding"
+    depends on HAS_SECRET_HIDING
+    help
+		The directmap contains mapping for most of the RAM which makes domain
+		memory easily accessible. While making the performance better, it also makes
+		the hypervisor more vulnerable to speculation attacks.
+
+		Enabling this feature will allow the user to decide whether the memory
+		is always mapped at boot or mapped only on demand (see the command line
+		option "directmap").
+
+		If unsure, say N.
+
 config MEM_ACCESS
 	def_bool MEM_ACCESS_ALWAYS_ON
 	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 7c1bdfc046..9b7e4721cd 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -174,6 +174,11 @@ paddr_t __ro_after_init mem_hotplug;
 static char __initdata opt_badpage[100] = "";
 string_param("badpage", opt_badpage);
 
+bool __ro_after_init opt_directmap = true;
+#ifdef CONFIG_HAS_SECRET_HIDING
+boolean_param("directmap", opt_directmap);
+#endif
+
 /*
  * no-bootscrub -> Free pages are not zeroed during boot.
  */
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 7561297a75..9d4f1f2d0d 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -167,6 +167,13 @@ extern unsigned long max_page;
 extern unsigned long total_pages;
 extern paddr_t mem_hotplug;
 
+extern bool opt_directmap;
+
+static inline bool has_directmap(void)
+{
+    return opt_directmap;
+}
+
 /*
  * Extra fault info types which are used to further describe
  * the source of an access violation.
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720931.1124032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvZ-0004v0-KD; Mon, 13 May 2024 13:41:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720931.1124032; Mon, 13 May 2024 13:41:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvZ-0004sm-CK; Mon, 13 May 2024 13:41:17 +0000
Received: by outflank-mailman (input) for mailman id 720931;
 Mon, 13 May 2024 13:41:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvX-0002zP-Ad
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:15 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7742508e-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:14 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:13 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.0.204:1425]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.61.78:2525]
 with esmtp (Farcaster)
 id 4bd6f625-72c4-4121-b554-dc3b0483863b; Mon, 13 May 2024 13:41:12 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:12 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:12 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7742508e-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607674; x=1747143674;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=NU5YSGcrFgwo4jtZeOfCQS23Pg2dIdFW+HjZiUxxomg=;
  b=Up9JRjjH4XIGA/9mEo5kGlM4VQG6+od02JFVAaOo+507e//FQOyDlFrV
   I2lm920L+VjSDzUP+Da7OmKUKE8bgFznPJq0L2pYFCX8pWHetN6h77BPp
   JSS1hGYOFt+u6oTgcoN9k2fYATEHVekj/TwJCyWlevH9DnL9w8t3gWRiV
   0=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="294692557"
X-Farcaster-Flow-ID: 4bd6f625-72c4-4121-b554-dc3b0483863b
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <jgrall@amazon.com>, Elias
 El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 (resend) 10/19] xen/page_alloc: Add a path for xenheap when there is no direct map
Date: Mon, 13 May 2024 13:40:37 +0000
Message-ID: <20240513134046.82605-11-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When there is not an always-mapped direct map, xenheap allocations need
to be mapped and unmapped on-demand.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    I have left the call to map_pages_to_xen() and destroy_xen_mappings()
    in the split heap for now. I am not entirely convinced this is necessary
    because in that setup only the xenheap would be always mapped and
    this doesn't contain any guest memory (aside the grant-table).
    So map/unmapping for every allocation seems unnecessary.

    Changes in v2:
        * Fix remaining wrong indentation in alloc_xenheap_pages()

    Changes since Hongyan's version:
        * Rebase
        * Fix indentation in alloc_xenheap_pages()
        * Fix build for arm32

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 9b7e4721cd..dfb2c05322 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2242,6 +2242,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe)
 void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 {
     struct page_info *pg;
+    void *ret;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2250,17 +2251,36 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
     if ( unlikely(pg == NULL) )
         return NULL;
 
+    ret = page_to_virt(pg);
+
+    if ( !has_directmap() &&
+         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
+                          PAGE_HYPERVISOR) )
+    {
+        /* Failed to map xenheap pages. */
+        free_heap_pages(pg, order, false);
+        return NULL;
+    }
+
     return page_to_virt(pg);
 }
 
 
 void free_xenheap_pages(void *v, unsigned int order)
 {
+    unsigned long va = (unsigned long)v & PAGE_MASK;
+
     ASSERT_ALLOC_CONTEXT();
 
     if ( v == NULL )
         return;
 
+    if ( !has_directmap() &&
+         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
+        dprintk(XENLOG_WARNING,
+                "Error while destroying xenheap mappings at %p, order %u\n",
+                v, order);
+
     free_heap_pages(virt_to_page(v), order, false);
 }
 
@@ -2284,6 +2304,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 {
     struct page_info *pg;
     unsigned int i;
+    void *ret;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2296,16 +2317,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
     if ( unlikely(pg == NULL) )
         return NULL;
 
+    ret = page_to_virt(pg);
+
+    if ( !has_directmap() &&
+         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
+                          PAGE_HYPERVISOR) )
+    {
+        /* Failed to map xenheap pages. */
+        free_domheap_pages(pg, order);
+        return NULL;
+    }
+
     for ( i = 0; i < (1u << order); i++ )
         pg[i].count_info |= PGC_xen_heap;
 
-    return page_to_virt(pg);
+    return ret;
 }
 
 void free_xenheap_pages(void *v, unsigned int order)
 {
     struct page_info *pg;
     unsigned int i;
+    unsigned long va = (unsigned long)v & PAGE_MASK;
 
     ASSERT_ALLOC_CONTEXT();
 
@@ -2317,6 +2350,12 @@ void free_xenheap_pages(void *v, unsigned int order)
     for ( i = 0; i < (1u << order); i++ )
         pg[i].count_info &= ~PGC_xen_heap;
 
+    if ( !has_directmap() &&
+         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
+        dprintk(XENLOG_WARNING,
+                "Error while destroying xenheap mappings at %p, order %u\n",
+                v, order);
+
     free_heap_pages(pg, order, true);
 }
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720932.1124040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vva-00053E-4T; Mon, 13 May 2024 13:41:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720932.1124040; Mon, 13 May 2024 13:41:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6VvZ-00052N-QS; Mon, 13 May 2024 13:41:17 +0000
Received: by outflank-mailman (input) for mailman id 720932;
 Mon, 13 May 2024 13:41:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6VvX-0002lY-PT
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:15 +0000
Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com
 [52.119.213.156]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7664c49f-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:13 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52005.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:12 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:6181]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 87cd883e-f817-4cc7-8910-c7f435c3254b; Mon, 13 May 2024 13:41:10 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:10 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:09 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7664c49f-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607674; x=1747143674;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=Xc4KQdAGUonE6oCgkT6a8PPvVb9mrXzRPtcULUVgVPo=;
  b=ojZJtXKegTqOz/eknVhUo+p/2MSvIWki91nVL3RJ7naDfjNHsEquwVXv
   0vJa1dVqNFmhxKkK9ydQJGWxFUvK0ISLNyNyOLozC6kVoueoi1w1JjlXZ
   GIf/NtYgE2Ha8GgNK4xPJfzcEbXxIMXo6VwN9v5DPxpy6/4/Yoba+qWD5
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="653770276"
X-Farcaster-Flow-ID: 87cd883e-f817-4cc7-8910-c7f435c3254b
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH V3 (resend) 09/19] x86/domain_page: Remove the fast paths when mfn is not in the directmap
Date: Mon, 13 May 2024 13:40:36 +0000
Message-ID: <20240513134046.82605-10-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When mfn is not in direct map, never use mfn_to_virt for any mappings.

We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with
arch_mfns_in_direct_map(mfn, 1) because these two are equivalent. The
extra comparison in arch_mfns_in_direct_map() looks different but because
DIRECTMAP_VIRT_END is always higher, it does not make any difference.

Lastly, domain_page_map_to_mfn() needs to gain to a special case for
the PMAP.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>

----

    Changes since Hongyan's version:
        * arch_mfn_in_direct_map() was renamed to arch_mfns_in_directmap()
        * add a special case for the PMAP in domain_page_map_to_mfn()

diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index 55e337aaf7..89caefc8a2 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -14,8 +14,10 @@
 #include <xen/sched.h>
 #include <xen/vmap.h>
 #include <asm/current.h>
+#include <asm/fixmap.h>
 #include <asm/flushtlb.h>
 #include <asm/hardirq.h>
+#include <asm/pmap.h>
 #include <asm/setup.h>
 
 static DEFINE_PER_CPU(struct vcpu *, override);
@@ -35,10 +37,11 @@ static inline struct vcpu *mapcache_current_vcpu(void)
     /*
      * When using efi runtime page tables, we have the equivalent of the idle
      * domain's page tables but current may point at another domain's VCPU.
-     * Return NULL as though current is not properly set up yet.
+     * Return the idle domains's vcpu on that core because the efi per-domain
+     * region (where the mapcache is) is in-sync with the idle domain.
      */
     if ( efi_rs_using_pgtables() )
-        return NULL;
+        return idle_vcpu[smp_processor_id()];
 
     /*
      * If guest_table is NULL, and we are running a paravirtualised guest,
@@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn)
     struct vcpu_maphash_entry *hashent;
 
 #ifdef NDEBUG
-    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
         return mfn_to_virt(mfn_x(mfn));
 #endif
 
     v = mapcache_current_vcpu();
-    if ( !v )
-        return mfn_to_virt(mfn_x(mfn));
+    if ( !v || !v->domain->arch.mapcache.inuse )
+    {
+        if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
+            return mfn_to_virt(mfn_x(mfn));
+        else
+        {
+            BUG_ON(system_state >= SYS_STATE_smp_boot);
+            return pmap_map(mfn);
+        }
+    }
 
     dcache = &v->domain->arch.mapcache;
     vcache = &v->arch.mapcache;
-    if ( !dcache->inuse )
-        return mfn_to_virt(mfn_x(mfn));
 
     perfc_incr(map_domain_page_count);
 
@@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr)
     if ( !va || va >= DIRECTMAP_VIRT_START )
         return;
 
+    if ( va >= FIXADDR_START && va < FIXADDR_TOP )
+    {
+        pmap_unmap((void *)ptr);
+        return;
+    }
+
     ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
 
     v = mapcache_current_vcpu();
@@ -237,7 +252,7 @@ int mapcache_domain_init(struct domain *d)
     unsigned int bitmap_pages;
 
 #ifdef NDEBUG
-    if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( !mem_hotplug && arch_mfn_in_directmap(0, max_page) )
         return 0;
 #endif
 
@@ -308,7 +323,7 @@ void *map_domain_page_global(mfn_t mfn)
             local_irq_is_enabled()));
 
 #ifdef NDEBUG
-    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
+    if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) )
         return mfn_to_virt(mfn_x(mfn));
 #endif
 
@@ -335,6 +350,23 @@ mfn_t domain_page_map_to_mfn(const void *ptr)
     if ( va >= DIRECTMAP_VIRT_START )
         return _mfn(virt_to_mfn(ptr));
 
+    /*
+     * The fixmap is stealing the top-end of the VMAP. So the check for
+     * the PMAP *must* happen first.
+     *
+     * Also, the fixmap translate a slot to an address backwards. The
+     * logic will rely on it to avoid any complexity. So check at
+     * compile time this will always hold.
+    */
+    BUILD_BUG_ON(fix_to_virt(FIX_PMAP_BEGIN) < fix_to_virt(FIX_PMAP_END));
+
+    if ( ((unsigned long)fix_to_virt(FIX_PMAP_END) <= va) &&
+         ((va & PAGE_MASK) <= (unsigned long)fix_to_virt(FIX_PMAP_BEGIN)) )
+    {
+        BUG_ON(system_state >= SYS_STATE_smp_boot);
+        return l1e_get_mfn(l1_fixmap[l1_table_offset(va)]);
+    }
+
     if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END )
         return vmap_to_mfn(va);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720937.1124062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvh-0006Su-MV; Mon, 13 May 2024 13:41:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720937.1124062; Mon, 13 May 2024 13:41:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvh-0006SU-H0; Mon, 13 May 2024 13:41:25 +0000
Received: by outflank-mailman (input) for mailman id 720937;
 Mon, 13 May 2024 13:41:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvg-0002lY-QO
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:24 +0000
Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c1a52f4-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:22 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2])
 by smtp-border-fw-6001.iad6.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:20 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:39746]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.60.166:2525]
 with esmtp (Farcaster)
 id 1e3f40c9-79ac-41ce-932f-75856514dfe8; Mon, 13 May 2024 13:41:19 +0000 (UTC)
Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:16 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:16 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c1a52f4-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607683; x=1747143683;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=YYAC4ge8TxViu8X0NtP8ExO8g2vNW+lgMuOpIFb06t0=;
  b=XuxowVYgY+unnSnSihIiT4zROppuesi1m7+7d4V+S1gT/53YJ9fJHW6e
   1KR5OdwTnNCAsdWpk9vpoMisMm8v0+vqtae1tU8EBwi+ErUPS5PD6hT20
   WW2tjMH7AuUYPQtr/2LcEzqwOxO6KeQ/WDNWVKxyk3X21gkCDX9B99I/u
   w=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="396086505"
X-Farcaster-Flow-ID: 1e3f40c9-79ac-41ce-932f-75856514dfe8
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <jgrall@amazon.com>, Elias
 El Yandouzi <eliasely@amazon.com>
Subject: [PATCH V3 (resend) 12/19] x86/setup: vmap heap nodes when they are outside the direct map
Date: Mon, 13 May 2024 13:40:39 +0000
Message-ID: <20240513134046.82605-13-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When we do not have a direct map, archs_mfn_in_direct_map() will always
return false, thus init_node_heap() will allocate xenheap pages from an
existing node for the metadata of a new node. This means that the
metadata of a new node is in a different node, slowing down heap
allocation.

Since we now have early vmap, vmap the metadata locally in the new node.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in v2:
        * vmap_contig_pages() was renamed to vmap_contig()
        * Fix indentation and coding style

    Changes from Hongyan's version:
        * arch_mfn_in_direct_map() was renamed to
          arch_mfns_in_direct_map()
        * Use vmap_contig_pages() rather than __vmap(...).
        * Add missing include (xen/vmap.h) so it compiles on Arm

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index dfb2c05322..3c0909f333 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -136,6 +136,7 @@
 #include <xen/sched.h>
 #include <xen/softirq.h>
 #include <xen/spinlock.h>
+#include <xen/vmap.h>
 
 #include <asm/flushtlb.h>
 #include <asm/page.h>
@@ -605,22 +606,44 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
         needed = 0;
     }
     else if ( *use_tail && nr >= needed &&
-              arch_mfns_in_directmap(mfn + nr - needed, needed) &&
               (!xenheap_bits ||
-               !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )
+              !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )
     {
-        _heap[node] = mfn_to_virt(mfn + nr - needed);
-        avail[node] = mfn_to_virt(mfn + nr - 1) +
-                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
+        {
+            _heap[node] = mfn_to_virt(mfn + nr - needed);
+            avail[node] = mfn_to_virt(mfn + nr - 1) +
+                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        }
+        else
+        {
+            mfn_t needed_start = _mfn(mfn + nr - needed);
+
+            _heap[node] = vmap_contig(needed_start, needed);
+            BUG_ON(!_heap[node]);
+            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
+                          sizeof(**avail) * NR_ZONES;
+        }
     }
     else if ( nr >= needed &&
-              arch_mfns_in_directmap(mfn, needed) &&
               (!xenheap_bits ||
-               !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
+              !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
     {
-        _heap[node] = mfn_to_virt(mfn);
-        avail[node] = mfn_to_virt(mfn + needed - 1) +
-                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        if ( arch_mfns_in_directmap(mfn, needed) )
+        {
+            _heap[node] = mfn_to_virt(mfn);
+            avail[node] = mfn_to_virt(mfn + needed - 1) +
+                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
+        }
+        else
+        {
+            mfn_t needed_start = _mfn(mfn);
+
+            _heap[node] = vmap_contig(needed_start, needed);
+            BUG_ON(!_heap[node]);
+            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
+                          sizeof(**avail) * NR_ZONES;
+        }
         *use_tail = false;
     }
     else if ( get_order_from_bytes(sizeof(**_heap)) ==
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720938.1124072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvk-0006uI-5G; Mon, 13 May 2024 13:41:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720938.1124072; Mon, 13 May 2024 13:41:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvj-0006tD-VZ; Mon, 13 May 2024 13:41:27 +0000
Received: by outflank-mailman (input) for mailman id 720938;
 Mon, 13 May 2024 13:41:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvi-0002zP-2X
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:26 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7d0f6584-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:25 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:21 +0000
Received: from EX19MTAUEC001.ant.amazon.com [10.0.29.78:8777]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.49.39:2525]
 with esmtp (Farcaster)
 id 1f592ada-6af3-40a6-8187-6ef26bab752b; Mon, 13 May 2024 13:41:20 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEC001.ant.amazon.com (10.252.135.222) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:18 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:18 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d0f6584-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607685; x=1747143685;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=roZUKjZFy3HRbWFvKqZPI52BDqwj8toyBUHm6ct47Gk=;
  b=P+C5/qpQjrPPVpYCsTGPxC548LmqXHTKl2UNl2J+70YEOwqD+LnQjUhQ
   dPiswTsJ/GCZFkJkpB61nNjzQsROO2LwbPx2zpmYqrq8+rVLpAAlmRG03
   CdyzjyBk0or0SFJ/QA/eYRzpm3XLJjcY5xE7/xHS1BlGXHxqgeSaYOrYW
   w=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88768912"
X-Farcaster-Flow-ID: 1f592ada-6af3-40a6-8187-6ef26bab752b
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 13/19] x86/setup: Do not create valid mappings when directmap=no
Date: Mon, 13 May 2024 13:40:40 +0000
Message-ID: <20240513134046.82605-14-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

Create empty mappings in the second e820 pass. Also, destroy existing
direct map mappings created in the first pass.

To make xenheap pages visible in guests, it is necessary to create empty
L3 tables in the direct map even when directmap=no, since guest cr3s
copy idle domain's L4 entries, which means they will share mappings in
the direct map if we pre-populate idle domain's L4 entries and L3
tables. A helper is introduced for this.

Also, after the direct map is actually gone, we need to stop updating
the direct map in update_xen_mappings().

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index f26c9799e4..919347d8c2 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -978,6 +978,57 @@ static struct domain *__init create_dom0(const module_t *image,
 /* How much of the directmap is prebuilt at compile time. */
 #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
 
+/*
+ * This either populates a valid direct map, or allocates empty L3 tables and
+ * creates the L4 entries for virtual address between [start, end) in the
+ * direct map depending on has_directmap();
+ *
+ * When directmap=no, we still need to populate empty L3 tables in the
+ * direct map region. The reason is that on-demand xenheap mappings are
+ * created in the idle domain's page table but must be seen by
+ * everyone. Since all domains share the direct map L4 entries, they
+ * will share xenheap mappings if we pre-populate the L4 entries and L3
+ * tables in the direct map region for all RAM. We also rely on the fact
+ * that L3 tables are never freed.
+ */
+static void __init populate_directmap(uint64_t pstart, uint64_t pend,
+                                      unsigned int flags)
+{
+    unsigned long vstart = (unsigned long)__va(pstart);
+    unsigned long vend = (unsigned long)__va(pend);
+
+    if ( pstart >= pend )
+        return;
+
+    BUG_ON(vstart < DIRECTMAP_VIRT_START);
+    BUG_ON(vend > DIRECTMAP_VIRT_END);
+
+    if ( has_directmap() )
+        /* Populate valid direct map. */
+        BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart),
+                                PFN_DOWN(pend - pstart), flags));
+    else
+    {
+        /* Create empty L3 tables. */
+        unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1);
+
+        for ( ; vaddr < vend; vaddr += (1UL << L4_PAGETABLE_SHIFT) )
+        {
+            l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)];
+
+            if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
+            {
+                mfn_t mfn = alloc_boot_pages(1, 1);
+                void *v = map_domain_page(mfn);
+
+                clear_page(v);
+                UNMAP_DOMAIN_PAGE(v);
+                l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR));
+            }
+        }
+    }
+}
+
 void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 {
     const char *memmap_type = NULL, *loader, *cmdline = "";
@@ -1601,8 +1652,17 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         map_e = min_t(uint64_t, e,
                       ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT);
 
-        /* Pass mapped memory to allocator /before/ creating new mappings. */
+        /*
+         * Pass mapped memory to allocator /before/ creating new mappings.
+         * The direct map for the bottom 4GiB has been populated in the first
+         * e820 pass. In the second pass, we make sure those existing mappings
+         * are destroyed when directmap=no.
+         */
         init_boot_pages(s, min(map_s, e));
+        if ( !has_directmap() )
+            destroy_xen_mappings((unsigned long)__va(s),
+                                 (unsigned long)__va(min(map_s, e)));
+
         s = map_s;
         if ( s < map_e )
         {
@@ -1610,6 +1670,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
             map_s = (s + mask) & ~mask;
             map_e &= ~mask;
             init_boot_pages(map_s, map_e);
+            if ( !has_directmap() )
+                destroy_xen_mappings((unsigned long)__va(map_s),
+                                     (unsigned long)__va(map_e));
         }
 
         if ( map_s > map_e )
@@ -1623,8 +1686,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 
             if ( map_e < end )
             {
-                map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e),
-                                 PFN_DOWN(end - map_e), PAGE_HYPERVISOR);
+                populate_directmap(map_e, end, PAGE_HYPERVISOR);
                 init_boot_pages(map_e, end);
                 map_e = end;
             }
@@ -1633,13 +1695,11 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         {
             /* This range must not be passed to the boot allocator and
              * must also not be mapped with _PAGE_GLOBAL. */
-            map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e),
-                             PFN_DOWN(e - map_e), __PAGE_HYPERVISOR_RW);
+            populate_directmap(map_e, e, __PAGE_HYPERVISOR_RW);
         }
         if ( s < map_s )
         {
-            map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s),
-                             PFN_DOWN(map_s - s), PAGE_HYPERVISOR);
+            populate_directmap(s, map_s, PAGE_HYPERVISOR);
             init_boot_pages(s, map_s);
         }
     }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720940.1124077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvk-00073F-Tj; Mon, 13 May 2024 13:41:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720940.1124077; Mon, 13 May 2024 13:41:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvk-00070B-Me; Mon, 13 May 2024 13:41:28 +0000
Received: by outflank-mailman (input) for mailman id 720940;
 Mon, 13 May 2024 13:41:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvj-0002lY-GI
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:27 +0000
Received: from smtp-fw-9105.amazon.com (smtp-fw-9105.amazon.com
 [207.171.188.204]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7d5be78f-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:25 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9105.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:18 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.0.204:27741]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.42.97:2525]
 with esmtp (Farcaster)
 id d9f5ac26-546a-43b3-a860-aee56dd191d0; Mon, 13 May 2024 13:41:17 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:14 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:14 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d5be78f-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607686; x=1747143686;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=FfhF1uGZGyTejdo4gJox/0kvgwcb9PzofNFjsbGWbZM=;
  b=XjaLRQENWz+r9dTS5ckM1rjgs5diZWpgHMsynhSbaH2jrlkfNfCow4dZ
   5pbTyujRWSphmBzgq9z91WjgnVHEYut9H8S3oeZDHQ9P9kMxPN6+Z1haf
   27tStvfQr/tRYAMoviu1gWQdfW9f278NUPB5ED5BccpSIDxzF0SMVS0LK
   k=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="726053710"
X-Farcaster-Flow-ID: d9f5ac26-546a-43b3-a860-aee56dd191d0
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Hongyan Xia
	<hongyxia@amazon.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 11/19] x86/setup: Leave early boot slightly earlier
Date: Mon, 13 May 2024 13:40:38 +0000
Message-ID: <20240513134046.82605-12-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Hongyan Xia <hongyxia@amazon.com>

When we do not have a direct map, memory for metadata of heap nodes in
init_node_heap() is allocated from xenheap, which needs to be mapped and
unmapped on demand. However, we cannot just take memory from the boot
allocator to create the PTEs while we are passing memory to the heap
allocator.

To solve this race, we leave early boot slightly sooner so that Xen PTE
pages are allocated from the heap instead of the boot allocator. We can
do this because the metadata for the 1st node is statically allocated,
and by the time we need memory to create mappings for the 2nd node, we
already have enough memory in the heap allocator in the 1st node.

Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index bd6b1184f5..f26c9799e4 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1751,6 +1751,22 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 
     numa_initmem_init(0, raw_max_page);
 
+    /*
+     * When we do not have a direct map, memory for metadata of heap nodes in
+     * init_node_heap() is allocated from xenheap, which needs to be mapped and
+     * unmapped on demand. However, we cannot just take memory from the boot
+     * allocator to create the PTEs while we are passing memory to the heap
+     * allocator during end_boot_allocator().
+     *
+     * To solve this race, we need to leave early boot before
+     * end_boot_allocator() so that Xen PTE pages are allocated from the heap
+     * instead of the boot allocator. We can do this because the metadata for
+     * the 1st node is statically allocated, and by the time we need memory to
+     * create mappings for the 2nd node, we already have enough memory in the
+     * heap allocator in the 1st node.
+     */
+    system_state = SYS_STATE_boot;
+
     if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) )
     {
         unsigned long lo = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
@@ -1782,8 +1798,6 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     else
         end_boot_allocator();
 
-    system_state = SYS_STATE_boot;
-
     bsp_stack = cpu_alloc_stack(0);
     if ( !bsp_stack )
         panic("No memory for BSP stack\n");
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:41:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:41:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720941.1124090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvm-0007Pt-DZ; Mon, 13 May 2024 13:41:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720941.1124090; Mon, 13 May 2024 13:41:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Vvm-0007PE-7s; Mon, 13 May 2024 13:41:30 +0000
Received: by outflank-mailman (input) for mailman id 720941;
 Mon, 13 May 2024 13:41:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvk-0002zP-32
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:28 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7e02d5f4-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:26 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:24 +0000
Received: from EX19MTAUEB001.ant.amazon.com [10.0.29.78:2195]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.49.39:2525]
 with esmtp (Farcaster)
 id f9f42ec9-ac81-45a7-bcb8-9513c302c028; Mon, 13 May 2024 13:41:23 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:23 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:22 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e02d5f4-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607687; x=1747143687;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=7GEtDZOeCCFB+yvu/Susicjynil2ILWZXCBC3LXl2pY=;
  b=NuoBBCNpv5cdCrPXRjVs5v6qWbL+EDIXmxMgO5NwGpUkh3lOUj0saO3h
   e4fbJnz8FX2pvKdnih2kyrWI+caIBULAKEE/boIZe0iQ15RH5MTUkHaYh
   V6v5jEFTDcOsy6QrMJc2LnZEOA6fF1+7Lqk5RUq+e/jb2wDiif9InEGHM
   A=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="725436999"
X-Farcaster-Flow-ID: f9f42ec9-ac81-45a7-bcb8-9513c302c028
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>, Jan Beulich <jbeulich@suse.com>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>
Subject: [PATCH V3 (resend) 15/19] Rename maddr_to_virt() calls
Date: Mon, 13 May 2024 13:40:42 +0000
Message-ID: <20240513134046.82605-16-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Until directmap gets completely removed, we'd still need to
keep some calls to mmaddr_to_virt() for xenheap pages or when the
directmap is enabled.

Rename the macro to maddr_to_directmap_virt() to flag them and
prevent further use of maddr_to_virt().

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
index 81f80c053a..ac016f3a04 100644
--- a/xen/arch/x86/dmi_scan.c
+++ b/xen/arch/x86/dmi_scan.c
@@ -277,7 +277,7 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len)
 			return "SMBIOS";
 		}
 	} else {
-		char __iomem *p = maddr_to_virt(0xF0000), *q;
+		char __iomem *p = maddr_to_directmap_virt(0xF0000), *q;
 		union {
 			struct dmi_eps dmi;
 			struct smbios3_eps smbios3;
@@ -364,7 +364,7 @@ static int __init dmi_iterate(void (*decode)(const struct dmi_header *))
 	dmi.size = 0;
 	smbios3.length = 0;
 
-	p = maddr_to_virt(0xF0000);
+	p = maddr_to_directmap_virt(0xF0000);
 	for (q = p; q < p + 0x10000; q += 16) {
 		if (!dmi.size) {
 			memcpy_fromio(&dmi, q, sizeof(dmi));
diff --git a/xen/arch/x86/include/asm/mach-default/bios_ebda.h b/xen/arch/x86/include/asm/mach-default/bios_ebda.h
index 42de6b2a5b..8cfe53d1f2 100644
--- a/xen/arch/x86/include/asm/mach-default/bios_ebda.h
+++ b/xen/arch/x86/include/asm/mach-default/bios_ebda.h
@@ -7,7 +7,7 @@
  */
 static inline unsigned int get_bios_ebda(void)
 {
-	unsigned int address = *(unsigned short *)maddr_to_virt(0x40E);
+	unsigned int address = *(unsigned short *)maddr_to_directmap_virt(0x40E);
 	address <<= 4;
 	return address;	/* 0 means none */
 }
diff --git a/xen/arch/x86/include/asm/page.h b/xen/arch/x86/include/asm/page.h
index c6891b52d4..bf7bf08ba4 100644
--- a/xen/arch/x86/include/asm/page.h
+++ b/xen/arch/x86/include/asm/page.h
@@ -240,11 +240,11 @@ void copy_page_sse2(void *to, const void *from);
 
 /* Convert between Xen-heap virtual addresses and machine addresses. */
 #define __pa(x)             (virt_to_maddr(x))
-#define __va(x)             (maddr_to_virt(x))
+#define __va(x)             (maddr_to_directmap_virt(x))
 
 /* Convert between Xen-heap virtual addresses and machine frame numbers. */
 #define __virt_to_mfn(va)   (virt_to_maddr(va) >> PAGE_SHIFT)
-#define __mfn_to_virt(mfn)  (maddr_to_virt((paddr_t)(mfn) << PAGE_SHIFT))
+#define __mfn_to_virt(mfn)  (maddr_to_directmap_virt((paddr_t)(mfn) << PAGE_SHIFT))
 
 /* Convert between machine frame numbers and page-info structures. */
 #define mfn_to_page(mfn)    (frame_table + mfn_to_pdx(mfn))
@@ -270,7 +270,7 @@ void copy_page_sse2(void *to, const void *from);
 #define virt_to_mfn(va)     __virt_to_mfn(va)
 #define mfn_to_directmap_virt(mfn)    __mfn_to_virt(mfn)
 #define virt_to_maddr(va)   __virt_to_maddr((unsigned long)(va))
-#define maddr_to_virt(ma)   __maddr_to_virt((unsigned long)(ma))
+#define maddr_to_directmap_virt(ma)   __maddr_to_directmap_virt((unsigned long)(ma))
 #define maddr_to_page(ma)   __maddr_to_page(ma)
 #define page_to_maddr(pg)   __page_to_maddr(pg)
 #define virt_to_page(va)    __virt_to_page(va)
diff --git a/xen/arch/x86/include/asm/x86_64/page.h b/xen/arch/x86/include/asm/x86_64/page.h
index 19ca64d792..a95ebc088f 100644
--- a/xen/arch/x86/include/asm/x86_64/page.h
+++ b/xen/arch/x86/include/asm/x86_64/page.h
@@ -48,7 +48,7 @@ static inline unsigned long __virt_to_maddr(unsigned long va)
     return xen_phys_start + va - XEN_VIRT_START;
 }
 
-static inline void *__maddr_to_virt(unsigned long ma)
+static inline void *__maddr_to_directmap_virt(unsigned long ma)
 {
     /* Offset in the direct map, accounting for pdx compression */
     unsigned long va_offset = maddr_to_directmapoff(ma);
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449..69181b0abe 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -664,7 +664,7 @@ void __init get_smp_config (void)
 
 static int __init smp_scan_config (unsigned long base, unsigned long length)
 {
-	unsigned int *bp = maddr_to_virt(base);
+	unsigned int *bp = maddr_to_directmap_virt(base);
 	struct intel_mp_floating *mpf;
 
 	Dprintk("Scan SMP from %p for %ld bytes.\n", bp,length);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 39aed5845d..1b02e2b6d5 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1764,7 +1764,7 @@ void __init efi_init_memory(void)
                 if ( map_pages_to_xen((unsigned long)mfn_to_directmap_virt(smfn),
                                     _mfn(smfn), emfn - smfn, prot) == 0 )
                     desc->VirtualStart =
-                        (unsigned long)maddr_to_virt(desc->PhysicalStart);
+                        (unsigned long)maddr_to_directmap_virt(desc->PhysicalStart);
                 else
                     printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
                         smfn, emfn - 1);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:52:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:52:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720959.1124102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W5u-0005Nb-ER; Mon, 13 May 2024 13:51:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720959.1124102; Mon, 13 May 2024 13:51:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W5u-0005NU-Ac; Mon, 13 May 2024 13:51:58 +0000
Received: by outflank-mailman (input) for mailman id 720959;
 Mon, 13 May 2024 13:51:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvx-0002lY-Tx
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:41 +0000
Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com
 [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 85cf85a8-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:39 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80008.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:37 +0000
Received: from EX19MTAUEC002.ant.amazon.com [10.0.44.209:36217]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.223:2525]
 with esmtp (Farcaster)
 id df3964f2-8ebf-477e-94f3-b615efdae6ad; Mon, 13 May 2024 13:41:36 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:31 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:31 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85cf85a8-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607699; x=1747143699;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=PLKm6zqnTr1kh3lkUidFqFNcSg3ycGZChl4qwlqzWSo=;
  b=KJWagw5D9fdR6ecsO306SqBxIvSajVAkKDhbXsR9ECw27XYHzF6Bxvbk
   lxrzOMMGzQeyXqf5CIX1H/xLN45a8BQX2yG2bLcYb5cQBef12AIfMmWaP
   nxddpaGDWM087DsHSHpWaf8hiFrThXiNPCthEsdsmLSmB0rVTXqxCLCyo
   E=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="88768953"
X-Farcaster-Flow-ID: df3964f2-8ebf-477e-94f3-b615efdae6ad
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 19/19] xen/arm64: Allow the admin to enable/disable the directmap
Date: Mon, 13 May 2024 13:40:46 +0000
Message-ID: <20240513134046.82605-20-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

Implement the same command line option as x86 to enable/disable the
directmap. By default this is kept enabled.

Also modify setup_directmap_mappings() to populate the L0 entries
related to the directmap area.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changes in v2:
        * Rely on the Kconfig option to enable Secret Hiding on Arm64
        * Use generic helper instead of arch_has_directmap()

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 743d343ffa..cccd5e4282 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -799,7 +799,7 @@ that enabling this option cannot guarantee anything beyond what underlying
 hardware guarantees (with, where available and known to Xen, respective
 tweaks applied).
 
-### directmap (x86)
+### directmap (arm64, x86)
 > `= <boolean>`
 
 > Default: `true`
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 0462960fc7..1cb495e334 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -7,6 +7,7 @@ config ARM_64
 	depends on !ARM_32
 	select 64BIT
 	select HAS_FAST_MULTIPLY
+	select HAS_SECRET_HIDING
 
 config ARM
 	def_bool y
diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 826864d25d..81115cce51 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -158,16 +158,27 @@ void __init switch_ttbr(uint64_t ttbr)
     update_identity_mapping(false);
 }
 
-/* Map the region in the directmap area. */
+/*
+ * This either populate a valid fdirect map, or allocates empty L1 tables
+ * and creates the L0 entries for the given region in the direct map
+ * depending on has_directmap().
+ *
+ * When directmap=no, we still need to populate empty L1 tables in the
+ * directmap region. The reason is that the root page-table (i.e. L0)
+ * is per-CPU and secondary CPUs will initialize their root page-table
+ * based on the pCPU0 one. So L0 entries will be shared if they are
+ * pre-populated. We also rely on the fact that L1 tables are never
+ * freed.
+ */
 static void __init setup_directmap_mappings(unsigned long base_mfn,
                                             unsigned long nr_mfns)
 {
+    unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1);
     int rc;
 
     /* First call sets the directmap physical and virtual offset. */
     if ( mfn_eq(directmap_mfn_start, INVALID_MFN) )
     {
-        unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1);
 
         directmap_mfn_start = _mfn(base_mfn);
         directmap_base_pdx = mfn_to_pdx(_mfn(base_mfn));
@@ -188,6 +199,24 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,
         panic("cannot add directmap mapping at %lx below heap start %lx\n",
               base_mfn, mfn_x(directmap_mfn_start));
 
+    if ( !has_directmap() )
+    {
+        vaddr_t vaddr = (vaddr_t)__mfn_to_virt(base_mfn);
+        lpae_t *root = this_cpu(xen_pgtable);
+        unsigned int i, slot;
+
+        slot = first_table_offset(vaddr);
+        nr_mfns += base_mfn - mfn_gb;
+        for ( i = 0; i < nr_mfns; i += BIT(XEN_PT_LEVEL_ORDER(0), UL), slot++ )
+        {
+            lpae_t *entry = &root[slot];
+
+            if ( !lpae_is_valid(*entry) && !create_xen_table(entry) )
+                panic("Unable to populate zeroeth slot %u\n", slot);
+        }
+        return;
+    }
+
     rc = map_pages_to_xen((vaddr_t)__mfn_to_virt(base_mfn),
                           _mfn(base_mfn), nr_mfns,
                           PAGE_HYPERVISOR_RW | _PAGE_BLOCK);
diff --git a/xen/arch/arm/include/asm/arm64/mm.h b/xen/arch/arm/include/asm/arm64/mm.h
index e0bd23a6ed..5888f29159 100644
--- a/xen/arch/arm/include/asm/arm64/mm.h
+++ b/xen/arch/arm/include/asm/arm64/mm.h
@@ -3,13 +3,10 @@
 
 extern DEFINE_PAGE_TABLE(xen_pgtable);
 
-/*
- * On ARM64, all the RAM is currently direct mapped in Xen.
- * Hence return always true.
- */
+/* On Arm64, the user can chose whether all the RAM is directmap. */
 static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
 {
-    return true;
+    return has_directmap();
 }
 
 void arch_setup_page_tables(void);
diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index def939172c..0f3ffab6ba 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -12,6 +12,7 @@
 #include <xen/grant_table.h>
 #include <xen/guest_access.h>
 #include <xen/mm.h>
+#include <xen/param.h>
 
 #include <xsm/xsm.h>
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d15987d6ea..6b06e2f4f5 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -778,6 +778,7 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
     cmdline_parse(cmdline);
 
     setup_mm();
+    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
 
     vm_init();
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:56:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:56:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720973.1124112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W9j-000740-T6; Mon, 13 May 2024 13:55:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720973.1124112; Mon, 13 May 2024 13:55:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W9j-00073t-QS; Mon, 13 May 2024 13:55:55 +0000
Received: by outflank-mailman (input) for mailman id 720973;
 Mon, 13 May 2024 13:55:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvo-0002zP-0B
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:32 +0000
Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com
 [52.119.213.156]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8106d847-112e-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 15:41:31 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-52005.iad7.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:31 +0000
Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:57545]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.42.97:2525]
 with esmtp (Farcaster)
 id f3d4a567-c04b-4550-b435-7b7102e74289; Mon, 13 May 2024 13:41:29 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:29 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:28 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8106d847-112e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607692; x=1747143692;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=Xrly80bM9b4Iylo4MumX8GuIMjwgPtJngKOUTgszWKw=;
  b=f2BK85Jm6jxXlGe0NpuTjacsYbNJNVZm0eookTlypkNlegOXJ/Qppuf2
   FHlXKdn0p5zpf9SU60Z2mOmwDBCkA6MVwMqNv3bTLfBI0EWVS6qZiucwi
   YisnG/XRWj+JQcwtEBEIS2Qz6/37PweB3UanTAIC8ayo4p+3hKcpVPCxq
   M=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="653770317"
X-Farcaster-Flow-ID: f3d4a567-c04b-4550-b435-7b7102e74289
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Stefano Stabellini <sstabellini@kernel.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 18/19] xen/arm64: Implement a mapcache for arm64
Date: Mon, 13 May 2024 13:40:45 +0000
Message-ID: <20240513134046.82605-19-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

At the moment, on arm64, map_domain_page() is implemented using
virt_to_mfn(). Therefore it is relying on the directmap.

In a follow-up patch, we will allow the admin to remove the directmap.
Therefore we want to implement a mapcache.

Thanksfully there is already one for arm32. So select ARCH_ARM_DOMAIN_PAGE
and add the necessary boiler plate to support 64-bit:
    - The page-table start at level 0, so we need to allocate the level
      1 page-table
    - map_domain_page() should check if the page is in the directmap. If
      yes, then use virt_to_mfn() to limit the performance impact
      when the directmap is still enabled (this will be selectable
      on the command line).

Take the opportunity to replace first_table_offset(...) with offsets[...].

Note that, so far, arch_mfns_in_directmap() always return true on
arm64. So the mapcache is not yet used. This will change in a
follow-up patch.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    There are a few TODOs:
        - It is becoming more critical to fix the mapcache
          implementation (this is not compliant with the Arm Arm)
        - Evaluate the performance

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 21d03d9f44..0462960fc7 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -1,7 +1,6 @@
 config ARM_32
 	def_bool y
 	depends on "$(ARCH)" = "arm32"
-	select ARCH_MAP_DOMAIN_PAGE
 
 config ARM_64
 	def_bool y
diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 2ec1ffe1dc..826864d25d 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -4,6 +4,7 @@
 #include <xen/mm.h>
 #include <xen/pfn.h>
 
+#include <asm/domain_page.h>
 #include <asm/setup.h>
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
@@ -237,6 +238,14 @@ void __init setup_mm(void)
     setup_frametable_mappings(ram_start, ram_end);
     max_page = PFN_DOWN(ram_end);
 
+    /*
+     * The allocators may need to use map_domain_page() (such as for
+     * scrubbing pages). So we need to prepare the domheap area first.
+     */
+    if ( !init_domheap_mappings(smp_processor_id()) )
+        panic("CPU%u: Unable to prepare the domheap page-tables\n",
+              smp_processor_id());
+
     init_staticmem_pages();
     init_sharedmem_pages();
 }
diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c
index ac2a6d0332..0f6ba48892 100644
--- a/xen/arch/arm/domain_page.c
+++ b/xen/arch/arm/domain_page.c
@@ -1,4 +1,5 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/domain_page.h>
 #include <xen/mm.h>
 #include <xen/pmap.h>
 #include <xen/vmap.h>
@@ -8,6 +9,8 @@
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
+#undef mfn_to_virt
+#define mfn_to_virt(va) __mfn_to_virt(mfn_x(mfn))
 
 /* cpu0's domheap page tables */
 static DEFINE_PAGE_TABLES(cpu0_dommap, DOMHEAP_SECOND_PAGES);
@@ -31,13 +34,30 @@ bool init_domheap_mappings(unsigned int cpu)
 {
     unsigned int order = get_order_from_pages(DOMHEAP_SECOND_PAGES);
     lpae_t *root = per_cpu(xen_pgtable, cpu);
+    lpae_t *first;
     unsigned int i, first_idx;
     lpae_t *domheap;
     mfn_t mfn;
 
+    /* Convenience aliases */
+    DECLARE_OFFSETS(offsets, DOMHEAP_VIRT_START);
+
     ASSERT(root);
     ASSERT(!per_cpu(xen_dommap, cpu));
 
+    /*
+     * On Arm64, the root is at level 0. Therefore we need an extra step
+     * to allocate the first level page-table.
+     */
+#ifdef CONFIG_ARM_64
+    if ( create_xen_table(&root[offsets[0]]) )
+        return false;
+
+    first = xen_map_table(lpae_get_mfn(root[offsets[0]]));
+#else
+    first = root;
+#endif
+
     /*
      * The domheap for cpu0 is initialized before the heap is initialized.
      * So we need to use pre-allocated pages.
@@ -58,16 +78,20 @@ bool init_domheap_mappings(unsigned int cpu)
      * domheap mapping pages.
      */
     mfn = virt_to_mfn(domheap);
-    first_idx = first_table_offset(DOMHEAP_VIRT_START);
+    first_idx = offsets[1];
     for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ )
     {
         lpae_t pte = mfn_to_xen_entry(mfn_add(mfn, i), MT_NORMAL);
         pte.pt.table = 1;
-        write_pte(&root[first_idx + i], pte);
+        write_pte(&first[first_idx + i], pte);
     }
 
     per_cpu(xen_dommap, cpu) = domheap;
 
+#ifdef CONFIG_ARM_64
+    xen_unmap_table(first);
+#endif
+
     return true;
 }
 
@@ -91,6 +115,10 @@ void *map_domain_page(mfn_t mfn)
     lpae_t pte;
     int i, slot;
 
+    /* Bypass the mapcache if the page is in the directmap */
+    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
+        return mfn_to_virt(mfn);
+
     local_irq_save(flags);
 
     /* The map is laid out as an open-addressed hash table where each
@@ -153,13 +181,25 @@ void *map_domain_page(mfn_t mfn)
 /* Release a mapping taken with map_domain_page() */
 void unmap_domain_page(const void *ptr)
 {
+    unsigned long va = (unsigned long)ptr;
     unsigned long flags;
     lpae_t *map = this_cpu(xen_dommap);
-    int slot = ((unsigned long)ptr - DOMHEAP_VIRT_START) >> SECOND_SHIFT;
+    unsigned int slot;
+
+    /* Below we assume that the domheap area doesn't start at 0 */
+    BUILD_BUG_ON(DOMHEAP_VIRT_START == 0);
 
-    if ( !ptr )
+    /*
+     * map_domain_page() may not have mapped anything if the address
+     * is part of the directmap. So ignore anything outside of the
+     * domheap.
+     */
+    if ( (va < DOMHEAP_VIRT_START) ||
+         ((va - DOMHEAP_VIRT_START) >= DOMHEAP_VIRT_SIZE) )
         return;
 
+    slot = (va - DOMHEAP_VIRT_START) >> SECOND_SHIFT;
+
     local_irq_save(flags);
 
     ASSERT(slot >= 0 && slot < DOMHEAP_ENTRIES);
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 60e0122cba..610dfa0466 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -432,6 +432,11 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
     } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
 }
 
+/* Helpers to allocate, map and unmap a Xen page-table */
+int create_xen_table(lpae_t *entry);
+lpae_t *xen_map_table(mfn_t mfn);
+void xen_unmap_table(const lpae_t *table);
+
 #endif /*  __ARCH_ARM_MM__ */
 /*
  * Local variables:
diff --git a/xen/arch/arm/include/asm/mmu/layout.h b/xen/arch/arm/include/asm/mmu/layout.h
index a3b546465b..c549420e8b 100644
--- a/xen/arch/arm/include/asm/mmu/layout.h
+++ b/xen/arch/arm/include/asm/mmu/layout.h
@@ -35,9 +35,13 @@
  *
  *  32G -  64G   Frametable: 56 bytes per page for 2TB of RAM
  *
- * 0x00000a8000000000 - 0x00007fffffffffff (512GB+117TB, L0 slots [21..255])
+ * 0x00000a8000000000 - 0x00007f7fffffffff (117TB, L0 slots [21..254])
  *  Unused
  *
+ * 0x00007f8000000000 - 0x00007fffffffffff (512GB, L0 slot [255])
+ *  (Relative offsets)
+ *  0  -    2G    Domheap: on-demand-mapped
+ *
  * 0x0000800000000000 - 0x000084ffffffffff (5TB, L0 slots [256..265])
  *  1:1 mapping of RAM
  *
@@ -130,6 +134,13 @@
 #define FRAMETABLE_SIZE        GB(32)
 #define FRAMETABLE_NR          (FRAMETABLE_SIZE / sizeof(*frame_table))
 
+#define DOMHEAP_VIRT_START     SLOT0(255)
+#define DOMHEAP_VIRT_SIZE      GB(2)
+
+#define DOMHEAP_ENTRIES        1024 /* 1024 2MB mapping slots */
+/* Number of domheap pagetable pages required at the second level (2MB mappings) */
+#define DOMHEAP_SECOND_PAGES (DOMHEAP_VIRT_SIZE >> FIRST_SHIFT)
+
 #define DIRECTMAP_VIRT_START   SLOT0(256)
 #define DIRECTMAP_SIZE         (SLOT0_ENTRY_SIZE * (266 - 256))
 #define DIRECTMAP_VIRT_END     (DIRECTMAP_VIRT_START + DIRECTMAP_SIZE - 1)
diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c
index 1ed1a53ab1..da33c6c52e 100644
--- a/xen/arch/arm/mmu/pt.c
+++ b/xen/arch/arm/mmu/pt.c
@@ -33,7 +33,7 @@ mm_printk(const char *fmt, ...) {}
 #define HYP_PT_ROOT_LEVEL 1
 #endif
 
-static lpae_t *xen_map_table(mfn_t mfn)
+lpae_t *xen_map_table(mfn_t mfn)
 {
     /*
      * During early boot, map_domain_page() may be unusable. Use the
@@ -45,7 +45,7 @@ static lpae_t *xen_map_table(mfn_t mfn)
     return map_domain_page(mfn);
 }
 
-static void xen_unmap_table(const lpae_t *table)
+void xen_unmap_table(const lpae_t *table)
 {
     /*
      * During early boot, xen_map_table() will not use map_domain_page()
@@ -228,7 +228,7 @@ void *ioremap(paddr_t pa, size_t len)
     return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE);
 }
 
-static int create_xen_table(lpae_t *entry)
+int create_xen_table(lpae_t *entry)
 {
     mfn_t mfn;
     void *p;
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:56:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:56:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720975.1124122 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W9r-0007LJ-6e; Mon, 13 May 2024 13:56:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720975.1124122; Mon, 13 May 2024 13:56:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6W9r-0007LC-3r; Mon, 13 May 2024 13:56:03 +0000
Received: by outflank-mailman (input) for mailman id 720975;
 Mon, 13 May 2024 13:56:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvt-0002lY-PI
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:37 +0000
Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com
 [207.171.190.10]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82b4e1b4-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:35 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-33001.sea14.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:26 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.0.204:36883]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.49.39:2525]
 with esmtp (Farcaster)
 id 1e2eaeee-dc0e-4963-bdc9-39279669553c; Mon, 13 May 2024 13:41:25 +0000 (UTC)
Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:25 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:24 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82b4e1b4-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607695; x=1747143695;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=BMLad5szd5dSnCh0YLtH/2Rm7w1Vj1S/B6o9FzZUpnM=;
  b=jxR/A4sxn65hNOYjvhf8d8lhNKCPLi4fp78Uj9QQ+kLgfp63Bls1ND6x
   BLsgoiXGEPeh0zRZsB1s7ZDNMtZsEnUB7nzPnXYXt8588xFpjQ7g51bs0
   hJ6Hlfzxh3nGLwlogunsDbq6+Vl1yqJtbOrQ3NUqR6gOin9X7ny5hn67C
   4=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="344160809"
X-Farcaster-Flow-ID: 1e2eaeee-dc0e-4963-bdc9-39279669553c
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Stefano Stabellini <sstabellini@kernel.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 16/19] xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
Date: Mon, 13 May 2024 13:40:43 +0000
Message-ID: <20240513134046.82605-17-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

The arm32 version of init_secondary_pagetables() will soon be re-used
for arm64 as well where the root table starts at level 0 rather than level 1.

So rename 'first' to 'root'.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changelog in v2:
        * Rebase
        * Fix typo

diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c
index 4ffc8254a4..e29b6f34f2 100644
--- a/xen/arch/arm/mmu/smpboot.c
+++ b/xen/arch/arm/mmu/smpboot.c
@@ -109,32 +109,32 @@ int prepare_secondary_mm(int cpu)
 #else
 int prepare_secondary_mm(int cpu)
 {
-    lpae_t *first;
+    lpae_t *root = alloc_xenheap_page();
 
     first = alloc_xenheap_page(); /* root == first level on 32-bit 3-level trie */
 
-    if ( !first )
+    if ( !root )
     {
-        printk("CPU%u: Unable to allocate the first page-table\n", cpu);
+        printk("CPU%u: Unable to allocate the root page-table\n", cpu);
         return -ENOMEM;
     }
 
     /* Initialise root pagetable from root of boot tables */
-    memcpy(first, per_cpu(xen_pgtable, 0), PAGE_SIZE);
-    per_cpu(xen_pgtable, cpu) = first;
+    memcpy(root, per_cpu(xen_pgtable, 0), PAGE_SIZE);
+    per_cpu(xen_pgtable, cpu) = root;
 
     if ( !init_domheap_mappings(cpu) )
     {
         printk("CPU%u: Unable to prepare the domheap page-tables\n", cpu);
         per_cpu(xen_pgtable, cpu) = NULL;
-        free_xenheap_page(first);
+        free_xenheap_page(root);
         return -ENOMEM;
     }
 
     clear_boot_pagetables();
 
     /* Set init_ttbr for this CPU coming up */
-    set_init_ttbr(first);
+    set_init_ttbr(root);
 
     return 0;
 }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 13:58:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 13:58:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.720987.1124132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6WBz-00004y-J4; Mon, 13 May 2024 13:58:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 720987.1124132; Mon, 13 May 2024 13:58:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6WBz-00004r-FW; Mon, 13 May 2024 13:58:15 +0000
Received: by outflank-mailman (input) for mailman id 720987;
 Mon, 13 May 2024 13:58:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvm-0002lY-3b
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:30 +0000
Received: from smtp-fw-9105.amazon.com (smtp-fw-9105.amazon.com
 [207.171.188.204]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f610767-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:27 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9105.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:23 +0000
Received: from EX19MTAUEB002.ant.amazon.com [10.0.0.204:13448]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with
 esmtp (Farcaster)
 id 4812b6f1-35cd-42d7-8ea7-3c70ed6b4a4d; Mon, 13 May 2024 13:41:22 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:21 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:21 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f610767-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607688; x=1747143688;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=mKCZV9DzUavScUGAYIjbkqz34voMdE7B7ItlMBncNvQ=;
  b=j+YjueYimKRBlt56RCHaHULTyp4SJTKdUkwrWU8gRKfi0k395dFLNMVt
   hrmvG7o3RjrPGMSSzJUpDqfu91z03AMx0HLBlhCWnRi9HBVWkZ2qz/LM7
   6ss8pTj/Fw0BYBaCt0CpVIZF5HfwEMuMIy0P4gEDVv5z8tX/TphAOuACM
   c=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="726053718"
X-Farcaster-Flow-ID: 4812b6f1-35cd-42d7-8ea7-3c70ed6b4a4d
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, "Elias El
 Yandouzi" <eliasely@amazon.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Lukasz Hawrylko
	<lukasz@hawrylko.pl>, "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	=?UTF-8?q?Mateusz=20M=C3=B3wka?= <mateusz.mowka@intel.com>
Subject: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Date: Mon, 13 May 2024 13:40:41 +0000
Message-ID: <20240513134046.82605-15-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Until directmap gets completely removed, we'd still need to
keep some calls to mfn_to_virt() for xenheap pages or when the
directmap is enabled.

Rename the macro to mfn_to_directmap_virt() to flag them and
prevent further use of mfn_to_virt().

Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 48538b5337..2bca3f9e87 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -336,6 +336,7 @@ static inline uint64_t gvirt_to_maddr(vaddr_t va, paddr_t *pa,
  */
 #define virt_to_mfn(va)     __virt_to_mfn(va)
 #define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+#define mfn_to_directmap_virt(mfn) mfn_to_virt(mfn)
 
 /* Convert between Xen-heap virtual addresses and page-info structures. */
 static inline struct page_info *virt_to_page(const void *v)
diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index 89caefc8a2..62d6fee0f4 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -81,14 +81,14 @@ void *map_domain_page(mfn_t mfn)
 
 #ifdef NDEBUG
     if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
-        return mfn_to_virt(mfn_x(mfn));
+        return mfn_to_directmap_virt(mfn_x(mfn));
 #endif
 
     v = mapcache_current_vcpu();
     if ( !v || !v->domain->arch.mapcache.inuse )
     {
         if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
-            return mfn_to_virt(mfn_x(mfn));
+            return mfn_to_directmap_virt(mfn_x(mfn));
         else
         {
             BUG_ON(system_state >= SYS_STATE_smp_boot);
@@ -324,7 +324,7 @@ void *map_domain_page_global(mfn_t mfn)
 
 #ifdef NDEBUG
     if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) )
-        return mfn_to_virt(mfn_x(mfn));
+        return mfn_to_directmap_virt(mfn_x(mfn));
 #endif
 
     return vmap(&mfn, 1);
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index b0cb96c3bc..d1482ae2f7 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -439,7 +439,7 @@ static int __init pvh_populate_p2m(struct domain *d)
                                  d->arch.e820[i].addr + d->arch.e820[i].size);
             enum hvm_translation_result res =
                  hvm_copy_to_guest_phys(mfn_to_maddr(_mfn(addr)),
-                                        mfn_to_virt(addr),
+                                        mfn_to_directmap_virt(addr),
                                         end - d->arch.e820[i].addr,
                                         v);
 
@@ -725,7 +725,7 @@ static int __init pvh_load_kernel(struct domain *d, const module_t *image,
 
     if ( initrd != NULL )
     {
-        rc = hvm_copy_to_guest_phys(last_addr, mfn_to_virt(initrd->mod_start),
+        rc = hvm_copy_to_guest_phys(last_addr, mfn_to_directmap_virt(initrd->mod_start),
                                     initrd_len, v);
         if ( rc )
         {
diff --git a/xen/arch/x86/include/asm/page.h b/xen/arch/x86/include/asm/page.h
index 350d1fb110..c6891b52d4 100644
--- a/xen/arch/x86/include/asm/page.h
+++ b/xen/arch/x86/include/asm/page.h
@@ -268,7 +268,7 @@ void copy_page_sse2(void *to, const void *from);
  */
 #define mfn_valid(mfn)      __mfn_valid(mfn_x(mfn))
 #define virt_to_mfn(va)     __virt_to_mfn(va)
-#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+#define mfn_to_directmap_virt(mfn)    __mfn_to_virt(mfn)
 #define virt_to_maddr(va)   __virt_to_maddr((unsigned long)(va))
 #define maddr_to_virt(ma)   __maddr_to_virt((unsigned long)(ma))
 #define maddr_to_page(ma)   __maddr_to_page(ma)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index efdf20f775..337363cf17 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -318,8 +318,8 @@ void __init arch_init_memory(void)
         iostart_pfn = max_t(unsigned long, pfn, 1UL << (20 - PAGE_SHIFT));
         ioend_pfn = min(rstart_pfn, 16UL << (20 - PAGE_SHIFT));
         if ( iostart_pfn < ioend_pfn )
-            destroy_xen_mappings((unsigned long)mfn_to_virt(iostart_pfn),
-                                 (unsigned long)mfn_to_virt(ioend_pfn));
+            destroy_xen_mappings((unsigned long)mfn_to_directmap_virt(iostart_pfn),
+                                 (unsigned long)mfn_to_directmap_virt(ioend_pfn));
 
         /* Mark as I/O up to next RAM region. */
         for ( ; pfn < rstart_pfn; pfn++ )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 919347d8c2..e0671ab3c3 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -399,7 +399,7 @@ void *__init bootstrap_map(const module_t *mod)
     void *ret;
 
     if ( system_state != SYS_STATE_early_boot )
-        return mod ? mfn_to_virt(mod->mod_start) : NULL;
+        return mod ? mfn_to_directmap_virt(mod->mod_start) : NULL;
 
     if ( !mod )
     {
@@ -1708,7 +1708,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     {
         set_pdx_range(mod[i].mod_start,
                       mod[i].mod_start + PFN_UP(mod[i].mod_end));
-        map_pages_to_xen((unsigned long)mfn_to_virt(mod[i].mod_start),
+        map_pages_to_xen((unsigned long)mfn_to_directmap_virt(mod[i].mod_start),
                          _mfn(mod[i].mod_start),
                          PFN_UP(mod[i].mod_end), PAGE_HYPERVISOR);
     }
diff --git a/xen/arch/x86/tboot.c b/xen/arch/x86/tboot.c
index ba0700d2d5..58df01dfb9 100644
--- a/xen/arch/x86/tboot.c
+++ b/xen/arch/x86/tboot.c
@@ -253,7 +253,7 @@ static void tboot_gen_xenheap_integrity(const uint8_t key[TB_KEY_SIZE],
             continue; /* skip tboot and its page tables */
 
         if ( is_page_in_use(page) && is_special_page(page) )
-            vmac_update(mfn_to_virt(mfn), PAGE_SIZE, &ctx);
+            vmac_update(mfn_to_directmap_virt(mfn), PAGE_SIZE, &ctx);
     }
     *mac = vmac(NULL, 0, nonce, NULL, &ctx);
 
diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index b2a280fba3..1697760d82 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -1247,19 +1247,25 @@ int memory_add(unsigned long spfn, unsigned long epfn, unsigned int pxm)
     i = virt_to_mfn(HYPERVISOR_VIRT_END - 1) + 1;
     if ( spfn < i )
     {
-        ret = map_pages_to_xen((unsigned long)mfn_to_virt(spfn), _mfn(spfn),
-                               min(epfn, i) - spfn, PAGE_HYPERVISOR);
-        if ( ret )
-            goto destroy_directmap;
+        if ( arch_mfns_in_directmap(spfn, min(epfn, i) - spfn ) )
+        {
+            ret = map_pages_to_xen((unsigned long)mfn_to_directmap_virt(spfn), _mfn(spfn),
+                                min(epfn, i) - spfn, PAGE_HYPERVISOR);
+            if ( ret )
+                goto destroy_directmap;
+        }
     }
     if ( i < epfn )
     {
         if ( i < spfn )
             i = spfn;
-        ret = map_pages_to_xen((unsigned long)mfn_to_virt(i), _mfn(i),
-                               epfn - i, __PAGE_HYPERVISOR_RW);
-        if ( ret )
-            goto destroy_directmap;
+        if ( arch_mfns_in_directmap(spfn, epfn - i) )
+        {
+            ret = map_pages_to_xen((unsigned long)mfn_to_directmap_virt(i), _mfn(i),
+                                epfn - i, __PAGE_HYPERVISOR_RW);
+            if ( ret )
+                goto destroy_directmap;
+        }
     }
 
     old_node_start = node_start_pfn(node);
@@ -1348,8 +1354,8 @@ destroy_frametable:
     NODE_DATA(node)->node_start_pfn = old_node_start;
     NODE_DATA(node)->node_spanned_pages = old_node_span;
  destroy_directmap:
-    destroy_xen_mappings((unsigned long)mfn_to_virt(spfn),
-                         (unsigned long)mfn_to_virt(epfn));
+    destroy_xen_mappings((unsigned long)mfn_to_directmap_virt(spfn),
+                         (unsigned long)mfn_to_directmap_virt(epfn));
 
     return ret;
 }
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index efbec00af9..39aed5845d 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1599,7 +1599,7 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end,
     for ( ; mfn < end; mfn = next )
     {
         l4_pgentry_t l4e = efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)];
-        unsigned long va = (unsigned long)mfn_to_virt(mfn);
+        unsigned long va = (unsigned long)mfn_to_directmap_virt(mfn);
 
         if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) )
             UNMAP_DOMAIN_PAGE(l3dst);
@@ -1757,15 +1757,18 @@ void __init efi_init_memory(void)
         if ( pfn_to_pdx(emfn - 1) < (DIRECTMAP_SIZE >> PAGE_SHIFT) &&
              pdx_is_region_compressible(mem_base, mem_npages) )
         {
-            if ( (unsigned long)mfn_to_virt(emfn - 1) >= HYPERVISOR_VIRT_END )
-                prot &= ~_PAGE_GLOBAL;
-            if ( map_pages_to_xen((unsigned long)mfn_to_virt(smfn),
-                                  _mfn(smfn), emfn - smfn, prot) == 0 )
-                desc->VirtualStart =
-                    (unsigned long)maddr_to_virt(desc->PhysicalStart);
-            else
-                printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
-                       smfn, emfn - 1);
+            if ( arch_mfns_in_directmap(smfn, emfn - smfn) )
+            {
+                if ( (unsigned long)mfn_to_directmap_virt(emfn - 1) >= HYPERVISOR_VIRT_END )
+                    prot &= ~_PAGE_GLOBAL;
+                if ( map_pages_to_xen((unsigned long)mfn_to_directmap_virt(smfn),
+                                    _mfn(smfn), emfn - smfn, prot) == 0 )
+                    desc->VirtualStart =
+                        (unsigned long)maddr_to_virt(desc->PhysicalStart);
+                else
+                    printk(XENLOG_ERR "Could not map MFNs %#lx-%#lx\n",
+                        smfn, emfn - 1);
+            }
         }
         else if ( !((desc->PhysicalStart + len - 1) >> (VADDR_BITS - 1)) &&
                   (extra = xmalloc(struct rt_extra)) != NULL )
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 3c0909f333..e1e98d4d59 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -611,8 +611,8 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
     {
         if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
         {
-            _heap[node] = mfn_to_virt(mfn + nr - needed);
-            avail[node] = mfn_to_virt(mfn + nr - 1) +
+            _heap[node] = mfn_to_directmap_virt(mfn + nr - needed);
+            avail[node] = mfn_to_directmap_virt(mfn + nr - 1) +
                           PAGE_SIZE - sizeof(**avail) * NR_ZONES;
         }
         else
@@ -631,8 +631,8 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
     {
         if ( arch_mfns_in_directmap(mfn, needed) )
         {
-            _heap[node] = mfn_to_virt(mfn);
-            avail[node] = mfn_to_virt(mfn + needed - 1) +
+            _heap[node] = mfn_to_directmap_virt(mfn);
+            avail[node] = mfn_to_directmap_virt(mfn + needed - 1) +
                           PAGE_SIZE - sizeof(**avail) * NR_ZONES;
         }
         else
diff --git a/xen/common/trace.c b/xen/common/trace.c
index 4e7b080e61..955509a0d8 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -229,7 +229,7 @@ static int alloc_trace_bufs(unsigned int pages)
         offset = t_info->mfn_offset[cpu];
 
         /* Initialize the buffer metadata */
-        per_cpu(t_bufs, cpu) = buf = mfn_to_virt(t_info_mfn_list[offset]);
+        per_cpu(t_bufs, cpu) = buf = mfn_to_directmap_virt(t_info_mfn_list[offset]);
         buf->cons = buf->prod = 0;
 
         printk(XENLOG_INFO "xentrace: p%d mfn %x offset %u\n",
@@ -268,7 +268,7 @@ out_dealloc:
             if ( !mfn )
                 break;
             ASSERT(!(mfn_to_page(_mfn(mfn))->count_info & PGC_allocated));
-            free_xenheap_pages(mfn_to_virt(mfn), 0);
+            free_xenheap_pages(mfn_to_directmap_virt(mfn), 0);
         }
     }
     free_xenheap_pages(t_info, get_order_from_pages(t_info_pages));
@@ -529,7 +529,7 @@ static unsigned char *next_record(const struct t_buf *buf, uint32_t *next,
     per_cpu_mfn_offset = t_info->mfn_offset[smp_processor_id()];
     mfn_list = (uint32_t *)t_info;
     mfn = mfn_list[per_cpu_mfn_offset + per_cpu_mfn_nr];
-    this_page = mfn_to_virt(mfn);
+    this_page = (unsigned char *)mfn_to_directmap_virt(mfn);
     if (per_cpu_mfn_nr + 1 >= opt_tbuf_size)
     {
         /* reached end of buffer? */
@@ -538,7 +538,7 @@ static unsigned char *next_record(const struct t_buf *buf, uint32_t *next,
     else
     {
         mfn = mfn_list[per_cpu_mfn_offset + per_cpu_mfn_nr + 1];
-        *next_page = mfn_to_virt(mfn);
+        *next_page = mfn_to_directmap_virt(mfn);
     }
     return this_page;
 }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 14:08:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 14:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721005.1124141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6WLQ-0004Wd-DN; Mon, 13 May 2024 14:08:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721005.1124141; Mon, 13 May 2024 14:08:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6WLQ-0004WW-AS; Mon, 13 May 2024 14:08:00 +0000
Received: by outflank-mailman (input) for mailman id 721005;
 Mon, 13 May 2024 14:07:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w2Lj=MQ=amazon.co.uk=prvs=8568adbc3=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6Vvr-0002lY-Os
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 13:41:35 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8031db8c-112e-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 15:41:30 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 13:41:28 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.0.204:18499]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.32.236:2525]
 with esmtp (Farcaster)
 id a8122ecf-7fd9-4f59-993e-fba829406aa3; Mon, 13 May 2024 13:41:27 +0000 (UTC)
Received: from EX19D008UEA001.ant.amazon.com (10.252.134.62) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:27 +0000
Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by
 EX19D008UEA001.ant.amazon.com (10.252.134.62) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Mon, 13 May 2024 13:41:26 +0000
Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com
 (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 13:41:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8031db8c-112e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715607690; x=1747143690;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=SVsOmDKOmFhMFa0LMfYH4rjGO+/XR9rqN+YrORlcGU8=;
  b=FmV0JThiGHomkIF1ZwyUKyGi+HV8jmCwydiLpEgrZ7lfcIvU6SSGkSvK
   gIwP86Va6QZmCa2H7Qtgjn2/Rlh+p4IqTiEExvu4NZxiU/uTnnqOI6VtA
   gccA902/jW1qEMULSH/OI5YljpEufZtKBvovWQTYYrBB67SCRDFolXefW
   k=;
X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; 
   d="scan'208";a="294692592"
X-Farcaster-Flow-ID: a8122ecf-7fd9-4f59-993e-fba829406aa3
From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Julien Grall
	<jgrall@amazon.com>, Stefano Stabellini <sstabellini@kernel.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Elias El Yandouzi
	<eliasely@amazon.com>
Subject: [PATCH V3 (resend) 17/19] xen/arm64: mm: Use per-pCPU page-tables
Date: Mon, 13 May 2024 13:40:44 +0000
Message-ID: <20240513134046.82605-18-eliasely@amazon.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240513134046.82605-1-eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Julien Grall <jgrall@amazon.com>

At the moment, on Arm64, every pCPU is sharing the same page-tables.

In a follow-up patch, we will allow the possibility to remove the
direct map and therefore it will be necessary to have a mapcache.

While we have plenty of spare virtual address space to reserve part
for each pCPU, it means that temporary mappings (e.g. guest memory)
could be accessible by every pCPU.

In order to increase our security posture, it would be better if
those mappings are only accessible by the pCPU doing the temporary
mapping.

In addition to that, a per-pCPU page-tables opens the way to have
per-domain mapping area.

Arm32 is already using per-pCPU page-tables so most of the code
can be re-used. Arm64 doesn't yet have support for the mapcache,
so a stub is provided (moved to its own header asm/domain_page.h).

Take the opportunity to fix a typo in a comment that is modified.

Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

----

    Changelog since v1:
        * Rebase
        * Fix typoes

diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c
index 293acb67e0..2ec1ffe1dc 100644
--- a/xen/arch/arm/arm64/mmu/mm.c
+++ b/xen/arch/arm/arm64/mmu/mm.c
@@ -76,6 +76,7 @@ static void __init prepare_runtime_identity_mapping(void)
     paddr_t id_addr = virt_to_maddr(_start);
     lpae_t pte;
     DECLARE_OFFSETS(id_offsets, id_addr);
+    lpae_t *root = this_cpu(xen_pgtable);
 
     if ( id_offsets[0] >= IDENTITY_MAPPING_AREA_NR_L0 )
         panic("Cannot handle ID mapping above %uTB\n",
@@ -86,7 +87,7 @@ static void __init prepare_runtime_identity_mapping(void)
     pte.pt.table = 1;
     pte.pt.xn = 0;
 
-    write_pte(&xen_pgtable[id_offsets[0]], pte);
+    write_pte(&root[id_offsets[0]], pte);
 
     /* Link second ID table */
     pte = pte_of_xenaddr((vaddr_t)xen_second_id);
diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c
index 3a43601623..ac2a6d0332 100644
--- a/xen/arch/arm/domain_page.c
+++ b/xen/arch/arm/domain_page.c
@@ -3,6 +3,8 @@
 #include <xen/pmap.h>
 #include <xen/vmap.h>
 
+#include <asm/domain_page.h>
+
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
diff --git a/xen/arch/arm/include/asm/arm32/mm.h b/xen/arch/arm/include/asm/arm32/mm.h
index 856f2dbec4..87a315db01 100644
--- a/xen/arch/arm/include/asm/arm32/mm.h
+++ b/xen/arch/arm/include/asm/arm32/mm.h
@@ -1,12 +1,6 @@
 #ifndef __ARM_ARM32_MM_H__
 #define __ARM_ARM32_MM_H__
 
-#include <xen/percpu.h>
-
-#include <asm/lpae.h>
-
-DECLARE_PER_CPU(lpae_t *, xen_pgtable);
-
 /*
  * Only a limited amount of RAM, called xenheap, is always mapped on ARM32.
  * For convenience always return false.
@@ -16,8 +10,6 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
     return false;
 }
 
-bool init_domheap_mappings(unsigned int cpu);
-
 static inline void arch_setup_page_tables(void)
 {
 }
diff --git a/xen/arch/arm/include/asm/domain_page.h b/xen/arch/arm/include/asm/domain_page.h
new file mode 100644
index 0000000000..e9f52685e2
--- /dev/null
+++ b/xen/arch/arm/include/asm/domain_page.h
@@ -0,0 +1,13 @@
+#ifndef __ASM_ARM_DOMAIN_PAGE_H__
+#define __ASM_ARM_DOMAIN_PAGE_H__
+
+#ifdef CONFIG_ARCH_MAP_DOMAIN_PAGE
+bool init_domheap_mappings(unsigned int cpu);
+#else
+static inline bool init_domheap_mappings(unsigned int cpu)
+{
+    return true;
+}
+#endif
+
+#endif /* __ASM_ARM_DOMAIN_PAGE_H__ */
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index 2bca3f9e87..60e0122cba 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -2,6 +2,9 @@
 #define __ARCH_ARM_MM__
 
 #include <xen/kernel.h>
+#include <xen/percpu.h>
+
+#include <asm/lpae.h>
 #include <asm/page.h>
 #include <public/xen.h>
 #include <xen/pdx.h>
diff --git a/xen/arch/arm/include/asm/mmu/mm.h b/xen/arch/arm/include/asm/mmu/mm.h
index c5e03a66bf..c03c3a51e4 100644
--- a/xen/arch/arm/include/asm/mmu/mm.h
+++ b/xen/arch/arm/include/asm/mmu/mm.h
@@ -2,6 +2,8 @@
 #ifndef __ARM_MMU_MM_H__
 #define __ARM_MMU_MM_H__
 
+DECLARE_PER_CPU(lpae_t *, xen_pgtable);
+
 /* Non-boot CPUs use this to find the correct pagetables. */
 extern uint64_t init_ttbr;
 
diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c
index da28d669e7..1ed1a53ab1 100644
--- a/xen/arch/arm/mmu/pt.c
+++ b/xen/arch/arm/mmu/pt.c
@@ -607,9 +607,9 @@ static int xen_pt_update(unsigned long virt,
     unsigned long left = nr_mfns;
 
     /*
-     * For arm32, page-tables are different on each CPUs. Yet, they share
-     * some common mappings. It is assumed that only common mappings
-     * will be modified with this function.
+     * Page-tables are different on each CPU. Yet, they share some common
+     * mappings. It is assumed that only common mappings will be modified
+     * with this function.
      *
      * XXX: Add a check.
      */
diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c
index f4bb424c3c..7b981456e6 100644
--- a/xen/arch/arm/mmu/setup.c
+++ b/xen/arch/arm/mmu/setup.c
@@ -28,17 +28,15 @@
  * PCPUs.
  */
 
-#ifdef CONFIG_ARM_64
-DEFINE_PAGE_TABLE(xen_pgtable);
-static DEFINE_PAGE_TABLE(xen_first);
-#define THIS_CPU_PGTABLE xen_pgtable
-#else
 /* Per-CPU pagetable pages */
 /* xen_pgtable == root of the trie (zeroeth level on 64-bit, first on 32-bit) */
 DEFINE_PER_CPU(lpae_t *, xen_pgtable);
 #define THIS_CPU_PGTABLE this_cpu(xen_pgtable)
 /* Root of the trie for cpu0, other CPU's PTs are dynamically allocated */
 static DEFINE_PAGE_TABLE(cpu0_pgtable);
+
+#ifdef CONFIG_ARM_64
+static DEFINE_PAGE_TABLE(xen_first);
 #endif
 
 /* Common pagetable leaves */
@@ -228,19 +226,22 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
     lpae_t pte, *p;
     int i;
 
+    p = cpu0_pgtable;
+
     phys_offset = boot_phys_offset;
 
+    /* arch_setup_page_tables() may need to access the root page-tables. */
+    per_cpu(xen_pgtable, 0) = cpu0_pgtable;
+
     arch_setup_page_tables();
 
 #ifdef CONFIG_ARM_64
     pte = pte_of_xenaddr((uintptr_t)xen_first);
     pte.pt.table = 1;
     pte.pt.xn = 0;
-    xen_pgtable[zeroeth_table_offset(XEN_VIRT_START)] = pte;
+    p[zeroeth_table_offset(XEN_VIRT_START)] = pte;
 
-    p = (void *) xen_first;
-#else
-    p = (void *) cpu0_pgtable;
+    p = xen_first;
 #endif
 
     /* Map xen second level page-table */
@@ -283,19 +284,11 @@ void __init setup_pagetables(unsigned long boot_phys_offset)
     pte.pt.table = 1;
     xen_second[second_table_offset(FIXMAP_ADDR(0))] = pte;
 
-#ifdef CONFIG_ARM_64
-    ttbr = (uintptr_t) xen_pgtable + phys_offset;
-#else
     ttbr = (uintptr_t) cpu0_pgtable + phys_offset;
-#endif
 
     switch_ttbr(ttbr);
 
     xen_pt_enforce_wnx();
-
-#ifdef CONFIG_ARM_32
-    per_cpu(xen_pgtable, 0) = cpu0_pgtable;
-#endif
 }
 
 void *__init arch_vmap_virt_end(void)
diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c
index e29b6f34f2..eb51d6aae9 100644
--- a/xen/arch/arm/mmu/smpboot.c
+++ b/xen/arch/arm/mmu/smpboot.c
@@ -7,6 +7,7 @@
 
 #include <xen/domain_page.h>
 
+#include <asm/domain_page.h>
 #include <asm/setup.h>
 
 /* Override macros from asm/page.h to make them work with mfn_t */
@@ -93,20 +94,6 @@ static void set_init_ttbr(lpae_t *root)
     unmap_domain_page(ptr);
 }
 
-#ifdef CONFIG_ARM_64
-int prepare_secondary_mm(int cpu)
-{
-    clear_boot_pagetables();
-
-    /*
-     * Set init_ttbr for this CPU coming up. All CPUs share a single setof
-     * pagetables, but rewrite it each time for consistency with 32 bit.
-     */
-    set_init_ttbr(xen_pgtable);
-
-    return 0;
-}
-#else
 int prepare_secondary_mm(int cpu)
 {
     lpae_t *root = alloc_xenheap_page();
@@ -138,7 +125,6 @@ int prepare_secondary_mm(int cpu)
 
     return 0;
 }
-#endif
 
 /*
  * Local variables:
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d242674381..d15987d6ea 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -42,6 +42,7 @@
 #include <asm/gic.h>
 #include <asm/cpuerrata.h>
 #include <asm/cpufeature.h>
+#include <asm/domain_page.h>
 #include <asm/platform.h>
 #include <asm/procinfo.h>
 #include <asm/setup.h>
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Mon May 13 15:28:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 15:28:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721030.1124152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Xan-0003LP-1s; Mon, 13 May 2024 15:27:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721030.1124152; Mon, 13 May 2024 15:27:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Xam-0003LI-V9; Mon, 13 May 2024 15:27:56 +0000
Received: by outflank-mailman (input) for mailman id 721030;
 Mon, 13 May 2024 15:27:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6Xam-0003LC-Hk
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 15:27:56 +0000
Received: from mail-qv1-xf33.google.com (mail-qv1-xf33.google.com
 [2607:f8b0:4864:20::f33])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5d591795-113d-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 17:27:53 +0200 (CEST)
Received: by mail-qv1-xf33.google.com with SMTP id
 6a1803df08f44-6a04c581ee3so13956916d6.0
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 08:27:53 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1d7040sm43574426d6.107.2024.05.13.08.27.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 08:27:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d591795-113d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715614072; x=1716218872; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=2r6U6XboQG2fdQVTVP/Ayy2jjx9leLzejn1rzqrKj98=;
        b=iofPF3uGNwNy7Zz5bszqoxlw6G0gCq8vxGLfFZ6t4sLjkyt0wJSMCQF5CEFvzHrSQ0
         Ol7oMG2UtIO0gnHYMrGyDHatJIdX9tmiReWLActwhBCB0HUQz0YnLcl5PDimq+Hn+nYM
         uwOulvg/qOhfU1W6S3BmgnvnxM5hEWXuUwIek=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715614072; x=1716218872;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2r6U6XboQG2fdQVTVP/Ayy2jjx9leLzejn1rzqrKj98=;
        b=epzzUIgsYm0VT3cekwy63N2Wc8+RTBCR7rbH5cocpTTVOmHK9WenkFySKwjfJZyIY/
         vfjX6JF0NkBVazUQ47be76PvMD8RH/xXhnXjcEMhuEaU0hpvLte03DhPRQc8wjHoIo6+
         x0Dy0DasUd0rgmb2AqhtQOpPTvXGSQLnNf6xwazKD3/2XGCx/AO/1xSToDXPPp34Q7vg
         Gcf7ydZhKroSTTCOYyi04UvOmiHO5slHl5FRoo+CNEAU0ssZmBcorJEaJhkOZfgaHSUb
         vPDUtA8bC9wnXPLctaZJ/YDuROJjrGTdyvawLQkq3sEBmPvWRMJ23BUR/VyZiS/HPjMx
         7lfQ==
X-Gm-Message-State: AOJu0YyiR0njppj33jVSTJCYqSWEz/oks9JS9REur0wef6JMBC5nlSyL
	a8fmba22JobVGB2s4AecU75Ec0y6hXnvzpztKl8PA9sztbkNc1Y1kFWbTiBdZ48=
X-Google-Smtp-Source: AGHT+IE8zhRpyTqM2dh3KZZmcJ+LjIx6vUP0uimEfK8L2aEnm7FLHZ3OtmdgASQg/B/azGV+TXbRUw==
X-Received: by 2002:a0c:9a9a:0:b0:6a3:2ab7:ecab with SMTP id 6a1803df08f44-6a32ab7edc9mr64221276d6.65.1715614072209;
        Mon, 13 May 2024 08:27:52 -0700 (PDT)
Date: Mon, 13 May 2024 17:27:50 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
Message-ID: <ZkIxdtiDc_pnPWdx@macbook>
References: <20240513111117.68828-1-eliasely@amazon.com>
 <20240513111117.68828-2-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513111117.68828-2-eliasely@amazon.com>

On Mon, May 13, 2024 at 11:10:59AM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Create a per-domain mapping of PV guest_root_pt as direct map is being
> removed.
> 
> Note that we do not map and unmap root_pgt for now since it is still a
> xenheap page.

I'm afraid this needs more context, at least for me to properly
understand.

I think I've figured out what create_perdomain_mapping() is supposed
to do, and I have to admit the interface is very awkward.

Anyway, attempted to provide some feedback.

> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
>     Changes in V3:
>         * Rename SHADOW_ROOT
>         * Haven't addressed the potentially over-allocation issue as I don't get it
> 
>     Changes in V2:
>         * Rework the shadow perdomain mapping solution in the follow-up patches
> 
>     Changes since Hongyan's version:
>         * Remove the final dot in the commit title
> 
> diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
> index ab7288cb36..5d710384df 100644
> --- a/xen/arch/x86/include/asm/config.h
> +++ b/xen/arch/x86/include/asm/config.h
> @@ -203,7 +203,7 @@ extern unsigned char boot_edid_info[128];
>  /* Slot 260: per-domain mappings (including map cache). */
>  #define PERDOMAIN_VIRT_START    (PML4_ADDR(260))
>  #define PERDOMAIN_SLOT_MBYTES   (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER))
> -#define PERDOMAIN_SLOTS         3
> +#define PERDOMAIN_SLOTS         4
>  #define PERDOMAIN_VIRT_SLOT(s)  (PERDOMAIN_VIRT_START + (s) * \
>                                   (PERDOMAIN_SLOT_MBYTES << 20))
>  /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */
> @@ -317,6 +317,14 @@ extern unsigned long xen_phys_start;
>  #define ARG_XLAT_START(v)        \
>      (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
>  
> +/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */
> +#define PV_ROOT_PT_MAPPING_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
> +#define PV_ROOT_PT_MAPPING_ENTRIES      MAX_VIRT_CPUS
> +
> +/* The address of a particular VCPU's PV_ROOT_PT */
> +#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \
> +    (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))

I know we are not there yet, but I wonder if we need to start having
some non-shared per-cpu mapping area in the page-tables.  Right now
this is shared between all the vCPUs, as it's per-domain space
(instead of per-vCPU).

> +
>  #define ELFSIZE 64
>  
>  #define ARCH_CRASH_SAVE_VMCOREINFO
> diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
> index f5daeb182b..8a97530607 100644
> --- a/xen/arch/x86/include/asm/domain.h
> +++ b/xen/arch/x86/include/asm/domain.h
> @@ -272,6 +272,7 @@ struct time_scale {
>  struct pv_domain
>  {
>      l1_pgentry_t **gdt_ldt_l1tab;
> +    l1_pgentry_t **root_pt_l1tab;
>  
>      atomic_t nr_l4_pages;
>  
> diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
> index d968bbbc73..efdf20f775 100644
> --- a/xen/arch/x86/mm.c
> +++ b/xen/arch/x86/mm.c
> @@ -505,6 +505,13 @@ void share_xen_page_with_guest(struct page_info *page, struct domain *d,
>      nrspin_unlock(&d->page_alloc_lock);
>  }
>  
> +#define pv_root_pt_idx(v) \
> +    ((v)->vcpu_id >> PAGETABLE_ORDER)
> +
> +#define pv_root_pt_pte(v) \
> +    ((v)->domain->arch.pv.root_pt_l1tab[pv_root_pt_idx(v)] + \
> +     ((v)->vcpu_id & (L1_PAGETABLE_ENTRIES - 1)))
> +
>  void make_cr3(struct vcpu *v, mfn_t mfn)
>  {
>      struct domain *d = v->domain;
> @@ -524,6 +531,13 @@ void write_ptbase(struct vcpu *v)
>  
>      if ( is_pv_vcpu(v) && v->domain->arch.pv.xpti )
>      {
> +        mfn_t guest_root_pt = _mfn(MASK_EXTR(v->arch.cr3, PAGE_MASK));
> +        l1_pgentry_t *pte = pv_root_pt_pte(v);
> +
> +        ASSERT(v == current);
> +
> +        l1e_write(pte, l1e_from_mfn(guest_root_pt, __PAGE_HYPERVISOR_RO));
> +
>          cpu_info->root_pgt_changed = true;
>          cpu_info->pv_cr3 = __pa(this_cpu(root_pgt));
>          if ( new_cr4 & X86_CR4_PCIDE )
> diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
> index 2a445bb17b..1b025986f7 100644
> --- a/xen/arch/x86/pv/domain.c
> +++ b/xen/arch/x86/pv/domain.c
> @@ -288,6 +288,21 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
>                                1U << GDT_LDT_VCPU_SHIFT);
>  }
>  
> +static int pv_create_root_pt_l1tab(struct vcpu *v)
> +{
> +    return create_perdomain_mapping(v->domain,
> +                                    PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v),
> +                                    1, v->domain->arch.pv.root_pt_l1tab,
> +                                    NULL);
> +}
> +
> +static void pv_destroy_root_pt_l1tab(struct vcpu *v)

The two 'v' parameters could be const here.

> +
> +{
> +    destroy_perdomain_mapping(v->domain,
> +                              PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v), 1);
> +}
> +
>  void pv_vcpu_destroy(struct vcpu *v)
>  {
>      if ( is_pv_32bit_vcpu(v) )
> @@ -297,6 +312,7 @@ void pv_vcpu_destroy(struct vcpu *v)
>      }
>  
>      pv_destroy_gdt_ldt_l1tab(v);
> +    pv_destroy_root_pt_l1tab(v);
>      XFREE(v->arch.pv.trap_ctxt);
>  }
>  
> @@ -311,6 +327,13 @@ int pv_vcpu_initialise(struct vcpu *v)
>      if ( rc )
>          return rc;
>  
> +    if ( v->domain->arch.pv.xpti )
> +    {
> +        rc = pv_create_root_pt_l1tab(v);
> +        if ( rc )
> +            goto done;
> +    }
> +
>      BUILD_BUG_ON(X86_NR_VECTORS * sizeof(*v->arch.pv.trap_ctxt) >
>                   PAGE_SIZE);
>      v->arch.pv.trap_ctxt = xzalloc_array(struct trap_info, X86_NR_VECTORS);
> @@ -346,10 +369,12 @@ void pv_domain_destroy(struct domain *d)
>  
>      destroy_perdomain_mapping(d, GDT_LDT_VIRT_START,
>                                GDT_LDT_MBYTES << (20 - PAGE_SHIFT));
> +    destroy_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START, PV_ROOT_PT_MAPPING_ENTRIES);

Line too long.  Also see comment below about using d->max_vcpus
instead of MAX_VIRT_CPUS.

>  
>      XFREE(d->arch.pv.cpuidmasks);
>  
>      FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab);
> +    FREE_XENHEAP_PAGE(d->arch.pv.root_pt_l1tab);
>  }
>  
>  void noreturn cf_check continue_pv_domain(void);
> @@ -371,6 +396,12 @@ int pv_domain_initialise(struct domain *d)
>          goto fail;
>      clear_page(d->arch.pv.gdt_ldt_l1tab);
>  
> +    d->arch.pv.root_pt_l1tab =
> +        alloc_xenheap_pages(0, MEMF_node(domain_to_node(d)));
> +    if ( !d->arch.pv.root_pt_l1tab )
> +        goto fail;
> +    clear_page(d->arch.pv.root_pt_l1tab);
> +
>      if ( levelling_caps & ~LCAP_faulting &&
>           (d->arch.pv.cpuidmasks = xmemdup(&cpuidmask_defaults)) == NULL )
>          goto fail;
> @@ -381,6 +412,11 @@ int pv_domain_initialise(struct domain *d)
>      if ( rc )
>          goto fail;
>  
> +    rc = create_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START,
> +                                  PV_ROOT_PT_MAPPING_ENTRIES, NULL, NULL);

Why not use d->max_vcpus here, instead of forcing up to MAX_VIRT_CPUS?

By the time pv_domain_initialise() is called max_vcpus should already
be initialized.  AFAICT it doesn't make a difference, because for the
call here only the L3 table is created, as last two parameters are
NULL, but still is more accurate to use max_vcpus.

> +    if ( rc )
> +        goto fail;
> +
>      d->arch.ctxt_switch = &pv_csw;
>  
>      d->arch.pv.xpti = is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu;
> diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
> index 630bdc3945..c1ae5013af 100644
> --- a/xen/arch/x86/x86_64/asm-offsets.c
> +++ b/xen/arch/x86/x86_64/asm-offsets.c
> @@ -80,6 +80,7 @@ void __dummy__(void)
>  
>  #undef OFFSET_EF
>  
> +    OFFSET(VCPU_id, struct vcpu, vcpu_id);
>      OFFSET(VCPU_processor, struct vcpu, processor);
>      OFFSET(VCPU_domain, struct vcpu, domain);
>      OFFSET(VCPU_vcpu_info, struct vcpu, vcpu_info_area.map);
> diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
> index df015589ce..c1377da7a5 100644
> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -162,7 +162,15 @@ FUNC_LOCAL(restore_all_guest)
>          and   %rsi, %rdi
>          and   %r9, %rsi
>          add   %rcx, %rdi
> +
> +        /*
> +         * The address in the vCPU cr3 is always mapped in the per-domain
> +         * pv_root_pt virt area.
> +         */
> +        imul  $PAGE_SIZE, VCPU_id(%rbx), %esi

Aren't some of the previous operations against %rsi now useless since
it gets unconditionally overwritten here?

and   %r9, %rsi
[...]
add   %rcx, %rsi

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 13 15:40:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 15:40:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721036.1124161 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Xms-0001nF-3T; Mon, 13 May 2024 15:40:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721036.1124161; Mon, 13 May 2024 15:40:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6Xms-0001n8-0z; Mon, 13 May 2024 15:40:26 +0000
Received: by outflank-mailman (input) for mailman id 721036;
 Mon, 13 May 2024 15:40:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6Xmq-0001n2-Or
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 15:40:24 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c2ebad9-113f-11ef-b4bb-af5377834399;
 Mon, 13 May 2024 17:40:22 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2e4b90b03a9so50924221fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 08:40:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b79bc83sm11388594f8f.16.2024.05.13.08.40.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 08:40:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c2ebad9-113f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715614822; x=1716219622; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=P8xZGMjf01HbyNWji+ISVHSj5gl4UxwZFFpPnC9HGAc=;
        b=MQ3bRmZ7FMm6oEOLBQS7TmOWT4VnPa2ECcYTU/79oscvCsz+A+jh953KdQj/L+GCtK
         nJQz0K/we5W5rVZrSu+vWUA7oDQfwWDc1ERGid335sCX52256JXnIdTenjMhpXgNmvCI
         JKF/MRm+xkGPO0M9/p554LXRYXlcEyGj9diDA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715614822; x=1716219622;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=P8xZGMjf01HbyNWji+ISVHSj5gl4UxwZFFpPnC9HGAc=;
        b=ZtOyIE9NsYRLJxB/+VvNHryR79dDPRdpoVRPurp6JuAfAwniSvuMMkAwovlBXLER06
         Vh6bOESWRfn/P3vtFnRu3930X1Nf/xL+kyylhiD3tYtezSk7ypuW9ugH+k3vvrvojEcc
         wCJgD4g4Jb/ikN6dGIe3BTGVlMn/8iL7AmtMmcruCbHd9AuuKUComIFEzFFdfEFcNhAS
         /X5d91IxK0CqvBeqv/o8UDMgNm/+iC2rYYqxDYKgaZBBy9Hh89KFNOyfesEtHw+HY2m1
         3jd4DTzHxrP/hh0CiWgUjs9e5sKV8ZzMcPtj4CEz+YOrGh0bsQ5m0doUkZpzxGMFNJKn
         lMkg==
X-Gm-Message-State: AOJu0YweDdVBCiBuWK+m4fDXdMAh/UFkHk8148G+MW79rLbgnePUEo80
	iFgcmoR3eiJCYHED99xTbtRAYU9gEDE+Z6Sw3H7B2ePlnk60df0hsNrWI51my3g=
X-Google-Smtp-Source: AGHT+IGaIxURuvDuIXdP/6zCWBbxx91s/O3PlQJB2bvNkLYtXxQPBFYrlggYrnusZ74cC3yKrdQptQ==
X-Received: by 2002:a2e:9b8b:0:b0:2e5:59a:591d with SMTP id 38308e7fff4ca-2e51fd3fe2emr71209121fa.4.1715614821996;
        Mon, 13 May 2024 08:40:21 -0700 (PDT)
Date: Mon, 13 May 2024 17:40:20 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Wei Liu <wei.liu2@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Wang <wawei@amazon.de>, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 02/19] x86/pv: Domheap pages should be mapped
 while relocating initrd
Message-ID: <ZkI0ZJKq9HT465VC@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-3-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-3-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:29PM +0000, Elias El Yandouzi wrote:
> From: Wei Liu <wei.liu2@citrix.com>
> 
> Xen shouldn't use domheap page as if they were xenheap pages. Map and
> unmap pages accordingly.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> Signed-off-by: Wei Wang <wawei@amazon.de>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
>     Changes in V3:
>         * Rename commit title
>         * Rework the for loop copying the pages
> 
>     Changes in V2:
>         * Get rid of mfn_to_virt
>         * Don't open code copy_domain_page()
> 
>     Changes since Hongyan's version:
>         * Add missing newline after the variable declaration
> 
> diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
> index d8043fa58a..807296c280 100644
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -618,18 +618,24 @@ int __init dom0_construct_pv(struct domain *d,
>          if ( d->arch.physaddr_bitsize &&
>               ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) )
>          {
> +            unsigned int nr_pages = 1UL << order;

Shouldn't you better initialize nr_pages to 'count' instead of 'order'
here?

Also note how 'order' at this point is not yet initialized to the
'count' based value, so I'm not sure from where that 'order' is coming
from.

In v2 you had:

+            unsigned long nr_pages;
+
             order = get_order_from_pages(count);
             page = alloc_domheap_pages(d, order, MEMF_no_scrub);
             if ( !page )
                 panic("Not enough RAM for domain 0 initrd\n");
+
+            nr_pages = 1UL << order;

nr_pages was derived from the 'order' value based on 'count'.  As said
above, I think you want to use just 'count' here, which is the rounded
up value of pages needed by initrd_len.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 13 16:49:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 16:49:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721047.1124171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6YrU-000463-Sx; Mon, 13 May 2024 16:49:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721047.1124171; Mon, 13 May 2024 16:49:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6YrU-00045w-QJ; Mon, 13 May 2024 16:49:16 +0000
Received: by outflank-mailman (input) for mailman id 721047;
 Mon, 13 May 2024 16:49:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YK/2=MQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6YrT-00045q-PN
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 16:49:15 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b9d19e32-1148-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 18:49:13 +0200 (CEST)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-792b8efed93so457469685a.3
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 09:49:13 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f194c3asm44370126d6.69.2024.05.13.09.49.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 May 2024 09:49:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9d19e32-1148-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715618952; x=1716223752; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=p7hokb4SEQ/ArDzEoy7A4TmgTIwDsfbPYMk1ioH9o8Q=;
        b=JlTaP7o0HgL6jnNAIZgeXAthJubL2rdh/eYgPoqakfRUkHL7IyGrkhHNkPCtTFNOT7
         Ir7iB425prpZxyoSPAmYKZiE5fhqwFYPSS2gC8GqRsFhyp2n0HKWuxvzOLLM1NmmZcOT
         5rV5TKI8hj0m0XCJMAY77vCzlutMURPObCcBQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715618952; x=1716223752;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=p7hokb4SEQ/ArDzEoy7A4TmgTIwDsfbPYMk1ioH9o8Q=;
        b=AHEY5xRdWNYi3XUEklm8dIGUAvsNwOTs7Gifi5p5ugtj5u7s5kH3ft5T8ZcswR71xR
         GH1FZ1KCKl3kV6Kx+L1zZQSglDnPUp087Py4PknMjJ6Ea7ZQX2AhMFuvLkM37Sh90ERC
         er1Sex1KmECD7SFMjGr7ojDGmmGQd9PZB4drq3oWL1X4ZJnpc51ZZieqA9di2/8PHglE
         VyaK6ILQN4Jvys+H3qmQdSbgb3EaDnBhnJ4u2nwZTzArmhqBzZ4dkdoh6Ynh2WLUUAkn
         UqF4kikj2sqUdnyx70YcUIRSX0cI9742+Ole6FhNQBeufFoXfyMBljasqXwkvubejE+k
         mv3A==
X-Gm-Message-State: AOJu0YzVIpByRjRdr24ngIM0bd/BhMMdPwNZ4HESpPWOkJzdkg1U8SG3
	UNlYe7W7/7/5ssR9AnrARt45d14ucIiyF5d/VObgadkqb7SRRGvQz64rk5noZBM=
X-Google-Smtp-Source: AGHT+IG7EA/L9JDanw6dYkwgUlDsZEk4K4ZNrm1S2vRyJiw8sGWJyCWj928qEaU93W4jhK6NowDFWg==
X-Received: by 2002:a05:6214:4302:b0:6a0:b3cc:ee06 with SMTP id 6a1803df08f44-6a16821eb2fmr127764936d6.37.1715618951695;
        Mon, 13 May 2024 09:49:11 -0700 (PDT)
Date: Mon, 13 May 2024 18:49:09 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 03/19] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
Message-ID: <ZkJEhZCRNnxGb_lc@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-4-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-4-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:30PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Building a PV dom0 is allocating from the domheap but uses it like the
> xenheap. Use the pages as they should be.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
>     Changes in V3:
>         * Fold following patch 'x86/pv: Map L4 page table for shim domain'
> 
>     Changes in V2:
>         * Clarify the commit message
>         * Break the patch in two parts
> 
>     Changes since Hongyan's version:
>         * Rebase
>         * Remove spurious newline
> 
> diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
> index 807296c280..ac910b438a 100644
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
>      l3_pgentry_t *l3tab = NULL, *l3start = NULL;
>      l2_pgentry_t *l2tab = NULL, *l2start = NULL;
>      l1_pgentry_t *l1tab = NULL, *l1start = NULL;
> +    mfn_t l4start_mfn = INVALID_MFN;
> +    mfn_t l3start_mfn = INVALID_MFN;
> +    mfn_t l2start_mfn = INVALID_MFN;
> +    mfn_t l1start_mfn = INVALID_MFN;
>  
>      /*
>       * This fully describes the memory layout of the initial domain. All
> @@ -710,22 +714,32 @@ int __init dom0_construct_pv(struct domain *d,
>          v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
>      }
>  
> +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
> +do {                                                    \
> +    unmap_domain_page(virt_var);                        \
> +    mfn_var = maddr_to_mfn(maddr);                      \
> +    maddr += PAGE_SIZE;                                 \
> +    virt_var = map_domain_page(mfn_var);                \

FWIW, I would do the advance after the map, so that the order matches
the name of the function.

> +} while ( false )
> +
>      if ( !compat )
>      {
>          maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
> -        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
> +        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
> +        l4tab = l4start;

You could even make the macro return virt_var, and so use it like:

l4tab = l4start = UNMAP_MAP_AND_ADVANCE(l4start_mfn, mpt_alloc);

?

Anyway, no strong opinion.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 13 17:20:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 17:20:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721051.1124182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ZLJ-0005xo-67; Mon, 13 May 2024 17:20:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721051.1124182; Mon, 13 May 2024 17:20:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ZLJ-0005x9-1l; Mon, 13 May 2024 17:20:05 +0000
Received: by outflank-mailman (input) for mailman id 721051;
 Mon, 13 May 2024 17:20:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RTuJ=MQ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s6ZLH-0005cG-LI
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 17:20:03 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0863e290-114d-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 19:20:02 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51f57713684so5815423e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 13 May 2024 10:20:02 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f27sm620457866b.127.2024.05.13.10.20.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 13 May 2024 10:20:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0863e290-114d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715620802; x=1716225602; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KZmfNVMPS0IBwYbYtAS1z3FtJotOQqYjhI7vFe6l/TU=;
        b=cgx9jmrOXnw/I5uRBrg8wbAfup2WHSHAq/7gBmIrSDBu7dU+rAM8ofLb7rtmk0hODO
         HUC52SAPwIhr0iRvHTPJnWTQMyDLtiYboXlxcsDODn4DfoaPOK39PhQ7r9eJULxutNwE
         uQhQw1p75lesRchFAbFVCs4MlKHOXL8fQhVjg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715620802; x=1716225602;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KZmfNVMPS0IBwYbYtAS1z3FtJotOQqYjhI7vFe6l/TU=;
        b=C2fsXRMblBPHoebumjsvKCjXIr0uANUBihWzs5LSvZ9tn7+tEuEN7H7jbJAzzNqGu5
         oMXHCo2AsRwfV5IH9cmJtRD0UuEF7uUTwDFNA//zv2EVl8nH1xga5pm+UoMM0oi4Xbrk
         biKNe94kRFJq14bvZW7Gv01UHrCHxzYEKHiZfLXIzEGCpaXb7a4g70DjBS8rbrCTvy75
         c9d3vxtGswcn2l3+EzuS/jEwzdxNPx6pM1A9BxT0ZCQgdh64mIs/+83/8k5soqUJfPVh
         9EfE2Zkg9joICt1NtpmjWL3GMIeB5w20qogLT2dwfVvD7u+nLNG+0ikV+doZaikpwLqi
         hSjw==
X-Forwarded-Encrypted: i=1; AJvYcCXFHk5Dt6UZEsjJjwdExMV1656jdQr5tFMQpfhck0PpF9ds6nTBG5kPhhQEx/DSpoXzS19+l0MbgR9FK7FwmHYakaE9FU8D7T8ubj1VQJo=
X-Gm-Message-State: AOJu0Yyn+bKZE3TIYQhF3Sm1cQKOanLjvk9yLCWChyGsCzFGJJBw7JWc
	B2AgxwcmdqtDsuHwj/Ue7bo1OuzpeHHDEq4lxYsHYf+ipU0HNzEixo2SYNfcrnk=
X-Google-Smtp-Source: AGHT+IHJKN0CS2sNekYc0Aw632qYyOpPcehU52iStFXZV8FQtruHfAZ1/Yd1F4qxJ/vdHcWsQQ0XkA==
X-Received: by 2002:a19:770b:0:b0:51e:f8ae:db35 with SMTP id 2adb3069b0e04-5220fe798b9mr6411995e87.43.1715620801714;
        Mon, 13 May 2024 10:20:01 -0700 (PDT)
Message-ID: <0890e0b6-ddd4-4e3a-b100-2bda5e880283@cloud.com>
Date: Mon, 13 May 2024 18:19:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4.5/8] tools/hvmloader: Further simplify SMP setup
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
 <20240509175057.1921538-1-andrew.cooper3@citrix.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <20240509175057.1921538-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09/05/2024 18:50, Andrew Cooper wrote:
> Now that we're using hypercalls to start APs, we can replace the 'ap_cpuid'
> global with a regular function parameter.  This requires telling the compiler
> that we'd like the parameter in a register rather than on the stack.
> 
> While adjusting, rename to cpu_setup().  It's always been used on the BSP,
> making the name ap_start() specifically misleading.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> 
> This is a trick I found for XTF, not that I've completed the SMP support yet.
> 
> I realise it's cheating slightly WRT 4.19, but it came out of the middle of a
> series targetted for 4.19.
> ---
>  tools/firmware/hvmloader/smp.c | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
> index 6ebf0b60faab..5d46eee1c5f4 100644
> --- a/tools/firmware/hvmloader/smp.c
> +++ b/tools/firmware/hvmloader/smp.c
> @@ -29,15 +29,15 @@
>  
>  #include <xen/vcpu.h>
>  
> -static int ap_callin, ap_cpuid;
> +static int ap_callin;
>  
> -static void ap_start(void)
> +static void __attribute__((regparm(1))) cpu_setup(unsigned int cpu)

I like it, but I'm not a fan of compiler attributes when there's sane
alternatives. We could pre-push the argument onto ap_stack to achieve
the same thing. As in, add a -4 offset to esp, and write "cpu" there.

  *(uint32_t*)ap.cpu_regs.x86_32.esp) = cpu;

That said, this is a solution as good as any other and it's definitely
better than a global, so take it or leave it.

With or without the proposed alternative...

Reviewed-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

>  {
> -    printf(" - CPU%d ... ", ap_cpuid);
> +    printf(" - CPU%d ... ", cpu);
>      cacheattr_init();
>      printf("done.\n");
>  
> -    if ( !ap_cpuid ) /* Used on the BSP too */
> +    if ( !cpu ) /* Used on the BSP too */
>          return;
>  
>      wmb();
> @@ -55,7 +55,6 @@ static void boot_cpu(unsigned int cpu)
>      static struct vcpu_hvm_context ap;
>  
>      /* Initialise shared variables. */
> -    ap_cpuid = cpu;
>      ap_callin = 0;
>      wmb();
>  
> @@ -63,9 +62,11 @@ static void boot_cpu(unsigned int cpu)
>      ap = (struct vcpu_hvm_context) {
>          .mode = VCPU_HVM_MODE_32B,
>          .cpu_regs.x86_32 = {
> -            .eip = (unsigned long)ap_start,
> +            .eip = (unsigned long)cpu_setup,
>              .esp = (unsigned long)ap_stack + ARRAY_SIZE(ap_stack),
>  
> +            .eax = cpu,
> +
>              /* Protected Mode, no paging. */
>              .cr0 = X86_CR0_PE,
>  
> @@ -105,7 +106,7 @@ void smp_initialise(void)
>      unsigned int i, nr_cpus = hvm_info->nr_vcpus;
>  
>      printf("Multiprocessor initialisation:\n");
> -    ap_start();
> +    cpu_setup(0);
>      for ( i = 1; i < nr_cpus; i++ )
>          boot_cpu(i);
>  }
> 
> base-commit: 53959cb8309919fc2f157a1c99e0af2ce280cb84



From xen-devel-bounces@lists.xenproject.org Mon May 13 20:12:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 20:12:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721067.1124192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6c1X-0000Z9-4z; Mon, 13 May 2024 20:11:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721067.1124192; Mon, 13 May 2024 20:11:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6c1X-0000Z2-1v; Mon, 13 May 2024 20:11:51 +0000
Received: by outflank-mailman (input) for mailman id 721067;
 Mon, 13 May 2024 20:11:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+RwE=MQ=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1s6c1W-0000Yu-8C
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 20:11:50 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 063e1dfd-1165-11ef-909d-e314d9c70b13;
 Mon, 13 May 2024 22:11:48 +0200 (CEST)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.17.1) with ESMTPS id 44DKBXDR030651
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Mon, 13 May 2024 16:11:39 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 44DKBXe5030650;
 Mon, 13 May 2024 13:11:33 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 063e1dfd-1165-11ef-909d-e314d9c70b13
Date: Mon, 13 May 2024 13:11:33 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org,
        Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
        Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Serious AMD-Vi(?) issue
Message-ID: <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
References: <ZgHwEGCsCLHiYU5J@mattapan.m5p.com>
 <ZgRXHQpamLIdu7dk@mattapan.m5p.com>
 <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
 <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZkHTC4RpUSpKj4wf@macbook>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Mon, May 13, 2024 at 10:44:59AM +0200, Roger Pau Monn� wrote:
> On Fri, May 10, 2024 at 09:09:54PM -0700, Elliott Mitchell wrote:
> > On Thu, Apr 18, 2024 at 09:33:31PM -0700, Elliott Mitchell wrote:
> > > 
> > > I suspect this is a case of there is some step which is missing from
> > > Xen's IOMMU handling.  Perhaps something which Linux does during an early
> > > DMA setup stage, but the current Xen implementation does lazily?
> > > Alternatively some flag setting or missing step?
> > > 
> > > I should be able to do another test approach in a few weeks, but I would
> > > love if something could be found sooner.
> > 
> > Turned out to be disturbingly easy to get the first entry when it
> > happened.  Didn't even need `dbench`, it simply showed once the OS was
> > fully loaded.  I did get some additional data points.
> > 
> > Appears this requires an AMD IOMMUv2.  A test system with known
> > functioning AMD IOMMUv1 didn't display the issue at all.
> > 
> > (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr fffffffdf8000000 flags 0x8 I
> 
> I would expect the address field to contain more information about the
> fault, but I'm not finding any information on the AMD-Vi specification
> apart from that it contains the DVA, which makes no sense when the
> fault is caused by an interrupt.
> 
> > (XEN) DDDD:bb:dd.f root @ 83b5f5 (3 levels) dfn=fffffffdf8000
> > (XEN)   L3[1f7] = 0 np
> 
> Attempting to print the page table walk for an Interrupt remapping
> fault is useless, we should likely avoid that when the I flag is set.

> > I find it surprising this required "iommu=debug" to get this level of
> > detail.  This amount of output seems more appropriate for "verbose".
> 
> "verbose" should also print this information.

Mostly I've noticed Xen's dmesg seems a bit sparse at default settings.
Confirming IOMMU was recognized and operational had been a challenge.  On
the flip side this does mean less potentially sensitive data gets in.

> > I strongly prefer to provide snippets.  There is a fair bit of output,
> > I'm unsure which portion is most pertinent.
> 
> I've already voiced my concern that I think what yo uare doing is not
> fair.  We are debugging this out of interest, and hence you refusing
> to provide all information just hampers our ability to debug, and
> makes us spend more time than required just thinking what snippets we
> need to ask for.
> 
> I will ask again, what's there in the Xen or the Linux dmesgs that you
> are so worried about leaking? Please provide an specific example.

I cannot point to specific data in Xen's dmesg which is known to be
sensitive.  On the flip side all the addresses could readily function as
a subliminal channel.

Might only be kernels from certain vendors, but hardware serial numbers
frequently make it into Linux's dmesg.  All the data coming from ACPI
tables could readily hide something.  Worse, data which seems harmless
now might later turn out to reveal things.

The usual approach is everyone has PGP keys and logs are kept private
on request.

> Why do you mask the device SBDF in the above snippet?  I would really
> like to understand what's so privacy relevant in a PCI SBDF number.

I doubt it reveals much.  Simply seems unlikely to help debugging and
therefore I prefer to mask it.  One more Xen dmesg line:

(XEN) AMD-Vi: Setup I/O page table: device id = 0xbbdd, type = 0x1, root table = 0xADDRADDR, domain = 0, paging mode = 3

> Does booting with `iommu=no-intremap` lead to any issues being
> reported?

I'll try that next time I restart the system.


Another viable approach.  I imagine one or more of the Xen developers
have computers with AMD processors.  I could send a pair of SATA devices
which are known to exhibit the behavior to someone.

The known reproductions have featured ASUS motherboards.  I doubt this is
a requirement, but if one of the main developers has such a system that
is a better target.  I also note these are plugged into motherboard SATA
ports.  It is possible add-on card SATA ports might not exhibit the
behavior.

Then you may discover not much log data is being provided simply due to
not much log data being generated.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Mon May 13 21:28:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 21:28:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721076.1124201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6dDa-0001Rj-HX; Mon, 13 May 2024 21:28:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721076.1124201; Mon, 13 May 2024 21:28:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6dDa-0001Rc-F3; Mon, 13 May 2024 21:28:22 +0000
Received: by outflank-mailman (input) for mailman id 721076;
 Mon, 13 May 2024 21:28:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6dDY-0001RW-SQ
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 21:28:20 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6dDY-00060p-Fi; Mon, 13 May 2024 21:28:20 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6dDY-0002Jt-7u; Mon, 13 May 2024 21:28:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=s5iFEEnN9UXy4r21ykiZB5NBzlGvJTTeFya/dDBwq4w=; b=CSz96nr42qkhMOWz0M5OSGbfkD
	rAbgaiiamMFx9tXem6qHZfI+GVImVy5oT1uG2/PgkXnvM1i/gC13hPocnb4Wq/9INtZo4dfmyjrbs
	zX/m1/Rjf4Pny/D8RBOcVrV0xt+B2fYPpWzl0ZcEbwztC4UL3pDYNvNddehm+ZYIttvI=;
Message-ID: <0625f4f2-daaa-4867-8c39-f8e9ce939ca0@xen.org>
Date: Mon, 13 May 2024 22:28:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] fix Rule 10.2 violation
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com,
 roger.pau@citrix.com, bertrand.marquis@arm.com, michal.orzel@amd.com
References: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

title: Is this the only violation we have in Xen? If so, then please add 
the subsystem in the title.

On 11/05/2024 00:37, Stefano Stabellini wrote:
> Change opt_conswitch to char to fix a violation of Rule 10.2.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> 
> diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
> index 2c363d9c1d..3a3a97bcbe 100644
> --- a/xen/drivers/char/console.c
> +++ b/xen/drivers/char/console.c
> @@ -49,7 +49,7 @@ string_param("console", opt_console);
>   /* Char 1: CTRL+<char1> is used to switch console input between Xen and DOM0 */
>   /* Char 2: If this character is 'x', then do not auto-switch to DOM0 when it */
>   /*         boots. Any other value, or omitting the char, enables auto-switch */
> -static unsigned char __read_mostly opt_conswitch[3] = "a";
> +static char __read_mostly opt_conswitch[3] = "a";

Looking at the rest of the code, we have:

#define switch_code (opt_conswitch[0] - 'a' + 1)

Can you confirm whether this is not somehow adding a new violation?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon May 13 21:55:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 21:55:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721082.1124211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ddM-0005gs-IP; Mon, 13 May 2024 21:55:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721082.1124211; Mon, 13 May 2024 21:55:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ddM-0005gl-Fi; Mon, 13 May 2024 21:55:00 +0000
Received: by outflank-mailman (input) for mailman id 721082;
 Mon, 13 May 2024 21:54:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6ddK-0005gf-Nu
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 21:54:58 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6ddK-0006QJ-FH; Mon, 13 May 2024 21:54:58 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6ddK-0003Vz-94; Mon, 13 May 2024 21:54:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EFaqGHI74Ifgk2yBG2wxZNsF+t3ZzIIpLTtX8JzjiX8=; b=gS07JZdBI+be9hCzVsYU4tP43O
	fMZlcHUo2iYog5++6hEVf9VM8CzDz88ZCDWS/tps7jia0meqLhus14jN+FOqizbam6cVSBNM1Zb6F
	rPp0L4jMF+80q21zuCFwOmiNKBvVURD6Dh8twNZsZuRropIkxaemeQMfB2MLXK54/CHw=;
Message-ID: <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
Date: Mon, 13 May 2024 22:54:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Content-Language: en-GB
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-2-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240425131119.2299629-2-luca.fancellu@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Luca,

On 25/04/2024 14:11, Luca Fancellu wrote:
> Currently the code is listing device tree reserve map regions
> as reserved memory for Xen, but they are not added into
> bootinfo.reserved_mem and they are fetched in multiple places
> using the same code sequence, causing duplication. Fix this
> by adding them to the bootinfo.reserved_mem at early stage.

Do we have enough space in bootinfo.reserved_mem for them?

The rest of the changes LGTM.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon May 13 22:16:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 May 2024 22:16:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721086.1124222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6dyM-0000Vw-7g; Mon, 13 May 2024 22:16:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721086.1124222; Mon, 13 May 2024 22:16:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6dyM-0000Vp-4W; Mon, 13 May 2024 22:16:42 +0000
Received: by outflank-mailman (input) for mailman id 721086;
 Mon, 13 May 2024 22:16:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6dyK-0000Vj-PP
 for xen-devel@lists.xenproject.org; Mon, 13 May 2024 22:16:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6dyK-0006oP-Ds; Mon, 13 May 2024 22:16:40 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6dyK-0004rY-7T; Mon, 13 May 2024 22:16:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=yJGxVBwvFv0P4HkwqjY3DR40Wmm6BCZQc3wHbEFDdJ4=; b=lZsNgEQ2jxhqD0QmK680gSAev9
	bDhs4OQZq4U9r6hqIeNJNjbBXJsiFKIDinKlzv3U1mKJLDQQgQwSCe8vCv0ZuSwKznrB9W+Jtsr22
	Tu16Otd9KUa7WbLGkoI8dqq5k85vo1x/A3v+BWFhP6wuY2Me7RIhETp4XbrQqj/qUYjY=;
Message-ID: <d295c7ea-d3e4-40c2-8704-6f3cc8a6a651@xen.org>
Date: Mon, 13 May 2024 23:16:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH 2/2] xen/arm: Rework dt_unreserved_regions to avoid
 recursion
Content-Language: en-GB
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-3-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240425131119.2299629-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Luca,

On 25/04/2024 14:11, Luca Fancellu wrote:
> The function dt_unreserved_regions is currently using recursion
> to compute the non overlapping ranges of a passed region against
> the reserved memory banks, in the spirit of removing the recursion
> to improve safety and also improve the scalability of the function,
> rework its code to use an iterative algorithm with the same result.
> 
> The function was taking an additional parameter 'first', but given
> the rework and given that the function was always initially called
> with this parameter as zero, remove the parameter and update the
> codebase to reflect the change.

In general, I am in favor to remove the recursion. I have some 
questions/remarks below though.

> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>   xen/arch/arm/include/asm/setup.h        |   8 +-
>   xen/arch/arm/include/asm/static-shmem.h |   7 +-
>   xen/arch/arm/kernel.c                   |   2 +-
>   xen/arch/arm/setup.c                    | 133 ++++++++++++++++--------
>   4 files changed, 96 insertions(+), 54 deletions(-)
> 
> diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
> index fc6967f9a435..24519b9ed969 100644
> --- a/xen/arch/arm/include/asm/setup.h
> +++ b/xen/arch/arm/include/asm/setup.h
> @@ -9,7 +9,12 @@
>   #define MAX_FDT_SIZE SZ_2M
>   
>   #define NR_MEM_BANKS 256
> +
> +#ifdef CONFIG_STATIC_SHM
>   #define NR_SHMEM_BANKS 32
> +#else
> +#define NR_SHMEM_BANKS 0
> +#endif
>   
>   #define MAX_MODULES 32 /* Current maximum useful modules */
>   
> @@ -215,8 +220,7 @@ void create_dom0(void);
>   
>   void discard_initial_modules(void);
>   void fw_unreserved_regions(paddr_t s, paddr_t e,
> -                           void (*cb)(paddr_t ps, paddr_t pe),
> -                           unsigned int first);
> +                           void (*cb)(paddr_t ps, paddr_t pe));
>   
>   size_t boot_fdt_info(const void *fdt, paddr_t paddr);
>   const char *boot_fdt_cmdline(const void *fdt);
> diff --git a/xen/arch/arm/include/asm/static-shmem.h b/xen/arch/arm/include/asm/static-shmem.h
> index 3b6569e5703f..1b7c7ea0e17d 100644
> --- a/xen/arch/arm/include/asm/static-shmem.h
> +++ b/xen/arch/arm/include/asm/static-shmem.h
> @@ -7,11 +7,11 @@
>   #include <asm/kernel.h>
>   #include <asm/setup.h>
>   
> -#ifdef CONFIG_STATIC_SHM
> -
>   /* Worst case /memory node reg element: (addrcells + sizecells) */
>   #define DT_MEM_NODE_REG_RANGE_SIZE ((NR_MEM_BANKS + NR_SHMEM_BANKS) * 4)
>   
> +#ifdef CONFIG_STATIC_SHM
> +
>   int make_resv_memory_node(const struct kernel_info *kinfo, int addrcells,
>                             int sizecells);
>   
> @@ -47,9 +47,6 @@ void shm_mem_node_fill_reg_range(const struct kernel_info *kinfo, __be32 *reg,
>   
>   #else /* !CONFIG_STATIC_SHM */
>   
> -/* Worst case /memory node reg element: (addrcells + sizecells) */
> -#define DT_MEM_NODE_REG_RANGE_SIZE (NR_MEM_BANKS * 4)
> -

I don't understand how this is related to the rest patch. Can you clarify?

>   static inline int make_resv_memory_node(const struct kernel_info *kinfo,
>                                           int addrcells, int sizecells)
>   {
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index 674388fa11a2..ecbeec518754 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -247,7 +247,7 @@ static __init int kernel_decompress(struct bootmodule *mod, uint32_t offset)
>        * Free the original kernel, update the pointers to the
>        * decompressed kernel
>        */
> -    fw_unreserved_regions(addr, addr + size, init_domheap_pages, 0);
> +    fw_unreserved_regions(addr, addr + size, init_domheap_pages);
>   
>       return 0;
>   }
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index c4e5c19b11d6..d737fe56e539 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -204,55 +204,97 @@ static void __init processor_id(void)
>   }
>   
>   static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
> -                                         void (*cb)(paddr_t ps, paddr_t pe),
> -                                         unsigned int first)
> +                                         void (*cb)(paddr_t ps, paddr_t pe))
>   {
> -    const struct membanks *reserved_mem = bootinfo_get_reserved_mem();
> -#ifdef CONFIG_STATIC_SHM
> -    const struct membanks *shmem = bootinfo_get_shmem();
> -    unsigned int offset;
> -#endif
> -    unsigned int i;
> -
>       /*
> -     * i is the current bootmodule we are evaluating across all possible
> -     * kinds.
> +     * The worst case scenario is to have N reserved region ovelapping the
> +     * passed one, so having N+1 regions in the stack
>        */
> -    for ( i = first; i < reserved_mem->nr_banks; i++ )
> -    {
> -        paddr_t r_s = reserved_mem->bank[i].start;
> -        paddr_t r_e = r_s + reserved_mem->bank[i].size;
> -
> -        if ( s < r_e && r_s < e )
> -        {
> -            dt_unreserved_regions(r_e, e, cb, i + 1);
> -            dt_unreserved_regions(s, r_s, cb, i + 1);
> -            return;
> -        }
> -    }
> -
> +    struct {
> +        paddr_t s;
> +        paddr_t e;
> +        unsigned int dict;
> +        unsigned int bank;
> +    } stack[NR_MEM_BANKS + NR_SHMEM_BANKS + 1];

If I am not mistaken, the variable could take up to 7KB of the stack. 
This is quite a bit (Xen stack is 32KB on Arm).

AFAIK, the function is not expected to be called concurrently. So it 
would be best to move to .initdata (by adding static __inidata).

Even with that in mind, do we actually need such a large stack?

> +    const struct membanks *mem_banks[] = {
> +        bootinfo_get_reserved_mem(),
>   #ifdef CONFIG_STATIC_SHM
> -    /*
> -     * When retrieving the corresponding shared memory addresses
> -     * below, we need to index the shmem->bank starting from 0, hence
> -     * we need to use i - reserved_mem->nr_banks.
> -    */
> -    offset = reserved_mem->nr_banks;
> -    for ( ; i - offset < shmem->nr_banks; i++ )
> +        bootinfo_get_shmem(),
> +#endif
> +    };
> +    unsigned int count = 0;
> +    unsigned int i, j;

i and j only seemed to be used within the loop. So can this be only be 
declared within it?

> +
> +    /* Push the initial region into the stack */
> +    stack[count].s = s;
> +    stack[count].e = e;
> +    stack[count].dict = 0;
> +    stack[count].bank = 0;
> +    count++;

You are repeating the 5 lines quite a bit. Can we add a wrapper for it? 
This would make the code a bit easier to read.

> +    while ( count > 0 )
>       {
> -        paddr_t r_s = shmem->bank[i - offset].start;
> -        paddr_t r_e = r_s + shmem->bank[i - offset].size;
> -
> -        if ( s < r_e && r_s < e )
> +        paddr_t b_s, b_e;

Coding style: Please add a newline.

> +        /* Take out last element of the stack */
> +        count--;
> +        i = stack[count].dict;
> +        j = stack[count].bank;
> +        b_s = stack[count].s;
> +        b_e = stack[count].e;
> +        /* Start checking for overlapping banks */
> +        for ( ; i < ARRAY_SIZE(mem_banks); i++ )
>           {
> -            dt_unreserved_regions(r_e, e, cb, i + 1);
> -            dt_unreserved_regions(s, r_s, cb, i + 1);
> -            return;
> +            for ( ; j < mem_banks[i]->nr_banks; j++ )
> +            {
> +                paddr_t r_s = mem_banks[i]->bank[j].start;
> +                paddr_t r_e = r_s + mem_banks[i]->bank[j].size;

As you are reworking the code, I would like us to avoid using 'start + 
size' because it could overflow.

I would prefer if we use either start, size or using 'start + size - 1'.

Also coding style: Newlinen missing


> +
> +                if ( b_s < r_e && r_s < b_e )
> +                {
> +                    /*

The line seems to be misindented compare to the other. Or maybe the rest is?

> +                    * Region is overlapping, push the two non overlap ranges in
> +                    * the stack
> +                    */
> +                    if ( (count + 2) < ARRAY_SIZE(stack) )
> +                    {
> +                        /*
> +                         * Push initial non overlapping part of the region into
> +                         * the stack
> +                         */
> +                        stack[count].s = b_s;
> +                        stack[count].e = r_s;
> +                        stack[count].dict = i;
> +                        stack[count].bank = j + 1;
> +                        count++;
> +                        /*
> +                         * Push last non overlapping part of the region into
> +                         * the stack
> +                         */
> +                        stack[count].s = r_e;
> +                        stack[count].e = b_e;
> +                        stack[count].dict = i;
> +                        stack[count].bank = j + 1;
> +                        count++;
> +                    }
> +                    else
> +                        panic("dt_unreserved_regions: stack space exausted");

Can you document in which circumstances this can happen?

> +
> +                    /*
> +                     * Retrieve the last region pushed to check against other
> +                     * banks, so break the loop through the banks and pop
> +                     * a new region from the stack
> +                     */
> +                    goto regions_pushed;
> +                }
> +            }
> +            j = 0;
>           }
> +        /*
> +         * This point is reached when the region doesn't overlap with any other
> +         * region, so call the callback on it.
> +         */
> +        cb(b_s, b_e);
> + regions_pushed:
>       }
> -#endif
> -
> -    cb(s, e);
>   }
>   
>   /*
> @@ -316,11 +358,10 @@ static bool __init bootmodules_overlap_check(struct bootmodules *bootmodules,
>   }
>   
>   void __init fw_unreserved_regions(paddr_t s, paddr_t e,
> -                                  void (*cb)(paddr_t ps, paddr_t pe),
> -                                  unsigned int first)
> +                                  void (*cb)(paddr_t ps, paddr_t pe))
>   {
>       if ( acpi_disabled )
> -        dt_unreserved_regions(s, e, cb, first);
> +        dt_unreserved_regions(s, e, cb);
>       else
>           cb(s, e);
>   }
> @@ -527,7 +568,7 @@ void __init discard_initial_modules(void)
>                !mfn_valid(maddr_to_mfn(e)) )
>               continue;
>   
> -        fw_unreserved_regions(s, e, init_domheap_pages, 0);
> +        fw_unreserved_regions(s, e, init_domheap_pages);
>       }
>   
>       mi->nr_mods = 0;
> @@ -702,7 +743,7 @@ void __init populate_boot_allocator(void)
>               }
>   #endif
>   
> -            fw_unreserved_regions(s, e, init_boot_pages, 0);
> +            fw_unreserved_regions(s, e, init_boot_pages);
>               s = n;
>           }
>       }

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 00:09:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 00:09:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721095.1124232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6fjL-0007DG-UE; Tue, 14 May 2024 00:09:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721095.1124232; Tue, 14 May 2024 00:09:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6fjL-0007D9-RC; Tue, 14 May 2024 00:09:19 +0000
Received: by outflank-mailman (input) for mailman id 721095;
 Tue, 14 May 2024 00:09:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s6fjJ-0007D1-Oy
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 00:09:17 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31d0078e-1186-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 02:09:15 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 492A0CE0D8B;
 Tue, 14 May 2024 00:09:10 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10D52C113CC;
 Tue, 14 May 2024 00:09:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31d0078e-1186-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715645349;
	bh=GWvmPigmSaXMoOQcMreKkRqhCcCxEac0yoFGbdxfUfM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UtBZ7Y9/pJZiybBO2iqLxZlvvLDeyBhC+kLneBUDClP+tK6j6LAGOJLTrNIJo6TWR
	 LcdT557aktQvQ3WTmOhn9xcFbdaCrJJr8/YHkGT63FN6Ghg+Y/eu1N1owIcpMy7lWv
	 bL+mNi8oTqWa1UNdbYsVxWcm537Mb+IETcz0UlAWEIwC61/V/XUxzIWUfaPd15tqAu
	 k/xn/d14TX/sldHco+ceEAWqFnZ589q6iABuam1uMXz6ACd4CB5ducrPOp5I9GLWNd
	 0h2V2kpZcHtsTCnC84vYUjnCcxLhNAqfpjUj8hLm1unDHIzlIVCfmsc/bX9nWvZtdS
	 06prqIm1xCsOQ==
Date: Mon, 13 May 2024 17:09:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, jbeulich@suse.com, roger.pau@citrix.com, 
    bertrand.marquis@arm.com, michal.orzel@amd.com
Subject: Re: [PATCH] fix Rule 10.2 violation
In-Reply-To: <0625f4f2-daaa-4867-8c39-f8e9ce939ca0@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405131706450.2544314@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop> <0625f4f2-daaa-4867-8c39-f8e9ce939ca0@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 13 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> title: Is this the only violation we have in Xen? If so, then please add the
> subsystem in the title.

The only remaining violations are about the use of the "toupper" macro.
Bugseng is recommending to add a cast to fix those or deviate toupper.


> On 11/05/2024 00:37, Stefano Stabellini wrote:
> > Change opt_conswitch to char to fix a violation of Rule 10.2.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > 
> > diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
> > index 2c363d9c1d..3a3a97bcbe 100644
> > --- a/xen/drivers/char/console.c
> > +++ b/xen/drivers/char/console.c
> > @@ -49,7 +49,7 @@ string_param("console", opt_console);
> >   /* Char 1: CTRL+<char1> is used to switch console input between Xen and
> > DOM0 */
> >   /* Char 2: If this character is 'x', then do not auto-switch to DOM0 when
> > it */
> >   /*         boots. Any other value, or omitting the char, enables
> > auto-switch */
> > -static unsigned char __read_mostly opt_conswitch[3] = "a";
> > +static char __read_mostly opt_conswitch[3] = "a";
> 
> Looking at the rest of the code, we have:
> 
> #define switch_code (opt_conswitch[0] - 'a' + 1)
> 
> Can you confirm whether this is not somehow adding a new violation?

No, this patch is to fix a violation exactly there.


From xen-devel-bounces@lists.xenproject.org Tue May 14 00:24:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 00:24:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721100.1124241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6fxn-0001eK-7z; Tue, 14 May 2024 00:24:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721100.1124241; Tue, 14 May 2024 00:24:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6fxn-0001eD-5H; Tue, 14 May 2024 00:24:15 +0000
Received: by outflank-mailman (input) for mailman id 721100;
 Tue, 14 May 2024 00:24:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s6fxm-0001e7-JW
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 00:24:14 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4949fa2a-1188-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 02:24:12 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C618A610A2;
 Tue, 14 May 2024 00:24:10 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86509C113CC;
 Tue, 14 May 2024 00:24:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4949fa2a-1188-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715646250;
	bh=GTyUx+S4XjZAH9bQEvATf+/466qA4dwSdokCELs0Y1E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ECyGqjk1yxXFD/EtC1w/pC3lmpCR2YNqoKk4/8yTeNIa5XKn5a2a4/JYwzl4pQ0GI
	 DZ58V0lOvSOoAxVR1LkDitzfc5qXwXq3TH0V8YtgE9PXSmD3zkq7SHgCBLXeTD7NfH
	 /1Z8roVBSY2JkPqnohPeFmtR/cnywis7IaP22URgpurZj/SeFa0H8SLNcwuaB/JiWI
	 f9+ZQHEyjkrArtpWn4xuNqzdTViTSE3PGQk8MYsO4i1KBfTHKRRVGY3qt2A7Ai7zF2
	 EHo6jgIpvKrT0Yz/itRz622KtI4AoGBdJEVroGshIyqQ+Yt5RGAXmvrT7IgTBmM2jw
	 q2skhsNUDTw+g==
Date: Mon, 13 May 2024 17:24:08 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Jan Beulich <JBeulich@suse.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>
Subject: Re: [PATCH] x86/cpufreq: Rename cpuid variable/parameters to cpu
In-Reply-To: <20240511194237.2596271-1-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405131723240.2544314@ubuntu-linux-20-04-desktop>
References: <20240511194237.2596271-1-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1955509460-1715646250=:2544314"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1955509460-1715646250=:2544314
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sat, 11 May 2024, Andrew Cooper wrote:
> Various functions have a parameter or local variable called cpuid, but this
> triggers a MISRA R5.3 violation because we also have a function called cpuid()
> which wraps the real CPUID instruction.
> 
> In all these cases, it's a Xen cpu index, which is far more commonly named
> just cpu in our code.
> 
> While adjusting these, fix a couple of other issues:
> 
>  * cpufreq_cpu_init() is on the end of a hypercall (with in-memory parameters,
>    even), making EFAULT the wrong error to use.  Use EOPNOTSUPP instead.
> 
>  * check_est_cpu() is wrong to tie EIST to just Intel, and nowhere else using
>    EIST makes this restriction.  Just check the feature itself, which is more
>    succinctly done after being folded into its single caller.
> 
>  * In powernow_cpufreq_update(), replace an opencoded cpu_online().
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> 
> cpu needs to stay signed for now in set_px_pminfo(), because of get_cpu_id().
> This can be cleaned up by making better use of BAD_APICID, but that's a much
> more involved change.
> ---
>  xen/arch/x86/acpi/cpu_idle.c              | 14 ++++----
>  xen/arch/x86/acpi/cpufreq/cpufreq.c       | 24 +++----------
>  xen/arch/x86/acpi/cpufreq/hwp.c           |  6 ++--
>  xen/arch/x86/acpi/cpufreq/powernow.c      |  6 ++--
>  xen/drivers/cpufreq/cpufreq.c             | 18 +++++-----
>  xen/drivers/cpufreq/utility.c             | 43 +++++++++++------------
>  xen/include/acpi/cpufreq/cpufreq.h        |  6 ++--
>  xen/include/acpi/cpufreq/processor_perf.h |  8 ++---
>  xen/include/xen/pmstat.h                  |  6 ++--
>  9 files changed, 57 insertions(+), 74 deletions(-)
> 
> diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
> index cfce4cc0408f..c8db1aa9913a 100644
> --- a/xen/arch/x86/acpi/cpu_idle.c
> +++ b/xen/arch/x86/acpi/cpu_idle.c
> @@ -1498,14 +1498,14 @@ static void amd_cpuidle_init(struct acpi_processor_power *power)
>          vendor_override = -1;
>  }
>  
> -uint32_t pmstat_get_cx_nr(uint32_t cpuid)
> +uint32_t pmstat_get_cx_nr(unsigned int cpu)
>  {
> -    return processor_powers[cpuid] ? processor_powers[cpuid]->count : 0;
> +    return processor_powers[cpu] ? processor_powers[cpu]->count : 0;
>  }
>  
> -int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
> +int pmstat_get_cx_stat(unsigned int cpu, struct pm_cx_stat *stat)
>  {
> -    struct acpi_processor_power *power = processor_powers[cpuid];
> +    struct acpi_processor_power *power = processor_powers[cpu];
>      uint64_t idle_usage = 0, idle_res = 0;
>      uint64_t last_state_update_tick, current_stime, current_tick;
>      uint64_t usage[ACPI_PROCESSOR_MAX_POWER] = { 0 };
> @@ -1522,7 +1522,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
>          return 0;
>      }
>  
> -    stat->idle_time = get_cpu_idle_time(cpuid);
> +    stat->idle_time = get_cpu_idle_time(cpu);
>      nr = min(stat->nr, power->count);
>  
>      /* mimic the stat when detail info hasn't been registered by dom0 */
> @@ -1572,7 +1572,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
>              idle_res += res[i];
>          }
>  
> -        get_hw_residencies(cpuid, &hw_res);
> +        get_hw_residencies(cpu, &hw_res);
>  
>  #define PUT_xC(what, n) do { \
>          if ( stat->nr_##what >= n && \
> @@ -1613,7 +1613,7 @@ int pmstat_get_cx_stat(uint32_t cpuid, struct pm_cx_stat *stat)
>      return 0;
>  }
>  
> -int pmstat_reset_cx_stat(uint32_t cpuid)
> +int pmstat_reset_cx_stat(unsigned int cpu)
>  {
>      return 0;
>  }
> diff --git a/xen/arch/x86/acpi/cpufreq/cpufreq.c b/xen/arch/x86/acpi/cpufreq/cpufreq.c
> index 2b6ef99678ae..a341f2f02063 100644
> --- a/xen/arch/x86/acpi/cpufreq/cpufreq.c
> +++ b/xen/arch/x86/acpi/cpufreq/cpufreq.c
> @@ -55,17 +55,6 @@ struct acpi_cpufreq_data *cpufreq_drv_data[NR_CPUS];
>  static bool __read_mostly acpi_pstate_strict;
>  boolean_param("acpi_pstate_strict", acpi_pstate_strict);
>  
> -static int check_est_cpu(unsigned int cpuid)
> -{
> -    struct cpuinfo_x86 *cpu = &cpu_data[cpuid];
> -
> -    if (cpu->x86_vendor != X86_VENDOR_INTEL ||
> -        !cpu_has(cpu, X86_FEATURE_EIST))
> -        return 0;
> -
> -    return 1;
> -}
> -
>  static unsigned extract_io(u32 value, struct acpi_cpufreq_data *data)
>  {
>      struct processor_performance *perf;
> @@ -530,7 +519,7 @@ static int cf_check acpi_cpufreq_cpu_init(struct cpufreq_policy *policy)
>          if (cpufreq_verbose)
>              printk("xen_pminfo: @acpi_cpufreq_cpu_init,"
>                     "HARDWARE addr space\n");
> -        if (!check_est_cpu(cpu)) {
> +        if (!cpu_has(c, X86_FEATURE_EIST)) {
>              result = -ENODEV;
>              goto err_unreg;
>          }
> @@ -690,15 +679,12 @@ static int __init cf_check cpufreq_driver_late_init(void)
>  }
>  __initcall(cpufreq_driver_late_init);
>  
> -int cpufreq_cpu_init(unsigned int cpuid)
> +int cpufreq_cpu_init(unsigned int cpu)
>  {
> -    int ret;
> -
>      /* Currently we only handle Intel, AMD and Hygon processor */
>      if ( boot_cpu_data.x86_vendor &
>           (X86_VENDOR_INTEL | X86_VENDOR_AMD | X86_VENDOR_HYGON) )
> -        ret = cpufreq_add_cpu(cpuid);
> -    else
> -        ret = -EFAULT;
> -    return ret;
> +        return cpufreq_add_cpu(cpu);
> +
> +    return -EOPNOTSUPP;
>  }
> diff --git a/xen/arch/x86/acpi/cpufreq/hwp.c b/xen/arch/x86/acpi/cpufreq/hwp.c
> index e61212803e71..59b57a4cef86 100644
> --- a/xen/arch/x86/acpi/cpufreq/hwp.c
> +++ b/xen/arch/x86/acpi/cpufreq/hwp.c
> @@ -506,11 +506,11 @@ static void cf_check hwp_set_misc_turbo(void *info)
>      }
>  }
>  
> -static int cf_check hwp_cpufreq_update(int cpuid, struct cpufreq_policy *policy)
> +static int cf_check hwp_cpufreq_update(unsigned int cpu, struct cpufreq_policy *policy)
>  {
> -    on_selected_cpus(cpumask_of(cpuid), hwp_set_misc_turbo, policy, 1);
> +    on_selected_cpus(cpumask_of(cpu), hwp_set_misc_turbo, policy, 1);
>  
> -    return per_cpu(hwp_drv_data, cpuid)->ret;
> +    return per_cpu(hwp_drv_data, cpu)->ret;
>  }
>  
>  static const struct cpufreq_driver __initconst_cf_clobber
> diff --git a/xen/arch/x86/acpi/cpufreq/powernow.c b/xen/arch/x86/acpi/cpufreq/powernow.c
> index 8a27ee82a5b0..69364e185562 100644
> --- a/xen/arch/x86/acpi/cpufreq/powernow.c
> +++ b/xen/arch/x86/acpi/cpufreq/powernow.c
> @@ -68,12 +68,12 @@ static void cf_check update_cpb(void *data)
>  }
>  
>  static int cf_check powernow_cpufreq_update(
> -    int cpuid, struct cpufreq_policy *policy)
> +    unsigned int cpu, struct cpufreq_policy *policy)
>  {
> -    if (!cpumask_test_cpu(cpuid, &cpu_online_map))
> +    if ( !cpu_online(cpu) )
>          return -EINVAL;
>  
> -    on_selected_cpus(cpumask_of(cpuid), update_cpb, policy, 1);
> +    on_selected_cpus(cpumask_of(cpu), update_cpb, policy, 1);
>  
>      return 0;
>  }
> diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
> index 36c800f4fa39..a74593b70577 100644
> --- a/xen/drivers/cpufreq/cpufreq.c
> +++ b/xen/drivers/cpufreq/cpufreq.c
> @@ -459,21 +459,21 @@ static void print_PPC(unsigned int platform_limit)
>  
>  int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
>  {
> -    int ret=0, cpuid;
> +    int ret=0, cpu;
>      struct processor_pminfo *pmpt;
>      struct processor_performance *pxpt;
>  
> -    cpuid = get_cpu_id(acpi_id);
> -    if ( cpuid < 0 || !perf )
> +    cpu = get_cpu_id(acpi_id);
> +    if ( cpu < 0 || !perf )
>      {
>          ret = -EINVAL;
>          goto out;
>      }
>      if ( cpufreq_verbose )
> -        printk("Set CPU acpi_id(%d) cpuid(%d) Px State info:\n",
> +        printk("Set CPU acpi_id(%d) cpu(%d) Px State info:\n",
>                 acpi_id, cpuid);

This cpuid should be changed as well?

Everything else looks OK to me. You can fix on commit.
--8323329-1955509460-1715646250=:2544314--


From xen-devel-bounces@lists.xenproject.org Tue May 14 00:27:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 00:27:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721106.1124252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6g1N-0002Ew-Mj; Tue, 14 May 2024 00:27:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721106.1124252; Tue, 14 May 2024 00:27:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6g1N-0002Ep-JY; Tue, 14 May 2024 00:27:57 +0000
Received: by outflank-mailman (input) for mailman id 721106;
 Tue, 14 May 2024 00:27:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6g1M-0002Ef-1c; Tue, 14 May 2024 00:27:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6g1L-0001Dd-Tl; Tue, 14 May 2024 00:27:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6g1L-0003Wr-Gk; Tue, 14 May 2024 00:27:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6g1L-0002KS-GD; Tue, 14 May 2024 00:27:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yJvm0iTW9VkAjCgt5CIzq2UDQwKWWfBZOCgFOWF87r8=; b=orisucgla9jNDAxLEyzTzAOHP1
	tWGnSaZW8c5sm8KpAPfyHStDlcWS/hIrjr+ctWYaIPK83SVJZvHLijf6++vBPRJem52VV/4JDq+hR
	1KMjaKkTR0jtmOjz91zVLsb0DTRREAbcNjtA7Ko9JcORmdNVByvfH76zs+QiqEkIG7kQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185990-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185990: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6d1346f1bcbf2724dee8af013cdab9f7b581435b
X-Osstest-Versions-That:
    linux=a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 May 2024 00:27:55 +0000

flight 185990 linux-linus real [real]
flight 185991 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185990/
http://logs.test-lab.xenproject.org/osstest/logs/185991/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt     10 host-ping-check-xen fail pass in 185991-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 185991-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 185991 like 185986
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 185991 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 185991 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 185991 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185986
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185986
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185986
 test-armhf-armhf-xl           8 xen-boot                     fail  like 185986
 test-armhf-armhf-libvirt-vhd  8 xen-boot                     fail  like 185986
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185986
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185986
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6d1346f1bcbf2724dee8af013cdab9f7b581435b
baseline version:
 linux                a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6

Last test of basis   185986  2024-05-12 23:41:44 Z    1 days
Testing same since   185990  2024-05-13 16:43:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Jason J. Herne" <jjherne@linux.ibm.com>
  Abel Vesa <abel.vesa@linaro.org>
  Adam Ford <aford173@gmail.com>
  Akhil R <akhilrajeev@nvidia.com>
  Alain Volmat <alain.volmat@foss.st.com>
  Alexander Egorenkov <egorenar@linux.ibm.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Stein <alexander.stein@ew.tq-group.com> # TQMa9352LA/CA
  Alexandre Torgue <alexandre.torgue@foss.st.com>
  Alexey Minnekhanov <alexeymin@postmarketos.org>
  Alice Guo <alice.guo@nxp.com>
  Alim Akhtar <alim.akhtar@samsung.com>
  Anand Moon <anand@edgeble.ai>
  Andre Przywara <andre.przywara@arm.com>
  Andreas Kemnade <andreas@kemnade.info>
  Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
  Andrew Davis <afd@ti.com>
  André Draszik <andre.draszik@linaro.org>
  Andy Yan <andyshrk@163.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Bambura <jenneron@postmarketos.org>
  Aren Moynihan <aren@peacevolution.org>
  Arnd Bergmann <arnd@arndb.de>
  Arthur Demchenkov <spinal.by@gmail.com>
  Artur Weber <aweber.kernel@gmail.com>
  Arınç ÜNAL <arinc.unal@arinc9.com>
  Balint Dobszay <balint.dobszay@arm.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Bhavya Kapoor <b-kapoor@ti.com>
  Biju Das <biju.das.jz@bp.renesas.com>
  Bjorn Andersson <andersson@kernel.org>
  Bjorn Andersson <quic_bjorande@quicinc.com>
  Boris Brezillon <boris.brezillon@collabora.com>
  Brandon Brnich <b-brnich@ti.com>
  Caleb Connolly <caleb.connolly@linaro.org>
  Carlos Song <carlos.song@nxp.com>
  Catalin Udma <catalin-dan.udma@nxp.com>
  Chen Wang <unicorn_wang@outlook.com>
  Chris Morgan <macromorgan@hotmail.com>
  Christian Borntraeger <borntraeger@linux.ibm.com>
  Christian Marangi <ansuelsmth@gmail.com>
  Christopher Obbard <chris.obbard@collabora.com>
  Chukun Pan <amadeus@jmu.edu.cn>
  Chun-Kuang Hu <chunkuang.hu@kernel.org>
  Ciprian Costea <ciprianmarian.costea@nxp.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Connor Abbott <cwabbott0@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Cristian Marussi <cristian.marussi@arm.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Danzberger <dd@embedd.com>
  Danila Tikhonov <danila@jiaxyga.com>
  Dario Binacchi <dario.binacchi@amarulasolutions.com>
  Dasnavis Sabiya <sabiya.d@ti.com>
  Dave Stevenson <dave.stevenson@raspberrypi.com>
  David Heidelberg <david@ixit.cz>
  David Hildenbrand <david@redhat.com>
  David Jander <david@protonic.nl>
  Denis Burkov <hitechshell@mail.ru>
  Diederik de Haas <didi.debian@cknow.org>
  Dinh Nguyen <dinguyen@kernel.org>
  Diogo Ivo <diogo.ivo@siemens.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Yashin <dmt.yashin@gmail.com>
  Dong Aisheng <aisheng.dong@nxp.com>
  Douglas Anderson <dianders@chromium.org>
  Dragan Simic <dsimic@manjaro.org>
  Duy Nguyen <duy.nguyen.rh@renesas.com>
  Elliot Berman <quic_eberman@quicinc.com>
  Elliot Berman <quic_eberman@quicinc.com> # sm8650-qrd
  Erico Nunes <nunes.erico@gmail.com>
  Fabio Estevam <festevam@denx.de>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Folker Schwesinger <dev@folker-schwesinger.de>
  Francesco Dolcini <francesco.dolcini@toradex.com>
  Frank Li <Frank.Li@nxp.com>
  Frank Oltmanns <frank@oltmanns.dev>
  Gabor Juhos <j4g8y7@gmail.com>
  Gabriel Fernandez <gabriel.fernandez@foss.st.com>
  Garrett Giordano <ggiordano@phytec.com>
  Gatien Chevallier <gatien.chevallier@foss.st.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Ghennadi Procopciuc <ghennadi.procopciuc@nxp.com>
  Ghennadi Procopciuc <ghennadi.procopciuc@oss.nxp.com>
  Gilles Talis <gilles.talis@gmail.com>
  Gregory CLEMENT <gregory.clement@bootlin.com>
  Guo Ren <guoren@kernel.org>
  Haibo Chen <haibo.chen@nxp.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko.stuebner@cherry.de>
  Heiko Stuebner <heiko@sntech.de>
  Hiago De Franco <hiago.franco@toradex.com>
  Holger Dengler <dengler@linux.ibm.com>
  Hugues Fruchet <hugues.fruchet@foss.st.com>
  Hui Liu <quic_huliu@quicinc.com>
  Huisong Li <lihuisong@huawei.com>
  Ian Ray <ian.ray@gehealthcare.com>
  Icenowy Zheng <uwu@icenowy.me>
  Inochi Amaoto <inochiama@outlook.com>
  Ivan T. Ivanov <iivanov@suse.de>
  Jacopo Mondi <jacopo@jmondi.org>
  Jai Luthra <j-luthra@ti.com>
  Jan Kiszka <jan.kiszka@siemens.com>
  Janosch Frank <frankja@linux.ibm.com>
  Jason J. Herne <jjherne@linux.ibm.com>
  Jason-ch Chen <Jason-ch.Chen@mediatek.com>
  Jason-JH.Lin <jason-jh.lin@mediatek.com>
  Jean Delvare <jdelvare@suse.de>
  Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
  Jens Wiklander <jens.wiklander@linaro.org>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jianfeng Liu <liujianfeng1994@gmail.com>
  Jianhua Lu <lujianhua000@gmail.com>
  Jimmy Hon <honyuenkwun@gmail.com>
  Jing Luo <jing@jing.rocks>
  Jisheng Zhang <jszhang@kernel.org>
  Joe Mason <buddyjojo06@outlook.com>
  Johan Hovold <johan+linaro@kernel.org>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Josua Mayer <josua@solid-run.com>
  Joy Zou <joy.zou@nxp.com>
  Judith Mendez <jm@ti.com>
  Jun Nie <jun.nie@linaro.org>
  Justin Stitt <justinstitt@google.com>
  Kathiravan Thirumoorthy <quic_kathirav@quicinc.com> # IPQ9574 and IPQ5332
  Komal Bajaj <quic_kbajaj@quicinc.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Konrad Dybcio <konradybcio@kernel.org>
  Krishna Kurapati <quic_kriskura@quicinc.com>
  Krishna Kurapati PSSNV <quic_kriskura@quicinc.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Li Zhijian <lizhijian@fujitsu.com>
  Ling Xu <quic_lxu5@quicinc.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Loic Poulain <loic.poulain@linaro.org>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Luca Ceresoli <luca.ceresoli@bootlin.com>
  Luca Weiss <luca.weiss@fairphone.com>
  Luca Weiss <luca@z3ntu.xyz>
  Lucas Stach <l.stach@pengutronix.de>
  Luke Wang <ziniu.wang_1@nxp.com>
  Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
  Marek Vasut <marex@denx.de>
  Markus Schneider-Pargmann <msp@baylibre.com>
  Matthew Rosato <mjrosato@linux.ibm.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Maulik Shah <quic_mkshah@quicinc.com>
  Max Krummenacher <max.krummenacher@toradex.com>
  Maxime Ripard <mripard@kernel.org>
  Maximilian Luz <luzmaximilian@gmail.com>
  Michael Grzeschik <m.grzeschik@pengutronix.de>
  Michael Riesch <michael.riesch@wolfvision.net>
  Michael Walle <mwalle@kernel.org>
  Miles Alan <m@milesalan.com>
  Mirela Rabulea <mirela.rabulea@nxp.com>
  Mohammad Shehar Yaar Tausif <sheharyaar48@gmail.com>
  Muhammed Efe Cetin <efectn@protonmail.com>
  Mukesh Ojha <quic_mojha@quicinc.com>
  Nathan Morrisson <nmorrisson@phytec.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nicolas Chauvet <kwizart@gmail.com>
  Niklas Cassel <cassel@kernel.org>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
  Nina Schoetterl-Glausch <nsg@linux.ibm.com>
  Nishanth Menon <nm@ti.com>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Oleksii Moisieiev <Oleksii_Moisieiev@epam.com>
  Ondrej Jirman <megi@xff.cz>
  Palmer Dabbelt <palmer@rivosinc.com>
  Pankaj Gupta <pankaj.gupta@nxp.com>
  Parthiban Nallathambi <parthiban@linumiz.com>
  Patrice Chotard <patrice.chotard@foss.st.com>
  Patrick Delaunay <patrick.delaunay@foss.st.com>
  Paweł Owoc <frut3k7@gmail.com>
  Peng Fan <peng.fan@nxp.com>
  Peter Griffin <peter.griffin@linaro.org>
  Peter Oberparleiter <oberpar@linux.ibm.com>
  Philipp Zabel <p.zabel@pengutronix.de>
  Poovendhan Selvaraj <quic_poovendh@quicinc.com>
  Rafał Miłecki <rafal@milecki.pl>
  Raphael Gallais-Pou <raphael.gallais-pou@foss.st.com>
  Raymond Hackley <raymondhackley@protonmail.com>
  Richard Acayan <mailingradian@gmail.com>
  Ritesh Kumar <quic_riteshk@quicinc.com>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Robert Nelson <robertcnelson@gmail.com>
  Roger Quadros <rogerq@kernel.org>
  Rong Zhang <i@rong.moe>
  Ryan Walklin <ryan@testtoast.com>
  Sam Protsenko <semen.protsenko@linaro.org>
  Sascha Hauer <s.hauer@pengutronix.de>
  Sebastian Raase <linux@sraa.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Sebastian Reichel <sre@kernel.org>
  Shawn Guo <shawnguo@kernel.org>
  Shengjiu Wang <shengjiu.wang@nxp.com>
  Sicelo A. Mhlongo <absicsz@gmail.com>
  Siddharth Manthan <siddharth.manthan@gmail.com>
  Stanislav Jakubek <stano.jakubek@gmail.com>
  Steen Hegelund <Steen.Hegelund@microchip.com>
  Stefan Eichenberger <stefan.eichenberger@toradex.com>
  Stefan Wahren <stefan.wahren@i2se.com>
  Stefan Wahren <wahrenst@gmx.net>
  Stephan Gerhold <stephan@gerhold.net>
  Stephen Boyd <sboyd@kernel.org>
  Stephen Boyd <swboyd@chromium.org>
  Sudeep Holla <sudeep.holla@arm.com>
  Sukrut Bellary <sukrut.bellary@linux.com>
  Sumanth Korikkar <sumanthk@linux.ibm.com>
  Sumit Garg <sumit.garg@linaro.org>
  Sven Schnelle <svens@linux.ibm.com>
  Thanh Le <thanh.le.xv@renesas.com>
  Thanh Quan <thanh.quan.xn@renesas.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Richter <tmricht@linux.ibm.com>
  Tim Harvey <tharvey@gateworks.com>
  Tom Brautaset <tbrautaset@gmail.com>
  Tommaso Merciai <tomm.merciai@gmail.com>
  Tony Lindgren <tony@atomide.com>
  Tudor Ambarus <tudor.ambarus@linaro.org>
  Udipto Goswami <quic_ugoswami@quicinc.com>
  Udit Kumar <u-kumar1@ti.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Umang Chheda <quic_uchheda@quicinc.com>
  Umang Jain <umang.jain@ideasonboard.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Uwe Kleine-König <uwe@kleine-koenig.org>
  Varun Sethi <v.sethi@nxp.com>
  Vasily Gorbik <gor@linux.ibm.com>
  Vignesh Raghavendra <vigneshr@ti.com>
  Vineeth Vijayan <vneethv@linux.ibm.com>
  Vitor Soares <vitor.soares@toradex.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>
  Wadim Mueller <wafgo01@gmail.com>
  wangkaiyuan <wangkaiyuan@inspur.com>
  Wei Fang <wei.fang@nxp.com>
  Wei Xu <xuwei5@hisilicon.com>
  Xilin Wu <wuxilin123@gmail.com> # on QCS8550 AYN Odin 2
  Xu Yang <xu.yang_2@nxp.com>
  Yang Xiwen <forbidden405@outlook.com>
  Yangyu Chen <cyy@cyyself.name>
  Yannick Fertre <yannick.fertre@foss.st.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Zev Weiss <zev@bewilderbeest.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   a38297e3fb01..6d1346f1bcbf  6d1346f1bcbf2724dee8af013cdab9f7b581435b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 14 00:32:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 00:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721114.1124262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6g5X-00040q-B9; Tue, 14 May 2024 00:32:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721114.1124262; Tue, 14 May 2024 00:32:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6g5X-00040j-80; Tue, 14 May 2024 00:32:15 +0000
Received: by outflank-mailman (input) for mailman id 721114;
 Tue, 14 May 2024 00:32:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s6g5V-00040d-Nm
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 00:32:13 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 65fffe14-1189-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 02:32:11 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 93397CE0FA4;
 Tue, 14 May 2024 00:32:06 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5AF4C113CC;
 Tue, 14 May 2024 00:32:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65fffe14-1189-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715646725;
	bh=H78/7RklkbUxxfqv1CXgGuILJwFQYgVzwrybslal0WI=;
	h=Date:From:To:cc:Subject:From;
	b=m6lUVAmZkStl1bWmJ+nltMt0IKuXWwRFgyClzrLxt73uNe08ObIODo/pqqfhXCjp9
	 4fL87XZT62qAAMzYnc72s9feXjLs+9rU8GaJw6UfdA5Dl/yWChdQdjmP6YQH0P3Gsg
	 e5EcxkbkCO/jgB8UFNyhF7OKGpO0VdxSc+T1PGqCKtAkGHe+fIJVNcaSGIfP0I8Iph
	 1g6wSx9pvkE51RXgikQOe6ZOQX9mJ4CG1RJMvQKwhTsFQ7BFLBwYTBER3uTurZpXMM
	 WgvNHNzB1O1nQX+Q0LE+DHLNPI7MnNSfalTkm2VlpgQdIWNjcmWYI8UBzczNTUhN1f
	 2n2+Mql5hhWyA==
Date: Mon, 13 May 2024 17:32:03 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, jbeulich@suse.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>
Subject: [PATCH] lib/strtoul: fix MISRA R10.2 violation
Message-ID: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Fix last violation of R10.2 by casting the result of toupper to plain
char. Note that we don't want to change toupper itself as it is a legacy
interface and it would cause more issues.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
I believe this is the last R10.2 violation

diff --git a/xen/lib/strtoul.c b/xen/lib/strtoul.c
index a378fe735e..345dcf9d8c 100644
--- a/xen/lib/strtoul.c
+++ b/xen/lib/strtoul.c
@@ -38,7 +38,7 @@ unsigned long simple_strtoul(
 
     while ( isxdigit(*cp) &&
             (value = isdigit(*cp) ? *cp - '0'
-                                  : toupper(*cp) - 'A' + 10) < base )
+                                  : (char)toupper(*cp) - 'A' + 10) < base )
     {
         result = result * base + value;
         cp++;


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:02:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:02:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721139.1124273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mAt-0007DG-Dy; Tue, 14 May 2024 07:02:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721139.1124273; Tue, 14 May 2024 07:02:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mAt-0007D9-98; Tue, 14 May 2024 07:02:11 +0000
Received: by outflank-mailman (input) for mailman id 721139;
 Tue, 14 May 2024 07:02:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mAs-0007D3-4x
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:02:10 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e1325311-11bf-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:02:08 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59b178b75bso1016672366b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:02:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b015ecsm694054966b.177.2024.05.14.00.02.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:02:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1325311-11bf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715670128; x=1716274928; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EQRhkDVX1LcDamhTiLZZoZLv/dajDYSw/6ksfLUt5MI=;
        b=PAJtZHtb9Mf7vIRoi45vzixKheWLfZXWyRje4aPjieDP0WvrKyAMPeO69FMIts6cna
         CbIlhklFLLmaKHn3KgbW4bkw2rncok7OhTCzVURbng/dSCdpymJZVd0UO46eBlA4ifOY
         XOcBtzXk0FpN7PaDiqm0GxDPRNYaCuELF0WqrPeL2H4uusvDndx2N+SVoA/5pXH8+F6t
         VXdDFgWKzEqt7KTLFEjdbw/HhJZVG98FNNdZOQAqltrI2OeB287N1hG2Ng+p/QWl3IKI
         6cZIL1f194To6ChRal56OMgarvEflWxUuzF/qgFumerWEHnHo/HwFr950g0HiNhmTO/E
         u8Ew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670128; x=1716274928;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=EQRhkDVX1LcDamhTiLZZoZLv/dajDYSw/6ksfLUt5MI=;
        b=cuuZ7yG7IfrvwAsbWR9E/eOLLmLWIpXZ5wPYhDlAvAzTuPEkAzlhW2cJqpZY5olUhk
         uAy30aXvj68imoLrmN6YD8mYO1br0BsURBQIW8ujQzLnhsYXDrwdMndZUEsIj4dK07nh
         ulZhHlZZObtrqnlR4eauhebJncnvTWSG5j682TXzXpA0iuUtmzVXgQ1v/xBuJ2nDvycl
         GTbtL/fXKWqgXrsjX8yDmFk9ujkJNvFbQiQnKcnCekwh993LQGamkSoV6ZXAJ/TeEYWH
         qIUFSRjrLL4sEDBLBC9bHOp6YuYjuk/ozLFPcC20QOTSTGk/VUIft4pUSspXmlo/w0qF
         yMpA==
X-Forwarded-Encrypted: i=1; AJvYcCVAQeTZ/cdUFHGRWrRHlsSMa2AcY2U2OYaSS5v71K567wf7WyLVah6OoyWyZ/iixChFHh22rHaka0l/uxNVp3uQ5zC7zfOKa/P9ZdA7zp0=
X-Gm-Message-State: AOJu0YxnDbxq2USAh4mApBEW5Ohtb6cjMPGot+3/xK7sfTZqeI6XmPut
	kB2o+nHJ1Aj0NQr/eLDUFXCZbn/5RrzSD3mdm1k7eqIm52UDXP18RIQDVyV6qQ==
X-Google-Smtp-Source: AGHT+IEaG8aj9u+XjBcmid97Z0/lXiRG1Quzk59GzpwLWNNZl2T9uLpkqdu9QGXYOcc9yLEu3U/j5w==
X-Received: by 2002:a17:906:30c7:b0:a59:a431:a8ce with SMTP id a640c23a62f3a-a5a2d55a7c3mr823465866b.2.1715670128172;
        Tue, 14 May 2024 00:02:08 -0700 (PDT)
Message-ID: <79ff7ec2-e4d4-4407-a5a1-98027c0a8c97@suse.com>
Date: Tue, 14 May 2024 09:02:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 04/13] xen/arm: add Dom0 cache coloring support
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240502165533.319988-1-carlo.nonato@minervasys.tech>
 <20240502165533.319988-5-carlo.nonato@minervasys.tech>
 <8fbd50df-6c9a-4177-8001-39c0f15e715e@suse.com>
 <CAG+AhRX3oUiGxwYphQDcKo_W+4V3kzMoDh_NVGQcCBXxYT0YAg@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRX3oUiGxwYphQDcKo_W+4V3kzMoDh_NVGQcCBXxYT0YAg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 17:53, Carlo Nonato wrote:
> Hi Jan,
> 
> On Mon, May 6, 2024 at 2:01 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 02.05.2024 18:55, Carlo Nonato wrote:
>>> Add a command line parameter to allow the user to set the coloring
>>> configuration for Dom0.
>>> A common configuration syntax for cache colors is introduced and
>>> documented.
>>> Take the opportunity to also add:
>>>  - default configuration notion.
>>>  - function to check well-formed configurations.
>>>
>>> Direct mapping Dom0 isn't possible when coloring is enabled, so
>>> CDF_directmap flag is removed when creating it.
>>>
>>> Based on original work from: Luca Miccio <lucmiccio@gmail.com>
>>>
>>> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
>>> Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
>>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>> albeit ...
>>
>>> v8:
>>> - added bound check on dom0_num_colors
>>> - default colors array set just once
>>> v7:
>>> - parse_color_config() doesn't accept leading/trailing commas anymore
>>> - removed alloc_colors() helper
>>> v6:
>>> - moved domain_llc_coloring_free() in this patch
>>
>> ... I'm having trouble seeing why this was done: Dom0 isn't ever cleaned up
>> after, is it?
> 
> domain_destroy() is never called for dom0, right?

Indeed, with perhaps the exception of late hwdom (assuming that feature hasn't
bit-rotted).

Jan

> If so, yes this is wrongly placed here.
> 
> Thanks.
> 
>> Jan



From xen-devel-bounces@lists.xenproject.org Tue May 14 07:05:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:05:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721142.1124281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mEC-0007kH-Qc; Tue, 14 May 2024 07:05:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721142.1124281; Tue, 14 May 2024 07:05:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mEC-0007kA-O1; Tue, 14 May 2024 07:05:36 +0000
Received: by outflank-mailman (input) for mailman id 721142;
 Tue, 14 May 2024 07:05:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mEB-0007k4-Na
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:05:35 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a20f472-11c0-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 09:05:31 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso42982111fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:05:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a178a9d68sm686572166b.80.2024.05.14.00.05.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:05:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a20f472-11c0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715670333; x=1716275133; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5JgMKTvgEP03nckpkq+PCRNqoamnknhD5+kUsUVCa38=;
        b=YAbtiIWC6nFgCOF6XRDRy6UfVd82ZB09tjA2lGe6KNAtAIY7mGrMBgB3tiOE/OksBB
         XUveKCggsaUvfbMpggTC4NpWf4MitJtYe1ZnsIsQyKy4uelajPEdqWvHOYx0u4tO2lNV
         5pUR/jbSPUTMx3fEBH1EWXgUPGPwdzFEeE/luY4O7aQyMYLDnV7HXy/kZggIU3OWo3v1
         qOtZ8EDYbR6NxycfgzaHJTF9wwP2/Z+RZmreFMdWTOdFvaZHmo7GEuLDD0qtMm8t4WIG
         Ufp1re1QwjLofCm4brf+qIjW98E9vq6sbCDDcgYDBBXOHTtyhmmYzpg1/XNPuDGbxySf
         K1cQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670333; x=1716275133;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5JgMKTvgEP03nckpkq+PCRNqoamnknhD5+kUsUVCa38=;
        b=cggmgTPbPsQOorGGPoZsDCk0/lCCbOhqug5iXzhCO/9xM69eeMx0bry7M4OHNsh3qH
         WNRRMf/N/0AgVHMVhqg4L0wdCkayNp6N8h2B0SePTkNqGYoL3+dvaO14dHZmxS/aBgwk
         7Wc/ETMik+RixkF+/DUReU8CJtLQzotryks6dG2y5uEJyo5JCHd9FUd7TDVdpr41DanV
         JMWVYp/iZdl3RM4Mb2v62Z+R+0KyZ1TyqCeyBZxoynHqJlm7kJVAdQ2Otx5LWfknQYzQ
         ayETw7ZsonYBpjoXJ7QC2gQr0nV9OG7m2V4wrHZBkMTFGCxYEDdJWu6d2aqwOyhhsePm
         S8gw==
X-Forwarded-Encrypted: i=1; AJvYcCUX3yP5613XgqNIyqYXBbO+hjX/AEn09OA/NJbW8NG5lzA8XCfeyi+j8ASp7oS+hqdrSr/fhk9irLA7UYiV9Bs444No6WbjDlc0G00/jbc=
X-Gm-Message-State: AOJu0YyvYLX0ZTTfw6YdngHGR90DUowVO0qCRD4yJlbW4uqILDkLJiPP
	7lBsg/p/l8cVIuyYkmz9b3axWngSf/0iIY2x/hrvKwtC9rz18nCzisdxtz+DEHggxgI5gGG6Bek
	=
X-Google-Smtp-Source: AGHT+IE+qypl3TYv3mtDpoooRDRZHZfMcaTzpxUDf6ugVInQxEb7xA23EbQVo3j2vcLFziWw/8rdaA==
X-Received: by 2002:a05:651c:149:b0:2e2:9842:a9d3 with SMTP id 38308e7fff4ca-2e52039dc3emr94524221fa.46.1715670332974;
        Tue, 14 May 2024 00:05:32 -0700 (PDT)
Message-ID: <c8a4ab8e-786a-4e85-80d4-4da0ea5d9129@suse.com>
Date: Tue, 14 May 2024 09:05:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 5/5] x86/MCE: optional build of AMD/Intel MCE code
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714640459.git.Sergiy_Kibrik@epam.com>
 <47d32ecff7b915bd23b6d13b76cedf4b39db71a2.1714640459.git.Sergiy_Kibrik@epam.com>
 <dc14d45e-b2b2-4414-a937-ae95115aa2cd@suse.com>
 <63dc5c56-19e4-405d-a016-03175dd704bf@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <63dc5c56-19e4-405d-a016-03175dd704bf@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 11:11, Sergiy Kibrik wrote:
> 06.05.24 14:32, Jan Beulich:
>> On 02.05.2024 11:21, Sergiy Kibrik wrote:
>>> Also global Intel-specific variables lmce_support & cmci_support have to be
>>> redefined if !INTEL, as they get checked in common code.
>>
>> Them being checked in common code may have different resolution strategies.
>> The justification here imo is that, right now, both variables are only ever
>> written by mce_intel.c. As mentioned for vmce_has_lmce(), there's nothing
>> fundamentally preventing MCG_CAP from having respective bits set on a non-
>> Intel CPU.
>>
> 
> so could these global variables just be moved to common code then? Like 
> arch/x86/cpu/mcheck/mce.c ?

That would likely be a better approach, yes.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:05:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:05:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721143.1124292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mET-00084p-2I; Tue, 14 May 2024 07:05:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721143.1124292; Tue, 14 May 2024 07:05:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mES-00084i-Vk; Tue, 14 May 2024 07:05:52 +0000
Received: by outflank-mailman (input) for mailman id 721143;
 Tue, 14 May 2024 07:05:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6mEQ-00082u-Ub
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:05:51 +0000
Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com
 [2607:f8b0:4864:20::72f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 646b236b-11c0-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:05:49 +0200 (CEST)
Received: by mail-qk1-x72f.google.com with SMTP id
 af79cd13be357-792d65cd7a8so150371385a.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:05:49 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2a45b5sm536329285a.62.2024.05.14.00.05.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 00:05:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 646b236b-11c0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715670348; x=1716275148; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=X40dseCSmhwoma2yJHjVIyCXRybcqL2OUtOW+9ivWME=;
        b=k2ioDzCW8TFGNco5OwO+oStVlchC7gpvApMceJUV5wX0GgqUTWXn+LOmg27N/fqgp6
         wbT5e4y832ls/QuaN9iqEgei9IzTER63anrqUUlcRg1tHPFyBL7Srtm+fQkYTnU14Dvs
         pFJGm+RHIGYyOmqv+lPijbC/JZulzWRFtDZns=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670348; x=1716275148;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=X40dseCSmhwoma2yJHjVIyCXRybcqL2OUtOW+9ivWME=;
        b=qqLN4faJuPX57VkPqF6u/bPUVmqyxpIKo2C/z/HVxgw37f98NPpYVwb9UCpdWgUObg
         cRVjvZVC/Ld85K/H7SAhwBj2K5DNFaxCWBajawrVLO1XHNuBaTIOv+gIGfOT02hgWm9k
         nTBhg3RP7TVyJ6yPcrKJRrnR9D+D7TvQb98t/r3X3V9tfBm3R9Y0XhEVtIbEhpAydndo
         vyaSMfP6cG/8D/T6M6Jtn3oMgKG2gozd51VPFG6s418bQPVkNjbxtWg17ly5U7HtzXUH
         4KFAKeqkeKvclnT2xQzLFLYM+/DPg5OESh8kjwMDQ6fKDSahPlcaiypjIL3CdnYZU5dx
         F1PA==
X-Gm-Message-State: AOJu0YxxagD7yLKieR5pfBYOcU4xBtbOL1W/BxA3hgdcslH164+7OaZs
	SJb1Adc7ExAg2eVo/BXr8h6Lmcv1sSW7W10eKJFYA5Ur7YC5i/cEYy+w2SFu0jWH6d7nSyvNZeV
	o
X-Google-Smtp-Source: AGHT+IG+j0uq98JnoZuVro5sSe8Tk8ar6MYifv+ydA3DMqQUXNAwwAKxRNLXZqBdyjxsMy8j49vqZw==
X-Received: by 2002:ae9:f819:0:b0:792:8d94:2bfa with SMTP id af79cd13be357-792c75ff9abmr1263266485a.71.1715670348026;
        Tue, 14 May 2024 00:05:48 -0700 (PDT)
Date: Tue, 14 May 2024 09:05:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH] x86/pv: Sanity check bytes_per_rep in rep_outs()
Message-ID: <ZkMNSbcB9S4YYcfy@macbook>
References: <20240511151642.2476555-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240511151642.2476555-1-andrew.cooper3@citrix.com>

On Sat, May 11, 2024 at 04:16:42PM +0100, Andrew Cooper wrote:
> I was playing with GCC-14, and gave the new static analyser (-fanalyze) a
> try.  One of the more serious things it came back with was this:
> 
>   In file included from ./arch/x86/include/asm/mm.h:10,
>                    from ./include/xen/mm.h:233,
>                    from ./include/xen/domain_page.h:12,
>                    from arch/x86/pv/emul-priv-op.c:10:
>   In function '__copy_from_guest_pv',
>       inlined from 'rep_outs' at arch/x86/pv/emul-priv-op.c:718:20:
>   ./arch/x86/include/asm/uaccess.h:174:9: warning: stack-based buffer overflow [CWE-121] [-Wanalyzer-out-of-bounds]
>     174 |         __asm__ __volatile__(                                           \
>         |         ^~~~~~~
>   ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
>     229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>         |             ^~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
>     236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
>         |     ^~~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
>     308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
>         |             ^~~~~~~~~~~~~~
>     'rep_outs': events 1-3
>       |
>       |arch/x86/pv/emul-priv-op.c:674:21:
>       |  674 | static int cf_check rep_outs(
>       |      |                     ^~~~~~~~
>       |      |                     |
>       |      |                     (1) entry to 'rep_outs'
>       |......
>       |  688 |     if ( !guest_io_okay(port, bytes_per_rep, curr, ctxt->regs) )
>       |      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |           |
>       |      |           (3) calling 'guest_io_okay' from 'rep_outs'
>       |......
>       |  710 |         unsigned int data = 0;
>       |      |                      ~~~~
>       |      |                      |
>       |      |                      (2) capacity: 4 bytes
>       |
>       +--> 'guest_io_okay': events 4-5
>              |
>              |  159 | static bool guest_io_okay(unsigned int port, unsigned int bytes,
>              |      |             ^~~~~~~~~~~~~
>              |      |             |
>              |      |             (4) entry to 'guest_io_okay'
>              |......
>              |  165 |     if ( iopl_ok(v, regs) )
>              |      |          ~~~~~~~~~~~~~~~~
>              |      |          |
>              |      |          (5) calling 'iopl_ok' from 'guest_io_okay'
>              |
>              +--> 'iopl_ok': event 6
>                     |
>                     |  148 | static bool iopl_ok(const struct vcpu *v, const struct cpu_user_regs *regs)
>                     |      |             ^~~~~~~
>                     |      |             |
>                     |      |             (6) entry to 'iopl_ok'
>                     |
>                   'iopl_ok': event 7
>                     |
>                     |./include/xen/bug.h:141:13:
>                     |  141 |     do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
>                     |      |             ^
>                     |      |             |
>                     |      |             (7) following 'false' branch...
>   arch/x86/pv/emul-priv-op.c:153:5: note: in expansion of macro 'ASSERT'
>                     |  153 |     ASSERT((v->arch.pv.iopl & ~X86_EFLAGS_IOPL) == 0);
>                     |      |     ^~~~~~
>                     |
>                   'iopl_ok': event 8
>                     |
>                     |./include/xen/bug.h:124:31:
>                     |  124 | #define assert_failed(msg) do {                                 \
>                     |      |                               ^
>                     |      |                               |
>                     |      |                               (8) ...to here
>   ./include/xen/bug.h:141:32: note: in expansion of macro 'assert_failed'
>                     |  141 |     do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
>                     |      |                                ^~~~~~~~~~~~~
>   arch/x86/pv/emul-priv-op.c:153:5: note: in expansion of macro 'ASSERT'
>                     |  153 |     ASSERT((v->arch.pv.iopl & ~X86_EFLAGS_IOPL) == 0);
>                     |      |     ^~~~~~
>                     |
>              <------+
>              |
>            'guest_io_okay': event 9
>              |
>              |  165 |     if ( iopl_ok(v, regs) )
>              |      |          ^~~~~~~~~~~~~~~~
>              |      |          |
>              |      |          (9) returning to 'guest_io_okay' from 'iopl_ok'
>              |
>       <------+
>       |
>     'rep_outs': events 10-13
>       |
>       |  688 |     if ( !guest_io_okay(port, bytes_per_rep, curr, ctxt->regs) )
>       |      |        ~  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |        |  |
>       |      |        |  (10) returning to 'rep_outs' from 'guest_io_okay'
>       |      |        (11) following 'true' branch...
>       |......
>       |  691 |     rc = read_segment(seg, &sreg, ctxt);
>       |      |     ~~   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |     |    |
>       |      |     |    (13) calling 'read_segment' from 'rep_outs'
>       |      |     (12) ...to here
>       |
>       +--> 'read_segment': events 14-18
>              |
>              |  493 | static int cf_check read_segment(
>              |      |                     ^~~~~~~~~~~~
>              |      |                     |
>              |      |                     (14) entry to 'read_segment'
>              |......
>              |  510 |     if ( ctxt->addr_size < 64 )
>              |      |        ~
>              |      |        |
>              |      |        (15) following 'false' branch...
>              |......
>              |  535 |         switch ( seg )
>              |      |         ~~~~~~
>              |      |         |
>              |      |         (16) ...to here
>              |......
>              |  538 |             if ( !is_x86_user_segment(seg) )
>              |      |                ~
>              |      |                |
>              |      |                (17) following 'false' branch (when 'seg <= 5')...
>              |  539 |                 return X86EMUL_UNHANDLEABLE;
>              |  540 |             reg->base = 0;
>              |      |             ~~~
>              |      |             |
>              |      |             (18) ...to here
>              |
>       <------+
>       |
>     'rep_outs': events 19-30
>       |
>       |  691 |     rc = read_segment(seg, &sreg, ctxt);
>       |      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |          |
>       |      |          (19) returning to 'rep_outs' from 'read_segment'
>       |  692 |     if ( rc != X86EMUL_OKAY )
>       |      |        ~
>       |      |        |
>       |      |        (20) following 'false' branch (when 'rc == 0')...
>       |......
>       |  695 |     if ( !sreg.p )
>       |      |     ~~ ~
>       |      |     |  |
>       |      |     |  (22) following 'false' branch...
>       |      |     (21) ...to here
>       |  696 |         return X86EMUL_UNHANDLEABLE;
>       |  697 |     if ( !sreg.s ||
>       |      |     ~~ ~ ~~~~~~~~~~
>       |      |     |  |         |
>       |      |     |  |         (26) following 'false' branch...
>       |      |     |  (24) following 'false' branch...
>       |      |     (23) ...to here
>       |  698 |          ((sreg.type & (_SEGMENT_CODE >> 8)) &&
>       |      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |                |
>       |      |                (25) ...to here
>       |  699 |           !(sreg.type & (_SEGMENT_WR >> 8))) )
>       |      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |......
>       |  706 |     poc->bpmatch = check_guest_io_breakpoint(curr, port, bytes_per_rep);
>       |      |     ~~~
>       |      |     |
>       |      |     (27) ...to here
>       |  707 |
>       |  708 |     while ( *reps < goal )
>       |      |             ~~~~~~~~~~~~
>       |      |                   |
>       |      |                   (28) following 'true' branch...
>       |  709 |     {
>       |  710 |         unsigned int data = 0;
>       |      |         ~~~~~~~~
>       |      |         |
>       |      |         (29) ...to here
>       |......
>       |  713 |         rc = pv_emul_virt_to_linear(sreg.base, offset, bytes_per_rep,
>       |      |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |              |
>       |      |              (30) calling 'pv_emul_virt_to_linear' from 'rep_outs'
>       |  714 |                                     sreg.limit, seg, ctxt, &addr);
>       |      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |
>       +--> 'pv_emul_virt_to_linear': events 31-33
>              |
>              |  580 | static int pv_emul_virt_to_linear(unsigned long base, unsigned long offset,
>              |      |            ^~~~~~~~~~~~~~~~~~~~~~
>              |      |            |
>              |      |            (31) entry to 'pv_emul_virt_to_linear'
>              |......
>              |  596 |     else if ( !__addr_ok(*addr) )
>              |      |             ~
>              |      |             |
>              |      |             (32) following 'false' branch...
>              |......
>              |  603 |     return rc;
>              |      |     ~~~~~~
>              |      |     |
>              |      |     (33) ...to here
>              |
>       <------+
>       |
>     'rep_outs': events 34-37
>       |
>       |  713 |         rc = pv_emul_virt_to_linear(sreg.base, offset, bytes_per_rep,
>       |      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |      |              |
>       |      |              (34) returning to 'rep_outs' from 'pv_emul_virt_to_linear'
>       |  714 |                                     sreg.limit, seg, ctxt, &addr);
>       |      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |  715 |         if ( rc != X86EMUL_OKAY )
>       |      |            ~
>       |      |            |
>       |      |            (35) following 'false' branch (when 'rc == 0')...
>       |......
>       |  718 |         if ( (rc = __copy_from_guest_pv(&data, (void __user *)addr,
>       |      |         ~~         ~
>       |      |         |          |
>       |      |         |          (37) inlined call to '__copy_from_guest_pv' from 'rep_outs'
>       |      |         (36) ...to here
>       |
>       +--> '__copy_from_guest_pv': events 38-41
>              |
>              |./arch/x86/include/asm/uaccess.h:294:8:
>              |  294 |     if (__builtin_constant_p(n)) {
>              |      |        ^
>              |      |        |
>              |      |        (38) following 'true' branch...
>              |  295 |         unsigned long ret;
>              |      |         ~~~~~~~~
>              |      |         |
>              |      |         (39) ...to here
>              |  296 |
>              |  297 |         switch (n) {
>              |      |         ~~~~~~
>              |      |         |
>              |      |         (40) following 'case 8:' branch...
>              |......
>              |  307 |         case 8:
>              |      |         ~~~~
>              |      |         |
>              |      |         (41) ...to here
>              |
>            '__copy_from_guest_pv': event 42
>              |
>              |  174 |         __asm__ __volatile__(                                           \
>              |      |         ^~~~~~~
>              |      |         |
>              |      |         (42) out-of-bounds write from byte 4 till byte 7 but 'data' ends at byte 4
>   ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
>              |  229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>              |      |             ^~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
>              |  236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
>              |      |     ^~~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
>              |  308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
>              |      |             ^~~~~~~~~~~~~~
>              |
>   ./arch/x86/include/asm/uaccess.h:174:9: note: write of 4 bytes to beyond the end of 'data'
>     174 |         __asm__ __volatile__(                                           \
>         |         ^~~~~~~
>   ./arch/x86/include/asm/uaccess.h:229:13: note: in expansion of macro 'get_unsafe_asm'
>     229 |     case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>         |             ^~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:236:5: note: in expansion of macro 'get_unsafe_size'
>     236 |     get_unsafe_size(x, ptr, size, UA_KEEP, retval, errret)
>         |     ^~~~~~~~~~~~~~~
>   ./arch/x86/include/asm/uaccess.h:308:13: note: in expansion of macro 'get_guest_size'
>     308 |             get_guest_size(*(uint64_t *)to, from, 8, ret, 8);
>         |             ^~~~~~~~~~~~~~
> 
>     ┌──────────────────────────────────────────────────────────────────────┐
>     │                    write of 'uint64_t' (8 bytes)                     │
>     └──────────────────────────────────────────────────────────────────────┘
>                      │                                   │
>                      │                                   │
>                      v                                   v
>     ┌──────────────────────────────────┐┌──────────────────────────────────┐
>     │  'data' (type: 'unsigned int')   ││        after valid range         │
>     └──────────────────────────────────┘└──────────────────────────────────┘
>     ├────────────────┬─────────────────┤├────────────────┬─────────────────┤
>                      │                                   │
>             ╭────────┴────────╮              ╭───────────┴──────────╮
>             │capacity: 4 bytes│              │⚠️  overflow of 4 bytes│
>             ╰─────────────────╯              ╰──────────────────────╯
> 
> What is happening is that it's seen that __copy_from_guest_pv() has a case for
> 8, and it doesn't have any information about bytes_per_rep which is an input
> to the function.
> 
> Architecturally, there's no 64-bit variant of OUTS in x86.  Leaving an
> assertion to this effect is enough to satisfy the analyser.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> 
> A little RFC, but the analyser is right - if rep_outs() were to be called with
> anything more than 4, the stack would be clobbered, and no developer is
> finding that bug the easy way...
> ---
>  xen/arch/x86/pv/emul-priv-op.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
> index f101510a1bab..1460ecc187be 100644
> --- a/xen/arch/x86/pv/emul-priv-op.c
> +++ b/xen/arch/x86/pv/emul-priv-op.c
> @@ -685,6 +685,8 @@ static int cf_check rep_outs(
>  
>      *reps = 0;
>  
> +    ASSERT(bytes_per_rep <= 4); /* i.e. 'data' being 4 bytes is fine. */

Don't we need this to be a BUG_ON() to satisfy the compiler also on
non-debug builds?

Or maybe:

if ( bytes_per_rep > 4 )
{
    ASSERT_UNREACHABLE();
    return X86EMUL_UNHANDLEABLE;
}

Would it be possible to add the check to guest_io_okay() itself?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:13:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:13:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721151.1124302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mLI-0001ld-TZ; Tue, 14 May 2024 07:12:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721151.1124302; Tue, 14 May 2024 07:12:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mLI-0001lW-Qj; Tue, 14 May 2024 07:12:56 +0000
Received: by outflank-mailman (input) for mailman id 721151;
 Tue, 14 May 2024 07:12:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mLH-0001lQ-J5
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:12:55 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6209a143-11c1-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:12:54 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso893067266b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:12:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17727sm682928166b.208.2024.05.14.00.12.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:12:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6209a143-11c1-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715670774; x=1716275574; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=t0iOPTvXAtHc4JjAuNcXyELg+l99+IOlHziW8W1joNA=;
        b=RUnhd9CsHfKn2K0aqe+H85GB8kOM/S6ArU2loJujspB6WAqP7p4Y/rFYqWZLMPRgRV
         RishIXuuNMJNAsX1KhAqOGz9NkMqff6urt858dY+FX1tBEte3q978Zjl7O0oWjwle+vk
         TrTXXPT/tXN6hjRAt0DSs6thooq7+fS6e3mwV2UmtS0fb26dgGc19sPjKZZDSFjAPOts
         1M8IHIA0h/Fy2YzDlM1KB+VzhDEOEonFdumYpOrjktucfdct9PKm3ibLYx6xPPJaXOBt
         vOs2Xpqo/t2vCffU98bA8yfKGR+5yMVRI/XHQ0PbJxgVztEZhB/JrdDWQaCruMiG1uR6
         qHVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670774; x=1716275574;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=t0iOPTvXAtHc4JjAuNcXyELg+l99+IOlHziW8W1joNA=;
        b=fsgJt8wb0WcCut0FOmsorpUAAQlhBPigCffRI10a0M+d7tEcq7F35nnsq22nitMkyV
         jEoQHbxumQAhPBz/pvKXOZGvogeWC+nmLGUlvLM9z+X8BVTXlYLZy83utlzP4MXu3jY4
         dR+84m3LZ8qa8r/ZBP6sHai0/7STVf0NHVosBl0ZlRFxv5BvsL8tbOljLZ6joDxiLwJP
         S6WOo+iLusGi61y+EMv2oECDKtuAmKgA1yihZwYxBU29Hvnsn5BkencFqeQiVT650s5+
         alXD7jbO/7fX6mR7Js6XZwmMUu+oz+Z8VSp7vDYsC2fetZzMP/RUxsN3bzjOQunOa08B
         IHMw==
X-Gm-Message-State: AOJu0YzZDR5KY1Q0Jd1gfS0kUtR1yjpUzeUEuGLqjQwa/PxaovfsaKnh
	xfN4D9yAd1yqyqdwb+iQUo4oImj3b3h7BMF1u3s5sGw9B3gClwx3roTslQqIfw==
X-Google-Smtp-Source: AGHT+IHHRQunVeudEypV59NPLMxwwJLUN/GSMWqxEeDbGmkKsU0kAUNYtZP86lHwUHPtmGnqZrv1kw==
X-Received: by 2002:a17:906:7f83:b0:a5a:8673:cccd with SMTP id a640c23a62f3a-a5a8673ce69mr80131366b.30.1715670773851;
        Tue, 14 May 2024 00:12:53 -0700 (PDT)
Message-ID: <2520da93-8b84-4a09-b944-fa065db03c18@suse.com>
Date: Tue, 14 May 2024 09:13:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Referencing domain struct from interrupt handler
Content-Language: en-US
To: Jens Wiklander <jens.wiklander@linaro.org>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 George Dunlap <george.dunlap@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
 <b965ee57-c6fc-459f-a5fd-fae47dc6ea9d@suse.com>
 <CAHUa44FsFi0F4tz3jN+d3WkR4dTPJ1HdUru+ME1YQyzMSbMG7Q@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAHUa44FsFi0F4tz3jN+d3WkR4dTPJ1HdUru+ME1YQyzMSbMG7Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.05.2024 09:10, Jens Wiklander wrote:
> On Fri, May 3, 2024 at 12:32 PM Jan Beulich <jbeulich@suse.com> wrote:
>> Furthermore, is it guaranteed that the IRQ handler won't interrupt code
>> fiddling with the domain list? I don't think it is, since
>> domlist_update_lock isn't acquired in an IRQ-safe manner. Looks like
>> you need to defer the operation on the domain until softirq or tasklet
>> context.
> 
> Thanks for the suggestion, I'm testing it as:
> static DECLARE_TASKLET(notif_sri_tasklet, notif_sri_action, NULL);
> 
> static void notif_irq_handler(int irq, void *data)
> {
>     tasklet_schedule(&notif_sri_tasklet);
> }
> 
> Where notif_sri_action() does what notif_irq_handler() did before
> (using rcu_lock_domain_by_id()).
> 
> I have one more question regarding this.
> 
> Even with the RCU lock if I understand it correctly, it's possible for
> domain_kill() to tear down the domain. Or as Julien explained it in
> another thread [3]:
>> CPU0: ffa_get_domain_by_vm_id() (return the domain as it is alive)
>>
>> CPU1: call domain_kill()
>> CPU1: teardown is called, free d->arch.tee (the pointer is not set to NULL)
>>
>> d->arch.tee is now a dangling pointer
>>
>> CPU0: access d->arch.tee
>>
>> This implies you may need to gain a global lock (I don't have a better
>> idea so far) to protect the IRQ handler against domains teardown.
> 
> I'm trying to address that (now in a tasklet) with:
>     /*
>      * domain_kill() calls ffa_domain_teardown() which will free
>      * d->arch.tee, but not set it to NULL. This can happen while holding
>      * the RCU lock.
>      *
>      * domain_lock() will stop rspin_barrier() in domain_kill(), unless
>      * we're already past rspin_barrier(), but then will d->is_dying be
>      * non-zero.
>      */
>     domain_lock(d);
>     if ( !d->is_dying )
>     {
>         struct ffa_ctx *ctx = d->arch.tee;
> 
>         ACCESS_ONCE(ctx->notif.secure_pending) = true;
>     }
>     domain_unlock(d);
> 
> It seems to work, but I'm worried I'm missing something or abusing
> domain_lock().

Well. Yes, this is one way of dealing with the issue. Yet as you suspect it
feels like an abuse of domain_lock(); that function would better be avoided
whenever possible. (It had some very unhelpful uses long ago.)

Another approach would generally be to do respective cleanup not from
underneath domain_kill(), but complete_domain_destroy(). It's not really
clear to me which of the two approaches is better in this case.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:14:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:14:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721154.1124311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mMT-0002HS-7b; Tue, 14 May 2024 07:14:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721154.1124311; Tue, 14 May 2024 07:14:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mMT-0002HL-4E; Tue, 14 May 2024 07:14:09 +0000
Received: by outflank-mailman (input) for mailman id 721154;
 Tue, 14 May 2024 07:14:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6mMS-0002HB-1Y
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:14:08 +0000
Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com
 [2607:f8b0:4864:20::a2d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d2ceef4-11c1-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:14:07 +0200 (CEST)
Received: by mail-vk1-xa2d.google.com with SMTP id
 71dfb90a1353d-4df2d7dca64so1814223e0c.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:14:07 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf27519esm537383085a.1.2024.05.14.00.14.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 00:14:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d2ceef4-11c1-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715670846; x=1716275646; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Zh1RffHNa9wwlkrdpu7DnVE6Qo+VeFhaWR5xsFAQZaI=;
        b=MCHg5V/+IR9/RY8ha3LcKyDI11TjswT4ZAJZJiWU5I1RK4IMEuWATq4KO0v6+LZJ51
         WRW71diUlv9HG14cvyw2o8iyK4xs0maz3ZueGVMr8E/u9jxN5Aq8nRCxseQQnYVGaTni
         jm9oyX9mthyZKtXmmQ+Ulh1RL/ef4VweatKbM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670846; x=1716275646;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Zh1RffHNa9wwlkrdpu7DnVE6Qo+VeFhaWR5xsFAQZaI=;
        b=vF9SVR3zgefKNZ0vCC6ZUoycdN7H/de/vytN+6Cj8QS9avo5H98y4r+1lbRNTJm6Q2
         UqfDR0AzkxNqH8amXvWgaWZRoQgL/QHBxn3XRJyleBJnIxVEEy2PyFVagY6Lmn2oRHy2
         nJjXSNottNMkjQizsbRued4O1FkIPsbwwp0NoIolvPhaIKORAzXMiy7Sr7eHMEwbSu6h
         rtWY08iGoJRCPKbNkSL5Dvsvr/EtezoAcYgMxyB8LAb44bSNxrvAuZe4hqmcZJKG2Eje
         x8+Rr7FHBQal9rDiv8RPRt1CMZmCrcPBcI2W3xlAlhgsaB9jB9MaQHFL7Oo6BtREDTmx
         XP3g==
X-Gm-Message-State: AOJu0YwZ7Y8Vbxt9wM4ZeAZcfXdgXfanP6WudwZ3svdeP0jX+NLNOQXi
	04iwPAEEQa2oiDfSJMAuvzn6pJYvFa+8Jy4eGq3RruFI62Yd6lWQzamgKyDdLRc=
X-Google-Smtp-Source: AGHT+IE6zKyOOEa4jemmAxyAmUQImoNPSLLz+QQK2RoExEXXd4YpR78cj5OTGjMXPuL3P7qpXT0GgA==
X-Received: by 2002:a05:6122:178f:b0:4cd:20ea:35aa with SMTP id 71dfb90a1353d-4df882d442cmr9958340e0c.8.1715670845954;
        Tue, 14 May 2024 00:14:05 -0700 (PDT)
Date: Tue, 14 May 2024 09:14:03 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH 1/4] x86/gen-cpuid: Minor cleanup
Message-ID: <ZkMPOxF2aR6wp1Vj@macbook>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-2-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240510224002.2324578-2-andrew.cooper3@citrix.com>

On Fri, May 10, 2024 at 11:39:59PM +0100, Andrew Cooper wrote:
> Rename INIT_FEATURE_NAMES to INIT_FEATURE_NAME_TO_VAL as we're about to gain a
> inverse mapping of the same thing.
> 
> Use dict.items() unconditionally.  iteritems() is a marginal perf optimsiation
> for Python2 only, and simply not worth the effort on a script this small.

My understanding is that what used to be iteritems() in Python 2 is
the behavior of items() in Python 3 (return a generator instead of a
copy of the dictionary list).

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:15:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:15:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721156.1124321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mNO-0002nV-FC; Tue, 14 May 2024 07:15:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721156.1124321; Tue, 14 May 2024 07:15:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mNO-0002nO-Cf; Tue, 14 May 2024 07:15:06 +0000
Received: by outflank-mailman (input) for mailman id 721156;
 Tue, 14 May 2024 07:15:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6mNN-0002nC-FX
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:15:05 +0000
Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com
 [2607:f8b0:4864:20::829])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ad85d7b8-11c1-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 09:15:01 +0200 (CEST)
Received: by mail-qt1-x829.google.com with SMTP id
 d75a77b69052e-43de92e234dso48584641cf.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:15:03 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e1a021d02sm21910431cf.52.2024.05.14.00.15.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 00:15:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad85d7b8-11c1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715670902; x=1716275702; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=LFx6pZn3u4Vj7n1yf/nEddnYd4CXuOKYOAOdot+NcLQ=;
        b=gtpxeCrwqgA2k5uPnKD1MvX2uFVSaYj+aCB2eKBAC4Y65xg3t5AeZPy4UGzC+7FbtJ
         Q2a0/Iw2YzM1q+VyB+iXMMF88oNCwn257q9dhFVvAzem3GzkMp7PMPZT0wlWWne7qIV5
         o5RFap2l0ef059BwF9Iwd8cZ+YFVJzQ8evPGE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715670902; x=1716275702;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LFx6pZn3u4Vj7n1yf/nEddnYd4CXuOKYOAOdot+NcLQ=;
        b=K0CFNKbaY7hd1UrgJ91UDx901dlKHSPqcz7i+MS5Qxl1mYDtagL8EW8w4Fczo+q15Q
         rC9fKMQQvNodofMaspbkRAffggD+xn/mZlLA/Ev+ZtEBjqo1kqRAe8JgLF0ynfmFU9E6
         W5WfxEJh0BXbLvaqfYby3Eg/vpCUq9xydXDuBkJqftFmIIGUfMquTzsCT84C3faKG7I+
         9Zmp4ZqdkUm455IBPq1TqgJrO8lq2bCSU3BhNK9IsaNKt5bTelT+9OJz5+k4SdUKryoJ
         TSOL6zuAzPIU9huvQDlHIuPxU7be2wb8gMeqQWLro7HixGgdhCEkEUuI4eExMQ0TtHLp
         0iew==
X-Gm-Message-State: AOJu0YzVhdEiSAfST6BZnrsYMVexFaUx54ALEHjxCccl9dSeXrIDIFIM
	C5ACRkYECsaehFJ9Zarj8x6s2CGvPLB/M5QoWSwcAdwi06TBC2fRgeiWshyqKtU=
X-Google-Smtp-Source: AGHT+IERNIrlNUndQqjXiDTBOS2biumSMHZcG4cX/jF/jARKhf+wWNGmllMPn/Hziv0n7HWkdne8/w==
X-Received: by 2002:ac8:5f4f:0:b0:43a:71be:f3ae with SMTP id d75a77b69052e-43dec39bf67mr287764001cf.29.1715670902219;
        Tue, 14 May 2024 00:15:02 -0700 (PDT)
Date: Tue, 14 May 2024 09:14:59 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH 2/4] tools/xen-cpuid: Rename decodes[] to leaf_info[]
Message-ID: <ZkMPc5puu9iCgDQU@macbook>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-3-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240510224002.2324578-3-andrew.cooper3@citrix.com>

On Fri, May 10, 2024 at 11:40:00PM +0100, Andrew Cooper wrote:
> From: Roger Pau Monné <roger.pau@citrix.com>
> 
> Split out of subsequent patch to aid legibility.

Maybe add: "No functional change intended".

> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:17:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:17:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721161.1124332 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mPx-0003PH-Sw; Tue, 14 May 2024 07:17:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721161.1124332; Tue, 14 May 2024 07:17:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mPx-0003PA-Pa; Tue, 14 May 2024 07:17:45 +0000
Received: by outflank-mailman (input) for mailman id 721161;
 Tue, 14 May 2024 07:17:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mPw-0003P4-Pz
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:17:44 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0cba3848-11c2-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 09:17:40 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so1075249966b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:17:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17814sm684420766b.206.2024.05.14.00.17.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:17:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cba3848-11c2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715671062; x=1716275862; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nOIhA9WGutZG1eFFZwMGqwKB0Jm2CMic0qQnHRgCzU8=;
        b=BfHbGSTcw76Mr+JR5tfUT9ZWifYsSQMsdtBYEyC7Sj7/ZvpGoSjhhWAv9wc8ph2tjx
         jrzeSSeF0v55MbF+YkYBaU68ni90FKCeGu8K7pGK378hIwzkDLeRlYAGAq+Tzssv7HgP
         VExA2fxhp8oC+bPnyUZ7KFq4QCWlsAYssSl0vZHTdyAzZep4wlZmJ83DOFlFc9mnJyQe
         M0wNyIDRkiBckHhpzDGmrQwmpzKXOAEkzW4JGDLyBugUFz2bKeHxg4iYpvQmg3Y0ZVuL
         X/IOAxJwqg+fO6Ul0VYgEgIKIVYxw+FXm2pPzXwsCbZUJC42pA/hNMI5yosyo8cxHJWc
         OZqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715671062; x=1716275862;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nOIhA9WGutZG1eFFZwMGqwKB0Jm2CMic0qQnHRgCzU8=;
        b=Vt3m7dtXH4JEJedqhyA3PAXa8ltNQMcN8QWVpjT0qo28K3455GD1orQZSyI1ZktmHo
         EQ56sv/AurUCFjNiVeBp5XlSOilOv9pzn4LHJtEHu1+muBmlO5FXL+zqy5nxu5LzIwyJ
         9nZ+9XWJHn7xaR9yMsvYzpH91GCxFnxuYSWXFYHYl4GfC3TweU3zkJzG+c6gosT+nDDu
         v6HvJFFgmHZb93dRNxW+KIzmpeMYmCoPsQkov9QWOxOydb8bAaZISGAsuOTiI5/4Hle7
         KVC6UcB8CyMyC5fkcw3Nctd8CZbLGRR1UuwJcxCjbURZBY8U5fitsTWwl23XAPwC81OD
         3Pig==
X-Forwarded-Encrypted: i=1; AJvYcCUcJxi4STzmT2t4K64PYR3RxVKNE9oxo0h9C6ej0NEFB0GgjRRpaismjFMp64TqUczwKc8xvdbRFM35OcvgG43NdaQnvxHJRH2wlYtic6Y=
X-Gm-Message-State: AOJu0YyaA9n0Jr2sBEfSk6Rgq/B8Ix9UxDc1jphe88j4yeE17gzWPX9D
	WPXOY/pV1Oxgj8u142BgXDlAHbfes8ky5NGHUuaLzZt8OA2cQnUldj/ZAm9Qbw==
X-Google-Smtp-Source: AGHT+IFl8pOZCFWPsiUEvgSZDJeu1QnF/6e1bh8rRdTq96Wc3qtI3ISvYot+4kD+hWjpVbys3Jo7Fg==
X-Received: by 2002:a17:907:2d13:b0:a59:c577:c5c3 with SMTP id a640c23a62f3a-a5a2d57a48emr1320356366b.29.1715671062189;
        Tue, 14 May 2024 00:17:42 -0700 (PDT)
Message-ID: <5c456b29-f458-4cd2-b92d-edb530de6959@suse.com>
Date: Tue, 14 May 2024 09:17:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 6/9] xen/arm64: bpi: Add missing code symbol
 annotations
Content-Language: en-US
To: Julien Grall <julien@xen.org>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, andrew.cooper3@citrix.com,
 edgar.iglesias@amd.com
References: <20240501035448.964625-1-edgar.iglesias@gmail.com>
 <20240501035448.964625-7-edgar.iglesias@gmail.com>
 <alpine.DEB.2.22.394.2405031656490.1151289@ubuntu-linux-20-04-desktop>
 <CAJy5ezpgXVp838aA_E8vJDARoO5iQOh07D+Gpn0kMCLj_iXyLw@mail.gmail.com>
 <398afdf1-8a11-4584-bb07-cf6fc2373d21@xen.org>
 <CAJy5ezpNQP5q0NVaGdXtt5BbJSzCMfTfFt-0RRgVq3+DqMbKgA@mail.gmail.com>
 <35c8fbfb-8b09-4aae-8c98-402dcd65ca2e@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <35c8fbfb-8b09-4aae-8c98-402dcd65ca2e@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.05.2024 19:37, Julien Grall wrote:
> 
> 
> On 07/05/2024 17:55, Edgar E. Iglesias wrote:
>> On Tue, May 7, 2024 at 11:57 AM Julien Grall <julien@xen.org> wrote:
>> Hi Julien,
> 
> Hi Edgar,
> 
>>
>> The reason I choose FUNC for the start of the symbol is because these
>> symbols contain
>> executable code (not only a table of pointers to code somewhere else)
>> and the ELF spec
>> says that STT_FUNC means the symbol contains functions or other executable
>> code (not only callable functions IIUC):
>>
>> "STT_FUNC The symbol is associated with a function or other executable code."
>> https://refspecs.linuxbase.org/elf/elf.pdf
>> (Symbol Table 1-20).
> 
> Thanks for the pointer. I originally did intend to suggest the change, 
> but then I saw the use of LABEL in x86 (such as svm_stgi_label). There 
> are a few others example with LABEL_LOCAL.
> 
> AFAICT, this is also executable code which the only difference that it 
> is not meant to be called by someone else. Furthermore, LABEL is using 
> DO_CODE_ALIGN(...) for the alignment which imply that it is intended to 
> be used by executable code. So I thought the only difference was whether 
> the label was intended to be used as a function.

No. See below.

>> I think using LABEL instead of GLOBAL for the _end labels of these
>> code sequences makes sense.
>> I'm happy to change the _start labels to LABEL too if you guys feel
>> that's better.
> 
> I have to admit I am little confused with the difference between LABEL 
> vs FUNC. I think I will need some guidance from Jan (he introduced 
> linkage.h).

For annotations the question is what is a "unit" of code. That wants to
be enclosed in FUNC() / END(). Any "inner" entry points or markers would
use LABEL(). On x86 I think it's mainly markers (i.e. addresses pointing
into code which we need e.g. for comparison operations on what Arm would
call PC) where we use LABEL().

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:22:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:22:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721164.1124341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mUc-0005UM-D5; Tue, 14 May 2024 07:22:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721164.1124341; Tue, 14 May 2024 07:22:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mUc-0005UF-A8; Tue, 14 May 2024 07:22:34 +0000
Received: by outflank-mailman (input) for mailman id 721164;
 Tue, 14 May 2024 07:22:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mUb-0005Tt-2T
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:22:33 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ba507622-11c2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:22:32 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59a8f0d941so1275418766b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:22:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfb83sm7135734a12.46.2024.05.14.00.22.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:22:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba507622-11c2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715671351; x=1716276151; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5OZM3jQfMRafwON+6Rr3i3c0ZrExWZEi7j42uNkEdFw=;
        b=Zbee86eXqJGZ7g2R1YZNOoS3g0lFt0KNqPWU1wMcwySv8K4zyhlFI3+qS0x+Y41hR9
         xqP2wiDq0lgYuKp4hFcTmogDieXqewpfopQkbDV/G0OOmbGz4FDQTTL8Kmqx3JnAWQJI
         4ac9RJyi71sRmEh4Tzd7O8QMVbwHTeS9yQdgqEWYwKmVsAUvbscj1AsJKNhMqYEbI0Lc
         7gmPvvnRwooejG8qPayZDtyqaNhcxRqdakmGB5NdUIbZzQrwZXztYcHw/EIjoBUnE3g7
         FP1Lb99eXu8pfBXdf3ENI5fWeamcXZEGK9z+OMjtIvCSQZXGRp5nm6DZzO+of19ZHlbl
         ksJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715671351; x=1716276151;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5OZM3jQfMRafwON+6Rr3i3c0ZrExWZEi7j42uNkEdFw=;
        b=lbtipPHCTO58aZ61/SFGZF3XFAtrry9mtdI4G2Tisbk4INRwZ00mlHAxgxap8rsD0r
         SbJnlnU5ktYHz64UFqhnBMKkFYGkM7oY8l0gWsQ0MBpBlDg9fytDfNTPtrgLgfTKBsT1
         s8cYOy/1Cg1ql8w0BMRoy8K6y8viHBnPMLKn9nKifIPvQ0cRg+eerCMRaLovkaCab/Sk
         qGsANY0Pe3VAxv4tEMhlVOcN3QH3l0SkMXaRdVfzrlTHcaSI7ZK6pzM4YLMpKKcgRhGL
         DnDioOVThblHKEFhMCMBiyq7v+jgzNNL1GR045jg3pTGLPwt85LPaizCKYuLrjOcnJsQ
         i8Cw==
X-Gm-Message-State: AOJu0YzZOwVbNmEhSled9k6Q4WFRagoHm4D6z55NF2xUMTDaa8wMPxpC
	7d6mlHUc1wp6/N2zgZEBPuEiHnYbK/yfMxB1zrE4H+y91K6fOM+g3WNa6TUlZQ==
X-Google-Smtp-Source: AGHT+IEX5x1WMdarNoWxfxkRtnwGqNzD2Mb3HMORyyF/9CMckXA9BluP696PNNYMmNixtmp5wtE4GQ==
X-Received: by 2002:a50:9353:0:b0:568:1248:9f49 with SMTP id 4fb4d7f45d1cf-5734d5bfbf9mr7364271a12.18.1715671351536;
        Tue, 14 May 2024 00:22:31 -0700 (PDT)
Message-ID: <8caac099-094d-4f16-b66b-8c8308684af7@suse.com>
Date: Tue, 14 May 2024 09:22:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/pv: Sanity check bytes_per_rep in rep_outs()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>
References: <20240511151642.2476555-1-andrew.cooper3@citrix.com>
 <ZkMNSbcB9S4YYcfy@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMNSbcB9S4YYcfy@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 09:05, Roger Pau Monné wrote:
> On Sat, May 11, 2024 at 04:16:42PM +0100, Andrew Cooper wrote:
>> Architecturally, there's no 64-bit variant of OUTS in x86.

This is the reason why personally ...

>>  Leaving an
>> assertion to this effect is enough to satisfy the analyser.

... I view adding such, in particular ...

>> --- a/xen/arch/x86/pv/emul-priv-op.c
>> +++ b/xen/arch/x86/pv/emul-priv-op.c
>> @@ -685,6 +685,8 @@ static int cf_check rep_outs(
>>  
>>      *reps = 0;
>>  
>> +    ASSERT(bytes_per_rep <= 4); /* i.e. 'data' being 4 bytes is fine. */
> 
> Don't we need this to be a BUG_ON() to satisfy the compiler also on
> non-debug builds?
> 
> Or maybe:
> 
> if ( bytes_per_rep > 4 )
> {
>     ASSERT_UNREACHABLE();
>     return X86EMUL_UNHANDLEABLE;
> }

... such a non-debug-build-covering form as dead code. With suitable
information to hand, I guess Eclair / Misra might even say so, too.

Instead I view the analyzer report here as a weakness of how analysis
is (and has to be) done.

Jan

> Would it be possible to add the check to guest_io_okay() itself?
> 
> Thanks, Roger.



From xen-devel-bounces@lists.xenproject.org Tue May 14 07:43:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:43:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721172.1124351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mof-00014l-3L; Tue, 14 May 2024 07:43:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721172.1124351; Tue, 14 May 2024 07:43:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6mof-00014e-0l; Tue, 14 May 2024 07:43:17 +0000
Received: by outflank-mailman (input) for mailman id 721172;
 Tue, 14 May 2024 07:43:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6mod-00014Y-HZ
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:43:15 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9d0806c2-11c5-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 09:43:11 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so1329662966b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:43:13 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b0195dsm686296066b.184.2024.05.14.00.43.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 00:43:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d0806c2-11c5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715672593; x=1716277393; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=O3T3F3wBgY2lowM3PHCpwiguoxntaMhIgnhnyvWn1vg=;
        b=W/AvKMm1UkURwdY99sPz/2MyIWkvZCREmyVMNziX/1KxGETxd1A+6s9AsUtLBJwQPD
         LkuIkA8MM39iQPw/DLFDR/B6LjNwA3alwMgkqCduFZubZrn57LOsOmZU9sqwMtl6YW5h
         QPJwV2iN20CTGVvXUE2ciYarzoNeP66nJZgLQ4Ku/b76yHcROVN2qaurHAQt49iAy/aQ
         LaLCHklJlNdAU3Md6lmtD+/65i4seVKyBhKzDsC2z4r94T+L9MtWucIWK3GY/qEszF49
         gC2Hfz/HW7y2WIirCRllWO+9VtqCQsMkw7TX2H00kwK/7EClw3e4fHV8iWokvbTJ9YAf
         NUHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715672593; x=1716277393;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=O3T3F3wBgY2lowM3PHCpwiguoxntaMhIgnhnyvWn1vg=;
        b=an/0xkXk8xC+D3gU1l38mq5Hkp3aADs6xm78jMXSKZKC1oSpEAvlRupULqkRLPAlhO
         q4gJBNOpgmpQuO53iFoatS3daD7WOwKfJ3xPJPysPOmfXuG1LDlz1IOu4963brWFYcs1
         OsPH2iGOQCsJ4AffJjToCsWowUBlgLjQbkavsDOCxFPOxcoLjCZZ33uA9y4LJVKQ2Y8w
         wzfDN11Kr/Y0OYre1Hs+Mt01+8sfLQeRYnCwMF0BxKfbQWyTureBcJQHGjjxL+GI1JTf
         DpRrHpWbGo1fljAvf4oEfPFyURaG/KMyxDU1+2Y80/qhf8YgKU58SCQoMxfl9BkkzKo+
         bVuA==
X-Forwarded-Encrypted: i=1; AJvYcCUsN1oaHSY7xCiaKVQXkCcapOaEthPvVvxRc9KqZp1WMry8Ya7yeiW9za1Qcvzk/03ga4T7li6tKfC5JsZwtLlgec6tKaDnNSQkJEm71G8=
X-Gm-Message-State: AOJu0Yzg/li6tsrUlGLzSKQCiBHeihpY8cRt0NqntXK6o3l7VV6kL09F
	pWiD6jQbXzIUcGM8yA5lQQ/9R8KZ/AzVKfczlv00EsKeeV+ZAimtqSOFy+CFe5zfKwpkKWVgp0o
	=
X-Google-Smtp-Source: AGHT+IGpzwXBf5dU5/rVk7zum2B2XClZ/K0f5UDayZ+j23o7xsQISy8R10sV2tSpxh3lXEB0mNhhiQ==
X-Received: by 2002:a17:907:d25:b0:a5a:2aed:ca2b with SMTP id a640c23a62f3a-a5a2d5c9fdcmr1041574366b.28.1715672592759;
        Tue, 14 May 2024 00:43:12 -0700 (PDT)
Message-ID: <ecb8814f-f9bf-4eb7-86ce-48bdd34f8bf9@suse.com>
Date: Tue, 14 May 2024 09:43:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/3] x86: detect PIT aliasing on ports other than
 0x4[0-3]
Content-Language: en-US
To: Jason Andryuk <jason.andryuk@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <0c45155a-2beb-4e69-bca3-cdf42ba22f2b@suse.com>
 <039b9ceb-4862-4e26-a344-e47fc04bd979@suse.com>
 <0cfaeb1f-947a-4e45-9f69-a0e3f8143e2a@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0cfaeb1f-947a-4e45-9f69-a0e3f8143e2a@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10.05.2024 19:40, Jason Andryuk wrote:
> On 2023-12-18 09:48, Jan Beulich wrote:
>> --- a/xen/arch/x86/time.c
>> +++ b/xen/arch/x86/time.c
>> @@ -425,6 +425,72 @@ static struct platform_timesource __init
>>       .resume = resume_pit,
>>   };
>>   
>> +unsigned int __initdata pit_alias_mask;
>> +
>> +static void __init probe_pit_alias(void)
>> +{
>> +    unsigned int mask = 0x1c;
>> +    uint8_t val = 0;
>> +
>> +    if ( !opt_probe_port_aliases )
>> +        return;
>> +
>> +    /*
>> +     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
>> +     * count is loaded independent of counting being / becoming enabled.  Thus
>> +     * we have a 16-bit value fully under our control, to write and then check
>> +     * whether we can also read it back unaltered.
>> +     */
>> +
>> +    /* Turn off speaker output and disable channel 2 counting. */
>> +    outb(inb(0x61) & 0x0c, 0x61);
>> +
>> +    outb((2 << 6) | (3 << 4) | (0 << 1), PIT_MODE); /* Mode 0, LSB/MSB. */
> 
> Channel 2, Lobyte/Hibyte, 0b000 Mode 0, (Binary)
> 
> #define PIT_MODE_CH2 (2 << 6)
> #define PIT_MODE0_16BIT ((3 << 4) | (0 << 1))
> 
> outb(PIT_MODE_CH2 | PIT_MODE0_16BIT, PIT_MODE);

Hmm. I can certainly see the value of introducing such #define-s, but then
while doing so one ought to also adjust other code using constants as done
here (for consistency).

>> +
>> +    do {
>> +        uint8_t val2;
>> +        unsigned int offs;
>> +
>> +        outb(val, PIT_CH2);
>> +        outb(val ^ 0xff, PIT_CH2);
>> +
>> +        /* Wait for the Null Count bit to clear. */
>> +        do {
>> +            /* Latch status. */
>> +            outb((3 << 6) | (1 << 5) | (1 << 3), PIT_MODE);
> 
> Read-back, Latch status,  read back timer channel 2

Was this meant as a request to extend the comment? If so, not quite,
as the line doesn't include any read-back. If not, I'm in trouble seeing
what you mean to tell me here (somewhat similar also for the first line
of your earlier comment still visible in context above).

>> +
>> +            /* Try to make sure we're actually having a PIT here. */
>> +            val2 = inb(PIT_CH2);
>> +            if ( (val2 & ~(3 << 6)) != ((3 << 4) | (0 << 1)) )
> 
> if ( (val2 & PIT_RB_MASK) != PIT_MODE0_16BIT )
> 
> I think particularly a define for PIT_MODE0_16BIT would be helpful to 
> show what is expected to be the same.
> 
>> +                return;
>> +        } while ( val2 & (1 << 6) );
> 
> I think Roger might have mentioned on an earlier version - would it make 
> sense to have a counter to prevent looping forever?

Well, as before: The issue with bounding such loops is that the bound is
going to be entirely arbitrary (and hence easily too large / too small).

> Also, FYI, I tested the series.  My test machine didn't show any aliasing.

That likely was an AMD one then? It's only Intel chipsets I've seen aliasing
on so far, but there it's (almost) all of them (with newer data sheets even
stating that behavior). We could, beyond shim, make the option default in
patch 1 be "false" for systems with AMD CPUs (on the assumption that those
wouldn't have Intel chipsets).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:53:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:53:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721179.1124361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6my8-00041C-VV; Tue, 14 May 2024 07:53:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721179.1124361; Tue, 14 May 2024 07:53:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6my8-000415-T1; Tue, 14 May 2024 07:53:04 +0000
Received: by outflank-mailman (input) for mailman id 721179;
 Tue, 14 May 2024 07:53:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6my7-00040z-OQ
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:53:03 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fd77d97a-11c6-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:53:02 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-420104e5336so15811415e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:53:02 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee934csm184556455e9.38.2024.05.14.00.53.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 00:53:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd77d97a-11c6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715673182; x=1716277982; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=tNqsSLUtvip3LDymvbn34Szd6re8FWDuq8rbRTdB1sA=;
        b=Vw28hktAy8GTxotGbxA0VN+/mLsPIFKs+Xyg9IunNtxscuWIUXCZWY0IuWwW5nIVlR
         XE+KNesByOo3DGNyvXiRfauWhQ9LPykeSw3qACW0tZFLRabj/NI6QD/490n2sFjdnXnS
         nD0SVYmO8ciYUL23SDxa7fD4j/+4VHqC/kWkA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715673182; x=1716277982;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tNqsSLUtvip3LDymvbn34Szd6re8FWDuq8rbRTdB1sA=;
        b=NaZm+OKvIrat2exzSu3Eiep3nTU5iQb5s88c/JlmYnnCHiKYhbnV/YbJEyBFvWD8+u
         0F5kR06ZNuuccZV1iMOnNcWg7XdmzYbpv/sJu8tUQeYLDDAzW1yVseVm+cOYAXp/8u86
         KbjEu1Ub1LW2tIdYw8K4Lu+ahMpb2WDTTsp5gAKIA+EY0fMOQUVcmSZWxd5pWqN/+Zhs
         i7fBnVpf+bxm1oS6Admlq0axw50KUnYhIzY8k9Gax+UDbvW5dl63Ccb30ufLAfTE1iMF
         WFRm9vYtpv4HUfWxwPLbi4ePgMo1aqTYt+HXAYUM14ZLNBWMRUtA9zyy6BKEwL3ptV1T
         BU8A==
X-Gm-Message-State: AOJu0Yz7V/3cUoDU03mBF9TELrSU7V0LHPfZs39y6EGeyfZ+BnsPWtos
	+dxehLPNwtiq3ijLe1gmoYKNudO0wYYiRxrtuHPhtYqwrE/+yY8TgevxvugdKBg=
X-Google-Smtp-Source: AGHT+IFuHJ9M4iMVdyKC1GjZhRRpCulrqslXw0w/ud/jqo+mlQ6Hn0OoFy+qOr/BHO+9Gpuukka8tA==
X-Received: by 2002:a05:600c:6548:b0:419:f533:ef88 with SMTP id 5b1f17b1804b1-41feaa42589mr107092315e9.11.1715673182020;
        Tue, 14 May 2024 00:53:02 -0700 (PDT)
Date: Tue, 14 May 2024 09:53:01 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature
 names
Message-ID: <ZkMYXSf9abR3J3FE@macbook>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-4-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240510224002.2324578-4-andrew.cooper3@citrix.com>

On Fri, May 10, 2024 at 11:40:01PM +0100, Andrew Cooper wrote:
> From: Roger Pau Monné <roger.pau@citrix.com>
> 
> Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
> data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
> are needed.
> 
> In xen-cpuid.c, have the compiler pad both leaf_info[] and feature_names[] if
> necessary.  This avoids needing complicated cross-checks.
> 
> As dump_leaf() rendered missing names as numbers, always dump leaves even if
> we don't have the leaf name.  This conversion was argumably missed in commit
> 59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> 
> Differences in names are:
> 
>  sysenter    -> sep
>  tm          -> tm1
>  ds-cpl      -> dscpl
>  est         -> eist
>  sse41       -> sse4-1
>  sse42       -> sse4-2
>  movebe      -> movbe
>  tsc-dl      -> tsc-deadline
>  rdrnd       -> rdrand
>  hyper       -> hypervisor
>  mmx+        -> mmext
>  fxsr+       -> ffxsr
>  pg1g        -> page1gb
>  3dnow+      -> 3dnowext
>  cmp         -> cmp-legacy
>  cr8d        -> cr8-legacy
>  lzcnt       -> abm
>  msse        -> misalignsse
>  3dnowpf     -> 3dnowprefetch
>  nodeid      -> nodeid-msr
>  dbx         -> dbext
>  tsc-adj     -> tsc-adjust
>  fdp-exn     -> fdp-excp-only
>  deffp       -> no-fpu-sel
>  <24>        -> bld
>  ppin        -> amd-ppin
>  lfence+     -> lfence-dispatch
>  ppin        -> intel-ppin
>  energy-ctrl -> energy-filtering
> 
> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
> one.  Several of the + names would be nice to keep as were, but doing so isn't
> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
> but we intentionally don't list them, so I'm not worried.
> 
> Thoughts?

I'm fine with this, we are now coherent between libxl, the Xen command
line cpuid= option and the output of xen-cpuid.

> 
> v3:
>  * Rework somewhat.
>  * Insert aliases of common_1d.
> ---
>  tools/misc/xen-cpuid.c | 15 ++++++---------
>  xen/tools/gen-cpuid.py | 21 +++++++++++++++++++++
>  2 files changed, 27 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> index 6ee835b22949..2f34694e9c57 100644
> --- a/tools/misc/xen-cpuid.c
> +++ b/tools/misc/xen-cpuid.c
> @@ -11,6 +11,7 @@
>  #include <xenguest.h>
>  
>  #include <xen-tools/common-macros.h>
> +#include <xen/lib/x86/cpuid-autogen.h>
>  
>  static uint32_t nr_features;
>  
> @@ -268,7 +269,7 @@ static const struct {
>      const char *name;
>      const char *abbr;
>      const char *const *strs;
> -} leaf_info[] = {
> +} leaf_info[FEATURESET_NR_ENTRIES] = {

Won't it be best to not specify the number of array elements here, as
we could then use a BUILD_BUG_ON() to detect when new leafs are added
to the featureset and thus adjust xen-cpuid.c?  Otherwise new
additions to the featureset will go unnoticed.

>      { "CPUID 0x00000001.edx",        "1d", str_1d },
>      { "CPUID 0x00000001.ecx",        "1c", str_1c },
>      { "CPUID 0x80000001.edx",       "e1d", str_e1d },
> @@ -291,6 +292,9 @@ static const struct {
>  
>  #define COL_ALIGN "24"
>  
> +static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
> +    INIT_FEATURE_VAL_TO_NAME;

I've also considered this when doing the original patch, but it seemed
worse to force each user of INIT_FEATURE_VAL_TO_NAME to have to
correctly size the array.  I would also use '* 32', as it's IMO
clearer and already used below when accessing the array.  I'm fine
if we want to go this way, but the extra Python code to add a last
array entry if required didn't seem that much TBH.

> +
>  static const char *const fs_names[] = {
>      [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
>      [XEN_SYSCTL_cpu_featureset_host]    = "Host",
> @@ -304,12 +308,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
>  {
>      unsigned i;
>  
> -    if ( !strs )
> -    {
> -        printf(" ???");
> -        return;
> -    }
> -
>      for ( i = 0; i < 32; ++i )
>          if ( leaf & (1u << i) )
>          {
> @@ -338,8 +336,7 @@ static void decode_featureset(const uint32_t *features,
>      for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
>      {
>          printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
> -        if ( leaf_info[i].name )
> -            dump_leaf(features[i], leaf_info[i].strs);
> +        dump_leaf(features[i], &feature_names[i * 32]);
>          printf("\n");
>      }
>  }
> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
> index 79d7f5c8e1c9..d0bb2e4a229f 100755
> --- a/xen/tools/gen-cpuid.py
> +++ b/xen/tools/gen-cpuid.py
> @@ -470,6 +470,27 @@ def write_results(state):
>      state.output.write(
>  """}
>  
> +""")
> +
> +    state.output.write(
> +"""
> +#define INIT_FEATURE_VAL_TO_NAME { \\
> +""")
> +
> +    for name, bit in sorted(state.values.items()):
> +        state.output.write(
> +            '    [%s] = "%s",\\\n' % (bit, name)
> +            )
> +
> +        # Add the other alias for 1d/e1d common bits
> +        if bit in state.common_1d:
> +            state.output.write(
> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
> +            )

Had no idea we had this aliases.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:53:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:53:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721180.1124372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6myk-0004TX-7l; Tue, 14 May 2024 07:53:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721180.1124372; Tue, 14 May 2024 07:53:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6myk-0004TQ-4x; Tue, 14 May 2024 07:53:42 +0000
Received: by outflank-mailman (input) for mailman id 721180;
 Tue, 14 May 2024 07:53:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zXVB=MR=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s6myi-0004TI-Q4
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:53:41 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on0629.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1307f4cf-11c7-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:53:39 +0200 (CEST)
Received: from AS9PR06CA0673.eurprd06.prod.outlook.com (2603:10a6:20b:49c::17)
 by AM9PR08MB6004.eurprd08.prod.outlook.com (2603:10a6:20b:285::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 07:53:36 +0000
Received: from AMS1EPF00000045.eurprd04.prod.outlook.com
 (2603:10a6:20b:49c:cafe::b0) by AS9PR06CA0673.outlook.office365.com
 (2603:10a6:20b:49c::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.25 via Frontend
 Transport; Tue, 14 May 2024 07:53:36 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS1EPF00000045.mail.protection.outlook.com (10.167.16.42) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21
 via Frontend Transport; Tue, 14 May 2024 07:53:35 +0000
Received: ("Tessian outbound 85e363419cbe:v315");
 Tue, 14 May 2024 07:53:35 +0000
Received: from c7385488fda6.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 210C976F-8B3D-4D5E-AE48-470CFC71CFA3.1; 
 Tue, 14 May 2024 07:53:29 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c7385488fda6.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 14 May 2024 07:53:29 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB7887.eurprd08.prod.outlook.com (2603:10a6:20b:52b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 07:53:26 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.052; Tue, 14 May 2024
 07:53:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1307f4cf-11c7-11ef-909d-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=UwWLz3powiR/h0Zc8Q+iUoO6al6BDbD84Bj3cLhbaRCGGs3J1JN31p2VbKraVY/Vm42C5DVUNq+X09IVzI/N8iz0kRd7cuUe/FKUPjMMMpoJJ+85Tj4acNY6Agnp+0/b0HXgU6iku2hFgPU1lIGVekVIFCuoXp1FlRwogDijLrFiJYhDv06Pcgar8hOZV8bjAYOXW4v5YOfz2m6YEt2mp3K+Z8iGEJvsfBfVv89wD6RoIKxyPxTaFWj/vYrw+Pv+3rWrrtyFdLx9KdKrNGMwWILbCt11dd524nTDXTNS8LIXl0bMpp/N9Cju7pdraNFtlHw3rKxRA3mS/K4IEkg9Fg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DERBBdoj0rOaglpD1IoOe+rXZPnkfSDKmKH3vSPMDQ0=;
 b=hr6koaKMspYWWXBmP2xanTBxDluxpsnrBkYgfoz6qD8e6YanC9Vr0yEQXFC1su+NzA/fGDqGmDYgMs+qE8vWe33NvtQsWwcDjiLdkYDp5xtt0IP0XZ3u4uCpx99436o+921H3dRHzDM/9FVOTlYFy7YJo8sN3K78ePOCW8T1H3bdOabrVputbbsmXMRbi0jc3B/LLJeRvnvLT5gTUGseVf2yEGemsD1lhZdfDtBZR+ycT6ICfsAeqvGmPdly9c+jdBjecN7l4UVl7mVpCHHtwXhgXDj5/q/ptEPSkMQf8qSjVJlwoDRPIBCXOMDG+WffuS7aOQdQfa9hs0I9F/tOtQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DERBBdoj0rOaglpD1IoOe+rXZPnkfSDKmKH3vSPMDQ0=;
 b=OmP3BnVb3R+FS5rJfXwFgX5f5IFZFDvfvkIvq3b7G0eBdZqcPY0jcE+G/QEZAzT40UejYI1qUku6Xgobu94yXJyBGAfbYppPAZ5vhAmV56dSwwBvT6uykJRzy8upx3CofwFU37S2G4oWaq2HWJCmxeOMQmuvqdqhOV9DgtnyPfg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: d8aa948aacce0b8f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DCGRiinJqNTgTPLBV1XUSjs+HhWlSEv6qGi7z1KQSGvChM5Niu1tRbLNzHQll64e08TAP88egeHc9tDyT407nvi3ltJgPaYE2gaDcnwxMizTT1+Ow7zWST2Vfwcb7FnZA7opJwGv+mYHzm+EGk8rXZcKiDhud7qlNi/kwy53aJOcEQORXoXLushuSQ90bYi0nTbSeF0CI8GkDCrzSnVqvms6Ov/jIy+V3gsE0zqn9D4Z05M9hdTEd6Z0GcU5CM6FSCxGRTq8fAOA3qzoDoGzEd7LTHE/yZDq+/6Jo1+HdDAqNW1M8RrTxt4moWWxvdIJ5vDYUCdI0bX6KbycKqYzXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DERBBdoj0rOaglpD1IoOe+rXZPnkfSDKmKH3vSPMDQ0=;
 b=ZFO/BOxW/muvEJ2shs5ijqmJCAd+Y9p8SBKTJ3B+JYIaz3wHXBMFe4AaiZDZi2XUfXUcBhzYjHdzylPLoYJmcXlkx3DtMeHcotWs1mjSsvhmhALn72Ynzf7vdWoo6PMq8UYkLe+hAywFZ07eyWTWX+yDHzG81eCQWGp5cYagwFQrTgJzfbUSR0RzEYLD6HCReXeeJCBk+AswGfCHMQjQh3PqOSXR2qShfLUAYpSFo20V8ehpdR+hdnYZq1ZB4E1q4KHqx1MqJdCJyhBX+i/k7Z3WjvZVi/Zm909Y/aK2f+J90k1nNdAgvH2LYEGUbarebfinmxarKIx8drLKmbGzcg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DERBBdoj0rOaglpD1IoOe+rXZPnkfSDKmKH3vSPMDQ0=;
 b=OmP3BnVb3R+FS5rJfXwFgX5f5IFZFDvfvkIvq3b7G0eBdZqcPY0jcE+G/QEZAzT40UejYI1qUku6Xgobu94yXJyBGAfbYppPAZ5vhAmV56dSwwBvT6uykJRzy8upx3CofwFU37S2G4oWaq2HWJCmxeOMQmuvqdqhOV9DgtnyPfg=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Thread-Topic: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Thread-Index: AQHalxIi36RR5t8uSEeoUslJ6d4QpLGV0j4AgACnKQA=
Date: Tue, 14 May 2024 07:53:25 +0000
Message-ID: <0A931CEA-59CA-4B1F-A8E4-A4CB990C0476@arm.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-2-luca.fancellu@arm.com>
 <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
In-Reply-To: <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB7887:EE_|AMS1EPF00000045:EE_|AM9PR08MB6004:EE_
X-MS-Office365-Filtering-Correlation-Id: 9b942e59-bd24-48e3-3167-08dc73eaf57d
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?NDFDVzhjcWwrQW8yeFU4ekgxM2xyNjg3YUsrUHdkU2pXS1FjdjJLM1lZeC9E?=
 =?utf-8?B?MjJuZlpqTVFCd2Z1M0tjRmtNWWUvMWgzYitUSVBIV2NDRjFOR3NtRzBaSEVR?=
 =?utf-8?B?TE5MWXN1Q1gzTTMwcjdvYmJxUGIvMkRjdmVzdlA3eDV5bWdSN0F5S3hSaXlJ?=
 =?utf-8?B?emVhcHZESitSKytFTDZmaEJIZ2JGY3paSWo5Q3Bwd3RHcWZpNURreEZzWDNP?=
 =?utf-8?B?aGhRQnVHK2xhbDVVM29ETG90RjNkWExET1ZhbW9XQWdKRnVUb2tvZ2RLOGpw?=
 =?utf-8?B?aVFWOXVVUnA0dGo5N3o5bUpUd0FpMmF4NFVwN2RzY3dLWEljb28zSzNQWkoy?=
 =?utf-8?B?OXJGR25wZDN3M2dUd282dGxaQWlrQlYrd2E4UGdpUzJOK2N2V2JvcDg2Zzlq?=
 =?utf-8?B?RElVSUtnZ01ybVZBV1FrdGUyWGlGdUFyejI4dWRzMldGVll2cC9EU3B6d1Vp?=
 =?utf-8?B?YThVcCtWZkV3TW5HSjRZT3dtOHdKemZIcldTaWVpcnJ2Z0NPSS9BYnpxSEhr?=
 =?utf-8?B?d2RScGZ4VFo1Mk1sckxjZ0xrNDdsWHFKN3FRaGtLejAvemlmc2hvUHgxVGZQ?=
 =?utf-8?B?WmhyRXpFbmszRlppQUUydVhwMDBTN1laRjI1d24vT0lhRUovVWpWVnpOSTNB?=
 =?utf-8?B?ODg2dENpYXNEalVlTzVmOExJOXozSFRVUmVJSDBVWUhRbElSUFhsbjZ4UzIv?=
 =?utf-8?B?ZEFEdDhMZUM3dW1WUmhSYTJiMkNmbitQenlqR25kTGxyVktpcEoxZDVURTFS?=
 =?utf-8?B?NE9JNkkwMjNoYk9HNjZ3T1pBMis4MURwQytlNnNzTlE3WWZiSG5DejROb3RR?=
 =?utf-8?B?ajhSRGJ0WHVYekpSOXlmVXgxTjlneWttODZtZ1QrNVpqbjBpNEFiU2FGRVJZ?=
 =?utf-8?B?Y1lpRUVUcHBDOVdHZzg2Z0l3MjZmQVhDVHhPSmlNOFo4QTBoWjFpZmpqd3Bh?=
 =?utf-8?B?UTNLck52dmwrR1J3MmpTS3YrcU5TWk5UTHZKUC94dUFVMzl2SzdtZHdGR0ZK?=
 =?utf-8?B?S0g2R0xOdGZXV0swVkhWYTFlVHpuL2tpZEsyVCtxczNUMUs2alRhcjBEc01Q?=
 =?utf-8?B?eXMzRTAvNmVsRVlLT0kzRG93aFVPSmVtSm9NOHBrWHNkTDIvZUV3UUlYY3BV?=
 =?utf-8?B?bDRwSm1YMittQlVLYi9kQytsaFgvMWFpWnBnSERkMUVkQ2x0RzRPbkx5REQr?=
 =?utf-8?B?STZKdGs2RUQ2MC9yRnZzUFN4RmMzUG92QXdsQkZlaXc1VEJud3F6cUlNMVhF?=
 =?utf-8?B?MWlXbFdJaHJ1dVdzVWxnK2RvSDRUaG0yT0lFTnhvS3c5Q2NMSlRYNStNVkZ5?=
 =?utf-8?B?OFVrZWE5emxNeVVoekNYZUNRU201VXdyMmM4aVc1ZUljdXVQV3djYkZaQyt3?=
 =?utf-8?B?UG1BZU9SRStzdUErTnhIVlNZYTlET0pCc2F1Q2xKcVhhZWhQVVh4NVNxanhv?=
 =?utf-8?B?Y29EQVJqUzhKME10bllvS2s5S0VJdjVzZnA2MjJEbTdRdkg1bTBVOHhKSXlM?=
 =?utf-8?B?OURoeWExSVRHT1p6cFBielhGd2tCdFdRZDdBWW1WQ01RTUJ6QVlZc01QT05B?=
 =?utf-8?B?WERNckZ2UWI1SlhDTUw5d1dIVklMOG5tank1T05UU2J1T2NQQkNZTUhHdE12?=
 =?utf-8?B?WXhRTGN4UEFQdG1KY2o1dW4zeUUwM1VrOXR2WnNiU1lFck1vVWdJQ3pBSVBD?=
 =?utf-8?B?QTVFakxGd1puYytrdTRKeWJCdVYwNVdyQ1FYd2dnblpqVUt5L2w0dWM5WDFJ?=
 =?utf-8?B?QUhwcklKdUdmQVBuQmsxY1lmZnJIczJ6d2xhaFB3VnVsZmhuclFZTG84cWpJ?=
 =?utf-8?B?dkMyUFpVbFc4V2p2WkQ1Zz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <457B5DA159A60F44A8F84A1DF7F7AD03@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7887
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS1EPF00000045.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bbf2beed-2072-4e25-6ea0-08dc73eaef99
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NVJMMVZJYmVrSW43TXF2aUJHMjdpL1ZMNElFMEFtT0RUb2pWN2RPanA3OTda?=
 =?utf-8?B?UE1RdXlnZkcwVUZ4T0JvUUxsTmhmRjk3NW5tY2QyWDZWelE1WWRDR2RNQnkv?=
 =?utf-8?B?bWthQjg4RWVDODJqTWRiRXp3MlRMdXloSnFCamIxRU5DYUMzQ055VWdNb29l?=
 =?utf-8?B?YWp4QnVldFFtZGxaYkMvVU96c2J5VjhwYUQ2RDJHcnlITjhFMlZWUmtGd1dK?=
 =?utf-8?B?R3ExSGNPV2NBMWptTUhlTWdHU0VndjJYYmR1MnlNUXJMTitWQjdsbm1zK1JJ?=
 =?utf-8?B?ZEdvRTZjR0k4aU1DUUlTZGg1WSt3TTZxWXFoMjU0OEFGTDVlR1prQWt5TWs4?=
 =?utf-8?B?eFZsVWVTSDNEUDZuTkJPSVQ4aVh1TzAzZnp1dS95RkJYVXBXRzdCYTlINFQ4?=
 =?utf-8?B?SUJvQzNkTGJraTc3aEE2MGdSSGJQalJTQkllTnZDY3EySUNGVXJldXJQNERZ?=
 =?utf-8?B?R3Y3a2ZMeStnTnIwL013R2ZxRmZocS8xS09oNUY5ZWIyY2JTNHNBSk1ETFJi?=
 =?utf-8?B?aVdORVo1VzJKNTZZT1JIOGoxVGQ4NTE4MStEcTBXTjk5bHpJKzRRRVAxTHNY?=
 =?utf-8?B?RktkV3hYOHRRSFJrSnppd2t6R3BtWWpLdUNsUjZpYTNGYi9YNDNidjZ5WHZq?=
 =?utf-8?B?SURlZXhvTkpCK1dDTUIrMFI5aWFKTDNUK3lQbFhkN1hETkVQUTE3Tkw4ME96?=
 =?utf-8?B?S0w2NTZ5SGQ4eDNVQzMreWpHMUNIYWo2ZE05RVcrTmExZmtlL1FGYkw4cWp6?=
 =?utf-8?B?OFBWeXBZM2tZeG9pQnFUaE9mMUxRKzJrVGg2d0JsaUNsckEwQWs3NTBNSWlU?=
 =?utf-8?B?MUZwbHBEd3dyejYra1AwTFZqVkdURjFBMi8xYUNwMWUzWEhqdVFFTTd3RGgr?=
 =?utf-8?B?RXZlSWR0ZWprelRPdVE0YUw1eGthVjQ4a1N2VjVORGxYMWh1bFI2WjVQVjVz?=
 =?utf-8?B?bmd6WGZiV0RMZGxRc0RHR2s1cGEvSktzdzdUcEpsaEtyNEVTZGFDRzk1eGNZ?=
 =?utf-8?B?WWY4emtOdGhqZzdPbXpXZGRnVU9yYlo0NFliRzFhRjdiUVhIS3hPbDRmOVlT?=
 =?utf-8?B?VVR1RzhtdGdUcU1TZXJYNGpVUXNQdmNBTzhBVmpsb0NSd3FKWFlpSS84MTMz?=
 =?utf-8?B?Q01VNWJ3SnQ5VVliK3o0Z2Z4aU5vV2xpK2JTVXhORkJpQ01ra1RTb1RVd0dU?=
 =?utf-8?B?TFJDSkxlVXJkQUFwNUpIc0xIUkdxRUV3L0lnOUtaYVdhU0pDN3NweXkydHV5?=
 =?utf-8?B?bVNiRmxiY2dKbmFqYXp3dXI4M3p6d1pHZFZDbUl5U3hRd2pvRnR4djFmdWc0?=
 =?utf-8?B?ZFkvQVdjTjZVWWhQYk53amlIVGE0WVdSN2NiTlJsejgxMm01UnNqVVlsa1Fj?=
 =?utf-8?B?cWxlWVMzWlFncXNRd3VBblBhZlRneXhuRFFhQ0VacUpCTEhIL3hZNXlHdFBl?=
 =?utf-8?B?emFEVVBSMHMzZkhrSU84RTM0U2s1QjljQ0JrNlM1NU5aRzJoL0RYSUp2Zmpq?=
 =?utf-8?B?TU1IUzJ6K0JUUklkcnhNcVJsQzhlMkZOTG9uaWRFMEFZeVlSR3RIamdNVzlI?=
 =?utf-8?B?RUEyNlNnbHgyclJGVURqendJR0hSVFFNSVZEc2xKZktKSGlaK3ZBUm1DLzRp?=
 =?utf-8?B?T2VwdnpKUGp1VGpxUDlpSHZiY2xTUHpSU2k2bVdPRXJ1Y3o2YVRoL3RDdEcr?=
 =?utf-8?B?L0d3bGZjMTZ3eEgxRnM0SFRxNDFjZHBDUG84QTYwZ0gwTlpLZmVxUkZuQVho?=
 =?utf-8?B?ZWVxM3VlQzQyVjFKaWVpUWZEeExXUW5UQTE4RUwxcUMvSFBFVUlJRkFxRVl1?=
 =?utf-8?B?SE5qbGs1WmdjbnVCMGZBUmdTRjR4NzJXNXp6YTA3RTgrNzVQdGVGaVhkSXNj?=
 =?utf-8?Q?tBMsrLlgS7NRD?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 07:53:35.7306
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9b942e59-bd24-48e3-3167-08dc73eaf57d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS1EPF00000045.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6004

SGkgSnVsaWVuLA0KDQpUaGFua3MgZm9yIGhhdmluZyBhIGxvb2sgb24gdGhlIHBhdGNoLA0KDQo+
IE9uIDEzIE1heSAyMDI0LCBhdCAyMjo1NCwgSnVsaWVuIEdyYWxsIDxqdWxpZW5AeGVuLm9yZz4g
d3JvdGU6DQo+IA0KPiBIaSBMdWNhLA0KPiANCj4gT24gMjUvMDQvMjAyNCAxNDoxMSwgTHVjYSBG
YW5jZWxsdSB3cm90ZToNCj4+IEN1cnJlbnRseSB0aGUgY29kZSBpcyBsaXN0aW5nIGRldmljZSB0
cmVlIHJlc2VydmUgbWFwIHJlZ2lvbnMNCj4+IGFzIHJlc2VydmVkIG1lbW9yeSBmb3IgWGVuLCBi
dXQgdGhleSBhcmUgbm90IGFkZGVkIGludG8NCj4+IGJvb3RpbmZvLnJlc2VydmVkX21lbSBhbmQg
dGhleSBhcmUgZmV0Y2hlZCBpbiBtdWx0aXBsZSBwbGFjZXMNCj4+IHVzaW5nIHRoZSBzYW1lIGNv
ZGUgc2VxdWVuY2UsIGNhdXNpbmcgZHVwbGljYXRpb24uIEZpeCB0aGlzDQo+PiBieSBhZGRpbmcg
dGhlbSB0byB0aGUgYm9vdGluZm8ucmVzZXJ2ZWRfbWVtIGF0IGVhcmx5IHN0YWdlLg0KPiANCj4g
RG8gd2UgaGF2ZSBlbm91Z2ggc3BhY2UgaW4gYm9vdGluZm8ucmVzZXJ2ZWRfbWVtIGZvciB0aGVt
Pw0KDQpTbyB3ZSBoYXZlIDI1NSBiYW5rcywgaW4gbXkgZXhwZXJpZW5jZSBJIHdvdWxkIHNheSBJ
4oCZdmUgbmV2ZXIgc2F3IHRvbyBtYW55IHJlc2VydmVkIHJlZ2lvbnMNCmluIHRoZSBEVCwgbWF5
YmUgYSBjb3VwbGUsIGJ1dCBJ4oCZdmUgYWx3YXlzIGhhZCB0byBkZWFsIHdpdGggZW1iZWRkZWQg
cGxhdGZvcm1zLg0KDQpJ4oCZdmUgdGVzdGVkIHRoaXMgb25lIHdpdGggQURMSU5LIEFWQSBib2Fy
ZCwgbjFzZHAsIEp1bm8sIHJhc3BiZXJyeSBwaSwgcWVtdSwgZnZwLg0KDQpJbiB5b3VyIGV4cGVy
aWVuY2UsIGhhdmUgeW91IHNlZW4gYW55IG51bWJlcnMgdGhhdCBjb3VsZCBiZSBjb25jZXJuaW5n
PyANCg0KQ2hlZXJzLA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:53:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:53:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721181.1124382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6myp-0004lF-Jk; Tue, 14 May 2024 07:53:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721181.1124382; Tue, 14 May 2024 07:53:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6myp-0004l8-Ff; Tue, 14 May 2024 07:53:47 +0000
Received: by outflank-mailman (input) for mailman id 721181;
 Tue, 14 May 2024 07:53:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6myo-0004kM-9U
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:53:46 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 152c3683-11c7-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 09:53:42 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41fd5dc0439so35501535e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 00:53:44 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccce24c0sm184376475e9.17.2024.05.14.00.53.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 00:53:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 152c3683-11c7-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715673224; x=1716278024; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=U/U2uxGMFphbtP9Y5GWb8Km1liP89pYaqmGeATZxIPw=;
        b=p+n0TH0x7rUSiVTgSl59nlJaQC7OpS+736m4nMpP3+eGdL0SADSyJdbTv/HcaEjULR
         SH42HobOKEaNFsJrGqaMsTCRQE/k4S+ttdo7W/BD0hV3gnsl5HY3+YMANF9FEnMGZ+d3
         /xsiR3mNQ4s2ZlV+95Grh8ZOJsD5TvSlJDB5c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715673224; x=1716278024;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=U/U2uxGMFphbtP9Y5GWb8Km1liP89pYaqmGeATZxIPw=;
        b=FcRs50Hc2N4OMhQwgd5At9r4QR5PUZeJuhjnvsG2WgAlivrx0bU0d5Sasz3wrVil7C
         4QAvscPiSAaZxF2/ruOx5fKdONJd7JBbygXMJK5ysb5YfgTHe4Nn0VaTaWbpc8/Y4T1D
         B8Gj/JXWQtZG9kRCPpbvO9PueNlPBU8hT5ZI0jv0OhlsBm/+iXIBnWKvPYQYaK7PO4m5
         8pRzfP/ny0iPa3L+mADiz8MyLCvZ43cOZiDERou2kZ2WYR+Ux4wvaJYaT4gX5xe/3gGZ
         xT9JZdDdSMPi6hhG6/+BsiNQflSyJRUsAyKeonNiuLurx5QOCsIFlpFzcAeRND0cCAIr
         yq8w==
X-Gm-Message-State: AOJu0Ywl0cv8alAEAdQRUQ123yObY092cU3ZUZ5J0Dd+gmon6wTnDNf2
	CI0HXQ5Thdbz76wSP3YoBysgtG44jFbsARAMRy2Cga1vtxVQMhXlsdFtSThoa1ViVM+yuuDlT8z
	F
X-Google-Smtp-Source: AGHT+IF/jlHTa2o1y+pCpg9TuTZDCLr6Q5LQjL/ljpNAD3ALZuMVyhsl48u30tDrJnIwadCspa4kUA==
X-Received: by 2002:a05:600c:4f42:b0:419:f241:632f with SMTP id 5b1f17b1804b1-41feac49273mr98204135e9.31.1715673223653;
        Tue, 14 May 2024 00:53:43 -0700 (PDT)
Date: Tue, 14 May 2024 09:53:42 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH 4/4] tools/xen-cpuid: Drop old names
Message-ID: <ZkMYhtFXeb5_vmCI@macbook>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-5-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240510224002.2324578-5-andrew.cooper3@citrix.com>

On Fri, May 10, 2024 at 11:40:02PM +0100, Andrew Cooper wrote:
> From: Roger Pau Monné <roger.pau@citrix.com>
> 
> Not used any more.  Split out of previous patch to aid legibility.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 07:55:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 07:55:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721187.1124391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6n0r-0005eD-UH; Tue, 14 May 2024 07:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721187.1124391; Tue, 14 May 2024 07:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6n0r-0005e6-Rf; Tue, 14 May 2024 07:55:53 +0000
Received: by outflank-mailman (input) for mailman id 721187;
 Tue, 14 May 2024 07:55:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zXVB=MR=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s6n0r-0005e0-1y
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 07:55:53 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05on20601.outbound.protection.outlook.com
 [2a01:111:f403:2613::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 62043398-11c7-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 09:55:52 +0200 (CEST)
Received: from AS9PR05CA0211.eurprd05.prod.outlook.com (2603:10a6:20b:494::13)
 by PA6PR08MB10737.eurprd08.prod.outlook.com (2603:10a6:102:3d3::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 07:55:48 +0000
Received: from AM3PEPF00009BA2.eurprd04.prod.outlook.com
 (2603:10a6:20b:494:cafe::7d) by AS9PR05CA0211.outlook.office365.com
 (2603:10a6:20b:494::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 07:55:48 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF00009BA2.mail.protection.outlook.com (10.167.16.27) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21
 via Frontend Transport; Tue, 14 May 2024 07:55:48 +0000
Received: ("Tessian outbound 082664cc04c1:v315");
 Tue, 14 May 2024 07:55:48 +0000
Received: from db70ad9e55eb.3
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1A3AF8A9-74BC-4200-A369-997E56741677.1; 
 Tue, 14 May 2024 07:55:37 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id db70ad9e55eb.3
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 14 May 2024 07:55:37 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB7887.eurprd08.prod.outlook.com (2603:10a6:20b:52b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 07:55:34 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.052; Tue, 14 May 2024
 07:55:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62043398-11c7-11ef-909d-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=oF2ReOiUB/0d3n9dvrzM+G+KKqq+tlCQHTGqYZlxYY1LlgL7tpDcBYPZ9XcI/D5QkdCMyybf0g5FJOGLA5tCnZYOACqXf66tinzIGBZ8SpYLG8Kvs0H594iYeC4awK+VME0hws6q+NAaaVFojUy88orDPIGXZ/Wrs+MzgRUP2GaaRnWAVR67ODbvf1W4r/KDfr/N0PcTms90JTMRdSfcHlc/clCgOCgDf1J78vI0Bgr+gIqIBlfiYbDlLvt6P/DISjJ2Kaw/tvS7DunLTOAgjsAYmoKi6xp+uSLWRmrEktKf+nvVmf02LWaqwClZFm11aHb4Dnst0yyeFuVWce1+ZQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/Ew+847jKNwKKO056iCqVpSt2bYtNQ767IWILIMJ9gA=;
 b=lhD3PL7mdDLtmqb+4xfdqILAWGuO3vX3fTdKXWX4HVFmREhNL3YYkgvqHs4hcQ7i4/QXKG6jlWskojyaeJUy99X9Bx9OIYgAe1l2uiYnvJq1NqjdB1eNBUwhAXESJBfhIFI+ZCYhNPk9uqH2Kke73HsKFQXzWKMwzcZ/vSEe/4crKf0LpTQ/5xj+6F7Vssc9gEBINDkyNnrkHxCVJ/YxxIK85Ra9uHrKhgnO8cHfPwN1u1KUQmR+VEhG/yd9sUAL4+Gx2LT3Kx4VcxgFZ622SxLw05JSvoaiafVnweKQD/ucMMxmZwdbzUvwNqNuVixR589yU6/71VD1FzNy8S75Fw==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/Ew+847jKNwKKO056iCqVpSt2bYtNQ767IWILIMJ9gA=;
 b=kzWIt3s12pWO5CIlhHPLqrcDUq+kaeo7Ezrud9lmNc+nYUMOJC8B3nx2gbvxb2RKyGytn5m801ee+pIythljDQxVL3LSuJ89q1JdkBV04hLU7EdozRaTUtYdkO+LO4X6xci30quX65wCctuQDz9dEHFW35I9L0HIShhnr3xoC2c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: da6083fd78f3b55d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cbwsl4j2EZOPcub1YkwM23Mdh+5Xt+v2G0KQoez6ljmAepV7n47E+uXISKx0vAtTL5u+zBBMNh8QDyG6htBikhrLcvDj8zntHR/iZ6SihkC9MUNDwvFlqdWO4w42FA/AtJbnCXlYoIwVne0uIZOpgn9pU6qXXykbpgXJzNaYufUcAFelVsape6AifWyeDIgBqj2fPCqEPuMPlx9CSQOsuyt/aHOAkIdh+TwI0AhvpA+lJWF9Fv+Lx31/b2pJQLJCs0fonC7kmNJRLta2I8c/W3YrDhAhFKDpxCNT4i8riU0zgwt/Nt+cCyDZzNm8CU7n9UZqzW6t+6xhrPo1wCdAmA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/Ew+847jKNwKKO056iCqVpSt2bYtNQ767IWILIMJ9gA=;
 b=cFt374c40yLJbllJc3i4lJWaSMQNU+YnsApLoWVprv19IpST95M0ChgfWOi5DoOa5K7++QtvbiFG+qr26EAJ00WLLRH2Te3HVDzUPlrGiCERdy0O1z5B56Tu7+1bJSUi29ZSKaTYJ6HXAnjbsO3mPCpfeW3WJRnuzYjTndJlEJpA0t0pzselH/QsxMWf7joY2FTbsgCzb6sb+ULBbZZByF2VZfSIlivQp7Wods0z+Hc3rlGXcWca0mJvD5EE/qYTZqlJSMYghfST3yGQSAi3RjJvkoomYtHfdH8Pd6NDw0DiRSSwHjHJfyGN4ydB1R5UVMuC/WgnU+OH+/f/VQhmtQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/Ew+847jKNwKKO056iCqVpSt2bYtNQ767IWILIMJ9gA=;
 b=kzWIt3s12pWO5CIlhHPLqrcDUq+kaeo7Ezrud9lmNc+nYUMOJC8B3nx2gbvxb2RKyGytn5m801ee+pIythljDQxVL3LSuJ89q1JdkBV04hLU7EdozRaTUtYdkO+LO4X6xci30quX65wCctuQDz9dEHFW35I9L0HIShhnr3xoC2c=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>, Penny Zheng <Penny.Zheng@arm.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage
Thread-Topic: [PATCH 3/7] xen/p2m: put reference for superpage
Thread-Index:
 AQHalVftK+2d6c8qh0SLUJVfhafWO7GL2AEAgAACwYCAAhGEAIAAEkYAgACocwCAABsXAIAAGy0AgAAMPQCAAAzdAIACI5OAgAVjhYA=
Date: Tue, 14 May 2024 07:55:33 +0000
Message-ID: <03410419-DEBC-4C73-89B0-0E04EA36070C@arm.com>
References: <20240423082532.776623-1-luca.fancellu@arm.com>
 <20240423082532.776623-4-luca.fancellu@arm.com>
 <b7d1d40b-c634-4830-aa4b-b0475bdff8fc@xen.org>
 <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com>
 <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org>
 <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org> <ZjyFxrvHJ04ZlBGg@macbook>
 <bd463f7a-eadf-441f-96e3-3a39f647f160@xen.org> <ZjyzTCOSeNvPq2Np@macbook>
 <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org> <ZjzIWsoRmgmWkgGh@macbook>
 <e1cf9c7c-581d-4bec-acaf-654d40b7ac07@xen.org>
In-Reply-To: <e1cf9c7c-581d-4bec-acaf-654d40b7ac07@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB7887:EE_|AM3PEPF00009BA2:EE_|PA6PR08MB10737:EE_
X-MS-Office365-Filtering-Correlation-Id: 2bb5efd6-a573-4eff-7c7a-08dc73eb4482
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?Uk5mcWZVZURtQ2lrYUUzZGtPTWlLY1VqeHVmd2RGR1hyMGZMNmlZK3NSenZh?=
 =?utf-8?B?bkRVYVV2akJwdXo2YkFZM1JscHQ0ZW95U2w1SjRIM3U2dk0rdGRJS2lJaVgr?=
 =?utf-8?B?TGlFRU4xVlNIR2V2KzRVODZZK29EL2xkdm9Nek9salVSU2ZBdzJFVVRjdlNo?=
 =?utf-8?B?M2t6STRWSEZiNWZ5YzlzTVBCSFVXajBHN2NJTjdhd0ErbGpVMnd0NmtkaHZl?=
 =?utf-8?B?YlRQS0NEa1hGbUVSOVh4Q2p1U3I3VlUxVTl6R0prcHVlUW8xU1dkZGxsaWtL?=
 =?utf-8?B?cDB5ZExQMjlLa3Z6YlNLZmp4RzFLZWRnWUVRWG1mWUpsV2oyNzAxUU5jYlhv?=
 =?utf-8?B?YjFOYkZ3bDV3czE0STV4Yzk4ZFYyUkxsRFlKQm1zTHFKd3krelBvYm1UY2R0?=
 =?utf-8?B?U3c3WERyY3ZaMkZXbUgwSlBjVDdjTjRYbmxmQVRVQTV6TFZvMnlOemZZbnYw?=
 =?utf-8?B?TnVKYW1Zc0JtMXRFQ3FFemNZNENKMjZmWlRGQ3FQVkE4NjJnTDA2R25YWTUy?=
 =?utf-8?B?bGREdjc5T2xUY0ZkaGFVdUdETkhoTG5Sb1ZHZnJhNUtoQ1A1ZzU3dndzQUtF?=
 =?utf-8?B?UzJSbDZhL21XZGtLWmpTUjhyek9hVE5YK3BDOEhkQWc2NFVkaUVDbmhiWE1P?=
 =?utf-8?B?RXZBQ3VIWWFTaEMyS3NhSjhCcURpVm1YVWRwOUUrRXBScjd5RHNsaVhaVmRj?=
 =?utf-8?B?UWhabzRQT1liS3ppTEdlUk5idTZVaGNJbnBmZG1wWmtGczQ5aWVGNmlmakda?=
 =?utf-8?B?V2d0aGZZMVViVVJ1a1hLUk9ST2M2N3dQTlZSMC91MGdqWjlRSFJmWXZrMVV0?=
 =?utf-8?B?SVd3VVNmL3pyWk1Ob01yYmsvcFhDNDVNSm9zdFlrVzUxejU4WENPRThJa1VJ?=
 =?utf-8?B?L1UrZmljeFZGcDM2eElEQ0NqTWtpZ2NuMnJ3T0NkRUQyQllJZTdBdmZjSDFr?=
 =?utf-8?B?KzVaT3dOZG1wYllMdGxPelBuQkJsRWtNS3F4SXQyaGRWQm9aenVuQm5IdXN6?=
 =?utf-8?B?eTBVVG95aUZld3dUM1JWZ0tOeUJRRHRpMVVMcnJQK1VjZVN6TVRvdWttVW9q?=
 =?utf-8?B?MFdZdWFJdk91VTlOSm9uZmdoVktvakF0NkZVeUkwYjdMdFdOK0YzeisrQkpj?=
 =?utf-8?B?L09Qd2ZBWlpPQTJlVWYzcWZzeUpVUWFpcVN2UUVOMlBVWjh3ZDRTbVpSQU9m?=
 =?utf-8?B?NEpKamt2aENyRHZGR2NTSlZKR2lpSWRraFk2cW9sRWJmd0VIS0VuazV3S1pC?=
 =?utf-8?B?M3R3VVZvN3p4UEsxYU9mMi9mZWhUTGUwdHdoR1YxblV0NjBGMDVkdkRiSGdu?=
 =?utf-8?B?U1FhNnFqQTNJVEZNV3p0NVZEdW9TMDRjUWdyc0Q2dmRjUTRqQklHSFROTTBl?=
 =?utf-8?B?R2xId2FOc0piSEhRZHNvTWRJU0NsRG5QTnZnbEo0cVB3MXpEOW5hbk5vY1lJ?=
 =?utf-8?B?Q2l3czRLNWszRkt0dm0yWDNwWEoxdnN5NHpaQ3FtcmhENk5qR1pYelgyNW1p?=
 =?utf-8?B?ODRrSDhTVng5R0R6S3dmekV6L1ZRYnNnaDIwVjJjQ2lyVXY0d2FBVGFaamxI?=
 =?utf-8?B?N1JmMjJnbHZSR1dEYk5hOGFZK2hwcGkzanNRUXVzWW83akFyREI0Nm92dFJ3?=
 =?utf-8?B?R3dXR2YzdE1vSkVEaVh0NXRHb3hranlHelhjeHpFb1crSUZQRTRwTGVmS0Z4?=
 =?utf-8?B?QUxnVkZNbnEyUVFlN2NtRnNTWWR1a0RxUlBmQUhPRFlaYjJrTWF5a3FUbWZT?=
 =?utf-8?B?U3MxcFh5SVBxKzlvalR0RVdObjJpcHNidldjZi9HSkM5TXA0TUN4YktjcFRx?=
 =?utf-8?B?RmNra1hFVVlncGZhVXRxUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <4CA6AE040F90F747AD052AD1685E6822@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7887
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF00009BA2.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c4c6c218-d275-419f-2738-08dc73eb3bee
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bFlTa2h1UTVQMVl1V3JQdzJDM2pFQllYVW96bjNhZ2pRR3FEODdNSWVRVzgw?=
 =?utf-8?B?eFRNaHRKN21PakJmekJ2WEpGb1E1Q1ZkalBxMmoxWUNIZFI1N0NLSjdZeFV2?=
 =?utf-8?B?TFg1Qk1VblZWaU1BeUNzVXlZV2p5anpnZnRPdVRZNXUxSCs2YWM2VWk4ZXdj?=
 =?utf-8?B?UW1mSHluMGkxK3R2cDVST1FiL05LUjhpM09kbFAwckFpTkN3aElPbCs1ZWU1?=
 =?utf-8?B?NWNuYjFuR2VzQ2p0V1dROVVTWU95TmtPS01IZ2NiUmp0WDN2c1RUWWUyZ29z?=
 =?utf-8?B?RXYrNUhEbVRYMGZWZ1NlcTdic0M4REVHMnhjZy9JMlNmdTViTFl1VXVNUXI1?=
 =?utf-8?B?N3ZNK2lONnZEU3FTazhtNmdCZFNnOUxEUXh4QVE3bVFUcE1SaVhhT2NtVVFK?=
 =?utf-8?B?NlUxWDdvYTFnUnBSZktpeUdGR0dWdXhJdHYvTEpNQXVKTFEzY2pENU1JQnl2?=
 =?utf-8?B?MVFvRmwwVDQ1eC9BYklHVkMxTmJ3cVpHcDRiYUtVU2k2MTkrTzdsempQc3ox?=
 =?utf-8?B?UE9yWUlkcFgxUDlDcHpDVWxiVmtLdmw2M1VJN1dLK2I2UWczL0h1OHJJVnZK?=
 =?utf-8?B?S2tsV2poNUhBb0s0NzFkTFRRUmxyZGZDQmYxYnhqcWE3eUVLaDREekIxejJp?=
 =?utf-8?B?N2pQbVlldlFNT1RqY2NkamF4dmU0Zm1FM0Fta2FXVzFtUXRqTWRyZlZsZ29y?=
 =?utf-8?B?NUZrdXErSDRUODFZajk2eVZDV3hFVVV5Ym1ndThESzhYazJEQy9IZ0VieHh4?=
 =?utf-8?B?MjNDZEV0MjlubjdhMDMvMzFpLzlVS2NkK3U0RXFXY3QrV09NU3YzeDRSL1lm?=
 =?utf-8?B?WG90SU1aWUhsZE1ZZDFYZ3pvSTBkVVE1eEluYWZpUDZtQUQyaEtnQXMrVERJ?=
 =?utf-8?B?RTRBNTlGZ1U3b3FNaTkvWTdML0pPaWV5dGUwb3VPVUlFOWVSdTBCZ3RMK2Rn?=
 =?utf-8?B?MVpMZC9kdzBNeThPMkVEOE94YmlWVDhabXllUDM4Y3pCdVYrajRsVU04VEhB?=
 =?utf-8?B?KzZ3NDM4QWwrWE9WTWtPd2wvVElRRjFUWXY2aGZaWTZaS1NLdnJ2ampDb2la?=
 =?utf-8?B?ZXhVQ2JJR282bGREWXdIbDBXQzg4cUdGQ0JlcVRTYVg4ZUk5YzNkTVUxaU8r?=
 =?utf-8?B?L2tZMXZPbFVDTTN6a3lMK3daZDhJSjA2ck9NOFRuS0ZGcnZwTklWUEIxUnRh?=
 =?utf-8?B?Ry9KaUNxY05rY2ZqbVhtd01LWWhhc3pyL0pJN0Njc05ZS3U0ZkIxbjl3N3RZ?=
 =?utf-8?B?NHk1bnVFTzNuZ2t4R3JMajJIWWw3MTVhY2NhOXR3L1EvYWY2YWxacmVBTkpQ?=
 =?utf-8?B?VUhaQ0VLSm8zZUZlRHZsODlLd2xRVmwzQUZRNlh3alBVZmxjNmpGcjVVUFA0?=
 =?utf-8?B?QVFWYmZ4UWVEcTl1SjAxK0Qza1J5bkFhUEJLOXZ4MDdKM3NsK3RoUG1wcjVI?=
 =?utf-8?B?UzdIbTdvd1JoZ2gvMXczUUR0aFJPRXhNTE1XVDJqakRYSWJWT3oyRitZUlJp?=
 =?utf-8?B?a2pxWDNuV1JkRFhPVFB0S3QzOEZRanNDdEJIRjUxSWlNYU4zM052UVlIUHE4?=
 =?utf-8?B?cVMxc21pTWk1ZFlpa05WMUEzSG5jVnlDNkNFTG94eTA5dWpvdnlOWUU4N2Jo?=
 =?utf-8?B?aVJpZllDVmNTZUZEamxaNUw3QnQwbEZxenFQZ09qUjlHK0JQY3doWjVHc2pP?=
 =?utf-8?B?VHJlN1ZHVlFYcTZ0MnI4VzlwZkRIY3RIa1Q3Tmc2Z2pJcXptMCs4NGI1RXNL?=
 =?utf-8?B?OXJhK2F2VVV4bVRUaU8yVm1qVFZXaHk5WWtsWlNMT1FGZzhucFpicGEvQjVN?=
 =?utf-8?B?dXdrY0FIRjdBM3BOa3M0bFdRT2JzY3paN3ZYVXYrMS9MSC9ER3BOMlZ3STVx?=
 =?utf-8?Q?biX/qDLD+cCBc?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 07:55:48.3210
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2bb5efd6-a573-4eff-7c7a-08dc73eb4482
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF00009BA2.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA6PR08MB10737

SGkgSnVsaWVuLA0KDQo+IA0KPj4gTm90ZSBhbHNvIHRoYXQgYSBmb3JlaWduIHVubWFwIHJlc3Vs
dGluZyBpbiBhIHBhZ2UgZnJlZSBpcyBhbHNvIG5vdA0KPj4gdGhlIGNvbW1vbiBjYXNlLCBhcyB0
aGF0IHNob3VsZCBvbmx5IGhhcHBlbiB3aGVuIHRoZSBmb3JlaWduIGRvbWFpbg0KPj4gaGFzIGJl
ZW4gZGVzdHJveWVkLCBvciB0aGUgcGFnZSBiYWxsb29uZWQgb3V0LCBzbyB0byBiZW5jaG1hcmsg
dGhlDQo+PiB3b3JzdCBjYXNlIHNvbWUgZWZmb3J0IHdpbGwgYmUgbmVlZGVkIGluIG9yZGVyIHRv
IG1vZGVsIHRoaXMNCj4+IHNjZW5hcmlvLg0KPiANCj4gR29vZCBjYWxsb3V0LiBJdCBtYXkgYmUg
ZWFzaWVyIHRvIHJlcHJvZHVjZSBpdCB3aXRoIHNvbWUgWFRGIHRlc3RzLiBVbmZvcnR1bmF0ZWx5
LCBJIGRvbid0IGhhdmUgdGhlIGJhbmR3aXRoIHRvIGxvb2sgYXQgaXQuIE1heWJlIEx1Y2EgY2Fu
Pw0KDQpVbmZvcnR1bmF0ZWx5IEkgZG9u4oCZdCBoYXZlIGJhbmR3aWR0aCBmb3IgdGhhdCwNCg0K
PiANCj4gT3RoZXJ3aXNlLCB3ZSBtYXkgaGF2ZSB0byBhY2NlcHQgbm90IHN1cHBvcnRpbmcgMUdC
IHN1cGVycGFnZSBmb3IgdGhlIHRpbWUgYmVpbmcgZm9yIHNoYXJlZCBtZW1vcnkuDQoNCkkgd2ls
bCB0cnkgdG8gZG8gaXQuIA0KDQpDaGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721197.1124401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6n8I-0000BL-5O; Tue, 14 May 2024 08:03:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721197.1124401; Tue, 14 May 2024 08:03:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6n8I-0000BE-2s; Tue, 14 May 2024 08:03:34 +0000
Received: by outflank-mailman (input) for mailman id 721197;
 Tue, 14 May 2024 08:03:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6n8F-0000Ac-WE
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:03:32 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6f169b24-11c8-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:03:22 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a88339780so62548066b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:03:24 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf91sm691714166b.1.2024.05.14.01.03.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:03:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f169b24-11c8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715673804; x=1716278604; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=B1LY1qdhRQMDOmOQYsJeYpKNF6BoOvEzcGeZU9pEyvA=;
        b=SD79d1vdQmQDkFni4NG/Heymy6kQLmx2zdqMvHGwa3+K8/0X9J/+OgSkVtxksY+n1D
         orC/NYifRspzRBr30FIwdpqiTy7kr75XCB/1IcAS6MIEcegpvqQdcRVBnLx3hFn+srYk
         TCanaXYN9ta8WabBkQ80gbv4uqsZE3ywfEt95olagjIhlBdGhYRE6c9rSDHEjAI0pLRm
         ya6sP13YP8/0cv2XvVVE6xmz638GW3R+uXIrExaba9czdkCms695OmOfnehTH9LSKDpt
         r9EjFZW3gHvuNdhgKEa/pp/ZB0h3ZO+7t81sLLYs8wHXmd13qZy+Y0qk8LiZdWbFNrR8
         4fFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715673804; x=1716278604;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=B1LY1qdhRQMDOmOQYsJeYpKNF6BoOvEzcGeZU9pEyvA=;
        b=TrSTPWjBrXDq6Hf0BNeO9IIZm+7AkJZFIh+0ll71flJAn6Kj1nt8otilCL3XXW5xTk
         U57GSi0A2hBPjclnT5MHAdxSBc9BxCZs7b41T79v5FtTVGwuO09GSOfekFEhyPwJWvVZ
         o4lIXjDmPn8WfKHfOCKGER3y8jl+/KL0f789qN0jEoFU29sfvdG4fDO7YrV7YlJcBnjE
         mCRM+JZJs48H53gju5b6OsnBFuKCU+1A9UCZoG5TYRXwFkwBkXtwPliokA21PRub9T7k
         ImSJmgmGy9nQgc6QfxY2mWAjgEeTS0lcr2rN+gHBDYCD7A8AtAPL45qX3FJ06dbN5s/6
         ImtA==
X-Gm-Message-State: AOJu0YwAbawf9/aBb4kwNcXxo8fAetrCSe1iizgUUx7ULPwHZ16L2O7n
	8URoi9c72YlPsLKTbkyBwLkgh1O87TNPYRG6i+FR0Mqh4v5sxBpXmzzRyPiVnQ==
X-Google-Smtp-Source: AGHT+IE6zymnrKmqHJ+7ljBSFMQD6NG0f7VxTiRwiD7lJ3PzB0CSs0318JVuU8uOf/9EdpUWIktp8g==
X-Received: by 2002:a17:906:fe4a:b0:a59:a8a4:a59d with SMTP id a640c23a62f3a-a5a2d66aa6dmr908319466b.62.1715673804137;
        Tue, 14 May 2024 01:03:24 -0700 (PDT)
Message-ID: <7998ecee-659d-4180-8b14-110c2907021f@suse.com>
Date: Tue, 14 May 2024 10:03:31 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Julien Grall <jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
 <20240513111117.68828-2-eliasely@amazon.com> <ZkIxdtiDc_pnPWdx@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkIxdtiDc_pnPWdx@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.05.2024 17:27, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 11:10:59AM +0000, Elias El Yandouzi wrote:
>> @@ -317,6 +317,14 @@ extern unsigned long xen_phys_start;
>>  #define ARG_XLAT_START(v)        \
>>      (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
>>  
>> +/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */
>> +#define PV_ROOT_PT_MAPPING_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
>> +#define PV_ROOT_PT_MAPPING_ENTRIES      MAX_VIRT_CPUS
>> +
>> +/* The address of a particular VCPU's PV_ROOT_PT */
>> +#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \
>> +    (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))
> 
> I know we are not there yet, but I wonder if we need to start having
> some non-shared per-cpu mapping area in the page-tables.  Right now
> this is shared between all the vCPUs, as it's per-domain space
> (instead of per-vCPU).

In turn requiring per-vCPU page tables, posing a problem when a guest
uses the same page tables for multiple vCPU-s.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:10:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:10:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721202.1124412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nF8-0001v8-SD; Tue, 14 May 2024 08:10:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721202.1124412; Tue, 14 May 2024 08:10:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nF8-0001v1-PG; Tue, 14 May 2024 08:10:38 +0000
Received: by outflank-mailman (input) for mailman id 721202;
 Tue, 14 May 2024 08:10:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6nF7-0001uv-LG
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:10:37 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 717f7e7a-11c9-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:10:36 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59b097b202so1127433766b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:10:36 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eb5ec296sm23926a12.35.2024.05.14.01.10.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:10:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 717f7e7a-11c9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715674236; x=1716279036; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ue2si8TDcodY21jNkP+jHA6IAfQUCqT37+yxOhXm0tw=;
        b=Ai9YUfdx11lomStVo6gXee/mwe8wV1gBFid9RoHp55b47otAgOOcaf4Nuny5i3FfXz
         mpzrxqLkvEqDj4WYXRR7EFN/bSeeUz2V2Pug+ITy3MizV9826gaOH+BfHy2SO+BXunfq
         NP3us77hJ/XyHDvZRnDXXsueXyFlcOmPvT0HTY8cma32b3VbSpZf0RxtHM4XDOlE+KsS
         Bbeo7KbZW9mDBEJvPyODcIaE57ku4llfGMZZiS0a2GbW1Qlf8oLLSIU7hLGJPy5pgmlv
         10LOcSOeTw8Mbtae2T/FjN2gKhnpSMQCprSJvJ05VXG26LjiNMNmDFhmCu+o85ujohOO
         wbJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715674236; x=1716279036;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ue2si8TDcodY21jNkP+jHA6IAfQUCqT37+yxOhXm0tw=;
        b=u2ookj3UjXyYX4D1gJS/6tkE0B4XLzUkIxYyP8akWsac4c/+8CTnmzwwTT/PIvZQQZ
         mUrCZGGhVbzip3aNAcX+bP47Sn/IaqBTeUCMupLD7sal4/VQnTot43lCiu9s2SWZKRNM
         M1bhdNhoDrRVBstYCzA++qO8/uk0CS7ufC3agrKZxnD7GEvw3jVGID1BtYIvK/urSYB3
         u7w1sgzchrtdFXBr6vXFEtrXABdzORRlXMSMuU3cWjeY6k1OKfA0H26N74FVvZJU3ALh
         oGGFE0MACK7/9ykPxkAyYRtwa/jxP45OB2bffvBiGvf61uof+n7ZojGCObvuvMxLvK/+
         isjw==
X-Forwarded-Encrypted: i=1; AJvYcCXvhSa0MNg4jWAkJcjyQ3gqkDTDGoOp5WR5mGNdo6KjedNxhwmDY2UklYrWpQTIF1zZNXXnd6Nxmyr97eN9nKlRPYUr6WjQk5tDkLeugXk=
X-Gm-Message-State: AOJu0YwsQLKsH9QWvF3IfUhBjwJ2dTE71SIl19xCs65hol1qvtFn9E6g
	YguUcMMimdO6JpNxBh4uVeo01/TfjKXzbKn6mLYejJoq/89/fjTvKKwdhdl0ZA==
X-Google-Smtp-Source: AGHT+IEap1g5MWmmr0O6R8azLrDaIln4p8Ak/Kkp89VxfqrQOy68ddbdUOV4CyeyKBeCI3Kcwcrelg==
X-Received: by 2002:a50:9353:0:b0:56c:5990:813e with SMTP id 4fb4d7f45d1cf-5734d5bfbd8mr7580752a12.13.1715674235927;
        Tue, 14 May 2024 01:10:35 -0700 (PDT)
Message-ID: <e6b525cb-a1f1-494b-8dfe-c9d86cfc8f9e@suse.com>
Date: Tue, 14 May 2024 10:10:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of
 mem_access.c
Content-Language: en-US
To: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: consulting@bugseng.com, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10.05.2024 14:32, Alessandro Zucchelli wrote:
> --- a/xen/arch/arm/include/asm/mem_access.h
> +++ b/xen/arch/arm/include/asm/mem_access.h
> @@ -17,6 +17,8 @@
>  #ifndef _ASM_ARM_MEM_ACCESS_H
>  #define _ASM_ARM_MEM_ACCESS_H
>  
> +#include <xen/types.h>
> +
>  static inline
>  bool p2m_mem_access_emulate_check(struct vcpu *v,
>                                    const struct vm_event_st *rsp)
> @@ -35,12 +37,28 @@ static inline bool p2m_mem_access_sanity_check(struct domain *d)
>   * Send mem event based on the access. Boolean return value indicates if trap
>   * needs to be injected into guest.
>   */
> +#ifdef CONFIG_MEM_ACCESS
>  bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec);
>  
>  struct page_info*
>  p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>                                    const struct vcpu *v);
> +#else
> +
> +static inline bool
> +p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec)
> +{
> +    return false;
> +}
> +
> +static inline struct page_info*
> +p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
> +                                  const struct vcpu *v)
> +{
> +    return NULL;
> +}
>  
> +#endif /*CONFIG_MEM_ACCESS*/

Why would each arch need to repeat these stubs? IOW why would they not
live in xen/mem_access.h?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:14:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:14:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721209.1124422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nIh-00031y-DJ; Tue, 14 May 2024 08:14:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721209.1124422; Tue, 14 May 2024 08:14:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nIh-00031r-Ai; Tue, 14 May 2024 08:14:19 +0000
Received: by outflank-mailman (input) for mailman id 721209;
 Tue, 14 May 2024 08:14:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p490=MR=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s6nIf-00031l-Kz
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:14:18 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f4826853-11c9-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:14:16 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.vpn.solinno.co.uk [172.19.84.10])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 9BE738006E;
 Tue, 14 May 2024 09:14:15 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id A839A20189; Tue, 14 May 2024 09:14:14 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4826853-11c9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715674455;
	bh=w1u90DsNaHSjyLaNrDcHOwtXTjvNeyUuUYKrG7Ev38M=;
	h=From:To:Cc:Subject:Date:From;
	b=BMkDFvsMH4xBk09gyB7SCtCwXsJ7ONM3Rn3sFhQ3Yak9AkHN6/+yR17Tt6zKaVD/A
	 /kBOj/KuGJ6HHA+62y1H/pRoVO4/12QZ5Pk7RSrUx0JgNLYXEz6PTxTNWeiXzz9u9l
	 n16u/APhAEFAauZvhbZv6+ghdSqWIqhdxljhA7sI=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH] tools/xentop: fix cpu% sort order
Date: Tue, 14 May 2024 09:13:44 +0100
Message-Id: <20240514081344.4499-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Although using integer comparison to compare doubles kind of
works, it's annoying to see domains slightly out of order when
sorting by cpu%.

Add a compare_dbl() function and update compare_cpu_pct() to
call it.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/xentop/xentop.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 545bd5e96d..99199caec9 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -85,6 +85,7 @@ static void set_delay(const char *value);
 static void set_prompt(const char *new_prompt, void (*func)(const char *));
 static int handle_key(int);
 static int compare(unsigned long long, unsigned long long);
+static int compare_dbl(double, double);
 static int compare_domains(xenstat_domain **, xenstat_domain **);
 static unsigned long long tot_net_bytes( xenstat_domain *, int);
 static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long *);
@@ -422,6 +423,16 @@ static int compare(unsigned long long i1, unsigned long long i2)
 	return 0;
 }
 
+/* Compares two double precision numbers, returning -1,0,1 for <,=,> */
+static int compare_dbl(double d1, double d2)
+{
+	if(d1 < d2)
+		return -1;
+	if(d1 > d2)
+		return 1;
+	return 0;
+}
+
 /* Comparison function for use with qsort.  Compares two domains using the
  * current sort field. */
 static int compare_domains(xenstat_domain **domain1, xenstat_domain **domain2)
@@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
 
 static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain *domain2)
 {
-	return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
+	return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));
 }
 
 /* Prints cpu percentage statistic */
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:16:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:16:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721213.1124432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nKo-0003ZU-Qm; Tue, 14 May 2024 08:16:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721213.1124432; Tue, 14 May 2024 08:16:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nKo-0003ZN-Mh; Tue, 14 May 2024 08:16:30 +0000
Received: by outflank-mailman (input) for mailman id 721213;
 Tue, 14 May 2024 08:16:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nKn-0003Z8-G8
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:16:29 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 41cb8f15-11ca-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:16:27 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 1FB383A4C0;
 Tue, 14 May 2024 04:16:25 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 181113A4BF;
 Tue, 14 May 2024 04:16:25 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 127BC3A4BE;
 Tue, 14 May 2024 04:16:24 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41cb8f15-11ca-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:mime-version:content-transfer-encoding;
	 s=sasl; bh=5h1V3cl8XHOKSgcuBJuKqihrCoEZnPy9xnGmQ9Ui9F8=; b=UGSz
	D04qMWlIn9AqJv2/CHP9Xf/mjskq5BtxDULt9CzLDowapaeQdWk14iqkOveFVbPO
	HmyCVku6FzYYBki/XNiVsRtBQdCAmN/6/VRiTZJcOXUtluogcdG8xIelusto2T+a
	TYP0Dij4GhWPbniudksGcfBwTCZ7fthYECeXYa8=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 0/6] x86: make Intel/AMD vPMU & MCE support configurable
Date: Tue, 14 May 2024 11:16:21 +0300
Message-Id: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
X-Pobox-Relay-ID:
 40DAC67E-11CA-11EF-9C88-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

This series aims to separate support of Intel & AMD CPUs in Xen build.
The code to drive both platforms used to be built unconditionally, until =
recent
introduction of CONFIG_{AMD,INTEL} Kconfig options.

This series extends coverage of these options on vpmu and mcheck subsyste=
ms,
which allows not to build Intel or AMD vpmu/mcheck support if CPU vendor'=
s support
was explicitly disabled.

Since v2 series one patch had been split into two patches for more clarit=
y,
hence series became one piece longer, and last patch changed significantl=
y.
Other changes are mostly cosmetic and tags addition.
Specific changes since v2 are provided per-patch.

v2 series here:
https://lore.kernel.org/xen-devel/cover.1714640459.git.Sergiy_Kibrik@epam=
.com/

  -Sergiy

Sergiy Kibrik (6):
  x86/vpmu: separate amd/intel vPMU code
  x86/intel: move vmce_has_lmce() routine to header
  x86/MCE: guard access to Intel/AMD-specific MCA MSRs
  x86/MCE: guard {intel/amd}_mcheck_init() calls
  x86/MCE: add default switch case in init_nonfatal_mce_checker()
  x86/MCE: optional build of AMD/Intel MCE code

 xen/arch/x86/cpu/Makefile           |  4 +++-
 xen/arch/x86/cpu/mcheck/Makefile    |  8 ++++----
 xen/arch/x86/cpu/mcheck/mce.c       |  8 ++++++++
 xen/arch/x86/cpu/mcheck/mce.h       |  5 +++++
 xen/arch/x86/cpu/mcheck/mce_intel.c |  8 --------
 xen/arch/x86/cpu/mcheck/non-fatal.c |  8 ++++++++
 xen/arch/x86/cpu/mcheck/vmce.c      | 13 ++++++++++---
 xen/arch/x86/cpu/vpmu.c             |  4 ++++
 xen/arch/x86/include/asm/mce.h      |  1 -
 xen/arch/x86/msr.c                  |  2 ++
 10 files changed, 44 insertions(+), 17 deletions(-)

--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:18:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:18:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721217.1124442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nMo-00048I-4R; Tue, 14 May 2024 08:18:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721217.1124442; Tue, 14 May 2024 08:18:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nMo-00048B-19; Tue, 14 May 2024 08:18:34 +0000
Received: by outflank-mailman (input) for mailman id 721217;
 Tue, 14 May 2024 08:18:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nMm-000483-9G
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:18:32 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 89b10015-11ca-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:18:28 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id B6DF02C401;
 Tue, 14 May 2024 04:18:27 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id AC85F2C3FE;
 Tue, 14 May 2024 04:18:27 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id E52922C3FD;
 Tue, 14 May 2024 04:18:26 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89b10015-11ca-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=6W+coo3XknLwYEm/aCICEBr73
	mGn+RWQy4lYwn4OC8E=; b=kHbgHxQQKm/+QulFNZUHDx2aMocV6pEtUBc8WTbcO
	PGsoC2uOt32ES60IWdGfwKfuej/4+E0X3nMOA+AN9GgBR73g63MDrc9TKkVirt6u
	Y0gkNk7bHOXW7E4Y7DPx93xZYmqa0k1uWQdwmv/d5ZlGZ2naHvw+TfKyh8UboaPR
	dk=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [XEN PATCH v3 1/6] x86/vpmu: separate amd/intel vPMU code
Date: Tue, 14 May 2024 11:18:25 +0300
Message-Id: <1deef9f4c7b362d8a0a33d0908293735953a066f.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 8A16083A-11CA-11EF-A903-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Build AMD vPMU when CONFIG_AMD is on, and Intel vPMU when CONFIG_INTEL
is on respectively, allowing for a plaftorm-specific build.

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>

---
changes in v3:
 - neither of the blank lines dropped
changes in v2:
 - drop static inline stubs, use #idef/#endif in vpmu_init)()
changes in v1:
 - switch to CONFIG_{AMD,INTEL} instead of CONFIG_{SVM,VMX}
---
 xen/arch/x86/cpu/Makefile | 4 +++-
 xen/arch/x86/cpu/vpmu.c   | 4 ++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/cpu/Makefile b/xen/arch/x86/cpu/Makefile
index 35561fe51d..eafce5f204 100644
--- a/xen/arch/x86/cpu/Makefile
+++ b/xen/arch/x86/cpu/Makefile
@@ -10,4 +10,6 @@ obj-y +=3D intel.o
 obj-y +=3D intel_cacheinfo.o
 obj-y +=3D mwait-idle.o
 obj-y +=3D shanghai.o
-obj-y +=3D vpmu.o vpmu_amd.o vpmu_intel.o
+obj-y +=3D vpmu.o
+obj-$(CONFIG_AMD) +=3D vpmu_amd.o
+obj-$(CONFIG_INTEL) +=3D vpmu_intel.o
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index b2e9881e06..a7bc0cd1fc 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -827,6 +827,7 @@ static int __init cf_check vpmu_init(void)
=20
     switch ( vendor )
     {
+#ifdef CONFIG_AMD
     case X86_VENDOR_AMD:
         ops =3D amd_vpmu_init();
         break;
@@ -834,10 +835,13 @@ static int __init cf_check vpmu_init(void)
     case X86_VENDOR_HYGON:
         ops =3D hygon_vpmu_init();
         break;
+#endif
=20
+#ifdef CONFIG_INTEL
     case X86_VENDOR_INTEL:
         ops =3D core2_vpmu_init();
         break;
+#endif
=20
     default:
         printk(XENLOG_WARNING "VPMU: Unknown CPU vendor: %d. "
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:20:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:20:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721220.1124451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nOd-0005rh-Dn; Tue, 14 May 2024 08:20:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721220.1124451; Tue, 14 May 2024 08:20:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nOd-0005ra-BE; Tue, 14 May 2024 08:20:27 +0000
Received: by outflank-mailman (input) for mailman id 721220;
 Tue, 14 May 2024 08:20:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6nOb-0005rU-Ml
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:20:25 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d007297b-11ca-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:20:24 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-5231efd80f2so2620452e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:20:24 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b0172fsm693066766b.183.2024.05.14.01.20.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:20:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d007297b-11ca-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715674824; x=1716279624; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9Zgh8G1aqBfIdEFiPl2LajYHgmz7y437ztgTEUef/IM=;
        b=daquiFQaPGlx+VN8xoiSUI8Sz7yjl43ZVCh3XqCIzU9nHEBPQn4QG4dP1r0md9yQpm
         sQUHZKliCNipt8pUAxU0/dCL3hhxk+x9+c1P0UP5KnSiAKuIvbiSXOyLOvsxbNv/9OSE
         tQDhplck/hqPLh8H7afyu94JLdiY3Qfz/9Wfq8bo0OGo3xwJOkq9BGbPpJ8qUPYy9mwb
         Bx3U71SRzYRsBXsjbEnqn/qw1TVBD+PU0NfBAfiMMI2vRrLXp8/A18QpzVuJEB78ho5d
         ivYGyJqezYlu6IBvRl0j6BuoruhVqDWkxZu8cDwOzApqLWpIsBfuLGl960tlpzyrdk1T
         ZGmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715674824; x=1716279624;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9Zgh8G1aqBfIdEFiPl2LajYHgmz7y437ztgTEUef/IM=;
        b=GGDbddsK+v10MPnqeqtvUJPzGI+s9/dAYVCnMvajxRt/AR7rLeh8rGpwdRbrcxuk3o
         /+4VygNj6pCmF1z90Lu/3m0bzNXWT2deMJ3UpYXfP5onzygZUCqsH4dmBP88JvvsW+ZY
         wFzDTklwMczH5NVTgNSoPn1PAUuQwFM1embKR41ZZNTpAiA+46iLuY/GYmDUQxm/TKRN
         GCpUhSnR3Rs7MTX/7yBquEEug9Zrss7oj1mGysEk1nvpRLm1UViylUHFr+5IcDdzM0jv
         9JDPXOzwAXio2ZsdMDCECpz4oQtNo2DzVrW9iahIMq4h5mv/A6c0lU6qzZBaQxJnRQIf
         faVQ==
X-Gm-Message-State: AOJu0YxMaS1fJMzXGkWrlm96AYiRwF6u+8S49l5LpGapf5e6j43CxmSB
	WGvZ/Q5p03Ns5hclD3LkDO25I4JwtEl4uR0iiXlB0TqGgwlzqOKyfL945G1DdA==
X-Google-Smtp-Source: AGHT+IGEHAN1vsFQid7mIRKMTBlXUy5mOBAdf4XSnOrKUDI4Jj4zpPMB0W3XhUGZjzGjbDwQb2a8jA==
X-Received: by 2002:a05:6512:33c5:b0:522:2a2c:759e with SMTP id 2adb3069b0e04-5222a2c76c2mr8441709e87.6.1715674823897;
        Tue, 14 May 2024 01:20:23 -0700 (PDT)
Message-ID: <0b95b04d-22c1-4ec9-8d3e-a4974e7610bc@suse.com>
Date: Tue, 14 May 2024 10:20:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Serious AMD-Vi(?) issue
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>, Elliott Mitchell <ehem+xen@m5p.com>
References: <e9b1c9c4-523b-481b-946e-37c7c18ea1d2@suse.com>
 <ZgHwEGCsCLHiYU5J@mattapan.m5p.com> <ZgRXHQpamLIdu7dk@mattapan.m5p.com>
 <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com> <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkHTC4RpUSpKj4wf@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.05.2024 10:44, Roger Pau Monné wrote:
> On Fri, May 10, 2024 at 09:09:54PM -0700, Elliott Mitchell wrote:
>> On Thu, Apr 18, 2024 at 09:33:31PM -0700, Elliott Mitchell wrote:
>>>
>>> I suspect this is a case of there is some step which is missing from
>>> Xen's IOMMU handling.  Perhaps something which Linux does during an early
>>> DMA setup stage, but the current Xen implementation does lazily?
>>> Alternatively some flag setting or missing step?
>>>
>>> I should be able to do another test approach in a few weeks, but I would
>>> love if something could be found sooner.
>>
>> Turned out to be disturbingly easy to get the first entry when it
>> happened.  Didn't even need `dbench`, it simply showed once the OS was
>> fully loaded.  I did get some additional data points.
>>
>> Appears this requires an AMD IOMMUv2.  A test system with known
>> functioning AMD IOMMUv1 didn't display the issue at all.
>>
>> (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr fffffffdf8000000 flags 0x8 I
> 
> I would expect the address field to contain more information about the
> fault, but I'm not finding any information on the AMD-Vi specification
> apart from that it contains the DVA, which makes no sense when the
> fault is caused by an interrupt.

Isn't the address above in the "magic" HT range (and hence still meaningful
as an address)?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:20:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:20:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721221.1124462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nOj-00069R-LB; Tue, 14 May 2024 08:20:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721221.1124462; Tue, 14 May 2024 08:20:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nOj-00069K-Ha; Tue, 14 May 2024 08:20:33 +0000
Received: by outflank-mailman (input) for mailman id 721221;
 Tue, 14 May 2024 08:20:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nOi-0005rU-SE
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:20:32 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d3fac39d-11ca-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:20:31 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id BBFB52C412;
 Tue, 14 May 2024 04:20:30 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id B2F2A2C411;
 Tue, 14 May 2024 04:20:30 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id CD3482C410;
 Tue, 14 May 2024 04:20:29 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3fac39d-11ca-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=KEoZJ3ChXpAiFd7BWBK3KL6eW
	PK5BEYxIe+pdjJWFys=; b=gzaZXJpLENUGgDM0xBcRY9iqBUCnu/W/4g3hKSbYC
	e044cikyROYigZv3icqup4fQFXZUT1I5au323iBQE7BIK99lE5FGoJzPiiicO1/N
	czWi5Oezuy39R8iFByjujTCi8RxtHobuFdzLw2fLGLYOkc7PlAELeBDzBKcJR4Ex
	aA=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to header
Date: Tue, 14 May 2024 11:20:27 +0300
Message-Id: <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 D3578E1A-11CA-11EF-BD28-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Moving this function out of mce_intel.c would make it possible to disable
build of Intel MCE code later on, because the function gets called from
common x86 code.

Also replace boilerplate code that checks for MCG_LMCE_P flag with
vmce_has_lmce(), which might contribute to readability a bit.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v3:
 - do not check for CONFIG_INTEL
 - remove CONFIG_INTEL from patch description
changes in v2:
 - move vmce_has_lmce() to cpu/mcheck/mce.h
 - move IS_ENABLED(CONFIG_INTEL) check inside vmce_has_lmce()
 - changed description
---
 xen/arch/x86/cpu/mcheck/mce.h       | 5 +++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ----
 xen/arch/x86/cpu/mcheck/vmce.c      | 5 ++---
 xen/arch/x86/include/asm/mce.h      | 1 -
 xen/arch/x86/msr.c                  | 2 ++
 5 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/mce.h b/xen/arch/x86/cpu/mcheck/mce.=
h
index 4806405f96..eba4b536c7 100644
--- a/xen/arch/x86/cpu/mcheck/mce.h
+++ b/xen/arch/x86/cpu/mcheck/mce.h
@@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v,=
 uint32_t msr)
     return 0;
 }
=20
+static inline bool vmce_has_lmce(const struct vcpu *v)
+{
+    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
+}
+
 struct mce_callbacks {
     void (*handler)(const struct cpu_user_regs *regs);
     bool (*check_addr)(uint64_t status, uint64_t misc, int addr_type);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mchec=
k/mce_intel.c
index 3f5199b531..af43281cc6 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -1050,7 +1050,3 @@ int vmce_intel_rdmsr(const struct vcpu *v, uint32_t=
 msr, uint64_t *val)
     return 1;
 }
=20
-bool vmce_has_lmce(const struct vcpu *v)
-{
-    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
-}
diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmc=
e.c
index 353d4f19b2..94d1f021e1 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -199,7 +199,7 @@ int vmce_rdmsr(uint32_t msr, uint64_t *val)
          * bits are always set in guest MSR_IA32_FEATURE_CONTROL by Xen,=
 so it
          * does not need to check them here.
          */
-        if ( cur->arch.vmce.mcg_cap & MCG_LMCE_P )
+        if ( vmce_has_lmce(cur) )
         {
             *val =3D cur->arch.vmce.mcg_ext_ctl;
             mce_printk(MCE_VERBOSE, "MCE: %pv: rd MCG_EXT_CTL %#"PRIx64"=
\n",
@@ -324,8 +324,7 @@ int vmce_wrmsr(uint32_t msr, uint64_t val)
         break;
=20
     case MSR_IA32_MCG_EXT_CTL:
-        if ( (cur->arch.vmce.mcg_cap & MCG_LMCE_P) &&
-             !(val & ~MCG_EXT_CTL_LMCE_EN) )
+        if ( vmce_has_lmce(cur) && !(val & ~MCG_EXT_CTL_LMCE_EN) )
             cur->arch.vmce.mcg_ext_ctl =3D val;
         else
             ret =3D -1;
diff --git a/xen/arch/x86/include/asm/mce.h b/xen/arch/x86/include/asm/mc=
e.h
index 6ce56b5b85..2ec47a71ae 100644
--- a/xen/arch/x86/include/asm/mce.h
+++ b/xen/arch/x86/include/asm/mce.h
@@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
 extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu =
*ctxt);
 extern int vmce_wrmsr(uint32_t msr, uint64_t val);
 extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
-extern bool vmce_has_lmce(const struct vcpu *v);
 extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);
=20
 DECLARE_PER_CPU(unsigned int, nr_mce_banks);
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 9babd441f9..b0ec96f021 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -24,6 +24,8 @@
=20
 #include <public/hvm/params.h>
=20
+#include "cpu/mcheck/mce.h"
+
 DEFINE_PER_CPU(uint32_t, tsc_aux);
=20
 int init_vcpu_msr_policy(struct vcpu *v)
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:21:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:21:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721224.1124472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nPI-00073r-UP; Tue, 14 May 2024 08:21:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721224.1124472; Tue, 14 May 2024 08:21:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nPI-00073g-Qn; Tue, 14 May 2024 08:21:08 +0000
Received: by outflank-mailman (input) for mailman id 721224;
 Tue, 14 May 2024 08:21:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6nPH-0006V6-N7
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:21:07 +0000
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com
 [2607:f8b0:4864:20::f35])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e6b5b63e-11ca-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:21:03 +0200 (CEST)
Received: by mail-qv1-xf35.google.com with SMTP id
 6a1803df08f44-69b59c10720so26600506d6.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:21:04 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1ff4f2sm51205216d6.134.2024.05.14.01.21.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 01:21:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6b5b63e-11ca-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715674864; x=1716279664; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GIMBNJDqAwbir+r9DLchpcuwfLcGZ29LEOsj57pQAcM=;
        b=Q16WWkC8N6njOrqxAUT+gV7wAPokNtqsF/mXmVkmafwGF/GfrY+WEuUqdXL28NjN0H
         o/C0gtOs6XH2xsKIlYnKTOpj/Sg1xUW0BticyF0WEHthohH4fwQzWWyX+YcHOPMez/cH
         TQi7xUj/fOKCb6tBEqYrGrN0uomm3cNc8CNOQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715674864; x=1716279664;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GIMBNJDqAwbir+r9DLchpcuwfLcGZ29LEOsj57pQAcM=;
        b=SOZLPvWUN6+sXobh8iv6vqKedz40k8HyFz3PjU9GROKxAxSc2jGMOHfSzD92UOepPX
         Li/dEvAK13yhxNRsWV4OZC7nkP+UbYyWG4TinJiIer1HYewmyen3GjHagb2jYs9PYuTG
         PcmLqvd8RHvQVwDeP/SR8KHRts2rOqLy9fi7tfDzesrVQpp2TMYyCvCT9WLDEbIiuxbh
         q/i4WxI6EdiB3xpTLdMoYevtfbK1PXeqftEIScAbQULrirL7byVY+GyUImW/+VumNUiC
         R0nL0s2KNz8J3/PL48c1z31wcsiUJLQm2EGozSl72xzXpdbkG4qusajzOLH4NWcSQC2r
         C51w==
X-Gm-Message-State: AOJu0Yw/5BhOl2E5PR3ehyl0mQhF7VSLj4IOvR54ZJN5pVL2FkX+mJ6o
	/geYT7YMZ+EvW4i2XqIBIxaqSUw4zTVGNVp44NOMjFpQpW2kS3ByVyPGCVxsg61nWC+Kjcn7z4T
	0
X-Google-Smtp-Source: AGHT+IEmpXF/rH4PMvzksUdWu+iUzOlbETi2ejGRE3LiQcWB6IUCegjFf+p0i229qLnS/iyiLoUIHw==
X-Received: by 2002:a05:6214:5b8a:b0:6a0:b905:97ba with SMTP id 6a1803df08f44-6a168213d8amr163333516d6.44.1715674863730;
        Tue, 14 May 2024 01:21:03 -0700 (PDT)
Date: Tue, 14 May 2024 10:21:01 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Wei Liu <wei.liu2@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Wang <wawei@amazon.de>, Hongyan Xia <hongyxia@amazon.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 04/19] x86: Lift mapcache variable to the
 arch level
Message-ID: <ZkMe7YxnpHdQ74R3@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-5-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240513134046.82605-5-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:31PM +0000, Elias El Yandouzi wrote:
> From: Wei Liu <wei.liu2@citrix.com>
> 
> It is going to be needed by HVM and idle domain as well, because without
> the direct map, both need a mapcache to map pages.

The idle domain is a PV domain, and so gets the mapcache already
initialized with the current code?

> 
> This commit lifts the mapcache variable up and initialise it a bit earlier
> for PV and HVM domains.

It would be good to write down why mapcache was only used for PV, so
to understand it is safe to use for HVM also.

> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> Signed-off-by: Wei Wang <wawei@amazon.de>
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:22:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:22:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721229.1124482 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nQk-00085o-C0; Tue, 14 May 2024 08:22:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721229.1124482; Tue, 14 May 2024 08:22:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nQk-00085h-7q; Tue, 14 May 2024 08:22:38 +0000
Received: by outflank-mailman (input) for mailman id 721229;
 Tue, 14 May 2024 08:22:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nQi-00085M-Od
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:22:36 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c6266cb-11cb-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:22:32 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id EE5BA2C41E;
 Tue, 14 May 2024 04:22:33 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id E536F2C41D;
 Tue, 14 May 2024 04:22:33 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 388772C41C;
 Tue, 14 May 2024 04:22:33 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c6266cb-11cb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=nwJV6QxpfW1VT27LacPTJtctg
	XhxVUKxAohHmFS/6/8=; b=l1PZlyraEegQH1RDFglOeRqO1bBQmuYIvdTzywUV8
	883BMOSgeaVdATiPuQmfK0qaAQmo2uUvL/MPa4/DqpDzR4rwBwN5fd8vTmpc/rdb
	E7J08pHmX5jVJodHZwZJjEgQctZnkm1uQQHa3cZT+eOCea2qrls/oXVVEB1Qeans
	KU=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 3/6] x86/MCE: guard access to Intel/AMD-specific MCA MSRs
Date: Tue, 14 May 2024 11:22:30 +0300
Message-Id: <15b01508ca14265f643ddfb508481012c21a8c81.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 1CE33192-11CB-11EF-B574-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Add build-time checks for newly introduced INTEL/AMD config options when
calling vmce_{intel/amd}_{rdmsr/wrmsr}() routines.
This way a platform-specific code can be omitted in vmce code, if this
platform is disabled in config.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
changes in v3:
 - neither of the blank lines dropped
changes in v2:
 - use #ifdef/#endif in switch instead of IS_ENABLED
 - fallback to returning default 0 if vendor not recognized
---
 xen/arch/x86/cpu/mcheck/vmce.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmc=
e.c
index 94d1f021e1..5abdf4cb5f 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -138,16 +138,20 @@ static int bank_mce_rdmsr(const struct vcpu *v, uin=
t32_t msr, uint64_t *val)
     default:
         switch ( boot_cpu_data.x86_vendor )
         {
+#ifdef CONFIG_INTEL
         case X86_VENDOR_CENTAUR:
         case X86_VENDOR_SHANGHAI:
         case X86_VENDOR_INTEL:
             ret =3D vmce_intel_rdmsr(v, msr, val);
             break;
+#endif
=20
+#ifdef CONFIG_AMD
         case X86_VENDOR_AMD:
         case X86_VENDOR_HYGON:
             ret =3D vmce_amd_rdmsr(v, msr, val);
             break;
+#endif
=20
         default:
             ret =3D 0;
@@ -271,14 +275,18 @@ static int bank_mce_wrmsr(struct vcpu *v, uint32_t =
msr, uint64_t val)
     default:
         switch ( boot_cpu_data.x86_vendor )
         {
+#ifdef CONFIG_INTEL
         case X86_VENDOR_INTEL:
             ret =3D vmce_intel_wrmsr(v, msr, val);
             break;
+#endif
=20
+#ifdef CONFIG_AMD
         case X86_VENDOR_AMD:
         case X86_VENDOR_HYGON:
             ret =3D vmce_amd_wrmsr(v, msr, val);
             break;
+#endif
=20
         default:
             ret =3D 0;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:22:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:22:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721230.1124491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nQt-0008NU-HZ; Tue, 14 May 2024 08:22:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721230.1124491; Tue, 14 May 2024 08:22:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nQt-0008NN-EW; Tue, 14 May 2024 08:22:47 +0000
Received: by outflank-mailman (input) for mailman id 721230;
 Tue, 14 May 2024 08:22:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6nQs-00085M-Dz
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:22:46 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 228be3f5-11cb-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:22:42 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so49610166b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:22:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a60eabd3csm302014266b.108.2024.05.14.01.22.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:22:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 228be3f5-11cb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715674964; x=1716279764; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4ghA2R+lU03Q6G/NuBzkmc5Rce/7J7EYxK/x13WwjxI=;
        b=GnXdAx2Yj7UXW5LNCByLGJj92VDMHiEj8Gy65jflATcqiNgETyxy044wOMvC8aCUI6
         CDBsn0Cia1yIY9NTXnRF+Nwl+wCquFNkZ+TMowVBWaLliKROQ97RCeOo4DysZdSfEOwa
         uupl65NCEH9TO6lK3+j5hgLPT4GsJw2W7u82houT6WF0E8I/v2Oq6g51M7JPQhoJzIOB
         JHHYkMQ8J3xQw1I3vAL8GFNMOjxDXYb+3mTSKpMmAxGsOoa8En3FWdQzerCYp4rSMMtE
         aRAegKq8gg6AUnjYx2crkfml0zrDET7mmxuShOXREtWsnCJrDSr1JVLjPsPX7PzGr+UI
         ZDpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715674964; x=1716279764;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4ghA2R+lU03Q6G/NuBzkmc5Rce/7J7EYxK/x13WwjxI=;
        b=WdTej+9jBDlAc6ISHXyMfTIrGv70jkKunK/P/3oMLnADMQ5Q/o6KChiWLqROM4HPwI
         fLmXSxa8NT8T/xbwROqW67bEk0YcSBZ3ohyfFiLVkZuZiw2NE0yWgqhAfTMKVZ3fyD8m
         4ixP1R3F9JiMZyPERz/GEsdSWax3biI5uksnTAiY9qJf3/FCzlVzxmhPNI4TQoqByyX7
         Dz3/0MvtOYoLTnEQQxfAR1A/w4IYPOeOeM2p2ke1f18d7Xm8eHyjFPm2xmDLYxE6c6ox
         t0trjv4/y5gSjPlJudkUQoS/I8Hu5sMQynwU5oYworhE689mUZe6Spl1l8J4kI+SESMM
         aHvw==
X-Gm-Message-State: AOJu0YxY9KY1hpAVryYITY/9fVD0CD/Q2a7o+TTcMapzMQb2eOn9VjXX
	lOVPiXR2Rgl8qZoAtFfAemX+rOcT3/XPjBs/hb8Qj/rb5aCVJZDGSRLWyhG1NA==
X-Google-Smtp-Source: AGHT+IGsxV3iQ+uyEOLYKcZoewiKqXDRnuwJVl01OOdOF83YbKFALYxkJUP5uvD9DyCibK6skmIJSA==
X-Received: by 2002:a17:906:da8b:b0:a59:bdb7:73f8 with SMTP id a640c23a62f3a-a5a2d66a3b4mr1040337666b.47.1715674964315;
        Tue, 14 May 2024 01:22:44 -0700 (PDT)
Message-ID: <08503965-4ba9-4fe3-a176-895623f4ad92@suse.com>
Date: Tue, 14 May 2024 10:22:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Serious AMD-Vi(?) issue
Content-Language: en-US
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <ZgHwEGCsCLHiYU5J@mattapan.m5p.com>
 <ZgRXHQpamLIdu7dk@mattapan.m5p.com>
 <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com> <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook> <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.05.2024 22:11, Elliott Mitchell wrote:
> On Mon, May 13, 2024 at 10:44:59AM +0200, Roger Pau Monné wrote:
>> Why do you mask the device SBDF in the above snippet?  I would really
>> like to understand what's so privacy relevant in a PCI SBDF number.
> 
> I doubt it reveals much.  Simply seems unlikely to help debugging and
> therefore I prefer to mask it.

SBDF in one place may be matchable against a memory address in another
place. _Any_ hiding of information is hindering analysis. Please can
you finally accept that it needs to be the person doing the analysis
to judge what is or is not relevant to them?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:24:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:24:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721234.1124501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nSh-0000iq-Sg; Tue, 14 May 2024 08:24:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721234.1124501; Tue, 14 May 2024 08:24:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nSh-0000ij-Q8; Tue, 14 May 2024 08:24:39 +0000
Received: by outflank-mailman (input) for mailman id 721234;
 Tue, 14 May 2024 08:24:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nSg-0000iZ-Cc
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:24:38 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 66826b46-11cb-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:24:37 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 968412C438;
 Tue, 14 May 2024 04:24:36 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 8E7272C437;
 Tue, 14 May 2024 04:24:36 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id AC54B2C436;
 Tue, 14 May 2024 04:24:35 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66826b46-11cb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=DOe4Y1iXkxyUp7tFZfnk+wWTi
	6MegcBusy617K955Vw=; b=dNnzQyHHXWisF2UCubKxGdLgpD9nzaxPIPYosfrfM
	s66sjbB3m2gP4ksCMxBS0LM600DhTUOMrCXG19R0A/Tb0cMxhWSKkhlKMJwTC/6I
	kVXYckDxmwS7mLtc5jhee8Fgrx+J9kexgsBbtT0t5Mp3AUW/DuOPaS78GvGh4kU1
	fk=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 4/6] x86/MCE: guard {intel/amd}_mcheck_init() calls
Date: Tue, 14 May 2024 11:24:34 +0300
Message-Id: <f79e54859b54e3fdc4aec1f41365169f4428cb8f.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 65E4360C-11CB-11EF-B5B5-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Guard calls to CPU-specific mcheck init routines in common MCE code
using new INTEL/AMD config options.

The purpose is not to build platform-specific mcheck code and calls to it=
,
if this platform is disabled in config.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
changes in v3:
- neither of the blank lines dropped
changes in v2:
 - use #ifdef/#endif in switch instead of IS_ENABLED
---
 xen/arch/x86/cpu/mcheck/mce.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.=
c
index d179e6b068..fb9dec5b89 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -760,11 +760,14 @@ void mcheck_init(struct cpuinfo_x86 *c, bool bsp)
=20
     switch ( c->x86_vendor )
     {
+#ifdef CONFIG_AMD
     case X86_VENDOR_AMD:
     case X86_VENDOR_HYGON:
         inited =3D amd_mcheck_init(c, bsp);
         break;
+#endif
=20
+#ifdef CONFIG_INTEL
     case X86_VENDOR_INTEL:
         switch ( c->x86 )
         {
@@ -774,6 +777,7 @@ void mcheck_init(struct cpuinfo_x86 *c, bool bsp)
             break;
         }
         break;
+#endif
=20
     default:
         break;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:26:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:26:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721244.1124511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nUh-0001HF-6k; Tue, 14 May 2024 08:26:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721244.1124511; Tue, 14 May 2024 08:26:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nUh-0001H8-45; Tue, 14 May 2024 08:26:43 +0000
Received: by outflank-mailman (input) for mailman id 721244;
 Tue, 14 May 2024 08:26:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nUg-0001Fw-IG
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:26:42 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aed7ce1c-11cb-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:26:39 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 9A8773A519;
 Tue, 14 May 2024 04:26:39 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 925253A518;
 Tue, 14 May 2024 04:26:39 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id AC8CB3A517;
 Tue, 14 May 2024 04:26:38 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aed7ce1c-11cb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=GoMafCLne/uUKmiYB7IOb56hQ
	84XAcKbV1FNoVwd924=; b=kWgqULECWj7S5aPYR9geqJ712VegbpJ30HHUx9wt0
	CudWqLcV13ohBJ8nf8myrELsuzJDcLaLwMd8kN53D7FIThuh/BWW+tP7UWs0SjaA
	P2EoIsSnWQvlh3FNtmCk0feTjhVEhgMYPvPrAg7/6f1n/kctj7+O3c8HhxA9GYuw
	UU=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 5/6] x86/MCE: add default switch case in init_nonfatal_mce_checker()
Date: Tue, 14 May 2024 11:26:36 +0300
Message-Id: <01aad263ad60f37ed74138b5abf2733361bb7d25.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 AF33E1E0-11CB-11EF-B499-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

The default switch case block is likely wanted here, to handle situation
e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, bu=
t
misleading message still gets logged anyway.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/x86/cpu/mcheck/non-fatal.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mchec=
k/non-fatal.c
index 33cacd15c2..c5d8e3aea9 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -29,9 +29,13 @@ static int __init cf_check init_nonfatal_mce_checker(v=
oid)
 		/* Assume we are on K8 or newer AMD or Hygon CPU here */
 		amd_nonfatal_mcheck_init(c);
 		break;
+
 	case X86_VENDOR_INTEL:
 		intel_nonfatal_mcheck_init(c);
 		break;
+
+	default:
+		return -ENODEV;
 	}
 	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n")=
;
 	return 0;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:28:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:28:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721247.1124522 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nWk-0001qe-IL; Tue, 14 May 2024 08:28:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721247.1124522; Tue, 14 May 2024 08:28:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nWk-0001qX-Fc; Tue, 14 May 2024 08:28:50 +0000
Received: by outflank-mailman (input) for mailman id 721247;
 Tue, 14 May 2024 08:28:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XrDB=MR=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s6nWj-0001qM-Pe
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:28:49 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbb6f8a2-11cb-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:28:48 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 714143ABE3;
 Tue, 14 May 2024 04:28:46 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 697223ABE2;
 Tue, 14 May 2024 04:28:46 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 2047F3ABE1;
 Tue, 14 May 2024 04:28:43 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbb6f8a2-11cb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=V/rp56yIvzwZLQTcJkKC13OhS
	ElWeNBBG3Y1G7bl8LU=; b=FX2BaCLA/Gq0MJ8Cs1ZR+MHETL5gc93Jc3p0/Issb
	3c04s2Q/HEWrFeVdipup8iXCbyOIH+dFHvDnPY9e9owRGIGwld3Opj2ImqwwTKJ9
	azu3C7xksbbu6EPs1FmDkzEidWadI3oSLqcB8/DVSkGQT/aaHZHJcax95JDlmZOt
	bI=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v3 6/6] x86/MCE: optional build of AMD/Intel MCE code
Date: Tue, 14 May 2024 11:28:39 +0300
Message-Id: <def311f3206f29f919a921d459442641fc293031.1715673586.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 F95E0F20-11CB-11EF-ADA0-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

Separate Intel/AMD-specific MCE code using CONFIG_{INTEL,AMD} config opti=
ons.
Now we can avoid build of mcheck code if support for specific platform is
intentionally disabled by configuration.

Also global variables lmce_support & cmci_support from Intel-specific
mce_intel.c have to moved to common mce.c, as they get checked in common =
code.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v3:
 - default return value of init_nonfatal_mce_checker() done in separate p=
atch
 - move lmce_support & cmci_support to common mce.c code
 - changed patch description
changes in v2:
 - fallback to original ordering in Makefile
 - redefine lmce_support & cmci_support global vars to false when !INTEL
 - changed patch description
---
 xen/arch/x86/cpu/mcheck/Makefile    | 8 ++++----
 xen/arch/x86/cpu/mcheck/mce.c       | 4 ++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ----
 xen/arch/x86/cpu/mcheck/non-fatal.c | 4 ++++
 4 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/Makefile b/xen/arch/x86/cpu/mcheck/M=
akefile
index f927f10b4d..e6cb4dd503 100644
--- a/xen/arch/x86/cpu/mcheck/Makefile
+++ b/xen/arch/x86/cpu/mcheck/Makefile
@@ -1,12 +1,12 @@
-obj-y +=3D amd_nonfatal.o
-obj-y +=3D mce_amd.o
+obj-$(CONFIG_AMD) +=3D amd_nonfatal.o
+obj-$(CONFIG_AMD) +=3D mce_amd.o
 obj-y +=3D mcaction.o
 obj-y +=3D barrier.o
-obj-y +=3D intel-nonfatal.o
+obj-$(CONFIG_INTEL) +=3D intel-nonfatal.o
 obj-y +=3D mctelem.o
 obj-y +=3D mce.o
 obj-y +=3D mce-apei.o
-obj-y +=3D mce_intel.o
+obj-$(CONFIG_INTEL) +=3D mce_intel.o
 obj-y +=3D non-fatal.o
 obj-y +=3D util.o
 obj-y +=3D vmce.o
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.=
c
index fb9dec5b89..32c1b2756b 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -38,6 +38,10 @@ DEFINE_PER_CPU_READ_MOSTLY(unsigned int, nr_mce_banks)=
;
 unsigned int __read_mostly firstbank;
 unsigned int __read_mostly ppin_msr;
 uint8_t __read_mostly cmci_apic_vector;
+bool __read_mostly cmci_support;
+
+/* If mce_force_broadcast =3D=3D 1, lmce_support will be disabled forcib=
ly. */
+bool __read_mostly lmce_support;
=20
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, poll_bankmask);
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, no_cmci_banks);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mchec=
k/mce_intel.c
index af43281cc6..dd812f4b8a 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -26,16 +26,12 @@
 #include "mcaction.h"
=20
 static DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, mce_banks_owned);
-bool __read_mostly cmci_support;
 static bool __read_mostly ser_support;
 static bool __read_mostly mce_force_broadcast;
 boolean_param("mce_fb", mce_force_broadcast);
=20
 static int __read_mostly nr_intel_ext_msrs;
=20
-/* If mce_force_broadcast =3D=3D 1, lmce_support will be disabled forcib=
ly. */
-bool __read_mostly lmce_support;
-
 /* Intel SDM define bit15~bit0 of IA32_MCi_STATUS as the MC error code *=
/
 #define INTEL_MCCOD_MASK 0xFFFF
=20
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mchec=
k/non-fatal.c
index c5d8e3aea9..52e16ad499 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -24,15 +24,19 @@ static int __init cf_check init_nonfatal_mce_checker(=
void)
 	 * Check for non-fatal errors every MCE_RATE s
 	 */
 	switch (c->x86_vendor) {
+#ifdef CONFIG_AMD
 	case X86_VENDOR_AMD:
 	case X86_VENDOR_HYGON:
 		/* Assume we are on K8 or newer AMD or Hygon CPU here */
 		amd_nonfatal_mcheck_init(c);
 		break;
+#endif
=20
+#ifdef CONFIG_INTEL
 	case X86_VENDOR_INTEL:
 		intel_nonfatal_mcheck_init(c);
 		break;
+#endif
=20
 	default:
 		return -ENODEV;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:38:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:38:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721256.1124532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ngP-0004l1-HJ; Tue, 14 May 2024 08:38:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721256.1124532; Tue, 14 May 2024 08:38:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ngP-0004ku-DQ; Tue, 14 May 2024 08:38:49 +0000
Received: by outflank-mailman (input) for mailman id 721256;
 Tue, 14 May 2024 08:38:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6ngN-0004ko-QB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:38:47 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60eaf483-11cd-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:38:46 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so549410766b.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:38:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed72bbsm7243366a12.57.2024.05.14.01.38.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:38:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60eaf483-11cd-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715675926; x=1716280726; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=s0zy3Vgr26FzbIVrWDxEElw5PwRCEREU9VQ5sdqIaFg=;
        b=Y2FGxuG0DayNUglqgVGvPNqq+hy2sKCK5nwYaC4c7/irmEY/RKrFl+BeOpJ7AGGp7Q
         rmbezcLFkOFBV/hUDwJKaURl9V0PR7y/Uo7LqNiQThosZUBe2ikgMtLOCrmkWsItOQsJ
         e6/XokQZehYeX8FJRgLIV4wvpEXi4JeNOyCnneRoDPxLRjD8p+p5947uGErctPt39su3
         6Jxf2+d+Sr4zgeuDV/IFLGkfJgC/+trQoiV+bUGJWQH3yEszs5aa/+jpIaFEUP17IU6n
         XvekGzU29Lfm07NDYpnmr4sFIkIp2TGfOuvTRS/EW/JKcNjwPOR7blSqUVcWKw8i4+lc
         s/vw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715675926; x=1716280726;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=s0zy3Vgr26FzbIVrWDxEElw5PwRCEREU9VQ5sdqIaFg=;
        b=oAGgzCDjcJ4BUkZHU2YWryG2pPYahL/Ye84TWYTxg4arcANQAeLqyOhk+WRI95xX95
         lrQFtUl0oV7pgtSWmUA8c/+uGQDfg0XyxSdtGXLByliglz6KKXfqh+Fvyj0x7bjLOcXS
         Shx7Px9pJT4xFXkQyEqc+e+wsWJK6GOrNvueIfT+b/MXritJk/FSYHTwCH4vywkntv8N
         52s79RnYP9T4yLxY+w8SX+AullnnoHReJgZ0OGJIS1Bx3xKJHF8d63Gy7pm1rE48WLGz
         15ICYZQH4pAUCl9H4CJcFt09OLR27zkGEGuribTC7KDzg5yVtQjaWLIoYUakh40wb/c4
         1HQg==
X-Gm-Message-State: AOJu0YzVygqzKBYq+93p1XL3MsXkRo6sCG9R8T9AuxZN6aRs1xvjyg6B
	PAZdZS77Kzxx+BJyJQdM0jD9U59y2jEWCLSHMeKHN2Y8Y5VRD3wUpcKD+U9FTw==
X-Google-Smtp-Source: AGHT+IF3zNqyloY34FyQc7U5Va/1blGyVAfedKwdvwAQ8WlL4ucKPB9NZ3WmaBCdPOfAzeRUxFsY8w==
X-Received: by 2002:aa7:c884:0:b0:573:5c18:c2d5 with SMTP id 4fb4d7f45d1cf-5735c18c2dcmr7146085a12.3.1715675926194;
        Tue, 14 May 2024 01:38:46 -0700 (PDT)
Message-ID: <1250a87c-e6c4-4c57-a7d0-f1fbdf7bb136@suse.com>
Date: Tue, 14 May 2024 10:38:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] fix Rule 10.2 violation
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
 george.dunlap@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Julien Grall <julien@xen.org>
References: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop>
 <0625f4f2-daaa-4867-8c39-f8e9ce939ca0@xen.org>
 <alpine.DEB.2.22.394.2405131706450.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405131706450.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 02:09, Stefano Stabellini wrote:
> On Mon, 13 May 2024, Julien Grall wrote:
>> Hi Stefano,
>>
>> title: Is this the only violation we have in Xen? If so, then please add the
>> subsystem in the title.
> 
> The only remaining violations are about the use of the "toupper" macro.
> Bugseng is recommending to add a cast to fix those or deviate toupper.
> 
> 
>> On 11/05/2024 00:37, Stefano Stabellini wrote:
>>> Change opt_conswitch to char to fix a violation of Rule 10.2.
>>>
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>>>
>>> diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
>>> index 2c363d9c1d..3a3a97bcbe 100644
>>> --- a/xen/drivers/char/console.c
>>> +++ b/xen/drivers/char/console.c
>>> @@ -49,7 +49,7 @@ string_param("console", opt_console);
>>>   /* Char 1: CTRL+<char1> is used to switch console input between Xen and
>>> DOM0 */
>>>   /* Char 2: If this character is 'x', then do not auto-switch to DOM0 when
>>> it */
>>>   /*         boots. Any other value, or omitting the char, enables
>>> auto-switch */
>>> -static unsigned char __read_mostly opt_conswitch[3] = "a";
>>> +static char __read_mostly opt_conswitch[3] = "a";
>>
>> Looking at the rest of the code, we have:
>>
>> #define switch_code (opt_conswitch[0] - 'a' + 1)
>>
>> Can you confirm whether this is not somehow adding a new violation?
> 
> No, this patch is to fix a violation exactly there.

Right. With a suitable subject prefix added:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:42:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:42:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721262.1124542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nkK-0007Rp-42; Tue, 14 May 2024 08:42:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721262.1124542; Tue, 14 May 2024 08:42:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nkK-0007Ri-13; Tue, 14 May 2024 08:42:52 +0000
Received: by outflank-mailman (input) for mailman id 721262;
 Tue, 14 May 2024 08:42:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6nkI-0007Rc-GY
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:42:50 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f11e5182-11cd-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:42:48 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-420104e5336so16158395e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:42:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4200e518984sm118855725e9.23.2024.05.14.01.42.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 01:42:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f11e5182-11cd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715676168; x=1716280968; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=MhqhIL0gBwwdJARQnLt/zpiJ59NDlGpFbNldB6nMz6E=;
        b=aOIl+lKsmeh1l92Ti0yaTyKMc/ZD3fQp+hZQTPS4by30NCS9vyPYqqx7+3gshhPUYi
         yufmgz9jAC55FOal7+VG58JVuANqlKstmkHAM1ymLOdfzaa7vdElJN/riVzcCWSySei6
         RErpwyNSPgZPjB3fEBLbp0vv7d8A84G9TtSBo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715676168; x=1716280968;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=MhqhIL0gBwwdJARQnLt/zpiJ59NDlGpFbNldB6nMz6E=;
        b=A6jaF3RLJEgMiGZh1SDL0r/UK7oarT5jy1kEGUD0PoBLrTvBg3ZuqXdiIgPsZ+fIho
         sqHZNOxvsqBtuFZekTy7YYuYvSswE4vZ0kW2d1owIGf14S+xHHACCzDg5XWuggR8C8J5
         ouvbGyK8TW/b//x27/DmDqFbbSpy2Fzvu79XLI0J/ADlFUCnbqjRi9cufooIKDKFLRLc
         pQFsQcQrKe9EwAp8ROYFWTMpHtaTjQzaYu4ci6ep1i5vsv0a/slFyfDRBp+HhMZkAVad
         5hBbtepbZ144UkYLrbcLtDFSHwPML9oFA0Wn3yV81STtNNiStQxnBoMTZoVmmkxWZPti
         qBpA==
X-Gm-Message-State: AOJu0Yy4F3mscyByaeHWOowCv2o6urfnyZocOA16n8/M16fILmC7symv
	lNn2SwyrYVkjGQ18a6n+qouGAbVbwfNS1mKV0gln2QHTJW3xhjkjFaL4fxr6CZo=
X-Google-Smtp-Source: AGHT+IG0/SRvK8+UJM05wdjtjgfPTjZsG9KL+zPOb6MitkzsuqYaRk+1cx4yRturpjn/mkPAZhwKXg==
X-Received: by 2002:a05:600c:35cd:b0:420:1a72:69dd with SMTP id 5b1f17b1804b1-4201a726bfamr32640725e9.10.1715676167799;
        Tue, 14 May 2024 01:42:47 -0700 (PDT)
Date: Tue, 14 May 2024 10:42:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Wang <wawei@amazon.de>, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 05/19] x86/mapcache: Initialise the mapcache
 for the idle domain
Message-ID: <ZkMkBrf2G5dQllHc@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-6-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-6-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:32PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> In order to use the mapcache in the idle domain, we also have to
> populate its page tables in the PERDOMAIN region, and we need to move
> mapcache_domain_init() earlier in arch_domain_create().

Oh, so this is the reason for the remark on the previous commit
message: idle domain init gets short-circuited earlier in
arch_domain_create() and never gets to the mapcache_domain_init()
call.

> Note, commit 'x86: lift mapcache variable to the arch level' has
> initialised the mapcache for HVM domains. With this patch, PV, HVM,
> idle domains now all initialise the mapcache.
> 
> Signed-off-by: Wei Wang <wawei@amazon.de>
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
> 	Changes in V2:
>           * Free resources if mapcache initialisation fails
>           * Remove `is_idle_domain()` check from `create_perdomain_mappings()`

I'm not seeing any is_idle_domain() in create_perdomain_mapping(), and
neither anything removed by this patch.

> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index 507d704f16..3303bdb53e 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -758,9 +758,16 @@ int arch_domain_create(struct domain *d,
>  
>      spin_lock_init(&d->arch.e820_lock);
>  
> +    if ( (rc = mapcache_domain_init(d)) != 0)
> +    {
> +        free_perdomain_mappings(d);
> +        return rc;


What about all the error paths below here that don't use the fail
label, don't you need to also call free_perdomain_mappings() on them?

Or alternatively arrange the fail label to be suitable to be used this
early if it's not already the case.

> +    }
> +
>      /* Minimal initialisation for the idle domain. */
>      if ( unlikely(is_idle_domain(d)) )
>      {
> +        struct page_info *pg = d->arch.perdomain_l3_pg;

const?

>          static const struct arch_csw idle_csw = {
>              .from = paravirt_ctxt_switch_from,
>              .to   = paravirt_ctxt_switch_to,
> @@ -771,6 +778,9 @@ int arch_domain_create(struct domain *d,
>  
>          d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>  
> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);

Albeit I think you could just use d->arch.perdomain_l3_pg directly
here and avoid the local pg variable?

Would you mind adding:

/* Slot 260: Per-domain mappings. */

I wonder if it won't be better to just use init_xen_l4_slots() and
special case the idle domain in there, instead of open-coding the L4
population for the idle domain like this.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 08:46:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:46:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721265.1124552 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nnO-00084t-HB; Tue, 14 May 2024 08:46:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721265.1124552; Tue, 14 May 2024 08:46:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6nnO-00084m-EJ; Tue, 14 May 2024 08:46:02 +0000
Received: by outflank-mailman (input) for mailman id 721265;
 Tue, 14 May 2024 08:46:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6nnN-00084g-8U
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:46:01 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 63545bd7-11ce-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 10:46:00 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59ad12efe3so592668666b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:46:00 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c2c7d79sm7241850a12.59.2024.05.14.01.45.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:45:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63545bd7-11ce-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715676359; x=1716281159; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1XAdd1MYGx0CrayOzgBjMlQADEkkAwdXEBr5JB4DeYY=;
        b=AeEBc2RNVLUxfiNRfkKKzTNJWf6zqnBOL9R0HoBNrSyehjV8RGFQA8KjPEePnQhFD2
         5wXsOOhJEvZhChFXs2bbyt6EqIqcQr1Fta4dMnyRozLmivQj1+LpyOguesFw0z3T0Hvj
         DhUMUD8fEtNlfo4WwW5EF7C4A5zICUWo9xf1gSY1fhhxGlguVaqqPXg2M9gkxETeIojK
         piMvuER7lJytEPD3pzBF29Mx2YT0IsMgNyESp35xupOfR04qsQ0d45aObEGMQ5U/zjw0
         DuQgg3pQKWxKBJ165GEyuascsgBRW2YWn0LOQb5vA0N/SHooBSapgec+WYfzIa2lYDxJ
         ZTPA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715676359; x=1716281159;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1XAdd1MYGx0CrayOzgBjMlQADEkkAwdXEBr5JB4DeYY=;
        b=Ja9yEFjG61/wjTr3+K085SU7WRVG4Ds44O6Xzfzczr57zmM64ZSU8MwInC4GlkZKJv
         iqmKDuF9/G8Iu15znife4pfEdmWZ97VwkrvZFJa4CtG9WPLFIpnwaF7lYFMzPczKXpib
         7yVXU9z4/FA9FcRw5B/1R+Qh/h9fGOBpdHRNvlYjRGrqoAkSeO6opDIQgiVZz9+a3Lna
         5VbmBaog1JEsQ00lW7h/No1KmB/P6GteyRAmN/cyni3ZM9mM51IZpQ7yKKeobZx0Cyhr
         dJnR2r/p6jlZSMsYourmO9GSeFJe7N1afSgIoLvjKWblFWDWjlhSG7F2mO4/j0jx5SMh
         TAFA==
X-Forwarded-Encrypted: i=1; AJvYcCUuKze5M7jN2BvJv5BcyaNKIknjXsFKQGhFXTdE85QEFUWZHT6o6DwS7u5d4ZQ1O5r+U1xxIyAGUfDEMd0eAynPFZfQ02T1H0qghKuL+Es=
X-Gm-Message-State: AOJu0YwsZaiZvVfYkNZ/ixM5DIA1gQZpXHLYXjC+Ab3aEYt9cTNOKRxu
	S9s6jBXSSX4kzvuD4czSZrgc09oLpFlUI2vsTt5tr42FPEmwYHP/ULG9eAJT9g==
X-Google-Smtp-Source: AGHT+IEofKuRVyAilE/2E/qhxyXB1uW0PdcCJ4yIirxWw369HYdKDL19hk2XuaugodTuSroYp2tnTQ==
X-Received: by 2002:a50:ab12:0:b0:572:3f41:25aa with SMTP id 4fb4d7f45d1cf-5734d5c0eb9mr11269796a12.11.1715676359623;
        Tue, 14 May 2024 01:45:59 -0700 (PDT)
Message-ID: <16990192-aceb-408f-9247-45f3b1f58e0a@suse.com>
Date: Tue, 14 May 2024 10:46:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] lib/strtoul: fix MISRA R10.2 violation
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 02:32, Stefano Stabellini wrote:
> Fix last violation of R10.2 by casting the result of toupper to plain
> char. Note that we don't want to change toupper itself as it is a legacy
> interface and it would cause more issues.

Can you point me at a single example where a new issue would arise? All
places I've spotted (including tolower() uses) would appear to benefit
from changing toupper() / tolower() themselves. Further, since they are
both wrapper macros only anyway, if any concern remained, fiddling with
the wrapper macros while leaving alone the underlying inline functions
would allow any such use site to simply be switched to using the inline
functions directly. As said, from looking at it I don't expect that
would be necessary, so instead I'd rather hope that eventually we can
do away with the wrapper macros, renaming the inline functions
accordingly.

Jan

> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
> I believe this is the last R10.2 violation
> 
> diff --git a/xen/lib/strtoul.c b/xen/lib/strtoul.c
> index a378fe735e..345dcf9d8c 100644
> --- a/xen/lib/strtoul.c
> +++ b/xen/lib/strtoul.c
> @@ -38,7 +38,7 @@ unsigned long simple_strtoul(
>  
>      while ( isxdigit(*cp) &&
>              (value = isdigit(*cp) ? *cp - '0'
> -                                  : toupper(*cp) - 'A' + 10) < base )
> +                                  : (char)toupper(*cp) - 'A' + 10) < base )
>      {
>          result = result * base + value;
>          cp++;



From xen-devel-bounces@lists.xenproject.org Tue May 14 08:52:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 08:52:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721273.1124561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ntm-0002Pg-5B; Tue, 14 May 2024 08:52:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721273.1124561; Tue, 14 May 2024 08:52:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ntm-0002PZ-2b; Tue, 14 May 2024 08:52:38 +0000
Received: by outflank-mailman (input) for mailman id 721273;
 Tue, 14 May 2024 08:52:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6ntk-0002PT-Rq
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 08:52:36 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e839b2d-11cf-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 10:52:34 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59ad344f7dso1081327566b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 01:52:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d532sm707783866b.18.2024.05.14.01.52.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 01:52:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e839b2d-11cf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715676754; x=1716281554; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bpUeof2BRKXt8gLYUv3Oxcsa+wXdHkEgyAzBUX4ATaw=;
        b=NzNOJo/3h455CL6yto4Y548ACeINqn6hTqXCxoV9u9Bd9HlBBdx/etUxI/zHEowOx+
         GIqgx2jw8McnaqGdKUiy+SXpp4oMhUJckA1V+wPx6lLXLeXDeKuEMkKfeO80/h4Fu4AX
         JOPBAhwlTzzk0/9TTr4pExs/B1dALiVnrfLPaKtSp4XPa0sJRScaD+Fxw3Ksxjebm2Gc
         Qg1qaYHutzXL6mlkEz4mOb511w+HkF+hesvlpnLnTlkmo1ZhN4hXvoHElxbFTyf/j+2I
         /63ydqLOW65d0cqRoae5UCcj3y73DvyLsIMJJFk3q5E1hkKpITDToB67Nd7LfR60DGch
         T/Ew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715676754; x=1716281554;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bpUeof2BRKXt8gLYUv3Oxcsa+wXdHkEgyAzBUX4ATaw=;
        b=UwdKMD+oBlTih2T9TGMhWp4mvKfg4GP8APKzgw3gtjWEXffsuT+M98IwvwzQ2MX9X4
         ISU+g6R4RGFiB6vemF3g7U58EP0fWxDeshDzS04fN/mU3u7d44Nxrty3Tb1kiBOiRLqQ
         viP1GI34A1QHd8I8iWwVNJFEw0TINoQokehSNF0kQLGBNbs0bCO67ksmn48Bk3tUZPTY
         Ycyvk/PHhgjrEhdWd6SxC9F/833ZvvNgamhAHU9JDqkFH1ummx6G+em+m5oUjVJKCacr
         xNmZ+MNCGQxV5mrAocilSQR+oB4thilLo2BDMEQoXChLs8pqd6bv08oSNERTFRzAkmsQ
         wTbA==
X-Gm-Message-State: AOJu0YyUWG5zL+bWxu2dqi4tOtjKH+KB6Cdp23xDKoLOPfti5Mdgv0Po
	+DkocosyaTtQn0T1Wwc9FprrE8gvFTSWmigpYNMIMVUZ5EuQAHA+sfa3aOn54g==
X-Google-Smtp-Source: AGHT+IEYzzHTkwKzQ1lzgT9lxMOTn10kiZoTkq81ixW+4Gbakk1w09juw68/5tTJNDW0kLnu5big9g==
X-Received: by 2002:a17:906:4ecb:b0:a59:cdf4:f939 with SMTP id a640c23a62f3a-a5a2d54c60cmr835308066b.3.1715676754106;
        Tue, 14 May 2024 01:52:34 -0700 (PDT)
Message-ID: <36b89590-9bd9-4a64-9150-f2819bcd4a0e@suse.com>
Date: Tue, 14 May 2024 10:52:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpufreq: Rename cpuid variable/parameters to cpu
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240511194237.2596271-1-andrew.cooper3@citrix.com>
 <alpine.DEB.2.22.394.2405131723240.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405131723240.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 02:24, Stefano Stabellini wrote:
> On Sat, 11 May 2024, Andrew Cooper wrote:
>> --- a/xen/drivers/cpufreq/cpufreq.c
>> +++ b/xen/drivers/cpufreq/cpufreq.c
>> @@ -459,21 +459,21 @@ static void print_PPC(unsigned int platform_limit)
>>  
>>  int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
>>  {
>> -    int ret=0, cpuid;
>> +    int ret=0, cpu;

Would be nice if the missing blanks were added here as well, while touching
the line.

>>      struct processor_pminfo *pmpt;
>>      struct processor_performance *pxpt;
>>  
>> -    cpuid = get_cpu_id(acpi_id);
>> -    if ( cpuid < 0 || !perf )
>> +    cpu = get_cpu_id(acpi_id);
>> +    if ( cpu < 0 || !perf )
>>      {
>>          ret = -EINVAL;
>>          goto out;
>>      }
>>      if ( cpufreq_verbose )
>> -        printk("Set CPU acpi_id(%d) cpuid(%d) Px State info:\n",
>> +        printk("Set CPU acpi_id(%d) cpu(%d) Px State info:\n",
>>                 acpi_id, cpuid);
> 
> This cpuid should be changed as well?

And with that adjustment
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:06:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:06:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721279.1124572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6o6T-0005Gp-8t; Tue, 14 May 2024 09:05:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721279.1124572; Tue, 14 May 2024 09:05:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6o6T-0005Gi-5z; Tue, 14 May 2024 09:05:45 +0000
Received: by outflank-mailman (input) for mailman id 721279;
 Tue, 14 May 2024 09:05:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y506=MR=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s6o6S-0005Gc-Ky
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:05:44 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 233dddc5-11d1-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:05:42 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so1213876766b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:05:41 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f3dsm691814566b.123.2024.05.14.02.05.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:05:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 233dddc5-11d1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715677540; x=1716282340; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=S32SOD8xQbnu1UbjDRIfbFTfLCH36Ju2ZdHupMbk0kY=;
        b=X96mPUizArhrLPa2yhmH6H+pK9NPjVRZV3pOHU70U8bbh1ACfCylapIYHpFwb2+lZ2
         D2L5yZpxx8+hlKH9aamAB1XBnKQ/iEMbu6d3Ue3Dqhk6kgLnbKFyGWuddTcY7O8vZQWc
         FYbUr1e72IbUxyi71J260sTIA9Jph8Znox52/MKv2Kq7Hinjcx3VbBhR8O7yyIa+cmVF
         iT+lFBL6oid4rLNDcCfTqJpr8Oc9sFdHe58ai4rS4p0AGlIRmbftSTObBT9GvK0gMe2m
         LAGUajzX6Q/cS/YhJH99sVnPYrIQeSkehlNxd+CCrmOOllAIydcxRRiLJZCvDXpELzgD
         Z2qQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715677540; x=1716282340;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=S32SOD8xQbnu1UbjDRIfbFTfLCH36Ju2ZdHupMbk0kY=;
        b=B81F3eWa0+WwLKWQP+OjoR6vo3NMW4Wq23oNc0z8s1u3+gpa3RJqemc/Pd81HyUXtp
         lENB9UsLnoFFo8c6xDOYpWGT+65+j0nDi2xqEvEwYP+5iz+bqI9dVgTg7w5CzdJMP2Kg
         xVXlPhxhlk2jCnZgD8wAPtytwe/a4th8284aGRTMN2trVxWSMOQ/dCCYJAJhVIgSkmKd
         6s1zPSOwhHFD56jj8wV0YiWty4mslR0nPR9wBJkjitivjK5yxIp9eowVHOIXz6FGKFqi
         Mp33BnYfUt/UpFkg1KyqmCsc3CDawJVWglhBlV0vqFDjNuIsKzeyJlSiqYMEDZ6Ejzw6
         5Wrw==
X-Forwarded-Encrypted: i=1; AJvYcCUzf609SezP4Nlnq1nquSDW5CmKDxohOPgtDi+tLYHZ3s7rx7woMZ20PjhebWdvOp2q7F0yF4Ur2PLHCeOZD0hdxJoq/iha2zJyX+sit8w=
X-Gm-Message-State: AOJu0YzZQ2PKE8p5DmQM8M3yxQgN0CMwiL+EvFtsoWuOmv1Knd2k08Fh
	x3SCnzWgfcDXcyanZQkn8hSE9jcQJKH76WnwKqxn/V9uFblgOuL+
X-Google-Smtp-Source: AGHT+IGGrLCLUalZh5FXkxk5v8rCa/dsNJpHGzP01hX5hyvYdY14iP46Xrzr7GSxpHaLyODrTQzICA==
X-Received: by 2002:a17:906:7110:b0:a59:a18e:3fd9 with SMTP id a640c23a62f3a-a5a2d54c365mr720666766b.5.1715677539982;
        Tue, 14 May 2024 02:05:39 -0700 (PDT)
Message-ID: <7084e5635fc52be7add2f67cdbdd335cad618f7d.camel@gmail.com>
Subject: Re: [RFC PATCH v2 0/5] Add bridge VLAN support
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Leigh Brown
 <leigh@solinno.co.uk>,  xen-devel@lists.xenproject.org
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Juergen Gross
	 <jgross@suse.com>, Jason Andryuk <jandryuk@gmail.com>
Date: Tue, 14 May 2024 11:05:39 +0200
In-Reply-To: <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
	 <f5300cd1-4200-4957-9e4b-9ba0c1f40fa2@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-09 at 16:53 +0100, Andrew Cooper wrote:
> On 08/05/2024 10:38 pm, Leigh Brown wrote:
> > Hello all,
> >=20
> > I realised over the weekend that there is a valid use case for
> > providing
> > a VIF to a domain that has access to multiple VLANs, e.g. a router.
> > Yes,
> > you can create a VIF per VLAN, but if you start having several
> > VLANs (as
> > I do), it would be nicer to create a single interface that has
> > access to
> > all the relevant VLANs (e.g. enX0.10, enX0.20, etc.).
> >=20
> > So, version 2 changes the name and type of the parameter from an
> > integer
> > called `vid' to a string called `vlan'. The vlan parameter is then
> > parsed by the vif-bridge script (actually, the functions called by
> > it in
> > xen-network-common.sh).
> >=20
> > As it quite a common practice to allocate VLANs in round numbers, I
> > also
> > implemented the ability to specify contiguous or non-contiguous
> > ranges.
> > You can specify whether a VLAN is tagged or untagged, and which
> > VLAN is
> > the PVID (only one PVID is allowed).=C2=A0 For example,
> >=20
> > vif =3D [ 'mac=3Dxx:xx:xx:xx:xx:xx, bridge=3Dbr0, vlan=3D10p/20-29' ]
> >=20
> > will setup the VIF so that 10 is the PVID and VLAN IDs 20 through
> > 29
> > are permitted with tags. Another example:
> >=20
> > vif =3D [ 'mac=3Dxx:xx:xx:xx:xx:xx, bridge=3Dbr0, vlan=3D1p/10+10x9' ]
> >=20
> > will setup the bridge to set 1 as the PVID and permit access with
> > tags for VLAN IDs 10, 20, 30, 40, 50, 60, 70, 80 and 90.
> >=20
> > This patch set enables this capability as follows:
> >=20
> > 1. Adds `vlan' as a new member of the libxl_device_nic structure;
> > 2. Adds support to read and write the vlan parameter from the
> > xenstore;
> > 3. Adds `vlan' as a new keyword for the vif configuration option;
> > 4. Adds support to assign the bridge VLANs in the Linux hotplug
> > scripts;
> > 5. Updated xl-network-configuration(5) manpage and example configs.
> >=20
> > Original blurb below:
> >=20
> > For many years I have been configuring VLANs on my Linux Dom0 by
> > creating VLAN interfaces for each VLAN I wanted to connect a domain
> > to and then a corresponding bridge. So I would tend to have things
> > like:
> >=20
> > enp0s0=C2=A0=C2=A0=C2=A0 -> br0=C2=A0=C2=A0=C2=A0=C2=A0 -> vif1, vif2
> > enp0s0.10 -> br0vl10 -> vif3, vif4
> > enp0s0.20 -> br0vl20 -> vif5
> > dummy0=C2=A0=C2=A0=C2=A0 -> br1=C2=A0=C2=A0=C2=A0=C2=A0 -> vif6
> >=20
> > I recently discovered that iproute2 supports creating bridge VLANs
> > that
> > allows you to assign a VLAN to each of the interfaces associated to
> > a
> > bridge. This allows a greatly simplified configuration where a
> > single
> > bridge can support all the domains, and the iproute2 bridge command
> > can
> > assign each VIF to the required VLAN.=C2=A0 This looks like this:
> >=20
> > # bridge vlan
> > port=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 vlan-id=C2=A0=20
> > enp0s0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 1 PVID Egress Untagged
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 10
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 20
> > br0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 1 PVID Egress Untagged
> > vif1.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 1 PVID Egress Untagged
> > vif2.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 1 PVID Egress Untagged
> > vif3.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 10 PVID Egress Untagged
> > vif4.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 10 PVID Egress Untagged
> > vif5.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 20 PVID Egress Untagged
> > vif6.0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 30 PVID Egress Untagged
> >=20
> > This patch set enables this capability as follows:
> >=20
> > 1. Adds `vid' as a new member of the libxl_device_nic structure;
> > 2. Adds support to read and write vid from the xenstore;
> > 3. Adds `vid' as a new keyword for the vif configuration option;
> > 4. Adds support for assign the bridge VLAN in the Linux hotplug
> > scripts.
> >=20
> > I don't believe NetBSD or FreeBSD support this capability, but if
> > they
> > do please point me in the direction of some documentation and/or
> > examples.
> >=20
> > NB: I'm not very familiar with Xen code base so may have missed
> > something important, although I have tested it and it is working
> > well
> > for me.
> >=20
> > Cheers,
> >=20
> > Leigh.
> >=20
> >=20
> > Leigh Brown (5):
> > =C2=A0 tools/libs/light: Add vlan field to libxl_device_nic
> > =C2=A0 tools/xl: add vlan keyword to vif option
> > =C2=A0 tools/hotplug/Linux: Add bridge VLAN support
> > =C2=A0 docs/man: document VIF vlan keyword
> > =C2=A0 tools/examples: Example Linux bridge VLAN config
> >=20
> > =C2=A0docs/man/xl-network-configuration.5.pod.in=C2=A0=C2=A0=C2=A0 |=C2=
=A0 38 ++++++
> > =C2=A0tools/examples/linux-bridge-vlan/README=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 |=C2=A0 68 +++++++++++
> > =C2=A0tools/examples/linux-bridge-vlan/br0.netdev=C2=A0=C2=A0 |=C2=A0=
=C2=A0 7 ++
> > =C2=A0tools/examples/linux-bridge-vlan/br0.network=C2=A0 |=C2=A0=C2=A0 =
8 ++
> > =C2=A0.../examples/linux-bridge-vlan/enp0s0.network |=C2=A0 16 +++
> > =C2=A0tools/hotplug/Linux/xen-network-common.sh=C2=A0=C2=A0=C2=A0=C2=A0=
 | 111
> > ++++++++++++++++++
> > =C2=A0tools/libs/light/libxl_nic.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 =
10 ++
> > =C2=A0tools/libs/light/libxl_types.idl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 1 +
> > =C2=A0tools/xl/xl_parse.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 +
> > =C2=A09 files changed, 261 insertions(+)
> > =C2=A0create mode 100644 tools/examples/linux-bridge-vlan/README
> > =C2=A0create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
> > =C2=A0create mode 100644 tools/examples/linux-bridge-vlan/br0.network
> > =C2=A0create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.networ=
k
> >=20
>=20
> This is past the last-post date, so Oleksii will need to decide
> whether
> he's happy to make an exception for it.
We also have Feature Freeze this week. But if maintainers think they
have additional/enough time for reviewing this patch series then we can
consider to have in Xen 4.19.

~ Oleksii

>=20
> Anthony is OoO for a month now, so if this is to get in for 4.19,
> then
> it will need reviewing by others.=C2=A0 I've CC'd a few plausible
> candidates...
>=20
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Tue May 14 09:08:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:08:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721284.1124582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6o9V-0005uD-Qr; Tue, 14 May 2024 09:08:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721284.1124582; Tue, 14 May 2024 09:08:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6o9V-0005u6-Nr; Tue, 14 May 2024 09:08:53 +0000
Received: by outflank-mailman (input) for mailman id 721284;
 Tue, 14 May 2024 09:08:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y506=MR=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s6o9U-0005u0-Bz
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:08:52 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9420cc0a-11d1-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:08:50 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59b178b75bso1043188866b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:08:50 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eb2459f0sm114109a12.93.2024.05.14.02.08.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:08:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9420cc0a-11d1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715677730; x=1716282530; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=/jmm0MhYebwHSh3avmuIVn24ItpVT9wlYrJnqbFthcM=;
        b=Pah5NSz2G4e/0HeWQEMG+ZpD9xw3MlRpu1ZSDva18QYOSvfQ8zrn3SsbDXjLQ4wSNq
         F4oINGLfAjgJAlRhnYL0tSgHhHdom/ON3zRkiQzQRR6E54oWJ2joranuG7RdiZDBTfJK
         4QgUbsAiOVyM7PjRsTsQRz7zoc7OG2bQ13oh4RW6wsUxbc4rv5CUCwp/JCnRgtbp7LlK
         7bapSs8JrvuGVkIkwSr6MRiuCE/Atm3zqXRIZMeEuEM42AnlhU6SlpFaTwgdut7sKIXA
         InH9gkRSpygQfXdng0V4ImYqOIwCy+ebbTuVJ6ZiAMnWKI4aPUN5iwCG9Eg2XjNilDmI
         j0Ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715677730; x=1716282530;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/jmm0MhYebwHSh3avmuIVn24ItpVT9wlYrJnqbFthcM=;
        b=MZEPHARWSiJhlMFHd61FhCUKawIO7Y4tW6BEW3QyzDhlXjCsJ8s+vSxUX+eS2bQjfN
         gwRCWc0TAkmq1fYwRit1Qk6AvE0T/IDOUosBBdUFCIv8dM0p6g/naPBrqKMjtCTYUXW7
         XEDmAP20KCzjS+T4bOcz2fZMojPdLzgP89Pe4+9vcSox4qIxiHohK5RpQA6iM6eh8K3E
         QIB92ik7/8mnb83Hu+Tn38rtycq9o+8WM/lhvNeb3g8ZsIGNNthD4UM2DHdjbGBXUzuf
         DqPst0UeZYDVpYRBZORLYB5eCsHYt6baChdD1CBqyvFMvnczG5E0HqDZLlX04cb8XHJU
         VSFg==
X-Forwarded-Encrypted: i=1; AJvYcCVIkFg0rZOr+jsrMXsDMClEdeVjDGDVPBP5UVT1SvZzOQH463M+bbnYY+TWhpdT2o5+2HcboLMlaFrWf63cMYxQfh9wcKbZM89fEmJmU5g=
X-Gm-Message-State: AOJu0YzJsS8qENXAS2nnmK0xdNAkpUD8BecWjR+eaM7XA5bwlyLi8P/6
	mPxa83owRN4/FfDn+XUPesn99cZkJh1rh3XhsRlqoGF1V3dT/RCn40bsRdBr
X-Google-Smtp-Source: AGHT+IFZ1xitOmkJOKnQqy+eGTL4E3GWC7HVhpsw5f7Q8NqNSUwybdAKLOTvtyjPFcmAAefXtJMpew==
X-Received: by 2002:aa7:d1d3:0:b0:574:eb26:74a with SMTP id 4fb4d7f45d1cf-574eb260839mr258819a12.21.1715677729799;
        Tue, 14 May 2024 02:08:49 -0700 (PDT)
Message-ID: <c3cea2b9118392c7b4ddddd750dd97ac96e8ec7c.camel@gmail.com>
Subject: Re: [PATCH for-4.19] libxl: fix population of the online vCPU
 bitmap for PVH
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Roger Pau Monne
	 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross
 <jgross@suse.com>,  Arthur Borsboom <arthurborsboom@gmail.com>, Leigh Brown
 <leigh@solinno.co.uk>
Date: Tue, 14 May 2024 11:08:48 +0200
In-Reply-To: <df58677c-d9a3-43a7-b5b4-85565ad73b12@citrix.com>
References: <20240510124913.49945-1-roger.pau@citrix.com>
	 <df58677c-d9a3-43a7-b5b4-85565ad73b12@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Fri, 2024-05-10 at 16:20 +0100, Andrew Cooper wrote:
> On 10/05/2024 1:49 pm, Roger Pau Monne wrote:
> > libxl passes some information to libacpi to create the ACPI table
> > for a PVH
> > guest, and among that information it's a bitmap of which vCPUs are
> > online
> > which can be less than the maximum number of vCPUs assigned to the
> > domain.
> >=20
> > While the population of the bitmap is done correctly for HVM based
> > on the
> > number of online vCPUs, for PVH the population of the bitmap is
> > done based on
> > the number of maximum vCPUs allowed.=C2=A0 This leads to all local APIC
> > entries in
> > the MADT being set as enabled, which contradicts the data in
> > xenstore if vCPUs
> > is different than maximum vCPUs.
> >=20
> > Fix by copying the internal libxl bitmap that's populated based on
> > the vCPUs
> > parameter.
> >=20
> > Reported-by: Arthur Borsboom <arthurborsboom@gmail.com>
> > Link: https://gitlab.com/libvirt/libvirt/-/issues/399
> > Reported-by: Leigh Brown <leigh@solinno.co.uk>
> > Fixes: 14c0d328da2b ('libxl/acpi: Build ACPI tables for HVMlite
> > guests')
> > Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> > ---
> > Note that the setup of hvm_info_table could be shared between PVH
> > and HVM, as
> > the fields are very limited, see hvm_build_set_params() for the HVM
> > side.
> > However this late in the release it's safer to just adjust the PVH
> > path.
> >=20
> > Also note the checksum is not provided when hvm_info_table is built
> > for PVH.
> > This is fine so far because such checksum is only consumed by
> > hvmloader and not
> > libacpi itself.
> >=20
> > It's a bugfix, so should be considered for 4.19.
>=20
> Bugfixes are still completely fair game at this point.
Considering that this is bugfixes I will be happy to have that in 4.19
release:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
>=20
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>



From xen-devel-bounces@lists.xenproject.org Tue May 14 09:11:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:11:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721288.1124591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oCJ-0008Qu-6y; Tue, 14 May 2024 09:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721288.1124591; Tue, 14 May 2024 09:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oCJ-0008Qn-4O; Tue, 14 May 2024 09:11:47 +0000
Received: by outflank-mailman (input) for mailman id 721288;
 Tue, 14 May 2024 09:11:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oCH-0008Qh-Lg
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:11:45 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb377b21-11d1-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:11:43 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5a88339780so80252666b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:11:43 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a3c65d06fsm533626966b.52.2024.05.14.02.11.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:11:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb377b21-11d1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715677903; x=1716282703; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=r+ElSVHZIfIVtmDPII7svjzGxPG6aSl1kCr1cOt4mfA=;
        b=Ka8JEI/wJpbxumujOsJEfXOmWbYEFtAqmDGe42gMhYebG74pmlJbR0yL091sRazuPa
         malXeS4eq/vXUEpvqYaW5m9gXHIggAF8iyIEllDy7FeQVozgTLbvCy0mMMD5dVOvyPNZ
         8KECm49DsgH40B+U2fIeFVpR2gcWMQS75QoFXXOA4eMlaVlRukNZGyhGmp/Hn9/oTcxq
         pro9Oee9Ry5cxWVk9IzX0zLqkUHzq6u9ozO2d49rChurrLDCESdjjhD+llthyeVIGzbb
         fZycaSXnjlE3aoIBWkYECKMW8MOg1YPQJjRVCRCQMIpiTV6gq0pjx0IVeYMnQSTa8+Q4
         3d/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715677903; x=1716282703;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=r+ElSVHZIfIVtmDPII7svjzGxPG6aSl1kCr1cOt4mfA=;
        b=i5j9xl2frAyqZIjM82lqxsKoiXm6veAPN65GfFitfIlM13WLZdddwAEtpT0TkpADBf
         CAWMx7fi0gJc0/k1W7PFdW2lOpJ3m0Z2ij4y1n9SWoOI01vnhY5GZYlcqldcCbRS/1HT
         K3M8S68OgmdAOKfX/LdeQuWC6xohfhWVKQhmyJquRbm9heZbD/xKJ5C3crNvwkU8yshf
         6MIx4g26hBKoCibojXBTb+C8/ujaUPSI73KSkhjrOzBTySx5Xfx4mMv3CYSmVjrWYmZo
         R4j3zvZy0RxnsyU7PxsXnMybRr/koOMQadmE0Xwwa7RxVQiRHo6fK3lIf+NbdBmCtmhA
         mt4Q==
X-Forwarded-Encrypted: i=1; AJvYcCX35ov3kYSaiAbluEMENdJdPpQiv7Xz4+FKNlq7lxxZdmcmbNONR3TBEAhfXWcZ5C41LW0KC+cel5Y2uP1VlmK31cH5rCfxOQTA+HFtMR8=
X-Gm-Message-State: AOJu0YyTVwy45N5TLgBG8xeh7QrATAKyVIVqn13DcMErWwf3tZl/kuFt
	SlRVIyUOr+U9erHA1OFPmdqa0BCt0b1EnoG7X4knX3vaP79eYUheJlrejvk74w==
X-Google-Smtp-Source: AGHT+IG0YZeFYzvWvWTlxsCW4oPs/lXcvs9ze2sLiQDo6e7UyvsEoR2u7Vf942DADP5l2optU5Y/TA==
X-Received: by 2002:a17:907:9710:b0:a5a:2d30:b8c1 with SMTP id a640c23a62f3a-a5a2d54c038mr1011278266b.14.1715677902809;
        Tue, 14 May 2024 02:11:42 -0700 (PDT)
Message-ID: <39badfc4-df85-46d4-8cfa-740592c60174@suse.com>
Date: Tue, 14 May 2024 11:11:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 3/3] xen/pci: address violations of MISRA C Rule
 20.7
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 xen-devel@lists.xenproject.org, Nicola Vetrini <nicola.vetrini@bugseng.com>
References: <cover.1714487169.git.nicola.vetrini@bugseng.com>
 <df48ce446cf6409eda04460109f3082b39c4f618.1714487169.git.nicola.vetrini@bugseng.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <df48ce446cf6409eda04460109f3082b39c4f618.1714487169.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.04.2024 16:28, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
>  xen/include/xen/pci_regs.h | 6 +++---
>  xen/include/xen/vpci.h     | 2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)

I notice you committed this with just my ack. I'm pretty sure Roger doesn't
mind, but formally his ack would have been required (hence why I didn't
commit it already before my vacation).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:14:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:14:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721290.1124602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oEm-0000jo-K4; Tue, 14 May 2024 09:14:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721290.1124602; Tue, 14 May 2024 09:14:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oEm-0000jh-HP; Tue, 14 May 2024 09:14:20 +0000
Received: by outflank-mailman (input) for mailman id 721290;
 Tue, 14 May 2024 09:14:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y506=MR=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s6oEl-0000jZ-DB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:14:19 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57beca1b-11d2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:14:18 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59e4136010so1373134166b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:14:18 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0a99sm7297420a12.52.2024.05.14.02.14.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:14:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57beca1b-11d2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715678058; x=1716282858; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=jCq1jEEBXsJms2qDxK6Vlb8MjfuVjmFZ2iS1y6IJjSo=;
        b=SoetO/Vwiv/8BAy0U0KdbkrOBT5VHxGx4xsYUUmlwEiUWiia8Pa6Nq8ZisOem8KN/Y
         bSR/e+VjDyMtkkCfYKvRGJj1Zgvl4oSPEffx66VMUSgjUy+t/hB0lOBAd1D96/HFHfkY
         d7AxJTQBSvvNWvlo47f9vxcIdKFEmPThkzPBDY86PGpn7S//KdanQ1zVE1jV0CW6qqn3
         ajTqCCm2Fspd6urgXAW0UUI2uz9SagMyttHQfNHsCdbE6DrpMZSexjuP0sRx+BjKk3+Q
         pRnK8/DGqF0lzdn4Zvd4r3k+cfDy7f2EkxehL6D7HFP5sdgTGI5zLGwEki0zBqiwKuve
         NTEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715678058; x=1716282858;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jCq1jEEBXsJms2qDxK6Vlb8MjfuVjmFZ2iS1y6IJjSo=;
        b=mIAfvNNeDo/TyothpXRjv/TCmRWvAxXsZcVzmWgtKPjW3kMGdNp3riqA3nlQek5zyL
         tmE3+ULM0QhYXnrKLWCFzmxwewZup2T/d6hg1IA3Ctyjjwm/7PA83VKQTLc1MWbWdsBA
         tDjij0KRBqOpTUd4Tms504XIEP8+NVt1WGgnJBdHOwlPBhkGXFNk2zhVO181ktI1FKMV
         VAKFEghoHXmKxf3fal68hlhVlNhk4YmlSyhpAwbSfRKJosyrsbROz2iLBAN3Np3bCBFm
         CcPH0zsDbxNrAPzsVeTq49H5waNlOyjD6DwKsN9LsFlfjw9o+AcjTFgU98D/B3xb4UI7
         vR/A==
X-Forwarded-Encrypted: i=1; AJvYcCXQCEcvH093hFWyUldZC7yMnNX3qT6uaagMol7mFnr4BDc0bHYhwUFw+GMUm1vGTggGQr1sz0v2JsUXym/UVZ08zFn5ihaBpbuvkgMinRk=
X-Gm-Message-State: AOJu0Yw+q28GZIW5johLhe47rCIdwpsvKRrZLITeZYv5uP028troHbko
	9S1BlBfjM/T6uUevnVMD+jYAAf9AnGmG76n/CNFdo36Ihn9SGOH2/HJSM5JH
X-Google-Smtp-Source: AGHT+IFAKVkPkjztU1ZccSXidEVHF9vGC/KrT7buxO0OeqNZLP2HHMwjn8vHTnDJjo0xRnVrqMIvQQ==
X-Received: by 2002:a50:9505:0:b0:572:6846:b899 with SMTP id 4fb4d7f45d1cf-5734d6b407dmr7680089a12.41.1715678057968;
        Tue, 14 May 2024 02:14:17 -0700 (PDT)
Message-ID: <7c651429b17c2e65f99d72d0595c07cbb3ef2861.camel@gmail.com>
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>
Date: Tue, 14 May 2024 11:14:17 +0200
In-Reply-To: <20240513085925.59324-1-roger.pau@citrix.com>
References: <20240513085925.59324-1-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-13 at 10:59 +0200, Roger Pau Monne wrote:
> There's no point in forcing a system wide update of the MTRRs on all
> processors
> when there are no changes to be propagated.=C2=A0 On AP startup it's only
> the AP
> that needs to write the system wide MTRR values in order to match the
> rest of
> the already online CPUs.
>=20
> We have occasionally seen the watchdog trigger during `xen-hptool
> cpu-online`
> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of
> the MTRRs
> on all the CPUs in the system.
>=20
> While there adjust the comment to clarify why the system-wide
> resetting of the
> MTRR registers is not needed for the purposes of mtrr_ap_init().
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> For consideration for 4.19: it's a bugfix of a rare instance of the
> watchdog
> triggering, but it's also a good performance improvement when
> performing
> cpu-online.
>=20
> Hopefully runtime changes to MTRR will affect a single MSR at a time,
> lowering
> the chance of the watchdog triggering due to the system-wide
> resetting of the
> range.
Considering it as a bugfix it will be good to have it in 4.19 release:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii

> ---
> =C2=A0xen/arch/x86/cpu/mtrr/main.c | 15 ++++++++-------
> =C2=A01 file changed, 8 insertions(+), 7 deletions(-)
>=20
> diff --git a/xen/arch/x86/cpu/mtrr/main.c
> b/xen/arch/x86/cpu/mtrr/main.c
> index 90b235f57e68..0a44ebbcb04f 100644
> --- a/xen/arch/x86/cpu/mtrr/main.c
> +++ b/xen/arch/x86/cpu/mtrr/main.c
> @@ -573,14 +573,15 @@ void mtrr_ap_init(void)
> =C2=A0	if (!mtrr_if || hold_mtrr_updates_on_aps)
> =C2=A0		return;
> =C2=A0	/*
> -	 * Ideally we should hold mtrr_mutex here to avoid mtrr
> entries changed,
> -	 * but this routine will be called in cpu boot time, holding
> the lock
> -	 * breaks it. This routine is called in two cases: 1.very
> earily time
> -	 * of software resume, when there absolutely isn't mtrr
> entry changes;
> -	 * 2.cpu hotadd time. We let mtrr_add/del_page hold
> cpuhotplug lock to
> -	 * prevent mtrr entry changes
> +	 * hold_mtrr_updates_on_aps takes care of preventing
> unnecessary MTRR
> +	 * updates when batch starting the CPUs (see
> +	 * mtrr_aps_sync_{begin,end}()).
> +	 *
> +	 * Otherwise just apply the current system wide MTRR values
> to this AP.
> +	 * Note this doesn't require synchronization with the other
> CPUs, as
> +	 * there are strictly no modifications of the current MTRR
> values.
> =C2=A0	 */
> -	set_mtrr(~0U, 0, 0, 0);
> +	mtrr_set_all();
> =C2=A0}
> =C2=A0
> =C2=A0/**



From xen-devel-bounces@lists.xenproject.org Tue May 14 09:15:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:15:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721293.1124612 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oFx-0001Fp-VS; Tue, 14 May 2024 09:15:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721293.1124612; Tue, 14 May 2024 09:15:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oFx-0001Fi-Qx; Tue, 14 May 2024 09:15:33 +0000
Received: by outflank-mailman (input) for mailman id 721293;
 Tue, 14 May 2024 09:15:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6oFx-0001FY-3m; Tue, 14 May 2024 09:15:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6oFw-0002gX-W6; Tue, 14 May 2024 09:15:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6oFw-0006vN-KY; Tue, 14 May 2024 09:15:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6oFw-0006fz-Jy; Tue, 14 May 2024 09:15:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4VusIsNXfx3BdSMM6BE7rFczZTrFhyUKPhJ9GmTJ8NE=; b=JOAFkzLNIwr4ON3Ed6JE2Rpnck
	WJZaQU3jbt2dDRwv3t38n5jBxOQNd14R+dL3uM39WK2Gap4IHTXBPkVIA07VGZNDhCLpulzCzJ/DX
	r+Sz785YMviwqKi4BgTc80E1J5QAb/Q3Di+PmBzqb0KkQC46e+gYw2afb/nM1Ibmjhfs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185992-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185992: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a7c840ba5fa78d7761b9fedc33d69cef44986d79
X-Osstest-Versions-That:
    linux=6d1346f1bcbf2724dee8af013cdab9f7b581435b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 May 2024 09:15:32 +0000

flight 185992 linux-linus real [real]
flight 185995 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185992/
http://logs.test-lab.xenproject.org/osstest/logs/185995/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt      8 xen-boot                 fail REGR. vs. 185990

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 185995-retest
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 185995-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 185995 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 185995 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 185995 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 185995 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185990
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 185990
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185990
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a7c840ba5fa78d7761b9fedc33d69cef44986d79
baseline version:
 linux                6d1346f1bcbf2724dee8af013cdab9f7b581435b

Last test of basis   185990  2024-05-13 16:43:50 Z    0 days
Testing same since   185992  2024-05-14 00:41:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Guerin <adam.guerin@intel.com>
  Ai Chao <aichao@kylinos.cn>
  Akhil R <akhilrajeev@nvidia.com>
  Akira Yokosawa <akiyks@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Aleksandr Mishin <amishin@t-argos.ru>
  Alexander Lobakin <aleksander.lobakin@intel.com>
  Alexey Dobriyan (Yandex) <adobriyan@gmail.com>
  Alexey Dobriyan <adobriyan@gmail.com>
  Alice Ryhl <aliceryhl@google.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrea Parri <parri.andrea@gmail.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Animesh Agarwal <animeshagarwal28@gmail.com>
  Anuj Gupta <anuj20.g@samsung.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Avadhut Naik <avadhut.naik@amd.com>
  Bagas Sanjaya <bagasdotme@gmail.com>
  Benno Lossin <benno.lossin@proton.me>
  Bilbao, Carlos <carlos.bilbao@amd.com>
  Bill O'Donnell <bodonnel@redhat.com>
  Bird, Tim <Tim.Bird@sony.com>
  Boqun Feng <boqun.feng@gmail.com>
  Breno Leitao <leitao@debian.org>
  Brian Kubisiak <brian@kubisiak.com>
  Carlos Bilbao <carlos.bilbao@amd.com>
  Chang S. Bae <chang.seok.bae@intel.com>
  Chen Jun <chenjun102@huawei.com>
  Chen Ni <nichen@iscas.ac.cn>
  Chenghai Huang <huangchenghai2@huawei.com>
  Christian Brauner <brauner@kernel.org>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Coly Li <colyli@suse.de>
  Conor Dooley <conor.dooley@microchip.com>
  Corentin Labbe <clabbe@baylibre.com>
  Damian Muszynski <damian.muszynski@intel.com>
  Damien Le Moal <dlemoal@kernel.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Jordan <daniel.m.jordan@oracle.com>
  Danilo Krummrich <dakr@redhat.com>
  David Gstir <david@sigma-star.at>
  David Howells <dhowells@redhat.com>
  David Oberhollenzer <david.oberhollenzer@sigma-star.at>
  David Rientjes <rientjes@google.com>
  David Sterba <dsterba@suse.com>
  David Wei <dw@davidwei.uk>
  Dawid Osuchowski <linux@osuchow.ski>
  Dennis Lam <dennis.lamerice@gmail.com>
  Dennis Maisenbacher <dennis.maisenbacher@wdc.com>
  Dirk Behme <dirk.behme@de.bosch.com>
  Dongliang Mu <dzm91@hust.edu.cn>
  Dustin L. Howett <dustin@howett.net>
  Edward Adam Davis <eadavis@qq.com>
  Eric Biggers <ebiggers@google.com>
  Eric Sandeen <sandeen@redhat.com>
  Erick Archer <erick.archer@outlook.com>
  Federico Vaga <federico.vaga@vaga.pv.it>
  Florian-Ewald Mueller <florian-ewald.mueller@ionos.com>
  Frank Li <Frank.Li@nxp.com>
  Frederic Weisbecker <frederic@kernel.org>
  Gabriel Krisman Bertazi <krisman@suse.de>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Giovanni Cabiddu <giovanni.cabiddu@intel.com>
  Guenter Roeck <linux@roeck-us.net>
  Gustavo A. R. Silva <gustavoars@kernel.org>
  Haifeng Xu <haifeng.xu@shopee.com>
  Hailey Mothershead <hailmo@amazon.com>
  Hans Holmberg <hans.holmberg@wdc.com>
  Haoyang Liu <tttturtleruss@hust.edu.cn>
  Herbert Xu <herbert@gondor.apana.org.au>
  Hyunmin Lee <hyunminlr@gmail.com>
  Ilya Denisyev <dev@elkcl.ru>
  INAGAKI Hiroshi <musashino.open@gmail.com>
  Ivan Orlov <ivan.orlov0322@gmail.com>
  Ivan Orlov <ivan.orlov@codethink.co.uk>
  Jack Wang <jinpu.wang@ionos.com>
  Jakub Kicinski <kuba@kernel.org>
  James Bottomley <James.Bottomley@HansenPartnership.com>
  Jarkko Sakkinen <jarkko.sakkinen@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org> # for TRUSTED_KEYS_TPM
  Jens Axboe <axboe@kernel.dk>
  Jerry Snitselaar <jsnitsel@redhat.com>
  Jeungwoo Yoo <casionwoo@gmail.com>
  Jia Jie Ho <jiajie.ho@starfivetech.com>
  Jianfeng Wang <jianfeng.w.wang@oracle.com>
  Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
  Jini Susan George <jinisusan.george@amd.com>
  Joachim Vandersmissen <git@jvdsn.com>
  Joel Fernandes (Google) <joel@joelfernandes.org>
  Joel Granados <j.granados@samsung.com>
  Johannes Berg <johannes.berg@intel.com>
  Johannes Thumshirn <johannes.thumshirn@wdc.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Garry <john.g.garry@oracle.com>
  Jonathan Corbet <corbet@lwn.net>
  Justin Stitt <justinstitt@google.com>
  Karel Balej <balejk@matfyz.cz>
  Karthikeyan Ramasubramanian <kramasub@chromium.org>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Keith Busch <kbusch@kernel.org>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krzysztof Błaszkowski <kb@sysmikro.com.pl>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Laura Nao <laura.nao@collabora.com>
  Li Hua <lihua@email.com>
  Li Nan <linan122@huawei.com>
  Li Zhijian <lizhijian@fujitsu.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lothar Rubusch <l.rubusch@gmail.com>
  Luca Weiss <luca.weiss@fairphone.com>
  Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
  Luis Henriques (SUSE) <luis.henriques@linux.dev>
  Luis Henriques <lhenriques@suse.de>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Lukas Wunner <lukas@wunner.de>
  Maki Hatano <Maki.Y.Hatano@gmail.com>
  Marco Elver <elver@google.com>
  Marek Vasut <marex@denx.de>
  Mario Limonciello <superm1@gmail.com>
  Mark Rutland <mark.rutland@arm.com>
  Matt Turner <mattst88@gmail.com>
  Matthew Mirvish <matthew@mm12.xyz>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Max Filippov <jcmvbkbc@gmail.com>
  Maxime Méré <maxime.mere@foss.st.com>
  Maíra Canal <mcanal@igalia.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Haener <michael.haener@siemens.com>
  Miguel Ojeda <ojeda@kernel.org>
  Mikko Perttunen <mperttunen@nvidia.com>
  Ming Lei <ming.lei@redhat.com>
  Nathan Chancellor <nathan@kernel.org>
  Neeraj Upadhyay (AMD) <neeraj.iitr10@gmail.com>
  Neeraj Upadhyay <neeraj.upadhyay@kernel.org>
  Nell Shamrell-Harrington <nells@linux.microsoft.com>
  Nhat Pham <nphamcs@gmail.com>
  Nikita Kiryushin <kiryushin@ancud.ru>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Pankaj Gupta <pankaj.gupta@nxp.com>
  Paul E. McKenney <paulmck@kernel.org>
  Pavel Begunkov <asml.silence@gmail.com>
  Prashant Malani <pmalani@chromium.org>
  Raghav Narang <dev@raxyte.com>
  Remington Brasga <rbrasga@uci.edu>
  Richard Weinberger <richard@nod.at>
  Roman Smirnov <r.smirnov@omp.ru>
  Ruyi Zhang <ruyi.zhang@samsung.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Sangyun Kim <sangyun.kim@snu.ac.kr>
  Sarat Mandava <mandavasarat@gmail.com>
  Saurav Shah <sauravshah.31@gmail.com>
  Silvio Gissi <sifonsec@amazon.com>
  Siming Wan <siming.wan@intel.com>
  Song Liu <song@kernel.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  syzbot+0a3683a0a6fecf909244@syzkaller.appspotmail.com
  Tejun Heo <tj@kernel.org>
  Thierry Reding <treding@nvidia.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Blum <thorsten.blum@toblux.com>
  Thorsten Leemhuis <linux@leemhuis.info>
  Tim Bird <tim.bird@sony.com>
  Tim Chen <tim.c.chen@linux.intel.com>
  Tom Lendacky <thomas.lendacky@amd.com>
  Tom Zanussi <tom.zanussi@linux.intel.com>
  Tyler Hicks (Microsoft) <code@tyhicks.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uladzislau Rezki (Sony) <urezki@gmail.com>
  Usama Arif <usamaarif642@gmail.com>
  Utkarsh Tripathi <utripathi2002@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vignesh Balasubramanian <vigbalas@amd.com>
  Vijay Nag <nagvijay@microsoft.com>
  Vitaly Chikunov <vt@altlinux.org>
  Vlastimil Babka <vbabka@suse.cz>
  Wedson Almeida Filho <walmeida@microsoft.com>
  Wenkai Lin <linwenkai6@hisilicon.com>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Xin Zeng <xin.zeng@intel.com>
  Xiongwei Song <xiongwei.song@windriver.com>
  Xiu Jianfeng <xiujianfeng@huawei.com>
  Yang Li <yang.lee@linux.alibaba.com>
  Yanteng Si <siyanteng@loongson.cn>
  Yu Kuai <yukuai3@huawei.com>
  Yujie Liu <yujie.liu@intel.com>
  Zenghui Yu <zenghui.yu@linux.dev>
  Zhang Yi <yi.zhang@huawei.com>
  Zhu Yanjun <yanjun.zhu@linux.dev>
  Zqiang <qiang.zhang1211@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 15941 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:15:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:15:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721297.1124622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oGK-0001mL-Am; Tue, 14 May 2024 09:15:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721297.1124622; Tue, 14 May 2024 09:15:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oGK-0001mC-6Y; Tue, 14 May 2024 09:15:56 +0000
Received: by outflank-mailman (input) for mailman id 721297;
 Tue, 14 May 2024 09:15:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y506=MR=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s6oGI-0001gM-77
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:15:54 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90728699-11d2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:15:53 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso1382174266b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:15:53 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a29d5f14bsm644823966b.213.2024.05.14.02.15.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:15:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90728699-11d2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715678153; x=1716282953; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=mxpvRQDrBmFqVClwAgNRnJyfpXAKPPWee9HFi9CKjGw=;
        b=J2wekfbNo3v2EEBfJVjF7XAwM0DopxUtQsU6nzeeSlMnuf32c9+q3joSRIiZKtDn+h
         oIq2jYXZPaSsvxEClm4BQYXgdv2EWFGoT++em1SuLGpnqWTYFOQHvGsQ+2eFRAhj+nGp
         ngJHKH16ZYGwvuh7r9QlC3OIUUZPVXGTqT9nIdVdtRztN0tpzqJdfjpTkIKOqfbowKF7
         sdZgUh0mGnUl3KJCd168qwKUJkxEnBe9e9QbfQUfa57je5dWEceE8EaaGuFepTFOzvZW
         KalvrPDG3J8XAXO6Wvezo2S4D3w86r/fS2SYE5AFFtXLEAatsV7N4l9e0DS6Ckzw/UOY
         8J9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715678153; x=1716282953;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mxpvRQDrBmFqVClwAgNRnJyfpXAKPPWee9HFi9CKjGw=;
        b=WQM03TIa37sXv3e1sC3oBSK/NTYIEjx3ac1dcilT/FR/+RhXk6Ydw+l9aTkMqvy41C
         ctYtopT+7Pc6SM8/QbXth9tXSMbRZy3WO2yqAmVcL2b/emv6G4/xUbQZHISy/omcVIF6
         2IOUT7a/mrys8NU76+EGZfTnfmuArTatUM2gkpQ8q0gXOe1VrMFD+m16fzVCUCCCKk/d
         myafxYejl8iYUEdx1OkQqxGR7VLxBCvhXfvynmECoSC91aBKdbdGvnoDy4JbboaRF/FW
         8KvZD2okcHkHoCorYZfihUtbUYj7PCONZfmV5ca7Ic8BUBznqJ3LrbKwzhuRtaUxlMKf
         HjKg==
X-Forwarded-Encrypted: i=1; AJvYcCX3X+asvvqr48ET7dipsuzSU8DimiXmfO9lPSmhvCTDPUNUNIYauSrKdXoRugB6IdVFH5//pEV4IXzq78/odtYKWDZPF/si/RIVKLHNC8Y=
X-Gm-Message-State: AOJu0YybMsMcgQodFkylpueusMfjdtwTFHhH6hRf4nQWAXUKxPxexIsG
	f6Bblu+4UXIchmwu1MvOhMaicXt1/qEpIps4npuKCxxKSQUB5o1g
X-Google-Smtp-Source: AGHT+IGuKWiZqJGBtJ+ZMzlnQMR+Kpoj9kpY0gvRdkkDsHn/CJL/axFBEBN3mcgSImLLyBv4mTr3VQ==
X-Received: by 2002:a17:906:ef01:b0:a5a:3da6:7712 with SMTP id a640c23a62f3a-a5a3da67767mr786054166b.71.1715678152788;
        Tue, 14 May 2024 02:15:52 -0700 (PDT)
Message-ID: <98d911ff46dfd5e9c3a6fb7007e7200def05ac75.camel@gmail.com>
Subject: Re: [PATCH for-4.19 v2] tools/xen-cpuid: switch to use cpu-policy
 defined names
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Andrew Cooper
	 <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	xen-devel@lists.xenproject.org
Date: Tue, 14 May 2024 11:15:51 +0200
In-Reply-To: <Zjy9zmA1kjgZt9s8@macbook>
References: <20240502114922.94288-1-roger.pau@citrix.com>
	 <2ddc6307-c9a6-4ee8-9689-c9c2c5a37142@suse.com> <Zjy9zmA1kjgZt9s8@macbook>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-09 at 14:13 +0200, Roger Pau Monn=C3=A9 wrote:
> On Thu, May 02, 2024 at 03:16:54PM +0200, Jan Beulich wrote:
> > On 02.05.2024 13:49, Roger Pau Monne wrote:
> > > Like it was done recently for libxl, switch to using the auto-
> > > generated feature
> > > names by the processing of cpufeatureset.h, this allows removing
> > > the open-coded
> > > feature names, and unifies the feature naming with libxl and the
> > > hypervisor.
> > >=20
> > > Introduce a newly auto-generated array that contains the feature
> > > names indexed
> > > at featureset bit position, otherwise using the existing
> > > INIT_FEATURE_NAMES
> > > would require iterating over the array elements until a match
> > > with the expected
> > > bit position is found.
> > >=20
> > > Note that leaf names need to be kept, as the current auto-
> > > generated data
> > > doesn't contain the leaf names.
> > >=20
> > > Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> >=20
> > Reviewed-by: Jan Beulich <jbeulich@suse.com>
>=20
> Oleksii, now that Jan has provided a Reviewed-by, can you provide a
> release-ack for this to go in?
Based that it is reviewed, I will be happy to have this in 4.19:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:20:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721307.1124631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oKi-0003mW-QM; Tue, 14 May 2024 09:20:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721307.1124631; Tue, 14 May 2024 09:20:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oKi-0003mP-Nl; Tue, 14 May 2024 09:20:28 +0000
Received: by outflank-mailman (input) for mailman id 721307;
 Tue, 14 May 2024 09:20:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6oKh-0003mI-GR
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:20:27 +0000
Received: from mail-yw1-x112b.google.com (mail-yw1-x112b.google.com
 [2607:f8b0:4864:20::112b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 320a25e0-11d3-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:20:25 +0200 (CEST)
Received: by mail-yw1-x112b.google.com with SMTP id
 00721157ae682-61bed5ce32fso57407657b3.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:20:25 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54d6afasm66365541cf.24.2024.05.14.02.20.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:20:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 320a25e0-11d3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715678424; x=1716283224; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vzF0v9ZrsRWn8EzKCq+LtNfaFlXPhw3/JvfSXyguUS8=;
        b=sL2dOOcLEE8Fq40yfQpb+Zp7D66unNpluSXpcjMxin1utUrptTB0HcdJBcaWm1RW/o
         KCAM2vuuC6+gQAX/48n2TpuTz0bgERNnYzOSP8fCOk/bwKSiyiPs7YpBQstBloMJ+YNZ
         vlCymnnvGkcUH9WVXrZd3654lCNRaudHvG2XI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715678424; x=1716283224;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vzF0v9ZrsRWn8EzKCq+LtNfaFlXPhw3/JvfSXyguUS8=;
        b=XqCGf/W3Awk1zi3k7f1c5RUPlUsE29qdBycaZ9GCX0zUXB/c+z4Xjo/9Z+/lzde9ID
         J3s6c3KNq0qdx4x0T0i4z9qSDaEb1E7DF95A7QBixO8Avsz4nORyBI93KHzZDs5+mQ72
         bZqjzg5OEWW95pYwhQqzOz926Da9zaKHiLOmY+Pe7cQVrhu+eElBdbvB3yoWz9JZIFUN
         NM/IEEAaFrNos7gEZFT9AyxIRLHpQ/WenOdKlDNlyVlb+bHMdTzjNzn6gF5l8L8Aggbd
         N5A61ngyaEyWSaU5non8/zp87MwwIKDxKQkNkqNs7mEh3OiKegdGtfE2CLdDHlEcDnpf
         RtiA==
X-Gm-Message-State: AOJu0YybnhrM5N237fDKDpd8VNauFHkyVSHBJ5h3oGsK+kkDiWusmy+x
	95bp5a8tHhROWXQm69G+H9h1j4hYgbe6v1b10wQ4jEFzW5Ku+2n6LCjz6HMO0dY=
X-Google-Smtp-Source: AGHT+IHg6YYOkkD6bd6AcpKdnJjLtOfz/i55XfXIHfBdLkoAJ4MpkIXZLFva4SVVZ3KDuYbZDuj8eQ==
X-Received: by 2002:a81:a0cd:0:b0:61b:33f6:9bbb with SMTP id 00721157ae682-622affdf582mr117239747b3.28.1715678423737;
        Tue, 14 May 2024 02:20:23 -0700 (PDT)
Date: Tue, 14 May 2024 11:20:21 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Message-ID: <ZkMs1cknBFBWZoJG@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-7-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:33PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Also add a helper function to retrieve it. Change arch_mfns_in_direct_map
> to check this option before returning.
> 
> This is added as a Kconfig option as well as a boot command line option.
> While being generic, the Kconfig option is only usable for x86 at the moment.
> 
> Note that there remains some users of the directmap at this point. The option
> is introduced now as it will be needed in follow-up patches.

It's hard for me to evaluate whether the option name and the help text
is correct, because the implementation is not yet complete.  It would
be best if this was introduced after the implementation has gone in,
so that the reviewer can evaluate that the text matches the
implementation.  Now it's mostly a promise of what's yet to be
implemented.

> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     Changes in V2:
>         * Introduce a Kconfig option
>         * Reword the commit message
>         * Make opt_directmap and helper generic
> 
>     Changes since Hongyan's version:
>         * Reword the commit message
>         * opt_directmap is only modified during boot so mark it as
>           __ro_after_init
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index e760f3266e..743d343ffa 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
>  hardware guarantees (with, where available and known to Xen, respective
>  tweaks applied).
>  
> +### directmap (x86)
> +> `= <boolean>`
> +
> +> Default: `true`
> +
> +Enable or disable the directmap region in Xen.

Enable or disable fully populating the directmap region in Xen.

> +
> +By default, Xen creates the directmap region which maps physical memory
                                                          ^ all?
> +in that region. Setting this to no will sparsely populate the directmap,

"Setting this to no" => "Disabling this option will sparsely..."

> +blocking exploits that leak secrets via speculative memory access in the
> +directmap.
> +
>  ### dma_bits
>  > `= <integer>`
>  
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 7e03e4bc55..b4ec0e582e 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -28,6 +28,7 @@ config X86
>  	select HAS_PCI_MSI
>  	select HAS_PIRQ
>  	select HAS_SCHED_GRANULARITY
> +	select HAS_SECRET_HIDING
>  	select HAS_UBSAN
>  	select HAS_VPCI if HVM
>  	select NEEDS_LIBELF
> diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
> index 98b66edaca..54d835f156 100644
> --- a/xen/arch/x86/include/asm/mm.h
> +++ b/xen/arch/x86/include/asm/mm.h
> @@ -622,11 +622,17 @@ void write_32bit_pse_identmap(uint32_t *l2);
>  /*
>   * x86 maps part of physical memory via the directmap region.
>   * Return whether the range of MFN falls in the directmap region.
> + *
> + * When boot command line sets directmap=no, the directmap will mostly be empty
> + * so this will always return false.
>   */
>  static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>  {
>      unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
>  
> +    if ( !has_directmap() )
> +        return false;
> +
>      return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
>  }
>  
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index f84e1cd79c..bd6b1184f5 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>      if ( highmem_start )
>          xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
>  
> +    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");

IMO this wants to be printed as part of the speculation mitigations, see
print_details() in spec_ctrl.c

> +
>      /*
>       * Walk every RAM region and map it in its entirety (on x86/64, at least)
>       * and notify it to the boot allocator.
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 565ceda741..856604068c 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -80,12 +80,29 @@ config HAS_PMAP
>  config HAS_SCHED_GRANULARITY
>  	bool
>  
> +config HAS_SECRET_HIDING
> +	bool
> +
>  config HAS_UBSAN
>  	bool
>  
>  config MEM_ACCESS_ALWAYS_ON
>  	bool
>  
> +config SECRET_HIDING
> +    bool "Secret hiding"
> +    depends on HAS_SECRET_HIDING

IMO 'SECRET_HIDING' is too generic, this wants a more specific name.
Maybe SPARCE_DIRECTMAP or some such.

> +    help
> +		The directmap contains mapping for most of the RAM which makes domain
> +		memory easily accessible. While making the performance better, it also makes
> +		the hypervisor more vulnerable to speculation attacks.
> +
> +		Enabling this feature will allow the user to decide whether the memory
> +		is always mapped at boot or mapped only on demand (see the command line
> +		option "directmap").
> +
> +		If unsure, say N.
> +
>  config MEM_ACCESS
>  	def_bool MEM_ACCESS_ALWAYS_ON
>  	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 7c1bdfc046..9b7e4721cd 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -174,6 +174,11 @@ paddr_t __ro_after_init mem_hotplug;
>  static char __initdata opt_badpage[100] = "";
>  string_param("badpage", opt_badpage);
>  
> +bool __ro_after_init opt_directmap = true;
> +#ifdef CONFIG_HAS_SECRET_HIDING
> +boolean_param("directmap", opt_directmap);
> +#endif
> +
>  /*
>   * no-bootscrub -> Free pages are not zeroed during boot.
>   */
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 7561297a75..9d4f1f2d0d 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -167,6 +167,13 @@ extern unsigned long max_page;
>  extern unsigned long total_pages;
>  extern paddr_t mem_hotplug;
>  
> +extern bool opt_directmap;
> +
> +static inline bool has_directmap(void)
> +{
> +    return opt_directmap;

This likely wants:

return IS_ENABLED(CONFIG_HAS_SECRET_HIDING) && opt_directmap;

So that when HAS_SECRET_HIDING is build time disabled the compiler can
likely eliminate the code.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:25:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:25:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721313.1124642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oPG-0005Fk-Cn; Tue, 14 May 2024 09:25:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721313.1124642; Tue, 14 May 2024 09:25:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oPG-0005Fd-8l; Tue, 14 May 2024 09:25:10 +0000
Received: by outflank-mailman (input) for mailman id 721313;
 Tue, 14 May 2024 09:25:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oPF-0005FX-Fo
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:25:09 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id da5b4ce0-11d3-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:25:07 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51fea3031c3so7071493e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:25:07 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c6d7sm700934666b.59.2024.05.14.02.25.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:25:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da5b4ce0-11d3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715678706; x=1716283506; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=mX3TB+de2A03snoFMDWj2SlfJr+ktFoulnQl/oak/pU=;
        b=RRH3O6J+fr0KJ+sCKahpz/65U4z7ybyqgHe5B9dkZZAGLXQr00/Yckyptxtl1/GHQN
         mfwGq5gPlr9A9tMhyLnWR5XmJVZhT9gvSnfxB5HpDBl1tO5noJwq0KMrroRC530qszfw
         8MxNNACNfFeURWkHuR0SesGtU1irMjolqyOYKsDwNqI8FtMstoSeoBZbqle1X5ajgytO
         Pn6oSh6ThaYmLdBtDgleAmLYeIK+4iZGNN3ydCW8bTJiK5AGeq91fcCG7T5PHGVTVhPK
         /P5jiGJEh+hDfSDGywzLIhsatS/XDHXHgRibuvpA83dIXFarRacT0MdgazAovNDV1s/u
         ZbaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715678706; x=1716283506;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mX3TB+de2A03snoFMDWj2SlfJr+ktFoulnQl/oak/pU=;
        b=GoU9q1NYZvqrb0+ZDFb6c22fVybz8+Xo015Jz/ElF9EZxuoZPyM3jfl7mgK8AWj+EN
         QF1UmO69sMLHrqb2dC7ZWiT51p05wxEyCXvbo4V1ssaXmbuSq+1ENuTJlMU+A8zn7vjN
         8NOzrKgHCh7nUm0isJl3vfFZKlMi8pXiuQ4XnDZ/2N7/4emL/dVEJ8wL8PCknI77a4PH
         T+umLjIpBfT7eWd7Vcs/Untji2WbwJrJ5nMGkY4eBCwUJbq/rc+MuWYoogMAu5ynOQtW
         AZgHkGo0Fb3FGPu+HCkS+jQ0zQj6jA6J7y4MVZcU3rxrnlDIBdloQtxmEyOvy9xBw5+I
         8PWg==
X-Forwarded-Encrypted: i=1; AJvYcCV+Nz4WNCiAYqEhYGJ8/8dHwmKcSQTt63EEMISKWrn7e72aosBBdfNtvoYwBARyyxz8nl52TZyubq0G0N3YwJ7Gx2/JgJAaLqn6Z3kgfE4=
X-Gm-Message-State: AOJu0Yx6JeyabCyhfcMC4TZkdRzyvQ5vEIhBcUx/SbKThT0EetvR6TBR
	JnpPpDe557K8rDUxHz2RwDbUCLYPScufS1CrssAnn9X8WT2DxTsAzIZ4qjHnOjFG5NAA1A3ybL4
	=
X-Google-Smtp-Source: AGHT+IHtTza15ZNLlFIOwB9SeocQPby51iuDtQ775c/csh/M8Tu3U9lu5oES5ymGeo1fl6luoHTITw==
X-Received: by 2002:a05:6512:1281:b0:522:3356:3b8 with SMTP id 2adb3069b0e04-5223356040bmr10026735e87.27.1715678706596;
        Tue, 14 May 2024 02:25:06 -0700 (PDT)
Message-ID: <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
Date: Tue, 14 May 2024 11:25:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Daniel Smith <dpsmith@apertussolutions.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 03.04.2024 08:16, Jan Beulich wrote:
> On 02.04.2024 19:06, Andrew Cooper wrote:
>> The commit makes a claim without any kind of justification.
> 
> Well, what does "have no business" leave open?
> 
>> The claim is false, and the commit broke lsevtchn in dom0.
> 
> Or alternatively lsevtchn was doing something that was never meant to work
> (from Xen's perspective).
> 
>>  It is also quite
>> obvious from XSM_TARGET that it has broken device model stubdoms too.
> 
> Why would that be "obvious"? What business would a stubdom have to look at
> Xen's side of an evtchn?
> 
>> Whether to return information about a xen-owned evtchn is a matter of policy,
>> and it's not acceptable to short circuit the XSM on the matter.
> 
> I can certainly accept this as one possible view point. As in so many cases
> I'm afraid I dislike you putting it as if it was the only possible one.
> 
> In summary: The supposed justification you claim is missing in the original
> change is imo also missing here then: What business would any entity in the
> system have to look at Xen's side of an event channel? Back at the time, 3
> people agreed that it's "none".

You've never responded to this reply of mine, or its follow-up. You also
didn't chime in on the discussion Daniel and I were having. I consider my
objections unaddressed, and in fact I continue to consider the change to
be wrong. Therefore it was inappropriate for you to commit it; it needs
reverting asap. If you're not going to do so, I will.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:40:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:40:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721323.1124652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ody-0001U4-Ja; Tue, 14 May 2024 09:40:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721323.1124652; Tue, 14 May 2024 09:40:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ody-0001Tx-Fw; Tue, 14 May 2024 09:40:22 +0000
Received: by outflank-mailman (input) for mailman id 721323;
 Tue, 14 May 2024 09:40:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6odx-0001SY-4i
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:40:21 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f98bdeae-11d5-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:40:19 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-34e667905d2so3615570f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:40:19 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbca98sm13211119f8f.112.2024.05.14.02.40.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:40:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f98bdeae-11d5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715679618; x=1716284418; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Cz6ZgprpL01AnBFkJIjoqqgc4nrYCjjDmKPGTX+S+lQ=;
        b=N9XTBSvMCqsn/DfwTeE2lr11aJciGpmwaA/kfpPRjJWb6/2wAMXMETBYiBDguITiDq
         w88J8L92uUPfGbvqLIN2W4ihWS2jm4YHIPKHCVutRTyUtgx+PHkbgb8zWwhzOcbEnhSO
         aMfhJAOFGelgNOMimxT416EhaD7hMeBlZ/xHU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715679618; x=1716284418;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Cz6ZgprpL01AnBFkJIjoqqgc4nrYCjjDmKPGTX+S+lQ=;
        b=a6mV+MBrXHnsL3C6QnoZ8wqLcVqwLJ0WKvJ/ajGuZZJywH+mrvNUkP7llkW6fXVn6s
         5pZtfxj1bONVErHMP2x9BGvQi7XaSEwxcBe3+lk+N+1hCuGxXXxIgVA2OX/87XJXxL+H
         Q6NVn4b05T+yCdm469VUGkMSp1tYeredzbw+UvG0JR5FPxLihp3sPFPc+zGJGLac1sRi
         yDmZxRedJKMzmwQpBxGydvp0/81oJi+dWJ17uC/PqyxLfnZ+RHSnQlBqWMqA6t9EL3rD
         hxXiWT/zYDuv2T48MeJZJF40MRzaJi+TfdF2Xpr7E4jRiZqlTuc/Epdh+l3WM1nxfkPk
         KHzg==
X-Gm-Message-State: AOJu0Yw+RLtioREzyTgJVXoa1wI9A7slZZVsSGP0ysUcsbMIt4MrDQ/u
	le27f+ufxUuOU5rpi+H/NIp+wrXLD16fZZ4Ahe/KJKHgkzvw9Pare1hZ8XYGwTz4xAbDdXQ0IWk
	h
X-Google-Smtp-Source: AGHT+IEuiPUSp88xCtbS36VqFmmn68MDC6tZUSihDB11SAq8lyr5gf+8Orle4O65EowrznHhi4SzNA==
X-Received: by 2002:adf:fe04:0:b0:34f:f540:bdc9 with SMTP id ffacd0b85a97d-3504a63300amr7932140f8f.9.1715679617892;
        Tue, 14 May 2024 02:40:17 -0700 (PDT)
Date: Tue, 14 May 2024 11:40:16 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Message-ID: <ZkMxgDPYgaKnmRRE@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-8-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
> From: Julien Grall <jgrall@amazon.com>
> 
> PMAP will be used in a follow-up patch to bootstrap map domain
> page infrastructure -- we need some way to map pages to setup the
> mapcache without a direct map.
> 
> The functions pmap_{map, unmap} open code {set, clear}_fixmap to break
> the loop.
> 
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     The PMAP infrastructure was upstream separately for Arm since
>     Hongyan sent the secret-free hypervisor series. So this is a new
>     patch to plumb the feature on x86.
> 
>     Changes in v2:
>         * Declare PMAP entries earlier in fixed_addresses
>         * Reword the commit message
> 
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index b4ec0e582e..56feb0c564 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -27,6 +27,7 @@ config X86
>  	select HAS_PCI
>  	select HAS_PCI_MSI
>  	select HAS_PIRQ
> +	select HAS_PMAP

Shouldn't it be selected by HAS_SECRET_HIDING rather than being
unconditionally selected?

>  	select HAS_SCHED_GRANULARITY
>  	select HAS_SECRET_HIDING
>  	select HAS_UBSAN
> diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
> index 516ec3fa6c..a7ac365fc6 100644
> --- a/xen/arch/x86/include/asm/fixmap.h
> +++ b/xen/arch/x86/include/asm/fixmap.h
> @@ -21,6 +21,8 @@
>  
>  #include <xen/acpi.h>
>  #include <xen/pfn.h>
> +#include <xen/pmap.h>
> +
>  #include <asm/apicdef.h>
>  #include <asm/msi.h>
>  #include <acpi/apei.h>
> @@ -53,6 +55,8 @@ enum fixed_addresses {
>      FIX_PV_CONSOLE,
>      FIX_XEN_SHARED_INFO,
>  #endif /* CONFIG_XEN_GUEST */
> +    FIX_PMAP_BEGIN,
> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,

This would better have

#ifdef CONFIG_HAS_PMAP

guards?

>      /* Everything else should go further down. */
>      FIX_APIC_BASE,
>      FIX_IO_APIC_BASE_0,
> diff --git a/xen/arch/x86/include/asm/pmap.h b/xen/arch/x86/include/asm/pmap.h
> new file mode 100644
> index 0000000000..62746e191d
> --- /dev/null
> +++ b/xen/arch/x86/include/asm/pmap.h
> @@ -0,0 +1,25 @@
> +#ifndef __ASM_PMAP_H__
> +#define __ASM_PMAP_H__
> +
> +#include <asm/fixmap.h>
> +
> +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn)
> +{
> +    unsigned long linear = (unsigned long)fix_to_virt(slot);
> +    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
> +
> +    ASSERT(!(l1e_get_flags(*pl1e) & _PAGE_PRESENT));
> +
> +    l1e_write_atomic(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR));
> +}
> +
> +static inline void arch_pmap_unmap(unsigned int slot)
> +{
> +    unsigned long linear = (unsigned long)fix_to_virt(slot);
> +    l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)];
> +
> +    l1e_write_atomic(pl1e, l1e_empty());
> +    flush_tlb_one_local(linear);
> +}
> +
> +#endif /* __ASM_PMAP_H__ */

We usually add a:

/*
 * Local variables:
 * mode: C
 * c-file-style: "BSD"
 * c-basic-offset: 4
 * indent-tabs-mode: nil
 * End:
 */

Footer.  No strict requirement, but it's nice so that your editor
already picks the correct defaults for tabs &c.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:40:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:40:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721324.1124661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oeI-0001sZ-Ue; Tue, 14 May 2024 09:40:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721324.1124661; Tue, 14 May 2024 09:40:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oeI-0001sR-Rz; Tue, 14 May 2024 09:40:42 +0000
Received: by outflank-mailman (input) for mailman id 721324;
 Tue, 14 May 2024 09:40:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oeH-0001SY-G7
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:40:41 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06271144-11d6-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 11:40:39 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so1263838466b.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:40:39 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eb5ec296sm111706a12.35.2024.05.14.02.40.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:40:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06271144-11d6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715679639; x=1716284439; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jlCLTqw00BwgOzSCcZqnQxUrm1RkTJWdJpeT/SE9zrY=;
        b=J8kzjHVgaBRmo2MklPZ/O4AlsYCiEdKjAXqhYwC8nZ+Xmk4G/i0B1iQrmLzlAXrFi5
         Nq0ALaHcJaxKSW9jo84XycYLjhuS4odfMmd+9s+97DAqniaZjVU2If6GzQyAo2FXFAwl
         7avM3IA/Ld/fiuwVh739tu8j7qMq05PESAz3krNtUhTLfu8O9aDUE4DIFl7Y68S+b20M
         ll0bb4QMSONlYjYUd6rVQTt08JwtxAKnm3xKVFtKwD5iOk7OyAK99vciTTl2tk9in0xX
         yvLL1wmEgPBfpc+q8U7UV1QLmOxPPITFQUW9JscpB+9/ZoH9IKY8Ai4y96KSQQHTDKvX
         0JxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715679639; x=1716284439;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jlCLTqw00BwgOzSCcZqnQxUrm1RkTJWdJpeT/SE9zrY=;
        b=DRkthe5clrpTrXxAstX2BvJgRjiskHywN0YnX0OIzJkPPmi/1/BHFPW9fbnpjL0/SU
         OKTkeOL7dPn3SStAk6LA+1ftUB6BG0rsdXTs92tedKU+5kEmz7BaMBVp4toD655vfDdw
         3Qs4s8weZv8KkXkuUTKabCLuN0z5isnc3PiR3vpGGUTuiWRYVDeOoXd26W3EjHW8z9BU
         lRtlmorW41zbZDTt+yuut3w9y+MP9Gc/Hylbz6h5l62lWClawy98OWDqi0/EjDsj4ay5
         AbFHfI4yn/7CPzGdPHRCSQ+BTeW/TojThCaLiPpdcJt1nwYXZDl5TAR9ApgjylcEOo+o
         n61Q==
X-Forwarded-Encrypted: i=1; AJvYcCW8QFTOp8hLNLg4cVH3ILlueuWZ8XeglrUPl84nDuvpbaBv9PnJSSdKmWh/kSB94FNKIjiELxGvSXfjZRifbTCzl9v6z/NtIXLQgeGjhOE=
X-Gm-Message-State: AOJu0YwSA4kxMbf41AQBHKn/0mVfjYt6qfF6ogwGl5JYLD6Jr4cg36lv
	TZsz67Y0Q/FTVW1+z4asRdZaAEL1YYEAoMRxj1OY2yCCXaKt5IP7/PonTN7mLg==
X-Google-Smtp-Source: AGHT+IFieXlOHeQ1/bLYZ/47+S4VlfvWbubmp6QSsPNPmOQVNqVBETnp/QMKXSlWMZB/ORDfkLWZQw==
X-Received: by 2002:a50:8d07:0:b0:571:e272:296 with SMTP id 4fb4d7f45d1cf-5734d67ee8emr8507092a12.27.1715679639180;
        Tue, 14 May 2024 02:40:39 -0700 (PDT)
Message-ID: <aae35115-95a0-40d2-bc83-c89de1e448cf@suse.com>
Date: Tue, 14 May 2024 11:40:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/cpu-policy: Fix migration from Ice Lake to Cascade
 Lake
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240507112957.1701824-1-andrew.cooper3@citrix.com>
 <20240507134540.1714274-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240507134540.1714274-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.05.2024 15:45, Andrew Cooper wrote:
> Ever since Xen 4.14, there has been a latent bug with migration.
> 
> While some toolstacks can level the features properly, they don't shink
> feat.max_subleaf when all features have been dropped.  This is because
> we *still* have not completed the toolstack side work for full CPU Policy
> objects.
> 
> As a consequence, even when properly feature levelled, VMs can't migrate
> "backwards" across hardware which reduces feat.max_subleaf.  One such example
> is Ice Lake (max_subleaf=2 for INTEL_PSFD) to Cascade Lake (max_subleaf=0).
> 
> Extend the max policies feat.max_subleaf to the hightest number Xen knows
> about, but leave the default policies matching the host.  This will allow VMs
> with a higher feat.max_subleaf than strictly necessary to migrate in.
> 
> Eventually we'll manage to teach the toolstack how to avoid creating such VMs
> in the first place, but there's still more work to do there.

Can you explain to me in how far "x86/CPUID: shrink max_{,sub}leaf fields
according to actual leaf contents" would not already have taken care of
this (and not just for sub-leaves of leaf 7), if only it (at least its
more recent versions) was ever seriously looked at? I realize there was
one todo item left there (addressing of which I could probably have used
some help with), but that shouldn't have entirely prevented any progress.
(If I'm not mistaken an earlier version had once gone in, but then needed
to be reverted.)

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:42:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:42:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721330.1124673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ofp-0002ew-9j; Tue, 14 May 2024 09:42:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721330.1124673; Tue, 14 May 2024 09:42:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ofp-0002ep-5d; Tue, 14 May 2024 09:42:17 +0000
Received: by outflank-mailman (input) for mailman id 721330;
 Tue, 14 May 2024 09:42:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6ofn-0002ef-9l
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:42:15 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3e9414d9-11d6-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:42:14 +0200 (CEST)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-34e0d8b737eso3737291f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:42:14 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8a77easm13156662f8f.53.2024.05.14.02.42.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 02:42:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e9414d9-11d6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715679734; x=1716284534; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vY59CoxrIbE4fN3ZkuMSxQYJSIbg908uYli81yCD2HI=;
        b=jjmnaIJ88nw+UbqFftW/W4Lj/oS2eqVmDH0Z+cNHOF+xwMYAsFvZ6l14lRHy5yEZeA
         lOAp6k8IL0qZHUELR8fkpCZt/963bD2e9z3e1p7M1q8JE96V3k3Z060Q/mJCeB+wQOrd
         HFuIR0P543cRl86d+QsPTgumVHZtW2a5WDZrE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715679734; x=1716284534;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vY59CoxrIbE4fN3ZkuMSxQYJSIbg908uYli81yCD2HI=;
        b=LbYeI0eJJNqhsvEgWpM19EtaW81SyZkdXCP02BPZrb4DcUpp8G76BgeBYTcAHXxfjd
         Kn60vpxt002UdAAQX1ed7dghrBUc7Nwc2K52SSJ7iA6exDx+nytw/3vrHC1BS1p9MtfD
         7HTf/x/w2/stG26van/whzSuTQhVatKHv0XA042iNfrTtie3xLUpMeSA+mQZocjiUY9q
         iV3SRNG3IBsdQtUEcJQJC1oPaVHkAFmmgFu+u93hO19c10q8FSBxjfSvTl5BWyr3DaeL
         PtzBSj19WSC9R6cJmXIX8v28k3xPZT84a7DGt8DyV5w7xKWixT+UT1mSLcHQBm53/l04
         AOoA==
X-Gm-Message-State: AOJu0YzKXHF3PfQEvQR2koMWMWdzyV2CUSjbYrXgSYgAyWR0tg5vxdjw
	NNZN8VW1D0yb7beNZnfJwySvAjARm46rq2itvXR2ike1p/dQ4ds0jIoevTlIOtA=
X-Google-Smtp-Source: AGHT+IH2/sXJvM+TaMc2oyky8D1AusD1bZZL2kd7W1S/+CVZMrMNhlkJsucRzxw/38zixWOIWszkFw==
X-Received: by 2002:adf:f78a:0:b0:34d:b3a0:5614 with SMTP id ffacd0b85a97d-3504989e643mr9414973f8f.8.1715679733811;
        Tue, 14 May 2024 02:42:13 -0700 (PDT)
Date: Tue, 14 May 2024 11:42:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH V3 (resend) 08/19] xen/x86: Add build assertion for
 fixmap entries
Message-ID: <ZkMx9O5MtR_eTr1g@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-9-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-9-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:35PM +0000, Elias El Yandouzi wrote:
> The early fixed addresses must all fit into the static L1 table.
> Introduce a build assertion to this end.
> 
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>      Changes in v2:
>          * New patch
> 
> diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
> index a7ac365fc6..904bee0480 100644
> --- a/xen/arch/x86/include/asm/fixmap.h
> +++ b/xen/arch/x86/include/asm/fixmap.h
> @@ -77,6 +77,11 @@ enum fixed_addresses {
>  #define FIXADDR_SIZE  (__end_of_fixed_addresses << PAGE_SHIFT)
>  #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
>  
> +static inline void fixaddr_build_assertion(void)
> +{
> +    BUILD_BUG_ON(FIX_PMAP_END > L1_PAGETABLE_ENTRIES - 1);
> +}

Just introduce the BUILD_BUG_ON somewhere else, no need for a new
function just for this.

Adding the BUILD_BUG_ON() to pmap_map() would be perfectly fine.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:43:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:43:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721333.1124682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ogf-0003Be-HQ; Tue, 14 May 2024 09:43:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721333.1124682; Tue, 14 May 2024 09:43:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ogf-0003BX-EM; Tue, 14 May 2024 09:43:09 +0000
Received: by outflank-mailman (input) for mailman id 721333;
 Tue, 14 May 2024 09:43:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oge-0002ef-Ps
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:43:08 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ead2d31-11d6-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:43:08 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51aa6a8e49aso7359082e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:43:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfde7sm7320571a12.48.2024.05.14.02.43.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:43:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ead2d31-11d6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715679787; x=1716284587; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lCIn8xBLTXn2biVgtxyZqDUw0oeSBP2s95BOkhXjPMY=;
        b=QRLo7bVjJfUkSdoFhlut5A0HNuTyo7qwOX5m82cgTY52TPinzl9w0jyEivO6XAQ2yl
         bVZEYpHzkWoK2fVLvaCQdqmlmpgaRpLX9/ep9dWC673uvKkqOw/Qi2dftiTFUF8uwKfq
         TUnBl8UvBGd2yj+edAh4KiHSUxdhgCKq2lVESPnOFGbBa0UAWrK0aV03lROnHDgEnqKp
         Gec4gVWBKSgS/b5VMhuZA9EwXwMt8Z3MB32xXW51zxbU6nsYq43THeXJ8UZDZCN3DYD9
         yN+ApDvtJig3zdLOW5tehbxYZEHz2CBbnJWoiPZC5QoKmZf9PeB5LVrH10TzMkwyuG+t
         rP4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715679787; x=1716284587;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lCIn8xBLTXn2biVgtxyZqDUw0oeSBP2s95BOkhXjPMY=;
        b=mWuVHJhtRMtZI1unLOuHi11fShpplf11rsbtIIOjDXRdVbHzgdcXCWEe7CEJHKep4s
         7yQzyTDs4IWvpPnB3rA51uskjs85u5eY4Gx432YZwNB92XzvHBuLgYYZL7ghRGN4HTcW
         LDJWJ4NEzrYfXIaNQvbWQyfZfTnxDoxfmbCuGvc/KvsypWD1wwvpBShRPbgFYv56FAkZ
         C94i38T5TeKwmGr1lg0OeU9tbaC3T3++EJ35bumARC2jVOExOJf8FTQK5HBLewQCVZO7
         0w0p4H/LWoBILPvCIXUafPNy7/J2KK2KsXdVG5wV6YCDrW9XhxYAl+A/rZxTuUVdgUkL
         BMNQ==
X-Gm-Message-State: AOJu0YwVr7Npzhct/+DvTiCUKr7aJJgzdgxP6BYuZ9Ky2S9cfNGAjFd5
	PSu29zFdlzFYMApV9gz9QGgCL4jI/PHXLPVjeS4MM0DqQIWMugkQVJwne2OSGg==
X-Google-Smtp-Source: AGHT+IHOzBLa/10digZoGTNiPwLkeJh/rgFERawkE0fMesKKOgC0YEkREvjI+3bmzWHmx7pQtq5gJg==
X-Received: by 2002:a05:6512:3a83:b0:51a:c8bb:fcf7 with SMTP id 2adb3069b0e04-5220fa7208dmr10148660e87.3.1715679787554;
        Tue, 14 May 2024 02:43:07 -0700 (PDT)
Message-ID: <87950da7-f765-4656-808f-6c63a29ced3d@suse.com>
Date: Tue, 14 May 2024 11:43:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com> <ZkMxgDPYgaKnmRRE@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMxgDPYgaKnmRRE@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 11:40, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
>> @@ -53,6 +55,8 @@ enum fixed_addresses {
>>      FIX_PV_CONSOLE,
>>      FIX_XEN_SHARED_INFO,
>>  #endif /* CONFIG_XEN_GUEST */
>> +    FIX_PMAP_BEGIN,
>> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
> 
> This would better have
> 
> #ifdef CONFIG_HAS_PMAP
> 
> guards?

That's useful only when the option can actually be off in certain
configurations, isn't it?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:45:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:45:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721335.1124692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oiz-0003kh-Ty; Tue, 14 May 2024 09:45:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721335.1124692; Tue, 14 May 2024 09:45:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oiz-0003ka-QN; Tue, 14 May 2024 09:45:33 +0000
Received: by outflank-mailman (input) for mailman id 721335;
 Tue, 14 May 2024 09:45:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oiz-0003kU-8y
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:45:33 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b4af3207-11d6-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:45:32 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51f12ccff5eso7413700e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:45:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bebb6d5sm7300261a12.34.2024.05.14.02.45.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:45:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4af3207-11d6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715679932; x=1716284732; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DVOcvwwuLFbSN2m1WLpcMqmAgm5xwYO31xM9tDej/xg=;
        b=fPNJanHhmfu8zCe29q/czxp1kn0FOZm1SxoSbgEyyBHIS9kwqAGgIXHKcv9Q+36tr6
         pV0fle5Ks7p+q+7ct012qq5LUAiEkxQDIG14Y6KzVCaPPl94sgtZA+Zv+Aly0N1EXqCb
         KeZh1qak4mQfRlADghvlEk7mKAk2FAeZoF+Q6whacqqqRBT3JZ72jtY5eZs3H5htBKz+
         TR0sqwKJw6W2GHvRHfA4I54Umh3X6lpLD/NC5kg7DlEVAaLrWiZCbg/2KQ+xM2R3cDSB
         BpqDNkKHYYlFdYdIcYO7PmltzCk2j7C6eluKUVyKlrEu4XOXW8Ae8pPWVQhoCu4qCaG6
         xG7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715679932; x=1716284732;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DVOcvwwuLFbSN2m1WLpcMqmAgm5xwYO31xM9tDej/xg=;
        b=wAvtV0ZQuKwngdfSqzfnVp/rdN1Agzll6WtIueVpCdx0B/BrRTR6nK/zS0p9n0GDkp
         Z/y7QKGRpzEGy9XOps5JVsFphrXBRn89B60sY7g/WhlCpB9yWkAGYwzsuGXiB+EDRtIu
         Na92cBgBTmw3Dvn+GlPdcGXCs4OVlJ3Ar9UhZKK+SLDdxJWlx9hJdeYmqJ/6qTKePUBz
         0o+BKSLNOJV9TK3iHR9IbbGYTcCT8g33UON7LggjizkkVRKoliZuS/X/crE7UZXqMs4a
         T8QVB553U351tV/T8UcgioEGWDeU4Z8LDdep5mDahW3XQa++mrQPo6YGcQJhEAQtU2jF
         KqLA==
X-Gm-Message-State: AOJu0YzbVqqTcsB/EfcGeBUa9AwIHhPgrzxzq2uLClXthiXK76eO7ML5
	puQwGbNxD9yD2MNODv9EL4AZ+yY04QzZhblASWy4n6lv4BLBTiBkoodX84eQNA==
X-Google-Smtp-Source: AGHT+IGojVhQq61pP0/4k4X1JBBOBbd0KmUNhAsTZkIYljp4nXJSFgiZSvEAxWCRRK/xLmK0OLB6og==
X-Received: by 2002:a19:6455:0:b0:51d:ed1:b44c with SMTP id 2adb3069b0e04-5221006f7e8mr7630891e87.19.1715679932001;
        Tue, 14 May 2024 02:45:32 -0700 (PDT)
Message-ID: <f77ec9b8-d780-42a7-b527-efba0e121286@suse.com>
Date: Tue, 14 May 2024 11:45:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 08/19] xen/x86: Add build assertion for fixmap
 entries
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Andrew Cooper <andrew.cooper3@citrix.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-9-eliasely@amazon.com> <ZkMx9O5MtR_eTr1g@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMx9O5MtR_eTr1g@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 11:42, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:35PM +0000, Elias El Yandouzi wrote:
>> The early fixed addresses must all fit into the static L1 table.
>> Introduce a build assertion to this end.
>>
>> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
>>
>> ----
>>
>>      Changes in v2:
>>          * New patch
>>
>> diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
>> index a7ac365fc6..904bee0480 100644
>> --- a/xen/arch/x86/include/asm/fixmap.h
>> +++ b/xen/arch/x86/include/asm/fixmap.h
>> @@ -77,6 +77,11 @@ enum fixed_addresses {
>>  #define FIXADDR_SIZE  (__end_of_fixed_addresses << PAGE_SHIFT)
>>  #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
>>  
>> +static inline void fixaddr_build_assertion(void)
>> +{
>> +    BUILD_BUG_ON(FIX_PMAP_END > L1_PAGETABLE_ENTRIES - 1);
>> +}
> 
> Just introduce the BUILD_BUG_ON somewhere else, no need for a new
> function just for this.

And especially not in an inline function (thus triggering the potential error
perhaps several dozen times in a highly parallel build).

> Adding the BUILD_BUG_ON() to pmap_map() would be perfectly fine.

Nevertheless we have build_assertions() functions in a couple of places, so
yes, there are precedents to doing so rather that putting the constructs at
more or less random places.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:52:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:52:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721343.1124702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6opA-0005g6-Ha; Tue, 14 May 2024 09:51:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721343.1124702; Tue, 14 May 2024 09:51:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6opA-0005fz-Di; Tue, 14 May 2024 09:51:56 +0000
Received: by outflank-mailman (input) for mailman id 721343;
 Tue, 14 May 2024 09:51:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6op9-0005ft-OC
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:51:55 +0000
Received: from mail-vs1-xe2f.google.com (mail-vs1-xe2f.google.com
 [2607:f8b0:4864:20::e2f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9835e41c-11d7-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:51:54 +0200 (CEST)
Received: by mail-vs1-xe2f.google.com with SMTP id
 ada2fe7eead31-47efc7f2625so1439722137.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:51:54 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df5b46a26sm66236281cf.80.2024.05.14.02.51.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:51:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9835e41c-11d7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715680313; x=1716285113; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=A/S/lX2fSmBt9sPmgcTXAePdsSzUzYBE1ivMHp5kgxc=;
        b=Y+8y9RsDEd3Pjons9r+p1RA6H4WFgVhNha943/Tq3WXUvOE1dZwl9CJq5Q83tyMvDN
         w8RTDGMXGEX+/SrmMaMnwYxuNGT/NlNmJZocKhZyMr09U2zphuAoo3z3IDV+zoTS0pXP
         HF1jap4QZTRQfd/LdWIN3Uh0NbdFjqt/+0vhU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715680313; x=1716285113;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=A/S/lX2fSmBt9sPmgcTXAePdsSzUzYBE1ivMHp5kgxc=;
        b=Kernmc4kVj8zFXZRYD0w0eGTntENLXUk6nMt6N6b03rVOq6/PBg5+pZag4zsCDKoRI
         skqZwTQON32glmbOJ21uZa00tBZQiEi/KOqwJd2t84gYVdQMZMaPbVpVCDyFxCDmYrun
         u5RGu5dTDk2W2LzyxRngYOvPFw3rRDad6pjrvSAiGhWYbN5cmmReR3hdW1QNV0MG9UEE
         N+iW5mJOxs2iwhvRTrYRqm/6Z4aYlX/rYAHn2yHZCzNKYH4zZ/gK//nygvRf3LZI9Z3z
         +abPTXL3B1LQLytDgo1+73k2P1HFBMiooVgyrnOwyHioAR4JiA4YM8a3DMlpT3RlLnJm
         4JMg==
X-Forwarded-Encrypted: i=1; AJvYcCWG3PAFTLn1ZrNtJ85iwLBY85VS++nnt8eeiciPlF4jdgxoOOjZl5CN8HK/NnVPwUqJfBCkIMKip7bomF0sMVUjyq77TO3m4+iwu5jX7N8=
X-Gm-Message-State: AOJu0YwGSk+J0DV06EO3SlWdc1BL72eHU4fOxDab/7+Ve6GmgbmJqdWK
	RBU+O/VG86atfOdJx0VeIZHvvQTLln4ulA3Er3bHaWTSHCw7OKynw/1BNyvTI2M=
X-Google-Smtp-Source: AGHT+IF6Xjqu08K5kF3pXYdQkM93xxrKhtg02zJq7iWauAMxe5+7kTz0KHjmy2R3OXeH7XEfh6MM2g==
X-Received: by 2002:a05:6102:d8d:b0:47b:ef4f:edb3 with SMTP id ada2fe7eead31-48077de2bd1mr12216766137.8.1715680313401;
        Tue, 14 May 2024 02:51:53 -0700 (PDT)
Message-ID: <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
Date: Tue, 14 May 2024 10:51:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Daniel Smith <dpsmith@apertussolutions.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 10:25 am, Jan Beulich wrote:
> On 03.04.2024 08:16, Jan Beulich wrote:
>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>> The commit makes a claim without any kind of justification.
>> Well, what does "have no business" leave open?
>>
>>> The claim is false, and the commit broke lsevtchn in dom0.
>> Or alternatively lsevtchn was doing something that was never meant to work
>> (from Xen's perspective).
>>
>>>  It is also quite
>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>> Why would that be "obvious"? What business would a stubdom have to look at
>> Xen's side of an evtchn?
>>
>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>> and it's not acceptable to short circuit the XSM on the matter.
>> I can certainly accept this as one possible view point. As in so many cases
>> I'm afraid I dislike you putting it as if it was the only possible one.
>>
>> In summary: The supposed justification you claim is missing in the original
>> change is imo also missing here then: What business would any entity in the
>> system have to look at Xen's side of an event channel? Back at the time, 3
>> people agreed that it's "none".
> You've never responded to this reply of mine, or its follow-up. You also
> didn't chime in on the discussion Daniel and I were having. I consider my
> objections unaddressed, and in fact I continue to consider the change to
> be wrong. Therefore it was inappropriate for you to commit it; it needs
> reverting asap. If you're not going to do so, I will.

You tried defending breaking a utility with "well it shouldn't exist then".

You don't have a leg to stand on, and two maintainers of relevant
subsystems here just got tired of bullshit being presented in place of
any credible argument for having done the change in the way you did.

The correct response was "Sorry I broke things.  Lets revert this for
now to unbreak, and I'll see about reworking it to not intentionally
subvert Xen's security mechanism".

As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
had for the principle of the change based on the absurdity of your
arguments.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:52:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:52:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721344.1124712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6opB-0005uf-R3; Tue, 14 May 2024 09:51:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721344.1124712; Tue, 14 May 2024 09:51:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6opB-0005uW-OJ; Tue, 14 May 2024 09:51:57 +0000
Received: by outflank-mailman (input) for mailman id 721344;
 Tue, 14 May 2024 09:51:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6opA-0005ft-OX
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:51:56 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9963fb68-11d7-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:51:56 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e6792ea67dso51926611fa.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:51:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d92sm697859066b.85.2024.05.14.02.51.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:51:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9963fb68-11d7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715680315; x=1716285115; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Ewjtb4si4PNJIEUXGOGdyvrW80i/rihnaPh645nuA5A=;
        b=F8fQ8OECISDldJXjP6GYvqEs3RlaNHY/OmR6LzNzZWO74F+ktwclbaVnIfAeMx8uZX
         yybZ8hVE5t+mcwlXec+cGQ3RYD2Qltgh6oQOH9qA2S7I3S5GcQwZqTwoylkbx0GsZo7v
         XE5Qbj1JfxeUZfjDtOW3mtWWOwT66vaFdu55WHc0FrmTNmfY+rUKIUNeL57SOgssvnMm
         pCUl/nFP3Hrdf5vUlWF2OTIhpxv/ms0MjZa1pSilglUSPby7TjWlEh1FjlJabDT9W8vv
         d6vMgOoioRDorAr9oGJfLHlFvUQ+uBBGHPP2gN2U8mUAYdRKUDBIu4THQYIT/maX2xs0
         EWXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715680315; x=1716285115;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ewjtb4si4PNJIEUXGOGdyvrW80i/rihnaPh645nuA5A=;
        b=RI7yLgSsiQ6guLRrlw7bt94rna0XqM9tr5Xxx37mWAL4i34D1LfDbdP2DHDM8S/V/p
         gt0ChKeLLjDYDVJ33XwN4D+trQLqszVWug3hDF19AoDxTI/TUs1K+hsy5yt17vPHnsp3
         xah1z7xx7KLg5qNMwtATSylSkbipe6V+JqQEIZB+Y0MGgn808CCYolQfejodgt3Mxwq8
         AMzEdqlnOsXTst5z/IQR5W+BtsTHB68By+lXcr+rZwxJZR0cnKKpQYjDDnCh5d2MTMKX
         7KC1M0EysaIAaCSxN/9dKW1qJIqkjoFvJY4ew3KxBr28gL1zE2YtNEXZ5DKld0cqryHv
         2ANw==
X-Forwarded-Encrypted: i=1; AJvYcCUyWoYYpvyV8BBK4avBe7oFv6xfgb/YaOqV9DuYjZRUBTy6djmhtmZmNS3aj/WshYdy0MgsQzy0jX1l+LbGwqAR5GCNGVtLJqfGOrMECUo=
X-Gm-Message-State: AOJu0YxhZ99fwcOfZVI1a/PFsxTpZH+611v0OUlqE2Q3BpruJU7fC3TI
	AY/BjsVbcUoUr0YmuCtUopRoGSwVOLLgxw6uvE1qdo+2t8BkQoEIM/wxZ0GLfg==
X-Google-Smtp-Source: AGHT+IEYe4/z6psHPdfjGnQteiVs+VasqzVpkxs/Xd0X+zw1euWpWGPBujsbTpwPTjWdrDoLubBpCg==
X-Received: by 2002:ac2:5495:0:b0:51d:9f10:71b7 with SMTP id 2adb3069b0e04-5220fd7ce6dmr8559259e87.28.1715680314429;
        Tue, 14 May 2024 02:51:54 -0700 (PDT)
Message-ID: <bd2a7cd3-4109-404e-9a15-30bb152bf878@suse.com>
Date: Tue, 14 May 2024 11:52:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 09/27] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-10-eliasely@amazon.com>
 <7031861b-f925-479c-b236-8b1809996ede@suse.com>
 <e465d95c-f1c8-4857-9bb5-1c37a3c4cc80@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e465d95c-f1c8-4857-9bb5-1c37a3c4cc80@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.05.2024 17:21, Elias El Yandouzi wrote:
>  > On 20/02/2024 10:28, Jan Beulich wrote:
>>> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>>> --- a/xen/arch/x86/pv/dom0_build.c
>>> +++ b/xen/arch/x86/pv/dom0_build.c
>>> @@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
>>>       l3_pgentry_t *l3tab = NULL, *l3start = NULL;
>>>       l2_pgentry_t *l2tab = NULL, *l2start = NULL;
>>>       l1_pgentry_t *l1tab = NULL, *l1start = NULL;
>>> +    mfn_t l4start_mfn = INVALID_MFN;
>>> +    mfn_t l3start_mfn = INVALID_MFN;
>>> +    mfn_t l2start_mfn = INVALID_MFN;
>>> +    mfn_t l1start_mfn = INVALID_MFN;
>>
>> The reason initializers are needed here is, aiui, the overly large scope
>> of these variables. For example ...
> 
> Correct, is it just an observation or do you want me to do anything?

Where possible reducing the scope of variables would be preferred. Hence
why I ...

>>> @@ -708,22 +712,32 @@ int __init dom0_construct_pv(struct domain *d,
>>>           v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
>>>       }
>>>   
>>> +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
>>> +do {                                                    \
>>> +    unmap_domain_page(virt_var);                        \
>>> +    mfn_var = maddr_to_mfn(maddr);                      \
>>> +    maddr += PAGE_SIZE;                                 \
>>> +    virt_var = map_domain_page(mfn_var);                \
>>> +} while ( false )
>>> +
>>>       if ( !compat )
>>>       {
>>>           maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
>>> -        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
>>> +        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
>>> +        l4tab = l4start;
>>>           clear_page(l4tab);
>>> -        init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)),
>>> -                          d, INVALID_MFN, true);
>>> -        v->arch.guest_table = pagetable_from_paddr(__pa(l4start));
>>> +        init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true);
>>> +        v->arch.guest_table = pagetable_from_mfn(l4start_mfn);
>>
>> ... looks to be required only here, while ...
>>
>>>       }
>>>       else
>>>       {
>>>           /* Monitor table already created by switch_compat(). */
>>> -        l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table));
>>> +        l4start_mfn = pagetable_get_mfn(v->arch.guest_table);
>>> +        l4start = l4tab = map_domain_page(l4start_mfn);
>>
>> ... in principle the use of the variable could be avoided here. Below
>> from here there's no further use of it.

... went into some detail towards that possibility.

>>> @@ -781,30 +797,34 @@ int __init dom0_construct_pv(struct domain *d,
>>>   
>>>       if ( compat )
>>>       {
>>> -        l2_pgentry_t *l2t;
>>> -
>>>           /* Ensure the first four L3 entries are all populated. */
>>>           for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
>>>           {
>>>               if ( !l3e_get_intpte(*l3tab) )
>>>               {
>>>                   maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
>>> -                l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
>>> -                clear_page(l2tab);
>>> -                *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT);
>>> +                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
>>> +                clear_page(l2start);
>>> +                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
>>>               }
>>
>> The updating of l2start is only conditional here, yet ...
>>
>>>               if ( i == 3 )
>>>                   l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
>>>           }
>>>   
>>> -        l2t = map_l2t_from_l3e(l3start[3]);
>>> -        init_xen_pae_l2_slots(l2t, d);
>>> -        unmap_domain_page(l2t);
>>> +        init_xen_pae_l2_slots(l2start, d);
>>
>> ... here you assume it points at the page referenced by the 3rd L3 entry.
> 
> Hmm, I missed it when sending the revision and indeed it doesn't look 
> correct.
> 
>> Question is why the original code is being replaced here in the first
>> place: It was already suitably mapping the page in question.
> 
> The code was already suitably mapping the pages in question. This patch 
> doesn't aim to make any functional change, just to rework how the 
> domheap pages are used. The goal of the series is to remove the mappings 
> from the directmap, which means those pages needs to be mapped and 
> unmapped when required.

But that doesn't address my question: If there's nothing wrong with the
earlier code, why does it need changing (right here)?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 09:53:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 09:53:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721350.1124721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oqk-0006mG-4M; Tue, 14 May 2024 09:53:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721350.1124721; Tue, 14 May 2024 09:53:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6oqk-0006m9-1X; Tue, 14 May 2024 09:53:34 +0000
Received: by outflank-mailman (input) for mailman id 721350;
 Tue, 14 May 2024 09:53:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6oqi-0006lv-D1
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 09:53:32 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d217edb4-11d7-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 11:53:31 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59c5c9c6aeso1325748166b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 02:53:31 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d92sm697859066b.85.2024.05.14.02.53.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 02:53:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d217edb4-11d7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715680411; x=1716285211; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WcSjuuUmwI/uTFm3SLnWKhOwvy97ImVZk9TT5SOLH2E=;
        b=BJBr9uegqn+PFMu4OBHB12grV/4RkHmyqPaK7M9akJxEJQd3KxmhO78kUZXcPLwHEf
         lM7cuP8XblcU6tk9YCn6c4Z1yM6HglhMQAilvXon29wRb/lxcTHFH+CxlW3O9Qfsbcfi
         5Z+smRQ7L/CG9fhibzG7iehIXo2nmTCAh95w8FYoPyHOJYvCbJswGm+B0j76rMBTBOBw
         8FMncNIGPmEZM3+OXJLHXN1KVE8b/9ZAvH09iLC7WzUaz5W+Mhs5l+ut08R8YxUZ+PGU
         vmGTWvNJSFJf8+siJJ8Iop2hjRvfZh3IwaR+J8HPHoEg8pQIfZY02Idc933jiMxr7K4A
         4Ddw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715680411; x=1716285211;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WcSjuuUmwI/uTFm3SLnWKhOwvy97ImVZk9TT5SOLH2E=;
        b=fWMaF3V1aS4jBkUGeX9jxDyNBBP7DKEjtCoC7/dydYA5hy3yvknM0eskPRRhiiVKhu
         ZQsUDgVmpuKK1uBBKEL1i1TvBZkvOP2L8gEP9K5lymEcM6ZlUqYZop6tjzpq3R9Cocv3
         xVSjLGehRj4yMlAicpL1mvFPoT2v/WLOoIZeLbjzeDArRo8zaj70qumpafCwtJiYS/YC
         vpH0o5GgWw68nIl9x+gHU2CrZttLC2eaptdNllIdjUXamsQFIfywN4nPZFW4ZnJi1gmo
         eQAaTTnT3aCyCrko3Zd9EQ5a2uH+wg4ts6AKHHxeyYuGTbyMlxsrV/jLixbnYXXqaz3X
         cAGw==
X-Forwarded-Encrypted: i=1; AJvYcCWubdCjqI8LG9nmk5GZSvSkpdEMHApTkVwJ7Z/4HEsPs+Tv6iT4iI+YiHciLJVFHTZEopl/sAEfSzY8k8t5jr5siQEg5e6KUuFX6XB4bXI=
X-Gm-Message-State: AOJu0Yxu7nKlaigYSHcHb5JAuTLH1wAuJS4N+peTEMkUwSUXSMxcSkjY
	iGgmYu8GWB1u9Na80dLbKm7dbv2lNbrOu0FgsytgbKu0YiQqobq9B3J96v9Ivw==
X-Google-Smtp-Source: AGHT+IHzEzFaT9851PgGPMxhJgxyFOzpd3SeGYOtjsx9PratNWuoD5LN44iXXHwoaF+PYCr7X1u6Zg==
X-Received: by 2002:a17:907:1b17:b0:a59:aa69:9791 with SMTP id a640c23a62f3a-a5a2d5eeb92mr1208196066b.34.1715680410801;
        Tue, 14 May 2024 02:53:30 -0700 (PDT)
Message-ID: <054dfde5-cfaa-470b-9d40-c0453bc4dbbb@suse.com>
Date: Tue, 14 May 2024 11:53:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 11/27] x86: Lift mapcache variable to the arch
 level
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Wang <wawei@amazon.de>, Hongyan Xia <hongyxia@amazon.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-12-eliasely@amazon.com>
 <1ff19cb2-851d-40cf-bb12-5b7074eee0b8@suse.com>
 <06e162de-55d6-4612-aa45-b1a350571d2b@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <06e162de-55d6-4612-aa45-b1a350571d2b@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.05.2024 17:22, Elias El Yandouzi wrote:
>>> This only lifts the mapcache variable up. Whether we populate the
>>> mapcache for a domain is unchanged in this patch.
>>
>> Is it? I wonder because of ...
>>
> 
> I agree, the commit message doesn't completely reflect the changes below.
> 
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -843,6 +843,8 @@ int arch_domain_create(struct domain *d,
>>>   
>>>       psr_domain_init(d);
>>>   
>>> +    mapcache_domain_init(d);
>>> +
>>>       if ( is_hvm_domain(d) )
>>>       {
>>>           if ( (rc = hvm_domain_initialise(d, config)) != 0 )
>>> @@ -850,8 +852,6 @@ int arch_domain_create(struct domain *d,
>>>       }
>>>       else if ( is_pv_domain(d) )
>>>       {
>>> -        mapcache_domain_init(d);
>>> -
>>>           if ( (rc = pv_domain_initialise(d)) != 0 )
>>>               goto fail;
>>>       }
>>
>> ... this and ...
>>
>>> --- a/xen/arch/x86/domain_page.c
>>> +++ b/xen/arch/x86/domain_page.c
>>> @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
>>>   #endif
>>>   
>>>       v = mapcache_current_vcpu();
>>> -    if ( !v || !is_pv_vcpu(v) )
>>> +    if ( !v )
>>>           return mfn_to_virt(mfn_x(mfn));
>>
>> ... this and yet more changes indicating otherwise.
>>
>> Yet if which domains have a mapcache set up is already changed here, I
>> wonder whether the idle domain shouldn't be taken care of here as well.
> 
> Do you suggest to fold here the following patch where the mapcache gets 
> initialized for idle domains?

Or the respective part(s) thereof.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721357.1124731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6p0f-0002CP-1i; Tue, 14 May 2024 10:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721357.1124731; Tue, 14 May 2024 10:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6p0e-0002CI-VJ; Tue, 14 May 2024 10:03:48 +0000
Received: by outflank-mailman (input) for mailman id 721357;
 Tue, 14 May 2024 10:03:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6p0e-0002CC-6o
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:03:48 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 40fc30d8-11d9-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 12:03:46 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so535506766b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:03:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7da8sm707505366b.99.2024.05.14.03.03.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 03:03:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40fc30d8-11d9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715681026; x=1716285826; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rdSZthp74TLUjLS/d377/SBlO1xL1eefpvnkDZAQNnQ=;
        b=X0IvX7GLpuTMJAa21Jnut6R0+rabPfLquGDvfvlZmH/cWNky5KsLTC58NU2GgcLo79
         Sb4qGjwbOe+4JNYg3vWIt7MzHuyNmv+xbdvhvJB4ZAK9HL0hEp0Tkf2y5BC6KZdev4cC
         T4upeB5un/bmHpyqCJ9Gg1CuGGmwN9Zl8W8PCE0wQjiKeo7J+HUcZFSvEGiYTLWHSEV/
         vOp2+npXfzZo5yKuOMHq1YV8N1INUsr1GWIW0QMuxbxF/hVeyQkgKpX0iznFHEAeS7Ba
         5qOKvzny4JQ2PawxwRkEuaUChmGwo7sJhtIKN8UgoaBylDPqAxlaObIqT9L90QdwRWGg
         3N6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715681026; x=1716285826;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rdSZthp74TLUjLS/d377/SBlO1xL1eefpvnkDZAQNnQ=;
        b=ORz8s1VVzNzPo8ZU5zuaw//qIslxmeIzScyoXTQOp3xhcyZhuG2qcViKCNqB9wkuBg
         H8YjEZGbcgr2no9h1AHYKtbArBSIaJoXd4yd9WKxqscwjK9h6pmUjHrKakvfDR27fAXQ
         lWXcCvZYPZzCKZ17l4w1XhquXQvgwrXnBkRTOdid5yG42ghT/2cUq87N3F/QY/Yr/Odx
         9imGgeCkJJKo+d4+zQWkflOJvs7widiECPOQ9y9mueWGWBqHeCUYutPu/w26FwQXO89K
         tCjQGR4mjO8VQNLOgfAjaQm6n3BARJOehIH0i3QPtn4SzidL+sUhro7W4cK35hfvxKzu
         P8kQ==
X-Forwarded-Encrypted: i=1; AJvYcCXqxuWwZhh2+KqfiW1IEFy29j2fgse4YuuzI5jQTknIKK2Ewd6KDySiN2weU/I9DgVjCMrEIt3W6anPAoSLyv//fzVIUJTScxofoQCCtOc=
X-Gm-Message-State: AOJu0Yy2ync8ecVyjLVM0WXp0aYwpoeubCciQCzEQpKSuIEibYVhox8d
	CS15cB9AyT+Qhj22f9oHm+ck+O/OJK6xaSFl1Mw7+zchupP7zs12aEVmefsuRQ==
X-Google-Smtp-Source: AGHT+IHhkPKcGHa7zpr+tjD5ufu50ISrCwlos9ftoIwiPdoh6zt/3CRtBbAC12aF8u6/mm1AISaNhA==
X-Received: by 2002:a17:906:194a:b0:a59:9b75:b90 with SMTP id a640c23a62f3a-a5a2d53ad53mr828755366b.2.1715681026293;
        Tue, 14 May 2024 03:03:46 -0700 (PDT)
Message-ID: <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
Date: Tue, 14 May 2024 12:03:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Daniel Smith <dpsmith@apertussolutions.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 11:51, Andrew Cooper wrote:
> On 14/05/2024 10:25 am, Jan Beulich wrote:
>> On 03.04.2024 08:16, Jan Beulich wrote:
>>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>>> The commit makes a claim without any kind of justification.
>>> Well, what does "have no business" leave open?
>>>
>>>> The claim is false, and the commit broke lsevtchn in dom0.
>>> Or alternatively lsevtchn was doing something that was never meant to work
>>> (from Xen's perspective).
>>>
>>>>  It is also quite
>>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>>> Why would that be "obvious"? What business would a stubdom have to look at
>>> Xen's side of an evtchn?
>>>
>>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>>> and it's not acceptable to short circuit the XSM on the matter.
>>> I can certainly accept this as one possible view point. As in so many cases
>>> I'm afraid I dislike you putting it as if it was the only possible one.
>>>
>>> In summary: The supposed justification you claim is missing in the original
>>> change is imo also missing here then: What business would any entity in the
>>> system have to look at Xen's side of an event channel? Back at the time, 3
>>> people agreed that it's "none".
>> You've never responded to this reply of mine, or its follow-up. You also
>> didn't chime in on the discussion Daniel and I were having. I consider my
>> objections unaddressed, and in fact I continue to consider the change to
>> be wrong. Therefore it was inappropriate for you to commit it; it needs
>> reverting asap. If you're not going to do so, I will.
> 
> You tried defending breaking a utility with "well it shouldn't exist then".
> 
> You don't have a leg to stand on, and two maintainers of relevant
> subsystems here just got tired of bullshit being presented in place of
> any credible argument for having done the change in the way you did.

Please can you finally get into the habit of not sending rude replies?

> The correct response was "Sorry I broke things.  Lets revert this for
> now to unbreak, and I'll see about reworking it to not intentionally
> subvert Xen's security mechanism".

I'm sorry, but I didn't break things. I made things more consistent with
the earlier change, as pointed out before: With your revert,
evtchn_status() is now (again) inconsistent with e.g. evtchn_send(). If
you were serious about this being something that needs leaving to XSM,
you'd have adjusted such further uses of consumer_is_xen() as well. But
you aren't. You're merely insisting on lsevtchn needing to continue to
work in a way it should never have worked, with a patch to improve the
situation already pending.

Just to state a very basic principle here again: Xen-internal event
channels ought to either be fully under XSM control when it comes to
domains attempting to access them (in whichever way), or they should
truly be Xen-internal, with access uniformly prevented. To me the
former option simply makes very little sense.

> As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
> had for the principle of the change based on the absurdity of your
> arguments.

No, pending objections are pending objections. Daniel's responses didn't
eliminate them.

As a separate aspect: I can't assume anymore that it is just coincidence
that you taking such a controversial action is at a time when I'm away.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:08:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:08:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721361.1124742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6p4d-0002n2-Gz; Tue, 14 May 2024 10:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721361.1124742; Tue, 14 May 2024 10:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6p4d-0002mv-EH; Tue, 14 May 2024 10:07:55 +0000
Received: by outflank-mailman (input) for mailman id 721361;
 Tue, 14 May 2024 10:07:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6p4c-0002mp-L0
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:07:54 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d4159e9c-11d9-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 12:07:53 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so571168666b.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:07:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea65c0sm7214879a12.10.2024.05.14.03.07.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 03:07:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4159e9c-11d9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715681273; x=1716286073; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ko3+Cad/ujOuVIZA7Ewbl9fV2GOO1z2XD5xu5aGCeXQ=;
        b=Yx1sBizinAZuU1QROeFO/S09iCOMy8l1bWytDrRhmDi7sOq+6Y2+oZio0BhtZn1GaT
         9MEC5Pn8w72qI6gUeUztVoSKOMdO3Uz0WT2xJiaTGmPfUy+tMaYc5TWWt9xo1Y/mBg/N
         iKLG/9Tfl3nzAMhlnVOTwgy+dVtWscXrFHpEvN3rEYUd6krXcHp+mTZv44K/VM17nlNc
         lSTWSNjwXu46NBQA4x7xln9F8TpF7j/P9J6oX5rm4v3B8k8UhCGgZMrKnaHj2Lq2j1QO
         niTBch0rJnBQRF1HwCy6mywx+R4nUycfKyyeZU11+787134rfL9BVdqEuE/s4WqD78OR
         CSCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715681273; x=1716286073;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ko3+Cad/ujOuVIZA7Ewbl9fV2GOO1z2XD5xu5aGCeXQ=;
        b=CpJIDWxGKbtGuqXtfz5BcGSUKTBEQSy5QH9XGiGzdgwMmkOWBTl/WGCFD5sA579H2p
         7SMVqGaer2Bu9ThevUdcsu3x6ygTo+8TYQaJkVtcREJxTaJgzxRxRNSIOMGOQ+folWfA
         UXtfo6RCrrnKbrCMRk7xWvP0flJkrVPjZbz957vv/rMLKBE8yTywVVajYD411r7OCOuv
         aNcC9iJdElK87Zb0JH0iYNmjRE7EjxRCtvQGEd4GPhQX8J0iN100HEPkbz5zKJ8DUonS
         dIW7NlHzdBlsVIUxLUtK2Y9QjILN1ji9ULhq2ZNwqI5DpILloVSdF9iIrYhCbwuXlSTq
         VafQ==
X-Forwarded-Encrypted: i=1; AJvYcCUPAAd3HonzvVJhD364OZr3fXI57kAygbCmwp7MWFXBfzFWRcyL4vV121s2T3AFZTPqw5oXTjSW//NePqgMPCJNdqrFcOUnRXM56JuoNgg=
X-Gm-Message-State: AOJu0YyZu2WdeBf8LvIzgTn9KosepXooGWI4Z/iiML5pDIyMveRCkinT
	DWnRs+IABLZiMybfn1e6XjrW6CGwu4fhhvcW7gAYSZ/i4rCJ6d64WSmPCwy5FA==
X-Google-Smtp-Source: AGHT+IEOOFZEoLHbriJz1HUEhj59g4q54KQ7u9ejgIDYv2ogwPLvq30o/IG7xH80RzX4FN9RDHD7/Q==
X-Received: by 2002:a50:8711:0:b0:572:7015:f303 with SMTP id 4fb4d7f45d1cf-5734d6f00d7mr10489266a12.35.1715681273176;
        Tue, 14 May 2024 03:07:53 -0700 (PDT)
Message-ID: <d84642c4-d578-4629-ac66-7461f5d143f1@suse.com>
Date: Tue, 14 May 2024 12:08:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 12/27] x86/mapcache: Initialise the mapcache
 for the idle domain
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Wei Wang <wawei@amazon.de>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-13-eliasely@amazon.com>
 <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
 <a74001e4-7ae1-48f7-854d-2a8aeb2ff8da@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a74001e4-7ae1-48f7-854d-2a8aeb2ff8da@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 11:35, Elias El Yandouzi wrote:
> On 20/02/2024 10:51, Jan Beulich wrote:
>> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -750,9 +750,16 @@ int arch_domain_create(struct domain *d,
>>>   
>>>       spin_lock_init(&d->arch.e820_lock);
>>>   
>>> +    if ( (rc = mapcache_domain_init(d)) != 0)
>>> +    {
>>> +        free_perdomain_mappings(d);
>>> +        return rc;
>>> +    }
>>> +
>>>       /* Minimal initialisation for the idle domain. */
>>>       if ( unlikely(is_idle_domain(d)) )
>>>       {
>>> +        struct page_info *pg = d->arch.perdomain_l3_pg;
>>>           static const struct arch_csw idle_csw = {
>>>               .from = paravirt_ctxt_switch_from,
>>>               .to   = paravirt_ctxt_switch_to,
>>> @@ -763,6 +770,9 @@ int arch_domain_create(struct domain *d,
>>>   
>>>           d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>>>   
>>> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
>>> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
>>> +
>>>           return 0;
>>>       }
>>
>> Why not add another call to mapcache_domain_init() right here, allowing
>> a more specific panic() to be invoked in case of failure (compared to
>> the BUG_ON() upon failure of creation of the idle domain as a whole)?
>> Then the other mapcache_domain_init() call doesn't need moving a 2nd
>> time in close succession.
> 
> To be honest, I don't really like the idea of having twice the same call 
> just for the benefit of having a panic() call in case of failure for the 
> idle domain.

Resulting in the problem Roger has now validly pointed out in reply to v3.
IOW the (more specific) panic() isn't the only reason; it would merely be
an imo desirable side effect.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:19:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:19:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721367.1124752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pFI-0005ci-JS; Tue, 14 May 2024 10:18:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721367.1124752; Tue, 14 May 2024 10:18:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pFI-0005cb-GO; Tue, 14 May 2024 10:18:56 +0000
Received: by outflank-mailman (input) for mailman id 721367;
 Tue, 14 May 2024 10:18:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6pFH-0005cV-Ni
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:18:55 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5de6c6b2-11db-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 12:18:54 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so538896666b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:18:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea65aasm7315199a12.5.2024.05.14.03.18.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 03:18:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5de6c6b2-11db-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715681934; x=1716286734; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Pt8j63seJHbsi0uIXJsUqTLZo+0aMi4S6sWK+XPqS28=;
        b=NQQhQ5LDEMLpMu9Xcq0qDOL6QEwxR3yhfJKCL9qgIl5RS2diBUQexMRE1UDhv848/8
         ZJOcurm3fQlZ3Cvnt+8qvg5S5Ni1lm5WmdTGiNt+2veR7hU79TCwvJjO7Q/gVQeTGBq6
         SY+5M26zexawBfRmztQoumVx0//qAUf+MmYKVvls74ZINvYEG14qCu1Z/+w3oGnN06RJ
         UzGyiY/afp5vMi8aUMTVcAihvZUfShpv6Ag0rFxAzO83+xge+pCnSGF7dHS9Ttij5kzE
         y7nm/G56eOCztJV7OGlIheP3NVkipQWdvIQEOw1wCDJrU+eRkSUUH25TczQ9/1L+yLnj
         VHXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715681934; x=1716286734;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Pt8j63seJHbsi0uIXJsUqTLZo+0aMi4S6sWK+XPqS28=;
        b=dyPK+wDFXmR7Q5YvOBZCuxpBoxRMS5/qAXhmaH8uq0h7Ab9ghCu6NycymzM5VJVQIn
         +CVi6IGERcwfqmPoelFVP8gcir9YPd6B25j7IIugvOYTtAHD0+xZ6vxIRIeS2SoQnBRC
         V6NZcRT4Rx3nFo2pmAw7DGMoBwv9BE8CIxRpjmg+1YuLdkgspoX/UIQJMBo/fcOx/RLd
         Lro1Ao9IaQ+987CZRHvMNtvoq4GDOQx9XPGhCKovpTS1vTIhY9vm5KhnX+HFhCj2U9I6
         FuIyxCIV3cSm6IviFxL0iGm4FVSJZZe7eg54rTE8Vo3Ab1aIdKHcaErgjOB3FcmyyDWi
         dicw==
X-Forwarded-Encrypted: i=1; AJvYcCWCkSmN3L2eQArY7vflu4caqIU5edheUMObR9dT64cc4Hl4rVLLIb2bGjHOisch5QxBKF0IpzfIrDlRogFh5rzcrgD2LKv22wTQ2/0jgGs=
X-Gm-Message-State: AOJu0Yyn0YMF7IVAQmNLSgDaBm7Aty9ib89zsq1SpbI95cZhXN4eHtU8
	Q67B6linIlgfVYGpRqGUSeOMRWJI9b54cJo24IZzIHK3fkkQCVt4m1KvUOaSkw==
X-Google-Smtp-Source: AGHT+IF44GkgoviWpQzA6w6+/Hf9pOtNXwBcFiyDX/h45zG3J6RjYjz4Y5nKaOe7vH2toMN5bBPBBQ==
X-Received: by 2002:a50:8e5d:0:b0:572:6ab0:6afc with SMTP id 4fb4d7f45d1cf-5734d6b1c30mr8276372a12.33.1715681933797;
        Tue, 14 May 2024 03:18:53 -0700 (PDT)
Message-ID: <71c96690-cd9a-4e1e-b2a0-7783dc13c35e@suse.com>
Date: Tue, 14 May 2024 12:19:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 13/27] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-14-eliasely@amazon.com>
 <18795bde-bfd5-41ab-bef5-f74819bae956@suse.com>
 <20fb9bf9-b24a-4119-807b-c0a4fd6b7439@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20fb9bf9-b24a-4119-807b-c0a4fd6b7439@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 12:50, Elias El Yandouzi wrote:
> On 20/02/2024 11:14, Jan Beulich wrote:
>> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>>> --- a/xen/arch/x86/Kconfig
>>> +++ b/xen/arch/x86/Kconfig
>>> @@ -29,6 +29,7 @@ config X86
>>>   	select HAS_UBSAN
>>>   	select HAS_VPCI if HVM
>>>   	select NEEDS_LIBELF
>>> +	select HAS_SECRET_HIDING
>>
>> Please respect alphabetic sorting. As to "secret hiding" - personally I
>> consider this too generic a term. This is about limiting the direct map. Why
>> not name the option then accordingly?
> 
> I think it is a fairly decent name, would you have any suggestion? 
> Otherwise I will just stick to it.

See how Roger, on v3, has now responded along the same lines? His naming
suggestion (with spelling adjusted) would be fine with me.

>>> +    eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
>>
>> Irrespective I don't see a need to replace the initializer by an assignment.
> 
> I guess it was to avoid the useless min() computation in case directmap 
> is disabled. I can put it back to what it was.

The compiler ought to be able to re-arrange code accordingly, if it thinks
the overall result will then be better.

>>> +config SECRET_HIDING
>>> +    bool "Secret hiding"
>>> +    depends on HAS_SECRET_HIDING
>>> +    ---help---
>>> +    The directmap contains mapping for most of the RAM which makes domain
>>> +    memory easily accessible. While making the performance better, it also makes
>>> +    the hypervisor more vulnerable to speculation attacks.
>>> +
>>> +    Enabling this feature will allow the user to decide whether the memory
>>> +    is always mapped at boot or mapped only on demand (see the command line
>>> +    option "directmap").
>>> +
>>> +    If unsure, say N.
>>
>> Also as an alternative did you consider making this new setting merely
>> control the default of opt_directmap? Otherwise the variable shouldn't exist
>> at all when the Kconfig option is off, but rather be #define-d to "true" in
>> that case.
> 
> I am not sure to understand why the option shouldn't exist at all when 
> Kconfig option is off.

I didn't say "option", but "variable", and ...

> If SECRET_HIDING option is off, then opt_directmap must be 
> unconditionally set to true. If SECRET_HIDING option is on, then 
> opt_directmap value depends on the commandline option.

... I did clearly say what I think you want to do, bringing things in line
with other opt_* that reduce to a constant when a certain CONFIG_* is not
defined.

> The corresponding wrapper, has_directmap(), will be used in multiple 
> location in follow-up patch. I don't really see how you want to do.

The wrapper is fine to have if, as per the earlier reply still visible in
context below, the variable itself can then be suitably static (and the
fallback #define local to that same C file). Otherwise I simply don't see
the value of the wrapper function.

>>> --- a/xen/include/xen/mm.h
>>> +++ b/xen/include/xen/mm.h
>>> @@ -165,6 +165,13 @@ extern unsigned long max_page;
>>>   extern unsigned long total_pages;
>>>   extern paddr_t mem_hotplug;
>>>   
>>> +extern bool opt_directmap;
>>> +
>>> +static inline bool has_directmap(void)
>>> +{
>>> +    return opt_directmap;
>>> +}
>>
>> If opt_directmap isn't static, I see little point in having such a wrapper.
>> If there are reasons, I think they want stating in the description.
> 
> I don't think there is a specific reason to be mentioned, if you really 
> wish to, I can remove it.
> 
>> On the whole: Is the placement of this patch in the series an indication
>> that as of here all directmap uses have gone away? If so, what's the rest of
>> the series about? Alternatively isn't use of this option still problematic
>> at this point of the series? Whichever way it is - this wants clarifying in
>> the description.
> 
> This patch is not an indication that all directmap uses have been 
> removed. We need to know in follow-up patch whether or not the option is 
> enabled and so we have to introduce this patch here.

There's a pretty clear indication: "directmap=off" means "no directmap".
It does not mean "a little less of direct mapping". Aiui that won't even
change by the end of the series. It's only the ratio which is going to
change.

> At this point in the series, the feature is not yet complete.

Right, and again - see how Roger, on v3, has now replied along the same
line.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:20:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:20:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721370.1124762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pGZ-00073A-Sh; Tue, 14 May 2024 10:20:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721370.1124762; Tue, 14 May 2024 10:20:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pGZ-000733-Pi; Tue, 14 May 2024 10:20:15 +0000
Received: by outflank-mailman (input) for mailman id 721370;
 Tue, 14 May 2024 10:20:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6pGY-00072t-NQ
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:20:14 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d41cdeb-11db-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 12:20:13 +0200 (CEST)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-34d7a32bdd3so3131395f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:20:13 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502baad07fsm13330820f8f.88.2024.05.14.03.20.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 03:20:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d41cdeb-11db-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715682013; x=1716286813; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=/NKhc87XdzYB48q5cDIUD8gUxz3tFXu5pueGeLBUV/Y=;
        b=jx63S3kA63ULNMfu/JcLyRNsXTr+Qb0Uz4G0JViHqcj45XFDevHXEeTo/lYjAJF2Bc
         8v8fuLLXyxc9xPfs66GOCSIY9THRocruJfrBmIHJxQ7RzRkY2/ezGhNGGxkM1vLeQMwx
         /HURYRmspk/n5zqnzpfez+pdoN/w8CA4cNzzc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715682013; x=1716286813;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/NKhc87XdzYB48q5cDIUD8gUxz3tFXu5pueGeLBUV/Y=;
        b=VoDz3QAwAn1x6KW13w5yMNq02S+uktTvEx2Foiltf8nKg7JQJqdmOh0pHJ9m/B3xDR
         DLgmrSSoHsryTbFZS1STpYKKUOv2djfAXatdLXVFPJRNtgv3a3IJYFCZdcKrNse/hIxh
         b02RRoJ38jd0TAAbRxWYkV2AkQxxvyeSr8/uOLQ6S1Z+CxktvKBiW62JKGM0BT+y2pG3
         3Ym9OPzSwjNZKorQbfm6pGz44Yr6aoMqJamzNQ+rjTa7lYU64zWVoozGj2C/e6Y9vk3B
         5Kjc6FcrPXk03A95SoEUVl2oAX+soHyLKud6m9LGhp01LAqKjIgRQo9OPHg139R6d4H+
         43GA==
X-Gm-Message-State: AOJu0YxyKTFrb6emkmcGMQUKHycq5K5C5yW66jembyL4UC0TEiLycm0C
	12B0swSLqR+YV6MimC/bOi3escMk00QykVLlfw9kZmIbdBDbacvPXbhjpi3rzHE=
X-Google-Smtp-Source: AGHT+IHtXb1Tx4pH/ctR3HLfMqrbHv/s/y7XKVKkMajmtWe1i2O9xjKJfDpnu0ezfz35tNljcKRJ+Q==
X-Received: by 2002:adf:c58a:0:b0:34a:56e7:5cc4 with SMTP id ffacd0b85a97d-351bfd2aa96mr4199767f8f.2.1715682013286;
        Tue, 14 May 2024 03:20:13 -0700 (PDT)
Date: Tue, 14 May 2024 12:20:11 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Message-ID: <ZkM625feJ8mMaM_p@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com>
 <ZkMs1cknBFBWZoJG@macbook>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZkMs1cknBFBWZoJG@macbook>

On Tue, May 14, 2024 at 11:20:21AM +0200, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:33PM +0000, Elias El Yandouzi wrote:
> > From: Hongyan Xia <hongyxia@amazon.com>
> > diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> > index 7561297a75..9d4f1f2d0d 100644
> > --- a/xen/include/xen/mm.h
> > +++ b/xen/include/xen/mm.h
> > @@ -167,6 +167,13 @@ extern unsigned long max_page;
> >  extern unsigned long total_pages;
> >  extern paddr_t mem_hotplug;
> >  
> > +extern bool opt_directmap;
> > +
> > +static inline bool has_directmap(void)
> > +{
> > +    return opt_directmap;
> 
> This likely wants:
> 
> return IS_ENABLED(CONFIG_HAS_SECRET_HIDING) && opt_directmap;

Er, sorry, this is wrong, should be:

return !IS_ENABLED(CONFIG_HAS_SECRET_HIDING) || opt_directmap;

Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:22:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:22:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721373.1124771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pIL-0007de-7c; Tue, 14 May 2024 10:22:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721373.1124771; Tue, 14 May 2024 10:22:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pIL-0007dX-4y; Tue, 14 May 2024 10:22:05 +0000
Received: by outflank-mailman (input) for mailman id 721373;
 Tue, 14 May 2024 10:22:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6pIK-0007cf-0z
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:22:04 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdc00472-11db-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 12:22:02 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-34f7d8bfaa0so3825610f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:22:02 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8a7907sm13238777f8f.63.2024.05.14.03.22.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 03:22:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdc00472-11db-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715682121; x=1716286921; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=9aEqg3pai1C+QnmBjJjdlQhpYcDZfyz4ua4D3W/WOTU=;
        b=O7sUvMNIoQkGHVwor8aHmQF279nl0r6sKY1I9n+ekpbZoO8VD2AUQNRLa2p4Pqw0Uz
         5wsFFde7DxrFEHXRFjask9wMm9UK+8CU1l8AFmPhmMwcZe8T+uOdTDLFwG+4i7m58hRa
         C506vbFR8bpVqbz/ICEj5DwAgxZj2VQ1ZhfNs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715682121; x=1716286921;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9aEqg3pai1C+QnmBjJjdlQhpYcDZfyz4ua4D3W/WOTU=;
        b=PV8ki54QXiNNWm7yoB/g3OmuxYiEQetMmjKyPinW5Zmx4hXkUMsJBZnrCZq52CJ7Me
         cCWYpcWzxPUnUfR4vZJACVOeGx+pWQWWGkP0tPtf57Ipw0fMxfLDhIjg74O3Li6BkWrx
         xpERIldD+jXJ6mjyoVhDIIm7TFvTAbCcG9vmQcNSXoqf9LLUK1abNl0P0ttlYpfhp+dT
         bSKho2g37E/uqfCYE8pD4AEwR7XiKUsNIwiRBX8vggFHlER2FBbqtmfJwT2DyveRwnoI
         ZstEr2GWNtKgvwnCKbCU2D+IGtfTmYnlP5eWmb1j/heaWPHNHjcEZ/0IW+O73Dg46wFT
         QZQg==
X-Gm-Message-State: AOJu0Yw8MWLg4VhNbIQLv3OMF6hhXtnU0kFeOpaGNoWNdoT7Wis93n9T
	qBwpsCQLRG4Q2NY4XgEMYelIvZAO/JxMxVkIhxdhQGRl0ob7UdCE7NMrbGGUfgM=
X-Google-Smtp-Source: AGHT+IFnCiHBLTKR5f4oq97PkKZYHF/SRzm/zZi1Wq8WEO1cgwZjJoaSBOxz8DBdNbtJo9d98hggKw==
X-Received: by 2002:a5d:550b:0:b0:34c:67d6:8dec with SMTP id ffacd0b85a97d-3504a62fec5mr9690654f8f.6.1715682121494;
        Tue, 14 May 2024 03:22:01 -0700 (PDT)
Date: Tue, 14 May 2024 12:22:00 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Elias El Yandouzi <eliasely@amazon.com>
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Message-ID: <ZkM7SK8U1Qsk7E3x@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com>
 <ZkMxgDPYgaKnmRRE@macbook>
 <87950da7-f765-4656-808f-6c63a29ced3d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <87950da7-f765-4656-808f-6c63a29ced3d@suse.com>

On Tue, May 14, 2024 at 11:43:14AM +0200, Jan Beulich wrote:
> On 14.05.2024 11:40, Roger Pau Monné wrote:
> > On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
> >> @@ -53,6 +55,8 @@ enum fixed_addresses {
> >>      FIX_PV_CONSOLE,
> >>      FIX_XEN_SHARED_INFO,
> >>  #endif /* CONFIG_XEN_GUEST */
> >> +    FIX_PMAP_BEGIN,
> >> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
> > 
> > This would better have
> > 
> > #ifdef CONFIG_HAS_PMAP
> > 
> > guards?
> 
> That's useful only when the option can actually be off in certain
> configurations, isn't it?

My comment earlier on this patch suggested to make CONFIG_HAS_PMAP be
selected by HAS_SECRET_HIDING, rather than being unconditionally
arch-selected (if that's possible, I certainly don't know the usage in
further patches).

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 10:26:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 10:26:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721377.1124781 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pMW-0000E4-Mw; Tue, 14 May 2024 10:26:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721377.1124781; Tue, 14 May 2024 10:26:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6pMW-0000Dx-KP; Tue, 14 May 2024 10:26:24 +0000
Received: by outflank-mailman (input) for mailman id 721377;
 Tue, 14 May 2024 10:26:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6pMV-00009b-Kl
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 10:26:23 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 694f7a52-11dc-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 12:26:23 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a8cd78701so17819666b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 03:26:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1789247csm700655266b.82.2024.05.14.03.26.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 03:26:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 694f7a52-11dc-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715682382; x=1716287182; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aKwppwpQhXQlhzsC4CfsCstbXtSO3y4fl5GkEUJUHrw=;
        b=LQyBzNzCVrRnSQUAMbvA9IytbqudaO8WQsYM1KswRSjeboUT9JKQ6K398QjyGpcmAj
         xbW7c2OnhpMFS3lDFE8qMbwZYDEjovUE+ZgQsq8z2HzE9z6CJ9Z1l3fIOPltNAHUOYxs
         6HJ3Y8Q0EZvExNfr1vfe955Mh0ikvsUKXBzvMjABvhxQ5xs6kL5kvAbfgxbkRLkrGOEg
         1RlKN7duAIAEMD1cDmor2/i9qgWqXfvhLD7YfxQh4UZDVf62cYIm8UEf5QLPq4uK2Tfp
         DsJgIpXWUkO7zLwSYXzy6Vxv8K5SEXmvfNELRl7viHTtwx9Z23qcm21YsUVLwpqQ91RC
         6tQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715682382; x=1716287182;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aKwppwpQhXQlhzsC4CfsCstbXtSO3y4fl5GkEUJUHrw=;
        b=FZ1k2+Gh4Txm6ExycfBWAoS72C2bvwJZpTv3fdZdzQIw9OFeDnAqXk91HvhxhgJ+1K
         J+gsoe57FBIDXNLZ32AQXoQYxQ/UUB9Flxnlfx3XUiGPGJyMBKkWJruMXD+OX7Y0Xnhx
         ItREzPUxV6vSUsGXDSDZaHzyGGlvz5nME3n3m8gVhFIO3MyyfCu9sPdSs2NNf2sqAQnL
         qb6EuPfoK8i9FHRTQ1q6Jl6R+YsdoblmJ7U9K6G3zktLjn1/4IezG7NKT0Gx3gFIWgi/
         Z3GplehdVZ1mwHD+4ThlrQl6p97nqTDg2tn8x5Rstq69Xv+IPBV/8bKVwTT39mp/xsde
         oTcg==
X-Gm-Message-State: AOJu0Yxtp6r5LKlaj3eHDSlQbcnUwvq/UlRapuKm2UQ3dZ+e96/QFpMj
	Os3iEJeGdPVNCilGjI7ShHkXcq6zQuYOCJttcWUAXJ0EOsxL71Z0LfqqfpYoOw==
X-Google-Smtp-Source: AGHT+IHDei9sT8ODxGLgdAW3fn4XilvkPTnu169M2ZGZbOvawLeSptZ2ZSIGkNpgUaPQgxGmueiDhg==
X-Received: by 2002:a17:906:7853:b0:a59:cdc9:6fd9 with SMTP id a640c23a62f3a-a5a2d5750acmr714300366b.21.1715682382483;
        Tue, 14 May 2024 03:26:22 -0700 (PDT)
Message-ID: <90621115-e370-49a1-b272-d20f861d6137@suse.com>
Date: Tue, 14 May 2024 12:26:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com> <ZkMxgDPYgaKnmRRE@macbook>
 <87950da7-f765-4656-808f-6c63a29ced3d@suse.com> <ZkM7SK8U1Qsk7E3x@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkM7SK8U1Qsk7E3x@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 12:22, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 11:43:14AM +0200, Jan Beulich wrote:
>> On 14.05.2024 11:40, Roger Pau Monné wrote:
>>> On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
>>>> @@ -53,6 +55,8 @@ enum fixed_addresses {
>>>>      FIX_PV_CONSOLE,
>>>>      FIX_XEN_SHARED_INFO,
>>>>  #endif /* CONFIG_XEN_GUEST */
>>>> +    FIX_PMAP_BEGIN,
>>>> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
>>>
>>> This would better have
>>>
>>> #ifdef CONFIG_HAS_PMAP
>>>
>>> guards?
>>
>> That's useful only when the option can actually be off in certain
>> configurations, isn't it?
> 
> My comment earlier on this patch suggested to make CONFIG_HAS_PMAP be
> selected by HAS_SECRET_HIDING, rather than being unconditionally
> arch-selected (if that's possible, I certainly don't know the usage in
> further patches).

Right, but in patch 6 HAS_SECRET_HIDING is selected unconditionally,
which would then also select HAS_PMAP. If, otoh, HAS_PMAP was selected
only when SECRET_HIDING (or whatever its name is going to be), then an
#ifdef would indeed be wanted here.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:09:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:09:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721407.1124792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6q2Q-0001Bw-T6; Tue, 14 May 2024 11:09:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721407.1124792; Tue, 14 May 2024 11:09:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6q2Q-0001Bp-PN; Tue, 14 May 2024 11:09:42 +0000
Received: by outflank-mailman (input) for mailman id 721407;
 Tue, 14 May 2024 11:09:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6q2Q-0001AG-3t
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:09:42 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73cbc523-11e2-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 13:09:37 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-51f0f6b613dso6798065e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 04:09:37 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccbe8f9bsm192197055e9.6.2024.05.14.04.09.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 04:09:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73cbc523-11e2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715684977; x=1716289777; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=t6e28PtWjPyOR6vJsno5O8bavbhaT20wT5u1Y1bDg/s=;
        b=ulkdQ2ERkYX6SHdLJ0682Nc2upQzvgIg76b5PSSXTpp0r6msBRiYtcW8Kp4imR1DFs
         FoMiy377ffXIn3z3raJSWX5coh142A94/qE784wm6dARQLut+VgCtsfv+beyoUrOcKnC
         tlak9VhzaZa7rmuL+8b38ydr9xnfvipnZsn+M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715684977; x=1716289777;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=t6e28PtWjPyOR6vJsno5O8bavbhaT20wT5u1Y1bDg/s=;
        b=KJxyCpWzaBZFUpCKlg2t5Z8cdjfb145f6qcJk4NCN0+Yh92gcN+GmYGssgZDFoOma/
         TTJx9Omcgvdu5oMFmlrykPbGRrUBqlTQmGFUTcLJO6rwF7SkRsTuDEiGba05I7qn60zK
         zM1itpm8GTgX5+xvQ2WbKe5FmKBpMIgv5nU6SZjXVZQnX4m/4yRxa/n++k4NtbCKi2NF
         UmYPb0MmRK2xIEm+A7AIPxdSKRBcIkduAymklAmqFzBRCPMwbFBdcb3hFH6mEGUqqLyM
         zSNQf96B2gOuCVPsvtU3MXNXE7F+joAjFwMOVri46lIXvi4k3gIjcQLIyI3mxf+JMH/f
         846A==
X-Forwarded-Encrypted: i=1; AJvYcCVK8otadumHDmceWiAIb3WQ54PZiDNLACBIASAOlsxE2MbPv90drZ5Ax+I69xU45Z6OFPHYIJ/NV3DtPlp8oydsrUkiAq5XagrT6YZsePs=
X-Gm-Message-State: AOJu0Yy711qQaTy5/HlUnpBRVtaTzyFYKa1FMcDrgi9YKLwP60FRRhEb
	AnnGTJYtg0nHrGhwqsujg84nBRJFOn4jO1wGqQ19jvOjAOONAlD5iF8GUK0JO0k=
X-Google-Smtp-Source: AGHT+IGmj9eLRnXuV9wapF6LmORCL/C4rRF6omSHYQQhIO6rdl0+ZodwRSrtb5Pk9L4/8+JfrEmcCA==
X-Received: by 2002:a05:6512:3da2:b0:522:2fa0:c3f5 with SMTP id 2adb3069b0e04-5222fa0cd89mr8799894e87.62.1715684976947;
        Tue, 14 May 2024 04:09:36 -0700 (PDT)
Message-ID: <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
Date: Tue, 14 May 2024 12:09:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240513085925.59324-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240513085925.59324-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13/05/2024 9:59 am, Roger Pau Monne wrote:
> There's no point in forcing a system wide update of the MTRRs on all processors
> when there are no changes to be propagated.  On AP startup it's only the AP
> that needs to write the system wide MTRR values in order to match the rest of
> the already online CPUs.
>
> We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
> on all the CPUs in the system.
>
> While there adjust the comment to clarify why the system-wide resetting of the
> MTRR registers is not needed for the purposes of mtrr_ap_init().
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
> triggering, but it's also a good performance improvement when performing
> cpu-online.
>
> Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
> the chance of the watchdog triggering due to the system-wide resetting of the
> range.

"Runtime" changes will only be during dom0 boot, if at all, but yes - it
is restricted to a single MTRR at a time.

It's XENPF_{add,del,read}_memtype, but it's only used by Classic Linux. 
PVOps only issues read_memtype.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:13:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:13:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721417.1124801 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6q6P-0002iF-GW; Tue, 14 May 2024 11:13:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721417.1124801; Tue, 14 May 2024 11:13:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6q6P-0002i8-Dn; Tue, 14 May 2024 11:13:49 +0000
Received: by outflank-mailman (input) for mailman id 721417;
 Tue, 14 May 2024 11:13:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6q6O-0002i2-0F
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:13:48 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6q6J-0004mK-Pq; Tue, 14 May 2024 11:13:43 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6q6J-0006tE-IY; Tue, 14 May 2024 11:13:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EWf7BbrwtV4179T2sSAukepkdAkoyiT8JYIsAXpZ4KU=; b=dWXULGt0GqPf1yim0182OPO0Ni
	l6D86dNyrxYC4LTAo+d+eLAVnYJBGbpb4ZUkUfG1SZW6OOjLs6+neHjzc2sx1TDFEv4r+qsgZFF2M
	30T01+mafD+aEkz4fWG6yx8zE1W2hXAUZj4cDD6QVk02/Pe/nUzg94JjcB4hPdGGAkss=;
Message-ID: <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
Date: Tue, 14 May 2024 12:13:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Daniel Smith <dpsmith@apertussolutions.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
 <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

(+ Oleksii as the release manager)

Chiming into the discussion as there seems there is disagreement.

On 14/05/2024 11:03, Jan Beulich wrote:
> On 14.05.2024 11:51, Andrew Cooper wrote:
>> On 14/05/2024 10:25 am, Jan Beulich wrote:
>>> On 03.04.2024 08:16, Jan Beulich wrote:
>>>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>>>> The commit makes a claim without any kind of justification.
>>>> Well, what does "have no business" leave open?
>>>>
>>>>> The claim is false, and the commit broke lsevtchn in dom0.
>>>> Or alternatively lsevtchn was doing something that was never meant to work
>>>> (from Xen's perspective).
>>>>
>>>>>   It is also quite
>>>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>>>> Why would that be "obvious"? What business would a stubdom have to look at
>>>> Xen's side of an evtchn?
>>>>
>>>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>>>> and it's not acceptable to short circuit the XSM on the matter.
>>>> I can certainly accept this as one possible view point. As in so many cases
>>>> I'm afraid I dislike you putting it as if it was the only possible one.
>>>>
>>>> In summary: The supposed justification you claim is missing in the original
>>>> change is imo also missing here then: What business would any entity in the
>>>> system have to look at Xen's side of an event channel? Back at the time, 3
>>>> people agreed that it's "none".
>>> You've never responded to this reply of mine, or its follow-up. You also
>>> didn't chime in on the discussion Daniel and I were having. I consider my
>>> objections unaddressed, and in fact I continue to consider the change to
>>> be wrong. Therefore it was inappropriate for you to commit it; it needs
>>> reverting asap. If you're not going to do so, I will.
>>
>> You tried defending breaking a utility with "well it shouldn't exist then".
>>
>> You don't have a leg to stand on, and two maintainers of relevant
>> subsystems here just got tired of bullshit being presented in place of
>> any credible argument for having done the change in the way you did.
> 
> Please can you finally get into the habit of not sending rude replies?
> 
>> The correct response was "Sorry I broke things.  Lets revert this for
>> now to unbreak, and I'll see about reworking it to not intentionally
>> subvert Xen's security mechanism".
> 
> I'm sorry, but I didn't break things. I made things more consistent with
> the earlier change, as pointed out before: With your revert,
> evtchn_status() is now (again) inconsistent with e.g. evtchn_send(). If
> you were serious about this being something that needs leaving to XSM,
> you'd have adjusted such further uses of consumer_is_xen() as well. But
> you aren't. You're merely insisting on lsevtchn needing to continue to
> work in a way it should never have worked, with a patch to improve the
> situation already pending.
> 
> Just to state a very basic principle here again: Xen-internal event
> channels ought to either be fully under XSM control when it comes to
> domains attempting to access them (in whichever way), or they should
> truly be Xen-internal, with access uniformly prevented. To me the
> former option simply makes very little sense.

I agree we need consistency on how we handle security policy event 
channel. Although, I don't have a strong opinion on which way to go.

For the commit message, it is not entirely clear what "broke lseventch 
in dom0" really mean. Is it lsevtchn would not stop or it will just not 
display the event channel?

If the former, isn't a sign that the tool needs to be harden a bit more? 
If the latter, then I would argue that consistency for the XSM policy is 
more important than displaying the event channel for now (the patch was 
also committed 3 years ago...).

So I would vote for a revert and, if desired, replacing with a patch 
that would change the XSM policy consistently. Alternatively, the 
consistency should be a blocker for Xen 4.19.

> 
>> As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
>> had for the principle of the change based on the absurdity of your
>> arguments.
> 
> No, pending objections are pending objections. Daniel's responses didn't
> eliminate them.

Indeed, this is rule 4 of the check-in policy:

4. There must be no "open" objections.

I don't view Jan's objections as unreasonable in particular for the 
consistency part.

> As a separate aspect: I can't assume anymore that it is just coincidence
> that you taking such a controversial action is at a time when I'm away.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:20:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:20:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721424.1124812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qCS-0004xP-53; Tue, 14 May 2024 11:20:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721424.1124812; Tue, 14 May 2024 11:20:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qCS-0004wr-0r; Tue, 14 May 2024 11:20:04 +0000
Received: by outflank-mailman (input) for mailman id 721424;
 Tue, 14 May 2024 11:20:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6qCQ-0004Vc-29
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:20:02 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6qCP-0004st-N3; Tue, 14 May 2024 11:20:01 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6qCP-00074s-H6; Tue, 14 May 2024 11:20:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=5/tVt2Dk1f1bUYt40TqN7l9uRdImKRcMPvz2p+l+8l4=; b=ekHZY/o2x9ASvWw9G8JozFRvst
	YMqAXpV9R4eVsWGpDZFEs7eCjTzYK9AJMK6Xv2Zu/lOIhO72OCJv6geyJHMA7szFLW0u7R2t3YOex
	Qn4sVIelKGB/s6hnKQ9l3wXeqLQdCq0vtO0sEz23qq75f4glQ1cBpYPehG10VF3yo0x0=;
Message-ID: <862a7088-240a-42c1-9d36-703840aa3b73@xen.org>
Date: Tue, 14 May 2024 12:19:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] fix Rule 10.2 violation
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
 george.dunlap@citrix.com, jbeulich@suse.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, michal.orzel@amd.com
References: <alpine.DEB.2.22.394.2405101634500.2544314@ubuntu-linux-20-04-desktop>
 <0625f4f2-daaa-4867-8c39-f8e9ce939ca0@xen.org>
 <alpine.DEB.2.22.394.2405131706450.2544314@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405131706450.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 14/05/2024 01:09, Stefano Stabellini wrote:
> On Mon, 13 May 2024, Julien Grall wrote:
>> Hi Stefano,
>>
>> title: Is this the only violation we have in Xen? If so, then please add the
>> subsystem in the title.
> 
> The only remaining violations are about the use of the "toupper" macro.
> Bugseng is recommending to add a cast to fix those or deviate toupper.

Ok. Please can you add a prefix in the title then?

> 
> 
>> On 11/05/2024 00:37, Stefano Stabellini wrote:
>>> Change opt_conswitch to char to fix a violation of Rule 10.2.
>>>
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>>>
>>> diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
>>> index 2c363d9c1d..3a3a97bcbe 100644
>>> --- a/xen/drivers/char/console.c
>>> +++ b/xen/drivers/char/console.c
>>> @@ -49,7 +49,7 @@ string_param("console", opt_console);
>>>    /* Char 1: CTRL+<char1> is used to switch console input between Xen and
>>> DOM0 */
>>>    /* Char 2: If this character is 'x', then do not auto-switch to DOM0 when
>>> it */
>>>    /*         boots. Any other value, or omitting the char, enables
>>> auto-switch */
>>> -static unsigned char __read_mostly opt_conswitch[3] = "a";
>>> +static char __read_mostly opt_conswitch[3] = "a";
>>
>> Looking at the rest of the code, we have:
>>
>> #define switch_code (opt_conswitch[0] - 'a' + 1)
>>
>> Can you confirm whether this is not somehow adding a new violation?
> 
> No, this patch is to fix a violation exactly there.

Thanks for the confirmation:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:48:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:48:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721432.1124822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qeG-0001QF-5u; Tue, 14 May 2024 11:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721432.1124822; Tue, 14 May 2024 11:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qeG-0001Q8-36; Tue, 14 May 2024 11:48:48 +0000
Received: by outflank-mailman (input) for mailman id 721432;
 Tue, 14 May 2024 11:48:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6qeE-0001Q0-RQ
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:48:46 +0000
Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com
 [2607:f8b0:4864:20::82e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea52c0b6-11e7-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 13:48:44 +0200 (CEST)
Received: by mail-qt1-x82e.google.com with SMTP id
 d75a77b69052e-43dfcbc4893so23429181cf.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 04:48:44 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54d6afasm67604621cf.24.2024.05.14.04.48.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 04:48:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea52c0b6-11e7-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715687323; x=1716292123; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Pdda6HZ6WUFsRUqYthgzkS5mAoIrF4iyAT4V8GimXJ8=;
        b=QJi3dQXcoKTQVB+kQgW0/sbYA56CeQz3SwoiQqBXGtUmBlaWIGd4fq8ukCgcWideh5
         +GCGAHcHw7Ba2zxixeA9jSKGq8HUAyTZa2OQYAKD+8s77S6J561H5+/p4ihdrSzkPr59
         6c0neaTSuZFVzDOieuqIRPX32TLX9zNtJJXAY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715687323; x=1716292123;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Pdda6HZ6WUFsRUqYthgzkS5mAoIrF4iyAT4V8GimXJ8=;
        b=KsUci26449NXizvOqHOxtTb2PGvWLDIUKpU5EQokcF8WM1nYJQZigYOQaafU55BSjA
         eqscp7dJgJi86MZGrynG0QYWu/j6PUdn9F3qTdlGb2GhMWuSvkVKPReF0Bzy0Yot903C
         xKy7xYWrxS1bpS/gt1/GoXa4y+AOnv9FKsgQMO32ehYMgZpI6We9VnIlddtaJzesbFp1
         e3b7pRpi8GT4EsXwtvrEwbe/gPxzg7WPz1tEKGpzkDEvzhkFlG1/FxAJfcXYa6MOQCci
         o5JyzeZWQXYnwwNxWNjcgKMWPT7l5hYskkw5NcnZ3i8zUfxsDCMLluVXOHbJWKUJG+dx
         +oNA==
X-Gm-Message-State: AOJu0Ywp5G126v1NF7PQW636OQIo2M3HH+iw8eppSaDNvLeDZQTdy9Sr
	HfxzZYz7OVhiaHBate9iVS3dFKPyVQCGfj8Kd66jPGfvVelHo2xVqribpAo4AFA=
X-Google-Smtp-Source: AGHT+IHMTLnsnmlLZMOjjuADMCd7cRwgAl3+97UqinJIqCLyTWFWB8ihTWVY1x/atOEhmyXBZVlJfw==
X-Received: by 2002:a05:622a:8b:b0:43a:dd9f:7aa8 with SMTP id d75a77b69052e-43dfdb1ea7bmr135540141cf.9.1715687323231;
        Tue, 14 May 2024 04:48:43 -0700 (PDT)
Date: Tue, 14 May 2024 13:48:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 09/19] x86/domain_page: Remove the fast paths
 when mfn is not in the directmap
Message-ID: <ZkNPmARt2EgLCBwk@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-10-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-10-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:36PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When mfn is not in direct map, never use mfn_to_virt for any mappings.
> 
> We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with
> arch_mfns_in_direct_map(mfn, 1) because these two are equivalent. The
> extra comparison in arch_mfns_in_direct_map() looks different but because
> DIRECTMAP_VIRT_END is always higher, it does not make any difference.
> 
> Lastly, domain_page_map_to_mfn() needs to gain to a special case for
> the PMAP.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> 
> ----
> 
>     Changes since Hongyan's version:
>         * arch_mfn_in_direct_map() was renamed to arch_mfns_in_directmap()
>         * add a special case for the PMAP in domain_page_map_to_mfn()
> 
> diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
> index 55e337aaf7..89caefc8a2 100644
> --- a/xen/arch/x86/domain_page.c
> +++ b/xen/arch/x86/domain_page.c
> @@ -14,8 +14,10 @@
>  #include <xen/sched.h>
>  #include <xen/vmap.h>
>  #include <asm/current.h>
> +#include <asm/fixmap.h>
>  #include <asm/flushtlb.h>
>  #include <asm/hardirq.h>
> +#include <asm/pmap.h>
>  #include <asm/setup.h>
>  
>  static DEFINE_PER_CPU(struct vcpu *, override);
> @@ -35,10 +37,11 @@ static inline struct vcpu *mapcache_current_vcpu(void)
>      /*
>       * When using efi runtime page tables, we have the equivalent of the idle
>       * domain's page tables but current may point at another domain's VCPU.
> -     * Return NULL as though current is not properly set up yet.
> +     * Return the idle domains's vcpu on that core because the efi per-domain
> +     * region (where the mapcache is) is in-sync with the idle domain.
>       */
>      if ( efi_rs_using_pgtables() )
> -        return NULL;
> +        return idle_vcpu[smp_processor_id()];

There's already an existing instance of idle_vcpu[smp_processor_id()]
down in the function, it might make sense to put this in a local
variable.

>  
>      /*
>       * If guest_table is NULL, and we are running a paravirtualised guest,
> @@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn)
>      struct vcpu_maphash_entry *hashent;
>  
>  #ifdef NDEBUG
> -    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
> +    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
>          return mfn_to_virt(mfn_x(mfn));
>  #endif
>  
>      v = mapcache_current_vcpu();
> -    if ( !v )
> -        return mfn_to_virt(mfn_x(mfn));
> +    if ( !v || !v->domain->arch.mapcache.inuse )
> +    {
> +        if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
> +            return mfn_to_virt(mfn_x(mfn));
> +        else
> +        {
> +            BUG_ON(system_state >= SYS_STATE_smp_boot);
> +            return pmap_map(mfn);
> +        }
> +    }
>  
>      dcache = &v->domain->arch.mapcache;
>      vcache = &v->arch.mapcache;
> -    if ( !dcache->inuse )
> -        return mfn_to_virt(mfn_x(mfn));
>  
>      perfc_incr(map_domain_page_count);
>  
> @@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr)
>      if ( !va || va >= DIRECTMAP_VIRT_START )
>          return;
>  
> +    if ( va >= FIXADDR_START && va < FIXADDR_TOP )

This should be a fixmap helper IMO. virt_is_fixmap(addr) or similar.
There's already an existing instance in virt_to_fix().

> +    {
> +        pmap_unmap((void *)ptr);
> +        return;
> +    }
> +
>      ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
>  
>      v = mapcache_current_vcpu();
> @@ -237,7 +252,7 @@ int mapcache_domain_init(struct domain *d)
>      unsigned int bitmap_pages;
>  
>  #ifdef NDEBUG
> -    if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
> +    if ( !mem_hotplug && arch_mfn_in_directmap(0, max_page) )
>          return 0;
>  #endif
>  
> @@ -308,7 +323,7 @@ void *map_domain_page_global(mfn_t mfn)
>              local_irq_is_enabled()));
>  
>  #ifdef NDEBUG
> -    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
> +    if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) )
>          return mfn_to_virt(mfn_x(mfn));
>  #endif
>  
> @@ -335,6 +350,23 @@ mfn_t domain_page_map_to_mfn(const void *ptr)
>      if ( va >= DIRECTMAP_VIRT_START )
>          return _mfn(virt_to_mfn(ptr));
>  
> +    /*
> +     * The fixmap is stealing the top-end of the VMAP. So the check for
> +     * the PMAP *must* happen first.
> +     *
> +     * Also, the fixmap translate a slot to an address backwards. The
> +     * logic will rely on it to avoid any complexity. So check at
> +     * compile time this will always hold.
> +    */
> +    BUILD_BUG_ON(fix_to_virt(FIX_PMAP_BEGIN) < fix_to_virt(FIX_PMAP_END));
> +
> +    if ( ((unsigned long)fix_to_virt(FIX_PMAP_END) <= va) &&
> +         ((va & PAGE_MASK) <= (unsigned long)fix_to_virt(FIX_PMAP_BEGIN)) )
> +    {

Can we place this as some kind of helper in fixmap.h?

It's already quite ugly, and could be useful in other places.

bool virt_in_fixmap_range(addr, start idx, end idx)

Or something similar.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:51:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:51:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721435.1124831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qgS-0003ON-Ik; Tue, 14 May 2024 11:51:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721435.1124831; Tue, 14 May 2024 11:51:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qgS-0003OG-Fq; Tue, 14 May 2024 11:51:04 +0000
Received: by outflank-mailman (input) for mailman id 721435;
 Tue, 14 May 2024 11:51:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6qgR-0003OA-Vg
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:51:03 +0000
Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com
 [2607:f8b0:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3bc89039-11e8-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 13:51:01 +0200 (CEST)
Received: by mail-oi1-x232.google.com with SMTP id
 5614622812f47-3c999d53e04so2410911b6e.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 04:51:01 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792db4083c1sm245330085a.76.2024.05.14.04.50.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 04:50:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bc89039-11e8-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715687460; x=1716292260; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=aDL66rKh/a2veKVvyvn4d1xqXs8dntitr9GgFylNT+I=;
        b=ZlYMYvM5E7VNqV3FbhSScO5/NKSue5KD7srXUMjub1VIIX8haErliC5aE5rzCfiMhv
         i/QqrJcVopGRWa+WCTcuJxGZxeR9q/I5k1p1DJEtchipT2d0kY619zC4na5w61V7t2Tr
         sl6xokKyOicJVRRBSxp3Jp2rlz3U92fE5FHcA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715687460; x=1716292260;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=aDL66rKh/a2veKVvyvn4d1xqXs8dntitr9GgFylNT+I=;
        b=CIX6m+W/LP1j9OVliY/+6BcYVIhgBXs+aK55+OhgX36buqMC/b8hloUtMwR1uIdE0G
         JB3jegj14Jfx8gGmX/MNtV3dmZSwNs4zGYxe6n47XK0ZmZmo+wgckZwpsuUJrSKxqEI0
         ho2RLSVM+E0gxH8TgmMP4R/2EItRIk81HPVYe7HY9WR2+kg3FqFw2Lm6lKJ2vJjaVjOa
         UxT+KdLWgZy9QgFrX/pwsxIp2dtghxKz78CNUg3uDd940+0MWbRBE5Xh5g6ZqCHezULL
         mr7+mh29VVjjt6uGaNKQ0OIe6+0qUE2+JSJ8QoDM7JT9qSUp0pI/QfyOxUbZ+VOfbkoj
         PIIg==
X-Forwarded-Encrypted: i=1; AJvYcCXbWiU3vBpDdKIqOutVXeyvuoWlCYykOxHSK905pHc6JQVrguKsOe3y6XqivA4/Xau4/0k8PiZdNynMdbdk1G+aNQpf0PRoXqyIiz3wAIk=
X-Gm-Message-State: AOJu0YycN7Wa+QJ6kWN5fEn1iMJim9NrhF/tekgz7G1rEjrn6aVxyaTH
	2188Lri+QN9QnLYcAjiys966WsqQXD9Wll5R0vxaecSqLbSkEnfJ0crzdQUkZYI=
X-Google-Smtp-Source: AGHT+IHQBr8J9M8C6o+w7iF8iqRimFJfh6Db9MyBa3OYWwHK10kzK+T4rEtIVNOlcCA5j0pv9/7Zkw==
X-Received: by 2002:a05:6808:4193:b0:3c7:a4d:d54d with SMTP id 5614622812f47-3c99704881bmr13929579b6e.15.1715687459877;
        Tue, 14 May 2024 04:50:59 -0700 (PDT)
Message-ID: <34214222-1fe3-41a7-9017-412347654e66@citrix.com>
Date: Tue, 14 May 2024 12:50:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 12:09 pm, Andrew Cooper wrote:
> On 13/05/2024 9:59 am, Roger Pau Monne wrote:
>> There's no point in forcing a system wide update of the MTRRs on all processors
>> when there are no changes to be propagated.  On AP startup it's only the AP
>> that needs to write the system wide MTRR values in order to match the rest of
>> the already online CPUs.
>>
>> We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
>> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
>> on all the CPUs in the system.
>>
>> While there adjust the comment to clarify why the system-wide resetting of the
>> MTRR registers is not needed for the purposes of mtrr_ap_init().
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> ---
>> For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
>> triggering, but it's also a good performance improvement when performing
>> cpu-online.
>>
>> Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
>> the chance of the watchdog triggering due to the system-wide resetting of the
>> range.
> "Runtime" changes will only be during dom0 boot, if at all, but yes - it
> is restricted to a single MTRR at a time.
>
> It's XENPF_{add,del,read}_memtype, but it's only used by Classic Linux. 
> PVOps only issues read_memtype.
>
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Having stared at the manuals, I expect the reason this is intermittent
even with dedicated testing is because on SMM entry, CR0.CD/NW are
specifically unmodified.

Therefore, an SMI hitting the critical region will proceed at a glacial
pace.

But it does occur to me that the rendezvous is a plain rendezvous, which
means it will also be taking NMIs because of the watchdog at 2Hz, and
those will be glacial too.

A further optimisation would be to not disable caches if there are no
updates to make.  This will be the overwhelming common case in general,
and 100% case on CPU hot{un,}plug, but as it is, getting rid of the
unnecessary rendezvous is still a massive improvement.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:51:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:51:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721438.1124842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qhH-0003ts-S6; Tue, 14 May 2024 11:51:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721438.1124842; Tue, 14 May 2024 11:51:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qhH-0003tl-O7; Tue, 14 May 2024 11:51:55 +0000
Received: by outflank-mailman (input) for mailman id 721438;
 Tue, 14 May 2024 11:51:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6qhG-0003sZ-HD
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:51:54 +0000
Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com
 [2607:f8b0:4864:20::82a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b2895d1-11e8-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 13:51:53 +0200 (CEST)
Received: by mail-qt1-x82a.google.com with SMTP id
 d75a77b69052e-43de92e234dso50157741cf.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 04:51:53 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54b58b5sm68207871cf.7.2024.05.14.04.51.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 04:51:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b2895d1-11e8-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715687512; x=1716292312; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GKt2WiSV0PMWdBp4aAdKjqCr+kejxER0yNV4tKTETYM=;
        b=PiVvXZrlLuyu7DyDFGCQkskzLLVJAM1CVb/9yKsOh538wMEvIY8Lls2wMNb/DcFJge
         qspOMsnpS4zq9iapQAdUH4c9y03P82SBzKIFlZcENT0RwauX23bfAuaUY6XQZsZHNm5W
         NOzoYhSO6rvhLMVDzEM3b5Ux1Hno3G1t1vvrg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715687512; x=1716292312;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GKt2WiSV0PMWdBp4aAdKjqCr+kejxER0yNV4tKTETYM=;
        b=CJsC5cjyzKQG8nlByqG70Zv26Ztg10gteNWPRFRgGjJO1/CDoRS7HTLetBhg/uRiTS
         wWW4IP3MJMHTJ1cdlfMPeXZ9Cv6z03nB5m5uP8x9EgRqcjwtAuO2/hWBTck7lI87XX7w
         Pt63DLV+caL8DtU593YwYWAn9QoD6HtpGFm9RFYDlXg0oktHAph34xaMG7POU40O5Uxs
         QWnxiFUEWkS3Vfa3QhNv1x4m56HkSSWi1eDxONcWIv1QDeQM1RIVNta3raMYJwtyugNr
         nDN3MNM1ygmnfzCl4HeF2tdhliPkltaP0KGTRjYiXz3ASo823K2R+FiPoYETbNRn4MAx
         DOlA==
X-Gm-Message-State: AOJu0Yw+bkwHsyJMsj1wDDr9IJ0m66M74FWGc+QH6G35spErGj7Vsqfo
	krsY53qSEeSUYIyk5KhFjJ47xYteL4/PldtCexnC1o9bzbEq3BBcxFKdcAqe9no=
X-Google-Smtp-Source: AGHT+IEfkjud1hhz/xujaxaGcccgP0hYzTkxTckE1wd3KQXis07UuU3cM+A/lZNiqEd+Aan5+DtlUQ==
X-Received: by 2002:ac8:5d02:0:b0:43e:26f5:7e5d with SMTP id d75a77b69052e-43e26f58193mr28881381cf.12.1715687512512;
        Tue, 14 May 2024 04:51:52 -0700 (PDT)
Date: Tue, 14 May 2024 13:51:50 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Elias El Yandouzi <eliasely@amazon.com>
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Message-ID: <ZkNQVh-N9UUPxD7F@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com>
 <ZkMxgDPYgaKnmRRE@macbook>
 <87950da7-f765-4656-808f-6c63a29ced3d@suse.com>
 <ZkM7SK8U1Qsk7E3x@macbook>
 <90621115-e370-49a1-b272-d20f861d6137@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <90621115-e370-49a1-b272-d20f861d6137@suse.com>

On Tue, May 14, 2024 at 12:26:29PM +0200, Jan Beulich wrote:
> On 14.05.2024 12:22, Roger Pau Monné wrote:
> > On Tue, May 14, 2024 at 11:43:14AM +0200, Jan Beulich wrote:
> >> On 14.05.2024 11:40, Roger Pau Monné wrote:
> >>> On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
> >>>> @@ -53,6 +55,8 @@ enum fixed_addresses {
> >>>>      FIX_PV_CONSOLE,
> >>>>      FIX_XEN_SHARED_INFO,
> >>>>  #endif /* CONFIG_XEN_GUEST */
> >>>> +    FIX_PMAP_BEGIN,
> >>>> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
> >>>
> >>> This would better have
> >>>
> >>> #ifdef CONFIG_HAS_PMAP
> >>>
> >>> guards?
> >>
> >> That's useful only when the option can actually be off in certain
> >> configurations, isn't it?
> > 
> > My comment earlier on this patch suggested to make CONFIG_HAS_PMAP be
> > selected by HAS_SECRET_HIDING, rather than being unconditionally
> > arch-selected (if that's possible, I certainly don't know the usage in
> > further patches).
> 
> Right, but in patch 6 HAS_SECRET_HIDING is selected unconditionally,
> which would then also select HAS_PMAP. If, otoh, HAS_PMAP was selected
> only when SECRET_HIDING (or whatever its name is going to be), then an
> #ifdef would indeed be wanted here.

Oh, indeed, I was meant to tie to SECRET_HIDING and not
HAS_SECRET_HIDING.  I have to admit (as I've already commented on the
patch) I don't much like those names, they are far too generic.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 11:57:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 11:57:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721445.1124851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qmM-0004bd-Gl; Tue, 14 May 2024 11:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721445.1124851; Tue, 14 May 2024 11:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qmM-0004bW-E5; Tue, 14 May 2024 11:57:10 +0000
Received: by outflank-mailman (input) for mailman id 721445;
 Tue, 14 May 2024 11:57:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6qmK-0004bQ-UC
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 11:57:08 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1606b68c-11e9-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 13:57:06 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59ae3efbb3so8664066b.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 04:57:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea65edsm7490971a12.7.2024.05.14.04.57.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 04:57:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1606b68c-11e9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715687826; x=1716292626; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P5t9oaiq0kcWuDwcVYP4BqAuLPAYeAqcFt76hLCI4xk=;
        b=O11loO6mwHj5obczyqrYCor9PmfiOEOYRfLRyZ1bo8UMQBOsFQ1Gx6STTpK/4DWpWP
         LNlCE+j+ItQj6SpD6ev4um3HQBKKOgTTqekY7wWydvkZC9kGxzLtQCr3UxsaaIngoecR
         4uLvbUn+PgzP1tOl61PO7WO6KTM1/PcpWJaCEIAO+2uPGDoFA76TA6S7gfVR/bHeYx+4
         K8s7QeeMAbAJnKoQnlcG7s1uDeDWljDeiRiinh/DTdcWaZC2vfmTF5gz4k2pOOrpV2zr
         i/rqbGwMw3TdxCp+gD6uZFLh9OQU0hERa6fHeptDvktdlmPgchBG9dCzoh5fNsm/wYb6
         4Z6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715687826; x=1716292626;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P5t9oaiq0kcWuDwcVYP4BqAuLPAYeAqcFt76hLCI4xk=;
        b=MNGQJzslrERbw2iCXP4niiVBBp4ovEekSBGQRys7y3y3KB6Me03lfdwcSecxSey4Gy
         I5G/7G5Q/w1iskrKn2TxYtuvvITnTzC944b+Tq9q6d4rdP3jGAjgt1qf1sRzmUDcGiZY
         jt5uwCmQUitObJgyO+XxEDV23qKxwKQ5IhwOMXCdxPvTKg11zPW+0leSlACYkirxBuaD
         hoxmkbXN+gF6i2tAPf0/Mo2POEf/EcAqLDzqNaKtPv4968I5mM5iL/dLxzs3hMPd0jnh
         9GvGurst+kuMlDvNSQJSMHBOGQlQqzHOo89U2VFB9lEFBE8dGZoLoihbSPBdvjzsWmPy
         cxHA==
X-Forwarded-Encrypted: i=1; AJvYcCWEiGyQO9LvuNjhHFlYeRgq8Aq6UPvrL3LKfwH+UT9k8nyHMmIIS43dF6fsXKbdKk5C8pNVZDUnVANztXOtDDeLbpChWeV+t846b5LWB2A=
X-Gm-Message-State: AOJu0YxHXZ4XUknuKR6bNz9bG8FKaKhL65HiOd+yrBaauK48u+V+TXXR
	73b6GcCMRT2yR3QTsikOPqZhQBKEVk/qdqvaTcC0StlhrW02zum4NJzmCEzWgQ==
X-Google-Smtp-Source: AGHT+IFkuva0e2NmdrS9VMZl0F6D938sa4MPV3j8Nk5XFoxtVhDK/RLGuKssZo0k4x+7Bq0uX2qFnQ==
X-Received: by 2002:a50:ccc8:0:b0:572:795a:b913 with SMTP id 4fb4d7f45d1cf-5734d67fd75mr10041045a12.34.1715687826216;
        Tue, 14 May 2024 04:57:06 -0700 (PDT)
Message-ID: <435ccfc3-d4e3-4da4-838f-4fb0e8987eab@suse.com>
Date: Tue, 14 May 2024 13:57:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240513085925.59324-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513085925.59324-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 10:59, Roger Pau Monne wrote:
> --- a/xen/arch/x86/cpu/mtrr/main.c
> +++ b/xen/arch/x86/cpu/mtrr/main.c
> @@ -573,14 +573,15 @@ void mtrr_ap_init(void)
>  	if (!mtrr_if || hold_mtrr_updates_on_aps)
>  		return;
>  	/*
> -	 * Ideally we should hold mtrr_mutex here to avoid mtrr entries changed,
> -	 * but this routine will be called in cpu boot time, holding the lock
> -	 * breaks it. This routine is called in two cases: 1.very earily time
> -	 * of software resume, when there absolutely isn't mtrr entry changes;
> -	 * 2.cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug lock to
> -	 * prevent mtrr entry changes
> +	 * hold_mtrr_updates_on_aps takes care of preventing unnecessary MTRR
> +	 * updates when batch starting the CPUs (see
> +	 * mtrr_aps_sync_{begin,end}()).
> +	 *
> +	 * Otherwise just apply the current system wide MTRR values to this AP.
> +	 * Note this doesn't require synchronization with the other CPUs, as
> +	 * there are strictly no modifications of the current MTRR values.
>  	 */
> -	set_mtrr(~0U, 0, 0, 0);
> +	mtrr_set_all();
>  }

While I agree with the change here, it doesn't go quite far enough. Originally
I meant to ask that, with this (supposedly) sole use of ~0U gone, you please
also drop the handling of that special case from set_mtrr(). But another
similar call exist in mtrr_aps_sync_end(). Yet while that's "fine" for the
boot case (watchdog being started only slightly later), it doesn't look to be
for the S3 resume one: The watchdog is re-enabled quite a bit earlier there.
I actually wonder whether mtrr_aps_sync_{begin,end}() wouldn't better
themselves invoke watchdog_{dis,en}able(), thus also making the boot case
explicitly safe, not just safe because of ordering.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:07:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721451.1124861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qwN-0006y4-GT; Tue, 14 May 2024 12:07:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721451.1124861; Tue, 14 May 2024 12:07:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6qwN-0006xx-Dy; Tue, 14 May 2024 12:07:31 +0000
Received: by outflank-mailman (input) for mailman id 721451;
 Tue, 14 May 2024 12:07:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6qwL-0006xr-J4
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:07:29 +0000
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com
 [2607:f8b0:4864:20::f35])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 88616362-11ea-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 14:07:28 +0200 (CEST)
Received: by mail-qv1-xf35.google.com with SMTP id
 6a1803df08f44-6a0f889877cso18240326d6.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 05:07:28 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1872f8sm52727256d6.55.2024.05.14.05.07.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 05:07:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88616362-11ea-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715688447; x=1716293247; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=oZjFV9qxtpZP1OtUKJ3si0WwVe7CzAFYin7mIAjiyKc=;
        b=kI8iRuSskHPiSmjfBvCHE6OEH1Qeft4KwqsXyGQGxobNZI+HL4xpM+WUlMB0e/gBaH
         /8JldORuBYaSVqwfBh1ly79+O9p3Mu0BXRuxT24OpG/U8em+56XsZWPnNyHTYIkMf8h4
         3GR0JawtY60zBV/JV99jB+Rgcw9WJ6QEn5sSY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715688447; x=1716293247;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=oZjFV9qxtpZP1OtUKJ3si0WwVe7CzAFYin7mIAjiyKc=;
        b=nmmk7zQDQujuAhKoZjQ8Y7y0jc+Mw2zmWhlxfyrKPSUWi+hCgJlrP0ktSSTC4tpTEq
         Pu0MBDx2MlGK3hmX7sqqXv/Q/gagDDQT6ttCkybjccDJ/C6TO5C4d7M7XaMxjgsRcE24
         9j0tqJ2+RgR3Fgsmm57uhjV5jlaN0TaTZhdpVU0iBQKIGdNTt62xf/GlZXOAWpZdWpp/
         96XXfW8jKTOw2aY8syXR94rBnenKxRyzKJb66tsiYZgTb0Wd7RdMol2VEWLrbZHDNeG0
         Ew9+sBJiOimWuvnhwfxPOvy5B8wzPQxmaP/kMsvXLThZ7Fhq7JLFj4rxGLzndL2LyNCU
         8I+g==
X-Forwarded-Encrypted: i=1; AJvYcCUCssnFCsFye9Q0J6WBY4RqS+zrpU13rW76NckwMzAwokvbQ21Uiz0c5kR2Icz3r53la2r1fJyJfRVLLaqEyyHlsLj/W9iB/2oNnMCqkUE=
X-Gm-Message-State: AOJu0YxBGKU+xklKe6qoM1YA+6hKKifMRqXnV/5JaxqZGtSLJAe5ShU6
	6Mno9ewNgfokCTc8DNO2D2hYkgDNTKLodBb4ybVqzmR6WR5J7vR3Si/x2XsIujtE13KD5BOcYg0
	N
X-Google-Smtp-Source: AGHT+IHOG4xrsXj7pUxVrWs7S4Hm9PsBNYXXrFMcQ6hJ9EiH8DRWQFEEZ/kntmaJZzo/ciE+NIw7Hg==
X-Received: by 2002:a05:6214:568c:b0:6a0:585d:a4c6 with SMTP id 6a1803df08f44-6a168138c3dmr142118446d6.6.1715688447529;
        Tue, 14 May 2024 05:07:27 -0700 (PDT)
Message-ID: <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
Date: Tue, 14 May 2024 13:07:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
To: Leigh Brown <leigh@solinno.co.uk>, xen-devel@lists.xenproject.org,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240514081344.4499-1-leigh@solinno.co.uk>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 9:13 am, Leigh Brown wrote:
> Although using integer comparison to compare doubles kind of
> works, it's annoying to see domains slightly out of order when
> sorting by cpu%.
>
> Add a compare_dbl() function and update compare_cpu_pct() to
> call it.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
> ---
>  tools/xentop/xentop.c | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
> index 545bd5e96d..99199caec9 100644
> --- a/tools/xentop/xentop.c
> +++ b/tools/xentop/xentop.c
> @@ -85,6 +85,7 @@ static void set_delay(const char *value);
>  static void set_prompt(const char *new_prompt, void (*func)(const char *));
>  static int handle_key(int);
>  static int compare(unsigned long long, unsigned long long);
> +static int compare_dbl(double, double);
>  static int compare_domains(xenstat_domain **, xenstat_domain **);
>  static unsigned long long tot_net_bytes( xenstat_domain *, int);
>  static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long *);
> @@ -422,6 +423,16 @@ static int compare(unsigned long long i1, unsigned long long i2)
>  	return 0;
>  }
>  
> +/* Compares two double precision numbers, returning -1,0,1 for <,=,> */
> +static int compare_dbl(double d1, double d2)
> +{
> +	if(d1 < d2)
> +		return -1;
> +	if(d1 > d2)
> +		return 1;
> +	return 0;
> +}
> +
>  /* Comparison function for use with qsort.  Compares two domains using the
>   * current sort field. */
>  static int compare_domains(xenstat_domain **domain1, xenstat_domain **domain2)
> @@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
>  
>  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain *domain2)
>  {
> -	return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
> +	return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));

Oh, we were doing an implicit double->unsigned long long conversion. 
Over the range 0.0 to 100.0, that ought to work as expected.  What kind
of out-of-order are you seeing?

Nevertheless, this should comparison should clearly be done using
doubles.  AFACT, get_cpu_pct() shouldn't ever return a NaN, so I think
this simple form is fine.

Oleksii: This is another bugfix to xentop, and should be considered for
4.19 at this point.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:16:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:16:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721454.1124872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6r4X-0000dU-B5; Tue, 14 May 2024 12:15:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721454.1124872; Tue, 14 May 2024 12:15:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6r4X-0000dN-86; Tue, 14 May 2024 12:15:57 +0000
Received: by outflank-mailman (input) for mailman id 721454;
 Tue, 14 May 2024 12:15:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/uDt=MR=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s6r4W-0000dF-52
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:15:56 +0000
Received: from smtp-190c.mail.infomaniak.ch (smtp-190c.mail.infomaniak.ch
 [2001:1600:4:17::190c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b4eb066d-11eb-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 14:15:52 +0200 (CEST)
Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch
 [10.7.10.108])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VdwLH3dTwzN03;
 Tue, 14 May 2024 14:15:51 +0200 (CEST)
Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VdwLD3B6RzB4h; Tue, 14 May 2024 14:15:48 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4eb066d-11eb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=digikod.net;
	s=20191114; t=1715688951;
	bh=6cLb3jtwfOqoRKJtvZPYLahhceLlkxg39lBtYJITgWQ=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=KRT/ov4rWMykef9xMhIwSZhHfcSSN6GQwPRDurzMGPbliQsS0VO6zSHgJ9Wbadi3l
	 OGIHTWGzorzDqVpD81oSg4u9D970KQGA1j7nSMc8pi7COPs75RE7gvSRn175mFw02a
	 0K4edTjsnRc2MHxsDF/mZpD6rApUOvuun3U3rT6U=
Date: Tue, 14 May 2024 14:15:46 +0200
From: =?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
To: Sean Christopherson <seanjc@google.com>, 
	Nicolas Saenz Julienne <nsaenz@amazon.com>
Cc: Borislav Petkov <bp@alien8.de>, 
	Dave Hansen <dave.hansen@linux.intel.com>, "H . Peter Anvin" <hpa@zytor.com>, 
	Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, Anna Trikalinou <atrikalinou@microsoft.com>, 
	Chao Peng <chao.p.peng@linux.intel.com>, Forrest Yuan Yu <yuanyu@google.com>, 
	James Gowans <jgowans@amazon.com>, James Morris <jamorris@linux.microsoft.com>, 
	John Andersen <john.s.andersen@intel.com>, "Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, 
	Marian Rotariu <marian.c.rotariu@gmail.com>, Mihai =?utf-8?B?RG9uyJt1?= <mdontu@bitdefender.com>, 
	=?utf-8?B?TmljdciZb3IgQ8OuyJt1?= <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, 
	Yu Zhang <yu.c.zhang@linux.intel.com>, =?utf-8?Q?=C8=98tefan_=C8=98icleru?= <ssicleru@bitdefender.com>, 
	dev@lists.cloudhypervisor.org, kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, 
	x86@kernel.org, xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
Message-ID: <20240514.OoPohLaejai6@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com>
 <20240506.ohwe7eewu0oB@digikod.net>
 <ZjmFPZd5q_hEBdBz@google.com>
 <20240507.ieghomae0UoC@digikod.net>
 <ZjpTxt-Bxia3bRwB@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZjpTxt-Bxia3bRwB@google.com>
X-Infomaniak-Routing: alpha

On Tue, May 07, 2024 at 09:16:06AM -0700, Sean Christopherson wrote:
> On Tue, May 07, 2024, Mickaël Salaün wrote:
> > > Actually, potential bad/crazy idea.  Why does the _host_ need to define policy?
> > > Linux already knows what assets it wants to (un)protect and when.  What's missing
> > > is a way for the guest kernel to effectively deprivilege and re-authenticate
> > > itself as needed.  We've been tossing around the idea of paired VMs+vCPUs to
> > > support VTLs and SEV's VMPLs, what if we usurped/piggybacked those ideas, with a
> > > bit of pKVM mixed in?
> > > 
> > > Borrowing VTL terminology, where VTL0 is the least privileged, userspace launches
> > > the VM at VTL0.  At some point, the guest triggers the deprivileging sequence and
> > > userspace creates VTL1.  Userpace also provides a way for VTL0 restrict access to
> > > its memory, e.g. to effectively make the page tables for the kernel's direct map
> > > writable only from VTL1, to make kernel text RO (or XO), etc.  And VTL0 could then
> > > also completely remove its access to code that changes CR0/CR4.
> > > 
> > > It would obviously require a _lot_ more upfront work, e.g. to isolate the kernel
> > > text that modifies CR0/CR4 so that it can be removed from VTL0, but that should
> > > be doable with annotations, e.g. tag relevant functions with __magic or whatever,
> > > throw them in a dedicated section, and then free/protect the section(s) at the
> > > appropriate time.
> > > 
> > > KVM would likely need to provide the ability to switch VTLs (or whatever they get
> > > called), and host userspace would need to provide a decent amount of the backend
> > > mechanisms and "core" policies, e.g. to manage VTL0 memory, teardown (turn off?)
> > > VTL1 on kexec(), etc.  But everything else could live in the guest kernel itself.
> > > E.g. to have CR pinning play nice with kexec(), toss the relevant kexec() code into
> > > VTL1.  That way VTL1 can verify the kexec() target and tear itself down before
> > > jumping into the new kernel. 
> > > 
> > > This is very off the cuff and have-wavy, e.g. I don't have much of an idea what
> > > it would take to harden kernel text patching, but keeping the policy in the guest
> > > seems like it'd make everything more tractable than trying to define an ABI
> > > between Linux and a VMM that is rich and flexible enough to support all the
> > > fancy things Linux does (and will do in the future).
> > 
> > Yes, we agree that the guest needs to manage its own policy.  That's why
> > we implemented Heki for KVM this way, but without VTLs because KVM
> > doesn't support them.
> > 
> > To sum up, is the VTL approach the only one that would be acceptable for
> > KVM?  
> 
> Heh, that's not a question you want to be asking.  You're effectively asking me
> to make an authorative, "final" decision on a topic which I am only passingly
> familiar with.
> 
> But since you asked it... :-)  Probably?
> 
> I see a lot of advantages to a VTL/VSM-like approach:
> 
>  1. Provides Linux-as-a guest the flexibility it needs to meaningfully advance
>     its security, with the least amount of policy built into the guest/host ABI.
> 
>  2. Largely decouples guest policy from the host, i.e. should allow the guest to
>     evolve/update it's policy without needing to coordinate changes with the host.
> 
>  3. The KVM implementation can be generic enough to be reusable for other features.
> 
>  4. Other groups are already working on VTL-like support in KVM, e.g. for VSM
>     itself, and potentially for VMPL/SVSM support.
> 
> IMO, #2 is a *huge* selling point.  Not having to coordinate changes across
> multiple code bases and/or organizations and/or maintainers is a big win for
> velocity, long term maintenance, and probably the very viability of HEKI.

Agree, this is our goal.

> 
> Providing the guest with the tools to define and implement its own policy means
> end users don't have to way for some third party, e.g. CSPs, to deploy the
> accompanying host-side changes, because there are no host-side changes.
> 
> And encapsulating everything in the guest drastically reduces the friction with
> changes in the kernel that interact with hardening, both from a technical and a
> social perspective.  I.e. giving the kernel (near) complete control over its
> destiny minimizes the number of moving parts, and will be far, far easier to sell
> to maintainers.  I would expect maintainers to react much more favorably to being
> handed tools to harden the kernel, as opposed to being presented a set of APIs
> that can be used to make the kernel compliant with _someone else's_ vision of
> what kernel hardening should look like.
> 
> E.g. imagine a new feature comes along that requires overriding CR0/CR4 pinning
> in a way that doesn't fit into existing policy.  If the VMM is involved in
> defining/enforcing the CR pinning policy, then supporting said new feature would
> require new guest/host ABI and an updated host VMM in order to make the new
> feature compatible with HEKI.  Inevitably, even if everything goes smoothly from
> an upstreaming perspective, that will result in guests that have to choose between
> HEKI and new feature X, because there is zero chance that all hosts that run Linux
> as a guest will be updated in advance of new feature X being deployed.

Sure. We need to find a generic-enough KVM interface to be able to
restrict a wide range of virtualization/hardware mechanisms (to not rely
too much on KVM changes) and delegate most of enforcement/emulation to
VTL1.  In short, policy definition owned by VTL0/guest, and policy
enforcement shared between KVM (coarse grained) and VTL1 (fine grained).

> 
> And if/when things don't go smoothly, odds are very good that kernel maintainers
> will eventually tire of having to coordinate and negotiate with QEMU and other
> VMMs, and will become resistant to continuing to support/extend HEKI.

Yes, that was our concern too and another reason why we choose to let
the guest handle its own security policy.

> 
> > If yes, that would indeed require a *lot* of work for something we're not
> > sure will be accepted later on.
> 
> Yes and no.  The AWS folks are pursuing VSM support in KVM+QEMU, and SVSM support
> is trending toward the paired VM+vCPU model.  IMO, it's entirely feasible to
> design KVM support such that much of the development load can be shared between
> the projects.  And having 2+ use cases for a feature (set) makes it _much_ more
> likely that the feature(s) will be accepted.
> 
> And similar to what Paolo said regarding HEKI not having a complete story, I
> don't see a clear line of sight for landing host-defined policy enforcement, as
> there are many open, non-trivial questions that need answers. I.e. upstreaming
> HEKI in its current form is also far from a done deal, and isn't guaranteed to
> be substantially less work when all is said and done.

I'm not sure to understand why "Heki not having a complete story".  The
goal is the same as the current kernel self-protection mechanisms.


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:19:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:19:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721459.1124882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6r8H-0001Et-Q5; Tue, 14 May 2024 12:19:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721459.1124882; Tue, 14 May 2024 12:19:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6r8H-0001Em-MS; Tue, 14 May 2024 12:19:49 +0000
Received: by outflank-mailman (input) for mailman id 721459;
 Tue, 14 May 2024 12:19:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6r8H-0001Ec-18; Tue, 14 May 2024 12:19:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6r8G-0005st-Tl; Tue, 14 May 2024 12:19:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6r8G-0006I9-GQ; Tue, 14 May 2024 12:19:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6r8G-000530-G2; Tue, 14 May 2024 12:19:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tBJXiatnGKfMB8jKbvTTqP/w/q8t6cEfxZU+6Ncws08=; b=HB4kdumgBCqgfsTXP5Msy7zY4I
	D+P6qdv648LACgExk/TARj2rURDj/iog4keWKNdq14ag8mh97xC0c5qy+ckbVxo9YRtsBrI7PnxFL
	Eg1JqtCta6w0J5AnQRdbm14S78rgZHgVFFTHysAR1ifRvuPiEXT2d9aRsPIffhG9ns5M=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185993-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 185993: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:guest-localmigrate/x10:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 May 2024 12:19:48 +0000

flight 185993 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185993/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 18 guest-localmigrate/x10 fail pass in 185987

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185987
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185987
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185987
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185987
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185987
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185987
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   185993  2024-05-14 01:51:55 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue May 14 12:24:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:24:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721466.1124892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rC9-0002l6-FO; Tue, 14 May 2024 12:23:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721466.1124892; Tue, 14 May 2024 12:23:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rC9-0002kz-CW; Tue, 14 May 2024 12:23:49 +0000
Received: by outflank-mailman (input) for mailman id 721466;
 Tue, 14 May 2024 12:23:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/uDt=MR=digikod.net=mic@srs-se1.protection.inumbo.net>)
 id 1s6rC8-0002kt-SC
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:23:48 +0000
Received: from smtp-bc0e.mail.infomaniak.ch (smtp-bc0e.mail.infomaniak.ch
 [45.157.188.14]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cfa71842-11ec-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 14:23:46 +0200 (CEST)
Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch
 [10.7.10.108])
 by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4VdwWP64pbzTXk;
 Tue, 14 May 2024 14:23:45 +0200 (CEST)
Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA
 id 4VdwWN5tzlzF2L; Tue, 14 May 2024 14:23:44 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfa71842-11ec-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=digikod.net;
	s=20191114; t=1715689425;
	bh=I0eZWdP4Q+KPu4Hx3s3mpwHuoQ0UvkeO9/nafOAr82w=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=HUYIhJQSGxJF3jwUizanlmonUX7sygOvcV8noUaAFzN0V3a3iKwuGB8/GEum6Clxq
	 EcKo1qBH3Un5VS8jWoGkoTi3TiKlCjIcaIL1kB+8poVzqExKvp5IQ2ywDDPeKqlugY
	 OVZlyAPCKt5gtA5k6143lcqKUer3NgtqPGvwYp+A=
Date: Tue, 14 May 2024 14:23:42 +0200
From: =?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
To: Nicolas Saenz Julienne <nsaenz@amazon.com>
Cc: Sean Christopherson <seanjc@google.com>, 
	Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
	"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, 
	Kees Cook <keescook@chromium.org>, Paolo Bonzini <pbonzini@redhat.com>, 
	Thomas Gleixner <tglx@linutronix.de>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Rick P Edgecombe <rick.p.edgecombe@intel.com>, 
	Alexander Graf <graf@amazon.com>, Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	Mihai =?utf-8?B?RG9uyJt1?= <mdontu@bitdefender.com>, =?utf-8?B?TmljdciZb3IgQ8OuyJt1?= <nicu.citu@icloud.com>, 
	Thara Gopinath <tgopinath@microsoft.com>, Trilok Soni <quic_tsoni@quicinc.com>, 
	Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, 
	Yu Zhang <yu.c.zhang@linux.intel.com>, =?utf-8?Q?=C8=98tefan_=C8=98icleru?= <ssicleru@bitdefender.com>, 
	dev@lists.cloudhypervisor.org, kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, 
	x86@kernel.org, xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
Message-ID: <20240514.mai3Ahdoo2qu@digikod.net>
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com>
 <20240506.ohwe7eewu0oB@digikod.net>
 <ZjmFPZd5q_hEBdBz@google.com>
 <20240507.ieghomae0UoC@digikod.net>
 <ZjpTxt-Bxia3bRwB@google.com>
 <D15VQ97L5M8J.1TDNQE6KLW6JO@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <D15VQ97L5M8J.1TDNQE6KLW6JO@amazon.com>
X-Infomaniak-Routing: alpha

On Fri, May 10, 2024 at 10:07:00AM +0000, Nicolas Saenz Julienne wrote:
> On Tue May 7, 2024 at 4:16 PM UTC, Sean Christopherson wrote:
> > > If yes, that would indeed require a *lot* of work for something we're not
> > > sure will be accepted later on.
> >
> > Yes and no.  The AWS folks are pursuing VSM support in KVM+QEMU, and SVSM support
> > is trending toward the paired VM+vCPU model.  IMO, it's entirely feasible to
> > design KVM support such that much of the development load can be shared between
> > the projects.  And having 2+ use cases for a feature (set) makes it _much_ more
> > likely that the feature(s) will be accepted.
> 
> Since Sean mentioned our VSM efforts, a small update. We were able to
> validate the concept of one KVM VM per VTL as discussed in LPC. Right
> now only for single CPU guests, but are in the late stages of bringing
> up MP support. The resulting KVM code is small, and most will be
> uncontroversial (I hope). If other obligations allow it, we plan on
> having something suitable for review in the coming months.

Looks good!

> 
> Our implementation aims to implement all the VSM spec necessary to run
> with Microsoft Credential Guard. But note that some aspects necessary
> for HVCI are not covered, especially the ones that depend on MBEC
> support, or some categories of secure intercepts.

We already implemented support for MBEC, so that should not be an issue.
We just need to find the best interface to configure it.

> 
> Development happens
> https://github.com/vianpl/{linux,qemu,kvm-unit-tests} and the vsm-next
> branch, but I'd advice against looking into it until we add some order
> to the rework. Regardless, feel free to get in touch.

Thanks for the update.

Could we schedule a PUCK meeting to synchronize and help each other?
What about June 12?


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:33:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:33:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721471.1124902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rLQ-0005YN-AI; Tue, 14 May 2024 12:33:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721471.1124902; Tue, 14 May 2024 12:33:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rLQ-0005YG-6x; Tue, 14 May 2024 12:33:24 +0000
Received: by outflank-mailman (input) for mailman id 721471;
 Tue, 14 May 2024 12:33:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6rLO-0005YA-Vm
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:33:22 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 267732e7-11ee-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 14:33:21 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-573061776e8so26546a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 05:33:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bfe2dd1sm7456691a12.58.2024.05.14.05.33.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 05:33:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 267732e7-11ee-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715690001; x=1716294801; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=B5YH4ZxQAJt8PMf0AVNDdRHjXLnrivmrZJoJ5IFdocU=;
        b=D7yn3UMhtXgiwk5uHEghFEA98IU1iN2j1kkozGZ7RJC620DHFkmDBdtZEfeygnNewp
         zutst37Vg/JgPADQAIHvqcvJ0RXCz8YcZMEcGxj8zDoMqjLBq+IvkWBEPLn6RXCrwhll
         7IcUcLfZCN2XotVlm1mjLCSqxO+cJ2/FEiacHoiYC1R5jfBaOBZVbh4nQscZSs/8chu+
         D12BQiG6FwaefCDvhisWDhz1+IMTpz6luOQ50PVAztGkKpKC7oS+vrmeZ5/Qe/bliZ9M
         mygipoT6ZTsU2O++ky+M8xVgFcqkszCqZkCitgJEXtVpsYQ0gkrXFXHpOstFxiISpI3s
         G0tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715690001; x=1716294801;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=B5YH4ZxQAJt8PMf0AVNDdRHjXLnrivmrZJoJ5IFdocU=;
        b=w+k6zwX5P0nYCQwN2nsGnfmKFH+PGsSXTWU3S0/S/9qtw6FAcvxkMN376Ym67LEHgw
         j/nD89CyZnQl28h8H2YHditqEfd3MM+3ucgWomf93SAdNC8OE1ItRy5JwbubmJOH+lAq
         6JWJUm2Xd0Tj9LwIs4x/mjWWB5fCxqnweKRg4Ols6WD/EOnDB4PTdpvoLVpLIt2rZCfk
         QbH9wDi7DUEBwO4jGl7IhhKgMSq+Cof255lj7O+Ww64ORftHY7PANBh2eCstEZgkE3y5
         6Uyho7oKZNAf3gREM8VYw87Hl/O+FCO9EhHuXcEOwGTCClz0krEAlmX/qzMtjaK6+dCK
         Of5g==
X-Gm-Message-State: AOJu0YwRlsN94uODiYSfe6AL/z4ZmQUHV99/v0E7JayzYsthnyXyPVLT
	IJm4vOj5yt20nUeT4TLvVItrcFTNFg2lE5XrrHDxaAzxm+QPAMbS/HYWxpm39w==
X-Google-Smtp-Source: AGHT+IGUbecmSAF5qaakNoPYwcdtRY4JPWgSypwTu9mQ2uoVDTBXyZa3iH7y1M2sqcnXvDmvQAiFKw==
X-Received: by 2002:a50:a417:0:b0:570:5b70:d76d with SMTP id 4fb4d7f45d1cf-5734d67aacamr8697555a12.28.1715690001300;
        Tue, 14 May 2024 05:33:21 -0700 (PDT)
Message-ID: <9807f36e-429d-4ebf-ae48-3f216330fd41@suse.com>
Date: Tue, 14 May 2024 14:33:28 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 07/19] xen/x86: Add support for the PMAP
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Julien Grall <jgrall@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-8-eliasely@amazon.com> <ZkMxgDPYgaKnmRRE@macbook>
 <87950da7-f765-4656-808f-6c63a29ced3d@suse.com> <ZkM7SK8U1Qsk7E3x@macbook>
 <90621115-e370-49a1-b272-d20f861d6137@suse.com> <ZkNQVh-N9UUPxD7F@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkNQVh-N9UUPxD7F@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 13:51, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 12:26:29PM +0200, Jan Beulich wrote:
>> On 14.05.2024 12:22, Roger Pau Monné wrote:
>>> On Tue, May 14, 2024 at 11:43:14AM +0200, Jan Beulich wrote:
>>>> On 14.05.2024 11:40, Roger Pau Monné wrote:
>>>>> On Mon, May 13, 2024 at 01:40:34PM +0000, Elias El Yandouzi wrote:
>>>>>> @@ -53,6 +55,8 @@ enum fixed_addresses {
>>>>>>      FIX_PV_CONSOLE,
>>>>>>      FIX_XEN_SHARED_INFO,
>>>>>>  #endif /* CONFIG_XEN_GUEST */
>>>>>> +    FIX_PMAP_BEGIN,
>>>>>> +    FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP,
>>>>>
>>>>> This would better have
>>>>>
>>>>> #ifdef CONFIG_HAS_PMAP
>>>>>
>>>>> guards?
>>>>
>>>> That's useful only when the option can actually be off in certain
>>>> configurations, isn't it?
>>>
>>> My comment earlier on this patch suggested to make CONFIG_HAS_PMAP be
>>> selected by HAS_SECRET_HIDING, rather than being unconditionally
>>> arch-selected (if that's possible, I certainly don't know the usage in
>>> further patches).
>>
>> Right, but in patch 6 HAS_SECRET_HIDING is selected unconditionally,
>> which would then also select HAS_PMAP. If, otoh, HAS_PMAP was selected
>> only when SECRET_HIDING (or whatever its name is going to be), then an
>> #ifdef would indeed be wanted here.
> 
> Oh, indeed, I was meant to tie to SECRET_HIDING and not
> HAS_SECRET_HIDING.  I have to admit (as I've already commented on the
> patch) I don't much like those names, they are far too generic.

And I commented to this same effect on v2 already, without being heard.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:35:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:35:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721474.1124913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rNp-00065O-NC; Tue, 14 May 2024 12:35:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721474.1124913; Tue, 14 May 2024 12:35:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rNp-00065H-IY; Tue, 14 May 2024 12:35:53 +0000
Received: by outflank-mailman (input) for mailman id 721474;
 Tue, 14 May 2024 12:35:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6rNo-00065B-Pj
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:35:52 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7fd8e58f-11ee-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 14:35:51 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59c5c9c6aeso1984066b.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 05:35:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781ce3fsm719053266b.4.2024.05.14.05.35.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 05:35:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fd8e58f-11ee-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715690151; x=1716294951; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Ytjp0ZQ9pzO1BDsCUdFRdYK9I9TqClMxI7mpuhutEEM=;
        b=On1MbY3TXKGuUl0DoSItSB+G6av4MGnk55DC0jfUTbeqmQqMNUTBqtcY8TxtB7Y1WI
         gIZI3X4jj0rvmJntlDES/pPzaeTHTBEYBy0wi9HkZ/5REFypm7ZD32yqy6ihbtzTNRyb
         RYouGZn5gatT5+gLWuwIQQPiwVKBTK7yJYjZTBKkovvioxc6hBInLBC6PnM3kVwSbr+f
         C/mGvvoNqk9AKnKOozwB/Ogngs+Toq0PEWlPxCA5b3LC+avv2sPL1DUt+dncXpCTXK7X
         jNbtUEEnUkGjsvIjGXArYem9t34DhUzoRrmQUrY0vWMCKex1o/havrh1IUjWQybmxOaf
         rKeA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715690151; x=1716294951;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ytjp0ZQ9pzO1BDsCUdFRdYK9I9TqClMxI7mpuhutEEM=;
        b=c/s6mQajOb9vDc+/iDrGJlqfwfsNdLL9Bg0zCT/YyLUlxn+ayXTBEd5awP5+7tY/+z
         7Y3UJzE4Kzh9Hgk8cmyVhk0JRAShtY2rub45gZHFR2y2XeVpDXoR2xygmQgAFveHJm4u
         ErGnM0Gs3K/BV8nw47QmUs86u7VjNog9VaydFLR5Q+eavdxpWjuzrzLOLKA7STAiCJW/
         fP6G4X5PUm9b4lPoCxevckV8mnylq/Mm8GiadiwyfxsrMNy4qyZW2KtNa2ixf9HryxWW
         0n2ixA0RHlSo663SUncT64NKqVaDvTfuvQof5tXIvgep9Rh2OGhGyGuEEY+Btka4giGg
         UM7w==
X-Forwarded-Encrypted: i=1; AJvYcCUaJOkEVStlG5LY1sHaD/+Usa+0krrX+AcghTTJndpr2pRZybVAxv9sG77iAdlMoN/tUth6Lr7X/pxrHdkH4HjlTFjv80s2Wlxbgg6+eOc=
X-Gm-Message-State: AOJu0YwhSxZRt7AnCr54/qbuV90t11J5JbSD1/Ms/AK+zFARyI64hem4
	DidfGGERwqZAs6GPA7P7VA/PWygqmNtPM8cOQYlX7Pg196RZReRHQHWQ3APZQg==
X-Google-Smtp-Source: AGHT+IGw4qWfr6bW+gfEvm0osWRfcCN5FzTJu8ZFOzZUg2q5PaHpBIkkTEBuiD944N7NXpAJOlDk2w==
X-Received: by 2002:a17:907:7da3:b0:a58:e3d9:e2d6 with SMTP id a640c23a62f3a-a5a2d672f76mr1089417566b.56.1715690151295;
        Tue, 14 May 2024 05:35:51 -0700 (PDT)
Message-ID: <c39e3c5b-1b6b-447e-93a9-46457535436b@suse.com>
Date: Tue, 14 May 2024 14:35:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
 <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
Cc: Leigh Brown <leigh@solinno.co.uk>, xen-devel@lists.xenproject.org,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 14:07, Andrew Cooper wrote:
> On 14/05/2024 9:13 am, Leigh Brown wrote:
>> Although using integer comparison to compare doubles kind of
>> works, it's annoying to see domains slightly out of order when
>> sorting by cpu%.
>>
>> Add a compare_dbl() function and update compare_cpu_pct() to
>> call it.
>>
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> ---
>>  tools/xentop/xentop.c | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
>> index 545bd5e96d..99199caec9 100644
>> --- a/tools/xentop/xentop.c
>> +++ b/tools/xentop/xentop.c
>> @@ -85,6 +85,7 @@ static void set_delay(const char *value);
>>  static void set_prompt(const char *new_prompt, void (*func)(const char *));
>>  static int handle_key(int);
>>  static int compare(unsigned long long, unsigned long long);
>> +static int compare_dbl(double, double);
>>  static int compare_domains(xenstat_domain **, xenstat_domain **);
>>  static unsigned long long tot_net_bytes( xenstat_domain *, int);
>>  static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long *);
>> @@ -422,6 +423,16 @@ static int compare(unsigned long long i1, unsigned long long i2)
>>  	return 0;
>>  }
>>  
>> +/* Compares two double precision numbers, returning -1,0,1 for <,=,> */
>> +static int compare_dbl(double d1, double d2)
>> +{
>> +	if(d1 < d2)
>> +		return -1;
>> +	if(d1 > d2)
>> +		return 1;
>> +	return 0;
>> +}
>> +
>>  /* Comparison function for use with qsort.  Compares two domains using the
>>   * current sort field. */
>>  static int compare_domains(xenstat_domain **domain1, xenstat_domain **domain2)
>> @@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
>>  
>>  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain *domain2)
>>  {
>> -	return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
>> +	return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));
> 
> Oh, we were doing an implicit double->unsigned long long conversion. 
> Over the range 0.0 to 100.0, that ought to work as expected.  What kind
> of out-of-order are you seeing?
> 
> Nevertheless, this should comparison should clearly be done using
> doubles.  AFACT, get_cpu_pct() shouldn't ever return a NaN, so I think
> this simple form is fine.

Just for completeness: INF would be similarly an issue, but hopefully cannot
come back from get_cpu_pct() either.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 12:36:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 12:36:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721478.1124921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rOd-0006bi-Tp; Tue, 14 May 2024 12:36:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721478.1124921; Tue, 14 May 2024 12:36:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rOd-0006bb-RD; Tue, 14 May 2024 12:36:43 +0000
Received: by outflank-mailman (input) for mailman id 721478;
 Tue, 14 May 2024 12:36:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p490=MR=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s6rOc-0006PE-4f
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 12:36:42 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9c9b417b-11ee-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 14:36:40 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 7D9628006E;
 Tue, 14 May 2024 13:36:39 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c9b417b-11ee-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715690199;
	bh=6qjycH+HuVrx/VRN8AxAfdr4wODMVUK2uSjIiZUnmfU=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=zQlTODQgwUgxYP9Rz49UUlCKTW65m5DZOkJHiTVl0vcqWdWIXAlkZctXL3pUeSHrO
	 uxQ0o1AfW+vRUBW1hmpzvkACqSPJfiuKlT1UUQVamLR86ONaNEIfM39ue5GBNOmtPd
	 Gs+XKaliExgJXhV++s4R4I/kv73ThoZrKnz/AQbY=
MIME-Version: 1.0
Date: Tue, 14 May 2024 13:36:39 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
In-Reply-To: <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
 <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
Message-ID: <d1fbc48ee89da3f4fe45a96161f9502e@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hello,

On 2024-05-14 13:07, Andrew Cooper wrote:
> On 14/05/2024 9:13 am, Leigh Brown wrote:
>> Although using integer comparison to compare doubles kind of
>> works, it's annoying to see domains slightly out of order when
>> sorting by cpu%.
>> 
>> Add a compare_dbl() function and update compare_cpu_pct() to
>> call it.
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> ---
>>  tools/xentop/xentop.c | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>> 
>> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
>> index 545bd5e96d..99199caec9 100644
>> --- a/tools/xentop/xentop.c
>> +++ b/tools/xentop/xentop.c
>> @@ -85,6 +85,7 @@ static void set_delay(const char *value);
>>  static void set_prompt(const char *new_prompt, void (*func)(const 
>> char *));
>>  static int handle_key(int);
>>  static int compare(unsigned long long, unsigned long long);
>> +static int compare_dbl(double, double);
>>  static int compare_domains(xenstat_domain **, xenstat_domain **);
>>  static unsigned long long tot_net_bytes( xenstat_domain *, int);
>>  static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long 
>> *);
>> @@ -422,6 +423,16 @@ static int compare(unsigned long long i1, 
>> unsigned long long i2)
>>  	return 0;
>>  }
>> 
>> +/* Compares two double precision numbers, returning -1,0,1 for <,=,> 
>> */
>> +static int compare_dbl(double d1, double d2)
>> +{
>> +	if(d1 < d2)
>> +		return -1;
>> +	if(d1 > d2)
>> +		return 1;
>> +	return 0;
>> +}
>> +
>>  /* Comparison function for use with qsort.  Compares two domains 
>> using the
>>   * current sort field. */
>>  static int compare_domains(xenstat_domain **domain1, xenstat_domain 
>> **domain2)
>> @@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
>> 
>>  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain 
>> *domain2)
>>  {
>> -	return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
>> +	return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));
> 
> Oh, we were doing an implicit double->unsigned long long conversion. 
> Over the range 0.0 to 100.0, that ought to work as expected.  What kind
> of out-of-order are you seeing?

Without patch:

xentop - 13:29:01   Xen 4.18.2
13 domains: 1 running, 12 blocked, 0 paused, 0 crashed, 0 dying, 0 
shutdown
Mem: 67030640k total, 33097800k used, 33932840k free    CPUs: 24 @ 
3693MHz
       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) 
MAXMEM(%)
   icecream --b---       2597    6.6    4194368    6.3    4195328       
6.3
      xendd --b---       4016    5.4     524268    0.8     525312       
0.8
   Domain-0 -----r       1059    1.7    1048576    1.6    1048576       
1.6
       neon --b---        826    1.1    2097216    3.1    2098176       
3.1
    blender --b---        121    0.2    1048640    1.6    1049600       
1.6
      bread --b---         69    0.1     524352    0.8     525312       
0.8
        bob --b---        502    0.3   16777284   25.0   16778240      
25.0
     cheese --b---        225    0.5    1048384    1.6    1049600       
1.6
    cassini --b---        489    0.4    3145792    4.7    3146752       
4.7
   chickpea --b---         67    0.1     524352    0.8     525312       
0.8
     lentil --b---         67    0.1     262208    0.4     263168       
0.4
    fusilli --b---        159    0.2     524352    0.8     525312       
0.8
      pizza --b---        359    0.5     524352    0.8     525312       
0.8

With patch:

xentop - 13:30:17   Xen 4.18.2
13 domains: 1 running, 12 blocked, 0 paused, 0 crashed, 0 dying, 0 
shutdown
Mem: 67030640k total, 33097788k used, 33932852k free    CPUs: 24 @ 
3693MHz
       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) 
MAXMEM(%)
      xendd --b---       4020    5.7     524268    0.8     525312       
0.8
   icecream --b---       2600    3.8    4194368    6.3    4195328       
6.3
   Domain-0 -----r       1060    1.5    1048576    1.6    1048576       
1.6
       neon --b---        827    1.1    2097216    3.1    2098176       
3.1
     cheese --b---        225    0.7    1048384    1.6    1049600       
1.6
      pizza --b---        359    0.5     524352    0.8     525312       
0.8
    cassini --b---        490    0.4    3145792    4.7    3146752       
4.7
    fusilli --b---        159    0.2     524352    0.8     525312       
0.8
        bob --b---        502    0.2   16777284   25.0   16778240      
25.0
    blender --b---        121    0.2    1048640    1.6    1049600       
1.6
      bread --b---         69    0.1     524352    0.8     525312       
0.8
   chickpea --b---         67    0.1     524352    0.8     525312       
0.8
     lentil --b---         67    0.1     262208    0.4     263168       
0.4

> Nevertheless, this should comparison should clearly be done using
> doubles.  AFACT, get_cpu_pct() shouldn't ever return a NaN, so I think
> this simple form is fine.
> 
> Oleksii: This is another bugfix to xentop, and should be considered for
> 4.19 at this point.
> 
> ~Andrew

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:05:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:05:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721489.1124931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rpx-0003tH-3D; Tue, 14 May 2024 13:04:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721489.1124931; Tue, 14 May 2024 13:04:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rpx-0003tA-07; Tue, 14 May 2024 13:04:57 +0000
Received: by outflank-mailman (input) for mailman id 721489;
 Tue, 14 May 2024 13:04:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6rpv-0003t4-HC
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:04:55 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8e72418d-11f2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:04:54 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-572b37afd73so93241a12.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:04:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a55275e80sm438942666b.8.2024.05.14.06.04.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:04:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e72418d-11f2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715691894; x=1716296694; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=owwrnOYg/hRDFZK+JFkSVjuytfCfeFkLX9e2EIhvdoE=;
        b=Lg3lLkLJCUDOnISrBH2tod8fscQGk44e3j8JEq/pIFpUPySuCFjw8ngw9+rzoO9FUB
         1qehOv/Q0AiluKw1BBP6EYQdqBx6nZN3+AHdCq+z/elUQjoIXrxeK8+DBatPtJzQ02UI
         bGstxmjcNNkFAwArFiyQ/pk9DiWWgLyxTHdN0mmnmpCORqZY3fW2gK+w3abjW9epe1TN
         7A8PNzC006VP7YrdenrVsIwHle1EYXAHzpqRSun5tkb7rWtt+zEZ7ya+lNjh2SgGJIZ6
         v7pARxz0dy3HVviTG9zCEi2Wtp7/Z0wSpok9XeRzFnJWG1BxVygTE/Aj3CzpDq+fPM/u
         za5A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715691894; x=1716296694;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=owwrnOYg/hRDFZK+JFkSVjuytfCfeFkLX9e2EIhvdoE=;
        b=GlUScnWvdr8UwTxTiQ+WSRnFNAWj9IU8iyYf2C36fBDxm9aZEogL1tEOOXRihCkvwa
         EwwPdigjl5AyjXnmOoQdIPlHPt8bohM0qaaQ/7Rw16bSv7k4ZAAu8yY8iPcmKaGXJ4D/
         eaKXu9rI7UlkIEXUfeNdRuCgZOP2/qtjMVtz69p3Aq/2HwKcpgXj8T6r6/YKIN2nK66P
         ma+ed49frAvkTU7ogkm3VzhetdyiR9xwttbOuWgCzRn+GLii/aRI5OSBjrBl/iBuOaUH
         tUPROtD8bXG8wvhAzprelEndYznwrLt/33X712cIPPGtCCsuq825OcxoHj30MIdQtt3L
         9aFg==
X-Gm-Message-State: AOJu0YzqdVIULL0yI7d3beQLFgbfpiqUM3dftamVGbQvJIazjR4XmZzP
	QdMmt1wVAXWrWt3+gQ/GeVuctBoI1KiwDQVtkZchB1ucx0LvBf8zqqFgm+4swKerth+2COOj+ew
	=
X-Google-Smtp-Source: AGHT+IHBR29rK0DZF7QT9dOOsKkppf0MKSNEF1GxleHS/Ed2Sde3UTAPQsHDarV7j3kxvswGnMqGRw==
X-Received: by 2002:a17:907:7daa:b0:a59:ddec:b339 with SMTP id a640c23a62f3a-a5a2d66b530mr1011658966b.50.1715691893640;
        Tue, 14 May 2024 06:04:53 -0700 (PDT)
Message-ID: <bed75dc2-1cd4-462f-9ab7-f06a2e596d15@suse.com>
Date: Tue, 14 May 2024 15:05:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature
 names
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <ZkMYXSf9abR3J3FE@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMYXSf9abR3J3FE@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 09:53, Roger Pau Monné wrote:
> On Fri, May 10, 2024 at 11:40:01PM +0100, Andrew Cooper wrote:
>> Differences in names are:
>>
>>  sysenter    -> sep
>>  tm          -> tm1
>>  ds-cpl      -> dscpl
>>  est         -> eist
>>  sse41       -> sse4-1
>>  sse42       -> sse4-2
>>  movebe      -> movbe
>>  tsc-dl      -> tsc-deadline
>>  rdrnd       -> rdrand
>>  hyper       -> hypervisor
>>  mmx+        -> mmext
>>  fxsr+       -> ffxsr
>>  pg1g        -> page1gb
>>  3dnow+      -> 3dnowext
>>  cmp         -> cmp-legacy
>>  cr8d        -> cr8-legacy
>>  lzcnt       -> abm
>>  msse        -> misalignsse
>>  3dnowpf     -> 3dnowprefetch
>>  nodeid      -> nodeid-msr
>>  dbx         -> dbext
>>  tsc-adj     -> tsc-adjust
>>  fdp-exn     -> fdp-excp-only
>>  deffp       -> no-fpu-sel
>>  <24>        -> bld
>>  ppin        -> amd-ppin
>>  lfence+     -> lfence-dispatch
>>  ppin        -> intel-ppin
>>  energy-ctrl -> energy-filtering
>>
>> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
>> one.  Several of the + names would be nice to keep as were, but doing so isn't
>> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
>> but we intentionally don't list them, so I'm not worried.
>>
>> Thoughts?
> 
> I'm fine with this, we are now coherent between libxl, the Xen command
> line cpuid= option and the output of xen-cpuid.

Hmm, consistency across the components is of course a fair goal. Otherwise I
would have suggested to consider putting in place overrides in feature_names[]
for those cases where e.g. the trailing + might indeed be neater (and shorter).

>> --- a/tools/misc/xen-cpuid.c
>> +++ b/tools/misc/xen-cpuid.c
>> @@ -11,6 +11,7 @@
>>  #include <xenguest.h>
>>  
>>  #include <xen-tools/common-macros.h>
>> +#include <xen/lib/x86/cpuid-autogen.h>
>>  
>>  static uint32_t nr_features;
>>  
>> @@ -268,7 +269,7 @@ static const struct {
>>      const char *name;
>>      const char *abbr;
>>      const char *const *strs;
>> -} leaf_info[] = {
>> +} leaf_info[FEATURESET_NR_ENTRIES] = {
> 
> Won't it be best to not specify the number of array elements here, as
> we could then use a BUILD_BUG_ON() to detect when new leafs are added
> to the featureset and thus adjust xen-cpuid.c?  Otherwise new
> additions to the featureset will go unnoticed.

I, too, would be in favor of that.

>> @@ -291,6 +292,9 @@ static const struct {
>>  
>>  #define COL_ALIGN "24"
>>  
>> +static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
>> +    INIT_FEATURE_VAL_TO_NAME;
> 
> I've also considered this when doing the original patch, but it seemed
> worse to force each user of INIT_FEATURE_VAL_TO_NAME to have to
> correctly size the array.  I would also use '* 32', as it's IMO
> clearer and already used below when accessing the array.  I'm fine
> if we want to go this way, but the extra Python code to add a last
> array entry if required didn't seem that much TBH.

Same here.

>> --- a/xen/tools/gen-cpuid.py
>> +++ b/xen/tools/gen-cpuid.py
>> @@ -470,6 +470,27 @@ def write_results(state):
>>      state.output.write(
>>  """}
>>  
>> +""")
>> +
>> +    state.output.write(
>> +"""
>> +#define INIT_FEATURE_VAL_TO_NAME { \\
>> +""")
>> +
>> +    for name, bit in sorted(state.values.items()):
>> +        state.output.write(
>> +            '    [%s] = "%s",\\\n' % (bit, name)
>> +            )
>> +
>> +        # Add the other alias for 1d/e1d common bits
>> +        if bit in state.common_1d:
>> +            state.output.write(
>> +                '    [%s] = "%s",\\\n' % (64 + bit, name)

I realize right here this 64 can't very well be expanded to a useful
expression ((FEATURESET_e1d - FEATURESET_1d) * 32); could I talk you
into at least adding a comment to this effect?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:05:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:05:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721490.1124942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rqG-0004CH-FR; Tue, 14 May 2024 13:05:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721490.1124942; Tue, 14 May 2024 13:05:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rqG-0004CA-CA; Tue, 14 May 2024 13:05:16 +0000
Received: by outflank-mailman (input) for mailman id 721490;
 Tue, 14 May 2024 13:05:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6rqF-0003t4-1a
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:05:15 +0000
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [2607:f8b0:4864:20::72e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9a024633-11f2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:05:14 +0200 (CEST)
Received: by mail-qk1-x72e.google.com with SMTP id
 af79cd13be357-78ef9ce897bso441213985a.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:05:14 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf31171esm560024885a.116.2024.05.14.06.05.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:05:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a024633-11f2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715691913; x=1716296713; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=VZlzMx15mGmoqywlKLUPpZ7ZCTX2G3tkZqOGbbqWMAc=;
        b=A+wSwPwziKfPwZ23uiViNNiwRzK7Eg2EYl6N5pEadN12/j5Po2xYAAU31PvH06FBah
         F2gu8x6xqurYghQEh+t6789FEo9N9R6/INzp8Rn3ZXXEfnZeQisAwkTkUD4TqUMULswh
         B0uYtnrus8P93N/rDSgl47r5fxKAHNXSroh1c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715691913; x=1716296713;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VZlzMx15mGmoqywlKLUPpZ7ZCTX2G3tkZqOGbbqWMAc=;
        b=s9jl1O6hpRXBtX69mijzHEs39VxZnZpV/9VCfUwy+GoBigI7OeAXlYJz0t4RhulMTP
         dkJSTawKOD43aqLhwC2qDg58WbfzfB+C6G5govJ3/i8zwvaPTgFEjVVRb/lFtmK1Uxjz
         50E1HyvKCAveC5+3U3EdfbjDNkjF9qUQtNYLyT4o72rgyiTqATh/jdyjfy+mzTzzeRDx
         B0dKLdGIQii0pYCiXHbUIShfHgotNbHv4u1mPEG6gOs64TANbN/FlyTfjw65YvrxvpQS
         vvhZ/nMVj0MMg5gmKtyoF0fK/bBcB6sMIJaezNH+xcgOmiN80Q+/sO1qzQu78VzgehnK
         MyqQ==
X-Gm-Message-State: AOJu0YwZcQdkoj83ASpkremn9aC0veg9AaBtXxc1ICbS7dAAvSsHwX/v
	jC+hL2z1jAjMKXs6O34eEwxOzfzJrcS0Cc/9XxDAgAv55HQRTg9jyga98Vwe2gQ=
X-Google-Smtp-Source: AGHT+IFIrovBvZy989qWX0hkNMjIMDALIASpmB3kYTBnDq+FwonlN1GJ2ZHEWGOD8W6HIVET9eI3/w==
X-Received: by 2002:a05:620a:254d:b0:792:7f5d:46a8 with SMTP id af79cd13be357-792c75f44ccmr1749144985a.56.1715691912729;
        Tue, 14 May 2024 06:05:12 -0700 (PDT)
Message-ID: <58577656-d1c9-4f54-9526-83bdbcaa37a6@citrix.com>
Date: Tue, 14 May 2024 14:05:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature
 names
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <ZkMYXSf9abR3J3FE@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkMYXSf9abR3J3FE@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 8:53 am, Roger Pau Monné wrote:
> On Fri, May 10, 2024 at 11:40:01PM +0100, Andrew Cooper wrote:
>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>> index 6ee835b22949..2f34694e9c57 100644
>> --- a/tools/misc/xen-cpuid.c
>> +++ b/tools/misc/xen-cpuid.c
>> @@ -11,6 +11,7 @@
>>  #include <xenguest.h>
>>  
>>  #include <xen-tools/common-macros.h>
>> +#include <xen/lib/x86/cpuid-autogen.h>
>>  
>>  static uint32_t nr_features;
>>  
>> @@ -268,7 +269,7 @@ static const struct {
>>      const char *name;
>>      const char *abbr;
>>      const char *const *strs;
>> -} leaf_info[] = {
>> +} leaf_info[FEATURESET_NR_ENTRIES] = {
> Won't it be best to not specify the number of array elements here, as
> we could then use a BUILD_BUG_ON() to detect when new leafs are added
> to the featureset and thus adjust xen-cpuid.c?  Otherwise new
> additions to the featureset will go unnoticed.

Hmm.  I suppose we have the same in libxl_cpuid.c so we should do so here.

I'll do an adjustment.

>
>>      { "CPUID 0x00000001.edx",        "1d", str_1d },
>>      { "CPUID 0x00000001.ecx",        "1c", str_1c },
>>      { "CPUID 0x80000001.edx",       "e1d", str_e1d },
>> @@ -291,6 +292,9 @@ static const struct {
>>  
>>  #define COL_ALIGN "24"
>>  
>> +static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
>> +    INIT_FEATURE_VAL_TO_NAME;
> I've also considered this when doing the original patch, but it seemed
> worse to force each user of INIT_FEATURE_VAL_TO_NAME to have to
> correctly size the array.  I would also use '* 32', as it's IMO
> clearer and already used below when accessing the array.  I'm fine
> if we want to go this way, but the extra Python code to add a last
> array entry if required didn't seem that much TBH.

I was looking to avoid the other BUILD_BUG_ON()'s, and in particular
bringing in known_features just for a build time check.

Given that there's only one instance right now, and no obvious other
usecase, I'd say this is better.  In terms of just xen-cpuid.c, it's
clearly correct whereas leaving it implicitly to
INIT_FEATURE_VAL_TO_NAME is not.

>
>> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
>> index 79d7f5c8e1c9..d0bb2e4a229f 100755
>> --- a/xen/tools/gen-cpuid.py
>> +++ b/xen/tools/gen-cpuid.py
>> @@ -470,6 +470,27 @@ def write_results(state):
>>      state.output.write(
>>  """}
>>  
>> +""")
>> +
>> +    state.output.write(
>> +"""
>> +#define INIT_FEATURE_VAL_TO_NAME { \\
>> +""")
>> +
>> +    for name, bit in sorted(state.values.items()):
>> +        state.output.write(
>> +            '    [%s] = "%s",\\\n' % (bit, name)
>> +            )
>> +
>> +        # Add the other alias for 1d/e1d common bits
>> +        if bit in state.common_1d:
>> +            state.output.write(
>> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
>> +            )
> Had no idea we had this aliases.

Without this, you get a bunch of numbers when rendering e1d for known
features (all hardware), and all dynamic policies on AMD/Hygon hardware.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:07:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:07:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721497.1124952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rsP-00051F-T6; Tue, 14 May 2024 13:07:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721497.1124952; Tue, 14 May 2024 13:07:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rsP-000518-O0; Tue, 14 May 2024 13:07:29 +0000
Received: by outflank-mailman (input) for mailman id 721497;
 Tue, 14 May 2024 13:07:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6rsP-00050y-24
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:07:29 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e9ad2926-11f2-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:07:28 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-43df732cb05so13213461cf.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:07:27 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e12b42801sm32659671cf.55.2024.05.14.06.07.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 06:07:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9ad2926-11f2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715692047; x=1716296847; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=nm55sy5L4tLfMCnAnqjy2v6Z96IfSoi9VR5YhkRI+3Q=;
        b=LF8cUrF72txFUib0uc+yptbfuDoADOZCuqbVH+KJ5NG0vEaje2+P6PKXlcah2njGtT
         g1aOU5Cx7VRjhIMmKKW9AS6HZXuZ1ruBGgjpKF2qltiOkId4kziKbG356Ma0k+JuPT1N
         J1jsoHBydAfMdqnqO5muCz6m1CQ4xEIDpNdvo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715692047; x=1716296847;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nm55sy5L4tLfMCnAnqjy2v6Z96IfSoi9VR5YhkRI+3Q=;
        b=YCQ2zox/xZ26GjIdfycPJrknIwhnjwlOowk5K/HfbA9N8lWUtx6QUGvUC/+h5NEXTM
         BySpplUPf1t2YwjRLdX1LHdojk0qgItd7CdPtNQ5UqNhJagHQGHsspS03Bvla+ZD6b8X
         dFOPdL5I12xdbBNX3uwSYX6zrx9UAgA0/0hY3jvSBOW2MwVMcyOAJ1QVyFCbceMUq0Kg
         tK5oY7W3dlfi7mB9zuIoS9EKcvA6fu+2ly6EDYi9WFnrqsiGLAYH1g/Q42mQQW7xO8Uw
         aeQrwEwLvANP81lF+PmTuX8F+V8BNRVqWF3Tia2DkwAPJl31pkWsRCVjRgFvxWJrTj/p
         5ChA==
X-Gm-Message-State: AOJu0YzK8AZYRjMa0o4Ep4VrEXB0np+J78cTAtK07IzNrmZe29UEqHlv
	/Yj5q1UeF1pNItn6G1P+HotQCLGAAAZrPT0VwxaZzeeZViASaece0hHSFNXBWyo=
X-Google-Smtp-Source: AGHT+IHXt0AY1pzIoqtAIJnRpSSOxNa7wjI1BYwXzcE4Won4cnbp7TdYp1JkEpGUo9hTHyTNCc/OEA==
X-Received: by 2002:a05:622a:3d1:b0:43a:4b03:8000 with SMTP id d75a77b69052e-43dfdf9c697mr108672621cf.67.1715692046548;
        Tue, 14 May 2024 06:07:26 -0700 (PDT)
Date: Tue, 14 May 2024 15:07:24 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 10/19] xen/page_alloc: Add a path for xenheap
 when there is no direct map
Message-ID: <ZkNiDMqk7ijWi7bK@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-11-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-11-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:37PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When there is not an always-mapped direct map, xenheap allocations need
> to be mapped and unmapped on-demand.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     I have left the call to map_pages_to_xen() and destroy_xen_mappings()
>     in the split heap for now. I am not entirely convinced this is necessary
>     because in that setup only the xenheap would be always mapped and
>     this doesn't contain any guest memory (aside the grant-table).
>     So map/unmapping for every allocation seems unnecessary.

I'm also concerned by this, did you test that
CONFIG_SEPARATE_XENHEAP=y works properly with the added {,un}map
calls?

If CONFIG_SEPARATE_XENHEAP=y I would expect the memory returned by
alloc_heap_pages(MEMZONE_XEN...) to already have the virtual mappings
created ahead?

The comment at the top of page_alloc.c also needs to be updated to
notice how the removal of the direct map affects xenheap allocations,
AFAICT a new combination is now possible:

CONFIG_SEPARATE_XENHEAP=n & CONFIG_NO_DIRECTMAP=y

>     Changes in v2:
>         * Fix remaining wrong indentation in alloc_xenheap_pages()
> 
>     Changes since Hongyan's version:
>         * Rebase
>         * Fix indentation in alloc_xenheap_pages()
>         * Fix build for arm32
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 9b7e4721cd..dfb2c05322 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2242,6 +2242,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe)
>  void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>  {
>      struct page_info *pg;
> +    void *ret;

virt_addr maybe? ret is what I would expect to store the return value
of the function usually.

>  
>      ASSERT_ALLOC_CONTEXT();
>  
> @@ -2250,17 +2251,36 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>      if ( unlikely(pg == NULL) )
>          return NULL;
>  
> +    ret = page_to_virt(pg);
> +
> +    if ( !has_directmap() &&
> +         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
> +                          PAGE_HYPERVISOR) )
> +    {
> +        /* Failed to map xenheap pages. */
> +        free_heap_pages(pg, order, false);
> +        return NULL;
> +    }
> +
>      return page_to_virt(pg);
>  }
>  
>  
>  void free_xenheap_pages(void *v, unsigned int order)
>  {
> +    unsigned long va = (unsigned long)v & PAGE_MASK;
> +
>      ASSERT_ALLOC_CONTEXT();
>  
>      if ( v == NULL )
>          return;
>  
> +    if ( !has_directmap() &&
> +         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
> +        dprintk(XENLOG_WARNING,
> +                "Error while destroying xenheap mappings at %p, order %u\n",
> +                v, order);
> +
>      free_heap_pages(virt_to_page(v), order, false);
>  }
>  
> @@ -2284,6 +2304,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>  {
>      struct page_info *pg;
>      unsigned int i;
> +    void *ret;
>  
>      ASSERT_ALLOC_CONTEXT();
>  
> @@ -2296,16 +2317,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>      if ( unlikely(pg == NULL) )
>          return NULL;
>  
> +    ret = page_to_virt(pg);
> +
> +    if ( !has_directmap() &&
> +         map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order,
> +                          PAGE_HYPERVISOR) )
> +    {
> +        /* Failed to map xenheap pages. */
> +        free_domheap_pages(pg, order);
> +        return NULL;
> +    }
> +
>      for ( i = 0; i < (1u << order); i++ )
>          pg[i].count_info |= PGC_xen_heap;
>  
> -    return page_to_virt(pg);
> +    return ret;
>  }
>  
>  void free_xenheap_pages(void *v, unsigned int order)
>  {
>      struct page_info *pg;
>      unsigned int i;
> +    unsigned long va = (unsigned long)v & PAGE_MASK;
>  
>      ASSERT_ALLOC_CONTEXT();
>  
> @@ -2317,6 +2350,12 @@ void free_xenheap_pages(void *v, unsigned int order)
>      for ( i = 0; i < (1u << order); i++ )
>          pg[i].count_info &= ~PGC_xen_heap;
>  
> +    if ( !has_directmap() &&
> +         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )
> +        dprintk(XENLOG_WARNING,
> +                "Error while destroying xenheap mappings at %p, order %u\n",
> +                v, order);

I don't think this should be a dprintk(), leaving mappings behind
could be a severe issue given the point of this work is to prevent
leaking data by having everything mapped on the direct map.

This needs to be a printk() IMO, I'm unsure whether freeing the memory
would need to be avoided if destroying the mappings failed, I can't
think of how we could recover from this gracefully.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:11:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:11:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721503.1124962 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rvo-00073g-9w; Tue, 14 May 2024 13:11:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721503.1124962; Tue, 14 May 2024 13:11:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6rvo-00073Z-6T; Tue, 14 May 2024 13:11:00 +0000
Received: by outflank-mailman (input) for mailman id 721503;
 Tue, 14 May 2024 13:10:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6rvn-00073T-Bh
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:10:59 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 666419bc-11f3-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 15:10:57 +0200 (CEST)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-792ecce9522so45373785a.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:10:57 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2fc587sm561120185a.75.2024.05.14.06.10.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 06:10:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 666419bc-11f3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715692256; x=1716297056; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Y1SfSHiXaC6QhrOH7TgxyNsbQ5kNUGDEUN9bq4b11pc=;
        b=qugr5v8b/LOa1CZ/wNassNsAlkZsfNS+6Sx+QnNPthtlJdeWiZqtvWjv5ft3GUp4cg
         9/VS7ONdLKeQrfMGVNPH7S0R20o3EZl/hJbvwa8NLf/cfQUBXIW2gGr1tTBFjRmd1Cv2
         AU7lJ2hM/j1O73oA6gNWO9UwoBIn3uk0f9a8c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715692256; x=1716297056;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Y1SfSHiXaC6QhrOH7TgxyNsbQ5kNUGDEUN9bq4b11pc=;
        b=uCP7zGnZoBA9NihUB/ScGd9CnkA0I+4pRcad1oPYV/GNLuoSpkeZ/WJFNYjYyEf5ZM
         /8sUJ/pDV+lUUQoV5bQN6ZwNJwjEGcMjB+2w6mcdJ/8SlcCZE9wfIWhZgaFh1QMtx0jI
         wTi+iFjzApKaI0SLnV3unwW+jOySaGOrvoKTdI9FvMxG1oxuO+aP8dXZ3nFUu9/bgVd5
         ylxVC/kOZ3deiq8bZe90bXu12g6au9IwitaeaMtKLYLGSkMBs/LZjm1+TpLMmqE+rBEb
         V6Dc0wifOxqVFzS8OQZlc2IYgCr0Fi4DTvD8tG98Yjn/SHhVWZII+GhtxTQaR5O7SmfY
         408Q==
X-Forwarded-Encrypted: i=1; AJvYcCVVnPEOqzfuF56r22VfiTJs8BwCLgkitqUc3P/C7db7AO00+gjQA89SBcTV85PnP3DA255HcUbJjvElNoQNYvTCbpWwALvuGahBz61eMxk=
X-Gm-Message-State: AOJu0YwBf5YqyjE+D8FUp2NPT37VyXpVco3JqLKjfa/Wdk9F8ugLCT8C
	mkw6Y14RR/BhueXpNzvC0n51xwOLfQQAnbCc8/e1B8WxKFfbuQK7epSLViTQwZk=
X-Google-Smtp-Source: AGHT+IEoWK9Gw+dQabeRnCApT3pQG2wPcZy4eaeTMLpby12DFrU20XLO4LEOGFCb1XWLgHpeeGNZxw==
X-Received: by 2002:a05:620a:102e:b0:78a:5fd9:46d7 with SMTP id af79cd13be357-792c75ffc85mr1447254385a.67.1715692255735;
        Tue, 14 May 2024 06:10:55 -0700 (PDT)
Date: Tue, 14 May 2024 15:10:52 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
Message-ID: <ZkNi3HE4V_ErzyhR@macbook>
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <435ccfc3-d4e3-4da4-838f-4fb0e8987eab@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <435ccfc3-d4e3-4da4-838f-4fb0e8987eab@suse.com>

On Tue, May 14, 2024 at 01:57:13PM +0200, Jan Beulich wrote:
> On 13.05.2024 10:59, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/cpu/mtrr/main.c
> > +++ b/xen/arch/x86/cpu/mtrr/main.c
> > @@ -573,14 +573,15 @@ void mtrr_ap_init(void)
> >  	if (!mtrr_if || hold_mtrr_updates_on_aps)
> >  		return;
> >  	/*
> > -	 * Ideally we should hold mtrr_mutex here to avoid mtrr entries changed,
> > -	 * but this routine will be called in cpu boot time, holding the lock
> > -	 * breaks it. This routine is called in two cases: 1.very earily time
> > -	 * of software resume, when there absolutely isn't mtrr entry changes;
> > -	 * 2.cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug lock to
> > -	 * prevent mtrr entry changes
> > +	 * hold_mtrr_updates_on_aps takes care of preventing unnecessary MTRR
> > +	 * updates when batch starting the CPUs (see
> > +	 * mtrr_aps_sync_{begin,end}()).
> > +	 *
> > +	 * Otherwise just apply the current system wide MTRR values to this AP.
> > +	 * Note this doesn't require synchronization with the other CPUs, as
> > +	 * there are strictly no modifications of the current MTRR values.
> >  	 */
> > -	set_mtrr(~0U, 0, 0, 0);
> > +	mtrr_set_all();
> >  }
> 
> While I agree with the change here, it doesn't go quite far enough. Originally
> I meant to ask that, with this (supposedly) sole use of ~0U gone, you please
> also drop the handling of that special case from set_mtrr(). But another
> similar call exist in mtrr_aps_sync_end(). Yet while that's "fine" for the
> boot case (watchdog being started only slightly later), it doesn't look to be
> for the S3 resume one: The watchdog is re-enabled quite a bit earlier there.
> I actually wonder whether mtrr_aps_sync_{begin,end}() wouldn't better
> themselves invoke watchdog_{dis,en}able(), thus also making the boot case
> explicitly safe, not just safe because of ordering.

Hm, I don't like disabling the watchdog, I guess it could be
acceptable here because both usages of mtrr_aps_sync_end() are limited
to specific scenarios (boot or resume from suspension).  I can prepare
a separate patch, but I don't think the watchdog disabling should be
part of this patch.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:13:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:13:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721506.1124972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ryG-0007mv-LJ; Tue, 14 May 2024 13:13:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721506.1124972; Tue, 14 May 2024 13:13:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ryG-0007mo-IS; Tue, 14 May 2024 13:13:32 +0000
Received: by outflank-mailman (input) for mailman id 721506;
 Tue, 14 May 2024 13:13:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p490=MR=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s6ryE-0007me-RI
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:13:30 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c166362f-11f3-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:13:29 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id AF27E8006E;
 Tue, 14 May 2024 14:13:28 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c166362f-11f3-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715692408;
	bh=PW7VzRkT3sECDlfBsJvN32ftKyvfnGRz/kyS0PGx+zU=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=c2OT/ZW4fM3eACUpQqeH9D2lhZiwCSGPMSBNFHP4WFYpA0cHDpxVd5nTp9sWfLlDm
	 q3NEOnL9D3MVbA/OEMzoATfz2iHQxHglp/EUD9vnP0vEvhpqNgUyWA/yuCFT4b90r0
	 NfkQdffVvbf6PI1Y1Nmz1LVioXDCu/ATG+nKb/Jw=
MIME-Version: 1.0
Date: Tue, 14 May 2024 14:13:28 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
In-Reply-To: <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
 <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
Message-ID: <38bce1599deae1b77f2d55afaa765744@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-05-14 13:07, Andrew Cooper wrote:
> On 14/05/2024 9:13 am, Leigh Brown wrote:
>> Although using integer comparison to compare doubles kind of
>> works, it's annoying to see domains slightly out of order when
>> sorting by cpu%.
>> 
>> Add a compare_dbl() function and update compare_cpu_pct() to
>> call it.
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> ---
>>  tools/xentop/xentop.c | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>> 
>> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
>> index 545bd5e96d..99199caec9 100644
>> --- a/tools/xentop/xentop.c
>> +++ b/tools/xentop/xentop.c
>> @@ -85,6 +85,7 @@ static void set_delay(const char *value);
>>  static void set_prompt(const char *new_prompt, void (*func)(const 
>> char *));
>>  static int handle_key(int);
>>  static int compare(unsigned long long, unsigned long long);
>> +static int compare_dbl(double, double);
>>  static int compare_domains(xenstat_domain **, xenstat_domain **);
>>  static unsigned long long tot_net_bytes( xenstat_domain *, int);
>>  static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long 
>> *);
>> @@ -422,6 +423,16 @@ static int compare(unsigned long long i1, 
>> unsigned long long i2)
>>  	return 0;
>>  }
>> 
>> +/* Compares two double precision numbers, returning -1,0,1 for <,=,> 
>> */
>> +static int compare_dbl(double d1, double d2)
>> +{
>> +	if(d1 < d2)
>> +		return -1;
>> +	if(d1 > d2)
>> +		return 1;
>> +	return 0;
>> +}
>> +
>>  /* Comparison function for use with qsort.  Compares two domains 
>> using the
>>   * current sort field. */
>>  static int compare_domains(xenstat_domain **domain1, xenstat_domain 
>> **domain2)
>> @@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
>> 
>>  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain 
>> *domain2)
>>  {
>> -	return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
>> +	return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));
> 
> Oh, we were doing an implicit double->unsigned long long conversion. 
> Over the range 0.0 to 100.0, that ought to work as expected.  What kind
> of out-of-order are you seeing?
> 
> Nevertheless, this should comparison should clearly be done using
> doubles.  AFACT, get_cpu_pct() shouldn't ever return a NaN, so I think
> this simple form is fine.
> 
> Oleksii: This is another bugfix to xentop, and should be considered for
> 4.19 at this point.
> 
> ~Andrew

Perhaps I overthought it, and this approach might be better:

--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -523,7 +523,8 @@ static double get_cpu_pct(xenstat_domain *domain)

  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain 
*domain2)
  {
-       return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
+       return -compare(get_cpu_pct(domain1) * 100.0,
+                       get_cpu_pct(domain2) * 100.0);
  }

  /* Prints cpu percentage statistic */


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:36:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:36:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721514.1124982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sKD-0003eu-Hp; Tue, 14 May 2024 13:36:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721514.1124982; Tue, 14 May 2024 13:36:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sKD-0003en-E8; Tue, 14 May 2024 13:36:13 +0000
Received: by outflank-mailman (input) for mailman id 721514;
 Tue, 14 May 2024 13:36:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6sKC-0003eh-VK
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:36:12 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ecc43df5-11f6-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 15:36:10 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f1b378ca5so9678554e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:36:10 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a50365669sm485869266b.193.2024.05.14.06.36.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:36:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecc43df5-11f6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715693770; x=1716298570; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tI1mCcdu3kmApZJIMxVjSs2Dx1/LGxlqUqxBx/MkAyA=;
        b=A1BNAZJZuHSJRRf3SNCYv9wxOhlG0q+zKRCvwnNgNBz7ORl1HVeSSLahoGEGlFXJov
         8opavyvn6bFJ7L4Ypa7ygaksn1KQIXdKWO7kGzjSk5KlFZZXmSn7BfNyTfogYBz1UVl4
         VZ03XqVY9rCLpaDw32+6GZc62TgZjsONoBhkI+yxJUi0Qd6oilSZbz4gQTnKhOlwlG0H
         6nO2WLtGmMZf5JLtmL5BJqv7k5Ai5t+VkuJp9Knl4C27Eo+c7VzEZB/is11krxvEEF5D
         Zbr3t/wlVaHq4Rp2U1k+TEz410cICHw25jiBfxScNhHP/BcUXr63FEQb41PcrcFgHtBO
         GByQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715693770; x=1716298570;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tI1mCcdu3kmApZJIMxVjSs2Dx1/LGxlqUqxBx/MkAyA=;
        b=m6v0YEWtH4Yc7vRKQrm4gKzTyvYTzo4zcBHdzOShTt0O1lZcmMf8MHfGPgCln3IOfx
         tQg9grT0900Bu26TmVXwrsNpSSElwrTdxghJtJHV+LDxc8V7UXxRVULuEjC4yqkhbj7V
         TnsdcJywngwtOAI+1g8ZbajK5dnzFBPmDDWzSxltQ4sfebwqmrHu4u+gbuXvb3Zz3/l3
         cOpK/sz4eVMNNr5V0do0btNnu0/ibHvg9hh5gUpUQWTPSS6M4k0lW3WxqyghFpdDznWN
         tJUStTbj+0R9rt6x3iAcQi1mNjq/lC2M22ez3vcSULMiISH10duyEtc3e+mOyDH0gFxM
         Ju9g==
X-Forwarded-Encrypted: i=1; AJvYcCVPag5Ws5FVyr832RBeW7lfKuQfXLDW7KXFvjgeieGFlRJ45LPftcfk/Vbzp1ZN0ASDJvNp/lUPYY/D+SW1Vifkf9Z8fhXJ3xNtTnCGSmg=
X-Gm-Message-State: AOJu0YyV/PgiQPE1LfH6+l050uaCQ6lazKLiWI51Hrt7k92jB3NWjGe9
	/ovif8vy/3W2WbnlM5s8Skt7JCRji6DTKvp/z01k60tjbX2xbe0kyyDRHNPhxg==
X-Google-Smtp-Source: AGHT+IFUTjblc7BihXGNoUH7eyzKt+dMlBDeIqSMNg3vCiD2lPqD22xAdA3kQwBrHCl+aFdnQ916xw==
X-Received: by 2002:a05:6512:1308:b0:520:36ea:9375 with SMTP id 2adb3069b0e04-522102783ffmr12844611e87.43.1715693769860;
        Tue, 14 May 2024 06:36:09 -0700 (PDT)
Message-ID: <5749f2f4-19e1-435f-9daf-31c248fb9a0b@suse.com>
Date: Tue, 14 May 2024 15:36:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <435ccfc3-d4e3-4da4-838f-4fb0e8987eab@suse.com> <ZkNi3HE4V_ErzyhR@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkNi3HE4V_ErzyhR@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 15:10, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 01:57:13PM +0200, Jan Beulich wrote:
>> On 13.05.2024 10:59, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/cpu/mtrr/main.c
>>> +++ b/xen/arch/x86/cpu/mtrr/main.c
>>> @@ -573,14 +573,15 @@ void mtrr_ap_init(void)
>>>  	if (!mtrr_if || hold_mtrr_updates_on_aps)
>>>  		return;
>>>  	/*
>>> -	 * Ideally we should hold mtrr_mutex here to avoid mtrr entries changed,
>>> -	 * but this routine will be called in cpu boot time, holding the lock
>>> -	 * breaks it. This routine is called in two cases: 1.very earily time
>>> -	 * of software resume, when there absolutely isn't mtrr entry changes;
>>> -	 * 2.cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug lock to
>>> -	 * prevent mtrr entry changes
>>> +	 * hold_mtrr_updates_on_aps takes care of preventing unnecessary MTRR
>>> +	 * updates when batch starting the CPUs (see
>>> +	 * mtrr_aps_sync_{begin,end}()).
>>> +	 *
>>> +	 * Otherwise just apply the current system wide MTRR values to this AP.
>>> +	 * Note this doesn't require synchronization with the other CPUs, as
>>> +	 * there are strictly no modifications of the current MTRR values.
>>>  	 */
>>> -	set_mtrr(~0U, 0, 0, 0);
>>> +	mtrr_set_all();
>>>  }
>>
>> While I agree with the change here, it doesn't go quite far enough. Originally
>> I meant to ask that, with this (supposedly) sole use of ~0U gone, you please
>> also drop the handling of that special case from set_mtrr(). But another
>> similar call exist in mtrr_aps_sync_end(). Yet while that's "fine" for the
>> boot case (watchdog being started only slightly later), it doesn't look to be
>> for the S3 resume one: The watchdog is re-enabled quite a bit earlier there.
>> I actually wonder whether mtrr_aps_sync_{begin,end}() wouldn't better
>> themselves invoke watchdog_{dis,en}able(), thus also making the boot case
>> explicitly safe, not just safe because of ordering.
> 
> Hm, I don't like disabling the watchdog, I guess it could be
> acceptable here because both usages of mtrr_aps_sync_end() are limited
> to specific scenarios (boot or resume from suspension).  I can prepare
> a separate patch, but I don't think the watchdog disabling should be
> part of this patch.

Not sure (as to being part of this patch). Of course it would be okay to
address the S3 side separately, whichever approach we use. Yet imo it
would also be okay to address both in one go, again whichever approach we
use. If you prefer a separate one, so be it.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:44:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:44:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721528.1124992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sS4-0005tw-91; Tue, 14 May 2024 13:44:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721528.1124992; Tue, 14 May 2024 13:44:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sS4-0005tp-5t; Tue, 14 May 2024 13:44:20 +0000
Received: by outflank-mailman (input) for mailman id 721528;
 Tue, 14 May 2024 13:44:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6sS3-0005tf-Jq; Tue, 14 May 2024 13:44:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6sS3-0007I5-GL; Tue, 14 May 2024 13:44:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6sS3-0000A1-3m; Tue, 14 May 2024 13:44:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6sS3-00078H-3Q; Tue, 14 May 2024 13:44:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ArwehWLkBx0PkwtMf0GBBUq0V+E0Vcxzdz6+alWw9ao=; b=zYoRzYWrbab6lZBu4kvbjqjupq
	/rqLEain00d8ncvvu4+y4D/xIglb74a2JJt8OzPz/vdW/QkSaXYLmJAsYUCldErE4XTzrvcwxi1OQ
	hZVk317QZyoETAPmnDt1vbtrvcRktM8fcIAcDho0NwMKbo+8Fnem6JU4leuAEgwZMpP4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185994-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 185994: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=2566522a55977e03c4df124b16aeaf38ad801bb1
X-Osstest-Versions-That:
    libvirt=9e59ba56c8a26156799a556fa79c9654a5d1acd4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 May 2024 13:44:19 +0000

flight 185994 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185994/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185988
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              2566522a55977e03c4df124b16aeaf38ad801bb1
baseline version:
 libvirt              9e59ba56c8a26156799a556fa79c9654a5d1acd4

Last test of basis   185988  2024-05-13 04:20:28 Z    1 days
Testing same since   185994  2024-05-14 04:20:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Privoznik <mprivozn@redhat.com>
  Rayhan Faizel <rayhan.faizel@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   9e59ba56c8..2566522a55  2566522a55977e03c4df124b16aeaf38ad801bb1 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:47:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:47:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721535.1125001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sV9-0006TZ-MZ; Tue, 14 May 2024 13:47:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721535.1125001; Tue, 14 May 2024 13:47:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sV9-0006TS-Js; Tue, 14 May 2024 13:47:31 +0000
Received: by outflank-mailman (input) for mailman id 721535;
 Tue, 14 May 2024 13:47:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6sV8-0006TK-U3
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:47:30 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8119555d-11f8-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 15:47:28 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51f0b6b682fso6081548e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:47:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179263d9sm728477466b.95.2024.05.14.06.47.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:47:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8119555d-11f8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715694448; x=1716299248; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6gXJutPpaMImNmnLoQy2AAn0UxkAqLU8P2eGlEX6wDQ=;
        b=bnulQbXExi9ue1bkWshncHj8dG+rOQ5kkf9s0nnmoTi27SSy7k9xD+JyZCxqMJWr93
         eRH3vsd8OQIwI3QIYaTanqr02uTA4j0XVRg6N6uSK1YwyVDjyDpbn3efrcdIxj9DPB9k
         +Sxr5/A7ZAoobLcZbXkDRyrgLgk79R/0umfTLGeLARAESwVl8/5jmR3sfRMla2sxcWLh
         UyuDX3MStX+9Z8cn/KjW9LRlOBXuyrcb3PUX0r3z0X9YPyCK7iWLltkfR9ElAGCihNnt
         N9j4IkuTZls46kw7xSwOy00cJpWpWwGJGXFgHIeC7WVQt0TYni3Jrw5XssJTi9yko2nA
         wa3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715694448; x=1716299248;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6gXJutPpaMImNmnLoQy2AAn0UxkAqLU8P2eGlEX6wDQ=;
        b=gGOPcmRsZSAQyoDscEin+8sfJL17sLE9cM42OZLjWNaz+7N7M1utdur71Aq4s3v/TX
         BZJ08V+tjhRszDJFKKWpD1pdPIH45ywOAt/XUmk2UhmbBk2LOrtkef23qmZ1+YSXHJgi
         NZrAzTTRQ74KsBJ3Tjt8EBzBrLoRkzN81mDhv487TAR9Ao9AfrbNusw2MJ7mu9scb41P
         3TLxsJLXis1prvI+0rmPBO9/Ae56kPYeSHM79vykLvRIeQfnv3WtKLA0/LWS8BENkjaT
         2QuvjyP6XjXTfa0ONHKZHK9L0MupddmUpdyqx5e4GhnnA3A/3qNyPDR+Dcl8sNykmD5i
         uuaA==
X-Forwarded-Encrypted: i=1; AJvYcCWzAjGOa3P91mJ8QCbyj4NZhU3VEjMXeOAeEWi3Lta00d/L8rqLUqyb7PHBI4UvYiwpamJT5P+w3YXtw7xFNdXCegNQjaUY/rRIzIydkL4=
X-Gm-Message-State: AOJu0YxG5T7v+AMf46UJCXLLwrhiwrgSNq+vtdwxcXhdFkB5DyJsmtXx
	YJp0nFUNd7HnwSC4mZQ3mMJhDjmNwbO5WmpwuP5Mysbrm1+t3BmlE9fP+cxohA==
X-Google-Smtp-Source: AGHT+IG735Iwwspnulg1QVlfLn8ujVWOlPAiIcH7zwTga1L4cgCKJr7z2b9ShHqiU8wC1CVlPIqJrA==
X-Received: by 2002:a05:6512:3c87:b0:51d:5ffd:efe4 with SMTP id 2adb3069b0e04-5220fc6df22mr15365916e87.1.1715694448234;
        Tue, 14 May 2024 06:47:28 -0700 (PDT)
Message-ID: <369a3c75-2e71-4fb8-8373-aae737099e3b@suse.com>
Date: Tue, 14 May 2024 15:47:35 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/misra: replace R13.1 with R13.2
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 michal.orzel@amd.com, bertrand.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405101625470.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405101625470.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 11.05.2024 01:32, Stefano Stabellini wrote:
> After looking closely at the R13.1 violations and potential fixes or
> deviations, upon further reflection and discussions, we realized that it
> is R13.2, limited to initializers list, that we want to comply with.
> 
> Link: https://marc.info/?l=xen-devel&m=170751643325805

The mail you point at is a reply to one where I raised an aspect that was
never really addressed: 'I'm afraid I don't understand the "eventually
limiting" part.' Therefore I also don't follow why we'd want to limit
applicability of this rule to just initializer lists. I'm pretty sure we
do not want expressions the result of which or any side effects of which
are unpredictable, no matter where such an expression is used.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:50:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:50:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721537.1125012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sY3-00080L-Le; Tue, 14 May 2024 13:50:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721537.1125012; Tue, 14 May 2024 13:50:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sY1-0007zp-P7; Tue, 14 May 2024 13:50:29 +0000
Received: by outflank-mailman (input) for mailman id 721537;
 Tue, 14 May 2024 13:50:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6sXv-0007tI-Aa
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:50:23 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e888dc05-11f8-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:50:22 +0200 (CEST)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2e2c70f0c97so68385441fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:50:22 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccbe8e2fsm195316105e9.3.2024.05.14.06.50.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:50:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e888dc05-11f8-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715694622; x=1716299422; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=2AOQGq9+skWEV6tKei9ANLmo3y1qxkBttfUvGHeRp28=;
        b=REKGe50FeLuClHZ/mo9PofLkJp4wpdhTIbr6tfAhksHSL7/aDVZ0lk0hwNgr15c8i0
         qHy3+CJgyiQqct6taEwx5PgOauFAE+eMDFpu26SFQV8CgX10W7u5XcHM/26V2rAOWoR5
         qTSf1Rk6b67Ilh4EB/yz6KNA3+sdlAmKuJ25Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715694622; x=1716299422;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=2AOQGq9+skWEV6tKei9ANLmo3y1qxkBttfUvGHeRp28=;
        b=Mii6L4ZBb4RXDgXNHAozUYQYRj82wqPCnRlmXXl+COnnTe4dmB9/yYqTIgRHFkqfe0
         lVtTPS4YM4zSSqVgaQST2GsvGoZZSh4LhjGnYDA6umCE61wvsegWfxg2CDZgCf4DkvB6
         Q0wMPw3ADDLaLRL2497lkIlddbLwd5ysPveJrJ9XwEUNOM6elBS7/Q6nrRjVfHPqsvnu
         8a/J3ymcqOoAkZ4bjyJmCfSaCcNXT8+lDM2z6JImAjapO1TkyeGgx/N0+a/GGPbV2bc/
         HW3I0uj8rEbiWRytgRqyVF+qeD6pb4rBQOUZOhBMcGX4kfM/ggA4ZIrTBwO3HVJUrP0b
         e62w==
X-Forwarded-Encrypted: i=1; AJvYcCW5+gCxzVoLRTIOB8aHDK9DurwuX3dAHIX3r/wTVp2sPpzq4eF7Be1J2/9ibYQYZUnJ/A0w23t+pIT3VxfJXydnYhvY+R0HOKOnMtdXGbQ=
X-Gm-Message-State: AOJu0YzOlbVY/e1AUlRg2l7gM6J0aZhrZqltjHtjbn6/T4nHUD5SUGck
	SNULZhhmrRTbgiH6MNWgL8LLI/l6V7BkdTbjq/5RQCBvZ1KlG0uCLmVXizYTk30=
X-Google-Smtp-Source: AGHT+IH4bZZoyFcDhZTeTRVN4PISwUKN2hEV1GssB4HTQ1//pKd8njil8PbWTSrXzWU/hYUAoki64w==
X-Received: by 2002:a2e:96ca:0:b0:2e3:603e:469b with SMTP id 38308e7fff4ca-2e5204ccf5cmr80919331fa.40.1715694619788;
        Tue, 14 May 2024 06:50:19 -0700 (PDT)
Message-ID: <dd689012-0554-43ae-b22b-035415ce3392@citrix.com>
Date: Tue, 14 May 2024 14:50:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 12:09 pm, Andrew Cooper wrote:
> On 13/05/2024 9:59 am, Roger Pau Monne wrote:
>> There's no point in forcing a system wide update of the MTRRs on all processors
>> when there are no changes to be propagated.  On AP startup it's only the AP
>> that needs to write the system wide MTRR values in order to match the rest of
>> the already online CPUs.
>>
>> We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
>> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
>> on all the CPUs in the system.
>>
>> While there adjust the comment to clarify why the system-wide resetting of the
>> MTRR registers is not needed for the purposes of mtrr_ap_init().
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> ---
>> For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
>> triggering, but it's also a good performance improvement when performing
>> cpu-online.
>>
>> Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
>> the chance of the watchdog triggering due to the system-wide resetting of the
>> range.
> "Runtime" changes will only be during dom0 boot, if at all, but yes - it
> is restricted to a single MTRR at a time.
>
> It's XENPF_{add,del,read}_memtype, but it's only used by Classic Linux. 
> PVOps only issues read_memtype.
>
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Actually no - this isn't safe in all cases.

There are BIOSes which get MTRRs wrong, and with the APs having UC
covering a wider region than the BSP.

In this case, creating consistency will alter the MTRRs on all CPUs
currently up, and we do need to perform the rendezvous in that case.

There are 3 cases:

1) Nothing to do.  This is the overwhemlingly common case.
2) Local changes only.  No broadcast, but we do need to enter CD mode.
3) Remote changes needed.  Needs full broadcast.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:52:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:52:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721542.1125022 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sa9-0001dE-Qn; Tue, 14 May 2024 13:52:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721542.1125022; Tue, 14 May 2024 13:52:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sa9-0001d7-NN; Tue, 14 May 2024 13:52:41 +0000
Received: by outflank-mailman (input) for mailman id 721542;
 Tue, 14 May 2024 13:52:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6sa7-0001ch-SK
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:52:39 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39412531-11f9-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 15:52:37 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-420160f8f52so14658395e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:52:37 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee9326sm196681255e9.35.2024.05.14.06.52.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:52:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39412531-11f9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715694757; x=1716299557; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=NPYcK4moord7NEYjb+taKzz99LnDLvwtdm22uiCo+5U=;
        b=ZA+pJhsuk4hYWZ1F/1q1vEDWsyNbsAknwB8J+cOtdhdrGQuZ4GdcAC/VQPrpEc4+4d
         eNuY4maXTN7uikw7Ntkp7fD0f4CyXZvrLH6DBN9TE7UxIiemmPblTSPsLCbsgz/p46Jl
         WoXhZhYLS1WShDwMmttm4LwLITIyCLUtNE9lc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715694757; x=1716299557;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NPYcK4moord7NEYjb+taKzz99LnDLvwtdm22uiCo+5U=;
        b=sTwZdm98OhsBkoCgUV3I+j/vL1xPytLhKCcuNTaN9+efszJ2gALVG5khe65/JfZNqd
         Z+Fu5tPhKEbrndfmWwxhpkzFpTWZbkst2zQKg6PzSEg3h87KNwbO+6vLJlj4AzamBinx
         XN+WOVzQDPOMHYE0+eKyaUnszzhA7LKMTRRAoCxpgysxAGdMv/Q8+qrL2qnWWeAnCVfe
         j0frjmpiuiuxaxSW0kJN401hOrneZlwiQlZWkFVTwDQeLG65dH14FM+IcUrZ3VGmuc7v
         ELbdBfnQ7oSWGc1dbDQBZKKnuh90vlYvG2KxiPzGjhTYKoGqk83lrEz2QRr1m4+34fPp
         disQ==
X-Gm-Message-State: AOJu0YzkrqEwnWrLUm7OvygY5DNtAqOMtejUEvSC7vycEN8U3mAMhLd8
	/c6iDYrDpRuLfuTcWWsjMmvKAFJlgNLGBhLRrYLMJ8j7yp0hdO9l83r0LxG2BH0=
X-Google-Smtp-Source: AGHT+IEFoMLmsJ3yTH2DzeoTGKa0+dJQuappipNH3Out5EdoFUvbDeGQ4uncemPAzd9cJGDEJGL4iw==
X-Received: by 2002:a05:600c:3d06:b0:420:11c1:b240 with SMTP id 5b1f17b1804b1-42011c1b30amr58425125e9.24.1715694757193;
        Tue, 14 May 2024 06:52:37 -0700 (PDT)
Message-ID: <2f83c912-c6bf-4f02-9c64-abe539d25a61@citrix.com>
Date: Tue, 14 May 2024 14:52:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
 <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
 <d1fbc48ee89da3f4fe45a96161f9502e@solinno.co.uk>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d1fbc48ee89da3f4fe45a96161f9502e@solinno.co.uk>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 1:36 pm, Leigh Brown wrote:
> Hello,
> 
> On 2024-05-14 13:07, Andrew Cooper wrote:
>> On 14/05/2024 9:13 am, Leigh Brown wrote:
>>> Although using integer comparison to compare doubles kind of
>>> works, it's annoying to see domains slightly out of order when
>>> sorting by cpu%.
>>>
>>> Add a compare_dbl() function and update compare_cpu_pct() to
>>> call it.
>>>
>>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>>> ---
>>>  tools/xentop/xentop.c | 13 ++++++++++++-
>>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
>>> index 545bd5e96d..99199caec9 100644
>>> --- a/tools/xentop/xentop.c
>>> +++ b/tools/xentop/xentop.c
>>> @@ -85,6 +85,7 @@ static void set_delay(const char *value);
>>>  static void set_prompt(const char *new_prompt, void (*func)(const
>>> char *));
>>>  static int handle_key(int);
>>>  static int compare(unsigned long long, unsigned long long);
>>> +static int compare_dbl(double, double);
>>>  static int compare_domains(xenstat_domain **, xenstat_domain **);
>>>  static unsigned long long tot_net_bytes( xenstat_domain *, int);
>>>  static bool tot_vbd_reqs(xenstat_domain *, int, unsigned long long *);
>>> @@ -422,6 +423,16 @@ static int compare(unsigned long long i1,
>>> unsigned long long i2)
>>>      return 0;
>>>  }
>>>
>>> +/* Compares two double precision numbers, returning -1,0,1 for <,=,> */
>>> +static int compare_dbl(double d1, double d2)
>>> +{
>>> +    if(d1 < d2)
>>> +        return -1;
>>> +    if(d1 > d2)
>>> +        return 1;
>>> +    return 0;
>>> +}
>>> +
>>>  /* Comparison function for use with qsort.  Compares two domains
>>> using the
>>>   * current sort field. */
>>>  static int compare_domains(xenstat_domain **domain1, xenstat_domain
>>> **domain2)
>>> @@ -523,7 +534,7 @@ static double get_cpu_pct(xenstat_domain *domain)
>>>
>>>  static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain
>>> *domain2)
>>>  {
>>> -    return -compare(get_cpu_pct(domain1), get_cpu_pct(domain2));
>>> +    return -compare_dbl(get_cpu_pct(domain1), get_cpu_pct(domain2));
>>
>> Oh, we were doing an implicit double->unsigned long long conversion. 
>> Over the range 0.0 to 100.0, that ought to work as expected.  What kind
>> of out-of-order are you seeing?
> 
> Without patch:
> 
> xentop - 13:29:01   Xen 4.18.2
> 13 domains: 1 running, 12 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
> Mem: 67030640k total, 33097800k used, 33932840k free    CPUs: 24 @ 3693MHz
>       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%)
>   icecream --b---       2597    6.6    4194368    6.3    4195328       6.3
>      xendd --b---       4016    5.4     524268    0.8     525312       0.8
>   Domain-0 -----r       1059    1.7    1048576    1.6    1048576       1.6
>       neon --b---        826    1.1    2097216    3.1    2098176       3.1
>    blender --b---        121    0.2    1048640    1.6    1049600       1.6
>      bread --b---         69    0.1     524352    0.8     525312       0.8
>        bob --b---        502    0.3   16777284   25.0   16778240      25.0
>     cheese --b---        225    0.5    1048384    1.6    1049600       1.6
>    cassini --b---        489    0.4    3145792    4.7    3146752       4.7
>   chickpea --b---         67    0.1     524352    0.8     525312       0.8
>     lentil --b---         67    0.1     262208    0.4     263168       0.4
>    fusilli --b---        159    0.2     524352    0.8     525312       0.8
>      pizza --b---        359    0.5     524352    0.8     525312       0.8
> 
> With patch:
> 
> xentop - 13:30:17   Xen 4.18.2
> 13 domains: 1 running, 12 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
> Mem: 67030640k total, 33097788k used, 33932852k free    CPUs: 24 @ 3693MHz
>       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%)
>      xendd --b---       4020    5.7     524268    0.8     525312       0.8
>   icecream --b---       2600    3.8    4194368    6.3    4195328       6.3
>   Domain-0 -----r       1060    1.5    1048576    1.6    1048576       1.6
>       neon --b---        827    1.1    2097216    3.1    2098176       3.1
>     cheese --b---        225    0.7    1048384    1.6    1049600       1.6
>      pizza --b---        359    0.5     524352    0.8     525312       0.8
>    cassini --b---        490    0.4    3145792    4.7    3146752       4.7
>    fusilli --b---        159    0.2     524352    0.8     525312       0.8
>        bob --b---        502    0.2   16777284   25.0   16778240      25.0
>    blender --b---        121    0.2    1048640    1.6    1049600       1.6
>      bread --b---         69    0.1     524352    0.8     525312       0.8
>   chickpea --b---         67    0.1     524352    0.8     525312       0.8
>     lentil --b---         67    0.1     262208    0.4     263168       0.4


Ah, so it's the rounding, and a straight cast discards the fractional part.

I think your patch is fine, although it could do with a mention of why
this goes wrong in the commit message.  I'm happy to adjust on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 13:54:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 13:54:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721545.1125032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sc2-0002AI-4f; Tue, 14 May 2024 13:54:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721545.1125032; Tue, 14 May 2024 13:54:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sc2-0002AB-21; Tue, 14 May 2024 13:54:38 +0000
Received: by outflank-mailman (input) for mailman id 721545;
 Tue, 14 May 2024 13:54:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6sc1-0002A3-B3
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 13:54:37 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7fe7adea-11f9-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 15:54:36 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59ad344f7dso13772866b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 06:54:36 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8b634cdcsm37029066b.93.2024.05.14.06.54.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 06:54:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fe7adea-11f9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715694876; x=1716299676; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=IEl4cS4JlSGyp+2J/90G08D4D8e8CcqOq2eZS6Go0QA=;
        b=KVWfXvmk1CpKEDgbDmMOgtgIpFpqg15TPHyliG07iXVT3iVKXXuzBasrNcfnPAMNUt
         thS4h8KtMbxPfTXedNi57913qx/MH2XZTbIdhdP+mU3ismNKV8c75AfBF8DXnGQqOufv
         DgDbazCwvo2SEOISaXKNwBOiBbEDWiJG6Gy6XpAapZoSkRJzLLQmDr51JZtEVDbHD4Vv
         kRNycGRVuFsZA4qU5ZI05qewG+CYU3k4FyoD2nNwN/gDt2JfRXZtQQZx2bPpscXFH5AJ
         uDC1kW95ry6u+GYYAmkGSEw9TsQLIcuuLTMgs2vpFN0Nn5ouyW3edMbyHbB5upwKzCDc
         QwMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715694876; x=1716299676;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=IEl4cS4JlSGyp+2J/90G08D4D8e8CcqOq2eZS6Go0QA=;
        b=lxqtPkxzkb61lI+BhKdDf6zxyDhxCoddch6fSlcFg4hbYIe1HpuR1Ax+Abh9eMXAev
         ifkCBjDRrzIL+cYqMuVJvaabUdZipz+JjnUhUGGS3WvJJuj9LYRLIJr9h9XI1+8MfbG1
         S5Q0Bi2/R0E24QzgBfoA5CVVcay5ldUez0tpsGbDqmI5JopXoqhPjevP4pdqqDkmC0TB
         u6QsYP4QpOsrE7elgGMqlIfClEV5GfIcqNdL2t8JUgir9yNGt6ea1KgodLW9OFcKMHjF
         w6BdibA0E3r306UoNnxwlq5twYhg0zy6E8JeDgX59gVWPTHpjnXjNF7ouZQZTDcfl7Aa
         VzDw==
X-Forwarded-Encrypted: i=1; AJvYcCVlywG/sEecWQpNkDS81gNCh/vBVG5gK1G2hOM0OpiQRkEJo38TH+WssP7GTTpedBaeSReuCezFJTvHca+Ts4zFnu6oQxVD7ZUUtCG/aiI=
X-Gm-Message-State: AOJu0YwAJqmHT6gclo5YIiBBscxdrA+Z4tDQka5XPluIz0kDaV/LIihX
	2XN+wQIhFwKdagzlN6nwylJ+NWCj4nRDsW5/H9Xba0x7S7PhZ+86qMHFrghLrQ==
X-Google-Smtp-Source: AGHT+IEZCKfW5rdR+MQ2JYOOxzYmlIXgLz6dbQTdwxTVVu02Aksvov9kxsJKcFv5V0rWCdeiCmq/XQ==
X-Received: by 2002:a17:906:7f81:b0:a5a:86e8:4218 with SMTP id a640c23a62f3a-a5a86e84346mr94710766b.61.1715694875588;
        Tue, 14 May 2024 06:54:35 -0700 (PDT)
Message-ID: <8f183016-4117-45f8-ad75-6fb8363a0b60@suse.com>
Date: Tue, 14 May 2024 15:54:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/misra: replace R13.1 with R13.2
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 michal.orzel@amd.com, bertrand.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405101625470.2544314@ubuntu-linux-20-04-desktop>
 <369a3c75-2e71-4fb8-8373-aae737099e3b@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <369a3c75-2e71-4fb8-8373-aae737099e3b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 15:47, Jan Beulich wrote:
> On 11.05.2024 01:32, Stefano Stabellini wrote:
>> After looking closely at the R13.1 violations and potential fixes or
>> deviations, upon further reflection and discussions, we realized that it
>> is R13.2, limited to initializers list, that we want to comply with.
>>
>> Link: https://marc.info/?l=xen-devel&m=170751643325805
> 
> The mail you point at is a reply to one where I raised an aspect that was
> never really addressed: 'I'm afraid I don't understand the "eventually
> limiting" part.' Therefore I also don't follow why we'd want to limit
> applicability of this rule to just initializer lists. I'm pretty sure we
> do not want expressions the result of which or any side effects of which
> are unpredictable, no matter where such an expression is used.

Oh, and a formal aspect: I don't think an initializer list counts as
expression. Instead it's a construct involving multiple expressions,
evaluation order of which is unspecified. Specifically

unsigned arr[2] = { i++, i++ };

would violate 13.1 but satisfy 13.2, because 13.2 is concerned of
only each individual expression (and this isn't a comma expression
but a list, somewhat similar to argument lists in function calls).

Surely the compiler would already choke on this, but I wanted to use
a simple example.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:11:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:11:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721554.1125041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ss7-0005s6-G3; Tue, 14 May 2024 14:11:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721554.1125041; Tue, 14 May 2024 14:11:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6ss7-0005rz-DG; Tue, 14 May 2024 14:11:15 +0000
Received: by outflank-mailman (input) for mailman id 721554;
 Tue, 14 May 2024 14:11:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6ss6-0005qF-K5
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:11:14 +0000
Received: from mail-oo1-xc35.google.com (mail-oo1-xc35.google.com
 [2607:f8b0:4864:20::c35])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0e4ff07-11fb-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 16:11:12 +0200 (CEST)
Received: by mail-oo1-xc35.google.com with SMTP id
 006d021491bc7-5b27c5603ddso2647521eaf.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:11:11 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f185520sm53864086d6.37.2024.05.14.07.11.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 07:11:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0e4ff07-11fb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715695870; x=1716300670; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=kTbCmzldkSuVzK/l9M/kWx9NnmGiRIbnkodtilfb1LE=;
        b=ipLdMt5Y2OGYwCnbaQvzAkKOKv6IO6zyh3fiKPTymDlxrVr+5VK//vLES+VPthjwYz
         3GCNCyAwLYRUVUVF/t1TRtl2C0nUmECfRez0T4Nf23POpMb9U5k77HDmi+Cw448uJ3+y
         kON1/Odf6jtPb/yYvyn3wAUkiWHcQE1rBgyEo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715695870; x=1716300670;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kTbCmzldkSuVzK/l9M/kWx9NnmGiRIbnkodtilfb1LE=;
        b=OJa42Rdm2t5ubPUQWSeWxCRRnjHi3+fUcMXTp8IGzoJR5fZABRUHfS+knlk4ER5v4I
         mq06B2r88taVnfJr7hjeduARnJ6gKqUynqvY7BMWEvBL7ohggaCGppDCtiFyb+lrkz+6
         Y4IcVyNPrRFJr82kYaUPgi/cvWHUQPZj6dfg1dfeQrpzB7o1Z14VtDeTxOUFmPZi4cNw
         f6rOopTA6ukviB4zCg0J0lDT64oRmlKNbr0jv4X+5kxgTpCBVSah0Jheq17EToDTdgzo
         FPPPuLODNQMtasvQYeDRTNnsAGkYlBAZxoGp4C46zhWoYZDTTLwvhLdhGuWFJ8aTdTWV
         Lqow==
X-Gm-Message-State: AOJu0YwncczUXd+HeJmAiF5KYA0uCh9VwW+nwRSUxLUwdtz97gBsOTUg
	4z4x4AuEURFgE+EMXUpj6mYC3sPcEHIn13lqMLd6rWEhlTLF+UEoS70cik12dfM=
X-Google-Smtp-Source: AGHT+IF1Jzn2YwLw4PBVy0Wb4XrnA83eHVGfMyo4h7joC0rfxnmJNbb6HPtEvI9wNw/buSmPOpKHtw==
X-Received: by 2002:a05:6358:b38c:b0:186:ea4:b1a3 with SMTP id e5c5f4694b2df-193bd00b481mr1187437755d.32.1715695870358;
        Tue, 14 May 2024 07:11:10 -0700 (PDT)
Date: Tue, 14 May 2024 16:11:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 11/19] x86/setup: Leave early boot slightly
 earlier
Message-ID: <ZkNw--0ilcMh5YeB@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-12-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-12-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:38PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When we do not have a direct map, memory for metadata of heap nodes in
> init_node_heap() is allocated from xenheap, which needs to be mapped and
> unmapped on demand. However, we cannot just take memory from the boot
> allocator to create the PTEs while we are passing memory to the heap
> allocator.
> 
> To solve this race, we leave early boot slightly sooner so that Xen PTE
> pages are allocated from the heap instead of the boot allocator. We can
> do this because the metadata for the 1st node is statically allocated,
> and by the time we need memory to create mappings for the 2nd node, we
> already have enough memory in the heap allocator in the 1st node.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index bd6b1184f5..f26c9799e4 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1751,6 +1751,22 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>  
>      numa_initmem_init(0, raw_max_page);
>  
> +    /*
> +     * When we do not have a direct map, memory for metadata of heap nodes in
> +     * init_node_heap() is allocated from xenheap, which needs to be mapped and
> +     * unmapped on demand.

Hm, maybe I'm confused, but isn't xenheap memory supposed to be always
mapped when in use?  In one of the previous patches xenheap memory is
unconditionally mapped in alloc_xenheap_pages().

IMO, this would better be worded as:  "... is allocated from xenheap,
which needs to be mapped at allocation and unmapped when freed."

>                             However, we cannot just take memory from the boot
> +     * allocator to create the PTEs while we are passing memory to the heap
> +     * allocator during end_boot_allocator().

Could you elaborate here, I don't obviously see why we can't consume
memory from the boot allocator.  Is it because under certain
conditions we might try to allocate memory from the boot allocator in
order to fulfill a call to map_pages_to_xen() and find the boot
allocator empty?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:15:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:15:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721557.1125052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sw9-0006nS-W1; Tue, 14 May 2024 14:15:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721557.1125052; Tue, 14 May 2024 14:15:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6sw9-0006nL-Sg; Tue, 14 May 2024 14:15:25 +0000
Received: by outflank-mailman (input) for mailman id 721557;
 Tue, 14 May 2024 14:15:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6sw9-0006nF-0I
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:15:25 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6794a4da-11fc-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:15:24 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59a0168c75so43043966b.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:15:24 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17cc5sm719942666b.205.2024.05.14.07.15.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:15:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6794a4da-11fc-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715696123; x=1716300923; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E1GLDoU14y5eUwMiaXoQUSfTMj41T6Dyac8dRbqUddk=;
        b=cHJmjORC3XKGoWUeTSoPVj1tqc7aCJ3bVJ5M6CXee40iwtAh5QFxAwDWK+YnxB3k19
         4Zypxt56j9wak90yVRwouKTi5tdYCt/+U4rsOAre6T2ZgvxzVb0BxkM7ERtBHRWDRfXY
         98SbBx66dQNyx33IoiEy8njpPU4t5539vLl1DJ9y67E8nPv1ZFlrq61YcLXJ7G7OHXKu
         FHnLLgRvFjP/C1B2EGqchgakusMZDFUIGNTG9f6HpPVRla860ggVu0cUuJbaJ8W2gf5s
         QC4AX4ixJsCC8snSC7zqDkK4FZmubIak8pqOwkiYrgdlH0BZLEdvN2T4jDmzm/AnaAjs
         Z/Pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715696123; x=1716300923;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=E1GLDoU14y5eUwMiaXoQUSfTMj41T6Dyac8dRbqUddk=;
        b=OWlGg46BtGYePNT11yu80OaUuhIkwZDd9XV9jQ8owZtv2wKYUL5bAlRAey9LotANAA
         bm0RWYE7SpDVUluQA7vVt1sphhF8wCYSqutRozfYmYBr5hxrHrib47aVwf14OV9bsBFD
         bfKuS81d/FGGE2dgQaA12fRAKBROzryGvfwUdDI4e/gMVIMcsSZkQqTp4SeCZqGA2edc
         iLsyh6DkfUVkdJmqggvFwVNfsQ49YJaH3D01RJmGjZtvTaiabYj21WRgjXQutjBxQljo
         LaAbRvFDE/5RXYKDZQgQ+P79fIusWViCRUmXi2W3XG7uEcHtmopPdMo5Fp6kF/cghUHl
         qpEQ==
X-Forwarded-Encrypted: i=1; AJvYcCXCVz4WCWKJO5IlsmBHnneP5Z2PqeHQ9iSWshbN1xU7gCD1x/kVU7vzwZjhzEuA6V4Cd2SA0NjQbxPLRMSrhwJT50LUlnWTBog+P+BsdPE=
X-Gm-Message-State: AOJu0Yxy0flG2itPF3e93YtxPeZFURprb4Jjn+UUP5w5WdwtsISvgRE2
	SOMYnFz6jQ2DF5qRTEGUsIAK6QltmLl+myfMxevN7LKMWcVFX1nJqJ9SKWSbFQ==
X-Google-Smtp-Source: AGHT+IFzhraK0ytrSmu2exG+GvhWAovJoylge2mhnUmJLeUs4iBiERVceC8XgJW5wGqSXHuG+0HjaA==
X-Received: by 2002:a17:906:2dd5:b0:a59:c728:5430 with SMTP id a640c23a62f3a-a5a2d677dc0mr837681066b.76.1715696123383;
        Tue, 14 May 2024 07:15:23 -0700 (PDT)
Message-ID: <a93c9b9b-4066-4ffd-9c61-73301044a3ab@suse.com>
Date: Tue, 14 May 2024 16:15:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3] tools/lsevtchn: Use errno macro to handle
 hypercall error cases
Content-Language: en-US
To: Matthew Barnes <matthew.barnes@cloud.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <8bde5ee38597a86334b86822de920802483a7179.1715100071.git.matthew.barnes@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8bde5ee38597a86334b86822de920802483a7179.1715100071.git.matthew.barnes@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.05.2024 13:04, Matthew Barnes wrote:
> @@ -24,7 +25,18 @@ int main(int argc, char **argv)
>          status.port = port;
>          rc = xc_evtchn_status(xch, &status);
>          if ( rc < 0 )
> -            break;
> +        {
> +            if ( errno == ESRCH )
> +            {
> +                fprintf(stderr, "Domain ID '%d' does not correspond to valid domain.\n", domid);
> +                break;
> +            }
> +
> +            if ( errno == EINVAL )
> +                break;
> +
> +            continue;
> +        }

Hmm, I'm not sure "black listing" certain error codes is useful. I'd have
expected a "white listing" approach, special casing just EACCES and EPERM
(which iirc is what XSM would return). I'm also not convinced of the
error message text of the ESRCH case you special case: There are valid
domain IDs which still cannot be used with rcu_lock_domain_by_any_id(),
e.g. DOM_IO and DOM_XEN.

I'd be curious to hear what others think.

Andrew, ftaod - this is the patch I've mentioned in reply to your revert
touching evtchn_status(). And as mentioned there - lsevtchn should never
have outright bailed on _any_ error it gets back (i.e. even ones coming
from XSM).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:24:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:24:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721563.1125062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t4U-0008Qb-Sg; Tue, 14 May 2024 14:24:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721563.1125062; Tue, 14 May 2024 14:24:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t4U-0008QU-PI; Tue, 14 May 2024 14:24:02 +0000
Received: by outflank-mailman (input) for mailman id 721563;
 Tue, 14 May 2024 14:24:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6t4S-0008QO-Tc
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:24:00 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b038561-11fd-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:23:59 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so20607166b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:23:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17fdesm727650266b.220.2024.05.14.07.23.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:23:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b038561-11fd-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715696639; x=1716301439; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JCRGASo4IlSi0Q0MtIL47B4S6DbaI3hI8yYBGuEb0pQ=;
        b=g/rjmTK4fE6Q3Pdyxt+vKdGd/Wb8cnv2hTRT9x3tsSwN0e6EAYtjMyUkFklbSOYwgB
         pps2UEJlb8B9UeFCerulvWrbJZUu3PQaQp3SIT+lmvWQwizkG3asppzPpokf04plqpOn
         3NVNfAEyG+FtbG+JCXD56ZfAUcLIpQ96Ux++Z728nB9gKrrf3coiuHNe9pjYwYyR+CmX
         AdDf3dHIIupz+vyA4nY/XjDg6FP743XNCs7dR0X+OT9wuMsS7ORDBSgKtVZ9bWufDw+t
         L44Dfiug+cWxXh66tRI+0RYY05pkh/f+dI+8jPES0OLQbvcGWB4EKj4LCcbCQOZGIPW/
         HWqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715696639; x=1716301439;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JCRGASo4IlSi0Q0MtIL47B4S6DbaI3hI8yYBGuEb0pQ=;
        b=YzAVO1/dnR5if/G1vvBHAu0mE8lUPu7Lxzy9q7EwB8kt9oa+SVsF7gLiIdycBFXltF
         D2rH2Jz3tr8nyUyff4pgBIckTpe+5obKdv4su/+8LIgKfg79+eDnaMWVBc/cVkDiBTHl
         h2ystE/Fj9OLa6MugYat8Osy7YGQFA33RxbTpBjxJhS9OOmonvn0fjJ7GffRSXsiesTh
         IJ//Fv/YtXpLDH+6c1VFtF7C+b0102EUi1YoWB12hi+7XwP8K1PEWGBUbYpsCPZohetG
         +H19Jri13x/G7BZTxBE6hNYVfJfjDwsVd6Q8Zm/YBy1BlDkOh2Nm1PGUJZzEiW1Mclj9
         o/Cg==
X-Gm-Message-State: AOJu0Yy2OlF12vS1TZ6YIXxc/eQWvhBaXermHrtBNgzPRM3nskFxaFo/
	WdF7Fp0S066xAOU/2GsD6t+nAq+zCpOIHu8rbvO+UPpYEbQmf4xqzvP4rvuDXQ==
X-Google-Smtp-Source: AGHT+IECsrh4EtsS3we+P+JjNKQta4lQtwZkQCy2OrIz549hLCfFg5bhPnivz8WUFzZKAmFfzC9dsw==
X-Received: by 2002:a17:907:1189:b0:a59:ba2b:590b with SMTP id a640c23a62f3a-a5a2d6bed92mr874956566b.71.1715696639305;
        Tue, 14 May 2024 07:23:59 -0700 (PDT)
Message-ID: <bcae705d-ebaf-48f8-8217-ad2d917f90e2@suse.com>
Date: Tue, 14 May 2024 16:24:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2 2/3] xen/x86: enable altp2m at create domain
 domctl
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Anthony PERARD <anthony@xenproject.org>,
 Juergen Gross <jgross@suse.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-3-roger.pau@citrix.com>
 <d43a704a-fd2a-4778-9250-a69b483016b4@citrix.com> <ZjyH6f-diiqLO-0D@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjyH6f-diiqLO-0D@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09.05.2024 10:23, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 08:38:07PM +0100, Andrew Cooper wrote:
>> On 08/05/2024 12:23 pm, Roger Pau Monne wrote:
>>> Enabling it using an HVM param is fragile, and complicates the logic when
>>> deciding whether options that interact with altp2m can also be enabled.
>>>
>>> Leave the HVM param value for consumption by the guest, but prevent it from
>>> being set.  Enabling is now done using the misc_flags field in
>>> xen_arch_domainconfig.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>> ---
>>> Changes since v1:
>>>  - New in this version.
>>
>> Ha.  So this is actually work that Petr has been wanting to do.
>>
>> Petr has a series hoping to make it into 4.19 (x86: Make MAX_ALTP2M
>> configurable), which just missed out on this side of things.
>>
>> altp2m is not architecture specific at all, and there's even support for
>> ARM out on the mailing list.  Therefore, the altp2m mode wants to be
>> common, just like the new MAX_ALTP2M setting already is.
> 
> Initially I had it as a set of XEN_DOMCTL_CDF_* flags, but it wasn't
> clear to me whether the modes could be shared between arches.
> 
>> Both fields can reasonably share uint32_t, but could you work with Petr
>> to make both halfs of this land cleanly.
> 
> I'm happy for Petr to pick this patch as part of the series if he
> feels like.
> 
> I assume the plan would be to add an XEN_DOMCTL_CDF_altp2m flag, and
> then a new field to signal the mode.
> 
>>
>> As to the HVMPARAM, I'd really quite like to delete it.  It was always a
>> bodge, and there's a full set of HVMOP_altp2m_* for a guest to use.
> 
> I've assumed we must keep HVM_PARAM_ALTP2M for backwards
> compatibility.
> 
>>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>>> index 20e83cf38bbd..dff790060605 100644
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -708,13 +711,33 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>          }
>>>      }
>>>  
>>> -    if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED )
>>> +    if ( config->arch.misc_flags & ~XEN_X86_MISC_FLAGS_ALL )
>>>      {
>>>          dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n",
>>>                  config->arch.misc_flags);
>>>          return -EINVAL;
>>>      }
>>>  
>>> +    if ( altp2m && (altp2m & (altp2m - 1)) )
>>> +    {
>>> +        dprintk(XENLOG_INFO, "Multiple altp2m options selected in flags: %#x\n",
>>> +                config->flags);
>>> +        return -EINVAL;
>>
>> I think this would be clearer to follow by having a 2 bit field called
>> altp2m_mode and check for <= 2.
> 
> Don't we need 3 bits, for mixed, external and limited modes?
> 
> We could do with 2 bits if we signal altp2m enabled in a different
> field, and then introduce a field to just contain the mode.

I think what Andrew meant is

#define XEN_X86_ALTP2M_MIXED   (1U << 1)
/* Enable altp2m external mode. */
#define XEN_X86_ALTP2M_EXT     (2U << 1)
/* Enable altp2m limited mode. */
#define XEN_X86_ALTP2M_LIMITED (3U << 1)

(leaving aside the x86-only vs common aspect). That would also eliminate
the ability to request multiple (conflicting) modes.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:27:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721568.1125072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t7f-0001GA-Ay; Tue, 14 May 2024 14:27:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721568.1125072; Tue, 14 May 2024 14:27:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t7f-0001G3-6z; Tue, 14 May 2024 14:27:19 +0000
Received: by outflank-mailman (input) for mailman id 721568;
 Tue, 14 May 2024 14:27:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6t7d-0001Fx-Ie
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:27:17 +0000
Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com
 [2607:f8b0:4864:20::82b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0ffdad04-11fe-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:27:16 +0200 (CEST)
Received: by mail-qt1-x82b.google.com with SMTP id
 d75a77b69052e-43de92e236fso40396061cf.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:27:16 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e251233c6sm13445221cf.84.2024.05.14.07.27.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 07:27:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ffdad04-11fe-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715696835; x=1716301635; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Rh9kVC5CL/PTjZKOoNWg1FSAjsf22ptRuCD3BlIjHzA=;
        b=My3xIcIlqN/gX2p0T8tJsBKcqyqiuK95RsGOXdRm2pejdx60rkfhh8JzK6oN3LEF2Z
         5Q7gcijOkFYaumQEHX2YKI5stLhtjoXSAaubD/nv1qfz+iABb5koVaXsmerxSEoqvHjE
         BHNwfNw0BpJcxlRQzUpiAIhXH7msFr38KjY9M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715696835; x=1716301635;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Rh9kVC5CL/PTjZKOoNWg1FSAjsf22ptRuCD3BlIjHzA=;
        b=JOOHzjO5Z/Qa8xqwa1IjMIT7AvhHu2AfDhldph9mlopGLsaU05j1LmVoTbARP0MKFY
         Bc+v02NuB1Cdh8+lqpyufCOE64eRd6NGjvan/sg4il+yh+UYLZ8zxMokmN1vtEv90rrs
         jerakSPkwZTFd8DCXmURqEAABV3h8yUWSLUE5otW/4GcOc/Np1YMj4KtZVlTSoiAhgSK
         82GWVTLxyCh+SZlJJ6NiMkZWZ9XzSbBFp4c5tLBu8d23s2joR81Yr3CFjPebz7Fb//YZ
         a9bXABiL9/WmEQNmqv60zdtIGAPOy2ulqN5RVBBc4eLZQS35+TOkWROFUQdFH1Y+Vgp1
         vjVg==
X-Gm-Message-State: AOJu0YynYIXjHC+cH/fJiUiTSH4TbY9JgYEtOl3LGZtg/uvBQo5VGpv0
	/cragsCd5uctf+Ew5m0u3zeq1Y9+oJpH0CSPchbNJJi47zt7h6GWBMnIl+QE0nV2p6cNJUPkjjg
	r
X-Google-Smtp-Source: AGHT+IH0M00lNZ551UtLzz8kx5B1WY2R48FaCeN1Ht6YvSkEhOQy4vzqBsB8xYbLN6dDOBCwZU1OFw==
X-Received: by 2002:a05:622a:34d:b0:43d:f9b6:d5bf with SMTP id d75a77b69052e-43dfdb4e58emr150195051cf.33.1715696835357;
        Tue, 14 May 2024 07:27:15 -0700 (PDT)
Date: Tue, 14 May 2024 16:27:13 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature
 names
Message-ID: <ZkN0wXbVu-sbCNdf@macbook>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <ZkMYXSf9abR3J3FE@macbook>
 <58577656-d1c9-4f54-9526-83bdbcaa37a6@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <58577656-d1c9-4f54-9526-83bdbcaa37a6@citrix.com>

On Tue, May 14, 2024 at 02:05:10PM +0100, Andrew Cooper wrote:
> On 14/05/2024 8:53 am, Roger Pau Monné wrote:
> > On Fri, May 10, 2024 at 11:40:01PM +0100, Andrew Cooper wrote:
> >> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> >> index 6ee835b22949..2f34694e9c57 100644
> >> --- a/tools/misc/xen-cpuid.c
> >> +++ b/tools/misc/xen-cpuid.c
> >> @@ -291,6 +292,9 @@ static const struct {
> >>  
> >>  #define COL_ALIGN "24"
> >>  
> >> +static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
> >> +    INIT_FEATURE_VAL_TO_NAME;
> > I've also considered this when doing the original patch, but it seemed
> > worse to force each user of INIT_FEATURE_VAL_TO_NAME to have to
> > correctly size the array.  I would also use '* 32', as it's IMO
> > clearer and already used below when accessing the array.  I'm fine
> > if we want to go this way, but the extra Python code to add a last
> > array entry if required didn't seem that much TBH.
> 
> I was looking to avoid the other BUILD_BUG_ON()'s, and in particular
> bringing in known_features just for a build time check.
> 
> Given that there's only one instance right now, and no obvious other
> usecase, I'd say this is better.  In terms of just xen-cpuid.c, it's
> clearly correct whereas leaving it implicitly to
> INIT_FEATURE_VAL_TO_NAME is not.

If you dislike my original attempt at doing this, what about casting
the literal array initializer created by gen-cpuid.py, so that the
result ends up looking like:

#define INIT_FEATURE_NAME_ARRAY (const char *[(FEATURESET_NR_ENTRIES + 1) * 32]) { \
...

Would that be better?

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:28:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:28:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721572.1125082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t9F-0001ml-K0; Tue, 14 May 2024 14:28:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721572.1125082; Tue, 14 May 2024 14:28:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6t9F-0001me-HJ; Tue, 14 May 2024 14:28:57 +0000
Received: by outflank-mailman (input) for mailman id 721572;
 Tue, 14 May 2024 14:28:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6t9E-0001mY-BI
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:28:56 +0000
Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com
 [2607:f8b0:4864:20::f34])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4a5259fb-11fe-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 16:28:54 +0200 (CEST)
Received: by mail-qv1-xf34.google.com with SMTP id
 6a1803df08f44-69b782287f9so22399266d6.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:28:54 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1cd250sm54259666d6.98.2024.05.14.07.28.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:28:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a5259fb-11fe-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715696933; x=1716301733; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=+0vaDs9ONyd+jd6O8LXAoP+3ueJtHfSC/x11kcgHlJM=;
        b=WJfE+uY/swiJiIPNARYI5qaRXzHZv6Kh6dPblD2Jk1lNPaq0AgjIGZ/+KGs7gGtVfC
         eRWZloe2ZXKZ73drxSNBUQ3wV/SWcM3xOnGN0ndB9VmTKJvwug63lb5MEkC2Emd/YxjM
         2QfR0sfhwebG/oYkApo91e9bKDdMxNBTr0zY8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715696933; x=1716301733;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+0vaDs9ONyd+jd6O8LXAoP+3ueJtHfSC/x11kcgHlJM=;
        b=nTS7zAMx79JMcGMhWiOsjAcOowW2HHgJEe6Qp/3F1YIFGmb8qe9NS0TVDP5e3ZS2Ui
         fpwWqja5LTTtxOZ2f0c+Xf9rrk2uXoR/et3vsapRH9x7tuVq9+qDWNph6zvBXbKzuUeh
         lfxoQpVORqwReqwjkNgaKl1DfCxrcom9qZkrXyNrVy/9+QOwOqNw++T2Dmw+X2rtfjXs
         8T+Zxu8onTjKcw4e1GqfFviRBwe5YrVdcBQLI5c4uyoPncLfeo9IRbfB/yRzul9kR7iw
         zwm9A6/5+HvVfi0jXFtX7h1aixDa9ObI23owCX1sc3gTJu0nWmjQgSXl/W9JUJNsYXy2
         4DOQ==
X-Gm-Message-State: AOJu0YzEM83llP9qAQOd/xjgkJ7sw/KWDuVeA32SLDIlSSCPCCwmJGT7
	vhnSfPO5PREKHGrrdCbbaDR6oTOjZMOUZxNEKVXFe2eSFHfmOm9xDi+bf04CkII=
X-Google-Smtp-Source: AGHT+IHBQXOjytTn8+ExGS0hsGaPG4cHfG6LqY0nogLjLm4G/jAYlsVwQr8UbLH2mh+BxIsUIcftvQ==
X-Received: by 2002:a05:6214:3c9f:b0:6a0:b19d:e0a3 with SMTP id 6a1803df08f44-6a1682406dbmr161962156d6.55.1715696933257;
        Tue, 14 May 2024 07:28:53 -0700 (PDT)
Message-ID: <3ce16a4d-bff1-4a6a-8f66-62b49191fe41@citrix.com>
Date: Tue, 14 May 2024 15:28:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpufreq: Rename cpuid variable/parameters to cpu
To: Jan Beulich <jbeulich@suse.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20240511194237.2596271-1-andrew.cooper3@citrix.com>
 <alpine.DEB.2.22.394.2405131723240.2544314@ubuntu-linux-20-04-desktop>
 <36b89590-9bd9-4a64-9150-f2819bcd4a0e@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <36b89590-9bd9-4a64-9150-f2819bcd4a0e@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 9:52 am, Jan Beulich wrote:
> On 14.05.2024 02:24, Stefano Stabellini wrote:
>> On Sat, 11 May 2024, Andrew Cooper wrote:
>>> --- a/xen/drivers/cpufreq/cpufreq.c
>>> +++ b/xen/drivers/cpufreq/cpufreq.c
>>> @@ -459,21 +459,21 @@ static void print_PPC(unsigned int platform_limit)
>>>  
>>>  int set_px_pminfo(uint32_t acpi_id, struct xen_processor_performance *perf)
>>>  {
>>> -    int ret=0, cpuid;
>>> +    int ret=0, cpu;
> Would be nice if the missing blanks were added here as well, while touching
> the line.

Will adjust.

>
>>>      struct processor_pminfo *pmpt;
>>>      struct processor_performance *pxpt;
>>>  
>>> -    cpuid = get_cpu_id(acpi_id);
>>> -    if ( cpuid < 0 || !perf )
>>> +    cpu = get_cpu_id(acpi_id);
>>> +    if ( cpu < 0 || !perf )
>>>      {
>>>          ret = -EINVAL;
>>>          goto out;
>>>      }
>>>      if ( cpufreq_verbose )
>>> -        printk("Set CPU acpi_id(%d) cpuid(%d) Px State info:\n",
>>> +        printk("Set CPU acpi_id(%d) cpu(%d) Px State info:\n",
>>>                 acpi_id, cpuid);
>> This cpuid should be changed as well?
> And with that adjustment

Yes - this was a victim of a "trivial" post-test fix.  I fixed it just
after sending out the email.

> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:30:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:30:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721575.1125091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tAo-0003Nf-Ug; Tue, 14 May 2024 14:30:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721575.1125091; Tue, 14 May 2024 14:30:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tAo-0003NY-Rt; Tue, 14 May 2024 14:30:34 +0000
Received: by outflank-mailman (input) for mailman id 721575;
 Tue, 14 May 2024 14:30:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6tAn-0003NS-KH
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:30:33 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 84392850-11fe-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 16:30:31 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-792b9790f6bso461171885a.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:30:31 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792cfb853b4sm370653485a.4.2024.05.14.07.30.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:30:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84392850-11fe-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715697030; x=1716301830; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=WF+xCR9+y5gvY1IMhdEF0ZG7V/i9WeHDKOUFw54uk4Q=;
        b=tX8hxlbzRQNTa6AdZs0QrgILoPoXxt3SWjdChY7hDekFDjUvX0gJ1l4p7/iBvcaqwF
         TZ92ohi0WB8Iu0RCv6IGaVmLDi+9XEbpYet64LscSBuR4blUCqQWy+TeO0bSg+MKP5LM
         kIdK1xK2WCDsvr9DWuSP50f3fdb3kMr4Gnyao=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715697030; x=1716301830;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WF+xCR9+y5gvY1IMhdEF0ZG7V/i9WeHDKOUFw54uk4Q=;
        b=n3+qz6T6TQAZNWSYF7qdkwY7XPY9Ykcf+iN0B/hdXU04QwqqPCXY0jsvEzNZvTKbOr
         7N1NtxJ7CU2FTsoONPQ4PTxgUzTIXfQoG9D6FebBVkzEUS7Zzj/WdwFwZiKiRilQPJu8
         rNLECAUJRns5Tlt/BvWp7hCyuLwXpaNFkrHVgZToOuGJnDi/bayjlJzkBTAy4jBN4fBN
         nwyunzuJROTy/iMpI7rxaMq02Pl5sJkansZ7WcyNR5E7qAo6J/F2xbZQXCv2BAyYH58/
         odi+4cC7txoM8ukyWm9RE+JLhyvjM0pgW18OHNI52YM0uchx/GQIoMSEYKWpnT8DCg7C
         aqMQ==
X-Gm-Message-State: AOJu0Yyv7qzNyJckQg1smk9z/niKLb87nvxhEsJr7qlM+OvRctM/xdCk
	uOlYOuAMvM2AKqe5/zemeBJj3LELpkDF8donxP3gv7CvuQjEK2eP+h/khyRYDWnWi+5jSPEQ7kC
	s
X-Google-Smtp-Source: AGHT+IErB0W7W02rIKSOTPw8ftQ5B12SG7CCy3qP5V/tySJ8i0TpjIYcnE7fsUYiiwRTkfJxldQYlg==
X-Received: by 2002:a05:620a:40c1:b0:792:bd1f:b72b with SMTP id af79cd13be357-792c7600bb4mr1691302085a.72.1715697030299;
        Tue, 14 May 2024 07:30:30 -0700 (PDT)
Message-ID: <86aa4550-c183-416b-83fc-044a0836341a@citrix.com>
Date: Tue, 14 May 2024 15:30:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] tools/xen-cpuid: Use automatically generated feature
 names
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <ZkMYXSf9abR3J3FE@macbook> <58577656-d1c9-4f54-9526-83bdbcaa37a6@citrix.com>
 <ZkN0wXbVu-sbCNdf@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkN0wXbVu-sbCNdf@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 3:27 pm, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 02:05:10PM +0100, Andrew Cooper wrote:
>> On 14/05/2024 8:53 am, Roger Pau Monné wrote:
>>> On Fri, May 10, 2024 at 11:40:01PM +0100, Andrew Cooper wrote:
>>>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>>>> index 6ee835b22949..2f34694e9c57 100644
>>>> --- a/tools/misc/xen-cpuid.c
>>>> +++ b/tools/misc/xen-cpuid.c
>>>> @@ -291,6 +292,9 @@ static const struct {
>>>>  
>>>>  #define COL_ALIGN "24"
>>>>  
>>>> +static const char *const feature_names[(FEATURESET_NR_ENTRIES + 1) << 5] =
>>>> +    INIT_FEATURE_VAL_TO_NAME;
>>> I've also considered this when doing the original patch, but it seemed
>>> worse to force each user of INIT_FEATURE_VAL_TO_NAME to have to
>>> correctly size the array.  I would also use '* 32', as it's IMO
>>> clearer and already used below when accessing the array.  I'm fine
>>> if we want to go this way, but the extra Python code to add a last
>>> array entry if required didn't seem that much TBH.
>> I was looking to avoid the other BUILD_BUG_ON()'s, and in particular
>> bringing in known_features just for a build time check.
>>
>> Given that there's only one instance right now, and no obvious other
>> usecase, I'd say this is better.  In terms of just xen-cpuid.c, it's
>> clearly correct whereas leaving it implicitly to
>> INIT_FEATURE_VAL_TO_NAME is not.
> If you dislike my original attempt at doing this, what about casting
> the literal array initializer created by gen-cpuid.py, so that the
> result ends up looking like:
>
> #define INIT_FEATURE_NAME_ARRAY (const char *[(FEATURESET_NR_ENTRIES + 1) * 32]) { \
> ...
>
> Would that be better?

That will trigger -Wvla, I think.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:34:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:34:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721579.1125102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tET-0004gj-GD; Tue, 14 May 2024 14:34:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721579.1125102; Tue, 14 May 2024 14:34:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tET-0004gc-D8; Tue, 14 May 2024 14:34:21 +0000
Received: by outflank-mailman (input) for mailman id 721579;
 Tue, 14 May 2024 14:34:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tER-0004gW-QX
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:34:19 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0afddf13-11ff-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 16:34:17 +0200 (CEST)
Received: from DS7PR05CA0021.namprd05.prod.outlook.com (2603:10b6:5:3b9::26)
 by PH8PR12MB7110.namprd12.prod.outlook.com (2603:10b6:510:22e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 14:34:12 +0000
Received: from DS3PEPF000099DF.namprd04.prod.outlook.com
 (2603:10b6:5:3b9:cafe::17) by DS7PR05CA0021.outlook.office365.com
 (2603:10b6:5:3b9::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.25 via Frontend
 Transport; Tue, 14 May 2024 14:34:12 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF000099DF.mail.protection.outlook.com (10.167.17.202) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:34:11 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:34:10 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:34:09 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:34:07 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0afddf13-11ff-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bmthUKqMQFr48Wknm4hAQ0XQHwpk26eIfMCvjJSN53pzGqpMgCqm4au++4appPZkAsZKpdDLu5khOeiWiFLrNWl8Xj005GJatDXg5PDhDK9gkB/b1tn0EHO3PH5Zjyd0Cryhk0PDOM0yvY2SW8XqNsnuWyy6ijP9kyk94Cn7gCPS0QxeijEOeoMko8onEji/a4eZM/x5dyJfkT6L5T0IASDLNzGaiycozEI5pXVm5S7icdRhMi7lmQEYyDE1G2IrkH/yJ/d5paEvCM6IoNiAzVM7oxQGjYSKYk5nh0vtBH1fplHBOcUSP7w+tFvED/NKerSX4UdX06oUdQU2m68AoA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zoZFrLAdEXJwSYNJ8SgMjskivwX10Bp1KtaRk01l4qk=;
 b=AYO+yUPfjEIffC2D6jF/4gwS/ORsyXlb6QmuG1l10q6D3ZDmqrPqhV8N/3aZ0THsRLg+jluL8RWo8bZEDnhndG6vN85md0RnXHAQXFiE2bc6bbDhNecObia9Ilqvn/E2ItS9iVYi+5iBp3lSTbbA9jt/7pzCC6n1ssA6tvB/cvwiKpYIifmX6WYOgdEA6ctLN2dJ/nXGCMN7CkFn01yrlcVhA31AXSUzoqhxrn6E3fCC+s8o54fQqrJR3gbYDaOSTfAPe3WTURhD+aOBWBe3noePHKGFhLKtM1iw06NtB546SkBKp81zUU+EDRa2mxzvrPfxxbEW+yUsroe9yZw8wQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zoZFrLAdEXJwSYNJ8SgMjskivwX10Bp1KtaRk01l4qk=;
 b=Z7hzp4jiK0Jxw8tQhcEB88a+1PgJbXqW3aZATfFGVd1UVwQuTQc+lpMmwslthst9HUJmYSHrDKFBI+J8qkRyuw9NY+FpXvgLNIu/ClixqbtPkj8YVjqqoup7GiPvQKnhEWM+fv9nofsOpl+gqrTbQl3enZVmvFKKUpjqScgoQ2E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v14 0/5] PCI devices passthrough on Arm, part 3
Date: Tue, 14 May 2024 10:33:52 -0400
Message-ID: <20240514143400.152280-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099DF:EE_|PH8PR12MB7110:EE_
X-MS-Office365-Filtering-Correlation-Id: dea21605-5bf5-46ef-46e7-08dc7422ec15
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?iIupmQriX3tfPajaHjlgKwFqTMvtw4anvV3tbF5vNLvZaAluITx5q/ItxAQb?=
 =?us-ascii?Q?ZpKRJIuO8Hnx1v2HRKCECHYtxAw/lkpP2R+dSzv6nLEg9E5sh401qziNxoA1?=
 =?us-ascii?Q?jnKjp/RyT2fsX2yD9vFFGfGEAeE0g3atAfY45v+znjFlSr8hp9O7HyTq/hry?=
 =?us-ascii?Q?pG2uihtLUgPuMWjYzBzIegULv8YODuX0/NxvCR6Pd7dRD+Z5rsPFpQdDsbrl?=
 =?us-ascii?Q?cK5i5EjOGcI/4Jyhf4cSlLmZACBzVjxRxToqufiBPa58JN0C7WcXT7WPTr02?=
 =?us-ascii?Q?DRHhjoj+ubkoMhb4XDM42zn1cy1kHyParBbg3geqSlvfl2vroa+woM/e8d8r?=
 =?us-ascii?Q?vJ9bDScNl9orXJm8DVnjdfjGl/Fz0HgvyJxCHbouRAafDN07KU4AISrsV3cp?=
 =?us-ascii?Q?iokeetGVfsnV5h60HUUeUKluRSr6LEJNFNEnsdT+986yjM0sMB3yDExqwPU0?=
 =?us-ascii?Q?FtNV05MMhqrka07SdFkdWw+jg9SD44JyE5VLONOAfA6BmrKml8kIkLmqR3eh?=
 =?us-ascii?Q?0BxxQ7NaCy7E1p5YC9iHDNLYlcQ/J8UOsYhXL8zP7naq2iMjs7PF6xyuu/2E?=
 =?us-ascii?Q?fJal8132tBpe+tnQmIlB2Zum8p7t3Tq2QUgfApQXFkwdEx/pR2zlIK5C8tVK?=
 =?us-ascii?Q?XhhWkWJ4UObAS4ze4DgcujUc4qP6FYC8xtSlQhoIksqmeH6kkL4G1dieeUDH?=
 =?us-ascii?Q?3BegtLw4fVcfgxwOfmwoVRnWOmCnZk2MNB4i2Jh4iRyVNakmdENNkqJT3iSB?=
 =?us-ascii?Q?yn9yE3oYkq4AW5bXwTUrNdSBlKWSab3Vpx+fS7CC14NRXKYfbf5h6d/pCDNj?=
 =?us-ascii?Q?0Cs9GP7Lrtq7N5jRvVTxnrVbL+6Vqo7pvPe1/r9ZYtlUejfMBFxjfZa+Y97c?=
 =?us-ascii?Q?47TMmtRakJ9fMSnjU5REkDEwJH1hcNjJ8MlThfo773pFnz1jgPhnDTZuaK9K?=
 =?us-ascii?Q?0lDc5ZxEnNQqHLybq06o0k1OZ+se0KXiXgxHUfr50Kf9uj/GXuefkfIhT4GY?=
 =?us-ascii?Q?fpPMACDLAvtFwPX1njDVFSgxz7vjI9Luyr+wOYsUUysrQ2Cd6bqwEBM2o15A?=
 =?us-ascii?Q?RR2kVXsMOSklwkEGXcVDowRfEGwl13D5sKO22bc5+uiPffq/vUsM4yW0EL3V?=
 =?us-ascii?Q?fp9nbsJkpJMaJTHSrgzqvOMKAaBx1pxBDP6/sxPs52LGepUx5GY9yIlUQe01?=
 =?us-ascii?Q?/3U1HHjyot23oZSl8kjlDi6Ow6qX+u9hJTA45nbkR7SHe132CRi1IEyBlaue?=
 =?us-ascii?Q?DJ3mGiQiB2sJ+HZ/AaZOzJKaI1wI2w3j2GhZZNPnJcGyYeV9IU1xCn6+n2Lr?=
 =?us-ascii?Q?UDJU1+zq80ZPIHv88PG2mOkV5f+y0JeMVXdTRklkE29yajWGrG0Nkbgloywb?=
 =?us-ascii?Q?SeQ1C0NoYvuXuYxhnjMuK04BUl0s?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:34:11.7201
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dea21605-5bf5-46ef-46e7-08dc7422ec15
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099DF.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7110

This is next version of vPCI rework. Aim of this series is to prepare
ground for introducing PCI support on ARM platform.

in v14:
 - drop first 9 patches as they were committed
 - updated ("vpci/header: emulate PCI_COMMAND register for guests")

in v13:
 - drop ("xen/arm: vpci: permit access to guest vpci space") as it was
   unnecessary

in v12:
 - I (Stewart) coordinated with Volodomyr to send this whole series. So,
   add my (Stewart) Signed-off-by to all patches.
 - The biggest change is to re-work the PCI_COMMAND register patch.
   Additional feedback has also been addressed - see individual patches.
 - Drop ("pci: msi: pass pdev to pci_enable_msi() function") and
   ("pci: introduce per-domain PCI rwlock") as they were committed
 - Rename ("rangeset: add rangeset_empty() function")
       to ("rangeset: add rangeset_purge() function")
 - Rename ("vpci/header: rework exit path in init_bars")
       to ("vpci/header: rework exit path in init_header()")

in v11:
 - Added my (Volodymyr) Signed-off-by tag to all patches
 - Patch "vpci/header: emulate PCI_COMMAND register for guests" is in
   intermediate state, because it was agreed to rework it once Stewart's
   series on register handling are in.
 - Addressed comments, please see patch descriptions for details.

in v10:

 - Removed patch ("xen/arm: vpci: check guest range"), proper fix
   for the issue is part of ("vpci/header: emulate PCI_COMMAND
   register for guests")
 - Removed patch ("pci/header: reset the command register when adding
   devices")
 - Added patch ("rangeset: add rangeset_empty() function") because
   this function is needed in ("vpci/header: handle p2m range sets
   per BAR")
 - Added ("vpci/header: handle p2m range sets per BAR") which addressed
   an issue discovered by Andrii Chepurnyi during virtio integration
 - Added ("pci: msi: pass pdev to pci_enable_msi() function"), which is
   prereq for ("pci: introduce per-domain PCI rwlock")
 - Fixed "Since v9/v8/... " comments in changelogs to reduce confusion.
   I left "Since" entries for older versions, because they were added
   by original author of the patches.

in v9:

v9 includes addressed commentes from a previous one. Also it
introduces a couple patches from Stewart. This patches are related to
vPCI use on ARM. Patch "vpci/header: rework exit path in init_bars"
was factored-out from "vpci/header: handle p2m range sets per BAR".

in v8:

The biggest change from previous, mistakenly named, v7 series is how
locking is implemented. Instead of d->vpci_rwlock we introduce
d->pci_lock which has broader scope, as it protects not only domain's
vpci state, but domain's list of PCI devices as well.

As we discussed in IRC with Roger, it is not feasible to rework all
the existing code to use the new lock right away. It was agreed that
any write access to d->pdev_list will be protected by **both**
d->pci_lock in write mode and pcidevs_lock(). Read access on other
hand should be protected by either d->pci_lock in read mode or
pcidevs_lock(). It is expected that existing code will use
pcidevs_lock() and new users will use new rw lock. Of course, this
does not mean that new users shall not use pcidevs_lock() when it is
appropriate.

Changes from previous versions are described in each separate patch.

Oleksandr Andrushchenko (4):
  vpci/header: emulate PCI_COMMAND register for guests
  vpci: add initial support for virtual PCI bus topology
  xen/arm: translate virtual PCI bus topology for guests
  xen/arm: account IO handlers for emulated PCI MSI-X

Volodymyr Babchuk (1):
  arm/vpci: honor access size when returning an error

 xen/arch/arm/vpci.c        | 63 +++++++++++++++++++++++------
 xen/drivers/Kconfig        |  4 ++
 xen/drivers/vpci/header.c  | 60 +++++++++++++++++++++++++---
 xen/drivers/vpci/msi.c     |  8 ++++
 xen/drivers/vpci/msix.c    |  7 ++++
 xen/drivers/vpci/vpci.c    | 81 ++++++++++++++++++++++++++++++++++++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/sched.h    | 10 ++++-
 xen/include/xen/vpci.h     | 27 +++++++++++++
 9 files changed, 242 insertions(+), 19 deletions(-)


base-commit: 46aa3031ae89ac1771f4159972edab65710e7349
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:34:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:34:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721580.1125112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tEe-0004y2-N2; Tue, 14 May 2024 14:34:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721580.1125112; Tue, 14 May 2024 14:34:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tEe-0004xv-KM; Tue, 14 May 2024 14:34:32 +0000
Received: by outflank-mailman (input) for mailman id 721580;
 Tue, 14 May 2024 14:34:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tEd-0004xK-7p
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:34:31 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 11bd1953-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:34:29 +0200 (CEST)
Received: from BY5PR16CA0033.namprd16.prod.outlook.com (2603:10b6:a03:1a0::46)
 by DM6PR12MB4057.namprd12.prod.outlook.com (2603:10b6:5:213::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 14:34:25 +0000
Received: from CO1PEPF000066EB.namprd05.prod.outlook.com
 (2603:10b6:a03:1a0:cafe::42) by BY5PR16CA0033.outlook.office365.com
 (2603:10b6:a03:1a0::46) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 14:34:25 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000066EB.mail.protection.outlook.com (10.167.249.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:34:24 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:34:23 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:34:21 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11bd1953-11ff-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T9xWYeEt/y4M2g1Rse67Wb5ca6ptMjIns5y2zTje/+qy5Spkml49G3GuqYnRSrIyx2OR86fKLTW1XqjMeoANHt4rSJqoVUrktFbfoxwop0ZtTZKrp+WJX34nEU/mQqxPS3viCqLdrVz5YjnfK7k5niWdV08SZEjtXfcw6/r0JcpyPFBmOulboPSkmufmiU1y5KNvO89slSvo7RJrtLQG4kGN2nZAaQbjGmErPTxQBAwnMg/YgwIqqonNUjh0q3bJ4ZkuAwT97MWOqC4eTFcxkU5IxYGhQP+anlDt4a1IDR88rE6yS3lhrB4znwyjH7/Bxt6cPe+S5CAoC68T62OyVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zUwBnsMoww8f8q0OFnkTt7/zeC8tzJGFYO6woXqxVXg=;
 b=VWgoFIpP8ruzJEm2eTp2Vo315U4r7BOWxnZmw+xOCyNB/gexT0Pdg1ft+yRnlhzR1LY8DXWUrEL2kkiThUrhoAkZEDDE5ZIKgGSRTTuvojYI0297veqo4uG0iJFtHGig8I704jQHv3gDYqPCodYonBXSq3c1jAZsQwgRULlYzajAfYfBSQcgYMfNtK69DF3URVHUWjqNgiQT/ej+Kj24dM1EOa+DksFbIbFAgm2QAgvwfEa/VXjiKc1lqhfX9ZS6Zcl7T1jTYXX8jD+8xY6HFibyDqjw1i0uy79Ubj03JKPMmf2TEBbaXp0ykksIaeIvzR53m8nLJs16Y5yyfw+sAA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zUwBnsMoww8f8q0OFnkTt7/zeC8tzJGFYO6woXqxVXg=;
 b=UwfE0G3R4vCNPgoz90RetIJrgQ/ceeqNn0fW2CU9UYs8//3Mwek02W9q8fLK5qfePXp+yHrmsX5Bzvd/cFtL2C5ccHpwC1Y3GGLXL+9dOTaCCbL3zyd69y8DF2pZrT9OHDHhCwMLhtQn7VNzcbcQZ+/y1IrhdoUtD1Ycrk98kxM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v14 1/5] vpci/header: emulate PCI_COMMAND register for guests
Date: Tue, 14 May 2024 10:33:53 -0400
Message-ID: <20240514143400.152280-2-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240514143400.152280-1-stewart.hildebrand@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EB:EE_|DM6PR12MB4057:EE_
X-MS-Office365-Filtering-Correlation-Id: 364e5c62-d5e9-4e98-1028-08dc7422f3bb
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?GtlWqU/re/zNBjVbhxZxiAoBPlVmZCANJkh6QxhvBUeEiljp51+ct6ltLQQS?=
 =?us-ascii?Q?Y/I3wKJmgjyeArBM1eUrJey6uMkMw9pnqhLewfpGhvvCmWKev+6D0/QiwWZ/?=
 =?us-ascii?Q?o7xKXsURKAH8Kfi8+ZIHdl2eZ7cyPh3BIO+gzh+C81qk3253a6Ru1veL9jPQ?=
 =?us-ascii?Q?E8RzSmH4XmIcDrkmJ5RYRmuIFaSh4tyuDIIjwfiyj//QpHFiut2fi80GLf8s?=
 =?us-ascii?Q?YBIQ0DM06cwiFPGpdBhH9L1C17SBJjNdFqgo1wYieYbntMLo6lwsybFf+8Fk?=
 =?us-ascii?Q?XdizuwkQkWeShl43Mgodr0lf6cXbFNd1NgzYp/P5xeGlq4xJBxb1VyWGry+E?=
 =?us-ascii?Q?JVa6HSAVsBgmE6xaPgCnBoc10iFexgoLmM8ThWWW7WgG+pRvT3fxW7+r+5K4?=
 =?us-ascii?Q?IqG1iacaeUhjZLJ8x2qvNyECViHMJHPv/TcGrm9kFErkTE2soZlJ7i2tOrLS?=
 =?us-ascii?Q?RjKOfHBsx3K0v4blxX7lPNpve92zyGVe5NgNvexsgNR4x7hNWzbilfEn4IJS?=
 =?us-ascii?Q?WAz1xq2CxvkvPikdhg35mdryg+0IquP63HcYk70/F60pucrkxQc3OykOfw1v?=
 =?us-ascii?Q?VTDbV8Ph1EknQ/Pvp5RwHZAymoOSCapTjmAFRuKkK/XgBPCdTZKFfEFmzuGK?=
 =?us-ascii?Q?ByMjHJdLRxy4EVAYqFFG1+ZeM+Aht6FGnFAcwO8lgKglHrWDs+acyUf7yABn?=
 =?us-ascii?Q?bfcNHcZlb9ntPr5hobgmhYQjLOIMDzPW8xntdYhGcA/s5UvQ83AWf+EMnGZt?=
 =?us-ascii?Q?mHS3c/++XJcYfh5QpwL6vSxdpDgZrFYWFFHTGnpp6MJHr0pcX+LNkVcj6wP3?=
 =?us-ascii?Q?2v2aE11hMiCCaeTvcJpHHqISmSt+jmqpWhtt4uB/ccDyVzvBjRmUHOW0G2YT?=
 =?us-ascii?Q?GoRXEGBLu+nPkuExaF+R23mL5Lw+1a0i6vIDaVc31UbB+wona8tMN01TBIKT?=
 =?us-ascii?Q?+YUbQGRoULMsCqdt2G3ojZ38eE5hlSk3tkARPPgmMtnB8L8g2NmDU1OtGQxu?=
 =?us-ascii?Q?bkbn5OSYJs9HIiH9pjLhCf18588/dxmaZiyS7/wKdC7lTa240C5h0nFLnqlH?=
 =?us-ascii?Q?IDDsO7za63NdwgxIefiHM3ma5f2GpTnyZJsDyZluoWB2cXlLGgdeeERWslMn?=
 =?us-ascii?Q?6ZR3ovikf0YWVFHceuXL9PrSvTXuiOKI3cTRP8SYQ9fXjD/mCMXYUuYuyfMM?=
 =?us-ascii?Q?wrZ0s0oyJGrvSgQ3x0vHZrRxVlADcZZyjtnQv3B3/FDKc9Ur59DDC810zw8y?=
 =?us-ascii?Q?UOrEeKQ7x9W1KqR7T8+j/bFI6e8lFouplEE8kVhyRfVD/0RhkTQgS/xOKx+9?=
 =?us-ascii?Q?yrFqgwowzTKvHIFahUHGVDnW?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:34:24.5194
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 364e5c62-d5e9-4e98-1028-08dc7422f3bb
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4057

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
guest's (domU) view of this will want to be zero (for now), the host
having set it to 1 should be preserved, or else we'd effectively be
giving the domU control of the bit. Thus, PCI_COMMAND register needs
proper emulation in order to honor host's settings.

According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
Device Control" the reset state of the command register is typically 0,
so when assigning a PCI device use 0 as the initial state for the
guest's (domU) view of the command register.

Here is the full list of command register bits with notes about
PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
also documented here since that is our current reference implementation
for PCI passthrough.

PCI_COMMAND_IO (bit 0)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if an I/O BAR is exposed to the guest.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
    don't yet support I/O BARs for domUs.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_MEMORY (bit 1)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if a Memory BAR is exposed to the guest.
  Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
    regions.
  Xen domU: For devices assigned to DomUs, memory decoding will be
    disabled at the time of initialization.

PCI_COMMAND_MASTER (bit 2)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_SPECIAL (bit 3)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_INVALIDATE (bit 4)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_VGA_PALETTE (bit 5)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_PARITY (bit 6)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_WAIT (bit 7)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: hardwire to 0
  QEMU: res_mask
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_SERR (bit 8)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_FAST_BACK (bit 9)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_INTX_DISABLE (bit 10)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU checks if INTx was mapped
    for a device. If it is not, then guest can't control
    PCI_COMMAND_INTX_DISABLE bit.
  Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
  Xen dom0: We allow dom0 to control this bit freely.

Bits 11-15
  PCIe 6.1: RsvdP
  PCI LB 3.0: Reserved
  QEMU: res_mask
  Xen domU/dom0: rsvdp_mask

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
RFC: There is an unaddressed question for Roger: should we update the
     guest view of the PCI_COMMAND_INTX_DISABLE bit in
     msi.c/msix.c:control_write()? See prior discussion at [1].
     In my opinion, I think we should make sure that hardware state and
     the guest view are consistent (i.e. don't lie to the guest).

[1] https://lore.kernel.org/xen-devel/86b25777-788c-4b9a-8166-a6f8174bedc9@suse.com/

In v14:
- check for 0->1 transition in INTX_DISABLE-setting logic in
  msi.c:control_write() to match msix.c:control_write()
- clear domU-controllable bits in header.c:init_header()

In v13:
- Update right away (don't defer) PCI_COMMAND_MEMORY bit in guest_cmd
  variable in cmd_write()
- Make comment single line in xen/drivers/vpci/msi.c:control_write()
- Rearrange memory decoding disabling snippet in init_header()

In v12:
- Rework patch using vpci_add_register_mask()
- Add bitmask #define in pci_regs.h according to PCIe 6.1 spec, except
  don't add the RO bits because they were RW in PCI LB 3.0 spec.
- Move and expand TODO comment about properly emulating bits
- Update guest_cmd in msi.c/msix.c:control_write()
- Simplify cmd_write(), thanks to rsvdp_mask
- Update commit description

In v11:
- Fix copy-paste mistake: vpci->msi should be vpci->msix
- Handle PCI_COMMAND_IO
- Fix condition for disabling INTx in the MSI-X code
- Show domU changes to only allowed bits
- Show PCI_COMMAND_MEMORY write only after P2M was altered
- Update comments in the code
In v10:
- Added cf_check attribute to guest_cmd_read
- Removed warning about non-zero cmd
- Updated comment MSI code regarding disabling INTX
- Used ternary operator in vpci_add_register() call
- Disable memory decoding for DomUs in init_bars()
In v9:
- Reworked guest_cmd_read
- Added handling for more bits
Since v6:
- fold guest's logic into cmd_write
- implement cmd_read, so we can report emulated INTx state to guests
- introduce header->guest_cmd to hold the emulated state of the
  PCI_COMMAND register for guests
Since v5:
- add additional check for MSI-X enabled while altering INTX bit
- make sure INTx disabled while guests enable MSI/MSI-X
Since v3:
- gate more code on CONFIG_HAS_MSI
- removed logic for the case when MSI/MSI-X not enabled
---
 xen/drivers/vpci/header.c  | 60 ++++++++++++++++++++++++++++++++++----
 xen/drivers/vpci/msi.c     |  8 +++++
 xen/drivers/vpci/msix.c    |  7 +++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/vpci.h     |  3 ++
 5 files changed, 73 insertions(+), 6 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 47648c395132..2491dbae8901 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -524,9 +524,21 @@ static void cf_check cmd_write(
 {
     struct vpci_header *header = data;
 
+    if ( !is_hardware_domain(pdev->domain) )
+    {
+        const struct vpci *vpci = pdev->vpci;
+
+        if ( (vpci->msi && vpci->msi->enabled) ||
+             (vpci->msix && vpci->msix->enabled) )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+
+        header->guest_cmd = cmd;
+    }
+
     /*
      * Let Dom0 play with all the bits directly except for the memory
-     * decoding one.
+     * decoding one. Bits that are not allowed for DomU are already
+     * handled above and by the rsvdp_mask.
      */
     if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) )
         /*
@@ -540,6 +552,14 @@ static void cf_check cmd_write(
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static uint32_t cf_check guest_cmd_read(
+    const struct pci_dev *pdev, unsigned int reg, void *data)
+{
+    const struct vpci_header *header = data;
+
+    return header->guest_cmd;
+}
+
 static void cf_check bar_write(
     const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
 {
@@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    /*
+     * Setup a handler for the command register.
+     *
+     * TODO: If support for emulated bits is added, re-visit how to handle
+     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
+     */
+    rc = vpci_add_register_mask(pdev->vpci,
+                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
+                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
+                                PCI_COMMAND_RSVDP_MASK |
+                                    (is_hwdom ? 0
+                                              : PCI_COMMAND_IO |
+                                                PCI_COMMAND_PARITY |
+                                                PCI_COMMAND_WAIT |
+                                                PCI_COMMAND_SERR |
+                                                PCI_COMMAND_FAST_BACK),
+                                0);
     if ( rc )
         return rc;
 
@@ -836,9 +870,23 @@ static int cf_check init_header(struct pci_dev *pdev)
     if ( pdev->ignore_bars )
         return 0;
 
-    /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
-    if ( cmd & PCI_COMMAND_MEMORY )
+
+    /*
+     * For DomUs, clear PCI_COMMAND_{MASTER,MEMORY,IO} and other
+     * DomU-controllable bits in PCI_COMMAND. Devices assigned to DomUs will
+     * start with memory decoding disabled, and modify_bars() will not be called
+     * at the end of this function.
+     */
+    if ( !is_hwdom )
+        cmd &= ~(PCI_COMMAND_VGA_PALETTE | PCI_COMMAND_INVALIDATE |
+                 PCI_COMMAND_SPECIAL | PCI_COMMAND_MASTER | PCI_COMMAND_MEMORY |
+                 PCI_COMMAND_IO);
+
+    header->guest_cmd = cmd;
+
+    /* Disable memory decoding before sizing. */
+    if ( !is_hwdom || (cmd & PCI_COMMAND_MEMORY) )
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY);
 
     for ( i = 0; i < num_bars; i++ )
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index 30adcf7df05d..3e414e69a432 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -57,6 +57,7 @@ static void cf_check control_write(
 
     if ( new_enabled )
     {
+        bool old_enabled = msi->enabled;
         /*
          * If the device is already enabled it means the number of
          * enabled messages has changed. Disable and re-enable the
@@ -70,6 +71,13 @@ static void cf_check control_write(
 
         if ( vpci_msi_arch_enable(msi, pdev, vectors) )
             return;
+
+        /* Make sure domU doesn't enable INTx while enabling MSI. */
+        if ( !old_enabled && !is_hardware_domain(pdev->domain) )
+        {
+            pci_intx(pdev, false);
+            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
     }
     else
         vpci_msi_arch_disable(msi, pdev);
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index 58c16ebdf283..fbe710ab92ef 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -135,6 +135,13 @@ static void cf_check control_write(
         }
     }
 
+    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
+    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
+    {
+        pci_intx(pdev, false);
+        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+    }
+
     msix->masked = new_masked;
     msix->enabled = new_enabled;
 
diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h
index 0bc18efabb74..250ba106dbd3 100644
--- a/xen/include/xen/pci_regs.h
+++ b/xen/include/xen/pci_regs.h
@@ -48,6 +48,7 @@
 #define  PCI_COMMAND_SERR	0x100	/* Enable SERR */
 #define  PCI_COMMAND_FAST_BACK	0x200	/* Enable back-to-back writes */
 #define  PCI_COMMAND_INTX_DISABLE 0x400 /* INTx Emulation Disable */
+#define  PCI_COMMAND_RSVDP_MASK	0xf800
 
 #define PCI_STATUS		0x06	/* 16 bits */
 #define  PCI_STATUS_IMM_READY	0x01	/* Immediate Readiness */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 6e4c972f35ed..2064d9354f5b 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -107,6 +107,9 @@ struct vpci {
         } bars[PCI_HEADER_NORMAL_NR_BARS + 1];
         /* At most 6 BARS + 1 expansion ROM BAR. */
 
+        /* Guest (domU only) view of the PCI_COMMAND register. */
+        uint16_t guest_cmd;
+
         /*
          * Store whether the ROM enable bit is set (doesn't imply ROM BAR
          * is mapped into guest p2m) if there's a ROM BAR on the device.
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:34:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:34:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721582.1125122 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tEp-0005Kl-4s; Tue, 14 May 2024 14:34:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721582.1125122; Tue, 14 May 2024 14:34:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tEp-0005Ke-1B; Tue, 14 May 2024 14:34:43 +0000
Received: by outflank-mailman (input) for mailman id 721582;
 Tue, 14 May 2024 14:34:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tEo-0004xK-IB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:34:42 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 18ab5345-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:34:41 +0200 (CEST)
Received: from MW4PR03CA0105.namprd03.prod.outlook.com (2603:10b6:303:b7::20)
 by SN7PR12MB6911.namprd12.prod.outlook.com (2603:10b6:806:261::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 14:34:36 +0000
Received: from CO1PEPF000066E8.namprd05.prod.outlook.com
 (2603:10b6:303:b7:cafe::a) by MW4PR03CA0105.outlook.office365.com
 (2603:10b6:303:b7::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 14:34:36 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000066E8.mail.protection.outlook.com (10.167.249.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:34:35 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:34:34 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:34:32 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18ab5345-11ff-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=C86yrNUpMdZq+nJsTt1XuYh0Fw3MoV5MPVmkSU+eSZd3qBjQicV3Aj1B+FAh1UqEDM6EXvBnhYyLFL+Aeo3ETXB0iZgrifM0wsnbmeHswoilLKYG5fK/coeMSfXZ7msjPZ+NlWAHA5MoJW6ubZTjGWXTnzJkgx9+fWviRuHrqIVQyLDoRdEVW2dDBqEbbGp1LjQKXRSMG01fL0Mq23ZbiB4RJJ00yDef4G6GfWobaVNg5A1ju7deJjFOK7hDzjMX6uwr4Fj+Og3wiTy8/3bWTgLYoCc0hAW/qwjfAnLv785vfJ/K97wfAtjk01lgZEiOfwfgi34ICWesOmxfqkYB5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=snqZUZKvXcG+0ruyZzznMKpfTsrySOP3X8wsZHFKOLM=;
 b=P6CdCkh81NS3syoOwydVTeCj8ee0kUJRzIGeDnHBWhwgoN2CdATaH7JO6aXvudqM6BjidbOfj9zyCVDUWLtei98fVo9i3HixE2ecoMipHlrTx52pRotgsclYJI3Fh8yyAk9A939XCSN5Mwm3ZkSgaU6kU/8g3U7AAXhejAt7zVHkews1oLh4hrkl9AOou2XooHOxHE31/9ORTIU1gUwMOJT1YWPu8ek9IxoujMDFx/rZDHahhWxfdkukKBNLqWtZ5jIG/cJ3mMfaXfLdTttQe4l18XbIcXrgqE1PcBZCvPtphY243yZvAgh+A2OkDe17Qwz63gh/v/rsT6x2V+Uxdg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=snqZUZKvXcG+0ruyZzznMKpfTsrySOP3X8wsZHFKOLM=;
 b=0TzH70P4RSpzIkBQp5+G6QXIJwjSp0cGsz4MrfjkZNOWHfioeURCRq0MgOdqva5UkKsqwBnLeSdrW+uO0gdqOAmzh5xDsGAYVdtEvj0bkxbO5xmXEXSVO6NaA5792OW94WttjLqoIQftPUVdqoqxPdf86zyFa/rJF/qaTeBq0ww=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v14 2/5] vpci: add initial support for virtual PCI bus topology
Date: Tue, 14 May 2024 10:33:54 -0400
Message-ID: <20240514143400.152280-3-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240514143400.152280-1-stewart.hildebrand@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066E8:EE_|SN7PR12MB6911:EE_
X-MS-Office365-Filtering-Correlation-Id: e10c92a1-2e7c-4b8d-d914-08dc7422fa4f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?IVwIASX5u+qR4utHccDLFQXnE7EQsaXNuJJvMcW01gKTsrAeVg+VwWsFk7S1?=
 =?us-ascii?Q?ASnNDy9K8ELwSgG7qHzfC+RsPbOjXobOf4pIf1wPx6Q+phTCIDUoq7NT8p3Q?=
 =?us-ascii?Q?HYJ1zYbMpr/nVIQ5/DsRD5GkglS2i1s6KjLCzZmNJPkzvf27DZBRPhf3L/Cc?=
 =?us-ascii?Q?8dCemJ5FOnSgrb4kVDDLEvYCasMfrWGrAdegQ3VF/gfRhfE+RrLQNShU4l9o?=
 =?us-ascii?Q?zuefA6JS9aJ/FCGMNWVnexC/cK39oPUszlTf3zIKm43aKJ+G5MQDzcglxKOL?=
 =?us-ascii?Q?1qe7UP6pKDON0AO41qtNcu+ZhEb19scbSuLPMOJoNFAByjcQxtcsCmQOmtjA?=
 =?us-ascii?Q?08Bk1cadOcLaXUFq9iFr5D7RBV5FDqyR2YzGNIJSOiPODqgbRfUCU8Zzerlb?=
 =?us-ascii?Q?7VIMlADsS41fgaIyl1abfB8EjNagV+N9pyHcptHVjHOusapZfnhVkeOJTCev?=
 =?us-ascii?Q?CbaZhACPLgwb0PKBE0yFA80NXW/3bWIdrXUNouDVu4PNOAYKcNXPi00mjru8?=
 =?us-ascii?Q?qrx3aCJ4Y4JFq3wiJSqomc2DtJjgA4Q1hzRkFRpcb5qoXr/Sk2fmUdP84SXN?=
 =?us-ascii?Q?Q31xxqS0TtTqHCHXLUo0x1PBQBaZiditXGHNrfKTnd8dvdF+wWcOCEWXbOWD?=
 =?us-ascii?Q?C+lBEDTAy4+ckcJKDZWj1fqOXUR6MCVcWLmJ6bXzasBVIFb7yzJNYjY/PP4T?=
 =?us-ascii?Q?2BwSWByxNFtnLcyGRf4We3ZZ1kYG9FVo0R5K/HuR5ddSqqyUq9lpF5Te7ZdC?=
 =?us-ascii?Q?JvKeZJ6jKl1ncJu14RHCbydrMpMtpwdR7UpzU7VFGTBkqB4vL5+yJjvODMxa?=
 =?us-ascii?Q?nbyiTw5NviUjWRr5jfktifNs1kFbTwMucI9kgbbJmNWRrnq6eXXhN+KxrX52?=
 =?us-ascii?Q?Jis91/kGxvK1LWukTTOEJpi2QcJXyEN3p9xPbNIGTXQklKzATho1RxwK3bFm?=
 =?us-ascii?Q?KZTmv0qUfeHFdjponjJVB6dOJ9HVMLDWuajvLJKtfy2d/sbtnKe4Ese0cXc4?=
 =?us-ascii?Q?GKhoaAw/2qHfYGNoOzLH1e7uPTL9ybb0Y8kpZfuBYlaOzQnOwSWba4Hp+uvK?=
 =?us-ascii?Q?KSlpbLofmY4CHijO7mABJBdnJ62UHUdn2VuyGD0RnwTEbHUfoUFjzZy4GPIh?=
 =?us-ascii?Q?UccAFIyzgIWK8yaW4RRrPSvkfe9s+coD40KMWlyd6xUSIf8Vr1lCRnLSEjg1?=
 =?us-ascii?Q?/vNbzO5pVvuNDyo5jYWolcDL0zrhphtiwpulO7188oL5NwV9jAiUjys0XAYH?=
 =?us-ascii?Q?6qNS2u3jyiS/K5V4+PQquuJfx38Tqi7BzVTaZiY64rZHqo+As9vpFx07awU8?=
 =?us-ascii?Q?bRTLujb+orULpK2hPI6KFg0gdneV/ImkOVItcpB3Th5gIaKyuWMOuqiIHhR2?=
 =?us-ascii?Q?Vqkju58=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:34:35.5296
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e10c92a1-2e7c-4b8d-d914-08dc7422fa4f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066E8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6911

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Please note, that at the moment only function 0 of a multifunction
device can be passed through.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v13:
- s/depends on/select/ in Kconfig
- check pdev->sbdf.fn instead of two booleans in add_virtual_device()
- comment #endifs in sched.h
- clarify comment about limits in vpci.h with seg/bus limit
In v11:
- Fixed code formatting
- Removed bogus write_unlock() call
- Fixed type for new_dev_number
In v10:
- Removed ASSERT(pcidevs_locked())
- Removed redundant code (local sbdf variable, clearing sbdf during
device removal, etc)
- Added __maybe_unused attribute to "out:" label
- Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the
  first patch where it is used (previously was in "vpci: add hooks for
  PCI device assign/de-assign")
In v9:
- Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart)
In v8:
- Added write lock in add_virtual_device
Since v6:
- re-work wrt new locking scheme
- OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device()
Since v5:
- s/vpci_add_virtual_device/add_virtual_device and make it static
- call add_virtual_device from vpci_assign_device and do not use
  REGISTER_VPCI_INIT machinery
- add pcidevs_locked ASSERT
- use DECLARE_BITMAP for vpci_dev_assigned_map
Since v4:
- moved and re-worked guest sbdf initializers
- s/set_bit/__set_bit
- s/clear_bit/__clear_bit
- minor comment fix s/Virtual/Guest/
- added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used
  later for counting the number of MMIO handlers required for a guest
  (Julien)
Since v3:
 - make use of VPCI_INIT
 - moved all new code to vpci.c which belongs to it
 - changed open-coded 31 to PCI_SLOT(~0)
 - added comments and code to reject multifunction devices with
   functions other than 0
 - updated comment about vpci_dev_next and made it unsigned int
 - implement roll back in case of error while assigning/deassigning devices
 - s/dom%pd/%pd
Since v2:
 - remove casts that are (a) malformed and (b) unnecessary
 - add new line for better readability
 - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
    functions are now completely gated with this config
 - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/drivers/Kconfig     |  4 +++
 xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched.h | 10 +++++++-
 xen/include/xen/vpci.h  | 12 +++++++++
 4 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..20050e9bb8b3 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	select HAS_VPCI
+
 endmenu
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 97e115dc5798..23722634d50b 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+static int add_virtual_device(struct pci_dev *pdev)
+{
+    struct domain *d = pdev->domain;
+    unsigned int new_dev_number;
+
+    if ( is_hardware_domain(d) )
+        return 0;
+
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
+    /*
+     * Each PCI bus supports 32 devices/slots at max or up to 256 when
+     * there are multi-function ones which are not yet supported.
+     */
+    if ( pdev->sbdf.fn )
+    {
+        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
+                 &pdev->sbdf);
+        return -EOPNOTSUPP;
+    }
+    new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map,
+                                         VPCI_MAX_VIRT_DEV);
+    if ( new_dev_number == VPCI_MAX_VIRT_DEV )
+        return -ENOSPC;
+
+    __set_bit(new_dev_number, &d->vpci_dev_assigned_map);
+
+    /*
+     * Both segment and bus number are 0:
+     *  - we emulate a single host bridge for the guest, e.g. segment 0
+     *  - with bus 0 the virtual devices are seen as embedded
+     *    endpoints behind the root complex
+     *
+     * TODO: add support for multi-function devices.
+     */
+    pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0);
+
+    return 0;
+}
+
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 void vpci_deassign_device(struct pci_dev *pdev)
 {
     unsigned int i;
@@ -49,6 +92,12 @@ void vpci_deassign_device(struct pci_dev *pdev)
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    if ( pdev->vpci->guest_sbdf.sbdf != ~0 )
+        __clear_bit(pdev->vpci->guest_sbdf.dev,
+                    &pdev->domain->vpci_dev_assigned_map);
+#endif
+
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
     {
@@ -103,6 +152,13 @@ int vpci_assign_device(struct pci_dev *pdev)
     INIT_LIST_HEAD(&pdev->vpci->handlers);
     spin_lock_init(&pdev->vpci->lock);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    pdev->vpci->guest_sbdf.sbdf = ~0;
+    rc = add_virtual_device(pdev);
+    if ( rc )
+        goto out;
+#endif
+
     for ( i = 0; i < NUM_VPCI_INIT; i++ )
     {
         rc = __start_vpci_array[i](pdev);
@@ -110,6 +166,7 @@ int vpci_assign_device(struct pci_dev *pdev)
             break;
     }
 
+ out: __maybe_unused;
     if ( rc )
         vpci_deassign_device(pdev);
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b84199598..2dcd1d1a4f8a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -486,7 +486,15 @@ struct domain
      * 2. pdev->vpci->lock
      */
     rwlock_t pci_lock;
-#endif
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /*
+     * The bitmap which shows which device numbers are already used by the
+     * virtual PCI bus topology and is used to assign a unique SBDF to the
+     * next passed through virtual PCI device.
+     */
+    DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV);
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+#endif /* CONFIG_HAS_PCI */
 
 #ifdef CONFIG_HAS_PASSTHROUGH
     struct domain_iommu iommu;
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 2064d9354f5b..980aded26fc1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -21,6 +21,14 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 
 #define VPCI_ECAM_BDF(addr)     (((addr) & 0x0ffff000) >> 12)
 
+/*
+ * Maximum number of devices supported by the virtual bus topology:
+ * each PCI bus supports 32 devices/slots at max or up to 256 when
+ * there are multi-function ones which are not yet supported.
+ * This limit implies only segment 0, bus 0 is supported.
+ */
+#define VPCI_MAX_VIRT_DEV       (PCI_SLOT(~0) + 1)
+
 #define REGISTER_VPCI_INIT(x, p)                \
   static vpci_register_init_t *const x##_entry  \
                __used_section(".data.vpci." p) = (x)
@@ -175,6 +183,10 @@ struct vpci {
             struct vpci_arch_msix_entry arch;
         } entries[];
     } *msix;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /* Guest SBDF of the device. */
+    pci_sbdf_t guest_sbdf;
+#endif
 #endif
 };
 
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:35:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:35:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721586.1125131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tF6-0005yo-CZ; Tue, 14 May 2024 14:35:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721586.1125131; Tue, 14 May 2024 14:35:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tF6-0005yf-9o; Tue, 14 May 2024 14:35:00 +0000
Received: by outflank-mailman (input) for mailman id 721586;
 Tue, 14 May 2024 14:34:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tF4-0005tT-VJ
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:34:59 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2214bc5a-11ff-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 16:34:57 +0200 (CEST)
Received: from CH2PR07CA0022.namprd07.prod.outlook.com (2603:10b6:610:20::35)
 by MN2PR12MB4254.namprd12.prod.outlook.com (2603:10b6:208:1d0::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Tue, 14 May
 2024 14:34:50 +0000
Received: from DS3PEPF000099DB.namprd04.prod.outlook.com
 (2603:10b6:610:20:cafe::ad) by CH2PR07CA0022.outlook.office365.com
 (2603:10b6:610:20::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26 via Frontend
 Transport; Tue, 14 May 2024 14:34:50 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF000099DB.mail.protection.outlook.com (10.167.17.197) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:34:49 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:34:49 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:34:42 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2214bc5a-11ff-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dvK8YW3OdSTp7BR35H5WYReHmVXCDzjxy/jatDpZ89ZsyNf2PvJ4pD+aYBareE2t/Ucifc+fFNDudN+P/wiGvl/q6ZJyzhV2mByNVr2W82H5NbW6RftrUf6aOMAM+KpDVl1YUtFg0RY94ZiFuQPYWKlEJ1MIraQgNZvRwVUR/lkwYcV9H2ZLVmNMc2UuT583mi0IbKgyj1YyJOCNQSmaF3oEAxz/NdQt5pVX7XBvLsvY3JryAINVIdqr3b0ilz+UOi8B94A6/aKF2tQ8/vyeXDHzfDy03vsMa1GGwH9AEl3Rzp8p8x83S6pvaMhW55TW587yo4JnvSdBaJ7lreeQ1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Fz5VFWknMZx6HHUaDNZrtZ7oahJwlLDFqbFpBhGUT+E=;
 b=F3nkW1OkQV5weYKJ4cf3BBBveZcM3DjcvqszT9p/6Fzpsjv8EpBgeuOZrRqq4Etb7kVTCV5XGddGegSEQk1QJqCwK4YxviyYoixNxNXPwN6WliK8u4zjOnLBtvFoOL5YgExWmB+JtUm710eaCHpezEDq1yf8NUrbSEzmLBWsnQWxX0cuW/IcHJAhc8/8oR047XG5uLfQ5/fe1ZFiaOtH2VOpkpfd9QQvrxqkU24TLK4ZUT60R0W/oQ3NjpXgP3fwrpCFznrdOLZk4BPwvzEdUgtVSRe8h0W1UebPaXyNx4M52jfHll42f8b6KGnMJyMpi0BWrHBw7mja0dao/hWdGA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Fz5VFWknMZx6HHUaDNZrtZ7oahJwlLDFqbFpBhGUT+E=;
 b=ijabFWS0YQIN1JvxDvuN7H+NOOrwD0hV5HSpXRs0XgI3RTSEpPbt8hY9n9Oempobwbc158gdlaC+zgq4IoYciVAa1dVRweIN+BYdOBjF1g0IdCjAWK4G5KB6cO5qyjFPImmMfn54hG7c8FTh/k5NTmO7kwgl8YVGSgjsf5oC3GE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v14 3/5] xen/arm: translate virtual PCI bus topology for guests
Date: Tue, 14 May 2024 10:33:55 -0400
Message-ID: <20240514143400.152280-4-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240514143400.152280-1-stewart.hildebrand@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099DB:EE_|MN2PR12MB4254:EE_
X-MS-Office365-Filtering-Correlation-Id: 021f1f9e-2393-4caf-2263-08dc742302d9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?72aDNwaCEi4ACEL7T6eSjo5aP4gesI8fE2UiN3kfdRYI1V23X4RabeFLe5Vy?=
 =?us-ascii?Q?RJnmkfln5SvXuu8IoihBb6XMAJ9swBHE3VtwZXMTnyOD5J9ZzwRYOR9NENFM?=
 =?us-ascii?Q?CUiLbDzXHHJjvByYBxKRH5NLxtzqxGrq56WJ4xj/0NVqOrkyRmZPXDKXDiXI?=
 =?us-ascii?Q?jyF+Sr8LLSYLJui2DdJKEAt6RwHGrK0WkRdfIFRv5WNpGck1A5/7QeDLFzH7?=
 =?us-ascii?Q?FjMMQ7NZLhn/poZzHaKcZJOfamqNDDlCv2qpDLYdqDAiNOnep8Kq4xLjN5ov?=
 =?us-ascii?Q?Wr409cFWcw/sDPlaNlqTjaZIKTv24UTMTjKTV+8xJEV6qCHUzGTy19Tmasxi?=
 =?us-ascii?Q?vStIh9CHfVN8WjRaXwEGWPJ4wGh6xn4rsZnteXGTiXQ9odLAtu0874O4QrWE?=
 =?us-ascii?Q?ihEsZG302Lf8QTRYoC2bYyBnuB07S1ro3RE7mednQQ3+O9mhMlNNMr6W8Civ?=
 =?us-ascii?Q?sa2Ylod0tiAmJhaIKLUMr29dlAx+z487aFTeEXpnL6GBNqRePBzRD/i7YXz4?=
 =?us-ascii?Q?87WeMmf/LLwo+WV6WdRqFmlQ5fGu+JRojJYFLA5kZ+vw+4+UMsz2nATT/2WV?=
 =?us-ascii?Q?TdXwCXIUBzzy6Y5Y0WW0iZZkI912Fg36YhInF4N5Id7cLd4S9scouRfxukfO?=
 =?us-ascii?Q?0bOPcjQEfQOCCZbhtTZH+FvkMK/R3pt2M3AsOqcomwelmD+BFILrmezC2nb4?=
 =?us-ascii?Q?KQ/jMtVvMn/KIqBYs9acZita64WfJSgQE0fPwBgqQ7VqlE6Xz1L3BJrG33MJ?=
 =?us-ascii?Q?gwJFQvGO2hZIwGWDV4svdzZ3GQqr3ppVMY3hjpRC5Eaqd2x9keohdZxYu1Kb?=
 =?us-ascii?Q?+DrN5VTT0raYl6kqQ1SpbxJKqoJ7iLjbl6u8NsEM6Qlm83zfyOQF22CQg9fk?=
 =?us-ascii?Q?HOv7iufP6opKwXqZrQ7RLf6uqg7Wr+NIpg7M4fR2jvdRRQORovI5P2cavAQz?=
 =?us-ascii?Q?r3NwNBBZZlWeuj34VDzdzbC2tdJUujsUtvocecVY02N7mSZ8CPmzooxk+TWJ?=
 =?us-ascii?Q?qNNFqWEm6V+fS+vTa8c7yJkZKzTCSPZl3v8Fwhq9veqaiE5MZgzN90by5qd+?=
 =?us-ascii?Q?cPIXbgOjxG7GZ0EWwCpDgjJjLhBS60HV0t5LbJtvncfJOL+opTyETcHb2DH6?=
 =?us-ascii?Q?BOqWQjzb/Rm2ewYmyj2v/aifqUwpgB76UVNZOUnjpwxKqIWrwGPJJwJe7wjY?=
 =?us-ascii?Q?1aSE5aiOSwL4tIWr88xSTOq3PoMMiiZkRNh0DpA2+7a89oKZYmP1agMsAfQU?=
 =?us-ascii?Q?6R+7PSyTDTOQJELFF2UymR4sfKbErd3AICFfN9I7KtF1fvIFR/ucN/YSVteL?=
 =?us-ascii?Q?XozmcQFufmPZ4u+hN8CLDZYsTD25GlKFZIY6RI5UefsitVpzEXGqNWTLggq3?=
 =?us-ascii?Q?y8pbD/wbtHv38fMoUNGV2im9MLEw?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:34:49.9292
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 021f1f9e-2393-4caf-2263-08dc742302d9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099DB.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4254

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v11:
- Fixed format issues
- Added ASSERT_UNREACHABLE() to the dummy implementation of
vpci_translate_virtual_device()
- Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
the logic in the function
Since v9:
- Commend about required lock replaced with ASSERT()
- Style fixes
- call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
Since v8:
- locks moved out of vpci_translate_virtual_device()
Since v6:
- add pcidevs locking to vpci_translate_virtual_device
- update wrt to the new locking scheme
Since v5:
- add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
  case to simplify ifdefery
- add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
- reset output register on failed virtual SBDF translation
Since v4:
- indentation fixes
- constify struct domain
- updated commit message
- updates to the new locking scheme (pdev->vpci_lock)
Since v3:
- revisit locking
- move code to vpci.c
Since v2:
 - pass struct domain instead of struct vcpu
 - constify arguments where possible
 - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/arch/arm/vpci.c     | 47 +++++++++++++++++++++++++++++++----------
 xen/drivers/vpci/vpci.c | 24 +++++++++++++++++++++
 xen/include/xen/vpci.h  | 12 +++++++++++
 3 files changed, 72 insertions(+), 11 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 3bc4bb55082a..7a6a0017d132 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -7,31 +7,51 @@
 
 #include <asm/mmio.h>
 
-static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
-                                     paddr_t gpa)
+static bool vpci_sbdf_from_gpa(struct domain *d,
+                               const struct pci_host_bridge *bridge,
+                               paddr_t gpa, pci_sbdf_t *sbdf)
 {
-    pci_sbdf_t sbdf;
+    bool translated = true;
+
+    ASSERT(sbdf);
 
     if ( bridge )
     {
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
-        sbdf.seg = bridge->segment;
-        sbdf.bus += bridge->cfg->busn_start;
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
+        sbdf->seg = bridge->segment;
+        sbdf->bus += bridge->cfg->busn_start;
     }
     else
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+    {
+        /*
+         * For the passed through devices we need to map their virtual SBDF
+         * to the physical PCI device being passed through.
+         */
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+        read_lock(&d->pci_lock);
+        translated = vpci_translate_virtual_device(d, sbdf);
+        read_unlock(&d->pci_lock);
+    }
 
-    return sbdf;
+    return translated;
 }
 
 static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
                           register_t *r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+    {
+        *r = ~0UL;
+        return 1;
+    }
+
     if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa),
                         1U << info->dabt.size, &data) )
     {
@@ -39,7 +59,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0ul;
+    *r = ~0UL;
 
     return 0;
 }
@@ -48,7 +68,12 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
                            register_t r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
+
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+        return 1;
 
     return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
                            1U << info->dabt.size, r);
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 23722634d50b..98b294f09688 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -81,6 +81,30 @@ static int add_virtual_device(struct pci_dev *pdev)
     return 0;
 }
 
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
+{
+    const struct pci_dev *pdev;
+
+    ASSERT(!is_hardware_domain(d));
+    ASSERT(rw_is_locked(&d->pci_lock));
+
+    for_each_pdev ( d, pdev )
+    {
+        if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) )
+        {
+            /* Replace guest SBDF with the physical one. */
+            *sbdf = pdev->sbdf;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
 void vpci_deassign_device(struct pci_dev *pdev)
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 980aded26fc1..7e5a0f0c50c1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -303,6 +303,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v)
 }
 #endif
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
+#else
+static inline bool vpci_translate_virtual_device(const struct domain *d,
+                                                 pci_sbdf_t *sbdf)
+{
+    ASSERT_UNREACHABLE();
+
+    return false;
+}
+#endif
+
 #endif
 
 /*
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:35:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:35:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721588.1125142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tFG-0006R3-RX; Tue, 14 May 2024 14:35:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721588.1125142; Tue, 14 May 2024 14:35:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tFG-0006Qr-O9; Tue, 14 May 2024 14:35:10 +0000
Received: by outflank-mailman (input) for mailman id 721588;
 Tue, 14 May 2024 14:35:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tFF-0004xK-Pe
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:35:09 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 295e54c2-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:35:09 +0200 (CEST)
Received: from DM6PR06CA0074.namprd06.prod.outlook.com (2603:10b6:5:336::7) by
 CY8PR12MB7610.namprd12.prod.outlook.com (2603:10b6:930:9a::11) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7544.52; Tue, 14 May 2024 14:35:04 +0000
Received: from DS3PEPF000099DC.namprd04.prod.outlook.com
 (2603:10b6:5:336:cafe::3c) by DM6PR06CA0074.outlook.office365.com
 (2603:10b6:5:336::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 14:35:04 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF000099DC.mail.protection.outlook.com (10.167.17.198) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:35:04 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:35:03 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:35:03 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:34:56 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 295e54c2-11ff-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PcftE2FfCPHS65k20P2jyZcy626YJ3HQ3xqs0aSFk9K9qBttw7Y0pKxHVyGfP1uCaZYF6rdbg9JE+PHPSCfn17e0kRTBy/rUbIJTWboGcSG7yMEJ7A+XxbVZqZkxDy9Og5DqzHwm198fxDEpx2tVgF2x8YZOLG9VXm/sDKr2ZSzDmw1oiH9WXk3Tw0Aupm5xaVbJPwi5g5Om/XejX4G2x1GY4ND82YO2yrTeymkgDoDzFc6CNd1p2ZGhfo0kmWRKqfaPG7ykSD7+WsS0WMhXz6SB8CfovpJ6FuqX1xGlQhyw7I0/pCLDYDg3vjBobUIfDQ6vZ/if1y+4TBUnrr65mQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=jSDIS7u8FBArnwQtMGorD6BVCk5kKDuvtvaxFhhABr0=;
 b=kUGTCcx+SNxWuYZ4AKNqHLogxzCOGNc0KD0WltYihxdUzkLF0kNxPgHQEHNYuePI5C9l1H5PlF+McPBHTIR3xYzXjARpavkhDtrw1Y/tDoyCmvmdbMtJigo7QkYGlyIHhq0/m8NIoW6peRzr5XHkK24YFMiC37baVLV++tJvpl1/4u25NuGFuJX5fR1rNXUfskRjQZeQ/0sjF1T8blC8K8wkRHjvVR3OXG7afSakmkirpPzUvAyz71eLhQFefB/hBCtuvH1qMEjHdRlzYMNGFCLL2WxyheDUzZfRdoU5YA0F9REPC3M8UIj5ok3dCag2M58GJn5GqMvCK235RehJWQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jSDIS7u8FBArnwQtMGorD6BVCk5kKDuvtvaxFhhABr0=;
 b=chB9KdJWptuT+nuuJb1FQmOKU/98g3cPsIAfwqYSJcJvJW3gmkq3YjucMELJsyg9TJJ6gew9wjxILp2pliuLOYXOwMyLxTKk0u1rehgjrEJsBtIDfgySn+LM3OSxjfYISfAiDTtKLlzbup+HaXjRAZDbH/BX9DHMnk2hBt4/UXA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Julien Grall <jgrall@amazon.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v14 4/5] xen/arm: account IO handlers for emulated PCI MSI-X
Date: Tue, 14 May 2024 10:33:56 -0400
Message-ID: <20240514143400.152280-5-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240514143400.152280-1-stewart.hildebrand@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099DC:EE_|CY8PR12MB7610:EE_
X-MS-Office365-Filtering-Correlation-Id: 054dbf8b-0e00-4edc-a5e0-08dc74230b3f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?lqUVDnFTdJhmX13chB8ahkNu+P3fLDVWZ3mcOm7ftOGSTPQ8BgJXUJd/G8+8?=
 =?us-ascii?Q?/+tsUSTJJVtiQpfURGSMaN2Tg0HfxyQO8SDPSmNZoI62djYolbftaz8TTKXM?=
 =?us-ascii?Q?qvIOXZ3ADZaPjBCukC6exT9YHxxwNQRPmGHWhnYtlFwyWNYCGFR9zoeO+Q7a?=
 =?us-ascii?Q?6qgflUzj0bcgROQjNU4+bAC/rsADv1qHTMIiDblMRmF9M56h50nICIzFZsdY?=
 =?us-ascii?Q?GsooMFaaY7dK5UNVlXNMGSJPtjRKcxEOho9YiUNQHHgkkv7Hk1eS3+SZaIjS?=
 =?us-ascii?Q?ov+Bgd7zMozN+qAR10Lr+r4g/Nh3tPJf/eR3hWLFrLoaVKIVVEZjjdxbEuhL?=
 =?us-ascii?Q?ER1Mpdsxz3XSs2aV5HWyY7pO/QdmjNwVfIj5X57dcfPpEdzY7wg9p58U33SS?=
 =?us-ascii?Q?8fSK8Ct6XnRv2Ii5N8zB4E41eoNKDUucH9bImNJwuhgMKY+LakowOciFZBig?=
 =?us-ascii?Q?9jprNAQPZFpbgpcOlxC6yIOLlUUu+6xmrUhZcGRjv8yH3rjCLebNEEV7S3mc?=
 =?us-ascii?Q?Tw+dG/RWCH8Q138w5jdAyGDOaCMbRjl6QmEv2lJoXWWVc1AdsF+HsrNpN4qA?=
 =?us-ascii?Q?59ixPbjaNrEVAZoblqd4N+wNqbTku1Rgh9K4TkzhtMhjUmL6Dp2kgV/bf9nW?=
 =?us-ascii?Q?rXn4a/ggVgHu3W/+yfkP1dzTjsiqc94akNK32DWCSvQT0UD9alpxtKGdfbj/?=
 =?us-ascii?Q?8re3dmp+YHRGDJnO1p24WDINEMD7duEf3JIjsqfWGbMZrMLX0Rm5B7i5Wcbs?=
 =?us-ascii?Q?pqHW8kP2ya7L2s+YZ1DJWsTJyFXMluS66VJDANg/UifPVY7Vp5s/gktEpMlP?=
 =?us-ascii?Q?/XL8XhnULgIeU2Qg5Rv6yPlcScCS/umth/qAC+2YRUkmFKfZdPR4geu+IQnI?=
 =?us-ascii?Q?7xl762tB6X0DqlI/FEfSZGLaUGp/hrHH1pLNBiEW5nrTaBKwQnU2aoa7Iybp?=
 =?us-ascii?Q?7k4jt0i6rIUg4KU50vwMvGYJ9xxr9G6o6x9bhaq0kNwcnDCMPyVLwHi+zh+E?=
 =?us-ascii?Q?444RAosHkAGdQOJJeYp/im9olnbTwNficeWf/0UxG8MAy8IjAox6YP1jM31z?=
 =?us-ascii?Q?IZeLBCLhUcID1tdEwHLSpfICisPg/MbPYTu6fX+wAtBuoA+nkjHxrYyTw9Ek?=
 =?us-ascii?Q?/eU3HR0bvcv3CRIuANTXlldemxRZCcNPxBDCnWfVrdrWWtdnYyBxb1yECd9b?=
 =?us-ascii?Q?T4ZJ4HslTuw6/NVc9SJKa596T7i3Yon8mNahrlFcdvzq15COsslCoOglTjZR?=
 =?us-ascii?Q?/rmGKX8Rsq6gSYMFNT4nAgHsV3h28Z1druvxPRoW85uFi/4GXRvwFd+wD5Pb?=
 =?us-ascii?Q?NY1TUhrLWa2LeKbD4YMI3FErgl86i/0+DsU1fbQIOHbG9deYkUWFtSuj1l+4?=
 =?us-ascii?Q?9+74Jf0Pkr/oshYy7IrKnx93J+PT?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:35:04.0384
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 054dbf8b-0e00-4edc-a5e0-08dc74230b3f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099DC.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7610

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

At the moment, we always allocate an extra 16 slots for IO handlers
(see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated
MSI-X registers we need to explicitly tell that we have additional IO
handlers, so those are accounted.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
This actually moved here from the part 2 of the prep work for PCI
passthrough on Arm as it seems to be the proper place for it.

Since v5:
- optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is
  defined unconditionally
New in v5
---
 xen/arch/arm/vpci.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 7a6a0017d132..348ba0fbc860 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -130,6 +130,8 @@ static int vpci_get_num_handlers_cb(struct domain *d,
 
 unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
+    unsigned int count;
+
     if ( !has_vpci(d) )
         return 0;
 
@@ -150,7 +152,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
      * For guests each host bridge requires one region to cover the
      * configuration space. At the moment, we only expose a single host bridge.
      */
-    return 1;
+    count = 1;
+
+    /*
+     * There's a single MSI-X MMIO handler that deals with both PBA
+     * and MSI-X tables per each PCI device being passed through.
+     * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV.
+     */
+    if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) )
+        count += VPCI_MAX_VIRT_DEV;
+
+    return count;
 }
 
 /*
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:35:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:35:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721593.1125152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tFV-00070s-3y; Tue, 14 May 2024 14:35:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721593.1125152; Tue, 14 May 2024 14:35:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tFV-00070i-1D; Tue, 14 May 2024 14:35:25 +0000
Received: by outflank-mailman (input) for mailman id 721593;
 Tue, 14 May 2024 14:35:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6tFT-0004xK-Q3
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:35:23 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060e.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31c564ed-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:35:23 +0200 (CEST)
Received: from MW3PR06CA0002.namprd06.prod.outlook.com (2603:10b6:303:2a::7)
 by SN7PR12MB7977.namprd12.prod.outlook.com (2603:10b6:806:340::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 14:35:18 +0000
Received: from SJ5PEPF000001CC.namprd05.prod.outlook.com
 (2603:10b6:303:2a:cafe::c2) by MW3PR06CA0002.outlook.office365.com
 (2603:10b6:303:2a::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 14:35:17 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ5PEPF000001CC.mail.protection.outlook.com (10.167.242.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 14:35:16 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:35:15 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 09:35:15 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 09:35:09 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31c564ed-11ff-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bgo3z8RKUAeMa3//VZ3E6/Ae26SG+P7cyiZBEFk3moTngK7Bz0Ag3Gzsi2adNamwxPM7HvBikDOT9SJ3mR9eTUSGc++JyrwHhox62XVHkS2Fav/9XKXUlmwOO5yTtfO7WxGyohe8r46FaXBrd4ZtqNfzChPjKVnAN+vILRao3J+NYa37Er5+t3nAZSVt4YG+NT4EYZrbBg0ywJ/uevMz0v+/grPbcDY7KfRLhGCEQOU8lKQAo5kzQZNAa1IdyFarz+X9HPIoow5svG/GGtLuH29osMf4vX05egH5VWuhTUSeceBXhHfWiDrNHKsKt1n9z3EZMdSaUS1GYCAcudcemw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RhlHUD9Uu3FrjzdtnmRTwtlA65attPhQDXfrRmCx33g=;
 b=WLMXxnfJwS3y8MFsQlfqpi4x4s4vPKNJrzd5ayzyobaOpFUfRdQZmmfzS25LsXc/Iaoj2Q+WDkppJyDdsy31gYgrFE6j8NSszLb1EKPVoh6EzYnO3P69EA/TjL8RHWhEzI5cNmP6cgMOZvDpiTLhtI76PJq9IslSLyTGO1m9ISWzdAgCw0nNbKKX4csT5g5sx3YqlNWxXX7fGVuXeeC0wMi83HdjHumcgbRGHVJxb2l3+VspNNGgoys7t8ktwamSMfV3Esq76xIlN4Y0DnXIEsb7YHA7TINn07HgvjOtO6b7u6uP7qvJZZxkircuNIyhEsvVScfbG7qPW9L/nC53Bg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RhlHUD9Uu3FrjzdtnmRTwtlA65attPhQDXfrRmCx33g=;
 b=oZ2V6bMennaAUQdIMkXrd8Z40e1W+DZlj9XdxEG57oVnuhpmT3RjykTbjOrEyrnyWP1OwqXiLR14q/8L4YNTp1HNzW58jCI/3C2moJ++X9Lyyj8Ms5GdgtOBGF86kCw5lIVgC0q36njeGuj200/NfNhTAjAS4BVtKj2LnzgA7ak=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v14 5/5] arm/vpci: honor access size when returning an error
Date: Tue, 14 May 2024 10:33:57 -0400
Message-ID: <20240514143400.152280-6-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240514143400.152280-1-stewart.hildebrand@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001CC:EE_|SN7PR12MB7977:EE_
X-MS-Office365-Filtering-Correlation-Id: b0ef63ab-8853-4624-2182-08dc742312ce
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?IRQdCSQ76CNm6BAyrpNWu5W0Gq18D7L2QRGuVj/HmSj/CzJC4jYK+lHewJNV?=
 =?us-ascii?Q?3Hs8qrW5bw2bfysC+foYio+wSwdyR00iSU04q6j1+Hc1sX2X2jzUytJHyJLx?=
 =?us-ascii?Q?gW7dVIyP+bag4ZBUBxMVcADEAiSs3RrPnsvZEIq0JP+UV+suf/LGZkzoIkPf?=
 =?us-ascii?Q?S6QxOk6ALoB3eH5Jk8pUuvMy3e90hT4exui8/iBSeTT3C4kWPsTyx48NmNB0?=
 =?us-ascii?Q?4VxOKrbK/vb7pGeSr2c+yIsJoZt3PqpOd0qBx+Z5rXYCFfN154T5XKRxKTnM?=
 =?us-ascii?Q?nYMJYcttO46TS8zBqB9BqKmITd/NEhYVToRZYNkLXLLqDu2p7dcSowzFI0mb?=
 =?us-ascii?Q?koP94USOU592ipnRLywEbvmygRezWq1amjayu1yNRk7nK+0u5WkmwWJG3p6m?=
 =?us-ascii?Q?4T3H8DOfcT2o9yzCbaKLsY5NR5ENRrON+R6qo4Bti31bcoGNUp2rv8mkAYqH?=
 =?us-ascii?Q?uA6IL3svTEDmxcfQEsZPLupFPm5oYNLDGKITcb3y83v3eE2iGzoFNaBRMC4c?=
 =?us-ascii?Q?RP6w1zSE9xoRwuBNDd4/v5oK2O99EUOM/BnqhjnU70FfmA1xozo3//Q36umc?=
 =?us-ascii?Q?ghy+x9sXELvjX3gOW6EPmQBtxto4KNbxdhy+tU26BykBsqnVNm/b6ZqV7cqw?=
 =?us-ascii?Q?GfgrgyFPPMYNNlIFNaP+CpSyiQ4PqQAszP7MZKieC8bDjAury1gtLU2JX5Ez?=
 =?us-ascii?Q?wH4TCQZ2Bbs4JBKHugGN+pI5oFQMUwlACJwWvIAf0qdots9GEDqDh0Q0bgCX?=
 =?us-ascii?Q?7J2AFiYG+MQ/zqZNgN6grXY2UCJ37dNOCzV6pK+5073+POfBs6UPF1/ffgUw?=
 =?us-ascii?Q?1yp2S1c0PM+ZoQDmDoiSbjnv3+haJ1ma/RKNajqRpXv/Lw+rd0zJ+KRn0IJP?=
 =?us-ascii?Q?qPc1/9slHeDj4EDH6OeK7o/5B/XicthcsON4g0KbEwWgp7cWdeeCa+xrnk8P?=
 =?us-ascii?Q?Os2hjyBVU8SpGKapdIsunQoJLEWK91c0AAecqSTsqzlEBzb2522+2mpzkcNv?=
 =?us-ascii?Q?9iRqV3dKhGYW+SFEIMxBHkRZHuJV4m9bKoeR3NzXDMaIxb4q+YtcuwVfFTEw?=
 =?us-ascii?Q?RjDhNqkYOKw0z8z7DlVwxPAy40KLieAIEckLM8FR3n96PMQDPnBaBe5TRDzf?=
 =?us-ascii?Q?Xx4N3VCl11FFtcBtaLfIOPzopF/ecGeND0sZCbJB1xo822faNOgTOM18SyJN?=
 =?us-ascii?Q?ZbM4Wz/lKvo34oZ7Kgu25AxVjo8ulytX55hmVFDWm3NuEkJGP43afHgZUjgY?=
 =?us-ascii?Q?jVP8iZ+sB00YHinpFPBPv4HXaFcfLjPWxIAi0uoPKYLvFovlxYQca7sJVDqB?=
 =?us-ascii?Q?xSMWNrPddZ9hirmZ4Q6OQPD5w+l9vkeBob5sYaftr9nE7pcn6/fNIaERq3hM?=
 =?us-ascii?Q?3sDw4nMLJBL+As1P2HEZIJkWBt7U?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 14:35:16.6850
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b0ef63ab-8853-4624-2182-08dc742312ce
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001CC.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7977

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

Guest can try to read config space using different access sizes: 8,
16, 32, 64 bits. We need to take this into account when we are
returning an error back to MMIO handler, otherwise it is possible to
provide more data than requested: i.e. guest issues LDRB instruction
to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
register.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
v9->10:
* New patch in v10.
---
 xen/arch/arm/vpci.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 348ba0fbc860..aaf9d9120c3d 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -41,6 +41,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 {
     struct pci_host_bridge *bridge = p;
     pci_sbdf_t sbdf;
+    const uint8_t access_size = (1 << info->dabt.size) * 8;
+    const uint64_t access_mask = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
@@ -48,7 +50,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 
     if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
     {
-        *r = ~0UL;
+        *r = access_mask;
         return 1;
     }
 
@@ -59,7 +61,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0UL;
+    *r = access_mask;
 
     return 0;
 }
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue May 14 14:36:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:36:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721600.1125162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tGY-0007sb-EH; Tue, 14 May 2024 14:36:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721600.1125162; Tue, 14 May 2024 14:36:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tGY-0007sU-AZ; Tue, 14 May 2024 14:36:30 +0000
Received: by outflank-mailman (input) for mailman id 721600;
 Tue, 14 May 2024 14:36:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6tGX-0007sO-Va
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:36:29 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5959c2d7-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:36:28 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a5a88339780so27992066b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:36:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b015f7sm742302166b.146.2024.05.14.07.36.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:36:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5959c2d7-11ff-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715697388; x=1716302188; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=R74w2d5dyci9/kJ/yEsGNOwynJklK0T+h9hxLHHyfc4=;
        b=OauMrxLf1Zc+sU9HLoRQCKyD70zNokQHoPVVVUJHd4+IXjGjvGyMhVsR6zwD1hoDxF
         gDPesSTkY0Plc9s7yUe8kjOdUoJyOYkkk8IQltJ/YqiAlor3MFX5xJD5d5X8DjFhGAyF
         qwz1/JXKi/cpIU1t0B1KaIOn1+myHwtiKqRLzbv0w37fM7pXRKzBJ7DIGdAA3J/2xucr
         vi06Y9A2p80xdssVGAw3gHVQW7WA3st68A84Weky758es2Rbg3NDvIHIzD+rmzi3QFNX
         L0c8f+/NTueIhypUNkOWHji50LuRBZNtgdXm0hKXJjF3+MZdWoGMESwIR7PcOZMoGsHy
         8USw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715697388; x=1716302188;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=R74w2d5dyci9/kJ/yEsGNOwynJklK0T+h9hxLHHyfc4=;
        b=nddVNhsYJCiggO/f7auQYoxcRry7o2C0VQBpg+9L4lA68n4EvSdg2jhfvy5g7vndOL
         ddzG1nzo0zHeK9VMu/exil8o2SYfoP/z4tu6N//vQKGhdN4gRGwA+YRkAAU9ZujmpWwD
         /YpG91hj+ts6CEPsJGZ5b0nYtC2U1/mDPq26Mmmoh0iiyep8wc4JUjY/QJD/omhYgXAp
         MQopQsP/AAZ9l36rx3kySbdB32xouClw0F83QSnqKPjbPwdjIMqeR9UzkoFOXdX1EV6e
         kz7AJ3StLQiyVdQcJBnW49EpJ35tIPrDRlT3FBvSRd8oC6WBTdFIZh3irP719uSACM02
         im3w==
X-Forwarded-Encrypted: i=1; AJvYcCWuLBopGNQYi5qDgSGmBJhwkKaSYtRznce6Myxv1brmsb/Bx301eMIXxbvo5yngTTTwbJgtVfW4Gvv/dAgCvbfmvHNpyGg4mNk+Ce+DdfU=
X-Gm-Message-State: AOJu0Yz8K9oGuFwtMYb/JIOVaEXMemojo6FgLW5Bplka4P8EVy6kWk2n
	2P+zKGJY5hHEW5+8xdxpb2+WT6AYpwsNqGMBqQxnZ6HLtbH/T1XBzuNFbzLzrQ==
X-Google-Smtp-Source: AGHT+IECD/gMxXe/ZZrDJC1Iapv1M6ue7FFistrCUcnDIGGRHXpvKSW6qNoIZB9hbH2DcZf0NxdZbw==
X-Received: by 2002:a17:906:7f8a:b0:a59:b6a8:4d7a with SMTP id a640c23a62f3a-a5a2d5d3d22mr854441066b.43.1715697388166;
        Tue, 14 May 2024 07:36:28 -0700 (PDT)
Message-ID: <05093548-7cd4-4969-825f-c40fee6d7879@suse.com>
Date: Tue, 14 May 2024 16:36:35 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2 3/3] xen/x86: remove foreign mappings from the
 p2m on teardown
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Community Manager <community.manager@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240508112323.38946-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.05.2024 13:23, Roger Pau Monne wrote:
> @@ -1043,7 +1045,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
>          break;
>  
>      case p2m_map_foreign:
> -        if ( !mfn_valid(nfn) )
> +        if ( !mfn_valid(nfn) || p2m != p2m_get_hostp2m(p2m->domain) )
>          {
>              ASSERT_UNREACHABLE();
>              return -EINVAL;
> @@ -1068,7 +1070,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
>          break;
>  
>      case p2m_map_foreign:
> -        if ( !mfn_valid(ofn) )
> +        if ( !mfn_valid(ofn) || p2m != p2m_get_hostp2m(p2m->domain) )
>          {
>              ASSERT_UNREACHABLE();
>              return -EINVAL;

I wonder whether these two hunks wouldn't more logically be part of patch 1
or 2 (probably not 1, as at that point the assertion isn't quite correct
yet). Then again it may be as (seemingly) unrelated there as it is here.
Thus
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:37:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:37:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721607.1125172 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tHV-0008Tf-M6; Tue, 14 May 2024 14:37:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721607.1125172; Tue, 14 May 2024 14:37:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tHV-0008TY-JP; Tue, 14 May 2024 14:37:29 +0000
Received: by outflank-mailman (input) for mailman id 721607;
 Tue, 14 May 2024 14:37:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6tHU-0008TO-Et
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:37:28 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7c73acbc-11ff-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:37:27 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e6792ea67dso59884421fa.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:37:27 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b015f7sm742302166b.146.2024.05.14.07.37.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:37:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c73acbc-11ff-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715697447; x=1716302247; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=NvP4yLvMnC1cVmbWzh4ZmXRmEhCmleFWN5rjcW+9gLk=;
        b=cy6KICB7SOYiTfsVdqYgDDqMCFbdohsH1Eftvw71duBozVZg+mKzpr66l2FO7soMr3
         tCynWqF2rRPSHsra1wOnPeuY3LpSdZoUNykRRqA0XpHSbM+T2GNq1glVc1wKI5yqmcuI
         XRb3fqeLZeCLJoVKQY07UU4hYvGKMCMa1Y/N2TUjPHwr+MgOGP2084MWb78oR8GtvmpY
         csTKOgEAS7A2ZvBsXfiUitZwhi6Vmunqrhk+jCtann2sjqmNk0caw5M8w8eexI3dI9US
         UYgDcVSZRva0zFw/i54fitzYInNKZlk4SiVhEXnHlMcbsrEY8md1V2p4epM5X9+b0CpV
         DGEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715697447; x=1716302247;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=NvP4yLvMnC1cVmbWzh4ZmXRmEhCmleFWN5rjcW+9gLk=;
        b=VBmc0Up6X6/3yjId0q3yYeoBrEqOfMGiL/R01AimLC6BO+Ugwxxph13WHnoxnrGt+1
         hKhlT8P8xMMR2S8Ty+W0zGn6LFiEikoSgazuCLRcMWBWpnV3Q809NYv99fpOwuTwTX7K
         PEwo7VEH7jny4dz7kJH1BPqN4EB3v+GFd4MB03w7sjd6EVGyFPpuKcuwu8m3q0wRzQm8
         r+XzhBITzuLqryJP0lcpnwR3zw7mswiuLwYjFCGy/zLmlGHQdweao1FjZMiNBRmPOs+K
         pqare7uirsgtcbXzeaj+AVt4huauBjKOkKI8CzFKghTGzLugP68AInQoVX0s+e3jV+y4
         VdaQ==
X-Forwarded-Encrypted: i=1; AJvYcCUoOXNXLT/pNe/z1ledbJF24xDkxGfmKkrXE6ktXm+sIt+61sTrwQhQHVQsPsJ3yJFdj+16sl+S4v/7qzQDMT/m/fknFcIfOPYducnc7cc=
X-Gm-Message-State: AOJu0YzEG2r/xa/3aXChHL/PEavuUsYQQ9BE3aWRfSOqYqV5nZBS513t
	M/WU6kQCS5w7u1QZTdI+voMfTTFif8iw+SNPHhvzEg7ussXdzuYSNm+7nWuA+w==
X-Google-Smtp-Source: AGHT+IEpGbgpVtl1MrlgrhkFmH7Ws7TmA7NO09ezKniJIOZaPhS6KFYni+aCG8FO0L3axAHki4m3rw==
X-Received: by 2002:a05:651c:2114:b0:2e1:cb0f:4e1e with SMTP id 38308e7fff4ca-2e51fc34061mr120325021fa.2.1715697445009;
        Tue, 14 May 2024 07:37:25 -0700 (PDT)
Message-ID: <74be423c-1210-452e-9450-f8006282144e@suse.com>
Date: Tue, 14 May 2024 16:37:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v2 1/3] xen/x86: account number of foreign
 mappings in the p2m
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240508112323.38946-1-roger.pau@citrix.com>
 <20240508112323.38946-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240508112323.38946-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.05.2024 13:23, Roger Pau Monne wrote:
> Such information will be needed in order to remove foreign mappings during
> teardown for HVM guests.
> 
> Right now the introduced counter is not consumed.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
Yet I think it shouldn't be committed (much) ahead of patch 3.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:44:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:44:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721614.1125181 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tNt-0002uF-E0; Tue, 14 May 2024 14:44:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721614.1125181; Tue, 14 May 2024 14:44:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tNt-0002u8-BC; Tue, 14 May 2024 14:44:05 +0000
Received: by outflank-mailman (input) for mailman id 721614;
 Tue, 14 May 2024 14:44:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6tNs-0002u2-Eh
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:44:04 +0000
Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com
 [2607:f8b0:4864:20::72c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67a96310-1200-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:44:03 +0200 (CEST)
Received: by mail-qk1-x72c.google.com with SMTP id
 af79cd13be357-792ce7a1fa8so346945485a.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:44:03 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf3106c2sm569422385a.106.2024.05.14.07.44.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 07:44:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67a96310-1200-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715697841; x=1716302641; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=yU54ZWAH083rLTWsflh2hCSJ7w8yWqPrKomATStyodM=;
        b=bfQ84rRcnI/7sE8zII/OOWR7e710PDNsqHJ2ZL51wj7Pt6J1yK76eBpFWfrSwf/Gct
         mJA2M4tcBHYxpdbL8aazrkyjhhWxOs2C8DjYI0KVyebOE5ZPrYpXJM9f3V/ZbhVZ74GS
         ao8XrLFc1wWJ7LW4yGrxFCgoetsVh6Wl3RPYM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715697841; x=1716302641;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yU54ZWAH083rLTWsflh2hCSJ7w8yWqPrKomATStyodM=;
        b=mKTpNFiADsjdDJOHjlLDCp9Tyspd6dYAHWw6HpRgPCO15XXUCfhhWwnAl518SiP0rp
         5ho51PUIO2r74z3/1WYtl7qhBwfABRieSZtCdVQBeAbUwoCiULw3SqVhbrdJZYw0MpAD
         hxRaV2zuNV9bEKPjcfO2zW2IPEfGRCq3HbZAg7GMkxR7BW45MwP1XbSmY5kbaRIBTq7E
         2W89UjHj6T4lKejx8IG80w3wg/TRHFZMrH0sB01jn/9I0DfBvaAbVm4Lh8qXYAnFqzsT
         T5qh19/hjXOjbY+Tz8VYdP2JorRMm4tYL75/zwilQ7JGKBZQg0ex9TviJC/aJ6WKFzyb
         VIMA==
X-Gm-Message-State: AOJu0YwOr+X2jnyYPM7RA8q058Sq9aMs5syPchFZaKGEz/HTzKVa8NQm
	TL+Ax2ZYMFF0bF5zvbTrG039UjSfSoJfzedj2tr3hXWYvP2LTsbhiu3vILwYbPk=
X-Google-Smtp-Source: AGHT+IHh99zfJ7zdzINhue8fHdXkNa7E4LKWeQms9Bv2K1OjP4wgYyFQww2wuaEldVKbYYLxWSbR/A==
X-Received: by 2002:a05:620a:9372:b0:792:e0b5:c029 with SMTP id af79cd13be357-792e0b5cc3bmr549214385a.21.1715697841444;
        Tue, 14 May 2024 07:44:01 -0700 (PDT)
Date: Tue, 14 May 2024 16:43:55 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
Message-ID: <ZkN4q6H591SeT6wH@macbook>
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
 <dd689012-0554-43ae-b22b-035415ce3392@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <dd689012-0554-43ae-b22b-035415ce3392@citrix.com>

On Tue, May 14, 2024 at 02:50:18PM +0100, Andrew Cooper wrote:
> On 14/05/2024 12:09 pm, Andrew Cooper wrote:
> > On 13/05/2024 9:59 am, Roger Pau Monne wrote:
> >> There's no point in forcing a system wide update of the MTRRs on all processors
> >> when there are no changes to be propagated.  On AP startup it's only the AP
> >> that needs to write the system wide MTRR values in order to match the rest of
> >> the already online CPUs.
> >>
> >> We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
> >> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
> >> on all the CPUs in the system.
> >>
> >> While there adjust the comment to clarify why the system-wide resetting of the
> >> MTRR registers is not needed for the purposes of mtrr_ap_init().
> >>
> >> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >> ---
> >> For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
> >> triggering, but it's also a good performance improvement when performing
> >> cpu-online.
> >>
> >> Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
> >> the chance of the watchdog triggering due to the system-wide resetting of the
> >> range.
> > "Runtime" changes will only be during dom0 boot, if at all, but yes - it
> > is restricted to a single MTRR at a time.
> >
> > It's XENPF_{add,del,read}_memtype, but it's only used by Classic Linux. 
> > PVOps only issues read_memtype.
> >
> > Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Actually no - this isn't safe in all cases.
> 
> There are BIOSes which get MTRRs wrong, and with the APs having UC
> covering a wider region than the BSP.
> 
> In this case, creating consistency will alter the MTRRs on all CPUs
> currently up, and we do need to perform the rendezvous in that case.

I'm confused, the state that gets applied in mtrr_set_all() is not
modified to match what's in the started AP registers.

An AP starting with a different set of MTRR registers than the saved
state will result in the MTRR state on the AP being changed, but not
the Xen state stored in mtrr_state, and hence there will be no changes
to synchronize.

> There are 3 cases:
> 
> 1) Nothing to do.  This is the overwhemlingly common case.
> 2) Local changes only.  No broadcast, but we do need to enter CD mode.
> 3) Remote changes needed.  Needs full broadcast.

Please bear with me, but I don't think 3) is possible during AP
bringup.  It's possible I'm missing a path where the differences in
the started AP MTRR state are somehow reconciled with the cached MTRR
state?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:51:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:51:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721626.1125192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tUe-0004ve-4R; Tue, 14 May 2024 14:51:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721626.1125192; Tue, 14 May 2024 14:51:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tUe-0004vX-1q; Tue, 14 May 2024 14:51:04 +0000
Received: by outflank-mailman (input) for mailman id 721626;
 Tue, 14 May 2024 14:51:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6tUc-0004k5-Nv
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:51:02 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61f0396e-1201-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:51:02 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56e48d0a632so340999a12.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:51:02 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17a06sm721622766b.219.2024.05.14.07.51.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:51:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61f0396e-1201-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715698261; x=1716303061; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=tUfldpWvaaTSiHO6/aigbPFj+us19e8xKvzDgSUsTcs=;
        b=dbft37rI8jixh1eYZuBfrP4/BXF4vhrJLjj990cdZCIXQkCTas8tkvOLphibhwiN+2
         aik7Nqe+SO1Ho9218FCHdEjpDaoKVs6NzML/Hv2Gwn2NwTUTAF9oArhg6fwXCG2VBu0I
         qaKEBEw2+G9Tqh4zZcHxBeoX1pvz8NhynRIo0P58viEsNPHdcmGclX9rltuQJSjUUlxo
         HjtHeUBiauGnI9EGdzwDlWnWfcvqrK8VNb1gyh31jFhT2uPfQGMt5sY5wJUFVHK4/9v1
         DqmTGsIb4OmoKYQdNexhomHoFIVcAuUxJRF7bKe1tI1hEJ8pDY1dS+M0FA/aOUx6cCtr
         8zBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715698261; x=1716303061;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tUfldpWvaaTSiHO6/aigbPFj+us19e8xKvzDgSUsTcs=;
        b=Rfg60OaHeap0ONnJxb7uadhyTuOieRNbe4eJKc6zEWCMbYkkdr7Dfdn0OU7KT8BlvV
         QUEL/vhDwYPaGnbI/qeshh8AKQscJF8uMTjtTs8BOkCaFv/3WAn39mPFrw6KeK/u4qsh
         zvQmy+V47ks6FLgRzbcuvDtCq04J8cgIxX+VUABIj3YX6Rc2FCXz9mTvkbDaoY7xtKHx
         lUCrTZIjf2mYwNxPtNGa9mqJaeyGvUMEs+zbf1xszUzf1T+srm8/A4iQJN0vKyQbVIOA
         v98tr7WVHbk8DOvJfg7V0L1mCNkWkjPjk7FjiCa0e0FFr/9IfBHJimkhBtsBlwVdtDpP
         DNDA==
X-Forwarded-Encrypted: i=1; AJvYcCWvjOW9rBD4M8S4XBd0rGDP4yVhq4z39BOm1+b9sDC+Rm1pVcp4wy2F5R8Nv74Jg4S0QQLKnPeGAwYnE5IF2vKJbXf97TeMJMPmPmR+g44=
X-Gm-Message-State: AOJu0Yx/53ZngpKn9Ml7aPzXU04iBNTJKkoweiqTA8ZXS+ayJxJveEpT
	nNYkVQFhLL3tis5//AbuRG/S/7TgTHf32C6K7lC2xf+2s040CCYhrI0EcN0DwA==
X-Google-Smtp-Source: AGHT+IHL9uIUI+8AgWp2RsFS0lsnJ1bdfHIWHfRQDn8oQBO5HIQcIYpKC1gqMmKJcdbNVbVAkKHuRA==
X-Received: by 2002:a17:906:3e0d:b0:a5a:1077:334 with SMTP id a640c23a62f3a-a5a2d57a39amr1284122766b.26.1715698261448;
        Tue, 14 May 2024 07:51:01 -0700 (PDT)
Message-ID: <dd145c67-8e3e-4b15-94f7-c7cd1f127d45@suse.com>
Date: Tue, 14 May 2024 16:51:08 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 01/19] x86: Create per-domain mapping of
 guest_root_pt
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-2-eliasely@amazon.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-2-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Create a per-domain mapping of PV guest_root_pt as direct map is being
> removed.
> 
> Note that we do not map and unmap root_pgt for now since it is still a
> xenheap page.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
>     Changes in V3:
>         * Rename SHADOW_ROOT
>         * Haven't addressed the potentially over-allocation issue as I don't get it

I thought I had explained in enough detail that the GDT/LDT area needs
quite a bit more space (2 times 64k per vCPU) than the root PT one (4k
per vCPU). Thus while d->arch.pv.gdt_ldt_l1tab really needs to point at
a full page (as long as not taking into account dynamic domain
properties), d->arch.pv.root_pt_l1tab doesn't need to (and hence might
better be allocated using xzalloc() / xzalloc_array(), even when also
not taking into account dynamic domain properties, i.e. vCPU count).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 14:58:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 14:58:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721634.1125202 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tbW-0006U0-Tp; Tue, 14 May 2024 14:58:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721634.1125202; Tue, 14 May 2024 14:58:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tbW-0006Tt-Qv; Tue, 14 May 2024 14:58:10 +0000
Received: by outflank-mailman (input) for mailman id 721634;
 Tue, 14 May 2024 14:58:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6tbV-0006Tn-6c
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 14:58:09 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60059651-1202-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 16:58:08 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so24343766b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 07:58:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c823dsm729289166b.100.2024.05.14.07.58.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 07:58:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60059651-1202-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715698688; x=1716303488; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wofELa6xhGTDOJO2X7cSH5wYetnwYKwX6A5ydQs9olQ=;
        b=HNYbCcROiZshAlHG0OyY8wY0cJmXTZGAjiKBmpUOFhyMXCzaWbj2D5dj+x1WFzm5dn
         63UCnRTQ8lIRRtZp2oGTiyAcIt348y94u50muyBQuyuue84Tz85NXKgJhehAnyA57gSb
         O2jFwBD4TMownz8BSBQCrPHrMuWJpnOZpl2xi2eSxMxIJdZByzyFrePtpFgfckxRxhVw
         qxb0oYGYihxwB0EFlgGT8FZWFvGhPHeMqRzVkSEaqQf2WE1LUu43XeC0KhwugljUwLUm
         gvd6LvVpJlAckR1ZA42+gliEp59UMQEJmgcalfn1fkRP1CcAfXA2pF+nRiJ4I4iR0ug9
         6hTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715698688; x=1716303488;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wofELa6xhGTDOJO2X7cSH5wYetnwYKwX6A5ydQs9olQ=;
        b=sk0aXmbuhT0F11RqlehoEYwtAqp67NpPJAaIdrtsz4SOVXyELu33Rfhn2epeBmJtR6
         ZW9AwV9WTpVm/hKO+hbtsl6ot7Jtp0TXyn1Nxxtv44mziWQsY1Ws3CKIMMa7N0xXzQui
         S5j1JIO1kd37MLthKatNIa2vLjtpwUUHnO6pDKfwGxK206OyPzBdBUIuxlwF4EW9P4Yk
         09Kys7Rz8e7C+XztL9/iwnDx7P2sWzlD1Fg44/7yY3c9XQxiPZjEzHehB0I9W3zv3CyE
         +ZvXgZpe88VOuztGD/ko3QZuSLGqtnkq32T2w4VvRu/tNeOlodUGsXyrihLM01C50hND
         U6iA==
X-Gm-Message-State: AOJu0YzvLaVdd7uC2G/Mfjgtj/w+ENhJ2aBANcB1fhlts0gKnm/WQVVE
	4SQfaIaew2XWjJ7L96luhqw1XAcs52VMpUBevx4v3gvI9ngUr5vd/O/pqP9dhA==
X-Google-Smtp-Source: AGHT+IHCTnYWAhCHBR5nm5fKjQmGckyZVlWCRJ9ouOaO2/8miaM8G2xf5P0/ieRLYTSkJk6J94pzdQ==
X-Received: by 2002:a17:906:ae51:b0:a59:cdc5:d6e8 with SMTP id a640c23a62f3a-a5a2d6668d5mr775356866b.50.1715698687755;
        Tue, 14 May 2024 07:58:07 -0700 (PDT)
Message-ID: <79afece5-a4c7-4426-b40f-b3db49fbdfc2@suse.com>
Date: Tue, 14 May 2024 16:58:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 03/19] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall <jgrall@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-4-eliasely@amazon.com> <ZkJEhZCRNnxGb_lc@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkJEhZCRNnxGb_lc@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.05.2024 18:49, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:30PM +0000, Elias El Yandouzi wrote:
>> @@ -710,22 +714,32 @@ int __init dom0_construct_pv(struct domain *d,
>>          v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
>>      }
>>  
>> +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
>> +do {                                                    \
>> +    unmap_domain_page(virt_var);                        \
>> +    mfn_var = maddr_to_mfn(maddr);                      \
>> +    maddr += PAGE_SIZE;                                 \
>> +    virt_var = map_domain_page(mfn_var);                \
> 
> FWIW, I would do the advance after the map, so that the order matches
> the name of the function.

Actually I was thinking kind of the same when looking at v3, even if I
may not have commented to that effect. Then again that goes somewhat
against the further suggestion below.

>> +} while ( false )
>> +
>>      if ( !compat )
>>      {
>>          maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
>> -        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
>> +        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
>> +        l4tab = l4start;
> 
> You could even make the macro return virt_var, and so use it like:
> 
> l4tab = l4start = UNMAP_MAP_AND_ADVANCE(l4start_mfn, mpt_alloc);
> 
> ?

Not quite, l4start also need to be an input to the macro:

    l4tab = l4start = UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);

Else unmap_domain_page() has nothing to act upon. If anything that would
then (imo) likely better be

    l4tab = UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);

with l4start still updated inside the macro.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:02:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:02:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721639.1125211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tg5-0008ET-F8; Tue, 14 May 2024 15:02:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721639.1125211; Tue, 14 May 2024 15:02:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6tg5-0008EM-C9; Tue, 14 May 2024 15:02:53 +0000
Received: by outflank-mailman (input) for mailman id 721639;
 Tue, 14 May 2024 15:02:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6tg4-0008EG-DX
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:02:52 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 067e3c4d-1203-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:02:47 +0200 (CEST)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-34da04e44a2so4524988f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:02:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbbd60sm13961896f8f.90.2024.05.14.08.02.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 08:02:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 067e3c4d-1203-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715698967; x=1716303767; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=2YMk/aXSIx/Qh4B8ngzoA9luXMIY+lh/iAPWI0bgU8c=;
        b=XfSbuxIcT631UyTaeJRS4w6bAl2zbOkg+dhagVQR3jsdPEHDjtePrqBIXp16Er0YpJ
         Zh6ccfeQhHLWSjVSjF5EeprMd92CGjhU8vy+hCnJPN6iBE2I9qmA9F1O0W5MbF8hq5in
         oCV+OBOTWUUGSgFdLxtqmJV2bhMSavxUOC21Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715698967; x=1716303767;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2YMk/aXSIx/Qh4B8ngzoA9luXMIY+lh/iAPWI0bgU8c=;
        b=F5uho0qXp/49xvG8Q8kSic59byNZxU4Nt0S22JsFhrUjObbgdI6gKL1y+pdMkyg13f
         hD2lRthdwbY9R9eGLg7rP0QlBR1ARb3szzYHF3QuZcJG9qlYheW5u1yV/S7WLQ2vy/fP
         4Y/LIG8w5DgY/laYpvqHge/QbJlA/zDS0Q7R/wWQMxfPleNXi62hjgn2kMAzSCPvuNOf
         Wclj5oBQQgP7Rfl5ril9LAGtf75qN4uvL7ec31K36x8m7RTmmi8+96fGFbMwdlcTjGST
         uQJDGH8C5rW8zUIuUHzLbnb090eou0Xm4+LjWxm1cSqAhnQ91xRgjSGUBGiAjt1iqzAI
         +sAg==
X-Gm-Message-State: AOJu0YxK+yzmYLIhdOtaDPywB8ZA0IkedXlA3mfI1o86iKv9mg6l4Ns3
	IY7uIY5PnzdHFel4K4aEPfNZwgAaP0yX1OhyjIee5RVfaS1seRZ0SC+E/GeZwE4=
X-Google-Smtp-Source: AGHT+IG2hIjYFNYYy2IZhutaQgmwju+nA2Xisu8qoeok/AqBJYQPZgxQVcnoSLZt/CBXkau/8BTvmA==
X-Received: by 2002:a5d:45d1:0:b0:34c:120a:fbed with SMTP id ffacd0b85a97d-3504a635932mr9546509f8f.15.1715698966963;
        Tue, 14 May 2024 08:02:46 -0700 (PDT)
Date: Tue, 14 May 2024 17:02:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 12/19] x86/setup: vmap heap nodes when they
 are outside the direct map
Message-ID: <ZkN9FWP3VMZS-s-y@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-13-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-13-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:39PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When we do not have a direct map, archs_mfn_in_direct_map() will always
> return false, thus init_node_heap() will allocate xenheap pages from an
> existing node for the metadata of a new node. This means that the
> metadata of a new node is in a different node, slowing down heap
> allocation.
> 
> Since we now have early vmap, vmap the metadata locally in the new node.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     Changes in v2:
>         * vmap_contig_pages() was renamed to vmap_contig()
>         * Fix indentation and coding style
> 
>     Changes from Hongyan's version:
>         * arch_mfn_in_direct_map() was renamed to
>           arch_mfns_in_direct_map()
>         * Use vmap_contig_pages() rather than __vmap(...).
>         * Add missing include (xen/vmap.h) so it compiles on Arm
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index dfb2c05322..3c0909f333 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -136,6 +136,7 @@
>  #include <xen/sched.h>
>  #include <xen/softirq.h>
>  #include <xen/spinlock.h>
> +#include <xen/vmap.h>
>  
>  #include <asm/flushtlb.h>
>  #include <asm/page.h>
> @@ -605,22 +606,44 @@ static unsigned long init_node_heap(int node, unsigned long mfn,
>          needed = 0;
>      }
>      else if ( *use_tail && nr >= needed &&
> -              arch_mfns_in_directmap(mfn + nr - needed, needed) &&
>                (!xenheap_bits ||
> -               !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )
> +              !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) )

Unrelated change?  The indentation was correct for this line and you
are breaking it.

>      {
> -        _heap[node] = mfn_to_virt(mfn + nr - needed);
> -        avail[node] = mfn_to_virt(mfn + nr - 1) +
> -                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
> +        if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
> +        {
> +            _heap[node] = mfn_to_virt(mfn + nr - needed);
> +            avail[node] = mfn_to_virt(mfn + nr - 1) +
> +                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
> +        }
> +        else
> +        {
> +            mfn_t needed_start = _mfn(mfn + nr - needed);
> +
> +            _heap[node] = vmap_contig(needed_start, needed);
> +            BUG_ON(!_heap[node]);
> +            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
> +                          sizeof(**avail) * NR_ZONES;
> +        }

You could shorten the blocks I think:

if ( arch_mfns_in_directmap(mfn + nr - needed, needed) )
    _heap[node] = mfn_to_virt(mfn + nr - needed);
else
    _heap[node] = vmap_contig(_mfn(mfn + nr - needed), needed);

BUG_ON(!_heap[node]);
avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
                  sizeof(**avail) * NR_ZONES;

So that more part of the logic is shared between both.

>      }
>      else if ( nr >= needed &&
> -              arch_mfns_in_directmap(mfn, needed) &&
>                (!xenheap_bits ||
> -               !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
> +              !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) )
>      {
> -        _heap[node] = mfn_to_virt(mfn);
> -        avail[node] = mfn_to_virt(mfn + needed - 1) +
> -                      PAGE_SIZE - sizeof(**avail) * NR_ZONES;
> +        if ( arch_mfns_in_directmap(mfn, needed) )
> +        {
> +            _heap[node] = mfn_to_virt(mfn);
> +            avail[node] = mfn_to_virt(mfn + needed - 1) +
> +                          PAGE_SIZE - sizeof(**avail) * NR_ZONES;
> +        }
> +        else
> +        {
> +            mfn_t needed_start = _mfn(mfn);
> +
> +            _heap[node] = vmap_contig(needed_start, needed);
> +            BUG_ON(!_heap[node]);
> +            avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) -
> +                          sizeof(**avail) * NR_ZONES;
> +        }

Same here.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:04:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:04:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721642.1125221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6th9-0000J3-Oy; Tue, 14 May 2024 15:03:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721642.1125221; Tue, 14 May 2024 15:03:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6th9-0000Iu-Ll; Tue, 14 May 2024 15:03:59 +0000
Received: by outflank-mailman (input) for mailman id 721642;
 Tue, 14 May 2024 15:03:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6th8-0000DO-AP
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:03:58 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2fa65987-1203-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:03:56 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5723edf0ae5so296823a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:03:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81bfsm731478666b.129.2024.05.14.08.03.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 08:03:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fa65987-1203-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715699036; x=1716303836; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qyAaFDYPFGlusW1kBXq8G9N5fsrl7gO/puN8GnKOyDE=;
        b=FGoOLM87q6JcpIJEJQQ4/AKgBA0VR8A0HbQx8rW5kV9CkDJ7v+ajhhpuOPamGxAeuH
         NKrr68z0mPUTrrGMItp9DgIgy3kHHtJwwF6SeakFPE6Sn5jN1m4QiVzZnaPR1rpTjFEO
         hrWC2wW5SQBe805rajL3SJXy5RuqcwvhGdb7pOMU8sX9Nxv06jx28VuRLyjAzCMpylvp
         6hNLWNJPRt9lXqhQjaups0gX64RkQ5N0w6oLfi+diOl0uJmyvlQDc+fCZq2aFEyNqo6k
         +W2JPelxTTqKsmrVeUKjQIUI5TfBoE6m1XuLZ0RsVhl28blh8hMVNQ0Y56Th8GfC45vD
         37xQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715699036; x=1716303836;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qyAaFDYPFGlusW1kBXq8G9N5fsrl7gO/puN8GnKOyDE=;
        b=IBe6MatKIxSGxWkDoH/pWpIWjx/bjGnf27kASuKB56gb2bUyMuSeqNZO95P8KmORME
         EugJdrt9DB7/HZSEC7TreS1EoFxg5TgkrRXMLQ0Yb7+u+YS+eRhyWwrnwXyz+946Tg0Z
         3NnVvbs067fqO7lnAydrNLDDa+IoCNXZaEK+Vak5fJtcxa5D2WFQSgiJqrThc/D7KGC9
         BoRCOuo20ADqiG9XY8wCj3jydn7NBdDQBdAFX84rx1t/XOu5eEZY5yVeRfBIWDhhZL6B
         20W3Q7npQ/LM9V52Xzzh2g9OoQsQoV8MZ3Ud/onuwMgI7RIghMyb0exAk8o7iCzUqr0V
         sM1w==
X-Forwarded-Encrypted: i=1; AJvYcCXtuVZMU8jcTVG0B0JhNylIrHsZS+AYV+mtRHsZOnFm1x0A9rDiwkp40k5u0Ht5mwms+KOsCEpTVolkn1WEwmfkUdagn0lA9MnRxLWIhNQ=
X-Gm-Message-State: AOJu0YwNFhXKgKBLCiWA0ztjv62jzdxOskFmWv8k1+jyarQs3rN/BWH/
	xhzzATh+5b7IsWFklaZYgBCk2C7PFtdKlX/Ia/75u/SOiH0D5CtaYg2BEzAD9A==
X-Google-Smtp-Source: AGHT+IEgj0LbQaQwe8FjQSwHYGVTzmu53Yc20g9mQkt6iI4448KhENaJOwX4A9nTq02VUm+jBG+7xw==
X-Received: by 2002:a17:906:a404:b0:a59:a8a4:a5a0 with SMTP id a640c23a62f3a-a5a2d5cb736mr911445166b.36.1715699036095;
        Tue, 14 May 2024 08:03:56 -0700 (PDT)
Message-ID: <ffe6fbc8-fbbb-44a4-b981-b43f3fb48433@suse.com>
Date: Tue, 14 May 2024 17:03:55 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 03/19] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-4-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-4-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
>      l3_pgentry_t *l3tab = NULL, *l3start = NULL;
>      l2_pgentry_t *l2tab = NULL, *l2start = NULL;
>      l1_pgentry_t *l1tab = NULL, *l1start = NULL;
> +    mfn_t l4start_mfn = INVALID_MFN;
> +    mfn_t l3start_mfn = INVALID_MFN;
> +    mfn_t l2start_mfn = INVALID_MFN;
> +    mfn_t l1start_mfn = INVALID_MFN;

Just to mention it here again: By limiting the scope of these I'm pretty
sure no initializer would be needed even "just in case" (really I don't
think they're needed even when the all have function scope, as producer
and consumer are always close together afaics; quite different from
l<N>start and l<N>tab).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:39:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:39:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721651.1125231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uFS-000867-Gm; Tue, 14 May 2024 15:39:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721651.1125231; Tue, 14 May 2024 15:39:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uFS-00085z-EH; Tue, 14 May 2024 15:39:26 +0000
Received: by outflank-mailman (input) for mailman id 721651;
 Tue, 14 May 2024 15:39:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6uFR-00085m-Ce
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:39:25 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 227a6774-1208-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:39:22 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-34e28e32ea4so3605668f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:39:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b79bc83sm13958607f8f.16.2024.05.14.08.39.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 08:39:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 227a6774-1208-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715701161; x=1716305961; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=8rFS0byviflQCNG2nFalv8QEIIekxcFumIxXCjB1Dx0=;
        b=UCka3p415KK721X0iRsphTLVmRaDRrTc73W54R3+IcXTwCbRuH/wl1RaLURfjog4MI
         2bvjv0iX0CVWSmJ/hiQljb3MSBiGSEpbPg8//aJfggUL7HtXT6nXmli6Na2x2AXSexqU
         0Lf3VROUhqabqopZ3FV7z/F/gujXHClyEa3kA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715701161; x=1716305961;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8rFS0byviflQCNG2nFalv8QEIIekxcFumIxXCjB1Dx0=;
        b=rq401qpyIBTKCSNx4AlVV9FQmOUplz6oqEPAKHsNHsIp8sARF7gw1Xi+3TllaJpUQS
         9AjnLyMtu5upEgPi2nn+GmkrBT1IPXWO6NA1zmzxu7zt+k0+EqxQPeaiV/WZgPUbGtco
         5ylFhMNGKjwIvNoF34i8hrEmNGr8pI9Wk40lx7p6XoF2i4sf/n8Agib9t8nlfegoIiOY
         5b5BUaFvrnFLGnbIWLZn3mnrcidotTnqxkneppL6V00G7YieZG3uZT/8+CkycKiPh3NP
         WQeeQB9p4Ae/A2yCrWKW7aPnY/ppygOqHmEv0YRC+0Fu96H9zB0Plw6o+m9v1diahWQP
         99Sw==
X-Gm-Message-State: AOJu0YzawTTFTuqjK/AuFBomz0nQREihWhpdCsPVkRQt7/VATgeV8CGh
	426b+Y3ZcPkTd/7GVVXiLoA3Q5EcQocA74l5Cu9Fjnjwyrysz+Z490VZiB3en2o=
X-Google-Smtp-Source: AGHT+IF2qc8pdS+Vc83sojdXB3vFa5nk+Jxva2fEq/ZkeStufP7ZbhO/GJ9eIR1/vB1qafBM244zqw==
X-Received: by 2002:a5d:53c5:0:b0:34c:e9b5:d746 with SMTP id ffacd0b85a97d-3504a632ff4mr10319302f8f.6.1715701161401;
        Tue, 14 May 2024 08:39:21 -0700 (PDT)
Date: Tue, 14 May 2024 17:39:20 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 13/19] x86/setup: Do not create valid
 mappings when directmap=no
Message-ID: <ZkOFqFrSs41UtjIU@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-14-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-14-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:40PM +0000, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Create empty mappings in the second e820 pass. Also, destroy existing
> direct map mappings created in the first pass.
> 
> To make xenheap pages visible in guests, it is necessary to create empty
> L3 tables in the direct map even when directmap=no, since guest cr3s
> copy idle domain's L4 entries, which means they will share mappings in
> the direct map if we pre-populate idle domain's L4 entries and L3
> tables. A helper is introduced for this.
> 
> Also, after the direct map is actually gone, we need to stop updating
> the direct map in update_xen_mappings().
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index f26c9799e4..919347d8c2 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -978,6 +978,57 @@ static struct domain *__init create_dom0(const module_t *image,
>  /* How much of the directmap is prebuilt at compile time. */
>  #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
>  
> +/*
> + * This either populates a valid direct map, or allocates empty L3 tables and
> + * creates the L4 entries for virtual address between [start, end) in the
> + * direct map depending on has_directmap();
> + *
> + * When directmap=no, we still need to populate empty L3 tables in the
> + * direct map region. The reason is that on-demand xenheap mappings are
> + * created in the idle domain's page table but must be seen by
> + * everyone. Since all domains share the direct map L4 entries, they
> + * will share xenheap mappings if we pre-populate the L4 entries and L3
> + * tables in the direct map region for all RAM. We also rely on the fact
> + * that L3 tables are never freed.
> + */
> +static void __init populate_directmap(uint64_t pstart, uint64_t pend,

paddr_t for both.

> +                                      unsigned int flags)
> +{
> +    unsigned long vstart = (unsigned long)__va(pstart);
> +    unsigned long vend = (unsigned long)__va(pend);
> +
> +    if ( pstart >= pend )
> +        return;
> +
> +    BUG_ON(vstart < DIRECTMAP_VIRT_START);
> +    BUG_ON(vend > DIRECTMAP_VIRT_END);
> +
> +    if ( has_directmap() )
> +        /* Populate valid direct map. */
> +        BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart),
> +                                PFN_DOWN(pend - pstart), flags));
> +    else
> +    {
> +        /* Create empty L3 tables. */
> +        unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1);
> +
> +        for ( ; vaddr < vend; vaddr += (1UL << L4_PAGETABLE_SHIFT) )

It might be clearer (by avoiding some of the bitops and masks to simply
do:

for ( unsigned int idx = l4_table_offset(vstart);
      idx <= l4_table_offset(vend);
      idx++ )
{
...

> +        {
> +            l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)];
> +
> +            if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
> +            {
> +                mfn_t mfn = alloc_boot_pages(1, 1);

Hm, why not use alloc_xen_pagetable()?

> +                void *v = map_domain_page(mfn);
> +
> +                clear_page(v);
> +                UNMAP_DOMAIN_PAGE(v);

Maybe use clear_domain_page()?

> +                l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR));
> +            }
> +        }
> +    }
> +}
> +
>  void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>  {
>      const char *memmap_type = NULL, *loader, *cmdline = "";
> @@ -1601,8 +1652,17 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>          map_e = min_t(uint64_t, e,
>                        ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT);
>  
> -        /* Pass mapped memory to allocator /before/ creating new mappings. */
> +        /*
> +         * Pass mapped memory to allocator /before/ creating new mappings.
> +         * The direct map for the bottom 4GiB has been populated in the first
> +         * e820 pass. In the second pass, we make sure those existing mappings
> +         * are destroyed when directmap=no.

Quite likely a stupid question, but why has the directmap been
populated for memory below 4GB?  IOW: why do we need to create those
mappings just to have them destroyed here.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:40:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:40:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721653.1125241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uGP-0001Kr-PX; Tue, 14 May 2024 15:40:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721653.1125241; Tue, 14 May 2024 15:40:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uGP-0001Kk-N5; Tue, 14 May 2024 15:40:25 +0000
Received: by outflank-mailman (input) for mailman id 721653;
 Tue, 14 May 2024 15:40:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y506=MR=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s6uGN-0001J7-So
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:40:23 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 46476bbc-1208-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:40:22 +0200 (CEST)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2e43c481b53so69436671fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:40:22 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0cda6b7sm17623261fa.29.2024.05.14.08.40.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 08:40:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46476bbc-1208-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715701221; x=1716306021; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=tU2yXDAuZSc5ve1dF3Ml055enoyzRf2YW4t0BMnSwZg=;
        b=RtycvfqHVCOKyBsBFDMoNNXj6fw8xEy/zC+p+hrpadzXFaHiOjfIfH1HflaDg4dSXw
         DA71ahajMRDCCjimjMxgwICuQiMk+QpID4jF6fLVe9v+l4tp1VUJ6uL43U5d+biWuSWr
         TjFS+UtnM8GNq30GfEwCKAjpmjW17SiGnmgQ6taEZhF6kxLu9vZUFdVdJ286++4yfj63
         UREoNQUwRMau+KqDuA8vYOklMocu3e1hkwUsErqnNe2kshAqT+9Pkx8TpRTMjY639iUJ
         EyJ44JLcAp4Yc7cp686MNLCHlpfqo8yBvnYWpE1Ru2KQXFGy1qVUnFOxe1BN9SS+y5CD
         tiYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715701221; x=1716306021;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tU2yXDAuZSc5ve1dF3Ml055enoyzRf2YW4t0BMnSwZg=;
        b=Ark5YlarR0RQi8eRUkg/SffUZqNISyhx10Qs+Bmr6RegHeD53e2Pw7TRAsm3s2pOsZ
         hfwWyLG49Sod4u7g4na5kkexpof3cPqXTgFWPoZKZB5VUXzFmH4FnkBrOiPmGAEmBWK9
         V6suMPTcrepRxkimGTBX+KFY8PoooTJlWa7i2d4lPl1HrBoLYi+9USGCjGA6zWM0PGLc
         HgHrZduUFuxiJvgDj992tJuPOiHn3QxdcbUC6FWbGAGnEDxpeCH9bCT+xffHNBQgvoRn
         iPSQU+5SoF3TBMsC4KH6mAh+OB3Z7l0uqdNcE948G0sECrUIlxTgO+mZ9XYilkhA0EZ8
         XjOg==
X-Gm-Message-State: AOJu0YxjbIveqEaAb9C7YSWzO6iEvNmRIp3kl9zsaoL3VRLucPeG1s9i
	ktaVondWzb9Q0+JGNm/s8s6yhBsVxMVMkEDQh5vNCObRRXaUV2g6h0w8JpaM
X-Google-Smtp-Source: AGHT+IFFMe/WXqEjDVJHc6BeTX4cEx9P9c4vvJSkaA/dHAqA2cEq+bbSE9KSN2XPy+biLXb1DpQHhw==
X-Received: by 2002:a2e:e02:0:b0:2d4:54f2:c409 with SMTP id 38308e7fff4ca-2e52028d78emr80870171fa.38.1715701220984;
        Tue, 14 May 2024 08:40:20 -0700 (PDT)
Message-ID: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
Subject: Proposal to Extend Feature Freeze Deadline
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, Henry Wang <xin.wang2@amd.com>, 
	kelly.choi@cloud.com
Date: Tue, 14 May 2024 17:40:19 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hello everyone,

We're observing fewer merged patches/series across several
architectures for the current 4.19 release in comparison to previous
release.

For example:
1. For Arm, significant features like Cache Coloring and PCI
Passthrough won't be fully merged. Thus, it would be beneficial to
commit at least the following two patch series:=20
[1]https://lore.kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@amd.c=
om/
 =20
[2]https://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.=
com/

2. For RISC-V, having the following patch series [3], mostly reviewed
with only one blocker [4], would be advantageous (As far as I know,
Andrew is planning to update his patch series):
[3]https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@=
gmail.com/
[4]
https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/

3. For PPC, it would be beneficial to have [5] merged:
[5]
https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptoreng=
ineering.com/

Extending the feature freeze deadline by one week, until May 24th,
would provide additional time for merges mentioned above. This, in
turn, would create space for more features and fixes for x86 and other
common elements. If we agree to extend the feature freeze deadline,
please feel free to outline what you would like to see in the 4.19
release. This will make it easier to track our final goals and
determine if they are realistically achievable.

I'd like to open the floor for discussion on this proposal. Does it
make sense, and would it be useful?

Have a great day!

Best regards,
  Olesii



From xen-devel-bounces@lists.xenproject.org Tue May 14 15:44:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:44:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721658.1125252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uJy-0002Ua-90; Tue, 14 May 2024 15:44:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721658.1125252; Tue, 14 May 2024 15:44:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uJy-0002UT-65; Tue, 14 May 2024 15:44:06 +0000
Received: by outflank-mailman (input) for mailman id 721658;
 Tue, 14 May 2024 15:44:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0FJY=MR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s6uJw-0002UN-6q
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:44:04 +0000
Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com
 [2607:f8b0:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c9069469-1208-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:44:02 +0200 (CEST)
Received: by mail-oi1-x22c.google.com with SMTP id
 5614622812f47-3c999556297so2590111b6e.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:44:02 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf33f1a5sm570691885a.131.2024.05.14.08.43.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 08:43:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9069469-1208-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715701441; x=1716306241; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uTkAbNEdl0lHbYWjk3Nr8h08VnGBDx80LnEL1ULdfWM=;
        b=F4MX+30S4mPIhVNg38gfRmec+NsSuznANOZ9xPmG43EB5rN4/uGM9CAN9oToK3KS33
         eiUmYr5c7scLRl8ngv8L7vp3evAi7yddje7azdZUTj9aVEldpfltFnZr4N8WmH2CB8w8
         I5ecHguhzQUwRvn2VvxrZ0mhXBTbICW6slInI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715701441; x=1716306241;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uTkAbNEdl0lHbYWjk3Nr8h08VnGBDx80LnEL1ULdfWM=;
        b=IYoMCTxWggcb4gI5WW6rJAeW17WMnWqG6oZDKom1lv+8FVxPbAJysiOsVZvi9BQGxA
         dNNuXGlI49c1s0u9hGRSweHMMneER4hdFM4ko31vYGQwvgXUBCFzmWyAzXC7pGQFE8RC
         9xeXy2bN6V2tc6ULtrirzboSXuR5SbJhNnNfy+uLN/dx+BmtiNg2/pty8ModwxNGarIE
         ZgDv3f7+/24nD+pFRo+FfYKKBjh8yMw7o5k9MbNsUIpV5E2i6W3HRmdDsVuQLeg0v0Ak
         Twxjs+I92sUsHpGK5CWG/m/+V1j1hoTL/1Tq1CkOBaulz+QlSK3ZjSL9WbL8sxOoWSpm
         2eyw==
X-Forwarded-Encrypted: i=1; AJvYcCU7TdMPfnr9hl3wCiWl+iDy9KCOLqHTzWJPxepDq3D/spDEmv3hm97gvd3MkYlGbfy7mpEaBIRgGGH5wgW5SODyIe6tBEbXCdG07ES0lSg=
X-Gm-Message-State: AOJu0YzHO0+8ezABMOvQzez/9cDb2TctqX4brczLWteRZ41uIBhg/Goh
	ce0K2ps5XAfYOl2eloeUUEXE8BZ0QgVXO2vJbTdtMowJyo9Hd1kOV3F1TllBIAw=
X-Google-Smtp-Source: AGHT+IHX8HhQDqPVX8X4NsPcU75FMtU4ifuRqzGmZ/gM3oMymo7snynAS5ditVMdhLjxi4aHvX0Ybg==
X-Received: by 2002:a05:6808:18a7:b0:3c9:9517:e0ab with SMTP id 5614622812f47-3c997072211mr17629639b6e.32.1715701440386;
        Tue, 14 May 2024 08:44:00 -0700 (PDT)
Message-ID: <bba91b8b-b1cc-4054-96e8-80a63a3b808a@citrix.com>
Date: Tue, 14 May 2024 16:43:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Proposal to Extend Feature Freeze Deadline
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, Henry Wang <xin.wang2@amd.com>,
 kelly.choi@cloud.com
References: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14/05/2024 4:40 pm, Oleksii K. wrote:
> Hello everyone,
>
> We're observing fewer merged patches/series across several
> architectures for the current 4.19 release in comparison to previous
> release.
>
> For example:
> 1. For Arm, significant features like Cache Coloring and PCI
> Passthrough won't be fully merged. Thus, it would be beneficial to
> commit at least the following two patch series: 
> [1]https://lore.kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@amd.com/
>   
> [2]https://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.com/
>
> 2. For RISC-V, having the following patch series [3], mostly reviewed
> with only one blocker [4], would be advantageous (As far as I know,
> Andrew is planning to update his patch series):
> [3]https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@gmail.com/
> [4]
> https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/
>
> 3. For PPC, it would be beneficial to have [5] merged:
> [5]
> https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptorengineering.com/
>
> Extending the feature freeze deadline by one week, until May 24th,
> would provide additional time for merges mentioned above. This, in
> turn, would create space for more features and fixes for x86 and other
> common elements. If we agree to extend the feature freeze deadline,
> please feel free to outline what you would like to see in the 4.19
> release. This will make it easier to track our final goals and
> determine if they are realistically achievable.
>
> I'd like to open the floor for discussion on this proposal. Does it
> make sense, and would it be useful?

Considering how many people are blocked on me, I'd welcome a little bit
longer to get the outstanding series/fixes to land.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:45:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:45:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721661.1125261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uLU-00030N-Il; Tue, 14 May 2024 15:45:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721661.1125261; Tue, 14 May 2024 15:45:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uLU-00030G-Fr; Tue, 14 May 2024 15:45:40 +0000
Received: by outflank-mailman (input) for mailman id 721661;
 Tue, 14 May 2024 15:45:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PE3g=MR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s6uLT-000308-KO
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:45:39 +0000
Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com
 [2607:f8b0:4864:20::72a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02146be8-1209-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:45:37 +0200 (CEST)
Received: by mail-qk1-x72a.google.com with SMTP id
 af79cd13be357-78ef9ce897bso455749685a.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:45:37 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a354a0a802sm43546d6.81.2024.05.14.08.45.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 08:45:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02146be8-1209-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715701536; x=1716306336; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=NHQQ0lQFDULRomqg48fqsY1jG2htG1eJGwOYJVORz3g=;
        b=Kv+pPeM+1z2PIHNXpzNXhaqsscOOFHcXTa8NjlJ7r6ooVGLpGkbSPSUtpdAM2aEwEw
         mucu81Iookc8B8C2KFUdZ+aLVGqe+x2B3XgnalIWPwwAba8/W3OGVSj/1L3ipVXzfo2c
         BoB/IGmgWer/9olX+1v9zMs1/paRmd+D1FNBU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715701536; x=1716306336;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NHQQ0lQFDULRomqg48fqsY1jG2htG1eJGwOYJVORz3g=;
        b=VFxWApSRg9Qn/OmBQ79HwLXHNKAnkNRDfRwqbXdhKjtKnHBqAQiarOhExd1JBqq0I9
         CQ0mndYGnYugSQRLKmC5pat+0aa7WKaiQQ0Kz2lVZB7ZyGUlVAZzoi2c8AJjTINIwkOO
         q1NhbUGcF9c7+ZcPCB/wHBpo4qpcTOyF1offuQZMOT1gKQSChFIX+mBp1gbPOq/2zyoU
         nnvHnhyfq9MyOHvaKPro0OvYF/jdJVkDkazKyMg86DbWWNP7Ifzd6Kbrv6Dpknzjh4BB
         b75VV9C0ywaEVOGvd3MmOjsGrreDv201zi6uzqiP3bQXa7jCKPfIRjHU0KzYlqQ010As
         ePQw==
X-Gm-Message-State: AOJu0YwLH3CrvI8Ri4aqdzSrqdg+FX9CkvaSADbRX/+7o+dbjbGP8Noo
	2AwAe9pI5UbI1Y6yD+qfuHHdklBY979xIcy+R7wrHjzCvWeqs3KwSZ2lZvvoIK8=
X-Google-Smtp-Source: AGHT+IHGW1UysBHmMO1yCAkzV9aoPlrXWsZWN4ZZLWv57KtzZTiXpFxdm/WvUHUxdlrrmwE7AMlQ4A==
X-Received: by 2002:a05:6214:3c9c:b0:6a0:ac0f:a59c with SMTP id 6a1803df08f44-6a168258eefmr147035916d6.46.1715701536369;
        Tue, 14 May 2024 08:45:36 -0700 (PDT)
Date: Tue, 14 May 2024 17:45:34 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Lukasz Hawrylko <lukasz@hawrylko.pl>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Mateusz =?utf-8?B?TcOzd2th?= <mateusz.mowka@intel.com>
Subject: Re: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Message-ID: <ZkOHHtRPTqC4Zvnp@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-15-eliasely@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240513134046.82605-15-eliasely@amazon.com>

On Mon, May 13, 2024 at 01:40:41PM +0000, Elias El Yandouzi wrote:
> Until directmap gets completely removed, we'd still need to
> keep some calls to mfn_to_virt() for xenheap pages or when the
> directmap is enabled.
> 
> Rename the macro to mfn_to_directmap_virt() to flag them and
> prevent further use of mfn_to_virt().

Both here and in the following patch, I'm afraid I'm unsure of it's
usefulness.  I'm leaning towards this being code churn for very little
benefit.

Also, I'm not sure I see how the patch prevents further usage of
mfn_to_virt(), as (for Arm) the existing macro is not removed.  If
anything I would prefer a comment clearly stating that the macro
operates on directmap space, and avoid the name change.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 14 15:46:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 15:46:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721663.1125272 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uMA-0003VR-R5; Tue, 14 May 2024 15:46:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721663.1125272; Tue, 14 May 2024 15:46:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uMA-0003VK-NT; Tue, 14 May 2024 15:46:22 +0000
Received: by outflank-mailman (input) for mailman id 721663;
 Tue, 14 May 2024 15:46:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gsOr=MR=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s6uM9-0003TG-Nv
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 15:46:21 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1ba3ac1b-1209-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 17:46:20 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-572e8028e0cso324142a12.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 08:46:20 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b014besm733507666b.161.2024.05.14.08.46.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 08:46:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ba3ac1b-1209-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715701579; x=1716306379; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UNs0PEvIyVMnqw5/1MDhjLmg7EgRSbvMronrwsuzX7g=;
        b=K7VbsCQu0j5Rhv81xGhBvfb0T1woMhftNmjtYTxn5v+iLKNJoqLSqClCzeQ2SHZh7J
         Y4iomtnLoxa5A5cfAa8Io5HDduXN+VmNmibT4+iB//Di30uGchaB2AZEMZRxO7yOmlhF
         KGoqCKf2C+BbG6w2lWGElwzW0uM/ZgZazXt5E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715701579; x=1716306379;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UNs0PEvIyVMnqw5/1MDhjLmg7EgRSbvMronrwsuzX7g=;
        b=N7lELOhLdgmG5xUY6oce+2MW3twGN5zijmv3KfzV9Hqq8SbapYqzAjcJ4brVW3F1XZ
         t6eFZ4Qw/6HDKSK3JfMH9HLHFVNHgJxgontcsDE22aywaRPR2LDKccnGc21j0emNhmaB
         tqKao3fJJFX4+TGsCADY85AqFbjYaqFds+lu3ksoptm4u0EHIyaatUZ2iZssoPVMsFon
         AO4PJvIypm/pzvWEXTIzYcrLgfapkM91L42SCCgppI7aIGzYN/NUH05ESAN7vGJ7kA8D
         LywE5iYWct6+op/CVS/Z4dhOAz6DrGz+8iInuJpZ0TUhSlI5Gygjbr38iEYZ40NYltzg
         jAZQ==
X-Gm-Message-State: AOJu0Yw8Ah/UPF6BbeoeNc+xaaEJywu6cmQmOWNLMe+h7jTXVWterDuY
	Kiv9kJ9y3Vp0DetFtYKOjmCKOrMWyZAsOqhm4sb9lPspL/Lq1Ofbht0lF2wYA3I=
X-Google-Smtp-Source: AGHT+IEBQ335cMZkyzoGqgaTiQq3j4uraHfDi/mnZSdqw33iKaFbSh2GRgTsy0OTrTHlHrK9OMZm/w==
X-Received: by 2002:a17:906:a203:b0:a59:a8a4:a59f with SMTP id a640c23a62f3a-a5a2d5d55e8mr666386266b.45.1715701579533;
        Tue, 14 May 2024 08:46:19 -0700 (PDT)
Message-ID: <ebdf9834-20f4-4779-8c33-4fb27d5f6d7f@cloud.com>
Date: Tue, 14 May 2024 16:46:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
To: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Julien Grall <jgrall@amazon.com>, Elias El Yandouzi <eliasely@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
 <20240513111117.68828-2-eliasely@amazon.com> <ZkIxdtiDc_pnPWdx@macbook>
 <7998ecee-659d-4180-8b14-110c2907021f@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <7998ecee-659d-4180-8b14-110c2907021f@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 09:03, Jan Beulich wrote:
> On 13.05.2024 17:27, Roger Pau Monné wrote:
>> On Mon, May 13, 2024 at 11:10:59AM +0000, Elias El Yandouzi wrote:
>>> @@ -317,6 +317,14 @@ extern unsigned long xen_phys_start;
>>>  #define ARG_XLAT_START(v)        \
>>>      (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
>>>  
>>> +/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */
>>> +#define PV_ROOT_PT_MAPPING_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
>>> +#define PV_ROOT_PT_MAPPING_ENTRIES      MAX_VIRT_CPUS
>>> +
>>> +/* The address of a particular VCPU's PV_ROOT_PT */
>>> +#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \
>>> +    (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))
>>
>> I know we are not there yet, but I wonder if we need to start having
>> some non-shared per-cpu mapping area in the page-tables.  Right now
>> this is shared between all the vCPUs, as it's per-domain space
>> (instead of per-vCPU).
> 
> In turn requiring per-vCPU page tables, posing a problem when a guest
> uses the same page tables for multiple vCPU-s.
> 
> Jan
> 

True. Having separate page tables per CPU is an unavoidable end goal for
a hypervisor claiming to hold no secrets, however. Otherwise any CPU can
still speculatively read the stacks of other CPUs and take well-timed
glances over mappings set transiently by any other CPU.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Tue May 14 16:23:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 16:23:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721674.1125282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uvf-0003Hf-Jr; Tue, 14 May 2024 16:23:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721674.1125282; Tue, 14 May 2024 16:23:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6uvf-0003HY-GG; Tue, 14 May 2024 16:23:03 +0000
Received: by outflank-mailman (input) for mailman id 721674;
 Tue, 14 May 2024 16:23:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VieR=MR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s6uve-0003HS-OS
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 16:23:02 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3babf022-120e-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 18:23:01 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a5a88339780so49246466b.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 09:23:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d7csm747656166b.140.2024.05.14.09.22.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 09:23:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3babf022-120e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715703781; x=1716308581; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=oRpWNAdQnTo0N4docaodQ5ZeZukKlehSHQL6Z/WP0yk=;
        b=TDVANDcfV7VzAIznUVDPU2NRRJ8IJwsJHpaNWFIhvvovzdZqeKubGtW9ycxel4cP8/
         G6F+cjn2Mdb0JcDFLG1OMoTy+2Tq/FhsKzsSCH/BniDD5uVcAafQJqdRHWUAwu7mQwjk
         fcxyU+KdsOErRCYz8689LshNKSJ5VCt9qUaIQzCXGS6F0TU+0xkezulZ7e/TNrxHxu7p
         WcPflgqhGGDmaCWBs2qM8lW1oivs18MEeRPzK1XoM/9faPA1IADiYnrTGcnG17FWtcfM
         WJCNilk8/wzmOHuOn7zOyGaGSREk6Och3bdBzOJ0ILGkaW8SqbXWeV0yiasM8fR/4pAX
         PUng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715703781; x=1716308581;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=oRpWNAdQnTo0N4docaodQ5ZeZukKlehSHQL6Z/WP0yk=;
        b=N0N0KOYEh9MCvbOjUn66ihofOK1jA3Q27FuJMVN4WadYY5ssDDChCqPL/CVeDgLhE2
         eGfSXfUOerNmtAg1J30PeyjuOQoq7JhkXTEJNt/EBpQ7P2BjEr786Df29fj0dhAmJEmM
         4+t84aXy+4j3ofazJk8hxBd3aaPbs3g/3xPsq6F7duIkMGwHagYDuzFOVoYXdYtRZPWr
         4m2AusLWKxgIVQleQuD/6BEUVnnQfRpnrA3zZZXmcEHqgyD0LyL3P3MX7kLceC3SCMGu
         2EjXYU5l/hRpAw/NHIFj1HHfQOYn9s5oGmxpnYOTf6PsanvkKDnAy2MZun8NwHqPCj7l
         7DKA==
X-Gm-Message-State: AOJu0YyTdPWPNP3ROEV4G3YHQSG7q3OETn5yDhqf5K0DwAzuNceSmzCm
	u1870BqLKjSgvr5bEgDshcKbuVYWVVUAoBcscL2u/PuBpRQj4AQhd+9L0ZM+Nw==
X-Google-Smtp-Source: AGHT+IFRMQjxQ73KMM3725entLj5SIT2+iszFANr7+FRawFJniLXz66SGb/jsIgXE9GoBylDXPfRGA==
X-Received: by 2002:a17:906:48cc:b0:a59:cdc9:6fdf with SMTP id a640c23a62f3a-a5a2d54c0c9mr821509366b.4.1715703780751;
        Tue, 14 May 2024 09:23:00 -0700 (PDT)
Message-ID: <df8cc51d-60ea-452b-809e-c75aab957929@suse.com>
Date: Tue, 14 May 2024 18:22:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-15-eliasely@amazon.com> <ZkOHHtRPTqC4Zvnp@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkOHHtRPTqC4Zvnp@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 17:45, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:41PM +0000, Elias El Yandouzi wrote:
>> Until directmap gets completely removed, we'd still need to
>> keep some calls to mfn_to_virt() for xenheap pages or when the
>> directmap is enabled.
>>
>> Rename the macro to mfn_to_directmap_virt() to flag them and
>> prevent further use of mfn_to_virt().
> 
> Both here and in the following patch, I'm afraid I'm unsure of it's
> usefulness.  I'm leaning towards this being code churn for very little
> benefit.

I expect this patch is a response to an earlier comment of mine. I'm
rather worried that at the time this series actually goes in, un-audited
mfn_to_virt() uses remain (perhaps because of introduction between patch
submission and its committing). Such uses would all very likely end in
crashes or worse, but they may not be found by testing.

> Also, I'm not sure I see how the patch prevents further usage of
> mfn_to_virt(), as (for Arm) the existing macro is not removed.  If
> anything I would prefer a comment clearly stating that the macro
> operates on directmap space, and avoid the name change.

But Arm isn't switched to this sparse direct map mode, I think? At which
point uses in Arm-specific code continue to be okay.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 14 17:16:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 17:16:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721687.1125292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6vlB-0002Dx-Bl; Tue, 14 May 2024 17:16:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721687.1125292; Tue, 14 May 2024 17:16:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6vlB-0002Dq-97; Tue, 14 May 2024 17:16:17 +0000
Received: by outflank-mailman (input) for mailman id 721687;
 Tue, 14 May 2024 17:16:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0ttm=MR=amazon.co.uk=prvs=857b575ae=eliasely@srs-se1.protection.inumbo.net>)
 id 1s6vl9-0002Dh-Ma
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 17:16:15 +0000
Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com
 [207.171.188.206]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8f028a5-1215-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 19:16:13 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-9106.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 May 2024 17:16:05 +0000
Received: from EX19MTAEUA001.ant.amazon.com [10.0.17.79:22193]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.14.167:2525]
 with esmtp (Farcaster)
 id e42f7b7e-387e-4fbc-bce0-ee8dc1ea35ec; Tue, 14 May 2024 17:16:03 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUA001.ant.amazon.com (10.252.50.223) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Tue, 14 May 2024 17:16:03 +0000
Received: from [10.95.97.194] (10.95.97.194) by EX19D018EUA002.ant.amazon.com
 (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Tue, 14 May
 2024 17:16:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8f028a5-1215-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715706974; x=1747242974;
  h=message-id:date:mime-version:subject:to:cc:references:
   from:in-reply-to:content-transfer-encoding;
  bh=7WMWzzWfADKU6GG+LTImiZZ3v3HOBZvXkyxAIdszVRA=;
  b=K4lHS4UEAEaeOeFW4a0bRg22vLazQAybOF3iylOdbKv+IhLemKMYRmvp
   nCgLqqaMMWH2Sgybj30WC66VzIHzSIIStBAOBM56E/rIEGA1kfsv5tGzt
   UeB9e/rbAGc21mVYPuyozMYBKF1hZBSwOVDwoZJtq+Kle+/LoigkMk5Qj
   g=;
X-IronPort-AV: E=Sophos;i="6.08,159,1712620800"; 
   d="scan'208";a="725759101"
X-Farcaster-Flow-ID: e42f7b7e-387e-4fbc-bce0-ee8dc1ea35ec
Message-ID: <699c6487-58c9-456f-8415-e3c525fa905e@amazon.com>
Date: Tue, 14 May 2024 18:15:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, <julien@xen.org>, <pdurrant@amazon.com>,
	<dwmw@amazon.com>, Julien Grall <jgrall@amazon.com>
References: <20240513111117.68828-1-eliasely@amazon.com>
 <20240513111117.68828-2-eliasely@amazon.com> <ZkIxdtiDc_pnPWdx@macbook>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <ZkIxdtiDc_pnPWdx@macbook>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-Originating-IP: [10.95.97.194]
X-ClientProxiedBy: EX19D040UWB004.ant.amazon.com (10.13.138.91) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

Hi Roger,

On 13/05/2024 16:27, Roger Pau Monné wrote:
>> diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
>> index 2a445bb17b..1b025986f7 100644
>> --- a/xen/arch/x86/pv/domain.c
>> +++ b/xen/arch/x86/pv/domain.c
>> @@ -288,6 +288,21 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
>>                                 1U << GDT_LDT_VCPU_SHIFT);
>>   }
>>   
>> +static int pv_create_root_pt_l1tab(struct vcpu *v)
>> +{
>> +    return create_perdomain_mapping(v->domain,
>> +                                    PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v),
>> +                                    1, v->domain->arch.pv.root_pt_l1tab,
>> +                                    NULL);
>> +}
>> +
>> +static void pv_destroy_root_pt_l1tab(struct vcpu *v)
> 
> The two 'v' parameters could be const here.

I could constify the parameters but the functions wouldn't be consistent 
with the two above for gdt/ldt.

>> @@ -381,6 +412,11 @@ int pv_domain_initialise(struct domain *d)
>>       if ( rc )
>>           goto fail;
>>   
>> +    rc = create_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START,
>> +                                  PV_ROOT_PT_MAPPING_ENTRIES, NULL, NULL);
> 
> Why not use d->max_vcpus here, instead of forcing up to MAX_VIRT_CPUS?
> 
> By the time pv_domain_initialise() is called max_vcpus should already
> be initialized.  AFAICT it doesn't make a difference, because for the
> call here only the L3 table is created, as last two parameters are
> NULL, but still is more accurate to use max_vcpus.


There is no particular reason. I think we can safely use d->max_vcpus.
>> diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
>> index df015589ce..c1377da7a5 100644
>> --- a/xen/arch/x86/x86_64/entry.S
>> +++ b/xen/arch/x86/x86_64/entry.S
>> @@ -162,7 +162,15 @@ FUNC_LOCAL(restore_all_guest)
>>           and   %rsi, %rdi
>>           and   %r9, %rsi
>>           add   %rcx, %rdi
>> +
>> +        /*
>> +         * The address in the vCPU cr3 is always mapped in the per-domain
>> +         * pv_root_pt virt area.
>> +         */
>> +        imul  $PAGE_SIZE, VCPU_id(%rbx), %esi
> 
> Aren't some of the previous operations against %rsi now useless since
> it gets unconditionally overwritten here?

I think I can just get rid off of:

     and   %r9, %rsi

> and   %r9, %rsi
> [...]
> add   %rcx, %rsi

The second operation you suggested is actually used to retrieve the VA 
of the PV_ROOT_PT.

Cheers,
Elias



From xen-devel-bounces@lists.xenproject.org Tue May 14 17:49:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 17:49:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721692.1125301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6wGj-0006OL-Rf; Tue, 14 May 2024 17:48:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721692.1125301; Tue, 14 May 2024 17:48:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6wGj-0006OE-Os; Tue, 14 May 2024 17:48:53 +0000
Received: by outflank-mailman (input) for mailman id 721692;
 Tue, 14 May 2024 17:48:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6wGj-0006O8-DH
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 17:48:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6wGj-0003lQ-29; Tue, 14 May 2024 17:48:53 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6wGi-00074A-Qz; Tue, 14 May 2024 17:48:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=XTMxr9qgFNEbabVWtjxqC8Gu8FdyBZ6yMETWB/9DvL0=; b=LRoAQ79pkTJ4VNQuK/XcKDOvVs
	y1TEiSHFE7MciszkS59EExTbDlr1SAy26k62ylwJ+tD0b459pueTEi6HI9XXu22xRcXPF+2S1+Ec4
	PpdnRsNtEZYROnMz/eU1oIVu0Ip7NtSQYlW+FFOyDxfwRNz8vM+Oh19IAssCyTgljY2c=;
Message-ID: <cf646df7-0e92-4138-8b54-471c72f44cb8@xen.org>
Date: Tue, 14 May 2024 18:48:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 5/5] arm/vpci: honor access size when returning an
 error
Content-Language: en-GB
To: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-6-stewart.hildebrand@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240514143400.152280-6-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stewart,

On 14/05/2024 15:33, Stewart Hildebrand wrote:
> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> 
> Guest can try to read config space using different access sizes: 8,
> 16, 32, 64 bits. We need to take this into account when we are
> returning an error back to MMIO handler, otherwise it is possible to
> provide more data than requested: i.e. guest issues LDRB instruction
> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
> register.
> 
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

With one remark below:

Acked-by: Julien Grall <jgrall@amazon.com>

> ---
> v9->10:
> * New patch in v10.
> ---
>   xen/arch/arm/vpci.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 348ba0fbc860..aaf9d9120c3d 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -41,6 +41,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>   {
>       struct pci_host_bridge *bridge = p;
>       pci_sbdf_t sbdf;
> +    const uint8_t access_size = (1 << info->dabt.size) * 8;
> +    const uint64_t access_mask = GENMASK_ULL(access_size - 1, 0);
>       /* data is needed to prevent a pointer cast on 32bit */
>       unsigned long data;
>   
> @@ -48,7 +50,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>   
>       if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
>       {
> -        *r = ~0UL;
> +        *r = access_mask;

The name 'access_mask' is a bit confusing. I would not expect such value 
for be returned to the guest. What about 'invalid'?

Also can you confirm whether patches #4 and #5 be committed without the 
rest of the series?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 19:31:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 19:31:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721715.1125312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6xrf-00052J-WA; Tue, 14 May 2024 19:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721715.1125312; Tue, 14 May 2024 19:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6xrf-00052C-Sw; Tue, 14 May 2024 19:31:07 +0000
Received: by outflank-mailman (input) for mailman id 721715;
 Tue, 14 May 2024 19:31:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gwnU=MR=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s6xre-000526-MA
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 19:31:06 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7ed73f13-1228-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 21:31:01 +0200 (CEST)
Received: from BN0PR08CA0028.namprd08.prod.outlook.com (2603:10b6:408:142::27)
 by DM3PR12MB9434.namprd12.prod.outlook.com (2603:10b6:0:4b::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Tue, 14 May
 2024 19:30:51 +0000
Received: from BN2PEPF00004FC0.namprd04.prod.outlook.com
 (2603:10b6:408:142:cafe::60) by BN0PR08CA0028.outlook.office365.com
 (2603:10b6:408:142::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 14 May 2024 19:30:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF00004FC0.mail.protection.outlook.com (10.167.243.186) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 19:30:51 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 14:30:50 -0500
Received: from [172.25.174.161] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 14:30:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ed73f13-1228-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LSkQRB9EoVnuHyr/a7Is8mY3zM0wG6Fo2hbvxlLCJGWbJr8NlK8B8v301zU6PB7pxMARXABC/ZiJJndvw7TAkZ3J4vgoOJ8kTW3dWVQGo4L0jZvGtpSPmF1/5MNDwTi5S06ua0rTYU78ZQKyDj6IKQp31Q9v+u9TbriYHO75mKRoHRvmX6KKt6vK0KGf1M9VIa6m3LVZ7Rh1xQn52WGf4QM63Zmwco5Q5894gmXnPcHkPaaDqqfL1G0Oc8KYHnuVyajtbswMMMLityfN36dfMoMvI1bfkHxsTXynCYw2hqTUjVEs6kHyRqx7owTudA1wCHa/rp8LpZbj6jZM0ThxJQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6HiN2/ngL8lzfBK0HAiA341EyVRfzfAdaOj4pY62+5k=;
 b=WyCW0CCLjcp3qU4ntToNlI3GG1ed/VNEw9H3/LmzYeM1Yie2+XUL8Rt0Wxf82eHgY4rffXSKnxtOL8BeFEc/TW0iVXpfhAPj9OYVinifXeHDVu0c9cmPRVLFMKPjgd491O1Cj94nsZnXYLjJLc/f5GANFIoi5v4kEPSYxFgv0qxhaNITIY71fTGW9oxdpfwLkQjxX+MDQx/+0EnLsdS5LMK61SvDjnp7CVPJRV/tSIZb7UyJIkzy5bmPb1G+pdBk/ebA3II1PCmDLFBW+BANUewMXv21ZpWYV5ehJaigQrx9U1EZa9gX8zbLNbjpJPYaWt8lI/WbRA9e85hXonpjEA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6HiN2/ngL8lzfBK0HAiA341EyVRfzfAdaOj4pY62+5k=;
 b=qNRe500esttmZQbwPsCcfi6jWIctHYg7KuJ0GhduYyBFt9XUwjS+ygfWpbiXTBxYos5cjf0iNTUrNvkv13kVrkYhVPsLcQGgIduN5t9JwyK6QbKPMSYxvrhTKywxA2+oa0ZCNdk9kbE0KKZeEd6gWzTa5K96JTx8DBQNRK4IcBw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <da33086c-10ff-42c4-b85a-0697c3659b8e@amd.com>
Date: Tue, 14 May 2024 15:30:49 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/3] x86: detect PIT aliasing on ports other than
 0x4[0-3]
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <0c45155a-2beb-4e69-bca3-cdf42ba22f2b@suse.com>
 <039b9ceb-4862-4e26-a344-e47fc04bd979@suse.com>
 <0cfaeb1f-947a-4e45-9f69-a0e3f8143e2a@amd.com>
 <ecb8814f-f9bf-4eb7-86ce-48bdd34f8bf9@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <ecb8814f-f9bf-4eb7-86ce-48bdd34f8bf9@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF00004FC0:EE_|DM3PR12MB9434:EE_
X-MS-Office365-Filtering-Correlation-Id: 497bffce-d136-4015-acc7-08dc744c5d6c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bm9obDBRSTJKc0dJYVhwRDVoQlZCV2pPUlc5RGlKZm9zUkF1M3k0REVIS2d5?=
 =?utf-8?B?U29lQVgybnkvcEFqcS91YnVBdGsvaDJOQ1dMclllaHltUDhqQXBOcmtxb1hE?=
 =?utf-8?B?TXBOa2Y0a3NoY1g0aExqZ2VhR09JY0NtRTJ3MHdCWVpMVFRHUFd4a0pyb210?=
 =?utf-8?B?d3JIQlNEMzVhRDU1OThha25lL3Rydnh5Q3M2RVFiM3hkRVhHLzNqTjVTOG82?=
 =?utf-8?B?OGY5V2dYaG5FeXRFNGFVa25ZUEdXMENNNTBjS0JRenhWVm1FbmE3MnFoVXZw?=
 =?utf-8?B?STV0U0VsalZoNmozNUNjMkRDV0JxN25CWDBrVExuaWtxUXUrQzBteWs3VEhx?=
 =?utf-8?B?OEhQWWx4cUdDRllGOHlGMTh3SFhrQ0ZVTjI5emVhSUl4RXlKSUp2V2RMbzlP?=
 =?utf-8?B?enRKWXlPcUwyU0pDNTBTNWdUbHVLd3J3UW8xeGdhSURCdHJOMGFJeHg5VFc5?=
 =?utf-8?B?bjA2YklyWnNpVThJZ2JZRkxVSDAwUDBZRnRvbDNWczZadmc3WjVua1MyT2oy?=
 =?utf-8?B?YXprYTZNclIwSXRYb3lwTGFsb2NwMTY4OFlMT0RLaUJzaFo4R3RFcERFZGtV?=
 =?utf-8?B?MjAyQ3J0c2VnZmxWc2dPMDlJTE01YnpCVmduVFlrRHdGZjBYRHUyS1pYOFpH?=
 =?utf-8?B?SU9jY09oK2drcXlaRUIzdTVSUXhpUCt0ZDFIOWRiYy9nbzIvRGJkeGc1aFJ1?=
 =?utf-8?B?RUpiNHUyMjhjWDI0bGdQTTJuejlCcG9Ib0FsN0tJMXBzclkzNGFsaDg5UUlv?=
 =?utf-8?B?R2QydHVQWGpMcE03NW9JeDVCRG0yVm5ZTnJLWTQxL0pXWWpPemtLWUdCeHBn?=
 =?utf-8?B?cHl4QXZ5YnExWGJvK0J6OTd0Qmg0Rm5rdTZUczhUVWdEaEUvMGxPREJQajkr?=
 =?utf-8?B?djU0dExSUitpZHZ4a0ZQRGFjazlNbEE5K2owcHhIMXlpbUlTMHBpcFZTT1VJ?=
 =?utf-8?B?UkMwSy9GMWFiRkJiTm1CRzBSR2V2eDdZS1oyWUNvMGIwVHZGSXJqd3pyTC81?=
 =?utf-8?B?alNRdmVhMkxINEh3ekdFVXByamk3WXVLdVZCME5qNEJIdkpzMmhWTWdnVWhO?=
 =?utf-8?B?VjBMNjFVcVhnc3RZUUVMeFFGWlB1dDRwZkhzc3JJVHgvdnhpd2tpdTg0a1JN?=
 =?utf-8?B?MC9pSDZJYURybTRlamlyYXllS0FRemNySjlLSTFaQnRuU1Zvdm43QnphOU5j?=
 =?utf-8?B?bDRCRFJSYzE4RlRMU1ptZzNST3lUbXJ1SXNxUDFyM1VTNFJRRm1HbHlBWCtC?=
 =?utf-8?B?aVZOR0MvS2RaaVpCYzZUMnJpWDROc1NFR3M1MUVKZXcxWnpxOU5wdnRieGJN?=
 =?utf-8?B?K1p0eWQvbEJ0eTUrVzlKVjhrMG1uT2tOWlRGOEE4ZkVodmF2QTV4eFpTMFZl?=
 =?utf-8?B?Y1E0aG8xak9XVnBWQ2wvQUxLUC9rNlJBK0RDVnB2U3AvMFp5dWllNFdaUHJs?=
 =?utf-8?B?Yk9EWGgzcFdMaVdhMDY4NkpkTGxXY3dnaGI4QzFFZFphUzlUUm9ic2dqOTRK?=
 =?utf-8?B?Z1IrTE9yN1Q2Vmtaa1ZZdUxrUDRFVWQ5a29hM0YwNTRBVjNQRXhkMXRWNWpW?=
 =?utf-8?B?b0IwbGc0ZzJzbEpyTnVDRUZOQ1BNb1RnMlAwQ3pMQkhocVp2emVRZ05saTV4?=
 =?utf-8?B?Z1ZvdmxtQTFjcEVFUFZvaWNRUjJaK3EreWpSWWRoUTdHVk9CNzlXTy8vV2Jr?=
 =?utf-8?B?ZVRVRFZqSDA2b2g0MEtFdHRRWVZIQnh2VE9OQW1vNXhiV2FBclFTVDRvZDkz?=
 =?utf-8?B?MXVtQktPeTJUbkZyd1Robm4wTktlc2hYL3lIODByRDhnVkxtQUMyemJITFkw?=
 =?utf-8?B?SkgyTkUxWnozRzdNMExacFZJTlhqQ1lqUDBXMVRqVW5xTjhzRjd0NFo4SVNF?=
 =?utf-8?Q?jnF1rSKKAgp6C?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 19:30:51.3038
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 497bffce-d136-4015-acc7-08dc744c5d6c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF00004FC0.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9434

On 2024-05-14 03:43, Jan Beulich wrote:
> On 10.05.2024 19:40, Jason Andryuk wrote:
>> On 2023-12-18 09:48, Jan Beulich wrote:
>>> --- a/xen/arch/x86/time.c
>>> +++ b/xen/arch/x86/time.c
>>> @@ -425,6 +425,72 @@ static struct platform_timesource __init
>>>        .resume = resume_pit,
>>>    };
>>>    
>>> +unsigned int __initdata pit_alias_mask;
>>> +
>>> +static void __init probe_pit_alias(void)
>>> +{
>>> +    unsigned int mask = 0x1c;
>>> +    uint8_t val = 0;
>>> +
>>> +    if ( !opt_probe_port_aliases )
>>> +        return;
>>> +
>>> +    /*
>>> +     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
>>> +     * count is loaded independent of counting being / becoming enabled.  Thus
>>> +     * we have a 16-bit value fully under our control, to write and then check
>>> +     * whether we can also read it back unaltered.
>>> +     */
>>> +
>>> +    /* Turn off speaker output and disable channel 2 counting. */
>>> +    outb(inb(0x61) & 0x0c, 0x61);
>>> +
>>> +    outb((2 << 6) | (3 << 4) | (0 << 1), PIT_MODE); /* Mode 0, LSB/MSB. */
>>
>> Channel 2, Lobyte/Hibyte, 0b000 Mode 0, (Binary)
>>
>> #define PIT_MODE_CH2 (2 << 6)
>> #define PIT_MODE0_16BIT ((3 << 4) | (0 << 1))
>>
>> outb(PIT_MODE_CH2 | PIT_MODE0_16BIT, PIT_MODE);
> 
> Hmm. I can certainly see the value of introducing such #define-s, but then
> while doing so one ought to also adjust other code using constants as done
> here (for consistency).

I had to look up all these bit values, so I think it's nicer with 
#defines-s.  Particularly, using PIT_MODE0_16BIT for the programming and 
checking shows the relationship.  I wasn't looking to make more work for 
you.  This function is self-contained, so just using them here for the 
time being seems reasonable.

>>> +
>>> +    do {
>>> +        uint8_t val2;
>>> +        unsigned int offs;
>>> +
>>> +        outb(val, PIT_CH2);
>>> +        outb(val ^ 0xff, PIT_CH2);
>>> +
>>> +        /* Wait for the Null Count bit to clear. */
>>> +        do {
>>> +            /* Latch status. */
>>> +            outb((3 << 6) | (1 << 5) | (1 << 3), PIT_MODE);
>>
>> Read-back, Latch status,  read back timer channel 2
> 
> Was this meant as a request to extend the comment? If so, not quite,
> as the line doesn't include any read-back. If not, I'm in trouble seeing
> what you mean to tell me here (somewhat similar also for the first line
> of your earlier comment still visible in context above).

Sorry, these were my notes as I was interpreting the bits.  I should 
have removed them from the email before sending as they aren't 
actionable comments.  Read back was in reference to writing (3 << 6) to 
the mode - not the action of read backing back the value.

>>> +
>>> +            /* Try to make sure we're actually having a PIT here. */
>>> +            val2 = inb(PIT_CH2);
>>> +            if ( (val2 & ~(3 << 6)) != ((3 << 4) | (0 << 1)) )
>>
>> if ( (val2 & PIT_RB_MASK) != PIT_MODE0_16BIT )
>>
>> I think particularly a define for PIT_MODE0_16BIT would be helpful to
>> show what is expected to be the same.
>>
>>> +                return;
>>> +        } while ( val2 & (1 << 6) );
>>
>> I think Roger might have mentioned on an earlier version - would it make
>> sense to have a counter to prevent looping forever?
> 
> Well, as before: The issue with bounding such loops is that the bound is
> going to be entirely arbitrary (and hence easily too large / too small).

Ah, yes.  Your response had slipped my mind.

>> Also, FYI, I tested the series.  My test machine didn't show any aliasing.
> 
> That likely was an AMD one then? It's only Intel chipsets I've seen aliasing
> on so far, but there it's (almost) all of them (with newer data sheets even
> stating that behavior). We could, beyond shim, make the option default in
> patch 1 be "false" for systems with AMD CPUs (on the assumption that those
> wouldn't have Intel chipsets).

Indeed, it was an AMD system, but my sample size is 1.

I didn't realize this was motivated by aliasing being common on Intel 
chipsets.  I think that would be useful to include in the commit messages.

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Tue May 14 19:32:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 19:32:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721719.1125322 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6xtQ-0005dD-Dh; Tue, 14 May 2024 19:32:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721719.1125322; Tue, 14 May 2024 19:32:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6xtQ-0005d6-AE; Tue, 14 May 2024 19:32:56 +0000
Received: by outflank-mailman (input) for mailman id 721719;
 Tue, 14 May 2024 19:32:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6xtO-0005cw-TM; Tue, 14 May 2024 19:32:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6xtO-0005WO-Qy; Tue, 14 May 2024 19:32:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s6xtO-0008Rs-Fb; Tue, 14 May 2024 19:32:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s6xtO-0005hO-FF; Tue, 14 May 2024 19:32:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sMsBWQZusE4UF1HQYkg32gYYz1t9lVRXSlJ4tcbSzlw=; b=riXDHiFY+AiJU8GQ6NmQ3kqh8T
	5/EhsL1lFtq5FB2/N7UB7s5rthQEs1SwCuTLcRRfQvFUSRSoKUEUfH1jBJXVBNYoLyxlolCtsv8UY
	4NlB9Ikqrgr+PsAy70rOZpt5PmBqjON61fyhcCo8IVV/yg65DVeVIKKCDNeX+RdBMWsg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185996-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185996: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a5131c3fdf2608f1c15f3809e201cf540eb28489
X-Osstest-Versions-That:
    linux=6d1346f1bcbf2724dee8af013cdab9f7b581435b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 May 2024 19:32:54 +0000

flight 185996 linux-linus real [real]
flight 185997 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/185996/
http://logs.test-lab.xenproject.org/osstest/logs/185997/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 19 guest-localmigrate/x10 fail pass in 185997-retest
 test-armhf-armhf-xl-arndale   8 xen-boot            fail pass in 185997-retest
 test-armhf-armhf-examine      8 reboot              fail pass in 185997-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 185990
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 185997 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 185997 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185990
 test-armhf-armhf-libvirt-vhd  8 xen-boot                     fail  like 185990
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 185990
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185990
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185990
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a5131c3fdf2608f1c15f3809e201cf540eb28489
baseline version:
 linux                6d1346f1bcbf2724dee8af013cdab9f7b581435b

Last test of basis   185990  2024-05-13 16:43:50 Z    1 days
Failing since        185992  2024-05-14 00:41:54 Z    0 days    2 attempts
Testing same since   185996  2024-05-14 09:18:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
  Adam Guerin <adam.guerin@intel.com>
  Ai Chao <aichao@kylinos.cn>
  Akhil R <akhilrajeev@nvidia.com>
  Akira Yokosawa <akiyks@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Aleksandr Mishin <amishin@t-argos.ru>
  Alex Shi <alexs@kernel.org>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Lobakin <aleksander.lobakin@intel.com>
  Alexey Dobriyan (Yandex) <adobriyan@gmail.com>
  Alexey Dobriyan <adobriyan@gmail.com>
  Alice Ryhl <aliceryhl@google.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrea Parri <parri.andrea@gmail.com>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Animesh Agarwal <animeshagarwal28@gmail.com>
  Anuj Gupta <anuj20.g@samsung.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Avadhut Naik <avadhut.naik@amd.com>
  Bagas Sanjaya <bagasdotme@gmail.com>
  Benno Lossin <benno.lossin@proton.me>
  Bilbao, Carlos <carlos.bilbao@amd.com>
  Bill O'Donnell <bodonnel@redhat.com>
  Bingsong Si <sibs@chinatelecom.cn>
  Bird, Tim <Tim.Bird@sony.com>
  Boqun Feng <boqun.feng@gmail.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Breno Leitao <leitao@debian.org>
  Brian Gerst <brgerst@gmail.com>
  Brian Kubisiak <brian@kubisiak.com>
  Carlos Bilbao <carlos.bilbao@amd.com>
  Chang S. Bae <chang.seok.bae@intel.com>
  Chen Jun <chenjun102@huawei.com>
  Chen Ni <nichen@iscas.ac.cn>
  Chenghai Huang <huangchenghai2@huawei.com>
  Christian Brauner <brauner@kernel.org>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Coly Li <colyli@suse.de>
  Conor Dooley <conor.dooley@microchip.com>
  Corentin Labbe <clabbe@baylibre.com>
  Damian Muszynski <damian.muszynski@intel.com>
  Damien Le Moal <dlemoal@kernel.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Jordan <daniel.m.jordan@oracle.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Gstir <david@sigma-star.at>
  David Howells <dhowells@redhat.com>
  David Oberhollenzer <david.oberhollenzer@sigma-star.at>
  David Rientjes <rientjes@google.com>
  David Sterba <dsterba@suse.com>
  David Wei <dw@davidwei.uk>
  Dawid Osuchowski <linux@osuchow.ski>
  Dennis Lam <dennis.lamerice@gmail.com>
  Dennis Maisenbacher <dennis.maisenbacher@wdc.com>
  Dirk Behme <dirk.behme@de.bosch.com>
  Dongliang Mu <dzm91@hust.edu.cn>
  Dr. David Alan Gilbert <linux@treblig.org>
  Dustin L. Howett <dustin@howett.net>
  Edward Adam Davis <eadavis@qq.com>
  Eric Biggers <ebiggers@google.com>
  Eric Sandeen <sandeen@redhat.com>
  Erick Archer <erick.archer@gmx.com>
  Erick Archer <erick.archer@outlook.com>
  Federico Vaga <federico.vaga@vaga.pv.it>
  Florian-Ewald Mueller <florian-ewald.mueller@ionos.com>
  Frank Li <Frank.Li@nxp.com>
  Frederic Weisbecker <frederic@kernel.org>
  Gabriel Krisman Bertazi <krisman@suse.de>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Giovanni Cabiddu <giovanni.cabiddu@intel.com>
  Guenter Roeck <linux@roeck-us.net>
  Guixiong Wei <weiguixiong@bytedance.com>
  Gustavo A. R. Silva <gustavoars@kernel.org>
  H. Peter Anvin (Intel) <hpa@zytor.com>
  H.J. Lu <hjl.tools@gmail.com>
  Haifeng Xu <haifeng.xu@shopee.com>
  Hailey Mothershead <hailmo@amazon.com>
  Hans Holmberg <hans.holmberg@wdc.com>
  Haoyang Liu <tttturtleruss@hust.edu.cn>
  Heiko Carstens <hca@linux.ibm.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Hyunmin Lee <hyunminlr@gmail.com>
  Ilya Denisyev <dev@elkcl.ru>
  INAGAKI Hiroshi <musashino.open@gmail.com>
  Ingo Molnar <mingo@kernel.org>
  Ivan Orlov <ivan.orlov0322@gmail.com>
  Ivan Orlov <ivan.orlov@codethink.co.uk>
  Jack Wang <jinpu.wang@ionos.com>
  Jakub Kicinski <kuba@kernel.org>
  James Bottomley <James.Bottomley@HansenPartnership.com>
  Jarkko Sakkinen <jarkko.sakkinen@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org> # for TRUSTED_KEYS_TPM
  Jens Axboe <axboe@kernel.dk>
  Jerry Snitselaar <jsnitsel@redhat.com>
  Jeungwoo Yoo <casionwoo@gmail.com>
  Jia Jie Ho <jiajie.ho@starfivetech.com>
  Jianfeng Wang <jianfeng.w.wang@oracle.com>
  Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
  Jini Susan George <jinisusan.george@amd.com>
  Jiri Olsa <jolsa@kernel.org>
  Joachim Vandersmissen <git@jvdsn.com>
  Joel Fernandes (Google) <joel@joelfernandes.org>
  Joel Granados <j.granados@samsung.com>
  Johannes Berg <johannes.berg@intel.com>
  Johannes Thumshirn <johannes.thumshirn@wdc.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Garry <john.g.garry@oracle.com>
  Jonathan Corbet <corbet@lwn.net>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Juergen Gross <jgross@suse.com>
  Julian Stecklina <julian.stecklina@cyberus-technology.de>
  Justin Stitt <justinstitt@google.com>
  K Prateek Nayak <kprateek.nayak@amd.com>
  Karel Balej <balejk@matfyz.cz>
  Karthikeyan Ramasubramanian <kramasub@chromium.org>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Keith Busch <kbusch@kernel.org>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Krzysztof Błaszkowski <kb@sysmikro.com.pl>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kyle Huey <khuey@kylehuey.com>
  Kyle Huey <me@kylehuey.com>
  Lai Jiangshan <jiangshan.ljs@antgroup.com>
  Laura Nao <laura.nao@collabora.com>
  Li Hua <lihua@email.com>
  Li Nan <linan122@huawei.com>
  Li RongQing <lirongqing@baidu.com>
  Li Zhijian <lizhijian@fujitsu.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lothar Rubusch <l.rubusch@gmail.com>
  Luca Weiss <luca.weiss@fairphone.com>
  Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
  Luis Henriques (SUSE) <luis.henriques@linux.dev>
  Luis Henriques <lhenriques@suse.de>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Lukas Wunner <lukas@wunner.de>
  Lukasz Luba <lukasz.luba@arm.com>
  Maki Hatano <Maki.Y.Hatano@gmail.com>
  Marco Elver <elver@google.com>
  Marek Vasut <marex@denx.de>
  Mario Limonciello <superm1@gmail.com>
  Mark Rutland <mark.rutland@arm.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Matt Turner <mattst88@gmail.com>
  Matthew Mirvish <matthew@mm12.xyz>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Max Filippov <jcmvbkbc@gmail.com>
  Maxime Méré <maxime.mere@foss.st.com>
  Maíra Canal <mcanal@igalia.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Haener <michael.haener@siemens.com>
  Miguel Ojeda <ojeda@kernel.org>
  Mikko Perttunen <mperttunen@nvidia.com>
  Ming Lei <ming.lei@redhat.com>
  Mingwei Zhang <mizhang@google.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Namhyung Kim <namhyung@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  Neeraj Upadhyay (AMD) <neeraj.iitr10@gmail.com>
  Neeraj Upadhyay <neeraj.upadhyay@kernel.org>
  Nell Shamrell-Harrington <nells@linux.microsoft.com>
  Nhat Pham <nphamcs@gmail.com>
  Nicholas Piggin <npiggin@gmail.com>
  Nick Desaulniers <ndesaulniers@google.com>
  Nikita Kiryushin <kiryushin@ancud.ru>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Pankaj Gupta <pankaj.gupta@nxp.com>
  Paul E. McKenney <paulmck@kernel.org>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Pavel Begunkov <asml.silence@gmail.com>
  Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Prashant Malani <pmalani@chromium.org>
  Qais Yousef <qyousef@layalina.io>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rafael J. Wysocki <rafael@kernel.org>
  Raghav Narang <dev@raxyte.com>
  Remington Brasga <rbrasga@uci.edu>
  Richard Weinberger <richard@nod.at>
  Roman Smirnov <r.smirnov@omp.ru>
  Ruyi Zhang <ruyi.zhang@samsung.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Sangyun Kim <sangyun.kim@snu.ac.kr>
  Sarat Mandava <mandavasarat@gmail.com>
  Saurabh Sengar <ssengar@linux.microsoft.com>
  Saurav Shah <sauravshah.31@gmail.com>
  Shrikanth Hegde <sshegde@linux.ibm.com>
  Silvio Gissi <sifonsec@amazon.com>
  Siming Wan <siming.wan@intel.com>
  Song Liu <song@kernel.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  syzbot+0a3683a0a6fecf909244@syzkaller.appspotmail.com
  Tejun Heo <tj@kernel.org>
  Thierry Reding <treding@nvidia.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Blum <thorsten.blum@toblux.com>
  Thorsten Leemhuis <linux@leemhuis.info>
  Tim Bird <tim.bird@sony.com>
  Tim Chen <tim.c.chen@linux.intel.com>
  Tom Lendacky <thomas.lendacky@amd.com>
  Tom Zanussi <tom.zanussi@linux.intel.com>
  Tong Tiangen <tongtiangen@huawei.com>
  Tony Luck <tony.luck@intel.com>
  Tyler Hicks (Microsoft) <code@tyhicks.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uladzislau Rezki (Sony) <urezki@gmail.com>
  Uros Bizjak <ubizjak@gmail.com>
  Usama Arif <usamaarif642@gmail.com>
  Utkarsh Tripathi <utripathi2002@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Valentin Schneider <vschneid@redhat.com>
  Vignesh Balasubramanian <vigbalas@amd.com>
  Vijay Nag <nagvijay@microsoft.com>
  Vincent Guittot <vincent.guittot@linaro.org>
  Viresh Kumar <viresh.kumar@linaro.org>
  Vitaly Chikunov <vt@altlinux.org>
  Vlastimil Babka <vbabka@suse.cz>
  Waiman Long <longman@redhat.com>
  Wedson Almeida Filho <walmeida@microsoft.com>
  Wei Yang <richard.weiyang@gmail.com>
  Wenkai Lin <linwenkai6@hisilicon.com>
  Will Deacon <will@kernel.org>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Xin Li (Intel) <xin@zytor.com>
  Xin Zeng <xin.zeng@intel.com>
  Xiongwei Song <xiongwei.song@windriver.com>
  Xiu Jianfeng <xiujianfeng@huawei.com>
  Yang Li <yang.lee@linux.alibaba.com>
  Yanteng Si <siyanteng@loongson.cn>
  Yu Kuai <yukuai3@huawei.com>
  YueHaibing <yuehaibing@huawei.com>
  Yujie Liu <yujie.liu@intel.com>
  Yuntao Wang <ytcoode@gmail.com>
  Zenghui Yu <zenghui.yu@linux.dev>
  Zhang Rui <rui.zhang@intel.com>
  Zhang Yi <yi.zhang@huawei.com>
  Zhu Yanjun <yanjun.zhu@linux.dev>
  Zqiang <qiang.zhang1211@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6d1346f1bcbf..a5131c3fdf26  a5131c3fdf2608f1c15f3809e201cf540eb28489 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721730.1125352 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yai-0003Ib-Bs; Tue, 14 May 2024 20:17:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721730.1125352; Tue, 14 May 2024 20:17:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yai-0003IS-96; Tue, 14 May 2024 20:17:40 +0000
Received: by outflank-mailman (input) for mailman id 721730;
 Tue, 14 May 2024 20:17:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yag-0002pc-OM
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:38 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 022e98eb-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:38 +0200 (CEST)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-34db6a29998so4736553f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:38 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502baacfdesm14649479f8f.73.2024.05.14.13.17.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 022e98eb-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717857; x=1716322657; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=QODbdNtYx//N7B7NxU/Urb2ggSQNvaYVgLtnma262hw=;
        b=b2WRmEMnJ5/HYPe/Vsnn8Cv6P4ryc4vRfepLZBSToq5QtgBHliENdzTiEmFF5dMfB4
         hxNT1JZB4w6OFiV48WZegb7CpSoLlXx6r4WiVKmaud5XRYt+tPsP2nm4a1FtMTY9AYRd
         1wXFa3/6fivvuzmA7UJUbJ97wlKzCAuv1JIvkZwkLouq0AK/ZbZRGfekM+GeYbQxwK2D
         tq3/W6Ly2ctc1SDYqDD6boePr7rMLy/xsjbHAIi3U8sztKJMnyVZFYL8j/3diN5QxSJ5
         mIkPmrtbAyKcIGsvGbz45+cAYsX0bwT4ZpyumqHKf9VXWyHmGUlHFi5VAdxYtyHjgMUa
         t0sQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717857; x=1716322657;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=QODbdNtYx//N7B7NxU/Urb2ggSQNvaYVgLtnma262hw=;
        b=wbu74hm/zIy4SOlhGW5mTzqiBCwskaejVyvrp3YAjPhrK/dfj74bOZpgjqdDZwjKHN
         LRDuisvGVN8sWFsvVjwevHq3tqv79Rn6CbpvLFSXxEi+6spqKGuwDQFPXEtRXWLxadwl
         IKyc/ILX5CCdolzow/RrTx+wi2tOJiDLkSIHOpCIEIkDYFyQsyY/sZhHPeL8TEp5idUq
         mFSovniqwQ1VIxB4JGO8JkUon9D8o1+ZZwGlvifpDqqNh7GakXicTTIDY0TaeQEZ4JJK
         AHAZpmLM99AsrOw+xC6BzIQDy0tdMwHC8fXXNaesyVelYnaW4o7YXcZdTTsvJZeLp1Qp
         9ZBw==
X-Gm-Message-State: AOJu0YxeRBLwApF6CvqmPQNUMdsDSS00fWMP47ACO4VxcrO2HfpQ3sju
	jJlPbedZlXkvVY/1uXPsYEohtZqnEynia3xkWA21YoW5VzcL6jQz7INpDUAy2dE=
X-Google-Smtp-Source: AGHT+IHpd7msz+LqA1MbybkOFxoKsVL/RYXHvO49nMw1p4/Qd434paujy0JPmUhMPHj+Sv1rXxQIhg==
X-Received: by 2002:a5d:58ce:0:b0:343:dc46:bb4b with SMTP id ffacd0b85a97d-3504aa62e41mr9383601f8f.59.1715717856670;
        Tue, 14 May 2024 13:17:36 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 2/9] xen/arm64: smc: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:07 +0200
Message-Id: <20240504115514.1063509-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/smc.S | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/smc.S b/xen/arch/arm/arm64/smc.S
index fc6b676e2e..68b05e8ddd 100644
--- a/xen/arch/arm/arm64/smc.S
+++ b/xen/arch/arm/arm64/smc.S
@@ -19,7 +19,7 @@
  *                          register_t a6, register_t a7,
  *                          struct arm_smccc_res *res)
  */
-ENTRY(__arm_smccc_1_0_smc)
+FUNC(__arm_smccc_1_0_smc)
         smc     #0
         ldr     x4, [sp]
         cbz     x4, 1f          /* No need to store the result */
@@ -27,12 +27,13 @@ ENTRY(__arm_smccc_1_0_smc)
         stp     x2, x3, [x4, #SMCCC_RES_a2]
 1:
         ret
+END(__arm_smccc_1_0_smc)
 
 /*
  * void arm_smccc_1_2_smc(const struct arm_smccc_1_2_regs *args,
  *                        struct arm_smccc_1_2_regs *res)
  */
-ENTRY(arm_smccc_1_2_smc)
+FUNC(arm_smccc_1_2_smc)
     /* Save `res` and free a GPR that won't be clobbered by SMC call */
     stp     x1, x19, [sp, #-16]!
 
@@ -69,3 +70,4 @@ ENTRY(arm_smccc_1_2_smc)
     /* Restore original x19 */
     ldp     xzr, x19, [sp], #16
     ret
+END(arm_smccc_1_2_smc)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721729.1125337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yah-0002tD-1k; Tue, 14 May 2024 20:17:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721729.1125337; Tue, 14 May 2024 20:17:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yag-0002sE-SN; Tue, 14 May 2024 20:17:38 +0000
Received: by outflank-mailman (input) for mailman id 721729;
 Tue, 14 May 2024 20:17:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yaf-0002pZ-TN
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:37 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ffccdc4e-122e-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:17:34 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41fc5645bb1so44415425e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:34 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-351b4af0b0asm7917767f8f.100.2024.05.14.13.17.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffccdc4e-122e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717853; x=1716322653; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=uqrG0bhOcLLfykWM1D6eXpiAcV0N0AIf9kwEdF2aCpQ=;
        b=QPhsEGln7zMI0paymqI9LnKUw/F/g7AYhxU7u4CFUjN0uzB6GIAPMwc2i+vLuqFrwv
         PmDnGBt5ZXQZuqdf1qgYTwf2YrkDNult4M9sSEYZb6ZBjCZzq7rQAFoZpQxyjFS2jDsb
         QJtHXxFdKl++xu3gFk6uASkGfaDUAEkel9enIJjB+yNNK2ZXXjJstHJNRBSXBpaatM4M
         VpQ7QxU2ML1cQ2LWt8OPQCS8jyam7i3o3oMnvVb9p4bLi0ai3i2Wxn4lGD5+vbtqh+SB
         5nFKvXFxS6PPGrKAoPGCRRLpSUIB0yK1t6E/qOLgpTF9Nstzfn22PBya9FF3ex9VktiD
         Y9aA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717853; x=1716322653;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uqrG0bhOcLLfykWM1D6eXpiAcV0N0AIf9kwEdF2aCpQ=;
        b=ts0oq6wKODpsPzKBMufBg4D4MeXA6/c3O1Eh7OukXNr4hVQekvdG2hsAkC/aUBAjf7
         c7DD0KWTxqgr5sQk9Gk4iwoJXdNqMqU9MY1cp9JFOaA9RsKoI0iaro223Et/TRP7aGfD
         ssvc5Fyu9e62BNDNOr+OuHYUY2Criv/nH2RYZKCF0+4+Ctpty593SDIYQDbgDhSzoVRR
         AGEP3RMDz9Haqbf8HeWkE++GKxDYrb0UH9Ot5lJ0Al5RJM5nAGi0VNj3ifK9STG91D/6
         Qts+cd4FCkPSIiKpByNF9bsLz6TETX4u+tJwCxqTvBGpunodbwxnHxnAbba5sw06D6R6
         Yzwg==
X-Gm-Message-State: AOJu0Yw3ycLuC9aY5DoTKhirLzNDyR9UOumcw0aQyyFk62nAD13YbM/m
	vRs3OUx/RaSulXV+MQyaV+6zfldZEWe78XVvNW1bPL8iuYaohswZgsxCbKjCFCo=
X-Google-Smtp-Source: AGHT+IFeAZwZVdqqi5E44z5On/1TXiVw9bvDcBC52b21ktHy6qDWgfRBLzUqOkBI50H4xpxbN9jIyg==
X-Received: by 2002:a05:600c:4585:b0:41f:cf63:3ace with SMTP id 5b1f17b1804b1-41fead65ab1mr94015115e9.34.1715717852599;
        Tue, 14 May 2024 13:17:32 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 0/9] xen/arm: arm64: Annotate code symbols
Date: Sat,  4 May 2024 13:55:05 +0200
Message-Id: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

On the way towards Xen safety certification we're evaluating the use
of tools to collect code-coverage/profiling information from execution
traces. Some tools rely on ELF symbols for code being declared with
type FUNC and having a symbol size.

We currently annotate some symbols but not all. Also, there seems to be
different ways to do the annotation.

This series now converts all the .S files under xen/arm/arm64 to use
the macros from xen/linkage.h and also adds missing code symbol
annotations for global symbols. There are still some local labels that
are left unmodified.

Best regards,
Edgar

ChangeLog:

v3 -> v4:
* Add newline between FUNC_LOCAL and PRINT_ID.
* Use LABEL instead of GLOBAL in bpi.S.
* Changed .global -> LABEL in entry.S (dropped Stefanos RB tag).

v2 -> v3:
* Convert and add annotations for all of xen/arm/arm64.
* Fix commit message for entry.S.

v1 -> v2:
* Drop RFC.
* Squash into a single patch to avoid confusion.


Edgar E. Iglesias (9):
  xen/arm64: entry: Add missing code symbol annotations
  xen/arm64: smc: Add missing code symbol annotations
  xen/arm64: sve: Add missing code symbol annotations
  xen/arm64: head: Add missing code symbol annotations
  xen/arm64: debug: Add missing code symbol annotations
  xen/arm64: bpi: Add missing code symbol annotations
  xen/arm64: mmu/head: Add missing code symbol annotations
  xen/arm64: cache: Use the generic xen/linkage.h macros
  xen/arm64: lib: Use the generic xen/linkage.h macros

 xen/arch/arm/arm64/bpi.S         | 20 ++++----
 xen/arch/arm/arm64/cache.S       |  4 +-
 xen/arch/arm/arm64/debug.S       |  6 ++-
 xen/arch/arm/arm64/entry.S       | 78 ++++++++++++++++++++------------
 xen/arch/arm/arm64/head.S        | 51 +++++++++++----------
 xen/arch/arm/arm64/lib/memchr.S  |  4 +-
 xen/arch/arm/arm64/lib/memcmp.S  |  4 +-
 xen/arch/arm/arm64/lib/memcpy.S  |  4 +-
 xen/arch/arm/arm64/lib/memmove.S |  4 +-
 xen/arch/arm/arm64/lib/memset.S  |  4 +-
 xen/arch/arm/arm64/lib/strchr.S  |  4 +-
 xen/arch/arm/arm64/lib/strcmp.S  |  4 +-
 xen/arch/arm/arm64/lib/strlen.S  |  4 +-
 xen/arch/arm/arm64/lib/strncmp.S |  4 +-
 xen/arch/arm/arm64/lib/strnlen.S |  4 +-
 xen/arch/arm/arm64/lib/strrchr.S |  4 +-
 xen/arch/arm/arm64/mmu/head.S    | 29 ++++++------
 xen/arch/arm/arm64/smc.S         |  6 ++-
 xen/arch/arm/arm64/sve-asm.S     |  9 ++--
 19 files changed, 142 insertions(+), 105 deletions(-)

-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721732.1125372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yal-0003oP-RU; Tue, 14 May 2024 20:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721732.1125372; Tue, 14 May 2024 20:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yal-0003oI-N0; Tue, 14 May 2024 20:17:43 +0000
Received: by outflank-mailman (input) for mailman id 721732;
 Tue, 14 May 2024 20:17:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yak-0002pc-Hm
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:42 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 045ad2f8-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:41 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-420180b5898so16412385e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:41 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f87c235b4sm239768445e9.11.2024.05.14.13.17.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 045ad2f8-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717860; x=1716322660; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oZl/wuDBhCq4h/1HN03exKmz7nONAZ/OXRP84qtToi4=;
        b=cUS6RI2fYIZoQzTxdL7pMcZeVMHv4yIkCUY/C+c0V1w8Qdbb7PozFLnJSVls0ib+fS
         Zz5/BWgHFLjCVewOI9uVhyeJG4DRUBv2QiGplGCO6sHC4k9pV4Er8zPH5f2ozjbO1CnW
         TDBZJWoy8CzcQledsNpH76vbk8kKzTiDNyWyTTSXp6kG8t2TlMGAQH+UjZiCAKRPWNSf
         0VbTIACuZHUMWh3o1edmvMv4SJOCwg8wDnFZpSiTblfxOBdLxtkLrChwNEc8SxUExq6l
         doNdE7XbmrAR77RYKyx6MUGuq/B9zOSUpObjQ3D1Vj8RaI+GPoPHwFw5Ip8b74Is2RdZ
         o7bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717860; x=1716322660;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oZl/wuDBhCq4h/1HN03exKmz7nONAZ/OXRP84qtToi4=;
        b=Jn4Uv/v1vb4aCro1cpE2iCkFXpTcQoRJu3FJiXuiANePyI0nLvZZVanuhk7C/6l2Oy
         VPoRRfIAJc5JyoCjipqY1FSCArpclpytCzPn/LP/sXTmjzu19QJryPTMwNaJLRbqUgTu
         kZIQXKhotVKSEWGKZUA8P3QnvilWUJogCtavFPd7g8R5wSensyRmEkgNQBhoXugWQLKf
         UZHgsg1sUydKuw4U78VrqJnNFHhekBeNrV2kDC00KJXbWxTmY0l5bUC8vwpOo5205zer
         WRkE8HFDlnBxjbIITmcznhSnVOh+BEPJ/hwgvK0dAAUAvEiE8FxrhsT6OQm6SZnlPWTf
         nYPg==
X-Gm-Message-State: AOJu0YwlA1cCYUCGZy2a4jkhgw2Jj8/Pn4kXqKRy5gNWs4vNKLKlDbTO
	bxdYVmw2Cr7paYNo8QmFRgWYi7spKqnwj5oEETg5w/C9mOPr5p6q/OZsVyH9z+k=
X-Google-Smtp-Source: AGHT+IFuSD60ZS4xJWrTf447rZXLlq7kk0j0gxhUPXxjzw8P3vbS1Ckr0YK/AKTwAQytA3rbxmA0RA==
X-Received: by 2002:a05:600c:4513:b0:41f:df08:5ef7 with SMTP id 5b1f17b1804b1-41feac5a4a7mr100198425e9.28.1715717860424;
        Tue, 14 May 2024 13:17:40 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 4/9] xen/arm64: head: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:09 +0200
Message-Id: <20240504115514.1063509-5-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/head.S | 51 +++++++++++++++++++++------------------
 1 file changed, 27 insertions(+), 24 deletions(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index fb297e9eb5..2fa07dc3a0 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -90,7 +90,7 @@
  * 4K-aligned address.
  */
 
-GLOBAL(start)
+FUNC(start)
         /*
          * DO NOT MODIFY. Image header expected by Linux boot-loaders.
          */
@@ -102,6 +102,7 @@ efi_head:
          */
         add     x13, x18, #0x16
         b       real_start           /* branch to kernel start */
+END(start)
         .quad   0                    /* Image load offset from start of RAM */
         .quad   _end - start         /* Effective size of kernel image, little-endian */
         .quad   __HEAD_FLAGS         /* Informative flags, little-endian */
@@ -223,7 +224,7 @@ section_table:
         .align  5
 #endif /* CONFIG_ARM_EFI */
 
-real_start:
+FUNC_LOCAL(real_start)
         /* BSS should be zeroed when booting without EFI */
         mov   x26, #0                /* x26 := skip_zero_bss */
 
@@ -263,9 +264,9 @@ primary_switched:
         mov   x1, x21                /* x1 := paddr(FDT) */
         ldr   x2, =start_xen
         b     launch
-ENDPROC(real_start)
+END(real_start)
 
-GLOBAL(init_secondary)
+FUNC(init_secondary)
         msr   DAIFSet, 0xf           /* Disable all interrupts */
 
         /* Find out where we are */
@@ -304,7 +305,7 @@ secondary_switched:
         /* Jump to C world */
         ldr   x2, =start_secondary
         b     launch
-ENDPROC(init_secondary)
+END(init_secondary)
 
 /*
  * Check if the CPU has been booted in Hypervisor mode.
@@ -313,7 +314,7 @@ ENDPROC(init_secondary)
  *
  * Clobbers x0 - x5
  */
-check_cpu_mode:
+FUNC_LOCAL(check_cpu_mode)
         PRINT_ID("- Current EL ")
         mrs   x5, CurrentEL
         print_reg x5
@@ -329,7 +330,7 @@ check_cpu_mode:
         PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n")
         PRINT_ID("- Please update the bootloader -\r\n")
         b fail
-ENDPROC(check_cpu_mode)
+END(check_cpu_mode)
 
 /*
  * Zero BSS
@@ -339,7 +340,7 @@ ENDPROC(check_cpu_mode)
  *
  * Clobbers x0 - x3
  */
-zero_bss:
+FUNC_LOCAL(zero_bss)
         /* Zero BSS only when requested */
         cbnz  x26, skip_bss
 
@@ -353,14 +354,14 @@ zero_bss:
 
 skip_bss:
         ret
-ENDPROC(zero_bss)
+END(zero_bss)
 
 /*
  * Initialize the processor for turning the MMU on.
  *
  * Clobbers x0 - x3
  */
-cpu_init:
+FUNC_LOCAL(cpu_init)
         PRINT_ID("- Initialize CPU -\r\n")
 
         /* Set up memory attribute type tables */
@@ -399,7 +400,7 @@ cpu_init:
          */
         msr spsel, #1
         ret
-ENDPROC(cpu_init)
+END(cpu_init)
 
 /*
  * Setup the initial stack and jump to the C world
@@ -411,7 +412,7 @@ ENDPROC(cpu_init)
  *
  * Clobbers x3
  */
-launch:
+FUNC_LOCAL(launch)
         ldr   x3, =init_data
         add   x3, x3, #INITINFO_stack /* Find the boot-time stack */
         ldr   x3, [x3]
@@ -421,13 +422,14 @@ launch:
 
         /* Jump to C world */
         br    x2
-ENDPROC(launch)
+END(launch)
 
 /* Fail-stop */
-fail:   PRINT_ID("- Boot failed -\r\n")
+FUNC_LOCAL(fail)
+        PRINT_ID("- Boot failed -\r\n")
 1:      wfe
         b     1b
-ENDPROC(fail)
+END(fail)
 
 #ifdef CONFIG_EARLY_PRINTK
 /*
@@ -438,14 +440,14 @@ ENDPROC(fail)
  *
  * Clobbers x0 - x1
  */
-init_uart:
+FUNC_LOCAL(init_uart)
         ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS
 #ifdef CONFIG_EARLY_UART_INIT
         early_uart_init x23, 0
 #endif
         PRINT("- UART enabled -\r\n")
         ret
-ENDPROC(init_uart)
+END(init_uart)
 
 /*
  * Print early debug messages.
@@ -454,7 +456,7 @@ ENDPROC(init_uart)
  * x23: Early UART base address
  * Clobbers x0-x1
  */
-ENTRY(asm_puts)
+FUNC(asm_puts)
         early_uart_ready x23, 1
         ldrb  w1, [x0], #1           /* Load next char */
         cbz   w1, 1f                 /* Exit on nul */
@@ -462,7 +464,7 @@ ENTRY(asm_puts)
         b     asm_puts
 1:
         ret
-ENDPROC(asm_puts)
+END(asm_puts)
 
 /*
  * Print a 64-bit number in hex.
@@ -471,7 +473,7 @@ ENDPROC(asm_puts)
  * x23: Early UART base address
  * Clobbers x0-x3
  */
-ENTRY(asm_putn)
+FUNC(asm_putn)
         adr_l x1, hex
         mov   x3, #16
 1:
@@ -484,7 +486,7 @@ ENTRY(asm_putn)
         subs  x3, x3, #1
         b.ne  1b
         ret
-ENDPROC(asm_putn)
+END(asm_putn)
 
 RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
 
@@ -493,16 +495,17 @@ RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
 /* This provides a C-API version of __lookup_processor_type
  * TODO: For now, the implementation return NULL every time
  */
-ENTRY(lookup_processor_type)
+FUNC(lookup_processor_type)
         mov  x0, #0
         ret
+END(lookup_processor_type)
 
 #ifdef CONFIG_ARM_EFI
 /*
  *  Function to transition from EFI loader in C, to Xen entry point.
  *  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
  */
-ENTRY(efi_xen_start)
+FUNC(efi_xen_start)
         /*
          * Preserve x0 (fdt pointer) across call to __flush_dcache_area,
          * restore for entry into Xen.
@@ -554,7 +557,7 @@ ENTRY(efi_xen_start)
         mov   x26, #1               /* x26 := skip_zero_bss */
 
         b     real_start_efi
-ENDPROC(efi_xen_start)
+END(efi_xen_start)
 
 #endif /* CONFIG_ARM_EFI */
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721731.1125362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yaj-0003XQ-Ip; Tue, 14 May 2024 20:17:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721731.1125362; Tue, 14 May 2024 20:17:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yaj-0003XE-FX; Tue, 14 May 2024 20:17:41 +0000
Received: by outflank-mailman (input) for mailman id 721731;
 Tue, 14 May 2024 20:17:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yai-0002pc-L0
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:40 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 03386757-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:40 +0200 (CEST)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4200ee78f35so26599295e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:40 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee9292sm206161835e9.37.2024.05.14.13.17.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03386757-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717858; x=1716322658; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6Lep3NQhPdutZe5qkYH7KS+rz9lJgEOzrCQ4vbmoNE8=;
        b=mgGIAupxBcVatInfvQ4BxfHwEEm9p8qdNGVhaXkuJYzCdkbOkH5RG64+bZ0hWuM73H
         7b0XOuUt5ZlmSVdGiu4iVwe40k+EdyUbrwZnmCFU9VuN4rbfCv+mz4js1RkQS5NrkgsK
         K/F/I3qO+3SjjGxtLmZY5q8nyE/AF730eAtfy7XJkbUaURhkN5nMj5yY5YdIuzBrNXYf
         5WIkL4FZgYfUSMcf81Y2xPnYLEdnZs7MWPboi2PDw2VVrVfXt9L7xSwlTiVn5lObthAP
         fmVviM6Jd1pzq/OFaso5mXH3jJjpRYKRAI4QD4WPE8iMXy1Ir2yP0QiDiCOiCz1KFwGw
         59uA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717858; x=1716322658;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=6Lep3NQhPdutZe5qkYH7KS+rz9lJgEOzrCQ4vbmoNE8=;
        b=I8bdI+sB7EH7FN6JjnGlwRGZ7JmjBDjwe6ZnarCQmgr7i2CdjHO0op4DU1yywq2EqY
         wyQyCzh5veEdsYJDDU24dkXoHuCEdyGdaYkXwIoKAzucQmL6vumlMRDIKD6iBPLI1HDW
         oiggX6zMKkSRfFRI0tVJWA4k2Qu1ODHEw7dd+CJzl0q2nk9CHBo2IJp7OtHieTKcPs6J
         Qi0l2d7SMOtjle5X0+iypEOUnXbEM+tjVVSyBgxvybgsLBTZhiT09nOgCmGRTKXS8o0w
         Yrwf739U0FD9DMSNMdmVOX2Mqo+C6pLSZ9FI7B3zkbUqQ35UqmSagXQnKDq0JCpGFBOu
         5uAQ==
X-Gm-Message-State: AOJu0YzfnWiAP24TVSeZyeMEGM7LTOhGHGCArmh2IXzIgZKlboACzsLf
	4YthWo2mBvcxU5+nHJ/5nJ8RcQ5C/Qp9XLxjCVpFrqYehX1+dhEZANCzCJAjtCI=
X-Google-Smtp-Source: AGHT+IG+uxGLVFCc15wbWYAuKJA+85Tw6jQmsZx4mBC2o//DARBcPQ+DWDRVLezNMadyokMM4wsJ5w==
X-Received: by 2002:a05:600c:4f4e:b0:418:a985:3ca with SMTP id 5b1f17b1804b1-41feac556e3mr162350105e9.31.1715717858610;
        Tue, 14 May 2024 13:17:38 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 3/9] xen/arm64: sve: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:08 +0200
Message-Id: <20240504115514.1063509-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/sve-asm.S | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/arm64/sve-asm.S b/xen/arch/arm/arm64/sve-asm.S
index 59dbefbbb2..2d8b895f07 100644
--- a/xen/arch/arm/arm64/sve-asm.S
+++ b/xen/arch/arm/arm64/sve-asm.S
@@ -161,9 +161,10 @@
 .endm
 
 /* Gets the current vector register size in bytes */
-GLOBAL(sve_get_hw_vl)
+FUNC(sve_get_hw_vl)
     _sve_rdvl 0, 1
     ret
+END(sve_get_hw_vl)
 
 /*
  * Save the SVE context
@@ -172,9 +173,10 @@ GLOBAL(sve_get_hw_vl)
  * x1 - pointer to buffer for P0-15
  * x2 - Save FFR if non-zero
  */
-GLOBAL(sve_save_ctx)
+FUNC(sve_save_ctx)
     sve_save 0, 1, x2
     ret
+END(sve_save_ctx)
 
 /*
  * Load the SVE context
@@ -183,9 +185,10 @@ GLOBAL(sve_save_ctx)
  * x1 - pointer to buffer for P0-15
  * x2 - Restore FFR if non-zero
  */
-GLOBAL(sve_load_ctx)
+FUNC(sve_load_ctx)
     sve_load 0, 1, x2
     ret
+END(sve_load_ctx)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721728.1125331 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yag-0002pu-Mu; Tue, 14 May 2024 20:17:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721728.1125331; Tue, 14 May 2024 20:17:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yag-0002pn-KQ; Tue, 14 May 2024 20:17:38 +0000
Received: by outflank-mailman (input) for mailman id 721728;
 Tue, 14 May 2024 20:17:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yaf-0002pc-8D
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:37 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00f8c882-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:36 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-420180b5897so18164075e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:36 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4201916e7c6sm67810195e9.12.2024.05.14.13.17.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00f8c882-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717855; x=1716322655; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=IBvV5FEc3tf5GrjG7+ucFNGYNubjsh1/ytD3egjnMDA=;
        b=PGrWGMrq9cZeLegpKs950NrAoyrbMKVZkeeY8nnEXyXLJ7v7TJXXYet0J+yleHFLtW
         V5xjvHllX/hx2wO7XkwZ5xgDXuDXM+n8wRzq+2cdosr0KmQQ7fnFcx4g7HOYfJBy/sdx
         Ii4vXQ+8vg1k+JXVgR6AetcXJEabFHgXQ1cQqQp/XTWyid8aI7YyKgxKRhcyU0604DW/
         oloRY0hIwV7ym7Jl88IOT8KnM0rw3Jrn7OszHucmyUp5cKLribtAfBEdlhjksgms/58M
         lUyQqhzfAgt8bSdfCzFAEaKmckFflYD/Zffm0FzrPduU7zmBJVWgrBktmPrZ9BNOyTJj
         0nOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717855; x=1716322655;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=IBvV5FEc3tf5GrjG7+ucFNGYNubjsh1/ytD3egjnMDA=;
        b=XXpaQ/rIqcB5+rIm3LowxC7iMuWvmGusvieHsqhmBkLjay6XvaOlSUItnp7UZKR7lg
         b6c8dKVFcgmiGVtYLm/OvrOJXVLQPGq9EEdrzxpFoUMZ2cwFGOcUt7y6OIHCwNQ5PhPp
         a/FzGTHon1zlO2RYJod/+Lvvy0lLHssnhyoh4YfMIL66x3J3lQyMvPkj/hhD0tQ6Gx6H
         MZJirvK5h9miN5Wn0rxPV182MctwCgIijNCaBTJL9B1B0+W+B0QDp1QoYkSaLZbuwpU0
         wAZmCcndMJz9P2scrkS3uoPGBS9Q02T1Hh4QsmSmBNs1kLVffjTYzghKxJr8tmwxikM6
         mhJw==
X-Gm-Message-State: AOJu0Yyjj55/k4+yMA5kn0YthViTOqHwNI8XfD9RdTkVV2ErrZ1ksq7g
	hgfUREMvRpja+EdPmgHyePpaJDfzjLrmn9RGXnM9K29tvfaNyM7RhCj0MTt/1zo=
X-Google-Smtp-Source: AGHT+IH8vPyrmb3cb5pvK75gMbwQemgCC+qqFqAZgZmOp97aqUU6jUfk3YlvyfSC4IedujUNhly3yQ==
X-Received: by 2002:a1c:4c03:0:b0:41c:5eb:4f8f with SMTP id 5b1f17b1804b1-41feaa38a6bmr103679465e9.15.1715717854695;
        Tue, 14 May 2024 13:17:34 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 1/9] xen/arm64: entry: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:06 +0200
Message-Id: <20240504115514.1063509-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/entry.S | 78 ++++++++++++++++++++++++--------------
 1 file changed, 50 insertions(+), 28 deletions(-)

diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index f963c923bb..6251135ebd 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -289,21 +289,25 @@
         b       do_bad_mode
         .endm
 
-hyp_sync_invalid:
+FUNC_LOCAL(hyp_sync_invalid)
         entry   hyp=1
         invalid BAD_SYNC
+END(hyp_sync_invalid)
 
-hyp_irq_invalid:
+FUNC_LOCAL(hyp_irq_invalid)
         entry   hyp=1
         invalid BAD_IRQ
+END(hyp_irq_invalid)
 
-hyp_fiq_invalid:
+FUNC_LOCAL(hyp_fiq_invalid)
         entry   hyp=1
         invalid BAD_FIQ
+END(hyp_fiq_invalid)
 
-hyp_error_invalid:
+FUNC_LOCAL(hyp_error_invalid)
         entry   hyp=1
         invalid BAD_ERROR
+END(hyp_error_invalid)
 
 /*
  * SError received while running in the hypervisor mode.
@@ -313,11 +317,12 @@ hyp_error_invalid:
  * simplicity, as SError should be rare and potentially fatal,
  * all interrupts are kept masked.
  */
-hyp_error:
+FUNC_LOCAL(hyp_error)
         entry   hyp=1
         mov     x0, sp
         bl      do_trap_hyp_serror
         exit    hyp=1
+END(hyp_error)
 
 /*
  * Synchronous exception received while running in the hypervisor mode.
@@ -327,7 +332,7 @@ hyp_error:
  * some of them. So we want to inherit the state from the interrupted
  * context.
  */
-hyp_sync:
+FUNC_LOCAL(hyp_sync)
         entry   hyp=1
 
         /* Inherit interrupts */
@@ -338,6 +343,7 @@ hyp_sync:
         mov     x0, sp
         bl      do_trap_hyp_sync
         exit    hyp=1
+END(hyp_sync)
 
 /*
  * IRQ received while running in the hypervisor mode.
@@ -352,7 +358,7 @@ hyp_sync:
  * would require some rework in some paths (e.g. panic, livepatch) to
  * ensure the ordering is enforced everywhere.
  */
-hyp_irq:
+FUNC_LOCAL(hyp_irq)
         entry   hyp=1
 
         /* Inherit D, A, F interrupts and keep I masked */
@@ -365,8 +371,9 @@ hyp_irq:
         mov     x0, sp
         bl      do_trap_irq
         exit    hyp=1
+END(hyp_irq)
 
-guest_sync:
+FUNC_LOCAL(guest_sync)
         /*
          * Save x0, x1 in advance
          */
@@ -413,8 +420,9 @@ fastpath_out_workaround:
         mov     x1, xzr
         eret
         sb
+END(guest_sync)
 
-wa2_ssbd:
+FUNC_LOCAL(wa2_ssbd)
 #ifdef CONFIG_ARM_SSBD
 alternative_cb arm_enable_wa2_handling
         b       wa2_end
@@ -450,42 +458,55 @@ wa2_end:
         mov     x0, xzr
         eret
         sb
-guest_sync_slowpath:
+END(wa2_ssbd)
+
+FUNC_LOCAL(guest_sync_slowpath)
         /*
          * x0/x1 may have been scratch by the fast path above, so avoid
          * to save them.
          */
         guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_sync, save_x0_x1=0
+END(guest_sync_slowpath)
 
-guest_irq:
+FUNC_LOCAL(guest_irq)
         guest_vector compat=0, iflags=IFLAGS__A__, trap=irq
+END(guest_irq)
 
-guest_fiq_invalid:
+FUNC_LOCAL(guest_fiq_invalid)
         entry   hyp=0, compat=0
         invalid BAD_FIQ
+END(guest_fiq_invalid)
 
-guest_error:
+FUNC_LOCAL(guest_error)
         guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_serror
+END(guest_error)
 
-guest_sync_compat:
+FUNC_LOCAL(guest_sync_compat)
         guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_sync
+END(guest_sync_compat)
 
-guest_irq_compat:
+FUNC_LOCAL(guest_irq_compat)
         guest_vector compat=1, iflags=IFLAGS__A__, trap=irq
+END(guest_irq_compat)
 
-guest_fiq_invalid_compat:
+FUNC_LOCAL(guest_fiq_invalid_compat)
         entry   hyp=0, compat=1
         invalid BAD_FIQ
+END(guest_fiq_invalid_compat)
 
-guest_error_compat:
+FUNC_LOCAL(guest_error_compat)
         guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_serror
+END(guest_error_compat)
 
-ENTRY(return_to_new_vcpu32)
+FUNC(return_to_new_vcpu32)
         exit    hyp=0, compat=1
-ENTRY(return_to_new_vcpu64)
+END(return_to_new_vcpu32)
+
+FUNC(return_to_new_vcpu64)
         exit    hyp=0, compat=0
+END(return_to_new_vcpu64)
 
-return_from_trap:
+FUNC_LOCAL(return_from_trap)
         msr     daifset, #IFLAGS___I_ /* Mask interrupts */
 
         ldr     x21, [sp, #UREGS_PC]            /* load ELR */
@@ -524,6 +545,7 @@ return_from_trap:
 
         eret
         sb
+END(return_from_trap)
 
 /*
  * Consume pending SError generated by the guest if any.
@@ -536,7 +558,7 @@ return_from_trap:
  * it. So the function will unmask SError exception for a small window and
  * then mask it again.
  */
-check_pending_guest_serror:
+FUNC_LOCAL(check_pending_guest_serror)
         /*
          * Save elr_el2 to check whether the pending SError exception takes
          * place while we are doing this sync exception.
@@ -562,13 +584,11 @@ check_pending_guest_serror:
          * exception handler, and the elr_el2 will be set to
          * abort_guest_exit_start or abort_guest_exit_end.
          */
-        .global abort_guest_exit_start
-abort_guest_exit_start:
+LABEL(abort_guest_exit_start)
 
         isb
 
-        .global abort_guest_exit_end
-abort_guest_exit_end:
+LABEL(abort_guest_exit_end)
         /* Mask PSTATE asynchronous abort bit, close the checking window. */
         msr     daifset, #IFLAGS__A__
 
@@ -586,7 +606,7 @@ abort_guest_exit_end:
         cset    x19, ne
 
         ret
-ENDPROC(check_pending_guest_serror)
+END(check_pending_guest_serror)
 
 /*
  * Exception vectors.
@@ -597,7 +617,7 @@ ENDPROC(check_pending_guest_serror)
         .endm
 
         .align  11
-ENTRY(hyp_traps_vector)
+FUNC(hyp_traps_vector)
         ventry  hyp_sync_invalid            /* Synchronous EL2t */
         ventry  hyp_irq_invalid             /* IRQ EL2t */
         ventry  hyp_fiq_invalid             /* FIQ EL2t */
@@ -617,6 +637,7 @@ ENTRY(hyp_traps_vector)
         ventry  guest_irq_compat            /* IRQ 32-bit EL0/EL1 */
         ventry  guest_fiq_invalid_compat    /* FIQ 32-bit EL0/EL1 */
         ventry  guest_error_compat          /* Error 32-bit EL0/EL1 */
+END(hyp_traps_vector)
 
 /*
  * struct vcpu *__context_switch(struct vcpu *prev, struct vcpu *next)
@@ -626,7 +647,7 @@ ENTRY(hyp_traps_vector)
  *
  * Returns prev in x0
  */
-ENTRY(__context_switch)
+FUNC(__context_switch)
         add     x8, x0, #VCPU_arch_saved_context
         mov     x9, sp
         stp     x19, x20, [x8], #16         /* store callee-saved registers */
@@ -647,6 +668,7 @@ ENTRY(__context_switch)
         ldr     lr, [x8]
         mov     sp, x9
         ret
+END(__context_switch)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721733.1125383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yao-00046s-4t; Tue, 14 May 2024 20:17:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721733.1125383; Tue, 14 May 2024 20:17:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yao-00046j-0M; Tue, 14 May 2024 20:17:46 +0000
Received: by outflank-mailman (input) for mailman id 721733;
 Tue, 14 May 2024 20:17:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yan-0002pZ-2K
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:45 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05323461-122f-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:17:43 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41fd5dc0480so38210245e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:43 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fe9599294sm69564455e9.1.2024.05.14.13.17.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05323461-122f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717862; x=1716322662; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iiprdCUaZ8IPJ2HAIHBPs+piSKbOcZYN0PLf+5sc450=;
        b=k9unhmmTPO3p28/dnEDtn1vC/ltlQJxpdKiYl7iY4uz4VG7mJp/icIuwcK0JxNEz4w
         yqxUsGiQj5CmX8nnWrNGioYdYfeIR5nIB7r1Vlu0sjVihXeu2xgv3xxqzlD/xODuk31Y
         L3QXA0vjvc1fc3TLhL9F1wVMc4WJ0A7eyQmvJcIoJOYQI3SqG92iUknKBc0DX1HIIwmD
         p6kRkSCIGqesR+Q/TR4zQeVFuNMsQ6L475jTvT2KkumAjJ1CnNlC35PhgoWld1g1Ly9y
         GLlYK8BsivE1CkLsgcgftEHwbDBP8cwHcaRGjEftGfZHZN1adVssMFZOOA7/wGerXFBf
         7cWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717862; x=1716322662;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iiprdCUaZ8IPJ2HAIHBPs+piSKbOcZYN0PLf+5sc450=;
        b=ByzVUxFsNqGd1I6fRAP4946mEr9LdA2f7SzE9PZ19CXWRyblylf1AVMVABFjTVTqq+
         RWyYUxpGsCvZQQ9aptHjmZnZ9FT0m7BL1kv0OiaODkXtM6vUhfO3z+qzbWk2YAwdiiN2
         dnV0N5lTwmCPwi+ANXd5UIeeYfzFBdeLgMKnDXvHtWNxYglsIXs9DrZ7h5H7RDHHkHCF
         ik4YrUJsTuPmaMTBhB0fUS+r7k8R3D5Vbw6I3eeorczCzf1DvW4BtP2nk7ZJJl+RLDqL
         xU9Sj5cCpPMe4ThzOqZhUsFUY0d6epfpyLqy4geSA4jB4n2FKufPmfNAHaT1p/fr+Tjn
         5+bw==
X-Gm-Message-State: AOJu0Yw9PmcsxfEvewpVufibMeWBu9nQDlZOI5pQSat8nXOKwEcQ25Rv
	iMcw67YEqr7bJrYC2j03Ro3O2lIJ76lxRf1ASaZ6ol1fZyeDa5Ep5gxI3mdhu1g=
X-Google-Smtp-Source: AGHT+IEgatADuwMOnHbGVnd50bJVk3aJQ122Wq/nW+vPJftub6+X/YV1Ttxp9uUzAs9H5gXfHezQ9w==
X-Received: by 2002:a05:600c:3548:b0:41a:9a6a:41e1 with SMTP id 5b1f17b1804b1-41feac5a3c3mr88789995e9.29.1715717862394;
        Tue, 14 May 2024 13:17:42 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 5/9] xen/arm64: debug: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:10 +0200
Message-Id: <20240504115514.1063509-6-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/debug.S | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/debug.S b/xen/arch/arm/arm64/debug.S
index 71cad9d762..c3d02c33d7 100644
--- a/xen/arch/arm/arm64/debug.S
+++ b/xen/arch/arm/arm64/debug.S
@@ -27,17 +27,19 @@
  * Print a character on the UART - this function is called by C
  * x0: character to print
  */
-GLOBAL(early_putch)
+FUNC(early_putch)
         ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS
         early_uart_ready x15, 1
         early_uart_transmit x15, w0
         ret
+END(early_putch)
 
 /* Flush the UART - this function is called by C */
-GLOBAL(early_flush)
+FUNC(early_flush)
         ldr   x15, =EARLY_UART_VIRTUAL_ADDRESS  /* x15 := VA UART base address */
         early_uart_ready x15, 1
         ret
+END(early_flush)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721734.1125392 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yar-0004Sh-F8; Tue, 14 May 2024 20:17:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721734.1125392; Tue, 14 May 2024 20:17:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yar-0004SQ-BX; Tue, 14 May 2024 20:17:49 +0000
Received: by outflank-mailman (input) for mailman id 721734;
 Tue, 14 May 2024 20:17:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yaq-0002pc-2i
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:48 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07be9db2-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:47 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-420116a6652so23635155e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:47 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42013bf1a5dsm100159525e9.30.2024.05.14.13.17.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07be9db2-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717867; x=1716322667; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1717D7j7Li0LpnJckxTcnvPunZIbhAasL7O0x4ESCyw=;
        b=k1QEDIxWIsN/Y5whsuTQFD0jUS/JyoVIouGDI8MpJ++ebF9RW2Puo/OU875R4Kp/bM
         W7pQwqoiKbIDz7Mbf/lzzJ3iRBhKFHDq33FzStFgET3nTNmZWlgJs+OQVub8NsO98FQP
         rXs085R87JgUCJoAS4HsmpK23A5vQl/xVNH7A9s1Trppm1t/ydFsPjPXl+peCFkMHAEf
         PBaQLfO7P3Lh2Wd2XkItUIGwIL1+NGJfbofLCdkA/G1CYTKPUOYn9SNjdYAj8lusIK5j
         /B69xk5FyHQq3SorjZEPf5+yCrP/es9cOEauUjk5AeYFF7n9t7Yzr4MjmocXpRRuNuz7
         pYow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717867; x=1716322667;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1717D7j7Li0LpnJckxTcnvPunZIbhAasL7O0x4ESCyw=;
        b=t2VDvpm7Rvv9uEjGhPjJXTkBkExBB6KYU1UDwvMm6HhXcCVxCQEwBVpFMw4A0ntClB
         NRxdrKG9cy0aoH/QGzjuYh1d/ijwNi05U1/Tgvl066YSS9mAlEQP3FvAH3XDmSLgBfwo
         peH//LfCdpWPO4TtYsOBK7Mm2jas+fnf8h730/ahj/jx2DsFGRiyWL4NQd6zgLHDubGU
         JHVd77QcHPRs/AXL8QtP4mux/koqn7Pbqnik/bMsFpsw6JxOGmJiwzowtWok7RL8aWng
         vYEa+uYIRz24i7Sc4JeZohcjlTq9L0c9y1RyFr9MZW2r1n/cHnxxRehxEm+5fy500s88
         y7Kg==
X-Gm-Message-State: AOJu0YzT98Kh84YJk1TS4eN1hCRMl0CXT8aue+UiXFwKfRtqRl6TCSnO
	/B/Nu5w3zhlEMWm88jyhyiApUHVsuNTxCkxIWq3zOG8Ww2GRwtBVLJydoAdWdqQ=
X-Google-Smtp-Source: AGHT+IGRh/W5Wv2GrhvlWrUjkBkCL2R4hNYgf3565GfKmcqojKnIg9g6JVwPHXk2eVzSaWDGoc/4jQ==
X-Received: by 2002:a05:600c:4f92:b0:420:177f:c2a6 with SMTP id 5b1f17b1804b1-420177fc636mr46639105e9.10.1715717866670;
        Tue, 14 May 2024 13:17:46 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 6/9] xen/arm64: bpi: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:11 +0200
Message-Id: <20240504115514.1063509-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index 4e63825220..d6b8defb80 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -52,14 +52,15 @@
  * micro-architectures in a system.
  */
     .align	11
-ENTRY(__bp_harden_hyp_vecs_start)
+FUNC(__bp_harden_hyp_vecs_start)
     .rept 4
     vectors hyp_traps_vector
     .endr
-ENTRY(__bp_harden_hyp_vecs_end)
+LABEL(__bp_harden_hyp_vecs_end)
+END(__bp_harden_hyp_vecs_start)
 
 .macro mitigate_spectre_bhb_loop count
-ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+FUNC(__mitigate_spectre_bhb_loop_start_\count)
     stp     x0, x1, [sp, #-16]!
     mov     x0, \count
 .Lspectre_bhb_loop\@:
@@ -68,11 +69,12 @@ ENTRY(__mitigate_spectre_bhb_loop_start_\count)
     b.ne    .Lspectre_bhb_loop\@
     sb
     ldp     x0, x1, [sp], #16
-ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+LABEL(__mitigate_spectre_bhb_loop_end_\count)
+END(__mitigate_spectre_bhb_loop_start_\count)
 .endm
 
 .macro smccc_workaround num smcc_id
-ENTRY(__smccc_workaround_smc_start_\num)
+FUNC(__smccc_workaround_smc_start_\num)
     sub     sp, sp, #(8 * 4)
     stp     x0, x1, [sp, #(8 * 2)]
     stp     x2, x3, [sp, #(8 * 0)]
@@ -81,13 +83,15 @@ ENTRY(__smccc_workaround_smc_start_\num)
     ldp     x2, x3, [sp, #(8 * 0)]
     ldp     x0, x1, [sp, #(8 * 2)]
     add     sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_smc_end_\num)
+LABEL(__smccc_workaround_smc_end_\num)
+END(__smccc_workaround_smc_start_\num)
 .endm
 
-ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+FUNC(__mitigate_spectre_bhb_clear_insn_start)
     clearbhb
     isb
-ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+LABEL(__mitigate_spectre_bhb_clear_insn_end)
+END(__mitigate_spectre_bhb_clear_insn_start)
 
 mitigate_spectre_bhb_loop 8
 mitigate_spectre_bhb_loop 24
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721735.1125402 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yas-0004jB-Ou; Tue, 14 May 2024 20:17:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721735.1125402; Tue, 14 May 2024 20:17:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yas-0004ie-Kp; Tue, 14 May 2024 20:17:50 +0000
Received: by outflank-mailman (input) for mailman id 721735;
 Tue, 14 May 2024 20:17:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yar-0002pc-MN
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:49 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08a85208-122f-11ef-909d-e314d9c70b13;
 Tue, 14 May 2024 22:17:49 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-351ae94323aso2848538f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:49 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbc56b7sm14402146f8f.114.2024.05.14.13.17.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08a85208-122f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717868; x=1716322668; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=G6nqh6fc2e8at7xBm6/0+3HSkQQZqnB7HPpzJ3rGv88=;
        b=igu+TTA351Gf6tHAdpHS/PegK1OViJ4NGxAzPqJBbMYgByE758pwNRc7tE0gPXUEWf
         Q5tO0511s8KcR0y4PS4wv6d5p6b/LuG9lKRhKc+8x0agV+jG/OGPHih5aa85mpMeKPS+
         OGntoqb23pEXnfHB/fJmBdAsQrbR6otM/A4nAFhVJYCbRte0Ax+bLd1i8qe0BaCXv0TC
         Dubl+1ABy3L7FODWdBFvsRWcqC3LEebpWPz3cxioS485d/vCuUYkfhM5oF/2a0piidDw
         BLgVy4iulo2I2nMd3ygyf9VwlSPe/oBWvAn8F2a4ihLew6Hv2jiSVv1cHRZfHei55cEk
         qQ9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717868; x=1716322668;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=G6nqh6fc2e8at7xBm6/0+3HSkQQZqnB7HPpzJ3rGv88=;
        b=q/Mx6Yj9gBCLd9nVUumilm5GWMCW3gUb5WbsYJ969Gu2xsNiSjK1XD04l2bERxWbmz
         tt1R8UXkWXUAKbsjzMHibLPlKVXfD8lXtNm4tXPvjAG7axku+WoPaaf2Lay//i1/SnrE
         iSgRmCcC3zPv9GtlCN2ObpwAmRXUnE4RLiSJVEhcttiXbzAdW07OFbx6rCNbQfglvE/Q
         LhYRFu3JomLgTZ6VnGHnfxBjoBj5RQg48uutjGA2KKtPsEttbEbMROOky/aEEZAWt4o+
         xO9rcfPloh8xi93d6fAHeRvq25Yxp1QvRbRC41sw52QmI+Nx8ifb5rf9+QjUuYNlzFvA
         YTTA==
X-Gm-Message-State: AOJu0YxXlZKoNycCUnmXLqiu0i70I4IbQomGIhgFy066GJQInbxOoVq6
	+MlNqWilUBMYH0Se3FdhlfpxqD/im39K+TlI0Gud7vEykclaGrankEqR4QPH1GM=
X-Google-Smtp-Source: AGHT+IHdh4XOg68Cp20JCc05AVisIn5+J16veP2rN6473ADgexjnvxIX4EWyaFtMQdkflILXmib/Zg==
X-Received: by 2002:adf:f10e:0:b0:350:484f:fe1a with SMTP id ffacd0b85a97d-3504a61c512mr10328931f8f.11.1715717867981;
        Tue, 14 May 2024 13:17:47 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 7/9] xen/arm64: mmu/head: Add missing code symbol annotations
Date: Sat,  4 May 2024 13:55:12 +0200
Message-Id: <20240504115514.1063509-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols
and add missing annotations.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/arch/arm/arm64/mmu/head.S | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S
index fa40b696dd..665a51a337 100644
--- a/xen/arch/arm/arm64/mmu/head.S
+++ b/xen/arch/arm/arm64/mmu/head.S
@@ -138,7 +138,7 @@
  *
  * Clobbers x0 - x4
  */
-create_page_tables:
+FUNC_LOCAL(create_page_tables)
         /* Prepare the page-tables for mapping Xen */
         ldr   x0, =XEN_VIRT_START
         create_table_entry boot_pgtable, boot_first, x0, 0, x1, x2, x3
@@ -260,7 +260,7 @@ virtphys_clash:
         /* Identity map clashes with boot_third, which we cannot handle yet */
         PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
         b     fail
-ENDPROC(create_page_tables)
+END(create_page_tables)
 
 /*
  * Turn on the Data Cache and the MMU. The function will return on the 1:1
@@ -273,7 +273,7 @@ ENDPROC(create_page_tables)
  *
  * Clobbers x0 - x5
  */
-enable_mmu:
+FUNC_LOCAL(enable_mmu)
         mov   x4, x0
         mov   x5, x1
         PRINT_ID("- Turning on paging -\r\n")
@@ -304,7 +304,7 @@ enable_mmu:
         PRINT_ID("- Paging turned on -\r\n")
 
         ret
-ENDPROC(enable_mmu)
+END(enable_mmu)
 
 /*
  * Enable mm (turn on the data cache and the MMU) for secondary CPUs.
@@ -316,7 +316,7 @@ ENDPROC(enable_mmu)
  *
  * Clobbers x0 - x6
  */
-ENTRY(enable_secondary_cpu_mm)
+FUNC(enable_secondary_cpu_mm)
         mov   x6, lr
 
         load_paddr x0, init_ttbr
@@ -328,7 +328,7 @@ ENTRY(enable_secondary_cpu_mm)
 
         /* Return to the virtual address requested by the caller. */
         ret
-ENDPROC(enable_secondary_cpu_mm)
+END(enable_secondary_cpu_mm)
 
 /*
  * Enable mm (turn on the data cache and the MMU) for the boot CPU.
@@ -340,7 +340,7 @@ ENDPROC(enable_secondary_cpu_mm)
  *
  * Clobbers x0 - x6
  */
-ENTRY(enable_boot_cpu_mm)
+FUNC(enable_boot_cpu_mm)
         mov   x6, lr
 
         bl    create_page_tables
@@ -365,7 +365,7 @@ ENTRY(enable_boot_cpu_mm)
          * by the caller.
          */
         b     remove_identity_mapping
-ENDPROC(enable_boot_cpu_mm)
+END(enable_boot_cpu_mm)
 
 /*
  * Remove the 1:1 map from the page-tables. It is not easy to keep track
@@ -377,7 +377,7 @@ ENDPROC(enable_boot_cpu_mm)
  *
  * Clobbers x0 - x1
  */
-remove_identity_mapping:
+FUNC_LOCAL(remove_identity_mapping)
         /*
          * Find the zeroeth slot used. Remove the entry from zeroeth
          * table if the slot is not XEN_ZEROETH_SLOT.
@@ -419,20 +419,21 @@ identity_mapping_removed:
         flush_xen_tlb_local
 
         ret
-ENDPROC(remove_identity_mapping)
+END(remove_identity_mapping)
 
 /* Fail-stop */
-fail:   PRINT("- Boot failed -\r\n")
+FUNC_LOCAL(fail)
+        PRINT("- Boot failed -\r\n")
 1:      wfe
         b     1b
-ENDPROC(fail)
+END(fail)
 
 /*
  * Switch TTBR
  *
  * x0    ttbr
  */
-ENTRY(switch_ttbr_id)
+FUNC(switch_ttbr_id)
         /* 1) Ensure any previous read/write have completed */
         dsb    ish
         isb
@@ -464,7 +465,7 @@ ENTRY(switch_ttbr_id)
         isb
 
         ret
-ENDPROC(switch_ttbr_id)
+END(switch_ttbr_id)
 
 /*
  * Local variables:
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721736.1125412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yav-00056D-4W; Tue, 14 May 2024 20:17:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721736.1125412; Tue, 14 May 2024 20:17:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yau-00055t-W3; Tue, 14 May 2024 20:17:52 +0000
Received: by outflank-mailman (input) for mailman id 721736;
 Tue, 14 May 2024 20:17:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yau-0002pZ-0F
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:52 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09685cae-122f-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:17:50 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-3504ad82a64so906570f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:50 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8a781asm14391479f8f.60.2024.05.14.13.17.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09685cae-122f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717869; x=1716322669; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UTCkkC5/83lm5FIVPOAdJc+T+IQBpLJenacee+31hfg=;
        b=S++4pqcCdiEzDfsoMSHYyD8QEbUgxmtIvG7lKcDzRD79pKr6MSTQhVmJg7tluOfW5h
         PZ6GMPSWT2fTsFFoXiALOMUzf/fejz0C7CgCIIl1lxqdWgXtOW3XUULOH4+vtcWnjVQp
         o6WyPz/jqV7WTg3PTnr4AP/aYL9hrfNo2cV2WB+1GEUaPvCjbu9Kcr2tYKpMVzdxuFgO
         DlVrD0QPx5aV5RjYeAW2JheSeOGslY9NZDU56EayCaAQezVewjk41Ueyiy8rWU2ZOw9s
         PpMaWOUOWwIrJAUEdtyRZr0WwcnCVA9Nxk5xcltiZlS0+aqcqEW16u917KyFRBbCIs+P
         xH8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717869; x=1716322669;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UTCkkC5/83lm5FIVPOAdJc+T+IQBpLJenacee+31hfg=;
        b=AM5adtUd8IilJIsjb7w19YyCNGpPjq6LjNAjsRneNBgj4FT3gawT8F36NGcd/xOHag
         Dr1Zgh6lelcklJtHUcHydX94kA/CWEIdpwOCke7HR+RtmWi6KGg8xPBjYv86TNZydlw1
         i3huyk7P7Si0iGwQ9uJKk/B2nNuIaALqRZLx/WnnAYgpLysJ/ZLq4pvXeW35KoeIChWS
         PjqLH0DZcAunCUzB5XVWgCWFHx9bcicQKFOjuRkopoYF4/bc61OmHPnRipYrwViPZWqu
         9eMdTF8G2xQw9G9SmPXqRMNP8V/kKsesE09z6QGwL8TXH/6CTfLXvEaN3sONcQR7wFp4
         CtJA==
X-Gm-Message-State: AOJu0YxKJt1jhGg8iDbVBdbnPzCO32Q2C1SZwL/r/ZslKVHh3f9qRTzM
	/35333NzRhwSVa3ZcXYxahL3cu5AKDHwto3FNyOJ7WA6IUqDolDchaIg7XDbqSo=
X-Google-Smtp-Source: AGHT+IF20G/v0oHsqh6yp0IUZm9ugRLi2VmhlB17neNBBFPDQMOljh8mtnwhP0tivXMW3PxJx7mSxw==
X-Received: by 2002:adf:f788:0:b0:349:eeaf:3a01 with SMTP id ffacd0b85a97d-3504a632de1mr8881343f8f.22.1715717869448;
        Tue, 14 May 2024 13:17:49 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 8/9] xen/arm64: cache: Use the generic xen/linkage.h macros
Date: Sat,  4 May 2024 13:55:13 +0200
Message-Id: <20240504115514.1063509-9-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/cache.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/cache.S b/xen/arch/arm/arm64/cache.S
index 9a88a2b497..66ed85f735 100644
--- a/xen/arch/arm/arm64/cache.S
+++ b/xen/arch/arm/arm64/cache.S
@@ -40,7 +40,7 @@
  *	- kaddr   - kernel address
  *	- size    - size in question
  */
-ENTRY(__flush_dcache_area)
+FUNC(__flush_dcache_area)
 	dcache_line_size x2, x3
 	add	x1, x0, x1
 	sub	x3, x2, #1
@@ -51,4 +51,4 @@ ENTRY(__flush_dcache_area)
 	b.lo	1b
 	dsb	sy
 	ret
-ENDPROC(__flush_dcache_area)
+END(__flush_dcache_area)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:17:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:17:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721737.1125421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yax-0005Uv-HH; Tue, 14 May 2024 20:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721737.1125421; Tue, 14 May 2024 20:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yax-0005Uj-Cv; Tue, 14 May 2024 20:17:55 +0000
Received: by outflank-mailman (input) for mailman id 721737;
 Tue, 14 May 2024 20:17:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uMtz=MR=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s6yav-0002pZ-SY
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:17:53 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0a60cb18-122f-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:17:52 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-34da35cd01cso5610277f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 13:17:52 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8a77easm14449404f8f.53.2024.05.14.13.17.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 May 2024 13:17:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a60cb18-122f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715717871; x=1716322671; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JCSDJt2hMmc69IRlsT9xTwbRCbI4uAHpKe8NaeMrf5E=;
        b=npiz4V1TpPyr1NaX5wz9BTV8T9pLHHjIfnAH2dKAwKcNqlSZrxA3D6ZCCM3FGITpVU
         slb0f/EA5qyiW2lj99fX7DcTanYkUnkYsbzVe1eoHNDrI0W9DHQprj6arqFK3/bXigw4
         QiDQ2ihN1AV9DG6F5PF5BHNo+rBXKteVcRFUtRSvdkzHYjP1ea3dE+sOxiS7a138aW0h
         ssvGPeDGA9hrwCcHThy37Xc3r3I3/W3aP5bfONZNzhe1JlHoCkltUJQdHiY1LZ/Qw0Ob
         Qbhq7jiQQKvNaQjZtYcDDz9kw4c0w+o3NJsI7ZdvhleofMMRbPnObKu5QWD9pX2HOZFm
         FfNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715717871; x=1716322671;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JCSDJt2hMmc69IRlsT9xTwbRCbI4uAHpKe8NaeMrf5E=;
        b=eWHRYPmYD+lEj3DNGuLLo3VedDAKehUKU6eZFRYQrp+lTtcsf1RqZY9xGEFc7WMKkI
         HzDVvlTVjwS9+RrB0QwY98SPr/2mPuBHcprlfc2td0frnRCBdnyLQ7pyMea0LyQowkbl
         JcpxCQT5qZo5IHgEHxTMD0FtK6cFzYrzjsEBqrYW17eoP9DOAOHE2dUB12kLHyo9328b
         Dh76LqvoJLqXOkUZ0UT9Y4NZ1fGaAXl4O7bLpxCbBMJ6gi0oh0fUZin814t9fckJcQrV
         YFNOO4YF0vx6/IaifskKvL//P65S8qPLfGsc/Qru+y94l8TB7O6F3K/1NkAB8PkamYxK
         uveg==
X-Gm-Message-State: AOJu0YwC5w3Ga9MGDOpkp85sr3FAZ6utBQ1OaO7aovpv4Rk5qp5zTbVQ
	Hr3l+127ZIShBCJNu6dNwCgUxqpVdHeT5DuLow8yKY8/WgAuGWsd6aZ97M9+8rg=
X-Google-Smtp-Source: AGHT+IEuJF3MSyUJ1H11afBcmSASfkLnxpTxrxB73BUnHkVzyKDDCPRiRj69qmBLVG/4AN8MGtvEBQ==
X-Received: by 2002:a5d:5908:0:b0:34e:93c1:7979 with SMTP id ffacd0b85a97d-3504a73e607mr10716553f8f.38.1715717870752;
        Tue, 14 May 2024 13:17:50 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	edgar.iglesias@amd.com
Subject: [PATCH v4 9/9] xen/arm64: lib: Use the generic xen/linkage.h macros
Date: Sat,  4 May 2024 13:55:14 +0200
Message-Id: <20240504115514.1063509-10-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Use the generic xen/linkage.h macros to annotate code symbols.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/arm64/lib/memchr.S  | 4 ++--
 xen/arch/arm/arm64/lib/memcmp.S  | 4 ++--
 xen/arch/arm/arm64/lib/memcpy.S  | 4 ++--
 xen/arch/arm/arm64/lib/memmove.S | 4 ++--
 xen/arch/arm/arm64/lib/memset.S  | 4 ++--
 xen/arch/arm/arm64/lib/strchr.S  | 4 ++--
 xen/arch/arm/arm64/lib/strcmp.S  | 4 ++--
 xen/arch/arm/arm64/lib/strlen.S  | 4 ++--
 xen/arch/arm/arm64/lib/strncmp.S | 4 ++--
 xen/arch/arm/arm64/lib/strnlen.S | 4 ++--
 xen/arch/arm/arm64/lib/strrchr.S | 4 ++--
 11 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/xen/arch/arm/arm64/lib/memchr.S b/xen/arch/arm/arm64/lib/memchr.S
index 81f113bb1c..3d8aeca3ca 100644
--- a/xen/arch/arm/arm64/lib/memchr.S
+++ b/xen/arch/arm/arm64/lib/memchr.S
@@ -29,7 +29,7 @@
  * Returns:
  *	x0 - address of first occurrence of 'c' or 0
  */
-ENTRY(memchr)
+FUNC(memchr)
 	and	w1, w1, #0xff
 1:	subs	x2, x2, #1
 	b.mi	2f
@@ -40,4 +40,4 @@ ENTRY(memchr)
 	ret
 2:	mov	x0, #0
 	ret
-ENDPROC(memchr)
+END(memchr)
diff --git a/xen/arch/arm/arm64/lib/memcmp.S b/xen/arch/arm/arm64/lib/memcmp.S
index 87c2537ffe..d77dd4ce52 100644
--- a/xen/arch/arm/arm64/lib/memcmp.S
+++ b/xen/arch/arm/arm64/lib/memcmp.S
@@ -57,7 +57,7 @@ pos		.req	x11
 limit_wd	.req	x12
 mask		.req	x13
 
-ENTRY(memcmp)
+FUNC(memcmp)
 	cbz	limit, .Lret0
 	eor	tmp1, src1, src2
 	tst	tmp1, #7
@@ -254,4 +254,4 @@ CPU_LE( rev	data2, data2 )
 .Lret0:
 	mov	result, #0
 	ret
-ENDPROC(memcmp)
+END(memcmp)
diff --git a/xen/arch/arm/arm64/lib/memcpy.S b/xen/arch/arm/arm64/lib/memcpy.S
index d90d20ef3e..1e04b79010 100644
--- a/xen/arch/arm/arm64/lib/memcpy.S
+++ b/xen/arch/arm/arm64/lib/memcpy.S
@@ -55,7 +55,7 @@ C_h	.req	x12
 D_l	.req	x13
 D_h	.req	x14
 
-ENTRY(memcpy)
+FUNC(memcpy)
 	mov	dst, dstin
 	cmp	count, #16
 	/*When memory length is less than 16, the accessed are not aligned.*/
@@ -197,4 +197,4 @@ ENTRY(memcpy)
 	tst	count, #0x3f
 	b.ne	.Ltail63
 	ret
-ENDPROC(memcpy)
+END(memcpy)
diff --git a/xen/arch/arm/arm64/lib/memmove.S b/xen/arch/arm/arm64/lib/memmove.S
index a49de845d0..14438dbe9c 100644
--- a/xen/arch/arm/arm64/lib/memmove.S
+++ b/xen/arch/arm/arm64/lib/memmove.S
@@ -56,7 +56,7 @@ C_h	.req	x12
 D_l	.req	x13
 D_h	.req	x14
 
-ENTRY(memmove)
+FUNC(memmove)
 	cmp	dstin, src
 	b.lo	memcpy
 	add	tmp1, src, count
@@ -193,4 +193,4 @@ ENTRY(memmove)
 	tst	count, #0x3f
 	b.ne	.Ltail63
 	ret
-ENDPROC(memmove)
+END(memmove)
diff --git a/xen/arch/arm/arm64/lib/memset.S b/xen/arch/arm/arm64/lib/memset.S
index 5bf751521b..367fa60175 100644
--- a/xen/arch/arm/arm64/lib/memset.S
+++ b/xen/arch/arm/arm64/lib/memset.S
@@ -53,7 +53,7 @@ dst		.req	x8
 tmp3w		.req	w9
 tmp3		.req	x9
 
-ENTRY(memset)
+FUNC(memset)
 	mov	dst, dstin	/* Preserve return value.  */
 	and	A_lw, val, #255
 	orr	A_lw, A_lw, A_lw, lsl #8
@@ -212,4 +212,4 @@ ENTRY(memset)
 	ands	count, count, zva_bits_x
 	b.ne	.Ltail_maybe_long
 	ret
-ENDPROC(memset)
+END(memset)
diff --git a/xen/arch/arm/arm64/lib/strchr.S b/xen/arch/arm/arm64/lib/strchr.S
index 0506b0ff7f..83fd81e8ef 100644
--- a/xen/arch/arm/arm64/lib/strchr.S
+++ b/xen/arch/arm/arm64/lib/strchr.S
@@ -27,7 +27,7 @@
  * Returns:
  *	x0 - address of first occurrence of 'c' or 0
  */
-ENTRY(strchr)
+FUNC(strchr)
 	and	w1, w1, #0xff
 1:	ldrb	w2, [x0], #1
 	cmp	w2, w1
@@ -37,4 +37,4 @@ ENTRY(strchr)
 	cmp	w2, w1
 	csel	x0, x0, xzr, eq
 	ret
-ENDPROC(strchr)
+END(strchr)
diff --git a/xen/arch/arm/arm64/lib/strcmp.S b/xen/arch/arm/arm64/lib/strcmp.S
index c6f42dd255..7677108e26 100644
--- a/xen/arch/arm/arm64/lib/strcmp.S
+++ b/xen/arch/arm/arm64/lib/strcmp.S
@@ -59,7 +59,7 @@ tmp3		.req	x9
 zeroones	.req	x10
 pos		.req	x11
 
-ENTRY(strcmp)
+FUNC(strcmp)
 	eor	tmp1, src1, src2
 	mov	zeroones, #REP8_01
 	tst	tmp1, #7
@@ -230,4 +230,4 @@ CPU_BE(	orr	syndrome, diff, has_nul )
 	lsr	data1, data1, #56
 	sub	result, data1, data2, lsr #56
 	ret
-ENDPROC(strcmp)
+END(strcmp)
diff --git a/xen/arch/arm/arm64/lib/strlen.S b/xen/arch/arm/arm64/lib/strlen.S
index fb6aaf1a6a..10feedaf81 100644
--- a/xen/arch/arm/arm64/lib/strlen.S
+++ b/xen/arch/arm/arm64/lib/strlen.S
@@ -56,7 +56,7 @@ pos		.req	x12
 #define REP8_7f 0x7f7f7f7f7f7f7f7f
 #define REP8_80 0x8080808080808080
 
-ENTRY(strlen)
+FUNC(strlen)
 	mov	zeroones, #REP8_01
 	bic	src, srcin, #15
 	ands	tmp1, srcin, #15
@@ -123,4 +123,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp1 )	/* Shift (tmp1 & 63).  */
 	csinv	data1, data1, xzr, le
 	csel	data2, data2, data2a, le
 	b	.Lrealigned
-ENDPROC(strlen)
+END(strlen)
diff --git a/xen/arch/arm/arm64/lib/strncmp.S b/xen/arch/arm/arm64/lib/strncmp.S
index a4a0f779f5..c00a641fc7 100644
--- a/xen/arch/arm/arm64/lib/strncmp.S
+++ b/xen/arch/arm/arm64/lib/strncmp.S
@@ -64,7 +64,7 @@ limit_wd	.req	x13
 mask		.req	x14
 endloop		.req	x15
 
-ENTRY(strncmp)
+FUNC(strncmp)
 	cbz	limit, .Lret0
 	eor	tmp1, src1, src2
 	mov	zeroones, #REP8_01
@@ -307,4 +307,4 @@ CPU_BE( orr	syndrome, diff, has_nul )
 .Lret0:
 	mov	result, #0
 	ret
-ENDPROC(strncmp)
+END(strncmp)
diff --git a/xen/arch/arm/arm64/lib/strnlen.S b/xen/arch/arm/arm64/lib/strnlen.S
index 81c8e8b54e..2059ba782c 100644
--- a/xen/arch/arm/arm64/lib/strnlen.S
+++ b/xen/arch/arm/arm64/lib/strnlen.S
@@ -59,7 +59,7 @@ limit_wd	.req	x14
 #define REP8_7f 0x7f7f7f7f7f7f7f7f
 #define REP8_80 0x8080808080808080
 
-ENTRY(strnlen)
+FUNC(strnlen)
 	cbz	limit, .Lhit_limit
 	mov	zeroones, #REP8_01
 	bic	src, srcin, #15
@@ -168,4 +168,4 @@ CPU_LE( lsr	tmp2, tmp2, tmp4 )	/* Shift (tmp1 & 63).  */
 .Lhit_limit:
 	mov	len, limit
 	ret
-ENDPROC(strnlen)
+END(strnlen)
diff --git a/xen/arch/arm/arm64/lib/strrchr.S b/xen/arch/arm/arm64/lib/strrchr.S
index 07059983f8..81033c0822 100644
--- a/xen/arch/arm/arm64/lib/strrchr.S
+++ b/xen/arch/arm/arm64/lib/strrchr.S
@@ -26,7 +26,7 @@
  * Returns:
  *	x0 - address of last occurrence of 'c' or 0
  */
-ENTRY(strrchr)
+FUNC(strrchr)
 	mov	x3, #0
 	and	w1, w1, #0xff
 1:	ldrb	w2, [x0], #1
@@ -37,4 +37,4 @@ ENTRY(strrchr)
 	b	1b
 2:	mov	x0, x3
 	ret
-ENDPROC(strrchr)
+END(strrchr)
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Tue May 14 20:32:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:32:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721770.1125431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yoW-0003bX-Q9; Tue, 14 May 2024 20:31:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721770.1125431; Tue, 14 May 2024 20:31:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6yoW-0003bQ-NJ; Tue, 14 May 2024 20:31:56 +0000
Received: by outflank-mailman (input) for mailman id 721770;
 Tue, 14 May 2024 20:31:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=loUn=MR=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s6yoV-0003bK-Mp
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:31:55 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fefc3ca4-1230-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:31:53 +0200 (CEST)
Received: from DS7PR03CA0328.namprd03.prod.outlook.com (2603:10b6:8:2b::30) by
 SA0PR12MB4430.namprd12.prod.outlook.com (2603:10b6:806:70::20) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.26; Tue, 14 May 2024 20:31:49 +0000
Received: from CH1PEPF0000AD7B.namprd04.prod.outlook.com
 (2603:10b6:8:2b:cafe::62) by DS7PR03CA0328.outlook.office365.com
 (2603:10b6:8:2b::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.25 via Frontend
 Transport; Tue, 14 May 2024 20:31:49 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000AD7B.mail.protection.outlook.com (10.167.244.58) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Tue, 14 May 2024 20:31:48 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 15:31:47 -0500
Received: from [172.18.62.51] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 15:31:46 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fefc3ca4-1230-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=efN4vsHwe/2TptbgIJrEQMzPEhMLOPd02xgPk1DzU/KmjCIfLGPMheFvTiIN/QNeHGt+3iXeN3jsNnzhCQlKu+bYMp2R/pTPa69Yl+r8UI5otHuso2cFWVum0qrgf8s44PKWbdRlOKNZ1TOR2GFLpsLnQ3ZEAamg8nvL60FPtvbqI/VoRPFWmDTiiBgEn2XT1nfGkei2cqOVSjYI5O64VfitqqV1NO9IAKTAmX3t33s+VJ2LYTwUuH1ovW45DEPHAqNMuO+0Zkcw5QdqT5P+0k+WCGk6R+Stn0zmk/RkMRGgpekFSRue1WbemcldW4NnK1n9Omdii3ZMZxHSMjk4Qw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5Tuv8uKyEOoj5fQkM8rpAb2/35C33Bxtn8x+6/yXHeU=;
 b=BhachzRcc5JMEe95RlerPxqZ+PxyO1ZJOXs19lWHnVxbZYVEi2NkeiA7Yo3hwpxHgTedjxfRwFG4Kcw2yEJF1/E9btXAoldq1mNxeZL7YNkie9mTZn7G6s3Rsn8kcHuWVRQuRgluKrM70oPvzb9bdbY0dy6IAw+CSXzlejLLpHvd8r+0AIhhuM7ToyTxy+X8x3Ql+HWtC/PIfYVn+2cqkkftTZitijD/on+geZs9oLQRGv5eG2srkt9c78HEk4owvigt4tduGsfkHQ3TGFoz+may6vjQkNfFwUhTXmdWdBJHc8x9g0uDJf5lNT+8Lxn6Fznbf3ij+jjhh5qCWQcjIQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Tuv8uKyEOoj5fQkM8rpAb2/35C33Bxtn8x+6/yXHeU=;
 b=fBw+yYVJWb8Ys4OGEuHLq3eRzQiy/6tvQQJF08uEZFcSI3xB+lflqesDhvEC8hUuwMflr0c3KQAotGemIG/Ue5B9YVZIFa8PDXsXA6JbydlfBwdEujf6sv9rmKxoqZTOb4Fn3giy6lkAM5M24K+EqtJAI/uf2sX4c14dZurxScw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <735c3e3c-d70a-40f5-aca7-f134bc5b98b1@amd.com>
Date: Tue, 14 May 2024 16:31:46 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 5/5] arm/vpci: honor access size when returning an
 error
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	"Michal Orzel" <michal.orzel@amd.com>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-6-stewart.hildebrand@amd.com>
 <cf646df7-0e92-4138-8b54-471c72f44cb8@xen.org>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <cf646df7-0e92-4138-8b54-471c72f44cb8@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7B:EE_|SA0PR12MB4430:EE_
X-MS-Office365-Filtering-Correlation-Id: e9df6e55-bdca-4159-737e-08dc7454e151
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|82310400017|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?U3pKckVQOTFPNTdJSWloUmY4QWN2blBhNGN3YjJ4aGN5UEIrNEdvaWZrZFdM?=
 =?utf-8?B?VzdZRVB6UUpRZFJkZ252ZXhQam5sNmNKREJ2Z09iVUUzM1pOeFZjb3U0OUdW?=
 =?utf-8?B?N0lLT1ZOTm9lU3BTZXpraWJQS1k0YUJQZGpNT0M0bjlRL3ZBY0pmNEdhcGw5?=
 =?utf-8?B?Zm00K0k5TVZNcE1STFJmQWh1RncyYUZuaEdIWW5OT3A2cStWeCticUVkTXVR?=
 =?utf-8?B?NTlaTERVSmZmM01vZmNDTlhnS0QzY0g3RXN3ZVdlUktjdkxwbDlBNHIzSUZy?=
 =?utf-8?B?RnVVVFR3bEZaMWdwazhQaFd1SWNJOEJZQW5pUmFxRHhaSU1iRThXc1FpRGI1?=
 =?utf-8?B?VHdFNng3TllxNm91RDVqNTVrZDBubzVPaHVnU2NOUkFKSXU2bi8vUS82TC9t?=
 =?utf-8?B?QkE2c2pMcEFDdjBQZlJrYTliNFVLQlpFME1aemRNVXVHc0hNeXpocEdFU0Iz?=
 =?utf-8?B?eDVDQ256c1VDOXFUZHBpOHFwYVUvdGcwUEVJNW54WkVzS2pqcGFlZjVZem8v?=
 =?utf-8?B?MmF6ZkhUNzVDd1A2ak1ZWHMzREd1UDJ3U29GNTQvRDRrU2UyWS9RZ0d5SFV5?=
 =?utf-8?B?ZWp6a21yaWdIK1djUGtuMTM4dTU2TWorN0dxRjVnQk8zdllZUjE1SE9TQko0?=
 =?utf-8?B?cjBLeDhWOUwrUUpENkY3SUQxRDRsZDkvRW9PaWl0dEp4RFErWVlXaEsvOFRL?=
 =?utf-8?B?a1QybEdXSkRZMXExTDJCU2t1c0lHbXdRdHoyN3N4WlFiM3dEcnByY3lBTHZ5?=
 =?utf-8?B?ZTE4eTJPVGRBc0Q3SDhYZGVCMkdVaEJ5aWZsZURLSkhpOFh5M3N6QjJ2SUc2?=
 =?utf-8?B?aHNXbC9pTmVkOTFZK0VWMG5aZmRwS3psMXUvWCszUmdaYk41L21ibVhvVTRN?=
 =?utf-8?B?YjNjK29HWHNVQXJ3bFFTSTNVaHRsTHV1Y2hBN2ZGcGZOcjJreEhldWRiYzFO?=
 =?utf-8?B?TnpNUGVoSVRsK21jVi8wcTdVMlNZSUZEUWVWbHpxUjdKVWJ1OHpPeERNLy9R?=
 =?utf-8?B?Mm5ac1NpL3oxRWt0Wm5vbEdwM2hZdlRqNnpxQnJERlVtdG1KeWpCYzNSQ2FE?=
 =?utf-8?B?c0U1b2tyR0NzWVQ4d1ViYlVpb2h2bGQ1dVcybUgzSDFBNUJZMmhDWCtZam5N?=
 =?utf-8?B?czVGWmFUcThWanRhbE5aWFJoOUlnbFZLUE5PajY0NlAreVpuZGFsMVlXNysx?=
 =?utf-8?B?R1F6bTd1dVF0RUVHTC9qVzIvK3FwTklobGZBSW9TTnN1Sk85bG4xUlNMSnFZ?=
 =?utf-8?B?QmUxMWRCdlNYU252WjVaNkV6U0xEQ3RaRDNiRVY4MFBxc2R5UXZFUlh3OTR5?=
 =?utf-8?B?SGhLNVRrK1NkN2hJaTJEZmZqK1JhSEdHQWMrUDlNZ2M3aWhoUEZQc0Y3VGl2?=
 =?utf-8?B?WUovaFZOdUJlZ2x4QThVVE9PNnBuNWk0c0t1ejU5Z2NnL1Y2QVZXd3VzZHJ1?=
 =?utf-8?B?RTdTTWFtaEVlN2J1ZWcxZ3J4Y0VoWWFsYXJuNGlNVkxwR3czRUxXV2ZyZjk4?=
 =?utf-8?B?VjFXOERpL1E5TnE4ZWt0UFYxSjJpbmd6S1lhUXFGMVJpR0hRaG9NL0JrN3hZ?=
 =?utf-8?B?RTFBT21hdDhHampETzE4SXFHUjM0SytPZjFsYWVpN3dSRlh3YkwreS9UbUpU?=
 =?utf-8?B?Qmc1TFJOQ1ZmNWJVNGVmUGxxOGZKM0FIY240M0t3Y29BSE5Ea1g4L2diWkdK?=
 =?utf-8?B?UXl5OWt2VWUrdExmaEFIT3JzSFphU3Baa0tkd0ppQ0FFR3oxcDRjL2MyWVpr?=
 =?utf-8?B?RzQvN2lxd243U0hpdVR5ajRZa3FkbmZqNVMzdCtjRVNEUC9PQktra2IxQXNy?=
 =?utf-8?B?NTNVTllTWTM1K2duTHRwTWNyQ3p6eHhnSVJlTDc1K1hRTFVKVXlOZC9uYjJZ?=
 =?utf-8?Q?hC/3slrsOmUR+?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(82310400017)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 20:31:48.5429
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9df6e55-bdca-4159-737e-08dc7454e151
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD7B.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4430

On 5/14/24 13:48, Julien Grall wrote:
> Hi Stewart,
> 
> On 14/05/2024 15:33, Stewart Hildebrand wrote:
>> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
>>
>> Guest can try to read config space using different access sizes: 8,
>> 16, 32, 64 bits. We need to take this into account when we are
>> returning an error back to MMIO handler, otherwise it is possible to
>> provide more data than requested: i.e. guest issues LDRB instruction
>> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
>> register.
>>
>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> 
> With one remark below:
> 
> Acked-by: Julien Grall <jgrall@amazon.com>

Thanks!

> 
>> ---
>> v9->10:
>> * New patch in v10.
>> ---
>>   xen/arch/arm/vpci.c | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
>> index 348ba0fbc860..aaf9d9120c3d 100644
>> --- a/xen/arch/arm/vpci.c
>> +++ b/xen/arch/arm/vpci.c
>> @@ -41,6 +41,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>>   {
>>       struct pci_host_bridge *bridge = p;
>>       pci_sbdf_t sbdf;
>> +    const uint8_t access_size = (1 << info->dabt.size) * 8;

I'd like to add a U suffix to the 1 to make it consistent with the
remaining occurrences in this file.

>> +    const uint64_t access_mask = GENMASK_ULL(access_size - 1, 0);
>>       /* data is needed to prevent a pointer cast on 32bit */
>>       unsigned long data;
>>   @@ -48,7 +50,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>>         if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
>>       {
>> -        *r = ~0UL;
>> +        *r = access_mask;
> 
> The name 'access_mask' is a bit confusing. I would not expect such value
> for be returned to the guest. What about 'invalid'?

That sounds good, I've made the change in my local tree.

> 
> Also can you confirm whether patches #4 and #5 be committed without
> the rest of the series?

Patch #4: no, it uses a constant defined in patch #2 ("vpci: add initial
support for virtual PCI bus topology").

Patch #5: conceptually, yes, but patch #3 ("xen/arm: translate virtual
PCI bus topology for guests") also modifies vpci_mmio_read(), so there
are rebase conflicts to resolve in both patches #3 and #5. Thinking more
about it, patch #5 falls more into the category of a fix than a feature,
so it probably should have been in the beginning of the series anyway.
Alright, I've reordered it and resolved the rebase conflicts in my local
tree.

Here's what the patch ("arm/vpci: honor access size when returning an
error") now looks like based on staging:

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 3bc4bb55082a..31e9e1d20751 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 {
     struct pci_host_bridge *bridge = p;
     pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    const uint8_t access_size = (1U << info->dabt.size) * 8;
+    const uint64_t invalid = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
@@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0ul;
+    *r = invalid;
 
     return 0;
 }

The patch ("xen/arm: translate virtual PCI bus topology for guests")
will then introduce a new use of the "invalid" variable.


From xen-devel-bounces@lists.xenproject.org Tue May 14 20:51:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 20:51:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721775.1125442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6z7k-0008AQ-Fg; Tue, 14 May 2024 20:51:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721775.1125442; Tue, 14 May 2024 20:51:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6z7k-0008AJ-Cj; Tue, 14 May 2024 20:51:48 +0000
Received: by outflank-mailman (input) for mailman id 721775;
 Tue, 14 May 2024 20:51:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0OWz=MR=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1s6z7j-00089q-86
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 20:51:47 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c50038e2-1233-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 22:51:44 +0200 (CEST)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.17.1) with ESMTPS id 44EKpS3x036440
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 14 May 2024 16:51:34 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 44EKpSDN036439;
 Tue, 14 May 2024 13:51:28 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c50038e2-1233-11ef-b4bb-af5377834399
Date: Tue, 14 May 2024 13:51:28 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>,
        Wei Liu <wl@xen.org>, Kelly Choi <kelly.choi@cloud.com>,
        Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Subject: Re: Serious AMD-Vi(?) issue
Message-ID: <ZkPO0D0/Jx4ZnRBO@mattapan.m5p.com>
References: <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com>
 <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
 <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook>
 <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
 <08503965-4ba9-4fe3-a176-895623f4ad92@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <08503965-4ba9-4fe3-a176-895623f4ad92@suse.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Tue, May 14, 2024 at 10:22:51AM +0200, Jan Beulich wrote:
> On 13.05.2024 22:11, Elliott Mitchell wrote:
> > On Mon, May 13, 2024 at 10:44:59AM +0200, Roger Pau Monn� wrote:
> >> Why do you mask the device SBDF in the above snippet?  I would really
> >> like to understand what's so privacy relevant in a PCI SBDF number.
> > 
> > I doubt it reveals much.  Simply seems unlikely to help debugging and
> > therefore I prefer to mask it.
> 
> SBDF in one place may be matchable against a memory address in another
> place. _Any_ hiding of information is hindering analysis. Please can
> you finally accept that it needs to be the person doing the analysis
> to judge what is or is not relevant to them?

Not going to happen as I'd accepted this long ago.  The usual approach
is all developers have PGP keys (needed for security issues anyway) and
you don't require all logs to be public.

I've noticed the core of the Xen project appears centered in the EU.  Yet
you're not catering to data privacy at all?  Or is this a service
exclusively provided to people who prove they're EU citizens?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Tue May 14 21:03:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 21:03:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721784.1125451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zJO-0002EY-EO; Tue, 14 May 2024 21:03:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721784.1125451; Tue, 14 May 2024 21:03:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zJO-0002ER-Bx; Tue, 14 May 2024 21:03:50 +0000
Received: by outflank-mailman (input) for mailman id 721784;
 Tue, 14 May 2024 21:03:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6zJM-0002EJ-PB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 21:03:48 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6zJG-00078u-1o; Tue, 14 May 2024 21:03:42 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6zJF-000174-Nm; Tue, 14 May 2024 21:03:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=bUtFDoucwgIQQ3W4NIzb5d8M24BnL6CN0ZmCDbDFVow=; b=zG+N2du3bHKcpAWTow+WfFnvzU
	ljE4ABJa2/NPu0OBUfKBrIScUlj/0/XGHe+f/XyPWEhQRlen5BtIYUREHarDrtUfyoDSMDc3hbf13
	ZD+Fum9GP/ytZaUJvnl/iaG+J2W4widiCb5dQVZ2g8I5fSeRZ6x4S3G9ge/gAu60pv5Y=;
Message-ID: <6ba241d8-77b7-4b1c-81f7-6558d4b38449@xen.org>
Date: Tue, 14 May 2024 22:03:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of
 mem_access.c
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: consulting@bugseng.com, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
 <e6b525cb-a1f1-494b-8dfe-c9d86cfc8f9e@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <e6b525cb-a1f1-494b-8dfe-c9d86cfc8f9e@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 14/05/2024 09:10, Jan Beulich wrote:
> On 10.05.2024 14:32, Alessandro Zucchelli wrote:
>> --- a/xen/arch/arm/include/asm/mem_access.h
>> +++ b/xen/arch/arm/include/asm/mem_access.h
>> @@ -17,6 +17,8 @@
>>   #ifndef _ASM_ARM_MEM_ACCESS_H
>>   #define _ASM_ARM_MEM_ACCESS_H
>>   
>> +#include <xen/types.h>
>> +
>>   static inline
>>   bool p2m_mem_access_emulate_check(struct vcpu *v,
>>                                     const struct vm_event_st *rsp)
>> @@ -35,12 +37,28 @@ static inline bool p2m_mem_access_sanity_check(struct domain *d)
>>    * Send mem event based on the access. Boolean return value indicates if trap
>>    * needs to be injected into guest.
>>    */
>> +#ifdef CONFIG_MEM_ACCESS
>>   bool p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec);
>>   
>>   struct page_info*
>>   p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>>                                     const struct vcpu *v);
>> +#else
>> +
>> +static inline bool
>> +p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct npfec npfec)
>> +{
>> +    return false;
>> +}
>> +
>> +static inline struct page_info*
>> +p2m_mem_access_check_and_get_page(vaddr_t gva, unsigned long flag,
>> +                                  const struct vcpu *v)
>> +{
>> +    return NULL;
>> +}
>>   
>> +#endif /*CONFIG_MEM_ACCESS*/
> 
> Why would each arch need to repeat these stubs? IOW why would they not
> live in xen/mem_access.h?

Because they are not used by nor defined in common code. It is pure 
coincidence they are named the same. If at some point, some code can be 
shared, then yes I would agree it could be common.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 21:06:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 21:06:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721789.1125461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zLa-0002mK-Q4; Tue, 14 May 2024 21:06:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721789.1125461; Tue, 14 May 2024 21:06:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zLa-0002mD-Ne; Tue, 14 May 2024 21:06:06 +0000
Received: by outflank-mailman (input) for mailman id 721789;
 Tue, 14 May 2024 21:06:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s6zLZ-0002m6-4K
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 21:06:05 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6zLY-0007C2-Q2; Tue, 14 May 2024 21:06:04 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s6zLY-0001Gi-KP; Tue, 14 May 2024 21:06:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Rp0QZ7JIHQgrzyYBjdX1OjwwMVcId1x6df9RifvwlRU=; b=bUg8lZ4uOAHk83LnR7kMqPOiJq
	jAxERM5lvurVpFmX7gTXW8ntczcMVEN9BEN03pLF9S+rmixFvOODle8m/KunRSL9SaNxUGPJisDRO
	nOgrA2mJ3wlKdtuPVvgndBDEcapyUrSxnsiDQbgL7/RRLX+H6lzqswJOiCddElctf5oA=;
Message-ID: <d41b633a-7702-41f6-b2b3-26321de53af2@xen.org>
Date: Tue, 14 May 2024 22:06:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Content-Language: en-GB
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-2-luca.fancellu@arm.com>
 <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
 <0A931CEA-59CA-4B1F-A8E4-A4CB990C0476@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <0A931CEA-59CA-4B1F-A8E4-A4CB990C0476@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 14/05/2024 08:53, Luca Fancellu wrote:
> Hi Julien,
> 
> Thanks for having a look on the patch,
> 
>> On 13 May 2024, at 22:54, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Luca,
>>
>> On 25/04/2024 14:11, Luca Fancellu wrote:
>>> Currently the code is listing device tree reserve map regions
>>> as reserved memory for Xen, but they are not added into
>>> bootinfo.reserved_mem and they are fetched in multiple places
>>> using the same code sequence, causing duplication. Fix this
>>> by adding them to the bootinfo.reserved_mem at early stage.
>>
>> Do we have enough space in bootinfo.reserved_mem for them?
> 
> So we have 255 banks, in my experience I would say I’ve never saw too many reserved regions
> in the DT, maybe a couple, but I’ve always had to deal with embedded platforms.
> 
> I’ve tested this one with ADLINK AVA board, n1sdp, Juno, raspberry pi, qemu, fvp.
> 
> In your experience, have you seen any numbers that could be concerning?
I know in the past we had to bump the memory banks a few times. But as 
you tested on a few platforms, I think we should be ok.

It would be best if this patch goes sooner than later to allow wider 
testing before we release 4.19.

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 14 21:36:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 21:36:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721801.1125472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zoU-0000UE-1P; Tue, 14 May 2024 21:35:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721801.1125472; Tue, 14 May 2024 21:35:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s6zoT-0000U7-V1; Tue, 14 May 2024 21:35:57 +0000
Received: by outflank-mailman (input) for mailman id 721801;
 Tue, 14 May 2024 21:35:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s6zoS-0000Tx-Qv
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 21:35:56 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id effe0537-1239-11ef-b4bb-af5377834399;
 Tue, 14 May 2024 23:35:53 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 606846135B;
 Tue, 14 May 2024 21:35:51 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id DA839C32786;
 Tue, 14 May 2024 21:35:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: effe0537-1239-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715722551;
	bh=gZ6eDocZ0a1RIgFarB/Rv3RWZzU6B+R8pgu9855wUEQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ozo5v5lz+HGp/j1jfmpJVCHUkJu929hCfb7/ThgC/1CJv4Z+zFntMtzRmrJsfMJVt
	 9sJiwY0T5MKswMnE8TVwidhoN/YWGDCs6wNgkBeD24nnTGtKqc5PrQm1qm3owaalL4
	 kC57P6dLQzhah9NH2HOP5cg3qJiGju69TjyGICYDi3Chuwvgp5T/PWWIOFAshBkF67
	 Wdc4WINL3j8SQSw7RlhQPoKcPXRGe1d4lAeL543gw/HADqZ4R9WKa3ZKQYvU6+3vsF
	 hLXEBlJN3M4b2UDoE5MzwXxomUtBEf5VIlWmKKFKRJ08v3Bg2Z7BDRapV2XIMsIVNx
	 ABWRSQ8LXsCQg==
Date: Tue, 14 May 2024 14:35:48 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <George.Dunlap@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Daniel Smith <dpsmith@apertussolutions.com>, 
    Xen-devel <xen-devel@lists.xenproject.org>, 
    "committers@xenproject.org" <committers@xenproject.org>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
In-Reply-To: <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405141411210.2544314@ubuntu-linux-20-04-desktop>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com> <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com> <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com> <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
 <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1627467485-1715721124=:2544314"
Content-ID: <alpine.DEB.2.22.394.2405141412250.2544314@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1627467485-1715721124=:2544314
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2405141412251.2544314@ubuntu-linux-20-04-desktop>

On Tue, 14 May 2024, Julien Grall wrote:
> Hi,
> 
> (+ Oleksii as the release manager)
> 
> Chiming into the discussion as there seems there is disagreement.
> 
> On 14/05/2024 11:03, Jan Beulich wrote:
> > On 14.05.2024 11:51, Andrew Cooper wrote:
> > > On 14/05/2024 10:25 am, Jan Beulich wrote:
> > > > On 03.04.2024 08:16, Jan Beulich wrote:
> > > > > On 02.04.2024 19:06, Andrew Cooper wrote:
> > > > > > The commit makes a claim without any kind of justification.
> > > > > Well, what does "have no business" leave open?
> > > > > 
> > > > > > The claim is false, and the commit broke lsevtchn in dom0.
> > > > > Or alternatively lsevtchn was doing something that was never meant to
> > > > > work
> > > > > (from Xen's perspective).
> > > > > 
> > > > > >   It is also quite
> > > > > > obvious from XSM_TARGET that it has broken device model stubdoms
> > > > > > too.
> > > > > Why would that be "obvious"? What business would a stubdom have to
> > > > > look at
> > > > > Xen's side of an evtchn?
> > > > > 
> > > > > > Whether to return information about a xen-owned evtchn is a matter
> > > > > > of policy,
> > > > > > and it's not acceptable to short circuit the XSM on the matter.
> > > > > I can certainly accept this as one possible view point. As in so many
> > > > > cases
> > > > > I'm afraid I dislike you putting it as if it was the only possible
> > > > > one.
> > > > > 
> > > > > In summary: The supposed justification you claim is missing in the
> > > > > original
> > > > > change is imo also missing here then: What business would any entity
> > > > > in the
> > > > > system have to look at Xen's side of an event channel? Back at the
> > > > > time, 3
> > > > > people agreed that it's "none".
> > > > You've never responded to this reply of mine, or its follow-up. You also
> > > > didn't chime in on the discussion Daniel and I were having. I consider
> > > > my
> > > > objections unaddressed, and in fact I continue to consider the change to
> > > > be wrong. Therefore it was inappropriate for you to commit it; it needs
> > > > reverting asap. If you're not going to do so, I will.
> > > 
> > > You tried defending breaking a utility with "well it shouldn't exist
> > > then".
> > > 
> > > You don't have a leg to stand on, and two maintainers of relevant
> > > subsystems here just got tired of bullshit being presented in place of
> > > any credible argument for having done the change in the way you did.
> > 
> > Please can you finally get into the habit of not sending rude replies?
> > 
> > > The correct response was "Sorry I broke things.  Lets revert this for
> > > now to unbreak, and I'll see about reworking it to not intentionally
> > > subvert Xen's security mechanism".
> > 
> > I'm sorry, but I didn't break things. I made things more consistent with
> > the earlier change, as pointed out before: With your revert,
> > evtchn_status() is now (again) inconsistent with e.g. evtchn_send(). If
> > you were serious about this being something that needs leaving to XSM,
> > you'd have adjusted such further uses of consumer_is_xen() as well. But
> > you aren't. You're merely insisting on lsevtchn needing to continue to
> > work in a way it should never have worked, with a patch to improve the
> > situation already pending.
> > 
> > Just to state a very basic principle here again: Xen-internal event
> > channels ought to either be fully under XSM control when it comes to
> > domains attempting to access them (in whichever way), or they should
> > truly be Xen-internal, with access uniformly prevented. To me the
> > former option simply makes very little sense.
> 
> I agree we need consistency on how we handle security policy event channel.
> Although, I don't have a strong opinion on which way to go.

Same here


> For the commit message, it is not entirely clear what "broke lseventch in
> dom0" really mean. Is it lsevtchn would not stop or it will just not display
> the event channel?
> 
> If the former, isn't a sign that the tool needs to be harden a bit more? If
> the latter, then I would argue that consistency for the XSM policy is more
> important than displaying the event channel for now (the patch was also
> committed 3 years ago...).

I realize 3 years have passed and it is a long time, but many
downstreams (including some which are widely used) don't rebase
regularly and we are still missing lots of tests from gitlab-ci. The
unfortunate result is that it can take years to realize there is a
breakage. We need more gitlab-ci (or OSSTest) tests.


> So I would vote for a revert and, if desired, replacing with a patch that
> would change the XSM policy consistently. Alternatively, the consistency
> should be a blocker for Xen 4.19.

I am convinced by Daniel's argument here:

https://marc.info/?l=xen-devel&m=171215093102694
https://marc.info/?l=xen-devel&m=171215073502479

I would ack Andrew's revert. If we decide to revert Andrew's revert, I
also think that we should make the alternative solution, whatever that
might be, a blocker for Xen 4.19.

My favorite alternative solition is Daniel's suggestion of adding a
check to the dummy XSM policy. I am not sure if this is the same thing
you mean with "change the XSM policy consistently".


> > > As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
> > > had for the principle of the change based on the absurdity of your
> > > arguments.
> > 
> > No, pending objections are pending objections. Daniel's responses didn't
> > eliminate them.
> 
> Indeed, this is rule 4 of the check-in policy:
> 
> 4. There must be no "open" objections.
> 
> I don't view Jan's objections as unreasonable in particular for the
> consistency part.
--8323329-1627467485-1715721124=:2544314--


From xen-devel-bounces@lists.xenproject.org Tue May 14 22:52:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 22:52:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721815.1125482 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s710M-0003jW-C4; Tue, 14 May 2024 22:52:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721815.1125482; Tue, 14 May 2024 22:52:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s710M-0003jP-9U; Tue, 14 May 2024 22:52:18 +0000
Received: by outflank-mailman (input) for mailman id 721815;
 Tue, 14 May 2024 22:52:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s710K-0003jJ-UB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 22:52:16 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9a00fa48-1244-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 00:52:14 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id A17EBCE1366;
 Tue, 14 May 2024 22:52:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D780C4AF08;
 Tue, 14 May 2024 22:52:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a00fa48-1244-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715727127;
	bh=I0J262r6gG+COYtQKQZ2a39Mr3dDQsi4g5lKC+PZavs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uT8EycxDfwmNJVrcfDH2ap/lcf3Qn0QGLskR/Fz1lhuegGBxuFU+q+6oJe6LmYx6J
	 MD2Q00Ml5Z7K05w8twSk61gE59Kv7TWtTAOpXjkAyivGL4rtzk2jVP5h9eRexkmbH+
	 dXFv2OnT0Gkfr41WhpRMYZjdOuMv5ljO3gS00iFetvtifBeEHZa4MTNrw2PUBuG5Qx
	 9n76r2i6KG5cNap9seVNNPaMh8rqZohkjfG0OXW0r3NEnaulCJkFe8jOnVuWUtgoKU
	 IA7/dPGa1qtl+IrEqdhhxC1xI3Wp0rLsNmiSCcXlqqh6HzBYomPbMRb+5eiXYe0A0Z
	 L7iokscJwpewg==
Date: Tue, 14 May 2024 15:52:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, nicola.vetrini@bugseng.com, 
    consulting@bugseng.com
Subject: Re: [PATCH] lib/strtoul: fix MISRA R10.2 violation
In-Reply-To: <16990192-aceb-408f-9247-45f3b1f58e0a@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405141540140.2544314@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop> <16990192-aceb-408f-9247-45f3b1f58e0a@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 14 May 2024, Jan Beulich wrote:
> On 14.05.2024 02:32, Stefano Stabellini wrote:
> > Fix last violation of R10.2 by casting the result of toupper to plain
> > char. Note that we don't want to change toupper itself as it is a legacy
> > interface and it would cause more issues.
> 
> Can you point me at a single example where a new issue would arise? All
> places I've spotted (including tolower() uses) would appear to benefit
> from changing toupper() / tolower() themselves. Further, since they are
> both wrapper macros only anyway, if any concern remained, fiddling with
> the wrapper macros while leaving alone the underlying inline functions
> would allow any such use site to simply be switched to using the inline
> functions directly. As said, from looking at it I don't expect that
> would be necessary, so instead I'd rather hope that eventually we can
> do away with the wrapper macros, renaming the inline functions
> accordingly.

If we change __toupper to return a plain char, then there are a few
other things we need to change for consistency, see below. To be honest
I thought it would cause more problems. I am OK to go with that if you
all agree. (Nicola please have a look in case this introduces more
issues elsewhere.)


diff --git a/xen/include/xen/ctype.h b/xen/include/xen/ctype.h
index 6dec944a37..6a6854e01c 100644
--- a/xen/include/xen/ctype.h
+++ b/xen/include/xen/ctype.h
@@ -15,9 +15,9 @@
 #define _X	0x40	/* hex digit */
 #define _SP	0x80	/* hard space (0x20) */
 
-extern const unsigned char _ctype[];
+extern const char _ctype[];
 
-#define __ismask(x) (_ctype[(int)(unsigned char)(x)])
+#define __ismask(x) (_ctype[(int)(char)(x)])
 
 #define isalnum(c)	((__ismask(c)&(_U|_L|_D)) != 0)
 #define isalpha(c)	((__ismask(c)&(_U|_L)) != 0)
@@ -34,14 +34,14 @@ extern const unsigned char _ctype[];
 #define isascii(c) (((unsigned char)(c))<=0x7f)
 #define toascii(c) (((unsigned char)(c))&0x7f)
 
-static inline unsigned char __tolower(unsigned char c)
+static inline char __tolower(char c)
 {
 	if (isupper(c))
 		c -= 'A'-'a';
 	return c;
 }
 
-static inline unsigned char __toupper(unsigned char c)
+static inline char __toupper(char c)
 {
 	if (islower(c))
 		c -= 'a'-'A';
diff --git a/xen/lib/ctype.c b/xen/lib/ctype.c
index 7b233a335f..27e5b61b1c 100644
--- a/xen/lib/ctype.c
+++ b/xen/lib/ctype.c
@@ -1,7 +1,7 @@
 #include <xen/ctype.h>
 
 /* for ctype.h */
-const unsigned char _ctype[] = {
+const char _ctype[] = {
     _C,_C,_C,_C,_C,_C,_C,_C,                        /* 0-7 */
     _C,_C|_S,_C|_S,_C|_S,_C|_S,_C|_S,_C,_C,         /* 8-15 */
     _C,_C,_C,_C,_C,_C,_C,_C,                        /* 16-23 */


From xen-devel-bounces@lists.xenproject.org Tue May 14 22:57:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 22:57:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721821.1125495 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s715i-0004Ml-11; Tue, 14 May 2024 22:57:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721821.1125495; Tue, 14 May 2024 22:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s715h-0004Me-UH; Tue, 14 May 2024 22:57:49 +0000
Received: by outflank-mailman (input) for mailman id 721821;
 Tue, 14 May 2024 22:57:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s715g-0004MF-TB
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 22:57:48 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 60bada91-1245-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 00:57:46 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 4621A61354;
 Tue, 14 May 2024 22:57:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B80A1C2BD10;
 Tue, 14 May 2024 22:57:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60bada91-1245-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715727465;
	bh=Fa3PKxp/yI/ts4ijVfHaICrW2Uv9/Hm2uNufvNgEoEs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RJpnlHULu92YKrhsJOrJKyWevHewDb9RvIovQgLCSVPVGAiNxtt/Mm2YIwcO+QR8N
	 vlBJna9CZcJrHL6lTh2votwHM6uuxtd5+I5BL1GVCrg/8xXrF+zpREjUlOGsijJNlA
	 gTqUoxgV0rvkE3IW04julBa+DU8eBEePGX0cIADqQ5Qbs6UIO5GStfnmQ+RDYJJXgg
	 69jZMs/m8veKJPieomNAWzAynu9Szlod5nkRo0igYrr7wL899uLjjdwExLufT6Neew
	 UE5nqmhVd0fMLqwgjYowB3q0WMNs8+w0oKlizrWCB4wVjRtb5nIA6uV9M/3EzhS7Ti
	 Rt5vj7OuvNF1g==
Date: Tue, 14 May 2024 15:57:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, julien@xen.org, michal.orzel@amd.com, 
    bertrand.marquis@arm.com, roger.pau@citrix.com, 
    xen-devel@lists.xenproject.org, roberto.bagnara@bugseng.com
Subject: Re: [PATCH] docs/misra: replace R13.1 with R13.2
In-Reply-To: <369a3c75-2e71-4fb8-8373-aae737099e3b@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405141556050.2544314@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405101625470.2544314@ubuntu-linux-20-04-desktop> <369a3c75-2e71-4fb8-8373-aae737099e3b@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 14 May 2024, Jan Beulich wrote:
> On 11.05.2024 01:32, Stefano Stabellini wrote:
> > After looking closely at the R13.1 violations and potential fixes or
> > deviations, upon further reflection and discussions, we realized that it
> > is R13.2, limited to initializers list, that we want to comply with.
> > 
> > Link: https://marc.info/?l=xen-devel&m=170751643325805
> 
> The mail you point at is a reply to one where I raised an aspect that was
> never really addressed: 'I'm afraid I don't understand the "eventually
> limiting" part.' Therefore I also don't follow why we'd want to limit
> applicability of this rule to just initializer lists.

I was trying to make a 1:1 replacement, but I see your point and I
agree. We'll have to discuss at the next MISRA C meeting whether to
accept R13.2 in full or partially.


From xen-devel-bounces@lists.xenproject.org Tue May 14 23:15:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 May 2024 23:15:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721832.1125506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s71N0-0007Ws-JY; Tue, 14 May 2024 23:15:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721832.1125506; Tue, 14 May 2024 23:15:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s71N0-0007Wl-GB; Tue, 14 May 2024 23:15:42 +0000
Received: by outflank-mailman (input) for mailman id 721832;
 Tue, 14 May 2024 23:15:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o5lq=MR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s71My-0007Wf-Ry
 for xen-devel@lists.xenproject.org; Tue, 14 May 2024 23:15:40 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0334487-1247-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 01:15:39 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2149061354;
 Tue, 14 May 2024 23:15:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4537FC2BD10;
 Tue, 14 May 2024 23:15:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0334487-1247-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715728537;
	bh=cknac0lzprvjGuU+wJs9zjIFW4bKs7DnewDXXFJb3QY=;
	h=Date:From:To:cc:Subject:From;
	b=hZ9dRbBaeFtCayr8f8fEauC7EV/6qvvyFp+KajEZKF++thQHKTAalAUwLzIH3a+UK
	 UKC93DFyXUTl1brfrIH4SZNGULhclb7HBA+LB3GCthH4On292ASiyTN+9rBZna1/jk
	 p0PTshEj/VcH0gcBncGCtPiFdaABPOhZ5Tgk5bg9bvwy+BHnwHi3kb/t7+drb+bt3B
	 G2UQ+0YNg6ZaI7o7wDBpPdkowe2uzRn6VhO4bgsnseaMNT8S8SJTf8GYj+7D5K6qp0
	 PYFGhaM4zEWQBbx4zo13oJiNXtPkqNuygwB4xa8SiEBGzzbQsfR+KTTICPOkjnxp4T
	 sUPngWk+L3LPg==
Date: Tue, 14 May 2024 16:15:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Michal Orzel <michal.orzel@amd.com>, roberto.bagnara@bugseng.com, 
    consulting@bugseng.com
Subject: [PATCH] docs/misra: add D4.12
Message-ID: <alpine.DEB.2.22.394.2405141611170.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Add D4.12 with the same explanation as the rules of the R21 series.
D4.12 refers to the standard library memory allocation functions and
similar third party libraries with memory allocation functions. It
doesn't refer to the in-tree implementation we have in Xen which is
subject to MISRA C rules and MISRA C scanning.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 80e5e972ad..bc8506add4 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -76,6 +76,11 @@ maintainers if you want to suggest a change.
        considered libraries from MISRA C point of view as they are
        imported in source form)
 
+   * - `Dir 4.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/D_04_12.c>`_
+     - Required
+     - Dynamic memory allocation shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Dir 4.14 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/D_04_14.c>`_
      - Required
      - The validity of values received from external sources shall be


From xen-devel-bounces@lists.xenproject.org Wed May 15 00:57:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 00:57:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721842.1125517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72xM-0004Hr-RZ; Wed, 15 May 2024 00:57:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721842.1125517; Wed, 15 May 2024 00:57:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72xM-0004Hk-Ly; Wed, 15 May 2024 00:57:20 +0000
Received: by outflank-mailman (input) for mailman id 721842;
 Wed, 15 May 2024 00:57:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DUSJ=MS=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s72xL-0004He-Go
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 00:57:19 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 139ede97-1256-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 02:57:18 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-572669fd9f9so1099843a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 17:57:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 139ede97-1256-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715734637; x=1716339437; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TgKeQCtSGXMOTtoEpL9MupCi/snafMG2ArhxtXmZLco=;
        b=ngJ3O788aO7Khdj9qPS0th7eRL3r5eVpDrkfq9Rjrz3OT7N22XHN74zLhWQzMisrtX
         PoIsfepri+NnR0ai5EapA/5yA3LFgqPu6GYk4H5BuDmH23LMShTnF6tBAe9yabg5BIO7
         VVxSEz7iyHoaWMjHCd/SX9hhlAhXOeQ+8LpmjFfY4gnNu1gv5iQE2W6ywviO9IKa+/cW
         pNfOQ9khesja+BWQrrkh+5O6en5hjVnTgbp2PsnGs/xrVrVrqL4Nzw82A9Ts3799SS1r
         1mVfsaFfXB6aT179s1o17DMSPZM/dhT4OdHh+h61SyJnLaFQUJSDWT520RE+3SZQyKzi
         SkVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715734637; x=1716339437;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=TgKeQCtSGXMOTtoEpL9MupCi/snafMG2ArhxtXmZLco=;
        b=vka1wDiXGIEPaPRGnTNNvLh7cxfNdxfYOSHW/tN03BceuTNLUVLSbqHQ0pclibt29o
         CeIFx2I+1jZmjkScNJe8fIvGNcqqNSgxnLdAgJAxJlQOa+6lWH9KlLFpDTjsqG9gLsIv
         hxmL/X8Hze72GABriHA+OXslpEsblcY9q4mmYDshYZ3Z+G0D2eNP7rLX1B/Hyh2oLn+k
         HKjRCMQr8atyFcUZbt7wgKi2hX/SZgBLFxWM54dX/tvEuOYY/l+RhGH25xjkb015NrOs
         G/ZIwuk1FR1to1yuSbmjKLLHZh7AlrRfUduFRYLUuHP1NbUG19hgEHUSiflgX2DoWcFT
         6eYA==
X-Gm-Message-State: AOJu0YyuXRY36eQfdXl+bxSWjoqGYFUx9rvIt343W+tOuIQ9vk/TKeY8
	pciR450SvuOsdsfq5OXdtMHoZ9Bmzfs4xvUvzh0Iw91/AA+Tlt11BQLXBvL3/43pKN8eAjgSGE1
	0YMs6vw7ou88OQnmoUK2rJHZObvhKIMie
X-Google-Smtp-Source: AGHT+IGtWA2CGE+sxULYJnuwa46PUC7MelEpKWom0NOkGcnLQHu4VCLUvA9ib8yUsb/t9fmIGkjfYgMQ96W5z+ZXB/U=
X-Received: by 2002:a50:d5ca:0:b0:572:a4eb:6682 with SMTP id
 4fb4d7f45d1cf-5734d6b1f9amr9772610a12.30.1715734637044; Tue, 14 May 2024
 17:57:17 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <20240508213824.55824-4-leigh@solinno.co.uk>
In-Reply-To: <20240508213824.55824-4-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Tue, 14 May 2024 20:57:05 -0400
Message-ID: <CAKf6xpuNJbf2ioVFyvu2L6YG2_eNRbqPF8tn7Yodyv5UA4uE6A@mail.gmail.com>
Subject: Re: [RFC PATCH v2 3/5] tools/hotplug/Linux: Add bridge VLAN support
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
	anthony.perard@citrix.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 6:55=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> wr=
ote:
>
> Update add_to_bridge shell function to read the vlan parameter
> from xenstore and set the bridge VLAN configuration for the VID.
>
> Add additional helper functions to parse the vlan specification,
> which consists of one or more of the follow:
>
> a) single VLAN (e.g. 10).
> b) contiguous range of VLANs (e.g. 10-15).
> c) discontiguous range with base, increment and count
>    (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).
>
> A single VLAN can be suffixed with "p" to indicate the PVID, or
> "u" to indicate untagged. A range of VLANs can be suffixed with
> "u" to indicate untagged.  A complex example would be:
>
>    vlan=3D1p/10-15/20-25u
>
> This capability only works when using the iproute2 bridge command,
> so a warning is issued if the vlan parameter is set and the bridge
> command is not available, as it will be ignored.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
> ---
>  tools/hotplug/Linux/xen-network-common.sh | 111 ++++++++++++++++++++++
>  1 file changed, 111 insertions(+)
>
> diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Li=
nux/xen-network-common.sh
> index 42fa704e8d..d9fb4f7355 100644
> --- a/tools/hotplug/Linux/xen-network-common.sh
> +++ b/tools/hotplug/Linux/xen-network-common.sh
> @@ -121,10 +121,113 @@ create_bridge () {
>      fi
>  }
>
> +_vif_vlan_add() {
> +    # References vlans, pvid and error variables from the calling functi=
on
> +    local -i vid=3D$1
> +    local flag=3D${2:-}
> +
> +    if (( vid < 1 || vid > 4094 )) ;then
> +        error=3D"vlan id $vid not between 1 and 4094"
> +        return
> +    fi
> +    if [[ -n "${vlans[$vid]}" ]] ;then
> +        error=3D"vlan id $vid specified more than once"
> +        return

You could do `fatal "vlan id $vid specified more than once"` and just
terminate the script at this point.  It would simplify your later code
if you use fatal more.

> +    fi
> +    case $flag in
> +     p) if (( pvid !=3D 0 )) ;then
> +            error=3D"more than one pvid specified ($vid and $pvid)"
> +            return
> +        fi
> +        pvid=3D$vid
> +        vlans[$vid]=3Dp ;;
> +     u) vlans[$vid]=3Du ;;
> +     *) vlans[$vid]=3Dt ;;
> +    esac
> +}
> +
> +_vif_vlan_parse_term() {
> +    # References error variable from the calling function
> +    local vid incr last term=3D${1:-}
> +
> +    if [[ $term =3D~ ^([0-9]+)([pu])?$ ]] ;then

I like that you split the different cases into multiple REs.

> +        _vif_vlan_add ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}
> +    elif [[ $term =3D~ ^([0-9]+)-([0-9]+)(u)?$ ]] ;then
> +        vid=3D${BASH_REMATCH[1]}
> +        last=3D${BASH_REMATCH[2]}
> +        if (( last >=3D vid )) ;then
> +            for (( ; vid<=3Dlast; vid++ )) ;do
> +                _vif_vlan_add $vid ${BASH_REMATCH[3]}
> +            done
> +        else
> +           error=3D"invalid vlan id range: $term"
> +        fi
> +    elif [[ $term =3D~ ^([0-9]+)\+([0-9]+)x([0-9]+)(u)?$ ]] ;then
> +        vid=3D${BASH_REMATCH[1]}
> +        incr=3D${BASH_REMATCH[2]}
> +        for (( j=3D${BASH_REMATCH[3]}; j>0; --j, vid+=3Dincr ))
> +        do
> +            _vif_vlan_add $vid ${BASH_REMATCH[4]}
> +        done
> +    else
> +        error=3D"invalid vlan specification: $term"
> +    fi
> +}
> +
> +_vif_vlan_validate_pvid() {
> +    # References vlans and pvid variables from the calling function
> +    if (( pvid =3D=3D 0 )) ;then
> +        if (( ${#vlans[@]} =3D=3D 1 )) ;then
> +            vlans[${!vlans[*]}]=3Dp
> +        else
> +            error=3D"pvid required for multiple vlan ids"
> +        fi
> +    fi
> +}
> +
> +_vif_vlan_setup() {
> +    # References vlans and dev variable from the calling function
> +    local vid cmd
> +
> +    bridge vlan del dev "$dev" vid 1
> +    for vid in ${!vlans[@]} ;do
> +        cmd=3D"bridge vlan add dev '$dev' vid $vid"
> +        case ${vlans[$vid]} in
> +             p) cmd=3D"$cmd pvid untagged" ;;
> +             u) cmd=3D"$cmd untagged" ;;
> +             t) ;;
> +        esac
> +        eval "$cmd"
> +    done
> +}
> +
> +_vif_vlan_membership() {
> +    # The vlans, pvid, dev and error variables are used by sub-functions
> +    local -A vlans=3D()
> +    local -a terms=3D()
> +    local -i i pvid=3D0
> +    local dev=3D$1 error=3D""
> +
> +    # Split the vlan specification string into its terms
> +    readarray -d / -t terms <<<$2
> +    for (( i=3D0; i<${#terms[@]}; ++i )) ;do
> +        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}
> +        [[ -n "$error" ]] && break
> +    done
> +
> +    [[ -z "$error" ]] && _vif_vlan_validate_pvid
> +    [[ -z "$error" ]] && _vif_vlan_setup
> +    [[ -z "$error" ]] && return 0
> +
> +    log error "$error"
> +    return 1
> +}
> +
>  # Usage: add_to_bridge bridge dev
>  add_to_bridge () {
>      local bridge=3D$1
>      local dev=3D$2
> +    local vlan=3D$(xenstore_read_default "$XENBUS_PATH/vlan" "")
>
>      # Don't add $dev to $bridge if it's already on the bridge.
>      if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
> @@ -134,6 +237,14 @@ add_to_bridge () {
>          else
>              ip link set ${dev} master ${bridge}
>          fi
> +        if [ -n "${vlan}" ] ;then
> +            if which bridge >&/dev/null; then
> +                log debug "configuring VLANs for ${dev} on ${bridge}"
> +                _vif_vlan_membership "${dev}" "${vlan}"
> +            else
> +                log warning "bridge command not available, ignoring vlan=
 config"

Do you think this should be fatal?  It seems to me that setting up the
connection but not applying the vlans could be a security issue.

This file before your patch was very close to posix sh.  Afterwards,
it definitely needs bash.  vif-bridge is /bin/bash, so it is fine.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Wed May 15 00:58:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 00:58:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721845.1125525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72y7-0004kx-11; Wed, 15 May 2024 00:58:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721845.1125525; Wed, 15 May 2024 00:58:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72y6-0004kq-UP; Wed, 15 May 2024 00:58:06 +0000
Received: by outflank-mailman (input) for mailman id 721845;
 Wed, 15 May 2024 00:58:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DUSJ=MS=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s72y6-0004kf-8T
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 00:58:06 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 302a0872-1256-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 02:58:05 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-56e47843cc7so867765a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 17:58:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 302a0872-1256-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715734685; x=1716339485; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9HHbrBkoepckM0AmPsXzpj3xxagqOFnXkdOz/Dig7TI=;
        b=b2Z6AiNolZOYUmjDfmJ/3pPTkuN445RIeH+WeNUZRpaMWPOoB5JhXDXjSOJe5cqanL
         9fQt9kPlxG8d4vLDXOGabwFg/D2xiutlUHaOyQWYE0/tHq1LqnjxaA+T/eHAdeaEfBOv
         Im09Jpix4JDm+nECy2bsr/uA3oHnpxxnH+V6BRPqZt4B9xa3fKJuwVG9OqH2elurxwZ9
         oUxcc2d/3ik+FS68rtDezMXz1TiEAgxjHNNy18NDFfFhMhWweB19qgpjkDz/f/wGt62o
         92YRkfhiu/hAAITqZWfdo573H9HBX65DdoPjTn/mnPulUDbADl8wQTgp0n4pTg/qtFU8
         SrMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715734685; x=1716339485;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9HHbrBkoepckM0AmPsXzpj3xxagqOFnXkdOz/Dig7TI=;
        b=X9JuQ2EznctG4ybjSbytaMttriRLj+dqdFcd94C+JF8VrasJkVOVr8Gg8SK/BOf9AO
         ITg1vodI4dNaTmViHc28XtVA2TEqjCYKvAs66J+vybuQLXxNZ7WMtUhP4M0DtQQDOrPq
         pRKPP10CgoJgF1BIU4C5dh13ix8d4+DoF5+QnnylHcgtTjW4voYGrvODZnMJnRoZ2jjO
         pCxPV/0ltbHn91MEZWfguG4t6Detae4CPL3VZQiYYui12tr4HVDCEgwzrmf3OUmEhlhM
         O1mj5FEXt83IBFmbkuLOPRs8e9UybCbW3oZbF1UavXRahntKAUJcU/c6rQs00a184+of
         yswA==
X-Gm-Message-State: AOJu0Yykk6RupUbiGA0IZDg02Nn/477ajcYSqWsSeDEvUVzSoZAp1ci0
	hhDR1zCRFvJUEIz8IyR+Hhsx3iQ8I4kwX6273EuRZn0dee1o99qTYS4plUPaN1pXSGoDE1UzCxb
	8cDshA47OKJ4/VCaIh33GCWCDuNg=
X-Google-Smtp-Source: AGHT+IHxi/S6V/m6R9xB1lBBLBBnHYJxi7+hBywOZsrbpaCAjeb837qjWoDnmRl/zY87RHObwJTseyGyxkCduq02euQ=
X-Received: by 2002:a50:d496:0:b0:572:93fa:d6fa with SMTP id
 4fb4d7f45d1cf-5734d43e905mr9234200a12.0.1715734685161; Tue, 14 May 2024
 17:58:05 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <20240508213824.55824-5-leigh@solinno.co.uk>
In-Reply-To: <20240508213824.55824-5-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Tue, 14 May 2024 20:57:53 -0400
Message-ID: <CAKf6xpviWyFxt4JMq23K0LxjDcP2zJF1Sqye=1jS2UAtwcaRww@mail.gmail.com>
Subject: Re: [RFC PATCH v2 4/5] docs/man: document VIF vlan keyword
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
	anthony.perard@citrix.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 5:39=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> wr=
ote:
>
> Document the new `vlan' keyword in xl-network-configuration(5).
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

One nit below

> ---
>  docs/man/xl-network-configuration.5.pod.in | 38 ++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/docs/man/xl-network-configuration.5.pod.in b/docs/man/xl-net=
work-configuration.5.pod.in
> index f3e379bcf8..c35c0922b3 100644
> --- a/docs/man/xl-network-configuration.5.pod.in
> +++ b/docs/man/xl-network-configuration.5.pod.in
> @@ -259,6 +259,44 @@ Specifies the MTU (i.e. the maximum size of an IP pa=
yload, exclusing headers). T

> +Note, one VLAN ID must be marked as the PVID. In the case of a vlan
> +specification consisting of a single VLAN ID (e.g. C<vlan=3D10>), the B<=
p> suffix
> +may be omitted. Specifying more than one untagged VLAN ID is an advanced
> +configuration - use with caution.
> +
> +For example:
> +
> +        'vlan=3D10' -- meaning a single VLAN that is the PVID.
> +       'vlan=3D10p/20' -- VLAN 10 is the PVID and VLAN 20 is tagged.
> +       'vlan=3D10p/100+10x4' -- VLANs 10, 100, 110, 120, 130, 140, 150.

Indent mismatch between 7 and 8 spaces.

> +
>  =3Dhead2 trusted / untrusted
>
>  An advisory setting for the frontend driver on whether the backend shoul=
d be
> --
> 2.39.2
>
>


From xen-devel-bounces@lists.xenproject.org Wed May 15 00:58:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 00:58:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721847.1125536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72yd-0005Fh-7y; Wed, 15 May 2024 00:58:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721847.1125536; Wed, 15 May 2024 00:58:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s72yd-0005Fa-5O; Wed, 15 May 2024 00:58:39 +0000
Received: by outflank-mailman (input) for mailman id 721847;
 Wed, 15 May 2024 00:58:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DUSJ=MS=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s72yc-0005FN-Ex
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 00:58:38 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 430cdcba-1256-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 02:58:37 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-5731ffcc905so1083403a12.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 17:58:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 430cdcba-1256-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715734717; x=1716339517; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GMaRqChvg6ywj2luDw1uwT8qy9H7hw8ShNEFLXpsHR0=;
        b=KMcSerrE+kEHIqC1zAqGB8vfhK+cXsrWJOZlOtlyRg5QaxaymbwKjMaGgceUhOXOS3
         XivawL8OrpNw7UwtsEQzmIxN8FMAYjwNk9lq4ht02/iozwrPxzmkPfvLqSGpyPRXN4Ti
         tuk+0VzYuHT7IDtgb4O6/h7wftTeRks6Z24BSFi5awj12+xqeAVXwR7Ldq/XxjXkXCw5
         DGGGJ4t3eoAXoz4IE9iDK/TaCFvpn5t1yN8bzsno1uYcs5mZNo44Uw7KACE9IxsglfXw
         kDuvC9GsFkdBRKDpZgSiY1kV9TY1uGcAwTjbcp+9OmXFclYoeX2rz1tf5yExmabYc8N2
         XdTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715734717; x=1716339517;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GMaRqChvg6ywj2luDw1uwT8qy9H7hw8ShNEFLXpsHR0=;
        b=E6BWyYilrYu5OqcG09kvn1PiDq+qpEh1aLOelSmobkN+qauSt3q+qml8xCFA63/ely
         sotKvt91LZQ0LMuRz45xRYepgcRyG+UCEs6cLe+sPdhAMuN9pcDh3xmBTs1VZ9K1sYbg
         Nw12YgQA2UvHbogbx27dO2EgaL73kWl5zz0dtcvxU95cizl6wMwGxxkY/dte8OPUCfpz
         9SwKV6s833rIahQXXJouHpK0T7SQ/p1rHip0LCpwUB9dnDrv/x/J6PkD9nMjyuTycTWT
         mk06SN0lGTbnMpld+SXs3vIZ9iCF5YHQ2ZppO7rSbsZwkF7I0xZ/hkETu2tNViSdsUmT
         H9JQ==
X-Gm-Message-State: AOJu0YxJVByEICjaYCW2NiBMWO4MsY7CH/eoJS12oI2f+jfjp/msdsx4
	MZ2A8hpr7W1pZxZK30ORZtIRKMERbIKkyPOZw1mCiTjp9Of/xeYLHbhW6+IN4q2vFdJBqwmngxQ
	BCn06cHmh6REkyWYFA7qq7JmHIFch/A==
X-Google-Smtp-Source: AGHT+IEdQpDwy8m9i2Rd8tGtRXIifUwAfX7QnVY9yK1Y8SXbL3W7WPFgjhJCNzD+XdNm7QJxeVHETSVmTP2ze5U1ucE=
X-Received: by 2002:a50:ccdd:0:b0:574:eb22:3629 with SMTP id
 4fb4d7f45d1cf-574eb223677mr1734724a12.19.1715734717001; Tue, 14 May 2024
 17:58:37 -0700 (PDT)
MIME-Version: 1.0
References: <20240508213824.55824-1-leigh@solinno.co.uk> <20240508213824.55824-6-leigh@solinno.co.uk>
In-Reply-To: <20240508213824.55824-6-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Tue, 14 May 2024 20:58:25 -0400
Message-ID: <CAKf6xpt+UTwY0hFuODhpeb6rWOB0kgbqrrFVprP7MZYOTJ59YA@mail.gmail.com>
Subject: Re: [RFC PATCH v2 5/5] tools/examples: Example Linux bridge VLAN config
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com, 
	anthony.perard@citrix.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 8, 2024 at 6:08=E2=80=AFPM Leigh Brown <leigh@solinno.co.uk> wr=
ote:>
> Add a new directory linux-bridge-vlan with examples files showing
> how to configure systemd-networkd to support a bridge VLAN
> configuration.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
> ---
>  tools/examples/linux-bridge-vlan/README       | 68 +++++++++++++++++++
>  tools/examples/linux-bridge-vlan/br0.netdev   |  7 ++
>  tools/examples/linux-bridge-vlan/br0.network  |  8 +++
>  .../examples/linux-bridge-vlan/enp0s0.network | 16 +++++
>  4 files changed, 99 insertions(+)
>  create mode 100644 tools/examples/linux-bridge-vlan/README
>  create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
>  create mode 100644 tools/examples/linux-bridge-vlan/br0.network
>  create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network

I think putting these in docs/misc/linux-bridge-vlan/ might be a
better location.

> diff --git a/tools/examples/linux-bridge-vlan/README b/tools/examples/lin=
ux-bridge-vlan/README
> new file mode 100644
> index 0000000000..83b9fa3fd6
> --- /dev/null
> +++ b/tools/examples/linux-bridge-vlan/README
> @@ -0,0 +1,68 @@
> +Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> +
> +Introduction
> +------------
> +
> +This directory contains example files to be placed in /etc/systemd/netwo=
rk
> +to enable a single bridge with multiple VLAN support.
> +
> +The example is to support the scenario where the Xen host network interf=
ace
> +is connected to an Ethernet switch configured as a trunk port. Each doma=
in
> +VIF can then be configured with one or more VLAN IDs, one of which will =
be
> +the PVID.
> +
> +The example files create a bridge device called br0, with a physical int=
erface
> +called enp0s0. You will need to update this with your system's device na=
me.
> +
> +Key points of the configuration are:
> +
> +1. In br0.netdev, VLANFiltering=3Don is set. This is required to ensure =
the
> +   VLAN tags are handled correctly.  If it is not set then the packets
> +   from the VIF interfaces will not have the correct VLAN tags set.
> +
> +2. In br0.network, a system IPv4 address is configured that can be updat=
ed
> +   according to your local network settings.
> +
> +3. In enp0s0.network, Bridge=3Dbr0 sets the bridge device to connect to.=
 There
> +   is also a [BridgeVLAN] section for each VLAN you want to give access
> +   to the switch. Note, if you want to create an internal VLAN private t=
o

For
"for each VLAN you want to give access to the switch"
do you mean:
"for each VLAN you want connected with the external network"
or
"for each VLAN you want accessible on the external network"
?

The "access to the switch" part I find unclear.

> +   the host, do not include its VLAN ID in this file.


> +Domain configuration
> +--------------------
> +
> +Add the vlan=3D keyword to the vif definition in the domain. The simples=
t
> +and most common example is a domain that wishes to connect to a single V=
LAN:
> +
> +vif =3D [ 'mac=3Dxx:xx:xx:xx:xx:xx, bridge=3Dbr0, vlan=3D10' ]
> +
> +If you wish to configure a domain to route between two VLANs, you have t=
wo
> +options. Option 1 is to create multiple interfaces on different VLANs:
> +
> +vif =3D [ 'mac=3Dxx:xx:xx:xx:xx:xx, bridge=3Dbr0, vlan=3D10',
> +       'max=3Dxx:xx:xx:xx:xx:xx, bridge=3Dbr0, vlan=3D20' ]

Hard tab here makes the diff look off, but the file would be aligned.

I think this is good content.  I'm not familiar with the systemd
networking stuff to give an R-b.  But it's only examples, so I think
it should be okay.  I'm not a maintainer, but it would be an Acked-by,
if I were.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Wed May 15 01:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 01:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721868.1125545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s73gS-0002dA-9Y; Wed, 15 May 2024 01:43:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721868.1125545; Wed, 15 May 2024 01:43:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s73gS-0002d3-6i; Wed, 15 May 2024 01:43:56 +0000
Received: by outflank-mailman (input) for mailman id 721868;
 Wed, 15 May 2024 01:43:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3gy8=MS=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s73gR-0002cx-Ei
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 01:43:55 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9444058c-125c-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 03:43:52 +0200 (CEST)
Received: from SA9PR13CA0060.namprd13.prod.outlook.com (2603:10b6:806:22::35)
 by CY5PR12MB6273.namprd12.prod.outlook.com (2603:10b6:930:22::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 01:43:45 +0000
Received: from SA2PEPF00003AE8.namprd02.prod.outlook.com
 (2603:10b6:806:22:cafe::a2) by SA9PR13CA0060.outlook.office365.com
 (2603:10b6:806:22::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26 via Frontend
 Transport; Wed, 15 May 2024 01:43:45 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00003AE8.mail.protection.outlook.com (10.167.248.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 01:43:44 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 20:43:44 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 20:43:42 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9444058c-125c-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LFBoMyfxkhFIxIy3S7xIiehYUxmhedabpojIPA/MfYoZdY2UokTfguasUcy89r0F/WFIcLLYPN0WvEEyifpnBDCdpVJB+q94fh1qKznZGBKzjQdqPLdF78WKm2aACsrKGugBq64C5eCfbPaqFX/jWbym0Qgy44m2kjyvtAgXkxoq/gc1Kpc6s/AOcyPAX7KZCgooXbel7jkDDS8DqDUNzmMi6eAAeUxTppkdPnuS6M/uPk23QwmSsIiDJPkmCggAYc4C9HJAnGwdvN3K0WJeNDpde1IZAo91DUo3zM6cIIncAZAIHpPMRS/JeVg3arCdBRGVDO+0vOm40Fq9H1ACiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=I9Mo9dcjzNCPuVSTk1Qb4E60fo4KMmya7iYoi3zTTWY=;
 b=TGLCnzRCB29j5QcYGXVlvVmfhICyUUfz0oOLZmzDUIFaadVL//H0ZeKrOLMnd0AItsfZbi4eaXoG3PS75tW9KpZbCOOXktCwd2s1VcaRy+vkeQSbcRsPglp2JidQOXkfFxzYrTaUbGIB98jwR3TMnSBSzNidZ44cz5xfo15fr5hcU6JG0HIulkVE7owAH0ynsgNCgOKLUwc1yb4K7N1wfzJush36t9Ye4sOkrb1Z2TKQb2ZdwOWsg86SK7rOk7xY1vVb444Z6XIeiSdPuVn9nSbzAnM3QfMmoKuAHeKWIaHX6HLfS7Yzd0QGDqyB8hcqI+fXU66zlLp5AlKPfkaekw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=I9Mo9dcjzNCPuVSTk1Qb4E60fo4KMmya7iYoi3zTTWY=;
 b=e4/5VL1Hg/ZkmMZiWziQL/vAVLpD4gpMWEipjaIc7sTOvOrw49b1D8Z3BtAXAcB25s0g8sqGSJKvTqfofv4HivmkdWnh+4PMEztyKJkzIjigAekdhLC8DYRufURSJ73f3g0y4IoluXgVcs7Z+1AcbOqNal2LBQ3Bev5un394cjE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>, <linux-kernel@vger.kernel.org>
CC: Henry Wang <xin.wang2@amd.com>, Juergen Gross <jgross@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Michal Orzel <michal.orzel@amd.com>
Subject: [PATCH] drivers/xen: Improve the late XenStore init protocol
Date: Wed, 15 May 2024 09:43:30 +0800
Message-ID: <20240515014330.1044617-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00003AE8:EE_|CY5PR12MB6273:EE_
X-MS-Office365-Filtering-Correlation-Id: c3a6b98b-32b5-4f2d-7f64-08dc7480750f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?qaa5iZHwudBr6Y7Q5p2814qTq2Rmlqeenf++Xa2C+u6cCutSNQGZO0HNRaO/?=
 =?us-ascii?Q?kAVe5wbHqXen9F7FoG4Om4fCmqLoC/anR0Dfgw1mcBduElhp1J4mC/n7J2ts?=
 =?us-ascii?Q?q3ZXu9ipOcXDkTgYCpynQDBdbbaZUmGfCfRd0VXn7e8Je6West5rjyOazkP5?=
 =?us-ascii?Q?DFkJLTNVmXV7ls9bpFVYE/DXzYM2r1CK++wsMU287YZCTok393L4DTqdjbGm?=
 =?us-ascii?Q?kUVbpS1zCAKqvKdKv1E4JIdNlxSabIC3JXsymmJoJbQI831MgfYD+QlPPOdj?=
 =?us-ascii?Q?pSOhW18qvFjKNhBPCpyHYBBoxNcAukm/nJBCQyv24g1eKpFIuzCIH5m1929f?=
 =?us-ascii?Q?zh7pVoUiUX+Tx1UGWlocge1vPdeEs2s0kljHyzs7WZlfe5WBe3AEgwRspG0A?=
 =?us-ascii?Q?2lcQ+3yxmvr2C/t/kbuOHcF1zLmx5We24teXZ+0xtAHSO1GmjbdlYtsKEbLB?=
 =?us-ascii?Q?ZETnF9Jm44PhJk9o6UiOsIKr38HsBo5QB+HR4FIuK1cplJIL9SVtrR6vEh8D?=
 =?us-ascii?Q?oKqP0rVLvIah+2O7cnOOVwXpL4VtvacminojTUDHG0uJs+0zAe4QSzWaAPQX?=
 =?us-ascii?Q?KVfR6xD45x3C1CfqXwwfRcTuJiDYUCuSPqURwBhj/97B/SIyl/OzVdzybkIY?=
 =?us-ascii?Q?bN1s4+8iIFNQBxVVmPozaqFBhjDrW38a/2woF3qxoDmlno2YVU+yc4+OPoXC?=
 =?us-ascii?Q?eKVbNOzhY+a5ybeeo2008K3Pb3+x/ZpvZIC9/ShE4Bhycl55G97dJtIRcOB1?=
 =?us-ascii?Q?0Wk32EHdUpk1HUUHOme3fslqV4OgGimNGfY8fbdGwEBsPJDKGesB+qUdENGH?=
 =?us-ascii?Q?VGfzP8HpgO/tvayTuUPLruYp7/xOjMXm1RCWv5CKf5hG08f4YOy+7jRsIQCe?=
 =?us-ascii?Q?4b/rp2nLn0beUhx4xhKSr4my3NFxx2FAhZjBINgO5v+Ypj0I1vV1eQkKdUFV?=
 =?us-ascii?Q?okj+DebYMAO1jOW+s7yWyooabzobJGTe0ggLdUUbbJhUkfW3os3Rd72/WzX/?=
 =?us-ascii?Q?wlTyoVtiF0qnLv/Jeo+wqTbqyErnUjI/dpy2mpZfC9ccSffOlQLcgsS7aoSu?=
 =?us-ascii?Q?B5fMw8cUvgj0DsYlaKnrOVJlOkDo4u0u6YhLRIwkZCISQ2uEO0v2bxTDcVqB?=
 =?us-ascii?Q?4WoXWvc90MybBwWP7y9Y5hhEZtuC9kbkoVzjaFbvu4ea5d7CQkgHjDp20iP2?=
 =?us-ascii?Q?8Pl1acSG3PD1xL3OmMHe5Pl8Ld80abgtdydyZf6jJ4MTT8WD2uSZJPM/Tioi?=
 =?us-ascii?Q?KlbrVOLOAqoBdZHosTfsxc5/Rstk3dNJSykC9cQwvjb8b+IKHIaHf/1gwrRB?=
 =?us-ascii?Q?V3m8ugt1a76aW/ZkDGeTHSXzjxWk3P7LNz8A/0iljLpj2073jmh9aNgmbbmm?=
 =?us-ascii?Q?WHlMaA6QJ4KpgPP8lyOjGN6n76CW?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 01:43:44.7728
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c3a6b98b-32b5-4f2d-7f64-08dc7480750f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00003AE8.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6273

Currently, the late XenStore init protocol is only triggered properly
for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
case that XenStore interface is allocated but not ready (the connection
status is not XENSTORE_CONNECTED), Linux should also wait until the
XenStore is set up properly.

Introduce a macro to describe the XenStore interface is ready, use
it in xenbus_probe_initcall() and xenbus_probe() to select the code
path of doing the late XenStore init protocol or not.

Take the opportunity to enhance the check of the allocated XenStore
interface can be properly mapped, and return error early if the
memremap() fails.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index 3205e5d724c8..8aec0ed1d047 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -72,6 +72,10 @@ EXPORT_SYMBOL_GPL(xen_store_evtchn);
 struct xenstore_domain_interface *xen_store_interface;
 EXPORT_SYMBOL_GPL(xen_store_interface);
 
+#define XS_INTERFACE_READY \
+	((xen_store_interface != NULL) && \
+	 (xen_store_interface->connection == XENSTORE_CONNECTED))
+
 enum xenstore_init xen_store_domain_type;
 EXPORT_SYMBOL_GPL(xen_store_domain_type);
 
@@ -751,9 +755,10 @@ static void xenbus_probe(void)
 {
 	xenstored_ready = 1;
 
-	if (!xen_store_interface) {
-		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
-					       XEN_PAGE_SIZE, MEMREMAP_WB);
+	if (!xen_store_interface || XS_INTERFACE_READY) {
+		if (!xen_store_interface)
+			xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
+						       XEN_PAGE_SIZE, MEMREMAP_WB);
 		/*
 		 * Now it is safe to free the IRQ used for xenstore late
 		 * initialization. No need to unbind: it is about to be
@@ -822,7 +827,7 @@ static int __init xenbus_probe_initcall(void)
 	if (xen_store_domain_type == XS_PV ||
 	    (xen_store_domain_type == XS_HVM &&
 	     !xs_hvm_defer_init_for_callback() &&
-	     xen_store_interface != NULL))
+	     XS_INTERFACE_READY))
 		xenbus_probe();
 
 	/*
@@ -831,7 +836,7 @@ static int __init xenbus_probe_initcall(void)
 	 * started, then probe.  It will be triggered when communication
 	 * starts happening, by waiting on xb_waitq.
 	 */
-	if (xen_store_domain_type == XS_LOCAL || xen_store_interface == NULL) {
+	if (xen_store_domain_type == XS_LOCAL || !XS_INTERFACE_READY) {
 		struct task_struct *probe_task;
 
 		probe_task = kthread_run(xenbus_probe_thread, NULL,
@@ -1014,6 +1019,12 @@ static int __init xenbus_init(void)
 			xen_store_interface =
 				memremap(xen_store_gfn << XEN_PAGE_SHIFT,
 					 XEN_PAGE_SIZE, MEMREMAP_WB);
+			if (!xen_store_interface) {
+				pr_err("%s: cannot map HVM_PARAM_STORE_PFN=%llx\n",
+				       __func__, v);
+				err = -ENOMEM;
+				goto out_error;
+			}
 			if (xen_store_interface->connection != XENSTORE_CONNECTED)
 				wait = true;
 		}
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 01:54:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 01:54:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721873.1125556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s73ql-0004X0-Aa; Wed, 15 May 2024 01:54:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721873.1125556; Wed, 15 May 2024 01:54:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s73ql-0004Wt-7S; Wed, 15 May 2024 01:54:35 +0000
Received: by outflank-mailman (input) for mailman id 721873;
 Wed, 15 May 2024 01:54:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s73qk-0004Wj-5o; Wed, 15 May 2024 01:54:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s73qj-0003wb-UB; Wed, 15 May 2024 01:54:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s73qj-0004AC-Lm; Wed, 15 May 2024 01:54:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s73qj-0001Vl-L8; Wed, 15 May 2024 01:54:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=AQVpqeTR0cFbgsJtJ2Gyt02Vg9ct6TlCrmkLu6CANWU=; b=ua/a5H+PbRqm+pLqrqvlM+XlQ5
	FBt5bCWbnxwYsAD3SOxJxl5gE4R3ZcuhSdme2ijkqLbu9uD+23qeLlaI8s5a6Ze9uH6hgjySoZECT
	B8WVW1KAhOpVIeWCPkeVS2T2T35wBuq53wwvw7i6Ia6P/EA4ffxgzzY9IKd7l3Cx1dzU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185999-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 185999: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=319a5125ca2649e6eb95670b4d721260025c187d
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 01:54:33 +0000

flight 185999 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185999/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  319a5125ca2649e6eb95670b4d721260025c187d
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   185976  2024-05-11 02:00:22 Z    3 days
Testing same since   185999  2024-05-14 23:02:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   46aa3031ae..319a5125ca  319a5125ca2649e6eb95670b4d721260025c187d -> smoke


From xen-devel-bounces@lists.xenproject.org Wed May 15 02:12:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 02:12:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721880.1125565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s748K-00080q-Ov; Wed, 15 May 2024 02:12:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721880.1125565; Wed, 15 May 2024 02:12:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s748K-00080j-MC; Wed, 15 May 2024 02:12:44 +0000
Received: by outflank-mailman (input) for mailman id 721880;
 Wed, 15 May 2024 02:12:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s748J-00080Z-Gj; Wed, 15 May 2024 02:12:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s748J-0004lO-DW; Wed, 15 May 2024 02:12:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s748J-0004au-4W; Wed, 15 May 2024 02:12:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s748J-0008R9-40; Wed, 15 May 2024 02:12:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=A8byFbvkfmO0olRQvq0rttrgby+ZLN0W6xVLmpqTpsQ=; b=RKGDJdF853lV3sfvplZeiO1HEe
	Uhbm9uoRHnPo3hiZQf2ovqzOUg1fiYm7zRxdkYXKFcnI6DMbjQkRkyoXTJxci2JpSQnka7FMnb1mU
	aJWuuuomB1Qv01t3eVfzqQsZ+WAM1Wm/6Kx8xV4SGQ/7VAFbiBpzMKZvtUax4HEnUV8g=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-185998-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 185998: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-raw:<job status>:broken:regression
    linux-linus:test-armhf-armhf-xl-raw:host-install(5):broken:regression
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start.2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b95dc87362aa57bdd0dcbad109ca5e5ef3cbb6c
X-Osstest-Versions-That:
    linux=a5131c3fdf2608f1c15f3809e201cf540eb28489
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 02:12:43 +0000

flight 185998 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/185998/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-raw         <job status>                 broken
 test-armhf-armhf-xl-raw       5 host-install(5)        broken REGR. vs. 185996
 test-armhf-armhf-xl           8 xen-boot                 fail REGR. vs. 185996
 test-armhf-armhf-libvirt      8 xen-boot                 fail REGR. vs. 185996

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-freebsd11-amd64 22 guest-start.2 fail blocked in 185996
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185996
 test-armhf-armhf-libvirt-vhd  8 xen-boot                     fail  like 185996
 test-armhf-armhf-examine      8 reboot                       fail  like 185996
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 185996
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185996
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4b95dc87362aa57bdd0dcbad109ca5e5ef3cbb6c
baseline version:
 linux                a5131c3fdf2608f1c15f3809e201cf540eb28489

Last test of basis   185996  2024-05-14 09:18:32 Z    0 days
Testing same since   185998  2024-05-14 19:41:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adrian Hunter <adrian.hunter@intel.com>
  Alice Ryhl <aliceryhl@google.com>
  Amer Al Shanawany <amer.shanawany@gmail.com>
  Andrea della Porta <andrea.porta@suse.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Anna-Maria Behnsen <anna-maria@linutronix.de>
  Antonio Borneo <antonio.borneo@foss.st.com>
  Anup Patel <apatel@ventanamicro.com>
  Arjan van de Ven <arjan@linux.intel.com>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Artur Rojek <contact@artur-rojek.eu>
  Baoqi Zhang <zhangbaoqi@loongson.cn>
  Besar Wicaksono <bwicaksono@nvidia.com>
  Biao Dong <dongbiao@loongson.cn>
  Biju Das <biju.das.jz@bp.renesas.com>
  Bitao Hu <yaoma@linux.alibaba.com>
  Björn Töpel <bjorn@rivosinc.com>
  Boqun Feng <boqun.feng@gmail.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Brennan Xavier McManus <bxmcmanus@gmail.com>
  chenqiwu <qiwu.chen@transsion.com>
  chenqiwu <qiwuchen55@gmail.com>
  Christian Brauner <brauner@kernel.org>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Daniel J Blueman <daniel@quora.org>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Gow <davidgow@google.com>
  David Stevens <stevensd@chromium.org>
  David Woodhouse <dwmw@amazon.co.uk>
  Dawei Li <dawei.li@shingroup.cn>
  Dongli Zhang <dongli.zhang@oracle.com>
  Dr. David Alan Gilbert <linux@treblig.org>
  Edward Liaw <edliaw@google.com>
  Eric Chanudet <echanude@redhat.com>
  Fabrice Gasnier <fabrice.gasnier@foss.st.com>
  Finn Thain <fthain@linux-m68k.org>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gavin Shan <gshan@redhat.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Geert Uytterhoeven <geert@linux-m68k.org>
  George Guo <guodongtai@kylinos.cn>
  Guanrui Huang <guanrui.huang@linux.alibaba.com>
  Guenter Roeck <linux@roeck-us.net>
  Haifeng Xu <haifeng.xu@shopee.com>
  Hao Chen <chenhao418@huawei.com>
  Ingo Molnar <mingo@kernel.org>
  Itaru Kitayama <itaru.kitayama@fujitsu.com>
  Ivan Orlov <ivan.orlov0322@gmail.com>
  Jacob Pan <jacob.jun.pan@linux.intel.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
  Jinjie Ruan <ruanjinjie@huawei.com>
  Jiri Slaby (SUSE) <jirislaby@kernel.org>
  Joel Granados <j.granados@samsung.com>
  John Hubbard <jhubbard@nvidia.com>
  John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Junhao He <hejunhao3@huawei.com>
  Keguang Zhang <keguang.zhang@gmail.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kunwu Chan <chentao@kylinos.cn>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Li Zhijian <lizhijian@fujitsu.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Lu Dai <dai.lu@exordes.com>
  Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
  Marc Zyngier <maz@kernel.org>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Michael Schmitz <schmitzmic@gmail.com>
  Mickaël Salaün <mic@digikod.net>
  Miguel Ojeda <ojeda@kernel.org>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nathan Chancellor <nathan@kernel.org>
  Nikhil Rao <nikhil.rao@intel.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Paul Cercueil <paul@crapouillou.net>
  Paul E. McKenney <paulmck@kernel.org>
  Peter Xu <peterx@redhat.com>
  Qiuxu Zhuo <qiuxu.zhuo@intel.com>
  Rae Moar <rmoar@google.com>
  Rafael J. Wysocki <rafael@kernel.org>
  Randy Dunlap <rdunlap@infradead.org>
  Reinette Chatre <reinette.chatre@intel.com>
  Rob Landley <rob@landley.net>
  Robin Murphy <robin.murphy@arm.com>
  Rodrigo Campos <rodrigo@sdfg.com.ar>
  Ryan Roberts <ryan.roberts@arm.com>
  Scott Mayhew <smayhew@redhat.com>
  Sean Christopherson <seanjc@google.com>
  Shiqi Liu <shiqiliu@hust.edu.cn>
  Shuah Khan <skhan@linuxfoundation.org>
  Simon Glass <sjg@chromium.org>
  Srikanth Aithal <sraithal@amd.com>
  Stafford Horne <shorne@gmail.com>
  Stefan Wahren <wahrenst@gmx.net>
  Stephen Boyd <swboyd@chromium.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Sudeep Holla <sudeep.holla@arm.com>
  Suzuki K Poulose <suzuki.poulose@arm.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Blum <thorsten.blum@toblux.com>
  Tianyang Zhang <zhangtianyang@loongson.cn>
  Tiezhu Yang <yangtiezhu@loongson.cn>
  Tom Lendacky <thomas.lendacky@amd.com>
  Tony Luck <tony.luck@intel.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Valentin Obst <kernel@valentinobst.de>
  Viresh Kumar <viresh.kumar@linaro.org>
  Wander Lairson Costa <wander@redhat.com>
  Will Deacon <will@kernel.org>
  Willy Tarreau <w@1wt.eu>
  xieming <xieming@kylinos.cn>
  Yicong Yang <yangyicong@hisilicon.com>
  Yo-Jung (Leo) Lin <0xff07@gmail.com>
  Yury Norov <yury.norov@gmail.com>
  Zenghui Yu <yuzenghui@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      broken  
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-xl-raw broken
broken-step test-armhf-armhf-xl-raw host-install(5)

Not pushing.

(No revision log; it would be 6941 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed May 15 03:13:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 03:13:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721892.1125576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s754n-0007Tz-2y; Wed, 15 May 2024 03:13:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721892.1125576; Wed, 15 May 2024 03:13:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s754m-0007Ts-WA; Wed, 15 May 2024 03:13:08 +0000
Received: by outflank-mailman (input) for mailman id 721892;
 Wed, 15 May 2024 03:13:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3gy8=MS=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s754m-0007Tm-6H
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 03:13:08 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0b4b57cd-1269-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 05:13:05 +0200 (CEST)
Received: from BY3PR05CA0054.namprd05.prod.outlook.com (2603:10b6:a03:39b::29)
 by IA1PR12MB6089.namprd12.prod.outlook.com (2603:10b6:208:3ef::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 03:13:01 +0000
Received: from CO1PEPF000066EA.namprd05.prod.outlook.com
 (2603:10b6:a03:39b:cafe::5b) by BY3PR05CA0054.outlook.office365.com
 (2603:10b6:a03:39b::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26 via Frontend
 Transport; Wed, 15 May 2024 03:13:01 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000066EA.mail.protection.outlook.com (10.167.249.5) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 03:13:01 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 14 May
 2024 22:13:00 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 14 May 2024 22:12:59 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b4b57cd-1269-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HimJzIvFx+IWD8/WbgkKOvdMeIZVjNDq6HJALyxZJdegn1O6qH7PDxE27JRK48fxwOSi6xiyUrz6Pl6/qCTqQh32oM+5n7JIpvMuz0yJQ10eDlvjYAC3GlRrVi7e9ydPRRuvK5gG62soTFuZm12Evp+3T/QsCyVzoUWtUYj/HGOw0QtSKfjMo4BT4J43Cjsd8mK3YZBT9PaX7+kPUkvdMaNT8V7q0UZ3RKS+fU7TstY05YzUycRcFznap9/tceNxfJAyv/jKm3P5lUARJWRPLeyyc8ihwu/RPw7sPB5MEhHpB7iYnYWov1FUjrZna9v2eR0D1+1UidU8ibhgibpJfA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EU9AFCOBovKvDEZHx/Rj/uksJtoh/0JsrUr2my3g+fc=;
 b=JQJsJGE6FTNFO2FkyQrjYfTxufxVT/vZpVFmxV9S8BuSHrOsQ8RnrlKiOaoKzoBL0iAu8u5KrlXdQnUPCgaeOdIJxJeraqlfh0riOR79Kmn7oqya1VRjy5sgaCxBXCl7HtdvlBuRzFfYdcDM8ldkYZLitunxGDXbqPqgZU/pMGkM+hczBD+Ua4pb2Vx34Y5Sp97GqgXu5UU+DvOU5icuFtfocjurKSPHO92G44+tHaITRFAMOczYZKeRZT5M0cYHiCHagoz0Yb+3dUPpd2AsCW1P+0ZqTbwyr5VzmPtVsfDZhkvo0Y6yNFw9EgBPUa7va707eGC4+b+TxjyinTo8rw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=gmail.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EU9AFCOBovKvDEZHx/Rj/uksJtoh/0JsrUr2my3g+fc=;
 b=uzuiHsslRqy5njkw+Wf84pkoebtVPsq1PY7IFdRcHU79SXfSsA9vec1O+I2KEwsQFf/7zU1xkVVsXmqDP4kbg54HK63hw1khuxHoAImKcOyyMPArDxOuzE6URpWESAQYs9M4BXca/Vq7lkzmnB/i1CzrLQ5vZijxu78CA9kBCbU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <32f16048-c1ba-4ebb-a752-a77ed529a99c@amd.com>
Date: Wed, 15 May 2024 11:12:58 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Proposal to Extend Feature Freeze Deadline
To: Oleksii K. <oleksii.kurochko@gmail.com>, <xen-devel@lists.xenproject.org>
CC: <committers@xenproject.org>, <kelly.choi@cloud.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>
References: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
 <bba91b8b-b1cc-4054-96e8-80a63a3b808a@citrix.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <bba91b8b-b1cc-4054-96e8-80a63a3b808a@citrix.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EA:EE_|IA1PR12MB6089:EE_
X-MS-Office365-Filtering-Correlation-Id: 1625f1b1-6503-4b49-9a6e-08dc748cede0
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017|3613699003;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?T0pjUlNFUTlvOWt5ZW1HSUR5S0hybjg0Q1F3OHo2SGh1SEZ3bjB5SkI0eVVX?=
 =?utf-8?B?eEtzQnFZTDVhNUhIVU55QVlUb0Y3ZTBTcnR4MHorZ1hSUndodk9CZjZrK3pP?=
 =?utf-8?B?QndMRHo2cXRjeHo3RTVSa0QvaGlWNStBbUtCL21Xa09XOGlqaGxQNEp0djdi?=
 =?utf-8?B?MVZsODZ0SlArUTNYcUxib2hscTl1Wms5V2M2VXc2aHRORGtUaG1zanhIVzVs?=
 =?utf-8?B?Q29pZGg1RmxrekRob21oQjA5UDN1V0tEdDlVMDBQM2Yra1ZpUmpzdnlpWmY3?=
 =?utf-8?B?NExHMEttRE5BQlBuS2RuUGxqMWNZTnAxb2UyZXpRT2ZSM0JXMTFFMHczTXB0?=
 =?utf-8?B?WGhqOGZoUmNidktkV21mMFovVnZ2Z2xsditObmFwV3l4UGoxTkg5N1JuWGVt?=
 =?utf-8?B?eWx4WFFNVFhIdHFlR2VjVmxMemRLSkkraHlhMGFKdWF5QUQ1MWlZRndWVmxK?=
 =?utf-8?B?VTY4aE1FNFVNWUo5cVI3eUxQa3MvOEY3SFhJVEY2ODdRWnJVSXBHRXlncFFU?=
 =?utf-8?B?SUZHVnlRL2VTUWVhMFJuWlZVZmVEcjNrODFlMVZRamEyVm5oNmlNbjh4Um13?=
 =?utf-8?B?WVBPNFYvTGJUT0RuTDN3TmtBd2ordTc5QVAzeTI0ZWthY1ZudDZkaEpialJP?=
 =?utf-8?B?b3Uwd2hIVFhhd24rS2s0Z0FZMHJDMUc1UVdTUUEzMjUvRVRJa2JWMFJZUHVl?=
 =?utf-8?B?MFpkVkpQaCtrNHBOVmFDREpocXRaNGpPSjh0N2x0TDhVeU1CbXRDZWMzWkh2?=
 =?utf-8?B?RTBLNU9ZbVZUVGgyVS9KQkUwek1yYUJScGs1OElTYWt2alhpdmlZdk1OOWlG?=
 =?utf-8?B?ZkxSV2NuK05Ka2hwa3hQcVNJQjJxTzVNK1NhcXlkcmVGa056bWZMeWNVc2wy?=
 =?utf-8?B?em13YVBpcTN6Zm1FM0FuTEZUa0pLUEI2NHpKR3NHZEVqRlNFOC9ia2xleGVG?=
 =?utf-8?B?ZCs4V3cxbnFvWU9QVFVmajViQndxdlNmY3ZmWEJUNW5GbmorM1ZHVzY3SEx1?=
 =?utf-8?B?V1JYWndHdytoK3dzSVFENER1NytwZkdiZnVud2FvZTZHUk1aNTQ0WEhqWmI5?=
 =?utf-8?B?TWxSV2JSZWVZUThCenJVclF3MUtGeVlrSHdWeFZoajQ1bW8rbk5HQzBVM3dH?=
 =?utf-8?B?UW8xRkFib3RDRDBVUGdiSXlGUWFHckFBbFVzNXFjMXRIUnVXVnJYMlcralBE?=
 =?utf-8?B?Uk5DNmRKTGtCbStYYXoxRDZBMjR5bzdRdm1kMjB3TjdSTzhKZHJUa25mN3Ji?=
 =?utf-8?B?Zk5rZy95OWk4YmpMMy9MMitmdnhVZUVWR1lpN3Q3T3VWaVQxamNWTzBDTVNM?=
 =?utf-8?B?TVE2L0tTMDRJYkkwR0FTTDFzcDlGMThVZU5NaFg0cWJFeDlsOHJPT1B6Y2Mw?=
 =?utf-8?B?S05qUFluWDNUdXJNYllETXNaSWZOd3paUm5TSUdJUlRmZFNTUE1EeVRSeWxa?=
 =?utf-8?B?QmZkWURkQUVqYUtwWDdDL1hBMjUzT1FYL05ZbmpoRUNuLy9DN1N3cUxLM29Y?=
 =?utf-8?B?eW1iYVZVTWpBd0RzQmwxV3VOVFUrL1BUU3V4TDdsZnNQZjlkREkrNGYwVnlu?=
 =?utf-8?B?L20zclNkeTl3ZC84bWRya2oyVDExTmlXQXBQMit0ZHE4c3d4QVl2NVlzOFl6?=
 =?utf-8?B?ZUlXUFlQWndrc2NWQkJMQ3NyQ1R2dzg2TFp6SmhOc3UvOWV5b0xZSEtCaDZH?=
 =?utf-8?B?ODdHRm9yTjA4dXJKb2cxblFzcXE0SmQ0dnVBWVpzSUhyUXRqMEx1eU9KMFl4?=
 =?utf-8?B?a1ovWW5MM1h2R0REUTdIUVlDRFNCTzh3YTRTN0hCNDYzMHBjRE82b2lwZjND?=
 =?utf-8?Q?17vkkxpcu+NrXsXkMug9oFuj6CMFsEyUhKRic=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017)(3613699003);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 03:13:01.3524
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1625f1b1-6503-4b49-9a6e-08dc748cede0
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EA.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6089

Hi Oleksii,

On 5/14/2024 11:43 PM, Andrew Cooper wrote:
> On 14/05/2024 4:40 pm, Oleksii K. wrote:
>> Hello everyone,
>>
>> We're observing fewer merged patches/series across several
>> architectures for the current 4.19 release in comparison to previous
>> release.
>>
>> For example:
>> 1. For Arm, significant features like Cache Coloring and PCI
>> Passthrough won't be fully merged. Thus, it would be beneficial to
>> commit at least the following two patch series:
>> [1]https://lore.kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@amd.com/
>>    
>> [2]https://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.com/
>>
>> 2. For RISC-V, having the following patch series [3], mostly reviewed
>> with only one blocker [4], would be advantageous (As far as I know,
>> Andrew is planning to update his patch series):
>> [3]https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@gmail.com/
>> [4]
>> https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/
>>
>> 3. For PPC, it would be beneficial to have [5] merged:
>> [5]
>> https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptorengineering.com/
>>
>> Extending the feature freeze deadline by one week, until May 24th,
>> would provide additional time for merges mentioned above. This, in
>> turn, would create space for more features and fixes for x86 and other
>> common elements. If we agree to extend the feature freeze deadline,
>> please feel free to outline what you would like to see in the 4.19
>> release. This will make it easier to track our final goals and
>> determine if they are realistically achievable.
>>
>> I'd like to open the floor for discussion on this proposal. Does it
>> make sense, and would it be useful?
> Considering how many people are blocked on me, I'd welcome a little bit
> longer to get the outstanding series/fixes to land.

It would be great if we can extend the deadline for a week, thank you! I 
will try my best to make progress of the two above-mentioned Arm series.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Wed May 15 03:27:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 03:27:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721898.1125585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s75IC-0001Qp-AR; Wed, 15 May 2024 03:27:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721898.1125585; Wed, 15 May 2024 03:27:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s75IC-0001Qi-7u; Wed, 15 May 2024 03:27:00 +0000
Received: by outflank-mailman (input) for mailman id 721898;
 Wed, 15 May 2024 03:26:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s75IB-0001QY-1G; Wed, 15 May 2024 03:26:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s75IA-000633-RV; Wed, 15 May 2024 03:26:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s75IA-0006Td-Fq; Wed, 15 May 2024 03:26:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s75IA-0002sI-FO; Wed, 15 May 2024 03:26:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XpGbq7tPSN1Mjg+MS0GYBEIYmauVTzcN+N2R7NKhz4E=; b=HkTuuDE7IVwSwdsRHDFYUAzaGd
	aHZBDrtxgV4tiKJiDnKkiF5n+OvOygGzZFmUyb0BVuJc3Jt+ga17bnDMrUukKIWCGxedJYm1sit9J
	E+VaA+CtrzLSnBKZmawVoNMz0f475RpAhBmA6oLNPPTdwUMxgWXT/r+OU5RQpqKI/mxI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186000-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186000: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=558a25366d66e9ff96784ed3aab476f808531bf3
X-Osstest-Versions-That:
    ovmf=4b6ee06a090d956f80b4a92fb9bf03098a372f39
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 03:26:58 +0000

flight 186000 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186000/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 558a25366d66e9ff96784ed3aab476f808531bf3
baseline version:
 ovmf                 4b6ee06a090d956f80b4a92fb9bf03098a372f39

Last test of basis   185967  2024-05-10 04:11:12 Z    4 days
Testing same since   186000  2024-05-15 01:41:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Shun Cheng Liu <adam.liu@tw.synaptics.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   4b6ee06a09..558a25366d  558a25366d66e9ff96784ed3aab476f808531bf3 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 15 06:21:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:21:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721915.1125596 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s780s-0006jB-9D; Wed, 15 May 2024 06:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721915.1125596; Wed, 15 May 2024 06:21:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s780s-0006j4-6Z; Wed, 15 May 2024 06:21:18 +0000
Received: by outflank-mailman (input) for mailman id 721915;
 Wed, 15 May 2024 06:21:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qki8=MS=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1s780q-0006iy-SE
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:21:16 +0000
Received: from mail-oa1-x2d.google.com (mail-oa1-x2d.google.com
 [2001:4860:4864:20::2d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5476c844-1283-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 08:21:14 +0200 (CEST)
Received: by mail-oa1-x2d.google.com with SMTP id
 586e51a60fabf-23f5a31d948so4201000fac.0
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 23:21:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5476c844-1283-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1715754073; x=1716358873; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Oaa85CsLwpAmYlsk+PQjdULpAI6iM+Mw19xv5yP3TZ4=;
        b=HVAXX/c3jIJTwmskp2oT2dylPplwFTih1Qxnw1y3kgdytf90mu84Yl2bxjmzwmSNG3
         ay3mkGgm92Zh7k35e3EII4EIkCj3MJxJA0KGAWYkx2pQAs+MGCGYDz5q3A8xgG2nCC02
         h2wOrCvcoQdRRbwAxEzvqnT3PvaA1inNhiPqnwMmhC6MkQCH3t2mDZzyjB0kaC4BsPkW
         9Gd6HQs4B0xiwbWndqDm4RXzj20VPTdFnB4B4RMvSrJYFa4Hm39R5Hwm/PjX2gDwpuzc
         caHGxETmjFCY39gvsOWzIUnd5gVDfi6E48kGJ9TpRdSMsjWbejMCJkyvD6XvDWHHU2NV
         xORg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715754073; x=1716358873;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Oaa85CsLwpAmYlsk+PQjdULpAI6iM+Mw19xv5yP3TZ4=;
        b=R5lrxgu0ssAHKqXgoe5G505jElAj0NpJhlJ3zgFfnOAWgFu/yCCZ3eb5f/pS8AlZzl
         k2fijIPKP0yN+5uogP0LuxOhsV/YlNf3xSTx1ixIKAO6F8QhMSJOOYNtrF/PWjzyK0mY
         p3tt7Zfu4U5Gevx2qXYZKQpXq2RuYsC6M8ugwEVKmX2im64RjNoz/BgpEZwTxpIHPBD0
         B8APqixhJxTAxtbNjrznN3tNQj9y/vxWRZcJTdrhzR3U9uqz1YjSrfvCl4utUZHRQGNu
         XHrpYvFLxguH/lAXh/mcLC/8JYAWjBr5n2u0ZNdwMlO/6GM/CvQUNnJmpHGs0a0N6f/j
         az1Q==
X-Gm-Message-State: AOJu0YzZQKaJx9OImwYAi0z2WX+fRNSsu3MNRZoMIbB2SH3bBUW9oGj4
	771gLKvQ0rsM9TNANQ1ldmoGaJZBMRM+xm+6PGiZKoMxun+bS0Zt/4WHXDhwbnTmDOoiVrNf7Gh
	+5fFxwYBrua5euqyOOGF4bcrCd7fLBUwDAH8QXw==
X-Google-Smtp-Source: AGHT+IFiKrS1Xh2ZFX6UaRD5rM/ntwFDS8tiiWJ7TW03KQVeC++TdNw6jogEh4TeE1y5k8BYm9NF/Pb9Hyd7B0aMu20=
X-Received: by 2002:a05:6871:514:b0:23c:4a71:4fc4 with SMTP id
 586e51a60fabf-24172e41cd9mr17519846fac.39.1715754073220; Tue, 14 May 2024
 23:21:13 -0700 (PDT)
MIME-Version: 1.0
References: <CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com>
 <b965ee57-c6fc-459f-a5fd-fae47dc6ea9d@suse.com> <CAHUa44FsFi0F4tz3jN+d3WkR4dTPJ1HdUru+ME1YQyzMSbMG7Q@mail.gmail.com>
 <2520da93-8b84-4a09-b944-fa065db03c18@suse.com>
In-Reply-To: <2520da93-8b84-4a09-b944-fa065db03c18@suse.com>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Wed, 15 May 2024 08:21:00 +0200
Message-ID: <CAHUa44F-e87A=4F84dSw8nYViEyOyOE-7n8Nm=L0PHWaAy4U8A@mail.gmail.com>
Subject: Re: Referencing domain struct from interrupt handler
To: Jan Beulich <jbeulich@suse.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	George Dunlap <george.dunlap@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 14, 2024 at 9:12=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 08.05.2024 09:10, Jens Wiklander wrote:
> > On Fri, May 3, 2024 at 12:32=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >> Furthermore, is it guaranteed that the IRQ handler won't interrupt cod=
e
> >> fiddling with the domain list? I don't think it is, since
> >> domlist_update_lock isn't acquired in an IRQ-safe manner. Looks like
> >> you need to defer the operation on the domain until softirq or tasklet
> >> context.
> >
> > Thanks for the suggestion, I'm testing it as:
> > static DECLARE_TASKLET(notif_sri_tasklet, notif_sri_action, NULL);
> >
> > static void notif_irq_handler(int irq, void *data)
> > {
> >     tasklet_schedule(&notif_sri_tasklet);
> > }
> >
> > Where notif_sri_action() does what notif_irq_handler() did before
> > (using rcu_lock_domain_by_id()).
> >
> > I have one more question regarding this.
> >
> > Even with the RCU lock if I understand it correctly, it's possible for
> > domain_kill() to tear down the domain. Or as Julien explained it in
> > another thread [3]:
> >> CPU0: ffa_get_domain_by_vm_id() (return the domain as it is alive)
> >>
> >> CPU1: call domain_kill()
> >> CPU1: teardown is called, free d->arch.tee (the pointer is not set to =
NULL)
> >>
> >> d->arch.tee is now a dangling pointer
> >>
> >> CPU0: access d->arch.tee
> >>
> >> This implies you may need to gain a global lock (I don't have a better
> >> idea so far) to protect the IRQ handler against domains teardown.
> >
> > I'm trying to address that (now in a tasklet) with:
> >     /*
> >      * domain_kill() calls ffa_domain_teardown() which will free
> >      * d->arch.tee, but not set it to NULL. This can happen while holdi=
ng
> >      * the RCU lock.
> >      *
> >      * domain_lock() will stop rspin_barrier() in domain_kill(), unless
> >      * we're already past rspin_barrier(), but then will d->is_dying be
> >      * non-zero.
> >      */
> >     domain_lock(d);
> >     if ( !d->is_dying )
> >     {
> >         struct ffa_ctx *ctx =3D d->arch.tee;
> >
> >         ACCESS_ONCE(ctx->notif.secure_pending) =3D true;
> >     }
> >     domain_unlock(d);
> >
> > It seems to work, but I'm worried I'm missing something or abusing
> > domain_lock().
>
> Well. Yes, this is one way of dealing with the issue. Yet as you suspect =
it
> feels like an abuse of domain_lock(); that function would better be avoid=
ed
> whenever possible. (It had some very unhelpful uses long ago.)
>
> Another approach would generally be to do respective cleanup not from
> underneath domain_kill(), but complete_domain_destroy(). It's not really
> clear to me which of the two approaches is better in this case.

Thanks for the feedback. I tried moving the freeing of d->arch.tee to
complete_domain_destroy() while keeping the rest of the cleanup as is.
It works as expected, I'll use this for the next version of the patch
set.

Cheers,
Jens


From xen-devel-bounces@lists.xenproject.org Wed May 15 06:24:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:24:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721918.1125606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s783v-0007Hi-NI; Wed, 15 May 2024 06:24:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721918.1125606; Wed, 15 May 2024 06:24:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s783v-0007Hb-KX; Wed, 15 May 2024 06:24:27 +0000
Received: by outflank-mailman (input) for mailman id 721918;
 Wed, 15 May 2024 06:24:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s783u-0007HT-3i
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:24:26 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c53ee473-1283-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 08:24:23 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a5cce2ce6so100670366b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 23:24:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d1fcsm806891466b.35.2024.05.14.23.24.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 23:24:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c53ee473-1283-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715754263; x=1716359063; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aPkD0J+zCGO8fA4AkyUm6JIhM5BJ112JzDVD+5OrljA=;
        b=FTZBusEcqOLQAyoBnQnGPS7E2KQxYnP9k8EQJkI0kmidYgACnLQxbWEamg6lTA581o
         q/AnY+7nlQqQxw9H7yu/vmqD/7nJEGV5Y+/oGynLJRUp+xiCuTKCanwKs0tGE6yXmnnT
         6XZdkYxnYQT0FgD4h6cXl3GC16gymjZir4ctXrUPmWdlv05/mc7+VubXHkpmnGK/ZMK5
         wxAUShgvJ2qLLcafxqjNU0rRCIuDJE/OXm/2dXhAOdOaKkGKh2G2P2DcmrWHmk6/RG3W
         RDlBclKUU0b/IzlEPgy7e5EJqEj2ZK8CG6K7m9y14SYt0tr92Auu5HhfW+/RR5+67IcP
         8m3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715754263; x=1716359063;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aPkD0J+zCGO8fA4AkyUm6JIhM5BJ112JzDVD+5OrljA=;
        b=P5KWhYkD0Ok9IzK3qMNzB/w56JB0lWWOTTWgddaVzXEqr3FSiZh5zK5QZiKT64A6Nf
         rS2gTfNSaRbv9EepPzJujjAE/wj+OgBpGSYpvuu3FcLZlDhMeHj6bv2/TrcZzDiZm0LV
         lH3gwAFrGAGsqBOdNi7lyFULECKmfmMQO8L20KR+ndgwaSGRbKFm8On+YiKwonE9DHRc
         s5hk2p2YD235DI1StN81sUCn6vETex1XSQshFr5hC4ai5SWeFrw+QGnIqPDy8r4PugVK
         Wk79fqgOShwsqAJ7Hm1fccd7gQ0S6khid1iKy5uyOC1H0+BLj1fYBB1J9w144MOGv9V+
         xXWg==
X-Forwarded-Encrypted: i=1; AJvYcCWdV+iBkN4ezICJWsNm1oPXVbqjQiNe4laojErA5VZehoKDnYAZWhfbzJzfmMI40o/+NmYERHHQNjZM4V7cZibRxXrvBD2AapriALl4f1c=
X-Gm-Message-State: AOJu0YyyxIBOmZIDveLcEwhNTx47zHIy+qM6T3lm8SG1IDbpMSsh6kAf
	G0wCsapcSXvovStUg9B1OJNimHjR/BgSlhRZWNB+aPcsS3uy2Q9XHlVyGqUFuw==
X-Google-Smtp-Source: AGHT+IG0NqCFvafMi9ru/mwYoYrUvvFeEPP2+gEKS8HlPdFsqS7tZ9t5q/yNuc+X/o/9qZ9lTtIsag==
X-Received: by 2002:a17:906:ad97:b0:a59:c833:d275 with SMTP id a640c23a62f3a-a5a2d58a4e7mr937037366b.30.1715754262952;
        Tue, 14 May 2024 23:24:22 -0700 (PDT)
Message-ID: <a3fa7be4-48ce-4b68-a8f7-f94df58d97de@suse.com>
Date: Wed, 15 May 2024 08:24:21 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] lib/strtoul: fix MISRA R10.2 violation
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org, nicola.vetrini@bugseng.com,
 consulting@bugseng.com
References: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop>
 <16990192-aceb-408f-9247-45f3b1f58e0a@suse.com>
 <alpine.DEB.2.22.394.2405141540140.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405141540140.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 00:52, Stefano Stabellini wrote:
> On Tue, 14 May 2024, Jan Beulich wrote:
>> On 14.05.2024 02:32, Stefano Stabellini wrote:
>>> Fix last violation of R10.2 by casting the result of toupper to plain
>>> char. Note that we don't want to change toupper itself as it is a legacy
>>> interface and it would cause more issues.
>>
>> Can you point me at a single example where a new issue would arise? All
>> places I've spotted (including tolower() uses) would appear to benefit
>> from changing toupper() / tolower() themselves. Further, since they are
>> both wrapper macros only anyway, if any concern remained, fiddling with
>> the wrapper macros while leaving alone the underlying inline functions
>> would allow any such use site to simply be switched to using the inline
>> functions directly. As said, from looking at it I don't expect that
>> would be necessary, so instead I'd rather hope that eventually we can
>> do away with the wrapper macros, renaming the inline functions
>> accordingly.
> 
> If we change __toupper to return a plain char, then there are a few
> other things we need to change for consistency, see below. To be honest
> I thought it would cause more problems. I am OK to go with that if you
> all agree. (Nicola please have a look in case this introduces more
> issues elsewhere.)
> 
> 
> diff --git a/xen/include/xen/ctype.h b/xen/include/xen/ctype.h
> index 6dec944a37..6a6854e01c 100644
> --- a/xen/include/xen/ctype.h
> +++ b/xen/include/xen/ctype.h
> @@ -15,9 +15,9 @@
>  #define _X	0x40	/* hex digit */
>  #define _SP	0x80	/* hard space (0x20) */
>  
> -extern const unsigned char _ctype[];
> +extern const char _ctype[];

Why would this be needed? I can't see a connection to toupper() / tolower().

> -#define __ismask(x) (_ctype[(int)(unsigned char)(x)])
> +#define __ismask(x) (_ctype[(int)(char)(x)])

This almost certainly is wrong. Whether plain char is signed or unsigned is
left to the compiler, and it being signed would result in possibly negative
array indexes. Again I can't see a connection to the issue at hand.

> @@ -34,14 +34,14 @@ extern const unsigned char _ctype[];
>  #define isascii(c) (((unsigned char)(c))<=0x7f)
>  #define toascii(c) (((unsigned char)(c))&0x7f)
>  
> -static inline unsigned char __tolower(unsigned char c)
> +static inline char __tolower(char c)
>  {
>  	if (isupper(c))
>  		c -= 'A'-'a';
>  	return c;
>  }
>  
> -static inline unsigned char __toupper(unsigned char c)
> +static inline char __toupper(char c)
>  {
>  	if (islower(c))
>  		c -= 'a'-'A';

This isn't what I had suggested. First I said to leave alone the double-
underscore prefixed functions, and only touch the wrapper macros (as a
precaution in case any use site exists which relies on present behavior).
And then I didn't suggest to alter parameter types; only the return type
would need adjustment, I think, for what you're aiming at:

#define tolower(c) ((char)__tolower(c))
#define toupper(c) ((char)__toupper(c))

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 06:32:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:32:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721923.1125616 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78Bk-0000dL-Es; Wed, 15 May 2024 06:32:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721923.1125616; Wed, 15 May 2024 06:32:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78Bk-0000dE-C5; Wed, 15 May 2024 06:32:32 +0000
Received: by outflank-mailman (input) for mailman id 721923;
 Wed, 15 May 2024 06:32:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s78Bi-0000d8-PX
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:32:30 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e66cd86f-1284-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 08:32:28 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59e4136010so121259366b.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 May 2024 23:32:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17865sm805246766b.204.2024.05.14.23.32.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 May 2024 23:32:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e66cd86f-1284-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715754748; x=1716359548; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dm16VyhRJj1s03ND3XgAo+uplnecyiMpJe/fxvBOSL8=;
        b=H8lMU3FwaGFTBzMEtQ1dYdS/WSENsVvKpoKmmjhSjtSRAW/qIQPeQpr/cNczQK6JcU
         zqlcHomS3YeAPvqBlTcJCl5ri1UpwiFEYito2iSx3beHTXGppy2Dd8cTiArO6rzuzcxe
         omkLyCwKoZ3wj6vAUm1qPn63L4/mE1wkeuGTP6huYPbKito4km0Q38VsgHTiTikYhK59
         +6x9EKTxlRS8nowZLU5UD44pXoc+noUQ6b9I4UcMoot75ml+TPLIxjW+VhyEOK44CBnM
         6dZvc3f3RQoRXxokaAE56OxvvE6aYMYYqsRVxoFNrrKR9DftVh9R9uAqCmTpWWsP6m9q
         mAuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715754748; x=1716359548;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dm16VyhRJj1s03ND3XgAo+uplnecyiMpJe/fxvBOSL8=;
        b=GQJ4cMhl3wzdDbPsua0me/n05I0walCmNr53R095bMmzdaydgPlnXwzVBPhf6VVF7W
         RQHNEF1UcLEmFFZwfpDnqIQ/CqvzX4s7JvkCi708EipUyCfnXKhVObAyxvz/3uJ70eey
         pXSMaoRYqcKoJ+puVj0W7E+0RkPMpfcp8xvjmcdpZPgJULSL24vTIB7r11CjDeeBdhwh
         foaezfcMQadZWGlJaK3kAsjT3qMu+XUIhQKOZewTuUJFps9mSBLeFsW8L3daDYibKW/v
         fjCpgjvcHsHskFBQMGGZ3k66oieN0iiERuskiN358idlDtC725Ay+wENjp5N2HAFhlwi
         EPSg==
X-Forwarded-Encrypted: i=1; AJvYcCVNakQHrf1b8UGSJk+O1YXAD1HAKz7vhKGlgCDfZJye0OWLQBogi6cPXq6PB+FkArDrwEPscTNKQxUTZHgYZ/5calVf0bjS802P0U6SeAo=
X-Gm-Message-State: AOJu0YxIzD9zRid9l9Dv21d6CfQP/pFYgcNH/7vbFfQDPJ45kWqvLt3r
	ko4OmxCXmktsEHplzU35h9uQ9DPkjM/ipKXu2zPKd06GpkC25Y0MDuh6UNciaw==
X-Google-Smtp-Source: AGHT+IHyoFDPCAbCNQM3NAjz/KUu2pFU8s7bTi1wyAatHP2L1dLOukNqDMHl0ha76IuUStLHDCaSiQ==
X-Received: by 2002:a17:906:e2c2:b0:a59:c3dd:db2a with SMTP id a640c23a62f3a-a5a2d53b04cmr973655466b.11.1715754748175;
        Tue, 14 May 2024 23:32:28 -0700 (PDT)
Message-ID: <20335f55-4b16-45e5-a8aa-868006b57ca5@suse.com>
Date: Wed, 15 May 2024 08:32:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 5/5] arm/vpci: honor access size when returning an
 error
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-6-stewart.hildebrand@amd.com>
 <cf646df7-0e92-4138-8b54-471c72f44cb8@xen.org>
 <735c3e3c-d70a-40f5-aca7-f134bc5b98b1@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <735c3e3c-d70a-40f5-aca7-f134bc5b98b1@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 22:31, Stewart Hildebrand wrote:
> Here's what the patch ("arm/vpci: honor access size when returning an
> error") now looks like based on staging:
> 
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 3bc4bb55082a..31e9e1d20751 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>  {
>      struct pci_host_bridge *bridge = p;
>      pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> +    const uint8_t access_size = (1U << info->dabt.size) * 8;

And why exactly uint8_t here, rather than unsigned int? See ./CODING_STYLE.

> +    const uint64_t invalid = GENMASK_ULL(access_size - 1, 0);

I'm not entirely convinced of uint64_t here either, but I'd view this as
more borderline than the uint8_t above. As per ...

> @@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>          return 1;
>      }
>  
> -    *r = ~0ul;
> +    *r = invalid;

... the original rhs here, unsigned long (or perhaps register_t) would seem
more appropriate, but I have no idea whether on Arm32 info->dabt.size can
end up being 3.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 06:50:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:50:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721929.1125636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TH-0003ya-5v; Wed, 15 May 2024 06:50:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721929.1125636; Wed, 15 May 2024 06:50:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TH-0003yT-34; Wed, 15 May 2024 06:50:39 +0000
Received: by outflank-mailman (input) for mailman id 721929;
 Wed, 15 May 2024 06:50:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NNmL=MS=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s78TF-0003jj-V0
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:50:37 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2405::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ec5c393-1287-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 08:50:37 +0200 (CEST)
Received: from SJ0PR05CA0069.namprd05.prod.outlook.com (2603:10b6:a03:332::14)
 by MW4PR12MB7032.namprd12.prod.outlook.com (2603:10b6:303:1e9::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 06:50:33 +0000
Received: from CO1PEPF000044F3.namprd05.prod.outlook.com
 (2603:10b6:a03:332:cafe::c5) by SJ0PR05CA0069.outlook.office365.com
 (2603:10b6:a03:332::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26 via Frontend
 Transport; Wed, 15 May 2024 06:50:32 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000044F3.mail.protection.outlook.com (10.167.241.73) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 06:50:32 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 01:50:28 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ec5c393-1287-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KQoA78y8HFU9ZFsEaRg5bbN31t/SNV528CmgOUxgyhnK4rGOFRcJo24xWF1B42xyoINcCqnl1Q++EEJPj9fej7unkoKL3yzHUJe7YGuTO3HqMYu5rrj0SBb645opa0yM4hlaxlEbA+5JK2WoetCLJBwvgscDj34l2dsEsb+pYB0/cuX9B3kXNrYeIXTdodKyTPxNsj6TPWXhyHcI4CmTDHO47nB6wPJBNspdIXtgFceLhBKHrNALmwdKR0Hqa4h1YRY2+hw+19AFIOPsXPpyQ+L5WCntB9TY+3s3VUxAklM6xqhd053PjSZnBwAYJO44a+gaokoZ1Geu6AAuXTnnHw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=L0DMDL2cmntv+GEqtM+DWFLmEeuaMCIJxvPHhNL1Yxc=;
 b=AghRJXHQRqutfEbZkB+0/jsgsfkbk5vPelaNr83SAJk/jrOryJEHbxPGtMy4Q/Mf2jQXZimFiKLZip07W85LTfLh2DUKimm04bMRAkse0XNHgjgBqVtJ6SPrqQ7Ur/lfinWJIydUltAMaeg/of16vSox7lRbJ9jVIP4UYA6CLlIjZdRtrSenb/kYQGnHHvmplyD0uA4tgncW6M1VL3yDzsHogVkoMC/VWWDR+PP7HO35obXvOgaMLK/7umNkxogUnmAUm6/ycOQk5uQwusAIIVK9Na4uEYJG3rs1kAJf5L2YKFA8WTx2xJp30kGu/UQzlgUDvuyUYeeKQqJy8seGxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L0DMDL2cmntv+GEqtM+DWFLmEeuaMCIJxvPHhNL1Yxc=;
 b=f1Vff4JHSba/TSjjQQogNcw3u08/wTKC7Guim6RtAzUGYKpE7cdiU/5MXV1OfBeeXLV24u0xEXaz46n9z5RDJAQE1QLW3ux4D8k9zei8Rtobp18WbDRvAm9GfeKMdSO1xMSCQEh7v87Z6tSrtplEMMgas6UzxQ1VESFl7rVbeqs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, "Rafael J .
 Wysocki" <rafael@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-acpi@vger.kernel.org>, Huang Rui
	<Ray.Huang@amd.com>, Jiqian Chen <Jiqian.Chen@amd.com>, Huang Rui
	<ray.huang@amd.com>
Subject: [RFC KERNEL PATCH v7 1/2] xen/pvh: Setup gsi for passthrough device
Date: Wed, 15 May 2024 14:50:10 +0800
Message-ID: <20240515065011.13797-2-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515065011.13797-1-Jiqian.Chen@amd.com>
References: <20240515065011.13797-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F3:EE_|MW4PR12MB7032:EE_
X-MS-Office365-Filtering-Correlation-Id: 793a59ad-cbba-492c-754c-08dc74ab50ee
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?BiwMgfZHtnz/lwEJzBjQILySzL9iirr9tnAVxQAwk4Ey0qhwshXV5ZwElOwu?=
 =?us-ascii?Q?o5reCALQIluqmFhi/WgRRv5YiZuhSBAmlWG3bwqPnUbBGzSmFRHJsrN9AXyf?=
 =?us-ascii?Q?NVF8PQSkFc3P6epfhSehVmtaUJK2NdMhx81aXuelTfzi/7qTZS/p2Nsqjaxx?=
 =?us-ascii?Q?94ORvqWuXIUZIfSMZfkFIqhU2khXop8EZ9/DenlGepeaaKftNclT7S6N4VCV?=
 =?us-ascii?Q?oDliIrMgsQZQLTAR31LpTdnq/UmjQiC+bQTTfJd8eW8QDRCCAbOLQZiRAMlf?=
 =?us-ascii?Q?0Ye72GK5fcEKiPhkx8Qpqs8ABKwv7eJQaTZxp7IL2drqjpqpku3O5OJp5BhY?=
 =?us-ascii?Q?JdELPhmVM8CLcLQMUTcu+Ub/3ZCC9CQdrzQFiD+RNORQm3RCAGjhjTZDvoGW?=
 =?us-ascii?Q?9gc/2ZGiaUNGHc8GChSQmbHDF9qSGcc1I/8E0KL5cLEyClpMcJkUcXBoAYbk?=
 =?us-ascii?Q?w0NZyw4aZFNBHfpsDyHc2ARUT607bSpjrh1AQp312QUs6ZlZ8bYikp4Vj5Tc?=
 =?us-ascii?Q?fCuPsTKo+REU/zjbyXfEEsE5yaVyvhLJl4EG01wBrt8o1HXzp7VlhX/Teb83?=
 =?us-ascii?Q?rgwOTL+Nr1j0Z+9tiFnWsqMk5scGPLnCx7dzg2NkTsMLshLknXzeqnDYab5m?=
 =?us-ascii?Q?dght+PxgJbspmpuhXUbQd0gm3Yn+JnC8TN5b0bhEjEQHJ5FiiEOJiROhCV70?=
 =?us-ascii?Q?3yoDCIfgKuZVVqxL9rWfWMh/euyP5155OmjUYr3MWKW3VAGC9yL9AQ6GbjTg?=
 =?us-ascii?Q?bBGQNzCGDo+eN0aeUpZIgc0fyjhHLY15BxARpcsTigabvrWbLCCvKxa5zcV1?=
 =?us-ascii?Q?dTvZXq+OmM/EC8mTKFZj9T7NLspyQ0ogfZcobgusN+aWNDhedqVSPR4/iMNG?=
 =?us-ascii?Q?E7xWHGDXX+NdQRFEqxt9Nt2AQiTdUWECMoyGRts499IPuAvA0gE+LlDEiNeh?=
 =?us-ascii?Q?K8NpKuazJLgrfqYm7N1QKaZusgrdYmhRd5Zw/hDeiokvUZEf+oWEouEDD4Jx?=
 =?us-ascii?Q?bNTCkKicMeLMsO/2dLbBK/TjBG5oMW41MDhpJSI96rpsMzISnvc/sxEoGMDr?=
 =?us-ascii?Q?q/X16C+D88fjWdU/OCclNU5sausoUGq49UFY/MT1K7DDC2L/GfK5/pSFiWQw?=
 =?us-ascii?Q?h21iG0vlcH5P9g5B+OskKgCIBPed4cWn8ejK4LHYce7ZNSFGvb0ZEXdjFZva?=
 =?us-ascii?Q?oc3Jzit4HOVRrw1mLkOiU3gS6ZDQltwVycvVmQDIL+wZEpPTiCShrHflL34N?=
 =?us-ascii?Q?glWAhc59X9pXtyIUIvPH0mYutGVZrcpzxAuha0kMqvqsPJ2JRFMB80pPd2nt?=
 =?us-ascii?Q?DMjsW+0m0DSv2ytTcSbK3cBcGo2PLwXe1ZNnUEb0t0Hsr0sKF8VAzGJsdH2a?=
 =?us-ascii?Q?Iahse5Mm0uMqixn5lrM2gHwrmPBq?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 06:50:32.4396
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 793a59ad-cbba-492c-754c-08dc74ab50ee
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F3.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7032

In PVH dom0, the gsis don't get registered, but the gsi of
a passthrough device must be configured for it to be able to be
mapped into a domU.

When assign a device to passthrough, proactively setup the gsi
of the device during that process.

Co-developed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 arch/x86/xen/enlighten_pvh.c       | 21 +++++++++++++
 drivers/acpi/pci_irq.c             |  2 +-
 drivers/xen/acpi.c                 | 50 ++++++++++++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c | 21 +++++++++++++
 include/linux/acpi.h               |  1 +
 include/xen/acpi.h                 | 10 ++++++
 6 files changed, 104 insertions(+), 1 deletion(-)

diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c
index 27a2a02ef8fb..711cdcbc6916 100644
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -4,6 +4,7 @@
 #include <linux/mm.h>
 
 #include <xen/hvc-console.h>
+#include <xen/acpi.h>
 
 #include <asm/bootparam.h>
 #include <asm/io_apic.h>
@@ -27,6 +28,26 @@
 bool __ro_after_init xen_pvh;
 EXPORT_SYMBOL_GPL(xen_pvh);
 
+int xen_pvh_setup_gsi(int gsi, int trigger, int polarity)
+{
+	int ret;
+	struct physdev_setup_gsi setup_gsi;
+
+	setup_gsi.gsi = gsi;
+	setup_gsi.triggering = (trigger == ACPI_EDGE_SENSITIVE ? 0 : 1);
+	setup_gsi.polarity = (polarity == ACPI_ACTIVE_HIGH ? 0 : 1);
+
+	ret = HYPERVISOR_physdev_op(PHYSDEVOP_setup_gsi, &setup_gsi);
+	if (ret == -EEXIST) {
+		xen_raw_printk("Already setup the GSI :%d\n", gsi);
+		ret = 0;
+	} else if (ret)
+		xen_raw_printk("Fail to setup GSI (%d)!\n", gsi);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(xen_pvh_setup_gsi);
+
 void __init xen_pvh_init(struct boot_params *boot_params)
 {
 	u32 msr;
diff --git a/drivers/acpi/pci_irq.c b/drivers/acpi/pci_irq.c
index ff30ceca2203..630fe0a34bc6 100644
--- a/drivers/acpi/pci_irq.c
+++ b/drivers/acpi/pci_irq.c
@@ -288,7 +288,7 @@ static int acpi_reroute_boot_interrupt(struct pci_dev *dev,
 }
 #endif /* CONFIG_X86_IO_APIC */
 
-static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
+struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
 {
 	struct acpi_prt_entry *entry = NULL;
 	struct pci_dev *bridge;
diff --git a/drivers/xen/acpi.c b/drivers/xen/acpi.c
index 6893c79fd2a1..9e2096524fbc 100644
--- a/drivers/xen/acpi.c
+++ b/drivers/xen/acpi.c
@@ -30,6 +30,7 @@
  * IN THE SOFTWARE.
  */
 
+#include <linux/pci.h>
 #include <xen/acpi.h>
 #include <xen/interface/platform.h>
 #include <asm/xen/hypercall.h>
@@ -75,3 +76,52 @@ int xen_acpi_notify_hypervisor_extended_sleep(u8 sleep_state,
 	return xen_acpi_notify_hypervisor_state(sleep_state, val_a,
 						val_b, true);
 }
+
+struct acpi_prt_entry {
+	struct acpi_pci_id      id;
+	u8                      pin;
+	acpi_handle             link;
+	u32                     index;
+};
+
+int xen_acpi_get_gsi_info(struct pci_dev *dev,
+						  int *gsi_out,
+						  int *trigger_out,
+						  int *polarity_out)
+{
+	int gsi;
+	u8 pin;
+	struct acpi_prt_entry *entry;
+	int trigger = ACPI_LEVEL_SENSITIVE;
+	int polarity = acpi_irq_model == ACPI_IRQ_MODEL_GIC ?
+				      ACPI_ACTIVE_HIGH : ACPI_ACTIVE_LOW;
+
+	if (!dev || !gsi_out || !trigger_out || !polarity_out)
+		return -EINVAL;
+
+	pin = dev->pin;
+	if (!pin)
+		return -EINVAL;
+
+	entry = acpi_pci_irq_lookup(dev, pin);
+	if (entry) {
+		if (entry->link)
+			gsi = acpi_pci_link_allocate_irq(entry->link,
+							 entry->index,
+							 &trigger, &polarity,
+							 NULL);
+		else
+			gsi = entry->index;
+	} else
+		gsi = -1;
+
+	if (gsi < 0)
+		return -EINVAL;
+
+	*gsi_out = gsi;
+	*trigger_out = trigger;
+	*polarity_out = polarity;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_acpi_get_gsi_info);
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 46c40ec8a18e..2b90d832d0a7 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -21,6 +21,9 @@
 #include <xen/events.h>
 #include <xen/pci.h>
 #include <xen/xen.h>
+#ifdef CONFIG_ACPI
+#include <xen/acpi.h>
+#endif
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
@@ -367,6 +370,9 @@ static int pcistub_match(struct pci_dev *dev)
 static int pcistub_init_device(struct pci_dev *dev)
 {
 	struct xen_pcibk_dev_data *dev_data;
+#ifdef CONFIG_ACPI
+	int gsi, trigger, polarity;
+#endif
 	int err = 0;
 
 	dev_dbg(&dev->dev, "initializing...\n");
@@ -435,6 +441,21 @@ static int pcistub_init_device(struct pci_dev *dev)
 			goto config_release;
 		pci_restore_state(dev);
 	}
+
+#ifdef CONFIG_ACPI
+	err = xen_acpi_get_gsi_info(dev, &gsi, &trigger, &polarity);
+	if (err) {
+		dev_err(&dev->dev, "Fail to get gsi info!\n");
+		goto config_release;
+	}
+
+	if (xen_initial_domain() && xen_pvh_domain()) {
+		err = xen_pvh_setup_gsi(gsi, trigger, polarity);
+		if (err)
+			goto config_release;
+	}
+#endif
+
 	/* Now disable the device (this also ensures some private device
 	 * data is setup before we export)
 	 */
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 34829f2c517a..f8690b02bba4 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -361,6 +361,7 @@ void acpi_unregister_gsi (u32 gsi);
 
 struct pci_dev;
 
+struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin);
 int acpi_pci_irq_enable (struct pci_dev *dev);
 void acpi_penalize_isa_irq(int irq, int active);
 bool acpi_isa_irq_available(int irq);
diff --git a/include/xen/acpi.h b/include/xen/acpi.h
index b1e11863144d..9b50027113f3 100644
--- a/include/xen/acpi.h
+++ b/include/xen/acpi.h
@@ -67,10 +67,20 @@ static inline void xen_acpi_sleep_register(void)
 		acpi_suspend_lowlevel = xen_acpi_suspend_lowlevel;
 	}
 }
+int xen_pvh_setup_gsi(int gsi, int trigger, int polarity);
 #else
 static inline void xen_acpi_sleep_register(void)
 {
 }
+
+static inline int xen_pvh_setup_gsi(int gsi, int trigger, int polarity)
+{
+	return -1;
+}
 #endif
 
+int xen_acpi_get_gsi_info(struct pci_dev *dev,
+						  int *gsi_out,
+						  int *trigger_out,
+						  int *polarity_out);
 #endif	/* _XEN_ACPI_H */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 06:50:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:50:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721928.1125626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TE-0003jw-Rn; Wed, 15 May 2024 06:50:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721928.1125626; Wed, 15 May 2024 06:50:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TE-0003jp-P5; Wed, 15 May 2024 06:50:36 +0000
Received: by outflank-mailman (input) for mailman id 721928;
 Wed, 15 May 2024 06:50:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NNmL=MS=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s78TD-0003jj-9O
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:50:35 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6c9d5701-1287-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 08:50:33 +0200 (CEST)
Received: from MW4PR04CA0047.namprd04.prod.outlook.com (2603:10b6:303:6a::22)
 by BY5PR12MB4050.namprd12.prod.outlook.com (2603:10b6:a03:207::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Wed, 15 May
 2024 06:50:29 +0000
Received: from CO1PEPF000044F0.namprd05.prod.outlook.com
 (2603:10b6:303:6a:cafe::1a) by MW4PR04CA0047.outlook.office365.com
 (2603:10b6:303:6a::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Wed, 15 May 2024 06:50:29 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000044F0.mail.protection.outlook.com (10.167.241.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 06:50:29 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 01:50:25 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c9d5701-1287-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fgychTMBqg3dLP9Rpi7XD6+K1Xuryn7R5YsfNG+XGJdLhT3U3bDhh1Axe80XC8jZYhqzjSMj67GsV+2mQkXCwqriJnq0OKCaoRpCdIOwViV0FIUGXfvohxgrMq7XbLk3rnP3dgUcAH9d9UhrQ0n0F65YdFLgfPl8HeamAr3wROfcY643hXRo7yV8lNbCP7bQHqIqbbDyjuZ1n74uyTx1HCidaSgwSZrWfZXtyk0vVwlxRJT/KPjM5GIsLoOCQ37vKpekwF75u4lTWpAPHdO6j73sarkn8w+4B01jBYTo33jYnElEM75CMZS87EMKCJ4SAC/z7jGikL7pP8zA7vkK6w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8u1a9zv+ID5+qnxPmAar9kgCl4ZyYytPz2Gob4RuFsw=;
 b=YKD23F813a+pkS5YMnFTdKc525Uf09a+3+8vjuGhaAgXUGjfg+FZ9f//CiRd1qWyHxHNMcyhmxi6ORlSZFeg9X5dKA1zFnHJMTe5VvvqUlsJJfhcpbxY2fH/Jf1MrW4sJOv28ecl+A/KAZrnt5WxEbZPa1zMHBEaf1khbzjyww9hNMy5axfsKkDvTKm5sMttyHIce6FMluozjZmnyDSBdQLBscfXyDQGYZlo9mlieN4vXWatfLuOBK55tRLGd3WfnVO5ZAasa0jLbXo/Q7fQtv7u+uEX8RLZ8emHCnt0xV1aBL6NJlyHoJVsXwo7ljzvEUv8UDmXyZjITB0fe93eSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8u1a9zv+ID5+qnxPmAar9kgCl4ZyYytPz2Gob4RuFsw=;
 b=c0h5DGPdF0mmlJH5qViVeZejRPqb5imY18Hcjjjic8Avcb9BgTY1XAffeqwe4LshooDkZKQ5OmXrN8jjgJvhx6Lf+6/qxRoQdbWTs3WzgBnfrmmQcn6TbCrEr27e+3wjTNDETH9jq26glv8pBsnFFBf/FUIrSKx5CNe2fDlXUCA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, "Rafael J .
 Wysocki" <rafael@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-acpi@vger.kernel.org>, Huang Rui
	<Ray.Huang@amd.com>, Jiqian Chen <Jiqian.Chen@amd.com>
Subject: [RFC KERNEL PATCH v7 0/2] Support device passthrough when dom0 is PVH on Xen
Date: Wed, 15 May 2024 14:50:09 +0800
Message-ID: <20240515065011.13797-1-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F0:EE_|BY5PR12MB4050:EE_
X-MS-Office365-Filtering-Correlation-Id: b4f0f5bb-3de9-4562-742a-08dc74ab4ee7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?/V96+cHaacDH9fm2Z+RMeiJZ6HoOjD/MHrZ/dvlYik934EJ+vEY1wsZeBDi6?=
 =?us-ascii?Q?+ShzFp91Ya7vDc9Uuo3GqxYdlpGPhRe8i1JdAfYA6Bt1yx/wDqe2RKNbutkA?=
 =?us-ascii?Q?0Drc4Vqr9eQYo62tW2oBBEIMupD6ce9ImzUeJEMAeMBnl5EXo67Rynyj4xtg?=
 =?us-ascii?Q?r5OyRUmAPMPZHOSkIiytySSlpG7s6/40OkcHS3S2hoxBDcTICo70vvXS/OGk?=
 =?us-ascii?Q?Y2aU2TdKxdRvf52OLIawKPGrmxCkpNKXRDCEgkkhwb0Nd8r679nUAfFp+B16?=
 =?us-ascii?Q?UwOoW1nxESWTAI6tX3z94s1nnU8FAWGMMQFjYnbRFeoK/0u/du08YBCmS441?=
 =?us-ascii?Q?1nvSIh942eKb7p+uXRLn8216ZAsUwVl5HSh8fCQ5T7W018OWLP34J0QtjfF+?=
 =?us-ascii?Q?wA8c8A0fnhLm69Jjg7Wf+tYRmULYHyIPgdpkoQ/bgXMOX2PC7XC48ZIXH/Rz?=
 =?us-ascii?Q?jq/MlkyRLzbOKgBeFiVUn5a59EybrZaKKWG2gMvfB7aUWfi8joNHh0gnHhnN?=
 =?us-ascii?Q?L7PbW0ku0XEL4pvxdTQu8mvVgrnXteYVGHGWd8AS3QeRy6Qbj0KczRm1vwOq?=
 =?us-ascii?Q?yAXkJuVh/sNPnC3NocKehOX871dD1L01+gQXD6ozSRzquS+TlMS/8wxGV3Nj?=
 =?us-ascii?Q?qV/YBE9lF+q+N5B1AyyMaRtGt0ojPcjFncK/mgzMB8ccQmAExR7y6cczFu12?=
 =?us-ascii?Q?hhUTBGKZsWjlQMDNS5s5kke6r7t45bvrtiktEIDoiVFp0JH50igBPNUAZlGa?=
 =?us-ascii?Q?GUqzNXskQ6hSeO9ZV+PB0be8c1vl3hXOEGgZC6rMWbNoAKfOxsl2MG/WsqK/?=
 =?us-ascii?Q?ZKOSvBPv9YU4GI7nF/TydUQqPdl7sBI3ntFrGlqOAl6tV53ExBmWkddhlviQ?=
 =?us-ascii?Q?bTR5O+sQq3PjVlQl9G0m28rL0QMgMx5zFN/g8zQ7I3o/hdzs/WV79i7xVpEv?=
 =?us-ascii?Q?Qm9a+ba2NCqHHDo7gQqKVLvZpAJ30oHPbFGtKLT6uAXKZ8wsiMljCn3YPzM8?=
 =?us-ascii?Q?lGFBArqREDWbY5zsjdLv1KI9lmVRTNW0SDQU2rBJUcy76gcMmf0pI/FUTSkB?=
 =?us-ascii?Q?fyKkq4qUaTbGu2ANFdDM0oaaybigyExhc5RKCx+nFLrDKMPOoMvG+Kk//Cqn?=
 =?us-ascii?Q?iV4ml/NwyAVkPPWVKd33QDXXbUQP0lD3UA7G7age8SPJkiEvPmHtiIxBbX3h?=
 =?us-ascii?Q?CpTxBCEpAiv+Z4oSWt5ExDNTylPbUPp2FWrRsgSw8uAY4VB1dJJ/oITyys3r?=
 =?us-ascii?Q?29ytAUyLFiuo2fyaHCGh2ZdokBlEkJHhrlaHIFl4ctWEm4UhxVjR4B8TS0W7?=
 =?us-ascii?Q?lqRZM1e+hOPdY4Z6GuSscYU48t+zYQX0V3NQGgA0axK1/w=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 06:50:29.0382
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b4f0f5bb-3de9-4562-742a-08dc74ab4ee7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4050

Hi All,
This is v7 series to support passthrough on Xen when dom0 is PVH.
v6->v7 change:
* the first patch of v6 was already merged into branch linux_next.
* patch#1: is the patch#2 of v6. move the implementation of function xen_acpi_get_gsi_info to
           file drivers/xen/acpi.c, that modification is more convenient for the subsequent
           patch to obtain gsi.
* patch#2: is the patch#3 of v6. add a new parameter "gsi" to struct pcistub_device and set
           gsi when pcistub initialize device. Then when userspace wants to get gsi by passing
           sbdf, we can return that gsi.


Best regards,
Jiqian Chen




v5->v6 change:
* patch#3: change to add a new syscall to translate irq to gsi, instead adding a new gsi sysfs.


v4->v5 changes:
* patch#1: Add Reviewed-by Stefano
* patch#2: Add Reviewed-by Stefano
* patch#3: No changes


v3->v4 changes:
* patch#1: change the comment of PHYSDEVOP_pci_device_state_reset; use a new function
           pcistub_reset_device_state to wrap __pci_reset_function_locked and xen_reset_device_state,
           and call pcistub_reset_device_state in pci_stub.c
* patch#2: remove map_pirq from xen_pvh_passthrough_gsi


v2->v3 changes:
* patch#1: add condition to limit do xen_reset_device_state for no-pv domain in pcistub_init_device.
* patch#2: Abandoning previous implementations that call unmask_irq. To setup gsi and map pirq for
           passthrough device in pcistub_init_device.
* patch#3: Abandoning previous implementations that adds new syscall to get gsi from irq. To add a new
           sysfs for gsi, then userspace can get gsi number from sysfs.


Below is the description of v2 cover letter:
This series of patches are the v2 of the implementation of passthrough when dom0 is PVH on Xen.
We sent the v1 to upstream before, but the v1 had so many problems and we got lots of suggestions.
I will introduce all issues that these patches try to fix and the differences between v1 and v2.

Issues we encountered:
1. pci_stub failed to write bar for a passthrough device.
Problem: when we run \u201csudo xl pci-assignable-add <sbdf>\u201d to assign a device, pci_stub will
call \u201cpcistub_init_device() -> pci_restore_state() -> pci_restore_config_space() ->
pci_restore_config_space_range() -> pci_restore_config_dword() -> pci_write_config_dword(), the pci
config write will trigger an io interrupt to bar_write() in the xen, but the bar->enabled was set before,
the write is not allowed now, and then when bar->Qemu config the passthrough device in xen_pt_realize(),
it gets invalid bar values.

Reason: the reason is that we don't tell vPCI that the device has been reset, so the current cached state
in pdev->vpci is all out of date and is different from the real device state.

Solution: to solve this problem, the first patch of kernel(xen/pci: Add xen_reset_device_state
function) and the fist patch of xen(xen/vpci: Clear all vpci status of device) add a new hypercall to
reset the state stored in vPCI when the state of real device has changed.
Thank Roger for the suggestion of this v2, and it is different from v1
(https://lore.kernel.org/xen-devel/20230312075455.450187-3-ray.huang@amd.com/), v1 simply allow domU to
write pci bar, it does not comply with the design principles of vPCI.

2. failed to do PHYSDEVOP_map_pirq when dom0 is PVH
Problem: HVM domU will do PHYSDEVOP_map_pirq for a passthrough device by using gsi. See
xen_pt_realize->xc_physdev_map_pirq and pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
will call into Xen, but in hvm_physdev_op(), PHYSDEVOP_map_pirq is not allowed.

Reason: In hvm_physdev_op(), the variable "currd" is PVH dom0 and PVH has no X86_EMU_USE_PIRQ flag, it
will fail at has_pirq check.

Solution: I think we may need to allow PHYSDEVOP_map_pirq when "currd" is dom0 (at present dom0 is PVH).
The second patch of xen(x86/pvh: Open PHYSDEVOP_map_pirq for PVH dom0) allow PVH dom0 do
PHYSDEVOP_map_pirq. This v2 patch is better than v1, v1 simply remove the has_pirq check
(xen https://lore.kernel.org/xen-devel/20230312075455.450187-4-ray.huang@amd.com/).

3. the gsi of a passthrough device doesn't be unmasked
 3.1 failed to check the permission of pirq
 3.2 the gsi of passthrough device was not registered in PVH dom0

Problem:
3.1 callback function pci_add_dm_done() will be called when qemu config a passthrough device for domU.
This function will call xc_domain_irq_permission()-> pirq_access_permitted() to check if the gsi has
corresponding mappings in dom0. But it didn\u2019t, so failed. See
XEN_DOMCTL_irq_permission->pirq_access_permitted, "current" is PVH dom0 and it return irq is 0.
3.2 it's possible for a gsi (iow: vIO-APIC pin) to never get registered on PVH dom0, because the
devices of PVH are using MSI(-X) interrupts. However, the IO-APIC pin must be configured for it to be
able to be mapped into a domU.

Reason: After searching codes, I find "map_pirq" and "register_gsi" will be done in function
vioapic_write_redirent->vioapic_hwdom_map_gsi when the gsi(aka ioapic's pin) is unmasked in PVH dom0.
So the two problems can be concluded to that the gsi of a passthrough device doesn't be unmasked.

Solution: to solve these problems, the second patch of kernel(xen/pvh: Unmask irq for passthrough device
in PVH dom0) call the unmask_irq() when we assign a device to be passthrough. So that passthrough devices
can have the mapping of gsi on PVH dom0 and gsi can be registered. This v2 patch is different from the
v1( kernel https://lore.kernel.org/xen-devel/20230312120157.452859-5-ray.huang@amd.com/,
kernel https://lore.kernel.org/xen-devel/20230312120157.452859-5-ray.huang@amd.com/ and
xen https://lore.kernel.org/xen-devel/20230312075455.450187-5-ray.huang@amd.com/),
v1 performed "map_pirq" and "register_gsi" on all pci devices on PVH dom0, which is unnecessary and may
cause multiple registration.

4. failed to map pirq for gsi
Problem: qemu will call xc_physdev_map_pirq() to map a passthrough device\u2019s gsi to pirq in function
xen_pt_realize(). But failed.

Reason: According to the implement of xc_physdev_map_pirq(), it needs gsi instead of irq, but qemu pass
irq to it and treat irq as gsi, it is got from file /sys/bus/pci/devices/xxxx:xx:xx.x/irq in function
xen_host_pci_device_get(). But actually the gsi number is not equal with irq. On PVH dom0, when it
allocates irq for a gsi in function acpi_register_gsi_ioapic(), allocation is dynamic, and follow the
principle of applying first, distributing first. And if you debug the kernel codes(see
function __irq_alloc_descs), you will find the irq number is allocated from small to large by order, but
the applying gsi number is not, gsi 38 may come before gsi 28, that causes gsi 38 get a smaller irq number
than gsi 28, and then gsi != irq.

Solution: we can record the relation between gsi and irq, then when userspace(qemu) want to use gsi, we
can do a translation. The third patch of kernel(xen/privcmd: Add new syscall to get gsi from irq) records
all the relations in acpi_register_gsi_xen_pvh() when dom0 initialize pci devices, and provide a syscall
for userspace to get the gsi from irq. The third patch of xen(tools: Add new function to get gsi from irq)
add a new function xc_physdev_gsi_from_irq() to call the new syscall added on kernel side.
And then userspace can use that function to get gsi. Then xc_physdev_map_pirq() will success. This v2 patch
is the same as v1( kernel https://lore.kernel.org/xen-devel/20230312120157.452859-6-ray.huang@amd.com/ and
xen https://lore.kernel.org/xen-devel/20230312075455.450187-6-ray.huang@amd.com/)

About the v2 patch of qemu, just change an included head file, other are similar to the v1 (
qemu https://lore.kernel.org/xen-devel/20230312092244.451465-19-ray.huang@amd.com/), just call
xc_physdev_gsi_from_irq() to get gsi from irq.

Jiqian Chen (2):
  xen/pvh: Setup gsi for passthrough device
  xen/privcmd: Add new syscall to get gsi from dev

 arch/x86/xen/enlighten_pvh.c       | 21 +++++++++++
 drivers/acpi/pci_irq.c             |  2 +-
 drivers/xen/acpi.c                 | 50 +++++++++++++++++++++++++
 drivers/xen/privcmd.c              | 28 ++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c | 59 ++++++++++++++++++++++++++++--
 include/linux/acpi.h               |  1 +
 include/uapi/xen/privcmd.h         |  7 ++++
 include/xen/acpi.h                 | 12 ++++++
 8 files changed, 176 insertions(+), 4 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 06:50:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 06:50:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721930.1125646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TM-0004G1-F9; Wed, 15 May 2024 06:50:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721930.1125646; Wed, 15 May 2024 06:50:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78TM-0004Fu-BY; Wed, 15 May 2024 06:50:44 +0000
Received: by outflank-mailman (input) for mailman id 721930;
 Wed, 15 May 2024 06:50:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NNmL=MS=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s78TL-0003jj-7N
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 06:50:43 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 71e2267f-1287-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 08:50:42 +0200 (CEST)
Received: from SJ0PR03CA0332.namprd03.prod.outlook.com (2603:10b6:a03:39c::7)
 by BY1PR12MB8445.namprd12.prod.outlook.com (2603:10b6:a03:523::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.25; Wed, 15 May
 2024 06:50:36 +0000
Received: from CO1PEPF000044F1.namprd05.prod.outlook.com
 (2603:10b6:a03:39c:cafe::df) by SJ0PR03CA0332.outlook.office365.com
 (2603:10b6:a03:39c::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Wed, 15 May 2024 06:50:36 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000044F1.mail.protection.outlook.com (10.167.241.71) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 06:50:35 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 01:50:32 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71e2267f-1287-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T9pgACrF4X3+B3IhRAwGmJgIO7E2pUB+fzIYbC+Vy5ayBTfssrnRoBl7R3nMCJvVLlkprYBrH2O7Z57K+vpW79gXqmaVld6E8/Jgc7LSooCjASdxNwfSWmje7jThUsygO7jm3c0kLfGZrxdeFp1Lp2Juo8sg5SuqKlXL4uJRTbaCDFvmGZDo1h8/TfsldloCIRDPsnkKXe1W5AOtXGOsn6yuA97nyFd56EdQ5OctnBsW6i++P2OOwbvTEuaYS/qTjQ8vxqbll7yS569Jg3jCccGhrG0JoO3Td3o6iCruJOCjhkXp2im23zEnK2XHet5GGVwB+gZwcb123gGXzlSJew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=vcwYxAcPB960myYrq5oQXoH0v7z+ym1e/y8k8X4re90=;
 b=Z+OxDFAGL3gjw6O8jzWIpW/xP5mgPJYFpYWgnnTSoSyq8kDqsU3VUXQ06uwdOFT4n/km5aukoS2v4S6T8nNfGhm9bHzJCAMI15r9qG1MyHGYV2JpYNcuvwmPPfFT6T3heyEzpHoq0A8uYbnZyxpQwHjpjmNWpYka6wnbzv3xLlnpz+/8V4yLPPnyXARufQxUBiaaQAjaVngJHxy7EIQfpeX1YDaLO5lWu0ndLCSYSg5ZbzK+7rQ2aBYbR5ThAosKMaoy6MHH4CSAwbiZ/Yfi2y9XVBARbcLFRaDYfJvRjTyFLGCiYePbtHuBEfPYJp12UKmEPT+nmhJ3W0m3LpUVXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vcwYxAcPB960myYrq5oQXoH0v7z+ym1e/y8k8X4re90=;
 b=PO8F082YiAagWvJtwAV/cpy5zj1sHwsqD18jMDH71aOTD5rr0pcWV6ZSoVW6JlQz/nCMsk5WNCwtnJ8wYmCSQQVZewmfYaaUDCavDRfMmH+6M/d1E1eV5pLkRiO5TRRRcA979bSn+vT9wFfr+iJONBTFxH4jz33WVpmd229O+1A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: Juergen Gross <jgross@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, "Rafael J .
 Wysocki" <rafael@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-acpi@vger.kernel.org>, Huang Rui
	<Ray.Huang@amd.com>, Jiqian Chen <Jiqian.Chen@amd.com>, Huang Rui
	<ray.huang@amd.com>
Subject: [RFC KERNEL PATCH v7 2/2] xen/privcmd: Add new syscall to get gsi from dev
Date: Wed, 15 May 2024 14:50:11 +0800
Message-ID: <20240515065011.13797-3-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515065011.13797-1-Jiqian.Chen@amd.com>
References: <20240515065011.13797-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F1:EE_|BY1PR12MB8445:EE_
X-MS-Office365-Filtering-Correlation-Id: 96522366-1522-4902-667c-08dc74ab52f1
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?XkkMZld4FOxuVGYl6xJEg2UAyYjIZxxtmqkNByGES0zePyuHySd/olj6zrh5?=
 =?us-ascii?Q?mWwDjTWAvsfJd0yey3LfcOAUPszuNilq8laBnuEoz9IntpPa9c5FViO3TgXo?=
 =?us-ascii?Q?Ug1cvQdQXwOqlrya3grdypEvs2vyQxQ6EErfLu+/TCFk+2gvmU4youfphJsW?=
 =?us-ascii?Q?FR7BBxCCBdk7JVCb7NyEABphhki69XtxygOpUU3hap/uP/RTHYP/7Tjb7W3U?=
 =?us-ascii?Q?7W0292Lt7ZJNq+AAQDZli1CXLf5Ip6kvWdEnSzLRtKF2+vNjg+DYGxiNYILd?=
 =?us-ascii?Q?N57QupOw+yfDFRd89buUkNROMHYS7gPmuv6/HkFILXkFERWIbx3j/6rfPdbC?=
 =?us-ascii?Q?+KBMxUk/Ua+jdDznUtxPaAI83CI3oksatkvOnfWPEkH2YuHGTkGdMtfesouD?=
 =?us-ascii?Q?5HhiiPfzgCNjG3a09x9zrY5Fo/5QBIkrg3tvJNJNVQEDbse3YKmLjhc8JLN/?=
 =?us-ascii?Q?onJIKcidrvAYq1E2FHLvKOeTYdfL6UFlCafFDx23TiFeSuN5LyQdqXQ3vCs6?=
 =?us-ascii?Q?jzHzHYY4eYfeU/AjUP6EBlRTTbU73iWnMBaaY3ajIuvd7zPlGbg5CGZx3vUU?=
 =?us-ascii?Q?FM2VBDDJTXjcYuUtEVLAMBKvIuzJ8qDCPNpCWrS+GHV7bhIJcBLOeDd2bQlW?=
 =?us-ascii?Q?uwu44AYA3SWqv+isjZvZKt40f2LGc1qB8d1HOSBfIEq1WA/tB9U3JbrobDDl?=
 =?us-ascii?Q?iPB2EMBgRWciL3Cs3k562LMjZ3xd5aNVnMOF7m4CunoeGb41+LIBMPhWbYQm?=
 =?us-ascii?Q?ahTMYTgWMoFK08UHtwsumw7XJ6jjRiEf0oAGkxw7ZQD2z4yagteGaDFlGP/Z?=
 =?us-ascii?Q?4mXB0ReJhMzYcHtoZPToYDMmLv0L5CP80uUKpcSDh0JKoO9EgD5oCSd2MbLP?=
 =?us-ascii?Q?skOv7/uxTrBHFAwYHOgbfXq5JXiAsmpbDcU3bDqkqNWKAaeZo05uSPFuq1dh?=
 =?us-ascii?Q?G1nUWPlAIsgeYXZ62GowjERojQMrYHr3j8laH/pfIJlpVt0RlhnDaZuUghlz?=
 =?us-ascii?Q?xmh66V7+X9+divEFLVDN8CDv5FXSBmtVKzc0D3mu7WWoOC+ZJbJram+XokXj?=
 =?us-ascii?Q?Cx+zs1YTtNzKm3us+MTpJzr7ZpfypcwvSsUlwvrOrFtLcKlnvGk8AvVGF5eW?=
 =?us-ascii?Q?jv4ZdbWMw5bN7tT1Vk/2OXo13C3Q+IooYfnvVc91xUlcfcaSypeWftnAfmJY?=
 =?us-ascii?Q?WnGf86RwvsZ1nuul+okQNw72HkmwvawgUQXZeKUxeHAKFKECkg3rE8nwYw3M?=
 =?us-ascii?Q?4xzF5Aqzha5suO9UX6ojE4YgVI6JNnMANuKC3BS4qhCXbAB0g2s/CSQNjUSk?=
 =?us-ascii?Q?hxudPpEMk7gk50JxxaDOxiXew89kHFmb8deMBV9t1W+FFs3dmHPGBd5+xHvd?=
 =?us-ascii?Q?daH9YUUrzEIQN5mPLJIhg7NKZcxA?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 06:50:35.7987
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 96522366-1522-4902-667c-08dc74ab52f1
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F1.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR12MB8445

In PVH dom0, it uses the linux local interrupt mechanism,
when it allocs irq for a gsi, it is dynamic, and follow
the principle of applying first, distributing first. And
the irq number is alloced from small to large, but the
applying gsi number is not, may gsi 38 comes before gsi 28,
it causes the irq number is not equal with the gsi number.
And when passthrough a device, QEMU will use device's gsi
number to do pirq mapping, but the gsi number is got from
file /sys/bus/pci/devices/<sbdf>/irq, irq!= gsi, so it will
fail when mapping.
And in current linux codes, there is no method to get gsi
for userspace.

For above purpose, record gsi of pcistub devices when init
pcistub and add a new syscall into privcmd to let userspace
can get gsi when they have a need.

Co-developed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
---
 drivers/xen/privcmd.c              | 28 ++++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c | 38 +++++++++++++++++++++++++++---
 include/uapi/xen/privcmd.h         |  7 ++++++
 include/xen/acpi.h                 |  2 ++
 4 files changed, 72 insertions(+), 3 deletions(-)

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 67dfa4778864..5953a03b5cb0 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -45,6 +45,9 @@
 #include <xen/page.h>
 #include <xen/xen-ops.h>
 #include <xen/balloon.h>
+#ifdef CONFIG_ACPI
+#include <xen/acpi.h>
+#endif
 
 #include "privcmd.h"
 
@@ -842,6 +845,27 @@ static long privcmd_ioctl_mmap_resource(struct file *file,
 	return rc;
 }
 
+static long privcmd_ioctl_gsi_from_dev(struct file *file, void __user *udata)
+{
+	struct privcmd_gsi_from_dev kdata;
+
+	if (copy_from_user(&kdata, udata, sizeof(kdata)))
+		return -EFAULT;
+
+#ifdef CONFIG_ACPI
+	kdata.gsi = pcistub_get_gsi_from_sbdf(kdata.sbdf);
+	if (kdata.gsi == -1)
+		return -EINVAL;
+#else
+	kdata.gsi = -1;
+#endif
+
+	if (copy_to_user(udata, &kdata, sizeof(kdata)))
+		return -EFAULT;
+
+	return 0;
+}
+
 #ifdef CONFIG_XEN_PRIVCMD_EVENTFD
 /* Irqfd support */
 static struct workqueue_struct *irqfd_cleanup_wq;
@@ -1529,6 +1553,10 @@ static long privcmd_ioctl(struct file *file,
 		ret = privcmd_ioctl_ioeventfd(file, udata);
 		break;
 
+	case IOCTL_PRIVCMD_GSI_FROM_DEV:
+		ret = privcmd_ioctl_gsi_from_dev(file, udata);
+		break;
+
 	default:
 		break;
 	}
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 2b90d832d0a7..4b62b4d377a9 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -56,6 +56,9 @@ struct pcistub_device {
 
 	struct pci_dev *dev;
 	struct xen_pcibk_device *pdev;/* non-NULL if struct pci_dev is in use */
+#ifdef CONFIG_ACPI
+	int gsi;
+#endif
 };
 
 /* Access to pcistub_devices & seized_devices lists and the initialize_devices
@@ -88,6 +91,9 @@ static struct pcistub_device *pcistub_device_alloc(struct pci_dev *dev)
 
 	kref_init(&psdev->kref);
 	spin_lock_init(&psdev->lock);
+#ifdef CONFIG_ACPI
+	psdev->gsi = -1;
+#endif
 
 	return psdev;
 }
@@ -220,6 +226,25 @@ static struct pci_dev *pcistub_device_get_pci_dev(struct xen_pcibk_device *pdev,
 	return pci_dev;
 }
 
+#ifdef CONFIG_ACPI
+int pcistub_get_gsi_from_sbdf(unsigned int sbdf)
+{
+	struct pcistub_device *psdev;
+	int domain = sbdf >> 16;
+	int bus = (sbdf >> 8) & 0xff;
+	int slot = (sbdf >> 3) & 0x1f;
+	int func = sbdf & 0x7;
+
+	psdev = pcistub_device_find(domain, bus, slot, func);
+
+	if (!psdev)
+		return -1;
+
+	return psdev->gsi;
+}
+EXPORT_SYMBOL_GPL(pcistub_get_gsi_from_sbdf);
+#endif
+
 struct pci_dev *pcistub_get_pci_dev_by_slot(struct xen_pcibk_device *pdev,
 					    int domain, int bus,
 					    int slot, int func)
@@ -367,14 +392,20 @@ static int pcistub_match(struct pci_dev *dev)
 	return found;
 }
 
-static int pcistub_init_device(struct pci_dev *dev)
+static int pcistub_init_device(struct pcistub_device *psdev)
 {
 	struct xen_pcibk_dev_data *dev_data;
+	struct pci_dev *dev;
 #ifdef CONFIG_ACPI
 	int gsi, trigger, polarity;
 #endif
 	int err = 0;
 
+	if (!psdev)
+		return -EINVAL;
+
+	dev = psdev->dev;
+
 	dev_dbg(&dev->dev, "initializing...\n");
 
 	/* The PCI backend is not intended to be a module (or to work with
@@ -448,6 +479,7 @@ static int pcistub_init_device(struct pci_dev *dev)
 		dev_err(&dev->dev, "Fail to get gsi info!\n");
 		goto config_release;
 	}
+	psdev->gsi = gsi;
 
 	if (xen_initial_domain() && xen_pvh_domain()) {
 		err = xen_pvh_setup_gsi(gsi, trigger, polarity);
@@ -495,7 +527,7 @@ static int __init pcistub_init_devices_late(void)
 
 		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
 
-		err = pcistub_init_device(psdev->dev);
+		err = pcistub_init_device(psdev);
 		if (err) {
 			dev_err(&psdev->dev->dev,
 				"error %d initializing device\n", err);
@@ -565,7 +597,7 @@ static int pcistub_seize(struct pci_dev *dev,
 		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
 
 		/* don't want irqs disabled when calling pcistub_init_device */
-		err = pcistub_init_device(psdev->dev);
+		err = pcistub_init_device(psdev);
 
 		spin_lock_irqsave(&pcistub_devices_lock, flags);
 
diff --git a/include/uapi/xen/privcmd.h b/include/uapi/xen/privcmd.h
index 8b8c5d1420fe..220e7670a113 100644
--- a/include/uapi/xen/privcmd.h
+++ b/include/uapi/xen/privcmd.h
@@ -126,6 +126,11 @@ struct privcmd_ioeventfd {
 	__u8 pad[2];
 };
 
+struct privcmd_gsi_from_dev {
+	__u32 sbdf;
+	int gsi;
+};
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: &privcmd_hypercall_t
@@ -157,5 +162,7 @@ struct privcmd_ioeventfd {
 	_IOW('P', 8, struct privcmd_irqfd)
 #define IOCTL_PRIVCMD_IOEVENTFD					\
 	_IOW('P', 9, struct privcmd_ioeventfd)
+#define IOCTL_PRIVCMD_GSI_FROM_DEV				\
+	_IOC(_IOC_NONE, 'P', 10, sizeof(struct privcmd_gsi_from_dev))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/include/xen/acpi.h b/include/xen/acpi.h
index 9b50027113f3..0bf5f4884456 100644
--- a/include/xen/acpi.h
+++ b/include/xen/acpi.h
@@ -83,4 +83,6 @@ int xen_acpi_get_gsi_info(struct pci_dev *dev,
 						  int *gsi_out,
 						  int *trigger_out,
 						  int *polarity_out);
+
+int pcistub_get_gsi_from_sbdf(unsigned int sbdf);
 #endif	/* _XEN_ACPI_H */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 07:09:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:09:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721941.1125655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78lc-0007M3-Vd; Wed, 15 May 2024 07:09:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721941.1125655; Wed, 15 May 2024 07:09:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78lc-0007Lw-Sr; Wed, 15 May 2024 07:09:36 +0000
Received: by outflank-mailman (input) for mailman id 721941;
 Wed, 15 May 2024 07:09:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s78lb-0007Ig-EV
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:09:35 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1460df76-128a-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:09:33 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 5FF4A4EE0738;
 Wed, 15 May 2024 09:09:32 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1460df76-128a-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Wed, 15 May 2024 09:09:32 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 bertrand.marquis@arm.com, julien@xen.org, Simone Ballarin
 <simone.ballarin@bugseng.com>, Doug Goldstein <cardoe@cardoe.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: Re: [XEN PATCH 03/10] automation/eclair_analysis: deviate macro
 count_args_ for MISRA Rule 20.7
In-Reply-To: <alpine.DEB.2.22.394.2405011254290.497719@ubuntu-linux-20-04-desktop>
References: <cover.1713885065.git.nicola.vetrini@bugseng.com>
 <7de407c218f0911e28b7c3f609a55636165166a8.1713885065.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2404241727520.3940@ubuntu-linux-20-04-desktop>
 <5b675fcdf688723bc6d4ea260d10004e@bugseng.com>
 <alpine.DEB.2.22.394.2405011254290.497719@ubuntu-linux-20-04-desktop>
Message-ID: <006e9d571de6383679ab7fdb84dfcdd7@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-01 21:54, Stefano Stabellini wrote:
> On Mon, 29 Apr 2024, Nicola Vetrini wrote:
>> On 2024-04-25 02:28, Stefano Stabellini wrote:
>> > On Tue, 23 Apr 2024, Nicola Vetrini wrote:
>> > > The count_args_ macro violates Rule 20.7, but it can't be made
>> > > compliant with Rule 20.7 without breaking its functionality. Since
>> > > it's very unlikely for this macro to be misused, it is deviated.
>> >
>> > That is OK but can't we use the SAF- framework to do it, given that it
>> > is just one macro?
>> >
>> > If not, this is also OK.
>> >
>> >
>> 
>> It would be more fragile, for no substantial gain
> 
> OK
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Hi,

I think this patch slipped through the cracks. I see it only has 
Stefano's R-by, so perhaps it needs a further ack?

Thanks,

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:15:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:15:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721945.1125666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78rJ-0000pq-N1; Wed, 15 May 2024 07:15:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721945.1125666; Wed, 15 May 2024 07:15:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s78rJ-0000pj-Jq; Wed, 15 May 2024 07:15:29 +0000
Received: by outflank-mailman (input) for mailman id 721945;
 Wed, 15 May 2024 07:15:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s78rI-0000pU-5N
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:15:28 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e70f3dfd-128a-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:15:26 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso109833366b.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 00:15:26 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17808sm818515166b.217.2024.05.15.00.15.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 00:15:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e70f3dfd-128a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715757326; x=1716362126; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=AqjT/Vw8JQwJFf9t9q08Z/+6wDrjcT15ZWT0eOsy1mo=;
        b=G4i4KkFqiWblnLXVi02IM3KAdpxITAVz7Y+3RupL3+OMqtDSeYz63brdfJhRyKUC7C
         zN3MdiqNwpMF8jdnsvYSUwEsk9cIphjDFQ6svU+o0BQqSFMPrUV91MpcOkSlQyFlYt/M
         UDl++t/qPuncto7osgnzUdNpNH4ajXcsh2jQndmyKRk4KLW82LClTvbtyIfsoXAb/5YM
         VtaAkO8nsYopsnqe6iNjWYF1hZdbv2V8PjihqVuHMDStxv8KX68YGnQJtcaG0qyXbZ7/
         LOmJF1xfY4JqBYEwBpOCZYwDk46ORssYQFt5vciS2TWg1/FVy5amLf8eNUq3bjyX5PSX
         CF+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715757326; x=1716362126;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=AqjT/Vw8JQwJFf9t9q08Z/+6wDrjcT15ZWT0eOsy1mo=;
        b=WDmM5GocyViDt9bDsqxu+c63VgRZrfQ/5CUGNxH0cv/Qxf3jAwZJ61oJ3A00RiTsn4
         7Yzfn7FVW0BR118+M4nBfxAi30zrikCstJ+iEPPDZkZhfd291jh4qSTMi1PVo8pAtwU3
         EgXcY22dEBFSFqA4q2rVxe3TdySQVb6LVsQg5uQV3dvaVHCRlAQy6nWgmwXqbl+ziK41
         YUk/vB9mfpGhfh4Y2rFgj1Tso6/UgIM3KY75O8xW/BunCzo0CYgqAYRSozzuTARfPtbx
         T2/AjaDSPrLV2qeBlifOQpHG+xVxpHlhkH5Pdr8aPgFBZ3LHQlsNM7K9kDattCvb5tM9
         ypPQ==
X-Gm-Message-State: AOJu0YyE0b1h9EBKNqNp0mXaRd6+vQtZM/U2x0aaSDCejCe+APBOjmYN
	hh4ct8pn5TlgHVpa9OPV8io4a2v7yPonmKP3VqALUTVrD8DkduR921cHCRD4Dw==
X-Google-Smtp-Source: AGHT+IEyyzYPWsBBdsqni4ZFE6NQulhlTyQ+K1YyQOV6qel+KJkKT4WcqGxPntiFKZgXI61UeVP+og==
X-Received: by 2002:a17:906:b346:b0:a5a:2d0d:2aef with SMTP id a640c23a62f3a-a5a2d55ef25mr1473080466b.21.1715757325782;
        Wed, 15 May 2024 00:15:25 -0700 (PDT)
Message-ID: <93b11c7f-a422-473e-ae0e-3f0a53ea0190@suse.com>
Date: Wed, 15 May 2024 09:15:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 03/10] automation/eclair_analysis: deviate macro
 count_args_ for MISRA Rule 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 bertrand.marquis@arm.com, julien@xen.org,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <cover.1713885065.git.nicola.vetrini@bugseng.com>
 <7de407c218f0911e28b7c3f609a55636165166a8.1713885065.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2404241727520.3940@ubuntu-linux-20-04-desktop>
 <5b675fcdf688723bc6d4ea260d10004e@bugseng.com>
 <alpine.DEB.2.22.394.2405011254290.497719@ubuntu-linux-20-04-desktop>
 <006e9d571de6383679ab7fdb84dfcdd7@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <006e9d571de6383679ab7fdb84dfcdd7@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 09:09, Nicola Vetrini wrote:
> On 2024-05-01 21:54, Stefano Stabellini wrote:
>> On Mon, 29 Apr 2024, Nicola Vetrini wrote:
>>> On 2024-04-25 02:28, Stefano Stabellini wrote:
>>>> On Tue, 23 Apr 2024, Nicola Vetrini wrote:
>>>>> The count_args_ macro violates Rule 20.7, but it can't be made
>>>>> compliant with Rule 20.7 without breaking its functionality. Since
>>>>> it's very unlikely for this macro to be misused, it is deviated.
>>>>
>>>> That is OK but can't we use the SAF- framework to do it, given that it
>>>> is just one macro?
>>>>
>>>> If not, this is also OK.
>>>>
>>>>
>>>
>>> It would be more fragile, for no substantial gain
>>
>> OK
>>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> I think this patch slipped through the cracks. I see it only has 
> Stefano's R-by, so perhaps it needs a further ack?

I don't think it does. All it needs is putting in. I guess I assumed
Stefano would be taking care of that, with your discussion (quickly)
settled.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721952.1125675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799F-0004MY-3b; Wed, 15 May 2024 07:34:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721952.1125675; Wed, 15 May 2024 07:34:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799F-0004MR-16; Wed, 15 May 2024 07:34:01 +0000
Received: by outflank-mailman (input) for mailman id 721952;
 Wed, 15 May 2024 07:34:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s799E-0004ML-0E
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:00 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7dfdebfe-128d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 09:33:58 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so135340766b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 00:33:58 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b178b1sm820131066b.191.2024.05.15.00.33.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 00:33:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7dfdebfe-128d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715758438; x=1716363238; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z6PonMRQvEhASBmXrBQYuUr+6RexroA7OI1t5pueh6U=;
        b=BP9uPE3LcnL5OsAW8SKa2gWmASp62vISAfGuHt5uPyH9yPpvjidytd6lMNPDkvaTa8
         89XD9u+LJuZLmiHdjP4edFPVE1tCPscMM+Ix1ekNnwLEFUUKk7kUrP29kfrz8Y+TaAd2
         uisoZJVfiiSh9BlhLWt1kNlgIqw7En7wod7Y9qXTkvuXj0zzx44sH0/tkJvIQireOnw5
         Dkljz9HWLsthdEa5aBhciVtsgvvE6/I3pKndnZquP1hZRNu9I3ZHEnwHBT51I8QGq+nx
         UuRnlPUchepLys87ymXe7Vfvx8e7hu5YGtSCw86sQCoF15njXjRmUWm5UWJ/vH/MNx+t
         /T+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715758438; x=1716363238;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z6PonMRQvEhASBmXrBQYuUr+6RexroA7OI1t5pueh6U=;
        b=g4B/mzTRScYa9CwDUiQjc3yVTPrBm2bgmEZgUS4t1PPiHZH85UlM72/v+6rMjbWBrf
         mS5o3+yFqREdS8QQ+zglGwW1llYSSBAxUBtvOJ0VyLtEQIRzn3fOg0hm1Qc6RQut0fqE
         9fe1Ac6q5wvqDonXKsthorBgpgTcjZr92ww3/TwJF3jRtuYQvli9WzNoSJNasGgSFgrK
         MMLV/EOvLrKhr5cNaurR9CVxg4y5LNSLk5HQq6Nmjj0GJaOGeRJHi3qwozRpnaIWDD1K
         Qu81R0pXPK3IeCVv6ZuXuwPzpkRbsD+6sJFDQpv2QCua0rz4jCTvV5ZpHRtPc40BdB6u
         Cg9g==
X-Forwarded-Encrypted: i=1; AJvYcCWmeKCcSAMF8YHzFmmXk91nle62h0vRXcRZbNfSbLFuKO4U9ztqtg/d1iWwj22us5IeV40PRDb8+mjzbkJwO+usR8czrIBTVHxkdIb5I2I=
X-Gm-Message-State: AOJu0Yz9O3XCuLziWa788uzswSnvPuIP6auzE+mNc7KDDjCuPbRSPKXa
	zxwmqjDpa0gacUMxoH8PXMpWxX2oKt3eIdfQDSSqNomJBmMT/uObRcZpDJMAqA==
X-Google-Smtp-Source: AGHT+IE/83SHFOdlL9UQghm8SQqKA1YYYHDsZ4Wl22oQF96QFMrT7y8EeRZqmQmU5oECakmouF8/rg==
X-Received: by 2002:a17:906:2b85:b0:a59:9f4e:4e3b with SMTP id a640c23a62f3a-a5a2d53ae89mr1026769066b.2.1715758438068;
        Wed, 15 May 2024 00:33:58 -0700 (PDT)
Message-ID: <a00b26e0-cf65-44fb-8d74-630bfdb20dfe@suse.com>
Date: Wed, 15 May 2024 09:33:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Daniel Smith <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, Julien Grall <julien@xen.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
 <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
 <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
 <alpine.DEB.2.22.394.2405141411210.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405141411210.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 23:35, Stefano Stabellini wrote:
> On Tue, 14 May 2024, Julien Grall wrote:
>> On 14/05/2024 11:03, Jan Beulich wrote:
>>> On 14.05.2024 11:51, Andrew Cooper wrote:
>>>> You tried defending breaking a utility with "well it shouldn't exist
>>>> then".
>>>>
>>>> You don't have a leg to stand on, and two maintainers of relevant
>>>> subsystems here just got tired of bullshit being presented in place of
>>>> any credible argument for having done the change in the way you did.
>>>
>>> Please can you finally get into the habit of not sending rude replies?
>>>
>>>> The correct response was "Sorry I broke things.  Lets revert this for
>>>> now to unbreak, and I'll see about reworking it to not intentionally
>>>> subvert Xen's security mechanism".
>>>
>>> I'm sorry, but I didn't break things. I made things more consistent with
>>> the earlier change, as pointed out before: With your revert,
>>> evtchn_status() is now (again) inconsistent with e.g. evtchn_send(). If
>>> you were serious about this being something that needs leaving to XSM,
>>> you'd have adjusted such further uses of consumer_is_xen() as well. But
>>> you aren't. You're merely insisting on lsevtchn needing to continue to
>>> work in a way it should never have worked, with a patch to improve the
>>> situation already pending.
>>>
>>> Just to state a very basic principle here again: Xen-internal event
>>> channels ought to either be fully under XSM control when it comes to
>>> domains attempting to access them (in whichever way), or they should
>>> truly be Xen-internal, with access uniformly prevented. To me the
>>> former option simply makes very little sense.
>>
>> I agree we need consistency on how we handle security policy event channel.
>> Although, I don't have a strong opinion on which way to go.
> 
> Same here
> 
> 
>> For the commit message, it is not entirely clear what "broke lseventch in
>> dom0" really mean. Is it lsevtchn would not stop or it will just not display
>> the event channel?
>>
>> If the former, isn't a sign that the tool needs to be harden a bit more? If
>> the latter, then I would argue that consistency for the XSM policy is more
>> important than displaying the event channel for now (the patch was also
>> committed 3 years ago...).
> 
> I realize 3 years have passed and it is a long time, but many
> downstreams (including some which are widely used) don't rebase
> regularly and we are still missing lots of tests from gitlab-ci. The
> unfortunate result is that it can take years to realize there is a
> breakage. We need more gitlab-ci (or OSSTest) tests.
> 
> 
>> So I would vote for a revert and, if desired, replacing with a patch that
>> would change the XSM policy consistently. Alternatively, the consistency
>> should be a blocker for Xen 4.19.
> 
> I am convinced by Daniel's argument here:
> 
> https://marc.info/?l=xen-devel&m=171215093102694

I particularly disagree with the "since it is access control and falls under
the purview of XSM" in there, without addressing my point regarding Xen-
internal resources. It is a fundamental hypervisor decision whether to leave
access to Xen-internal resources to XSM control. If that decision ended up
being "yes", then I agree XSM maintainers may ack a respective change. If
that decision as "no", though, acking would purely fall to REST for code
like what is being touched here.

Just to further clarify: If it was "yes" above, other Xen-internal resources
then also ought to be domain accessible based on XSM policy. I don't think
that's the case e.g. for Xen-private memory.

IOW I can't help the impression that both the patch and the ack were
provided looking at just the one special case, driven by the (perceived)
tool breakage (see below).

> https://marc.info/?l=xen-devel&m=171215073502479

In there he said in particular: "And it is incorrect because as again you
have not articulated why the lsevtchn behavior is wrong and thus whether
this is the valid corrective action." Daniel, did you even look at the code
when saying so? With the revert in place, lsevtchn is still going to fall
on the nose when XSM denies access to a particular channel. I didn't think
this needed calling out explicitly; the tool needs fixing.

> I would ack Andrew's revert. If we decide to revert Andrew's revert, I
> also think that we should make the alternative solution, whatever that
> might be, a blocker for Xen 4.19.
> 
> My favorite alternative solition is Daniel's suggestion of adding a
> check to the dummy XSM policy. I am not sure if this is the same thing
> you mean with "change the XSM policy consistently".

I don't think it would be, but I also don't know what exact check was
thought about. I think I was quite clear about evtchn_send()'s similar
code (there may be more). All of these want to behave the same: All
involving XSM, or all not doing so. This is the kind of thing where I
don't think any "majority" can trump technical aspects. If the verdict was
"XSM", then yes, my original patch would have moved us in the wrong
direction. But then a plain revert is insufficient, and the blaming in
there also should have been done at least differently, if at all.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721954.1125686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799r-0004ny-BD; Wed, 15 May 2024 07:34:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721954.1125686; Wed, 15 May 2024 07:34:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799r-0004nq-8T; Wed, 15 May 2024 07:34:39 +0000
Received: by outflank-mailman (input) for mailman id 721954;
 Wed, 15 May 2024 07:34:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s799p-0004ka-UU
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:37 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9493c0c2-128d-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:34:36 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 83BF24EE0738;
 Wed, 15 May 2024 09:34:35 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9493c0c2-128d-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 0/4] address violations of MISRA C Rule 20.7
Date: Wed, 15 May 2024 09:34:29 +0200
Message-Id: <cover.1715757982.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi all,

this series aims to refactor some macros that cause violations of MISRA C Rule
20.7 ("Expressions resulting from the expansion of macro parameters shall be
enclosed in parentheses"). All the macros touched by these patches are in some
way involved in violations, and the strategy adopted to bring them into
compliance is to add parentheses around macro arguments where needed.

Nicola Vetrini (4):
  x86/vpmu: address violations of MISRA C Rule 20.7
  x86/hvm: address violations of MISRA C Rule 20.7
  x86_64/uaccess: address violations of MISRA C Rule 20.7
  x86_64/cpu_idle: address violations of MISRA C Rule 20.7

 xen/arch/x86/cpu/vpmu_amd.c               | 4 ++--
 xen/arch/x86/hvm/mtrr.c                   | 2 +-
 xen/arch/x86/hvm/rtc.c                    | 2 +-
 xen/arch/x86/include/asm/hvm/save.h       | 2 +-
 xen/arch/x86/include/asm/x86_64/uaccess.h | 7 ++++---
 xen/arch/x86/x86_64/cpu_idle.c            | 2 +-
 6 files changed, 10 insertions(+), 9 deletions(-)

-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721955.1125696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799s-00052t-J2; Wed, 15 May 2024 07:34:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721955.1125696; Wed, 15 May 2024 07:34:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799s-00052m-GT; Wed, 15 May 2024 07:34:40 +0000
Received: by outflank-mailman (input) for mailman id 721955;
 Wed, 15 May 2024 07:34:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s799q-0004ka-US
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:38 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94e67609-128d-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:34:37 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 3FC664EE0745;
 Wed, 15 May 2024 09:34:36 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94e67609-128d-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 1/4] x86/vpmu: address violations of MISRA C Rule 20.7
Date: Wed, 15 May 2024 09:34:30 +0200
Message-Id: <ecba64bb8295fa27f0ddbb0905b0983a3572b1ae.1715757982.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715757982.git.nicola.vetrini@bugseng.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/arch/x86/cpu/vpmu_amd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
index db2fa420e14a..97e6315bd9f7 100644
--- a/xen/arch/x86/cpu/vpmu_amd.c
+++ b/xen/arch/x86/cpu/vpmu_amd.c
@@ -25,8 +25,8 @@
 
 #define is_guest_mode(msr) ((msr) & (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
 #define is_pmu_enabled(msr) ((msr) & (1ULL << MSR_F10H_EVNTSEL_EN_SHIFT))
-#define set_guest_mode(msr) (msr |= (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
-#define is_overflowed(msr) (!((msr) & (1ULL << (MSR_F10H_COUNTER_LENGTH-1))))
+#define set_guest_mode(msr) ((msr) |= (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
+#define is_overflowed(msr) (!((msr) & (1ULL << (MSR_F10H_COUNTER_LENGTH - 1))))
 
 static unsigned int __read_mostly num_counters;
 static const u32 __read_mostly *counters;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721956.1125701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799s-000567-TZ; Wed, 15 May 2024 07:34:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721956.1125701; Wed, 15 May 2024 07:34:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799s-000553-Oi; Wed, 15 May 2024 07:34:40 +0000
Received: by outflank-mailman (input) for mailman id 721956;
 Wed, 15 May 2024 07:34:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s799r-0004nt-Iu
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:39 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 959f3b2c-128d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 09:34:38 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 85EC14EE0747;
 Wed, 15 May 2024 09:34:37 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 959f3b2c-128d-11ef-909d-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 3/4] x86_64/uaccess: address violations of MISRA C Rule 20.7
Date: Wed, 15 May 2024 09:34:32 +0200
Message-Id: <b131427dc03c4b89141bab7648523b81bf5186a6.1715757982.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715757982.git.nicola.vetrini@bugseng.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

xlat_malloc_init is touched for consistency, despite the construct
being already deviated.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/arch/x86/include/asm/x86_64/uaccess.h | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/include/asm/x86_64/uaccess.h b/xen/arch/x86/include/asm/x86_64/uaccess.h
index ba79f950fba9..c6fa3fd381bc 100644
--- a/xen/arch/x86/include/asm/x86_64/uaccess.h
+++ b/xen/arch/x86/include/asm/x86_64/uaccess.h
@@ -26,15 +26,16 @@ void free_compat_arg_xlat(struct vcpu *v);
 #define xlat_page_start ((unsigned long)COMPAT_ARG_XLAT_VIRT_BASE)
 #define xlat_page_size  COMPAT_ARG_XLAT_SIZE
 #define xlat_page_left_size(xlat_page_current) \
-    (xlat_page_start + xlat_page_size - xlat_page_current)
+    (xlat_page_start + xlat_page_size - (xlat_page_current))
 
 #define xlat_malloc_init(xlat_page_current)    do { \
-    xlat_page_current = xlat_page_start; \
+    (xlat_page_current) = xlat_page_start; \
 } while (0)
 
 extern void *xlat_malloc(unsigned long *xlat_page_current, size_t size);
 
-#define xlat_malloc_array(_p, _t, _c) ((_t *) xlat_malloc(&_p, sizeof(_t) * _c))
+#define xlat_malloc_array(_p, _t, _c) ((_t *) xlat_malloc(&(_p), \
+                                                          sizeof(_t) * (_c)))
 
 /*
  * Valid if in +ve half of 48-bit address space, or above Xen-reserved area.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721957.1125705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799t-0005BH-8D; Wed, 15 May 2024 07:34:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721957.1125705; Wed, 15 May 2024 07:34:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799t-00059G-1d; Wed, 15 May 2024 07:34:41 +0000
Received: by outflank-mailman (input) for mailman id 721957;
 Wed, 15 May 2024 07:34:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s799r-0004ka-Uk
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:39 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9548029b-128d-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:34:37 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id DAE474EE0739;
 Wed, 15 May 2024 09:34:36 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9548029b-128d-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 2/4] x86/hvm: address violations of MISRA C Rule 20.7
Date: Wed, 15 May 2024 09:34:31 +0200
Message-Id: <6d14b3283005cf1a30c4fa24f9841586a41e2b1b.1715757982.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715757982.git.nicola.vetrini@bugseng.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/arch/x86/hvm/mtrr.c             | 2 +-
 xen/arch/x86/hvm/rtc.c              | 2 +-
 xen/arch/x86/include/asm/hvm/save.h | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
index 32f74c1db03b..1079851f70ed 100644
--- a/xen/arch/x86/hvm/mtrr.c
+++ b/xen/arch/x86/hvm/mtrr.c
@@ -16,7 +16,7 @@
 #include <public/hvm/e820.h>
 
 /* Get page attribute fields (PAn) from PAT MSR. */
-#define pat_cr_2_paf(pat_cr,n)  ((((uint64_t)pat_cr) >> ((n)<<3)) & 0xff)
+#define pat_cr_2_paf(pat_cr, n)  ((((uint64_t)(pat_cr)) >> ((n) << 3)) & 0xff)
 
 /* Effective mm type lookup table, according to MTRR and PAT. */
 static const uint8_t mm_type_tbl[MTRR_NUM_TYPES][X86_NUM_MT] = {
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index 4bb1c7505534..72c7bdbfcd02 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -45,7 +45,7 @@
 #define vrtc_domain(x) (container_of(x, struct pl_time, vrtc)->domain)
 #define vrtc_vcpu(x)   (pt_global_vcpu_target(vrtc_domain(x)))
 #define epoch_year     1900
-#define get_year(x)    (x + epoch_year)
+#define get_year(x)    ((x) + epoch_year)
 
 enum rtc_mode {
    rtc_mode_no_ack,
diff --git a/xen/arch/x86/include/asm/hvm/save.h b/xen/arch/x86/include/asm/hvm/save.h
index 8149aa113cb4..ec8de029319d 100644
--- a/xen/arch/x86/include/asm/hvm/save.h
+++ b/xen/arch/x86/include/asm/hvm/save.h
@@ -50,7 +50,7 @@ int _hvm_check_entry(struct hvm_domain_context *h,
                           HVM_SAVE_LENGTH(x), true) == 0 )      \
     {                                                           \
         ptr = &(h)->data[(h)->cur];                             \
-        h->cur += HVM_SAVE_LENGTH(x);                           \
+        (h)->cur += HVM_SAVE_LENGTH(x);                         \
     }                                                           \
     ptr; })
 
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:34:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:34:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721958.1125714 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799t-0005IJ-OH; Wed, 15 May 2024 07:34:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721958.1125714; Wed, 15 May 2024 07:34:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s799t-0005Eo-EK; Wed, 15 May 2024 07:34:41 +0000
Received: by outflank-mailman (input) for mailman id 721958;
 Wed, 15 May 2024 07:34:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s799s-0004nt-8d
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:34:40 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 95fcfef3-128d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 09:34:39 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 310834EE0748;
 Wed, 15 May 2024 09:34:38 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95fcfef3-128d-11ef-909d-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 4/4] x86_64/cpu_idle: address violations of MISRA C Rule 20.7
Date: Wed, 15 May 2024 09:34:33 +0200
Message-Id: <6c84f6f0bf93a00ab1acda2d39d31b1f3d5bf8aa.1715757982.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715757982.git.nicola.vetrini@bugseng.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/arch/x86/x86_64/cpu_idle.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/x86_64/cpu_idle.c b/xen/arch/x86/x86_64/cpu_idle.c
index fcd6fc0fc212..cc9febc03d60 100644
--- a/xen/arch/x86/x86_64/cpu_idle.c
+++ b/xen/arch/x86/x86_64/cpu_idle.c
@@ -93,7 +93,7 @@ long compat_set_cx_pminfo(uint32_t acpi_id,
             return -EFAULT; \
         guest_from_compat_handle(states, (_s_)->states); \
 \
-        for ( i = 0; i < _s_->count; i++ ) \
+        for ( i = 0; i < (_s_)->count; i++ ) \
         { \
            if ( unlikely(copy_from_guest_offset(&state, states, i, 1)) ) \
                return -EFAULT; \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 07:35:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:35:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721970.1125735 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79B4-0007SU-99; Wed, 15 May 2024 07:35:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721970.1125735; Wed, 15 May 2024 07:35:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79B4-0007SN-6U; Wed, 15 May 2024 07:35:54 +0000
Received: by outflank-mailman (input) for mailman id 721970;
 Wed, 15 May 2024 07:35:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s79B3-0007SA-GJ
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:35:53 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c1e66429-128d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 09:35:52 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso145094066b.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 00:35:52 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d2dcsm820694166b.44.2024.05.15.00.35.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 00:35:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1e66429-128d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715758552; x=1716363352; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lWSzj71CoUYpciKV+ThBycq38u3zBqG1EESkCslYEhE=;
        b=UzLFFvYaHeEnI2eIn5Us7koW4LFic+qCcGlDtkkYucCgGXwCU3O7nKRURZP+WxmQ/B
         ooK7Jg3NeBM25kHNILfSRuJtaogWyKftegc28ZSlHm8ieP48kFN8P44AnNSpsPn+AtLW
         j23BDE+/WjxmPy7SD+0oAKaT77LondFDvIU8jHBdB4uju80Nl92MQoFh+VRAiXbhBmHf
         IB5Os5DP+sRa66zdmEH6yRCGbsZQxGIAognMcCqLwYQvfwm7kNFsjdmnJcMPqn5CV2Jm
         zNSo5BDJ/WPNWzKc824i7ooCY9MEIZQePn9tFYginfqzLXY4bQrg3xknPVmIT73XV/iA
         GJvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715758552; x=1716363352;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lWSzj71CoUYpciKV+ThBycq38u3zBqG1EESkCslYEhE=;
        b=tWDdRRRkIjKyCiL09QCV3Pg38hLDtqOsQutdNFx3muOWw7zkHMTdWbwl5s5oSBfSnN
         bUELy+UwfcUbXk0wehbdumEhb1t1Wp4Go2XCUqwbF8uMVUd+AruUO9DaJ28hcCCI67zU
         HguduC8DPkkc1PA8YPw0majla/Zdv1YDpjWVwT72mlQaXbkO1Od7JWCDzA7A3VsEIvTc
         VVwikh8E27sWewEfvfy10vDHe9XVufla+aJ9mjzqVVGk9GTiL12WOb44ID/krHnYkVh7
         qaM2GQvdcqFncxvfw80tIvC3dcTqFMAjb5uJJeKabogCFv+RKW6p68CCXt5Og0NFf4sa
         cqLA==
X-Forwarded-Encrypted: i=1; AJvYcCViZSSXnniir5/BJelzz8J4ffHwExWE841yvOkEjR4/Tf5eHlZ6cjRKQvYBYgxWhkC1GMlambKTHdpK9GQluNxDIc9OVFyRNEp51Swn3t4=
X-Gm-Message-State: AOJu0Yw+ORoDcgx1aNExJcfrt5Pxy1fa5AS0RIjj3PbKvzWkbilriaMc
	GVIbMVV80ntoq9ZUPQbv75pSvLIXWa+HHphVEOJMFtID7tQsTGWuYwE7AnX8Rg==
X-Google-Smtp-Source: AGHT+IF1OrqJ5l2SNwEbkVug/7e2pgUHNcvJWTks/0smXfwL0q5G2lCN70yH/bigXODJfNswmNGjkg==
X-Received: by 2002:a17:906:d935:b0:a59:d243:945 with SMTP id a640c23a62f3a-a5a2d66a888mr955341666b.57.1715758552072;
        Wed, 15 May 2024 00:35:52 -0700 (PDT)
Message-ID: <bd1072c5-1533-46b7-b6c8-fab1c0f80ab3@suse.com>
Date: Wed, 15 May 2024 09:35:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/misra: add D4.12
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Julien Grall <julien@xen.org>,
 Michal Orzel <michal.orzel@amd.com>, roberto.bagnara@bugseng.com,
 consulting@bugseng.com, xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405141611170.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405141611170.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 01:15, Stefano Stabellini wrote:
> Add D4.12 with the same explanation as the rules of the R21 series.
> D4.12 refers to the standard library memory allocation functions and
> similar third party libraries with memory allocation functions. It
> doesn't refer to the in-tree implementation we have in Xen which is
> subject to MISRA C rules and MISRA C scanning.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> 
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index 80e5e972ad..bc8506add4 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -76,6 +76,11 @@ maintainers if you want to suggest a change.
>         considered libraries from MISRA C point of view as they are
>         imported in source form)
>  
> +   * - `Dir 4.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/D_04_12.c>`_
> +     - Required
> +     - Dynamic memory allocation shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_

I'm having trouble connecting this remark with the directive. We do have
dynamic memory allocation routines, and we use them. It doesn't really
matter that they don't come from an external library, does it?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 07:48:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:48:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721980.1125746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79NR-0001NZ-BO; Wed, 15 May 2024 07:48:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721980.1125746; Wed, 15 May 2024 07:48:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79NR-0001NS-8i; Wed, 15 May 2024 07:48:41 +0000
Received: by outflank-mailman (input) for mailman id 721980;
 Wed, 15 May 2024 07:48:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s79NQ-0001NM-28
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:48:40 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8a1bdf34-128f-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:48:38 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59b097b202so98903666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 00:48:37 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f78sm816140166b.133.2024.05.15.00.48.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 00:48:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a1bdf34-128f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715759317; x=1716364117; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XAeOWAQ85i6Y/taxO0T1QDkrC2eLuexqW6xx/aGJLQ8=;
        b=Ldqju7mvR/GYuUilw9wIbVKlUaT7zvOU/DWJTnql945gEfDXHTC9uBt8imNxBFL+Mu
         jbSjJjzszCYG0flFPq+yYH44kibPXKRewDAW2OcAyOtmgFZFa90YhzuhPo+2nrh/Yn0o
         tbqcLi2qdf/jYelRbGPUrvSISwzmVY0ai0bl817uCdqn5+mkOFI5GbG+N1zgkETmoKJh
         OEJuKIXHWj6DMlXFNkjXF9mp9iQ3YXJ2YOLZFovFMoJSigoY3S4FFrA0d4JwYOeLelS8
         Fx3AWcPg4i6AUnbiaGKSepK92m/Ep4sCPFZTjV4Ebx9JEa6RqwZxn6wPNycjPlfN0pqY
         9wRw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715759317; x=1716364117;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XAeOWAQ85i6Y/taxO0T1QDkrC2eLuexqW6xx/aGJLQ8=;
        b=a5ajB8vqxwUMYHNkcg0mYU7pGClRUNFbgtriQZhqGzDP4oyxyAE7MDWCi4JYhZjI5h
         QydjynWIqj4brv24k29FYRbBHHeHYEX3xVpyZ13h0QCC1UuuVz+xAHNgWnfspnkVzhV2
         sBFFwOWQXq/hdaE+U99gQ3LBP/kuqzPCkHXQ3SxicIhJlOUAIsyB2QbdZKh4/zlbBtqJ
         YCS4BVH5C1xPFnZRY/GrMo4sKehUkBg5paE7H4bQ5rKX3d/3v4CrUAzILV9SbMYnFdbh
         X3JO7JYx6PQIpme/GATp094ZiAbI2Puu++T83XT6KfLchThSDz+LmiVGGRnF+R2H5UL9
         fm3w==
X-Forwarded-Encrypted: i=1; AJvYcCXK/I7PA8qiFJf6Ws0EjtwYn9z6VGOuG7LsTeqywXRfbRQFBC3L92o2rQPyWmO2i3REH8APfb+KacZ6LtAEkKvPNV1yVUij2ceaS6rc2PU=
X-Gm-Message-State: AOJu0YwSKj7Qu/toMzi3fF6DnHWO3DmvUdIRTxQPjEsyKOQ5Vg4Ar78L
	ZDRidPvNnrOHn5JU3eKNJPvReskYNuowYyvd2c9d/zqqV8/L/Z7FXiQc67hKEw==
X-Google-Smtp-Source: AGHT+IGdDTmlkSZQCynwqbh9Q7+7UpEDgBPpiV+ImH46ZJfFNCwAqvE6BbKImryx/ZIVTEGQWljRRw==
X-Received: by 2002:a17:906:da8d:b0:a59:a112:add2 with SMTP id a640c23a62f3a-a5a2d673908mr1294427666b.69.1715759317414;
        Wed, 15 May 2024 00:48:37 -0700 (PDT)
Message-ID: <c67c1b8b-e14b-4c30-a381-1b89aedcddb9@suse.com>
Date: Wed, 15 May 2024 09:48:36 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 0/4] address violations of MISRA C Rule 20.7
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>, xen-devel@lists.xenproject.org
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cover.1715757982.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Oleksii,

On 15.05.2024 09:34, Nicola Vetrini wrote:
> Hi all,
> 
> this series aims to refactor some macros that cause violations of MISRA C Rule
> 20.7 ("Expressions resulting from the expansion of macro parameters shall be
> enclosed in parentheses"). All the macros touched by these patches are in some
> way involved in violations, and the strategy adopted to bring them into
> compliance is to add parentheses around macro arguments where needed.
> 
> Nicola Vetrini (4):
>   x86/vpmu: address violations of MISRA C Rule 20.7
>   x86/hvm: address violations of MISRA C Rule 20.7
>   x86_64/uaccess: address violations of MISRA C Rule 20.7
>   x86_64/cpu_idle: address violations of MISRA C Rule 20.7

for 4.18 we took a relaxed approach towards (simple) changes for Misra purposes.
I wonder whether you mean to permit the same for 4.19, or whether series like
this one rather want/need delaying until after branching.

Jan

>  xen/arch/x86/cpu/vpmu_amd.c               | 4 ++--
>  xen/arch/x86/hvm/mtrr.c                   | 2 +-
>  xen/arch/x86/hvm/rtc.c                    | 2 +-
>  xen/arch/x86/include/asm/hvm/save.h       | 2 +-
>  xen/arch/x86/include/asm/x86_64/uaccess.h | 7 ++++---
>  xen/arch/x86/x86_64/cpu_idle.c            | 2 +-
>  6 files changed, 10 insertions(+), 9 deletions(-)
> 



From xen-devel-bounces@lists.xenproject.org Wed May 15 07:52:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 07:52:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721983.1125756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79Qk-0003Ud-Q3; Wed, 15 May 2024 07:52:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721983.1125756; Wed, 15 May 2024 07:52:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79Qk-0003UW-My; Wed, 15 May 2024 07:52:06 +0000
Received: by outflank-mailman (input) for mailman id 721983;
 Wed, 15 May 2024 07:52:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=opSn=MS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s79Qj-0003UQ-5G
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 07:52:05 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0473f657-1290-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 09:52:03 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id DC49F4EE0738;
 Wed, 15 May 2024 09:52:01 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0473f657-1290-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Subject: [XEN PATCH] automation/eclair_analysis: fully deviate MISRA C Rules 21.9 and 21.10
Date: Wed, 15 May 2024 09:51:59 +0200
Message-Id: <6990027e12e108a2eaee7300931b1dd01c30795d.1715758910.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These rules are concerned with the use of facilities provided by the
C Standard Library (qsort, bsearch for rule 21.9, and those provided
by <time.h> for rule 21.10).

Xen provides in its source code its own implementation of some of these
functions and macros, therefore a justification is provided for allowing
uses of these functions in the project.

The rules are also marked as clean as a consequence.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 .../eclair_analysis/ECLAIR/deviations.ecl      | 14 ++++++++++++++
 .../eclair_analysis/ECLAIR/monitored.ecl       |  2 ++
 automation/eclair_analysis/ECLAIR/tagging.ecl  |  2 +-
 docs/misra/deviations.rst                      | 18 ++++++++++++++++++
 4 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index 5eb7368a7322..1478c64a5af1 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -477,6 +477,20 @@ leads to a violation of the Rule are deviated."
 -config=MC3R1.R20.12,macros+={deliberate, "name(GENERATE_CASE)&&loc(file(deliberate_generate_case))"}
 -doc_end
 
+#
+# Series 21.
+#
+
+-doc_begin="Xen does not use the functions provided by the Standard Library, but
+implements a set of functions that share the same names as their Standard Library equivalent.
+The implementation of these functions is available in source form, so the undefined, unspecified
+or implementation-defined behaviors contemplated by the C Standard do not apply.
+If some undefined or unspecified behavior does arise in the implementation, it
+falls under the jurisdiction of other MISRA rules."
+-config=MC3R1.R21.9,reports+={deliberate, "any()"}
+-config=MC3R1.R21.10,reports+={deliberate, "any()"}
+-doc_end
+
 #
 # General
 #
diff --git a/automation/eclair_analysis/ECLAIR/monitored.ecl b/automation/eclair_analysis/ECLAIR/monitored.ecl
index 69308ea51c01..9da709dc889c 100644
--- a/automation/eclair_analysis/ECLAIR/monitored.ecl
+++ b/automation/eclair_analysis/ECLAIR/monitored.ecl
@@ -36,12 +36,14 @@
 -enable=MC3R1.R20.4
 -enable=MC3R1.R20.9
 -enable=MC3R1.R2.1
+-enable=MC3R1.R21.10
 -enable=MC3R1.R21.13
 -enable=MC3R1.R21.17
 -enable=MC3R1.R21.18
 -enable=MC3R1.R21.19
 -enable=MC3R1.R21.20
 -enable=MC3R1.R21.21
+-enable=MC3R1.R21.9
 -enable=MC3R1.R2.2
 -enable=MC3R1.R22.2
 -enable=MC3R1.R22.4
diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index a14bda5033df..acea15f486a1 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 161134771c77..b5450c38f66c 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -411,6 +411,24 @@ Deviations related to MISRA C:2012 Rules:
        construct is deviated only in Translation Units that present a violation
        of the Rule due to uses of this macro.
      - Tagged as `deliberate` for ECLAIR.
+     
+   * - R21.9
+     - Xen does not use the `bsearch` and `qsort` functions provided by the C
+       Standard Library, but provides in source form its own implementation,
+       therefore any unspecified or undefined behavior associated to the
+       functions provided by the Standard Library does not apply. Any such
+       behavior that may exist in such functions is therefore under the
+       jurisdiction of other MISRA C rules.
+     - Project-wide deviation, tagged as `deliberate` for ECLAIR.
+
+   * - R21.10
+     - Xen does not use the facilities provided by the `\<time.h\>` provided by the C
+       Standard Library, but provides in source form its own implementation,
+       therefore any unspecified, undefined or implementation-defined behavior
+       associated to the functions provided by the Standard Library does not
+       apply. Any such behavior that may exist in such functions is therefore
+       under the jurisdiction of other MISRA C rules.
+     - Project-wide deviation, tagged as `deliberate` for ECLAIR.
 
 Other deviations:
 -----------------
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 08:11:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 08:11:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.721998.1125766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79jn-0007Y1-UA; Wed, 15 May 2024 08:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 721998.1125766; Wed, 15 May 2024 08:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s79jn-0007Xu-Ov; Wed, 15 May 2024 08:11:47 +0000
Received: by outflank-mailman (input) for mailman id 721998;
 Wed, 15 May 2024 08:11:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s79jm-0007Vf-Lu; Wed, 15 May 2024 08:11:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s79jm-0004FI-HE; Wed, 15 May 2024 08:11:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s79jm-00048u-5M; Wed, 15 May 2024 08:11:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s79jm-0007SH-4o; Wed, 15 May 2024 08:11:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nVZnygFN4wCvboLgfao5V98VriavpqPajRZGPpkk7xk=; b=zOFvVb0BXd+WiaNl8A6OvEMz/8
	7LCOpg0YELdDRLWjhg4IaIKuM7Garx35/+BjT9UPn2AZBX46BxIY9T9aUtzzTNoHywEwdwc+K1Qx8
	pAv/SAETY/gCX7NCF+p9h4QYgOalqX2CHvtU0slQK+YGqNYHLI/dDmp1EPbRMjsFcY7Q=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186001-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186001: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 08:11:46 +0000

flight 186001 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186001/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185993
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185993
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185993
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185993
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185993
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185993
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   186001  2024-05-15 01:51:54 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed May 15 08:53:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 08:53:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722012.1125775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ANU-00053j-Uz; Wed, 15 May 2024 08:52:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722012.1125775; Wed, 15 May 2024 08:52:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ANU-00053c-S0; Wed, 15 May 2024 08:52:48 +0000
Received: by outflank-mailman (input) for mailman id 722012;
 Wed, 15 May 2024 08:52:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ANT-00053W-Is
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 08:52:47 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f4f25ec-1298-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 10:52:45 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-574d1a1c36aso1398426a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 01:52:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01968sm829902066b.166.2024.05.15.01.52.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 01:52:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f4f25ec-1298-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715763165; x=1716367965; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=usjtIK6sqrUaoYx074lh8bs5SFfkVgAx2UHaIyHmabk=;
        b=ATxtK8tKseQ67EQ+jU+5B5dw8U+Ip5e3FtDNledRDMF0CpWr4GozdFjLHU8xlXfTx6
         cxpSTOrNuspVnlj2rMDblrR6anoA0uAst1jLqaWR4P32Qe9x0xM1DEZ8TiVuJMGUvrA+
         ND49bDDCa2HLjD+4jUWY1kqL7RsvqRTUAV4jCl3rdfnp8Wd4g61JdE9YODccubYUvTHH
         6ZdbW93hML+0VnwsEGJpejEJykVZzCewRuiMXbo7kknbXVPiYAI9Xw7vKFU8mSWb6hFJ
         w8Wf11ng+mm2qrG4BKdhslKwyPwQesttZpUTyjxl7RpxeBjo/mzUVPhHF5mvFkn/Srgb
         1Mtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715763165; x=1716367965;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=usjtIK6sqrUaoYx074lh8bs5SFfkVgAx2UHaIyHmabk=;
        b=Pe+4AQpTjble22G6n25Rd0fcLy2JEvhzMCISS+Fpd/4g7LuZXGdImkv9bxbEJc2yZZ
         fkNaoAuDP6wyKKgitUmLNBjzhZp+cGUsuV5ogzc4Or17tHISbdQmAQG6q9a28GbeG5Ne
         ldZ4pEiJZXayZXC1KAH2LdLy/yLU5i9vcEhZnm8qDkw9GerY4jAfxTQ1yAR5+xKo2XL+
         yG9X7tQNdTsu8tBPZjUcwdSYrxnJk5iicCThC0LewCii6/fOzm1HJlIFgo0VVpDP/ILv
         ua4ToTHHnOs3RjsoBnCig3NSl4QbMYjCrc0lBB85ClpSaieoPmJRoLK0qHeJngJPokJE
         JFmg==
X-Forwarded-Encrypted: i=1; AJvYcCUdPFkGj/8b/fo7cFqkBLgR+QnKIcni/jlvNNkP4anyVtgd6GRyn58RVkT+hJ0YI8ccKcjcr3ATCeDlkfv4CmD0DVj/FQ8lI+jJmi9y66Y=
X-Gm-Message-State: AOJu0Ywr762s0ejUgEGNSJcvieOcYBqK5Bo6ZgZojvfsUiN5wymKZxtf
	f7Tt4w4l7iahl4oo0fsp/nPnQOUQ1uGBSeuQjqRy7mwBZqS/zJD9MTpDZ3oaKQ==
X-Google-Smtp-Source: AGHT+IGoZLkLN7hkv0qzmpMTyLOeTGFb8VrgBJGTIvAnHKnXVk6tc6XPs28aas6wUuHj1f2uJPRMXw==
X-Received: by 2002:a17:907:76d3:b0:a59:c7d7:8b0f with SMTP id a640c23a62f3a-a5a2d57a717mr931264966b.29.1715763164751;
        Wed, 15 May 2024 01:52:44 -0700 (PDT)
Message-ID: <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
Date: Wed, 15 May 2024 10:52:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 12:15, Oleksii Kurochko wrote:
> The following generic functions were introduced:
> * test_bit
> * generic__test_and_set_bit
> * generic__test_and_clear_bit
> * generic__test_and_change_bit
> 
> Also, the patch introduces the following generics which are
> used by the functions mentioned above:
> * BITOP_BITS_PER_WORD
> * BITOP_MASK
> * BITOP_WORD
> * BITOP_TYPE
> 
> These functions and macros can be useful for architectures
> that don't have corresponding arch-specific instructions.

Logically this paragraph may better move ahead of the BITOP_* one.

> Because of that x86 has the following check in the macros test_bit(),
> __test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit():
>     if ( bitop_bad_size(addr) ) __bitop_bad_size();
> It was necessary to make bitop bad size check generic too, so
> arch_check_bitop_size() was introduced.

Not anymore, with the most recent adjustments? There's nothing arch-
specific anymore in the checking.

> @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
>   * If two examples of this operation race, one can appear to succeed
>   * but actually fail.  You must protect multiple accesses with a lock.
>   */
> -static inline int __test_and_set_bit(int nr, void *addr)
> +static inline int arch__test_and_set_bit(int nr, volatile void *addr)

I think I raised this point before: Why arch__ here, ...

> @@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
>   * If two examples of this operation race, one can appear to succeed
>   * but actually fail.  You must protect multiple accesses with a lock.
>   */
> -static inline int __test_and_clear_bit(int nr, void *addr)
> +static inline int arch__test_and_clear_bit(int nr, volatile void *addr)

... here, ...

> @@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
>  
>      return oldbit;
>  }
> -#define __test_and_clear_bit(nr, addr) ({               \
> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> -    __test_and_clear_bit(nr, addr);                     \
> -})
> +#define arch__test_and_clear_bit arch__test_and_clear_bit
>  
>  /* WARNING: non atomic and it can be reordered! */
> -static inline int __test_and_change_bit(int nr, void *addr)
> +static inline int arch__test_and_change_bit(int nr, volatile void *addr)

... and here, while ...

> @@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
>      return oldbit;
>  }
>  
> -#define test_bit(nr, addr) ({                           \
> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> +#define arch_test_bit(nr, addr) ({                      \

... just arch_ here? I don't like the double underscore infixes very
much, but I'm okay with them as long as they're applied consistently.

> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -65,10 +65,144 @@ static inline int generic_flsl(unsigned long x)
>   * scope
>   */
>  
> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
> +
> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> +
> +extern void __bitop_bad_size(void);
> +
>  /* --------------------- Please tidy above here --------------------- */
>  
>  #include <asm/bitops.h>
>  
> +#ifndef arch_check_bitop_size
> +
> +#define bitop_bad_size(addr) sizeof(*(addr)) < sizeof(bitop_uint_t)

Nit: Missing parentheses around the whole expression.

> +#define arch_check_bitop_size(addr) \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size();

Apart from the arch_ prefix that now wants dropping, this macro (if we
want one in the first place) also wants writing in a way such that it
can be used as a normal expression, without double semicolons or other
anomalies (potentially) resulting at use sites. E.g.

#define check_bitop_size(addr) do { \
    if ( bitop_bad_size(addr) )     \
        __bitop_bad_size();         \
} while ( 0 )

> +#endif /* arch_check_bitop_size */
> +
> +/**
> + * generic__test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline bool
> +generic__test_and_set_bit(unsigned long nr, volatile void *addr)

The original per-arch functions all use "int" for their first parameter.
Here you use unsigned long, without any mention in the description of the
potential behavioral change. Which is even worse given that then x86 ends
up inconsistent with Arm and PPC in this regard, by ...

> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old | mask;
> +    return (old & mask);
> +}
> +
> +/**
> + * generic__test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline bool
> +generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old & ~mask;
> +    return (old & mask);
> +}
> +
> +/* WARNING: non atomic and it can be reordered! */
> +static always_inline bool
> +generic__test_and_change_bit(unsigned long nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old ^ mask;
> +    return (old & mask);
> +}
> +/**
> + * generic_test_bit - Determine whether a bit is set
> + * @nr: bit number to test
> + * @addr: Address to start counting from
> + */
> +static always_inline bool generic_test_bit(int nr, const volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +
> +    return (*p & mask);
> +}
> +
> +static always_inline bool
> +__test_and_set_bit(unsigned long nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_set_bit
> +#define arch__test_and_set_bit generic__test_and_set_bit
> +#endif
> +
> +    return arch__test_and_set_bit(nr, addr);

... silently truncating and sign-converting nr here.

As to generic_test_bit() - please don't cast away const-ness there.

> +}
> +#define __test_and_set_bit(nr, addr) ({             \
> +    arch_check_bitop_size(addr);                    \
> +    __test_and_set_bit(nr, addr);                   \
> +})
> +
> +static always_inline bool
> +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)

Oddly enough here at least you use bitop_uint_t, but that's still ...

> +{
> +#ifndef arch__test_and_clear_bit
> +#define arch__test_and_clear_bit generic__test_and_clear_bit
> +#endif
> +
> +    return arch__test_and_clear_bit(nr, addr);

... meaning a signedness conversion on x86 then. And beware: You can't
simply change x86'es code to use bitop_uint_t. The underlying insns used
interpret the bit position as a signed number, i.e. permitting accesses
below the incoming pointer (whether it really makes sense to be that way
is a separate question). I'm afraid I have no good suggestion how to deal
with that: Any approach I can think of is either detrimental to the
generic implementation or would have unwanted effects on the x86 one.
Others, anyone?

> --- a/xen/include/xen/types.h
> +++ b/xen/include/xen/types.h
> @@ -64,6 +64,12 @@ typedef __u64 __be64;
>  
>  typedef unsigned int __attribute__((__mode__(__pointer__))) uintptr_t;
>  
> +#ifndef BITOP_TYPE
> +#define BITOP_BITS_PER_WORD 32
> +
> +typedef uint32_t bitop_uint_t;
> +#endif

I think you mentioned to me before why this needs to live here, not in
xen/bitops.h. Yet I don't recall the reason, and the description (hint,
hint) doesn't say anything either.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 08:58:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 08:58:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722014.1125786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ASU-0005do-Gv; Wed, 15 May 2024 08:57:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722014.1125786; Wed, 15 May 2024 08:57:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ASU-0005dh-E0; Wed, 15 May 2024 08:57:58 +0000
Received: by outflank-mailman (input) for mailman id 722014;
 Wed, 15 May 2024 08:57:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AST-0005db-GE
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 08:57:57 +0000
Received: from pb-smtp20.pobox.com (pb-smtp20.pobox.com [173.228.157.52])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 36f5f391-1299-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 10:57:54 +0200 (CEST)
Received: from pb-smtp20.pobox.com (unknown [127.0.0.1])
 by pb-smtp20.pobox.com (Postfix) with ESMTP id 7A0CB1BA5F;
 Wed, 15 May 2024 04:57:52 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp20.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp20.pobox.com (Postfix) with ESMTP id 710C81BA5E;
 Wed, 15 May 2024 04:57:52 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp20.pobox.com (Postfix) with ESMTPSA id D29581BA5D;
 Wed, 15 May 2024 04:57:48 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36f5f391-1299-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:mime-version:content-transfer-encoding;
	 s=sasl; bh=u9TIevOX68nz9tHM5jazPM24+VdTJI7U4Q4KeYnwHgE=; b=QJsq
	68m/oPobnWqYvsVuZlcxWgBCEm5FFb03ZbpkqS0hHGeST5ySNHcudypgJkBfAIgv
	tNhZWuhELqlVf5nZ14FyIJpuOfjssADdLUQPyCRB5frTrjZ3KRk63EIenrqm1GOL
	iF8WVdZg0Xae4mr/eMQO71j/a+qnVwNKOEjJ+Ec=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Paul Durrant <paul@xen.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 00/15] x86: make cpu virtualization support configurable
Date: Wed, 15 May 2024 11:57:45 +0300
Message-Id: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
X-Pobox-Relay-ID:
 3450D298-1299-11EF-AAB3-F515D2CDFF5E-90055647!pb-smtp20.pobox.com
Content-Transfer-Encoding: quoted-printable

This is yet another attempt to provide a means to render the cpu virtuali=
zation
technology support in Xen configurable.
Currently, irrespectively of the target platform, both AMD-V and Intel VT=
-x
drivers are built.
The series adds three new Kconfig controls, ALT2PM, SVM and VMX, that can=
 be
used to switch to a finer-grained configuration for a given platform, and
reduce dead code.

The code separation is done using the new config guards.

In this series, comparing to v1, new option CONFIG_ALT2PM introduced, tha=
t
can enable/disable altp2m support independently on VMX.
More specific changes are provided in per-patch changelog.=20

v1 series here:
https://lore.kernel.org/xen-devel/20240416061845.3467822-1-Sergiy_Kibrik@=
epam.com/

 -Sergiy

Sergiy Kibrik (11):
  x86/monitor: guard altp2m usage
  x86/p2m: guard altp2m routines
  x86/p2m: move altp2m-related code to separate file
  x86: introduce CONFIG_ALTP2M Kconfig option
  x86/p2m: guard altp2m code with CONFIG_ALTP2M option
  x86: guard cpu_has_{svm/vmx} macros with CONFIG_{SVM/VMX}
  x86/vpmu: guard vmx/svm calls with cpu_has_{vmx,svm}
  x86/traps: clean up superfluous #idef-s
  x86/domain: clean up superfluous #idef-s
  x86/vmx: guard access to cpu_has_vmx_* in common code
  iommu/vt-d: guard vmx_pi_hooks_* calls with cpu_has_vmx

Xenia Ragiadakou (4):
  x86: introduce AMD-V and Intel VT-x Kconfig options
  x86/oprofile: guard svm specific symbols with CONFIG_SVM
  x86/ioreq: guard VIO_realmode_completion with CONFIG_VMX
  x86/hvm: make AMD-V and Intel VT-x support configurable

 xen/arch/x86/Kconfig                    |  25 +
 xen/arch/x86/cpu/vpmu_amd.c             |   8 +-
 xen/arch/x86/cpu/vpmu_intel.c           |  20 +-
 xen/arch/x86/domain.c                   |   4 +-
 xen/arch/x86/hvm/Makefile               |   4 +-
 xen/arch/x86/hvm/emulate.c              |   2 +
 xen/arch/x86/hvm/hvm.c                  |   2 +-
 xen/arch/x86/hvm/ioreq.c                |   2 +
 xen/arch/x86/hvm/monitor.c              |   4 +-
 xen/arch/x86/hvm/viridian/viridian.c    |   4 +-
 xen/arch/x86/include/asm/altp2m.h       |   5 +-
 xen/arch/x86/include/asm/cpufeature.h   |   6 +-
 xen/arch/x86/include/asm/hvm/hvm.h      |   2 +-
 xen/arch/x86/include/asm/hvm/vmx/vmcs.h |   4 +-
 xen/arch/x86/include/asm/p2m.h          |  17 +-
 xen/arch/x86/mm/Makefile                |   5 +-
 xen/arch/x86/mm/altp2m.c                | 630 +++++++++++++++++++++++
 xen/arch/x86/mm/hap/Makefile            |   2 +-
 xen/arch/x86/mm/p2m-basic.c             |  19 +-
 xen/arch/x86/mm/p2m-ept.c               |   2 +-
 xen/arch/x86/mm/p2m.c                   | 632 +-----------------------
 xen/arch/x86/mm/p2m.h                   |   3 +
 xen/arch/x86/oprofile/op_model_athlon.c |   2 +-
 xen/arch/x86/traps.c                    |   9 +-
 xen/drivers/passthrough/vtd/iommu.c     |   6 +-
 xen/include/xen/sched.h                 |   2 +-
 26 files changed, 738 insertions(+), 683 deletions(-)

--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:00:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:00:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722017.1125795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AUR-0006V7-R6; Wed, 15 May 2024 08:59:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722017.1125795; Wed, 15 May 2024 08:59:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AUR-0006V0-OY; Wed, 15 May 2024 08:59:59 +0000
Received: by outflank-mailman (input) for mailman id 722017;
 Wed, 15 May 2024 08:59:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AUQ-0006Uu-QN
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 08:59:58 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 80707de0-1299-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 10:59:57 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 20A1E1AE64;
 Wed, 15 May 2024 04:59:56 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 189EB1AE63;
 Wed, 15 May 2024 04:59:56 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id A7BEE1AE62;
 Wed, 15 May 2024 04:59:54 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80707de0-1299-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=O/4uva/ALHfMHswcx9quBNHY7
	OZITBZUuNjaXjzXO0M=; b=aXt/fvr59eEmqkiT/cC1Rrc82iZ7ZlIEtG6aYNXH9
	vO49tnnHkQbM8QNyAAE0E9q8r4yXzT334oonenzJ6+aIMhP6N2X/NXZZN7HL0AjB
	EL1d6GdF2eWj6LEKAi3HF1ZYzlGwjWJGzJxVEgudtHM4mYffK1+1D2ErC5zgGf+u
	Fc=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Subject: [XEN PATCH v2 01/15] x86: introduce AMD-V and Intel VT-x Kconfig options
Date: Wed, 15 May 2024 11:59:52 +0300
Message-Id: <3f2168a337a192336e9a7fb797185c39978db11b.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 7F500188-1299-11EF-B3E3-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

From: Xenia Ragiadakou <burzalodowa@gmail.com>

Introduce two new Kconfig options, SVM and VMX, to allow code
specific to each virtualization technology to be separated and, when not
required, stripped.
CONFIG_SVM will be used to enable virtual machine extensions on platforms=
 that
implement the AMD Virtualization Technology (AMD-V).
CONFIG_VMX will be used to enable virtual machine extensions on platforms=
 that
implement the Intel Virtualization Technology (Intel VT-x).

Both features depend on HVM support.

Since, at this point, disabling any of them would cause Xen to not compil=
e,
the options are enabled by default if HVM and are not selectable by the u=
ser.

No functional change intended.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - simplify kconfig expression to def_bool HVM
 - keep file list in Makefile in alphabetical order
changes in v1:
 - change kconfig option name AMD_SVM/INTEL_VMX -> SVM/VMX
---
 xen/arch/x86/Kconfig         | 6 ++++++
 xen/arch/x86/hvm/Makefile    | 4 ++--
 xen/arch/x86/mm/Makefile     | 3 ++-
 xen/arch/x86/mm/hap/Makefile | 2 +-
 4 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 7e03e4bc55..8c9f8431f0 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -122,6 +122,12 @@ config HVM
=20
 	  If unsure, say Y.
=20
+config SVM
+	def_bool HVM
+
+config VMX
+	def_bool HVM
+
 config XEN_SHSTK
 	bool "Supervisor Shadow Stacks"
 	depends on HAS_AS_CET_SS
diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index 3464191544..8434badc64 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -1,5 +1,5 @@
-obj-y +=3D svm/
-obj-y +=3D vmx/
+obj-$(CONFIG_SVM) +=3D svm/
+obj-$(CONFIG_VMX) +=3D vmx/
 obj-y +=3D viridian/
=20
 obj-y +=3D asid.o
diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile
index 0803ac9297..0128ca7ab6 100644
--- a/xen/arch/x86/mm/Makefile
+++ b/xen/arch/x86/mm/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_MEM_SHARING) +=3D mem_sharing.o
 obj-$(CONFIG_HVM) +=3D nested.o
 obj-$(CONFIG_HVM) +=3D p2m.o
 obj-y +=3D p2m-basic.o
-obj-$(CONFIG_HVM) +=3D p2m-ept.o p2m-pod.o p2m-pt.o
+obj-$(CONFIG_VMX) +=3D p2m-ept.o
+obj-$(CONFIG_HVM) +=3D p2m-pod.o p2m-pt.o
 obj-y +=3D paging.o
 obj-y +=3D physmap.o
diff --git a/xen/arch/x86/mm/hap/Makefile b/xen/arch/x86/mm/hap/Makefile
index 8ef54b1faa..98c8a87819 100644
--- a/xen/arch/x86/mm/hap/Makefile
+++ b/xen/arch/x86/mm/hap/Makefile
@@ -3,4 +3,4 @@ obj-y +=3D guest_walk_2.o
 obj-y +=3D guest_walk_3.o
 obj-y +=3D guest_walk_4.o
 obj-y +=3D nested_hap.o
-obj-y +=3D nested_ept.o
+obj-$(CONFIG_VMX) +=3D nested_ept.o
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:02:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:02:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722024.1125805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AWQ-0007yl-5q; Wed, 15 May 2024 09:02:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722024.1125805; Wed, 15 May 2024 09:02:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AWQ-0007ye-3O; Wed, 15 May 2024 09:02:02 +0000
Received: by outflank-mailman (input) for mailman id 722024;
 Wed, 15 May 2024 09:02:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AWO-0007yU-Ky
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:02:00 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c972cd5e-1299-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:01:59 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id B7E011AE9D;
 Wed, 15 May 2024 05:01:58 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id AEFA81AE9C;
 Wed, 15 May 2024 05:01:58 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id CD3601AE9B;
 Wed, 15 May 2024 05:01:57 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c972cd5e-1299-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=0h3VDZqzxvclJXa5ig4GCZlFK
	w9kvRilSok7LMNliA4=; b=tYioIXoEbVaYnBVUQFf+TZ979ZYN5ixxzYuibjcWs
	YOlpH0krxzhbyvOsprIjv3PACrLl6Td7ZBK7EnAUtxKARZsd38sTTEbh9w4PnkMj
	0Oknt+u/3yqEtTgH/uu3fXxJDxYvO+4t35m5rjoqD0IBtWJ1UqT2wtA3rdNRwcWR
	Zc=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Tamas K Lengyel <tamas@tklengyel.com>
Subject: [XEN PATCH v2 02/15] x86/monitor: guard altp2m usage
Date: Wed, 15 May 2024 12:01:55 +0300
Message-Id: <01767c3f98a88999d4b8ed3ae742ad66a0921ba3.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 C8B7F45C-1299-11EF-9E0A-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

Explicitly check whether altp2m is on for domain when getting altp2m inde=
x.
If explicit call to altp2m_active() always returns false, DCE will remove
call to altp2m_vcpu_idx().

The puspose of that is later to be able to disable altp2m support and
exclude its code from the build completely, when not supported by target
platform (as of now it's supported for VT-d only).

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Tamas K Lengyel <tamas@tklengyel.com>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - patch description changed, removed VMX mentioning
 - guard by altp2m_active() instead of hvm_altp2m_supported()
---
 xen/arch/x86/hvm/monitor.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/monitor.c b/xen/arch/x86/hvm/monitor.c
index 2a8ff07ec9..74621000b2 100644
--- a/xen/arch/x86/hvm/monitor.c
+++ b/xen/arch/x86/hvm/monitor.c
@@ -262,6 +262,8 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t g=
fn, uint32_t pfec,
     struct vcpu *curr =3D current;
     vm_event_request_t req =3D {};
     paddr_t gpa =3D (gfn_to_gaddr(gfn) | (gla & ~PAGE_MASK));
+    unsigned int altp2m_idx =3D altp2m_active(curr->domain) ?
+                              altp2m_vcpu_idx(curr) : 0;
     int rc;
=20
     ASSERT(curr->arch.vm_event->send_event);
@@ -270,7 +272,7 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t g=
fn, uint32_t pfec,
      * p2m_get_mem_access() can fail from a invalid MFN and return -ESRC=
H
      * in which case access must be restricted.
      */
-    rc =3D p2m_get_mem_access(curr->domain, gfn, &access, altp2m_vcpu_id=
x(curr));
+    rc =3D p2m_get_mem_access(curr->domain, gfn, &access, altp2m_idx);
=20
     if ( rc =3D=3D -ESRCH )
         access =3D XENMEM_access_n;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:04:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:04:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722028.1125815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AYP-0000BH-KF; Wed, 15 May 2024 09:04:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722028.1125815; Wed, 15 May 2024 09:04:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AYP-0000BA-HU; Wed, 15 May 2024 09:04:05 +0000
Received: by outflank-mailman (input) for mailman id 722028;
 Wed, 15 May 2024 09:04:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AYN-0000AC-St
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:04:03 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 131bb5d6-129a-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:04:03 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 665621AEBA;
 Wed, 15 May 2024 05:04:02 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 5F0061AEB9;
 Wed, 15 May 2024 05:04:02 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 3AA061AEB5;
 Wed, 15 May 2024 05:04:01 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 131bb5d6-129a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=8cHhszYK7gh07npz053WlMjUF
	pjZMrbumLTp0bEkDA8=; b=gGb2MAXvbOitMXo/4x42RkhX1u793PsoDq7gFhhlb
	ULcmUOnDUObjxDPLQKtEQuz6HM05+yM7rjDCEUP/zAPFDVEeXJt/OFCXVIMl0vLk
	N+Jw6PNtJmS/oPWmvgzaTBjyjTzntAuS163wlN8pOcxAhizyOB5tUjE+uG5A0V8e
	ro=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Tamas K Lengyel <tamas@tklengyel.com>
Subject: [XEN PATCH v2 03/15] x86/p2m: guard altp2m routines
Date: Wed, 15 May 2024 12:03:59 +0300
Message-Id: <d4c537b1bcac4b301b16336ea3407834fc3076b1.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 12469D44-129A-11EF-9194-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

Initialize and bring down altp2m only when it is supported by the platfor=
m,
e.g. VMX. Also guard p2m_altp2m_propagate_change().
The puspose of that is the possiblity to disable altp2m support and exclu=
de its
code from the build completely, when it's not supported by the target pla=
tform.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Tamas K Lengyel <tamas@tklengyel.com>
---
 xen/arch/x86/mm/p2m-basic.c | 19 +++++++++++--------
 xen/arch/x86/mm/p2m-ept.c   |  2 +-
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-basic.c b/xen/arch/x86/mm/p2m-basic.c
index 8599bd15c6..90106997d7 100644
--- a/xen/arch/x86/mm/p2m-basic.c
+++ b/xen/arch/x86/mm/p2m-basic.c
@@ -126,13 +126,15 @@ int p2m_init(struct domain *d)
         return rc;
     }
=20
-    rc =3D p2m_init_altp2m(d);
-    if ( rc )
+    if ( hvm_altp2m_supported() )
     {
-        p2m_teardown_hostp2m(d);
-        p2m_teardown_nestedp2m(d);
+        rc =3D p2m_init_altp2m(d);
+        if ( rc )
+        {
+            p2m_teardown_hostp2m(d);
+            p2m_teardown_nestedp2m(d);
+        }
     }
-
     return rc;
 }
=20
@@ -195,11 +197,12 @@ void p2m_final_teardown(struct domain *d)
 {
     if ( is_hvm_domain(d) )
     {
+        if ( hvm_altp2m_supported() )
+            p2m_teardown_altp2m(d);
         /*
-         * We must tear down both of them unconditionally because
-         * we initialise them unconditionally.
+         * We must tear down nestedp2m unconditionally because
+         * we initialise it unconditionally.
          */
-        p2m_teardown_altp2m(d);
         p2m_teardown_nestedp2m(d);
     }
=20
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index f83610cb8c..d264df5b14 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -986,7 +986,7 @@ out:
     if ( is_epte_present(&old_entry) )
         ept_free_entry(p2m, &old_entry, target);
=20
-    if ( entry_written && p2m_is_hostp2m(p2m) )
+    if ( entry_written && p2m_is_hostp2m(p2m) && hvm_altp2m_supported())
     {
         ret =3D p2m_altp2m_propagate_change(d, _gfn(gfn), mfn, order, p2=
mt, p2ma);
         if ( !rc )
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:05:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:05:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722031.1125826 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AZe-0000h3-VY; Wed, 15 May 2024 09:05:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722031.1125826; Wed, 15 May 2024 09:05:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AZe-0000gw-SW; Wed, 15 May 2024 09:05:22 +0000
Received: by outflank-mailman (input) for mailman id 722031;
 Wed, 15 May 2024 09:05:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cqhY=MS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7AZd-0000gm-Hg
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:05:21 +0000
Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com
 [2607:f8b0:4864:20::112a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40422978-129a-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 11:05:19 +0200 (CEST)
Received: by mail-yw1-x112a.google.com with SMTP id
 00721157ae682-6114c9b4d83so58847617b3.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:05:19 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54b579csm80678431cf.5.2024.05.15.02.05.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 02:05:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40422978-129a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715763918; x=1716368718; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=93qIpFPUbjkLMI3HgTtDzyyJ14Dx2k56D1NQkwSoxGA=;
        b=K0ToZ4UdpydnONylM2O6YytfzeErlgxkPPs+tXxdykuzYjkpprH2WqDnPmzFWPZi67
         XV3PkjzwFFOyDCZIsjV0sh46Rrwt7Iplzv31r0fo1WL5GZsKCYzUIDHvhdQniHxWtaeU
         H+nG4VV9ixkezCK3SqQ9PjDEmGNYzGj+OIUHA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715763918; x=1716368718;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=93qIpFPUbjkLMI3HgTtDzyyJ14Dx2k56D1NQkwSoxGA=;
        b=GhhfVHdS9+BFd8IxWu4OkhcrkbmZVR9Ve5R2SSwyaSmOj/ooO3/s0t6Mg9+KjUs+Dt
         0r48Db9lvFLCGQHANZFSCyWOONxWxXfhX7OvBg5VaVHR11ixG0aRsWaU8S/f6yaewy69
         MqLWqnREO6VGg28gViVmyUQHLRGSQRsv4oEpd4sJ0z04hckT8eRAUIodblhG59ZT4CCc
         psvMP81z8XngGzUMrBW/NrVn8jytCh+PkqFd5eFFv/9x2vkiOCrJuozEo9aCEgdtwe7w
         eBZ6eg/AFZfgwZdh05XkbrGBf52x00YC8fQgYHOT8n26MjCarvMZ5Tx45DqK9cUJxdfP
         d5lA==
X-Gm-Message-State: AOJu0YxG96G0JItc8USotsvPlkEvac7XlDmuPJCF7/5le+l9OaYziDG+
	vCXmQaZE3UaPEARqLMQSzCpF5Cx9AIFAaDy3VvnABV/6oDgE2odRY8WEvB5pHSY=
X-Google-Smtp-Source: AGHT+IE6Pq6gQv3c0ltrNwytdcxQuSZygVg2GohsEkcsl/w21paUbLSg0wQK4xxeSoA3+Tqxy0uWhA==
X-Received: by 2002:a05:690c:fc1:b0:61b:91e3:f954 with SMTP id 00721157ae682-622affc6413mr174115057b3.8.1715763917540;
        Wed, 15 May 2024 02:05:17 -0700 (PDT)
Date: Wed, 15 May 2024 11:05:15 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt
Message-ID: <ZkR6y3kuvKySKwRm@macbook>
References: <20240513111117.68828-1-eliasely@amazon.com>
 <20240513111117.68828-2-eliasely@amazon.com>
 <ZkIxdtiDc_pnPWdx@macbook>
 <699c6487-58c9-456f-8415-e3c525fa905e@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <699c6487-58c9-456f-8415-e3c525fa905e@amazon.com>

On Tue, May 14, 2024 at 06:15:57PM +0100, Elias El Yandouzi wrote:
> Hi Roger,
> 
> On 13/05/2024 16:27, Roger Pau Monné wrote:
> > > diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
> > > index 2a445bb17b..1b025986f7 100644
> > > --- a/xen/arch/x86/pv/domain.c
> > > +++ b/xen/arch/x86/pv/domain.c
> > > @@ -288,6 +288,21 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
> > >                                 1U << GDT_LDT_VCPU_SHIFT);
> > >   }
> > > +static int pv_create_root_pt_l1tab(struct vcpu *v)
> > > +{
> > > +    return create_perdomain_mapping(v->domain,
> > > +                                    PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v),
> > > +                                    1, v->domain->arch.pv.root_pt_l1tab,
> > > +                                    NULL);
> > > +}
> > > +
> > > +static void pv_destroy_root_pt_l1tab(struct vcpu *v)
> > 
> > The two 'v' parameters could be const here.
> 
> I could constify the parameters but the functions wouldn't be consistent
> with the two above for gdt/ldt.

The fact they are not const for the other helpers would also need
fixing at some point IMO, it's best if those are already using the
correct type.

> > > diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
> > > index df015589ce..c1377da7a5 100644
> > > --- a/xen/arch/x86/x86_64/entry.S
> > > +++ b/xen/arch/x86/x86_64/entry.S
> > > @@ -162,7 +162,15 @@ FUNC_LOCAL(restore_all_guest)
> > >           and   %rsi, %rdi
> > >           and   %r9, %rsi
> > >           add   %rcx, %rdi
> > > +
> > > +        /*
> > > +         * The address in the vCPU cr3 is always mapped in the per-domain
> > > +         * pv_root_pt virt area.
> > > +         */
> > > +        imul  $PAGE_SIZE, VCPU_id(%rbx), %esi
> > 
> > Aren't some of the previous operations against %rsi now useless since
> > it gets unconditionally overwritten here?
> 
> I think I can just get rid off of:
> 
>     and   %r9, %rsi
> 
> > and   %r9, %rsi
> > [...]
> > add   %rcx, %rsi
> 
> The second operation you suggested is actually used to retrieve the VA of
> the PV_ROOT_PT.

Oh, yes, sorry, got confused when looking at the source file together
with the diff, it's only the `and` that can be removed.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:06:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:06:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722034.1125835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AaW-0001DI-6v; Wed, 15 May 2024 09:06:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722034.1125835; Wed, 15 May 2024 09:06:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AaW-0001DB-4I; Wed, 15 May 2024 09:06:16 +0000
Received: by outflank-mailman (input) for mailman id 722034;
 Wed, 15 May 2024 09:06:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AaU-000103-OI
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:06:14 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f8a6286-129a-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:06:12 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 1CFFA1B78B;
 Wed, 15 May 2024 05:06:10 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 13D771B78A;
 Wed, 15 May 2024 05:06:10 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 62E661B789;
 Wed, 15 May 2024 05:06:06 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f8a6286-129a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=5ppiqUMhZa/TUsVhmPyp9SIdu
	yJb6XDnR4WLAsE0R6c=; b=me7UyMbtH9QPwTtfAqu5sqbPrsCuFWQBnNOHBGXjU
	ylqUgHLKlokaVboruREVIZf7ST6k2Iv2RDdK/sxzmHHmb4/XIcyk9gu5Edhvujrk
	TpjdU81m7AJ/BlYX9S2wu93x7ZHHgXBBsEAix1W3Mr3uKgK2cQsx8mfUakaHBAZK
	sc=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Tamas K Lengyel <tamas@tklengyel.com>
Subject: [XEN PATCH v2 04/15] x86/p2m: move altp2m-related code to separate file
Date: Wed, 15 May 2024 12:06:02 +0300
Message-Id: <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 5CE01CA4-129A-11EF-B0DF-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

Move altp2m code from generic p2m.c file to altp2m.c, so it is kept separ=
ately
and can possibly be disabled in the build. We may want to disable it when
building for specific platform only, that doesn't support alternate p2m.

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Tamas K Lengyel <tamas@tklengyel.com>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v2:
 - no double blank lines
 - no unrelated re-formatting
 - header #include-s ordering
 - changed patch description
---
 xen/arch/x86/mm/altp2m.c | 630 ++++++++++++++++++++++++++++++++++++++
 xen/arch/x86/mm/p2m.c    | 632 +--------------------------------------
 xen/arch/x86/mm/p2m.h    |   3 +
 3 files changed, 635 insertions(+), 630 deletions(-)

diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
index a04297b646..6fe1e9ed6b 100644
--- a/xen/arch/x86/mm/altp2m.c
+++ b/xen/arch/x86/mm/altp2m.c
@@ -7,6 +7,8 @@
 #include <asm/hvm/hvm.h>
 #include <asm/p2m.h>
 #include <asm/altp2m.h>
+#include <public/hvm/hvm_op.h>
+#include <xen/event.h>
 #include "mm-locks.h"
 #include "p2m.h"
=20
@@ -151,6 +153,634 @@ void p2m_teardown_altp2m(struct domain *d)
     }
 }
=20
+int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t=
 *mfn,
+                               p2m_type_t *t, p2m_access_t *a,
+                               bool prepopulate)
+{
+    *mfn =3D ap2m->get_entry(ap2m, gfn, t, a, 0, NULL, NULL);
+
+    /* Check host p2m if no valid entry in alternate */
+    if ( !mfn_valid(*mfn) && !p2m_is_hostp2m(ap2m) )
+    {
+        struct p2m_domain *hp2m =3D p2m_get_hostp2m(ap2m->domain);
+        unsigned int page_order;
+        int rc;
+
+        *mfn =3D p2m_get_gfn_type_access(hp2m, gfn, t, a, P2M_ALLOC | P2=
M_UNSHARE,
+                                       &page_order, 0);
+
+        rc =3D -ESRCH;
+        if ( !mfn_valid(*mfn) || *t !=3D p2m_ram_rw )
+            return rc;
+
+        /* If this is a superpage, copy that first */
+        if ( prepopulate && page_order !=3D PAGE_ORDER_4K )
+        {
+            unsigned long mask =3D ~((1UL << page_order) - 1);
+            gfn_t gfn_aligned =3D _gfn(gfn_x(gfn) & mask);
+            mfn_t mfn_aligned =3D _mfn(mfn_x(*mfn) & mask);
+
+            rc =3D ap2m->set_entry(ap2m, gfn_aligned, mfn_aligned, page_=
order, *t, *a, 1);
+            if ( rc )
+                return rc;
+        }
+    }
+
+    return 0;
+}
+
+void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
+{
+    if ( altp2m_active(v->domain) )
+        p2m_switch_vcpu_altp2m_by_id(v, idx);
+}
+
+bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
+{
+    struct domain *d =3D v->domain;
+    bool rc =3D false;
+
+    if ( idx >=3D MAX_ALTP2M )
+        return rc;
+
+    altp2m_list_lock(d);
+
+    if ( d->arch.altp2m_eptp[idx] !=3D mfn_x(INVALID_MFN) )
+    {
+        if ( p2m_set_altp2m(v, idx) )
+            altp2m_vcpu_update_p2m(v);
+        rc =3D 1;
+    }
+
+    altp2m_list_unlock(d);
+    return rc;
+}
+
+/*
+ * Read info about the gfn in an altp2m, locking the gfn.
+ *
+ * If the entry is valid, pass the results back to the caller.
+ *
+ * If the entry was invalid, and the host's entry is also invalid,
+ * return to the caller without any changes.
+ *
+ * If the entry is invalid, and the host entry was valid, propagate
+ * the host's entry to the altp2m (retaining page order), and indicate
+ * that the caller should re-try the faulting instruction.
+ */
+bool p2m_altp2m_get_or_propagate(struct p2m_domain *ap2m, unsigned long =
gfn_l,
+                                 mfn_t *mfn, p2m_type_t *p2mt,
+                                 p2m_access_t *p2ma, unsigned int *page_=
order)
+{
+    p2m_type_t ap2mt;
+    p2m_access_t ap2ma;
+    unsigned int cur_order;
+    unsigned long mask;
+    gfn_t gfn;
+    mfn_t amfn;
+    int rc;
+
+    /*
+     * NB we must get the full lock on the altp2m here, in addition to
+     * the lock on the individual gfn, since we may change a range of
+     * gfns below.
+     */
+    p2m_lock(ap2m);
+
+    amfn =3D get_gfn_type_access(ap2m, gfn_l, &ap2mt, &ap2ma, 0, &cur_or=
der);
+
+    if ( cur_order > *page_order )
+        cur_order =3D *page_order;
+
+    if ( !mfn_eq(amfn, INVALID_MFN) )
+    {
+        p2m_unlock(ap2m);
+        *mfn  =3D amfn;
+        *p2mt =3D ap2mt;
+        *p2ma =3D ap2ma;
+        *page_order =3D cur_order;
+        return false;
+    }
+
+    /* Host entry is also invalid; don't bother setting the altp2m entry=
. */
+    if ( mfn_eq(*mfn, INVALID_MFN) )
+    {
+        p2m_unlock(ap2m);
+        *page_order =3D cur_order;
+        return false;
+    }
+
+    /*
+     * If this is a superpage mapping, round down both frame numbers
+     * to the start of the superpage.  NB that we repupose `amfn`
+     * here.
+     */
+    mask =3D ~((1UL << cur_order) - 1);
+    amfn =3D _mfn(mfn_x(*mfn) & mask);
+    gfn =3D _gfn(gfn_l & mask);
+
+    /* Override the altp2m entry with its default access. */
+    *p2ma =3D ap2m->default_access;
+
+    rc =3D p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
+    p2m_unlock(ap2m);
+
+    if ( rc )
+    {
+        gprintk(XENLOG_ERR,
+                "failed to set entry for %"PRI_gfn" -> %"PRI_mfn" altp2m=
 %u, rc %d\n",
+                gfn_l, mfn_x(amfn), vcpu_altp2m(current).p2midx, rc);
+        domain_crash(ap2m->domain);
+    }
+
+    return true;
+}
+
+enum altp2m_reset_type {
+    ALTP2M_RESET,
+    ALTP2M_DEACTIVATE
+};
+
+static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
+                             enum altp2m_reset_type reset_type)
+{
+    struct p2m_domain *p2m;
+
+    ASSERT(idx < MAX_ALTP2M);
+    p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
+
+    p2m_lock(p2m);
+
+    p2m_flush_table_locked(p2m);
+
+    if ( reset_type =3D=3D ALTP2M_DEACTIVATE )
+        p2m_free_logdirty(p2m);
+
+    /* Uninit and reinit ept to force TLB shootdown */
+    ept_p2m_uninit(p2m);
+    ept_p2m_init(p2m);
+
+    p2m->min_remapped_gfn =3D gfn_x(INVALID_GFN);
+    p2m->max_remapped_gfn =3D 0;
+
+    p2m_unlock(p2m);
+}
+
+void p2m_flush_altp2m(struct domain *d)
+{
+    unsigned int i;
+
+    altp2m_list_lock(d);
+
+    for ( i =3D 0; i < MAX_ALTP2M; i++ )
+    {
+        p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
+        d->arch.altp2m_eptp[i] =3D mfn_x(INVALID_MFN);
+        d->arch.altp2m_visible_eptp[i] =3D mfn_x(INVALID_MFN);
+    }
+
+    altp2m_list_unlock(d);
+}
+
+static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
+                               p2m_access_t hvmmem_default_access)
+{
+    struct p2m_domain *hostp2m, *p2m;
+    int rc;
+
+    ASSERT(idx < MAX_ALTP2M);
+
+    p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
+    hostp2m =3D p2m_get_hostp2m(d);
+
+    p2m_lock(p2m);
+
+    rc =3D p2m_init_logdirty(p2m);
+
+    if ( rc )
+        goto out;
+
+    /* The following is really just a rangeset copy. */
+    rc =3D rangeset_merge(p2m->logdirty_ranges, hostp2m->logdirty_ranges=
);
+
+    if ( rc )
+    {
+        p2m_free_logdirty(p2m);
+        goto out;
+    }
+
+    p2m->default_access =3D hvmmem_default_access;
+    p2m->domain =3D hostp2m->domain;
+    p2m->global_logdirty =3D hostp2m->global_logdirty;
+    p2m->min_remapped_gfn =3D gfn_x(INVALID_GFN);
+    p2m->max_mapped_pfn =3D p2m->max_remapped_gfn =3D 0;
+
+    p2m_init_altp2m_ept(d, idx);
+
+ out:
+    p2m_unlock(p2m);
+
+    return rc;
+}
+
+int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
+{
+    int rc =3D -EINVAL;
+    struct p2m_domain *hostp2m =3D p2m_get_hostp2m(d);
+
+    if ( idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+        return rc;
+
+    altp2m_list_lock(d);
+
+    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D=3D
+         mfn_x(INVALID_MFN) )
+        rc =3D p2m_activate_altp2m(d, idx, hostp2m->default_access);
+
+    altp2m_list_unlock(d);
+    return rc;
+}
+
+int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
+                         xenmem_access_t hvmmem_default_access)
+{
+    int rc =3D -EINVAL;
+    unsigned int i;
+    p2m_access_t a;
+    struct p2m_domain *hostp2m =3D p2m_get_hostp2m(d);
+
+    if ( hvmmem_default_access > XENMEM_access_default ||
+         !xenmem_access_to_p2m_access(hostp2m, hvmmem_default_access, &a=
) )
+        return rc;
+
+    altp2m_list_lock(d);
+
+    for ( i =3D 0; i < MAX_ALTP2M; i++ )
+    {
+        if ( d->arch.altp2m_eptp[i] !=3D mfn_x(INVALID_MFN) )
+            continue;
+
+        rc =3D p2m_activate_altp2m(d, i, a);
+
+        if ( !rc )
+            *idx =3D i;
+
+        break;
+    }
+
+    altp2m_list_unlock(d);
+    return rc;
+}
+
+int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
+{
+    struct p2m_domain *p2m;
+    int rc =3D -EBUSY;
+
+    if ( !idx || idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) =
)
+        return rc;
+
+    rc =3D domain_pause_except_self(d);
+    if ( rc )
+        return rc;
+
+    rc =3D -EBUSY;
+    altp2m_list_lock(d);
+
+    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=3D
+         mfn_x(INVALID_MFN) )
+    {
+        p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
+
+        if ( !_atomic_read(p2m->active_vcpus) )
+        {
+            p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
+            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D
+                mfn_x(INVALID_MFN);
+            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP=
)] =3D
+                mfn_x(INVALID_MFN);
+            rc =3D 0;
+        }
+    }
+
+    altp2m_list_unlock(d);
+
+    domain_unpause_except_self(d);
+
+    return rc;
+}
+
+int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
+{
+    struct vcpu *v;
+    int rc =3D -EINVAL;
+
+    if ( idx >=3D MAX_ALTP2M )
+        return rc;
+
+    rc =3D domain_pause_except_self(d);
+    if ( rc )
+        return rc;
+
+    rc =3D -EINVAL;
+    altp2m_list_lock(d);
+
+    if ( d->arch.altp2m_visible_eptp[idx] !=3D mfn_x(INVALID_MFN) )
+    {
+        for_each_vcpu( d, v )
+            if ( p2m_set_altp2m(v, idx) )
+                altp2m_vcpu_update_p2m(v);
+
+        rc =3D 0;
+    }
+
+    altp2m_list_unlock(d);
+
+    domain_unpause_except_self(d);
+
+    return rc;
+}
+
+int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
+                          gfn_t old_gfn, gfn_t new_gfn)
+{
+    struct p2m_domain *hp2m, *ap2m;
+    p2m_access_t a;
+    p2m_type_t t;
+    mfn_t mfn;
+    int rc =3D -EINVAL;
+
+    if ( idx >=3D  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D=3D
+         mfn_x(INVALID_MFN) )
+        return rc;
+
+    hp2m =3D p2m_get_hostp2m(d);
+    ap2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
+
+    p2m_lock(hp2m);
+    p2m_lock(ap2m);
+
+    if ( gfn_eq(new_gfn, INVALID_GFN) )
+    {
+        mfn =3D ap2m->get_entry(ap2m, old_gfn, &t, &a, 0, NULL, NULL);
+        rc =3D mfn_valid(mfn)
+             ? p2m_remove_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K)
+             : 0;
+        goto out;
+    }
+
+    rc =3D altp2m_get_effective_entry(ap2m, old_gfn, &mfn, &t, &a,
+                                    AP2MGET_prepopulate);
+    if ( rc )
+        goto out;
+
+    rc =3D altp2m_get_effective_entry(ap2m, new_gfn, &mfn, &t, &a,
+                                    AP2MGET_query);
+    if ( rc )
+        goto out;
+
+    if ( !ap2m->set_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K, t, a,
+                          (current->domain !=3D d)) )
+    {
+        rc =3D 0;
+
+        if ( gfn_x(new_gfn) < ap2m->min_remapped_gfn )
+            ap2m->min_remapped_gfn =3D gfn_x(new_gfn);
+        if ( gfn_x(new_gfn) > ap2m->max_remapped_gfn )
+            ap2m->max_remapped_gfn =3D gfn_x(new_gfn);
+    }
+
+ out:
+    p2m_unlock(ap2m);
+    p2m_unlock(hp2m);
+    return rc;
+}
+
+int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
+                                mfn_t mfn, unsigned int page_order,
+                                p2m_type_t p2mt, p2m_access_t p2ma)
+{
+    struct p2m_domain *p2m;
+    unsigned int i;
+    unsigned int reset_count =3D 0;
+    unsigned int last_reset_idx =3D ~0;
+    int ret =3D 0;
+
+    if ( !altp2m_active(d) )
+        return 0;
+
+    altp2m_list_lock(d);
+
+    for ( i =3D 0; i < MAX_ALTP2M; i++ )
+    {
+        p2m_type_t t;
+        p2m_access_t a;
+
+        if ( d->arch.altp2m_eptp[i] =3D=3D mfn_x(INVALID_MFN) )
+            continue;
+
+        p2m =3D d->arch.altp2m_p2m[i];
+
+        /* Check for a dropped page that may impact this altp2m */
+        if ( mfn_eq(mfn, INVALID_MFN) &&
+             gfn_x(gfn) + (1UL << page_order) > p2m->min_remapped_gfn &&
+             gfn_x(gfn) <=3D p2m->max_remapped_gfn )
+        {
+            if ( !reset_count++ )
+            {
+                p2m_reset_altp2m(d, i, ALTP2M_RESET);
+                last_reset_idx =3D i;
+            }
+            else
+            {
+                /* At least 2 altp2m's impacted, so reset everything */
+                for ( i =3D 0; i < MAX_ALTP2M; i++ )
+                {
+                    if ( i =3D=3D last_reset_idx ||
+                         d->arch.altp2m_eptp[i] =3D=3D mfn_x(INVALID_MFN=
) )
+                        continue;
+
+                    p2m_reset_altp2m(d, i, ALTP2M_RESET);
+                }
+
+                ret =3D 0;
+                break;
+            }
+        }
+        else if ( !mfn_eq(get_gfn_type_access(p2m, gfn_x(gfn), &t, &a, 0=
,
+                                              NULL), INVALID_MFN) )
+        {
+            int rc =3D p2m_set_entry(p2m, gfn, mfn, page_order, p2mt, p2=
ma);
+
+            /* Best effort: Don't bail on error. */
+            if ( !ret )
+                ret =3D rc;
+
+            p2m_put_gfn(p2m, gfn);
+        }
+        else
+            p2m_put_gfn(p2m, gfn);
+    }
+
+    altp2m_list_unlock(d);
+
+    return ret;
+}
+
+/*
+ * Set/clear the #VE suppress bit for a page.  Only available on VMX.
+ */
+int p2m_set_suppress_ve(struct domain *d, gfn_t gfn, bool suppress_ve,
+                        unsigned int altp2m_idx)
+{
+    int rc;
+    struct xen_hvm_altp2m_suppress_ve_multi sve =3D {
+        altp2m_idx, suppress_ve, 0, 0, gfn_x(gfn), gfn_x(gfn), 0
+    };
+
+    if ( !(rc =3D p2m_set_suppress_ve_multi(d, &sve)) )
+        rc =3D sve.first_error;
+
+    return rc;
+}
+
+/*
+ * Set/clear the #VE suppress bit for multiple pages.  Only available on=
 VMX.
+ */
+int p2m_set_suppress_ve_multi(struct domain *d,
+                              struct xen_hvm_altp2m_suppress_ve_multi *s=
ve)
+{
+    struct p2m_domain *host_p2m =3D p2m_get_hostp2m(d);
+    struct p2m_domain *ap2m =3D NULL;
+    struct p2m_domain *p2m =3D host_p2m;
+    uint64_t start =3D sve->first_gfn;
+    int rc =3D 0;
+
+    if ( sve->view > 0 )
+    {
+        if ( sve->view >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP=
) ||
+             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)=
] =3D=3D
+             mfn_x(INVALID_MFN) )
+            return -EINVAL;
+
+        p2m =3D ap2m =3D array_access_nospec(d->arch.altp2m_p2m, sve->vi=
ew);
+    }
+
+    p2m_lock(host_p2m);
+
+    if ( ap2m )
+        p2m_lock(ap2m);
+
+    while ( sve->last_gfn >=3D start )
+    {
+        p2m_access_t a;
+        p2m_type_t t;
+        mfn_t mfn;
+        int err =3D 0;
+
+        if ( (err =3D altp2m_get_effective_entry(p2m, _gfn(start), &mfn,=
 &t, &a,
+                                               AP2MGET_query)) &&
+             !sve->first_error )
+        {
+            sve->first_error_gfn =3D start; /* Save the gfn of the first=
 error */
+            sve->first_error =3D err; /* Save the first error code */
+        }
+
+        if ( !err && (err =3D p2m->set_entry(p2m, _gfn(start), mfn,
+                                           PAGE_ORDER_4K, t, a,
+                                           sve->suppress_ve)) &&
+             !sve->first_error )
+        {
+            sve->first_error_gfn =3D start; /* Save the gfn of the first=
 error */
+            sve->first_error =3D err; /* Save the first error code */
+        }
+
+        /* Check for continuation if it's not the last iteration. */
+        if ( sve->last_gfn >=3D ++start && hypercall_preempt_check() )
+        {
+            rc =3D -ERESTART;
+            break;
+        }
+    }
+
+    sve->first_gfn =3D start;
+
+    if ( ap2m )
+        p2m_unlock(ap2m);
+
+    p2m_unlock(host_p2m);
+
+    return rc;
+}
+
+int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
+                        unsigned int altp2m_idx)
+{
+    struct p2m_domain *host_p2m =3D p2m_get_hostp2m(d);
+    struct p2m_domain *ap2m =3D NULL;
+    struct p2m_domain *p2m;
+    mfn_t mfn;
+    p2m_access_t a;
+    p2m_type_t t;
+    int rc =3D 0;
+
+    if ( altp2m_idx > 0 )
+    {
+        if ( altp2m_idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPT=
P) ||
+             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP=
)] =3D=3D
+             mfn_x(INVALID_MFN) )
+            return -EINVAL;
+
+        p2m =3D ap2m =3D array_access_nospec(d->arch.altp2m_p2m, altp2m_=
idx);
+    }
+    else
+        p2m =3D host_p2m;
+
+    gfn_lock(host_p2m, gfn, 0);
+
+    if ( ap2m )
+        p2m_lock(ap2m);
+
+    mfn =3D p2m->get_entry(p2m, gfn, &t, &a, 0, NULL, suppress_ve);
+    if ( !mfn_valid(mfn) )
+        rc =3D -ESRCH;
+
+    if ( ap2m )
+        p2m_unlock(ap2m);
+
+    gfn_unlock(host_p2m, gfn, 0);
+
+    return rc;
+}
+
+int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m=
_idx,
+                                   uint8_t visible)
+{
+    int rc =3D 0;
+
+    altp2m_list_lock(d);
+
+    /*
+     * Eptp index is correlated with altp2m index and should not exceed
+     * min(MAX_ALTP2M, MAX_EPTP).
+     */
+    if ( altp2m_idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) |=
|
+         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =3D=
=3D
+         mfn_x(INVALID_MFN) )
+        rc =3D -EINVAL;
+    else if ( visible )
+        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_E=
PTP)] =3D
+            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)=
];
+    else
+        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_E=
PTP)] =3D
+            mfn_x(INVALID_MFN);
+
+    altp2m_list_unlock(d);
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index ce742c12e0..7c422a2d7e 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -500,7 +500,7 @@ int p2m_alloc_table(struct p2m_domain *p2m)
     return 0;
 }
=20
-static int __must_check
+int __must_check
 p2m_remove_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
                  unsigned int page_order)
 {
@@ -1329,7 +1329,7 @@ p2m_getlru_nestedp2m(struct domain *d, struct p2m_d=
omain *p2m)
     return p2m;
 }
=20
-static void
+void
 p2m_flush_table_locked(struct p2m_domain *p2m)
 {
     struct page_info *top, *pg;
@@ -1729,481 +1729,6 @@ int unmap_mmio_regions(struct domain *d,
     return i =3D=3D nr ? 0 : i ?: ret;
 }
=20
-int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t=
 *mfn,
-                               p2m_type_t *t, p2m_access_t *a,
-                               bool prepopulate)
-{
-    *mfn =3D ap2m->get_entry(ap2m, gfn, t, a, 0, NULL, NULL);
-
-    /* Check host p2m if no valid entry in alternate */
-    if ( !mfn_valid(*mfn) && !p2m_is_hostp2m(ap2m) )
-    {
-        struct p2m_domain *hp2m =3D p2m_get_hostp2m(ap2m->domain);
-        unsigned int page_order;
-        int rc;
-
-        *mfn =3D p2m_get_gfn_type_access(hp2m, gfn, t, a, P2M_ALLOC | P2=
M_UNSHARE,
-                                       &page_order, 0);
-
-        rc =3D -ESRCH;
-        if ( !mfn_valid(*mfn) || *t !=3D p2m_ram_rw )
-            return rc;
-
-        /* If this is a superpage, copy that first */
-        if ( prepopulate && page_order !=3D PAGE_ORDER_4K )
-        {
-            unsigned long mask =3D ~((1UL << page_order) - 1);
-            gfn_t gfn_aligned =3D _gfn(gfn_x(gfn) & mask);
-            mfn_t mfn_aligned =3D _mfn(mfn_x(*mfn) & mask);
-
-            rc =3D ap2m->set_entry(ap2m, gfn_aligned, mfn_aligned, page_=
order, *t, *a, 1);
-            if ( rc )
-                return rc;
-        }
-    }
-
-    return 0;
-}
-
-void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
-{
-    if ( altp2m_active(v->domain) )
-        p2m_switch_vcpu_altp2m_by_id(v, idx);
-}
-
-bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
-{
-    struct domain *d =3D v->domain;
-    bool rc =3D false;
-
-    if ( idx >=3D MAX_ALTP2M )
-        return rc;
-
-    altp2m_list_lock(d);
-
-    if ( d->arch.altp2m_eptp[idx] !=3D mfn_x(INVALID_MFN) )
-    {
-        if ( p2m_set_altp2m(v, idx) )
-            altp2m_vcpu_update_p2m(v);
-        rc =3D 1;
-    }
-
-    altp2m_list_unlock(d);
-    return rc;
-}
-
-/*
- * Read info about the gfn in an altp2m, locking the gfn.
- *
- * If the entry is valid, pass the results back to the caller.
- *
- * If the entry was invalid, and the host's entry is also invalid,
- * return to the caller without any changes.
- *
- * If the entry is invalid, and the host entry was valid, propagate
- * the host's entry to the altp2m (retaining page order), and indicate
- * that the caller should re-try the faulting instruction.
- */
-bool p2m_altp2m_get_or_propagate(struct p2m_domain *ap2m, unsigned long =
gfn_l,
-                                 mfn_t *mfn, p2m_type_t *p2mt,
-                                 p2m_access_t *p2ma, unsigned int *page_=
order)
-{
-    p2m_type_t ap2mt;
-    p2m_access_t ap2ma;
-    unsigned int cur_order;
-    unsigned long mask;
-    gfn_t gfn;
-    mfn_t amfn;
-    int rc;
-
-    /*
-     * NB we must get the full lock on the altp2m here, in addition to
-     * the lock on the individual gfn, since we may change a range of
-     * gfns below.
-     */
-    p2m_lock(ap2m);
-
-    amfn =3D get_gfn_type_access(ap2m, gfn_l, &ap2mt, &ap2ma, 0, &cur_or=
der);
-
-    if ( cur_order > *page_order )
-        cur_order =3D *page_order;
-
-    if ( !mfn_eq(amfn, INVALID_MFN) )
-    {
-        p2m_unlock(ap2m);
-        *mfn  =3D amfn;
-        *p2mt =3D ap2mt;
-        *p2ma =3D ap2ma;
-        *page_order =3D cur_order;
-        return false;
-    }
-
-    /* Host entry is also invalid; don't bother setting the altp2m entry=
. */
-    if ( mfn_eq(*mfn, INVALID_MFN) )
-    {
-        p2m_unlock(ap2m);
-        *page_order =3D cur_order;
-        return false;
-    }
-
-    /*
-     * If this is a superpage mapping, round down both frame numbers
-     * to the start of the superpage.  NB that we repupose `amfn`
-     * here.
-     */
-    mask =3D ~((1UL << cur_order) - 1);
-    amfn =3D _mfn(mfn_x(*mfn) & mask);
-    gfn =3D _gfn(gfn_l & mask);
-
-    /* Override the altp2m entry with its default access. */
-    *p2ma =3D ap2m->default_access;
-
-    rc =3D p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
-    p2m_unlock(ap2m);
-
-    if ( rc )
-    {
-        gprintk(XENLOG_ERR,
-                "failed to set entry for %"PRI_gfn" -> %"PRI_mfn" altp2m=
 %u, rc %d\n",
-                gfn_l, mfn_x(amfn), vcpu_altp2m(current).p2midx, rc);
-        domain_crash(ap2m->domain);
-    }
-
-    return true;
-}
-
-enum altp2m_reset_type {
-    ALTP2M_RESET,
-    ALTP2M_DEACTIVATE
-};
-
-static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
-                             enum altp2m_reset_type reset_type)
-{
-    struct p2m_domain *p2m;
-
-    ASSERT(idx < MAX_ALTP2M);
-    p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
-
-    p2m_lock(p2m);
-
-    p2m_flush_table_locked(p2m);
-
-    if ( reset_type =3D=3D ALTP2M_DEACTIVATE )
-        p2m_free_logdirty(p2m);
-
-    /* Uninit and reinit ept to force TLB shootdown */
-    ept_p2m_uninit(p2m);
-    ept_p2m_init(p2m);
-
-    p2m->min_remapped_gfn =3D gfn_x(INVALID_GFN);
-    p2m->max_remapped_gfn =3D 0;
-
-    p2m_unlock(p2m);
-}
-
-void p2m_flush_altp2m(struct domain *d)
-{
-    unsigned int i;
-
-    altp2m_list_lock(d);
-
-    for ( i =3D 0; i < MAX_ALTP2M; i++ )
-    {
-        p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
-        d->arch.altp2m_eptp[i] =3D mfn_x(INVALID_MFN);
-        d->arch.altp2m_visible_eptp[i] =3D mfn_x(INVALID_MFN);
-    }
-
-    altp2m_list_unlock(d);
-}
-
-static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
-                               p2m_access_t hvmmem_default_access)
-{
-    struct p2m_domain *hostp2m, *p2m;
-    int rc;
-
-    ASSERT(idx < MAX_ALTP2M);
-
-    p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
-    hostp2m =3D p2m_get_hostp2m(d);
-
-    p2m_lock(p2m);
-
-    rc =3D p2m_init_logdirty(p2m);
-
-    if ( rc )
-        goto out;
-
-    /* The following is really just a rangeset copy. */
-    rc =3D rangeset_merge(p2m->logdirty_ranges, hostp2m->logdirty_ranges=
);
-
-    if ( rc )
-    {
-        p2m_free_logdirty(p2m);
-        goto out;
-    }
-
-    p2m->default_access =3D hvmmem_default_access;
-    p2m->domain =3D hostp2m->domain;
-    p2m->global_logdirty =3D hostp2m->global_logdirty;
-    p2m->min_remapped_gfn =3D gfn_x(INVALID_GFN);
-    p2m->max_mapped_pfn =3D p2m->max_remapped_gfn =3D 0;
-
-    p2m_init_altp2m_ept(d, idx);
-
- out:
-    p2m_unlock(p2m);
-
-    return rc;
-}
-
-int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
-{
-    int rc =3D -EINVAL;
-    struct p2m_domain *hostp2m =3D p2m_get_hostp2m(d);
-
-    if ( idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
-        return rc;
-
-    altp2m_list_lock(d);
-
-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D=3D
-         mfn_x(INVALID_MFN) )
-        rc =3D p2m_activate_altp2m(d, idx, hostp2m->default_access);
-
-    altp2m_list_unlock(d);
-    return rc;
-}
-
-int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
-                         xenmem_access_t hvmmem_default_access)
-{
-    int rc =3D -EINVAL;
-    unsigned int i;
-    p2m_access_t a;
-    struct p2m_domain *hostp2m =3D p2m_get_hostp2m(d);
-
-    if ( hvmmem_default_access > XENMEM_access_default ||
-         !xenmem_access_to_p2m_access(hostp2m, hvmmem_default_access, &a=
) )
-        return rc;
-
-    altp2m_list_lock(d);
-
-    for ( i =3D 0; i < MAX_ALTP2M; i++ )
-    {
-        if ( d->arch.altp2m_eptp[i] !=3D mfn_x(INVALID_MFN) )
-            continue;
-
-        rc =3D p2m_activate_altp2m(d, i, a);
-
-        if ( !rc )
-            *idx =3D i;
-
-        break;
-    }
-
-    altp2m_list_unlock(d);
-    return rc;
-}
-
-int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
-{
-    struct p2m_domain *p2m;
-    int rc =3D -EBUSY;
-
-    if ( !idx || idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) =
)
-        return rc;
-
-    rc =3D domain_pause_except_self(d);
-    if ( rc )
-        return rc;
-
-    rc =3D -EBUSY;
-    altp2m_list_lock(d);
-
-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=3D
-         mfn_x(INVALID_MFN) )
-    {
-        p2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
-
-        if ( !_atomic_read(p2m->active_vcpus) )
-        {
-            p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
-            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D
-                mfn_x(INVALID_MFN);
-            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP=
)] =3D
-                mfn_x(INVALID_MFN);
-            rc =3D 0;
-        }
-    }
-
-    altp2m_list_unlock(d);
-
-    domain_unpause_except_self(d);
-
-    return rc;
-}
-
-int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
-{
-    struct vcpu *v;
-    int rc =3D -EINVAL;
-
-    if ( idx >=3D MAX_ALTP2M )
-        return rc;
-
-    rc =3D domain_pause_except_self(d);
-    if ( rc )
-        return rc;
-
-    rc =3D -EINVAL;
-    altp2m_list_lock(d);
-
-    if ( d->arch.altp2m_visible_eptp[idx] !=3D mfn_x(INVALID_MFN) )
-    {
-        for_each_vcpu( d, v )
-            if ( p2m_set_altp2m(v, idx) )
-                altp2m_vcpu_update_p2m(v);
-
-        rc =3D 0;
-    }
-
-    altp2m_list_unlock(d);
-
-    domain_unpause_except_self(d);
-
-    return rc;
-}
-
-int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
-                          gfn_t old_gfn, gfn_t new_gfn)
-{
-    struct p2m_domain *hp2m, *ap2m;
-    p2m_access_t a;
-    p2m_type_t t;
-    mfn_t mfn;
-    int rc =3D -EINVAL;
-
-    if ( idx >=3D  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =3D=3D
-         mfn_x(INVALID_MFN) )
-        return rc;
-
-    hp2m =3D p2m_get_hostp2m(d);
-    ap2m =3D array_access_nospec(d->arch.altp2m_p2m, idx);
-
-    p2m_lock(hp2m);
-    p2m_lock(ap2m);
-
-    if ( gfn_eq(new_gfn, INVALID_GFN) )
-    {
-        mfn =3D ap2m->get_entry(ap2m, old_gfn, &t, &a, 0, NULL, NULL);
-        rc =3D mfn_valid(mfn)
-             ? p2m_remove_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K)
-             : 0;
-        goto out;
-    }
-
-    rc =3D altp2m_get_effective_entry(ap2m, old_gfn, &mfn, &t, &a,
-                                    AP2MGET_prepopulate);
-    if ( rc )
-        goto out;
-
-    rc =3D altp2m_get_effective_entry(ap2m, new_gfn, &mfn, &t, &a,
-                                    AP2MGET_query);
-    if ( rc )
-        goto out;
-
-    if ( !ap2m->set_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K, t, a,
-                          (current->domain !=3D d)) )
-    {
-        rc =3D 0;
-
-        if ( gfn_x(new_gfn) < ap2m->min_remapped_gfn )
-            ap2m->min_remapped_gfn =3D gfn_x(new_gfn);
-        if ( gfn_x(new_gfn) > ap2m->max_remapped_gfn )
-            ap2m->max_remapped_gfn =3D gfn_x(new_gfn);
-    }
-
- out:
-    p2m_unlock(ap2m);
-    p2m_unlock(hp2m);
-    return rc;
-}
-
-int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
-                                mfn_t mfn, unsigned int page_order,
-                                p2m_type_t p2mt, p2m_access_t p2ma)
-{
-    struct p2m_domain *p2m;
-    unsigned int i;
-    unsigned int reset_count =3D 0;
-    unsigned int last_reset_idx =3D ~0;
-    int ret =3D 0;
-
-    if ( !altp2m_active(d) )
-        return 0;
-
-    altp2m_list_lock(d);
-
-    for ( i =3D 0; i < MAX_ALTP2M; i++ )
-    {
-        p2m_type_t t;
-        p2m_access_t a;
-
-        if ( d->arch.altp2m_eptp[i] =3D=3D mfn_x(INVALID_MFN) )
-            continue;
-
-        p2m =3D d->arch.altp2m_p2m[i];
-
-        /* Check for a dropped page that may impact this altp2m */
-        if ( mfn_eq(mfn, INVALID_MFN) &&
-             gfn_x(gfn) + (1UL << page_order) > p2m->min_remapped_gfn &&
-             gfn_x(gfn) <=3D p2m->max_remapped_gfn )
-        {
-            if ( !reset_count++ )
-            {
-                p2m_reset_altp2m(d, i, ALTP2M_RESET);
-                last_reset_idx =3D i;
-            }
-            else
-            {
-                /* At least 2 altp2m's impacted, so reset everything */
-                for ( i =3D 0; i < MAX_ALTP2M; i++ )
-                {
-                    if ( i =3D=3D last_reset_idx ||
-                         d->arch.altp2m_eptp[i] =3D=3D mfn_x(INVALID_MFN=
) )
-                        continue;
-
-                    p2m_reset_altp2m(d, i, ALTP2M_RESET);
-                }
-
-                ret =3D 0;
-                break;
-            }
-        }
-        else if ( !mfn_eq(get_gfn_type_access(p2m, gfn_x(gfn), &t, &a, 0=
,
-                                              NULL), INVALID_MFN) )
-        {
-            int rc =3D p2m_set_entry(p2m, gfn, mfn, page_order, p2mt, p2=
ma);
-
-            /* Best effort: Don't bail on error. */
-            if ( !ret )
-                ret =3D rc;
-
-            p2m_put_gfn(p2m, gfn);
-        }
-        else
-            p2m_put_gfn(p2m, gfn);
-    }
-
-    altp2m_list_unlock(d);
-
-    return ret;
-}
-
 /*** Audit ***/
=20
 #if P2M_AUDIT
@@ -2540,159 +2065,6 @@ int xenmem_add_to_physmap_one(
     return rc;
 }
=20
-/*
- * Set/clear the #VE suppress bit for a page.  Only available on VMX.
- */
-int p2m_set_suppress_ve(struct domain *d, gfn_t gfn, bool suppress_ve,
-                        unsigned int altp2m_idx)
-{
-    int rc;
-    struct xen_hvm_altp2m_suppress_ve_multi sve =3D {
-        altp2m_idx, suppress_ve, 0, 0, gfn_x(gfn), gfn_x(gfn), 0
-    };
-
-    if ( !(rc =3D p2m_set_suppress_ve_multi(d, &sve)) )
-        rc =3D sve.first_error;
-
-    return rc;
-}
-
-/*
- * Set/clear the #VE suppress bit for multiple pages.  Only available on=
 VMX.
- */
-int p2m_set_suppress_ve_multi(struct domain *d,
-                              struct xen_hvm_altp2m_suppress_ve_multi *s=
ve)
-{
-    struct p2m_domain *host_p2m =3D p2m_get_hostp2m(d);
-    struct p2m_domain *ap2m =3D NULL;
-    struct p2m_domain *p2m =3D host_p2m;
-    uint64_t start =3D sve->first_gfn;
-    int rc =3D 0;
-
-    if ( sve->view > 0 )
-    {
-        if ( sve->view >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP=
) ||
-             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)=
] =3D=3D
-             mfn_x(INVALID_MFN) )
-            return -EINVAL;
-
-        p2m =3D ap2m =3D array_access_nospec(d->arch.altp2m_p2m, sve->vi=
ew);
-    }
-
-    p2m_lock(host_p2m);
-
-    if ( ap2m )
-        p2m_lock(ap2m);
-
-    while ( sve->last_gfn >=3D start )
-    {
-        p2m_access_t a;
-        p2m_type_t t;
-        mfn_t mfn;
-        int err =3D 0;
-
-        if ( (err =3D altp2m_get_effective_entry(p2m, _gfn(start), &mfn,=
 &t, &a,
-                                               AP2MGET_query)) &&
-             !sve->first_error )
-        {
-            sve->first_error_gfn =3D start; /* Save the gfn of the first=
 error */
-            sve->first_error =3D err; /* Save the first error code */
-        }
-
-        if ( !err && (err =3D p2m->set_entry(p2m, _gfn(start), mfn,
-                                           PAGE_ORDER_4K, t, a,
-                                           sve->suppress_ve)) &&
-             !sve->first_error )
-        {
-            sve->first_error_gfn =3D start; /* Save the gfn of the first=
 error */
-            sve->first_error =3D err; /* Save the first error code */
-        }
-
-        /* Check for continuation if it's not the last iteration. */
-        if ( sve->last_gfn >=3D ++start && hypercall_preempt_check() )
-        {
-            rc =3D -ERESTART;
-            break;
-        }
-    }
-
-    sve->first_gfn =3D start;
-
-    if ( ap2m )
-        p2m_unlock(ap2m);
-
-    p2m_unlock(host_p2m);
-
-    return rc;
-}
-
-int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
-                        unsigned int altp2m_idx)
-{
-    struct p2m_domain *host_p2m =3D p2m_get_hostp2m(d);
-    struct p2m_domain *ap2m =3D NULL;
-    struct p2m_domain *p2m;
-    mfn_t mfn;
-    p2m_access_t a;
-    p2m_type_t t;
-    int rc =3D 0;
-
-    if ( altp2m_idx > 0 )
-    {
-        if ( altp2m_idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPT=
P) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP=
)] =3D=3D
-             mfn_x(INVALID_MFN) )
-            return -EINVAL;
-
-        p2m =3D ap2m =3D array_access_nospec(d->arch.altp2m_p2m, altp2m_=
idx);
-    }
-    else
-        p2m =3D host_p2m;
-
-    gfn_lock(host_p2m, gfn, 0);
-
-    if ( ap2m )
-        p2m_lock(ap2m);
-
-    mfn =3D p2m->get_entry(p2m, gfn, &t, &a, 0, NULL, suppress_ve);
-    if ( !mfn_valid(mfn) )
-        rc =3D -ESRCH;
-
-    if ( ap2m )
-        p2m_unlock(ap2m);
-
-    gfn_unlock(host_p2m, gfn, 0);
-
-    return rc;
-}
-
-int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m=
_idx,
-                                   uint8_t visible)
-{
-    int rc =3D 0;
-
-    altp2m_list_lock(d);
-
-    /*
-     * Eptp index is correlated with altp2m index and should not exceed
-     * min(MAX_ALTP2M, MAX_EPTP).
-     */
-    if ( altp2m_idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) |=
|
-         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =3D=
=3D
-         mfn_x(INVALID_MFN) )
-        rc =3D -EINVAL;
-    else if ( visible )
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_E=
PTP)] =3D
-            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)=
];
-    else
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_E=
PTP)] =3D
-            mfn_x(INVALID_MFN);
-
-    altp2m_list_unlock(d);
-
-    return rc;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/p2m.h b/xen/arch/x86/mm/p2m.h
index 04308cfb6d..635f5a7f45 100644
--- a/xen/arch/x86/mm/p2m.h
+++ b/xen/arch/x86/mm/p2m.h
@@ -22,6 +22,9 @@ static inline void p2m_free_logdirty(struct p2m_domain =
*p2m) {}
 int p2m_init_altp2m(struct domain *d);
 void p2m_teardown_altp2m(struct domain *d);
=20
+void p2m_flush_table_locked(struct p2m_domain *p2m);
+int __must_check p2m_remove_entry(struct p2m_domain *p2m, gfn_t gfn, mfn=
_t mfn,
+                                  unsigned int page_order);
 void p2m_nestedp2m_init(struct p2m_domain *p2m);
 int p2m_init_nestedp2m(struct domain *d);
 void p2m_teardown_nestedp2m(struct domain *d);
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:08:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:08:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722039.1125845 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AcW-0001qu-Ll; Wed, 15 May 2024 09:08:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722039.1125845; Wed, 15 May 2024 09:08:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AcW-0001qn-JG; Wed, 15 May 2024 09:08:20 +0000
Received: by outflank-mailman (input) for mailman id 722039;
 Wed, 15 May 2024 09:08:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AcV-0001qh-9A
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:08:19 +0000
Received: from pb-smtp20.pobox.com (pb-smtp20.pobox.com [173.228.157.52])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aa98e4b2-129a-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:08:18 +0200 (CEST)
Received: from pb-smtp20.pobox.com (unknown [127.0.0.1])
 by pb-smtp20.pobox.com (Postfix) with ESMTP id 547F11BBAA;
 Wed, 15 May 2024 05:08:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp20.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp20.pobox.com (Postfix) with ESMTP id 4C5F31BBA9;
 Wed, 15 May 2024 05:08:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp20.pobox.com (Postfix) with ESMTPSA id D60251BB9F;
 Wed, 15 May 2024 05:08:12 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa98e4b2-129a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=4MpdoyoH5pC3p0CFUByX+/dhz
	ItVV3D2aq0lad6vARM=; b=slkyQBGbA00raAtKOXj3hYlqwltBL6XIVBq5bBpXU
	nukisjagm1x3pCikaxgkYmrQEGV4gla6hIW/It/FJpHcGHXDhX9VkHLRepkzC1bX
	6KhU1LL0k6Ftduu70/WXtIF6QbzlBD4ea1Ch9w3+jJFF3JCv/t+nZHwBIXUZNKTx
	ZQ=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Tamas K Lengyel <tamas@tklengyel.com>
Subject: [XEN PATCH v2 05/15] x86: introduce CONFIG_ALTP2M Kconfig option
Date: Wed, 15 May 2024 12:08:09 +0300
Message-Id: <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 A8418106-129A-11EF-8833-F515D2CDFF5E-90055647!pb-smtp20.pobox.com
Content-Transfer-Encoding: quoted-printable

Add new option to make altp2m code inclusion optional.
Currently altp2m support provided for VT-d only, so option is dependant o=
n VMX.

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Tamas K Lengyel <tamas@tklengyel.com>
---
 xen/arch/x86/Kconfig | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 8c9f8431f0..2872b031a7 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -358,6 +358,11 @@ config REQUIRE_NX
 	  was unavailable. However, if enabled, Xen will no longer boot on
 	  any CPU which is lacking NX support.
=20
+config ALTP2M
+	bool "Alternate P2M support"
+	def_bool y
+	depends on VMX && EXPERT
+
 endmenu
=20
 source "common/Kconfig"
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:09:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:09:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722042.1125856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AdZ-0002Ni-Uq; Wed, 15 May 2024 09:09:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722042.1125856; Wed, 15 May 2024 09:09:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AdZ-0002Nb-Rv; Wed, 15 May 2024 09:09:25 +0000
Received: by outflank-mailman (input) for mailman id 722042;
 Wed, 15 May 2024 09:09:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7AdY-0002NQ-4q
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:09:24 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d1fb0662-129a-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:09:23 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59e4136010so139778666b.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:09:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01501sm828618266b.165.2024.05.15.02.09.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 02:09:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1fb0662-129a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715764162; x=1716368962; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=NwWYu6W19dcNsz5ctSGjawhXFHCqIvASoM6FkqWc5nE=;
        b=H244GmrlFlXTpnc4I0ieW+o62Fctc7l+fs8RK44epGkIDaWHifz1WCRrRXEeRUWblw
         VV49JTEymj7kBYpuZu26va2MO1kPUJDI0Lo2RGA3p5u1o5h05O1I++6+X0STZGu3ChH/
         OJ6cQ6WdaK1tP+TrR3V49tqU4WLsNObpCbGbvwacvLwqMjASghWYdcQ/XDpEcWhkhqx3
         k5muj/1cZuUErVaQe5ItTgdow2cuR3YNXqLL4IlCKJC31kE1kt7qmoljK7tnx2ST7ikO
         +equ+T7wCqPtHbmAIHNFyPu1p6SuGZo6PIVzfFxhiJF41kEU3ICTGVD7SNlsfpKwRCdC
         +ntw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715764162; x=1716368962;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=NwWYu6W19dcNsz5ctSGjawhXFHCqIvASoM6FkqWc5nE=;
        b=G+E4CCmIYH83oipS6XL8FQ/kEYmBlUKhIIpN1L2uqV+ivzhJ2bMILA8lyYF+VIaYQv
         yeWfwdf0r1qXeKquvb49W7Lgg4p9jnzpvYi9HXfHza9xrM2y2qPVAitFkkfFwPtFGp15
         WDcD10MTD3s+a6p6qW2H1A5VPiU8LaIO2dgxgw8RUkGBtaeE4ia8D5WIjYv28TKe4PqF
         +hWeNA8zllhAzQ950S+C3569w1aMc1wLAZKmCfjvMWz0ByZesXyMUMMKg+tCZukom1MA
         TapPXbHQKM0PuQKjwomFjQL9n3kOBaQU7OuLx1lVZ5oaD/qb6agcXakZuqS23H0l+JWK
         ZM5w==
X-Forwarded-Encrypted: i=1; AJvYcCV4HWQL3Txtp9l6R/ck5F16ywMdDv8eF4zjgjlYtkOBY5gub+je/C+u7AghSnDFQwRJxAGv6HF/nT3uLwTitiney217EkCKouZm2dVoNhc=
X-Gm-Message-State: AOJu0YwFLj075qx3LE/0O3TiRQ9Vux8d+Bh7fNP+oLynXkbeYVu7vMiQ
	GJJ8J9+jX//wTDGxsO4gwvswFdnXaIolYVnxX4u13RSbm3VOSfNQAlkUxmnoqA==
X-Google-Smtp-Source: AGHT+IFwyo9gv1k9D6Y1o2Dg+qJsvqZWAr7/yB+qqiMomzF1EabSOvY5V46Rf4bj7WK19Pp3GhqY+Q==
X-Received: by 2002:a17:906:d207:b0:a59:bacc:b082 with SMTP id a640c23a62f3a-a5a2d65f32fmr910496566b.60.1715764162415;
        Wed, 15 May 2024 02:09:22 -0700 (PDT)
Message-ID: <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
Date: Wed, 15 May 2024 11:09:21 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 12:15, Oleksii Kurochko wrote:
> Changes in V9:
>  - update return type of fls and flsl() to unsigned int to be aligned with other
>    bit ops.

But this then needs carrying through to ...

> --- a/xen/arch/arm/include/asm/arm64/bitops.h
> +++ b/xen/arch/arm/include/asm/arm64/bitops.h
> @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long __ffs(unsigned long word)
>   */
>  #define ffz(x)  __ffs(~(x))
>  
> -static inline int flsl(unsigned long x)
> +static inline int arch_flsl(unsigned long x)

... e.g. here. You don't want to introduce signed/unsigned mismatches.

Also why do you keep "inline" here, while ...

> --- a/xen/arch/x86/include/asm/bitops.h
> +++ b/xen/arch/x86/include/asm/bitops.h
> @@ -425,7 +425,7 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
>   *
>   * This is defined the same way as ffs.
>   */
> -static inline int flsl(unsigned long x)
> +static always_inline int arch_flsl(unsigned long x)

... you switch to always_inline here?

(replying out of order)

> --- a/xen/arch/arm/include/asm/arm32/bitops.h
> +++ b/xen/arch/arm/include/asm/arm32/bitops.h
> @@ -1,7 +1,7 @@
>  #ifndef _ARM_ARM32_BITOPS_H
>  #define _ARM_ARM32_BITOPS_H
>  
> -#define flsl fls
> +#define arch_flsl fls

It's the Arm maintainers to ultimately judge, but I'd be inclined to suggest

#define arch_flsl arch_fls

instead. That's not only behaviorally closer to what was there before, but
also reduces (a tiny bit) the amount of work the compiler needs to carry out.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:10:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:10:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722045.1125866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AeW-000444-6d; Wed, 15 May 2024 09:10:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722045.1125866; Wed, 15 May 2024 09:10:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AeW-00043x-45; Wed, 15 May 2024 09:10:24 +0000
Received: by outflank-mailman (input) for mailman id 722045;
 Wed, 15 May 2024 09:10:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AeV-0002ND-0n
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:10:23 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3a04abb-129a-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 11:10:21 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id F0DCA35309;
 Wed, 15 May 2024 05:10:18 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id E724035308;
 Wed, 15 May 2024 05:10:18 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id EB06535307;
 Wed, 15 May 2024 05:10:17 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3a04abb-129a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=f44NE1zYxVTaI52BPRLBMRpI7
	SYEqvO5KN8XxTWD4lc=; b=L5ANc3ibwOg99vpedtd13LOXpq8+AHFs/mozX2pID
	Ly25w6xMYD0VQvBKjxlo6a6MLcrePO6FIhfHZ2XSW7DbwTcKLWbcM/5iN5fSm0co
	U6uggbJqays0D3L3dM90Ly+riCR8YJ8d8Zvmi2OoSptqmAVtrJYActm/Jec9F15v
	WM=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Tamas K Lengyel <tamas@tklengyel.com>
Subject: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with CONFIG_ALTP2M option
Date: Wed, 15 May 2024 12:10:16 +0300
Message-Id: <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 F2D1549E-129A-11EF-A0C8-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Instead of using generic CONFIG_HVM option switch to a bit more specific
CONFIG_ALTP2M option for altp2m support. Also guard altp2m routines, so t=
hat
they can be disabled completely in the build -- when target platform does=
 not
actually support altp2m (AMD-V & ARM as of now).

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Tamas K Lengyel <tamas@tklengyel.com>
---
changes in v2:
 - use separate CONFIG_ALTP2M option instead of CONFIG_VMX
---
 xen/arch/x86/include/asm/altp2m.h  |  5 ++++-
 xen/arch/x86/include/asm/hvm/hvm.h |  2 +-
 xen/arch/x86/include/asm/p2m.h     | 17 ++++++++++++++++-
 xen/arch/x86/mm/Makefile           |  2 +-
 4 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/include/asm/altp2m.h b/xen/arch/x86/include/asm=
/altp2m.h
index e5e59cbd68..092b13e231 100644
--- a/xen/arch/x86/include/asm/altp2m.h
+++ b/xen/arch/x86/include/asm/altp2m.h
@@ -7,7 +7,7 @@
 #ifndef __ASM_X86_ALTP2M_H
 #define __ASM_X86_ALTP2M_H
=20
-#ifdef CONFIG_HVM
+#ifdef CONFIG_ALTP2M
=20
 #include <xen/types.h>
 #include <xen/sched.h>         /* for struct vcpu, struct domain */
@@ -38,7 +38,10 @@ static inline bool altp2m_active(const struct domain *=
d)
 }
=20
 /* Only declaration is needed. DCE will optimise it out when linking. */
+void altp2m_vcpu_initialise(struct vcpu *v);
+void altp2m_vcpu_destroy(struct vcpu *v);
 uint16_t altp2m_vcpu_idx(const struct vcpu *v);
+int altp2m_vcpu_enable_ve(struct vcpu *v, gfn_t gfn);
 void altp2m_vcpu_disable_ve(struct vcpu *v);
=20
 #endif
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/as=
m/hvm/hvm.h
index 0c9e6f1564..4f03dd7af8 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -670,7 +670,7 @@ static inline bool hvm_hap_supported(void)
 /* returns true if hardware supports alternate p2m's */
 static inline bool hvm_altp2m_supported(void)
 {
-    return hvm_funcs.caps.altp2m;
+    return IS_ENABLED(CONFIG_ALTP2M) && hvm_funcs.caps.altp2m;
 }
=20
 /* Returns true if we have the minimum hardware requirements for nested =
virt */
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2=
m.h
index 111badf89a..855e69d24a 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -581,9 +581,9 @@ static inline gfn_t mfn_to_gfn(const struct domain *d=
, mfn_t mfn)
         return _gfn(mfn_x(mfn));
 }
=20
-#ifdef CONFIG_HVM
 #define AP2MGET_prepopulate true
 #define AP2MGET_query false
+#ifdef CONFIG_ALTP2M
=20
 /*
  * Looks up altp2m entry. If the entry is not found it looks up the entr=
y in
@@ -593,6 +593,15 @@ static inline gfn_t mfn_to_gfn(const struct domain *=
d, mfn_t mfn)
 int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t=
 *mfn,
                                p2m_type_t *t, p2m_access_t *a,
                                bool prepopulate);
+#else
+static inline int altp2m_get_effective_entry(struct p2m_domain *ap2m,
+                                             gfn_t gfn, mfn_t *mfn,
+                                             p2m_type_t *t, p2m_access_t=
 *a,
+                                             bool prepopulate)
+{
+    ASSERT_UNREACHABLE();
+    return -EOPNOTSUPP;
+}
 #endif
=20
 /* Init the datastructures for later use by the p2m code */
@@ -909,8 +918,14 @@ static inline bool p2m_set_altp2m(struct vcpu *v, un=
signed int idx)
 /* Switch alternate p2m for a single vcpu */
 bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx);
=20
+#ifdef CONFIG_ALTP2M
 /* Check to see if vcpu should be switched to a different p2m. */
 void p2m_altp2m_check(struct vcpu *v, uint16_t idx);
+#else
+static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
+{
+}
+#endif
=20
 /* Flush all the alternate p2m's for a domain */
 void p2m_flush_altp2m(struct domain *d);
diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile
index 0128ca7ab6..d7d57b8190 100644
--- a/xen/arch/x86/mm/Makefile
+++ b/xen/arch/x86/mm/Makefile
@@ -1,7 +1,7 @@
 obj-y +=3D shadow/
 obj-$(CONFIG_HVM) +=3D hap/
=20
-obj-$(CONFIG_HVM) +=3D altp2m.o
+obj-$(CONFIG_ALTP2M) +=3D altp2m.o
 obj-$(CONFIG_HVM) +=3D guest_walk_2.o guest_walk_3.o guest_walk_4.o
 obj-$(CONFIG_SHADOW_PAGING) +=3D guest_walk_4.o
 obj-$(CONFIG_MEM_ACCESS) +=3D mem_access.o
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:12:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:12:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722054.1125875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AgU-00057p-J8; Wed, 15 May 2024 09:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722054.1125875; Wed, 15 May 2024 09:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AgU-00057i-G4; Wed, 15 May 2024 09:12:26 +0000
Received: by outflank-mailman (input) for mailman id 722054;
 Wed, 15 May 2024 09:12:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AgT-00057a-5M
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:12:25 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3d5b5cf2-129b-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:12:24 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 8EBC035323;
 Wed, 15 May 2024 05:12:22 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 85A2E35322;
 Wed, 15 May 2024 05:12:22 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id BACBF35321;
 Wed, 15 May 2024 05:12:21 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d5b5cf2-129b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=qFhah2+IM6y4Lo8xEY359yzvT
	fmohoeQSwTfFz6nmRk=; b=eu/PFh/nU5wjH8/eyMA4htpoRljgr5TLhrPQZvP/J
	G+88V7JjokqbK2sBCJp2dJzEH2hY3ds/9POLMCcYD02J77U0idpYupFqgT1krSr4
	BTcHzpRthJVXAJrj7Mmo/8KXZuhrFC01iCkFk8FvDBD8mrbbu/rGPLOjgh7UwSSt
	Ag=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with CONFIG_{SVM/VMX}
Date: Wed, 15 May 2024 12:12:19 +0300
Message-Id: <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 3C9B4828-129B-11EF-89AB-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

As we now have SVM/VMX config options for enabling/disabling these featur=
es
completely in the build, it may be feasible to add build-time checks to
cpu_has_{svm,vmx} macros. These are used extensively thoughout HVM code, =
so
we won't have to add extra #ifdef-s to check whether svm/vmx has been ena=
bled,
while DCE cleans up calls to vmx/svm functions, if their code not being b=
uilt.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/x86/include/asm/cpufeature.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/include/asm/cpufeature.h b/xen/arch/x86/include=
/asm/cpufeature.h
index 9bc553681f..17f5aed000 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
 #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
 #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
 #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
-#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
+#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
+                                  boot_cpu_has(X86_FEATURE_VMX))
 #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
 #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
 #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
@@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
=20
 /* CPUID level 0x80000001.ecx */
 #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
-#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
+#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
+                                  boot_cpu_has(X86_FEATURE_SVM))
 #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
 #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
 #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:14:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:14:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722061.1125886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AiT-0005fE-TM; Wed, 15 May 2024 09:14:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722061.1125886; Wed, 15 May 2024 09:14:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AiT-0005f7-Qs; Wed, 15 May 2024 09:14:29 +0000
Received: by outflank-mailman (input) for mailman id 722061;
 Wed, 15 May 2024 09:14:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AiT-0005f1-6K
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:14:29 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 86f77944-129b-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 11:14:27 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 3162E35340;
 Wed, 15 May 2024 05:14:26 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 297753533F;
 Wed, 15 May 2024 05:14:26 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 24B493533E;
 Wed, 15 May 2024 05:14:25 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86f77944-129b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=QBVPADKDiu5x/oX6jZVkmGsvt
	8awM0kSX8QUYnF6Lfs=; b=TLIfs3Y4jdHaaq+/b+rvFIdlA1tnRxUpDOwtLlTt4
	huIdPD5Y4LoAp6EaFithiBn9sPSqHe/KqGbqBV4JRA/7M3iUG4lX1WQ4ffjPJN8J
	fMEZ4ura1w72RPxHsPXilErs1SNsjTrkD7jOnM9r1YeLLlozDmkGucnKz80hgay2
	fc=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 08/15] x86/vpmu: guard vmx/svm calls with cpu_has_{vmx,svm}
Date: Wed, 15 May 2024 12:14:22 +0300
Message-Id: <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 86266CC0-129B-11EF-BA87-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

If VMX/SVM disabled in the build, we may still want to have vPMU drivers =
for
PV guests. Yet some calls to vmx/svm-related routines needs to be guarded=
 then.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
 xen/arch/x86/cpu/vpmu_amd.c   |  8 ++++----
 xen/arch/x86/cpu/vpmu_intel.c | 20 ++++++++++----------
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
index db2fa420e1..40b0c8932f 100644
--- a/xen/arch/x86/cpu/vpmu_amd.c
+++ b/xen/arch/x86/cpu/vpmu_amd.c
@@ -290,7 +290,7 @@ static int cf_check amd_vpmu_save(struct vcpu *v,  bo=
ol to_guest)
     context_save(v);
=20
     if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
-         is_msr_bitmap_on(vpmu) )
+         is_msr_bitmap_on(vpmu) && cpu_has_svm )
         amd_vpmu_unset_msr_bitmap(v);
=20
     if ( to_guest )
@@ -363,7 +363,7 @@ static int cf_check amd_vpmu_do_wrmsr(unsigned int ms=
r, uint64_t msr_content)
             return 0;
         vpmu_set(vpmu, VPMU_RUNNING);
=20
-        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
+        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
              amd_vpmu_set_msr_bitmap(v);
     }
=20
@@ -372,7 +372,7 @@ static int cf_check amd_vpmu_do_wrmsr(unsigned int ms=
r, uint64_t msr_content)
         (is_pmu_enabled(msr_content) =3D=3D 0) && vpmu_is_set(vpmu, VPMU=
_RUNNING) )
     {
         vpmu_reset(vpmu, VPMU_RUNNING);
-        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
+        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
              amd_vpmu_unset_msr_bitmap(v);
         release_pmu_ownership(PMU_OWNER_HVM);
     }
@@ -415,7 +415,7 @@ static void cf_check amd_vpmu_destroy(struct vcpu *v)
 {
     struct vpmu_struct *vpmu =3D vcpu_vpmu(v);
=20
-    if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
+    if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
         amd_vpmu_unset_msr_bitmap(v);
=20
     xfree(vpmu->context);
diff --git a/xen/arch/x86/cpu/vpmu_intel.c b/xen/arch/x86/cpu/vpmu_intel.=
c
index cd414165df..10c34a5691 100644
--- a/xen/arch/x86/cpu/vpmu_intel.c
+++ b/xen/arch/x86/cpu/vpmu_intel.c
@@ -269,7 +269,7 @@ static inline void __core2_vpmu_save(struct vcpu *v)
     if ( !is_hvm_vcpu(v) )
         rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, core2_vpmu_cxt->global_statu=
s);
     /* Save MSR to private context to make it fork-friendly */
-    else if ( mem_sharing_enabled(v->domain) )
+    else if ( mem_sharing_enabled(v->domain) && cpu_has_vmx )
         vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
                            &core2_vpmu_cxt->global_ctrl);
 }
@@ -288,7 +288,7 @@ static int cf_check core2_vpmu_save(struct vcpu *v, b=
ool to_guest)
=20
     /* Unset PMU MSR bitmap to trap lazy load. */
     if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
-         cpu_has_vmx_msr_bitmap )
+         cpu_has_vmx && cpu_has_vmx_msr_bitmap )
         core2_vpmu_unset_msr_bitmap(v);
=20
     if ( to_guest )
@@ -333,7 +333,7 @@ static inline void __core2_vpmu_load(struct vcpu *v)
         wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, core2_vpmu_cxt->global_ctrl);
     }
     /* Restore MSR from context when used with a fork */
-    else if ( mem_sharing_is_fork(v->domain) )
+    else if ( mem_sharing_is_fork(v->domain) && cpu_has_vmx )
         vmx_write_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
                             core2_vpmu_cxt->global_ctrl);
 }
@@ -442,7 +442,7 @@ static int cf_check core2_vpmu_alloc_resource(struct =
vcpu *v)
     if ( !acquire_pmu_ownership(PMU_OWNER_HVM) )
         return 0;
=20
-    if ( is_hvm_vcpu(v) )
+    if ( is_hvm_vcpu(v) && cpu_has_vmx )
     {
         if ( vmx_add_host_load_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, 0) )
             goto out_err;
@@ -513,7 +513,7 @@ static int core2_vpmu_msr_common_check(u32 msr_index,=
 int *type, int *index)
         __core2_vpmu_load(current);
         vpmu_set(vpmu, VPMU_CONTEXT_LOADED);
=20
-        if ( is_hvm_vcpu(current) && cpu_has_vmx_msr_bitmap )
+        if ( is_hvm_vcpu(current) && cpu_has_vmx && cpu_has_vmx_msr_bitm=
ap )
             core2_vpmu_set_msr_bitmap(current);
     }
     return 1;
@@ -584,7 +584,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int =
msr, uint64_t msr_content)
         if ( msr_content & fixed_ctrl_mask )
             return -EINVAL;
=20
-        if ( is_hvm_vcpu(v) )
+        if ( is_hvm_vcpu(v) && cpu_has_vmx )
             vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
                                &core2_vpmu_cxt->global_ctrl);
         else
@@ -653,7 +653,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int =
msr, uint64_t msr_content)
             if ( blocked )
                 return -EINVAL;
=20
-            if ( is_hvm_vcpu(v) )
+            if ( is_hvm_vcpu(v) && cpu_has_vmx)
                 vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
                                    &core2_vpmu_cxt->global_ctrl);
             else
@@ -672,7 +672,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int =
msr, uint64_t msr_content)
         wrmsrl(msr, msr_content);
     else
     {
-        if ( is_hvm_vcpu(v) )
+        if ( is_hvm_vcpu(v) && cpu_has_vmx )
             vmx_write_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, msr_conten=
t);
         else
             wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, msr_content);
@@ -706,7 +706,7 @@ static int cf_check core2_vpmu_do_rdmsr(unsigned int =
msr, uint64_t *msr_content)
             *msr_content =3D core2_vpmu_cxt->global_status;
             break;
         case MSR_CORE_PERF_GLOBAL_CTRL:
-            if ( is_hvm_vcpu(v) )
+            if ( is_hvm_vcpu(v) && cpu_has_vmx )
                 vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, msr_con=
tent);
             else
                 rdmsrl(MSR_CORE_PERF_GLOBAL_CTRL, *msr_content);
@@ -808,7 +808,7 @@ static void cf_check core2_vpmu_destroy(struct vcpu *=
v)
     vpmu->context =3D NULL;
     xfree(vpmu->priv_context);
     vpmu->priv_context =3D NULL;
-    if ( is_hvm_vcpu(v) && cpu_has_vmx_msr_bitmap )
+    if ( is_hvm_vcpu(v) && cpu_has_vmx && cpu_has_vmx_msr_bitmap )
         core2_vpmu_unset_msr_bitmap(v);
     release_pmu_ownership(PMU_OWNER_HVM);
     vpmu_clear(vpmu);
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:16:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:16:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722067.1125896 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AkS-0006Fu-BG; Wed, 15 May 2024 09:16:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722067.1125896; Wed, 15 May 2024 09:16:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AkS-0006Fn-8E; Wed, 15 May 2024 09:16:32 +0000
Received: by outflank-mailman (input) for mailman id 722067;
 Wed, 15 May 2024 09:16:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AkQ-0006Ff-VH
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:16:30 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d03c5259-129b-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:16:30 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 1956435482;
 Wed, 15 May 2024 05:16:29 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 11C5A35481;
 Wed, 15 May 2024 05:16:29 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 6DC4E35480;
 Wed, 15 May 2024 05:16:28 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d03c5259-129b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=299ufJ2kZ2D6V92PsDN8HDonp
	5MtRuFSITeTwpBue9Q=; b=rb+6o5hp4FAI7YCe5i8W0vZBh6ACm1yvFfWcphJE3
	I43FEEViadokhM9ME3L2rzVfiBMtPhO3OHmhZ81bD7l2n4RYexuCyYHi/Zi8QUQ7
	Y2kIViauLGzLeW3zUm+//qflMIXP/AlhtRfLvko/qXCbHX9BeLEyU1tUAsvL46YT
	lA=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 09/15] x86/traps: clean up superfluous #idef-s
Date: Wed, 15 May 2024 12:16:26 +0300
Message-Id: <7f0b98062ce67ad8176670efbe3c3ebdb43d2b1c.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 CFA4903E-129B-11EF-85B5-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

Remove preprocessor checks for CONFIG_HVM option, because expressions cov=
ered
by these checks are already guarded by cpu_has_vmx, which itself depends
on CONFIG_HVM option (via CONFIG_VMX).

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
 xen/arch/x86/traps.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index d554c9d41e..7b8ee45edf 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -676,7 +676,6 @@ void vcpu_show_execution_state(struct vcpu *v)
=20
     vcpu_pause(v); /* acceptably dangerous */
=20
-#ifdef CONFIG_HVM
     /*
      * For VMX special care is needed: Reading some of the register stat=
e will
      * require VMCS accesses. Engaging foreign VMCSes involves acquiring=
 of a
@@ -689,7 +688,6 @@ void vcpu_show_execution_state(struct vcpu *v)
         ASSERT(!in_irq());
         vmx_vmcs_enter(v);
     }
-#endif
=20
     /* Prevent interleaving of output. */
     flags =3D console_lock_recursive_irqsave();
@@ -714,10 +712,8 @@ void vcpu_show_execution_state(struct vcpu *v)
         console_unlock_recursive_irqrestore(flags);
     }
=20
-#ifdef CONFIG_HVM
     if ( cpu_has_vmx && is_hvm_vcpu(v) )
         vmx_vmcs_exit(v);
-#endif
=20
     vcpu_unpause(v);
 }
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:18:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:18:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722073.1125906 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AmX-0006pe-NU; Wed, 15 May 2024 09:18:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722073.1125906; Wed, 15 May 2024 09:18:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AmX-0006pX-JH; Wed, 15 May 2024 09:18:41 +0000
Received: by outflank-mailman (input) for mailman id 722073;
 Wed, 15 May 2024 09:18:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AmV-0006pP-T5
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:18:39 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1c97be60-129c-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:18:38 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id B52891B823;
 Wed, 15 May 2024 05:18:36 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id ABE891B822;
 Wed, 15 May 2024 05:18:36 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 754F41B821;
 Wed, 15 May 2024 05:18:32 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c97be60-129c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=fy/w8iemesK+0sMkUtfBzP+Xq
	P828REQrPH5zx3qcTg=; b=Q3VLjGZbU7sn/9aAeJU9fMnGgN4K8zE0a09b+taR+
	nRuV90Rc40+i49K7NkjmPJV3JPrGZkiKyLJwAPV8foMCVwwFjBAPJ4tqRoF+5EcH
	F6YsmFjfI4043X4RKlIs29J/uwbmdQ6NJLZGkPWt1Hi20cLJ4sJ0VmcnQ6jQjFwo
	X8=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 10/15] x86/domain: clean up superfluous #idef-s
Date: Wed, 15 May 2024 12:18:29 +0300
Message-Id: <67d6604e8f66468c02f0c2e60315fc9251b69beb.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 1992272E-129C-11EF-BDB2-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

Remove preprocessor checks for CONFIG_HVM option, because expressions cov=
ered
by these checks are already guarded by cpu_has_svm, which itself depends
on CONFIG_HVM option (via CONFIG_SVM).

No functional change intended.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
 xen/arch/x86/domain.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 20e83cf38b..5c7fb7fc73 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1708,11 +1708,9 @@ static void load_segments(struct vcpu *n)
         if ( !(n->arch.flags & TF_kernel_mode) )
             SWAP(gsb, gss);
=20
-#ifdef CONFIG_HVM
         if ( cpu_has_svm && (uregs->fs | uregs->gs) <=3D 3 )
             fs_gs_done =3D svm_load_segs(n->arch.pv.ldt_ents, LDT_VIRT_S=
TART(n),
                                        n->arch.pv.fs_base, gsb, gss);
-#endif
     }
=20
     if ( !fs_gs_done )
@@ -2025,7 +2023,7 @@ static void __context_switch(void)
=20
     write_ptbase(n);
=20
-#if defined(CONFIG_PV) && defined(CONFIG_HVM)
+#if defined(CONFIG_PV)
     /* Prefetch the VMCB if we expect to use it later in the context swi=
tch */
     if ( cpu_has_svm && is_pv_64bit_domain(nd) && !is_idle_domain(nd) )
         svm_load_segs_prefetch();
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:20:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:20:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722079.1125916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AoU-0000Iz-1x; Wed, 15 May 2024 09:20:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722079.1125916; Wed, 15 May 2024 09:20:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AoT-0000In-Up; Wed, 15 May 2024 09:20:41 +0000
Received: by outflank-mailman (input) for mailman id 722079;
 Wed, 15 May 2024 09:20:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AoS-0000IZ-Lk
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:20:40 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 651df9bb-129c-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:20:39 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id C7026354B1;
 Wed, 15 May 2024 05:20:38 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id BFB1A354B0;
 Wed, 15 May 2024 05:20:38 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id E6416354AF;
 Wed, 15 May 2024 05:20:37 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 651df9bb-129c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=/ajn3Y23Ptep8PMnKlNl2X5Wf
	quTjK0FGfT5lFyKeSU=; b=QnFLjNbpEx2LlU/C+InuGlrAXqd3tZQQwI2HcsMzA
	fBenES8OC368m2TAhkh3M/kdbLk8cDyq4GTF2DW2p2wHbmODIgTSbR6YzOcUWgTh
	gw5lKkA66wNCOV6HHWp5J/k7C8++FxtaS6mZTgjYPVSDgPFk8endKLuTwOMPIfyz
	tM=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Subject: [XEN PATCH v2 11/15] x86/oprofile: guard svm specific symbols with CONFIG_SVM
Date: Wed, 15 May 2024 12:20:36 +0300
Message-Id: <8174a35669a8dffa10141c7fea64b7c1f6dfbe4e.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 645A633E-129C-11EF-964D-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

From: Xenia Ragiadakou <burzalodowa@gmail.com>

The symbol svm_stgi_label is AMD-V specific so guard its usage in common =
code
with CONFIG_SVM.

Since SVM depends on HVM, it can be used alone.
Also, use #ifdef instead of #if.

No functional change intended.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/x86/oprofile/op_model_athlon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/oprofile/op_model_athlon.c b/xen/arch/x86/oprof=
ile/op_model_athlon.c
index 69fd3fcc86..a9c7b87d67 100644
--- a/xen/arch/x86/oprofile/op_model_athlon.c
+++ b/xen/arch/x86/oprofile/op_model_athlon.c
@@ -320,7 +320,7 @@ static int cf_check athlon_check_ctrs(
 	struct vcpu *v =3D current;
 	unsigned int const nr_ctrs =3D model->num_counters;
=20
-#if CONFIG_HVM
+#ifdef CONFIG_SVM
 	struct cpu_user_regs *guest_regs =3D guest_cpu_user_regs();
=20
 	if (!guest_mode(regs) &&
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:22:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:22:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722085.1125926 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AqT-0001MC-DS; Wed, 15 May 2024 09:22:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722085.1125926; Wed, 15 May 2024 09:22:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AqT-0001M5-AW; Wed, 15 May 2024 09:22:45 +0000
Received: by outflank-mailman (input) for mailman id 722085;
 Wed, 15 May 2024 09:22:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AqR-0001Lx-VX
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:22:44 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ae898d4c-129c-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:22:43 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 065C2354C2;
 Wed, 15 May 2024 05:22:42 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id F17AB354C1;
 Wed, 15 May 2024 05:22:41 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 2DF7B354C0;
 Wed, 15 May 2024 05:22:40 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae898d4c-129c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=tJNFbcXuEgYbrw2Axr3CVP5ay
	5NL+SPdbHdUF/h0LRg=; b=jt8Mq1AGP4+oII64Bn3PXSibqbibUvfLBESky3FfJ
	90oYUrtwv20hIhw8vR43aOW1L4UqhXCclV9lxCHV42YCcMjF+5CjyitlCTJeY6uL
	HlqUr/gcpf1zIFom2JkHfd5oHIqu2Q1UqRgPHBRVhE0ioCMD7UtF7sVHvxTOHgGd
	c8=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [XEN PATCH v2 12/15] x86/vmx: guard access to cpu_has_vmx_* in common code
Date: Wed, 15 May 2024 12:22:39 +0300
Message-Id: <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 ADD01248-129C-11EF-BAA2-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

There're several places in common code, outside of arch/x86/hvm/vmx,
where cpu_has_vmx_* get accessed without checking if VMX present first.
We may want to guard these macros, as they read global variables defined
inside vmx-specific files -- so VMX can be made optional later on.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
---
Here I've tried a different approach from prev.patches [1,2] -- instead o=
f
modifying whole set of cpu_has_{svm/vmx}_* macros, we can:
 1) do not touch SVM part at all, because just as Andrew pointed out they=
're
used inside arch/x86/hvm/svm only.
 2) track several places in common code where cpu_has_vmx_* features are
checked out and guard them using cpu_has_vmx condition
 3) two of cpu_has_vmx_* macros being used in common code are checked in =
a bit
more tricky way, so instead of making complex conditionals even more comp=
licated,
we can instead integrate cpu_has_vmx condition inside these two macros.

This patch aims to replace [1,2] from v1 series by doing steps above.

 1. https://lore.kernel.org/xen-devel/20240416064402.3469959-1-Sergiy_Kib=
rik@epam.com/
 2. https://lore.kernel.org/xen-devel/20240416064606.3470052-1-Sergiy_Kib=
rik@epam.com/
---
changes in v2:
 - do not touch SVM code and macros
 - drop vmx_ctrl_has_feature()
 - guard cpu_has_vmx_* macros in common code instead
changes in v1:
 - introduced helper routine vmx_ctrl_has_feature() and used it for all
   cpu_has_vmx_* macros
---
 xen/arch/x86/hvm/hvm.c                  | 2 +-
 xen/arch/x86/hvm/viridian/viridian.c    | 4 ++--
 xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 4 ++--
 xen/arch/x86/traps.c                    | 5 +++--
 4 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c5..ab75de9779 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5180,7 +5180,7 @@ int hvm_debug_op(struct vcpu *v, int32_t op)
     {
         case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
         case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
-            if ( !cpu_has_monitor_trap_flag )
+            if ( !cpu_has_vmx || !cpu_has_monitor_trap_flag )
                 return -EOPNOTSUPP;
             break;
         default:
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viri=
dian/viridian.c
index 0496c52ed5..657c6a3ea7 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -196,7 +196,7 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint=
32_t leaf,
         res->a =3D CPUID4A_RELAX_TIMER_INT;
         if ( viridian_feature_mask(d) & HVMPV_hcall_remote_tlb_flush )
             res->a |=3D CPUID4A_HCALL_REMOTE_TLB_FLUSH;
-        if ( !cpu_has_vmx_apic_reg_virt )
+        if ( !cpu_has_vmx || !cpu_has_vmx_apic_reg_virt )
             res->a |=3D CPUID4A_MSR_BASED_APIC;
         if ( viridian_feature_mask(d) & HVMPV_hcall_ipi )
             res->a |=3D CPUID4A_SYNTHETIC_CLUSTER_IPI;
@@ -236,7 +236,7 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint=
32_t leaf,
=20
     case 6:
         /* Detected and in use hardware features. */
-        if ( cpu_has_vmx_virtualize_apic_accesses )
+        if ( cpu_has_vmx && cpu_has_vmx_virtualize_apic_accesses )
             res->a |=3D CPUID6A_APIC_OVERLAY;
         if ( cpu_has_vmx_msr_bitmap || (read_efer() & EFER_SVME) )
             res->a |=3D CPUID6A_MSR_BITMAPS;
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/inclu=
de/asm/hvm/vmx/vmcs.h
index 58140af691..aa05f9cf6e 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -306,7 +306,7 @@ extern u64 vmx_ept_vpid_cap;
 #define cpu_has_vmx_vnmi \
     (vmx_pin_based_exec_control & PIN_BASED_VIRTUAL_NMIS)
 #define cpu_has_vmx_msr_bitmap \
-    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
+    (cpu_has_vmx && vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_=
BITMAP)
 #define cpu_has_vmx_secondary_exec_control \
     (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
 #define cpu_has_vmx_tertiary_exec_control \
@@ -347,7 +347,7 @@ extern u64 vmx_ept_vpid_cap;
 #define cpu_has_vmx_vmfunc \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS)
 #define cpu_has_vmx_virt_exceptions \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
+    (cpu_has_vmx && vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_V=
IRT_EXCEPTIONS)
 #define cpu_has_vmx_pml \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
 #define cpu_has_vmx_mpx \
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 7b8ee45edf..3595bb379a 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1130,7 +1130,7 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, =
uint32_t leaf,
         if ( !is_hvm_domain(d) || subleaf !=3D 0 )
             break;
=20
-        if ( cpu_has_vmx_apic_reg_virt )
+        if ( cpu_has_vmx && cpu_has_vmx_apic_reg_virt )
             res->a |=3D XEN_HVM_CPUID_APIC_ACCESS_VIRT;
=20
         /*
@@ -1139,7 +1139,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, =
uint32_t leaf,
          * and wrmsr in the guest will run without VMEXITs (see
          * vmx_vlapic_msr_changed()).
          */
-        if ( cpu_has_vmx_virtualize_x2apic_mode &&
+        if ( cpu_has_vmx &&
+             cpu_has_vmx_virtualize_x2apic_mode &&
              cpu_has_vmx_apic_reg_virt &&
              cpu_has_vmx_virtual_intr_delivery )
             res->a |=3D XEN_HVM_CPUID_X2APIC_VIRT;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:24:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:24:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722088.1125936 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AsR-0001tP-Ns; Wed, 15 May 2024 09:24:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722088.1125936; Wed, 15 May 2024 09:24:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AsR-0001tI-LD; Wed, 15 May 2024 09:24:47 +0000
Received: by outflank-mailman (input) for mailman id 722088;
 Wed, 15 May 2024 09:24:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AsR-0001tC-2W
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:24:47 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f7d6d3ca-129c-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:24:46 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id F0554354E1;
 Wed, 15 May 2024 05:24:44 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id E83D2354E0;
 Wed, 15 May 2024 05:24:44 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 42EF7354DE;
 Wed, 15 May 2024 05:24:44 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7d6d3ca-129c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=lfpK0MUD/x59alsKeTEo/v19a
	RiX/271n/gqtVYSVVQ=; b=mgpjqU5zYcdWNnEYuplOJbgzNb2Etbw8ls45J8vYf
	dVAVR2IKZeSbAdmR5l3Hm3NcgWWdAcS7dZNdVZU6UlQs36pqRXldu0WI9ppq/p6P
	lfmUD6OoyKZYLfoB6Dg9Ft7FtRe6bsYAttvRh5jnueqWdPk/+tyRJJm924qBvuUY
	bk=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Subject: [XEN PATCH v2 13/15] x86/ioreq: guard VIO_realmode_completion with CONFIG_VMX
Date: Wed, 15 May 2024 12:24:42 +0300
Message-Id: <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 F72CFF8C-129C-11EF-9E53-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

From: Xenia Ragiadakou <burzalodowa@gmail.com>

VIO_realmode_completion is specific to vmx realmode, so guard the complet=
ion
handling code with CONFIG_VMX. Also, guard VIO_realmode_completion itself=
 by
CONFIG_VMX, instead of generic CONFIG_X86.

No functional change intended.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
changes in v1:
 - put VIO_realmode_completion enum under #ifdef CONFIG_VMX
---
 xen/arch/x86/hvm/emulate.c | 2 ++
 xen/arch/x86/hvm/ioreq.c   | 2 ++
 xen/include/xen/sched.h    | 2 +-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index ab1bc51683..d60b1f6f4d 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -2667,7 +2667,9 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt=
 *hvmemul_ctxt,
         break;
=20
     case VIO_mmio_completion:
+#ifdef CONFIG_VMX
     case VIO_realmode_completion:
+#endif
         BUILD_BUG_ON(sizeof(hvio->mmio_insn) < sizeof(hvmemul_ctxt->insn=
_buf));
         hvio->mmio_insn_bytes =3D hvmemul_ctxt->insn_buf_bytes;
         memcpy(hvio->mmio_insn, hvmemul_ctxt->insn_buf, hvio->mmio_insn_=
bytes);
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 4eb7a70182..b37bbd660b 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -33,6 +33,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion com=
pletion)
 {
     switch ( completion )
     {
+#ifdef CONFIG_VMX
     case VIO_realmode_completion:
     {
         struct hvm_emulate_ctxt ctxt;
@@ -43,6 +44,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion com=
pletion)
=20
         break;
     }
+#endif
=20
     default:
         ASSERT_UNREACHABLE();
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b841995..50a58fe428 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -152,7 +152,7 @@ enum vio_completion {
     VIO_no_completion,
     VIO_mmio_completion,
     VIO_pio_completion,
-#ifdef CONFIG_X86
+#ifdef CONFIG_VMX
     VIO_realmode_completion,
 #endif
 };
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:26:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:26:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722092.1125946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AuR-0002Qp-2k; Wed, 15 May 2024 09:26:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722092.1125946; Wed, 15 May 2024 09:26:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AuR-0002Qi-08; Wed, 15 May 2024 09:26:51 +0000
Received: by outflank-mailman (input) for mailman id 722092;
 Wed, 15 May 2024 09:26:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AuQ-0002Qc-7I
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:26:50 +0000
Received: from pb-smtp2.pobox.com (pb-smtp2.pobox.com [64.147.108.71])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 412e715d-129d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:26:49 +0200 (CEST)
Received: from pb-smtp2.pobox.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 2A79C354FB;
 Wed, 15 May 2024 05:26:48 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp2.pobox.com (Postfix) with ESMTP id 218BA354F8;
 Wed, 15 May 2024 05:26:48 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp2.pobox.com (Postfix) with ESMTPSA id 57FC3354F7;
 Wed, 15 May 2024 05:26:47 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 412e715d-129d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=QWpi+mcgV19u04TlYTDH6KKf8
	W1LgMYc9I836I7tdAs=; b=WL9hLhGSSOj29npYKRo+om4seXCql3jkvXEmLUT2+
	h/XCrDC8Ny2rtUE3fGsAZlUvpUtUA7SjW0GLPOJcAjeLwHFjShk/g71Rg45xqmWo
	aL/pEdZlpiaaoBmhRYkGGfT5ztyxf4M880EGVT/9cZVDZFwT0DB3VqrdfTKRwxP5
	EQ=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: [XEN PATCH v2 14/15] iommu/vt-d: guard vmx_pi_hooks_* calls with cpu_has_vmx
Date: Wed, 15 May 2024 12:26:45 +0300
Message-Id: <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 408AB73C-129D-11EF-88AA-25B3960A682E-90055647!pb-smtp2.pobox.com
Content-Transfer-Encoding: quoted-printable

VMX posted interrupts support can now be excluded from x86 build along wi=
th
VMX code itself, but still we may want to keep the possibility to use
VT-d IOMMU driver in non-HVM setups.
So we guard vmx_pi_hooks_{assign/deassign} with some checks for such a ca=
se.

No functional change intended here.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
 xen/drivers/passthrough/vtd/iommu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthroug=
h/vtd/iommu.c
index e13be244c1..ad78282250 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2772,7 +2772,7 @@ static int cf_check reassign_device_ownership(
=20
     if ( !QUARANTINE_SKIP(target, pdev->arch.vtd.pgd_maddr) )
     {
-        if ( !has_arch_pdevs(target) )
+        if ( cpu_has_vmx && !has_arch_pdevs(target) )
             vmx_pi_hooks_assign(target);
=20
 #ifdef CONFIG_PV
@@ -2806,7 +2806,7 @@ static int cf_check reassign_device_ownership(
     }
     if ( ret )
     {
-        if ( !has_arch_pdevs(target) )
+        if ( cpu_has_vmx && !has_arch_pdevs(target) )
             vmx_pi_hooks_deassign(target);
         return ret;
     }
@@ -2824,7 +2824,7 @@ static int cf_check reassign_device_ownership(
         write_unlock(&target->pci_lock);
     }
=20
-    if ( !has_arch_pdevs(source) )
+    if ( cpu_has_vmx && !has_arch_pdevs(source) )
         vmx_pi_hooks_deassign(source);
=20
     /*
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:29:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:29:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722096.1125956 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AwW-00033v-H8; Wed, 15 May 2024 09:29:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722096.1125956; Wed, 15 May 2024 09:29:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7AwW-00033o-ER; Wed, 15 May 2024 09:29:00 +0000
Received: by outflank-mailman (input) for mailman id 722096;
 Wed, 15 May 2024 09:28:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LJut=MS=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s7AwV-00033i-Cf
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:28:59 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8d253930-129d-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 11:28:57 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 2D1751B99F;
 Wed, 15 May 2024 05:28:55 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 241921B99E;
 Wed, 15 May 2024 05:28:55 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.126])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id BDBFA1B99D;
 Wed, 15 May 2024 05:28:51 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d253930-129d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=80FdUwFU0qesWxVN03JYJt5oG
	HTecFn91NcTkV99tSQ=; b=Nx08oJYmkt6CIH8GtCIdMtc70SlAKVFD//4Ch9H/o
	bAq+qsAb4hvNWEBb5/Y6nFL8sLNzNSSOMBeUVQoLQ4BJmTDdoCOqXcUPx9ozWlRd
	axPyxa1btcjLWshB3DvgYRWY4UoSkZZx7J5ZT2IrJOwqaFudljszZ532Sn2+QwYT
	TA=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Subject: [XEN PATCH v2 15/15] x86/hvm: make AMD-V and Intel VT-x support configurable
Date: Wed, 15 May 2024 12:28:48 +0300
Message-Id: <3ad7c0279da67e564713140fb5b247349cf4dccc.1715761386.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 8AB37632-129D-11EF-98F5-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

From: Xenia Ragiadakou <burzalodowa@gmail.com>

Provide the user with configuration control over the cpu virtualization s=
upport
in Xen by making SVM and VMX options user selectable.

To preserve the current default behavior, both options depend on HVM and
default to value of HVM.

To prevent users from unknowingly disabling virtualization support, make =
the
controls user selectable only if EXPERT is enabled.

No functional change intended.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
changes in v2:
 - remove dependency of build options IOMMU/AMD_IOMMU on VMX/SVM options
---
 xen/arch/x86/Kconfig | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 2872b031a7..62621c7271 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -123,10 +123,24 @@ config HVM
 	  If unsure, say Y.
=20
 config SVM
-	def_bool HVM
+	bool "AMD-V" if EXPERT
+	depends on HVM
+	default HVM
+	help
+	  Enables virtual machine extensions on platforms that implement the
+	  AMD Virtualization Technology (AMD-V).
+	  If your system includes a processor with AMD-V support, say Y.
+	  If in doubt, say Y.
=20
 config VMX
-	def_bool HVM
+	bool "Intel VT-x" if EXPERT
+	depends on HVM
+	default HVM
+	help
+	  Enables virtual machine extensions on platforms that implement the
+	  Intel Virtualization Technology (Intel VT-x).
+	  If your system includes a processor with Intel VT-x support, say Y.
+	  If in doubt, say Y.
=20
 config XEN_SHSTK
 	bool "Supervisor Shadow Stacks"
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 09:29:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:29:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722099.1125966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Awy-0003Zn-OY; Wed, 15 May 2024 09:29:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722099.1125966; Wed, 15 May 2024 09:29:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Awy-0003Zc-Li; Wed, 15 May 2024 09:29:28 +0000
Received: by outflank-mailman (input) for mailman id 722099;
 Wed, 15 May 2024 09:29:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Awx-0003Ui-Ci
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:29:27 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9f673f1c-129d-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:29:26 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-572ba002a6bso1423712a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:29:26 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01517sm830642166b.157.2024.05.15.02.29.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 02:29:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f673f1c-129d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715765366; x=1716370166; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LpcJ7sW1Zg4pRLASa07jt2b4VErmFG99QkQLm4bcba4=;
        b=DIsv7n/4hBnhmgruIymTx3nrgjmpxJeUdkkoMJade/ThesqK5jRFUNN3Albc2VUNUK
         QQB+Vwqfi31oWfZ5wZvF5BumO7R/A9lpT4Ktk1CyCLiUei0OZcxt/D0nqq886K6WbDwi
         hA6PpskvLkzh22TW0rB2rIoswGpPE162qKxgMlhuHpdsvuAbEuWU9jYPBWcKs2h0a5hy
         0D/bhOt9xImBQjUcjyU5psbq3lnNn2R9xLrF4emN2UhOaGtUGfY32wBI3wrr/x+gnIpH
         e2q7emumsifu37mom2kgQhyXI16b8VD/8vx0AWXs8W8NqbW/AYcsu+HFDEv49mj6K/Wy
         GHaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715765366; x=1716370166;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LpcJ7sW1Zg4pRLASa07jt2b4VErmFG99QkQLm4bcba4=;
        b=eU+YJlPCCCaai0o/AEw3jPONs0AmXPzzKMU1Xr1jAGCtNlAzpvR1c3dCQvpaKUnZfM
         OOKpbgEMeRLJs1nBes0nh2hUHyS8HtQfGCR7qD8Yiaa1Vi0GebsdQbPGiOTTs35OlJT4
         1Prw4MA4TcKNt65f9o7ZyQPOIpaw8LJf0a63TZLgnJgcQJT3zwOnoW0y1obrlMRqC6wx
         ynmqgHqLG8UPLl2UvqKc70Y3EVf8AEjoD6LFc5KaxAG32vxKZ8gZjbScFBirY+EUeoVj
         4qnfgGQxPNU9JSZ/y7HSLjguLRlPMU/VwkYXXMXDG4qSo/x+cYZX4llxDU47ZvVLNK4n
         xJJQ==
X-Forwarded-Encrypted: i=1; AJvYcCVDD2f9wSn+pxUxPnb4rJ5XLjZQDBYqRVR9+eeyln2bXwId1D7Ealzs9rnRbbemAq9XRrrZRCFgZB0rXg1B9UpVZruu4VMetL51VwjaUR0=
X-Gm-Message-State: AOJu0Yz3HrtAbiEHX8MP2flHrDwml8eE5sDX2zrRUvddVCxr1a9ag4fm
	kM1D9jCKDKLAGl5CaKDFAH6pH+FCyCspNMvoQXDqDo/DY19cfHk5ewVaiI9zWQ==
X-Google-Smtp-Source: AGHT+IHdeQ8Y6mtgqOFy+gUIvy/Obijae+NFXBrcGhwE+LQoIzZdsI7SEBSIyI6vBId1ktAK3tV6pA==
X-Received: by 2002:a17:906:32d9:b0:a59:dd9d:6da5 with SMTP id a640c23a62f3a-a5a2d54c07amr970583566b.3.1715765366136;
        Wed, 15 May 2024 02:29:26 -0700 (PDT)
Message-ID: <548088ad-9d99-465a-b103-aa59831a0d43@suse.com>
Date: Wed, 15 May 2024 11:29:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 05/15] xen/riscv: introduce bitops.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <c81251aa51abe383f60a24770a07a9aa6d673a21.1714988096.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c81251aa51abe383f60a24770a07a9aa6d673a21.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 12:15, Oleksii Kurochko wrote:
> Changes in V9:
>  - add Acked-by: Jan Beulich <jbeulich@suse.com>
>  - drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects
>    to work with 32-bit quantities.
>  - s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.

Yet then ...

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/bitops.h
> @@ -0,0 +1,137 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2012 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_BITOPS_H
> +#define _ASM_RISCV_BITOPS_H
> +
> +#include <asm/system.h>
> +
> +#if BITOP_BITS_PER_WORD == 64
> +#define __AMO(op)   "amo" #op ".d"
> +#elif BITOP_BITS_PER_WORD == 32
> +#define __AMO(op)   "amo" #op ".w"
> +#else
> +#error "Unexpected BITS_PER_LONG"

... there's a leftover here.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:38:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:38:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722109.1125985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B5g-0006aW-Ox; Wed, 15 May 2024 09:38:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722109.1125985; Wed, 15 May 2024 09:38:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B5g-0006aP-MV; Wed, 15 May 2024 09:38:28 +0000
Received: by outflank-mailman (input) for mailman id 722109;
 Wed, 15 May 2024 09:38:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cqhY=MS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7B5f-0006LE-Vb
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:38:28 +0000
Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com
 [2607:f8b0:4864:20::836])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e12df604-129e-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:38:27 +0200 (CEST)
Received: by mail-qt1-x836.google.com with SMTP id
 d75a77b69052e-43dff9b28f6so37824781cf.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:38:26 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54d85a0sm80129021cf.23.2024.05.15.02.38.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 02:38:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e12df604-129e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715765906; x=1716370706; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=DV+GBFpznnQ0ghbtfomW5vQYJJo5lnkwSFr/X/117Oc=;
        b=hQDoic9o0Y6ZQVOCHum4QM89W9f4EnR/srWQW6Sql8cp2+WvZayH/B5+HKJi9uG/Wt
         QHXXqZlVZNrMkE4+84AtB4Of+P0Ru8id31sOZTTesZuJNSXibmoqyONW/3YZH83ucxjV
         Z6S4yB0L6f6C8x6Ux+qb55OdXG9N3Nme7NqX0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715765906; x=1716370706;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DV+GBFpznnQ0ghbtfomW5vQYJJo5lnkwSFr/X/117Oc=;
        b=s/S1lumTnnGfCTDAX4ZTFHajSXR1g8uFYsXo5nniMcrVKfoAfUavW/F8qokGhVrN6d
         nKylpgIEOmlJLbM1wHEnJcSWV0yHGFKGxMHdaNIo8ZU9M7voShJ3fertH/H3nukuDbud
         9ICceviEOvJ/iG7I6K/v5CstA7r/cbTZ3v+d7DDerUjt1ka30PjJKDHTOdBbT66smS8e
         OWlJ2CiqB+l8pDWG+dEO37X8vOkonGHqCfTZ6Sa2DtkE2slj8cJeUzaJFzfQ4H+U+3Fq
         8jUKvPaDea+f9vfENHNAH4jz+GJZUasOxy2mlHKcZiwyoTcfbJIAgj522I1Z8FAzbj9j
         Z5dA==
X-Gm-Message-State: AOJu0YxIea+nZyOS4xqy36K3Is83ZmWMjizGJ9Ep/z4my83K88bbwLIZ
	UJVUh8euu4Vc9V8ufx4OnQuQMvp8Um3sfYwG/wwVj2/wl4XCLqVi8lS3ycIN94g=
X-Google-Smtp-Source: AGHT+IE4qpNQLP7Qrdr/KkjUz/cXjiOCBegz/8tBIcUTDv91glNe6ed6KyqlrMXI0lTSXCDb2F45Rg==
X-Received: by 2002:a05:622a:490:b0:43b:14a:f9ac with SMTP id d75a77b69052e-43dfcdb68fbmr248721281cf.14.1715765904790;
        Wed, 15 May 2024 02:38:24 -0700 (PDT)
Date: Wed, 15 May 2024 11:38:22 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
	dwmw@amazon.com, Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Lukasz Hawrylko <lukasz@hawrylko.pl>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Mateusz =?utf-8?B?TcOzd2th?= <mateusz.mowka@intel.com>,
	Elias El Yandouzi <eliasely@amazon.com>
Subject: Re: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Message-ID: <ZkSCju4RicXRuAvu@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-15-eliasely@amazon.com>
 <ZkOHHtRPTqC4Zvnp@macbook>
 <df8cc51d-60ea-452b-809e-c75aab957929@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <df8cc51d-60ea-452b-809e-c75aab957929@suse.com>

On Tue, May 14, 2024 at 06:22:59PM +0200, Jan Beulich wrote:
> On 14.05.2024 17:45, Roger Pau Monné wrote:
> > On Mon, May 13, 2024 at 01:40:41PM +0000, Elias El Yandouzi wrote:
> >> Until directmap gets completely removed, we'd still need to
> >> keep some calls to mfn_to_virt() for xenheap pages or when the
> >> directmap is enabled.
> >>
> >> Rename the macro to mfn_to_directmap_virt() to flag them and
> >> prevent further use of mfn_to_virt().
> > 
> > Both here and in the following patch, I'm afraid I'm unsure of it's
> > usefulness.  I'm leaning towards this being code churn for very little
> > benefit.
> 
> I expect this patch is a response to an earlier comment of mine. I'm
> rather worried that at the time this series actually goes in, un-audited
> mfn_to_virt() uses remain (perhaps because of introduction between patch
> submission and its committing). Such uses would all very likely end in
> crashes or worse, but they may not be found by testing.

I see, would be good to note the intention on the commit message then.

> > Also, I'm not sure I see how the patch prevents further usage of
> > mfn_to_virt(), as (for Arm) the existing macro is not removed.  If
> > anything I would prefer a comment clearly stating that the macro
> > operates on directmap space, and avoid the name change.
> 
> But Arm isn't switched to this sparse direct map mode, I think? At which
> point uses in Arm-specific code continue to be okay.

Right, it's just that Arm will have both mfn_to_virt() and
mfn_to_directmap_virt() which seems a bit confusing when they are
actually the same implementation.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:38:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:38:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722108.1125975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B5T-0006Ll-IX; Wed, 15 May 2024 09:38:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722108.1125975; Wed, 15 May 2024 09:38:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B5T-0006LQ-Fh; Wed, 15 May 2024 09:38:15 +0000
Received: by outflank-mailman (input) for mailman id 722108;
 Wed, 15 May 2024 09:38:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7B5R-0006LE-Vd
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:38:13 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8f52c1d-129e-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:38:12 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-5709cb80b03so1560115a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:38:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c2b8f7fsm8632048a12.66.2024.05.15.02.38.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 02:38:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8f52c1d-129e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715765892; x=1716370692; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bbI9mjALL/Uem3MryWIJwbx0cofU2nEQgjy/50sozdc=;
        b=eZIOK1ltmRu89DDsfg61NkeNOpXSOQpgPGtBLimfwotaEkXU8povAl7vX5WEFAn/UT
         r5zfsot2Xzb5PxVQDxrQNc85NF88Mr2C/jMLpE+88dubuusLcC18o3rliI6R89XnDWTy
         chf66Z5C+kCkExDlwORsYLxKQv+kbRLrdNu8B3Q7K19Ajak1iPMITLnxUpC1goOu1SC8
         w9eOFipNQEpgoCbXIZ2IxoeRwgWntZCCdA6zYJXg9iuaCXhQYaUNM+POCWhsRd+Rt6eL
         LIX47TnLmQFlE6o3BkKkumZnuZMCGH1xKwy+lUVRDOHamzV6qk7fFE2wqGkjepWR9m6o
         Y+lQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715765892; x=1716370692;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bbI9mjALL/Uem3MryWIJwbx0cofU2nEQgjy/50sozdc=;
        b=LIv+/w/NV3Phwau2ykeLtRLC7pRuRI6p3C7KCE+nJx65JvYQ+BRyaVOHknlYkkzboW
         pTyr6pl0GlV6DEKbItFf13ywxs60PpN1cHlEI53e6v7lXCD7Q2pwty5zn57p8gz36iVE
         AriUjA0ZvOI/u29cP/oLCipv+h8uPB9YszJpiMVS2HBCMjwV2uTFKVFUDzKU9jzoag0/
         /w+A/B/6gmmlUD2AVpXvIlze5sD6Q98ztbmvntxmLZLTrcMNcmjTEQvVCULXsWWkyfsL
         9kR/LTWlb+mUdSjKsKJQxaokOeaJ0RSKeInPAhrB4jDBbSVKzhedyHoq5bah9G2wDaCn
         kJ8w==
X-Forwarded-Encrypted: i=1; AJvYcCVZq+31rhgM6ZpOt/JdCJw6Zu9aeKhCbzlmtSNOK1stm3GE21FL41MbloYAu5xXVN8i0uI0rvmUUl28qu/Z4XpjNZEcTF3z4bf+jt74qX4=
X-Gm-Message-State: AOJu0YwzwbRGItsqTk+X1zKsBoghEs5o3/GZ0Gyl/X+rE028U9vRn/E5
	ddAGVyrJkaOYVvv9AsD49y3PGNcJ+GIJIFo9jKo5etdv1wR6KKQs7906K1e1lA==
X-Google-Smtp-Source: AGHT+IFyUN+R9hDd3Ajx0Wz0suMTh+HRvpPfDC4ugJ4DKT4m4Vnrm/laOZjxHHzTsaGAPNvUi2mWLw==
X-Received: by 2002:a50:8757:0:b0:56e:2294:e2e6 with SMTP id 4fb4d7f45d1cf-5734d67ae3dmr10688265a12.26.1715765892131;
        Wed, 15 May 2024 02:38:12 -0700 (PDT)
Message-ID: <fee5fee5-f501-4bad-8dac-934822457db8@suse.com>
Date: Wed, 15 May 2024 11:38:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 06/15] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <37eb9cd10cc10b84a3314039f6de6563aaabe793.1714988096.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <37eb9cd10cc10b84a3314039f6de6563aaabe793.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 12:15, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/cmpxchg.h
> @@ -0,0 +1,239 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright (C) 2014 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_CMPXCHG_H
> +#define _ASM_RISCV_CMPXCHG_H
> +
> +#include <xen/compiler.h>
> +#include <xen/lib.h>
> +
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +#define _amoswap_generic(ptr, new, ret, sfx) \
> +    asm volatile ( \
> +        " amoswap" sfx " %0, %2, %1" \
> +        : "=r" (ret), "+A" (*(ptr)) \
> +        : "r" (new) \
> +        : "memory" );
> +
> +/*
> + * For LR and SC, the A extension requires that the address held in rs1 be
> + * naturally aligned to the size of the operand (i.e., eight-byte aligned
> + * for 64-bit words and four-byte aligned for 32-bit words).
> + * If the address is not naturally aligned, an address-misaligned exception
> + * or an access-fault exception will be generated.
> + *
> + * Thereby:
> + * - for 1-byte xchg access the containing word by clearing low two bits.
> + * - for 2-byte xchg access the containing word by clearing bit 1.
> + *
> + * If resulting 4-byte access is still misalgined, it will fault just as
> + * non-emulated 4-byte access would.
> + */
> +#define emulate_xchg_1_2(ptr, new, lr_sfx, sc_sfx) \
> +({ \
> +    uint32_t *aligned_ptr; \
> +    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
> +    unsigned int new_val_bit = \
> +        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
> +    unsigned long mask = \
> +        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
> +    unsigned int new_ = (new) << new_val_bit; \
> +    unsigned int old; \
> +    unsigned int scratch; \
> +    \
> +    aligned_ptr = (uint32_t *)((unsigned long)(ptr) & ~alignment_mask); \
> +    \
> +    asm volatile ( \
> +        "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
> +        "   andn  %[scratch], %[old], %[mask]\n" \
> +        "   or   %[scratch], %[scratch], %z[new_]\n" \
> +        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
> +        "   bnez %[scratch], 0b\n" \
> +        : [old] "=&r" (old), [scratch] "=&r" (scratch), \
> +          [ptr_] "+A" (*aligned_ptr) \
> +        : [new_] "rJ" (new_), [mask] "r" (mask) \
> +        : "memory" ); \
> +    \
> +    (__typeof__(*(ptr)))((old & mask) >> new_val_bit); \
> +})
> +
> +/*
> + * This function doesn't exist, so you'll get a linker error
> + * if something tries to do an invalid xchg().
> + */
> +extern unsigned long __bad_xchg(volatile void *ptr, int size);
> +
> +static always_inline unsigned long __xchg(volatile void *ptr, unsigned long new, int size)

Nit: Line too long.

I also find "int size" odd: This can't be negative, can it? Ought to
be "unsigned int" then. Same for __cmpxchg() then.

> +#define xchg(ptr, x) \
> +({ \
> +    __typeof__(*(ptr)) n_ = (x); \
> +    (__typeof__(*(ptr))) \
> +        __xchg((ptr), (unsigned long)n_, sizeof(*(ptr))); \

Nit: I'm pretty sure I mentioned before that the parentheses around
the first argument are unnecessary. Same for cmpxchg() then.

With these taken care of
Acked-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/arch/riscv/include/asm/config.h
> +++ b/xen/arch/riscv/include/asm/config.h
> @@ -119,6 +119,8 @@
>  
>  #define BITS_PER_LLONG 64
>  
> +#define BITS_PER_BYTE 8
> +
>  /* xen_ulong_t is always 64 bits */
>  #define BITS_PER_XEN_ULONG 64

I'm in no way going to insist, but imo this would better be inserted
ahead of BYTES_PER_LONG.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:42:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:42:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722117.1125996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B9E-0000yq-7m; Wed, 15 May 2024 09:42:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722117.1125996; Wed, 15 May 2024 09:42:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7B9E-0000yj-5I; Wed, 15 May 2024 09:42:08 +0000
Received: by outflank-mailman (input) for mailman id 722117;
 Wed, 15 May 2024 09:42:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7B9D-0000yd-Ih
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:42:07 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 643ead0e-129f-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:42:06 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so3329866a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:42:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c323d3asm8639408a12.90.2024.05.15.02.42.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 02:42:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 643ead0e-129f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715766126; x=1716370926; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eKqlAuc7F5Y4rxj0U5Gq5uII2uWKzDs6jAXSHw/Xv5I=;
        b=CsEAAncJ7GWGqe1wbo3mBZBmID0o/IeywKB3wXTonb0PR8OPeKMCkK1f924iRZA2lD
         rOQ3vVX6Szbc1y4l+7K9z53Cy9nggkISD7ytjKowg/9zKkbVoCuq9BgcsqIP+AMcnC1g
         ZsqbUGWPLz1Bu7vP1k02r7kAF9b+DDBGsAH1eQILRrYsV86uICo5+I6tOvUAbZdpWzZm
         FLymX7f3q72HUBo0H+2JeFNSXx2Kefz9ZEHtgbI+BiIjnjWxDUwaFFPuPwVu920K5fIe
         h3U2AfrLPi8TcRNrk/oDTh6TKldXsdVkR59+B77nmaOlmSDxHNERkyuaTmBfTLToYgKc
         L4wA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715766126; x=1716370926;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eKqlAuc7F5Y4rxj0U5Gq5uII2uWKzDs6jAXSHw/Xv5I=;
        b=KN3b5t+stRUGWVamwT83xY2t31vyolpyTtfMVGFQ4L4mt9Q1F+ewZkJJjmAXoDFEuJ
         UvPeepOy6Qj3fRO6p7GXBRvnILb/zgXGdOEJRsV5B23S4rMqNmncsN0y6GUGW7PDlH2R
         E7xMQqPy8bKoqCYrujRk7j6NxBanSEoJcqvmUnmjseVo+qbWuAirqZ5lNTjPaSjDcHGi
         vuWHfQ0V+AJjuIqSny+C62MKgP5vV0MNc1RoaMp9F8/4Qp7OYmotxOmr15aeO1XHVND4
         68WbsYCuiMF11XW9JPf4K9A1dX9j3rDk5+0BsRTTMgrlkg3DL/CqCHMDe6nnhfh8z9SX
         Gf8g==
X-Gm-Message-State: AOJu0YyFLTl210Ck/SdJ3z9Ha9y12u1f4yqdNQ4NGtR2FjfCe8gPWlYA
	H0O/a/P0c0QDgcViI58F3knY7OiN102qclYo5dphGTOkE2FRaavCVbhp7iW8Hg==
X-Google-Smtp-Source: AGHT+IESUncwEkoGxob5ePb5DglU5QlZfsoY53BgS3BYyYgcQTTDHhLy5Gu+QJLOL3+ppKhbPsnGGA==
X-Received: by 2002:aa7:c485:0:b0:572:7d77:179d with SMTP id 4fb4d7f45d1cf-573326edde8mr14112600a12.5.1715766125863;
        Wed, 15 May 2024 02:42:05 -0700 (PDT)
Message-ID: <da8f0d28-f3d5-4d64-963f-954adc7dcb45@suse.com>
Date: Wed, 15 May 2024 11:42:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-15-eliasely@amazon.com> <ZkOHHtRPTqC4Zvnp@macbook>
 <df8cc51d-60ea-452b-809e-c75aab957929@suse.com> <ZkSCju4RicXRuAvu@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkSCju4RicXRuAvu@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 11:38, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 06:22:59PM +0200, Jan Beulich wrote:
>> On 14.05.2024 17:45, Roger Pau Monné wrote:
>>> On Mon, May 13, 2024 at 01:40:41PM +0000, Elias El Yandouzi wrote:
>>>> Until directmap gets completely removed, we'd still need to
>>>> keep some calls to mfn_to_virt() for xenheap pages or when the
>>>> directmap is enabled.
>>>>
>>>> Rename the macro to mfn_to_directmap_virt() to flag them and
>>>> prevent further use of mfn_to_virt().
>>>
>>> Both here and in the following patch, I'm afraid I'm unsure of it's
>>> usefulness.  I'm leaning towards this being code churn for very little
>>> benefit.
>>
>> I expect this patch is a response to an earlier comment of mine. I'm
>> rather worried that at the time this series actually goes in, un-audited
>> mfn_to_virt() uses remain (perhaps because of introduction between patch
>> submission and its committing). Such uses would all very likely end in
>> crashes or worse, but they may not be found by testing.
> 
> I see, would be good to note the intention on the commit message then.
> 
>>> Also, I'm not sure I see how the patch prevents further usage of
>>> mfn_to_virt(), as (for Arm) the existing macro is not removed.  If
>>> anything I would prefer a comment clearly stating that the macro
>>> operates on directmap space, and avoid the name change.
>>
>> But Arm isn't switched to this sparse direct map mode, I think? At which
>> point uses in Arm-specific code continue to be okay.
> 
> Right, it's just that Arm will have both mfn_to_virt() and
> mfn_to_directmap_virt() which seems a bit confusing when they are
> actually the same implementation.

Right, I agree it ends up slightly confusing. I don't think though that we
need to keep both longer term; we can likely switch back relatively soon.
We just need some "settling" period to allow people to notice and adjust
their code in whatever they have in the works.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 09:49:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 09:49:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722122.1126005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BGG-0001vj-0a; Wed, 15 May 2024 09:49:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722122.1126005; Wed, 15 May 2024 09:49:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BGF-0001vc-UF; Wed, 15 May 2024 09:49:23 +0000
Received: by outflank-mailman (input) for mailman id 722122;
 Wed, 15 May 2024 09:49:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7BGE-0001vE-9K
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 09:49:22 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 678f7763-12a0-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 11:49:21 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a5a88339780so150790966b.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 02:49:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17cd0sm830766866b.214.2024.05.15.02.49.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 02:49:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 678f7763-12a0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715766561; x=1716371361; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PMIoX1C6AAf1aqGV3Zp/glLRSmmmteOWja094gIAFXU=;
        b=ZGVksqJtKFgGUbKe9V1xRhVjF2D3TjQboi2FIR4QsdCZ5fw4ID/pmZ8ho72dUNu5LA
         YdhQAhfx0uu6fuzm84Exz1YoGLx7ebJA37Qn49otEBXrI3un7pWkFp9Ocp/9ZOymEEFS
         WhdmDRe5zcerTHeiQKq61GsE7wU70GG8eUo+vVyUvQ5to5ZK1/4/X6cdSxXz5jUHwcVR
         ltxet2sjZ94ynYCm1WABdmcdzQbFhdq0LKuoLA2zLGUWfbzxiq4isDkY3vZGDbkx6/q+
         XhU85uDcH5DkvzJKMufkYhMBvkJRnyH4nMOVUWOmzE8UdLh4QjqzQnKW0RIVqwPfjsml
         ICOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715766561; x=1716371361;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PMIoX1C6AAf1aqGV3Zp/glLRSmmmteOWja094gIAFXU=;
        b=I9Bjqz2sVqVo/Y9+L3xsCFBhLnGg9KSbAKm8oUbSNdB5fSv+hdiRuOE9jLF8aYU8TI
         eNhDWmPmM+bDQnPZ1348hUVytqsgGM9SJB2ssMgXiEoUORhI5VMOH75eorin3Zd5DGrw
         J/siM/RKIvXiL1j9oeu3G6Hx/UMAWD2j0iUXqANM+RzR5o8/wy4VHmDBRUYQTSRmsnuT
         D1jf2eD50Q2WXVlWONpmeqPfpwiPp7FYaXj1VuJTpHyV692qLleFdmLkC7reipE/BgIZ
         q25pgaEk60mvCbcXfAplb/vg2Y8fey9VlrwvRlRXUL0F/i9LSiXiAkv0MQyQucZ68BwN
         I9jA==
X-Forwarded-Encrypted: i=1; AJvYcCW/n4i9x7qxjdjJzZiB+qg6OVrKaM49cjGv7gq+xFdn2TSMrmH0GFTor9hulSGUDpiNquznUyFly28DBXcxlGTbduCXp7IpM+q/1tkv0fQ=
X-Gm-Message-State: AOJu0YzCi4N9R8Vm8YyECMyafbFmQvoR1vE5oLYcwZRTd0j2uFmRPF5m
	+ZC49Fr3oKrhvNddSFUnBtWOOipkJYztR7W66L9PStHGz1Hic1SUjKhIg7fwww==
X-Google-Smtp-Source: AGHT+IENfqmBIg3MHEvj7+UWx2aIl3QCPOZk8d7eEDCwliMr8LNdf9Cnaocnqk4NpKgNucv68Z/4/w==
X-Received: by 2002:a17:906:134d:b0:a59:9eab:162b with SMTP id a640c23a62f3a-a5a2d5d01abmr1016382266b.35.1715766560927;
        Wed, 15 May 2024 02:49:20 -0700 (PDT)
Message-ID: <7a4e4b99-bedc-4742-9119-2bad9bf468de@suse.com>
Date: Wed, 15 May 2024 11:49:19 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 07/15] xen/riscv: introduce atomic.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <1b8d0d3e665adc10222a9e07cedab7bcb40d88f2.1714988096.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1b8d0d3e665adc10222a9e07cedab7bcb40d88f2.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.05.2024 12:15, Oleksii Kurochko wrote:
> Changes in V9:
>  - update the defintion of write_atomic macros:
>    drop the return value as this macros isn't expeceted to return something
>    drop unnessary parentheses around p.

Yet then what about add_sized()? With that also tidied
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 10:06:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 10:06:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722129.1126016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BW3-0006yj-Au; Wed, 15 May 2024 10:05:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722129.1126016; Wed, 15 May 2024 10:05:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BW3-0006yc-7i; Wed, 15 May 2024 10:05:43 +0000
Received: by outflank-mailman (input) for mailman id 722129;
 Wed, 15 May 2024 10:05:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s7BW1-0006xB-Sd
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 10:05:42 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2607::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aebf3125-12a2-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 12:05:40 +0200 (CEST)
Received: from DB9PR01CA0025.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:1d8::30) by AS4PR08MB8142.eurprd08.prod.outlook.com
 (2603:10a6:20b:58d::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 10:05:36 +0000
Received: from DU6PEPF0000B61B.eurprd02.prod.outlook.com
 (2603:10a6:10:1d8:cafe::c3) by DB9PR01CA0025.outlook.office365.com
 (2603:10a6:10:1d8::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.25 via Frontend
 Transport; Wed, 15 May 2024 10:05:36 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU6PEPF0000B61B.mail.protection.outlook.com (10.167.8.132) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21
 via Frontend Transport; Wed, 15 May 2024 10:05:36 +0000
Received: ("Tessian outbound ba75727f6dca:v315");
 Wed, 15 May 2024 10:05:36 +0000
Received: from 1be2dac11444.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4306D45D-15FF-4DE0-9B3C-0338FB4B1341.1; 
 Wed, 15 May 2024 10:05:24 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1be2dac11444.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 May 2024 10:05:24 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DU0PR08MB7413.eurprd08.prod.outlook.com (2603:10a6:10:351::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Wed, 15 May
 2024 10:05:21 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7544.052; Wed, 15 May 2024
 10:05:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aebf3125-12a2-11ef-909d-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=SFUm932Tz4AnUD9SF28nLDFabLNOc2t9gmBFYgTtGS075ZdKZEIJGh7xwZhc5YkvYBPBIlLst4crLUEWJl9DJPIxVmudawQatMJZ/q2fRzmB5Li7fBfo59+6QJEwGEXs7SA7+lUNyGaz/0COF0sqdP4D78/e2Vka1RN/jOZKs/Qos5R0gc5Bz1hP80iTZPAiocC+r8vEu197XSxn8ImFvzwqhvfRV5UbaVe0CPCtbvE9YEktk4ElXsWUYUQBbwz5JR3duiCWXFxU/Rl7/qN6UztWbMLJEXSfzsQd8k0rxAcmVmUPtMN+IwaVI9ZMPdaG+WZgU38pchsEncHCWloA7g==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OaDAydz/Q82yXYNpdavomn0aM1mGqVxyXJ57CCbCVRQ=;
 b=EvpfcOfouGo59zgKlx0J0EyzlFRcfIYC/QGhnoU+ydL/5MlAYKUH0HYSFK+WnlpxnPG2uEKuyoB5ONPn7yOTtAG98JFZOeVdyPJTSYwBLBEE6IzmQTul0jA6fmiSXgNv0qP6OGQpH/JafMpm1mstmOvFtzwqZjxCHAghKnO2cE+MQyHHGBjqWfrNm+Mx4u977H+gIg5Podud6PoiHbBcL8zlcWIZWzu0+6bUjjp/mGswsaxJyoU4GHPALgPrGLImuyspX0rN59+dSlqL9yW0hcYI+Sur4V73ucmuJybQtN4tnvEKoGK3Yt+vrNjsG4DAN+dbxZ5OrZHpYYJ4z7JrmA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OaDAydz/Q82yXYNpdavomn0aM1mGqVxyXJ57CCbCVRQ=;
 b=nBb9CHxJwzf+Ex+1D9AiksMZl8SiWpt0jyqaBMT37StP2Hq9Rp3PEWZVR6+AUqhbtdvsC4ppwVkLlyuL9Exx1TYQZ/OCrDdhAfvuMy9uudLKo+m4QaOiPHiBzJ7wm1zX0NphHJtjYg78awc1U9TFyd6nXfB789e7keAM60Q+KEo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: c23f8325cbf8baba
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=apFVMNhwC/MqDi1uTPrg4BNNiTeGnWP3cstnkYmzT7oPuP+38WGZaLagqRuZLpn2a1leve9bwtMXOX7hchVk3FP2eSqUTOiC7WfzQYg1SH6DwUeIJUXubrP9Acc+y/TF98xtRykm6G26WSsK7AxycsW9+PC82RiCUFfdoHa45oQdimQqWaQZwxUQJKs7OjU4NgUS795KKx8Bj+TYuNR+0LzZ6uq045LXbc3KEwKOTCBCsdQU94ney9fIPL02wpiYIF+NXcajtwjU5vvJTYJn5bNETqOCBWv6K2FkxKkhDqM2MyRl2GotZc/VK7TaPOzlCT1FztiENn0Y8Gw1jhgGHw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OaDAydz/Q82yXYNpdavomn0aM1mGqVxyXJ57CCbCVRQ=;
 b=aO9gLypaWLjxQx6SlNpbkwXRkaHKT8ukAVHX6WUpA4nI3rAz01Qq2uxMD4isoZYvgAXDD06T5P5Cz1WLg90jMwQK1tQ/NqMzbrQJ67Hrl+MfEmLPj8YiXA2cTonaSkUZYUn2FS/eShLolxamxQVDSXFyqsojZaeLopuOroAIicDqrNlx4iT6oIn7rO1Alju8zMF8xYfLOeTumoUpdXj1Op/fS2uWpsrnxcZgcLNA61l4xep/unciFJmv4qyQDVVFBpirTRZBoUoNWuxMdMd0m39K1rnF4jS/4bUW2kyQr3PIKzESuF400sWRgeYSjn0xye9JgaSFnHYmq4doYG1tJA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OaDAydz/Q82yXYNpdavomn0aM1mGqVxyXJ57CCbCVRQ=;
 b=nBb9CHxJwzf+Ex+1D9AiksMZl8SiWpt0jyqaBMT37StP2Hq9Rp3PEWZVR6+AUqhbtdvsC4ppwVkLlyuL9Exx1TYQZ/OCrDdhAfvuMy9uudLKo+m4QaOiPHiBzJ7wm1zX0NphHJtjYg78awc1U9TFyd6nXfB789e7keAM60Q+KEo=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Thread-Topic: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Thread-Index: AQHalxIi36RR5t8uSEeoUslJ6d4QpLGV0j4AgACnKQCAAN2CAIAA2a+A
Date: Wed, 15 May 2024 10:05:21 +0000
Message-ID: <F353B76B-D159-43A0-898F-EF8048342B07@arm.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-2-luca.fancellu@arm.com>
 <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
 <0A931CEA-59CA-4B1F-A8E4-A4CB990C0476@arm.com>
 <d41b633a-7702-41f6-b2b3-26321de53af2@xen.org>
In-Reply-To: <d41b633a-7702-41f6-b2b3-26321de53af2@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DU0PR08MB7413:EE_|DU6PEPF0000B61B:EE_|AS4PR08MB8142:EE_
X-MS-Office365-Filtering-Correlation-Id: 06359aaf-eef6-4fd3-e32c-08dc74c690ed
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?aEowN0g5NWRNMk9HZVpWNWhNRHFuVWZuTERLYi9ybXB2MXVYRk1oRlNUWkU5?=
 =?utf-8?B?MVU4S2YwWnZLUlg0czdMSDlIUXNuNVNFWkdNVnpNKzQxbUxqN3NZU3NYVWJU?=
 =?utf-8?B?dCsrTExOZ0tZVWVQOHpIamthNWM1S0lVVW5ERU5qeGRsTnlYazAzS0NUMXRn?=
 =?utf-8?B?aHZhcEIrWHByeEVVN3dnd2VlVmVBLzk1cGc1amo4RmRlR04vL1FLQldNMkJW?=
 =?utf-8?B?MkFIM0RGN29HTFI2bVozcXFwQy9kNlIxTEhZTmNTenlNd3lLOTN2amdoQlkx?=
 =?utf-8?B?SkhJK1ltYWJlQzN6VVJ4am9DS1NIVWVhT2JXZXJiQkpsNGhPN0NKS3BTWGho?=
 =?utf-8?B?K2ZuTE9aTERyTlhMUzRqR2prcTdwUFplVGlVQWlDdHhGZWpDTjkwMFlKditX?=
 =?utf-8?B?OXowdHIzVzBHemhxaDFLT3VUQk9zSXVFWlVwUVVlRlpsQm5IWEJiRVBjM3pO?=
 =?utf-8?B?eWlqa0dubHhzZlVrbDFRT0owRVZWOXg1dCtXNnhGQkFzRDlYdlMzZ0tKUlB5?=
 =?utf-8?B?WnhvbE9FR0hpT3JTcHZJZVRwM0gyYlBhRVlVbHhyUFdkUS9xSlpSaW5lOEgy?=
 =?utf-8?B?RVdONjJ0R0xqZHFLS1Z2bkhMcDVMZHg0ZGtLSmp0OTNSMnNWZ3FHVTV2RjFz?=
 =?utf-8?B?ZnJKMWVsa25OUzlGVlVkTXl2a00yR0lvbk9ZeXRhZlJ4MGZObEd3VnJBTkp0?=
 =?utf-8?B?VkRRTitNL2JFQ09HalRQNG5ZN3FLdDIrOUQ1aDlmM2tTMmhKcEh0aGZ1dno5?=
 =?utf-8?B?ZWl2ZnUvYllRQWFraFBTVFlwWWJzSkRDR1BlajFlREtsT0lKVWc0N0VZL3dn?=
 =?utf-8?B?NytlSWkyY1VNTys0SFJXVEpxSWpWOFhHQ0NCZzd5U0JBQUxIZGJIajVrUnps?=
 =?utf-8?B?aFR1VTUwSFZMV0pyOFgzamRBSXZBSzNLMGVSSUorMXljeG9UT2E2Uks4cmE4?=
 =?utf-8?B?aHNkY0JmRm1TMmwzdzZKZkFMakJ2TGdOU0hoZElaSXc2R2JBektNRHYxK3M2?=
 =?utf-8?B?SmFEbnRsbndxRVVHaVlDRjMwbHhTWGFhdE43aERWdTAyYk9Rc1F5WWFBdm5a?=
 =?utf-8?B?Sk1MVUthUXUyZTQ4Ym9BNG9PTFlFWTYvOXdyMHI3eERLZEdhM1B2LzFQWGkw?=
 =?utf-8?B?VmdRNjVBVEdWbFZRaWpObHd3eXZSakR4WjZIYzNtcXFIQnI4OW1uKzVMUzZU?=
 =?utf-8?B?cXAyZU5oSVczNU5CVkVNOWZNN05Vayt4U0RZWkM5aVcyNUU3a0NPVnZKd0tl?=
 =?utf-8?B?OHBtQnVKTHRuQWFGWFJiTUFrQXQzR1pJazdUSWduTytPblY0YWFzOGVwdUgx?=
 =?utf-8?B?V2hkUVNoYVJWV096ZU4yaVVOS2JsYzhIUGNpcWhqYTN1RmJ2Y0NKTDV3UXZC?=
 =?utf-8?B?Q3RTOWYwSXZXTUhMbXEvRldDMjJjQk10UWhWczhHNnNpbjlEQzlyMXE0eGRz?=
 =?utf-8?B?Y1JsNTltdUxjdkplNHBuWHFPamJraHBUSWtqSkRvd1F4WDVmcjErRWUxQ1pV?=
 =?utf-8?B?UXMxMFlTN0tOMGo0S1VXc3ZhZWRGYS9hV3U3M1VLYklEekYya0gxNTNnUW5N?=
 =?utf-8?B?QkZOVUtBZzZCTW1KWUhOeGYrS2x5Z3NTZWdHQWRQR2RrbXVWUWw5QTRQMW1G?=
 =?utf-8?B?RmY3VGQzZ09YT3RXMEV6amZqQmxnRko5YzBGeDJOZU11a0hCWEtsQzFlOXZV?=
 =?utf-8?B?U0lBWDgwb0NoVGhoRFdiSVAxMmJoSXRDd1BScjMvb2lTZ2kyU1YvMDErVW1o?=
 =?utf-8?B?UTVUSkF4YTVrcFh0K1poa1g4WFZUUzU5ZUpmTXRBSFRGd3lpOHppM2FiWDB1?=
 =?utf-8?B?bFZGRXM5LzErSTZNenJmZz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <314FCE7D8F06A04BAA0793B203249D75@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7413
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU6PEPF0000B61B.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f1111403-7fbe-492a-ba3b-08dc74c6882d
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|35042699013|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?MzFNaVFFNFpFQTg2Tm5iWnpOcnBaU3R3M0k3V2ZjNnd3MklWT1pQeFVBR0sz?=
 =?utf-8?B?TVZQU2E2ZVlmVCtzY2ZXYUJmdmgwU3Z1dWhEVGpoL2ZleFk3cVdKZkliR2lt?=
 =?utf-8?B?MWEwcSsyZFFzRU5sT2Q5aVk4SUpZWlNMNVlETFR0VjZKNUpjYUhhd2ZtTTVy?=
 =?utf-8?B?SXN5aFNLRHhnczBwUHdKa2JPYWpIemh5dFdBY1RtUDRVMEUwdjROUjAzSnox?=
 =?utf-8?B?cFZUenZpbE5GR0NMTm1DRWVORVlqbWd1dVYrUnNmV3dTNVNPTXlyR09sa1RS?=
 =?utf-8?B?OHZUZUdPdkZKazB5Zmg1MmhDL0c4b3J3Ti8xVEh1QXZoNVYxNERJb3FOakdr?=
 =?utf-8?B?eTBQbHpQcTdjWngvSENNSng4dmpuVEZsTTVka2orK1dJam0zclJQWnJBd2JK?=
 =?utf-8?B?YkZvV1J1djJEM2RIUVJiMkxzVy9RcWdJdzlxV3FudFpFZHJLdDQ5SWczNC9Q?=
 =?utf-8?B?ZmtSTWRLN25tZDVGQ0RIZ1hRN0JrSG02WWNKeEt1Rmd5bXNlNTAxMUZzT3lC?=
 =?utf-8?B?YUZPMXBWMDFJOWNvMmJ3Qkwxd1RpeC9zcURON3VEcmptZ1pCM2NaMkVSSXpQ?=
 =?utf-8?B?ZHVNQjk1WDlnaW9KQ2ZhVm1FT0hNS2toRzB6a2VxdSsyUEJSNXR1Um5WNHcz?=
 =?utf-8?B?Qnh5UDhLN013ekpGOXJOTXdwS2hwZ0dnQmQ5OVppOHlvOG9Yb044enFMYlNp?=
 =?utf-8?B?REtKdG9XWlZudDdiODFtSmFvYlZSSlJpaWMyUGNzdGQ3dVBCbEcyK2R1Tkdj?=
 =?utf-8?B?WVZNbnpNc1pJOXNxV3F1cVVkcVlvVWJUUTlORU5JVWdtd1NvSWszWmlzbndn?=
 =?utf-8?B?cXVQWkpBcUQ2OGFPcVlpUjRiWUxBbmNaNlpYNEVhandDdXVtMWs4L2FtR2hu?=
 =?utf-8?B?QnR4ek9MY3QxdTQyQU12aHo0Y1N6TTBlNzNIRHA5bEdYTkZkZHdGZDFDaTJm?=
 =?utf-8?B?bFd1M3c3V2EybDh1TmlMeDA3RGNtcWNoOUJjRzkwdCtWQ3d1Z0ZRMy90K3lI?=
 =?utf-8?B?OENoWHYyR0xOVlU1YnBjcFROYS9lUHJmVFlQS1FjcWhpcGQvUEZLQkd4WTR5?=
 =?utf-8?B?NHE1bGJEY0RaZkFuT0ZBWS9nNGlBTUJORzJwUlFKU1pGYXJydnRUMy9qRFhG?=
 =?utf-8?B?Ti9NQmZ3RlpnMWlQaGRFVEVLVllncU83NUtCTkxsYmpaMU9yVWd1L0RRN1hs?=
 =?utf-8?B?YmFUbEg1R1hGNmF0UnVSUVQxcDJQS1ZCVk8xdlZqV2ZUT3lBN01rWGs4SFFH?=
 =?utf-8?B?MnZJVDZVMVAzRmxDckI5b0ovRWU3TkxyYkVGeG96cTMvK0RsK2ZLZG5ENzkx?=
 =?utf-8?B?eCtRWm5uanp6Y2V6TWM3VVcyNlZrakFmYldxNmN0UEFibTlpNFVUQlVpR0NK?=
 =?utf-8?B?THZodXJsNW94MmhldDV4QWlTUkl3OG9SeHoyM1JoMnl2V1VpSjlWSXp4WHNv?=
 =?utf-8?B?b0xKNmFQd2d4WGdkQi9BSXRNMHgyTGdxOXIzQ2s2R3NJZzZhcnBadlVVVHFG?=
 =?utf-8?B?MHZ4VWdGTitNdko4ZC9GOVE1R2lTWjIzWWJDTWpxcGwreXZvbmpJdVRNOWg5?=
 =?utf-8?B?YmRVNGZZZis4NSt6amhZdlBtakZTMTFTTnpXc1lwU2NVTm0vbTdBQklVOGtv?=
 =?utf-8?B?QzBNTTVuNXphRlpaTlhEMzNaOVJtRGJ5Z2ZqNWREL0tqclpoNlFkVG1Ubkpy?=
 =?utf-8?B?UXNBQm90T2xpM0htTlI4MHdVeXFWU2YzUjB2Tm9TaDJWQjYwK2dSaEhDTGI5?=
 =?utf-8?B?QkJPcGVuRFZxMzdDQzRybkRiTC9mZ3JHdnRlYnJ6TS9QSnVBNzQ2OEhlWThx?=
 =?utf-8?B?dDVFNEVLOTZlWFJtY3FpSUZQMWloRHF1RXpPTGwwa0dLd2NHTjV1a2tHVlVn?=
 =?utf-8?Q?v1DRJrNtKGXrQ?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(82310400017)(35042699013)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 10:05:36.3623
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 06359aaf-eef6-4fd3-e32c-08dc74c690ed
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU6PEPF0000B61B.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8142

DQoNCj4gT24gMTQgTWF5IDIwMjQsIGF0IDIyOjA2LCBKdWxpZW4gR3JhbGwgPGp1bGllbkB4ZW4u
b3JnPiB3cm90ZToNCj4gDQo+IEhpLA0KPiANCj4gT24gMTQvMDUvMjAyNCAwODo1MywgTHVjYSBG
YW5jZWxsdSB3cm90ZToNCj4+IEhpIEp1bGllbiwNCj4+IFRoYW5rcyBmb3IgaGF2aW5nIGEgbG9v
ayBvbiB0aGUgcGF0Y2gsDQo+Pj4gT24gMTMgTWF5IDIwMjQsIGF0IDIyOjU0LCBKdWxpZW4gR3Jh
bGwgPGp1bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4+PiANCj4+PiBIaSBMdWNhLA0KPj4+IA0KPj4+
IE9uIDI1LzA0LzIwMjQgMTQ6MTEsIEx1Y2EgRmFuY2VsbHUgd3JvdGU6DQo+Pj4+IEN1cnJlbnRs
eSB0aGUgY29kZSBpcyBsaXN0aW5nIGRldmljZSB0cmVlIHJlc2VydmUgbWFwIHJlZ2lvbnMNCj4+
Pj4gYXMgcmVzZXJ2ZWQgbWVtb3J5IGZvciBYZW4sIGJ1dCB0aGV5IGFyZSBub3QgYWRkZWQgaW50
bw0KPj4+PiBib290aW5mby5yZXNlcnZlZF9tZW0gYW5kIHRoZXkgYXJlIGZldGNoZWQgaW4gbXVs
dGlwbGUgcGxhY2VzDQo+Pj4+IHVzaW5nIHRoZSBzYW1lIGNvZGUgc2VxdWVuY2UsIGNhdXNpbmcg
ZHVwbGljYXRpb24uIEZpeCB0aGlzDQo+Pj4+IGJ5IGFkZGluZyB0aGVtIHRvIHRoZSBib290aW5m
by5yZXNlcnZlZF9tZW0gYXQgZWFybHkgc3RhZ2UuDQo+Pj4gDQo+Pj4gRG8gd2UgaGF2ZSBlbm91
Z2ggc3BhY2UgaW4gYm9vdGluZm8ucmVzZXJ2ZWRfbWVtIGZvciB0aGVtPw0KPj4gU28gd2UgaGF2
ZSAyNTUgYmFua3MsIGluIG15IGV4cGVyaWVuY2UgSSB3b3VsZCBzYXkgSeKAmXZlIG5ldmVyIHNh
dyB0b28gbWFueSByZXNlcnZlZCByZWdpb25zDQo+PiBpbiB0aGUgRFQsIG1heWJlIGEgY291cGxl
LCBidXQgSeKAmXZlIGFsd2F5cyBoYWQgdG8gZGVhbCB3aXRoIGVtYmVkZGVkIHBsYXRmb3Jtcy4N
Cj4+IEnigJl2ZSB0ZXN0ZWQgdGhpcyBvbmUgd2l0aCBBRExJTksgQVZBIGJvYXJkLCBuMXNkcCwg
SnVubywgcmFzcGJlcnJ5IHBpLCBxZW11LCBmdnAuDQo+PiBJbiB5b3VyIGV4cGVyaWVuY2UsIGhh
dmUgeW91IHNlZW4gYW55IG51bWJlcnMgdGhhdCBjb3VsZCBiZSBjb25jZXJuaW5nPw0KPiBJIGtu
b3cgaW4gdGhlIHBhc3Qgd2UgaGFkIHRvIGJ1bXAgdGhlIG1lbW9yeSBiYW5rcyBhIGZldyB0aW1l
cy4gQnV0IGFzIHlvdSB0ZXN0ZWQgb24gYSBmZXcgcGxhdGZvcm1zLCBJIHRoaW5rIHdlIHNob3Vs
ZCBiZSBvay4NCj4gDQo+IEl0IHdvdWxkIGJlIGJlc3QgaWYgdGhpcyBwYXRjaCBnb2VzIHNvb25l
ciB0aGFuIGxhdGVyIHRvIGFsbG93IHdpZGVyIHRlc3RpbmcgYmVmb3JlIHdlIHJlbGVhc2UgNC4x
OS4NCj4gDQo+IEFja2VkLWJ5OiBKdWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPg0KDQpZ
ZXMgaXQgd291bGQgbWFrZSBzZW5zZSwgdGhpcyBwYXRjaCBtYWtlcyBzZW5zZSBvbiBpdHMgb3du
LCB3b3VsZCB5b3UvYW55b25lIGNvbW1pdCBpdCBzZXBhcmF0ZWx5IHdoaWxlIEkgd29yayBvbiB0
aGUgc2Vjb25kDQpwYXRjaD8gDQoNCg==


From xen-devel-bounces@lists.xenproject.org Wed May 15 10:07:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 10:07:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722133.1126025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BYB-0007Xx-Lh; Wed, 15 May 2024 10:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722133.1126025; Wed, 15 May 2024 10:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7BYB-0007Xq-J8; Wed, 15 May 2024 10:07:55 +0000
Received: by outflank-mailman (input) for mailman id 722133;
 Wed, 15 May 2024 10:07:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7BY9-0007Xg-RE
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 10:07:53 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fd54a3bd-12a2-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 12:07:51 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56e37503115so1031509a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 03:07:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0af0sm8733503a12.49.2024.05.15.03.07.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 03:07:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd54a3bd-12a2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715767671; x=1716372471; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=izkW4rYDfqThlgnBovCexIj5NbvN3Vagt0ihNNRBqB4=;
        b=NgWcn3/VyhAfCI4cRqFc0HfB0G1MzPNK2dFB30BMg5FD4dgkKDXawPglcD5Bd9uts2
         mjXRbwzdciCh5O6ilidcW/5yUqD1mOQls8oU/wgnncfiKNo2Lq9rrlXyKKVVO1YtLQg/
         rte5VvEOd1BVWzY0kvBL5iI78eltVePT5K7gLA5F6n2nKfgq5FoHvlZnBi+imP4IBgSM
         O7Q0kl1Mz9RNuDHLLTZxKMOjD5yQrDbTnG6q/MbHkVTyaARn7ee/8qXGGygpIqQQnNgc
         +dPMR+QjGYUE8ecMPkQWl/LpNm4vqNnugeAASLoKZKpmZulEcSqveTAGzLiOTZ5R7EkD
         FU1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715767671; x=1716372471;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=izkW4rYDfqThlgnBovCexIj5NbvN3Vagt0ihNNRBqB4=;
        b=FJFKf72Yk/0VgRM29f0l6dbPhtOu2coLkMbZDxmQsoBk8oTgZr5NrHvqprYn0vqBsM
         VPj2JNpuaVdtTLupGjPkaEegA0k3VXmewBOH8Nnw1ftSL80By9P5O6vIebn9FjDgW4k5
         CSd4HlKzDykHvqdYl5Sc2DoL4/14kjLEqMGgGPEioOCZ06vtLQFqs+wutadla0xcXQQ9
         LXM2YBd/aqc5S4hA0fr169H/kcJ48s9ISOBttntwiRAtnLAkdY8HbLA8gUgU8xO2a1qG
         8I3PoCWv5VHWW9+ge7k2K4kKQ1uoxNyqWbw8f9BKd14V0ekNP8K//gKGaBMRPUg8SxDh
         gKtg==
X-Gm-Message-State: AOJu0YyyAmBNrt3VdfhJKbplDtToUqPti6b2YDBzjh8/f7Cwh0tLJZsk
	g0YtOg0DT4UifQMnwhtewZgwSluNa7t2Ko0O5Nvt2iQj2mTlbFMVW3yPwt333w==
X-Google-Smtp-Source: AGHT+IHD+YfedbYHHLJ3p5ggtpqgm3k5lbBegZoGP2RudgEJoB1Tw838SaxLSCjniRyjlIbUKKpZlw==
X-Received: by 2002:a05:6402:27cd:b0:573:555e:6d89 with SMTP id 4fb4d7f45d1cf-573555e6e72mr10905245a12.1.1715767671063;
        Wed, 15 May 2024 03:07:51 -0700 (PDT)
Message-ID: <44af358a-9510-4056-826c-3be99dc25830@suse.com>
Date: Wed, 15 May 2024 12:07:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com> <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com> <Zjjg2ueqgjmn-MS3@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zjjg2ueqgjmn-MS3@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 15:53, Roger Pau Monné wrote:
> On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
>> On 06.05.2024 14:42, Roger Pau Monné wrote:
>>> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
>>>> Make the variable a tristate, with (as done elsewhere) a negative value
>>>> meaning "default". Since all use sites need looking at, also rename it
>>>> to match our usual "opt_*" pattern. While touching it, also move it to
>>>> .data.ro_after_init.
>>>
>>> I guess I need to look at further patches, as given the feedback on
>>> the past version I think we agreed we want to set ATS unconditionally
>>> disabled by default, and hence I'm not sure I see the point of the
>>> tri-state if enabling ATS will require an explicit opt-in on the
>>> command line (ats=1).
>>
>> With the present wording in the VT-d spec (which we've now had vague
>> indication that it may not be meant that way) there needs to be
>> tristate behavior:
>> - With "ats=0" ATS won't be used.
>> - With "ats=1" ATS will be used for all ATS-capable devices.
>> - Without either option ATS will be used for devices where firmware
>>   mandates its use.
> 
> I'm afraid I don't agree to this behavior.  Regardless of what the
> firmware requests ATS must only be enabled on user-request (iow: when
> the ats=1 command line option is passed).  Otherwise ATS must remain
> disabled for all devices.  It's not fine for firmware to trigger the
> enabling of a feature that's not supported on Xen.

Well. On one hand I can see your point. Otoh with the spec still being the
way it is, on systems mandating ATS use for at least one device we'd then
simply need to deem Xen unsupported there altogether. The goal of the
series, though, is to make things work as mandated by the spec on such
systems, which to me implies we need to consider use of ATS supported in
such cases (and only for those specific devices, i.e. still without
considering use of "ats" on the command line supported).

If and when the spec was changed to clarify the flag is a performance hint,
not a functional requirement, then we could do as you suggest. At which
point, as mentioned before, opt_ats may be possible to become a plain
boolean variable.

>>>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>>>  
>>>>          if ( use_ats(pdev, iommu, ivrs_dev) )
>>>> -            dte->i = ats_enabled;
>>>> +            dte->i = true;
>>>
>>> Might be easier to just use:
>>>
>>> dte->i = use_ats(pdev, iommu, ivrs_dev);
>>
>> I'm hesitant here, as in principle we might be overwriting a "true" by
>> "false" then.
> 
> Hm, but that would be fine, what's the point in enabling the IOMMU to
> reply to ATS requests if ATS is not enabled on the device?
> 
> IOW: overwriting a "true" with a "false" seem like the correct
> behavior if it's based on the output of use_ats().

I don't think so, unless there were flow guarantees excluding the possibility
of taking this path twice without intermediately disabling the device again.
Down from here the enabling of ATS is gated on use_ats(). Hence if, in an
earlier invocation, we enabled ATS (and set dte->i), we wouldn't turn off ATS
below (there's only code to turn it on), yet with what you suggest we'd clear
dte->i.

Thinking about it: Maybe your comment roots in you meaning to leverage here
that use_ats() is not supposed to return different values for the same device,
when invoked multiple times. If so, I'm afraid I'm hesitant to make use of
such a property when I can easily avoid it.

>>>> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
>>>>  
>>>>  static inline int pci_ats_device(int seg, int bus, int devfn)
>>>>  {
>>>> -    if ( !ats_enabled )
>>>> +    if ( !opt_ats )
>>>>          return 0;
>>>
>>> Can't you remove that check altogether now, since you are adding an
>>> opt_ats check to use_ats()?
>>
>> Two reasons why not: For one this isn't AMD-specific code, and hence
>> shouldn't be tied to the AMD-specific use_ats(). In principle VT-d
>> code should be okay to call here, too. And then
>> amd_iommu_disable_domain_device() doesn't use use_ats(), but does call
>> here.
> 
> Oh, that's confusing, I didn't realize use_ats was AMD specific code.
> It should have some kind of prefix to avoid this kind of confusion.

Hmm, the function being static in an AMD-only file, I would have thought that
makes it clear enough that it's AMD-specific.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 10:42:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 10:42:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722144.1126035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7C5t-0007UY-9H; Wed, 15 May 2024 10:42:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722144.1126035; Wed, 15 May 2024 10:42:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7C5t-0007UR-6U; Wed, 15 May 2024 10:42:45 +0000
Received: by outflank-mailman (input) for mailman id 722144;
 Wed, 15 May 2024 10:42:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7C5s-0007UL-G4
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 10:42:44 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id db5fee4d-12a7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 12:42:42 +0200 (CEST)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2e428242a38so95670931fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 03:42:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17922159sm845269766b.97.2024.05.15.03.42.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 03:42:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db5fee4d-12a7-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715769761; x=1716374561; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+gi4XzWWe4SUgS2l4ABdUI2Bf6HqL4MIDOC3y3/WBfA=;
        b=KPapCHZq1lqk4KvJif1qI9oN5H+BpP2QXLikKm8WO0f0PrG+aNPin4pg/VMAfwn/F8
         zi+urhcRuHSFdDeRjMcUGTFAefw0f9pot95pmDhi5GuRSVTnB3IJyrHksX6R0EH14DIr
         wNO4PQS9JCmeJ0dxkghg6MSUUyXluKpMRoOnddJrVgZtT8GkqxvHACI6rw4PqaPvuoPf
         oBUC+ts2eL0pu8ys3JgQ9PSYSUKvUpUvBVLOAcVQ8DVysJ99zbWVLNG8CmW5lOI+2mIa
         ffkdZXpsbYSkFpZea/Mn9ZhQJlP5h1TqePAtzbooEa+sXGG97YqFxf72Oo/i1kxEmgMN
         u2GQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715769761; x=1716374561;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+gi4XzWWe4SUgS2l4ABdUI2Bf6HqL4MIDOC3y3/WBfA=;
        b=imtEy45zwNcn3i4gUt7kAEh+3f4SRomA1/TitL7pwPJELDQzsrvpHrOb7wkw8WNkxD
         jzMGqRHGgL4dJncQyAOzOSn3vgV8jZOzclS9WY1/Ct26WLHAxRSG6QCDOs87LWJXJqlE
         6QLBekphk/Fvi7Jn8ynx9IGOh0WYt1hz7Hwyfu1WmOWzdFBkgcNeuRtc/99oOQ4XG0lK
         MgjW8A4kzn/QsirM14hSt9PEPCM7eaGv1eY5h/6tXCCLT4JCKMe9DhH8sRYsSF2cAAfW
         9FodrRFZmXRRdK8NksrvzXdve9ZSfzgh+7hnfZ63JnA6LAFSLayf3WnyJx6g/FnCQv4m
         nTJA==
X-Gm-Message-State: AOJu0YzwSvq743voDeuMNj7kknJzvJdJXf+zNAFgtA6VGnJRusyZzl46
	ie2meEgJDlRmoJHFv0pcRqrUx3/9FOqR/EKaQh+Thi5OgunBqv9Fen2Ur+U/bw==
X-Google-Smtp-Source: AGHT+IFvB+EZRM3CIOi6UXZ8dyGIkezBPeRMGtDwz6aOPTCltQuw67uT/VutOWhMMlsKEjDcqcW++g==
X-Received: by 2002:a2e:9dc6:0:b0:2e6:f602:a02f with SMTP id 38308e7fff4ca-2e6f602a208mr13095381fa.52.1715769761468;
        Wed, 15 May 2024 03:42:41 -0700 (PDT)
Message-ID: <e9281523-a807-4889-80f0-a13804188af9@suse.com>
Date: Wed, 15 May 2024 12:42:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 06/12] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com> <ZjjdZRPluS0YIazc@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZjjdZRPluS0YIazc@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.05.2024 15:38, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:16:11AM +0100, Jan Beulich wrote:
>> When the flag is set, permit Dom0 to control the device (no worse than
>> what we had before and in line with other "best effort" behavior we use
>> when it comes to Dom0),
> 
> I think we should somehow be able to signal dom0 that this device
> might not operate as expected, otherwise dom0 might use it and the
> device could silently malfunction due to ATS not being enabled.

Whatever signaling we invented, no Dom0 would be required to respect it,
and for (perhaps quite) some time no Dom0 kernel would even exist to query
that property.

> Otherwise we should just hide the device from dom0.

This would feel wrong to me, almost like a regression from what we had
before.

> I assume setting the IOMMU context entry to passthrough mode would
> also be fine for such devices that require ATS?

I'm afraid I'm lacking the connection of the question to what is being
done here. Can you perhaps provide some more context? To provide some
context from my side: Using pass-through mode would be excluded when Dom0
is PVH. Hence why I'm not getting why we would want to even just consider
doing so.

Yet, looking at the spec, in pass-through mode translation requests are
treated as UR. So maybe your question was towards there needing to be
handling (whichever way) for the case where pass-through mode was
requested for PV Dom0? The only half-way sensible thing to do in that case
that I can think of right now would be to ignore that command line option,
just like we do when Dom0 is PVH. Yet that would equally apply to use of
"ats" on the command line, i.e. would likely first require yet another
separate patch. Except that in the "ats" case it may be reasonable to
instead panic(), for there being conflicting requests on the command line
(and it being unclear which one would be better to ignore).

>> --- a/xen/drivers/passthrough/vtd/iommu.c
>> +++ b/xen/drivers/passthrough/vtd/iommu.c
>> @@ -2364,6 +2364,26 @@ static int cf_check intel_iommu_add_devi
>>      if ( ret )
>>          dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
>>                  pdev->domain);
>> +    else if ( !pdev->broken )
>> +    {
>> +        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
>> +        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
>> +
>> +        /*
>> +         * Prevent the device from getting assigned to an unprivileged domain
>> +         * when firmware indicates ATS is required, but ATS could not be enabled
>> +         * or was not explicitly enabled via command line option.
>> +         */
>> +        if ( satc && satc->atc_required &&
>> +             (!drhd || ats_device(pdev, drhd) <= 0 ||
>> +              !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) ||
>> +              opt_ats < 0) )
> 
> Do you need the opt_ats check here?
> 
> I don't think it's possible for pci_ats_enabled() to return true if
> opt_ats is <= 0, and hence the opt_ats < 0 check can be dropped from
> the conditional?

In the present tristate mode of opt_ats a device can have ATS enabled when
opt_ats is -1 (i.e. no command line override): For devices with ATC_REQUIRED
set.

>> @@ -2375,12 +2395,26 @@ static int cf_check intel_iommu_enable_d
>>  
>>      pci_vtd_quirk(pdev);
>>  
>> -    if ( ret <= 0 )
>> -        return ret;
>> +    if ( ret <= 0 ||
>> +         (ret = enable_ats_device(pdev, &drhd->iommu->ats_devices)) < 0 ||
>> +         opt_ats < 0 )
> 
> Shouldn't this be opt_ats <= 0?

No, again not as long as this variable is a tristate one.

>> --- a/xen/drivers/passthrough/vtd/x86/ats.c
>> +++ b/xen/drivers/passthrough/vtd/x86/ats.c
>> @@ -45,8 +45,9 @@ int ats_device(const struct pci_dev *pde
>>  {
>>      struct acpi_drhd_unit *ats_drhd;
>>      unsigned int pos, expfl = 0;
>> +    const struct acpi_satc_unit *satc;
>>  
>> -    if ( opt_ats <= 0 || !iommu_qinval )
>> +    if ( !opt_ats || !iommu_qinval )
>>          return 0;
> 
> FWIW, I find this change confusing, hence my request earlier that
> opt_ats must be set to 0 or 1 by the point it gets used.

Right, but as said in particular on the subthread of patch 5, for now it has
to remain a full tristate. Whereas if the spec was changed, I expect the
variable could be switched to bool, and hence no overriding from -1 to 0/1
would be needed anymore at all.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 10:50:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 10:50:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722147.1126046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7CDA-0000rV-0i; Wed, 15 May 2024 10:50:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722147.1126046; Wed, 15 May 2024 10:50:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7CD9-0000rO-Te; Wed, 15 May 2024 10:50:15 +0000
Received: by outflank-mailman (input) for mailman id 722147;
 Wed, 15 May 2024 10:50:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=733o=MS=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s7CD9-0000rF-7k
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 10:50:15 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e76d17a9-12a8-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 12:50:12 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-572d83e3c7eso1582221a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 03:50:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e76d17a9-12a8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715770211; x=1716375011; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=ZRYbz1CnCBep73DA1rCnx+h16X0jJnQs+wXCW8hOioI=;
        b=HvSo2ILBsBfBV1fzk8G6KeHzupwav78h2mTsxojBRL5az800LLp2BEbD/t2eJ9Is+O
         gjjzwja1UqfYHxbO9gnngh/IIvXQch6quVIXlQr22RpRJwOykJMm23fJaS0vU8zHNv6m
         leQPQn2x9NQbZDU37nPovi3PDILxSudkWLiE0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715770211; x=1716375011;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ZRYbz1CnCBep73DA1rCnx+h16X0jJnQs+wXCW8hOioI=;
        b=ldfzgZmO1hADGqEKAo07Br7puTRARGyE1C3mdcz/hfIKHqTybVHlu/SYqSkoGy47Od
         ZFIwaNwlkvunEgttxZD9sKUCwKLFw0vdF7dO3XY7EImyGy/huQQN+0txWfsmZDv8v6Y6
         WBapzZ7w0L/haI5F4cIrW/57glF5D1jeTm8CzrxWPtdLVARzGqpOKjzqfrZXvXy/KMkr
         RhnmjQ6d5rj8Va/DGBNjIYZWh4nFT57Bx7DDozcfWqTCResxK9A06Hf/HWctaf6XRzzD
         2HgmtbEiiWqVANP7LaJYM06ghL2i3HxRpF4TUF/SiY9ktXfT7827y0aQ6oDobvt/O10M
         tztQ==
X-Forwarded-Encrypted: i=1; AJvYcCWcDzxvbA+buXtLlxoxxIxWjgmCYSD2W3r5wlh8EETwJ+al/bt6KGmX3MvVELlfWIi7rB+I2gA+tz9K0t67h+I0+bANY41xIY6hMJcs6Ro=
X-Gm-Message-State: AOJu0Yw5hGkdFuqT29qjLQr2wi+NC04l6tNHawSz7cVrXu3mL7JziR2B
	wFdk8V8zJSnjRWrvooiOmsXW7qLY49GF+ZwgGVKmKjjvFhjPZhlkXAFLCX/AkzdiAciXIfBbRmd
	Ew8GRT62vlEkr+wnWzk/usHDrtyrrVH5egOJ89w==
X-Google-Smtp-Source: AGHT+IFJDezroE65n2WnhM7PTSrRF8xl/ULCEfNsdqblLyro66lc00Cii7rPSQSzyljOD26PPAbjVZR20jgRjrab/yo=
X-Received: by 2002:a50:870d:0:b0:572:5fff:accd with SMTP id
 4fb4d7f45d1cf-5734d590348mr10699828a12.6.1715770211310; Wed, 15 May 2024
 03:50:11 -0700 (PDT)
MIME-Version: 1.0
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
In-Reply-To: <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 15 May 2024 11:49:35 +0100
Message-ID: <CAO-mL=x6P3oX=FXw=3bciHPbR2RAA5GymMVNY33_xiH_vu8juQ@mail.gmail.com>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, George Dunlap <George.Dunlap@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Daniel Smith <dpsmith@apertussolutions.com>, Xen-devel <xen-devel@lists.xenproject.org>, 
	"committers@xenproject.org" <committers@xenproject.org>
Content-Type: multipart/alternative; boundary="000000000000f6438506187be1a3"

--000000000000f6438506187be1a3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi all,

As Stefano has mentioned, we have the maintainers and committers call later
today.
Let's use this time to better collaborate and discuss any differences in
opinions we have. It will also give everyone a chance to explain their
viewpoints.

Andy, please can I remind you to keep the language clean and professional
in line with our standards as a community.

Many thanks,
Kelly Choi

Community Manager
Xen Project


On Tue, May 14, 2024 at 10:51=E2=80=AFAM Andrew Cooper <andrew.cooper3@citr=
ix.com>
wrote:

> On 14/05/2024 10:25 am, Jan Beulich wrote:
> > On 03.04.2024 08:16, Jan Beulich wrote:
> >> On 02.04.2024 19:06, Andrew Cooper wrote:
> >>> The commit makes a claim without any kind of justification.
> >> Well, what does "have no business" leave open?
> >>
> >>> The claim is false, and the commit broke lsevtchn in dom0.
> >> Or alternatively lsevtchn was doing something that was never meant to
> work
> >> (from Xen's perspective).
> >>
> >>>  It is also quite
> >>> obvious from XSM_TARGET that it has broken device model stubdoms too.
> >> Why would that be "obvious"? What business would a stubdom have to loo=
k
> at
> >> Xen's side of an evtchn?
> >>
> >>> Whether to return information about a xen-owned evtchn is a matter of
> policy,
> >>> and it's not acceptable to short circuit the XSM on the matter.
> >> I can certainly accept this as one possible view point. As in so many
> cases
> >> I'm afraid I dislike you putting it as if it was the only possible one=
.
> >>
> >> In summary: The supposed justification you claim is missing in the
> original
> >> change is imo also missing here then: What business would any entity i=
n
> the
> >> system have to look at Xen's side of an event channel? Back at the
> time, 3
> >> people agreed that it's "none".
> > You've never responded to this reply of mine, or its follow-up. You als=
o
> > didn't chime in on the discussion Daniel and I were having. I consider =
my
> > objections unaddressed, and in fact I continue to consider the change t=
o
> > be wrong. Therefore it was inappropriate for you to commit it; it needs
> > reverting asap. If you're not going to do so, I will.
>
> You tried defending breaking a utility with "well it shouldn't exist then=
".
>
> You don't have a leg to stand on, and two maintainers of relevant
> subsystems here just got tired of bullshit being presented in place of
> any credible argument for having done the change in the way you did.
>
> The correct response was "Sorry I broke things.  Lets revert this for
> now to unbreak, and I'll see about reworking it to not intentionally
> subvert Xen's security mechanism".
>
> As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
> had for the principle of the change based on the absurdity of your
> arguments.
>
> ~Andrew
>

--000000000000f6438506187be1a3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,=C2=A0<div><br></div><div>As Stefano has mentioned,=
 we have the maintainers and committers call later today.</div><div>Let&#39=
;s use this time to better collaborate and discuss any differences in opini=
ons we have. It will also give everyone a chance to explain their viewpoint=
s.=C2=A0=C2=A0</div><div><br></div><div>Andy, please can I remind you to ke=
ep the language clean and professional in line with our standards as a comm=
unity.=C2=A0</div><div><br clear=3D"all"><div><div dir=3D"ltr" class=3D"gma=
il_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Many=
 thanks,</div><div>Kelly Choi</div><div><br></div><div><div style=3D"color:=
rgb(136,136,136)">Community Manager</div><div style=3D"color:rgb(136,136,13=
6)">Xen Project=C2=A0<br></div></div></div></div></div><br></div></div><br>=
<div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Ma=
y 14, 2024 at 10:51=E2=80=AFAM Andrew Cooper &lt;<a href=3D"mailto:andrew.c=
ooper3@citrix.com">andrew.cooper3@citrix.com</a>&gt; wrote:<br></div><block=
quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1=
px solid rgb(204,204,204);padding-left:1ex">On 14/05/2024 10:25 am, Jan Beu=
lich wrote:<br>
&gt; On 03.04.2024 08:16, Jan Beulich wrote:<br>
&gt;&gt; On 02.04.2024 19:06, Andrew Cooper wrote:<br>
&gt;&gt;&gt; The commit makes a claim without any kind of justification.<br=
>
&gt;&gt; Well, what does &quot;have no business&quot; leave open?<br>
&gt;&gt;<br>
&gt;&gt;&gt; The claim is false, and the commit broke lsevtchn in dom0.<br>
&gt;&gt; Or alternatively lsevtchn was doing something that was never meant=
 to work<br>
&gt;&gt; (from Xen&#39;s perspective).<br>
&gt;&gt;<br>
&gt;&gt;&gt;=C2=A0 It is also quite<br>
&gt;&gt;&gt; obvious from XSM_TARGET that it has broken device model stubdo=
ms too.<br>
&gt;&gt; Why would that be &quot;obvious&quot;? What business would a stubd=
om have to look at<br>
&gt;&gt; Xen&#39;s side of an evtchn?<br>
&gt;&gt;<br>
&gt;&gt;&gt; Whether to return information about a xen-owned evtchn is a ma=
tter of policy,<br>
&gt;&gt;&gt; and it&#39;s not acceptable to short circuit the XSM on the ma=
tter.<br>
&gt;&gt; I can certainly accept this as one possible view point. As in so m=
any cases<br>
&gt;&gt; I&#39;m afraid I dislike you putting it as if it was the only poss=
ible one.<br>
&gt;&gt;<br>
&gt;&gt; In summary: The supposed justification you claim is missing in the=
 original<br>
&gt;&gt; change is imo also missing here then: What business would any enti=
ty in the<br>
&gt;&gt; system have to look at Xen&#39;s side of an event channel? Back at=
 the time, 3<br>
&gt;&gt; people agreed that it&#39;s &quot;none&quot;.<br>
&gt; You&#39;ve never responded to this reply of mine, or its follow-up. Yo=
u also<br>
&gt; didn&#39;t chime in on the discussion Daniel and I were having. I cons=
ider my<br>
&gt; objections unaddressed, and in fact I continue to consider the change =
to<br>
&gt; be wrong. Therefore it was inappropriate for you to commit it; it need=
s<br>
&gt; reverting asap. If you&#39;re not going to do so, I will.<br>
<br>
You tried defending breaking a utility with &quot;well it shouldn&#39;t exi=
st then&quot;.<br>
<br>
You don&#39;t have a leg to stand on, and two maintainers of relevant<br>
subsystems here just got tired of bullshit being presented in place of<br>
any credible argument for having done the change in the way you did.<br>
<br>
The correct response was &quot;Sorry I broke things.=C2=A0 Lets revert this=
 for<br>
now to unbreak, and I&#39;ll see about reworking it to not intentionally<br=
>
subvert Xen&#39;s security mechanism&quot;.<br>
<br>
As it stands, you&#39;re 2-1 outvoted, and wasted any sympathy I may have<b=
r>
had for the principle of the change based on the absurdity of your<br>
arguments.<br>
<br>
~Andrew<br>
</blockquote></div>

--000000000000f6438506187be1a3--


From xen-devel-bounces@lists.xenproject.org Wed May 15 11:58:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 11:58:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722266.1126104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7DHI-0004sE-R7; Wed, 15 May 2024 11:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722266.1126104; Wed, 15 May 2024 11:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7DHI-0004s7-Nx; Wed, 15 May 2024 11:58:36 +0000
Received: by outflank-mailman (input) for mailman id 722266;
 Wed, 15 May 2024 11:58:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7DHH-0004rq-94
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 11:58:35 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 74c2eb54-12b2-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 13:58:34 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2e538a264f7so70880691fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 04:58:34 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f87d20488sm265726055e9.25.2024.05.15.04.58.33
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 04:58:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74c2eb54-12b2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715774314; x=1716379114; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=vO4leLrtvRFUHAgIvSBgaS0l2vESgB2VTMS6Hsi8+9Y=;
        b=kQ90EwthndEEEkovvfZbcgRO+Z7VHB6pzepT+LGfVeRLYoZkZcyV2mEz+AszOr3q/l
         88mgZyDvcjyohMM5qSguOy4Zf3NlpDl+bTDv1GsELZVnxhSx5DG353UtIeWBgIZyQypl
         lVmbYJgLBTFJEVsYBbQCvi1ZJwgKZ0YICU9cA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715774314; x=1716379114;
        h=content-transfer-encoding:autocrypt:subject:from:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vO4leLrtvRFUHAgIvSBgaS0l2vESgB2VTMS6Hsi8+9Y=;
        b=FujimUmA2FUw4A9wuSeHGNZaz5RbhXE0cNOtwiA3xfgc/TMUesXuv1jfXQZD0JmiLv
         vbZ/TEDLS1AmUJcb6LDoDvgQ0Z2J3EzmbyLr1l8KMrQSAPmLiCyQ2LBJXu5IiJTeGHve
         A9vkh8lz0LJO8Buemh2ps8FV65C9MQyMkFyvSOWtBkwDIhidUW0dG0+ud2kxHgD+FriN
         NfM27NYJfpXei7Nq9CVEm1EyPtRMCWS4iU5Lp+rWIgtEHl52pufMj9wrsyt3PWM7x96e
         nb0AzltcYpFQ19/XO1D1E95woF0h5xgUb1PUELHcgNEvbMsMG77bUldWnwnLGi9sHB8h
         UmNA==
X-Gm-Message-State: AOJu0YzQLnie84B6NCK5W5PbBTscgTPBli81hcgOpbiGFuVLzSgXqohn
	wao1DWQkXg/I/t3LAWph5lS+MlmFeUYgSuPY7ZtOi8oe3L6Ilwn7++Yj2XZJ0FSsFpEfyKba2dp
	g
X-Google-Smtp-Source: AGHT+IEyQKsoelT+y3YmJ7mQl1NgHRL57EyYoa95atihJb97sSW2SPp2GvN45RCrwnGqetOhkHIKmQ==
X-Received: by 2002:a2e:9355:0:b0:2df:e0c4:8429 with SMTP id 38308e7fff4ca-2e51fd4afb7mr101390861fa.18.1715774313676;
        Wed, 15 May 2024 04:58:33 -0700 (PDT)
Message-ID: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
Date: Wed, 15 May 2024 12:58:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-GB
To: xen-devel <xen-devel@lists.xenproject.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Xen crash in scheduler during cpu hotplug
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Just so it doesn't get lost.  In XenRT, we've found:

> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
> (XEN) CPU:    45
> (XEN) RIP:    e008:[<ffff82d040244cbf>]
> common/sched/credit.c#csched_load_balance+0x41/0x877
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
> 0000000000000000
> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
> ffff83103723d450
> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8: 
> 0000000000000000
> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
> 0000000000000009
> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
> ffff831037253720
> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
> 0000000000f526e0
> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
> 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d040244cbf>
> (common/sched/credit.c#csched_load_balance+0x41/0x877):
> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
> 8b 4e 28 48
> <snip>
> (XEN) Xen call trace:
> (XEN)    [<ffff82d040244cbf>] R
> common/sched/credit.c#csched_load_balance+0x41/0x877
> (XEN)    [<ffff82d040245a18>] F
> common/sched/credit.c#csched_schedule+0x36a/0x69f
> (XEN)    [<ffff82d040252644>] F common/sched/core.c#do_schedule+0xe8/0x433
> (XEN)    [<ffff82d0402572dd>] F common/sched/core.c#schedule+0x2e5/0x2f9
> (XEN)    [<ffff82d040232f35>] F common/softirq.c#__do_softirq+0x94/0xbe
> (XEN)    [<ffff82d040232fc8>] F do_softirq+0x13/0x15
> (XEN)    [<ffff82d0403075ef>] F arch/x86/domain.c#idle_loop+0x92/0xe6
> (XEN)
> (XEN) Pagetable walk from 0000000000000010:
> (XEN)  L4[0x000] = 000000103ffca063 ffffffffffffffff
> (XEN)  L3[0x000] = 000000103ffc9063 ffffffffffffffff
> (XEN)  L2[0x000] = 000000103ffc8063 ffffffffffffffff
> (XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 45:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: 0000000000000010
> (XEN) ****************************************

which we can reproduce reliably enough with a shell script doing
xen-hptool cpu-{off,on}line.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 12:40:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 12:40:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722302.1126130 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7DvN-0004fB-H5; Wed, 15 May 2024 12:40:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722302.1126130; Wed, 15 May 2024 12:40:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7DvN-0004f4-Db; Wed, 15 May 2024 12:40:01 +0000
Received: by outflank-mailman (input) for mailman id 722302;
 Wed, 15 May 2024 12:40:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7DvL-0004di-Vr
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 12:40:00 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3cd1bdcf-12b8-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 14:39:57 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so1644309a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 05:39:57 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7da8sm858312266b.99.2024.05.15.05.39.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 05:39:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cd1bdcf-12b8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715776797; x=1716381597; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RRZIdD1PNnstrzGUpN0wWe5cyhSUkt1nQ1HDw0F7er0=;
        b=deS54PoYealClqpwQ9KesEi6VZhxPaBPU9CFdVerowozzQ1gcLXW2G7Ti3dudHpvY6
         F0Kd8HAxnCBt2EBP70YEm+ZCVjUx0sOdLGCBktIXp0e8EtZmdKTZZuQb2yWR0Ovbm2v6
         G3rncWED6ay5VnAs/W6WW77Hz8Cv9OKNBqMRXXjb8DODIOZPjVZzVZ3YuKsR5HIkoHfM
         TQ30KbRLEsOm2OqZ7lhCU+bXO43qGxYQiSbU4tM4Umcbai126IJ345eTLdogh/zehx7D
         pUCMy02IV41KaUFTj9rm3ql4Uw0reV92uoOYHZjTxpaGJ8B4QenVRRsOMshsxmuFslwk
         zC5g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715776797; x=1716381597;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RRZIdD1PNnstrzGUpN0wWe5cyhSUkt1nQ1HDw0F7er0=;
        b=RYr0h+xC/2C2fd08FfXrw0pZYXfiiIGwiLtZsTmz8Jjmax359A5AAZusOekh69hMCO
         QHK71ZUEVz+cIEsUV8mZ9JAQkWz7CByF71OIlkLESu5WCEqzSh6zdOtGNUbP8AftFnZQ
         bT+xFwvL4HU1v9iEJ1NVn3/BdCwlTsb+Cda9ymP8qAa5m1cpbs+jaBbVY/nWWX+19U4x
         el4L7SuUtuIyhdKCDk+mEHUsISp4Ftrg3F17IfbfdpR7fUOPMbucNGBI0KY3RYV+qUS6
         /g09L0LUX1RhtLqkmLYuA5+y82sEzqfhcSYLJWWm9Ng5wdGC6H1ZiholaBjUexEdWOVs
         VtPw==
X-Gm-Message-State: AOJu0YzOYT9P+rhZ0RBUmLETVmgli3JlKrAfu6Kdz4dQQAakrGVHPWi0
	ypxS81U7/zT81OGUxouduYCg9YA2+7YQXeXhE3YeOPZXFDUh3TpS//v9V2kK1g==
X-Google-Smtp-Source: AGHT+IGiCYxA5YQhPYvH/pavad0RaXapzzlqWb6hs8VTH/x8Ulix++6eOwCQ0kj9ySY+IsPimKZD/A==
X-Received: by 2002:a17:906:6944:b0:a55:a895:46ae with SMTP id a640c23a62f3a-a5a2d67db94mr990392866b.63.1715776797094;
        Wed, 15 May 2024 05:39:57 -0700 (PDT)
Message-ID: <7553291c-f88d-49c7-a1bd-5f8dbd21fd2b@suse.com>
Date: Wed, 15 May 2024 14:39:55 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen crash in scheduler during cpu hotplug
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
References: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 13:58, Andrew Cooper wrote:
> Just so it doesn't get lost.  In XenRT, we've found:
> 
>> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
>> (XEN) CPU:    45
>> (XEN) RIP:    e008:[<ffff82d040244cbf>]
>> common/sched/credit.c#csched_load_balance+0x41/0x877
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
>> 0000000000000000
>> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
>> ffff83103723d450
>> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8: 
>> 0000000000000000
>> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
>> 0000000000000009
>> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
>> ffff831037253720
>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
>> 0000000000f526e0
>> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
>> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
>> 0000000000000000
>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>> (XEN) Xen code around <ffff82d040244cbf>
>> (common/sched/credit.c#csched_load_balance+0x41/0x877):
>> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
>> 8b 4e 28 48
>> <snip>
>> (XEN) Xen call trace:
>> (XEN)    [<ffff82d040244cbf>] R
>> common/sched/credit.c#csched_load_balance+0x41/0x877

While this is of course pretty little information, I've still tried to
decipher it, first noticing it's credit1 that's being used here. Once
forcing csched_load_balance() non-inline (no idea why it is a separate
function in your build), I can see a sufficiently matching pattern at
approximately the same offset into the function. That's

    const struct cpupool *c = get_sched_res(cpu)->cpupool;
    ...
    const cpumask_t *online = c->res_valid;
    ...
    BUG_ON(get_sched_res(cpu) != snext->unit->res);

overlapping, with the crash being on the middle of the quoted lines.
IOW the CPU pool is still NULL for this sched resource. Cc-ing
Jürgen for possible clues ...

Jan

>> (XEN)    [<ffff82d040245a18>] F
>> common/sched/credit.c#csched_schedule+0x36a/0x69f
>> (XEN)    [<ffff82d040252644>] F common/sched/core.c#do_schedule+0xe8/0x433
>> (XEN)    [<ffff82d0402572dd>] F common/sched/core.c#schedule+0x2e5/0x2f9
>> (XEN)    [<ffff82d040232f35>] F common/softirq.c#__do_softirq+0x94/0xbe
>> (XEN)    [<ffff82d040232fc8>] F do_softirq+0x13/0x15
>> (XEN)    [<ffff82d0403075ef>] F arch/x86/domain.c#idle_loop+0x92/0xe6
>> (XEN)
>> (XEN) Pagetable walk from 0000000000000010:
>> (XEN)  L4[0x000] = 000000103ffca063 ffffffffffffffff
>> (XEN)  L3[0x000] = 000000103ffc9063 ffffffffffffffff
>> (XEN)  L2[0x000] = 000000103ffc8063 ffffffffffffffff
>> (XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 45:
>> (XEN) FATAL PAGE FAULT
>> (XEN) [error_code=0000]
>> (XEN) Faulting linear address: 0000000000000010
>> (XEN) ****************************************
> 
> which we can reproduce reliably enough with a shell script doing
> xen-hptool cpu-{off,on}line.
> 
> ~Andrew
> 



From xen-devel-bounces@lists.xenproject.org Wed May 15 12:55:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 12:55:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722331.1126155 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EA1-0000Kh-1U; Wed, 15 May 2024 12:55:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722331.1126155; Wed, 15 May 2024 12:55:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EA0-0000Ka-VB; Wed, 15 May 2024 12:55:08 +0000
Received: by outflank-mailman (input) for mailman id 722331;
 Wed, 15 May 2024 12:55:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7E9z-0000KQ-UM; Wed, 15 May 2024 12:55:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7E9z-00019t-P1; Wed, 15 May 2024 12:55:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7E9z-0006VU-Bb; Wed, 15 May 2024 12:55:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7E9z-0003xA-BA; Wed, 15 May 2024 12:55:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7px3zIGpPMsFVlGqOXgjqY5uIjqQM/Vz06zV+skWM5I=; b=rhBmnWlln1kPcMh2m5f7xBj50t
	50bDmMIsdi5uA6itY3fzSqsDw9WAmMdX/s2JssVnu3bLlDCaXmHeTcE30+KtNjGSwFB5IgRsyNudC
	DElXWMCRYF0YW6J6y4mfyrh/tqymay09QWLYaVWzzxoWVV2nxxOgnsH1d/2kwT+TsOdE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186002-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186002: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4f8b6f25eb1e51febd426da764a0b0ea652ad238
X-Osstest-Versions-That:
    linux=a5131c3fdf2608f1c15f3809e201cf540eb28489
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 12:55:07 +0000

flight 186002 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186002/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185996
 test-armhf-armhf-libvirt-vhd  8 xen-boot                     fail  like 185996
 test-armhf-armhf-examine      8 reboot                       fail  like 185996
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185996
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185996
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185996
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4f8b6f25eb1e51febd426da764a0b0ea652ad238
baseline version:
 linux                a5131c3fdf2608f1c15f3809e201cf540eb28489

Last test of basis   185996  2024-05-14 09:18:32 Z    1 days
Failing since        185998  2024-05-14 19:41:48 Z    0 days    2 attempts
Testing same since   186002  2024-05-15 02:16:07 Z    0 days    1 attempts

------------------------------------------------------------
357 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   a5131c3fdf26..4f8b6f25eb1e  4f8b6f25eb1e51febd426da764a0b0ea652ad238 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed May 15 12:59:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 12:59:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722337.1126166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EEX-0001gm-Jy; Wed, 15 May 2024 12:59:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722337.1126166; Wed, 15 May 2024 12:59:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EEX-0001gf-Gm; Wed, 15 May 2024 12:59:49 +0000
Received: by outflank-mailman (input) for mailman id 722337;
 Wed, 15 May 2024 12:59:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Htxv=MS=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1s7EEW-0001gX-IR
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 12:59:48 +0000
Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com
 [2607:f8b0:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 01aee579-12bb-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 14:59:47 +0200 (CEST)
Received: by mail-oi1-x22b.google.com with SMTP id
 5614622812f47-3c9abbb9efbso1098292b6e.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 05:59:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01aee579-12bb-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715777986; x=1716382786; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BkVGQ0yCW+Kcc4HDyyz/k3gv3ZB4NHmjbpXeWCF7e+8=;
        b=LwUbr6jyhS8D0QSPNjfTy//7od5cSCDuA2H/iP6pPjQzsqJrQzINXQqro5vlShVO9F
         yl30eEDjV81F88QkY/pUAYDMX4qYb9MoqolN7MShvtQKJXAdiLkBmyLUVNKeAIxDePEc
         ZNI0OPqPgwOWjSyayHFG7qAqFPAXYnqCHmcQc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715777986; x=1716382786;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BkVGQ0yCW+Kcc4HDyyz/k3gv3ZB4NHmjbpXeWCF7e+8=;
        b=J2TNInUK+R2cUk8WT4KHCywAIYyBvgBjvfpkkSvxk3pwUTMInnHSmtCrGUjN5zoJ3o
         8n4mHMlhLLW3Nb99sIIe1n1BnxcJ/oW9zpGrPV9dmTJdUt43F+gMoaoqEK+igUVMM/bl
         7YNaY4DyU2sCy+1AYAiYGCYzZ55zzzHEu8gRBafKCdNBEdxCECI82wxRbYJSzRHj/Mxp
         zlZATkU06aq1QXxwn/Nmfiz76gvbXEaqQ/qERoQIqThQMQW1tTWZ2yBoeem8H3/h80fd
         cs30hnoA1ssQ8tujvD0zwNERNeoHq4sJpS9IHKXHkvwOQDFYiUjJn42dmiMtVM4ka1yY
         MtKA==
X-Forwarded-Encrypted: i=1; AJvYcCUfI3uquX+Oizqta+UU4e2mVoI3u6ekCKsuprClGbXFyXRYtggkBBtnt0gIl0pr1kx2j5I0imSaBjCbdQcjUna1rMmX3zusDQjTIFbWoiw=
X-Gm-Message-State: AOJu0Yz9nlr5qdi3yFlY7HEtuismB/p5c5L/iqCW28SPhK6jBpLalgg+
	LnMITS9utGNxp6uzxhx2+MWT2+5scgJ80Bl7+RDTs3jlr27+YcDpqMXiCKSyUP7Vbx39vTBJRdb
	QibQNSQNvoTTzAUdot9gRN8CHEj8i6BznBwYC4Q==
X-Google-Smtp-Source: AGHT+IHNPJTgQV6SbwXzpCan+9bKdOmsJIDH9yB5dV2Pc6wGnhjJkvsLlMENdEmFgwLgFQ7rmGEWrwg+pLU37bsC32M=
X-Received: by 2002:a05:6870:1596:b0:23c:e63c:4564 with SMTP id
 586e51a60fabf-24111c626e7mr8911279fac.7.1715777986145; Wed, 15 May 2024
 05:59:46 -0700 (PDT)
MIME-Version: 1.0
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
In-Reply-To: <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 15 May 2024 13:59:35 +0100
Message-ID: <CA+zSX=bbVVJzA=a-ZDgTashtu8-t+BqWVpzWxahOve+WGZLzrQ@mail.gmail.com>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Julien Grall <julien@xen.org>, Daniel Smith <dpsmith@apertussolutions.com>, 
	Xen-devel <xen-devel@lists.xenproject.org>, 
	"committers@xenproject.org" <committers@xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 14, 2024 at 10:51=E2=80=AFAM Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>
> On 14/05/2024 10:25 am, Jan Beulich wrote:
> > On 03.04.2024 08:16, Jan Beulich wrote:
> >> On 02.04.2024 19:06, Andrew Cooper wrote:
> >>> The commit makes a claim without any kind of justification.
> >> Well, what does "have no business" leave open?
> >>
> >>> The claim is false, and the commit broke lsevtchn in dom0.
> >> Or alternatively lsevtchn was doing something that was never meant to =
work
> >> (from Xen's perspective).
> >>
> >>>  It is also quite
> >>> obvious from XSM_TARGET that it has broken device model stubdoms too.
> >> Why would that be "obvious"? What business would a stubdom have to loo=
k at
> >> Xen's side of an evtchn?
> >>
> >>> Whether to return information about a xen-owned evtchn is a matter of=
 policy,
> >>> and it's not acceptable to short circuit the XSM on the matter.
> >> I can certainly accept this as one possible view point. As in so many =
cases
> >> I'm afraid I dislike you putting it as if it was the only possible one=
.
> >>
> >> In summary: The supposed justification you claim is missing in the ori=
ginal
> >> change is imo also missing here then: What business would any entity i=
n the
> >> system have to look at Xen's side of an event channel? Back at the tim=
e, 3
> >> people agreed that it's "none".
> > You've never responded to this reply of mine, or its follow-up. You als=
o
> > didn't chime in on the discussion Daniel and I were having. I consider =
my
> > objections unaddressed, and in fact I continue to consider the change t=
o
> > be wrong. Therefore it was inappropriate for you to commit it; it needs
> > reverting asap. If you're not going to do so, I will.
>
> You tried defending breaking a utility with "well it shouldn't exist then=
".
>
> You don't have a leg to stand on, and two maintainers of relevant
> subsystems here just got tired of bullshit being presented in place of
> any credible argument for having done the change in the way you did.
>
> The correct response was "Sorry I broke things.  Lets revert this for
> now to unbreak, and I'll see about reworking it to not intentionally
> subvert Xen's security mechanism".
>
> As it stands, you're 2-1 outvoted, and wasted any sympathy I may have
> had for the principle of the change based on the absurdity of your
> arguments.

We have a simple process for dealing with this situation, Andy, which
you didn't follow.  You can't just go checking things in because you
feel strongly about it.

 -George


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:11:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:11:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722344.1126176 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EPk-0004fq-OO; Wed, 15 May 2024 13:11:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722344.1126176; Wed, 15 May 2024 13:11:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EPk-0004fj-KP; Wed, 15 May 2024 13:11:24 +0000
Received: by outflank-mailman (input) for mailman id 722344;
 Wed, 15 May 2024 13:11:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7EPj-0004fd-Ii
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:11:23 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0125ebe-12bc-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:11:22 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so176359166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:11:22 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a178a9f0fsm855140866b.69.2024.05.15.06.11.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:11:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0125ebe-12bc-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715778681; x=1716383481; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=w9/MpnHlK4WnFuZfgJjdmI8kmoLoZoI3C3K5nxMI4YU=;
        b=D8DMLGu/2hiqXqb8V7B+PYmk+IZXP8iipZT82AugJ5WYgjIQfF7d60OycQJHCIlJ0/
         69kiPggFWV4SM6NZ8eVnWI8od2ubORv8l9x2Pj0ORenKfdUhgJMNydKtNhIq/RNRmITN
         d+BxemhfLNUYMsH4zPXm2+RUZtPEYnv42yfh3Gmi7vKypZocOTwedWawGX2LxG8EYQEu
         O/5Hz9htiUTJu/NwVnC1NurJlPLtv1uCS1Dip2p0v/DG0MFQQWUCCWaL59KrtH/E5mH/
         UsMEhCmDE9QK5tgFFZZC7zlrCIAQxYSoxRx/zdVYS2EXfDxuyOxi5Gcsu/f+YdDNMyA1
         E9iQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715778681; x=1716383481;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=w9/MpnHlK4WnFuZfgJjdmI8kmoLoZoI3C3K5nxMI4YU=;
        b=eS4epKFXExTrkQdvf++noaHahgzjiZIPDAbQd+avoKTD7PdEGAkMeEcZOQC45GrnJE
         wYCEVsKA7gE5X74unHssZmZnIHrE3U924HImLxWT5TcrCDUiB0VZTeUanLku88o2zCHB
         10M/lug0H1Z8gKKD4+jmhQlOVqhZljVFacHyCf8apDGc2DgY75c8e4obnLxjLMYqyu5w
         2WeEC+c0gNOpFW3PkUwUCvotUSYM385ALRrsZUAUmWD4nxMYuM7U9LQ2UCqa0XTjlu2F
         hklUJmg2OmuG6fFAnK1xdVlzL0z7FRMTci71e/yJn5Rn2xFQVPRAThoeDoCgH8oy9jMy
         hVxw==
X-Forwarded-Encrypted: i=1; AJvYcCWoObqCrtRgW+fVxm8Okpzen/3LTRDtnEaNqmwiQ/KIXVkL1JtL8vqgRO2T83OAyjWdnyYIRe61Q6IdUSKEStGMgxV23EbzTPesAC6CAig=
X-Gm-Message-State: AOJu0Yx4rQN40t6XtkjXd8ph4xCB4cxnVXX90xOr/VcePZGB8ED6UbT3
	g5kDdY4vd2ky3Ys60f7CehISL26ki95+8XjKYZHZYIpmKMWM8x6YdWh09tjjPw==
X-Google-Smtp-Source: AGHT+IGAK53JKe8uY1mJvAlgbOf9Fq7VNlvkTNDoR57VN4Sw25olMDFhsAntNvufZiJF+E2XM/gIIA==
X-Received: by 2002:a17:907:3609:b0:a5a:8849:2205 with SMTP id a640c23a62f3a-a5a884925ffmr321408866b.42.1715778681643;
        Wed, 15 May 2024 06:11:21 -0700 (PDT)
Message-ID: <b693b6ba-4ff9-44e8-bf35-590143a077cf@suse.com>
Date: Wed, 15 May 2024 15:11:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 04/19] x86: Lift mapcache variable to the arch
 level
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Wang <wawei@amazon.de>, Hongyan Xia <hongyxia@amazon.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-5-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-5-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -851,6 +851,8 @@ int arch_domain_create(struct domain *d,
>  
>      psr_domain_init(d);
>  
> +    mapcache_domain_init(d);

This new placement is re-done right away in the next patch. To me this is
another hint at it wanting considering to deal with the idle domain here
as well, right away.

> --- a/xen/arch/x86/domain_page.c
> +++ b/xen/arch/x86/domain_page.c
> @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
>  #endif
>  
>      v = mapcache_current_vcpu();
> -    if ( !v || !is_pv_vcpu(v) )
> +    if ( !v )
>          return mfn_to_virt(mfn_x(mfn));

While I don't mind this and ...

> @@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr)
>      ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
>  
>      v = mapcache_current_vcpu();
> -    ASSERT(v && is_pv_vcpu(v));
> +    ASSERT(v);

... this change being done already here, I don't think that's necessary.
Nor is it really related to what the subject and (so far at least)
description say is being done here. In which case the description wants
to at least mention why the adjustments are pulled ahead earlier than
where they're strictly needed.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:22:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:22:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722348.1126185 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EaK-0007Cd-Ge; Wed, 15 May 2024 13:22:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722348.1126185; Wed, 15 May 2024 13:22:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EaK-0007CW-Dn; Wed, 15 May 2024 13:22:20 +0000
Received: by outflank-mailman (input) for mailman id 722348;
 Wed, 15 May 2024 13:22:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7EaI-0007CQ-I6
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:22:18 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 25b22d65-12be-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 15:22:16 +0200 (CEST)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-78f05341128so523799685a.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:22:16 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2fca82sm677730685a.89.2024.05.15.06.22.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:22:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25b22d65-12be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715779335; x=1716384135; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gkwRZyhZIsCkGM+O4ci548+VvxsRNca7Pek4Id5P3uE=;
        b=nC3jBt5b3wgqrPpK251wM+nzLqv2PAFIrKPREzlsZk9xlnHzAf+/rhRUHF5yXlvuSf
         Y1CjkwHS+NpI3dnGC1bnRIPbjurzBISHoARhoIKe/QjTXq9KDQFaqkPajjHu46PcGumv
         BUmEdk/dbXYS0NOrMejFLtsOqYGB7ZZ1rPLYQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715779335; x=1716384135;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gkwRZyhZIsCkGM+O4ci548+VvxsRNca7Pek4Id5P3uE=;
        b=R8htyluwLXHFnZ8gYDQfoJtaZ4WHV3Dp5XzZFpPyDuZ/WsITy8AkmHOX4uMr2nHYSB
         Jki3yVUbkYbub1a4tvEsYHVX2DkFe5jFYMhbh+RZe+z4FXjg4JY8UQzg6HXYvP+VTnlK
         LwBjRauxqtWRiqLv4MjRwIvWWD3pfsjhsoR65XVHwpBfkk9lR65vWxgKgbfLY+xblDrY
         w0HpNiZ4S5c0awdPgHasA43nvHvPebjv+Sl3fIxwGB22FvggHcX2Rq/yxOPQ3+z4bDLj
         n8+FTd3iewRCb7IS2bLgDMr0PAXusgXjF1HUOGJNIdGYIRW5aZ78UKsZSECMQ62UaF7w
         7IAA==
X-Gm-Message-State: AOJu0YylBYgqaucVgQ9KmTn44SUuZG6oa6u8rPL9/VEwt0hNP/mqAJk8
	pFEmlnvH3oDA1uPLdfpTV7fjvO417GnTpmQ+pMVFxEH9XRJ6VYASvQrxM3yD71A=
X-Google-Smtp-Source: AGHT+IEFFyFDs9z7a2aga/FFMV0LyyHsktk1U3Gv14xPxSYqFQwgAmyIw2e4ld7icrUK1ZV4wNizfQ==
X-Received: by 2002:ae9:f212:0:b0:790:c7f3:abce with SMTP id af79cd13be357-792c75b0916mr1720212385a.37.1715779335192;
        Wed, 15 May 2024 06:22:15 -0700 (PDT)
Message-ID: <9fdbf8a1-56a6-4be3-96a0-fcded0382cf0@citrix.com>
Date: Wed, 15 May 2024 14:22:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen crash in scheduler during cpu hotplug
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>
References: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
 <7553291c-f88d-49c7-a1bd-5f8dbd21fd2b@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7553291c-f88d-49c7-a1bd-5f8dbd21fd2b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 1:39 pm, Jan Beulich wrote:
> On 15.05.2024 13:58, Andrew Cooper wrote:
>> Just so it doesn't get lost.  In XenRT, we've found:
>>
>>> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
>>> (XEN) CPU:    45
>>> (XEN) RIP:    e008:[<ffff82d040244cbf>]
>>> common/sched/credit.c#csched_load_balance+0x41/0x877
>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
>>> 0000000000000000
>>> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
>>> ffff83103723d450
>>> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8: 
>>> 0000000000000000
>>> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
>>> 0000000000000009
>>> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
>>> ffff831037253720
>>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
>>> 0000000000f526e0
>>> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
>>> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
>>> 0000000000000000
>>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>>> (XEN) Xen code around <ffff82d040244cbf>
>>> (common/sched/credit.c#csched_load_balance+0x41/0x877):
>>> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
>>> 8b 4e 28 48
>>> <snip>
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff82d040244cbf>] R
>>> common/sched/credit.c#csched_load_balance+0x41/0x877
> While this is of course pretty little information, I've still tried to
> decipher it, first noticing it's credit1 that's being used here. Once
> forcing csched_load_balance() non-inline (no idea why it is a separate
> function in your build), I can see a sufficiently matching pattern at
> approximately the same offset into the function. That's
>
>     const struct cpupool *c = get_sched_res(cpu)->cpupool;
>     ...
>     const cpumask_t *online = c->res_valid;
>     ...
>     BUG_ON(get_sched_res(cpu) != snext->unit->res);
>
> overlapping, with the crash being on the middle of the quoted lines.
> IOW the CPU pool is still NULL for this sched resource. Cc-ing
> Jürgen for possible clues ...

We've seen it in 4.13, 4.17 and upstream, after Roger extended the
existing CPU hotplug testing to try and reproduce the MTRR watchdog
failure.  We've found yet another "no irq for handler" from this too.

It's always a deference at NULL+0x10, somewhere within csched_schedule().

---8<---
#!/bin/bash

set -e

CPUS=$(xl info nr_cpus)

while :
do
    for C in $(seq 1 $(($CPUS-1)))
    do
        echo "Offline $C"
        xen-hptool cpu-offline $C
        echo "Online $C"
        xen-hptool cpu-online $C
    done
done
---8<---

does rather well at reproducing the crash.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:28:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:28:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722352.1126198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Eg5-0007pI-4k; Wed, 15 May 2024 13:28:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722352.1126198; Wed, 15 May 2024 13:28:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Eg5-0007pB-29; Wed, 15 May 2024 13:28:17 +0000
Received: by outflank-mailman (input) for mailman id 722352;
 Wed, 15 May 2024 13:28:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=733o=MS=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s7Eg3-0007p5-8s
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:28:15 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fb279952-12be-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:28:14 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-56e56ee8d5cso1956875a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:28:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb279952-12be-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715779693; x=1716384493; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=o2yFgV5wbekf3+V9EnlkTrKV7kIXvg0Qz2m+9CZK1zc=;
        b=KwjdA1qpfoBu3xRrqUvPjwN2k4PEMsJPvQ8LFjtuKCopXpZ8oUKT5JmjyaIlRWO1jB
         W0IhC4cqw1JY+Fgi9lgilnxvJ9SA/AVCFfNnDV8ARRbMPwO3XMPlJdFaLf+saqaGUWwb
         3Z+2/eIKnRR1m1Vt92unYoT8hTWOPaqzf5kYE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715779693; x=1716384493;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=o2yFgV5wbekf3+V9EnlkTrKV7kIXvg0Qz2m+9CZK1zc=;
        b=Zzu1pQ8glXfkBDOfxHOGYilmeMihEeN96noHjF7ebMlG/lkPBQX69m6I9TNQpFDXKW
         th2aSx3vUHMHB0c8HYa3nWMQe0WIU4h32fk/4N09ertBx2KwgtSdBmCtIuGL0brTfqx2
         zH2a9JxFbSl00OgK0mVnl+N8hmQ6GhvnT72FNUF7mSCNe7PK7W9aiirQNoIwq//x8N93
         7uav9O4xipQrB7QB+mD8qY0Y/t2udZbCf2Fo/1qarDFWwhqYKbjFsVs/SY1R5uQ958dp
         QphYzglHLmX6i6/x5YfwxSTtGrd4sQK5olvU+QS5jojVWvodfKvDA85uqUggat0iKvYI
         anww==
X-Forwarded-Encrypted: i=1; AJvYcCUOuuiM6DHVwmKDkcE0nQBReUF6NWaTCvbCmsY5J3efI2ulLFXMb+xS5PyW/pavP8XctLTxlFZWX0nwbPLGB5Fyf7MnLhViLVqeVcQyVpY=
X-Gm-Message-State: AOJu0Yy5DaEUEQNGy8ueQenWT69EbvZr9W3kBF03CPDZbAidtwztcrmB
	Kc0seGLCmDZjeN3ocuBf5BSSofW+UAHxJhp6m6q8pBX1h3psJXmleI9wZjmGc8XQow1zSHkcelC
	1upiv3XDES3vtbZ6bh81rFjyhExM2MLVT9nFBHg==
X-Google-Smtp-Source: AGHT+IEIf2sUaKa4QiGoA8ai9I+590BRFLEmJMLzXLk6bwOEZ5PUhFwasICKPOn86nchDbr/pfD7thrwOZkCR5RkgOE=
X-Received: by 2002:a17:906:2b48:b0:a59:c9ce:338b with SMTP id
 a640c23a62f3a-a5a2d5f1479mr1162978166b.35.1715779693256; Wed, 15 May 2024
 06:28:13 -0700 (PDT)
MIME-Version: 1.0
References: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
 <bba91b8b-b1cc-4054-96e8-80a63a3b808a@citrix.com> <32f16048-c1ba-4ebb-a752-a77ed529a99c@amd.com>
In-Reply-To: <32f16048-c1ba-4ebb-a752-a77ed529a99c@amd.com>
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 15 May 2024 14:27:37 +0100
Message-ID: <CAO-mL=x0=oyOY_ZVcECz2ZrVf8Mtz7pYk8XXWfjM2Ww08W=jbA@mail.gmail.com>
Subject: Re: Proposal to Extend Feature Freeze Deadline
To: Henry Wang <xin.wang2@amd.com>
Cc: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org, 
	committers@xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="0000000000002150db06187e179a"

--0000000000002150db06187e179a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Oleksii,

If there are no objections by tomorrow, let's assume by lazy consensus that
we will extend the timeline by a week.
If anyone objects to this, please reply to this email.

Many thanks,
Kelly Choi

Community Manager
Xen Project


On Wed, May 15, 2024 at 4:13=E2=80=AFAM Henry Wang <xin.wang2@amd.com> wrot=
e:

> Hi Oleksii,
>
> On 5/14/2024 11:43 PM, Andrew Cooper wrote:
> > On 14/05/2024 4:40 pm, Oleksii K. wrote:
> >> Hello everyone,
> >>
> >> We're observing fewer merged patches/series across several
> >> architectures for the current 4.19 release in comparison to previous
> >> release.
> >>
> >> For example:
> >> 1. For Arm, significant features like Cache Coloring and PCI
> >> Passthrough won't be fully merged. Thus, it would be beneficial to
> >> commit at least the following two patch series:
> >> [1]
> https://lore.kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@amd.co=
m/
> >>
> >> [2]
> https://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.c=
om/
> >>
> >> 2. For RISC-V, having the following patch series [3], mostly reviewed
> >> with only one blocker [4], would be advantageous (As far as I know,
> >> Andrew is planning to update his patch series):
> >> [3]
> https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@g=
mail.com/
> >> [4]
> >>
> https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.co=
m/
> >>
> >> 3. For PPC, it would be beneficial to have [5] merged:
> >> [5]
> >>
> https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptore=
ngineering.com/
> >>
> >> Extending the feature freeze deadline by one week, until May 24th,
> >> would provide additional time for merges mentioned above. This, in
> >> turn, would create space for more features and fixes for x86 and other
> >> common elements. If we agree to extend the feature freeze deadline,
> >> please feel free to outline what you would like to see in the 4.19
> >> release. This will make it easier to track our final goals and
> >> determine if they are realistically achievable.
> >>
> >> I'd like to open the floor for discussion on this proposal. Does it
> >> make sense, and would it be useful?
> > Considering how many people are blocked on me, I'd welcome a little bit
> > longer to get the outstanding series/fixes to land.
>
> It would be great if we can extend the deadline for a week, thank you! I
> will try my best to make progress of the two above-mentioned Arm series.
>
> Kind regards,
> Henry
>

--0000000000002150db06187e179a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Oleksii,<div><br></div><div>If there are no objections =
by tomorrow, let&#39;s assume by lazy consensus=C2=A0that we will extend th=
e timeline by a week.=C2=A0</div><div>If anyone=C2=A0objects to this, pleas=
e reply to this=C2=A0email.</div><div><br clear=3D"all"><div><div dir=3D"lt=
r" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D=
"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></div><div><div =
style=3D"color:rgb(136,136,136)">Community Manager</div><div style=3D"color=
:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></div></div></div><br><=
/div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a=
ttr">On Wed, May 15, 2024 at 4:13=E2=80=AFAM Henry Wang &lt;<a href=3D"mail=
to:xin.wang2@amd.com">xin.wang2@amd.com</a>&gt; wrote:<br></div><blockquote=
 class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so=
lid rgb(204,204,204);padding-left:1ex">Hi Oleksii,<br>
<br>
On 5/14/2024 11:43 PM, Andrew Cooper wrote:<br>
&gt; On 14/05/2024 4:40 pm, Oleksii K. wrote:<br>
&gt;&gt; Hello everyone,<br>
&gt;&gt;<br>
&gt;&gt; We&#39;re observing fewer merged patches/series across several<br>
&gt;&gt; architectures for the current 4.19 release in comparison to previo=
us<br>
&gt;&gt; release.<br>
&gt;&gt;<br>
&gt;&gt; For example:<br>
&gt;&gt; 1. For Arm, significant features like Cache Coloring and PCI<br>
&gt;&gt; Passthrough won&#39;t be fully merged. Thus, it would be beneficia=
l to<br>
&gt;&gt; commit at least the following two patch series:<br>
&gt;&gt; [1]<a href=3D"https://lore.kernel.org/xen-devel/20240511005611.831=
25-1-xin.wang2@amd.com/" rel=3D"noreferrer" target=3D"_blank">https://lore.=
kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@amd.com/</a><br>
&gt;&gt;=C2=A0 =C2=A0 <br>
&gt;&gt; [2]<a href=3D"https://lore.kernel.org/xen-devel/20240424033449.168=
398-1-xin.wang2@amd.com/" rel=3D"noreferrer" target=3D"_blank">https://lore=
.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.com/</a><br>
&gt;&gt;<br>
&gt;&gt; 2. For RISC-V, having the following patch series [3], mostly revie=
wed<br>
&gt;&gt; with only one blocker [4], would be advantageous (As far as I know=
,<br>
&gt;&gt; Andrew is planning to update his patch series):<br>
&gt;&gt; [3]<a href=3D"https://lore.kernel.org/xen-devel/cover.1713347222.g=
it.oleksii.kurochko@gmail.com/" rel=3D"noreferrer" target=3D"_blank">https:=
//lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko@gmail.com=
/</a><br>
&gt;&gt; [4]<br>
&gt;&gt; <a href=3D"https://patchew.org/Xen/20240313172716.2325427-1-andrew=
.cooper3@citrix.com/" rel=3D"noreferrer" target=3D"_blank">https://patchew.=
org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/</a><br>
&gt;&gt;<br>
&gt;&gt; 3. For PPC, it would be beneficial to have [5] merged:<br>
&gt;&gt; [5]<br>
&gt;&gt; <a href=3D"https://lore.kernel.org/xen-devel/cover.1712893887.git.=
sanastasio@raptorengineering.com/" rel=3D"noreferrer" target=3D"_blank">htt=
ps://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptorengine=
ering.com/</a><br>
&gt;&gt;<br>
&gt;&gt; Extending the feature freeze deadline by one week, until May 24th,=
<br>
&gt;&gt; would provide additional time for merges mentioned above. This, in=
<br>
&gt;&gt; turn, would create space for more features and fixes for x86 and o=
ther<br>
&gt;&gt; common elements. If we agree to extend the feature freeze deadline=
,<br>
&gt;&gt; please feel free to outline what you would like to see in the 4.19=
<br>
&gt;&gt; release. This will make it easier to track our final goals and<br>
&gt;&gt; determine if they are realistically achievable.<br>
&gt;&gt;<br>
&gt;&gt; I&#39;d like to open the floor for discussion on this proposal. Do=
es it<br>
&gt;&gt; make sense, and would it be useful?<br>
&gt; Considering how many people are blocked on me, I&#39;d welcome a littl=
e bit<br>
&gt; longer to get the outstanding series/fixes to land.<br>
<br>
It would be great if we can extend the deadline for a week, thank you! I <b=
r>
will try my best to make progress of the two above-mentioned Arm series.<br=
>
<br>
Kind regards,<br>
Henry<br>
</blockquote></div>

--0000000000002150db06187e179a--


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:35:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:35:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722355.1126210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Emu-0001cR-TT; Wed, 15 May 2024 13:35:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722355.1126210; Wed, 15 May 2024 13:35:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Emu-0001cK-Os; Wed, 15 May 2024 13:35:20 +0000
Received: by outflank-mailman (input) for mailman id 722355;
 Wed, 15 May 2024 13:35:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Emt-0001cE-Is
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:35:19 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f7f9eb4b-12bf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:35:18 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so175080166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:35:18 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a813cd3dasm234533866b.45.2024.05.15.06.35.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:35:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7f9eb4b-12bf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715780117; x=1716384917; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5lVIV7SdRR2/xZ3RC6TEhrkP/iWMaqIOeKVzrJaSm3g=;
        b=Hb2tUNo/9DNFCBqEkVPHdpKP4+2n/OtD0cwa4rpVb5EKonp88zWPGis+UdR2Cmz6He
         KEry5AdxGkQ7W3kAWtsCaWmSjIP1vaplZpT4X5xbVqa17NxgCbL8jzdyqklWFV32n/uf
         xJnnBOSOzB5qyZGhHzThl3J4FfOrFmNGZB2DwW1WBGsUp3fJUKka0bK8g6SBReIFEZod
         /+GSPbMtv46Z/iwOBdAb2jSDJ2/gsLOSZX3yqYVrxTnE5aPUfdAHygRTIeH0tHKPtd94
         XjAbx42m8H+rJ5bIFhGDmfG1eIobli2qk27hl1itOhFtimlbUoovJI3ooRJ3BUseoGFo
         9hMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715780117; x=1716384917;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5lVIV7SdRR2/xZ3RC6TEhrkP/iWMaqIOeKVzrJaSm3g=;
        b=B78VoZzSj+bZmEGv8tEbrEF2pPxp81s/veheyw//Rl15jLgPd8qBbfMGwOaS54m3NZ
         6mLwfIXBN7ucHUYOBOH0A0wCPZ6oYCHVJ7ZSiJWhHLnbZ7YjXmtIr5tJ1Q8nXLv5CPSb
         LHYOOYfXTYdi5X63g0XhabFqqmHXpJZ04qFm35ag7OHqtSzi5qgJUW41fHK2IOiL+QkQ
         0IzjQ0Mg0nPI/IZayhWJu7G4icrnH7ULAxe3kG+3sAzoHhfx9uCAVPwhcmNZcWbbW4Ls
         f8Xl27XdbSw7t+JruvqSdcbH+5PhQeMjn1ID50xNSTaxnoS35zZkyAjGC5pD1/L0n4Fi
         BJHg==
X-Gm-Message-State: AOJu0YzwyyWzCp6jpgizrkfXFt6pk7cNB3xGUozFGoG5zeyxaQiyCm1+
	j8Btb7DJy20GSjUsdyi2reziYJodJ+VTzX+eymOu/2n69iMVz8RaapWijf4Mm0uHxi+wHMwPXng
	=
X-Google-Smtp-Source: AGHT+IGBXko6mFBtn6CS5dy/lVH8hKVKaXD08OXyKoSaF3VDf1vRICSGMwM8FHnGBByynCOaBQDAmg==
X-Received: by 2002:a17:906:22c7:b0:a59:be21:3577 with SMTP id a640c23a62f3a-a5a2d5cd472mr1026405166b.43.1715780117637;
        Wed, 15 May 2024 06:35:17 -0700 (PDT)
Message-ID: <dc80cf26-8900-4848-84dd-6f3e47d59f57@suse.com>
Date: Wed, 15 May 2024 15:35:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86: respect mapcache_domain_init() failing
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The function itself properly handles and hands onwards failure from
create_perdomain_mapping(). Therefore its caller should respect possible
failure, too.

Fixes: 4b28bf6ae90b ("x86: re-introduce map_domain_page() et al")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Effectively split off of "x86/mapcache: Initialise the mapcache for the
idle domain", except that the change to respect the error wasn't even
mentioned there. Plus we likely want to backport this.

--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -858,7 +858,8 @@ int arch_domain_create(struct domain *d,
     }
     else if ( is_pv_domain(d) )
     {
-        mapcache_domain_init(d);
+        if ( (rc = mapcache_domain_init(d)) != 0 )
+            goto fail;
 
         if ( (rc = pv_domain_initialise(d)) != 0 )
             goto fail;


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:38:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:38:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722361.1126218 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EqM-0002G8-Cd; Wed, 15 May 2024 13:38:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722361.1126218; Wed, 15 May 2024 13:38:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EqM-0002G1-9g; Wed, 15 May 2024 13:38:54 +0000
Received: by outflank-mailman (input) for mailman id 722361;
 Wed, 15 May 2024 13:38:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8u5V=MS=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s7EqL-0002Fo-HF
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:38:53 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 77f67b57-12c0-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:38:52 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-56e1baf0380so1985314a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:38:52 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea6586sm9063284a12.14.2024.05.15.06.38.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:38:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77f67b57-12c0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715780332; x=1716385132; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+IHLvH0jBQ36Nd1AJeKYFlGQwwOrZNPOSXcdtFh5fo0=;
        b=cf+Y3OmecZ8/nV55fkyz38u7H8aOImAAygZyBuzDT6C80dSFSlj8ueDvoVRYJHEH1V
         JAZ5hkLJvgMV3S45g24hLT6ZsNbOgsmv2C6vmIhk5ZqHKmyc0kvgUroQMHvfUwh3utIZ
         KL34Oar5nDRtwBQmys5P94uR9b5Ac0iabHQfLTV9Nca8+ERP6HjaMVqJVxHgULDBkoPk
         Eo3yzT3VBEpw6jBlbWoxUiA+HdiKYiNlSyvVreCx8YWSVOW6eOzV1rTqKHqjhvid+lH+
         0WJ8elYTeQDiflAwHwSBCwKe2nmgzU5MHRnjzJD5ErSfenyEIScWV/cs5PBMBL/7sO5y
         m4LA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715780332; x=1716385132;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+IHLvH0jBQ36Nd1AJeKYFlGQwwOrZNPOSXcdtFh5fo0=;
        b=fgRXiZhHw8nPp2vG5hniar4ec8t9N5SVPnFqmadwXKnEkT2eFc/jI2o6b4SSLIySPn
         dOXiDYSEkCZ2ijY7P0bBpf5XDwRdiGxzWhFWCC5tCshpR2Uk6S1QqLUNh7IqShoqD4dK
         1wMQS0TzqJKOTaACzQGkPuAEUAP4/zKulFFBepLCpzCIQwNKKx1LBtBfWTyStlk3/J1t
         G/81yZ8pSN1xKpHS2E4qKAA7oXeJstu7UAu1COGnK7vtHv0XZymZQe5EG3+/+gTudOhi
         HKWhJgHYZmzb8fPqnnsfta5VggQIV/dhtSL0hAfpPqlWPioYDy5rRMprxcb0Y+JKzoU1
         seLA==
X-Gm-Message-State: AOJu0YzCPkGrUUN1pcGGtfWbtrlnOZ4i8AVCJNg95nbycaLPnR4OhzON
	eaGbc7I0EKJL7CeYhwbo8S4mCd90UjT7YWj9Mi9UVULszZ/PTDAlqSHXvPmNrx8=
X-Google-Smtp-Source: AGHT+IGOgXOmPffKTpU+z1Rl1VO+wY9OPMHGn9q9sNEJ9dvaC+U8AelZcxuPQDnZ4g+MNGa8/f5UAg==
X-Received: by 2002:a50:998e:0:b0:572:3cc4:2dcf with SMTP id 4fb4d7f45d1cf-5734d5be329mr12201843a12.14.1715780332150;
        Wed, 15 May 2024 06:38:52 -0700 (PDT)
Message-ID: <6766f9f1-2ceb-42d8-8ddd-f98671106579@suse.com>
Date: Wed, 15 May 2024 15:38:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen crash in scheduler during cpu hotplug
To: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>
References: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
 <7553291c-f88d-49c7-a1bd-5f8dbd21fd2b@suse.com>
 <9fdbf8a1-56a6-4be3-96a0-fcded0382cf0@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <9fdbf8a1-56a6-4be3-96a0-fcded0382cf0@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 15.05.24 15:22, Andrew Cooper wrote:
> On 15/05/2024 1:39 pm, Jan Beulich wrote:
>> On 15.05.2024 13:58, Andrew Cooper wrote:
>>> Just so it doesn't get lost.  In XenRT, we've found:
>>>
>>>> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
>>>> (XEN) CPU:    45
>>>> (XEN) RIP:    e008:[<ffff82d040244cbf>]
>>>> common/sched/credit.c#csched_load_balance+0x41/0x877
>>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>>> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
>>>> 0000000000000000
>>>> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
>>>> ffff83103723d450
>>>> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8:
>>>> 0000000000000000
>>>> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
>>>> 0000000000000009
>>>> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
>>>> ffff831037253720
>>>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
>>>> 0000000000f526e0
>>>> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
>>>> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
>>>> 0000000000000000
>>>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>>>> (XEN) Xen code around <ffff82d040244cbf>
>>>> (common/sched/credit.c#csched_load_balance+0x41/0x877):
>>>> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
>>>> 8b 4e 28 48
>>>> <snip>
>>>> (XEN) Xen call trace:
>>>> (XEN)    [<ffff82d040244cbf>] R
>>>> common/sched/credit.c#csched_load_balance+0x41/0x877
>> While this is of course pretty little information, I've still tried to
>> decipher it, first noticing it's credit1 that's being used here. Once
>> forcing csched_load_balance() non-inline (no idea why it is a separate
>> function in your build), I can see a sufficiently matching pattern at
>> approximately the same offset into the function. That's
>>
>>      const struct cpupool *c = get_sched_res(cpu)->cpupool;
>>      ...
>>      const cpumask_t *online = c->res_valid;
>>      ...
>>      BUG_ON(get_sched_res(cpu) != snext->unit->res);
>>
>> overlapping, with the crash being on the middle of the quoted lines.
>> IOW the CPU pool is still NULL for this sched resource. Cc-ing
>> Jürgen for possible clues ...
> 
> We've seen it in 4.13, 4.17 and upstream, after Roger extended the
> existing CPU hotplug testing to try and reproduce the MTRR watchdog
> failure.  We've found yet another "no irq for handler" from this too.
> 
> It's always a deference at NULL+0x10, somewhere within csched_schedule().

I think I've found the reason.

In schedule_cpu_add() the cpupool and granularity are set only after
releasing the scheduling lock. I think those must be inside the locked
region.

Can you give this one a try (not tested at all)?

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 0cb33831d2..babac7aad6 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -3176,6 +3176,8 @@ int schedule_cpu_add(unsigned int cpu, struct cpupool *c)

      sr->scheduler = new_ops;
      sr->sched_priv = ppriv;
+    sr->granularity = cpupool_get_granularity(c);
+    sr->cpupool = c;

      /*
       * Reroute the lock to the per pCPU lock as /last/ thing. In fact,
@@ -3188,8 +3190,6 @@ int schedule_cpu_add(unsigned int cpu, struct cpupool *c)
      /* _Not_ pcpu_schedule_unlock(): schedule_lock has changed! */
      spin_unlock_irqrestore(old_lock, flags);

-    sr->granularity = cpupool_get_granularity(c);
-    sr->cpupool = c;
      /* The  cpu is added to a pool, trigger it to go pick up some work */
      cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ);


Juergen



From xen-devel-bounces@lists.xenproject.org Wed May 15 13:41:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:41:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722364.1126229 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Esa-0004CZ-OW; Wed, 15 May 2024 13:41:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722364.1126229; Wed, 15 May 2024 13:41:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Esa-0004CS-LH; Wed, 15 May 2024 13:41:12 +0000
Received: by outflank-mailman (input) for mailman id 722364;
 Wed, 15 May 2024 13:41:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=733o=MS=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s7EsZ-0004CM-Iw
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:41:11 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c94f0ed1-12c0-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 15:41:09 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso158423666b.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:41:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c94f0ed1-12c0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715780469; x=1716385269; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=wmrbt5g3xW5fHBu0hr4SidkuSpkmyoXxC17lF1jRack=;
        b=TL7lv6DIctA/jMNh8xZ/1h8JLvTSymcJAuLwWgsFEOFE0UmyIQI/hyJRHc8jku6XV4
         5pii10qzsI/e3eJYqnU/D6xzjDlKzl7xyaTF3tPdYcJCA6iE6DIHP72QwfetfCdMXpaT
         f9xUFbAnuf8+4b2qas8/1w5otD5lASVvvwVGg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715780469; x=1716385269;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=wmrbt5g3xW5fHBu0hr4SidkuSpkmyoXxC17lF1jRack=;
        b=uZPxvvc7PxfnAUuGyVqAA2rnsQjNZJRghur0P4iHWldjka6HlYrd7IMG3ZfWKAkv2u
         eJytoWq1OcsEw6fju2d74j3lQxPpaS4fgc9j60eQypDspzvkiG5/qdf7jat9fDFh+zWS
         IhLh4hZBgBCIzxQN5d1hAD6fBowWLlEKCTbPteTuhsqtnCdcS3IR5c5UTXquoWjIonXr
         LQKjVcf/uT8qb15DzKaM/fHh9uBcsgG0F+741va7QjOxcWr4WE+qETYmOkmZIhFSWcLd
         6+YKyuHOgZl92I82ZVL7pf0fvNjh+rWdKMWJjj0hd08T43a/NV7mxJ31c79bj48K/KAq
         pdaA==
X-Forwarded-Encrypted: i=1; AJvYcCXR/sp+5EtDUAzVrshyb7wdccxYM0R+xQscZcuSpknf+gWMxiYOQtOO37SIBGR+ywVXpEnZlwg9GDDq4eh09Z/SGgW2yDPFa/Bbhd3VrZI=
X-Gm-Message-State: AOJu0YwTWlj+LVSZihjyxFewah6N3LLWQHSIirmm8PZ1HUM8F8KqBPJT
	RPL1cHWDGlfCXtR+QEgFUJX7162lV7KND6/GveIkIaBFSx1J/pdbeIjc7oSzQBXABM+B0zZs8+3
	YnbAoNjt99SfYIzX2aiIGV4XaLgfQ034glw7Abg==
X-Google-Smtp-Source: AGHT+IEscZaGsDpASJG1curHpIeR7pTebTZ9pxb2og0x4CArt/kD6LFJNrQieG1bc96DAjGzRza62K0inC2mkZyn1Fc=
X-Received: by 2002:a17:906:7f95:b0:a59:c833:d274 with SMTP id
 a640c23a62f3a-a5a2d5cb2a0mr1621341866b.37.1715780468723; Wed, 15 May 2024
 06:41:08 -0700 (PDT)
MIME-Version: 1.0
References: <c2ce4002-58d5-48a3-949c-3c361c78c0ac@suse.com>
 <ZhdNxWNpM0KCzz8E@mattapan.m5p.com> <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com> <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com> <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook> <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
 <08503965-4ba9-4fe3-a176-895623f4ad92@suse.com> <ZkPO0D0/Jx4ZnRBO@mattapan.m5p.com>
In-Reply-To: <ZkPO0D0/Jx4ZnRBO@mattapan.m5p.com>
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 15 May 2024 14:40:31 +0100
Message-ID: <CAO-mL=z=VsGx2Cimw7bW5bx8BBF+xfckEMBafCmqSi8GpFtdXw@mail.gmail.com>
Subject: Re: Serious AMD-Vi(?) issue
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: multipart/alternative; boundary="0000000000005a04a206187e452b"

--0000000000005a04a206187e452b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Elliott,

Most of our developers are based in the EU timezone, however we are a
worldwide community.
The Xen Project is an open source community that everyone contributes to
and we do not divide how we provide help, based on location.

As explained previously, we are happy to help resolve issues and provide
advice where necessary. However, to do this, our developers need the
relevant information to provide accurate resolutions. Given that our
developers have repeatedly voiced their concerns, and are debugging this
out of interest, please help us by providing all the necessary information.

Until we have this information, it will be very difficult to help you
further. Should anything change, we would be glad to assist you.

Many thanks,
Kelly Choi

Community Manager
Xen Project


On Tue, May 14, 2024 at 9:51=E2=80=AFPM Elliott Mitchell <ehem+xen@m5p.com>=
 wrote:

> On Tue, May 14, 2024 at 10:22:51AM +0200, Jan Beulich wrote:
> > On 13.05.2024 22:11, Elliott Mitchell wrote:
> > > On Mon, May 13, 2024 at 10:44:59AM +0200, Roger Pau Monn=C3=A9 wrote:
> > >> Why do you mask the device SBDF in the above snippet?  I would reall=
y
> > >> like to understand what's so privacy relevant in a PCI SBDF number.
> > >
> > > I doubt it reveals much.  Simply seems unlikely to help debugging and
> > > therefore I prefer to mask it.
> >
> > SBDF in one place may be matchable against a memory address in another
> > place. _Any_ hiding of information is hindering analysis. Please can
> > you finally accept that it needs to be the person doing the analysis
> > to judge what is or is not relevant to them?
>
> Not going to happen as I'd accepted this long ago.  The usual approach
> is all developers have PGP keys (needed for security issues anyway) and
> you don't require all logs to be public.
>
> I've noticed the core of the Xen project appears centered in the EU.  Yet
> you're not catering to data privacy at all?  Or is this a service
> exclusively provided to people who prove they're EU citizens?
>
>
> --
> (\___(\___(\______          --=3D> 8-) EHM <=3D--          ______/)___/)_=
__/)
>  \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
>   \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
> 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
>
>
>

--0000000000005a04a206187e452b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello Elliott,<div><br></div><div>Most of our developers a=
re based in the EU timezone, however we are a worldwide community.=C2=A0</d=
iv><div>The Xen Project is an open source community that everyone contribut=
es to and we do not divide how we provide help, based on location.=C2=A0</d=
iv><div><br></div><div>As explained previously, we are happy to help resolv=
e issues and provide advice where necessary. However, to do this, our devel=
opers need the relevant information to provide accurate resolutions. Given =
that our developers have repeatedly voiced their concerns, and are debuggin=
g this out of interest, please help us by=C2=A0providing all the necessary =
information.=C2=A0</div><div><br></div><div>Until we have this information,=
 it will be very difficult to help you further. Should anything change, we =
would be glad to assist you.</div><div><br clear=3D"all"><div><div dir=3D"l=
tr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=
=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></div><div><d=
iv style=3D"color:rgb(136,136,136)">Community Manager</div><div style=3D"co=
lor:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></div></div></div><b=
r></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmai=
l_attr">On Tue, May 14, 2024 at 9:51=E2=80=AFPM Elliott Mitchell &lt;<a hre=
f=3D"mailto:ehem%2Bxen@m5p.com">ehem+xen@m5p.com</a>&gt; wrote:<br></div><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft:1px solid rgb(204,204,204);padding-left:1ex">On Tue, May 14, 2024 at 10:=
22:51AM +0200, Jan Beulich wrote:<br>
&gt; On 13.05.2024 22:11, Elliott Mitchell wrote:<br>
&gt; &gt; On Mon, May 13, 2024 at 10:44:59AM +0200, Roger Pau Monn=C3=A9 wr=
ote:<br>
&gt; &gt;&gt; Why do you mask the device SBDF in the above snippet?=C2=A0 I=
 would really<br>
&gt; &gt;&gt; like to understand what&#39;s so privacy relevant in a PCI SB=
DF number.<br>
&gt; &gt; <br>
&gt; &gt; I doubt it reveals much.=C2=A0 Simply seems unlikely to help debu=
gging and<br>
&gt; &gt; therefore I prefer to mask it.<br>
&gt; <br>
&gt; SBDF in one place may be matchable against a memory address in another=
<br>
&gt; place. _Any_ hiding of information is hindering analysis. Please can<b=
r>
&gt; you finally accept that it needs to be the person doing the analysis<b=
r>
&gt; to judge what is or is not relevant to them?<br>
<br>
Not going to happen as I&#39;d accepted this long ago.=C2=A0 The usual appr=
oach<br>
is all developers have PGP keys (needed for security issues anyway) and<br>
you don&#39;t require all logs to be public.<br>
<br>
I&#39;ve noticed the core of the Xen project appears centered in the EU.=C2=
=A0 Yet<br>
you&#39;re not catering to data privacy at all?=C2=A0 Or is this a service<=
br>
exclusively provided to people who prove they&#39;re EU citizens?<br>
<br>
<br>
-- <br>
(\___(\___(\______=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 --=3D&gt; 8-) EHM &lt;=
=3D--=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ______/)___/)___/)<br>
=C2=A0\BS (=C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"mail=
to:ehem%2Bsigmsg@m5p.com" target=3D"_blank">ehem+sigmsg@m5p.com</a>=C2=A0 P=
GP 87145445=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A0 )=C2=A0 =C2=A0/=
<br>
=C2=A0 \_CS\=C2=A0 =C2=A0|=C2=A0 _____=C2=A0 -O #include &lt;stddisclaimer.=
h&gt; O-=C2=A0 =C2=A0_____=C2=A0 |=C2=A0 =C2=A0/=C2=A0 _/<br>
8A19\___\_|_/58D2 7E3D DDF4 7BA6 &lt;-PGP-&gt; 41D1 B375 37D0 8714\_|_/___/=
5445<br>
<br>
<br>
</blockquote></div>

--0000000000005a04a206187e452b--


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:44:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:44:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722370.1126238 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EwD-0005B8-5A; Wed, 15 May 2024 13:44:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722370.1126238; Wed, 15 May 2024 13:44:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7EwD-0005B1-2b; Wed, 15 May 2024 13:44:57 +0000
Received: by outflank-mailman (input) for mailman id 722370;
 Wed, 15 May 2024 13:44:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7EwB-0005Ae-N8
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:44:55 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4f882467-12c1-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:44:54 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so130911666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:44:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1789247csm855446966b.82.2024.05.15.06.44.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:44:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f882467-12c1-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715780694; x=1716385494; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=rsvZ4ZNwSL4ar7njQK2oEdaNIK9eyRQLP3Whn7rPoVA=;
        b=GjuHo9TD4DYAOaqTOYmxsAwnnwt91mD0cIA3U84P+6cdYfs1SJnKBgaaxqsnpLvSw6
         GweIAkpTn0c436CIlQEE/PxeOiZuzNaSif3SMyznapvNxUabc6UR1+kLlGPHklVxN6IV
         2BiRKcSFd7/+EuPn3l5ywDkya9d3Og14yWUOfWiNsYqhsp5xwbU8NJ+Eb+ASlMqlNxpI
         0ef3gAPXrLQV4OcDAL+DGqVq+myqIgbLRcVe+aD6+8w6F5PMe0419wKYzhH0gV1Gj+gQ
         m3cZCELfzWX5BjbfM9EM4grXJc5pXY6hWMjA3QZnAbIBr4SgVjmyjf/RKg528klhJLc7
         2P8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715780694; x=1716385494;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rsvZ4ZNwSL4ar7njQK2oEdaNIK9eyRQLP3Whn7rPoVA=;
        b=TNzuymiU8dlp0lcUzduV2eeD6d+lqLipA6+1sO1310ZkvDyavPsDzWELgqlZCBsxJl
         VRmZmhaxUVVZjSST7CtjCmIpuv2uUcvsWdyZk57GBsVXSuz1wD9t6WrU9NqN3qZ0FLS0
         ss/bZ/MDPS5NhlNWTHtaFEgAxvNIZjHw2ygYf06XlLhCXhm3lELZz6YjgMaiq0F/9UWY
         cx0rqoEQgwwTGz6aa5TzkDhtGoKbIAr1T91HDZHd3Z7Kg18Ujf796SaVYOaYP9mqL6ef
         sjJQ+OPqkMU0RBGaz8LXbJzAXWZ4TiyH521lWM3FL4DETxIwWxVqOZQsek2x4f/wHGcN
         jG6A==
X-Gm-Message-State: AOJu0Yz+5OrCoOL8AxOVucvNyfV30XTsW5gB7AyGDMgMsy8oC+RUXBWk
	el0g0Y1QWr2sN3TiSXChxUZHquu2T5rPwlNLA6KkqQSluGWptoA5gvFA68aneNhODZzej7D6j7g
	=
X-Google-Smtp-Source: AGHT+IFPl22Nm+eYub6HG43ApKZBKwINqmG8s00dYMo4fh8VMSLGuEUKyhUJpc2j2l3MpwplUmthEw==
X-Received: by 2002:a17:906:3512:b0:a59:a282:5dbd with SMTP id a640c23a62f3a-a5a2d676474mr1548369966b.66.1715780694006;
        Wed, 15 May 2024 06:44:54 -0700 (PDT)
Message-ID: <f58e140d-4f47-48c0-a1de-8aeeac436393@suse.com>
Date: Wed, 15 May 2024 15:44:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 05/19] x86/mapcache: Initialise the mapcache
 for the idle domain
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Wang <wawei@amazon.de>,
 Julien Grall <jgrall@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-6-eliasely@amazon.com> <ZkMkBrf2G5dQllHc@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMkBrf2G5dQllHc@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 10:42, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:32PM +0000, Elias El Yandouzi wrote:
>> @@ -771,6 +778,9 @@ int arch_domain_create(struct domain *d,
>>  
>>          d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>>  
>> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
>> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
> 
> Albeit I think you could just use d->arch.perdomain_l3_pg directly
> here and avoid the local pg variable?
> 
> Would you mind adding:
> 
> /* Slot 260: Per-domain mappings. */
> 
> I wonder if it won't be better to just use init_xen_l4_slots() and
> special case the idle domain in there, instead of open-coding the L4
> population for the idle domain like this.

That would require changes to init_xen_l4_slots(), afaics, which I'm
not sure we actually want (for, perhaps, being more intrusive than adding
the two lines here).

However, with this I'd like to get back to my earlier remark regarding
the (further) moving of the mapcache_domain_init() invocation: You need
to touch this idle domain specific path anyway. With that I view your
earlier argument as yet more weak, even leaving aside the previously
indicated fallout that this further movement causes.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:45:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:45:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722371.1126249 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ewk-0005b1-DT; Wed, 15 May 2024 13:45:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722371.1126249; Wed, 15 May 2024 13:45:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ewk-0005au-AO; Wed, 15 May 2024 13:45:30 +0000
Received: by outflank-mailman (input) for mailman id 722371;
 Wed, 15 May 2024 13:45:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7Ewi-0005ZN-Ax
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:45:28 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 62f28f27-12c1-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:45:27 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-56e47843cc7so1668659a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:45:27 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574f6b8b9d7sm1179292a12.82.2024.05.15.06.45.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 06:45:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62f28f27-12c1-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715780726; x=1716385526; darn=lists.xenproject.org;
        h=mime-version:user-agent:references:in-reply-to:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Y67gsmKBYcde4btQwZ+FO2JNODx48qfwNGD5Nx/qsAw=;
        b=YBwgEHNyzSH6wTL61tttRSkpKOL+CYTFzLeRYw4CTLCaVJmAQRp7s2gfzewd3RF4Bw
         XbTU96TP7m6OO7dQed2icMbV2mQj9jU/HlxmbazX9oytAGhmAGjNiAP4Fbl4fP6frjtW
         +f+I+4ZHW146phlCF1lyYaADMiujSoZ9dnkiIqqjW6RLYhyeZyFQnDzKcqYGlKikO5iw
         uASRcONTdTBj/Gomi4wJFWP2/JaVPZ60NTBjWaN0DC3ehcfptSdypGbJzN6xfc4Fwgiz
         02cuy6w5I8D5S1vyizScuTo05mYXYx8mvPRUh87YfIr/hvGyq28TvND04mFWvgMjmF7D
         XjIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715780726; x=1716385526;
        h=mime-version:user-agent:references:in-reply-to:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Y67gsmKBYcde4btQwZ+FO2JNODx48qfwNGD5Nx/qsAw=;
        b=Eab8xHeu0BPpeUPoenW1cmzAd2++kCVdx4WZ3Un6btLqsyTeiaQS187ttvGD6qeUur
         NR/aO/qx1SxUv56gRGhX96Z02PKpcv7GoiMpfh9dnzXuae/jXfpMKjRPdXSx5IwFSOrU
         b+6Vp/HIznC62n+8MAqDTjIADpTbSUYde0+FghuFoPW0lC1x67LyBII3Z/aS5hUraE1r
         Fmqk71mL3a0/au9+TXPX8a7dssAw01mbje938326xlTnAMkaiTHRohqPPLuBCVjcFCmu
         E6cryYhnpA+9QkFbkbCdV73jqCdHEe9aVwgFYfmRdrmWVaAM0QrOmcuteK/DMsy76SCp
         QVlQ==
X-Gm-Message-State: AOJu0YwSTknWOcpbYeERklnzE4fWtN92vMA7my2GErRWrywseRedW0Cv
	WTM/87Y07dAXPpW4wNswNxUtwcaRmTz8dyzCjKWv/VSk2CXJP9ES
X-Google-Smtp-Source: AGHT+IHEvTO9Mp+c3PNHe5Ig08X3Pq1tWxbWS4/XISTL0dj0ozo76nzrVrwkgW/OaKEK5zhrySZ6Lg==
X-Received: by 2002:a50:c358:0:b0:574:ebfb:6d97 with SMTP id 4fb4d7f45d1cf-574ebfb6e33mr2657636a12.36.1715780726070;
        Wed, 15 May 2024 06:45:26 -0700 (PDT)
Message-ID: <b07f608f1ede847bfc4ac4d75ad7d3e97bca67d0.camel@gmail.com>
Subject: Re: Proposal to Extend Feature Freeze Deadline
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Kelly Choi <kelly.choi@cloud.com>, Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, committers@xenproject.org, Andrew Cooper
	 <andrew.cooper3@citrix.com>
Date: Wed, 15 May 2024 15:45:24 +0200
In-Reply-To: <CAO-mL=x0=oyOY_ZVcECz2ZrVf8Mtz7pYk8XXWfjM2Ww08W=jbA@mail.gmail.com>
References: <a72f75b9edc4255761ff676d6a09a966290a24df.camel@gmail.com>
	 <bba91b8b-b1cc-4054-96e8-80a63a3b808a@citrix.com>
	 <32f16048-c1ba-4ebb-a752-a77ed529a99c@amd.com>
	 <CAO-mL=x0=oyOY_ZVcECz2ZrVf8Mtz7pYk8XXWfjM2Ww08W=jbA@mail.gmail.com>
Content-Type: multipart/alternative; boundary="=-xZCGicTfQJQ2gos0fqvV"
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

--=-xZCGicTfQJQ2gos0fqvV
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Kelly,

On Wed, 2024-05-15 at 14:27 +0100, Kelly Choi wrote:
> Hi Oleksii,
>=20
> If there are no objections by tomorrow, let's assume by lazy
> consensus=C2=A0that we will extend the timeline by a week.=C2=A0
> If anyone=C2=A0objects to this, please reply to this=C2=A0email.
I will send a separate email tomorrow if no one objects.

~ Oleksii
>=20
> Many thanks,
> Kelly Choi
>=20
> Community Manager
> Xen Project=C2=A0
>=20
>=20
> On Wed, May 15, 2024 at 4:13=E2=80=AFAM Henry Wang <xin.wang2@amd.com> wr=
ote:
> > Hi Oleksii,
> >=20
> > On 5/14/2024 11:43 PM, Andrew Cooper wrote:
> > > On 14/05/2024 4:40 pm, Oleksii K. wrote:
> > >> Hello everyone,
> > >>
> > >> We're observing fewer merged patches/series across several
> > >> architectures for the current 4.19 release in comparison to
> > previous
> > >> release.
> > >>
> > >> For example:
> > >> 1. For Arm, significant features like Cache Coloring and PCI
> > >> Passthrough won't be fully merged. Thus, it would be beneficial
> > to
> > >> commit at least the following two patch series:
> > >>
> > [1]https://lore.kernel.org/xen-devel/20240511005611.83125-1-xin.wang2@a=
md.com/
> > >>=C2=A0 =C2=A0=20
> > >>
> > [2]https://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@=
amd.com/
> > >>
> > >> 2. For RISC-V, having the following patch series [3], mostly
> > reviewed
> > >> with only one blocker [4], would be advantageous (As far as I
> > know,
> > >> Andrew is planning to update his patch series):
> > >>
> > [3]https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kuroc=
hko@gmail.com/
> > >> [4]
> > >>
> > https://patchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.=
com/
> > >>
> > >> 3. For PPC, it would be beneficial to have [5] merged:
> > >> [5]
> > >>
> > https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@rapto=
rengineering.com/
> > >>
> > >> Extending the feature freeze deadline by one week, until May
> > 24th,
> > >> would provide additional time for merges mentioned above. This,
> > in
> > >> turn, would create space for more features and fixes for x86 and
> > other
> > >> common elements. If we agree to extend the feature freeze
> > deadline,
> > >> please feel free to outline what you would like to see in the
> > 4.19
> > >> release. This will make it easier to track our final goals and
> > >> determine if they are realistically achievable.
> > >>
> > >> I'd like to open the floor for discussion on this proposal. Does
> > it
> > >> make sense, and would it be useful?
> > > Considering how many people are blocked on me, I'd welcome a
> > little bit
> > > longer to get the outstanding series/fixes to land.
> >=20
> > It would be great if we can extend the deadline for a week, thank
> > you! I=20
> > will try my best to make progress of the two above-mentioned Arm
> > series.
> >=20
> > Kind regards,
> > Henry


--=-xZCGicTfQJQ2gos0fqvV
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<html><head><style>pre,code,address {
  margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
  margin-top: 0.2em;
  margin-bottom: 0.2em;
}
ol,ul {
  margin-top: 0em;
  margin-bottom: 0em;
}
blockquote {
  margin-top: 0em;
  margin-bottom: 0em;
}
</style></head><body><div>Hi Kelly,</div><div><br></div><div>On Wed, 2024-0=
5-15 at 14:27 +0100, Kelly Choi wrote:</div><blockquote type=3D"cite" style=
=3D"margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div=
 dir=3D"ltr">Hi Oleksii,<div><br></div><div>If there are no objections by t=
omorrow, let's assume by lazy consensus&nbsp;that we will extend the timeli=
ne by a week.&nbsp;</div><div>If anyone&nbsp;objects to this, please reply =
to this&nbsp;email.</div></div></blockquote><div>I will send a separate ema=
il tomorrow if no one objects.</div><div><br></div><div>~ Oleksii</div><blo=
ckquote type=3D"cite" style=3D"margin:0 0 0 .8ex; border-left:2px #729fcf s=
olid;padding-left:1ex"><div dir=3D"ltr"><div><br clear=3D"all"><div><div di=
r=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div=
 dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></div><di=
v><div style=3D"color:rgb(136,136,136)">Community Manager</div><div style=
=3D"color:rgb(136,136,136)">Xen Project&nbsp;<br></div></div></div></div></=
div><br></div></div><div><br></div><div class=3D"gmail_quote"><div dir=3D"l=
tr" class=3D"gmail_attr">On Wed, May 15, 2024 at 4:13=E2=80=AFAM Henry Wang=
 &lt;<a href=3D"mailto:xin.wang2@amd.com">xin.wang2@amd.com</a>&gt; wrote:<=
br></div><blockquote type=3D"cite" style=3D"margin:0 0 0 .8ex; border-left:=
2px #729fcf solid;padding-left:1ex"><div>Hi Oleksii,<br></div><div><br>On 5=
/14/2024 11:43 PM, Andrew Cooper wrote:<br>&gt; On 14/05/2024 4:40 pm, Olek=
sii K. wrote:<br>&gt;&gt; Hello everyone,<br>&gt;&gt;<br>&gt;&gt; We're obs=
erving fewer merged patches/series across several<br>&gt;&gt; architectures=
 for the current 4.19 release in comparison to previous<br>&gt;&gt; release=
.<br>&gt;&gt;<br>&gt;&gt; For example:<br>&gt;&gt; 1. For Arm, significant =
features like Cache Coloring and PCI<br>&gt;&gt; Passthrough won't be fully=
 merged. Thus, it would be beneficial to<br>&gt;&gt; commit at least the fo=
llowing two patch series:<br>&gt;&gt; [1]<a href=3D"https://lore.kernel.org=
/xen-devel/20240511005611.83125-1-xin.wang2@amd.com/" rel=3D"noreferrer" ta=
rget=3D"_blank">https://lore.kernel.org/xen-devel/20240511005611.83125-1-xi=
n.wang2@amd.com/</a><br>&gt;&gt;&nbsp; &nbsp; <br>&gt;&gt; [2]<a href=3D"ht=
tps://lore.kernel.org/xen-devel/20240424033449.168398-1-xin.wang2@amd.com/"=
 rel=3D"noreferrer" target=3D"_blank">https://lore.kernel.org/xen-devel/202=
40424033449.168398-1-xin.wang2@amd.com/</a><br>&gt;&gt;<br>&gt;&gt; 2. For =
RISC-V, having the following patch series [3], mostly reviewed<br>&gt;&gt; =
with only one blocker [4], would be advantageous (As far as I know,<br>&gt;=
&gt; Andrew is planning to update his patch series):<br>&gt;&gt; [3]<a href=
=3D"https://lore.kernel.org/xen-devel/cover.1713347222.git.oleksii.kurochko=
@gmail.com/" rel=3D"noreferrer" target=3D"_blank">https://lore.kernel.org/x=
en-devel/cover.1713347222.git.oleksii.kurochko@gmail.com/</a><br>&gt;&gt; [=
4]<br>&gt;&gt; <a href=3D"https://patchew.org/Xen/20240313172716.2325427-1-=
andrew.cooper3@citrix.com/" rel=3D"noreferrer" target=3D"_blank">https://pa=
tchew.org/Xen/20240313172716.2325427-1-andrew.cooper3@citrix.com/</a><br>&g=
t;&gt;<br>&gt;&gt; 3. For PPC, it would be beneficial to have [5] merged:<b=
r>&gt;&gt; [5]<br>&gt;&gt; <a href=3D"https://lore.kernel.org/xen-devel/cov=
er.1712893887.git.sanastasio@raptorengineering.com/" rel=3D"noreferrer" tar=
get=3D"_blank">https://lore.kernel.org/xen-devel/cover.1712893887.git.sanas=
tasio@raptorengineering.com/</a><br>&gt;&gt;<br>&gt;&gt; Extending the feat=
ure freeze deadline by one week, until May 24th,<br>&gt;&gt; would provide =
additional time for merges mentioned above. This, in<br>&gt;&gt; turn, woul=
d create space for more features and fixes for x86 and other<br>&gt;&gt; co=
mmon elements. If we agree to extend the feature freeze deadline,<br>&gt;&g=
t; please feel free to outline what you would like to see in the 4.19<br>&g=
t;&gt; release. This will make it easier to track our final goals and<br>&g=
t;&gt; determine if they are realistically achievable.<br>&gt;&gt;<br>&gt;&=
gt; I'd like to open the floor for discussion on this proposal. Does it<br>=
&gt;&gt; make sense, and would it be useful?<br>&gt; Considering how many p=
eople are blocked on me, I'd welcome a little bit<br>&gt; longer to get the=
 outstanding series/fixes to land.<br></div><div><br>It would be great if w=
e can extend the deadline for a week, thank you! I <br>will try my best to =
make progress of the two above-mentioned Arm series.<br></div><div><br>Kind=
 regards,<br>Henry<br></div></blockquote></div></blockquote><div><br></div>=
<div><span></span></div></body></html>

--=-xZCGicTfQJQ2gos0fqvV--


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:53:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:53:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722381.1126259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F3y-0008Ox-9I; Wed, 15 May 2024 13:52:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722381.1126259; Wed, 15 May 2024 13:52:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F3y-0008Oq-4n; Wed, 15 May 2024 13:52:58 +0000
Received: by outflank-mailman (input) for mailman id 722381;
 Wed, 15 May 2024 13:52:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7F3x-0008Og-Id; Wed, 15 May 2024 13:52:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7F3x-00027k-FX; Wed, 15 May 2024 13:52:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7F3x-00084E-6W; Wed, 15 May 2024 13:52:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7F3x-00060K-6G; Wed, 15 May 2024 13:52:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BpCBTHEkvBgoUDfbYyrG1+8aLkaFgguBJpva96p0/EA=; b=orML7XyU0MtTnPSd6QaI6T5STZ
	GqRsCeZgYnZxMeUZ4fmrKTNTM70UpTiw2Tu38qhwtlDdfzUMtIhB5Vh8cy/BD18zLwaBsDM2/mtnm
	8zAJG/+vghWuMq/PHPwu09q+X6ebTw1AXYBtVTF5wMB699QlDHdBeUMx5a21HigGp8bI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186003-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186003: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=2bcf14eabf02063fc7850a6b578f9bd6e26a6e60
X-Osstest-Versions-That:
    libvirt=2566522a55977e03c4df124b16aeaf38ad801bb1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 13:52:57 +0000

flight 186003 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186003/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185994
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              2bcf14eabf02063fc7850a6b578f9bd6e26a6e60
baseline version:
 libvirt              2566522a55977e03c4df124b16aeaf38ad801bb1

Last test of basis   185994  2024-05-14 04:20:38 Z    1 days
Testing same since   186003  2024-05-15 04:20:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhiram Tilak <atp.exp@gmail.com>
  Martin Kletzander <mkletzan@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Rayhan Faizel <rayhan.faizel@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   2566522a55..2bcf14eabf  2bcf14eabf02063fc7850a6b578f9bd6e26a6e60 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:55:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:55:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722386.1126268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F5u-0000WC-J4; Wed, 15 May 2024 13:54:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722386.1126268; Wed, 15 May 2024 13:54:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F5u-0000W5-GV; Wed, 15 May 2024 13:54:58 +0000
Received: by outflank-mailman (input) for mailman id 722386;
 Wed, 15 May 2024 13:54:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7F5s-0000Vv-Ev
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:54:56 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b48ab2a1-12c2-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 15:54:53 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59a387fbc9so207511366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:54:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm873729666b.14.2024.05.15.06.54.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:54:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b48ab2a1-12c2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715781293; x=1716386093; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T8KUSKK4f/TeN8ca0cnroVOq86D1LfE9MjRgPGGCXIE=;
        b=O1ypHrVxbvk35x/5t0IZG8z84hRKnb6EVvd9HqsSWxflhX2QUxLzS8V+xZjX6xBMFb
         eXWwiXr95irDrnSjLT87zaeC54E3ja1kyHsbVOYDUBJy8EzrtGCPazp3GYGpyDtiNYX0
         4W+JUYM/cZypVP0NNYZoIgZHFhGhOBuTrsAkSgKarZn3n5vR69A/TRu2mS8DrRf9GDPP
         uxk7RrRNX/UDeA8bNsoW9jpXeWQKw3JIP6Q3VIxbd6E874Qz5IyFx/tLzYkVUCJ+xXhP
         qWNZhk94dRcyRmAxH0E2S7Gds1Eeq0mNbi9xV6qFMl8efJFFtXSTMmp4BbwrzKHaEY0L
         ZWjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715781293; x=1716386093;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=T8KUSKK4f/TeN8ca0cnroVOq86D1LfE9MjRgPGGCXIE=;
        b=gQlttVNVQ80nWGqedJqaD4nMWiTPGFNjbfvqjLJ1EZ6qVadUHYLui+cR9TMWZA/D6w
         L1lUkBUNGLBERs5J18G9RAr8bHF3kLmGjg32LAny9Mjw4LZencvLB9W97T3CZc1R8zeL
         /2cmCEFYcsbMZljoKcqiNmzfsidQ8PVYYgUijhlKeBNCL8MSXFbW/A4Fh83pEwsAppVt
         0QlAOkZ5ld9E1Nn3GY8tZ65bzjDostXuLi6C/ZSMdsgflXrFIXDNnyrPr8q5igry4Sqx
         OkxWCp0FiM9grocXDAkhgP+JnT0NTyMWHuft3UQ2OzzKgz4bKVlQ8J1T/7a6lMvded0S
         +iPA==
X-Gm-Message-State: AOJu0YwEVtBF5rUU4pXsNO7p+PDHG7kBd7HFEjHrkfpwYr1y45K8AARO
	fSSssQi9DzBvbhlrBtzLgjxKYfpE42yvD2usKgwcNeIbp5T93DL0cmMJrYzZ0Q==
X-Google-Smtp-Source: AGHT+IH0mG1n3fdPPmn9yZbacvekDjwXnHL0KXJbHTifGCKaDYC/MOdnRraLuzqNhV3OUhEKrscZeQ==
X-Received: by 2002:a17:906:7815:b0:a59:c3bb:b83b with SMTP id a640c23a62f3a-a5a2d54c915mr890643566b.1.1715781292958;
        Wed, 15 May 2024 06:54:52 -0700 (PDT)
Message-ID: <086f1bbe-2b86-493e-8936-81f420500672@suse.com>
Date: Wed, 15 May 2024 15:54:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <jgrall@amazon.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com> <ZkMs1cknBFBWZoJG@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkMs1cknBFBWZoJG@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 11:20, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:33PM +0000, Elias El Yandouzi wrote:
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
>>  hardware guarantees (with, where available and known to Xen, respective
>>  tweaks applied).
>>  
>> +### directmap (x86)
>> +> `= <boolean>`
>> +
>> +> Default: `true`
>> +
>> +Enable or disable the directmap region in Xen.
> 
> Enable or disable fully populating the directmap region in Xen.

Elias, would you please take care to address earlier review comments
before sending a new version? This and ...

>> +
>> +By default, Xen creates the directmap region which maps physical memory
>                                                           ^ all?
>> +in that region. Setting this to no will sparsely populate the directmap,
> 
> "Setting this to no" => "Disabling this option will sparsely..."

... this is what I had already asked for in reply to v2, of course with
different wording.

>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>>      if ( highmem_start )
>>          xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
>>  
>> +    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
> 
> IMO this wants to be printed as part of the speculation mitigations, see
> print_details() in spec_ctrl.c

And not "on" / "off", but "full" / "sparse" (and word order changed accordingly)
perhaps.

>> --- a/xen/common/Kconfig
>> +++ b/xen/common/Kconfig
>> @@ -80,12 +80,29 @@ config HAS_PMAP
>>  config HAS_SCHED_GRANULARITY
>>  	bool
>>  
>> +config HAS_SECRET_HIDING
>> +	bool
>> +
>>  config HAS_UBSAN
>>  	bool
>>  
>>  config MEM_ACCESS_ALWAYS_ON
>>  	bool
>>  
>> +config SECRET_HIDING
>> +    bool "Secret hiding"
>> +    depends on HAS_SECRET_HIDING
> 
> IMO 'SECRET_HIDING' is too generic, this wants a more specific name.
> Maybe SPARCE_DIRECTMAP or some such.

This is another aspect I had raised on v2 already. SPARSE_DIRECTMAP
would be fine with me.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:55:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:55:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722387.1126279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F61-0000nA-PH; Wed, 15 May 2024 13:55:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722387.1126279; Wed, 15 May 2024 13:55:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7F61-0000n3-Mf; Wed, 15 May 2024 13:55:05 +0000
Received: by outflank-mailman (input) for mailman id 722387;
 Wed, 15 May 2024 13:55:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7F60-0000mQ-Ka
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:55:04 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id baa4d409-12c2-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:55:03 +0200 (CEST)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2e538a264f7so72617041fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:55:03 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0bbcc5fsm20471331fa.20.2024.05.15.06.55.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 06:55:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: baa4d409-12c2-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715781303; x=1716386103; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=OhqwbTjcYPKqjA3fcdjahtltKIvrjVq+INTIIqzGjtI=;
        b=FSmOWBf3iT5U0NQltLJUcRWz8OAkH1psFjCRtyUcRn1itkBub02bJjnKA/gyzBErDb
         f4PKnvKzyAPECdcjc+czCUILuemetxVRa6fYC2OXWrQ2Sbrbo9FNZ7KdI+PdtoOVwRhK
         za566tcRBnfsJCHJYM1IvQuBUiqZEbyZ26OT3BxC5qT4o4+fs7xXy8+bI/Mi1OtH/kSp
         Z0HOfoOqV1TioeejXA3uwgf0k2vaAVEEmYrfKW1iQmBggPUKanF3BoszEh4RcA/1YgVw
         OF3IwxHCNDgc3hJeFFXB0F+Xymy3kWmXTWfz0xu85Yntr7i74rkTiX3crj5iVANaZHmI
         W/LQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715781303; x=1716386103;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OhqwbTjcYPKqjA3fcdjahtltKIvrjVq+INTIIqzGjtI=;
        b=C3ElbWDUc10vFX8zRBt19onQPsMnaF+Jwd+AoQaLHXx10NX4FVg/yJNyBg68ZXXetF
         EfaJPGQ1oWQ/ISHA8YzuTlu8rcEL0RwdRjQGy8tAxlDZlDrji9xd6VXpQai1l6M6dByF
         jmOH+G3NqaaWNY35j/1P62kVuYjH7sXw5n9NMlDUGZjRpGIqMbDXUY9HlDExYv7XEjky
         VNVqo/5z9KjefyNXkgrhTo0+5czHbobl947h/Qcpu7p4B28W7zLGmVe0wyQXrcZsez0i
         y3OZz8PcrJklIpiS1uq7DEBPh/o45Yi48TJm5JlQZ2gnd1TYAITppdcawx28qcIj+UES
         ZpEA==
X-Forwarded-Encrypted: i=1; AJvYcCUVGHpp+KkAMrtq5Zgo9vkIWoYgDagQJ0HHHTBXIxOtR2Hk/kmF8rA3bulCrERK0n6khTi80GS5uUNaXNaJ7+ek4eW4mJ6ubBWhVZMMWY8=
X-Gm-Message-State: AOJu0YzkAIFE+mX6Qcg7VHA0kY7nNNi/ctwBo9x9iZLfP7/dm4eSf2El
	jRTrzneZqNWhExcrIdAFa+58OVWThZrxkwNs4qeIBJ1c3PSWEqXD
X-Google-Smtp-Source: AGHT+IGybDgSo05OkMM/MRG6inWht+8PNtPdDwzHGj+MKiMbwreRKFd4h5wtPObIQYQVC7TLYuHL+Q==
X-Received: by 2002:a2e:848f:0:b0:2de:48ef:c3e1 with SMTP id 38308e7fff4ca-2e51fd5949emr127218011fa.21.1715781302885;
        Wed, 15 May 2024 06:55:02 -0700 (PDT)
Message-ID: <46fd5c698fc13ffa3e8e82ed507bcfa67a9ef819.camel@gmail.com>
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Wed, 15 May 2024 15:55:01 +0200
In-Reply-To: <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
	 <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 11:09 +0200, Jan Beulich wrote:
> On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > Changes in V9:
> > =C2=A0- update return type of fls and flsl() to unsigned int to be
> > aligned with other
> > =C2=A0=C2=A0 bit ops.
>=20
> But this then needs carrying through to ...
>=20
> > --- a/xen/arch/arm/include/asm/arm64/bitops.h
> > +++ b/xen/arch/arm/include/asm/arm64/bitops.h
> > @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long
> > __ffs(unsigned long word)
> > =C2=A0 */
> > =C2=A0#define ffz(x)=C2=A0 __ffs(~(x))
> > =C2=A0
> > -static inline int flsl(unsigned long x)
> > +static inline int arch_flsl(unsigned long x)
>=20
> ... e.g. here. You don't want to introduce signed/unsigned
> mismatches.
Do you mean that generic flsl() uses 'unsigned int' as a return type,
but arch_flsl continue to use 'int'?

>=20
> Also why do you keep "inline" here, while ...
>=20
> > --- a/xen/arch/x86/include/asm/bitops.h
> > +++ b/xen/arch/x86/include/asm/bitops.h
> > @@ -425,7 +425,7 @@ static always_inline unsigned int
> > arch_ffsl(unsigned long x)
> > =C2=A0 *
> > =C2=A0 * This is defined the same way as ffs.
> > =C2=A0 */
> > -static inline int flsl(unsigned long x)
> > +static always_inline int arch_flsl(unsigned long x)
>=20
> ... you switch to always_inline here?
Because Adnrew's patch with bitops.h for x86 changes to always_inline,
so to be consistent, at least, for architecture.

~ Oleksii

>=20
> (replying out of order)
>=20
> > --- a/xen/arch/arm/include/asm/arm32/bitops.h
> > +++ b/xen/arch/arm/include/asm/arm32/bitops.h
> > @@ -1,7 +1,7 @@
> > =C2=A0#ifndef _ARM_ARM32_BITOPS_H
> > =C2=A0#define _ARM_ARM32_BITOPS_H
> > =C2=A0
> > -#define flsl fls
> > +#define arch_flsl fls
>=20
> It's the Arm maintainers to ultimately judge, but I'd be inclined to
> suggest
>=20
> #define arch_flsl arch_fls
>=20
> instead. That's not only behaviorally closer to what was there
> before, but
> also reduces (a tiny bit) the amount of work the compiler needs to
> carry out.
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Wed May 15 13:59:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:59:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722393.1126289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FAJ-0001ib-As; Wed, 15 May 2024 13:59:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722393.1126289; Wed, 15 May 2024 13:59:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FAJ-0001iU-7N; Wed, 15 May 2024 13:59:31 +0000
Received: by outflank-mailman (input) for mailman id 722393;
 Wed, 15 May 2024 13:59:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7FAI-0001gO-Kj
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:59:30 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 589b0c98-12c3-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 15:59:28 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-572e48f91e9so1946478a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:59:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179263d9sm867656566b.95.2024.05.15.06.59.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 06:59:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 589b0c98-12c3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715781568; x=1716386368; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OFQIoriKADdLf2q2ccnXgAC/FxAXTbJKGfE/7oPxiDI=;
        b=YPpPS9EaEUEX6hnghacYp40zPqL6SaneKuD9qIisXwT2/5r0nL6sg3vP3s+TGmyphq
         pSY/upWUrEGzFbh0AjUR0sSXrL24U+DNT2BILg09J9I0NKCc9bsntIVwM4TXUFyg9FYe
         odWo1ayPP65zZSkPs2P/sSyclBR8EClQQnFAXu/zzAzaaM5f3/es8xt8QIwRBg4O2moA
         lGgbKl6tjfZW6vLl2K+Ec/5mUYhi+VOK6qf+2apEIC7BKgFGp2z5+cNeXHdCoYkocQCH
         akRVB8CGTbvwtJ/08uYYozNAY5NM1Z1sQSB1aZKodU4tKelOFfviLjl/Eqq3lnJy+DJy
         D4Wg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715781568; x=1716386368;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OFQIoriKADdLf2q2ccnXgAC/FxAXTbJKGfE/7oPxiDI=;
        b=g+/jstCQNKmzkPZfR96Y7RWgInEzy0JRvKlLPKPTKwtueN/P0TuqRk8u93vQpNK70w
         4yrRK+L55G79ZHoo980SPU89xJfEzseBZNOjmh/zMQpiEDNjWDhDJHvLhOWhCPZdhzug
         N0Z7cdejIBXnCfZEFlclviiRTGqLzmZTy2Rl+TukNnFK/QctVwic4kIfOBPu64tvjG0k
         cjjvEYLFKKH4t7MKL4806OoEPF2YHD3TA7F9WWKD3GFdEmqAWfXOaGLt/jPafzT8eBWw
         eegvG/7GVsEW5SBz6Sjs5/ihtSIfK+j8u2+U3wpQd2d2wheFhVbechZouQQxRt1Y58AH
         x3Cg==
X-Forwarded-Encrypted: i=1; AJvYcCUPyII6wXqh+u6sSav6Looz7pAA7v84NevVmwKeUvADDWxR/+bupPQG+Jg14QdcT9VjzPPMdtO/o1hnkoaXwQV4t2D1CnlhJRiw/PpC/EU=
X-Gm-Message-State: AOJu0YwD6S+V5E/uzRe/5baTSD3Hh6Ok7NZK+eaeWmiJZcSpfj5pbxu1
	xGqi3FO+VbllOkCwb6vkxxOB8QNTr92DJKM6OFmE7eDjUUWGLsdgvgTWtmfn5A==
X-Google-Smtp-Source: AGHT+IFGtIs9iUISP5QhtW8k7pZuc5sctlyQjyTv+C90iyu5OGJ4e6aqq2HggArriRAEqsfQ1q2XZw==
X-Received: by 2002:a17:907:31c2:b0:a58:eba0:6716 with SMTP id a640c23a62f3a-a5a2d66a40bmr1400994366b.60.1715781568238;
        Wed, 15 May 2024 06:59:28 -0700 (PDT)
Message-ID: <34e1d27b-efc9-495e-b88a-2720997ed9da@suse.com>
Date: Wed, 15 May 2024 15:59:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-7-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -80,12 +80,29 @@ config HAS_PMAP
>  config HAS_SCHED_GRANULARITY
>  	bool
>  
> +config HAS_SECRET_HIDING
> +	bool
> +
>  config HAS_UBSAN
>  	bool
>  
>  config MEM_ACCESS_ALWAYS_ON
>  	bool
>  
> +config SECRET_HIDING
> +    bool "Secret hiding"
> +    depends on HAS_SECRET_HIDING
> +    help
> +		The directmap contains mapping for most of the RAM which makes domain
> +		memory easily accessible. While making the performance better, it also makes
> +		the hypervisor more vulnerable to speculation attacks.
> +
> +		Enabling this feature will allow the user to decide whether the memory
> +		is always mapped at boot or mapped only on demand (see the command line
> +		option "directmap").
> +
> +		If unsure, say N.
> +
>  config MEM_ACCESS
>  	def_bool MEM_ACCESS_ALWAYS_ON
>  	prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON

Surely there's a better place to add this new setting than between two
dependent options (MEM_ACCESS_ALWAYS_ON and MEM_ACCESS).

> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -167,6 +167,13 @@ extern unsigned long max_page;
>  extern unsigned long total_pages;
>  extern paddr_t mem_hotplug;
>  
> +extern bool opt_directmap;
> +
> +static inline bool has_directmap(void)
> +{
> +    return opt_directmap;
> +}

As indicated before, with the Kconfig setting off I think we want to
have an alternative

#define opt_directmap true

There's no need to impact generated code by needing to look at a "variable"
which is never going to change value.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 13:59:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 13:59:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722395.1126298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FAi-0002NT-Lm; Wed, 15 May 2024 13:59:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722395.1126298; Wed, 15 May 2024 13:59:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FAi-0002NM-JE; Wed, 15 May 2024 13:59:56 +0000
Received: by outflank-mailman (input) for mailman id 722395;
 Wed, 15 May 2024 13:59:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7FAh-000270-Jp
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 13:59:55 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 68668cfa-12c3-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 15:59:55 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f174e316eso8399808e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 06:59:55 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52399dcb951sm221027e87.268.2024.05.15.06.59.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 06:59:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68668cfa-12c3-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715781595; x=1716386395; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=O5sUAN4xsv0QPdY4uy80pJLa1Lng9pvUAPqZnicw+mI=;
        b=EWqqrV0Kggb9snwiVrpsZXiToKH7Jb00pGR8IVs7PhnfJ0TSbahjs2S9uUeRoSIopd
         W7iK53D/mng2xqARkEKeWYf7bhuhACftvCSrNRrLkJPZfmXqtT/lGeyVuhKzosbqz+1G
         NG+DZHiKdyLWc5LqdOPgr7trtXHFSe2jk11Y0OnVB3cpZAiFX/e+eXgoKcRDcFPDG2Sz
         XVCLuixe2re5brRyYvLkB8twmQiafyQd3R8vjMxzZO/fEnFriD14Nr9AjC71mWR6M7bP
         Z3hO4p3YlBHWFwxqk5ztj8m+DpvSkVt0mwdGzU901I31juG25TMtHtLDKurAk+/lozEk
         AkGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715781595; x=1716386395;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=O5sUAN4xsv0QPdY4uy80pJLa1Lng9pvUAPqZnicw+mI=;
        b=e+LeMZ3jVOhNXfH3lSziu3Hn9OvgpQiln82xTShPAh2qAV0LR6AZfsMfe5dV6BPlUb
         IVDZXjc8EAHK6EnmBg5mnNzcZ4awWggg1dcAf2OkRQL80B+8WUyiCRrnEt/eFXtGwsjm
         dkh9EKmLXOoIGPhaxF47ECIyNxdMy2csI8tIIBmQ4vxtoFwg60LfDlVZGQaSwSQO47d6
         2geRfZ9jeGFwtYoUKLJeTpWhM8CM3uSnSDZCg6Fe4hwA+mGa45nfJt1Gf5GqSphXBACS
         nnxOiPN0n4HySPgXR1xNqsXEaYXUQWb8Fagx+nkDmRHKhZdRuL9lTwRbe8ILcbdtpmDW
         gVuA==
X-Forwarded-Encrypted: i=1; AJvYcCUfmUpZKNlwYG/2bDrZYGOrtYS1aPVr5+sb6yw3/nQBllUu3HU2LurtPDBotjL1MUP2QYNssj2uk+iTlKUdnihnGmnYi6tb0MiINiDM8UQ=
X-Gm-Message-State: AOJu0YygitlQP+ifSbSzd5Hxs6BDN3Ui/sORMKN8U0iOE/u+rTOk31cT
	6nqrOMvy36XHJFJaV9Xecop+DQYZsyZebmKB0TPsCpEfYC4fhGew
X-Google-Smtp-Source: AGHT+IG81bAVXSpuO/CY87+IFx6HU5uWaNCUuEjB8fcZx1dazynChwJpD7rVU3H1XzbHLN8d2/PZmQ==
X-Received: by 2002:a05:6512:3e0e:b0:522:2edc:c858 with SMTP id 2adb3069b0e04-5224b73a706mr9054704e87.34.1715781594542;
        Wed, 15 May 2024 06:59:54 -0700 (PDT)
Message-ID: <47858beea78e165ac0675b696f78f0301cf107d3.camel@gmail.com>
Subject: Re: [PATCH v9 07/15] xen/riscv: introduce atomic.h
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
Date: Wed, 15 May 2024 15:59:53 +0200
In-Reply-To: <7a4e4b99-bedc-4742-9119-2bad9bf468de@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <1b8d0d3e665adc10222a9e07cedab7bcb40d88f2.1714988096.git.oleksii.kurochko@gmail.com>
	 <7a4e4b99-bedc-4742-9119-2bad9bf468de@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 11:49 +0200, Jan Beulich wrote:
> On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > Changes in V9:
> > =C2=A0- update the defintion of write_atomic macros:
> > =C2=A0=C2=A0 drop the return value as this macros isn't expeceted to re=
turn
> > something
> > =C2=A0=C2=A0 drop unnessary parentheses around p.
>=20
> Yet then what about add_sized()? With that also tidied
> Acked-by: Jan Beulich <jbeulich@suse.com>
Sure, parentheses around p in add_sized() should be dropped too.
I will update the patch in the next version of the patch series.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 15 14:03:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:03:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722404.1126309 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FDy-0004D0-4Y; Wed, 15 May 2024 14:03:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722404.1126309; Wed, 15 May 2024 14:03:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FDy-0004Ct-16; Wed, 15 May 2024 14:03:18 +0000
Received: by outflank-mailman (input) for mailman id 722404;
 Wed, 15 May 2024 14:03:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7FDw-0004BW-F8
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:03:16 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df202ac6-12c3-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:03:14 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so184729066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 07:03:14 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a50365669sm625656366b.193.2024.05.15.07.03.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 07:03:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df202ac6-12c3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715781794; x=1716386594; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ZpRkRANNIioplkw+kPkU6qK+ewThZFhCVvAQJd+Nbwc=;
        b=XTLQjGYK2m/D38oE1t6J17k43GEW0sPczbEFR7u16V2oXqm2TnmIv4ly1PyPjlpUdS
         1ZlnE7m2twN1ZlAVhq6FCCC0JG/GfwXCW/doXC9bZVHbsklH1dt6pzhZu8E6VFAAdtRi
         QMGuFgjI9qb7zeF625M1Bqg7VQJhfiN9qB0MlxgKVR/tIIMu5/pnCvctiBocQKaKRvEO
         Gan2NdidZU2QHpqXCQNM0/gxrUStSuBLfqLoXgKpdIqfDDtTJLDgRYzb426mVQY5VCe0
         iOCqYYucpjV+kjDYyUbLP9k9NugKfP7BooIMuPwKMWW/BlmyXpnYg33kXk2ZdsBqQXIf
         Qhqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715781794; x=1716386594;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ZpRkRANNIioplkw+kPkU6qK+ewThZFhCVvAQJd+Nbwc=;
        b=RV5y51InNsrlzlFsSR0AJMnTsGP5b60Z1M4rgxwcYB73/TPhKRNq/l8bqKWj+lycvd
         Np28OUOwuBeErvutFwnHmIha7Cfcfro24zMkzDvbZitkyeP/ae6Vdl4T4uHzHm89zbgi
         rALoGZza4B33PjI56iUCrAfOM1ZjD5+ZER6gDsvhRPI0VfVv2RgP88608qQWDuw4Nlqf
         W1Dl/n2dJr4TcGx8ydAf/qSuYjpnjkOUcRbMKXg7fX9aUDUVeQRjh0nrm/OujqWrpJfq
         UzakZYxqUSpGmu8504Pio9NRsXXRWc4aYb6VS53RG4hjdSKvI/XcWkHEu3XYSHWS52+L
         RLwg==
X-Forwarded-Encrypted: i=1; AJvYcCWG6TRpKsfQ7LeU9vH9RjJWd9vkG2Oj8+Ol9NPptkTQtYPUGnWqywF1zx03/Uh6Vhswt+5S0AFp8uZUqhd9pQLIyGjUq6xoVsoapetZPgs=
X-Gm-Message-State: AOJu0YyaN2e4KADk9EAChY/DzTABKSq/Wtv35cFF15mCmV7py3Zsk7CK
	MfTdRHecClEtaM5Us6gwkDJg9ML/xwDe5Jzu1b0Ka/LzwVSsDW6WBA+Z83KF+Q==
X-Google-Smtp-Source: AGHT+IEe+ZWUtjwwLA8Mi2hbyOOfcHtrqI6oHi42cwIUmPIEZQq7xxsB8kNSVytzggVNRp/vE5yPHA==
X-Received: by 2002:a17:906:75a:b0:a59:fca5:ccaa with SMTP id a640c23a62f3a-a5a2d53b020mr1008479966b.13.1715781793975;
        Wed, 15 May 2024 07:03:13 -0700 (PDT)
Message-ID: <6d10f5ea-991a-4070-9c4e-cf7c692d2759@suse.com>
Date: Wed, 15 May 2024 16:03:12 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 08/19] xen/x86: Add build assertion for fixmap
 entries
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-9-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-9-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> The early fixed addresses must all fit into the static L1 table.
> Introduce a build assertion to this end.
> 
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>      Changes in v2:
>          * New patch
> 
> diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h
> index a7ac365fc6..904bee0480 100644
> --- a/xen/arch/x86/include/asm/fixmap.h
> +++ b/xen/arch/x86/include/asm/fixmap.h
> @@ -77,6 +77,11 @@ enum fixed_addresses {
>  #define FIXADDR_SIZE  (__end_of_fixed_addresses << PAGE_SHIFT)
>  #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
>  
> +static inline void fixaddr_build_assertion(void)
> +{
> +    BUILD_BUG_ON(FIX_PMAP_END > L1_PAGETABLE_ENTRIES - 1);
> +}
> +
>  extern void __set_fixmap(
>      enum fixed_addresses idx, unsigned long mfn, unsigned long flags);
>  

In addition to earlier comments that were given - this looks like it wants to
be part of another patch (the previous one? a subsequent one?), rather than
being standalone. Such an check makes sense in connection with code actually
leveraging the assumption checked.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 14:07:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722408.1126319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FHw-0005BP-KB; Wed, 15 May 2024 14:07:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722408.1126319; Wed, 15 May 2024 14:07:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FHw-0005BH-GW; Wed, 15 May 2024 14:07:24 +0000
Received: by outflank-mailman (input) for mailman id 722408;
 Wed, 15 May 2024 14:07:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7FHu-0005B6-Vf
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:07:22 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7284826c-12c4-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 16:07:21 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-56e1baf0380so2051158a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 07:07:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0035sm9051531a12.50.2024.05.15.07.07.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 07:07:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7284826c-12c4-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715782041; x=1716386841; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=k0PfqxX7BVmMsAZzKFvyNkhPpeR8HETprZKZjx0qju0=;
        b=bILRp+3GTIFNODGJBNIcgQ+8zKoJIDNICSAQU9J5ZAU1uM6CWUKsuvkyX23zNtadm1
         eKHxO7PD1ZgSZCo4hyMR0OLDUNdKY/kaYk2N+9tYZbR7gVjvDuMtlbF0YIAhSsFSHUPA
         uGPE32a8sPwnjsbefM/oqsnO2a+OuFbAVaGx27ae+vkpNomKDt0HvHZCP+wX9PjIMR4q
         oc2dFzx79f74WgoGKraABuzxVcyo4WKEf0dUV7iTrk7k45iReYKwMjyztjes00gIYsjY
         8TWyYoJu0UwG1nS1jfDBrgBVJIVTbB/wK5aM0wfwTOdIQTdOfPf9q2716GRJ7fr/zcnd
         TTMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715782041; x=1716386841;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=k0PfqxX7BVmMsAZzKFvyNkhPpeR8HETprZKZjx0qju0=;
        b=ALRj8R3snc8/y5UY9qFQYpUeUHrsFHQrBmO7qrX5E++b/OI2kFYuwiB3JJJ1AleDa2
         crc/aGT6+q17TXY189R37TKmDxD8T68fi1FSNAURNqGYI4a9vbXIAoc2JPBkmPYIuPxh
         RnIMrTGO3Wn++KfXjA4Kc4+5RME8UYmmPJAIOW9g4jw5w7bhcRfDRzBgcQZbFZFiJ3CL
         Ppao6pVsWHoHogwLexddORCQ8vBxjVU3iZMGzQ/oZt3Cua5CXkRCyapOEZfoCIFPrUCt
         d7jUL+T0t4xVw5JnZZJGrN3KBbjhh/vwXENqg5spmY4M0s6ScJWqf/7r788By4ppPb5L
         IQfA==
X-Forwarded-Encrypted: i=1; AJvYcCUg7Es2P8PPXbw9+o1zbYFsOd8r0l214Jc12/O0UU/Cy/G/Jh8No/dI2zMxaegck6apexcOJFyy2dumdZRbL3T4uptZNW2BBa0pYGOv6nQ=
X-Gm-Message-State: AOJu0Ywv2aJCb0zXNjABKviOGftXOFaEyefWfYbv1Y9Z7ocynF2K90yQ
	9wR3uul1R2Kb4j120kUhBeiBEZdRR3ODVbJfIDV6wJnhxzmkiDi/Dl5Z+p26OA==
X-Google-Smtp-Source: AGHT+IGnMvc9sGONal+ESoQrP2J9BZXybaiKAnruUK+Z1n4UzwMOo+1hI8fMLgmDmd/Due0jcqJAeQ==
X-Received: by 2002:a50:d59d:0:b0:56d:c91c:dfc2 with SMTP id 4fb4d7f45d1cf-5734d67eca7mr11808269a12.31.1715782041212;
        Wed, 15 May 2024 07:07:21 -0700 (PDT)
Message-ID: <ea90fd21-1ec4-46b5-b157-810466b3f0ab@suse.com>
Date: Wed, 15 May 2024 16:07:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
 <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
 <46fd5c698fc13ffa3e8e82ed507bcfa67a9ef819.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <46fd5c698fc13ffa3e8e82ed507bcfa67a9ef819.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 15:55, Oleksii K. wrote:
> On Wed, 2024-05-15 at 11:09 +0200, Jan Beulich wrote:
>> On 06.05.2024 12:15, Oleksii Kurochko wrote:
>>> Changes in V9:
>>>  - update return type of fls and flsl() to unsigned int to be
>>> aligned with other
>>>    bit ops.
>>
>> But this then needs carrying through to ...
>>
>>> --- a/xen/arch/arm/include/asm/arm64/bitops.h
>>> +++ b/xen/arch/arm/include/asm/arm64/bitops.h
>>> @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long
>>> __ffs(unsigned long word)
>>>   */
>>>  #define ffz(x)  __ffs(~(x))
>>>  
>>> -static inline int flsl(unsigned long x)
>>> +static inline int arch_flsl(unsigned long x)
>>
>> ... e.g. here. You don't want to introduce signed/unsigned
>> mismatches.
> Do you mean that generic flsl() uses 'unsigned int' as a return type,
> but arch_flsl continue to use 'int'?

Yes.

>> Also why do you keep "inline" here, while ...
>>
>>> --- a/xen/arch/x86/include/asm/bitops.h
>>> +++ b/xen/arch/x86/include/asm/bitops.h
>>> @@ -425,7 +425,7 @@ static always_inline unsigned int
>>> arch_ffsl(unsigned long x)
>>>   *
>>>   * This is defined the same way as ffs.
>>>   */
>>> -static inline int flsl(unsigned long x)
>>> +static always_inline int arch_flsl(unsigned long x)
>>
>> ... you switch to always_inline here?
> Because Adnrew's patch with bitops.h for x86 changes to always_inline,
> so to be consistent, at least, for architecture.

And why not extend this to Arm?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 14:22:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:22:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722414.1126329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FW1-0000tE-Q6; Wed, 15 May 2024 14:21:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722414.1126329; Wed, 15 May 2024 14:21:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FW1-0000t7-NC; Wed, 15 May 2024 14:21:57 +0000
Received: by outflank-mailman (input) for mailman id 722414;
 Wed, 15 May 2024 14:21:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7FW1-0000t1-9l
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:21:57 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7aec620e-12c6-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:21:54 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso183487866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 07:21:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17892444sm860426766b.76.2024.05.15.07.21.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 07:21:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7aec620e-12c6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715782914; x=1716387714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VDjKHeVmEF8gDoTf7gHB2hCYAf1lLBmvUSTuuW7v6x8=;
        b=WURRCYYewgip3zSh1apRWAzjUXKalfmAW53mWmQ/sqpTMA4QnxL+GVK1yn3zxh7da+
         8t36Ht6M8eZCv1GvrlP3O7L2jZFbCs5trFLfL0Z+lUyJ6UNLApYUjI/FvNZnB1P5isWR
         3taKLG/MCMF01fJSllpG99iEL0GK3+E+kwuMpYT+XJQVo098AIx18OxEToKoBWbAY0XE
         2YO2K5pdql5HVLwFlGT/Pgx6sQZrctRKttPPMLWPlEuci2qjV4C2Bsox7piEycAygZVL
         Y90edCvtN59S08B3M3o0HCnC9gw/3A0zxwgYPYj0v/Vs31XXDKDzhjkZ2csOmXTuYo+l
         iRSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715782914; x=1716387714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VDjKHeVmEF8gDoTf7gHB2hCYAf1lLBmvUSTuuW7v6x8=;
        b=IEzYccoBO562QXSEIYXinllNQMUKfryVNpGvoNfx1PAndeW92/NkSkdW+9ZThmfhVm
         XKUZlHrq0oCQgNCsvsvQ3/JsPhalzgI7BQowNEBWW6a46mqEuW3qocWV1QAjJwdQWDM2
         u7bxMom2JMK4aHOGBKx1blRNRG8CDYXbQ5ETYX+c+ukKZdcxl3zCG6ZUpGjE8D78EK4i
         uNfubLmhzRQbteChzH5ERp7WNSrrhWEEzMd7esGar1OfRinwKUNXu+hkgZCWpb5n8Vhc
         x9jvwhbYsXxnpRAe13NMdfc2WA032klBTsquJraTJaZjoyfQfTSprWVCOpXgHoS84dOU
         zt5g==
X-Forwarded-Encrypted: i=1; AJvYcCU6x7UqJ5oHk+w1iQLestUCX2cr/+DSU5n2OP2QJ0aLjPZg7PnAeFymuf4QJgpe/TvhbL5noFvN8ppMVZMMsjXpK30GvBFeDy4301v7cos=
X-Gm-Message-State: AOJu0Yy3XxaZiNpaLi+DHzZCyJwT+df5e52Z2B4ysAKnpvXLuwmsiZrN
	6kiqHp3CLeTPUxobbV2TR+4fwJ7oXljP19CMO+AIo/N2IQOTO4KSzYzBsVJADw==
X-Google-Smtp-Source: AGHT+IFMjyS9z0y/NWW5GiglPbeEuz+v5V6NnYCb3H0G3lPOOGrs2jV2sZhGSQw7q8cwbaebyX/f5Q==
X-Received: by 2002:a17:907:1b0b:b0:a59:a3ef:21eb with SMTP id a640c23a62f3a-a5a2d675b19mr1303577666b.73.1715782914321;
        Wed, 15 May 2024 07:21:54 -0700 (PDT)
Message-ID: <4a47654c-54c4-4ada-8139-e5ca9d35af86@suse.com>
Date: Wed, 15 May 2024 16:21:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 09/19] x86/domain_page: Remove the fast paths
 when mfn is not in the directmap
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-10-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-10-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> @@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn)
>      struct vcpu_maphash_entry *hashent;
>  
>  #ifdef NDEBUG
> -    if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) )
> +    if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
>          return mfn_to_virt(mfn_x(mfn));
>  #endif
>  
>      v = mapcache_current_vcpu();
> -    if ( !v )
> -        return mfn_to_virt(mfn_x(mfn));
> +    if ( !v || !v->domain->arch.mapcache.inuse )
> +    {
> +        if ( arch_mfns_in_directmap(mfn_x(mfn), 1) )
> +            return mfn_to_virt(mfn_x(mfn));
> +        else
> +        {
> +            BUG_ON(system_state >= SYS_STATE_smp_boot);
> +            return pmap_map(mfn);
> +        }
> +    }
>  
>      dcache = &v->domain->arch.mapcache;
>      vcache = &v->arch.mapcache;
> -    if ( !dcache->inuse )
> -        return mfn_to_virt(mfn_x(mfn));

Is this case (the logic for which you move up) actually possible? I.e.
can we observe a domain here which hasn't made it through
mapcache_domain_init() (where ->inuse is set)?

> @@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr)
>      if ( !va || va >= DIRECTMAP_VIRT_START )
>          return;
>  
> +    if ( va >= FIXADDR_START && va < FIXADDR_TOP )
> +    {
> +        pmap_unmap((void *)ptr);

Misra is going to object to this casting away of const. It's rather
pmap_unmap() which wants changing, to accept a pointer-to-const.

> @@ -335,6 +350,23 @@ mfn_t domain_page_map_to_mfn(const void *ptr)
>      if ( va >= DIRECTMAP_VIRT_START )
>          return _mfn(virt_to_mfn(ptr));
>  
> +    /*
> +     * The fixmap is stealing the top-end of the VMAP. So the check for
> +     * the PMAP *must* happen first.

Not really. You could also ...

> +     * Also, the fixmap translate a slot to an address backwards. The
> +     * logic will rely on it to avoid any complexity. So check at
> +     * compile time this will always hold.
> +    */
> +    BUILD_BUG_ON(fix_to_virt(FIX_PMAP_BEGIN) < fix_to_virt(FIX_PMAP_END));
> +
> +    if ( ((unsigned long)fix_to_virt(FIX_PMAP_END) <= va) &&
> +         ((va & PAGE_MASK) <= (unsigned long)fix_to_virt(FIX_PMAP_BEGIN)) )
> +    {
> +        BUG_ON(system_state >= SYS_STATE_smp_boot);
> +        return l1e_get_mfn(l1_fixmap[l1_table_offset(va)]);
> +    }
> +
>      if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END )

... put it into the body of this if() then. Which might be preferable to
keep the non-global-mapping case straight / quick.

Jan

>          return vmap_to_mfn(va);
>  



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:22:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:22:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722415.1126339 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FWX-0001Lp-11; Wed, 15 May 2024 14:22:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722415.1126339; Wed, 15 May 2024 14:22:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FWW-0001Li-UZ; Wed, 15 May 2024 14:22:28 +0000
Received: by outflank-mailman (input) for mailman id 722415;
 Wed, 15 May 2024 14:22:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7FWV-0001LJ-Gs
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:22:27 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d61b016-12c6-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 16:22:25 +0200 (CEST)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2db17e8767cso98427971fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 07:22:25 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-351d21c8106sm284788f8f.60.2024.05.15.07.22.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 07:22:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d61b016-12c6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715782945; x=1716387745; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=RtwMB00+s5qMmn/kqwmbEpnVdTHjnFeYVb3ciF6v32M=;
        b=KoGHTQJ4f2lSUimq4VVDewbC3msBWD4RJeB8FosAKEXYd1rhGeuvcVtY+ZgMAJrGCR
         uRDE8UtIt8Vgy9lHzlYe4qnt3y+bhQMUEahNo/KH4wlMVO1+7mDF7Ps4voroAt0oo9uF
         X7JTQY5uKgQdHxMQCw1RHYOJVY7VM39uYckWA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715782945; x=1716387745;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RtwMB00+s5qMmn/kqwmbEpnVdTHjnFeYVb3ciF6v32M=;
        b=P7hf3JyiBjU2Lm2yo/f9llHLKXUYdZddo7fzSwrLg0/cD4hMfiZ0W+T5vopvPk6t7f
         Yx+DG012MI9+VHVciJg8foVxmrfgGP0aIEHcgZIf3uvokwXo6b7voS/oH68qd2DCRA4z
         r+56yvGkx1a45F3l6FPaT2XXoHLV38pi6uWzxBroSNC6bm9tX/Pc5pqHER8hwqKTPZtg
         QGIazyAph/ACtOh4FWK72Xzl+ZJBgyv8juJlrBFO4CP2EqOqxS3P+V9lyW/skzLRsBjQ
         d7awS5W1ePxMDAFRBqdCdFtg64Vqi8+hedmZFQ9nQHSqQRZgU0Vuj6MmdqAS30ws62No
         gxVw==
X-Gm-Message-State: AOJu0YxPjQp+LPdnekk7ULgYYrpV8NQLK3tfDlsF0DimSHbd37LJ8vND
	b8y7Ez1eZRi/SuSJ//RPus3BIlw1Wt/RVxMz+N/XHiZFjQH6C696/Xkl/0Q7ELw=
X-Google-Smtp-Source: AGHT+IGdRvyP7NWoa+ddjY6+WWxZOOgtzddKaEM9rI1w6GZ05qB3qUuyMqLILy1LLJNpKll5gBynow==
X-Received: by 2002:a05:651c:1542:b0:2de:7046:b8f8 with SMTP id 38308e7fff4ca-2e51fd2e6a8mr162451611fa.5.1715782943154;
        Wed, 15 May 2024 07:22:23 -0700 (PDT)
Message-ID: <4ddc5acc-fc7b-4ed6-8e63-412f34da00b2@citrix.com>
Date: Wed, 15 May 2024 15:22:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen crash in scheduler during cpu hotplug
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>
References: <41ae17df-fb06-40b3-a98a-02f6f495f169@citrix.com>
 <7553291c-f88d-49c7-a1bd-5f8dbd21fd2b@suse.com>
 <9fdbf8a1-56a6-4be3-96a0-fcded0382cf0@citrix.com>
 <6766f9f1-2ceb-42d8-8ddd-f98671106579@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <6766f9f1-2ceb-42d8-8ddd-f98671106579@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 2:38 pm, Jürgen Groß wrote:
> On 15.05.24 15:22, Andrew Cooper wrote:
>> On 15/05/2024 1:39 pm, Jan Beulich wrote:
>>> On 15.05.2024 13:58, Andrew Cooper wrote:
>>>> Just so it doesn't get lost.  In XenRT, we've found:
>>>>
>>>>> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
>>>>> (XEN) CPU:    45
>>>>> (XEN) RIP:    e008:[<ffff82d040244cbf>]
>>>>> common/sched/credit.c#csched_load_balance+0x41/0x877
>>>>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>>>>> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
>>>>> 0000000000000000
>>>>> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
>>>>> ffff83103723d450
>>>>> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8:
>>>>> 0000000000000000
>>>>> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
>>>>> 0000000000000009
>>>>> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
>>>>> ffff831037253720
>>>>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
>>>>> 0000000000f526e0
>>>>> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
>>>>> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
>>>>> 0000000000000000
>>>>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>>>>> (XEN) Xen code around <ffff82d040244cbf>
>>>>> (common/sched/credit.c#csched_load_balance+0x41/0x877):
>>>>> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
>>>>> 8b 4e 28 48
>>>>> <snip>
>>>>> (XEN) Xen call trace:
>>>>> (XEN)    [<ffff82d040244cbf>] R
>>>>> common/sched/credit.c#csched_load_balance+0x41/0x877
>>> While this is of course pretty little information, I've still tried to
>>> decipher it, first noticing it's credit1 that's being used here. Once
>>> forcing csched_load_balance() non-inline (no idea why it is a separate
>>> function in your build), I can see a sufficiently matching pattern at
>>> approximately the same offset into the function. That's
>>>
>>>      const struct cpupool *c = get_sched_res(cpu)->cpupool;
>>>      ...
>>>      const cpumask_t *online = c->res_valid;
>>>      ...
>>>      BUG_ON(get_sched_res(cpu) != snext->unit->res);
>>>
>>> overlapping, with the crash being on the middle of the quoted lines.
>>> IOW the CPU pool is still NULL for this sched resource. Cc-ing
>>> Jürgen for possible clues ...
>>
>> We've seen it in 4.13, 4.17 and upstream, after Roger extended the
>> existing CPU hotplug testing to try and reproduce the MTRR watchdog
>> failure.  We've found yet another "no irq for handler" from this too.
>>
>> It's always a deference at NULL+0x10, somewhere within
>> csched_schedule().
>
> I think I've found the reason.
>
> In schedule_cpu_add() the cpupool and granularity are set only after
> releasing the scheduling lock. I think those must be inside the locked
> region.
>
> Can you give this one a try (not tested at all)?
>
> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
> index 0cb33831d2..babac7aad6 100644
> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -3176,6 +3176,8 @@ int schedule_cpu_add(unsigned int cpu, struct
> cpupool *c)
>
>      sr->scheduler = new_ops;
>      sr->sched_priv = ppriv;
> +    sr->granularity = cpupool_get_granularity(c);
> +    sr->cpupool = c;
>
>      /*
>       * Reroute the lock to the per pCPU lock as /last/ thing. In fact,
> @@ -3188,8 +3190,6 @@ int schedule_cpu_add(unsigned int cpu, struct
> cpupool *c)
>      /* _Not_ pcpu_schedule_unlock(): schedule_lock has changed! */
>      spin_unlock_irqrestore(old_lock, flags);
>
> -    sr->granularity = cpupool_get_granularity(c);
> -    sr->cpupool = c;
>      /* The  cpu is added to a pool, trigger it to go pick up some
> work */
>      cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ);

This change seems to be standing up to the test, in a way that the
previous form very much didn't.

Thanks for the quick fix.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722423.1126359 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Faa-0002Jz-PB; Wed, 15 May 2024 14:26:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722423.1126359; Wed, 15 May 2024 14:26:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Faa-0002Js-MD; Wed, 15 May 2024 14:26:40 +0000
Received: by outflank-mailman (input) for mailman id 722423;
 Wed, 15 May 2024 14:26:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7FaZ-00025L-2u
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:39 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 239c571f-12c7-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 16:26:37 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F10A214BF;
 Wed, 15 May 2024 07:27:01 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 849F63F7A6;
 Wed, 15 May 2024 07:26:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 239c571f-12c7-11ef-909d-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Date: Wed, 15 May 2024 15:26:19 +0100
Message-Id: <20240515142626.3480640-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The current static shared memory code is using bootinfo banks when it
needs to find the number of borrowers, so every time assign_shared_memory
is called, the bank is searched in the bootinfo.shmem structure.

There is nothing wrong with it, however the bank can be used also to
retrieve the start address and size and also to pass less argument to
assign_shared_memory. When retrieving the information from the bootinfo
bank, it's also possible to move the checks on alignment to
process_shm_node in the early stages.

So create a new function find_shm_bank_by_id() which takes a
'struct shared_meminfo' structure and the shared memory ID, to look for a
bank with a matching ID, take the physical host address and size from the
bank, pass the bank to assign_shared_memory() removing the now unnecessary
arguments and finally remove the acquire_nr_borrower_domain() function
since now the information can be extracted from the passed bank.
Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
case of errors (unlikely), as said above, move the checks on alignment
to process_shm_node.

Drawback of this change is that now the bootinfo are used also when the
bank doesn't need to be allocated, however it will be convinient later
to use it as an argument for assign_shared_memory when dealing with
the use case where the Host physical address is not supplied by the user.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2 changes:
 - fix typo commit msg, renamed find_shm() to find_shm_bank_by_id(),
   swap region size check different from zero and size alignment, remove
   not necessary BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 101 +++++++++++++++++++-----------------
 1 file changed, 54 insertions(+), 47 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 78881dd1d3f7..0afc86c43f85 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -19,29 +19,22 @@ static void __init __maybe_unused build_assertions(void)
                  offsetof(struct shared_meminfo, bank)));
 }
 
-static int __init acquire_nr_borrower_domain(struct domain *d,
-                                             paddr_t pbase, paddr_t psize,
-                                             unsigned long *nr_borrowers)
+static const struct membank __init *
+find_shm_bank_by_id(const struct membanks *shmem, const char *shm_id)
 {
-    const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
 
-    /* Iterate reserved memory to find requested shm bank. */
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
     {
-        paddr_t bank_start = shmem->bank[bank].start;
-        paddr_t bank_size = shmem->bank[bank].size;
-
-        if ( (pbase == bank_start) && (psize == bank_size) )
+        if ( strncmp(shm_id, shmem->bank[bank].shmem_extra->shm_id,
+                     MAX_SHM_ID_LENGTH) == 0 )
             break;
     }
 
     if ( bank == shmem->nr_banks )
-        return -ENOENT;
+        return NULL;
 
-    *nr_borrowers = shmem->bank[bank].shmem_extra->nr_shm_borrowers;
-
-    return 0;
+    return &shmem->bank[bank];
 }
 
 /*
@@ -103,14 +96,18 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     return smfn;
 }
 
-static int __init assign_shared_memory(struct domain *d,
-                                       paddr_t pbase, paddr_t psize,
-                                       paddr_t gbase)
+static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       const struct membank *shm_bank)
 {
     mfn_t smfn;
     int ret = 0;
     unsigned long nr_pages, nr_borrowers, i;
     struct page_info *page;
+    paddr_t pbase, psize;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+    nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
     printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
            d, pbase, pbase + psize);
@@ -135,14 +132,6 @@ static int __init assign_shared_memory(struct domain *d,
         }
     }
 
-    /*
-     * Get the right amount of references per page, which is the number of
-     * borrower domains.
-     */
-    ret = acquire_nr_borrower_domain(d, pbase, psize, &nr_borrowers);
-    if ( ret )
-        return ret;
-
     /*
      * Instead of letting borrower domain get a page ref, we add as many
      * additional reference as the number of borrowers when the owner
@@ -199,6 +188,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
 
     dt_for_each_child_node(node, shm_node)
     {
+        const struct membank *boot_shm_bank;
         const struct dt_property *prop;
         const __be32 *cells;
         uint32_t addr_cells, size_cells;
@@ -212,6 +202,23 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
 
+        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
+        {
+            printk("%pd: invalid \"xen,shm-id\" property", d);
+            return -EINVAL;
+        }
+        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
+
+        boot_shm_bank = find_shm_bank_by_id(bootinfo_get_shmem(), shm_id);
+        if ( !boot_shm_bank )
+        {
+            printk("%pd: static shared memory bank not found: '%s'", d, shm_id);
+            return -ENOENT;
+        }
+
+        pbase = boot_shm_bank->start;
+        psize = boot_shm_bank->size;
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -221,20 +228,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
-        psize = dt_read_paddr(cells, size_cells);
-        if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
-        {
-            printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
-                   d, pbase, gbase);
-            return -EINVAL;
-        }
-        if ( !IS_ALIGNED(psize, PAGE_SIZE) )
-        {
-            printk("%pd: size 0x%"PRIpaddr" is not suitably aligned\n",
-                   d, psize);
-            return -EINVAL;
-        }
+        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
         for ( i = 0; i < PFN_DOWN(psize); i++ )
             if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
@@ -251,13 +245,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
             owner_dom_io = false;
 
-        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
-        {
-            printk("%pd: invalid \"xen,shm-id\" property", d);
-            return -EINVAL;
-        }
-        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
-
         /*
          * DOMID_IO is a fake domain and is not described in the Device-Tree.
          * Therefore when the owner of the shared region is DOMID_IO, we will
@@ -270,8 +257,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
              * We found the first borrower of the region, the owner was not
              * specified, so they should be assigned to dom_io.
              */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d,
-                                       pbase, psize, gbase);
+            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                       boot_shm_bank);
             if ( ret )
                 return ret;
         }
@@ -439,12 +426,32 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
     size = dt_next_cell(size_cells, &cell);
 
+    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+    {
+        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+               paddr);
+        return -EINVAL;
+    }
+
+    if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
+    {
+        printk("fdt: guest address 0x%"PRIpaddr" is not suitably aligned.\n",
+               gaddr);
+        return -EINVAL;
+    }
+
     if ( !size )
     {
         printk("fdt: the size for static shared memory region can not be zero\n");
         return -EINVAL;
     }
 
+    if ( !IS_ALIGNED(size, PAGE_SIZE) )
+    {
+        printk("fdt: size 0x%"PRIpaddr" is not suitably aligned\n", size);
+        return -EINVAL;
+    }
+
     end = paddr + size;
     if ( end <= paddr )
     {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722422.1126349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FaZ-00025d-Jt; Wed, 15 May 2024 14:26:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722422.1126349; Wed, 15 May 2024 14:26:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FaZ-00025W-GJ; Wed, 15 May 2024 14:26:39 +0000
Received: by outflank-mailman (input) for mailman id 722422;
 Wed, 15 May 2024 14:26:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7FaY-00025L-DH
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:38 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 22ff3bf6-12c7-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 16:26:37 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DDB501042;
 Wed, 15 May 2024 07:27:00 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7114E3F7A6;
 Wed, 15 May 2024 07:26:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22ff3bf6-12c7-11ef-909d-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 0/7] Static shared memory followup v2 - pt2
Date: Wed, 15 May 2024 15:26:18 +0100
Message-Id: <20240515142626.3480640-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This serie is a partial rework of this other serie:
https://patchwork.kernel.org/project/xen-devel/cover/20231206090623.1932275-1-Penny.Zheng@arm.com/

The original serie is addressing an issue of the static shared memory feature
that impacts the memory footprint of other component when the feature is
enabled, another issue impacts the device tree generation for the guests when
the feature is enabled and used and the last one is a missing feature that is
the option to have a static shared memory region that is not from the host
address space.

This serie is handling some comment on the original serie and it is splitting
the rework in two part, this first part is addressing the memory footprint issue
and the device tree generation and currently is fully merged
(https://patchwork.kernel.org/project/xen-devel/cover/20240418073652.3622828-1-luca.fancellu@arm.com/),
this serie is addressing the static shared memory allocation from the Xen heap.

Luca Fancellu (5):
  xen/arm: Lookup bootinfo shm bank during the mapping
  xen/arm: Wrap shared memory mapping code in one function
  xen/arm: Parse xen,shared-mem when host phys address is not provided
  xen/arm: Rework heap page allocation outside allocate_bank_memory
  xen/arm: Implement the logic for static shared memory from Xen heap

Penny Zheng (2):
  xen/p2m: put reference for level 2 superpage
  xen/docs: Describe static shared memory when host address is not
    provided

 docs/misc/arm/device-tree/booting.txt   |  52 ++-
 xen/arch/arm/arm32/mmu/mm.c             |  11 +-
 xen/arch/arm/dom0less-build.c           |   4 +-
 xen/arch/arm/domain_build.c             |  84 +++--
 xen/arch/arm/include/asm/domain_build.h |   9 +-
 xen/arch/arm/mmu/p2m.c                  |  57 +++-
 xen/arch/arm/setup.c                    |  14 +-
 xen/arch/arm/static-shmem.c             | 435 +++++++++++++++++-------
 8 files changed, 482 insertions(+), 184 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722424.1126369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fad-0002aA-1b; Wed, 15 May 2024 14:26:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722424.1126369; Wed, 15 May 2024 14:26:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fac-0002a3-TH; Wed, 15 May 2024 14:26:42 +0000
Received: by outflank-mailman (input) for mailman id 722424;
 Wed, 15 May 2024 14:26:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Fab-0002KG-3F
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:41 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 2438264b-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:38 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0FEB81516;
 Wed, 15 May 2024 07:27:03 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 97A8B3F7A6;
 Wed, 15 May 2024 07:26:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2438264b-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 2/7] xen/arm: Wrap shared memory mapping code in one function
Date: Wed, 15 May 2024 15:26:20 +0100
Message-Id: <20240515142626.3480640-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Wrap the code and logic that is calling assign_shared_memory
and map_regions_p2mt into a new function 'handle_shared_mem_bank',
it will become useful later when the code will allow the user to
don't pass the host physical address.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2 changes:
 - add blank line, move owner_dom_io computation inside
   handle_shared_mem_bank in order to reduce args count, remove
   not needed BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 87 ++++++++++++++++++++++---------------
 1 file changed, 53 insertions(+), 34 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 0afc86c43f85..8a14d120690c 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -181,6 +181,53 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
     return 0;
 }
 
+static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
+                                         const char *role_str,
+                                         const struct membank *shm_bank)
+{
+    bool owner_dom_io = true;
+    paddr_t pbase, psize;
+    int ret;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+
+    /*
+     * "role" property is optional and if it is defined explicitly,
+     * then the owner domain is not the default "dom_io" domain.
+     */
+    if ( role_str != NULL )
+        owner_dom_io = false;
+
+    /*
+     * DOMID_IO is a fake domain and is not described in the Device-Tree.
+     * Therefore when the owner of the shared region is DOMID_IO, we will
+     * only find the borrowers.
+     */
+    if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
+         (!owner_dom_io && strcmp(role_str, "owner") == 0) )
+    {
+        /*
+         * We found the first borrower of the region, the owner was not
+         * specified, so they should be assigned to dom_io.
+         */
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        if ( ret )
+            return ret;
+    }
+
+    if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
+    {
+        /* Set up P2M foreign mapping for borrower domain. */
+        ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
+                               _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
+        if ( ret )
+            return ret;
+    }
+
+    return 0;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -195,9 +242,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         paddr_t gbase, pbase, psize;
         int ret = 0;
         unsigned int i;
-        const char *role_str;
+        const char *role_str = NULL;
         const char *shm_id;
-        bool owner_dom_io = true;
 
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
@@ -238,39 +284,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                 return -EINVAL;
             }
 
-        /*
-         * "role" property is optional and if it is defined explicitly,
-         * then the owner domain is not the default "dom_io" domain.
-         */
-        if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
-            owner_dom_io = false;
+        /* "role" property is optional */
+        dt_property_read_string(shm_node, "role", &role_str);
 
-        /*
-         * DOMID_IO is a fake domain and is not described in the Device-Tree.
-         * Therefore when the owner of the shared region is DOMID_IO, we will
-         * only find the borrowers.
-         */
-        if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
-             (!owner_dom_io && strcmp(role_str, "owner") == 0) )
-        {
-            /*
-             * We found the first borrower of the region, the owner was not
-             * specified, so they should be assigned to dom_io.
-             */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
-                                       boot_shm_bank);
-            if ( ret )
-                return ret;
-        }
-
-        if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
-        {
-            /* Set up P2M foreign mapping for borrower domain. */
-            ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
-                                   _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
-            if ( ret )
-                return ret;
-        }
+        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
+        if ( ret )
+            return ret;
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722425.1126375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fad-0002dr-CP; Wed, 15 May 2024 14:26:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722425.1126375; Wed, 15 May 2024 14:26:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fad-0002cl-5R; Wed, 15 May 2024 14:26:43 +0000
Received: by outflank-mailman (input) for mailman id 722425;
 Wed, 15 May 2024 14:26:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Fac-0002KG-3E
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:42 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 2504f786-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:40 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5782715A1;
 Wed, 15 May 2024 07:27:04 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AB4BE3F7A6;
 Wed, 15 May 2024 07:26:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2504f786-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v2 3/7] xen/p2m: put reference for level 2 superpage
Date: Wed, 15 May 2024 15:26:21 +0100
Message-Id: <20240515142626.3480640-4-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

We are doing foreign memory mapping for static shared memory, and
there is a great possibility that it could be super mapped.
But today, p2m_put_l3_page could not handle superpages.

This commits implements a new function p2m_put_l2_superpage to handle
2MB superpages, specifically for helping put extra references for
foreign superpages.

Modify relinquish_p2m_mapping as well to take into account preemption
when type is foreign memory and order is above 9 (2MB).

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2:
 - Do not handle 1GB super page as there might be some issue where
   a lot of calls to put_page(...) might be issued which could lead
   to free memory that is a long operation.
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
---
 xen/arch/arm/mmu/p2m.c | 57 +++++++++++++++++++++++++++++-------------
 1 file changed, 40 insertions(+), 17 deletions(-)

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 41fcca011cf4..29fdb3b87dd0 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
     return rc;
 }
 
-/*
- * Put any references on the single 4K page referenced by pte.
- * TODO: Handle superpages, for now we only take special references for leaf
- * pages (specifically foreign ones, which can't be super mapped today).
- */
-static void p2m_put_l3_page(const lpae_t pte)
+/* Put any references on the single 4K page referenced by mfn. */
+static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
 {
-    mfn_t mfn = lpae_get_mfn(pte);
-
-    ASSERT(p2m_is_valid(pte));
-
     /*
      * TODO: Handle other p2m types
      *
@@ -771,16 +763,44 @@ static void p2m_put_l3_page(const lpae_t pte)
      * flush the TLBs if the page is reallocated before the end of
      * this loop.
      */
-    if ( p2m_is_foreign(pte.p2m.type) )
+    if ( p2m_is_foreign(type) )
     {
         ASSERT(mfn_valid(mfn));
         put_page(mfn_to_page(mfn));
     }
     /* Detect the xenheap page and mark the stored GFN as invalid. */
-    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
+    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
         page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
 }
 
+/* Put any references on the superpage referenced by mfn. */
+static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
+{
+    unsigned int i;
+    unsigned int level_order = XEN_PT_LEVEL_ORDER(3);
+
+    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
+    {
+        p2m_put_l3_page(mfn, type);
+
+        mfn = mfn_add(mfn, 1 << level_order);
+    }
+}
+
+/* Put any references on the page referenced by pte. */
+static void p2m_put_page(const lpae_t pte, unsigned int level)
+{
+    mfn_t mfn = lpae_get_mfn(pte);
+
+    ASSERT(p2m_is_valid(pte));
+
+    /* We have a second level 2M superpage */
+    if ( p2m_is_superpage(pte, level) && (level == 2) )
+        return p2m_put_l2_superpage(mfn, pte.p2m.type);
+    else if ( level == 3 )
+        return p2m_put_l3_page(mfn, pte.p2m.type);
+}
+
 /* Free lpae sub-tree behind an entry */
 static void p2m_free_entry(struct p2m_domain *p2m,
                            lpae_t entry, unsigned int level)
@@ -809,9 +829,10 @@ static void p2m_free_entry(struct p2m_domain *p2m,
 #endif
 
         p2m->stats.mappings[level]--;
-        /* Nothing to do if the entry is a super-page. */
-        if ( level == 3 )
-            p2m_put_l3_page(entry);
+        /* TODO: Currently we don't handle 1GB super-page. */
+        if ( level >= 2 )
+            p2m_put_page(entry, level);
+
         return;
     }
 
@@ -1558,9 +1579,11 @@ int relinquish_p2m_mapping(struct domain *d)
 
         count++;
         /*
-         * Arbitrarily preempt every 512 iterations.
+         * Arbitrarily preempt every 512 iterations or when type is foreign
+         * mapping and the order is above 9 (2MB).
          */
-        if ( !(count % 512) && hypercall_preempt_check() )
+        if ( (!(count % 512) || (p2m_is_foreign(t) && (order > 9))) &&
+             hypercall_preempt_check() )
         {
             rc = -ERESTART;
             break;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722426.1126389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fae-00033y-Jv; Wed, 15 May 2024 14:26:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722426.1126389; Wed, 15 May 2024 14:26:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fae-000334-Fs; Wed, 15 May 2024 14:26:44 +0000
Received: by outflank-mailman (input) for mailman id 722426;
 Wed, 15 May 2024 14:26:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Fad-0002KG-3T
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:43 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 259d49db-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:41 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 853561042;
 Wed, 15 May 2024 07:27:05 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F2BAA3F7A6;
 Wed, 15 May 2024 07:26:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 259d49db-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 4/7] xen/arm: Parse xen,shared-mem when host phys address is not provided
Date: Wed, 15 May 2024 15:26:22 +0100
Message-Id: <20240515142626.3480640-5-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Handle the parsing of the 'xen,shared-mem' property when the host physical
address is not provided, this commit is introducing the logic to parse it,
but the functionality is still not implemented and will be part of future
commits.

Rework the logic inside process_shm_node to check the shm_id before doing
the other checks, because it ease the logic itself, add more comment on
the logic.
Now when the host physical address is not provided, the value
INVALID_PADDR is chosen to signal this condition and it is stored as
start of the bank, due to that change also early_print_info_shmem and
init_sharedmem_pages are changed, to don't handle banks with start equal
to INVALID_PADDR.

Another change is done inside meminfo_overlap_check, to skip banks that
are starting with the start address INVALID_PADDR, that function is used
to check banks from reserved memory, shared memory and ACPI and since
the comment above the function states that wrapping around is not handled,
it's unlikely for these bank to have the start address as INVALID_PADDR.
Same change is done inside consider_modules, find_unallocated_memory and
dt_unreserved_regions functions, in order to skip banks that starts with
INVALID_PADDR from any computation.
The changes above holds because of this consideration.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2 changes:
 - fix comments, add parenthesis to some conditions, remove unneeded
   variables, remove else branch, increment counter in the for loop,
   skip INVALID_PADDR start banks from also consider_modules,
   find_unallocated_memory and dt_unreserved_regions. (Michal)
---
 xen/arch/arm/arm32/mmu/mm.c |  11 +++-
 xen/arch/arm/domain_build.c |   5 ++
 xen/arch/arm/setup.c        |  14 +++-
 xen/arch/arm/static-shmem.c | 125 +++++++++++++++++++++++++-----------
 4 files changed, 111 insertions(+), 44 deletions(-)

diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c
index 23150122f7c4..3d46f32831bf 100644
--- a/xen/arch/arm/arm32/mmu/mm.c
+++ b/xen/arch/arm/arm32/mmu/mm.c
@@ -128,8 +128,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
     nr += reserved_mem->nr_banks;
     for ( ; i - nr < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - nr].start;
-        paddr_t r_e = r_s + shmem->bank[i - nr].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - nr].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - nr].size;
 
         if ( s < r_e && r_s < e )
         {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index f6550809cfdf..4febba69ef63 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -926,6 +926,11 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
         for ( j = 0; j < mem_banks[i]->nr_banks; j++ )
         {
             start = mem_banks[i]->bank[j].start;
+
+            /* Shared memory banks can contain INVALID_PADDR as start */
+            if ( INVALID_PADDR == start )
+                continue;
+
             end = mem_banks[i]->bank[j].start + mem_banks[i]->bank[j].size;
             res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
                                         PFN_DOWN(end - 1));
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d242674381d4..016859f6982a 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -265,8 +265,15 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
     nr += reserved_mem->nr_banks;
     for ( ; i - nr < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - nr].start;
-        paddr_t r_e = r_s + shmem->bank[i - nr].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - nr].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - nr].size;
 
         if ( s < r_e && r_s < e )
         {
@@ -297,7 +304,8 @@ static bool __init meminfo_overlap_check(const struct membanks *mem,
         bank_start = mem->bank[i].start;
         bank_end = bank_start + mem->bank[i].size;
 
-        if ( region_end <= bank_start || region_start >= bank_end )
+        if ( INVALID_PADDR == bank_start || region_end <= bank_start ||
+             region_start >= bank_end )
             continue;
         else
         {
diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 8a14d120690c..ddaacbc77740 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -265,6 +265,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
+        if ( INVALID_PADDR == pbase )
+        {
+            printk("%pd: host physical address must be chosen by users at the moment", d);
+            return -EINVAL;
+        }
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -377,7 +383,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
 {
     const struct fdt_property *prop, *prop_id, *prop_role;
     const __be32 *cell;
-    paddr_t paddr, gaddr, size, end;
+    paddr_t paddr = INVALID_PADDR;
+    paddr_t gaddr, size, end;
     struct membanks *mem = bootinfo_get_shmem();
     struct shmem_membank_extra *shmem_extra = bootinfo_get_shmem_extra();
     unsigned int i;
@@ -432,24 +439,37 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     if ( !prop )
         return -ENOENT;
 
+    cell = (const __be32 *)prop->data;
     if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) )
     {
-        if ( len == dt_cells_to_size(size_cells + address_cells) )
-            printk("fdt: host physical address must be chosen by users at the moment.\n");
-
-        printk("fdt: invalid `xen,shared-mem` property.\n");
-        return -EINVAL;
+        if ( len == dt_cells_to_size(address_cells + size_cells) )
+            device_tree_get_reg(&cell, address_cells, size_cells, &gaddr,
+                                &size);
+        else
+        {
+            printk("fdt: invalid `xen,shared-mem` property.\n");
+            return -EINVAL;
+        }
     }
+    else
+    {
+        device_tree_get_reg(&cell, address_cells, address_cells, &paddr,
+                            &gaddr);
+        size = dt_next_cell(size_cells, &cell);
 
-    cell = (const __be32 *)prop->data;
-    device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
-    size = dt_next_cell(size_cells, &cell);
+        if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+        {
+            printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+                paddr);
+            return -EINVAL;
+        }
 
-    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
-    {
-        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
-               paddr);
-        return -EINVAL;
+        end = paddr + size;
+        if ( end <= paddr )
+        {
+            printk("fdt: static shared memory region %s overflow\n", shm_id);
+            return -EINVAL;
+        }
     }
 
     if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
@@ -471,39 +491,64 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
         return -EINVAL;
     }
 
-    end = paddr + size;
-    if ( end <= paddr )
-    {
-        printk("fdt: static shared memory region %s overflow\n", shm_id);
-        return -EINVAL;
-    }
-
     for ( i = 0; i < mem->nr_banks; i++ )
     {
         /*
          * Meet the following check:
-         * 1) The shm ID matches and the region exactly match
-         * 2) The shm ID doesn't match and the region doesn't overlap
-         * with an existing one
+         * - when host address is provided:
+         *   1) The shm ID matches and the region exactly match
+         *   2) The shm ID doesn't match and the region doesn't overlap
+         *      with an existing one
+         * - when host address is not provided:
+         *   1) The shm ID matches and the region size exactly match
          */
-        if ( paddr == mem->bank[i].start && size == mem->bank[i].size )
+        bool paddr_assigned = (INVALID_PADDR == paddr);
+
+        if ( strncmp(shm_id, shmem_extra[i].shm_id, MAX_SHM_ID_LENGTH) == 0 )
         {
-            if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                         MAX_SHM_ID_LENGTH) == 0  )
+            /*
+             * Regions have same shm_id (cases):
+             * 1) physical host address is supplied:
+             *    - OK:   paddr is equal and size is equal (same region)
+             *    - Fail: paddr doesn't match or size doesn't match (there
+             *            cannot exists two shmem regions with same shm_id)
+             * 2) physical host address is NOT supplied:
+             *    - OK:   size is equal (same region)
+             *    - Fail: size is not equal (same shm_id must identify only one
+             *            region, there can't be two different regions with same
+             *            shm_id)
+             */
+            bool start_match = paddr_assigned ? (paddr == mem->bank[i].start) :
+                                                true;
+
+            if ( start_match && (size == mem->bank[i].size) )
                 break;
             else
             {
-                printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n",
+                printk("fdt: different shared memory region could not share the same shm ID %s\n",
                        shm_id);
                 return -EINVAL;
             }
         }
-        else if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                          MAX_SHM_ID_LENGTH) != 0 )
+
+        /*
+         * Regions have different shm_id (cases):
+         * 1) physical host address is supplied:
+         *    - OK:   paddr different, or size different (case where paddr
+         *            is equal but psize is different are wrong, but they
+         *            are handled later when checking for overlapping)
+         *    - Fail: paddr equal and size equal (the same region can't be
+         *            identified with different shm_id)
+         * 2) physical host address is NOT supplied:
+         *    - OK:   Both have different shm_id so even with same size they
+         *            can exists
+         */
+        if ( !paddr_assigned || (paddr != mem->bank[i].start) ||
+             (size != mem->bank[i].size) )
             continue;
         else
         {
-            printk("fdt: different shared memory region could not share the same shm ID %s\n",
+            printk("fdt: xen,shm-id %s does not match for all the nodes using the same region\n",
                    shm_id);
             return -EINVAL;
         }
@@ -513,7 +558,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     {
         if (i < mem->max_banks)
         {
-            if ( check_reserved_regions_overlap(paddr, size) )
+            if ( (paddr != INVALID_PADDR) &&
+                 check_reserved_regions_overlap(paddr, size) )
                 return -EINVAL;
 
             /* Static shared memory shall be reserved from any other use. */
@@ -583,13 +629,13 @@ void __init early_print_info_shmem(void)
 {
     const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
+    unsigned int printed = 0;
 
-    for ( bank = 0; bank < shmem->nr_banks; bank++ )
-    {
-        printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", bank,
-               shmem->bank[bank].start,
-               shmem->bank[bank].start + shmem->bank[bank].size - 1);
-    }
+    for ( bank = 0; bank < shmem->nr_banks; bank++, printed++ )
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", printed,
+                shmem->bank[bank].start,
+                shmem->bank[bank].start + shmem->bank[bank].size - 1);
 }
 
 void __init init_sharedmem_pages(void)
@@ -598,7 +644,8 @@ void __init init_sharedmem_pages(void)
     unsigned int bank;
 
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
-        init_staticmem_bank(&shmem->bank[bank]);
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            init_staticmem_bank(&shmem->bank[bank]);
 }
 
 int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722427.1126399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fag-0003Nf-WB; Wed, 15 May 2024 14:26:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722427.1126399; Wed, 15 May 2024 14:26:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fag-0003NW-SS; Wed, 15 May 2024 14:26:46 +0000
Received: by outflank-mailman (input) for mailman id 722427;
 Wed, 15 May 2024 14:26:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Fae-0002KG-5N
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:44 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 264d3fa6-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:42 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9E08814BF;
 Wed, 15 May 2024 07:27:06 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2BB353F7A6;
 Wed, 15 May 2024 07:26:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 264d3fa6-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 5/7] xen/arm: Rework heap page allocation outside allocate_bank_memory
Date: Wed, 15 May 2024 15:26:23 +0100
Message-Id: <20240515142626.3480640-6-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The function allocate_bank_memory allocates pages from the heap and
maps them to the guest using guest_physmap_add_page.

As a preparation work to support static shared memory bank when the
host physical address is not provided, Xen needs to allocate memory
from the heap, so rework allocate_bank_memory moving out the page
allocation in a new function called allocate_domheap_memory.

The function allocate_domheap_memory takes a callback function and
a pointer to some extra information passed to the callback and this
function will be called for every region, until a defined size is
reached.

In order to keep allocate_bank_memory functionality, the callback
passed to allocate_domheap_memory is a wrapper for
guest_physmap_add_page.

Let allocate_domheap_memory be externally visible, in order to use
it in the future from the static shared memory module.

Take the opportunity to change the signature of allocate_bank_memory
and remove the 'struct domain' parameter, which can be retrieved from
'struct kernel_info'.

No functional changes is intended.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2:
 - Reduced scope of pg var in allocate_domheap_memory, removed not
   necessary BUG_ON(), changed callback to return bool and fix
   comment. (Michal)
---
 xen/arch/arm/dom0less-build.c           |  4 +-
 xen/arch/arm/domain_build.c             | 79 +++++++++++++++++--------
 xen/arch/arm/include/asm/domain_build.h |  9 ++-
 3 files changed, 62 insertions(+), 30 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242f4..20ddf6f8f250 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -60,12 +60,12 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
 
     mem->nr_banks = 0;
     bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
                                bank_size) )
         goto fail;
 
     bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
                                bank_size) )
         goto fail;
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4febba69ef63..fd8babacb9df 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -417,30 +417,15 @@ static void __init allocate_memory_11(struct domain *d,
 }
 
 #ifdef CONFIG_DOM0LESS_BOOT
-bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                                 gfn_t sgfn, paddr_t tot_size)
+bool __init allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                                    alloc_domheap_mem_cb cb, void *extra)
 {
-    struct membanks *mem = kernel_info_get_mem(kinfo);
-    int res;
-    struct page_info *pg;
-    struct membank *bank;
-    unsigned int max_order = ~0;
-
-    /*
-     * allocate_bank_memory can be called with a tot_size of zero for
-     * the second memory bank. It is not an error and we can safely
-     * avoid creating a zero-size memory bank.
-     */
-    if ( tot_size == 0 )
-        return true;
-
-    bank = &mem->bank[mem->nr_banks];
-    bank->start = gfn_to_gaddr(sgfn);
-    bank->size = tot_size;
+    unsigned int max_order = UINT_MAX;
 
     while ( tot_size > 0 )
     {
         unsigned int order = get_allocation_size(tot_size);
+        struct page_info *pg;
 
         order = min(max_order, order);
 
@@ -463,17 +448,61 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
             continue;
         }
 
-        res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
-        if ( res )
-        {
-            dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        if ( !cb(d, pg, order, extra) )
             return false;
-        }
 
-        sgfn = gfn_add(sgfn, 1UL << order);
         tot_size -= (1ULL << (PAGE_SHIFT + order));
     }
 
+    return true;
+}
+
+static bool __init guest_map_pages(struct domain *d, struct page_info *pg,
+                                   unsigned int order, void *extra)
+{
+    gfn_t *sgfn = (gfn_t *)extra;
+    int res;
+
+    BUG_ON(!sgfn);
+    res = guest_physmap_add_page(d, *sgfn, page_to_mfn(pg), order);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        return false;
+    }
+
+    *sgfn = gfn_add(*sgfn, 1UL << order);
+
+    return true;
+}
+
+bool __init allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                                 paddr_t tot_size)
+{
+    struct membanks *mem = kernel_info_get_mem(kinfo);
+    struct domain *d = kinfo->d;
+    struct membank *bank;
+
+    /*
+     * allocate_bank_memory can be called with a tot_size of zero for
+     * the second memory bank. It is not an error and we can safely
+     * avoid creating a zero-size memory bank.
+     */
+    if ( tot_size == 0 )
+        return true;
+
+    bank = &mem->bank[mem->nr_banks];
+    bank->start = gfn_to_gaddr(sgfn);
+    bank->size = tot_size;
+
+    /*
+     * Allocate pages from the heap until tot_size is zero and map them to the
+     * guest using guest_map_pages, passing the starting gfn as extra parameter
+     * for the map operation.
+     */
+    if ( !allocate_domheap_memory(d, tot_size, guest_map_pages, &sgfn) )
+        return false;
+
     mem->nr_banks++;
     kinfo->unassigned_mem -= bank->size;
 
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 45936212ca21..e712afbc7fbf 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -5,9 +5,12 @@
 #include <asm/kernel.h>
 
 typedef __be32 gic_interrupt_t[3];
-
-bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                          gfn_t sgfn, paddr_t tot_size);
+typedef bool (*alloc_domheap_mem_cb)(struct domain *d, struct page_info *pg,
+                                     unsigned int order, void *extra);
+bool allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                             alloc_domheap_mem_cb cb, void *extra);
+bool allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                          paddr_t tot_size);
 int construct_domain(struct domain *d, struct kernel_info *kinfo);
 int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit);
 int make_chosen_node(const struct kernel_info *kinfo);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722428.1126404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fah-0003Rt-EG; Wed, 15 May 2024 14:26:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722428.1126404; Wed, 15 May 2024 14:26:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fah-0003R9-6H; Wed, 15 May 2024 14:26:47 +0000
Received: by outflank-mailman (input) for mailman id 722428;
 Wed, 15 May 2024 14:26:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Faf-0002KG-Fq
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:45 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 26facb00-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:43 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AA8EF1516;
 Wed, 15 May 2024 07:27:07 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3EA633F7A6;
 Wed, 15 May 2024 07:26:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26facb00-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 6/7] xen/arm: Implement the logic for static shared memory from Xen heap
Date: Wed, 15 May 2024 15:26:24 +0100
Message-Id: <20240515142626.3480640-7-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This commit implements the logic to have the static shared memory banks
from the Xen heap instead of having the host physical address passed from
the user.

When the host physical address is not supplied, the physical memory is
taken from the Xen heap using allocate_domheap_memory, the allocation
needs to occur at the first handled DT node and the allocated banks
need to be saved somewhere, so introduce the 'shm_heap_banks' static
global variable of type 'struct meminfo' that will hold the banks
allocated from the heap, its field .shmem_extra will be used to point
to the bootinfo shared memory banks .shmem_extra space, so that there
is not further allocation of memory and every bank in shm_heap_banks
can be safely identified by the shm_id to reconstruct its traceability
and if it was allocated or not.

A search into 'shm_heap_banks' will reveal if the banks were allocated
or not, in case the host address is not passed, and the callback given
to allocate_domheap_memory will store the banks in the structure and
map them to the current domain, to do that, some changes to
acquire_shared_memory_bank are made to let it differentiate if the bank
is from the heap and if it is, then assign_pages is called for every
bank.

When the bank is already allocated, for every bank allocated with the
corresponding shm_id, handle_shared_mem_bank is called and the mapping
are done.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2 changes:
 - add static inline get_shmem_heap_banks(), given the changes to the
   struct membanks interface. Rebase changes due to removal of
   owner_dom_io arg from handle_shared_mem_bank.
   Change save_map_heap_pages return type given the changes to the
   allocate_domheap_memory callback type.
---
 xen/arch/arm/static-shmem.c | 186 ++++++++++++++++++++++++++++++------
 1 file changed, 155 insertions(+), 31 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index ddaacbc77740..9c3a83042d8b 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -9,6 +9,22 @@
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
 
+typedef struct {
+    struct domain *d;
+    paddr_t gbase;
+    const char *role_str;
+    struct shmem_membank_extra *bank_extra_info;
+} alloc_heap_pages_cb_extra;
+
+static struct meminfo __initdata shm_heap_banks = {
+    .common.max_banks = NR_MEM_BANKS
+};
+
+static inline struct membanks *get_shmem_heap_banks(void)
+{
+    return container_of(&shm_heap_banks.common, struct membanks, common);
+}
+
 static void __init __maybe_unused build_assertions(void)
 {
     /*
@@ -64,7 +80,8 @@ static bool __init is_shm_allocated_to_domio(paddr_t pbase)
 }
 
 static mfn_t __init acquire_shared_memory_bank(struct domain *d,
-                                               paddr_t pbase, paddr_t psize)
+                                               paddr_t pbase, paddr_t psize,
+                                               bool bank_from_heap)
 {
     mfn_t smfn;
     unsigned long nr_pfns;
@@ -84,19 +101,31 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     d->max_pages += nr_pfns;
 
     smfn = maddr_to_mfn(pbase);
-    res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+    if ( bank_from_heap )
+        /*
+         * When host address is not provided, static shared memory is
+         * allocated from heap and shall be assigned to owner domain.
+         */
+        res = assign_pages(maddr_to_page(pbase), nr_pfns, d, 0);
+    else
+        res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+
     if ( res )
     {
-        printk(XENLOG_ERR
-               "%pd: failed to acquire static memory: %d.\n", d, res);
-        d->max_pages -= nr_pfns;
-        return INVALID_MFN;
+        printk(XENLOG_ERR "%pd: failed to %s static memory: %d.\n", d,
+               bank_from_heap ? "assign" : "acquire", res);
+        goto fail;
     }
 
     return smfn;
+
+ fail:
+    d->max_pages -= nr_pfns;
+    return INVALID_MFN;
 }
 
 static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       bool bank_from_heap,
                                        const struct membank *shm_bank)
 {
     mfn_t smfn;
@@ -109,10 +138,7 @@ static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
     psize = shm_bank->size;
     nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
-    printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
-           d, pbase, pbase + psize);
-
-    smfn = acquire_shared_memory_bank(d, pbase, psize);
+    smfn = acquire_shared_memory_bank(d, pbase, psize, bank_from_heap);
     if ( mfn_eq(smfn, INVALID_MFN) )
         return -EINVAL;
 
@@ -183,6 +209,7 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
 
 static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
                                          const char *role_str,
+                                         bool bank_from_heap,
                                          const struct membank *shm_bank)
 {
     bool owner_dom_io = true;
@@ -192,6 +219,9 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
     pbase = shm_bank->start;
     psize = shm_bank->size;
 
+    printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
+           d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
+
     /*
      * "role" property is optional and if it is defined explicitly,
      * then the owner domain is not the default "dom_io" domain.
@@ -211,7 +241,8 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
          * We found the first borrower of the region, the owner was not
          * specified, so they should be assigned to dom_io.
          */
-        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                   bank_from_heap, shm_bank);
         if ( ret )
             return ret;
     }
@@ -228,6 +259,39 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
     return 0;
 }
 
+static bool __init save_map_heap_pages(struct domain *d, struct page_info *pg,
+                                       unsigned int order, void *extra)
+{
+    alloc_heap_pages_cb_extra *b_extra = (alloc_heap_pages_cb_extra *)extra;
+    int idx = shm_heap_banks.common.nr_banks;
+    int ret = -ENOSPC;
+
+    BUG_ON(!b_extra);
+
+    if ( idx < shm_heap_banks.common.max_banks )
+    {
+        shm_heap_banks.bank[idx].start = page_to_maddr(pg);
+        shm_heap_banks.bank[idx].size = (1ULL << (PAGE_SHIFT + order));
+        shm_heap_banks.bank[idx].shmem_extra = b_extra->bank_extra_info;
+        shm_heap_banks.common.nr_banks++;
+
+        ret = handle_shared_mem_bank(b_extra->d, b_extra->gbase,
+                                     b_extra->role_str, true,
+                                     &shm_heap_banks.bank[idx]);
+        if ( !ret )
+        {
+            /* Increment guest physical address for next mapping */
+            b_extra->gbase += shm_heap_banks.bank[idx].size;
+            return true;
+        }
+    }
+
+    printk("Failed to allocate static shared memory from Xen heap: (%d)\n",
+           ret);
+
+    return false;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -265,37 +329,97 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
-        if ( INVALID_PADDR == pbase )
-        {
-            printk("%pd: host physical address must be chosen by users at the moment", d);
-            return -EINVAL;
-        }
+        /* "role" property is optional */
+        dt_property_read_string(shm_node, "role", &role_str);
 
         /*
-         * xen,shared-mem = <pbase, gbase, size>;
-         * TODO: pbase is optional.
+         * xen,shared-mem = <[pbase,] gbase, size>;
+         * pbase is optional.
          */
         addr_cells = dt_n_addr_cells(shm_node);
         size_cells = dt_n_size_cells(shm_node);
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
-        for ( i = 0; i < PFN_DOWN(psize); i++ )
-            if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
-            {
-                printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
-                       d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
-                return -EINVAL;
-            }
+        if ( pbase != INVALID_PADDR )
+        {
+            /* guest phys address is after host phys address */
+            gbase = dt_read_paddr(cells + addr_cells, addr_cells);
+
+            for ( i = 0; i < PFN_DOWN(psize); i++ )
+                if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
+                {
+                    printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
+                        d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
+                    return -EINVAL;
+                }
+
+            /* The host physical address is supplied by the user */
+            ret = handle_shared_mem_bank(d, gbase, role_str, false,
+                                         boot_shm_bank);
+            if ( ret )
+                return ret;
+        }
+        else
+        {
+            /*
+             * The host physical address is not supplied by the user, so it
+             * means that the banks needs to be allocated from the Xen heap,
+             * look into the already allocated banks from the heap.
+             */
+            const struct membank *alloc_bank =
+                find_shm_bank_by_id(get_shmem_heap_banks(), shm_id);
 
-        /* "role" property is optional */
-        dt_property_read_string(shm_node, "role", &role_str);
+            /* guest phys address is right at the beginning */
+            gbase = dt_read_paddr(cells, addr_cells);
 
-        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
-        if ( ret )
-            return ret;
+            if ( !alloc_bank )
+            {
+                alloc_heap_pages_cb_extra cb_arg = { d, gbase, role_str,
+                    boot_shm_bank->shmem_extra };
+
+                /* shm_id identified bank is not yet allocated */
+                if ( !allocate_domheap_memory(NULL, psize, save_map_heap_pages,
+                                              &cb_arg) )
+                {
+                    printk(XENLOG_ERR
+                           "Failed to allocate (%"PRIpaddr"MB) pages as static shared memory from heap\n",
+                           psize >> 20);
+                    return -EINVAL;
+                }
+            }
+            else
+            {
+                /* shm_id identified bank is already allocated */
+                const struct membank *end_bank =
+                        &shm_heap_banks.bank[shm_heap_banks.common.nr_banks];
+                paddr_t gbase_bank = gbase;
+
+                /*
+                 * Static shared memory banks that are taken from the Xen heap
+                 * are allocated sequentially in shm_heap_banks, so starting
+                 * from the first bank found identified by shm_id, the code can
+                 * just advance by one bank at the time until it reaches the end
+                 * of the array or it finds another bank NOT identified by
+                 * shm_id
+                 */
+                for ( ; alloc_bank < end_bank; alloc_bank++ )
+                {
+                    if ( strncmp(shm_id, alloc_bank->shmem_extra->shm_id,
+                                 MAX_SHM_ID_LENGTH) != 0 )
+                        break;
+
+                    ret = handle_shared_mem_bank(d, gbase_bank, role_str, true,
+                                                 alloc_bank);
+                    if ( ret )
+                        return ret;
+
+                    /* Increment guest physical address for next mapping */
+                    gbase_bank += alloc_bank->size;
+                }
+            }
+        }
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:26:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:26:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722429.1126411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fah-0003Zr-Pf; Wed, 15 May 2024 14:26:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722429.1126411; Wed, 15 May 2024 14:26:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Fah-0003YX-K7; Wed, 15 May 2024 14:26:47 +0000
Received: by outflank-mailman (input) for mailman id 722429;
 Wed, 15 May 2024 14:26:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRjG=MS=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s7Fag-0002KG-D8
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:26:46 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 27a825e7-12c7-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:26:44 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F22A715A1;
 Wed, 15 May 2024 07:27:08 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 51BAD3F7A6;
 Wed, 15 May 2024 07:26:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27a825e7-12c7-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v2 7/7] xen/docs: Describe static shared memory when host address is not provided
Date: Wed, 15 May 2024 15:26:25 +0100
Message-Id: <20240515142626.3480640-8-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240515142626.3480640-1-luca.fancellu@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

This commit describe the new scenario where host address is not provided
in "xen,shared-mem" property and a new example is added to the page to
explain in details.

Take the occasion to fix some typos in the page.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v2:
 - Add Michal R-by
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-10-Penny.Zheng@arm.com/
   with some changes in the commit message.
---
 docs/misc/arm/device-tree/booting.txt | 52 ++++++++++++++++++++-------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2f6..ac4bad6fe5e0 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -590,7 +590,7 @@ communication.
     An array takes a physical address, which is the base address of the
     shared memory region in host physical address space, a size, and a guest
     physical address, as the target address of the mapping.
-    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
+    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >;
 
     It shall also meet the following criteria:
     1) If the SHM ID matches with an existing region, the address range of the
@@ -601,8 +601,8 @@ communication.
     The number of cells for the host address (and size) is the same as the
     guest pseudo-physical address and they are inherited from the parent node.
 
-    Host physical address is optional, when missing Xen decides the location
-    (currently unimplemented).
+    Host physical address is optional, when missing Xen decides the location.
+    e.g. xen,shared-mem = < [guest address] [size] >;
 
 - role (Optional)
 
@@ -629,7 +629,7 @@ chosen {
         role = "owner";
         xen,shm-id = "my-shared-mem-0";
         xen,shared-mem = <0x10000000 0x10000000 0x10000000>;
-    }
+    };
 
     domU1 {
         compatible = "xen,domain";
@@ -640,25 +640,36 @@ chosen {
         vpl011;
 
         /*
-         * shared memory region identified as 0x0(xen,shm-id = <0x0>)
-         * is shared between Dom0 and DomU1.
+         * shared memory region "my-shared-mem-0" is shared
+         * between Dom0 and DomU1.
          */
         domU1-shared-mem@10000000 {
             compatible = "xen,domain-shared-memory-v1";
             role = "borrower";
             xen,shm-id = "my-shared-mem-0";
             xen,shared-mem = <0x10000000 0x50000000 0x10000000>;
-        }
+        };
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between DomU1 and DomU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * DomU1 and DomU2.
          */
         domU1-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x60000000 0x20000000>;
-        }
+        };
+
+        /*
+         * shared memory region "my-shared-mem-2" is shared between
+         * DomU1 and DomU2.
+         */
+        domU1-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "owner";
+            xen,shared-mem = <0x80000000 0x20000000>;
+        };
 
         ......
 
@@ -672,14 +683,21 @@ chosen {
         cpus = <1>;
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between domU1 and domU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * domU1 and domU2.
          */
         domU2-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x70000000 0x20000000>;
-        }
+        };
+
+        domU2-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "borrower";
+            xen,shared-mem = <0x90000000 0x20000000>;
+        };
 
         ......
     };
@@ -699,3 +717,11 @@ shared between DomU1 and DomU2. It will get mapped at 0x60000000 in DomU1 guest
 physical address space, and at 0x70000000 in DomU2 guest physical address space.
 DomU1 and DomU2 are both the borrower domain, the owner domain is the default
 owner domain DOMID_IO.
+
+For the static shared memory region "my-shared-mem-2", since host physical
+address is not provided by user, Xen will automatically allocate 512MB
+from heap as static shared memory to be shared between DomU1 and DomU2.
+The automatically allocated static shared memory will get mapped at
+0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
+guest physical address space. DomU1 is explicitly defined as the owner domain,
+and DomU2 is the borrower domain.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 15 14:27:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:27:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722435.1126429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FbB-0005xb-Fk; Wed, 15 May 2024 14:27:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722435.1126429; Wed, 15 May 2024 14:27:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FbB-0005xO-Cq; Wed, 15 May 2024 14:27:17 +0000
Received: by outflank-mailman (input) for mailman id 722435;
 Wed, 15 May 2024 14:27:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6wQf=MS=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s7Fb9-0005pX-UO
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:27:16 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 38a0a0c4-12c7-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 16:27:13 +0200 (CEST)
Received: from SA9PR13CA0139.namprd13.prod.outlook.com (2603:10b6:806:27::24)
 by MN0PR12MB5954.namprd12.prod.outlook.com (2603:10b6:208:37d::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 14:27:10 +0000
Received: from SA2PEPF00003AE7.namprd02.prod.outlook.com
 (2603:10b6:806:27:cafe::8e) by SA9PR13CA0139.outlook.office365.com
 (2603:10b6:806:27::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.10 via Frontend
 Transport; Wed, 15 May 2024 14:27:10 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SA2PEPF00003AE7.mail.protection.outlook.com (10.167.248.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 14:27:09 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 09:27:08 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 09:27:07 -0500
Received: from [172.18.62.51] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 15 May 2024 09:27:06 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38a0a0c4-12c7-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nRN/NWowAMa0CcFnVMuLoFG15i/j0niUdCQym4mqB7EPiN1JqQWnw72b1YTB5BX6bLeXkPreJVulTtnr27Xgks/aZ8M4eOfsdzXy93C/MhhKSI4c+9kmspfmX8N4CnWT/ceKhwBQ+2DPwP+onLNzKkLLdKynzTCzpjzTmGpqx8yuU6zzUHYLEl6dFkSf/7YBAeT0V734SnWRyjhffOXelEQSP8WnyoxIw/y7mhT/OeGxuZ7JkqpoufczX1M5PhTCYWREvSHqWp82UjkOewBYwpEWAzVD2FVMKrD08RicrqdZ1J6+23PqeE5JpkLKb+kM4x/rc9z5eEiEayIZHytWJw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=miWKji/1XntNKyYIcFoAx/UroAi6DyHJuVvRVQCCt7A=;
 b=TtMew6jD/3kCQzeEq9QQeHewSC2cgdqRLwQWizRMoDH0zwjNzHwvP3RhlqEFRkvst/nhp2Ax4uPn5S92nhz5yIqXcrHkFl2TU/IPLRts64JPlOweuiXkvFtC0iASoBJjlC1L3UDzpkp48VY1EienAaoexX/ee2uON3Zq2XUv/oJ/i4t8XHDn6zcgCZObpJMOPwe3ioYM6j6uixz1VtR6oa55Muos5Jv0KlevD+Yh0rG1tfgtzhDXxkv1lRQy/m+anlFjK80o5wYvfVZkQ2IVwMe75Jhsh4KzbGB3aNLp7QQUsS/cAJ/a3sVcSyrbTHKQC7QYjjhd5PNPU/FUFJ2UgA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=miWKji/1XntNKyYIcFoAx/UroAi6DyHJuVvRVQCCt7A=;
 b=2J/WU6mCSX635BmrZ2MhKf2NJuKlb7FbzPZvEzDiqbMeWQ5+nw3NBcMPorJocGEUpV4Bo/6Vu/3pgvAG8YKG4hFgVLjsKyzKuMa3a/j7OsIeyjIQrU9Q78riMlb9zKHob/I2qSV5AXY5uXwDOuakJPvTu/T+KLgQA0fe7K9h8EI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <4eab3820-d357-42cb-a367-a7a0966e7e3b@amd.com>
Date: Wed, 15 May 2024 10:27:05 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v7 1/5] xen/vpci: Clear all vpci status of device
To: <xen-devel@lists.xenproject.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Oleksii K. <oleksii.kurochko@gmail.com>
CC: Jiqian Chen <Jiqian.Chen@amd.com>, Jan Beulich <jbeulich@suse.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, "George Dunlap"
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, "Daniel P .
 Smith" <dpsmith@apertussolutions.com>, Huang Rui <Ray.Huang@amd.com>
References: <20240419035340.608833-1-Jiqian.Chen@amd.com>
 <20240419035340.608833-2-Jiqian.Chen@amd.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <20240419035340.608833-2-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00003AE7:EE_|MN0PR12MB5954:EE_
X-MS-Office365-Filtering-Correlation-Id: 1d5006f1-ef14-4344-1490-08dc74eb1af3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|7416005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?TER5eHNHQ2NiUisyaGJQd0diNVJIQ3VjWlkzUlN1Y0VHUk8xbFJkOEZ5aUJC?=
 =?utf-8?B?K2x1cnVFS3BCNEVZOEFTK2xSYUhiZWhKYWdyajdXOGxxUFZTK211djJId0xG?=
 =?utf-8?B?QTV2Qis2S2lrSDNxVWljZG1iL1lBYzBQUENxTEdEZTZCeDFrSmxmVFhFaGxJ?=
 =?utf-8?B?Z2xSSjdETW1HYVRFOHpITi9sOTJvSjkwajRLOStySkN3YlFOaXczd0crenJR?=
 =?utf-8?B?c213OGNFWW0ydE5ZZjR2Z3lTYTNXYnlFVGRpMlA5blVVLzdJYk5LYTZQMFQ4?=
 =?utf-8?B?a2JoY2kxVDZNQUhGTzVieitIK2N0dm1oN2ZtT25VZDdlcysvbFVBVzkzUEcx?=
 =?utf-8?B?MDNGcUhTVGlwajg4RGdsWGJzZEJuNmVOa0NuL2NjMi9oL2hTNHp5ZnREaFVp?=
 =?utf-8?B?NHA1aHViUEZ2S0VMb1VIaFh3UGZCeWIzWktReVMrOW9DaTFYa0x3VWJQeFZs?=
 =?utf-8?B?N21SS3NNRTJiejh0ZnlBcXdEYmZickViRFZ5T0NSNG84aEROWWxWalpSUStx?=
 =?utf-8?B?a1piQUdSNmNwdW05ZzF0eG1rNHZvUkx4TGQyRUU0T1kyd3NLa09NMVNoQS84?=
 =?utf-8?B?ZlFlU2hLUDF2UlJYYW1mU3RKZ1RQeEZROTVwQUhIWE5XUHBXQWpSR2RqR21i?=
 =?utf-8?B?UDJETGJHeTlxZE8wK0dKenp6dmJtRTZYZllxNm9vQjJLSDNKM2JOeEdFSzM0?=
 =?utf-8?B?WlRvaSt3bXdXQ3ZhWW5nNERLK09VV1VRZzdvZkFSOWZUOEZyZjVMbk95OVZT?=
 =?utf-8?B?bDExYmVLS2V2TTJFM1ROc2M5ZFplS0hOYldYcmd6SG5KTE5ZWDRjQlV6djBO?=
 =?utf-8?B?NG9QMmo5YWRsL09nS2ovK1FybDdHNU9tcVFLczFWcEFCK2RLMkdpWHZaczJ1?=
 =?utf-8?B?NXJQMFhJRnVIMndjV0xlaU5xcXVMZnlwRmJhdkxzNW5wRzhadWhJMEQ4ejlC?=
 =?utf-8?B?SUlmOWxXL3E3ZlNFQ1R0dlV3TmhJRnBBOFVhZUZBdTFzc3ZvRnRVRGo4SC9z?=
 =?utf-8?B?R3VOZkY3VXA2VnpBNS8rYXJCazVOZHdYd1B3UWhCRkRzNW50a3g5S1FSVnhH?=
 =?utf-8?B?QWtvb0RMeENSK2JieHlIeldRSXpiL1hYVDdVVnZtcVhiUkhXbWgyQ3lzTXhm?=
 =?utf-8?B?OTM5T0xDbEsrYkZDOXpDWWR2eXpYeE1qM2drNkFtMERjKzQwc3p0aVFWMi9I?=
 =?utf-8?B?MUUwN3JaNzZYREdkOTVYbWJrK1lZWVIwOW42TFFFRW00M1V3QkdTNlhRamJq?=
 =?utf-8?B?aXZJakxIeVg2dzZHalZ4aklzeWpBVnRvcWVFeWdaRUw0eE1EU3lGaVlWWDhM?=
 =?utf-8?B?TDNTd0YzOUlyVGJwOXFRQXBpKzZ0emFSUVMvaHM5RTJuQm1rVHZqTitnd2hm?=
 =?utf-8?B?UzFlWHYzejNSSVpreGpBc0FDbkNBbEh5cnNubHd3MGtDQ0JmMUhzV0FaaHNw?=
 =?utf-8?B?K3Buc1pVdnVncTRXcHdRSEN0RE8ya21yWkdKZTNLNStuUTBjKzBWQjNySkxj?=
 =?utf-8?B?Y1ZSN0tMenA2cTc3VEVlTmtoRXdhbENPajUwUkZsZWxTYmw2VUUwcTMwdGhM?=
 =?utf-8?B?L2NKK25zK0wwc29VVDYxOVlMN2xSSDk0SzVhWWdaNDk3UEdDQVJvQWRhNG1z?=
 =?utf-8?B?ZVRXell1bkxpNHliMXM5Z01RZDJPZ1F6bFZXa3RnTGlUNytJcjFOcCszZ0tN?=
 =?utf-8?B?MUF0czMzSmcrYldNZlJ0dlUxdHZlN0k1Slplc0pBcERlRkRISXMvOTdVdXZS?=
 =?utf-8?Q?3tu+bvcxuxNXjbJVIZr2WOudsGrh3LOWriJJMCG?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(7416005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 14:27:09.7573
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d5006f1-ef14-4344-1490-08dc74eb1af3
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00003AE7.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5954

On 4/18/24 23:53, Jiqian Chen wrote:
> When a device has been reset on dom0 side, the vpci on Xen
> side won't get notification, so the cached state in vpci is
> all out of date compare with the real device state.
> To solve that problem, add a new hypercall to clear all vpci
> device state. When the state of device is reset on dom0 side,
> dom0 can call this hypercall to notify vpci.
> 
> Signed-off-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Could we consider this patch for 4.19? It's independent of the rest of
this series, and it fixes a real issue observed on both Arm and x86. The
Linux counterpart has already been merged in linux-next [0].

[0] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240515&id=b272722511d5e8ae580f01830687b8a6b2717f01


From xen-devel-bounces@lists.xenproject.org Wed May 15 14:34:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 14:34:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722460.1126448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FiF-0000UC-Ak; Wed, 15 May 2024 14:34:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722460.1126448; Wed, 15 May 2024 14:34:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7FiF-0000U5-81; Wed, 15 May 2024 14:34:35 +0000
Received: by outflank-mailman (input) for mailman id 722460;
 Wed, 15 May 2024 14:34:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DTPh=MS=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s7FiD-0000Tz-HA
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 14:34:33 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3cf22f48-12c8-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 16:34:31 +0200 (CEST)
Received: from BN9PR03CA0657.namprd03.prod.outlook.com (2603:10b6:408:13b::32)
 by DM4PR12MB6397.namprd12.prod.outlook.com (2603:10b6:8:b4::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Wed, 15 May
 2024 14:34:26 +0000
Received: from BN3PEPF0000B371.namprd21.prod.outlook.com
 (2603:10b6:408:13b:cafe::3e) by BN9PR03CA0657.outlook.office365.com
 (2603:10b6:408:13b::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26 via Frontend
 Transport; Wed, 15 May 2024 14:34:26 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN3PEPF0000B371.mail.protection.outlook.com (10.167.243.168) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Wed, 15 May 2024 14:34:26 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 09:34:26 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 09:34:25 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 15 May 2024 09:34:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cf22f48-12c8-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R5I5k/828S1h0q2SiFFgvpojhsLSPhpFGGyLvVLMyFoMmbZ88rpFA+Kj+e+BxszefHfF7JfG2JpDK0dmPl8ytiVSxWJO3Xe7fwB3kX7PHwxXgHhzbprwlBBu8YFj3u/wFaARESMcMezH34dENEA0wmdiHJFbqZP/M8l2B8eA9LrsaRjJF2Lq6wnbmsaD+a/kCA2emCNplXg/fVREPHIXZoUUwp6JLxlXO6ZLokUAG07w1PGUB2/ltA4kOYl2K7rbCV4DaJ5TaDvpt1CgRte72tVJGb2dLmEjb0SrFZ0D444sWBQmf8Yo2qYlzOJFLpX2LcVm5j61rM2qntPqRfIieA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=na9INcQpOnXYA2PqNBCo4QBb/elEfAJ15EFW6fuAr3M=;
 b=LRzPQrmturGMSYqrdMWK5ok/NWPAtLVvksJ8uNe39nD/mp1DDU6LvYVrthWJxgFYv4ACDbPbX8dpfMAAd1C4LDVCYmrvI1cCRIi4MamgLIQb6fUJ6oukglC+81eH01tfe8CgHXk4mZe2pSM5c3d+viNRojAOr/rcuxNLXw6yMy18Nmm24TdX/6JvkVjX20WCt3FYgiC0ZsJx7JNVirzuj7GKo02idqftSrrPUSM1BMbxspKGjPlPd1cAGQo/6E3kiy2zS2fQACbbaozAGCiBygw9xURQM1jLB8RbYgWaF36EY8GqKP7W4Qnb96u2zMFXrfzxWZ6OWmJWn0Q4JPor1w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=gmail.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=na9INcQpOnXYA2PqNBCo4QBb/elEfAJ15EFW6fuAr3M=;
 b=2A4xHK5NKnLh6/IgjDM4PNN+4HmWlBWV9+NcuIkfp3i07G9GfgepHSDPCUTQ/zkbJdm1B/Alt5IwP0ol4jE49zqTnQV57foI/8ooxFXYnCDaoeSAo/G48XmuWYI8CStJP6K6yp9SJWUCmQEHF26u757ahYXUd4ak+HLsOrVdDdc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <04e98df8-439e-4b17-9444-0bba19dde635@amd.com>
Date: Wed, 15 May 2024 16:34:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible
 header
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Shawn Anastasio
	<sanastasio@raptorengineering.com>, Rahul Singh <rahul.singh@arm.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <4ef575389520479cee0a79fccfc20e6891594d61.1714988096.git.oleksii.kurochko@gmail.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <4ef575389520479cee0a79fccfc20e6891594d61.1714988096.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B371:EE_|DM4PR12MB6397:EE_
X-MS-Office365-Filtering-Correlation-Id: ba5f7ad3-2b0f-480f-c2b7-08dc74ec1f51
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005|7416005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZXZvNm1ESHBhekgvVyt1QlZrUEFsWi9iOGNzeU43RXFRSVBrakJvYzVHNWdV?=
 =?utf-8?B?bGhZa0pLZUg3cEJyZElFTGN6a2svOEdUanRaQm5KczRmejMxVTArRloxSE4y?=
 =?utf-8?B?aTN2MGQrN3Z3VERaQ3hlZHJjWkZGKzczQWhhQlgxTTNacWdkRmNvcnJSK0gw?=
 =?utf-8?B?QXU3WWlGNDM5dkwwcXJDMGlLbTA2NTlxK2p6ell4SDdvR1c5NzR5NFhGSTlR?=
 =?utf-8?B?dXl6N2l0b3REMVVYRVJ5NmJvNXRkUWdlY3pnaVVuYTArekFDeENJWlpsVUtN?=
 =?utf-8?B?bWFxN2w5d1g0UE42c0RJSFRMOXI2VmFSNmVoNHFoY2hDZjJvQnVrQXd1RnZ0?=
 =?utf-8?B?R3JhZGtob0lES09jU3ZDcWh5R0ZndkNvUnFUbHZhaFNQb0tROWVHWnBSV0g0?=
 =?utf-8?B?T1R3K01zd3hSNGt1VHBBNXRETWoyMm1GNENzVHRoWHhhWjcrVVBLeVROSk1r?=
 =?utf-8?B?NHU4RHgrTjZWMjI2MFNZM2RkZjM5TmdGVHlEak5mVHpzSE9DS0NhVTJoSXpk?=
 =?utf-8?B?SW5KbG5vWWxpQW1YRFhja01uUDRpOFIyNzVjY3Y5d0ptMmJOK3M3UitSbUpu?=
 =?utf-8?B?Y2laZDNQcU92d05Od09wZ0NXc0RZS2YxSXQ2cjZkZEhwUlgvSGcwemI5M1Ja?=
 =?utf-8?B?aUJIajlvditIazhsRWdhSTFFRTc1WGVGdXlyNmh6MTd6dEIxSzJSTTFiVERB?=
 =?utf-8?B?RmVIVTNxRGhBNEVjS0hSR1BMcnBXdzN6eWp1YjFZZzdYN3Jvd1lid3VQNVVu?=
 =?utf-8?B?UFFBaDBva3FyNm5IMGpwVERucSt0bURUMDhod2hOZ2xlL0hjWVlZUDdBMzUy?=
 =?utf-8?B?RUxPcTIyb1hlcFphR2hDNmV4WEZtZDNmZUh5TEtneThpMSsxeG1DRHQreFoz?=
 =?utf-8?B?US9oOGF2WTdtaWowdzhkMkVXemQ5eDRJOVdIRmJydXg1cGZMZzVTazAzc3RR?=
 =?utf-8?B?eWRJU0NCNWlWYkx6MFNxZVZLWlJZN1JRL1FoK1NqNStmT0Z5MFRDSlhkWVI0?=
 =?utf-8?B?S0JHUTRVZWl0dkR2bUVOWlRiVDI4L3Z5M3F1YlhVektiSjlVQTk0Y2h2Umc3?=
 =?utf-8?B?OXVUdmVTa2FCSHR6THFUTGhrd1lwRHZ6YWJOTlBqczg2c2R1UXM0VlBQY1BF?=
 =?utf-8?B?eXEzeXBCMlJRZmFBYlNlTDN4VVZ3bytidVJXQ0VhdHBLUjBVVE96aVNWR2Vy?=
 =?utf-8?B?a2NnSVAxUFdOS1djR2FJemxBUWFTRkpUalhBendKNGw4b1RmcXhORGtRRHUz?=
 =?utf-8?B?M0M2VEw0NW5EaUdIQ3dTaGgvMCtzcXNpWXJyYzhtVml2eE5vVDNITUFTdTBM?=
 =?utf-8?B?RzBocDVUY1VGOUswNUc4WWlMdEdhSkIxNzdPdmhPK05OS3lEQXdNTnVHZ0xp?=
 =?utf-8?B?Mm15VVg4WkhmTWtLQkpleFExbWVEaEVvRndhcE8zY2hhVzdNcWthMk50WHNE?=
 =?utf-8?B?Q0RZNksxQlF0TXFEZDVLUEZHYldMZ0N1bEgraW1VSEpvQUtyUkRJYXM1M2dP?=
 =?utf-8?B?ZUVGZEhGVmZ0VDdLNlE4NWxwTElrSFk0SEp6RnFEbXpwSXBsSXA5cGRPVEhW?=
 =?utf-8?B?ME5wU3NFbERaNjR3emNWVjVzSlUrUXBwK3g2WmNJNmczejZWZFhrWkRxZGxi?=
 =?utf-8?B?SS9uV2FMWUhCeGRPcHlKcllwM3hnUkFLaDVOc05rdTFWMlAyMGQrbm81YjQv?=
 =?utf-8?B?QXZXVjJoZFpZS0UvRk54ZjdZQ095K0ZTWk9Gd2NDWUd6b0RYOExsQXRWTWdE?=
 =?utf-8?B?c2RESFVDM3BZVU4yd0xXZGIzT3Z1OWhScU0rZmZvelVSLzJVUWd5R0R0MlRv?=
 =?utf-8?B?VkNZZzkva0VJbzRSYzJGZz09?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 14:34:26.5854
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ba5f7ad3-2b0f-480f-c2b7-08dc74ec1f51
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B371.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6397



On 06/05/2024 12:15, Oleksii Kurochko wrote:
> 
> 
> The mentioned macros exist only because of Linux compatible purpose.
> 
> The patch defines __ffs() in terms of Xen bitops and it is safe
> to define in this way ( as __ffs() - 1 ) as considering that __ffs()
> was defined as __builtin_ctzl(x), which has undefined behavior when x=0,
> so it is assumed that such cases are not encountered in the current code.
> 
> To not include <xen/linux-compat.h> to Xen library files __ffs() and __ffz()
> were defined locally in find-next-bit.c.
> 
> Except __ffs() usage in find-next-bit.c only one usage of __ffs() leave
> in smmu-v3.c. It seems that it __ffs can be changed to ffsl(x)-1 in
> this file, but to keep smmu-v3.c looks close to linux it was deciced just
> to define __ffs() in xen/linux-compat.h and include it in smmu-v3.c
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Wed May 15 15:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722476.1126459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GG8-0006fE-SB; Wed, 15 May 2024 15:09:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722476.1126459; Wed, 15 May 2024 15:09:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GG8-0006f7-NC; Wed, 15 May 2024 15:09:36 +0000
Received: by outflank-mailman (input) for mailman id 722476;
 Wed, 15 May 2024 15:09:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/K8o=MS=arm.com=Rahul.Singh@srs-se1.protection.inumbo.net>)
 id 1s7GG6-0006f1-TO
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:09:35 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 (mail-dbaeur03on20601.outbound.protection.outlook.com
 [2a01:111:f403:260d::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 22ab411c-12cd-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:09:33 +0200 (CEST)
Received: from AM0PR03CA0079.eurprd03.prod.outlook.com (2603:10a6:208:69::20)
 by PAXPR08MB7336.eurprd08.prod.outlook.com (2603:10a6:102:223::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Wed, 15 May
 2024 15:09:31 +0000
Received: from AM4PEPF00027A5D.eurprd04.prod.outlook.com
 (2603:10a6:208:69:cafe::c) by AM0PR03CA0079.outlook.office365.com
 (2603:10a6:208:69::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Wed, 15 May 2024 15:09:31 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00027A5D.mail.protection.outlook.com (10.167.16.69) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21
 via Frontend Transport; Wed, 15 May 2024 15:09:30 +0000
Received: ("Tessian outbound daa456608199:v315");
 Wed, 15 May 2024 15:09:30 +0000
Received: from a741a34fe2d8.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0BA9C8B2-CB1E-4CC5-9F0C-5E3ADFA4B803.1; 
 Wed, 15 May 2024 15:08:36 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a741a34fe2d8.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 May 2024 15:08:36 +0000
Received: from AS8PR08MB7158.eurprd08.prod.outlook.com (2603:10a6:20b:404::24)
 by DU2PR08MB10279.eurprd08.prod.outlook.com (2603:10a6:10:46e::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Wed, 15 May
 2024 15:08:33 +0000
Received: from AS8PR08MB7158.eurprd08.prod.outlook.com
 ([fe80::3a3a:3b79:47ad:2313]) by AS8PR08MB7158.eurprd08.prod.outlook.com
 ([fe80::3a3a:3b79:47ad:2313%4]) with mapi id 15.20.7587.026; Wed, 15 May 2024
 15:08:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22ab411c-12cd-11ef-909d-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=Sm+y31RBMu6gwEEZ2AFnf0aDjLOC0MqsB3hiBJ9CSMX8o/RtitL28FdJWYCKk/RjMPq/X2Ed/bAEq+qvHeyJbnKjZFKY/NAZNGluAktLVPPX/jl7i2X7PqNceu+/pYZL/Ut+uijapF58OjXAa8+DDPvn5oGzswgv7BTozQgL6KCo9f6w+wk4d0Z79omrfBnY6l3Q3/0VRx5b+Cv6fVG9AMHEToBghRc2DedP7aq/gO55NyhQguWmRmRNAsGsSUCTTxx6ir8JX91DntFxnP/EwhCNUe3jYMU78YdmO47qg+Z0uqprLo2lsOfy8BRXhZIKmj13UNJvij6yE/ri61GAVw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=v/+9rMwEnhWdLab1Y2cQ+a1OelbnrTzOuGzbl5m1z7U=;
 b=NRjART/6JllAJmdbKzy7zFpbqiPGkaToo98CAKDAPqHFRmQq57m1tdWFrCw/RaREW8Q+dUgsq9CplM1OlTGaOlZ0H1ZGnDkpkh0F1ILJhlItX8g0Ogrt1vi7+HAonTnazNI8f7DMsZGBh0/Kiv1flWakeIzZB7iTrsN/vi/2nnXnJJfDNjL5UMIUkYflTZoofYxvs7RmLG//GHqFwjeaL+H4tWaFz/zPl+cVwTMAwdrN2p0Bhd82eEGTlI7C3odsgWJ0aY1XEIY4pHCA6E/zYNj3ajbFbfPZS6DGtt28LumN9I1sC/LqX2/wmGOzjB7WluQa9OaU6qCaoX3YHflW/A==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v/+9rMwEnhWdLab1Y2cQ+a1OelbnrTzOuGzbl5m1z7U=;
 b=JMQ9NoHec/7vgaYYAZMhWEbaw9K2P8e1RRH1P0lQ6tQlFvJijZR0kRtbf/76oW1OTw4g/WUeZhsyZlgWyHlxj8NVMaZLH818zcvXCWsOhhSieanEkvEXFC+7sIaVg30UxPJd7qS5K+1sghVEDVEHvdoLlPQR6vXv9Y41orqfezA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 728256f609e0e07e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EDqZa6dtA3XKpy1IMFxjryF35G9wJV0sZBxXxg4aRqcH0Do8Dhk7du/LYuqmu0wTazFoeRSB+ETZ3yuehzEE5rAroq1hXoL1cHLMCmAdUflDvlFcKakWkwQx2N3Gd9t6IzlSLbFDHvi6fU+3aNFGP6n9lk+PtCvcHK4oNtlDDHH7LRjcpcPUrTa6R4dIvGdzpp7c6nF+xsIHdeAIAFWf5gxVN4dNXGWkpV8mKlYt1sxvExyV02T4GRKx+X1GJuMFxoLy2LHxDF9rhdsCLdtuTSWTKWidQaqh2XcxLBBG+XiaHVb0JhFCBxQQsH/uOXujkXVrzq7WCos/y8Z8/m2SDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=v/+9rMwEnhWdLab1Y2cQ+a1OelbnrTzOuGzbl5m1z7U=;
 b=hD0oxacsan4XDhhOxP2JQCB6CBt+QUwBVQaR+/wyNZ0hmHNtLxLt/t6dsUUTPYW78a8rapy7Nx3CO4vzcBH9Lrzc1W/J5uMc4CS0Lb7Eqt7oRVOoMOyuU+FZ7vhYp6ZaEx6mMfuaefXrDbgTsSVBZw1JVB1PrphB0ywTlg9jRaNr+m/3jVUx9Tfwggq81X9A6RQz2Sj+s8DnRZLhhB0b99Zm+9QMBsuLDFtv1JsfDFF9E5cyh16U/yc4yxza40Jih3UA4Rfali1sTXuK1SLmf+7Qre+udkeW5kV4pZcaBxDf/b09AWgYfLkJkF25QI+jCk3Hn/03wKClEjKxAblxHA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v/+9rMwEnhWdLab1Y2cQ+a1OelbnrTzOuGzbl5m1z7U=;
 b=JMQ9NoHec/7vgaYYAZMhWEbaw9K2P8e1RRH1P0lQ6tQlFvJijZR0kRtbf/76oW1OTw4g/WUeZhsyZlgWyHlxj8NVMaZLH818zcvXCWsOhhSieanEkvEXFC+7sIaVg30UxPJd7qS5K+1sghVEDVEHvdoLlPQR6vXv9Y41orqfezA=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Shawn Anastasio
	<sanastasio@raptorengineering.com>
Subject: Re: [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible
 header
Thread-Topic: [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible
 header
Thread-Index: AQHan55fe9tWwhK0EkCZuWBTgB6qOLGYdCCA
Date: Wed, 15 May 2024 15:08:33 +0000
Message-ID: <69E3438E-1AA0-430B-9D92-B55DC0261D40@arm.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <4ef575389520479cee0a79fccfc20e6891594d61.1714988096.git.oleksii.kurochko@gmail.com>
In-Reply-To:
 <4ef575389520479cee0a79fccfc20e6891594d61.1714988096.git.oleksii.kurochko@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AS8PR08MB7158:EE_|DU2PR08MB10279:EE_|AM4PEPF00027A5D:EE_|PAXPR08MB7336:EE_
X-MS-Office365-Filtering-Correlation-Id: 67e185af-1bcf-4f28-c594-08dc74f10595
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|7416005|1800799015|366007|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?N0lJK1ltQW5wcFFrTkZpWnJVRUE5RnVJamthREQ5Zm1IbGo2ZFJtL3liMHpX?=
 =?utf-8?B?bkpYdWpaRjdPN215Ym5UbjBtVkpMRE5NNXYzVU8yVHVtWjNXb0huVmV4NHlW?=
 =?utf-8?B?dy9KeER0eWw2aWlGMzZwKzBaZmNtSG1MVHVXbUtUR1d3eFM3ZVBmbUM2bjdo?=
 =?utf-8?B?czFteEZDVXRpSzRUT2RlaDN3KzFDeXJCODYzbWk0RTd6alJiam16TlYycXF4?=
 =?utf-8?B?QUVkcWlZWFcyVGVEMWhpVUpNSi9qbm5zL0hEenhlOFdKazJjWTdtZHhKZ0lZ?=
 =?utf-8?B?MmhLZW12Sm9MRUE0VVlJNWUzbW5aWGZNdE9VZE1ZWHJCNUo4WU9JU1lJNGk5?=
 =?utf-8?B?Z0VlcVFpR3hraGV2RXNiTDhNS0I4d3JtM0RiUXliakdNbllzM1RKYVR1T25Y?=
 =?utf-8?B?VkdzbXQxc3gwdG1Wd3RlQVN0U25RVndOZHNFdFltRElBV1cyNlE4WTNjN0Zs?=
 =?utf-8?B?R3ZUT1Flb1cwMStoM3pyYXlHb0xoWTdWY1ZlRkVZZWdWSGV5aDhmbkdWZVVV?=
 =?utf-8?B?NHVzVHpzQ1JnYjVtMmY5aTlmYjlwOGtEbWUveUJVdWhyTG1kdGg0Vmg2RGht?=
 =?utf-8?B?b05lY0oycFh5Q1dVZy8raEtoM3NEY2U3L1ZaQ2pNaXFsUi9IUFFMUlRGUWR4?=
 =?utf-8?B?OXZqOGc2L1FRRmowb3JZUk9QK2l1aExwcnRucWxwdlcrNmZ2aHFwNWIrTXBt?=
 =?utf-8?B?VnAydmNXbzNqc2xybmVBVTd1TWNPZDVzN0V0REJxanhScE85TFYvMWZoQnFk?=
 =?utf-8?B?TE1PQzcrUTdKTTJlNmVZeGZJZXN0WHRrU0JyVEwvaWYydmF3Z0xNanVMbnp4?=
 =?utf-8?B?Q2dKS3hGSlQweXdBUE9UUExuL2ZPc1FIcHBLYnlPcjl1SGtRVG9LTGdSczN1?=
 =?utf-8?B?UWYzZDhSYXJETjJ3MjA0YkN2NkJMVWxDOUZSSFR2dCtVRmhqdzYwdlk5R0Jk?=
 =?utf-8?B?T2pLdkduSVJhazloQ2tyUTAwTjVxYnZnOWkyRTlqRFJXb3ZjMkpqcVJmT2ox?=
 =?utf-8?B?aDB4S2hQNzdrdHNWa3RvU2p3RjVvdmJuaFp5UWhKQkxVS2orZENhQmgxelRn?=
 =?utf-8?B?c3FjbHJ0ZTJuR2VSWDVEbjRsYXlveXFoYnVnVFFjSWhvYVBxalhIMkZrMlN5?=
 =?utf-8?B?b2J5UlVwVUl2QUNQUkE2THd4Mm13SW9uZ1BFczl4SDd0ZlQrS2ZMOFgvQjd5?=
 =?utf-8?B?QnhCVGdPbXJKT1NURDIyaEpwRVVVUHV3K2oydVlPaDlveWhpemhRWFJCY0o1?=
 =?utf-8?B?R3lQcjcvSlVteEhRSG9CU2tyd3NuaXNiM0prNC9sS3FldGw5RDNKQXljTmND?=
 =?utf-8?B?Y3gyRi9wQzJjVXZPYWxQN3FXMndUdlpwSC80N0dSK1lXOFp0YS8yblFzMkhS?=
 =?utf-8?B?NE5ERkdGRCtsVnQ3dHppQWpOTTR1Q25Ed3d0WC9CdkdJcjZEZUs0Q2xiSmFE?=
 =?utf-8?B?OFI4WWdFWHk3WHhTZVBoc05wYitVRFE4OHlqUmxHNUZFVzNBeWp3Z1NPMU1U?=
 =?utf-8?B?Ti9LbEo2bVQ4eHg5SjVaeFlTVXJJK1g5bUtRQnQ1ZmJTcGlkalk0bFdsWThM?=
 =?utf-8?B?ZzRaaTl5Mk5JOGp3YTFpZVpyMVR6eUlwR2t1bGNsT1hZNWg3T1hYelUyWC83?=
 =?utf-8?B?NTVhdE8rVFRMb0JZRCtLdmo5NzRBLy9zRmZJNnYxUDZTbUdITlhoTXY1ZFZi?=
 =?utf-8?B?VzF6OURIeUZKZmlZQkxOdGFUbk1SS1hlV20zRXkzeTZFbXJOOVl0WXI3emtu?=
 =?utf-8?B?YTYzRFI1TnRKNlNRSENFUFE2TTFmbTFISDl3SERHclcrcTZ3ck1xL1BhcGJO?=
 =?utf-8?B?M0gyZGh4ZmR3dUpSdVBhdz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB7158.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <78D7E8EFF3274F4B88852585BB58CEF6@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB10279
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00027A5D.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6e9124b2-f735-4128-bd6d-08dc74f0e375
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?dWpQcGdZSmFENExhL0J3RGVWc3Y4R05uNlFyS01WbHdvVkhoK1gwOFl1cWRa?=
 =?utf-8?B?b2NtTWdaSURRVThPU0NKeU12R0MwTWFpOU9FbFV0Nk0wRnk1TU5MNk5vMXZZ?=
 =?utf-8?B?cmNGZUdadU9ZNkJOanZQWllkMk1DdlVKT0E1UzR0YjhwRUxCQjQ0MHdTL2ZG?=
 =?utf-8?B?WHhpa1B3RXVMcHI0a0g5cm9CYTY0L1hYUFI5UkNsQ0Y1Y1d4MjdFbVIxR1Fp?=
 =?utf-8?B?K1IwMGNYWTZwcjZJUGdkbDMzSjRLdE5hT3luV1ZGMlBkUEoxbkdtQkgwV3Mx?=
 =?utf-8?B?SWpIbnZpSVpjMmR3d1NkNlh1OS81SVQwSUZYUFFGVnNrN0VCdHRxcUNUZjNU?=
 =?utf-8?B?d3lJbUlNK2VwcCtRazErT3dYRTBzN1Bqa0QvcXJVYzJ4RXY2R0lxUUkrN1p4?=
 =?utf-8?B?ZUlnZWdvMEFwd1F5Ym4xaHNKOVF2TVViZkJ4S1NkS3ZIZmV5Tm04eUNjVnZZ?=
 =?utf-8?B?OXE1UE51NFZnSDgrZnVWQ21GRlJoSkpTVzBHaTlXUG9ocFNnRklseWFxL3JM?=
 =?utf-8?B?cWduUGlGUWYyZFY1aDRnUTJvb09McEFSL0h4MWQ2RkJrK0M3ZVpCUTh2aG5Q?=
 =?utf-8?B?bU5MQWJ3ejNWYXlFbFN6OEJqdjdEMU5iaTZaMlRKTkFXSzJRUEhqL0dJWFBV?=
 =?utf-8?B?VWJXM2R1aENTT2s0R0gxOUtZbGtEUjE3UDNJVDVCOGZNaEx5eEVpZlhyUkRP?=
 =?utf-8?B?ckR0SFRzSnJVRW9XUG5USjJFeWJlanM1L0VucWRIUnVRZ1pkM0diNWJLWHpP?=
 =?utf-8?B?bjlFdjJvMWE1UFg3eTRuSG83SSsxMklNODNITW5JV0cwb2NwQmE2aVFRdlVY?=
 =?utf-8?B?Wnh1RFpEZE5FU01sTE1ITy9UOU9Ra2htSVRmWk9DNkhLR2RsMWFsc0YrYXNZ?=
 =?utf-8?B?cVBIQzZ5Yi9zaUtJOStYM0VwQ2pnVTIybXAzRjduRGtKRHl4SW9XeGgwaVhB?=
 =?utf-8?B?Um4rSkxvUWxleFVIZ29LUXd4d0N6OHEzM2NtMlR1RVdoQkdxY2JiVUI2TFJY?=
 =?utf-8?B?dzlITUhPZEdjQUhlTEVtVldkbjlxcUhKZEMzZ01iWjVDRlRDdEJVV2c0ZWRs?=
 =?utf-8?B?MU16UkJiSnJDeHJwTDF2RUo1TlpUMnJhdE54S2pXWXFKQ1lQaEdQMCtabjYr?=
 =?utf-8?B?UDRNQURWL2Y3aks4VEhLVlB1bll5dzBvV09oV2w1UWdBeGxqVy9TbXI4bWs2?=
 =?utf-8?B?eTNDYWEzTEZ0a3p1cDNQMVR6TW5oemxyQUcrbllpU3lDU0xOUFZGNUYyanhS?=
 =?utf-8?B?Q1FnME9KVXkxbXZoYzNmdVp3QTJLNGFhTkVYLzB5YmhXR1JBTGpIR3RUYmxL?=
 =?utf-8?B?bkN5NGRjTVhmOUVTSnJkeWI1NkoveHR5aG5kNmZIUFRnZkE0VDJQNG1YdTJF?=
 =?utf-8?B?Wm51MzAxbGJweTJhakZqZFJVVmpML0w5RlFGYkFsMGN6R1p4WkNIZVJMYTNZ?=
 =?utf-8?B?TXpBVkIwZ1dHNzBGSlRoTmN5VXA1SEJQcE1zMm1aWVFPREJGeDFoRHNRVkIv?=
 =?utf-8?B?MHZaTnlWTEdtUUdxNElRYm5NWjVWa2hLd25wTmRBdnU5T0hJQ0kwaWRHT0NJ?=
 =?utf-8?B?dHBucTgrbyt3UUJzeEtLRjNhai9jcXlFYzVHMTlNbVlYOWoxTXNjb1ZXYTBF?=
 =?utf-8?B?eS8wSVRpWklDWTRzQWE4akJQa2Y5NGFVRFIydGZvRnBMY1BBcmhCQlRwR29M?=
 =?utf-8?B?YllIa2EzRENHeWJvK1BzZVkvLzlEZU5vd2RTUkxxaVg5VHBsVVpJaGFRZEJM?=
 =?utf-8?B?NDEwWmpIbTFnY2ZNeEs5WmJ2eVlHWmJZQjdlNFB4eXpFTWtsWW9KZE5aVkRx?=
 =?utf-8?Q?6O2wuimYr1ZdeNLdVp+GRN2cUr54B6mNZrE3o=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 15:09:30.5972
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 67e185af-1bcf-4f28-c594-08dc74f10595
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00027A5D.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7336

SGkgT2xla3NpaSwNCg0KPiBPbiA2IE1heSAyMDI0LCBhdCAxMToxNeKAr0FNLCBPbGVrc2lpIEt1
cm9jaGtvIDxvbGVrc2lpLmt1cm9jaGtvQGdtYWlsLmNvbT4gd3JvdGU6DQo+IA0KPiBUaGUgbWVu
dGlvbmVkIG1hY3JvcyBleGlzdCBvbmx5IGJlY2F1c2Ugb2YgTGludXggY29tcGF0aWJsZSBwdXJw
b3NlLg0KPiANCj4gVGhlIHBhdGNoIGRlZmluZXMgX19mZnMoKSBpbiB0ZXJtcyBvZiBYZW4gYml0
b3BzIGFuZCBpdCBpcyBzYWZlDQo+IHRvIGRlZmluZSBpbiB0aGlzIHdheSAoIGFzIF9fZmZzKCkg
LSAxICkgYXMgY29uc2lkZXJpbmcgdGhhdCBfX2ZmcygpDQo+IHdhcyBkZWZpbmVkIGFzIF9fYnVp
bHRpbl9jdHpsKHgpLCB3aGljaCBoYXMgdW5kZWZpbmVkIGJlaGF2aW9yIHdoZW4geD0wLA0KPiBz
byBpdCBpcyBhc3N1bWVkIHRoYXQgc3VjaCBjYXNlcyBhcmUgbm90IGVuY291bnRlcmVkIGluIHRo
ZSBjdXJyZW50IGNvZGUuDQo+IA0KPiBUbyBub3QgaW5jbHVkZSA8eGVuL2xpbnV4LWNvbXBhdC5o
PiB0byBYZW4gbGlicmFyeSBmaWxlcyBfX2ZmcygpIGFuZCBfX2ZmeigpDQo+IHdlcmUgZGVmaW5l
ZCBsb2NhbGx5IGluIGZpbmQtbmV4dC1iaXQuYy4NCj4gDQo+IEV4Y2VwdCBfX2ZmcygpIHVzYWdl
IGluIGZpbmQtbmV4dC1iaXQuYyBvbmx5IG9uZSB1c2FnZSBvZiBfX2ZmcygpIGxlYXZlDQo+IGlu
IHNtbXUtdjMuYy4gSXQgc2VlbXMgdGhhdCBpdCBfX2ZmcyBjYW4gYmUgY2hhbmdlZCB0byBmZnNs
KHgpLTEgaW4NCj4gdGhpcyBmaWxlLCBidXQgdG8ga2VlcCBzbW11LXYzLmMgbG9va3MgY2xvc2Ug
dG8gbGludXggaXQgd2FzIGRlY2ljZWQganVzdA0KPiB0byBkZWZpbmUgX19mZnMoKSBpbiB4ZW4v
bGludXgtY29tcGF0LmggYW5kIGluY2x1ZGUgaXQgaW4gc21tdS12My5jDQo+IA0KPiBTaWduZWQt
b2ZmLWJ5OiBPbGVrc2lpIEt1cm9jaGtvIDxvbGVrc2lpLmt1cm9jaGtvQGdtYWlsLmNvbT4NCj4g
QWNrZWQtYnk6IFNoYXduIEFuYXN0YXNpbyA8c2FuYXN0YXNpb0ByYXB0b3JlbmdpbmVlcmluZy5j
b20+DQo+IFJldmlld2VkLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQoNCkZv
ciBTTU1VdjMgY2hhbmdlczoNCkFja2VkLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2luZ2hAYXJt
LmNvbT4NCg0KUmVnYXJkcywNClJhaHVsDQog


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:13:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:13:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722479.1126469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GK2-0000MT-Ej; Wed, 15 May 2024 15:13:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722479.1126469; Wed, 15 May 2024 15:13:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GK2-0000MM-AR; Wed, 15 May 2024 15:13:38 +0000
Received: by outflank-mailman (input) for mailman id 722479;
 Wed, 15 May 2024 15:13:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7GK1-0000MG-1G
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:13:37 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b360b3bf-12cd-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:13:35 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e0a34b2899so105228021fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:13:35 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01631sm871191666b.153.2024.05.15.08.13.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:13:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b360b3bf-12cd-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715786015; x=1716390815; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SVV2/hLcQw3aZJvTlD+Rpate6U19tAQsl4o8Ekmm/AM=;
        b=E/3JL3r8khZi8Y9wmIDPylOYY7rd8ofcbvXhKnyapFXLvVhYrmEkuxVsDdebpiPJWs
         YvHKaRgUVMiHctjgeynmB8BMn2f1UhYHY0+/QQzue8nUl56078NOEOtBclFbyCHzRCb/
         KX4mQ1qczbQUqifnc58dnR5xgWgPaLCFvoglu3GwC2xh9r7eah/LZfWMRY0ujkuRQ1wo
         dXswWnMbGLVd4f2Lb3XyEHgI7eTheCMSXsxnA8nD4i0QSjF8paWMIHdGzcSQnTdfLCrJ
         KrENmKKpT9C7wdLxBdrQ+GRIEFoRDzQPhVNRuKtMq9KUoxt+W2WV+KnTTrKa5KlYY7F2
         L3qA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786015; x=1716390815;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SVV2/hLcQw3aZJvTlD+Rpate6U19tAQsl4o8Ekmm/AM=;
        b=Ho3tdnGZ6RygaLcHFZsLH/WYJlOBwvtAy12QEjBGQLplZmTzqqN4UPaOGNXfgCbjg6
         e0xvrQMH46lwYryiWtWtau0MMVQIzKp/JYGszqrHli0A6HJ9LoXvTJfdcIJcgDTqjyPJ
         eVco9L1MOztLQ6oXW7NrWp6buSQyggudlSfxVdVtKvHwu+m8SXJTqWSZySxt3OERtwPz
         ZH0XK5IX6gZ2PAeKPi7lKHm0VCU83FJRh4jYC7opIriKutbggxLDsFZL9koRSMb3+tkW
         WqUvElyY9x5q0SSRkXft8yEy7is/abr8dS0RFmTWNiqq62tB1+rxtCyAoiG6vUo6SaI/
         +XkA==
X-Forwarded-Encrypted: i=1; AJvYcCXQiE/jr0PJ5P3QTDdTgyKoTYN3FBy9maxYr+IrCMNn/3NvaQ5lU9smNOktFQKsAvkYpdlNxbD61S9ZKiALAcqG88T0krDBUr/aZ4B0sTw=
X-Gm-Message-State: AOJu0YztI6cBaRw9VUS4mOG/qYy5TE6yTu2pkmCOP5Xg/eTxQl4TSbkX
	IWnMCF5cCMZP4qhaT7r/DGswoktJ/b14tzRKsevPqghxNEKVjrvZ+r3z6F/deA==
X-Google-Smtp-Source: AGHT+IG5Q+a+LL1E4Ytt9dDWDhz0WZk2ZmmTqV1Yfi8mTlThy/GSd/zlhU1M3sLvr+ZwiZfylYa/kQ==
X-Received: by 2002:ac2:4284:0:b0:51f:567b:c399 with SMTP id 2adb3069b0e04-52210277b79mr13825215e87.62.1715786015352;
        Wed, 15 May 2024 08:13:35 -0700 (PDT)
Message-ID: <f64f50f5-fc0d-465e-9904-db6352dad35d@suse.com>
Date: Wed, 15 May 2024 17:13:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 10/19] xen/page_alloc: Add a path for xenheap
 when there is no direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-11-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-11-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When there is not an always-mapped direct map, xenheap allocations need
> to be mapped and unmapped on-demand.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     I have left the call to map_pages_to_xen() and destroy_xen_mappings()
>     in the split heap for now. I am not entirely convinced this is necessary
>     because in that setup only the xenheap would be always mapped and
>     this doesn't contain any guest memory (aside the grant-table).
>     So map/unmapping for every allocation seems unnecessary.

Kind of depends on whether you expect guest data to potentially be copied
into memory that cam from the Xen heap, even if just transiently.

> @@ -2317,6 +2350,12 @@ void free_xenheap_pages(void *v, unsigned int order)
>      for ( i = 0; i < (1u << order); i++ )
>          pg[i].count_info &= ~PGC_xen_heap;
>  
> +    if ( !has_directmap() &&
> +         destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) )

PAGE_SIZE << order ?

Also, on x86 at least, to limit the impact of this, I think there wants to
be a prereq change to modify_xen_mappings() limiting the final flush_area()
there to just the one page altered, when it is just a single page that is
being fiddled with.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:22:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:22:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722485.1126478 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GSo-0002Ws-7m; Wed, 15 May 2024 15:22:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722485.1126478; Wed, 15 May 2024 15:22:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GSo-0002Wl-4l; Wed, 15 May 2024 15:22:42 +0000
Received: by outflank-mailman (input) for mailman id 722485;
 Wed, 15 May 2024 15:22:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7GSn-0002Wf-Fz
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:22:41 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f73a3c5a-12ce-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:22:39 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e4b90b03a9so77107631fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:22:39 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfc2fsm9186024a12.39.2024.05.15.08.22.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:22:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f73a3c5a-12ce-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715786559; x=1716391359; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=z90oHeb7BmjBCbCyQiMaLsBrK5r6Ef7VhFkt7CjQhXM=;
        b=f4DTcB2je5xJE+gMFG4D54osvPP3YbDcNq9u0d3C2w3BLkLfM4wbYD5CbkmTT6r9jx
         WNowqD/MfnWeKRuQUvRs7B4Cio4/1f/61jscx+nh/eMuWhDXYXDAr64e63OS6SBGSAXL
         a4OoUomjo7veNA9iFpeMJzxlZvvmSfRXNMClzrMUcJCy+1ycW2fx5czv2dIbXcaSxtRA
         Y84VAG7YVsllA1HpuWIdVTwGlkTZclphYmA9zPgS85JI1v+rH7AsZnFS8uUB7t6t0Sx/
         JjWgTis3kB+RPHnqbezDmTmo48qL0iOm3cYb9dcaj2eefTGhgXytG7YdaUyWiQ7XURoh
         UPzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786559; x=1716391359;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=z90oHeb7BmjBCbCyQiMaLsBrK5r6Ef7VhFkt7CjQhXM=;
        b=BOPg7TCqHK9nsPcMEQshxkcGZodofR56tSUzPA0WDm0vu1w7EauRw4nkRfBm+q4kYF
         JpGuwDU5/++5Ell89npLCMOy8ndGE1MvK70cIvFBdCm6tTacw4V8AjqZ0ZRCMSlr6eyQ
         lnYQSSsRBzEIoRhTzExIL8CRs8jnD7EgowSQ3wkvI67oo4fWeJEDl8dkEIXyWK+siWyy
         PXdFx6Uoi67WvOebxvh/8aQDlJf2YEN4B2MtMzAA44wH3UoHNiExU6CZtqmlq1r6fPTt
         cloFfkycUo/YgB4igzlXJ/OQlrxJulf034sO/ki1eTShEkUdtjefrTXwjyfX9ST0/rme
         Sd7Q==
X-Forwarded-Encrypted: i=1; AJvYcCW62BMrwkj4SmsuwNM7y3CZvM7jswlc9JtXTUYGdYx7vKI1BeN6rU/bGshpN11fLI+9vEX515z34BXqP+YuHE9Q5HQfhXBSW/lQLrhGjYg=
X-Gm-Message-State: AOJu0Yxx3sRRBAyZuKPPhGmeEPFt52He8oIrjs+MG7rkZcW+NkAvpWjU
	0/AbmpyO/hQIaiKB39BEJUGYnT1zJ7VNkdGvQlpVrLnchER9LtEvVo17Rfl8hA==
X-Google-Smtp-Source: AGHT+IGny/G5TgpmQnUjj2uWAuuhJMA91VyHHQYfBP9dBM1G/itLhJh0G66cvVRg+H+d1yuet/rB5g==
X-Received: by 2002:a19:e05a:0:b0:518:ce4b:17ef with SMTP id 2adb3069b0e04-5221006cdd3mr10961201e87.60.1715786558788;
        Wed, 15 May 2024 08:22:38 -0700 (PDT)
Message-ID: <bf49520a-691c-4876-a582-20a6745d4a56@suse.com>
Date: Wed, 15 May 2024 17:22:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 11/19] x86/setup: Leave early boot slightly
 earlier
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-12-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-12-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1751,6 +1751,22 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>  
>      numa_initmem_init(0, raw_max_page);
>  
> +    /*
> +     * When we do not have a direct map, memory for metadata of heap nodes in
> +     * init_node_heap() is allocated from xenheap, which needs to be mapped and
> +     * unmapped on demand. However, we cannot just take memory from the boot
> +     * allocator to create the PTEs while we are passing memory to the heap
> +     * allocator during end_boot_allocator().
> +     *
> +     * To solve this race, we need to leave early boot before
> +     * end_boot_allocator() so that Xen PTE pages are allocated from the heap
> +     * instead of the boot allocator. We can do this because the metadata for
> +     * the 1st node is statically allocated, and by the time we need memory to
> +     * create mappings for the 2nd node, we already have enough memory in the
> +     * heap allocator in the 1st node.
> +     */
> +    system_state = SYS_STATE_boot;
> +
>      if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) )
>      {
>          unsigned long lo = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
> @@ -1782,8 +1798,6 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>      else
>          end_boot_allocator();
>  
> -    system_state = SYS_STATE_boot;
> -
>      bsp_stack = cpu_alloc_stack(0);
>      if ( !bsp_stack )
>          panic("No memory for BSP stack\n");

I'm pretty wary of this movement, even more so when Arm isn't switched at
the same time. It has (virtually?) always been the case that this state
switch happens _after_ end_boot_allocator(), and I wouldn't be surprised
if there was a dependency on that somewhere. I realize you've been telling
use that at Amazon you've been running with an earlier variant of these
changes for a long time, and you not having hit issues with this is a good
sign. But I'm afraid it's not a proof.

As to possible alternatives - as pointed out by Roger, the comment / patch
description aren't entirely clear as to what exactly needs working around.
One possibility might be to introduce an x86-only boolean controlling from
when on to use the heap allocator for page table allocations, thus
decoupling that from system_state.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:23:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:23:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722488.1126488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GTz-00032W-IX; Wed, 15 May 2024 15:23:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722488.1126488; Wed, 15 May 2024 15:23:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GTz-00032P-E2; Wed, 15 May 2024 15:23:55 +0000
Received: by outflank-mailman (input) for mailman id 722488;
 Wed, 15 May 2024 15:23:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6wQf=MS=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s7GTy-00032F-R8
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:23:54 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 226ed3c2-12cf-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:23:52 +0200 (CEST)
Received: from BN8PR03CA0035.namprd03.prod.outlook.com (2603:10b6:408:94::48)
 by IA0PR12MB8350.namprd12.prod.outlook.com (2603:10b6:208:40d::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Wed, 15 May
 2024 15:23:49 +0000
Received: from BN1PEPF00004684.namprd03.prod.outlook.com
 (2603:10b6:408:94:cafe::86) by BN8PR03CA0035.outlook.office365.com
 (2603:10b6:408:94::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Wed, 15 May 2024 15:23:49 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN1PEPF00004684.mail.protection.outlook.com (10.167.243.90) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Wed, 15 May 2024 15:23:49 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 10:23:48 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 10:23:48 -0500
Received: from [172.18.62.51] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 15 May 2024 10:23:47 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 226ed3c2-12cf-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oJ9AAeVPJuQpISXGpV31MYInpFq4wjJPCjyadebfdJjk76uSrbdUmIMMJlIXHKsJsb8TZ1ym0csngMwljG/Lc+/bizMg6whtduY1PkcHNxH6Vxx7sp8f+oW6+P70dBSa7QhBEzid1PUUNJV3xaiJv/g+rhxv+uWt5l8Kc74rR948xM37JU3xDG0J3LuLRLnA5t5xx15IqE7cqvgCalE+RO1i9vWSXSr5NlepnHk5GsiR7fWv5z2pGRfteDKEezTXRTU3/+rwPOu5G1P7KQjl7E8Mlt8Q4PJVdjc5CeW/vVnIiW60MUjI66B4dWZakgWYNoGbQ3L33TICiwcz8g8RdA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UzRJOqZMNHQhajyVvCP+iCeSVE5aHh1KCTZRWHK+bPI=;
 b=S3DG30MLhZeauuuMi7lbvpVaYDZCwVuGJODU3VGfF/iQM95m+9CfQhFQSAXHLIijDZF/pJh83B9xWxvfoQIm1+A2l2SVCQ3awL6s2SCxLiRip+iZmbFJrg4fVY7wbpU6bjbmnIknrCKr4mBzJnTRAyHnwRVCSiva+Wl8Mh0R4SB6ejAjae7r8wl4+Rd5FQvNP/5/wrfplxFW34welByTpwKZFD1qob+iWxTI8BsGI8PIKiStlzew6g0CejuCssiZqHXg+fORo6DnD684Ty0fCkKp6nvFEpAAdowGKYSuLPTFeXnunP8OGRWPWYYG9byuUaNL2jdE5HJaKP9nRvxoBQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UzRJOqZMNHQhajyVvCP+iCeSVE5aHh1KCTZRWHK+bPI=;
 b=IPn/IjlsNejTVu2IWYmeTHwV5fhEWOVeE3XvudrZPffI+GX8DNyrs1sN74GK555a+zXRC0TzCUgilNLNv7SkVMOFhjCn7vvjssRsJThCh2oUwHXhfvoAb+2IyMVzA9wUCcc6wf7L0v2LWNw4rOM5ZnMv7MHZpOnhYZ9ifbueH50=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <21878244-b04d-4dfa-a8e2-347e9d40832e@amd.com>
Date: Wed, 15 May 2024 11:23:47 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 5/5] arm/vpci: honor access size when returning an
 error
To: Jan Beulich <jbeulich@suse.com>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>,
	<xen-devel@lists.xenproject.org>
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-6-stewart.hildebrand@amd.com>
 <cf646df7-0e92-4138-8b54-471c72f44cb8@xen.org>
 <735c3e3c-d70a-40f5-aca7-f134bc5b98b1@amd.com>
 <20335f55-4b16-45e5-a8aa-868006b57ca5@suse.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <20335f55-4b16-45e5-a8aa-868006b57ca5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN1PEPF00004684:EE_|IA0PR12MB8350:EE_
X-MS-Office365-Filtering-Correlation-Id: 6842ecd3-4543-4cd3-7cab-08dc74f30534
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?d3Q3UUNYbmxMVy9EUkVaaUJEVTlSanFyZ0hZM2tPTG1FU1VndzFwSGhOeUNy?=
 =?utf-8?B?Y3pIS2ZFdC9TcmViM2IybFlDaDBnUTY5SXFNV0xyYWlyZnBlcWhuVmxpcUNY?=
 =?utf-8?B?ZXAwU2d5b0w0Vy9OdXhObm56SzJzV1VCOFZqMWVGYVNjUk50QmZyRDBkSUZX?=
 =?utf-8?B?cXlrUzF5bjJ2Q2U1Ulliem85L3FJVzhkODlPTDE0WStLN1c1WmpLVWVDTVdX?=
 =?utf-8?B?NjVGclJIM043ZXQ4Rzc3ditWc0hUVlFwd3hXNUtCRmJ4ZGJyb3A1YmZpb3Zv?=
 =?utf-8?B?aUZCOFNsdldPOVExV3Y5ZVErRU9WQ0FoS2tNbHZoSXp6bEZSTWdjNHVLRzgz?=
 =?utf-8?B?UTRpQWhZdUtFb245Q2N0M2U5OEtwUzBjZUhFYTh5aiswcmoyaUVKNk5BTEFo?=
 =?utf-8?B?bzlwUWE4K0RrWlJsdWtLenBBaVZqb1pYY1JjVTNGZEdid0VOVU9OM3ZuK0p6?=
 =?utf-8?B?RVRyK0wwcFFEUkhTeGNBOEttZG1oN3FWYW9JQmN1U0pOOGhLa21ZY3BzYjJP?=
 =?utf-8?B?c2tiSDBoZFpaQUxyTTQycnR0U0ZoZXpvcU8yM01vYjhtTmF2Ky9rL0JYbWZw?=
 =?utf-8?B?SUc4dDR0WGlDekdIS3ltYjNwNVY4Rk1zaFFNQk16VHNpeCtWd1VoUWtGNlp6?=
 =?utf-8?B?S2pINy9mUW8xTjV6TU9TOE5XdWNjb3loaTNYU044b1d4ZHhQRENMVUVqZkFI?=
 =?utf-8?B?Qmp1Z2FUVDBjNDR4MlZYTkNYTDVFY3BXc3FPQnFzckxvZHVVa3RTRG16UVJG?=
 =?utf-8?B?V2IySnJxR21BVGRHc3piaURTdFd2M0lsQzZYUU5WRC8yaXJRVDdVK3E2R0lC?=
 =?utf-8?B?RHVrNVlhSVBwM01IcU1wUGxoK1BKeUZLdVFPNXo0dWtNMGNsK1RDVzNQSU9V?=
 =?utf-8?B?alJ3cTVWQ1Y5QjlxQUxqRWZyckNKb0NjSkY1eGxGNFZUNS8zckdyVUYwQ3ls?=
 =?utf-8?B?VkJCZ0hBdUV6MHUvYXpGbm1TRGxLdGtvLy9CaUp0TUxkV082RUtzTmhDclUy?=
 =?utf-8?B?eEdlRi9RQWVEbzhPR1EvN08rSkRHcnludk1uZXhRdmkzeTZ2azVUSnhocWxO?=
 =?utf-8?B?YVp6U3RieXBTZm52OXRjcU54dmZZek9kWVp2bFptRjdmbzBWaEcxTFRScFhF?=
 =?utf-8?B?dDcwanAxMWc4cHlPeHVCZHBDSytqaWl0bXlTbElOMmt3UnpXNE1SVDd3TW44?=
 =?utf-8?B?YVNNTUR6WktRV0Urc2VWaUhKdCtwbDluZDBUd05xcFAxdnhja3hOeGh4M0VI?=
 =?utf-8?B?NVFiVCt1Vmk0SUVSVkJGRnBrK2prWlRKUC9GRFh4MHZlUlYzdEpESytlZlVU?=
 =?utf-8?B?UC93d1QvTnpYZ3RkRHFmWDh0K25SMXlEZUxmZ3FEUWVSQWxGSnk1M2JrUUxv?=
 =?utf-8?B?MHlmaXQvekhFWlE3eWlNYklFaXNYMjU5b3VnblZIUHZYa01Xa28vU25jdHVK?=
 =?utf-8?B?MDdlek01emRicEpIV0VZZEhvT0h3ZjY2MXBqMHVHN0dVTXA4aVdTUXVWcTQz?=
 =?utf-8?B?UXBmWmowbytMaDNMNDNjOG9UY2hjZkR0U1Fka2wvK1lyTXplNnd0bWpHOWJC?=
 =?utf-8?B?dnhjcmxXS0Q1aldLaFl3dUFyRnd3RFdoZ3dkUm9oOXZLY0lCMHhQaTJUck9W?=
 =?utf-8?B?QUYzZDg1SE5DOUQ2MkFpL3VUVkkzMm5xN3FvRUdVa0kyYXdyakpaT0RxUnNY?=
 =?utf-8?B?QnIxNThpbW1FMERYMExSUERTMGpVUU4zcXljRjEwa0xVSlJXWnRNTCtJR3Bs?=
 =?utf-8?B?ZlJLc3U4bS9POWxhbG4rYjVLdThoNU5YRjI0Rk9vY2psbjEwTzI1c0taU2xs?=
 =?utf-8?B?NnRtRUhzOU44N2EvbHBKRXM4cWkrZkFXYzhGV1hxcFloS2Z6Ri9uMlNHVlNL?=
 =?utf-8?Q?NaVzvz9XC3Xpz?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 15:23:49.2656
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6842ecd3-4543-4cd3-7cab-08dc74f30534
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1PEPF00004684.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8350

On 5/15/24 02:32, Jan Beulich wrote:
> On 14.05.2024 22:31, Stewart Hildebrand wrote:
>> Here's what the patch ("arm/vpci: honor access size when returning an
>> error") now looks like based on staging:
>>
>> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
>> index 3bc4bb55082a..31e9e1d20751 100644
>> --- a/xen/arch/arm/vpci.c
>> +++ b/xen/arch/arm/vpci.c
>> @@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>>  {
>>      struct pci_host_bridge *bridge = p;
>>      pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
>> +    const uint8_t access_size = (1U << info->dabt.size) * 8;
> 
> And why exactly uint8_t here, rather than unsigned int? See ./CODING_STYLE.

I'll change to unsigned int.

> 
>> +    const uint64_t invalid = GENMASK_ULL(access_size - 1, 0);
> 
> I'm not entirely convinced of uint64_t here either, but I'd view this as
> more borderline than the uint8_t above. As per ...
> 
>> @@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>>          return 1;
>>      }
>>  
>> -    *r = ~0ul;
>> +    *r = invalid;
> 
> ... the original rhs here, unsigned long (or perhaps register_t) would seem
> more appropriate, but I have no idea whether on Arm32 info->dabt.size can
> end up being 3.

Well, it depends which spec we look at. In the ARMv7 ARM, 3 is reserved.
But in the ARMv8 ARM, in the aarch32 section, 3 appears to be valid...

Anyway, since the value returned in r can only be as wide as register_t
due to the way our mmio handlers are implemented, I'll change to
register_t for now to match the lhs.


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:25:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:25:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722491.1126497 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GVl-0003ac-SB; Wed, 15 May 2024 15:25:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722491.1126497; Wed, 15 May 2024 15:25:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GVl-0003aV-Pg; Wed, 15 May 2024 15:25:45 +0000
Received: by outflank-mailman (input) for mailman id 722491;
 Wed, 15 May 2024 15:25:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8u5V=MS=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s7GVk-0003aP-VP
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:25:44 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 65336009-12cf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:25:44 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2384333CBF;
 Wed, 15 May 2024 15:25:43 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E8227136A8;
 Wed, 15 May 2024 15:25:42 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /eAeN/bTRGaQZgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 15 May 2024 15:25:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65336009-12cf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715786743; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=j3CjhN97k6PpzDWO9RwH5pVj4WdhyqdEUtnghfTzbUA=;
	b=QZZ0E2eYd1pEz8IYgr65VMi7Io/yWe3pNagFN9Pr2b7x+n9MFPN2YQMceiqDtWpEfqYu38
	NdT8WqBdmBovXp2uGE5ZZ+MpnbBE9AfNwJJQduXPsPkMVJw3FClYGUwN1ukPo0zWrTVguI
	bPuJSXOXi6v9LOl/MmnHBmh3ZIPHUXY=
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=QZZ0E2eY
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1715786743; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=j3CjhN97k6PpzDWO9RwH5pVj4WdhyqdEUtnghfTzbUA=;
	b=QZZ0E2eYd1pEz8IYgr65VMi7Io/yWe3pNagFN9Pr2b7x+n9MFPN2YQMceiqDtWpEfqYu38
	NdT8WqBdmBovXp2uGE5ZZ+MpnbBE9AfNwJJQduXPsPkMVJw3FClYGUwN1ukPo0zWrTVguI
	bPuJSXOXi6v9LOl/MmnHBmh3ZIPHUXY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH] xen/sched: set all sched_resource data inside locked region for new cpu
Date: Wed, 15 May 2024 17:25:39 +0200
Message-Id: <20240515152539.18714-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Result: default: False [-2.01 / 50.00];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	MID_CONTAINS_FROM(1.00)[];
	R_MISSING_CHARSET(0.50)[];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.10)[text/plain];
	MX_GOOD(-0.01)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim,suse.com:email];
	FROM_EQ_ENVFROM(0.00)[];
	ARC_NA(0.00)[];
	FROM_HAS_DN(0.00)[];
	TO_DN_SOME(0.00)[];
	MIME_TRACE(0.00)[0:+];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	RCPT_COUNT_FIVE(0.00)[5];
	DKIM_TRACE(0.00)[suse.com:+]
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: 2384333CBF
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Flag: NO
X-Spam-Score: -2.01

When adding a cpu to a scheduler, set all data items of struct
sched_resource inside the locked region, as otherwise a race might
happen (e.g. when trying to access the cpupool of the cpu):

(XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
(XEN) CPU:    45
(XEN) RIP:    e008:[<ffff82d040244cbf>]
common/sched/credit.c#csched_load_balance+0x41/0x877
(XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
(XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
0000000000000000
(XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
ffff83103723d450
(XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8:
0000000000000000
(XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
0000000000000009
(XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
ffff831037253720
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
0000000000f526e0
(XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040244cbf>
(common/sched/credit.c#csched_load_balance+0x41/0x877):
(XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
8b 4e 28 48
<snip>
(XEN) Xen call trace:
(XEN)    [<ffff82d040244cbf>] R
common/sched/credit.c#csched_load_balance+0x41/0x877
(XEN)    [<ffff82d040245a18>] F
common/sched/credit.c#csched_schedule+0x36a/0x69f
(XEN)    [<ffff82d040252644>] F common/sched/core.c#do_schedule+0xe8/0x433
(XEN)    [<ffff82d0402572dd>] F common/sched/core.c#schedule+0x2e5/0x2f9
(XEN)    [<ffff82d040232f35>] F common/softirq.c#__do_softirq+0x94/0xbe
(XEN)    [<ffff82d040232fc8>] F do_softirq+0x13/0x15
(XEN)    [<ffff82d0403075ef>] F arch/x86/domain.c#idle_loop+0x92/0xe6

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: a8c6c623192e ("sched: clarify use cases of schedule_cpu_switch()")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/common/sched/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 0cb33831d2..babac7aad6 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -3176,6 +3176,8 @@ int schedule_cpu_add(unsigned int cpu, struct cpupool *c)
 
     sr->scheduler = new_ops;
     sr->sched_priv = ppriv;
+    sr->granularity = cpupool_get_granularity(c);
+    sr->cpupool = c;
 
     /*
      * Reroute the lock to the per pCPU lock as /last/ thing. In fact,
@@ -3188,8 +3190,6 @@ int schedule_cpu_add(unsigned int cpu, struct cpupool *c)
     /* _Not_ pcpu_schedule_unlock(): schedule_lock has changed! */
     spin_unlock_irqrestore(old_lock, flags);
 
-    sr->granularity = cpupool_get_granularity(c);
-    sr->cpupool = c;
     /* The  cpu is added to a pool, trigger it to go pick up some work */
     cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ);
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Wed May 15 15:27:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:27:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722495.1126508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GXB-0004CU-9Q; Wed, 15 May 2024 15:27:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722495.1126508; Wed, 15 May 2024 15:27:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GXB-0004CN-6r; Wed, 15 May 2024 15:27:13 +0000
Received: by outflank-mailman (input) for mailman id 722495;
 Wed, 15 May 2024 15:27:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7GX9-0004CH-TQ
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:27:11 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 98fd3008-12cf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:27:10 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-41fd5dc03easo43372415e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:27:10 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41ff5e3dcfbsm198590245e9.22.2024.05.15.08.27.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:27:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98fd3008-12cf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715786830; x=1716391630; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zQ4vCvZuXow6Eob5on1IQXVBYXzXKgL9qUx+au2x4to=;
        b=qAVUKfMVTlG3TZp/mUIBHfWmYh2JbmrD/vg5aBvDyTAut3aKHjqIr5c3xAFIDZIiam
         DjXk+LUSK2vTQmnQkS6d41Hqpmc3ssZ9cSNQ7ngvnftVO/J/7l2Na2jq2fogDeApD7TH
         541ZFcgs9Dby69fQO127oeJeqE2qRH5r7jF+o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786830; x=1716391630;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zQ4vCvZuXow6Eob5on1IQXVBYXzXKgL9qUx+au2x4to=;
        b=GwMICFl+JWbkuLOO3LmLu7gulhscio+z/T3NS2tVcBxoSjqHq18cFOm6gh0WjbpqhH
         4V3dnh4vpYhQ6Q/hk33JV6eJJNlglSJ0bEscVcDuz84tYzoF2fN5Qs60phTUVJ6Xo58a
         QKM7w45pxU6wspxG+XrnC89dCDdt8UWv3Tv1CDHrdboqdFTkqM99a9/zl8S+SWBnTK83
         lJ8bgxRtwazvB09Wh/P1EaYPyuuQmkVA6iDW5hFDv/M2Fz/GMvxY56GIqD8wk/jo899Z
         9yG8qsyGC8vjHkfPX79hdBkblFVu5/POoiiXvznV10rDVqdBB9ntkey2TK78qKT9vqNk
         kgWw==
X-Forwarded-Encrypted: i=1; AJvYcCUTurXfyNFRFCF8nETHh676ECEN2j4CawQ8/JM4XiWn+SH0/T1BE9GYmsLImmPx8QnWhm6NVqT67PZPSXS3sr/jefhPaU1PFK/HTEbzTXo=
X-Gm-Message-State: AOJu0Yxo7oFeZUUNFkHt4Q3+D3gD1EgcUZ8LcvA08T+pQ4YEzryfLH/i
	p3iEvU1n8YZJQhhiQdkkqcBWwXUV9XU6wHBJHGUZKlZ519WQAEO8Z46XCHznLT8=
X-Google-Smtp-Source: AGHT+IENSsO5GFJCHmzUKO5wr1M34sOLZdmmSZBkcxgHkoFCy7dZBdRQDo2DRz2nMngd3BgotYbvog==
X-Received: by 2002:a05:600c:4f06:b0:420:1094:65d with SMTP id 5b1f17b1804b1-42010940753mr98376135e9.12.1715786830173;
        Wed, 15 May 2024 08:27:10 -0700 (PDT)
Message-ID: <b04a9f8c-aed4-4672-a97a-22e5bdec51e5@citrix.com>
Date: Wed, 15 May 2024 16:27:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/sched: set all sched_resource data inside locked
 region for new cpu
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>
References: <20240515152539.18714-1-jgross@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240515152539.18714-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 4:25 pm, Juergen Gross wrote:
> When adding a cpu to a scheduler, set all data items of struct
> sched_resource inside the locked region, as otherwise a race might
> happen (e.g. when trying to access the cpupool of the cpu):
>
> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
> (XEN) CPU:    45
> (XEN) RIP:    e008:[<ffff82d040244cbf>]
> common/sched/credit.c#csched_load_balance+0x41/0x877
> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
> 0000000000000000
> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
> ffff83103723d450
> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8:
> 0000000000000000
> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
> 0000000000000009
> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
> ffff831037253720
> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
> 0000000000f526e0
> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
> 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d040244cbf>
> (common/sched/credit.c#csched_load_balance+0x41/0x877):
> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
> 8b 4e 28 48
> <snip>
> (XEN) Xen call trace:
> (XEN)    [<ffff82d040244cbf>] R
> common/sched/credit.c#csched_load_balance+0x41/0x877
> (XEN)    [<ffff82d040245a18>] F
> common/sched/credit.c#csched_schedule+0x36a/0x69f
> (XEN)    [<ffff82d040252644>] F common/sched/core.c#do_schedule+0xe8/0x433
> (XEN)    [<ffff82d0402572dd>] F common/sched/core.c#schedule+0x2e5/0x2f9
> (XEN)    [<ffff82d040232f35>] F common/softirq.c#__do_softirq+0x94/0xbe
> (XEN)    [<ffff82d040232fc8>] F do_softirq+0x13/0x15
> (XEN)    [<ffff82d0403075ef>] F arch/x86/domain.c#idle_loop+0x92/0xe6
>
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Fixes: a8c6c623192e ("sched: clarify use cases of schedule_cpu_switch()")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thankyou for the quick turnaround.  I'll fix the wrapping of the commit
message on commit.  Not sure what went wrong on my original outgoing email.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:28:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:28:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722500.1126517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GYE-0004iw-If; Wed, 15 May 2024 15:28:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722500.1126517; Wed, 15 May 2024 15:28:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GYE-0004ip-Fo; Wed, 15 May 2024 15:28:18 +0000
Received: by outflank-mailman (input) for mailman id 722500;
 Wed, 15 May 2024 15:28:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7GYC-0004fG-Gv
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:28:16 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bfd47dbb-12cf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:28:15 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51f45104ef0so7758449e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:28:15 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574f5a5b029sm1393481a12.7.2024.05.15.08.28.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:28:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfd47dbb-12cf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715786895; x=1716391695; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JrPsDs9ut17QqTfzK4d9EG7k8uvHNySSnR9FioeK9Dc=;
        b=OCPJy9BfH9OrW8WIlKVNuIw96uMTnxCj2ErBxFoQqkXZ++w7NiCMgS0LTwiHQ+uQON
         1wAkiIvu4bTf68WcKJI7znSwbPquxz832uQ9m7/LCkWo7QXz//TYvLR83fGTE+JaLcLQ
         g8Y2cOwA4wSjbi8IjEv+wVPESiJBFktca7cZhjUoBUuRJIgZ7B7c4HOzS/fO5hQYhc8d
         gcApV8Z0EcNDEcAsN3gO7UhCJzqVE2uJXQLcRX3Oi936A1B5R8+a/Tbgpp6G0dysPvdO
         tbpdyP3g3YVYCgfdcNYDhtE+iBPQm8csmED+IAVUrW4QW4Qq8WYXkHDsNhufgUqdvnBE
         ON1w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786895; x=1716391695;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JrPsDs9ut17QqTfzK4d9EG7k8uvHNySSnR9FioeK9Dc=;
        b=lIHArr+sWwpLqKgJzLctOQd6qQLQ19ZgJwJoEJDiOsNdfSg/fpTU1TJVO8GB2GZmuQ
         6wJzKToZ8iSAIo1KiA4KsPYTMAK74BOJYyxUIrQnG2W+A3kJ2PJaa3HW6fNrXMsEvP/L
         +Cqngnq3iCFLNlnIzxCBkX1ydSWRJjFWpsJeZ01UC90lEvEGMzrI749rAVsckwy356Lz
         RZg0VZvATFI6rOmKzkUS3kBN6YHNrLlANviKFpM1U9N3JADNnTuHbeA/6ME5973WvUsa
         EZA1D4N+ULTFrAaReDr6zyfVZwhdjBoV3IOe2lO3UY+etSmG9FT0XcASV0z5go72xoVH
         V2/Q==
X-Forwarded-Encrypted: i=1; AJvYcCXVN9oF/SlcPhSKD7nZZ2GrlTkUJrvDhfN+g8CZTaWMCB+6PhtmK2pCes/pNqhOu2l0joBhlgG8qxTYnAB9KkmZZik+oGPm0XDIcyQnd7M=
X-Gm-Message-State: AOJu0YyRrpY/cneo9QoC1Hg+Gyj3EhJznMwSw48AywnaNKIOdT8ulNa5
	3wgsc4fy/sz3/SSRxv9GyON9Gh8swkMicijUM19SDReN0c/Nch4VHaAKRnxmfQ==
X-Google-Smtp-Source: AGHT+IFKiO+Cpn8yXFaH5r/w0HgP53s3M2i0tnrgGY3i9D3/MguqmeomJHxMRdb6x9WgA4PlQhGG1A==
X-Received: by 2002:a05:6512:4889:b0:519:5e81:276d with SMTP id 2adb3069b0e04-5220fb776fcmr9789497e87.31.1715786895341;
        Wed, 15 May 2024 08:28:15 -0700 (PDT)
Message-ID: <5edd6d8a-f014-4e80-b7d9-d4748d44e671@suse.com>
Date: Wed, 15 May 2024 17:28:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 12/19] x86/setup: vmap heap nodes when they
 are outside the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-13-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-13-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> When we do not have a direct map, archs_mfn_in_direct_map() will always
> return false, thus init_node_heap() will allocate xenheap pages from an
> existing node for the metadata of a new node. This means that the
> metadata of a new node is in a different node, slowing down heap
> allocation.
> 
> Since we now have early vmap, vmap the metadata locally in the new node.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> ----
> 
>     Changes in v2:
>         * vmap_contig_pages() was renamed to vmap_contig()
>         * Fix indentation and coding style
> 
>     Changes from Hongyan's version:
>         * arch_mfn_in_direct_map() was renamed to
>           arch_mfns_in_direct_map()
>         * Use vmap_contig_pages() rather than __vmap(...).
>         * Add missing include (xen/vmap.h) so it compiles on Arm
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index dfb2c05322..3c0909f333 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c

Just one thing on top of what Roger has said: This being the only file
that is touched, why "x86/setup:" as the subject prefix. It'll misguide
people to assume x86 code is what is (mainly) being touched, unless they
actually look into and scroll through the patch.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:29:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:29:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722503.1126528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZT-0005HA-TT; Wed, 15 May 2024 15:29:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722503.1126528; Wed, 15 May 2024 15:29:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZT-0005Gs-Qg; Wed, 15 May 2024 15:29:35 +0000
Received: by outflank-mailman (input) for mailman id 722503;
 Wed, 15 May 2024 15:29:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7GZS-0005FX-Rc
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:29:34 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id edb054f3-12cf-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:29:32 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51ff65b1e14so7777988e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:29:32 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52399dcb951sm245275e87.268.2024.05.15.08.29.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 08:29:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: edb054f3-12cf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715786972; x=1716391772; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=LLYV9Be1Dq+oNQmVMaP7Vc1g1afNBBpOJb9dBHJL8kY=;
        b=C3t8wYMR9vSEY7rKcDu9/tQJ+FztkFyxZiPgx7m2BKK5qa5KftA3rsc9Is8F4S9I35
         irnwT6I5kyeO/k2xhHt4e3ZGcb1y/jzHy9dgo/zqsuH7Nj5jiDuwmr6lrxynfiseVtvz
         xlzjfi2JpB2Wwyc0OZ1VrTm3IKFbUik8YM6sKVLFcJL3IdKUFTZP676ZGs3kdGQ8lPbb
         L5qLP0AiOzVNSjDmnGukc4arwEh+TInRxIqI2q7Ofa/ju9mt2qsKXO0pRrueg2Ij4I5Q
         8+kmHs+RZexi7v1BT6AlNgmnE3La/EFBKi10Gq86v9TdVum+30Et51BFHdvboTO7595q
         Rkbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786972; x=1716391772;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LLYV9Be1Dq+oNQmVMaP7Vc1g1afNBBpOJb9dBHJL8kY=;
        b=XZx450ZSH8hN70QfV4dTL2pl4AmtgcWcXImcUct64jZXNnVscyXnfjp8WivKQ+9VH3
         vj3p7eDwKJVfiJbIpd8IJjB5RqyiTGBu1RwBgUMY+ON/dekXANqamw5bS90Uh/9911zE
         52kmbqwIA4lXQzCLEbi5o3ushSzi0QpgXDrwFm4AzGsm3lRaD3CZNvLy6rysB7fs5fiW
         kmhUiTXgO32sXshx1/imj3G6ZiHQHLggfSAyuOfGDJoI3ov7U2WZKmmcg0v2wjdpsY2d
         HZvbWOJePq1DDNTTUec0ZGJAASxS35UZ3CfY180ob8RxQ+s1xrCnnyFjARmU2hfAD4Ns
         P1xg==
X-Forwarded-Encrypted: i=1; AJvYcCVOcSpCjFqFPNKF5GK+Iwtcn0gnLKhOZWXQ4/VnzETOLiflL7D2+nUobd1eb2SewKvs78EFpz5/0EQgyU4SKcavvLklEn9ZEgsPtu4XEqM=
X-Gm-Message-State: AOJu0YxEVJnEjT5EYwwTaNkWDydPhx4qt+A7hKBQGjRNAanbE0IQHurY
	mXgJbTZ2YMAZflIaSltpqPTeXn3X53yocdmehv3o6yRZuev93bKU
X-Google-Smtp-Source: AGHT+IFGaEcfXsayyUIGxtXeQkgmUtylapRpFir45HMr69F5BFFS80+s46v3s+vHu4GNEAjtaL85Aw==
X-Received: by 2002:a05:6512:3d19:b0:51a:b110:3214 with SMTP id 2adb3069b0e04-5221007029cmr12595614e87.49.1715786972064;
        Wed, 15 May 2024 08:29:32 -0700 (PDT)
Message-ID: <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 15 May 2024 17:29:30 +0200
In-Reply-To: <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
	 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 10:52 +0200, Jan Beulich wrote:
> On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > The following generic functions were introduced:
> > * test_bit
> > * generic__test_and_set_bit
> > * generic__test_and_clear_bit
> > * generic__test_and_change_bit
> >=20
> > Also, the patch introduces the following generics which are
> > used by the functions mentioned above:
> > * BITOP_BITS_PER_WORD
> > * BITOP_MASK
> > * BITOP_WORD
> > * BITOP_TYPE
> >=20
> > These functions and macros can be useful for architectures
> > that don't have corresponding arch-specific instructions.
>=20
> Logically this paragraph may better move ahead of the BITOP_* one.
>=20
> > Because of that x86 has the following check in the macros
> > test_bit(),
> > __test_and_set_bit(), __test_and_clear_bit(),
> > __test_and_change_bit():
> > =C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();
> > It was necessary to make bitop bad size check generic too, so
> > arch_check_bitop_size() was introduced.
>=20
> Not anymore, with the most recent adjustments? There's nothing arch-
> specific anymore in the checking.
>=20
> > @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr,
> > volatile void *addr)
> > =C2=A0 * If two examples of this operation race, one can appear to
> > succeed
> > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesses wi=
th a
> > lock.
> > =C2=A0 */
> > -static inline int __test_and_set_bit(int nr, void *addr)
> > +static inline int arch__test_and_set_bit(int nr, volatile void
> > *addr)
>=20
> I think I raised this point before: Why arch__ here, ...
>=20
> > @@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr,
> > volatile void *addr)
> > =C2=A0 * If two examples of this operation race, one can appear to
> > succeed
> > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesses wi=
th a
> > lock.
> > =C2=A0 */
> > -static inline int __test_and_clear_bit(int nr, void *addr)
> > +static inline int arch__test_and_clear_bit(int nr, volatile void
> > *addr)
>=20
> ... here, ...
>=20
> > @@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int
> > nr, void *addr)
> > =C2=A0
> > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > =C2=A0}
> > -#define __test_and_clear_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > -=C2=A0=C2=A0=C2=A0 __test_and_clear_bit(nr, addr);=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > -})
> > +#define arch__test_and_clear_bit arch__test_and_clear_bit
> > =C2=A0
> > =C2=A0/* WARNING: non atomic and it can be reordered! */
> > -static inline int __test_and_change_bit(int nr, void *addr)
> > +static inline int arch__test_and_change_bit(int nr, volatile void
> > *addr)
>=20
> ... and here, while ...
>=20
> > @@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr,
> > const volatile void *addr)
> > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > =C2=A0}
> > =C2=A0
> > -#define test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > +#define arch_test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 \
>=20
> ... just arch_ here? I don't like the double underscore infixes very
> much, but I'm okay with them as long as they're applied consistently.

Common code and x86 use __test_and_clear_bit(), and this patch provides
a generic version of __test_and_clear_bit(). To emphasize that
generic__test_and_clear_bit() is a common implementation of
__test_and_clear_bit(), the double underscore was retained. Also,
test_and_clear_bit() exists and if one day it will be needed to provide
a generic version of it, then it will be needed to have
generic__test_and_clear_bit() and generic_test_and_clear_bit()

A similar logic was chosen for test_bit.


>=20
> > --- a/xen/include/xen/bitops.h
> > +++ b/xen/include/xen/bitops.h
> > @@ -65,10 +65,144 @@ static inline int generic_flsl(unsigned long
> > x)
> > =C2=A0 * scope
> > =C2=A0 */
> > =C2=A0
> > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > BITOP_BITS_PER_WORD))
> > +
> > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > +
> > +extern void __bitop_bad_size(void);
> > +
> > =C2=A0/* --------------------- Please tidy above here -----------------=
-
> > --- */
> > =C2=A0
> > =C2=A0#include <asm/bitops.h>
> > =C2=A0
> > +#ifndef arch_check_bitop_size
> > +
> > +#define bitop_bad_size(addr) sizeof(*(addr)) <
> > sizeof(bitop_uint_t)
>=20
> Nit: Missing parentheses around the whole expression.
>=20
> > +#define arch_check_bitop_size(addr) \
> > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();
>=20
> Apart from the arch_ prefix that now wants dropping, this macro (if
> we
> want one in the first place)=C2=A0
What do you mean by 'want' here? I thought it is pretty necessary from
safety point of view to have this check.
Except arch_ prefix does it make sense to drop "#ifndef
arch_check_bitop_size" around this macros?

> also wants writing in a way such that it
> can be used as a normal expression, without double semicolons or
> other
> anomalies (potentially) resulting at use sites. E.g.
>=20
> #define check_bitop_size(addr) do { \
> =C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) )=C2=A0=C2=A0=C2=A0=C2=A0 \
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __bitop_bad_size();=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> } while ( 0 )
>=20
> > +#endif /* arch_check_bitop_size */
> > +
> > +/**
> > + * generic__test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
> > +static always_inline bool
> > +generic__test_and_set_bit(unsigned long nr, volatile void *addr)
>=20
> The original per-arch functions all use "int" for their first
> parameter.
> Here you use unsigned long, without any mention in the description of
> the
> potential behavioral change. Which is even worse given that then x86
> ends
> up inconsistent with Arm and PPC in this regard, by ...
>=20
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +
> > +/**
> > + * generic__test_and_clear_bit - Clear a bit and return its old
> > value
> > + * @nr: Bit to clear
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
> > +static always_inline bool
> > +generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +
> > +/* WARNING: non atomic and it can be reordered! */
> > +static always_inline bool
> > +generic__test_and_change_bit(unsigned long nr, volatile void
> > *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +/**
> > + * generic_test_bit - Determine whether a bit is set
> > + * @nr: bit number to test
> > + * @addr: Address to start counting from
> > + */
> > +static always_inline bool generic_test_bit(int nr, const volatile
> > void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +
> > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > +}
> > +
> > +static always_inline bool
> > +__test_and_set_bit(unsigned long nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_set_bit
> > +#define arch__test_and_set_bit generic__test_and_set_bit
> > +#endif
> > +
> > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
>=20
> ... silently truncating and sign-converting nr here.
Missed that fact. AFAIU there is no specific reason for bit number to
be 'int' for this function, so it makes sense to update x86's prototype
of arch__test_and_set_bit() to:
   static inline int arch__test_and_set_bit(unsigned long nr, volatile
   void *addr).
  =20
But probably I can't use 'unsigned long' here too, as it should a
compilation error around 'btsl' instruction.

So it can be or 'unsinged int' or 'int'.
>=20
> As to generic_test_bit() - please don't cast away const-ness there.
>=20
> > +}
> > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 arch_check_bitop_size(addr);=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> > +})
> > +
> > +static always_inline bool
> > +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
>=20
> Oddly enough here at least you use bitop_uint_t, but that's still ...
>=20
> > +{
> > +#ifndef arch__test_and_clear_bit
> > +#define arch__test_and_clear_bit generic__test_and_clear_bit
> > +#endif
> > +
> > +=C2=A0=C2=A0=C2=A0 return arch__test_and_clear_bit(nr, addr);
>=20
> ... meaning a signedness conversion on x86 then. And beware: You
> can't
> simply change x86'es code to use bitop_uint_t. The underlying insns
> used
> interpret the bit position as a signed number, i.e. permitting
> accesses
> below the incoming pointer (whether it really makes sense to be that
> way
> is a separate question). I'm afraid I have no good suggestion how to
> deal
> with that: Any approach I can think of is either detrimental to the
> generic implementation or would have unwanted effects on the x86 one.
> Others, anyone?
Is the signedness conversion here a big problem? I suppose no one will
pass a negative number to nr.

It seems to me that it would be better to use 'int' everywhere and not
use bitop_uint_t for 'nr' since it is just a bit position. 'Int'
provides enough range for possible bit number.

>=20
> > --- a/xen/include/xen/types.h
> > +++ b/xen/include/xen/types.h
> > @@ -64,6 +64,12 @@ typedef __u64 __be64;
> > =C2=A0
> > =C2=A0typedef unsigned int __attribute__((__mode__(__pointer__)))
> > uintptr_t;
> > =C2=A0
> > +#ifndef BITOP_TYPE
> > +#define BITOP_BITS_PER_WORD 32
> > +
> > +typedef uint32_t bitop_uint_t;
> > +#endif
>=20
> I think you mentioned to me before why this needs to live here, not
> in
> xen/bitops.h. Yet I don't recall the reason, and the description
> (hint,
> hint) doesn't say anything either.
If I remember correctly ( after this phrase I think I have to update
the description ) the reason was that if I put that to xen/bitops.h:

    ...
    #include <asm/bitops.h>
   =20
    #ifndef BITOP_TYPE
    #define BITOP_BITS_PER_WORD 32
    /* typedef uint32_t bitop_uint_t; */
    #endif
    ...

Then we will have an issue that we can't use the generic definition of=20
BITOP_BITS_PER_WORD and bitop_uint_t in asm/bitops.h as it is defined
after inclusion of <asm/bitops.h>.

But if to put it to <xen/types.h> then such problem won't occur.

If it still makes sense, then I'll update the description in the next
patch version.

~ Oleksii

>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Wed May 15 15:29:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:29:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722504.1126538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZd-0005ZQ-8P; Wed, 15 May 2024 15:29:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722504.1126538; Wed, 15 May 2024 15:29:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZd-0005ZJ-5X; Wed, 15 May 2024 15:29:45 +0000
Received: by outflank-mailman (input) for mailman id 722504;
 Wed, 15 May 2024 15:29:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cqhY=MS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7GZc-0005Yo-63
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:29:44 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f39ccf4d-12cf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:29:43 +0200 (CEST)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-792bcfde2baso594194085a.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:29:43 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf30f93fsm688476085a.102.2024.05.15.08.29.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 08:29:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f39ccf4d-12cf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715786982; x=1716391782; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=3sxGkmPVD5HLFrwQ1yQahTr4XsqgTq5I3qCRTvh/WaE=;
        b=Uwk+wrDrXKT7wuf5wQ8lhJMT7/SXTLwH+9ko2OA/VX5bG3I8ns7k3ndeggdr6rI1Mi
         wVH6SOXMMk9KuG0jL3u6yjgyIfbJyzy1MF/SO6s8kcrhh6/77XxTRp26EiTMdo7B3060
         8PocyYRPbJlX8vAP4V6JeJoDDj/nNZWLEufEQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715786982; x=1716391782;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=3sxGkmPVD5HLFrwQ1yQahTr4XsqgTq5I3qCRTvh/WaE=;
        b=h27vKPreQTxfzGwxZ9vW5QZmvFop1wu3MDYb12mWD+Mry1/MjjieS3BMc8DwohzGuo
         egZIGMnyg62jnFmP/IIipoq4TYmMhx1Ui7qADZEG+jdLgleO6vqs1oEEi6gRDa1pAYBI
         7B0mY0nHI1OYRqVlefhF88sGrQQpDIGHhv1MfL5cuuboyOUF/14Po1hrRW/9gXPn+jAP
         isgVriRDgnFPWQuolFRYHI1UXuobcOnk9Z8vkd0OxOxd8aCI3AR+ZCimKOvxH3BY+xVk
         vkDfV3JE43LheNoqKKq+qxr1PfG7YlJKwgWIUeJQ0eGah7jR+NQK2LwJZsBKCi3a74RE
         Jv2A==
X-Gm-Message-State: AOJu0YyLmm/psvmrrikDP368eHiqF4sKDtEJ5zBK16zBwuxFcJhg0xbM
	FVck2jAWGPcDFf3Cj6oB4k5PHED6+zeJ9a6rRyQfbHxse4abQ2f+tIw/fjhaRvJvZX31A/z4s7W
	0
X-Google-Smtp-Source: AGHT+IHrUfB3FnVXKRvNxQUkNyVWWW7N5YXwvqj3UY+11WN1ZeTw0FIO4srwbU/vABbkqlaVLEcTUw==
X-Received: by 2002:a05:620a:4393:b0:792:e8ab:9826 with SMTP id af79cd13be357-792e8ab983emr773594885a.62.1715786981746;
        Wed, 15 May 2024 08:29:41 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity masks
Date: Wed, 15 May 2024 17:29:25 +0200
Message-ID: <20240515152925.77197-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Print the CPU affinity masks as numeric ranges instead of plain hexadecimal
bitfields.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/irq.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 80ba8d9fe912..3b951d81bd6d 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1934,10 +1934,10 @@ void do_IRQ(struct cpu_user_regs *regs)
                 if ( ~irq < nr_irqs && irq_desc_initialized(desc) )
                 {
                     spin_lock(&desc->lock);
-                    printk("IRQ%d a=%04lx[%04lx,%04lx] v=%02x[%02x] t=%s s=%08x\n",
-                           ~irq, *cpumask_bits(desc->affinity),
-                           *cpumask_bits(desc->arch.cpu_mask),
-                           *cpumask_bits(desc->arch.old_cpu_mask),
+                    printk("IRQ%d a={%*pbl}[{%*pbl},{%*pbl}] v=%02x[%02x] t=%s s=%08x\n",
+                           ~irq, CPUMASK_PR(desc->affinity),
+                           CPUMASK_PR(desc->arch.cpu_mask),
+                           CPUMASK_PR(desc->arch.old_cpu_mask),
                            desc->arch.vector, desc->arch.old_vector,
                            desc->handler->typename, desc->status);
                     spin_unlock(&desc->lock);
@@ -2638,7 +2638,7 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
         if ( !set_affinity )
             printk("Cannot set affinity for IRQ%u\n", irq);
         else if ( break_affinity )
-            printk("Broke affinity for IRQ%u, new: %*pb\n",
+            printk("Broke affinity for IRQ%u, new: {%*pbl}\n",
                    irq, CPUMASK_PR(affinity));
     }
 
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 15 15:29:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:29:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722505.1126548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZi-0005rp-GM; Wed, 15 May 2024 15:29:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722505.1126548; Wed, 15 May 2024 15:29:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GZi-0005ri-CV; Wed, 15 May 2024 15:29:50 +0000
Received: by outflank-mailman (input) for mailman id 722505;
 Wed, 15 May 2024 15:29:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1aeg=MS=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7GZg-0005Yo-1q
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:29:48 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f59a0cc7-12cf-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:29:46 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 86BB580068;
 Wed, 15 May 2024 16:29:45 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f59a0cc7-12cf-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715786985;
	bh=nja7hs+1Fl4A9h1PMkBe/mv1CUxj2LF9liD70bT+zlA=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=OS9z4LABvhs1zpA3tPsN8Pv0YXdJsg8wiLF9/ooaiaDBYxf6EzvgU3zBvA1xl4KVG
	 n3PR8+/Nfj9B7FtuWKAAGMSjpPEGb10inZSjKFhHSTKObMsBCBmfYopj0gQPyInaSg
	 iKPWYINh7cVZ2YZSDtYZt0/yqV1eO4sYBv3CRWds=
MIME-Version: 1.0
Date: Wed, 15 May 2024 16:29:45 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
 anthony.perard@citrix.com
Subject: Re: [RFC PATCH v2 3/5] tools/hotplug/Linux: Add bridge VLAN support
In-Reply-To: <CAKf6xpuNJbf2ioVFyvu2L6YG2_eNRbqPF8tn7Yodyv5UA4uE6A@mail.gmail.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
 <20240508213824.55824-4-leigh@solinno.co.uk>
 <CAKf6xpuNJbf2ioVFyvu2L6YG2_eNRbqPF8tn7Yodyv5UA4uE6A@mail.gmail.com>
Message-ID: <f7a94db1deb1c3197d1f77e8e774ee53@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Jason,

On 2024-05-15 01:57, Jason Andryuk wrote:
> On Wed, May 8, 2024 at 6:55 PM Leigh Brown <leigh@solinno.co.uk> wrote:
>> 
>> Update add_to_bridge shell function to read the vlan parameter
>> from xenstore and set the bridge VLAN configuration for the VID.
>> 
>> Add additional helper functions to parse the vlan specification,
>> which consists of one or more of the follow:
>> 
>> a) single VLAN (e.g. 10).
>> b) contiguous range of VLANs (e.g. 10-15).
>> c) discontiguous range with base, increment and count
>>    (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).
>> 
>> A single VLAN can be suffixed with "p" to indicate the PVID, or
>> "u" to indicate untagged. A range of VLANs can be suffixed with
>> "u" to indicate untagged.  A complex example would be:
>> 
>>    vlan=1p/10-15/20-25u
>> 
>> This capability only works when using the iproute2 bridge command,
>> so a warning is issued if the vlan parameter is set and the bridge
>> command is not available, as it will be ignored.
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> ---
>>  tools/hotplug/Linux/xen-network-common.sh | 111 
>> ++++++++++++++++++++++
>>  1 file changed, 111 insertions(+)
>> 
>> diff --git a/tools/hotplug/Linux/xen-network-common.sh 
>> b/tools/hotplug/Linux/xen-network-common.sh
>> index 42fa704e8d..d9fb4f7355 100644
>> --- a/tools/hotplug/Linux/xen-network-common.sh
>> +++ b/tools/hotplug/Linux/xen-network-common.sh
>> @@ -121,10 +121,113 @@ create_bridge () {
>>      fi
>>  }
>> 
>> +_vif_vlan_add() {
>> +    # References vlans, pvid and error variables from the calling 
>> function
>> +    local -i vid=$1
>> +    local flag=${2:-}
>> +
>> +    if (( vid < 1 || vid > 4094 )) ;then
>> +        error="vlan id $vid not between 1 and 4094"
>> +        return
>> +    fi
>> +    if [[ -n "${vlans[$vid]}" ]] ;then
>> +        error="vlan id $vid specified more than once"
>> +        return
> 
> You could do `fatal "vlan id $vid specified more than once"` and just
> terminate the script at this point.  It would simplify your later code
> if you use fatal more.

I will do that.

>> +    fi
>> +    case $flag in
>> +     p) if (( pvid != 0 )) ;then
>> +            error="more than one pvid specified ($vid and $pvid)"
>> +            return
>> +        fi
>> +        pvid=$vid
>> +        vlans[$vid]=p ;;
>> +     u) vlans[$vid]=u ;;
>> +     *) vlans[$vid]=t ;;
>> +    esac
>> +}
>> +
>> +_vif_vlan_parse_term() {
>> +    # References error variable from the calling function
>> +    local vid incr last term=${1:-}
>> +
>> +    if [[ $term =~ ^([0-9]+)([pu])?$ ]] ;then
> 
> I like that you split the different cases into multiple REs.
> 
>> +        _vif_vlan_add ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}
>> +    elif [[ $term =~ ^([0-9]+)-([0-9]+)(u)?$ ]] ;then
>> +        vid=${BASH_REMATCH[1]}
>> +        last=${BASH_REMATCH[2]}
>> +        if (( last >= vid )) ;then
>> +            for (( ; vid<=last; vid++ )) ;do
>> +                _vif_vlan_add $vid ${BASH_REMATCH[3]}
>> +            done
>> +        else
>> +           error="invalid vlan id range: $term"
>> +        fi
>> +    elif [[ $term =~ ^([0-9]+)\+([0-9]+)x([0-9]+)(u)?$ ]] ;then
>> +        vid=${BASH_REMATCH[1]}
>> +        incr=${BASH_REMATCH[2]}
>> +        for (( j=${BASH_REMATCH[3]}; j>0; --j, vid+=incr ))
>> +        do
>> +            _vif_vlan_add $vid ${BASH_REMATCH[4]}
>> +        done
>> +    else
>> +        error="invalid vlan specification: $term"
>> +    fi
>> +}
>> +
>> +_vif_vlan_validate_pvid() {
>> +    # References vlans and pvid variables from the calling function
>> +    if (( pvid == 0 )) ;then
>> +        if (( ${#vlans[@]} == 1 )) ;then
>> +            vlans[${!vlans[*]}]=p
>> +        else
>> +            error="pvid required for multiple vlan ids"
>> +        fi
>> +    fi
>> +}
>> +
>> +_vif_vlan_setup() {
>> +    # References vlans and dev variable from the calling function
>> +    local vid cmd
>> +
>> +    bridge vlan del dev "$dev" vid 1
>> +    for vid in ${!vlans[@]} ;do
>> +        cmd="bridge vlan add dev '$dev' vid $vid"
>> +        case ${vlans[$vid]} in
>> +             p) cmd="$cmd pvid untagged" ;;
>> +             u) cmd="$cmd untagged" ;;
>> +             t) ;;
>> +        esac
>> +        eval "$cmd"
>> +    done
>> +}
>> +
>> +_vif_vlan_membership() {
>> +    # The vlans, pvid, dev and error variables are used by 
>> sub-functions
>> +    local -A vlans=()
>> +    local -a terms=()
>> +    local -i i pvid=0
>> +    local dev=$1 error=""
>> +
>> +    # Split the vlan specification string into its terms
>> +    readarray -d / -t terms <<<$2
>> +    for (( i=0; i<${#terms[@]}; ++i )) ;do
>> +        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}
>> +        [[ -n "$error" ]] && break
>> +    done
>> +
>> +    [[ -z "$error" ]] && _vif_vlan_validate_pvid
>> +    [[ -z "$error" ]] && _vif_vlan_setup
>> +    [[ -z "$error" ]] && return 0
>> +
>> +    log error "$error"
>> +    return 1
>> +}
>> +
>>  # Usage: add_to_bridge bridge dev
>>  add_to_bridge () {
>>      local bridge=$1
>>      local dev=$2
>> +    local vlan=$(xenstore_read_default "$XENBUS_PATH/vlan" "")
>> 
>>      # Don't add $dev to $bridge if it's already on the bridge.
>>      if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
>> @@ -134,6 +237,14 @@ add_to_bridge () {
>>          else
>>              ip link set ${dev} master ${bridge}
>>          fi
>> +        if [ -n "${vlan}" ] ;then
>> +            if which bridge >&/dev/null; then
>> +                log debug "configuring VLANs for ${dev} on ${bridge}"
>> +                _vif_vlan_membership "${dev}" "${vlan}"
>> +            else
>> +                log warning "bridge command not available, ignoring 
>> vlan config"
> 
> Do you think this should be fatal?  It seems to me that setting up the
> connection but not applying the vlans could be a security issue.

Agreed, I will do that.

> This file before your patch was very close to posix sh.  Afterwards,
> it definitely needs bash.  vif-bridge is /bin/bash, so it is fine.

Great - I would not like to rewrite it in Posix shell!

> Regards,
> Jason

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:30:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:30:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722512.1126558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GaV-0007pL-Oa; Wed, 15 May 2024 15:30:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722512.1126558; Wed, 15 May 2024 15:30:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GaV-0007pE-Kv; Wed, 15 May 2024 15:30:39 +0000
Received: by outflank-mailman (input) for mailman id 722512;
 Wed, 15 May 2024 15:30:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1aeg=MS=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7GaT-0005FX-La
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:30:37 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 12d56de2-12d0-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:30:35 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id E2A9880068;
 Wed, 15 May 2024 16:30:34 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12d56de2-12d0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715787034;
	bh=9h8ymMqtRs462F/ihippUsQtICketsWkIzl7UyFP8uA=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=2PKjIe7NNtR8Nzp83aPPRIuS2tSG96MmKGh2BPsJfSzy8VXHIEvfwR1nx59WzAzSh
	 P5MoHNwh5eOaoq2bHhWyKvIFtSidTjvdBN1DJOnuvdaUsq7QwubA8SRHDq3+dLXlBg
	 B6A3yhqaUrf48rJTrcH0s+U1UP8adIQtmPxpgCxc=
MIME-Version: 1.0
Date: Wed, 15 May 2024 16:30:34 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
 anthony.perard@citrix.com
Subject: Re: [RFC PATCH v2 4/5] docs/man: document VIF vlan keyword
In-Reply-To: <CAKf6xpviWyFxt4JMq23K0LxjDcP2zJF1Sqye=1jS2UAtwcaRww@mail.gmail.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
 <20240508213824.55824-5-leigh@solinno.co.uk>
 <CAKf6xpviWyFxt4JMq23K0LxjDcP2zJF1Sqye=1jS2UAtwcaRww@mail.gmail.com>
Message-ID: <c62b512c3b1ea58fb07e50df85b4ebd6@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Jason,

On 2024-05-15 01:57, Jason Andryuk wrote:
> On Wed, May 8, 2024 at 5:39 PM Leigh Brown <leigh@solinno.co.uk> wrote:
>> 
>> Document the new `vlan' keyword in xl-network-configuration(5).
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
> 
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> 
> One nit below
> 
>> ---
>>  docs/man/xl-network-configuration.5.pod.in | 38 
>> ++++++++++++++++++++++
>>  1 file changed, 38 insertions(+)
>> 
>> diff --git a/docs/man/xl-network-configuration.5.pod.in 
>> b/docs/man/xl-network-configuration.5.pod.in
>> index f3e379bcf8..c35c0922b3 100644
>> --- a/docs/man/xl-network-configuration.5.pod.in
>> +++ b/docs/man/xl-network-configuration.5.pod.in
>> @@ -259,6 +259,44 @@ Specifies the MTU (i.e. the maximum size of an IP 
>> payload, exclusing headers). T
> 
>> +Note, one VLAN ID must be marked as the PVID. In the case of a vlan
>> +specification consisting of a single VLAN ID (e.g. C<vlan=10>), the 
>> B<p> suffix
>> +may be omitted. Specifying more than one untagged VLAN ID is an 
>> advanced
>> +configuration - use with caution.
>> +
>> +For example:
>> +
>> +        'vlan=10' -- meaning a single VLAN that is the PVID.
>> +       'vlan=10p/20' -- VLAN 10 is the PVID and VLAN 20 is tagged.
>> +       'vlan=10p/100+10x4' -- VLANs 10, 100, 110, 120, 130, 140, 150.
> 
> Indent mismatch between 7 and 8 spaces.

FWIW I will sort that too :-)

>> +
>>  =head2 trusted / untrusted
>> 
>>  An advisory setting for the frontend driver on whether the backend 
>> should be
>> --
>> 2.39.2
>> 
>> 

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:30:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:30:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722513.1126568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gac-000878-UP; Wed, 15 May 2024 15:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722513.1126568; Wed, 15 May 2024 15:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gac-000871-Ra; Wed, 15 May 2024 15:30:46 +0000
Received: by outflank-mailman (input) for mailman id 722513;
 Wed, 15 May 2024 15:30:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7Gab-0005FX-40
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:30:45 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 17cfa1b2-12d0-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:30:43 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-4200ee78f33so34575625e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:30:43 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee9318sm237354295e9.30.2024.05.15.08.30.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:30:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17cfa1b2-12d0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715787043; x=1716391843; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zj8mPpsF8CehAc+fVyF9EYE50ZAsJsk+SsYK1uJQ4FM=;
        b=YvEXs/KDGKc4llWCqMo6Lupzd9/sFkMsoowbEaqXci5yucbLUcFSb23rga7X078jMQ
         tKnj93O1Xp3MAiAV78MlPEmeNV9DGp6b2sJNGnaTjxJhjqRAHi3n3YXZ/Lp/sfQ25ILH
         dbhpevIIlwgbmPmhT5eO8P5k0x61Ii89x10yA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715787043; x=1716391843;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zj8mPpsF8CehAc+fVyF9EYE50ZAsJsk+SsYK1uJQ4FM=;
        b=smnO9z1Rek7RsKrJJhSeFGnQvvny1YFe1aRTKv0pWzPFOed/iiPVLIOz34RPVCX82J
         gS+Yvsf1OSf8ElHSVX5iW+AWuJqmsvp/jASPd8NifSsyGmIfG98trbMLHAovXF/mSsaw
         Vidkxnhl/OSxIHqdS9CVpEAThsGMRWvytf0k+opyS3nSPBsLbGosxOdyuFVW12GZNgWN
         9zHXOF4yhWOz4m0uFyl4q71WAR9ppTr9/vsZxjQhi6aG/YZmFoL0Jeq2/KM0t8Gi0niV
         B8SzIBrqp/lr1Wfx8fvwMsc48x2EiTMzFlgGDQ7yNJlAM8URLMMRD/stsoWBzyMSNFdj
         q8qw==
X-Forwarded-Encrypted: i=1; AJvYcCX7/sKfZQzTLaGwBDGbkTbRD02OAo8vGQtWHnbzGV99CRDW9hFj9hTDRyMPVL/OVH3zrN+5eyIGstsuMZqSL3yhZLcwQMfct6WP7Kj7YtQ=
X-Gm-Message-State: AOJu0Yz88RKP0DK2wuui9eMDhkSOh1NCW7cco8wY5wz7VZx/2Sgpn1lS
	Y3YzGNtlDLW/M+NCBo5Zf03qO/yFza0lh0kwajpa7cExbX5ErStCYHgUn3fHyRs=
X-Google-Smtp-Source: AGHT+IGOHe96vAhRPQYH5kDRhrLOvIPw9ymnMGA3EEek5o311ivNoXXXig5K88Ec5qtya1lJInW5SA==
X-Received: by 2002:a05:600c:3b8c:b0:41c:258:9b70 with SMTP id 5b1f17b1804b1-41feac4913amr117990575e9.28.1715787043016;
        Wed, 15 May 2024 08:30:43 -0700 (PDT)
Message-ID: <c897b41d-2fa5-4db6-b591-60e15b646d6d@citrix.com>
Date: Wed, 15 May 2024 16:30:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity
 masks
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240515152925.77197-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240515152925.77197-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 4:29 pm, Roger Pau Monne wrote:
> Print the CPU affinity masks as numeric ranges instead of plain hexadecimal
> bitfields.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Ha - I was going to write exactly the same patch, but you beat me to it.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:32:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:32:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722522.1126578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gbt-0000Xj-8P; Wed, 15 May 2024 15:32:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722522.1126578; Wed, 15 May 2024 15:32:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gbt-0000Xc-40; Wed, 15 May 2024 15:32:05 +0000
Received: by outflank-mailman (input) for mailman id 722522;
 Wed, 15 May 2024 15:32:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7Gbr-0000V1-Q6
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:32:03 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 476b3fde-12d0-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 17:32:03 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-52389c1308dso1648071e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:32:03 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea65b5sm9252304a12.6.2024.05.15.08.31.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 08:32:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 476b3fde-12d0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715787123; x=1716391923; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=bV5W8qkpNA6NJqllCxpyCGe72hVUACqRwRXGeTpi5L4=;
        b=aQZvLU1/GKoY+BPaJuICWv5vUkkBrcuCfpccaEwHKV9BbhDPE/iwLt0kfnvoA2sBia
         idWE6giOQLDq3+53t4rc2t35uvFGoyA/PAqYuG83FwQYlukVdTGbvA0zz6S/QW0lbHvP
         OWveBVmK+/Mg8Uk52igMVCVvPGA4DIZAbwLoR76IYC6PjAM0bzqsTrw21MURgwcU9n7Z
         6kShFwXIkZHaA/2VOANfnUXPtTMvYNumH2Sso52J/N/zkvvpt/4CWdOWoJcDnW8FkG+v
         9hE9y08OJXr1d59aKWYwveRs5itJUyaDj2C2/VYC3Ubk8qw3u9PRtZ8wR/WQ22x92hWK
         8i7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715787123; x=1716391923;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bV5W8qkpNA6NJqllCxpyCGe72hVUACqRwRXGeTpi5L4=;
        b=NasNhn3laaXJGzy6/WvXtrPTOuFnx4hTJPLRRDKChX4xBYZ/NEn+u8uBIclj2IeuGe
         UXzo31+HPz9mVIdBYZ4a6gvFSBXTXPAq6a4mlpOqrVDs3KwVrL9PkIGxonZpORk7KCuz
         KrFi8sYHhXa9sI2BsVYPeqjx2Bf/nOjpmmQO9ZUYjM4WaHvNWh1JOh94wzAseDjHJ1uV
         9bl06U5orzglEZ1LoIuw74O1xdnoAPuV404Ep4od3bufX6KCp8kNWjmXAiz+qfwGTYbM
         eJz/KsTjRk0nTgow6pU+VGXaH8XtuP2O7qZdtyX2qApmsLwdBa9n0vWyqK++5wdFvtyu
         JPvg==
X-Forwarded-Encrypted: i=1; AJvYcCWDF1ggT2e/FPlS0hI3wwgjHtpqotT/6D641sTPEt8iOi97TxkxiS69vLfGl+fWvBPPj4iI8dUzbE7Iof9oFl3x8N65VdxDr7wZvtN6LN4=
X-Gm-Message-State: AOJu0YxZm1DU3rAXmPtF7ZfVk65gSEiJXYajmeY+YxmaSvPv5WSXKMJj
	+RSKWuyMoYfxt1mB2PSB02VO/VOt0dCYB2xCjwwnPDDA0AfzDXVR
X-Google-Smtp-Source: AGHT+IEjg2E4GcwP/o0Qm0xWsyOPhy2uqP93kEieiqj87h8jTpNzzb5D15fuvUH7eo8zjEHDyOeR0Q==
X-Received: by 2002:a05:6512:2149:b0:516:e8b1:9ed5 with SMTP id 2adb3069b0e04-52210279e56mr8863916e87.59.1715787120647;
        Wed, 15 May 2024 08:32:00 -0700 (PDT)
Message-ID: <2b5281df104e265211b35a1d8fdc556857426574.camel@gmail.com>
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Wed, 15 May 2024 17:31:59 +0200
In-Reply-To: <ea90fd21-1ec4-46b5-b157-810466b3f0ab@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
	 <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
	 <46fd5c698fc13ffa3e8e82ed507bcfa67a9ef819.camel@gmail.com>
	 <ea90fd21-1ec4-46b5-b157-810466b3f0ab@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 16:07 +0200, Jan Beulich wrote:
> On 15.05.2024 15:55, Oleksii K. wrote:
> > On Wed, 2024-05-15 at 11:09 +0200, Jan Beulich wrote:
> > > On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > > > Changes in V9:
> > > > =C2=A0- update return type of fls and flsl() to unsigned int to be
> > > > aligned with other
> > > > =C2=A0=C2=A0 bit ops.
> > >=20
> > > But this then needs carrying through to ...
> > >=20
> > > > --- a/xen/arch/arm/include/asm/arm64/bitops.h
> > > > +++ b/xen/arch/arm/include/asm/arm64/bitops.h
> > > > @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long
> > > > __ffs(unsigned long word)
> > > > =C2=A0 */
> > > > =C2=A0#define ffz(x)=C2=A0 __ffs(~(x))
> > > > =C2=A0
> > > > -static inline int flsl(unsigned long x)
> > > > +static inline int arch_flsl(unsigned long x)
> > >=20
> > > ... e.g. here. You don't want to introduce signed/unsigned
> > > mismatches.
> > Do you mean that generic flsl() uses 'unsigned int' as a return
> > type,
> > but arch_flsl continue to use 'int'?
>=20
> Yes.
>=20
> > > Also why do you keep "inline" here, while ...
> > >=20
> > > > --- a/xen/arch/x86/include/asm/bitops.h
> > > > +++ b/xen/arch/x86/include/asm/bitops.h
> > > > @@ -425,7 +425,7 @@ static always_inline unsigned int
> > > > arch_ffsl(unsigned long x)
> > > > =C2=A0 *
> > > > =C2=A0 * This is defined the same way as ffs.
> > > > =C2=A0 */
> > > > -static inline int flsl(unsigned long x)
> > > > +static always_inline int arch_flsl(unsigned long x)
> > >=20
> > > ... you switch to always_inline here?
> > Because Adnrew's patch with bitops.h for x86 changes to
> > always_inline,
> > so to be consistent, at least, for architecture.
>=20
> And why not extend this to Arm?
No specific reason, just wanted to do minimal amount of necessary
changes. I'll do that in the the next patch version.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:33:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:33:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722526.1126587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gda-0001Op-M7; Wed, 15 May 2024 15:33:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722526.1126587; Wed, 15 May 2024 15:33:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Gda-0001Oi-J8; Wed, 15 May 2024 15:33:50 +0000
Received: by outflank-mailman (input) for mailman id 722526;
 Wed, 15 May 2024 15:33:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cqhY=MS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7GdZ-0001Oc-C3
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:33:49 +0000
Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com
 [2607:f8b0:4864:20::f32])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8526bb73-12d0-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:33:47 +0200 (CEST)
Received: by mail-qv1-xf32.google.com with SMTP id
 6a1803df08f44-69b6c2e9ed9so33662166d6.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:33:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1797ebsm65188416d6.4.2024.05.15.08.33.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 08:33:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8526bb73-12d0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715787226; x=1716392026; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=52F00axdSTxZ9nhEnnOlzDw6Q1ldPywncpLvDK9T3SI=;
        b=TdWLZZRS0mrJDuoyuqe8u011l/QXi1v+WeHMD1gwQ43BsMwTIenjJ3V4f8K9JcEgR+
         Er5cuViE55XaZpr+r9s2HqQNtlFGOECImoBcbjBqlAJi37o88fdAlIW6cDjWjiNkV2pz
         baWH4Nr5xCkEkgeCwOtfaNATtYJ86210ySBaQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715787226; x=1716392026;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=52F00axdSTxZ9nhEnnOlzDw6Q1ldPywncpLvDK9T3SI=;
        b=MjY91L0ckaBgPKNlfjjx5bWmvITHi4pok9Zz7sLLDJie5V/Cz7awsSSZKH3oxL1Cad
         CTI/A+iuyxvOeFhQdbnkxw/IR7IMlC1wPgFLvy55xFyeJ9wvCAkPr4Ci1rfRK948Yv6l
         GMuAn/+ZRR8qRxENDVJKyNWwJVcHPy/XbKwm2de1Woj7B71UbhAF02yULrC+W+lSP296
         gI5mgIY+Yj+GbQURG46jEGty+MoYaSKIJGKfZvuw3p1XXEhnu2N9YvWsvvXgIV53JcN/
         m/eweYWm3ECpEPTnHcjC/i7yoZa/XcYIGnr4+W9VR2h3HcEUzl6BUw93bSBXpT1gq+De
         DTmA==
X-Gm-Message-State: AOJu0YxMpbgc2f5aIXTRUj++rHbiPnB3JPgaHsJe9OoErAb7QQBkMgfJ
	IEyTqTunifjxbMlLWOA8Mn+vKffPCWZwlBFKXaUS3Vh48LeZz1+RbMWXPa1c44o=
X-Google-Smtp-Source: AGHT+IEBw2r1+DksohsjXZ4YRUULUzuLdfIQ3igz6CNdMJuRft/q2T0JsZb/kKTyw5ZatjZmDDuSWg==
X-Received: by 2002:a05:6214:2e41:b0:6a0:b905:97ab with SMTP id 6a1803df08f44-6a1682349d4mr196213776d6.52.1715787226198;
        Wed, 15 May 2024 08:33:46 -0700 (PDT)
Date: Wed, 15 May 2024 17:33:43 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Elias El Yandouzi <eliasely@amazon.com>
Subject: Re: [PATCH] x86: respect mapcache_domain_init() failing
Message-ID: <ZkTV1w7B2UyvJSHU@macbook>
References: <dc80cf26-8900-4848-84dd-6f3e47d59f57@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <dc80cf26-8900-4848-84dd-6f3e47d59f57@suse.com>

On Wed, May 15, 2024 at 03:35:15PM +0200, Jan Beulich wrote:
> The function itself properly handles and hands onwards failure from
> create_perdomain_mapping(). Therefore its caller should respect possible
> failure, too.
> 
> Fixes: 4b28bf6ae90b ("x86: re-introduce map_domain_page() et al")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:42:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:42:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722534.1126598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GlN-0004Gv-DX; Wed, 15 May 2024 15:41:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722534.1126598; Wed, 15 May 2024 15:41:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GlN-0004Go-9j; Wed, 15 May 2024 15:41:53 +0000
Received: by outflank-mailman (input) for mailman id 722534;
 Wed, 15 May 2024 15:41:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7GlM-0004CF-DG
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:41:52 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a5581f4f-12d1-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:41:50 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-56e6a1edecfso2772384a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:41:50 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574f6b8b9d7sm1341992a12.82.2024.05.15.08.41.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:41:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5581f4f-12d1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715787710; x=1716392510; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=k1F+NXhIM/iWN8rTZcrc7o7sEQYIQOhmQW6f5r3DggA=;
        b=WHZxoy2PJLGPdLEYC2+CqvrETVfq2bbXGL1XbAPhSWrk0Lw/AyZmUq+9dIEM9GudNN
         hzJIFPgD2rp+2vewmfFbmmb8EwC6TluTG5lQmk2+O0CFpU/6ZCpAqMkH+H3IDTPyNoff
         pzrzTAgz8Ac0Z1kJLjdKi9/HVVWIXo9+tUgg6p0GchFRgfGuKk6Lx10cEPiA3Vuvklzr
         XsiwOh4kBhlkIgHLI9GFIwgRKEehkj0g7rFvmZfAb5abutRviq2GavBXoIYi8RaMGEJk
         uswL2ZAwqHJ0kLz0bIE4GEllTZVIT44rYA7ZIMMLnpaWZ/Wosx++gHR6gIWi98sIuEtM
         Vk6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715787710; x=1716392510;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=k1F+NXhIM/iWN8rTZcrc7o7sEQYIQOhmQW6f5r3DggA=;
        b=VUT1kcwZ6TiIHSyEk7OWgNKhepSFHE4ctVvFzDv2jhNneASyPytFy0OePys8NRMfia
         wCdio7V08kWRpwYMJlYg3MX8lO8pi5Rs8qsod/Rf3BSM7+kGGPDzTQ0Ccza+icu2OD/S
         yFJt+gvjpZHOp/lxPyFx+DU7QUiR/aVT8zlpi5cO0C79YE8KIGPM3koXepei6Bc+CLfL
         HKahndz11xH0sTRsMAHyxKlnrGwIF/kGf2USObcPlNA+Nveij6Ny514y4dWaC/+QeSMt
         eWXudCjSQFX86E1YfDX5GDYknJaOEKfZafJG/VPpwfQiKUE0AT/0I57NuV5OfWvI8gGP
         X5/g==
X-Forwarded-Encrypted: i=1; AJvYcCUb7/dU1+aHEpBCqa94eaJ/jsmgsBZAz8rTYbC7vZcnXy0mIIfSYF3TkNaHFUDiMNexQp0RSGOYXh2RxTONJKOE37P0REjgvaNwcTR+Obs=
X-Gm-Message-State: AOJu0YxcrxVdi+8/zoU1fJnrl/hP1ydPqihh04wiA2sAE9sFW7gIot/a
	jbMcmVySS3jY/hxZCTEwzwPm3G/84+yD07igR/DfIbeZ/vYONB72js8HTTNsgQ==
X-Google-Smtp-Source: AGHT+IEFWQzMGkyuz5LE3c99VHzYFOZVpDE1dvCNp/Lf+MdDDpIXJmuVJmF9kgyDI3zv2+vknKq4bw==
X-Received: by 2002:a50:bb05:0:b0:572:8aab:4420 with SMTP id 4fb4d7f45d1cf-5734d6de8c1mr11562042a12.39.1715787709821;
        Wed, 15 May 2024 08:41:49 -0700 (PDT)
Message-ID: <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
Date: Wed, 15 May 2024 17:41:48 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 17:29, Oleksii K. wrote:
> On Wed, 2024-05-15 at 10:52 +0200, Jan Beulich wrote:
>> On 06.05.2024 12:15, Oleksii Kurochko wrote:
>>> The following generic functions were introduced:
>>> * test_bit
>>> * generic__test_and_set_bit
>>> * generic__test_and_clear_bit
>>> * generic__test_and_change_bit
>>>
>>> Also, the patch introduces the following generics which are
>>> used by the functions mentioned above:
>>> * BITOP_BITS_PER_WORD
>>> * BITOP_MASK
>>> * BITOP_WORD
>>> * BITOP_TYPE
>>>
>>> These functions and macros can be useful for architectures
>>> that don't have corresponding arch-specific instructions.
>>
>> Logically this paragraph may better move ahead of the BITOP_* one.
>>
>>> Because of that x86 has the following check in the macros
>>> test_bit(),
>>> __test_and_set_bit(), __test_and_clear_bit(),
>>> __test_and_change_bit():
>>>     if ( bitop_bad_size(addr) ) __bitop_bad_size();
>>> It was necessary to make bitop bad size check generic too, so
>>> arch_check_bitop_size() was introduced.
>>
>> Not anymore, with the most recent adjustments? There's nothing arch-
>> specific anymore in the checking.
>>
>>> @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr,
>>> volatile void *addr)
>>>   * If two examples of this operation race, one can appear to
>>> succeed
>>>   * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>>   */
>>> -static inline int __test_and_set_bit(int nr, void *addr)
>>> +static inline int arch__test_and_set_bit(int nr, volatile void
>>> *addr)
>>
>> I think I raised this point before: Why arch__ here, ...
>>
>>> @@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr,
>>> volatile void *addr)
>>>   * If two examples of this operation race, one can appear to
>>> succeed
>>>   * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>>   */
>>> -static inline int __test_and_clear_bit(int nr, void *addr)
>>> +static inline int arch__test_and_clear_bit(int nr, volatile void
>>> *addr)
>>
>> ... here, ...
>>
>>> @@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int
>>> nr, void *addr)
>>>  
>>>      return oldbit;
>>>  }
>>> -#define __test_and_clear_bit(nr, addr) ({               \
>>> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
>>> -    __test_and_clear_bit(nr, addr);                     \
>>> -})
>>> +#define arch__test_and_clear_bit arch__test_and_clear_bit
>>>  
>>>  /* WARNING: non atomic and it can be reordered! */
>>> -static inline int __test_and_change_bit(int nr, void *addr)
>>> +static inline int arch__test_and_change_bit(int nr, volatile void
>>> *addr)
>>
>> ... and here, while ...
>>
>>> @@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr,
>>> const volatile void *addr)
>>>      return oldbit;
>>>  }
>>>  
>>> -#define test_bit(nr, addr) ({                           \
>>> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
>>> +#define arch_test_bit(nr, addr) ({                      \
>>
>> ... just arch_ here? I don't like the double underscore infixes very
>> much, but I'm okay with them as long as they're applied consistently.
> 
> Common code and x86 use __test_and_clear_bit(), and this patch provides
> a generic version of __test_and_clear_bit(). To emphasize that
> generic__test_and_clear_bit() is a common implementation of
> __test_and_clear_bit(), the double underscore was retained. Also,
> test_and_clear_bit() exists and if one day it will be needed to provide
> a generic version of it, then it will be needed to have
> generic__test_and_clear_bit() and generic_test_and_clear_bit()
> 
> A similar logic was chosen for test_bit.

Right, but in all of your reply arch_ doesn't appear at all. Yet the
question was: Why then not arch__test_bit(), to match the other arch
helpers?

>>> --- a/xen/include/xen/bitops.h
>>> +++ b/xen/include/xen/bitops.h
>>> @@ -65,10 +65,144 @@ static inline int generic_flsl(unsigned long
>>> x)
>>>   * scope
>>>   */
>>>  
>>> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) %
>>> BITOP_BITS_PER_WORD))
>>> +
>>> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
>>> +
>>> +extern void __bitop_bad_size(void);
>>> +
>>>  /* --------------------- Please tidy above here ------------------
>>> --- */
>>>  
>>>  #include <asm/bitops.h>
>>>  
>>> +#ifndef arch_check_bitop_size
>>> +
>>> +#define bitop_bad_size(addr) sizeof(*(addr)) <
>>> sizeof(bitop_uint_t)
>>
>> Nit: Missing parentheses around the whole expression.
>>
>>> +#define arch_check_bitop_size(addr) \
>>> +    if ( bitop_bad_size(addr) ) __bitop_bad_size();
>>
>> Apart from the arch_ prefix that now wants dropping, this macro (if
>> we
>> want one in the first place) 
> What do you mean by 'want' here? I thought it is pretty necessary from
> safety point of view to have this check.

I don't question the check. What I was questioning is the need for a
macro to wrap that, seeing how x86 did without. I'm not outright
objecting to such a macro, though.

> Except arch_ prefix does it make sense to drop "#ifndef
> arch_check_bitop_size" around this macros?

Of course, as with arch_ dropped from the name there's no intention to
allow arch overrides.

>>> +/**
>>> + * generic__test_and_set_bit - Set a bit and return its old value
>>> + * @nr: Bit to set
>>> + * @addr: Address to count from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>> +static always_inline bool
>>> +generic__test_and_set_bit(unsigned long nr, volatile void *addr)
>>
>> The original per-arch functions all use "int" for their first
>> parameter.
>> Here you use unsigned long, without any mention in the description of
>> the
>> potential behavioral change. Which is even worse given that then x86
>> ends
>> up inconsistent with Arm and PPC in this regard, by ...
>>
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old | mask;
>>> +    return (old & mask);
>>> +}
>>> +
>>> +/**
>>> + * generic__test_and_clear_bit - Clear a bit and return its old
>>> value
>>> + * @nr: Bit to clear
>>> + * @addr: Address to count from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>> +static always_inline bool
>>> +generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old & ~mask;
>>> +    return (old & mask);
>>> +}
>>> +
>>> +/* WARNING: non atomic and it can be reordered! */
>>> +static always_inline bool
>>> +generic__test_and_change_bit(unsigned long nr, volatile void
>>> *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old ^ mask;
>>> +    return (old & mask);
>>> +}
>>> +/**
>>> + * generic_test_bit - Determine whether a bit is set
>>> + * @nr: bit number to test
>>> + * @addr: Address to start counting from
>>> + */
>>> +static always_inline bool generic_test_bit(int nr, const volatile
>>> void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +
>>> +    return (*p & mask);
>>> +}
>>> +
>>> +static always_inline bool
>>> +__test_and_set_bit(unsigned long nr, volatile void *addr)
>>> +{
>>> +#ifndef arch__test_and_set_bit
>>> +#define arch__test_and_set_bit generic__test_and_set_bit
>>> +#endif
>>> +
>>> +    return arch__test_and_set_bit(nr, addr);
>>
>> ... silently truncating and sign-converting nr here.
> Missed that fact. AFAIU there is no specific reason for bit number to
> be 'int' for this function, so it makes sense to update x86's prototype
> of arch__test_and_set_bit() to:
>    static inline int arch__test_and_set_bit(unsigned long nr, volatile
>    void *addr).
>    
> But probably I can't use 'unsigned long' here too, as it should a
> compilation error around 'btsl' instruction.
> 
> So it can be or 'unsinged int' or 'int'.
>>
>> As to generic_test_bit() - please don't cast away const-ness there.
>>
>>> +}
>>> +#define __test_and_set_bit(nr, addr) ({             \
>>> +    arch_check_bitop_size(addr);                    \
>>> +    __test_and_set_bit(nr, addr);                   \
>>> +})
>>> +
>>> +static always_inline bool
>>> +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
>>
>> Oddly enough here at least you use bitop_uint_t, but that's still ...
>>
>>> +{
>>> +#ifndef arch__test_and_clear_bit
>>> +#define arch__test_and_clear_bit generic__test_and_clear_bit
>>> +#endif
>>> +
>>> +    return arch__test_and_clear_bit(nr, addr);
>>
>> ... meaning a signedness conversion on x86 then. And beware: You
>> can't
>> simply change x86'es code to use bitop_uint_t. The underlying insns
>> used
>> interpret the bit position as a signed number, i.e. permitting
>> accesses
>> below the incoming pointer (whether it really makes sense to be that
>> way
>> is a separate question). I'm afraid I have no good suggestion how to
>> deal
>> with that: Any approach I can think of is either detrimental to the
>> generic implementation or would have unwanted effects on the x86 one.
>> Others, anyone?
> Is the signedness conversion here a big problem? I suppose no one will
> pass a negative number to nr.
> 
> It seems to me that it would be better to use 'int' everywhere and not
> use bitop_uint_t for 'nr' since it is just a bit position. 'Int'
> provides enough range for possible bit number.

Indeed, and that's then hopefully less of a risk as to introducing hard
to spot issues. Provided Arm and PPC are okay with that type then as well.

>>> --- a/xen/include/xen/types.h
>>> +++ b/xen/include/xen/types.h
>>> @@ -64,6 +64,12 @@ typedef __u64 __be64;
>>>  
>>>  typedef unsigned int __attribute__((__mode__(__pointer__)))
>>> uintptr_t;
>>>  
>>> +#ifndef BITOP_TYPE
>>> +#define BITOP_BITS_PER_WORD 32
>>> +
>>> +typedef uint32_t bitop_uint_t;
>>> +#endif
>>
>> I think you mentioned to me before why this needs to live here, not
>> in
>> xen/bitops.h. Yet I don't recall the reason, and the description
>> (hint,
>> hint) doesn't say anything either.
> If I remember correctly ( after this phrase I think I have to update
> the description ) the reason was that if I put that to xen/bitops.h:
> 
>     ...
>     #include <asm/bitops.h>
>     
>     #ifndef BITOP_TYPE
>     #define BITOP_BITS_PER_WORD 32
>     /* typedef uint32_t bitop_uint_t; */
>     #endif
>     ...
> 
> Then we will have an issue that we can't use the generic definition of 
> BITOP_BITS_PER_WORD and bitop_uint_t in asm/bitops.h as it is defined
> after inclusion of <asm/bitops.h>.
> 
> But if to put it to <xen/types.h> then such problem won't occur.
> 
> If it still makes sense, then I'll update the description in the next
> patch version.

I see. But we don't need to allow for arch overrides here anymore, so in
xen/bitops.h couldn't we as well have

#define BITOP_BITS_PER_WORD 32
typedef uint32_t bitop_uint_t;

... (if necessary)

#include <asm/bitops.h>

?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:51:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:51:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722541.1126607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GuB-0006BM-5v; Wed, 15 May 2024 15:50:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722541.1126607; Wed, 15 May 2024 15:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7GuB-0006BF-3U; Wed, 15 May 2024 15:50:59 +0000
Received: by outflank-mailman (input) for mailman id 722541;
 Wed, 15 May 2024 15:50:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Gu9-0006B9-K8
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:50:57 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea461344-12d2-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:50:55 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56e69a51a33so1752361a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:50:55 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01968sm875337866b.166.2024.05.15.08.50.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:50:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea461344-12d2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715788255; x=1716393055; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gCBH8tcSc17q3r2TgdLdiA8tYhMQC+mdJZ2R4Rxncy8=;
        b=VNHQzaXGtgzKQZEqwhL7EMmEGNVul3x3VpbPurhqMB7BSpoY1nUJjv+kVaDm/fknQW
         ip5ADBFzwzFED7idp6kmRm4STpzTLEctD/FQpqFMvmY46BpYkIXs6Ca3OJxELGAJf4Aw
         KpBH3L34+Dmpg/1JXwZpsgzImHZmFqDsc0ViM78Mu9/PxHgsl4IrZPdw9DzwdOM6flKc
         2Na+TXVZ1kqOHbEPT0KhcV3HwXSwh1uKpezMqudvATXY6fJ5zOc/j0Kvj/EMlMgbkrOP
         flhlsQdrXK8xESvVjTLqvD5mdDXx7rz3x83lkTCZx+/xNT3ETLe+K+b+waoNmCF0Qvrm
         KsPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715788255; x=1716393055;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gCBH8tcSc17q3r2TgdLdiA8tYhMQC+mdJZ2R4Rxncy8=;
        b=rlD6/ls3MFxxgIF4bAc62fjE2mFQNqE92L68zhTMjU0jmXC9BDU2MJNMZcw/l6qRuv
         THOPp02iOf5bmhC5CkGPuLPqrut3P7ht/TvFvzupNsV+q3ppKpocX/eHU1JAYm+UVtFf
         1nzdtEAUIByPbDlv1EKenerpfzHiBx+7/fGMq/I1tnt5X9NXWmeBHocTcechh4dYDA1O
         9b548sb37bAHT0oU4ScPwIrWKriG49wUxL6AqMtDwt+5zTket2FeufHpEHQuOReK1K1j
         tqRKItt4aNCeG3vsPetmq59mbXH9/YKwzZNbob7+H1GEtv9sTi0Ic6BOtWnQS4tekIdJ
         VrRg==
X-Gm-Message-State: AOJu0Yzj92GdpAojmO/ux2aejQ4KcOTrRjm6E6MWZaGKyb/m/U267GR1
	yfNnbiX53czcX6oQlpZ+8Tfw0MtUBFTgduCnEzSEchcBMg1VmKBYj69EXOdU9g==
X-Google-Smtp-Source: AGHT+IFQRA/khRP0tEFvlCmdIDIlH4A5ofEHWuCXHkO+6/yE3ODVRsFf/6tqw8VBH9pl00Wna3R6mw==
X-Received: by 2002:a17:906:1406:b0:a5a:7a4e:7e80 with SMTP id a640c23a62f3a-a5a7a4e7effmr540744366b.72.1715788255171;
        Wed, 15 May 2024 08:50:55 -0700 (PDT)
Message-ID: <256495e6-cd0c-4b6f-a1c0-78cce2338eb5@suse.com>
Date: Wed, 15 May 2024 17:50:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 13/19] x86/setup: Do not create valid mappings
 when directmap=no
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, pdurrant@amazon.com,
 dwmw@amazon.com, Hongyan Xia <hongyxia@amazon.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall <jgrall@amazon.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-14-eliasely@amazon.com> <ZkOFqFrSs41UtjIU@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkOFqFrSs41UtjIU@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.05.2024 17:39, Roger Pau Monné wrote:
> On Mon, May 13, 2024 at 01:40:40PM +0000, Elias El Yandouzi wrote:
>> +        {
>> +            l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)];
>> +
>> +            if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
>> +            {
>> +                mfn_t mfn = alloc_boot_pages(1, 1);
> 
> Hm, why not use alloc_xen_pagetable()?
> 
>> +                void *v = map_domain_page(mfn);
>> +
>> +                clear_page(v);
>> +                UNMAP_DOMAIN_PAGE(v);
> 
> Maybe use clear_domain_page()?

Or else use unmap_domain_page(). v is going out of scope just afterwards,
and UNMAP_DOMAIN_PAGE() is intended to be use when that's not the case.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 15:59:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 15:59:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722545.1126618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7H2M-00081U-Rf; Wed, 15 May 2024 15:59:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722545.1126618; Wed, 15 May 2024 15:59:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7H2M-00081N-Oe; Wed, 15 May 2024 15:59:26 +0000
Received: by outflank-mailman (input) for mailman id 722545;
 Wed, 15 May 2024 15:59:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7H2L-0007zS-FM
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 15:59:25 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 18e02180-12d4-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 17:59:23 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso197825266b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 08:59:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17870sm871813566b.216.2024.05.15.08.59.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 08:59:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18e02180-12d4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715788763; x=1716393563; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FCcAy8OcZ4luyl8LZClZRMMbTK0tl1r5Psh2m0WNWzU=;
        b=hDntHQUWKoDVEpNqGKjAzParQoDKoxLrijO4C2KsiMzDzNBndS8Cgps9tPFfCJZEQA
         EaKVxc3kP8j7eip6O4mn3Wj9ep9z8OYZRyGUpqC5mqdybDDcICtRCpdrf0zlqV1Sb4Ml
         qtYTBVXs5qzuAt8hkMVymkk8Bi1obqP/dhJlrEq+0+PgXmJ1QU3zhbWFSmGC5kqr29Qu
         lTNf6/gqiVZeFjd4xYF6vez+T2f9H7x2YIvP1KKsIROam8P8CxefIUV37cS6VtO2Z3bn
         erUd3Pv8NeeYrKvCREoVBgjDAbw/uxVufEHXDBEuDYNRtTci5jlg9lJyj8SRWUQ7+kRU
         UncA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715788763; x=1716393563;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FCcAy8OcZ4luyl8LZClZRMMbTK0tl1r5Psh2m0WNWzU=;
        b=NL/XZBsYxrLBX2+N7YLZicZ9C2EjgZOoXTaGMKCUTgf96CliWghZtyjBKNkZRHFflT
         Zc9wDerhCFTSBcDgC+ikVgA0OU3sUNzB5jCXh+OSnugUte7OLL1Vn4g78Fgc3rAY5XGD
         O7clzvejH6AbQThDnWv/ZBz14v6/DysBmLtv5XvkR3GHWcZNldZ8llT3jXbbJH/tR+yJ
         7Oe5kJ2vMzEyAkNq4l5TUto4xzjstKBQ4K2ryc+47h4ZEw4KXtd4rFNrbLTEmcO823Ct
         9vRACYupRgXtbEpM1ExnCRtya/VM2JN44dXBsGYSfZCqkgkZ1V69BPQbBaFXQhbCcqYH
         Z/wg==
X-Forwarded-Encrypted: i=1; AJvYcCUrqh25GXsJgwEEzw9plksAP0TeL3KTRtLhjc7tH3ugGbD6AoIZnhbQj8pYsM99DZT6hMBCjBGkOs+e/hKA6Uvh2XzEeK3wDQsQGD+n9SE=
X-Gm-Message-State: AOJu0Yz4HMTddnYRHcoe3jdyvISdNckqSH1dJLM/Dq7yo4iMnnhNDeST
	CJWthW1vmA7W3YnZaV1ni7c0agHPyGZ3CSGLXZZh+Fr5FmYdaU1RzzC6cuOJcA==
X-Google-Smtp-Source: AGHT+IFZonrQnUx4F9nz1NYGj3AeOvBlLeQ69rctvJVQJz2D3ovvy0l1M+pBwBk1YlQCjM9mqS4xVw==
X-Received: by 2002:a17:907:980d:b0:a5a:5496:3c76 with SMTP id a640c23a62f3a-a5a54963cdfmr1092765466b.6.1715788762767;
        Wed, 15 May 2024 08:59:22 -0700 (PDT)
Message-ID: <e3e75469-c356-4a02-a3aa-a9f3d277fdb1@suse.com>
Date: Wed, 15 May 2024 17:59:21 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 13/19] x86/setup: Do not create valid mappings
 when directmap=no
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-14-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-14-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Create empty mappings in the second e820 pass. Also, destroy existing
> direct map mappings created in the first pass.
> 
> To make xenheap pages visible in guests, it is necessary to create empty
> L3 tables in the direct map even when directmap=no, since guest cr3s
> copy idle domain's L4 entries, which means they will share mappings in
> the direct map if we pre-populate idle domain's L4 entries and L3
> tables. A helper is introduced for this.

Hmm. On one hand this may reduce memory consumption some, when large
ranges of MFNs aren't allocated as Xen heap pages. Otoh this increases
memory needs when Xen heap pages actually need mapping. I wonder whether
the (presumably less intrusive) change of merely altering permissions
from PAGE_HYPERVISOR to _PAGE_NONE|MAP_SMALL_PAGES wouldn't be better.

> Also, after the direct map is actually gone, we need to stop updating
> the direct map in update_xen_mappings().

What is this about? You only alter setup.c here.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 16:02:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 16:02:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722550.1126628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7H5O-0002VQ-DE; Wed, 15 May 2024 16:02:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722550.1126628; Wed, 15 May 2024 16:02:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7H5O-0002VJ-Aa; Wed, 15 May 2024 16:02:34 +0000
Received: by outflank-mailman (input) for mailman id 722550;
 Wed, 15 May 2024 16:02:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EWeN=MS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7H5N-0002P9-5S
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 16:02:33 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 860324cb-12d4-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 18:02:26 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so197292966b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 09:02:26 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cd627e681sm93257066b.150.2024.05.15.09.02.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 09:02:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 860324cb-12d4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715788946; x=1716393746; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wtYr/WbH2LhECipyaU9Fs3pn4IHaginS0y/0imMygQc=;
        b=Q3UXiIq1YRqjqqR4rH2/fzqf85pCKAEMybMO35aVvmXpzNL5EufLW37jOnM39drI8g
         Do6ZQ2l914Su1Ua8estta/+MGuEBVJ1xY8XJRvxlK7Nv6QPLev8tF54SXTN5gqG/Inj4
         n/pYWgqZhYfeUjjLuuPKLlZ1MaARK1OgElNSvoTGGsU1JcyO9RkpjEaXLpbRCspBnbmq
         q89ZUA/JuJHP7wNBCpT0CF0l/bbwfqdS9t5gB3fUsviNLBW8sYMa7CjF7XLVAouJY4JW
         efbs9OwIgPAUiAi39rRq19r77cm9I2w2Kb+Oyz04oOBdjCZBdLBCts14NNhzKb5gQmsf
         13WA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715788946; x=1716393746;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wtYr/WbH2LhECipyaU9Fs3pn4IHaginS0y/0imMygQc=;
        b=rbt9it00focNIl6kpiQ29YDpgjCUQY8EoMLOBdihZ4t7/OGI92TSkUoFTvmxkvfA07
         6p+U+RXeVWT2SfqEaLPlQFutB2M9UIEjnlPJ1MmDdx4JU4BZr89fU2BKv07hNMG3bbdD
         CHOM0UlGsptkXKgEsp850P4uj8wL9OXf0DsG5ggpLgJvkVx1RU4IRqGEwd1h9l6SJlRu
         ctoYqmGHynOBb/mepYpdo2wLRA9OnPxFMvePhdbuGV2xkcxh2xdTNSuQgcaOe54SFXf0
         w3kcvYyV3MUPpvq9kaNtQw4gVnTzGDmtX6E+YIXT/FEpkvAbPxK1Ak7rPznJNtakm5Qb
         Y5Pg==
X-Forwarded-Encrypted: i=1; AJvYcCXc9CPRleS3V3dVAjEwcOjrcV5UAGxc5tWQqMkW2pGihy8/CXMUaHXqg8d5IqSeiHSstv8IzZ5Aib84T2n9hsnE4HZSJuKzdHyFrwn0eeg=
X-Gm-Message-State: AOJu0YxA9tPa6IJIg6/Q61vliGQujyiHcXZ2mSZeTuXE0uxSusYKeyo+
	Ry+VF/wHwwajqFNJ5YpqGbK0sYhQs5aFNgQIEE93jvMfkIe+JYJsRvFJCsrLEw==
X-Google-Smtp-Source: AGHT+IHVeKhFzP79Ov6cJnYYB+OT4guvdNKwuLbsnCwBuhvHflzmRk/2bWqwvw/9oVEfUnEdVuUUbg==
X-Received: by 2002:a17:906:4f15:b0:a5a:89a3:323d with SMTP id a640c23a62f3a-a5a89a333a9mr373828166b.72.1715788945858;
        Wed, 15 May 2024 09:02:25 -0700 (PDT)
Message-ID: <03813fe4-39e6-4ac1-a6bf-eeda0eb6e034@suse.com>
Date: Wed, 15 May 2024 18:02:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-7-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
>  hardware guarantees (with, where available and known to Xen, respective
>  tweaks applied).
>  
> +### directmap (x86)
> +> `= <boolean>`
> +
> +> Default: `true`
> +
> +Enable or disable the directmap region in Xen.
> +
> +By default, Xen creates the directmap region which maps physical memory
> +in that region. Setting this to no will sparsely populate the directmap,
> +blocking exploits that leak secrets via speculative memory access in the
> +directmap.

Along the lines of remarks on comments and descriptions: I think we ought to
reserve "directmap=no" to a future where there really is the option of not
having anything direct-mapped. Right now imo that option form ought to be
invalid, and only "directmap=sparse" should be recognized (alongside
"directmap=yes" of course).

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 15 16:10:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 16:10:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722554.1126638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7HCr-000476-4z; Wed, 15 May 2024 16:10:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722554.1126638; Wed, 15 May 2024 16:10:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7HCr-00046z-2N; Wed, 15 May 2024 16:10:17 +0000
Received: by outflank-mailman (input) for mailman id 722554;
 Wed, 15 May 2024 16:10:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1aeg=MS=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7HCq-00046t-DW
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 16:10:16 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9d5f2675-12d5-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 18:10:15 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 85A3E80068;
 Wed, 15 May 2024 17:10:14 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d5f2675-12d5-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715789414;
	bh=OjqAW+TAGPOZIhhrPnD6hgyoZ74oVlIcnOnEsUGmRhY=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=LTo3lADku09O/lUuo/3NLVPMr9EJwBdPm9x8P6w5mnp4Ga0mtjBnNf6mmdPpICVij
	 BqZbI7byS/dAZwf7fEEnEVPBkI1zA8fkVzKwrsmRb3DRgxZ/4JNQbt6IdrNVZBOjFV
	 zOkyP3kl5g6m3VbsheOzJrJOGs6Sdz1cZBmmnYi8=
MIME-Version: 1.0
Date: Wed, 15 May 2024 17:10:14 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
 anthony.perard@citrix.com
Subject: Re: [RFC PATCH v2 5/5] tools/examples: Example Linux bridge VLAN
 config
In-Reply-To: <CAKf6xpt+UTwY0hFuODhpeb6rWOB0kgbqrrFVprP7MZYOTJ59YA@mail.gmail.com>
References: <20240508213824.55824-1-leigh@solinno.co.uk>
 <20240508213824.55824-6-leigh@solinno.co.uk>
 <CAKf6xpt+UTwY0hFuODhpeb6rWOB0kgbqrrFVprP7MZYOTJ59YA@mail.gmail.com>
Message-ID: <ec51767e47d0a5b9baea46173b1e9e9e@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Jason,

On 2024-05-15 01:58, Jason Andryuk wrote:
> On Wed, May 8, 2024 at 6:08 PM Leigh Brown <leigh@solinno.co.uk> 
> wrote:>
>> Add a new directory linux-bridge-vlan with examples files showing
>> how to configure systemd-networkd to support a bridge VLAN
>> configuration.
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> ---
>>  tools/examples/linux-bridge-vlan/README       | 68 
>> +++++++++++++++++++
>>  tools/examples/linux-bridge-vlan/br0.netdev   |  7 ++
>>  tools/examples/linux-bridge-vlan/br0.network  |  8 +++
>>  .../examples/linux-bridge-vlan/enp0s0.network | 16 +++++
>>  4 files changed, 99 insertions(+)
>>  create mode 100644 tools/examples/linux-bridge-vlan/README
>>  create mode 100644 tools/examples/linux-bridge-vlan/br0.netdev
>>  create mode 100644 tools/examples/linux-bridge-vlan/br0.network
>>  create mode 100644 tools/examples/linux-bridge-vlan/enp0s0.network
> 
> I think putting these in docs/misc/linux-bridge-vlan/ might be a
> better location.

No problem, will move.

>> diff --git a/tools/examples/linux-bridge-vlan/README 
>> b/tools/examples/linux-bridge-vlan/README
>> new file mode 100644
>> index 0000000000..83b9fa3fd6
>> --- /dev/null
>> +++ b/tools/examples/linux-bridge-vlan/README
>> @@ -0,0 +1,68 @@
>> +Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
>> +=====================================================================
>> +
>> +Introduction
>> +------------
>> +
>> +This directory contains example files to be placed in 
>> /etc/systemd/network
>> +to enable a single bridge with multiple VLAN support.
>> +
>> +The example is to support the scenario where the Xen host network 
>> interface
>> +is connected to an Ethernet switch configured as a trunk port. Each 
>> domain
>> +VIF can then be configured with one or more VLAN IDs, one of which 
>> will be
>> +the PVID.
>> +
>> +The example files create a bridge device called br0, with a physical 
>> interface
>> +called enp0s0. You will need to update this with your system's device 
>> name.
>> +
>> +Key points of the configuration are:
>> +
>> +1. In br0.netdev, VLANFiltering=on is set. This is required to ensure 
>> the
>> +   VLAN tags are handled correctly.  If it is not set then the 
>> packets
>> +   from the VIF interfaces will not have the correct VLAN tags set.
>> +
>> +2. In br0.network, a system IPv4 address is configured that can be 
>> updated
>> +   according to your local network settings.
>> +
>> +3. In enp0s0.network, Bridge=br0 sets the bridge device to connect 
>> to. There
>> +   is also a [BridgeVLAN] section for each VLAN you want to give 
>> access
>> +   to the switch. Note, if you want to create an internal VLAN 
>> private to
> 
> For
> "for each VLAN you want to give access to the switch"
> do you mean:
> "for each VLAN you want connected with the external network"
> or
> "for each VLAN you want accessible on the external network"
> ?
> The "access to the switch" part I find unclear.
> 

On re-reading it is not as clear as I'd hope - I will adjust.

>> +   the host, do not include its VLAN ID in this file.
> 
> 
>> +Domain configuration
>> +--------------------
>> +
>> +Add the vlan= keyword to the vif definition in the domain. The 
>> simplest
>> +and most common example is a domain that wishes to connect to a 
>> single VLAN:
>> +
>> +vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10' ]
>> +
>> +If you wish to configure a domain to route between two VLANs, you 
>> have two
>> +options. Option 1 is to create multiple interfaces on different 
>> VLANs:
>> +
>> +vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10',
>> +       'max=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=20' ]
> 
> Hard tab here makes the diff look off, but the file would be aligned.

Will fix up so all the indents are spaced, to be consistent.

> I think this is good content.  I'm not familiar with the systemd
> networking stuff to give an R-b.  But it's only examples, so I think
> it should be okay.  I'm not a maintainer, but it would be an Acked-by,
> if I were.
> 
> Regards,
> Jason

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Wed May 15 17:03:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 17:03:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722560.1126649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7I2C-0005zw-1p; Wed, 15 May 2024 17:03:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722560.1126649; Wed, 15 May 2024 17:03:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7I2B-0005zp-TF; Wed, 15 May 2024 17:03:19 +0000
Received: by outflank-mailman (input) for mailman id 722560;
 Wed, 15 May 2024 17:03:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ly6O=MS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7I2A-0005zj-PW
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 17:03:18 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02c3c83f-12dd-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 19:03:11 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e1fa1f1d9bso123908871fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 10:03:11 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d9192sm2609570e87.241.2024.05.15.10.03.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 10:03:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02c3c83f-12dd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715792591; x=1716397391; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=CudvKSAjP6QR2eUIDcgcD3uTS4NZg1oCeQIxxS4n4eo=;
        b=D4XoW+3wRD3xBOmMmo3TzJk12XucXybVNcAggTx8H5x6CEM8syD4LXb3vHroKXVd5k
         wdebNEIjgwaVTP8AOLSFOZEglgQItRnJK2UGDm0/gBsuLKVole1yEJelpD4peRi7Wt25
         l2S4OXfSjDi2CODVkk+g9vJ4wFTaVJ93W4q4MRLN075Us4XDyWpZW1rtBDxs0QrpI/wt
         9iQc3zaXXC9NXoyd/5/tkH32i1DjQbMucbPUdnTIVgJSv3PgeXsyPF+jgWUXYl/SLsv9
         z2IFquGp0kN4FDOprWaslsENyqZ7KqLAxcEEaLndXN+gzV0LHBNKhLvnPoZHyiTYui0t
         POFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715792591; x=1716397391;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CudvKSAjP6QR2eUIDcgcD3uTS4NZg1oCeQIxxS4n4eo=;
        b=vlsxZ0r5T2SNxukskG5VbrwJD6Any6JrbtSUiKvOOXf2QCZVEAaiqqKpK1URsPiCqq
         jOdt4voamNpcYm/Oln4Annf4kGnkbBbfqUr3L6Di/HoaUeB6zMPedSFCwPzYg1eAixlp
         SLr2frFlwjbWcgJsCHMjxRtYvmokFbhc6vqnjajPSReOvUwLdeeHIHd4ecofTYsHWKbX
         i2UpbMiXdbr0yf3IUOSEz5xU+5CJtV9pSLIZVa+3JchObXs165ZnU7uGd0Vw8zCrF5t4
         IlCShy2soMscrpbkW1PmZKBOGN7TP/OY/mv0UWO2C/7z3QluYG9tIWuPXbofmpc0xFsL
         oc0w==
X-Forwarded-Encrypted: i=1; AJvYcCUCkREbuEjU9PR8qbVrjgp7xC0tamw42lXnB3xk8+hOW/foFyjWETs0MKusvyVdz0td/3V6QjvpoYDE39Mfr5/XPDuPB/0oXhCgepiExp0=
X-Gm-Message-State: AOJu0YwEGB77JSsDQgrx4kE0ilkNObPsgTOV0cuVOmujVu5wUGeULMfy
	fdd63AJp6lAqFqUI7lpBSSlDClyEUBJwNMeJUObTMPeHRHeuiLko
X-Google-Smtp-Source: AGHT+IHMk4b45aUupTs5AUMmsysZNygLlrTy77fZNAttbqJfDEHsSSwb1PHHGZHRUBz41qJw1IvM/w==
X-Received: by 2002:a05:6512:70f:b0:523:af1f:89d9 with SMTP id 2adb3069b0e04-523af1f8aa7mr374443e87.58.1715792590780;
        Wed, 15 May 2024 10:03:10 -0700 (PDT)
Message-ID: <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 15 May 2024 19:03:09 +0200
In-Reply-To: <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
	 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
	 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
	 <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 17:41 +0200, Jan Beulich wrote:
> On 15.05.2024 17:29, Oleksii K. wrote:
> > On Wed, 2024-05-15 at 10:52 +0200, Jan Beulich wrote:
> > > On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > > > The following generic functions were introduced:
> > > > * test_bit
> > > > * generic__test_and_set_bit
> > > > * generic__test_and_clear_bit
> > > > * generic__test_and_change_bit
> > > >=20
> > > > Also, the patch introduces the following generics which are
> > > > used by the functions mentioned above:
> > > > * BITOP_BITS_PER_WORD
> > > > * BITOP_MASK
> > > > * BITOP_WORD
> > > > * BITOP_TYPE
> > > >=20
> > > > These functions and macros can be useful for architectures
> > > > that don't have corresponding arch-specific instructions.
> > >=20
> > > Logically this paragraph may better move ahead of the BITOP_*
> > > one.
> > >=20
> > > > Because of that x86 has the following check in the macros
> > > > test_bit(),
> > > > __test_and_set_bit(), __test_and_clear_bit(),
> > > > __test_and_change_bit():
> > > > =C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();
> > > > It was necessary to make bitop bad size check generic too, so
> > > > arch_check_bitop_size() was introduced.
> > >=20
> > > Not anymore, with the most recent adjustments? There's nothing
> > > arch-
> > > specific anymore in the checking.
> > >=20
> > > > @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr,
> > > > volatile void *addr)
> > > > =C2=A0 * If two examples of this operation race, one can appear to
> > > > succeed
> > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesse=
s with
> > > > a
> > > > lock.
> > > > =C2=A0 */
> > > > -static inline int __test_and_set_bit(int nr, void *addr)
> > > > +static inline int arch__test_and_set_bit(int nr, volatile void
> > > > *addr)
> > >=20
> > > I think I raised this point before: Why arch__ here, ...
> > >=20
> > > > @@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int
> > > > nr,
> > > > volatile void *addr)
> > > > =C2=A0 * If two examples of this operation race, one can appear to
> > > > succeed
> > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesse=
s with
> > > > a
> > > > lock.
> > > > =C2=A0 */
> > > > -static inline int __test_and_clear_bit(int nr, void *addr)
> > > > +static inline int arch__test_and_clear_bit(int nr, volatile
> > > > void
> > > > *addr)
> > >=20
> > > ... here, ...
> > >=20
> > > > @@ -243,13 +237,10 @@ static inline int
> > > > __test_and_clear_bit(int
> > > > nr, void *addr)
> > > > =C2=A0
> > > > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > > > =C2=A0}
> > > > -#define __test_and_clear_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > -=C2=A0=C2=A0=C2=A0 __test_and_clear_bit(nr, addr);=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > -})
> > > > +#define arch__test_and_clear_bit arch__test_and_clear_bit
> > > > =C2=A0
> > > > =C2=A0/* WARNING: non atomic and it can be reordered! */
> > > > -static inline int __test_and_change_bit(int nr, void *addr)
> > > > +static inline int arch__test_and_change_bit(int nr, volatile
> > > > void
> > > > *addr)
> > >=20
> > > ... and here, while ...
> > >=20
> > > > @@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr,
> > > > const volatile void *addr)
> > > > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > > > =C2=A0}
> > > > =C2=A0
> > > > -#define test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +#define arch_test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > >=20
> > > ... just arch_ here? I don't like the double underscore infixes
> > > very
> > > much, but I'm okay with them as long as they're applied
> > > consistently.
> >=20
> > Common code and x86 use __test_and_clear_bit(), and this patch
> > provides
> > a generic version of __test_and_clear_bit(). To emphasize that
> > generic__test_and_clear_bit() is a common implementation of
> > __test_and_clear_bit(), the double underscore was retained. Also,
> > test_and_clear_bit() exists and if one day it will be needed to
> > provide
> > a generic version of it, then it will be needed to have
> > generic__test_and_clear_bit() and generic_test_and_clear_bit()
> >=20
> > A similar logic was chosen for test_bit.
>=20
> Right, but in all of your reply arch_ doesn't appear at all.
I am a little confused here. According to my logic, should it be
arch___test_and_set_bit() and generic___test_and_set_bit()?

If you are asking why there is no generic implementation for
test_and_clear_bit() without the double underscores, the reason is that
Arm, PPC, and x86 don't use generic code and rely on specific
instructions for this operation. Therefore, I didn't see much sense in
providing a generic version of test_and_clear_bit(), at least, for now.


>  Yet the
> question was: Why then not arch__test_bit(), to match the other arch
> helpers?
Because no one uses __test_bit(). Everywhere is used test_bit().

>=20
> > > > --- a/xen/include/xen/bitops.h
> > > > +++ b/xen/include/xen/bitops.h
> > > > @@ -65,10 +65,144 @@ static inline int generic_flsl(unsigned
> > > > long
> > > > x)
> > > > =C2=A0 * scope
> > > > =C2=A0 */
> > > > =C2=A0
> > > > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > > > BITOP_BITS_PER_WORD))
> > > > +
> > > > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > > > +
> > > > +extern void __bitop_bad_size(void);
> > > > +
> > > > =C2=A0/* --------------------- Please tidy above here -------------=
-
> > > > ----
> > > > --- */
> > > > =C2=A0
> > > > =C2=A0#include <asm/bitops.h>
> > > > =C2=A0
> > > > +#ifndef arch_check_bitop_size
> > > > +
> > > > +#define bitop_bad_size(addr) sizeof(*(addr)) <
> > > > sizeof(bitop_uint_t)
> > >=20
> > > Nit: Missing parentheses around the whole expression.
> > >=20
> > > > +#define arch_check_bitop_size(addr) \
> > > > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();
> > >=20
> > > Apart from the arch_ prefix that now wants dropping, this macro
> > > (if
> > > we
> > > want one in the first place)=C2=A0
> > What do you mean by 'want' here? I thought it is pretty necessary
> > from
> > safety point of view to have this check.
>=20
> I don't question the check. What I was questioning is the need for a
> macro to wrap that, seeing how x86 did without. I'm not outright
> objecting to such a macro, though.
We can follow x86 approach as we there's no any more intention to allow
arch overrides.

>=20
> > Except arch_ prefix does it make sense to drop "#ifndef
> > arch_check_bitop_size" around this macros?
>=20
> Of course, as with arch_ dropped from the name there's no intention
> to
> allow arch overrides.
>=20
> > > > +/**
> > > > + * generic__test_and_set_bit - Set a bit and return its old
> > > > value
> > > > + * @nr: Bit to set
> > > > + * @addr: Address to count from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > > > +static always_inline bool
> > > > +generic__test_and_set_bit(unsigned long nr, volatile void
> > > > *addr)
> > >=20
> > > The original per-arch functions all use "int" for their first
> > > parameter.
> > > Here you use unsigned long, without any mention in the
> > > description of
> > > the
> > > potential behavioral change. Which is even worse given that then
> > > x86
> > > ends
> > > up inconsistent with Arm and PPC in this regard, by ...
> > >=20
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +
> > > > +/**
> > > > + * generic__test_and_clear_bit - Clear a bit and return its
> > > > old
> > > > value
> > > > + * @nr: Bit to clear
> > > > + * @addr: Address to count from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > > > +static always_inline bool
> > > > +generic__test_and_clear_bit(bitop_uint_t nr, volatile void
> > > > *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +
> > > > +/* WARNING: non atomic and it can be reordered! */
> > > > +static always_inline bool
> > > > +generic__test_and_change_bit(unsigned long nr, volatile void
> > > > *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +/**
> > > > + * generic_test_bit - Determine whether a bit is set
> > > > + * @nr: bit number to test
> > > > + * @addr: Address to start counting from
> > > > + */
> > > > +static always_inline bool generic_test_bit(int nr, const
> > > > volatile
> > > > void *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > > > +}
> > > > +
> > > > +static always_inline bool
> > > > +__test_and_set_bit(unsigned long nr, volatile void *addr)
> > > > +{
> > > > +#ifndef arch__test_and_set_bit
> > > > +#define arch__test_and_set_bit generic__test_and_set_bit
> > > > +#endif
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
> > >=20
> > > ... silently truncating and sign-converting nr here.
> > Missed that fact. AFAIU there is no specific reason for bit number
> > to
> > be 'int' for this function, so it makes sense to update x86's
> > prototype
> > of arch__test_and_set_bit() to:
> > =C2=A0=C2=A0 static inline int arch__test_and_set_bit(unsigned long nr,
> > volatile
> > =C2=A0=C2=A0 void *addr).
> > =C2=A0=C2=A0=20
> > But probably I can't use 'unsigned long' here too, as it should a
> > compilation error around 'btsl' instruction.
> >=20
> > So it can be or 'unsinged int' or 'int'.
> > >=20
> > > As to generic_test_bit() - please don't cast away const-ness
> > > there.
> > >=20
> > > > +}
> > > > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 arch_check_bitop_size(addr);=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 \
> > > > +})
> > > > +
> > > > +static always_inline bool
> > > > +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> > >=20
> > > Oddly enough here at least you use bitop_uint_t, but that's still
> > > ...
> > >=20
> > > > +{
> > > > +#ifndef arch__test_and_clear_bit
> > > > +#define arch__test_and_clear_bit generic__test_and_clear_bit
> > > > +#endif
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 return arch__test_and_clear_bit(nr, addr);
> > >=20
> > > ... meaning a signedness conversion on x86 then. And beware: You
> > > can't
> > > simply change x86'es code to use bitop_uint_t. The underlying
> > > insns
> > > used
> > > interpret the bit position as a signed number, i.e. permitting
> > > accesses
> > > below the incoming pointer (whether it really makes sense to be
> > > that
> > > way
> > > is a separate question). I'm afraid I have no good suggestion how
> > > to
> > > deal
> > > with that: Any approach I can think of is either detrimental to
> > > the
> > > generic implementation or would have unwanted effects on the x86
> > > one.
> > > Others, anyone?
> > Is the signedness conversion here a big problem? I suppose no one
> > will
> > pass a negative number to nr.
> >=20
> > It seems to me that it would be better to use 'int' everywhere and
> > not
> > use bitop_uint_t for 'nr' since it is just a bit position. 'Int'
> > provides enough range for possible bit number.
>=20
> Indeed, and that's then hopefully less of a risk as to introducing
> hard
> to spot issues. Provided Arm and PPC are okay with that type then as
> well.
Then I will update prototypes of generic functions correspondingly.

>=20
> > > > --- a/xen/include/xen/types.h
> > > > +++ b/xen/include/xen/types.h
> > > > @@ -64,6 +64,12 @@ typedef __u64 __be64;
> > > > =C2=A0
> > > > =C2=A0typedef unsigned int __attribute__((__mode__(__pointer__)))
> > > > uintptr_t;
> > > > =C2=A0
> > > > +#ifndef BITOP_TYPE
> > > > +#define BITOP_BITS_PER_WORD 32
> > > > +
> > > > +typedef uint32_t bitop_uint_t;
> > > > +#endif
> > >=20
> > > I think you mentioned to me before why this needs to live here,
> > > not
> > > in
> > > xen/bitops.h. Yet I don't recall the reason, and the description
> > > (hint,
> > > hint) doesn't say anything either.
> > If I remember correctly ( after this phrase I think I have to
> > update
> > the description ) the reason was that if I put that to
> > xen/bitops.h:
> >=20
> > =C2=A0=C2=A0=C2=A0 ...
> > =C2=A0=C2=A0=C2=A0 #include <asm/bitops.h>
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 #ifndef BITOP_TYPE
> > =C2=A0=C2=A0=C2=A0 #define BITOP_BITS_PER_WORD 32
> > =C2=A0=C2=A0=C2=A0 /* typedef uint32_t bitop_uint_t; */
> > =C2=A0=C2=A0=C2=A0 #endif
> > =C2=A0=C2=A0=C2=A0 ...
> >=20
> > Then we will have an issue that we can't use the generic definition
> > of=20
> > BITOP_BITS_PER_WORD and bitop_uint_t in asm/bitops.h as it is
> > defined
> > after inclusion of <asm/bitops.h>.
> >=20
> > But if to put it to <xen/types.h> then such problem won't occur.
> >=20
> > If it still makes sense, then I'll update the description in the
> > next
> > patch version.
>=20
> I see. But we don't need to allow for arch overrides here anymore, so
> in
> xen/bitops.h couldn't we as well have
>=20
> #define BITOP_BITS_PER_WORD 32
> typedef uint32_t bitop_uint_t;
>=20
> ... (if necessary)
>=20
> #include <asm/bitops.h>
>=20
> ?
Good point. If there is not need to allow for arch overrides then we
can use suggested above approach. Thanks.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 15 17:25:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 17:25:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722568.1126658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7IN0-0001A1-KU; Wed, 15 May 2024 17:24:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722568.1126658; Wed, 15 May 2024 17:24:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7IN0-00019u-Hq; Wed, 15 May 2024 17:24:50 +0000
Received: by outflank-mailman (input) for mailman id 722568;
 Wed, 15 May 2024 17:24:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7IMz-00019o-Bs
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 17:24:49 +0000
Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com
 [2607:f8b0:4864:20::f34])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06b39987-12e0-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 19:24:47 +0200 (CEST)
Received: by mail-qv1-xf34.google.com with SMTP id
 6a1803df08f44-6a04bae513cso26120406d6.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 10:24:47 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179900sm66201596d6.19.2024.05.15.10.24.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 10:24:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06b39987-12e0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715793886; x=1716398686; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=z1nN9Ar+qfmXEaldLdkdVicnTVi7R8lyro+Fz4srAmg=;
        b=jpmrj/lQVJ+9tPBhYy8DQZX+MEmC18wSeEdKo6v2Evpnr1smyz3mpvdzDQLE82bm4O
         eSeTPanXBzcQoR/iESb+60c145d+YXZV/x0rzDPnqqBjFwBgObWlrNBUvRuf39MxkhxX
         oprk3PfbtFqhBlG0fydGRDRy8xAFHVvukh+SE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715793886; x=1716398686;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=z1nN9Ar+qfmXEaldLdkdVicnTVi7R8lyro+Fz4srAmg=;
        b=SpXZa5y1FqWxhEkhGe2BIqlntyie7YZ4t9fXUDH2lfJIhhl1oxQrf1fsBeWc8/RfWX
         7wxK/ZfZoFLnLOehOcUChh4peIwnlv+8+0BldEws02n4v79yOdZ02s/dw4vwwTXOtiVW
         bQJX1dgLYhUrMMn2KjO+cPLWsOwuBBJK/YtNrBqtV2BELZNOSQfrZ6S1EBKN9Xs3ouMS
         /46LMTVTcH8ufDP22f/6IfJ+zE2Lbm3lS/MMbLP2kMV6D8cycKEHwugn3iw2h3PdUd5+
         OSoFoZcp4iJPEK/Wmz1i27MwwqTrcVnQcIy0sEkDAEV8y1NS39dIk54Vnh2pSOfoxxYJ
         /esQ==
X-Gm-Message-State: AOJu0Yyr6Y48iGFdzEKILZ+1vj3YDN49w/JMnJ21gl0UG65rZokB9oiO
	ex73IQIudLRINbKBzMykUmTSoHLPs338JLwBXpU2iH9eBbiT7KIJZ8/dygIsw+k=
X-Google-Smtp-Source: AGHT+IGdw9AG2XeUJ28f6yMJbXM1IBdkgQiHzyYnNYIXdLu+8pK5dtrqRxTVCz+1t401kvj+l3a/mw==
X-Received: by 2002:a05:6214:550a:b0:6a0:5446:dbc1 with SMTP id 6a1803df08f44-6a168144b8bmr174723556d6.2.1715793886121;
        Wed, 15 May 2024 10:24:46 -0700 (PDT)
Message-ID: <ddc6fe6c-e7bb-4cf1-876a-843a6c5a2877@citrix.com>
Date: Wed, 15 May 2024 18:24:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH v2 4/5] docs/man: document VIF vlan keyword
To: Leigh Brown <leigh@solinno.co.uk>, Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, anthony.perard@citrix.com
References: <20240508213824.55824-1-leigh@solinno.co.uk>
 <20240508213824.55824-5-leigh@solinno.co.uk>
 <CAKf6xpviWyFxt4JMq23K0LxjDcP2zJF1Sqye=1jS2UAtwcaRww@mail.gmail.com>
 <c62b512c3b1ea58fb07e50df85b4ebd6@solinno.co.uk>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c62b512c3b1ea58fb07e50df85b4ebd6@solinno.co.uk>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 4:30 pm, Leigh Brown wrote:
> Hi Jason,
>
> On 2024-05-15 01:57, Jason Andryuk wrote:
>> On Wed, May 8, 2024 at 5:39 PM Leigh Brown <leigh@solinno.co.uk> wrote:
>>>
>>> Document the new `vlan' keyword in xl-network-configuration(5).
>>>
>>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>>
>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>
>> One nit below
>>
>>> ---
>>>  docs/man/xl-network-configuration.5.pod.in | 38 ++++++++++++++++++++++
>>>  1 file changed, 38 insertions(+)
>>>
>>> diff --git a/docs/man/xl-network-configuration.5.pod.in
>>> b/docs/man/xl-network-configuration.5.pod.in
>>> index f3e379bcf8..c35c0922b3 100644
>>> --- a/docs/man/xl-network-configuration.5.pod.in
>>> +++ b/docs/man/xl-network-configuration.5.pod.in
>>> @@ -259,6 +259,44 @@ Specifies the MTU (i.e. the maximum size of an
>>> IP payload, exclusing headers). T
>>
>>> +Note, one VLAN ID must be marked as the PVID. In the case of a vlan
>>> +specification consisting of a single VLAN ID (e.g. C<vlan=10>), the
>>> B<p> suffix
>>> +may be omitted. Specifying more than one untagged VLAN ID is an
>>> advanced
>>> +configuration - use with caution.
>>> +
>>> +For example:
>>> +
>>> +        'vlan=10' -- meaning a single VLAN that is the PVID.
>>> +       'vlan=10p/20' -- VLAN 10 is the PVID and VLAN 20 is tagged.
>>> +       'vlan=10p/100+10x4' -- VLANs 10, 100, 110, 120, 130, 140, 150.
>>
>> Indent mismatch between 7 and 8 spaces.
>
> FWIW I will sort that too :-)

This should be merged into patch 2 which introduces the vlan keyword.

As the indentation is the only issue, I'll fix it on commit while merging.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 18:26:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 18:26:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722573.1126667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7JKB-0001kN-TN; Wed, 15 May 2024 18:25:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722573.1126667; Wed, 15 May 2024 18:25:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7JKB-0001kG-QQ; Wed, 15 May 2024 18:25:59 +0000
Received: by outflank-mailman (input) for mailman id 722573;
 Wed, 15 May 2024 18:25:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o41/=MS=amazon.co.uk=prvs=858ff6e72=eliasely@srs-se1.protection.inumbo.net>)
 id 1s7JKA-0001kA-Ca
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 18:25:58 +0000
Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 909f8931-12e8-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 20:25:55 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6])
 by smtp-border-fw-2101.iad2.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 18:25:52 +0000
Received: from EX19MTAEUB002.ant.amazon.com [10.0.43.254:12661]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.39.135:2525]
 with esmtp (Farcaster)
 id 89e6cdf7-ef88-4cc4-bee1-a023a310ff98; Wed, 15 May 2024 18:25:50 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUB002.ant.amazon.com (10.252.51.79) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Wed, 15 May 2024 18:25:47 +0000
Received: from [192.168.26.160] (10.106.82.11) by
 EX19D018EUA002.ant.amazon.com (10.252.50.146) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Wed, 15 May 2024 18:25:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 909f8931-12e8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715797556; x=1747333556;
  h=message-id:date:mime-version:subject:to:references:from:
   in-reply-to:content-transfer-encoding;
  bh=C4vQ91EDfpC1i/6V1M2/7OmUKz0oO4cYLUlwCL4b68Q=;
  b=OtVbTjsA7cZIcXl09rJeJhEz/d8hKwrOIzNEA1fSko3hTnYGr8+Kmme0
   VIyfFqb40PMR5VxbIJnMA40vjwd21Edl+HPcV82eZeMLOLvRE1frVJeNP
   nxjtvs6HHLBQpVY93Uv6SuHQsyNDMA3VeDHXti3v5kbZmsFE/8QA3WUFz
   Y=;
X-IronPort-AV: E=Sophos;i="6.08,162,1712620800"; 
   d="scan'208";a="401391469"
X-Farcaster-Flow-ID: 89e6cdf7-ef88-4cc4-bee1-a023a310ff98
Message-ID: <bda3386e-26c5-4efd-b7ad-00f3643523fa@amazon.com>
Date: Wed, 15 May 2024 19:25:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 01/19] x86: Create per-domain mapping of
 guest_root_pt
To: <xen-devel@lists.xenproject.org>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-2-eliasely@amazon.com>
 <dd145c67-8e3e-4b15-94f7-c7cd1f127d45@suse.com>
Content-Language: en-US
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <dd145c67-8e3e-4b15-94f7-c7cd1f127d45@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.106.82.11]
X-ClientProxiedBy: EX19D039UWB003.ant.amazon.com (10.13.138.93) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

Hi Jan,

On 14/05/2024 15:51, Jan Beulich wrote:
> On 13.05.2024 15:40, Elias El Yandouzi wrote:
>> From: Hongyan Xia <hongyxia@amazon.com>
>>
>> Create a per-domain mapping of PV guest_root_pt as direct map is being
>> removed.
>>
>> Note that we do not map and unmap root_pgt for now since it is still a
>> xenheap page.
>>
>> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
>> Signed-off-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
>>
>> ----
>>      Changes in V3:
>>          * Rename SHADOW_ROOT
>>          * Haven't addressed the potentially over-allocation issue as I don't get it
> 
> I thought I had explained in enough detail that the GDT/LDT area needs
> quite a bit more space (2 times 64k per vCPU) than the root PT one (4k
> per vCPU). Thus while d->arch.pv.gdt_ldt_l1tab really needs to point at
> a full page (as long as not taking into account dynamic domain
> properties), d->arch.pv.root_pt_l1tab doesn't need to (and hence might
> better be allocated using xzalloc() / xzalloc_array(), even when also
> not taking into account dynamic domain properties, i.e. vCPU count).

I just understood your point and yes you're correct I was 
over-allocating... Sorry, it took me so long to get it.

I'll go instead with:

@@ -371,6 +396,12 @@ int pv_domain_initialise(struct domain *d)
          goto fail;
      clear_page(d->arch.pv.gdt_ldt_l1tab);

+    d->arch.pv.root_pt_l1tab =
+        xzalloc_array(l1_pgentry_t *,
+                      DIV_ROUND_UP(d->max_vcpus, L1_PAGETABLE_ENTRIES));
+    if ( !d->arch.pv.root_pt_l1tab )
+        goto fail;
+
      if ( levelling_caps & ~LCAP_faulting &&
           (d->arch.pv.cpuidmasks = xmemdup(&cpuidmask_defaults)) == NULL )
          goto fail;

However, I noticed quite a weird bug while doing some testing. I may 
need your expertise to find the root cause.

In the case where I have more vCPUs than pCPUs (and let's consider we 
have one pCPU for two vCPUs), I noticed that I would always get a page 
fault in dom0 kernel (5.10.0-13-amd64) at the exact same location. I did 
a bit of investigation but I couldn't come to a clear conclusion. 
Looking at the stack trace [1], I have the feeling the crash occurs in a 
loop or a recursive call.

I tried to identify where the crash occurred using addr2line:

 > addr2line -e vmlinux-5.10.0-29-amd64 0xffffffff810218a0
debian/build/build_amd64_none_amd64/arch/x86/xen/mmu_pv.c:880

It turns out to point on the closing bracket of the function 
xen_mm_unpin_all()[2].

I thought the crash could happen while returning from the function in 
the assembly epilogue but the output of objdump doesn't even show the 
address.

The only theory I could think of was that because we only have one pCPU, 
we may never execute one of the two vCPUs, and never setup the mapping 
to the guest_root_pt in write_ptbase(), hence the page fault. This is 
just a random theory, I couldn't find any hint suggesting it would be 
the case though. Any idea how I could debug this?

[1] https://pastebin.com/UaGRaV6a
[2] https://github.com/torvalds/linux/blob/v5.10/arch/x86/xen/mmu_pv.c#L880

Elias


From xen-devel-bounces@lists.xenproject.org Wed May 15 18:58:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 18:58:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722580.1126682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Jpq-0006Xc-DJ; Wed, 15 May 2024 18:58:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722580.1126682; Wed, 15 May 2024 18:58:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Jpq-0006XV-9q; Wed, 15 May 2024 18:58:42 +0000
Received: by outflank-mailman (input) for mailman id 722580;
 Wed, 15 May 2024 18:58:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7Jpp-0006XJ-3E; Wed, 15 May 2024 18:58:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7Jpp-0008Dz-1X; Wed, 15 May 2024 18:58:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7Jpo-0002N2-L9; Wed, 15 May 2024 18:58:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7Jpo-0004LD-KR; Wed, 15 May 2024 18:58:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=obfehCD0tIbyTFektNUTnAwYO4zXJO74i41bLZJYMKk=; b=3ij10gLclsJyJ6f/voExXTSUr/
	PTUiM8KNhqgGmlTu3p30JS8iRz9greON5IfLdBNbVPwc54kr9DOKiKzIZAP12jfckRbfb6DlPrnTm
	i/QsK5vxhgGIK1R/eMWTIirPlyLTkyAQk406QiOmNHWF71LqrCEwu+LxOuETRg6/1578=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186004-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186004: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:xen-boot:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=319a5125ca2649e6eb95670b4d721260025c187d
X-Osstest-Versions-That:
    xen=46aa3031ae89ac1771f4159972edab65710e7349
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 18:58:40 +0000

flight 186004 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186004/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt      8 xen-boot                     fail  like 185983
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186001
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186001
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186001
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186001
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186001
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  319a5125ca2649e6eb95670b4d721260025c187d
baseline version:
 xen                  46aa3031ae89ac1771f4159972edab65710e7349

Last test of basis   186001  2024-05-15 01:51:54 Z    0 days
Testing same since   186004  2024-05-15 08:40:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   46aa3031ae..319a5125ca  319a5125ca2649e6eb95670b4d721260025c187d -> master


From xen-devel-bounces@lists.xenproject.org Wed May 15 18:59:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 18:59:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722586.1126692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Jqu-0007DT-Px; Wed, 15 May 2024 18:59:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722586.1126692; Wed, 15 May 2024 18:59:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Jqu-0007DM-Ms; Wed, 15 May 2024 18:59:48 +0000
Received: by outflank-mailman (input) for mailman id 722586;
 Wed, 15 May 2024 18:59:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RHxB=MS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7Jqs-000735-Mk
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 18:59:46 +0000
Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com
 [2607:f8b0:4864:20::f2e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4aa57925-12ed-11ef-909d-e314d9c70b13;
 Wed, 15 May 2024 20:59:46 +0200 (CEST)
Received: by mail-qv1-xf2e.google.com with SMTP id
 6a1803df08f44-6a0f889877cso23932796d6.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 11:59:44 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a16068cee4sm65728846d6.8.2024.05.15.11.59.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 11:59:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4aa57925-12ed-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715799583; x=1716404383; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=V2/WvhCWtgZ5MsKAPwOX/iQz6ESjwDd7SAlTx3Kd59g=;
        b=OtJDf1Fa3pwa/gThi31JMkmomTsgXl05pSN8uGPuU4l/dLRr4VX90JFt2GLV7F9cgQ
         IQDuY33xR+xlMGqrwbXlXJXzHA/A6X3nqiLBT4KNeVxtOLW7S6fd9rW86kE/AZREdYLc
         Nz9chp5vIbcIE75SBjsh/in3AqE0bohXA0gnk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715799583; x=1716404383;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=V2/WvhCWtgZ5MsKAPwOX/iQz6ESjwDd7SAlTx3Kd59g=;
        b=SZoTk8aM3vHEoQLE68VUjUQRZBAf7yIbnhSJHe7QNSfcn+WElsnRtHOQlf0+8RRovn
         bVYjjjnFe9hozyjoAG0Sne7NX/4U/qhzFYYZcBP9tpPgYraIgG68XXHRYap/qSbp6XbU
         z5QwCV8XlUfwrEfMUmLlQaVzY68zWLyzX1KCAhXmPL/QUKvhMHGcDMpqUYdUdxTT9KpM
         C29M1rAvJmx95NgRSlpu5X3QySjFxEtzzaEAP0E9TGiNPAv/m5FzXp6933D587RcIpdj
         BxYwKUtQAx6Jwbyl7VTjKeXjMl/g7oYxrezaGV37xJnZ1zWogf8wbEiKieGy+ZaE/bDa
         M+Pw==
X-Gm-Message-State: AOJu0Yw7dFpMxuEuGAI9BKmbcmi8GBkkBGYVlnA6hrmaEP8F6EHlAr0F
	WalWhpB6qDquxKNVQu0jHjsR6H+zPKPV+q22zkTmWvcUhWwI7CbbAgI/jF2zTy8=
X-Google-Smtp-Source: AGHT+IElJVgK7hN8t25dFARsWPG5ihF7do8FCY2B2M57YN4f+sc37waMpQX1mZXLpyWutEvwGiJ/IA==
X-Received: by 2002:a05:6214:5a13:b0:6a0:d220:861f with SMTP id 6a1803df08f44-6a168244e10mr162380586d6.59.1715799583460;
        Wed, 15 May 2024 11:59:43 -0700 (PDT)
Message-ID: <f679bb5c-d1d7-4b24-9025-17419255a1d2@citrix.com>
Date: Wed, 15 May 2024 19:59:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/mtrr: avoid system wide rendezvous when
 setting AP MTRRs
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240513085925.59324-1-roger.pau@citrix.com>
 <2d245c04-3bf9-4b9d-ad02-e754dcbefa28@citrix.com>
 <dd689012-0554-43ae-b22b-035415ce3392@citrix.com> <ZkN4q6H591SeT6wH@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkN4q6H591SeT6wH@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 3:43 pm, Roger Pau Monné wrote:
> On Tue, May 14, 2024 at 02:50:18PM +0100, Andrew Cooper wrote:
>> On 14/05/2024 12:09 pm, Andrew Cooper wrote:
>>> On 13/05/2024 9:59 am, Roger Pau Monne wrote:
>>>> There's no point in forcing a system wide update of the MTRRs on all processors
>>>> when there are no changes to be propagated.  On AP startup it's only the AP
>>>> that needs to write the system wide MTRR values in order to match the rest of
>>>> the already online CPUs.
>>>>
>>>> We have occasionally seen the watchdog trigger during `xen-hptool cpu-online`
>>>> in one Intel Cascade Lake box with 448 CPUs due to the re-setting of the MTRRs
>>>> on all the CPUs in the system.
>>>>
>>>> While there adjust the comment to clarify why the system-wide resetting of the
>>>> MTRR registers is not needed for the purposes of mtrr_ap_init().
>>>>
>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> ---
>>>> For consideration for 4.19: it's a bugfix of a rare instance of the watchdog
>>>> triggering, but it's also a good performance improvement when performing
>>>> cpu-online.
>>>>
>>>> Hopefully runtime changes to MTRR will affect a single MSR at a time, lowering
>>>> the chance of the watchdog triggering due to the system-wide resetting of the
>>>> range.
>>> "Runtime" changes will only be during dom0 boot, if at all, but yes - it
>>> is restricted to a single MTRR at a time.
>>>
>>> It's XENPF_{add,del,read}_memtype, but it's only used by Classic Linux. 
>>> PVOps only issues read_memtype.
>>>
>>> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Actually no - this isn't safe in all cases.
>>
>> There are BIOSes which get MTRRs wrong, and with the APs having UC
>> covering a wider region than the BSP.
>>
>> In this case, creating consistency will alter the MTRRs on all CPUs
>> currently up, and we do need to perform the rendezvous in that case.
> I'm confused, the state that gets applied in mtrr_set_all() is not
> modified to match what's in the started AP registers.
>
> An AP starting with a different set of MTRR registers than the saved
> state will result in the MTRR state on the AP being changed, but not
> the Xen state stored in mtrr_state, and hence there will be no changes
> to synchronize.
>
>> There are 3 cases:
>>
>> 1) Nothing to do.  This is the overwhemlingly common case.
>> 2) Local changes only.  No broadcast, but we do need to enter CD mode.
>> 3) Remote changes needed.  Needs full broadcast.
> Please bear with me, but I don't think 3) is possible during AP
> bringup.  It's possible I'm missing a path where the differences in
> the started AP MTRR state are somehow reconciled with the cached MTRR
> state?

[Summarising a conversation on Matrix]

The problem case is when the BSP has an MTRR covering [$X, $X+2) and an
AP has has an MTRR covering [$X, $X+3).

This is a firmware bug (asymmetric settings), but it has been observed
in practice.  The resolution in this case ought to be to update all CPUs
to use [$X, $X+3), if that is the more UC direction.

However, it appears that Xen always resolves asymmetry like this by
choosing the BSP setting.  Therefore, (whether we should or not), we
don't have a case where observing an AP state results in a change of
state on other CPUs.

Therefore while case 3 exists in reality, we're not losing it as a side
effect of this patch.

So we'll take the improvement here and defer the other bugs to a future
juncture.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 15 20:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 20:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722610.1126733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7LId-0004wt-Iy; Wed, 15 May 2024 20:32:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722610.1126733; Wed, 15 May 2024 20:32:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7LId-0004wm-GS; Wed, 15 May 2024 20:32:31 +0000
Received: by outflank-mailman (input) for mailman id 722610;
 Wed, 15 May 2024 20:32:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ex/G=MS=flex--seanjc.bounces.google.com=32RtFZgYKCV4OA6JF8CKKCHA.8KITAJ-9ARAHHEOPO.TAJLNKFA8P.KNC@srs-se1.protection.inumbo.net>)
 id 1s7LIc-0004wg-0a
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 20:32:30 +0000
Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com
 [2607:f8b0:4864:20::449])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3e67b601-12fa-11ef-b4bb-af5377834399;
 Wed, 15 May 2024 22:32:27 +0200 (CEST)
Received: by mail-pf1-x449.google.com with SMTP id
 d2e1a72fcca58-6f441afba80so8253764b3a.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 13:32:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e67b601-12fa-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20230601; t=1715805146; x=1716409946; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id
         :reply-to;
        bh=iyv87Dbnyu7gm4af/LI0TOX4gLeF6QvKnfsIZOOyrfc=;
        b=mrcMteNs1xZy+Mfqg7mll9bhPraNijLUC+A5TJ7HzRNIwOwvBWq5NvM9PLDnCYE5Uw
         l+ApH7DYFY3FjcEtBocjQ+RA/qHzQBG0v0gPTre7rLWhUlKaMdAQCvIwtdLOtuRzSr0G
         K4OS1Czmat9IitkR+05XH8PD6lQYBzmUW7ej7iOUG+UyvFyxAmjn0nbpiv99zP0iGY0e
         3fsc48AlQo8eOGoaVkYbUnF/8h8GLj/9NN9Jhn0RlMoOpN4CvB9tpdTvTPdB4oOSDxoS
         XUbHfYj50bd6TNiKgFmQuFZutEF/xTPCwgbGPB9N2jdxdBEYOH7Rx7yotFSSQc2yq5tF
         C5UA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715805146; x=1716409946;
        h=content-transfer-encoding:cc:to:from:subject:message-id:references
         :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=iyv87Dbnyu7gm4af/LI0TOX4gLeF6QvKnfsIZOOyrfc=;
        b=dbwg79EU+Te9TTlI3hiRA2gnpVrhO3nAAC/XlweMWJtOlE4ZBW2ecZoWZAjH/1iPnV
         P0VZm3Z3HeNZ18eayuw1dN7qnflvGJ1zSQPwWosCxP7TNuaQP/spMUTeQZQNnNVm6j+V
         yvUqFmhB3xT4V0NolzKVuuPF7aZUpfHlpekNHfkxYfManG1xSq09fn2LaF/b066V3YRf
         HbgHzTXcpUkL/hVXdVXgrZhL2m7cEc8UmPdUaep6HXSranTipixYTOf0qB1hgNmXN2Ps
         poShF+MWXgytFwr9itq6F0GpBsTd7GrGn+E08HDSMVWiKILhOyiaevdN8WbXnOEMAxpH
         FYzA==
X-Forwarded-Encrypted: i=1; AJvYcCW6lDIiPhQYMWm5ulzdm4221yPIWP1v9LfVnaNFD3a3UI+e5S/ZZ53P0CIbAWurjAnCNgKNgTVaDgly//FhnH32AbxkeMQR5T6g5iBwI+Y=
X-Gm-Message-State: AOJu0YxIAEsSlbzEgSZTuWpH11O26VNeu3dfuzcsIaz6jB65Jg5UXoEh
	B8LF+IHlsZPIa+fDrse8dxYgAdMdYNsPpcAEFol0SpzkX00Trh8qqAMW93KFuDOQFQ1KxfLj5MG
	TUQ==
X-Google-Smtp-Source: AGHT+IECDBDOitjYW0H0HtQ+B5fogjE5oB/VbBha173B4uff3BsqArnMeFZozd93oJEEJOGvPRJpNAKBnf4=
X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37])
 (user=seanjc job=sendgmr) by 2002:a05:6a00:1d26:b0:6ea:baf6:57a3 with SMTP id
 d2e1a72fcca58-6f4e0431a08mr1169393b3a.6.1715805145969; Wed, 15 May 2024
 13:32:25 -0700 (PDT)
Date: Wed, 15 May 2024 13:32:24 -0700
In-Reply-To: <20240514.mai3Ahdoo2qu@digikod.net>
Mime-Version: 1.0
References: <20240503131910.307630-1-mic@digikod.net> <20240503131910.307630-4-mic@digikod.net>
 <ZjTuqV-AxQQRWwUW@google.com> <20240506.ohwe7eewu0oB@digikod.net>
 <ZjmFPZd5q_hEBdBz@google.com> <20240507.ieghomae0UoC@digikod.net>
 <ZjpTxt-Bxia3bRwB@google.com> <D15VQ97L5M8J.1TDNQE6KLW6JO@amazon.com> <20240514.mai3Ahdoo2qu@digikod.net>
Message-ID: <ZkUb2IWj4Z9FziCb@google.com>
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
From: Sean Christopherson <seanjc@google.com>
To: "=?utf-8?Q?Micka=C3=ABl_Sala=C3=BCn?=" <mic@digikod.net>
Cc: Nicolas Saenz Julienne <nsaenz@amazon.com>, Borislav Petkov <bp@alien8.de>, 
	Dave Hansen <dave.hansen@linux.intel.com>, "H . Peter Anvin" <hpa@zytor.com>, 
	Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, 
	Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, 
	Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, 
	Rick P Edgecombe <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>, 
	Angelina Vu <angelinavu@linux.microsoft.com>, 
	Anna Trikalinou <atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>, 
	Forrest Yuan Yu <yuanyu@google.com>, James Gowans <jgowans@amazon.com>, 
	James Morris <jamorris@linux.microsoft.com>, John Andersen <john.s.andersen@intel.com>, 
	"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>, 
	"Mihai =?utf-8?B?RG9uyJt1?=" <mdontu@bitdefender.com>, 
	"=?utf-8?B?TmljdciZb3IgQ8OuyJt1?=" <nicu.citu@icloud.com>, Thara Gopinath <tgopinath@microsoft.com>, 
	Trilok Soni <quic_tsoni@quicinc.com>, Wei Liu <wei.liu@kernel.org>, 
	Will Deacon <will@kernel.org>, Yu Zhang <yu.c.zhang@linux.intel.com>, 
	"=?utf-8?Q?=C8=98tefan_=C8=98icleru?=" <ssicleru@bitdefender.com>, dev@lists.cloudhypervisor.org, 
	kvm@vger.kernel.org, linux-hardening@vger.kernel.org, 
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-security-module@vger.kernel.org, qemu-devel@nongnu.org, 
	virtualization@lists.linux-foundation.org, x86@kernel.org, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 14, 2024, Micka=C3=ABl Sala=C3=BCn wrote:
> On Fri, May 10, 2024 at 10:07:00AM +0000, Nicolas Saenz Julienne wrote:
> > Development happens
> > https://github.com/vianpl/{linux,qemu,kvm-unit-tests} and the vsm-next
> > branch, but I'd advice against looking into it until we add some order
> > to the rework. Regardless, feel free to get in touch.
>=20
> Thanks for the update.
>=20
> Could we schedule a PUCK meeting to synchronize and help each other?
> What about June 12?

June 12th works on my end.


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:13:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:13:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722623.1126744 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Mrw-0002I5-NZ; Wed, 15 May 2024 22:13:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722623.1126744; Wed, 15 May 2024 22:13:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Mrw-0002Hy-Ke; Wed, 15 May 2024 22:13:04 +0000
Received: by outflank-mailman (input) for mailman id 722623;
 Wed, 15 May 2024 22:13:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Mrv-0002Hs-9o
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:13:03 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 496b380a-1308-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 00:13:00 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 58DCECE17D4;
 Wed, 15 May 2024 22:12:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BDD7C116B1;
 Wed, 15 May 2024 22:12:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 496b380a-1308-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715811176;
	bh=uEMbC0YIDzL0nk9QPXgGfz2S2MW5Shth3LoY2FW5OdQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Uo36pxTvhxl4mk4sUbbvA8Yn3iGTcGfFfIUr2Nt7dml8QFmHBMP2Jx860DxA3l+om
	 sUb8eSRdtGEhbYx9bwcQSEjBU5wXp07fIXsQyQjNd/kQD3xMAiMWCLdaU140qk4tEF
	 oTR3dZ2gPtHHs2Xx1tRKNlXs7w/hvtBuNmHHG8Yz/EzMITR6OwX0Y2rja2gnbztRog
	 sKOGXojwA6a6q+JvDxBdj4CwkdNF6XFjEJuM+Wq3Xvaf+Et6K/p+3Zm/CNrCcRSwfF
	 haSlMvA4wHdwvgiWsDuH5vqd5EwnUfdj+b5MI61P6pl0ju5g8BTkzmCWWJbuX63xYW
	 KG5UGO33JT6oQ==
Date: Wed, 15 May 2024 15:12:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>
Subject: Re: [XEN PATCH] automation/eclair_analysis: fully deviate MISRA C
 Rules 21.9 and 21.10
In-Reply-To: <6990027e12e108a2eaee7300931b1dd01c30795d.1715758910.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405151511520.2544314@ubuntu-linux-20-04-desktop>
References: <6990027e12e108a2eaee7300931b1dd01c30795d.1715758910.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Nicola Vetrini wrote:
> These rules are concerned with the use of facilities provided by the
> C Standard Library (qsort, bsearch for rule 21.9, and those provided
> by <time.h> for rule 21.10).
> 
> Xen provides in its source code its own implementation of some of these
> functions and macros, therefore a justification is provided for allowing
> uses of these functions in the project.
> 
> The rules are also marked as clean as a consequence.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  .../eclair_analysis/ECLAIR/deviations.ecl      | 14 ++++++++++++++
>  .../eclair_analysis/ECLAIR/monitored.ecl       |  2 ++
>  automation/eclair_analysis/ECLAIR/tagging.ecl  |  2 +-
>  docs/misra/deviations.rst                      | 18 ++++++++++++++++++
>  4 files changed, 35 insertions(+), 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> index 5eb7368a7322..1478c64a5af1 100644
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -477,6 +477,20 @@ leads to a violation of the Rule are deviated."
>  -config=MC3R1.R20.12,macros+={deliberate, "name(GENERATE_CASE)&&loc(file(deliberate_generate_case))"}
>  -doc_end
>  
> +#
> +# Series 21.
> +#
> +
> +-doc_begin="Xen does not use the functions provided by the Standard Library, but
> +implements a set of functions that share the same names as their Standard Library equivalent.
> +The implementation of these functions is available in source form, so the undefined, unspecified
> +or implementation-defined behaviors contemplated by the C Standard do not apply.
> +If some undefined or unspecified behavior does arise in the implementation, it
> +falls under the jurisdiction of other MISRA rules."
> +-config=MC3R1.R21.9,reports+={deliberate, "any()"}
> +-config=MC3R1.R21.10,reports+={deliberate, "any()"}
> +-doc_end
> +
>  #
>  # General
>  #
> diff --git a/automation/eclair_analysis/ECLAIR/monitored.ecl b/automation/eclair_analysis/ECLAIR/monitored.ecl
> index 69308ea51c01..9da709dc889c 100644
> --- a/automation/eclair_analysis/ECLAIR/monitored.ecl
> +++ b/automation/eclair_analysis/ECLAIR/monitored.ecl
> @@ -36,12 +36,14 @@
>  -enable=MC3R1.R20.4
>  -enable=MC3R1.R20.9
>  -enable=MC3R1.R2.1
> +-enable=MC3R1.R21.10
>  -enable=MC3R1.R21.13
>  -enable=MC3R1.R21.17
>  -enable=MC3R1.R21.18
>  -enable=MC3R1.R21.19
>  -enable=MC3R1.R21.20
>  -enable=MC3R1.R21.21
> +-enable=MC3R1.R21.9
>  -enable=MC3R1.R2.2
>  -enable=MC3R1.R22.2
>  -enable=MC3R1.R22.4
> diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
> index a14bda5033df..acea15f486a1 100644
> --- a/automation/eclair_analysis/ECLAIR/tagging.ecl
> +++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
> @@ -19,7 +19,7 @@
>  
>  -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
>  
> --service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
> +-service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
>  }
>  
>  -setq=target,getenv("XEN_TARGET_ARCH")
> diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
> index 161134771c77..b5450c38f66c 100644
> --- a/docs/misra/deviations.rst
> +++ b/docs/misra/deviations.rst
> @@ -411,6 +411,24 @@ Deviations related to MISRA C:2012 Rules:
>         construct is deviated only in Translation Units that present a violation
>         of the Rule due to uses of this macro.
>       - Tagged as `deliberate` for ECLAIR.
> +     
> +   * - R21.9
> +     - Xen does not use the `bsearch` and `qsort` functions provided by the C
> +       Standard Library, but provides in source form its own implementation,
> +       therefore any unspecified or undefined behavior associated to the
> +       functions provided by the Standard Library does not apply. Any such
> +       behavior that may exist in such functions is therefore under the
> +       jurisdiction of other MISRA C rules.
> +     - Project-wide deviation, tagged as `deliberate` for ECLAIR.
> +
> +   * - R21.10
> +     - Xen does not use the facilities provided by the `\<time.h\>` provided by the C
> +       Standard Library, but provides in source form its own implementation,
> +       therefore any unspecified, undefined or implementation-defined behavior
> +       associated to the functions provided by the Standard Library does not
> +       apply. Any such behavior that may exist in such functions is therefore
> +       under the jurisdiction of other MISRA C rules.
> +     - Project-wide deviation, tagged as `deliberate` for ECLAIR.
>  
>  Other deviations:
>  -----------------
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:19:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:19:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722627.1126754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Mxg-00039t-BK; Wed, 15 May 2024 22:19:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722627.1126754; Wed, 15 May 2024 22:19:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Mxg-00039m-86; Wed, 15 May 2024 22:19:00 +0000
Received: by outflank-mailman (input) for mailman id 722627;
 Wed, 15 May 2024 22:18:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Mxe-00039d-SJ
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:18:58 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e7bf5d4-1309-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 00:18:56 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 504D660AAD;
 Wed, 15 May 2024 22:18:55 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C0B5CC116B1;
 Wed, 15 May 2024 22:18:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e7bf5d4-1309-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715811535;
	bh=hWPGgn2veiW2Ir9WKsDAPLO5QYf+tJ46ZxyS1oYFi/s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fLE2wcivHd34N8/JCEQHwnXrRDLKVrnBQEhD79p+w8c5V05OAKDzxqjnDKJjYUEvw
	 hUpuIPZ2YTknBqDZkIFbUKL1UtvKv+tHWzsxv7ANt8ntXhGaFU59lKq/YlWeqvouEE
	 9VH0vKPxGsZSydsRE9SP0Pc1eJm7Fd/wJaHK0g8cpSzI14iGxCiTYnAWPN67i1kAFq
	 f1o0Ssvq/KgDksR3Nl2cW0SpatLWhU2IoOp+x12S4s/AaJ0CyLgdPHepFPImIVsj8a
	 pXJw4fYXSW3t/oRpwJtoQsULLiJ7D1qRKB8+4eJ4v98U4GPLUBzmiJwRvh/93L+504
	 GbJqnYPflZfHw==
Date: Wed, 15 May 2024 15:18:52 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Michal Orzel <michal.orzel@amd.com>, roberto.bagnara@bugseng.com, 
    consulting@bugseng.com, xen-devel@lists.xenproject.org
Subject: Re: [PATCH] docs/misra: add D4.12
In-Reply-To: <bd1072c5-1533-46b7-b6c8-fab1c0f80ab3@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405151513530.2544314@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405141611170.2544314@ubuntu-linux-20-04-desktop> <bd1072c5-1533-46b7-b6c8-fab1c0f80ab3@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Jan Beulich wrote:
> On 15.05.2024 01:15, Stefano Stabellini wrote:
> > Add D4.12 with the same explanation as the rules of the R21 series.
> > D4.12 refers to the standard library memory allocation functions and
> > similar third party libraries with memory allocation functions. It
> > doesn't refer to the in-tree implementation we have in Xen which is
> > subject to MISRA C rules and MISRA C scanning.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > 
> > diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> > index 80e5e972ad..bc8506add4 100644
> > --- a/docs/misra/rules.rst
> > +++ b/docs/misra/rules.rst
> > @@ -76,6 +76,11 @@ maintainers if you want to suggest a change.
> >         considered libraries from MISRA C point of view as they are
> >         imported in source form)
> >  
> > +   * - `Dir 4.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/D_04_12.c>`_
> > +     - Required
> > +     - Dynamic memory allocation shall not be used
> > +     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
> 
> I'm having trouble connecting this remark with the directive. We do have
> dynamic memory allocation routines, and we use them. It doesn't really
> matter that they don't come from an external library, does it?

Similarly to the 21.x rules series, it makes a difference if they are
external libraries or code within the project. The rule points out that
the standard library memory allocation functions can lead to undefined
behavior. On the other hand, our own implementation under xen.git is
subject to MISRA C scanning and all the other MISRA C rules.

The example in the link above, shows a use-after-free error that in our
case it should be caught by other MISRA C rules scanning.


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:30:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:30:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722632.1126764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7N90-00063j-GX; Wed, 15 May 2024 22:30:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722632.1126764; Wed, 15 May 2024 22:30:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7N90-00063c-Cd; Wed, 15 May 2024 22:30:42 +0000
Received: by outflank-mailman (input) for mailman id 722632;
 Wed, 15 May 2024 22:30:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7N8z-00063W-7a
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:30:41 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c0fe8f32-130a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 00:30:38 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5A58A615F7;
 Wed, 15 May 2024 22:30:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3D22C116B1;
 Wed, 15 May 2024 22:30:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0fe8f32-130a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715812237;
	bh=BhaxKrSJ2V074gHXzgPwRKtFyji5OeLt5R3Sl21D4t0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DC2NxJVEy31yOLjZq9huooHYPwBBmjBjscRNb0ZIy3x6OEpnlOzGNuQT92ALOTRRV
	 1NNhrZjfbKC0lEtLDZOmnnkdgi080nHff9IsAxUVb/vA1ieHqy5fJddxyiP7BlRu07
	 HG78AuDwrZh9dO0iHbi3gN/jF1hi1vGY/F/9TLhql3jUuWpIr9rRHNB99sBSVcfHte
	 0joH++aTxyWzb7rwjAJ0PSCxJrYrGnt6ObW8dQm/61+/4UlfEcEzW7mcCJgjOv2vxt
	 OcBtySvKnw9a3yyXI8RbwBEuusgl/lGV/8pqUE8U0okm/NYcsTjYEWnd+a8Bzvuj3M
	 kdb3NJxOwTmtg==
Date: Wed, 15 May 2024 15:30:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] drivers/xen: Improve the late XenStore init protocol
In-Reply-To: <20240515014330.1044617-1-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405151524270.2544314@ubuntu-linux-20-04-desktop>
References: <20240515014330.1044617-1-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Henry Wang wrote:
> Currently, the late XenStore init protocol is only triggered properly
> for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
> case that XenStore interface is allocated but not ready (the connection
> status is not XENSTORE_CONNECTED), Linux should also wait until the
> XenStore is set up properly.
> 
> Introduce a macro to describe the XenStore interface is ready, use
> it in xenbus_probe_initcall() and xenbus_probe() to select the code
> path of doing the late XenStore init protocol or not.
> 
> Take the opportunity to enhance the check of the allocated XenStore
> interface can be properly mapped, and return error early if the
> memremap() fails.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Please add a Fixes: tag


> ---
>  drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++++++++++-----
>  1 file changed, 16 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
> index 3205e5d724c8..8aec0ed1d047 100644
> --- a/drivers/xen/xenbus/xenbus_probe.c
> +++ b/drivers/xen/xenbus/xenbus_probe.c
> @@ -72,6 +72,10 @@ EXPORT_SYMBOL_GPL(xen_store_evtchn);
>  struct xenstore_domain_interface *xen_store_interface;
>  EXPORT_SYMBOL_GPL(xen_store_interface);
>  
> +#define XS_INTERFACE_READY \
> +	((xen_store_interface != NULL) && \
> +	 (xen_store_interface->connection == XENSTORE_CONNECTED))
> +
>  enum xenstore_init xen_store_domain_type;
>  EXPORT_SYMBOL_GPL(xen_store_domain_type);
>  
> @@ -751,9 +755,10 @@ static void xenbus_probe(void)
>  {
>  	xenstored_ready = 1;
>  
> -	if (!xen_store_interface) {
> -		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
> -					       XEN_PAGE_SIZE, MEMREMAP_WB);
> +	if (!xen_store_interface || XS_INTERFACE_READY) {
> +		if (!xen_store_interface)

These two nested if's don't make sense to me. If XS_INTERFACE_READY
succeeds, it means that  ((xen_store_interface != NULL) &&
(xen_store_interface->connection == XENSTORE_CONNECTED)).

So it is not possible that xen_store_interface == NULL immediately
after. Right?


> +			xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
> +						       XEN_PAGE_SIZE, MEMREMAP_WB);
>  		/*
>  		 * Now it is safe to free the IRQ used for xenstore late
>  		 * initialization. No need to unbind: it is about to be
> @@ -822,7 +827,7 @@ static int __init xenbus_probe_initcall(void)
>  	if (xen_store_domain_type == XS_PV ||
>  	    (xen_store_domain_type == XS_HVM &&
>  	     !xs_hvm_defer_init_for_callback() &&
> -	     xen_store_interface != NULL))
> +	     XS_INTERFACE_READY))
>  		xenbus_probe();
>  
>  	/*
> @@ -831,7 +836,7 @@ static int __init xenbus_probe_initcall(void)
>  	 * started, then probe.  It will be triggered when communication
>  	 * starts happening, by waiting on xb_waitq.
>  	 */
> -	if (xen_store_domain_type == XS_LOCAL || xen_store_interface == NULL) {
> +	if (xen_store_domain_type == XS_LOCAL || !XS_INTERFACE_READY) {
>  		struct task_struct *probe_task;
>  
>  		probe_task = kthread_run(xenbus_probe_thread, NULL,
> @@ -1014,6 +1019,12 @@ static int __init xenbus_init(void)
>  			xen_store_interface =
>  				memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>  					 XEN_PAGE_SIZE, MEMREMAP_WB);
> +			if (!xen_store_interface) {
> +				pr_err("%s: cannot map HVM_PARAM_STORE_PFN=%llx\n",
> +				       __func__, v);
> +				err = -ENOMEM;

I think this should -EINVAL


> +				goto out_error;
> +			}
>  			if (xen_store_interface->connection != XENSTORE_CONNECTED)
>  				wait = true;
>  		}
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:43:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:43:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722636.1126774 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NL8-0008R9-Ie; Wed, 15 May 2024 22:43:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722636.1126774; Wed, 15 May 2024 22:43:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NL8-0008R2-E7; Wed, 15 May 2024 22:43:14 +0000
Received: by outflank-mailman (input) for mailman id 722636;
 Wed, 15 May 2024 22:43:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NL7-0008Qs-4l
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:43:13 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f6c4a01-130c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 00:43:09 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0860CCE17E3;
 Wed, 15 May 2024 22:43:04 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69AB4C116B1;
 Wed, 15 May 2024 22:43:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f6c4a01-130c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715812983;
	bh=3B2QPX/ga/0AMCH1IWm6CBpgkx7uEqSubo5omjdhkDk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=u24GfXM+55qXQfRDSHTj7tOt/UBYWn8d03ITH9yqoUOUz0DE9ym3wps5ouvjLqkD8
	 a1Yo1fBSyHCnYd7/XXMFHpIEPBIlWziMzzM617W3iH/CCwKUbob/SfKDQQFBCqO+I5
	 cSJWq9JMkxCPG2hywt13urQRLcmrXjnr3+trK26+x7qCWykCHZ7DonYu1hNcxrGY5N
	 LeewcLRKzPkQ2dRgR1wnQn16N4UnhjQowcp7q9e7ariyXbSrCx8sXBgnvGYt5vdlzW
	 qGUA4tymb2FEjFyBe28qFtit/JYt+Ib2P9ldeKMU/6Tj02VYyi33wFsIzhcqI7c4wD
	 sgEZAyaSOmU4A==
Date: Wed, 15 May 2024 15:42:59 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bjorn Helgaas <bhelgaas@google.com>, 
    "Rafael J . Wysocki" <rafael@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org, 
    linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, 
    Huang Rui <Ray.Huang@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC KERNEL PATCH v7 2/2] xen/privcmd: Add new syscall to get
 gsi from dev
In-Reply-To: <20240515065011.13797-3-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405151537430.2544314@ubuntu-linux-20-04-desktop>
References: <20240515065011.13797-1-Jiqian.Chen@amd.com> <20240515065011.13797-3-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Jiqian Chen wrote:
> In PVH dom0, it uses the linux local interrupt mechanism,
> when it allocs irq for a gsi, it is dynamic, and follow
> the principle of applying first, distributing first. And
> the irq number is alloced from small to large, but the
> applying gsi number is not, may gsi 38 comes before gsi 28,
> it causes the irq number is not equal with the gsi number.
> And when passthrough a device, QEMU will use device's gsi
> number to do pirq mapping, but the gsi number is got from
> file /sys/bus/pci/devices/<sbdf>/irq, irq!= gsi, so it will
> fail when mapping.
> And in current linux codes, there is no method to get gsi
> for userspace.
> 
> For above purpose, record gsi of pcistub devices when init
> pcistub and add a new syscall into privcmd to let userspace
> can get gsi when they have a need.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>  drivers/xen/privcmd.c              | 28 ++++++++++++++++++++++
>  drivers/xen/xen-pciback/pci_stub.c | 38 +++++++++++++++++++++++++++---
>  include/uapi/xen/privcmd.h         |  7 ++++++
>  include/xen/acpi.h                 |  2 ++
>  4 files changed, 72 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> index 67dfa4778864..5953a03b5cb0 100644
> --- a/drivers/xen/privcmd.c
> +++ b/drivers/xen/privcmd.c
> @@ -45,6 +45,9 @@
>  #include <xen/page.h>
>  #include <xen/xen-ops.h>
>  #include <xen/balloon.h>
> +#ifdef CONFIG_ACPI
> +#include <xen/acpi.h>
> +#endif
>  
>  #include "privcmd.h"
>  
> @@ -842,6 +845,27 @@ static long privcmd_ioctl_mmap_resource(struct file *file,
>  	return rc;
>  }
>  
> +static long privcmd_ioctl_gsi_from_dev(struct file *file, void __user *udata)
> +{
> +	struct privcmd_gsi_from_dev kdata;
> +
> +	if (copy_from_user(&kdata, udata, sizeof(kdata)))
> +		return -EFAULT;
> +
> +#ifdef CONFIG_ACPI
> +	kdata.gsi = pcistub_get_gsi_from_sbdf(kdata.sbdf);
> +	if (kdata.gsi == -1)
> +		return -EINVAL;
> +#else
> +	kdata.gsi = -1;

Should we return an error instead, like -EINVAL, to make the behavior
more similar to the CONFIG_ACPI case?


> +#endif
> +
> +	if (copy_to_user(udata, &kdata, sizeof(kdata)))
> +		return -EFAULT;
> +
> +	return 0;
> +}
> +
>  #ifdef CONFIG_XEN_PRIVCMD_EVENTFD
>  /* Irqfd support */
>  static struct workqueue_struct *irqfd_cleanup_wq;
> @@ -1529,6 +1553,10 @@ static long privcmd_ioctl(struct file *file,
>  		ret = privcmd_ioctl_ioeventfd(file, udata);
>  		break;
>  
> +	case IOCTL_PRIVCMD_GSI_FROM_DEV:
> +		ret = privcmd_ioctl_gsi_from_dev(file, udata);
> +		break;
> +
>  	default:
>  		break;
>  	}
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index 2b90d832d0a7..4b62b4d377a9 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -56,6 +56,9 @@ struct pcistub_device {
>  
>  	struct pci_dev *dev;
>  	struct xen_pcibk_device *pdev;/* non-NULL if struct pci_dev is in use */
> +#ifdef CONFIG_ACPI
> +	int gsi;
> +#endif
>  };
>  
>  /* Access to pcistub_devices & seized_devices lists and the initialize_devices
> @@ -88,6 +91,9 @@ static struct pcistub_device *pcistub_device_alloc(struct pci_dev *dev)
>  
>  	kref_init(&psdev->kref);
>  	spin_lock_init(&psdev->lock);
> +#ifdef CONFIG_ACPI
> +	psdev->gsi = -1;
> +#endif
>  
>  	return psdev;
>  }
> @@ -220,6 +226,25 @@ static struct pci_dev *pcistub_device_get_pci_dev(struct xen_pcibk_device *pdev,
>  	return pci_dev;
>  }
>  
> +#ifdef CONFIG_ACPI
> +int pcistub_get_gsi_from_sbdf(unsigned int sbdf)
> +{
> +	struct pcistub_device *psdev;
> +	int domain = sbdf >> 16;
> +	int bus = (sbdf >> 8) & 0xff;
> +	int slot = (sbdf >> 3) & 0x1f;
> +	int func = sbdf & 0x7;

you can use PCI_DEVFN PCI_SLOT PCI_FUNC pci_domain_nr instead of open
coding.


> +
> +	psdev = pcistub_device_find(domain, bus, slot, func);
> +
> +	if (!psdev)
> +		return -1;
> +
> +	return psdev->gsi;
> +}
> +EXPORT_SYMBOL_GPL(pcistub_get_gsi_from_sbdf);
> +#endif
> +
>  struct pci_dev *pcistub_get_pci_dev_by_slot(struct xen_pcibk_device *pdev,
>  					    int domain, int bus,
>  					    int slot, int func)
> @@ -367,14 +392,20 @@ static int pcistub_match(struct pci_dev *dev)
>  	return found;
>  }
>  
> -static int pcistub_init_device(struct pci_dev *dev)
> +static int pcistub_init_device(struct pcistub_device *psdev)
>  {
>  	struct xen_pcibk_dev_data *dev_data;
> +	struct pci_dev *dev;
>  #ifdef CONFIG_ACPI
>  	int gsi, trigger, polarity;
>  #endif
>  	int err = 0;
>  
> +	if (!psdev)
> +		return -EINVAL;
> +
> +	dev = psdev->dev;
> +
>  	dev_dbg(&dev->dev, "initializing...\n");
>  
>  	/* The PCI backend is not intended to be a module (or to work with
> @@ -448,6 +479,7 @@ static int pcistub_init_device(struct pci_dev *dev)
>  		dev_err(&dev->dev, "Fail to get gsi info!\n");
>  		goto config_release;
>  	}
> +	psdev->gsi = gsi;
>  
>  	if (xen_initial_domain() && xen_pvh_domain()) {
>  		err = xen_pvh_setup_gsi(gsi, trigger, polarity);
> @@ -495,7 +527,7 @@ static int __init pcistub_init_devices_late(void)
>  
>  		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
>  
> -		err = pcistub_init_device(psdev->dev);
> +		err = pcistub_init_device(psdev);
>  		if (err) {
>  			dev_err(&psdev->dev->dev,
>  				"error %d initializing device\n", err);
> @@ -565,7 +597,7 @@ static int pcistub_seize(struct pci_dev *dev,
>  		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
>  
>  		/* don't want irqs disabled when calling pcistub_init_device */
> -		err = pcistub_init_device(psdev->dev);
> +		err = pcistub_init_device(psdev);
>  
>  		spin_lock_irqsave(&pcistub_devices_lock, flags);
>  
> diff --git a/include/uapi/xen/privcmd.h b/include/uapi/xen/privcmd.h
> index 8b8c5d1420fe..220e7670a113 100644
> --- a/include/uapi/xen/privcmd.h
> +++ b/include/uapi/xen/privcmd.h
> @@ -126,6 +126,11 @@ struct privcmd_ioeventfd {
>  	__u8 pad[2];
>  };
>  
> +struct privcmd_gsi_from_dev {
> +	__u32 sbdf;
> +	int gsi;
> +};
> +
>  /*
>   * @cmd: IOCTL_PRIVCMD_HYPERCALL
>   * @arg: &privcmd_hypercall_t
> @@ -157,5 +162,7 @@ struct privcmd_ioeventfd {
>  	_IOW('P', 8, struct privcmd_irqfd)
>  #define IOCTL_PRIVCMD_IOEVENTFD					\
>  	_IOW('P', 9, struct privcmd_ioeventfd)
> +#define IOCTL_PRIVCMD_GSI_FROM_DEV				\
> +	_IOC(_IOC_NONE, 'P', 10, sizeof(struct privcmd_gsi_from_dev))
>  
>  #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
> diff --git a/include/xen/acpi.h b/include/xen/acpi.h
> index 9b50027113f3..0bf5f4884456 100644
> --- a/include/xen/acpi.h
> +++ b/include/xen/acpi.h
> @@ -83,4 +83,6 @@ int xen_acpi_get_gsi_info(struct pci_dev *dev,
>  						  int *gsi_out,
>  						  int *trigger_out,
>  						  int *polarity_out);
> +
> +int pcistub_get_gsi_from_sbdf(unsigned int sbdf);
>  #endif	/* _XEN_ACPI_H */
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:48:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:48:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722642.1126784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NPs-0000an-3Z; Wed, 15 May 2024 22:48:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722642.1126784; Wed, 15 May 2024 22:48:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NPs-0000ag-0I; Wed, 15 May 2024 22:48:08 +0000
Received: by outflank-mailman (input) for mailman id 722642;
 Wed, 15 May 2024 22:48:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NPq-0000aa-Ow
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:48:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 302a992b-130d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 00:48:04 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DEE536149C;
 Wed, 15 May 2024 22:48:02 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 333A2C116B1;
 Wed, 15 May 2024 22:48:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 302a992b-130d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715813282;
	bh=Z3h8GZPro4Gg/YXBHF3gVOMMrtbcUlssNlWYhZennmI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MmXEba/9yr1+M4up9V6EyZPnVRZe62ibYuQwgwYqO08vY99OvVOZxylZdOfVV2Uau
	 VgpauAeuaeKgKkTL6ly8CHStwzwRvw8SOM2i1Lm6/av9OLsyviU9r0oeriCUDHPafq
	 8qMRZe600w+JoO7XmAMcd1zkoGRtkcDGBmo03sTAtAMXhddoihjKgf8o+pDljR47VE
	 MfmBOn5Vz4HnS6zXEq9Ss+CNDJxjBB5YM31EMZvCa5hm2Y/ij6HO4eAwmPtTw18rvb
	 jXHzidch/tM+abrFlPfPQXls08seWrfVPP1cLs6irt4dGju/a2/uzk4aLV/EDWE6Xa
	 atMG5MEkxKFqw==
Date: Wed, 15 May 2024 15:47:58 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, nicola.vetrini@bugseng.com, 
    consulting@bugseng.com
Subject: Re: [PATCH] lib/strtoul: fix MISRA R10.2 violation
In-Reply-To: <a3fa7be4-48ce-4b68-a8f7-f94df58d97de@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405151547260.2544314@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405131729180.2544314@ubuntu-linux-20-04-desktop> <16990192-aceb-408f-9247-45f3b1f58e0a@suse.com> <alpine.DEB.2.22.394.2405141540140.2544314@ubuntu-linux-20-04-desktop> <a3fa7be4-48ce-4b68-a8f7-f94df58d97de@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Jan Beulich wrote:
> On 15.05.2024 00:52, Stefano Stabellini wrote:
> > On Tue, 14 May 2024, Jan Beulich wrote:
> >> On 14.05.2024 02:32, Stefano Stabellini wrote:
> >>> Fix last violation of R10.2 by casting the result of toupper to plain
> >>> char. Note that we don't want to change toupper itself as it is a legacy
> >>> interface and it would cause more issues.
> >>
> >> Can you point me at a single example where a new issue would arise? All
> >> places I've spotted (including tolower() uses) would appear to benefit
> >> from changing toupper() / tolower() themselves. Further, since they are
> >> both wrapper macros only anyway, if any concern remained, fiddling with
> >> the wrapper macros while leaving alone the underlying inline functions
> >> would allow any such use site to simply be switched to using the inline
> >> functions directly. As said, from looking at it I don't expect that
> >> would be necessary, so instead I'd rather hope that eventually we can
> >> do away with the wrapper macros, renaming the inline functions
> >> accordingly.
> > 
> > If we change __toupper to return a plain char, then there are a few
> > other things we need to change for consistency, see below. To be honest
> > I thought it would cause more problems. I am OK to go with that if you
> > all agree. (Nicola please have a look in case this introduces more
> > issues elsewhere.)
> > 
> > 
> > diff --git a/xen/include/xen/ctype.h b/xen/include/xen/ctype.h
> > index 6dec944a37..6a6854e01c 100644
> > --- a/xen/include/xen/ctype.h
> > +++ b/xen/include/xen/ctype.h
> > @@ -15,9 +15,9 @@
> >  #define _X	0x40	/* hex digit */
> >  #define _SP	0x80	/* hard space (0x20) */
> >  
> > -extern const unsigned char _ctype[];
> > +extern const char _ctype[];
> 
> Why would this be needed? I can't see a connection to toupper() / tolower().
> 
> > -#define __ismask(x) (_ctype[(int)(unsigned char)(x)])
> > +#define __ismask(x) (_ctype[(int)(char)(x)])
> 
> This almost certainly is wrong. Whether plain char is signed or unsigned is
> left to the compiler, and it being signed would result in possibly negative
> array indexes. Again I can't see a connection to the issue at hand.
> 
> > @@ -34,14 +34,14 @@ extern const unsigned char _ctype[];
> >  #define isascii(c) (((unsigned char)(c))<=0x7f)
> >  #define toascii(c) (((unsigned char)(c))&0x7f)
> >  
> > -static inline unsigned char __tolower(unsigned char c)
> > +static inline char __tolower(char c)
> >  {
> >  	if (isupper(c))
> >  		c -= 'A'-'a';
> >  	return c;
> >  }
> >  
> > -static inline unsigned char __toupper(unsigned char c)
> > +static inline char __toupper(char c)
> >  {
> >  	if (islower(c))
> >  		c -= 'a'-'A';
> 
> This isn't what I had suggested. First I said to leave alone the double-
> underscore prefixed functions, and only touch the wrapper macros (as a
> precaution in case any use site exists which relies on present behavior).
> And then I didn't suggest to alter parameter types; only the return type
> would need adjustment, I think, for what you're aiming at:
> 
> #define tolower(c) ((char)__tolower(c))
> #define toupper(c) ((char)__toupper(c))

Oh I see. This is much more similar to the original suggestion from
Bugseng. Let me send a v2.


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:52:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:52:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722645.1126793 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NTo-0002bs-JL; Wed, 15 May 2024 22:52:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722645.1126793; Wed, 15 May 2024 22:52:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NTo-0002bl-GN; Wed, 15 May 2024 22:52:12 +0000
Received: by outflank-mailman (input) for mailman id 722645;
 Wed, 15 May 2024 22:52:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NTm-0002bf-J6
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 22:52:10 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c23560c1-130d-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 00:52:09 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 48DF1614C5;
 Wed, 15 May 2024 22:52:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7FAACC116B1;
 Wed, 15 May 2024 22:52:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c23560c1-130d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715813528;
	bh=yq9aJoe7mPRaP2y/18GJw006zYIktPsyCWvFrHtNf4A=;
	h=Date:From:To:cc:Subject:From;
	b=R9VgG3rV/tBB+KTwEnCAO5FBDL+EkermOybBCbWvtQf5M488en5V+KzpmwGX+TmJA
	 xC02yINB2owbuDPAPY/M+i2flFh/PALWXQuVfG3O92BYNprK6qhEYRANSwmL4xL85A
	 MdG//8imY7VVhhGMsnVoYyzAj5e9OJDgzgRDjS5kqhih9owTmMWBxeC79M++BtScdy
	 f5ZJJ74ZAaWlJT2QrSEiRGDg9oHmkLlkgJ4BOi18dJC6dV7ev1h1tiHCdMao8cLyeu
	 SxVHNaZdJyzoK8GKZH9DtcUUatfpzlSbAJ08fR9GESxfL7BjiW71Wfnwjg/dP1Z79w
	 cLihdRBv9wjUw==
Date: Wed, 15 May 2024 15:52:04 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>, 
    nicola.vetrini@bugseng.com, consulting@bugseng.com, jbeulich@suse.com
Subject: [PATCH v2] include/ctype.h: fix MISRA R10.2 violation 
Message-ID: <alpine.DEB.2.22.394.2405151549020.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

The value returned by __toupper is used in arithmetic operations causing
MISRA C 10.2 violations. Cast to plain char in the toupper macro. Also
do the same in tolower for consistency.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
Changes in v2:
- fix the toupper macro instead

diff --git a/xen/include/xen/ctype.h b/xen/include/xen/ctype.h
index 6dec944a37..773ac27aa4 100644
--- a/xen/include/xen/ctype.h
+++ b/xen/include/xen/ctype.h
@@ -48,7 +48,7 @@ static inline unsigned char __toupper(unsigned char c)
 	return c;
 }
 
-#define tolower(c) __tolower(c)
-#define toupper(c) __toupper(c)
+#define tolower(c) ((char)__tolower(c))
+#define toupper(c) ((char)__toupper(c))
 
 #endif


From xen-devel-bounces@lists.xenproject.org Wed May 15 22:55:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 22:55:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722648.1126805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NWo-00039K-2J; Wed, 15 May 2024 22:55:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722648.1126805; Wed, 15 May 2024 22:55:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NWn-00039F-T7; Wed, 15 May 2024 22:55:17 +0000
Received: by outflank-mailman (input) for mailman id 722648;
 Wed, 15 May 2024 22:55:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7NWn-000395-AH; Wed, 15 May 2024 22:55:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7NWn-00049G-8j; Wed, 15 May 2024 22:55:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7NWm-0007zn-VP; Wed, 15 May 2024 22:55:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7NWm-0005QF-Ur; Wed, 15 May 2024 22:55:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Go/qdM1Z5BdRmihFueQz5XAdshCcxRDVTbIohM7Qq/E=; b=jrK/aNEKT73dq12jwQooRwtVXJ
	zCQpaJPXMKW4t1JwXA42l1nW/wk0nfqDSlZh945Cm0OjYANt9CrKsU53/orXwDig4OywCqBJ+fiRx
	FC8XsRB1O7NPGpzR+FFxkYIlNYxvRENkxdCZC7dXEUWcVBw0bXLpRig7wNOBfAyOP5TA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186006-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186006: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
X-Osstest-Versions-That:
    xen=319a5125ca2649e6eb95670b4d721260025c187d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 May 2024 22:55:16 +0000

flight 186006 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186006/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
baseline version:
 xen                  319a5125ca2649e6eb95670b4d721260025c187d

Last test of basis   185999  2024-05-14 23:02:13 Z    0 days
Testing same since   186006  2024-05-15 20:04:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   319a5125ca..abd00b037d  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:08:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:08:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722657.1126814 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Njo-0005RO-2x; Wed, 15 May 2024 23:08:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722657.1126814; Wed, 15 May 2024 23:08:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Njo-0005RH-0M; Wed, 15 May 2024 23:08:44 +0000
Received: by outflank-mailman (input) for mailman id 722657;
 Wed, 15 May 2024 23:08:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Njm-0005RB-3C
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:08:42 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f1f9c52-1310-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:08:39 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 5D735CE17F2;
 Wed, 15 May 2024 23:08:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED724C116B1;
 Wed, 15 May 2024 23:08:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f1f9c52-1310-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715814512;
	bh=XqjCuxiB2MnS1WYIMeDQH7gJ6WD9EqRTdV+F492/gLw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VWypi5lBK2x7yQ0NI1MMlBECA0+HSqVfaNaMDHLNjucNrkXxCM8VrigYggHzitAIA
	 N4O5OnwFrkHb265DvcSHDoSEePIJ2LSHn/9zH9ck+tQ7+JK8Q1fLW2X5UUanfaANLd
	 KBcko0Eue4dM5TTYB+oW9Xvhb7X6czPsx52iBjV0FZy/hP5N5JMKxuKn/3FT/Rs756
	 BWrq5QnpbWfD7C/64/udGSu2n2gwwpoza5/oPAxQX/W1X/ZVp7B08A/xGQ7XB/EUDn
	 BWjYZeQuCtnK84i7M4UWGPsamo1o6oHthmo1Fx+/XQza4D55f18c4/xxPGV2nAAqdo
	 BQJo+rNMlFynQ==
Date: Wed, 15 May 2024 16:08:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v5 6/8] xen: mapcache: Pass the ram_addr offset to
 xen_map_cache()
In-Reply-To: <20240503014449.1046238-7-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151605550.2544314@ubuntu-linux-20-04-desktop>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com> <20240503014449.1046238-7-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Pass the ram_addr offset to xen_map_cache.
> This is in preparation for adding grant mappings that need
> to compute the address within the RAMBlock.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---
>  hw/xen/xen-mapcache.c         | 16 +++++++++++-----
>  include/sysemu/xen-mapcache.h |  2 ++
>  system/physmem.c              |  9 +++++----
>  3 files changed, 18 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index ec95445696..26bc38a9e3 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
>                               void *vaddr,
>                               hwaddr size,
>                               hwaddr address_index,
> -                             bool dummy)
> +                             bool dummy,
> +                             ram_addr_t ram_offset)
>  {
>      uint8_t *vaddr_base;
>      xen_pfn_t *pfns;
> @@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
>  
>  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>                                         hwaddr phys_addr, hwaddr size,
> +                                       ram_addr_t ram_offset,
>                                         uint8_t lock, bool dma, bool is_write)
>  {
>      MapCacheEntry *entry, *pentry = NULL,
> @@ -337,14 +339,16 @@ tryagain:
>      if (!entry) {
>          entry = g_new0(MapCacheEntry, 1);
>          pentry->next = entry;
> -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> +                         ram_offset);
>      } else if (!entry->lock) {
>          if (!entry->vaddr_base || entry->paddr_index != address_index ||
>                  entry->size != cache_size ||
>                  !test_bits(address_offset >> XC_PAGE_SHIFT,
>                      test_bit_size >> XC_PAGE_SHIFT,
>                      entry->valid_mapping)) {
> -            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
> +            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> +                             ram_offset);
>          }
>      }
>  
> @@ -391,13 +395,15 @@ tryagain:
>  
>  uint8_t *xen_map_cache(MemoryRegion *mr,
>                         hwaddr phys_addr, hwaddr size,
> +                       ram_addr_t ram_addr_offset,
>                         uint8_t lock, bool dma,
>                         bool is_write)
>  {
>      uint8_t *p;
>  
>      mapcache_lock(mapcache);
> -    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
> +    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> +                               lock, dma, is_write);
>      mapcache_unlock(mapcache);
>      return p;
>  }
> @@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
>  
>      xen_remap_bucket(mc, entry, entry->vaddr_base,
> -                     cache_size, address_index, false);
> +                     cache_size, address_index, false, new_phys_addr);

Everything else makes sense, but I don't understand how can it be that
new_phys_addr is the block->offset here?


>      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
>                  test_bit_size >> XC_PAGE_SHIFT,
>                  entry->valid_mapping)) {
> diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> index 1ec9e66752..b5e3ea1bc0 100644
> --- a/include/sysemu/xen-mapcache.h
> +++ b/include/sysemu/xen-mapcache.h
> @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
>  void xen_map_cache_init(phys_offset_to_gaddr_t f,
>                          void *opaque);
>  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
> +                       ram_addr_t ram_addr_offset,
>                         uint8_t lock, bool dma,
>                         bool is_write);
>  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
>  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
>                                       hwaddr phys_addr,
>                                       hwaddr size,
> +                                     ram_addr_t ram_addr_offset,
>                                       uint8_t lock,
>                                       bool dma,
>                                       bool is_write)
> diff --git a/system/physmem.c b/system/physmem.c
> index b7847db1a2..33d09f7571 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>           */
>          if (xen_mr_is_memory(block->mr)) {
>              return xen_map_cache(block->mr, block->offset + addr,
> -                                 len, lock, lock,
> -                                 is_write);
> +                                 len, block->offset,
> +                                 lock, lock, is_write);
>          }
>  
>          block->host = xen_map_cache(block->mr, block->offset,
> -                                    block->max_length, 1,
> -                                    lock, is_write);
> +                                    block->max_length,
> +                                    block->offset,
> +                                    1, lock, is_write);
>      }

This is OK but it is really making it clear that there is a mistake in
the code. OK for now but it is something to fix in the future.


>      return ramblock_ptr(block, addr);
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:14:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:14:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722662.1126823 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Npe-0007cv-NJ; Wed, 15 May 2024 23:14:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722662.1126823; Wed, 15 May 2024 23:14:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Npe-0007co-Kg; Wed, 15 May 2024 23:14:46 +0000
Received: by outflank-mailman (input) for mailman id 722662;
 Wed, 15 May 2024 23:14:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Npd-0007cb-Jx
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:14:45 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e8910190-1310-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:14:43 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id A8F22CE17FD;
 Wed, 15 May 2024 23:14:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E27A0C116B1;
 Wed, 15 May 2024 23:14:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8910190-1310-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715814877;
	bh=5nZURLaoNtXce9Y8qA5k4kGoOJSAV9Ep36Do2UBcKbk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=o+P9KyhIWTR7JvTiRtLy6cWFYqZBr6hZq9+C98cXoDmgBnyj3zZiaWjc7cmvPdhcy
	 7qVcwv85o5J63IgCni/bLyevNJtCOC3OFiAkuZLpGVVF8wysShk0NqASA8Gf61bqhi
	 uPxjsHZSZP4FqCxVEQNXWc1U/+Htv0/HEn5lOEtQfVA5ZcHbYw3n1tvb96Hg0Ql3IZ
	 qxR01yxMHEFMXR5QMWUOv7hPbvhmX0MpMddqgC7YN9zrusTkumjlXnVH/Hb8Rk2/d/
	 ZuVni1h8Rn252G8EULwVMP/WNgr6jffDaZ4scYw+OielC3ZKPD/lzYQEvYqmH3hNuz
	 k1fzuEP8wmeTQ==
Date: Wed, 15 May 2024 16:14:35 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v5 7/8] xen: mapcache: Add support for grant mappings
In-Reply-To: <20240503014449.1046238-8-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151614270.2544314@ubuntu-linux-20-04-desktop>
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com> <20240503014449.1046238-8-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add a second mapcache for grant mappings. The mapcache for
> grants needs to work with XC_PAGE_SIZE granularity since
> we can't map larger ranges than what has been granted to us.
> 
> Like with foreign mappings (xen_memory), machines using grants
> are expected to initialize the xen_grants MR and map it
> into their address-map accordingly.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Wed May 15 23:17:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:17:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722665.1126834 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Nsd-0008CV-3c; Wed, 15 May 2024 23:17:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722665.1126834; Wed, 15 May 2024 23:17:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Nsd-0008CO-0z; Wed, 15 May 2024 23:17:51 +0000
Received: by outflank-mailman (input) for mailman id 722665;
 Wed, 15 May 2024 23:17:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Nsc-0008CI-BT
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:17:50 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 56ea886b-1311-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 01:17:49 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 03A87CE17FC;
 Wed, 15 May 2024 23:17:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8E88C116B1;
 Wed, 15 May 2024 23:17:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56ea886b-1311-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815063;
	bh=lk9gQmDGPeoyLQma8nQqXia3ZfDePyKZ/vakkk79Nag=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=coc6ix9srwRSIn78ao3hTZZFlaFZOKbMfDhBVCquBz/g/AThDNYOxx9LXrIyNHExc
	 PDeq9HtX4mMH5MrGg9Kp43brgEWzkoXdWPHIEf7AOaRqap69URBKBTgwe0xzzhxLKj
	 WLARO8TbtcftDoQsp4hwHSliqJxViKFwIdb2e/IVS0ZwxgbIfU5k4zb2gxUUIzy7Za
	 kBOX77mdc7pUikvlSg4fToODyNE4buVQXLfz9N8vwSAuVbddZ/Yt1r71kn8U5ZfkIz
	 RkhpPKfOcJbqnQ8nW/H2/VG5kABtFu9YXdzJUKtcsgE0472pWYmgCzxj3yxG0LsuQO
	 DUy4cAmVlbxqg==
Date: Wed, 15 May 2024 16:17:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [XEN PATCH 1/4] x86/vpmu: address violations of MISRA C Rule
 20.7
In-Reply-To: <ecba64bb8295fa27f0ddbb0905b0983a3572b1ae.1715757982.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405151617240.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com> <ecba64bb8295fa27f0ddbb0905b0983a3572b1ae.1715757982.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  xen/arch/x86/cpu/vpmu_amd.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
> index db2fa420e14a..97e6315bd9f7 100644
> --- a/xen/arch/x86/cpu/vpmu_amd.c
> +++ b/xen/arch/x86/cpu/vpmu_amd.c
> @@ -25,8 +25,8 @@
>  
>  #define is_guest_mode(msr) ((msr) & (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
>  #define is_pmu_enabled(msr) ((msr) & (1ULL << MSR_F10H_EVNTSEL_EN_SHIFT))
> -#define set_guest_mode(msr) (msr |= (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
> -#define is_overflowed(msr) (!((msr) & (1ULL << (MSR_F10H_COUNTER_LENGTH-1))))
> +#define set_guest_mode(msr) ((msr) |= (1ULL << MSR_F10H_EVNTSEL_GO_SHIFT))
> +#define is_overflowed(msr) (!((msr) & (1ULL << (MSR_F10H_COUNTER_LENGTH - 1))))

is_overflowed is just a cosmetic change


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:18:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:18:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722668.1126843 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NtQ-0000HC-BU; Wed, 15 May 2024 23:18:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722668.1126843; Wed, 15 May 2024 23:18:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NtQ-0000H5-8v; Wed, 15 May 2024 23:18:40 +0000
Received: by outflank-mailman (input) for mailman id 722668;
 Wed, 15 May 2024 23:18:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NtP-0008CI-7P
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:18:39 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 75529a13-1311-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 01:18:38 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DB985615E4;
 Wed, 15 May 2024 23:18:36 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CCF1EC116B1;
 Wed, 15 May 2024 23:18:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75529a13-1311-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815116;
	bh=SCXizz/EilxvN1gX2HeXJkgWZPC7AaUPWUwCx3Q36s0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hzjfUaoZiEE+jdsQN82i5QlmhradRn3/W5Bv/xsaYNJPBsEIVGiQ/JzDJvkrdWcz+
	 GLfgfp4AOcxbDU8gbe6Hv8xrTUHBBQw7VhqQNe1u+9YWO2ArKwwR5LmMHMHzBAqDG/
	 Ifw5xkUQKa+sb/mW6SZe7GVik9frix5TQbr0J2GC2T2DsQS/b/+D/7iyVOvSUA46hE
	 /7Fjj8hvK3Nyfwp48l+zh6i7ETuhCjzQZ0Q7TO+q1V58AFlyXtEdu86du6ugsjwheC
	 WarMLghdZ2tCV3GOri7IhxPr6j84Lu2CDzmh6grZ+toBptiOt/4oQr7d2H5F6D8iXl
	 K27uHIE8JQcnw==
Date: Wed, 15 May 2024 16:18:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [XEN PATCH 2/4] x86/hvm: address violations of MISRA C Rule
 20.7
In-Reply-To: <6d14b3283005cf1a30c4fa24f9841586a41e2b1b.1715757982.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405151618170.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com> <6d14b3283005cf1a30c4fa24f9841586a41e2b1b.1715757982.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/x86/hvm/mtrr.c             | 2 +-
>  xen/arch/x86/hvm/rtc.c              | 2 +-
>  xen/arch/x86/include/asm/hvm/save.h | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
> index 32f74c1db03b..1079851f70ed 100644
> --- a/xen/arch/x86/hvm/mtrr.c
> +++ b/xen/arch/x86/hvm/mtrr.c
> @@ -16,7 +16,7 @@
>  #include <public/hvm/e820.h>
>  
>  /* Get page attribute fields (PAn) from PAT MSR. */
> -#define pat_cr_2_paf(pat_cr,n)  ((((uint64_t)pat_cr) >> ((n)<<3)) & 0xff)
> +#define pat_cr_2_paf(pat_cr, n)  ((((uint64_t)(pat_cr)) >> ((n) << 3)) & 0xff)

just a cosmetic change


>  /* Effective mm type lookup table, according to MTRR and PAT. */
>  static const uint8_t mm_type_tbl[MTRR_NUM_TYPES][X86_NUM_MT] = {
> diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
> index 4bb1c7505534..72c7bdbfcd02 100644
> --- a/xen/arch/x86/hvm/rtc.c
> +++ b/xen/arch/x86/hvm/rtc.c
> @@ -45,7 +45,7 @@
>  #define vrtc_domain(x) (container_of(x, struct pl_time, vrtc)->domain)
>  #define vrtc_vcpu(x)   (pt_global_vcpu_target(vrtc_domain(x)))
>  #define epoch_year     1900
> -#define get_year(x)    (x + epoch_year)
> +#define get_year(x)    ((x) + epoch_year)
>  
>  enum rtc_mode {
>     rtc_mode_no_ack,
> diff --git a/xen/arch/x86/include/asm/hvm/save.h b/xen/arch/x86/include/asm/hvm/save.h
> index 8149aa113cb4..ec8de029319d 100644
> --- a/xen/arch/x86/include/asm/hvm/save.h
> +++ b/xen/arch/x86/include/asm/hvm/save.h
> @@ -50,7 +50,7 @@ int _hvm_check_entry(struct hvm_domain_context *h,
>                            HVM_SAVE_LENGTH(x), true) == 0 )      \
>      {                                                           \
>          ptr = &(h)->data[(h)->cur];                             \
> -        h->cur += HVM_SAVE_LENGTH(x);                           \
> +        (h)->cur += HVM_SAVE_LENGTH(x);                         \
>      }                                                           \
>      ptr; })
>  
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:19:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:19:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722671.1126855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NuK-0000rr-LR; Wed, 15 May 2024 23:19:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722671.1126855; Wed, 15 May 2024 23:19:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NuK-0000rk-HF; Wed, 15 May 2024 23:19:36 +0000
Received: by outflank-mailman (input) for mailman id 722671;
 Wed, 15 May 2024 23:19:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NuI-0000m2-Re
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:19:34 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94dd56c1-1311-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:19:32 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id A9BA6CE17E5;
 Wed, 15 May 2024 23:19:29 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A20D2C116B1;
 Wed, 15 May 2024 23:19:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94dd56c1-1311-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815168;
	bh=XlwaEmW5wYY+zIrqokwynsWAIbhKBf192GhnqdDk0ec=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ao0qrwnO3RraVDr3G3HzKiK0+gX5yJnMKRJbTEMgOypYZ64SN4TJ9jch7s+xJ8coZ
	 wzsjSGYzcRBvWM3J5uE9vy5zvjb5u7k3+UEQa+I0V+zUxWEz7kcn9skZpghRPpZE4Q
	 l3zAaIomeeXAc6cok8bU7EsOp8vSGWTiXiIR/KD6+iIBSXOZvc34rlzBq5XP0VZt6n
	 JkPEgmcdx9FsmPRASEH9vXOKxji5ngfUs43SHWOifHTnhx/qscg1aR27nsOSS6WGpm
	 doBtt2YoJJmpGphVWuObGTiEKNgDz3A2u1AzKNZw+ZeRJh5x/3qc40VTpQA44uWSAU
	 T003I1jQ0AmVQ==
Date: Wed, 15 May 2024 16:19:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [XEN PATCH 3/4] x86_64/uaccess: address violations of MISRA C
 Rule 20.7
In-Reply-To: <b131427dc03c4b89141bab7648523b81bf5186a6.1715757982.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405151619200.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com> <b131427dc03c4b89141bab7648523b81bf5186a6.1715757982.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> xlat_malloc_init is touched for consistency, despite the construct
> being already deviated.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:20:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:20:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722673.1126865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Nud-0001R2-Tm; Wed, 15 May 2024 23:19:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722673.1126865; Wed, 15 May 2024 23:19:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Nud-0001QP-OK; Wed, 15 May 2024 23:19:55 +0000
Received: by outflank-mailman (input) for mailman id 722673;
 Wed, 15 May 2024 23:19:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7Nuc-0000m2-Ns
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:19:54 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a1a2efae-1311-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:19:53 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 99FCD61616;
 Wed, 15 May 2024 23:19:51 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02FC9C116B1;
 Wed, 15 May 2024 23:19:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1a2efae-1311-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815191;
	bh=P2TqCEm07IO71XzjbE99tAAsu3UFAKe/yxzprP+w9EM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=PDVJTVGYdzhywT2OU93OBnMdiCf7NIp2Sxc/cwaM13/SuAbz+iwi20u5b4DhLUQUQ
	 cT4cAUzKi/Pzp00a+DixUD82sp3qacUVWVEx3kvnjzZ8Kug1nC6dx54eu7S5sPfMiI
	 Ve/9XRuGvwl+WxWzDkzwAPKgLMejKKb0GFFqsEKK8MzB4dW78CurJ7iR93fAGHtH6A
	 PGjQ3VORgfSMErIyeGGaoPOdp22XOsVlds3RiteITGYCCmoMWr51naLqkhrC8DwznD
	 ZX+oZfISkuVUX0jxqkPWtKbm68LU3A/1g4SRnrbQTSuEtL0ZdpqnEj1oelkE6A5i0+
	 tVhyAf/Av4buw==
Date: Wed, 15 May 2024 16:19:48 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [XEN PATCH 4/4] x86_64/cpu_idle: address violations of MISRA C
 Rule 20.7
In-Reply-To: <6c84f6f0bf93a00ab1acda2d39d31b1f3d5bf8aa.1715757982.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405151619410.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com> <6c84f6f0bf93a00ab1acda2d39d31b1f3d5bf8aa.1715757982.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Wed May 15 23:25:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:25:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722676.1126874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NzX-0003nF-Hu; Wed, 15 May 2024 23:24:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722676.1126874; Wed, 15 May 2024 23:24:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7NzX-0003n8-EO; Wed, 15 May 2024 23:24:59 +0000
Received: by outflank-mailman (input) for mailman id 722676;
 Wed, 15 May 2024 23:24:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7NzW-0003n2-Ri
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:24:58 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 566791af-1312-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 01:24:57 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D648BCE1371;
 Wed, 15 May 2024 23:24:53 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C044C116B1;
 Wed, 15 May 2024 23:24:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 566791af-1312-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815493;
	bh=cOy32dOQ9cyam8WQ2lP2EYDTH/jxSOlJ+JgnjXKKEWY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=W+CJdMRzJ6GT76WYxSUbl+zeiWK5yN2pBAB/ouYmlZ8EQZ7hexYAt6kyqJ71uE8Nn
	 toopXoF0yeJBPoLQmTCALfzA7S7BXFl070TXQkUIW6O+OoSlSSczIGCtXVrEi7p9bM
	 sf30ifJe6fGDAZqTp2yY7UyEGIM2Wqk6vvVekmmkkHaFEqdQEkQQS2kP/OeIye61x5
	 64ZlYMnHIWKQuQVxQp1nO6L5fK+WX7kAG989UGbdhgolp6s/zUQDuKVyiDzHxEzQMv
	 uR/HDkXDSc6QDcWpJQtfUfX7LIB/weO2JAUjAIPMf12jOZLS9SfAhLxLkUxCaSSx9o
	 zhiM7LcX+kWCg==
Date: Wed, 15 May 2024 16:24:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v4 4/9] xen/arm64: head: Add missing code symbol
 annotations
In-Reply-To: <20240504115514.1063509-5-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151624120.2544314@ubuntu-linux-20-04-desktop>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com> <20240504115514.1063509-5-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 4 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/head.S | 51 +++++++++++++++++++++------------------
>  1 file changed, 27 insertions(+), 24 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
> index fb297e9eb5..2fa07dc3a0 100644
> --- a/xen/arch/arm/arm64/head.S
> +++ b/xen/arch/arm/arm64/head.S
> @@ -90,7 +90,7 @@
>   * 4K-aligned address.
>   */
>  
> -GLOBAL(start)
> +FUNC(start)
>          /*
>           * DO NOT MODIFY. Image header expected by Linux boot-loaders.
>           */
> @@ -102,6 +102,7 @@ efi_head:
>           */
>          add     x13, x18, #0x16
>          b       real_start           /* branch to kernel start */
> +END(start)
>          .quad   0                    /* Image load offset from start of RAM */
>          .quad   _end - start         /* Effective size of kernel image, little-endian */
>          .quad   __HEAD_FLAGS         /* Informative flags, little-endian */
> @@ -223,7 +224,7 @@ section_table:
>          .align  5
>  #endif /* CONFIG_ARM_EFI */
>  
> -real_start:
> +FUNC_LOCAL(real_start)
>          /* BSS should be zeroed when booting without EFI */
>          mov   x26, #0                /* x26 := skip_zero_bss */
>  
> @@ -263,9 +264,9 @@ primary_switched:
>          mov   x1, x21                /* x1 := paddr(FDT) */
>          ldr   x2, =start_xen
>          b     launch
> -ENDPROC(real_start)
> +END(real_start)
>  
> -GLOBAL(init_secondary)
> +FUNC(init_secondary)
>          msr   DAIFSet, 0xf           /* Disable all interrupts */
>  
>          /* Find out where we are */
> @@ -304,7 +305,7 @@ secondary_switched:
>          /* Jump to C world */
>          ldr   x2, =start_secondary
>          b     launch
> -ENDPROC(init_secondary)
> +END(init_secondary)
>  
>  /*
>   * Check if the CPU has been booted in Hypervisor mode.
> @@ -313,7 +314,7 @@ ENDPROC(init_secondary)
>   *
>   * Clobbers x0 - x5
>   */
> -check_cpu_mode:
> +FUNC_LOCAL(check_cpu_mode)
>          PRINT_ID("- Current EL ")
>          mrs   x5, CurrentEL
>          print_reg x5
> @@ -329,7 +330,7 @@ check_cpu_mode:
>          PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n")
>          PRINT_ID("- Please update the bootloader -\r\n")
>          b fail
> -ENDPROC(check_cpu_mode)
> +END(check_cpu_mode)
>  
>  /*
>   * Zero BSS
> @@ -339,7 +340,7 @@ ENDPROC(check_cpu_mode)
>   *
>   * Clobbers x0 - x3
>   */
> -zero_bss:
> +FUNC_LOCAL(zero_bss)
>          /* Zero BSS only when requested */
>          cbnz  x26, skip_bss
>  
> @@ -353,14 +354,14 @@ zero_bss:
>  
>  skip_bss:
>          ret
> -ENDPROC(zero_bss)
> +END(zero_bss)
>  
>  /*
>   * Initialize the processor for turning the MMU on.
>   *
>   * Clobbers x0 - x3
>   */
> -cpu_init:
> +FUNC_LOCAL(cpu_init)
>          PRINT_ID("- Initialize CPU -\r\n")
>  
>          /* Set up memory attribute type tables */
> @@ -399,7 +400,7 @@ cpu_init:
>           */
>          msr spsel, #1
>          ret
> -ENDPROC(cpu_init)
> +END(cpu_init)
>  
>  /*
>   * Setup the initial stack and jump to the C world
> @@ -411,7 +412,7 @@ ENDPROC(cpu_init)
>   *
>   * Clobbers x3
>   */
> -launch:
> +FUNC_LOCAL(launch)
>          ldr   x3, =init_data
>          add   x3, x3, #INITINFO_stack /* Find the boot-time stack */
>          ldr   x3, [x3]
> @@ -421,13 +422,14 @@ launch:
>  
>          /* Jump to C world */
>          br    x2
> -ENDPROC(launch)
> +END(launch)
>  
>  /* Fail-stop */
> -fail:   PRINT_ID("- Boot failed -\r\n")
> +FUNC_LOCAL(fail)
> +        PRINT_ID("- Boot failed -\r\n")
>  1:      wfe
>          b     1b
> -ENDPROC(fail)
> +END(fail)
>  
>  #ifdef CONFIG_EARLY_PRINTK
>  /*
> @@ -438,14 +440,14 @@ ENDPROC(fail)
>   *
>   * Clobbers x0 - x1
>   */
> -init_uart:
> +FUNC_LOCAL(init_uart)
>          ldr   x23, =CONFIG_EARLY_UART_BASE_ADDRESS
>  #ifdef CONFIG_EARLY_UART_INIT
>          early_uart_init x23, 0
>  #endif
>          PRINT("- UART enabled -\r\n")
>          ret
> -ENDPROC(init_uart)
> +END(init_uart)
>  
>  /*
>   * Print early debug messages.
> @@ -454,7 +456,7 @@ ENDPROC(init_uart)
>   * x23: Early UART base address
>   * Clobbers x0-x1
>   */
> -ENTRY(asm_puts)
> +FUNC(asm_puts)
>          early_uart_ready x23, 1
>          ldrb  w1, [x0], #1           /* Load next char */
>          cbz   w1, 1f                 /* Exit on nul */
> @@ -462,7 +464,7 @@ ENTRY(asm_puts)
>          b     asm_puts
>  1:
>          ret
> -ENDPROC(asm_puts)
> +END(asm_puts)
>  
>  /*
>   * Print a 64-bit number in hex.
> @@ -471,7 +473,7 @@ ENDPROC(asm_puts)
>   * x23: Early UART base address
>   * Clobbers x0-x3
>   */
> -ENTRY(asm_putn)
> +FUNC(asm_putn)
>          adr_l x1, hex
>          mov   x3, #16
>  1:
> @@ -484,7 +486,7 @@ ENTRY(asm_putn)
>          subs  x3, x3, #1
>          b.ne  1b
>          ret
> -ENDPROC(asm_putn)
> +END(asm_putn)
>  
>  RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
>  
> @@ -493,16 +495,17 @@ RODATA_SECT(.rodata.idmap, hex, "0123456789abcdef")
>  /* This provides a C-API version of __lookup_processor_type
>   * TODO: For now, the implementation return NULL every time
>   */
> -ENTRY(lookup_processor_type)
> +FUNC(lookup_processor_type)
>          mov  x0, #0
>          ret
> +END(lookup_processor_type)
>  
>  #ifdef CONFIG_ARM_EFI
>  /*
>   *  Function to transition from EFI loader in C, to Xen entry point.
>   *  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>   */
> -ENTRY(efi_xen_start)
> +FUNC(efi_xen_start)
>          /*
>           * Preserve x0 (fdt pointer) across call to __flush_dcache_area,
>           * restore for entry into Xen.
> @@ -554,7 +557,7 @@ ENTRY(efi_xen_start)
>          mov   x26, #1               /* x26 := skip_zero_bss */
>  
>          b     real_start_efi
> -ENDPROC(efi_xen_start)
> +END(efi_xen_start)
>  
>  #endif /* CONFIG_ARM_EFI */
>  
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:26:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:26:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722681.1126884 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7O1J-0004LQ-Rl; Wed, 15 May 2024 23:26:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722681.1126884; Wed, 15 May 2024 23:26:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7O1J-0004LJ-P2; Wed, 15 May 2024 23:26:49 +0000
Received: by outflank-mailman (input) for mailman id 722681;
 Wed, 15 May 2024 23:26:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7O1I-0004L9-PV
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:26:48 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 984a72b2-1312-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:26:46 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2C0FF615F9;
 Wed, 15 May 2024 23:26:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97CC3C116B1;
 Wed, 15 May 2024 23:26:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 984a72b2-1312-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715815604;
	bh=LzNTcPEq45Y5y113TkBRtB7OYJJfblhfpm5W8bOZ0Gg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ftKBI6wmFh7MyRmXyCDk/Y3FORTCFVwJIFvp5c/wbDFh+VJnh/a499XbvbHkjiZUm
	 B7wfwmpQQlkbH0614Cpjv+X7BwVieNIq53Lq/4NBcVLKFwVAw3lPnEZuNeQp8oollQ
	 AiyZmwjOgRm72oGx+zBBZiZHIWM0hLOO86yDdPoLIrkyjSFLtrdEwrnsZmsRlYCYPO
	 evUj5gnhgppL9ie4nZvpD3MUIJokq44Cy++M+5o3sso9jduh7FHSraUtpCGZ/ISOW0
	 s0QYkkKdAbFKksa9Hsg3BKwRX1cDHzz2SeIAIbr2BVtwyrVXsRUfHcHyzyteg/B3jh
	 wLIArug6bpB+g==
Date: Wed, 15 May 2024 16:26:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v4 6/9] xen/arm64: bpi: Add missing code symbol
 annotations
In-Reply-To: <20240504115514.1063509-7-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151625140.2544314@ubuntu-linux-20-04-desktop>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com> <20240504115514.1063509-7-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 4 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/bpi.S | 20 ++++++++++++--------
>  1 file changed, 12 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
> index 4e63825220..d6b8defb80 100644
> --- a/xen/arch/arm/arm64/bpi.S
> +++ b/xen/arch/arm/arm64/bpi.S
> @@ -52,14 +52,15 @@
>   * micro-architectures in a system.
>   */
>      .align	11
> -ENTRY(__bp_harden_hyp_vecs_start)
> +FUNC(__bp_harden_hyp_vecs_start)
>      .rept 4
>      vectors hyp_traps_vector
>      .endr
> -ENTRY(__bp_harden_hyp_vecs_end)
> +LABEL(__bp_harden_hyp_vecs_end)
> +END(__bp_harden_hyp_vecs_start)
>  
>  .macro mitigate_spectre_bhb_loop count
> -ENTRY(__mitigate_spectre_bhb_loop_start_\count)
> +FUNC(__mitigate_spectre_bhb_loop_start_\count)
>      stp     x0, x1, [sp, #-16]!
>      mov     x0, \count
>  .Lspectre_bhb_loop\@:
> @@ -68,11 +69,12 @@ ENTRY(__mitigate_spectre_bhb_loop_start_\count)
>      b.ne    .Lspectre_bhb_loop\@
>      sb
>      ldp     x0, x1, [sp], #16
> -ENTRY(__mitigate_spectre_bhb_loop_end_\count)
> +LABEL(__mitigate_spectre_bhb_loop_end_\count)
> +END(__mitigate_spectre_bhb_loop_start_\count)
>  .endm
>  
>  .macro smccc_workaround num smcc_id
> -ENTRY(__smccc_workaround_smc_start_\num)
> +FUNC(__smccc_workaround_smc_start_\num)
>      sub     sp, sp, #(8 * 4)
>      stp     x0, x1, [sp, #(8 * 2)]
>      stp     x2, x3, [sp, #(8 * 0)]
> @@ -81,13 +83,15 @@ ENTRY(__smccc_workaround_smc_start_\num)
>      ldp     x2, x3, [sp, #(8 * 0)]
>      ldp     x0, x1, [sp, #(8 * 2)]
>      add     sp, sp, #(8 * 4)
> -ENTRY(__smccc_workaround_smc_end_\num)
> +LABEL(__smccc_workaround_smc_end_\num)
> +END(__smccc_workaround_smc_start_\num)
>  .endm
>  
> -ENTRY(__mitigate_spectre_bhb_clear_insn_start)
> +FUNC(__mitigate_spectre_bhb_clear_insn_start)
>      clearbhb
>      isb
> -ENTRY(__mitigate_spectre_bhb_clear_insn_end)
> +LABEL(__mitigate_spectre_bhb_clear_insn_end)
> +END(__mitigate_spectre_bhb_clear_insn_start)
>  
>  mitigate_spectre_bhb_loop 8
>  mitigate_spectre_bhb_loop 24
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:49:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:49:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722692.1126893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ON4-0000Yc-K3; Wed, 15 May 2024 23:49:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722692.1126893; Wed, 15 May 2024 23:49:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ON4-0000YV-HC; Wed, 15 May 2024 23:49:18 +0000
Received: by outflank-mailman (input) for mailman id 722692;
 Wed, 15 May 2024 23:49:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7ON3-0000YP-0g
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:49:17 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bbdbd44d-1315-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:49:14 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2E4F261620;
 Wed, 15 May 2024 23:49:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50169C32789;
 Wed, 15 May 2024 23:49:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbdbd44d-1315-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715816952;
	bh=6smpIIM4q21kF+COMUHH7hPoxtPPscHSzvhQtiS2sc4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kYZKbpGqfslKILkTgtizsVnZ2BwKRxrtVkEO/taMEJ3tcYNE2AlsLIgg1Mo6F+l0U
	 y4eNfCxjVcZYOZXZ9+1hZn2JCzP+7mC23B10Kr8zj33N7MKBAaIB5v5p3E40/gWcGs
	 zjBA0PpyDislSCRuIsUEG4RmxlGJV32OuCPnEWwi4lkm7Q8xqmaouyIDU8h0DGK1pq
	 ruDnhZLUrXV/JyskPC+fh6EwLt6EHrRXkO5F9xs9nAPcgXVaSjUXE+13JVzxOHXxNz
	 bT/Rud9tBwKyh5vkIVfefMyak80mPDHqVk9T5/cxs7d57yS2I28er3KcOanpOaTbBj
	 IhlznuXYmdpmQ==
Date: Wed, 15 May 2024 16:49:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v4 1/9] xen/arm64: entry: Add missing code symbol
 annotations
In-Reply-To: <20240504115514.1063509-2-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151647590.2544314@ubuntu-linux-20-04-desktop>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com> <20240504115514.1063509-2-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 4 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/entry.S | 78 ++++++++++++++++++++++++--------------
>  1 file changed, 50 insertions(+), 28 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
> index f963c923bb..6251135ebd 100644
> --- a/xen/arch/arm/arm64/entry.S
> +++ b/xen/arch/arm/arm64/entry.S
> @@ -289,21 +289,25 @@
>          b       do_bad_mode
>          .endm
>  
> -hyp_sync_invalid:
> +FUNC_LOCAL(hyp_sync_invalid)
>          entry   hyp=1
>          invalid BAD_SYNC
> +END(hyp_sync_invalid)
>  
> -hyp_irq_invalid:
> +FUNC_LOCAL(hyp_irq_invalid)
>          entry   hyp=1
>          invalid BAD_IRQ
> +END(hyp_irq_invalid)
>  
> -hyp_fiq_invalid:
> +FUNC_LOCAL(hyp_fiq_invalid)
>          entry   hyp=1
>          invalid BAD_FIQ
> +END(hyp_fiq_invalid)
>  
> -hyp_error_invalid:
> +FUNC_LOCAL(hyp_error_invalid)
>          entry   hyp=1
>          invalid BAD_ERROR
> +END(hyp_error_invalid)
>  
>  /*
>   * SError received while running in the hypervisor mode.
> @@ -313,11 +317,12 @@ hyp_error_invalid:
>   * simplicity, as SError should be rare and potentially fatal,
>   * all interrupts are kept masked.
>   */
> -hyp_error:
> +FUNC_LOCAL(hyp_error)
>          entry   hyp=1
>          mov     x0, sp
>          bl      do_trap_hyp_serror
>          exit    hyp=1
> +END(hyp_error)
>  
>  /*
>   * Synchronous exception received while running in the hypervisor mode.
> @@ -327,7 +332,7 @@ hyp_error:
>   * some of them. So we want to inherit the state from the interrupted
>   * context.
>   */
> -hyp_sync:
> +FUNC_LOCAL(hyp_sync)
>          entry   hyp=1
>  
>          /* Inherit interrupts */
> @@ -338,6 +343,7 @@ hyp_sync:
>          mov     x0, sp
>          bl      do_trap_hyp_sync
>          exit    hyp=1
> +END(hyp_sync)
>  
>  /*
>   * IRQ received while running in the hypervisor mode.
> @@ -352,7 +358,7 @@ hyp_sync:
>   * would require some rework in some paths (e.g. panic, livepatch) to
>   * ensure the ordering is enforced everywhere.
>   */
> -hyp_irq:
> +FUNC_LOCAL(hyp_irq)
>          entry   hyp=1
>  
>          /* Inherit D, A, F interrupts and keep I masked */
> @@ -365,8 +371,9 @@ hyp_irq:
>          mov     x0, sp
>          bl      do_trap_irq
>          exit    hyp=1
> +END(hyp_irq)
>  
> -guest_sync:
> +FUNC_LOCAL(guest_sync)
>          /*
>           * Save x0, x1 in advance
>           */
> @@ -413,8 +420,9 @@ fastpath_out_workaround:
>          mov     x1, xzr
>          eret
>          sb
> +END(guest_sync)
>  
> -wa2_ssbd:
> +FUNC_LOCAL(wa2_ssbd)
>  #ifdef CONFIG_ARM_SSBD
>  alternative_cb arm_enable_wa2_handling
>          b       wa2_end
> @@ -450,42 +458,55 @@ wa2_end:
>          mov     x0, xzr
>          eret
>          sb
> -guest_sync_slowpath:
> +END(wa2_ssbd)
> +
> +FUNC_LOCAL(guest_sync_slowpath)
>          /*
>           * x0/x1 may have been scratch by the fast path above, so avoid
>           * to save them.
>           */
>          guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_sync, save_x0_x1=0
> +END(guest_sync_slowpath)
>  
> -guest_irq:
> +FUNC_LOCAL(guest_irq)
>          guest_vector compat=0, iflags=IFLAGS__A__, trap=irq
> +END(guest_irq)
>  
> -guest_fiq_invalid:
> +FUNC_LOCAL(guest_fiq_invalid)
>          entry   hyp=0, compat=0
>          invalid BAD_FIQ
> +END(guest_fiq_invalid)
>  
> -guest_error:
> +FUNC_LOCAL(guest_error)
>          guest_vector compat=0, iflags=IFLAGS__AI_, trap=guest_serror
> +END(guest_error)
>  
> -guest_sync_compat:
> +FUNC_LOCAL(guest_sync_compat)
>          guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_sync
> +END(guest_sync_compat)
>  
> -guest_irq_compat:
> +FUNC_LOCAL(guest_irq_compat)
>          guest_vector compat=1, iflags=IFLAGS__A__, trap=irq
> +END(guest_irq_compat)
>  
> -guest_fiq_invalid_compat:
> +FUNC_LOCAL(guest_fiq_invalid_compat)
>          entry   hyp=0, compat=1
>          invalid BAD_FIQ
> +END(guest_fiq_invalid_compat)
>  
> -guest_error_compat:
> +FUNC_LOCAL(guest_error_compat)
>          guest_vector compat=1, iflags=IFLAGS__AI_, trap=guest_serror
> +END(guest_error_compat)
>  
> -ENTRY(return_to_new_vcpu32)
> +FUNC(return_to_new_vcpu32)
>          exit    hyp=0, compat=1
> -ENTRY(return_to_new_vcpu64)
> +END(return_to_new_vcpu32)
> +
> +FUNC(return_to_new_vcpu64)
>          exit    hyp=0, compat=0
> +END(return_to_new_vcpu64)
>  
> -return_from_trap:
> +FUNC_LOCAL(return_from_trap)
>          msr     daifset, #IFLAGS___I_ /* Mask interrupts */
>  
>          ldr     x21, [sp, #UREGS_PC]            /* load ELR */
> @@ -524,6 +545,7 @@ return_from_trap:
>  
>          eret
>          sb
> +END(return_from_trap)
>  
>  /*
>   * Consume pending SError generated by the guest if any.
> @@ -536,7 +558,7 @@ return_from_trap:
>   * it. So the function will unmask SError exception for a small window and
>   * then mask it again.
>   */
> -check_pending_guest_serror:
> +FUNC_LOCAL(check_pending_guest_serror)
>          /*
>           * Save elr_el2 to check whether the pending SError exception takes
>           * place while we are doing this sync exception.
> @@ -562,13 +584,11 @@ check_pending_guest_serror:
>           * exception handler, and the elr_el2 will be set to
>           * abort_guest_exit_start or abort_guest_exit_end.
>           */
> -        .global abort_guest_exit_start
> -abort_guest_exit_start:
> +LABEL(abort_guest_exit_start)
>  
>          isb
>  
> -        .global abort_guest_exit_end
> -abort_guest_exit_end:
> +LABEL(abort_guest_exit_end)
>          /* Mask PSTATE asynchronous abort bit, close the checking window. */
>          msr     daifset, #IFLAGS__A__
>  
> @@ -586,7 +606,7 @@ abort_guest_exit_end:
>          cset    x19, ne
>  
>          ret
> -ENDPROC(check_pending_guest_serror)
> +END(check_pending_guest_serror)
>  
>  /*
>   * Exception vectors.
> @@ -597,7 +617,7 @@ ENDPROC(check_pending_guest_serror)
>          .endm
>  
>          .align  11
> -ENTRY(hyp_traps_vector)
> +FUNC(hyp_traps_vector)
>          ventry  hyp_sync_invalid            /* Synchronous EL2t */
>          ventry  hyp_irq_invalid             /* IRQ EL2t */
>          ventry  hyp_fiq_invalid             /* FIQ EL2t */
> @@ -617,6 +637,7 @@ ENTRY(hyp_traps_vector)
>          ventry  guest_irq_compat            /* IRQ 32-bit EL0/EL1 */
>          ventry  guest_fiq_invalid_compat    /* FIQ 32-bit EL0/EL1 */
>          ventry  guest_error_compat          /* Error 32-bit EL0/EL1 */
> +END(hyp_traps_vector)
>  
>  /*
>   * struct vcpu *__context_switch(struct vcpu *prev, struct vcpu *next)
> @@ -626,7 +647,7 @@ ENTRY(hyp_traps_vector)
>   *
>   * Returns prev in x0
>   */
> -ENTRY(__context_switch)
> +FUNC(__context_switch)
>          add     x8, x0, #VCPU_arch_saved_context
>          mov     x9, sp
>          stp     x19, x20, [x8], #16         /* store callee-saved registers */
> @@ -647,6 +668,7 @@ ENTRY(__context_switch)
>          ldr     lr, [x8]
>          mov     sp, x9
>          ret
> +END(__context_switch)
>  
>  /*
>   * Local variables:
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 15 23:49:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 May 2024 23:49:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722694.1126903 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ONe-0001EI-RM; Wed, 15 May 2024 23:49:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722694.1126903; Wed, 15 May 2024 23:49:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ONe-0001EB-Op; Wed, 15 May 2024 23:49:54 +0000
Received: by outflank-mailman (input) for mailman id 722694;
 Wed, 15 May 2024 23:49:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kF2M=MS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7ONd-0001Cu-4t
 for xen-devel@lists.xenproject.org; Wed, 15 May 2024 23:49:53 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d11c8945-1315-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 01:49:50 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 4680761629;
 Wed, 15 May 2024 23:49:49 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9FDB0C32782;
 Wed, 15 May 2024 23:49:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d11c8945-1315-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715816989;
	bh=pgqYyDEf+Hogp85BIs4KiLUDR2b7ViWBJweZVMzx4KQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=s0+aqCk544LbrAxr/jeV+Svrr383ZD0bpQ4KchGUGnfjpw2xsiFz4PQMtvS4rYmuS
	 Nu4rFsSTG1hu7Cx4zpdjPWfMHUoSss8AAuw/yXTFR5BqJp2gghq70k3ASjbp/dlYnX
	 WgVcsZBD43Jfe5IfqRFRLhoC+VHV4pm1n9V70v77YSZ3Kw6ig3Mgi3g00/uQ+iZEjk
	 q8M/u4sR8cRZvIpLPX43UUWzy7v+qSsLcbp8k4o17RHQKJeQz4UbUZv235EeOxm8RK
	 NkcuEi9R4uu3D3bC/2HpnvonCrhLcym0ToqXECb+Rqgt9eNgeA5epNwHKZb54CzjU+
	 62z8RlAbhdlFA==
Date: Wed, 15 May 2024 16:49:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    bertrand.marquis@arm.com, michal.orzel@amd.com, Volodymyr_Babchuk@epam.com, 
    jbeulich@suse.com, andrew.cooper3@citrix.com, edgar.iglesias@amd.com
Subject: Re: [PATCH v4 7/9] xen/arm64: mmu/head: Add missing code symbol
 annotations
In-Reply-To: <20240504115514.1063509-8-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405151649210.2544314@ubuntu-linux-20-04-desktop>
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com> <20240504115514.1063509-8-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 4 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Use the generic xen/linkage.h macros to annotate code symbols
> and add missing annotations.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/arm64/mmu/head.S | 29 +++++++++++++++--------------
>  1 file changed, 15 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S
> index fa40b696dd..665a51a337 100644
> --- a/xen/arch/arm/arm64/mmu/head.S
> +++ b/xen/arch/arm/arm64/mmu/head.S
> @@ -138,7 +138,7 @@
>   *
>   * Clobbers x0 - x4
>   */
> -create_page_tables:
> +FUNC_LOCAL(create_page_tables)
>          /* Prepare the page-tables for mapping Xen */
>          ldr   x0, =XEN_VIRT_START
>          create_table_entry boot_pgtable, boot_first, x0, 0, x1, x2, x3
> @@ -260,7 +260,7 @@ virtphys_clash:
>          /* Identity map clashes with boot_third, which we cannot handle yet */
>          PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
>          b     fail
> -ENDPROC(create_page_tables)
> +END(create_page_tables)
>  
>  /*
>   * Turn on the Data Cache and the MMU. The function will return on the 1:1
> @@ -273,7 +273,7 @@ ENDPROC(create_page_tables)
>   *
>   * Clobbers x0 - x5
>   */
> -enable_mmu:
> +FUNC_LOCAL(enable_mmu)
>          mov   x4, x0
>          mov   x5, x1
>          PRINT_ID("- Turning on paging -\r\n")
> @@ -304,7 +304,7 @@ enable_mmu:
>          PRINT_ID("- Paging turned on -\r\n")
>  
>          ret
> -ENDPROC(enable_mmu)
> +END(enable_mmu)
>  
>  /*
>   * Enable mm (turn on the data cache and the MMU) for secondary CPUs.
> @@ -316,7 +316,7 @@ ENDPROC(enable_mmu)
>   *
>   * Clobbers x0 - x6
>   */
> -ENTRY(enable_secondary_cpu_mm)
> +FUNC(enable_secondary_cpu_mm)
>          mov   x6, lr
>  
>          load_paddr x0, init_ttbr
> @@ -328,7 +328,7 @@ ENTRY(enable_secondary_cpu_mm)
>  
>          /* Return to the virtual address requested by the caller. */
>          ret
> -ENDPROC(enable_secondary_cpu_mm)
> +END(enable_secondary_cpu_mm)
>  
>  /*
>   * Enable mm (turn on the data cache and the MMU) for the boot CPU.
> @@ -340,7 +340,7 @@ ENDPROC(enable_secondary_cpu_mm)
>   *
>   * Clobbers x0 - x6
>   */
> -ENTRY(enable_boot_cpu_mm)
> +FUNC(enable_boot_cpu_mm)
>          mov   x6, lr
>  
>          bl    create_page_tables
> @@ -365,7 +365,7 @@ ENTRY(enable_boot_cpu_mm)
>           * by the caller.
>           */
>          b     remove_identity_mapping
> -ENDPROC(enable_boot_cpu_mm)
> +END(enable_boot_cpu_mm)
>  
>  /*
>   * Remove the 1:1 map from the page-tables. It is not easy to keep track
> @@ -377,7 +377,7 @@ ENDPROC(enable_boot_cpu_mm)
>   *
>   * Clobbers x0 - x1
>   */
> -remove_identity_mapping:
> +FUNC_LOCAL(remove_identity_mapping)
>          /*
>           * Find the zeroeth slot used. Remove the entry from zeroeth
>           * table if the slot is not XEN_ZEROETH_SLOT.
> @@ -419,20 +419,21 @@ identity_mapping_removed:
>          flush_xen_tlb_local
>  
>          ret
> -ENDPROC(remove_identity_mapping)
> +END(remove_identity_mapping)
>  
>  /* Fail-stop */
> -fail:   PRINT("- Boot failed -\r\n")
> +FUNC_LOCAL(fail)
> +        PRINT("- Boot failed -\r\n")
>  1:      wfe
>          b     1b
> -ENDPROC(fail)
> +END(fail)
>  
>  /*
>   * Switch TTBR
>   *
>   * x0    ttbr
>   */
> -ENTRY(switch_ttbr_id)
> +FUNC(switch_ttbr_id)
>          /* 1) Ensure any previous read/write have completed */
>          dsb    ish
>          isb
> @@ -464,7 +465,7 @@ ENTRY(switch_ttbr_id)
>          isb
>  
>          ret
> -ENDPROC(switch_ttbr_id)
> +END(switch_ttbr_id)
>  
>  /*
>   * Local variables:
> -- 
> 2.40.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:30:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722704.1126922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P0d-0002zq-Us; Thu, 16 May 2024 00:30:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722704.1126922; Thu, 16 May 2024 00:30:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P0d-0002zj-Rn; Thu, 16 May 2024 00:30:11 +0000
Received: by outflank-mailman (input) for mailman id 722704;
 Thu, 16 May 2024 00:30:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P0c-0002zb-NL
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:30:10 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 707c00c7-131b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:30:07 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 958DCCE1808;
 Thu, 16 May 2024 00:29:59 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4667FC116B1;
 Thu, 16 May 2024 00:29:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 707c00c7-131b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819398;
	bh=q95dNPvd+2hCI4U/34Jz0uKLv0pHekZk6C/qrWY/vEg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rNyn0vu3alstK8Hqtl9AvfDkjHp41m/cGOnSZAcdPlmdEt9kk6KMnV8zhkPkw7UNa
	 /IGC+I8+jPyjX7dkzNRwh65gNZnUBue6JyRiWiPJ8CV6Rxtuzv6GHLBhUzrM9PZAUT
	 HqUIQsT5xJtvKm6IS916XX56fPcUm1AZG1g5/GdZZWXr4P8UKFuZKAKJibfZW9arhm
	 2UA96VyCm8pMz3vd1/TdEY/V1g/GjX3ZVvplKq8thyAcGKK9LsmPQoUGvaglRcaD+v
	 eCMkSLDWCxtWRfX1sx7jrahO8OhelhYxpWfDEd2E1armDWuerJUQ25VB+ypH6vNU5U
	 iXTEtXOoSXobA==
Date: Wed, 15 May 2024 17:29:55 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>, 
    Petre Pircalabu <ppircalabu@bitdefender.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Tamas K Lengyel <tamas@tklengyel.com>
Subject: Re: [XEN PATCH v2 02/15] x86/monitor: guard altp2m usage
In-Reply-To: <01767c3f98a88999d4b8ed3ae742ad66a0921ba3.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151729470.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <01767c3f98a88999d4b8ed3ae742ad66a0921ba3.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Explicitly check whether altp2m is on for domain when getting altp2m index.
> If explicit call to altp2m_active() always returns false, DCE will remove
> call to altp2m_vcpu_idx().
> 
> The puspose of that is later to be able to disable altp2m support and
> exclude its code from the build completely, when not supported by target
> platform (as of now it's supported for VT-d only).
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Tamas K Lengyel <tamas@tklengyel.com>
> CC: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> changes in v2:
>  - patch description changed, removed VMX mentioning
>  - guard by altp2m_active() instead of hvm_altp2m_supported()
> ---
>  xen/arch/x86/hvm/monitor.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/hvm/monitor.c b/xen/arch/x86/hvm/monitor.c
> index 2a8ff07ec9..74621000b2 100644
> --- a/xen/arch/x86/hvm/monitor.c
> +++ b/xen/arch/x86/hvm/monitor.c
> @@ -262,6 +262,8 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t gfn, uint32_t pfec,
>      struct vcpu *curr = current;
>      vm_event_request_t req = {};
>      paddr_t gpa = (gfn_to_gaddr(gfn) | (gla & ~PAGE_MASK));
> +    unsigned int altp2m_idx = altp2m_active(curr->domain) ?
> +                              altp2m_vcpu_idx(curr) : 0;
>      int rc;
>  
>      ASSERT(curr->arch.vm_event->send_event);
> @@ -270,7 +272,7 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t gfn, uint32_t pfec,
>       * p2m_get_mem_access() can fail from a invalid MFN and return -ESRCH
>       * in which case access must be restricted.
>       */
> -    rc = p2m_get_mem_access(curr->domain, gfn, &access, altp2m_vcpu_idx(curr));
> +    rc = p2m_get_mem_access(curr->domain, gfn, &access, altp2m_idx);
>  
>      if ( rc == -ESRCH )
>          access = XENMEM_access_n;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:31:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:31:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722708.1126932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P2E-00044v-93; Thu, 16 May 2024 00:31:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722708.1126932; Thu, 16 May 2024 00:31:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P2E-00044o-5c; Thu, 16 May 2024 00:31:50 +0000
Received: by outflank-mailman (input) for mailman id 722708;
 Thu, 16 May 2024 00:31:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P2D-0003iX-2P
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:31:49 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id adc7ac1d-131b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:31:48 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 7449261636;
 Thu, 16 May 2024 00:31:46 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CF92FC116B1;
 Thu, 16 May 2024 00:31:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: adc7ac1d-131b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819506;
	bh=3Mx+mu9GUIW6ZMQAR9lVY3xbYV4lhP/F+VSt9iwW/oI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RPSZWTFkoisj584mOG8Nb5kcZCKd26mJ8IN93FeFRSCwu8FcCKLU4QQIIgzNdjSH5
	 up/3eNJDpo2fcsHOvlJAfn7KhDsMVaOsCLAlr3BVwhVPApgYFrvzk7UpYH8SD0WdGG
	 dBkgIiHAiZ9rmzNsAZlYgWYIzleVw3ndM6mEY1TKlO79JZ6FnHWyRmiLRiqANn0mLH
	 bldLzARtLb+4cFYrQv0DeG53jkXBJSCjmT7VSXHgIm/10criumbv/H1wTekJBAEJLF
	 ZBK4qessnSZi/p8zR8uiYeWzFVBs0cF0xksvAwlfCo1xEcGKL2Pp4jUVlSH1BD8Im+
	 de90rNzp1YbgA==
Date: Wed, 15 May 2024 17:31:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Tamas K Lengyel <tamas@tklengyel.com>
Subject: Re: [XEN PATCH v2 03/15] x86/p2m: guard altp2m routines
In-Reply-To: <d4c537b1bcac4b301b16336ea3407834fc3076b1.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151731190.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <d4c537b1bcac4b301b16336ea3407834fc3076b1.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Initialize and bring down altp2m only when it is supported by the platform,
> e.g. VMX. Also guard p2m_altp2m_propagate_change().
> The puspose of that is the possiblity to disable altp2m support and exclude its
> code from the build completely, when it's not supported by the target platform.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Tamas K Lengyel <tamas@tklengyel.com>
> ---
>  xen/arch/x86/mm/p2m-basic.c | 19 +++++++++++--------
>  xen/arch/x86/mm/p2m-ept.c   |  2 +-
>  2 files changed, 12 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/x86/mm/p2m-basic.c b/xen/arch/x86/mm/p2m-basic.c
> index 8599bd15c6..90106997d7 100644
> --- a/xen/arch/x86/mm/p2m-basic.c
> +++ b/xen/arch/x86/mm/p2m-basic.c
> @@ -126,13 +126,15 @@ int p2m_init(struct domain *d)
>          return rc;
>      }
>  
> -    rc = p2m_init_altp2m(d);
> -    if ( rc )
> +    if ( hvm_altp2m_supported() )
>      {
> -        p2m_teardown_hostp2m(d);
> -        p2m_teardown_nestedp2m(d);
> +        rc = p2m_init_altp2m(d);
> +        if ( rc )
> +        {
> +            p2m_teardown_hostp2m(d);
> +            p2m_teardown_nestedp2m(d);
> +        }
>      }
> -
>      return rc;
>  }
>  
> @@ -195,11 +197,12 @@ void p2m_final_teardown(struct domain *d)
>  {
>      if ( is_hvm_domain(d) )
>      {
> +        if ( hvm_altp2m_supported() )
> +            p2m_teardown_altp2m(d);
>          /*
> -         * We must tear down both of them unconditionally because
> -         * we initialise them unconditionally.
> +         * We must tear down nestedp2m unconditionally because
> +         * we initialise it unconditionally.
>           */
> -        p2m_teardown_altp2m(d);
>          p2m_teardown_nestedp2m(d);
>      }
>  
> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
> index f83610cb8c..d264df5b14 100644
> --- a/xen/arch/x86/mm/p2m-ept.c
> +++ b/xen/arch/x86/mm/p2m-ept.c
> @@ -986,7 +986,7 @@ out:
>      if ( is_epte_present(&old_entry) )
>          ept_free_entry(p2m, &old_entry, target);
>  
> -    if ( entry_written && p2m_is_hostp2m(p2m) )
> +    if ( entry_written && p2m_is_hostp2m(p2m) && hvm_altp2m_supported())
>      {
>          ret = p2m_altp2m_propagate_change(d, _gfn(gfn), mfn, order, p2mt, p2ma);
>          if ( !rc )

This is a matter of taste but I would put hvm_altp2m_supported() first.

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:33:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:33:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722711.1126942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P3z-0004fL-JS; Thu, 16 May 2024 00:33:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722711.1126942; Thu, 16 May 2024 00:33:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P3z-0004fE-Gi; Thu, 16 May 2024 00:33:39 +0000
Received: by outflank-mailman (input) for mailman id 722711;
 Thu, 16 May 2024 00:33:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P3y-0004f8-0t
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:33:38 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id edf9d4a1-131b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 02:33:35 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id A972861637;
 Thu, 16 May 2024 00:33:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 268C2C116B1;
 Thu, 16 May 2024 00:33:33 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: edf9d4a1-131b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819614;
	bh=wufE5MAe99888uZBmU34n3fYHQZDQKudRYwmkN0nIzw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HertGY37cas/q50fkWRT0guRLPsZ5oNhbJoVvxYZ47ZU0bR48vAfL9QVQhEMsQODo
	 2TsOjJeEu9YOqi1LG2wktyuRJHy0VXm5bCH4GaWvydeHyU9kmIF3pV8hH0JC897QAp
	 gCka/H1Qf1O/Pd9HZJBGtB/QcJaUkIcqUVRoAeFindvJxoaVDNCBOfeiCazz68Gkc9
	 IaqQgqF5aCmew8Lvsr78e9J74OogTB1bcJs26cwvPse2kM6XWTn/cyWr9UO7R3sZc7
	 ulQfbaLT9T6ia9xh93JHPxSQgsLIlB4GTus4s4aX3MG9+KR3/xg4JLRtW/PzQy0anV
	 GwKe2M+RJMW1w==
Date: Wed, 15 May 2024 17:33:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Tamas K Lengyel <tamas@tklengyel.com>
Subject: Re: [XEN PATCH v2 05/15] x86: introduce CONFIG_ALTP2M Kconfig
 option
In-Reply-To: <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151732490.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Add new option to make altp2m code inclusion optional.
> Currently altp2m support provided for VT-d only, so option is dependant on VMX.
> 
> No functional change intended.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Tamas K Lengyel <tamas@tklengyel.com>
> ---
>  xen/arch/x86/Kconfig | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 8c9f8431f0..2872b031a7 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -358,6 +358,11 @@ config REQUIRE_NX
>  	  was unavailable. However, if enabled, Xen will no longer boot on
>  	  any CPU which is lacking NX support.
>  
> +config ALTP2M
> +	bool "Alternate P2M support"
> +	def_bool y
> +	depends on VMX && EXPERT

I think we need a small help text here, something like

Alternate-p2m allows a guest to manage multiple p2m guest physical "memory
views" (as opposed to a single p2m). Useful for memory introspection.


>  endmenu
>  
>  source "common/Kconfig"
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:36:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:36:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722714.1126952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P6E-0005Ct-VT; Thu, 16 May 2024 00:35:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722714.1126952; Thu, 16 May 2024 00:35:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P6E-0005Cm-S8; Thu, 16 May 2024 00:35:58 +0000
Received: by outflank-mailman (input) for mailman id 722714;
 Thu, 16 May 2024 00:35:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P6D-0005Ce-39
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:35:57 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f783f5a-131c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:35:54 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 23CCACE1814;
 Thu, 16 May 2024 00:35:50 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B4ACC32782;
 Thu, 16 May 2024 00:35:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f783f5a-131c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819749;
	bh=ubQXcp4hG8ObvAs5nalAUPiEO7p0iDgBvitvg1NYWAc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=FGglG3sWTO61vbQYervNNy2sxZ47UfZKPsuHvpz1milpqA2B6BUMkdx+EYX6ykczw
	 LnS/sCR776ljEEuclPfY+2vjH2hiYPzExQR2OPJNhW9NnANk8rYWY++CmqNXsOzTw0
	 ztd5XMJBxRnTxfOVqczyYQJV4O8eP0rqwrviBNs92ZtCJnUzpMOd3maAqFJdjYc00q
	 phhiGrX1f+YqYP63uULuxXZoKnSdFX2OEcuX+dZJokrac0mI4sZyHtAGVjqAlVWpkt
	 CYXeZALJqYbN/6DpeFqPjw7k/4c46WnTDxUNuYE1U1rBiVfJkdDz0k+Es56LHFQOlW
	 dLvwq46unRINQ==
Date: Wed, 15 May 2024 17:35:45 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Tamas K Lengyel <tamas@tklengyel.com>
Subject: Re: [XEN PATCH v2 04/15] x86/p2m: move altp2m-related code to separate
 file
In-Reply-To: <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151735350.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Move altp2m code from generic p2m.c file to altp2m.c, so it is kept separately
> and can possibly be disabled in the build. We may want to disable it when
> building for specific platform only, that doesn't support alternate p2m.
> 
> No functional change intended.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Tamas K Lengyel <tamas@tklengyel.com>
> CC: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> changes in v2:
>  - no double blank lines
>  - no unrelated re-formatting
>  - header #include-s ordering
>  - changed patch description
> ---
>  xen/arch/x86/mm/altp2m.c | 630 ++++++++++++++++++++++++++++++++++++++
>  xen/arch/x86/mm/p2m.c    | 632 +--------------------------------------
>  xen/arch/x86/mm/p2m.h    |   3 +
>  3 files changed, 635 insertions(+), 630 deletions(-)
> 
> diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
> index a04297b646..6fe1e9ed6b 100644
> --- a/xen/arch/x86/mm/altp2m.c
> +++ b/xen/arch/x86/mm/altp2m.c
> @@ -7,6 +7,8 @@
>  #include <asm/hvm/hvm.h>
>  #include <asm/p2m.h>
>  #include <asm/altp2m.h>
> +#include <public/hvm/hvm_op.h>
> +#include <xen/event.h>
>  #include "mm-locks.h"
>  #include "p2m.h"
>  
> @@ -151,6 +153,634 @@ void p2m_teardown_altp2m(struct domain *d)
>      }
>  }
>  
> +int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t *mfn,
> +                               p2m_type_t *t, p2m_access_t *a,
> +                               bool prepopulate)
> +{
> +    *mfn = ap2m->get_entry(ap2m, gfn, t, a, 0, NULL, NULL);
> +
> +    /* Check host p2m if no valid entry in alternate */
> +    if ( !mfn_valid(*mfn) && !p2m_is_hostp2m(ap2m) )
> +    {
> +        struct p2m_domain *hp2m = p2m_get_hostp2m(ap2m->domain);
> +        unsigned int page_order;
> +        int rc;
> +
> +        *mfn = p2m_get_gfn_type_access(hp2m, gfn, t, a, P2M_ALLOC | P2M_UNSHARE,
> +                                       &page_order, 0);
> +
> +        rc = -ESRCH;
> +        if ( !mfn_valid(*mfn) || *t != p2m_ram_rw )
> +            return rc;
> +
> +        /* If this is a superpage, copy that first */
> +        if ( prepopulate && page_order != PAGE_ORDER_4K )
> +        {
> +            unsigned long mask = ~((1UL << page_order) - 1);
> +            gfn_t gfn_aligned = _gfn(gfn_x(gfn) & mask);
> +            mfn_t mfn_aligned = _mfn(mfn_x(*mfn) & mask);
> +
> +            rc = ap2m->set_entry(ap2m, gfn_aligned, mfn_aligned, page_order, *t, *a, 1);
> +            if ( rc )
> +                return rc;
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
> +{
> +    if ( altp2m_active(v->domain) )
> +        p2m_switch_vcpu_altp2m_by_id(v, idx);
> +}
> +
> +bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
> +{
> +    struct domain *d = v->domain;
> +    bool rc = false;
> +
> +    if ( idx >= MAX_ALTP2M )
> +        return rc;
> +
> +    altp2m_list_lock(d);
> +
> +    if ( d->arch.altp2m_eptp[idx] != mfn_x(INVALID_MFN) )
> +    {
> +        if ( p2m_set_altp2m(v, idx) )
> +            altp2m_vcpu_update_p2m(v);
> +        rc = 1;
> +    }
> +
> +    altp2m_list_unlock(d);
> +    return rc;
> +}
> +
> +/*
> + * Read info about the gfn in an altp2m, locking the gfn.
> + *
> + * If the entry is valid, pass the results back to the caller.
> + *
> + * If the entry was invalid, and the host's entry is also invalid,
> + * return to the caller without any changes.
> + *
> + * If the entry is invalid, and the host entry was valid, propagate
> + * the host's entry to the altp2m (retaining page order), and indicate
> + * that the caller should re-try the faulting instruction.
> + */
> +bool p2m_altp2m_get_or_propagate(struct p2m_domain *ap2m, unsigned long gfn_l,
> +                                 mfn_t *mfn, p2m_type_t *p2mt,
> +                                 p2m_access_t *p2ma, unsigned int *page_order)
> +{
> +    p2m_type_t ap2mt;
> +    p2m_access_t ap2ma;
> +    unsigned int cur_order;
> +    unsigned long mask;
> +    gfn_t gfn;
> +    mfn_t amfn;
> +    int rc;
> +
> +    /*
> +     * NB we must get the full lock on the altp2m here, in addition to
> +     * the lock on the individual gfn, since we may change a range of
> +     * gfns below.
> +     */
> +    p2m_lock(ap2m);
> +
> +    amfn = get_gfn_type_access(ap2m, gfn_l, &ap2mt, &ap2ma, 0, &cur_order);
> +
> +    if ( cur_order > *page_order )
> +        cur_order = *page_order;
> +
> +    if ( !mfn_eq(amfn, INVALID_MFN) )
> +    {
> +        p2m_unlock(ap2m);
> +        *mfn  = amfn;
> +        *p2mt = ap2mt;
> +        *p2ma = ap2ma;
> +        *page_order = cur_order;
> +        return false;
> +    }
> +
> +    /* Host entry is also invalid; don't bother setting the altp2m entry. */
> +    if ( mfn_eq(*mfn, INVALID_MFN) )
> +    {
> +        p2m_unlock(ap2m);
> +        *page_order = cur_order;
> +        return false;
> +    }
> +
> +    /*
> +     * If this is a superpage mapping, round down both frame numbers
> +     * to the start of the superpage.  NB that we repupose `amfn`
> +     * here.
> +     */
> +    mask = ~((1UL << cur_order) - 1);
> +    amfn = _mfn(mfn_x(*mfn) & mask);
> +    gfn = _gfn(gfn_l & mask);
> +
> +    /* Override the altp2m entry with its default access. */
> +    *p2ma = ap2m->default_access;
> +
> +    rc = p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
> +    p2m_unlock(ap2m);
> +
> +    if ( rc )
> +    {
> +        gprintk(XENLOG_ERR,
> +                "failed to set entry for %"PRI_gfn" -> %"PRI_mfn" altp2m %u, rc %d\n",
> +                gfn_l, mfn_x(amfn), vcpu_altp2m(current).p2midx, rc);
> +        domain_crash(ap2m->domain);
> +    }
> +
> +    return true;
> +}
> +
> +enum altp2m_reset_type {
> +    ALTP2M_RESET,
> +    ALTP2M_DEACTIVATE
> +};
> +
> +static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
> +                             enum altp2m_reset_type reset_type)
> +{
> +    struct p2m_domain *p2m;
> +
> +    ASSERT(idx < MAX_ALTP2M);
> +    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +
> +    p2m_lock(p2m);
> +
> +    p2m_flush_table_locked(p2m);
> +
> +    if ( reset_type == ALTP2M_DEACTIVATE )
> +        p2m_free_logdirty(p2m);
> +
> +    /* Uninit and reinit ept to force TLB shootdown */
> +    ept_p2m_uninit(p2m);
> +    ept_p2m_init(p2m);
> +
> +    p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
> +    p2m->max_remapped_gfn = 0;
> +
> +    p2m_unlock(p2m);
> +}
> +
> +void p2m_flush_altp2m(struct domain *d)
> +{
> +    unsigned int i;
> +
> +    altp2m_list_lock(d);
> +
> +    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    {
> +        p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
> +        d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
> +        d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
> +    }
> +
> +    altp2m_list_unlock(d);
> +}
> +
> +static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
> +                               p2m_access_t hvmmem_default_access)
> +{
> +    struct p2m_domain *hostp2m, *p2m;
> +    int rc;
> +
> +    ASSERT(idx < MAX_ALTP2M);
> +
> +    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +    hostp2m = p2m_get_hostp2m(d);
> +
> +    p2m_lock(p2m);
> +
> +    rc = p2m_init_logdirty(p2m);
> +
> +    if ( rc )
> +        goto out;
> +
> +    /* The following is really just a rangeset copy. */
> +    rc = rangeset_merge(p2m->logdirty_ranges, hostp2m->logdirty_ranges);
> +
> +    if ( rc )
> +    {
> +        p2m_free_logdirty(p2m);
> +        goto out;
> +    }
> +
> +    p2m->default_access = hvmmem_default_access;
> +    p2m->domain = hostp2m->domain;
> +    p2m->global_logdirty = hostp2m->global_logdirty;
> +    p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
> +    p2m->max_mapped_pfn = p2m->max_remapped_gfn = 0;
> +
> +    p2m_init_altp2m_ept(d, idx);
> +
> + out:
> +    p2m_unlock(p2m);
> +
> +    return rc;
> +}
> +
> +int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
> +{
> +    int rc = -EINVAL;
> +    struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> +
> +    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> +        return rc;
> +
> +    altp2m_list_lock(d);
> +
> +    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> +         mfn_x(INVALID_MFN) )
> +        rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);
> +
> +    altp2m_list_unlock(d);
> +    return rc;
> +}
> +
> +int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
> +                         xenmem_access_t hvmmem_default_access)
> +{
> +    int rc = -EINVAL;
> +    unsigned int i;
> +    p2m_access_t a;
> +    struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> +
> +    if ( hvmmem_default_access > XENMEM_access_default ||
> +         !xenmem_access_to_p2m_access(hostp2m, hvmmem_default_access, &a) )
> +        return rc;
> +
> +    altp2m_list_lock(d);
> +
> +    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    {
> +        if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
> +            continue;
> +
> +        rc = p2m_activate_altp2m(d, i, a);
> +
> +        if ( !rc )
> +            *idx = i;
> +
> +        break;
> +    }
> +
> +    altp2m_list_unlock(d);
> +    return rc;
> +}
> +
> +int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
> +{
> +    struct p2m_domain *p2m;
> +    int rc = -EBUSY;
> +
> +    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> +        return rc;
> +
> +    rc = domain_pause_except_self(d);
> +    if ( rc )
> +        return rc;
> +
> +    rc = -EBUSY;
> +    altp2m_list_lock(d);
> +
> +    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
> +         mfn_x(INVALID_MFN) )
> +    {
> +        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +
> +        if ( !_atomic_read(p2m->active_vcpus) )
> +        {
> +            p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
> +            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =
> +                mfn_x(INVALID_MFN);
> +            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP)] =
> +                mfn_x(INVALID_MFN);
> +            rc = 0;
> +        }
> +    }
> +
> +    altp2m_list_unlock(d);
> +
> +    domain_unpause_except_self(d);
> +
> +    return rc;
> +}
> +
> +int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
> +{
> +    struct vcpu *v;
> +    int rc = -EINVAL;
> +
> +    if ( idx >= MAX_ALTP2M )
> +        return rc;
> +
> +    rc = domain_pause_except_self(d);
> +    if ( rc )
> +        return rc;
> +
> +    rc = -EINVAL;
> +    altp2m_list_lock(d);
> +
> +    if ( d->arch.altp2m_visible_eptp[idx] != mfn_x(INVALID_MFN) )
> +    {
> +        for_each_vcpu( d, v )
> +            if ( p2m_set_altp2m(v, idx) )
> +                altp2m_vcpu_update_p2m(v);
> +
> +        rc = 0;
> +    }
> +
> +    altp2m_list_unlock(d);
> +
> +    domain_unpause_except_self(d);
> +
> +    return rc;
> +}
> +
> +int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
> +                          gfn_t old_gfn, gfn_t new_gfn)
> +{
> +    struct p2m_domain *hp2m, *ap2m;
> +    p2m_access_t a;
> +    p2m_type_t t;
> +    mfn_t mfn;
> +    int rc = -EINVAL;
> +
> +    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> +         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> +         mfn_x(INVALID_MFN) )
> +        return rc;
> +
> +    hp2m = p2m_get_hostp2m(d);
> +    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +
> +    p2m_lock(hp2m);
> +    p2m_lock(ap2m);
> +
> +    if ( gfn_eq(new_gfn, INVALID_GFN) )
> +    {
> +        mfn = ap2m->get_entry(ap2m, old_gfn, &t, &a, 0, NULL, NULL);
> +        rc = mfn_valid(mfn)
> +             ? p2m_remove_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K)
> +             : 0;
> +        goto out;
> +    }
> +
> +    rc = altp2m_get_effective_entry(ap2m, old_gfn, &mfn, &t, &a,
> +                                    AP2MGET_prepopulate);
> +    if ( rc )
> +        goto out;
> +
> +    rc = altp2m_get_effective_entry(ap2m, new_gfn, &mfn, &t, &a,
> +                                    AP2MGET_query);
> +    if ( rc )
> +        goto out;
> +
> +    if ( !ap2m->set_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K, t, a,
> +                          (current->domain != d)) )
> +    {
> +        rc = 0;
> +
> +        if ( gfn_x(new_gfn) < ap2m->min_remapped_gfn )
> +            ap2m->min_remapped_gfn = gfn_x(new_gfn);
> +        if ( gfn_x(new_gfn) > ap2m->max_remapped_gfn )
> +            ap2m->max_remapped_gfn = gfn_x(new_gfn);
> +    }
> +
> + out:
> +    p2m_unlock(ap2m);
> +    p2m_unlock(hp2m);
> +    return rc;
> +}
> +
> +int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
> +                                mfn_t mfn, unsigned int page_order,
> +                                p2m_type_t p2mt, p2m_access_t p2ma)
> +{
> +    struct p2m_domain *p2m;
> +    unsigned int i;
> +    unsigned int reset_count = 0;
> +    unsigned int last_reset_idx = ~0;
> +    int ret = 0;
> +
> +    if ( !altp2m_active(d) )
> +        return 0;
> +
> +    altp2m_list_lock(d);
> +
> +    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    {
> +        p2m_type_t t;
> +        p2m_access_t a;
> +
> +        if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
> +            continue;
> +
> +        p2m = d->arch.altp2m_p2m[i];
> +
> +        /* Check for a dropped page that may impact this altp2m */
> +        if ( mfn_eq(mfn, INVALID_MFN) &&
> +             gfn_x(gfn) + (1UL << page_order) > p2m->min_remapped_gfn &&
> +             gfn_x(gfn) <= p2m->max_remapped_gfn )
> +        {
> +            if ( !reset_count++ )
> +            {
> +                p2m_reset_altp2m(d, i, ALTP2M_RESET);
> +                last_reset_idx = i;
> +            }
> +            else
> +            {
> +                /* At least 2 altp2m's impacted, so reset everything */
> +                for ( i = 0; i < MAX_ALTP2M; i++ )
> +                {
> +                    if ( i == last_reset_idx ||
> +                         d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
> +                        continue;
> +
> +                    p2m_reset_altp2m(d, i, ALTP2M_RESET);
> +                }
> +
> +                ret = 0;
> +                break;
> +            }
> +        }
> +        else if ( !mfn_eq(get_gfn_type_access(p2m, gfn_x(gfn), &t, &a, 0,
> +                                              NULL), INVALID_MFN) )
> +        {
> +            int rc = p2m_set_entry(p2m, gfn, mfn, page_order, p2mt, p2ma);
> +
> +            /* Best effort: Don't bail on error. */
> +            if ( !ret )
> +                ret = rc;
> +
> +            p2m_put_gfn(p2m, gfn);
> +        }
> +        else
> +            p2m_put_gfn(p2m, gfn);
> +    }
> +
> +    altp2m_list_unlock(d);
> +
> +    return ret;
> +}
> +
> +/*
> + * Set/clear the #VE suppress bit for a page.  Only available on VMX.
> + */
> +int p2m_set_suppress_ve(struct domain *d, gfn_t gfn, bool suppress_ve,
> +                        unsigned int altp2m_idx)
> +{
> +    int rc;
> +    struct xen_hvm_altp2m_suppress_ve_multi sve = {
> +        altp2m_idx, suppress_ve, 0, 0, gfn_x(gfn), gfn_x(gfn), 0
> +    };
> +
> +    if ( !(rc = p2m_set_suppress_ve_multi(d, &sve)) )
> +        rc = sve.first_error;
> +
> +    return rc;
> +}
> +
> +/*
> + * Set/clear the #VE suppress bit for multiple pages.  Only available on VMX.
> + */
> +int p2m_set_suppress_ve_multi(struct domain *d,
> +                              struct xen_hvm_altp2m_suppress_ve_multi *sve)
> +{
> +    struct p2m_domain *host_p2m = p2m_get_hostp2m(d);
> +    struct p2m_domain *ap2m = NULL;
> +    struct p2m_domain *p2m = host_p2m;
> +    uint64_t start = sve->first_gfn;
> +    int rc = 0;
> +
> +    if ( sve->view > 0 )
> +    {
> +        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> +             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
> +             mfn_x(INVALID_MFN) )
> +            return -EINVAL;
> +
> +        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
> +    }
> +
> +    p2m_lock(host_p2m);
> +
> +    if ( ap2m )
> +        p2m_lock(ap2m);
> +
> +    while ( sve->last_gfn >= start )
> +    {
> +        p2m_access_t a;
> +        p2m_type_t t;
> +        mfn_t mfn;
> +        int err = 0;
> +
> +        if ( (err = altp2m_get_effective_entry(p2m, _gfn(start), &mfn, &t, &a,
> +                                               AP2MGET_query)) &&
> +             !sve->first_error )
> +        {
> +            sve->first_error_gfn = start; /* Save the gfn of the first error */
> +            sve->first_error = err; /* Save the first error code */
> +        }
> +
> +        if ( !err && (err = p2m->set_entry(p2m, _gfn(start), mfn,
> +                                           PAGE_ORDER_4K, t, a,
> +                                           sve->suppress_ve)) &&
> +             !sve->first_error )
> +        {
> +            sve->first_error_gfn = start; /* Save the gfn of the first error */
> +            sve->first_error = err; /* Save the first error code */
> +        }
> +
> +        /* Check for continuation if it's not the last iteration. */
> +        if ( sve->last_gfn >= ++start && hypercall_preempt_check() )
> +        {
> +            rc = -ERESTART;
> +            break;
> +        }
> +    }
> +
> +    sve->first_gfn = start;
> +
> +    if ( ap2m )
> +        p2m_unlock(ap2m);
> +
> +    p2m_unlock(host_p2m);
> +
> +    return rc;
> +}
> +
> +int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
> +                        unsigned int altp2m_idx)
> +{
> +    struct p2m_domain *host_p2m = p2m_get_hostp2m(d);
> +    struct p2m_domain *ap2m = NULL;
> +    struct p2m_domain *p2m;
> +    mfn_t mfn;
> +    p2m_access_t a;
> +    p2m_type_t t;
> +    int rc = 0;
> +
> +    if ( altp2m_idx > 0 )
> +    {
> +        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> +             mfn_x(INVALID_MFN) )
> +            return -EINVAL;
> +
> +        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +    }
> +    else
> +        p2m = host_p2m;
> +
> +    gfn_lock(host_p2m, gfn, 0);
> +
> +    if ( ap2m )
> +        p2m_lock(ap2m);
> +
> +    mfn = p2m->get_entry(p2m, gfn, &t, &a, 0, NULL, suppress_ve);
> +    if ( !mfn_valid(mfn) )
> +        rc = -ESRCH;
> +
> +    if ( ap2m )
> +        p2m_unlock(ap2m);
> +
> +    gfn_unlock(host_p2m, gfn, 0);
> +
> +    return rc;
> +}
> +
> +int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> +                                   uint8_t visible)
> +{
> +    int rc = 0;
> +
> +    altp2m_list_lock(d);
> +
> +    /*
> +     * Eptp index is correlated with altp2m index and should not exceed
> +     * min(MAX_ALTP2M, MAX_EPTP).
> +     */
> +    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> +         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> +         mfn_x(INVALID_MFN) )
> +        rc = -EINVAL;
> +    else if ( visible )
> +        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> +            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)];
> +    else
> +        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> +            mfn_x(INVALID_MFN);
> +
> +    altp2m_list_unlock(d);
> +
> +    return rc;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
> index ce742c12e0..7c422a2d7e 100644
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -500,7 +500,7 @@ int p2m_alloc_table(struct p2m_domain *p2m)
>      return 0;
>  }
>  
> -static int __must_check
> +int __must_check
>  p2m_remove_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
>                   unsigned int page_order)
>  {
> @@ -1329,7 +1329,7 @@ p2m_getlru_nestedp2m(struct domain *d, struct p2m_domain *p2m)
>      return p2m;
>  }
>  
> -static void
> +void
>  p2m_flush_table_locked(struct p2m_domain *p2m)
>  {
>      struct page_info *top, *pg;
> @@ -1729,481 +1729,6 @@ int unmap_mmio_regions(struct domain *d,
>      return i == nr ? 0 : i ?: ret;
>  }
>  
> -int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t *mfn,
> -                               p2m_type_t *t, p2m_access_t *a,
> -                               bool prepopulate)
> -{
> -    *mfn = ap2m->get_entry(ap2m, gfn, t, a, 0, NULL, NULL);
> -
> -    /* Check host p2m if no valid entry in alternate */
> -    if ( !mfn_valid(*mfn) && !p2m_is_hostp2m(ap2m) )
> -    {
> -        struct p2m_domain *hp2m = p2m_get_hostp2m(ap2m->domain);
> -        unsigned int page_order;
> -        int rc;
> -
> -        *mfn = p2m_get_gfn_type_access(hp2m, gfn, t, a, P2M_ALLOC | P2M_UNSHARE,
> -                                       &page_order, 0);
> -
> -        rc = -ESRCH;
> -        if ( !mfn_valid(*mfn) || *t != p2m_ram_rw )
> -            return rc;
> -
> -        /* If this is a superpage, copy that first */
> -        if ( prepopulate && page_order != PAGE_ORDER_4K )
> -        {
> -            unsigned long mask = ~((1UL << page_order) - 1);
> -            gfn_t gfn_aligned = _gfn(gfn_x(gfn) & mask);
> -            mfn_t mfn_aligned = _mfn(mfn_x(*mfn) & mask);
> -
> -            rc = ap2m->set_entry(ap2m, gfn_aligned, mfn_aligned, page_order, *t, *a, 1);
> -            if ( rc )
> -                return rc;
> -        }
> -    }
> -
> -    return 0;
> -}
> -
> -void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
> -{
> -    if ( altp2m_active(v->domain) )
> -        p2m_switch_vcpu_altp2m_by_id(v, idx);
> -}
> -
> -bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
> -{
> -    struct domain *d = v->domain;
> -    bool rc = false;
> -
> -    if ( idx >= MAX_ALTP2M )
> -        return rc;
> -
> -    altp2m_list_lock(d);
> -
> -    if ( d->arch.altp2m_eptp[idx] != mfn_x(INVALID_MFN) )
> -    {
> -        if ( p2m_set_altp2m(v, idx) )
> -            altp2m_vcpu_update_p2m(v);
> -        rc = 1;
> -    }
> -
> -    altp2m_list_unlock(d);
> -    return rc;
> -}
> -
> -/*
> - * Read info about the gfn in an altp2m, locking the gfn.
> - *
> - * If the entry is valid, pass the results back to the caller.
> - *
> - * If the entry was invalid, and the host's entry is also invalid,
> - * return to the caller without any changes.
> - *
> - * If the entry is invalid, and the host entry was valid, propagate
> - * the host's entry to the altp2m (retaining page order), and indicate
> - * that the caller should re-try the faulting instruction.
> - */
> -bool p2m_altp2m_get_or_propagate(struct p2m_domain *ap2m, unsigned long gfn_l,
> -                                 mfn_t *mfn, p2m_type_t *p2mt,
> -                                 p2m_access_t *p2ma, unsigned int *page_order)
> -{
> -    p2m_type_t ap2mt;
> -    p2m_access_t ap2ma;
> -    unsigned int cur_order;
> -    unsigned long mask;
> -    gfn_t gfn;
> -    mfn_t amfn;
> -    int rc;
> -
> -    /*
> -     * NB we must get the full lock on the altp2m here, in addition to
> -     * the lock on the individual gfn, since we may change a range of
> -     * gfns below.
> -     */
> -    p2m_lock(ap2m);
> -
> -    amfn = get_gfn_type_access(ap2m, gfn_l, &ap2mt, &ap2ma, 0, &cur_order);
> -
> -    if ( cur_order > *page_order )
> -        cur_order = *page_order;
> -
> -    if ( !mfn_eq(amfn, INVALID_MFN) )
> -    {
> -        p2m_unlock(ap2m);
> -        *mfn  = amfn;
> -        *p2mt = ap2mt;
> -        *p2ma = ap2ma;
> -        *page_order = cur_order;
> -        return false;
> -    }
> -
> -    /* Host entry is also invalid; don't bother setting the altp2m entry. */
> -    if ( mfn_eq(*mfn, INVALID_MFN) )
> -    {
> -        p2m_unlock(ap2m);
> -        *page_order = cur_order;
> -        return false;
> -    }
> -
> -    /*
> -     * If this is a superpage mapping, round down both frame numbers
> -     * to the start of the superpage.  NB that we repupose `amfn`
> -     * here.
> -     */
> -    mask = ~((1UL << cur_order) - 1);
> -    amfn = _mfn(mfn_x(*mfn) & mask);
> -    gfn = _gfn(gfn_l & mask);
> -
> -    /* Override the altp2m entry with its default access. */
> -    *p2ma = ap2m->default_access;
> -
> -    rc = p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
> -    p2m_unlock(ap2m);
> -
> -    if ( rc )
> -    {
> -        gprintk(XENLOG_ERR,
> -                "failed to set entry for %"PRI_gfn" -> %"PRI_mfn" altp2m %u, rc %d\n",
> -                gfn_l, mfn_x(amfn), vcpu_altp2m(current).p2midx, rc);
> -        domain_crash(ap2m->domain);
> -    }
> -
> -    return true;
> -}
> -
> -enum altp2m_reset_type {
> -    ALTP2M_RESET,
> -    ALTP2M_DEACTIVATE
> -};
> -
> -static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
> -                             enum altp2m_reset_type reset_type)
> -{
> -    struct p2m_domain *p2m;
> -
> -    ASSERT(idx < MAX_ALTP2M);
> -    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> -
> -    p2m_lock(p2m);
> -
> -    p2m_flush_table_locked(p2m);
> -
> -    if ( reset_type == ALTP2M_DEACTIVATE )
> -        p2m_free_logdirty(p2m);
> -
> -    /* Uninit and reinit ept to force TLB shootdown */
> -    ept_p2m_uninit(p2m);
> -    ept_p2m_init(p2m);
> -
> -    p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
> -    p2m->max_remapped_gfn = 0;
> -
> -    p2m_unlock(p2m);
> -}
> -
> -void p2m_flush_altp2m(struct domain *d)
> -{
> -    unsigned int i;
> -
> -    altp2m_list_lock(d);
> -
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> -    {
> -        p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
> -        d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
> -        d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
> -    }
> -
> -    altp2m_list_unlock(d);
> -}
> -
> -static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
> -                               p2m_access_t hvmmem_default_access)
> -{
> -    struct p2m_domain *hostp2m, *p2m;
> -    int rc;
> -
> -    ASSERT(idx < MAX_ALTP2M);
> -
> -    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> -    hostp2m = p2m_get_hostp2m(d);
> -
> -    p2m_lock(p2m);
> -
> -    rc = p2m_init_logdirty(p2m);
> -
> -    if ( rc )
> -        goto out;
> -
> -    /* The following is really just a rangeset copy. */
> -    rc = rangeset_merge(p2m->logdirty_ranges, hostp2m->logdirty_ranges);
> -
> -    if ( rc )
> -    {
> -        p2m_free_logdirty(p2m);
> -        goto out;
> -    }
> -
> -    p2m->default_access = hvmmem_default_access;
> -    p2m->domain = hostp2m->domain;
> -    p2m->global_logdirty = hostp2m->global_logdirty;
> -    p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
> -    p2m->max_mapped_pfn = p2m->max_remapped_gfn = 0;
> -
> -    p2m_init_altp2m_ept(d, idx);
> -
> - out:
> -    p2m_unlock(p2m);
> -
> -    return rc;
> -}
> -
> -int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
> -{
> -    int rc = -EINVAL;
> -    struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> -
> -    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> -        return rc;
> -
> -    altp2m_list_lock(d);
> -
> -    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> -         mfn_x(INVALID_MFN) )
> -        rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);
> -
> -    altp2m_list_unlock(d);
> -    return rc;
> -}
> -
> -int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
> -                         xenmem_access_t hvmmem_default_access)
> -{
> -    int rc = -EINVAL;
> -    unsigned int i;
> -    p2m_access_t a;
> -    struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> -
> -    if ( hvmmem_default_access > XENMEM_access_default ||
> -         !xenmem_access_to_p2m_access(hostp2m, hvmmem_default_access, &a) )
> -        return rc;
> -
> -    altp2m_list_lock(d);
> -
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> -    {
> -        if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
> -            continue;
> -
> -        rc = p2m_activate_altp2m(d, i, a);
> -
> -        if ( !rc )
> -            *idx = i;
> -
> -        break;
> -    }
> -
> -    altp2m_list_unlock(d);
> -    return rc;
> -}
> -
> -int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
> -{
> -    struct p2m_domain *p2m;
> -    int rc = -EBUSY;
> -
> -    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> -        return rc;
> -
> -    rc = domain_pause_except_self(d);
> -    if ( rc )
> -        return rc;
> -
> -    rc = -EBUSY;
> -    altp2m_list_lock(d);
> -
> -    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
> -         mfn_x(INVALID_MFN) )
> -    {
> -        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> -
> -        if ( !_atomic_read(p2m->active_vcpus) )
> -        {
> -            p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
> -            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =
> -                mfn_x(INVALID_MFN);
> -            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP)] =
> -                mfn_x(INVALID_MFN);
> -            rc = 0;
> -        }
> -    }
> -
> -    altp2m_list_unlock(d);
> -
> -    domain_unpause_except_self(d);
> -
> -    return rc;
> -}
> -
> -int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
> -{
> -    struct vcpu *v;
> -    int rc = -EINVAL;
> -
> -    if ( idx >= MAX_ALTP2M )
> -        return rc;
> -
> -    rc = domain_pause_except_self(d);
> -    if ( rc )
> -        return rc;
> -
> -    rc = -EINVAL;
> -    altp2m_list_lock(d);
> -
> -    if ( d->arch.altp2m_visible_eptp[idx] != mfn_x(INVALID_MFN) )
> -    {
> -        for_each_vcpu( d, v )
> -            if ( p2m_set_altp2m(v, idx) )
> -                altp2m_vcpu_update_p2m(v);
> -
> -        rc = 0;
> -    }
> -
> -    altp2m_list_unlock(d);
> -
> -    domain_unpause_except_self(d);
> -
> -    return rc;
> -}
> -
> -int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
> -                          gfn_t old_gfn, gfn_t new_gfn)
> -{
> -    struct p2m_domain *hp2m, *ap2m;
> -    p2m_access_t a;
> -    p2m_type_t t;
> -    mfn_t mfn;
> -    int rc = -EINVAL;
> -
> -    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> -         mfn_x(INVALID_MFN) )
> -        return rc;
> -
> -    hp2m = p2m_get_hostp2m(d);
> -    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> -
> -    p2m_lock(hp2m);
> -    p2m_lock(ap2m);
> -
> -    if ( gfn_eq(new_gfn, INVALID_GFN) )
> -    {
> -        mfn = ap2m->get_entry(ap2m, old_gfn, &t, &a, 0, NULL, NULL);
> -        rc = mfn_valid(mfn)
> -             ? p2m_remove_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K)
> -             : 0;
> -        goto out;
> -    }
> -
> -    rc = altp2m_get_effective_entry(ap2m, old_gfn, &mfn, &t, &a,
> -                                    AP2MGET_prepopulate);
> -    if ( rc )
> -        goto out;
> -
> -    rc = altp2m_get_effective_entry(ap2m, new_gfn, &mfn, &t, &a,
> -                                    AP2MGET_query);
> -    if ( rc )
> -        goto out;
> -
> -    if ( !ap2m->set_entry(ap2m, old_gfn, mfn, PAGE_ORDER_4K, t, a,
> -                          (current->domain != d)) )
> -    {
> -        rc = 0;
> -
> -        if ( gfn_x(new_gfn) < ap2m->min_remapped_gfn )
> -            ap2m->min_remapped_gfn = gfn_x(new_gfn);
> -        if ( gfn_x(new_gfn) > ap2m->max_remapped_gfn )
> -            ap2m->max_remapped_gfn = gfn_x(new_gfn);
> -    }
> -
> - out:
> -    p2m_unlock(ap2m);
> -    p2m_unlock(hp2m);
> -    return rc;
> -}
> -
> -int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
> -                                mfn_t mfn, unsigned int page_order,
> -                                p2m_type_t p2mt, p2m_access_t p2ma)
> -{
> -    struct p2m_domain *p2m;
> -    unsigned int i;
> -    unsigned int reset_count = 0;
> -    unsigned int last_reset_idx = ~0;
> -    int ret = 0;
> -
> -    if ( !altp2m_active(d) )
> -        return 0;
> -
> -    altp2m_list_lock(d);
> -
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> -    {
> -        p2m_type_t t;
> -        p2m_access_t a;
> -
> -        if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
> -            continue;
> -
> -        p2m = d->arch.altp2m_p2m[i];
> -
> -        /* Check for a dropped page that may impact this altp2m */
> -        if ( mfn_eq(mfn, INVALID_MFN) &&
> -             gfn_x(gfn) + (1UL << page_order) > p2m->min_remapped_gfn &&
> -             gfn_x(gfn) <= p2m->max_remapped_gfn )
> -        {
> -            if ( !reset_count++ )
> -            {
> -                p2m_reset_altp2m(d, i, ALTP2M_RESET);
> -                last_reset_idx = i;
> -            }
> -            else
> -            {
> -                /* At least 2 altp2m's impacted, so reset everything */
> -                for ( i = 0; i < MAX_ALTP2M; i++ )
> -                {
> -                    if ( i == last_reset_idx ||
> -                         d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
> -                        continue;
> -
> -                    p2m_reset_altp2m(d, i, ALTP2M_RESET);
> -                }
> -
> -                ret = 0;
> -                break;
> -            }
> -        }
> -        else if ( !mfn_eq(get_gfn_type_access(p2m, gfn_x(gfn), &t, &a, 0,
> -                                              NULL), INVALID_MFN) )
> -        {
> -            int rc = p2m_set_entry(p2m, gfn, mfn, page_order, p2mt, p2ma);
> -
> -            /* Best effort: Don't bail on error. */
> -            if ( !ret )
> -                ret = rc;
> -
> -            p2m_put_gfn(p2m, gfn);
> -        }
> -        else
> -            p2m_put_gfn(p2m, gfn);
> -    }
> -
> -    altp2m_list_unlock(d);
> -
> -    return ret;
> -}
> -
>  /*** Audit ***/
>  
>  #if P2M_AUDIT
> @@ -2540,159 +2065,6 @@ int xenmem_add_to_physmap_one(
>      return rc;
>  }
>  
> -/*
> - * Set/clear the #VE suppress bit for a page.  Only available on VMX.
> - */
> -int p2m_set_suppress_ve(struct domain *d, gfn_t gfn, bool suppress_ve,
> -                        unsigned int altp2m_idx)
> -{
> -    int rc;
> -    struct xen_hvm_altp2m_suppress_ve_multi sve = {
> -        altp2m_idx, suppress_ve, 0, 0, gfn_x(gfn), gfn_x(gfn), 0
> -    };
> -
> -    if ( !(rc = p2m_set_suppress_ve_multi(d, &sve)) )
> -        rc = sve.first_error;
> -
> -    return rc;
> -}
> -
> -/*
> - * Set/clear the #VE suppress bit for multiple pages.  Only available on VMX.
> - */
> -int p2m_set_suppress_ve_multi(struct domain *d,
> -                              struct xen_hvm_altp2m_suppress_ve_multi *sve)
> -{
> -    struct p2m_domain *host_p2m = p2m_get_hostp2m(d);
> -    struct p2m_domain *ap2m = NULL;
> -    struct p2m_domain *p2m = host_p2m;
> -    uint64_t start = sve->first_gfn;
> -    int rc = 0;
> -
> -    if ( sve->view > 0 )
> -    {
> -        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
> -             mfn_x(INVALID_MFN) )
> -            return -EINVAL;
> -
> -        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
> -    }
> -
> -    p2m_lock(host_p2m);
> -
> -    if ( ap2m )
> -        p2m_lock(ap2m);
> -
> -    while ( sve->last_gfn >= start )
> -    {
> -        p2m_access_t a;
> -        p2m_type_t t;
> -        mfn_t mfn;
> -        int err = 0;
> -
> -        if ( (err = altp2m_get_effective_entry(p2m, _gfn(start), &mfn, &t, &a,
> -                                               AP2MGET_query)) &&
> -             !sve->first_error )
> -        {
> -            sve->first_error_gfn = start; /* Save the gfn of the first error */
> -            sve->first_error = err; /* Save the first error code */
> -        }
> -
> -        if ( !err && (err = p2m->set_entry(p2m, _gfn(start), mfn,
> -                                           PAGE_ORDER_4K, t, a,
> -                                           sve->suppress_ve)) &&
> -             !sve->first_error )
> -        {
> -            sve->first_error_gfn = start; /* Save the gfn of the first error */
> -            sve->first_error = err; /* Save the first error code */
> -        }
> -
> -        /* Check for continuation if it's not the last iteration. */
> -        if ( sve->last_gfn >= ++start && hypercall_preempt_check() )
> -        {
> -            rc = -ERESTART;
> -            break;
> -        }
> -    }
> -
> -    sve->first_gfn = start;
> -
> -    if ( ap2m )
> -        p2m_unlock(ap2m);
> -
> -    p2m_unlock(host_p2m);
> -
> -    return rc;
> -}
> -
> -int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
> -                        unsigned int altp2m_idx)
> -{
> -    struct p2m_domain *host_p2m = p2m_get_hostp2m(d);
> -    struct p2m_domain *ap2m = NULL;
> -    struct p2m_domain *p2m;
> -    mfn_t mfn;
> -    p2m_access_t a;
> -    p2m_type_t t;
> -    int rc = 0;
> -
> -    if ( altp2m_idx > 0 )
> -    {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> -             mfn_x(INVALID_MFN) )
> -            return -EINVAL;
> -
> -        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> -    }
> -    else
> -        p2m = host_p2m;
> -
> -    gfn_lock(host_p2m, gfn, 0);
> -
> -    if ( ap2m )
> -        p2m_lock(ap2m);
> -
> -    mfn = p2m->get_entry(p2m, gfn, &t, &a, 0, NULL, suppress_ve);
> -    if ( !mfn_valid(mfn) )
> -        rc = -ESRCH;
> -
> -    if ( ap2m )
> -        p2m_unlock(ap2m);
> -
> -    gfn_unlock(host_p2m, gfn, 0);
> -
> -    return rc;
> -}
> -
> -int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> -                                   uint8_t visible)
> -{
> -    int rc = 0;
> -
> -    altp2m_list_lock(d);
> -
> -    /*
> -     * Eptp index is correlated with altp2m index and should not exceed
> -     * min(MAX_ALTP2M, MAX_EPTP).
> -     */
> -    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> -         mfn_x(INVALID_MFN) )
> -        rc = -EINVAL;
> -    else if ( visible )
> -        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> -            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)];
> -    else
> -        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> -            mfn_x(INVALID_MFN);
> -
> -    altp2m_list_unlock(d);
> -
> -    return rc;
> -}
> -
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/x86/mm/p2m.h b/xen/arch/x86/mm/p2m.h
> index 04308cfb6d..635f5a7f45 100644
> --- a/xen/arch/x86/mm/p2m.h
> +++ b/xen/arch/x86/mm/p2m.h
> @@ -22,6 +22,9 @@ static inline void p2m_free_logdirty(struct p2m_domain *p2m) {}
>  int p2m_init_altp2m(struct domain *d);
>  void p2m_teardown_altp2m(struct domain *d);
>  
> +void p2m_flush_table_locked(struct p2m_domain *p2m);
> +int __must_check p2m_remove_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
> +                                  unsigned int page_order);
>  void p2m_nestedp2m_init(struct p2m_domain *p2m);
>  int p2m_init_nestedp2m(struct domain *d);
>  void p2m_teardown_nestedp2m(struct domain *d);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:38:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722718.1126961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P8Q-0005qi-DK; Thu, 16 May 2024 00:38:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722718.1126961; Thu, 16 May 2024 00:38:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P8Q-0005qb-Ap; Thu, 16 May 2024 00:38:14 +0000
Received: by outflank-mailman (input) for mailman id 722718;
 Thu, 16 May 2024 00:38:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P8O-0005qV-Tx
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:38:12 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9154ebae-131c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:38:11 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0F9BDCE1777;
 Thu, 16 May 2024 00:38:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CCAC0C116B1;
 Thu, 16 May 2024 00:38:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9154ebae-131c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819887;
	bh=4g34A0o9035cOxa64wo1nxAwIGq6I4tY15eiVmR9ZFk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OBPm5VI7Mo00yMGrPN3Uryq2ssiCVZkvThihTJ/Es7c+B/aErEok2MkPTt18kqRWK
	 4EwIGpMS5ONVx0KWhWhjHGU1M3hQxE/bNxI/P3xD71dq68mOsPgcewHwbJ0eKjHdr/
	 5eIusoKK/8GB6cpB3gWRI58cuBotr4i/DeKRD76Ly4iWJYqV6kJ0pSdmzh9YFWzUXv
	 gTeWYb1TcCXvINFXQgdSmO1pbgv/3FyzZMyKUnjoQAjWxJ+iNL6naJwpYVeX4eocfy
	 Zai1ZLruAK6szPM6T0ZbNebEjJC7IT1lZ3Mmk3OFlrli651PZ8/NCWVO5+j4bw1oiR
	 jpQdLsEBTpSJQ==
Date: Wed, 15 May 2024 17:38:04 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Tamas K Lengyel <tamas@tklengyel.com>
Subject: Re: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with CONFIG_ALTP2M
 option
In-Reply-To: <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151737150.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Instead of using generic CONFIG_HVM option switch to a bit more specific
> CONFIG_ALTP2M option for altp2m support. Also guard altp2m routines, so that
> they can be disabled completely in the build -- when target platform does not
> actually support altp2m (AMD-V & ARM as of now).
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Tamas K Lengyel <tamas@tklengyel.com>
> ---
> changes in v2:
>  - use separate CONFIG_ALTP2M option instead of CONFIG_VMX
> ---
>  xen/arch/x86/include/asm/altp2m.h  |  5 ++++-
>  xen/arch/x86/include/asm/hvm/hvm.h |  2 +-
>  xen/arch/x86/include/asm/p2m.h     | 17 ++++++++++++++++-
>  xen/arch/x86/mm/Makefile           |  2 +-
>  4 files changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/x86/include/asm/altp2m.h b/xen/arch/x86/include/asm/altp2m.h
> index e5e59cbd68..092b13e231 100644
> --- a/xen/arch/x86/include/asm/altp2m.h
> +++ b/xen/arch/x86/include/asm/altp2m.h
> @@ -7,7 +7,7 @@
>  #ifndef __ASM_X86_ALTP2M_H
>  #define __ASM_X86_ALTP2M_H
>  
> -#ifdef CONFIG_HVM
> +#ifdef CONFIG_ALTP2M
>  
>  #include <xen/types.h>
>  #include <xen/sched.h>         /* for struct vcpu, struct domain */
> @@ -38,7 +38,10 @@ static inline bool altp2m_active(const struct domain *d)
>  }
>  
>  /* Only declaration is needed. DCE will optimise it out when linking. */
> +void altp2m_vcpu_initialise(struct vcpu *v);
> +void altp2m_vcpu_destroy(struct vcpu *v);
>  uint16_t altp2m_vcpu_idx(const struct vcpu *v);
> +int altp2m_vcpu_enable_ve(struct vcpu *v, gfn_t gfn);
>  void altp2m_vcpu_disable_ve(struct vcpu *v);
>  
>  #endif
> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
> index 0c9e6f1564..4f03dd7af8 100644
> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -670,7 +670,7 @@ static inline bool hvm_hap_supported(void)
>  /* returns true if hardware supports alternate p2m's */
>  static inline bool hvm_altp2m_supported(void)
>  {
> -    return hvm_funcs.caps.altp2m;
> +    return IS_ENABLED(CONFIG_ALTP2M) && hvm_funcs.caps.altp2m;
>  }
>  
>  /* Returns true if we have the minimum hardware requirements for nested virt */
> diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
> index 111badf89a..855e69d24a 100644
> --- a/xen/arch/x86/include/asm/p2m.h
> +++ b/xen/arch/x86/include/asm/p2m.h
> @@ -581,9 +581,9 @@ static inline gfn_t mfn_to_gfn(const struct domain *d, mfn_t mfn)
>          return _gfn(mfn_x(mfn));
>  }
>  
> -#ifdef CONFIG_HVM
>  #define AP2MGET_prepopulate true
>  #define AP2MGET_query false
> +#ifdef CONFIG_ALTP2M

Is it necessary? Can't we just replace CONFIG_HVM with CONFIG_ALTP2M on
the same line



From xen-devel-bounces@lists.xenproject.org Thu May 16 00:38:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:38:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722720.1126972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P8n-0006GE-Lf; Thu, 16 May 2024 00:38:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722720.1126972; Thu, 16 May 2024 00:38:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7P8n-0006G7-Ic; Thu, 16 May 2024 00:38:37 +0000
Received: by outflank-mailman (input) for mailman id 722720;
 Thu, 16 May 2024 00:38:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7P8m-0005qV-Mg
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:38:36 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0ed0ad9-131c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:38:36 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C716A615F7;
 Thu, 16 May 2024 00:38:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7A100C116B1;
 Thu, 16 May 2024 00:38:33 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0ed0ad9-131c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715819914;
	bh=UHDD/UovjdkBTkxgFuBNpZd8jvAC/eLri3nFPg2FeUQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WB3aDR13KNoeXE840XddpR9vOZXDcJre3yiOHwGt2gGAHN9Va1LsSnEkCApSa2wrO
	 06OACOJYmeHT0TsDcc4/6iDsQUMOaTd4iwVeMJhl1O7VnjwU/BHS9rlSYTE8TacB7n
	 8z8ssDzt66jR08ytVRrGAhQ6Oh82RdihzRBBydZrfoeUrG46omxc5+mgOSRwZQhvLH
	 9YkbyM0nVAq5LZpXfXe5kMUNpI2iyhYnyX06fDgT5xK3lrsDxG4IIpvaH07ujJQSdi
	 CYuyFjPWd+2H5CWmruD0omV2z7uf5kaqfytKI+DBagtKbTR/KqjfYxnxnGEqFb547+
	 2Hl3iZ6v5z9Ug==
Date: Wed, 15 May 2024 17:38:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
In-Reply-To: <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151738130.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> As we now have SVM/VMX config options for enabling/disabling these features
> completely in the build, it may be feasible to add build-time checks to
> cpu_has_{svm,vmx} macros. These are used extensively thoughout HVM code, so
> we won't have to add extra #ifdef-s to check whether svm/vmx has been enabled,
> while DCE cleans up calls to vmx/svm functions, if their code not being built.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 16 00:44:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:44:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722728.1126982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PEi-0000X9-9s; Thu, 16 May 2024 00:44:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722728.1126982; Thu, 16 May 2024 00:44:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PEi-0000X2-71; Thu, 16 May 2024 00:44:44 +0000
Received: by outflank-mailman (input) for mailman id 722728;
 Thu, 16 May 2024 00:44:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PEh-0000Ww-Lw
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:44:43 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b4aabdd-131d-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:44:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id E22CF60EDE;
 Thu, 16 May 2024 00:44:40 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7BE99C116B1;
 Thu, 16 May 2024 00:44:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b4aabdd-131d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820280;
	bh=R9a8H5UBpfaQ+OyUCtoxTLUElqPo418XLs/v9CCV27I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pUiDEPh5qCcFjCcTELaoBO9J84XhqIO3Webhs34G+4zIVpuMpv7KbtKiTTHJApwiW
	 EyPiyEaLsYDt0JIdyVuylDHtn0uNFq6Eng5oSNb3/WkKZiSeW+VAGH4hRFUSAP3nfD
	 6lExl0sjEpfuB1fUTrW6cowTqbgAUpGxgm8KMZsl+yJvo1/zH9PjRWig9i47mMO2XX
	 CLC9VFbISVxhTDq99xjj5wqqcnPJHyyeE+yaFKr3KX4SkIE3ci2f+YL+ND/NPDedsD
	 hbs1xD8txXzD+/3Uj8IOKZQ/FCMqrQUmWBUwlaLeIPxUrgE29yafqVef/KCSl1b0aK
	 19d4fgcd+Pj8w==
Date: Wed, 15 May 2024 17:44:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 08/15] x86/vpmu: guard vmx/svm calls with
 cpu_has_{vmx,svm}
In-Reply-To: <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151740180.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> If VMX/SVM disabled in the build, we may still want to have vPMU drivers for
> PV guests. Yet some calls to vmx/svm-related routines needs to be guarded then.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Question to the x86 maintainers: are we sure we want to support the case
where VMX/SVM is disabled in the build but still we want to run PV
guests with vPMU?

If the question is not, could we simplify this simply by making vpmu_amd
dependent on CONFIG_SVM and vpmu_intel dependent on CONFIG_VMX?

I realize that it is possible and technically correct to disable
CONFIG_SVM (or VMX) to run on AMD hardware (or Intel) with plain PV
guests only. But do we want to support it? I wonder if we could make
things easier by avoiding to support this configuration until somebody
asks for it.


> ---
>  xen/arch/x86/cpu/vpmu_amd.c   |  8 ++++----
>  xen/arch/x86/cpu/vpmu_intel.c | 20 ++++++++++----------
>  2 files changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
> index db2fa420e1..40b0c8932f 100644
> --- a/xen/arch/x86/cpu/vpmu_amd.c
> +++ b/xen/arch/x86/cpu/vpmu_amd.c
> @@ -290,7 +290,7 @@ static int cf_check amd_vpmu_save(struct vcpu *v,  bool to_guest)
>      context_save(v);
>  
>      if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
> -         is_msr_bitmap_on(vpmu) )
> +         is_msr_bitmap_on(vpmu) && cpu_has_svm )
>          amd_vpmu_unset_msr_bitmap(v);
>  
>      if ( to_guest )
> @@ -363,7 +363,7 @@ static int cf_check amd_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
>              return 0;
>          vpmu_set(vpmu, VPMU_RUNNING);
>  
> -        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
> +        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
>               amd_vpmu_set_msr_bitmap(v);
>      }
>  
> @@ -372,7 +372,7 @@ static int cf_check amd_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
>          (is_pmu_enabled(msr_content) == 0) && vpmu_is_set(vpmu, VPMU_RUNNING) )
>      {
>          vpmu_reset(vpmu, VPMU_RUNNING);
> -        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
> +        if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
>               amd_vpmu_unset_msr_bitmap(v);
>          release_pmu_ownership(PMU_OWNER_HVM);
>      }
> @@ -415,7 +415,7 @@ static void cf_check amd_vpmu_destroy(struct vcpu *v)
>  {
>      struct vpmu_struct *vpmu = vcpu_vpmu(v);
>  
> -    if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) )
> +    if ( is_hvm_vcpu(v) && is_msr_bitmap_on(vpmu) && cpu_has_svm )
>          amd_vpmu_unset_msr_bitmap(v);
>  
>      xfree(vpmu->context);
> diff --git a/xen/arch/x86/cpu/vpmu_intel.c b/xen/arch/x86/cpu/vpmu_intel.c
> index cd414165df..10c34a5691 100644
> --- a/xen/arch/x86/cpu/vpmu_intel.c
> +++ b/xen/arch/x86/cpu/vpmu_intel.c
> @@ -269,7 +269,7 @@ static inline void __core2_vpmu_save(struct vcpu *v)
>      if ( !is_hvm_vcpu(v) )
>          rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, core2_vpmu_cxt->global_status);
>      /* Save MSR to private context to make it fork-friendly */
> -    else if ( mem_sharing_enabled(v->domain) )
> +    else if ( mem_sharing_enabled(v->domain) && cpu_has_vmx )
>          vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
>                             &core2_vpmu_cxt->global_ctrl);
>  }
> @@ -288,7 +288,7 @@ static int cf_check core2_vpmu_save(struct vcpu *v, bool to_guest)
>  
>      /* Unset PMU MSR bitmap to trap lazy load. */
>      if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
> -         cpu_has_vmx_msr_bitmap )
> +         cpu_has_vmx && cpu_has_vmx_msr_bitmap )
>          core2_vpmu_unset_msr_bitmap(v);
>  
>      if ( to_guest )
> @@ -333,7 +333,7 @@ static inline void __core2_vpmu_load(struct vcpu *v)
>          wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, core2_vpmu_cxt->global_ctrl);
>      }
>      /* Restore MSR from context when used with a fork */
> -    else if ( mem_sharing_is_fork(v->domain) )
> +    else if ( mem_sharing_is_fork(v->domain) && cpu_has_vmx )
>          vmx_write_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
>                              core2_vpmu_cxt->global_ctrl);
>  }
> @@ -442,7 +442,7 @@ static int cf_check core2_vpmu_alloc_resource(struct vcpu *v)
>      if ( !acquire_pmu_ownership(PMU_OWNER_HVM) )
>          return 0;
>  
> -    if ( is_hvm_vcpu(v) )
> +    if ( is_hvm_vcpu(v) && cpu_has_vmx )
>      {
>          if ( vmx_add_host_load_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, 0) )
>              goto out_err;
> @@ -513,7 +513,7 @@ static int core2_vpmu_msr_common_check(u32 msr_index, int *type, int *index)
>          __core2_vpmu_load(current);
>          vpmu_set(vpmu, VPMU_CONTEXT_LOADED);
>  
> -        if ( is_hvm_vcpu(current) && cpu_has_vmx_msr_bitmap )
> +        if ( is_hvm_vcpu(current) && cpu_has_vmx && cpu_has_vmx_msr_bitmap )
>              core2_vpmu_set_msr_bitmap(current);
>      }
>      return 1;
> @@ -584,7 +584,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
>          if ( msr_content & fixed_ctrl_mask )
>              return -EINVAL;
>  
> -        if ( is_hvm_vcpu(v) )
> +        if ( is_hvm_vcpu(v) && cpu_has_vmx )
>              vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
>                                 &core2_vpmu_cxt->global_ctrl);
>          else
> @@ -653,7 +653,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
>              if ( blocked )
>                  return -EINVAL;
>  
> -            if ( is_hvm_vcpu(v) )
> +            if ( is_hvm_vcpu(v) && cpu_has_vmx)
>                  vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
>                                     &core2_vpmu_cxt->global_ctrl);
>              else
> @@ -672,7 +672,7 @@ static int cf_check core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
>          wrmsrl(msr, msr_content);
>      else
>      {
> -        if ( is_hvm_vcpu(v) )
> +        if ( is_hvm_vcpu(v) && cpu_has_vmx )
>              vmx_write_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, msr_content);
>          else
>              wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, msr_content);
> @@ -706,7 +706,7 @@ static int cf_check core2_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
>              *msr_content = core2_vpmu_cxt->global_status;
>              break;
>          case MSR_CORE_PERF_GLOBAL_CTRL:
> -            if ( is_hvm_vcpu(v) )
> +            if ( is_hvm_vcpu(v) && cpu_has_vmx )
>                  vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL, msr_content);
>              else
>                  rdmsrl(MSR_CORE_PERF_GLOBAL_CTRL, *msr_content);
> @@ -808,7 +808,7 @@ static void cf_check core2_vpmu_destroy(struct vcpu *v)
>      vpmu->context = NULL;
>      xfree(vpmu->priv_context);
>      vpmu->priv_context = NULL;
> -    if ( is_hvm_vcpu(v) && cpu_has_vmx_msr_bitmap )
> +    if ( is_hvm_vcpu(v) && cpu_has_vmx && cpu_has_vmx_msr_bitmap )
>          core2_vpmu_unset_msr_bitmap(v);
>      release_pmu_ownership(PMU_OWNER_HVM);
>      vpmu_clear(vpmu);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:45:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:45:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722731.1126991 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PFo-00012a-It; Thu, 16 May 2024 00:45:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722731.1126991; Thu, 16 May 2024 00:45:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PFo-00012T-GG; Thu, 16 May 2024 00:45:52 +0000
Received: by outflank-mailman (input) for mailman id 722731;
 Thu, 16 May 2024 00:45:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PFm-00012J-Sz
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:45:50 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a38cfcbe-131d-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:45:49 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B304061616;
 Thu, 16 May 2024 00:45:48 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0F5DC116B1;
 Thu, 16 May 2024 00:45:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a38cfcbe-131d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820348;
	bh=897RYxPsxTHuVRECkO/nYLXZsj4jFRfOyYR5LD47M9k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ew50qXNyYZ/dLI5g3rNOXjNXUUY1nIH4lj7k4okyqpVR4VjjUAaA+Ia/t4BBbNhGI
	 AcOzBFyqhcTv0MKhjC7/xWwPC+1b7rDGQbPzfIK0Im9TSNHHUpWKP0r7zGvx+tozqy
	 24WN/ZiIUffM03GMPydQVpeM5CjKhb4bfaEi8JQSGHN/T+bs2YLK5x5F96Dr+aWxLz
	 ISloXLiPW99a96ZWxOT5sl37Kii6tvF4niOknC2DpithuzfVU9o1A6ju0puQzjU+9P
	 UVtZVefZBZBzTZbJM9UHiUXiGE1L1qzCHuxZb7sl18m5rvQUTk3HL4nGHz64iM7jCK
	 8pa+MeDDw02Hw==
Date: Wed, 15 May 2024 17:45:45 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 09/15] x86/traps: clean up superfluous #idef-s
In-Reply-To: <7f0b98062ce67ad8176670efbe3c3ebdb43d2b1c.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151745360.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <7f0b98062ce67ad8176670efbe3c3ebdb43d2b1c.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Remove preprocessor checks for CONFIG_HVM option, because expressions covered
> by these checks are already guarded by cpu_has_vmx, which itself depends
> on CONFIG_HVM option (via CONFIG_VMX).
> 
> No functional change intended.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Nice!

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 16 00:46:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:46:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722734.1127001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PGI-0001XT-RS; Thu, 16 May 2024 00:46:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722734.1127001; Thu, 16 May 2024 00:46:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PGI-0001XM-Oj; Thu, 16 May 2024 00:46:22 +0000
Received: by outflank-mailman (input) for mailman id 722734;
 Thu, 16 May 2024 00:46:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PGH-0001X0-Id
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:46:21 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b4514dca-131d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 02:46:18 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id E76BF615F7;
 Thu, 16 May 2024 00:46:16 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 885C6C116B1;
 Thu, 16 May 2024 00:46:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4514dca-131d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820376;
	bh=Ywx0ivV5tmJ3US9Cz8KoOFAzjmhBYnAUBKluw6FBhBM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DHSj11xJgObeQhXinNetbsOQZBKwkO13t6a0hbc3PzjeYGjF9aVSAuPIlT/p2qsaD
	 eUsFopVLFiEeUkBSxKZEEGydT7er1dZsQkv/sY19NGKLcrqZpLPUqcRicSxEoMwJHP
	 YdaWGU68nhFLgGYuDTkJfRRpBtoF2BGjRfYHGIkCGFUZMTpFrpRNMQwcwW94aiFdHI
	 XE0KA+u3ukcaAWJm8fXBhtjN+R5PE9RaYxinioXc3tQD5MCl+C7yXPlnzCG7Q0kjwm
	 vtbVtQJt9hhtueaX+LxusfFGclsgq4vX4hMKj5ATIKcYzlG60c9DraNjxGX0OwSFk5
	 vVEdpwRpkamSw==
Date: Wed, 15 May 2024 17:46:13 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 10/15] x86/domain: clean up superfluous #idef-s
In-Reply-To: <67d6604e8f66468c02f0c2e60315fc9251b69beb.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151746070.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <67d6604e8f66468c02f0c2e60315fc9251b69beb.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> Remove preprocessor checks for CONFIG_HVM option, because expressions covered
> by these checks are already guarded by cpu_has_svm, which itself depends
> on CONFIG_HVM option (via CONFIG_SVM).
> 
> No functional change intended.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 16 00:51:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:51:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722739.1127012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PKg-0003Hk-Bw; Thu, 16 May 2024 00:50:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722739.1127012; Thu, 16 May 2024 00:50:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PKg-0003Hd-9F; Thu, 16 May 2024 00:50:54 +0000
Received: by outflank-mailman (input) for mailman id 722739;
 Thu, 16 May 2024 00:50:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PKf-0003HX-67
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:50:53 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 561dac4a-131e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:50:51 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 47117CE1777;
 Thu, 16 May 2024 00:50:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 382ABC116B1;
 Thu, 16 May 2024 00:50:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 561dac4a-131e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820643;
	bh=H5GDzVsLacn2FXdHs5959QxH720DDh9p9jEmKXmQ7mo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gLPRlz2oGP60zqtSTKdKpBjEmntrLnGHu7YXcGk86uvrqsAisdmsm5J3zERVd/nOt
	 hoRz0OxqHyeqrtHLG+LSLv/E9yzlMQ/P9HlhBDErrdE2cisrcq6K/3kTULjvZRIIGj
	 rSurlujMuYRiPZc17aKG8R7ZYQrabSXh6s6HsiGqdXsYFo3D74vAfHC4uCo2nPiWSc
	 TofYlp5ISa/RcdqRDwnHo2TnJ7SmQrCxUkh0nbUT+towIGwfJ+2hUVnZDjJMNKgGhX
	 BCFA48DCq4vpydawGCxSCjhbXziYuCY5yavl+LTbZ+wOm5yBd+BOZ80xQy7hGnz8hH
	 COg4dX6emQf0Q==
Date: Wed, 15 May 2024 17:50:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Paul Durrant <paul@xen.org>, Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH v2 12/15] x86/vmx: guard access to cpu_has_vmx_* in
 common code
In-Reply-To: <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151748080.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> There're several places in common code, outside of arch/x86/hvm/vmx,
> where cpu_has_vmx_* get accessed without checking if VMX present first.
> We may want to guard these macros, as they read global variables defined
> inside vmx-specific files -- so VMX can be made optional later on.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Jan Beulich <jbeulich@suse.com>
> ---
> Here I've tried a different approach from prev.patches [1,2] -- instead of
> modifying whole set of cpu_has_{svm/vmx}_* macros, we can:
>  1) do not touch SVM part at all, because just as Andrew pointed out they're
> used inside arch/x86/hvm/svm only.
>  2) track several places in common code where cpu_has_vmx_* features are
> checked out and guard them using cpu_has_vmx condition
>  3) two of cpu_has_vmx_* macros being used in common code are checked in a bit
> more tricky way, so instead of making complex conditionals even more complicated,
> we can instead integrate cpu_has_vmx condition inside these two macros.
> 
> This patch aims to replace [1,2] from v1 series by doing steps above.
> 
>  1. https://lore.kernel.org/xen-devel/20240416064402.3469959-1-Sergiy_Kibrik@epam.com/
>  2. https://lore.kernel.org/xen-devel/20240416064606.3470052-1-Sergiy_Kibrik@epam.com/

I am missing some of the previous discussions but why can't we just fix
all of the cpu_has_vmx_* #defines in vmcs.h to also check for
cpu_has_vmx?

That seems easier and simpler than to add add-hoc checks at the invocations?


> ---
> changes in v2:
>  - do not touch SVM code and macros
>  - drop vmx_ctrl_has_feature()
>  - guard cpu_has_vmx_* macros in common code instead
> changes in v1:
>  - introduced helper routine vmx_ctrl_has_feature() and used it for all
>    cpu_has_vmx_* macros
> ---
>  xen/arch/x86/hvm/hvm.c                  | 2 +-
>  xen/arch/x86/hvm/viridian/viridian.c    | 4 ++--
>  xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 4 ++--
>  xen/arch/x86/traps.c                    | 5 +++--
>  4 files changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 9594e0a5c5..ab75de9779 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -5180,7 +5180,7 @@ int hvm_debug_op(struct vcpu *v, int32_t op)
>      {
>          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
>          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
> -            if ( !cpu_has_monitor_trap_flag )
> +            if ( !cpu_has_vmx || !cpu_has_monitor_trap_flag )
>                  return -EOPNOTSUPP;
>              break;
>          default:
> diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
> index 0496c52ed5..657c6a3ea7 100644
> --- a/xen/arch/x86/hvm/viridian/viridian.c
> +++ b/xen/arch/x86/hvm/viridian/viridian.c
> @@ -196,7 +196,7 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
>          res->a = CPUID4A_RELAX_TIMER_INT;
>          if ( viridian_feature_mask(d) & HVMPV_hcall_remote_tlb_flush )
>              res->a |= CPUID4A_HCALL_REMOTE_TLB_FLUSH;
> -        if ( !cpu_has_vmx_apic_reg_virt )
> +        if ( !cpu_has_vmx || !cpu_has_vmx_apic_reg_virt )
>              res->a |= CPUID4A_MSR_BASED_APIC;
>          if ( viridian_feature_mask(d) & HVMPV_hcall_ipi )
>              res->a |= CPUID4A_SYNTHETIC_CLUSTER_IPI;
> @@ -236,7 +236,7 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
>  
>      case 6:
>          /* Detected and in use hardware features. */
> -        if ( cpu_has_vmx_virtualize_apic_accesses )
> +        if ( cpu_has_vmx && cpu_has_vmx_virtualize_apic_accesses )
>              res->a |= CPUID6A_APIC_OVERLAY;
>          if ( cpu_has_vmx_msr_bitmap || (read_efer() & EFER_SVME) )
>              res->a |= CPUID6A_MSR_BITMAPS;
> diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
> index 58140af691..aa05f9cf6e 100644
> --- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
> +++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
> @@ -306,7 +306,7 @@ extern u64 vmx_ept_vpid_cap;
>  #define cpu_has_vmx_vnmi \
>      (vmx_pin_based_exec_control & PIN_BASED_VIRTUAL_NMIS)
>  #define cpu_has_vmx_msr_bitmap \
> -    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
> +    (cpu_has_vmx && vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
>  #define cpu_has_vmx_secondary_exec_control \
>      (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
>  #define cpu_has_vmx_tertiary_exec_control \
> @@ -347,7 +347,7 @@ extern u64 vmx_ept_vpid_cap;
>  #define cpu_has_vmx_vmfunc \
>      (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS)
>  #define cpu_has_vmx_virt_exceptions \
> -    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
> +    (cpu_has_vmx && vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
>  #define cpu_has_vmx_pml \
>      (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
>  #define cpu_has_vmx_mpx \
> diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
> index 7b8ee45edf..3595bb379a 100644
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -1130,7 +1130,7 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf,
>          if ( !is_hvm_domain(d) || subleaf != 0 )
>              break;
>  
> -        if ( cpu_has_vmx_apic_reg_virt )
> +        if ( cpu_has_vmx && cpu_has_vmx_apic_reg_virt )
>              res->a |= XEN_HVM_CPUID_APIC_ACCESS_VIRT;
>  
>          /*
> @@ -1139,7 +1139,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf,
>           * and wrmsr in the guest will run without VMEXITs (see
>           * vmx_vlapic_msr_changed()).
>           */
> -        if ( cpu_has_vmx_virtualize_x2apic_mode &&
> +        if ( cpu_has_vmx &&
> +             cpu_has_vmx_virtualize_x2apic_mode &&
>               cpu_has_vmx_apic_reg_virt &&
>               cpu_has_vmx_virtual_intr_delivery )
>              res->a |= XEN_HVM_CPUID_X2APIC_VIRT;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:51:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722743.1127021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PLO-0003v3-N4; Thu, 16 May 2024 00:51:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722743.1127021; Thu, 16 May 2024 00:51:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PLO-0003uw-KM; Thu, 16 May 2024 00:51:38 +0000
Received: by outflank-mailman (input) for mailman id 722743;
 Thu, 16 May 2024 00:51:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PLO-0003g4-1l
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:51:38 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 71f45764-131e-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 02:51:36 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C7DA9615F7;
 Thu, 16 May 2024 00:51:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B1CEC116B1;
 Thu, 16 May 2024 00:51:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71f45764-131e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820694;
	bh=ZYfQX7cCFHwsa34Np2fSwh5Cvagy23Ynk+cwVLEA6Xo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iKQTMCVPkvCYZ4a0n+LBjQbuyipNZ2cIbuiVV+qjD3+RIAwdRIM0+2G/0mhT8xoYg
	 jQnoBSfUwMbA9xDWBoRBi59zmgeNdA9DsMFvJQzy6nx+IXIZf3wfhIIBlhz+sBnbq+
	 QuLFsxxdsFcKAEhkR9Qlhg4VFgj+xXbEn1ALoQhT25Ib6dVnd+JtwPEhHS7mOMHpnG
	 keuqRfPA60sxPN6JerkTw6/p2N6rhQkSkdZP0cQFr2kAYhiofs6jQuKVErgHZV+Q6g
	 Wnw3vYq15KatMJ6SMP8/sghGFuMp4L6qXsrf+jmnxFdZ4Tt5QHkFV+cubdzbkMELZb
	 YIh1/TSY/8G6g==
Date: Wed, 15 May 2024 17:51:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Xenia Ragiadakou <burzalodowa@gmail.com>, 
    Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 13/15] x86/ioreq: guard VIO_realmode_completion
 with CONFIG_VMX
In-Reply-To: <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151751230.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> From: Xenia Ragiadakou <burzalodowa@gmail.com>
> 
> VIO_realmode_completion is specific to vmx realmode, so guard the completion
> handling code with CONFIG_VMX. Also, guard VIO_realmode_completion itself by
> CONFIG_VMX, instead of generic CONFIG_X86.
> 
> No functional change intended.
> 
> Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 16 00:54:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:54:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722745.1127032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PO3-0004ve-2n; Thu, 16 May 2024 00:54:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722745.1127032; Thu, 16 May 2024 00:54:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PO2-0004vX-WD; Thu, 16 May 2024 00:54:23 +0000
Received: by outflank-mailman (input) for mailman id 722745;
 Thu, 16 May 2024 00:54:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7PO2-0004vR-3n
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:54:22 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d26cb61b-131e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:54:19 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 73601CE1802;
 Thu, 16 May 2024 00:54:16 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7AFCC32782;
 Thu, 16 May 2024 00:54:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d26cb61b-131e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820855;
	bh=MuE4HQ2FPbsqGcRxJFuoymWVDev/utL3Oe71h6ns99k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GKOIpo1FX331UVAEdSRsOxDFZY/oMHeYlfNgqFcSLKDxCaIW00LeJYcOqd+bnueTU
	 vYtdQum8FsVgcuf6HfEK5lf3jPs1Nd7G9CweVUtDAg5CTxMPlBim96ykubD5JYGK4C
	 UQqmPvSr1GK1NxLXoEo0UPwZsv5JWbRt4GnQhoh1AVQ/I4u/n+Qfj3GDjclxejrT1W
	 eqMV5GqChiPn+qn7FI9Adl8HyqlvAWba+5DI9zF6emaeZBNIIaM9bHYY5Tzfp5Rp7I
	 VWY2nLDGpL+bXeOSkTz7TV+depz1SA8I3ZMWZUu2uPqJSCIpJZj4YNNnBoUaVAWkvj
	 CQwpfRBstSN3g==
Date: Wed, 15 May 2024 17:54:13 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 14/15] iommu/vt-d: guard vmx_pi_hooks_* calls with
 cpu_has_vmx
In-Reply-To: <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151751530.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> VMX posted interrupts support can now be excluded from x86 build along with
> VMX code itself, but still we may want to keep the possibility to use
> VT-d IOMMU driver in non-HVM setups.
> So we guard vmx_pi_hooks_{assign/deassign} with some checks for such a case.
> 
> No functional change intended here.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

I know that Andrew was keep on having a separate Kconfig option for
VT-D, separate from VMX. But still, couldn't we make the VT-D Kconfig
option depending on CONFIG_VMX?

To me, VT-D should require VMX, without VMX it should not be possible to
enable VT-D.

This comment goes in the same direction of my previous comment regarding
the vpmu: we are trying to make things more configurable and flexible
and that's good, but we don't necessary need to make all possible
combination work. VT-D without VMX is another one of those combination
that I would only enable after a customer asks.


> ---
>  xen/drivers/passthrough/vtd/iommu.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
> index e13be244c1..ad78282250 100644
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2772,7 +2772,7 @@ static int cf_check reassign_device_ownership(
>  
>      if ( !QUARANTINE_SKIP(target, pdev->arch.vtd.pgd_maddr) )
>      {
> -        if ( !has_arch_pdevs(target) )
> +        if ( cpu_has_vmx && !has_arch_pdevs(target) )
>              vmx_pi_hooks_assign(target);
>  
>  #ifdef CONFIG_PV
> @@ -2806,7 +2806,7 @@ static int cf_check reassign_device_ownership(
>      }
>      if ( ret )
>      {
> -        if ( !has_arch_pdevs(target) )
> +        if ( cpu_has_vmx && !has_arch_pdevs(target) )
>              vmx_pi_hooks_deassign(target);
>          return ret;
>      }
> @@ -2824,7 +2824,7 @@ static int cf_check reassign_device_ownership(
>          write_unlock(&target->pci_lock);
>      }
>  
> -    if ( !has_arch_pdevs(source) )
> +    if ( cpu_has_vmx && !has_arch_pdevs(source) )
>          vmx_pi_hooks_deassign(source);
>  
>      /*
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 00:54:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 00:54:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722748.1127042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7POc-0005UZ-9J; Thu, 16 May 2024 00:54:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722748.1127042; Thu, 16 May 2024 00:54:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7POc-0005US-6r; Thu, 16 May 2024 00:54:58 +0000
Received: by outflank-mailman (input) for mailman id 722748;
 Thu, 16 May 2024 00:54:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7POa-0004vR-Q3
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 00:54:56 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e92e20f5-131e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 02:54:56 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 166806163D;
 Thu, 16 May 2024 00:54:55 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6AF2DC32786;
 Thu, 16 May 2024 00:54:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e92e20f5-131e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715820894;
	bh=w+bUDSePBm3kOiKXdGcHoqpjZkpwd5sLkggqM0vWmbs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VJAqJVXb7Cre7G4723V8LbB4C6iNfBcCCNMRvdKVF0nQ7K9tdl8PeRqNjpCVrXz6f
	 C4GbCCWJKa7CVLMxABG0npL51lYLhICySpUdHVJ8deMsIQEy2+P654QXFAJMpMJ8Av
	 pS0EdzT9pRTFIQxDQGS5ShWfpprKAEt8FObY5ri4mu4wP8tqCpp5M/1R0X1pLuOD6S
	 FLms5q+U+JtLk8yafxEBv+q40titS8OlhyMsaJ5MwyL/FGJzaYQusg1f0/myrCD4jt
	 liCmO7GuC4M/UtZj8tgfcz1+PTHIREjgU43/b0z1yakD+/lnfEw+xjvTtyvbWwzCjM
	 VHBoQpnPgIejA==
Date: Wed, 15 May 2024 17:54:51 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Xenia Ragiadakou <burzalodowa@gmail.com>, 
    Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 15/15] x86/hvm: make AMD-V and Intel VT-x support
 configurable
In-Reply-To: <3ad7c0279da67e564713140fb5b247349cf4dccc.1715761386.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405151754420.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <3ad7c0279da67e564713140fb5b247349cf4dccc.1715761386.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Sergiy Kibrik wrote:
> From: Xenia Ragiadakou <burzalodowa@gmail.com>
> 
> Provide the user with configuration control over the cpu virtualization support
> in Xen by making SVM and VMX options user selectable.
> 
> To preserve the current default behavior, both options depend on HVM and
> default to value of HVM.
> 
> To prevent users from unknowingly disabling virtualization support, make the
> controls user selectable only if EXPERT is enabled.
> 
> No functional change intended.
> 
> Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Thu May 16 01:04:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 01:04:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722757.1127052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PYB-0007p0-5L; Thu, 16 May 2024 01:04:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722757.1127052; Thu, 16 May 2024 01:04:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7PYB-0007os-2H; Thu, 16 May 2024 01:04:51 +0000
Received: by outflank-mailman (input) for mailman id 722757;
 Thu, 16 May 2024 01:04:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7PY9-0007oi-Ij; Thu, 16 May 2024 01:04:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7PY9-00069A-Fd; Thu, 16 May 2024 01:04:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7PY9-0004Kf-5S; Thu, 16 May 2024 01:04:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7PY9-0006lb-4i; Thu, 16 May 2024 01:04:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zjTyPG944ZiiiJSWpJ9kqoV8eeB8yOZv+XS1aoTDprA=; b=FQ2XVxJ4P6QI59z4Y3SroikzTu
	+rmh8NcCWGnkXzF7fVS4x7ZEToT94D7iZB8ydWn1NQRYfAow5xLS2lEQbEHRVtthPFQVD/duJjD1t
	cxEhiVYZ4MqB5PtD0pNZf3CdhSqmd5Mx62pvPpGOolpAP0MCCSLYonNCnoSKZrhkVBNA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186005-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186005: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:host-ping-check-xen:fail:allowable
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=1b294a1f35616977caddaddf3e9d28e576a1adbc
X-Osstest-Versions-That:
    linux=4f8b6f25eb1e51febd426da764a0b0ea652ad238
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 01:04:49 +0000

flight 186005 linux-linus real [real]
flight 186008 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186005/
http://logs.test-lab.xenproject.org/osstest/logs/186008/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 186008-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     10 host-ping-check-xen      fail REGR. vs. 186002

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186008 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186008 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186002
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186002
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186002
 test-armhf-armhf-libvirt-vhd  8 xen-boot                     fail  like 186002
 test-armhf-armhf-examine      8 reboot                       fail  like 186002
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186002
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186002
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186002
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                1b294a1f35616977caddaddf3e9d28e576a1adbc
baseline version:
 linux                4f8b6f25eb1e51febd426da764a0b0ea652ad238

Last test of basis   186002  2024-05-15 02:16:07 Z    0 days
Testing same since   186005  2024-05-15 13:12:03 Z    0 days    1 attempts

------------------------------------------------------------
452 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4f8b6f25eb1e..1b294a1f3561  1b294a1f35616977caddaddf3e9d28e576a1adbc -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu May 16 01:25:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 01:25:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722765.1127061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ps5-0004W0-SB; Thu, 16 May 2024 01:25:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722765.1127061; Thu, 16 May 2024 01:25:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ps5-0004Vt-Pg; Thu, 16 May 2024 01:25:25 +0000
Received: by outflank-mailman (input) for mailman id 722765;
 Thu, 16 May 2024 01:25:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Ps4-0004Vk-9X
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 01:25:24 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 27157e62-1323-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 03:25:22 +0200 (CEST)
Received: from MN2PR05CA0018.namprd05.prod.outlook.com (2603:10b6:208:c0::31)
 by CH2PR12MB4262.namprd12.prod.outlook.com (2603:10b6:610:af::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May
 2024 01:25:14 +0000
Received: from BL02EPF0001A107.namprd05.prod.outlook.com
 (2603:10b6:208:c0:cafe::8d) by MN2PR05CA0018.outlook.office365.com
 (2603:10b6:208:c0::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 01:25:14 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF0001A107.mail.protection.outlook.com (10.167.241.136) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 01:25:14 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 15 May
 2024 20:25:13 -0500
Received: from [172.31.69.246] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 15 May 2024 20:25:12 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27157e62-1323-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZUwY4z/BNf0i9CeyyIUmfNIDs3jVw9yM+kHYkC922OmOqhbS4M42QEfR2+DrMlUyQyrPFjH6JZnCVPljMfREnjcc1WAmMsSiie6pcaSpfpZbsi9nv5Fshvh2fiJRcvOocOs187ANM/A5aHLRYyq4MdkddJ4X2PoOBS0f64cjmi3KX8GIcyXa61rJ+I/fNE4ZxPQtZ4GT83GbAKluuBZoIlwa4e7dGChQioPZ01FF0GPl5XUb8jBstrjR8Bq7gZh5ovJlBipmoKLiXvc7Yoe8fpc3b6CRLecG3jMPSF3sRU9FGjQWgZGViR0vSF+J4pnu6RHFWD3JMTrbJKyaGiJYYA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hl6KpEUCprwIyfEVtjKZLElxzKs+PEABLDLKQP7oidM=;
 b=b0M5ud0A+QzfbwQEt73RKvjp0fnjHX/a8bFpGqV2/F1cFnhCYthqWHb171FWvk4GIiQxvtJTwvFawjnA/PxHeVdPY05jXonGLGXM5JF0hiy0a2tW0dO/XEF0dbP0ytYnNDQDL6fZrbv/PtuPGkWvEmtf6SWyXf0FOMsYegEaD8qA6td+jfqaiNNHJj9Ovr0X4cAkHCL/mabVujYOfFw821qs5oYB87KQdU76vlx+zlrTTZte9k/OVArNKXOaq0MfgbGxBwbKh6nxrmq7IAJ+nSTb3WOZq/eVW1dLVYrzQqtf7VThvRS9d0AdTl6cWrHQXWVI7Pyrh0OwuicbTkJIEA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hl6KpEUCprwIyfEVtjKZLElxzKs+PEABLDLKQP7oidM=;
 b=PRaAIAzjraUJq6ggN0SY5GxizpbvGWSr0Gg6Nu3Qxi7PvRYS89J7iUHOVMQCO8PKZqvCHWnXkWZjY3S5kInnb/tLiKFUhcB4mOLN2zxtVZGL08ki4Zb9IghLHGNvY2U2BjgCGfKKUjMWho0D2b6s4MXILRtg29xf6vYuwYkl1zU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <028f29be-0393-4a57-83e2-ea27fe0320d5@amd.com>
Date: Thu, 16 May 2024 09:25:11 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] drivers/xen: Improve the late XenStore init protocol
To: Stefano Stabellini <sstabellini@kernel.org>
CC: <xen-devel@lists.xenproject.org>, <linux-kernel@vger.kernel.org>, "Juergen
 Gross" <jgross@suse.com>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Michal Orzel <michal.orzel@amd.com>
References: <20240515014330.1044617-1-xin.wang2@amd.com>
 <alpine.DEB.2.22.394.2405151524270.2544314@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2405151524270.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A107:EE_|CH2PR12MB4262:EE_
X-MS-Office365-Filtering-Correlation-Id: 7b50aafa-ee8b-4675-1c77-08dc7547099e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?T1MzdnNSTSszR1hVMHdmTWxnUEdiM2w4K1FTN3FpMGF5MTRibjNFSW0zQVhn?=
 =?utf-8?B?djJrcjJpMnltQ2NrRjEwblc5TUkrS0ZPSHBpYzM4ajltT2tqcjdlM1NtZGxh?=
 =?utf-8?B?cGZsS2JyQlgyc3l4R0sySGJodGk5RU0rMCt0OTBIV1VqVHB3REh3bnAybU00?=
 =?utf-8?B?djRzODliM0JwZVVtVmN1NWY4OFU2dlNaSWZoSnNnQnF4eHFWRHpSOHNoUHls?=
 =?utf-8?B?UDFVbE5xekV1Qk5xVk1VZHc5YXd6NnYxTHB5TnMrcmVIWTNBbUYrT215OUxR?=
 =?utf-8?B?OUhWWHkvOGJmY3BJcGxNeTlpZ2J6ZXkvU1hxWXZGUFh0Y1RaVGNtQW9aOVVU?=
 =?utf-8?B?SHVhQklFQ0p5K1lOWVpEZlpXN01OdEpDZEl0UjQ0djFLb0FQOVRHdjJlRXA5?=
 =?utf-8?B?clZXbWhDS3lLQ1R5QlNYT0IwZlVLMnJqWVI1TEZwRGpzY3QvMHlPTXJ1dGVD?=
 =?utf-8?B?bjFFZlhQcXJIci9MRi93czk4WlhkZTU4SGI2NjBjUHduQ1ByQi8yVTJucGVU?=
 =?utf-8?B?L0hvaUJnb1MvdHFJYVlCekpYb01HMXB1UnBzNXg2SkE4d3lXWVpYYXBQakxx?=
 =?utf-8?B?UVlGb2VHNlFGYXhhcHBFMWVNNEo0U0p4ZTRuRDVPYVJ1dzlETmt4RGJZdndt?=
 =?utf-8?B?clRTSDFUblRPdDQ1UGgzOGZzNUczM25zU09nNER0VmpsUmh6RG1GWWJieVZP?=
 =?utf-8?B?Sk1lSzI3RzRnbzVCOVdSVFM1MHZKc0xRU2E5WTdLaFRaSlVSVnB2dEdZUWVJ?=
 =?utf-8?B?bUZON0lORmVZU2Q5TkFXTWhocW9hcmFFZU1CcEVLMjlNcHRmaHFpclpoNEk2?=
 =?utf-8?B?V1IwT0k0WjdUb3luZ0luVWQ4eHdOWXRWdGlubnZOdm9md0ZaKzBWNE5PeHcw?=
 =?utf-8?B?SkR1S1V2Ti9zQXdjVFpUV3ZnSTZoUmFZZDBERlZxVzNUUzJ4VG5MT3pCWFBX?=
 =?utf-8?B?V0NHcVhwb1FHbmdReVFJVW1LRFkvcVNvSmxwL1dkYWdtVmhaQkUxSVVpSmo1?=
 =?utf-8?B?TDlMYmxUelh0eVI5VnptendDdlVwYlVieThWdWx3YWtUNlFCNjVOZnM5MDl4?=
 =?utf-8?B?L1VsWUdIV1FiN3RWdlJNQXBST2phYjhkcTZ2RkJWZjhVbktTNXN6aHJRb0FH?=
 =?utf-8?B?clNsSHZTdm84eUt0bStITThJSzlvT29KK2tOQU1UNHdNb2RIL280UVltd29F?=
 =?utf-8?B?dTI0cnFZYzlvQ0NxbFVGaWVEUURDUG9kVmNBbm9FY0RuenlDUWdTVXl4ZTFp?=
 =?utf-8?B?UGMrdDMrYmNwbWUxYXRMTTJOZU1CTWpmOGRSeHFpL1lSVWZNYUhvQU40c1BX?=
 =?utf-8?B?Mi84TDg2d3pFZkNwU0NvUmxoUDBIVklpaG91L1dUQ3A3Qm1XU29qcTd6dmJN?=
 =?utf-8?B?TlBLUDV2WC95NFkreW1DaGVFam5oOEY1Z2wxRm9CSWlpbWxwY0hwUXJteU4x?=
 =?utf-8?B?Sk9ERi9JWGM3Rk9aQ3dSeFdyTzUwczdoSnE3Y3RGbytqa1RSQkhmcFUyOUJC?=
 =?utf-8?B?MG1EZXRyZVpVSUUvbkZZOGxkMkFLYmlqd2NhODk3aTZmd1hEU2hvSFlhOVFJ?=
 =?utf-8?B?QkljSnZJTWY3L0JUVmhkZnhxcXBIUXNsY0p1cVl0THhuckJ4RkJyRlBoMmVr?=
 =?utf-8?B?Y3ZYZ09OajBZaVR0Z2E4eTVVT0JlVUdnTnpuTkh1VlBSTkI2bzB5ODhVN2oy?=
 =?utf-8?B?UXNIeDlLd2l2V21HUHZUczZQaWI3Q25iemVTekMxMUM1YkYzRUl6Tno5em5E?=
 =?utf-8?B?M2xoVzRLUEI5Z0hsWGQxVk51Y04vSGEvWHl2cXpRM0cvaDM3RlN6TXFVZTUv?=
 =?utf-8?Q?9B0xr7ZMo3/SblJDfDUNHJEPKZfDFAm++Hx2w=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 01:25:14.3783
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b50aafa-ee8b-4675-1c77-08dc7547099e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A107.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4262

Hi Stefano,

On 5/16/2024 6:30 AM, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Henry Wang wrote:
>> Currently, the late XenStore init protocol is only triggered properly
>> for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
>> case that XenStore interface is allocated but not ready (the connection
>> status is not XENSTORE_CONNECTED), Linux should also wait until the
>> XenStore is set up properly.
>>
>> Introduce a macro to describe the XenStore interface is ready, use
>> it in xenbus_probe_initcall() and xenbus_probe() to select the code
>> path of doing the late XenStore init protocol or not.
>>
>> Take the opportunity to enhance the check of the allocated XenStore
>> interface can be properly mapped, and return error early if the
>> memremap() fails.
>>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> Please add a Fixes: tag

Sure. Will do.

>> ---
>>   drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++++++++++-----
>>   1 file changed, 16 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
>> index 3205e5d724c8..8aec0ed1d047 100644
>> --- a/drivers/xen/xenbus/xenbus_probe.c
>> +++ b/drivers/xen/xenbus/xenbus_probe.c
>> @@ -72,6 +72,10 @@ EXPORT_SYMBOL_GPL(xen_store_evtchn);
>>   struct xenstore_domain_interface *xen_store_interface;
>>   EXPORT_SYMBOL_GPL(xen_store_interface);
>>   
>> +#define XS_INTERFACE_READY \
>> +	((xen_store_interface != NULL) && \
>> +	 (xen_store_interface->connection == XENSTORE_CONNECTED))
>> +
>>   enum xenstore_init xen_store_domain_type;
>>   EXPORT_SYMBOL_GPL(xen_store_domain_type);
>>   
>> @@ -751,9 +755,10 @@ static void xenbus_probe(void)
>>   {
>>   	xenstored_ready = 1;
>>   
>> -	if (!xen_store_interface) {
>> -		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>> -					       XEN_PAGE_SIZE, MEMREMAP_WB);
>> +	if (!xen_store_interface || XS_INTERFACE_READY) {
>> +		if (!xen_store_interface)
> These two nested if's don't make sense to me. If XS_INTERFACE_READY
> succeeds, it means that  ((xen_store_interface != NULL) &&
> (xen_store_interface->connection == XENSTORE_CONNECTED)).
>
> So it is not possible that xen_store_interface == NULL immediately
> after. Right?

I think this is because we want to free the irq for the late init case, 
otherwise the init-dom0less will fail. For the xenstore PFN allocated 
case, the connection is already set to CONNECTED when we execute 
init-dom0less. But I agree with you, would below diff makes more sense 
to you?

diff --git a/drivers/xen/xenbus/xenbus_probe.c 
b/drivers/xen/xenbus/xenbus_probe.c
index 8aec0ed1d047..b8005b651a29 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -76,6 +76,8 @@ EXPORT_SYMBOL_GPL(xen_store_interface);
         ((xen_store_interface != NULL) && \
          (xen_store_interface->connection == XENSTORE_CONNECTED))

+static bool xs_late_init = false;
+
  enum xenstore_init xen_store_domain_type;
  EXPORT_SYMBOL_GPL(xen_store_domain_type);

@@ -755,7 +757,7 @@ static void xenbus_probe(void)
  {
         xenstored_ready = 1;

-       if (!xen_store_interface || XS_INTERFACE_READY) {
+       if (xs_late_init) {
                 if (!xen_store_interface)
                         xen_store_interface = memremap(xen_store_gfn << 
XEN_PAGE_SHIFT,
XEN_PAGE_SIZE, MEMREMAP_WB);
@@ -937,6 +939,8 @@ static irqreturn_t xenbus_late_init(int irq, void 
*unused)
         int err;
         uint64_t v = 0;

+       xs_late_init = true;
+
         err = hvm_get_parameter(HVM_PARAM_STORE_PFN, &v);
         if (err || !v || !~v)
                 return IRQ_HANDLED;

>> +			xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>> +						       XEN_PAGE_SIZE, MEMREMAP_WB);
>>   		/*
>>   		 * Now it is safe to free the IRQ used for xenstore late
>>   		 * initialization. No need to unbind: it is about to be
>> @@ -822,7 +827,7 @@ static int __init xenbus_probe_initcall(void)
>>   	if (xen_store_domain_type == XS_PV ||
>>   	    (xen_store_domain_type == XS_HVM &&
>>   	     !xs_hvm_defer_init_for_callback() &&
>> -	     xen_store_interface != NULL))
>> +	     XS_INTERFACE_READY))
>>   		xenbus_probe();
>>   
>>   	/*
>> @@ -831,7 +836,7 @@ static int __init xenbus_probe_initcall(void)
>>   	 * started, then probe.  It will be triggered when communication
>>   	 * starts happening, by waiting on xb_waitq.
>>   	 */
>> -	if (xen_store_domain_type == XS_LOCAL || xen_store_interface == NULL) {
>> +	if (xen_store_domain_type == XS_LOCAL || !XS_INTERFACE_READY) {
>>   		struct task_struct *probe_task;
>>   
>>   		probe_task = kthread_run(xenbus_probe_thread, NULL,
>> @@ -1014,6 +1019,12 @@ static int __init xenbus_init(void)
>>   			xen_store_interface =
>>   				memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>>   					 XEN_PAGE_SIZE, MEMREMAP_WB);
>> +			if (!xen_store_interface) {
>> +				pr_err("%s: cannot map HVM_PARAM_STORE_PFN=%llx\n",
>> +				       __func__, v);
>> +				err = -ENOMEM;
> I think this should -EINVAL

Will change.

Kind regards,
Henry

>> +				goto out_error;
>> +			}
>>   			if (xen_store_interface->connection != XENSTORE_CONNECTED)
>>   				wait = true;
>>   		}
>> -- 
>> 2.34.1
>>



From xen-devel-bounces@lists.xenproject.org Thu May 16 02:12:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 02:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722772.1127072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7QbD-0006ny-6t; Thu, 16 May 2024 02:12:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722772.1127072; Thu, 16 May 2024 02:12:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7QbD-0006nr-3s; Thu, 16 May 2024 02:12:03 +0000
Received: by outflank-mailman (input) for mailman id 722772;
 Thu, 16 May 2024 02:12:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t9d+=MT=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s7QbB-0006nk-Aj
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 02:12:01 +0000
Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com
 [2607:f8b0:4864:20::831])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac9c5aa8-1329-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 04:11:58 +0200 (CEST)
Received: by mail-qt1-x831.google.com with SMTP id
 d75a77b69052e-43e1d15a46eso23178271cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 19:11:58 -0700 (PDT)
Received: from shine.lan
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df57110a9sm90701811cf.69.2024.05.15.19.11.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 19:11:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac9c5aa8-1329-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715825517; x=1716430317; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=H0WIxuR6UQgg2tu21CpoSpp7DsryaH1MZJlm+1o+riU=;
        b=lxzqKJwwsCfPDMCXp2ezy59JeLGHNmNKWigvFL/cyvGHbN90USEKlyfk6h/5aqvVRW
         oy7P7L8gXiojGuSJBLXLwx+w7NDYX0Q5SnB9TWqvj7lYDOaUmkq/JAHKkEVdehJrBNM5
         Oxtn2mjWsBlVbP4U+vkC4QKFf2f44lEekEnsevffsMD6UcOjrWQ4pONmM1Q0RBHtCvsd
         DbNqTajdIYtNXGxCMjxEaeOaRz0p6ib11WMROP5t83Niv5hwPPitilCQUOWMpryU1+cJ
         g9soBI5jXy8VVsi+s7p8x3s2eLan+Zsxno5Jz7xmyQfBXoiNJMl3IDmXHXGZe+Vu2qI2
         xC7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715825517; x=1716430317;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=H0WIxuR6UQgg2tu21CpoSpp7DsryaH1MZJlm+1o+riU=;
        b=GPw9/eP6zn68MRzk9qmRwgY2tWbE+U0+n9lDgOI//VXESwT7huZQizif5lOpiUadxE
         NBUsnKEk444Abclm0hmaHrkyFmBETOanDF8DWJUdkpOSHUeH5WWq07hPy1iO3NI5YVWn
         nNSWEStSyOlinjPvxBTL1Z7+2levqHJ6Wq1d48mg1lyZTmIOSIqHcangZ8SIYfchpsBD
         Tfrf/hTwtCXYmXOjqY64XztujBp0BlGDwd2lSJBbu1vnEOeuTwW3Tr4BJ6FcV460wzxj
         lhrHJn5gpuFw9mU/LmTiljJ7wLyBsVZVqOI79dmK+YrC8udv1uSoPs+dam6OwxQgPKaa
         hmHg==
X-Gm-Message-State: AOJu0Yx5vFp3jKudk+0RuwhIv4tNmIcFlKVunsAAcekac133WiJ3VIJ+
	zJZUm08awL8OZnj7JRB26xeh3/YsDhgqBTv8pCe9nTQya0B3jT+PU+VfWQ==
X-Google-Smtp-Source: AGHT+IFBg8FlMwn9BilWSFWMmQLzOhU7hCJg/ZbIoQYiwnJQaQHG6Qz2Ho4hq2iXGcyt4kvM/eH5dw==
X-Received: by 2002:a05:622a:1ba8:b0:43e:3b2f:e2 with SMTP id d75a77b69052e-43e3b2f0860mr18344851cf.37.1715825516591;
        Wed, 15 May 2024 19:11:56 -0700 (PDT)
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v3] libxl: Enable stubdom cdrom changing
Date: Wed, 15 May 2024 22:10:10 -0400
Message-Id: <20240516021010.3783-1-jandryuk@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Jason Andryuk <jason.andryuk@amd.com>

From: Jason Andryuk <jandryuk@gmail.com>

To change the cd-rom medium, libxl will:
 - QMP eject the medium from QEMU
 - block-detach the old PV disk
 - block-attach the new PV disk
 - QMP change the medium to the new PV disk by fdset-id

The QMP code is reused, and remove and attach are implemented here.

The stubdom must internally handle adding /dev/xvdc to the appropriate
fdset.  libxl in dom0 doesn't see the result of adding to the fdset as
that is internal to the stubdom, but the fdset's opaque fields will be
set to stub-devid:$devid, so libxl can identify it.  $devid is common
between the stubdom and libxl, so it can be identified on both side.
The stubdom will name the device xvdY regardless of the guest name hdY,
sdY, or xvdY, but the stubdom will be assigned the same devid
facilitating lookup.  Because the stubdom add-fd call is asynchronous,
libxl needs to poll query-fdsets to identify when add-fd has completed.

For cd-eject, we still need to attach the empty vbd.  This is necessary
since xenstore is used to determine that hdc exists.  Otherwise after
eject, hdc would be gone and the cd-insert would fail to find the drive
to insert new media.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
QEMU inside the stubdom can be sandboxed disallowing opening the devices
by path name, so QMP add-fd is used like with dom0 QEMU depriv.

v1 allowed ERROR_JSON_CONFIG_EMPTY "because a stubdom don't have a json
config," but it is re-introduced in v2 since the guest should be acted
on.  It seems fine in testing.  The missing JSON was probably from
OpenXT having disabled JSON configs in the past.

v2
Use query-fdsets for removal and addition - stub-devid:$devid
Re-use aodev instead of 2nd aodev_del
Rename some functions
Support sdX vdevs
Get stubdomid from cis->disk_domid in stubdom callbacks
Use if (rc) not if (rc != 0)
Remove comment about libxl_device_disk_remove
Use EGC_GC not STATE_AO_GC
Re-work and eliminate cdrom_insert_stubdom_query_fdset_retry
Change some messages
Allow missing removal fdset in case it wasn't added during startup.
Drop LOGD(... rc=%d)

v3:
Remove LOG DEBUGs
s/timeout_retry/retry_timer/
Init stubdom_fdset = -1 and use as flag to pase fdset-id
Drop AO_GC
Avoid non-error goto
Expect ERROR_TIMEOUT as non-error
Drop some comments
close payload_fd and set to -1
Remove limit on query-fdset retries

By removing the limit on query-fdset retries, we lose an explicity error
message for the stubdom not performing the add-fd, but we give more time
which may let more cases succeed.
---
 docs/misc/stubdom.txt         |  10 +
 tools/libs/light/libxl_disk.c | 384 ++++++++++++++++++++++++++++++++--
 2 files changed, 373 insertions(+), 21 deletions(-)

diff --git a/docs/misc/stubdom.txt b/docs/misc/stubdom.txt
index c717a95d17..64c220db20 100644
--- a/docs/misc/stubdom.txt
+++ b/docs/misc/stubdom.txt
@@ -127,6 +127,16 @@ Limitations:
  - at most 26 emulated disks are supported (more are still available as PV disks)
  - graphics output (VNC/SDL/Spice) not supported
 
+CD-ROM changing:
+
+To change the CD-ROM medium, libxl will:
+ - QMP eject the medium from QEMU
+ - block-detach the old PV disk
+ - block-attach the new PV disk
+ - QMP change the medium to the new PV disk by fdset-id
+
+The stubdom must internally add /dev/xvdc to an fdset in QEMU with opaque set
+to "stub-devid:$devid".  libxl will lookup the fdset with that string.
 
                                    PV-GRUB
                                    =======
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index 1f1e5c6567..d76f39ea42 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -829,21 +829,118 @@ int libxl_device_disk_getinfo(libxl_ctx *ctx, uint32_t domid,
     return rc;
 }
 
+/*
+ * Search through the query-fdsets JSON looking for a matching devid.
+ *
+ * If found, return the fdset-id integer (>=0).
+ *
+ * If not found, return ERROR_NOTFOUND.
+ *
+ * On error, return libxl ERROR_*.
+ */
+static int query_fdsets_find_fdset(libxl__gc *gc,
+                                   const libxl__json_object *response,
+                                   int devid)
+{
+    const libxl__json_object *fdset;
+    const char *needle = GCSPRINTF("stub-devid:%d", devid);
+    int i, j, rc;
+
+    /* query-fdsets returns:
+     * [
+     *   { "fds": [
+     *       { "fd": 30,
+     *         "opaque": "stub-devid:2080"
+     *       }
+     *     ],
+     *     "fdset-id": 1
+     *   }
+     * ]
+     */
+    for (i = 0; (fdset = libxl__json_array_get(response, i)); i++) {
+        const libxl__json_object *fdset_id;
+        const libxl__json_object *fds;
+        const libxl__json_object *fd;
+
+        fdset_id = libxl__json_map_get("fdset-id", fdset, JSON_INTEGER);
+        if (!fdset_id) {
+            rc = ERROR_QEMU_API;
+            goto out;
+        }
+
+        fds = libxl__json_map_get("fds", fdset, JSON_ARRAY);
+        if (!fds) {
+            rc = ERROR_QEMU_API;
+            goto out;
+        }
+        for (j = 0; (fd = libxl__json_array_get(fds, j)); j++) {
+            const libxl__json_object *fd_num;
+            const libxl__json_object *opaque;
+            const char *opaque_str;
+
+            fd_num = libxl__json_map_get("fd", fd, JSON_INTEGER);
+            if (!fd_num) {
+                rc = ERROR_QEMU_API;
+                goto out;
+            }
+            opaque = libxl__json_map_get("opaque", fd, JSON_STRING);
+            if (!opaque) {
+                continue;
+            }
+
+            opaque_str = libxl__json_object_get_string(opaque);
+            if (strcmp(opaque_str, needle) == 0) {
+                return libxl__json_object_get_integer(fdset_id);
+            }
+        }
+    }
+    rc = ERROR_NOTFOUND;
+
+ out:
+    return rc;
+}
+
 typedef struct {
     libxl__ao *ao;
+    libxl__ao_device aodev;
     libxl_domid domid;
+    libxl_domid disk_domid;
     libxl_device_disk *disk;
     libxl_device_disk disk_saved;
     libxl__ev_slowlock qmp_lock;
     int dm_ver;
     libxl__ev_time time;
+    libxl__ev_time retry_timer;
     libxl__ev_qmp qmp;
+    int stubdom_fdset;
 } libxl__cdrom_insert_state;
 
 static void cdrom_insert_lock_acquired(libxl__egc *, libxl__ev_slowlock *,
                                        int rc);
 static void cdrom_insert_qmp_connected(libxl__egc *, libxl__ev_qmp *,
                                        const libxl__json_object *, int rc);
+static void cdrom_insert_stubdom_query_fdset_rm(libxl__egc *egc,
+                                                libxl__ev_qmp *qmp,
+                                                const libxl__json_object *resp,
+                                                int rc);
+static void cdrom_insert_stubdom_parse_fdset_rm(libxl__egc *egc,
+                                                libxl__ev_qmp *qmp,
+                                                const libxl__json_object *resp,
+                                                int rc);
+static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *,
+                                         const libxl__json_object *, int rc);
+static void cdrom_insert_stubdom_disk_remove_cb(libxl__egc *egc,
+                                                 libxl__ao_device *aodev);
+static void cdrom_insert_stubdom_disk_add_cb(libxl__egc *egc,
+                                             libxl__ao_device *aodev);
+static void cdrom_insert_stubdom_query_fdset(libxl__egc *egc,
+                                             libxl__ev_time *ev,
+                                             const struct timeval *abs,
+                                             int rc);
+static void cdrom_insert_stubdom_parse_fdset(libxl__egc *egc,
+                                             libxl__ev_qmp *qmp,
+                                             const libxl__json_object *response,
+                                             int rc);
 static void cdrom_insert_ejected(libxl__egc *egc, libxl__ev_qmp *,
                                  const libxl__json_object *, int rc);
 static void cdrom_insert_addfd_cb(libxl__egc *egc, libxl__ev_qmp *,
@@ -865,6 +962,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
     libxl_device_disk *disks = NULL;
     int rc;
     libxl__cdrom_insert_state *cis;
+    libxl_domid stubdomid;
 
     GCNEW(cis);
     cis->ao = ao;
@@ -876,6 +974,8 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
     cis->qmp_lock.ao = ao;
     cis->qmp_lock.domid = domid;
     libxl__ev_time_init(&cis->time);
+    libxl__ev_time_init(&cis->retry_timer);
+    cis->stubdom_fdset = -1;
     libxl__ev_qmp_init(&cis->qmp);
     cis->qmp.ao = ao;
     cis->qmp.domid = domid;
@@ -892,12 +992,6 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    if (libxl_get_stubdom_id(ctx, domid) != 0) {
-        LOGD(ERROR, domid, "cdrom-insert doesn't work for stub domains");
-        rc = ERROR_INVAL;
-        goto out;
-    }
-
     cis->dm_ver = libxl__device_model_version_running(gc, domid);
     if (cis->dm_ver == -1) {
         LOGD(ERROR, domid, "Cannot determine device model version");
@@ -905,7 +999,22 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    disks = libxl__device_list(gc, &libxl__disk_devtype, domid, &num);
+    stubdomid = libxl_get_stubdom_id(CTX, cis->domid);
+    if (stubdomid == 0) {
+        cis->disk_domid = domid;
+    } else {
+        cis->disk_domid = stubdomid;
+        disk->backend = LIBXL_DISK_BACKEND_PHY;
+    }
+
+    if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL &&
+        stubdomid) {
+        LOGD(ERROR, domid, "cdrom-insert doesn't work for Mini-OS stubdoms");
+        rc = ERROR_INVAL;
+        goto out;
+    }
+
+    disks = libxl__device_list(gc, &libxl__disk_devtype, cis->disk_domid, &num);
     for (i = 0; i < num; i++) {
         if (disks[i].is_cdrom && !strcmp(disk->vdev, disks[i].vdev))
         {
@@ -920,7 +1029,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    rc = libxl__device_disk_setdefault(gc, domid, disk, false);
+    rc = libxl__device_disk_setdefault(gc, cis->disk_domid, disk, false);
     if (rc) goto out;
 
     if (!disk->pdev_path) {
@@ -994,7 +1103,12 @@ static void cdrom_insert_qmp_connected(libxl__egc *egc, libxl__ev_qmp *qmp,
         QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid);
     else
         QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid);
-    qmp->callback = cdrom_insert_ejected;
+
+    if (libxl_get_stubdom_id(CTX, cis->domid))
+        qmp->callback = cdrom_insert_stubdom_query_fdset_rm;
+    else
+        qmp->callback = cdrom_insert_ejected;
+
     rc = libxl__ev_qmp_send(egc, qmp, "eject", args);
     if (rc) goto out;
     return;
@@ -1002,6 +1116,211 @@ out:
     cdrom_insert_done(egc, cis, rc); /* must be last */
 }
 
+static void cdrom_insert_stubdom_query_fdset_rm(libxl__egc *egc,
+                                                libxl__ev_qmp *qmp,
+                                                const libxl__json_object *resp,
+                                                int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+
+    if (rc) goto out;
+
+    /* Only called for qemu-xen/linux stubdom. */
+    assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN);
+
+    cis->qmp.callback = cdrom_insert_stubdom_parse_fdset_rm;
+
+    rc = libxl__ev_qmp_send(egc, &cis->qmp, "query-fdsets", NULL);
+    if (rc) goto out;
+
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_parse_fdset_rm(libxl__egc *egc,
+                                                libxl__ev_qmp *qmp,
+                                                const libxl__json_object *resp,
+                                                int rc)
+{
+    EGC_GC;
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    int devid;
+    int fdset;
+
+    if (rc) goto out;
+
+    /* Only called for qemu-xen/linux stubdom. */
+    assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN);
+
+    devid = libxl__device_disk_dev_number(cis->disk->vdev, NULL, NULL);
+    fdset = query_fdsets_find_fdset(gc, resp, devid);
+    if (fdset < 0) {
+        rc = fdset;
+        goto out;
+    }
+
+    LOGD(DEBUG, cis->domid, "Found fdset %d", fdset);
+
+    libxl__json_object *args = NULL;
+
+    libxl__qmp_param_add_integer(gc, &args, "fdset-id", fdset);
+
+    cis->qmp.callback = cdrom_insert_stubdom_ejected;
+
+    rc = libxl__ev_qmp_send(egc, &cis->qmp, "remove-fd", args);
+    if (rc) goto out;
+
+    return;
+
+ out:
+    if (rc == ERROR_NOTFOUND) {
+        LOGD(DEBUG, cis->domid, "No fdset found - skipping remove-fd");
+        cdrom_insert_stubdom_ejected(egc, qmp, resp, 0);
+    } else {
+        cdrom_insert_done(egc, cis, rc); /* must be last */
+    }
+}
+
+
+static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *qmp,
+                                         const libxl__json_object *response,
+                                         int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    libxl__device *device;
+    STATE_AO_GC(cis->ao);
+    domid_t stubdomid = cis->disk_domid;
+
+    if (rc) goto out;
+
+    GCNEW(device);
+    rc = libxl__device_from_disk(gc, stubdomid, cis->disk, device);
+    if (rc) goto out;
+
+    /* stubdom PV block dev eject */
+    libxl__prepare_ao_device(ao, &cis->aodev);
+    cis->aodev.action = LIBXL__DEVICE_ACTION_REMOVE;
+    cis->aodev.dev = device;
+    cis->aodev.callback = cdrom_insert_stubdom_disk_remove_cb;
+    cis->aodev.force.flag = LIBXL__FORCE_OFF;
+    libxl__initiate_device_generic_remove(egc, &cis->aodev);
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_disk_remove_cb(libxl__egc *egc,
+                                                 libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev);
+    domid_t stubdomid = cis->disk_domid;
+
+    if (aodev->rc) {
+        LOGD(ERROR, aodev->dev->domid, "Unable to remove stubdom PV disk id %u",
+             aodev->dev->devid);
+        goto out;
+    }
+
+    /* block dev insert - this may be inserting an empty disk for eject. */
+    libxl__prepare_ao_device(ao, &cis->aodev);
+    /* set an ao callback to end up in cdrom_insert_ejected */
+    cis->aodev.callback = cdrom_insert_stubdom_disk_add_cb;
+    libxl__device_disk_add(egc, stubdomid, cis->disk, &cis->aodev);
+
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, aodev->rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_disk_add_cb(libxl__egc *egc,
+                                             libxl__ao_device *aodev)
+{
+    EGC_GC;
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev);
+
+    if (aodev->rc) {
+        LOGD(ERROR, aodev->dev->domid, "Unable to insert stubdom PV disk id %u",
+             aodev->dev->devid);
+        goto out;
+    }
+
+    cdrom_insert_stubdom_query_fdset(egc, &cis->retry_timer, NULL,
+                                     ERROR_TIMEDOUT);
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, aodev->rc);
+}
+
+static void cdrom_insert_stubdom_query_fdset(libxl__egc *egc,
+                                             libxl__ev_time *ev,
+                                             const struct timeval *abs,
+                                             int rc)
+{
+    EGC_GC;
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(ev, *cis, retry_timer);
+
+    if (rc != ERROR_TIMEDOUT) goto out;
+
+    /* Only called for qemu-xen/linux stubdom. */
+    assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN);
+
+    cis->qmp.callback = cdrom_insert_stubdom_parse_fdset;
+
+    rc = libxl__ev_qmp_send(egc, &cis->qmp, "query-fdsets", NULL);
+    if (rc) goto out;
+
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_parse_fdset(libxl__egc *egc,
+                                             libxl__ev_qmp *qmp,
+                                             const libxl__json_object *response,
+                                             int rc)
+{
+    EGC_GC;
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    int devid;
+    int fdset;
+
+    if (rc) goto out;
+
+    /* Only called for qemu-xen/linux stubdom. */
+    assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN);
+
+    devid = libxl__device_disk_dev_number(cis->disk->vdev, NULL, NULL);
+    fdset = query_fdsets_find_fdset(gc, response, devid);
+    if (fdset < 0) {
+        rc = fdset;
+        goto out;
+    }
+
+    cis->stubdom_fdset = fdset;
+
+    LOGD(DEBUG, cis->domid, "Found fdset %d", cis->stubdom_fdset);
+    cdrom_insert_ejected(egc, &cis->qmp, NULL, rc);
+    return;
+
+ out:
+    if (rc == ERROR_NOTFOUND) {
+        rc = libxl__ev_time_register_rel(cis->ao, &cis->retry_timer,
+                                         cdrom_insert_stubdom_query_fdset,
+                                         200);
+        if (rc) goto out;
+        return;
+    }
+
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
 static void cdrom_insert_ejected(libxl__egc *egc,
                                  libxl__ev_qmp *qmp,
                                  const libxl__json_object *response,
@@ -1026,7 +1345,7 @@ static void cdrom_insert_ejected(libxl__egc *egc,
 
     if (rc) goto out;
 
-    rc = libxl__device_from_disk(gc, domid, disk, &device);
+    rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device);
     if (rc) goto out;
     be_path = libxl__device_backend_path(gc, &device);
     libxl_path = libxl__device_libxl_path(gc, &device);
@@ -1082,6 +1401,7 @@ static void cdrom_insert_ejected(libxl__egc *egc,
     if (rc) goto out;
 
     if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN &&
+        libxl_get_stubdom_id(CTX, cis->domid) == 0 &&
         disk->format != LIBXL_DISK_FORMAT_EMPTY) {
         libxl__json_object *args = NULL;
 
@@ -1094,7 +1414,7 @@ static void cdrom_insert_ejected(libxl__egc *egc,
             goto out;
         }
 
-        /* This free form parameter is not use by QEMU or libxl. */
+        /* This free form parameter is not used by QEMU or non-stubdom libxl. */
         QMP_PARAMETERS_SPRINTF(&args, "opaque", "%s:%s",
                                libxl_disk_format_to_string(disk->format),
                                disk->pdev_path);
@@ -1116,7 +1436,15 @@ out:
         cdrom_insert_done(egc, cis, rc); /* must be last */
     } else if (!has_callback) {
         /* Only called if no asynchronous callback are set. */
-        cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */
+        if (libxl_get_stubdom_id(CTX, cis->domid) &&
+            disk->format != LIBXL_DISK_FORMAT_EMPTY) {
+            LOGD(DEBUG, cis->domid,
+                 "stubdom %d needs to perform add-fd internally",
+                 libxl_get_stubdom_id(CTX, cis->domid));
+            cdrom_insert_addfd_cb(egc, qmp, NULL, 0); /* must be last */
+        } else  {
+            cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */
+        }
     }
 }
 
@@ -1135,17 +1463,25 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc,
     /* convenience aliases */
     libxl_device_disk *disk = cis->disk;
 
-    close(qmp->payload_fd);
+    if (qmp->payload_fd >= 0)
+    {
+        close(qmp->payload_fd);
+    }
     qmp->payload_fd = -1;
 
     if (rc) goto out;
 
-    o = libxl__json_map_get("fdset-id", response, JSON_INTEGER);
-    if (!o) {
-        rc = ERROR_FAIL;
-        goto out;
+    /* response non-NULL only for non-stubdom */
+    if (cis->stubdom_fdset == -1) {
+        o = libxl__json_map_get("fdset-id", response, JSON_INTEGER);
+        if (!o) {
+            rc = ERROR_FAIL;
+            goto out;
+        }
+        fdset = libxl__json_object_get_integer(o);
+    } else {
+        fdset = cis->stubdom_fdset;
     }
-    fdset = libxl__json_object_get_integer(o);
 
     devid = libxl__device_disk_dev_number(disk->vdev, NULL, NULL);
     qmp->callback = cdrom_insert_inserted;
@@ -1158,8 +1494,13 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc,
     if (libxl__qmp_ev_qemu_compare_version(qmp, 2, 8, 0) >= 0) {
         QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid);
         QMP_PARAMETERS_SPRINTF(&args, "filename", "/dev/fdset/%d", fdset);
-        libxl__qmp_param_add_string(gc, &args, "format",
-            libxl__qemu_disk_format_string(disk->format));
+        if (response) {
+            libxl__qmp_param_add_string(gc, &args, "format",
+                libxl__qemu_disk_format_string(disk->format));
+        } else {
+            /* Stubdom is using blockdev /dev/xvd* */
+            libxl__qmp_param_add_string(gc, &args, "format", "host_device");
+        }
         rc = libxl__ev_qmp_send(egc, qmp, "blockdev-change-medium", args);
     } else {
         QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid);
@@ -1196,7 +1537,7 @@ static void cdrom_insert_inserted(libxl__egc *egc,
 
     if (rc) goto out;
 
-    rc = libxl__device_from_disk(gc, domid, disk, &device);
+    rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device);
     if (rc) goto out;
     be_path = libxl__device_backend_path(gc, &device);
     libxl_path = libxl__device_libxl_path(gc, &device);
@@ -1281,6 +1622,7 @@ static void cdrom_insert_done(libxl__egc *egc,
     EGC_GC;
 
     libxl__ev_time_deregister(gc, &cis->time);
+    libxl__ev_time_deregister(gc, &cis->retry_timer);
     libxl__ev_qmp_dispose(gc, &cis->qmp);
     if (cis->qmp.payload_fd >= 0) close(cis->qmp.payload_fd);
     libxl__ev_slowlock_unlock(gc, &cis->qmp_lock);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 02:22:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 02:22:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722778.1127082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Qky-0000aM-7c; Thu, 16 May 2024 02:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722778.1127082; Thu, 16 May 2024 02:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Qky-0000aF-4i; Thu, 16 May 2024 02:22:08 +0000
Received: by outflank-mailman (input) for mailman id 722778;
 Thu, 16 May 2024 02:22:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t9d+=MT=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s7Qkx-0000Zq-Gz
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 02:22:07 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16c4c973-132b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 04:22:06 +0200 (CEST)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-79305314956so11837585a.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 19:22:06 -0700 (PDT)
Received: from shine.lan
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a32e640233sm47296986d6.5.2024.05.15.19.22.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 19:22:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16c4c973-132b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715826124; x=1716430924; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+LlVXVnOsbp9AhquhYqJUISTf8i2JBzvpJscD8RA+R4=;
        b=NLQV/fRVQbP+idHvsmD/PsLUCtHd4smLff1fhNvZJ1mEuG2j8CS/rY9eQQxFglFlxW
         mUZiOureGv8me2DcJ5xQYOjPGgrN4fHza8+j1MJYAv63lC3W6nVuiJfTDzN2IVymhzaM
         DM+hYyg+Qu6ozxxjHmlKQXXEdM0rtMuqf0bh8iKt6inLDnff2aZx+EhFPLHaNMUTgXJh
         ZySUyJfH7lQHk8urg4mlZzl4v6KAc2CpE+am+qwdzYDs2iVRTW4XDEPTFBy/wy11LbKa
         gnLtzb25pjpghXyCiYvHwjNz8OludhUN7yG+TqdUxLXAgsHWZGuJglRaUJ996vq4fTGH
         DLnQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715826124; x=1716430924;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=+LlVXVnOsbp9AhquhYqJUISTf8i2JBzvpJscD8RA+R4=;
        b=jFLdSks89EnDdCTWMeLWyp72yK29fxrXmCtHYdPxhnzK13320XkVBtEYcL+Z5zwWZO
         E1V54iiD9nDWKKOqRNgRS6RdzM2ClqFs/VZSTWxrEalT28kR7XV7U5MmPmOYnn2urLrq
         5LyhJJnfBnfyGgsqDXbo/VfBc1odTjzOmfkWp4oSQwAtGrRyQQr5HcYygXZIsJcAlybn
         jSvpmqTJvMXK06IgF7+jWVfee5oiTEzej/rF8b+HU5kC7jjCBiVdvXVi6/rsygf7gmtV
         3ndHC1+wl8smAMcmBA7+8kYLWNqk5BIZpRYaN7RGduYtdCkHyWzTeEWOHerJv8Zz+cpI
         8Ilw==
X-Gm-Message-State: AOJu0Yy55yVsSNiLoEhZuok59r3X0vqOGQMsX5aeepGmFfra57Qqh9vF
	SQCPO9k8qCjubk/FNvzJZIU0FoqxFZRTECKrk9Jvjj6OQkla+x3rC0KfwA==
X-Google-Smtp-Source: AGHT+IGm4UkkoJyHu/MIRkY0KnFaw1Je1nQ+WJBSgRfsD8C9T0DAVqGpScorH3s07/4R6N1nc2noDg==
X-Received: by 2002:a05:6214:3bc2:b0:6a0:7df4:90b8 with SMTP id 6a1803df08f44-6a1681db1b5mr194253356d6.35.1715826124317;
        Wed, 15 May 2024 19:22:04 -0700 (PDT)
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>
Subject: [PATCH] CHANGELOG: Mention libxl blktap/tapback support
Date: Wed, 15 May 2024 22:21:53 -0400
Message-Id: <20240516022153.5008-1-jandryuk@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Jason Andryuk <jason.andryuk@amd.com>

From: Jason Andryuk <jason.andryuk@amd.com>

Add entry for backendtype=tap support in libxl.  blktap needs some
changes to work with libxl, which haven't been merged.  They are
available from this PR: https://github.com/xapi-project/blktap/pull/394

Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
 CHANGELOG.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index c43c45d8d4..438e307253 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
      for IPIs and Physical addressing mode for external interrupts.
  - Add a new 9pfs backend running as a daemon in dom0. First user is
    Xenstore-stubdom now being able to support full Xenstore trace capability.
+ - libxl support for backendtype=tap with tapback.
 
 ### Removed
 - caml-stubdom.  It hasn't built since 2014, was pinned to Ocaml 4.02, and has
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 02:22:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 02:22:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722779.1127092 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7QlB-0000vR-F4; Thu, 16 May 2024 02:22:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722779.1127092; Thu, 16 May 2024 02:22:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7QlB-0000vK-C8; Thu, 16 May 2024 02:22:21 +0000
Received: by outflank-mailman (input) for mailman id 722779;
 Thu, 16 May 2024 02:22:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t9d+=MT=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s7QlA-0000Zq-Fb
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 02:22:20 +0000
Received: from mail-yw1-x112e.google.com (mail-yw1-x112e.google.com
 [2607:f8b0:4864:20::112e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1eac1423-132b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 04:22:19 +0200 (CEST)
Received: by mail-yw1-x112e.google.com with SMTP id
 00721157ae682-61d35d266e7so89796137b3.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 19:22:19 -0700 (PDT)
Received: from shine.lan
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e17a8315csm49857151cf.32.2024.05.15.19.22.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 15 May 2024 19:22:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eac1423-132b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715826138; x=1716430938; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=H15xTstOQCjjtsIpAdxJAvKFKY+gIA01CkEDZA9sFY0=;
        b=WRopWRggIKQqOibs18+UAbixrLfHwQ0ViQWLNBB5EBN+LoPFXBKmGzEhJH/qLXLHHE
         0DwPjtnbhN+QK6QD/WHDusolF3X6hsFVpt53Xl5sKjb0RnR4BEOhlIZj1XxVXc0dm2/J
         b0kQ6EQS5bvtrIhWKtldvnpySyibh4dUARCY9we6f8vm3JI/3gXkWqW+2aJ1/uE1cKdW
         7Qk9Z0DcNyXT5+CSto7yHJTCgbPevq0NO2W0Z+SJLApFVrEFq/4hJcA7ofmCPnZ2b20S
         Jvkri87YqkOcLoCwFfky5Y/6p2lqtngltzXdiJJ5E3LoPJVcqhZkW3r6gS0Kg4kpz0Yh
         E37Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715826138; x=1716430938;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=H15xTstOQCjjtsIpAdxJAvKFKY+gIA01CkEDZA9sFY0=;
        b=G+w4N6nEnbM/mt/a2EFeFCWimdkNDEvANl/U7c+c4OP6kNp2cXVO0w8idBY1l+hbKJ
         PKMmZDzJ6qaxDC+p3Xu1eRFf4UXZCu/pGFtiU7pqKU/UVsftXKcoZRcSnTiIqaOa76IC
         u0DRelFA2MN3XEfEs6NHRA24bpfrvoqh2CpXnPVWSXpTytDsAyQBGpWw6VRp8DwnlVoq
         N3bEJd6+mNoZn2oFyAgZdRJCgNw1cY+dmffuznB7dWHnoQ0Ry48sD0zx9RJjePAzN5Ez
         FLnd2+5ru8mWmADnRmX2mkvi7Z/ZCwlwShRd9Re8m6FFpChs0fk/Xxo1BjzYRZqE5SM1
         iKPA==
X-Gm-Message-State: AOJu0YxF95ApNXbya4xR501w3mLQA+VTPFwE4/LpkKp2nGWw03Gi3fj6
	f8UhL2Egt0WfPEio975z7afq3UiMVmLjPH8o+68U9LjfZ7lIw547fjBJXA==
X-Google-Smtp-Source: AGHT+IHP9HhJ4g0luGl1dh4SVQ+af924DTrAwhF+FQCGPUOUjaM05KMvs+4f5/2vTAIbEZQGhv/PIw==
X-Received: by 2002:a05:690c:6c87:b0:61b:33c8:7bce with SMTP id 00721157ae682-622afff705emr179976697b3.31.1715826137844;
        Wed, 15 May 2024 19:22:17 -0700 (PDT)
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH] hotplug: Restore block-tap phy compatibility
Date: Wed, 15 May 2024 22:22:12 -0400
Message-Id: <20240516022212.5034-1-jandryuk@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Jason Andryuk <jason.andryuk@amd.com>

From: Jason Andryuk <jason.andryuk@amd.com>

backendtype=phy using the blktap kernel module needs to use write_dev,
but tapback can't support that.  tapback should perform better, but make
the script compatible with the old kernel module again.

Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
This is tested with tapback, but untested with phy/blkback since I don't
have the kernel module setup.

 tools/hotplug/Linux/block-tap | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/tools/hotplug/Linux/block-tap b/tools/hotplug/Linux/block-tap
index 126e472786..8661ce77e3 100755
--- a/tools/hotplug/Linux/block-tap
+++ b/tools/hotplug/Linux/block-tap
@@ -20,9 +20,13 @@
 #
 # Disks should be specified using the following syntax:
 #
+# For use with tapback (vbd3) (preferred):
 # vdev=xvda,backendtype=tap,format=vhd,target=/srv/target.vhd
 #
-# format is either "aio" (for raw files), or "vhd"
+# For use with blkback and the blktap2 kernel module:
+# script=block-tap,vdev=xvda,target=<type>:<file>
+#
+# format/<type> is either "aio" (for raw files), or "vhd"
 
 dir=$(dirname "$0")
 . "$dir/block-common.sh"
@@ -205,12 +209,17 @@ add()
 
     xenstore_write "$XENBUS_PATH/pid" "$pid"
     xenstore_write "$XENBUS_PATH/minor" "$minor"
-    # dev, as a unix socket, would end up with major:minor 0:0 in
-    # physical-device if write_dev were used.  tapback would be thrown off by
-    # that incorrect minor, so just skip writing physical-device.
-    xenstore_write "$XENBUS_PATH/physical-device-path" "$dev"
 
-    success
+    if [ "$XENBUS_TYPE" = "vbd3" ] ; then
+        # $dev, as a unix socket, has major:minor 0:0.  If write_dev writes
+        # physical-device, tapback would use that incorrect minor 0.  So don't
+        # write physical-device.
+        xenstore_write "$XENBUS_PATH/physical-device-path" "$dev"
+
+        success
+    else
+        write_dev "$dev"
+    fi
 
     release_lock "block"
 }
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 04:50:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 04:50:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722794.1127103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7T49-0006IR-Gi; Thu, 16 May 2024 04:50:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722794.1127103; Thu, 16 May 2024 04:50:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7T49-0006Ht-B2; Thu, 16 May 2024 04:50:05 +0000
Received: by outflank-mailman (input) for mailman id 722794;
 Thu, 16 May 2024 04:50:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7T48-00067y-Fm; Thu, 16 May 2024 04:50:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7T48-0002gs-Dt; Thu, 16 May 2024 04:50:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7T48-0006eD-2W; Thu, 16 May 2024 04:50:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7T48-0002CI-1z; Thu, 16 May 2024 04:50:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nEq6vfxa19GkunGPMaFdRHvIZp8K+c24hnpUO2xsbLw=; b=X82mPdQsqO8jsgfda+jiD57bmq
	LbZUUkv84+/bdEqbCrcgTTpnHSPmy6nIgxO4m0L4gsH0JzY2cZbCXKoNmpnw36095CXv4tpzamr/X
	Ity/90wWVPI6q3EmBtqDusyQ/QPmiUljA7FAF4rDy+vW3IZe5kSbWZS84gnwS+4tQR/Q=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186009-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186009: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=762a848223e0fd61aa87b4e040c83dd2c490fb79
X-Osstest-Versions-That:
    xen=abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 04:50:04 +0000

flight 186009 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186009/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  762a848223e0fd61aa87b4e040c83dd2c490fb79
baseline version:
 xen                  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4

Last test of basis   186006  2024-05-15 20:04:01 Z    0 days
Testing same since   186009  2024-05-16 01:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Nicola Vetrini <nicola.vetrini@bugseng.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   abd00b037d..762a848223  762a848223e0fd61aa87b4e040c83dd2c490fb79 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 16 05:22:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 05:22:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722801.1127111 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7TYm-0002lH-Qs; Thu, 16 May 2024 05:21:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722801.1127111; Thu, 16 May 2024 05:21:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7TYm-0002lA-Nz; Thu, 16 May 2024 05:21:44 +0000
Received: by outflank-mailman (input) for mailman id 722801;
 Thu, 16 May 2024 05:21:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hlyC=MT=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1s7TYl-0002l4-KM
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 05:21:43 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2bd96466-1344-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 07:21:40 +0200 (CEST)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.17.1) with ESMTPS id 44G5LMOX042847
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Thu, 16 May 2024 01:21:28 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 44G5LL92042846;
 Wed, 15 May 2024 22:21:21 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2bd96466-1344-11ef-b4bb-af5377834399
Date: Wed, 15 May 2024 22:21:21 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Kelly Choi <kelly.choi@cloud.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org,
        Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
        Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Subject: Re: Serious AMD-Vi(?) issue
Message-ID: <ZkWX0QgWRCQrlRv8@mattapan.m5p.com>
References: <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com>
 <ZiDBc3ye2wqmBAfq@mattapan.m5p.com>
 <f0bdb386-0870-4468-846c-6c8a91eaf806@suse.com>
 <ZiH0G5kN6m+wlNjn@mattapan.m5p.com>
 <Zj7vkp4r0EY9rxT4@mattapan.m5p.com>
 <ZkHTC4RpUSpKj4wf@macbook>
 <ZkJz9U2A/mVKnkjA@mattapan.m5p.com>
 <08503965-4ba9-4fe3-a176-895623f4ad92@suse.com>
 <ZkPO0D0/Jx4ZnRBO@mattapan.m5p.com>
 <CAO-mL=z=VsGx2Cimw7bW5bx8BBF+xfckEMBafCmqSi8GpFtdXw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAO-mL=z=VsGx2Cimw7bW5bx8BBF+xfckEMBafCmqSi8GpFtdXw@mail.gmail.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Wed, May 15, 2024 at 02:40:31PM +0100, Kelly Choi wrote:
> 
> As explained previously, we are happy to help resolve issues and provide
> advice where necessary. However, to do this, our developers need the
> relevant information to provide accurate resolutions. Given that our
> developers have repeatedly voiced their concerns, and are debugging this
> out of interest, please help us by providing all the necessary information.
> 
> Until we have this information, it will be very difficult to help you
> further. Should anything change, we would be glad to assist you.

Usually private submission of logs (PGP) is acceptable.

Note, I am not claiming Xen's `dmesg` contains truly concerning
information.  The issue is there is enough data for problematic
information to unintentionally leak in.  Alternatively no pieces would
be individually concerning, but all together information may leak.

Hopefully ACPI table addresses nor table order are effected by the
motherboard serial number, yet those could readily leak information.

So far this is acting like a major bug.  The paucity of reports is likely
due to few people using RAID1 with flash (most people relying greater
reliability even before the first large studies came out).


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Thu May 16 06:42:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 06:42:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722812.1127122 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7UoE-0005Bh-85; Thu, 16 May 2024 06:41:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722812.1127122; Thu, 16 May 2024 06:41:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7UoE-0005Ba-4y; Thu, 16 May 2024 06:41:46 +0000
Received: by outflank-mailman (input) for mailman id 722812;
 Thu, 16 May 2024 06:41:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7UoD-0005BU-BR
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 06:41:45 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5bf8e787-134f-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 08:41:44 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a387fbc9so294723666b.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 May 2024 23:41:43 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b015besm959373766b.144.2024.05.15.23.41.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 May 2024 23:41:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5bf8e787-134f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715841703; x=1716446503; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=DcyxgiWudPuLQJqexzwYYv9BWv3V3urNz/mkOmQHMY4=;
        b=XlbGo1u2m+MonrTGCUS7eBDH4WtSEAXwC/hGGEjFtPy/3MWXAsy3EdiDBV79g3xhsq
         Fd9mX+r7CqHCRYqriVpQWq0Qebh53lTJmhB5MEgLSSUvK0reJjfuz/eA52Yksso/rW8m
         PKNtrWKZwa8c5tnT+WcICztOwDKckOLmYVr1u7eBP3H7G4jjUW0tJjUHeT/s1JF6C/Ee
         TXVAi7qM+7XV0HpcOMu3vApGOO6Oc5FDT33DpzebLD1xMPGSYqmRFQ4WI6V7pVm+gBWL
         t9wsvN2HSWe6ZmKybd1EeM8CzKATckHDeWKirheVNPdYHtjBZKrAPGf8UcpuVG85CDGp
         x+zA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715841703; x=1716446503;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=DcyxgiWudPuLQJqexzwYYv9BWv3V3urNz/mkOmQHMY4=;
        b=ZaHn+r67AQmzx9UasbWyHPYDPytCe4mY/EnRBxqh69wmDw+eWlwl3hfdjr1WhU8M67
         C0ipM6ST1XHU1kH4SaqAMvI0iaqf7jLQtFkYFJVTY0Kq7jZHW/ZhoNjvHXYgsl5mbIyE
         kg8gLqJye/LAYNgl9ggkUxnmTYHTgwlYyK75qF6zkmde1Kpa17bm2x0HxDH7g/RMKHwi
         zgHSGKqwZYq6NIVfuUfmhzlMsP5ia+QYNSdiZVCgJ0gMbTmSy1MTCUEluITzMccCzB1F
         6ZQH/7VsdprxLwz0qZfJmeHcHIXyJvLRPdM6X2kGwmtxZDmo6H5PO8ggjLkwt03oEY80
         PxPA==
X-Forwarded-Encrypted: i=1; AJvYcCXcgNhDsB0GJi3Rbr/50oxoKRzmEkxNxbDCjJ5F0wRoQYnWRQf3w5wNPO26lTFwZ98rn0CSJLDD2F2kPIMVqX7xWjNaJc8oNzmdFHB5mN8=
X-Gm-Message-State: AOJu0YxdE0o2hrML8KdBWzFnaOtlYe67Nx8yjK6MQtFqOeBiJtoLbAt9
	QpGpD5gQZhqVOJWRhwKlTTLMwsJWcJwn55zpb1Q35+eEa4LCcYUa4vK4bq66Fg==
X-Google-Smtp-Source: AGHT+IFutTKVtufbLszoM37xzD0o5/dH0SW98ZP1VHRQnwU7efBGgx4xmr2ywX9/3v2pYvM3D868Dg==
X-Received: by 2002:a17:906:2288:b0:a59:ca33:683f with SMTP id a640c23a62f3a-a5a2d5859dcmr1082778466b.28.1715841703330;
        Wed, 15 May 2024 23:41:43 -0700 (PDT)
Message-ID: <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
Date: Thu, 16 May 2024 08:41:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Daniel Smith <dpsmith@apertussolutions.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 11:25, Jan Beulich wrote:
> On 03.04.2024 08:16, Jan Beulich wrote:
>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>> The commit makes a claim without any kind of justification.
>>
>> Well, what does "have no business" leave open?
>>
>>> The claim is false, and the commit broke lsevtchn in dom0.
>>
>> Or alternatively lsevtchn was doing something that was never meant to work
>> (from Xen's perspective).
>>
>>>  It is also quite
>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>>
>> Why would that be "obvious"? What business would a stubdom have to look at
>> Xen's side of an evtchn?
>>
>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>> and it's not acceptable to short circuit the XSM on the matter.
>>
>> I can certainly accept this as one possible view point. As in so many cases
>> I'm afraid I dislike you putting it as if it was the only possible one.
>>
>> In summary: The supposed justification you claim is missing in the original
>> change is imo also missing here then: What business would any entity in the
>> system have to look at Xen's side of an event channel? Back at the time, 3
>> people agreed that it's "none".
> 
> You've never responded to this reply of mine, or its follow-up. You also
> didn't chime in on the discussion Daniel and I were having. I consider my
> objections unaddressed, and in fact I continue to consider the change to
> be wrong. Therefore it was inappropriate for you to commit it; it needs
> reverting asap. If you're not going to do so, I will.

For the record: With Andrew having clarified that in his revert's
description:

"The claim is false; it broke lsevtchn in dom0, a debugging utility which
 absolutely does care about all of the domain's event channels."

"all of the domain's event channels" means to include event channels Xen
uses for its own, and merely puts in the domain's table for ease of
handling, I've agreed that - in the absence of any better debugging
means - the revert may stay in place. For context, my prior understanding
of the issue was that lsevtchn simply stops probing further channels when
getting back any kind of error from EVTCHNOP_status (which I continue to
think wants addressing there, by a future version of a patch that was
already sent). However, there are follow-on concerns I have:

1) In the discussion George claimed that exposing status information in
an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
how a similar assumption by CPU designers has led to a flood of
vulnerabilities over the last 6+ years. Information exposure imo is never
okay, unless it can be _proven_ that absolutely nothing "useful" can be
inferred from it. (I'm having difficulty seeing how such a proof might
look like.)

2) Me pointing out that the XSM hook might similarly get in the way of
debugging, Andrew suggested that this is not an issue because any sensible
XSM policy used in such an environment would grant sufficient privilege to
Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
for Xen-internal event channels. The debugging argument then becomes weak,
as in that case the XSM hook is possibly going to get in the way.

3) In the discussion Andrew further gave the impression that evtchn_send()
had no XSM check. Yet it has; the difference to evtchn_status() is that
the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
evtchn_status() may indeed be useful for debugging, evtchn_send() may be
similarly useful to allow getting a stuck channel unstuck.)

In summary I continue to think that an outright revert was inappropriate.
DomU-s should continue to be denied status information on Xen-internal
event channels, unconditionally and independent of whether dummy, silo, or
Flask is in use.

Plus, as stated before, evtchn_send() would better remain in sync in this
regard with evtchn_status(). The situation is less clear for evtchn_close()
and evtchn_bind_vcpu(): Those indeed have no XSM checks while they do deny
operation on Xen-internal channels. It is likely more far fetched to
assume permitting them for debugging purposes might in fact help in rare
situations. Yet it may still be a matter of consistency to keep them in
sync with the other two. (Note that there's also evtchn_usable(), which
might then also need tweaking itself or at the use sites.)

FTR, it is going to be only then that I would consider the cumulative
result as eligible for backporting. For this purpose, at the risk of
being flamed again, it might still be easier to revert the revert and then
put in place a change meeting the above criteria. That could then be taken
for backport as is.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 06:54:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 06:54:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722819.1127132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7V0H-0007DA-Be; Thu, 16 May 2024 06:54:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722819.1127132; Thu, 16 May 2024 06:54:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7V0H-0007D3-7V; Thu, 16 May 2024 06:54:13 +0000
Received: by outflank-mailman (input) for mailman id 722819;
 Thu, 16 May 2024 06:54:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7V0G-0007Cx-1y
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 06:54:12 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17bee40b-1351-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 08:54:10 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by LV2PR12MB5773.namprd12.prod.outlook.com (2603:10b6:408:17b::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Thu, 16 May
 2024 06:54:05 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Thu, 16 May 2024
 06:54:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17bee40b-1351-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fKguUsOok+qAQ2ZiPq6BqdvWPpQB5zwvIyAH1xB+grtOhcrhAPiKVwPzTahWzoeDyk5gbLy/ycNgk07Y0/rK4wAa5eV0k4kAXX4uFi36JihREFR6C2lXZm80Hrw773hrCXlXr/TjTPY+NlCMjhk2FCMJdjTHX9V8wn0f2Sv1E84Ep88VS7E0r0xLJfbjFWGimQBBPTj+OYq9B/Pq+cvhCu8OjBuwxbtDGQ5FjbA8Ud23y/WRKREWNDBCjIVEzvZQ8/getR9b5BIuBTXsGkvUiGhAdWlYyCaJ1tUuSGxjS90fuQz255d3ItdZ8s3p9WgHFMxSRfcXoaKzsPWrJ0AKVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xM34b5ZgBUIBqiV2bj2PEtymnUsgM9DU/OPdFf9NOq4=;
 b=Rga4r2IfDQNUPByIMGOvLPakjtO4omScD2lhtC3c+pJkCX6mkLSpWXva4h5KvK1ODSVjjWeEVgYINHZ361MyRcQqvYVegG0eMYiP29rx14lCKYRzMmGvgCV75XFdu9hXR8aoGlZ4Ze2OQxY3tTx7pI/gw+xXvz/fDcXVBep3MQbWQm6pfZsa5aI8IVjKRFCcqp01R+xCFXIQsj8EekXoIeS25Zo2S0YYZItq4y+rXztdH8uMVrUDovcVCUbxCdBS+abandNPfTWSMrZUDNCD13vtgiD4chuVKXFxuDPVzyfrQvmK+Sktsd5pq1CBLqqbQ5Inj2J81FkKe7LL1ob2Bg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xM34b5ZgBUIBqiV2bj2PEtymnUsgM9DU/OPdFf9NOq4=;
 b=LdpoUf+gUwIcU/5YeCWQou2c+pqgD8Ob8Bxjeo+amQdNRZgamRs6Hz5gG6FD8JaBiNG1zbn4v4b+gSCaWPikpNLkodvHNBBt6oHOAEQUoTt3jxcun6g29evBPwjq+6zwOnbkyRngBMMxTLt/RCdi5eFETbz7A/sdClU6jwT6Vlc=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Juergen Gross <jgross@suse.com>, Bjorn Helgaas <bhelgaas@google.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v7 2/2] xen/privcmd: Add new syscall to get gsi
 from dev
Thread-Topic: [RFC KERNEL PATCH v7 2/2] xen/privcmd: Add new syscall to get
 gsi from dev
Thread-Index: AQHappQxRQ+jZBZ+9UGzYgGu09lQUrGY5VCAgAENwoA=
Date: Thu, 16 May 2024 06:54:05 +0000
Message-ID:
 <BL1PR12MB584969F16D93CC4A5C8E1F0FE7ED2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240515065011.13797-1-Jiqian.Chen@amd.com>
 <20240515065011.13797-3-Jiqian.Chen@amd.com>
 <alpine.DEB.2.22.394.2405151537430.2544314@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2405151537430.2544314@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|LV2PR12MB5773:EE_
x-ms-office365-filtering-correlation-id: a47bb112-6c77-4fee-d989-08dc7574fa42
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?dzdiYWZWdmtMQUYzREpXQUtFamdtWFJJekhiWE55d2ZKZkpLemVFRmdxMHZL?=
 =?utf-8?B?Rnk0RnFoUUVlNHg0M21DNk9kM3ZKMTZoa0oyL05Femp1QzVjS3UvSzBXRnRP?=
 =?utf-8?B?YmE2U3AzaVpVYjRJeVQwZUdtdFBUQ3IwV1RKNzJrSExGNmg5MFFhaUpKQjYx?=
 =?utf-8?B?aW1md1BOZy9acnlXRTY2OXhuMmdQanpoeW5TTHkxcU1KVmxhUGFKS095bjlu?=
 =?utf-8?B?bys5czZwNy9MZXZLSUpNdXZ6aHdhNTZuWnQwbzI0UmErRWduQWxubVkxazZE?=
 =?utf-8?B?UWxIQmkzOXIzUi9kWHhiNTNISHhKMkR3UVZYMFhuMzYvd1FrN3NHS1BFZCsv?=
 =?utf-8?B?MjUrM0dPcE95eUxJa0prRW9kaGxLOWlwZzMyeUpIYSt3L2JoVys2R0QzYmdF?=
 =?utf-8?B?dEJUNDBJcEJubU95ZEZZMXhLLzh6d1o3MHNxMkMzeUx3M2E0QWlmTDZFUUpD?=
 =?utf-8?B?bzdmU0pRSk5nNk05SG00Qy95M1N1K2lJZmRYNzJyQ0ptVzZYenZnaWQvNXdK?=
 =?utf-8?B?UFVZUDZ4R3ViMm9LUENyKzVHSmwzTFAwbGozekJUT1p3OVN5d2Ftd2VNREVI?=
 =?utf-8?B?ZGZ3S0d2alFOWkZjUk1USTNnMWhBZWdBZ0paTHlaRUMzYUFKeWdpaERiUHkx?=
 =?utf-8?B?VTBXYXdFdW1LSk1wRHdNOThXUHo0Z0tLMW81VDl1eWhqR3o4Nk55bzBSRU41?=
 =?utf-8?B?Um4zSmRpcTdHUHM4YUIyZ05wR3RCMHhrcG1kM1VvL2ZQdGxyNG5IMktoaitX?=
 =?utf-8?B?dUdTQTZ4QzM3VFg5QlBNbm1TZ1lkNS9jWlRnTUxxU2FNMTVFWWlWWmlVVU96?=
 =?utf-8?B?NytOZ3lnZnA2QWxML0tXbU8wSThlMy9lMWlFd1VGWEZBcUJKeVo0NmdxWU82?=
 =?utf-8?B?NXpSR1BxQXR0cjlyakorQWF0cnp4dFpNTG0rMGRTU2xCdXBKcWwyYUlTMEdH?=
 =?utf-8?B?MDE3S25SKzZ0T0NYN2g1b0dmR1JxeE9CWkhFMkxmdEVUV3JoWSt5N2ZHVzJ1?=
 =?utf-8?B?YnhQOHRDdFI0NzVnQWU3a3l2eWR5akVQS3liZHExMFJKS0FSODBCK2N6Umpx?=
 =?utf-8?B?ZjhJMlhmVXkza25Kc1FmSUhtYWJEWFpYZFE4YjRFbGU0VmpkcGVvM3dkbzZS?=
 =?utf-8?B?QTdoYkJlYjhsSHZMSldGTGZaNHlUMXQ2QjM1YThEMWx3VHlFS2xVRm4vSkFM?=
 =?utf-8?B?c1RQaWxsWStINGlILzdxTFlnKzdHRUZsRjdMMlpOa0pDYlpTZ0xpZENLRXVz?=
 =?utf-8?B?bk5SUmFMck9MQWdvODdFNHd6YTNESTV1U1JLc1pKV00xbmRtZlFXMEM4NDJQ?=
 =?utf-8?B?MlVCWlp1bTZYQzRxZ3grbkpFcVRsTCtadWxIMFhIelEvZkhJSE9NSStmS0hE?=
 =?utf-8?B?aEVaTE1icXhRUFE0UjE2WVcrd05ETjQxUzJZdjloRHVXR1RWcFd4UmEwRHJI?=
 =?utf-8?B?NllBMlRaMUluY3RtcHgzTUswa0hXVWVYNTdzbW9KSkRRbTBRbEFqa2JyZmRm?=
 =?utf-8?B?bDdrUEwrM3JNalZQbGM2RE1LdmxDRklOWnJRdURyOXp3Smt5T1R2VUJDUERw?=
 =?utf-8?B?bmVqU2NYbFJNQ1FJdy9CWFF6NFRpZWliS2hEdEdwaG56VmVpaGNSemlqalAw?=
 =?utf-8?B?Mk9iZHc1c1RRMHptLzZCTERMN2lVcEZwbnMwdjlDT3B5bUttSkwycTFzeVMw?=
 =?utf-8?B?c29JMDYrb0NVczA5SGQxenZhaTVyNE5sd2ljYnMrbE5kWXhMSzIrUStZQjNC?=
 =?utf-8?B?eUxPK1dFN1htaFBKZXA3dXcwaURtWDFoZ2w5SlVhTWNNQlhmSC9WUGlBQVZO?=
 =?utf-8?B?MnIzZTJmNFNWTkFwcFhnUT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?NWw1YUQ2V3A3TGJJdG1YZlM4c2E3azFUTFBpclZzR3dLN0xPZ2U0QTVxVTNR?=
 =?utf-8?B?b1FFcEVHdnFacXpyV3dxRzFaRFhXRjc1bUFadnA1aFM4NDdDWHFaOUhjQ1d2?=
 =?utf-8?B?cGhJRDlLN01ZbEpCUWg1VDNKY0xFQm84R3N3SXl1VVhrYU1VSGVMaW02a2kv?=
 =?utf-8?B?TkdZdFZEVVRISGYzNkNHRlc5bU50TXRXQm9SNmN5L1MrWEVXQlM2NXNJYWhm?=
 =?utf-8?B?M3lIcS9sMm4rZStjeVcrR2phZXJldDV0UnE1N3B2Y1p4YVNlTktzQlE3ZU5q?=
 =?utf-8?B?SjBOQ0xiaFFMTlZnVjhvOXdUbVJweXFnK0dmUE9CRnI4WlloaktVa0ZZOVZT?=
 =?utf-8?B?YldQNWJxd1A1N2d0NDFEVE8vUWd0VGVEbEFMakl2SEo0d01kK1RIOTdZYVRm?=
 =?utf-8?B?R1ZWMTBtdFdVZzF3cVJ4QVRIVmJ5U1BqekpDOW1Nc09HRUZtcTJ5aDJ3SGFl?=
 =?utf-8?B?Rm4zdW0xeTNsYWhqZURwZ3lYekpzb0dObGd5Y2FCTXlnMm5pVWtMbDJjaXV2?=
 =?utf-8?B?YmV5ZzRxQWw3ZWtBS3RqdjJzK0NOVXZMYXVld2JVTG9iWlVEZGU4MDNud1g0?=
 =?utf-8?B?ckVUcTFjL0VFL3BXUUwvRW1CUGVJQ0pHTUZlK1BLUEdnODl6UVRQenpDajFs?=
 =?utf-8?B?bXhsdXNybTQ2MzkreDlQT0xVZjZhNWl0VGY5aTZjcGNDNzYxbTQwYUphZFNw?=
 =?utf-8?B?ZURXNGpqNkxmVzliL2ZFa0RoWEZ4bE9KSVNkNHlDNGNWeUJBZ01LS2didEJN?=
 =?utf-8?B?ZHBjNVpMUjFwbjJrYlNrQXc0aUJ2eG1BNzBNWUxTZWs4OVpWeWljYWNlbXJ6?=
 =?utf-8?B?WUVZQ090eVVsdURHT0hLV2lRN1NVcGsvVVNhSFZPbXNsNVVQMmJab21lR2xj?=
 =?utf-8?B?M2tLYWR0dktxQUsrTUtrQ3BoOHdrS2JpbHVkSU0xUGxXMThFd01FSHBBcUVC?=
 =?utf-8?B?WldNUi91RXlLc2l2b3BsSnY5ZEZKc2gvc0xHMG50WVJiUkFPTVRESmRUNnl3?=
 =?utf-8?B?QWFKK3ZCd1hzY3hBVG9qVFUzenlacnJsWC9QTnN2N3N5V1FPZW5ZRlFSTzl5?=
 =?utf-8?B?NDJoVjl5MHNXM3pZOTdwZTg2MmhIWDhWVElzcUNSa3JMcEFLWFdjRmNoOHBN?=
 =?utf-8?B?Z3hGVWg2Qy9QYzNFUzRCS1ZMc1c1RzdEb0NOYXl1ODYrVEtaTExZYUZaMmNW?=
 =?utf-8?B?Nnl4RVA0cUNDeEZJaXlHRXZFbDQ0bWlkdVdQNWtBcytiWmlxdDQxcVdGbHkz?=
 =?utf-8?B?Qnp3c0tGdS9naC9YbC8yYVZEM24wZy9ibFFpMUxiRzhIbHBnL25Ud0dwMUVi?=
 =?utf-8?B?V1BPdVNXN3FSRVRYSUJwRXdpelJBV3YxUW1CQ2VIaE55elFveXNnNnNrNjN1?=
 =?utf-8?B?Y2hXZ1B0MFN6STVrUk1DNjMrREhPL3B5WWNTc2d0SDdEM1AwazM4OW9uaERy?=
 =?utf-8?B?SnZmd2pLcTBpQzZYM3dsdFkxOU1CbmNVSWxUc0Fqdno1aEduSm1PTm1vOTV1?=
 =?utf-8?B?TkhvS3FHd0FHVE1OWnpIeUNwR3hkVjhsaGZsV20rSEp0bWxMWi9MT0NXaUJR?=
 =?utf-8?B?RG5oR0lVUWxiNFZtYURGVUpkRG9qcTRiNVJ5K2dZajdEais0NmZvQW9pdmQ0?=
 =?utf-8?B?SUNSZm1ScGl0cHlxWUxDRVhNLzU4U1Y2V1JVeHBCNElsUHhLTjZNeWZKMVA1?=
 =?utf-8?B?QjB0a2ZodzJaWjNxUy96cS9RWmV4RFFVcGtWZVVLc01BS0FlMFJLanBtY3hS?=
 =?utf-8?B?NU0rdEZ5S1ZnZGlsZDdOVXNRWFRKdk1OY2ZaVFZhZmtrNU9saEoyamphS1Rh?=
 =?utf-8?B?OFI1Mkg2dldMQkxuVEFXQUtYT3ZPWENTNVNtSHJVZURzbTFFRXdjN1ZYdWlC?=
 =?utf-8?B?ZnVoSmpNdEdwMHMyNUdaWnN5S3dBZDRWR1RNdTVSTm5PMVVLUUxDMzJBUGNE?=
 =?utf-8?B?WDZPbFp0cFliV3NKM1gwV0pCNVA3N3JpNkR2cFByUXNTSDQvM2M3NkcxQU9C?=
 =?utf-8?B?V3dkNGtQTGtDb3pTUVhMTittSlI1TWJWaDY3L1lRQmdvZkcwZlBrZjhLc0J3?=
 =?utf-8?B?d3RlakRmclRmMGMranRPUlZrc3R3MmYwdktZa1UzSGViMUJiY2ZDL3U2cTlY?=
 =?utf-8?Q?4DEE=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <DBD9AC12B24A724A99225388A515DDC6@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a47bb112-6c77-4fee-d989-08dc7574fa42
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 May 2024 06:54:05.4973
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Dilf7WJlqIDtaDoz9SUlYgBvNsnvAaoKxQMV8UMHwv5wLrJSDr/CpItfSl0SkPOTbOdOhi/wOiPeyGX7+R05Og==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5773

T24gMjAyNC81LzE2IDA2OjQyLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+IE9uIFdlZCwg
MTUgTWF5IDIwMjQsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4gSW4gUFZIIGRvbTAsIGl0IHVzZXMg
dGhlIGxpbnV4IGxvY2FsIGludGVycnVwdCBtZWNoYW5pc20sDQo+PiB3aGVuIGl0IGFsbG9jcyBp
cnEgZm9yIGEgZ3NpLCBpdCBpcyBkeW5hbWljLCBhbmQgZm9sbG93DQo+PiB0aGUgcHJpbmNpcGxl
IG9mIGFwcGx5aW5nIGZpcnN0LCBkaXN0cmlidXRpbmcgZmlyc3QuIEFuZA0KPj4gdGhlIGlycSBu
dW1iZXIgaXMgYWxsb2NlZCBmcm9tIHNtYWxsIHRvIGxhcmdlLCBidXQgdGhlDQo+PiBhcHBseWlu
ZyBnc2kgbnVtYmVyIGlzIG5vdCwgbWF5IGdzaSAzOCBjb21lcyBiZWZvcmUgZ3NpIDI4LA0KPj4g
aXQgY2F1c2VzIHRoZSBpcnEgbnVtYmVyIGlzIG5vdCBlcXVhbCB3aXRoIHRoZSBnc2kgbnVtYmVy
Lg0KPj4gQW5kIHdoZW4gcGFzc3Rocm91Z2ggYSBkZXZpY2UsIFFFTVUgd2lsbCB1c2UgZGV2aWNl
J3MgZ3NpDQo+PiBudW1iZXIgdG8gZG8gcGlycSBtYXBwaW5nLCBidXQgdGhlIGdzaSBudW1iZXIg
aXMgZ290IGZyb20NCj4+IGZpbGUgL3N5cy9idXMvcGNpL2RldmljZXMvPHNiZGY+L2lycSwgaXJx
IT0gZ3NpLCBzbyBpdCB3aWxsDQo+PiBmYWlsIHdoZW4gbWFwcGluZy4NCj4+IEFuZCBpbiBjdXJy
ZW50IGxpbnV4IGNvZGVzLCB0aGVyZSBpcyBubyBtZXRob2QgdG8gZ2V0IGdzaQ0KPj4gZm9yIHVz
ZXJzcGFjZS4NCj4+DQo+PiBGb3IgYWJvdmUgcHVycG9zZSwgcmVjb3JkIGdzaSBvZiBwY2lzdHVi
IGRldmljZXMgd2hlbiBpbml0DQo+PiBwY2lzdHViIGFuZCBhZGQgYSBuZXcgc3lzY2FsbCBpbnRv
IHByaXZjbWQgdG8gbGV0IHVzZXJzcGFjZQ0KPj4gY2FuIGdldCBnc2kgd2hlbiB0aGV5IGhhdmUg
YSBuZWVkLg0KPj4NCj4+IENvLWRldmVsb3BlZC1ieTogSHVhbmcgUnVpIDxyYXkuaHVhbmdAYW1k
LmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEppcWlhbiBDaGVuIDxKaXFpYW4uQ2hlbkBhbWQuY29t
Pg0KPj4gLS0tDQo+PiAgZHJpdmVycy94ZW4vcHJpdmNtZC5jICAgICAgICAgICAgICB8IDI4ICsr
KysrKysrKysrKysrKysrKysrKysNCj4+ICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1
Yi5jIHwgMzggKysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tDQo+PiAgaW5jbHVkZS91YXBp
L3hlbi9wcml2Y21kLmggICAgICAgICB8ICA3ICsrKysrKw0KPj4gIGluY2x1ZGUveGVuL2FjcGku
aCAgICAgICAgICAgICAgICAgfCAgMiArKw0KPj4gIDQgZmlsZXMgY2hhbmdlZCwgNzIgaW5zZXJ0
aW9ucygrKSwgMyBkZWxldGlvbnMoLSkNCj4+DQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4v
cHJpdmNtZC5jIGIvZHJpdmVycy94ZW4vcHJpdmNtZC5jDQo+PiBpbmRleCA2N2RmYTQ3Nzg4NjQu
LjU5NTNhMDNiNWNiMCAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3ByaXZjbWQuYw0KPj4g
KysrIGIvZHJpdmVycy94ZW4vcHJpdmNtZC5jDQo+PiBAQCAtNDUsNiArNDUsOSBAQA0KPj4gICNp
bmNsdWRlIDx4ZW4vcGFnZS5oPg0KPj4gICNpbmNsdWRlIDx4ZW4veGVuLW9wcy5oPg0KPj4gICNp
bmNsdWRlIDx4ZW4vYmFsbG9vbi5oPg0KPj4gKyNpZmRlZiBDT05GSUdfQUNQSQ0KPj4gKyNpbmNs
dWRlIDx4ZW4vYWNwaS5oPg0KPj4gKyNlbmRpZg0KPj4gIA0KPj4gICNpbmNsdWRlICJwcml2Y21k
LmgiDQo+PiAgDQo+PiBAQCAtODQyLDYgKzg0NSwyNyBAQCBzdGF0aWMgbG9uZyBwcml2Y21kX2lv
Y3RsX21tYXBfcmVzb3VyY2Uoc3RydWN0IGZpbGUgKmZpbGUsDQo+PiAgCXJldHVybiByYzsNCj4+
ICB9DQo+PiAgDQo+PiArc3RhdGljIGxvbmcgcHJpdmNtZF9pb2N0bF9nc2lfZnJvbV9kZXYoc3Ry
dWN0IGZpbGUgKmZpbGUsIHZvaWQgX191c2VyICp1ZGF0YSkNCj4+ICt7DQo+PiArCXN0cnVjdCBw
cml2Y21kX2dzaV9mcm9tX2RldiBrZGF0YTsNCj4+ICsNCj4+ICsJaWYgKGNvcHlfZnJvbV91c2Vy
KCZrZGF0YSwgdWRhdGEsIHNpemVvZihrZGF0YSkpKQ0KPj4gKwkJcmV0dXJuIC1FRkFVTFQ7DQo+
PiArDQo+PiArI2lmZGVmIENPTkZJR19BQ1BJDQo+PiArCWtkYXRhLmdzaSA9IHBjaXN0dWJfZ2V0
X2dzaV9mcm9tX3NiZGYoa2RhdGEuc2JkZik7DQo+PiArCWlmIChrZGF0YS5nc2kgPT0gLTEpDQo+
PiArCQlyZXR1cm4gLUVJTlZBTDsNCj4+ICsjZWxzZQ0KPj4gKwlrZGF0YS5nc2kgPSAtMTsNCj4g
DQo+IFNob3VsZCB3ZSByZXR1cm4gYW4gZXJyb3IgaW5zdGVhZCwgbGlrZSAtRUlOVkFMLCB0byBt
YWtlIHRoZSBiZWhhdmlvcg0KPiBtb3JlIHNpbWlsYXIgdG8gdGhlIENPTkZJR19BQ1BJIGNhc2U/
DQpPSywgd2lsbCByZXR1cm4gLUVJTlZBTCBpZiBub3QgY29uZmlnIGFjcGkuDQpMaWtlOg0Kc3Rh
dGljIGxvbmcgcHJpdmNtZF9pb2N0bF9nc2lfZnJvbV9kZXYoc3RydWN0IGZpbGUgKmZpbGUsIHZv
aWQgX191c2VyICp1ZGF0YSkNCnsNCiNpZmRlZiBDT05GSUdfQUNQSQ0KCXN0cnVjdCBwcml2Y21k
X2dzaV9mcm9tX2RldiBrZGF0YTsNCg0KCWlmIChjb3B5X2Zyb21fdXNlcigma2RhdGEsIHVkYXRh
LCBzaXplb2Yoa2RhdGEpKSkNCgkJcmV0dXJuIC1FRkFVTFQ7DQoNCglrZGF0YS5nc2kgPSBwY2lz
dHViX2dldF9nc2lfZnJvbV9zYmRmKGtkYXRhLnNiZGYpOw0KCWlmIChrZGF0YS5nc2kgPT0gLTEp
DQoJCXJldHVybiAtRUlOVkFMOw0KDQoJaWYgKGNvcHlfdG9fdXNlcih1ZGF0YSwgJmtkYXRhLCBz
aXplb2Yoa2RhdGEpKSkNCgkJcmV0dXJuIC1FRkFVTFQ7DQoNCglyZXR1cm4gMDsNCiNlbHNlDQoJ
cmV0dXJuIC1FSU5WQUw7DQojZW5kaWYNCn0NCg0KPiANCj4gDQo+PiArI2VuZGlmDQo+PiArDQo+
PiArCWlmIChjb3B5X3RvX3VzZXIodWRhdGEsICZrZGF0YSwgc2l6ZW9mKGtkYXRhKSkpDQo+PiAr
CQlyZXR1cm4gLUVGQVVMVDsNCj4+ICsNCj4+ICsJcmV0dXJuIDA7DQo+PiArfQ0KPj4gKw0KPj4g
ICNpZmRlZiBDT05GSUdfWEVOX1BSSVZDTURfRVZFTlRGRA0KPj4gIC8qIElycWZkIHN1cHBvcnQg
Ki8NCj4+ICBzdGF0aWMgc3RydWN0IHdvcmtxdWV1ZV9zdHJ1Y3QgKmlycWZkX2NsZWFudXBfd3E7
DQo+PiBAQCAtMTUyOSw2ICsxNTUzLDEwIEBAIHN0YXRpYyBsb25nIHByaXZjbWRfaW9jdGwoc3Ry
dWN0IGZpbGUgKmZpbGUsDQo+PiAgCQlyZXQgPSBwcml2Y21kX2lvY3RsX2lvZXZlbnRmZChmaWxl
LCB1ZGF0YSk7DQo+PiAgCQlicmVhazsNCj4+ICANCj4+ICsJY2FzZSBJT0NUTF9QUklWQ01EX0dT
SV9GUk9NX0RFVjoNCj4+ICsJCXJldCA9IHByaXZjbWRfaW9jdGxfZ3NpX2Zyb21fZGV2KGZpbGUs
IHVkYXRhKTsNCj4+ICsJCWJyZWFrOw0KPj4gKw0KPj4gIAlkZWZhdWx0Og0KPj4gIAkJYnJlYWs7
DQo+PiAgCX0NCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1
Yi5jIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0dWIuYw0KPj4gaW5kZXggMmI5MGQ4
MzJkMGE3Li40YjYyYjRkMzc3YTkgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNp
YmFjay9wY2lfc3R1Yi5jDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1
Yi5jDQo+PiBAQCAtNTYsNiArNTYsOSBAQCBzdHJ1Y3QgcGNpc3R1Yl9kZXZpY2Ugew0KPj4gIA0K
Pj4gIAlzdHJ1Y3QgcGNpX2RldiAqZGV2Ow0KPj4gIAlzdHJ1Y3QgeGVuX3BjaWJrX2RldmljZSAq
cGRldjsvKiBub24tTlVMTCBpZiBzdHJ1Y3QgcGNpX2RldiBpcyBpbiB1c2UgKi8NCj4+ICsjaWZk
ZWYgQ09ORklHX0FDUEkNCj4+ICsJaW50IGdzaTsNCj4+ICsjZW5kaWYNCj4+ICB9Ow0KPj4gIA0K
Pj4gIC8qIEFjY2VzcyB0byBwY2lzdHViX2RldmljZXMgJiBzZWl6ZWRfZGV2aWNlcyBsaXN0cyBh
bmQgdGhlIGluaXRpYWxpemVfZGV2aWNlcw0KPj4gQEAgLTg4LDYgKzkxLDkgQEAgc3RhdGljIHN0
cnVjdCBwY2lzdHViX2RldmljZSAqcGNpc3R1Yl9kZXZpY2VfYWxsb2Moc3RydWN0IHBjaV9kZXYg
KmRldikNCj4+ICANCj4+ICAJa3JlZl9pbml0KCZwc2Rldi0+a3JlZik7DQo+PiAgCXNwaW5fbG9j
a19pbml0KCZwc2Rldi0+bG9jayk7DQo+PiArI2lmZGVmIENPTkZJR19BQ1BJDQo+PiArCXBzZGV2
LT5nc2kgPSAtMTsNCj4+ICsjZW5kaWYNCj4+ICANCj4+ICAJcmV0dXJuIHBzZGV2Ow0KPj4gIH0N
Cj4+IEBAIC0yMjAsNiArMjI2LDI1IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2RldiAqcGNpc3R1Yl9k
ZXZpY2VfZ2V0X3BjaV9kZXYoc3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXYsDQo+PiAgCXJl
dHVybiBwY2lfZGV2Ow0KPj4gIH0NCj4+ICANCj4+ICsjaWZkZWYgQ09ORklHX0FDUEkNCj4+ICtp
bnQgcGNpc3R1Yl9nZXRfZ3NpX2Zyb21fc2JkZih1bnNpZ25lZCBpbnQgc2JkZikNCj4+ICt7DQo+
PiArCXN0cnVjdCBwY2lzdHViX2RldmljZSAqcHNkZXY7DQo+PiArCWludCBkb21haW4gPSBzYmRm
ID4+IDE2Ow0KPj4gKwlpbnQgYnVzID0gKHNiZGYgPj4gOCkgJiAweGZmOw0KPj4gKwlpbnQgc2xv
dCA9IChzYmRmID4+IDMpICYgMHgxZjsNCj4+ICsJaW50IGZ1bmMgPSBzYmRmICYgMHg3Ow0KPiAN
Cj4geW91IGNhbiB1c2UgUENJX0RFVkZOIFBDSV9TTE9UIFBDSV9GVU5DIHBjaV9kb21haW5fbnIg
aW5zdGVhZCBvZiBvcGVuDQo+IGNvZGluZy4NClRoYW5rcywgd2lsbCBjaGFuZ2UgdG8gdXNlIHRo
ZXNlIGluIG5leHQgdmVyc2lvbi4NCkJ1dCBwY2lfZG9tYWluX25yIHJlcXVpcmVzIHBhc3Npbmcg
aW4gcGNpX2Rldi4NCldpbGwgY2hhbmdlIGxpa2U6DQoJaW50IGRvbWFpbiA9IChzYmRmID4+IDE2
KSAmIDB4ZmZmZjsNCglpbnQgYnVzID0gUENJX0JVU19OVU0oc2JkZik7DQoJaW50IHNsb3QgPSBQ
Q0lfU0xPVChzYmRmKTsNCglpbnQgZnVuYyA9IFBDSV9GVU5DKHNiZGYpOw0KDQo+IA0KPiANCj4+
ICsNCj4+ICsJcHNkZXYgPSBwY2lzdHViX2RldmljZV9maW5kKGRvbWFpbiwgYnVzLCBzbG90LCBm
dW5jKTsNCj4+ICsNCj4+ICsJaWYgKCFwc2RldikNCj4+ICsJCXJldHVybiAtMTsNCj4+ICsNCj4+
ICsJcmV0dXJuIHBzZGV2LT5nc2k7DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHBjaXN0
dWJfZ2V0X2dzaV9mcm9tX3NiZGYpOw0KPj4gKyNlbmRpZg0KPj4gKw0KPj4gIHN0cnVjdCBwY2lf
ZGV2ICpwY2lzdHViX2dldF9wY2lfZGV2X2J5X3Nsb3Qoc3RydWN0IHhlbl9wY2lia19kZXZpY2Ug
KnBkZXYsDQo+PiAgCQkJCQkgICAgaW50IGRvbWFpbiwgaW50IGJ1cywNCj4+ICAJCQkJCSAgICBp
bnQgc2xvdCwgaW50IGZ1bmMpDQo+PiBAQCAtMzY3LDE0ICszOTIsMjAgQEAgc3RhdGljIGludCBw
Y2lzdHViX21hdGNoKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAgCXJldHVybiBmb3VuZDsNCj4+
ICB9DQo+PiAgDQo+PiAtc3RhdGljIGludCBwY2lzdHViX2luaXRfZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpkZXYpDQo+PiArc3RhdGljIGludCBwY2lzdHViX2luaXRfZGV2aWNlKHN0cnVjdCBwY2lz
dHViX2RldmljZSAqcHNkZXYpDQo+PiAgew0KPj4gIAlzdHJ1Y3QgeGVuX3BjaWJrX2Rldl9kYXRh
ICpkZXZfZGF0YTsNCj4+ICsJc3RydWN0IHBjaV9kZXYgKmRldjsNCj4+ICAjaWZkZWYgQ09ORklH
X0FDUEkNCj4+ICAJaW50IGdzaSwgdHJpZ2dlciwgcG9sYXJpdHk7DQo+PiAgI2VuZGlmDQo+PiAg
CWludCBlcnIgPSAwOw0KPj4gIA0KPj4gKwlpZiAoIXBzZGV2KQ0KPj4gKwkJcmV0dXJuIC1FSU5W
QUw7DQo+PiArDQo+PiArCWRldiA9IHBzZGV2LT5kZXY7DQo+PiArDQo+PiAgCWRldl9kYmcoJmRl
di0+ZGV2LCAiaW5pdGlhbGl6aW5nLi4uXG4iKTsNCj4+ICANCj4+ICAJLyogVGhlIFBDSSBiYWNr
ZW5kIGlzIG5vdCBpbnRlbmRlZCB0byBiZSBhIG1vZHVsZSAob3IgdG8gd29yayB3aXRoDQo+PiBA
QCAtNDQ4LDYgKzQ3OSw3IEBAIHN0YXRpYyBpbnQgcGNpc3R1Yl9pbml0X2RldmljZShzdHJ1Y3Qg
cGNpX2RldiAqZGV2KQ0KPj4gIAkJZGV2X2VycigmZGV2LT5kZXYsICJGYWlsIHRvIGdldCBnc2kg
aW5mbyFcbiIpOw0KPj4gIAkJZ290byBjb25maWdfcmVsZWFzZTsNCj4+ICAJfQ0KPj4gKwlwc2Rl
di0+Z3NpID0gZ3NpOw0KPj4gIA0KPj4gIAlpZiAoeGVuX2luaXRpYWxfZG9tYWluKCkgJiYgeGVu
X3B2aF9kb21haW4oKSkgew0KPj4gIAkJZXJyID0geGVuX3B2aF9zZXR1cF9nc2koZ3NpLCB0cmln
Z2VyLCBwb2xhcml0eSk7DQo+PiBAQCAtNDk1LDcgKzUyNyw3IEBAIHN0YXRpYyBpbnQgX19pbml0
IHBjaXN0dWJfaW5pdF9kZXZpY2VzX2xhdGUodm9pZCkNCj4+ICANCj4+ICAJCXNwaW5fdW5sb2Nr
X2lycXJlc3RvcmUoJnBjaXN0dWJfZGV2aWNlc19sb2NrLCBmbGFncyk7DQo+PiAgDQo+PiAtCQll
cnIgPSBwY2lzdHViX2luaXRfZGV2aWNlKHBzZGV2LT5kZXYpOw0KPj4gKwkJZXJyID0gcGNpc3R1
Yl9pbml0X2RldmljZShwc2Rldik7DQo+PiAgCQlpZiAoZXJyKSB7DQo+PiAgCQkJZGV2X2Vycigm
cHNkZXYtPmRldi0+ZGV2LA0KPj4gIAkJCQkiZXJyb3IgJWQgaW5pdGlhbGl6aW5nIGRldmljZVxu
IiwgZXJyKTsNCj4+IEBAIC01NjUsNyArNTk3LDcgQEAgc3RhdGljIGludCBwY2lzdHViX3NlaXpl
KHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiAgCQlzcGluX3VubG9ja19pcnFyZXN0b3JlKCZwY2lz
dHViX2RldmljZXNfbG9jaywgZmxhZ3MpOw0KPj4gIA0KPj4gIAkJLyogZG9uJ3Qgd2FudCBpcnFz
IGRpc2FibGVkIHdoZW4gY2FsbGluZyBwY2lzdHViX2luaXRfZGV2aWNlICovDQo+PiAtCQllcnIg
PSBwY2lzdHViX2luaXRfZGV2aWNlKHBzZGV2LT5kZXYpOw0KPj4gKwkJZXJyID0gcGNpc3R1Yl9p
bml0X2RldmljZShwc2Rldik7DQo+PiAgDQo+PiAgCQlzcGluX2xvY2tfaXJxc2F2ZSgmcGNpc3R1
Yl9kZXZpY2VzX2xvY2ssIGZsYWdzKTsNCj4+ICANCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL3Vh
cGkveGVuL3ByaXZjbWQuaCBiL2luY2x1ZGUvdWFwaS94ZW4vcHJpdmNtZC5oDQo+PiBpbmRleCA4
YjhjNWQxNDIwZmUuLjIyMGU3NjcwYTExMyAxMDA2NDQNCj4+IC0tLSBhL2luY2x1ZGUvdWFwaS94
ZW4vcHJpdmNtZC5oDQo+PiArKysgYi9pbmNsdWRlL3VhcGkveGVuL3ByaXZjbWQuaA0KPj4gQEAg
LTEyNiw2ICsxMjYsMTEgQEAgc3RydWN0IHByaXZjbWRfaW9ldmVudGZkIHsNCj4+ICAJX191OCBw
YWRbMl07DQo+PiAgfTsNCj4+ICANCj4+ICtzdHJ1Y3QgcHJpdmNtZF9nc2lfZnJvbV9kZXYgew0K
Pj4gKwlfX3UzMiBzYmRmOw0KPj4gKwlpbnQgZ3NpOw0KPj4gK307DQo+PiArDQo+PiAgLyoNCj4+
ICAgKiBAY21kOiBJT0NUTF9QUklWQ01EX0hZUEVSQ0FMTA0KPj4gICAqIEBhcmc6ICZwcml2Y21k
X2h5cGVyY2FsbF90DQo+PiBAQCAtMTU3LDUgKzE2Miw3IEBAIHN0cnVjdCBwcml2Y21kX2lvZXZl
bnRmZCB7DQo+PiAgCV9JT1coJ1AnLCA4LCBzdHJ1Y3QgcHJpdmNtZF9pcnFmZCkNCj4+ICAjZGVm
aW5lIElPQ1RMX1BSSVZDTURfSU9FVkVOVEZECQkJCQlcDQo+PiAgCV9JT1coJ1AnLCA5LCBzdHJ1
Y3QgcHJpdmNtZF9pb2V2ZW50ZmQpDQo+PiArI2RlZmluZSBJT0NUTF9QUklWQ01EX0dTSV9GUk9N
X0RFVgkJCQlcDQo+PiArCV9JT0MoX0lPQ19OT05FLCAnUCcsIDEwLCBzaXplb2Yoc3RydWN0IHBy
aXZjbWRfZ3NpX2Zyb21fZGV2KSkNCj4+ICANCj4+ICAjZW5kaWYgLyogX19MSU5VWF9QVUJMSUNf
UFJJVkNNRF9IX18gKi8NCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL3hlbi9hY3BpLmggYi9pbmNs
dWRlL3hlbi9hY3BpLmgNCj4+IGluZGV4IDliNTAwMjcxMTNmMy4uMGJmNWY0ODg0NDU2IDEwMDY0
NA0KPj4gLS0tIGEvaW5jbHVkZS94ZW4vYWNwaS5oDQo+PiArKysgYi9pbmNsdWRlL3hlbi9hY3Bp
LmgNCj4+IEBAIC04Myw0ICs4Myw2IEBAIGludCB4ZW5fYWNwaV9nZXRfZ3NpX2luZm8oc3RydWN0
IHBjaV9kZXYgKmRldiwNCj4+ICAJCQkJCQkgIGludCAqZ3NpX291dCwNCj4+ICAJCQkJCQkgIGlu
dCAqdHJpZ2dlcl9vdXQsDQo+PiAgCQkJCQkJICBpbnQgKnBvbGFyaXR5X291dCk7DQo+PiArDQo+
PiAraW50IHBjaXN0dWJfZ2V0X2dzaV9mcm9tX3NiZGYodW5zaWduZWQgaW50IHNiZGYpOw0KPj4g
ICNlbmRpZgkvKiBfWEVOX0FDUElfSCAqLw0KPj4gLS0gDQo+PiAyLjM0LjENCj4+DQoNCi0tIA0K
QmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:04:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:04:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722823.1127141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VAX-0000s0-1H; Thu, 16 May 2024 07:04:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722823.1127141; Thu, 16 May 2024 07:04:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VAW-0000rt-Ux; Thu, 16 May 2024 07:04:48 +0000
Received: by outflank-mailman (input) for mailman id 722823;
 Thu, 16 May 2024 07:04:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7VAV-0000rn-GQ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:04:47 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 93d6da37-1352-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 09:04:46 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-574b3d6c0f3so2963359a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:04:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01631sm948048866b.153.2024.05.16.00.04.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:04:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93d6da37-1352-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715843085; x=1716447885; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Zpc41xGROJJynsJRtAZSKaAEcXphCP8ViT6H4bQeDPc=;
        b=X4/hOUd2DK7mqW6SG0SPfjUnT2ubtcWZd6B/gAct4o9jiidIu38RhOjsFWe23thlpb
         lnY0SDMEBXcEld5X+U+mfhd2vOHdY4JsVxTNh8HM1ibrqo8dj5sMs5PbC4JrFTf+fLLM
         rumraPSa6/iR/mdLF5pMjvJQHKPt0Cs6oQVWHEH7p/pph4daSHH1Y+obLKSq7Wxa2YMp
         b4tsljAhjMwc//t3R/G8kx9FtQlhLihSwgJpt3/C54bqsVRnKL4s8sOEbXhcoxawLQSC
         fQLXRUo7Dr7cWg9+sstc5304LLY5zAkXIWo/MJcenuQ6OEowhGMWyUJSTQgQ4/q4fHi8
         /+VA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715843085; x=1716447885;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Zpc41xGROJJynsJRtAZSKaAEcXphCP8ViT6H4bQeDPc=;
        b=OPgfX+mAWyGOocU/Zj+hlmHlS00L9DdOtdp9HC00Ug0KGAoiRPxGZZdYZhyTtdF5h5
         XSMqu+P3Su2So1Zr6vOU4b+JIfY5cbCl7UtMjhko13JXBrFR/ehleZd9PxdGd02yDYH4
         6GVkx/31XPZ/QVjK/v6x4FVwNzJuKdTGj2Spa2JCuVRTd7uqxD7JlkBJDMXpRxNk+QuD
         OjRRRpjJHfdDoWxnX9QJeXyMxkejeKVE46sBXDg7aBaNmczBBVENMH+hXKJx0wprsXOm
         R+APYn9Bg7EZ+GrKbp1CfuH4F+6MrkL2gVR7mXgQIqrrmsPRCoXKNjK/asBSY4UvhagB
         zJVQ==
X-Forwarded-Encrypted: i=1; AJvYcCXCjCJ10uFtnmGV9hbia2CPO/TVkDnekjuQZKrsV1x+V6pksCYjXXjm6C6uDYVrf/IK8sEADglFH1KPhuNM3fX8d2kjFt5QcLv5A48A71M=
X-Gm-Message-State: AOJu0YwE8ZCcynZ0dFXK5BkTtNV1lQx39mu1m7RjCOdGGiA3SKICXm01
	TaCxhlKhhxidbnAntcfX4/GwPNOTT6BBQUBY7MK7ktIesYBwfWfofQJPNGoB9g==
X-Google-Smtp-Source: AGHT+IFR1WmGeyJ9MRydx8kvWbB32wih4cugjErKY++43tS5EixybAzG830K9AVX1i0zvWn8SWiNrA==
X-Received: by 2002:a17:906:f6da:b0:a5a:1ba1:1e7b with SMTP id a640c23a62f3a-a5a2d584119mr1177032566b.19.1715843085571;
        Thu, 16 May 2024 00:04:45 -0700 (PDT)
Message-ID: <77c6e497-631d-4f92-bd38-8ab7dba4841d@suse.com>
Date: Thu, 16 May 2024 09:04:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
 <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
 <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 19:03, Oleksii K. wrote:
> On Wed, 2024-05-15 at 17:41 +0200, Jan Beulich wrote:
>> On 15.05.2024 17:29, Oleksii K. wrote:
>>> On Wed, 2024-05-15 at 10:52 +0200, Jan Beulich wrote:
>>>> On 06.05.2024 12:15, Oleksii Kurochko wrote:
>>>>> The following generic functions were introduced:
>>>>> * test_bit
>>>>> * generic__test_and_set_bit
>>>>> * generic__test_and_clear_bit
>>>>> * generic__test_and_change_bit
>>>>>
>>>>> Also, the patch introduces the following generics which are
>>>>> used by the functions mentioned above:
>>>>> * BITOP_BITS_PER_WORD
>>>>> * BITOP_MASK
>>>>> * BITOP_WORD
>>>>> * BITOP_TYPE
>>>>>
>>>>> These functions and macros can be useful for architectures
>>>>> that don't have corresponding arch-specific instructions.
>>>>
>>>> Logically this paragraph may better move ahead of the BITOP_*
>>>> one.
>>>>
>>>>> Because of that x86 has the following check in the macros
>>>>> test_bit(),
>>>>> __test_and_set_bit(), __test_and_clear_bit(),
>>>>> __test_and_change_bit():
>>>>>     if ( bitop_bad_size(addr) ) __bitop_bad_size();
>>>>> It was necessary to make bitop bad size check generic too, so
>>>>> arch_check_bitop_size() was introduced.
>>>>
>>>> Not anymore, with the most recent adjustments? There's nothing
>>>> arch-
>>>> specific anymore in the checking.
>>>>
>>>>> @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr,
>>>>> volatile void *addr)
>>>>>   * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>>   * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>>   */
>>>>> -static inline int __test_and_set_bit(int nr, void *addr)
>>>>> +static inline int arch__test_and_set_bit(int nr, volatile void
>>>>> *addr)
>>>>
>>>> I think I raised this point before: Why arch__ here, ...
>>>>
>>>>> @@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int
>>>>> nr,
>>>>> volatile void *addr)
>>>>>   * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>>   * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>>   */
>>>>> -static inline int __test_and_clear_bit(int nr, void *addr)
>>>>> +static inline int arch__test_and_clear_bit(int nr, volatile
>>>>> void
>>>>> *addr)
>>>>
>>>> ... here, ...
>>>>
>>>>> @@ -243,13 +237,10 @@ static inline int
>>>>> __test_and_clear_bit(int
>>>>> nr, void *addr)
>>>>>  
>>>>>      return oldbit;
>>>>>  }
>>>>> -#define __test_and_clear_bit(nr, addr) ({               \
>>>>> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
>>>>> -    __test_and_clear_bit(nr, addr);                     \
>>>>> -})
>>>>> +#define arch__test_and_clear_bit arch__test_and_clear_bit
>>>>>  
>>>>>  /* WARNING: non atomic and it can be reordered! */
>>>>> -static inline int __test_and_change_bit(int nr, void *addr)
>>>>> +static inline int arch__test_and_change_bit(int nr, volatile
>>>>> void
>>>>> *addr)
>>>>
>>>> ... and here, while ...
>>>>
>>>>> @@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr,
>>>>> const volatile void *addr)
>>>>>      return oldbit;
>>>>>  }
>>>>>  
>>>>> -#define test_bit(nr, addr) ({                           \
>>>>> -    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
>>>>> +#define arch_test_bit(nr, addr) ({                      \
>>>>
>>>> ... just arch_ here? I don't like the double underscore infixes
>>>> very
>>>> much, but I'm okay with them as long as they're applied
>>>> consistently.
>>>
>>> Common code and x86 use __test_and_clear_bit(), and this patch
>>> provides
>>> a generic version of __test_and_clear_bit(). To emphasize that
>>> generic__test_and_clear_bit() is a common implementation of
>>> __test_and_clear_bit(), the double underscore was retained. Also,
>>> test_and_clear_bit() exists and if one day it will be needed to
>>> provide
>>> a generic version of it, then it will be needed to have
>>> generic__test_and_clear_bit() and generic_test_and_clear_bit()
>>>
>>> A similar logic was chosen for test_bit.
>>
>> Right, but in all of your reply arch_ doesn't appear at all.
> I am a little confused here. According to my logic, should it be
> arch___test_and_set_bit() and generic___test_and_set_bit()?

Why 3 underscores in a row? I'm clearly not following.

> If you are asking why there is no generic implementation for
> test_and_clear_bit() without the double underscores, the reason is that
> Arm, PPC, and x86 don't use generic code and rely on specific
> instructions for this operation. Therefore, I didn't see much sense in
> providing a generic version of test_and_clear_bit(), at least, for now.

No, there was no question in that direction. And hence ...

>>  Yet the
>> question was: Why then not arch__test_bit(), to match the other arch
>> helpers?
> Because no one uses __test_bit(). Everywhere is used test_bit().

... this seems unrelated (constrained by my earlier lack of following you).

(Later) Wait, maybe I've finally figured it: You use arch__test_and_*()
because those underlie __test_and_*(), but arch_test_bit() because there's
solely test_bit() (same for the generic_* naming). I guess I can accept
that then, despite the slight anomaly you point out, resulting in the
question towards 3 underscores in a row. To clarify, my thinking was more
towards there not possibly being generic forms of test_and_*() (i.e. no
possible set of arch_test_and_*() or generic_test_and_*()), thus using
double inner underscores in arch__test_*() and generic__test_*() to
signify that those are purely internal functions, which aren't supposed to
be called directly.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:06:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:06:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722827.1127153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VBw-0001Sv-Fy; Thu, 16 May 2024 07:06:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722827.1127153; Thu, 16 May 2024 07:06:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VBw-0001So-BB; Thu, 16 May 2024 07:06:16 +0000
Received: by outflank-mailman (input) for mailman id 722827;
 Thu, 16 May 2024 07:06:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Jc0F=MT=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s7VBu-0001Si-Up
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:06:14 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c76db7f8-1352-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 09:06:12 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 4C9C74EE0739;
 Thu, 16 May 2024 09:06:12 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c76db7f8-1352-11ef-b4bb-af5377834399
MIME-Version: 1.0
Date: Thu, 16 May 2024 09:06:12 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [XEN PATCH 2/4] x86/hvm: address violations of MISRA C Rule 20.7
In-Reply-To: <alpine.DEB.2.22.394.2405151618170.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <6d14b3283005cf1a30c4fa24f9841586a41e2b1b.1715757982.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405151618170.2544314@ubuntu-linux-20-04-desktop>
Message-ID: <b78517fce0f119200e07863ce9eac152@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-16 01:18, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that 
>> all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>> 
>> No functional change.
>> 
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> 
>> ---
>>  xen/arch/x86/hvm/mtrr.c             | 2 +-
>>  xen/arch/x86/hvm/rtc.c              | 2 +-
>>  xen/arch/x86/include/asm/hvm/save.h | 2 +-
>>  3 files changed, 3 insertions(+), 3 deletions(-)
>> 
>> diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
>> index 32f74c1db03b..1079851f70ed 100644
>> --- a/xen/arch/x86/hvm/mtrr.c
>> +++ b/xen/arch/x86/hvm/mtrr.c
>> @@ -16,7 +16,7 @@
>>  #include <public/hvm/e820.h>
>> 
>>  /* Get page attribute fields (PAn) from PAT MSR. */
>> -#define pat_cr_2_paf(pat_cr,n)  ((((uint64_t)pat_cr) >> ((n)<<3)) & 
>> 0xff)
>> +#define pat_cr_2_paf(pat_cr, n)  ((((uint64_t)(pat_cr)) >> ((n) << 
>> 3)) & 0xff)
> 
> just a cosmetic change
> 

No. The point here is that pat_cr could in principle be something like 1 
+ 2, which without parentheses may not expand as intended (in this case 
because of the nearby cast). Then there's also the style fix on n

> 
>>  /* Effective mm type lookup table, according to MTRR and PAT. */
>>  static const uint8_t mm_type_tbl[MTRR_NUM_TYPES][X86_NUM_MT] = {
>> diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
>> index 4bb1c7505534..72c7bdbfcd02 100644
>> --- a/xen/arch/x86/hvm/rtc.c
>> +++ b/xen/arch/x86/hvm/rtc.c
>> @@ -45,7 +45,7 @@
>>  #define vrtc_domain(x) (container_of(x, struct pl_time, 
>> vrtc)->domain)
>>  #define vrtc_vcpu(x)   (pt_global_vcpu_target(vrtc_domain(x)))
>>  #define epoch_year     1900
>> -#define get_year(x)    (x + epoch_year)
>> +#define get_year(x)    ((x) + epoch_year)
>> 
>>  enum rtc_mode {
>>     rtc_mode_no_ack,
>> diff --git a/xen/arch/x86/include/asm/hvm/save.h 
>> b/xen/arch/x86/include/asm/hvm/save.h
>> index 8149aa113cb4..ec8de029319d 100644
>> --- a/xen/arch/x86/include/asm/hvm/save.h
>> +++ b/xen/arch/x86/include/asm/hvm/save.h
>> @@ -50,7 +50,7 @@ int _hvm_check_entry(struct hvm_domain_context *h,
>>                            HVM_SAVE_LENGTH(x), true) == 0 )      \
>>      {                                                           \
>>          ptr = &(h)->data[(h)->cur];                             \
>> -        h->cur += HVM_SAVE_LENGTH(x);                           \
>> +        (h)->cur += HVM_SAVE_LENGTH(x);                         \
>>      }                                                           \
>>      ptr; })
>> 
>> --
>> 2.34.1
>> 

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:17:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:17:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722834.1127163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VMY-0003ai-E8; Thu, 16 May 2024 07:17:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722834.1127163; Thu, 16 May 2024 07:17:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VMY-0003ab-9O; Thu, 16 May 2024 07:17:14 +0000
Received: by outflank-mailman (input) for mailman id 722834;
 Thu, 16 May 2024 07:17:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7VMW-0003aV-RL
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:17:12 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5047675a-1354-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 09:17:11 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-56e48d0a632so3835784a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:17:11 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0035sm10127275a12.50.2024.05.16.00.17.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:17:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5047675a-1354-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715843831; x=1716448631; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=W5MeLCkLCvsz71byx6cErlNzGkhmYwaMtdCJstKIW74=;
        b=DqejDtyzNyRx5MMEXqLlZYCqYBVeFxg5Y7x8Dxc5vDIm4RTEdWdcNNNBbp5buZGkZb
         WtCBJS54u6pCq4EVuWoEPJ3fQ69MQ5X8Fz5zykV2nF9ySOLesYyqnrbfz2Qzl7aFYOVG
         X+Kuzcy7/09f2ZwgTVaXlOKUijTxAFqMdkvGxot7qKIPKAnXCtBb1NUZYXJagsuxQlhU
         pAwmLy6bzF1MaP9pLduE3eF7NtT1iwaI17l2pniz/emAAVNOy/SV3gsBQZj3h+FCfNI7
         oK+lQm6GBmQAgIgx4FAaRENaC8Z1SGuWbczUCuAm6MfQ/AvwC2OvwxS27c54vH/Sfk62
         Y9PA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715843831; x=1716448631;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=W5MeLCkLCvsz71byx6cErlNzGkhmYwaMtdCJstKIW74=;
        b=ZI76AjHf9KbhdC/J8s+J7mo0gPI5NMpPSrKylbdGO+c0FyruiiwH56PUY5RgpMdAX4
         1wGkXPMt0d4Lc9haKNGQ+IzwxZkqrxfI2fLFI8RDgBK3rGW+Qqh89QMN2jXaCzt/16vU
         BFGEUHL5xai81zq7gPZKhAorKNZ4J595rUVsfcIIwuM7k1uFssc3Y29WARSAvKY9hx2D
         1bdPGocjp2gfPC7B6IOPctMA/EQu3I4H/wa/E89LSsDziQTH07HBEZOYY0z/deL/xGIq
         X+KnbkdTUrUjRZeY/j7VvTwC+6jyoi1eA0xd7UopRHhodgM6zXpUDUNkj6FLldBQrXdZ
         S7vw==
X-Gm-Message-State: AOJu0YybLy77xpLYFzd7IBM4nDhZPpZoDmyc6Pry6imvPD/iE0roo+Xk
	jjYp3gFBYxBWzArBh9sEurkvuUE7LP2Z2hqTPZGs4tr9UrGPCbYV2FOdT4sSQw==
X-Google-Smtp-Source: AGHT+IEWK7UFczPO82/ECAbemvYreZ2oDBp6W5qew16fQMB6SZaLurhDWlSc5JLtKbsWO3DxdIJbew==
X-Received: by 2002:a50:d615:0:b0:56c:522f:53e1 with SMTP id 4fb4d7f45d1cf-5734d5d13b1mr18833901a12.17.1715843831302;
        Thu, 16 May 2024 00:17:11 -0700 (PDT)
Message-ID: <b50d0a83-fab4-4f59-bf4d-5c5593923f34@suse.com>
Date: Thu, 16 May 2024 09:17:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 01/19] x86: Create per-domain mapping of
 guest_root_pt
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-2-eliasely@amazon.com>
 <dd145c67-8e3e-4b15-94f7-c7cd1f127d45@suse.com>
 <bda3386e-26c5-4efd-b7ad-00f3643523fa@amazon.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <bda3386e-26c5-4efd-b7ad-00f3643523fa@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 20:25, Elias El Yandouzi wrote:
> However, I noticed quite a weird bug while doing some testing. I may 
> need your expertise to find the root cause.

Looks like you've overflowed the dom0 kernel stack, most likely because
of recurring nested exceptions.

> In the case where I have more vCPUs than pCPUs (and let's consider we 
> have one pCPU for two vCPUs), I noticed that I would always get a page 
> fault in dom0 kernel (5.10.0-13-amd64) at the exact same location. I did 
> a bit of investigation but I couldn't come to a clear conclusion. 
> Looking at the stack trace [1], I have the feeling the crash occurs in a 
> loop or a recursive call.
> 
> I tried to identify where the crash occurred using addr2line:
> 
>  > addr2line -e vmlinux-5.10.0-29-amd64 0xffffffff810218a0
> debian/build/build_amd64_none_amd64/arch/x86/xen/mmu_pv.c:880
> 
> It turns out to point on the closing bracket of the function 
> xen_mm_unpin_all()[2].
> 
> I thought the crash could happen while returning from the function in 
> the assembly epilogue but the output of objdump doesn't even show the 
> address.
> 
> The only theory I could think of was that because we only have one pCPU, 
> we may never execute one of the two vCPUs, and never setup the mapping 
> to the guest_root_pt in write_ptbase(), hence the page fault. This is 
> just a random theory, I couldn't find any hint suggesting it would be 
> the case though. Any idea how I could debug this?

I guess you want to instrument Xen enough to catch the top level fault (or
the 2nd from top, depending on where the nesting actually starts) to see
why that happens. Quite likely some guest mapping isn't set up properly.

Jan

> [1] https://pastebin.com/UaGRaV6a
> [2] https://github.com/torvalds/linux/blob/v5.10/arch/x86/xen/mmu_pv.c#L880
> 
> Elias
> 



From xen-devel-bounces@lists.xenproject.org Thu May 16 07:25:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:25:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722836.1127172 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VUu-0005hs-4W; Thu, 16 May 2024 07:25:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722836.1127172; Thu, 16 May 2024 07:25:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VUu-0005hl-1c; Thu, 16 May 2024 07:25:52 +0000
Received: by outflank-mailman (input) for mailman id 722836;
 Thu, 16 May 2024 07:25:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7VUt-0005hf-Eq
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:25:51 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 84c43919-1355-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 09:25:49 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59a9d66a51so252292866b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:25:49 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a60eabd3csm555382066b.108.2024.05.16.00.25.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:25:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84c43919-1355-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715844349; x=1716449149; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=g8CsRgJbNnPgWfuPER+ZlNlEhiVyDxqLDBIYMv25Pyk=;
        b=KEbGIDwcHdIZhTCU+YCSOZvxmrcWwz2qsPRyUqHyHnxT5uR7fuohGySPd2z+MPI0J5
         h3MEXznW0yDIGz8zImr48W/HopqF67w2GVa+vO8844HLzH8u454Y+2V1oBvOWgyOY0QR
         CWBRoYCvjlHudq9JO7B8iavMUGi/uGL4O7VikKIGx8e34LTXlOCZn+ORQcgl8CacW/e4
         AehQw3PeUNnu4UAwOXF5efFcrhxjlkj/AIsXWEBwkY0Ehr4g2tM3nS7NhTb/7kBNBovl
         cPuVRzOHAb4z6sBwCzMUaYp2l1UJpYCeW4wDVv+ZZN8xIxVdqHdxd2sMXcsBlnR0HRGt
         dpGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715844349; x=1716449149;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=g8CsRgJbNnPgWfuPER+ZlNlEhiVyDxqLDBIYMv25Pyk=;
        b=vfystCRaCdqkKsWowEC8GgK9kFi7074WBtPXOLr5qwMQr+hcOU1q7GZ14f4jASsquq
         mRo96dXluqarsYVWcRsXQat+MPm1SCGi2e8QgfSW9ttumxFxlMJuIdcXP2bmg7/ycHjN
         mD/MYsv1P1lYHt482f5MaoEu9KvCVDpqZRVagGeNERal1vL+4bSSDOzOhhqrKmZPZZxk
         q+uPjojstOqxuvS1MZjDXaAMp3q2MZFk64eaUXfYcf7V4ZyhQB07jL4xzfUFi9MoaMIn
         Skqdx7LT0J8OQ/Fa3iqcxhTz0kTl6BasZp/q5kPgmC4bpIzKJqShztrvV0W8XBP5G2uF
         x7+w==
X-Gm-Message-State: AOJu0YxsMCkvPBWvIkORFOYkLs54FHgbdE2ELuUyiVuqSHDooyAeTQWV
	kmZXbtDUWbKIN2675ItGLd9ZXNBpIFmH1wNLpSE48kYTbO/tnt90JFo1mx7fZw==
X-Google-Smtp-Source: AGHT+IFxMeQfNqT7Ee98nTqgWi+Po6xmaZntUn8ovYs+6HgoJELrGV+J2wAvO5PawlNgP+rC71iCew==
X-Received: by 2002:a17:906:d552:b0:a59:9c4d:da3c with SMTP id a640c23a62f3a-a5a2d5cca61mr1385266766b.40.1715844348605;
        Thu, 16 May 2024 00:25:48 -0700 (PDT)
Message-ID: <02449ca7-27c6-4fcf-ad74-0e570307f17b@suse.com>
Date: Thu, 16 May 2024 09:25:47 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 08/15] x86/vpmu: guard vmx/svm calls with
 cpu_has_{vmx,svm}
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151740180.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151740180.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:44, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> If VMX/SVM disabled in the build, we may still want to have vPMU drivers for
>> PV guests. Yet some calls to vmx/svm-related routines needs to be guarded then.
>>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> 
> Question to the x86 maintainers: are we sure we want to support the case
> where VMX/SVM is disabled in the build but still we want to run PV
> guests with vPMU?
> 
> If the question is not, could we simplify this simply by making vpmu_amd
> dependent on CONFIG_SVM and vpmu_intel dependent on CONFIG_VMX?
> 
> I realize that it is possible and technically correct to disable
> CONFIG_SVM (or VMX) to run on AMD hardware (or Intel) with plain PV
> guests only. But do we want to support it? I wonder if we could make
> things easier by avoiding to support this configuration until somebody
> asks for it.

I think we want to allow for such a configuration; whether that's deemed
a supported one is an orthogonal question. Much like you can set PV=n and
HVM=n at the same time, yielding a largely useless hypervisor (where
perhaps even the question of whether it's support may raise eyebrows).

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:32:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:32:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722840.1127182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vb1-0007om-Pv; Thu, 16 May 2024 07:32:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722840.1127182; Thu, 16 May 2024 07:32:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vb1-0007of-Ma; Thu, 16 May 2024 07:32:11 +0000
Received: by outflank-mailman (input) for mailman id 722840;
 Thu, 16 May 2024 07:32:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Vb0-0007oG-TT
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:32:10 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 670981c6-1356-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 09:32:08 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-51f40b5e059so597239e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:32:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5734d4cf324sm9367897a12.24.2024.05.16.00.32.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:32:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 670981c6-1356-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715844728; x=1716449528; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=K6J6MYFMpo/ba/j/SxvlVyzLvMneb53SH9bFhZT4yIw=;
        b=QxFRud024Abx/uUmhAOP3K6eqjT+znQRfc71lwqX19AZA2+yLCyNFGWYUmmOyoadJS
         eOQ6YGJxZHLuJijB8Y6IX35vWzfSoPnAX0qMCk+9I7IJ3Sv0ebreC44ZZZpyGdemamYB
         sTblA4Duzr5d7USIRZCMscCENd2OoRX+uonDkOkw3uzY91Hbjz+Zxxgsa+Yf6GFC8+LN
         NYb4t4VDnVwZ4PxupENXd2gfMRqYtQSEm0D87KrAwTNtDlJZ0y+GSCSGe2NzsWw6kXbx
         heCdq39SlJoQO4IkJM4xA7j4COqBxqZMsDUKzibrDU1e4Etvo1ZcUEFB/XWGxHy+TkWk
         Oz8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715844728; x=1716449528;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=K6J6MYFMpo/ba/j/SxvlVyzLvMneb53SH9bFhZT4yIw=;
        b=NlfUWxsfR4U+CLqhddDJxFVXqxvm9ZAM/3FVBUcWeZbhLIJFun3EsDZUf49s5eobSW
         go5ClyYB3dSBGlQUn/D7d61Pzg9WUjQKcHumr6ZWw6SpMOEyBF59fieA/XwAqRGdebaU
         FwbFnS256pqwT96TzZ74G5JsWZiLstX8RjJZM8p273kRTVJF/CPO6zTEQH4o71aqGHW/
         UmIAn16Dm74TOYvF4MTvl4SWjaYc2mDzGbXfWbf2oR8QhkaD11FCS4pj2jQw8TDiaVPZ
         oVuP7x+R4PhsoO/CfIr5q3kFKsoRo36d/D4i5lrADhLjAQDScOkWvsrG+REOuxAFEVSV
         nOyw==
X-Gm-Message-State: AOJu0YxwKHSUWnUQnPRvnPgncUqqlq0JH5J/hPvQzi9AVX06Zb3GMdCu
	9PmSke7iCpSqZdFI56ezhvNlJcv7G//i3z+EOSSkJYPm3U20vRvAZS0qW5o4tA==
X-Google-Smtp-Source: AGHT+IFMQvrr4tsJQRejO7bHoPNJVPUa+rldU1D7kE7VI2rMLjkAQdTAiwEOcXtyxzjNO3XScG4rCQ==
X-Received: by 2002:a19:9119:0:b0:51b:afe0:3f4c with SMTP id 2adb3069b0e04-5220ff70bafmr11345373e87.48.1715844728440;
        Thu, 16 May 2024 00:32:08 -0700 (PDT)
Message-ID: <4f2c1486-1b58-4c8d-a74e-ad83bf9cc4c7@suse.com>
Date: Thu, 16 May 2024 09:32:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 12/15] x86/vmx: guard access to cpu_has_vmx_* in
 common code
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151748080.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151748080.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:50, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> There're several places in common code, outside of arch/x86/hvm/vmx,
>> where cpu_has_vmx_* get accessed without checking if VMX present first.
>> We may want to guard these macros, as they read global variables defined
>> inside vmx-specific files -- so VMX can be made optional later on.
>>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
>> CC: Andrew Cooper <andrew.cooper3@citrix.com>
>> CC: Jan Beulich <jbeulich@suse.com>
>> ---
>> Here I've tried a different approach from prev.patches [1,2] -- instead of
>> modifying whole set of cpu_has_{svm/vmx}_* macros, we can:
>>  1) do not touch SVM part at all, because just as Andrew pointed out they're
>> used inside arch/x86/hvm/svm only.
>>  2) track several places in common code where cpu_has_vmx_* features are
>> checked out and guard them using cpu_has_vmx condition
>>  3) two of cpu_has_vmx_* macros being used in common code are checked in a bit
>> more tricky way, so instead of making complex conditionals even more complicated,
>> we can instead integrate cpu_has_vmx condition inside these two macros.
>>
>> This patch aims to replace [1,2] from v1 series by doing steps above.
>>
>>  1. https://lore.kernel.org/xen-devel/20240416064402.3469959-1-Sergiy_Kibrik@epam.com/
>>  2. https://lore.kernel.org/xen-devel/20240416064606.3470052-1-Sergiy_Kibrik@epam.com/
> 
> I am missing some of the previous discussions but why can't we just fix
> all of the cpu_has_vmx_* #defines in vmcs.h to also check for
> cpu_has_vmx?
> 
> That seems easier and simpler than to add add-hoc checks at the invocations?

I'd like to take the question on step further: Following 0b5f149338e3
("x86/HVM: hide SVM/VMX when their enabling is prohibited by firmware"),
is this change needed at all? IOW is there a path left where cpu_has_vmx
may be false, by any cpu_has_vmx_* may still yield true?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:37:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:37:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722847.1127192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VgJ-0008RK-CR; Thu, 16 May 2024 07:37:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722847.1127192; Thu, 16 May 2024 07:37:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7VgJ-0008RD-9T; Thu, 16 May 2024 07:37:39 +0000
Received: by outflank-mailman (input) for mailman id 722847;
 Thu, 16 May 2024 07:37:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7VgI-0008R7-F8
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:37:38 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ab10a1c-1357-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 09:37:37 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-52192578b95so519270e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:37:37 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574fdb36a35sm1485104a12.17.2024.05.16.00.37.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:37:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ab10a1c-1357-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715845057; x=1716449857; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xRrzT3j6noehZdxpl39UjpRiiFrlamncfeu4/Bf4T7s=;
        b=CacvMaCf3cZQGbFQH7ZzEGydTd/1ZC/FaMBuIcK4JSHytyCDddJ4huDv3ho3ymROoq
         2P4VMTJHHg29yJ9CBQNvnqkL2nshzry0cnZUbuETA3FyyHjBlGbVv2I+0ITG0h0PdFeo
         WWS6oAcrP5URf70fyw285J8qekTdlh3GVKqbL+9xC6KA0L+gz9H+qer4Ab87rMjEW8KM
         yn9HYqWcc+2Fss1JfyRa9uUUA0YCQQyn8Z2oBdA9oSbFdA8lZyF0KrOH574y6EmSsPhn
         XDxMRqSpsbCtt0zjpEkANfw9TydxpOkMJPInUpWbtWILg/qRMYInysT4I8ma48fjwB1S
         LhaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715845057; x=1716449857;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xRrzT3j6noehZdxpl39UjpRiiFrlamncfeu4/Bf4T7s=;
        b=HgZgyALK+AZBvrn0Zt74gBxLdsZU3wf3KVMLlxAi1Z722sAUYvFypEiwrWx7tDQN9l
         tRkBrjIQCFXuviuN/Yb73wDUa7dS/ke9nT8KfpqaPLtQgnDQwFg/jRr4Z6AxzGrMVOGr
         1AeZ9RLq0vA2WQMh/qNm6WUHL9522UmjKSFZ4qOM+Oz4TWvLDNww2Dsqspz4i1ZHfqVE
         fUm6YIz20K4ujG7is00NAVYuWoI/ppKhHLRb2qPXvt/9wbpMCbxHaYYWd9SVnd87OKhC
         xRHtVp/OkhL0MKADGfVcRf87Ox2mrldwRfG2srWHUpPH3BAa9wIZD/u9WN/hKZjmITvF
         RnrQ==
X-Gm-Message-State: AOJu0YySCxlXrsFe0drxKZoMFNwf5iCabkXLZtBP8l7VEx8NExCCDLNm
	pQvh2MhX3OZKbgpWiwI82ViC1dxLHpoWB6+2ZW3S6XN6P6s3xOsK/CGXqHPpDQ==
X-Google-Smtp-Source: AGHT+IFoFlwUS/wE8TL76Cs1xUFfqKJYTFmDZ4VMN0u5ZvHgBd7616VFX2j0mX366ohukzrnqNVJKg==
X-Received: by 2002:a05:6512:1247:b0:51e:e703:c878 with SMTP id 2adb3069b0e04-5220fa71de0mr16168959e87.14.1715845056723;
        Thu, 16 May 2024 00:37:36 -0700 (PDT)
Message-ID: <3104deae-0d27-438e-83ea-035a7786fd4a@suse.com>
Date: Thu, 16 May 2024 09:37:35 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 14/15] iommu/vt-d: guard vmx_pi_hooks_* calls with
 cpu_has_vmx
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151751530.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151751530.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:54, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> VMX posted interrupts support can now be excluded from x86 build along with
>> VMX code itself, but still we may want to keep the possibility to use
>> VT-d IOMMU driver in non-HVM setups.
>> So we guard vmx_pi_hooks_{assign/deassign} with some checks for such a case.
>>
>> No functional change intended here.
>>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> 
> I know that Andrew was keep on having a separate Kconfig option for
> VT-D, separate from VMX. But still, couldn't we make the VT-D Kconfig
> option depending on CONFIG_VMX?
> 
> To me, VT-D should require VMX, without VMX it should not be possible to
> enable VT-D.
> 
> This comment goes in the same direction of my previous comment regarding
> the vpmu: we are trying to make things more configurable and flexible
> and that's good, but we don't necessary need to make all possible
> combination work. VT-D without VMX is another one of those combination
> that I would only enable after a customer asks.

Well. Imo again the configuration should be permitted. VMX and INTEL_IOMMU
ought to be default to INTEL, but permit being turned on/off in all cases.
(That's btw part of the reason why I continue to be unhappy with it being
INTEL where really INTEL_CPU was meant. If what is INTEL now would be
INTEL_CPU, INTEL could be an umbrella option for all three, or two if we
were to tie VMX to INTEL_CPU.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:41:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:41:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722852.1127201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vjj-0002Ch-TB; Thu, 16 May 2024 07:41:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722852.1127201; Thu, 16 May 2024 07:41:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vjj-0002Ca-QC; Thu, 16 May 2024 07:41:11 +0000
Received: by outflank-mailman (input) for mailman id 722852;
 Thu, 16 May 2024 07:41:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Vji-0002CE-Dw
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:41:10 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a71da56e-1357-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 09:41:06 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-574d1a1c36aso2985700a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:41:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a5ecd37a3sm570486466b.79.2024.05.16.00.41.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:41:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a71da56e-1357-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715845265; x=1716450065; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UUi9nDgpZQHSi7udgEBCMlqbXC6UNa3fLlCLL5Lanow=;
        b=R9ph+h7nAz84KPph13hAEUhrsERhhJmwYfYeOdgFc2vWyg1iKenxF0GitFU0M5qyP2
         hrjoOG6x48kRzCilDClaA8V4fquVrgnm/2mzB+CX65V2HRbyEw4JPPiyt5KOEdFIHLYj
         QXHdkrljlflkRe8lGdCC+GqAHklEA1P2zysT73yy5i1baV4c3k4DzTiKPiwXIuw6D1jv
         Ldg+NdtnpJa+SObFDrasseMTU88QlOapLP9Z4Bs5A346+dnyaol7mv5hI/v6saEAH9/y
         nVd5FfIV0GxS+3FWVUo/iaKqd83CmD+x441wPanG0wIRCUMfmKSiUSRxBxzJjxkhhGJo
         DXBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715845265; x=1716450065;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UUi9nDgpZQHSi7udgEBCMlqbXC6UNa3fLlCLL5Lanow=;
        b=eR7V7XUEMsAo1faJTB0mY6nq5luWG3Kbtmhtq5YvpXDW+JNGY6I8pAoj97kLB5zg0w
         rBpHIZvAwh0UcccZb99putq2IF0/uaARE9Re6ho8SdjfepKc5sR8XaCYgI3mwE0Hp3OJ
         D0HKuG4RHmJblQ1g1hEpY3PXpVkzuaAVmvyfdiKjUjHQweXdj40xb9qkbQUcIsXjKcjU
         Ry9TvWt76/crnRCRqGyxqx864X1QN1wH07OG2e8Js0OYo+1VhpV5VGRwWdaB9esSoMxB
         H6QDGIF6nO4mJHxsJCjeKtrdEfQrzkDPJs3+3nsQpr3JAIoIpjymaiLqkhrHSL9F2UbU
         bkBA==
X-Forwarded-Encrypted: i=1; AJvYcCVpHoXnqeg1t3peYhh+BP3eYWValY/IqIXzulZhM4FzN70m1PKc5CxXMa7xJsjgEi24lAM6YSY7XVAc9z/OhJLUJEdy+cFKoSj8iFNMZpM=
X-Gm-Message-State: AOJu0YygJg42dXvbGBEER1+tmVEHrxMZXjCQnrCo3p+YIKiFUxcNG7k0
	3KoJLP9E7SS3CJm7z/acHJIS9ce7exPfZ8ZbbhlG9sm7803sD/zrcQ7nZFJuyQ==
X-Google-Smtp-Source: AGHT+IEJgmyZ46oMLxX2hOixtDzwhjsLAOPvaUbr+EVnBftqqWJJ2K2w0cGLOLjvy62ej/Pny2H/Pg==
X-Received: by 2002:a17:906:8d7:b0:a59:a0c1:2624 with SMTP id a640c23a62f3a-a5a2d5cb855mr1190695266b.39.1715845265485;
        Thu, 16 May 2024 00:41:05 -0700 (PDT)
Message-ID: <64083e01-edf1-4395-a9d7-82e82d220de7@suse.com>
Date: Thu, 16 May 2024 09:41:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] hotplug: Restore block-tap phy compatibility
Content-Language: en-US
To: Jason Andryuk <jandryuk@gmail.com>
Cc: Jason Andryuk <jason.andryuk@amd.com>,
 Anthony PERARD <anthony@xenproject.org>, xen-devel@lists.xenproject.org
References: <20240516022212.5034-1-jandryuk@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516022212.5034-1-jandryuk@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 04:22, Jason Andryuk wrote:
> From: Jason Andryuk <jason.andryuk@amd.com>
> 
> From: Jason Andryuk <jason.andryuk@amd.com>

Two identical From: (also in another patch of yours, while in yet another one
you have two _different_ ones, when only one will survive into the eventual
commit anyway)?

> backendtype=phy using the blktap kernel module needs to use write_dev,
> but tapback can't support that.  tapback should perform better, but make
> the script compatible with the old kernel module again.
> 
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

Should there be a Fixes: tag here?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 07:46:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 07:46:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722857.1127211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vp4-0002oL-FB; Thu, 16 May 2024 07:46:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722857.1127211; Thu, 16 May 2024 07:46:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Vp4-0002oE-CS; Thu, 16 May 2024 07:46:42 +0000
Received: by outflank-mailman (input) for mailman id 722857;
 Thu, 16 May 2024 07:46:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Vp2-0002ni-DJ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 07:46:40 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6debf468-1358-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 09:46:39 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-574ea5184abso3188069a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 00:46:39 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d6dsm954840066b.71.2024.05.16.00.46.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 00:46:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6debf468-1358-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715845599; x=1716450399; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=3HfXHcoIHBOaBUN7lKT3rSNX7UxonJ/JKyL/obFsIHg=;
        b=WvaDC6RpNdhz3nxE577v5bDr517Ab7+/V4IjJxntATu0U2yCDuXYg6TkH+fnNdoydT
         nWDhQt//DI9seBaSl5aq4fBoXRsM7oQwMu/WIl7DSGoMnUyFG1HCf5opVfEyx0cA3z+P
         WNROwDoNDsVifjZEEPIAOWzbXHxnoQYNt3EMHtNf3TxSFRd/2qcRtpHRiHfvFOSv8YWb
         NTTcTPc1556iYiKvWLE7fDTD/GA9Tp+59yqUqaA4LzZnl6cZAT0ELuQUEHNMNDEzLqfz
         GLTQViu8iMOTTK92oqTeLVH4NjTACyX/tYHnKw1MrggwOrssV1GR5LfAydoOzQQzjptG
         D9+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715845599; x=1716450399;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3HfXHcoIHBOaBUN7lKT3rSNX7UxonJ/JKyL/obFsIHg=;
        b=L8xt9PQAZrnx0lKYZ4k8l6M03DAIGTzYzlPRgS9Gp/ipxG7VZ9AJoOozbZEjaD5Wgs
         O7dKl+zuk7AfAyzNQ4A2U7BSuzyskjUL/Cq5n1NTi/+uzsR67pcqT5Yto1AQ/2OYHNk1
         vrxlxYBTDcBfnORGXhUv6rwPZ3/nSZk4iKs1QEQXaPjKh2n+hlRdSRQpjTBo21tyAVuA
         7FhCMhWfNv6ksjJiUcxTbb3M2E2l7BlyfgZ0eehdOJZCLy9Pt8vGVznUKXWfcTYp2m2r
         790d7I5I0gst7amjiDNwhG2JD695+AyDLVGIPfVzDGybvvt17dt+1fjDVJMunJWEBowW
         nHnQ==
X-Forwarded-Encrypted: i=1; AJvYcCXTZTVJ5ueGlAzgAb/JaVR/tcl3A1V0FGS8q4lCZB7oHCSlusTiZShwmxXIO9KC8b0psp6Ih8PuEhTK6IFeo6sqf3d123W/biQLOyMuI5U=
X-Gm-Message-State: AOJu0YyWvqoX/u9FY4B+U2GPYDJKqiOIBj0Kgs2LRpFr1m7/9wW9PtUe
	bLus8ffSn7VgPqRwRG/dc/0PrVXg9E/fIZsYBStnE3B+qHljGtS4L/slkKYMlA==
X-Google-Smtp-Source: AGHT+IEA+/gi17v0zCRWRsyrYohJbaA4otgoSd22aQEVGKiaAOb5smdmPYGDsR2l67ngFOkBguSxMg==
X-Received: by 2002:a17:906:e943:b0:a5a:8a63:9fee with SMTP id a640c23a62f3a-a5a8a63a051mr447767366b.3.1715845599035;
        Thu, 16 May 2024 00:46:39 -0700 (PDT)
Message-ID: <6d83a588-c190-4f86-92d3-da48c929563f@suse.com>
Date: Thu, 16 May 2024 09:46:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/sched: set all sched_resource data inside locked
 region for new cpu
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>, Juergen Gross <jgross@suse.com>,
 xen-devel@lists.xenproject.org
References: <20240515152539.18714-1-jgross@suse.com>
 <b04a9f8c-aed4-4672-a97a-22e5bdec51e5@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b04a9f8c-aed4-4672-a97a-22e5bdec51e5@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.05.2024 17:27, Andrew Cooper wrote:
> On 15/05/2024 4:25 pm, Juergen Gross wrote:
>> When adding a cpu to a scheduler, set all data items of struct
>> sched_resource inside the locked region, as otherwise a race might
>> happen (e.g. when trying to access the cpupool of the cpu):
>>
>> (XEN) ----[ Xen-4.19.0-1-d  x86_64  debug=y  Tainted:     H  ]----
>> (XEN) CPU:    45
>> (XEN) RIP:    e008:[<ffff82d040244cbf>]
>> common/sched/credit.c#csched_load_balance+0x41/0x877
>> (XEN) RFLAGS: 0000000000010092   CONTEXT: hypervisor
>> (XEN) rax: ffff82d040981618   rbx: ffff82d040981618   rcx:
>> 0000000000000000
>> (XEN) rdx: 0000003ff68cd000   rsi: 000000000000002d   rdi:
>> ffff83103723d450
>> (XEN) rbp: ffff83207caa7d48   rsp: ffff83207caa7b98   r8:
>> 0000000000000000
>> (XEN) r9:  ffff831037253cf0   r10: ffff83103767c3f0   r11:
>> 0000000000000009
>> (XEN) r12: ffff831037237990   r13: ffff831037237990   r14:
>> ffff831037253720
>> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
>> 0000000000f526e0
>> (XEN) cr3: 000000005bc2f000   cr2: 0000000000000010
>> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss:
>> 0000000000000000
>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>> (XEN) Xen code around <ffff82d040244cbf>
>> (common/sched/credit.c#csched_load_balance+0x41/0x877):
>> (XEN)  48 8b 0c 10 48 8b 49 08 <48> 8b 79 10 48 89 bd b8 fe ff ff 49
>> 8b 4e 28 48
>> <snip>
>> (XEN) Xen call trace:
>> (XEN)    [<ffff82d040244cbf>] R
>> common/sched/credit.c#csched_load_balance+0x41/0x877
>> (XEN)    [<ffff82d040245a18>] F
>> common/sched/credit.c#csched_schedule+0x36a/0x69f
>> (XEN)    [<ffff82d040252644>] F common/sched/core.c#do_schedule+0xe8/0x433
>> (XEN)    [<ffff82d0402572dd>] F common/sched/core.c#schedule+0x2e5/0x2f9
>> (XEN)    [<ffff82d040232f35>] F common/softirq.c#__do_softirq+0x94/0xbe
>> (XEN)    [<ffff82d040232fc8>] F do_softirq+0x13/0x15
>> (XEN)    [<ffff82d0403075ef>] F arch/x86/domain.c#idle_loop+0x92/0xe6
>>
>> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Fixes: a8c6c623192e ("sched: clarify use cases of schedule_cpu_switch()")
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Tested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Thankyou for the quick turnaround.  I'll fix the wrapping of the commit
> message on commit.  Not sure what went wrong on my original outgoing email.

Didn't you rush this in a little too fast? No matter how obvious it might seem,
there was no sched/ maintainer ack so far ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 08:12:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 08:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722871.1127221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WDm-0000uo-W4; Thu, 16 May 2024 08:12:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722871.1127221; Thu, 16 May 2024 08:12:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WDm-0000uh-Ta; Thu, 16 May 2024 08:12:14 +0000
Received: by outflank-mailman (input) for mailman id 722871;
 Thu, 16 May 2024 08:12:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7WDl-0000ub-N9
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 08:12:13 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fefd52f9-135b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 10:12:11 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5709cb80b03so3204515a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 01:12:11 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574ed753a6dsm2720087a12.77.2024.05.16.01.12.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 01:12:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fefd52f9-135b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715847131; x=1716451931; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cQI8KB3NBiKLrVA543O5ijA8MSOfQOypRYx1X/hQDOw=;
        b=Tf+YdUMjYIN6f790xc1JUDZQfagwO03aYmc4z56tNrjGwINHRksLaAOceV9amP5O+p
         P5qdoLtoAalAacIdB7B2A5wLG3pUWzphKDGb4npZ2ezlzIt73aMUADbAKGaqrznR15aw
         HfAZaoE/S+IMWXVwetMzanCBzzDVmIuWGCooW2nrW9moaYQ6webDqIE6OKqxlXQyVuXo
         nn9kdYcHjp6QWXoZecl6RW9zZ/FSIY9DSnWh7JaKc1nQDgxL2eoLhqdcPGqVnmqw/lOV
         M17ZEy7OHj+lms3nSgVAv8NO36pe3wkiE9BBRWI4RykczdDBTiIdS+4B5WNH+ACGFzTn
         Z4Hw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715847131; x=1716451931;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cQI8KB3NBiKLrVA543O5ijA8MSOfQOypRYx1X/hQDOw=;
        b=U8yO2RXFnjd7fJSniBEs2Vlw0UzicPqhb586bVxM2J160QL0N+1C8cemo1BYc2F6QI
         Y0rwocsBKSgCzXopqQ+NvXDc1Uc+axaMx6/OmuCJWgfuCcBaVsXIRagiCtTFLmsFi25C
         MWtbcoWSYqX9j5WgUbOLb+AEaiQZN2odLA+Nh/+hNhSL6VFesvrD62A94/ZsFfM2FqY/
         wk87eBDw2XzcDPfZBvJyZ7S6l7fzKmf0Lp0Jh/GuImDkFu+lnggLyB2SnFwhMY1+rfVa
         i8IWMNnu9dsQR41BWD1bzoaVZemcWA3wFPan8nCLYvKle9xR51iP0Mcy6l0HrxaFjiX/
         KAfA==
X-Forwarded-Encrypted: i=1; AJvYcCUui/7X3xJHX/0/fSkQ2MINXm/2K6I3sBsCcdiLiaBqgHFHjNWUNTAYWn3sfkX3o4yTz1avII/0ttW4PL/f4WS6NJ3fWUR4WU3ISMS+zvc=
X-Gm-Message-State: AOJu0YxPvib1i0xRRC5QaKBg5F0z4oViiVplN1vz3Dho9JmtB0v9YBxO
	0SqehAmS5gyaPF+9Dg733is857qnONTZo2iZU3ahDkIRUrvmZiYXGu6f7asSHg==
X-Google-Smtp-Source: AGHT+IFUEAT2Un9IUPbpLOVq2Z4f+3YXh6qVoe6kvgW8YHG8txZ7v4UnkYXJiGy7wh2K19FIOrBDyg==
X-Received: by 2002:a50:9505:0:b0:572:6846:b899 with SMTP id 4fb4d7f45d1cf-5734d6b407dmr11630908a12.41.1715847130898;
        Thu, 16 May 2024 01:12:10 -0700 (PDT)
Message-ID: <e32accc2-1391-471e-afd1-e67dc681105b@suse.com>
Date: Thu, 16 May 2024 10:12:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] include/ctype.h: fix MISRA R10.2 violation
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Julien Grall <julien@xen.org>,
 nicola.vetrini@bugseng.com, consulting@bugseng.com,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405151549020.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151549020.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 00:52, Stefano Stabellini wrote:
> The value returned by __toupper is used in arithmetic operations causing
> MISRA C 10.2 violations. Cast to plain char in the toupper macro. Also
> do the same in tolower for consistency.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu May 16 08:16:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 08:16:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722874.1127232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WHP-0001SR-FO; Thu, 16 May 2024 08:15:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722874.1127232; Thu, 16 May 2024 08:15:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WHP-0001SK-Cg; Thu, 16 May 2024 08:15:59 +0000
Received: by outflank-mailman (input) for mailman id 722874;
 Thu, 16 May 2024 08:15:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7WHN-0001SA-Vt; Thu, 16 May 2024 08:15:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7WHN-00070v-TK; Thu, 16 May 2024 08:15:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7WHN-0003vP-II; Thu, 16 May 2024 08:15:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7WHN-0000NZ-Hr; Thu, 16 May 2024 08:15:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=usxY3pD6djznybJNOKCZoQwn472bgcjyASXElZ0v7GA=; b=qVTKo4Xv+Y1yB1+ok0BpxPrn9n
	Z2R18CsKzROPVhFQhiATjhCKHOH1kye8bLjhGlQG0V/gBwIQONAqPsLGX4mh6exncQOqtPADLGowl
	bo4AS4EE/97kDr2BlLPezAQt+AMw7RKsM28LBy0dUmqVypms8jquEslgLPVKydfjhsLQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186007-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186007: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
X-Osstest-Versions-That:
    xen=319a5125ca2649e6eb95670b4d721260025c187d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 08:15:57 +0000

flight 186007 xen-unstable real [real]
flight 186012 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186007/
http://logs.test-lab.xenproject.org/osstest/logs/186012/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 186012-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 186004
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186012 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186012 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186004
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186004
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186004
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186004
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186004
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
baseline version:
 xen                  319a5125ca2649e6eb95670b4d721260025c187d

Last test of basis   186004  2024-05-15 08:40:36 Z    0 days
Testing same since   186007  2024-05-15 23:08:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   319a5125ca..abd00b037d  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4 -> master


From xen-devel-bounces@lists.xenproject.org Thu May 16 08:31:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 08:31:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722896.1127285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WVz-0005x6-Dk; Thu, 16 May 2024 08:31:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722896.1127285; Thu, 16 May 2024 08:31:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7WVz-0005wz-Aw; Thu, 16 May 2024 08:31:03 +0000
Received: by outflank-mailman (input) for mailman id 722896;
 Thu, 16 May 2024 08:31:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7WVy-0005vF-OG
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 08:31:02 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0c052d0-135e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 10:31:01 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-56e56ee8d5cso3372726a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 01:31:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfe87sm10168552a12.53.2024.05.16.01.31.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 01:31:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0c052d0-135e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715848261; x=1716453061; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=d1IoVMqRJbAXFb72224xcMh9cMkSitNu9e/WI/HTkxo=;
        b=Bnpv2jy2HR87b9E9vdPoQ1EnMZmW5US9lBR/Ywe5Sy3LhTRy6HGSz2N808G542/T68
         6MGzAJMCKdQkq05rUxXdfz1/t0Q8kHHcQAo80ONjrs0SjUMq+5AFJ9XQ2hcDqUnhidwF
         sclAf6bKmrQn7yaiI9VyLE/dOWoNQrXd40o/0miFdRmeso5iY80lbzL/yWHmCn1/YbeP
         +Gw91EMGdQp/8PNFP8eNxCdW8/WpNIkx/50O+SxfdKclGUpjpYcK2/1yxxTtMd0JuY6Q
         xREwksdHoM5LvMkqncI3bMVkGwhdongJygpM8F5c+ru068OiakfSnM8L900WetTRXDva
         RWlA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715848261; x=1716453061;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=d1IoVMqRJbAXFb72224xcMh9cMkSitNu9e/WI/HTkxo=;
        b=Etkx+nFp13Mpwtxpf4WUBg8Rw53HiPhJv+SjNmGiaeSUiup9bLBZY8Fci9iK0lqzmh
         U/UxhfVLS7AklQUzSfy2MY4pwrRwbdUiFY9AM0EbOzS4C61CMijUVP37mu5n8QiVTLqu
         kiT7izqqSnVTh6vMmbbCgWniRtmDClcmMoVEeogX08XVdyAWgpOLZgzwSWMjov9drW69
         EqS2F93JrO7Qr9Uif0zFRqsFvGgMg8myZ8qEUMPST3NKuOK+kWBfScHGyfDUQQHRfVqD
         ASlSjE/PhUfzEhzUeC8VjY50JhijrvipGxjKaFAoJhTejYk7FhLfSIk31y6/c3Kvm+Vy
         QIVg==
X-Gm-Message-State: AOJu0Yy07UjftVeqfthx4JFtbrO7md86IQV+Hhw2wkAvg8fZyk8tiFhe
	82eG6J3QBc5Zninlgfh3siFdQGemIiu9p8viBN+Tzhq9DtaW6D62wMSnO1Lae6T8GwfjM01A3iA
	=
X-Google-Smtp-Source: AGHT+IHgkjlAr7Sa2xPuqTiLDfOw2PQ281z/X8UFlOghtzZ1Tk+UzPwI1hg8VDypTxkhL0i4VH+lBg==
X-Received: by 2002:a50:c311:0:b0:574:ebba:8f95 with SMTP id 4fb4d7f45d1cf-574ebba9061mr4427817a12.24.1715848261280;
        Thu, 16 May 2024 01:31:01 -0700 (PDT)
Message-ID: <209eb68a-6bd4-4969-bccc-94a1224cffa3@suse.com>
Date: Thu, 16 May 2024 10:30:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.17.4
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Julien Grall <julien@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>
References: <b726b105-9204-4a72-8fbc-ceaa8e74f3ec@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b726b105-9204-4a72-8fbc-ceaa8e74f3ec@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.03.2024 13:23, Jan Beulich wrote:
> the release is due in two to three weeks. Please point out backports you find
> missing from the respective staging branch, but which you consider relevant.
> 
> Note that this is going to be the last Xen Project coordinated ordinary stable
> release from this branch; the branch will move into security-only support mode
> afterwards.

Noticing only now: This was premature; there ought to be a regular 4.17.5, as
full support extends up until early June. I have no idea where I was looking
when deciding to write the above.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 08:57:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 08:57:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722900.1127295 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Wva-0001ok-Dy; Thu, 16 May 2024 08:57:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722900.1127295; Thu, 16 May 2024 08:57:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Wva-0001od-BS; Thu, 16 May 2024 08:57:30 +0000
Received: by outflank-mailman (input) for mailman id 722900;
 Thu, 16 May 2024 08:57:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7WvZ-0001oX-An
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 08:57:29 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 523d398f-1362-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 10:57:28 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59cdd185b9so398758566b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 01:57:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdc72d31fsm121865766b.169.2024.05.16.01.57.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 01:57:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 523d398f-1362-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715849847; x=1716454647; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dNC6vf8/COs9uc/pu94tL09PzkEQMKSrQx7KSdEb/UY=;
        b=a6T1kS8vyvbHm02NHUTUJsTsfJQfKJHFfWAClraRk1k458SIPZ7RQAJHJUjRi89PqO
         hooXi5QJ6yXCopEC9OCFo9ycOVxDA6zQVkEd19Z4V1GGQK6KR0HzmVdhpG4T0jR5XLfZ
         /SGKYzjzP2+d0CYUtoO9VLKyy192Kk/+NOt9lS3Xg1JU5N2HKgiXHd/m/bMCbaluAvIo
         4GKOWz4TfjSDHtecqg8EScIWaxygr0fPyGjSW0E4LAi25k18ofBy1R+cgPYH55ZfUl9G
         6ILMruRIoKIi2riudIha3Vgc7bfkTp3ixxwYymFzZ3ZimCOKRDVN5ylkq5APGNyvX74Q
         7dAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715849847; x=1716454647;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dNC6vf8/COs9uc/pu94tL09PzkEQMKSrQx7KSdEb/UY=;
        b=TkzlD8Ud7Sde/cNNAed213ihww28I3/mq50I0/jg6F3xVPT6b0p5Elr8/g/i2DDfZm
         vi+t/2Rz6J+PBDjdgdQ9YIzVp0dy/t+LbivhVzI40kq4JEIexNNk7781911qCY7uvXSb
         ublpfur5/4PvCUNqjCsF0dutHxDS1wGKpywnRolJasjQb3TqYd5HAmMbdOhBWoeEoAhk
         MXBep+TDie3WIH2IlfBlhXbqo6o7iKFYiPretpsFsVYhe++rRIO0f2MVvDW1La0qawqP
         7MD/aQVwen1dr30rIVzVvYPpqLoi6dZHQn9bHoU4Sj1GTKxk2ZsD+JQKBfH9+cVLCaFW
         SPsw==
X-Forwarded-Encrypted: i=1; AJvYcCWO9so7Npzg2BvWCMvwV0jASnOVKBwto5812BU9ysvKagXf44vs2O4VbWZhXSoswVdMqtTKvY7XJW6ceiZ6kn8KktKRpxl3w5T+prEVMXA=
X-Gm-Message-State: AOJu0Yz0U5eurJMTWyBZDZA6w3F15v3Niyq+ushHA4CRw30kj/bdjFVG
	+jPRwo4zZOTTJYfbn36xGabqIIyIu5TUg3seEHJg6QIA1ACt33EHGe53MGCvng==
X-Google-Smtp-Source: AGHT+IHRRwrb/eV6oZLMV0JUaLV6T1X6kq2vUwY5KGCfwr38ahJB1Kv7n4lvfMWKyI7im0WGrXivOw==
X-Received: by 2002:a17:906:2845:b0:a59:f2d2:49b1 with SMTP id a640c23a62f3a-a5a2d1b0df0mr1536495066b.9.1715849847518;
        Thu, 16 May 2024 01:57:27 -0700 (PDT)
Message-ID: <06db6010-3d70-4caa-ae34-3e35418fa835@suse.com>
Date: Thu, 16 May 2024 10:57:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 14/19] Rename mfn_to_virt() calls
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 xen-devel@lists.xenproject.org
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-15-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240513134046.82605-15-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.05.2024 15:40, Elias El Yandouzi wrote:
> Until directmap gets completely removed, we'd still need to
> keep some calls to mfn_to_virt() for xenheap pages or when the
> directmap is enabled.
> 
> Rename the macro to mfn_to_directmap_virt() to flag them and
> prevent further use of mfn_to_virt().

What I'm missing here is mentioning of criteria by which it is okay to
continue to use the (renamed) macro in the specific cases where they're
simply renamed. That might then also serve as a reference for people
needing to update their code. This may also mean the patch actually may
want splitting, grouping changes for similar reasons together and
dropping the old macro only as a last, final step.

Furthermore, also taking the next patch into account, it's not clear to
me why e.g. page_to_virt() is okay to survive.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:11:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:11:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722910.1127306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7X9B-0005TD-IK; Thu, 16 May 2024 09:11:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722910.1127306; Thu, 16 May 2024 09:11:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7X9B-0005T6-FX; Thu, 16 May 2024 09:11:33 +0000
Received: by outflank-mailman (input) for mailman id 722910;
 Thu, 16 May 2024 09:11:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7X9A-0005Su-36
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:11:32 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47b43502-1364-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:11:30 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-5709cb80b03so3316263a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:11:29 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfcc1sm9984154a12.45.2024.05.16.02.11.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:11:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47b43502-1364-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715850689; x=1716455489; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lT/9utrNTe0rj3FLn4NaMDWp6jMiAah10FuyiBzbjqQ=;
        b=gtAiN/nHhPOG6snx5J+84A1NpMNB/CBRVhluL5dyET/sQr1OdW/OrxFDruKCeJk48S
         Noxa4Lz4fENqbXmhVuER/dAGHtZERh1W7pZWasNoJy5xDS0KOkyCnNtud7uawXTuuxgF
         SkI55iUIIt+vdo0Z6pXNI4GseusjXOkLzzhBYBgKQtMZZsu5WxkkdBgQjahVwYcUBCSO
         qFJzzcCYGQMDr206EDbWF/tLoT7rcCBFTaORx8wybTltP0TE5KJzGFDTCqmhvexs326y
         MrCMMgqjJF0nbbptpJ9AoMNqTEYBlzbk/BMnYKW/K6+kDE8xzINjDL0vXChX/Wdf56HW
         EZyw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715850689; x=1716455489;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lT/9utrNTe0rj3FLn4NaMDWp6jMiAah10FuyiBzbjqQ=;
        b=pB75h3WsvewHDCZz/KLOXU677lrj/sRJkADDngoVhLaJFgczUmBihzv+Yh2KlBGK7q
         Z/fsCFhOzDRBG2LGCBdMQ9hp9wpw3vEzosnAMO3qp1vUT+zpVLUlXLHzmfWn/dr2xwuo
         wIKKcT0IhrV0bBd250YkkBA7CoEUzMKwC36uD/JUg2+pRu5ercjGzqp1eRm1zDr317IC
         2IfCoUX/diD0Y/K90vL+RUwzs9LFl9PfrYH+fqiLJReDoYsz1vkpBRKsW5zyNUVFA/lX
         nzZZn5eYHrWvPQo+S38oBjHTsowG0aHPBs3Jeq7LBXRExPKdETDKwv1bGYMN7lKG5jQ1
         flqA==
X-Forwarded-Encrypted: i=1; AJvYcCX4r2UWqP2aDv7D+bjMI29zNgp6LBhTCPWtvhnm3Mpeas6fwcnQKV7YMWpOPxLtHNffvqohOhyl3jitCWMqtsdYvWcEp16yCbQtfr7IgVw=
X-Gm-Message-State: AOJu0YxDBlbHeZ4AxAnyzD2hWjMgHWsR4Etw1wvIhgD9s0cfphjBzGzF
	9oSpfw3xOn2e+PtKDFrCD6w73bvbEd0JxWKNrX0jlFZXTOerM9Y9V/J/yYEEc09fxJqMBYc34dQ
	=
X-Google-Smtp-Source: AGHT+IFJQGBgPxapaYMmPa/aU3u4tgRQS1lVGNRbKS6q12vTmyZQEmATytW+CHtl9XiYgnhyhtKMCw==
X-Received: by 2002:a50:aa94:0:b0:572:a198:49c5 with SMTP id 4fb4d7f45d1cf-5734d5be7a2mr10903060a12.16.1715850688855;
        Thu, 16 May 2024 02:11:28 -0700 (PDT)
Message-ID: <e7971561-02c5-4445-af4b-f18b0a6cefde@suse.com>
Date: Thu, 16 May 2024 11:11:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 1/5] vpci/header: emulate PCI_COMMAND register for
 guests
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-2-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240514143400.152280-2-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 16:33, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
> to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
> guest's (domU) view of this will want to be zero (for now), the host
> having set it to 1 should be preserved, or else we'd effectively be
> giving the domU control of the bit. Thus, PCI_COMMAND register needs
> proper emulation in order to honor host's settings.
> 
> According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
> Device Control" the reset state of the command register is typically 0,
> so when assigning a PCI device use 0 as the initial state for the
> guest's (domU) view of the command register.
> 
> Here is the full list of command register bits with notes about
> PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
> also documented here since that is our current reference implementation
> for PCI passthrough.
> 
> PCI_COMMAND_IO (bit 0)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU sets this bit to 1 in
>     hardware if an I/O BAR is exposed to the guest.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
>     don't yet support I/O BARs for domUs.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_MEMORY (bit 1)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU sets this bit to 1 in
>     hardware if a Memory BAR is exposed to the guest.
>   Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
>     regions.
>   Xen domU: For devices assigned to DomUs, memory decoding will be
>     disabled at the time of initialization.
> 
> PCI_COMMAND_MASTER (bit 2)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_SPECIAL (bit 3)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_INVALIDATE (bit 4)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_VGA_PALETTE (bit 5)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_PARITY (bit 6)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_WAIT (bit 7)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: hardwire to 0
>   QEMU: res_mask
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_SERR (bit 8)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_FAST_BACK (bit 9)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_INTX_DISABLE (bit 10)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU checks if INTx was mapped
>     for a device. If it is not, then guest can't control
>     PCI_COMMAND_INTX_DISABLE bit.
>   Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> Bits 11-15
>   PCIe 6.1: RsvdP
>   PCI LB 3.0: Reserved
>   QEMU: res_mask
>   Xen domU/dom0: rsvdp_mask
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one further nit:

> --- a/xen/drivers/vpci/msi.c
> +++ b/xen/drivers/vpci/msi.c
> @@ -57,6 +57,7 @@ static void cf_check control_write(
>  
>      if ( new_enabled )
>      {
> +        bool old_enabled = msi->enabled;
>          /*
>           * If the device is already enabled it means the number of
>           * enabled messages has changed. Disable and re-enable the

Blank line please to separate declaration(s) from statement(s) (or, in this
case, a comment ahead of a statement). Happy to insert while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:20:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:20:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722924.1127320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XHR-0007Js-FH; Thu, 16 May 2024 09:20:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722924.1127320; Thu, 16 May 2024 09:20:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XHR-0007JK-Av; Thu, 16 May 2024 09:20:05 +0000
Received: by outflank-mailman (input) for mailman id 722924;
 Thu, 16 May 2024 09:20:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7XHP-0006Gj-KW
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:20:03 +0000
Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com
 [2607:f8b0:4864:20::734])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 796884b5-1365-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 11:20:02 +0200 (CEST)
Received: by mail-qk1-x734.google.com with SMTP id
 af79cd13be357-78f02c96c52so480748685a.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:20:02 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-793017facabsm95003685a.88.2024.05.16.02.20.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 02:20:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 796884b5-1365-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715851201; x=1716456001; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=wx6W+ENlg5PRro4DaaU11kdGbIPXqjVb6ItABsLuQ9w=;
        b=nZYzH3cuRe50fJ9FBITUVcvfGrSvn47VKLUXCUjFj8K4q9KBe40OjNvH49jMSgfFcE
         q2Qmm/EgKcb/9nPEqZcd9U+w5KtCf8KjkdrQxd70q4ZR7Hhqxmlg9zB/yKNRih9nQNjY
         rhOjfZmnB+/9SYeMpH4sgpp6CHzVZDUDDShLI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715851201; x=1716456001;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wx6W+ENlg5PRro4DaaU11kdGbIPXqjVb6ItABsLuQ9w=;
        b=lbyT2QHKnf5VW03KtPtnzi9NPEd2qEFWQ9t9FQorFffapdCt5/9N9BjkW0/gfMLLjZ
         B0thHlGmnHBDQj49L68ka3N6ae1FjLSV6pCsE6wesf1eGR09zypS8HGoVttqy34r3M31
         NdhYtE5btZmZN9bs6vKcw/6HZ62b8YY1EKstiMa9XP6/+IgvioBgadhS7QDLSgPxUWW9
         RKv9iMLS2RNbxpzTZMkxUgc3CP9Ufln2c1C/qtEBb8lcEXMHnNACQhPJDLjGIcHO4Oa0
         qRqDuDTqmvtFyQ2gAS0NgMkJnpPgrN6gmSLWkpQkfOPrUprVbQ0O0OkqKiGX5ybq4+ZI
         fRag==
X-Forwarded-Encrypted: i=1; AJvYcCVKLKR3IAtQblab84RfsQgfVEBlbp5GzYMS6GuwtrTaXYkFrPMPF/fRK41ck3BK7ISvthPOjfSdv9bw8eAbdynH7Xw0xfC08VFUUPgWkhA=
X-Gm-Message-State: AOJu0YyJsAhvTq1wd77K1HmvJQ3Hn+EajzY/Ppo9HlxbqBgV6qCOmgjz
	04JLc3hdza3ep5QBNhx+xhvzWOHECxyWpZnBZbOtfeCU3sbSvZM86J0TjA+Qla4=
X-Google-Smtp-Source: AGHT+IFBgbpaG+gm/fpEGXhcgTZ//0BPdTq1H4gMyQB8ODnBMFM52h2/0ZyWQvjIozD5yw2uGU78Fg==
X-Received: by 2002:a05:620a:c93:b0:792:c175:b12f with SMTP id af79cd13be357-792c7600ee8mr1860050685a.63.1715851201414;
        Thu, 16 May 2024 02:20:01 -0700 (PDT)
Date: Thu, 16 May 2024 11:19:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Elias El Yandouzi <eliasely@amazon.com>, xen-devel@lists.xenproject.org,
	julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
	Hongyan Xia <hongyxia@amazon.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Message-ID: <ZkXPvgRupZw31nGw@macbook>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com>
 <ZkMs1cknBFBWZoJG@macbook>
 <086f1bbe-2b86-493e-8936-81f420500672@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <086f1bbe-2b86-493e-8936-81f420500672@suse.com>

On Wed, May 15, 2024 at 03:54:51PM +0200, Jan Beulich wrote:
> On 14.05.2024 11:20, Roger Pau Monné wrote:
> > On Mon, May 13, 2024 at 01:40:33PM +0000, Elias El Yandouzi wrote:
> >> --- a/docs/misc/xen-command-line.pandoc
> >> +++ b/docs/misc/xen-command-line.pandoc
> >> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
> >>  hardware guarantees (with, where available and known to Xen, respective
> >>  tweaks applied).
> >>  
> >> +### directmap (x86)
> >> +> `= <boolean>`
> >> +
> >> +> Default: `true`
> >> +
> >> +Enable or disable the directmap region in Xen.
> > 
> > Enable or disable fully populating the directmap region in Xen.
> 
> Elias, would you please take care to address earlier review comments
> before sending a new version? This and ...
> 
> >> +
> >> +By default, Xen creates the directmap region which maps physical memory
> >                                                           ^ all?
> >> +in that region. Setting this to no will sparsely populate the directmap,
> > 
> > "Setting this to no" => "Disabling this option will sparsely..."
> 
> ... this is what I had already asked for in reply to v2, of course with
> different wording.
> 
> >> --- a/xen/arch/x86/setup.c
> >> +++ b/xen/arch/x86/setup.c
> >> @@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
> >>      if ( highmem_start )
> >>          xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
> >>  
> >> +    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
> > 
> > IMO this wants to be printed as part of the speculation mitigations, see
> > print_details() in spec_ctrl.c
> 
> And not "on" / "off", but "full" / "sparse" (and word order changed accordingly)
> perhaps.

I've been thinking about this, and I'm leaning towards calling this
new mode "ondemand" rather than "sparse".  The fact that the direct
map ends up sparely populated is a consequence of populating it on
demand, and hence the later would be more descriptive IMO.

(Same for the Kconfig option then ONDEMAND_DIRECTMAP, or some such)

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:23:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:23:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722933.1127329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XKr-0008Go-0v; Thu, 16 May 2024 09:23:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722933.1127329; Thu, 16 May 2024 09:23:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XKq-0008Gh-UP; Thu, 16 May 2024 09:23:36 +0000
Received: by outflank-mailman (input) for mailman id 722933;
 Thu, 16 May 2024 09:23:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7XKp-0008Gb-Q1
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:23:35 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f7e9de29-1365-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 11:23:34 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-5238b5c080cso671455e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:23:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm6954042a12.92.2024.05.16.02.23.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:23:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7e9de29-1365-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715851414; x=1716456214; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=d9Unq06GS3W7hSL9K5ZoogCfkJuM3N5NCg/yz3EDx2k=;
        b=J0JR/5HprA+1vSdZXmDQu+6AV/uL7WdEPM1qVwgRfKLY6IXz2ABBLvDSjSJ+XKBeQF
         /wcsYSDW1T79a6/KHT0dVbX3B2OrdG/4S7dHuGq/3jKQg3VW1T/FMvI3KtIEujlqJZh+
         miqI9bbQJp0Ud8+v8QlRQLliGqNyI+3gl3prtWyS65mjiVhhiwC97Ty1P8QqegNlxeKh
         290Ic+w1l+OavudOzt7WIltFGUYNd3Lt60ENoiyc9z+lCEiCfbBUHXAPQHTluom2W+cQ
         7rAELVY95hjPvFU/XOfNtYy37Do+N1iyctC8xdbGaU15qttwzXIYQ/S7dcb/OJ6E2Mfv
         oSwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715851414; x=1716456214;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=d9Unq06GS3W7hSL9K5ZoogCfkJuM3N5NCg/yz3EDx2k=;
        b=BqD2PrbmYfcMGPsN6pbV0ukgz8qAItmYYeRaJ5Hm/vU5r/WK52R86FdeOuTJ7zfOiX
         p2rtaCExSzW84+B5MPNqLZghneF8tbHK5LFGQlI0p9oti0V2W64KF+km2+EQOdU1Zxg5
         Z87XqlVW7nhkwdpaXO2shiI/Y3Enz/xxoiSCBBk+8ohqRk7F3JrmGc8S6cLLs+JeMjTv
         9AmBzcQJpLJESJyd+F8BYmvMhJKaF3RWw+AA+N8EJu/skOlD7e8YRcKN+z4c3JlSbiF4
         5s/IE7xDtTc/IXOubK/JOqLo4+9dTyigkiXfc6DF92cMGHsGSQ+v+gxGYhNbXNxWBeLl
         Ynxw==
X-Forwarded-Encrypted: i=1; AJvYcCX0THigcoOM8C83Wu9QS71RCq5qh3PJ9+7faLGVwucNY/ItWH7TefUKU/8Kv6h1QQQKFeCg7SQTiAbxe+uGiQ5lghg3/BtLpkcFOWJrkzg=
X-Gm-Message-State: AOJu0Yy4oAwrLPP2EsnskLr/IbAlm8xwKu2MsNLN/uneMp0KINBuuPRs
	/VNHhEx4Zohu6j8lNQnGoPN/SPAbmYkM8/0EZlo3G3tt1s72N8SX5ct8+l2DIg==
X-Google-Smtp-Source: AGHT+IHqhH3tWmRkdQKnimlbiPtAbxk/0IoZ/FQ90eDPqkqnlPgUf15BdlmNwpYPP/F5s/c6VYxGpg==
X-Received: by 2002:a05:6512:1310:b0:516:cc06:fa03 with SMTP id 2adb3069b0e04-5221006ccdemr14170268e87.56.1715851413935;
        Thu, 16 May 2024 02:23:33 -0700 (PDT)
Message-ID: <806a2978-19fb-4d31-ab6a-35ea7317c8de@suse.com>
Date: Thu, 16 May 2024 11:23:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 2/5] vpci: add initial support for virtual PCI bus
 topology
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240514143400.152280-1-stewart.hildebrand@amd.com>
 <20240514143400.152280-3-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240514143400.152280-3-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 16:33, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Assign SBDF to the PCI devices being passed through with bus 0.
> The resulting topology is where PCIe devices reside on the bus 0 of the
> root complex itself (embedded endpoints).
> This implementation is limited to 32 devices which are allowed on
> a single PCI bus.
> 
> Please note, that at the moment only function 0 of a multifunction
> device can be passed through.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

I'm largely okay, so
Acked-by: Jan Beulich <jbeulich@suse.com>

Still my wariness remains towards the multi-function bit (it being set is
at least misleading when the device then has no further functions), and I
also remain not fully convinced by the earlier reply towards phantom
functions. While those indeed don't have any config space, is their setting
up in the IOMMU tables actually handled correctly for this virtual SBDF
case? It may well be (as the virtual coordinates _should_ indeed not matter
at all), but that's not entirely obvious.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:24:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:24:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722936.1127338 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XM6-0000Mb-AO; Thu, 16 May 2024 09:24:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722936.1127338; Thu, 16 May 2024 09:24:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XM6-0000MU-7G; Thu, 16 May 2024 09:24:54 +0000
Received: by outflank-mailman (input) for mailman id 722936;
 Thu, 16 May 2024 09:24:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7XM4-0000MM-TC
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:24:52 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2564a71e-1366-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:24:50 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-574ea5184abso3353931a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:24:50 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm6954042a12.92.2024.05.16.02.24.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:24:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2564a71e-1366-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715851490; x=1716456290; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6RobZ2jZYqDsBFhsz2KpkAATm3NB9MFYo2sy3PSylCU=;
        b=cWD2zsKk3rXj4Hd2OMdU5Ml328F86MSiMSeGvgn8MswsNVF3OuGu5yGjiY3OZt3tvx
         i40fcW9/Sd+S1ioJPDuqnyom6tL4l+mV4itZQzGTAkDzn2Q7xVdjMYUYQ5O4/Su7//PU
         dvA3/hk+pR2wmzm8aFpxJdBfMKXhmDRH+7mcP7OvrFnpvx9tTRZUw6qoRk7kizC+CO/m
         wCMN5LK50aHkp+HQ8Z+RoTuFIkR0Pehmv24unp6ZwPz+kvyxJcEO2DQu7uqcLM5lPGGD
         A9V1piodOmlsdLdlaVomaRapymQi+MVYW5Xd3xbDVIfRgpD5DPrK7z35++HLn85s2qDB
         Y4VQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715851490; x=1716456290;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6RobZ2jZYqDsBFhsz2KpkAATm3NB9MFYo2sy3PSylCU=;
        b=TV/HLYvbfy4b9gbkhnc33ouQ6EHgvJ45rFH4zvkvMMa7oWtbdwhOWUmIZWlTdKIl+e
         jIEy4IJK5DF9cq5RywIEvxu4w0VLZZXecuiddCJLhV4y34/HivtT6yGGI3dj+FmHTD4q
         sTz4TvAKWUszVbY913zS1pM+/23UAnpjDBpMVbzy59u57/J6Ug/25fdMW33cR4BrK7oR
         Zhme774geP42uqk+CWEVUOtycWpnTp43UpfDTY3bdABWvJH3Jc4CnoagcJ6Bxo3GAZCL
         frX5/vgJ/ENHgC5mQoFUwpQrXYlzQm0c6QlYVG6IDYGRIXT/mhy71f2EMZm+gPcpSWUb
         bgZg==
X-Forwarded-Encrypted: i=1; AJvYcCUsAj35XH+AIQ6Qm3pNv75ObZ2Y0T/5XKVVvcfzbgWIlWJyuUeeM4XxmaKMFX2Fn1pHCOT9vB5YGbrwGKjA3Wp1+phUneBW/iVIoaLiq9A=
X-Gm-Message-State: AOJu0YyXjeyF4PQQpsZn0hSMiKzlhqtXeM94U9SYx29LFtzQ1m199fxk
	35+TINAYyNIAg93r7cnFrzIBwpXxxVqVSwLGlYIW6V/ypf0QdJBEjWC+tKyEqw==
X-Google-Smtp-Source: AGHT+IFifYw2nnZKN0QmbYKITGs7Yd9qkg+0YSEpVRluJvKijuOpfNfxqbtvHqYEZPugBw3jDVxqag==
X-Received: by 2002:a50:9fc9:0:b0:56e:24a5:587a with SMTP id 4fb4d7f45d1cf-5734d5bec0fmr14752032a12.11.1715851490291;
        Thu, 16 May 2024 02:24:50 -0700 (PDT)
Message-ID: <cc7620c8-a458-451e-badb-8e46d2cb9d90@suse.com>
Date: Thu, 16 May 2024 11:24:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3 (resend) 06/19] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Elias El Yandouzi <eliasely@amazon.com>, xen-devel@lists.xenproject.org,
 julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <jgrall@amazon.com>
References: <20240513134046.82605-1-eliasely@amazon.com>
 <20240513134046.82605-7-eliasely@amazon.com> <ZkMs1cknBFBWZoJG@macbook>
 <086f1bbe-2b86-493e-8936-81f420500672@suse.com> <ZkXPvgRupZw31nGw@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkXPvgRupZw31nGw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 11:19, Roger Pau Monné wrote:
> On Wed, May 15, 2024 at 03:54:51PM +0200, Jan Beulich wrote:
>> On 14.05.2024 11:20, Roger Pau Monné wrote:
>>> On Mon, May 13, 2024 at 01:40:33PM +0000, Elias El Yandouzi wrote:
>>>> --- a/docs/misc/xen-command-line.pandoc
>>>> +++ b/docs/misc/xen-command-line.pandoc
>>>> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
>>>>  hardware guarantees (with, where available and known to Xen, respective
>>>>  tweaks applied).
>>>>  
>>>> +### directmap (x86)
>>>> +> `= <boolean>`
>>>> +
>>>> +> Default: `true`
>>>> +
>>>> +Enable or disable the directmap region in Xen.
>>>
>>> Enable or disable fully populating the directmap region in Xen.
>>
>> Elias, would you please take care to address earlier review comments
>> before sending a new version? This and ...
>>
>>>> +
>>>> +By default, Xen creates the directmap region which maps physical memory
>>>                                                           ^ all?
>>>> +in that region. Setting this to no will sparsely populate the directmap,
>>>
>>> "Setting this to no" => "Disabling this option will sparsely..."
>>
>> ... this is what I had already asked for in reply to v2, of course with
>> different wording.
>>
>>>> --- a/xen/arch/x86/setup.c
>>>> +++ b/xen/arch/x86/setup.c
>>>> @@ -1517,6 +1517,8 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>>>>      if ( highmem_start )
>>>>          xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
>>>>  
>>>> +    printk("Booting with directmap %s\n", has_directmap() ? "on" : "off");
>>>
>>> IMO this wants to be printed as part of the speculation mitigations, see
>>> print_details() in spec_ctrl.c
>>
>> And not "on" / "off", but "full" / "sparse" (and word order changed accordingly)
>> perhaps.
> 
> I've been thinking about this, and I'm leaning towards calling this
> new mode "ondemand" rather than "sparse".  The fact that the direct
> map ends up sparely populated is a consequence of populating it on
> demand, and hence the later would be more descriptive IMO.
> 
> (Same for the Kconfig option then ONDEMAND_DIRECTMAP, or some such)

Fine with me, fwiw.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:39:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:39:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722946.1127350 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xa3-0002dS-KT; Thu, 16 May 2024 09:39:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722946.1127350; Thu, 16 May 2024 09:39:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xa3-0002dL-FQ; Thu, 16 May 2024 09:39:19 +0000
Received: by outflank-mailman (input) for mailman id 722946;
 Thu, 16 May 2024 09:39:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Xa1-0002dF-T7
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:39:17 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 29388c6a-1368-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 11:39:16 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so277307466b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:39:16 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cd7asm972793666b.10.2024.05.16.02.39.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:39:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29388c6a-1368-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715852355; x=1716457155; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HUx+ikfbzMlTBtMQjf1KJcraRogTi1XwyFjuggwj9xY=;
        b=g1LYsC5sKiAS975IdxQRfQatRaNcMgbxelC1/+mg+WtDzeEqaQVoREx16aySR7SYk6
         Qq0CuNI6BO78jiRGEV4bJMrVk/pLTV07kP7lQ+o2F0lTNuwy1xS5k9I1FaIqg7zuTNYI
         gVocE6ew6a+0dEd72wG3+SzmwyHuAPBbDrdjCV1wt0FA3YLKRoc8Z901bAOw7AzsDlhL
         TQUU7DhXTfiWt4xHPL1DRem+02iyD2hyzclSqgHnN5bDld2SxH+Z/8g3DoDA+JG3ZVbZ
         hIA7r0oRW/9EUGgzeRqaoqib/z7IvZd4Bth7rce1rBMBKOjQavLmKaxYC+ISrbt7tLYQ
         V+QA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715852355; x=1716457155;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HUx+ikfbzMlTBtMQjf1KJcraRogTi1XwyFjuggwj9xY=;
        b=JAyfu2SurwyK4h5wzGa99vwKbqgjaMAaytZWRfGulU3Q8g7siElNx2WwIf47j/CcSg
         wMpsRIWiFjaAdekk0LZv8NqCRT4zrqs4FK6pOKNZ4Kgj3yCKsIJITvtCXIrRMu8c2JKW
         sClFGtxOfNQjsr2lwgegMnPFachUE/7Sx7B6iSoMlDuKYKZQ7EicYSVTOHuh7kBJ8S1/
         oj7Cdz8BqhnoT5si/34zAcEZbHwd6uXdjS3JtSSUiGPa84nBL9uJdtGvKlISkYVGr+c+
         GQp3tAmqtDSBA2iHpUsLwpBiX6V/Dcpnhju4x+h8QzbwSOl1dJ/f3e6g6nHNqs2xcDpi
         4lZw==
X-Forwarded-Encrypted: i=1; AJvYcCVlMb/lsuPr8V+QK52VXOwRRoCQfgAjzyqEhqKqNxTIKUNcZtqrMdgcWPiK0SZyPeW7P9D7aA0QUTnrcF5QUuWKMOuuQ7J9sxYHXh9sjlI=
X-Gm-Message-State: AOJu0Yw30z+Rheln3W+zLSz3tRnLr7s7g22VgwMrTUrLDSAoHdOKCpxi
	qIrZFmqQ86O5ZsfhSFVaAmEr8aWF1mEv2m2JjkK6BW2BseGnO78mRqSTA4Tiow==
X-Google-Smtp-Source: AGHT+IE0hBWNFw6r3ZpPo8koa8LM790YzWePh8QsCap2vy3StnrXZQhFtOW554SowyxUbjOB4tV6pA==
X-Received: by 2002:a17:907:3601:b0:a55:9195:362c with SMTP id a640c23a62f3a-a5a2d66a3ffmr1404933866b.49.1715852355676;
        Thu, 16 May 2024 02:39:15 -0700 (PDT)
Message-ID: <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
Date: Thu, 16 May 2024 11:39:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to
 header
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 10:20, Sergiy Kibrik wrote:
> Moving this function out of mce_intel.c would make it possible to disable
> build of Intel MCE code later on, because the function gets called from
> common x86 code.

Why "would"? "Will" or "is going to" would seem more to the point to me.
But anyway.

> --- a/xen/arch/x86/cpu/mcheck/mce.h
> +++ b/xen/arch/x86/cpu/mcheck/mce.h
> @@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v, uint32_t msr)
>      return 0;
>  }
>  
> +static inline bool vmce_has_lmce(const struct vcpu *v)
> +{
> +    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
> +}

Is there a particular reason this is placed here, rather than ...

> --- a/xen/arch/x86/include/asm/mce.h
> +++ b/xen/arch/x86/include/asm/mce.h
> @@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
>  extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu *ctxt);
>  extern int vmce_wrmsr(uint32_t msr, uint64_t val);
>  extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
> -extern bool vmce_has_lmce(const struct vcpu *v);
>  extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);

... in the file the declaration was in, thus avoiding ...

> --- a/xen/arch/x86/msr.c
> +++ b/xen/arch/x86/msr.c
> @@ -24,6 +24,8 @@
>  
>  #include <public/hvm/params.h>
>  
> +#include "cpu/mcheck/mce.h"

... the need for such a non-standard, cross-directory #include?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:44:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:44:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722950.1127359 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xep-0004iE-3b; Thu, 16 May 2024 09:44:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722950.1127359; Thu, 16 May 2024 09:44:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xep-0004i7-0k; Thu, 16 May 2024 09:44:15 +0000
Received: by outflank-mailman (input) for mailman id 722950;
 Thu, 16 May 2024 09:44:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Xen-0004i1-W0
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:44:13 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d971b0a2-1368-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:44:11 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5731ffcc905so3460809a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:44:11 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8a55f2e8sm279374766b.126.2024.05.16.02.44.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:44:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d971b0a2-1368-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715852651; x=1716457451; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jwPUykx1X9pX6Ql4hdD+Q07RasmI59xDJhwjudis6nI=;
        b=J6ZuAeMz1vE+fwkzx5iorzO1026S2QVhNvQ7riGO+kC39cP6aghpYnYQIeszVBFLay
         Lr5LBdnW0ic2sDu8EKVkpus8gx705L5GMtYPll4bV8Ebc7y65BC1hjPUxodMIddOT5X+
         1ExZk5jIBIbwZ6KVaOR+5f8MER1a2ExtvalJpFL7zeMfzom0/Xf1+wL4+7zbb3jaJkoy
         lRhQkVx1GkE8qxrsatdd/HL4fSG6IW2JTyE/CxV27OY4SJBOCNpQdMFMRC2kwqPhL7Rn
         LaLVjM68rBCVGZXM9XHiu1H3d5Rq5i3qXduUEdgzQ/ESFSojNd78W55xkOsIGSUsY5fa
         c/Ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715852651; x=1716457451;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jwPUykx1X9pX6Ql4hdD+Q07RasmI59xDJhwjudis6nI=;
        b=bXxgdDcbsJ7bcbhzf6w4S08une8A8CPVPlkyfAickQp5bv7CrhCxxRYR6aArqAmfSk
         mJDHfHjc3nYhZboc+/jk/TVj4gO271KiiABocwMT8yGfISQvw/b2/aPwJpeJ77xInxSG
         RwEZ4hytfFa+IyWmhqL1hWhBMtwCSBXnfzQhTJ0D62O3x7U3g7VySYYbcn2tq4jE/gST
         EG2Zs6Wp4Ms49gmobjIRWRbhtS8tcP/pPR/ZQ3RfNDBqufS2aWDrVUwvBA0nr2WrmIp2
         S3QUMXaENYxk3UpB3pTYfvRZOPQNhHLG6YL5k7KbbTbj++IEznjzXhDoeUO7F/D7KrKD
         bHDQ==
X-Forwarded-Encrypted: i=1; AJvYcCW4Zh7iv1uOEM7h3fHOLn0VO+J3joTQtHbD926VHwsQ8rvaMv9Ra6Mtqim+A5wIw/hnWcqArEqC652bL4gs9Z6X8H/IoERtf+0+8eDvitc=
X-Gm-Message-State: AOJu0YzNWPXLkh3xBVIlkjiYtY5EsqsSONSJrHc+cCZIzIhBTTsl55WE
	6bVLLNCmCaRgLtxmEGgbt/qRRM+upH8CX9uOmXu6gd2VbEsMbD3vKK5p7PwvYw==
X-Google-Smtp-Source: AGHT+IFeTAPs8gquwPfYVcg5IcYeVDhDVWUtgbnw16PzBGMUKX4lX9h4GSJzPbNrwbKl1Gms0+TB7A==
X-Received: by 2002:a17:907:3f28:b0:a59:bd78:34ca with SMTP id a640c23a62f3a-a5a2d5c89f3mr1458778366b.36.1715852651355;
        Thu, 16 May 2024 02:44:11 -0700 (PDT)
Message-ID: <7cffa5fc-7e23-4acf-98c3-56929025bbd9@suse.com>
Date: Thu, 16 May 2024 11:44:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 5/6] x86/MCE: add default switch case in
 init_nonfatal_mce_checker()
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <01aad263ad60f37ed74138b5abf2733361bb7d25.1715673586.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <01aad263ad60f37ed74138b5abf2733361bb7d25.1715673586.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 10:26, Sergiy Kibrik wrote:
> The default switch case block is likely wanted here, to handle situation
> e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, but
> misleading message still gets logged anyway.

With "likely" dropped I'm okay with this as far as the addition of the default
label goes. However, ...

> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
> @@ -29,9 +29,13 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>  		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>  		amd_nonfatal_mcheck_init(c);
>  		break;
> +
>  	case X86_VENDOR_INTEL:
>  		intel_nonfatal_mcheck_init(c);
>  		break;
> +
> +	default:
> +		return -ENODEV;
>  	}
>  	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>  	return 0;

... converting this to an error return still isn't justified. On one hand it's
benign because we still don't check return values from initcalls. Yet otoh it
isn't really an error, is it?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:46:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:46:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722955.1127369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xh1-0005JB-ID; Thu, 16 May 2024 09:46:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722955.1127369; Thu, 16 May 2024 09:46:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xh1-0005J4-FJ; Thu, 16 May 2024 09:46:31 +0000
Received: by outflank-mailman (input) for mailman id 722955;
 Thu, 16 May 2024 09:46:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Xh0-0005Iy-EO
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:46:30 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b5fd056-1369-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 11:46:29 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5a5cce2ce6so248364966b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:46:29 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a7b53fc8fsm368333366b.38.2024.05.16.02.46.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:46:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b5fd056-1369-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715852789; x=1716457589; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uoB7/keanyFaN2lwuhb3jZv9pDHbUfKzN3oRLYa1Wtk=;
        b=HJ1Df1RxZ7Uzd5gJmntOzRuov6Z0NEAfaT1dMv2JP535NNFxF2mveXvdWzEASm4Kuz
         Jcua36w9CMKVFX4bcStrrB0Sb1Cy5F4obgvUWN8P9irhelQJ8eZuUvr94ziy3Uty4t0Y
         mR0w9BYNf81p1l1uL38nIVvE3OCcQZvGQ+gbHkqQ1WnRj6BS5exg/FMvNVDghaoklzlE
         aqipiorcKrFdaLknyD4C8wsCI7inuDUgaltfPIlhS76vTpGYSsWNAHF4l7UJJ2VwNCpS
         BLyj+tPbtLFAIz9C6ofb7adrH2EafouvsYa7RmPi+VzWxVSG5BPQUEVaYqJUeNQv+DK/
         JeUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715852789; x=1716457589;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uoB7/keanyFaN2lwuhb3jZv9pDHbUfKzN3oRLYa1Wtk=;
        b=Bvnq6csnFqRGPw4sNVVTDRCuO7YppbkowTBZAcafR6uFoM25q/A9dIswPNvTMgs2m2
         lNu3c0a57DbqQOk0U7sUbm6yH1LGz993hcWVrmQg19VADwHncnY8LZtQFwrg7vxoOcCz
         Ic8Wb80/nNVPkjj+EwHArIuQyztSKRmvuGhrZYYSjsSd85VAl+bhyfnckqLqIJBK5rZl
         pF770Tg0CYQqOo4ZVRCCFXng8Cwen0/mIDhdsHePVrfDaUFgF9/BbBpisXyVsYEGITC6
         l/wwlfEjeQvmnEjSq8P4XWYQEb+APTJ4ODl3Q+kEQpHE9ImzDmiSHNOvSP4mNz5Tnary
         Bovw==
X-Forwarded-Encrypted: i=1; AJvYcCVaUANYZb9fOoWWm1+2Q8pZz7S1Zpv83Q3/wGAsMjhtGvm0FRybKJWL4f20lsm6h/vhnKi3HLB2SEiaUwh5/KxH6ZVuAITqAGyWWUpQr10=
X-Gm-Message-State: AOJu0Yyg4Kuu+DMZUAcNN+aY5RCevWNgzQy9oaVUpD15JR2PggrYGmt3
	OcMTcH4+w4gerWym52AJ/8HfQwSyBH5qXvOgMBSidUfYhSKEgwEAv9y7REXWiQ==
X-Google-Smtp-Source: AGHT+IFC/jODb1+ATe6fbHrsnVq5ZDtKBMYxaJ1At2gBM6gvD6DL7eiTSCgz50kPO16jKNf9T13rIg==
X-Received: by 2002:a17:906:b79a:b0:a59:aa69:9790 with SMTP id a640c23a62f3a-a5a2d54c87fmr1143782366b.8.1715852788813;
        Thu, 16 May 2024 02:46:28 -0700 (PDT)
Message-ID: <ea51dc26-e098-457f-a44c-262742b317d5@suse.com>
Date: Thu, 16 May 2024 11:46:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 5/6] x86/MCE: add default switch case in
 init_nonfatal_mce_checker()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <01aad263ad60f37ed74138b5abf2733361bb7d25.1715673586.git.Sergiy_Kibrik@epam.com>
 <7cffa5fc-7e23-4acf-98c3-56929025bbd9@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7cffa5fc-7e23-4acf-98c3-56929025bbd9@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 11:44, Jan Beulich wrote:
> On 14.05.2024 10:26, Sergiy Kibrik wrote:
>> The default switch case block is likely wanted here, to handle situation
>> e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, but
>> misleading message still gets logged anyway.
> 
> With "likely" dropped I'm okay with this as far as the addition of the default
> label goes. However, ...
> 
>> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
>> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
>> @@ -29,9 +29,13 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>>  		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>>  		amd_nonfatal_mcheck_init(c);
>>  		break;
>> +
>>  	case X86_VENDOR_INTEL:
>>  		intel_nonfatal_mcheck_init(c);
>>  		break;
>> +
>> +	default:
>> +		return -ENODEV;
>>  	}
>>  	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>>  	return 0;
> 
> ... converting this to an error return still isn't justified. On one hand it's
> benign because we still don't check return values from initcalls. Yet otoh it
> isn't really an error, is it?

I realize earlier in the function there are error returns, too. I question at
least the former of them as well. And the latter shouldn't be an error at least
when the vendor isn't going to be handled in the switch().

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722960.1127389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnQ-0007qS-Bf; Thu, 16 May 2024 09:53:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722960.1127389; Thu, 16 May 2024 09:53:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnQ-0007qL-8s; Thu, 16 May 2024 09:53:08 +0000
Received: by outflank-mailman (input) for mailman id 722960;
 Thu, 16 May 2024 09:53:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7XnP-0007bz-5C
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:07 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 177c4865-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:05 +0200 (CEST)
Received: from PA7P264CA0360.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:37c::15)
 by SJ2PR12MB8882.namprd12.prod.outlook.com (2603:10b6:a03:537::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Thu, 16 May
 2024 09:53:02 +0000
Received: from SN1PEPF00036F42.namprd05.prod.outlook.com
 (2603:10a6:102:37c:cafe::f5) by PA7P264CA0360.outlook.office365.com
 (2603:10a6:102:37c::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 09:53:00 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F42.mail.protection.outlook.com (10.167.248.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:52:59 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:52:53 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 177c4865-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PrA0excvJ2pqALielsNgWFp/2jOmQkCzIiCYFjKpwkjcgqP5H1h1JrBXBA9BOa4rOYq7NvhKmE5r539Z2bg/ByK7TntGg7/WjB5j6pcvFsj0No2LCaz2RPbO0H98VtNwziITbGIMdOPNYka1vRzdJ+iN90uN3rUq/zg53INJYV0oI3c3FQ4KoRQq4tk4j0Kk3PIpRk1mMZHA7tnUGOBfWyqQvH1Zwh8y0Dp3NuMhBtFdkAkDjyju3LIwStXfI2jV37Q6SCA7nAIPoLH5ITH7R7LWckUZlUj5noa8JVskPD/lkw7FWOGeZQN/vDC7tCUsE/bnW/rdRwLvVdJ7NSmmmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EeE3uZCJJMp49WYoeMdmwPSgIvA5T34m4ty5QgqPjT4=;
 b=FVLssQnpJpUCsNWsnLavxEBjiJy+/eduVP4dEYeBEkvvSjzU16az8gi1TYdWrzQ9o70is0CAOrz1i/uTb5nLl1y3GUlahIiTk4EPP1nHn2+H4wfCk+YwviBSL5RS1yobYfF98wzmiTRm+eSUcRJe2aLPnF0KI4Q/ef+IAtIcaBZ5DvjCsUNS78l0w1daux3ocdPhV0FH2ROdSXyEb17UrDC5vm7W+6WFi2PEbGiWKvpw0uAaPxJwKULGUgGouFQEXdraz87h6SFr0rXQS9OCo0KqVK7zMfWkwKLGButGMQSnLZMiP7W+iKcBsR2YndLHZugqrgVw9VyGfrCHikq/Nw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EeE3uZCJJMp49WYoeMdmwPSgIvA5T34m4ty5QgqPjT4=;
 b=vZFOWLw95jpRblQDWifxdc/V3tJiK/RKNa9bOKLPOC3xSynMXH1oUL8sOhwA3p/MWAuJP708RFqSoFVNicyDtKKB7HXKU95CopsnD+iPCjPwtJY5+zeaHpacP1MT6ydastwh+WwXvuuW0wCUNbcQcgdKc+WeIBGguyAcnPKPHoE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Date: Thu, 16 May 2024 17:52:31 +0800
Message-ID: <20240516095235.64128-2-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516095235.64128-1-Jiqian.Chen@amd.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F42:EE_|SJ2PR12MB8882:EE_
X-MS-Office365-Filtering-Correlation-Id: d2294fa0-b2b6-40cb-ba57-08dc758df88e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|7416005|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?aweMeH+AuoZeRE7AOJeeE25yBGx4nXT8hHQ8PX9jO5DnTSSD5zboUnRvJSiW?=
 =?us-ascii?Q?TMoR88kh69AIHAiWFZRDmqg6D9mwNKB6XReFimdRUqsnx0JQm/JMMTwh5OhG?=
 =?us-ascii?Q?W+SXkdos39POZvNrA8F0eBAl7j9Eh8FA3TnEUkTKOtwQ9o1n2lWhx0KhyN+S?=
 =?us-ascii?Q?T7al16IjVOF08aLOR7YaENY7OCn6G/dOEEZrup/gcwWqTX6txETGcKE5YeZk?=
 =?us-ascii?Q?5zaRP95VjcJVmNYFgNwFGNIjenOFjDg5u+ZoH0d7OPm5KsRjf30SK8At5Nfu?=
 =?us-ascii?Q?+F2zhOtSfKaPGiDMXwGGhZFOHlIQ5CYdgGXB4K4MdOh6Q6dRba6fUG2et6ev?=
 =?us-ascii?Q?E47lbOuG0Z8n2mpWZl0degbInepfEVrqn+R68o15AyjGYXnKdE6LgtdB1lJ/?=
 =?us-ascii?Q?xA3dqGQDitJRuxgVlImc09w5qehKXHdPw2iCX57gLoaIvgIRYEL0HwZHKt7r?=
 =?us-ascii?Q?+Ofz9fo01K9DX1XbLRh7wg1hbUdTFW8MveVnCe6RGd8vFtEw4VWtmhCpng9d?=
 =?us-ascii?Q?Kn5eknYbtAZZDYo8+bszIGVW02qRCXshCMvtaZZlepFNN/J55O+MEduIrTdS?=
 =?us-ascii?Q?XuKiSioii7DPSVMJ+o2k0RMfiGRe3KFuOiU352pblvh519ueRQAFTCnXLRUB?=
 =?us-ascii?Q?DKRWx/0DRiiR/r5eg109SHziOZ6OCRj7atAXZJjENw8YkLZBKIKxcpqupnZE?=
 =?us-ascii?Q?tekxcXOhobwiRgkqa+cEYfmSyVq5PqpHduI2ioTPwMcKBFs5xTY1V13k0/ig?=
 =?us-ascii?Q?FZxK6bOVttohp52UHqRPYe13RrZGwS6MpTvaS1tcmQHylMvSb1eJF6Ry5YIp?=
 =?us-ascii?Q?6BtgHSy71qfdFZFQwWEaq52adUTmYfyh1P9LqmWQJ8oJuo7UqFo3FyvMIijl?=
 =?us-ascii?Q?4sSsUafztb/vXoXs8I6RdDtDE1G78yM1J/x4ZAhIvAxr+2hi5sagNp3bGivg?=
 =?us-ascii?Q?WxgBamTfuPwejI2dsEIUMnsWLCFgiENB6tqI0ZruIbbp6PPqfG0SLAcUnzDd?=
 =?us-ascii?Q?lC+jg/BLWDfLeKaHaVF9zLGj7Ym2zB8TLvu4DUhsERYjsFYB/trnx708laGD?=
 =?us-ascii?Q?c80c1IhEFcIhf1WyHDtY/oLYmSuUrc1/zlmXiyeu22reqnRiLFNGXsb7tiR6?=
 =?us-ascii?Q?3uC1/piueFTul3ZARgr1NjK7ODemj25b/tlr6BhVytKziOEa6+J7zPem5ljx?=
 =?us-ascii?Q?QCee847rcsg6Kv4F4s8B2m7mtKzWbHB6/Mbz1WCRMsJhc6Xmb9rQ7j8D00HQ?=
 =?us-ascii?Q?UM3J/FjwwDWQ2xDL/VNBwvO3tM++l4ogYNo9hpViCzcynKqtA1BEqRPzkXn8?=
 =?us-ascii?Q?GBJ8ma8wIKnC6E6+A3lrjCyWLVT8Y9g/vNq5GPJj78jYEcL8s5eMq4xJcKTW?=
 =?us-ascii?Q?pW5WDAw2RtxSeFgEz155U/5puagN?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(7416005)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:52:59.9928
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d2294fa0-b2b6-40cb-ba57-08dc758df88e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F42.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8882

When a device has been reset on dom0 side, the vpci on Xen
side won't get notification, so the cached state in vpci is
all out of date compare with the real device state.
To solve that problem, add a new hypercall to clear all vpci
device state. When the state of device is reset on dom0 side,
dom0 can call this hypercall to notify vpci.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/x86/hvm/hypercall.c |  1 +
 xen/drivers/pci/physdev.c    | 36 ++++++++++++++++++++++++++++++++++++
 xen/drivers/vpci/vpci.c      | 10 ++++++++++
 xen/include/public/physdev.h |  7 +++++++
 xen/include/xen/vpci.h       |  6 ++++++
 5 files changed, 60 insertions(+)

diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 14679dd82971..56fbb69ab201 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -84,6 +84,7 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_pci_mmcfg_reserved:
     case PHYSDEVOP_pci_device_add:
     case PHYSDEVOP_pci_device_remove:
+    case PHYSDEVOP_pci_device_state_reset:
     case PHYSDEVOP_dbgp_op:
         if ( !is_hardware_domain(currd) )
             return -ENOSYS;
diff --git a/xen/drivers/pci/physdev.c b/xen/drivers/pci/physdev.c
index 42db3e6d133c..73dc8f058b0e 100644
--- a/xen/drivers/pci/physdev.c
+++ b/xen/drivers/pci/physdev.c
@@ -2,6 +2,7 @@
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
 #include <xen/init.h>
+#include <xen/vpci.h>
 
 #ifndef COMPAT
 typedef long ret_t;
@@ -67,6 +68,41 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         break;
     }
 
+    case PHYSDEVOP_pci_device_state_reset: {
+        struct physdev_pci_device dev;
+        struct pci_dev *pdev;
+        pci_sbdf_t sbdf;
+
+        if ( !is_pci_passthrough_enabled() )
+            return -EOPNOTSUPP;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&dev, arg, 1) != 0 )
+            break;
+        sbdf = PCI_SBDF(dev.seg, dev.bus, dev.devfn);
+
+        ret = xsm_resource_setup_pci(XSM_PRIV, sbdf.sbdf);
+        if ( ret )
+            break;
+
+        pcidevs_lock();
+        pdev = pci_get_pdev(NULL, sbdf);
+        if ( !pdev )
+        {
+            pcidevs_unlock();
+            ret = -ENODEV;
+            break;
+        }
+
+        write_lock(&pdev->domain->pci_lock);
+        ret = vpci_reset_device_state(pdev);
+        write_unlock(&pdev->domain->pci_lock);
+        pcidevs_unlock();
+        if ( ret )
+            printk(XENLOG_ERR "%pp: failed to reset PCI device state\n", &sbdf);
+        break;
+    }
+
     default:
         ret = -ENOSYS;
         break;
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 97e115dc5798..424aec2d5c46 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -115,6 +115,16 @@ int vpci_assign_device(struct pci_dev *pdev)
 
     return rc;
 }
+
+int vpci_reset_device_state(struct pci_dev *pdev)
+{
+    ASSERT(pcidevs_locked());
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
+    vpci_deassign_device(pdev);
+    return vpci_assign_device(pdev);
+}
+
 #endif /* __XEN__ */
 
 static int vpci_register_cmp(const struct vpci_register *r1,
diff --git a/xen/include/public/physdev.h b/xen/include/public/physdev.h
index f0c0d4727c0b..f5bab1f29779 100644
--- a/xen/include/public/physdev.h
+++ b/xen/include/public/physdev.h
@@ -296,6 +296,13 @@ DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_add_t);
  */
 #define PHYSDEVOP_prepare_msix          30
 #define PHYSDEVOP_release_msix          31
+/*
+ * Notify the hypervisor that a PCI device has been reset, so that any
+ * internally cached state is regenerated.  Should be called after any
+ * device reset performed by the hardware domain.
+ */
+#define PHYSDEVOP_pci_device_state_reset     32
+
 struct physdev_pci_device {
     /* IN */
     uint16_t seg;
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 6e4c972f35ed..93b1c1d72c05 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -30,6 +30,7 @@ int __must_check vpci_assign_device(struct pci_dev *pdev);
 
 /* Remove all handlers and free vpci related structures. */
 void vpci_deassign_device(struct pci_dev *pdev);
+int __must_check vpci_reset_device_state(struct pci_dev *pdev);
 
 /* Add/remove a register handler. */
 int __must_check vpci_add_register_mask(struct vpci *vpci,
@@ -266,6 +267,11 @@ static inline int vpci_assign_device(struct pci_dev *pdev)
 
 static inline void vpci_deassign_device(struct pci_dev *pdev) { }
 
+static inline int __must_check vpci_reset_device_state(struct pci_dev *pdev)
+{
+    return 0;
+}
+
 static inline void vpci_dump_msi(void) { }
 
 static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722959.1127379 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnP-0007cC-5i; Thu, 16 May 2024 09:53:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722959.1127379; Thu, 16 May 2024 09:53:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnP-0007c5-2Z; Thu, 16 May 2024 09:53:07 +0000
Received: by outflank-mailman (input) for mailman id 722959;
 Thu, 16 May 2024 09:53:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7XnN-0007bz-Pg
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:05 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15b61a2f-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:03 +0200 (CEST)
Received: from PA7P264CA0369.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:37c::8)
 by BY5PR12MB4226.namprd12.prod.outlook.com (2603:10b6:a03:203::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 09:52:59 +0000
Received: from SN1PEPF00036F42.namprd05.prod.outlook.com
 (2603:10a6:102:37c:cafe::81) by PA7P264CA0369.outlook.office365.com
 (2603:10a6:102:37c::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 09:52:57 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F42.mail.protection.outlook.com (10.167.248.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:52:56 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:52:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15b61a2f-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ggzsajg6AryCSFveiDLB6yaP9lK0dr1L5t7B3d/RU7c5Yewa+pfpCWHWrbuY5EspuTLbahuaTpN5OGDFQf2f2NhJnkCPlJ94tJSgOhkr5KJ1xcsQ2DTfFrB4HNRZn5uY4n3tU3nnGemBjWZAzCCeMKEvl6C1rSDccGUHs36Avi7JAoeScSvDuDh5PCleAH4XMGpX9WKnb41mS3T+U2O9fZLxRbFiNvYtgnPRCf3UcSYHdg7QjJg08oCNitxVlavsKhwEu1qoQ7/qLQohiU1if3P0alaAhYI5hTj/zKVsCWozjCVp4dhtTuX5ZePG7cDpsuz2s/1l4/qW791i9141Ag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=U576NGSoFyCkRp52I/6MmY0fAnRgoGTDI9GuJhwTsVk=;
 b=d7XdHG9LbJhd3FVBz8u2xMLK3sBRnsr+qBpLvFxzWiMPwc0NJisi1tjKTe0QOAIIPDCcu2CHVxiBhC4yChZuR/LRknZDj8RKmL1JiyX+4L59r06/VjQfcQHfUrMYPEWIwnugI5vL9prPxojU2zkrue8Y2HPO3upeIg2Lk0DgPaCeLdNiEXUXeoSyAG9J9F+QHBMS5Rf9WM4Dg+yUjyEOgohIeRx9Fi/8Iue/6dxUvb3EOV+tFP2KuK1rlKQTDeojt/mzB9ZnSGUIiWIffb0XFx5PhQYN2Jf4okG6bDW88Id+rFhieoW/yZ4pyD3CsfsQNxSbHaSy6HwuFegNV53aIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=U576NGSoFyCkRp52I/6MmY0fAnRgoGTDI9GuJhwTsVk=;
 b=OYCi99igHOJKg2/rv7xbXW3zhwb7CMkZdhX7VihOvQgM04DIAVGv0P+wgSQrtVKBj3p3gl1/KI5cVeJLC6wC4aneIIYIrGuRCg2D/AuR0N6u1MTt5NxyPRui6P1ZCA0/eqaoFLOJRHFidk+KBakz3WQ0EXnjB0dKKQAvQyn2NRc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>
Subject: [XEN PATCH v8 0/5] Support device passthrough when dom0 is PVH on Xen
Date: Thu, 16 May 2024 17:52:30 +0800
Message-ID: <20240516095235.64128-1-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F42:EE_|BY5PR12MB4226:EE_
X-MS-Office365-Filtering-Correlation-Id: 00e3306b-f881-40fc-8261-08dc758df690
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|7416005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?sk2Q7nXAKBbdqxstER/EWx5vxgP7SRjU+8Ts7nC+APCuaf3vRmJy4Y1r1Z/z?=
 =?us-ascii?Q?Lj5lQwtH8kErOeNAI8UY9xArf9WZdzD54JquFOGsXjS3PsjRUhrNOPhrK5hL?=
 =?us-ascii?Q?1r7/eKryAj++7xHiR+O5Z7KBV7JQBgwOkglnwvql8/p0m5L+3Bx+K/Dgnmpg?=
 =?us-ascii?Q?rSdQiZXeigqHOYcOuJSCPREcywTi3wlkH8G+8aJUPAnvKeIp17HxxWttoAbe?=
 =?us-ascii?Q?D7vx9cQ6jtFmWBM65/GecXP9sgnjcHD2BtNoz3EIAS1KbS5/M91rJ+Gbt7Ca?=
 =?us-ascii?Q?Nc4EGpSjyudWoLXABE2lZcqOnQmBwJxUoGSU4GTgs8GOR+0FHzLoWWh47YVr?=
 =?us-ascii?Q?wBDC8slNbFiKVkZBBuKjtmfTCvHY7uvY5tcGB1gT6vj7+wDQFZkcccN628O0?=
 =?us-ascii?Q?+h/AboPhRHm0jDfAlzAm9k8bFkkcTtTsMmnMtorE0sGa1faQtBHtaAEPQoKy?=
 =?us-ascii?Q?uNr9W94tjW1nfQgQeXIvQjwh2XtZA7IQvDlaajdd4N2oQRwVFytw0gywTNw4?=
 =?us-ascii?Q?QAVUOQd0ooLZakUdG2XMqqxBlSk4LCl8Tnw1gRyyzvOjINIRMIlRusH9JXwC?=
 =?us-ascii?Q?mciIwiBHOWYPMjFAit1NQn7IwlD8fsDPFRIi6qRJsNixcNvpyW7HPRFfI5DU?=
 =?us-ascii?Q?G9yS+a2+Qgdq9pqpSXFLhoxYs/zV2Et4+uFH7DEsOzB+OjY9MGryQM5Oq/dQ?=
 =?us-ascii?Q?VllXtaNB4D6aLw4K7sFu+5+4IUXpJajO9b7rMYmcWyrSylSi8FoS4Q9xguOt?=
 =?us-ascii?Q?29cc0b3iP5UEOY8r7fR6FbbekU0eDloz/Z6b/BqtJkB2zWBgDPMNQLQDSv8w?=
 =?us-ascii?Q?Fxlix45c0PgxbHLyH/CPJrVfFK+TIPwU7sZD3HwuntJ7h60e1N7rUbm5MwSM?=
 =?us-ascii?Q?maNc6uVorK1upChEnInTV4dKvjB1iWtEPOwp5EbKDEL0buRnPoVC0NZXSJNV?=
 =?us-ascii?Q?3PcdljdLuZTd8le4MITwstG4CQQZlBxv7dWTF1A4D1oPtvUY/b/IRc6TxmGr?=
 =?us-ascii?Q?HmbF8AilJTOgfoLngIHItNYgkrO4xlThKlyfy9RRTiG3drWOPy/5TnFGHyMG?=
 =?us-ascii?Q?D7ltrE0K8lJw1ZCXg5s1xdSs7vwhIq/dD+UcsBhnlvivmvvCXHkAVS6qd09y?=
 =?us-ascii?Q?7eo71tb8TW2rst69/310rt/eV6CB/YXSEexK/rJWmbWOSF6eLhBnK+AEo71D?=
 =?us-ascii?Q?krVMjeacKoz7cvSfvdBSDFoAFUyppDiIMgN5TmRRRqlemcgU1SUoPl8+enoO?=
 =?us-ascii?Q?d23ga0E1WA+vaTZfb54sSwwwAIglXQIOf34ftcdV4T5pcoa64rViv86EJ6QP?=
 =?us-ascii?Q?tuqf8YDS2yKt81ZrmSbnKZO+ockFMAMAZTfBxWqr7m0XqA=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(7416005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:52:56.6647
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 00e3306b-f881-40fc-8261-08dc758df690
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F42.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4226

Hi All,
This is v8 series to support passthrough when dom0 is PVH
v6->v7 changes:
* patch#2: Add the domid check(domid == DOMID_SELF) to prevent self map when guest doesn't use pirq.
           That check was missed in the previous version.
* patch#4: Due to changes in the implementation of obtaining gsi in the kernel. Change to add a new function
           to get gsi by passing in the sbdf of pci device.
* patch#5: Remove the parameter "is_gsi", when there exist gsi, in pci_add_dm_done use a new function
           pci_device_set_gsi to do map_pirq and grant permission. That gets more intuitive code logic.


Best regards,
Jiqian Chen



v6->v7 changes:
* patch#4: Due to changes in the implementation of obtaining gsi in the kernel. Change to add a new function
           to get gsi from irq, instead of gsi sysfs.
* patch#5: Fix the issue with variable usage, rc->r.


v5->v6 changes:
* patch#1: Add Reviewed-by Stefano and Stewart. Rebase code and change old function vpci_remove_device,
           vpci_add_handlers to vpci_deassign_device, vpci_assign_device
* patch#2: Add Reviewed-by Stefano
* patch#3: Remove unnecessary "ASSERT(!has_pirq(currd));"
* patch#4: Fix some coding style issues below directory tools
* patch#5: Modified some variable names and code logic to make code easier to be understood, which to use
           gsi by default and be compatible with older kernel versions to continue to use irq


v4->v5 changes:
* patch#1: add pci_lock wrap function vpci_reset_device_state
* patch#2: move the check of self map_pirq to physdev.c, and change to check if the caller has PIRQ flag, and
           just break for PHYSDEVOP_(un)map_pirq in hvm_physdev_op
* patch#3: return -EOPNOTSUPP instead, and use ASSERT(!has_pirq(currd));
* patch#4: is the patch#5 in v4 because patch#5 in v5 has some dependency on it. And add the handling of errno
           and add the Reviewed-by Stefano
* patch#5: is the patch#4 in v4. New implementation to add new hypercall XEN_DOMCTL_gsi_permission to grant gsi


v3->v4 changes:
* patch#1: change the comment of PHYSDEVOP_pci_device_state_reset; move printings behind pcidevs_unlock
* patch#2: add check to prevent PVH self map
* patch#3: new patch, The implementation of adding PHYSDEVOP_setup_gsi for PVH is treated as a separate patch
* patch#4: new patch to solve the map_pirq problem of PVH dom0. use gsi to grant irq permission in
           XEN_DOMCTL_irq_permission.
* patch#5: to be compatible with previous kernel versions, when there is no gsi sysfs, still use irq
v4 link:
https://lore.kernel.org/xen-devel/20240105070920.350113-1-Jiqian.Chen@amd.com/T/#t

v2->v3 changes:
* patch#1: move the content out of pci_reset_device_state and delete pci_reset_device_state; add
           xsm_resource_setup_pci check for PHYSDEVOP_pci_device_state_reset; add description for
		   PHYSDEVOP_pci_device_state_reset;
* patch#2: du to changes in the implementation of the second patch on kernel side(that it will do setup_gsi and
           map_pirq when assigning a device to passthrough), add PHYSDEVOP_setup_gsi for PVH dom0, and we need
		   to support self mapping.
* patch#3: du to changes in the implementation of the second patch on kernel side(that adds a new sysfs for gsi
           instead of a new syscall), so read gsi number from the sysfs of gsi.
v3 link:
https://lore.kernel.org/xen-devel/20231210164009.1551147-1-Jiqian.Chen@amd.com/T/#t

v2 link:
https://lore.kernel.org/xen-devel/20231124104136.3263722-1-Jiqian.Chen@amd.com/T/#t
Below is the description of v2 cover letter:
This series of patches are the v2 of the implementation of passthrough when dom0 is PVH on Xen.
We sent the v1 to upstream before, but the v1 had so many problems and we got lots of suggestions.
I will introduce all issues that these patches try to fix and the differences between v1 and v2.

Issues we encountered:
1. pci_stub failed to write bar for a passthrough device.
Problem: when we run \u201csudo xl pci-assignable-add <sbdf>\u201d to assign a device, pci_stub will call
pcistub_init_device() -> pci_restore_state() -> pci_restore_config_space() ->
pci_restore_config_space_range() -> pci_restore_config_dword() -> pci_write_config_dword()\u201d, the pci config
write will trigger an io interrupt to bar_write() in the xen, but the
bar->enabled was set before, the write is not allowed now, and then when 
bar->Qemu config the
passthrough device in xen_pt_realize(), it gets invalid bar values.

Reason: the reason is that we don't tell vPCI that the device has been reset, so the current cached state in
pdev->vpci is all out of date and is different from the real device state.

Solution: to solve this problem, the first patch of kernel(xen/pci: Add xen_reset_device_state
function) and the fist patch of xen(xen/vpci: Clear all vpci status of device) add a new hypercall to reset the
state stored in vPCI when the state of real device has changed.
Thank Roger for the suggestion of this v2, and it is different from
v1 (https://lore.kernel.org/xen-devel/20230312075455.450187-3-ray.huang@amd.com/), v1 simply allow domU to write
pci bar, it does not comply with the design principles of vPCI.

2. failed to do PHYSDEVOP_map_pirq when dom0 is PVH
Problem: HVM domU will do PHYSDEVOP_map_pirq for a passthrough device by using gsi. See
xen_pt_realize->xc_physdev_map_pirq and pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq will call
into Xen, but in hvm_physdev_op(), PHYSDEVOP_map_pirq is not allowed.

Reason: In hvm_physdev_op(), the variable "currd" is PVH dom0 and PVH has no X86_EMU_USE_PIRQ flag, it will fail
at has_pirq check.

Solution: I think we may need to allow PHYSDEVOP_map_pirq when "currd" is dom0 (at present dom0 is PVH). The
second patch of xen(x86/pvh: Open PHYSDEVOP_map_pirq for PVH dom0) allow PVH dom0 do PHYSDEVOP_map_pirq. This v2
patch is better than v1, v1 simply remove the has_pirq check
(xen https://lore.kernel.org/xen-devel/20230312075455.450187-4-ray.huang@amd.com/).

3. the gsi of a passthrough device doesn't be unmasked
 3.1 failed to check the permission of pirq
 3.2 the gsi of passthrough device was not registered in PVH dom0

Problem:
3.1 callback function pci_add_dm_done() will be called when qemu config a passthrough device for domU.
This function will call xc_domain_irq_permission()-> pirq_access_permitted() to check if the gsi has corresponding
mappings in dom0. But it didn\u2019t, so failed. See XEN_DOMCTL_irq_permission->pirq_access_permitted, "current"
is PVH dom0 and it return irq is 0.
3.2 it's possible for a gsi (iow: vIO-APIC pin) to never get registered on PVH dom0, because the devices of PVH
are using MSI(-X) interrupts. However, the IO-APIC pin must be configured for it to be able to be mapped into a domU.

Reason: After searching codes, I find "map_pirq" and "register_gsi" will be done in function
vioapic_write_redirent->vioapic_hwdom_map_gsi when the gsi(aka ioapic's pin) is unmasked in PVH dom0.
So the two problems can be concluded to that the gsi of a passthrough device doesn't be unmasked.

Solution: to solve these problems, the second patch of kernel(xen/pvh: Unmask irq for passthrough device in PVH dom0)
call the unmask_irq() when we assign a device to be passthrough. So that passthrough devices can have the mapping of
gsi on PVH dom0 and gsi can be registered. This v2 patch is different from the
v1( kernel https://lore.kernel.org/xen-devel/20230312120157.452859-5-ray.huang@amd.com/,
kernel https://lore.kernel.org/xen-devel/20230312120157.452859-5-ray.huang@amd.com/ and
xen https://lore.kernel.org/xen-devel/20230312075455.450187-5-ray.huang@amd.com/),
v1 performed "map_pirq" and "register_gsi" on all pci devices on PVH dom0, which is unnecessary and may cause
multiple registration.

4. failed to map pirq for gsi
Problem: qemu will call xc_physdev_map_pirq() to map a passthrough device\u2019s gsi to pirq in function
xen_pt_realize(). But failed.

Reason: According to the implement of xc_physdev_map_pirq(), it needs gsi instead of irq, but qemu pass irq to it and
treat irq as gsi, it is got from file /sys/bus/pci/devices/xxxx:xx:xx.x/irq in function xen_host_pci_device_get().
But actually the gsi number is not equal with irq. On PVH dom0, when it allocates irq for a gsi in
function acpi_register_gsi_ioapic(), allocation is dynamic, and follow the principle of applying first, distributing
first. And if you debug the kernel codes(see function __irq_alloc_descs), you will find the irq number is allocated
from small to large by order, but the applying gsi number is not, gsi 38 may come before gsi 28, that causes gsi 38
get a smaller irq number than gsi 28, and then gsi != irq.

Solution: we can record the relation between gsi and irq, then when userspace(qemu) want to use gsi, we can do a
translation. The third patch of kernel(xen/privcmd: Add new syscall to get gsi from irq) records all the relations
in acpi_register_gsi_xen_pvh() when dom0 initialize pci devices, and provide a syscall for userspace to get the gsi
from irq. The third patch of xen(tools: Add new function to get gsi from irq) add a new function
xc_physdev_gsi_from_irq() to call the new syscall added on kernel side.
And then userspace can use that function to get gsi. Then xc_physdev_map_pirq() will success. This v2 patch is the
same as v1( kernel https://lore.kernel.org/xen-devel/20230312120157.452859-6-ray.huang@amd.com/ and
xen https://lore.kernel.org/xen-devel/20230312075455.450187-6-ray.huang@amd.com/)

About the v2 patch of qemu, just change an included head file, other are similar to the
v1 ( qemu https://lore.kernel.org/xen-devel/20230312092244.451465-19-ray.huang@amd.com/), just call
xc_physdev_gsi_from_irq() to get gsi from irq.

Jiqian Chen (5):
  xen/vpci: Clear all vpci status of device
  x86/pvh: Allow (un)map_pirq when dom0 is PVH
  x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
  tools: Add new function to get gsi from dev
  domctl: Add XEN_DOMCTL_gsi_permission to grant gsi

 tools/include/xen-sys/Linux/privcmd.h |  7 +++
 tools/include/xencall.h               |  2 +
 tools/include/xenctrl.h               |  7 +++
 tools/libs/call/core.c                |  5 +++
 tools/libs/call/libxencall.map        |  2 +
 tools/libs/call/linux.c               | 15 +++++++
 tools/libs/call/private.h             |  9 ++++
 tools/libs/ctrl/xc_domain.c           | 15 +++++++
 tools/libs/ctrl/xc_physdev.c          |  4 ++
 tools/libs/light/libxl_pci.c          | 63 +++++++++++++++++++++++++++
 xen/arch/x86/domctl.c                 | 31 +++++++++++++
 xen/arch/x86/hvm/hypercall.c          |  8 ++++
 xen/arch/x86/physdev.c                | 24 ++++++++++
 xen/drivers/pci/physdev.c             | 36 +++++++++++++++
 xen/drivers/vpci/vpci.c               | 10 +++++
 xen/include/public/domctl.h           |  9 ++++
 xen/include/public/physdev.h          |  7 +++
 xen/include/xen/vpci.h                |  6 +++
 xen/xsm/flask/hooks.c                 |  1 +
 19 files changed, 261 insertions(+)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722961.1127399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnY-0008Ae-P9; Thu, 16 May 2024 09:53:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722961.1127399; Thu, 16 May 2024 09:53:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7XnY-0008AW-Kn; Thu, 16 May 2024 09:53:16 +0000
Received: by outflank-mailman (input) for mailman id 722961;
 Thu, 16 May 2024 09:53:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7XnX-0007bz-Be
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:15 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20615.outbound.protection.outlook.com
 [2a01:111:f400:7e88::615])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1bc561bf-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:13 +0200 (CEST)
Received: from PA7P264CA0355.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:37c::24)
 by DS0PR12MB8271.namprd12.prod.outlook.com (2603:10b6:8:fb::6) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.28; Thu, 16 May 2024 09:53:09 +0000
Received: from SN1PEPF00036F42.namprd05.prod.outlook.com
 (2603:10a6:102:37c:cafe::34) by PA7P264CA0355.outlook.office365.com
 (2603:10a6:102:37c::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 09:53:07 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F42.mail.protection.outlook.com (10.167.248.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:53:06 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:52:57 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bc561bf-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iAPRRoU7/BI9Gf3JPfgkdkgwy3bwou0kOM/nRVxchVReYBxKuZV+vfQvDcf1hU5YR57akyl/Iuz1EfKuKUJs+vcw25tpa/R/SkrtU1MzkTb7baH53plRocShpxTeGOgOawy0NF+vv15woPQ7ORsh4yF7vvOXJFOmO8f1BV2yWxNQJBBPpMrQYiNL/rJZEc4QiBZ2KCmRIOfonpdIZ+W5T/bPBrchBWgwJ57VnZlvdzpIwdPfpF9E0d4uEhgk+teSjwsCgzPWNukeX3L37r0irkHmfoyQhdb+ufmLM0kRNVsURMpWI3jahbcnSnO/hetqnFeeAIRLufveghBZ7PwVEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=S/Ec9q5A5ur4le3zKaKlsOkcz/0csbeUoraafjUCch8=;
 b=Dz20fxenNZRTWAAHnTqI8/3y4Z/wKVXKJYv4QhuxgjVF52jOUeLobwSRc4CPKn402hVzUD6Xa5DOokzeAo1hM6lTQlaMMzChgD/HPbq0CHBJmgTC9zT4t0kyC6NbkEGSC1aNKJGMcfFEZ9w6Wfg11DHK+P4Es+5s+rdGscltN2+UbOXppDaxjwJt7tlf6l6iX83Q6uYNqDPPt6tZP29PXX5f7DLborndzKwjsqj6MmP/a+Cb+vJFKfVmo7KcpNsz97A6+vp6MZkoLc6JS2XTBRZybYTLB3bk6BlFOuHckZ87toZQT/SiajJgBBzFHHIwoqZpkvSIQy1X1IJtXq6z4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=S/Ec9q5A5ur4le3zKaKlsOkcz/0csbeUoraafjUCch8=;
 b=yBRpxeUbTxZI4eXn66FP5RuU2ZJTRdnp5rkKbeQYlCu0e7cjh49VrEtVcgwECa/iyDrnplbr0UoUzj2QwZLzwKKPln0xu6gyyO0K5rCmL5qs1Hk50bIX/Vxqk6YMlEA97eDR82U0wC5kecXgCv9qjFisx/qUXred7NYkeB+1hOM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: [XEN PATCH v8 2/5] x86/pvh: Allow (un)map_pirq when dom0 is PVH
Date: Thu, 16 May 2024 17:52:32 +0800
Message-ID: <20240516095235.64128-3-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516095235.64128-1-Jiqian.Chen@amd.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F42:EE_|DS0PR12MB8271:EE_
X-MS-Office365-Filtering-Correlation-Id: 54ecdb5b-a85e-4e4a-c7e5-08dc758dfc88
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|7416005|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?3PDSWubrASEYM7A8ism4geb92zMjOFvEBobGcGuHDpkpg7UBAUp5bzQ1Tddu?=
 =?us-ascii?Q?7hijW5JmvmARZkIPyEeKj6ujGrBg87v0D2IgUze9VKA7RXShlmzEhPLShTQ6?=
 =?us-ascii?Q?5e62LtZQPuKh3Dqzy1WDqJRxoPfWuzl2WspspxhZZtttXe6FD+Oj785Su5HE?=
 =?us-ascii?Q?cPGfATvF508MQ2XwIW1b0FModtodLh3PYQIVl8YmxQ9ZKjyt9dF1h2Ajd+hZ?=
 =?us-ascii?Q?YtElcC+1DbsXmzC6+OOyVQKpis/AfCWSOzkQ6Pu328FtxDdm0YXGuU2EO4Ma?=
 =?us-ascii?Q?F08TRlrq9Wqgpq7rH4beZsaE4bB9NPu/IZOCHakBx8L893ssUqXruRv7Ad84?=
 =?us-ascii?Q?vO+XsZTw05HbfEX2B5sh8Gla3cFqo2Gqs1JvH6731NwyUUb6rQCVKvinFGKF?=
 =?us-ascii?Q?6ggRxsXYsYcYzoqPvVLBxqT6Y7VB0b4H1ZUBlyDC61VAgRaMczX9nFiAub2Z?=
 =?us-ascii?Q?rr72kI9IlcYD7kEOJhPr5w7oVVfCJYdfK+fbzWeI3/lxAGQt3BvKTjcJr2+8?=
 =?us-ascii?Q?JkNtMC/nPh9mvCxYUE1PlMdLUe/9aekl3nuBwuVhInnHn+2IPUZpgxpejacD?=
 =?us-ascii?Q?nFtIKmeyBLxM9GbNZpUAoioHSCBrMc82ILSH+dNV83/GtVmoOq24Nz4gpLWP?=
 =?us-ascii?Q?xz3wb0BkmaKwNBpoIMycFUd61QZJmL/7OVDrxK580iCNQUpadnkGDD38nraG?=
 =?us-ascii?Q?ADwU56HmfIZK5BIub9j7b47LuHCH+XPxnvnFxz84AZr+EFGbTzdywB75JXRw?=
 =?us-ascii?Q?39proVguhmQr2NbuD05DGkprL7vyx29akRSsm/Rs6OdrIk4iD4lhgSL8guEM?=
 =?us-ascii?Q?Xcqo2CziGdkd050qppSQpFj8NuUPfS9QIiCvVaWAlR2053sJpTD0rrEuHYdr?=
 =?us-ascii?Q?c1+mRi6dSyuaGASUY0yV9LL83aK9393k5iLUFf6hAuqAo88fP/9SEKFTebiD?=
 =?us-ascii?Q?pMWjwvXUzCoewdPawkQEMtfYnOqXroZoss5pZ4NdxY776Db+Sf0B0xWCSeO4?=
 =?us-ascii?Q?cBuKr0Cl7BCeb9Mz0o/lwszBLa0IIcoH9Ezyy+SGC0Mtpxkm/BZn7D2eqylY?=
 =?us-ascii?Q?GPGdXw6wFqHGcv1Qo70KXAbYbHscdO51jdexPbJWyKk5rY9KdSadgkwrc6lO?=
 =?us-ascii?Q?xjUCHK9E2XC2p9CNPhHppnC6ZqnOLBIErv8QJ1x1rZANZRp5pPm/r7vrBFJF?=
 =?us-ascii?Q?0SMv4HI/vCbDRDgNyZz6aUDFgpvyCOILnQAe4fNOhMuuxKq/i02GIOzVmBRY?=
 =?us-ascii?Q?rVTZUfPm7yulqycTUKJVsOqq1RLY+jHSI1I4//mkrIjv4umM8mQehMDR4CJQ?=
 =?us-ascii?Q?7pMRDJii/NDsQjsc9D6HdI8P7cYZZVNgqWQ+GkxR/04sxdEPjzC2yFZTItcL?=
 =?us-ascii?Q?hUHnEpwCWMGMdxB8zAyIUHNjeiJn?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:53:06.6803
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 54ecdb5b-a85e-4e4a-c7e5-08dc758dfc88
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F42.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8271

If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
a passthrough device by using gsi, see
xen_pt_realize->xc_physdev_map_pirq and
pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
is not allowed because currd is PVH dom0 and PVH has no
X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.

So, allow PHYSDEVOP_map_pirq when dom0 is PVH and also allow
PHYSDEVOP_unmap_pirq for the failed path to unmap pirq. And
add a new check to prevent self map when caller has no PIRQ
flag.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/x86/hvm/hypercall.c |  2 ++
 xen/arch/x86/physdev.c       | 24 ++++++++++++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 56fbb69ab201..d49fb8b548a3 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -74,6 +74,8 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     {
     case PHYSDEVOP_map_pirq:
     case PHYSDEVOP_unmap_pirq:
+        break;
+
     case PHYSDEVOP_eoi:
     case PHYSDEVOP_irq_status_query:
     case PHYSDEVOP_get_free_pirq:
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 7efa17cf4c1e..1337f95171cd 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -305,11 +305,23 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_map_pirq: {
         physdev_map_pirq_t map;
         struct msi_info msi;
+        struct domain *d;
 
         ret = -EFAULT;
         if ( copy_from_guest(&map, arg, 1) != 0 )
             break;
 
+        d = rcu_lock_domain_by_any_id(map.domid);
+        if ( d == NULL )
+            return -ESRCH;
+        /* If caller is the same HVM guest as current, check pirq flag */
+        if ( !is_pv_domain(d) && !has_pirq(d) && map.domid == DOMID_SELF )
+        {
+            rcu_unlock_domain(d);
+            return -EOPNOTSUPP;
+        }
+        rcu_unlock_domain(d);
+
         switch ( map.type )
         {
         case MAP_PIRQ_TYPE_MSI_SEG:
@@ -343,11 +355,23 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 
     case PHYSDEVOP_unmap_pirq: {
         struct physdev_unmap_pirq unmap;
+        struct domain *d;
 
         ret = -EFAULT;
         if ( copy_from_guest(&unmap, arg, 1) != 0 )
             break;
 
+        d = rcu_lock_domain_by_any_id(unmap.domid);
+        if ( d == NULL )
+            return -ESRCH;
+        /* If caller is the same HVM guest as current, check pirq flag */
+        if ( !is_pv_domain(d) && !has_pirq(d) && unmap.domid == DOMID_SELF )
+        {
+            rcu_unlock_domain(d);
+            return -EOPNOTSUPP;
+        }
+        rcu_unlock_domain(d);
+
         ret = physdev_unmap_pirq(unmap.domid, unmap.pirq);
         break;
     }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722962.1127410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xnb-0008Rl-1B; Thu, 16 May 2024 09:53:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722962.1127410; Thu, 16 May 2024 09:53:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xna-0008Re-S9; Thu, 16 May 2024 09:53:18 +0000
Received: by outflank-mailman (input) for mailman id 722962;
 Thu, 16 May 2024 09:53:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7XnZ-0007bz-Iq
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:17 +0000
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2407::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1d984174-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:15 +0200 (CEST)
Received: from SA1P222CA0195.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:3c4::8)
 by BL1PR12MB5873.namprd12.prod.outlook.com (2603:10b6:208:395::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Thu, 16 May
 2024 09:53:11 +0000
Received: from SN1PEPF00036F3E.namprd05.prod.outlook.com
 (2603:10b6:806:3c4:cafe::e2) by SA1P222CA0195.outlook.office365.com
 (2603:10b6:806:3c4::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 09:53:11 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F3E.mail.protection.outlook.com (10.167.248.22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:53:10 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:53:06 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d984174-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CNWcmJsw83un9TlRBUhAEexfgDypF8G1MUZfJeCoP4gRVoYp7/wVacMEB5SorwT25b+OlRl1+BoXZF7JpCZfEGcjuW7ooJ3ICEd7SoXnzFMGWhE2I7l76ydjY8EDRZ/sFTiTY7Fb0l6CvDskWFRcI7VWB51Lqkv4r4HdmGB/e4xCVxbWB2A5MVuem6+Hbs0sBAuhlS2P7G5rSNIEqXtX71wYbbI7/JyJj3622W7AuQWkitDXnLrMWMiHfpD6rE4129+B+/7UGIkSIqrI/zrmubUs7tJ9sHPyc0CZGU5HicHIbIZ4ehZyWzrLo/d35CaUMBVz4Fcz4q6DSEQKHoEDaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5Ljn9VtnUEyifgNA3cosOzMocDUuiDW1at9hMSJ3P5c=;
 b=ZaMRj0I8SqBiB/fX2KVEtQ+JmnxhBjX4Ql4zUUvS3L37kKWrIM0ohTQYTQSOxgFrmXeRCpXUf4RwNdhnavg+pcpj5URDwwKPFnYDkjw+afNr2ANzFG2ZN0zGYYtQqfBVVYOzLDn1nTPZ89fIhb+L3rbzfd29Joi6Ysgs4baSrsSZ/OCR+t1bPMjP7nAYrgFmEw1SUku1YSA7wz6pMUHJJndczOESdjAE4DeqNQb+fj3r++sDjsSy8NOa3gFpnpAr0ZmPxLCs9xqQa3hLeHVEjEX5JinaJk8Iqb4VDJfYLK0rr+mOsrjx5SQQ2wwiqEfa28zWuhfO5/mkblqZaa3jGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Ljn9VtnUEyifgNA3cosOzMocDUuiDW1at9hMSJ3P5c=;
 b=qv0ambQ0qHDP+5KxBE/c6ZUD7Y4mdflJLkN4p8quBstHdh1c76NNPrMUOuaoz8TnOO7lTATODgT2Xh2Zh+VPnwaBWx9cE6/yT+FVVFNSPF/fRDoShPxqRxnqZ0aLCcMpWYAp/JbsqXVvzGUVq/QwzieA/J7PWu9jNLL90KiRB2M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: [RFC XEN PATCH v8 4/5] tools: Add new function to get gsi from dev
Date: Thu, 16 May 2024 17:52:34 +0800
Message-ID: <20240516095235.64128-5-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516095235.64128-1-Jiqian.Chen@amd.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F3E:EE_|BL1PR12MB5873:EE_
X-MS-Office365-Filtering-Correlation-Id: 9ddf47c0-3f80-49be-9d42-08dc758dff0f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005|7416005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?8U0MbQPC05dg2ofgPqGVAyNxFTAehg15x6J1n1U5HjFfX+z0un4u1tNpeoXS?=
 =?us-ascii?Q?jxboY/GgEGqVJss1DuO5VsKsP52K/eMx+lR0com+fykk5rbMZHrNgkiLYuxg?=
 =?us-ascii?Q?rpd1ZzAvwq8qRWEji+/AZeLwWFBUCR425GLIingZ1BjD+EoQyNgTdt/uXcV9?=
 =?us-ascii?Q?BcOWRb/q7UoxnNhEZJgK9Gq92QvPSRkggEAn2KO8T7qp7K7DzugKNFwK0QNr?=
 =?us-ascii?Q?odAX7duhrQYyfIxxRq3YRH30RLKRIS544jMzC7afL2wg8Qvx4/YGlwpImf1z?=
 =?us-ascii?Q?pJO3WpSsbsWFQTy3N53SJQjoWJCwVzoL6gZE7ytQhFEDMIfsxGX9exf1/kQt?=
 =?us-ascii?Q?xNfF2nVKgjzhhhdC7qacnZvmMSh3XTvxZ7YYDw92BqF5DqPgfKr97JW7mq32?=
 =?us-ascii?Q?optisaHfkDFsz8X6XBjZ0fICk+CeoGuXisP2hc+OJOQbn75F3+E2D9g8Rf++?=
 =?us-ascii?Q?TqEwft9Ua0fn+GfE/xrRqmGtXFhTwXfvqALFORowGw6mSQGhQ9lXHLGlBI6b?=
 =?us-ascii?Q?w4kXwt/njoen+TIHv/f3bKW6J3t/QzXvxgHJBu5ubqxcNdXHNN9kxf1cN4pK?=
 =?us-ascii?Q?nJ0AIgGTnK4FpU5qyrcDaPWHHZmia6WQO2g36Bg3+qvyZOeGULkBWc2tg7oq?=
 =?us-ascii?Q?35VA1yN946ANoE9FDQ3M/mnz6c08HY+OcMH8azAig6lisckUe6I7MSuArlPC?=
 =?us-ascii?Q?UqU+3h9HM3TxRqw13VQpZIXAxmnW8yBV+5aPnb/1jhY5AVOYObnkSlwhPAyW?=
 =?us-ascii?Q?8BZ1dndGjANGBMR59Oqyj/B+0qjpHns+aqho27LetEFo1ILXUt7FoQ7cGq+G?=
 =?us-ascii?Q?K62/FtejCcoms6OkntdXyW73KJ7q70iMIaI94PAjOFdkluyZVWmqMZt1JLbm?=
 =?us-ascii?Q?XnS0XArCGy1+d1qQEwILXp5f33MuVOz1cRs010JgYIFIJluU7TEpTs3OOeBT?=
 =?us-ascii?Q?1XyUEX/eOkiJjuQRpedbkWnyZ7uVM36v8XLlvA8YUBOi3GZKyB8zRSowNTEJ?=
 =?us-ascii?Q?EtIUIWeyPUdvYRGJdwfDU/oCXaBLhkRyGkfS7IPtPyrlPhquqkVVQElJd7tJ?=
 =?us-ascii?Q?uJJIS2Uhp6tLLmY9rBf6sQ+TzMnW0iq+ScDD0TsyXDU79mlNQJes8t99Yc+n?=
 =?us-ascii?Q?e04X+LbEvJaouVUqn+M2nMzia3QpxQWvBHObzLw039hs25uQGH3754tEE/ZR?=
 =?us-ascii?Q?azxW77+I8Ntrq+3+Dq4Na4I/4+6Adnj0sGYuxMbxk2AQZFUShX/mcU3F2vGj?=
 =?us-ascii?Q?EnCckLXefh7BD5LyXlGjrHN2Kf4WtcwRksYZTryqPs/8lFlsPxltbwGcNO7G?=
 =?us-ascii?Q?cmnzEVM/7wrCzq+uV19YEWgm1OwP4PWuItfyiM7TodFUcdh00O2Wa1y75LAP?=
 =?us-ascii?Q?gG+devM=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:53:10.9350
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ddf47c0-3f80-49be-9d42-08dc758dff0f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F3E.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5873

In PVH dom0, it uses the linux local interrupt mechanism,
when it allocs irq for a gsi, it is dynamic, and follow
the principle of applying first, distributing first. And
irq number is alloced from small to large, but the applying
gsi number is not, may gsi 38 comes before gsi 28, that
causes the irq number is not equal with the gsi number.
And when passthrough a device, QEMU will use its gsi number
to do pirq mapping, see xen_pt_realize->xc_physdev_map_pirq,
but the gsi number is got from file
/sys/bus/pci/devices/<sbdf>/irq, so it will fail when mapping.
And in current codes, there is no method to get gsi for
userspace.

For above purpose, add new function to get gsi. And call this
function before xc_physdev_(un)map_pirq

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Chen Jiqian <Jiqian.Chen@amd.com>
---
 tools/include/xen-sys/Linux/privcmd.h |  7 +++++++
 tools/include/xencall.h               |  2 ++
 tools/include/xenctrl.h               |  2 ++
 tools/libs/call/core.c                |  5 +++++
 tools/libs/call/libxencall.map        |  2 ++
 tools/libs/call/linux.c               | 15 +++++++++++++++
 tools/libs/call/private.h             |  9 +++++++++
 tools/libs/ctrl/xc_physdev.c          |  4 ++++
 tools/libs/light/libxl_pci.c          | 23 +++++++++++++++++++++++
 9 files changed, 69 insertions(+)

diff --git a/tools/include/xen-sys/Linux/privcmd.h b/tools/include/xen-sys/Linux/privcmd.h
index bc60e8fd55eb..977f1a058797 100644
--- a/tools/include/xen-sys/Linux/privcmd.h
+++ b/tools/include/xen-sys/Linux/privcmd.h
@@ -95,6 +95,11 @@ typedef struct privcmd_mmap_resource {
 	__u64 addr;
 } privcmd_mmap_resource_t;
 
+typedef struct privcmd_gsi_from_dev {
+	__u32 sbdf;
+	int gsi;
+} privcmd_gsi_from_dev_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: &privcmd_hypercall_t
@@ -114,6 +119,8 @@ typedef struct privcmd_mmap_resource {
 	_IOC(_IOC_NONE, 'P', 6, sizeof(domid_t))
 #define IOCTL_PRIVCMD_MMAP_RESOURCE				\
 	_IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
+#define IOCTL_PRIVCMD_GSI_FROM_DEV				\
+	_IOC(_IOC_NONE, 'P', 10, sizeof(privcmd_gsi_from_dev_t))
 #define IOCTL_PRIVCMD_UNIMPLEMENTED				\
 	_IOC(_IOC_NONE, 'P', 0xFF, 0)
 
diff --git a/tools/include/xencall.h b/tools/include/xencall.h
index fc95ed0fe58e..750aab070323 100644
--- a/tools/include/xencall.h
+++ b/tools/include/xencall.h
@@ -113,6 +113,8 @@ int xencall5(xencall_handle *xcall, unsigned int op,
              uint64_t arg1, uint64_t arg2, uint64_t arg3,
              uint64_t arg4, uint64_t arg5);
 
+int xen_oscall_gsi_from_dev(xencall_handle *xcall, unsigned int sbdf);
+
 /* Variant(s) of the above, as needed, returning "long" instead of "int". */
 long xencall2L(xencall_handle *xcall, unsigned int op,
                uint64_t arg1, uint64_t arg2);
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 499685594427..841db41ad7e4 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1641,6 +1641,8 @@ int xc_physdev_unmap_pirq(xc_interface *xch,
                           uint32_t domid,
                           int pirq);
 
+int xc_physdev_gsi_from_dev(xc_interface *xch, uint32_t sbdf);
+
 /*
  *  LOGGING AND ERROR REPORTING
  */
diff --git a/tools/libs/call/core.c b/tools/libs/call/core.c
index 02c4f8e1aefa..6dae50c9a6ba 100644
--- a/tools/libs/call/core.c
+++ b/tools/libs/call/core.c
@@ -173,6 +173,11 @@ int xencall5(xencall_handle *xcall, unsigned int op,
     return osdep_hypercall(xcall, &call);
 }
 
+int xen_oscall_gsi_from_dev(xencall_handle *xcall, unsigned int sbdf)
+{
+    return osdep_oscall(xcall, sbdf);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/call/libxencall.map b/tools/libs/call/libxencall.map
index d18a3174e9dc..b92a0b5dc12c 100644
--- a/tools/libs/call/libxencall.map
+++ b/tools/libs/call/libxencall.map
@@ -10,6 +10,8 @@ VERS_1.0 {
 		xencall4;
 		xencall5;
 
+		xen_oscall_gsi_from_dev;
+
 		xencall_alloc_buffer;
 		xencall_free_buffer;
 		xencall_alloc_buffer_pages;
diff --git a/tools/libs/call/linux.c b/tools/libs/call/linux.c
index 6d588e6bea8f..92c740e176f2 100644
--- a/tools/libs/call/linux.c
+++ b/tools/libs/call/linux.c
@@ -85,6 +85,21 @@ long osdep_hypercall(xencall_handle *xcall, privcmd_hypercall_t *hypercall)
     return ioctl(xcall->fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
 }
 
+int osdep_oscall(xencall_handle *xcall, unsigned int sbdf)
+{
+    privcmd_gsi_from_dev_t dev_gsi = {
+        .sbdf = sbdf,
+        .gsi = -1,
+    };
+
+    if (ioctl(xcall->fd, IOCTL_PRIVCMD_GSI_FROM_DEV, &dev_gsi)) {
+        PERROR("failed to get gsi from dev");
+        return -1;
+    }
+
+    return dev_gsi.gsi;
+}
+
 static void *alloc_pages_bufdev(xencall_handle *xcall, size_t npages)
 {
     void *p;
diff --git a/tools/libs/call/private.h b/tools/libs/call/private.h
index 9c3aa432efe2..cd6eb5a3e66f 100644
--- a/tools/libs/call/private.h
+++ b/tools/libs/call/private.h
@@ -57,6 +57,15 @@ int osdep_xencall_close(xencall_handle *xcall);
 
 long osdep_hypercall(xencall_handle *xcall, privcmd_hypercall_t *hypercall);
 
+#if defined(__linux__)
+int osdep_oscall(xencall_handle *xcall, unsigned int sbdf);
+#else
+static inline int osdep_oscall(xencall_handle *xcall, unsigned int sbdf)
+{
+    return -1;
+}
+#endif
+
 void *osdep_alloc_pages(xencall_handle *xcall, size_t nr_pages);
 void osdep_free_pages(xencall_handle *xcall, void *p, size_t nr_pages);
 
diff --git a/tools/libs/ctrl/xc_physdev.c b/tools/libs/ctrl/xc_physdev.c
index 460a8e779ce8..c1458f3a38b5 100644
--- a/tools/libs/ctrl/xc_physdev.c
+++ b/tools/libs/ctrl/xc_physdev.c
@@ -111,3 +111,7 @@ int xc_physdev_unmap_pirq(xc_interface *xch,
     return rc;
 }
 
+int xc_physdev_gsi_from_dev(xc_interface *xch, uint32_t sbdf)
+{
+    return xen_oscall_gsi_from_dev(xch->xcall, sbdf);
+}
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 96cb4da0794e..7e44d4c3ae2b 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1406,6 +1406,12 @@ static bool pci_supp_legacy_irq(void)
 #endif
 }
 
+#define PCI_DEVID(bus, devfn)\
+            ((((uint16_t)(bus)) << 8) | ((devfn) & 0xff))
+
+#define PCI_SBDF(seg, bus, devfn) \
+            ((((uint32_t)(seg)) << 16) | (PCI_DEVID(bus, devfn)))
+
 static void pci_add_dm_done(libxl__egc *egc,
                             pci_add_state *pas,
                             int rc)
@@ -1418,6 +1424,7 @@ static void pci_add_dm_done(libxl__egc *egc,
     unsigned long long start, end, flags, size;
     int irq, i;
     int r;
+    uint32_t sbdf;
     uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
     uint32_t domainid = domid;
     bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid);
@@ -1486,6 +1493,13 @@ static void pci_add_dm_done(libxl__egc *egc,
         goto out_no_irq;
     }
     if ((fscanf(f, "%u", &irq) == 1) && irq) {
+        sbdf = PCI_SBDF(pci->domain, pci->bus,
+                        (PCI_DEVFN(pci->dev, pci->func)));
+        r = xc_physdev_gsi_from_dev(ctx->xch, sbdf);
+        /* if fail, keep using irq; if success, r is gsi, use gsi */
+        if (r != -1) {
+            irq = r;
+        }
         r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq);
         if (r < 0) {
             LOGED(ERROR, domainid, "xc_physdev_map_pirq irq=%d (error=%d)",
@@ -2172,8 +2186,10 @@ static void pci_remove_detached(libxl__egc *egc,
     int  irq = 0, i, stubdomid = 0;
     const char *sysfs_path;
     FILE *f;
+    uint32_t sbdf;
     uint32_t domainid = prs->domid;
     bool isstubdom;
+    int r;
 
     /* Convenience aliases */
     libxl_device_pci *const pci = &prs->pci;
@@ -2239,6 +2255,13 @@ skip_bar:
     }
 
     if ((fscanf(f, "%u", &irq) == 1) && irq) {
+        sbdf = PCI_SBDF(pci->domain, pci->bus,
+                        (PCI_DEVFN(pci->dev, pci->func)));
+        r = xc_physdev_gsi_from_dev(ctx->xch, sbdf);
+        /* if fail, keep using irq; if success, r is gsi, use gsi */
+        if (r != -1) {
+            irq = r;
+        }
         rc = xc_physdev_unmap_pirq(ctx->xch, domid, irq);
         if (rc < 0) {
             /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722963.1127419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xnc-0000HQ-98; Thu, 16 May 2024 09:53:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722963.1127419; Thu, 16 May 2024 09:53:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xnc-0000HD-4s; Thu, 16 May 2024 09:53:20 +0000
Received: by outflank-mailman (input) for mailman id 722963;
 Thu, 16 May 2024 09:53:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7XnZ-0007bz-T2
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:17 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1d203c57-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:15 +0200 (CEST)
Received: from PA7P264CA0370.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:37c::26)
 by PH7PR12MB6420.namprd12.prod.outlook.com (2603:10b6:510:1fc::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May
 2024 09:53:12 +0000
Received: from SN1PEPF00036F42.namprd05.prod.outlook.com
 (2603:10a6:102:37c:cafe::39) by PA7P264CA0370.outlook.office365.com
 (2603:10a6:102:37c::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 09:53:11 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F42.mail.protection.outlook.com (10.167.248.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:53:10 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:53:01 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d203c57-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y1rE27mHKPAIUX1BJpa/4cvELdUqOIcj4+RK8KkrnIEyHhS92dq8wtrydkYWfKjgLRiMulJwyYEN5PkKSQHlNnrBL/yB6TwYURfuyaBraPcEGXhnASh++0hL6ds1lbOAY0D8uQ26TNXjKw64X16SDnDihY3VqCxDLMr8mg86qPfVQbMh1e7+xJVInUqXHit8SdE5BKoMcWfDkm/g1iEPUAj9Rq6Gqz0oLWZK26O/vphuFWjE381flI64HY5Ma1r1nYF56Tul57UYKDoob/cacAgHBsLGNOL1GjYqecjFXiRxPFxkRWoW7OSgDNbzL2/KbQAxkvfu5zJ/CRf5o6dkUg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cppfuA3qPefeFbRlxjFzL7F92H5/rzEberk3T4m9jZg=;
 b=fcm4xvMIShHbeX96S0BHoIqkBm0YWyC8e+8Ug8tNRfBtc03MQa3Wrk2KHZVnX+kI6QoFvdqXOahE7BiLygYfz21BIq9pAM3A9XAsoao7zlMqmtS16LRKsnnf8uLVIyw/kyZTT4hZMwMeykJvORDT5kozHlcIl2S+oyVnBQ3BdWQcXON+fjhgQhDwMKFUSVKWonpXvCnVqjnHR+VAnEz2N3yWFPwJ+4/EFUo4OtpbMd7MLIdLDKfyKGnl3hnmUCq4lWCIf31fxYzQrPG7ApVv9c9bg0dbhqP6bTMd1wysdlV1DN6aBq4+dfF025GaUcjDiwyUgK3NdQco/IYjR+yJrQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cppfuA3qPefeFbRlxjFzL7F92H5/rzEberk3T4m9jZg=;
 b=xWXWI5fIIGkwBGQxGEcLOX6B7MVYi09ZSUHxS7kcOvKVHNoXIg3MqqgBvv9dfWyBmpqnCCibRdupi/djZ0jVodEW2Fuz1wiEGO3f0GwauWlilSK9Ois2zrgBtdL1tZWEwHMlzXDQDB1HShfVVLQtu+21SqeXHVKk7BwKhbYhOgU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: [XEN PATCH v8 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
Date: Thu, 16 May 2024 17:52:33 +0800
Message-ID: <20240516095235.64128-4-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516095235.64128-1-Jiqian.Chen@amd.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F42:EE_|PH7PR12MB6420:EE_
X-MS-Office365-Filtering-Correlation-Id: 7f640649-82d3-4fd6-c0d1-08dc758dfee8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|7416005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?3jq60muSCtN8IshWF/GnKWzVw89ox2+Mdddu/BemG+6m1WzKWwIvquQNPp1g?=
 =?us-ascii?Q?h/nW0U/t82uLkQcitKlATvcUX2gPHimkKFZKbKhuVfiDtZnmgclhRCmBvDF/?=
 =?us-ascii?Q?2Xujc8IXLZZGLtUhe05XPshI7IfnfyTnyFd2oSapBJC8ujfXTDX4FtWXI2w5?=
 =?us-ascii?Q?WC5S23fJ83ZjkwZIeQvi97lFyFDA4UGyMY17RMF49bnGkTcoXwq5CTtkrF46?=
 =?us-ascii?Q?gdYLsZzbGgehBqb5IqJEEnyTThICO6wGojK2ye2e3c7QzaGTHQHNgiPcdmId?=
 =?us-ascii?Q?zO8l1S/CXqLSjj+cX2JzC8MJRzQwdBd+RgSfJ3O4GQ+lT3EhUunBWdveomRV?=
 =?us-ascii?Q?r5JfZ+9E3ll8N6RZstFiyth81pd3B4gk34wjrYg2d3HBzYIg+1aoD8zHu1+k?=
 =?us-ascii?Q?yICHHhh/m6MNA69lJmYkI/0rqV5A8LUDHgOfuuiGGmFOCNuenJJRPNYmDnn0?=
 =?us-ascii?Q?fCq755yWU7VhI6jgaagtPLesaEtR4SFcN86ANPR0rjM59mlkocRj+3hzGlLB?=
 =?us-ascii?Q?YUNZdpOvgnor6u4IyqKGKEFh1KzXBglf/eeeU6dZ2BAfRZDC5MRWTaT7FImT?=
 =?us-ascii?Q?qn00h4ucms/gZI49nVPPiqQGqxHtsaRBmWuNl+726vTXAP/EtzZwayBvAfoc?=
 =?us-ascii?Q?5xxcC/iNJ5saWm0eNaOxPONJyQcL0WlD89ApBzKrG1HJMHeZVcFRtmaPh5SU?=
 =?us-ascii?Q?2PQO9RU5Rn3R040aMyhvG/2PMJZLgtwGEIjMS3l6KV/uqilqLuG0Ez+/pqP1?=
 =?us-ascii?Q?e8U86qmp2U4e9r/QS3N6BFKd+i+xDPPyKOtgVZe9HodlaEtT+j8kDGL0pSpf?=
 =?us-ascii?Q?XaQdvfb9UP7W9wMiKFujZ8ZsqNeMjX3J2f2rNVPDbZxS/zKd6QlPJSEoaBD5?=
 =?us-ascii?Q?D7wzIy9W1xDAINLpSN0m3ZMNAHBG/B61ibr1n7C57wz5CAB5TlJyMbyVy03l?=
 =?us-ascii?Q?boEo768ERjd2OlcqbKFAkx5908dMF8dGZIsn8cAs0v/cywccmrRCDQThamKM?=
 =?us-ascii?Q?lmXBr2Cj0J+1F6gYdDaPqSno5rkVjMCp/A+EtkFGAdSDjiGz/7BZOKdwjw+r?=
 =?us-ascii?Q?MrzFEsUz28Rw1UVZp0J5IAD/olqNGMFGAQU229a1XvBLREwE4qdjL1rCc8d/?=
 =?us-ascii?Q?TzJ+Zyh1mdvI6gz9CrMiX6XWdG6pJDt8s3qgq7GIsj+JQ7GWUzTpnAfo2OSB?=
 =?us-ascii?Q?AnFihhrNZBPXJF4J8Iz7hUhYyW7h9rHWSEnb0J6rzkwrl5D2esyGLuYLm++t?=
 =?us-ascii?Q?reC8MTSkY9N5MmdMXuY+PoOWtzoEoYK9RHlrtLgnB9Y/ZrSYFHQlR52GI36n?=
 =?us-ascii?Q?+W5aIB3lpwykEtPz1u80S9kCcKpDZteOWgeKe3cLVCv2VlpH5seKbgSTdIP2?=
 =?us-ascii?Q?Ht+Z//1yZRQIH5GaNFZR8RHfH9U0?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(7416005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:53:10.6647
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f640649-82d3-4fd6-c0d1-08dc758dfee8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F42.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6420

On PVH dom0, the gsis don't get registered, but
the gsi of a passthrough device must be configured for it to
be able to be mapped into a hvm domU.
On Linux kernel side, it calles PHYSDEVOP_setup_gsi for
passthrough devices to register gsi when dom0 is PVH.
So, add PHYSDEVOP_setup_gsi for above purpose.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
---
 xen/arch/x86/hvm/hypercall.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index d49fb8b548a3..98e3c6b176ff 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -76,6 +76,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_unmap_pirq:
         break;
 
+    case PHYSDEVOP_setup_gsi:
+        if ( !is_hardware_domain(currd) )
+            return -EOPNOTSUPP;
+        break;
+
     case PHYSDEVOP_eoi:
     case PHYSDEVOP_irq_status_query:
     case PHYSDEVOP_get_free_pirq:
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:53:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:53:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722964.1127429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xnf-0000dg-OK; Thu, 16 May 2024 09:53:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722964.1127429; Thu, 16 May 2024 09:53:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xnf-0000dT-KY; Thu, 16 May 2024 09:53:23 +0000
Received: by outflank-mailman (input) for mailman id 722964;
 Thu, 16 May 2024 09:53:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7Xne-0007bz-3w
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:53:22 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1fa43e0b-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:53:20 +0200 (CEST)
Received: from PH0PR07CA0076.namprd07.prod.outlook.com (2603:10b6:510:f::21)
 by SA3PR12MB8438.namprd12.prod.outlook.com (2603:10b6:806:2f6::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 09:53:14 +0000
Received: from SN1PEPF00036F3D.namprd05.prod.outlook.com
 (2603:10b6:510:f:cafe::54) by PH0PR07CA0076.outlook.office365.com
 (2603:10b6:510:f::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30 via Frontend
 Transport; Thu, 16 May 2024 09:53:14 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F3D.mail.protection.outlook.com (10.167.248.21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 09:53:13 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 04:53:09 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1fa43e0b-136a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=en+yW7As1fcorNDSSKNdqp/lo4/ntiAwgEZsAEfxu0oUpnPGcGuk3Id5RiUJzn7deIGBXY/HTE+qGwWk360k8s4sawPgUgTtpaHnkwjaXp/jH13smz72YSrH6VwZWlHAEkW0qB0PJh0ugH2LTP+OoR2xzqkPVi5sB2yX9O68x3lRDerAYDBl5ULvZGexurbgFh0BXKJdD1BtWkb0GA64nRwoFh2tTU1T39AF3xePURCyPlQ9yf5Qi9uNitfZVCO2dkMLcKktXc/+GslO2xNP4JQGu3LBNtuaOeG16VJjSli7th3zVGFed2bbpy7gowTVJl1pDdQozr1CQeYC5cV0XQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=+PVXKQu58hDwgfbrYDE/kcfnEXNdd+32q921wt8QLfU=;
 b=lHkeH8k6mbKH92U2607Xz0G2Hjm42ShQQYelsYxbjn6wCRdReEaLGMApvNtyIKo/jISMHhEtdkNOnoGrEBeEkkv/C/SaO+umcwtawEas/ssS/a9H0yWqpQL7xF26yxLcXWU4KDlzkA1d4bIS61Me3VKtUk9EkfR/rykS+sBUWwaJwWNYS4dOxxv1auaJPpWYPwcRCK7wozrY26oOev5TpXVSEUVP3TrzVTOu/B1xky3DmwnBqBjCqr1rq3PfJWwInCPQivIX4+zOQEt9hDyEWSi1MexxwvLf628As2tKU+u9FxQpK4fVzS1WMl/IIhhvUackp4A65IXP4KLhs3vbDg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+PVXKQu58hDwgfbrYDE/kcfnEXNdd+32q921wt8QLfU=;
 b=ZaHjO+b19cD9IVru6l5lmSlLGZJ6CY/Gt/u/86G7RLUdttvyIN7pZrdKo4VMrbltn12lRAs2s3u8Hd6q1+QAqg0MkCALMXVKRS7/qy74Tms3yecnUTI+BtrkguMUArhLKMBKql2NjlgqjZltzokQHA1wT2ovTadqkYbf0MQmgGM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony@xenproject.org>, "Anthony
 PERARD" <anthony.perard@cloud.com>, Juergen Gross <jgross@suse.com>, "Daniel
 P . Smith" <dpsmith@apertussolutions.com>, Stewart Hildebrand
	<Stewart.Hildebrand@amd.com>, Huang Rui <Ray.Huang@amd.com>, Jiqian Chen
	<Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to grant gsi
Date: Thu, 16 May 2024 17:52:35 +0800
Message-ID: <20240516095235.64128-6-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516095235.64128-1-Jiqian.Chen@amd.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F3D:EE_|SA3PR12MB8438:EE_
X-MS-Office365-Filtering-Correlation-Id: 6e1f936c-5b6c-4433-bbde-08dc758e00dd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|7416005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?PTOxP3HXRO/oXx3ZfDWkb+W12ioUwyLKMwH2Su8T48UTKyDOU7IeoDOmesBZ?=
 =?us-ascii?Q?nAqZzMLWEx8SkewNpInH5oQhZqPyA3b/mm3LJih/oxWY1B3jHQSIqdlUIZ2p?=
 =?us-ascii?Q?kBZfZ7+doHTtlMeKCqx3i365RjD1Po0qc/6aEd6UhwPhHeVMjHY7F3U6i1Iy?=
 =?us-ascii?Q?L+XB1PDkuwZgCaeZf3EED5DLDH+aw/xf0mC5iZEcl1cHAXrd68rRFjZ6JkX8?=
 =?us-ascii?Q?wj3XZpWrx5Pw7P52RwerxWWP4z9p/v3ZsXwVfMdeTf6qZ8mndSF3lusvXAHa?=
 =?us-ascii?Q?REQiqNw8DuWIZWp3XgaKMekxYp717N2XIX2QGWkcyl3vcLzK1d7050JH5MwX?=
 =?us-ascii?Q?F+HX+Q0vwc55bDKFvwOR0N8hQcvmKizM8GT8yfui24e7j3nlaSYFl3hxeVUy?=
 =?us-ascii?Q?qq/hA17aFCMk/HA7ux9SSNqv9nV4TAbKeb/PR19PucSaMk8UEL4VBrxM10Y1?=
 =?us-ascii?Q?ncKzdu4Oaq51Q9jhgQyv1G/YgoDMxpiS1ICnFiR8eADetaGpA3M4u0FuG/Lx?=
 =?us-ascii?Q?p3bOVDQPix7XrAIHkZ+y74AEYOCvaDbzqd3TYu14L1xH95VkrnzQBwWO6Nxs?=
 =?us-ascii?Q?6PofegC99j25FoX74FsDTMDxcgZbNsKl9fPPCZcxq8wFlA4fDH5h5MP//hwQ?=
 =?us-ascii?Q?yz41K4+hkstjaLU0DQtZWyPHz7utsP6tQx6oagoIkMkR7HxqbZga2jWU6SGD?=
 =?us-ascii?Q?6YIFGR6/xygul93PgSxbMcQvEEaZWxPEotxyY8HH6V10qSpDzixbVydLOqF+?=
 =?us-ascii?Q?nm5lwJR3zUbb7NfcYAhLB4KtxU98L0Chn2jkFh6hmL5Eh5c6O+873NZ8NkR7?=
 =?us-ascii?Q?N7Ki8efBSjYhSaC2L625s0blxh/wh7UWL/+umwMUWK6HL5jYIjKKIS6qi4YI?=
 =?us-ascii?Q?qKPYgrjd0s1N3D7a6g+cwBbQNn/6mXsNGHSv4l+nZDNLxAFyDmVIdCmD0Egn?=
 =?us-ascii?Q?CaTUYeROEw9Rjt4r1RmR5E/31pPhfaTwlLgfQjqblJvhDakXfF/ltqr7jEtj?=
 =?us-ascii?Q?FCgBbHv8/+4AUcKLAEcSUeDiFH7K18bS8lC0dZHVWkA6sw3nEqnrF1INRPMq?=
 =?us-ascii?Q?rbv3KWJ+8Vrq7qaU2tMUbzvhE42+u65qYukyFnfQNv/3+xWMCNt9It6vHh2+?=
 =?us-ascii?Q?r5rjgCmD+1XhKnin7JrvBv3JhzeRrr6hHqu9n7VY6a73wW3jKzUBIUUKoLFa?=
 =?us-ascii?Q?Zbl1rZIKSZYNGB2L4TTlU5fXXYVQfyj7kVj6xhqETlk3BUotVw5cCIco99ce?=
 =?us-ascii?Q?0hDiJqVSw1Rn18YYg0OgJ3jQV6oU0JXAVg/AzLoRHMRMCEmJXf2NyVk4+yy8?=
 =?us-ascii?Q?ZcvSysWnQHyNTjMRvWQFT24fJ5Bqta91pktxZdUtwxCawfMNaIyhEUfEunYL?=
 =?us-ascii?Q?erJ5kFQ=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(7416005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 09:53:13.9480
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e1f936c-5b6c-4433-bbde-08dc758e00dd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F3D.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB8438

Some type of domain don't have PIRQ, like PVH, when
passthrough a device to guest on PVH dom0, callstack
pci_add_dm_done->XEN_DOMCTL_irq_permission will failed
at domain_pirq_to_irq.

So, add a new hypercall to grant/revoke gsi permission
when dom0 is not PV or dom0 has not PIRQ flag.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
---
 tools/include/xenctrl.h      |  5 +++
 tools/libs/ctrl/xc_domain.c  | 15 ++++++++
 tools/libs/light/libxl_pci.c | 72 ++++++++++++++++++++++++++++--------
 xen/arch/x86/domctl.c        | 31 ++++++++++++++++
 xen/include/public/domctl.h  |  9 +++++
 xen/xsm/flask/hooks.c        |  1 +
 6 files changed, 117 insertions(+), 16 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 841db41ad7e4..c21a79d74be3 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1382,6 +1382,11 @@ int xc_domain_irq_permission(xc_interface *xch,
                              uint32_t pirq,
                              bool allow_access);
 
+int xc_domain_gsi_permission(xc_interface *xch,
+                             uint32_t domid,
+                             uint32_t gsi,
+                             bool allow_access);
+
 int xc_domain_iomem_permission(xc_interface *xch,
                                uint32_t domid,
                                unsigned long first_mfn,
diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index f2d9d14b4d9f..8540e84fda93 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -1394,6 +1394,21 @@ int xc_domain_irq_permission(xc_interface *xch,
     return do_domctl(xch, &domctl);
 }
 
+int xc_domain_gsi_permission(xc_interface *xch,
+                             uint32_t domid,
+                             uint32_t gsi,
+                             bool allow_access)
+{
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_gsi_permission,
+        .domain = domid,
+        .u.gsi_permission.gsi = gsi,
+        .u.gsi_permission.allow_access = allow_access,
+    };
+
+    return do_domctl(xch, &domctl);
+}
+
 int xc_domain_iomem_permission(xc_interface *xch,
                                uint32_t domid,
                                unsigned long first_mfn,
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 7e44d4c3ae2b..1d1b81dd2844 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1412,6 +1412,37 @@ static bool pci_supp_legacy_irq(void)
 #define PCI_SBDF(seg, bus, devfn) \
             ((((uint32_t)(seg)) << 16) | (PCI_DEVID(bus, devfn)))
 
+static int pci_device_set_gsi(libxl_ctx *ctx,
+                              libxl_domid domid,
+                              libxl_device_pci *pci,
+                              bool map,
+                              int *gsi_back)
+{
+    int r, gsi, pirq;
+    uint32_t sbdf;
+
+    sbdf = PCI_SBDF(pci->domain, pci->bus, (PCI_DEVFN(pci->dev, pci->func)));
+    r = xc_physdev_gsi_from_dev(ctx->xch, sbdf);
+    *gsi_back = r;
+    if (r < 0)
+        return r;
+
+    gsi = r;
+    pirq = r;
+    if (map)
+        r = xc_physdev_map_pirq(ctx->xch, domid, gsi, &pirq);
+    else
+        r = xc_physdev_unmap_pirq(ctx->xch, domid, pirq);
+    if (r)
+        return r;
+
+    r = xc_domain_gsi_permission(ctx->xch, domid, gsi, map);
+    if (r && errno == EOPNOTSUPP)
+        r = xc_domain_irq_permission(ctx->xch, domid, gsi, map);
+
+    return r;
+}
+
 static void pci_add_dm_done(libxl__egc *egc,
                             pci_add_state *pas,
                             int rc)
@@ -1424,10 +1455,10 @@ static void pci_add_dm_done(libxl__egc *egc,
     unsigned long long start, end, flags, size;
     int irq, i;
     int r;
-    uint32_t sbdf;
     uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
     uint32_t domainid = domid;
     bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid);
+    int gsi;
 
     /* Convenience aliases */
     bool starting = pas->starting;
@@ -1485,6 +1516,19 @@ static void pci_add_dm_done(libxl__egc *egc,
     fclose(f);
     if (!pci_supp_legacy_irq())
         goto out_no_irq;
+
+    r = pci_device_set_gsi(ctx, domid, pci, 1, &gsi);
+    if (gsi >= 0) {
+        if (r < 0) {
+            rc = ERROR_FAIL;
+            LOGED(ERROR, domainid,
+                  "pci_device_set_gsi gsi=%d (error=%d)", gsi, errno);
+            goto out;
+        } else {
+            goto process_permissive;
+        }
+    }
+    /* if gsi < 0, keep using irq */
     sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                 pci->bus, pci->dev, pci->func);
     f = fopen(sysfs_path, "r");
@@ -1493,13 +1537,6 @@ static void pci_add_dm_done(libxl__egc *egc,
         goto out_no_irq;
     }
     if ((fscanf(f, "%u", &irq) == 1) && irq) {
-        sbdf = PCI_SBDF(pci->domain, pci->bus,
-                        (PCI_DEVFN(pci->dev, pci->func)));
-        r = xc_physdev_gsi_from_dev(ctx->xch, sbdf);
-        /* if fail, keep using irq; if success, r is gsi, use gsi */
-        if (r != -1) {
-            irq = r;
-        }
         r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq);
         if (r < 0) {
             LOGED(ERROR, domainid, "xc_physdev_map_pirq irq=%d (error=%d)",
@@ -1519,6 +1556,7 @@ static void pci_add_dm_done(libxl__egc *egc,
     }
     fclose(f);
 
+process_permissive:
     /* Don't restrict writes to the PCI config space from this VM */
     if (pci->permissive) {
         if ( sysfs_write_bdf(gc, SYSFS_PCIBACK_DRIVER"/permissive",
@@ -2186,10 +2224,10 @@ static void pci_remove_detached(libxl__egc *egc,
     int  irq = 0, i, stubdomid = 0;
     const char *sysfs_path;
     FILE *f;
-    uint32_t sbdf;
     uint32_t domainid = prs->domid;
     bool isstubdom;
     int r;
+    int gsi;
 
     /* Convenience aliases */
     libxl_device_pci *const pci = &prs->pci;
@@ -2245,6 +2283,15 @@ skip_bar:
     if (!pci_supp_legacy_irq())
         goto skip_legacy_irq;
 
+    r = pci_device_set_gsi(ctx, domid, pci, 0, &gsi);
+    if (gsi >= 0) {
+        if (r < 0) {
+            LOGED(ERROR, domainid,
+                  "pci_device_set_gsi gsi=%d (error=%d)", gsi, errno);
+        }
+        goto skip_legacy_irq;
+    }
+    /* if gsi < 0, keep using irq */
     sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                            pci->bus, pci->dev, pci->func);
 
@@ -2255,13 +2302,6 @@ skip_bar:
     }
 
     if ((fscanf(f, "%u", &irq) == 1) && irq) {
-        sbdf = PCI_SBDF(pci->domain, pci->bus,
-                        (PCI_DEVFN(pci->dev, pci->func)));
-        r = xc_physdev_gsi_from_dev(ctx->xch, sbdf);
-        /* if fail, keep using irq; if success, r is gsi, use gsi */
-        if (r != -1) {
-            irq = r;
-        }
         rc = xc_physdev_unmap_pirq(ctx->xch, domid, irq);
         if (rc < 0) {
             /*
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 9a72d57333e9..9b8a08b2a81d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -237,6 +237,37 @@ long arch_do_domctl(
         break;
     }
 
+    case XEN_DOMCTL_gsi_permission:
+    {
+        unsigned int gsi = domctl->u.gsi_permission.gsi;
+        int allow = domctl->u.gsi_permission.allow_access;
+
+        if ( is_pv_domain(current->domain) || has_pirq(current->domain) )
+        {
+            ret = -EOPNOTSUPP;
+            break;
+        }
+
+        if ( gsi >= nr_irqs_gsi )
+        {
+            ret = -EINVAL;
+            break;
+        }
+
+        if ( !irq_access_permitted(current->domain, gsi) ||
+             xsm_irq_permission(XSM_HOOK, d, gsi, allow) )
+        {
+            ret = -EPERM;
+            break;
+        }
+
+        if ( allow )
+            ret = irq_permit_access(d, gsi);
+        else
+            ret = irq_deny_access(d, gsi);
+        break;
+    }
+
     case XEN_DOMCTL_getpageframeinfo3:
     {
         unsigned int num = domctl->u.getpageframeinfo3.num;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b08..47e95f9ee824 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -447,6 +447,13 @@ struct xen_domctl_irq_permission {
 };
 
 
+/* XEN_DOMCTL_gsi_permission */
+struct xen_domctl_gsi_permission {
+    uint32_t gsi;
+    uint8_t allow_access;    /* flag to specify enable/disable of x86 gsi access */
+};
+
+
 /* XEN_DOMCTL_iomem_permission */
 struct xen_domctl_iomem_permission {
     uint64_aligned_t first_mfn;/* first page (physical page number) in range */
@@ -1277,6 +1284,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_gsi_permission                87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1299,6 +1307,7 @@ struct xen_domctl {
         struct xen_domctl_setdomainhandle   setdomainhandle;
         struct xen_domctl_setdebugging      setdebugging;
         struct xen_domctl_irq_permission    irq_permission;
+        struct xen_domctl_gsi_permission    gsi_permission;
         struct xen_domctl_iomem_permission  iomem_permission;
         struct xen_domctl_ioport_permission ioport_permission;
         struct xen_domctl_hypercall_init    hypercall_init;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 5e88c71b8e22..a5b134c91101 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -685,6 +685,7 @@ static int cf_check flask_domctl(struct domain *d, int cmd)
     case XEN_DOMCTL_shadow_op:
     case XEN_DOMCTL_ioport_permission:
     case XEN_DOMCTL_ioport_mapping:
+    case XEN_DOMCTL_gsi_permission:
 #endif
 #ifdef CONFIG_HAS_PASSTHROUGH
     /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 09:55:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 09:55:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722972.1127438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xpq-0002RB-4g; Thu, 16 May 2024 09:55:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722972.1127438; Thu, 16 May 2024 09:55:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xpq-0002R4-21; Thu, 16 May 2024 09:55:38 +0000
Received: by outflank-mailman (input) for mailman id 722972;
 Thu, 16 May 2024 09:55:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Xpo-0002Qy-FX
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 09:55:36 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 703aa883-136a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 11:55:34 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59a5f81af4so277263566b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 02:55:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cd85sm978438466b.15.2024.05.16.02.55.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 02:55:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 703aa883-136a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715853334; x=1716458134; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bPEWXnJFwSWKDG4uety0TCU2Veg1SqzueaAjNBWUjN8=;
        b=W8Q7JezUr6Udk4GD4ufThdcZBBOwFReJBJq6jV9953MooQHwVO9ytP/dolR7kt1iAr
         7J5zV3J/TxJ/tbnfN7O+zOLLxp2gYpceRegVY0NfE/RUesQx/2xfmxon2Wfrlh/sJUgi
         uFN0tWEzAXdhR92cQlAXcsg6Ff//t7OEPYf2JVG3nzQQWoCXMu+Zbkc9Y64cOl8oD5Pu
         M6MWVZ6FZE4st/CyCMhgeZloyUS9afn6m1gR2oKP0CHuES7SSi/Nshgcm7AO1bIGbY6q
         v2u8ZyRQV9m5ZBoraSe/W/wkWUtml92SNfTcA033IOTviRsWKNjWHfCZjBU+beNM8Gpq
         QhLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715853334; x=1716458134;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bPEWXnJFwSWKDG4uety0TCU2Veg1SqzueaAjNBWUjN8=;
        b=OH5TUFqUcnC2bA5/dsGtFJZGnhNLWv/XZGpB2qT78d1UEfwU2dXeGliS2Z5ZsETuUc
         nbmQblFsGzaY6/vGaEjThNTLtWf88nnYHaarxgr0LB+eiU1X1bwAyS3T52Fo3zmQUWws
         N6T6aTIWyHbHzrG49I1v4F0cLJJmdrqzxskLpKvyPcAV7fD02BYpb53TbrzvO+/DbhO4
         7KvhsNzD8CF+9bLTTa6xwonlFO4zwJDP9XBYFbgwK7qgOtd7njnAw+lwzC4h5WaGLCT3
         FoIs5Fwf6/VWdc3GBObf8SmeJxEjPGSCq4ZzLU8S4fRBas0XQm+SFGhll4CKi7U75qTZ
         fRew==
X-Forwarded-Encrypted: i=1; AJvYcCVj1SA7UgOhy0KYOCsIzq3LwSerdl+AZ4hyqnYJZrG1Cibfhn0yTWVYMcagewT66ntZw5/lhotlGWsLw3CTl5HLLy3FcgiwRRklaTn0+mU=
X-Gm-Message-State: AOJu0YytBFE2qEKwbYxO/wbrKmJyYvI0L2hNF+qVw9wnT+mOQKAMVTyl
	eaHwvQWapm+p3kA046IdjiQ1Hvcu5ZrDaNU97KK61ekyOYS8SxPgwdZojkXHNg==
X-Google-Smtp-Source: AGHT+IEXv69L8au3xd3YE4PBVv1qBalmdacjFNWTQwVLZTnmknG0XF6Qbeji/Zns3Y7BrjBEuVlMaQ==
X-Received: by 2002:a17:906:3197:b0:a59:cf38:5339 with SMTP id a640c23a62f3a-a5a2d55a6e6mr1317821666b.6.1715853333761;
        Thu, 16 May 2024 02:55:33 -0700 (PDT)
Message-ID: <beff1c5b-8a6c-41b9-a451-e5f656c654b6@suse.com>
Date: Thu, 16 May 2024 11:55:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 6/6] x86/MCE: optional build of AMD/Intel MCE code
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <def311f3206f29f919a921d459442641fc293031.1715673586.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <def311f3206f29f919a921d459442641fc293031.1715673586.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.05.2024 10:28, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/cpu/mcheck/mce.c
> +++ b/xen/arch/x86/cpu/mcheck/mce.c
> @@ -38,6 +38,10 @@ DEFINE_PER_CPU_READ_MOSTLY(unsigned int, nr_mce_banks);
>  unsigned int __read_mostly firstbank;
>  unsigned int __read_mostly ppin_msr;
>  uint8_t __read_mostly cmci_apic_vector;
> +bool __read_mostly cmci_support;
> +
> +/* If mce_force_broadcast == 1, lmce_support will be disabled forcibly. */
> +bool __read_mostly lmce_support;

While moving these, did you consider switching to __ro_after_init? Preferably
with that
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722991.1127465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxi-0005Vx-GZ; Thu, 16 May 2024 10:03:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722991.1127465; Thu, 16 May 2024 10:03:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxi-0005VE-Af; Thu, 16 May 2024 10:03:46 +0000
Received: by outflank-mailman (input) for mailman id 722991;
 Thu, 16 May 2024 10:03:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxh-0005ER-3k
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:45 +0000
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2407::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9305f67a-136b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:03:42 +0200 (CEST)
Received: from BYAPR02CA0030.namprd02.prod.outlook.com (2603:10b6:a02:ee::43)
 by SA1PR12MB8161.namprd12.prod.outlook.com (2603:10b6:806:330::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Thu, 16 May
 2024 10:03:39 +0000
Received: from CO1PEPF000042AC.namprd03.prod.outlook.com
 (2603:10b6:a02:ee:cafe::94) by BYAPR02CA0030.outlook.office365.com
 (2603:10b6:a02:ee::43) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 10:03:36 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000042AC.mail.protection.outlook.com (10.167.243.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:36 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:35 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:35 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:34 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9305f67a-136b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Skzxkw/gc/elR/dfcS6eEaw8+V/2BeVfRfuS34yE2wgg11K3Hag3bZQ3xPAA53zgBBRDY4YD6xI/RKOgUbsEsXVwLZ/F7/lXlhTYW092/5woUx9b2DKD39pTdnk6LQGpaPrGPbWgrqY2Gar/yckqpW44OBFTSfiOrwsG9GFPYPguDXFAx6KbbtSonRJ6ld7ZswgwT3Q4ihJ4yxcXKxvoDFaH1RInF2OzAkTsSkqiH6yYWxHJaWfik7Esy6ezB4ugHRBEBj4p2RxGbBj/mu9mivT5wLKPgkACL+yaUUMI7BXRw3nCNyE27L8NKEE5oUNx8fBKp/YKYgWshzyRDYgu5w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OSVZTcTs9mm7sB/dXnHKN/ALb10cEIdfRSBxLdDzF1s=;
 b=O2EvHeSWLPSXIXPaLyI7eBpUdXuVw2Jd/qQZXZX2GIhEYqQMu8jBHyiY+lj+6sgSTtsxxHPDvGdE6nfcYAEshhuIf8R7rsnvQ/TR4PBtJ/MNnz/mPUaKgLRy0DrlW8dZS+54nnEPVUG1LZjAksNFXwrIBp/SxghPJ34uf5iwNH2vHnWHDcJsvtwKn+Svh8BJbLXOnD2sI+WG1+rvcaaPW2vZAvDN9KaTIL+bQd2EcAxIcamLZSVIj0QFjqLroKmL5x4nj2kuxCq6OujW88bkqa7nbpeXdSwBncTcX13vg+GY0dx1cmFRQrscoUMJ2A7bzzY2LptFuGCDUmb7LnqmRg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OSVZTcTs9mm7sB/dXnHKN/ALb10cEIdfRSBxLdDzF1s=;
 b=138iIMtSB3GcV36u1I678e6tFnqagssZLIsNdtqr73aBpUCJUalMvKui+0tguH0+lfwBwz/UTP+WoM4/UZvBCk8OhD7Fn4QUNf9wrmwwe62xUJOjXHlS1KpPc63rfPvYsCtuPcHbjOCNsfo+VHLdcd+knGzNtuE9yXGkHr+rkuA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Subject: [PATCH v2 1/8] xen/common/dt-overlay: Fix lock issue when add/remove the device
Date: Thu, 16 May 2024 18:03:23 +0800
Message-ID: <20240516100330.1433265-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042AC:EE_|SA1PR12MB8161:EE_
X-MS-Office365-Filtering-Correlation-Id: ade187ed-93ef-4405-8e54-08dc758f73df
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?2oMr5RY6wH5/euGsSa4U+iE4PAOvkIApsUxYg9XGiyXP4jQaTqpI37HJC0p4?=
 =?us-ascii?Q?ya9dSamv2eHhJMzp/w5V39/4GlFxZxY/7JnscBIOjPCb/WuRKVmrnQO0FVth?=
 =?us-ascii?Q?/4dDo6XXhKPJC/NnZHE5uUwjAT6gWEznNb1nXcdw7K1OGQ8jryuQWBxlZs9j?=
 =?us-ascii?Q?MTs/s6hqwJDJCC1tfwtCcnkI5VKTSomjBGaFC5RWFtyiPLqxWcxXKVlVzKEN?=
 =?us-ascii?Q?Juy5bTyu6+cZFdgs2b50pDkyTr/PdEa9xQD+5EJycjhDkGiXKua1oxEWsEZk?=
 =?us-ascii?Q?z5K0H4QI11wHcUwR+a0G71T7aIckJ+jmcPUh2O2HI/YlMf3B+WQmsaLSNHhP?=
 =?us-ascii?Q?BvruTb3RbwtzWpWbZSO9oiNj6L8G71RnAEUS894cP5jTgRWTKtJWdQ2PH4de?=
 =?us-ascii?Q?hGZK+Qn5zqUYanBsyffucJtvOznI0VYFig3DWX9OiJrVn71gBewQcIl8gtcI?=
 =?us-ascii?Q?CmoCBYMEN9KRqKmi/oRPTfAPXPpTo0yvVFjA1jpql3dwDqpJVA7dMlWozuxi?=
 =?us-ascii?Q?J0MThH56zaffFLAQAgibqyi8FVNEBMWotK1LqYGDDIwjZhlx+LRsAu5bg2vy?=
 =?us-ascii?Q?EYcUcYFSjtw/lrmvc64y0rDL5PNY8bs+Ct8TaOQKXENRSj9w7deAtpfiP9MD?=
 =?us-ascii?Q?gS8gdNpOtBeliqtUGm2CWcS73CINvujyhIEJV0k+XY1kyR7xe19oIN0Osyqj?=
 =?us-ascii?Q?cCqDphgmyUk6Y6/MjHxrkIiDmx2cJi9jmnuMVSdGMdNOH2Q0miuCqqHsX0YH?=
 =?us-ascii?Q?GBZvpjhoKoOMbZ00hM80jpZApKQAHvBFGhzfmIUW23uRydtLMGTg7LWZBLPw?=
 =?us-ascii?Q?+LVpLX5RKToNBYkLp9w3P5jV/5jNHCRKcoBYV6fEUVRsTBtZHODcjEHdPn0r?=
 =?us-ascii?Q?nJ3rrroc7QOi9hEfka4iUn+K8FCxmduKaMdBnNtaG2FhQgIFrXMkXwQWgcA7?=
 =?us-ascii?Q?hXP4KUHbZr27+YUW1/EXmP40HIR+P2H7us53pAspGjSg03yChtqS2o0iVryo?=
 =?us-ascii?Q?QVBYAU7DoIbk0ik+L7AQgC8a8SfNDcvCZIKBgYUprEije0zNk6Tiy9UjHb/p?=
 =?us-ascii?Q?CR9VfEe7QN4fGDfSLINauD6P1sU9W+Mb6CENhuzKbSJKA7B9W9wLrZw9FhSf?=
 =?us-ascii?Q?sS7j07hj2mVpl9kNwcdTOAvcGTMrYQ71MNtai3WxecOVEfk+eWb1wQ3mA6av?=
 =?us-ascii?Q?FDTtgpLQChhBnsdqom7ttVMfDFPCz+Jngb8fq9Idni+8h8QkHyRvfewljS5E?=
 =?us-ascii?Q?dIw5lYpyBuZffaEaADRXPpqg8jeZLUKNpE2kCQ14SY68Ba5g9m8ulNEFUob9?=
 =?us-ascii?Q?Hai4R9g8pK/+BSt22yUWjY+/uzrBUbKqlSgy4kxJGPTyaJYhPr6boLUzm/Ch?=
 =?us-ascii?Q?Om8ACnM=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:36.3165
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ade187ed-93ef-4405-8e54-08dc758f73df
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042AC.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8161

If CONFIG_DEBUG=y, below assertion will be triggered:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
[...]
(XEN) Xen call trace:
(XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
(XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
(XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
(XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
(XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
(XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
(XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
(XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
(XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
(XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ****************************************

This is because iommu_remove_dt_device() is called without taking the
dt_host_lock. dt_host_lock is meant to ensure that the DT node will not
disappear behind back. So fix the issue by taking the lock as soon as
getting hold of overlay_node.

Similar issue will be observed in adding the dtbo:
(XEN) Assertion 'system_state < SYS_STATE_active || rw_is_locked(&dt_host_lock)'
failed at xen-source/xen/drivers/passthrough/device_tree.c:192
(XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
[...]
(XEN) Xen call trace:
(XEN)    [<00000a00002594f4>] iommu_add_dt_device+0x7c/0x17c (PC)
(XEN)    [<00000a0000259494>] iommu_add_dt_device+0x1c/0x17c (LR)
(XEN)    [<00000a0000267db4>] handle_device+0x68/0x1e8
(XEN)    [<00000a0000208ba8>] dt_overlay_sysctl+0x9d4/0xb84
(XEN)    [<00000a000027342c>] arch_do_sysctl+0x24/0x38
(XEN)    [<00000a0000231ac8>] do_sysctl+0x9ac/0xa34
(XEN)    [<00000a0000274b70>] traps.c#do_trap_hypercall+0x230/0x2dc
(XEN)    [<00000a0000276330>] do_trap_guest_sync+0x478/0x688
(XEN)    [<00000a000025e480>] entry.o#guest_sync_slowpath+0xa8/0xd8

This is because the lock is released too early. So fix the issue by
releasing the lock after handle_device().

Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Take the lock as soon as getting hold of overlay_node. Also
  release the lock after handle_device() when adding dtbo.
v1.1:
- Move the unlock position before the check of rc.
---
 xen/common/dt-overlay.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 1b197381f6..9cece79067 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -429,18 +429,24 @@ static int remove_nodes(const struct overlay_track *tracker)
         if ( overlay_node == NULL )
             return -EINVAL;
 
+        write_lock(&dt_host_lock);
+
         rc = remove_descendant_nodes_resources(overlay_node);
         if ( rc )
+        {
+            write_unlock(&dt_host_lock);
             return rc;
+        }
 
         rc = remove_node_resources(overlay_node);
         if ( rc )
+        {
+            write_unlock(&dt_host_lock);
             return rc;
+        }
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
 
-        write_lock(&dt_host_lock);
-
         rc = dt_overlay_remove_node(overlay_node);
         if ( rc )
         {
@@ -604,8 +610,6 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return rc;
         }
 
-        write_unlock(&dt_host_lock);
-
         prev_node->allnext = next_node;
 
         overlay_node = dt_find_node_by_path(overlay_node->full_name);
@@ -619,6 +623,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
         rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
                            tr->iomem_ranges,
                            tr->irq_ranges);
+        write_unlock(&dt_host_lock);
         if ( rc )
         {
             printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722990.1127459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxi-0005Tc-7e; Thu, 16 May 2024 10:03:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722990.1127459; Thu, 16 May 2024 10:03:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxi-0005TV-3y; Thu, 16 May 2024 10:03:46 +0000
Received: by outflank-mailman (input) for mailman id 722990;
 Thu, 16 May 2024 10:03:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxh-0005T3-2T
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:45 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 939f7113-136b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:03:44 +0200 (CEST)
Received: from MW4PR03CA0337.namprd03.prod.outlook.com (2603:10b6:303:dc::12)
 by BY5PR12MB4115.namprd12.prod.outlook.com (2603:10b6:a03:20f::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 10:03:38 +0000
Received: from CO1PEPF000044FC.namprd21.prod.outlook.com
 (2603:10b6:303:dc:cafe::62) by MW4PR03CA0337.outlook.office365.com
 (2603:10b6:303:dc::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 10:03:38 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:37 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:37 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:36 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:35 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 939f7113-136b-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ajw8jvLIdS0hWT6baNa0RZztzpKE35uPM2NqX3o+g4QDdMctbOyacesvJISS9V4lkMTWXxwUZc45CQtKpuq6f5BeQ068x3vwo43XtxTG3MW/ak25k61NzM0x9MFGUVUpL41A0B8nKP076xx/SqZgb3VOthMyw/ouCm2hBWRyZrahJOy7DFZPSl/alBmyjVx26Wm08D+/o/0vRIV1CSY9SjD8OhjUFICW3K85i5iemcMDrEV4DyjkDJ0aNjSr8iHm0Ou+K5JtqMcVmzpoaL64cvZDdackoWxGR4x4k/++Isj5eBYC7RJMmE2+vMmNmM+2SCqDRW6dnprWOAX+HPhJqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7coXZqUivStKOWByVVC93sA3+vqD9Aj0YK2CmeBB7tg=;
 b=m4l7p6qOF1lRreW1l3CBIqWZKzMv3E/eRQyWes6HlXU7afJOK1Os5GpzuCaaSUGhAYnC++9kNXSahM3gNStDtfr8J50Toi2GW1DJiZc21QO7V3UsfhiuIFAkAQHr8eb4bAQLZA8SWyU66qI0UQnE/48qF3+64DU+NHeO8MNjIXFLSlP+pVy5l33AZgDNDWgeAGb6MHzxZD5av5CtVneyPFtT+oQ0lt1tB8REnV7OjQipKFZBJx1nhGje82TmTISY5gehBPtGnDG3yGJBSZnbhfBnR3unTLsE6kWBYsaXcQ+ky6LdRzPpWlx7eOsPyWvQgl6VtJojMeYOETNCp0waSQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7coXZqUivStKOWByVVC93sA3+vqD9Aj0YK2CmeBB7tg=;
 b=wY5yQibSag/CEOhjV55+Z1bmmARMTm7lc2/3XCwvpWnXWfeFW0DK93DLHFe8d5SfCocjKaub9K/y9mMEK5Ynxkk2/B8tQi5jUw6E9383q8Nzbk6JUZFM9Xw3UPl0fkjpe9EeAHeHmccVMYVoOQ3XDRlIXUhS6pZPtRqrffuBpv8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Anthony PERARD <anthony.perard@cloud.com>
Subject: [PATCH v2 2/8] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Thu, 16 May 2024 18:03:24 +0800
Message-ID: <20240516100330.1433265-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|BY5PR12MB4115:EE_
X-MS-Office365-Filtering-Correlation-Id: f596d05e-10a9-408c-7a0a-08dc758f74cd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?hHCAM0qA+YQejaY8Ejkz5P90V/g78QFOFlZkGkZdpZmnCNzTPawDr+0lXK/Q?=
 =?us-ascii?Q?lOTtamM87/KlPnlc8MkZjtHP/yAcc4PN9a/D2IBRgTOcozS16oDgolr2v+uX?=
 =?us-ascii?Q?TGcBa5mFRj3nh8Y/iVq1/T+yUIJYLpRC4rHLBbulJcq+oUTbUmSSeBpCFTS9?=
 =?us-ascii?Q?ORnTVWgN45NpQwlupFDuyIPoZ3u1LSVYHl+IQDySLn7wQyqpwzv8cDxXgFdu?=
 =?us-ascii?Q?dOL0Lx9DyxDMd1t5njMLom0BcSJt4kXLhGZX8hCoLTM6mn/GZTg4ErdSLUuz?=
 =?us-ascii?Q?sPEKg6SGmgXKAJugQakVU0+PrBE0/WzB4orVQlkQFhEFQ505SBS+zf9ZClhd?=
 =?us-ascii?Q?gDcDW5GTHm2JppcJ6VeUaDeWbVq0XV0x5xzQlDefMWpgMVxfRzOQJrHgqEkh?=
 =?us-ascii?Q?bWZPQfSIl+kOTA5sOeWY6Y7waVBCbQvtpxIcI7RHp9GdPk9582wGQn2niyM7?=
 =?us-ascii?Q?LshtjNrlK9w4PcZKEAiVOtt6Umty4ukkJJJAppExNqn9RfCwEu0t4SW3bpVF?=
 =?us-ascii?Q?BwOM6kMk4KxjkSx4ycDQBsJcqoZ9Bmy8v2Sj8P5joVKNc36ndQTeLJclR7Gg?=
 =?us-ascii?Q?L3Os2MnluclWzvhp763uf9JEDW6VTZP5h5FJiB50P3AaXtzsvnU9EfqctCZZ?=
 =?us-ascii?Q?mtCkOfso2cOHlUYFs2m/tLiqp6qspX2l1xS/m40TIehXk/iGqrugXPfY4JDs?=
 =?us-ascii?Q?Odkb+qQKPxywyHqyztvW/8noMMcaF2o3u6i5W9R2JL5ayNM+3Y1Wsb+l2BHA?=
 =?us-ascii?Q?2asJFfsywu2VMVAB/yWsEMXiCvbgCX6bSsKTsNl1SfsYAVYLdTlJbIjGu10W?=
 =?us-ascii?Q?4C/owNzJr7pcD+aExUzvbhT7o6CvYC887Y2jVliRk30xtWh+OgdjB1hJQs47?=
 =?us-ascii?Q?O4tor6PA733nEoBw2P7sxwf3kFeV5UkytnwtRAICt6Fai1V6jtzlLFD/FJKy?=
 =?us-ascii?Q?c0JYfYp4nh5lHzxG1PeoUrmQzaYRzBxCvVfkAavJomHZc/J2W/LyTfU8F+da?=
 =?us-ascii?Q?aUPgv/pJqh4OF5Xf1K0ijKyWPBnze+9J9lFJmhAEVs3TzE+btbfmDrgXv7C3?=
 =?us-ascii?Q?JuGuIYXUjd5tmO4Q5RimWd2YuEzK73v8nF3sH9js55uRjKKrclEpufHIS0KE?=
 =?us-ascii?Q?JyUaR7Pf/TCAuhdR9Z7/BBND8CvdXAyskPEwFwHlceSvipLqcfvrE5IPhj0j?=
 =?us-ascii?Q?WOFFuW4kXWIt1WUlhMEas8HVQElslbUi8qvA6hF3wTJRxmt6QRCekJaxno4u?=
 =?us-ascii?Q?pt57ATXgaUfjHk1YAaTocWGZod+Keusm7AmlTCMso2v4D26om5HAWW9UnZhL?=
 =?us-ascii?Q?MtJxlvSiH69Ffxoh15YPG/B2UKTIyVsCRX4r/AdiHdCMyy25VWobm1DtJZOv?=
 =?us-ascii?Q?Dn5W6FE=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:37.8445
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f596d05e-10a9-408c-7a0a-08dc758f74cd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FC.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4115

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".
Add the missing "," in the cmdtable.

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony.perard@cloud.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch
---
 tools/xl/xl_cmdtable.c  | 2 +-
 tools/xl/xl_vmcontrol.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1f3c6b5897 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
       "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>"
+      "add/remove <.dtbo>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722992.1127479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxk-0005xn-K3; Thu, 16 May 2024 10:03:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722992.1127479; Thu, 16 May 2024 10:03:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxk-0005xe-H0; Thu, 16 May 2024 10:03:48 +0000
Received: by outflank-mailman (input) for mailman id 722992;
 Thu, 16 May 2024 10:03:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxj-0005ER-DS
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:47 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94896344-136b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:03:45 +0200 (CEST)
Received: from BYAPR02CA0028.namprd02.prod.outlook.com (2603:10b6:a02:ee::41)
 by SJ1PR12MB6268.namprd12.prod.outlook.com (2603:10b6:a03:455::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May
 2024 10:03:42 +0000
Received: from CO1PEPF000042AC.namprd03.prod.outlook.com
 (2603:10b6:a02:ee:cafe::21) by BYAPR02CA0028.outlook.office365.com
 (2603:10b6:a02:ee::41) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 10:03:42 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000042AC.mail.protection.outlook.com (10.167.243.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:42 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:39 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:38 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:37 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94896344-136b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iNrI7o2xjMNkq21QYDZzguE6Bz7SOkDDO4Fy5LXSna4r1/7+gARcEoHgXuJiCzaonF6lqw1opEybh7lZhGeigrlhYRbgW3pfnm9R7cZ78VzeDiXz7WWDveD7KA52AmA9y/1QIfmvAFwZzDIosAFIjLyshZYsJ/ClJx3ZOWALgpYJkUcYn7Pew9VAoEB5aSFvtJ26d7/fTzY8gdXem1+g3jD/z+HbC33fMYsF055ng0NuQ3cl1xe+Yn2QCdowDkUwVW/CTDeCdpAV4/u9wlBmSOIAS7xw5hxbbi01bqp2TZiMDDv22JFVfbAAZCmwatL4pyGSrv/O3HkVdZ+s1dHWaQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4FOmUBM5q170h0MhyoHdpU1Ky6atwZpLP5xhhuCrh8w=;
 b=N1LFR8bfk3prN+aYNEQ9M7Q/a+nG+KH16btoCgBT8w4IgCv0fJnzbz4KsiQ6PDfeYUhXBOO46WbrA2EPn6ZjD90Bk/orXgTbsmsFaIk0RjrQeEh37KE3ME3eKT1nPEWtRV6Y/qYO4qvPF/MkixKV1ruORTT1lo55gYkF6Hinq/wMh9bkOhipyE8wY8BsN1ZY9bhwPoff+Jd1B4nuDqFDOIlECK/IwzUl6wXchCGCKpzJlOwbKctWzWE6SaFkS52+RlMP7V4vHgdlDT4dI38Ume5h1shfPLyqy4Jc4a8HOkgQUc11zFYmWDPLKBv8MiWZek+Ohh0KwHYL52ES7ZN2xw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4FOmUBM5q170h0MhyoHdpU1Ky6atwZpLP5xhhuCrh8w=;
 b=JXJoBe4xTAOMCFxY81+f73Gq61Ikop2fzkTNhKKqgIn/SmcK2hnACZtYJlXKVsgI5WxJa9JOtmFX6oQ4FDWDZslhnTnLKAzQ81fee0wQ3hU7YoCFEklPDOJxOsWP6ONABC7RwTQ06/T8ajGyLkBgVULzWKPVTiBTBfo6IuCoLME=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
Date: Thu, 16 May 2024 18:03:25 +0800
Message-ID: <20240516100330.1433265-4-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042AC:EE_|SJ1PR12MB6268:EE_
X-MS-Office365-Filtering-Correlation-Id: 8c8c6576-5a73-45d8-2bf5-08dc758f7767
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?p28vscojw1AxysNMGx8aJEqRwFe5dHP0+wm4yebhfDLEBSwOCT5alRUt/amg?=
 =?us-ascii?Q?hAu91/s145djjhuge+TqvSZJ4kBDEOdKSNl/DIeXI/p2OItssMaAa5JTBUYs?=
 =?us-ascii?Q?pAli7/7nxq51fWR0ZTE3nADl4RSPgdDZ3/8LlftjnPtZwVpX4XzljWWleQX6?=
 =?us-ascii?Q?scSLZOTYPic+PoZGwmMVzck9qhJ5cIhIWjD/WDsVNl+SdZ6NNw+XmwXQEMBY?=
 =?us-ascii?Q?5o5xzjX30/PvV2XE4ywM0dPoeS9Iwg3oCPhC1Td00IUiqTlBlEzVdPPyyKqc?=
 =?us-ascii?Q?rMtPEzD6utq+t9eo+zuE62N2Fy9Su5EeQMzq/r1t1v/m3wq+JWrJE9n//mqL?=
 =?us-ascii?Q?d8AeuTFSVGb/60Ex95qXrd0ioaRGvwwyHWKooobg5GAGq1gmLyJWnPKhrSii?=
 =?us-ascii?Q?jp0RVo7Qq0/r5e2vi2X48HW6/a40LvCby1DGrBqikUfI9M04Qdb5hOQ7zz9U?=
 =?us-ascii?Q?5LJZt5nnQ/TD9ugk+xv/i1xx3i9xVPDQ6/SmBw8MijA/qKlN/ViVRey0SSCV?=
 =?us-ascii?Q?A+6TuPiWbMK5u7TIcgesfg79Ct6zPFQxDWD7n4vtrsblRuqhwKwj5FOxh2q7?=
 =?us-ascii?Q?Qr6+VsK5r/+nvagholt2ibxfTtO4agqSPusVwTA0UTMEumthvQ7BOvk5QXq+?=
 =?us-ascii?Q?NYPD8WB3DhYDLrTr7clIibk5utgekSuJOwysbiuS7lPV11cxK11S28NU4khE?=
 =?us-ascii?Q?E/rXAzag9X64tCOoTSPS0U7wOn6nSZYw9APBuLW3SBRtaA2tj/I3cMbCGBQA?=
 =?us-ascii?Q?cfVwErQh0bDEjqljAr1wLx262L8P7IAGzTc5bFXCRdyN42TVmGqQIG65c4Zg?=
 =?us-ascii?Q?ZMXWa0sWyTkXYLD9gaMD9mo2neWtZVKcMSloVkwdP3nbXlYuv7nNM/qDe7Zb?=
 =?us-ascii?Q?9KqKAuOH5OhMnMLoPaCg+zn3Gu1o49CFYuYXShKNbtsv/VIDVgOTMdEAYfst?=
 =?us-ascii?Q?yqozVHxHP5m+kgtYVctZ6XzNH6QqvCkwtMO4U+pEmvo6fbNK3I/BVIxRyXTd?=
 =?us-ascii?Q?70aQe69m+1d889gvadwl0Lv1QavskBNMqR6XgLnjD8m9eO7PbTBofrA5wKqo?=
 =?us-ascii?Q?6rY6M2q+2KxzNhFdSky8KCFeoDN/cup3AGOiy1cr4e9cGYt5Lid2yD7JKrI8?=
 =?us-ascii?Q?wHtU4vfvXhQkpBfP6iMJbIMGSOXPFzLy2lbUlTJS8tWd4TbEMCq/pluiZzZt?=
 =?us-ascii?Q?IngpgNwInN6omMzHT7QK4+lisTuTkbtv2aQqepWyKoi+7825VIW3nchUIx6z?=
 =?us-ascii?Q?Ko3iL/PpMX92v91m0vIfWZmnaQOlc8x0YoL1HsLSFlwDo7lq6NlmchUxZ7Ny?=
 =?us-ascii?Q?+JzJBRRa6eIUL7AIoEPdUnYQV3QaZJHTHun2XtydAMWYOFkKGD4pTjXDbMLm?=
 =?us-ascii?Q?pZmq8WVnkkoUopqBRz00x987xTrC?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:42.2384
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c8c6576-5a73-45d8-2bf5-08dc758f7767
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042AC.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6268

There are some use cases in which the dom0less domUs need to have
the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
example, the dynamic dtbo feature allows the domain to be assigned
a device that is behind the IOMMU at runtime. For these use cases,
we need to have a way to specify the domain will need the IOMMU
mapping at domain construction time.

Introduce a "passthrough" DT property for Dom0less DomUs following
the same entry as the xl.cfg. Currently only provide two options,
i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
construction time based on the property.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch to replace the original patch in v1:
  "[PATCH 03/15] xen/arm: Always enable IOMMU"
---
 docs/misc/arm/device-tree/booting.txt | 13 +++++++++++++
 xen/arch/arm/dom0less-build.c         |  7 +++++--
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..61f9082553 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -260,6 +260,19 @@ with the following properties:
     value specified by Xen command line parameter gnttab_max_maptrack_frames
     (or its default value if unspecified, i.e. 1024) is used.
 
+- passthrough
+
+    A string property specifying whether IOMMU mappings are enabled for the
+    domain and hence whether it will be enabled for passthrough hardware.
+    Possible property values are:
+
+    - "enabled"
+    IOMMU mappings are enabled for the domain.
+
+    - "disabled"
+    IOMMU mappings are disabled for the domain and so hardware may not be
+    passed through. This option is the default if this property is missing.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..1396a102e1 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -848,6 +848,7 @@ static int __init construct_domU(struct domain *d,
 void __init create_domUs(void)
 {
     struct dt_device_node *node;
+    const char *dom0less_iommu;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
 
@@ -895,8 +896,10 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
-             iommu_enabled )
+        if ( iommu_enabled &&
+             ((!dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
+               !strcmp(dom0less_iommu, "enabled")) ||
+              dt_find_compatible_node(node, NULL, "multiboot,device-tree")) )
             d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722989.1127449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxe-0005Ee-TM; Thu, 16 May 2024 10:03:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722989.1127449; Thu, 16 May 2024 10:03:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxe-0005EX-Px; Thu, 16 May 2024 10:03:42 +0000
Received: by outflank-mailman (input) for mailman id 722989;
 Thu, 16 May 2024 10:03:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxd-0005ER-Bf
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:41 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 90be1afc-136b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:03:39 +0200 (CEST)
Received: from MW4PR04CA0249.namprd04.prod.outlook.com (2603:10b6:303:88::14)
 by MW6PR12MB8664.namprd12.prod.outlook.com (2603:10b6:303:23c::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Thu, 16 May
 2024 10:03:35 +0000
Received: from CO1PEPF000044FB.namprd21.prod.outlook.com
 (2603:10b6:303:88:cafe::ce) by MW4PR04CA0249.outlook.office365.com
 (2603:10b6:303:88::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 16 May 2024 10:03:35 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044FB.mail.protection.outlook.com (10.167.241.201) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:34 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:34 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:33 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90be1afc-136b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xv21quXJJPUM1XQrsjr/ttbijV6QR1Y+hnZ4YOB+zg/QF1GPhQvDgI3Hqw+boUmU4YKRrXWOMZgUvUlu/iShIv7aN4EhAygQiY/8iwfstKYjqk978poPz9Kz054xwKkfJBkOVar/jLi0eQYkdndGfLvWUpvsRWFKpDH6pwX9Yw2IQqIPusZVo1XMvUlQcGIReyiz53yWbU+RrKrK6VlaYDTqS2c5+D2+8nkEAhmzLy7aJ6Aofvhx7udF+FBvgjdXjtBCgdM2oAqMulpV3CCBqa0dRwsL7tdbNhYEc9p4QkPlsuZS5Y+/O26u44N+2EICEvHLlvK6FWC0kgIf2jmOZg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=a/pgNhx9ywTPwFg7fK3LV9WnW4v2x0UNVhg62vcL7cw=;
 b=kev7jBOFws8DukOh3Tph3jjf3Q4YLXtd7pkn2kfjAVoCYm3PRhzO65gyP6RXRmfQSb5cIiRBs8D8L73+2aq0vO6WHzm+wE2iKc6NVpBP7pYNHaMSzdnWJ9YXGfHm9EnP7+AQZEcqiYEVMcT7oos3f2yxRj2aI//E0rdTHtDjLxr6ELG0ePfTSplk/cUzwBQuJukCfLegkWdfuDOeKY8425uDtyj/goGfaErm6vYXZj1L9RbIAmSwzY+J9IU+M8Bnat9FsR15CqPHm6XdCoOjkrhv8v6w0zZS7EvyJmtV8IHJLyXFTn+6jhJactdBir32+IOXeJ29aHlkBhpqHWEwBQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=a/pgNhx9ywTPwFg7fK3LV9WnW4v2x0UNVhg62vcL7cw=;
 b=p8sgbdoyRTJiNKjEQzdjsWQvwNtQfnRo1zdYpXBBG7b0ZWs2HbDXkK5Xfn8kcwqqv9N30DWA8XqzZ6dfvSGiT9cTw2Lfa5/K0F6BJ0nQUzUmYMhNkgZ2Egq6j7+pLxMQHh7pgeJm+woRqsglABPYqEWYuRAluWvMVHPTr/CJp+M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Anthony PERARD
	<anthony@xenproject.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	George Dunlap <george.dunlap@citrix.com>, "Nick Rosbrook"
	<rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>, "Andrew Cooper"
	<andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/8] Remaining patches for dynamic node programming using overlay dtbo
Date: Thu, 16 May 2024 18:03:22 +0800
Message-ID: <20240516100330.1433265-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FB:EE_|MW6PR12MB8664:EE_
X-MS-Office365-Filtering-Correlation-Id: d4b562db-f632-458a-7dd6-08dc758f72e9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|7416005|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?6X9jAmoW21R4lO9Dejfuo90UCHE8992osksi3mV4/asHAhEQzcYc/fP/1+5Y?=
 =?us-ascii?Q?yJ6LVu+jzx5YusI3o/3WhJ00oYLyd8cwVihUPjOJw06DNZGBFDH8FmkqEGVE?=
 =?us-ascii?Q?zkHDE0I3X2RVJ5nI9YJbEyi218Y3CQNO7bOsFvOKj92ByFVOYUw2f1mr2uMu?=
 =?us-ascii?Q?q/2pssGheDj2LIWZchmmG+3UFc4GwXjkifUHBfrcTp/nScYPy8I8FoaFgg7M?=
 =?us-ascii?Q?vtBxBbcnu2R3bIqd7j9a8xwFjVeUW8RPBVPkBrP9tDXu2LDbQCWTD1HZmw7R?=
 =?us-ascii?Q?+l5AAfiGV7POzxBx82sRDeAvkEOEPIerFcVM4YW3kLhcJFeByXF4qUgcgP9C?=
 =?us-ascii?Q?tg7NCYA1ip3G3VXXrAiXJWLGlYkpV1M2duiLGnFp/n0lJDyUEhJtCm8LbaWC?=
 =?us-ascii?Q?t2NnIh2DE0A3RCwqjpT+OWLOeH1isH07YFmrC2PhIHVA0MgXwGx0Fyate8Kn?=
 =?us-ascii?Q?rPdt5+cXYY40MYMqCkibrZY1H5zqpmlHp8VMcK3AkFHRokXouETkLBD/LfNE?=
 =?us-ascii?Q?oFdgd5TIpRCl8eno+u7qCP05+/mdZKqhTEsJkcD9FCRuRvUtXfA5ZYJnyoLe?=
 =?us-ascii?Q?hfvrZygGHxemKPXuojqz7DEaJwa9JpcE+R9MCrnyH9NDLes9oq7sAi+SX/Ix?=
 =?us-ascii?Q?ZuTUBOacKiDWoZDcCdo0+Y0sBVzzsVvkH3OuaVr2gxY4OeDBTyyrFTebuVfI?=
 =?us-ascii?Q?NYuTBIzkLrJN2QO+ET0Aqx2vZxiodmermCLKKLfVVWX77x1F75s2RxDwFVgc?=
 =?us-ascii?Q?DISOxxl/ERTx4GUzyjxlOPOhtIj/jSBPY6SOYQYvvsUMCUAqSRZYuQ1Xl09v?=
 =?us-ascii?Q?jMtvxVqfszO7mbpX8ynJ9YJgFuXL4tVhHgYi/SQKv6PWBlggo3nmg1cRPsUU?=
 =?us-ascii?Q?DMp3LxPC5+UAoFhAJwd7EEglVqnX4CtcZZV+RpqX14ypwjWT1Zk6dD6ifRad?=
 =?us-ascii?Q?FStjjZ4LLYQJjIiG9oXgIMzlfMEEadAt22n/3bzWWRuQ7ZiyD5G81828jNhG?=
 =?us-ascii?Q?QNadSWNIsad4NWdlXTsbDc4wO5P0ijzKj9zA8FZo2FwWkYTB6q6r3Zsh99py?=
 =?us-ascii?Q?BWIt3sR0ODwhlKZ3JrXVCBJVjg3AlxeL1QzEbzo1Ad2PLrDgL2o0FTrxVrzu?=
 =?us-ascii?Q?WcH0q4qCi5fmvUMcWcDW1zSxBilpM6+pl4cFwAla49spiuX0HzVx+BjqGZ0u?=
 =?us-ascii?Q?34rwi1MJ6yuYTkm7Bjm0+NiEsjttu8zFvJ2ZOFodrScQ07bog+H/FIz+Amyo?=
 =?us-ascii?Q?iL63fo59j5lh7JGMrG2tzJbL0Z+LaoWWJPhjWLgkJG4tpvHZsHijaC5Ropg9?=
 =?us-ascii?Q?vuj+aa8f8lOxNTQI3j+zbsF/?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:34.6726
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d4b562db-f632-458a-7dd6-08dc758f72e9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FB.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8664

Hi all,

This is the remaining series for the full functional "dynamic node
programming using overlay dtbo" feature. The first part [1] has
already been merged.

Quoting from the original series, the first part has already made
Xen aware of new device tree node which means updating the dt_host
with overlay node information, and in this series, the goal is to
map IRQ and IOMMU during runtime, where we will do the actual IOMMU
and IRQ mapping and unmapping to a running domain. Also, documentation
of the "dynamic node programming using overlay dtbo" feature is added.

Patch 1 and 2 are fixes of the existing code which is noticed during
my local tests, details please see the commit message.

Gitlab CI for this series can be found in [1].

[1] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1293126857

Henry Wang (6):
  xen/common/dt-overlay: Fix lock issue when add/remove the device
  tools/xl: Correct the help information and exit code of the dt-overlay
    command
  xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
  tools/arm: Introduce the "nr_spis" xl config entry
  xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations
  tools: Introduce the "xl dt-overlay {attach,detach}" commands

Vikram Garhwal (2):
  xen/arm/gic: Allow routing/removing interrupt to running VMs
  docs: Add device tree overlay documentation

 docs/man/xl.cfg.5.pod.in              |  11 +
 docs/misc/arm/device-tree/booting.txt |  13 +
 docs/misc/arm/overlay.txt             |  99 ++++++
 tools/golang/xenlight/helpers.gen.go  |   2 +
 tools/golang/xenlight/types.gen.go    |   1 +
 tools/include/libxl.h                 |  15 +-
 tools/include/xenctrl.h               |   3 +
 tools/libs/ctrl/xc_dt_overlay.c       |  31 ++
 tools/libs/light/libxl_arm.c          |   4 +-
 tools/libs/light/libxl_dt_overlay.c   |  30 +-
 tools/libs/light/libxl_types.idl      |   1 +
 tools/xl/xl_cmdtable.c                |   4 +-
 tools/xl/xl_parse.c                   |   3 +
 tools/xl/xl_vmcontrol.c               |  39 ++-
 xen/arch/arm/dom0less-build.c         |   7 +-
 xen/arch/arm/domctl.c                 |   3 +
 xen/arch/arm/gic-vgic.c               |   8 +-
 xen/arch/arm/gic.c                    |  15 -
 xen/arch/arm/vgic/vgic.c              |   5 +-
 xen/common/dt-overlay.c               | 418 +++++++++++++++++++++-----
 xen/include/public/domctl.h           |  15 +
 xen/include/public/sysctl.h           |   7 +-
 xen/include/xen/dt-overlay.h          |   7 +
 23 files changed, 615 insertions(+), 126 deletions(-)
 create mode 100644 docs/misc/arm/overlay.txt

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722993.1127489 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxp-0006Iq-2I; Thu, 16 May 2024 10:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722993.1127489; Thu, 16 May 2024 10:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxo-0006Ia-UQ; Thu, 16 May 2024 10:03:52 +0000
Received: by outflank-mailman (input) for mailman id 722993;
 Thu, 16 May 2024 10:03:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxn-0005T3-SB
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:51 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 97a9dba8-136b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:03:51 +0200 (CEST)
Received: from MW4PR03CA0155.namprd03.prod.outlook.com (2603:10b6:303:8d::10)
 by SA1PR12MB6798.namprd12.prod.outlook.com (2603:10b6:806:25a::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 10:03:45 +0000
Received: from CO1PEPF000042A8.namprd03.prod.outlook.com
 (2603:10b6:303:8d:cafe::cc) by MW4PR03CA0155.outlook.office365.com
 (2603:10b6:303:8d::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 16 May 2024 10:03:45 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000042A8.mail.protection.outlook.com (10.167.243.37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:44 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:40 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:39 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97a9dba8-136b-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iDu6A9qbnYr0zSFV7lGoYQdZBuadHzWR2oz6a1lXG+WioqD/hnVKYFcEYobpU/O9XPC21JtAUKsWl5sSKI9f29Z79HSMTgo4FYwm3MmhfVbSz2VvqgYkup10iL/Kdy9ByYIjSRwpcWLbIQ49Rd6bshUUIQPTATxDNfpWH23eBO/Oat1sXick66oWsM9JVYMVWG1pXRohTO0ctpSz4jmBwP3XusGQ7wJYBGO619U3in3ETN+Pow7rg3uuFiJR+vngeAEKWtlLYKgrFktezX/LUOUlKd+v0bgjNebASGSDmSLlDlCthZNMZZ3DvQxaUrSexZPz/5jChGX2QqXVSEZXDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wWIi7CD7ZXy7QHzhQvNqeQnVv2B/6KurT+iAYX495cs=;
 b=FA8DXEaMb8ttkGqE1+eT0uFILbEeo+rjq8k6OTUUD2amOPQwm0s4xmpRB7UyVHpdD8uTrQxBPc76biOg04OtrlpwtnPSGXoTSky4wthu92KIQCNJIbjuvV4bO4CsSxpLCNE8L5iybR99fyO3BaVqGt5eBYxoVqfl5oca+A3xcI82dwmyNhSLrWwSP2wWyUO0C/m03JuLgCAJZ7r/DiSNwb0c7xIeHGtCfehd/Pq0aqhePxbYTtyWTgw3dMgfBYCTytikdbjZ6Lt6f4S5xqF/js1PJRvD0+5OZrj1oCjhYGMGxNvia5u0kUIVd4rjPVWVYhp2UwAvWZN1VmWJDVRUKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wWIi7CD7ZXy7QHzhQvNqeQnVv2B/6KurT+iAYX495cs=;
 b=4g1YPQZsRLLJ8Zk163NgtuJD3JvPOvaTaMBSepjWtddhd96hhX00j7UVo8i/J7/H1lhDxnf2HCTrko4s4Bt32zBIOUhiQl6YHgLOjEw/7kotk+naJhwBD0o4sf0UIdVp3x2p6KQjdxRQhywf9cYdregRfbXYGml++UH6iQxXTZI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
	<rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 4/8] tools/arm: Introduce the "nr_spis" xl config entry
Date: Thu, 16 May 2024 18:03:26 +0800
Message-ID: <20240516100330.1433265-5-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042A8:EE_|SA1PR12MB6798:EE_
X-MS-Office365-Filtering-Correlation-Id: d2388b5e-93f9-4d4e-1eea-08dc758f78cc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?sanhQ7uQGc63rD9omLVewJWBeEfHtF+cHwOzttmWVDkgzTz3vIiUgsFmxeTs?=
 =?us-ascii?Q?oxo9MrCwoZzKpMzBc1XfpvmJxe4usytmojaqqPIq2t7ZnzgPY6+IJdhn1a3x?=
 =?us-ascii?Q?KwGU8ttFmcioqehcV8WhyI5SSpnxXdfntwvJjgYA+f82EKOySzJaBgfOyo79?=
 =?us-ascii?Q?0BcIyPXFgvNewyz9suPAQPfibeYBVp2TS55M1yY2OcRPGxLuqp54uo9RgLfN?=
 =?us-ascii?Q?NghM6ZI6FbA6fFni6B28tk19KFD17yylICx9g2PXOb3l0chblIKHgGc1ZOi4?=
 =?us-ascii?Q?SSCCuzs9adKnZx73nbiiBdNhuz3w6a2BoeqQFYMsj+y81nx39xXkzs3pWKwm?=
 =?us-ascii?Q?k8i2//ev+TGQQT3nqYvus9q0UDBiMU+NwUmaJ7tdka4g3RrN4qF+G1QZOshi?=
 =?us-ascii?Q?ajWmtui8KKS27jVLaSitYu3fsrUZQ539iNSGcWShVdX5a27DG6ztK/GnP/V5?=
 =?us-ascii?Q?aMtMVmUNGGtf7MC6Szqir7O25wlPn+qbQ1YYmXT+gHuh/3r07RQChKEw7C+J?=
 =?us-ascii?Q?SXnmNOCYbnvmXcb1pdI12/8w8rk/e4uUHUVUSKQwsCiFLhXYDRZpMFmqqwGi?=
 =?us-ascii?Q?xAuszootAydux5M3Tma2fKTk5rUOQZaf6A7Fs9CpEFL6eJ4FJRpxlT1HUfvU?=
 =?us-ascii?Q?xABsL883fO0P7gbA5Y406MTWK2PmnTZNk1tZFF4F2H6P1YoC+3ABH0i6fg70?=
 =?us-ascii?Q?Oi1ZkQMsRD+K6fY8lBpW1myIxscZekcvZRaLdzHdM7nuscJTIgRuvlJ2Ypt2?=
 =?us-ascii?Q?+cvqJmofifNc+zG3FtGIOBZmGxMaYsSm7yWFoeKkRptVuUSGH8gfwwINnloR?=
 =?us-ascii?Q?xIJ98aNwQtNTDRnjwxhHghXE+F8jgkW/F5K7zIgwOpebPTbA3j0STX0EYX99?=
 =?us-ascii?Q?olfXyMnzjPoxO9SMdDksum+sycIqB1/09EpvWg8MptDeiIizC/93DHC/hzbn?=
 =?us-ascii?Q?E9H06Bu0O3XZGLE3YgTVI9CHrKdzlD//Wo4iq0K4vx+USquSWYubMHvlqP4u?=
 =?us-ascii?Q?xH44BavYDbx0CY4w/7NuNU9l5KwQYUatcAgnY1WLa+9BucfBXN7FoFhQ9JIt?=
 =?us-ascii?Q?Yldk+Spx7ayz8tztKVW1Y4ZhQZnsOGVTctWQXYteBHSSa1yru2NlnB62hymJ?=
 =?us-ascii?Q?j3ZV6WLBKaR4dqhWbfWMzU9RvpYjadoLLvaWXW+SIKOffBXmFarN8nOALxsl?=
 =?us-ascii?Q?CVuVG9kuG6gJKf5TEtUiElF+BPnQOu6u4nykNPu2vW/RrBCVnTBvQCLkPMEG?=
 =?us-ascii?Q?JHyJQe68yszaFlyFbouBWDbDh1HOsAZj351yR7gFGlwcMlJVz5Et985GZFQ/?=
 =?us-ascii?Q?hU8yU60tYzezZXg74XJD5pnB9KWITWz/umcnZM98hw6x2zkbVTe/O7mwhhu7?=
 =?us-ascii?Q?JygtJoamCL+x4kdZPc/G07hjYp6r?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:44.5621
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d2388b5e-93f9-4d4e-1eea-08dc758f78cc
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042A8.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6798

Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.

Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.

Update the doc and the golang bindings accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch to replace the original patch in v1:
  "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
---
 docs/man/xl.cfg.5.pod.in             | 11 +++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/libs/light/libxl_arm.c         |  4 ++--
 tools/libs/light/libxl_types.idl     |  1 +
 tools/xl/xl_parse.c                  |  3 +++
 6 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..6a2d86065e 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,17 @@ raised.
 
 =back
 
+=over 4
+
+=item B<nr_spis="NR_SPIS">
+
+A 32-bit optional integer parameter specifying the number of SPIs (Shared
+Peripheral Interrupts) to allocate for the domain. If the `nr_spis` parameter
+is missing, the max number of SPIs calculated by the toolstack based on the
+devices allocated for the domain will be used.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 78bdb08b15..757ccaf035 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
+x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
+xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index ccfe18019e..b7b4ba88af 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -597,6 +597,7 @@ ArchArm struct {
 GicVersion GicVersion
 Vuart VuartType
 SveVl SveType
+NrSpis uint32
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1cb89fa584..a4029e3ac8 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 
     LOG(DEBUG, "Configure the domain");
 
-    config->arch.nr_spis = nr_spis;
-    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
+    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
+    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
 
     switch (d_config->b_info.arch_arm.gic_version) {
     case LIBXL_GIC_VERSION_DEFAULT:
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 470122e768..3f143f405d 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
                                ("sve_vl", libxl_sve_type),
+                               ("nr_spis", uint32),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index ab09d0288b..4aa99029b5 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2933,6 +2933,9 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
+        b_info->arch_arm.nr_spis = l;
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722994.1127499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxq-0006Zg-Bi; Thu, 16 May 2024 10:03:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722994.1127499; Thu, 16 May 2024 10:03:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxq-0006ZR-8V; Thu, 16 May 2024 10:03:54 +0000
Received: by outflank-mailman (input) for mailman id 722994;
 Thu, 16 May 2024 10:03:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxo-0005T3-VL
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:52 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 986b2861-136b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:03:52 +0200 (CEST)
Received: from MW4PR03CA0162.namprd03.prod.outlook.com (2603:10b6:303:8d::17)
 by PH0PR12MB7905.namprd12.prod.outlook.com (2603:10b6:510:28b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Thu, 16 May
 2024 10:03:48 +0000
Received: from CO1PEPF000044FA.namprd21.prod.outlook.com
 (2603:10b6:303:8d:cafe::b1) by MW4PR03CA0162.outlook.office365.com
 (2603:10b6:303:8d::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 16 May 2024 10:03:48 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:47 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:46 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:46 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:43 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 986b2861-136b-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Yv1u04U91wvqYeyAINSCS4oMSzvqtre4QfoGR/A97lMO1m3nxbh9wtaBHZfZZO3CqPi5UoKmWQWYpClmAqWq3DMjOknDGFDi53PadQDZyUUEGgHt/o7YkV2xS31uGz3z8AwoKNk+RhgBJFEczXbmR/RWRqQCun6Lkt9Qgk7sGwcyAH4O6dqMu7HL02bjKOco12KGlOLG4m12ShST2idV/dIBCcpsktiXYCd9MQdQqW7qAFJ56yZvcL1XGEYj4i/O3V1o8c6h/1iCusImSUQMWus8Sfin683Tm1O+14euR54IKtNxz7vIOezY4rXzmyZaMmWraG7TCTCbXfSRD/AXnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=mF2OBpKMBYdkRqVcGsWEJvyB5o1cjSWHP++bO6wuCaM=;
 b=dBPN5QNFY9M/livjelK5E63yc/YObvezt8U44Rus9Tu+ssbngS0J8sM4meJjchV7ukRIPc0AI8OVj1214lLdwaCdJ8zgEes04ZqYsKMx5WEjEbixTfDLtcguve7nXNC3E9jmtsQLKIJsNyb0hFhXDi8hiSSFq0rkpDUfqtlZMfALHfdVYnchyZkvu/pphDaoyGhfv4WePoKCWHkNDfjFBUVLFIAimyLAiZeJl3rkoIr+LO1csTX+ErOVt/NWSOzp9p3erNVQ0AoFR7j6YzET9eqR0Oi8gunVTQ7E1q0UoZqJVKENM51rEVVMmGpJlgHVFZjTtviG2uXrrDx6nQKi1w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=mF2OBpKMBYdkRqVcGsWEJvyB5o1cjSWHP++bO6wuCaM=;
 b=G3h/kFl2E2YtmsELkOZ2ReZmJo315KXNXkkP3LMrLFk5U2VjCTPTAhcHLrjIe09l8KHSQbBYG2ZnW8WZ84C8xjPiALEcGWQYcHyUCiKZJLy56dpCoTIzUjZtr8U7KFT59tkPpiQialV7HI2ut9yWH4w4CepzXhhICrlsQEa8O98=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Vikram Garhwal <fnu.vikram@xilinx.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>
Subject: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to running VMs
Date: Thu, 16 May 2024 18:03:27 +0800
Message-ID: <20240516100330.1433265-6-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|PH0PR12MB7905:EE_
X-MS-Office365-Filtering-Correlation-Id: 66647cf5-16f1-4d7b-8f9c-08dc758f7a90
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?w96Id55op/KRIy+G9ieLph2A179n/n4oqikxATjwFbxX3XnYpGnlwv5CBoY5?=
 =?us-ascii?Q?gCyCBGvr3JFYYSKq0/Q2yhstBkogASEmkcK8S8K9gzi+dyc4ck6johJYoADW?=
 =?us-ascii?Q?xRjB50yh8JSl8YmTw24hsMZTrdD63ZSMXHiQCRN6owCf8yqzuHD91RMBmwaK?=
 =?us-ascii?Q?W4R2L5EmzTasfpZ7xn/9nxm2IE62sbqvkSD9habxBKLbRNJz4pUEsZbOLcwt?=
 =?us-ascii?Q?xijpUjI2O6ORlC2PmcNXr0+QNLVTBn1L5Qm5yY4raHvIDD7ACD8vxX0HKwzy?=
 =?us-ascii?Q?Bx8Uez/UDseIebbZDRHG1++DCdE4/qr9PQ9Kdye3rkw9w1QqgjWC6+4AYDiz?=
 =?us-ascii?Q?C59+HszPvu75CerFZd5Y5nTX2Td4MTrKh8lNsMg0sv6ZRSnyE1aGSteVy7mm?=
 =?us-ascii?Q?lMSAFc8dDSNQfQYw0uFLRrTpD1Jn5ycFeguWMUd9ACj7m4pHexRF6JyRdeOA?=
 =?us-ascii?Q?zCjhKk0LFIyhcygnzEori2aJQmlYL1iqE+IWeAwuh6ryRJQ/2NzojxQ+MRw7?=
 =?us-ascii?Q?FluIGDTOm3NF76cmi/Dqs9T33IQN4pAGTjX5yaBZOvny2jN57fwCIWzpN0Zc?=
 =?us-ascii?Q?5X903Z+VYa9dMy93+N2k8RjkZARqxnwZSHhtqaJoZligg3xp/kKzJe99y1oO?=
 =?us-ascii?Q?++uJXTohUhOx4iQyBlLE50j1Rbhjb5DsgvULIKxkgI4zltVHWa6pI9zNJa9Q?=
 =?us-ascii?Q?WxbfuihvvZ8rEZk+gwNcIGVaKaCIInpAlcb4YDl0Lm6qldZKByyiQ2Uu/X2K?=
 =?us-ascii?Q?nlYXGIAsgWJLYGNwLZMaIfeovLbLresF+6pofMuVyjlaLJkT2f8MS1go8y8v?=
 =?us-ascii?Q?9AcMJdCS1gIEIdFFhILbuGDYfQPvYeRq3Ckr1PvSzqfUMRjOC/NOtn27OKSD?=
 =?us-ascii?Q?sX0zHQQfumKUaGmc7N9MeEA+87Xe8A2572G0gb9emUPqREyK/VyQK+ptNLaA?=
 =?us-ascii?Q?wEN7wkzsIztvA3vLSulAMwyZrqdqUaiKHr4DsHfRaCThhvLJLRnZjAs6MNSq?=
 =?us-ascii?Q?Wz9xB2OJH19cT9C4plpO0+0Ui/xjnaq9jZrEWKntnqkSrodi5p7iLreJRPEg?=
 =?us-ascii?Q?5TazlRfty6ZA+eXKCzWfvTU+DNMGWgbGiVcdg9qmyxPpgkx2mpvKbTAx08cQ?=
 =?us-ascii?Q?ZLdSJBjEZw8wfBE4nR+TwHkMLfRoKx/hedp+UEye/hlRtJjq8n/YZ4W8da8v?=
 =?us-ascii?Q?loPkHCWyeXsVCbSjtaGffXiEtCXX9OrBcMNkrWHkpy6yBl98QCEQ4Ri/UUIm?=
 =?us-ascii?Q?NZ+eoePvMu9RPRuOzbkflP5319/s0RumiOklX0flVFSEr5KM5uWHDKP3m2+Q?=
 =?us-ascii?Q?jVPq4ZRYjYeftd2VINhcXmo0dIdsfFVCGk4xp4m66tKL8EM54R/qU8EHJrZH?=
 =?us-ascii?Q?zMFZjRFtYOukK5GEf1pei+4ffKj1?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:47.5272
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 66647cf5-16f1-4d7b-8f9c-08dc758f7a90
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FA.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7905

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Currently, routing/removing physical interrupts are only allowed at
the domain creation/destroy time. For use cases such as dynamic device
tree overlay adding/removing, the routing/removing of physical IRQ to
running domains should be allowed.

Removing the above-mentioned domain creation/dying check. Since this
will introduce interrupt state unsync issues for cases when the
interrupt is active or pending in the guest, therefore for these cases
we simply reject the operation. Do it for both new and old vGIC
implementations.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Reject the case where the IRQ is active or pending in guest.
---
 xen/arch/arm/gic-vgic.c  |  8 ++++++--
 xen/arch/arm/gic.c       | 15 ---------------
 xen/arch/arm/vgic/vgic.c |  5 +++--
 3 files changed, 9 insertions(+), 19 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 56490dbc43..d1608415f8 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     {
         /* The VIRQ should not be already enabled by the guest */
         if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             p->desc = desc;
         else
             ret = -EBUSY;
     }
     else
     {
-        if ( desc && p->desc != desc )
+        if ( desc && p->desc != desc &&
+             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
+              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )
             ret = -EINVAL;
         else
             p->desc = NULL;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86de..3ebd89940a 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
     ASSERT(virq < vgic_num_irqs(d));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-    if ( d->creation_finished )
-        return -EBUSY;
-
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
     if ( ret )
         return ret;
@@ -167,13 +159,6 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
     ASSERT(test_bit(_IRQ_GUEST, &desc->status));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * Removing an interrupt while the domain is running may have
-     * undesirable effect on the vGIC emulation.
-     */
-    if ( !d->is_dying )
-        return -EBUSY;
-
     desc->handler->shutdown(desc);
 
     /* EOI the IRQ if it has not been done by the guest */
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b9463a5f27..785ef2b192 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -877,7 +877,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
     if ( connect )                      /* assign a mapped IRQ */
     {
         /* The VIRQ should not be already enabled by the guest */
-        if ( !irq->hw && !irq->enabled )
+        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
         {
             irq->hw = true;
             irq->hwintid = desc->irq;
@@ -887,7 +887,8 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
     }
     else                                /* remove a mapped IRQ */
     {
-        if ( desc && irq->hwintid != desc->irq )
+        if ( desc && irq->hwintid != desc->irq &&
+             (irq->active || irq->pending_latch) )
         {
             ret = -EINVAL;
         }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722995.1127509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxu-00071L-Lc; Thu, 16 May 2024 10:03:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722995.1127509; Thu, 16 May 2024 10:03:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxu-000714-Hx; Thu, 16 May 2024 10:03:58 +0000
Received: by outflank-mailman (input) for mailman id 722995;
 Thu, 16 May 2024 10:03:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxs-0005ER-TE
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:57 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20607.outbound.protection.outlook.com
 [2a01:111:f400:7e88::607])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 99e22697-136b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:03:54 +0200 (CEST)
Received: from MW4P222CA0022.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::27)
 by MW4PR12MB7216.namprd12.prod.outlook.com (2603:10b6:303:226::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May
 2024 10:03:50 +0000
Received: from CO1PEPF000044FD.namprd21.prod.outlook.com
 (2603:10b6:303:114:cafe::41) by MW4P222CA0022.outlook.office365.com
 (2603:10b6:303:114::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 10:03:50 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044FD.mail.protection.outlook.com (10.167.241.203) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:50 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:49 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:48 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:46 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99e22697-136b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SzQl4oC4aqwmsE01f/6t/ZOPpDuu2OXSOZnTZMof0AJkaz4k4e46VqcSsMgpMoQNdVoOIY/9r92ht1uSnDbZmS/kt+fdgoRaSJmJL6eggQ4O3nBWJvqwl9+URHc6UstuNbkklNvmOn6eCS1KwvsnRySGWtOyrPDil3ar02qmvwldB+tO1rbXUkIhKlF4ozM2bZ9Tf636C6yQJu7vdkw6g7kXyfZe5CP67acHtGADv5WwNLxbDuirDgpvh70TFghSih8b4CtoLtH8u3c7gQBPcBk+bny8j7TwZ5qzk/7UKC+V1c1EJFqHawJ4q9xt1/kcdSbQbi+2NBaPf36Fxigjxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=rQdHNGe+exEXW3biYqAzR1o3kzypd+Kl5wlHYuzIz7g=;
 b=jDVarwmQNumQeCnO6vlEHV0+tGYfK0jtCfRZNR+OEaEcwDSID1rr1MtYBFxLJ2Ysocl2MAkb5xWwG1Lj9YTfQDkdbDJCLGcKH76DrUZKFRr2LvSOheMvkjm75sQKLqK2Y6oHl+ZYIZZnvHsz2UTFNB+NDasP0bIhGTUw5iDznAA4P1bYtrdANBW1kL4QPbjk1cpoZRC2l5mXGbiSdSxCC1h9SMrG2fAFuCbIZpUeX6//B4LagHk5XP2c7K/vHwqE8ytDn/K/GVvlLvXEjIXTO6G7aGqUT0cV3LtZE0lMmqTMyIEyYDCNJRv1YOn6Q6TQyhJ7HpUtdqrw8H5IhtosfA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rQdHNGe+exEXW3biYqAzR1o3kzypd+Kl5wlHYuzIz7g=;
 b=lvR9g0YEn//fjhoof7iZwr3o0rdpktzOe28FSOoy1Os8AkM5vjWLRkXKeAOQFSorcFaZhUHbIWdjVf2GX59EcJ7mcnYke/bL6DM6y1DEIWxygv0wI4KIzlUGHHE/wyTvQ/tvKy/0ECPPirVjk9fbPKSs8fcsIONzkLuWTCGDwsg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH v2 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations
Date: Thu, 16 May 2024 18:03:28 +0800
Message-ID: <20240516100330.1433265-7-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FD:EE_|MW4PR12MB7216:EE_
X-MS-Office365-Filtering-Correlation-Id: 01f7cbec-ff38-4212-e416-08dc758f7c2b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?Te+EBV2CpKCi2W/w0ahW0wrJr6MCpWgKhZ1SWwMV8+V6PEKwDkcEs/d8Zp+3?=
 =?us-ascii?Q?kC5N7YH9r5GdbAxBzwRaNiZomJi6akssyp+zgJIGAOZWTzOVQU+EgIKjst4u?=
 =?us-ascii?Q?npLrxCRbyRCRUGKkiNio6d9GhF9Kske1fsTH+hJ6eI4zJpT1wyQV1LwKCHDy?=
 =?us-ascii?Q?C/hUjNjCWKvOmjGtESrkGSTmVaNrpA/h9NxYP8TXM/tD2F8tna74vbQwtoSe?=
 =?us-ascii?Q?m/zEUP/LqIvq6G7kUpdqwj84bD+ERgKl7OHwdyt47k1r0525AtrF+b1kHqhn?=
 =?us-ascii?Q?RuDlLPD9KVQ8h/UH+C03aHe+katwI+JZAnZL6sbYzrFfCtltu8seTMtAwSgi?=
 =?us-ascii?Q?K8+qmciCdmtmyVaS8jGNQc6S4lEWiJEbaHo2hP6fA+P3zCuGNi71X7pCwasM?=
 =?us-ascii?Q?ml7rtKZUcjJYl8hU4br2eK+LBOc8h5tkUVpA/KZnl2HCU1FEKfkvX06sB+RC?=
 =?us-ascii?Q?qJrlQnlBN+B1s1w6d22V4WLaqa3/cM23ICHNgMeOZmpxVJ5fEQVGgytSftHA?=
 =?us-ascii?Q?HwXciwxHUx0QERDW6j31KJEy7lPrMHUSdNjLcJhw7SJ0DjKNOJZj2bKdNfxn?=
 =?us-ascii?Q?LzS3N5oTRlTgkkVwMLDaNi/hXuseXqRbyzY5aInvlfx/obNYPQN7WdJKFPQO?=
 =?us-ascii?Q?MK0GKfV9mkSLNEz2yUKcAgug2RUZrdvpDIziUeOOhJexYM3BZmVlv3sFfsLA?=
 =?us-ascii?Q?tcORE6wyX7baXpnz8GvURW2Sgf1mA9g8chZKJRwSlWnT90TvYIfKGfTNgo3w?=
 =?us-ascii?Q?rQkAZww55r7/4NdCgwaePPuXBQ6hDxdhR9/CqvnaVn7CgANWgYTQzZpRngQD?=
 =?us-ascii?Q?JhR0iJyEv5jplR2hY9/VndPY/lWU0r0gphI+aAHth3Ymv6gHkwcHADwnTEKD?=
 =?us-ascii?Q?IgkpqaPOmttfEetLF6G7UBdmZT4ApP3E2pGCr6HKZV3+h2Oik5dRbNtnr472?=
 =?us-ascii?Q?Heh/2Ou5ufOCDQYVrRFr1OSfFytV04S5iC0qrLUZg+VyAe9mi/SxK2cr49MW?=
 =?us-ascii?Q?y8i0t4KzHT0Iz1bZhfX9TECovCLOCyT1Y1Nau/xTfmYUs2dDsko/Wv4RwX8h?=
 =?us-ascii?Q?wkc/gKOS5TXm48H8A/dUHQIzPle6L/r9SrJSaPdrKZjH3VrNH61a2JeDdLL7?=
 =?us-ascii?Q?MVNkXfglo5zjLIEl3RArKWQagqvoXDvKsRp+sl5uiNL6icfW6LNAr6DG0tY8?=
 =?us-ascii?Q?AB2K37v4G0y45SrPCjocQhSLMdWic6mL7tjlItuzShmimhudlseN0ZEWQXct?=
 =?us-ascii?Q?Q3jSjcaDpjTL2GDJvxPsh6M1zeBvT0jKsrMXcNCmJi2Bf6hfjri2MfyI8sbo?=
 =?us-ascii?Q?OLbmeaK8Hjy6GRba/onomDGgRjCuyhW2bEDewk+Vn2Tjab/3lIdOjgBmW3KO?=
 =?us-ascii?Q?Viyx5j4=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:50.2073
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 01f7cbec-ff38-4212-e416-08dc758f7c2b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FD.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7216

In order to support the dynamic dtbo device assignment to a running
VM, the add/remove of the DT overlay and the attach/detach of the
device from the DT overlay should happen separately. Therefore,
repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
overlay to Xen device tree, instead of assigning the device to the
hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with
operations XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH} to do/undo the
device assignment to the domain.

The hypervisor firstly checks the DT overlay passed from the toolstack
is valid. Then the device nodes are retrieved from the overlay tracker
based on the DT overlay. The attach/detach of the device is implemented
by map/unmap the IRQ and IOMMU resources. Note that with these changes,
the device de-registration from the IOMMU driver should only happen at
the time when the DT overlay is removed from the Xen device tree.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
v2:
- New patch.
---
 xen/arch/arm/domctl.c        |   3 +
 xen/common/dt-overlay.c      | 415 ++++++++++++++++++++++++++++-------
 xen/include/public/domctl.h  |  15 ++
 xen/include/public/sysctl.h  |   7 +-
 xen/include/xen/dt-overlay.h |   7 +
 5 files changed, 366 insertions(+), 81 deletions(-)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad56efb0f5..12a12ee781 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -5,6 +5,7 @@
  * Copyright (c) 2012, Citrix Systems
  */
 
+#include <xen/dt-overlay.h>
 #include <xen/errno.h>
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
@@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+    case XEN_DOMCTL_dt_overlay:
+        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 9cece79067..593e985949 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -356,24 +356,100 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
     return 0;
 }
 
+static int remove_irq(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+
+    /*
+     * IRQ should always have access unless there are duplication of
+     * of irqs in device tree. There are few cases of xen device tree
+     * where there are duplicate interrupts for the same node.
+     */
+    if (!irq_access_permitted(d, s))
+        return 0;
+    /*
+     * TODO: We don't handle shared IRQs for now. So, it is assumed that
+     * the IRQs was not shared with another domain.
+     */
+    rc = irq_deny_access(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s);
+        return rc;
+    }
+
+    rc = release_guest_irq(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to release irq %ld\n", s);
+        return rc;
+    }
+
+    return rc;
+}
+
+static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d);
+}
+
+static int remove_iomem(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+    p2m_type_t t;
+    mfn_t mfn;
+
+    mfn = p2m_lookup(d, _gfn(s), &t);
+    if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL )
+        return -EINVAL;
+
+    rc = iomem_deny_access(d, s, e);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n",
+               d, s, e);
+        return rc;
+    }
+
+    rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s));
+    if ( rc )
+        return rc;
+
+    return rc;
+}
+
+static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d);
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
-static int remove_node_resources(struct dt_device_node *device_node)
+static int remove_node_resources(struct dt_device_node *device_node,
+                                 struct domain *d)
 {
     int rc = 0;
     unsigned int len;
     domid_t domid;
 
-    domid = dt_device_used_by(device_node);
+    if ( !d )
+    {
+        domid = dt_device_used_by(device_node);
 
-    dt_dprintk("Checking if node %s is used by any domain\n",
-               device_node->full_name);
+        dt_dprintk("Checking if node %s is used by any domain\n",
+                   device_node->full_name);
 
-    /* Remove the node if only it's assigned to hardware domain or domain io. */
-    if ( domid != hardware_domain->domain_id && domid != DOMID_IO )
-    {
-        printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n",
-               device_node->full_name, domid);
-        return -EINVAL;
+        /*
+         * We also check if device is assigned to DOMID_IO as when a domain
+         * is destroyed device is assigned to DOMID_IO.
+         */
+        if ( domid != DOMID_IO )
+        {
+            printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n",
+                   device_node->full_name, DOMID_IO, domid);
+            return -EINVAL;
+        }
     }
 
     /* Check if iommu property exists. */
@@ -381,9 +457,12 @@ static int remove_node_resources(struct dt_device_node *device_node)
     {
         if ( dt_device_is_protected(device_node) )
         {
-            rc = iommu_remove_dt_device(device_node);
-            if ( rc < 0 )
-                return rc;
+            if ( !list_empty(&device_node->domain_list) )
+            {
+                rc = iommu_deassign_dt_device(d, device_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
     }
 
@@ -392,7 +471,8 @@ static int remove_node_resources(struct dt_device_node *device_node)
 
 /* Remove all descendants from IOMMU. */
 static int
-remove_descendant_nodes_resources(const struct dt_device_node *device_node)
+remove_descendant_nodes_resources(const struct dt_device_node *device_node,
+                                  struct domain *d)
 {
     int rc = 0;
     struct dt_device_node *child_node;
@@ -402,12 +482,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node)
     {
         if ( child_node->child )
         {
-            rc = remove_descendant_nodes_resources(child_node);
+            rc = remove_descendant_nodes_resources(child_node, d);
             if ( rc )
                 return rc;
         }
 
-        rc = remove_node_resources(child_node);
+        rc = remove_node_resources(child_node, d);
         if ( rc )
             return rc;
     }
@@ -420,8 +500,7 @@ static int remove_nodes(const struct overlay_track *tracker)
 {
     int rc = 0;
     struct dt_device_node *overlay_node;
-    unsigned int j;
-    struct domain *d = hardware_domain;
+    unsigned int j, len;
 
     for ( j = 0; j < tracker->num_nodes; j++ )
     {
@@ -431,18 +510,15 @@ static int remove_nodes(const struct overlay_track *tracker)
 
         write_lock(&dt_host_lock);
 
-        rc = remove_descendant_nodes_resources(overlay_node);
-        if ( rc )
-        {
-            write_unlock(&dt_host_lock);
-            return rc;
-        }
-
-        rc = remove_node_resources(overlay_node);
-        if ( rc )
+        /* Check if iommu property exists. */
+        if ( dt_get_property(overlay_node, "iommus", &len) )
         {
-            write_unlock(&dt_host_lock);
-            return rc;
+            if ( dt_device_is_protected(overlay_node) )
+            {
+                rc = iommu_remove_dt_device(overlay_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
@@ -457,22 +533,6 @@ static int remove_nodes(const struct overlay_track *tracker)
         write_unlock(&dt_host_lock);
     }
 
-    /* Remove IRQ access. */
-    if ( tracker->irq_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
-   /* Remove mmio access. */
-    if ( tracker->iomem_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
     return rc;
 }
 
@@ -536,9 +596,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     xfree(entry->nodes_address);
 
-    rangeset_destroy(entry->irq_ranges);
-    rangeset_destroy(entry->iomem_ranges);
-
     xfree(entry);
 
  out:
@@ -620,15 +677,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return -EFAULT;
         }
 
-        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
-                           tr->iomem_ranges,
-                           tr->irq_ranges);
         write_unlock(&dt_host_lock);
-        if ( rc )
-        {
-            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-            return rc;
-        }
 
         /* Keep overlay_node address in tracker. */
         tr->nodes_address[j] = (unsigned long)overlay_node;
@@ -638,9 +687,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
 }
 /*
  * Adds device tree nodes under target node.
- * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
- * is done to avoid the removal of device_tree generation, iomem regions mapping
- * to hardware domain done by handle_node().
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
  */
 static long handle_add_overlay_nodes(void *overlay_fdt,
                                      uint32_t overlay_fdt_size)
@@ -774,20 +821,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
         goto err;
     }
 
-    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
-    if (tr->irq_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IRQ rangeset failed");
-        goto err;
-    }
-
-    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
-    if (tr->iomem_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
-        goto err;
-    }
-
     rc = add_nodes(tr, nodes_full_path);
     if ( rc )
     {
@@ -843,14 +876,205 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     xfree(tr->nodes_address);
     xfree(tr->fdt);
 
-    rangeset_destroy(tr->irq_ranges);
-    rangeset_destroy(tr->iomem_ranges);
-
     xfree(tr);
 
     return rc;
 }
 
+static long handle_detach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry, *temp, *track;
+    bool found_entry = false;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            track = entry;
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        rc = -EINVAL;
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Detaching nodes is supported only for prior added dtbo.\n");
+        goto out;
+
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = remove_descendant_nodes_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+
+        rc = remove_node_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+        write_unlock(&dt_host_lock);
+
+        rc = remove_all_irqs(entry->irq_ranges, d);
+        if ( rc )
+            goto out;
+
+        rc = remove_all_iomems(entry->iomem_ranges, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove IRQ access. */
+    if ( entry->irq_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove mmio access. */
+    if ( entry->iomem_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    return rc;
+}
+
+static long handle_attach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry, *temp, *track;
+    bool found_entry = false;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            track = entry;
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        rc = -EINVAL;
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Attaching nodes is supported only for prior added dtbo.\n");
+        goto out;
+
+    }
+
+    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
+    if (entry->irq_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IRQ rangeset failed");
+        goto out;
+    }
+
+    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
+    if (entry->iomem_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
+                           entry->iomem_ranges, entry->irq_ranges);
+        write_unlock(&dt_host_lock);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+            goto out;
+        }
+    }
+
+    spin_unlock(&overlay_lock);
+
+    return 0;
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+    return rc;
+}
+
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     long ret;
@@ -890,6 +1114,45 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
     return ret;
 }
 
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
+{
+    long ret;
+    void *overlay_fdt;
+
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH &&
+         op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH )
+        return -EOPNOTSUPP;
+
+    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
+        return -EINVAL;
+
+    if ( op->pad[0] || op->pad[1] || op->pad[2] )
+        return -EINVAL;
+
+    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+    if ( overlay_fdt == NULL )
+        return -ENOMEM;
+
+    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+    if ( ret )
+    {
+        gprintk(XENLOG_ERR, "copy from guest failed\n");
+        xfree(overlay_fdt);
+
+        return -EFAULT;
+    }
+
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH )
+        ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+    else
+        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+
+    xfree(overlay_fdt);
+
+    return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..95bfb3c0e8 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+#if defined(__arm__) || defined (__aarch64__)
+struct xen_domctl_dt_overlay {
+    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
+    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
+#define XEN_DOMCTL_DT_OVERLAY_ATTACH                3
+#define XEN_DOMCTL_DT_OVERLAY_DETACH                4
+    uint8_t overlay_op;                     /* IN: Attach or detach. */
+    uint8_t pad[3];                         /* IN: Must be zero. */
+};
+#endif
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1288,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_dt_overlay                    87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1351,9 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+#if defined(__arm__) || defined (__aarch64__)
+        struct xen_domctl_dt_overlay        dt_overlay;
+#endif
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index febaa4b16a..b613babdf9 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1187,11 +1187,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
 #if defined(__arm__) || defined (__aarch64__)
 /*
  * XEN_SYSCTL_dt_overlay
- * Performs addition/removal of device tree nodes under parent node using dtbo.
- * This does in three steps:
- *  - Adds/Removes the nodes from dt_host.
- *  - Adds/Removes IRQ permission for the nodes.
- *  - Adds/Removes MMIO accesses.
+ * Performs addition/removal of device tree nodes under parent node using dtbo
+ * from dt_host.
  */
 struct xen_sysctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
index c0567741ee..64c9e34a8a 100644
--- a/xen/include/xen/dt-overlay.h
+++ b/xen/include/xen/dt-overlay.h
@@ -14,6 +14,7 @@
 #include <xen/device_tree.h>
 #include <xen/list.h>
 #include <xen/rangeset.h>
+#include <public/domctl.h>
 
 /*
  * overlay_track describes information about added nodes through dtbo.
@@ -42,12 +43,18 @@ struct xen_sysctl_dt_overlay;
 
 #ifdef CONFIG_OVERLAY_DTB
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
 #else
 #include <xen/errno.h>
 static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     return -EOPNOTSUPP;
 }
+static inline long dt_overlay_domctl(struct domain *d,
+                                     struct xen_domctl_dt_overlay *op)
+{
+    return -EOPNOTSUPP;
+}
 #endif
 
 #endif /* __XEN_DT_OVERLAY_H__ */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722996.1127514 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxv-00075v-3q; Thu, 16 May 2024 10:03:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722996.1127514; Thu, 16 May 2024 10:03:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxu-000750-S6; Thu, 16 May 2024 10:03:58 +0000
Received: by outflank-mailman (input) for mailman id 722996;
 Thu, 16 May 2024 10:03:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxt-0005ER-TW
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:58 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2405::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9a731e23-136b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:03:55 +0200 (CEST)
Received: from MW4PR03CA0153.namprd03.prod.outlook.com (2603:10b6:303:8d::8)
 by BL3PR12MB6428.namprd12.prod.outlook.com (2603:10b6:208:3b7::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Thu, 16 May
 2024 10:03:52 +0000
Received: from CO1PEPF000042A8.namprd03.prod.outlook.com
 (2603:10b6:303:8d:cafe::d7) by MW4PR03CA0153.outlook.office365.com
 (2603:10b6:303:8d::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 10:03:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000042A8.mail.protection.outlook.com (10.167.243.37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:51 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:50 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:50 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a731e23-136b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lu2btGx3SJHxfm7pkjGq8DxuXJGXF6D10E/DrZuLMG/HEaX9JRweTVtbApZv9WT01DjwKfLFzUQQWK1pVkHuNhsvPAXGZF9kKQikoBF36ZxAd9Sj2k8OgCaVy3JXzFC+dMQeI2AKuHvps0kXoj1tPvjiwmFb3anqMAAadovizni0NFEFeZDhERCuSJryj3ZMfU1ecG1K068etYljHmOde/+iD4Yd5z9BJCInwnwxVkUmiHo0Ch+DHFiTvrtDnmE4ufvAc7E0+t8Ex0SIJSelbAx2qUP0ri/GqcUi+ssGqLxzD1ysBKTs3bVLIfG5M42tuCSxFJbYWMI8tBXAeGgUBg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=nl/BNnEwGnkfp7ugboqdbNn2EBw+cplgSH3kLI39gkI=;
 b=hJgq1eHJforyIRyB/BFw6pOQvSM0F4LGPwb+T3CLIKVoS4hqNltUiG7mzDrj+U52D9cH8hQnlG7UtbT5227ZUb9j7Hl6Tb7e2w6Xzs/K40bUdqRg+MhLaDGRsAkrpOC4tbKeVqd4spRQflbfAheAivkeYmtXN3/YRlqoaHOY9jhKj/ANiro3XOu5i4/i3uRzcNbx2Nqi6DmkiRSzMmDqTVisXzdN5NeW5fAUAdz/OTpRRkSzLTzT0uMq1OXCVlgaOj+FDxi0VKs+XYkLIcmnxUse09/4O7i+rdKhCgcuPSY7Y5ea7BpSDEuc5bQOdIgVw9l6z5+YlVP7Ok/V+J/TJg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nl/BNnEwGnkfp7ugboqdbNn2EBw+cplgSH3kLI39gkI=;
 b=eb/vQBtuMWIfNdwxD9ltn6debbnOpc+7PSbaj3lDB6p0/PYwcwmPe4UlO9oC8cDYJQ32wWKFiqXTXqnCq96MNhASRougp1MxkkFLf0jKsXhTN/6lBVO5R3UQLQE1/c2kDpKu8FbmgNdMfFDT2CAX4aYjpjMOJBY+KeLEpAY9dXo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 7/8] tools: Introduce the "xl dt-overlay {attach,detach}" commands
Date: Thu, 16 May 2024 18:03:29 +0800
Message-ID: <20240516100330.1433265-8-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042A8:EE_|BL3PR12MB6428:EE_
X-MS-Office365-Filtering-Correlation-Id: 1d26934f-4574-4069-9ef0-08dc758f7d19
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?TZy7jMliXsiLUTDfo09ghojyymhC1KLC0CRovvdLSdQjzUL54AxcPh/jorGv?=
 =?us-ascii?Q?THi4hp81jUB+BXMEtOOG5qH9Wx2AptIo83Etz1gqbVtAgtrmwWiM2Oxb32p5?=
 =?us-ascii?Q?5qxCP+IdZCDwPdVg30bpuPqdyNcNDPU5MnuiFFBsAPBtHKDlNWjFypW7tGu4?=
 =?us-ascii?Q?Idj0fHC6FbXH2C0jvqpYio+V1U3sVXlZJiJyM13bwR9XF+2dzWhiTsDzhzkF?=
 =?us-ascii?Q?s6owlBrwL9U9Q89lzK6gsWgEhIYeZ4yNbJa52jcGFx6exwuRlqUS5YdcgrOT?=
 =?us-ascii?Q?DUD02Q5gNDCQl4+rOhvgBQ6Z3QAfRFrKxfMjtAY0NMHB/kACtaIMzBleS9Ne?=
 =?us-ascii?Q?dG+tDR0+qrPfVSUqcZOz76VOw+GPjaYLpge6P9ruO6YXDqrt6k641UEmszBK?=
 =?us-ascii?Q?sUVpMRub21dyzscPUF61iHhttLyzmBxQQBeulkoFg35VGQrsUBqvv0zaVvIt?=
 =?us-ascii?Q?HBB+wqvVVzCNk6Gugw2i7OU0+f9qLsAIhjFJBNa6X/li4e2HprqDppY4qD+E?=
 =?us-ascii?Q?+adB5IJ5efqxw1wI79zti6cSqiE8L7cJW4eJZH1TfY6d2DIRQ20f5gJCESvg?=
 =?us-ascii?Q?OEuQFtV76f9kAnz5y0U5dkihfqOUvW3BD+esTQnSiz75u3w9DacdL4vA+seC?=
 =?us-ascii?Q?tm5lGYhQr4aGBh2MijXS2AphsCZ0ecDqI1Z8aebVjiL3YlXALdy9tYmGFBcI?=
 =?us-ascii?Q?AnEw6C8e/T9MMqAwW1GC03lb17mB/8OaWZwJxqwtP6poabUk6ytpO2JBxKgJ?=
 =?us-ascii?Q?CSJzbrBwM9AbrborIcG2IHonTBMXSjTdeUxcriTZAOv1aW6eY3C83mKXCYef?=
 =?us-ascii?Q?tAtZpg+LP7KtOVkh+Bm6ovJJUDmCBl/c64KkDQ9GeEMcH2+gY52S7Tiw7QDp?=
 =?us-ascii?Q?gqxVP+dH8u+b5sOVJ3UOkO/qc7n6/zVWR3GSMRrQQuI0fLHsxcFdXnvWSDRe?=
 =?us-ascii?Q?gVWCPJ9uXeeQ7UndGQDPCdLPIKEKAjJeUH3ioW1Sxx/zenOgE5shK7eTNY6Q?=
 =?us-ascii?Q?hyUMuBhdHSx69jcOxlCDAtCPvTU31myTyPoqdZFEcWj2wbmX7RWwdB0nEzBu?=
 =?us-ascii?Q?3tNbRZfr5xADn4FoCBYtcnKrpks3BK8agsg41s2csN0qyxTmWnxAhS0bj04I?=
 =?us-ascii?Q?P5vmUbxJ7A0LP3YwdBVDyRkIaMXOwR+aJ6Abtrk0ckPucLjMzGVP7pxBjJbU?=
 =?us-ascii?Q?4yCsiTAh+TSRYg3rL8VY82kCaXu33vofr21gZZEzCtTH0UjqcmHiyA5QpdIm?=
 =?us-ascii?Q?W374p4sQ/KolzOjhAShZLknXv+u7mJN7rDs5hJt04oFrYFLbZI1B6eZgUCAk?=
 =?us-ascii?Q?DZ06ghBIt0+XY0K9ZFnKSOYH7BXSOC8CM9qIPhgr1O7SZfDWikjZE4dmzJzz?=
 =?us-ascii?Q?hpjHgzy8dTgxeLYs95hyj+OoN0In?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:51.7809
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d26934f-4574-4069-9ef0-08dc758f7d19
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042A8.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6428

With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach/detach devices from the provided DT overlay to domains.
Support this by introducing a new set of "xl dt-overlay" commands and
related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
rework the command option parsing logic.

Since the addition of these two commands modifies the existing libxl
API libxl_dt_overlay(), also provide the backward compatible for it.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- New patch.
---
 tools/include/libxl.h               | 15 ++++++++++++-
 tools/include/xenctrl.h             |  3 +++
 tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++++++++
 tools/libs/light/libxl_dt_overlay.c | 30 ++++++++++++++++++++------
 tools/xl/xl_cmdtable.c              |  4 ++--
 tools/xl/xl_vmcontrol.c             | 33 +++++++++++++++++++----------
 6 files changed, 96 insertions(+), 20 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..27aab4bcee 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -2549,8 +2549,21 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 #if defined(__arm__) || defined(__aarch64__)
-int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
+#define LIBXL_DT_OVERLAY_ADD                   1
+#define LIBXL_DT_OVERLAY_REMOVE                2
+#define LIBXL_DT_OVERLAY_ATTACH                3
+#define LIBXL_DT_OVERLAY_DETACH                4
+
+int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay,
                      uint32_t overlay_size, uint8_t overlay_op);
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x041900
+int libxl_dt_overlay_0x041800(libxl_ctx *ctx, void *overlay,
+                              uint32_t overlay_size, uint8_t overlay_op);
+{
+    return libxl_dt_overlay(ctx, 0, overlay, overlay_size, overlay_op);
+}
+#define libxl_dt_overlay libxl_dt_overlay_0x041800
+#endif
 #endif
 
 /*
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..9ceca0cffc 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id);
 #endif
 
 /* Compat shims */
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
index c2224c4d15..ea1da522d1 100644
--- a/tools/libs/ctrl/xc_dt_overlay.c
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -48,3 +48,34 @@ err:
 
     return err;
 }
+
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id)
+{
+    int err;
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_dt_overlay,
+        .domain = domain_id,
+        .u.dt_overlay = {
+            .overlay_op = overlay_op,
+            .overlay_fdt_size = overlay_fdt_size,
+        }
+    };
+
+    DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+        goto err;
+
+    set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, overlay_fdt);
+
+    return err;
+}
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
index a6c709a6dc..9110b1efd2 100644
--- a/tools/libs/light/libxl_dt_overlay.c
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -41,8 +41,8 @@ static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size)
     return 0;
 }
 
-int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
-                     uint8_t overlay_op)
+int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay_dt,
+                     uint32_t overlay_dt_size, uint8_t overlay_op)
 {
     int rc;
     int r;
@@ -57,11 +57,29 @@ int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
         rc = 0;
     }
 
-    r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
-
-    if (r) {
-        LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
+    switch (overlay_op)
+    {
+    case LIBXL_DT_OVERLAY_ADD:
+    case LIBXL_DT_OVERLAY_REMOVE:
+        r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
+        if (r) {
+            LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
+            rc = ERROR_FAIL;
+        }
+        break;
+    case LIBXL_DT_OVERLAY_ATTACH:
+    case LIBXL_DT_OVERLAY_DETACH:
+        r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size,
+                                 overlay_op, domain_id);
+        if (r) {
+            LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__);
+            rc = ERROR_FAIL;
+        }
+        break;
+    default:
+        LOG(ERROR, "%s: Invalid overlay dtb op.", __func__);
         rc = ERROR_FAIL;
+        break;
     }
 
 out:
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 1f3c6b5897..37770b20e3 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = {
 #ifdef LIBXL_HAVE_DT_OVERLAY
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
-      "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>",
+      "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain",
+      "<operation=add|remove> <.dtbo> OR <operation=attach|detach> <.dtbo> <Domain>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 02575d5d36..53d1fa3655 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1268,32 +1268,43 @@ int main_create(int argc, char **argv)
 #ifdef LIBXL_HAVE_DT_OVERLAY
 int main_dt_overlay(int argc, char **argv)
 {
-    const char *overlay_ops = NULL;
     const char *overlay_config_file = NULL;
     void *overlay_dtb = NULL;
     int rc;
     uint8_t op;
     int overlay_dtb_size = 0;
-    const int overlay_add_op = 1;
-    const int overlay_remove_op = 2;
+    uint32_t domain_id = 0;
 
     if (argc < 2) {
         help("dt-overlay");
         return EXIT_FAILURE;
     }
 
-    overlay_ops = argv[1];
-    overlay_config_file = argv[2];
-
-    if (strcmp(overlay_ops, "add") == 0)
-        op = overlay_add_op;
-    else if (strcmp(overlay_ops, "remove") == 0)
-        op = overlay_remove_op;
+    if (strcmp(argv[optind], "add") == 0)
+        op = LIBXL_DT_OVERLAY_ADD;
+    else if (strcmp(argv[optind], "remove") == 0)
+        op = LIBXL_DT_OVERLAY_REMOVE;
+    else if (strcmp(argv[optind], "attach") == 0)
+        op = LIBXL_DT_OVERLAY_ATTACH;
+    else if (strcmp(argv[optind], "detach") == 0)
+        op = LIBXL_DT_OVERLAY_DETACH;
     else {
         fprintf(stderr, "Invalid dt overlay operation\n");
         return EXIT_FAILURE;
     }
 
+    overlay_config_file = argv[optind+1];
+
+    if (op == LIBXL_DT_OVERLAY_ATTACH || op == LIBXL_DT_OVERLAY_DETACH) {
+        if (argc <= optind + 2) {
+            fprintf(stderr, "Missing domain ID\n");
+            help("dt-overlay");
+            return EXIT_FAILURE;
+        } else {
+            domain_id = strtol(argv[optind+2], NULL, 10);
+        }
+    }
+
     if (overlay_config_file) {
         rc = libxl_read_file_contents(ctx, overlay_config_file,
                                       &overlay_dtb, &overlay_dtb_size);
@@ -1309,7 +1320,7 @@ int main_dt_overlay(int argc, char **argv)
         return EXIT_FAILURE;
     }
 
-    rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    rc = libxl_dt_overlay(ctx, domain_id, overlay_dtb, overlay_dtb_size, op);
 
     free(overlay_dtb);
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:04:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:04:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.722997.1127529 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxw-0007aI-IV; Thu, 16 May 2024 10:04:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 722997.1127529; Thu, 16 May 2024 10:04:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Xxw-0007ZA-EN; Thu, 16 May 2024 10:04:00 +0000
Received: by outflank-mailman (input) for mailman id 722997;
 Thu, 16 May 2024 10:03:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MS9v=MT=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7Xxv-0005T3-4q
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:59 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9cb6c228-136b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:03:58 +0200 (CEST)
Received: from PH7P220CA0105.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:32d::20)
 by DM4PR12MB5867.namprd12.prod.outlook.com (2603:10b6:8:66::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29; Thu, 16 May
 2024 10:03:54 +0000
Received: from CO1PEPF000042AA.namprd03.prod.outlook.com
 (2603:10b6:510:32d:cafe::3c) by PH7P220CA0105.outlook.office365.com
 (2603:10b6:510:32d::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29 via Frontend
 Transport; Thu, 16 May 2024 10:03:54 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000042AA.mail.protection.outlook.com (10.167.243.39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:53 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:03:52 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 05:03:51 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cb6c228-136b-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X6cDwxO82zVZLhGS1fy16FpelQP4WNG3Ys4u65CT48A/rvuVSWZQFSn3cORmA6L6v/oLpxHRUQJ6hs718YpXuGiLiYqre2+rjnDiLWLgkCWvn4/r0bonA1Cdq+lyhCa1bemVJtNntBz3EKMekuB3zy1+nh9DWtcs0zCM4uVxT2u9fet84BH/Jg/kVdqW3J/abTA6txstnJ9c6ykvrj3C/9x769vxCBFh4Quz/9G/8Xj4jJD2l0WRvpwkPE3Vvpco9BYFLVBFCh7e3bfdRAQgfqYhUBd7NaKDm6/BxRkd5VyesAcAECA24pjlaiKi6YLRAT32qGG2BOdbo/U6DDaruA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=v/Vywe/ogvtzCaenHHy7pk7r94uYXwgaQHO2LYJtq4Y=;
 b=d2Ac9Yv57g5c8k4qvUqRR9iV7ICX4xZMvKKRt+EhyRK0oHE3d9x3FeXPGPUyt08kSb9Xm9RXX03IoO+LeKbGFse7Ogt+jb+q0PcI+vX2SdQsbAWfxyHxkh7UzrvNs9/4j4ZEXbuTuorgOhgHaX1G1P4oAi/+SIHNYihVUiTL3iYsPrSW9DnnBYKEbTF0TxE9kHxYL3ly3wUyC5W+UXfK0M7oyN4ohO9WLEu3cYtXz3xxLUSNG8D8yZgTi2TVTnBZ4CK4nzf8Ki8+EKSpNGh9cfiB7NCiko1z6+v6EgH5wda/j8B7Fq1TiVQqNpwDy3hfb/4lTrdZWPAozU63P0VW4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v/Vywe/ogvtzCaenHHy7pk7r94uYXwgaQHO2LYJtq4Y=;
 b=dvIZDONT3W5ainpu7h0e34Hl0u3ueCu0dg9BqYh7BlftTnMGXXKxptrQWFQcOvJFaomsHUgHWTIWeu1TAbYZgk4Ux9KeexCKzab7RnSRFNUhGfF+UREKv4+LSJ7xkj2n0D87FSpC6hvBhSNL4vp6W/xy8jxqI3mgL2VWFPAjI8w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Vikram Garhwal <fnu.vikram@xilinx.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>
Subject: [PATCH v2 8/8] docs: Add device tree overlay documentation
Date: Thu, 16 May 2024 18:03:30 +0800
Message-ID: <20240516100330.1433265-9-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042AA:EE_|DM4PR12MB5867:EE_
X-MS-Office365-Filtering-Correlation-Id: ff737319-4604-4d2a-f520-08dc758f7e49
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?AHvP9pqvWSLHwdxEjSz16tS/r58hqMsutxIYmn6IiroeSLWQyfn9kX4tpRyJ?=
 =?us-ascii?Q?AmV8/UfBjjKJm75JMrHMo941QljY9LDp+gGhLXvOLYkR2VkbcMK8ysGyEM6c?=
 =?us-ascii?Q?8zlvdJHLaCZCZahsXb4He44Qimqaq0hfDnpXnY+9SYuU8fksT7nnTm05mOkj?=
 =?us-ascii?Q?i/3GrwgjOzKbjDurB/QrpUr9C0y6mXq+BgsAwFE4E45NEDWKtR5D2zj/vmoR?=
 =?us-ascii?Q?NCi7sjFJvBgEqZQY8HdBcC+A8OKetwVvhBGwK/h5ZEuubLNXqCvIzLZUY3AS?=
 =?us-ascii?Q?lwJG03P77iKvAhjZ+NRauotuj/Dr9dlNgte47Jp9jSF0UPvzt2Ywhq+H36Zn?=
 =?us-ascii?Q?uT5OGyxe/F+30XP38lWiODlOsGz9FNd5r8328GyrP6o+vPYpAaxTClZNfVMO?=
 =?us-ascii?Q?bhmXFLM7pC+8vnZrL7UlGvqunS+fKZKNzbGGDVGdv4qrodshpStc6L9tOFgd?=
 =?us-ascii?Q?M6DOmYJc+vIfrg4js8wLJgqAeYxcycJgV5ZoRoQ81Zwb+3JD6XDicAL//6qJ?=
 =?us-ascii?Q?8/wzBI5im86uFr47ZLacqh/2erKSrI2DPBPXHR67hp79GM7nm39XF4lfJaL3?=
 =?us-ascii?Q?YyXz88O6+zi2XZhiqv7BVD47hhHCZennWSGfExxZsdR7SMBkx/FG8tv5945S?=
 =?us-ascii?Q?GSfiAbkOoPmBHREnCFmckV9rxVsuiQzXKzG84341TNepDaFaUqxaW/wyA8iZ?=
 =?us-ascii?Q?gQskrn3HBctOhmaBHpXF5iJDkB8V4RFf8mLpu0bqWliXhS7pN5Ev7sGZB99/?=
 =?us-ascii?Q?JQLCWjy/dqNh6j3GaK3kc2XBGoLXuWtjohsnJ59VuFxnqJz2XOngKtkNBVWA?=
 =?us-ascii?Q?BJoqaSpF4OXcLz1T9IkvUufp3QF79eerG/KI3VXE8JgkK34vpt7PBTv7jtNU?=
 =?us-ascii?Q?XQH1iHBdBQoyzs7HzeZQRP4PaBky0fTSRCH2bJx3guApuRwdrr+H6vkgWQMX?=
 =?us-ascii?Q?5ao63aAJklHHeZPB3ywvYEDkpHs5TT904k4J05t+yiFWLOPQ3e/PNEfuWua6?=
 =?us-ascii?Q?wLW3YDH/U1Y5xYCSNKurSSHwPjEHyV6hUL2jO9YTApdEV5L4QWYeybcX0Ue9?=
 =?us-ascii?Q?GtP5AKT3QqMUcI61A0C0vvwFV6xlzLDweaTXs00k6egbNqaC3RlWhF/ALWCZ?=
 =?us-ascii?Q?9bqgbF3wJa6LxZSg9MDJQITH4dIbIvtPC1NzcPinbGoc7dVthylf/Y8i7zqP?=
 =?us-ascii?Q?xH+FNFP4pgf5dCfWbRBhJlstbnKkfKF+IM04Yo7peNMewhLYUKZJkhmhr3ko?=
 =?us-ascii?Q?GMnRC3joNAeX9MUEOjY5ry6Du3PryJOX3iVAwizV+Je8YPlmle5aGTQXcbom?=
 =?us-ascii?Q?9kM6g1BQ6xsBTkvL5Uu+ckyb?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:53.7910
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ff737319-4604-4d2a-f520-08dc758f7e49
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042AA.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5867

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Update the content based on the changes in this version.
---
 docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
 create mode 100644 docs/misc/arm/overlay.txt

diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
new file mode 100644
index 0000000000..811a6de369
--- /dev/null
+++ b/docs/misc/arm/overlay.txt
@@ -0,0 +1,99 @@
+# Device Tree Overlays support in Xen
+
+Xen now supports dynamic device assignment to running domains,
+i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and
+attaching/detaching them to/from a running domain with given $domid.
+
+Dynamic node assignment works in two steps:
+
+## Add/Remove device tree overlay to/from Xen device tree
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
+
+## Attach/Detach device from the DT overlay to/from domain
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor attach/detach the device to/from the user-provided $domid by
+   mapping/unmapping node resources in the DT overlay.
+
+# Examples
+
+Here are a few examples on how to use it.
+
+## Dom0 device add
+
+For assigning a device tree overlay to Dom0, user should firstly properly
+prepare the DT overlay. More information about device tree overlays can be
+found in [1]. Then, in Dom0, enter the following:
+
+    (dom0) xl dt-overlay add overlay.dtbo
+
+This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
+
+To assign the newly added device from the dtbo to Dom0:
+
+    (dom0) xl dt-overlay attach overlay.dtbo 0
+
+Next, if the user wants to add the same device tree overlay to dom0
+Linux, execute the following:
+
+    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally if needed, the relevant Linux kernel drive can be loaded using:
+
+    (dom0) modprobe module_name.ko
+
+## Dom0 device remove
+
+For removing the device from Dom0, first detach the device from Dom0:
+
+    (dom0) xl dt-overlay detach overlay.dtbo 0
+
+NOTE: The user is expected to unload any Linux kernel modules which
+might be accessing the devices in overlay.dtbo before detach the device.
+Detaching devices without unloading the modules might result in a crash.
+
+Then remove the overlay from Xen device tree:
+
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+## DomU device add/remove
+
+All the nodes in dtbo will be assigned to a domain; the user will need
+to prepare the dtb for the domU. For example, the `interrupt-parent` property
+of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`.
+Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
+
+User will need to create the DomU with below properties properly configured
+in the xl config file:
+- `iomem`
+- `passthrough` (if IOMMU is needed)
+
+User will also need to modprobe the relevant drivers.
+
+Example for domU device add:
+
+    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
+    (dom0) xl dt-overlay attach overlay.dtbo $domid
+    (dom0) xl console $domid                         # To access $domid console
+
+Next, if the user needs to modify/prepare the overlay.dtbo suitable for
+the domU:
+
+    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally, if needed, the relevant Linux kernel drive can be probed:
+
+    (domU) modprobe module_name.ko
+
+Example for domU overlay remove:
+
+    (dom0) xl dt-overlay detach overlay.dtbo $domid
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:09:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:09:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723018.1127539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y3O-0001uK-9f; Thu, 16 May 2024 10:09:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723018.1127539; Thu, 16 May 2024 10:09:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y3O-0001uD-5h; Thu, 16 May 2024 10:09:38 +0000
Received: by outflank-mailman (input) for mailman id 723018;
 Thu, 16 May 2024 10:09:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Y3N-0001u7-IQ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:09:37 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 66309779-136c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:09:36 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so281386566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:09:36 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d92sm975284866b.98.2024.05.16.03.09.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:09:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66309779-136c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715854176; x=1716458976; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=JR9aeLFq4ryr+Yj+8/m/OwHuECrOwQkncyie1IKBjiI=;
        b=a2br3zxdMqlgKQuES3zJkCUdm+O14uBjQaGnjiH1RiD6gZyXhrAo9w+5Et0HIy5rHE
         CUJdu202i69xB0/tHhdujBUnGHAaqmHtE32HE7pQQtBzbSQO7GI0wEaYczDx6BbEs+1l
         8oS54irTGaF+9xOYQRgplfnOoew1iYZfwOFIXWOjZ3i3b635TzX/4iNLmCGx2tw7Eaik
         YzlHF0JkQMluWvNpcXkrDgJRTIoS5sAFtWUIS93fFTg62DaORMpDApnqMK/DgbplvRhD
         BL70vVGETM+QO/r2XXggg7FOqL96r9C6Itm6UZIxwvrXknwkgpx8/+wndca7qtIEw2rw
         sjAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715854176; x=1716458976;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JR9aeLFq4ryr+Yj+8/m/OwHuECrOwQkncyie1IKBjiI=;
        b=YQf0pRVApm8iWtjnoD2uI7xK9ooz3JrlGngjx5k/eRCsaZ690aUXSO/YSKX8ROXf1I
         z4tAOLdNeBU3yAhjMzQ6fZHDWHCsHt1Lm2YLZaF52UR4W4BVsGdqwCqcuj9nEWz7GY3v
         a0heljgNXwBjiksJ3xVEaeYNzf9IvNtkphC7pF2wuWqu/Q0PQwWqyAk4LnLc9mkJ52h2
         lzjycPaxFEP7Koh+KOo91mkKNvEaLuI1h4O06E0DKdILCM3gnfgQtkE+QnE0im9BD8Xp
         pyHLE9JZnIVxHdDHN/1azLDMkentttySX+sTKhL+8A8jIlMmtOEkZyV6nj1jemTDqwRp
         cx2g==
X-Forwarded-Encrypted: i=1; AJvYcCVLUueNOuCLFjP8NLNJ5qByNLDJyVriNuOecz9ev6skEDmSgtmJ68WVOKCWJDgaFhe+hEfeVfdRXDhNxxqdA683J23ZpPc94lIoDLkyYFg=
X-Gm-Message-State: AOJu0Yxvr4hHA8Y4RMkoxtheHwS9NegRxxQ8OdriQl3l4C2tKUXBrHIZ
	NeWEdHXP1iBHAyk9JWbONHLJwfMPoYSuJPO1vkbWzdD4zkOf2420zrVSZ/5EcQ==
X-Google-Smtp-Source: AGHT+IFV5M8qk/sfaer7kl772vjhVQrVuwUWZr29IEfUV6zihYFi+G1sgVcyN9ZxGIIXWlisuy4zdw==
X-Received: by 2002:a17:907:1118:b0:a59:c5c2:a320 with SMTP id a640c23a62f3a-a5a2d581b19mr1207651966b.23.1715854175994;
        Thu, 16 May 2024 03:09:35 -0700 (PDT)
Message-ID: <55062fd5-89cc-4c68-a303-24922192344d@suse.com>
Date: Thu, 16 May 2024 12:09:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 01/15] x86: introduce AMD-V and Intel VT-x Kconfig
 options
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <3f2168a337a192336e9a7fb797185c39978db11b.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3f2168a337a192336e9a7fb797185c39978db11b.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 10:59, Sergiy Kibrik wrote:
> From: Xenia Ragiadakou <burzalodowa@gmail.com>
> 
> Introduce two new Kconfig options, SVM and VMX, to allow code
> specific to each virtualization technology to be separated and, when not
> required, stripped.
> CONFIG_SVM will be used to enable virtual machine extensions on platforms that
> implement the AMD Virtualization Technology (AMD-V).
> CONFIG_VMX will be used to enable virtual machine extensions on platforms that
> implement the Intel Virtualization Technology (Intel VT-x).
> 
> Both features depend on HVM support.
> 
> Since, at this point, disabling any of them would cause Xen to not compile,
> the options are enabled by default if HVM and are not selectable by the user.
> 
> No functional change intended.
> 
> Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

In principle I'm okay with this, so
Acked-by: Jan Beulich <jbeulich@suse.com>
Nevertheless, ...

> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -122,6 +122,12 @@ config HVM
>  
>  	  If unsure, say Y.
>  
> +config SVM
> +	def_bool HVM
> +
> +config VMX
> +	def_bool HVM

... VT-x having a sub-technology abbreviated as SVM (Secure Virtual Memory),
I wonder if we wouldn't be better off using AMD_SVM and INTEL_VMX. Andrew,
Roger?

> --- a/xen/arch/x86/mm/Makefile
> +++ b/xen/arch/x86/mm/Makefile
> @@ -10,6 +10,7 @@ obj-$(CONFIG_MEM_SHARING) += mem_sharing.o
>  obj-$(CONFIG_HVM) += nested.o
>  obj-$(CONFIG_HVM) += p2m.o
>  obj-y += p2m-basic.o
> -obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o p2m-pt.o
> +obj-$(CONFIG_VMX) += p2m-ept.o
> +obj-$(CONFIG_HVM) += p2m-pod.o p2m-pt.o
>  obj-y += paging.o
>  obj-y += physmap.o

Just as a remark, with no request for you to change anything right here:
With SVM=n and SHADOW_PAGING=n, I don't think we really need to build
p2m-pt.c. Yet of course this may require adjustments outside of the
Makefile to then actually build correctly.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:14:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723033.1127559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7k-0005HF-2N; Thu, 16 May 2024 10:14:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723033.1127559; Thu, 16 May 2024 10:14:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7j-0005H7-W0; Thu, 16 May 2024 10:14:07 +0000
Received: by outflank-mailman (input) for mailman id 723033;
 Thu, 16 May 2024 10:14:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7Y7i-00052X-Tt
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:14:06 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05f3e1b8-136d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:14:05 +0200 (CEST)
Received: from MN2PR20CA0059.namprd20.prod.outlook.com (2603:10b6:208:235::28)
 by SA1PR12MB8641.namprd12.prod.outlook.com (2603:10b6:806:388::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 10:14:01 +0000
Received: from BN2PEPF00004FC0.namprd04.prod.outlook.com
 (2603:10b6:208:235:cafe::b5) by MN2PR20CA0059.outlook.office365.com
 (2603:10b6:208:235::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 10:14:01 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF00004FC0.mail.protection.outlook.com (10.167.243.186) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:14:01 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:13:58 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05f3e1b8-136d-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YRQp0yuCNR3BeI3WPxoz2WCg/EKLyFAoYbtLLSFUJ2s/BJtutN5R+j4f1UWcENaUVjn1ZW2mM9IvbA3LSy6bRVbJJGpcNfgIbREcFLawVolHNEzLawinhrPJt2XOjamJ6bhREGUzgcPk6CKW31ArayB9KGAf4CnDDxVLGHm6GJxM82oVFVR8MXjo0jyYkEP9tsGiydBT1ZqOgAHNSPilsxBLWfYPL6p4y/Zl2nVLvdTMVfKAoimwdj4mdIQ3QubFml005uuIE6dxTVW+NXR1plgpAbRZwWI6EY+Ic1iHO6YHnbT+fqwcbVlhPf5fcJDOeRENcIkjoQXwV3HD/y+auA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zEMEB3wKzvfQ/AFFTBXplx0jxAU0bH2rBrLniIkwOOA=;
 b=AIsbOFUg3VneErNZ05xdoHsABdN+L8GRRfsQe5v82N2HPuvFpC4NLtHGqFQcyhkkfOnroM8XjZVCUeiuJ1wVHGhEFH1Ou/4k8Z68C7ScDJR8EJoCYRyvEVAxrfv8oeLTMVL8QG9FNWe2nnGcG48iUkDrBA2BxBgGPEq83CSbwibPvMeudfBn8cR8eR451XCZS5Rh6hVYheEobKDfIi+6pZDCh5WDYEv2yMOS7MXkvIzf1mEZbw1gUVbFJMfc9pQfX9mnOkBl+um97cXnhEdFGSziMCqKT7ActEVeN8B7vqdZwdNk2vjKn3mrFQdIuvAVxNPzaju7n4jJaNgBJnGAYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nongnu.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zEMEB3wKzvfQ/AFFTBXplx0jxAU0bH2rBrLniIkwOOA=;
 b=sEw+PTHaaM1FtBHH001p/x2iUaszlu+tQhNb6KBsCdMz4x6FwbyEjX31cb8FCwrAh0CVhV1MtPcAX3SKlkyjLW7hHv4cvZhpdbNzt4Ig6u0dFp+ZlXlq5y3j7bO3CP+oTYM4c7dCLmGQm/d569ciloFROf8fjQLR6lM6Cv04OvM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <qemu-devel@nongnu.org>, Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>
CC: <xen-devel@lists.xenproject.org>, Huang Rui <Ray.Huang@amd.com>, "Jiqian
 Chen" <Jiqian.Chen@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: [RFC QEMU PATCH v7 1/1] xen/pci: get gsi for passthrough devices
Date: Thu, 16 May 2024 18:13:38 +0800
Message-ID: <20240516101338.86763-2-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240516101338.86763-1-Jiqian.Chen@amd.com>
References: <20240516101338.86763-1-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF00004FC0:EE_|SA1PR12MB8641:EE_
X-MS-Office365-Filtering-Correlation-Id: 3b0a7b4e-0268-48e6-06a1-08dc7590e839
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?/XYQwvs2o10Nbm2rItE1LMjJME7VAqbb2XvB90/DZn8G4TZczpFce9FeK10M?=
 =?us-ascii?Q?kmkR9aEpwbyTGkXWaU0VEZA64z5ffjuGzbEgXHsDUGR8rMY2x6yDqQ7f+gbf?=
 =?us-ascii?Q?X4zaTpJGA6sjKmlXGMlxyD0Jhwvmszd6ar5DD0BE3moGCkvLEkOiNmTgWv4t?=
 =?us-ascii?Q?+NgYJhizyKjrSAbNgjiUytGMWrvGbXNykzHVMA88bfuDyIQPit+bRMmdQOTF?=
 =?us-ascii?Q?+bUyVVRMT6X6Oj4wNHOMwqD36dsa0tHVeDYmHj+zUV5haVHgF436cyxrTKW5?=
 =?us-ascii?Q?aQtpGyna6m/maF61EGcJzsc+xSmjxpVOq91NdteZ7MaegiSguqP6iM1+34Ca?=
 =?us-ascii?Q?qt8oisbCRxH0hc3fx4tnbG1BZ82BuFDMIHwSOAUr2RhTtuzXUgh/0IxpPaxd?=
 =?us-ascii?Q?i8c75Xg0xyZZupFPYWGtuUD9ujeHzCY5eyqeAgekAyHH5lgyvPV5ahIeTR9J?=
 =?us-ascii?Q?9A9kVGwHgnD4++AKTH3gn1J4i4JTOeJIuDntzWT1OHemZIRhK0pekPYZ2Wlp?=
 =?us-ascii?Q?E+/ivoBZ28YJZCl32fZ1D4BrzXtgKeHWTiAuAqqfatfu31PahxXqX9Zaa3hI?=
 =?us-ascii?Q?+IdV1A0NBpF48dQbvFxt3+v2dqcQy5OGSyJvbci0uduOcMEEYMnBDR/hOPu6?=
 =?us-ascii?Q?8G4spXOFroYqXr9UtvJiMsyKNKzkxv8wJQjw5Ao3AdBm6qgul5zNXsMGJ1Qy?=
 =?us-ascii?Q?8T6atpv/R0MR2cYiH8m5QKs32w504fUzqQI7TvYrPxSQ7mBsn9cUq1oRjIUy?=
 =?us-ascii?Q?tgr9OR7Y9xCPy5xkRhYZTgF0vAOpENalBNRhGAI7claXlinztjq6lvaogU7b?=
 =?us-ascii?Q?G2Ci+ypx7ZhbV5HhFKhqyTHjtmTxUUfhd3uJBWjSaViba5sy7dWyUsSUkcfZ?=
 =?us-ascii?Q?qo7DHdciaaKLaH1g+jHdLQlkkzKPS/pWeWs/Q3Qnt4TtFKhpKyVPwasObyeE?=
 =?us-ascii?Q?gxnvRu5NkMR4b9kCL1k10t7K+nH7HUCMLNJqqSrADCkqDwPZXIkyXqnsiy7g?=
 =?us-ascii?Q?ayDpnANqO7jFF5CtLRsesXL9BcJdDKTBxFnI5CAOH+/crIbW0n/k44X2uzkL?=
 =?us-ascii?Q?TCHEF9fskR1YagobmdThhpxgoDZL5dS440UdlZcRmXJXjw9234bFMxbP1P2R?=
 =?us-ascii?Q?LoBulWnZnn/6ZNitJT/PqlfJYeeK5OmMOE5z6vNclxvisg5mQGwCoBbyFH6t?=
 =?us-ascii?Q?5h1TKNa/EoaHJLqVHP1KmpdzdK/YurhLPiObAwtP574RWqiDP7zoUkLSiG0B?=
 =?us-ascii?Q?eC635rdf9mqfd1U8aG7eKT4Ggnpn8LdqsbJid1TxkFypJSukNdhhNRg4qJRt?=
 =?us-ascii?Q?cwarUGEBkNRwAVE0dxpGlWH/1uodUFAocGruFumN344IVueVGhugnO7bmqzT?=
 =?us-ascii?Q?3KfchVY=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:14:01.1090
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3b0a7b4e-0268-48e6-06a1-08dc7590e839
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF00004FC0.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8641

In PVH dom0, it uses the linux local interrupt mechanism,
when it allocs irq for a gsi, it is dynamic, and follow
the principle of applying first, distributing first. And
the irq number is alloced from small to large, but the
applying gsi number is not, may gsi 38 comes before gsi
28, that causes the irq number is not equal with the gsi
number. And when passthrough a device, qemu wants to use
gsi to map pirq, xen_pt_realize->xc_physdev_map_pirq, but
the gsi number is got from file
/sys/bus/pci/devices/<sbdf>/irq in current code, so it
will fail when mapping.

Get gsi by using new function supported by Xen tools.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
---
 hw/xen/xen-host-pci-device.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
index 8c6e9a1716a2..2fe6a60434ba 100644
--- a/hw/xen/xen-host-pci-device.c
+++ b/hw/xen/xen-host-pci-device.c
@@ -10,6 +10,7 @@
 #include "qapi/error.h"
 #include "qemu/cutils.h"
 #include "xen-host-pci-device.h"
+#include "hw/xen/xen_native.h"
 
 #define XEN_HOST_PCI_MAX_EXT_CAP \
     ((PCIE_CONFIG_SPACE_SIZE - PCI_CONFIG_SPACE_SIZE) / (PCI_CAP_SIZEOF + 4))
@@ -329,12 +330,17 @@ int xen_host_pci_find_ext_cap_offset(XenHostPCIDevice *d, uint32_t cap)
     return -1;
 }
 
+#define PCI_SBDF(seg, bus, dev, func) \
+            ((((uint32_t)(seg)) << 16) | \
+            (PCI_BUILD_BDF(bus, PCI_DEVFN(dev, func))))
+
 void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
                              uint8_t bus, uint8_t dev, uint8_t func,
                              Error **errp)
 {
     ERRP_GUARD();
     unsigned int v;
+    uint32_t sdbf;
 
     d->config_fd = -1;
     d->domain = domain;
@@ -364,11 +370,16 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
     }
     d->device_id = v;
 
-    xen_host_pci_get_dec_value(d, "irq", &v, errp);
-    if (*errp) {
-        goto error;
+    sdbf = PCI_SBDF(domain, bus, dev, func);
+    d->irq = xc_physdev_gsi_from_dev(xen_xc, sdbf);
+    /* fail to get gsi, fallback to irq */
+    if (d->irq == -1) {
+        xen_host_pci_get_dec_value(d, "irq", &v, errp);
+        if (*errp) {
+            goto error;
+        }
+        d->irq = v;
     }
-    d->irq = v;
 
     xen_host_pci_get_hex_value(d, "class", &v, errp);
     if (*errp) {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:14:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723032.1127548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7i-00052m-Ri; Thu, 16 May 2024 10:14:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723032.1127548; Thu, 16 May 2024 10:14:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7i-00052f-P3; Thu, 16 May 2024 10:14:06 +0000
Received: by outflank-mailman (input) for mailman id 723032;
 Thu, 16 May 2024 10:14:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hz7T=MT=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7Y7h-00052X-Fx
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:14:05 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 04af569b-136d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:14:03 +0200 (CEST)
Received: from MN2PR20CA0056.namprd20.prod.outlook.com (2603:10b6:208:235::25)
 by PH7PR12MB5594.namprd12.prod.outlook.com (2603:10b6:510:134::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 10:13:59 +0000
Received: from BN2PEPF00004FC0.namprd04.prod.outlook.com
 (2603:10b6:208:235:cafe::69) by MN2PR20CA0056.outlook.office365.com
 (2603:10b6:208:235::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 16 May 2024 10:13:59 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF00004FC0.mail.protection.outlook.com (10.167.243.186) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:13:58 +0000
Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 05:13:56 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04af569b-136d-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZIS2bynvrcPvBeM+VaQRh76BhfeoIkwdC3SEo7XPhoUFY8jJuGuDDjb/4CpalRc41NDbjVqAvA4jl4mvzjxPpFer24ra6dy8zUxuREU8A57DMfQ0d/ddH/hVTWYWpQLoBWD9bmXypQ6l0krPvhjGD4mOY3d7ar+/aO+laha+wsBJ7PMkPvWp9Hf8fXhMlaWW4A3BjfuHM5aVFRDJkf7i9z5zWjDnvDp2+PEE2E7PlzO6d5cgmvnkGxOWqPd8GemzyoRKjgSqqxrrXkORU1mqvFnlmPe4/cuaGU7DW4DAW+AvN7TCKJC59DZwDD8hEbhlgNCLUycViiupKGeHx1N+KQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zgznFSSStOfkFRmozyGTygvqO4riUbrmtAZEM7Wq7T4=;
 b=EtYNf7J6Tb56LCbvepZP4mpWRkaQFpQtJUG8PLrkseZdAP/ttuajQJre0YLj8j/0zN6OOQWIhLCYHEbXmZ5L8ZDbDMC3bbOf7djslzbzpf3xw6yIucd1p3/P1IrrUIj9NjoNBXUQtFdfQGRO9Kjef20svEyd1boBQWm1aAqoAlgmZU3XVLk/KORU8dMWAfOh0HUA0xwEnPJnO45lGIe1+0xylFTRSrdjxpoNbBvcupc1GQEuVQaxdnoE9jdBC+t7BeLtTFv7LP3qDFnmFBWL8X2vTy7tXKLUdvdVcsdhVbbk458F/ryD2hbt10mHP9v0J2Wtl1t/7+I2++SUjoBWoQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nongnu.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zgznFSSStOfkFRmozyGTygvqO4riUbrmtAZEM7Wq7T4=;
 b=VLRxx7kQLk42gY0kX+b2uHWdVKYhmnP1qjiRFd+BH6HNbyfAzBmjeaA8o0gw76nUI/F/fu6qViMMwBttNrf9Bcf2HUEYK554IxB2D8wk1jylKEnmZEkk0bhJDaE5zPMldZ59X7pKsc7XNG+AmuLmedT32EJm4fOIuT4wjbNKSj4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Jiqian Chen <Jiqian.Chen@amd.com>
To: <qemu-devel@nongnu.org>, Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>
CC: <xen-devel@lists.xenproject.org>, Huang Rui <Ray.Huang@amd.com>, "Jiqian
 Chen" <Jiqian.Chen@amd.com>
Subject: [RFC QEMU PATCH v7 0/1] Support device passthrough when dom0 is PVH on Xen
Date: Thu, 16 May 2024 18:13:37 +0800
Message-ID: <20240516101338.86763-1-Jiqian.Chen@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com
 (10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF00004FC0:EE_|PH7PR12MB5594:EE_
X-MS-Office365-Filtering-Correlation-Id: 1337c990-1155-4805-3e5d-08dc7590e6cc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?TB4dredHahDU2xvntDyE4vjxnaPkImyt3qC1CzaQsHMfXr4mpTVN4H3XF7+F?=
 =?us-ascii?Q?UD7ULUaY0PV9QRSRc58FdfUwQ8XOJ3IGSBVpQJ55uG3gGsEnpWDRMslYPXje?=
 =?us-ascii?Q?MYYf2AysAadC74fawVChRd66SeqH6aQlfZETjxv70lJ13I2yVAAQfMxNwGnW?=
 =?us-ascii?Q?Uffu9z0QltnePeagsBkS2ast92sUy/5A+tRBvgepe1ekdJU7VT6VDu786QjU?=
 =?us-ascii?Q?dBJMTgn+Dws6TV24O1kbRzwAITt+TWQbb5nLWVRdRmnJ8+2rMC0pKOtushOh?=
 =?us-ascii?Q?bVP81ABqywMgxFFTEWkRKU69D8Ya2VoamIN4QYA64uo5orxza2mXmvEp5dGx?=
 =?us-ascii?Q?clKYfoXjSXdZvMEa7IPM6fE8imED+GQ8lEbc46BFpa8+NyiRdn89YBwPW9j/?=
 =?us-ascii?Q?eZVVal2fIy0st9h3mCA7Xngzr1dVK45bg6uOYnWyBf2zQWwVLrEmQQriegl6?=
 =?us-ascii?Q?Br5EZU1LnHQ3+eKZ+sJ6CvI+kSK3L+qxPXdkU25PUlRq+gtpfMrw0EHxcfAx?=
 =?us-ascii?Q?Vb3rO2sryyFeciowFhx0ZX/VMf1Nelv9R9ZxVDmLbdHlSsFLzr0oP4PKJtbt?=
 =?us-ascii?Q?r091qHNNgKmb0UkftRVJj5EiA+JMSjR0kWwLcCTGhx6augaioAFO/yqA47oI?=
 =?us-ascii?Q?CZKyjoEOGOsLgEf+SclO3a107KVfs5lFUNDw04YTmyKtdd+jl1RdacwJb51G?=
 =?us-ascii?Q?CBRpVdLoTe7rjQduHnW3+Mdzk9SKVmW5bZko2m0VIfyQmRnCqTHGcfqxxaNW?=
 =?us-ascii?Q?3QoXJfls6v3yKlwlLCsc0bcHwH+FiJmVIH21N58HYGcOG1DD1hbkVd0ZGXfc?=
 =?us-ascii?Q?xvhvwhqiNKHnuMbdg+uUlLddCE+55QkXlxkaG3g+q0f/JxDjjrpbcbetWJeX?=
 =?us-ascii?Q?ZZwJFA3GCS/n0rTTmDMBAaJ0eEXFM0uu74YXC7I3YjVGdDcXY1XsnRIZolr9?=
 =?us-ascii?Q?2XBV9XkSySMdZ3Q/SaGuX/ChCCec7p+t5HurEh8f4h4UdbZxcRNRp8nQjoGX?=
 =?us-ascii?Q?Pu3FKFHPBIbdG2iiuQpkEAeM5O8SisFyMK1VSp4RDlK5TDoVZk7lMZppl/57?=
 =?us-ascii?Q?EzjBh+OqjgRON2b/eIbOto1czBn7NMCWldy/Ao/9Eq1lNOJbnFXa0VfU8iZF?=
 =?us-ascii?Q?JHrsTEdeH4MKIZjVLVj268FKiT3Yo8R6yNO9Lq7pA+fo6VPEapbIhsZcfaqS?=
 =?us-ascii?Q?B+Ffhaspc/MX4VTUiDgiTCtsZSolHf08AsTp2jqy1eithAQNrbRxan8tM3pc?=
 =?us-ascii?Q?7pW3z45bl5SpJHLymd0FUfGbi4F7pvvpZQGqaIqRRuznJekF5UHaCZ+INrHb?=
 =?us-ascii?Q?nfyRJV0jcn8agoQzgfMgu3A2?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:13:58.7340
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1337c990-1155-4805-3e5d-08dc7590e6cc
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF00004FC0.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5594

Hi All,
This is v7 series to support passthrough on Xen when dom0 is PVH.
v6->v7 changes:
* Due to changes in the implementation of obtaining gsi in the kernel and Xen. Change to use
  xc_physdev_gsi_from_dev, that requires passing in sbdf instead of irq.


Best regards,
Jiqian Chen



v5->v6 changes:
* Due to changes in the implementation of obtaining gsi in the kernel and Xen. Change to use
  xc_physdev_gsi_from_irq, instead of gsi sysfs.


v4->v5 changes:
* Add review by Stefano


v3->v4 changes:
* Add gsi into struct XenHostPCIDevice and use gsi number that read from gsi sysfs
  if it exists, if there is no gsi sysfs, still use irq.


v2->v3 changes:
* Du to changes in the implementation of the second patch on kernel side(that adds
  a new sysfs for gsi instead of a new syscall), so read gsi number from the sysfs of gsi.


Below is the description of v2 cover letter:
This patch is the v2 of the implementation of passthrough when dom0 is PVH on Xen.
Issues we encountered:
1. failed to map pirq for gsi
Problem: qemu will call xc_physdev_map_pirq() to map a passthrough device\u2019s gsi to pirq in
function xen_pt_realize(). But failed.

Reason: According to the implement of xc_physdev_map_pirq(), it needs gsi instead of irq,
but qemu pass irq to it and treat irq as gsi, it is got from file
/sys/bus/pci/devices/xxxx:xx:xx.x/irq in function xen_host_pci_device_get(). But actually
the gsi number is not equal with irq. On PVH dom0, when it allocates irq for a gsi in
function acpi_register_gsi_ioapic(), allocation is dynamic, and follow the principle of
applying first, distributing first. And if you debug the kernel codes
(see function __irq_alloc_descs), you will find the irq number is allocated from small to
large by order, but the applying gsi number is not, gsi 38 may come before gsi 28, that
causes gsi 38 get a smaller irq number than gsi 28, and then gsi != irq.

Solution: we can record the relation between gsi and irq, then when userspace(qemu) want
to use gsi, we can do a translation. The third patch of kernel(xen/privcmd: Add new syscall
to get gsi from irq) records all the relations in acpi_register_gsi_xen_pvh() when dom0
initialize pci devices, and provide a syscall for userspace to get the gsi from irq. The
third patch of xen(tools: Add new function to get gsi from irq) add a new function
xc_physdev_gsi_from_irq() to call the new syscall added on kernel side.
And then userspace can use that function to get gsi. Then xc_physdev_map_pirq() will success.

This v2 on qemu side is the same as the v1 
(qemu https://lore.kernel.org/xen-devel/20230312092244.451465-19-ray.huang@amd.com/), just call
xc_physdev_gsi_from_irq() to get gsi from irq.

Jiqian Chen (1):
  xen/pci: get gsi for passthrough devices

 hw/xen/xen-host-pci-device.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:14:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:14:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723035.1127572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7k-0005V3-R4; Thu, 16 May 2024 10:14:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723035.1127572; Thu, 16 May 2024 10:14:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Y7k-0005TO-Lx; Thu, 16 May 2024 10:14:08 +0000
Received: by outflank-mailman (input) for mailman id 723035;
 Thu, 16 May 2024 10:14:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yRp+=MT=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s7Y7j-00052e-QL
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:14:07 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06f97efc-136d-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:14:06 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-56e6a1edecfso4312011a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:14:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06f97efc-136d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715854445; x=1716459245; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=GKCXxpkICfYIaEzaewiT1/VwlZgIz/HgNpx+JI+R2Dw=;
        b=Cy1CMp1EFNyCihcD2IzC+X5CE93FlI4+k/a4YQzHA6seYLN89477dxErd48T8CGEjc
         OmnxaKegSeUkNS4xTzqg81GiVzHJz/dCm/nbG8CvL2tWUJj+GbUOrFw/JW+5sKijRh7z
         krMIthF7E0SH36+8F+Yf6VTOMAJTbfKD85B2M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715854445; x=1716459245;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GKCXxpkICfYIaEzaewiT1/VwlZgIz/HgNpx+JI+R2Dw=;
        b=lVQnrqSAY+ykuA9w9ZkMjqGR0LSh8pKLk+a7toRubQ0IRGiEhJptBMt+JjnkNcE66Z
         SVrAxdwEkh1QErkKw2OjASHT7j/kN1EplCjCeNRYZO7oVz9zRFvELthrJ96x+LpUqX0X
         5eksBQ02MJu1wY5vQgMPxAT1iDEihxCc+IsTixbheWLmyIZ0jQUi5/CABLI4DpoyaetM
         dKIFCya9JbZTNl9+4JfnKkJ9Mjb5tdIiv0ZBvuYUuZNIBTgwnzIXmlH1dWZ3djVrGyim
         GvmNwNEyl3VsCSxF3llTjR055h7P5GKfWeLQ4rHvKeJ34Qs3Gfks/2kPWr1wpR7OtFor
         PkPA==
X-Gm-Message-State: AOJu0YxQuwqvKbcSG9BPSGr5Su/zWrf7gmEqIU6q2rIrh3EZk66KK56j
	eM/IVa1wnlpLhUKDa4rdJnuYdPivTks45qWXuJf45eRrTJrgdiJji+3HkSmjhXMmLsKF3T2MKYC
	DH0EQelFmWA0J5+czKJjsK5FzErpXSGt+950moa2jUH5sWPzVcEt4qA==
X-Google-Smtp-Source: AGHT+IHbJX3233x6N0PE+zLEyio0aEicm/SjprHbvZjU4WLUo+mgh4vXxvCguS/gJNFXLbdE5KwPqS89fxek76MTXVE=
X-Received: by 2002:a50:d55e:0:b0:574:f09a:e7aa with SMTP id
 4fb4d7f45d1cf-574f09ae8dbmr4047615a12.21.1715854445248; Thu, 16 May 2024
 03:14:05 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Thu, 16 May 2024 11:13:29 +0100
Message-ID: <CAO-mL=y4b1N8hDenjNo9JWw9Pu+xR14=5RHrwb0fzystKTr1Ww@mail.gmail.com>
Subject: [REMINDER] - Submit your Design Sessions for Xen Summit!
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000b21f1706188f7eaf"

--000000000000b21f1706188f7eaf
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Xen Community,

*Our design sessions are now open for Xen Summit! *

If you've attended Xen Summit before, you might be familiar with the
process.

For anyone who hasn't done so before, please follow the instructions below,
using the link to create an account
<https://design-sessions.xenproject.org/>.
Once you've created your account, you'll be asked to verify using the code
below. That's it, you're in!

We aim to have design sessions with a virtual element (using Jitsi). This
is free for the community to join in and listen, and will be hosted by an
attendee in the session.

If you're going to be an *in-person attendee*, you can propose a session an=
d
vote on sessions you would like to see discussed.
If you're going to be a *virtual attendee*, you can vote on sessions you
would like to see discussed. You can still propose a session, but please
note there will be no professional audio or visual equipment in person for
these sessions.

The final schedule will be allocated and arranged by the highest-voted
sessions.

Virtual links for the community to join Xen Summit design sessions will be
shared closer to the event.

If you have any questions, please let me know.

For in-person tickets, click here
<https://events.linuxfoundation.org/xen-project-summit/register/>.

------------------------------


We look forward to holding the Design Sessions at the upcoming Xen Project
Summit. The design sessions will be on Wednesday, 5 June, and Thursday, 6
June 2024.

We encourage everyone to submit a Design Session, the verification code is:
=E2=80=9C*LFXEN24*=E2=80=9D.

*SUBMIT A DESIGN SESSION* <https://design-sessions.xenproject.org/>



The process involves the following steps:

   - Anyone interested can propose
   <https://design-sessions.xenproject.org/discussion/new> a topic.
   - All participants review the list of sessions
   <https://design-sessions.xenproject.org/list/discussion>, indicating
   their interest in attending each one.
   - The session scheduler optimizes the schedule
   <https://design-sessions.xenproject.org/schedule> to accommodate as many
   preferences as possible.

Participants can also propose long-form talks by adding [TALK] to the
session title.

For suggested topics, sample Design Session submissions, and more tips
check out the Xen Design Session page
<https://events.linuxfoundation.org/xen-project-summit/program/design-sessi=
ons/>
for
more information.


Best Regards,
Xen Project Events Team


------------------------------


Many thanks,
Kelly Choi

Community Manager
Xen Project

--000000000000b21f1706188f7eaf
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div dir=3D"ltr" class=3D"gmail_signature"><div dir=
=3D"ltr"><div>Hello Xen Community,</div><div><b><br></b></div><div><b>Our=
=C2=A0<span class=3D"gmail-il">design</span>=C2=A0<span class=3D"gmail-il">=
sessions</span>=C2=A0are now open for Xen Summit!=C2=A0</b></div><div><br><=
/div><div>If you&#39;ve attended Xen Summit before, you might be familiar w=
ith the process.=C2=A0</div><div><br></div><div>For anyone who hasn&#39;t d=
one so before, please follow the instructions below, using the=C2=A0<a href=
=3D"https://design-sessions.xenproject.org/" target=3D"_blank">link to crea=
te an account</a>.=C2=A0</div><div>Once you&#39;ve created your account, yo=
u&#39;ll be asked to verify using the code below. That&#39;s it, you&#39;re=
 in!</div><div><br></div><div>We aim to have=C2=A0<span class=3D"gmail-il">=
design</span>=C2=A0<span class=3D"gmail-il">sessions</span>=C2=A0with a vir=
tual element (using Jitsi). This is free for the community to join in and l=
isten, and will be hosted by an attendee in the=C2=A0<span class=3D"gmail-i=
l">session</span>.=C2=A0<br></div><div><br></div><div>If you&#39;re going t=
o be an=C2=A0<b><i>in-person attendee</i></b>, you can propose a=C2=A0<span=
 class=3D"gmail-il">session</span>=C2=A0and vote on=C2=A0<span class=3D"gma=
il-il">sessions</span>=C2=A0you would like to see discussed.=C2=A0</div><di=
v>If you&#39;re going to be a=C2=A0<b><i>virtual attendee</i></b>, you can =
vote on=C2=A0<span class=3D"gmail-il">sessions</span>=C2=A0you would like t=
o see discussed. You can still propose a=C2=A0<span class=3D"gmail-il">sess=
ion</span>, but please note there will be no professional audio or visual e=
quipment in person for these=C2=A0<span class=3D"gmail-il">sessions</span>.=
=C2=A0</div><div><br></div><div>The final schedule will be allocated and ar=
ranged by the highest-voted=C2=A0<span class=3D"gmail-il">sessions</span>.=
=C2=A0</div><div><br></div><div>Virtual links for the community to join Xen=
 Summit=C2=A0<span class=3D"gmail-il">design</span>=C2=A0<span class=3D"gma=
il-il">sessions</span>=C2=A0will be shared closer to the event.=C2=A0=C2=A0=
<br></div><div><br></div><div>If you=C2=A0have any questions,=C2=A0please l=
et me know.</div><div><p class=3D"MsoNormal">For in-person tickets,=C2=A0<a=
 href=3D"https://events.linuxfoundation.org/xen-project-summit/register/" t=
arget=3D"_blank">click here</a>.=C2=A0</p><p class=3D"MsoNormal"><u></u></p=
><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" sty=
le=3D"width:949.612px"><tbody><tr><td style=3D"padding:0.75pt"><div class=
=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><hr size=3D"1" =
width=3D"100%" noshade align=3D"center" style=3D"color:rgb(160,160,160)"></=
div><p class=3D"MsoNormal"></p></td></tr></tbody></table></div><div><br></d=
iv></div></div></div><div class=3D"gmail_quote"><div><div lang=3D"EN-US" id=
=3D"m_6486684977004734091m_-2685003709366885060body" style=3D"overflow-wrap=
: break-word;"><div><div id=3D"m_6486684977004734091m_-2685003709366885060m=
ail-editor-reference-message-container"><div><table border=3D"0" cellspacin=
g=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px;float:lef=
t"><tbody><tr><td style=3D"background:rgb(239,240,242);padding:0in"><div al=
ign=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" widt=
h=3D"100%" style=3D"width:1051.99px;float:left"><tbody><tr><td style=3D"pad=
ding:0in"><div align=3D"center"><table border=3D"0" cellspacing=3D"0" cellp=
adding=3D"0" width=3D"100%" style=3D"width:1051.99px;background:white"><tbo=
dy><tr><td style=3D"padding:0in"><div align=3D"center"><table border=3D"0" =
cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px=
"><tbody><tr><td valign=3D"top" style=3D"padding:0in"><table border=3D"0" c=
ellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"100%" style=3D"w=
idth:1051.99px"><tbody><tr><td style=3D"padding:0in"><table border=3D"0" ce=
llspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px">=
<tbody><tr><td style=3D"padding:0in"><div align=3D"center"><table border=3D=
"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.=
99px"><tbody><tr><td style=3D"padding:0in"><p class=3D"MsoNormal" align=3D"=
center" style=3D"text-align:center"><img border=3D"0" width=3D"699" id=3D"m=
_6486684977004734091m_-2685003709366885060_x0000_i1028" src=3D"https://cust=
om.cvent.com/64779B2AC74D4D2BBFF4459DE93A5253/pix/d770244914e047019777029a3=
2e20c53.png?d=3D699" class=3D"gmail-CToWUd gmail-a6T" tabindex=3D"0" style=
=3D"cursor: pointer; outline: 0px; width: 7.2812in;"></p></td></tr></tbody>=
</table></div><p class=3D"MsoNormal" align=3D"center" style=3D"text-align:c=
enter"></p><div align=3D"center"><table border=3D"0" cellspacing=3D"0" cell=
padding=3D"0" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td style=
=3D"padding:0in"></td></tr></tbody></table></div><p class=3D"MsoNormal" ali=
gn=3D"center" style=3D"text-align:center"><span style=3D"font-size:10pt;fon=
t-family:&quot;Times New Roman&quot;,serif"><u></u><u></u></span></p></td><=
/tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></d=
iv></td></tr></tbody></table></div></td></tr><tr><td style=3D"padding:0in;w=
ord-break:break-word"><div align=3D"center"><table border=3D"0" cellspacing=
=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px;background=
:white;float:left"><tbody><tr><td style=3D"padding:0in;float:left"><div ali=
gn=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=
=3D"100%" style=3D"width:1051.99px;float:left"><tbody><tr><td valign=3D"top=
" style=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=
=3D"0" align=3D"left" width=3D"100%" style=3D"width:1051.99px;word-break:br=
eak-word"><tbody><tr><td style=3D"padding:0in"><table border=3D"0" cellspac=
ing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px"><tbody=
><tr><td style=3D"padding:11.25pt 11.25pt 0in"><table border=3D"0" cellspac=
ing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1021.99px"><tbody=
><tr><td style=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" cellpa=
dding=3D"0" width=3D"100%" style=3D"width:1021.99px"><tbody><tr><td style=
=3D"padding:3.75pt 0in"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"=
><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:black">=
We look forward to holding the=C2=A0<span class=3D"gmail-il">Design</span>=
=C2=A0<span class=3D"gmail-il">Sessions</span>=C2=A0at the upcoming Xen Pro=
ject Summit. The=C2=A0<span class=3D"gmail-il">design</span>=C2=A0<span cla=
ss=3D"gmail-il">sessions</span>=C2=A0will be on Wednesday, 5 June, and Thur=
sday, 6 June 2024.<br><br>We encourage everyone to submit a=C2=A0<span clas=
s=3D"gmail-il">Design</span>=C2=A0<span class=3D"gmail-il">Session</span>, =
the=C2=A0</span><span style=3D"font-size:10.5pt;font-family:Arial,sans-seri=
f;color:rgb(33,33,33)">verification code</span><span style=3D"font-size:10.=
5pt;font-family:Arial,sans-serif;color:black">=C2=A0is: =E2=80=9C</span><b>=
<span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,3=
3,33)">LFXEN24</span></b><span style=3D"font-size:10.5pt;font-family:Arial,=
sans-serif;color:black">=E2=80=9D.<u></u><u></u></span></p></td></tr></tbod=
y></table></td></tr></tbody></table></td></tr></tbody></table></td></tr></t=
body></table></td></tr><tr><td valign=3D"top" style=3D"padding:0in"><table =
border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"10=
0%" style=3D"width:1051.99px;float:left"><tbody><tr><td style=3D"padding:0i=
n"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" s=
tyle=3D"width:1051.99px;border-radius:15px"><tbody><tr><td style=3D"padding=
:11.25pt"><div align=3D"center"><table border=3D"1" cellspacing=3D"0" cellp=
adding=3D"0" style=3D"background:rgb(114,190,68);border:4.5pt solid rgb(114=
,190,68)"><tbody><tr><td style=3D"border:none;padding:3.75pt 15pt"><p class=
=3D"MsoNormal" align=3D"center" style=3D"text-align:center"><span style=3D"=
color:black"><a href=3D"https://design-sessions.xenproject.org/" target=3D"=
_blank"><b><span style=3D"font-size:13pt;font-family:Arial,sans-serif;color=
:white;text-decoration-line:none">SUBMIT A=C2=A0<span class=3D"gmail-il">DE=
SIGN</span>=C2=A0<span class=3D"gmail-il">SESSION</span></span></b></a></sp=
an></p></td></tr></tbody></table></div><p class=3D"MsoNormal" align=3D"cent=
er" style=3D"text-align:center"><span style=3D"font-size:1pt">=C2=A0</span>=
</p></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td v=
align=3D"top" style=3D"padding:0in;word-break:break-word"><table border=3D"=
0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"100%" style=
=3D"width:1051.99px;float:left"><tbody><tr><td style=3D"padding:0in;float:l=
eft"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%"=
 style=3D"width:872.825px;word-break:break-word"><tbody><tr><td style=3D"pa=
dding:0in 11.25pt 11.25pt"><table border=3D"0" cellspacing=3D"0" cellpaddin=
g=3D"0" width=3D"100%" style=3D"width:842.825px"><tbody><tr><td style=3D"pa=
dding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D=
"100%" style=3D"width:842.825px"><tbody><tr><td style=3D"padding:3.75pt 0in=
"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-s=
ize:10.5pt;font-family:Arial,sans-serif;color:rgb(33,37,41)">The process in=
volves the following steps:</span><span style=3D"font-size:10.5pt;font-fami=
ly:Arial,sans-serif;color:black"><u></u><u></u></span></p></td></tr><tr><td=
 style=3D"padding:3.75pt 0in;word-break:break-word"><ul type=3D"disc"><li c=
lass=3D"MsoNormal" style=3D"margin-left:15px;color:black;line-height:13.5pt=
"><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33=
,37,41)">Anyone interested can=C2=A0</span><a href=3D"https://design-sessio=
ns.xenproject.org/discussion/new" target=3D"_blank"><span style=3D"font-siz=
e:10.5pt;font-family:Arial,sans-serif;color:rgb(13,110,253)">propose</span>=
</a><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(=
33,37,41)">=C2=A0a topic.</span><span style=3D"font-size:10.5pt;font-family=
:Arial,sans-serif"><u></u><u></u></span></li><li class=3D"MsoNormal" style=
=3D"margin-left:15px;color:black;line-height:13.5pt"><span style=3D"font-si=
ze:10.5pt;font-family:Arial,sans-serif;color:rgb(33,37,41)">All participant=
s review the=C2=A0</span><a href=3D"https://design-sessions.xenproject.org/=
list/discussion" target=3D"_blank"><span style=3D"font-size:10.5pt;font-fam=
ily:Arial,sans-serif;color:rgb(13,110,253)">list of=C2=A0<span class=3D"gma=
il-il">sessions</span></span></a><span style=3D"font-size:10.5pt;font-famil=
y:Arial,sans-serif;color:rgb(33,37,41)">, indicating their interest in atte=
nding each one.</span><span style=3D"font-size:10.5pt;font-family:Arial,san=
s-serif"><u></u><u></u></span></li><li class=3D"MsoNormal" style=3D"margin-=
left:15px;color:black;line-height:13.5pt"><span style=3D"font-size:10.5pt;f=
ont-family:Arial,sans-serif;color:rgb(33,37,41)">The=C2=A0<span class=3D"gm=
ail-il">session</span>=C2=A0scheduler optimizes the=C2=A0</span><a href=3D"=
https://design-sessions.xenproject.org/schedule" target=3D"_blank"><span st=
yle=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(13,110,253)"=
>schedule</span></a><span style=3D"font-size:10.5pt;font-family:Arial,sans-=
serif;color:rgb(33,37,41)">=C2=A0to accommodate as many preferences as poss=
ible.</span><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif"><=
u></u><u></u></span></li></ul></td></tr><tr><td style=3D"padding:3.75pt 0in=
;word-break:break-word"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"=
><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,=
37,41)">Participants can also propose long-form talks by adding [TALK] to t=
he=C2=A0<span class=3D"gmail-il">session</span>=C2=A0title.</span><span sty=
le=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:black"><u></u><u>=
</u></span></p></td></tr><tr><td style=3D"padding:3.75pt 0in;word-break:bre=
ak-word"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D=
"font-size:10.5pt;font-family:Arial,sans-serif;color:black">For suggested t=
opics, sample=C2=A0<span class=3D"gmail-il">Design</span>=C2=A0<span class=
=3D"gmail-il">Session</span>=C2=A0submissions, and more tips check out the=
=C2=A0</span><a href=3D"https://events.linuxfoundation.org/xen-project-summ=
it/program/design-sessions/" target=3D"_blank"><span style=3D"font-size:10.=
5pt;font-family:Arial,sans-serif">Xen=C2=A0<span class=3D"gmail-il">Design<=
/span>=C2=A0<span class=3D"gmail-il">Session</span>=C2=A0page</span></a><sp=
an style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:black">=C2=
=A0for more information.<u></u><u></u></span></p></td></tr><tr><td style=3D=
"padding:3.75pt 0in;word-break:break-word"><p class=3D"MsoNormal" style=3D"=
line-height:13.5pt"><span style=3D"font-size:10.5pt;font-family:Arial,sans-=
serif;color:black"><br>Best Regards,<br>Xen Project Events Team<u></u><u></=
u></span></p></td></tr></tbody></table></td></tr></tbody></table></td></tr>=
</tbody></table></td></tr></tbody></table></td></tr></tbody></table></div><=
/td></tr></tbody></table></div></td></tr><tr><td style=3D"padding:0in"><div=
 align=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" w=
idth=3D"100%" style=3D"width:1051.99px;background:rgb(98,171,48);float:left=
"><tbody><tr><td style=3D"padding:0in"><div align=3D"center"><table border=
=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:10=
51.99px;float:left"><tbody><tr><td valign=3D"top" style=3D"padding:0in"><ta=
ble border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=
=3D"100%" style=3D"width:1051.99px"><tbody><tr><td style=3D"padding:0in"><t=
able border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=
=3D"width:1051.99px"><tbody><tr><td style=3D"padding:11.25pt"></td></tr></t=
body></table></td></tr></tbody></table></td></tr></tbody></table></div></td=
></tr></tbody></table></div></td></tr></tbody></table></div></td></tr></tbo=
dy></table><p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p><table border=3D"=
0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.9=
9px"><tbody><tr><td style=3D"padding:0.75pt"><div class=3D"MsoNormal" align=
=3D"center" style=3D"text-align:center"><hr size=3D"1" width=3D"100%" nosha=
de align=3D"center" style=3D"color:rgb(160,160,160)"></div><p class=3D"MsoN=
ormal"><br></p></td></tr></tbody></table></div></div></div></div></div></di=
v><div><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_s=
ignature"><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div=
><br></div><div><div style=3D"color:rgb(136,136,136)">Community Manager</di=
v><div style=3D"color:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></=
div></div></div></div>

--000000000000b21f1706188f7eaf--


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:20:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:20:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723069.1127606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YDn-000138-TG; Thu, 16 May 2024 10:20:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723069.1127606; Thu, 16 May 2024 10:20:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YDn-000131-QX; Thu, 16 May 2024 10:20:23 +0000
Received: by outflank-mailman (input) for mailman id 723069;
 Thu, 16 May 2024 10:20:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YDn-00012v-6f
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:20:23 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e65f4a4b-136d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:20:21 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso305073966b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:20:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b0177asm972536566b.173.2024.05.16.03.20.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:20:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e65f4a4b-136d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715854820; x=1716459620; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ydp7O/RXU9B2RxfhCENK2SvhA9RmVIcwvX7guS8hQzc=;
        b=YS8Td27hXd+sSp2R9Sj+iRuHi98WwEy8zEbBZOeV2XY9psWplNd/F49BrP4w7hEYa9
         gd4v13aMoOvfrQ4J3ZlQShgo+f7Spc9TQDrAdRm1QVaHq9Lwy2cTJunmwHVl4msff8NY
         nfuRQPAmm+MaQ7d3nm32pVxFmmOQl1g0bd3+AKXUzHnIk8bK2seMl2eTmp7btM3XeNYp
         aTGBJxjX2ITERxnkA8QrjPONDCbpxEvkP0sByXzsOcRQLfjpzWWMXLim32gwmESjmQvg
         Zmil0tztCuS3vU95IOb0StLlVLMTlape2NKT5ooeoox6iBQOYTIZVwC+JNV+5p+xd1sy
         L8Tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715854820; x=1716459620;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ydp7O/RXU9B2RxfhCENK2SvhA9RmVIcwvX7guS8hQzc=;
        b=HXMV2X1lM1MjSL5SE+xGtqQN/FnsR7JPNkaYDhZD6FIsBbPHezRT4YID9oJ3ytu0km
         lrwVjS8DPSwTZJUqc9aD3AC2BaXjsrPyjwJE+4/MzmjcILL+Y5EYQia1Uq/rj7TiT18l
         uphdGVdk+75g4b/ep+ADtuMX09gVf2JQpVroZvbNdvmIt40vJAjGK+j9KQPMX9mhubXw
         y5+aZ+aYY7D4W+3tQUccjaie4yfR8ydX3YeBhEeOu2tIIsyy1U4PGYJatj7Esto5ScjT
         gnaML0DBE13DtB2ybdghkBMPk3r7p/37uSB4OZ4vHEqZ2s3g3xbn6IAtD4ka84LXY59F
         +4Og==
X-Forwarded-Encrypted: i=1; AJvYcCVs0SLo/Fdk4QutJrDDNhylRrimSe15wq922aTrupLGyvKgApruamj6SiC0VTYSQGn8aZm0ve3d6qnUYU/qC3NadsP57ffA7zHT/BnnI0g=
X-Gm-Message-State: AOJu0YwBVbHG+jju72Tw84/8YTHT4GKZvmxfaEAbkYYYF5zSGtbAwXuB
	+4LZSR7Y+Xrp1lzgBURiiDl8jCGjx82DfOcTamLXefa3y7fsfoGttfw2R/j4hA==
X-Google-Smtp-Source: AGHT+IEiQw4B2ESmjtaRVj2ir/c//ql9hmJseoZPSICA9hCOkKzLCCDOhfroJpcwOoX5Qd3cNS9hOQ==
X-Received: by 2002:a17:906:fc05:b0:a59:ae9b:c661 with SMTP id a640c23a62f3a-a5a2d5d49b3mr1112159666b.40.1715854820437;
        Thu, 16 May 2024 03:20:20 -0700 (PDT)
Message-ID: <1e0dfa77-680c-4ab3-9634-e787517c3d85@suse.com>
Date: Thu, 16 May 2024 12:20:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 02/15] x86/monitor: guard altp2m usage
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <01767c3f98a88999d4b8ed3ae742ad66a0921ba3.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <01767c3f98a88999d4b8ed3ae742ad66a0921ba3.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:01, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/hvm/monitor.c
> +++ b/xen/arch/x86/hvm/monitor.c
> @@ -262,6 +262,8 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t gfn, uint32_t pfec,
>      struct vcpu *curr = current;
>      vm_event_request_t req = {};
>      paddr_t gpa = (gfn_to_gaddr(gfn) | (gla & ~PAGE_MASK));
> +    unsigned int altp2m_idx = altp2m_active(curr->domain) ?
> +                              altp2m_vcpu_idx(curr) : 0;
>      int rc;
>  
>      ASSERT(curr->arch.vm_event->send_event);
> @@ -270,7 +272,7 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t gfn, uint32_t pfec,
>       * p2m_get_mem_access() can fail from a invalid MFN and return -ESRCH
>       * in which case access must be restricted.
>       */
> -    rc = p2m_get_mem_access(curr->domain, gfn, &access, altp2m_vcpu_idx(curr));
> +    rc = p2m_get_mem_access(curr->domain, gfn, &access, altp2m_idx);

I think something like this wants taking care of by the function itself.
asm/altp2m.h already has certain stubs (the conditional of which I expect
a later change will switch to CONFIG_ALTP2M); you'd need to just add one
more. Then the other (and any future) users of the function would be
covered as well.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:26:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:26:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723081.1127618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YJP-0002lQ-ID; Thu, 16 May 2024 10:26:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723081.1127618; Thu, 16 May 2024 10:26:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YJP-0002lJ-Df; Thu, 16 May 2024 10:26:11 +0000
Received: by outflank-mailman (input) for mailman id 723081;
 Thu, 16 May 2024 10:26:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YJO-0002lD-FK
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:26:10 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b58c674a-136e-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:26:08 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e428242a38so6996551fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:26:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17892450sm969297966b.63.2024.05.16.03.26.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:26:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b58c674a-136e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715855168; x=1716459968; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jstn+m1oTnzwXt5NJbmy4qbQU0ne++k/74yPZCYCL9c=;
        b=JnP8nuuaFa/oKFXtMCdQBOxd8XV5D0GhwhFd04VRzxuEeVkte6VXKqW4JCjr7QF41G
         5ASuO4VhLOH3PPPaw6CSL3QAmb/j4ZhBygWA0urxIbD2ymIABxhLVLNwjpbJ8KSOzE7n
         aPINQHvAB4KWWQrGJXoA0+K4h+kUn05pazXtbML5g9fIAARYRt7gj1qpVlRgRUPpuiAH
         U30HCsM/325DEUqZF8hM5mkl20f+J+D5921zh0Ju0nC+3qX11Rtgs8FZYD7YQP/BxvAq
         TsK/6sf3TSzU0Ug0FSYHiNItcQFwvSf2Znb/N19U6frLq6+OxALmgnwnr63DdV9pCyT0
         22AA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715855168; x=1716459968;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jstn+m1oTnzwXt5NJbmy4qbQU0ne++k/74yPZCYCL9c=;
        b=smEvZKPvC/Dgr3vgR/LpF5LrX1XavoWR4xT2PsLTRmKmX7k7s9jzTouM2pAgtdYFwM
         ThTEpGGx9kdtKEtQIJJXqTX3/HL6nbI2ipQwefNC2ajYq23Yk3W5Cu6wKWE94es3x9UT
         T8IPrLnz+iEiKeW3g2bHR4YdHDAnJWkT8EaKLNeoGFYH1Umci+oX1990RJT0gxHRnWON
         mSNn1VTlfCjeWtJHpPQ8nJiUOU2zQqUykEd2v4LSvQzeKsY7A5c7oom0d4nUn4Tp9dkK
         z7kufytChN3nOxNdg0Pii5y6t+fMrIWdyuUByrjrULik4d4x/nANFP/B2WPYH8xmV6/f
         2eDg==
X-Forwarded-Encrypted: i=1; AJvYcCU7vQutOpjhuzzvNHshtgq3mapfkLZ/6uUTAFLi8q7Zdsegu7XaYDOjZinYuKmWyhmKQ3fR2VPmVkts9Vz1yjt3m4gXymSZYjt1I8umkww=
X-Gm-Message-State: AOJu0YyWKSRUoNiXuEhKZXeBuKC0OW9tgPQMpest2joUFztcasbI0hXu
	/QyCaNw7ulKdvP8IaDVguCOQlm5GbGnAiUEojMN4of2dyrF0vBNbgCxuVgVVuQ==
X-Google-Smtp-Source: AGHT+IFoL++S7bKYiDkYjwQv/Hd00Yn+rjxzX+1IVi0OvG1j4hzkVM1vVyTJG2h/lnkSSPgLmJ3CYQ==
X-Received: by 2002:a05:651c:2114:b0:2e1:cb0f:4e1e with SMTP id 38308e7fff4ca-2e51fc34061mr170353091fa.2.1715855168059;
        Thu, 16 May 2024 03:26:08 -0700 (PDT)
Message-ID: <72e5802d-db4f-4d9b-a2ea-fc767ef2dd44@suse.com>
Date: Thu, 16 May 2024 12:26:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 03/15] x86/p2m: guard altp2m routines
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <d4c537b1bcac4b301b16336ea3407834fc3076b1.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d4c537b1bcac4b301b16336ea3407834fc3076b1.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:03, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/mm/p2m-basic.c
> +++ b/xen/arch/x86/mm/p2m-basic.c
> @@ -126,13 +126,15 @@ int p2m_init(struct domain *d)
>          return rc;
>      }
>  
> -    rc = p2m_init_altp2m(d);
> -    if ( rc )
> +    if ( hvm_altp2m_supported() )
>      {
> -        p2m_teardown_hostp2m(d);
> -        p2m_teardown_nestedp2m(d);
> +        rc = p2m_init_altp2m(d);
> +        if ( rc )
> +        {
> +            p2m_teardown_hostp2m(d);
> +            p2m_teardown_nestedp2m(d);
> +        }

With less code churn and less indentation:

    rc = hvm_altp2m_supported() ? p2m_init_altp2m(d) : 0;
    if ( rc )
    {
        p2m_teardown_hostp2m(d);
        p2m_teardown_nestedp2m(d);
    }

>      }
> -
>      return rc;
>  }

Please don't remove the blank line ahead of the main return of a function.

> --- a/xen/arch/x86/mm/p2m-ept.c
> +++ b/xen/arch/x86/mm/p2m-ept.c
> @@ -986,7 +986,7 @@ out:
>      if ( is_epte_present(&old_entry) )
>          ept_free_entry(p2m, &old_entry, target);
>  
> -    if ( entry_written && p2m_is_hostp2m(p2m) )
> +    if ( entry_written && p2m_is_hostp2m(p2m) && hvm_altp2m_supported())

I agree with Stefano's ordering comment here, btw.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:34:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:34:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723101.1127626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YRI-0006qL-8d; Thu, 16 May 2024 10:34:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723101.1127626; Thu, 16 May 2024 10:34:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YRI-0006qE-64; Thu, 16 May 2024 10:34:20 +0000
Received: by outflank-mailman (input) for mailman id 723101;
 Thu, 16 May 2024 10:34:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7YRG-0006p9-Rh
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:34:18 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d89aa13a-136f-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:34:16 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-52388d9ca98so1115318e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:34:16 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d36a6sm2882529e87.164.2024.05.16.03.34.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 03:34:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d89aa13a-136f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715855656; x=1716460456; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=hjwzz+vFkptf2K4QAxJOv9UPNzPEVbyefBu/50UUC4Q=;
        b=dvG9tyTG93YCVz48Wl88ayEonNlzyjwB1/aP6mB7ztPlSXo2OLFFVy69yyZMfXhCwc
         eUlZ21WYbe0VsyZtE8FCrVILNkKQLvcn33oC8n2iOrRmqgC6TeqOEuj14FSAy5hqrfOJ
         8z8zddDyPpAy6Hh8CUWoG1OcJvuugZJX6UYUbVIk9TA+1ypcPV5I1ncDzRzYF+GGoMrC
         YegKi8FLmxxxk2cJVTPNDF//NMzIExNGZdXBtBabkMAfACqMEx4zWbXpIVXFfRnkDek9
         UpaEsY032N7uMla6i9wj3Yf2NMN6ZXioUffwgmMLQ74QBbsME327VwkLT2VGZGq480s1
         1mWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715855656; x=1716460456;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=hjwzz+vFkptf2K4QAxJOv9UPNzPEVbyefBu/50UUC4Q=;
        b=DW2UcF8iFDbp/XQuqKZE+AkH86bYcGFH/KxQqyg0+mTPrhczZa+GcKLq8at4getHUJ
         CxdbhT3WTkKn5E+iSCJ/vzpIMYqtWwgqte1cM9AZLwzxo5I62k8j1+MV1D7PID7HpymT
         9r5ajpZVstRI1UglIUr4EECyJOjsGiHU+hwj4NrmBCaO5tzGaZ6ynwLqzrfrxc1M4Ak+
         gCvCIza+SPJSNSGw5gYWH10uJ51EoYxDy7eUOW1DlsYoX+Tix/lf1ULXVSMZHGJ6D1iD
         ZRXlUemT3ujl0HDbXGNEwo+kuvS+GOpArs5OSmUd6sgOHhJGQSCXffWRfMcWe23W8j26
         k0/w==
X-Forwarded-Encrypted: i=1; AJvYcCVct8kdIAvNWwWOyaPdKPFXchmsYaNHMyhhdcjMa9Np/X4UpGqHG3DbtJ3cdJfe7ofqIGKtZjY1SlT06CwZUnFVl9LclNrUAbCx6ZmJeAA=
X-Gm-Message-State: AOJu0YxdoUIh6/FeXLlimxdptO6IqKPDAIsajGPE0bYXbkKhJfa9u+cK
	egEJrg2l8TkvODxkjWfNggIw0TGYix8epCvqfdQ/MGphPcM0VM+d
X-Google-Smtp-Source: AGHT+IGtoLh5Go/7uW6EW6y0xCflCMhSac2wPXoh9TYPL0vfMGjEZbWe/nJr+iTYs+HeUqZlmNT26Q==
X-Received: by 2002:ac2:5f59:0:b0:521:92f6:3d34 with SMTP id 2adb3069b0e04-5220fd7c838mr14336002e87.22.1715855655878;
        Thu, 16 May 2024 03:34:15 -0700 (PDT)
Message-ID: <c8bd47e8f8558d88b4d5a4a09ea10728006fd4d3.camel@gmail.com>
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Thu, 16 May 2024 12:34:14 +0200
In-Reply-To: <77c6e497-631d-4f92-bd38-8ab7dba4841d@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
	 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
	 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
	 <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
	 <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
	 <77c6e497-631d-4f92-bd38-8ab7dba4841d@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-16 at 09:04 +0200, Jan Beulich wrote:
> On 15.05.2024 19:03, Oleksii K. wrote:
> > On Wed, 2024-05-15 at 17:41 +0200, Jan Beulich wrote:
> > > On 15.05.2024 17:29, Oleksii K. wrote:
> > > > On Wed, 2024-05-15 at 10:52 +0200, Jan Beulich wrote:
> > > > > On 06.05.2024 12:15, Oleksii Kurochko wrote:
> > > > > > The following generic functions were introduced:
> > > > > > * test_bit
> > > > > > * generic__test_and_set_bit
> > > > > > * generic__test_and_clear_bit
> > > > > > * generic__test_and_change_bit
> > > > > >=20
> > > > > > Also, the patch introduces the following generics which are
> > > > > > used by the functions mentioned above:
> > > > > > * BITOP_BITS_PER_WORD
> > > > > > * BITOP_MASK
> > > > > > * BITOP_WORD
> > > > > > * BITOP_TYPE
> > > > > >=20
> > > > > > These functions and macros can be useful for architectures
> > > > > > that don't have corresponding arch-specific instructions.
> > > > >=20
> > > > > Logically this paragraph may better move ahead of the BITOP_*
> > > > > one.
> > > > >=20
> > > > > > Because of that x86 has the following check in the macros
> > > > > > test_bit(),
> > > > > > __test_and_set_bit(), __test_and_clear_bit(),
> > > > > > __test_and_change_bit():
> > > > > > =C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size=
();
> > > > > > It was necessary to make bitop bad size check generic too,
> > > > > > so
> > > > > > arch_check_bitop_size() was introduced.
> > > > >=20
> > > > > Not anymore, with the most recent adjustments? There's
> > > > > nothing
> > > > > arch-
> > > > > specific anymore in the checking.
> > > > >=20
> > > > > > @@ -183,7 +180,7 @@ static inline int test_and_set_bit(int
> > > > > > nr,
> > > > > > volatile void *addr)
> > > > > > =C2=A0 * If two examples of this operation race, one can appear
> > > > > > to
> > > > > > succeed
> > > > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple acc=
esses
> > > > > > with
> > > > > > a
> > > > > > lock.
> > > > > > =C2=A0 */
> > > > > > -static inline int __test_and_set_bit(int nr, void *addr)
> > > > > > +static inline int arch__test_and_set_bit(int nr, volatile
> > > > > > void
> > > > > > *addr)
> > > > >=20
> > > > > I think I raised this point before: Why arch__ here, ...
> > > > >=20
> > > > > > @@ -232,7 +226,7 @@ static inline int
> > > > > > test_and_clear_bit(int
> > > > > > nr,
> > > > > > volatile void *addr)
> > > > > > =C2=A0 * If two examples of this operation race, one can appear
> > > > > > to
> > > > > > succeed
> > > > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple acc=
esses
> > > > > > with
> > > > > > a
> > > > > > lock.
> > > > > > =C2=A0 */
> > > > > > -static inline int __test_and_clear_bit(int nr, void *addr)
> > > > > > +static inline int arch__test_and_clear_bit(int nr,
> > > > > > volatile
> > > > > > void
> > > > > > *addr)
> > > > >=20
> > > > > ... here, ...
> > > > >=20
> > > > > > @@ -243,13 +237,10 @@ static inline int
> > > > > > __test_and_clear_bit(int
> > > > > > nr, void *addr)
> > > > > > =C2=A0
> > > > > > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > > > > > =C2=A0}
> > > > > > -#define __test_and_clear_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_siz=
e();=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > -=C2=A0=C2=A0=C2=A0 __test_and_clear_bit(nr, addr);=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > -})
> > > > > > +#define arch__test_and_clear_bit arch__test_and_clear_bit
> > > > > > =C2=A0
> > > > > > =C2=A0/* WARNING: non atomic and it can be reordered! */
> > > > > > -static inline int __test_and_change_bit(int nr, void
> > > > > > *addr)
> > > > > > +static inline int arch__test_and_change_bit(int nr,
> > > > > > volatile
> > > > > > void
> > > > > > *addr)
> > > > >=20
> > > > > ... and here, while ...
> > > > >=20
> > > > > > @@ -307,8 +295,7 @@ static inline int variable_test_bit(int
> > > > > > nr,
> > > > > > const volatile void *addr)
> > > > > > =C2=A0=C2=A0=C2=A0=C2=A0 return oldbit;
> > > > > > =C2=A0}
> > > > > > =C2=A0
> > > > > > -#define test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > -=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_siz=
e();=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > +#define arch_test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > >=20
> > > > > ... just arch_ here? I don't like the double underscore
> > > > > infixes
> > > > > very
> > > > > much, but I'm okay with them as long as they're applied
> > > > > consistently.
> > > >=20
> > > > Common code and x86 use __test_and_clear_bit(), and this patch
> > > > provides
> > > > a generic version of __test_and_clear_bit(). To emphasize that
> > > > generic__test_and_clear_bit() is a common implementation of
> > > > __test_and_clear_bit(), the double underscore was retained.
> > > > Also,
> > > > test_and_clear_bit() exists and if one day it will be needed to
> > > > provide
> > > > a generic version of it, then it will be needed to have
> > > > generic__test_and_clear_bit() and generic_test_and_clear_bit()
> > > >=20
> > > > A similar logic was chosen for test_bit.
> > >=20
> > > Right, but in all of your reply arch_ doesn't appear at all.
> > I am a little confused here. According to my logic, should it be
> > arch___test_and_set_bit() and generic___test_and_set_bit()?
>=20
> Why 3 underscores in a row? I'm clearly not following.
>=20
> > If you are asking why there is no generic implementation for
> > test_and_clear_bit() without the double underscores, the reason is
> > that
> > Arm, PPC, and x86 don't use generic code and rely on specific
> > instructions for this operation. Therefore, I didn't see much sense
> > in
> > providing a generic version of test_and_clear_bit(), at least, for
> > now.
>=20
> No, there was no question in that direction. And hence ...
>=20
> > > =C2=A0Yet the
> > > question was: Why then not arch__test_bit(), to match the other
> > > arch
> > > helpers?
> > Because no one uses __test_bit(). Everywhere is used test_bit().
>=20
> ... this seems unrelated (constrained by my earlier lack of following
> you).
>=20
> (Later) Wait, maybe I've finally figured it: You use
> arch__test_and_*()
> because those underlie __test_and_*(), but arch_test_bit() because
> there's
> solely test_bit() (same for the generic_* naming).
Yes, that what I meant.

>  I guess I can accept
> that then, despite the slight anomaly you point out, resulting in the
> question towards 3 underscores in a row. To clarify, my thinking was
> more
> towards there not possibly being generic forms of test_and_*() (i.e.
> no
> possible set of arch_test_and_*() or generic_test_and_*()), thus
> using
> double inner underscores in arch__test_*() and generic__test_*() to
> signify that those are purely internal functions, which aren't
> supposed to
> be called directly.
I understand your point regarding functions that start with "__".
For example, __test_and_clear_bit() is used not only internally (in
terms of architecture code) but also in common code, so it is not
strictly internal. I may have misunderstood what "internal function"
means in this context.

I thought that, at least for bit operations, "__bit_operation" means
that the bit operation is non-atomic and can be reordered, which
implies that it's not a good idea to use it in common code without
additional steps.

Anyway, I am not sure I understand which approach I should use in this
patch. You mentioned that possibly test_and_() can't have a generic
form, meaning it won't be a set of arch_test_and_() functions.

So, can I rename arch__test_() and generic__test_() to arch_test_() and
generic_test_(), respectively, and use the renamed functions in
_test_and*() in xen/bitops.h? Is my understanding correct?
If my understanding is correct, I am happy to apply mentioned changes
in the next patch version.

~ Oleksii


>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:37:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723106.1127637 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YUF-0007vR-MT; Thu, 16 May 2024 10:37:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723106.1127637; Thu, 16 May 2024 10:37:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YUF-0007vK-J6; Thu, 16 May 2024 10:37:23 +0000
Received: by outflank-mailman (input) for mailman id 723106;
 Thu, 16 May 2024 10:37:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YUE-0007vC-CY
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:37:22 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 467a7ac8-1370-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:37:21 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-5723edf0ae5so2885738a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:37:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7faasm965483366b.136.2024.05.16.03.37.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:37:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 467a7ac8-1370-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715855841; x=1716460641; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YwVAVtNNMa1Hc0vHyqcKhW7ESVYZx23wYJZgcsrtoXs=;
        b=DX+khVK6LxYnBV29EbwaL3OIU9xopPNqRqQTJHRDR+6v+co/dWApNEGrhF3XaxfGos
         vxhOf17WTAN2nw0rBvtOMJJ5GLW5dwN6h6TtlwvyQydwcy5j3eKTWU0TR4ER43MADmr/
         qez0ythIdLwxy+gnuO408SspGkVkCqAS9fmpgjA2d8a8Y+EBfXCGqN3DW4o5IpoBcMHI
         1Q9DpCMI5GwD/ca6FCyQhkWD5+siQXEMMGh6dZalG6+8+Qi8JwA+brTmFYAO5eXX4MVv
         tvYla0KltrO+KjloZN4cUUuSSIS1jCCrisnuUlVQswd76yGXHReYLywgsw1S3LdZXqBA
         50Mg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715855841; x=1716460641;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YwVAVtNNMa1Hc0vHyqcKhW7ESVYZx23wYJZgcsrtoXs=;
        b=DGTVAs05D8VlvZS7Yg6dRVNaGb/MR92SDacZtriTdYZZSjibsRkwb3hX8/sPO6OCaT
         VlruHdTBkyuH2dr38FApWjLtTuwkxQLhi8uHQUloZliVzWGcqb4pWyatyM4FwhQYx1S3
         RAsBhZKijOQMHg1tcrEMljbPPysJzHthLoHyB52bXpy03F/IiBGcWix38bNS8tDUZFft
         7MucelicX2+LIMKUlvayqisNYtuvzZUlng2ky0YZmTMFWXBVfXMvwDXfcCcWkhLMFxBA
         MFvMn+Qk2qZ/8eJdT9hOqmO/IMtLbJiKEGiojCbbUGwSDzn/u20P04D2uAzDUv5ze+b4
         wg+A==
X-Forwarded-Encrypted: i=1; AJvYcCWxdB+2cwutHtMQFDzgnPPc5NvhdozhWWMtbHT6WD6oz/SKQdB+KyoI7b3G24YT6LaeYTBpMqDYxF1xmkE84DaOw38CVoMELqVI1MOGbB4=
X-Gm-Message-State: AOJu0YyKoxphpeKp+yWZ/VodJA66jEvyJXvmp0zBfrCN+3QxZY4vT/Bx
	IxdlQOAo5ZFTD9GIVs/HDQSlfT+I82ePJ4fHCwjDyZvlguIp4mVvQ+wg+mB45Q==
X-Google-Smtp-Source: AGHT+IFhWd78SGpTXKw6a9azhaPHpXGw5tNC5QM4X+LEKzSMTtzmJvZPsW7r8AWV2tgWaFRRKuLGKA==
X-Received: by 2002:a17:907:2d86:b0:a5a:423:a69f with SMTP id a640c23a62f3a-a5a2d53b9bemr1454620366b.9.1715855839819;
        Thu, 16 May 2024 03:37:19 -0700 (PDT)
Message-ID: <7681c0b5-9f16-44e0-9ad0-48bb262e58e6@suse.com>
Date: Thu, 16 May 2024 12:37:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 04/15] x86/p2m: move altp2m-related code to
 separate file
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:06, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -500,7 +500,7 @@ int p2m_alloc_table(struct p2m_domain *p2m)
>      return 0;
>  }
>  
> -static int __must_check
> +int __must_check
>  p2m_remove_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn,
>                   unsigned int page_order)

With there now being a separate declaration, __must_check doesn't need
repeating here. Then
Acked-by: Jan Beulich <jbeulich@suse.com>

I'll see whether this applies okay ahead of the earlier patches; if it
does, I'll commit it with said adjustment.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:41:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:41:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723109.1127646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YXx-0001WW-82; Thu, 16 May 2024 10:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723109.1127646; Thu, 16 May 2024 10:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YXx-0001WP-5Z; Thu, 16 May 2024 10:41:13 +0000
Received: by outflank-mailman (input) for mailman id 723109;
 Thu, 16 May 2024 10:41:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YXv-0001UR-8s
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:41:11 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ce4cf024-1370-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:41:09 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a59b81d087aso263430866b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:41:09 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdce8f916sm126129866b.223.2024.05.16.03.41.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:41:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce4cf024-1370-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715856068; x=1716460868; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8rTbD/cvfzhzDm6OSlrL0nm/TpngG0yb23GR8C0xZtQ=;
        b=RP1SCy46d5ZcHZ4ZYg6pBcORcVlIf37WnfBi2UOv2B7Hs9TD9UKXDzVkg7/2xSRmLH
         9+bduNZOzKeO0+FUNNMNAM9dNqSb1n8ZM6oXvSUwtC3eGFqMDLDdkcdZBIAoY2I/ewwt
         mX+urcDUodWXYYf8ndNpMfZp4eZ4GgkYS5UoPT03I76B1fBy/V/86FcsM1xDtWvXXvBG
         T7+ouJ0N0i5VXTNP8OurhX3u391/3xyHqkcl++HhI6QW+vLbqxB3jWkNsiiZtEPK0+MI
         GenRwNCPM37y3C8SV4wjRLLU9E6Xmt+GYNi6fXgzk0BmrHWvRYWqGygoZSzV6oa+dGQg
         faaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715856068; x=1716460868;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8rTbD/cvfzhzDm6OSlrL0nm/TpngG0yb23GR8C0xZtQ=;
        b=kVuXor1ifeo8iZ3gkqhT5BWXr3saurwjdvyR39z5Ttv8bWzAmsfXq98h4jTzTab4hs
         QWNWv421yJ7gVM+1OcX2ryiBBJ/yOwjoBYK1BHTNbo48tsauBiSzzySuxX0kdB0+ih8F
         YMTq1nzYb+OISkV8wg+pZLWPFPH21dj8zejPXP3S2sfzOPxx9zXemvAfvMU1uv5aoaDJ
         0nC40BXJUWhyzo6LUY4ZENy71OJ7i6P44pkbE02se5yginiY3pGHBDG3qH1zqvtQ22yX
         +KqK7YpH0y0juwO7HScCED+gWDSg4AwZ4z/Rz++rKYRNnYM7rFYu1LmGTG5ygoGDRReo
         ENyw==
X-Forwarded-Encrypted: i=1; AJvYcCW6DpJYG0kzXhr5LQ///Rd6nlIAMQC9HTCyL3+cpjopU8oo7VVYXUUTs0BAW3N++kyJ2ZijLgqNCNXj3vA7nDpPiLi7KuFu8LzTmNFTLJc=
X-Gm-Message-State: AOJu0Yxb9QTVeGSBjYZCJwu4U+T/onXM1Xff2Z8JWbE4m8ip5wbypBuy
	RRRTuiAhhR+NGKOlG91K6ESA1SqoIhsGd1VCDR4CdoNwlUWrWx6myioAjN8Muavbpbq9xXp9U60
	=
X-Google-Smtp-Source: AGHT+IEUJy3Gp2/4Y1gFgIyjekpL4JYgPd9g4eMbL4svhFz0fdbkfyeTwvUtmH3eu4NGQ4XpzxdbYQ==
X-Received: by 2002:a17:906:7f95:b0:a59:c833:d274 with SMTP id a640c23a62f3a-a5a2d5cb2a0mr1858832766b.37.1715856068675;
        Thu, 16 May 2024 03:41:08 -0700 (PDT)
Message-ID: <cc4fb70e-0990-4f61-83a2-7063f3277dee@suse.com>
Date: Thu, 16 May 2024 12:41:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 05/15] x86: introduce CONFIG_ALTP2M Kconfig option
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:08, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -358,6 +358,11 @@ config REQUIRE_NX
>  	  was unavailable. However, if enabled, Xen will no longer boot on
>  	  any CPU which is lacking NX support.
>  
> +config ALTP2M
> +	bool "Alternate P2M support"
> +	def_bool y

Since you have "bool" already, here you mean "default".

> +	depends on VMX && EXPERT

This doesn't look right. Typical distro builds will be with EXPERT=n. Such
builds would suddenly end up without altp2m support then. What I think you
mean is a conditional prompt. I.e. overall

config ALTP2M
	bool "Alternate P2M support" if EXPERT
	default y
	depends on VMX
	help
	  ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:50:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:50:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723114.1127656 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YgP-0003eq-2O; Thu, 16 May 2024 10:49:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723114.1127656; Thu, 16 May 2024 10:49:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YgO-0003ej-Vg; Thu, 16 May 2024 10:49:56 +0000
Received: by outflank-mailman (input) for mailman id 723114;
 Thu, 16 May 2024 10:49:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YgN-0003eY-Oz
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:49:55 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06ea452b-1372-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:49:53 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-572af0b12b8so3109872a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:49:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcad0362sm7099189a12.20.2024.05.16.03.49.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:49:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06ea452b-1372-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715856593; x=1716461393; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bl/XB3ZRlZVeGhQxU9bpIlpMMOJOvBs3A0fid+cTgSs=;
        b=IdJJ+hk4Qi0sEgpXQHfmXfBmtsX/JdVnEISAXlxLzbUFo+G0CXyKBFeu0l06iz+M1w
         PnZ8j0l40z2QThKsTyEX3jf3nQQ+VLly9M1a4jT9jeLbuxwBEW5SfCsz9uyumHUkCPl5
         G6/MPxIJk02qaY9WJyJp34neyeqHnm835WdxVxNN749Rj/p/YgR0b68qLRx+mbyJspcb
         9w6ocPUogyYG8xrH5Cn/8gPGiEM2SgPryMNNiSKr1Vxm/u5dtAQMFdmHZ061P7fHUQZ+
         yICu5I2PMxC1SiTRTipngy/fapXAsE47h3ZE8xbp8JOVs4z7kd1EuQ40O5dTilTYmb4n
         kQgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715856593; x=1716461393;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bl/XB3ZRlZVeGhQxU9bpIlpMMOJOvBs3A0fid+cTgSs=;
        b=qzTxyQHP2cnZmCJo0JxgcHZ4xAGMo1zr0PpeeoBceNRY6EIBXnJ6Lo6h8bj12OF7If
         CO6G3wmMMY0f1zfCH+ouwJFjJrTotZdCTI22I2AgdF90PuKjGeeLznQANoMAHXZkpdH7
         il5XBiejHTPyyJJ1vV/xagk4vMa5WxvPXuNNKKfTkho+2U8Sg5rho2NEZ8ZeDgmX2JNp
         7Sj6Q9/HAV3DprkWNDpKxxD3YpfyZIZh8+tITTCBr46Rx6dIx0Prpp+zNHpJZEBFG5k4
         xziz2mmRqhN9GHKTSfCP8GAEX94dHHmILsQ3Hlz4JiQk/+0NA+B9qj0pVkh/+e5jDzR/
         oN5w==
X-Forwarded-Encrypted: i=1; AJvYcCXzvikP+yaWoItPnefNK11vJmBn/wHswpmDByG10eJ6ELHClyVXB+pgKf4B0a0lzB3eNYIoaC+NG99iLQZ0AzUtccK10vxR0L5U6mlxFFo=
X-Gm-Message-State: AOJu0YzKtqZS1apa9rpsH/N0iNwZ5U/Rt5OEbncy9PPeD3Tb5xbp9U9P
	ebqYpHFdpTLbIUsYPOkfMAH2+1pt6GoHmcV7damqHNvRYRWGNpLvLoPPtZZ6sw==
X-Google-Smtp-Source: AGHT+IG7H/Z+VkkPWcbXJlig5TIVBFBGq8GZYFhY4lMHxwGHaCqvQs4H4rHY3Hilk//oyGzqLlgpDA==
X-Received: by 2002:a50:8711:0:b0:572:7015:f303 with SMTP id 4fb4d7f45d1cf-5734d6f00d7mr15720119a12.35.1715856593132;
        Thu, 16 May 2024 03:49:53 -0700 (PDT)
Message-ID: <5a121770-9bf3-40c6-ae91-8dc70174dad5@suse.com>
Date: Thu, 16 May 2024 12:49:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
 <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
 <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
 <77c6e497-631d-4f92-bd38-8ab7dba4841d@suse.com>
 <c8bd47e8f8558d88b4d5a4a09ea10728006fd4d3.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c8bd47e8f8558d88b4d5a4a09ea10728006fd4d3.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 12:34, Oleksii K. wrote:
> On Thu, 2024-05-16 at 09:04 +0200, Jan Beulich wrote:
>> (Later) Wait, maybe I've finally figured it: You use
>> arch__test_and_*()
>> because those underlie __test_and_*(), but arch_test_bit() because
>> there's
>> solely test_bit() (same for the generic_* naming).
> Yes, that what I meant.
> 
>>  I guess I can accept
>> that then, despite the slight anomaly you point out, resulting in the
>> question towards 3 underscores in a row. To clarify, my thinking was
>> more
>> towards there not possibly being generic forms of test_and_*() (i.e.
>> no
>> possible set of arch_test_and_*() or generic_test_and_*()), thus
>> using
>> double inner underscores in arch__test_*() and generic__test_*() to
>> signify that those are purely internal functions, which aren't
>> supposed to
>> be called directly.
> I understand your point regarding functions that start with "__".
> For example, __test_and_clear_bit() is used not only internally (in
> terms of architecture code) but also in common code, so it is not
> strictly internal. I may have misunderstood what "internal function"
> means in this context.
> 
> I thought that, at least for bit operations, "__bit_operation" means
> that the bit operation is non-atomic and can be reordered, which
> implies that it's not a good idea to use it in common code without
> additional steps.

Correct, up to the comma; those may very well be used in common code,
provided non-atomic forms indeed suffice. But in my reply I didn't talk
about double-underscore-prefixes in names of involved functions. I
talked about inner double underscores.

> Anyway, I am not sure I understand which approach I should use in this
> patch. You mentioned that possibly test_and_() can't have a generic
> form, meaning it won't be a set of arch_test_and_() functions.
> 
> So, can I rename arch__test_() and generic__test_() to arch_test_() and
> generic_test_(), respectively, and use the renamed functions in
> _test_and*() in xen/bitops.h? Is my understanding correct?

You could. You could also stick to what you have now - as said, I can
accept that with the worked out explanation. Or you could switch to
using arch__test_bit() and generic__test_bit(), thus having the double
inner underscores identify "internal to the implementation" functions.
My preference would be in backwards order of what I have just enumerated
as possible options. I wonder whether really no-one else has any opinion
here ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723119.1127667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YmE-0006uW-Mc; Thu, 16 May 2024 10:55:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723119.1127667; Thu, 16 May 2024 10:55:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YmE-0006uP-Jd; Thu, 16 May 2024 10:55:58 +0000
Received: by outflank-mailman (input) for mailman id 723119;
 Thu, 16 May 2024 10:55:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YmD-0006uF-No
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:55:57 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df6a27eb-1372-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:55:56 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-5731ffcc905so3565441a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 03:55:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea6a36sm10340940a12.12.2024.05.16.03.55.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 03:55:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df6a27eb-1372-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715856956; x=1716461756; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MjkVqo8bkow5ECw80PpyS/0vdZeMISSEkzAn1+y5NbQ=;
        b=M8hCb510E6Cxpclyg5SPpdkTx5iR78QE30orklmlZGT7KQyW86rcT4ImT+XG68JuSd
         AG4wWk7o1byBP/a8PEzo+eIP+rkCsAWeenz5r+g4tnKEsdfisvdVMWCrHfEXM9yQksej
         SNmmovp+mUev9wh9fzYCChg2GBagZQXIcVr9hjDxIiXrMnBE1z9BywDS3tvN8Z7yg2jP
         TU/PuSNT0BMgQKb9fkyIozWPjEajrJ9IzyNtE73+Dc+RZywUVs5FM1PEsxWuLeDpbTje
         q0UAH/PVGmjsR5CIMAygdFjtIyBONBrhnguWGIrg7xnMFBmuZ3p4hZRjO64PDNB6vn1i
         W+3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715856956; x=1716461756;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MjkVqo8bkow5ECw80PpyS/0vdZeMISSEkzAn1+y5NbQ=;
        b=u1oV+nZtp54C8kSLr1H8Nif3i0G8/mwoB4XPF0ewD3A+5iCBJg7gWMA1Bzwsah47SK
         AyIaGjqyKdsaIsebiiBq+8oMN+RTTOtIqwjzdsAUxA3V7soRnHW4+OEZFkpC+89bJk1o
         Tll6YpwEY0KXZq9zTjiCVqOKHUR5k4AxfufHzvKK5ktTDHBlUwSJ/cVFVm6nAZgIGwGW
         QLziJnomWelQH7WLQB1bIwOqgdSo9XpJLaV37XelJgglu3oEcPOsuqakjFx/R4+2cywZ
         lSVm063cAT4mblDyoToHpn+RiXfhK+NDd+JTjjK2m1FiZph6Y9zK3FXj18aQziO2ZW+J
         4hVQ==
X-Gm-Message-State: AOJu0Yws67fDqdL91Dt6YJSsHWca+HRA4rmklt52pW/5Dd1tUk719obd
	HZSdTcHDfIW8FHOtEZUPWWWvNvCHo12ovQ7ACenYWhUFnNBcJm6xfA0o/cQLGQ==
X-Google-Smtp-Source: AGHT+IEefVfFJDysmrhcHJ8HB66T2COninvqwSkIkf2/UkJi4HAZkb44yPdMot7+iRlE8y32CkKhJg==
X-Received: by 2002:a05:6402:1a4d:b0:573:50d8:3fc0 with SMTP id 4fb4d7f45d1cf-57350d84107mr11904843a12.11.1715856956394;
        Thu, 16 May 2024 03:55:56 -0700 (PDT)
Message-ID: <04633485-24cc-4834-a64f-57193facd482@suse.com>
Date: Thu, 16 May 2024 12:55:55 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with
 CONFIG_ALTP2M option
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151737150.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151737150.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:38, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> --- a/xen/arch/x86/include/asm/p2m.h
>> +++ b/xen/arch/x86/include/asm/p2m.h
>> @@ -581,9 +581,9 @@ static inline gfn_t mfn_to_gfn(const struct domain *d, mfn_t mfn)
>>          return _gfn(mfn_x(mfn));
>>  }
>>  
>> -#ifdef CONFIG_HVM
>>  #define AP2MGET_prepopulate true
>>  #define AP2MGET_query false
>> +#ifdef CONFIG_ALTP2M
> 
> Is it necessary? Can't we just replace CONFIG_HVM with CONFIG_ALTP2M on
> the same line

No, because of where those constants are used.

Sergiy, you want to move the new #ifdef down by a line though, such that
the #define-s are separated from it by the blank line that presently (with
your change applied) follows the #ifdef line.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723123.1127677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ymv-0007bh-Um; Thu, 16 May 2024 10:56:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723123.1127677; Thu, 16 May 2024 10:56:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Ymv-0007ba-Rv; Thu, 16 May 2024 10:56:41 +0000
Received: by outflank-mailman (input) for mailman id 723123;
 Thu, 16 May 2024 10:56:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujgU=MT=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7Ymu-0007XN-3m
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:56:40 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f7bf26b8-1372-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:56:39 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2ADF580079;
 Thu, 16 May 2024 11:56:37 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 0717220189; Thu, 16 May 2024 11:56:37 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7bf26b8-1372-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715856997;
	bh=LkdaObEdft/zf4OSzmj6Efknxagk941/qBlw5l8npDY=;
	h=From:To:Cc:Subject:Date:From;
	b=DfPwOGuyitd8UeDM6qs0VoU3k1rZHEqjkWf2+vXMiuHfTv/SvV3TYz25OsJwS0+Jl
	 UVGDa4lB98HBximp0AYEvue5X6jrWRbTDKYYnyFu49iKpia9zcK8cIhqJGNtXnFDvc
	 LYyQpvmmeTGq382c0pmBosp6SaBeg8ECRMwl5n/0=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v3 0/4] Add bridge VLAN support
Date: Thu, 16 May 2024 11:56:08 +0100
Message-Id: <20240516105612.15306-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hello All,

I have addressed all the feedback from Jason and merged patch 4 and 2 as
requested by Andrew.

Summary of changes from RFC v2 to v3:

- xen-network-common.sh: use fatal() directly instead of $error variable
- xen-network-common.sh: bridge command not being available is now fatal
- Move examples to docs/misc/linux-bridge-vlan
- README: Make explanation of [BridgeVLAN] section clearer
- Use spaces consistently for indentation
- Merge previous patch 2 and 4 into a single patch 2


Blurb for RFC v2:

I realised over the weekend that there is a valid use case for providing
a VIF to a domain that has access to multiple VLANs, e.g. a router. Yes,
you can create a VIF per VLAN, but if you start having several VLANs (as
I do), it would be nicer to create a single interface that has access to
all the relevant VLANs (e.g. enX0.10, enX0.20, etc.).

So, version 2 changes the name and type of the parameter from an integer
called `vid' to a string called `vlan'. The vlan parameter is then
parsed by the vif-bridge script (actually, the functions called by it in
xen-network-common.sh).

As it quite a common practice to allocate VLANs in round numbers, I also
implemented the ability to specify contiguous or non-contiguous ranges.
You can specify whether a VLAN is tagged or untagged, and which VLAN is
the PVID (only one PVID is allowed).  For example,

vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20-29' ]

will setup the VIF so that 10 is the PVID and VLAN IDs 20 through 29
are permitted with tags. Another example:

vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=1p/10+10x9' ]

will setup the bridge to set 1 as the PVID and permit access with
tags for VLAN IDs 10, 20, 30, 40, 50, 60, 70, 80 and 90.

This patch set enables this capability as follows:

1. Adds `vlan' as a new member of the libxl_device_nic structure;
2. Adds support to read and write the vlan parameter from the xenstore;
3. Adds `vlan' as a new keyword for the vif configuration option;
4. Adds support to assign the bridge VLANs in the Linux hotplug scripts;
5. Updated xl-network-configuration(5) manpage and example configs.


Original blurb below:

For many years I have been configuring VLANs on my Linux Dom0 by
creating VLAN interfaces for each VLAN I wanted to connect a domain
to and then a corresponding bridge. So I would tend to have things
like:

enp0s0    -> br0     -> vif1, vif2
enp0s0.10 -> br0vl10 -> vif3, vif4
enp0s0.20 -> br0vl20 -> vif5
dummy0    -> br1     -> vif6

I recently discovered that iproute2 supports creating bridge VLANs that
allows you to assign a VLAN to each of the interfaces associated to a
bridge. This allows a greatly simplified configuration where a single
bridge can support all the domains, and the iproute2 bridge command can
assign each VIF to the required VLAN.  This looks like this:

# bridge vlan
port              vlan-id
enp0s0            1 PVID Egress Untagged
                  10
                  20
br0               1 PVID Egress Untagged
vif1.0            1 PVID Egress Untagged
vif2.0            1 PVID Egress Untagged
vif3.0            10 PVID Egress Untagged
vif4.0            10 PVID Egress Untagged
vif5.0            20 PVID Egress Untagged
vif6.0            30 PVID Egress Untagged

This patch set enables this capability as follows:

1. Adds `vid' as a new member of the libxl_device_nic structure;
2. Adds support to read and write vid from the xenstore;
3. Adds `vid' as a new keyword for the vif configuration option;
4. Adds support for assign the bridge VLAN in the Linux hotplug scripts.

I don't believe NetBSD or FreeBSD support this capability, but if they
do please point me in the direction of some documentation and/or examples.

NB: I'm not very familiar with Xen code base so may have missed
something important, although I have tested it and it is working well
for me.

Cheers,

Leigh.

-- 

Leigh Brown (4):
  tools/libs/light: Add vlan field to libxl_device_nic
  tools/xl: add vlan keyword to vif option
  Update add_to_bridge shell function to read the vlan parameter from
    xenstore and set the bridge VLAN configuration for the VID.
  tools/examples: Example Linux bridge VLAN config

 docs/man/xl-network-configuration.5.pod.in |  38 ++++++++
 docs/misc/linux-bridge-vlan/README         |  68 ++++++++++++++
 docs/misc/linux-bridge-vlan/br0.netdev     |   7 ++
 docs/misc/linux-bridge-vlan/br0.network    |   8 ++
 docs/misc/linux-bridge-vlan/enp0s0.network |  16 ++++
 tools/hotplug/Linux/xen-network-common.sh  | 103 +++++++++++++++++++++
 tools/libs/light/libxl_nic.c               |  10 ++
 tools/libs/light/libxl_types.idl           |   1 +
 tools/xl/xl_parse.c                        |   2 +
 9 files changed, 253 insertions(+)
 create mode 100644 docs/misc/linux-bridge-vlan/README
 create mode 100644 docs/misc/linux-bridge-vlan/br0.netdev
 create mode 100644 docs/misc/linux-bridge-vlan/br0.network
 create mode 100644 docs/misc/linux-bridge-vlan/enp0s0.network

-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723124.1127687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn4-0007vy-9F; Thu, 16 May 2024 10:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723124.1127687; Thu, 16 May 2024 10:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn4-0007vn-6c; Thu, 16 May 2024 10:56:50 +0000
Received: by outflank-mailman (input) for mailman id 723124;
 Thu, 16 May 2024 10:56:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujgU=MT=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7Yn2-0007M7-8e
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:56:48 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fcf3a761-1372-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 12:56:46 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2997080079;
 Thu, 16 May 2024 11:56:46 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 03ED920189; Thu, 16 May 2024 11:56:46 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcf3a761-1372-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715857006;
	bh=IqQbHL4zd6sroMW8oqRG7fEFGbUFht56fidjDQZye10=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=AhpBQQ3x4Lb7jWtkqMmnz4tg6Gmc8b+edVc4M0iWm8UQ11z8en4i2Mj7HQdjuYoD7
	 oGVrLAf1XEiTy5TMWzh7IEMqAJBsgjXrsVySQPmeHw9G9VFB6OT2WKF85/unQxZLuv
	 rUeolKYwZtQsKcQM3xneLSPrmP+dv1nrx3jANIrY=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v3 1/4] tools/libs/light: Add vlan field to libxl_device_nic
Date: Thu, 16 May 2024 11:56:09 +0100
Message-Id: <20240516105612.15306-2-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240516105612.15306-1-leigh@solinno.co.uk>
References: <20240516105612.15306-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add `vlan' string field to libxl_device_nic, to allow a VLAN
configuration to be specified for the VIF when adding it to the
bridge device.

Update libxl_nic.c to read and write the vlan field from the
xenstore.

This provides the capability for supported operating systems (e.g.
Linux) to perform VLAN filtering on bridge ports.  The Xen
hotplug scripts need to be updated to read this information from
the xenstore and perform the required configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

---
 tools/libs/light/libxl_nic.c     | 10 ++++++++++
 tools/libs/light/libxl_types.idl |  1 +
 2 files changed, 11 insertions(+)

diff --git a/tools/libs/light/libxl_nic.c b/tools/libs/light/libxl_nic.c
index d6bf06fc34..d861e3726d 100644
--- a/tools/libs/light/libxl_nic.c
+++ b/tools/libs/light/libxl_nic.c
@@ -233,6 +233,11 @@ static int libxl__set_xenstore_nic(libxl__gc *gc, uint32_t domid,
         flexarray_append(back, GCSPRINTF("%u", nic->mtu));
     }
     
+    if (nic->vlan) {
+        flexarray_append(back, "vlan");
+        flexarray_append(back, libxl__strdup(gc, nic->vlan));
+    }
+
     flexarray_append(back, "bridge");
     flexarray_append(back, libxl__strdup(gc, nic->bridge));
     flexarray_append(back, "handle");
@@ -313,6 +318,11 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const char *libxl_path,
         nic->mtu = LIBXL_DEVICE_NIC_MTU_DEFAULT;
     }
 
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                GCSPRINTF("%s/vlan", libxl_path),
+				(const char **)(&nic->vlan));
+    if (rc) goto out;
+
     rc = libxl__xs_read_checked(gc, XBT_NULL,
                                 GCSPRINTF("%s/mac", libxl_path), &tmp);
     if (rc) goto out;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 7d8bd5d216..5c510dc272 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -809,6 +809,7 @@ libxl_device_nic = Struct("device_nic", [
     ("backend_domname", string),
     ("devid", libxl_devid),
     ("mtu", integer),
+    ("vlan", string),
     ("model", string),
     ("mac", libxl_mac),
     ("ip", string),
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723125.1127693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn4-0007y0-KM; Thu, 16 May 2024 10:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723125.1127693; Thu, 16 May 2024 10:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn4-0007xX-DH; Thu, 16 May 2024 10:56:50 +0000
Received: by outflank-mailman (input) for mailman id 723125;
 Thu, 16 May 2024 10:56:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujgU=MT=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7Yn2-0007XN-ML
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:56:48 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fdaeafff-1372-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:56:48 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 5B26B8009E;
 Thu, 16 May 2024 11:56:47 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 32D1520189; Thu, 16 May 2024 11:56:47 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdaeafff-1372-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715857007;
	bh=0RL/5syMYmKDu7gc+zffZXfng7BleYOM9Jn0BEO2t7o=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=eryxgq4RS00dz0XWpf0ry83hU9EsoIJaowHO9dpyp+xEGYa6C9J2Ecl9oDNTtbwIo
	 P1U3pu3D62zCQ5M0OF1oMg5ooOavCYPITm9RDL5jsfvS36j3W6oYUQDRGajMGODqcf
	 8edB1TFYEXpuoPDcMs2HG7sjurrrUzaI7jfx/L1Y=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v3 2/4] tools/xl: add vlan keyword to vif option
Date: Thu, 16 May 2024 11:56:10 +0100
Message-Id: <20240516105612.15306-3-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240516105612.15306-1-leigh@solinno.co.uk>
References: <20240516105612.15306-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update parse_nic_config() to support a new `vlan' keyword. This
keyword specifies the VLAN configuration to assign to the VIF when
attaching it to the bridge port, on operating systems that support
the capability (e.g. Linux). The vlan keyword will allow one or
more VLANs to be configured on the VIF when adding it to the bridge
port. This will be done by the vif-bridge script and functions.

Document the new `vlan' keyword in xl-network-configuration(5).

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

---
 docs/man/xl-network-configuration.5.pod.in | 38 ++++++++++++++++++++++
 tools/xl/xl_parse.c                        |  2 ++
 2 files changed, 40 insertions(+)

diff --git a/docs/man/xl-network-configuration.5.pod.in b/docs/man/xl-network-configuration.5.pod.in
index f3e379bcf8..dfc35e72c6 100644
--- a/docs/man/xl-network-configuration.5.pod.in
+++ b/docs/man/xl-network-configuration.5.pod.in
@@ -259,6 +259,44 @@ Specifies the MTU (i.e. the maximum size of an IP payload, exclusing headers). T
 default value is 1500 but, if the VIF is attached to a bridge, it will be set to match
 unless overridden by this parameter.
 
+=head2 vlan
+
+Specifies the VLAN configuration. The format of this parameter is one or more
+VLAN IDs or ranges separated by forward slashes. Each term can be:
+
+=over
+
+=item *
+
+B<vlan> - a single VLAN ID in the range 1 to 4094. This can optionally followed
+by a B<p> to indicate the PVID or by a B<u> to indicate an untagged VLAN. C<p>
+implies B<u>.
+
+=item *
+
+B<vlan1>-B<vlan2> - a range of VLAN IDs from B<vlan1> to B<vlan2>, both between
+1 and 4094 and B<vlan1> being less than or equal to B<vlan2>. This can be
+optionally followed by a B<u> to indicate that the range of VLANs are untagged.
+
+=item *
+
+B<vlan>+B<offset>xB<count> - describing a range of VLAN IDs starting at B<vlan>
+with B<count> additional entries, each incremented by B<offset>. This can be 
+optionally followed by a B<u> to indicate that the range of VLANs are untagged.
+
+=back
+
+Note, one VLAN ID must be marked as the PVID. In the case of a vlan 
+specification consisting of a single VLAN ID (e.g. C<vlan=10>), the B<p> suffix
+may be omitted. Specifying more than one untagged VLAN ID is an advanced 
+configuration - use with caution.
+
+For example:
+
+        'vlan=10' -- meaning a single VLAN that is the PVID.
+        'vlan=10p/20' -- VLAN 10 is the PVID and VLAN 20 is tagged.
+        'vlan=10p/100+10x4' -- VLANs 10, 100, 110, 120, 130, 140, 150.
+
 =head2 trusted / untrusted
 
 An advisory setting for the frontend driver on whether the backend should be
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index ed983200c3..7546fe7e7a 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -565,6 +565,8 @@ int parse_nic_config(libxl_device_nic *nic, XLU_Config **config, char *token)
         nic->devid = parse_ulong(oparg);
     } else if (MATCH_OPTION("mtu", token, oparg)) {
         nic->mtu = parse_ulong(oparg);
+    } else if (MATCH_OPTION("vlan", token, oparg)) {
+        replace_string(&nic->vlan, oparg);
     } else if (!strcmp("trusted", token)) {
         libxl_defbool_set(&nic->trusted, true);
     } else if (!strcmp("untrusted", token)) {
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723126.1127697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn5-000863-0e; Thu, 16 May 2024 10:56:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723126.1127697; Thu, 16 May 2024 10:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn4-000853-P2; Thu, 16 May 2024 10:56:50 +0000
Received: by outflank-mailman (input) for mailman id 723126;
 Thu, 16 May 2024 10:56:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujgU=MT=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7Yn3-0007XN-MV
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:56:49 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe152f91-1372-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:56:48 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 0C42C8009F;
 Thu, 16 May 2024 11:56:48 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id D4C4B20189; Thu, 16 May 2024 11:56:47 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe152f91-1372-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715857008;
	bh=/LeYpkSLtC63z1CHHwG/PrlVgENABot2k3FS2Sid64E=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=RnuQNhykSjB2NiCV5Wh4HaLqxR5NpHTnhfMZ5lYVizmPF9dA46LeV6Bqj8SM6BXMP
	 Om5n+o72WDQwxxpkXaC8YqJvS3sF9qKkbXo/ky/qwe+xIMN3IOFmRVNppYJmDpNWDJ
	 3GBsd5Ts/U8OPirHDvBKAkstrBiGxCsgSz4SBFVw=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v3 3/4] tools/hotplug/Linux: Add bridge VLAN support
Date: Thu, 16 May 2024 11:56:11 +0100
Message-Id: <20240516105612.15306-4-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240516105612.15306-1-leigh@solinno.co.uk>
References: <20240516105612.15306-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update add_to_bridge shell function to read the vlan parameter from
xenstore and set the bridge VLAN configuration for the VID.

Add additional helper functions to parse the vlan specification,
which consists of one or more of the following:

a) single VLAN (e.g. 10).
b) contiguous range of VLANs (e.g. 10-15).
c) discontiguous range with base, increment and count
   (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).

A single VLAN can be suffixed with "p" to indicate the PVID, or
"u" to indicate untagged. A range of VLANs can be suffixed with
"u" to indicate untagged.  A complex example would be:

   vlan=1p/10-15/20-25u

This capability requires the iproute2 bridge command to be
installed.  An error will be generated if the vlan parameter is
set and the bridge command is not available.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

---
 tools/hotplug/Linux/xen-network-common.sh | 103 ++++++++++++++++++++++
 1 file changed, 103 insertions(+)

diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Linux/xen-network-common.sh
index 42fa704e8d..fa7615ce0f 100644
--- a/tools/hotplug/Linux/xen-network-common.sh
+++ b/tools/hotplug/Linux/xen-network-common.sh
@@ -121,10 +121,105 @@ create_bridge () {
     fi
 }
 
+_vif_vlan_add() {
+    # References vlans and pvid variables from the calling function
+    local -i vid=$1
+    local flag=${2:-}
+
+    if (( vid < 1 || vid > 4094 )) ;then
+        fatal "vlan id $vid not between 1 and 4094"
+    fi
+    if [[ -n "${vlans[$vid]}" ]] ;then
+        fatal "vlan id $vid specified more than once"
+    fi
+    case $flag in
+     p) if (( pvid != 0 )) ;then
+            fatal "more than one pvid specified ($vid and $pvid)"
+        fi
+        pvid=$vid
+        vlans[$vid]=p ;;
+     u) vlans[$vid]=u ;;
+     *) vlans[$vid]=t ;;
+    esac
+}
+
+_vif_vlan_parse_term() {
+    local vid incr last term=${1:-}
+
+    if [[ $term =~ ^([0-9]+)([pu])?$ ]] ;then
+        _vif_vlan_add ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}
+    elif [[ $term =~ ^([0-9]+)-([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        last=${BASH_REMATCH[2]}
+        if (( last >= vid )) ;then
+            for (( ; vid<=last; vid++ )) ;do
+                _vif_vlan_add $vid ${BASH_REMATCH[3]}
+            done
+        else
+            fatal "invalid vlan id range: $term"
+        fi
+    elif [[ $term =~ ^([0-9]+)\+([0-9]+)x([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        incr=${BASH_REMATCH[2]}
+        for (( j=${BASH_REMATCH[3]}; j>0; --j, vid+=incr ))
+        do
+            _vif_vlan_add $vid ${BASH_REMATCH[4]}
+        done
+    else
+        fatal "invalid vlan specification: $term"
+    fi
+}
+
+_vif_vlan_validate_pvid() {
+    # References vlans and pvid variables from the calling function
+    if (( pvid == 0 )) ;then
+        if (( ${#vlans[@]} == 1 )) ;then
+            vlans[${!vlans[*]}]=p
+        else
+            fatal "pvid required when using multiple vlan ids"
+        fi
+    fi
+}
+
+_vif_vlan_setup() {
+    # References vlans and dev variable from the calling function
+    local vid cmd
+
+    bridge vlan del dev "$dev" vid 1
+    for vid in ${!vlans[@]} ;do
+        cmd="bridge vlan add dev '$dev' vid $vid"
+        case ${vlans[$vid]} in
+             p) cmd="$cmd pvid untagged" ;;
+             u) cmd="$cmd untagged" ;;
+             t) ;;
+        esac
+        eval "$cmd"
+    done
+}
+
+_vif_vlan_membership() {
+    # The vlans, pvid and dev variables are used by sub-functions
+    local -A vlans=()
+    local -a terms=()
+    local -i i pvid=0
+    local dev=$1
+
+    # Split the vlan specification string into its terms
+    readarray -d / -t terms <<<$2
+    for (( i=0; i<${#terms[@]}; ++i )) ;do
+        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}
+    done
+
+    _vif_vlan_validate_pvid
+    _vif_vlan_setup
+    return 0
+}
+
 # Usage: add_to_bridge bridge dev
 add_to_bridge () {
     local bridge=$1
     local dev=$2
+    local vlan=$(xenstore_read_default "$XENBUS_PATH/vlan" "")
 
     # Don't add $dev to $bridge if it's already on the bridge.
     if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
@@ -134,6 +229,14 @@ add_to_bridge () {
         else
             ip link set ${dev} master ${bridge}
         fi
+        if [ -n "${vlan}" ] ;then
+            log debug "configuring vlans for ${dev} on ${bridge}"
+            if which bridge >&/dev/null; then
+                _vif_vlan_membership "${dev}" "${vlan}"
+            else
+                fatal "vlan configuration failed: bridge command not found"
+            fi
+        fi
     else
         log debug "$dev already on bridge $bridge"
     fi
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 10:56:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 10:56:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723127.1127717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn6-0000Cs-Ah; Thu, 16 May 2024 10:56:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723127.1127717; Thu, 16 May 2024 10:56:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Yn6-0000B5-5I; Thu, 16 May 2024 10:56:52 +0000
Received: by outflank-mailman (input) for mailman id 723127;
 Thu, 16 May 2024 10:56:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujgU=MT=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7Yn4-0007XN-Mn
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:56:50 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fea2b9dd-1372-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 12:56:49 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id F1AB280079;
 Thu, 16 May 2024 11:56:48 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id D064B20189; Thu, 16 May 2024 11:56:48 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fea2b9dd-1372-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715857008;
	bh=BO2dM/EaQO5+8TZ0KmpaERvUueDUoaoFR9BWzwbr6DA=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=EQYwpQy7HufR+rblgxoXxtD7nZEfkOlkGVQDgC3TeiuRPjUzB7z/62j3xlKmXofmF
	 iKGFI98OHiYvL6kqirsRH8k6pm7wwR0i9yCvk5PH9J4Ya0KizVGJwkV/L7I5MsKlil
	 b1IzO1/efCBSRYf9AwW5GWqlIWCT76AK4/cMjlyc=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v3 4/4] docs/misc: Example Linux bridge VLAN config
Date: Thu, 16 May 2024 11:56:12 +0100
Message-Id: <20240516105612.15306-5-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240516105612.15306-1-leigh@solinno.co.uk>
References: <20240516105612.15306-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new directory linux-bridge-vlan with example files showing
how to configure systemd-networkd to support a bridge VLAN
configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

---
 docs/misc/linux-bridge-vlan/README         | 68 ++++++++++++++++++++++
 docs/misc/linux-bridge-vlan/br0.netdev     |  7 +++
 docs/misc/linux-bridge-vlan/br0.network    |  8 +++
 docs/misc/linux-bridge-vlan/enp0s0.network | 16 +++++
 4 files changed, 99 insertions(+)
 create mode 100644 docs/misc/linux-bridge-vlan/README
 create mode 100644 docs/misc/linux-bridge-vlan/br0.netdev
 create mode 100644 docs/misc/linux-bridge-vlan/br0.network
 create mode 100644 docs/misc/linux-bridge-vlan/enp0s0.network

diff --git a/docs/misc/linux-bridge-vlan/README b/docs/misc/linux-bridge-vlan/README
new file mode 100644
index 0000000000..9a048bca39
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/README
@@ -0,0 +1,68 @@
+Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
+=====================================================================
+
+Introduction
+------------
+
+This directory contains example files to be placed in /etc/systemd/network
+to enable a single bridge with multiple VLAN support.
+
+The example is to support the scenario where the Xen host network interface
+is connected to an Ethernet switch configured as a trunk port. Each domain
+VIF can then be configured with one or more VLAN IDs, one of which will be
+the PVID.
+
+The example files create a bridge device called br0, with a physical interface 
+called enp0s0. You will need to update this with your system's device name.
+
+Key points of the configuration are:
+
+1. In br0.netdev, VLANFiltering=on is set. This is required to ensure the
+   VLAN tags are handled correctly.  If it is not set then the packets
+   from the VIF interfaces will not have the correct VLAN tags set.
+
+2. In br0.network, a system IPv4 address is configured that can be updated
+   according to your local network settings.
+
+3. In enp0s0.network, Bridge=br0 sets the bridge device to connect to. There
+   is also a [BridgeVLAN] section for each VLAN allowed on the external
+   interface. Note, if you want to create an internal VLAN private to the
+   host, do not include its VLAN ID in this file.
+
+
+Domain configuration
+--------------------
+
+Add the vlan= keyword to the vif definition in the domain. The simplest
+and most common example is a domain that wishes to connect to a single VLAN:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10' ]
+
+If you wish to configure a domain to route between two VLANs, you have two
+options. Option 1 is to create multiple interfaces on different VLANs:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10',
+        'max=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=20' ]
+
+Alternatively, you can create single interface:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20' ]
+
+In the domain, you would, for example, use enX0 for VLAN 10 and enX0.20 for 
+VLAN 20.
+
+
+Hints and tips
+--------------
+
+You can run the following commands on dom0 or a driver domain:
+
+1. To check if vlan_filtering is enabled:
+   # cat /sys/devices/virtual/net/<name>/bridge/vlan_filtering
+
+2. To check the bridge port VLAN assignments:
+   # bridge vlan
+
+3. To check the vlan setting in the xenstore (dom0 only):
+   # xenstore-ls -f | grep 'vlan ='
+
diff --git a/docs/misc/linux-bridge-vlan/br0.netdev b/docs/misc/linux-bridge-vlan/br0.netdev
new file mode 100644
index 0000000000..ae1fe487c3
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/br0.netdev
@@ -0,0 +1,7 @@
+[NetDev]
+Name=br0
+Kind=bridge
+MACAddress=xx:xx:xx:xx:xx:xx
+
+[Bridge]
+VLANFiltering=on
diff --git a/docs/misc/linux-bridge-vlan/br0.network b/docs/misc/linux-bridge-vlan/br0.network
new file mode 100644
index 0000000000..b56203b66a
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/br0.network
@@ -0,0 +1,8 @@
+[Match]
+Name=br0
+
+[Network]
+DNS=8.8.8.8
+#Domains=example.com
+Address=10.1.1.10/24
+Gateway=10.1.1.1
diff --git a/docs/misc/linux-bridge-vlan/enp0s0.network b/docs/misc/linux-bridge-vlan/enp0s0.network
new file mode 100644
index 0000000000..6ee3154dfc
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/enp0s0.network
@@ -0,0 +1,16 @@
+[Match]
+Name=enp0s0
+
+[Network]
+Bridge=br0
+
+# If Jumbo frames are required
+#[Link]
+#MTUBytes=9000
+
+[BridgeVLAN]
+VLAN=10
+
+[BridgeVLAN]
+VLAN=20
+
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:01:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:01:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723139.1127726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YrK-0003wh-Sj; Thu, 16 May 2024 11:01:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723139.1127726; Thu, 16 May 2024 11:01:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YrK-0003wa-Pd; Thu, 16 May 2024 11:01:14 +0000
Received: by outflank-mailman (input) for mailman id 723139;
 Thu, 16 May 2024 11:01:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7YrJ-0003w4-HG
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:01:13 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9ad18ca5-1373-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:01:11 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a59a0e4b773so297206866b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:01:11 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01968sm970196766b.166.2024.05.16.04.01.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:01:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ad18ca5-1373-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715857271; x=1716462071; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sqQ4w7l1Hd2xGalWtg1CBoGl5a/pFXWJTW4wk5umoQM=;
        b=ACi/1ROWl3j/idDpFzWP7fkQWWDAZNA6OrAgKZ2EUt2WyV9EIYXTBIFq4Isi2sgxNI
         ylWbfUGguZu5pUzeDk+9hpagD3wFNnsECf+SJLR/AsnKE6myqWO0wZTOBi0NHVX0uFRs
         HuPu5uaFHa6pRFO1ni1JHNi7+aiv80GaRiTDykGBib4l09XXHxWmpUhERx4Tl1nipgN7
         DYgCHym1WL9smJIlHpBgaVOM6h0RIVFfkeZnmKc8j1vaxBjRoq48OK+lZn1KfGaPMl6x
         8oM/PIGF+KlWbvPTlUliJZBhJw+pYQAlX+QAcWHtdnIQh5qCXfdrXN/1U1cnFwoNXPoS
         XzFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715857271; x=1716462071;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sqQ4w7l1Hd2xGalWtg1CBoGl5a/pFXWJTW4wk5umoQM=;
        b=wbZu/pMmNQb4TP3RuB7UedXQvph3F1hLitvm8o9M64G4QmgUMh3hDJJNq7w+vV8Ght
         9sUCpDL9INm0rl12ssJptBYhxcIj8BKnnE8cKvwRB/2ZIv3YWGdRkZY6jj8i5645bMpB
         vUQEYehE+MTNSmpyU4NgG8Te0dBLxOghMmJNdLNkX282qTGvBli6a3gdZ9x5EepWTgsT
         wjG7rY3kjnVFRCAKS6Frdjo5QiYJa032uG/fo8E9qF4NEzrNg+Z49w14cwRrR2OpMzJS
         iV5bQvUUBWCGaZ2wW9LsntWhtR+rGgVKg5H2VXly2wYkvlJ6cWXK3WU4gmV9wmRuy7Nn
         QNfw==
X-Forwarded-Encrypted: i=1; AJvYcCUQYuac85rFiPASaXoaYsQz7r9GobjAeXN/mDYRpcYcxSiqcIyH3/1aNUBNGGqnb8Ad/+sLQoJqz1BqsEMKDgH8yxP+Onz39Muyuu7HnpM=
X-Gm-Message-State: AOJu0Ywvu993+kxpaR7zR3jOB4lMC8LKUdtS+gsHtnDaQF4q8kUhBmA4
	ZyjaYdaA+GuDgiF2tUJOYNPGRM1x2EOOctgRvDZ/a1lqlHeNNi4GkoATjUhYrA==
X-Google-Smtp-Source: AGHT+IHikjxLEl1yD+WW6Jy+WgDfU+T5kKhfhBJo0i/OpwnlOxbz6cg4fbBqhB6mHiL4M+jCs0lcnA==
X-Received: by 2002:a17:906:194a:b0:a59:9b75:b90 with SMTP id a640c23a62f3a-a5a2d53ad53mr1259918466b.2.1715857270737;
        Thu, 16 May 2024 04:01:10 -0700 (PDT)
Message-ID: <29604726-8768-4b33-a8ef-eba6ad112d44@suse.com>
Date: Thu, 16 May 2024 13:01:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with
 CONFIG_ALTP2M option
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:10, Sergiy Kibrik wrote:
> @@ -38,7 +38,10 @@ static inline bool altp2m_active(const struct domain *d)
>  }
>  
>  /* Only declaration is needed. DCE will optimise it out when linking. */
> +void altp2m_vcpu_initialise(struct vcpu *v);
> +void altp2m_vcpu_destroy(struct vcpu *v);
>  uint16_t altp2m_vcpu_idx(const struct vcpu *v);
> +int altp2m_vcpu_enable_ve(struct vcpu *v, gfn_t gfn);
>  void altp2m_vcpu_disable_ve(struct vcpu *v);

These additions look unrelated, as long as the description says nothing in
this regard.

> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -670,7 +670,7 @@ static inline bool hvm_hap_supported(void)
>  /* returns true if hardware supports alternate p2m's */
>  static inline bool hvm_altp2m_supported(void)
>  {
> -    return hvm_funcs.caps.altp2m;
> +    return IS_ENABLED(CONFIG_ALTP2M) && hvm_funcs.caps.altp2m;

Which in turn raises the question whether the altp2m struct field shouldn't
become conditional upon CONFIG_ALTP2M too (or rather: instead, as the change
here then would need to be done differently). Yet maybe that would entail
further changes elsewhere, so may well better be left for later.

> --- a/xen/arch/x86/mm/Makefile
> +++ b/xen/arch/x86/mm/Makefile
> @@ -1,7 +1,7 @@
>  obj-y += shadow/
>  obj-$(CONFIG_HVM) += hap/
>  
> -obj-$(CONFIG_HVM) += altp2m.o
> +obj-$(CONFIG_ALTP2M) += altp2m.o

This change I think wants to move to patch 5.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:07:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723150.1127736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YxN-0005kk-KE; Thu, 16 May 2024 11:07:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723150.1127736; Thu, 16 May 2024 11:07:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7YxN-0005kd-Hi; Thu, 16 May 2024 11:07:29 +0000
Received: by outflank-mailman (input) for mailman id 723150;
 Thu, 16 May 2024 11:07:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7YxM-0005kX-RT
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:07:28 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b51ca98-1374-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:07:28 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-571ba432477so3504245a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:07:28 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdfed0e27sm106366766b.213.2024.05.16.04.07.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:07:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b51ca98-1374-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715857647; x=1716462447; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=d7fw3Uxcr4l9sdB/f+rweJ+gcxrQnNehLShuUNgR3KY=;
        b=hYKUAVZGhcTfSlHSqfWpT0RVbvPxucMxBwe/oOqbyj+IzFwnANQtN8wnBePU+PL5t0
         hDbq9lWB4/OTCi6mYMVlS1q3huqTWtcPiJKmPIbzbQvGlibw32DdoOVRu+nmOf7HpjcT
         TIne7ugE9HPPmkqOD51/VbDnrDyHwUpfe5EKI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715857647; x=1716462447;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=d7fw3Uxcr4l9sdB/f+rweJ+gcxrQnNehLShuUNgR3KY=;
        b=vPFz4VDckjPpba2vDn6zMXM1uqMl/QEMJBx0JBO/L14kpv1og+UKzhOtv+XTDJSEE7
         isllG4x5KgBEbgRiiNII8RuFMMLY31fwgWhMM6WRb/AzWo8MXsTNwqaexIFfA/0ksQA0
         +/sdMNycyOSwEeH1EvFpWDjfr1pLuQIdqqgHXhjtfrslulRqf2wJPFrQ1VLJGxKcx1ez
         dR1+RMSVsmjqxXxGe3waTd2XBqrZmFExHUSnE1PFAwwkVTDyMsVfsIan58XUZzA9bcPH
         VOZo4m1DjtEGdVPIqCjvj46VcEAbBpeJWmeCAqmrWkLiVMy7YS1clsR90CC6hgMa8rp6
         63jA==
X-Gm-Message-State: AOJu0YyXuvvodZ7DtOQSyUTtyTFtf2w51hziUQo5mu5ZsBp71iF/P74z
	FDUlxwFFa02cUyeSGaVoLAcbCZatYv8PrWRkQXx5PTwvkO3fnuhjEDGpB0H5Su4LsQj4WzGnreW
	w
X-Google-Smtp-Source: AGHT+IGGpyRYjM6N3YDPwFtNQWX6fJQLi0nnLcNNrEAq2yFSprh1p9QmRUgJo7VZgaeP3HwFjemtcA==
X-Received: by 2002:a17:907:36f:b0:a59:ba18:2fb9 with SMTP id a640c23a62f3a-a5a2d534e9cmr1120447566b.12.1715857645954;
        Thu, 16 May 2024 04:07:25 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH] tools: Add install/uninstall targets to tests/x86_emulator
Date: Thu, 16 May 2024 12:07:10 +0100
Message-Id: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Bring test_x86_emulator in line with other tests by adding
install/uninstall rules.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
 tools/tests/x86_emulator/Makefile | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
index 834b2112e7fe..30edf7e0185d 100644
--- a/tools/tests/x86_emulator/Makefile
+++ b/tools/tests/x86_emulator/Makefile
@@ -269,8 +269,15 @@ clean:
 .PHONY: distclean
 distclean: clean
 
-.PHONY: install uninstall
-install uninstall:
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
+	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
+
+.PHONY: uninstall
+uninstall:
+	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
+
 
 .PHONY: run32 clean32
 ifeq ($(XEN_COMPILE_ARCH),x86_64)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:13:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:13:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723155.1127746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Z2c-0000al-5d; Thu, 16 May 2024 11:12:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723155.1127746; Thu, 16 May 2024 11:12:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Z2c-0000ae-3B; Thu, 16 May 2024 11:12:54 +0000
Received: by outflank-mailman (input) for mailman id 723155;
 Thu, 16 May 2024 11:12:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Z2b-0000aX-9t
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:12:53 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3bf10cbd-1375-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:12:51 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51f12ccff5eso792411e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:12:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c322c06sm10568492a12.85.2024.05.16.04.12.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:12:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bf10cbd-1375-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715857970; x=1716462770; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Adhs2keN3Ebn4OSV/eDbqb4Oz5KNW04jrIJ6ap6R5wU=;
        b=EwgrkQ1mdKZSARcvrYg5GHoCNyMbWYQkgnDZvXsX0071J8IyyhOeKgYSb1TFo4Ig+9
         f7kLOa1kCM1J9FzVArovCHpd/DKJUePCUBAe/aqzpre5t0YpDDjqIBAivKMFLLN9eq/m
         KHR5sORfw/SYPOgfJuCmmje71V01SGvxWHsEnXYTEFP33+iuL6l8Lv9MUhRd91sD1Lpc
         sQIoRJiHTfmiSBDjb6nAyQXYg0N9cIy1axf6gymocMXSJsqjWQK5YIWrfNH1+KciKDoZ
         962mVnB702Emk6KwWGt05mwW04Xy8bmr3FjTcKONLxYPSZ/Q8LZCtM91gBazxKTRUxrb
         axUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715857970; x=1716462770;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Adhs2keN3Ebn4OSV/eDbqb4Oz5KNW04jrIJ6ap6R5wU=;
        b=f2+2ElCukQCaIpaXCtyDpNlSDSHH0PKDGS4QVx+qBfncb1g7pTUaCJ7mloJZcgLfv2
         vh64982zz8Lp/5XBv/FNVpXMd4fQ7yzmASMSwcZgfqapjfPxKtx81zlVOHkxW9KnY7W3
         DC5rnwlJzM+l0SVlAKTjN8if60l0jWla8kWgtGPTdF/QygUtOf6RkTOvMdzyL5tX+MzF
         tFIlWkJANcwIhoyZSY4Eml+/i8J7Qpe+GnC/+C+4bei5Af6QZCFtAVV5Ny+CRhsIPz3m
         hf6ZJW9nkPxW0V9lPCNPPiZCjMxHWb8oI/03yEZZUjWHwcXfV+BjfiQ2+5Ff4hb1gDr7
         ivog==
X-Forwarded-Encrypted: i=1; AJvYcCXXQmEROq7p1d4e5QeXUXHtu1ufH4gWljgSUzWXoD3vs1D7GLM5nkFHPDcnGXu/vnUpeybffgMieOcdyGqKb/zKlDDmIKFrI/gHmeQthm8=
X-Gm-Message-State: AOJu0YxIvnKViNTv/86oo45LdHff5HwhJpL2t2DAXUlJMDv4dx5d29E5
	DW3uo/UOOnUnYo6kYhRzp00x/FCYrMJmIrVlcv6F5LnnIEUE9uZRbvoPt5F7RQ==
X-Google-Smtp-Source: AGHT+IERJ0/GDFYbkKIqdo6yMFqTa9a8SpruvrgTiqhp1jlEDjG5PxM2TB9YWhyDfLjDJa8yqsTZGA==
X-Received: by 2002:a19:e01e:0:b0:51e:ef3f:e74d with SMTP id 2adb3069b0e04-522105847a4mr12845452e87.62.1715857970479;
        Thu, 16 May 2024 04:12:50 -0700 (PDT)
Message-ID: <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
Date: Thu, 16 May 2024 13:12:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:12, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/include/asm/cpufeature.h
> +++ b/xen/arch/x86/include/asm/cpufeature.h
> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>  #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>  #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>  #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
> +                                  boot_cpu_has(X86_FEATURE_VMX))
>  #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>  #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>  #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>  
>  /* CPUID level 0x80000001.ecx */
>  #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
> +                                  boot_cpu_has(X86_FEATURE_SVM))
>  #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>  #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>  #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)

Hmm, leaving aside the style issue (stray blanks after opening parentheses,
and as a result one-off indentation on the wrapped lines) I'm not really
certain we can do this. The description goes into detail why we would want
this, but it doesn't cover at all why it is safe for all present (and
ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
just to derive further knowledge from that, without them being directly
related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
for example. While it looks to be okay there, it may give you an idea of
what I mean.

Things might become better separated if instead for such checks we used
host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
that's still pretty far out, I'm afraid.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:16:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:16:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723164.1127757 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Z5l-0001qU-LP; Thu, 16 May 2024 11:16:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723164.1127757; Thu, 16 May 2024 11:16:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7Z5l-0001qN-Gk; Thu, 16 May 2024 11:16:09 +0000
Received: by outflank-mailman (input) for mailman id 723164;
 Thu, 16 May 2024 11:16:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7Z5k-0001qF-4P
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:16:08 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id afe0ff66-1375-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:16:05 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-571c2055cb1so920788a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:16:05 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574ea5d755bsm3343542a12.51.2024.05.16.04.16.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:16:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afe0ff66-1375-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715858165; x=1716462965; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=oVGuElW2+RdLtsUoiSqvoN2DvuwbqRicQPDBXdv6SCI=;
        b=HRh0P4O/DH9g2GmGbZEhdQET9GHbbGQt+iYnlniZnSQuN+W0yCvec545RMys1bOOjY
         xztwwj3Ild9D9HQzDuK89rHZQytfOiigCOmrmyOUaWrVS1sO1uw+p+4xPK9rGqQJCpyD
         6xzA1GvgnnMBj056Z05ihijTSXxq+o+6K1TH+dLzVK9fwTWtaUtXxv2exRK7WPFF8wlV
         DVuP2Eyhrx/3Qb+9E2LuUhJFGkvsc/B+hsCv8neLNWroDC58AHAAv+LgyETtvgaJvN3u
         eOVx63vRmHxMGh2gQ2A7GCilm64KFuWS7G5Bv2bgkMNk2aiDrPx4UnmANSxvfk8wCIa0
         d3kg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858165; x=1716462965;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=oVGuElW2+RdLtsUoiSqvoN2DvuwbqRicQPDBXdv6SCI=;
        b=NcELTBbYcMlLDAm8wHhghDuLjO4P5OyabgbeKLmoYTfLdK+EFBWX2QhNSyO4vAaW8y
         l4XNlcFKLnaXjn9Lw1s2UDS+KAJzeEAQpLYus0L47PJkqONmlwNfQDAaBpK5dwS9YOAm
         t+okjBfVjzJtvJDbdZv5I2BK2w6WhzBnOkebHZi3l5ibSyHNXhQFpSv5Y9SP0ECC9tTy
         fmIcRAM4IIccfuShjDn6fvLd/28NavREkFr8mB+XoZttLzoTT3YaO0jr0abP4A84pKDu
         tPjxcCVcyodBdCLjJ5zASlgsHananCkpgDyCTmm0LtBWbLyNrSOxonW77WNmi/jXkY6e
         hwAg==
X-Forwarded-Encrypted: i=1; AJvYcCUhGYW15FtCA5868Xt34CtfRz/6NJdOMHUFXw0nmJ1M2nIEgGfJfjfaRWTer5unRzo3n9Si+jcPkauk0LlcZdmyzNhp1esmJoVyg41XulM=
X-Gm-Message-State: AOJu0YyCB1mvnmdlaR/AjY13LqhSma/WfQNnko4876JqExwEKMVgqY75
	RV+8qKwsaZQGUMq7xK2Kk26XtvuKCqwAUPWIgcIZ0Uo9XrQwkI3h3onZANH89+cBSngjvzVMHTI
	=
X-Google-Smtp-Source: AGHT+IHI36nNeKI9stQGcSG/sERMn7XS+KrkIa5iDGhC+hZzw9m42rfl6qDe2/sId3Kp4Jin7N2K/g==
X-Received: by 2002:a50:9f06:0:b0:572:67d9:6c3b with SMTP id 4fb4d7f45d1cf-5734d67f864mr16295577a12.26.1715858165048;
        Thu, 16 May 2024 04:16:05 -0700 (PDT)
Message-ID: <cf8cfb82-7a69-4690-8afb-d971dbe8b7e3@suse.com>
Date: Thu, 16 May 2024 13:16:03 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 13:07, Alejandro Vallejo wrote:
> Bring test_x86_emulator in line with other tests by adding
> install/uninstall rules.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

I'd expect such a change to come with a word towards what use the binary has
on the installed system. Imo we should not randomly put binaries in place
when there's - afaict - absolutely no use for them outside of development.

Jan

> --- a/tools/tests/x86_emulator/Makefile
> +++ b/tools/tests/x86_emulator/Makefile
> @@ -269,8 +269,15 @@ clean:
>  .PHONY: distclean
>  distclean: clean
>  
> -.PHONY: install uninstall
> -install uninstall:
> +.PHONY: install
> +install: all
> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
> +
> +.PHONY: uninstall
> +uninstall:
> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
> +
>  
>  .PHONY: run32 clean32
>  ifeq ($(XEN_COMPILE_ARCH),x86_64)



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:23:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:23:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723169.1127767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZCf-0005R1-AQ; Thu, 16 May 2024 11:23:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723169.1127767; Thu, 16 May 2024 11:23:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZCf-0005Qu-7w; Thu, 16 May 2024 11:23:17 +0000
Received: by outflank-mailman (input) for mailman id 723169;
 Thu, 16 May 2024 11:23:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ZCe-0005Qk-8s
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:23:16 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id afe38048-1376-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:23:15 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so254659266b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:23:15 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17886sm973560666b.210.2024.05.16.04.23.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:23:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afe38048-1376-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715858594; x=1716463394; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2guZqs3YJwFfLDpBOb46zWp8VJ6xG7fIxOb5wDm9pOE=;
        b=GPh3hLL7dZaZL8lGAJspj3k84s1BgKQQxMJ0gjCRXXu9BmHG+AKvwigGYf6eEKlsYR
         QW/lX5BwTCDRK4S8LI1nua27fSdmMYq9CI49b936EpLqfTNPINSw/ZOjYFahT8j+HaWh
         +OGoc9fgkX4P/6h/4FEEvqP5PZXlGFLH+hwSxkqEYhDFtWY6EIBpM8kVwqB40zTzLYFZ
         LlrrVY1r1F7XVVN/g69GedYx1c0RDosoFAnfS2kBX8Nmc73xcm9TOoskTZM2Q92EntIN
         PBgKbb4SzLB2rkBqK4qzWcGt0bDxzG3KdWcgYV4ofp4G9uaoGptSRrO846RM3XNOuzHA
         tOwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858594; x=1716463394;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2guZqs3YJwFfLDpBOb46zWp8VJ6xG7fIxOb5wDm9pOE=;
        b=BqUCEyzqXUTCPdYoQixLHo3APeJuMcARTc9IcOlaWcMmLhHSs7AyLyNyhP19A8an/+
         dBpFIRD8/DRtl7Mi54Imwyzi+gJUoUh3yJcHrbTnKFDhOAqRn+6Q1X+wkgfLC9mv9TFr
         aTyRrpUBoBWYl6N7Q0olEQhQYDIUFAXmsExGA87yZGOcHZ85e5TCX0OK6NUOTO4hXP1p
         ygx34aqftAcXOw8IZcHoxDTrr5xfniF4+IXU92rTjqP7E3fFNKttV96rtKkw0Tl1GkC/
         x8COTadNVd6n15VutF7s9/bVnhNyU21yRimQgx2KLEFBXEb651djwdQqO2TM8PY9qBEr
         Kk2g==
X-Forwarded-Encrypted: i=1; AJvYcCUsVyTjSDnm9ofYBc1qQkaxaY5zmQMIw+jcEoXP1CE22TJ+rRsD5fRpKwV+QoRFPDU2CW0dz9P6xCE+muHmXJ/RMo22Per7lRbOrBLD45w=
X-Gm-Message-State: AOJu0Yzm6Oh5Okv5pNuvdkqFO6aoYSwf5fephwGTDVn5KtJsIqbvx2z4
	1Q47qeijutU9wgcvgIsfebuV/aiSQK3b3eCDjU3cqpphXkkMVz7CXUOyHnqPUA==
X-Google-Smtp-Source: AGHT+IFDLMqw6BUXhf+dDp3AyOriERa4mpMvo8hb4dk4YgZMzzvkwZd/x1H7HxiFRaghmTIZ4ILFkg==
X-Received: by 2002:a17:906:fe4a:b0:a59:a8a4:a59d with SMTP id a640c23a62f3a-a5a2d66aa6dmr1442062566b.62.1715858594611;
        Thu, 16 May 2024 04:23:14 -0700 (PDT)
Message-ID: <7d780eff-a64f-40dd-a377-2af05bbd2eee@suse.com>
Date: Thu, 16 May 2024 13:23:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 08/15] x86/vpmu: guard vmx/svm calls with
 cpu_has_{vmx,svm}
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <fbd17194026a7e61bac2198e3b468d498f45d064.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:14, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/cpu/vpmu_amd.c
> +++ b/xen/arch/x86/cpu/vpmu_amd.c
> @@ -290,7 +290,7 @@ static int cf_check amd_vpmu_save(struct vcpu *v,  bool to_guest)
>      context_save(v);
>  
>      if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
> -         is_msr_bitmap_on(vpmu) )
> +         is_msr_bitmap_on(vpmu) && cpu_has_svm )
>          amd_vpmu_unset_msr_bitmap(v);

Assuming the change in the earlier patch can actually be established to be
safe, along the lines of an earlier comment from Stefano the addition may
want to move earlier in the overall conditionals (here and below). In fact
I wonder whether it wouldn't be neater to have

#define is_svm_vcpu(v) (cpu_has_svm && is_hvm_vcpu(v))

at the top of the file, and then use that throughout to replace is_hvm_vcpu().
Same on the Intel side then, obviously.

> @@ -288,7 +288,7 @@ static int cf_check core2_vpmu_save(struct vcpu *v, bool to_guest)
>  
>      /* Unset PMU MSR bitmap to trap lazy load. */
>      if ( !vpmu_is_set(vpmu, VPMU_RUNNING) && is_hvm_vcpu(v) &&
> -         cpu_has_vmx_msr_bitmap )
> +         cpu_has_vmx && cpu_has_vmx_msr_bitmap )

Aren't you elsewhere adding IS_ENABLED() to cpu_has_vmx_*, rendering this (and
similar changes further down) redundant?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723178.1127799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-00071r-TH; Thu, 16 May 2024 11:28:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723178.1127799; Thu, 16 May 2024 11:28:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-0006z4-KF; Thu, 16 May 2024 11:28:26 +0000
Received: by outflank-mailman (input) for mailman id 723178;
 Thu, 16 May 2024 11:28:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHd-0006b7-QU
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:25 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68341d63-1377-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:28:24 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-5731ffcc905so3612976a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:24 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68341d63-1377-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858903; x=1716463703; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BlMJnuVPgAuWx5mzHbOSNLc3FPf2ltoLWAdUx3vSNgk=;
        b=gzVMOmwwqbE8u6F1Alm2kSXpRqxOTycAxF0lf0ewQX3WoV8k0MdCdBpmzf4axl8csN
         dn4hTFR97WvSpiqI1Hjs++H/y+e+tCchefjZsqSMKR4MZK+DaWlRjvP36e9h9u0SnrEg
         eFrWiADKTh++V7CeFaFap4Vglk/LBLMceyqiLUUmM1ozFGogeKqy8BjjmmL3li16kfbL
         RsDaC31zL8Swpw+c850H+TaYJ/9wnGUDCKpbPtDzH+JVc/3ODYSraicWJFLNFdf/ZcgZ
         GiO6pwDigqbuNnLZQqkJ9UNrbgl86Jv6FC0XeyAdTqPO1WLBnuBfvsw4J61VBzUHS+yU
         hJ0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858903; x=1716463703;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BlMJnuVPgAuWx5mzHbOSNLc3FPf2ltoLWAdUx3vSNgk=;
        b=GWGEllRKoEOdN/FQ1LNzwEc5VRplE+fr1EYXCCfPnU31ybS4njQw1quJAjrccQ/MnY
         KlH5TF5fkBIIcONLHnOHiWh2eA65YeeqaOZ6Vv9iX8d1eon2QkJSvFQ91thZNO8UJdfN
         ltnV1kdOjo9xq16rwvhSKH1MQUzr7uiWO1ZzASEZzC7Q6vjTzO5/eRwiIrYQt81q/kNb
         s+p1AZvRBK4KtV/+S2GvD6mRgUK7sn2ozcgiyTyAT0/IReLvUdcp8VsBKiUdE/5dr7SN
         dWLmVoWe52pdcjLOJjhu1n60iUHh6AHmwHHP2wo2bK4B3MlqgqvQ6txZxy+unMHcugPr
         NMgg==
X-Gm-Message-State: AOJu0YxMDP5GFSx3QiGwKdX5MZVOBkd9u3WP0/o542qe0CY5DmKuJe36
	7C0k3vHnsAlkU+QoFFVNEWXERjxsLyxFSOIMD967FsycK/DkK704yz0LGw==
X-Google-Smtp-Source: AGHT+IF0k5pWTfdbbFTMbNHJJ4YGIIz1Hmc94hXY1We+20u83JMGUYKVzWzEzNTo2L/vf/NhpK+Sgg==
X-Received: by 2002:a50:ccdd:0:b0:574:eb22:3629 with SMTP id 4fb4d7f45d1cf-574eb223677mr4705030a12.19.1715858903583;
        Thu, 16 May 2024 04:28:23 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH for-4.19? v3 3/6] docs/man: Add altp2m_count parameter to the xl.cfg manual
Date: Thu, 16 May 2024 11:28:10 +0000
Message-Id: <12aced8fdd0f2f8e0d1298895393edb8f741d641.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Update manual pages to include detailed information about the altp2m_count
configuration parameter.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 docs/man/xl.cfg.5.pod.in | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..5c09610cf4 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2039,6 +2039,20 @@ a single guest HVM domain. B<This option is deprecated, use the option
 B<Note>: While the option "altp2mhvm" is deprecated, legacy applications for
 x86 systems will continue to work using it.

+=item B<altp2m_count=NUMBER>
+
+Specifies the maximum number of alternate-p2m views available to the guest.
+This setting is crucial in domain introspection scenarios that require
+multiple physical-to-machine (p2m) memory mappings to be established
+simultaneously.
+
+Enabling multiple p2m views may increase memory usage. It is advisable to
+review and adjust the B<shadow_memory> setting as necessary to accommodate
+the additional memory requirements.
+
+B<Note>: This option is ignored if B<altp2m> is disabled. The default value
+is 10.
+
 =item B<nestedhvm=BOOLEAN>

 Enable or disables guest access to hardware virtualisation features,
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723175.1127776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHc-0006bQ-S4; Thu, 16 May 2024 11:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723175.1127776; Thu, 16 May 2024 11:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHc-0006bJ-PZ; Thu, 16 May 2024 11:28:24 +0000
Received: by outflank-mailman (input) for mailman id 723175;
 Thu, 16 May 2024 11:28:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHb-0006b6-I7
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:23 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67415070-1377-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:28:22 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so5922160a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:22 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67415070-1377-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858902; x=1716463702; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9wb2Zg0RGsFll9U2ozXeIHtw+XG8+EPJB3Bp+jbvzsQ=;
        b=g7wKBayxZQW2hP/3F3o+/3QY0FMUX11Lolz56PlL1BcipTtCM8EU2dOfsMDF099u5I
         5/nSYuBv2hEjSCc8UKV56BN3HyjXWUDSTEypV251/IC0salJzWSCwzHIQChFveCkU9fK
         Xg1TIv0E0MMh0rLgg6OoyhRsk9y8Vt/OsDYukG6SjUUd2k8J5VLCjg9o4VLmDBb8Wx5c
         fNygT+MWsbcnmDheQrePOUJYtCp8k8s50eiNNOGplKuLywbUFYTRwk7d120BmSQ3oAcI
         nRY073JE+58KDWhXDFcSuJ3RNXgPSq4gcE43krDbX0hG+Uuad6YK3AlQM7lO7ZahrKyL
         w6aA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858902; x=1716463702;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9wb2Zg0RGsFll9U2ozXeIHtw+XG8+EPJB3Bp+jbvzsQ=;
        b=O1dgDcTsZV0h9uPukvovN41m7cOtKABh1gqhmgvqVzdDSwgz8kWSrWwZpGYkWuWjoZ
         N4Q5CV2naeiPvpZHNEhOCcnuJjSWplDv5L2VwgabSLe9Yy+vzJf4lHTsMK23lCAGaAcK
         Xo8dlSJSVQCyPdnyNbtO1Knpm96ZenF0Z4wO+iGBF0cGEQebvoIwHFsm0LteLLfnR4XJ
         jl7ZQGQNwz8NKvfNN6pQmbGGEAlxhCeFIIZn5/W6lVt54Ur/XMxRB9+SmtgBth1O7W2s
         7WeirwOeLywaeInf9d7pZdhOxIEOXfAAlJSJ6SAfP5PFugPY6BQWphdxTW3w1UhGQZ16
         TERw==
X-Gm-Message-State: AOJu0Yxu/YYQQ6INtWau/r+355I7c8THe7xX6vdSf0mwk4cv7SAEXDbo
	ZZn3dL6KmCNCVnGGkBOlxqc+axa9oHF7jRnxyu6fnzOfa9J8LuiKNPWSpE3J
X-Google-Smtp-Source: AGHT+IGSJea0rRaSHcwu8qEp0BoExYhQPeYNJ6GI0Qo2/ni19nG/mqFw9W2LQDFxddiHBZUnOB9Ueg==
X-Received: by 2002:a05:6402:278c:b0:574:eb36:9cf5 with SMTP id 4fb4d7f45d1cf-574eb369d9fmr7759577a12.11.1715858901825;
        Thu, 16 May 2024 04:28:21 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH for-4.19? v3 1/6] x86/p2m: Add braces for better code clarity
Date: Thu, 16 May 2024 11:28:08 +0000
Message-Id: <36cb7af0de191b6a8d64931225f21a42c2c04529.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

No functional change.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/x86/mm/p2m.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index ce742c12e0..eb7996170d 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -106,6 +106,7 @@ void p2m_change_entry_type_global(struct domain *d,
         unsigned int i;

         for ( i = 0; i < MAX_ALTP2M; i++ )
+        {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -114,6 +115,7 @@ void p2m_change_entry_type_global(struct domain *d,
                 change_entry_type_global(altp2m, ot, nt);
                 p2m_unlock(altp2m);
             }
+        }
     }

     p2m_unlock(hostp2m);
@@ -139,6 +141,7 @@ void p2m_memory_type_changed(struct domain *d)
         unsigned int i;

         for ( i = 0; i < MAX_ALTP2M; i++ )
+        {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -147,6 +150,7 @@ void p2m_memory_type_changed(struct domain *d)
                 _memory_type_changed(altp2m);
                 p2m_unlock(altp2m);
             }
+        }
     }

     p2m_unlock(hostp2m);
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723177.1127792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-0006uk-FW; Thu, 16 May 2024 11:28:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723177.1127792; Thu, 16 May 2024 11:28:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-0006sK-9r; Thu, 16 May 2024 11:28:26 +0000
Received: by outflank-mailman (input) for mailman id 723177;
 Thu, 16 May 2024 11:28:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHd-0006b7-4l
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:25 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 67cd5efb-1377-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:28:23 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-520f9d559f6so807061e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:23 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67cd5efb-1377-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858903; x=1716463703; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aWD/Qk3szSfEZBTtbpcC8O4b8u+9LVoa4Hx6fikLkdc=;
        b=OAKSN4zl9kKwGdQLBWRMJB9b9kfhMWeWGWoJrYXfQM+7bjt7OFLUMfEUABYRlzXOQF
         uDcSzUfzdIftid9NaWRFtI/jIu5fo92rQTSKgAm6zNZZJFElvyHG6K+haJeA2qWYfMNn
         DziNiECtOOiSCg7gTrlAxuM11+0YgzshHr59pfwctJXT+CgRqMU/3XnFWNyr2xh9SLCJ
         msc7tbERB3uEuWibBMOj3btwV2rSj+auk/wnflHUaLB0YPw8QK0LvITfuyuF2QkJk7hZ
         kX68SrgkCJ/AW35T5jP101ll9Cm9j7hGlHTH78zzpcEyiQ9gf9Vnwy4UQjvz2AI/eHWy
         Pbcw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858903; x=1716463703;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=aWD/Qk3szSfEZBTtbpcC8O4b8u+9LVoa4Hx6fikLkdc=;
        b=br+nRm8I9SQeMESpNj6hx98KYeURRn289JV/DbHENoXBokbSlA/dP+DHslRJx1wFZz
         3pCVNJYv9bwBljyjTQ0GCKwEBGOSkqTES7AtAAg53Xl1876gWTP5xxoskxHQoB3JgoV+
         WGgZFZ1OX7YOEpix5l/Wnu8KQrmXSPeZ/UNV6s0tKprnji8YHFk6RjeOw8M+rWISHcU3
         Azh9a2NYPJmTGQpZoaec03vedKqyXbcv+jdXdolkE5UKGBuajRhn29W1a8pFBgR0NhDF
         SPFtOGjEoGPhr6y50lIgRsyJ1Lhj0PlIvGYy8IsvbGVIbTm3LpsyxaOBY+bEySwBbBty
         1WNw==
X-Gm-Message-State: AOJu0Yy8+R0ZOx5iS+OypjZ9pjEbfZhq46coi/qnTAlrGGUNMcuDJx6G
	PXolxTeALjH4+uNb4RWUITGBKvB9qHD57y991IXvVUs3yL08x6AFAVMNX63g
X-Google-Smtp-Source: AGHT+IGL1s/Lp2LGNzl8vhiI2q4MxCc8OCCAMNjP6bbLOuclhZG+4EypeAZqby+BnX5zKsvL2FiZvg==
X-Received: by 2002:ac2:5b1e:0:b0:519:1047:7eac with SMTP id 2adb3069b0e04-5220fb746edmr12699563e87.23.1715858902834;
        Thu, 16 May 2024 04:28:22 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH for-4.19? v3 2/6] tools/xl: Add altp2m_count parameter
Date: Thu, 16 May 2024 11:28:09 +0000
Message-Id: <71133f351e0a5d6b993c608e8d33e62487d6142c.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Introduce a new altp2m_count parameter to control the maximum number of altp2m
views a domain can use. By default, if altp2m_count is unspecified and altp2m
is enabled, the value is set to 10, reflecting the legacy behavior.

This change is preparatory; it establishes the groundwork for the feature but
does not activate it.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/golang/xenlight/helpers.gen.go | 2 ++
 tools/golang/xenlight/types.gen.go   | 1 +
 tools/include/libxl.h                | 8 ++++++++
 tools/libs/light/libxl_create.c      | 9 +++++++++
 tools/libs/light/libxl_types.idl     | 1 +
 tools/xl/xl_parse.c                  | 4 ++++
 6 files changed, 25 insertions(+)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 78bdb08b15..40c247a0d0 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1158,6 +1158,7 @@ if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
 x.Altp2M = Altp2MMode(xc.altp2m)
+x.Altp2MCount = uint32(xc.altp2m_count)
 x.VmtraceBufKb = int(xc.vmtrace_buf_kb)
 if err := x.Vpmu.fromC(&xc.vpmu);err != nil {
 return fmt.Errorf("converting field Vpmu: %v", err)
@@ -1674,6 +1675,7 @@ if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
 xc.altp2m = C.libxl_altp2m_mode(x.Altp2M)
+xc.altp2m_count = C.uint32_t(x.Altp2MCount)
 xc.vmtrace_buf_kb = C.int(x.VmtraceBufKb)
 if err := x.Vpmu.toC(&xc.vpmu); err != nil {
 return fmt.Errorf("converting field Vpmu: %v", err)
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index ccfe18019e..a3ae8ef35a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -602,6 +602,7 @@ ArchX86 struct {
 MsrRelaxed Defbool
 }
 Altp2M Altp2MMode
+Altp2MCount uint32
 VmtraceBufKb int
 Vpmu Defbool
 }
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..1f149a8015 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1239,6 +1239,14 @@ typedef struct libxl__ctx libxl_ctx;
  */
 #define LIBXL_HAVE_ALTP2M 1

+/*
+ * LIBXL_HAVE_ALTP2M_COUNT
+ * If this is defined, then libxl supports setting the maximum number of
+ * alternate p2m tables.
+ */
+#define LIBXL_HAVE_ALTP2M_COUNT 1
+#define LIBXL_ALTP2M_COUNT_DEFAULT (~(uint32_t)0)
+
 /*
  * LIBXL_HAVE_REMUS
  * If this is defined, then libxl supports remus.
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 41252ec553..236b8c1965 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -483,6 +483,15 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
         return -ERROR_INVAL;
     }

+    if (b_info->altp2m_count == LIBXL_ALTP2M_COUNT_DEFAULT) {
+        if ((libxl_defbool_val(b_info->u.hvm.altp2m) ||
+            b_info->altp2m != LIBXL_ALTP2M_MODE_DISABLED))
+            /* Reflect the default legacy count */
+            b_info->altp2m_count = 10;
+        else
+            b_info->altp2m_count = 0;
+    }
+
     /* Assume that providing a bootloader user implies enabling restrict. */
     libxl_defbool_setdefault(&b_info->bootloader_restrict,
                              !!b_info->bootloader_user);
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..eb306fedf5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -728,6 +728,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     # Alternate p2m is not bound to any architecture or guest type, as it is
     # supported by x86 HVM and ARM support is planned.
     ("altp2m", libxl_altp2m_mode),
+    ("altp2m_count", uint32, {'init_val': 'LIBXL_ALTP2M_COUNT_DEFAULT'}),

     # Size of preallocated vmtrace trace buffers (in KBYTES).
     # Use zero value to disable this feature.
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..048ab6be0d 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2063,6 +2063,10 @@ void parse_config_data(const char *config_source,
         }
     }

+    if (!xlu_cfg_get_long(config, "altp2m_count", &l, 1)) {
+        b_info->altp2m_count = l;
+    }
+
     if (!xlu_cfg_get_long(config, "vmtrace_buf_kb", &l, 1) && l) {
         b_info->vmtrace_buf_kb = l;
     }
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723176.1127788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-0006qu-87; Thu, 16 May 2024 11:28:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723176.1127788; Thu, 16 May 2024 11:28:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHe-0006qn-39; Thu, 16 May 2024 11:28:26 +0000
Received: by outflank-mailman (input) for mailman id 723176;
 Thu, 16 May 2024 11:28:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHb-0006b7-K3
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:23 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 66ba2f71-1377-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:28:21 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51f4d2676d1so820681e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:21 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66ba2f71-1377-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858901; x=1716463701; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Mh2JHdIwpJztutV+fZB/VMbWO9UcZet+J0mJ7x6u9uA=;
        b=FyVEIoKBXcu2dXPz1hqTO9NbZnisIsJy/9AN8C+htXaWZsw41x0RRK4GCEnNupEfoo
         86Ix9j9DDraQFEQGFjhgMNwclL6cuwJHISYaEC4GMlash94NtABRD/jDy0t+EvZE0rGU
         kplI+YL5AdDKq7ej84n2nEC6XYx4c8HPyZeFsy7tMBxWOzKykwhpKyibjkFqTdsGdHXm
         RyNHfCmUEbW2MyV/ME+gBXm/345+tlVO3kCaS4Xyf0cMCrvshlILx/37mZa+mIIGlIhM
         sFp250KCGBzmu1ksN2dqmmriaFD34p2wnZKviQeKbYrQ3PBpsGrawFE1LTFnR7nTcBLQ
         PZdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858901; x=1716463701;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Mh2JHdIwpJztutV+fZB/VMbWO9UcZet+J0mJ7x6u9uA=;
        b=lz18qIs33NtS2KH+Upb7FdFNG4Wlrx6sorGQGD97cf6Y49getzeuwwyIYle3AUo05C
         QFsnhw5qoZWi5j6GHRG+KlRILPCZiybxCQxhwkqegsaMnO6JNKHAO6F4BTBTPD259xoQ
         ICU4hxteHvHDfJV+SBgiBx+SsLDMlVdEduuo0/iKp23ZLUoJ+0BLwqj8r9gy8HjSHz1f
         BAP+V1tR+mR8wHERsXCoi8hdJPHDWVIszzUEuMy1PkNcZapwFmq9FxnwCw7GjKx+4kZ4
         3cxZWNtARvgJlj5RaMsXD53gib1F3M7kt0EMGdHbqFWEGjEtgRWsBN2l6RPju08WAeHz
         kKQQ==
X-Gm-Message-State: AOJu0YxmukrCRlz4cqNqmXCpWuL7myle54E7d/2bzUWxL/TKw0n1izFv
	L+6O+9eRCmwWRiHb8pOH7AuFSl+nTGq7qBg/D73vIqvNXQpXjeEnVLdvkzwr
X-Google-Smtp-Source: AGHT+IGe49aSY0cIVxVwUB7medlul4WDBkOrkONa9aIMfGlMW9FWP5EXeNqwNGPt7Mi/hVzWc3AwMA==
X-Received: by 2002:ac2:5a05:0:b0:521:d17b:3948 with SMTP id 2adb3069b0e04-52210275dd7mr11189735e87.60.1715858900731;
        Thu, 16 May 2024 04:28:20 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>
Subject: [PATCH for-4.19? v3 0/6] x86: Make MAX_ALTP2M configurable
Date: Thu, 16 May 2024 11:28:07 +0000
Message-Id: <cover.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This series introduces the ability to configure the maximum number of altp2m
tables during domain creation. Previously, the limits were hardcoded to a
maximum of 10. This change allows for greater flexibility in environments that
require more or fewer altp2m views.

Adjustments include:
- "Prepare" commits with style changes.
- Adding a new `altp2m_count` parameter to xl.
- Adding a new `altp2m_count` parameter to the OCaml bindings.
- Adding a new `altp2m_count` parameter to the xl.cfg manual.
- Adding a new `nr_altp2m` field into the `xen_domctl_createdomain`, which,
  after sanity checks, is stored in newly introduced `nr_altp2m` field of
  `struct domain` - leaving room for other architectures to implement the
  altp2m feature.
- Replacing MAX_ALTP2M macro occurrences with `domain->nr_altp2m`.

This enhancement is particularly relevant for users leveraging Xen's features
for virtual machine introspection.

Changes since v2:
- Changed max_altp2m to nr_altp2m.
- Moved arch-dependent check from xen/common/domain.c to xen/arch/x86/domain.c.
- Replaced min(d->nr_altp2m, MAX_EPTP) occurences for just d->nr_altp2m.
- Replaced array_index_nospec(altp2m_idx, ...) for just altp2m_idx.
- Shortened long lines.
- Removed unnecessary comments in altp2m_vcpu_initialise/destroy.
- Moved nr_altp2m field after max_ fields in xen_domctl_createdomain.
- Removed the commit that adjusted the initial allocation of pages from 256
  to 1024. This means that after these patches, technically, the nr_altp2m will
  be capped to (256 - 1 + vcpus + MAX_NESTEDP2M) instead of MAX_EPTP (512).
  Future work will be needed to fix this.

Petr Beneš (6):
  x86/p2m: Add braces for better code clarity
  tools/xl: Add altp2m_count parameter
  docs/man: Add altp2m_count parameter to the xl.cfg manual
  x86: Make the maximum number of altp2m views configurable
  tools/libxl: Activate the altp2m_count feature
  tools/ocaml: Add altp2m_count parameter

 docs/man/xl.cfg.5.pod.in             | 14 +++++
 tools/golang/xenlight/helpers.gen.go |  2 +
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/include/libxl.h                |  8 +++
 tools/libs/light/libxl_create.c      | 10 ++++
 tools/libs/light/libxl_types.idl     |  1 +
 tools/ocaml/libs/xc/xenctrl.ml       |  1 +
 tools/ocaml/libs/xc/xenctrl.mli      |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c  | 11 ++--
 tools/xl/xl_parse.c                  |  4 ++
 xen/arch/x86/domain.c                | 12 +++++
 xen/arch/x86/hvm/hvm.c               |  8 ++-
 xen/arch/x86/hvm/vmx/vmx.c           |  2 +-
 xen/arch/x86/include/asm/domain.h    |  7 ++-
 xen/arch/x86/include/asm/p2m.h       |  4 +-
 xen/arch/x86/mm/altp2m.c             | 23 +++++++-
 xen/arch/x86/mm/hap/hap.c            |  6 +--
 xen/arch/x86/mm/mem_access.c         | 23 ++++----
 xen/arch/x86/mm/mem_sharing.c        |  2 +-
 xen/arch/x86/mm/p2m-ept.c            | 10 ++--
 xen/arch/x86/mm/p2m.c                | 80 ++++++++++++++--------------
 xen/common/domain.c                  |  1 +
 xen/include/public/domctl.h          |  5 +-
 xen/include/xen/sched.h              |  2 +
 24 files changed, 161 insertions(+), 77 deletions(-)

--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723180.1127824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHg-0007jZ-Rj; Thu, 16 May 2024 11:28:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723180.1127824; Thu, 16 May 2024 11:28:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHg-0007is-Lt; Thu, 16 May 2024 11:28:28 +0000
Received: by outflank-mailman (input) for mailman id 723180;
 Thu, 16 May 2024 11:28:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHg-0006b6-66
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:28 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a2d68c9-1377-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:28:27 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-572669fd9f9so3346789a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:27 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a2d68c9-1377-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858907; x=1716463707; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lcXsJFjS9m5+2bzRAoHvtJjIOqkMU5GtwndPgJCwhX4=;
        b=FWWYWfDreGrMzpIK2IDFXfRKtGx+biIrQU41MFruHink8j+mCoPA43ryt75rnGfhf0
         IjAm7qjEoA/27cULb5hD66pkbej8DwvpUxRm65k+b4m9Q+RfDU1CewEW1t/3J7oSQS99
         ISk9PN/eg4LgvYFPa0vWxzGL7uzpFKmbOpUUfzMAoSF8Y6O4a3Ri9ajasLERagsr2k1A
         t303GUQRGPCvgsSHU7YlYHCU9Rz1Bz0oWVEfNuVpK0kYC0H45OUv8HmhA8Twkw3fYT+F
         jZLg+WYilupzgc/w3EDqyuxTOiiqe516lmYfd+U/giYEssH+6jT1bWLy0ny054faS/ri
         De6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858907; x=1716463707;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=lcXsJFjS9m5+2bzRAoHvtJjIOqkMU5GtwndPgJCwhX4=;
        b=sDL+yKVUjDM3B7lT67vZ+VRnEft3kyH0pi/oRYMmycWiNu90u0Bez9dNOb3lOcCDcW
         h/neoZZZyfwM33wLWvB1RhEBr8Rf08AHE/5h1bisVAC47u3l/ovpMciznwL06KWfd3O3
         uW5DQg8LTdPLMjT++k/+QMkygmGPpZ6wpjwpBpKw0JrSJPL4Iith2NwcAIWOBCQ6C+PW
         PhRSl0ZomRpa+UA0BfQEWsWa8R1Hg2ePBjB05LkoqClqzgK9DQ+2N8u2E9Qllp01J1lr
         PNT1K8PABlh+xlimYk//rf2SW8ue4tBcTs744Rx52cJkdojt70wLf+TQA+wcVe6oO3Fk
         Y7Eg==
X-Gm-Message-State: AOJu0YwrMYVFIf2nXJhDuEomBNfNGHCMXS4eEkwCycAc6IzwCc/Mlyse
	LA2UqrZQzqv4Uw2CKrhcfMbwa9+5xTZVAEsxUcr6thjE1/OWB0AapbDDUg==
X-Google-Smtp-Source: AGHT+IEDNx0Eao0z1ZBy+RPyD3phVLCE34VncoyEhxg5SSGon5U7YHDOnSbpDqNvuzkTzzEeJEbSSw==
X-Received: by 2002:a50:8d4e:0:b0:571:bed1:3a27 with SMTP id 4fb4d7f45d1cf-5734d6f2d33mr13268941a12.38.1715858905672;
        Thu, 16 May 2024 04:28:25 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH for-4.19? v3 5/6] tools/libxl: Activate the altp2m_count feature
Date: Thu, 16 May 2024 11:28:12 +0000
Message-Id: <d478dbe742f9b8abd9d2e843a9c7b2a8a156157f.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This commit activates the previously introduced altp2m_count parameter,
establishing the connection between libxl and Xen.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/libs/light/libxl_create.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 236b8c1965..f5eb16d0dc 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -657,6 +657,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
             .max_grant_frames = b_info->max_grant_frames,
             .max_maptrack_frames = b_info->max_maptrack_frames,
             .grant_opts = XEN_DOMCTL_GRANT_version(b_info->max_grant_version),
+            .nr_altp2m = b_info->altp2m_count,
             .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SHIFT),
             .cpupool_id = info->poolid,
         };
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723179.1127817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHg-0007YX-5Y; Thu, 16 May 2024 11:28:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723179.1127817; Thu, 16 May 2024 11:28:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHf-0007Wf-Ut; Thu, 16 May 2024 11:28:27 +0000
Received: by outflank-mailman (input) for mailman id 723179;
 Thu, 16 May 2024 11:28:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHf-0006b6-65
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:27 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 69306036-1377-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:28:26 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a0013d551so163905766b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:25 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69306036-1377-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858905; x=1716463705; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iwSyWpfwO+8Pp/PIuWiuZytsATZP0WHIz6nOkOvuXw0=;
        b=SuKNqkuXZTyq0Ytp4vwfYKZtYacJ4esIyj7DcJT06Dc1qJvAHl8cnJ3CGJV2tlt2Mi
         o6hQox7g8TdrDvvhO29fKKiEvO8sMGGsVhyvLmnezXQoXO+Eahze2CSGuoO7PwSNvNzt
         huoti5yjdKaVWjdS+9PK/7HZPOZzBSg5RKNlPd+utsSsuSBZ2vMbbKPcq1AL1GRBwPBW
         NRi/phnkCG36oV+uvLfsM9z+5UudkkXR5pQjTEz56OySuZOQENYga8c3IbqsCHOwz47g
         PQvQApPUz2RnSh6f1oZYLTmYubKbD3Up5Zk4I3hlox4Pn4z2S48vXuESMzIG8Re3IWCQ
         HqwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858905; x=1716463705;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iwSyWpfwO+8Pp/PIuWiuZytsATZP0WHIz6nOkOvuXw0=;
        b=xGzsTEUShujKnNwi685L6tKgg3xOIiDH3OJpxzDqzxvWmrJd7ViVKaKLFT+Dj7Ad7x
         3vx42g0vViICYUDZQrzCcZROjosMaT5dkG8Re9pFtmIpxYm5NmdGNsngs41VN9SZZndG
         93Z+7P/UrlrPXWaxhhEBv+ZtGoa4fkIcgiTbxKcGAPe/vra3Ry3N9mcupo1LTrDo6rt6
         rkPGMCG5FHtKsH1UFMAbeQHM5OVt/PE7u2kT3BWZhcHiuIfiQxAZFAJCdYIyeIM+/jJN
         kIRbA2fRCZdsy7KJNPn3C1G0OzL2aJ2WeDxK8QpHofk/6jiYTSj5n4QqTaIWDG+lyJYt
         a2iQ==
X-Gm-Message-State: AOJu0YyHjimPGP7eT6xYNaWmvSeipCXj1mBYq3MJIL/nwbkds0nUBt8m
	7fSeKFc+UxKvFe8LzmJjxbA3J5BtHnQVkhjnC3sLD95z5hVsaHq3OYoNnSqH
X-Google-Smtp-Source: AGHT+IFeOFO64ZLbiZ4i/uTkzx634CIh9FArxl5qsmEwaJ0WEcvuPBin44FpZpdKvWhzkF38fATy5Q==
X-Received: by 2002:aa7:c35a:0:b0:574:b106:57db with SMTP id 4fb4d7f45d1cf-574b1065a14mr13248546a12.21.1715858904921;
        Thu, 16 May 2024 04:28:24 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH for-4.19? v3 4/6] x86: Make the maximum number of altp2m views configurable
Date: Thu, 16 May 2024 11:28:11 +0000
Message-Id: <b6bf7e5ba0deab4550b9d15c327015fe7d94d463.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This commit introduces the ability to configure the maximum number of altp2m
views for the domain during its creation. Previously, the limits were hardcoded
to a maximum of 10. This change allows for greater flexibility in environments
that require more or fewer altp2m views.

The maximum configurable limit for max_altp2m on x86 is now set to MAX_EPTP
(512). This cap is linked to the architectural limit of the EPTP-switching
VMFUNC, which supports up to 512 entries. Despite there being no inherent need
for limiting max_altp2m in scenarios not utilizing VMFUNC, decoupling these
components would necessitate substantial code changes.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 xen/arch/x86/domain.c             | 12 +++++
 xen/arch/x86/hvm/hvm.c            |  8 +++-
 xen/arch/x86/hvm/vmx/vmx.c        |  2 +-
 xen/arch/x86/include/asm/domain.h |  7 ++-
 xen/arch/x86/include/asm/p2m.h    |  4 +-
 xen/arch/x86/mm/altp2m.c          | 23 +++++++++-
 xen/arch/x86/mm/hap/hap.c         |  6 +--
 xen/arch/x86/mm/mem_access.c      | 23 ++++------
 xen/arch/x86/mm/mem_sharing.c     |  2 +-
 xen/arch/x86/mm/p2m-ept.c         | 10 ++--
 xen/arch/x86/mm/p2m.c             | 76 +++++++++++++++----------------
 xen/common/domain.c               |  1 +
 xen/include/public/domctl.h       |  5 +-
 xen/include/xen/sched.h           |  2 +
 14 files changed, 108 insertions(+), 73 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 00a3aaa576..3bd18cb2d0 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -685,6 +685,18 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }

+    if ( config->nr_altp2m && !hvm_altp2m_supported() )
+    {
+        dprintk(XENLOG_INFO, "altp2m requested but not available\n");
+        return -EINVAL;
+    }
+
+    if ( config->nr_altp2m > MAX_EPTP )
+    {
+        dprintk(XENLOG_INFO, "nr_altp2m must be <= %lu\n", MAX_EPTP);
+        return -EINVAL;
+    }
+
     if ( config->vmtrace_size )
     {
         unsigned int size = config->vmtrace_size;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c5..77e4016bdb 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4639,6 +4639,12 @@ static int do_altp2m_op(
         goto out;
     }

+    if ( d->nr_altp2m == 0 )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
     if ( (rc = xsm_hvm_altp2mhvm_op(XSM_OTHER, d, mode, a.cmd)) )
         goto out;

@@ -5228,7 +5234,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)
     if ( !hvm_is_singlestep_supported() )
         return;

-    if ( p2midx >= MAX_ALTP2M )
+    if ( p2midx >= v->domain->nr_altp2m )
         return;

     v->arch.hvm.single_step = true;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 5f67a48592..76ee09b701 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -4888,7 +4888,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
         {
             unsigned int i;

-            for ( i = 0; i < MAX_ALTP2M; ++i )
+            for ( i = 0; i < currd->nr_altp2m; ++i )
             {
                 if ( currd->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
                     continue;
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index f5daeb182b..3935328781 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -258,11 +258,10 @@ struct paging_vcpu {
     struct shadow_vcpu shadow;
 };

-#define MAX_NESTEDP2M 10
-
-#define MAX_ALTP2M      10 /* arbitrary */
 #define INVALID_ALTP2M  0xffff
 #define MAX_EPTP        (PAGE_SIZE / sizeof(uint64_t))
+#define MAX_NESTEDP2M   10
+
 struct p2m_domain;
 struct time_scale {
     int shift;
@@ -353,7 +352,7 @@ struct arch_domain

     /* altp2m: allow multiple copies of host p2m */
     bool altp2m_active;
-    struct p2m_domain *altp2m_p2m[MAX_ALTP2M];
+    struct p2m_domain **altp2m_p2m;
     mm_lock_t altp2m_list_lock;
     uint64_t *altp2m_eptp;
     uint64_t *altp2m_visible_eptp;
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 111badf89a..6faa105baf 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -881,7 +881,7 @@ static inline struct p2m_domain *p2m_get_altp2m(struct vcpu *v)
     if ( index == INVALID_ALTP2M )
         return NULL;

-    BUG_ON(index >= MAX_ALTP2M);
+    BUG_ON(index >= v->domain->nr_altp2m);

     return v->domain->arch.altp2m_p2m[index];
 }
@@ -891,7 +891,7 @@ static inline bool p2m_set_altp2m(struct vcpu *v, unsigned int idx)
 {
     struct p2m_domain *orig;

-    BUG_ON(idx >= MAX_ALTP2M);
+    BUG_ON(idx >= v->domain->nr_altp2m);

     if ( idx == vcpu_altp2m(v).p2midx )
         return false;
diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
index a04297b646..98dbb50d47 100644
--- a/xen/arch/x86/mm/altp2m.c
+++ b/xen/arch/x86/mm/altp2m.c
@@ -13,6 +13,11 @@
 void
 altp2m_vcpu_initialise(struct vcpu *v)
 {
+    struct domain *d = v->domain;
+
+    if ( d->nr_altp2m == 0 )
+        return;
+
     if ( v != current )
         vcpu_pause(v);

@@ -28,8 +33,12 @@ altp2m_vcpu_initialise(struct vcpu *v)
 void
 altp2m_vcpu_destroy(struct vcpu *v)
 {
+    struct domain *d = v->domain;
     struct p2m_domain *p2m;

+    if ( d->nr_altp2m == 0 )
+        return;
+
     if ( v != current )
         vcpu_pause(v);

@@ -120,7 +129,12 @@ int p2m_init_altp2m(struct domain *d)
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);

     mm_lock_init(&d->arch.altp2m_list_lock);
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, d->nr_altp2m);
+
+    if ( !d->arch.altp2m_p2m )
+        return -ENOMEM;
+
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         d->arch.altp2m_p2m[i] = p2m = p2m_init_one(d);
         if ( p2m == NULL )
@@ -141,7 +155,10 @@ void p2m_teardown_altp2m(struct domain *d)
     unsigned int i;
     struct p2m_domain *p2m;

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    if ( !d->arch.altp2m_p2m )
+        return;
+
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( !d->arch.altp2m_p2m[i] )
             continue;
@@ -149,6 +166,8 @@ void p2m_teardown_altp2m(struct domain *d)
         d->arch.altp2m_p2m[i] = NULL;
         p2m_free_one(p2m);
     }
+
+    XFREE(d->arch.altp2m_p2m);
 }

 /*
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index d2011fde24..501fd9848b 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -515,7 +515,7 @@ int hap_enable(struct domain *d, u32 mode)
             d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
         }

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             rv = p2m_alloc_table(d->arch.altp2m_p2m[i]);
             if ( rv != 0 )
@@ -538,7 +538,7 @@ void hap_final_teardown(struct domain *d)
     unsigned int i;

     if ( hvm_altp2m_supported() )
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
             p2m_teardown(d->arch.altp2m_p2m[i], true, NULL);

     /* Destroy nestedp2m's first */
@@ -590,7 +590,7 @@ void hap_teardown(struct domain *d, bool *preempted)
         FREE_XENHEAP_PAGE(d->arch.altp2m_eptp);
         FREE_XENHEAP_PAGE(d->arch.altp2m_visible_eptp);

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             p2m_teardown(d->arch.altp2m_p2m[i], false, preempted);
             if ( preempted && *preempted )
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 60a0cce68a..015b93447a 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -347,12 +347,11 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, uint32_t nr,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[altp2m_idx];
     }

     if ( !xenmem_access_to_p2m_access(p2m, access, &a) )
@@ -403,12 +402,11 @@ long p2m_set_mem_access_multi(struct domain *d,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[altp2m_idx];
     }

     p2m_lock(p2m);
@@ -466,12 +464,11 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access,
     }
     else if ( altp2m_idx ) /* altp2m view 0 is treated as the hostp2m */
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = d->arch.altp2m_p2m[altp2m_idx];
     }

     return _p2m_get_mem_access(p2m, gfn, access);
@@ -486,7 +483,7 @@ void arch_p2m_set_access_required(struct domain *d, bool access_required)
     if ( altp2m_active(d) )
     {
         unsigned int i;
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             struct p2m_domain *p2m = d->arch.altp2m_p2m[i];

diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index da28266ef0..83bb9dd5df 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -912,7 +912,7 @@ static int nominate_page(struct domain *d, gfn_t gfn,

         altp2m_list_lock(d);

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             ap2m = d->arch.altp2m_p2m[i];
             if ( !ap2m )
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index f83610cb8c..90b50a0fcf 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1293,7 +1293,7 @@ static void ept_set_ad_sync(struct domain *d, bool value)
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             struct p2m_domain *p2m;

@@ -1500,15 +1500,15 @@ void setup_ept_dump(void)

 void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
 {
-    struct p2m_domain *p2m = array_access_nospec(d->arch.altp2m_p2m, i);
+    struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
     struct ept_data *ept;

     p2m->ept.ad = hostp2m->ept.ad;
     ept = &p2m->ept;
     ept->mfn = pagetable_get_pfn(p2m_get_pagetable(p2m));
-    d->arch.altp2m_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
-    d->arch.altp2m_visible_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
+    d->arch.altp2m_eptp[i] = ept->eptp;
+    d->arch.altp2m_visible_eptp[i] = ept->eptp;
 }

 unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
@@ -1519,7 +1519,7 @@ unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
             continue;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index eb7996170d..a178645c69 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -105,7 +105,7 @@ void p2m_change_entry_type_global(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -140,7 +140,7 @@ void p2m_memory_type_changed(struct domain *d)
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -913,7 +913,7 @@ void p2m_change_type_range(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -986,7 +986,7 @@ int p2m_finish_type_change(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -1780,7 +1780,7 @@ bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
     struct domain *d = v->domain;
     bool rc = false;

-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= d->nr_altp2m )
         return rc;

     altp2m_list_lock(d);
@@ -1886,8 +1886,8 @@ static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
 {
     struct p2m_domain *p2m;

-    ASSERT(idx < MAX_ALTP2M);
-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ASSERT(idx < d->nr_altp2m);
+    p2m = d->arch.altp2m_p2m[idx];

     p2m_lock(p2m);

@@ -1912,7 +1912,7 @@ void p2m_flush_altp2m(struct domain *d)

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
         d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
@@ -1928,9 +1928,9 @@ static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
     struct p2m_domain *hostp2m, *p2m;
     int rc;

-    ASSERT(idx < MAX_ALTP2M);
+    ASSERT(idx < d->nr_altp2m);

-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    p2m = d->arch.altp2m_p2m[idx];
     hostp2m = p2m_get_hostp2m(d);

     p2m_lock(p2m);
@@ -1968,12 +1968,12 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
     int rc = -EINVAL;
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);

-    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( idx >= d->nr_altp2m )
         return rc;

     altp2m_list_lock(d);

-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
+    if ( d->arch.altp2m_eptp[idx] ==
          mfn_x(INVALID_MFN) )
         rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);

@@ -1995,7 +1995,7 @@ int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             continue;
@@ -2017,7 +2017,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     struct p2m_domain *p2m;
     int rc = -EBUSY;

-    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( !idx || idx >= d->nr_altp2m )
         return rc;

     rc = domain_pause_except_self(d);
@@ -2027,17 +2027,17 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     rc = -EBUSY;
     altp2m_list_lock(d);

-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
+    if ( d->arch.altp2m_eptp[idx] !=
          mfn_x(INVALID_MFN) )
     {
-        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+        p2m = d->arch.altp2m_p2m[idx];

         if ( !_atomic_read(p2m->active_vcpus) )
         {
             p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
-            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =
+            d->arch.altp2m_eptp[idx] =
                 mfn_x(INVALID_MFN);
-            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP)] =
+            d->arch.altp2m_visible_eptp[idx] =
                 mfn_x(INVALID_MFN);
             rc = 0;
         }
@@ -2055,7 +2055,7 @@ int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
     struct vcpu *v;
     int rc = -EINVAL;

-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= d->nr_altp2m )
         return rc;

     rc = domain_pause_except_self(d);
@@ -2090,13 +2090,12 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
     mfn_t mfn;
     int rc = -EINVAL;

-    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
-         mfn_x(INVALID_MFN) )
+    if ( idx >= d->nr_altp2m ||
+         d->arch.altp2m_eptp[idx] == mfn_x(INVALID_MFN) )
         return rc;

     hp2m = p2m_get_hostp2m(d);
-    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ap2m = d->arch.altp2m_p2m[idx];

     p2m_lock(hp2m);
     p2m_lock(ap2m);
@@ -2152,7 +2151,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         p2m_type_t t;
         p2m_access_t a;
@@ -2175,7 +2174,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
             else
             {
                 /* At least 2 altp2m's impacted, so reset everything */
-                for ( i = 0; i < MAX_ALTP2M; i++ )
+                for ( i = 0; i < d->nr_altp2m; i++ )
                 {
                     if ( i == last_reset_idx ||
                          d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
@@ -2575,12 +2574,11 @@ int p2m_set_suppress_ve_multi(struct domain *d,

     if ( sve->view > 0 )
     {
-        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( sve->view >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[sve->view] == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
+        p2m = ap2m = d->arch.altp2m_p2m[sve->view];
     }

     p2m_lock(host_p2m);
@@ -2643,12 +2641,11 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,

     if ( altp2m_idx > 0 )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = ap2m = d->arch.altp2m_p2m[altp2m_idx];
     }
     else
         p2m = host_p2m;
@@ -2679,17 +2676,16 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,

     /*
      * Eptp index is correlated with altp2m index and should not exceed
-     * min(MAX_ALTP2M, MAX_EPTP).
+     * d->nr_altp2m.
      */
-    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-         mfn_x(INVALID_MFN) )
+    if ( altp2m_idx >= d->nr_altp2m ||
+         d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
         rc = -EINVAL;
     else if ( visible )
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
-            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)];
+        d->arch.altp2m_visible_eptp[altp2m_idx] =
+            d->arch.altp2m_eptp[altp2m_idx];
     else
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
+        d->arch.altp2m_visible_eptp[altp2m_idx] =
             mfn_x(INVALID_MFN);

     altp2m_list_unlock(d);
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6773f7fb90..a10a70e9d4 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -610,6 +610,7 @@ struct domain *domain_create(domid_t domid,
     if ( config )
     {
         d->options = config->flags;
+        d->nr_altp2m = config->nr_altp2m;
         d->vmtrace_size = config->vmtrace_size;
     }

diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..60a871f123 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -21,7 +21,7 @@
 #include "hvm/save.h"
 #include "memory.h"

-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017

 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -86,6 +86,9 @@ struct xen_domctl_createdomain {

     uint32_t grant_opts;

+    /* Number of altp2ms to allocate. */
+    uint32_t nr_altp2m;
+
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b841995..18cc0748a1 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -602,6 +602,8 @@ struct domain
         unsigned int guest_request_sync          : 1;
     } monitor;

+    unsigned int nr_altp2m;    /* Number of altp2m tables */
+
     unsigned int vmtrace_size; /* Buffer size in bytes, or 0 to disable. */

 #ifdef CONFIG_ARGO
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:28:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:28:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723181.1127837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHj-0008BJ-5V; Thu, 16 May 2024 11:28:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723181.1127837; Thu, 16 May 2024 11:28:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZHj-0008B7-1q; Thu, 16 May 2024 11:28:31 +0000
Received: by outflank-mailman (input) for mailman id 723181;
 Thu, 16 May 2024 11:28:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=S52g=MT=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s7ZHh-0006b7-LA
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:28:29 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6a7b8689-1377-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:28:28 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-572a93890d1so3445325a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:28:28 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcede889sm7083043a12.92.2024.05.16.04.28.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:28:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a7b8689-1377-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715858907; x=1716463707; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CZx7e4I8f24AmBOZ73vCNLclq5iufdJNqbBIJFjDxwY=;
        b=Sc+mGZyYiWezTycnwzIkred2qGbRW+6rpfIGtNqySYtcrSrpBgrs2p/ku0YoqdLwvb
         Tdx2bJkN6eCAQbnlVkptbdxrpg5YYHIuYQR1OEZeBNScLuqYIZ7ugdRFy4NFjKlMyCoI
         4MeRaJy03JgtdJ9Kk/Ol551OB2wyZMEAVqN+zCjn7xmObnitvGXRfLgPyeNmXRJV9JvZ
         1vyEj0l4h4geTuhJQkGa1M3lUp+3zX6OZdeeoMIzKQwes6W/+gEzWX+k3IKW9FD2/dv5
         sB47X6tLcizzLiNGoVvob1ewIvDg2FQGiEJ5+nECFq3IKqwJwivPkJzN8cf8O+fsBFDM
         ZrLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715858907; x=1716463707;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=CZx7e4I8f24AmBOZ73vCNLclq5iufdJNqbBIJFjDxwY=;
        b=iIJXLTJMcQOloRgan+VZJI+2Ym3hokvgT1WFn8QydS5vW1FoPsuhn9ls+Kx8+zABAU
         EuEjJEJJsK98zeHwD+qS1VlCbva3YppnvM9sAshy2dVjKEOICONEDc9EqNw0eaB1Ws1N
         TtXBQEcjVoJ/7WHjJh0yi5+rIGCIMDQfYDgiApKzMTNroR341kjC83J/p+cM7s6ORqEO
         sVHi8gyBTt6YU7shdG/EuSJSz25j+PfTP7ACzL3Bhu2OjU88UGpRuU/oh3dre9uUhcc4
         TGMlFNl25GWctkaAff+9ultl39coGEGyFKgkbwv6rUDw9use0b6LgnbHM7yF43VMwff5
         jlmA==
X-Gm-Message-State: AOJu0YzLtRB9VNNm1goczEVocIKBoOUkt+Bj8Ci0UDgdyqXhUlGwKajg
	tE8MkH+o+0D6V2pzxJM4wK8xMd00C2gZkPFMWu+44Y8309gXHpbnCVNB/xOh
X-Google-Smtp-Source: AGHT+IF340ISLMPoj07I1Jt9JQ5Xug7pRVHWxjYlGkF/DFVxJW3+Izogh9HGDdXfmKU9eVffMT049g==
X-Received: by 2002:a50:bb0f:0:b0:56f:e7b9:e67d with SMTP id 4fb4d7f45d1cf-5734d5cec3bmr19668133a12.12.1715858907443;
        Thu, 16 May 2024 04:28:27 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Christian Lindig <christian.lindig@cloud.com>
Subject: [PATCH for-4.19? v3 6/6] tools/ocaml: Add altp2m_count parameter
Date: Thu, 16 May 2024 11:28:13 +0000
Message-Id: <8537c5deeb409580868ec8d88171146dc7138662.1715858136.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715858136.git.w1benny@gmail.com>
References: <cover.1715858136.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Allow developers using the OCaml bindings to set the altp2m_count parameter.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
---
 tools/ocaml/libs/xc/xenctrl.ml      |  1 +
 tools/ocaml/libs/xc/xenctrl.mli     |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c | 11 +++++++----
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index 55923857ec..dfb3d331c9 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -85,6 +85,7 @@ type domctl_create_config =
     max_grant_frames: int;
     max_maptrack_frames: int;
     max_grant_version: int;
+    altp2m_count: int;
     vmtrace_buf_kb: int32;
     cpupool_id: int32;
     arch: arch_domainconfig;
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 9b4b45db3a..ff0e309c56 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -77,6 +77,7 @@ type domctl_create_config = {
   max_grant_frames: int;
   max_maptrack_frames: int;
   max_grant_version: int;
+  altp2m_count: int;
   vmtrace_buf_kb: int32;
   cpupool_id: int32;
   arch: arch_domainconfig;
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index 2b6d3c09df..1f544cd2e4 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -210,9 +210,10 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
 #define VAL_MAX_GRANT_VERSION   Field(config, 8)
-#define VAL_VMTRACE_BUF_KB      Field(config, 9)
-#define VAL_CPUPOOL_ID          Field(config, 10)
-#define VAL_ARCH                Field(config, 11)
+#define VAL_ALTP2M_COUNT        Field(config, 9)
+#define VAL_VMTRACE_BUF_KB      Field(config, 10)
+#define VAL_CPUPOOL_ID          Field(config, 11)
+#define VAL_ARCH                Field(config, 12)

 	uint32_t domid = Int_val(wanted_domid);
 	uint64_t vmtrace_size = Int32_val(VAL_VMTRACE_BUF_KB);
@@ -230,6 +231,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 		.max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES),
 		.grant_opts =
 		    XEN_DOMCTL_GRANT_version(Int_val(VAL_MAX_GRANT_VERSION)),
+		.nr_altp2m = Int_val(VAL_ALTP2M_COUNT),
 		.vmtrace_size = vmtrace_size,
 		.cpupool_id = Int32_val(VAL_CPUPOOL_ID),
 	};
@@ -257,7 +259,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #if defined(__i386__) || defined(__x86_64__)

 		/* Quick & dirty check for ABI changes. */
-		BUILD_BUG_ON(sizeof(cfg) != 64);
+		BUILD_BUG_ON(sizeof(cfg) != 68);

         /* Mnemonics for the named fields inside xen_x86_arch_domainconfig */
 #define VAL_EMUL_FLAGS          Field(arch_domconfig, 0)
@@ -288,6 +290,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #undef VAL_ARCH
 #undef VAL_CPUPOOL_ID
 #undef VAL_VMTRACE_BUF_KB
+#undef VAL_ALTP2M_COUNT
 #undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:31:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:31:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723194.1127846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZKH-0003Jt-IP; Thu, 16 May 2024 11:31:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723194.1127846; Thu, 16 May 2024 11:31:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZKH-0003Jm-Fu; Thu, 16 May 2024 11:31:09 +0000
Received: by outflank-mailman (input) for mailman id 723194;
 Thu, 16 May 2024 11:31:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7ZKG-0003Jg-Jw
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:31:08 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c88ded90-1377-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:31:05 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5206a5854adso755933e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:31:05 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed17b8sm10456692a12.55.2024.05.16.04.31.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:31:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c88ded90-1377-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715859065; x=1716463865; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=7eES0enU6K30VZCAG3M0bmIoFFsSRcG+0mogoPG61rA=;
        b=em+P0DBWNWQNW419luz+38MFt+0VYrl1KONZ9m0SlFVigOEu1uAs5lp0tJg6ZYpJA6
         1J+oMFZ3AFsrjlow3K3yWEiMamXs0/sa3+IWKL2w3t5SbmGIVpc5WeAs72A2RN3eY7kN
         pPOkfoU4f5uZ0yjhdUNImxdLuGGr/eg3j2HH8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715859065; x=1716463865;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=7eES0enU6K30VZCAG3M0bmIoFFsSRcG+0mogoPG61rA=;
        b=XpTg0L5u8EKGafNz2xvPbLT22/p9mKKySwnRdbpuJ/MwE6zvsTqLs3S9Ziol1obzzy
         3pqlqjfQl47mqHKoDz/gQtmUc2dy0O/Hn/tewENjIiR2N/mJTRMyI6gMvwZ1N/fmuRyJ
         hQrpA6zX2kZKZWyq367r4/T5zkgpulsr56BWS3k6HCWqrOcF2W4+RMPV+zBF0d/hYtb+
         A8MRTMUKaBLlJl7BMlNEWmmqPsGmy3Y/p3mEbDsmFnYE7HWANa+nL+l/xBtQD5icVJpN
         BEEQTfPDcce+amt1DhyiR3aoTGk+cnmjeVLaqImPp0xFIeuZo5e7lGu+QEGxHiR67DiE
         x0kw==
X-Gm-Message-State: AOJu0YxElog3a+otReNYsn04qV4D7t9xR26qA7DNWsD8EccEmccG8z2I
	Pz0VA/ifzKMIYznstd4rkQjkKg5yLWyIuhgkchRPunzKxpuAQkRtV7ph7ptgbKShn4IOANCsE3a
	h
X-Google-Smtp-Source: AGHT+IHXVoEftk3i1p1rm5wrPzpW1+xaJlcnZsQHCYGBs1diEX6qnDShhwN9ttLosOwgdsFR2cG+FA==
X-Received: by 2002:a05:6512:3ca9:b0:523:8bce:3db7 with SMTP id 2adb3069b0e04-5238bce3e92mr4606066e87.68.1715859064823;
        Thu, 16 May 2024 04:31:04 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Fouad Hilly <fouad.hilly@cloud.com>
Subject: [PATCH] x86/ucode: Further fixes to identify "ucode already up to date"
Date: Thu, 16 May 2024 12:31:03 +0100
Message-Id: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When the revision in hardware is newer than anything Xen has to hand,
'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
because it doesn't know whether the revisions across the system are symmetric
or not.  This involves the patch getting all the way into the
apply_microcode() hooks before being found to be too old.

This is all a giant mess and needs an overhaul, but in the short term simply
adjust the apply_microcode() to return -EEXIST.

Also, unconditionally print the preexisting microcode revision on boot.  It's
relevant information which is otherwise unavailable if Xen doesn't find new
microcode to use.

Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Fouad Hilly <fouad.hilly@cloud.com>

Sorry Fouad, but this collides with your `--force` series once again.
Hopefully it might make things fractionally easier.

Background: For 06-55-04 (Skylake server, stepping 4 specifically), there's a
recent production firmware update which has a newer microcode revision than
exists in the Intel public microcode repository.  It's causing a mess in our
automated testing, although it is finding good bugs...
---
 xen/arch/x86/cpu/microcode/amd.c   | 7 +++++--
 xen/arch/x86/cpu/microcode/core.c  | 2 ++
 xen/arch/x86/cpu/microcode/intel.c | 7 +++++--
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
index 17e68697d5bf..f76a563c8b84 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -222,12 +222,15 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
     uint32_t rev, old_rev = sig->rev;
     enum microcode_match_result result = microcode_fits(patch);
 
+    if ( result == MIS_UCODE )
+        return -EINVAL;
+
     /*
      * Allow application of the same revision to pick up SMT-specific changes
      * even if the revision of the other SMT thread is already up-to-date.
      */
-    if ( result != NEW_UCODE && result != SAME_UCODE )
-        return -EINVAL;
+    if ( result == OLD_UCODE )
+        return -EEXIST;
 
     if ( check_final_patch_levels(sig) )
     {
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 762111838f5f..519798dca4af 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -881,6 +881,8 @@ int __init early_microcode_init(unsigned long *module_map,
 
     ucode_ops.collect_cpu_info();
 
+    printk(XENLOG_INFO "Boot microcode revision: 0x%08x\n", this_cpu(cpu_sig).rev);
+
     /*
      * Some hypervisors deliberately report a microcode revision of -1 to
      * mean that they will not accept microcode updates.
diff --git a/xen/arch/x86/cpu/microcode/intel.c b/xen/arch/x86/cpu/microcode/intel.c
index 96f34b336b21..f505aa1b7888 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -294,10 +294,13 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
 
     result = microcode_update_match(patch);
 
-    if ( result != NEW_UCODE &&
-         !(opt_ucode_allow_same && result == SAME_UCODE) )
+    if ( result == MIS_UCODE )
         return -EINVAL;
 
+    if ( result == OLD_UCODE ||
+         (result == SAME_UCODE && !opt_ucode_allow_same) )
+        return -EEXIST;
+
     wbinvd();
 
     wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)patch->data);
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 11:35:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:35:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723204.1127857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZOQ-0004Th-3y; Thu, 16 May 2024 11:35:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723204.1127857; Thu, 16 May 2024 11:35:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZOP-0004Ta-W4; Thu, 16 May 2024 11:35:25 +0000
Received: by outflank-mailman (input) for mailman id 723204;
 Thu, 16 May 2024 11:35:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7ZOO-0004TU-R3
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:35:24 +0000
Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com
 [2607:f8b0:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 615ea7e8-1378-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:35:23 +0200 (CEST)
Received: by mail-oi1-x22a.google.com with SMTP id
 5614622812f47-3c99c66a40fso46747b6e.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:35:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792eb161bbcsm311246885a.101.2024.05.16.04.35.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:35:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 615ea7e8-1378-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715859321; x=1716464121; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=nBEBKPB35mNcVwksdSFx/MSARJZfyiJx4PSMiiGT4bQ=;
        b=n1kvDsjRe0ypxlZtiyn7WM9b7MiG3TlznbzqmhJdn2LqwI0MBwgEws9XFREifv/dxC
         pgh7o0xV36bH1l7c8kQzEPcqe83bYAi/SNToHvSv7Exh34getsKogHMpcOngVKK3hors
         uQuxrJSy/kQGr533VRsX4pnfwiaZ6QJ3+4VgY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715859321; x=1716464121;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nBEBKPB35mNcVwksdSFx/MSARJZfyiJx4PSMiiGT4bQ=;
        b=E/YMqBVMzb2QTpqBMs5u686ikpWqq0kv/ZUthc+pXXa4QIlSv2DT+uxC4/WXrDGnQr
         9DaCiofYGAx++981MzbHj7iJiS0HnBBeaBXWcFi3J1ChK7XrMR+gN9wm1FgEReA0oHtp
         4wNu/TMSBTs/xWfwMw62U53uTK0dbDwogpGSMmWFeZ/vMSYIAifowF0e5O0uY4TZW63n
         CkMzEf72j3x/HjdLmaRsxJ9WKC5/PtEOfLljFs+CoKtgVdv0xeyOAd7z7NFnTiICh5F2
         ilQ5JpOy2NnN4C+AulLIUur7C/W0Lz5EchV+KMPQyvsM/bM2FcN6wEyfmjqN49Kvsrop
         E5yA==
X-Gm-Message-State: AOJu0YzuriCUK6R6mVfYlYPHrq4jv4BcDYIy30HMHrq0p8dVmkbEmQb6
	7lqSMKjj9vYEqVNh63EhZr+38vXy3xw+kpmyQQsspk/V0DJOXxm8xJJLJ7dyW5c=
X-Google-Smtp-Source: AGHT+IH84BgWpR5JzLbArnHdBU75mkXFyzkcEqYUThLfhB1UruUj5XCItLsyVv82UfUYwjxTZnmq2A==
X-Received: by 2002:a05:6808:4408:b0:3c9:93f1:c7ac with SMTP id 5614622812f47-3c9970cdc49mr17668074b6e.40.1715859320262;
        Thu, 16 May 2024 04:35:20 -0700 (PDT)
Date: Thu, 16 May 2024 13:35:17 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
Message-ID: <ZkXvdWf2mVkPW458@macbook>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240516110710.3446-1-alejandro.vallejo@cloud.com>

On Thu, May 16, 2024 at 12:07:10PM +0100, Alejandro Vallejo wrote:
> Bring test_x86_emulator in line with other tests by adding
> install/uninstall rules.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
>  tools/tests/x86_emulator/Makefile | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
> index 834b2112e7fe..30edf7e0185d 100644
> --- a/tools/tests/x86_emulator/Makefile
> +++ b/tools/tests/x86_emulator/Makefile
> @@ -269,8 +269,15 @@ clean:
>  .PHONY: distclean
>  distclean: clean
>  
> -.PHONY: install uninstall
> -install uninstall:
> +.PHONY: install
> +install: all
> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
> +
> +.PHONY: uninstall
> +uninstall:
> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
> +

FWIW, should you check that HOSTCC == CC before installing?  Otherwise
I'm unsure of the result in cross-compiled builds, as the x86_emulator
is built with HOSTCC, not CC.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:45:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:45:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723223.1127884 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZXZ-0000z5-6F; Thu, 16 May 2024 11:44:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723223.1127884; Thu, 16 May 2024 11:44:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZXZ-0000yy-1B; Thu, 16 May 2024 11:44:53 +0000
Received: by outflank-mailman (input) for mailman id 723223;
 Thu, 16 May 2024 11:44:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ZXX-0000yp-SR
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:44:51 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b3cc336d-1379-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:44:50 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a5cdd6cfae7so201874966b.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:44:50 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0af0sm10403816a12.49.2024.05.16.04.44.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:44:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3cc336d-1379-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715859890; x=1716464690; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sPpXE8yYsiVypob5r3TlAADd7rGNXxVldr3zbRZj9N4=;
        b=MYeVQJY2qTtEITPWfHugVzWsgbZIHu3yJgzZp97ZzKrR7vZxprufwYERvR8h8QlyC3
         1VV1v/o3ZLbggeYI1G/xCC9WejxAc+mFpwtH0bwdjUVFEnlIeVicSI+ds4fNWLvHYAu1
         S4cHS5AbR24sBrvaQGpG0ksnO5i9x7Xd8y/Mps1OmwpCxTYgd2LjmdVChIsqJLQYxFyR
         BAmQbaQAARm2uPJX7eQ5Fbn631z0NGN3iZGdURQaRvCEWHdbQgmRyNntHd0EGtJSeb8E
         WLdcfPCELb/DY1b5DHG8F/jIdMwdMhlMtsF+/hSx2kp3KjTmKViYI4DZr+xjwXxlaXeA
         EsMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715859890; x=1716464690;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sPpXE8yYsiVypob5r3TlAADd7rGNXxVldr3zbRZj9N4=;
        b=jKG7mBYQDarFAGGIkI8enpuL03d4bmz0mRG/wmO7CVqN2mDe60dswCp9m8CqYlRyhp
         FEiJrgvWexskP58szmLfhyOGmtao76NLcMry7toPMpll2L6nOv4RCgbDVWGpUA0t/wx/
         jjdYyKSSJ9No+DxUqNfujXzIW/N5N+gHKOt/h9TRnrlqGYXWk2qEjuR9ZF3GhL0cKMR5
         VLBPDw8UUwLEBQNRLt6CXLY9SNeCoQTePFYyL77mvvFmvtKZAca3e6kw1t/NjC8GYty7
         XBXq2ZImWkXIHKs6fEfS4KpmGGab0ui0tVk4WMJ4sI0OIiqAARU99xZrAcHJf2B9X0Uc
         ulrA==
X-Forwarded-Encrypted: i=1; AJvYcCVI/xYO+I7NTSx9gcSSw7XNeW1gTT1TVGVxvdelfnuYYLD5d1SyqL42sGFpp9knG9mQnhQUHXkf2585tGmNPi8WlfpyZEpVq23Een0rC5w=
X-Gm-Message-State: AOJu0YxZk3jfa6nbd7feM6h6MRyhp9Zxi0+A1Qtd8byCQAxZ2W1wq70V
	kVQSjWGRu00gTSi/ldpVBXLk9Y5zHkqIz544CyrShRAdj68CTWjnzAysHPobLA==
X-Google-Smtp-Source: AGHT+IEf/ZiRmzfovEEA1TGrkFaR/f49orm9OX6DvASdIWXHJsx6bwse6SJPwaDuPKa7ZvrYUrMmkg==
X-Received: by 2002:a17:906:6a10:b0:a59:9636:f3e with SMTP id a640c23a62f3a-a5a118c520bmr2144877066b.33.1715859890221;
        Thu, 16 May 2024 04:44:50 -0700 (PDT)
Message-ID: <7d8fa536-f196-46e8-870f-0cfb57c8a65d@suse.com>
Date: Thu, 16 May 2024 13:44:48 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/ucode: Further fixes to identify "ucode already up to
 date"
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 13:31, Andrew Cooper wrote:
> When the revision in hardware is newer than anything Xen has to hand,
> 'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
> because it doesn't know whether the revisions across the system are symmetric
> or not.  This involves the patch getting all the way into the
> apply_microcode() hooks before being found to be too old.
> 
> This is all a giant mess and needs an overhaul, but in the short term simply
> adjust the apply_microcode() to return -EEXIST.
> 
> Also, unconditionally print the preexisting microcode revision on boot.  It's
> relevant information which is otherwise unavailable if Xen doesn't find new
> microcode to use.

Since you do this for the BSP only, I'm okay with that. Doing this for all
CPUs would have added too much verbosity imo, and I would then have asked
to log the pre-existing revision only when no update would be done by us.

> Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one small request related to the remark above:

> --- a/xen/arch/x86/cpu/microcode/core.c
> +++ b/xen/arch/x86/cpu/microcode/core.c
> @@ -881,6 +881,8 @@ int __init early_microcode_init(unsigned long *module_map,
>  
>      ucode_ops.collect_cpu_info();
>  
> +    printk(XENLOG_INFO "Boot microcode revision: 0x%08x\n", this_cpu(cpu_sig).rev);

Can this please be "BSP" or "Boot CPU" instead of just "Boot", to clarify
which CPU's information this is? I'm pretty sure you too have hit systems
where firmware doesn't update all cores.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:46:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:46:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723228.1127892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZYy-0001nt-GH; Thu, 16 May 2024 11:46:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723228.1127892; Thu, 16 May 2024 11:46:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZYy-0001nl-DZ; Thu, 16 May 2024 11:46:20 +0000
Received: by outflank-mailman (input) for mailman id 723228;
 Thu, 16 May 2024 11:46:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ZYx-0001nd-RZ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:46:19 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e8135b12-1379-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 13:46:17 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so3259576a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:46:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed0af0sm10403816a12.49.2024.05.16.04.46.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 04:46:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8135b12-1379-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715859977; x=1716464777; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RM3byCZX1/6byX6gI2+BOAfTlRNvy3a3bQJ5HRPzyAM=;
        b=fNrhVQkpRw7mlELruqAfWnd/WnPzT5Y9bFHt5F5Ps98Slyy/8DBUETaeZMrHULHpvO
         ap44keJP3hYgLjPvT/Qw4t28irFUDFrtvwcdRk/MA125v8Wwl6403jq2xfrVf855LZAi
         uoKz6mWxOa+is97dkx3pUin8dBoFUjyBTOR11LGWlNh2QKYQMIg1gPHaFMLswSgBNVHK
         W9j6P+3E2Z55M31b9r+csZ4zykHSzX2kHJeIRUWIENNz/b/1ZjQ+ChXYUjXPyGE/Kfp3
         c6W0NYMRx/a563acn4X5l9dhrMLgVZCNpNBpWTYxkiOz9fz3mC14c4KWC3QzYgtLRL3G
         Hbuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715859977; x=1716464777;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RM3byCZX1/6byX6gI2+BOAfTlRNvy3a3bQJ5HRPzyAM=;
        b=D5nD8IYY00qsx8TPI6eznvcY9ktN2Rh1iftgejvUUMBmAD7q4hi3ofSSpgz8Nv7YsX
         EflB4VPioxLV/YWPR7dK1sevalcPuuHauNsz/tUr/Fd3T+seaon0rFXFW5+EUQo+PxZ1
         luOLGR0o/g12gXeaP60jiKD1wrMPO/nIe37jXN7FVHsqYvUczsCaQwQZWexDoeaJR2qa
         l/5AT9TAG1EpigvQ1AacJFUtOAw0y62Pt9NLvxQxOEc1/IPvXMuY2iN6bjhQljytYLgv
         sEFq6e1thJjYtkjDWjTybPfv33H0h9C9tuSJdQF9u+6bvJ/TSDthFos6X3SQ+sqpNtS0
         mGtg==
X-Gm-Message-State: AOJu0YzNaF6DXULxkGYGH+k7JQ99mcbJ1OQ1EZvd97uA5Waddg78z46B
	yI2rcqu4e8oa7se+mRrhkmcH+FgLXn0lwdXqCCvfho6Y4sqcW0BvHTv/75vzHg==
X-Google-Smtp-Source: AGHT+IHWeBxrUZjQ799dn9Ph9Re16jDB/a0CNvm72JgTdJwNfwOcdgcr9ptusBMCcaeubud65JAuIg==
X-Received: by 2002:a50:d650:0:b0:572:2f0d:f4cb with SMTP id 4fb4d7f45d1cf-5734d5c16eamr11557639a12.1.1715859977330;
        Thu, 16 May 2024 04:46:17 -0700 (PDT)
Message-ID: <6367019b-2683-4422-a4f6-df692b0f6a5e@suse.com>
Date: Thu, 16 May 2024 13:46:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 04/15] x86/p2m: move altp2m-related code to
 separate file
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, Sergiy Kibrik <Sergiy_Kibrik@epam.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <d791a552c3d9331d9b2d40c91e6568932ea6afb7.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151735350.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151735350.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:35, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> Move altp2m code from generic p2m.c file to altp2m.c, so it is kept separately
>> and can possibly be disabled in the build. We may want to disable it when
>> building for specific platform only, that doesn't support alternate p2m.
>>
>> No functional change intended.
>>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
>> CC: Tamas K Lengyel <tamas@tklengyel.com>
>> CC: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Ouch, I'm sorry, Stefano, I failed to put this in while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 11:50:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 11:50:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723230.1127902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZdO-00040h-2B; Thu, 16 May 2024 11:50:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723230.1127902; Thu, 16 May 2024 11:50:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZdN-00040a-VH; Thu, 16 May 2024 11:50:53 +0000
Received: by outflank-mailman (input) for mailman id 723230;
 Thu, 16 May 2024 11:50:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7ZdM-0003z0-NO
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 11:50:52 +0000
Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com
 [2607:f8b0:4864:20::833])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8aaba0b9-137a-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 13:50:51 +0200 (CEST)
Received: by mail-qt1-x833.google.com with SMTP id
 d75a77b69052e-43df44ef3e3so31621331cf.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 04:50:51 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df56b32c8sm95749471cf.77.2024.05.16.04.50.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 04:50:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8aaba0b9-137a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715860250; x=1716465050; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=TmQDnGpTJoPIGtRonZuWFNkU/cBlfrWhnFK1NH1oGFc=;
        b=HTtx+A5by6VlyjSwIS68h6tPdGGBgLh1IjPUJAIVI8jgX2GMysi0A31ZFz2vrnoNiB
         9qAppv11UmZuwfIPKltpbK+kTcRz8AQW4J+qMkH5GtI7GZxvkSz1nfxZzJqsBOX4SGkn
         Up8eMKQF/hKNuuramAA+xEVuBYys6j1eL+dxk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715860250; x=1716465050;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TmQDnGpTJoPIGtRonZuWFNkU/cBlfrWhnFK1NH1oGFc=;
        b=Kybu7vxhFGcRiG9MuJgHcdj5H9KDPG99jFpSK44kKLDm4mmhyu3l6fUWh854Z8+ST1
         ZurKGJYmUY50RG/5qWhCIz3mRpcm5n1AF+hROL0bGC53rDmuz41n4W+MZqVpUyOiabXw
         X9mVA/N2awuCuf6BsyxuSJcKOiCHn4TS5xv4Kl1FGUSuMf61BCkZkzB4UR0scrl0nNxM
         C9G90/WpKjbhM254aTwzKDkE7F0jjFwhARQiyy3LVPhOpkBT0xHRK0kiSmW/crf/Mw80
         TymmzBPr037jKpY8IIGVOEPFCQe8zj3n2MMkW2atb/cEa2tVlI4FoMVo+7r+KfBgtM9G
         jcPw==
X-Gm-Message-State: AOJu0YzEOoUtE+ulhUiMKhThgbeXjkBCIT4Q+4G4CdW4wiYs+z8Hgkfg
	MWNCqaHwB7073XM6A0ERfgG4SO9rThuu7QqOAhjRLcACIBWotRH02AYCKmBUy40=
X-Google-Smtp-Source: AGHT+IFZoYT+RdWncCaxy3+YEbZj7r549kxO+RLlRUpZQkbxFnfUbMqnMSR/SXGO5njo5lOg1FLhNg==
X-Received: by 2002:ac8:5dd0:0:b0:43e:3da7:b39e with SMTP id d75a77b69052e-43e3da7b40amr15101541cf.65.1715860250458;
        Thu, 16 May 2024 04:50:50 -0700 (PDT)
Date: Thu, 16 May 2024 13:50:47 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>,
	Fouad Hilly <fouad.hilly@cloud.com>
Subject: Re: [PATCH] x86/ucode: Further fixes to identify "ucode already up
 to date"
Message-ID: <ZkXzFwbYfFI6DeP8@macbook>
References: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240516113103.3018940-1-andrew.cooper3@citrix.com>

On Thu, May 16, 2024 at 12:31:03PM +0100, Andrew Cooper wrote:
> When the revision in hardware is newer than anything Xen has to hand,
> 'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
> because it doesn't know whether the revisions across the system are symmetric
> or not.  This involves the patch getting all the way into the
> apply_microcode() hooks before being found to be too old.
> 
> This is all a giant mess and needs an overhaul, but in the short term simply
> adjust the apply_microcode() to return -EEXIST.
> 
> Also, unconditionally print the preexisting microcode revision on boot.  It's
> relevant information which is otherwise unavailable if Xen doesn't find new
> microcode to use.
> 
> Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Fouad Hilly <fouad.hilly@cloud.com>
> 
> Sorry Fouad, but this collides with your `--force` series once again.
> Hopefully it might make things fractionally easier.
> 
> Background: For 06-55-04 (Skylake server, stepping 4 specifically), there's a
> recent production firmware update which has a newer microcode revision than
> exists in the Intel public microcode repository.  It's causing a mess in our
> automated testing, although it is finding good bugs...
> ---
>  xen/arch/x86/cpu/microcode/amd.c   | 7 +++++--
>  xen/arch/x86/cpu/microcode/core.c  | 2 ++
>  xen/arch/x86/cpu/microcode/intel.c | 7 +++++--
>  3 files changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
> index 17e68697d5bf..f76a563c8b84 100644
> --- a/xen/arch/x86/cpu/microcode/amd.c
> +++ b/xen/arch/x86/cpu/microcode/amd.c
> @@ -222,12 +222,15 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
>      uint32_t rev, old_rev = sig->rev;
>      enum microcode_match_result result = microcode_fits(patch);
>  
> +    if ( result == MIS_UCODE )
> +        return -EINVAL;
> +
>      /*
>       * Allow application of the same revision to pick up SMT-specific changes
>       * even if the revision of the other SMT thread is already up-to-date.
>       */
> -    if ( result != NEW_UCODE && result != SAME_UCODE )
> -        return -EINVAL;
> +    if ( result == OLD_UCODE )
> +        return -EEXIST;

Won't it be simpler to just add this check ahead of the existing one,
so that you can leave the code as-is, iow:

    if ( result == OLD_UCODE )
        return -EEXIST;

    /*
     * Allow application of the same revision to pick up SMT-specific changes
     * even if the revision of the other SMT thread is already up-to-date.
     */
    if ( result != NEW_UCODE && result != SAME_UCODE )
        return -EINVAL;

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:09:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:09:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723250.1127913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZvJ-0000ro-RK; Thu, 16 May 2024 12:09:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723250.1127913; Thu, 16 May 2024 12:09:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZvJ-0000rg-Oa; Thu, 16 May 2024 12:09:25 +0000
Received: by outflank-mailman (input) for mailman id 723250;
 Thu, 16 May 2024 12:09:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7ZvJ-0000rX-Ex; Thu, 16 May 2024 12:09:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7ZvJ-0002nF-Bo; Thu, 16 May 2024 12:09:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7ZvJ-00049C-15; Thu, 16 May 2024 12:09:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7ZvJ-0002lS-0U; Thu, 16 May 2024 12:09:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nPoq1b5GhqdXbh+bwrb0RHXzk9I4f17b6INFOMYnnFY=; b=Pl4LTg/CrYdVt0NjcMbvcTs66p
	XJOieh9GE41EldVzVOjfccj874EwwBn6+zZ1jQpMJdJiwE2uHHdqiYP+y6hM91ET4moyNG7roX7tW
	rvgb5H7qsCiFRd63qv2LRCmko3sHj2Id7NZs1OyWyUPwnnF5xUJK4Lkohu92vqlX20Ao=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186014-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186014: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6628bacc81f35b41bca18948fe9f405da441906f
X-Osstest-Versions-That:
    xen=762a848223e0fd61aa87b4e040c83dd2c490fb79
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 12:09:25 +0000

flight 186014 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186014/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  6628bacc81f35b41bca18948fe9f405da441906f
baseline version:
 xen                  762a848223e0fd61aa87b4e040c83dd2c490fb79

Last test of basis   186009  2024-05-16 01:00:26 Z    0 days
Testing same since   186014  2024-05-16 09:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Rahul Singh <rahul.singh@arm.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   762a848223..6628bacc81  6628bacc81f35b41bca18948fe9f405da441906f -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:11:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:11:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723255.1127923 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZxP-0002iu-75; Thu, 16 May 2024 12:11:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723255.1127923; Thu, 16 May 2024 12:11:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ZxP-0002in-4K; Thu, 16 May 2024 12:11:35 +0000
Received: by outflank-mailman (input) for mailman id 723255;
 Thu, 16 May 2024 12:11:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ZxN-0002iZ-Dj
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:11:33 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6de47f9f-137d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 14:11:30 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59b58fe083so268977966b.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:11:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdc72d31fsm139788766b.169.2024.05.16.05.11.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:11:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6de47f9f-137d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715861490; x=1716466290; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=R96vRc0eklM1hVZMcOU4eHuMkP4p7QxrjGnDHYjU0P4=;
        b=EMFxF/RyipKHyb4+g/RjjH1q7cYIdNWvCEQuT/U3F0Ek2Tf8XS9PaTtM1VIcm7NIBw
         yi1hxOZqLP7MdV6CtUMYHo7eqzkdyRBXkc6ntH2B3YEEpFGnHtln+b2QUUJcMYkFwOqb
         L2vV5iziH2CausY5YtrfdC+sFghCd9AowV6P6q0jCzGlzHensaDCrnWubQ+3H1XugePn
         8gL76ltcGU1cHXrq3QWcRrrhcUz+7mkzYcgBbssdmFPgTJi0BWdEd0PGJn3D97JVt8M5
         3E8/OwAQT2ScuvSlEhCE8tqTfJOeE7FRpEqS1id7eKf4CHlFXaJHrnA2nYVSlqUnWT/a
         6S2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715861490; x=1716466290;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=R96vRc0eklM1hVZMcOU4eHuMkP4p7QxrjGnDHYjU0P4=;
        b=DEMqJubp0/ELVOycyWjgIV1XA4CIgbrHix8CbPiKQUSvd5t7VkNbtDtl0w3TosO/b1
         5yDNilkTRhSCUEd78stACwpVCVpmV5NUQ/bOqdRJefIBD+psfuCbFa7t24XQ/sM4g3w7
         bdpMQhHvqA9CFdqG+2QPBYbCvd0efjtF8Z67L4Q6K3GIBKI5LOEDOiBxfAnIV/Qo7DWi
         /0WFL7XMMrUO95FU/8YGwi0mADQq7B/TN4RzqWDWqmR9vI+j+hUMDztkMbdHZiwUHVYK
         2WnWQ32mlV2IlKcp/5GcBJ55PAHc9GgTR0ckMU3z+SfM0OegwjvGUarUP0x97ldeNhDW
         JUhg==
X-Forwarded-Encrypted: i=1; AJvYcCXzr25z6V3UxmTksRfLuLXGTPvisMtlrIlyxHJeRPKfljTyTEKVIKDT5Xe01zyrk3MU7homBBsovqgtRoQswuo4xNRSf7PJLuyMN2uCK6I=
X-Gm-Message-State: AOJu0Yyn8LrN1nIS1F8Y2w5EGuxrGZIlNdBPDRfusconI5wnDvj7CdMM
	6FDFmIeDjwDJgJR0X6oAIIH9S4Tty7jEGWgf3yM8w3OJBcTpBqXLNiUHmSiQoQ==
X-Google-Smtp-Source: AGHT+IFgS54du9FRDlAGTvmTet+3Ty52KD6gyAISMPQkRYPqyBA4GK/osw3+1HqaYd9Ga2zmxY/8yw==
X-Received: by 2002:a17:906:ae8b:b0:a59:cb09:41d6 with SMTP id a640c23a62f3a-a5a2d65f294mr1113835766b.49.1715861490343;
        Thu, 16 May 2024 05:11:30 -0700 (PDT)
Message-ID: <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
Date: Thu, 16 May 2024 14:11:28 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 13/15] x86/ioreq: guard VIO_realmode_completion
 with CONFIG_VMX
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Xenia Ragiadakou <burzalodowa@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:24, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/hvm/emulate.c
> +++ b/xen/arch/x86/hvm/emulate.c
> @@ -2667,7 +2667,9 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
>          break;
>  
>      case VIO_mmio_completion:
> +#ifdef CONFIG_VMX
>      case VIO_realmode_completion:
> +#endif
>          BUILD_BUG_ON(sizeof(hvio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
>          hvio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
>          memcpy(hvio->mmio_insn, hvmemul_ctxt->insn_buf, hvio->mmio_insn_bytes);

This change doesn't buy us anything, does it?

> --- a/xen/arch/x86/hvm/ioreq.c
> +++ b/xen/arch/x86/hvm/ioreq.c
> @@ -33,6 +33,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion completion)
>  {
>      switch ( completion )
>      {
> +#ifdef CONFIG_VMX
>      case VIO_realmode_completion:
>      {
>          struct hvm_emulate_ctxt ctxt;
> @@ -43,6 +44,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion completion)
>  
>          break;
>      }
> +#endif
>  
>      default:
>          ASSERT_UNREACHABLE();

And while this change is needed for the goal of the series, I wonder whether
it wouldn't better be arch_vcpu_ioreq_completion() as whole that then gets
stubbed out.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:15:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:15:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723258.1127932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a1Z-00042u-NF; Thu, 16 May 2024 12:15:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723258.1127932; Thu, 16 May 2024 12:15:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a1Z-00042n-Kk; Thu, 16 May 2024 12:15:53 +0000
Received: by outflank-mailman (input) for mailman id 723258;
 Thu, 16 May 2024 12:15:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7a1Z-00042h-7W
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:15:53 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0923fe4b-137e-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 14:15:51 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-572e8028e0cso3318623a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:15:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5750d24c8c1sm559715a12.72.2024.05.16.05.15.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:15:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0923fe4b-137e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715861751; x=1716466551; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vMEbeCF6PHmkPkmy8h6WYnlrDVpl+7aVfReOgi2GNOU=;
        b=gba97ilcJ6iY68r5EFLt4Q+EsirJBSGiwWbu8VEiEeFnGS40MKKUDwPhad6+d0A2md
         ee8uSAprQ/ftR8ChqT5SHdTsizpbiTKgeTmKelOLLDm0et/piMmz892rexOOlXYkWjZZ
         lPg/aFzRH4dYWRPmMnoxwAa/yVTFmZk/pESbx8WRLwI1XUnvjl32N/PYvZqpIfK3gZfc
         cL/eYAVIJItflDiz5R1d/YZSBzEhm1CuKQh/oqilzidybuZ9H1pAiO4udI/HQ8hbQtci
         2Ca5PB8zKO1AzvEeNrj0ZmMQLW4r2luhjHamYanG4bSuBZwr3V+cXGpdTXFCIuOzYozt
         A6+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715861751; x=1716466551;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vMEbeCF6PHmkPkmy8h6WYnlrDVpl+7aVfReOgi2GNOU=;
        b=Bx+sO963RphrS2qg2yeIFq2TAG+ms8zY1teLu+SfPhfsdUpIJDXr2ezkwAH7wDVHzd
         7e2JELNtOaShKViFDKax05aN7pESWf6LQMjv+hmy31LKsk5gEthhGOu9QX6y9WhG88wM
         m6/YFLuqQQ6z7SnCHK8mcy99mIjghDOPKGc4kF1KVDtCoxoffxigghBb6VJR48EiTHH/
         LGTRIzYNplKagU3725BCa+CGef5/7z1xlvEdEomul2KrwxD5Z5K4BYItpOt64JVhy8DT
         Vo47rG5aRicTrpHLRSwKR/nAgtMjFd79bqviLxxa8LT9MVIwSwJ2+PDu9o60nPI6+0AD
         gwIQ==
X-Forwarded-Encrypted: i=1; AJvYcCVA+Z60bEZgl23vzZ6xJ4LUWUSLhTWInnCv1PEzBkBBBBogKmnAhaCJUa4TE7KK/7tS1QUh00QXMZ13KJRlXuJe4boP0xVpvmN8caMBQKA=
X-Gm-Message-State: AOJu0YwU/hhrd7WulF2oXzZPTSw9B5ClSyHBWE8svBD/KzAwX7f14MXt
	TyFN7ombqNPLe2IqfNCwIHkpc9PmtHORj4fp8/PFgCwyN0ngtAjXVvsQamuYQQ==
X-Google-Smtp-Source: AGHT+IH3UOx9xjFVRVc8cnJmjwmPjB0cVB3acOyVHMSucv1SWNCFcMwBeirw1iiSKYMvP796Ws6UTw==
X-Received: by 2002:a50:8d18:0:b0:574:ebfb:6d93 with SMTP id 4fb4d7f45d1cf-574ebfb6ddemr4478343a12.10.1715861750790;
        Thu, 16 May 2024 05:15:50 -0700 (PDT)
Message-ID: <c43a554e-4b21-4a3b-92f4-60633f61f67e@suse.com>
Date: Thu, 16 May 2024 14:15:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 14/15] iommu/vt-d: guard vmx_pi_hooks_* calls with
 cpu_has_vmx
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.05.2024 11:26, Sergiy Kibrik wrote:
> VMX posted interrupts support can now be excluded from x86 build along with
> VMX code itself, but still we may want to keep the possibility to use
> VT-d IOMMU driver in non-HVM setups.
> So we guard vmx_pi_hooks_{assign/deassign} with some checks for such a case.

But both function already have a stub each. Isn't is merely a matter of
changing when those stubs come into play?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:16:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:16:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723262.1127943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a1x-0004Xz-Ur; Thu, 16 May 2024 12:16:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723262.1127943; Thu, 16 May 2024 12:16:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a1x-0004Xq-S5; Thu, 16 May 2024 12:16:17 +0000
Received: by outflank-mailman (input) for mailman id 723262;
 Thu, 16 May 2024 12:16:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7a1w-0004Se-QG
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:16:16 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1766a1fd-137e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:16:15 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-574b3d6c0f3so3379228a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:16:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1766a1fd-137e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715861775; x=1716466575; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Yn+oXlbgVr7a9k+BINSVHmajhWBEJ0sHm25nDF+ijag=;
        b=Bf2AdiSCkzCVv4mwrKDDUmDi3sCamPTf3YSdpJNDyQf5PeZNUBY9hFS84M1CKPTvtv
         3W3coyWkrA7GcCr82TGn0bXKCP3CUJcBW0/BHWUklqKiOFuY7+s/9eRb3f02OrqxfmO0
         U3kJk2w48eYbdF/dU2H216brBzPDlHIc1ErSqopNtOryvOH8EOkOy5nRyV7H+s7Vrlx+
         rNd+7b2mWp7wCvz/KdbBO7xE/vW432DVxQVIkTqdzoq5JG6A+uM2S05ruoiUuL12ZoJz
         FPAkworUhZv1yVIq4lpj4h22Dnw6OYXvQoZSPmElLykLfd/LvkjO+PegpHPIC+wrpIgk
         OLjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715861775; x=1716466575;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Yn+oXlbgVr7a9k+BINSVHmajhWBEJ0sHm25nDF+ijag=;
        b=wTz9DEfFVOLe8ZaIua3bNUvqrEkCOfb6SjcvhxXvy35r8Ny0F+kF68kz88Tg5bft+f
         g5oWO+jI9VnFRSx0a9ejgBjV8xkwIqc4xJafXrdZIxiDqfytWZAuX7JwixrIEvQT5aOu
         VBBmNFIbtYKybxUvVKixRqSoAIm+aUBDiuTgUz93NmFZA0bakUuGQpSvCmumXuQMRGZy
         F+fOl94gUZFlpla+Z+SxwdG98WNkxRtF+c6JPo/AXgIH999Vy+vG5xTRdC3kkWLH4T9Y
         +4ad5nW4xU9KuZGtIU9HR7dzjWl1LsQ7kBIOweGxvmQ/PFTBIMftAuZUc63Zzbp8fQNc
         pw+Q==
X-Forwarded-Encrypted: i=1; AJvYcCWAK7PTvQBHLLm6EqXdYMF6D2sQPl50HlIxrhu8RSoW6qQGA0YtQfjf1q+l+hX7vTArOzCgf/GhbC5vgi6wsg/PqxCiJiD+oG8jDqozsOg=
X-Gm-Message-State: AOJu0Yw+r18xFM3NkOC0Lybp6UUIbRqqaueCRstg8TV64ZhMR5EvYGuT
	dkNg+NyUmTx6YkvA28g4wteMHgRmYyc9mSsFzgvO0tpo89N6vliz2GfXl3JLZX+GbLf4jODQ57X
	6vKnKc4+fsBW/sz5Skh5UinFK61s=
X-Google-Smtp-Source: AGHT+IFnT5fCN2Q3WSyadS6/AHfALVLYnoDOnQyI+y3w4CmbQMi4Z9f5oXw7pPPeYA3fC3u8vr/OEv8Pt+tb1RiaPk8=
X-Received: by 2002:a50:d496:0:b0:572:3bcc:8370 with SMTP id
 4fb4d7f45d1cf-5734d4417b4mr11339654a12.0.1715861774886; Thu, 16 May 2024
 05:16:14 -0700 (PDT)
MIME-Version: 1.0
References: <20240503014449.1046238-1-edgar.iglesias@gmail.com>
 <20240503014449.1046238-7-edgar.iglesias@gmail.com> <alpine.DEB.2.22.394.2405151605550.2544314@ubuntu-linux-20-04-desktop>
In-Reply-To: <alpine.DEB.2.22.394.2405151605550.2544314@ubuntu-linux-20-04-desktop>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 16 May 2024 14:16:03 +0200
Message-ID: <CAJy5ezoB+MZDhPN0iWcTTTy_ufdVwK1qOpU-5M4aJ+Ujfv9TQg@mail.gmail.com>
Subject: Re: [PATCH v5 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: qemu-devel@nongnu.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony PERARD <anthony@xenproject.org>, 
	Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
	David Hildenbrand <david@redhat.com>, =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 16, 2024 at 1:08=E2=80=AFAM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> On Fri, 3 May 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Pass the ram_addr offset to xen_map_cache.
> > This is in preparation for adding grant mappings that need
> > to compute the address within the RAMBlock.
> >
> > No functional changes.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > ---
> >  hw/xen/xen-mapcache.c         | 16 +++++++++++-----
> >  include/sysemu/xen-mapcache.h |  2 ++
> >  system/physmem.c              |  9 +++++----
> >  3 files changed, 18 insertions(+), 9 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index ec95445696..26bc38a9e3 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
> >                               void *vaddr,
> >                               hwaddr size,
> >                               hwaddr address_index,
> > -                             bool dummy)
> > +                             bool dummy,
> > +                             ram_addr_t ram_offset)
> >  {
> >      uint8_t *vaddr_base;
> >      xen_pfn_t *pfns;
> > @@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
> >
> >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> >                                         hwaddr phys_addr, hwaddr size,
> > +                                       ram_addr_t ram_offset,
> >                                         uint8_t lock, bool dma, bool is=
_write)
> >  {
> >      MapCacheEntry *entry, *pentry =3D NULL,
> > @@ -337,14 +339,16 @@ tryagain:
> >      if (!entry) {
> >          entry =3D g_new0(MapCacheEntry, 1);
> >          pentry->next =3D entry;
> > -        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, d=
ummy);
> > +        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, d=
ummy,
> > +                         ram_offset);
> >      } else if (!entry->lock) {
> >          if (!entry->vaddr_base || entry->paddr_index !=3D address_inde=
x ||
> >                  entry->size !=3D cache_size ||
> >                  !test_bits(address_offset >> XC_PAGE_SHIFT,
> >                      test_bit_size >> XC_PAGE_SHIFT,
> >                      entry->valid_mapping)) {
> > -            xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy);
> > +            xen_remap_bucket(mc, entry, NULL, cache_size, address_inde=
x, dummy,
> > +                             ram_offset);
> >          }
> >      }
> >
> > @@ -391,13 +395,15 @@ tryagain:
> >
> >  uint8_t *xen_map_cache(MemoryRegion *mr,
> >                         hwaddr phys_addr, hwaddr size,
> > +                       ram_addr_t ram_addr_offset,
> >                         uint8_t lock, bool dma,
> >                         bool is_write)
> >  {
> >      uint8_t *p;
> >
> >      mapcache_lock(mapcache);
> > -    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma,=
 is_write);
> > +    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_o=
ffset,
> > +                               lock, dma, is_write);
> >      mapcache_unlock(mapcache);
> >      return p;
> >  }
> > @@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(Ma=
pCache *mc,
> >      trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> >
> >      xen_remap_bucket(mc, entry, entry->vaddr_base,
> > -                     cache_size, address_index, false);
> > +                     cache_size, address_index, false, new_phys_addr);
>
> Everything else makes sense, but I don't understand how can it be that
> new_phys_addr is the block->offset here?
>

Agreed, this should be old_phys_addr propagated via phys_offset from
xen_add_to_physmap().
The reason this didn't have an effect here is that we currently only
use the ram_addr offset for grants,
and xen_replace_cache_entry is never called for grants. Anyway, I'll
fix this in the next version!




>
> >      if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> >                  test_bit_size >> XC_PAGE_SHIFT,
> >                  entry->valid_mapping)) {
> > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcach=
e.h
> > index 1ec9e66752..b5e3ea1bc0 100644
> > --- a/include/sysemu/xen-mapcache.h
> > +++ b/include/sysemu/xen-mapcache.h
> > @@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_=
offset,
> >  void xen_map_cache_init(phys_offset_to_gaddr_t f,
> >                          void *opaque);
> >  uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size=
,
> > +                       ram_addr_t ram_addr_offset,
> >                         uint8_t lock, bool dma,
> >                         bool is_write);
> >  ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
> > @@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_=
gaddr_t f,
> >  static inline uint8_t *xen_map_cache(MemoryRegion *mr,
> >                                       hwaddr phys_addr,
> >                                       hwaddr size,
> > +                                     ram_addr_t ram_addr_offset,
> >                                       uint8_t lock,
> >                                       bool dma,
> >                                       bool is_write)
> > diff --git a/system/physmem.c b/system/physmem.c
> > index b7847db1a2..33d09f7571 100644
> > --- a/system/physmem.c
> > +++ b/system/physmem.c
> > @@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *bloc=
k, ram_addr_t addr,
> >           */
> >          if (xen_mr_is_memory(block->mr)) {
> >              return xen_map_cache(block->mr, block->offset + addr,
> > -                                 len, lock, lock,
> > -                                 is_write);
> > +                                 len, block->offset,
> > +                                 lock, lock, is_write);
> >          }
> >
> >          block->host =3D xen_map_cache(block->mr, block->offset,
> > -                                    block->max_length, 1,
> > -                                    lock, is_write);
> > +                                    block->max_length,
> > +                                    block->offset,
> > +                                    1, lock, is_write);
> >      }
>
> This is OK but it is really making it clear that there is a mistake in
> the code. OK for now but it is something to fix in the future.
>
>
> >      return ramblock_ptr(block, addr);
> > --
> > 2.40.1
> >


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:17:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:17:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723268.1127953 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a3E-0005ZW-C5; Thu, 16 May 2024 12:17:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723268.1127953; Thu, 16 May 2024 12:17:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7a3E-0005ZP-8v; Thu, 16 May 2024 12:17:36 +0000
Received: by outflank-mailman (input) for mailman id 723268;
 Thu, 16 May 2024 12:17:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7a3D-0005ZH-26
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:17:35 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 45ce19e4-137e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:17:33 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so319152266b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:17:33 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f32sm981391066b.106.2024.05.16.05.17.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:17:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45ce19e4-137e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715861853; x=1716466653; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KdmzI0pblVMHjBethUop9xBQnLciYn98soZ8P6ketGo=;
        b=b8GLsb3JnRpSHf7KuP89TDXX0Jk4FJ8eOChJ4oUdXTLBSLedXyzIQfVx06BuGOTosq
         VhKG67g/nVR51KJ8JdKWUcKqO00l992zvvBT0LQEaslSKNMWhxFW3eS/WxUVCjsFGJam
         gxzGQMQt+5YWQ34F5xBtS0JppC6o4cCVgq3t1jcuYxL28/vPj94mvZd4a63ZFVsM6VBJ
         PMgZ0mtE0iRqqjVrxUDqabrdy4zu8OuXWQ3QKQ5Vgy6vCWOr/yG+UENm69RbkS30Xs+r
         BQdehxpBNIjQ2Lm7fKMlWDpeO3NMgtyTNQp4T8Mr3ZgkXafQ8WqpCgx8Ya+qdq2I2vCv
         3CGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715861853; x=1716466653;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KdmzI0pblVMHjBethUop9xBQnLciYn98soZ8P6ketGo=;
        b=FGdUm2tom0Q239NtHJend4O16t4QO5+1uOd9HbtLTT02FMOckO3EhBZ99xImuUP2fa
         /OGCC1nylaoFuKeFUMjze1z0dVZKv3bsUAFYkuCHMbB+X5jHK4SFvFRzeOw18WStuV+W
         ZcHRR92ikVH0yFmav71iZ7gqO5lvzdI20hZSxk8plLBODlcEXQhq28S5JdK7xqdHG64n
         ZKx5QJmVdLtZyRC/OPlhdQ89zfBKQVmm9DH6BAc97h5wjRMIoGVfadfIvI4Sgnsf+8jK
         fK6gJXNnsyVWrLlaGkF4O1koMHXcVatTf47tntR8bSgUQpv+v2ONNhPgzUssoFm1O5m5
         n+9A==
X-Gm-Message-State: AOJu0YybcMXv1h39kLxkVtV7oMbW+tJXurxZbfphlDnIIhHsbpPazDMC
	KXtpx3/+qFYvvWg79FrupaRBu6MKvxlemFvYx+wyHgHSa4rFFoBwzO3kMQqrEw==
X-Google-Smtp-Source: AGHT+IFqJ0x2u1Q10KzSjaxa4Awg/iKZLxNthUfs22g+aaesWeFD2P5/DXKD4NBE+4zcUw5KOqz8wg==
X-Received: by 2002:a17:906:b081:b0:a5a:5c98:c594 with SMTP id a640c23a62f3a-a5a5c98c636mr846301766b.65.1715861853075;
        Thu, 16 May 2024 05:17:33 -0700 (PDT)
Message-ID: <13bbced0-a704-48c5-83e0-dbf46e2783ee@suse.com>
Date: Thu, 16 May 2024 14:17:31 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 15/15] x86/hvm: make AMD-V and Intel VT-x support
 configurable
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, Xenia Ragiadakou <burzalodowa@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <3ad7c0279da67e564713140fb5b247349cf4dccc.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151754420.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151754420.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 02:54, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>> From: Xenia Ragiadakou <burzalodowa@gmail.com>
>>
>> Provide the user with configuration control over the cpu virtualization support
>> in Xen by making SVM and VMX options user selectable.
>>
>> To preserve the current default behavior, both options depend on HVM and
>> default to value of HVM.
>>
>> To prevent users from unknowingly disabling virtualization support, make the
>> controls user selectable only if EXPERT is enabled.
>>
>> No functional change intended.
>>
>> Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu May 16 12:29:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:29:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723272.1127963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aEo-0000dt-Bv; Thu, 16 May 2024 12:29:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723272.1127963; Thu, 16 May 2024 12:29:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aEo-0000dm-9M; Thu, 16 May 2024 12:29:34 +0000
Received: by outflank-mailman (input) for mailman id 723272;
 Thu, 16 May 2024 12:29:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7aEn-0000de-17
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:29:33 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0779cfc-137f-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 14:29:28 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so293598366b.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:29:28 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894eb1sm978064266b.77.2024.05.16.05.29.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:29:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0779cfc-137f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715862568; x=1716467368; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fXVS35DRWL5MxvMZYr1vkxLTGtIPJS670c3yyt/vKK0=;
        b=bI1KsyOyfLUpHDeUi3tj6kF0BrcqF0O3FFaQ2KgyzwLApoRXBAju6whGkcLeBCzJlA
         o2vSJt7eT1X/hvcYvH9yNMJA3BfRWKvfSnuka5/8OWuxAGmHRFWRw0fgkEmaD1MSJVV+
         9welR4DY8tvXoWGSmf5nbgL07LrYLIeYzyjB8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715862568; x=1716467368;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=fXVS35DRWL5MxvMZYr1vkxLTGtIPJS670c3yyt/vKK0=;
        b=AKxLn9N3QO/2AXnLUkUlr9B7o9Cx6sojkVYeLHxuDKx8Oj5FjFC3u3l/YJa8AhlJGg
         VVqnKBNSpALCLOYDkn/S9TtK1QMCvHHxaF+wg6ZNd6Gr8KOMeU9acqIO5U6qOiII76yM
         mYkMAIQKRLeRanzSZHnf+ijivrB70xg2QZkzunuPjbtSDPKbFsTM7COq0blwAB6Burgz
         7aveaI4d7kRFKcgfYKP7X/9pgUPL4JTRgo1XwlpslyB7x+M8DzFmz6w6Ebi0DdCQJpcw
         9tqQN+OtSkDsrx0KNZxhnQvnRRj94oxjnbKucbpJxVRkZX/YUPLaA5GhNCMb1NS0eBP5
         IKrQ==
X-Gm-Message-State: AOJu0Yy1BaTchC48oZskFKeehkIo0bHqn0hdVnr84zYFd4bb7nsMHkqW
	/hYc0D59xs1rj9+VR8akdaRFCu/R+WDwy+G+7MpnjSEfwEAGwtLC1yHMm9IdEfo=
X-Google-Smtp-Source: AGHT+IEEk79PY6c/JBYBCzUGCzWhh0AMdXMCJqxePBrDc+Qb498ep3AZaoNlKRFcszH+TMj18PXVCQ==
X-Received: by 2002:a17:906:1358:b0:a59:9eab:1622 with SMTP id a640c23a62f3a-a5a2d665e06mr1183512266b.56.1715862568328;
        Thu, 16 May 2024 05:29:28 -0700 (PDT)
Message-ID: <7965ba74-9177-44b7-9253-7ab5a77fd88f@cloud.com>
Date: Thu, 16 May 2024 13:29:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
 <ZkXvdWf2mVkPW458@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZkXvdWf2mVkPW458@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16/05/2024 12:35, Roger Pau Monné wrote:
> On Thu, May 16, 2024 at 12:07:10PM +0100, Alejandro Vallejo wrote:
>> Bring test_x86_emulator in line with other tests by adding
>> install/uninstall rules.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>>  tools/tests/x86_emulator/Makefile | 11 +++++++++--
>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
>> index 834b2112e7fe..30edf7e0185d 100644
>> --- a/tools/tests/x86_emulator/Makefile
>> +++ b/tools/tests/x86_emulator/Makefile
>> @@ -269,8 +269,15 @@ clean:
>>  .PHONY: distclean
>>  distclean: clean
>>  
>> -.PHONY: install uninstall
>> -install uninstall:
>> +.PHONY: install
>> +install: all
>> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
>> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
>> +
>> +.PHONY: uninstall
>> +uninstall:
>> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
>> +
> 
> FWIW, should you check that HOSTCC == CC before installing?  Otherwise
> I'm unsure of the result in cross-compiled builds, as the x86_emulator
> is built with HOSTCC, not CC.
> 
> Thanks, Roger.

Right...

More generally, should we do s/CC/HOSTCC/ on all compiler checks? I see
no particular reason to do them on $(CC) rather than the actual compiler
used during build.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:30:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:30:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723275.1127973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aFd-00029V-L1; Thu, 16 May 2024 12:30:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723275.1127973; Thu, 16 May 2024 12:30:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aFd-00029O-IR; Thu, 16 May 2024 12:30:25 +0000
Received: by outflank-mailman (input) for mailman id 723275;
 Thu, 16 May 2024 12:30:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7aFb-00010i-Ok
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:30:23 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 10bf3634-1380-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:30:23 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59c5c9c6aeso308664666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:30:23 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d6dsm980953966b.71.2024.05.16.05.30.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:30:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10bf3634-1380-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715862622; x=1716467422; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=JdK0drs0oIPnJU9lfAwWPoE+YMXHxQGT1xqLV4ohw2k=;
        b=PjTyCQ6NzberdIr17Ppenq+bbA4IBcPA361TOsq/T83xE/yKa4hOLXay2js/rO+OHY
         njEylUBdqdf0sGKiWPQGSvMRvJ9Ix+5mKIL0x60JuP5VDZsrE2N7uc1JugaFH8WxPNlB
         aOXggOmrPX27Lvrv7i0SJz08Tanjx5AAGhU7g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715862622; x=1716467422;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JdK0drs0oIPnJU9lfAwWPoE+YMXHxQGT1xqLV4ohw2k=;
        b=IpcJbjNBMUhPTCHxBd6baMMbSW/pkqfllexhdrXa/rmLZtXNvW1rHMbE+bBmDSvLvC
         07kob10q6OtRetdrYNv/Ocmb/gRyN8Ndnt+ZK++Z7zsNX6OQGnyLEmP4zgreFpn+Uc1v
         f+xptHzeFmp+a1iiIGGvwGGP6OS82OhvxhLQCayhIkfHtkPlcjffaK5T0K1hYsQIy2iG
         2D6EEkJ2dIWGYvvcx/n2enC1EjiE0FVSj/UoKt970bZvXtw/SecOF+ShQGBrzcGBNYL5
         6biJ2+JTMNvIo2f70lKuqXmokDYQ/qU78s3AZWOCMAxVx82bZXzinoGgVJwWe9OWbBqK
         Tw+Q==
X-Gm-Message-State: AOJu0YzLy5PuxZeeUYi3gr3VxHdSkpAWcCepGpA+wECUudmTOiosa9rc
	9KNlCyvSxnBBXVm/KBaRDW10YKt8TlUom1yWpNuQyeMa1hY6POSJ/5VPXIKFAfM=
X-Google-Smtp-Source: AGHT+IFC8scBM6GPoDJ5CrF/425mQ4aCv1Lon2Rw/uSwLOWMtvf/VoOxuxEwtzpva2aiQQ8U863ERQ==
X-Received: by 2002:a17:907:7da3:b0:a58:e3d9:e2d6 with SMTP id a640c23a62f3a-a5a2d672f76mr1592713266b.56.1715862622490;
        Thu, 16 May 2024 05:30:22 -0700 (PDT)
Message-ID: <4bc4f2f5-6914-445a-a9a1-a609c0c3cf1f@citrix.com>
Date: Thu, 16 May 2024 13:30:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/ucode: Further fixes to identify "ucode already up to
 date"
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>, Fouad Hilly <fouad.hilly@cloud.com>
References: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
 <ZkXzFwbYfFI6DeP8@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkXzFwbYfFI6DeP8@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16/05/2024 12:50 pm, Roger Pau Monné wrote:
> On Thu, May 16, 2024 at 12:31:03PM +0100, Andrew Cooper wrote:
>> When the revision in hardware is newer than anything Xen has to hand,
>> 'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
>> because it doesn't know whether the revisions across the system are symmetric
>> or not.  This involves the patch getting all the way into the
>> apply_microcode() hooks before being found to be too old.
>>
>> This is all a giant mess and needs an overhaul, but in the short term simply
>> adjust the apply_microcode() to return -EEXIST.
>>
>> Also, unconditionally print the preexisting microcode revision on boot.  It's
>> relevant information which is otherwise unavailable if Xen doesn't find new
>> microcode to use.
>>
>> Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Fouad Hilly <fouad.hilly@cloud.com>
>>
>> Sorry Fouad, but this collides with your `--force` series once again.
>> Hopefully it might make things fractionally easier.
>>
>> Background: For 06-55-04 (Skylake server, stepping 4 specifically), there's a
>> recent production firmware update which has a newer microcode revision than
>> exists in the Intel public microcode repository.  It's causing a mess in our
>> automated testing, although it is finding good bugs...
>> ---
>>  xen/arch/x86/cpu/microcode/amd.c   | 7 +++++--
>>  xen/arch/x86/cpu/microcode/core.c  | 2 ++
>>  xen/arch/x86/cpu/microcode/intel.c | 7 +++++--
>>  3 files changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
>> index 17e68697d5bf..f76a563c8b84 100644
>> --- a/xen/arch/x86/cpu/microcode/amd.c
>> +++ b/xen/arch/x86/cpu/microcode/amd.c
>> @@ -222,12 +222,15 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
>>      uint32_t rev, old_rev = sig->rev;
>>      enum microcode_match_result result = microcode_fits(patch);
>>  
>> +    if ( result == MIS_UCODE )
>> +        return -EINVAL;
>> +
>>      /*
>>       * Allow application of the same revision to pick up SMT-specific changes
>>       * even if the revision of the other SMT thread is already up-to-date.
>>       */
>> -    if ( result != NEW_UCODE && result != SAME_UCODE )
>> -        return -EINVAL;
>> +    if ( result == OLD_UCODE )
>> +        return -EEXIST;
> Won't it be simpler to just add this check ahead of the existing one,
> so that you can leave the code as-is, iow:
>
>     if ( result == OLD_UCODE )
>         return -EEXIST;
>
>     /*
>      * Allow application of the same revision to pick up SMT-specific changes
>      * even if the revision of the other SMT thread is already up-to-date.
>      */
>     if ( result != NEW_UCODE && result != SAME_UCODE )
>         return -EINVAL;
>
> Thanks, Roger.

Not really, no.  That still leaves this piece of logic which is
misleading IMO.

MIS_UCODE is the only -EINVAL worthy case.

Every other *_UCODE constant needs to be 0 or -EEXIST, depending on
allow-same/--force.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu May 16 12:31:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:31:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723278.1127983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aGK-0002y0-UM; Thu, 16 May 2024 12:31:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723278.1127983; Thu, 16 May 2024 12:31:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aGK-0002xt-Qh; Thu, 16 May 2024 12:31:08 +0000
Received: by outflank-mailman (input) for mailman id 723278;
 Thu, 16 May 2024 12:31:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7aGJ-0002vB-Tt
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:31:07 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 292ba30a-1380-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 14:31:04 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a599a298990so341816566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:31:04 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8b634cdcsm290995666b.93.2024.05.16.05.31.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:31:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 292ba30a-1380-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715862663; x=1716467463; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=J6Jl3scWQzYKH3aqoYM+MbdxOBcOQosmLLJsxmjmqf0=;
        b=L1SmPFzptd1OCCvjQjdEOt+u/Dmwhjt9pREWDd/QFCbCHOFE23ia0uDvaXFvZ/dbXM
         65ROdiv2XuFeXs4xPzh2jpqiNMhR7CIQ0UtNp8NwEIBj9Y/76vg0eacHkPGLH8ko2pLv
         wEFPv4qtJE4bdFNcDS9iShD+7ZGcmGVfbiYTdgR2W9bkm/NV4+l49axqOriW/xQJEBFX
         Mcq0iQmaUBy2kzMupiZVAgcn9lPEmSkSirrHsW4bPwZv2npbVtek6q+/9aBaO+lC9q4k
         /4Zb6pozYJclfxuFugP+8ZzvZW/wjG+vxzieNbM0zlGiysUTgoKButzy+BGlAs7ywzPj
         CEUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715862663; x=1716467463;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=J6Jl3scWQzYKH3aqoYM+MbdxOBcOQosmLLJsxmjmqf0=;
        b=F1Lqfon4DUC3IW+rcXxrKZkDZWvep+If/ict0npGOweX8ZWO8Nv9yGGuF3GQbDzpka
         UDDvtzmt10H4RMF5gG9vFCBdVY+qTGdigxTFn21Yf18sFl4ePcije3B5cQcjWL6hMBww
         G7TLKlTWjMkPS2Go4u/vR0EZY7wCtV9E4pr26nyeHjQ9kZHmTQCcwUxPIWoIGdSTojNC
         soQ43nXf/Rlg735Ae4+DGazGOeLObsRNKPbIT17+ftVKdg+YynYQZ6BRKCQvXpsq08mp
         U8CNcWKSkSpE0O3+FOoChSNcCd43zX9Lg1dSp+pfbQ+GlF7OtDbiXZImwqP70oSjCw3E
         mHqA==
X-Forwarded-Encrypted: i=1; AJvYcCWQksVrvLym/W36zuuCRxwShjSZBTcWUP9xKS8q9JX7OURKtdFbKGJNWOFu7qedJ6jLYOIvRDr2mkOsOGZ4DQ1Ehbk26/x6uDz8BoER01I=
X-Gm-Message-State: AOJu0Yyl0MEpxyep7tt5LYRZFnEGm3kFWYifmKxMPF0RGAQfg4UZaeBl
	Cib8mU/yqpBoBKQhI03agVK8IYy2GfZT1KSqQvDmJs+kPOg9bBAP6NPom2M8SA==
X-Google-Smtp-Source: AGHT+IGBEN96TpjIslGxdcss8E719zdXnuxhhMxmdiylrd3LuaRUjFkCajtn/0gnm4fqIx/h2OJCmw==
X-Received: by 2002:a17:906:34c7:b0:a5a:4683:e961 with SMTP id a640c23a62f3a-a5a4683eaf3mr975258966b.52.1715862663448;
        Thu, 16 May 2024 05:31:03 -0700 (PDT)
Message-ID: <28138e7d-26c7-4faa-86f1-a1c7b175114b@suse.com>
Date: Thu, 16 May 2024 14:31:02 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and
 related operations
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-7-xin.wang2@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516100330.1433265-7-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 12:03, Henry Wang wrote:
> +static long handle_detach_overlay_nodes(struct domain *d,
> +                                        const void *overlay_fdt,
> +                                        uint32_t overlay_fdt_size)
> +{
> +    int rc;
> +    unsigned int j;
> +    struct overlay_track *entry, *temp, *track;
> +    bool found_entry = false;
> +
> +    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> +    if ( rc )
> +        return rc;
> +
> +    spin_lock(&overlay_lock);
> +
> +    /*
> +     * First check if dtbo is correct i.e. it should one of the dtbo which was
> +     * used when dynamically adding the node.
> +     * Limitation: Cases with same node names but different property are not
> +     * supported currently. We are relying on user to provide the same dtbo
> +     * as it was used when adding the nodes.
> +     */
> +    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> +    {
> +        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
> +        {
> +            track = entry;

Random question (not doing a full review of the DT code): What use is
this (and the track variable itself)? It's never used further down afaics.
Same for attach.

> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op {
>  typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
>  DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
>  
> +#if defined(__arm__) || defined (__aarch64__)

Nit: Consistent use of blanks please (also again below).

> +struct xen_domctl_dt_overlay {
> +    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
> +    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
> +#define XEN_DOMCTL_DT_OVERLAY_ATTACH                3
> +#define XEN_DOMCTL_DT_OVERLAY_DETACH                4

While the numbers don't really matter much, picking 3 and 4 rather than,
say, 1 and 2 still looks a little odd.

> --- a/xen/include/xen/dt-overlay.h
> +++ b/xen/include/xen/dt-overlay.h
> @@ -14,6 +14,7 @@
>  #include <xen/device_tree.h>
>  #include <xen/list.h>
>  #include <xen/rangeset.h>
> +#include <public/domctl.h>

Why? All you need here ...

> @@ -42,12 +43,18 @@ struct xen_sysctl_dt_overlay;
>  
>  #ifdef CONFIG_OVERLAY_DTB
>  long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
> +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);

... is a forward declaration of struct xen_domctl_dt_overlay.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:31:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:31:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723280.1127993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aGa-0003N6-7Z; Thu, 16 May 2024 12:31:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723280.1127993; Thu, 16 May 2024 12:31:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aGa-0003Mz-4p; Thu, 16 May 2024 12:31:24 +0000
Received: by outflank-mailman (input) for mailman id 723280;
 Thu, 16 May 2024 12:31:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7aGY-00010i-LH
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:31:22 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3356c494-1380-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:31:21 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-520f9d559f6so902460e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:31:21 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c34fed1sm10355362a12.96.2024.05.16.05.31.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:31:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3356c494-1380-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715862680; x=1716467480; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=wPrPh7gWtAYkvlpDuB8QIJDgNh4So4IIXhJClXiAi+A=;
        b=RaMFKC0A0vUk/lvH8P+1UZ5GfbhuKUUvsK0e0PjURe1UzrlHMRGFPVSx+n78xI6wAG
         f87FZR9/xnKeoRfz99BxZAHb/djx/Sg/jMUOCyjYbWk3lRexZaoq8V0PJRUDBZTKnEAG
         +GZlZxrvGGY83IVNpGkiwOBiQygFm3DYV5dWs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715862680; x=1716467480;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wPrPh7gWtAYkvlpDuB8QIJDgNh4So4IIXhJClXiAi+A=;
        b=o3Nath6moY+Z8hoy9wQy3ecHHHKJmUECHzIeNDVwLthhXxvJw5micuJd/2DStme/oS
         H1hd2CZfHhsJrPG+VpXp8C3DMF8rTxs11Cd+NnraPUJsMKWp8jt+CgWDLxbgoIoOzC5l
         OKnlDsIZyZDQBseQFYBuMg0tPXyDMbosxoCx2H8yYrf1UUwkH/PD2d6mpcWZhDiLIwkE
         +acdehVkrEptQusmlzOV8wnoAGHcvGydUJi2dGRb5kZ0vUCPIOX1xoqGWkwoddWhiSt/
         GgtUAMjDIaf5oSY5nob4nxTF8oRBqq4UNArtbLCP/zFtMit8BFKBpLWzYFLfjcxFRmZH
         2V3A==
X-Forwarded-Encrypted: i=1; AJvYcCUVQCOeY1kfcljMQod1q8fBPDpgtIgSIQUrSGlYJAVtmJfhOIsC+uLOFYmwQcZQsMtatUGiJ7GiwZyBgjUh0Avo6HjXSzwHeCjnheb50Y8=
X-Gm-Message-State: AOJu0YxfB0/jBFowuFGLyqMjHaG7UHgYiR65yshxVE6iHoOUR/9njUsp
	eJ29V5I6K99NCTTQQCqI6IuXFkeyQ7BfKqz6aANmIpCrcaFYO4hsw4nSHN/JlC4=
X-Google-Smtp-Source: AGHT+IF3nDhH9LYtt/iQWz4RRaI6H2Gn+1QrjsiyCqehySKP46LMIOQnBqrSYG8g4ifRufiESkQ7nQ==
X-Received: by 2002:a19:8c56:0:b0:51d:a87e:27e6 with SMTP id 2adb3069b0e04-5221006e5a4mr11256617e87.67.1715862680481;
        Thu, 16 May 2024 05:31:20 -0700 (PDT)
Message-ID: <f4621fbf-50b8-45a2-aa4c-b94572753573@citrix.com>
Date: Thu, 16 May 2024 13:31:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/ucode: Further fixes to identify "ucode already up to
 date"
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
 <7d8fa536-f196-46e8-870f-0cfb57c8a65d@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7d8fa536-f196-46e8-870f-0cfb57c8a65d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16/05/2024 12:44 pm, Jan Beulich wrote:
> On 16.05.2024 13:31, Andrew Cooper wrote:
>> When the revision in hardware is newer than anything Xen has to hand,
>> 'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
>> because it doesn't know whether the revisions across the system are symmetric
>> or not.  This involves the patch getting all the way into the
>> apply_microcode() hooks before being found to be too old.
>>
>> This is all a giant mess and needs an overhaul, but in the short term simply
>> adjust the apply_microcode() to return -EEXIST.
>>
>> Also, unconditionally print the preexisting microcode revision on boot.  It's
>> relevant information which is otherwise unavailable if Xen doesn't find new
>> microcode to use.
> Since you do this for the BSP only, I'm okay with that. Doing this for all
> CPUs would have added too much verbosity imo, and I would then have asked
> to log the pre-existing revision only when no update would be done by us.
>
>> Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with one small request related to the remark above:
>
>> --- a/xen/arch/x86/cpu/microcode/core.c
>> +++ b/xen/arch/x86/cpu/microcode/core.c
>> @@ -881,6 +881,8 @@ int __init early_microcode_init(unsigned long *module_map,
>>  
>>      ucode_ops.collect_cpu_info();
>>  
>> +    printk(XENLOG_INFO "Boot microcode revision: 0x%08x\n", this_cpu(cpu_sig).rev);
> Can this please be "BSP" or "Boot CPU" instead of just "Boot", to clarify
> which CPU's information this is? I'm pretty sure you too have hit systems
> where firmware doesn't update all cores.

I'll switch to BSP.

I have further plans (4.20 at this point) to reduce logspam.

* The AP boot path should warn if it finds a revision which isn't the
bsp_orig version, cached version, or thread-0 revision.  Most of what is
printed right now is expected and normal in the system.

* Late load should print once, not once per CPU.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:37:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:37:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723291.1128004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aMQ-0005GH-Us; Thu, 16 May 2024 12:37:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723291.1128004; Thu, 16 May 2024 12:37:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aMQ-0005GA-Pc; Thu, 16 May 2024 12:37:26 +0000
Received: by outflank-mailman (input) for mailman id 723291;
 Thu, 16 May 2024 12:37:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7aMP-0005G4-Na
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:37:25 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0c00cb18-1381-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:37:24 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so322895566b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:37:24 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17f0dsm975929066b.211.2024.05.16.05.37.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 05:37:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c00cb18-1381-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715863044; x=1716467844; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j8XEvVeiMVNR9flj7k3kSfSfD9pzbPRxdD0zGTFaP9Y=;
        b=ODWsp09n566ml/ConQ2dbyz17U4W5QDQ0erTnqrua8lytzoy3Qwu2cnBa3n/BrSSEu
         1ljgk+GlO3s95UaMXO+GniJtl1R4JImeMbPrIPEUT0Y1dWEOcNBtWsfaVYu2wqicbyYN
         WsdPV+8rJeQSGtq3U69I/jTuZoovgezwt/Dwvn6opMjcMlgO51860XCDI+6gDipt781O
         gCmhDXooERABmH6FD4XRFvoj1IiG4NMpVdUOBp3PtOwrraru34H0kvOiGVszqY+mxy8z
         MOFWsq7Rcag6BCFSuWVR0yGTLv9TmUPXjRxhow5Q0y5CTyAFbaLkAATsZEdeiRmdg74q
         vLNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715863044; x=1716467844;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=j8XEvVeiMVNR9flj7k3kSfSfD9pzbPRxdD0zGTFaP9Y=;
        b=hDjKoo5OYwAYrYRbKxgjlD2KdWonC0zlYYR5MFxQkBTE6Az/BYcZhkjMgdFE4Ckk/9
         r0MINIj9g0z+Lqvwnd+nTDH3ouIcpggBgWQQ2osd7Zs9FpRLggI/ITNAPMEIrg5LGY9L
         wI4hrcM05sx9/RIH7p4rE24BITyC8CN0SNioPDOXXAdw2+BOQ3xbQcnOHKH4mkFBT0qF
         wqjD8kSDo6I9m2NDCGUXmlCvF3pbSmVrW4ReD9YZWbFFLdMf94v4cOZIYc6MwkRYNUfm
         7PMHPMeZ1vgsSvCgHEGKZVwHuzk6rD8HWzmTv3tI8KDpycLIFGKufghRKM4P/IcvyRNT
         p+BA==
X-Gm-Message-State: AOJu0YzKeJlounTvpnT3RIN3arQSPHSAZE9D02hsdx4M3whHzV3Ks3Jf
	u44EC8hSPVI2wy5i5ZIeDgb19fXBAjSUH8nwcbSY6gb0OZwEML3E0ImN2PSuSg==
X-Google-Smtp-Source: AGHT+IHqUb76w5A9zgx0D9op5fXp+0mipdfVjXA0f9uvCIvzYXXOZRLu1I00fgSv7V9/MUOxKAxMDg==
X-Received: by 2002:a17:906:13c8:b0:a59:cbb5:e09e with SMTP id a640c23a62f3a-a5a2d6669b6mr1251785766b.58.1715863043962;
        Thu, 16 May 2024 05:37:23 -0700 (PDT)
Message-ID: <ce573d24-1d52-48a8-b3f0-8636dacd5c9b@suse.com>
Date: Thu, 16 May 2024 14:37:22 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
 <ZkXvdWf2mVkPW458@macbook> <7965ba74-9177-44b7-9253-7ab5a77fd88f@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7965ba74-9177-44b7-9253-7ab5a77fd88f@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 14:29, Alejandro Vallejo wrote:
> On 16/05/2024 12:35, Roger Pau Monné wrote:
>> On Thu, May 16, 2024 at 12:07:10PM +0100, Alejandro Vallejo wrote:
>>> Bring test_x86_emulator in line with other tests by adding
>>> install/uninstall rules.
>>>
>>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>>> ---
>>>  tools/tests/x86_emulator/Makefile | 11 +++++++++--
>>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
>>> index 834b2112e7fe..30edf7e0185d 100644
>>> --- a/tools/tests/x86_emulator/Makefile
>>> +++ b/tools/tests/x86_emulator/Makefile
>>> @@ -269,8 +269,15 @@ clean:
>>>  .PHONY: distclean
>>>  distclean: clean
>>>  
>>> -.PHONY: install uninstall
>>> -install uninstall:
>>> +.PHONY: install
>>> +install: all
>>> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
>>> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
>>> +
>>> +.PHONY: uninstall
>>> +uninstall:
>>> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
>>> +
>>
>> FWIW, should you check that HOSTCC == CC before installing?  Otherwise
>> I'm unsure of the result in cross-compiled builds, as the x86_emulator
>> is built with HOSTCC, not CC.
>>
>> Thanks, Roger.
> 
> Right...
> 
> More generally, should we do s/CC/HOSTCC/ on all compiler checks? I see
> no particular reason to do them on $(CC) rather than the actual compiler
> used during build.

No. There really is a mix here, intentionally. Anything built through testcase.mk
is using CC, and hence respective checking needs to use CC, too. That said, I
don't think the split is done quite correctly just yet, which may raise the
question of whether having the split is actually worth it.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 12:45:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 12:45:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723294.1128012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aUG-00006s-Jf; Thu, 16 May 2024 12:45:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723294.1128012; Thu, 16 May 2024 12:45:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aUG-00006l-H9; Thu, 16 May 2024 12:45:32 +0000
Received: by outflank-mailman (input) for mailman id 723294;
 Thu, 16 May 2024 12:45:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7aUF-00006f-8Y
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 12:45:31 +0000
Received: from mail-ua1-x932.google.com (mail-ua1-x932.google.com
 [2607:f8b0:4864:20::932])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2d1cf653-1382-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 14:45:30 +0200 (CEST)
Received: by mail-ua1-x932.google.com with SMTP id
 a1e0cc1a2514c-7f79ef6704eso2222600241.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 05:45:30 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1d8e7dsm75303776d6.121.2024.05.16.05.45.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 05:45:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d1cf653-1382-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715863529; x=1716468329; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=xJ6WrOoPMYCYkS5FRW7xUJVq0lN7VWF/Dq61F4trvd4=;
        b=LFvI9407nWhI3ZJAhfCwRpeiH2072NPRihNYtcr8HbiHZRxNBHD3WBv/xHLL0XjNSP
         JAz1+Vl610zghhwadxBFshH9DqrIAc0hiMttRX+c5Gv9/QAv1oUHkCHigSpXSxiIFqzL
         OqXTCsO74tF6NlD6Et9Cxsjb4DTsbxrgBK108=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715863529; x=1716468329;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xJ6WrOoPMYCYkS5FRW7xUJVq0lN7VWF/Dq61F4trvd4=;
        b=kZyDebSw7XLiaomZV2YI08V09N9t4HZzhD/cIOKWO9kYx7qBMtZsCFnHu1DtMFzKkC
         TwdPI8hqco2k8EwlS/Us9C4YtMhlGo3dZruLoLlxD+BvPQ47gJoqTeqQMWMu/Ffjrl5g
         znypil8rFO4DtRmfbjnS6yQoKaurkF90SzxORyt+X7OcBfOSf8dt9ntOresgn4Hc3T6C
         x0JZc6lXtE+mBSnvJ4A86R+N5QbVvvRkx+uEwkzdWJ7lIZIEwPHOBY04RrLk5OGjPXsY
         kvD1+rwFEGi3D3t9CsI0/W4bbaLRHJaLUyahZCvwJs+Z9K66C6/GuskpxEujWnNkWxb5
         72Ng==
X-Gm-Message-State: AOJu0YzxQRA7c8GmW3OlOijlW+wfxFXuZhBpBTHsHuVmoZ8Fmtanq3EB
	9FBc4xf+/jwBzB5fR3nYSiC8t/vEAaziMH4hMBzuW5CY+XCGWleMnV6u5bK4WOg=
X-Google-Smtp-Source: AGHT+IEt8MsvWBDLChCZDq1yp4DaJRjvXNKsK064s9UQXVewO8sJSoWFqWQQxDQB9ejDq+nXWOEMcA==
X-Received: by 2002:a05:6122:4595:b0:4ca:80c5:753d with SMTP id 71dfb90a1353d-4df8829f482mr17125759e0c.4.1715863527550;
        Thu, 16 May 2024 05:45:27 -0700 (PDT)
Date: Thu, 16 May 2024 14:45:25 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>,
	Fouad Hilly <fouad.hilly@cloud.com>
Subject: Re: [PATCH] x86/ucode: Further fixes to identify "ucode already up
 to date"
Message-ID: <ZkX_5W-b2J6eR7Py@macbook>
References: <20240516113103.3018940-1-andrew.cooper3@citrix.com>
 <ZkXzFwbYfFI6DeP8@macbook>
 <4bc4f2f5-6914-445a-a9a1-a609c0c3cf1f@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4bc4f2f5-6914-445a-a9a1-a609c0c3cf1f@citrix.com>

On Thu, May 16, 2024 at 01:30:21PM +0100, Andrew Cooper wrote:
> On 16/05/2024 12:50 pm, Roger Pau Monné wrote:
> > On Thu, May 16, 2024 at 12:31:03PM +0100, Andrew Cooper wrote:
> >> When the revision in hardware is newer than anything Xen has to hand,
> >> 'microcode_cache' isn't set up.  Then, `xen-ucode` initiates the update
> >> because it doesn't know whether the revisions across the system are symmetric
> >> or not.  This involves the patch getting all the way into the
> >> apply_microcode() hooks before being found to be too old.
> >>
> >> This is all a giant mess and needs an overhaul, but in the short term simply
> >> adjust the apply_microcode() to return -EEXIST.
> >>
> >> Also, unconditionally print the preexisting microcode revision on boot.  It's
> >> relevant information which is otherwise unavailable if Xen doesn't find new
> >> microcode to use.
> >>
> >> Fixes: 648db37a155a ("x86/ucode: Distinguish "ucode already up to date"")
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >> ---
> >> CC: Jan Beulich <JBeulich@suse.com>
> >> CC: Roger Pau Monné <roger.pau@citrix.com>
> >> CC: Fouad Hilly <fouad.hilly@cloud.com>
> >>
> >> Sorry Fouad, but this collides with your `--force` series once again.
> >> Hopefully it might make things fractionally easier.
> >>
> >> Background: For 06-55-04 (Skylake server, stepping 4 specifically), there's a
> >> recent production firmware update which has a newer microcode revision than
> >> exists in the Intel public microcode repository.  It's causing a mess in our
> >> automated testing, although it is finding good bugs...
> >> ---
> >>  xen/arch/x86/cpu/microcode/amd.c   | 7 +++++--
> >>  xen/arch/x86/cpu/microcode/core.c  | 2 ++
> >>  xen/arch/x86/cpu/microcode/intel.c | 7 +++++--
> >>  3 files changed, 12 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
> >> index 17e68697d5bf..f76a563c8b84 100644
> >> --- a/xen/arch/x86/cpu/microcode/amd.c
> >> +++ b/xen/arch/x86/cpu/microcode/amd.c
> >> @@ -222,12 +222,15 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
> >>      uint32_t rev, old_rev = sig->rev;
> >>      enum microcode_match_result result = microcode_fits(patch);
> >>  
> >> +    if ( result == MIS_UCODE )
> >> +        return -EINVAL;
> >> +
> >>      /*
> >>       * Allow application of the same revision to pick up SMT-specific changes
> >>       * even if the revision of the other SMT thread is already up-to-date.
> >>       */
> >> -    if ( result != NEW_UCODE && result != SAME_UCODE )
> >> -        return -EINVAL;
> >> +    if ( result == OLD_UCODE )
> >> +        return -EEXIST;
> > Won't it be simpler to just add this check ahead of the existing one,
> > so that you can leave the code as-is, iow:
> >
> >     if ( result == OLD_UCODE )
> >         return -EEXIST;
> >
> >     /*
> >      * Allow application of the same revision to pick up SMT-specific changes
> >      * even if the revision of the other SMT thread is already up-to-date.
> >      */
> >     if ( result != NEW_UCODE && result != SAME_UCODE )
> >         return -EINVAL;
> >
> > Thanks, Roger.
> 
> Not really, no.  That still leaves this piece of logic which is
> misleading IMO.
> 
> MIS_UCODE is the only -EINVAL worthy case.
> 
> Every other *_UCODE constant needs to be 0 or -EEXIST, depending on
> allow-same/--force.

OK, my main concern was the previous logic wouldn't allow a newly
introduced state to get past the return -EINVAL, while the new logic
could possibly allow it to pass through.

I don't think adding states is that common, and if you prefer it that
way it's fine.

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:06:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:06:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723301.1128023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ao7-0005Bf-3E; Thu, 16 May 2024 13:06:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723301.1128023; Thu, 16 May 2024 13:06:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ao7-0005BY-0b; Thu, 16 May 2024 13:06:03 +0000
Received: by outflank-mailman (input) for mailman id 723301;
 Thu, 16 May 2024 13:06:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yDT=MT=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s7ao5-0005BS-1q
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:06:01 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 097353e9-1385-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:05:59 +0200 (CEST)
Received: from MN2PR14CA0028.namprd14.prod.outlook.com (2603:10b6:208:23e::33)
 by CY8PR12MB7171.namprd12.prod.outlook.com (2603:10b6:930:5c::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May
 2024 13:05:54 +0000
Received: from BL6PEPF0001AB4A.namprd04.prod.outlook.com
 (2603:10b6:208:23e:cafe::3e) by MN2PR14CA0028.outlook.office365.com
 (2603:10b6:208:23e::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 13:05:54 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB4A.mail.protection.outlook.com (10.167.242.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 13:05:54 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:05:53 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:05:53 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 08:05:52 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 097353e9-1385-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Bzn55rLE3VRLhvFEvkH+Rj19/RNE7YAbRG39fS+Dv/vdRmkaugZVqY8IBb6zYjc1COi+p40iWXzSy84AxJ9GBsq6imgmJO2upb1WM6otM4XM1c0BB33nM1tS114Gdu4loKp+1ByVzWyeo0W9CLjdvYipXU18owKG/BWg/YK6nFrWbcLr9SQhboe4iHobEtqxpOGiTEVu/gxmjZeVlysatFrohiMpgsZU6oeNsuZEo4+rlaeNZtEvntCs7277rlDnSIgZ2+MlLq+ATmMOpHR+qchXxrbTVTXfzlaHWYklky9OiA0mkZ5Ga3beh/Ag6YsY0JtFp62gJsqxWwYzB2q2RQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4UnRlEiOLmIelgMslq7Jt2mZlLO6gUYQq8cE7ygBmek=;
 b=AMJnZHSkK3K5d82D921q39CA/1J6j5NdyMa50W8PEV9Dpgqcr9amQ5ZFqQtScmxNEUodHyue4kYh1EkkXgbRrNPo7qbQ8rGaCVYU39nG2Zme0mDqZ6cXw2O20vJD0AISHafK4bVW/x2AfNq7FuWt9fSvUdnS+mIHaSzi+CN3OZPnK8ZVHl336S4Hv0WGRgLwcFUUt8RyZg+PT+zo4aHRFosCG60upxwooPcmpoWqgR5S8UeXOvww3sc5wlEnyyK0pVG2e7AyJUj0zb0BvtqZ7uUj82B4A2YtlXeyy9PlBHqyEr4svnQ4NEUUZwaRk3mtcIBZoVhd1DsB+DD46qkT/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4UnRlEiOLmIelgMslq7Jt2mZlLO6gUYQq8cE7ygBmek=;
 b=mzXEEqwedK2lJQl7nH2i/GaBpdHk7iRLFoMgS4vt0fT/JiXpQmLnAtrizBhjjq9SjsEjWPJ+9VLXXKThLAM+o/XjXO7+Emw9PpwvkLpJDdSjQawxfQYZXklC6S0jtixxGiNdikmireS85c3+Q95LqSEbcu89nKzJKvZAoUvmyPc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <d5de1a67-fb91-43b3-b7d3-e5162f479a11@amd.com>
Date: Thu, 16 May 2024 15:05:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/7] xen/arm: Lookup bootinfo shm bank during the
 mapping
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-2-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-2-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4A:EE_|CY8PR12MB7171:EE_
X-MS-Office365-Filtering-Correlation-Id: baa0abef-b24c-48b4-6059-08dc75a8eb3f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VVE5K1ZkWHZnVmJZanhvTkk1TFpJSjllU2hRTTNVUlZXUHN0S21LbkRhODBG?=
 =?utf-8?B?YmJiSXBYWEorU0taVGRKRnhacHE2VjMxYUhXTXJEKythUHQ5b2JEYW4vVXVi?=
 =?utf-8?B?ZktXK2tkWWdUU1ZGb0hRbW8wZWIwWVpHbmMwNGw5bGdpMnpLR2IzbS94dENl?=
 =?utf-8?B?WFVvMitEenA1clUxYmJySUN3N0pCTTF0THVrdG1FMEFROGY5akxKemk5R3pM?=
 =?utf-8?B?WTRPVkxnc29GQkJkN0dhaUN2NHNZdlhONFdEWEIxYVI3c0xRMXBabHhGbzRa?=
 =?utf-8?B?STlXRmE1aDJVVnVLUkl2Q0Q0SnlPc2tiMVU3MnlnUzBPUTB2SVBOYWVidGta?=
 =?utf-8?B?TDRwVzFlY3NOSWg2dm9pNmxPczlCUDB3UEhOdE0wTmxCdkRrWnRROHg0OFAv?=
 =?utf-8?B?eVlmLzdpUG1QcmF0K1Q0ZURuY3hXcmpqekpNMFV5ckhVTW0xTnhxVmFSZ25U?=
 =?utf-8?B?Z2pvUkZLT3pTUCtESHZGZG1LUzd3VWw0dDdLR0IzaUc1VWNIVjQ0VnlSandB?=
 =?utf-8?B?Tkt1bDY0YndWWE8weXVNamhMNVU5MlVzd2dxb0luZzJFNkNMQllGMk4xaFlt?=
 =?utf-8?B?aUplZDU5VGUrajAvdHRNSDM2YmsrdHNsVXR6bmRXTXBFR0lwV3RWY1RUUFY3?=
 =?utf-8?B?bzdtK3NhK2RkM2Fhd0JaejRJVEhOTGgrYWlwNVllaEVWQmw3RnhnRk1oakpk?=
 =?utf-8?B?YkhVU1FSOTYvNHgySnRDTzRkTnNRQzJJWEZTUkVkMytmN1ZRY3RXSnNSZkQx?=
 =?utf-8?B?cVcvNTlnTE1XcVJkSGtPRGNPRjJ3Q3hzKzB5YVlETHVkREg5Znp3bmM5K1Nv?=
 =?utf-8?B?YXpOZ2Q1RGUvRVVYUzFnTDYvSGx1S3pYdG9uRmZEekViS1V4NjdOL2d5QzhT?=
 =?utf-8?B?L29FZWEvRFJheEtYL25xbUNsT2ZzV3JFeFl6SG5Bcis1eTQ0ODBUQjZmN3Vk?=
 =?utf-8?B?YURna2c5d3JpcWF0UWErVXJ0TURuZ2prV2kyWVVCTUxQdnNReHFEWmFnQkJN?=
 =?utf-8?B?TVNrOVNYcVlsbzBMdk9VSDBIeFZLdGJkd0VPTzRSZ1k5MkdCL3pmMVlsRndp?=
 =?utf-8?B?Q0ZabzZsYzdHUnZuRzFkVWt2cHpqc28yVWg3dDF1T2pmam5BeUdYUC9ocm02?=
 =?utf-8?B?UWYyRk41MlBGM3drbEhQL1FqL1oySXQ1WEIzS0g3dHJ4MVptODU3UmN5WnIz?=
 =?utf-8?B?QzZTSEp5a0QyM1gvNkw3YmxxR3FhRldvaXF5MEZabC8zR1pJa24xUUtzbmtS?=
 =?utf-8?B?YndtUmZ6Vjh2Y2I0TFhwWEZJRFo5WERzOGpGK3ZRUWlpMHFPdm81TmlUVG1I?=
 =?utf-8?B?cUp2WlpQYWR2VjVrZUE1aFNFUThBRFA1U2JERWJuQi9iaFkvQXN4ZU9SdHI2?=
 =?utf-8?B?eVR4NURuSCs0elJNWS9BQWt2eFMzNXJ3Y0VNVk1HWTQvUjkwWkFGUitBZWs2?=
 =?utf-8?B?UkJ0anlxYWlkMnZQZFdqbVUvS2VPV1BqY3F0d2ExWmxIaWhBR1IwOUNlUkw0?=
 =?utf-8?B?MWhISjBTRGlGVCtDU0dxd1N4UmloR3RCQzZOcUxKaWgzMnc2ODZRUS9UeGs4?=
 =?utf-8?B?VVhVUUNVdTZUVmJoWUl2QWVTTlJ3VkoreTJObFB1U0FLdWF4N2pOTG5seVRz?=
 =?utf-8?B?SDlDTnJ6bGl1ZjZ3cW1HVXZqZkFvY0kvK0xlMTArZDRidDE5bzFtdkZlWVlQ?=
 =?utf-8?B?OHdaVDMxemp2dWt5NnE3Rk1mTG8rRnZyN0s4ZjFBMDl1bHpXby9GZUZaYlFM?=
 =?utf-8?B?aHBNaCtEd1FyckdEN0w1Q2dwZXI2NFBNaEV5T2lLOHZmRFQxYUhDZmJIZXhl?=
 =?utf-8?B?elJCVDRnNlQ4SmM1VGU1VlhSdnZvTU9JaWV4cm1WZlliN1lOVDBMZWswc1JH?=
 =?utf-8?Q?3TcpEDjYfwqif?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 13:05:54.1210
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: baa0abef-b24c-48b4-6059-08dc75a8eb3f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB4A.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7171

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> The current static shared memory code is using bootinfo banks when it
> needs to find the number of borrowers, so every time assign_shared_memory
> is called, the bank is searched in the bootinfo.shmem structure.
> 
> There is nothing wrong with it, however the bank can be used also to
> retrieve the start address and size and also to pass less argument to
s/argument/arguments

> assign_shared_memory. When retrieving the information from the bootinfo
> bank, it's also possible to move the checks on alignment to
> process_shm_node in the early stages.
> 
> So create a new function find_shm_bank_by_id() which takes a
> 'struct shared_meminfo' structure and the shared memory ID, to look for a
> bank with a matching ID, take the physical host address and size from the
> bank, pass the bank to assign_shared_memory() removing the now unnecessary
> arguments and finally remove the acquire_nr_borrower_domain() function
> since now the information can be extracted from the passed bank.
> Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
> case of errors (unlikely), as said above, move the checks on alignment
> to process_shm_node.
> 
> Drawback of this change is that now the bootinfo are used also when the
> bank doesn't need to be allocated, however it will be convinient later
s/convinient/convenient

> to use it as an argument for assign_shared_memory when dealing with
> the use case where the Host physical address is not supplied by the user.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v2 changes:
>  - fix typo commit msg, renamed find_shm() to find_shm_bank_by_id(),
>    swap region size check different from zero and size alignment, remove
>    not necessary BUGON(). (Michal)
> ---
>  xen/arch/arm/static-shmem.c | 101 +++++++++++++++++++-----------------
>  1 file changed, 54 insertions(+), 47 deletions(-)
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 78881dd1d3f7..0afc86c43f85 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -19,29 +19,22 @@ static void __init __maybe_unused build_assertions(void)
>                   offsetof(struct shared_meminfo, bank)));
>  }
> 
> -static int __init acquire_nr_borrower_domain(struct domain *d,
> -                                             paddr_t pbase, paddr_t psize,
> -                                             unsigned long *nr_borrowers)
> +static const struct membank __init *
> +find_shm_bank_by_id(const struct membanks *shmem, const char *shm_id)
>  {
> -    const struct membanks *shmem = bootinfo_get_shmem();
>      unsigned int bank;
> 
> -    /* Iterate reserved memory to find requested shm bank. */
>      for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
>      {
> -        paddr_t bank_start = shmem->bank[bank].start;
> -        paddr_t bank_size = shmem->bank[bank].size;
> -
> -        if ( (pbase == bank_start) && (psize == bank_size) )
> +        if ( strncmp(shm_id, shmem->bank[bank].shmem_extra->shm_id,
Does it really need to be strncmp? You validated it a few times already.

Other than that:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:08:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:08:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723305.1128032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aqC-0006FG-J0; Thu, 16 May 2024 13:08:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723305.1128032; Thu, 16 May 2024 13:08:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7aqC-0006F3-GQ; Thu, 16 May 2024 13:08:12 +0000
Received: by outflank-mailman (input) for mailman id 723305;
 Thu, 16 May 2024 13:08:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7aqC-0006DO-0F
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:08:12 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5864ec18-1385-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:08:10 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5a1054cf61so362489466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 06:08:10 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf60sm993297866b.14.2024.05.16.06.08.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 06:08:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5864ec18-1385-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715864890; x=1716469690; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Vxg8B5JCecBGum66lthlq5hRvVO2wT4u2t755TfKbSU=;
        b=cjt0139jNuhYQetkBAYvbKpgReT2AmqC1QuN0DI3jwwOWH7yXJyyAnWJ88K8KNUf0n
         AVZBa/ug/Oi+2cdUszS6XopdNFBYUYLpsLZ+m1bpeHMZSkAR27W3lzUVsC0qKZTRxH8V
         25WjH0CKs1bI5byEKlOrwSWT/pPB+zHfZppD6J6ai2v52lYr9G2Fy7Cw2vPbXx8rowmp
         LxDjLuY5uxX/snOadscEP1/R/Na6Rdc4WH8vCX7/BjUekcHLmw7wadQ+yAHqElXm0fb9
         WocvlVdASzfmd29I3iyf1v4t9UoQ93qLs4a8Fh0nPg8TalcteGG0j7WWHHYUveMQZKFE
         p+Sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715864890; x=1716469690;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Vxg8B5JCecBGum66lthlq5hRvVO2wT4u2t755TfKbSU=;
        b=KOR6h4+F4gZWP7gn3523WNrt+dx6kHqPEf8JJuZmYjLBi+7uR9ChVdU4H8Ku4kMZy4
         s4tWtN4KlvjU+NrXlMB60Bby0qYJRhqNry2fbvw58wvjv65bFrOZ27TBeM2hXnfDdM8x
         7TgQshkM9FcQC11PVUVr7qbg7JmcWw4yw5RqxlrNh6cuOPeEW+1am9WlsQqUQ/VU+a3q
         yzIu2l7BbyIklepwsmj6q5OpSljlVUX0ebn9Ex4VbBVNJZcxzDkyHyuI+16YdOrb5nsk
         jIXuwLuFuS6ZRrrF7J0PoPG4HYZxrmi0AdRQsoewa5mffYQxT1G0bSTp1KFIyGlppwIg
         cncQ==
X-Forwarded-Encrypted: i=1; AJvYcCWiekbFLHV6g3IBpi6AIg3RGn5zNM5xr0QqNpUJDrzjjuatqmt9vazSWagsj3Jo5GBqWEpRiZe61/6eVDP76gy47UjRnodeVqC8Sdf7yI8=
X-Gm-Message-State: AOJu0Yz1e7mlCI8CF3+msewLHgRLko/BL15aPg+ab4ZvfDUHbLQq3jhL
	BVqUDf/reQn3cfWARMa+H/5DZN//mZuBzsWJIls1mwTxe/TZUv3sq0ZQ/bJPyw==
X-Google-Smtp-Source: AGHT+IFzWBlJszmQfQiZi0hqkZt0GdNVGjDd5j7j7kIRZ2AVL+BMbMBh8py9sCQ81439QRTdtHeHkQ==
X-Received: by 2002:a17:906:4882:b0:a59:a727:1e81 with SMTP id a640c23a62f3a-a5a2d57a381mr1847560166b.20.1715864890209;
        Thu, 16 May 2024 06:08:10 -0700 (PDT)
Message-ID: <fb708441-b302-4727-8131-62435a54d99c@suse.com>
Date: Thu, 16 May 2024 15:08:08 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Content-Language: en-US
To: Jiqian Chen <Jiqian.Chen@amd.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
 Huang Rui <Ray.Huang@amd.com>, xen-devel@lists.xenproject.org
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516095235.64128-2-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 11:52, Jiqian Chen wrote:
> @@ -67,6 +68,41 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          break;
>      }
>  
> +    case PHYSDEVOP_pci_device_state_reset: {
> +        struct physdev_pci_device dev;
> +        struct pci_dev *pdev;
> +        pci_sbdf_t sbdf;
> +
> +        if ( !is_pci_passthrough_enabled() )
> +            return -EOPNOTSUPP;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&dev, arg, 1) != 0 )
> +            break;
> +        sbdf = PCI_SBDF(dev.seg, dev.bus, dev.devfn);
> +
> +        ret = xsm_resource_setup_pci(XSM_PRIV, sbdf.sbdf);
> +        if ( ret )
> +            break;

Daniel, is re-use of this hook appropriate here?

> +        pcidevs_lock();
> +        pdev = pci_get_pdev(NULL, sbdf);
> +        if ( !pdev )
> +        {
> +            pcidevs_unlock();
> +            ret = -ENODEV;
> +            break;
> +        }
> +
> +        write_lock(&pdev->domain->pci_lock);
> +        ret = vpci_reset_device_state(pdev);
> +        write_unlock(&pdev->domain->pci_lock);
> +        pcidevs_unlock();

Can't this be done right after the write_lock()?

> +        if ( ret )
> +            printk(XENLOG_ERR "%pp: failed to reset PCI device state\n", &sbdf);

s/PCI/vPCI/ (at least as long as nothing else is done here)

> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -115,6 +115,16 @@ int vpci_assign_device(struct pci_dev *pdev)
>  
>      return rc;
>  }
> +
> +int vpci_reset_device_state(struct pci_dev *pdev)
> +{
> +    ASSERT(pcidevs_locked());

Is this necessary for ...

> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
> +
> +    vpci_deassign_device(pdev);
> +    return vpci_assign_device(pdev);

... either of these calls? Both functions do themselves only have the
2nd of the asserts you add.

> --- a/xen/include/public/physdev.h
> +++ b/xen/include/public/physdev.h
> @@ -296,6 +296,13 @@ DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_add_t);
>   */
>  #define PHYSDEVOP_prepare_msix          30
>  #define PHYSDEVOP_release_msix          31
> +/*
> + * Notify the hypervisor that a PCI device has been reset, so that any
> + * internally cached state is regenerated.  Should be called after any
> + * device reset performed by the hardware domain.
> + */
> +#define PHYSDEVOP_pci_device_state_reset     32

Nit: Just a single blank as a separator will do here, for going past the
columnar arrangement of other #define-s anyway.

>  struct physdev_pci_device {
>      /* IN */
>      uint16_t seg;

Is re-using this struct for this new sub-op sufficient? IOW are all
possible resets equal, and hence it doesn't need specifying what kind of
reset was done? For example, other than FLR most reset variants reset all
functions in one go aiui. Imo that would better require only a single
hypercall, just to avoid possible confusion. It also reads as if FLR would
not reset as many registers as other reset variants would.

This may seem to not matter right now, but this is a stable interface you
add, and hence modifying it later will be cumbersome, if possible at all.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:15:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:15:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723311.1128047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7axT-0000bj-BQ; Thu, 16 May 2024 13:15:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723311.1128047; Thu, 16 May 2024 13:15:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7axT-0000bc-8l; Thu, 16 May 2024 13:15:43 +0000
Received: by outflank-mailman (input) for mailman id 723311;
 Thu, 16 May 2024 13:15:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7axR-0000bW-Tz
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:15:41 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64308ffc-1386-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:15:40 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-572669fd9f9so3501033a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 06:15:40 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bebb56fsm10527900a12.33.2024.05.16.06.15.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 06:15:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64308ffc-1386-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715865339; x=1716470139; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MnSS4hSoCWtkefUSxw1ivBpTMCEMzONTA3RqAl7piS4=;
        b=IG0joNHKxScd6OkrD/9oQqLgvsHJf9ASpgaFOzs5F3ZdJ3gRBH5ydfFrD3mPD7KI/K
         ZFcelOf99FsEYeD+CjrEZQJjLtw6L37qinAHPkghwZdFtHkMONcxlnF8JeVW3EfE8Wno
         HGtvELuhjxv94hh23GbuU64JAFpGhi1p4fae0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715865339; x=1716470139;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MnSS4hSoCWtkefUSxw1ivBpTMCEMzONTA3RqAl7piS4=;
        b=HIM6nFIA5jUBB+y6YvVrfZwX6ZAO/j3k2nzYtLPWYPJrkczb/EfgdDWAInzpPNAdb3
         GA7DVP026xh23/c+mPLfZ3MiojROhljKyY3GtVDgVGBnoFWGtRpX1o+Qgea1anpkLjbi
         pfeP3knOGJlwIXRliaNF3xQLpxm6AONDZd5AK3quBrRB6z6s9m1pf1jr7s8Vt/VzawoR
         VI8tRt+2rSGljToa6+uKqZjZ6jXNo3ye81bBZuFP+Ah6vvXCuj+WxcxkEp2wmZMGMSJO
         HXpQeh3RgS1JIM9w3S8lCWxgOyPfbGD5cqfTKbby7Kn74A40/f386ZubuP8qVpLs7A3x
         b96A==
X-Forwarded-Encrypted: i=1; AJvYcCXSqo9HuQLUUwT+1RbMBac0Mljgge/x5Cq+GVpkqJ4o7qu/ycDXgv1vD2dTCMjk/NcoHDQosLQUoe3qDzS/yPimeJT/o0ieeRtHxpp9by4=
X-Gm-Message-State: AOJu0YxRrwgySoCbyDj7xaer+iph5V08YoLwh6syysbNj+/znIZh4zX8
	bLtWBywlxJ0xlzPgdXb+QF12Hh03/57LR87t7VBMCxhsCHn9tQ5SvP5q+0w7esQ=
X-Google-Smtp-Source: AGHT+IHxTT7K41Nkg6kQcl8zYMmNCxsyoYCpa+lvNkL7GumtZfjOOO4o8Nao7aVMbfGHLEw9pMOf/A==
X-Received: by 2002:aa7:d294:0:b0:574:eb43:3865 with SMTP id 4fb4d7f45d1cf-574eb433921mr4634714a12.32.1715865339456;
        Thu, 16 May 2024 06:15:39 -0700 (PDT)
Message-ID: <1c98ba15-fcbf-4fa1-aa0e-39de8bdfb5fa@cloud.com>
Date: Thu, 16 May 2024 14:15:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
 <cf8cfb82-7a69-4690-8afb-d971dbe8b7e3@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <cf8cfb82-7a69-4690-8afb-d971dbe8b7e3@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi,

On 16/05/2024 12:16, Jan Beulich wrote:
> On 16.05.2024 13:07, Alejandro Vallejo wrote:
>> Bring test_x86_emulator in line with other tests by adding
>> install/uninstall rules.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> 
> I'd expect such a change to come with a word towards what use the binary has
> on the installed system.

First of all, having some tests installed and some not is highly
confusing (See [1]).

> Imo we should not randomly put binaries in place
> when there's - afaict - absolutely no use for them outside of development

I don't disagree, but that's (imo) a packaging matter. For reference
XenServer has a xen-dom0-tests package for this very purpose, so we can
install it or withhold it at will.

Being able to validate that the development invariants still hold on
whatever machine you happen to be running on is generally a good thing,
I reckon.

Cheers,
Alejandro

==============================

[1] (grepped for the install target in the tests folder)

[snip]

--
./paging-mempool/Makefile:install: all
./paging-mempool/Makefile-      $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
./paging-mempool/Makefile-      $(INSTALL_PROG) $(TARGET)
$(DESTDIR)$(LIBEXEC_BIN)
--
./xenstore/Makefile:install: all
./xenstore/Makefile-    $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
./xenstore/Makefile-    $(if $(TARGETS),$(INSTALL_PROG) $(TARGETS)
$(DESTDIR)$(LIBEXEC_BIN))
--
./depriv/Makefile:install: all
./depriv/Makefile-      $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
./depriv/Makefile-      $(INSTALL_PROG) $(INSTALL_PRIVBIN)
$(DESTDIR)$(LIBEXEC_BIN)
--
./cpu-policy/Makefile:install: all
./cpu-policy/Makefile-  $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
./cpu-policy/Makefile-  $(if $(TARGETS),$(INSTALL_PROG) $(TARGETS)
$(DESTDIR)$(LIBEXEC_BIN))

[snip]



From xen-devel-bounces@lists.xenproject.org Thu May 16 13:20:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:20:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723316.1128057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b1s-0002Jp-Sf; Thu, 16 May 2024 13:20:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723316.1128057; Thu, 16 May 2024 13:20:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b1s-0002Ji-Pk; Thu, 16 May 2024 13:20:16 +0000
Received: by outflank-mailman (input) for mailman id 723316;
 Thu, 16 May 2024 13:20:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yDT=MT=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s7b1r-0002Jc-6b
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:20:15 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06e02175-1387-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:20:13 +0200 (CEST)
Received: from PH0PR07CA0004.namprd07.prod.outlook.com (2603:10b6:510:5::9) by
 CH2PR12MB4071.namprd12.prod.outlook.com (2603:10b6:610:7b::16) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.30; Thu, 16 May 2024 13:20:06 +0000
Received: from SN1PEPF000397B2.namprd05.prod.outlook.com
 (2603:10b6:510:5:cafe::be) by PH0PR07CA0004.outlook.office365.com
 (2603:10b6:510:5::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Thu, 16 May 2024 13:20:06 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF000397B2.mail.protection.outlook.com (10.167.248.56) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 13:20:05 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:20:05 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:20:05 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 08:20:04 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06e02175-1387-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YweDm7nwuT1PqW36+eplneGvOibIh+WnWhiccyWRsrKEI3gfdzKYPmLQLSl1Tsa/K8Aci61wbVm7824isAu1L6gnnRuJZmUaXb7L0jQ6pQNJHj3A6L3Tq8PoNCdc1hsNVcZdg0IIn63c8hOwMrWYc0edQ8W/HF1ISFf+Eccxv5syj44A8/H/s/9ghPZq0oWlNZ+B5X/d/FcdySD6SJAaBROKxjaeaHwfKV1c6kHVPOL3JiIGBF1bkeXRDykIcLr+CfJCrFF4Fp6nMAKmGJWRICvsjeWI7SzgMxJdmFjk180NpuMd17PXxszOmllRORrAVotFvzF7rvL4YMrMrAB1Dg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HQNxe9XW3c06hYbC2BOFWXBtWoD/EIj0yaL3VUHYHVM=;
 b=JqNJlkE3zoDKL7q6/zr4cls9nUtoE3zPZrEoAZaUqw1/glsw3N7+VUv5PypJx+GfHGLbZ3nPXQKZ7iwPBL8I1gq7CQRv3KSWIpS3UXelB4PDNzdJ/19fVc6+V6T5+a0qYp+S4YMWX4dkdAn/DQ+kZhPUnNs7ekIP7MJ9RNNyIvkpY/ZxTUdBynSps/FCZg7siLW55jdpSPv9Vj2wbmVrPL5F3o5ZRN8ayL4nRjZdUu62NNOEQ8KFDXwG4WC35QHI3UHA9ZuGsKJGGerWOcsVS0oLO2jkeQapipiw6Fk2ZT/sdhM8ESe2ifCjO1PKKEmt5dn/JJbr5qeDn1HblR1NoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HQNxe9XW3c06hYbC2BOFWXBtWoD/EIj0yaL3VUHYHVM=;
 b=Sr5mt5a34+BZITsmo2ezOplNwyH/UhNInulMkRdkHFVQ/gPMk/Z3lssu/07ltGl3U4C2y/SAgptBCdkRIxlsD8I9tSPNTyYz5cLETyYB6qd440vJZowqv1X+gmEfEXJDUhiLDnCoPXSvUxajaZlupr89B0yd4e2UuFf9ARTgOz8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <854f4353-6a02-48a1-99f3-0ce65950e790@amd.com>
Date: Thu, 16 May 2024 15:19:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/7] xen/arm: Wrap shared memory mapping code in one
 function
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-3-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF000397B2:EE_|CH2PR12MB4071:EE_
X-MS-Office365-Filtering-Correlation-Id: 6391f37d-94f1-4428-26bc-08dc75aae702
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?UDlMYjlYUysvUjNZNXRaMXVqSVBOQ2J5Q2x5eU52cFJQWEF0aHVEZktjbW4v?=
 =?utf-8?B?VE5SOFFTdUsxVkJtUEd3NVBwOTQ3YVpLaTBaSU52Zzg0dTl2T0hXVkRLb0FF?=
 =?utf-8?B?WUhONnNLNUZSSGdUWHhzL1lnM0NiS1lIODkxYWQyUnFGSjRUM2FsU0ZSQU9M?=
 =?utf-8?B?SGlBdUhMaXFPUEkzS0pCRG1pMzdxRGZ1MXpvUEhIWWkzUDk5Nkt5Wk44QnFD?=
 =?utf-8?B?a0swaDk3VGswaWhaSDkwMVNYMFdwaGN4R1hpcWhRbmVYUDk4emZLSTlCYTU2?=
 =?utf-8?B?OXZCaXJ2aHJsK1hOSFh0TlN2Z0RtemcxcFFDK1FLZy9VZkZkcjNEeEgzMkZY?=
 =?utf-8?B?UUZZdGpNVFlzczlkY0xCMTFRZkhGV2k5ZmVUUlR2em94RjNGR1BOMHBmYlJO?=
 =?utf-8?B?SW9rTDVGTjhIR0RuQXNTVm80UmpJN3MySHpEQ0tOV0Zxdm9uK3B5NncrMmpl?=
 =?utf-8?B?UGdUS0Q4SjdtWWF6eStxcXpUUXRmY0ZTYUtxVGRIck5VRmsxeHpGOWd1bndP?=
 =?utf-8?B?Uy9YMGpIN0lEeWNZMkZVUkhDM1ZqeHlIUmVoaUk5NTBSc2tHeVNvRzl3NmNm?=
 =?utf-8?B?VTFBVHVDa2lOZHp1SXlVbzBXWmRjZnVuVzVoT2Q3c0p5SDZBU2NMV3g0L1hx?=
 =?utf-8?B?SEZvZmpnNlZxRFplaVltQjdjMjBtMlRzTEp3NUg1R3NYbTZONzRUa3dUM0FR?=
 =?utf-8?B?cCtHdFJ4a1JaSmFXZGZRS0xvM2t5Nkp3Z3ZKV1lRNlZGSUdzRldGUWUybzc2?=
 =?utf-8?B?OTNUcnlEWC93NUxSN3ZDT1V4ZHFaL3lJWk1KSHIwT1RnV3U1RUJPNm44dUZw?=
 =?utf-8?B?RmpML1AzbDZMMTVwSXkyQTl3dlIwWW5mWWI2K1BDSWQzZ3NwM0FkY1RmNUkv?=
 =?utf-8?B?ekVSWUdIWHA5ejViY3VwTnRhbEJpZW5WZXFjZDQ2MEJCZjBNeThvUHB3WlZn?=
 =?utf-8?B?Y1hDTTIwbU8xblpHWnNZcm13RjJDTGc1SUp5WENEMm1Td3NmbXNPMmx0UjVy?=
 =?utf-8?B?RDAwYmdoaXM5WDdIMFVGb3o1VXRSdVhiUU15U0NGSlI1b2NsZko4YmxhNUtk?=
 =?utf-8?B?dGtLNzAyUnF6OTJqSk9DQndBd1VKV0h1aGVHVlFYczI2b0JrQlZEZzJFRjBB?=
 =?utf-8?B?ZzdiTlBZeHJBbTFYRU1OMFRyR2xqZDc2dGdseGpGMW9Fdjc5dVk0bjZyYUhX?=
 =?utf-8?B?TVNqdW5PM0ltTmdDZTMrYzVGZ0tSbytocHNTSUdYY3o1aHZOUTZReDBDa2U5?=
 =?utf-8?B?alJTejFWUzZoUjNvbGpBKzJEWXVTQVdJNmZMd0FmaDhOZjlzZTBtVVNhTHRG?=
 =?utf-8?B?clN1a0krcWx4d0hQREwyZEdUZkJmSzU2VzIzUG9wWEJpaHBlTlZtZlRZRmpD?=
 =?utf-8?B?RXFCMng3NUdnMkJpcDFxZVJJTzEwaFc0dkp3b0NiNEV2NTBFYUNhL3cwbmlO?=
 =?utf-8?B?cUFWZ1MrbVJVODdhc0dTdEQzNTRrMDZhaVNFcGREM0QzSUdRRk01VWpKZlJC?=
 =?utf-8?B?NDI1YWRCSEVJMit0bXB5SUp6L2xBMHVhKzNjQUlqdUc3WVNKbkZ1eEVCQ0xh?=
 =?utf-8?B?V1ZwUFBIRzNDdWpvbFVVNTh5a09zWXhwbTIxU3JFZmtYUEQ3VzJuRHZHYzcx?=
 =?utf-8?B?RjJCaDFtRjk1QUVxclNqRC9vVnZlc0w3V3dBbCs3bEpSdEN1dnREUGhOdXlH?=
 =?utf-8?B?d0xoT0w3bFU4YnYzMzU0K3VTbHhqWDlyYWcwUHpHaE5qeGo1UWE1Rjk3UVl3?=
 =?utf-8?B?L2orRElLVW03VzZGQ1JjNmw4eE1zNjdVOFRrZlV3R1BiWGFLZ3BaNU54cTQ4?=
 =?utf-8?B?MXdlTHhEb2JyckZoNHQ4YVEzWEFLZkZwV3lsdWZkUUt6NU1VQ3Y3NE1TRmps?=
 =?utf-8?Q?6qieWnCCz+3u1?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 13:20:05.9710
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6391f37d-94f1-4428-26bc-08dc75aae702
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF000397B2.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4071

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> Wrap the code and logic that is calling assign_shared_memory
> and map_regions_p2mt into a new function 'handle_shared_mem_bank',
> it will become useful later when the code will allow the user to
> don't pass the host physical address.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v2 changes:
>  - add blank line, move owner_dom_io computation inside
>    handle_shared_mem_bank in order to reduce args count, remove
>    not needed BUGON(). (Michal)
> ---
>  xen/arch/arm/static-shmem.c | 87 ++++++++++++++++++++++---------------
>  1 file changed, 53 insertions(+), 34 deletions(-)
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 0afc86c43f85..8a14d120690c 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -181,6 +181,53 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
>      return 0;
>  }
> 
> +static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
> +                                         const char *role_str,
> +                                         const struct membank *shm_bank)
> +{
> +    bool owner_dom_io = true;
> +    paddr_t pbase, psize;
> +    int ret;
> +
> +    pbase = shm_bank->start;
> +    psize = shm_bank->size;
> +
> +    /*
> +     * "role" property is optional and if it is defined explicitly,
> +     * then the owner domain is not the default "dom_io" domain.
> +     */
> +    if ( role_str != NULL )
> +        owner_dom_io = false;
> +
> +    /*
> +     * DOMID_IO is a fake domain and is not described in the Device-Tree.
> +     * Therefore when the owner of the shared region is DOMID_IO, we will
> +     * only find the borrowers.
> +     */
> +    if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
> +         (!owner_dom_io && strcmp(role_str, "owner") == 0) )
> +    {
> +        /*
> +         * We found the first borrower of the region, the owner was not
> +         * specified, so they should be assigned to dom_io.
> +         */
> +        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
> +    {
> +        /* Set up P2M foreign mapping for borrower domain. */
> +        ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
> +                               _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
>  int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>                         const struct dt_device_node *node)
>  {
> @@ -195,9 +242,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          paddr_t gbase, pbase, psize;
>          int ret = 0;
>          unsigned int i;
> -        const char *role_str;
> +        const char *role_str = NULL;
>          const char *shm_id;
> -        bool owner_dom_io = true;
> 
>          if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
>              continue;
> @@ -238,39 +284,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>                  return -EINVAL;
>              }
> 
> -        /*
> -         * "role" property is optional and if it is defined explicitly,
> -         * then the owner domain is not the default "dom_io" domain.
> -         */
> -        if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
> -            owner_dom_io = false;
> +        /* "role" property is optional */
> +        dt_property_read_string(shm_node, "role", &role_str);
This now violates a MISRA rule saying that if a function returns a value, this value needs to be checked.
I think you should check if return value is non zero and if such, assign role_str NULL (thus removing such
assignment from a definition).

Other than that:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:22:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:22:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723320.1128066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b43-0003lP-7T; Thu, 16 May 2024 13:22:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723320.1128066; Thu, 16 May 2024 13:22:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b43-0003lI-4r; Thu, 16 May 2024 13:22:31 +0000
Received: by outflank-mailman (input) for mailman id 723320;
 Thu, 16 May 2024 13:22:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7b41-0003lC-Dq
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:22:29 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 578b406f-1387-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:22:28 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-34dc8d3fbf1so6593486f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 06:22:28 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbc887sm18860516f8f.98.2024.05.16.06.22.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 06:22:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 578b406f-1387-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715865747; x=1716470547; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=VRvChKd2TDTGb3agmd1ge7fLU7AVjjJM1gAP/8DEzUM=;
        b=HqBkatX3NkFISInxHVILdC4pOBU8JlBiLUbZijiwgpZNEGn1gZld0pul/ZgCxkaIje
         fhm8vYIN7AMkwVwzeVD2AJYzAzpcAobaCdHdZ91L7pYnLuH8FNrRaUrT6a18tNfxbmKV
         Dy4a+BjfPNNTK+0KYFjhU/jfhhTwHSpkuxNbQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715865747; x=1716470547;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=VRvChKd2TDTGb3agmd1ge7fLU7AVjjJM1gAP/8DEzUM=;
        b=HGIOKd03JWJfm/vXFMyNkSgBy34x6JESR17CuxfmA6uc+E6Fr0MDNWDI2T04qkFFJi
         Y5OARuWkQAr14I9GMNAUc4t9Aynz0iNk1FS2Z2KJbnsFkI41LtvhiABQhLnDNG9cStK2
         lUHqUuteyCHbcnCQgcfM9PQSbZ745yurVFCJ8xp1b7eGpaFDHQZeMNq+MOe/wxqjTKzk
         k8TrhiYxtFRe+b7TA3ToWDTZevTicrTDhV37GZz0b3LRJvUoRd1NQNOvFxhXIPWaJj4x
         6Cr1BD5Ct/C8fcl+BFFqSuZSo1UWlpRIBv4pHH0dqKAsa03RPiMA7o9E0MQ+Gr1zzEuQ
         t2Ew==
X-Gm-Message-State: AOJu0YxJVvR/HReE8IKmAENCvR3q9iunMr725BAm44u/aVdIFzREvPBJ
	oJC1aaOo8zQF4IMOoSSDLFCUb8r5kAX4uiG6FLTXsAofn5ipAbqGxJU2aoIeM6KruAPjn5Su69b
	5
X-Google-Smtp-Source: AGHT+IHOE0n0q0KE8F+wiBQXdbhx1wmyzDz1YXxcAFqBBVSvhXLTsrQNdFK+hxK+kYgcJ9ZDCYfiWw==
X-Received: by 2002:a05:6000:798:b0:351:b7c8:3f08 with SMTP id ffacd0b85a97d-351b7c83fa5mr13449591f8f.10.1715865747276;
        Thu, 16 May 2024 06:22:27 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19] xen/x86: limit interrupt movement done by fixup_irqs()
Date: Thu, 16 May 2024 15:22:24 +0200
Message-ID: <20240516132224.86005-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current check used in fixup_irqs() to decide whether to move around
interrupts is based on the affinity mask, but such mask can have all bits set,
and hence is unlikely to be a subset of the input mask.  For example if an
interrupt has an affinity mask of all 1s, any input to fixup_irqs() that's not
an all set CPU mask would cause that interrupt to be shuffled around
unconditionally.

What fixup_irqs() care about is evacuating interrupts from CPUs not set on the
input CPU mask, and for that purpose it should check whether the interrupt is
assigned to a CPU not present in the input mask.

Note that the shuffling done by fixup_irqs() is racy: the old destination
target is not allowed to drain any pending interrupts before the new
destination is set, which can lead to spurious 'No irq handler for vector ...'
messages.  While the proposed change doesn't fix the issue, limiting the
amount of shuffling to only strictly necessary reduces the chances of stray
interrupts.

While there also adjust the comment as to the purpose of fixup_irqs().

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/include/asm/irq.h | 2 +-
 xen/arch/x86/irq.c             | 9 +++++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index 413994d2133b..33dd7667137b 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -130,7 +130,7 @@ void free_domain_pirqs(struct domain *d);
 int map_domain_emuirq_pirq(struct domain *d, int pirq, int emuirq);
 int unmap_domain_pirq_emuirq(struct domain *d, int pirq);
 
-/* Reset irq affinities to match the given CPU mask. */
+/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
 void fixup_irqs(const cpumask_t *mask, bool verbose);
 void fixup_eoi(void);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 3b951d81bd6d..223f813f6ceb 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2527,7 +2527,7 @@ static int __init cf_check setup_dump_irqs(void)
 }
 __initcall(setup_dump_irqs);
 
-/* Reset irq affinities to match the given CPU mask. */
+/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
 void fixup_irqs(const cpumask_t *mask, bool verbose)
 {
     unsigned int irq;
@@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
                 release_old_vec(desc);
         }
 
-        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
+        /*
+         * Avoid shuffling the interrupt around if it's assigned to a CPU set
+         * that's all covered by the requested affinity mask.
+         */
+        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
+        if ( !desc->action || cpumask_subset(affinity, mask) )
         {
             spin_unlock(&desc->lock);
             continue;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu May 16 13:24:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:24:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723324.1128077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b6I-0004Td-ON; Thu, 16 May 2024 13:24:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723324.1128077; Thu, 16 May 2024 13:24:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7b6I-0004TW-LM; Thu, 16 May 2024 13:24:50 +0000
Received: by outflank-mailman (input) for mailman id 723324;
 Thu, 16 May 2024 13:24:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=S4LL=MT=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s7b6H-0004TQ-4C
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:24:49 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 (mail-am0eur02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2606::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aa0b4e39-1387-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:24:46 +0200 (CEST)
Received: from DUZPR01CA0231.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:4b4::12) by AM9PR08MB6243.eurprd08.prod.outlook.com
 (2603:10a6:20b:2db::5) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 13:24:37 +0000
Received: from DU6PEPF00009528.eurprd02.prod.outlook.com
 (2603:10a6:10:4b4:cafe::99) by DUZPR01CA0231.outlook.office365.com
 (2603:10a6:10:4b4::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend
 Transport; Thu, 16 May 2024 13:24:37 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU6PEPF00009528.mail.protection.outlook.com (10.167.8.9) with
 Microsoft SMTP
 Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7587.21 via
 Frontend Transport; Thu, 16 May 2024 13:24:36 +0000
Received: ("Tessian outbound 9d9bf1c5d85a:v315");
 Thu, 16 May 2024 13:24:36 +0000
Received: from 213ed3c0f3d1.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 AB566322-554D-4906-9126-BA5EBF54EC6E.1; 
 Thu, 16 May 2024 13:24:29 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 213ed3c0f3d1.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 May 2024 13:24:29 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB7353.eurprd08.prod.outlook.com (2603:10a6:20b:444::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May
 2024 13:24:24 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.028; Thu, 16 May 2024
 13:24:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa0b4e39-1387-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=bFA+GNPf8YhM63yJe4ZFBGYNNUsNOgxidhOdxgpOyT9J7et28uAiLHR/3FB4TWprz7spw4ey5yeDqcbLQCHPDuoNzIQiKypzILDI5HJzkaZ7KvC/dc57kIyZr/qjOr+BME8Wr1RVNttqDui741nzJuMehAnXeDaLuR5pUUhB5iSugQ/EpxLZy3WOOvo1R2PvC2oLyzqKmxnszEpdG5gYb6DIg5xLu39AMy1Z/H6bd7VByHeUw0RmzF1UVjWE2yWSYf/jJZ8nnwM8qUq9FmPmJ2NnkxODe7sunkVY+tb7nbMMqedws14cQErFeiSZtiokV1HcQeE0za4/HOdog0CXCg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fXXbJmbvR5oz9UhDa4nLJI+eTKj1bhiYZ1E+64sKjEA=;
 b=QclJ76Kx0v1hQs8i170OMcQSFcL8q19vLJKmt/O1HCa+pr9LmzFS7qdiaU2F0oIxuUJM2IUQkWbJfp+Cslj4zlEuZ3bwaXq37h+LfvSlOsBu0GzFh7550v8R4D+3i5nnZjtibjaW100XIT+IyiWOf3vWAifP/+oFegVGCWaAMRNXpN5BJGmviOqgcV15V7GB7nUmDR8i9JLyOKiOrn7MdGodSsfVmkiPKnPx3wwmvD0//vEpMrRGyIP4JBnmo0FqMcRVpdPkhyxMgTRiVqg1N+HI0vNQn5y3cL4i9m3hDqWp+TE9mjoA6AUwEbqRjcbechGvoNE2ZGQNLgc65y4P8w==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fXXbJmbvR5oz9UhDa4nLJI+eTKj1bhiYZ1E+64sKjEA=;
 b=al04fV83ndiO9A4aoFF2OPG/T8PshLNW55jjZ2+fdpCidSfY/irboytpR5/n8jw9Uhkrheftmblv0J7hyXM9aznX6i6gC9vpqfFdN8LEF6X8WnI4N7A0+HAQgMgbgFcO+YaUmOliU7bwrjcynSlUNxC5BiXNyz9kqUKyE4Ivd5s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: efa1791889e6ca4d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Tl/movNA4USbwPBnTMXmn1PsVLF2MIrtmgrbLQYzztsLvwru9cTe/fVqCLPKPIf++0XoEm7MFiAZ+FNB4/5jErqjjGRufs+ILCZxESTtrA4g7JYymSyHiueufLbWPhwAVXQk6l+NKtFZ0Zf1/d+FC6uZ6Cz+YsVx6Os1OuAW9PNF0wWJ2v85P0synMDiQ/Wr1BX6v4Kcibvpod8tgPyjyVd+FrW7o1Gub6kvxP11C7GixmPxnE8k6Ai6ivSkY0ydcjhcj33GODTPHI0iDf0nUB9pH1GOsxdo1FRj/tpbyn850XFRZGwGraZx7I7fjvkmoaD2REHN4Ju2bo5mHXWakA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fXXbJmbvR5oz9UhDa4nLJI+eTKj1bhiYZ1E+64sKjEA=;
 b=fOmFww3OiM75NZnK2gmEqzj1b/foNGrVPbO59QMRPYPXNBU+K+0yg+xUY2i5NEaO+GVhO9jbcDxEevixkj+TYO3hA2mesLLozFTS89OGtUmlO2MUB447qWEtIT9fmKbi6seI9ayjsDTkK5SBOQGjGU+oi/jNopp1Moe71bsYGEPmc+tThi0rifmn9YItA9maI95BuR5m94E6zsobS2s3/bRBraZ0bNKzpTw1A0bVx1JHEY2J06UXxn3lxZ/56OcgoE+8NsauGK1jb5KHyHNzxABtPVljENwGJN2QJV/OVWc+KSVeS22uG9rnFuFzTkamzv7qZkJrNo/UeppYlZP4sg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fXXbJmbvR5oz9UhDa4nLJI+eTKj1bhiYZ1E+64sKjEA=;
 b=al04fV83ndiO9A4aoFF2OPG/T8PshLNW55jjZ2+fdpCidSfY/irboytpR5/n8jw9Uhkrheftmblv0J7hyXM9aznX6i6gC9vpqfFdN8LEF6X8WnI4N7A0+HAQgMgbgFcO+YaUmOliU7bwrjcynSlUNxC5BiXNyz9kqUKyE4Ivd5s=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Topic: [PATCH v2 2/7] xen/arm: Wrap shared memory mapping code in one
 function
Thread-Index: AQHaptP615847oF42UGhJ4Bg9mW8m7GZ2dcAgAABMIA=
Date: Thu, 16 May 2024 13:24:24 +0000
Message-ID: <8509E09D-2CED-4DD0-AE4F-F6CEBC63AABE@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-3-luca.fancellu@arm.com>
 <854f4353-6a02-48a1-99f3-0ce65950e790@amd.com>
In-Reply-To: <854f4353-6a02-48a1-99f3-0ce65950e790@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB7353:EE_|DU6PEPF00009528:EE_|AM9PR08MB6243:EE_
X-MS-Office365-Filtering-Correlation-Id: ff57a3cf-9919-4366-c2ef-08dc75ab8813
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?aUcwZ0toWUNmT2drWEo5UEl6aEN5YWMrQUVNMXRKWFdPWldLQjJiWXBnSFor?=
 =?utf-8?B?T2ZlTnlzTjNxUG5VOTdvazdyVU5OWjR6ZXNqeldyMUd0UDA1MWl0dzE2RGJn?=
 =?utf-8?B?THJzWTc2dzByUElQaGwvZGRXYjgyMFJCWGpLRFVHNE9HdEVYTzZGdVErbUJ5?=
 =?utf-8?B?QjN1T3RSRjBvMWYydlhQcS95TTZwZWxxT0pYL2ZlcmlneHl2Y3ZVZnJXUVBR?=
 =?utf-8?B?dnQ4WW5yQkdvRjRXT2RxTm9oOGR3bkE2UFR1ckx6dDE1cUdPYUFOc0wxRVN6?=
 =?utf-8?B?VjJHcWZMcFplQy8rQVNZc0pTYnJ5NWxkcUlqZDdlMExVL0dSWkE5WnVLcktv?=
 =?utf-8?B?UUNTQnJ4MGhuTkIvL2pBK0hpZk5MNW1sL1c3SlVTaHdMbUV2VXEyalM5cmdU?=
 =?utf-8?B?YVY2YWtXcHBFZnhOSXNoZmdCZ3JDRWIxQnVFem04RUw0RUlyV3RCL3ZGcVlr?=
 =?utf-8?B?cVB1Y2txUjRwTkJockQxSW0xMDZJdk1TWFBkeEMyODlHVFMwR25DSXRqL09V?=
 =?utf-8?B?TGNDZE9USHpjNHZhaDNXL3BmV0kycmY2a0R5Z0EyeUFRRW9CVHo5UjFJS1Mv?=
 =?utf-8?B?b056Mis3eGdGdzdkZ21BK3d3NWg2UHJtUU1RWFdON1g3OUVQVjMrcXpHVFUr?=
 =?utf-8?B?bkJLMzlsYVowNTV6Rkx5R0wwaE9MRzY0VXlWVFl3MjdzenF0WlV0ZmtCNXdT?=
 =?utf-8?B?a2wzVVhIL3lmNU8xK3JhOElrTWs1cDRQZExiek54SlZqTXoyNXA0Nmc0eUpP?=
 =?utf-8?B?ZDN1OVEwQ24zSVdKT3NLMFpHYWVXajdVNzJuVG5LRTVYMkloVjdoaTgyYW1q?=
 =?utf-8?B?R3Q4VVZNTnVTOXNnK20wMEJpb040YlhrZTNlUjRKanUrTmZuRjh3am5TcHNF?=
 =?utf-8?B?MjV2a0ZaSXZxODFINWhXMVl5RGJsT2dielhCWlVqU21vSU5sU204UVZCTjBj?=
 =?utf-8?B?VEdiVlU5eUQwamg5UU1PMy9meG1ROWFERDNOSTVIMVdCYWtYdC9Fa21uSFhM?=
 =?utf-8?B?b2N4eEVsejhENlJQVEVUc3FkM3N2N0dNcGNkczU0L1RCTVRwRGdVM2dMcXo2?=
 =?utf-8?B?bWMrRlpJQ2Qzc09UM0tUdDRnZEhVcHRlYlR0ZVREa21MRHhvMWJmTmk5R0ll?=
 =?utf-8?B?bkNObGxXYUNuekl5eEtIVGNlUFFoTGRlc1g1YndWR1ppaEI4ajFnS1V3ejN6?=
 =?utf-8?B?SlltL2V0em8rM0dDVk1SYjB2T2RWT1A2WkN6bkFnQ0xsb2ttVFpwQ09KS2xE?=
 =?utf-8?B?bTM3UWQwNzVLOHVZUUFOVlJOZ0RhOWFneUJXM3dFenovNzR5dytQT2FxYmJX?=
 =?utf-8?B?eG13QjErcFdaMHdJM09DSkVQcEJIWHJJMm1iZzJKQlpGN1FPSU1VNDFhU3FZ?=
 =?utf-8?B?QVFnRDllYlh3dnRPRHFyM2NmdW5sTGMycHFObURCNjZybmthbVVabmtLazZw?=
 =?utf-8?B?TmtJZW1RVzhiZFlEOENISStIL1luWEEvdFdOcUZNa1dBSCtOOFl2cXkwZG8r?=
 =?utf-8?B?elRkY0lmMzNZMmpuemhDbUo2SHg3ejltdXdhZUtEZXFtVDN5UnA5TEJLclBE?=
 =?utf-8?B?eW4rZEJOMTJhRzJqMFBlQk1MejJzWDBjQ2NQbHVtVVl0YnNkUHd6OXgxdnMz?=
 =?utf-8?B?MW90ZWUxaFliRVkvbkNYTURtYndzRzJnMys0cFhDR08vTzVuL1VFQ21pczJU?=
 =?utf-8?B?UXBFVE5ha0dJNTF2NHg1Q1JaU2FYUUVWaXNLaTZLT1l5TUg5bS9ab01GM3kv?=
 =?utf-8?B?YWtraXppRTRDVXV6UURUSWpsY0cwM0EyWHJIRW8zUjFJTHhpVUt1YWJCZW5Y?=
 =?utf-8?B?L1lWczMvWHNFL21DRjRZdz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <8CC11CF333AB4A43B057114406A1AD85@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7353
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU6PEPF00009528.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	42d3e5de-901f-4963-b324-08dc75ab8102
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|35042699013|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?aVpmYzNsaENmS2RRZmRXbkxDd2EzekNmRVVCK2tvdlpEYzEvSm9pUDJYQjU5?=
 =?utf-8?B?NFJGRDZ5NnRNU05WOVZiRjBMdHZaY2NlWkJTYk9Bd2hiUEdiZnZkUEhncG8y?=
 =?utf-8?B?YzdsU1k1dmhwVkU1cDZyK0Fnd2tSd2dZN3p0eUxoL0RCSlBDeVYrT0dWRU9i?=
 =?utf-8?B?N0FrczVBSmhqZ3dVM3ZvLzBzYmcrdERBajdwcTFVanhoQ3VyWWJlOTVSTENX?=
 =?utf-8?B?MGdVMXJLbVFBYVNiUER2aUplKzg1Y3drTy9sYzYvT3dNMU5YS0RlQjJSaGg1?=
 =?utf-8?B?RG1sMUMvanJtSEd6bXBGMnlvTTNRMG8walorTWY1bm43UHd6ZDFMTlR6QU1o?=
 =?utf-8?B?M3dRZHZteXYxWU12NlpLRkh2MlBaNzVJVkxvazVpRHhocHpXMmRLblBubE1Q?=
 =?utf-8?B?K1pUN3JLeStHVGVHMG03UGE2Z3gycDhPU3hhWnJSMkczQ0VXMkVnT3AwbTUx?=
 =?utf-8?B?UGpiR3FPQk1keVFXd0hEWEdnMDZaNTJpakFzSU45V0c5azdnbU83U2VtWE00?=
 =?utf-8?B?bk83YjZwbVJmT0JSUW1TeXFldi9KU0VMb3p3MGZJU2h5QzhUL0dSVjg0UnRq?=
 =?utf-8?B?eElIU3RBNDBXYjRDbG5BMXBXaTFZbG9sa1B3SHVXK1M0YzJONUJHSjNvWHQz?=
 =?utf-8?B?aUZMRHM5YXgrdXIzNWJUQXo2NjdjbDNvODFkQWdXTHg3QXhsUjdGTWptTWtz?=
 =?utf-8?B?bnFBalZzR1ZBR2JlZmV3enpTVm5OQjhvZ0Y0WGI4RGdCUU82S0FTZXBOOHFh?=
 =?utf-8?B?YTlHRVNwYnpPemlGMWIwQkh2OElhMUpBUFNQc0pjTU1vVlNuNWdEdU1ZSGlU?=
 =?utf-8?B?RWRITHlpVXFMT1F5emM4b0dLUGZ0MVNnUnhLNDNyTFJ0ZXRTQ1Q3S1lQNXZP?=
 =?utf-8?B?TFk4b2hlWk9NbVkwWlVZZ1BaZGNZZjk2bWFVcUxJdTV3eFpGTlk4ZUNvVi9o?=
 =?utf-8?B?MmdGS09nTTFDb0Z4TUlBTFNlNzJpT1pJSlA5ODhLME8xYnlsenR2dk1FeU5m?=
 =?utf-8?B?NU1FamNQUFNEUXZ5N2RFOE5acE11NW0xeDZZZDdKTlE2eTY1ck1LSjY2d25r?=
 =?utf-8?B?Z0V5VXNBaFkvUFRPTEFIQXFLMmorUzJYWDBETEtvbHBjRnJ5M2E4KzVvWVRz?=
 =?utf-8?B?Ym5yMGJ2ZEVqU3R2MENDK2FhQk85Q2NRR05DbFhVZi9pK3VLaHhUQkc4aEJE?=
 =?utf-8?B?UnZUV2lkWENwV3phT1ptV3ZoMUdPUGI2djRpYnRUUzNqb0xMWFU2dEwxVmJm?=
 =?utf-8?B?am5wMkxWaWRTSUcwemY1Zkl4a3RWS3orRU9ZNThTQVI1SlJXTVFmNndvWWxx?=
 =?utf-8?B?aE1LdHZsd0l6Z1Z0M1hCUExKZ05YMThOemFpeUwveFNHYlVTeXJVSWR3RHBx?=
 =?utf-8?B?dmRERUI3T3ZRZlBjRkV6RmlwRndaNjNGMmJxaVMwTkdYQVRTdG15Z1k2cFIx?=
 =?utf-8?B?TkpTVFYwRFluMCtFNFNQUXhycUdoeGUzSXFOaUdxdHN4eUp6dWhJcDQ3T1Bu?=
 =?utf-8?B?WXZzWWhMSEFWb1pDNlBEYkRBbGlyR1hZZTdKeUxTdVlqM01rRmYyRE9RN01H?=
 =?utf-8?B?ZVRmeklrWEFRNXdxaS9wZnBTT3FoeHlQVnRoTVcxUG5ZRzdNRlNtNDdFdnRV?=
 =?utf-8?B?M21pbzBsamlyV0drOEVEUW9aQUdITE9LZlJ1ZVhKWjB5eVkxNjhHM3puckM4?=
 =?utf-8?B?a05PTmxIYk9QTXhRemhvaHRVQnRUNllYSXBzbVJMYXNnYld0TC82MXc5WWw5?=
 =?utf-8?B?VlhVb2daQ1pHaHdTMGc1SkI5Z0xMRkVib2tyTjJyaGY5dkRUVkRicnFiS2xa?=
 =?utf-8?B?RW1VM0RKMjJ2bEJtTTAvMUlIb1hUYTlFT1AzUDhobHlWMkZ6amFjMmN3UXlP?=
 =?utf-8?Q?s465g5wdjlEtK?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(1800799015)(35042699013)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 13:24:36.2465
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ff57a3cf-9919-4366-c2ef-08dc75ab8813
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU6PEPF00009528.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6243

SGkgTWljaGFsLA0KDQo+PiANCj4+IC0gICAgICAgIC8qDQo+PiAtICAgICAgICAgKiAicm9sZSIg
cHJvcGVydHkgaXMgb3B0aW9uYWwgYW5kIGlmIGl0IGlzIGRlZmluZWQgZXhwbGljaXRseSwNCj4+
IC0gICAgICAgICAqIHRoZW4gdGhlIG93bmVyIGRvbWFpbiBpcyBub3QgdGhlIGRlZmF1bHQgImRv
bV9pbyIgZG9tYWluLg0KPj4gLSAgICAgICAgICovDQo+PiAtICAgICAgICBpZiAoIGR0X3Byb3Bl
cnR5X3JlYWRfc3RyaW5nKHNobV9ub2RlLCAicm9sZSIsICZyb2xlX3N0cikgPT0gMCApDQo+PiAt
ICAgICAgICAgICAgb3duZXJfZG9tX2lvID0gZmFsc2U7DQo+PiArICAgICAgICAvKiAicm9sZSIg
cHJvcGVydHkgaXMgb3B0aW9uYWwgKi8NCj4+ICsgICAgICAgIGR0X3Byb3BlcnR5X3JlYWRfc3Ry
aW5nKHNobV9ub2RlLCAicm9sZSIsICZyb2xlX3N0cik7DQo+IFRoaXMgbm93IHZpb2xhdGVzIGEg
TUlTUkEgcnVsZSBzYXlpbmcgdGhhdCBpZiBhIGZ1bmN0aW9uIHJldHVybnMgYSB2YWx1ZSwgdGhp
cyB2YWx1ZSBuZWVkcyB0byBiZSBjaGVja2VkLg0KPiBJIHRoaW5rIHlvdSBzaG91bGQgY2hlY2sg
aWYgcmV0dXJuIHZhbHVlIGlzIG5vbiB6ZXJvIGFuZCBpZiBzdWNoLCBhc3NpZ24gcm9sZV9zdHIg
TlVMTCAodGh1cyByZW1vdmluZyBzdWNoDQo+IGFzc2lnbm1lbnQgZnJvbSBhIGRlZmluaXRpb24p
Lg0KDQpTdXJlLCBJ4oCZbGwgZG8gaXQuDQoNCj4gDQo+IE90aGVyIHRoYW4gdGhhdDoNCj4gUmV2
aWV3ZWQtYnk6IE1pY2hhbCBPcnplbCA8bWljaGFsLm9yemVsQGFtZC5jb20+DQo+IA0KPiB+TWlj
aGFsDQoNCkNoZWVycywNCkx1Y2E=


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:30:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:30:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723328.1128087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bBF-00059a-9U; Thu, 16 May 2024 13:29:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723328.1128087; Thu, 16 May 2024 13:29:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bBF-00059T-5Z; Thu, 16 May 2024 13:29:57 +0000
Received: by outflank-mailman (input) for mailman id 723328;
 Thu, 16 May 2024 13:29:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7bBE-00059N-L8
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:29:56 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 60a44188-1388-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:29:53 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51f174e316eso939004e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 06:29:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c322c85sm10573149a12.84.2024.05.16.06.29.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 06:29:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60a44188-1388-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715866192; x=1716470992; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zDEyhE5ASknvhoXEIMpQKHP43jM+oRuvjSH2AGEV8to=;
        b=HAngb/TcIPFT9qrtVyofQpZSQ4wxk4URUcHcefkrddd2GCB3BsKBMpIY9L9+hZwyfz
         FM1ADCb/ehFTyURQFIkPmEiAGfJlXAQpbYkgnAp83+Km2CVZOd6i55QT/Nr/bNSc1zTZ
         aQdCnOVk8L/nYxkAD9dKg1moD6rnDu6977gLoRQAg7S76S58UR8bZYakc4A1h68LuJW4
         th7xLS6AwOxFnzqPXheirTmzTr2nHMdQbAdz+SbyHTVpcWlt3Z8OsoktZZlEEet+SRm+
         GU5fiTa40NZuQCAdXGzTq7CXqNaq3dWGaBEvHEsi/8hkxCMFB91FwloHqoL4myJKNEKf
         +yUw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715866192; x=1716470992;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zDEyhE5ASknvhoXEIMpQKHP43jM+oRuvjSH2AGEV8to=;
        b=OSdN4SVwavHg+RTFonA+zRgty6H8mam3i8iqOvoFai1oIysSZfM3jqsqkJ1QMlO5NE
         rspNGH/dEnfQNdh1sfHJ7cLRHsraZjEjXoGDp8nij7BRM9KLpI+GXqMDBsYHkaSQuT3O
         II/lELR7BBNcKz54/dZu403XRqvD+B9me6VoSzhMVmvZeeyhB3pABna0UQb6fAuu6lKD
         mC4w7aD8moJCris0l8NdGnp34r56q51VaUWPSBxBZk5xb5YhjRESs91gF7dWWXbCfcN/
         TUS8QEXFLR0qAveiicK5y0DYaZri9BBwn5niDpFntq0TWonqZ6nYmtPnyOSRQEf3wgYS
         XYWQ==
X-Forwarded-Encrypted: i=1; AJvYcCWKy2MxDqOKksAZNgFu5fXLOy2eodWgJb2wLn+R58JixtW4uDKV/MpEATBMKiXDU+VjN8SKIjG0cF81NLTTzWi2Tj7UTW+DhAk1XA/Gymw=
X-Gm-Message-State: AOJu0Yx4mM5yxB7X/uncHWJ/zw67OvW05N0amS/wHiu7pZaIyCdGD3Ec
	3hdtsYEXcLgRk8Xz+D4wc61hfsv5lJ/QEyyUcHVrqdqhFofb5HfIgIKVSVv9ZQ==
X-Google-Smtp-Source: AGHT+IH0fxs0mgXVgSU5rvT6dcBio4Ee8gWbkJCyzHlyVC9igq9fQ6h/RrwhFYtxaWvV0q6N/7qDZw==
X-Received: by 2002:a05:6512:6ca:b0:523:8f5e:4aa0 with SMTP id 2adb3069b0e04-5238f5e4b78mr3932674e87.63.1715866192458;
        Thu, 16 May 2024 06:29:52 -0700 (PDT)
Message-ID: <673505f6-2d0f-4ea7-ab9f-c132adce28c4@suse.com>
Date: Thu, 16 May 2024 15:29:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 2/5] x86/pvh: Allow (un)map_pirq when dom0 is PVH
Content-Language: en-US
To: Jiqian Chen <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
 Huang Rui <Ray.Huang@amd.com>, xen-devel@lists.xenproject.org
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-3-Jiqian.Chen@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516095235.64128-3-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 11:52, Jiqian Chen wrote:
> If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
> a passthrough device by using gsi, see
> xen_pt_realize->xc_physdev_map_pirq and
> pci_add_dm_done->xc_physdev_map_pirq.

xen_pt_realize() is in qemu, which imo wants saying here (for being a different
repo), the more that pci_add_dm_done() is in libxl.

> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -74,6 +74,8 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      {
>      case PHYSDEVOP_map_pirq:
>      case PHYSDEVOP_unmap_pirq:
> +        break;

I think this could do with a comment as to why it's permitted as well as giving
a reference to where further restrictions are enforced (or simply mentioning
the constraint of this only being permitted for management of other domains).

> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -305,11 +305,23 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_map_pirq: {
>          physdev_map_pirq_t map;
>          struct msi_info msi;
> +        struct domain *d;
>  
>          ret = -EFAULT;
>          if ( copy_from_guest(&map, arg, 1) != 0 )
>              break;
>  
> +        d = rcu_lock_domain_by_any_id(map.domid);
> +        if ( d == NULL )
> +            return -ESRCH;
> +        /* If caller is the same HVM guest as current, check pirq flag */

The caller is always current. What I think you mean is "caller is same as
the subject domain". I'm also having trouble with seeing the usefulness
of saying "check pirq flag". Instead I think you want to state the
restriction here that you actually mean to enforce (which would also mean
mentioning PVH in some way, to distinguish from the "normal HVM" case).

> +        if ( !is_pv_domain(d) && !has_pirq(d) && map.domid == DOMID_SELF )

You exclude DOMID_SELF but not the domain's ID? Why not simply check d
being current->domain, thus covering both cases? Plus you could use
rcu_lock_domain_by_id() to exclude DOMID_SELF, and you could use
rcu_lock_remote_domain_by_id() to exclude the local domain altogether.
Finally I'm not even sure you need the RCU lock here (else you could
use knownalive_domain_from_domid()). But perhaps that's better to cover
the qemu-in-stubdom case, which we have to consider potentially malicious.

I'm also inclined to suggest to use is_hvm_domain() here in favor of
!is_pv_domain().

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:42:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:42:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723335.1128098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bNF-0001Dm-C0; Thu, 16 May 2024 13:42:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723335.1128098; Thu, 16 May 2024 13:42:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bNF-0001Df-7H; Thu, 16 May 2024 13:42:21 +0000
Received: by outflank-mailman (input) for mailman id 723335;
 Thu, 16 May 2024 13:42:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yDT=MT=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s7bNE-0001DZ-1m
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:42:20 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1bdf6754-138a-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:42:17 +0200 (CEST)
Received: from DS7PR06CA0022.namprd06.prod.outlook.com (2603:10b6:8:2a::10) by
 DM4PR12MB6373.namprd12.prod.outlook.com (2603:10b6:8:a4::7) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.26; Thu, 16 May 2024 13:42:14 +0000
Received: from CY4PEPF0000EDD4.namprd03.prod.outlook.com
 (2603:10b6:8:2a:cafe::e2) by DS7PR06CA0022.outlook.office365.com
 (2603:10b6:8:2a::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 16 May 2024 13:42:14 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000EDD4.mail.protection.outlook.com (10.167.241.208) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 13:42:13 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:42:13 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 08:42:12 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bdf6754-138a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JBdEPLBzdP3WZUhwM71IRit+AdCL7whylgEf4fVC01QflS6M0Mb82023BCVZz26XfesBf9aJ3mw6PLfdF2lL0tNnsqywTUeANB2GytlxPH5udYtqrGL9TRPqr+YvXWlShPWwRp7Rd+OYxfNDi8UCrZEnAUnNxJoK/SLp207uk6+XZfrup1WFT+KybdFIicMFEA2BJNSIi/A0owJNN7luFdKpy/Py9t/kbVAWS6ogfhRWX0HdPUCM4P+KYoz0dctNUwYtjeK1jL4jFKDz6+nxgFhFaWHAUc51SLczgs8F/S0P9CIVIeSEqrvT05Sx7bqukp+3n4lQnC5Y6AZ7Ag0YfA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EbCiiT6Z2rWvizVA1JY6tTL7+yCkRWhu3zFgm1eXdOY=;
 b=JiU/F563CjXMoBQFz+PIeqcGvm6X/0fxAfUn/n0s3aWimZlCiaDkQInbkL6QHFZCyIq9LHKF6Yg+u/uLq9RlrJ8ygU8WP/8+Sxm3ql9yrEd+N1JY9TaftoqPQsvjFwMvdkRHmybTnX+qrWNpK1KM/IJFsg8KjGrs7ribnkAMvV5XIiS94oQnOI/XglrVTA5IezvjEvFoxrQHzPrvh8tP1q63RwHN6iKEf+Vbai4Rq9xbjvWPsvdwQI8TGOWvkF3ZflS+niFOEDxPX5rrrtxuS4Ai+uiyLPbmGJbwD2QdHwZdJrROO/bY119JDNYWBay0x1zocaP851ZlJCxvRKKnbQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EbCiiT6Z2rWvizVA1JY6tTL7+yCkRWhu3zFgm1eXdOY=;
 b=bU+HyrUfkVvkeB0mahjznjTXfGqQQ4+w7ScNb5Kf2ICpw0oUG7kztKOXgONQBcwDQfgS1Qfq/lfx3Hqg+/gRYigvqGOBzB30aeJqIngAikO9RZueh7vX6GYIoF34XxMuvDH9JiAzLMmFvJtTU03jR1ejOd6KB+zFettYONJKhTc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <4141bf2c-5190-4e8e-89bd-42382439e517@amd.com>
Date: Thu, 16 May 2024 15:42:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/7] xen/p2m: put reference for level 2 superpage
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Penny Zheng <Penny.Zheng@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-4-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD4:EE_|DM4PR12MB6373:EE_
X-MS-Office365-Filtering-Correlation-Id: 5bd79731-80bf-46c2-399b-08dc75adfe81
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZElRV2t5eVFZdHFiSUVOL2wxNCs1eDNrRDY4b3ZNbVdKdDVWWUFaeklmU1Yx?=
 =?utf-8?B?aWs2bzBUVmkrTkJ2bGl2VlBIN0gyNU9DOEJXTkhKajZFUFdQOVhyOGoxcmE5?=
 =?utf-8?B?bmpOQzY3bUw3TFRmVFJVVHo4eTk1YWVLdVdmRlc1YjhnOGtqNmlyWnNmK1FC?=
 =?utf-8?B?MlV0YWIwNnRXVTlPYWFPTXI1MFJTWms3U0xpSGZZRWZzaHJFdVBQQmFqRDNx?=
 =?utf-8?B?NGx2YnQraE5LR3FwME4rZU5DU1p6eUt4QURFV2RXYnRDblM3WDNhalNvVzRH?=
 =?utf-8?B?OHVucTV2MjFrWlFGMnR3SFlhK2pDZFBoK0hCVVZKTlB6cmQ3STR0RzFka2Mz?=
 =?utf-8?B?dG9RRDAvUlMzWndCUGdueHdIbHJOZ0FtdTJLOERwWklndWdXMjJSS2t6N1N3?=
 =?utf-8?B?czlBbS9DeVBzRU81bGp0RzFpOTRPRDdBbjJJeDdUNmMzY3JOQW1tOHN0cS8w?=
 =?utf-8?B?LzNDek01eHVMT1BObkc0bllFNUpaQUtCcDlzeTlMTFovaTNNZGpDZXltN29U?=
 =?utf-8?B?UEtMU0ZRSnk5MERrMzFCNUZ5aFN3UWt1K25hMkVhMzZPME5aZ2ZzNncvVUpt?=
 =?utf-8?B?cHI3RlNpNnFCMWpPMThWVG83S2xnbElCRDFScURpVDM5YTFXblpjNFk3N05q?=
 =?utf-8?B?U1BLRVphOGRUdWh2ZUJDQjAzcjZnNnZhZEdXcDJPVnFMOXpyM3NGS1VtdW1F?=
 =?utf-8?B?YXpyOGRMdndWR2o0aTRwb0dUeE5CZFc2U3lRZ3FGVURodHdWTzJIMWNmczM2?=
 =?utf-8?B?UmEzdDNMdHIyRDVQNzlUMHQvYXB2N0xUSFRxNy9BQ0IwWnNCamJvem52UXZE?=
 =?utf-8?B?cHVOcjQrbVpsY0t3cVhSdXYwOFNacEZUL1RJWVBkdXVrQTB2YkRHUmZON1hS?=
 =?utf-8?B?SUQxT04wTE4wVTBzcUxNQklhZEp5Nnp6dU4yVldOV3BjZXIrUGNEL3dFVVkz?=
 =?utf-8?B?ZkkwQTNxYStuOFNtbm9WdWpITlpPTHlXTHdTQlhPVUh5cHlrcGxFVW0xaGRD?=
 =?utf-8?B?NFhzTEFCM3pFNGxxTWtTbUJ2K2tpelJydEQzVElWTSsvU2llMXpqQ1ArTElB?=
 =?utf-8?B?YUNPeVgrYitIdGtKOEN5Sk9EbHIzQ0hnU2xqcWJMR3BadzhrMEVBeWV4WU43?=
 =?utf-8?B?ajRhRWFwWnNoWStnYnMvaElGSThwM09OSXZJeHZJRi9ETzIrNjNFTGMzbEZ2?=
 =?utf-8?B?Vlphc0pORnlRcGhrZmcydzVET3VwZDVaeVExWEcwd0ozcDY1MXJld25CTnZ0?=
 =?utf-8?B?NDZIZEZKbzBKckx1ellVam9oU1BzaG96cENiY2FNUU00QTEyL2hlOTVMVDBX?=
 =?utf-8?B?TUIrbFcvMXhPck96cUVCSkRTZmhkbzBVZ3ZBcTV3MTZmSlVMaXZYWm1ydWFO?=
 =?utf-8?B?MTBrci9zNUJqa1RmbVJ1WEhJdUF4WjlOWXJSWUxSNitUYkJKOEpVV0Nwdkts?=
 =?utf-8?B?VUdPY3pLVDI5S3hidmE3N0FmdHdzUFNxQ3NoUk5jb0J5SFRSUmJ1cEpKYUpr?=
 =?utf-8?B?UU1GdTduODB3bWhjK2prUzAyM29wS1R5U2dMdFo3U3dXRmsySjBpUmtRWXBr?=
 =?utf-8?B?TVBPK2R4SnI3U0ljTVhNVlE1VWNQZ1M2U3kvYU5TQzZ2dVJST3BBUzA2amJ4?=
 =?utf-8?B?TWlQQVdWc3RmS3lqWUxSbFU0ZE9HdWUweVdPUURWSzBtbTZ5RWNFZUg5ZXJw?=
 =?utf-8?B?MGxIQXZSZU1NZk05Z0R6YVl6VnJnRDBXN2JKbzZSbFlwSTIyMGhwWDFuNHhz?=
 =?utf-8?Q?FVwC00Zrpeqb3l6LPnrFHMpHadAGlaR8Ni5CYh7?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 13:42:13.8504
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5bd79731-80bf-46c2-399b-08dc75adfe81
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6373

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
s/could/can

> 
> This commits implements a new function p2m_put_l2_superpage to handle
> 2MB superpages, specifically for helping put extra references for
> foreign superpages.
> 
> Modify relinquish_p2m_mapping as well to take into account preemption
> when type is foreign memory and order is above 9 (2MB).
I don't see the reasoning why we don't handle 1GB super pages. It would be beneficial
to include such in commit msg as well as in the code.

> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v2:
>  - Do not handle 1GB super page as there might be some issue where
>    a lot of calls to put_page(...) might be issued which could lead
>    to free memory that is a long operation.
> v1:
>  - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
> ---
>  xen/arch/arm/mmu/p2m.c | 57 +++++++++++++++++++++++++++++-------------
>  1 file changed, 40 insertions(+), 17 deletions(-)
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index 41fcca011cf4..29fdb3b87dd0 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>      return rc;
>  }
> 
> -/*
> - * Put any references on the single 4K page referenced by pte.
> - * TODO: Handle superpages, for now we only take special references for leaf
> - * pages (specifically foreign ones, which can't be super mapped today).
> - */
> -static void p2m_put_l3_page(const lpae_t pte)
> +/* Put any references on the single 4K page referenced by mfn. */
> +static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
>  {
> -    mfn_t mfn = lpae_get_mfn(pte);
> -
> -    ASSERT(p2m_is_valid(pte));
> -
>      /*
>       * TODO: Handle other p2m types
>       *
> @@ -771,16 +763,44 @@ static void p2m_put_l3_page(const lpae_t pte)
>       * flush the TLBs if the page is reallocated before the end of
>       * this loop.
>       */
> -    if ( p2m_is_foreign(pte.p2m.type) )
> +    if ( p2m_is_foreign(type) )
>      {
>          ASSERT(mfn_valid(mfn));
>          put_page(mfn_to_page(mfn));
>      }
>      /* Detect the xenheap page and mark the stored GFN as invalid. */
> -    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
> +    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
>          page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
>  }
> 
> +/* Put any references on the superpage referenced by mfn. */
> +static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
> +{
> +    unsigned int i;
> +    unsigned int level_order = XEN_PT_LEVEL_ORDER(3);
If we know the third level order is always 0 no matter the page shift, does it make sense to have this variable?

> +
> +    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
> +    {
> +        p2m_put_l3_page(mfn, type);
> +
> +        mfn = mfn_add(mfn, 1 << level_order);
> +    }
> +}
> +
> +/* Put any references on the page referenced by pte. */
> +static void p2m_put_page(const lpae_t pte, unsigned int level)
> +{
> +    mfn_t mfn = lpae_get_mfn(pte);
> +
> +    ASSERT(p2m_is_valid(pte));
> +
> +    /* We have a second level 2M superpage */
> +    if ( p2m_is_superpage(pte, level) && (level == 2) )
> +        return p2m_put_l2_superpage(mfn, pte.p2m.type);
> +    else if ( level == 3 )
> +        return p2m_put_l3_page(mfn, pte.p2m.type);
> +}
> +
>  /* Free lpae sub-tree behind an entry */
>  static void p2m_free_entry(struct p2m_domain *p2m,
>                             lpae_t entry, unsigned int level)
> @@ -809,9 +829,10 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>  #endif
> 
>          p2m->stats.mappings[level]--;
> -        /* Nothing to do if the entry is a super-page. */
> -        if ( level == 3 )
> -            p2m_put_l3_page(entry);
> +        /* TODO: Currently we don't handle 1GB super-page. */
As a future reader of the code it would be beneficial to say why.

> +        if ( level >= 2 )
> +            p2m_put_page(entry, level);
> +
>          return;
>      }
> 
> @@ -1558,9 +1579,11 @@ int relinquish_p2m_mapping(struct domain *d)
> 
>          count++;
>          /*
> -         * Arbitrarily preempt every 512 iterations.
> +         * Arbitrarily preempt every 512 iterations or when type is foreign
> +         * mapping and the order is above 9 (2MB).
>           */
> -        if ( !(count % 512) && hypercall_preempt_check() )
> +        if ( (!(count % 512) || (p2m_is_foreign(t) && (order > 9))) &&
Instead of (order > 9), use XEN_PT_LEVEL_ORDER(2)

> +             hypercall_preempt_check() )
>          {
>              rc = -ERESTART;
>              break;
> --
> 2.34.1
> 

~Michal



From xen-devel-bounces@lists.xenproject.org Thu May 16 13:49:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:49:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723347.1128107 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bTs-0002SN-2A; Thu, 16 May 2024 13:49:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723347.1128107; Thu, 16 May 2024 13:49:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bTr-0002SG-VL; Thu, 16 May 2024 13:49:11 +0000
Received: by outflank-mailman (input) for mailman id 723347;
 Thu, 16 May 2024 13:49:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7bTq-0002SA-SF
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:49:10 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 11744277-138b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:49:08 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5727dc6d3edso3569800a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 06:49:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bebb6d5sm10530402a12.34.2024.05.16.06.49.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 06:49:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11744277-138b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715867348; x=1716472148; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=hzbt2g5wYfhQ2IZeexNtkD/ZoJ5JqZpxfBpMIie+F30=;
        b=TD4+gLnzv6Fl8gvnt9KGmkd48FbE38sZLbH7bnZBK2MspWx/CLDQKMsD3vsc/mkad2
         bARTudkf6io1QHjwANFSrMnf3msIHmx2eKR1gyaVlBnqdBteeIlnCGJE+z+fBB0pBC+5
         fQgXdBiDfQgPHG1QNF5CBsPOYBlYqz7jZmvEARdw3h4VAreY/O5mEV5b1ltER89epDSB
         T5jbRYFbUVJpjBxl82caRC+zX+1tc5X4UIR75sEBsQEbwViv+it2Rir5HKnqQFvHPaSB
         Sh3ddR1bPmeQmSmzAupqDfyTb5nTVyUhWLov/6u1kFDvTOmF0f2glHDdOdEIkOGa5WDl
         TrTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715867348; x=1716472148;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hzbt2g5wYfhQ2IZeexNtkD/ZoJ5JqZpxfBpMIie+F30=;
        b=JaksHCnudHTTXDGwdf4PfWlhGsw1QhtASkUu/2XpjOoLsa/lFFZ1P2FHi3doxMEGCm
         kJ/tPITerPZ1fBgSf391nxDYaE9+MDxZYbD3cO5HY1rN9mk6nRZB2H6gQndVWis6O+Ma
         l5rKy82g7/q3qSod02wfPUlbpipcErzGrUZChO2rakJelTy7VfYIGfpFXZv7IYu6xuoS
         lZ9DY1GhlG0QlvORf0LRwuU8Ct9ek00oaWERVCylB9QxKnQiY++yf1J+YJX/fCr0De8M
         ItuLGak1Rh7tuyWyW17/g9B0C+UL4TXxIDQWgCJHkM1VCubeOM27cDKu7DILo3w1LO4i
         fy0w==
X-Forwarded-Encrypted: i=1; AJvYcCXVWgPcKEJACP6yjwEAfAcs/iss+nQR+/l08y1ZyTs8Z7YAfJuLL2XogFaEGYWad9B+o/avGsiHATq0vDdZqjoeg+wvj7NUgUubrdqmaK0=
X-Gm-Message-State: AOJu0YwEiO2ZO+SXQFNKCYZWVMbU4YJZb8xBuJklebinkP/KD9xHz0/g
	YyHPVx3medy1VK++i1kNTXMHXxGyhYMiOl5ZG6g9CPcqrfsRpipFZd2rBlmrLA==
X-Google-Smtp-Source: AGHT+IHG9SfYYixUR7aDMq79JzGLosE2BMZIndbhbghaN7aoOm4Yy0X7phgYf/VxDVyXWFXYayNlqw==
X-Received: by 2002:a50:baeb:0:b0:572:6675:258f with SMTP id 4fb4d7f45d1cf-5734d5ceb2dmr12470457a12.24.1715867348209;
        Thu, 16 May 2024 06:49:08 -0700 (PDT)
Message-ID: <34f6ac6d-0bad-478a-86c1-a124df5504c8@suse.com>
Date: Thu, 16 May 2024 15:49:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
To: Jiqian Chen <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
 Huang Rui <Ray.Huang@amd.com>, xen-devel@lists.xenproject.org
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-4-Jiqian.Chen@amd.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516095235.64128-4-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 11:52, Jiqian Chen wrote:
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -76,6 +76,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_unmap_pirq:
>          break;
>  
> +    case PHYSDEVOP_setup_gsi:
> +        if ( !is_hardware_domain(currd) )
> +            return -EOPNOTSUPP;
> +        break;
> +
>      case PHYSDEVOP_eoi:
>      case PHYSDEVOP_irq_status_query:
>      case PHYSDEVOP_get_free_pirq:

Below here we have a hardware-domain-only block already. Any reason not
to add to there? Yes, that uses -ENOSYS. Imo that wants changing anyway,
but even without that to me it would seem more consistent overall to have
the new case simply added there.

Just to double check: Is there a respective Linux patch already (if so,
cross-linking the patches may be helpful)? Or does PVH Linux invoke the
sub-op already anyway, just that so far it fails?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:52:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:52:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723350.1128117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bXM-0004z8-GF; Thu, 16 May 2024 13:52:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723350.1128117; Thu, 16 May 2024 13:52:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bXM-0004yt-Db; Thu, 16 May 2024 13:52:48 +0000
Received: by outflank-mailman (input) for mailman id 723350;
 Thu, 16 May 2024 13:52:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6R0U=MT=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s7bXL-0004v6-DZ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:52:47 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20610.outbound.protection.outlook.com
 [2a01:111:f400:7e88::610])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9285b3e0-138b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:52:46 +0200 (CEST)
Received: from SJ0PR13CA0129.namprd13.prod.outlook.com (2603:10b6:a03:2c6::14)
 by MW3PR12MB4394.namprd12.prod.outlook.com (2603:10b6:303:54::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29; Thu, 16 May
 2024 13:52:42 +0000
Received: from CO1PEPF000044F0.namprd05.prod.outlook.com
 (2603:10b6:a03:2c6:cafe::9e) by SJ0PR13CA0129.outlook.office365.com
 (2603:10b6:a03:2c6::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.12 via Frontend
 Transport; Thu, 16 May 2024 13:52:41 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044F0.mail.protection.outlook.com (10.167.241.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 13:52:40 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:52:40 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 08:52:39 -0500
Received: from [172.25.174.161] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 08:52:39 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9285b3e0-138b-11ef-909d-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FvDcpkHR/OqiTxtQHbcvsg5iNPgiyBsegq95yESeCfpoV8ZZJ05xJJVWj6LZnn/X4YWQzT+BZgMDxCeQHiSedD+wBPWLmiSEGGuZZEDLDw660cA5LAkBntT0+jek3luv4jOOZnYtFsrelufIa0TiADTbkd+bhl7nZIN7a5R+1heR4hXDz34vCCv6GsaYniHXQFNu2Rxh/bvF/OuBs9b4jwaiWtX2SnbsdT/M50wLkjVe4WqhxUElLLzOyJ2BgMkt1LXjdEaZP/VyUTTp5Dv67V9u2zQeWCFoysHi6Q9HkFdktneU2ASxt3JmJYzxlX8Zks06I5aKiSWR2jcKK8TpUg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dUj5FOqbKH7wwT1u1EJdGkPcJ91VQKVB3nhPZyk750A=;
 b=jfRU/A5eTVg+yAxNntV+2VcNGBJFxJ2GLRmm97B9Sfmvd7D8yOFaCC8F3ZzftmKf7H5k6joL1FuCTtYm8oT6kaiNPDCmYVAHupgEe+U44F8+3weuBmou9xMmQwdp0mJY5HLzadhN7nsWLrRoYVuuhYWoE21Z0wXIP0YfTP6kY/rPIl2dswMRLAE70XMphoZnpdSIBeWc1F5z91/Vy7cdwniUia2aubkJmsqj3SASSYGL3ZH3R4usRtyQ97Li0E/c1+NCP/YeV45pnxhoyvkzSgbK6n85ttRsfVZaFOK0YH4UxrfvyLzxdhUBjmugJP5hnZpkUPrNguR0juiwb5uoMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dUj5FOqbKH7wwT1u1EJdGkPcJ91VQKVB3nhPZyk750A=;
 b=PZ6H76y2FKlDS/nyKhhA5oPL89xTRzFE7QSpYVQ9LSMKxvxWhV+dut02Cen9Qbd1vCbgo8q04CWK7YM9zQdmz0mT96RRJ1umGfamDR1R2n7HrbVVIUAApn581qsbLzQ/25HYHu7nyZ4ikdOf3bxwpvsziT9LyweCqp3QtU8GOKE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <9678073f-82d5-4402-b5a0-e24985c1446b@amd.com>
Date: Thu, 16 May 2024 09:52:38 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] hotplug: Restore block-tap phy compatibility
To: Jan Beulich <jbeulich@suse.com>, Jason Andryuk <jandryuk@gmail.com>
CC: Anthony PERARD <anthony@xenproject.org>, <xen-devel@lists.xenproject.org>
References: <20240516022212.5034-1-jandryuk@gmail.com>
 <64083e01-edf1-4395-a9d7-82e82d220de7@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <64083e01-edf1-4395-a9d7-82e82d220de7@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F0:EE_|MW3PR12MB4394:EE_
X-MS-Office365-Filtering-Correlation-Id: 84ec1f20-3d86-4d9e-0bc7-08dc75af7453
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RnNkRGdpRzR0WldmdkVsYWV6aWRDWGZ1K2NscHhBVHJwdjd6THhiWjRGcEF0?=
 =?utf-8?B?aWxvWjFYWlVLck5CZkU5WGVoNEpubENYaCt2NGVFeGdoTTJ5TnJ3OEVXNHNj?=
 =?utf-8?B?RVpJd2F1S3ROaDVmSHFWTGcrRUpNd1Zkcm5yWWdzazlFeitEbFNKR1FVQjRQ?=
 =?utf-8?B?ZjNRVGd6b2E3NnR3eGtuajFLU3llVE8wanl6cGFoZDM5YlpYYzlNYlBFczNO?=
 =?utf-8?B?d2NTYldzS0JhcTNXaitsWGFzTGd5QVVub1FKVExkdGgxQjNLcUVhdXhFM0Fa?=
 =?utf-8?B?ODJhYTdESndaVjFYQU5hZUdBeDJIQU5zTnB4YjR6eW5yTG1iOTFOV0RLbTE0?=
 =?utf-8?B?TWZuaVRhVnp1RENFUmw0aGNaU1kvSHdMQVlSRUFpSXBadmZ3WWZxcldCSnRv?=
 =?utf-8?B?aVhyVXJVVmRZSWd5Z2RTclNiMWhCUmIxaFlNMzB1WWV1SVc4d3g0cXQyY0hV?=
 =?utf-8?B?Y21heFlwWFE3L0ErN3RWVkpXcm5palZNNlZoZjB0VWhKUURyTnJKSjdSbDBk?=
 =?utf-8?B?MStEaXBtaWZhWWdRZ2tzME1OS3JkTlBLRFExa3JsNzVRbno4bitDWStrZXRu?=
 =?utf-8?B?UVNCanE4M1drRDBFVDNMSnhob29vSkJlTmtWYnNSc3A3anBCUXpYN0FsaXdZ?=
 =?utf-8?B?eDd1ek00RC9WaU9aaVVOWjdyV1Zxd0FaOEtVa0hLSEhhcU41ajNSS3oyd3I5?=
 =?utf-8?B?dHRIdVZPYlJKTkxQVXlTWmIxajVDajVDam4vWUVJbjh5Vk9YWG5RSnIyYU93?=
 =?utf-8?B?Q2hHS09naXVFR2FCY0sxekV1YnFCdEhUWHJCTnNxcDFXY0EvRDVTdzZRQVdB?=
 =?utf-8?B?NlpUOXlxWXFZYnZyWUtiY082ZTVHdExxd09wZTdDVStFSE5ENmFnWjhIZk9E?=
 =?utf-8?B?U1p6Q2x0RWxwaU9SRzFoRUhZNFlSaHB0TDF4Tm5ES1BiOVhHWFJET3VNbVJ2?=
 =?utf-8?B?dU41dmszamJzbzgraXRvUk9wZlJiL1lEdlkwN2o0VFNBcDhzdDBKNU54TzUr?=
 =?utf-8?B?ZmszbXo2MEJ6ZHJnSXkra2w0bGozMlNwYUlGa2tDaE5TMHEyL2xmRFNYVzdi?=
 =?utf-8?B?bE1WeHNLamhvRXczN0cxZUJJWHZjSFVQemJNUDBuTkNnL2Q3dWwrZFlXdFlq?=
 =?utf-8?B?MlBMWHlwMFBtRFBCZmZlMlJ1enJpMmJnSzBSYWNqdFpTaW1Yclp2UE52TlFw?=
 =?utf-8?B?SEFKU1hrUlBlekVYc3ptSzJXb3dpRk5JUGU1dUVsdlkyL1lvdFNHTW4rQUJT?=
 =?utf-8?B?V3psQWxka0FFb1puNHNFVmswejljMTVUTTBVYkxKV2p4cHlnQko5TTZpZDFr?=
 =?utf-8?B?Q2NjWHBQMHplbmMrSHYwVDFSV21JSUsyTGhDaDNPaFNXWDBWWXAzU2ZHbGY1?=
 =?utf-8?B?Nk1FNG50dWwvMDZEYThMN0Rab3pPMHlTNmt4SWZYWUVPNlBFWkkvMlVwWER2?=
 =?utf-8?B?MUUxMDA4TGwvQmtRZDBpSGVDRnZ3ZEJYekNGUml4OGxpSEgwT0tHaGdPVVlC?=
 =?utf-8?B?Ny9sZHZhMjJHUUg5Mld2N2Y3cE5vWFF2bkM4ekNaY0huUlA4eEV4OFQ1bHFP?=
 =?utf-8?B?MnMzZHdPcEw5M3ZVNjZMOHprcXZPc3h3aVJ0SGZXalVFcDdlaFBucGZWZFRP?=
 =?utf-8?B?bFVtZ3NYSFVtYzRKbThvdFdhVEIza21JVWx1Z0RMMngrSVU3OVF5NEhmRFpO?=
 =?utf-8?B?U0Q2SjhSMlRvdEhRSjlrRTh0dWI1UWVmS1FTZVZ1ZmQrdzF6TGxMUGNmNHkr?=
 =?utf-8?B?Zm1CVUhnZmtub3VRQUxOeitXa3JXWWMrT2o5eHZ3RVRkbGlsWlhuZzk3eHM2?=
 =?utf-8?B?cmFpWlltQy9XM2k2QjZzdjE2TXgrZ21RbzdhanduVDJWYkpycVRaT005b1d5?=
 =?utf-8?Q?SoxNvPnQBp0Wu?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 13:52:40.9062
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 84ec1f20-3d86-4d9e-0bc7-08dc75af7453
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4394

On 2024-05-16 03:41, Jan Beulich wrote:
> On 16.05.2024 04:22, Jason Andryuk wrote:
>> From: Jason Andryuk <jason.andryuk@amd.com>
>>
>> From: Jason Andryuk <jason.andryuk@amd.com>
> 
> Two identical From: (also in another patch of yours, while in yet another one
> you have two _different_ ones, when only one will survive into the eventual
> commit anyway)?

Sorry about that.  Since I was sending from my gmail account, I thought 
I needed explicit From: lines to ensure the authorship was listed w/ 
amd.com.  I generated the patches with `git format-patch --from`, to get 
the explicit From: lines, and then sent with `git send-email`.  The 
send-email step then inserted the additional lines.  I guess it added 
 From amd.com since I had changed to that address in .gitconfig.

>> backendtype=phy using the blktap kernel module needs to use write_dev,
>> but tapback can't support that.  tapback should perform better, but make
>> the script compatible with the old kernel module again.
>>
>> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
> 
> Should there be a Fixes: tag here?

That makes sense.

Fixes: 76a484193d ("hotplug: Update block-tap")

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723356.1128126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-0005u7-8G; Thu, 16 May 2024 13:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723356.1128126; Thu, 16 May 2024 13:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-0005u0-56; Thu, 16 May 2024 13:59:42 +0000
Received: by outflank-mailman (input) for mailman id 723356;
 Thu, 16 May 2024 13:59:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be0-0005tl-CV
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:40 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 87df4c9d-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:37 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.nyi.internal (Postfix) with ESMTP id 46DEB1380191;
 Thu, 16 May 2024 09:59:36 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Thu, 16 May 2024 09:59:36 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:35 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87df4c9d-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1715867976; x=1715954376; bh=dUbs2+uz0vOo0vFkty7Eibm00fKFXkvo
	gmBd4lo7OJc=; b=Y7MzFKiRBBoELttX+rtt5aF+4Vyf2+WvK2bF1bUl0ak1eCm6
	4uGi0V5I5XyBGe3U2xdaPQ18XL1HSkiCXvqD+yY/mSq2I9XGXxwaOiJzn9/FAZfX
	WhOocCnR5OJ4zdpG3DRJjSeV/ARKzQhZSp2DuWE0+moMh+0MMk2qR15Wpnem7Pw7
	pk59BqedT3/9dqfHvuYOdT/5HqSiV53ZIyf7YAvH/L83GBSjTsG5NYHOWAg7abve
	cdg0Dd/xPy2njRRbbDgKDHexrenAfwM7FXiomNx0Rsp/f6ZCQkFeapeNjc3SNZkD
	ij1k2ow4HPXo0jSSnPlaudgVQhbwISCMbH66Ng==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm3; t=1715867976; x=1715954376; bh=dUbs2+uz0vOo0
	vFkty7Eibm00fKFXkvogmBd4lo7OJc=; b=K28l3qbsnX/EdSH0aACq7vyxiR9N1
	xDYmJJl079yRag3/DEh01qUB6RnNN5itZ6oTvTO/0N23u7ol2UqqxR8CC0qCm4fD
	vLwBS0MyPaKoQUWijKNkQazJTb54Vw/tCw4tyUsV2GgOkaBJWhOvyvmNLOU1nNLm
	bMTVjD57qLKcDmXqSA1fj5tIs2k5DodOgPD6DD0af0G2nRGatwaJbmjHvLLHkQa6
	2PduqV7nivmk54X+cn7ietcmQ5hF2AuYXoRzIG//OyNEHWH49FlCv8rUtwpd7vAG
	Thrr9IkTUfqbww+ePNM1GePQVVhoVlPjOns4pjmEGYPdvTqrTerp5k9+w==
X-ME-Sender: <xms:SBFGZqlDYr84dAzUhFO9cyX4rosxk-mJOpwoexU9jv0bMKXbsPJHZA>
    <xme:SBFGZh3BYvsbwxEOELmxEBzkhKESeRryZXe4vb4d4Z3DoznZuBX8iPU_KLmBgTQt9
    VcoKSxoimboyw>
X-ME-Received: <xmr:SBFGZoqfqObvfOOatbTqLGR5A59Fws88HJiXyYKK7ts_ZNfvbuDUGoXsFkcFl4DTFirqGNLklwm5PjAzp0e8kYeyMSj6wpPhOmo3FY2ezyi1RyRr-kQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggt
    khhiuceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
    eqnecuggftrfgrthhtvghrnhepleekhfduleetleelleetteevfeefteffkeetteejheel
    gfegkeelgeehhfdthedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg
    hilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdr
    tghomh
X-ME-Proxy: <xmx:SBFGZulVPbmBrRMvnuumobHk4wK5T_1SI8cFZRl_0ofWGaGenW9b8Q>
    <xmx:SBFGZo2uXHfSyRXcuT_0ATIyaNOP3B1Bco-H2SPdGZFY4uEfPo1Q4Q>
    <xmx:SBFGZlucK8hN986FCakJAQ0UTrJN4o-iN46nmxTI0A3zAuz5cbHHIA>
    <xmx:SBFGZkXmjOnXK6Zem32oYFNTowLInLaF8eXU-yD4xjjs6Fjn-hPmRg>
    <xmx:SBFGZl_VRzYgYr_hTVVPoZKOLX8FAJnoT1PtrGK1V7zJgovMUQBmHeFW>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH 00/12] automation: Add build and test for Linux stubdomain
Date: Thu, 16 May 2024 15:58:21 +0200
Message-ID: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Initial patches can be applied independently but all are needed before the
"automation: Add linux stubdom build and smoke test" patch.
And later "libxl: Allow stubdomain to control interupts of PCI device" and
"automation: update kernel for x86 tests" is needed before PCI passthrough
test (but both can be committed earlier as they don't depend on others).

See the "automation: Add linux stubdom build and smoke test" patch description
for more details.

Note the Alpine version bump requires rebuilding containers, but so does the
actual test patch (extra dependencies), so it probably makes sense to do it at
the same time.

Marek Marczykowski-Górecki (12):
  automation: include domU kernel messages in the console output log
  automation: update fedora build to F39
  automation: switch to alpine:3.19
  automation: increase verbosity of starting a domain
  automation: prevent grub unpacking initramfs
  RFC: automation: Add linux stubdom build and smoke test
  libxl: Allow stubdomain to control interupts of PCI device
  automation: update kernel for x86 tests
  WIP: automation: temporarily add 'testlab' tag to stubdomain build
  automation: stubdom test with PCI passthrough
  automation: stubdom test with boot from CDROM
  [DO NOT MERGE] switch to my containers fork

 automation/build/alpine/3.18-arm64v8.dockerfile           |  49 +--
 automation/build/alpine/3.18.dockerfile                   |  51 +--
 automation/build/alpine/3.19-arm64v8.dockerfile           |  52 ++-
 automation/build/alpine/3.19.dockerfile                   |  60 +++-
 automation/build/fedora/29.dockerfile                     |  46 +--
 automation/build/fedora/39.dockerfile                     |  46 ++-
 automation/gitlab-ci/build.yaml                           |  85 ++--
 automation/gitlab-ci/test.yaml                            |  87 ++--
 automation/scripts/build                                  |  12 +-
 automation/scripts/containerize                           |   4 +-
 automation/scripts/qemu-alpine-x86_64.sh                  |   2 +-
 automation/scripts/qemu-smoke-dom0-arm32.sh               |   2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh               |   2 +-
 automation/scripts/qubes-x86-64.sh                        | 153 ++++++-
 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile |  65 +---
 automation/tests-artifacts/alpine/3.18.dockerfile         |  66 +---
 automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile |  65 +++-
 automation/tests-artifacts/alpine/3.19.dockerfile         |  72 +++-
 automation/tests-artifacts/kernel/6.1.19.dockerfile       |  40 +--
 automation/tests-artifacts/kernel/6.1.90.dockerfile       |  40 ++-
 tools/libs/light/libxl_pci.c                              |   8 +-
 21 files changed, 614 insertions(+), 393 deletions(-)
 delete mode 100644 automation/build/alpine/3.18-arm64v8.dockerfile
 delete mode 100644 automation/build/alpine/3.18.dockerfile
 create mode 100644 automation/build/alpine/3.19-arm64v8.dockerfile
 create mode 100644 automation/build/alpine/3.19.dockerfile
 delete mode 100644 automation/build/fedora/29.dockerfile
 create mode 100644 automation/build/fedora/39.dockerfile
 delete mode 100644 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
 delete mode 100644 automation/tests-artifacts/alpine/3.18.dockerfile
 create mode 100644 automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile
 create mode 100644 automation/tests-artifacts/alpine/3.19.dockerfile
 delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/6.1.90.dockerfile

base-commit: 319a5125ca2649e6eb95670b4d721260025c187d
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723360.1128167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be6-0006vC-F8; Thu, 16 May 2024 13:59:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723360.1128167; Thu, 16 May 2024 13:59:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be6-0006us-A9; Thu, 16 May 2024 13:59:46 +0000
Received: by outflank-mailman (input) for mailman id 723360;
 Thu, 16 May 2024 13:59:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be4-0006bS-Hh
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:44 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 89f4f4df-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:41 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id 20B841140102;
 Thu, 16 May 2024 09:59:40 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Thu, 16 May 2024 09:59:40 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:38 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89f4f4df-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867980; x=1715954380; bh=v8UvuOInE1
	5TzCQTzCfX1F4m6ztY1tE/KixumMSvRV0=; b=4S2FrE7dCFylYj9l8v4/KoW/kk
	50P+ca1qmgc0bDalyGezoapVGmT9vZzM6DV74E3ZSqbh6pxILDVX9PqCIDFsj+de
	F6AKT0QcrpaOyFHVGoAwzo3T9Qk7Va9p9f2OTmFS6Ah+FGlviP4o6EQZSzZ1XcTv
	W2j1I1O+5coU9ELiZsv+NgV8waOkWmI9rJacgclW9uYgvKo0GFT6WYQZvz0LRIP8
	aN4JQfKV689rIrvm7V7Y5c/zkDpjCTSAOgG0D1XiHGoKoSMV9N94BQIERY9ruGq6
	nwGB4E1OMqgu1GxLLW1KOpsN70b/h25sGUDiKy6WKwnFs6xYuc+x0afO9zlQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867980; x=
	1715954380; bh=v8UvuOInE15TzCQTzCfX1F4m6ztY1tE/KixumMSvRV0=; b=C
	4BvatJCXI7ML75S7WioE4RI79cKW0HLmwuCK2tWmSOwRo2rIV4T/E9Lm+gx2rFO/
	72sLRdnUG5dDKzaNWygJzmdi1h65nPUcoC/93maeIUP1DgG5tbrESixMk/x73Fyh
	RzKs5kq+8k3qqSABqshvCwLONZAH7/jHl/KMf2uVHHI347nxjLrk5AWg1U8f0U7x
	5tgmYJdnrDAd+1v3YFOqCK2/nhg5ut1+KWkh7u4nMXgn53tFoWaCcQOL5vJ4OrLL
	YspxLw88Z44YCS8ret+IGAf/yP/sHNR8UjJ8W7/5+Ehpa7BfXNUFHBY9nn5eL6KJ
	unSsCZjXXfr0nlX1ll+2g==
X-ME-Sender: <xms:SxFGZv-12Q-BY79hkkiTZGJ4WHiKRblljWOnkJV5vCA9SAxZWPq2VA>
    <xme:SxFGZruzP4rLAdYjH0X15K2S_ewEPTvsOM-hX46QsqKscL47nqYdWTZFq-V5rs2Nr
    jy7CiOtnuHl9g>
X-ME-Received: <xmr:SxFGZtDgXer1czBsJm_SmD7AAGdyZTI6WC0aqmbwngvYIN-9Ekz5nwpjiomJExbRBaH1W6P_nbItJ9qCzz3EPcebz6SZLZ9FGlRsToTBax2DcnzUTbg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeel
    fedtfefgfffghfevhfehvdeileehheffueekieetfeffhfetgefggfejudfggeenucffoh
    hmrghinhepghhithhlrggsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr
    rghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhngh
    hslhgrsgdrtghomh
X-ME-Proxy: <xmx:SxFGZrcElwZq7f0rigzM6n0ZLBqO3N9OFZ9pdsiHZ_EtroNPsQXRnA>
    <xmx:TBFGZkMwgcPwheHON_DVbe9OcWH2BpY1lXSqHBa_8PpudGu9vWft5w>
    <xmx:TBFGZtlaCGiAbPZ9JjWa0THYtbflsXzXoS7tR8mbJYQCwYsSuVjlnw>
    <xmx:TBFGZus11y16llLwd2oamkKVN7ypqlOn9v4VisvToVh_wW4bo6TYlA>
    <xmx:TBFGZprihWeehxwM4OzIEGNIUbTLf88nRKjt5HfDp1qQ7uSGxJ5w4-1J>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 03/12] automation: switch to alpine:3.19
Date: Thu, 16 May 2024 15:58:24 +0200
Message-ID: <a681e1d414c0735128cdde70d6b521d8c96e6e45.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Alpine 3.19 is needed for upcoming stubdomain tests, as linux stubdomain
build requires dracut-core package (dracut-install tool specifically)
which isn't available in 3.18. While technically it will be needed only
in the x86_64 builds, switch Alpine version everywhere for uniformity.
Note this bumps kernel version requirement on docker runners -
dracut-install uses faccessat2() syscall which was introduced in Linux
5.8.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/build/alpine/3.18-arm64v8.dockerfile           | 49 +------
 automation/build/alpine/3.18.dockerfile                   | 51 +------
 automation/build/alpine/3.19-arm64v8.dockerfile           | 49 ++++++-
 automation/build/alpine/3.19.dockerfile                   | 51 ++++++-
 automation/gitlab-ci/build.yaml                           | 56 +++----
 automation/gitlab-ci/test.yaml                            | 52 +++---
 automation/scripts/containerize                           |  4 +-
 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile | 65 +--------
 automation/tests-artifacts/alpine/3.18.dockerfile         | 66 +--------
 automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile | 65 ++++++++-
 automation/tests-artifacts/alpine/3.19.dockerfile         | 67 ++++++++-
 11 files changed, 288 insertions(+), 287 deletions(-)
 delete mode 100644 automation/build/alpine/3.18-arm64v8.dockerfile
 delete mode 100644 automation/build/alpine/3.18.dockerfile
 create mode 100644 automation/build/alpine/3.19-arm64v8.dockerfile
 create mode 100644 automation/build/alpine/3.19.dockerfile
 delete mode 100644 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
 delete mode 100644 automation/tests-artifacts/alpine/3.18.dockerfile
 create mode 100644 automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile
 create mode 100644 automation/tests-artifacts/alpine/3.19.dockerfile

diff --git a/automation/build/alpine/3.18-arm64v8.dockerfile b/automation/build/alpine/3.18-arm64v8.dockerfile
deleted file mode 100644
index 91e90220240f..000000000000
--- a/automation/build/alpine/3.18-arm64v8.dockerfile
+++ /dev/null
@@ -1,49 +0,0 @@
-FROM --platform=linux/arm64/v8 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apk --no-cache add \
-  \
-  # xen build deps
-  argp-standalone \
-  autoconf \
-  bash \
-  bison \
-  curl \
-  dev86 \
-  dtc-dev \
-  flex \
-  gcc \
-  git \
-  iasl \
-  libaio-dev \
-  libfdt \
-  linux-headers \
-  make \
-  musl-dev  \
-  ncurses-dev \
-  ocaml \
-  ocaml-findlib \
-  patch  \
-  python3-dev \
-  py3-setuptools \
-  texinfo \
-  util-linux-dev \
-  xz-dev \
-  yajl-dev \
-  zlib-dev \
-  \
-  # qemu build deps
-  glib-dev \
-  libattr \
-  libcap-ng-dev \
-  pixman-dev \
-  # qubes test deps
-  openssh-client \
-  fakeroot \
diff --git a/automation/build/alpine/3.18.dockerfile b/automation/build/alpine/3.18.dockerfile
deleted file mode 100644
index 8d5dac05b01f..000000000000
--- a/automation/build/alpine/3.18.dockerfile
+++ /dev/null
@@ -1,51 +0,0 @@
-FROM --platform=linux/amd64 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apk --no-cache add \
-  \
-  # xen build deps
-  argp-standalone \
-  autoconf \
-  bash \
-  bison \
-  clang \
-  curl \
-  dev86 \
-  flex \
-  g++ \
-  gcc \
-  git \
-  grep \
-  iasl \
-  libaio-dev \
-  libc6-compat \
-  linux-headers \
-  make \
-  musl-dev  \
-  ncurses-dev \
-  ocaml \
-  ocaml-findlib \
-  patch  \
-  python3-dev \
-  py3-setuptools \
-  texinfo \
-  util-linux-dev \
-  xz-dev \
-  yajl-dev \
-  zlib-dev \
-  \
-  # qemu build deps
-  glib-dev \
-  libattr \
-  libcap-ng-dev \
-  ninja \
-  pixman-dev \
-  # livepatch-tools deps
-  elfutils-dev \
diff --git a/automation/build/alpine/3.19-arm64v8.dockerfile b/automation/build/alpine/3.19-arm64v8.dockerfile
new file mode 100644
index 000000000000..158cf465a9ff
--- /dev/null
+++ b/automation/build/alpine/3.19-arm64v8.dockerfile
@@ -0,0 +1,49 @@
+FROM --platform=linux/arm64/v8 alpine:3.19
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apk --no-cache add \
+  \
+  # xen build deps
+  argp-standalone \
+  autoconf \
+  bash \
+  bison \
+  curl \
+  dev86 \
+  dtc-dev \
+  flex \
+  gcc \
+  git \
+  iasl \
+  libaio-dev \
+  libfdt \
+  linux-headers \
+  make \
+  musl-dev  \
+  ncurses-dev \
+  ocaml \
+  ocaml-findlib \
+  patch  \
+  python3-dev \
+  py3-setuptools \
+  texinfo \
+  util-linux-dev \
+  xz-dev \
+  yajl-dev \
+  zlib-dev \
+  \
+  # qemu build deps
+  glib-dev \
+  libattr \
+  libcap-ng-dev \
+  pixman-dev \
+  # qubes test deps
+  openssh-client \
+  fakeroot \
diff --git a/automation/build/alpine/3.19.dockerfile b/automation/build/alpine/3.19.dockerfile
new file mode 100644
index 000000000000..0be6d7c85fe7
--- /dev/null
+++ b/automation/build/alpine/3.19.dockerfile
@@ -0,0 +1,51 @@
+FROM --platform=linux/amd64 alpine:3.19
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apk --no-cache add \
+  \
+  # xen build deps
+  argp-standalone \
+  autoconf \
+  bash \
+  bison \
+  clang \
+  curl \
+  dev86 \
+  flex \
+  g++ \
+  gcc \
+  git \
+  grep \
+  iasl \
+  libaio-dev \
+  libc6-compat \
+  linux-headers \
+  make \
+  musl-dev  \
+  ncurses-dev \
+  ocaml \
+  ocaml-findlib \
+  patch  \
+  python3-dev \
+  py3-setuptools \
+  texinfo \
+  util-linux-dev \
+  xz-dev \
+  yajl-dev \
+  zlib-dev \
+  \
+  # qemu build deps
+  glib-dev \
+  libattr \
+  libcap-ng-dev \
+  ninja \
+  pixman-dev \
+  # livepatch-tools deps
+  elfutils-dev \
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 69665ec5b11f..b186289bbd82 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -258,9 +258,9 @@
 
 # Arm test artifacts
 
-alpine-3.18-arm64-rootfs-export:
+alpine-3.19-arm64-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18-arm64v8
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19-arm64v8
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
   artifacts:
@@ -318,9 +318,9 @@ qemu-system-ppc64-8.1.0-ppc64-export:
 
 # x86_64 test artifacts
 
-alpine-3.18-rootfs-export:
+alpine-3.19-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
   artifacts:
@@ -344,15 +344,15 @@ kernel-6.1.19-export:
 
 # Build jobs needed for tests
 
-alpine-3.18-gcc:
+alpine-3.19-gcc:
   extends: .gcc-x86-64-build
   variables:
-    CONTAINER: alpine:3.18
+    CONTAINER: alpine:3.19
 
-alpine-3.18-gcc-debug:
+alpine-3.19-gcc-debug:
   extends: .gcc-x86-64-build-debug
   variables:
-    CONTAINER: alpine:3.18
+    CONTAINER: alpine:3.19
 
 debian-stretch-gcc-debug:
   extends: .gcc-x86-64-build-debug
@@ -435,57 +435,57 @@ debian-bookworm-gcc-debug-arm64-randconfig:
     CONTAINER: debian:bookworm-arm64v8
     RANDCONFIG: y
 
-alpine-3.18-gcc-arm64:
+alpine-3.19-gcc-arm64:
   extends: .gcc-arm64-build
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
 
-alpine-3.18-gcc-debug-arm64:
+alpine-3.19-gcc-debug-arm64:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
 
-alpine-3.18-gcc-arm64-randconfig:
+alpine-3.19-gcc-arm64-randconfig:
   extends: .gcc-arm64-build
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     RANDCONFIG: y
 
-alpine-3.18-gcc-debug-arm64-randconfig:
+alpine-3.19-gcc-debug-arm64-randconfig:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     RANDCONFIG: y
 
-alpine-3.18-gcc-debug-arm64-staticmem:
+alpine-3.19-gcc-debug-arm64-staticmem:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
       CONFIG_EXPERT=y
       CONFIG_UNSUPPORTED=y
       CONFIG_STATIC_MEMORY=y
 
-alpine-3.18-gcc-debug-arm64-static-shared-mem:
+alpine-3.19-gcc-debug-arm64-static-shared-mem:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
       CONFIG_UNSUPPORTED=y
       CONFIG_STATIC_MEMORY=y
       CONFIG_STATIC_SHM=y
 
-alpine-3.18-gcc-debug-arm64-boot-cpupools:
+alpine-3.19-gcc-debug-arm64-boot-cpupools:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
-alpine-3.18-gcc-debug-arm64-earlyprintk:
+alpine-3.19-gcc-debug-arm64-earlyprintk:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
       CONFIG_EARLY_UART_CHOICE_PL011=y
       CONFIG_EARLY_UART_BASE_ADDRESS=0x9000000
@@ -588,15 +588,15 @@ debian-bookworm-gcc-arm64-cppcheck:
 
 # Build jobs not needed for tests
 
-alpine-3.18-clang:
+alpine-3.19-clang:
   extends: .clang-x86-64-build
   variables:
-    CONTAINER: alpine:3.18
+    CONTAINER: alpine:3.19
 
-alpine-3.18-clang-debug:
+alpine-3.19-clang-debug:
   extends: .clang-x86-64-build-debug
   variables:
-    CONTAINER: alpine:3.18
+    CONTAINER: alpine:3.19
 
 archlinux-gcc:
   extends: .gcc-x86-64-build
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index ad249fa0a5d9..f62d426a8d34 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -3,7 +3,7 @@
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
 
 .arm64-test-needs: &arm64-test-needs
-  - alpine-3.18-arm64-rootfs-export
+  - alpine-3.19-arm64-rootfs-export
   - kernel-5.19-arm64-export
   - qemu-system-aarch64-6.0.0-arm64-export
 
@@ -11,7 +11,7 @@
   - qemu-system-aarch64-6.0.0-arm32-export
 
 .x86-64-test-needs: &x86-64-test-needs
-  - alpine-3.18-rootfs-export
+  - alpine-3.19-rootfs-export
   - kernel-6.1.19-export
 
 .qemu-arm64:
@@ -100,7 +100,7 @@
   extends: .test-jobs-common
   variables:
     # the test controller runs on RPi4
-    CONTAINER: alpine:3.18-arm64v8
+    CONTAINER: alpine:3.19-arm64v8
     LOGFILE: smoke-test.log
     PCIDEV: "03:00.0"
     PCIDEV_INTR: "MSI-X"
@@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
     - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.19-gcc-arm64
 
 xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
   extends: .xilinx-arm64
@@ -157,7 +157,7 @@ xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
     - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh gem-passthrough 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.19-gcc-arm64
 
 adl-smoke-x86-64-gcc-debug:
   extends: .adl-x86-64
@@ -165,7 +165,7 @@ adl-smoke-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 adl-smoke-x86-64-dom0pvh-gcc-debug:
   extends: .adl-x86-64
@@ -173,7 +173,7 @@ adl-smoke-x86-64-dom0pvh-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh dom0pvh 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 adl-suspend-x86-64-gcc-debug:
   extends: .adl-x86-64
@@ -181,7 +181,7 @@ adl-suspend-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh s3 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 adl-pci-pv-x86-64-gcc-debug:
   extends: .adl-x86-64
@@ -189,7 +189,7 @@ adl-pci-pv-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh pci-pv 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 adl-pci-hvm-x86-64-gcc-debug:
   extends: .adl-x86-64
@@ -197,7 +197,7 @@ adl-pci-hvm-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 zen3p-smoke-x86-64-gcc-debug:
   extends: .zen3p-x86-64
@@ -205,7 +205,7 @@ zen3p-smoke-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 zen3p-smoke-x86-64-dom0pvh-gcc-debug:
   extends: .zen3p-x86-64
@@ -213,7 +213,7 @@ zen3p-smoke-x86-64-dom0pvh-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh dom0pvh 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 zen3p-pci-hvm-x86-64-gcc-debug:
   extends: .zen3p-x86-64
@@ -221,7 +221,7 @@ zen3p-pci-hvm-x86-64-gcc-debug:
     - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc-debug
+    - alpine-3.19-gcc-debug
 
 qemu-smoke-dom0-arm64-gcc:
   extends: .qemu-arm64
@@ -229,7 +229,7 @@ qemu-smoke-dom0-arm64-gcc:
     - ./automation/scripts/qemu-smoke-dom0-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.19-gcc-arm64
 
 qemu-smoke-dom0-arm64-gcc-debug:
   extends: .qemu-arm64
@@ -237,7 +237,7 @@ qemu-smoke-dom0-arm64-gcc-debug:
     - ./automation/scripts/qemu-smoke-dom0-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64
+    - alpine-3.19-gcc-debug-arm64
 
 qemu-smoke-dom0less-arm64-gcc:
   extends: .qemu-arm64
@@ -245,7 +245,7 @@ qemu-smoke-dom0less-arm64-gcc:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.19-gcc-arm64
 
 qemu-smoke-dom0less-arm64-gcc-debug:
   extends: .qemu-arm64
@@ -253,7 +253,7 @@ qemu-smoke-dom0less-arm64-gcc-debug:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64
+    - alpine-3.19-gcc-debug-arm64
 
 qemu-smoke-dom0less-arm64-gcc-debug-gicv3:
   extends: .qemu-arm64
@@ -261,7 +261,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-gicv3:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh gicv3 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64
+    - alpine-3.19-gcc-debug-arm64
 
 qemu-smoke-dom0less-arm64-gcc-debug-staticmem:
   extends: .qemu-arm64
@@ -269,7 +269,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-staticmem:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh static-mem 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64-staticmem
+    - alpine-3.19-gcc-debug-arm64-staticmem
 
 qemu-smoke-dom0less-arm64-gcc-debug-staticheap:
  extends: .qemu-arm64
@@ -277,7 +277,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-staticheap:
    - ./automation/scripts/qemu-smoke-dom0less-arm64.sh static-heap 2>&1 | tee ${LOGFILE}
  needs:
    - *arm64-test-needs
-   - alpine-3.18-gcc-debug-arm64
+   - alpine-3.19-gcc-debug-arm64
 
 qemu-smoke-dom0less-arm64-gcc-debug-static-shared-mem:
   extends: .qemu-arm64
@@ -285,7 +285,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-static-shared-mem:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh static-shared-mem 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64-static-shared-mem
+    - alpine-3.19-gcc-debug-arm64-static-shared-mem
 
 qemu-smoke-dom0less-arm64-gcc-debug-boot-cpupools:
   extends: .qemu-arm64
@@ -293,7 +293,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-boot-cpupools:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh boot-cpupools 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64-boot-cpupools
+    - alpine-3.19-gcc-debug-arm64-boot-cpupools
 
 qemu-smoke-dom0less-arm64-gcc-debug-earlyprintk:
   extends: .qemu-arm64
@@ -301,14 +301,14 @@ qemu-smoke-dom0less-arm64-gcc-debug-earlyprintk:
     - ./automation/scripts/qemu-smoke-dom0less-arm64.sh earlyprintk 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-debug-arm64-earlyprintk
+    - alpine-3.19-gcc-debug-arm64-earlyprintk
 
 qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
   extends: .qemu-arm64
   script:
     - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
   needs:
-    - alpine-3.18-gcc-arm64
+    - alpine-3.19-gcc-arm64
     - qemu-system-aarch64-6.0.0-arm64-export
 
 qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
@@ -316,7 +316,7 @@ qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
   script:
     - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
   needs:
-    - alpine-3.18-gcc-debug-arm64
+    - alpine-3.19-gcc-debug-arm64
     - qemu-system-aarch64-6.0.0-arm64-export
 
 qemu-smoke-dom0-arm32-gcc:
@@ -389,7 +389,7 @@ qemu-alpine-x86_64-gcc:
     - ./automation/scripts/qemu-alpine-x86_64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *x86-64-test-needs
-    - alpine-3.18-gcc
+    - alpine-3.19-gcc
 
 qemu-smoke-x86-64-gcc:
   extends: .qemu-x86-64
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index acdef1b54813..ae82e727b0b6 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -24,8 +24,8 @@ die() {
 #
 BASE="registry.gitlab.com/xen-project/xen"
 case "_${CONTAINER}" in
-    _alpine) CONTAINER="${BASE}/alpine:3.18" ;;
-    _alpine-arm64v8) CONTAINER="${BASE}/alpine:3.18-arm64v8" ;;
+    _alpine) CONTAINER="${BASE}/alpine:3.19" ;;
+    _alpine-arm64v8) CONTAINER="${BASE}/alpine:3.19-arm64v8" ;;
     _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;;
     _riscv64) CONTAINER="${BASE}/archlinux:current-riscv64" ;;
     _centos7) CONTAINER="${BASE}/centos:7" ;;
diff --git a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile b/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
deleted file mode 100644
index 0e5ae7f2b4d8..000000000000
--- a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
+++ /dev/null
@@ -1,65 +0,0 @@
-FROM --platform=linux/arm64/v8 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-RUN \
-  # apk
-  apk update && \
-  \
-  # xen runtime deps
-  apk add musl && \
-  apk add openrc && \
-  apk add busybox && \
-  apk add sudo && \
-  apk add dbus && \
-  apk add bash && \
-  apk add python3 && \
-  apk add zlib && \
-  apk add ncurses && \
-  apk add yajl && \
-  apk add libaio && \
-  apk add xz && \
-  apk add util-linux && \
-  apk add argp-standalone && \
-  apk add libfdt && \
-  apk add glib && \
-  apk add pixman && \
-  apk add curl && \
-  apk add udev && \
-  \
-  # Xen
-  cd / && \
-  # Minimal ramdisk environment in case of cpio output
-  rc-update add udev && \
-  rc-update add udev-trigger && \
-  rc-update add udev-settle && \
-  rc-update add networking sysinit && \
-  rc-update add loopback sysinit && \
-  rc-update add bootmisc boot && \
-  rc-update add devfs sysinit && \
-  rc-update add dmesg sysinit && \
-  rc-update add hostname boot && \
-  rc-update add hwclock boot && \
-  rc-update add hwdrivers sysinit && \
-  rc-update add killprocs shutdown && \
-  rc-update add modloop sysinit && \
-  rc-update add modules boot && \
-  rc-update add mount-ro shutdown && \
-  rc-update add savecache shutdown && \
-  rc-update add sysctl boot && \
-  rc-update add local default && \
-  cp -a /sbin/init /init && \
-  echo "ttyS0" >> /etc/securetty && \
-  echo "hvc0" >> /etc/securetty && \
-  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
-  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
-  passwd -d "root" root && \
-  \
-  # Create rootfs
-  cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile
deleted file mode 100644
index 9cde6c9ad4da..000000000000
--- a/automation/tests-artifacts/alpine/3.18.dockerfile
+++ /dev/null
@@ -1,66 +0,0 @@
-FROM --platform=linux/amd64 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-RUN \
-  # apk
-  apk update && \
-  \
-  # xen runtime deps
-  apk add musl && \
-  apk add libgcc && \
-  apk add openrc && \
-  apk add busybox && \
-  apk add sudo && \
-  apk add dbus && \
-  apk add bash && \
-  apk add python3 && \
-  apk add zlib && \
-  apk add ncurses && \
-  apk add yajl && \
-  apk add libaio && \
-  apk add xz && \
-  apk add util-linux && \
-  apk add argp-standalone && \
-  apk add libfdt && \
-  apk add glib && \
-  apk add pixman && \
-  apk add curl && \
-  apk add udev && \
-  apk add pciutils && \
-  apk add libelf && \
-  \
-  # Xen
-  cd / && \
-  # Minimal ramdisk environment in case of cpio output
-  rc-update add udev && \
-  rc-update add udev-trigger && \
-  rc-update add udev-settle && \
-  rc-update add loopback sysinit && \
-  rc-update add bootmisc boot && \
-  rc-update add devfs sysinit && \
-  rc-update add dmesg sysinit && \
-  rc-update add hostname boot && \
-  rc-update add hwclock boot && \
-  rc-update add hwdrivers sysinit && \
-  rc-update add modules boot && \
-  rc-update add killprocs shutdown && \
-  rc-update add mount-ro shutdown && \
-  rc-update add savecache shutdown && \
-  rc-update add local default && \
-  cp -a /sbin/init /init && \
-  echo "ttyS0" >> /etc/securetty && \
-  echo "hvc0" >> /etc/securetty && \
-  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
-  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
-  echo > /etc/modules && \
-  passwd -d "root" root && \
-  \
-  # Create rootfs
-  cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
diff --git a/automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile b/automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile
new file mode 100644
index 000000000000..68a5831bca72
--- /dev/null
+++ b/automation/tests-artifacts/alpine/3.19-arm64v8.dockerfile
@@ -0,0 +1,65 @@
+FROM --platform=linux/arm64/v8 alpine:3.19
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen runtime deps
+  apk add musl && \
+  apk add openrc && \
+  apk add busybox && \
+  apk add sudo && \
+  apk add dbus && \
+  apk add bash && \
+  apk add python3 && \
+  apk add zlib && \
+  apk add ncurses && \
+  apk add yajl && \
+  apk add libaio && \
+  apk add xz && \
+  apk add util-linux && \
+  apk add argp-standalone && \
+  apk add libfdt && \
+  apk add glib && \
+  apk add pixman && \
+  apk add curl && \
+  apk add udev && \
+  \
+  # Xen
+  cd / && \
+  # Minimal ramdisk environment in case of cpio output
+  rc-update add udev && \
+  rc-update add udev-trigger && \
+  rc-update add udev-settle && \
+  rc-update add networking sysinit && \
+  rc-update add loopback sysinit && \
+  rc-update add bootmisc boot && \
+  rc-update add devfs sysinit && \
+  rc-update add dmesg sysinit && \
+  rc-update add hostname boot && \
+  rc-update add hwclock boot && \
+  rc-update add hwdrivers sysinit && \
+  rc-update add killprocs shutdown && \
+  rc-update add modloop sysinit && \
+  rc-update add modules boot && \
+  rc-update add mount-ro shutdown && \
+  rc-update add savecache shutdown && \
+  rc-update add sysctl boot && \
+  rc-update add local default && \
+  cp -a /sbin/init /init && \
+  echo "ttyS0" >> /etc/securetty && \
+  echo "hvc0" >> /etc/securetty && \
+  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
+  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
+  passwd -d "root" root && \
+  \
+  # Create rootfs
+  cd / && \
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
diff --git a/automation/tests-artifacts/alpine/3.19.dockerfile b/automation/tests-artifacts/alpine/3.19.dockerfile
new file mode 100644
index 000000000000..6d665daedfa4
--- /dev/null
+++ b/automation/tests-artifacts/alpine/3.19.dockerfile
@@ -0,0 +1,67 @@
+FROM --platform=linux/amd64 alpine:3.19
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen runtime deps
+  apk add musl && \
+  apk add libgcc && \
+  apk add openrc && \
+  apk add busybox && \
+  apk add sudo && \
+  apk add dbus && \
+  apk add bash && \
+  apk add python3 && \
+  apk add zlib && \
+  apk add ncurses && \
+  apk add yajl && \
+  apk add libaio && \
+  apk add xz && \
+  apk add util-linux && \
+  apk add argp-standalone && \
+  apk add libfdt && \
+  apk add glib && \
+  apk add pixman && \
+  apk add curl && \
+  apk add udev && \
+  apk add pciutils && \
+  apk add libelf && \
+  apk add libdw && \
+  \
+  # Xen
+  cd / && \
+  # Minimal ramdisk environment in case of cpio output
+  rc-update add udev && \
+  rc-update add udev-trigger && \
+  rc-update add udev-settle && \
+  rc-update add loopback sysinit && \
+  rc-update add bootmisc boot && \
+  rc-update add devfs sysinit && \
+  rc-update add dmesg sysinit && \
+  rc-update add hostname boot && \
+  rc-update add hwclock boot && \
+  rc-update add hwdrivers sysinit && \
+  rc-update add modules boot && \
+  rc-update add killprocs shutdown && \
+  rc-update add mount-ro shutdown && \
+  rc-update add savecache shutdown && \
+  rc-update add local default && \
+  cp -a /sbin/init /init && \
+  echo "ttyS0" >> /etc/securetty && \
+  echo "hvc0" >> /etc/securetty && \
+  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
+  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
+  echo > /etc/modules && \
+  passwd -d "root" root && \
+  \
+  # Create rootfs
+  cd / && \
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723357.1128133 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-0005xM-Hh; Thu, 16 May 2024 13:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723357.1128133; Thu, 16 May 2024 13:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-0005wF-CY; Thu, 16 May 2024 13:59:42 +0000
Received: by outflank-mailman (input) for mailman id 723357;
 Thu, 16 May 2024 13:59:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be1-0005tl-2U
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:41 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8871f64b-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:38 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id 8B0711140163;
 Thu, 16 May 2024 09:59:37 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Thu, 16 May 2024 09:59:37 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:36 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8871f64b-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867977; x=1715954377; bh=Vx5DcvP0Oc
	GsO9ZlwIl0/mxQ0w+8bvA0AKy34SdFnpQ=; b=LB0jqGR4r+UX55IsXmrypEm75t
	u7ftVY+Sn5U7djolifbMlCoBWpDWzJiiNDuiJ4vUewSEzca6Y+Ammx7/Co0zy6B1
	Y5X/iqEsm/cDldOm3S9qBrLpg9UhJiRkXWcwklqUZH60hP9m+ah2jn3RNyT+w8Ld
	hqPk20yZbiV2LfZpKeVrbviljQp8Okieuj7s8Ah9b9iXUN+luGEsV7kKZTzsamCI
	Hv+0Mg6PnJaiY8vbXXZI3NaY1ipY6zSmX+efBqHyzSpqh3wA/xkwF/W5Mn0oOFHz
	KHace7spXuSPZ8HGSwtaRFLFraErWP7D91qj3QkwxijelkSDhp5jkl+/2Paw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867977; x=
	1715954377; bh=Vx5DcvP0OcGsO9ZlwIl0/mxQ0w+8bvA0AKy34SdFnpQ=; b=R
	F3oAknZY5YdcRc1OmGxE/qxEOabYFci2Ei958Aca9ZAiYNZhloGQJ0ioOzsxN8WC
	NeLqOQw+NU/BALyPqLkLk+x8MWLjs33PAhFrR0+gdLx2LV+YdFyMof07ndJzAC+e
	2womreuZ4YMlXVLC2jVhZJJ/qKeVDlXGmLfxpXIPD5EtUS5ZEGRy9soFa6LOr/Ks
	LgxWrA1Q3LSGhie3ebtmOFomnY+rQcjebofIhkv7Flcis0gdUr1z2k1rn7TyImOK
	U0td/cpuRo31GtV1xBvIyMVxmfU8wu5MuVX3bxYVLKak0EzIFBFi30AxCeRWBCqD
	roH3lSuVl4EdPMdLjTPnw==
X-ME-Sender: <xms:SRFGZnlrMQFQcVVN-uj7DvFjfKZV4eCdAKxs8Bw8tiTBqQkLo37jlw>
    <xme:SRFGZq2UGzIqNDoxsg1h_eC8E_zi1zhjzX6V0TekegkU2sPiHJSr6raTcyRUMkcuZ
    3jh3HD5g2-Vjw>
X-ME-Received: <xmr:SRFGZtoTLip1MyUbiPtwa0CXI67t_RQVb-i69tzkkOO0-Fx2RFIfoK_6zssPQ1EKm7uGffpyLR44VHvgyRQoLf_R1yeLjLR6JO0bzztw6GANGHHYH_w>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:SRFGZvm7T_lvyMJi2P0xg7e8GbHFyZxsbjy3OT0auyBBubLiXWLZyA>
    <xmx:SRFGZl2DpD35iJv_E7_XI330i7_dDaXC16djg2km_JTUX7A8xLrIsw>
    <xmx:SRFGZusYtE275epXvcTyfuY35cGT-yR_fy3khAdRqFnklkcMr2Yc6Q>
    <xmx:SRFGZpVPUxm4hcEGVr4gwRjldyxjQ2j5PJq0cw-Pqi8i0_OH_J3L5g>
    <xmx:SRFGZuyQUFS8qXm3rcgKO8y9j3YIqYdzRev0XtKDmvgmhhOv0PK1IR9i>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 01/12] automation: include domU kernel messages in the console output log
Date: Thu, 16 May 2024 15:58:22 +0200
Message-ID: <eb081e91f8c82d3293d6935c0567df96903d0b6c.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/scripts/qubes-x86-64.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index d81ed7b931cf..4beeff17d31b 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -131,6 +131,8 @@ mkdir sys
 rm var/run
 echo "#!/bin/sh
 
+echo 8 > /proc/sys/kernel/printk
+
 ${domU_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723359.1128156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be5-0006cp-5B; Thu, 16 May 2024 13:59:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723359.1128156; Thu, 16 May 2024 13:59:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be5-0006cg-2L; Thu, 16 May 2024 13:59:45 +0000
Received: by outflank-mailman (input) for mailman id 723359;
 Thu, 16 May 2024 13:59:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be3-0005tl-L6
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:43 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8ab89abf-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:42 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.nyi.internal (Postfix) with ESMTP id 5F64313803A4;
 Thu, 16 May 2024 09:59:41 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Thu, 16 May 2024 09:59:41 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:40 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ab89abf-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867981; x=1715954381; bh=Y5qffu5/Vu
	+XKaybYq8e5lXVQKHQBbeNSAMA/00cIO8=; b=vm4+SF2jJWHszzivlgxwOtKZcj
	1mVOHWWFAdq0r8scQu2OImIwabRzlhxn77GJ9KgVEIwABGWx04zMncfpo99UElSB
	A6TFB9Gxf3VBttMHRePH7qQhYBG3mv2KGG4SJDpoq0ndI/vygRc+VT8nz6I6OAjg
	SUa3sREWuLL2UgBJpu0yOur4UgXxB0M9o8k11n10mJXpmfbQhAZr9o4uBgW7jWAK
	HIQB0OJZweTsujJbrLLHRm/mVs++i3PHXUgsPWuKOqqXV/kAONZhjItCFajTxvj4
	ezj3rvk0bywETFfnmNxy1pJcBS+UgPsmKsC1zbdtd5uUTRQjrUG6DeDm9LGA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867981; x=
	1715954381; bh=Y5qffu5/Vu+XKaybYq8e5lXVQKHQBbeNSAMA/00cIO8=; b=Y
	CVeDLVNaIoCLUCRAmQdHFHrr9kD3hZh3Eyq7mL3zaMUFRc/mVUQWyxihsO7mJOlk
	/ZPP/OvmjPbnDXo3mJRadYly/GIsWSOjAQD3S5dTD3rnuVf8niXWLycjltEyJ4Y9
	BNB3x83K7mH/SPI6yQa1U5JTFoM23puj+TSIvkXeGvUHDSZkSZRYtrG84FH8Na6Q
	SKx4SFN0XjOiRnXAyToQehiOIO4I7Q+0f4yPP1JGNqvF4uFl45aZ+5a1tYmk90kV
	vZ0rAJsEAqY+yDuQHuYeqgkJJx6CdcsPpLSp5LYIbyl/SYnGhJGZhR1dek5icQtt
	/ApqeFH4rAdFNvirLhSBw==
X-ME-Sender: <xms:TRFGZkpGclxMeGcuB5Gc8C-j2EX3-Yw8sfCf6u_b4Hc4BFWB0cNH2w>
    <xme:TRFGZqqF_EcsUzO-FH-WoE0RCTvqGyrFoZ9zVhI6O9vhgJCdwjvhqBy7G3KsDf9K1
    tS6wimLcuB5pQ>
X-ME-Received: <xmr:TRFGZpP9syI9pRNTE6e4ZndMI7-BDyzt3KF6TCMSpOxhRqQpJVzNX-o97oaOADSHmFVkE5708e25l4XagiXj75cLL1kiFQ5zyZlsGElEYb66V22QyJQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:TRFGZr513P4qKWprGV2XHjJuqaMWmpDgoYBl74q_haRvcZjLQE_NtA>
    <xmx:TRFGZj6aEiFTFFLQuNpGUzDcaS-lV0LGNeILPLEhpMbFTpSw79vUgQ>
    <xmx:TRFGZrj8jUH1WLqb4OyOjmc7OP8XlECUTSvPBd8kVMVDlmJ9TDs3ug>
    <xmx:TRFGZt57v3afGRzOPEt2moZzDuQ85pv2jtBUGgo0JQfXrrPvfjwIRw>
    <xmx:TRFGZllBhFeSN8FLC2a6_zgb6u1EgZu50iVcMRKAHsG09znf-G39zhIS>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 04/12] automation: increase verbosity of starting a domain
Date: Thu, 16 May 2024 15:58:25 +0200
Message-ID: <cebd305832fab02532fba4b63d2b3c64801753ef.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

And start collecting qemu log earlier, so it isn't lost in case of a
timeout during domain startup.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/scripts/qemu-alpine-x86_64.sh    | 2 +-
 automation/scripts/qemu-smoke-dom0-arm32.sh | 2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh | 2 +-
 automation/scripts/qubes-x86-64.sh          | 4 ++--
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 8e398dcea34b..a188d60ea6f3 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -56,7 +56,7 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
index d91648905669..3d208cd55bfa 100755
--- a/automation/scripts/qemu-smoke-dom0-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
@@ -21,7 +21,7 @@ echo "#!/bin/bash
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > ./root/xen.start
 echo "bash /root/xen.start" >> ./etc/init.d/xen-watchdog
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index e0bb37af3610..afc24074eef8 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -52,7 +52,7 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 4beeff17d31b..bd620b0d9273 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -112,7 +112,6 @@ echo \"${passed}\"
 "
 
     dom0_check="
-tail -F /var/log/xen/qemu-dm-domU.log &
 until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.log; do
     sleep 1
 done
@@ -167,7 +166,8 @@ ifconfig xenbr0 192.168.0.1
 
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
-xl create /etc/xen/domU.cfg
+tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
+xl -vvv create /etc/xen/domU.cfg
 ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723361.1128171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be6-0006zI-Q9; Thu, 16 May 2024 13:59:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723361.1128171; Thu, 16 May 2024 13:59:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be6-0006y2-Ky; Thu, 16 May 2024 13:59:46 +0000
Received: by outflank-mailman (input) for mailman id 723361;
 Thu, 16 May 2024 13:59:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be5-0006bS-6e
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:45 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8b79953b-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:43 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.nyi.internal (Postfix) with ESMTP id A72EC13814A2;
 Thu, 16 May 2024 09:59:42 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Thu, 16 May 2024 09:59:42 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:41 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b79953b-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867982; x=1715954382; bh=cBlSjeC6zd
	zYMoWYLWhLp1CCK8q23xDxzAsx9E+dnrY=; b=mo6UKpsRNc7gr9AnBbyV8Cs/wV
	MnFgDUgiGTtBCZ5f0n04z5oFFW4xQQN1bahC8/TeFnmZXbWFCdB2FrECcyqQWKox
	RNgXHQcM85IdpAuNgzMzpkSXdEaOQmOd2+Kvo5uXkgGzpH+IASF4LLGVcjwXDA/v
	xoE/q98GDarHuyLYOVwzcVVZ20MqohdwHI1G1BwotOWx5S9LjI43SxOiPVNK9xaU
	wRUXxli5/5n/J/+X+aujx08Mtn075TtkXyfRNrxatpDmhuuO+YYLLSZjblfTeIu+
	XgeklA6dwlwQbyr8YJ4qby6BzgxTAxJzWiEKiXANl/65b1+zRAeFY82fqpTw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867982; x=
	1715954382; bh=cBlSjeC6zdzYMoWYLWhLp1CCK8q23xDxzAsx9E+dnrY=; b=c
	gNdw/XMpKM3xQC6wzV+yp3TnZlF7DG4A/9JrdGJPKx55dh7nyZYYYRddKsiAwjAN
	2D7mk5J+6ywb2iBxF5pfW5cbqQ9o3FJjeLDh5p3u4uDpnirT2ao4CaRYiuJfRKGF
	2tJSFwEPfeQ9btJ7EX6Hbp+CleXu0AVFdY5C5ew1lk6vsZ4JGJaALx4mVx74qmap
	vIOYOYb4VQbQokmnr8RhcSLiQO70P/ZUVQ3I3ClpbXXgarzrL3vgAZUH2tAvZkCw
	LcC4o0hHdRbUwcJ4BAOjG5uYeYxU7pW4H0sjVJ191NqLrh1WEITiIPmEkwAun6Lo
	ANlJKAGIgJrl/zA7+FQVQ==
X-ME-Sender: <xms:ThFGZnhFynRtGm38dSNdA-sqVDpBS8wRvDAlsbRkK-233wybPlRn3Q>
    <xme:ThFGZkCAH9fwY1xraaPGJilF7FCwclgMTrXNIrSHvtGfA_LGhI2QFkPzCs5iGTKvl
    1iy_bxCXkMZ8g>
X-ME-Received: <xmr:ThFGZnFzA5iUnvJOFUrdiz550aGf-AVQacCV5ndu0p1t3z9xd6AayN_8VJmdRgSN5eblJl_272EA-envOEM3td0StcbjQTlVs8HER2uEpGHZ56TB6Vo>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:ThFGZkQZP4JG8zpgu9vu4kU7V-Q9U4gIrwQRHAALmlwicxqpjVZG3g>
    <xmx:ThFGZkyehKSm4gpyYaud5px6YRRUnlrLe1p64BsUBTcqNSlkVgYmag>
    <xmx:ThFGZq5Kpom9gtlTOgOp2px37rjkVNKX9VHh5hwkavg896gOBRLpMA>
    <xmx:ThFGZpyx6zFgswY6e9RGnl_Y7ZmnDqTSmJhQHDGxUepJBCjcP07f4A>
    <xmx:ThFGZg8YoMkqQfUJyb7cOe6eVSxev94H0a7_z1BaZ0q5NjdCqMfCwS4F>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 05/12] automation: prevent grub unpacking initramfs
Date: Thu, 16 May 2024 15:58:26 +0200
Message-ID: <f56b08c94bacb493651f0121660aeaece34f3df7.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It fails on larger initramfs (~250MB one), let Linux do it.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/scripts/qubes-x86-64.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index bd620b0d9273..77cb0d45815d 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -189,7 +189,7 @@ CONTROLLER=control@thor.testnet
 echo "
 multiboot2 (http)/gitlab-ci/xen $CONSOLE_OPTS loglvl=all guest_loglvl=all dom0_mem=4G console_timestamps=boot $extra_xen_opts
 module2 (http)/gitlab-ci/vmlinuz console=hvc0 root=/dev/ram0 earlyprintk=xen
-module2 (http)/gitlab-ci/initrd-dom0
+module2 --nounzip (http)/gitlab-ci/initrd-dom0
 " > $TFTP/grub.cfg
 
 cp -f binaries/xen $TFTP/xen
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723362.1128179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be7-00077m-DO; Thu, 16 May 2024 13:59:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723362.1128179; Thu, 16 May 2024 13:59:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be7-00076P-42; Thu, 16 May 2024 13:59:47 +0000
Received: by outflank-mailman (input) for mailman id 723362;
 Thu, 16 May 2024 13:59:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be6-0006bS-6e
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:46 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8c3a470a-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:44 +0200 (CEST)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id E763D11400E9;
 Thu, 16 May 2024 09:59:43 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute4.internal (MEProxy); Thu, 16 May 2024 09:59:43 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:42 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c3a470a-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867983; x=1715954383; bh=mzdnDiyd0w
	yM7KrExNqycAdweGbpvBzSfJbl9cbzN/g=; b=g2vPIMHedE9HToZItkYtCegsib
	mayoJQ6/W4vR7HFVyvwit7sz6blxVUTSO3ySuu4/6gWC5Jn4mEPi0ODPYgXQ5/0y
	CBa6Kdg3Jk/t2eEXTH1vkJe1cMz71tW/qDH4gessgBlwYbUZt2GxaXSUZdCT2MrI
	upsGqV8grDZ9mpDHm1AyqsnaaVBLcN5lJSyFXMq+y7RRp1e28v3UYvHgPdboSKLi
	RN/rIDHtFGuCvc3Qs3AFCxDUV/GLylwySrUon0gLLY4iFdj2PBrOvXc+vUhYVMSH
	1UXTc21HrilCrVeZfR5nSE9gRh5aMk16kBb7r7nR7RsJn8LEG0j567ySce+A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867983; x=
	1715954383; bh=mzdnDiyd0wyM7KrExNqycAdweGbpvBzSfJbl9cbzN/g=; b=T
	nTJZpOKfR6EvpzmUZJuwcyXkW7tb5p7Kifu9ZqlEAGfvfxt2ifm1cjub4mi0WBEY
	nznjCo4/pOwKaLT4q5eqOv2ZKgnfhd+xWPs/f8e2gXU8x0049uZdqo8nDAbHpMnt
	zNWF0DE9akff30mLjzWo5Z0Lf2PSrQuzVfgmYud1Xjb0uDAXDY7l2egZdc6T/7aH
	sDuHOOgfZjq5pDaR4xTEGkvgxfHXj8GMD5P1aH6w91EICexJ/ooVDypNX7mjY4Bf
	PvxOwd3Bdqb8E10h8OtV9ZYIINsxwurAS0l7l9Yg77FU4OY5gmmEtMNVqKa6wegM
	eVsoBxhGhAfgLkJoZolKw==
X-ME-Sender: <xms:TxFGZiVeNbjwbkmHPrzgMON-2NzC31TE6qA_QBvQ6Bh5nrsAK5e7Vg>
    <xme:TxFGZulvYInR9hLQg8GTKiQYNSsKhLN-CrtB6ZDv-6aHXIwXfN0bwtPwKlo8K8cmE
    dComhPhzV-wMQ>
X-ME-Received: <xmr:TxFGZmZUx1Lxf2vwUjTC8V04PPt8ej-RTMgXQKmPpfAqRvyUvqMO01Q7Zyqs7p-6oSYA6knh7O3leYAkzqwPXQFJbzT5YGXjHmlmgRjKlqkDDac8d_U>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeff
    ffeiueffueejleevteduffehvddvgfetkeehlefgvefggfehhefhfeffuefhtdenucffoh
    hmrghinhepghhithhlrggsrdgtohhmpdhgihhthhhusgdrtghomhenucevlhhushhtvghr
    ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:TxFGZpX_ndsJOxO6DBNTZm6BiJYEZ--QtodBjcb1vUCqiFL0Hyjuqw>
    <xmx:TxFGZsl0hYVFfXmBD-EX9qUz3qN_I_cFm5ZstWDh9vbUw4l9Cv7vAA>
    <xmx:TxFGZufr8ThSe7yBs30mtDRC5O3LbHs_EBSxPY0vuuKBvnL1dhdbuA>
    <xmx:TxFGZuGMhNbDI5ehEr4KHvHXSWkCOyzMkqgcxEEet1NFcK1NYkzT7w>
    <xmx:TxFGZmhHvw8qTGtLItxWZgjS1Kpf-Fjg_aXRWeO25bkTnSmi8f88bBGq>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 06/12] RFC: automation: Add linux stubdom build and smoke test
Date: Thu, 16 May 2024 15:58:27 +0200
Message-ID: <c11feff249928f7e06a91c9b60633e1f74bbd6fd.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add minimal linux-stubdom smoke test. It starts a simple HVM with
linux-stubdom. The actual stubdom implementation is taken from Qubes OS
and then stripped off Qubes-specific code. In particular, the remaining
code does _not_ support:
 - direct kernel boot (implemented by relaying on specific guest disk
   laying in Qubes OS)
 - graphical console (used Qubes GUI agent injected into
   stubdomain's qemu)
 - audio input/output (used Qubes audio agent inside stubdomain)
 - USB passthrough (used qrexec <-> usbip proxy inside stubdomain)
 - setting up DHCP server (assumes guest addressing used in Qubes OS)

For this smoke test, the relevant part is missing direct kernel boot, as
that's used in other smoke tests. Solve this by preparing disk image
with proper bootloader (grub) installed. Since the test script is
running on arm64 to control x86_64 box, it cannot (easily) install grub
directly. For this reason, prepare bootsector as part of the Xen build
(which runs on x86_64) and then prepend do the disk image during the
test (and adjust partitions table afterwards).

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
The test is implemented using hardware runner, because some of the
further tests will require it (for example PCI passthrough with
stubdomain). But if there is strong desire to have stubdomain tested
inside qemu tests (to be included in patchew runs), it is probably an
option for this basic smoke test.

For now I'm keeping stubdomain code (build and glue scripts) in separate
repository on my github account. This is far from ideal. What would be
preferred option? New repository on xenbits? Or add directly into
xen.git (stubdom directory)? Honestly, I'd rather avoid the latter, as
from packager point of view those are mostly separate beings (similar to
qemu, where many use distribution-provide one instead of the one bundled
with Xen) and it's convenient to not need to rebuild stubdomain on every
hypervisor change (like a security patch).

Another topic is QEMU version inside stubdomain. It needs to be a
separate build due to vastly different configure options, so I cannot
reuse the qemu binary built for dom0 (or distribution-provided one if
Xen is configured to use it). But also, at this moment qemu for
stubdomain needs few extra patches that are not upstream yet.
What should be the proper solution here (after upstreaming all the
patches)?

Generally, I try to add tests early, even though there is still some
work to do for proper stubdomain integration into upstream Xen, so any
cleanups and future changes (like the CDROM libxl patches by Jason
Andryuk) can be made with more confidence and reduce risk of
regressions.

The patch is RFC only because of the stubdom repository location.
---
 automation/build/alpine/3.19-arm64v8.dockerfile   |  2 +-
 automation/build/alpine/3.19.dockerfile           |  9 ++-
 automation/gitlab-ci/build.yaml                   |  3 +-
 automation/gitlab-ci/test.yaml                    |  8 +-
 automation/scripts/build                          | 12 ++-
 automation/scripts/qubes-x86-64.sh                | 87 +++++++++++++++-
 automation/tests-artifacts/alpine/3.19.dockerfile |  6 +-
 7 files changed, 123 insertions(+), 4 deletions(-)

diff --git a/automation/build/alpine/3.19-arm64v8.dockerfile b/automation/build/alpine/3.19-arm64v8.dockerfile
index 158cf465a9ff..12810f87ecc6 100644
--- a/automation/build/alpine/3.19-arm64v8.dockerfile
+++ b/automation/build/alpine/3.19-arm64v8.dockerfile
@@ -47,3 +47,5 @@ RUN apk --no-cache add \
   # qubes test deps
   openssh-client \
   fakeroot \
+  sfdisk \
+  e2fsprogs \
diff --git a/automation/build/alpine/3.19.dockerfile b/automation/build/alpine/3.19.dockerfile
index 0be6d7c85fe7..108284613987 100644
--- a/automation/build/alpine/3.19.dockerfile
+++ b/automation/build/alpine/3.19.dockerfile
@@ -49,3 +49,12 @@ RUN apk --no-cache add \
   pixman-dev \
   # livepatch-tools deps
   elfutils-dev \
+  # stubdom deps
+  dracut-core \
+  quilt \
+  gnupg \
+  libseccomp-dev \
+  glib-static \
+  gmp-dev \
+  mpc1-dev \
+  mpfr-dev \
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index b186289bbd82..783a0687ba34 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -323,9 +323,11 @@ alpine-3.19-rootfs-export:
   image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+    - cp /grub-core.img binaries/grub-core.img
   artifacts:
     paths:
       - binaries/initrd.tar.gz
+      - binaries/grub-core.img
   tags:
     - x86_64
 
@@ -353,6 +355,7 @@ alpine-3.19-gcc-debug:
   extends: .gcc-x86-64-build-debug
   variables:
     CONTAINER: alpine:3.19
+    STUBDOM_LINUX: y
 
 debian-stretch-gcc-debug:
   extends: .gcc-x86-64-build-debug
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index f62d426a8d34..80d10eb7f476 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -199,6 +199,14 @@ adl-pci-hvm-x86-64-gcc-debug:
     - *x86-64-test-needs
     - alpine-3.19-gcc-debug
 
+adl-stubdom-hvm-x86-64-gcc-debug:
+  extends: .adl-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh stubdom-hvm 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.19-gcc-debug
+
 zen3p-smoke-x86-64-gcc-debug:
   extends: .zen3p-x86-64
   script:
diff --git a/automation/scripts/build b/automation/scripts/build
index b3c71fb6fb60..f7c85b07a8d0 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -102,4 +102,16 @@ else
     # build Xen
     cp -r dist binaries/
     if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fi
+
+    if [[ "${STUBDOM_LINUX}" == "y" ]]; then
+        git clone https://github.com/marmarek/qubes-vmm-xen-stubdom-linux -b for-upstream2
+        make -j$(nproc) -C qubes-vmm-xen-stubdom-linux get-sources
+        CFLAGS="-I${PWD}/dist/install/usr/local/include" \
+            LDFLAGS="-L${PWD}/dist/install/usr/local/lib" \
+            LD_LIBRARY_PATH="${PWD}/dist/install/usr/local/lib" \
+            PATH="${PWD}/dist/install/usr/local/bin:$PATH" \
+            make -j$(nproc) -C qubes-vmm-xen-stubdom-linux -f Makefile.stubdom all
+        cp qubes-vmm-xen-stubdom-linux/build/linux/arch/x86/boot/bzImage binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-kernel
+        cp qubes-vmm-xen-stubdom-linux/build/rootfs/stubdom-linux-rootfs binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-rootfs
+    fi
 fi
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 77cb0d45815d..fc73403dbadf 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -18,6 +18,37 @@ memory = 512
 vif = [ "bridge=xenbr0", ]
 disk = [ ]
 '
+domU_disk_path=
+
+### helper functions
+
+build_domU_disk() {
+    local kernel="$1"
+    local initrd="$2"
+    local rootfs="$3"
+    local output="$4"
+    local grubcfg="$rootfs/boot/grub2/grub.cfg"
+    local kernel_cmdline="root=/dev/xvda1 console=hvc0 earlyprintk=xen"
+
+    mkdir -p "$rootfs/boot/grub2"
+    cp "$kernel" "$rootfs/boot/vmlinuz"
+    echo "linux /boot/vmlinuz $kernel_cmdline" >> "$grubcfg"
+    if [ -n "$initrd" ]; then
+        cp "$initrd" "$rootfs/boot/initrd.img"
+        echo "initrd /boot/initrd.img" >> "$grubcfg"
+    fi
+    echo "boot" >> "$grubcfg"
+    size=$(du -sm "$rootfs")
+    size=${size%%	*}
+    # add 5M margin
+    size=$(( size + 5 ))
+    mke2fs -d "$rootfs" "$output.part1" ${size}m
+    cat "$rootfs/usr/lib/grub/i386-pc/boot_hybrid.img" binaries/grub-core.img > "$output"
+    # align for the partition 1 start (2048 sectors)
+    truncate -s $((2048 * 512)) "$output"
+    cat "$output.part1" >> "$output"
+    echo ",,linux,*" | sfdisk "$output"
+}
 
 ### test: smoke test & smoke test PVH
 if [ -z "${test_variant}" ] || [ "${test_variant}" = "dom0pvh" ]; then
@@ -116,6 +147,41 @@ until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.
     sleep 1
 done
 "
+
+### test: stubdom-hvm
+elif [ "${test_variant}" = "stubdom-hvm" ]; then
+    passed="ping test passed"
+
+    domU_config='
+type = "hvm"
+name = "domU"
+memory = 512
+vif = [ "bridge=xenbr0", ]
+disk = [ "/srv/disk.img,format=raw,vdev=xvda" ]
+device_model_version = "qemu-xen"
+device_model_stubdomain_override = 1
+on_reboot = "destroy"
+# libxl configures vkb backend to be dom0 instead of the stubdomain, defer
+# changing that until there is consensus what to do about VGA output (VNC)
+vkb_device = 0
+'
+    domU_check="
+ifconfig eth0 192.168.0.2
+until ping -c 10 192.168.0.1; do
+    sleep 1
+done
+echo \"${passed}\"
+"
+    dom0_check="
+set +x
+until grep -q \"${passed}\" /var/log/xen/console/guest-domU.log; do
+    sleep 1
+done
+set -x
+echo \"${passed}\"
+"
+
+    domU_disk_path=/srv/disk.img
 fi
 
 # DomU
@@ -137,8 +203,17 @@ ${domU_check}
 chmod +x etc/local.d/xen.start
 echo "rc_verbose=yes" >> etc/rc.conf
 sed -i -e 's/^Welcome/domU \0/' etc/issue
-find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
 cd ..
+if [ -n "$domU_disk_path" ]; then
+    build_domU_disk \
+        "binaries/bzImage" \
+        "" \
+        "rootfs" \
+        "binaries/disk.img"
+else
+    (cd rootfs; find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz)
+fi
+
 rm -rf rootfs
 
 # DOM0 rootfs
@@ -152,6 +227,9 @@ mkdir srv
 mkdir sys
 rm var/run
 cp -ar ../binaries/dist/install/* .
+if [ -n "$domU_disk_path" ]; then
+    cp ../binaries/disk.img "./$domU_disk_path"
+fi
 
 echo "#!/bin/bash
 
@@ -164,8 +242,9 @@ ifconfig eth0 up
 ifconfig xenbr0 up
 ifconfig xenbr0 192.168.0.1
 
-# get domU console content into test log
+# get domU (and possibly its stubdom) console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
+tail -F /var/log/xen/console/guest-domU-dm.log 2>/dev/null | sed -e \"s/^/(domU-dm) /\" &
 tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
 xl -vvv create /etc/xen/domU.cfg
 ${dom0_check}
@@ -178,7 +257,9 @@ echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
-cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
+if [ -r ../binaries/domU-rootfs.cpio.gz ]; then
+    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
+fi
 find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
diff --git a/automation/tests-artifacts/alpine/3.19.dockerfile b/automation/tests-artifacts/alpine/3.19.dockerfile
index 6d665daedfa4..cfb2cb30fb30 100644
--- a/automation/tests-artifacts/alpine/3.19.dockerfile
+++ b/automation/tests-artifacts/alpine/3.19.dockerfile
@@ -35,6 +35,8 @@ RUN \
   apk add pciutils && \
   apk add libelf && \
   apk add libdw && \
+  apk add grub-bios && \
+  apk add libseccomp && \
   \
   # Xen
   cd / && \
@@ -64,4 +66,6 @@ RUN \
   \
   # Create rootfs
   cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var && \
+  # Prepare boot sector for HVM disk
+  grub-mkimage -o /grub-core.img -O i386-pc -p '(hd0,msdos1)/boot/grub2' boot part_msdos ext2 linux biosdisk configfile normal
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723358.1128138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-000672-S5; Thu, 16 May 2024 13:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723358.1128138; Thu, 16 May 2024 13:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be2-00065g-Os; Thu, 16 May 2024 13:59:42 +0000
Received: by outflank-mailman (input) for mailman id 723358;
 Thu, 16 May 2024 13:59:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be2-0005tl-2V
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:42 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 892b6787-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:39 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id C09711140166;
 Thu, 16 May 2024 09:59:38 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Thu, 16 May 2024 09:59:38 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:37 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 892b6787-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867978; x=1715954378; bh=+2WEMyTqWK
	NhMj6QhIZf6GFM0RxapZhBZiw+isG8OcI=; b=Z8D8X4OxCVYdraaM0Va/bcbPBt
	YytTvaPlLVjGpxok0H798l4gAb74IL80W2jZV7H74Ibxjh+F8pX+lBNVzKYbR2xM
	xUKVeGieIHyDsxwqZd9kfUDsDX7EE1w2cKCEIWmmxR1AHH9tV7flFiOqp8ACSjs+
	ILOMDtkl7XHvZvlVDioYGRyw9N+5qqQBkFALgxemwmOPAwIaqFWiBVqTJprdcmPo
	JHUW5eAJp2Ceq3If3Yi1u0pIFKhzvnj0JXmH3wh6Tu0phZWbb5ZR6nJJbYaw4IXl
	rTrGTr2zhMKVD28pyV12fN/V1eKE4CImqFbhAMOWIwCja2Ja+pIncN6EwgOg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867978; x=
	1715954378; bh=+2WEMyTqWKNhMj6QhIZf6GFM0RxapZhBZiw+isG8OcI=; b=g
	7Zi8nXMH8cRi7RR9rb80J20MasOsBpjvr4hXfKiYm+Hg0zMprWZAMHMTXxHVXVy0
	h4J3IcLZwXv3y521N6Nxei7xuDEGg42iEMq7k8YZYsF6y4XdQDSoUu/zuJ42akrz
	pSdFou5uMsQ3+CEOXfoYb2UK8Q2KdzQVo3QhDZ9izDFB0+5A9JDNSq47oCO52zO5
	zknoHIBDpHmchRi0IKY0IkyX1cK+Ku3jKGEEGTmbBTpyihvdYHBs25rPPWUjzDdv
	qkf0B799KA4bt/7SUx+Z+Tvpe1fDuJXbKEmAdBZX0HWLkuNg5Q65epqzLa5y1V8K
	HuwT86oAWKfZLD4aemptA==
X-ME-Sender: <xms:ShFGZnKaN3on3Ok5C8AAajtfF81to7P9hVLA4vF7xRZzZDy_k6Mf4g>
    <xme:ShFGZrJKfYMis3wo8f9GG9giUewBj_6yyLhOH52tll8NolAWoFafxzm7CEkRqSiL7
    IJzRfl4Z361VQ>
X-ME-Received: <xmr:ShFGZvsR7bwh6uX03E-6N8_STVHcLhGBjCTN9m4eIIcUbjD1SFARXUnGvavXSzIX5HyxxlK_Pqgk10PJeE7oqSJiEk6GvX-j7AQXXuy6mYalY53hTYk>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgjedtucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:ShFGZgaWX_E6k6-5NlHC0rvwXVv4ymLOws_DkYMGYE2j-PZkB0n71w>
    <xmx:ShFGZuYISGQ8SiUIsoKYk21cS9IDSZVb4oUj2GBdnnN1QGtwwV0Ycg>
    <xmx:ShFGZkAKmZPkG8yQmEU5X4a31wBIYw-H69N1Od5S2TOkJEYHeF2teg>
    <xmx:ShFGZsZu_Lw2q7Six690TAwtCDlob2H1z5ahRrCZsj6BxNCu_D2e7w>
    <xmx:ShFGZuF3ADW2Zq5FeQ4nGOO83F6m7jPl3kHARaqUX5JsPsw-YzklFPbv>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 02/12] automation: update fedora build to F39
Date: Thu, 16 May 2024 15:58:23 +0200
Message-ID: <ae23eeb2e08c2fa95d975bfad64b818870e370e6.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fedora 29 is long EOL

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/build/fedora/29.dockerfile | 46 +----------------------------
 automation/build/fedora/39.dockerfile | 46 ++++++++++++++++++++++++++++-
 automation/gitlab-ci/build.yaml       |  4 +-
 3 files changed, 48 insertions(+), 48 deletions(-)
 delete mode 100644 automation/build/fedora/29.dockerfile
 create mode 100644 automation/build/fedora/39.dockerfile

diff --git a/automation/build/fedora/29.dockerfile b/automation/build/fedora/29.dockerfile
deleted file mode 100644
index f473ae13e7c1..000000000000
--- a/automation/build/fedora/29.dockerfile
+++ /dev/null
@@ -1,46 +0,0 @@
-FROM --platform=linux/amd64 fedora:29
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-# install Xen depends
-RUN dnf -y install \
-        clang \
-        gcc \
-        gcc-c++ \
-        ncurses-devel \
-        zlib-devel \
-        openssl-devel \
-        python-devel \
-        python3-devel \
-        libuuid-devel \
-        pkgconfig \
-        flex \
-        bison \
-        libaio-devel \
-        glib2-devel \
-        yajl-devel \
-        pixman-devel \
-        glibc-devel \
-        make \
-        binutils \
-        git \
-        wget \
-        acpica-tools \
-        python-markdown \
-        patch \
-        checkpolicy \
-        dev86 \
-        xz-devel \
-        bzip2 \
-        nasm \
-        ocaml \
-        ocaml-findlib \
-        golang \
-        # QEMU
-        ninja-build \
-    && dnf clean all && \
-    rm -rf /var/cache/dnf
-
-RUN useradd --create-home user
-USER user
-WORKDIR /build
diff --git a/automation/build/fedora/39.dockerfile b/automation/build/fedora/39.dockerfile
new file mode 100644
index 000000000000..054f73444060
--- /dev/null
+++ b/automation/build/fedora/39.dockerfile
@@ -0,0 +1,46 @@
+FROM --platform=linux/amd64 fedora:39
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+# install Xen depends
+RUN dnf -y install \
+        clang \
+        gcc \
+        gcc-c++ \
+        ncurses-devel \
+        zlib-devel \
+        openssl-devel \
+        python-devel \
+        python3-devel \
+        libuuid-devel \
+        pkgconfig \
+        flex \
+        bison \
+        libaio-devel \
+        glib2-devel \
+        yajl-devel \
+        pixman-devel \
+        glibc-devel \
+        make \
+        binutils \
+        git \
+        wget \
+        acpica-tools \
+        python-markdown \
+        patch \
+        checkpolicy \
+        dev86 \
+        xz-devel \
+        bzip2 \
+        nasm \
+        ocaml \
+        ocaml-findlib \
+        golang \
+        # QEMU
+        ninja-build \
+    && dnf clean all && \
+    rm -rf /var/cache/dnf
+
+RUN useradd --create-home user
+USER user
+WORKDIR /build
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5b4..69665ec5b11f 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -691,12 +691,12 @@ debian-bookworm-32-gcc-debug:
 fedora-gcc:
   extends: .gcc-x86-64-build
   variables:
-    CONTAINER: fedora:29
+    CONTAINER: fedora:39
 
 fedora-gcc-debug:
   extends: .gcc-x86-64-build-debug
   variables:
-    CONTAINER: fedora:29
+    CONTAINER: fedora:39
 
 # Ubuntu Trusty's Clang is 3.4 while Xen requires 3.5
 
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723363.1128197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be9-0007kl-P9; Thu, 16 May 2024 13:59:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723363.1128197; Thu, 16 May 2024 13:59:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be9-0007kG-Ka; Thu, 16 May 2024 13:59:49 +0000
Received: by outflank-mailman (input) for mailman id 723363;
 Thu, 16 May 2024 13:59:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be7-0005tl-Gu
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:47 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8d0b9a40-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:46 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id 46BA51140163;
 Thu, 16 May 2024 09:59:45 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Thu, 16 May 2024 09:59:45 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:44 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d0b9a40-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867985; x=1715954385; bh=i0tm0hLqA8
	ZLzyhanisejpktcCRo0bXMjTCqyw0gS+4=; b=c+7KzSezfvynmGBLoAmYd+WTzm
	6Eo7ZAqWKL10udGCmQnv0LPk46g+EJiDfycE5Y1eSsJ04LaABvlGo20C2K4K106E
	csJDxEnM2QQZvviBMBbh1V8dRAnNgxY/1ZqwNkPB9ktA6I+bs8Xt0ZJfAmx/JftK
	QW+vQlbaye3hJ0sDB2nDnJNOmRjjVLAJ/ZwktGZcXLnHUcUTqBE7LxoFoD63N6r2
	+KxTsI6hneuNhEgbWx9R+fvZZR42j6bg2IDy3xpS0jKpL7gtiv4YJoiQzvlxO/V4
	B/GS85GqAg3JnKefGi3gs1Du+c7T06SSBAVG31pHF5yadUKhSX7RnGIY/nrw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867985; x=
	1715954385; bh=i0tm0hLqA8ZLzyhanisejpktcCRo0bXMjTCqyw0gS+4=; b=K
	08V5F5EbgXuDdcOHoGgBxEk0szGPAhh9I5EUblwRDQy+eoo+LzPnD3UTckbiZu09
	QDrZi/774q0lBut6vTjZE4lz48pyZdyq07nK1WvVNn9okEoMh9/nNDps++M+M+Ic
	myiLzxLLyklVB39B2CogGJ9AeqPXL5fz6zU3B2kFE2Iv3nRUdfc14t2k78oN3gjD
	86Xzz/hSH1rLW8fDbx7Ajdh1de/CCmTmP2yV4zhtahVtgi6jepRO1UyG76Jd3KLS
	bWrbIsKChgP3j7hhGsZWmNYgKzlYM+jke9s/qe01U85/JolXr3W/I+l9YZYrAoV2
	2Xy8gGKG6gZ0SGtPEYjBw==
X-ME-Sender: <xms:UBFGZqHaAJf9PhvLyW3Y7WrB5gYi3IlxiwTaNllysIrUDWIRmnWLkg>
    <xme:UBFGZrXYK4IiN_qU0nIeWYdqnpofSCfgRor4s5xjJeM9j4O9SePu3Px09BEkcinKh
    ksMaocrmsf1ig>
X-ME-Received: <xmr:UBFGZkI7moV1pKWgal91L19OClOnSMBgJfPOAIzWqwLtyCpztvT9ROMsVEL8dKrpNmeGZPE8SBNvQprJyX-NwIwFjtHxPMpa4PycEbdjtWgqBwNl1SA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:URFGZkHzo5XiLHxifXOswET2_AifUan_9W_FgQ64Mcm3EqjkblmTpA>
    <xmx:URFGZgW9it1-Xqi1JzwMPC7HPztPgDVCev0DyoLlTLjfhXfkb5d67g>
    <xmx:URFGZnOVZxGqk3AKhkBRp8iPS0aNtRkRy8yGUUWGBApgp712Lk927A>
    <xmx:URFGZn2FAWwISqFzZZQxnpwLquN4-SQfFhSebICN44Tkvg3Y4AK8wA>
    <xmx:URFGZnQWdIAcx4KpH1NHdSdrZPv3-y-wT3hjlT78rNxYjlc4DH9WPI6D>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 07/12] libxl: Allow stubdomain to control interupts of PCI device
Date: Thu, 16 May 2024 15:58:28 +0200
Message-ID: <aeba05a5ec7a0657217933f165ee0ac7e86e1b1b.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Especially allow it to control MSI/MSI-X enabling bits. This part only
writes a flag to a sysfs, the actual implementation is on the kernel
side.

This requires Linux >= 5.10 in dom0 (or relevant patch backported).

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libs/light/libxl_pci.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 96cb4da0794e..6f357b70b815 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1513,6 +1513,14 @@ static void pci_add_dm_done(libxl__egc *egc,
             rc = ERROR_FAIL;
             goto out;
         }
+    } else if (libxl_is_stubdom(ctx, domid, NULL)) {
+        /* Allow acces to MSI enable flag in PCI config space for the stubdom */
+        if ( sysfs_write_bdf(gc, SYSFS_PCIBACK_DRIVER"/allow_interrupt_control",
+                             pci) < 0 ) {
+            LOGD(ERROR, domainid, "Setting allow_interrupt_control for device");
+            rc = ERROR_FAIL;
+            goto out;
+        }
     }
 
 out_no_irq:
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723364.1128202 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beA-0007ov-5e; Thu, 16 May 2024 13:59:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723364.1128202; Thu, 16 May 2024 13:59:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7be9-0007no-UG; Thu, 16 May 2024 13:59:49 +0000
Received: by outflank-mailman (input) for mailman id 723364;
 Thu, 16 May 2024 13:59:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7be8-0006bS-2I
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:48 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8dc5c438-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:47 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.nyi.internal (Postfix) with ESMTP id 7B8C413814B2;
 Thu, 16 May 2024 09:59:46 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Thu, 16 May 2024 09:59:46 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:45 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8dc5c438-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867986; x=1715954386; bh=u6dbejfCbQ
	Ta4IPIW9KKt75kBi6AoNdi5qidbD/kO3Q=; b=H+gYi5L/MQNUlVI+sOc+kOKGaa
	eTr8K5spBGn6SfiU6ug0XIg68Q3AeXNYFfQtabr0CkYt+DzrRmDj/q221D1E6Fmb
	39KFsskZD83+QoYzPbpBET/rda8MB+9H6iUDronNtulyctZTkxbkKiBvwug388fS
	RBYubncENuDLhVQV18yncZCUQ8FeLL1snR+9V3YnZ4U04Z4glPeODai5emRA4m6T
	co6PkhE6K/CrA1R7dwEk3tjxKPPdZxa0epyzPCY4tDNj+6Q2QanopMq9jSUgE0oQ
	7neNjmTOvr1wBgDS48kGY8sl7WAIqhe138tvTCjOGGMaZWire7ttU16+HIlQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867986; x=
	1715954386; bh=u6dbejfCbQTa4IPIW9KKt75kBi6AoNdi5qidbD/kO3Q=; b=a
	xOQXQBOmO9Cew+s8XEfxNFpCMqXk7oJnXsr2yZR0jDE47PccfcStqXCuCQrOD/FZ
	jOYK3nx9qvxnUEdHOTA42GudDsj5u+E6f30JtJyzuG7o5jMh4Sln8a0dbTVvkpI+
	BuqYypnCR5hcJZdrZSOOdjfY/XnxOCeZz4T6Fx1qhWqbzBGvRdys/5hODWXC/T3Z
	EGVDV1VntIQLAg6ApLobPyftnAceX6SOluZiUTUtPd6V//6zWG7k7UhXfaJ/Df3L
	szDsxodzwiU9D/pkjGcq7z/Oy4ah46i652ydAWrghD5ycThxHcbuyCFqW7Qn+z6k
	cUm6cFvv+GSi/C794wHjA==
X-ME-Sender: <xms:UhFGZob3svXFchL8CGBhzoHl4GkYrOZ4nJ1-AI_9FgPFHz_J4bY80Q>
    <xme:UhFGZjbkExPkKHDQO8timU8ZmFOugnBwcB52W8QDuUZJteM4etHd50qpB-ygxEkOY
    C99NmkrwxHVnA>
X-ME-Received: <xmr:UhFGZi8HqSt62M-REjbjlAUjZqdCVqprBefN2-VlU3phOntc9sXnUKnHosA7BfSlDBmwVhu6kE7vBwJtl8LPlAgnHWWtZxMTVCOug3JogX-qXwVYzT8>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    lefggfeuffegjeduheeigeetueegfeekjeetgffhvdeiffeiieejgfduieekgeenucffoh
    hmrghinhepghhithhlrggsrdgtohhmpdhkvghrnhgvlhdrohhrghenucevlhhushhtvghr
    ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:UhFGZiq1oxmRMqeeelWuPJeHARMbcdtVc5rnBe_YYrdtOKPYc1tM1g>
    <xmx:UhFGZjqjc6qSR9z_0fHbaa8bD_f3WXfF_GNimwI6U4rqlxr2X1PETw>
    <xmx:UhFGZgQ4Qb4_ztJ56V0iBZhGQGztkjHX7UqIEN3_jfACfx__DdCguA>
    <xmx:UhFGZjoDZijGDbkPV3svToZYkGi5LbSqEGXJi7kerCGyIRd5stmTGg>
    <xmx:UhFGZiUab4JiFD0crPZqL6pIRHuALWrMg2U7xjQkRBA_RhGXY5N_uPdg>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 08/12] automation: update kernel for x86 tests
Date: Thu, 16 May 2024 15:58:29 +0200
Message-ID: <2e6dc425056db1267768fd1bd0a20c444b1f1610.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Update 6.1.x kernel to the latest version in this branch. This is
especially needed to include MSI-X related fixes for stubdomain
("xen-pciback: Consider INTx disabled when MSI/MSI-X is enabled").

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/gitlab-ci/build.yaml                     |  4 +-
 automation/gitlab-ci/test.yaml                      |  2 +-
 automation/tests-artifacts/kernel/6.1.19.dockerfile | 40 +--------------
 automation/tests-artifacts/kernel/6.1.90.dockerfile | 40 ++++++++++++++-
 4 files changed, 43 insertions(+), 43 deletions(-)
 delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/6.1.90.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 783a0687ba34..9b9e5464f179 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -331,9 +331,9 @@ alpine-3.19-rootfs-export:
   tags:
     - x86_64
 
-kernel-6.1.19-export:
+kernel-6.1.90-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.90
   script:
     - mkdir binaries && cp /bzImage binaries/bzImage
   artifacts:
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 80d10eb7f476..e3910f4c1a9f 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -12,7 +12,7 @@
 
 .x86-64-test-needs: &x86-64-test-needs
   - alpine-3.19-rootfs-export
-  - kernel-6.1.19-export
+  - kernel-6.1.90-export
 
 .qemu-arm64:
   extends: .test-jobs-common
diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile
deleted file mode 100644
index 3a4096780d20..000000000000
--- a/automation/tests-artifacts/kernel/6.1.19.dockerfile
+++ /dev/null
@@ -1,40 +0,0 @@
-FROM --platform=linux/amd64 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV LINUX_VERSION=6.1.19
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        libssl-dev \
-        bc \
-        curl \
-        flex \
-        bison \
-        libelf-dev \
-        && \
-    apt-get autoremove -y && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-
-# Build the kernel
-RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
-    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
-    cd linux-"$LINUX_VERSION" && \
-    make defconfig && \
-    make xen.config && \
-    scripts/config --enable BRIDGE && \
-    scripts/config --enable IGC && \
-    cp .config .config.orig && \
-    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
-    make -j$(nproc) bzImage && \
-    cp arch/x86/boot/bzImage / && \
-    cd /build && \
-    rm -rf linux-"$LINUX_VERSION"*
diff --git a/automation/tests-artifacts/kernel/6.1.90.dockerfile b/automation/tests-artifacts/kernel/6.1.90.dockerfile
new file mode 100644
index 000000000000..46cadf02ca78
--- /dev/null
+++ b/automation/tests-artifacts/kernel/6.1.90.dockerfile
@@ -0,0 +1,40 @@
+FROM --platform=linux/amd64 debian:bookworm
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LINUX_VERSION=6.1.90
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        libssl-dev \
+        bc \
+        curl \
+        flex \
+        bison \
+        libelf-dev \
+        && \
+    apt-get autoremove -y && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
+
+# Build the kernel
+RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
+    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
+    cd linux-"$LINUX_VERSION" && \
+    make defconfig && \
+    make xen.config && \
+    scripts/config --enable BRIDGE && \
+    scripts/config --enable IGC && \
+    cp .config .config.orig && \
+    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
+    make -j$(nproc) bzImage && \
+    cp arch/x86/boot/bzImage / && \
+    cd /build && \
+    rm -rf linux-"$LINUX_VERSION"*
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723365.1128214 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beB-0008EX-J8; Thu, 16 May 2024 13:59:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723365.1128214; Thu, 16 May 2024 13:59:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beB-0008D1-DI; Thu, 16 May 2024 13:59:51 +0000
Received: by outflank-mailman (input) for mailman id 723365;
 Thu, 16 May 2024 13:59:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7beA-0005tl-4C
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:50 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8e8c1d35-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:48 +0200 (CEST)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailfout.nyi.internal (Postfix) with ESMTP id BE49813803AD;
 Thu, 16 May 2024 09:59:47 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute4.internal (MEProxy); Thu, 16 May 2024 09:59:47 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:46 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e8c1d35-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867987; x=1715954387; bh=PRF53+BCaf
	9Aziy/eN2Em2AqRlh6rGUA7cP+LaPIH+E=; b=OGnAGl2U4SmMOB08+dj3TM59DA
	3o063TgAuCrFfnkmkOmQMfTfCEm5WB/Evf32CpJ3UByl3vp3STuKN2SVc92ux2+f
	zatHfYBYM7oIWtMBrGo40Kad+umSEyZAGjyWawBCDakpcs2UJ2YLOXkSYMv6oQp9
	j6/AM4gxGuVdE5pI3+39vL023vG4WhKQlXMhTIVLa9HCMfTaJteXzMbRVsQ5ntLu
	kWzepZVZlK3d5Xf9x7Liunwyx9AtnXf0tAHDYVkYNyMjGZqTs84/LXiltJyubg8L
	bsUuECRGVeNu7ChnB0vrdWhixE2wyHloq4BwGBVV3OlENIxVErg0y9jvJArg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867987; x=
	1715954387; bh=PRF53+BCaf9Aziy/eN2Em2AqRlh6rGUA7cP+LaPIH+E=; b=N
	RekwLBVqJaWLsmYT9L5zafMGv1FbQhx5EBQWlVhMuMJF2JlSvf2qjM3n9zXYbrzK
	jMOU1pkjHZaN8nriurCyOrp62afmbi2uE7WZBX1H2Lex4XCPsqBBsZNk2G9FP6Zc
	hHB2fpRXVOmDC/Mi6eAjBEmdhhgwATEu0hye5JwmqNL0iYBwXOGEDg3bsMurWE5Z
	zNoTZsqos3yLS++8kbjBUS2p0UD0wtN+rNi0P6l7h/9E2nHlM1IloMRFPP88ZF2+
	a+FOfuwFLa5O3YQ25B2FFGT66ZUF6VjjpubKz2+TMAWIS74g31U+Mcx6/sb7xaCs
	AOLuhgT2WJ5fRdQg50+FA==
X-ME-Sender: <xms:UxFGZtu4Mq9n0JyajEr5Fs8liEivH2o_cKzQlPrq8-x873ZOJQGrJg>
    <xme:UxFGZmeTh8aFai1-gOKVzipAt5-gt0PP8a4fOG3ONhutxMYzo3BA45umMVOEoxQZ6
    HUxRdkmpkEl9Q>
X-ME-Received: <xmr:UxFGZgylfII-0R9q3noViuvX7yz9T7CxGvVb8RcOU6D8s_vK9KS6at98NdXyxpCol79lOqFKE_hQjNCf9XsoJMksjXdpMhYamaX7U7lTm_ZD4Imc_Ck>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:UxFGZkN6Tl-78y4tuYURBgbMWQH1Jyr64VTCsQWY_yd0PH8sbQGKHA>
    <xmx:UxFGZt8ESiLFsmKeC675NkLgntIBLqZ6xB7q35xImnLNFt0qsI0W-A>
    <xmx:UxFGZkUJX3o0JZ7IDt6ZVcUnS-I5U8LjjKhRT0KgZFn6Zn4uyRJuuw>
    <xmx:UxFGZueBBFGPC7hAJw2nhfDaDEMEbJR03X1DOqoMXKNOR7x4HMOXjA>
    <xmx:UxFGZoZyo-S-JtR2Hxl-CkgFiinbg-d-74GoXAPoCynfX2Fuf3ZStx8->
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 09/12] WIP: automation: temporarily add 'testlab' tag to stubdomain build
Date: Thu, 16 May 2024 15:58:30 +0200
Message-ID: <ef65781a69879d15a334f1afe644762d4f9c0bd7.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Make it run on newer runners that have new enough kernel for
dracut-install.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/gitlab-ci/build.yaml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 9b9e5464f179..134a01d03efa 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -356,6 +356,9 @@ alpine-3.19-gcc-debug:
   variables:
     CONTAINER: alpine:3.19
     STUBDOM_LINUX: y
+  tags:
+  - x86_64
+  - testlab
 
 debian-stretch-gcc-debug:
   extends: .gcc-x86-64-build-debug
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723366.1128226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beD-0000A1-20; Thu, 16 May 2024 13:59:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723366.1128226; Thu, 16 May 2024 13:59:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beC-000096-Ok; Thu, 16 May 2024 13:59:52 +0000
Received: by outflank-mailman (input) for mailman id 723366;
 Thu, 16 May 2024 13:59:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7beA-0006bS-EW
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:50 +0000
Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com
 [103.168.172.151]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8f468cfc-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:49 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.nyi.internal (Postfix) with ESMTP id 0F07D13803A4;
 Thu, 16 May 2024 09:59:49 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Thu, 16 May 2024 09:59:49 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:47 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f468cfc-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867989; x=1715954389; bh=neWLSYtrwE
	5oJ6YyyAYbJGxLAujbQDz+IR1NVL0iM3w=; b=R25Te2hxyjQXXz6SNJaBblO2sZ
	PJb7NAWjSbryuR3D/6jNbC/Q4OHo02DtHIUpW3eU60r8b8fl0XRB4/X6kB3CpV5n
	8yST4bI0zfYAsexvbuW/MqaJdkQGdyOvE3hr0OYNGZSCQdsMR0jLvJqkj4SHt6ck
	/0Tuo6eDsvPN5e7t8HVdIexEXcq/GVPu1OERzFrEXLX02miR9ZbgSsEqIvLVkk4T
	ZLj3LlBjUhZXiJXVKkxucVTSuyhTXB/+OmuNBp2n9lj1EDCbjGTPryL04RXB2N7q
	aaa3HpI0+k3R/t6tSertGzb2j6sOPY3w58MD3RKta/AQSlW/Vwm5yqA5rnWA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867989; x=
	1715954389; bh=neWLSYtrwE5oJ6YyyAYbJGxLAujbQDz+IR1NVL0iM3w=; b=T
	qrNwIAbphsbDC2cHlgh1nEhVghfdWyDHWyRe8jp9gnHiMjfMy+THePOxfVqg0jGh
	JXpy3HUcM7UVkZe4+63IjwCek2SVRE8junLognkckSFaFuKfzcQ9Cd1fkhDLq8Ej
	3It5xlXSg5c5R6f97JwbsawvxAEpsQb5twJE9F0SwAMLfSrhBuYmR/mtNW+pqDw/
	hRsiHZwk6Bw1uTntwM288ciyoWyXIPKm+B8o4B+jqplqa5vZDUNd/8WCWInWRa4X
	+vddsbJLhtHcHZD7Ha60slxCYfi2HfXUe3KrL6rALtvKbY4f3tiUraizWE03/aPj
	xgkxhbL9DvgIPq7ECMXag==
X-ME-Sender: <xms:VBFGZrT7oOB9MQ6u8uZvJeMJoQitOP9MMiyeTmDk8B3CwMS5i_saRw>
    <xme:VBFGZsz_lLBBzDzEuprvrhWoQP0PTdPomonLwPvLjInhrYlgLXOy7GD9QIvApcqj6
    W5-022WPGZBRw>
X-ME-Received: <xmr:VBFGZg2KsrzY_Qzq2Wd7Vpspa3hUhpl-VrrFWwl7_bDDI9r17-GKgpIVGjcwf8ryCpuH8mfrWAZTlQT6BMXU7RT1jxljUZ-g_GYNqGrPCSmXokeA_4w>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:VBFGZrDh647gYotZwpGNTrV7rUAa-QK67xc1wvA4slKv9NSoXs_a5Q>
    <xmx:VBFGZkj8trFA-utu_xvvQ22hoJySfdQQDLzh4RChNTKLuEdvAiOmaA>
    <xmx:VBFGZvq22JQN6nlAiWE6SHUwsz_y9d8oSsIeti_tKQLxzSd0h72SOg>
    <xmx:VBFGZvgVwAFVNx6o0-R7oKiNYMvr3EpHzbR1n_6apMrzqlbVNQXGiQ>
    <xmx:VRFGZttBX2dFBfjsiavjICC2GIrZ7fd9diiaVasvcUJTdI8MLy1pXRpo>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 10/12] automation: stubdom test with PCI passthrough
Date: Thu, 16 May 2024 15:58:31 +0200
Message-ID: <16c7001cfec1cb7a601414b745655656ea63fe29.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Based on the initial stubdomain test and existing PCI passthrough tests,
add one that combines both.
Schedule it on the AMD runner, as it has less tests right now.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/gitlab-ci/test.yaml     |  8 ++++++++
 automation/scripts/qubes-x86-64.sh | 30 +++++++++++++++++++++++++-----
 2 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index e3910f4c1a9f..76cc430ae00f 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -231,6 +231,14 @@ zen3p-pci-hvm-x86-64-gcc-debug:
     - *x86-64-test-needs
     - alpine-3.19-gcc-debug
 
+zen3p-pci-stubdom-x86-64-gcc-debug:
+  extends: .zen3p-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh pci-stubdom 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.19-gcc-debug
+
 qemu-smoke-dom0-arm64-gcc:
   extends: .qemu-arm64
   script:
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index fc73403dbadf..816c16fbab3e 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -98,8 +98,8 @@ ping -c 10 192.168.0.2 || exit 1
 echo \"${passed}\"
 "
 
-### test: pci-pv, pci-hvm
-elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ]; then
+### test: pci-pv, pci-hvm, pci-stubdom
+elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ] || [ "${test_variant}" = "pci-stubdom" ]; then
 
     if [ -z "$PCIDEV" ]; then
         echo "Please set 'PCIDEV' variable with BDF of test network adapter" >&2
@@ -109,15 +109,35 @@ elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ]; then
 
     passed="pci test passed"
 
-    domU_config='
+    domain_type="${test_variant#pci-}"
+    if [ "$test_variant" = "pci-stubdom" ]; then
+        domain_type="hvm"
+        domU_config='
+type = "hvm"
+disk = [ "/srv/disk.img,format=raw,vdev=xvda" ]
+device_model_version = "qemu-xen"
+device_model_stubdomain_override = 1
+on_reboot = "destroy"
+# libxl configures vkb backend to be dom0 instead of the stubdomain, defer
+# changing that until there is consensus what to do about VGA output (VNC)
+vkb_device = 0
+'
+        domU_disk_path=/srv/disk.img
+    else
+        domU_config='
 type = "'${test_variant#pci-}'"
-name = "domU"
 kernel = "/boot/vmlinuz"
 ramdisk = "/boot/initrd-domU"
 extra = "root=/dev/ram0 console=hvc0 earlyprintk=xen"
+disk = [ ]
+'
+    fi
+
+    # common part
+    domU_config="$domU_config"'
+name = "domU"
 memory = 512
 vif = [ ]
-disk = [ ]
 pci = [ "'$PCIDEV',seize=1" ]
 on_reboot = "destroy"
 '
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723367.1128236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beE-0000Vh-FP; Thu, 16 May 2024 13:59:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723367.1128236; Thu, 16 May 2024 13:59:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beE-0000V0-5n; Thu, 16 May 2024 13:59:54 +0000
Received: by outflank-mailman (input) for mailman id 723367;
 Thu, 16 May 2024 13:59:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7beC-0006bS-N1
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:52 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9094c71b-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 15:59:52 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id 3D62D11400E9;
 Thu, 16 May 2024 09:59:51 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Thu, 16 May 2024 09:59:51 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:49 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9094c71b-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1715867991; x=1715954391; bh=+fHnK5t5dh
	ObVIi07JV7R8fbSKdAnIHiqOJTfPBBf/o=; b=xih3xk6iUC4oRz+Ikirrjztdz7
	N8eXJSIh5k+/jg3O3zYpAjbiXc4CR3EmmJ/FQuioLz1+GjxfeqNYcrln3Vmpom5g
	/4mLHwvyxuYch0IWuoyxCgiVYbM+WDWESzDY7BSiwXFl2kYqhFoBKZTgGeW7l0Xf
	nNWyKa5JrLhdfLPSISdXT1znSyUr+On/KKrAsP62LMs0oK6ypwNDvfoJq/HDBdXV
	OMIwmv45bQxyLHxpdyu/ehyK236xsKZF7pTa8khMPwf1Az/zzvNeYnGI+YN0yHph
	rIls2xqlOPL3Otr+4/ygltYHCVTSBAGjDXwLK6Ofp+l4DRiaokBMLAsUB8Og==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867991; x=
	1715954391; bh=+fHnK5t5dhObVIi07JV7R8fbSKdAnIHiqOJTfPBBf/o=; b=g
	dqtQML5KABpcZ5M6iT+EzKk5qgZeGkMKV5d5rNvX/p6owhz6F3w7v1GPqQW9ysRt
	xqSz3Fc1VpPKkHixdv8HUOScrBxrr2SVH+D3CgYwHvTC1uSTr5PJ1AYmgO2IGA57
	8gc0ov1U96rnmvpKYFflCTHJaERDlTMEgjAuy2exNGX5qS/f5SCwGKsvrer8UtVv
	JdKT/oj/AFXZh8/ded4Od83+i8RePrHSoP9TyJ2re9VYxA/hoZIeDwr+fppMRp/7
	z/YdnsibCMp1thSKVQWBFA3CWcMPr7XgYBvYE4iYNsI1Lc8Ngj6zqEfMesC1BAmX
	0sBR1oQQPXpYBi3YQ4KYQ==
X-ME-Sender: <xms:VhFGZvOZx2AFw7_bwjcP0bra6gWa4x0NMvtAoOqSsxjDcA0jMu4r4Q>
    <xme:VhFGZp_6RnepA9pLnytdaywnlzb99BeaM-cQBDAUEyI_-N430yFzcyRBbGIooPeQv
    0QET1zb59U_SA>
X-ME-Received: <xmr:VhFGZuTnV3LzJw4XfgObY_wG9QtNafPe5nJE8J6RoI-gY1c5Mp1lKwJzNd0fu_0ZCfGnDZWfVcWE9su1OznZWseOYa6LpeOn4fbLCp0q62zXm4EF-g0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:VhFGZjsA2GwYiTirVCJDoQ4ASosjvfV5uvJj8Gn8D_VkLIFeOFQPyQ>
    <xmx:VhFGZnff1AQ3FvgVlRTfrnp3xAGw6MjqFQfGmroytvSWTCfBgOzQ_g>
    <xmx:VhFGZv2rAgSRyZDGmQ1WhdeErujnBM-dCzMD9PRRIJUB5Efz4F41sQ>
    <xmx:VhFGZj8agoEeYLOX5cAN0soz-9YRGfWTNcgUkDGsvevwN90i_vGrWw>
    <xmx:VxFGZj60YxH2btEKbMMRtQSwNmZ_7zq7y7bX9zybj23TTR2ydMb2mifG>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 11/12] automation: stubdom test with boot from CDROM
Date: Thu, 16 May 2024 15:58:32 +0200
Message-ID: <4c08412876def58402f639b5cef2b9c7d5fffa96.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Based on the initial stubdomain test add booting from CDOM. It's
significantly different in terms of emulated devices (contrary to PV
disk, the cdrom is backed by qemu), so test that path too.
Schedule it on the AMD runner, as it has less tests right now.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/build/alpine/3.19-arm64v8.dockerfile   |  1 +-
 automation/gitlab-ci/build.yaml                   |  2 +-
 automation/gitlab-ci/test.yaml                    |  8 ++-
 automation/scripts/qubes-x86-64.sh                | 58 +++++++++++-----
 automation/tests-artifacts/alpine/3.19.dockerfile |  3 +-
 5 files changed, 56 insertions(+), 16 deletions(-)

diff --git a/automation/build/alpine/3.19-arm64v8.dockerfile b/automation/build/alpine/3.19-arm64v8.dockerfile
index 12810f87ecc6..03a3f28ff686 100644
--- a/automation/build/alpine/3.19-arm64v8.dockerfile
+++ b/automation/build/alpine/3.19-arm64v8.dockerfile
@@ -49,3 +49,4 @@ RUN apk --no-cache add \
   fakeroot \
   sfdisk \
   e2fsprogs \
+  xorriso \
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 134a01d03efa..f1e6a6144c90 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -324,10 +324,12 @@ alpine-3.19-rootfs-export:
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
     - cp /grub-core.img binaries/grub-core.img
+    - cp /grub-core-eltorito.img binaries/grub-core-eltorito.img
   artifacts:
     paths:
       - binaries/initrd.tar.gz
       - binaries/grub-core.img
+      - binaries/grub-core-eltorito.img
   tags:
     - x86_64
 
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 76cc430ae00f..4e4dca91c26e 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -239,6 +239,14 @@ zen3p-pci-stubdom-x86-64-gcc-debug:
     - *x86-64-test-needs
     - alpine-3.19-gcc-debug
 
+zen3p-stubdom-hvm-cdboot-x86-64-gcc-debug:
+  extends: .zen3p-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh stubdom-hvm-cdboot 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.19-gcc-debug
+
 qemu-smoke-dom0-arm64-gcc:
   extends: .qemu-arm64
   script:
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 816c16fbab3e..b4f5c846ffe3 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -19,6 +19,7 @@ vif = [ "bridge=xenbr0", ]
 disk = [ ]
 '
 domU_disk_path=
+domU_disk_type=disk
 
 ### helper functions
 
@@ -27,27 +28,47 @@ build_domU_disk() {
     local initrd="$2"
     local rootfs="$3"
     local output="$4"
+    local img_type="$5"
     local grubcfg="$rootfs/boot/grub2/grub.cfg"
-    local kernel_cmdline="root=/dev/xvda1 console=hvc0 earlyprintk=xen"
+    local kernel_cmdline
 
     mkdir -p "$rootfs/boot/grub2"
     cp "$kernel" "$rootfs/boot/vmlinuz"
+    if [ "$img_type" = "disk" ]; then
+        kernel_cmdline="root=/dev/xvda1 console=hvc0 earlyprintk=xen"
+    elif [ "$img_type" = "cdrom" ]; then
+        kernel_cmdline="root=/dev/sr0 console=hvc0 earlyprintk=xen"
+    fi
     echo "linux /boot/vmlinuz $kernel_cmdline" >> "$grubcfg"
     if [ -n "$initrd" ]; then
         cp "$initrd" "$rootfs/boot/initrd.img"
         echo "initrd /boot/initrd.img" >> "$grubcfg"
     fi
     echo "boot" >> "$grubcfg"
-    size=$(du -sm "$rootfs")
-    size=${size%%	*}
-    # add 5M margin
-    size=$(( size + 5 ))
-    mke2fs -d "$rootfs" "$output.part1" ${size}m
-    cat "$rootfs/usr/lib/grub/i386-pc/boot_hybrid.img" binaries/grub-core.img > "$output"
-    # align for the partition 1 start (2048 sectors)
-    truncate -s $((2048 * 512)) "$output"
-    cat "$output.part1" >> "$output"
-    echo ",,linux,*" | sfdisk "$output"
+    if [ "$img_type" = "disk" ]; then
+        size=$(du -sm "$rootfs")
+        size=${size%%	*}
+        # add 5M margin
+        size=$(( size + 5 ))
+        mke2fs -d "$rootfs" "$output.part1" ${size}m
+        cat "$rootfs/usr/lib/grub/i386-pc/boot_hybrid.img" binaries/grub-core.img > "$output"
+        # align for the partition 1 start (2048 sectors)
+        truncate -s $((2048 * 512)) "$output"
+        cat "$output.part1" >> "$output"
+        echo ",,linux,*" | sfdisk "$output"
+    elif [ "$img_type" = "cdrom" ]; then
+        cp binaries/grub-core-eltorito.img "$rootfs/boot/"
+        xorriso -as mkisofs \
+            -o "$output" \
+            -b boot/grub-core-eltorito.img \
+            -no-emul-boot \
+            -boot-load-size 4 \
+            -boot-info-table \
+            "$rootfs"
+    else
+        echo "Invalid img_type: $img_type" >&2
+        return 1
+    fi
 }
 
 ### test: smoke test & smoke test PVH
@@ -168,16 +189,22 @@ until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.
 done
 "
 
-### test: stubdom-hvm
-elif [ "${test_variant}" = "stubdom-hvm" ]; then
+### test: stubdom-hvm, stubdom-hvm-cdboot
+elif [ "${test_variant}" = "stubdom-hvm" ] || [ "${test_variant}" = "stubdom-hvm-cdboot" ] ; then
     passed="ping test passed"
 
+    disk_opts=
+    if [ "${test_variant}" = "stubdom-hvm-cdboot" ]; then
+        disk_opts=",devtype=cdrom"
+        domU_disk_type="cdrom"
+    fi
+
     domU_config='
 type = "hvm"
 name = "domU"
 memory = 512
 vif = [ "bridge=xenbr0", ]
-disk = [ "/srv/disk.img,format=raw,vdev=xvda" ]
+disk = [ "/srv/disk.img,format=raw,vdev=xvda'"$disk_opts"'" ]
 device_model_version = "qemu-xen"
 device_model_stubdomain_override = 1
 on_reboot = "destroy"
@@ -229,7 +256,8 @@ if [ -n "$domU_disk_path" ]; then
         "binaries/bzImage" \
         "" \
         "rootfs" \
-        "binaries/disk.img"
+        "binaries/disk.img" \
+        "$domU_disk_type"
 else
     (cd rootfs; find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz)
 fi
diff --git a/automation/tests-artifacts/alpine/3.19.dockerfile b/automation/tests-artifacts/alpine/3.19.dockerfile
index cfb2cb30fb30..7632b694c3da 100644
--- a/automation/tests-artifacts/alpine/3.19.dockerfile
+++ b/automation/tests-artifacts/alpine/3.19.dockerfile
@@ -68,4 +68,5 @@ RUN \
   cd / && \
   tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var && \
   # Prepare boot sector for HVM disk
-  grub-mkimage -o /grub-core.img -O i386-pc -p '(hd0,msdos1)/boot/grub2' boot part_msdos ext2 linux biosdisk configfile normal
+  grub-mkimage -o /grub-core.img -O i386-pc -p '(hd0,msdos1)/boot/grub2' boot part_msdos ext2 linux biosdisk configfile normal && \
+  grub-mkimage -o /grub-core-eltorito.img -O i386-pc-eltorito -p '(cd)/boot/grub2' boot part_msdos ext2 linux biosdisk configfile normal iso9660
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 13:59:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 13:59:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723368.1128247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beG-0000yJ-Rb; Thu, 16 May 2024 13:59:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723368.1128247; Thu, 16 May 2024 13:59:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7beG-0000y4-MD; Thu, 16 May 2024 13:59:56 +0000
Received: by outflank-mailman (input) for mailman id 723368;
 Thu, 16 May 2024 13:59:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7beE-0005tl-PR
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 13:59:54 +0000
Received: from fhigh5-smtp.messagingengine.com
 (fhigh5-smtp.messagingengine.com [103.168.172.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 91559875-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 15:59:53 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id 76D441140191;
 Thu, 16 May 2024 09:59:52 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Thu, 16 May 2024 09:59:52 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 09:59:51 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91559875-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to; s=fm3; t=1715867992; x=1715954392; bh=8Lwmc35RtboctQggSkgi0
	X2iYEdiNrOG6NFi8XBFJfU=; b=Y07+ofj+C6yaj8CQ62l94Sw9BFdAcITaXDbUd
	oWyna9YEHPrJLiYuedx+YCE15d80c+bmJ8Kc/dci9UUZ0wQb/bARIHmOfXHEfAYx
	q5HCiHRXNnwv1yI7lpi8sfFKBdPffc2+bwx7zV1EKvpgNRX7rE3mwc+5sKkZAipS
	jtRRzbEQqk6HpfRnuorguonI58g2hgNy3QU7+nt8ADaW47vRMPi91WKR+6nygN2W
	/535QV7Z22QbtoM1JBd5XrgrPU1aOD7n/hEyIszi5pg5M6U4/QigTzosy1Ux/mfb
	lL+IKh3FFwiE/dXJKrRoXo9VjDmq13CmBIFMZhG8Wp1/nA01g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:date:date:feedback-id:feedback-id:from:from
	:in-reply-to:in-reply-to:message-id:mime-version:references
	:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1715867992; x=
	1715954392; bh=8Lwmc35RtboctQggSkgi0X2iYEdiNrOG6NFi8XBFJfU=; b=G
	XaxK/XmnS8GPzNtSwQ9lYdOpRtfzAX4k1UF7pBTDxBF/LCvqAuL7tRCWxhdDy/w4
	frsSh0+VusIHys6Tp1RhQXWsjXX4jv0Gzh43qqCfd9mXzMqbIO2jnXi0SOPNQFtP
	0zxA1l9BI+Du3NzxOSdfWlCdjkyVGr908MucYli7crb+2D5X258x5Xi0OEa6UaeX
	Mo0qp9ND9OzidDt923q1wtuzJYWWEA3+KlFyYb4XVu9ruVg57eNbajVeTRtoytVJ
	H1snnNogValp3Nvh9k5A9icgTM53xwcWT8lp9XGnL2pVJQasG0qvg2KW0eLxK4V2
	tKXsdQPbAAzr0JfHgcbxw==
X-ME-Sender: <xms:WBFGZl4nLb70zh8qrl2tqAhKDyU_E9g93nnWIxjEIxKPqKpWwJ_0ng>
    <xme:WBFGZi5TNqFRN41AR8nB7ZJ-o8UmbMgF_UkE_E-qGn1V8WnmVD8u389qFeEF99AuD
    ShUKwqQUj0z8w>
X-ME-Received: <xmr:WBFGZscWpfyTzaDfcQkqaIrWjxvhFd1WpRliT5-1NjMd6vURJrwhL0hvHC2jOer4WdwHay9UWW6AhU7RXX-wlIZeGjzOTLRWU5rOO5wkq8ylrNp8ibg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehuddgieelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueet
    hefgieehheeftdeuleduhffgieelueejhffggfeiieevtefhfeffffeftdehnecuffhomh
    grihhnpehgihhtlhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr
    mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh
    hlrggsrdgtohhm
X-ME-Proxy: <xmx:WBFGZuLoGLuJWaZq3hsWG5sizxh9_tOz9y6ruvH5K13oschCr_vtNQ>
    <xmx:WBFGZpL4cfPXWHlaZqDptJ-mjLtD0Ms6_-9HOhkrNbAvhp_S3zsCZw>
    <xmx:WBFGZnyeWFAnU57AzBuWSkQPAOPl-xp0pAEOfjjuhTTw5qT59znh5A>
    <xmx:WBFGZlJ0DOlBUtaoHNgHllOsmnO7KqHYttd6uarA1NyCC0GqFQaOlg>
    <xmx:WBFGZp3Xbzd0XyOsRetrv-AqPwiSpboj2G_1_HSQucziPf5qI0_e5Xt6>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 12/12] [DO NOT MERGE] switch to my containers fork
Date: Thu, 16 May 2024 15:58:33 +0200
Message-ID: <c9d3a79f5620f14326e54bf1f56ef0c18740d031.1715867907.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

---
 automation/gitlab-ci/build.yaml | 19 ++++++++++++++++---
 automation/gitlab-ci/test.yaml  |  9 ++++++++-
 2 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index f1e6a6144c90..88a59692a881 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -260,7 +260,7 @@
 
 alpine-3.19-arm64-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19-arm64v8
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/alpine:3.19-arm64v8
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
   artifacts:
@@ -320,7 +320,7 @@ qemu-system-ppc64-8.1.0-ppc64-export:
 
 alpine-3.19-rootfs-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/alpine:3.19
   script:
     - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
     - cp /grub-core.img binaries/grub-core.img
@@ -335,7 +335,7 @@ alpine-3.19-rootfs-export:
 
 kernel-6.1.90-export:
   extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.90
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/kernel:6.1.90
   script:
     - mkdir binaries && cp /bzImage binaries/bzImage
   artifacts:
@@ -350,11 +350,13 @@ kernel-6.1.90-export:
 
 alpine-3.19-gcc:
   extends: .gcc-x86-64-build
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19
 
 alpine-3.19-gcc-debug:
   extends: .gcc-x86-64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19
     STUBDOM_LINUX: y
@@ -445,28 +447,33 @@ debian-bookworm-gcc-debug-arm64-randconfig:
 
 alpine-3.19-gcc-arm64:
   extends: .gcc-arm64-build
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
 
 alpine-3.19-gcc-debug-arm64:
   extends: .gcc-arm64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
 
 alpine-3.19-gcc-arm64-randconfig:
   extends: .gcc-arm64-build
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
     RANDCONFIG: y
 
 alpine-3.19-gcc-debug-arm64-randconfig:
   extends: .gcc-arm64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
     RANDCONFIG: y
 
 alpine-3.19-gcc-debug-arm64-staticmem:
   extends: .gcc-arm64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
@@ -476,6 +483,7 @@ alpine-3.19-gcc-debug-arm64-staticmem:
 
 alpine-3.19-gcc-debug-arm64-static-shared-mem:
   extends: .gcc-arm64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
@@ -485,6 +493,7 @@ alpine-3.19-gcc-debug-arm64-static-shared-mem:
 
 alpine-3.19-gcc-debug-arm64-boot-cpupools:
   extends: .gcc-arm64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19-arm64v8
     EXTRA_XEN_CONFIG: |
@@ -598,11 +607,13 @@ debian-bookworm-gcc-arm64-cppcheck:
 
 alpine-3.19-clang:
   extends: .clang-x86-64-build
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19
 
 alpine-3.19-clang-debug:
   extends: .clang-x86-64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: alpine:3.19
 
@@ -698,11 +709,13 @@ debian-bookworm-32-gcc-debug:
 
 fedora-gcc:
   extends: .gcc-x86-64-build
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: fedora:39
 
 fedora-gcc-debug:
   extends: .gcc-x86-64-build-debug
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
   variables:
     CONTAINER: fedora:39
 
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 4e4dca91c26e..0f36036d8275 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -1,6 +1,6 @@
 .test-jobs-common:
   stage: test
-  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  image: registry.gitlab.com/xen-project/people/marmarek/xen/${CONTAINER}
 
 .arm64-test-needs: &arm64-test-needs
   - alpine-3.19-arm64-rootfs-export
@@ -16,6 +16,7 @@
 
 .qemu-arm64:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:bookworm-arm64v8
     LOGFILE: qemu-smoke-arm64.log
@@ -29,6 +30,7 @@
 
 .qemu-arm32:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:bookworm-arm64v8
     LOGFILE: qemu-smoke-arm32.log
@@ -42,6 +44,7 @@
 
 .qemu-x86-64:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:bookworm
     LOGFILE: qemu-smoke-x86-64.log
@@ -55,6 +58,7 @@
 
 .qemu-riscv64:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: archlinux:current-riscv64
     LOGFILE: qemu-smoke-riscv64.log
@@ -68,6 +72,7 @@
 
 .qemu-ppc64le:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:bullseye-ppc64le
     LOGFILE: qemu-smoke-ppc64le.log
@@ -81,6 +86,7 @@
 
 .xilinx-arm64:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: ubuntu:xenial-xilinx
     LOGFILE: qemu-smoke-xilinx.log
@@ -129,6 +135,7 @@
 # Test jobs
 build-each-commit-gcc:
   extends: .test-jobs-common
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:bookworm
     XEN_TARGET_ARCH: x86_64
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Thu May 16 14:01:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 14:01:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723378.1128256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bgD-0005Y7-Lc; Thu, 16 May 2024 14:01:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723378.1128256; Thu, 16 May 2024 14:01:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bgD-0005Y0-Im; Thu, 16 May 2024 14:01:57 +0000
Received: by outflank-mailman (input) for mailman id 723378;
 Thu, 16 May 2024 14:01:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7bgC-0005XX-U7
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 14:01:56 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id da249614-138c-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 16:01:54 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59ad12efe3so152006066b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 07:01:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eb5ec296sm3404078a12.35.2024.05.16.07.01.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 07:01:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da249614-138c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715868114; x=1716472914; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ujnsNAx1jCOYPPKGm0C2coQ22R14RMeSetkSQ4kgQpI=;
        b=YpIgfhOHWA19edn1dYkNmAQ1xZTyx4rrbZKQEZVRO5O0I8R8cYmYFlqoMUJgIlNAsc
         GKzyk3bRRCKWig3Bwh/TmmuiDKH30RaDbEekctkBuAu9fh1ikafHkh/R6v70sRZQuL7+
         zwc0VxzXRnw4SSPC/vlGMY7vqYRPo0tZrerfncs5gFKDscKV1xiYL7d8IhVOLAFtkjv4
         EotZZkUmPJMnDoWZwz12+G/M7wXt9MKjWhmg8iPN4Rz2nJq3fvUJ0a7AKAhvhQKwAmxH
         1rRoLntZ0xzxL202WebN7zRnE8l9ykns6K05YqT/+U0mSY+aH8oVrF+Uu4uyNgq/cGil
         JGgg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715868114; x=1716472914;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ujnsNAx1jCOYPPKGm0C2coQ22R14RMeSetkSQ4kgQpI=;
        b=GWrjTegF2QOD/2vaCoUz/MkQOvki0vMD0i72pNi/7Bkumecn+S02J6A6ns8EtB0w0f
         OhmYT+8AG6V9HFpU97mg5pOqP4G/vaolxOYzezzxT0neyA9HfJK+SIdNlZEvL8JJCiia
         31GFXdUsJBjf2Yz0YzniH2QkP3TKCIE6pXx6olrece9n7/ILaMBal5E0qim6Qqux1K5+
         J22cIS8O7SWteuZ1Gho3C2wId2sccO6D5ftBFQCfe34dfb7dZEXe39pjGEuGwqQlqKFu
         52G2QGbXtGgKPqlQ8vewv9yAXxgTyZC2UNSYTOuh62X4NjkpRxI4sRR5ZOUaAPKYL4nZ
         b3Fg==
X-Forwarded-Encrypted: i=1; AJvYcCWl1xfURu1JMNNpN1SQMcxdh85sAgkBydpdlu03u6upvbhnRHOaOj/3KKMhtKEQjUU1RuMugCTBzho/rSUyoC/n2v7gNyg8sin9nJZmY3Q=
X-Gm-Message-State: AOJu0YxELyJmG7r0mFsPwGGXGRL6vZSG01KIQc4SgLcxNvnthuDyDXL+
	obVrS63xM05u2K0+triPwIKm6fhrkpys/3bKVfrD99BFc501AZ1TaX24mp/CkA==
X-Google-Smtp-Source: AGHT+IHvWE+TRNm9kP+7gFH5qyrY6kA1hpq9BGUCIFPQ1f08XDQ3RlBLHMAkblTfiUjObY4aRCN0Jg==
X-Received: by 2002:a50:d741:0:b0:572:983e:4aaa with SMTP id 4fb4d7f45d1cf-5734d67f615mr16940880a12.31.1715868114267;
        Thu, 16 May 2024 07:01:54 -0700 (PDT)
Message-ID: <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
Date: Thu, 16 May 2024 16:01:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Content-Language: en-US
To: Jiqian Chen <Jiqian.Chen@amd.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
 Huang Rui <Ray.Huang@amd.com>, xen-devel@lists.xenproject.org
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516095235.64128-6-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 11:52, Jiqian Chen wrote:
> Some type of domain don't have PIRQ, like PVH, when
> passthrough a device to guest on PVH dom0, callstack
> pci_add_dm_done->XEN_DOMCTL_irq_permission will failed
> at domain_pirq_to_irq.
> 
> So, add a new hypercall to grant/revoke gsi permission
> when dom0 is not PV or dom0 has not PIRQ flag.

Honestly I find this hard to follow, and thus not really making clear why
no other existing mechanism could be used.

> Signed-off-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---

Below here in an RFC patch you typically would want to put specific items
you're seeking feedback on. Without that it's hard to tell why this is
marked RFC.

> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -237,6 +237,37 @@ long arch_do_domctl(
>          break;
>      }
>  
> +    case XEN_DOMCTL_gsi_permission:
> +    {
> +        unsigned int gsi = domctl->u.gsi_permission.gsi;
> +        int allow = domctl->u.gsi_permission.allow_access;

bool?

> +        if ( is_pv_domain(current->domain) || has_pirq(current->domain) )
> +        {
> +            ret = -EOPNOTSUPP;
> +            break;
> +        }

Such a restriction imo wants explaining in a comment.

> +        if ( gsi >= nr_irqs_gsi )
> +        {
> +            ret = -EINVAL;
> +            break;
> +        }
> +
> +        if ( !irq_access_permitted(current->domain, gsi) ||

I.e. assuming IRQ == GSI? Is that a valid assumption when any number of
source overrides may be surfaced by ACPI?

> +             xsm_irq_permission(XSM_HOOK, d, gsi, allow) )

Here I'm pretty sure you can't very well re-use an existing hook, as the
value of interest is in a different numbering space, and a possible hook
function has no way of knowing which one it is. Daniel?

> +        {
> +            ret = -EPERM;
> +            break;
> +        }
> +
> +        if ( allow )
> +            ret = irq_permit_access(d, gsi);
> +        else
> +            ret = irq_deny_access(d, gsi);

As above I'm afraid you can't assume IRQ == GSI.

> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -447,6 +447,13 @@ struct xen_domctl_irq_permission {
>  };
>  
>  
> +/* XEN_DOMCTL_gsi_permission */
> +struct xen_domctl_gsi_permission {
> +    uint32_t gsi;
> +    uint8_t allow_access;    /* flag to specify enable/disable of x86 gsi access */
> +};

Explicit padding please, including a check that it's zero on input.

> +
> +
>  /* XEN_DOMCTL_iomem_permission */

No double blank lines please. In fact you will want to break the double blank
lines in leading context, inserting in the middle.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 14:02:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 14:02:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723381.1128267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bgr-0006Az-UA; Thu, 16 May 2024 14:02:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723381.1128267; Thu, 16 May 2024 14:02:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7bgr-0006Aq-RU; Thu, 16 May 2024 14:02:37 +0000
Received: by outflank-mailman (input) for mailman id 723381;
 Thu, 16 May 2024 14:02:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=G7yJ=MT=amazon.es=prvs=85909aada=nsaenz@srs-se1.protection.inumbo.net>)
 id 1s7bgq-00069X-6h
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 14:02:36 +0000
Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com
 [207.171.184.29]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f13a966b-138c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 16:02:35 +0200 (CEST)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-9102.sea19.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 May 2024 14:02:24 +0000
Received: from EX19MTAEUC001.ant.amazon.com [10.0.17.79:61258]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.38.98:2525]
 with esmtp (Farcaster)
 id 669cf238-0252-433a-b5ae-10131f992275; Thu, 16 May 2024 14:02:23 +0000 (UTC)
Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by
 EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1258.28; Thu, 16 May 2024 14:02:23 +0000
Received: from localhost (10.13.235.138) by EX19D004EUC001.ant.amazon.com
 (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Thu, 16 May
 2024 14:02:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f13a966b-138c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1715868155; x=1747404155;
  h=mime-version:content-transfer-encoding:date:message-id:
   cc:from:to:references:in-reply-to:subject;
  bh=PII+OyY3HRsFOpjW5l++XBk5W4fZ/zADl3BCmJlUORM=;
  b=MxGCVqenHOOu1amiqFPIJj8iHolMiELzMBkfgtz7aRgui5laU37yroIm
   PQlMALS83R8GC1Hep2GTybOqW92bpd5lOId6oaujqhVvjc75ecQ6PUtaJ
   kfxBDC2COnweQbgDZQbxg4MreqHCUZ5EY8+IykAJjK5klmj0rZDibvbBM
   g=;
X-IronPort-AV: E=Sophos;i="6.08,164,1712620800"; 
   d="scan'208";a="419475431"
Subject: Re: [RFC PATCH v3 3/5] KVM: x86: Add notifications for Heki policy
 configuration and violation
X-Farcaster-Flow-ID: 669cf238-0252-433a-b5ae-10131f992275
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="UTF-8"
Date: Thu, 16 May 2024 14:02:09 +0000
Message-ID: <D1B4HKJAJG21.2DH9F3E1Q6J9L@amazon.com>
CC: Sean Christopherson <seanjc@google.com>, Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>, "H . Peter Anvin" <hpa@zytor.com>,
	Ingo Molnar <mingo@redhat.com>, Kees Cook <keescook@chromium.org>, "Paolo
 Bonzini" <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, "Vitaly
 Kuznetsov" <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, "Rick P
 Edgecombe" <rick.p.edgecombe@intel.com>, Alexander Graf <graf@amazon.com>,
	Angelina Vu <angelinavu@linux.microsoft.com>, Anna Trikalinou
	<atrikalinou@microsoft.com>, Chao Peng <chao.p.peng@linux.intel.com>,
	"Forrest Yuan Yu" <yuanyu@google.com>, James Gowans <jgowans@amazon.com>,
	James Morris <jamorris@linux.microsoft.com>, John Andersen
	<john.s.andersen@intel.com>, "Madhavan T . Venkataraman"
	<madvenka@linux.microsoft.com>, Marian Rotariu <marian.c.rotariu@gmail.com>,
	=?utf-8?q?Mihai_Don=C8=9Bu?= <mdontu@bitdefender.com>,
	=?utf-8?q?Nicu=C8=99or_C=C3=AE=C8=9Bu?= <nicu.citu@icloud.com>, Thara
 Gopinath <tgopinath@microsoft.com>, "Trilok Soni" <quic_tsoni@quicinc.com>,
	Wei Liu <wei.liu@kernel.org>, Will Deacon <will@kernel.org>, Yu Zhang
	<yu.c.zhang@linux.intel.com>, =?utf-8?q?=C8=98tefan_=C8=98icleru?=
	<ssicleru@bitdefender.com>, <dev@lists.cloudhypervisor.org>,
	<kvm@vger.kernel.org>, <linux-hardening@vger.kernel.org>,
	<linux-hyperv@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-security-module@vger.kernel.org>, <qemu-devel@nongnu.org>,
	<virtualization@lists.linux-foundation.org>, <x86@kernel.org>,
	<xen-devel@lists.xenproject.org>
From: Nicolas Saenz Julienne <nsaenz@amazon.com>
To: =?utf-8?q?Micka=C3=ABl_Sala=C3=BCn?= <mic@digikod.net>
X-Mailer: aerc 0.17.0-129-gd582ac682cdf-dirty
References: <20240503131910.307630-1-mic@digikod.net>
 <20240503131910.307630-4-mic@digikod.net> <ZjTuqV-AxQQRWwUW@google.com>
 <20240506.ohwe7eewu0oB@digikod.net> <ZjmFPZd5q_hEBdBz@google.com>
 <20240507.ieghomae0UoC@digikod.net> <ZjpTxt-Bxia3bRwB@google.com>
 <D15VQ97L5M8J.1TDNQE6KLW6JO@amazon.com> <20240514.mai3Ahdoo2qu@digikod.net>
In-Reply-To: <20240514.mai3Ahdoo2qu@digikod.net>
X-Originating-IP: [10.13.235.138]
X-ClientProxiedBy: EX19D046UWA001.ant.amazon.com (10.13.139.112) To
 EX19D004EUC001.ant.amazon.com (10.252.51.190)

On Tue May 14, 2024 at 12:23 PM UTC, Micka=C3=ABl Sala=C3=BCn wrote:
> > Development happens
> > https://github.com/vianpl/{linux,qemu,kvm-unit-tests} and the vsm-next
> > branch, but I'd advice against looking into it until we add some order
> > to the rework. Regardless, feel free to get in touch.
>
> Thanks for the update.
>
> Could we schedule a PUCK meeting to synchronize and help each other?
> What about June 12?

Sounds great! June 12th works for me.

Nicolas


From xen-devel-bounces@lists.xenproject.org Thu May 16 14:22:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 14:22:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723420.1128277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7c0B-0004PU-Gi; Thu, 16 May 2024 14:22:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723420.1128277; Thu, 16 May 2024 14:22:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7c0B-0004PN-D0; Thu, 16 May 2024 14:22:35 +0000
Received: by outflank-mailman (input) for mailman id 723420;
 Thu, 16 May 2024 14:22:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7c0A-0004PH-1d
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 14:22:34 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bb8321f4-138f-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 16:22:32 +0200 (CEST)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2e564cad1f1so11152431fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 07:22:31 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d18344e9sm24121811fa.139.2024.05.16.07.22.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 07:22:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb8321f4-138f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715869351; x=1716474151; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=51qAFqO3J8VvzLgLj/wFYjT3KCxnpJFhUgC1HQ2a31g=;
        b=GDjxurYwqPq9zTC61x5IF8+7BujmOJut0e+sipE0/yA+smkeAv4ysW8A0Q7nq/L0er
         UoMHFKjPVGgDirqsc9fjSzCr3a0kkVOYjTCg5W1kN4mdzeGg2nmwg93TjHFSEyKCrTtV
         qUzK43xdV+DOIKWX78v8Cy8mDBp3RXlar8f3NG+80a27a7s5Lzr80eCvuVFKcBmgG2x8
         xgj7qqaORWuiZaz64tkZJpc7V+LNTr/ssYPVcrPdoEU/lOVbywwq6IP4bRJ5jdbtkShg
         fixAZE7ziIWj2tjyryvbs8/gwpLGKqGTP0x8/PA7wUUCT9RH14ENyoAJlPYEOJw5cHf/
         IU8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715869351; x=1716474151;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=51qAFqO3J8VvzLgLj/wFYjT3KCxnpJFhUgC1HQ2a31g=;
        b=hb1qJFSKwKiQI21qrSvw1H5wN2YlZ4C6m+TJ4Ray6c26I1ynrw7yJ0zyDoZkIZNlaJ
         s1Uj96qvlaev4ffuSot/8XE6PF3yYoyWaMUs2q2+6D7j9i4qIzsLIIPNzNo935Kfwzql
         JY/h2Yy7JbwKMntLl2gHDXPhgMMXSnXOE0xdMrLB+6u9QqnDJL4XW1ypQxQHpz6OC2NG
         ulStPNugAsKsXA0Dc6Aaqop2DHqcl9QriWziXwXNrqHNvhDN4lBsgwyNHs6dodDpZO6h
         dLOjSyIcEpUUow2RQFpiTRWN2mpOlicBb3hm3PvacjqVWiKbBLqH7L7TE1OpIsIa9pLk
         1zcg==
X-Gm-Message-State: AOJu0YyEHzAsjKtLw3VK7d+Uu32nr+R87F5FrTK3GXzTLMvVyzzN4+LY
	LUHolT42sItHU504UDIrCjaRtU0mwhCCiV1xNzDFZYtzikD8T9uhj4SEIynC
X-Google-Smtp-Source: AGHT+IHGsHzTv6LLbt1WjFRX+DiJ8oBBAQAbN2MFHtmINivy7ck/5dKY1fM892GB6vgEuPdjIZ+kLw==
X-Received: by 2002:a2e:8397:0:b0:2e6:f710:6056 with SMTP id 38308e7fff4ca-2e6f710618cmr31304871fa.18.1715869350870;
        Thu, 16 May 2024 07:22:30 -0700 (PDT)
Message-ID: <d7456a1a3d8a872297d65785466ae07fdf0bcbfd.camel@gmail.com>
Subject: Feature Freeze Deadline Extended to May 24
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Thu, 16 May 2024 16:22:30 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hello everyone,

Since there were no objections to extending the Feature Freeze Deadline
Proposal, I would like to inform you that the deadline has been
extended to May 24.

At the moment, I don't see any reason to extend other deadlines, so
they will remain the same.

Have a nice day!

Best regards,
 Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 16 14:46:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 14:46:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723431.1128286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7cNe-0002Ao-Ah; Thu, 16 May 2024 14:46:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723431.1128286; Thu, 16 May 2024 14:46:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7cNe-0002Ah-8C; Thu, 16 May 2024 14:46:50 +0000
Received: by outflank-mailman (input) for mailman id 723431;
 Thu, 16 May 2024 14:46:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7cNc-0002Ab-AD
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 14:46:48 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1eadd530-1393-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 16:46:46 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59a387fbc9so366977266b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 07:46:46 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d350sm987052566b.23.2024.05.16.07.46.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 07:46:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eadd530-1393-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715870806; x=1716475606; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jsp98pY2/VuhI4d3314NHm3LuKttPruFuyIFk+c0kVI=;
        b=ih4nwJhYQpuRiPnwUi/4YX0QzR3zexxM21PIkTYmLx8eZa1eChTDoBUNL+mjMI5fG/
         +7HVod6C/crScqGqE0vQ6150+qtiQH/2GtEtrylGqOwySo2Z+Dw7ZxzBu3IfkjQhzubW
         Bu+XOaKd5FyV5DNYY2zaAemt6Pr+5r1g+hu3I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715870806; x=1716475606;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jsp98pY2/VuhI4d3314NHm3LuKttPruFuyIFk+c0kVI=;
        b=G+MxYFBWWYyCGVqr6qehvNVOYdfcSLvV093WfuMTy95P8N6VYhKbWzw6MtZgPgVEP7
         vZsc4B8I7sdzhXnkOstb0PJeKw/uhPYEA2HmuGSQRjlC3oFZjfZvBI3RoiDUB/SSnslR
         c7WLKoAQDCMXy0Odk0QIvPSyrDN/LJUA5jkjdCGCQln4cMkmptF5dGLq9TcF0LayjWwp
         KWv8sfBvh+J/ja0hbSwKK3Wv1hwrI+JfZrb6OpSNMxDL9t2FEuB7y2ZqAXcI43N/VYqy
         y+iPS4cdEs70dSAzEdBekfU0HH6rpF48JZDNJjzFIDwKv9GyEYfDHSSusoAPuTHJwhww
         ecsw==
X-Gm-Message-State: AOJu0Yx8cNNO619yOJ7mVlZkhcYIxckeM8klV7RgeFlFyUH7QXqEoJea
	6Q1OBiu7cFd5+xskErNmwokcldeKDKrbSMwsJ+dUR6TYDnItSpDFS5jF2LLXC+Q=
X-Google-Smtp-Source: AGHT+IHSWvC31JTUtyx26mZPc9St04wgzrLKBoXO2cvY7dolOMcBeSskSOlZ0Qz1XrTL4uVePPjykQ==
X-Received: by 2002:a17:906:5fd5:b0:a59:c28a:7ec2 with SMTP id a640c23a62f3a-a5a2d5d49bemr1145265966b.41.1715870806228;
        Thu, 16 May 2024 07:46:46 -0700 (PDT)
Message-ID: <26f1627c-4325-42f0-bbb1-0bd9be35bbce@cloud.com>
Date: Thu, 16 May 2024 15:46:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
To: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
 <ZkXvdWf2mVkPW458@macbook> <7965ba74-9177-44b7-9253-7ab5a77fd88f@cloud.com>
 <ce573d24-1d52-48a8-b3f0-8636dacd5c9b@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ce573d24-1d52-48a8-b3f0-8636dacd5c9b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi,

On 16/05/2024 13:37, Jan Beulich wrote:
> On 16.05.2024 14:29, Alejandro Vallejo wrote:
>> On 16/05/2024 12:35, Roger Pau Monné wrote:
>>> On Thu, May 16, 2024 at 12:07:10PM +0100, Alejandro Vallejo wrote:
>>>> Bring test_x86_emulator in line with other tests by adding
>>>> install/uninstall rules.
>>>>
>>>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>>>> ---
>>>>  tools/tests/x86_emulator/Makefile | 11 +++++++++--
>>>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
>>>> index 834b2112e7fe..30edf7e0185d 100644
>>>> --- a/tools/tests/x86_emulator/Makefile
>>>> +++ b/tools/tests/x86_emulator/Makefile
>>>> @@ -269,8 +269,15 @@ clean:
>>>>  .PHONY: distclean
>>>>  distclean: clean
>>>>  
>>>> -.PHONY: install uninstall
>>>> -install uninstall:
>>>> +.PHONY: install
>>>> +install: all
>>>> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
>>>> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
>>>> +
>>>> +.PHONY: uninstall
>>>> +uninstall:
>>>> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
>>>> +
>>>
>>> FWIW, should you check that HOSTCC == CC before installing?  Otherwise
>>> I'm unsure of the result in cross-compiled builds, as the x86_emulator
>>> is built with HOSTCC, not CC.
>>>
>>> Thanks, Roger.
>>
>> Right...
>>
>> More generally, should we do s/CC/HOSTCC/ on all compiler checks? I see
>> no particular reason to do them on $(CC) rather than the actual compiler
>> used during build.
> 
> No. There really is a mix here, intentionally. Anything built through testcase.mk
> is using CC, and hence respective checking needs to use CC, too. That said, I
> don't think the split is done quite correctly just yet, which may raise the
> question of whether having the split is actually worth it.
> 
> Jan

I'm a bit puzzled by this. Why do we compile pieces of the test binary
with different toolchains?

At a glance it seems inconsequential in the native case and
fully broken on the cross-compiled case (which I guess is what Roger was
hinting at and I failed to notice).

Why the distinction? What am I missing?

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:01:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:01:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723437.1128300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7cbM-0006Pt-Nx; Thu, 16 May 2024 15:01:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723437.1128300; Thu, 16 May 2024 15:01:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7cbM-0006Pm-LF; Thu, 16 May 2024 15:01:00 +0000
Received: by outflank-mailman (input) for mailman id 723437;
 Thu, 16 May 2024 15:00:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7cbL-0006Pg-H8
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:00:59 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 19893287-1395-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 17:00:57 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-574bf7ab218so3945861a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:00:57 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7f78sm980320666b.133.2024.05.16.08.00.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 08:00:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19893287-1395-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715871657; x=1716476457; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nc44owd5c/D2OxMYl0HxeeYTTxhG+6KI2hLgPGk6oxs=;
        b=Tj4deZd+xTv+Jj6mS2zLy4wU3iOoh5N7t6Chx7TnQgWyTSw438ZUggbrhzh5lNktu6
         Yu34qw5cL+mrL/4q9BbyI1D8hKDq5j6jrJNaeTyubfjCZ1zs+xrbEOxUWJqLvwvf0dSp
         PV54jHWg9zVEay/n67bkWx3dZRPZbFfp5mDUZVPC8w49843Kntv8uj9waguVzPvHdY/A
         bl1GoBi9xfm+DeDSh1RNOZ2SY6XCcspwsatm7jBM0iiJCJqp7/MMy2mUSYeJZE9GKFGd
         C3Fo0e7yuoQUXOueviq3TJ+91YKKuhm3mwYKYnFjJTUwLNlAh3oCdw09wi2E6kPMCnIr
         aGnA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715871657; x=1716476457;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nc44owd5c/D2OxMYl0HxeeYTTxhG+6KI2hLgPGk6oxs=;
        b=jlfYbuzJnUnu+yyUlLfvRb2kP0B0ddKakqeFpXogE9svqhaAgGGPOunj7ChbiLK7jz
         PV5jZ+MtT0TFBrlN0Yqo3Y8pFL+AM/DdMcoXWQQXbpY76sO5FQFuX4knhDHw/P8UMW/F
         p3edAhkddcr7yX/9WmzJZE8cSF+seArSmrxt4KVrdRXJpIPvTe8bAI4hZZ8yoonnqina
         Ye7nJexZK4CwI9E8Qm7D4esV/Sfa/Zqz2DLsKhcK5dK003vM2ionRyhtdzkJfgvB85ZX
         ElLcCZ8cRdGRlHM3FYcaEuR/jmMdfC8yGWQf5pG6m+rlXzVHUeZQw7NSrgBoEkyx0omR
         oY5A==
X-Forwarded-Encrypted: i=1; AJvYcCUlXJwc3JeHXVGN2bwnLzicW1BBt0O3tPPJmDjsBCdxrhpCtrCFg8srWGKYkIMJ6pnzVGgj+h2OSFk2li41Ddkg+cwxWE8TUoEn3OOz1sE=
X-Gm-Message-State: AOJu0YxGbQpb0RHQUQ/EDxh7ERt+/NPv3b3ebMNNbMokPloknvZL7WL+
	N3B4xZk8YQUbQHKUpEAxEHLNSt0IyKoKHRh9/WU4IBVjx1fwSnaBPfhpcspXaA==
X-Google-Smtp-Source: AGHT+IHq5qqWkWy9RKnJcvVn80s1LOCXbHvz8i0h4fo7ygXwskAR3puJy3aBHkjNphyGztQm8sa1lA==
X-Received: by 2002:a17:907:76d3:b0:a59:c7d7:8b0f with SMTP id a640c23a62f3a-a5a2d57a717mr1165649666b.29.1715871656707;
        Thu, 16 May 2024 08:00:56 -0700 (PDT)
Message-ID: <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com>
Date: Thu, 16 May 2024 17:00:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240516132224.86005-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240516132224.86005-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 15:22, Roger Pau Monne wrote:
> --- a/xen/arch/x86/irq.c
> +++ b/xen/arch/x86/irq.c
> @@ -2527,7 +2527,7 @@ static int __init cf_check setup_dump_irqs(void)
>  }
>  __initcall(setup_dump_irqs);
>  
> -/* Reset irq affinities to match the given CPU mask. */
> +/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
>  void fixup_irqs(const cpumask_t *mask, bool verbose)
>  {

Evacuating is one purpose. Updating affinity, if need be, is another. I've
been wondering more than once though whether it is actually correct /
necessary for ->affinity to be updated by the function. As it stands you
don't remove the respective code, though.

> @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>                  release_old_vec(desc);
>          }
>  
> -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
> +        /*
> +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
> +         * that's all covered by the requested affinity mask.
> +         */
> +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
> +        if ( !desc->action || cpumask_subset(affinity, mask) )
>          {
>              spin_unlock(&desc->lock);
>              continue;

First my understanding of how the two CPU sets are used: ->affinity is
merely a representation of where the IRQ is permitted to be handled.
->arch.cpu_mask describes all CPUs where the assigned vector is valid
(and would thus need cleaning up when a new vector is assigned). Neither
of the two needs to be a strict subset of the other.

(It's not really clear whether ->arch.cpu_mask is [supposed to be] a
subset of cpu_online_map.)

If that understanding of mine is correct, going from just ->arch.cpu_mask
doesn't look quite right to me, as that may include CPUs not in ->affinity.
As in: Things could be further limited, by also ANDing in ->affinity.

At the same time your(?) and my variant suffer from cpumask_subset()
possibly returning true despite the CPU the IRQ is presently being
handled on being the one that we're in the process of bringing down. In
that case we absolutely cannot skip the move. (I'd like to note that
there are only two possible input values of "mask" for the function. The
case I think you've been looking into is when it's &cpu_online_map. In
which case cpumask_subset() is going to always return true with your
change in place, if I'm not mistaken. That seems to make your change
questionable. Yet with that I guess I'm overlooking something.)

Plus there remains the question of whether updating ->affinity can indeed
be skipped in more cases than it is right now (prior to you patch), or
whether up front we may want to get rid of that updating (in line, I think,
with earlier changes we did elsewhere) altogether.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:26:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:26:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723446.1128309 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7czR-0003Ad-OO; Thu, 16 May 2024 15:25:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723446.1128309; Thu, 16 May 2024 15:25:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7czR-0003AW-Lv; Thu, 16 May 2024 15:25:53 +0000
Received: by outflank-mailman (input) for mailman id 723446;
 Thu, 16 May 2024 15:25:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7czQ-0003AM-2I; Thu, 16 May 2024 15:25:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7czP-0006D3-Uz; Thu, 16 May 2024 15:25:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7czP-0001S5-O7; Thu, 16 May 2024 15:25:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7czP-000734-Nd; Thu, 16 May 2024 15:25:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gYPVFS6+j5Q9NNQJO/NiEUuYnV03YOwOI4B99lQONHk=; b=dMa/C2AW/Aj527icnvqm203Njx
	z6QghdGRpwV+DXY10EQkpFpmowZM25AUVXfyqcYGj79hiRzGJEbmqM8d8CJ5EiX3ih26ridMyoZsT
	lgr8EePQXiavWW/s9/TbuGjQZSYQ8B8f8RM8KRCohy0Xo83CezdK3Iq1LqORwFz5pft0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186011-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186011: tolerable FAIL - PUSHED
X-Osstest-Failures:
    libvirt:test-arm64-arm64-libvirt-qcow2:guest-start:fail:heisenbug
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=8b133e82fc62188b04dcdb2ddfc5589a48222a2c
X-Osstest-Versions-That:
    libvirt=2bcf14eabf02063fc7850a6b578f9bd6e26a6e60
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 15:25:51 +0000

flight 186011 libvirt real [real]
flight 186017 libvirt real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186011/
http://logs.test-lab.xenproject.org/osstest/logs/186017/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-qcow2 13 guest-start       fail pass in 186017-retest

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check fail in 186017 never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check fail in 186017 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186003
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              8b133e82fc62188b04dcdb2ddfc5589a48222a2c
baseline version:
 libvirt              2bcf14eabf02063fc7850a6b578f9bd6e26a6e60

Last test of basis   186003  2024-05-15 04:20:33 Z    1 days
Testing same since   186011  2024-05-16 04:20:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andi Chandler <andi@gowling.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   2bcf14eabf..8b133e82fc  8b133e82fc62188b04dcdb2ddfc5589a48222a2c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:30:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:30:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723454.1128320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7d3R-0004Dp-9g; Thu, 16 May 2024 15:30:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723454.1128320; Thu, 16 May 2024 15:30:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7d3R-0004DM-6Y; Thu, 16 May 2024 15:30:01 +0000
Received: by outflank-mailman (input) for mailman id 723454;
 Thu, 16 May 2024 15:30:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7d3P-0004DB-Uo; Thu, 16 May 2024 15:29:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7d3P-0006HU-RN; Thu, 16 May 2024 15:29:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7d3P-0001Vj-F4; Thu, 16 May 2024 15:29:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7d3P-0000FT-Eh; Thu, 16 May 2024 15:29:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aSw/1Mn4lVI8Ap0pVO2CITmSQMqmhkPoTSaIZyp5eng=; b=dxEowkTc8k/hzfCL0eIfjF4Sfl
	8hukxXDDbJ7NOCqg50OeOWqTx1R7TEa3eNAKt5B+TY32NGvC3NBktg7kp3x3jxx8kybWUpYYxK0oq
	6DYVsj8o1goR1ljngvQOVgHn99Gk15R2AfckOIuemetptrP035lxavxxWLEE9GSNx59g=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186010-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186010: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
X-Osstest-Versions-That:
    linux=1b294a1f35616977caddaddf3e9d28e576a1adbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 15:29:59 +0000

flight 186010 linux-linus real [real]
flight 186015 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186010/
http://logs.test-lab.xenproject.org/osstest/logs/186015/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   8 xen-boot                 fail REGR. vs. 186005

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail pass in 186015-retest
 test-armhf-armhf-xl           8 xen-boot            fail pass in 186015-retest
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 186015-retest
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 186015-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 186005

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 186015 like 186005
 test-armhf-armhf-xl         15 migrate-support-check fail in 186015 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 186015 never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 186015 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 186015 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 186015 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 186015 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 186015 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186005
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
baseline version:
 linux                1b294a1f35616977caddaddf3e9d28e576a1adbc

Last test of basis   186005  2024-05-15 13:12:03 Z    1 days
Testing same since   186010  2024-05-16 01:12:45 Z    0 days    1 attempts

------------------------------------------------------------
509 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 50492 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:37:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:37:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723466.1128330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dAt-0005wS-48; Thu, 16 May 2024 15:37:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723466.1128330; Thu, 16 May 2024 15:37:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dAs-0005wL-Vv; Thu, 16 May 2024 15:37:42 +0000
Received: by outflank-mailman (input) for mailman id 723466;
 Thu, 16 May 2024 15:37:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7dAs-0005wF-EJ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:37:42 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3b50c8b8-139a-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 17:37:41 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so329887566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:37:41 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b015f7sm1011801766b.146.2024.05.16.08.37.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 08:37:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b50c8b8-139a-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715873861; x=1716478661; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=26bFaUae7TNhu5UW1vOw6adqNZFB5Qaki6RPRXQQBPA=;
        b=EGMys0x30O/2Zyn/sNOoTB2ufSJ3uV5fK4oayL5f5ZAq6XQpH+3EEI/ZAeRLPZeRMM
         QhrmBWxfBJfJkKhsYisCUmHqMgfrcmLSnmzHut7OPIp04WfeMtW2HFaQQQd7kGsQQXZA
         gQiA7k7eJaqn81f+RxuO3/hx3tEbjNP4mLblk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715873861; x=1716478661;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=26bFaUae7TNhu5UW1vOw6adqNZFB5Qaki6RPRXQQBPA=;
        b=Qw+DFWvqYNMd28RN+JZ7sdhkqfExHmjkhdU/7r9UooduyUCzyOQ10I1Z7ScxS7X1fO
         SQL7I5T56/H8mEEx3m6+P+PvPMtmilNNaKRZdJn6tey4wL9rQaCXtsoUne3pHRL+7Bqc
         YzVsexRFlSWTUKL+1q1VPiEgsJ+kwhyH9i33kWUMDEfiuUdR2kLLgsfzs8URQ0ffHbDc
         lah9LkiKsBOuhXXi18pRcsqlRMCprLleUrxy2ZLd4SdP3PeskW0uLOx95gvW/y084B0c
         POZWuOfHIruHWTwftally/hPfagfYfchrFVGxcs8r2DpUARVn96y280WNMvjIF+QRk+T
         aplw==
X-Gm-Message-State: AOJu0YzmAeUP+FPmPwnKeep0oIuAn52L9NOPVRG+8sr221PceARBXxm3
	EV898LMgQ5MnsRU+1LSwMmHM9VHh5Gb1x69tf8Cyulyu+BEdSk/2BMRukz35hb0=
X-Google-Smtp-Source: AGHT+IFDhaYurIZFhAK7+Eyp/XaFNz3itMwvfVT6n9EppRrHo3MobI8PzBuABkP8NGHHQRt7VyMYzw==
X-Received: by 2002:a17:906:f111:b0:a5a:28cc:ff85 with SMTP id a640c23a62f3a-a5a2d58545amr1244604566b.28.1715873860899;
        Thu, 16 May 2024 08:37:40 -0700 (PDT)
Message-ID: <dc37751e-cf68-42af-909a-ad9a9ec3833f@cloud.com>
Date: Thu, 16 May 2024 16:37:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] tools/xg: Clean up xend-style overrides for CPU
 policies
To: Anthony PERARD <anthony.perard@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>,
 Juergen Gross <jgross@suse.com>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
 <20240207173957.19811-4-alejandro.vallejo@cloud.com>
 <2b4b05d1-154d-43b8-a29b-aa99332a8eeb@perard>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <2b4b05d1-154d-43b8-a29b-aa99332a8eeb@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30/04/2024 15:42, Anthony PERARD wrote:
> On Wed, Feb 07, 2024 at 05:39:57PM +0000, Alejandro Vallejo wrote:
>> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
>> index 5699a26b946e..cee0be80ba5b 100644
>> --- a/tools/libs/guest/xg_cpuid_x86.c
>> +++ b/tools/libs/guest/xg_cpuid_x86.c
>> @@ -772,49 +616,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>               * apic_id_size values greater than 7.  Limit the value to
>>               * 7 for now.
>>               */
>> -            if ( p->policy.extd.nc < 0x7f )
>> +            if ( cur->policy.extd.nc < 0x7f )
>>              {
>> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
>> -                    p->policy.extd.apic_id_size++;
>> +                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )
>> +                    cur->policy.extd.apic_id_size++;
>>  
>> -                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
>> +                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
>>              }
>>              break;
>>          }
>>      }
>>  
>> -    nr_leaves = ARRAY_SIZE(p->leaves);
>> -    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
>> -    if ( rc )
>> +    if ( xend || msr )
>>      {
>> -        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
>> -        goto out;
>> +        // The overrides are over the serialised form of the policy
> 
> Comments should use /* */

Ugh, yes.

> 
>> +        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
>> +            goto out;
>> +
>> +        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
>> +            goto out;
>> +        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
>> +            goto out;
> 
> What if `xend` is set, but `msr` isn't? Looks like there's going to be a
> segv in xc_msr_policy() because it doesn't check that `msr` is actually
> set.
> 
> 
> Thanks,
> 

OOPS! Yes, msrs was meant to have the same check I added for
xc_cpuid_xend_policy. Will do.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:48:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:48:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723476.1128370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dLA-0000Zp-P0; Thu, 16 May 2024 15:48:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723476.1128370; Thu, 16 May 2024 15:48:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dLA-0000Zi-Lf; Thu, 16 May 2024 15:48:20 +0000
Received: by outflank-mailman (input) for mailman id 723476;
 Thu, 16 May 2024 15:48:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7dLA-0008LQ-1R
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:48:20 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b7a75bb4-139b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 17:48:19 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-420298ff5b1so9908645e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:48:19 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee9292sm273969565e9.37.2024.05.16.08.48.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:48:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7a75bb4-139b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715874499; x=1716479299; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+z9GUm6j8WzszIWdyc3UrfaOV2gWSdTHpFYF8lhndm8=;
        b=ZetVf7jb3gRKjPFSBfAffCi8AyMsUIznWc4QbVFVZ3r9y0UaP9undzOApaVqq9LtN9
         qTdjxXIHll2VMMl0CEYqh/9X4LtEovcF+nASxO4lM3OD8i3h9eACOGjA/pZJJB4z7Pv5
         UMeLQ+vP7H+evWzV5CcPbtGagl/KJbc089kkRJXNiqffScJ8Uz6gyHx7B6BNoW7MsTHr
         KlPIdJk6v4L1ewoGYObMY+VLD3I5FTxN+b2Ri+1H/EWcWik1eKzsg9rnu2j1xYrFN0ne
         dkeZjlmI+wxyimn3KUbTlV3P49yEMfompFWRpC2KACORYsDGLRsEvT7Kvm2EZVvPAN1r
         aIZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715874499; x=1716479299;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+z9GUm6j8WzszIWdyc3UrfaOV2gWSdTHpFYF8lhndm8=;
        b=GJ/aoMXJoyIU8jW5crucGE0caY8TcSSj9qkTy+F7QOSRrf9Lt/s9cGikWO3Qk2xOGb
         7nkVvDn0fWLNdGhSG2E7mnIG59V26lQkM2li+VqPNNgV3lfEMZWXf0SmJSLMfMTNRerI
         4hM66rGexXLZGlYForLReH298tExhKdgrSA+DRfBKZB3DZbJJRzWwDRO9Lz4HzhQB4n0
         cAzN84CJA3n0jdHUFvcg4fuh/GUo0a8pYmHCN2NY81v+mHt+RE/LVRK6hOPL5A617oqR
         kqTQZMl94PtaT43gQpfTaTZx2zNiF8ge9p4sgYxE20sqAp33KmTEVZQB+1Jp9QHlo7Cj
         LYqg==
X-Forwarded-Encrypted: i=1; AJvYcCVpqJTZ0MotG7O5qJm/qCFrsnPl4QJekOh4pIXc/+MUwmQo36IHbMEBAnJ0qa1jL7yWr1ZB+GhhBmYVzal+ZrSm8hiYCNltyTxT+FvBBFA=
X-Gm-Message-State: AOJu0YzEUbL9KR1jeZE/pkQSzLYK5nRNG9moUoogmevoCNfC/fNKAIgK
	VecUpMSoOEI6r50sh3gjhEtRrL2ITQ5GqW99GIzAg+5Z1N4HYNmR
X-Google-Smtp-Source: AGHT+IGfNbqzq+QNyeonmsqylEpUTqR0Yl/VLa0jqDvd+rb0LT+CkFvkUWEMXBZQKhr0f0w8Mk8vVg==
X-Received: by 2002:a05:600c:4707:b0:41b:edf4:4071 with SMTP id 5b1f17b1804b1-41fead64731mr176758575e9.35.1715874498887;
        Thu, 16 May 2024 08:48:18 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v6 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()
Date: Thu, 16 May 2024 17:48:02 +0200
Message-Id: <20240516154804.1114245-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 hw/xen/xen-mapcache.c         | 16 +++++++++++-----
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c              |  9 +++++----
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
                              void *vaddr,
                              hwaddr size,
                              hwaddr address_index,
-                             bool dummy)
+                             bool dummy,
+                             ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
     xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
+                                       ram_addr_t ram_offset,
                                        uint8_t lock, bool dma, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
     if (!entry) {
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
-        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                         ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
                 !test_bits(address_offset >> XC_PAGE_SHIFT,
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
-            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                             ram_offset);
         }
     }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
                        hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
     uint8_t *p;
 
     mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+                               lock, dma, is_write);
     mapcache_unlock(mapcache);
     return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false);
+                     cache_size, address_index, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
                         void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
                                      hwaddr phys_addr,
                                      hwaddr size,
+                                     ram_addr_t ram_addr_offset,
                                      uint8_t lock,
                                      bool dma,
                                      bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
          */
         if (xen_mr_is_memory(block->mr)) {
             return xen_map_cache(block->mr, block->offset + addr,
-                                 len, lock, lock,
-                                 is_write);
+                                 len, block->offset,
+                                 lock, lock, is_write);
         }
 
         block->host = xen_map_cache(block->mr, block->offset,
-                                    block->max_length, 1,
-                                    lock, is_write);
+                                    block->max_length,
+                                    block->offset,
+                                    1, lock, is_write);
     }
 
     return ramblock_ptr(block, addr);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 15:48:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:48:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723473.1128340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL5-0008GN-44; Thu, 16 May 2024 15:48:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723473.1128340; Thu, 16 May 2024 15:48:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL5-0008GG-0I; Thu, 16 May 2024 15:48:15 +0000
Received: by outflank-mailman (input) for mailman id 723473;
 Thu, 16 May 2024 15:48:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7dL3-0008G5-6y
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:48:13 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b29c2323-139b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 17:48:11 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-51f99f9e0faso1127787e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:48:11 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4201916e7c6sm134563085e9.12.2024.05.16.08.48.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:48:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b29c2323-139b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715874490; x=1716479290; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=bj5mbrg4u13toU1ZMHMQUOYAMIi/iXYNkmjkQDeQwMW0bc6bpM6WA1uzq+iagJcJu3
         f1IJMawXAD5LDbxU2idU6XWYrBu8K7Urh+QEHt87Qi22ZMBaxq6vj0d8RlU3ANCurJ6a
         DNenn++Y8PWkaVBD0TF4ujhv9Nc+WtAax9fIkkjNbL4yjrp3zHzdg/ullslwr+CoL3Ze
         HhNF4DD6834YMZfitg4Ld/BlrB1hb6g1np9T7hIka9I0SKjLfe38WWX8YePEnYGZirjb
         +rMg0qTwkpoJCFXgoOh/GXROBb79rrxD2DKUruw7GmKxNiKelFa2eIECY+lBW6DENXyA
         8suA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715874490; x=1716479290;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=GO87EKIyebVt1vb8/8PcrrrxttAdwFlGeMY6VQh10XPckExG8Wr4bTn3LfHSLdzJor
         A/aA5f84JIgJYdmpKqopBarjlVVqd0P6zYjk2SclifSVkF2tBvP1xUuy4yHWEi+B0XJd
         /sfpV4ZHwJ75F/C0zj6zgyccEnXdbWcME4qAWoFf4gh2uf7Y+HYZVMdOFUI4YyPgqugm
         RV23FzY9cUCfgdnlvMoqn9TDJ4Y4JESg6UDwtShd6m5LNgiM26XDxuTiXODgVSSk+oMk
         aGETJf2P4rz492AtQkFf2TrBpijS23q2hQ+R395a4+IzldGnVByX4jr+xEGtZiQyK0RT
         5E7w==
X-Forwarded-Encrypted: i=1; AJvYcCVaf3jWqQ/4MUv1qnzsfvVLj8ry2jeHRGDFyZIECLZuRebIRloBn14bCqsivm+weWbt1/q8+Frnty/7hVpYgz4mnbSaONbzpqt0v9E18zg=
X-Gm-Message-State: AOJu0Yz33OB/AF+cdtvFV2HhGyKdc6ejQVBKDuK7Z0lufKPzMNdVvHVF
	oqSUaJh/8KScU/Q2S5yRNDYGsCLdU+LKGD6ylf+9WsW4ZLMyYH/O
X-Google-Smtp-Source: AGHT+IHDyhM75rM+/tSd8KHYsc2vk/F/hAtrKLe2KyivanA+rMbH+7GZB6FdXDOstgD6teeWWR65AQ==
X-Received: by 2002:a05:6512:b15:b0:51a:ca75:9ffe with SMTP id 2adb3069b0e04-5221057b50bmr17367040e87.42.1715874490119;
        Thu, 16 May 2024 08:48:10 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v6 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
Date: Thu, 16 May 2024 17:47:57 +0200
Message-Id: <20240516154804.1114245-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 54 ++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE     (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
     /* For most cases (>99.9%), the page address is the same. */
     MapCacheEntry *last_entry;
     unsigned long max_mcache_size;
-    unsigned int mcache_bucket_shift;
+    unsigned int bucket_shift;
+    unsigned long bucket_size;
 
     phys_offset_to_gaddr_t phys_offset_to_gaddr;
     QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
                                            void *opaque,
+                                           unsigned int bucket_shift,
                                            unsigned long max_size)
 {
     unsigned long size;
     MapCache *mc;
 
+    assert(bucket_shift >= XC_PAGE_SHIFT);
+
     mc = g_new0(MapCache, 1);
 
     mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
     QTAILQ_INIT(&mc->locked_entries);
 
+    mc->bucket_shift = bucket_shift;
+    mc->bucket_size = 1UL << bucket_shift;
     mc->max_mcache_size = max_size;
 
     mc->nr_buckets =
         (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
-         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+          (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+         (bucket_shift - XC_PAGE_SHIFT));
 
     size = mc->nr_buckets * sizeof(MapCacheEntry);
     size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
 {
     struct rlimit rlimit_as;
     unsigned long max_mcache_size;
+    unsigned int bucket_shift;
+
+    if (HOST_LONG_BITS == 32) {
+        bucket_shift = 16;
+    } else {
+        bucket_shift = 20;
+    }
 
     if (geteuid() == 0) {
         rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
         }
     }
 
-    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+    mapcache = xen_map_cache_init_single(f, opaque,
+                                         bucket_shift,
+                                         max_mcache_size);
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
     entry->valid_mapping = NULL;
 
     for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
     }
 
     /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
     bool dummy = false;
 
 tryagain:
-    address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = phys_addr >> mc->bucket_shift;
+    address_offset = phys_addr & (mc->bucket_size - 1);
 
     trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
         return mc->last_entry->vaddr_base + address_offset;
     }
 
-    /* size is always a multiple of MCACHE_BUCKET_SIZE */
+    /* size is always a multiple of mc->bucket_size */
     if (size) {
         cache_size = size + address_offset;
-        if (cache_size % MCACHE_BUCKET_SIZE) {
-            cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+        if (cache_size % mc->bucket_size) {
+            cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
         }
     } else {
-        cache_size = MCACHE_BUCKET_SIZE;
+        cache_size = mc->bucket_size;
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
         trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
         raddr = RAM_ADDR_INVALID;
     } else {
-        raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+        raddr = (reventry->paddr_index << mc->bucket_shift) +
              ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
     }
     mapcache_unlock(mc);
@@ -585,8 +597,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     hwaddr address_index, address_offset;
     hwaddr test_bit_size, cache_size = size;
 
-    address_index  = old_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = old_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = old_phys_addr >> mc->bucket_shift;
+    address_offset = old_phys_addr & (mc->bucket_size - 1);
 
     assert(size);
     /* test_bit_size is always a multiple of XC_PAGE_SIZE */
@@ -595,8 +607,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         test_bit_size += XC_PAGE_SIZE - (test_bit_size % XC_PAGE_SIZE);
     }
     cache_size = size + address_offset;
-    if (cache_size % MCACHE_BUCKET_SIZE) {
-        cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+    if (cache_size % mc->bucket_size) {
+        cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -609,8 +621,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
-    address_index  = new_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = new_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = new_phys_addr >> mc->bucket_shift;
+    address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 15:48:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:48:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723474.1128351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL6-0008Uw-BB; Thu, 16 May 2024 15:48:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723474.1128351; Thu, 16 May 2024 15:48:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL6-0008Up-72; Thu, 16 May 2024 15:48:16 +0000
Received: by outflank-mailman (input) for mailman id 723474;
 Thu, 16 May 2024 15:48:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7dL4-0008G5-Kp
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:48:14 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b3daf367-139b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 17:48:13 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-42016c8daa7so31427305e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:48:13 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-420193b0903sm135397305e9.47.2024.05.16.08.48.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:48:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3daf367-139b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715874492; x=1716479292; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=aho6bzCe+jDxNs4ys9vUT726OAdpjtZWNUfY80B3aVOIwdymqUwl2YyBOKYtA3Gh70
         SKtQzIib9/xxdPHaI1fKpD2STAPKLCPmbtA1N52qlyLQlb4R3htuz1HsbVOzQg8h1IDd
         wPglEJE7s6b7lIagoemGctAfZl6jQd6Mj9ScDgllSVk9vaCAT/vE3yEOWiZ3QrAwUReF
         DBvYhzwABOteiJQz0onRLbgs9Gc1JlwFj6T1Yj7BHn9VaYjNWck/sKqFaBMnnjrUu9vE
         gbFIst0ELd9NvA3nPNJXgC9xg08m2ohknXWReJ+AYHJ5Zy3m+vVnqKxQD9l9P5YHbaKo
         C+Xw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715874492; x=1716479292;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=GRsrSdnR84iZGK98B+bMxOdD617Bk5gB9erpadPna3HHrOpmFE68EjgqYQ3Sj6o5z1
         CddrzOs2hTwPFAusqcp1LIYFxSliBQfsP934FWllMgk5scSuKssl4LKg4sVQ1/n2iR0U
         yrEYwq5e5vol5RS4OjVQ0kYf1MS2UTH6uHTlSOmczk4ipb0jcIWKqWTM4YpRniVrwcPv
         Ziex6xe4XM+MLnBh0YrZV3vncV1kDsxTWhUcjDAaPszmsGuYQU1cGYm5gwroiR76XxI6
         4PpaFJQnCYNgEMK6ycFiHwqhl/6THa+hSZ+T74voDU4/LGBhmY3ftbSaRGT27SNgyb5X
         EaUw==
X-Forwarded-Encrypted: i=1; AJvYcCVxsFHXRuhZS219ix2H84yIBOuk2oJAT4SNX1ZTGas5vT93vuJxVRtiJ+LVoNh+JjjXYJmzqbWsGKX5atGUTfQLl3wM7TvKGCJSqnfHgcA=
X-Gm-Message-State: AOJu0Yy74WKXIGLFaT8hn2ntdCiLYAOxFNwUwbNbIQY9vW+cRGh19kF6
	4hdMRXKinhk2/LvfBrLpQukl1/2j2UvFA8mTJTho2gpIqu3sPXt8
X-Google-Smtp-Source: AGHT+IFjr0fT67BiVLzPhzx72CFQaxGN4vZsogPbdN+2UbY4Li7I/8td4kLMOJN0uGIq1S8SJxxrwg==
X-Received: by 2002:a05:600c:154e:b0:41f:e959:9053 with SMTP id 5b1f17b1804b1-41fead6ae10mr166768825e9.38.1715874492348;
        Thu, 16 May 2024 08:48:12 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v6 2/8] xen: mapcache: Unmap first entries in buckets
Date: Thu, 16 May 2024 17:47:58 +0200
Message-Id: <20240516154804.1114245-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
         return;
     }
     entry->lock--;
-    if (entry->lock > 0 || pentry == NULL) {
+    if (entry->lock > 0) {
         return;
     }
 
-    pentry->next = entry->next;
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
     if (munmap(entry->vaddr_base, entry->size) != 0) {
         perror("unmap fails");
         exit(-1);
     }
+
     g_free(entry->valid_mapping);
-    g_free(entry);
+    if (pentry) {
+        pentry->next = entry->next;
+        g_free(entry);
+    } else {
+        memset(entry, 0, sizeof *entry);
+    }
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 15:48:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:48:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723475.1128355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL6-00006c-If; Thu, 16 May 2024 15:48:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723475.1128355; Thu, 16 May 2024 15:48:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dL6-00005s-EP; Thu, 16 May 2024 15:48:16 +0000
Received: by outflank-mailman (input) for mailman id 723475;
 Thu, 16 May 2024 15:48:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7dL5-0008LQ-LJ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:48:15 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b4e7868a-139b-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 17:48:14 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e538a264e0so12460321fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:48:14 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42016b88f99sm146828625e9.10.2024.05.16.08.48.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:48:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4e7868a-139b-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715874494; x=1716479294; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YP28Iks2aCd2MOvUAbQc4yYuz/leCz2VtuKFu0fMfns=;
        b=Htw5IuoKJOBVAtAAeIs9v+QWS5XZlSFOmftRCQWV3aNYJeqn1UeW61GtqyzbEJ1HxC
         /BTI6n18XkvkFuhdcRuMoNEGISDGGcCXqHXJkdsQkP9fzofnBXFO0NJX5SCVsngAia/K
         Cp1PryhEnY5KvLUiqAg7FbE2Mcir2cVl21EXbYmXWAQhq6xOFoBo8vfxRuyhn23XDA6x
         Ve+gkgPbkcttAyRZgwDpros33L1VkqlAukeoqaWVnye34iECtjHOx2OSuEwF0quh4WtB
         GmJIsgwV9cfluutacU0f1OwF+e2FxMZ2rO4l9PR2yvBFnGFS5YLMYWSVuY2jbETA4ZNE
         RDoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715874494; x=1716479294;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YP28Iks2aCd2MOvUAbQc4yYuz/leCz2VtuKFu0fMfns=;
        b=G1MDIHW2H9rKhgk4S+6G6N3Qp9jR4pW85vDCwIFBLO6MeDMwslbdiyhP2SYi4xEZxs
         OpwhZcSvk7THAMPrC0WQvLfgU1A31bWjzgAO4tWQIkLyREx9SpQyNzUO39n7Q+WHSPeY
         SYOttzThvclZR9SVFHVPtzliQIvN8AII4GzQbss7XZzDru1zJ3aOxybQ+q20qNtJaUN6
         gQTi4wiuQkgTFip9V/qwk+EiKmJMPcVoOzp8LjnX9d/S+MRCe0DInW8Hhe81j2Qm3qAe
         ey8DUGR/vwHO1VTOKQ84Z+9Aszz0z1t2Elk6nV8zswL+L7MRHBFpMpaWXzy3VfLSoLrV
         7zQQ==
X-Forwarded-Encrypted: i=1; AJvYcCXE6GcneH3GWcpb43+KwUCxdk9wMjzC6HqjuKUUZ37Q+3RcIJXj8Afg2ms7hFYv1wHWT+1YxhxQrs3g2ciNoW2WxtqyMbtYqmfQJWcJaFw=
X-Gm-Message-State: AOJu0YyPp/gbQv/ZFyBZULiGEsYejpD8BOnS0Rzmfct01Ess9JUHiHYf
	jofSXWRAbPhmbZcCSiEzjKKQv10Ff8XoXhpZkLZP8cej1i4FT51z
X-Google-Smtp-Source: AGHT+IEaN/+VHmkJjv/znpWgW3h0+GqXzWUJj0uCRLf2DykoyCNb/asHTon7nVels7WBBHQIR1MU9A==
X-Received: by 2002:a2e:be9f:0:b0:2e3:ba0e:de12 with SMTP id 38308e7fff4ca-2e51ff5cf48mr209958761fa.22.1715874494287;
        Thu, 16 May 2024 08:48:14 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v6 3/8] xen: Add xen_mr_is_memory()
Date: Thu, 16 May 2024 17:47:59 +0200
Message-Id: <20240516154804.1114245-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
---
 hw/xen/xen-hvm-common.c | 10 ++++++++--
 include/sysemu/xen.h    |  8 ++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    return mr == &xen_memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
                    Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
         return;
     }
 
-    if (mr == &xen_memory) {
+    if (xen_mr_is_memory(mr)) {
         return;
     }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
     XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-    if (section->mr == &xen_memory) {
+    if (xen_mr_is_memory(section->mr)) {
         return;
     } else {
         if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
     g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 15:48:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:48:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723477.1128380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dLE-0000tX-6x; Thu, 16 May 2024 15:48:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723477.1128380; Thu, 16 May 2024 15:48:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dLE-0000tN-30; Thu, 16 May 2024 15:48:24 +0000
Received: by outflank-mailman (input) for mailman id 723477;
 Thu, 16 May 2024 15:48:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DxoZ=MT=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1s7dLC-0008G5-Kd
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:48:22 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b8673f45-139b-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 17:48:20 +0200 (CEST)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-4202c1d19d5so5298845e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:48:20 -0700 (PDT)
Received: from gmail.com ([213.0.35.158]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4202d81104esm15573575e9.16.2024.05.16.08.48.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:48:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8673f45-139b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715874500; x=1716479300; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SZNZmEv4DlBVnZCXhmwJDrvpI/tlo6jDsglpwLe0/DI=;
        b=ZnnV3iJ2fi/HahhdF6PipRHwVkQ5gFWllroZvH9Z5YSqxbUQHt8quz7unSLJhoUL6C
         MV0cLmY0WQPf/pgY9JHl5Vdxkbmt9MrFqV1HGFnehjY6z2rXtgKpyydPxWLo1hZ6pykp
         lYbBNnZZB9vGEXjY0PFd2ichswIq9FU6KgesUviOvosLLPucC1ypkdxcAlOvTTjZ+Exo
         8oJbFYuosp5rJco8BzdCCmlQhajnig16RhXs1ZBC1vv84WQHIkI/W6JPhClj7+5Eq+QI
         jx+op81NzJDcvP9NW9ZV15omd12IjTadhykDeo7F2Lkt0KkuJgs9+XwCzqlqEEipb9Lg
         PZAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715874500; x=1716479300;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SZNZmEv4DlBVnZCXhmwJDrvpI/tlo6jDsglpwLe0/DI=;
        b=BVtQtlao7dQSAaHiBYJMBZAnAiZ4VMODpvVODxMvtSbh2KX5PQjM0wz75aHouHHPjY
         eBKgsZ5dM/mjNs2/ve3C0A01yWg5dJmcniQOEgIXbtjRX7VE8+1yLmBMwZPUOAEjzaLJ
         uNycQD8OqOv2shXzZhmFcLmvml5xlOhIuW+La8bAoD2qTLULeaU2qLjLBkdvCKFQiJlv
         VYilC+MtJsuSDCDc6jRd4s+uSlTTnfW7tbNPMhdy60X+0Jb+V+UqUWDJmkHL+X12ULU1
         pCYO5SJav3alhus1h7YOQAkXV8KMsEZgTqM8NNVz558BYeH3tdfoCmpbypM19SqGlOYE
         zCYg==
X-Forwarded-Encrypted: i=1; AJvYcCXchq4XgbkvNdhPAuTrSJ5N2Nnht7LWJ3cYw2KZtHUiioj7YwjqwkXrQV7+esWqJr8B5LbN9j/b9Mub6HrSeJZfp4jrLi5/qXceQdhU3bY=
X-Gm-Message-State: AOJu0Yxi2F1prA+K6uV2EBE9ss04g1r+qSi3zfMtwUc804TDXuE4KH62
	xMbgQuT5n13MMAT4xhZxE7hdi0PqgAI1pKwu/FZOB+8dUmgVp508
X-Google-Smtp-Source: AGHT+IFlCJi5rUE9UtwNiDnxVECe0nVXb762LBypQ4XdGn83o5AP+HgeD6LDXCJWzztZIQsYhKzTvw==
X-Received: by 2002:a05:600c:4754:b0:41f:fca0:8c04 with SMTP id 5b1f17b1804b1-41ffca08dcemr132474315e9.11.1715874500186;
        Thu, 16 May 2024 08:48:20 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v6 7/8] xen: mapcache: Add support for grant mappings
Date: Thu, 16 May 2024 17:48:03 +0200
Message-Id: <20240516154804.1114245-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-hvm-common.c         |  12 ++-
 hw/xen/xen-mapcache.c           | 163 ++++++++++++++++++++++++++------
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h            |   7 ++
 4 files changed, 152 insertions(+), 33 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-    return mr == &xen_memory;
+    return mr == &xen_memory || mr == &xen_grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    return mr == &xen_grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..1cbc2aeaa9 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include <sys/resource.h>
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include <xenevtchn.h>
+#include <xengnttab.h>
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
     unsigned long *valid_mapping;
     uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
     uint8_t flags;
     hwaddr size;
     struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     unsigned long max_mcache_size;
     unsigned int bucket_shift;
 
+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
+    if (xen_region_gnttabdev == NULL) {
+        error_report("mapcache: Failed to open gnttab device");
+        exit(EXIT_FAILURE);
+    }
+
     if (HOST_LONG_BITS == 32) {
         bucket_shift = 16;
     } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     mapcache = xen_map_cache_init_single(f, opaque,
                                          bucket_shift,
                                          max_mcache_size);
+
+    /*
+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+     * map anything beyond the number of pages granted to us.
+     */
+    mapcache_grants = xen_map_cache_init_single(f, opaque,
+                                                XC_PAGE_SHIFT,
+                                                max_mcache_size);
+
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
                              hwaddr size,
                              hwaddr address_index,
                              bool dummy,
+                             bool grant,
+                             bool is_write,
                              ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
-    xen_pfn_t *pfns;
+    uint32_t *refs = NULL;
+    xen_pfn_t *pfns = NULL;
     int *err;
     unsigned int i;
     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
     trace_xen_remap_bucket(address_index);
 
-    pfns = g_new0(xen_pfn_t, nb_pfn);
+    if (grant) {
+        refs = g_new0(uint32_t, nb_pfn);
+    } else {
+        pfns = g_new0(xen_pfn_t, nb_pfn);
+    }
     err = g_new0(int, nb_pfn);
 
     if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
     g_free(entry->valid_mapping);
     entry->valid_mapping = NULL;
 
-    for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+    if (grant) {
+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+        for (i = 0; i < nb_pfn; i++) {
+            refs[i] = grant_base + i;
+        }
+    } else {
+        for (i = 0; i < nb_pfn; i++) {
+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+        }
     }
 
-    /*
-     * If the caller has requested the mapping at a specific address use
-     * MAP_FIXED to make sure it's honored.
-     */
+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
+
     if (!dummy) {
-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
-                                           PROT_READ | PROT_WRITE,
-                                           vaddr ? MAP_FIXED : 0,
-                                           nb_pfn, pfns, err);
+        if (grant) {
+            int prot = PROT_READ;
+
+            if (is_write) {
+                prot |= PROT_WRITE;
+            }
+
+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
+            assert(vaddr == NULL);
+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
+                                                         nb_pfn,
+                                                         xen_domid, refs,
+                                                         prot);
+        } else {
+            /*
+             * If the caller has requested the mapping at a specific address use
+             * MAP_FIXED to make sure it's honored.
+             *
+             * We don't yet support upgrading mappings from RO to RW, to handle
+             * models using ordinary address_space_rw(), foreign mappings ignore
+             * is_write and are always mapped RW.
+             */
+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
+                                               PROT_READ | PROT_WRITE,
+                                               vaddr ? MAP_FIXED : 0,
+                                               nb_pfn, pfns, err);
+        }
         if (vaddr_base == NULL) {
-            perror("xenforeignmemory_map2");
+            perror(grant ? "xengnttab_map_domain_grant_refs"
+                           : "xenforeignmemory_map2");
             exit(-1);
         }
     } else {
@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,
         }
     }
 
+    g_free(refs);
     g_free(pfns);
     g_free(err);
 }
@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
                                        ram_addr_t ram_offset,
-                                       uint8_t lock, bool dma, bool is_write)
+                                       uint8_t lock, bool dma,
+                                       bool grant, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
                   *free_entry = NULL, *free_pentry = NULL;
@@ -340,7 +400,7 @@ tryagain:
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                         ram_offset);
+                         grant, is_write, ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
@@ -348,7 +408,7 @@ tryagain:
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                             ram_offset);
+                             grant, is_write, ram_offset);
         }
     }
 
@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
+    bool grant = xen_mr_is_grants(mr);
+    MapCache *mc = grant ? mapcache_grants : mapcache;
     uint8_t *p;
 
-    mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
-                               lock, dma, is_write);
-    mapcache_unlock(mapcache);
+    if (grant) {
+        /*
+         * Grants are only supported via address_space_map(). Anything
+         * else is considered a user/guest error.
+         *
+         * QEMU generally doesn't expect these mappings to ever fail, so
+         * if this happens we report an error message and abort().
+         */
+        if (!lock) {
+            error_report("Trying access a grant reference without mapping it.");
+            abort();
+        }
+    }
+
+    mapcache_lock(mc);
+    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
+                               lock, dma, grant, is_write);
+    mapcache_unlock(mc);
     return p;
 }
 
@@ -449,7 +525,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 {
-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    ram_addr_t addr;
+
+    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    if (addr == RAM_ADDR_INVALID) {
+        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
+    }
+
+    return addr;
 }
 
 static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
@@ -460,6 +543,7 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     hwaddr paddr_index;
     hwaddr size;
     int found = 0;
+    int rc;
 
     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
         if (reventry->vaddr_req == buffer) {
@@ -502,7 +586,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     }
 
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
-    if (munmap(entry->vaddr_base, entry->size) != 0) {
+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
+        rc = xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
+                             entry->size >> mc->bucket_shift);
+    } else {
+        rc = munmap(entry->vaddr_base, entry->size);
+    }
+
+    if (rc) {
         perror("unmap fails");
         exit(-1);
     }
@@ -521,14 +612,24 @@ typedef struct XenMapCacheData {
     uint8_t *buffer;
 } XenMapCacheData;
 
+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
+{
+    mapcache_lock(mc);
+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
+    mapcache_unlock(mc);
+}
+
+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
+{
+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
+}
+
 static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
     XenMapCacheData *data = opaque;
 
-    mapcache_lock(mapcache);
-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
-    mapcache_unlock(mapcache);
-
+    xen_invalidate_map_cache_entry_all(data->buffer);
     aio_co_wake(data->co);
 }
 
@@ -543,9 +644,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
                                 xen_invalidate_map_cache_entry_bh, &data);
         qemu_coroutine_yield();
     } else {
-        mapcache_lock(mapcache);
-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
-        mapcache_unlock(mapcache);
+        xen_invalidate_map_cache_entry_all(buffer);
     }
 }
 
@@ -597,6 +696,7 @@ void xen_invalidate_map_cache(void)
     bdrv_drain_all();
 
     xen_invalidate_map_cache_single(mapcache);
+    xen_invalidate_map_cache_single(mapcache_grants);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
@@ -632,13 +732,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) == 0);
+
     address_index  = new_phys_addr >> mc->bucket_shift;
     address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false, old_phys_addr);
+                     cache_size, address_index, false,
+                     false, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 65a51aac2e..3d796235dc 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -16,6 +16,7 @@
 #include <xen/hvm/ioreq.h>
 
 extern MemoryRegion xen_memory;
+extern MemoryRegion xen_grants;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
 
@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
     do { } while (0)
 #endif
 
+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
+
 static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
 {
     return shared_page->vcpu_ioreq[i].vp_eport;
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index dc72f83bcb..19dccf4d71 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
 bool xen_mr_is_memory(MemoryRegion *mr);
+bool xen_mr_is_grants(MemoryRegion *mr);
 
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
     return false;
 }
 
+static inline bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 16 15:56:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:56:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723496.1128390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dTH-0004ST-VQ; Thu, 16 May 2024 15:56:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723496.1128390; Thu, 16 May 2024 15:56:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dTH-0004SM-Sn; Thu, 16 May 2024 15:56:43 +0000
Received: by outflank-mailman (input) for mailman id 723496;
 Thu, 16 May 2024 15:56:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7dTG-0004Rs-NR
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:56:42 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2ba055e-139c-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 17:56:41 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-43e0d564136so31906821cf.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:56:41 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e11c215adsm64911511cf.59.2024.05.16.08.56.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:56:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2ba055e-139c-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715875000; x=1716479800; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=H1Ld4sZ2GVD2soCDd3RxE7hNwjqhiWJsJNoQFAT72gk=;
        b=BFcaVUX0FzIyKxM8BMlr0S5tur99uazBIX0BKZ2lSOu3hmM7gxv380LUp+/FY+NjTR
         SIVIEfyZfaamDuW8pOzImU0AFR9Q9EbkTV5+pr7O7FRjW1F5ZUx4701zJ9GcewwbDQJ1
         NJEHt5CzNJPBinALAHarckIRGG0R3E8SaB2mY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875000; x=1716479800;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=H1Ld4sZ2GVD2soCDd3RxE7hNwjqhiWJsJNoQFAT72gk=;
        b=jdiAX2lxqRij8KEj+VmaQf5F6H7QKCNmsKl3ccxRlvbDYUerSava3FE1i9DwUZT8Y5
         XG6ypU5KBu/7TEc7hFu90Z+sAzdtL4RGaE9YUUhceLhUUSV9avfpI+znoAXprKItHEkD
         Of3nuB+RrC4Qy+2IWbPXNPLLXVi/nHCUPu+mjlrgKTi52X+5oyZ2HXqKZiC3GNlvV7yW
         jXbN+lzRVK2ax3S+WeUCN4zPEtw6TT2M9xphcTzDxImLMKrPcqnfCGVB6//TegwqT3Iv
         SCB5j9FR47OLxyegtyQiFosKaJnYVUgPMx60+SmzZ4onAd7uZ+Xw7ogWxRsAXcAAoR/i
         6XPg==
X-Forwarded-Encrypted: i=1; AJvYcCVuO8gq6MifvukO1OlOxgMW6Ge6/B90MDrYL2aZubOwMiauWdai4D9mB4sq4V6/tGtBO+I0V1pvKf2+zCztLkQI01iYQTwofcCA07Si/Jg=
X-Gm-Message-State: AOJu0Yy/7DZ9QzT0GT/VNpv6lVGajmjAHEbn8Mk6GWkwFuUIKzQ7g1cw
	dqAiNGrJHk4CFpxDK7X+eGsQ2Bx+8dvUGqCxEaLMOltcCofvhsDw0/f30Xu/hdSEjgUdjSjdLwh
	o
X-Google-Smtp-Source: AGHT+IE0Ti/uMqcgvVLOs8lI3Xll6tbDbt2WCYfYD0DK3JphTjOAcahuRH/qNKV631hZPCtV2hZ4iQ==
X-Received: by 2002:a05:622a:5592:b0:436:7bdb:d952 with SMTP id d75a77b69052e-43dfdac1fe2mr227129111cf.19.1715875000481;
        Thu, 16 May 2024 08:56:40 -0700 (PDT)
Date: Thu, 16 May 2024 17:56:38 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Message-ID: <ZkYstvGkdffWMn2z@macbook>
References: <20240516132224.86005-1-roger.pau@citrix.com>
 <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com>

On Thu, May 16, 2024 at 05:00:54PM +0200, Jan Beulich wrote:
> On 16.05.2024 15:22, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/irq.c
> > +++ b/xen/arch/x86/irq.c
> > @@ -2527,7 +2527,7 @@ static int __init cf_check setup_dump_irqs(void)
> >  }
> >  __initcall(setup_dump_irqs);
> >  
> > -/* Reset irq affinities to match the given CPU mask. */
> > +/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
> >  void fixup_irqs(const cpumask_t *mask, bool verbose)
> >  {
> 
> Evacuating is one purpose. Updating affinity, if need be, is another. I've
> been wondering more than once though whether it is actually correct /
> necessary for ->affinity to be updated by the function. As it stands you
> don't remove the respective code, though.

Yeah, I didn't want to get into updating ->affinity in this patch, so
decided to leave that as-is.

Note however that if we shuffle the interrupt around we should update
->affinity, so that the new destination is part of ->affinity?

Otherwise we could end up with the interrupt assigned to CPU(s) that
are not part of the ->affinity mask.  Maybe that's OK, TBH I'm not
sure I understand the purpose of the ->affinity mask, hence why I've
decided to leave it alone in this patch.

> 
> > @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
> >                  release_old_vec(desc);
> >          }
> >  
> > -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
> > +        /*
> > +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
> > +         * that's all covered by the requested affinity mask.
> > +         */
> > +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
> > +        if ( !desc->action || cpumask_subset(affinity, mask) )
> >          {
> >              spin_unlock(&desc->lock);
> >              continue;
> 
> First my understanding of how the two CPU sets are used: ->affinity is
> merely a representation of where the IRQ is permitted to be handled.
> ->arch.cpu_mask describes all CPUs where the assigned vector is valid
> (and would thus need cleaning up when a new vector is assigned). Neither
> of the two needs to be a strict subset of the other.

Oh, so it's allowed to have the interrupt target a CPU
(->arch.cpu_mask) that's not set in the affinity mask?

> 
> (It's not really clear whether ->arch.cpu_mask is [supposed to be] a
> subset of cpu_online_map.)

Not according to the description in arch_irq_desc:

        /*
         * Except for high priority interrupts @cpu_mask may have bits set for
         * offline CPUs.  Consumers need to be careful to mask this down to
         * online ones as necessary.  There is supposed to always be a non-
         * empty intersection with cpu_online_map.
         */

So ->arch.cpu_mask can (according to the comment) not strictly be a
subset of cpu_online_map.

Note this is only possible when using logical destination mode, so
removing that would turn the destination field into an unsigned int
that would point to a single CPU that must be present in
cpu_online_map.

> If that understanding of mine is correct, going from just ->arch.cpu_mask
> doesn't look quite right to me, as that may include CPUs not in ->affinity.
> As in: Things could be further limited, by also ANDing in ->affinity.

Hm, my expectation would be that ->arch.cpu_mask is a subset of
->affinity, but even if it's not, what we do care in fixup_cpus() is
what CPUs the interrupt targets, as we need to move the interrupt if
the target set is not in the input mask set.  I don't think ->affinity
should be taken into account for that decision, it should be done
based exclusively on which CPU(s) the interrupt target
(->arch.cpu_mask).

> At the same time your(?) and my variant suffer from cpumask_subset()
> possibly returning true despite the CPU the IRQ is presently being
> handled on being the one that we're in the process of bringing down.

No, I'm not sure I see how that can happen.  The CPU we are bringing
down will always be clear from the input CPU mask, and hence
cpumask_subset(->arch.cpu_mask, mask) will only return true if all the
set CPUs in ->arch.cpu_mask are also set in mask.  IOW: none of the
possible target destinations is a CPU to be removed.

> In
> that case we absolutely cannot skip the move. (I'd like to note that
> there are only two possible input values of "mask" for the function. The
> case I think you've been looking into is when it's &cpu_online_map.

Well, it's cpu_online_map which already has the CPU to be offlined
cleared.  See the call to cpumask_clear_cpu() ahead of calling
fixup_irqs().

> In
> which case cpumask_subset() is going to always return true with your
> change in place, if I'm not mistaken. That seems to make your change
> questionable. Yet with that I guess I'm overlooking something.)

I might we wrong, but I think you are missing that the to be offlined
CPU has been removed from cpu_online_map by the time it gets passed
to fixup_irqs().

> Plus there remains the question of whether updating ->affinity can indeed
> be skipped in more cases than it is right now (prior to you patch), or
> whether up front we may want to get rid of that updating (in line, I think,
> with earlier changes we did elsewhere) altogether.

I'm not sure about ->affinity TBH, that's why I've opted to leave it
alone for the time being.  It doesn't seem like a very helpful field
right now.

I would expect it to be mostly static, and populated with the set of
CPUs that are closer to (NUMA-wise) the device the interrupt belongs
to, but I'm not seeing any of this.  It seems to be set arbitrarily to
the CPU that binds the interrupt.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:58:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:58:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723500.1128400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dUs-00050w-9Z; Thu, 16 May 2024 15:58:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723500.1128400; Thu, 16 May 2024 15:58:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dUs-00050p-6p; Thu, 16 May 2024 15:58:22 +0000
Received: by outflank-mailman (input) for mailman id 723500;
 Thu, 16 May 2024 15:58:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7dUq-00050d-VO
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:58:20 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1d573a35-139d-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 17:58:19 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a5a1054cf61so392829466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:58:19 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b1773esm988903366b.199.2024.05.16.08.58.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 08:58:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d573a35-139d-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715875099; x=1716479899; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to
         :cc:subject:date:message-id:reply-to;
        bh=WRq67dEpvaxgg3QFeQBkNlihBhDVXflo99nfG1nxZ9s=;
        b=ZmCCs8olgBFGT70LuQaNK7IzA9VqbOTJUw/oWgGLgXoCB4re2uiSocyGfXJIYyloRK
         dI214P1a1gBDBozH5C3xB0sLSJtdgbYNo41fukT9sWUF3C3QVEXyI/VKPFfRNngfFKOs
         uunrN19QhmV1Rq/JQwMgGWyojL+FPvMFZmfgs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875099; x=1716479899;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:from:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WRq67dEpvaxgg3QFeQBkNlihBhDVXflo99nfG1nxZ9s=;
        b=ou9QoHIumOma8Pege8W08ub9cI+N2CxU7VxVT8PQNRjMIsj6p86dxU79wf76qJB29M
         seEQC1OCYRbstdsq12G1X5Ig2LPIyd1N7N+WUodR5tKV18j6MSecDGUntbRKlEWGRZZl
         uSAsV+7wSb0N7p8wjF9M12KqlAhRVBNkkWcwHD5555OW/yPhpNNIgCqlvU99Z6RX6UFZ
         RXZ0Nisanv8GhK8NjxhJCydKfwkUX5PuyC43PAzY9uHMWW9IX1cuXW9jSORTHhMAXKvQ
         Qm+ELxa9KmccVAFCfCbd9zYo3zzQt50TBOLRxvHGhmLBeUwzSGvdmfO3QlwlHbh3Erws
         +FPA==
X-Gm-Message-State: AOJu0YxgerqdleeD2BY+n1E5jLfEYTTz9BDi9Yqrl1HZUI/8f/oWweLF
	f50HveiaVC1CKqxyAbptzXM7XU0v/HtT0kciD3fB9/awPwycgGvpRT97mbl+e58=
X-Google-Smtp-Source: AGHT+IEalo+iCyDxRsMRkR66EVlLNoNI56gNz5b0vi9JhiIsk2Zfcn0NlJlO1OdtYlx/gbgygi4+rg==
X-Received: by 2002:a17:906:f6ca:b0:a59:c0a6:25cd with SMTP id a640c23a62f3a-a5a2d53ae0fmr1815506166b.5.1715875098612;
        Thu, 16 May 2024 08:58:18 -0700 (PDT)
Message-ID: <8b7e9a18-c6e7-4482-9e5a-cdf06b67ea71@cloud.com>
Date: Thu, 16 May 2024 16:58:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] tools/xg: Clean up xend-style overrides for CPU
 policies
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: anthony@xenproject.org
Cc: Xen-devel <xen-devel@lists.xenproject.org>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
 <20240207173957.19811-4-alejandro.vallejo@cloud.com>
 <2b4b05d1-154d-43b8-a29b-aa99332a8eeb@perard>
 <dc37751e-cf68-42af-909a-ad9a9ec3833f@cloud.com>
Content-Language: en-GB
In-Reply-To: <dc37751e-cf68-42af-909a-ad9a9ec3833f@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16/05/2024 16:37, Alejandro Vallejo wrote:
> On 30/04/2024 15:42, Anthony PERARD wrote:
>> On Wed, Feb 07, 2024 at 05:39:57PM +0000, Alejandro Vallejo wrote:
>>> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
>>> index 5699a26b946e..cee0be80ba5b 100644
>>> --- a/tools/libs/guest/xg_cpuid_x86.c
>>> +++ b/tools/libs/guest/xg_cpuid_x86.c
>>> @@ -772,49 +616,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>>               * apic_id_size values greater than 7.  Limit the value to
>>>               * 7 for now.
>>>               */
>>> -            if ( p->policy.extd.nc < 0x7f )
>>> +            if ( cur->policy.extd.nc < 0x7f )
>>>              {
>>> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
>>> -                    p->policy.extd.apic_id_size++;
>>> +                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )
>>> +                    cur->policy.extd.apic_id_size++;
>>>  
>>> -                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
>>> +                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
>>>              }
>>>              break;
>>>          }
>>>      }
>>>  
>>> -    nr_leaves = ARRAY_SIZE(p->leaves);
>>> -    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
>>> -    if ( rc )
>>> +    if ( xend || msr )
>>>      {
>>> -        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
>>> -        goto out;
>>> +        // The overrides are over the serialised form of the policy
>>
>> Comments should use /* */
> 
> Ugh, yes.
> 
>>
>>> +        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
>>> +            goto out;
>>> +
>>> +        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
>>> +            goto out;
>>> +        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
>>> +            goto out;
>>
>> What if `xend` is set, but `msr` isn't? Looks like there's going to be a
>> segv in xc_msr_policy() because it doesn't check that `msr` is actually
>> set.
>>
>>
>> Thanks,
>>
> 
> OOPS! Yes, msrs was meant to have the same check I added for
> xc_cpuid_xend_policy. Will do.
> 
> Cheers,
> Alejandro

Ugh answered to the old email. pinging to the xenproject one now.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 16 15:58:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 15:58:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723501.1128409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dUu-0005G6-Kd; Thu, 16 May 2024 15:58:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723501.1128409; Thu, 16 May 2024 15:58:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dUu-0005Fz-Hn; Thu, 16 May 2024 15:58:24 +0000
Received: by outflank-mailman (input) for mailman id 723501;
 Thu, 16 May 2024 15:58:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7dUt-0005Bz-AS
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 15:58:23 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e6b1b5e-139d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 17:58:21 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51f74fa2a82so1242533e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 08:58:21 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f39d2b8esm3036181e87.247.2024.05.16.08.58.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 08:58:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e6b1b5e-139d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715875101; x=1716479901; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=vSBBP6nHUw5/EgYTvPXK355r/xid6w2q4FGOgBFquYA=;
        b=D7Ntz+FR1ou+dAUaAmCXpJHtRZRUZt9yU838wtjIu81w4YxiTHa4tfCc9iEl7auWPc
         TVFxw1kpkJWzA81xKsx6hwVxGBHPmsETRAsdodCyAIgEiy+MaWnkU/gh8RzWVEY8oxqM
         mpbG2CmvSThRBBsOL1WNtYiPikEkY1BHtUfQOg9usQ+dbA7jTKKulFn+MRiF8wzWLNhS
         z9RZXhPXww6PEZvRVaPHx7OUeRNAmjTXII9+5waSepJN3zvXzbs7QPN5U9R+faD7d8i3
         C2DmEe0srKQvdzESfepbScYZyCS1ZhbduGAlftPW1nlDkyke1v6CBis5E+ej7X0PVV7J
         wsvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875101; x=1716479901;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vSBBP6nHUw5/EgYTvPXK355r/xid6w2q4FGOgBFquYA=;
        b=Y4VvfqxRZYadxVL2Gsyi6Cjj/DNjvXVFTwMMANWwZac0boaLhdvWQDI94W7HxhuWsy
         bo9ymp2ffyWiCEHrA94pUalRChx5oW3MhgOfR0NMrlpo7wxJMsBsAzZFnFm+YD9bRx0K
         f0RpPVhBh0ocHFZ+F9jmG8YUzNPU5iO7lCLK2gagSZP9sw85ErpGL9X/RXXrQdVUXVkM
         ONZRPZtJuol20OJcvqMnS2dJsxgf3+pQ9Qlknx50rhm5emzxUcGhXVhYO3kF5WZDqY0j
         UOrmN6GtSfAX4hWLzhu5ZqwQ9AyBhTrAogavjWKciqPoLacayRrsTzZyJaRmEwuTdP5I
         RIQg==
X-Forwarded-Encrypted: i=1; AJvYcCVQC0H/MeSxpFXgyV0w16ALmaxDJD8sJ2jDZr+COXt1Sg9SlHRhOXc0cf6Ccx/s8e0td6l8ydrETQbwq4HMJlfR6YuFs+f1AFTubyMzlRQ=
X-Gm-Message-State: AOJu0YyIc+Rh4nkKDh7qMGGzeX5p437arkrMLH91rjKsHLMl9UgMeC7H
	BNF+G/yl9eS/zCu+flhJ1plnuI8BRklrM+nRxcsdBCwumePBiCVQ
X-Google-Smtp-Source: AGHT+IGfM2yyADC8EM6rYOxXvEdM1veVmRerjEDPwE6B9fCMi5vdWm6DdHQZX+WI5aErhPqeZp93Pg==
X-Received: by 2002:ac2:4108:0:b0:51d:1d42:3eef with SMTP id 2adb3069b0e04-5220fd7b8b9mr16766553e87.29.1715875100564;
        Thu, 16 May 2024 08:58:20 -0700 (PDT)
Message-ID: <7f8ef88d0a8bd3e40b213d19dfc80cd3fd2db298.camel@gmail.com>
Subject: Re: [XEN PATCH 0/4] address violations of MISRA C Rule 20.7
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com, 
	ayan.kumar.halder@amd.com, consulting@bugseng.com, Andrew Cooper
	 <andrew.cooper3@citrix.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>, Nicola Vetrini <nicola.vetrini@bugseng.com>, 
	xen-devel@lists.xenproject.org
Date: Thu, 16 May 2024 17:58:19 +0200
In-Reply-To: <c67c1b8b-e14b-4c30-a381-1b89aedcddb9@suse.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
	 <c67c1b8b-e14b-4c30-a381-1b89aedcddb9@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 09:48 +0200, Jan Beulich wrote:
> Oleksii,
>=20
> On 15.05.2024 09:34, Nicola Vetrini wrote:
> > Hi all,
> >=20
> > this series aims to refactor some macros that cause violations of
> > MISRA C Rule
> > 20.7 ("Expressions resulting from the expansion of macro parameters
> > shall be
> > enclosed in parentheses"). All the macros touched by these patches
> > are in some
> > way involved in violations, and the strategy adopted to bring them
> > into
> > compliance is to add parentheses around macro arguments where
> > needed.
> >=20
> > Nicola Vetrini (4):
> > =C2=A0 x86/vpmu: address violations of MISRA C Rule 20.7
> > =C2=A0 x86/hvm: address violations of MISRA C Rule 20.7
> > =C2=A0 x86_64/uaccess: address violations of MISRA C Rule 20.7
> > =C2=A0 x86_64/cpu_idle: address violations of MISRA C Rule 20.7
>=20
> for 4.18 we took a relaxed approach towards (simple) changes for
> Misra purposes.
> I wonder whether you mean to permit the same for 4.19, or whether
> series like
> this one rather want/need delaying until after branching.
Lets follow the same approach for 4.19.

Sorry for delayed answer.

~ Oleksii

>=20
> Jan
>=20
> > =C2=A0xen/arch/x86/cpu/vpmu_amd.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 4 ++--
> > =C2=A0xen/arch/x86/hvm/mtrr.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 2 +-
> > =C2=A0xen/arch/x86/hvm/rtc.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | =
2 +-
> > =C2=A0xen/arch/x86/include/asm/hvm/save.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 | 2 +-
> > =C2=A0xen/arch/x86/include/asm/x86_64/uaccess.h | 7 ++++---
> > =C2=A0xen/arch/x86/x86_64/cpu_idle.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 2 +-
> > =C2=A06 files changed, 10 insertions(+), 9 deletions(-)
> >=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu May 16 16:04:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:04:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723511.1128422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dal-00008o-8t; Thu, 16 May 2024 16:04:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723511.1128422; Thu, 16 May 2024 16:04:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dal-00008g-6O; Thu, 16 May 2024 16:04:27 +0000
Received: by outflank-mailman (input) for mailman id 723511;
 Thu, 16 May 2024 16:04:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7dak-00008X-4x
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:04:26 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f68018a5-139d-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 18:04:24 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56e56ee8d5cso4210440a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:04:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eaace761sm3547085a12.91.2024.05.16.09.04.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 09:04:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f68018a5-139d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715875463; x=1716480263; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DkCeUJQ8pzOvFfOP1vuE5+6yP6OumOQEifPIhE54p8A=;
        b=AWj0V8966vSPkDKOfxpC8i9YZrL/cR06zFBvYQ/srYKxJspjknUOOw6yQ8eBF54aSq
         44ZK0kFFQy4VQAHFI45LbZITZY/znQH618ak83kN7QKLLhMngHbC775Dyy7ZzLv5sQWw
         5jlnlvNlQ2+ex3ARbAraFPi/vWHt+y6tOOpQHmRPGjnePdECn7VxAPcK+rmUrKnNUkoB
         cFJ4iPN3lQt50QEzAfNYXr7c5CNRpB2+/7/UGTlBjsPbs5OTvmzxwP5cHWK62IluXWsp
         6O5Kjbs/d62jit1tLbCTH4r8aAsVunCVZcCN8R83nBLZ+JnOxFkKKQ/gRQxf5P2VBSSu
         zJjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875463; x=1716480263;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DkCeUJQ8pzOvFfOP1vuE5+6yP6OumOQEifPIhE54p8A=;
        b=fcSnYojY5cQrtLf/iCm1CAI22juwz6ZjrLwRvG6/XpYxGSc1Z/xnSBNBPjG8wt5q9M
         tTTREwgYe/gmn3H/6iNpLzx0FJhKADJmWxQ+t31ijXrJqm7oMMkJep4CoZcdl1QIqskz
         wkxduX8lo/ZJZ1Ghykmevi77AoWjSlImw1Na/+iu8jJIFtBJsmwE6zcz6+01zT3BJdC8
         g2xUMwzKVOroVOdh1fpiQdPBcF/qyvHuC7oimicUa/0eJquH+DAsWuScXCLqZHcEAXdm
         9vyhy+swvtmsOfTkeUibQWWAeZSQSI+h7C1ikTIW+AONeQOl2dIDBxJgl5/i4a64fYs7
         lR6w==
X-Forwarded-Encrypted: i=1; AJvYcCXR/dvLfwrlxIGNLq+QsulsZTOwT+hVM26IjPokZ0Hxyjiq+aHoELTIooZOUM1qd0u3qCnmSxoaj6v3lEHCZJ7y/NSWm/oWiPF4+nEc/d4=
X-Gm-Message-State: AOJu0Yz1cgTMHKBuHCsMF0WlYnc0IM8kp26hfDsvm/7svg4VbX3gbEt5
	C4GgrwRHfgerb9+A9gB/Bf/hbFee5YSFfMPL5+0/qGPkbcnopm7N65xwBpkd4K/n79efgqs6zIM
	=
X-Google-Smtp-Source: AGHT+IEBDONIco+vd/kiafnD6BJB2vUIQ1nLQHvT1Uls0XMboV5ZqtMLezi/M9wHeSjgJNQJ0AOBdA==
X-Received: by 2002:a50:9ecd:0:b0:56b:cfef:b2de with SMTP id 4fb4d7f45d1cf-5734d67ecb2mr13736134a12.26.1715875463455;
        Thu, 16 May 2024 09:04:23 -0700 (PDT)
Message-ID: <b77790c6-a5b5-40d9-9387-5101ace14a4c@suse.com>
Date: Thu, 16 May 2024 18:04:22 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240516132224.86005-1-roger.pau@citrix.com>
 <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com> <ZkYstvGkdffWMn2z@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkYstvGkdffWMn2z@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 17:56, Roger Pau Monné wrote:
> On Thu, May 16, 2024 at 05:00:54PM +0200, Jan Beulich wrote:
>> On 16.05.2024 15:22, Roger Pau Monne wrote:
>>> @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>>>                  release_old_vec(desc);
>>>          }
>>>  
>>> -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
>>> +        /*
>>> +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
>>> +         * that's all covered by the requested affinity mask.
>>> +         */
>>> +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
>>> +        if ( !desc->action || cpumask_subset(affinity, mask) )
>>>          {
>>>              spin_unlock(&desc->lock);
>>>              continue;
>>[...]
>> In
>> which case cpumask_subset() is going to always return true with your
>> change in place, if I'm not mistaken. That seems to make your change
>> questionable. Yet with that I guess I'm overlooking something.)
> 
> I might we wrong, but I think you are missing that the to be offlined
> CPU has been removed from cpu_online_map by the time it gets passed
> to fixup_irqs().

Just on this part (I'll need to take more time to reply to other parts):
No, I've specifically paid attention to that fact. Yet for this particular
observation of mine is doesn't matter. If mask == &cpu_online_map, then
no matter what is in cpu_online_map

        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);

will mask things down to a subset of cpu_online_map, and hence

        if ( !desc->action || cpumask_subset(affinity, mask) )

(effectively being

        if ( !desc->action || cpumask_subset(affinity, &cpu_online_map) )

) is nothing else than

        if ( !desc->action || true )

. Yet that doesn't feel quite right.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 16:06:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:06:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723515.1128434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dcL-0000l1-L0; Thu, 16 May 2024 16:06:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723515.1128434; Thu, 16 May 2024 16:06:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dcL-0000ku-GM; Thu, 16 May 2024 16:06:05 +0000
Received: by outflank-mailman (input) for mailman id 723515;
 Thu, 16 May 2024 16:06:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7dcK-0000kk-L0
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:06:04 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 316caaf5-139e-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 18:06:02 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-51f12ccff5eso1256187e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:06:02 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f39d2bb6sm3035910e87.249.2024.05.16.09.06.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 09:06:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 316caaf5-139e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715875562; x=1716480362; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=l9pJ8URzD8F8P9KTJjOC5G7YdujGU97mixlcEmZ8O5s=;
        b=DdUzhPIivJXEAZcbcJ565zWHrB6Tie9DbGPPLNifffsXRqKnY5XkwdjBWUNtgm+R7p
         2Kbdhk79RaF1IqIj6U5pzP+vr3x7W7DL2P0O8Hs5GMSdC1zPOp4RA8mzAS+p1WnGmtY1
         E8foDvCSinUuAT7V2Z+s8/bM5s12pnQx445aV/dvody+f1dh2HHeUi1DdkdzD9h23+3h
         eo59fFfYk4Zs6qh65x6cBTwvak9zSI1FTiovOHuoK2yXhMxSQuiLKLru8Jf7r3xsHjyK
         NXAa7OOwGgp0wxxEZMpDh8QcwNIetDhGEFDk4bunnf861DkuGfcqlfKZN1pOB5g00hqx
         gvtQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875562; x=1716480362;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=l9pJ8URzD8F8P9KTJjOC5G7YdujGU97mixlcEmZ8O5s=;
        b=XslHSKfPar0z3VwVoTXVzjrFg/USXaindlLFEdpbFCRLDdQu1h/VU2lt2mAbd2wxX/
         Xv5I0Q5JCsAJyQsSaCqq+VZqsz2Ju76XNlodNN3JnopEFd9hlD3+BRJqDfedWDvva+4s
         W5QAfYi4WESNCVlG0WUbP+ehBrGmO4YP68NyolbqSopW0vwEZKJzvsFKxkv5N7LKQ6BU
         tnTrKLjzcDYckZLKQD2kvom3wFobZRfNLVexBhtYkG3pw9uqJ6IMV+SaG3mMoKgifUYe
         IsUiR13I+fF+zb4bqrmAjhxsl4BGY06zD3rWZ+wdsQXckNb1BdVthy1oeqBBWsTj4IR6
         woHA==
X-Forwarded-Encrypted: i=1; AJvYcCVpDr8o4yWesnVXXQgOHcAQvPo2S6NNRq/UvM6TUO/sFKb10GW19x6q/aKrz2/qPSTUm/cSX5GJfhRC5slKZutQuN15eMf0tVLlt0VpA4c=
X-Gm-Message-State: AOJu0YzUw16vX+HFB1Ff311iLACYouR/4rJ0cMDlWa5jIqOSYdIE8eZp
	a6hZwSnD3HF5k5mLopltJLC+POkha5drgGB4O79Kod9oGr37Br0H
X-Google-Smtp-Source: AGHT+IFEtxYj6Zwn+npuIJRp9SGh+e1xPPr+S+TIoa2CN+pxTpwTlZ1qsapC1meDfsdvnG6gVynAwA==
X-Received: by 2002:a05:6512:398a:b0:520:b0fe:6548 with SMTP id 2adb3069b0e04-52210584babmr16202300e87.64.1715875562040;
        Thu, 16 May 2024 09:06:02 -0700 (PDT)
Message-ID: <36c070d3d22da67c77c0d4752d6fc74faf85fbe5.camel@gmail.com>
Subject: Re: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity
 masks
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Roger Pau Monne
	 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>
Date: Thu, 16 May 2024 18:06:01 +0200
In-Reply-To: <c897b41d-2fa5-4db6-b591-60e15b646d6d@citrix.com>
References: <20240515152925.77197-1-roger.pau@citrix.com>
	 <c897b41d-2fa5-4db6-b591-60e15b646d6d@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 16:30 +0100, Andrew Cooper wrote:
> On 15/05/2024 4:29 pm, Roger Pau Monne wrote:
> > Print the CPU affinity masks as numeric ranges instead of plain
> > hexadecimal
> > bitfields.
> >=20
> > Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>=20
> Ha - I was going to write exactly the same patch, but you beat me to
> it.
>=20
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Looks good to me for having in Xen 4.19 release.

Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 16 16:08:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:08:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723517.1128443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7deB-0001X0-U1; Thu, 16 May 2024 16:07:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723517.1128443; Thu, 16 May 2024 16:07:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7deB-0001Wt-R6; Thu, 16 May 2024 16:07:59 +0000
Received: by outflank-mailman (input) for mailman id 723517;
 Thu, 16 May 2024 16:07:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7deA-0001Wn-EF
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:07:58 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 75440604-139e-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 18:07:56 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e564cad1f6so10796761fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:07:56 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e6f834e5eesm5463791fa.131.2024.05.16.09.07.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 09:07:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75440604-139e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715875676; x=1716480476; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=W68Q7qb73jEOcW2eXjc2JWV11ah8c1Z6pjSN1W3UmTw=;
        b=gGcdRueuMqJVHnM9qC74Fxw45QBrSEjsB3IsIO80no2isQOXqhhfY0rcI70jleG8RZ
         zxS5kzB2gySDDocjNdTL7k4Hs8J+yPK1O7dSeL+Z8q/D7RDQ/XIBslnHN/LjYX8sb2qA
         Vd1Rc5njw0wy5teOdy+L3Pt7JiWi8/3pGKZNgQallOnfleMFnzMvOC08208tLRXuqvPt
         8Y9rWnx62fXGbtr8Q46+RynbcrNII1uBgWgn2ZCE2mw3+vRMPMoGfdPnBr08bKTbkoMZ
         19bkO8LO+WRM14A383QZB75uWdFTyjTqMVD4nugjVgeCRkWAcusSKqqFIMyBBKNB1Y4+
         YJtA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875676; x=1716480476;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=W68Q7qb73jEOcW2eXjc2JWV11ah8c1Z6pjSN1W3UmTw=;
        b=nhdsWX2lbZv5a+zQohaNjotswwJhcfnO3gytDbN7n5ruhAjRUMuij/4UMBgwOM0HIw
         m1zyV5n8U2UAVdlnVR4pMKAmt4ATbRlb/I38cMrMZu6hlKKX1Gel2EBBvQxBcetpYtoG
         wLGts/YF+EgO8IbHlqiLntmZyFazE9NVXo+7rq/SzcMr5GBsxQtOBRMvpwLy2+vqixF0
         2D/JoRdD2BKnNs0NKK37/UM7DTlB5BJVg3czbP+EJ6urm1NfeZKCShyXOu3L1oYknLiO
         3pIsSQ5jmIYFZEhKy4viyjrylSRvL3nukIEiZKT9qdnKYqoaKnYtdKFD2s+D4s2QL0i1
         3IXA==
X-Forwarded-Encrypted: i=1; AJvYcCUTQB8vQhjPU9bSMMkWuvDLhDkLI0PpnUdq872SHHrM2EhRZC0fzdNBKk7EQuG09Suh7ZNCiKg+T3VzbcxPb0NQDIpbwVz6hz8DIFiB9E8=
X-Gm-Message-State: AOJu0Ywt9vgULLO8YcSFzxr8K2KTKe2FxIN4EBaS2auCOPOlv+Ig3mBv
	Lxd57Zlid6uVFmXHaqsx9m5J6g860zfYKFMA3UYmWTCu74tLw3ek
X-Google-Smtp-Source: AGHT+IGtLjKN1a+5Kp/Vgxso5pZpz9M6Hgb4/VVOwbEQR7Yi7CDWKProidWD2KkSIlpUTBw0cmXMqg==
X-Received: by 2002:a2e:a555:0:b0:2e1:a8db:2bca with SMTP id 38308e7fff4ca-2e5205e2972mr154874781fa.48.1715875675776;
        Thu, 16 May 2024 09:07:55 -0700 (PDT)
Message-ID: <0bb2becc5b5d327f6b3b878b876da78034bcebd6.camel@gmail.com>
Subject: Re: [XEN PATCH v7 1/5] xen/vpci: Clear all vpci status of device
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>, 
	xen-devel@lists.xenproject.org, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>
Cc: Jiqian Chen <Jiqian.Chen@amd.com>, Jan Beulich <jbeulich@suse.com>, 
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George
 Dunlap <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Anthony PERARD
 <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, "Daniel P .
 Smith" <dpsmith@apertussolutions.com>, Huang Rui <Ray.Huang@amd.com>
Date: Thu, 16 May 2024 18:07:54 +0200
In-Reply-To: <4eab3820-d357-42cb-a367-a7a0966e7e3b@amd.com>
References: <20240419035340.608833-1-Jiqian.Chen@amd.com>
	 <20240419035340.608833-2-Jiqian.Chen@amd.com>
	 <4eab3820-d357-42cb-a367-a7a0966e7e3b@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 10:27 -0400, Stewart Hildebrand wrote:
> On 4/18/24 23:53, Jiqian Chen wrote:
> > When a device has been reset on dom0 side, the vpci on Xen
> > side won't get notification, so the cached state in vpci is
> > all out of date compare with the real device state.
> > To solve that problem, add a new hypercall to clear all vpci
> > device state. When the state of device is reset on dom0 side,
> > dom0 can call this hypercall to notify vpci.
> >=20
> > Signed-off-by: Huang Rui <ray.huang@amd.com>
> > Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> > Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>=20
> Could we consider this patch for 4.19? It's independent of the rest
> of
> this series, and it fixes a real issue observed on both Arm and x86.
> The
> Linux counterpart has already been merged in linux-next [0].
>=20
> [0]
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commi=
t/?h=3Dnext-20240515&id=3Db272722511d5e8ae580f01830687b8a6b2717f01

Sure! Fixes should be merged.

Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Thu May 16 16:08:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:08:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723518.1128452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dem-0001zI-5X; Thu, 16 May 2024 16:08:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723518.1128452; Thu, 16 May 2024 16:08:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dem-0001zB-35; Thu, 16 May 2024 16:08:36 +0000
Received: by outflank-mailman (input) for mailman id 723518;
 Thu, 16 May 2024 16:08:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LYhW=MT=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7dej-0001z3-UR
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:08:33 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8ac7477f-139e-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 18:08:32 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so330037266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:08:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a178a9d81sm991765466b.83.2024.05.16.09.08.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 09:08:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ac7477f-139e-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715875712; x=1716480512; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=u3GWIFoUQ8dNniA006Izn//1I6UsOs2gb+h/yuBgfR8=;
        b=E1yqt19h7i8rL/WS6ZbGpqNx+iWlIeklZIN4wbKv7GXBMYjSrlcjwUxuYIXuyf8O6F
         zr8FlZfQZpHvKQOEN3Qcm1WDkEa+fzS91HZBa+rMzhpxyoTPyq4uwonsdGkQJPQkEE8z
         teCrHCHZJDRPw6vkhmf3FfiCBebLwf+mPXasFSNkZbVlLY4KBPzusfgiLn5eeXTTgDVy
         A/cTKSZ1wdUIBgihc9MFhDIv/8t4BZQe37YmNPnov1bQ0B8e2ooQWwF0K0elyzJ9RDqj
         2afawqnXFh9y2Ci7C+4V8MSzUAM38bgGiyzwRSaerPhNO7w7uPYlDZ3yrwtJQxPZDnaD
         eUAA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715875712; x=1716480512;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=u3GWIFoUQ8dNniA006Izn//1I6UsOs2gb+h/yuBgfR8=;
        b=Z7mDasjkou3JePQ9dneQpBkZ6+D1ljzv+UivjrdbzH2Tm8CM6gCMYK+QEko8fWW71o
         IcipTswITckgU9/esD4b9IjoMXerhsg1dtcfm8QVTG3PHt075l5MR+2Ndg6stWMqvYh0
         GlIWnMogc5jH+N/NODDUG2ZihYOfcg4pVT3pNbVUbrL0Gn/mz1QmOkJdFrPiC4URcbet
         qPQnDpdjjdt+34DqgP/wFRXAhbxuOSNCbafuGzMa4oxMsLADmX7JYm6ypfq2R28LvR7U
         BMWggwF3piToGIX54GJPelOClXCteZROWRQZG/cHiQPXGv5j21ytKbelnk0oD0lpKPmZ
         9QjA==
X-Forwarded-Encrypted: i=1; AJvYcCUM2/bcgNYrHeoiEOezvv2OSXo4153ELk4yL2Hda5WvpkcNXguJrwxemGXII4EasendkDC9p1Ab7iZJHTV8bMYwvtxF/SaEAtHZ51u5hnU=
X-Gm-Message-State: AOJu0YyywTRlhOAtchSg7jYM6LTd69Dkvtsxh3euhJec/2e6nLGKiooJ
	7/XWnu4RwMYn8jhL9Xk8UK/yDxMXKNw7x+6oJmXZC8I6Wyzf/dlPgUPThbWHDQ==
X-Google-Smtp-Source: AGHT+IHbGaM09obCanNKoIlXey0+aLDlMv2Un3ZIfyDfJnmo3m7Lk0ezoiWMgxdW5kkS878eXBeFlw==
X-Received: by 2002:a17:907:7daa:b0:a5a:4705:ad36 with SMTP id a640c23a62f3a-a5a4705b758mr1302297766b.16.1715875712173;
        Thu, 16 May 2024 09:08:32 -0700 (PDT)
Message-ID: <6d19ef9b-ccd8-4ae0-9e9e-ff7f3d92b333@suse.com>
Date: Thu, 16 May 2024 18:08:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 0/4] address violations of MISRA C Rule 20.7
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>, xen-devel@lists.xenproject.org
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <c67c1b8b-e14b-4c30-a381-1b89aedcddb9@suse.com>
 <7f8ef88d0a8bd3e40b213d19dfc80cd3fd2db298.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7f8ef88d0a8bd3e40b213d19dfc80cd3fd2db298.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 17:58, Oleksii K. wrote:
> On Wed, 2024-05-15 at 09:48 +0200, Jan Beulich wrote:
>> On 15.05.2024 09:34, Nicola Vetrini wrote:
>>> this series aims to refactor some macros that cause violations of
>>> MISRA C Rule
>>> 20.7 ("Expressions resulting from the expansion of macro parameters
>>> shall be
>>> enclosed in parentheses"). All the macros touched by these patches
>>> are in some
>>> way involved in violations, and the strategy adopted to bring them
>>> into
>>> compliance is to add parentheses around macro arguments where
>>> needed.
>>>
>>> Nicola Vetrini (4):
>>>   x86/vpmu: address violations of MISRA C Rule 20.7
>>>   x86/hvm: address violations of MISRA C Rule 20.7
>>>   x86_64/uaccess: address violations of MISRA C Rule 20.7
>>>   x86_64/cpu_idle: address violations of MISRA C Rule 20.7
>>
>> for 4.18 we took a relaxed approach towards (simple) changes for
>> Misra purposes.
>> I wonder whether you mean to permit the same for 4.19, or whether
>> series like
>> this one rather want/need delaying until after branching.
> Lets follow the same approach for 4.19.

Well, okay. But if you don't say now until when this is okay, you'll
need to announce the "stop" very prominently later on, so no-one
misses it.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 16 16:18:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:18:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723528.1128462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7doA-00051g-51; Thu, 16 May 2024 16:18:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723528.1128462; Thu, 16 May 2024 16:18:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7doA-00051Z-2F; Thu, 16 May 2024 16:18:18 +0000
Received: by outflank-mailman (input) for mailman id 723528;
 Thu, 16 May 2024 16:18:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7do8-00051A-MG
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:18:16 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e63d4adc-139f-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 18:18:15 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51f174e316eso1188051e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:18:15 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d8edbsm3008735e87.239.2024.05.16.09.18.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 09:18:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e63d4adc-139f-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715876295; x=1716481095; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=3zHgXCJVqG6j26NuClzdMhWEUX+xEfpJpSOC/7Qz8Ow=;
        b=FkZhrFkNvYe9HQNcK0WDALM4RKX+pqsOKsP9Mql+T2/9LD9U5o9mTkJLG5aFwVRz/r
         nUubjYIXc8l/n4ImIwOfxZk3MErCoOsMWJpOveaqOgQbWy+EshZkbL9xsB1IdVpnqFl6
         ++ClUs6GAfTadp7szsWZ4dDFU4DoQcJZyXMsDjAM/La8o1Y1RhyR75EYvg9lUmkKtlwH
         n5jvT3LPcuATuUhB7olGVZRhnSqdcc1ICJrv7IzKmkjKmvatOq3D0C+dVz3PpQEEZBC7
         bM5KW7sz3hNACyLV1B0YoHvt0rVcivprdhFQwQWQABWBI6/Bh+cxvD/J8EW+zbBX2/KA
         YPKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715876295; x=1716481095;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3zHgXCJVqG6j26NuClzdMhWEUX+xEfpJpSOC/7Qz8Ow=;
        b=Uvaf0Y0asn5CPD44s4p8J7ZuXEir676JPQ8coxUX40S1e9HBhWAp6miLOlfh5W45q0
         RyhSvPTwwfSqHRCrgWS6HMR6yzbTAyceacOSZXd4Un69Sf6C8omJ29sJOvf3G1AnQYjC
         5hbkPQcwjEDpV00ATMXloc2jAU4Ve+dxSQZHF1AZHady1ylQvlPo7Viao4cvWPPZdiE6
         Ep9YKUQbEZt9y4rM1uK92600guE9z4sngXaYKkBCwwqySwaW34JTV7wzd1Mpt9rFNZWA
         AdtHYk4RnafWl2eyBam/DMex7V6Hu4O1u7U2O8kfWZ+vtHgi+Wr8MbmuFa8seAMEnn2K
         /jzw==
X-Gm-Message-State: AOJu0Yy23NxRU9RPqZSfxq6KG4lzXzYwZrjvVef2YVQ/YFo4tFMfWvVV
	fCXkXn1OHPnUVyo0ft86tfP8//ytJbTuNeswjiCs88ObkgGHdvyQ
X-Google-Smtp-Source: AGHT+IH5pl8FslVbh4Ll9+N5ni/SfuxT4XEbxtNZgPTgPtw1Bnyw1kMVtQkAMbyCGgpydvUk9wMXAg==
X-Received: by 2002:a05:6512:1046:b0:521:9f04:27c1 with SMTP id 2adb3069b0e04-5220ff74b99mr14344142e87.66.1715876294855;
        Thu, 16 May 2024 09:18:14 -0700 (PDT)
Message-ID: <603054a7234c5262f10df552dad1c0919dacaf90.camel@gmail.com>
Subject: Re: [PATCH for-4.19] tools/xentop: fix cpu% sort order
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Leigh Brown
 <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org
Date: Thu, 16 May 2024 18:18:14 +0200
In-Reply-To: <2f83c912-c6bf-4f02-9c64-abe539d25a61@citrix.com>
References: <20240514081344.4499-1-leigh@solinno.co.uk>
	 <195184dd-46e3-4bd0-b82b-cf41000e4db1@citrix.com>
	 <d1fbc48ee89da3f4fe45a96161f9502e@solinno.co.uk>
	 <2f83c912-c6bf-4f02-9c64-abe539d25a61@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

T24gVHVlLCAyMDI0LTA1LTE0IGF0IDE0OjUyICswMTAwLCBBbmRyZXcgQ29vcGVyIHdyb3RlOgo+
IE9uIDE0LzA1LzIwMjQgMTozNiBwbSwgTGVpZ2ggQnJvd24gd3JvdGU6Cj4gPiBIZWxsbywKPiA+
IAo+ID4gT24gMjAyNC0wNS0xNCAxMzowNywgQW5kcmV3IENvb3BlciB3cm90ZToKPiA+ID4gT24g
MTQvMDUvMjAyNCA5OjEzIGFtLCBMZWlnaCBCcm93biB3cm90ZToKPiA+ID4gPiBBbHRob3VnaCB1
c2luZyBpbnRlZ2VyIGNvbXBhcmlzb24gdG8gY29tcGFyZSBkb3VibGVzIGtpbmQgb2YKPiA+ID4g
PiB3b3JrcywgaXQncyBhbm5veWluZyB0byBzZWUgZG9tYWlucyBzbGlnaHRseSBvdXQgb2Ygb3Jk
ZXIgd2hlbgo+ID4gPiA+IHNvcnRpbmcgYnkgY3B1JS4KPiA+ID4gPiAKPiA+ID4gPiBBZGQgYSBj
b21wYXJlX2RibCgpIGZ1bmN0aW9uIGFuZCB1cGRhdGUgY29tcGFyZV9jcHVfcGN0KCkgdG8KPiA+
ID4gPiBjYWxsIGl0Lgo+ID4gPiA+IAo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IExlaWdoIEJyb3du
IDxsZWlnaEBzb2xpbm5vLmNvLnVrPgo+ID4gPiA+IC0tLQo+ID4gPiA+IMKgdG9vbHMveGVudG9w
L3hlbnRvcC5jIHwgMTMgKysrKysrKysrKysrLQo+ID4gPiA+IMKgMSBmaWxlIGNoYW5nZWQsIDEy
IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKPiA+ID4gPiAKPiA+ID4gPiBkaWZmIC0tZ2l0
IGEvdG9vbHMveGVudG9wL3hlbnRvcC5jIGIvdG9vbHMveGVudG9wL3hlbnRvcC5jCj4gPiA+ID4g
aW5kZXggNTQ1YmQ1ZTk2ZC4uOTkxOTljYWVjOSAxMDA2NDQKPiA+ID4gPiAtLS0gYS90b29scy94
ZW50b3AveGVudG9wLmMKPiA+ID4gPiArKysgYi90b29scy94ZW50b3AveGVudG9wLmMKPiA+ID4g
PiBAQCAtODUsNiArODUsNyBAQCBzdGF0aWMgdm9pZCBzZXRfZGVsYXkoY29uc3QgY2hhciAqdmFs
dWUpOwo+ID4gPiA+IMKgc3RhdGljIHZvaWQgc2V0X3Byb21wdChjb25zdCBjaGFyICpuZXdfcHJv
bXB0LCB2b2lkCj4gPiA+ID4gKCpmdW5jKShjb25zdAo+ID4gPiA+IGNoYXIgKikpOwo+ID4gPiA+
IMKgc3RhdGljIGludCBoYW5kbGVfa2V5KGludCk7Cj4gPiA+ID4gwqBzdGF0aWMgaW50IGNvbXBh
cmUodW5zaWduZWQgbG9uZyBsb25nLCB1bnNpZ25lZCBsb25nIGxvbmcpOwo+ID4gPiA+ICtzdGF0
aWMgaW50IGNvbXBhcmVfZGJsKGRvdWJsZSwgZG91YmxlKTsKPiA+ID4gPiDCoHN0YXRpYyBpbnQg
Y29tcGFyZV9kb21haW5zKHhlbnN0YXRfZG9tYWluICoqLCB4ZW5zdGF0X2RvbWFpbgo+ID4gPiA+
ICoqKTsKPiA+ID4gPiDCoHN0YXRpYyB1bnNpZ25lZCBsb25nIGxvbmcgdG90X25ldF9ieXRlcygg
eGVuc3RhdF9kb21haW4gKiwKPiA+ID4gPiBpbnQpOwo+ID4gPiA+IMKgc3RhdGljIGJvb2wgdG90
X3ZiZF9yZXFzKHhlbnN0YXRfZG9tYWluICosIGludCwgdW5zaWduZWQgbG9uZwo+ID4gPiA+IGxv
bmcgKik7Cj4gPiA+ID4gQEAgLTQyMiw2ICs0MjMsMTYgQEAgc3RhdGljIGludCBjb21wYXJlKHVu
c2lnbmVkIGxvbmcgbG9uZyBpMSwKPiA+ID4gPiB1bnNpZ25lZCBsb25nIGxvbmcgaTIpCj4gPiA+
ID4gwqDCoMKgwqAgcmV0dXJuIDA7Cj4gPiA+ID4gwqB9Cj4gPiA+ID4gCj4gPiA+ID4gKy8qIENv
bXBhcmVzIHR3byBkb3VibGUgcHJlY2lzaW9uIG51bWJlcnMsIHJldHVybmluZyAtMSwwLDEgZm9y
Cj4gPiA+ID4gPCw9LD4gKi8KPiA+ID4gPiArc3RhdGljIGludCBjb21wYXJlX2RibChkb3VibGUg
ZDEsIGRvdWJsZSBkMikKPiA+ID4gPiArewo+ID4gPiA+ICvCoMKgwqAgaWYoZDEgPCBkMikKPiA+
ID4gPiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIC0xOwo+ID4gPiA+ICvCoMKgwqAgaWYoZDEgPiBk
MikKPiA+ID4gPiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDE7Cj4gPiA+ID4gK8KgwqDCoCByZXR1
cm4gMDsKPiA+ID4gPiArfQo+ID4gPiA+ICsKPiA+ID4gPiDCoC8qIENvbXBhcmlzb24gZnVuY3Rp
b24gZm9yIHVzZSB3aXRoIHFzb3J0LsKgIENvbXBhcmVzIHR3bwo+ID4gPiA+IGRvbWFpbnMKPiA+
ID4gPiB1c2luZyB0aGUKPiA+ID4gPiDCoCAqIGN1cnJlbnQgc29ydCBmaWVsZC4gKi8KPiA+ID4g
PiDCoHN0YXRpYyBpbnQgY29tcGFyZV9kb21haW5zKHhlbnN0YXRfZG9tYWluICoqZG9tYWluMSwK
PiA+ID4gPiB4ZW5zdGF0X2RvbWFpbgo+ID4gPiA+ICoqZG9tYWluMikKPiA+ID4gPiBAQCAtNTIz
LDcgKzUzNCw3IEBAIHN0YXRpYyBkb3VibGUgZ2V0X2NwdV9wY3QoeGVuc3RhdF9kb21haW4KPiA+
ID4gPiAqZG9tYWluKQo+ID4gPiA+IAo+ID4gPiA+IMKgc3RhdGljIGludCBjb21wYXJlX2NwdV9w
Y3QoeGVuc3RhdF9kb21haW4gKmRvbWFpbjEsCj4gPiA+ID4geGVuc3RhdF9kb21haW4KPiA+ID4g
PiAqZG9tYWluMikKPiA+ID4gPiDCoHsKPiA+ID4gPiAtwqDCoMKgIHJldHVybiAtY29tcGFyZShn
ZXRfY3B1X3BjdChkb21haW4xKSwKPiA+ID4gPiBnZXRfY3B1X3BjdChkb21haW4yKSk7Cj4gPiA+
ID4gK8KgwqDCoCByZXR1cm4gLWNvbXBhcmVfZGJsKGdldF9jcHVfcGN0KGRvbWFpbjEpLAo+ID4g
PiA+IGdldF9jcHVfcGN0KGRvbWFpbjIpKTsKPiA+ID4gCj4gPiA+IE9oLCB3ZSB3ZXJlIGRvaW5n
IGFuIGltcGxpY2l0IGRvdWJsZS0+dW5zaWduZWQgbG9uZyBsb25nCj4gPiA+IGNvbnZlcnNpb24u
wqAKPiA+ID4gT3ZlciB0aGUgcmFuZ2UgMC4wIHRvIDEwMC4wLCB0aGF0IG91Z2h0IHRvIHdvcmsg
YXMgZXhwZWN0ZWQuwqAKPiA+ID4gV2hhdCBraW5kCj4gPiA+IG9mIG91dC1vZi1vcmRlciBhcmUg
eW91IHNlZWluZz8KPiA+IAo+ID4gV2l0aG91dCBwYXRjaDoKPiA+IAo+ID4geGVudG9wIC0gMTM6
Mjk6MDHCoMKgIFhlbiA0LjE4LjIKPiA+IDEzIGRvbWFpbnM6IDEgcnVubmluZywgMTIgYmxvY2tl
ZCwgMCBwYXVzZWQsIDAgY3Jhc2hlZCwgMCBkeWluZywgMAo+ID4gc2h1dGRvd24KPiA+IE1lbTog
NjcwMzA2NDBrIHRvdGFsLCAzMzA5NzgwMGsgdXNlZCwgMzM5MzI4NDBrIGZyZWXCoMKgwqAgQ1BV
czogMjQgQAo+ID4gMzY5M01Iego+ID4gwqDCoMKgwqDCoCBOQU1FwqAgU1RBVEXCoMKgIENQVShz
ZWMpIENQVSglKcKgwqDCoMKgIE1FTShrKSBNRU0oJSnCoCBNQVhNRU0oaykKPiA+IE1BWE1FTSgl
KQo+ID4gwqAgaWNlY3JlYW0gLS1iLS0twqDCoMKgwqDCoMKgIDI1OTfCoMKgwqAgNi42wqDCoMKg
IDQxOTQzNjjCoMKgwqAgNi4zwqDCoMKgCj4gPiA0MTk1MzI4wqDCoMKgwqDCoMKgIDYuMwo+ID4g
wqDCoMKgwqAgeGVuZGQgLS1iLS0twqDCoMKgwqDCoMKgIDQwMTbCoMKgwqAgNS40wqDCoMKgwqAg
NTI0MjY4wqDCoMKgIDAuOMKgwqDCoMKgCj4gPiA1MjUzMTLCoMKgwqDCoMKgwqAgMC44Cj4gPiDC
oCBEb21haW4tMCAtLS0tLXLCoMKgwqDCoMKgwqAgMTA1OcKgwqDCoCAxLjfCoMKgwqAgMTA0ODU3
NsKgwqDCoCAxLjbCoMKgwqAKPiA+IDEwNDg1NzbCoMKgwqDCoMKgwqAgMS42Cj4gPiDCoMKgwqDC
oMKgIG5lb24gLS1iLS0twqDCoMKgwqDCoMKgwqAgODI2wqDCoMKgIDEuMcKgwqDCoCAyMDk3MjE2
wqDCoMKgIDMuMcKgwqDCoAo+ID4gMjA5ODE3NsKgwqDCoMKgwqDCoCAzLjEKPiA+IMKgwqAgYmxl
bmRlciAtLWItLS3CoMKgwqDCoMKgwqDCoCAxMjHCoMKgwqAgMC4ywqDCoMKgIDEwNDg2NDDCoMKg
wqAgMS42wqDCoMKgCj4gPiAxMDQ5NjAwwqDCoMKgwqDCoMKgIDEuNgo+ID4gwqDCoMKgwqAgYnJl
YWQgLS1iLS0twqDCoMKgwqDCoMKgwqDCoCA2OcKgwqDCoCAwLjHCoMKgwqDCoCA1MjQzNTLCoMKg
wqAgMC44wqDCoMKgwqAKPiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+IMKgwqDCoMKgwqDC
oCBib2IgLS1iLS0twqDCoMKgwqDCoMKgwqAgNTAywqDCoMKgIDAuM8KgwqAgMTY3NzcyODTCoMKg
IDI1LjDCoMKgCj4gPiAxNjc3ODI0MMKgwqDCoMKgwqAgMjUuMAo+ID4gwqDCoMKgIGNoZWVzZSAt
LWItLS3CoMKgwqDCoMKgwqDCoCAyMjXCoMKgwqAgMC41wqDCoMKgIDEwNDgzODTCoMKgwqAgMS42
wqDCoMKgCj4gPiAxMDQ5NjAwwqDCoMKgwqDCoMKgIDEuNgo+ID4gwqDCoCBjYXNzaW5pIC0tYi0t
LcKgwqDCoMKgwqDCoMKgIDQ4OcKgwqDCoCAwLjTCoMKgwqAgMzE0NTc5MsKgwqDCoCA0LjfCoMKg
wqAKPiA+IDMxNDY3NTLCoMKgwqDCoMKgwqAgNC43Cj4gPiDCoCBjaGlja3BlYSAtLWItLS3CoMKg
wqDCoMKgwqDCoMKgIDY3wqDCoMKgIDAuMcKgwqDCoMKgIDUyNDM1MsKgwqDCoCAwLjjCoMKgwqDC
oAo+ID4gNTI1MzEywqDCoMKgwqDCoMKgIDAuOAo+ID4gwqDCoMKgIGxlbnRpbCAtLWItLS3CoMKg
wqDCoMKgwqDCoMKgIDY3wqDCoMKgIDAuMcKgwqDCoMKgIDI2MjIwOMKgwqDCoCAwLjTCoMKgwqDC
oAo+ID4gMjYzMTY4wqDCoMKgwqDCoMKgIDAuNAo+ID4gwqDCoCBmdXNpbGxpIC0tYi0tLcKgwqDC
oMKgwqDCoMKgIDE1OcKgwqDCoCAwLjLCoMKgwqDCoCA1MjQzNTLCoMKgwqAgMC44wqDCoMKgwqAK
PiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+IMKgwqDCoMKgIHBpenphIC0tYi0tLcKgwqDC
oMKgwqDCoMKgIDM1OcKgwqDCoCAwLjXCoMKgwqDCoCA1MjQzNTLCoMKgwqAgMC44wqDCoMKgwqAK
PiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+IAo+ID4gV2l0aCBwYXRjaDoKPiA+IAo+ID4g
eGVudG9wIC0gMTM6MzA6MTfCoMKgIFhlbiA0LjE4LjIKPiA+IDEzIGRvbWFpbnM6IDEgcnVubmlu
ZywgMTIgYmxvY2tlZCwgMCBwYXVzZWQsIDAgY3Jhc2hlZCwgMCBkeWluZywgMAo+ID4gc2h1dGRv
d24KPiA+IE1lbTogNjcwMzA2NDBrIHRvdGFsLCAzMzA5Nzc4OGsgdXNlZCwgMzM5MzI4NTJrIGZy
ZWXCoMKgwqAgQ1BVczogMjQgQAo+ID4gMzY5M01Iego+ID4gwqDCoMKgwqDCoCBOQU1FwqAgU1RB
VEXCoMKgIENQVShzZWMpIENQVSglKcKgwqDCoMKgIE1FTShrKSBNRU0oJSnCoCBNQVhNRU0oaykK
PiA+IE1BWE1FTSglKQo+ID4gwqDCoMKgwqAgeGVuZGQgLS1iLS0twqDCoMKgwqDCoMKgIDQwMjDC
oMKgwqAgNS43wqDCoMKgwqAgNTI0MjY4wqDCoMKgIDAuOMKgwqDCoMKgCj4gPiA1MjUzMTLCoMKg
wqDCoMKgwqAgMC44Cj4gPiDCoCBpY2VjcmVhbSAtLWItLS3CoMKgwqDCoMKgwqAgMjYwMMKgwqDC
oCAzLjjCoMKgwqAgNDE5NDM2OMKgwqDCoCA2LjPCoMKgwqAKPiA+IDQxOTUzMjjCoMKgwqDCoMKg
wqAgNi4zCj4gPiDCoCBEb21haW4tMCAtLS0tLXLCoMKgwqDCoMKgwqAgMTA2MMKgwqDCoCAxLjXC
oMKgwqAgMTA0ODU3NsKgwqDCoCAxLjbCoMKgwqAKPiA+IDEwNDg1NzbCoMKgwqDCoMKgwqAgMS42
Cj4gPiDCoMKgwqDCoMKgIG5lb24gLS1iLS0twqDCoMKgwqDCoMKgwqAgODI3wqDCoMKgIDEuMcKg
wqDCoCAyMDk3MjE2wqDCoMKgIDMuMcKgwqDCoAo+ID4gMjA5ODE3NsKgwqDCoMKgwqDCoCAzLjEK
PiA+IMKgwqDCoCBjaGVlc2UgLS1iLS0twqDCoMKgwqDCoMKgwqAgMjI1wqDCoMKgIDAuN8KgwqDC
oCAxMDQ4Mzg0wqDCoMKgIDEuNsKgwqDCoAo+ID4gMTA0OTYwMMKgwqDCoMKgwqDCoCAxLjYKPiA+
IMKgwqDCoMKgIHBpenphIC0tYi0tLcKgwqDCoMKgwqDCoMKgIDM1OcKgwqDCoCAwLjXCoMKgwqDC
oCA1MjQzNTLCoMKgwqAgMC44wqDCoMKgwqAKPiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+
IMKgwqAgY2Fzc2luaSAtLWItLS3CoMKgwqDCoMKgwqDCoCA0OTDCoMKgwqAgMC40wqDCoMKgIDMx
NDU3OTLCoMKgwqAgNC43wqDCoMKgCj4gPiAzMTQ2NzUywqDCoMKgwqDCoMKgIDQuNwo+ID4gwqDC
oCBmdXNpbGxpIC0tYi0tLcKgwqDCoMKgwqDCoMKgIDE1OcKgwqDCoCAwLjLCoMKgwqDCoCA1MjQz
NTLCoMKgwqAgMC44wqDCoMKgwqAKPiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+IMKgwqDC
oMKgwqDCoCBib2IgLS1iLS0twqDCoMKgwqDCoMKgwqAgNTAywqDCoMKgIDAuMsKgwqAgMTY3Nzcy
ODTCoMKgIDI1LjDCoMKgCj4gPiAxNjc3ODI0MMKgwqDCoMKgwqAgMjUuMAo+ID4gwqDCoCBibGVu
ZGVyIC0tYi0tLcKgwqDCoMKgwqDCoMKgIDEyMcKgwqDCoCAwLjLCoMKgwqAgMTA0ODY0MMKgwqDC
oCAxLjbCoMKgwqAKPiA+IDEwNDk2MDDCoMKgwqDCoMKgwqAgMS42Cj4gPiDCoMKgwqDCoCBicmVh
ZCAtLWItLS3CoMKgwqDCoMKgwqDCoMKgIDY5wqDCoMKgIDAuMcKgwqDCoMKgIDUyNDM1MsKgwqDC
oCAwLjjCoMKgwqDCoAo+ID4gNTI1MzEywqDCoMKgwqDCoMKgIDAuOAo+ID4gwqAgY2hpY2twZWEg
LS1iLS0twqDCoMKgwqDCoMKgwqDCoCA2N8KgwqDCoCAwLjHCoMKgwqDCoCA1MjQzNTLCoMKgwqAg
MC44wqDCoMKgwqAKPiA+IDUyNTMxMsKgwqDCoMKgwqDCoCAwLjgKPiA+IMKgwqDCoCBsZW50aWwg
LS1iLS0twqDCoMKgwqDCoMKgwqDCoCA2N8KgwqDCoCAwLjHCoMKgwqDCoCAyNjIyMDjCoMKgwqAg
MC40wqDCoMKgwqAKPiA+IDI2MzE2OMKgwqDCoMKgwqDCoCAwLjQKPiAKPiAKPiBBaCwgc28gaXQn
cyB0aGUgcm91bmRpbmcsIGFuZCBhIHN0cmFpZ2h0IGNhc3QgZGlzY2FyZHMgdGhlIGZyYWN0aW9u
YWwKPiBwYXJ0Lgo+IAo+IEkgdGhpbmsgeW91ciBwYXRjaCBpcyBmaW5lLCBhbHRob3VnaCBpdCBj
b3VsZCBkbyB3aXRoIGEgbWVudGlvbiBvZgo+IHdoeQo+IHRoaXMgZ29lcyB3cm9uZyBpbiB0aGUg
Y29tbWl0IG1lc3NhZ2UuwqAgSSdtIGhhcHB5IHRvIGFkanVzdCBvbgo+IGNvbW1pdC4KRmVlbCBm
cmVlIHRvIG1lcmdlIGl0IGFzIEkgYW0gY29uc2lkZXJpbmcgaXQgYXMgYnVnZml4OgogUmVsZWFz
ZS1hY2tlZC1ieTogT2xla3NpaSBLdXJvY2hrbyA8b2xla3NpaS5rdXJvY2hrb0BnbWFpbC5jb20+
Cgp+IE9sZWtzaWkKPiAKPiB+QW5kcmV3Cgo=



From xen-devel-bounces@lists.xenproject.org Thu May 16 16:24:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:24:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723531.1128473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dtf-0007cm-Ob; Thu, 16 May 2024 16:23:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723531.1128473; Thu, 16 May 2024 16:23:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dtf-0007cf-LS; Thu, 16 May 2024 16:23:59 +0000
Received: by outflank-mailman (input) for mailman id 723531;
 Thu, 16 May 2024 16:23:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h+4+=MT=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7dte-0007cW-DQ
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:23:58 +0000
Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com
 [2607:f8b0:4864:20::833])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b1986313-13a0-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 18:23:57 +0200 (CEST)
Received: by mail-qt1-x833.google.com with SMTP id
 d75a77b69052e-43e415ae282so2494881cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:23:57 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df54d866csm98141201cf.26.2024.05.16.09.23.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 09:23:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1986313-13a0-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715876636; x=1716481436; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=uy4zgb499BsINYGIPPQXmZMX26VnMsPc9ouE/cxZoHw=;
        b=h563zotVsxFm+RsvXA2QJrCkrTV14MJ6QRulxWumfwcOiMB+jNdX0XawFxZUCHwKj2
         G8TqRioOvRy37iDPl218vKMfDY9I9sDeZ27jtbuWG1F1a7mWUFUaIL2rWGN5C0X6g5bu
         84Ywi5X4j8KIoKwMu0Aa4CucmZtGzV9MGtDTI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715876636; x=1716481436;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=uy4zgb499BsINYGIPPQXmZMX26VnMsPc9ouE/cxZoHw=;
        b=KiyPgwqfcwbtuOxnlj4voIr1L5CAygVvyKe0VPOrmCXvFun8fDvlqAOxwQEGrtMIh0
         5zdbk4Vz8qSdOdImLOieR9iLIAEs2PaUb/Bl4XshCxIthRRnQGqzsiDczM7sC8WKTaHq
         evr8lHyOtKYXS2uFs6O19JWLsFk+/nfAOVq16FOHfKlDO1m5OTTuh0zU+RHckFensGSH
         5CQ2wMDZeNURP/fkqdWVTqWwJijxDdjkM7yJm67tX2hRFVHa1Byas5L0nvg859Z2n3HX
         I6PGSxopOUmtyh57fslHYyfoe23pVzbL5nv7RXW8bFLR2iIswIuVn5vORnCUe14vektl
         aZRg==
X-Forwarded-Encrypted: i=1; AJvYcCW5Dm6t6b+kqDvndet5gROCRYluHZQrh4S0IyKbvuF8MF7S2mYDvMJIAKqJCu23DGSz++7+sDx5z6YR5EpAFpf2U0tysigOeDPIT1GDcug=
X-Gm-Message-State: AOJu0YxRNuhpfWF5jS9eIqcxjJMyDnqjFPqoXMhvf+G4+WaRTx8mjV2I
	CJVz1erxQsYPAauiW+7lB7EexhpyLAb3x076iTUJ3iCqm8LlS50d4sNruyhkTsEYgzWvWcEsV8E
	H
X-Google-Smtp-Source: AGHT+IH44hfJDMCo7AaENTgWo/BQ/QhwUJT4IJP/V0BW6/yIKuD8H9VuEfw/SpYXEwS6GWz4IvBF1Q==
X-Received: by 2002:ac8:5fcd:0:b0:43b:a7c:3ee2 with SMTP id d75a77b69052e-43dfda8ceeamr260019421cf.11.1715876636074;
        Thu, 16 May 2024 09:23:56 -0700 (PDT)
Date: Thu, 16 May 2024 18:23:53 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Message-ID: <ZkYzGZ3tDg9dRbnF@macbook>
References: <20240516132224.86005-1-roger.pau@citrix.com>
 <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com>
 <ZkYstvGkdffWMn2z@macbook>
 <b77790c6-a5b5-40d9-9387-5101ace14a4c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b77790c6-a5b5-40d9-9387-5101ace14a4c@suse.com>

On Thu, May 16, 2024 at 06:04:22PM +0200, Jan Beulich wrote:
> On 16.05.2024 17:56, Roger Pau Monné wrote:
> > On Thu, May 16, 2024 at 05:00:54PM +0200, Jan Beulich wrote:
> >> On 16.05.2024 15:22, Roger Pau Monne wrote:
> >>> @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
> >>>                  release_old_vec(desc);
> >>>          }
> >>>  
> >>> -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
> >>> +        /*
> >>> +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
> >>> +         * that's all covered by the requested affinity mask.
> >>> +         */
> >>> +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
> >>> +        if ( !desc->action || cpumask_subset(affinity, mask) )
> >>>          {
> >>>              spin_unlock(&desc->lock);
> >>>              continue;
> >>[...]
> >> In
> >> which case cpumask_subset() is going to always return true with your
> >> change in place, if I'm not mistaken. That seems to make your change
> >> questionable. Yet with that I guess I'm overlooking something.)
> > 
> > I might we wrong, but I think you are missing that the to be offlined
> > CPU has been removed from cpu_online_map by the time it gets passed
> > to fixup_irqs().
> 
> Just on this part (I'll need to take more time to reply to other parts):
> No, I've specifically paid attention to that fact. Yet for this particular
> observation of mine is doesn't matter. If mask == &cpu_online_map, then
> no matter what is in cpu_online_map
> 
>         cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
> 
> will mask things down to a subset of cpu_online_map, and hence
> 
>         if ( !desc->action || cpumask_subset(affinity, mask) )
> 
> (effectively being
> 
>         if ( !desc->action || cpumask_subset(affinity, &cpu_online_map) )
> 
> ) is nothing else than
> 
>         if ( !desc->action || true )
> 
> . Yet that doesn't feel quite right.

Oh, I get it now.  Ideally we would use cpu_online_map with the to be
removed CPU set, but that's complicated in this context.

For the purposes here we might as well avoid the AND of
->arch.cpu_mask with cpu_online_map and just check:

if ( !desc->action || cpumask_subset(desc->arch.cpu_mask, mask) )

As even if ->arch.cpu_mask has non-online CPUs set aside from the to
be offlined CPU, it would just mean that we might be shuffling more
than strictly necessary.  Note this will only be an issue with cluster
mode, physical mode must always have a single online CPU set in
->arch.cpu_mask.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 16 16:29:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 16:29:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723536.1128482 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dyP-0008Vg-8j; Thu, 16 May 2024 16:28:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723536.1128482; Thu, 16 May 2024 16:28:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7dyP-0008VZ-5z; Thu, 16 May 2024 16:28:53 +0000
Received: by outflank-mailman (input) for mailman id 723536;
 Thu, 16 May 2024 16:28:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=O3ow=MT=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s7dyN-0008VR-JV
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 16:28:51 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6023bfdb-13a1-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 18:28:49 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-572a93890d1so3960404a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 09:28:49 -0700 (PDT)
Received: from [10.80.5.21] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea651asm10572126a12.11.2024.05.16.09.28.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 09:28:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6023bfdb-13a1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715876929; x=1716481729; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sYshh0QI0PrXPFS5i6kmqo4IRlBxVxLPl4wGtP83Wcc=;
        b=XIFb7pSiTrsz0eo5glvbv/rXLqEv2tQwMexNNv11hYxLGc8cOn6TfXW1r7uPcvsSVP
         9a/xSG/vple1TNaKLwOI9vszW4dMZVXHVcxYiN1DJ2lTpum8I/um3/K+mpO/7p6g82i3
         v/dm3OiPUFNIdvBn9JD4SI44Fzhy9rHJiXNuo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715876929; x=1716481729;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sYshh0QI0PrXPFS5i6kmqo4IRlBxVxLPl4wGtP83Wcc=;
        b=hNX9YfGlqzbIUBuL5fgOfPJXSN5uLKNhrDQ6Hv+Cvxt2M8ta4QAF0b663aA0cURpBz
         3MF5b4jtd72Gkekl7bFHBwVD0x8DGykEWC+P2BZthQaa3QfC7V1MHz+VLHr8Ln9BrTbU
         HfLdOMWVthRcHMH+9G7IV312dvlZ3XvQNB/qMRyXS3r2QwTUTY00lbRqA1T3jMfALudA
         pTkeLgN5x4pjqfj7lwEOIgVPeictHC/n+Mtq31MP5UavavHzWs063Wjhuq5CF7HhUSdN
         iw4Im3XTeSL6Na4DLF+1t4soU0Yc1981gAlxJOjj26HAiXaKtXcatArYrqc5JFp3j+d6
         hKuA==
X-Gm-Message-State: AOJu0YwxYlIrjzxyt0b37HtTpo8XVZqJ5QZkzyEz78qrxWKkUNAFzSov
	8I/OGVe/hLWQ38RzCUbXT1l7RTu2d5lb5g+hHZFRSI0GMEOTuWcKTWVX1vpkQAQ=
X-Google-Smtp-Source: AGHT+IFwtqFNdOmhuTbmewPDpOKdfsBFMHbGRKypLdnZt42ifFJqVgEK8ybhcLmPeLNzN2MNoRLjfw==
X-Received: by 2002:a50:c317:0:b0:575:c1a:4948 with SMTP id 4fb4d7f45d1cf-5750c1a5334mr1863831a12.4.1715876929193;
        Thu, 16 May 2024 09:28:49 -0700 (PDT)
Message-ID: <61966287-10a6-415c-9902-82f5e19e35f4@cloud.com>
Date: Thu, 16 May 2024 17:28:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] tools/xg: Move xc_cpu_policy_t to xenguest.h
To: Anthony PERARD <anthony@xenproject.org>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>,
 Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
 <20240207173957.19811-2-alejandro.vallejo@cloud.com>
 <33f7fbc5-6bbf-4f84-9d83-96cbb106d7de@perard>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <33f7fbc5-6bbf-4f84-9d83-96cbb106d7de@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30/04/2024 15:13, Anthony PERARD wrote:
> On Wed, Feb 07, 2024 at 05:39:55PM +0000, Alejandro Vallejo wrote:
>> diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
>> index e01f494b772a..4e9078fdee4d 100644
>> --- a/tools/include/xenguest.h
>> +++ b/tools/include/xenguest.h
>> @@ -784,7 +784,13 @@ xen_pfn_t *xc_map_m2p(xc_interface *xch,
>>                        unsigned long *mfn0);
>>  
>>  #if defined(__i386__) || defined(__x86_64__)
>> -typedef struct xc_cpu_policy xc_cpu_policy_t;
>> +#include <xen/lib/x86/cpu-policy.h>
> 
> I don't think it's a good idea to expose "cpu-policy.h" to "xenguest.h",
> and it's going to break the build of every tools outside of xen
> repository that are using xenguest.h. xenguest.h is installed on a
> system, but cpu-policy.h isn't.
> 
>> +typedef struct xc_cpu_policy {
>> +    struct cpu_policy policy;
>> +    xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
>> +    xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
>> +} xc_cpu_policy_t;
> 
> Would using an accessor be an option to access `leaves` and `msrs` for
> "xen-cpuid" tool? That would avoid the need to expose the "cpu-policy.h"
> headers.
> 
> With accessors, we might not need to expose xc_cpu_policy_serialise() to
> the world anymore, and it could be internal to libxenguest, probably, I
> haven't check.
> 
> Thanks,
> 

That would work out for xen-cpuid.

I'm on the fence in the general case because I think it'd be
advantageous to let clients (i.e: xl) manipulate directly the
deserialized form of the policy. That would allow flipping features on
or off a heck of lot easier. We could create per-feature accessors, but
then we'll end up with _a lot_ of them.

I guess we'll get there when we get there. The xen-cpuid case definitely
doesn't warrant it. I'll give this a go.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 16 17:13:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 17:13:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723546.1128492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7efd-0003T4-II; Thu, 16 May 2024 17:13:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723546.1128492; Thu, 16 May 2024 17:13:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7efd-0003Sx-FZ; Thu, 16 May 2024 17:13:33 +0000
Received: by outflank-mailman (input) for mailman id 723546;
 Thu, 16 May 2024 17:13:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7efc-0003Sr-5y
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 17:13:32 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9e7868fb-13a7-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 19:13:31 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e27277d2c1so14616691fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 10:13:31 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccee9335sm273119305e9.29.2024.05.16.10.13.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 10:13:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e7868fb-13a7-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715879611; x=1716484411; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=9YH3Talo0JKHHZ8DSNTWBh0/kHsJ5KxWgJ4aT1Vo8AU=;
        b=nrU4bPpDJJJR4MJj1X9TLgJ0RcDeevVyYpi6FgiIpvtvj5yFcNHSf12kWDC8En8Ynw
         hxdK2kzO2EBjYNjSZvrnKTQNtUrA0OB3btqA23FrEv3we1Lmj6xo8IIX/fDI9TyN+t79
         BW8W3LE5OIUwqxcG/Ixm2c1ebWIyzoK30t3y0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715879611; x=1716484411;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9YH3Talo0JKHHZ8DSNTWBh0/kHsJ5KxWgJ4aT1Vo8AU=;
        b=duZjD3it9nCarl2m+QOkuctX644mUuS6ngPx4L4QL8F0kOh7VyawCs0aoV+Y2QTAP/
         jC+D+qIb7mZk/GUeg8/4SYLd8ZTFbyWjGpa42oFtqZt3ziBOjzroG9mY2NzoaGmsyiQt
         62gyttoP+oac2lNEbZ6fZXwygJpFTc13/336tDIVmt2XSVuQSwLCAFxZqQv3MW3+vtbh
         mUZh2G95DQJ/ylu1xheCJX9tKFE7IoLtL9z0uZDkBLfBPlwfwxoDF6aJ70+8VY6xJLIp
         Jp8pbAHMwrePAWqasxXIKuPP5CgbjBm6DmnZqDTYGzTNRzds0P1orCfE3B+YN/EN1MJY
         FZXA==
X-Forwarded-Encrypted: i=1; AJvYcCUqe7BJL/z4lIqkYxDYEUFTd3Ke8FL5I8jr0dCFgW0mKCqmDB4pVKveiOk3SGC111SXSr4+B+RTX06NMddjkJdWg1wUabZjKFmW4HOZzH8=
X-Gm-Message-State: AOJu0YwJkhYlqSWzROLX5dh4LMT+II6CaFT2BvI9G4yuDOB454g3FC/Z
	KlIVCwYnghQFqRARds8DSURORm5LEKuCHiXyL/49Wain0rnEOuB2980q3WrU3hM=
X-Google-Smtp-Source: AGHT+IFUwIISMRFIOPH+Jc0/vqr6ceYlVmRJoaiSAaJFHYaKfB889HtTIq/ikAb8r7BBvKSruUeqEw==
X-Received: by 2002:a2e:a98a:0:b0:2e6:f769:5124 with SMTP id 38308e7fff4ca-2e6f769532emr47748271fa.39.1715879610595;
        Thu, 16 May 2024 10:13:30 -0700 (PDT)
Message-ID: <06844367-cc1d-42f7-b925-ba78b4532b11@citrix.com>
Date: Thu, 16 May 2024 18:13:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity
 masks
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240515152925.77197-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240515152925.77197-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15/05/2024 4:29 pm, Roger Pau Monne wrote:
> Print the CPU affinity masks as numeric ranges instead of plain hexadecimal
> bitfields.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>  xen/arch/x86/irq.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
> index 80ba8d9fe912..3b951d81bd6d 100644
> --- a/xen/arch/x86/irq.c
> +++ b/xen/arch/x86/irq.c
> @@ -1934,10 +1934,10 @@ void do_IRQ(struct cpu_user_regs *regs)
>                  if ( ~irq < nr_irqs && irq_desc_initialized(desc) )
>                  {
>                      spin_lock(&desc->lock);
> -                    printk("IRQ%d a=%04lx[%04lx,%04lx] v=%02x[%02x] t=%s s=%08x\n",
> -                           ~irq, *cpumask_bits(desc->affinity),
> -                           *cpumask_bits(desc->arch.cpu_mask),
> -                           *cpumask_bits(desc->arch.old_cpu_mask),
> +                    printk("IRQ%d a={%*pbl}[{%*pbl},{%*pbl}] v=%02x[%02x] t=%s s=%08x\n",

Looking at this more closely, there's still some information obfuscation
going on.

How about "... a={} o={} n={} v=..."

so affinity, old and new masks are all stated explicitly, instead of
having to remember what the square brackets mean, and in particular that
the masks are backwards?

Happy to adjust on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 16 17:46:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 17:46:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723556.1128502 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7fBd-0001Yg-5B; Thu, 16 May 2024 17:46:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723556.1128502; Thu, 16 May 2024 17:46:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7fBd-0001YZ-2V; Thu, 16 May 2024 17:46:37 +0000
Received: by outflank-mailman (input) for mailman id 723556;
 Thu, 16 May 2024 17:46:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7fBc-0001YP-LN; Thu, 16 May 2024 17:46:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7fBc-0000mb-Hk; Thu, 16 May 2024 17:46:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7fBb-0007G2-Pc; Thu, 16 May 2024 17:46:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7fBb-0006Eu-PE; Thu, 16 May 2024 17:46:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=GiDcpczqUKjxNsH3ksK7YHn4ViJJeDCskpG94KREuKA=; b=F9OayrR5jlQQNG+yZTUwO0DNqO
	5P+AjKcwBYDLG8EtP3O6vXDgtbxGETloaiLVdfJij2zXi0Nfdu5p4WEDYBCB3uxmfoHVHG0CYbCyX
	IEaFA23u621plJKKsWFr63a1e0ggLLKku+Dqa4ghAVp7w3PF9nBYY7o1ijY32ayKhQxo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186016-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186016: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ae7584f63678cd9adc1c2f3a1e813b67a6b24544
X-Osstest-Versions-That:
    xen=6628bacc81f35b41bca18948fe9f405da441906f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 17:46:35 +0000

flight 186016 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186016/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ae7584f63678cd9adc1c2f3a1e813b67a6b24544
baseline version:
 xen                  6628bacc81f35b41bca18948fe9f405da441906f

Last test of basis   186014  2024-05-16 09:00:25 Z    0 days
Testing same since   186016  2024-05-16 13:02:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Sergiy Kibrik <Sergiy_Kibrik@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6628bacc81..ae7584f636  ae7584f63678cd9adc1c2f3a1e813b67a6b24544 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 16 18:58:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 18:58:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723572.1128540 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIv-0005Wd-WC; Thu, 16 May 2024 18:58:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723572.1128540; Thu, 16 May 2024 18:58:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIv-0005V3-Oz; Thu, 16 May 2024 18:58:13 +0000
Received: by outflank-mailman (input) for mailman id 723572;
 Thu, 16 May 2024 18:58:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7gIt-00057x-QT
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 18:58:11 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3d75aad3-13b6-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 20:58:10 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso373774066b.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 11:58:10 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d9dsm1003072566b.132.2024.05.16.11.58.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 11:58:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d75aad3-13b6-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715885890; x=1716490690; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BcRpXVcdvZAYil4Irc1oaRVZc77Zx6hRkAA/T1NmAPk=;
        b=OqR99uHbYpRJqUdjk4Rv0b9dYEz4uiLCSlVq4J+V6HqrdtvZqz2vDTl2tXRj/n5ahi
         LRpLIVj9gTQowy/ubjFNov2eITWWCL+Aw2JyRslY+ibSLQRWAPK5G4jWzzP5Yg9P9ND4
         xcl8XC83tBdoPcwEgzRj+vG1PMH4J5DZvPkg4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715885890; x=1716490690;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BcRpXVcdvZAYil4Irc1oaRVZc77Zx6hRkAA/T1NmAPk=;
        b=PyE6MGkAQeBV0inKbQ/1qli6ZAhlCjZjNMdNElsius0a5dcQClD6APsZeGdC06ClEd
         J8I/T0Wm97dRWh4wIKAbkVWp41e64dvkX+CdrmjNX6Nr9BxMTTPOiu32Oj9dEOHpJeY1
         uMfwJ7c3Aq0GuFTOma7BwfVQnieL3v5CMbLLvsFUmLcgGHj8H0LX8Q10OgeKKfKTbFxZ
         nvv1XpCJZ5eXmq0yewtJ6tJPlL5kbQXo2/M2rvC42CwYLyyZpR6/Ykq/nj6liygpKilk
         CyFvcFhfV63fV2AZUJ1XcWRgYErcKdgdwsWBagDHEDIhkX4WlzaUqXYxrDqi9SN2hFLz
         vaIA==
X-Gm-Message-State: AOJu0YyY3notNXiM2y4ZVkj6h/ntWY3geq1MICF3ptQXv5IlxXn8JlG4
	U4totogfFUrnm+pxrR/WCYtRiedc+pGPdAuIcc7d3R7ZcTKAR8b2bpdG8voUScYJE3DQXoKecFB
	b
X-Google-Smtp-Source: AGHT+IHHDTG9ObchtcVV/jVsqnfEynMKD2ibsufesvRXQQvkctYIHq8TbMcia1ICMivvqlAwStUKYQ==
X-Received: by 2002:a17:906:ecb4:b0:a5a:2d30:b8c5 with SMTP id a640c23a62f3a-a5a2d5356f5mr1471090766b.16.1715885890249;
        Thu, 16 May 2024 11:58:10 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: [PATCH v2 3/4] tools/{c,o}xenstored: Don't link against libsystemd
Date: Thu, 16 May 2024 19:58:03 +0100
Message-Id: <20240516185804.3309725-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the local freestanding wrapper instead.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Török <edwin.torok@cloud.com>

v2:
 * Redo almost from scratch, using the freestanding wrapper instead.
---
 tools/ocaml/xenstored/Makefile        | 2 --
 tools/ocaml/xenstored/systemd_stubs.c | 2 +-
 tools/xenstored/Makefile              | 5 -----
 tools/xenstored/posix.c               | 4 ++--
 4 files changed, 3 insertions(+), 10 deletions(-)

diff --git a/tools/ocaml/xenstored/Makefile b/tools/ocaml/xenstored/Makefile
index e8aaecf2e630..a8b8bb64698e 100644
--- a/tools/ocaml/xenstored/Makefile
+++ b/tools/ocaml/xenstored/Makefile
@@ -4,8 +4,6 @@ include $(OCAML_TOPLEVEL)/common.make
 
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
-CFLAGS-$(CONFIG_SYSTEMD)  += $(SYSTEMD_CFLAGS)
-LDFLAGS-$(CONFIG_SYSTEMD) += $(SYSTEMD_LIBS)
 
 CFLAGS  += $(CFLAGS-y)
 CFLAGS  += $(APPEND_CFLAGS)
diff --git a/tools/ocaml/xenstored/systemd_stubs.c b/tools/ocaml/xenstored/systemd_stubs.c
index f4c875075abe..7dbbdd35bf30 100644
--- a/tools/ocaml/xenstored/systemd_stubs.c
+++ b/tools/ocaml/xenstored/systemd_stubs.c
@@ -25,7 +25,7 @@
 
 #if defined(HAVE_SYSTEMD)
 
-#include <systemd/sd-daemon.h>
+#include <xen-sd-notify.h>
 
 CAMLprim value ocaml_sd_notify_ready(value ignore)
 {
diff --git a/tools/xenstored/Makefile b/tools/xenstored/Makefile
index e0897ed1ba30..09adfe1d5064 100644
--- a/tools/xenstored/Makefile
+++ b/tools/xenstored/Makefile
@@ -9,11 +9,6 @@ xenstored: LDLIBS += $(LDLIBS_libxenctrl)
 xenstored: LDLIBS += -lrt
 xenstored: LDLIBS += $(SOCKET_LIBS)
 
-ifeq ($(CONFIG_SYSTEMD),y)
-$(XENSTORED_OBJS-y): CFLAGS += $(SYSTEMD_CFLAGS)
-xenstored: LDLIBS += $(SYSTEMD_LIBS)
-endif
-
 TARGETS := xenstored
 
 .PHONY: all
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index d88c82d972d7..6037d739d013 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -27,7 +27,7 @@
 #include <sys/socket.h>
 #include <sys/un.h>
 #if defined(HAVE_SYSTEMD)
-#include <systemd/sd-daemon.h>
+#include <xen-sd-notify.h>
 #endif
 #include <xen-tools/xenstore-common.h>
 
@@ -393,7 +393,7 @@ void late_init(bool live_update)
 #if defined(HAVE_SYSTEMD)
 	if (!live_update) {
 		sd_notify(1, "READY=1");
-		fprintf(stderr, SD_NOTICE "xenstored is ready\n");
+		fprintf(stderr, "xenstored is ready\n");
 	}
 #endif
 }
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 18:58:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 18:58:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723573.1128545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIw-0005fM-AT; Thu, 16 May 2024 18:58:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723573.1128545; Thu, 16 May 2024 18:58:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIw-0005e6-6B; Thu, 16 May 2024 18:58:14 +0000
Received: by outflank-mailman (input) for mailman id 723573;
 Thu, 16 May 2024 18:58:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7gIv-00050W-Ch
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 18:58:13 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3decdf70-13b6-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 20:58:11 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a599a298990so398021866b.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 11:58:11 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d9dsm1003072566b.132.2024.05.16.11.58.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 11:58:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3decdf70-13b6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715885891; x=1716490691; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9RPPdXWzS+h1KReJUUX36RCkolCXpVxyjNBaSjStXp4=;
        b=O1ac5C0XJC9tJOfKcgJDYNTduuEDcREbhs/ovnpoWPM46u+s51fE9Da4LuZlAiof3I
         HRG4u2XencSUCCOU/mZHLS8bhkqn2TTQ5dso0+20cP7WJ9XXIi7ebUC1DNR4GbFSCB3v
         foAYu3c36F8bKls78CJJgyLbQrjMS6vNKtGcU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715885891; x=1716490691;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9RPPdXWzS+h1KReJUUX36RCkolCXpVxyjNBaSjStXp4=;
        b=oivlzXgvwFM5t6WrGcMG3MOZzNnG178uBQba8VHRpYO9IX0c+AUCxwEExU/JzV0u1W
         cCk1h4SVY3s8qi0/IIZjsJHI6BlvLh0desj3LnM/YY7kDaLv77EofCT9yxIJ/ANNPeRB
         iGc9YcgzYRWA5D4dkleEXjlzpqjcq4LX899OGk2w/fAHpX4qCRkd84RDQSQxI3JpDiuG
         bw9nrSS+kIt0OoY86j8qk5QhYIwoXhDbaM8yhBjVTSRctouEpnPMelvlDw+CkfpgL5fi
         mP3E+1X9WENNOlINDO0yg4+NlyiyDrhuZmUN//2mSTMhnJo95sS6d+n7xgvYkvK/UbWk
         bncA==
X-Gm-Message-State: AOJu0YxigT5bE63CjPeW+XsHxLiDE71nWNJl1xAGX6pK51Hchm7Cru2S
	cG+tCwsNivJeW2s/OSAthVY8NyDd0HjfIDQXsepmNafmoBWpHMbarA9i7/ITUQGaKunyWzWsI/8
	8
X-Google-Smtp-Source: AGHT+IEaucKEm0iXV/PnaVzx9IBaFmUmggPKs+Sh4PcPdsCfdTNeiXwPZCN45OUBuEBa7qPU3ADYmg==
X-Received: by 2002:a17:907:1b0b:b0:a59:a3ef:21eb with SMTP id a640c23a62f3a-a5a2d675b19mr1635240566b.73.1715885891059;
        Thu, 16 May 2024 11:58:11 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: [PATCH v2 4/4] tools: Drop libsystemd as a dependency
Date: Thu, 16 May 2024 19:58:04 +0100
Message-Id: <20240516185804.3309725-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There are no more users, and we want to disuade people from introducing new
users just for sd_notify() and friends.  Drop the dependency.

We still want the overall --with{,out}-systemd to gate the generation of the
service/unit/mount/etc files.

Rerun autogen.sh, and mark the dependency as removed in the build containers.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Török <edwin.torok@cloud.com>

v2:
 * Only strip out the library check.
---
 automation/build/archlinux/current.dockerfile |   1 +
 .../build/suse/opensuse-leap.dockerfile       |   1 +
 .../build/suse/opensuse-tumbleweed.dockerfile |   1 +
 automation/build/ubuntu/focal.dockerfile      |   1 +
 config/Tools.mk.in                            |   2 -
 m4/systemd.m4                                 |   9 -
 tools/configure                               | 256 ------------------
 7 files changed, 4 insertions(+), 267 deletions(-)

diff --git a/automation/build/archlinux/current.dockerfile b/automation/build/archlinux/current.dockerfile
index 3e37ab5c40c1..d29f1358c2bd 100644
--- a/automation/build/archlinux/current.dockerfile
+++ b/automation/build/archlinux/current.dockerfile
@@ -37,6 +37,7 @@ RUN pacman -S --refresh --sysupgrade --noconfirm --noprogressbar --needed \
         sdl2 \
         spice \
         spice-protocol \
+        # systemd for Xen < 4.19
         systemd \
         transfig \
         usbredir \
diff --git a/automation/build/suse/opensuse-leap.dockerfile b/automation/build/suse/opensuse-leap.dockerfile
index 48d0d50d005d..3ef33458a07c 100644
--- a/automation/build/suse/opensuse-leap.dockerfile
+++ b/automation/build/suse/opensuse-leap.dockerfile
@@ -59,6 +59,7 @@ RUN zypper install -y --no-recommends \
         'pkgconfig(sdl2)' \
         python3-devel \
         python3-setuptools \
+        # systemd-devel for Xen < 4.19
         systemd-devel \
         tar \
         transfig \
diff --git a/automation/build/suse/opensuse-tumbleweed.dockerfile b/automation/build/suse/opensuse-tumbleweed.dockerfile
index 53542ba1f4d9..a793601c876d 100644
--- a/automation/build/suse/opensuse-tumbleweed.dockerfile
+++ b/automation/build/suse/opensuse-tumbleweed.dockerfile
@@ -60,6 +60,7 @@ RUN zypper install -y --no-recommends \
         'pkgconfig(sdl2)' \
         python3-devel \
         python3-setuptools \
+        # systemd-devel for Xen < 4.19
         systemd-devel \
         tar \
         transfig \
diff --git a/automation/build/ubuntu/focal.dockerfile b/automation/build/ubuntu/focal.dockerfile
index 7c6a4d07566b..078abd56d56f 100644
--- a/automation/build/ubuntu/focal.dockerfile
+++ b/automation/build/ubuntu/focal.dockerfile
@@ -34,6 +34,7 @@ RUN apt-get update && \
         libnl-3-dev \
         ocaml-nox \
         libfindlib-ocaml-dev \
+        # libsystemd-dev for Xen < 4.19
         libsystemd-dev \
         transfig \
         pandoc \
diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index b54ab21f966b..50fbef841f3f 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -52,8 +52,6 @@ CONFIG_PYGRUB       := @pygrub@
 CONFIG_LIBFSIMAGE   := @libfsimage@
 
 CONFIG_SYSTEMD      := @systemd@
-SYSTEMD_CFLAGS      := @SYSTEMD_CFLAGS@
-SYSTEMD_LIBS        := @SYSTEMD_LIBS@
 XEN_SYSTEMD_DIR     := @SYSTEMD_DIR@
 XEN_SYSTEMD_MODULES_LOAD := @SYSTEMD_MODULES_LOAD@
 CONFIG_9PFS         := @ninepfs@
diff --git a/m4/systemd.m4 b/m4/systemd.m4
index 112dc11b5e05..e4fe51a8ba5f 100644
--- a/m4/systemd.m4
+++ b/m4/systemd.m4
@@ -41,15 +41,6 @@ AC_DEFUN([AX_ALLOW_SYSTEMD_OPTS], [
 ])
 
 AC_DEFUN([AX_CHECK_SYSTEMD_LIBS], [
-	PKG_CHECK_MODULES([SYSTEMD], [libsystemd-daemon],,
-		[PKG_CHECK_MODULES([SYSTEMD], [libsystemd >= 209])]
-        )
-	dnl pkg-config older than 0.24 does not set these for
-	dnl PKG_CHECK_MODULES() worth also noting is that as of version 208
-	dnl of systemd pkg-config --cflags currently yields no extra flags yet.
-	AC_SUBST([SYSTEMD_CFLAGS])
-	AC_SUBST([SYSTEMD_LIBS])
-
 	AS_IF([test "x$SYSTEMD_DIR" = x], [
 	    dnl In order to use the line below we need to fix upstream systemd
 	    dnl to properly ${prefix} for child variables in
diff --git a/tools/configure b/tools/configure
index 3d557234b319..b8faa1d520c5 100755
--- a/tools/configure
+++ b/tools/configure
@@ -9766,262 +9766,6 @@ $as_echo "#define HAVE_SYSTEMD 1" >>confdefs.h
 
 			systemd=y
 
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd-daemon\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd-daemon" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd-daemon\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd-daemon" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd-daemon" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd-daemon" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-	as_fn_error $? "Package requirements (libsystemd >= 209) were not met:
-
-$SYSTEMD_PKG_ERRORS
-
-Consider adjusting the PKG_CONFIG_PATH environment variable if you
-installed software in a non-standard prefix.
-
-Alternatively, you may set the environment variables SYSTEMD_CFLAGS
-and SYSTEMD_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details." "$LINENO" 5
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-	{ { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
-as_fn_error $? "The pkg-config script could not be found or is too old.  Make sure it
-is in your PATH or set the PKG_CONFIG environment variable to the full
-path to pkg-config.
-
-Alternatively, you may set the environment variables SYSTEMD_CFLAGS
-and SYSTEMD_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details.
-
-To get pkg-config, see <http://pkg-config.freedesktop.org/>.
-See \`config.log' for more details" "$LINENO" 5; }
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
-
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-	as_fn_error $? "Package requirements (libsystemd >= 209) were not met:
-
-$SYSTEMD_PKG_ERRORS
-
-Consider adjusting the PKG_CONFIG_PATH environment variable if you
-installed software in a non-standard prefix.
-
-Alternatively, you may set the environment variables SYSTEMD_CFLAGS
-and SYSTEMD_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details." "$LINENO" 5
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-	{ { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
-as_fn_error $? "The pkg-config script could not be found or is too old.  Make sure it
-is in your PATH or set the PKG_CONFIG environment variable to the full
-path to pkg-config.
-
-Alternatively, you may set the environment variables SYSTEMD_CFLAGS
-and SYSTEMD_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details.
-
-To get pkg-config, see <http://pkg-config.freedesktop.org/>.
-See \`config.log' for more details" "$LINENO" 5; }
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
-
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
-
-
-
 	if test "x$SYSTEMD_DIR" = x; then :
 
 	    	    	    	    	    	    	    	    	    	    	    	    	    SYSTEMD_DIR="\$(prefix)/lib/systemd/system/"
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 18:58:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 18:58:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723570.1128523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIu-0005FD-FE; Thu, 16 May 2024 18:58:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723570.1128523; Thu, 16 May 2024 18:58:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIu-0005F6-BR; Thu, 16 May 2024 18:58:12 +0000
Received: by outflank-mailman (input) for mailman id 723570;
 Thu, 16 May 2024 18:58:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7gIs-00050W-OI
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 18:58:10 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3c78c0c0-13b6-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 20:58:09 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59e4136010so388950866b.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 11:58:09 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d9dsm1003072566b.132.2024.05.16.11.58.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 11:58:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c78c0c0-13b6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715885888; x=1716490688; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=C7RjB5NwR6WzQg0yeEc59qpmK1kkZdlkcceJ3D90NrA=;
        b=jFwUuVaf09yeB/Wxgxz6gjWQ4WjIW/8LCVvFirQOfKmNRj7AG52yeCGSxR8T/pJ5dE
         bpiX1Lm1l5kQKVVsi47S9YFpbHYC7uc9p9tOqXiAak4qOWDLFSE7Nq0bOG9T83PTiT9R
         hr/jyhy4Rq1NHrh9UlcbkhXs6F2om2d6TuyVw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715885888; x=1716490688;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=C7RjB5NwR6WzQg0yeEc59qpmK1kkZdlkcceJ3D90NrA=;
        b=CQBHXzas8b5fAoO6/gd0si8WjuRZhx8IV02WnCeuZhzj5p8ccGnBVfJLIP4ILT55yX
         eEi6Pi64o0dpi8EjsPAhQixnTVtXd9CC4dbCLZ1Bdirj9QNe/nQ7y/IQMKmrOjZl+vbc
         nGGUcUxa7HtYeeMhRz7n3ynoPcrqSqI9r8Oz+n27Q+SDpVGqZklYPs7PYxvbMZJShWG0
         E1yaXBkDVP51y8Ck5HfkZbFvq4wsONU2KNVQP9n9cb6XSfit4U1QMPP1BcKLiomREr8B
         B7q6zgsw0vGC3yi5ozCiuv/WyggxwcDxr2fzpXhRFdij79Za1MpRoGPd5dt0EQkBgOq4
         4tQA==
X-Gm-Message-State: AOJu0YynvFIDv0l4S00KcaGN3sYP3Oz11WNsEqE/ERyWX4zpyRpjHcjS
	KLd2MCGUW+yNF0SWJR12LT6IjwS9dQnwjjWOvPqitJt0X2QODsJEVJ116tZK3fMAlnNQ0TvRrRy
	n
X-Google-Smtp-Source: AGHT+IGjUYxNLa2ANB9M8W3Yq72DLYSKo/fpopvLQVj9Ov9vaL9WDbnLs5DR144E5aX43nVMPh4PcQ==
X-Received: by 2002:a17:906:130f:b0:a5a:51dc:d1a with SMTP id a640c23a62f3a-a5a51dc13bdmr1026234166b.3.1715885888168;
        Thu, 16 May 2024 11:58:08 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: [PATCH v2 1/4] LICENSES: Add MIT-0 (MIT No Attribution)
Date: Thu, 16 May 2024 19:58:01 +0100
Message-Id: <20240516185804.3309725-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We are about to import code licensed under MIT-0.  It's compatible for us to
use, so identify it as a permitted license.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Török <edwin.torok@cloud.com>

v2:
 * New
---
 LICENSES/MIT-0 | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 LICENSES/MIT-0

diff --git a/LICENSES/MIT-0 b/LICENSES/MIT-0
new file mode 100644
index 000000000000..70fb90ee3486
--- /dev/null
+++ b/LICENSES/MIT-0
@@ -0,0 +1,31 @@
+Valid-License-Identifier: MIT-0
+
+SPDX-URL: https://spdx.org/licenses/MIT-0.html
+
+Usage-Guide:
+
+  To use the MIT-0 License put the following SPDX tag/value pair into a
+  comment according to the placement guidelines in the licensing rules
+  documentation:
+    SPDX-License-Identifier: MIT-0
+
+License-Text:
+
+MIT No Attribution
+
+Copyright <year> <copyright holder>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 18:58:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 18:58:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723569.1128513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIt-00050o-9G; Thu, 16 May 2024 18:58:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723569.1128513; Thu, 16 May 2024 18:58:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIt-00050h-5K; Thu, 16 May 2024 18:58:11 +0000
Received: by outflank-mailman (input) for mailman id 723569;
 Thu, 16 May 2024 18:58:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7gIr-00050W-Rc
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 18:58:09 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3bad26e6-13b6-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 20:58:07 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so288234066b.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 11:58:07 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d9dsm1003072566b.132.2024.05.16.11.58.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 11:58:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bad26e6-13b6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715885887; x=1716490687; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=zcr1R+3jXcw0CmtqFZJpukM9gBQdwIGdy8JTCpdlTf8=;
        b=d9G9XjcCefaqYZDH3lFMZ1QGHUOU603xm1RDenNvrvLs5EzrM0VDSk9WouHSsRo1Qd
         YksXhReWaNANr3Q2knKVq2Oh7S7CemrMTe50cxZqzrO8I8hNS5YU0hwmiOeWSp64qFvw
         X97Fc3S8ACQWLNJNk8CutOZuwS5s+IeB7TEzs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715885887; x=1716490687;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=zcr1R+3jXcw0CmtqFZJpukM9gBQdwIGdy8JTCpdlTf8=;
        b=ilQGW+CAqoJSEkGgXiTyb6c3JuSbdimhnGYpM3vGU73jR/8sWolvqGD7ZFc3AiZzB1
         r46zcGpdw939/ipF0PydbfcD3CxcOww8Gu1hvK/3vBknPR33w8rcbkFDgHPLKFU6aB+k
         MWifNe57TGQDOLIhe9L9p1vewAYBwMDVjf/62jrxaPxxNVEd973mb31mHrOY2OBlw/K9
         1jQxw6GzxrtjQukKDC4rwPAH9vtsjfjuQQEWhhB5VEvWJGoM9tV9oQ+OiqqgSC6nFnOD
         4QgZbcSm1jWhQ+pidzuGXcfOAG5QqeDn1SlVwowlI1CFpAukUUj+gcySKY+IGeNvX3GS
         Ov1w==
X-Gm-Message-State: AOJu0YwXYU5dwTxfrXXDbnVzLFgeGjlNv/p6QtJfn3UmOpC2mHafAGyH
	MiG5z/xt0Uhoy4Xh2klsSaYBKWUxshiD9fYPhmh3K3iTOIT1X9kNDfxlcs0BfW3CkJqKeofNgES
	x
X-Google-Smtp-Source: AGHT+IHGyydj41kfIUIurnmukhVRSCcCUMp2GV5ov6QeWFgr6KyvVGayrdb7K365XV4AIEVrFPnIRg==
X-Received: by 2002:a17:907:10d5:b0:a59:ced4:25b0 with SMTP id a640c23a62f3a-a5a2d5cb153mr1840131466b.34.1715885887222;
        Thu, 16 May 2024 11:58:07 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: [PATCH v2 for-4.19 0/4] Drop libsystemd
Date: Thu, 16 May 2024 19:58:00 +0100
Message-Id: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On advise from the systemd leadership.

v2:
 * Import the free-standing example and use that to retain existing
   functionality.

Andrew Cooper (4):
  LICENSES: Add MIT-0 (MIT No Attribution)
  tools: Import standalone sd_notify() implementation from systemd
  tools/{c,o}xenstored: Don't link against libsystemd
  tools: Drop libsystemd as a dependency

 LICENSES/MIT-0                                |  31 +++
 automation/build/archlinux/current.dockerfile |   1 +
 .../build/suse/opensuse-leap.dockerfile       |   1 +
 .../build/suse/opensuse-tumbleweed.dockerfile |   1 +
 automation/build/ubuntu/focal.dockerfile      |   1 +
 config/Tools.mk.in                            |   2 -
 m4/systemd.m4                                 |   9 -
 tools/configure                               | 256 ------------------
 tools/include/xen-sd-notify.h                 |  98 +++++++
 tools/ocaml/xenstored/Makefile                |   2 -
 tools/ocaml/xenstored/systemd_stubs.c         |   2 +-
 tools/xenstored/Makefile                      |   5 -
 tools/xenstored/posix.c                       |   4 +-
 13 files changed, 136 insertions(+), 277 deletions(-)
 create mode 100644 LICENSES/MIT-0
 create mode 100644 tools/include/xen-sd-notify.h


base-commit: 977d98e67c2e929c62aa1f495fc4c6341c45abb5
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 18:58:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 18:58:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723571.1128533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIv-0005TU-MB; Thu, 16 May 2024 18:58:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723571.1128533; Thu, 16 May 2024 18:58:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gIv-0005TN-Ii; Thu, 16 May 2024 18:58:13 +0000
Received: by outflank-mailman (input) for mailman id 723571;
 Thu, 16 May 2024 18:58:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7gIt-00050W-NV
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 18:58:11 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d0a2313-13b6-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 20:58:10 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-574f7c0bab4so4099333a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 11:58:10 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d9dsm1003072566b.132.2024.05.16.11.58.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 11:58:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d0a2313-13b6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715885889; x=1716490689; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uqlRzkVHjotj0GkagN3hJ6XGCuDGrgNSXxfSAPXzj6I=;
        b=P38v09faVcPoIOXU4vjcHZkYpHsmilKZQQWhP2jnhW6NnDNMlke1R38WIg/DTowNfe
         WdRyVnFZF1I4l6tS9LreT0kIPeybnRiWrQpHha96bbcB/yJ3+n6d40xLqH4jlZdc4OdV
         ZgQ1+GumqmOUpc5wm7aNdDfchUpxT0LIDNPsM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715885889; x=1716490689;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=uqlRzkVHjotj0GkagN3hJ6XGCuDGrgNSXxfSAPXzj6I=;
        b=cWbdrIiYqpFNR7kRlwErU4me4t3D3zbEzvdwmWAsYPMSTL7AmPE4gzl0lAdMFnnh3c
         nUyw4QDM9TV7vSuUh09aoj5uaYCwLHyWKKmDxbelRhYUDAJjsbzvNcIIiHJq5IVrAjQF
         v4QD+FZT+aCHgID86NsU9TZ8FqS7C1xim0ukV5i3QHBizRvjJITtFAMeqaLk1FAkpmYE
         4w4noqs76Uy8tt7krInS5kE0HVVCGnlkb4jkXvUihoFfjiioa7bCWgdeRdRVNKFiO62O
         1SnJ1zfpDnjt3RnCq5XkwxovgSkCdDpPiUZBinx1FeXNk5L75lStD7UJ9mEW1zmCnwcM
         XPgg==
X-Gm-Message-State: AOJu0YytiHqm6h3IaY+CK2mbtzKPTAFN42ZstOTCq239KyCoef62dBc2
	KoL8YFoy4I1ouSuu4L7UqONxrvRZC4JpfNiYRET3zEW/EUfLMN3qD4Guuu5i9QgADILzDX+2MF7
	Z
X-Google-Smtp-Source: AGHT+IEyWF6HXBnqO7J1kvSEwSr9tmQ4oJzGkpMwGHxblxwyZ36RhocSFZ8BIEIScVc6p5idf5CITQ==
X-Received: by 2002:a17:906:da8d:b0:a5a:7a1:5da3 with SMTP id a640c23a62f3a-a5a2d09e4b4mr1964968566b.0.1715885889559;
        Thu, 16 May 2024 11:58:09 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: [PATCH v2 2/4] tools: Import standalone sd_notify() implementation from systemd
Date: Thu, 16 May 2024 19:58:02 +0100
Message-Id: <20240516185804.3309725-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

... in order to avoid linking against the whole of libsystemd.

Only minimal changes to the upstream copy, to function as a drop-in
replacement for sd_notify() and as a header-only library.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Török <edwin.torok@cloud.com>

v2:
 * New
---
 tools/include/xen-sd-notify.h | 98 +++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)
 create mode 100644 tools/include/xen-sd-notify.h

diff --git a/tools/include/xen-sd-notify.h b/tools/include/xen-sd-notify.h
new file mode 100644
index 000000000000..eda9d8b22d9e
--- /dev/null
+++ b/tools/include/xen-sd-notify.h
@@ -0,0 +1,98 @@
+/* SPDX-License-Identifier: MIT-0 */
+
+/*
+ * Implement the systemd notify protocol without external dependencies.
+ * Supports both readiness notification on startup and on reloading,
+ * according to the protocol defined at:
+ * https://www.freedesktop.org/software/systemd/man/latest/sd_notify.html
+ * This protocol is guaranteed to be stable as per:
+ * https://systemd.io/PORTABILITY_AND_STABILITY/
+ *
+ * Differences from the upstream copy:
+ * - Rename/rework as a drop-in replacement for systemd/sd-daemon.h
+ * - Only take the subset Xen cares about
+ * - Respect -Wdeclaration-after-statement
+ */
+
+#ifndef XEN_SD_NOTIFY
+#define XEN_SD_NOTIFY
+
+#include <errno.h>
+#include <stddef.h>
+#include <stdlib.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <unistd.h>
+
+static inline void xen_sd_closep(int *fd) {
+  if (!fd || *fd < 0)
+    return;
+
+  close(*fd);
+  *fd = -1;
+}
+
+static inline int xen_sd_notify(const char *message) {
+  union sockaddr_union {
+    struct sockaddr sa;
+    struct sockaddr_un sun;
+  } socket_addr = {
+    .sun.sun_family = AF_UNIX,
+  };
+  size_t path_length, message_length;
+  ssize_t written;
+  const char *socket_path;
+  int __attribute__((cleanup(sd_closep))) fd = -1;
+
+  /* Verify the argument first */
+  if (!message)
+    return -EINVAL;
+
+  message_length = strlen(message);
+  if (message_length == 0)
+    return -EINVAL;
+
+  /* If the variable is not set, the protocol is a noop */
+  socket_path = getenv("NOTIFY_SOCKET");
+  if (!socket_path)
+    return 0; /* Not set? Nothing to do */
+
+  /* Only AF_UNIX is supported, with path or abstract sockets */
+  if (socket_path[0] != '/' && socket_path[0] != '@')
+    return -EAFNOSUPPORT;
+
+  path_length = strlen(socket_path);
+  /* Ensure there is room for NUL byte */
+  if (path_length >= sizeof(socket_addr.sun.sun_path))
+    return -E2BIG;
+
+  memcpy(socket_addr.sun.sun_path, socket_path, path_length);
+
+  /* Support for abstract socket */
+  if (socket_addr.sun.sun_path[0] == '@')
+    socket_addr.sun.sun_path[0] = 0;
+
+  fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0);
+  if (fd < 0)
+    return -errno;
+
+  if (connect(fd, &socket_addr.sa, offsetof(struct sockaddr_un, sun_path) + path_length) != 0)
+    return -errno;
+
+  written = write(fd, message, message_length);
+  if (written != (ssize_t) message_length)
+    return written < 0 ? -errno : -EPROTO;
+
+  return 1; /* Notified! */
+}
+
+static inline int sd_notify(int unset_environment, const char *message) {
+    int r = xen_sd_notify(message);
+
+    if (unset_environment)
+        unsetenv("NOTIFY_SOCKET");
+
+    return r;
+}
+
+#endif /* XEN_SD_NOTIFY */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 16 19:15:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 19:15:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723590.1128562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gZy-00032f-Pf; Thu, 16 May 2024 19:15:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723590.1128562; Thu, 16 May 2024 19:15:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gZy-00032Y-Mv; Thu, 16 May 2024 19:15:50 +0000
Received: by outflank-mailman (input) for mailman id 723590;
 Thu, 16 May 2024 19:15:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7gZx-00032S-Iu
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 19:15:49 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b321d055-13b8-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 21:15:47 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e6792ea67fso14936681fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 12:15:47 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0bbd6a7sm24594531fa.6.2024.05.16.12.15.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 12:15:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b321d055-13b8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715886947; x=1716491747; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=EPaBePz3BonMngO7cdhCKA9KQJOsT7QDQu8wSi91ybY=;
        b=dj82P4XAKaBUBTF1qXMgCxjr/vWWFIga2nONSF1JSNMZgNyIJoDz4dFjChQ0QpUlq9
         zAH72OemAg4S3/4iScfg6k1+9Qr4EOTnGdUJZ9hcMaOwqI4ktZZNdMIgVVyH8/fVvhHa
         KxOLYe9Q0oWl14WiNbphIBiDdMLfOfYJdY+6w0W2zLFfC26SLhCDDMPwpGoyF2ZdMGZY
         3zRhclcoRSCKy0KbgcKEOC6I5Swk7eCcbWmnTgWtjdcgoLBx+ISRTMjeNOWsRXRdMuoy
         qY5fyIUTYMjCTDP1tjwEv7bYMZyvL8g+gqh6mWfOr7SaC2sHUc1w2LkKEV/OPPPdORdT
         Cd6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715886947; x=1716491747;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EPaBePz3BonMngO7cdhCKA9KQJOsT7QDQu8wSi91ybY=;
        b=pN/+LxSaHrim7iRb1AmydIo1NFAeB6H0RgZbIRPTHZ/KTyW6Pf25K07DxBqUHae/Wv
         gET9XE+tCOfGx/01QMgO5qQ6ECTaFiYQZ2EW9FB+j158JyIefn7ilZpFAsJmbmBqM9kw
         a9jSJ0gkqwsnGaFRr6RbV60skOF3Ogfx+8gRoat4yAQaqmyXXvbwBNo+Wy/C6re+eIZr
         Zt4eQCCxoevwSJx31VF0UqOS7WBsOdfMv+DS+p+CHAvXw2ghTx7W/A6YCSI6DVeHFSzF
         uYjoIayEpBPjO92zFMUp03V3qpf/82DBuxsIH0olcwiPGJts35IzjU/z7lj9yrejg3BO
         KY5w==
X-Forwarded-Encrypted: i=1; AJvYcCUCDKGfP9SPNV6hzp/AQPapoDPoNxM7VXWTDEs5R4+ErOS8a4xlIeW2AdyIK1t/4s64Xu8k3xFnVLU9BEBD6B2+Ri3IuU1LbdTquMjq15Q=
X-Gm-Message-State: AOJu0YwZSZEuAQsEasqT1Lo8JdJ49rWVFV1dRayTN9d8XoxnWVzAACy/
	LqCLgR+C1U87sVj5eUZvvit3H6Dofho2vdcLsLlk1OeiDoBaQfvo
X-Google-Smtp-Source: AGHT+IH/QAgWew3vdVb/siW5lJyXYzpLrvjv7UqnF+iFproJwF7vzzjyIyp6JOwHT3JKOIj+AJnBSg==
X-Received: by 2002:a2e:9253:0:b0:2e0:4cbb:858a with SMTP id 38308e7fff4ca-2e520062823mr118711191fa.27.1715886946319;
        Thu, 16 May 2024 12:15:46 -0700 (PDT)
Message-ID: <66ac3d6aaf0e393ce76580f274c222d26c0aa0a1.camel@gmail.com>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Daniel Smith <dpsmith@apertussolutions.com>, 
 Xen-devel <xen-devel@lists.xenproject.org>, "committers@xenproject.org"
 <committers@xenproject.org>
Date: Thu, 16 May 2024 21:15:45 +0200
In-Reply-To: <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
	 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
	 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
	 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
	 <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
	 <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-14 at 12:13 +0100, Julien Grall wrote:
> Hi,
>=20
> (+ Oleksii as the release manager)
>=20
> Chiming into the discussion as there seems there is disagreement.
>=20
> On 14/05/2024 11:03, Jan Beulich wrote:
> > On 14.05.2024 11:51, Andrew Cooper wrote:
> > > On 14/05/2024 10:25 am, Jan Beulich wrote:
> > > > On 03.04.2024 08:16, Jan Beulich wrote:
> > > > > On 02.04.2024 19:06, Andrew Cooper wrote:
> > > > > > The commit makes a claim without any kind of justification.
> > > > > Well, what does "have no business" leave open?
> > > > >=20
> > > > > > The claim is false, and the commit broke lsevtchn in dom0.
> > > > > Or alternatively lsevtchn was doing something that was never
> > > > > meant to work
> > > > > (from Xen's perspective).
> > > > >=20
> > > > > > =C2=A0 It is also quite
> > > > > > obvious from XSM_TARGET that it has broken device model
> > > > > > stubdoms too.
> > > > > Why would that be "obvious"? What business would a stubdom
> > > > > have to look at
> > > > > Xen's side of an evtchn?
> > > > >=20
> > > > > > Whether to return information about a xen-owned evtchn is a
> > > > > > matter of policy,
> > > > > > and it's not acceptable to short circuit the XSM on the
> > > > > > matter.
> > > > > I can certainly accept this as one possible view point. As in
> > > > > so many cases
> > > > > I'm afraid I dislike you putting it as if it was the only
> > > > > possible one.
> > > > >=20
> > > > > In summary: The supposed justification you claim is missing
> > > > > in the original
> > > > > change is imo also missing here then: What business would any
> > > > > entity in the
> > > > > system have to look at Xen's side of an event channel? Back
> > > > > at the time, 3
> > > > > people agreed that it's "none".
> > > > You've never responded to this reply of mine, or its follow-up.
> > > > You also
> > > > didn't chime in on the discussion Daniel and I were having. I
> > > > consider my
> > > > objections unaddressed, and in fact I continue to consider the
> > > > change to
> > > > be wrong. Therefore it was inappropriate for you to commit it;
> > > > it needs
> > > > reverting asap. If you're not going to do so, I will.
> > >=20
> > > You tried defending breaking a utility with "well it shouldn't
> > > exist then".
> > >=20
> > > You don't have a leg to stand on, and two maintainers of relevant
> > > subsystems here just got tired of bullshit being presented in
> > > place of
> > > any credible argument for having done the change in the way you
> > > did.
> >=20
> > Please can you finally get into the habit of not sending rude
> > replies?
> >=20
> > > The correct response was "Sorry I broke things.=C2=A0 Lets revert thi=
s
> > > for
> > > now to unbreak, and I'll see about reworking it to not
> > > intentionally
> > > subvert Xen's security mechanism".
> >=20
> > I'm sorry, but I didn't break things. I made things more consistent
> > with
> > the earlier change, as pointed out before: With your revert,
> > evtchn_status() is now (again) inconsistent with e.g.
> > evtchn_send(). If
> > you were serious about this being something that needs leaving to
> > XSM,
> > you'd have adjusted such further uses of consumer_is_xen() as well.
> > But
> > you aren't. You're merely insisting on lsevtchn needing to continue
> > to
> > work in a way it should never have worked, with a patch to improve
> > the
> > situation already pending.
> >=20
> > Just to state a very basic principle here again: Xen-internal event
> > channels ought to either be fully under XSM control when it comes
> > to
> > domains attempting to access them (in whichever way), or they
> > should
> > truly be Xen-internal, with access uniformly prevented. To me the
> > former option simply makes very little sense.
>=20
> I agree we need consistency on how we handle security policy event=20
> channel. Although, I don't have a strong opinion on which way to go.
>=20
> For the commit message, it is not entirely clear what "broke
> lseventch=20
> in dom0" really mean. Is it lsevtchn would not stop or it will just
> not=20
> display the event channel?
>=20
> If the former, isn't a sign that the tool needs to be harden a bit
> more?=20
> If the latter, then I would argue that consistency for the XSM policy
> is=20
> more important than displaying the event channel for now (the patch
> was=20
> also committed 3 years ago...).
>=20
> So I would vote for a revert and, if desired, replacing with a patch=20
> that would change the XSM policy consistently. Alternatively, the=20
> consistency should be a blocker for Xen 4.19.
Sorry for the delayed response.

I am not deeply familiar with the technical details surrounding XSM,
but if I understand Daniel's point correctly, the original change
violates the access control framework. This suggests to me that the
revert should be merged.

However, I have a question: if we merge this revert, does it mean that
using channels a user ( domain ) will be able to get information about
certain events such as EVTCHNSTAT_unbound, EVTCHNSTAT_interdomain,
EVTCHNSTAT_pirq, EVTCHNSTAT_virq, and EVTCHNSTAT_IPI (based on the code
of lseventch.c)? Is this information really so critical that it cannot
be exposed for some time until a patch that changes the XSM policy
consistently is provided and merged?

If this information is indeed critical and should not be exposed, I
think we can consider Daniel's suggestion to add a check to the dummy
XSM policy as a solution.

~ Oleksii
>=20
> >=20
> > > As it stands, you're 2-1 outvoted, and wasted any sympathy I may
> > > have
> > > had for the principle of the change based on the absurdity of
> > > your
> > > arguments.
> >=20
> > No, pending objections are pending objections. Daniel's responses
> > didn't
> > eliminate them.
>=20
> Indeed, this is rule 4 of the check-in policy:
>=20
> 4. There must be no "open" objections.
>=20
> I don't view Jan's objections as unreasonable in particular for the=20
> consistency part.
>=20
> > As a separate aspect: I can't assume anymore that it is just
> > coincidence
> > that you taking such a controversial action is at a time when I'm
> > away.
>=20
> Cheers,
>=20



From xen-devel-bounces@lists.xenproject.org Thu May 16 19:21:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 19:21:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723595.1128573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gfI-0005D0-Ci; Thu, 16 May 2024 19:21:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723595.1128573; Thu, 16 May 2024 19:21:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7gfI-0005Ct-98; Thu, 16 May 2024 19:21:20 +0000
Received: by outflank-mailman (input) for mailman id 723595;
 Thu, 16 May 2024 19:21:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0hAQ=MT=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7gfG-00059F-VK
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 19:21:18 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7793b6ac-13b9-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 21:21:18 +0200 (CEST)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-52232d0e5ceso1536959e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 12:21:16 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-523940af3f2sm767433e87.266.2024.05.16.12.21.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 12:21:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7793b6ac-13b9-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715887276; x=1716492076; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=mmaUfRjshwvwnk3PFzLF0Gg+hca6VCpx7x9A7NqF9v4=;
        b=Q8LB5HjqkMEXsDLlmckhgVASQYfQHG95SQn8jwPprlVmPehz6ti2gOqTJ/LKQ+6U5P
         ruQvsW4Sp5kAJ8C1qKJirws1Pg28TfaH1296gu63UzDJW+HSbD5H+LizSbwaVeQ8o11x
         YI3mBFXGujK4O6vRAmUoYxbN/w1kzGNRoHLKn6JCMYeIFzeVk19Lpl5XNzjsiikWYIyY
         pmFXFJNx4pcRQQNdO+n1UF1ks6p8oEmZ0mu0dJq+ukI8nbnqz4IQ04kOO34CNrd9Jufr
         pBvWmGNGTaYGKJtt83u7JlvAXBgFPIsRUJZR2EsH+CyYb0bXVs9+urnDKjbIBjmk17Pr
         ki6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715887276; x=1716492076;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mmaUfRjshwvwnk3PFzLF0Gg+hca6VCpx7x9A7NqF9v4=;
        b=WlLC7UAYYdci4RE1ffZHhW6pM5GNkHkVjt8mjQkpiKTg2085YfuayjCBHiR0nEzRbG
         5EtXtDHCUvS63NPsESC1+7do4kYdfPX9Fr2TDOv/xfbLfw/LTk0KRjiSakYTcfVauEcL
         utYnaxa7xWAl/R6TuYuBSMIvoZc09EukYVqzN89JrZ5+jIu04fk+AlkPsfNnzX88jbFQ
         qZsHk8RfU7lRIc2QpHXRJzx9FQpNBjjk04r81VWiqaXNnM+3SL12qZBsR3Mu5yhfEU+U
         OtxozgJPTXCv+OoHWYnTz+Zl7XKDRP6Nx/TpdhQlE8yLYC3r+YRHWSfjWgMY+ZFo+S6F
         O4xQ==
X-Forwarded-Encrypted: i=1; AJvYcCXvJfcdAnFqrYzVY4qgMUUUKujL6IFxjwCdn/6X6OT9PBky3wD3192UEt9PljvV43u6zPcSZv/vMwXquaXpyqxJJng9ktHNVDblFjVLIEY=
X-Gm-Message-State: AOJu0YxUQ7xchmaA5sy542M6LL9ShmVJ82k1ls09LCpFeKLtoRK0NEzs
	YeEustibMEQFsEPcTZabAGzy2QfhGzk151LG0qCfJCBG5gbfS/u1YCFSJILq
X-Google-Smtp-Source: AGHT+IFm/OBu+XSX0FEsDFbIWw2Dh0xa4cjHCpTjBovW0f73PbQUCdqdt2SYTpoQCTm4tgTFQPmyog==
X-Received: by 2002:a05:6512:3293:b0:51d:9818:33fa with SMTP id 2adb3069b0e04-5221027603cmr11898394e87.68.1715887276201;
        Thu, 16 May 2024 12:21:16 -0700 (PDT)
Message-ID: <0bdb1e3952e3527083cb63295dc16c3403f0ed7e.camel@gmail.com>
Subject: Re: [XEN PATCH 0/4] address violations of MISRA C Rule 20.7
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com, 
	ayan.kumar.halder@amd.com, consulting@bugseng.com, Andrew Cooper
	 <andrew.cooper3@citrix.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>, Nicola Vetrini <nicola.vetrini@bugseng.com>, 
	xen-devel@lists.xenproject.org
Date: Thu, 16 May 2024 21:21:14 +0200
In-Reply-To: <6d19ef9b-ccd8-4ae0-9e9e-ff7f3d92b333@suse.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
	 <c67c1b8b-e14b-4c30-a381-1b89aedcddb9@suse.com>
	 <7f8ef88d0a8bd3e40b213d19dfc80cd3fd2db298.camel@gmail.com>
	 <6d19ef9b-ccd8-4ae0-9e9e-ff7f3d92b333@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-16 at 18:08 +0200, Jan Beulich wrote:
> > > for 4.18 we took a relaxed approach towards (simple) changes for
> > > Misra purposes.
> > > I wonder whether you mean to permit the same for 4.19, or whether
> > > series like
> > > this one rather want/need delaying until after branching.
> > Lets follow the same approach for 4.19.
>=20
> Well, okay. But if you don't say now until when this is okay, you'll
> need to announce the "stop" very prominently later on, so no-one
> misses it.
For me it is okay until we don't have Hard Code Release deadline.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Thu May 16 19:57:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 19:57:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723609.1128582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7hEF-0002zW-4d; Thu, 16 May 2024 19:57:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723609.1128582; Thu, 16 May 2024 19:57:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7hEF-0002zP-1y; Thu, 16 May 2024 19:57:27 +0000
Received: by outflank-mailman (input) for mailman id 723609;
 Thu, 16 May 2024 19:57:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WdV4=MT=redhat.com=david@srs-se1.protection.inumbo.net>)
 id 1s7hED-0002zJ-N5
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 19:57:25 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.133.124])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 81709bfa-13be-11ef-b4bb-af5377834399;
 Thu, 16 May 2024 21:57:21 +0200 (CEST)
Received: from mail-io1-f69.google.com (mail-io1-f69.google.com
 [209.85.166.69]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-596-3jAgD47jPbamzHPPWV8Lsw-1; Thu, 16 May 2024 15:57:19 -0400
Received: by mail-io1-f69.google.com with SMTP id
 ca18e2360f4ac-7e1e05c39easo554525139f.2
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 12:57:19 -0700 (PDT)
Received: from [10.21.161.194] ([65.132.165.41])
 by smtp.gmail.com with ESMTPSA id
 8926c6da1cb9f-4893700e294sm4197464173.36.2024.05.16.12.57.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 12:57:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81709bfa-13be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1715889440;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=XZrAGdL2yy7itxdPxNw4dGHuJWRBIMxVNdHPyNKpVMw=;
	b=LxbqQ7ZU19SeQFOlbRTzor0Yb78wnHprtlM40mHflZf7bjiD4hEcO+uUTzvJplFgS0db8e
	OTRq2YhookjLalt9MqZVJHhTwwNmoCABt609GxbT8ZySpD5VwTE4IiEPFp5Z1knz8QhAra
	2fGbPdldYpayLMdJbkVkCdCy/EWAs7g=
X-MC-Unique: 3jAgD47jPbamzHPPWV8Lsw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715889438; x=1716494238;
        h=content-transfer-encoding:in-reply-to:organization:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XZrAGdL2yy7itxdPxNw4dGHuJWRBIMxVNdHPyNKpVMw=;
        b=oTtDmAmLcU/QcKklqg82rL9se4UgHZEDROHpieWUvAZ8Z+HM3CsP/3LiMhiLWfy/56
         m+JQjRMsaXfsPuRBoSqdLcGF84lKaAqSLsWFY2S37+naAjImJCCiiZ9rS6NqceF7Og0m
         TsKgijbRG9qdFbWVa02nsJFVYq6S7HHPOhuOjSGNCBvA3Yli4+b0IHNKgJwcN2p6kiAQ
         UF35aB0uzTnsad2o5usP+wzcmqkX+bt8NPd0kE6UZM6bKeuKfk2bKeRJoSOBmm93wY+m
         LPMvFqNrZ4ak05+ClQF7RwAWr0jT/oAJv+PbKdXaMu/wsM5HmlZRPx5H0qDuzYtSjDqM
         LPrQ==
X-Forwarded-Encrypted: i=1; AJvYcCV7DFHRKfA02dmGRjEkWnlGr0UjKaFy2M4Io9hv8LicDKmuKqDWvsOitkWPUqYDXpIJWbQ1woVL0nX0bvWtcf290dFB+39RGnSpnmTmKKE=
X-Gm-Message-State: AOJu0YycGgbXH4hciOeGjz7cgileUeKKX5yZ2uHAcWPz43DUzX4rGs89
	21zzkqIAm+NVUhad4riOSOew/gHVr4iXEfu6AGSmThIfpEbASkCE3ym7VFYonGH34FHrL1blDfL
	pWy0nHdTv5z/e+vh8eVI10QrffLZQ0MK8HmaBRy9Imj/wWCEkn5uEIPMh5CajuEIZ
X-Received: by 2002:a6b:720c:0:b0:7e1:5ea0:269a with SMTP id ca18e2360f4ac-7e1b522278bmr2137213939f.21.1715889438265;
        Thu, 16 May 2024 12:57:18 -0700 (PDT)
X-Google-Smtp-Source: AGHT+IHJrPIdPGoj1UhOKTY5HiktV41Se6NkBKqGIK6yxnAXtLBYoBJZyNlXXQ6aYtqDwujgAO+xcQ==
X-Received: by 2002:a6b:720c:0:b0:7e1:5ea0:269a with SMTP id ca18e2360f4ac-7e1b522278bmr2137213039f.21.1715889437992;
        Thu, 16 May 2024 12:57:17 -0700 (PDT)
Message-ID: <94297019-3940-450d-94ef-2d862ab55f66@redhat.com>
Date: Thu, 16 May 2024 21:57:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 6/8] xen: mapcache: Pass the ram_addr offset to
 xen_map_cache()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 xen-devel@lists.xenproject.org
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
 <20240516154804.1114245-7-edgar.iglesias@gmail.com>
From: David Hildenbrand <david@redhat.com>
Autocrypt: addr=david@redhat.com; keydata=
 xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ
 dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL
 QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp
 XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK
 Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9
 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt
 WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc
 UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv
 jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb
 B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk
 ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW
 AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q
 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp
 rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf
 wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4
 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l
 pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd
 KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE
 BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs
 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF
 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9
 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz
 Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb
 T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A
 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk
 CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G
 NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75
 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx
 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS
 lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv
 AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa
 N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3
 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB
 boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq
 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f
 XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ
 a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq
 Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6
 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8
 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E
 th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr
 jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt
 WNyWQQ==
Organization: Red Hat
In-Reply-To: <20240516154804.1114245-7-edgar.iglesias@gmail.com>
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 16.05.24 17:48, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Pass the ram_addr offset to xen_map_cache.
> This is in preparation for adding grant mappings that need
> to compute the address within the RAMBlock.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> ---

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Thu May 16 20:30:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 20:30:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723612.1128593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7hjq-0001Ze-Kf; Thu, 16 May 2024 20:30:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723612.1128593; Thu, 16 May 2024 20:30:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7hjq-0001ZX-Gf; Thu, 16 May 2024 20:30:06 +0000
Received: by outflank-mailman (input) for mailman id 723612;
 Thu, 16 May 2024 20:30:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7hjo-0001FA-PP
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 20:30:04 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 12754f9b-13c3-11ef-909d-e314d9c70b13;
 Thu, 16 May 2024 22:30:03 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5BD2661765;
 Thu, 16 May 2024 20:30:01 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D4F1C113CC;
 Thu, 16 May 2024 20:29:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12754f9b-13c3-11ef-909d-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715891401;
	bh=L6qDb7RoyyQ5xN5zU57yiUI3gcioSC+go1Mb4ZijJrI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KJUXM0vzrZPLc6p437NziW5Yk7eq5iEHIK+sjVtzFT4rK56q5IZ8obZkgLd2mtkYN
	 fdeAJHDYSCG9jdTTJDbr66RCXMsXKXIvfKlx2hN3AEEgz/nwOQQ0D8Xonbh4YsJK8p
	 EXmAc8w4Z0w6daWYRndieZL+t859t2fIw6C4ef41UnCpGtpx7xeG71FQZzjtS0QNE9
	 y97yqv6iFI+ceKgitBkvmBsj0ntBdcW6DUUoS0g0/EYunDJN/FEDLauW6eC6V/TiOw
	 5vMLM5O6emQBdRfxDxhOpbmwX/AcpkdBTXwap1v+/0xg3iSqEb9dL3g5KXOBdrT/ww
	 CdAI33djG0qfQ==
Date: Thu, 16 May 2024 13:29:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, Bjorn Helgaas <bhelgaas@google.com>, 
    "Rafael J . Wysocki" <rafael@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
    "linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, 
    "Huang, Ray" <Ray.Huang@amd.com>
Subject: Re: [RFC KERNEL PATCH v7 2/2] xen/privcmd: Add new syscall to get
 gsi from dev
In-Reply-To: <BL1PR12MB584969F16D93CC4A5C8E1F0FE7ED2@BL1PR12MB5849.namprd12.prod.outlook.com>
Message-ID: <alpine.DEB.2.22.394.2405161329330.2544314@ubuntu-linux-20-04-desktop>
References: <20240515065011.13797-1-Jiqian.Chen@amd.com> <20240515065011.13797-3-Jiqian.Chen@amd.com> <alpine.DEB.2.22.394.2405151537430.2544314@ubuntu-linux-20-04-desktop> <BL1PR12MB584969F16D93CC4A5C8E1F0FE7ED2@BL1PR12MB5849.namprd12.prod.outlook.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 May 2024, Chen, Jiqian wrote:
> On 2024/5/16 06:42, Stefano Stabellini wrote:
> > On Wed, 15 May 2024, Jiqian Chen wrote:
> >> In PVH dom0, it uses the linux local interrupt mechanism,
> >> when it allocs irq for a gsi, it is dynamic, and follow
> >> the principle of applying first, distributing first. And
> >> the irq number is alloced from small to large, but the
> >> applying gsi number is not, may gsi 38 comes before gsi 28,
> >> it causes the irq number is not equal with the gsi number.
> >> And when passthrough a device, QEMU will use device's gsi
> >> number to do pirq mapping, but the gsi number is got from
> >> file /sys/bus/pci/devices/<sbdf>/irq, irq!= gsi, so it will
> >> fail when mapping.
> >> And in current linux codes, there is no method to get gsi
> >> for userspace.
> >>
> >> For above purpose, record gsi of pcistub devices when init
> >> pcistub and add a new syscall into privcmd to let userspace
> >> can get gsi when they have a need.
> >>
> >> Co-developed-by: Huang Rui <ray.huang@amd.com>
> >> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> >> ---
> >>  drivers/xen/privcmd.c              | 28 ++++++++++++++++++++++
> >>  drivers/xen/xen-pciback/pci_stub.c | 38 +++++++++++++++++++++++++++---
> >>  include/uapi/xen/privcmd.h         |  7 ++++++
> >>  include/xen/acpi.h                 |  2 ++
> >>  4 files changed, 72 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> >> index 67dfa4778864..5953a03b5cb0 100644
> >> --- a/drivers/xen/privcmd.c
> >> +++ b/drivers/xen/privcmd.c
> >> @@ -45,6 +45,9 @@
> >>  #include <xen/page.h>
> >>  #include <xen/xen-ops.h>
> >>  #include <xen/balloon.h>
> >> +#ifdef CONFIG_ACPI
> >> +#include <xen/acpi.h>
> >> +#endif
> >>  
> >>  #include "privcmd.h"
> >>  
> >> @@ -842,6 +845,27 @@ static long privcmd_ioctl_mmap_resource(struct file *file,
> >>  	return rc;
> >>  }
> >>  
> >> +static long privcmd_ioctl_gsi_from_dev(struct file *file, void __user *udata)
> >> +{
> >> +	struct privcmd_gsi_from_dev kdata;
> >> +
> >> +	if (copy_from_user(&kdata, udata, sizeof(kdata)))
> >> +		return -EFAULT;
> >> +
> >> +#ifdef CONFIG_ACPI
> >> +	kdata.gsi = pcistub_get_gsi_from_sbdf(kdata.sbdf);
> >> +	if (kdata.gsi == -1)
> >> +		return -EINVAL;
> >> +#else
> >> +	kdata.gsi = -1;
> > 
> > Should we return an error instead, like -EINVAL, to make the behavior
> > more similar to the CONFIG_ACPI case?
> OK, will return -EINVAL if not config acpi.
> Like:
> static long privcmd_ioctl_gsi_from_dev(struct file *file, void __user *udata)
> {
> #ifdef CONFIG_ACPI
> 	struct privcmd_gsi_from_dev kdata;
> 
> 	if (copy_from_user(&kdata, udata, sizeof(kdata)))
> 		return -EFAULT;
> 
> 	kdata.gsi = pcistub_get_gsi_from_sbdf(kdata.sbdf);
> 	if (kdata.gsi == -1)
> 		return -EINVAL;
> 
> 	if (copy_to_user(udata, &kdata, sizeof(kdata)))
> 		return -EFAULT;
> 
> 	return 0;
> #else
> 	return -EINVAL;
> #endif
> }


Yep that's fine



> >> +#endif
> >> +
> >> +	if (copy_to_user(udata, &kdata, sizeof(kdata)))
> >> +		return -EFAULT;
> >> +
> >> +	return 0;
> >> +}
> >> +
> >>  #ifdef CONFIG_XEN_PRIVCMD_EVENTFD
> >>  /* Irqfd support */
> >>  static struct workqueue_struct *irqfd_cleanup_wq;
> >> @@ -1529,6 +1553,10 @@ static long privcmd_ioctl(struct file *file,
> >>  		ret = privcmd_ioctl_ioeventfd(file, udata);
> >>  		break;
> >>  
> >> +	case IOCTL_PRIVCMD_GSI_FROM_DEV:
> >> +		ret = privcmd_ioctl_gsi_from_dev(file, udata);
> >> +		break;
> >> +
> >>  	default:
> >>  		break;
> >>  	}
> >> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> >> index 2b90d832d0a7..4b62b4d377a9 100644
> >> --- a/drivers/xen/xen-pciback/pci_stub.c
> >> +++ b/drivers/xen/xen-pciback/pci_stub.c
> >> @@ -56,6 +56,9 @@ struct pcistub_device {
> >>  
> >>  	struct pci_dev *dev;
> >>  	struct xen_pcibk_device *pdev;/* non-NULL if struct pci_dev is in use */
> >> +#ifdef CONFIG_ACPI
> >> +	int gsi;
> >> +#endif
> >>  };
> >>  
> >>  /* Access to pcistub_devices & seized_devices lists and the initialize_devices
> >> @@ -88,6 +91,9 @@ static struct pcistub_device *pcistub_device_alloc(struct pci_dev *dev)
> >>  
> >>  	kref_init(&psdev->kref);
> >>  	spin_lock_init(&psdev->lock);
> >> +#ifdef CONFIG_ACPI
> >> +	psdev->gsi = -1;
> >> +#endif
> >>  
> >>  	return psdev;
> >>  }
> >> @@ -220,6 +226,25 @@ static struct pci_dev *pcistub_device_get_pci_dev(struct xen_pcibk_device *pdev,
> >>  	return pci_dev;
> >>  }
> >>  
> >> +#ifdef CONFIG_ACPI
> >> +int pcistub_get_gsi_from_sbdf(unsigned int sbdf)
> >> +{
> >> +	struct pcistub_device *psdev;
> >> +	int domain = sbdf >> 16;
> >> +	int bus = (sbdf >> 8) & 0xff;
> >> +	int slot = (sbdf >> 3) & 0x1f;
> >> +	int func = sbdf & 0x7;
> > 
> > you can use PCI_DEVFN PCI_SLOT PCI_FUNC pci_domain_nr instead of open
> > coding.
> Thanks, will change to use these in next version.
> But pci_domain_nr requires passing in pci_dev.
> Will change like:
> 	int domain = (sbdf >> 16) & 0xffff;
> 	int bus = PCI_BUS_NUM(sbdf);
> 	int slot = PCI_SLOT(sbdf);
> 	int func = PCI_FUNC(sbdf);

That's fine


 
> >> +
> >> +	psdev = pcistub_device_find(domain, bus, slot, func);
> >> +
> >> +	if (!psdev)
> >> +		return -1;
> >> +
> >> +	return psdev->gsi;
> >> +}
> >> +EXPORT_SYMBOL_GPL(pcistub_get_gsi_from_sbdf);
> >> +#endif
> >> +
> >>  struct pci_dev *pcistub_get_pci_dev_by_slot(struct xen_pcibk_device *pdev,
> >>  					    int domain, int bus,
> >>  					    int slot, int func)
> >> @@ -367,14 +392,20 @@ static int pcistub_match(struct pci_dev *dev)
> >>  	return found;
> >>  }
> >>  
> >> -static int pcistub_init_device(struct pci_dev *dev)
> >> +static int pcistub_init_device(struct pcistub_device *psdev)
> >>  {
> >>  	struct xen_pcibk_dev_data *dev_data;
> >> +	struct pci_dev *dev;
> >>  #ifdef CONFIG_ACPI
> >>  	int gsi, trigger, polarity;
> >>  #endif
> >>  	int err = 0;
> >>  
> >> +	if (!psdev)
> >> +		return -EINVAL;
> >> +
> >> +	dev = psdev->dev;
> >> +
> >>  	dev_dbg(&dev->dev, "initializing...\n");
> >>  
> >>  	/* The PCI backend is not intended to be a module (or to work with
> >> @@ -448,6 +479,7 @@ static int pcistub_init_device(struct pci_dev *dev)
> >>  		dev_err(&dev->dev, "Fail to get gsi info!\n");
> >>  		goto config_release;
> >>  	}
> >> +	psdev->gsi = gsi;
> >>  
> >>  	if (xen_initial_domain() && xen_pvh_domain()) {
> >>  		err = xen_pvh_setup_gsi(gsi, trigger, polarity);
> >> @@ -495,7 +527,7 @@ static int __init pcistub_init_devices_late(void)
> >>  
> >>  		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
> >>  
> >> -		err = pcistub_init_device(psdev->dev);
> >> +		err = pcistub_init_device(psdev);
> >>  		if (err) {
> >>  			dev_err(&psdev->dev->dev,
> >>  				"error %d initializing device\n", err);
> >> @@ -565,7 +597,7 @@ static int pcistub_seize(struct pci_dev *dev,
> >>  		spin_unlock_irqrestore(&pcistub_devices_lock, flags);
> >>  
> >>  		/* don't want irqs disabled when calling pcistub_init_device */
> >> -		err = pcistub_init_device(psdev->dev);
> >> +		err = pcistub_init_device(psdev);
> >>  
> >>  		spin_lock_irqsave(&pcistub_devices_lock, flags);
> >>  
> >> diff --git a/include/uapi/xen/privcmd.h b/include/uapi/xen/privcmd.h
> >> index 8b8c5d1420fe..220e7670a113 100644
> >> --- a/include/uapi/xen/privcmd.h
> >> +++ b/include/uapi/xen/privcmd.h
> >> @@ -126,6 +126,11 @@ struct privcmd_ioeventfd {
> >>  	__u8 pad[2];
> >>  };
> >>  
> >> +struct privcmd_gsi_from_dev {
> >> +	__u32 sbdf;
> >> +	int gsi;
> >> +};
> >> +
> >>  /*
> >>   * @cmd: IOCTL_PRIVCMD_HYPERCALL
> >>   * @arg: &privcmd_hypercall_t
> >> @@ -157,5 +162,7 @@ struct privcmd_ioeventfd {
> >>  	_IOW('P', 8, struct privcmd_irqfd)
> >>  #define IOCTL_PRIVCMD_IOEVENTFD					\
> >>  	_IOW('P', 9, struct privcmd_ioeventfd)
> >> +#define IOCTL_PRIVCMD_GSI_FROM_DEV				\
> >> +	_IOC(_IOC_NONE, 'P', 10, sizeof(struct privcmd_gsi_from_dev))
> >>  
> >>  #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
> >> diff --git a/include/xen/acpi.h b/include/xen/acpi.h
> >> index 9b50027113f3..0bf5f4884456 100644
> >> --- a/include/xen/acpi.h
> >> +++ b/include/xen/acpi.h
> >> @@ -83,4 +83,6 @@ int xen_acpi_get_gsi_info(struct pci_dev *dev,
> >>  						  int *gsi_out,
> >>  						  int *trigger_out,
> >>  						  int *polarity_out);
> >> +
> >> +int pcistub_get_gsi_from_sbdf(unsigned int sbdf);
> >>  #endif	/* _XEN_ACPI_H */
> >> -- 
> >> 2.34.1
> >>
> 
> -- 
> Best regards,
> Jiqian Chen.
> 


From xen-devel-bounces@lists.xenproject.org Thu May 16 20:40:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 20:40:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723617.1128604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7htv-000470-Jr; Thu, 16 May 2024 20:40:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723617.1128604; Thu, 16 May 2024 20:40:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7htv-00046t-F9; Thu, 16 May 2024 20:40:31 +0000
Received: by outflank-mailman (input) for mailman id 723617;
 Thu, 16 May 2024 20:40:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7htt-00046j-PL; Thu, 16 May 2024 20:40:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7htt-0003uH-Ku; Thu, 16 May 2024 20:40:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7htt-0004MP-6j; Thu, 16 May 2024 20:40:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7htt-0003zf-6N; Thu, 16 May 2024 20:40:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hn1NpSOhAMdoe72OoOHL4E+bPptHRM8/7ZTYx/+qMPE=; b=Vqyk+sXkhTMySR56i3kQXfbrkB
	xp7eO7vYoA6ehFCNFkN7vWv2hcinoXThvD+NpEu5hbs2GnICArUdGLgl04bWFVarCRwvuASEwWac2
	fVi3OTS1hAMrcobrP3ZuITHihyH1EwNhOI8Q95b2UgrDoQRpbfppwAhSMLxIXZjra6aI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186013-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186013: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=762a848223e0fd61aa87b4e040c83dd2c490fb79
X-Osstest-Versions-That:
    xen=abd00b037da5ffa4e8c4508a5df0cd6eabb805a4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 20:40:29 +0000

flight 186013 xen-unstable real [real]
flight 186020 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186013/
http://logs.test-lab.xenproject.org/osstest/logs/186020/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186020-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186007
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186007
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186007
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186007
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186007
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186007
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  762a848223e0fd61aa87b4e040c83dd2c490fb79
baseline version:
 xen                  abd00b037da5ffa4e8c4508a5df0cd6eabb805a4

Last test of basis   186007  2024-05-15 23:08:41 Z    0 days
Testing same since   186013  2024-05-16 08:20:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Nicola Vetrini <nicola.vetrini@bugseng.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   abd00b037d..762a848223  762a848223e0fd61aa87b4e040c83dd2c490fb79 -> master


From xen-devel-bounces@lists.xenproject.org Thu May 16 21:08:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 21:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723632.1128621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7iKf-0008BI-Ru; Thu, 16 May 2024 21:08:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723632.1128621; Thu, 16 May 2024 21:08:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7iKf-0008BB-Nr; Thu, 16 May 2024 21:08:09 +0000
Received: by outflank-mailman (input) for mailman id 723632;
 Thu, 16 May 2024 21:08:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s7iKf-0008B5-9P
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 21:08:09 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s7iKe-0004dK-Rw; Thu, 16 May 2024 21:08:08 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s7iKe-0006Ou-KY; Thu, 16 May 2024 21:08:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YGvNSBqcETXrlL4Ed/249zRhDeO/mCl4MMWac+UEL08=; b=cngKL4brLIrpxCE2M3bT5vxdtc
	+KHxWtaKYCwcEAz31QXjHW2zCQ8EFLOWazGLwuzsgYTdD//has9nM144T1W9yiaqfpSyM8OTgbCLd
	9nrVm6BQ4ot51P+p/M/sas2aZ2HGJLh3/0ALautjgMtyj2Mx7Zh7tCa5mVmnJd+93pgI=;
Message-ID: <afa0d571-5b89-4e7b-a8e0-c0f9692a19fc@xen.org>
Date: Thu, 16 May 2024 22:08:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 0/9] xen/arm: arm64: Annotate code symbols
Content-Language: en-GB
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com,
 Volodymyr_Babchuk@epam.com, jbeulich@suse.com, andrew.cooper3@citrix.com,
 edgar.iglesias@amd.com
References: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240504115514.1063509-1-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Edgar,

On 04/05/2024 12:55, Edgar E. Iglesias wrote:
> Edgar E. Iglesias (9):
>    xen/arm64: entry: Add missing code symbol annotations
>    xen/arm64: smc: Add missing code symbol annotations
>    xen/arm64: sve: Add missing code symbol annotations
>    xen/arm64: head: Add missing code symbol annotations
>    xen/arm64: debug: Add missing code symbol annotations
>    xen/arm64: bpi: Add missing code symbol annotations
>    xen/arm64: mmu/head: Add missing code symbol annotations
>    xen/arm64: cache: Use the generic xen/linkage.h macros
>    xen/arm64: lib: Use the generic xen/linkage.h macros

The series is now committed. Thanks for the clean-up!

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 16 21:23:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 21:23:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723645.1128668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7iZL-0003u3-Gl; Thu, 16 May 2024 21:23:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723645.1128668; Thu, 16 May 2024 21:23:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7iZL-0003tw-CU; Thu, 16 May 2024 21:23:19 +0000
Received: by outflank-mailman (input) for mailman id 723645;
 Thu, 16 May 2024 21:23:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s7iZJ-0003tq-Ud
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 21:23:18 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s7iZJ-00050N-Ke; Thu, 16 May 2024 21:23:17 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s7iZJ-0007OK-FS; Thu, 16 May 2024 21:23:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=h23PA+5jhWdPSgoGaMOBNbWiqUUduCxH34QORiLEckc=; b=j5/yeZVmiaBoBcQ75kESnGuFew
	tfZ6bpoZlz+Kc6yWPAXXt53FE76a2+1EOJlftmn8Q4WEobgCD/g1LQ/eiphoWVFe5KLty/f1lH50c
	qZBKgMEhqzbyH/tAL/B/rLDMrY4fGao8Oj654QFW0VjJh0yvBibRMzEJD7p5SvnGyOd8=;
Message-ID: <fa9a1728-98a1-4b04-8965-8ec01d33b804@xen.org>
Date: Thu, 16 May 2024 22:23:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH 1/2] xen/arm: Add DT reserve map regions to
 bootinfo.reserved_mem
Content-Language: en-GB
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240425131119.2299629-1-luca.fancellu@arm.com>
 <20240425131119.2299629-2-luca.fancellu@arm.com>
 <a9629975-87d1-457b-b6aa-cbeb91fd4854@xen.org>
 <0A931CEA-59CA-4B1F-A8E4-A4CB990C0476@arm.com>
 <d41b633a-7702-41f6-b2b3-26321de53af2@xen.org>
 <F353B76B-D159-43A0-898F-EF8048342B07@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <F353B76B-D159-43A0-898F-EF8048342B07@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Luca,

On 15/05/2024 11:05, Luca Fancellu wrote:
> 
> 
>> On 14 May 2024, at 22:06, Julien Grall <julien@xen.org> wrote:
>>
>> Hi,
>>
>> On 14/05/2024 08:53, Luca Fancellu wrote:
>>> Hi Julien,
>>> Thanks for having a look on the patch,
>>>> On 13 May 2024, at 22:54, Julien Grall <julien@xen.org> wrote:
>>>>
>>>> Hi Luca,
>>>>
>>>> On 25/04/2024 14:11, Luca Fancellu wrote:
>>>>> Currently the code is listing device tree reserve map regions
>>>>> as reserved memory for Xen, but they are not added into
>>>>> bootinfo.reserved_mem and they are fetched in multiple places
>>>>> using the same code sequence, causing duplication. Fix this
>>>>> by adding them to the bootinfo.reserved_mem at early stage.
>>>>
>>>> Do we have enough space in bootinfo.reserved_mem for them?
>>> So we have 255 banks, in my experience I would say I’ve never saw too many reserved regions
>>> in the DT, maybe a couple, but I’ve always had to deal with embedded platforms.
>>> I’ve tested this one with ADLINK AVA board, n1sdp, Juno, raspberry pi, qemu, fvp.
>>> In your experience, have you seen any numbers that could be concerning?
>> I know in the past we had to bump the memory banks a few times. But as you tested on a few platforms, I think we should be ok.
>>
>> It would be best if this patch goes sooner than later to allow wider testing before we release 4.19.
>>
>> Acked-by: Julien Grall <jgrall@amazon.com>
> 
> Yes it would make sense, this patch makes sense on its own, would you/anyone commit it separately while I work on the second
> patch?

Thank you for the confirmation. This is now committed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 16 22:15:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 22:15:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723656.1128681 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7jNP-00045n-9m; Thu, 16 May 2024 22:15:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723656.1128681; Thu, 16 May 2024 22:15:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7jNP-00045g-6o; Thu, 16 May 2024 22:15:03 +0000
Received: by outflank-mailman (input) for mailman id 723656;
 Thu, 16 May 2024 22:15:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7jNO-00045a-Tg
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 22:15:02 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bb7b0e04-13d1-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 00:15:00 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id B1662CE1788;
 Thu, 16 May 2024 22:14:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15BEEC113CC;
 Thu, 16 May 2024 22:14:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb7b0e04-13d1-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715897693;
	bh=vpH+75ReyQpNGKULQBkFlwini+ATp8/4byOP7+8GzsQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TUNCjXxUY4Ng1MJLrBwCMmYu0soYhY5pssyMR5cKiQwNGDE7NAQQS5L/SoL5YDDdK
	 NAmUWFU90G7zodqd2Aggiu6TIdgO7yi4S+eV4tXKsoULkEmx/771dFbbiz8QHIXPqg
	 DThnhvaTVf7aQWcI9ugoN1cFLzIjObXM00Ac/pMb7A98SwrB3HTpSd5slDLIUnDMYm
	 NDukcwmGyEvsRW/zcztwqoKNSiQBt21118XISJ4fgNmkONhjMOzpzj3BCRg+AaHEV9
	 FfmNIURJtMdZC8oMU3gRE8lYwO6pwPXlXJ79ynn463VWXrwacU1AaDyX3vlkecoNpa
	 VPRe5HbbBRUGQ==
Date: Thu, 16 May 2024 15:14:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, 
    George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Christian Lindig <christian.lindig@citrix.com>, 
    =?UTF-8?Q?Edwin_T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>
Subject: Re: [PATCH v2 1/4] LICENSES: Add MIT-0 (MIT No Attribution)
In-Reply-To: <20240516185804.3309725-2-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405161514300.2544314@ubuntu-linux-20-04-desktop>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com> <20240516185804.3309725-2-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1797338315-1715897693=:2544314"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1797338315-1715897693=:2544314
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Andrew Cooper wrote:
> We are about to import code licensed under MIT-0.  It's compatible for us to
> use, so identify it as a permitted license.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin Török <edwin.torok@cloud.com>
> 
> v2:
>  * New
> ---
>  LICENSES/MIT-0 | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>  create mode 100644 LICENSES/MIT-0
> 
> diff --git a/LICENSES/MIT-0 b/LICENSES/MIT-0
> new file mode 100644
> index 000000000000..70fb90ee3486
> --- /dev/null
> +++ b/LICENSES/MIT-0
> @@ -0,0 +1,31 @@
> +Valid-License-Identifier: MIT-0
> +
> +SPDX-URL: https://spdx.org/licenses/MIT-0.html
> +
> +Usage-Guide:
> +
> +  To use the MIT-0 License put the following SPDX tag/value pair into a
> +  comment according to the placement guidelines in the licensing rules
> +  documentation:
> +    SPDX-License-Identifier: MIT-0
> +
> +License-Text:
> +
> +MIT No Attribution
> +
> +Copyright <year> <copyright holder>
> +
> +Permission is hereby granted, free of charge, to any person obtaining a copy
> +of this software and associated documentation files (the "Software"), to deal
> +in the Software without restriction, including without limitation the rights
> +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> +copies of the Software, and to permit persons to whom the Software is
> +furnished to do so.
> +
> +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> +SOFTWARE.
> -- 
> 2.30.2
> 
--8323329-1797338315-1715897693=:2544314--


From xen-devel-bounces@lists.xenproject.org Thu May 16 22:22:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 22:22:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723660.1128690 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7jUr-0006Dp-0L; Thu, 16 May 2024 22:22:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723660.1128690; Thu, 16 May 2024 22:22:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7jUq-0006Di-Tw; Thu, 16 May 2024 22:22:44 +0000
Received: by outflank-mailman (input) for mailman id 723660;
 Thu, 16 May 2024 22:22:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7jUq-0006DY-0j; Thu, 16 May 2024 22:22:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7jUp-00064e-Rt; Thu, 16 May 2024 22:22:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7jUp-0007kF-IW; Thu, 16 May 2024 22:22:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7jUp-0004k0-I1; Thu, 16 May 2024 22:22:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=syQeTUfYeX5lpfkBAOG9RZKlHqsXYRwtmq3Ec0RRmYs=; b=qKwz+Tfbo81uXZ9U5GoHTtVyfs
	zl5kvFnLU697LX0G2Loju57ZTsMjjVCFVec86c5PfFxVAcxZJ3uCOxlRiZaM1gz5e94cHBHcEWlqu
	NsIHZGBneTUzwb/0dd4ms6pxHdCN7CGUX0IqgOwrXl8ErmyLzut0VJhiAaj13DngJvRU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186019-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186019: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=977d98e67c2e929c62aa1f495fc4c6341c45abb5
X-Osstest-Versions-That:
    xen=ae7584f63678cd9adc1c2f3a1e813b67a6b24544
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 May 2024 22:22:43 +0000

flight 186019 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186019/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  977d98e67c2e929c62aa1f495fc4c6341c45abb5
baseline version:
 xen                  ae7584f63678cd9adc1c2f3a1e813b67a6b24544

Last test of basis   186016  2024-05-16 13:02:02 Z    0 days
Testing same since   186019  2024-05-16 18:02:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ae7584f636..977d98e67c  977d98e67c2e929c62aa1f495fc4c6341c45abb5 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 16 23:07:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 23:07:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723673.1128701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kBz-0004Xu-5U; Thu, 16 May 2024 23:07:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723673.1128701; Thu, 16 May 2024 23:07:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kBz-0004Xn-2U; Thu, 16 May 2024 23:07:19 +0000
Received: by outflank-mailman (input) for mailman id 723673;
 Thu, 16 May 2024 23:07:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9+o=MT=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s7kBx-0004Xh-8v
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 23:07:17 +0000
Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com
 [103.168.172.148]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08c9b460-13d9-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 01:07:15 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.nyi.internal (Postfix) with ESMTP id 6C835138148E;
 Thu, 16 May 2024 19:07:14 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Thu, 16 May 2024 19:07:14 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 16 May 2024 19:07:12 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08c9b460-13d9-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715900834;
	 x=1715987234; bh=Xbp6CAWgID8AeK7m7gdmyM1hQxHlHcEfEmPP7h9liDg=; b=
	P0YYATx4KDP1JBFMqxMHNhmLO9rS+j3jX8bpNQ1vreYOZTCnwx9NMDKU8sVQZ7X0
	yeGmzsC0ALx53VVOa2QeXtiYrwx0sGIk+oyDtpN6e/4P2qM7HjXK/TMer8xuOm2g
	Jc7GZ+3jGnKoYozHvBBDrttgKiMaejDJM00BRJ9y6UUl/oMwT0M1yn5TTyA43CI8
	A1Ajet9Wv8CRPXeaRk7BTnxWhSjdleknqa/a5vY8JNuYZG+TosyFZ+3uvIRpTT/Q
	gJVVRYlheJBHdRn1ncaJlAK/Cpm/9bkaPR/qNcjp1+WPsVaxeYfccqxf4JWw+CMS
	VU8CayJcR6C5Xmxe460u0w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715900834; x=1715987234; bh=Xbp6CAWgID8AeK7m7gdmyM1hQxHl
	HcEfEmPP7h9liDg=; b=LYorJPv9yz3l75eOR72FMXmlmQclBancDuDo3Ucaqz9P
	MphLQjP8EW/fHf9r7G3W022aw8jDgz9VevEbsGJFlityovksb9nwzz3BwYfWuPA8
	DH7zdHtBKdAHFRC2kUNDvngHwtjgP54a3Qt311WtrLIcKT2HA43gBBYwSSG7Gg6j
	BtaUZYJvLRZfZJEy/LL8HDr27bft+H+cUAIWhc952LdkoR6BxGL2GvHzS0dyryK8
	X+OpgAOCZI91B7+CilqVw8ID+qgLMTiKSPK5DrY4n51n+nZH2LkQLCLS3DLgUopi
	AJ/EPyR/2C1xY8NisZCtFuFL+XFYh/EE/U7zZlI1Mw==
X-ME-Sender: <xms:opFGZvzA6ChIE1-3jbFF9CiRXZwXeuJ0881cPd_InEdXxG7P1jLjZw>
    <xme:opFGZnSvwkAYHwQ-1EBoZ6Wtw6m_3EZ1sJ9I-7yuiNXf2deSKmgCXI3dJmq8ShL9d
    mKqLO7ScUMciA>
X-ME-Received: <xmr:opFGZpVE18oeh_dW45TsgMI5vre90_P_82QABQE-dsXcLwBH52ysvSCDvFIvDvks1lRLuoaTQaZeYk9gxCzZbPCTgumCT5lJ8Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehvddgudeiucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtroertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnheptdet
    vdfhkedutedvleffgeeutdektefhtefhfffhfeetgefhieegledvtddtkedtnecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:opFGZph9hXkayskPMt4V2gF9Blgo5sGKnpKof1PDcz383ODKGAjM_g>
    <xmx:opFGZhDOEIHHugNdWAI4ryUn3e0VeOoxpRJHooGJfm7E9nmP503mvA>
    <xmx:opFGZiK58GQ_KsI_EfmVPW46ptaS_4JeMtHrBFu5xDCB8ijpsJLnsA>
    <xmx:opFGZgDxyNpMo5LlZmZqkbPNq_VJuonmGrN2FYUm_PyC6OGBXRnIKg>
    <xmx:opFGZls01ety3u9xpjgqbermP_uHNcxq3L8Q3TXZ3o-pG9EQO5VHikf9>
Feedback-ID: i1568416f:Fastmail
Date: Fri, 17 May 2024 01:07:09 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	Edwin =?utf-8?B?VMO2csO2aw==?= <edwin.torok@cloud.com>
Subject: Re: [PATCH v2 2/4] tools: Import standalone sd_notify()
 implementation from systemd
Message-ID: <ZkaRnYqQJGfNURyb@mail-itl>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-3-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="CkPsxyEqsUgIAhd0"
Content-Disposition: inline
In-Reply-To: <20240516185804.3309725-3-andrew.cooper3@citrix.com>


--CkPsxyEqsUgIAhd0
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Fri, 17 May 2024 01:07:09 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	Edwin =?utf-8?B?VMO2csO2aw==?= <edwin.torok@cloud.com>
Subject: Re: [PATCH v2 2/4] tools: Import standalone sd_notify()
 implementation from systemd

On Thu, May 16, 2024 at 07:58:02PM +0100, Andrew Cooper wrote:
> ... in order to avoid linking against the whole of libsystemd.
>=20
> Only minimal changes to the upstream copy, to function as a drop-in
> replacement for sd_notify() and as a header-only library.

Maybe add explicit link to the original source?

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin T=C3=B6r=C3=B6k <edwin.torok@cloud.com>
>=20
> v2:
>  * New
> ---
>  tools/include/xen-sd-notify.h | 98 +++++++++++++++++++++++++++++++++++
>  1 file changed, 98 insertions(+)
>  create mode 100644 tools/include/xen-sd-notify.h
>=20
> diff --git a/tools/include/xen-sd-notify.h b/tools/include/xen-sd-notify.h
> new file mode 100644
> index 000000000000..eda9d8b22d9e
> --- /dev/null
> +++ b/tools/include/xen-sd-notify.h
> @@ -0,0 +1,98 @@

=2E..

> +static inline void xen_sd_closep(int *fd) {

Static inline is one of the changes vs upstream, and gitlab-ci is not
happy about it:

/builds/xen-project/patchew/xen/tools/xenstored/../../tools/include/xen-sd-=
notify.h:45:3: error: cleanup argument not a function
   45 |   int __attribute__((cleanup(sd_closep))) fd =3D -1;
      |   ^~~

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--CkPsxyEqsUgIAhd0
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZGkZ0ACgkQ24/THMrX
1yyr+wf/di0Bt03NikRjB4TFM7zPsNqDzMD/NrS2oiKUJo9xR6rvGGYhZrf2WL69
zUbFa4FRs6MZmlUrmzchp1/zA/0V4n7Cu0o8H7GMNF0yQHq+EyASigWCAm0Mp4vq
0ljMWG8/MoN0BkwTYu1vDdaVDPVlExLrTATSjSrrcc54Epp1ccVqbH6SztQTKDOF
vsZ0KInAT22ytWrEps8+23jDI1SQGQEv7gmuKUHKeHLJhKuppH8Td3AgGy9W2/Td
UpKeyzfV346n9AZxqPLftG1OrB4fQ44l1fwv2Xhd8PN9+bnfQRKZMHjDMVWz0iy8
v+tXmDfpUdbI8YZAtUhd8JY5nyTOIw==
=KL2a
-----END PGP SIGNATURE-----

--CkPsxyEqsUgIAhd0--


From xen-devel-bounces@lists.xenproject.org Thu May 16 23:09:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 23:09:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723676.1128711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kEK-0005fG-If; Thu, 16 May 2024 23:09:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723676.1128711; Thu, 16 May 2024 23:09:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kEK-0005f9-Ef; Thu, 16 May 2024 23:09:44 +0000
Received: by outflank-mailman (input) for mailman id 723676;
 Thu, 16 May 2024 23:09:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lir8=MT=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s7kEJ-0005f3-7E
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 23:09:43 +0000
Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com
 [2607:f8b0:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60223b7a-13d9-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 01:09:42 +0200 (CEST)
Received: by mail-ot1-x330.google.com with SMTP id
 46e09a7af769-6f0ef6bee72so513499a34.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 16:09:42 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2fc67esm848531585a.74.2024.05.16.16.09.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 May 2024 16:09:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60223b7a-13d9-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715900981; x=1716505781; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=1/yXC9IbDu4bX39MTjlu6X71SGfOrBhuEPXKFnZ8qVI=;
        b=p05X2GLkN9F4ptxw8yRIyOCHz7Bsvq9ohyTnbWfatBvYTLFP71RqH4wYQDN179QNVd
         Fe8KQUZUftkFl/o0mIIM/I5RTajLGGvSJ3jjG+0B0iZp4m8GxUjrA+21X5N9jmUqaJkO
         6Pjfaf5wiv+pRzBfjAU5WnLyOu6MmuGzTvO7s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715900981; x=1716505781;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1/yXC9IbDu4bX39MTjlu6X71SGfOrBhuEPXKFnZ8qVI=;
        b=wmNEZKjXIFiXxoX2Abuc6bdZ7BQrtKIZ0l+ajnojMkQM0TP2f7ffiOVB3YQUT+BxHE
         IbTFAmPdmWwoWiH8YMjuRMN9EqXdcpcvnlVSmjPwHtq2RMF9jXj8PdmraPHzm4Ka413K
         1v2inJkMF7DeL3w4q/rbEhmbOOQM4L2M4nuxOSnE6OiQc51tgdlvwtChMqz9WKxie4tv
         PMi/lDASqO3I1IvBA9jbleG7KzLtosNcKivPLy3aRf490zs+0kKP3o3iFHCDkdaJvH+H
         BdSu8b3ltKIrJCB1CcsnUgYwBxAq8S5v6Vdf4hkWuGeINEWvJdU4b7nvzw77cQlB3GCi
         do2w==
X-Gm-Message-State: AOJu0YwgbOAQNe0nFUgdQ/E0i5wlu4FAJ7WG5MDN2i//U1RMJ5hH0Q33
	92EuLe4SZtJ28prGwK6pUXglT3VQSil6Axua8kNZlwGhKNB+EjDc8MvI3uO5vT8=
X-Google-Smtp-Source: AGHT+IEruoVDF5DrobftJJFEu+IHFRfL/kmHd/0Ep2KHK/nb+PW+Z/yL1wFkZwVwBBlZhDEPktgCpw==
X-Received: by 2002:a05:6830:ed9:b0:6f0:8c66:ca1e with SMTP id 46e09a7af769-6f0e91141dcmr20422863a34.2.1715900980730;
        Thu, 16 May 2024 16:09:40 -0700 (PDT)
Message-ID: <2192f69a-3ad3-4cfc-b4eb-2f589bea7099@citrix.com>
Date: Fri, 17 May 2024 00:09:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] tools: Import standalone sd_notify()
 implementation from systemd
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-3-andrew.cooper3@citrix.com>
 <ZkaRnYqQJGfNURyb@mail-itl>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkaRnYqQJGfNURyb@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 17/05/2024 12:07 am, Marek Marczykowski-Górecki wrote:
> On Thu, May 16, 2024 at 07:58:02PM +0100, Andrew Cooper wrote:
>> ... in order to avoid linking against the whole of libsystemd.
>>
>> Only minimal changes to the upstream copy, to function as a drop-in
>> replacement for sd_notify() and as a header-only library.
> Maybe add explicit link to the original source?

Well, it is in the top of the file...

> diff --git a/tools/include/xen-sd-notify.h
> b/tools/include/xen-sd-notify.h new file mode 100644 index
> 000000000000..eda9d8b22d9e --- /dev/null +++
> b/tools/include/xen-sd-notify.h @@ -0,0 +1,98 @@ +/*
> SPDX-License-Identifier: MIT-0 */ + +/* + * Implement the systemd
> notify protocol without external dependencies. + * Supports both
> readiness notification on startup and on reloading, + * according to
> the protocol defined at: + *
> https://www.freedesktop.org/software/systemd/man/latest/sd_notify.html+
> * This protocol is guaranteed to be stable as per: + *
> https://systemd.io/PORTABILITY_AND_STABILITY/+ *



>> +static inline void xen_sd_closep(int *fd) {
> Static inline is one of the changes vs upstream, and gitlab-ci is not
> happy about it:
>
> /builds/xen-project/patchew/xen/tools/xenstored/../../tools/include/xen-sd-notify.h:45:3: error: cleanup argument not a function
>    45 |   int __attribute__((cleanup(sd_closep))) fd = -1;
>       |   ^~~
>

Yes - victim of a last minute refactor.  I've fixed it up locally.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 16 23:42:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 May 2024 23:42:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723694.1128720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kjS-0003cN-30; Thu, 16 May 2024 23:41:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723694.1128720; Thu, 16 May 2024 23:41:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7kjS-0003cG-0O; Thu, 16 May 2024 23:41:54 +0000
Received: by outflank-mailman (input) for mailman id 723694;
 Thu, 16 May 2024 23:41:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HcFO=MT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7kjQ-0003c8-Ni
 for xen-devel@lists.xenproject.org; Thu, 16 May 2024 23:41:52 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dbc6cee5-13dd-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 01:41:49 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 9DFA0CE198B;
 Thu, 16 May 2024 23:41:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5661C2BD11;
 Thu, 16 May 2024 23:41:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbc6cee5-13dd-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715902903;
	bh=s2yTsINqAzvFL10MGMb7Hyz2mo3TKMWfKU796GIYCLo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WwY1johu5/CQkU/uXnib6RCkHPs3XhGStL9CV9iyYtVCYkQ38PNgjEGH+wSvkXdqL
	 9SvVzKoOzBa3CkhWQfu+liRGLDTo5wfj0KRIo2XVg5vXRwVYfsSLHJlIDSolOD2Y3o
	 Gnk3EMhsEiUQLTL7Acw24BQs/Mv6ueUft8boxJ3BJ+MBsBIAQMeLzIy9sxoSxoeoYG
	 XWtztxL+0pbHcBcVXNWhDkhy4xL1rIW4n+3eDg8xCiZfIBgvPnyDGtkFeHehy+y78q
	 gcweMsnF5gDg1O2t+I1xTUxyeuvOMfnLHF2qN5oWpvRyAzxZJqlFbvtXQHh/pscwwy
	 6BC725GdXeg2Q==
Date: Thu, 16 May 2024 16:41:41 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Sergiy Kibrik <Sergiy_Kibrik@epam.com>, xen-devel@lists.xenproject.org, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Subject: Re: [XEN PATCH v2 14/15] iommu/vt-d: guard vmx_pi_hooks_* calls with
 cpu_has_vmx
In-Reply-To: <3104deae-0d27-438e-83ea-035a7786fd4a@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405161640350.2544314@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <73072e5b2ec40ad28d4bcfb9bb0870f3838bb726.1715761386.git.Sergiy_Kibrik@epam.com> <alpine.DEB.2.22.394.2405151751530.2544314@ubuntu-linux-20-04-desktop>
 <3104deae-0d27-438e-83ea-035a7786fd4a@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 May 2024, Jan Beulich wrote:
> On 16.05.2024 02:54, Stefano Stabellini wrote:
> > On Wed, 15 May 2024, Sergiy Kibrik wrote:
> >> VMX posted interrupts support can now be excluded from x86 build along with
> >> VMX code itself, but still we may want to keep the possibility to use
> >> VT-d IOMMU driver in non-HVM setups.
> >> So we guard vmx_pi_hooks_{assign/deassign} with some checks for such a case.
> >>
> >> No functional change intended here.
> >>
> >> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> > 
> > I know that Andrew was keep on having a separate Kconfig option for
> > VT-D, separate from VMX. But still, couldn't we make the VT-D Kconfig
> > option depending on CONFIG_VMX?
> > 
> > To me, VT-D should require VMX, without VMX it should not be possible to
> > enable VT-D.
> > 
> > This comment goes in the same direction of my previous comment regarding
> > the vpmu: we are trying to make things more configurable and flexible
> > and that's good, but we don't necessary need to make all possible
> > combination work. VT-D without VMX is another one of those combination
> > that I would only enable after a customer asks.
> 
> Well. Imo again the configuration should be permitted.

FYI Andrew said the same thing as you on Matrix, so I withdraw my
suggestion.


From xen-devel-bounces@lists.xenproject.org Fri May 17 00:12:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 00:12:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723704.1128731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lD9-0001BE-Nz; Fri, 17 May 2024 00:12:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723704.1128731; Fri, 17 May 2024 00:12:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lD9-0001B7-LG; Fri, 17 May 2024 00:12:35 +0000
Received: by outflank-mailman (input) for mailman id 723704;
 Fri, 17 May 2024 00:12:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7lD7-0001AY-PZ
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 00:12:33 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 25d2ee9e-13e2-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 02:12:32 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D4E16CE1993;
 Fri, 17 May 2024 00:12:27 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53BF8C113CC;
 Fri, 17 May 2024 00:12:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25d2ee9e-13e2-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715904747;
	bh=p6pl74FbIfmFlxT8dPPcHg1hmaZ68bkLMFjCfCRTNWA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=R1x2qYiFCITWjUuNvFhGiZmVwK179ukISANCJksJ6wbQ+/G/u/KmockHBJPVy+Fom
	 pBX5yFWxieQVdcz1/x/v0xK1dFzWi9wTkamJLB98Prrj9StO0zHS6f9xfwsjpWlViM
	 tChCGBaGdHJMAWtqM4/KiS3Xbum4DDD3eSiiyTga+aP6xzV3qF5hKlKOXgfpUMBZ2C
	 IsTS7ybYME94DPodma/kjj77/V57oRN3pBI1vrhawx8YM0neWgFU9fqye6K1jtX4SW
	 SRJB+GVl5QWDJCtw/Qo1fiDXA7pt2m+wh7OvdzoociA27ytwOMSzWD3awwC000q7eL
	 ySjpM1R8Gag7g==
Date: Thu, 16 May 2024 17:12:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 6/8] xen: mapcache: Pass the ram_addr offset to
 xen_map_cache()
In-Reply-To: <20240516154804.1114245-7-edgar.iglesias@gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405161712150.2544314@ubuntu-linux-20-04-desktop>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com> <20240516154804.1114245-7-edgar.iglesias@gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 May 2024, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Pass the ram_addr offset to xen_map_cache.
> This is in preparation for adding grant mappings that need
> to compute the address within the RAMBlock.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Fri May 17 00:50:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 00:50:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723712.1128741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lnL-00078r-Fm; Fri, 17 May 2024 00:49:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723712.1128741; Fri, 17 May 2024 00:49:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lnL-00078k-DA; Fri, 17 May 2024 00:49:59 +0000
Received: by outflank-mailman (input) for mailman id 723712;
 Fri, 17 May 2024 00:49:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7lnK-00078a-Po; Fri, 17 May 2024 00:49:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7lnK-00019h-Ly; Fri, 17 May 2024 00:49:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7lnK-0005l2-DJ; Fri, 17 May 2024 00:49:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7lnK-0003g9-Cm; Fri, 17 May 2024 00:49:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/bHwsa+kWSltTbGxIfG7WIhvfaJAWa+/eBnYz44eYDg=; b=pBcTiVzdCqZ6qtb+ljcZD4ZSY3
	r0Mt+WSOX483eBQboGqehG2D31rmLwCLJETVfNiQZgNEJCFCCK59rXuLzFlx3UFQ914ZDreSnb7vb
	zEP7PDhy8512satyFV3grC2wR3AHIPTxs82M1jYBYeBF86+Ct9T7ivxz9C3yGkdtLgIs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186018-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186018: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
X-Osstest-Versions-That:
    linux=1b294a1f35616977caddaddf3e9d28e576a1adbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 00:49:58 +0000

flight 186018 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186018/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot         fail in 186010 pass in 186018
 test-armhf-armhf-xl-rtds      8 xen-boot         fail in 186010 pass in 186018
 test-armhf-armhf-xl-multivcpu  8 xen-boot        fail in 186010 pass in 186018
 test-armhf-armhf-xl           8 xen-boot         fail in 186010 pass in 186018
 test-armhf-armhf-xl-raw       8 xen-boot         fail in 186010 pass in 186018
 test-armhf-armhf-libvirt      8 xen-boot         fail in 186010 pass in 186018
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186010
 test-armhf-armhf-xl-credit1   8 xen-boot                   fail pass in 186010

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186010 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186010 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186005
 test-armhf-armhf-examine      8 reboot                       fail  like 186005
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186005
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186005
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186005
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
baseline version:
 linux                1b294a1f35616977caddaddf3e9d28e576a1adbc

Last test of basis   186005  2024-05-15 13:12:03 Z    1 days
Testing same since   186010  2024-05-16 01:12:45 Z    0 days    2 attempts

------------------------------------------------------------
509 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   1b294a1f3561..3c999d1ae3c7  3c999d1ae3c75991902a1a7dad0cb62c2a3008b4 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri May 17 00:53:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 00:53:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723718.1128751 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lqE-00009k-Tk; Fri, 17 May 2024 00:52:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723718.1128751; Fri, 17 May 2024 00:52:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7lqE-00009Z-Qz; Fri, 17 May 2024 00:52:58 +0000
Received: by outflank-mailman (input) for mailman id 723718;
 Fri, 17 May 2024 00:52:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7lqD-00009Q-N1
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 00:52:57 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cbf43a37-13e7-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 02:52:56 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 545E56175C;
 Fri, 17 May 2024 00:52:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10DAFC113CC;
 Fri, 17 May 2024 00:52:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbf43a37-13e7-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715907174;
	bh=TegGSMyB0JVbO6jlmENdPDoxx20cRA2GXqpfoyUXYVk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gvKGGnuyc4WhtBlCJuhbccSsXprf1zrb45rBV9rU6MXH0h7E4npLggND/8prIh3n5
	 K7JEMSUkE4XcEo4u9g1ewA+wlynihMT99qyPZB13/mfI4jbJbb9FRlpGFG5qLar9mc
	 xyhX1+h33s9YED2fgglcV3pZkRRlJ9Uah4wjuf+dTYijQwCtoLhvOkgEg1ijMcpHnU
	 Aw3eYvTbFutAx6hVWW6WSw3g5YI+m6riav9Jduf2VKXWM0FXSVKov2441NZOZc+FqV
	 d//YZdCaSdsze9BwKzw6StxG7qsrDLjUg5IC/eLX6DqsARE5O5dhDY2pSkfi2leVMu
	 YRWyR2ZHYrAqg==
Date: Thu, 16 May 2024 17:52:51 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    Juergen Gross <jgross@suse.com>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] drivers/xen: Improve the late XenStore init protocol
In-Reply-To: <028f29be-0393-4a57-83e2-ea27fe0320d5@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405161743170.2544314@ubuntu-linux-20-04-desktop>
References: <20240515014330.1044617-1-xin.wang2@amd.com> <alpine.DEB.2.22.394.2405151524270.2544314@ubuntu-linux-20-04-desktop> <028f29be-0393-4a57-83e2-ea27fe0320d5@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-217339045-1715907173=:2544314"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-217339045-1715907173=:2544314
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Henry Wang wrote:
> Hi Stefano,
> 
> On 5/16/2024 6:30 AM, Stefano Stabellini wrote:
> > On Wed, 15 May 2024, Henry Wang wrote:
> > > Currently, the late XenStore init protocol is only triggered properly
> > > for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
> > > case that XenStore interface is allocated but not ready (the connection
> > > status is not XENSTORE_CONNECTED), Linux should also wait until the
> > > XenStore is set up properly.
> > > 
> > > Introduce a macro to describe the XenStore interface is ready, use
> > > it in xenbus_probe_initcall() and xenbus_probe() to select the code
> > > path of doing the late XenStore init protocol or not.
> > > 
> > > Take the opportunity to enhance the check of the allocated XenStore
> > > interface can be properly mapped, and return error early if the
> > > memremap() fails.
> > > 
> > > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > > Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> > Please add a Fixes: tag
> 
> Sure. Will do.
> 
> > > ---
> > >   drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++++++++++-----
> > >   1 file changed, 16 insertions(+), 5 deletions(-)
> > > 
> > > diff --git a/drivers/xen/xenbus/xenbus_probe.c
> > > b/drivers/xen/xenbus/xenbus_probe.c
> > > index 3205e5d724c8..8aec0ed1d047 100644
> > > --- a/drivers/xen/xenbus/xenbus_probe.c
> > > +++ b/drivers/xen/xenbus/xenbus_probe.c
> > > @@ -72,6 +72,10 @@ EXPORT_SYMBOL_GPL(xen_store_evtchn);
> > >   struct xenstore_domain_interface *xen_store_interface;
> > >   EXPORT_SYMBOL_GPL(xen_store_interface);
> > >   +#define XS_INTERFACE_READY \
> > > +	((xen_store_interface != NULL) && \
> > > +	 (xen_store_interface->connection == XENSTORE_CONNECTED))
> > > +
> > >   enum xenstore_init xen_store_domain_type;
> > >   EXPORT_SYMBOL_GPL(xen_store_domain_type);
> > >   @@ -751,9 +755,10 @@ static void xenbus_probe(void)
> > >   {
> > >   	xenstored_ready = 1;
> > >   -	if (!xen_store_interface) {
> > > -		xen_store_interface = memremap(xen_store_gfn <<
> > > XEN_PAGE_SHIFT,
> > > -					       XEN_PAGE_SIZE, MEMREMAP_WB);
> > > +	if (!xen_store_interface || XS_INTERFACE_READY) {
> > > +		if (!xen_store_interface)
> > These two nested if's don't make sense to me. If XS_INTERFACE_READY
> > succeeds, it means that  ((xen_store_interface != NULL) &&
> > (xen_store_interface->connection == XENSTORE_CONNECTED)).
> > 
> > So it is not possible that xen_store_interface == NULL immediately
> > after. Right?
> 
> I think this is because we want to free the irq for the late init case,
> otherwise the init-dom0less will fail. For the xenstore PFN allocated case,
> the connection is already set to CONNECTED when we execute init-dom0less. But
> I agree with you, would below diff makes more sense to you?
> 
> diff --git a/drivers/xen/xenbus/xenbus_probe.c
> b/drivers/xen/xenbus/xenbus_probe.c
> index 8aec0ed1d047..b8005b651a29 100644
> --- a/drivers/xen/xenbus/xenbus_probe.c
> +++ b/drivers/xen/xenbus/xenbus_probe.c
> @@ -76,6 +76,8 @@ EXPORT_SYMBOL_GPL(xen_store_interface);
>         ((xen_store_interface != NULL) && \
>          (xen_store_interface->connection == XENSTORE_CONNECTED))
> 
> +static bool xs_late_init = false;
> +
>  enum xenstore_init xen_store_domain_type;
>  EXPORT_SYMBOL_GPL(xen_store_domain_type);
> 
> @@ -755,7 +757,7 @@ static void xenbus_probe(void)
>  {
>         xenstored_ready = 1;
> 
> -       if (!xen_store_interface || XS_INTERFACE_READY) {
> +       if (xs_late_init) {
>                 if (!xen_store_interface)
>                         xen_store_interface = memremap(xen_store_gfn <<


I would just remove the outer 'if' and do this:


	if (!xen_store_interface)
		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
				XEN_PAGE_SIZE, MEMREMAP_WB);
	/*
	 * Now it is safe to free the IRQ used for xenstore late
	 * initialization. No need to unbind: it is about to be
	 * bound again from xb_init_comms. Note that calling
	 * unbind_from_irqhandler now would result in xen_evtchn_close()
	 * being called and the event channel not being enabled again
	 * afterwards, resulting in missed event notifications.
	 */
	if (xs_init_irq > 0)
		free_irq(xs_init_irq, &xb_waitq);


I think this should work fine in all cases. I am unsure if
xs_init_irq==0 is possible valid value for xs_init_irq. If it is not,
then we are fine. If 0 is a possible valid irq number, then we should
initialize xs_init_irq to -1, and here check for xs_init_irq >= 0.
--8323329-217339045-1715907173=:2544314--


From xen-devel-bounces@lists.xenproject.org Fri May 17 01:15:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:15:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723729.1128765 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mCB-00035u-OI; Fri, 17 May 2024 01:15:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723729.1128765; Fri, 17 May 2024 01:15:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mCB-00035n-LY; Fri, 17 May 2024 01:15:39 +0000
Received: by outflank-mailman (input) for mailman id 723729;
 Fri, 17 May 2024 01:15:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7mC9-00035h-TG
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:15:37 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f5bbd00b-13ea-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 03:15:35 +0200 (CEST)
Received: from BL0PR02CA0004.namprd02.prod.outlook.com (2603:10b6:207:3c::17)
 by DM6PR12MB4121.namprd12.prod.outlook.com (2603:10b6:5:220::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 01:15:27 +0000
Received: from BL02EPF00021F69.namprd02.prod.outlook.com
 (2603:10b6:207:3c:cafe::f3) by BL0PR02CA0004.outlook.office365.com
 (2603:10b6:207:3c::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Fri, 17 May 2024 01:15:27 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF00021F69.mail.protection.outlook.com (10.167.249.5) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 01:15:27 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 20:15:26 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 20:15:26 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 20:15:24 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5bbd00b-13ea-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aEfwZpMIE6FUgjS+jL00zhCz24VhiSRJVHJ4r3I0xTdGX9oh1PlOgrLz6UWygvKXna1pwFNFAqWYHU9s8DBu1DgamfuMClN0FvecHKoW9RVM05JzbME2SaYShdJhnV9MiT7pdZHByatO+FCvPi5aXYJEoHYUGZSkuGCoF61aWVFQcj6TOpGzMOQ5B8sdkn3/dpY9n8gAt4HLlRsIg46pka+O0IIbi4lZxYHGEC0wMbJjCXAdQTKBZVdx5m/IfW/A2ZQowie/gIu3BTPHyX4MXhlGe97NYNkmRLBPqUZan1qNH8nEdzaa829ZJ6kCj+N9iM7EZ0EZ2tWAbJZp3X2Ouw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wk82CmnnafP/ty0a2WVhVTjambpmPt6exWQt3tANaj0=;
 b=ULrr3xHOZ9FjxAvLaTozYdZVpNrqXgYUigVo4iydmCPf5xbMsDuqLtbQspa63TpWWeZKAESww6YQHPTbddx2Zv1osEPloFAUsg/xyBoEvMFMIfBcBptiffXTuT0RUsQi9Eqw+Ut3SO+XZBm+QIaWU2UBS8dTfH4zycjz//OFt4GQv+OyZKI5fJxPItkEj/TFfRK6FhAC0ahgJrQNMz2FRe2hB/JraMSwe+i3Moqhu2bvOhrDBEpGeRx20Fl4RWCT0GfeIh0ri+08D0uZntq4XTyMOzGz2AcyEQDRvcQDoO/8MKzjtsw+Z1u8k7adH9rCFOg6h851Yqch4tXJFe2Tag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wk82CmnnafP/ty0a2WVhVTjambpmPt6exWQt3tANaj0=;
 b=jVL+PmIrq6rPiXFCfGQU9G0JzM+qL596OdyvWC1w+63KXndIHVb3pH66by2GKO1RvG+KKKbkDxMGjGpgNfM/vkbUX35Z4cmLRTku34zxCHJqcT5ROvDTNS4ngQeXrC6LJ4Vxp9hYNVLM+VPtPtdZXfRYzD7dAqBLibvw/SBoEZA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>, <linux-kernel@vger.kernel.org>
CC: Henry Wang <xin.wang2@amd.com>, Juergen Gross <jgross@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Michal Orzel <michal.orzel@amd.com>
Subject: [PATCH v2] drivers/xen: Improve the late XenStore init protocol
Date: Fri, 17 May 2024 09:15:16 +0800
Message-ID: <20240517011516.1451087-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF00021F69:EE_|DM6PR12MB4121:EE_
X-MS-Office365-Filtering-Correlation-Id: 97b12dc7-9b4b-420f-c37a-08dc760ed603
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?2ljjx2ZimPn2IMlx+2PP1wKTUhKyPacOVvnUwWjFenJbZKCDzp+7hPr2aoB/?=
 =?us-ascii?Q?pKvvno9m0BdNAe60rkwiH0O2gT73KyEmmYwfOAj/oEHnfaXMglPySCpM4Ylv?=
 =?us-ascii?Q?rnk2Sfc6gfzpoFh5AYXfYZeKFLvGLBp6banem0zCZzwqnrUeN7hZeWe330LM?=
 =?us-ascii?Q?G5X686EEVajH3XYXe3Ce/Ze2TkYsX+vSzP5THBHzgCv+Fz1y50uMj7hH8tAm?=
 =?us-ascii?Q?tNhQIM8PO3iN2vBEwQJy9szGowbpgDU6uu3ibWStlQmzWYxDWv9QUQkh+HLL?=
 =?us-ascii?Q?nr3QAWKRTqeZ1ZvXtGzyjZ/ANERGpB/2TXOktE1NIitNwPKq0KtvBlCsbrHY?=
 =?us-ascii?Q?id0DZQsM4ojZbEMTfW/bi4qxtC4j5S3Fn2PjfYqqmmGUcAX2nONZExh8WBes?=
 =?us-ascii?Q?Ax5geCVTyDcM68W+gvAAON9hkIpneZuc11KUMEXZWeBeezLJGuVZsnlgALHm?=
 =?us-ascii?Q?W/leEv6TTI+C5o5inTH3DpUFASIb7nqttc0qaZQGp9M57sujnvl0KO2cIzSt?=
 =?us-ascii?Q?HW9tOlSwh19UrbBD1lH9he1T8iOHs+6k5OjMrLXTVu0b/+CheGiDMxRTKF3v?=
 =?us-ascii?Q?R2Ftbw22dOQHZsjKVuAi0inac4VwJ1qUNBjf2sobkyC5XUjFDTg2BCG6yqZv?=
 =?us-ascii?Q?2syqpKO8qIQf1Xk10Zo/poDm577NfY3fqSR60ChbKLiUPgw2ZtREpjXAVgLM?=
 =?us-ascii?Q?1ks2aaHVX0i6oHy1xaieE2m5Gf7GuHjx3tnDOZJtD7zgrjDOd9TpVNraQoqW?=
 =?us-ascii?Q?LHpK5CotRY9bA1FHF9CYfZPqlixEIgI0Hz3pgMeQWc7xNDoAv87GimulGKM4?=
 =?us-ascii?Q?VqzWeKa7GzSr33C+FKSIQXrgRqkyJRnAFqq4CZtjpP8o9MQs59rIMkPmhZWT?=
 =?us-ascii?Q?kSE/jzWrlYKPt246YXfp7OmuASLiJvbARTUjhM+BOLbexPqMqQIqjvG8m7jz?=
 =?us-ascii?Q?3fMJqEcOqaiALneWb2m7mDiuCXsCbJK39OqA8yqANFqca3lBBjQNTKEyWT42?=
 =?us-ascii?Q?OXAnIe+YgzAJI1jgKpzfgp03VDaFF66YlKUkSC//PSEwI0xfKE5V4ixUajlT?=
 =?us-ascii?Q?JK5BDH+RUT4msLIXN194GEeRJcHVhGnH2q9pe/i2QbKApa0YLLVLtJx85oPa?=
 =?us-ascii?Q?DxBkJbG+1GJglo1yNkWF3hMEUJ6ybRtof6UIkSrHIsgzOGxn3DPSi4ea4jaW?=
 =?us-ascii?Q?BjR5g4xsrYwsUCeugtVEAJqiR/126tlduSFzxAxmr67HNyGVl1DTZGIPYJKE?=
 =?us-ascii?Q?rDVgtF0AETl7AhkzhAu4Wi3oKWr/NQCVmdqdqz8a6+/fjRL7o5H659YLZMmK?=
 =?us-ascii?Q?hu9Gr5oHnw5bNfGV/Zmtgm4rUQt1iVZ8Vr0rHyb/KBAKFDE9eCm9126RLqut?=
 =?us-ascii?Q?DILRIKVrajZvo6YTdiUZX+iTa5FO?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 01:15:27.1489
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 97b12dc7-9b4b-420f-c37a-08dc760ed603
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF00021F69.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4121

Currently, the late XenStore init protocol is only triggered properly
for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
case that XenStore interface is allocated but not ready (the connection
status is not XENSTORE_CONNECTED), Linux should also wait until the
XenStore is set up properly.

Introduce a macro to describe the XenStore interface is ready, use
it in xenbus_probe_initcall() to select the code path of doing the
late XenStore init protocol or not. Since now we have more than one
condition for XenStore late init, rework the check in xenbus_probe()
for the free_irq().

Take the opportunity to enhance the check of the allocated XenStore
interface can be properly mapped, and return error early if the
memremap() fails.

Fixes: 5b3353949e89 ("xen: add support for initializing xenstore later as HVM domain")
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
v2:
- Use -EINVAL for the memremap() check. (Stefano)
- Add Fixes: tag. (Stefano)
- Rework the condition for free_irq() in xenbus_probe(). (Stefano)
---
 drivers/xen/xenbus/xenbus_probe.c | 36 ++++++++++++++++++++-----------
 1 file changed, 23 insertions(+), 13 deletions(-)

diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index 3205e5d724c8..1a9ded0cddcb 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -65,13 +65,17 @@
 #include "xenbus.h"
 
 
-static int xs_init_irq;
+static int xs_init_irq = -1;
 int xen_store_evtchn;
 EXPORT_SYMBOL_GPL(xen_store_evtchn);
 
 struct xenstore_domain_interface *xen_store_interface;
 EXPORT_SYMBOL_GPL(xen_store_interface);
 
+#define XS_INTERFACE_READY \
+	((xen_store_interface != NULL) && \
+	 (xen_store_interface->connection == XENSTORE_CONNECTED))
+
 enum xenstore_init xen_store_domain_type;
 EXPORT_SYMBOL_GPL(xen_store_domain_type);
 
@@ -751,19 +755,19 @@ static void xenbus_probe(void)
 {
 	xenstored_ready = 1;
 
-	if (!xen_store_interface) {
+	if (!xen_store_interface)
 		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
 					       XEN_PAGE_SIZE, MEMREMAP_WB);
-		/*
-		 * Now it is safe to free the IRQ used for xenstore late
-		 * initialization. No need to unbind: it is about to be
-		 * bound again from xb_init_comms. Note that calling
-		 * unbind_from_irqhandler now would result in xen_evtchn_close()
-		 * being called and the event channel not being enabled again
-		 * afterwards, resulting in missed event notifications.
-		 */
+	/*
+	 * Now it is safe to free the IRQ used for xenstore late
+	 * initialization. No need to unbind: it is about to be
+	 * bound again from xb_init_comms. Note that calling
+	 * unbind_from_irqhandler now would result in xen_evtchn_close()
+	 * being called and the event channel not being enabled again
+	 * afterwards, resulting in missed event notifications.
+	 */
+	if (xs_init_irq >= 0)
 		free_irq(xs_init_irq, &xb_waitq);
-	}
 
 	/*
 	 * In the HVM case, xenbus_init() deferred its call to
@@ -822,7 +826,7 @@ static int __init xenbus_probe_initcall(void)
 	if (xen_store_domain_type == XS_PV ||
 	    (xen_store_domain_type == XS_HVM &&
 	     !xs_hvm_defer_init_for_callback() &&
-	     xen_store_interface != NULL))
+	     XS_INTERFACE_READY))
 		xenbus_probe();
 
 	/*
@@ -831,7 +835,7 @@ static int __init xenbus_probe_initcall(void)
 	 * started, then probe.  It will be triggered when communication
 	 * starts happening, by waiting on xb_waitq.
 	 */
-	if (xen_store_domain_type == XS_LOCAL || xen_store_interface == NULL) {
+	if (xen_store_domain_type == XS_LOCAL || !XS_INTERFACE_READY) {
 		struct task_struct *probe_task;
 
 		probe_task = kthread_run(xenbus_probe_thread, NULL,
@@ -1014,6 +1018,12 @@ static int __init xenbus_init(void)
 			xen_store_interface =
 				memremap(xen_store_gfn << XEN_PAGE_SHIFT,
 					 XEN_PAGE_SIZE, MEMREMAP_WB);
+			if (!xen_store_interface) {
+				pr_err("%s: cannot map HVM_PARAM_STORE_PFN=%llx\n",
+				       __func__, v);
+				err = -EINVAL;
+				goto out_error;
+			}
 			if (xen_store_interface->connection != XENSTORE_CONNECTED)
 				wait = true;
 		}
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 01:17:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:17:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723733.1128775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mEN-0003ee-3X; Fri, 17 May 2024 01:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723733.1128775; Fri, 17 May 2024 01:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mEN-0003eX-0j; Fri, 17 May 2024 01:17:55 +0000
Received: by outflank-mailman (input) for mailman id 723733;
 Fri, 17 May 2024 01:17:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7mEL-0003eP-H7
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:17:53 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 46efe2cc-13eb-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 03:17:51 +0200 (CEST)
Received: from BN9PR03CA0709.namprd03.prod.outlook.com (2603:10b6:408:ef::24)
 by MW5PR12MB5599.namprd12.prod.outlook.com (2603:10b6:303:194::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Fri, 17 May
 2024 01:17:47 +0000
Received: from BN3PEPF0000B074.namprd04.prod.outlook.com
 (2603:10b6:408:ef:cafe::68) by BN9PR03CA0709.outlook.office365.com
 (2603:10b6:408:ef::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Fri, 17 May 2024 01:17:47 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN3PEPF0000B074.mail.protection.outlook.com (10.167.243.119) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 01:17:46 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 20:17:46 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 20:17:44 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46efe2cc-13eb-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Bs/Q2jv0yG8Cn7b+ptGujytF/rTmER1KRPEPyDGmxgmaNf8utFBfLzP+zhX6knag0xmb5L/cZ0xVIp5z15HKxC5uZuaZd2R5PhVPHx3ztQNLSG/tnWmUQQAV3qHQLnmwFPUGqJvB3MXENzoqro9I6JkYkcAfo8GZwSR9SJHPk2wcV8I0KFw+lB0cihQyc0jqhMZUSCiKryeHmFK93dRBYy48NsfY5jlermTP5lP4HjhlsCeJyDYVq1lUVRIy9mmhpea1vUn4w+ecZ0Sdznie/KYVwPz86L50tI5Nxl7gynSP1VthMpbKgjPkvUWaBjqUhNP664R+GzrkYycVKioe8Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Kp85KC0psuXALGt1cPe+mlXd7lcMqilrHi8mw7kjljQ=;
 b=PeEQBbpWtWNRLigApQDsH77U5eonflQ7bbYX++NCiWED+FRfsC/1OpNdA2DyQ4zPVVNPwgtw/Kt6vqkXRlV7RsXA+0f/Kn+SuXydLXTDYQxzVo56pbmFkc7hoiSH7YC8AH1CjhUOrIfA70tsZKmwFq6+4kpc5G0KuTXYwP9oort+RRdXOHxmOIK+zqDGul4hTbND/WpAitWW+1n5nd0xgqaJBbJVqgjLwwRGNLzitw8Cx5isSiSUUFWEK4/k5x8hWj+m3dMw1lo+JyT4JHv/rwAcf0tyGwqz17gHbv8OAdHyVoI4KV8ezAQm4rph49dccgir4zLiTfPM9+3mtFK2gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Kp85KC0psuXALGt1cPe+mlXd7lcMqilrHi8mw7kjljQ=;
 b=PCBlgm/JytrDJ/4YOM67Mu/tTNpBvhOpYuiLekWU15SMbdKp/HWRyilW/kj8uvHjISrKw2fQruigUbpY4orYgncI8rp/YpKcLOoLP+AcC+nteKWzdPmOE8kbYzzU/uaQUzV7s5c2WfDDFPi8gweGyL5RkjT2dxuPue0zMrkpEVE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <1539e7ab-f27f-4cea-bb79-4c341bb3c69c@amd.com>
Date: Fri, 17 May 2024 09:17:43 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] drivers/xen: Improve the late XenStore init protocol
To: Stefano Stabellini <sstabellini@kernel.org>
CC: <xen-devel@lists.xenproject.org>, <linux-kernel@vger.kernel.org>, "Juergen
 Gross" <jgross@suse.com>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Michal Orzel <michal.orzel@amd.com>
References: <20240515014330.1044617-1-xin.wang2@amd.com>
 <alpine.DEB.2.22.394.2405151524270.2544314@ubuntu-linux-20-04-desktop>
 <028f29be-0393-4a57-83e2-ea27fe0320d5@amd.com>
 <alpine.DEB.2.22.394.2405161743170.2544314@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2405161743170.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B074:EE_|MW5PR12MB5599:EE_
X-MS-Office365-Filtering-Correlation-Id: 2b122ba2-0dff-4fa6-5dea-08dc760f293e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?TG9NY0lrajJjZ2Ezb1IyV1dzdDJBY2dCbTA5T1llckVzSnJjb01HSlNCcVAy?=
 =?utf-8?B?d2dCMUNKMTk0WS9ZS2RMenpkNzl4dUNaNEZ5U2JYU1ZkamZrQnVQcmZqako5?=
 =?utf-8?B?M05EWHdpbm52YnUrWDNYbkRrVmhZS09lNVlhZ3htTHltWFhJUGhrS1Z4cVlH?=
 =?utf-8?B?NGt6dTVyWXNndDZOdzZkWGw2TlVpcGVDblRLNHhqelloOVpJNjdRS0ZrYkVR?=
 =?utf-8?B?dk5rellRbnlYSmlhZE96R2orL3VINjZ0V1pXNS9Ra1llMFFQOStzdll6eGRG?=
 =?utf-8?B?bzlDd0pzN05zR0U5c1h5bG5LNVBRWGd0c3RORE8zdS9Wb0s2bndwZHhpV3RL?=
 =?utf-8?B?Z2xPNzI0V2lsR0pxYUUvVC8xOWJ4Y01EL3JGWDF2M3VMZzVxRVFxemlJQkcv?=
 =?utf-8?B?RHF4TFR6Z0dSMXdJUDhNTVdOOG5CbjNtVzNyTVpqcU1FaXp0d3Q5KzBpQTNm?=
 =?utf-8?B?eElidTFIUmw3bjArbUtHNTQ0QklZQUFrbkxLMk9tMEY4Mmxnb1JvQlZZeC9a?=
 =?utf-8?B?VjhpRVBpc2h0emtFTVhzUk5VTW1meWFUdURCNEY2TGNYUThNWGx5dVJNYlNq?=
 =?utf-8?B?S0NhZ0xETU52Z3pmcGI2NUxHZkdBOThLOWI1MlM4ajVtMjhnVUs3Y3lkRmE4?=
 =?utf-8?B?STYreHpQSEhkQzZiWmt4OWlobkVYK3BqblB1ZGNJSkJabWlvdjFTTWcxWDRt?=
 =?utf-8?B?QUgremlxanBNaEFvVHVzOUxuYkEvWGZRME9YYy9LUzJBZzRqWmtnbWJKOU12?=
 =?utf-8?B?WlBpNkh6QUF3UStzclgrL1l5UHdqK2g0andydVhJNGJLQnV1RkFNOUlmWU1N?=
 =?utf-8?B?aTFwSGw5ajhKaW5Oc1VlZXJ0aCtNeC9DMklOalhMUU1tYXBRNGduVTBPd2Fz?=
 =?utf-8?B?UVdrSnpYYXhQRGlWYUhDQXVLOElrTVdnYWpKZ2pwYkVrQlhzSzQ4TThTclJi?=
 =?utf-8?B?a0RxYWVOWHFPdVVXVU1QazNuVktlRzJCKzZqRW1TS1k0Q00rb2ZVVEtFYU1p?=
 =?utf-8?B?Szd6bUVoVUp4NVNhWitYMThEcWdWd1JaRGlmc1h2ZEZNcENKWXl6NTk4b0Zi?=
 =?utf-8?B?dE0zNlB1ZEJRTnJUb2ZrSW5WaklyTDFqSGQrTDM0aWJWZzZhKzJTa0ZRcEMw?=
 =?utf-8?B?RVljbXkrbXBYbVgyU0NHbG1Bc3VvY05RdFZjOS9qdTRrbGl5ZGd6UU5CcGUv?=
 =?utf-8?B?VmR5TVlvR0hFaEdqUkdnNGRkWm5ueENHbEkvQS9HWG4xYVAxTWtzUlo2T3A5?=
 =?utf-8?B?NjdEVHBRQk91dEM5NjF0WUdrRUl2YjNMK2FoUmpxdFcxY2lTYkR2QXQ4dDdw?=
 =?utf-8?B?WXRzNmRMU1pzU3gyS1B2ZDgzeUJvdWd2dWpXSzN6VDY2VFdBdnE3VjRqTFM1?=
 =?utf-8?B?SWxod2syZHFIQWExWGZFdXdsencrWThoNXVMYllsejExdFViN0hid0ZJT2Vw?=
 =?utf-8?B?Z0MrMmdZTy9ZZVhVbG5XT1Z4R0E4eVdOYVArT1BINi95K2xDTTNxWkF2dC9p?=
 =?utf-8?B?VWFUWEZJQjVNTU5zaDl2L3UyUXcvTnpsOERGREtXeS9jRDRlNy9RM1RaSXg0?=
 =?utf-8?B?SnV6Q2xPQS85REl0VkkvSzhoNTkzZjBYeU4zbzIrZEQ3WkFqUWNobmtURjN1?=
 =?utf-8?B?RklJRWJMeFlDdzMxVFhNbTArYnZraHBab2ZubjBIaUlVdE1tYzVFUmtsdkdu?=
 =?utf-8?B?R2FkUnJObHkrU20wTjM0WW9hVlJhN1c0UjQ1M0hKV2lkcGQ4aVlncGtqUUU2?=
 =?utf-8?B?bXdaa2U1M2pFWnlzdXNUWGs3RkhsZGJ5SU1QeVliSG5PU0hRQm5yYlFZVnBl?=
 =?utf-8?B?Ump0ajlHdnVFcVNBUmxvaGVUd3dGS2tONDNSMWZOVE0xRG1iN2tKTHIxQzcw?=
 =?utf-8?Q?UG2kNWVV4kE6b?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 01:17:46.7839
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b122ba2-0dff-4fa6-5dea-08dc760f293e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B074.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5599

Hi Stefano,

On 5/17/2024 8:52 AM, Stefano Stabellini wrote:
> On Thu, 16 May 2024, Henry Wang wrote:
>>>>    enum xenstore_init xen_store_domain_type;
>>>>    EXPORT_SYMBOL_GPL(xen_store_domain_type);
>>>>    @@ -751,9 +755,10 @@ static void xenbus_probe(void)
>>>>    {
>>>>    	xenstored_ready = 1;
>>>>    -	if (!xen_store_interface) {
>>>> -		xen_store_interface = memremap(xen_store_gfn <<
>>>> XEN_PAGE_SHIFT,
>>>> -					       XEN_PAGE_SIZE, MEMREMAP_WB);
>>>> +	if (!xen_store_interface || XS_INTERFACE_READY) {
>>>> +		if (!xen_store_interface)
>>> These two nested if's don't make sense to me. If XS_INTERFACE_READY
>>> succeeds, it means that  ((xen_store_interface != NULL) &&
>>> (xen_store_interface->connection == XENSTORE_CONNECTED)).
>>>
>>> So it is not possible that xen_store_interface == NULL immediately
>>> after. Right?
>> I think this is because we want to free the irq for the late init case,
>> otherwise the init-dom0less will fail. For the xenstore PFN allocated case,
>> the connection is already set to CONNECTED when we execute init-dom0less. But
>> I agree with you, would below diff makes more sense to you?
>>
>> diff --git a/drivers/xen/xenbus/xenbus_probe.c
>> b/drivers/xen/xenbus/xenbus_probe.c
>> index 8aec0ed1d047..b8005b651a29 100644
>> --- a/drivers/xen/xenbus/xenbus_probe.c
>> +++ b/drivers/xen/xenbus/xenbus_probe.c
>> @@ -76,6 +76,8 @@ EXPORT_SYMBOL_GPL(xen_store_interface);
>>          ((xen_store_interface != NULL) && \
>>           (xen_store_interface->connection == XENSTORE_CONNECTED))
>>
>> +static bool xs_late_init = false;
>> +
>>   enum xenstore_init xen_store_domain_type;
>>   EXPORT_SYMBOL_GPL(xen_store_domain_type);
>>
>> @@ -755,7 +757,7 @@ static void xenbus_probe(void)
>>   {
>>          xenstored_ready = 1;
>>
>> -       if (!xen_store_interface || XS_INTERFACE_READY) {
>> +       if (xs_late_init) {
>>                  if (!xen_store_interface)
>>                          xen_store_interface = memremap(xen_store_gfn <<
>
> I would just remove the outer 'if' and do this:
>
>
> 	if (!xen_store_interface)
> 		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
> 				XEN_PAGE_SIZE, MEMREMAP_WB);
> 	/*
> 	 * Now it is safe to free the IRQ used for xenstore late
> 	 * initialization. No need to unbind: it is about to be
> 	 * bound again from xb_init_comms. Note that calling
> 	 * unbind_from_irqhandler now would result in xen_evtchn_close()
> 	 * being called and the event channel not being enabled again
> 	 * afterwards, resulting in missed event notifications.
> 	 */
> 	if (xs_init_irq > 0)
> 		free_irq(xs_init_irq, &xb_waitq);
>
>
> I think this should work fine in all cases.

Thanks. I followed your suggestion in v2.

>   I am unsure if
> xs_init_irq==0 is possible valid value for xs_init_irq. If it is not,
> then we are fine. If 0 is a possible valid irq number, then we should
> initialize xs_init_irq to -1, and here check for xs_init_irq >= 0.

Yeah the xs_init_irq==0 is a valid value. I followed your latter comment 
to init it to -1 and check it >=0.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Fri May 17 01:21:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:21:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723736.1128785 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mHk-0005Wr-Ir; Fri, 17 May 2024 01:21:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723736.1128785; Fri, 17 May 2024 01:21:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mHk-0005Wk-Ez; Fri, 17 May 2024 01:21:24 +0000
Received: by outflank-mailman (input) for mailman id 723736;
 Fri, 17 May 2024 01:21:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7mHi-0005We-To
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:21:22 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c481a277-13eb-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 03:21:21 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 1CE5761776;
 Fri, 17 May 2024 01:21:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE404C113CC;
 Fri, 17 May 2024 01:21:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c481a277-13eb-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715908879;
	bh=Zi382DsT9ynimDIZGqjpN+RCzEfNhUvXJn7moYDmWNA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DkbJ6V2+sqbaP6eFh859w2TQFMizovr4bES0ZsieQ/qFmKpIQ5XDTReLXTKqC+Ypu
	 b3CTo9qT/Gh0L5QoqUnxFWxxDG8nvn6yR9/fHcpl4mwDihyoKkAlUNUsONBm9IDCKH
	 m2mSNp048JPQN6HnIYPTn8TgQOJW3BE0cHmqkPrs1nEcETSkw+hjEa5ioF5ow3xYQy
	 0ZC39AkCshzTGVBv5HL9DhaLmZuR2RfIFOA+bEmouprF5Rz/Uk+f/cy2eE1WymAWMC
	 nVRwywUk/0znyOOvkRTfhgK1td1emCKCJP5ivGsUv56Nuy3uFQy/r1L2xU4fpE/jLd
	 bVV8YRagdL8aw==
Date: Thu, 16 May 2024 18:21:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <George.Dunlap@citrix.com>, 
    Daniel Smith <dpsmith@apertussolutions.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
In-Reply-To: <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com> <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com> <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 May 2024, Jan Beulich wrote:
> 1) In the discussion George claimed that exposing status information in
> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
> how a similar assumption by CPU designers has led to a flood of
> vulnerabilities over the last 6+ years. Information exposure imo is never
> okay, unless it can be _proven_ that absolutely nothing "useful" can be
> inferred from it. (I'm having difficulty seeing how such a proof might
> look like.)

Many would agree that it is better not to expose status information in
an uncontrolled manner. Anyway, let's focus on the actionable.


> 2) Me pointing out that the XSM hook might similarly get in the way of
> debugging, Andrew suggested that this is not an issue because any sensible
> XSM policy used in such an environment would grant sufficient privilege to
> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
> for Xen-internal event channels. The debugging argument then becomes weak,
> as in that case the XSM hook is possibly going to get in the way.
>
> 3) In the discussion Andrew further gave the impression that evtchn_send()
> had no XSM check. Yet it has; the difference to evtchn_status() is that
> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
> similarly useful to allow getting a stuck channel unstuck.)
> 
> In summary I continue to think that an outright revert was inappropriate.
> DomU-s should continue to be denied status information on Xen-internal
> event channels, unconditionally and independent of whether dummy, silo, or
> Flask is in use.

I think DomU-s should continue to be denied status information on
Xen-internal event channels *based on the default dummy, silo, or Flask
policy*. It is not up to us to decide the security policy, only to
enforce it and provide sensible defaults.

In any case, the XSM_TARGET check in evtchn_status seems to do what we
want?

evtchn_send uses XSM_HOOK, which is weaker, but it doesn't seem to be an
issue because (ignoring the consumer_is_xen check) there is a if(!lchn)
check that would fail on invalid event channels?


> Plus, as stated before, evtchn_send() would better remain in sync in this
> regard with evtchn_status(). The situation is less clear for evtchn_close()
> and evtchn_bind_vcpu(): Those indeed have no XSM checks while they do deny
> operation on Xen-internal channels. It is likely more far fetched to
> assume permitting them for debugging purposes might in fact help in rare
> situations. Yet it may still be a matter of consistency to keep them in
> sync with the other two. (Note that there's also evtchn_usable(), which
> might then also need tweaking itself or at the use sites.)
> 
> FTR, it is going to be only then that I would consider the cumulative
> result as eligible for backporting. For this purpose, at the risk of
> being flamed again, it might still be easier to revert the revert and then
> put in place a change meeting the above criteria. That could then be taken
> for backport as is.

I think we want to keep the revert because we need to unbreak lsevtchn.c.
Your point about consistency is valid and it would be good to go in that
direction but to me it is not the kind of thing that we would make
release-blocking.


From xen-devel-bounces@lists.xenproject.org Fri May 17 01:22:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:22:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723740.1128795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mIW-00066V-TX; Fri, 17 May 2024 01:22:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723740.1128795; Fri, 17 May 2024 01:22:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mIW-00066O-Qp; Fri, 17 May 2024 01:22:12 +0000
Received: by outflank-mailman (input) for mailman id 723740;
 Fri, 17 May 2024 01:22:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s7mIW-000644-6x
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:22:12 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id de7f6c19-13eb-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 03:22:10 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 62F2FCE19A1;
 Fri, 17 May 2024 01:21:59 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC266C113CC;
 Fri, 17 May 2024 01:21:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de7f6c19-13eb-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715908918;
	bh=/+69Lf2vDX6gQntQK69tY9Tw2kBCjXsZWh6uIw2z2+U=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=r3ModDtGISt7KGEtcdnrchRxmpJyTWN+T77c9M9MSXgNhqXLH8SVghfzsHCafFlCW
	 /ZeuCQb6sIYhnTfBHsbdXD7LhIS4m5p4GZBup/74EW8zjXS2FEu5HJ1Itj67uJn5N0
	 /8lqOxRgWaq480aacBo0ptQgOxZ4EWDAFoIhmpYpqVmw6VuQy+D3TrtGrjKPiSXWue
	 m5ILlc8RZPTn84+hUSL7bc90XffLGR8KSlj1R/rN3OwzKYtUiypU/hablIeIkBg2Bt
	 EUzd8SCUKtNIXcxWf16nfYF5qYEfHF/L4gnJQCEN5yXNLtrArog+9xYmC/Mh6RE6sR
	 GtVf/nbHM6t7Q==
Date: Thu, 16 May 2024 18:21:55 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH v2] drivers/xen: Improve the late XenStore init
 protocol
In-Reply-To: <20240517011516.1451087-1-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405161821240.2544314@ubuntu-linux-20-04-desktop>
References: <20240517011516.1451087-1-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 May 2024, Henry Wang wrote:
> Currently, the late XenStore init protocol is only triggered properly
> for the case that HVM_PARAM_STORE_PFN is ~0ULL (invalid). For the
> case that XenStore interface is allocated but not ready (the connection
> status is not XENSTORE_CONNECTED), Linux should also wait until the
> XenStore is set up properly.
> 
> Introduce a macro to describe the XenStore interface is ready, use
> it in xenbus_probe_initcall() to select the code path of doing the
> late XenStore init protocol or not. Since now we have more than one
> condition for XenStore late init, rework the check in xenbus_probe()
> for the free_irq().
> 
> Take the opportunity to enhance the check of the allocated XenStore
> interface can be properly mapped, and return error early if the
> memremap() fails.
> 
> Fixes: 5b3353949e89 ("xen: add support for initializing xenstore later as HVM domain")
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> v2:
> - Use -EINVAL for the memremap() check. (Stefano)
> - Add Fixes: tag. (Stefano)
> - Rework the condition for free_irq() in xenbus_probe(). (Stefano)
> ---
>  drivers/xen/xenbus/xenbus_probe.c | 36 ++++++++++++++++++++-----------
>  1 file changed, 23 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
> index 3205e5d724c8..1a9ded0cddcb 100644
> --- a/drivers/xen/xenbus/xenbus_probe.c
> +++ b/drivers/xen/xenbus/xenbus_probe.c
> @@ -65,13 +65,17 @@
>  #include "xenbus.h"
>  
>  
> -static int xs_init_irq;
> +static int xs_init_irq = -1;
>  int xen_store_evtchn;
>  EXPORT_SYMBOL_GPL(xen_store_evtchn);
>  
>  struct xenstore_domain_interface *xen_store_interface;
>  EXPORT_SYMBOL_GPL(xen_store_interface);
>  
> +#define XS_INTERFACE_READY \
> +	((xen_store_interface != NULL) && \
> +	 (xen_store_interface->connection == XENSTORE_CONNECTED))
> +
>  enum xenstore_init xen_store_domain_type;
>  EXPORT_SYMBOL_GPL(xen_store_domain_type);
>  
> @@ -751,19 +755,19 @@ static void xenbus_probe(void)
>  {
>  	xenstored_ready = 1;
>  
> -	if (!xen_store_interface) {
> +	if (!xen_store_interface)
>  		xen_store_interface = memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>  					       XEN_PAGE_SIZE, MEMREMAP_WB);
> -		/*
> -		 * Now it is safe to free the IRQ used for xenstore late
> -		 * initialization. No need to unbind: it is about to be
> -		 * bound again from xb_init_comms. Note that calling
> -		 * unbind_from_irqhandler now would result in xen_evtchn_close()
> -		 * being called and the event channel not being enabled again
> -		 * afterwards, resulting in missed event notifications.
> -		 */
> +	/*
> +	 * Now it is safe to free the IRQ used for xenstore late
> +	 * initialization. No need to unbind: it is about to be
> +	 * bound again from xb_init_comms. Note that calling
> +	 * unbind_from_irqhandler now would result in xen_evtchn_close()
> +	 * being called and the event channel not being enabled again
> +	 * afterwards, resulting in missed event notifications.
> +	 */
> +	if (xs_init_irq >= 0)
>  		free_irq(xs_init_irq, &xb_waitq);
> -	}
>  
>  	/*
>  	 * In the HVM case, xenbus_init() deferred its call to
> @@ -822,7 +826,7 @@ static int __init xenbus_probe_initcall(void)
>  	if (xen_store_domain_type == XS_PV ||
>  	    (xen_store_domain_type == XS_HVM &&
>  	     !xs_hvm_defer_init_for_callback() &&
> -	     xen_store_interface != NULL))
> +	     XS_INTERFACE_READY))
>  		xenbus_probe();
>  
>  	/*
> @@ -831,7 +835,7 @@ static int __init xenbus_probe_initcall(void)
>  	 * started, then probe.  It will be triggered when communication
>  	 * starts happening, by waiting on xb_waitq.
>  	 */
> -	if (xen_store_domain_type == XS_LOCAL || xen_store_interface == NULL) {
> +	if (xen_store_domain_type == XS_LOCAL || !XS_INTERFACE_READY) {
>  		struct task_struct *probe_task;
>  
>  		probe_task = kthread_run(xenbus_probe_thread, NULL,
> @@ -1014,6 +1018,12 @@ static int __init xenbus_init(void)
>  			xen_store_interface =
>  				memremap(xen_store_gfn << XEN_PAGE_SHIFT,
>  					 XEN_PAGE_SIZE, MEMREMAP_WB);
> +			if (!xen_store_interface) {
> +				pr_err("%s: cannot map HVM_PARAM_STORE_PFN=%llx\n",
> +				       __func__, v);
> +				err = -EINVAL;
> +				goto out_error;
> +			}
>  			if (xen_store_interface->connection != XENSTORE_CONNECTED)
>  				wait = true;
>  		}
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 17 01:22:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:22:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723741.1128805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mIc-0006NL-3u; Fri, 17 May 2024 01:22:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723741.1128805; Fri, 17 May 2024 01:22:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mIc-0006NE-0s; Fri, 17 May 2024 01:22:18 +0000
Received: by outflank-mailman (input) for mailman id 723741;
 Fri, 17 May 2024 01:22:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y5lu=MU=apertussolutions.com=dpsmith@srs-se1.protection.inumbo.net>)
 id 1s7mIb-0005xt-1v
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:22:17 +0000
Received: from sender3-of-o58.zoho.com (sender3-of-o58.zoho.com
 [136.143.184.58]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e3d10a20-13eb-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 03:22:14 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715908929895770.4070882730078;
 Thu, 16 May 2024 18:22:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3d10a20-13eb-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; t=1715908931; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=g9hOY0sUkPWdFO0zo052wg8MMKbsy7k+eOdgd20H6Ty+wa0fW5d7kSFSu/5ohtiGVJyJiP62+Aw5Mxrnn/a0QHNu7yF42ckWpq0IhAP8O2jB9GDH2McQ+m4xWKtJ2BmPT800B0KFzuu1UWWZR26FAxNKH5tUymSAJ1xL7kfVRFo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715908931; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=/ZP9OcrdM117h/0SaihIOKMG/tlbPUthFhAXt6FuLOg=; 
	b=g2QIPX+6q4W+QRdsbd8hX4eGprDPRgk0rz/VRYhdB8tQo4Ffhdl/cOhlPnyb45Z1Oues/TINDgzwtelHhxQ1kPoNJlHniL+lyEeIdIpTIsS0uaHSDv4MAroKQXGXV8cXe5bWCXEWm5XYCYWBEcEf4BZqmMYO3m0zwfJTx9V9Qy8=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715908931;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Message-ID:Date:Date:MIME-Version:Subject:Subject:To:To:Cc:Cc:References:From:From:In-Reply-To:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=/ZP9OcrdM117h/0SaihIOKMG/tlbPUthFhAXt6FuLOg=;
	b=nmFuiUWiPMrxFb6eVDp9FUWnJRAlTnveKU+/FiqmuTbhzMm//Zn31nTYRqJJ/7gp
	2/4gSPGlDoONZeKmlwUY1y6yPSuyoEk7y40zpdZLrLPk1m6mXh005iy8rArzs80e1aQ
	+40UCagB4c+Gn34UpzRejxBrx/ltbxckU+AaeVfQ=
Message-ID: <5059a536-2495-4121-8aba-5a2542ff9987@apertussolutions.com>
Date: Thu, 16 May 2024 21:22:02 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <George.Dunlap@citrix.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
Content-Language: en-US
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
In-Reply-To: <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External

On 5/16/24 02:41, Jan Beulich wrote:
> On 14.05.2024 11:25, Jan Beulich wrote:
>> On 03.04.2024 08:16, Jan Beulich wrote:
>>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>>> The commit makes a claim without any kind of justification.
>>>
>>> Well, what does "have no business" leave open?
>>>
>>>> The claim is false, and the commit broke lsevtchn in dom0.
>>>
>>> Or alternatively lsevtchn was doing something that was never meant to work
>>> (from Xen's perspective).
>>>
>>>>   It is also quite
>>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>>>
>>> Why would that be "obvious"? What business would a stubdom have to look at
>>> Xen's side of an evtchn?
>>>
>>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>>> and it's not acceptable to short circuit the XSM on the matter.
>>>
>>> I can certainly accept this as one possible view point. As in so many cases
>>> I'm afraid I dislike you putting it as if it was the only possible one.
>>>
>>> In summary: The supposed justification you claim is missing in the original
>>> change is imo also missing here then: What business would any entity in the
>>> system have to look at Xen's side of an event channel? Back at the time, 3
>>> people agreed that it's "none".
>>
>> You've never responded to this reply of mine, or its follow-up. You also
>> didn't chime in on the discussion Daniel and I were having. I consider my
>> objections unaddressed, and in fact I continue to consider the change to
>> be wrong. Therefore it was inappropriate for you to commit it; it needs
>> reverting asap. If you're not going to do so, I will.
> 
> For the record: With Andrew having clarified that in his revert's
> description:
> 
> "The claim is false; it broke lsevtchn in dom0, a debugging utility which
>   absolutely does care about all of the domain's event channels."
> 
> "all of the domain's event channels" means to include event channels Xen
> uses for its own, and merely puts in the domain's table for ease of
> handling, I've agreed that - in the absence of any better debugging
> means - the revert may stay in place. For context, my prior understanding
> of the issue was that lsevtchn simply stops probing further channels when
> getting back any kind of error from EVTCHNOP_status (which I continue to
> think wants addressing there, by a future version of a patch that was
> already sent). However, there are follow-on concerns I have:
> 
> 1) In the discussion George claimed that exposing status information in
> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
> how a similar assumption by CPU designers has led to a flood of
> vulnerabilities over the last 6+ years. Information exposure imo is never
> okay, unless it can be _proven_ that absolutely nothing "useful" can be
> inferred from it. (I'm having difficulty seeing how such a proof might
> look like.)

Jan, I would agree with you that any resources exposed to the guest, 
even if that resource is status information, should be behind an XSM 
check. I would, however, have to disagree the position over proving 
absolutely nothing "useful" can be inferred. Prior to spectre becoming 
commonly aware, no one considered proving there needed to be protections 
against out-of-order instruction execution being used to bypass branch 
conditions. Predicting the future will more often than not fail, but 
with an XSM check in place, the dummy/default policy can just be updated 
with the general safe decision and others can use FLASK to provide fine 
grain access.

> 2) Me pointing out that the XSM hook might similarly get in the way of
> debugging, Andrew suggested that this is not an issue because any sensible
> XSM policy used in such an environment would grant sufficient privilege to
> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
> for Xen-internal event channels. The debugging argument then becomes weak,
> as in that case the XSM hook is possibly going to get in the way.

And this is where we have a fundamental difference. Event channels are 
XSM labeled objects that are always connected to a guest. If they were 
truly Xen-internal, then a guest would have no way to get 
access/reference them. Again, I never disagreed with whether the guest 
should or should not be able to access them. I did ask for a better 
explanation than, "has no business", which is a statement of opinion not 
of fact or reason. The point is these event channels are a resource 
attached to the guest and access control decisions to them should be 
made under XSM. Injecting a hard-coded access control in front of the 
XSM check subverted/invalidated rules in the existing FLASK policy.

> 3) In the discussion Andrew further gave the impression that evtchn_send()
> had no XSM check. Yet it has; the difference to evtchn_status() is that
> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
> similarly useful to allow getting a stuck channel unstuck.)

A more appropriate default might be XSM_OTHER with conditions with in 
the hook  as to whether the policy should be XSM_HOOK, XSM_TARGET or 
flat denial.

> In summary I continue to think that an outright revert was inappropriate.
> DomU-s should continue to be denied status information on Xen-internal
> event channels, unconditionally and independent of whether dummy, silo, or
> Flask is in use.

Any guest facing resources, regardless if the backing end is the 
hypervisor, should be protected with XSM. This allows the maintainers to 
codify what they believe is a sane policy in the dummy policy and the 
end user can use FLASK to enforce what they believe is acceptable.

> Plus, as stated before, evtchn_send() would better remain in sync in this
> regard with evtchn_status(). The situation is less clear for evtchn_close()
> and evtchn_bind_vcpu(): Those indeed have no XSM checks while they do deny
> operation on Xen-internal channels. It is likely more far fetched to
> assume permitting them for debugging purposes might in fact help in rare
> situations. Yet it may still be a matter of consistency to keep them in
> sync with the other two. (Note that there's also evtchn_usable(), which
> might then also need tweaking itself or at the use sites.)

Just because that is how it was, doesn't mean it was correct. I had a 
discussion with one of the original authors of FLASK before taking up 
the maintainership and he felt there were likely more XSM checks that 
should have been in place originally. He considered it a first order 
approximation of what should be protected.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Fri May 17 01:36:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 01:36:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723757.1128815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mW8-0001Au-Ar; Fri, 17 May 2024 01:36:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723757.1128815; Fri, 17 May 2024 01:36:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7mW8-0001An-6y; Fri, 17 May 2024 01:36:16 +0000
Received: by outflank-mailman (input) for mailman id 723757;
 Fri, 17 May 2024 01:36:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7mW6-0001Ab-GJ
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 01:36:14 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d7eb2712-13ed-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 03:36:13 +0200 (CEST)
Received: from MW4PR03CA0309.namprd03.prod.outlook.com (2603:10b6:303:dd::14)
 by LV8PR12MB9183.namprd12.prod.outlook.com (2603:10b6:408:193::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 01:36:10 +0000
Received: from SJ5PEPF000001D2.namprd05.prod.outlook.com
 (2603:10b6:303:dd:cafe::45) by MW4PR03CA0309.outlook.office365.com
 (2603:10b6:303:dd::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Fri, 17 May 2024 01:36:09 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ5PEPF000001D2.mail.protection.outlook.com (10.167.242.54) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 01:36:09 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 20:36:08 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 20:36:08 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 20:36:06 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7eb2712-13ed-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gbaTTCM9d8O4r0uAjK8PDbO/zeR0kbf7AA1vioUO9b9Oxv9NmLHDQLimSxtKHhdQiJtcTK8VdCMzo6mXz6g9xKck4TGUnV9+Li431oEQ664c08SpArKdyx+854a55tIrPZ+Zqvqej2A9/T/1dVl92DELKhQ23RAbFGtpZjKMha5XYgry7DdGQM1Pn0F09gOkyTy6HMn8N+M6F/MtZD9Gs35iqmp1ETekouvLqbcdSDmRxdNaMWZYgrHEyt2rV7lJiNXGpyKu4DVJiMwKi7uKpjeJaKWiqZXr40sD1yRkr8sI5YHhGjU0vHEetNQVZojbicdsS20F+qE/TKYctcMHFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Q6J0+vyCm0AQ02DpB102APuTG9J+GZ8JwgNCnPIcxMs=;
 b=S4MoQnIHacoeimQc7yKTZEroqle/2bvopAQ3WO93SYvc5x3Jw6SxOhNRQScR02GlK/PO+KIK2O4fhvBFOvXriJdk+beZOAH0FBBWT+LZBF1ub84adS+wIFwVzsNOZqH3zd9elZMeQqejrW0ThZBF7WTDhq+yDOC/PzYpJ0zsxcju8PWqGRe/gIUdazj5xldJyygujCL2P0rgqpeBUBwJRlydFD80R14VXHR4LW2y2bbedC00k+5jy8blIYhizPRB3i4f2KuGdah1VJ880yRroMkXz4N+aQDCu3g6U4czS/w7z+xpZuOAAGZQHotQBqYQ4/xRYaAAo6e/e2FN0HQTQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q6J0+vyCm0AQ02DpB102APuTG9J+GZ8JwgNCnPIcxMs=;
 b=BfaT4qj2I/TN2Y4P7aELLJnV8giO5l0WHVRIeXDYChimeC2fIZJPAIA07v3yqsHTlH0W3mTOy+A8TDhITzG13R4wdLk0dzIXKVOriQK2kbRjWNFXCbZyxEe/susa73KTlXcGWz8sShKDzCK6lHhvdvvjz+fhlhCl8kPRfyI0+Ug=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <1f02615c-916c-4ab8-babc-17a117c56e87@amd.com>
Date: Fri, 17 May 2024 09:36:05 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and
 related operations
To: Jan Beulich <jbeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, <xen-devel@lists.xenproject.org>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-7-xin.wang2@amd.com>
 <28138e7d-26c7-4faa-86f1-a1c7b175114b@suse.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <28138e7d-26c7-4faa-86f1-a1c7b175114b@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D2:EE_|LV8PR12MB9183:EE_
X-MS-Office365-Filtering-Correlation-Id: d572f72d-a3b1-4ffd-7d5e-08dc7611ba5f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QXJPQXk0OFZjdFBReHZOeWR3UktSa29UMXNYRnByZDFzbUF0V0VhKzN2YkFt?=
 =?utf-8?B?Zm9GUjBZb0lvckhxK1dxSlFWM0RuOHBPeUE2VEgxSUJpelVlOFIwY2VSMWNJ?=
 =?utf-8?B?N2RKNExkR2pFL1BNZFZSK0UyeGovRXRqUFAyQ3B0Q1JIM285YlpsbU1Nc3I3?=
 =?utf-8?B?alFuR0gvdkZEVktmRjRuUUY3SlNyUm1xUEN4eEEvRTBmWGNoSnJPSWlSQkw1?=
 =?utf-8?B?RThEdVNYbFRYZnlwMWJ4QytOczRzNWF3YnVkNWh1dEMxR0x6bmszbklnWjYy?=
 =?utf-8?B?M3YzckhXejc4ZHdPQUtOUnplWVcwRTByLzMvYjRwbXgrQW5MNjdiZ1V1Q2oy?=
 =?utf-8?B?YzJvTjVlNWRJYkZnYkl1RE56d2hpZHJVVncwdTRIb3g0VUZEZTJ2cmhxQ2Fr?=
 =?utf-8?B?R21hVTE2RnlXWFVqNUpsS0t3TVQvcTVwWUFLQTR4endFWUt6aHQvK0k3eDhC?=
 =?utf-8?B?M1ZEbUV5UUNnK2ZZZlR3amhrNXAyYWZBKy9IMExMZ0krS0dpS2s2YWVHdDZY?=
 =?utf-8?B?Zzk4TnNIUDQyOVQxV2dsYkJUNkxPemQ4ZXc0bTk2OFl4NGNmTXgvRmZEaEI0?=
 =?utf-8?B?OWN6WHFiYUcxcldkekxTL3k5Y2FEQlhBZVB3ZGU5NXZFWTJJdmRlTXROY0Z2?=
 =?utf-8?B?VHZ2aEYyU2tYNXhMZHkvdWtHejd0RmltV0p2Rjd1cXkvMWhjaXhsNUhBR1ZO?=
 =?utf-8?B?c0lrUEdsdTRrRkNCRHdXQmdtY0ZaLy9GTzlSS2I2VkkrZlBnYUt6NUhzTHJa?=
 =?utf-8?B?a2NLcFhRZkJKcXBlTUd2Zm9wQWFJMDR4bjVkakMyZHNBdkNLVzBNaGlVWmx2?=
 =?utf-8?B?NXpVNDYxdHJPQjM3R3FOc09mQWowTTc5QUR4RW1zdlUydTZPMUppcFgrWTNp?=
 =?utf-8?B?UWQ1RldLWHhSdWNnWEpGdmNxa2hLdU12TzZwbjJqWjBVdHBQVGtrRUFveWEx?=
 =?utf-8?B?ejVnR1FDaVhuQXRzSEZXYmt0TTNHcDZEeEpCamlSY2ZHZXIwZDJNb0lIWUN5?=
 =?utf-8?B?OVFFSisrUXNQZ2pDZHF5ODhxb1pBdFBRS0JQOTM5SzdNT3dWVThnd2lRSUt4?=
 =?utf-8?B?aFowelVEVWhtQXJET3gzQkpYaEFyMEoxV25lV3JRbXQvdzc4OUV4YnA5QXc2?=
 =?utf-8?B?TW15M2o1anFQYmVoekptbVBxcGtCV1AzRWlYTHBCYUZSbkt3K2NmejVBNWR5?=
 =?utf-8?B?cDR1VlZNeTlLeVJGMmlhOFM3aTQrMHM4ZTJRdlVkNUZBWHE5SFVjMUJuRHFL?=
 =?utf-8?B?a2xjbVA4NFFOakljTVBJaTFkaitobkVEUXJjOGFyWkRyTUNVQmh4VFBSbnFJ?=
 =?utf-8?B?SWFUYUNCcnFHYjMvY1hqMWRKbjNuR0dJcllKVk41QkRLaDFMZUZlMzBGWWRv?=
 =?utf-8?B?WW9udVd2QlBuOVFWNjdqS1dmUSsrZVFNMFhralNzdTVXcElXMkdSVTB1cVBF?=
 =?utf-8?B?WGdNZERrQjVVUUx4ZzR2VkJBR1pUYk9oc1MyTjdBTXNiWU5acXFmTkRPS1ZK?=
 =?utf-8?B?Z3hqY2tVc3VRNHU4QnVxYS9iWXJKeUUxNnhYOEYyMjBZOHpvTmJpdTlBSUlX?=
 =?utf-8?B?aGpYUmpLdjRaemJ1THNacUFab1UrSGhSN1ZldEUxRGc4WElqZHF3Z0l6NG1u?=
 =?utf-8?B?b2cxWEtzYWdyRzdsYlBNVkQxMkJNaXlDa1ZUZENsOVhwaU52MWFDMkl1K3pr?=
 =?utf-8?B?aFZhaWE1Qi9vaDZucEgySE9CWnVBZWp1ZzBTblJWNEdzMWJ2cnc0eWtkelZR?=
 =?utf-8?B?Wnc3REN1eFg2cHBHVkNqaGhTRC9XV1ZzY2VDam5xY2RYelZlTDVPNWpCUDV3?=
 =?utf-8?B?R205ampMSmwrWlpLN2F6MlArTlNPL29Da0p0NlB1cjlKUmtJYTJMOC91SkZq?=
 =?utf-8?Q?+Td5h77YPY4Vo?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 01:36:09.1703
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d572f72d-a3b1-4ffd-7d5e-08dc7611ba5f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001D2.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9183

Hi Jan,

As usual, thanks for the review!

On 5/16/2024 8:31 PM, Jan Beulich wrote:
> On 16.05.2024 12:03, Henry Wang wrote:
>
> +    /*
> +     * First check if dtbo is correct i.e. it should one of the dtbo which was
> +     * used when dynamically adding the node.
> +     * Limitation: Cases with same node names but different property are not
> +     * supported currently. We are relying on user to provide the same dtbo
> +     * as it was used when adding the nodes.
> +     */
> +    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> +    {
> +        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
> +        {
> +            track = entry;
> Random question (not doing a full review of the DT code): What use is
> this (and the track variable itself)? It's never used further down afaics.
> Same for attach.

I think you are correct, it is a copy paste of the existing code and the 
track variable is indeed useless. So in v3, I will simply drop it and 
mention this clean-up in commit message. Also I realized that the exact 
logic of finding the entry is duplicated third times, so I will also 
extract the logic to a function.

>> --- a/xen/include/public/domctl.h
>> +++ b/xen/include/public/domctl.h
>> @@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op {
>>   typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
>>   DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
>>   
>> +#if defined(__arm__) || defined (__aarch64__)
> Nit: Consistent use of blanks please (also again below).

Good catch. Will fix it.

>> +struct xen_domctl_dt_overlay {
>> +    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
>> +    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
>> +#define XEN_DOMCTL_DT_OVERLAY_ATTACH                3
>> +#define XEN_DOMCTL_DT_OVERLAY_DETACH                4
> While the numbers don't really matter much, picking 3 and 4 rather than,
> say, 1 and 2 still looks a little odd.

Well although I agree with you it is indeed a bit odd, the problem of 
this is that, in current implementation I reused the libxl_dt_overlay() 
(with proper backward compatible) to deliver the sysctl and domctl 
depend on the op, and we have:
#define LIBXL_DT_OVERLAY_ADD                   1
#define LIBXL_DT_OVERLAY_REMOVE                2
#define LIBXL_DT_OVERLAY_ATTACH                3
#define LIBXL_DT_OVERLAY_DETACH                4

Then the op-number is passed from the toolstack to Xen, and checked in 
dt_overlay_domctl(). So with this implementation the attach/detach op 
number should be 3 and 4 since 1 and 2 have different meanings.

But I realized that I can also implement a similar API, say 
libxl_dt_overlay_domain() and that way we can reuse 1 and 2 and there is 
not even need to provide backward compatible of libxl_dt_overlay(). So 
would you mind sharing your preference on which approach would you like 
more? Thanks!

>> --- a/xen/include/xen/dt-overlay.h
>> +++ b/xen/include/xen/dt-overlay.h
>> @@ -14,6 +14,7 @@
>>   #include <xen/device_tree.h>
>>   #include <xen/list.h>
>>   #include <xen/rangeset.h>
>> +#include <public/domctl.h>
> Why? All you need here ...
>
>> @@ -42,12 +43,18 @@ struct xen_sysctl_dt_overlay;
>>   
>>   #ifdef CONFIG_OVERLAY_DTB
>>   long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
>> +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
> ... is a forward declaration of struct xen_domctl_dt_overlay.

Oh indeed. Will fix this. Thanks!

Kind regards,
Henry

>
> Jan



From xen-devel-bounces@lists.xenproject.org Fri May 17 02:20:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 02:20:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723768.1128824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nCW-0001xn-FV; Fri, 17 May 2024 02:20:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723768.1128824; Fri, 17 May 2024 02:20:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nCW-0001xP-C0; Fri, 17 May 2024 02:20:04 +0000
Received: by outflank-mailman (input) for mailman id 723768;
 Fri, 17 May 2024 02:20:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UZ95=MU=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s7nCU-0001dU-UD
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 02:20:02 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f74a1f8a-13f3-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 04:20:02 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-5708d8beec6so4150020a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 19:20:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f74a1f8a-13f3-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715912401; x=1716517201; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=13/tEnsX00az+Z78WeQi1Ma/WwplAv72LsKUOMKIlZY=;
        b=cpRWHl3JLxWcfG+BK0k1eMp8n0QBx8JISWf1MtnST0FH5Bif2/a4LP49B2Rgrr+2bs
         kjPxxAyFzTwcBjAm+cxxRMVSbO8cN/i/rFQX7ImQ+je0ifYiJbkjN/b6skdXUt9+MJrG
         hL7clAmSXFKdxO4YKSxIEjlOGM5sG03q7N2xnZAjHFUUKhGn2vTZcjER6WyjAmosnjtB
         JdBjzB/XYFdCs+vOWzKuudhrQ3ey3m4uH/FQFn4lZVhIed7d6O+xE7S4UX1BeTMxkZcZ
         xrOCq4PjBdTuN8YOTaFt3n3sVefuBNN6gZBxgT6MHJRXVEfhLTfsVV03MkJ9VbnhrDnt
         RZaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715912401; x=1716517201;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=13/tEnsX00az+Z78WeQi1Ma/WwplAv72LsKUOMKIlZY=;
        b=bpTaHy82lha02hYiaujGIFqmvF3Fi2E5BwpbsE49LmU3N3ntFuxnI+0rLU2mOAHp5a
         W9lZ8T8xq1Xf5vx2xMzcF/Mr2CQBqYYXFWh+TPwyN0ltMCJ13G6Gy225ySJ717fMRRq8
         TC71O3kg/2SIpUEt+aQv6W9i1eswUZdoCKedspUh2tk9Y6TTcg4FzSRDGWPhk0Ot6d3B
         mXq7hlS/sJr+3UzsB5UPinDicW77YUldCAlu6B1qAItx8kkyK1wl8oFNKQgp0Wer16ab
         0i4o3LgrGk1jlHe41JM+t2+2MMb46yYteh5btYvLRi2V1WyE7R4IiNC6GnR89hwrgJfO
         N+hA==
X-Gm-Message-State: AOJu0YzQTFtHUQrsoimI/bnX9PSrNjquGaJIeJrzhY+5KGL/YInXrz2Y
	AZr49v7zYwG4qn1R6Wn8/+H7EygtuWesmqoswxADOfIIXQNOEP+yXTEpaRs4vEOdOxQzgtoFpMd
	yIj4wkx8geMIBOQOqNV2lxLrwKk4=
X-Google-Smtp-Source: AGHT+IFQhIrAP4kf4JQCCdSWLCRSARgpMDd/QyLTW7I0F455I8yRy5zLbZOHaU9gy0Ej/s2Gfm34+r7+QV6Lu4XZirQ=
X-Received: by 2002:a50:c049:0:b0:574:ebdd:9ab6 with SMTP id
 4fb4d7f45d1cf-574ebdd9c5emr10676238a12.6.1715912401293; Thu, 16 May 2024
 19:20:01 -0700 (PDT)
MIME-Version: 1.0
References: <20240516105612.15306-1-leigh@solinno.co.uk> <20240516105612.15306-4-leigh@solinno.co.uk>
In-Reply-To: <20240516105612.15306-4-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Thu, 16 May 2024 22:19:49 -0400
Message-ID: <CAKf6xpvX2T3xhx59b0_X3e+9BpYuMZS84P=ghisZFD3yWsFssw@mail.gmail.com>
Subject: Re: [PATCH v3 3/4] tools/hotplug/Linux: Add bridge VLAN support
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Anthony Perard <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 16, 2024 at 6:56=E2=80=AFAM Leigh Brown <leigh@solinno.co.uk> w=
rote:
>
> Update add_to_bridge shell function to read the vlan parameter from
> xenstore and set the bridge VLAN configuration for the VID.
>
> Add additional helper functions to parse the vlan specification,
> which consists of one or more of the following:
>
> a) single VLAN (e.g. 10).
> b) contiguous range of VLANs (e.g. 10-15).
> c) discontiguous range with base, increment and count
>    (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).
>
> A single VLAN can be suffixed with "p" to indicate the PVID, or
> "u" to indicate untagged. A range of VLANs can be suffixed with
> "u" to indicate untagged.  A complex example would be:
>
>    vlan=3D1p/10-15/20-25u
>
> This capability requires the iproute2 bridge command to be
> installed.  An error will be generated if the vlan parameter is
> set and the bridge command is not available.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>
> ---
>  tools/hotplug/Linux/xen-network-common.sh | 103 ++++++++++++++++++++++
>  1 file changed, 103 insertions(+)
>
> diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Li=
nux/xen-network-common.sh
> index 42fa704e8d..fa7615ce0f 100644
> --- a/tools/hotplug/Linux/xen-network-common.sh
> +++ b/tools/hotplug/Linux/xen-network-common.sh

> +_vif_vlan_setup() {
> +    # References vlans and dev variable from the calling function
> +    local vid cmd
> +
> +    bridge vlan del dev "$dev" vid 1

Is vid 1 always added, so you need to remove it before customizing?

> +    for vid in ${!vlans[@]} ;do
> +        cmd=3D"bridge vlan add dev '$dev' vid $vid"
> +        case ${vlans[$vid]} in
> +             p) cmd=3D"$cmd pvid untagged" ;;
> +             u) cmd=3D"$cmd untagged" ;;
> +             t) ;;
> +        esac
> +        eval "$cmd"

Sorry if I missed this last time, but `eval` shouldn't be necessary.

        local args

        case ${vlans[$vid]} in
             p) args=3D"pvid untagged" ;;
             u) args=3D"untagged" ;;
             t) ;;
        esac
        bridge vlan add dev "$dev" vid "$vid" $args

args unquoted so it expands into maybe two args.  You could also make
args an array and do "${args[@]}"

> +    done
> +}
> +
> +_vif_vlan_membership() {
> +    # The vlans, pvid and dev variables are used by sub-functions
> +    local -A vlans=3D()
> +    local -a terms=3D()
> +    local -i i pvid=3D0
> +    local dev=3D$1
> +
> +    # Split the vlan specification string into its terms
> +    readarray -d / -t terms <<<$2
> +    for (( i=3D0; i<${#terms[@]}; ++i )) ;do
> +        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}

Because terms is not in double quotes, wouldn't it expand out?  Then
any whitespace would be dropped when calling _vif_vlan_parse_term.
Your %% only drops trailing spaces too.  Maybe you want
"${terms//[[:space:]]/}" to remove all spaces?

Stylistically, you use (( )) more than I would.  I'd do:

local term
for term in "${terms[@]}" ; do
     _vif_vlan_parse_term "$term"

But you can keep it your way if you prefer.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Fri May 17 02:22:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 02:22:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723771.1128835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nF5-0003Os-Ry; Fri, 17 May 2024 02:22:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723771.1128835; Fri, 17 May 2024 02:22:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nF5-0003Ol-PF; Fri, 17 May 2024 02:22:43 +0000
Received: by outflank-mailman (input) for mailman id 723771;
 Fri, 17 May 2024 02:22:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UZ95=MU=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s7nF4-0003Of-TL
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 02:22:42 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5619a371-13f4-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 04:22:41 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so4142329a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 19:22:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5619a371-13f4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715912560; x=1716517360; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XgeOXt05rT1J/mgAopx8fctx0EFrgq63XJy/wg1V9uo=;
        b=ZlesoXka+kCr6PDFAHGG3kDPlGLbPzodyRA+ASBO6pRp+mDMqj6bb6xjn0O/AEAftb
         xy6VraaIJ8zOu1O0AXFA7avypAIpwua7PWds2WZD4Dg79Oczo+EMrVpP9tBIHH1SmE9q
         4uJfP7ANbpaBADgczoFswW+MViaEDmR8NlFpKvMBCu/vHngXg5x2+mTgUI2QUofI9tWR
         7djhbkj3AQlAdRbeAPqo9DF+Ti2T/z6KY+bTlk9596NyxaAwn9Aa82tBjSzWwSUYWJau
         Sy6fiKyoC3T2GitpRaO6nALI43GGGmY1saEbK+pDR9NM/qmIZHkgOkJRTr69VXuKKhUX
         MqEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715912560; x=1716517360;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XgeOXt05rT1J/mgAopx8fctx0EFrgq63XJy/wg1V9uo=;
        b=wHjG1uPAfbq3XVB+mrFmjbbNPOKCoE/qYFnoL78eRxp5G2IxG/pEZcQNiFY/I213l3
         7AdEzF+WVEHnWCFauKmP+NM2uOXX8YyUmQvXe90A6HCet6fWKSqVJBH9mFQdveXwLED8
         ISx4HcHmRWvu4Y5agk/9bb+rZjtbee/SqnZdbkzsNM5a9kDa5ZysmgQ8VAdllS2Rb2jk
         ZEN1N07H8LcuW+oNqTG/ZK36i0f/yIkFT11+jMVKvlX+f98B0IJ/HfZPmflhzre7Drpy
         mmGjKbOaHeznRPgKWMv+mEcPrwaMkNcfSwd2NqVrwfxjdWQgGHEdqoSUQQ8E4olAr4DQ
         d3Vw==
X-Gm-Message-State: AOJu0Yx7IgTls7voqt+xIu1o19YI+KCAiVjNy+eZy3DkRg3Bpmp6fLGk
	PTrlYwcElFdR8ESykI1FDNT31MnyvDMfXeX8O5c2bwbDFaZGItJS+kmWIAbsXY6rYF7JRN76iAx
	Z0V0t0vi6zzwvjlxC+vBwiDXakQ8irQ==
X-Google-Smtp-Source: AGHT+IGOJfDYsfvpUKHT6NnvrIkRD6jwpPzibmMgocBERSPbFSqcMh8pWWSQmF/7DJgjN/kMj/8wbg9qSXtXLRfNA6s=
X-Received: by 2002:a50:9e6d:0:b0:572:8aab:441c with SMTP id
 4fb4d7f45d1cf-5734d6ed908mr13252761a12.26.1715912559739; Thu, 16 May 2024
 19:22:39 -0700 (PDT)
MIME-Version: 1.0
References: <20240516021010.3783-1-jandryuk@gmail.com>
In-Reply-To: <20240516021010.3783-1-jandryuk@gmail.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Thu, 16 May 2024 22:22:27 -0400
Message-ID: <CAKf6xps6KaP7BoJdiOvESX4CBk3Q25-khh0OZTkMH_j6J1MwJQ@mail.gmail.com>
Subject: Re: [PATCH v3] libxl: Enable stubdom cdrom changing
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 15, 2024 at 10:11=E2=80=AFPM Jason Andryuk <jandryuk@gmail.com>=
 wrote:
>
> From: Jason Andryuk <jason.andryuk@amd.com>

Sorry, I mess this up.  This From is incorrect.

> From: Jason Andryuk <jandryuk@gmail.com>

This is correct.

> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

These are correct.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Fri May 17 03:02:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:02:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723777.1128846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nrb-0002Zv-QW; Fri, 17 May 2024 03:02:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723777.1128846; Fri, 17 May 2024 03:02:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7nrb-0002Zo-M0; Fri, 17 May 2024 03:02:31 +0000
Received: by outflank-mailman (input) for mailman id 723777;
 Fri, 17 May 2024 03:02:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7nra-0002Ze-Q3; Fri, 17 May 2024 03:02:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7nra-0002tY-KZ; Fri, 17 May 2024 03:02:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7nra-000375-CU; Fri, 17 May 2024 03:02:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7nra-0004aY-C1; Fri, 17 May 2024 03:02:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+pIowa0YaB4iA+sbJQbKzLXJ3cm/EHBkHc/UPtxese4=; b=axfsO+kLHIWgZW3vzpGQJkiWGH
	QJKcqHwCuftOKchG++i7RphoV3CiN/CzXz0dzdbADUuCtS4md1oSNnQQ8ZOU+CL+mR3ly7WxSffwQ
	ueAKZoUw7gXyrtP6ClU0qgUvnUIF2DMZwobkpNzNvhq4UX6ld0CCw4JKIv0KcQVeBSaQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186022-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186022: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
X-Osstest-Versions-That:
    xen=977d98e67c2e929c62aa1f495fc4c6341c45abb5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 03:02:30 +0000

flight 186022 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186022/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
baseline version:
 xen                  977d98e67c2e929c62aa1f495fc4c6341c45abb5

Last test of basis   186019  2024-05-16 18:02:05 Z    0 days
Testing same since   186022  2024-05-16 23:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Edgar E. Iglesias <edgar.iglesias@amd.com>
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   977d98e67c..53dc37829c  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 17 03:23:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:23:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723783.1128855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oBM-0006PN-Dz; Fri, 17 May 2024 03:22:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723783.1128855; Fri, 17 May 2024 03:22:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oBM-0006PG-BN; Fri, 17 May 2024 03:22:56 +0000
Received: by outflank-mailman (input) for mailman id 723783;
 Fri, 17 May 2024 03:22:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7oBK-0006PA-Oe
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 03:22:54 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2408::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id be683921-13fc-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 05:22:53 +0200 (CEST)
Received: from PH7PR03CA0028.namprd03.prod.outlook.com (2603:10b6:510:339::18)
 by DM3PR12MB9413.namprd12.prod.outlook.com (2603:10b6:8:1af::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 03:22:48 +0000
Received: from SN1PEPF000252A1.namprd05.prod.outlook.com
 (2603:10b6:510:339:cafe::f8) by PH7PR03CA0028.outlook.office365.com
 (2603:10b6:510:339::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29 via Frontend
 Transport; Fri, 17 May 2024 03:22:48 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF000252A1.mail.protection.outlook.com (10.167.242.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 03:22:47 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:22:44 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:22:02 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 22:21:59 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be683921-13fc-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WCrKt3jND6ubuKXsINud70JZHnV0lNzSXOqsXyTvIaxJJjqcsKXaK7x5By9HtT3OIsz7TzyUNAoa2epfmMCf0QwvyLQjETNUdskqm3tp/42IyWaRwode3/cn10bgPwfTCC894JZCSAXSOdF8dRhMmbBx6fQ9QdWVzTf78OMwdXKjfLW//Hhbvnx3GpcbCPNzEiZ+93R1Od/V9N9koSjj2xxMqYkEIyg50Zr00oG7UPyyR/CN6UXO3iZ+DiKGFi0J+ixg294eLNahR6DTPi1nrNGffmGlLoPOqsBHvXt9HA+IVSjUdcWWDaHuvHpg2UBUB/KCRAGCbnYknNvAg+Kb6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=AxU6vjuvVdOW/ywnLEm/VDAKtSigYMJONUnO/ZgkylA=;
 b=EcbiQJ8ERHn3C8lROE6Z4zS2Ky8iNv6GNTgz7/B4jefJxomsyV/XWZcGxLHXS+dIqopRct03s35Q0Yh2cZ/xnA8J/hRwkWefE4fNzvszVGnT38EVxU1sOC0DXUWjKaF27/I94ZS0svyvsD53mZy0SFR/oizYfTcBppGdF39G9RF4HX9Nzd0xNXoMz68VQrvgVZNBep0G6NO4/fYm6v+D3xSuvEz3/BOfUh6YQ+QJnCGHYN5bT/elxlTyuRqPyntQTC4Y8u88hw9ef002YRL0E/Cx/lqnVlLDmCKzBYhOR1d7Kix953I6atR5fWvaQv+Z+mT94v1whfyBoH6csSOZ4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AxU6vjuvVdOW/ywnLEm/VDAKtSigYMJONUnO/ZgkylA=;
 b=R7VHd7YiqWv7SmuMiIqL+ZUTtOsh0wyMu3U2LUgpGfEf18y26kssrXb10F63zS8V9CLrRkI6zFKD7ot9pqddVlDUyGxd3XGydasJohL3QTBUbFpb8Tu227C5SUkIylTPjtbLhMHRNOWT8HJgfj67SHAbCpxhqZTO3TemuyNSb+U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Anthony PERARD
	<anthony@xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v3 0/4] Guest XenStore page allocation for 11 Dom0less domUs
Date: Fri, 17 May 2024 11:21:52 +0800
Message-ID: <20240517032156.1490515-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF000252A1:EE_|DM3PR12MB9413:EE_
X-MS-Office365-Filtering-Correlation-Id: e9f31c08-1c31-4087-a14f-08dc7620a02b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?DqsI+RJvLtXVKscPqyFU08PoQTL82k8F0s8CuprgE66bG0EfKlkVHGeQxGvs?=
 =?us-ascii?Q?Z0pP/n/FWkKOwflWI+m3Uus5gUBjyq9XSz5EOToyrUnJmdQ8ivm7kKot0emR?=
 =?us-ascii?Q?F6aE5iaPaxU7cw48v9GzFysDqr62BBVDEahVSAEeSzoCqgDWTSiQ0LfI5LPq?=
 =?us-ascii?Q?+YaEMd5oaQ4hZgqJkOqZE1/glBHMnn4VH3pc7Itmfhn/QJexECrrsh+W1VrF?=
 =?us-ascii?Q?eGg3T1ZndoomPtC/Bt7RmbInwyQ9ISAOGaFjovCOG33hGTTUepISTl7tbCIp?=
 =?us-ascii?Q?RupzaWVzsLgwpov1erqHilrx78Im0Ql1GMT4AMH/xrAZ7DKUyGB7l26ShWXX?=
 =?us-ascii?Q?+LtcF9Bxu8Kb3C60csKNAa185pZ7TStcaXnBWmEpNMEFhPMNdieMise7bMR7?=
 =?us-ascii?Q?3sztI3V3FHiW5PsDasOnM+qBeudub3o+v2AiyGTHCEsiGgX2TDzdeGzW6rLk?=
 =?us-ascii?Q?891/h2SVYQbZgdV9BlyMvDAh3grntCRt2ubJZQkSaCDG7enUFUHiR98Dk7Th?=
 =?us-ascii?Q?mxBMT0kOjo61pEv3odU5ZV1Uvf+jFNWtXOe3eZJtTSPTfxPxFF/5dssEKOH/?=
 =?us-ascii?Q?EBZ3oFXq7a+C7d7fgs7kRYrUzRjebqOXP6uBOB5c993Ynwwu1muIExFhq0oS?=
 =?us-ascii?Q?YsEtP5Z+z3vU5r3WsYJqtZRmF5L125qr/FKwskXrsW22qkc309PtzBTG5OxP?=
 =?us-ascii?Q?BBft6vY7NV/g8teSftfSvqOJFn8sN7MDo+VdM38g9rkrpvwCIht7ndRRVAAb?=
 =?us-ascii?Q?bj/QbR9ZdF0j0ETDst7TkiKZTbvd7gRQyriiFEsptQO0pugq5OFZO0yTPSaz?=
 =?us-ascii?Q?z/ZWFFIkqiWIEBn61sdzzNLI2GLb90c6V6WgX6cxIqxNr70dsvMw4mdVHaEQ?=
 =?us-ascii?Q?Fe7YkKEXpCu/kzAb10YCMIfh9Ck8lVFC4AXUMOFKbvAnjJrSVs60WB3eDeb8?=
 =?us-ascii?Q?0wCxW2Fq92mkK3X0qFj+LCMb3AaovJkDIUaerXMcnL67OZJ13CLScThWWqnZ?=
 =?us-ascii?Q?xD+klsBInfCmtbUL1AOA4mSrmO38nRKj9Fnlf8PZms3jd5pTyQV1W4QUMZr7?=
 =?us-ascii?Q?/G/h1NSc0X47XBrt6ZAe8KDRxWBsXqKZprSzdiPMkuR92vrcztFVdwvvfeDO?=
 =?us-ascii?Q?tDQPUl2rXZ2zR7SuHYj4nhBk/PtwsKDtDSISC/dTvFSAzuinxqJI4uySyPuM?=
 =?us-ascii?Q?NGq2mii6yVobtYWxgxOb65aT6hxRvz1rMkbM8a2fBVPc8pIQuHYcmYA5Os+p?=
 =?us-ascii?Q?I3wRECLckd/J0gdREuIS4G2cahkkBZfwrgypMPcuTcff1WJ4jkzD9Lm19Q2f?=
 =?us-ascii?Q?yPI1Yyy1UngUMXCJtIn8u+dkM5VnZ/BKg45ntNrpQeKMbw=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 03:22:47.7387
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9f31c08-1c31-4087-a14f-08dc7620a02b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF000252A1.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9413

Hi all,

This series is trying to fix the reported guest magic region alloc
issue for 11 Dom0less domUs, an error message can seen from the
init-dom0less application on 1:1 direct-mapped Dom0less DomUs:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

This is because populate_physmap() automatically assumes gfn == mfn
for direct mapped domains. This cannot be true for the magic pages
that are allocated later for 1:1 Dom0less DomUs from the init-dom0less
helper application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list
is empty at that time.

In [1] I've tried to fix this issue by the domctl approach, and
discussions in [2] and [3] indicates that a domctl is not really
necessary, as we can simplify the issue to "allocate the Dom0less
guest magic regions at the Dom0less DomU build time and pass the
region base PFN to init-dom0less application". The later on
discussion [4] reached an agreement that we only need to allocate
one single page for XenStore, and set the HVM_PARAM_STORE_PFN from
hypervisor with some Linux XenStore late init protocol improvements.
Therefore, this series tries to fix the issue based on all discussions.
The first patch puts a restriction that static shared memory on
direct-mapped DomUs should also be direct mapped, as otherwise it will
clash [5]. Patch 2 allocates the XenStore page from Xen and set the
initial connection status to RECONNECTED. Patch 3 is the update of the
init-dom0less application with all of the changes. Patch 4 is the doc
change to reflect the changes introduced by this series.

**NOTE**: This series should work with the Linux change [6].

[1] https://lore.kernel.org/xen-devel/20240409045357.236802-1-xin.wang2@amd.com/
[2] https://lore.kernel.org/xen-devel/c7857223-eab8-409a-b618-6ec70f6165aa@apertussolutions.com/
[3] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2404251508470.3940@ubuntu-linux-20-04-desktop/
[4] https://lore.kernel.org/xen-devel/d33ea00d-890d-45cc-9583-64c953abd70f@xen.org/
[5] https://lore.kernel.org/xen-devel/686ba256-f8bf-47e7-872f-d277bf7df0aa@xen.org/
[6] https://lore.kernel.org/xen-devel/20240517011516.1451087-1-xin.wang2@amd.com/

Henry Wang (4):
  xen/arm/static-shmem: Static-shmem should be direct-mapped for
    direct-mapped domains
  xen/arm: Alloc XenStore page for Dom0less DomUs from hypervisor
  tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
  docs/features/dom0less: Update the late XenStore init protocol

 docs/features/dom0less.pandoc         | 12 +++---
 docs/misc/arm/device-tree/booting.txt |  3 ++
 tools/helpers/init-dom0less.c         | 58 +++++++--------------------
 xen/arch/arm/dom0less-build.c         | 44 +++++++++++++++++++-
 xen/arch/arm/static-shmem.c           |  6 +++
 5 files changed, 73 insertions(+), 50 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 03:23:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:23:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723785.1128865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oBv-0006ql-MP; Fri, 17 May 2024 03:23:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723785.1128865; Fri, 17 May 2024 03:23:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oBv-0006qe-Jf; Fri, 17 May 2024 03:23:31 +0000
Received: by outflank-mailman (input) for mailman id 723785;
 Fri, 17 May 2024 03:23:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7oBt-0006l6-I6
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 03:23:29 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d34aecf3-13fc-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 05:23:28 +0200 (CEST)
Received: from PH7PR03CA0003.namprd03.prod.outlook.com (2603:10b6:510:339::8)
 by CH3PR12MB9283.namprd12.prod.outlook.com (2603:10b6:610:1cd::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 03:23:23 +0000
Received: from SN1PEPF000252A1.namprd05.prod.outlook.com
 (2603:10b6:510:339:cafe::4f) by PH7PR03CA0003.outlook.office365.com
 (2603:10b6:510:339::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29 via Frontend
 Transport; Fri, 17 May 2024 03:23:23 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF000252A1.mail.protection.outlook.com (10.167.242.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 03:23:22 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:22:45 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:22:06 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 22:22:04 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d34aecf3-13fc-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n9e5/ovVTWjkuuRvPCpt/+SZKX1pA6iH5NwxqjN7Eqt9Nt9yOk7jUI6wYZgSB/loO8hslJCdDKWh7LTpBeG2aKtKYJaHJ0ItlkdbD76iKza/dlmY28J0bRomijpsvkrmK2TiIZw0VgIpZCnCFqjeNp2n3W7bT/dv+jd0X+xfRG1ohhLkOMPXiv/NVbQK3oUPsGZQZIhv4n7CE3w5YeYrB9DRNn8UT1I9MQwl7xRG0lS+UwBurs/szKjHH06zvuc2p19uiEWb6RMLJ35ZCRlW45GfQCAsuxOTNeh6p6qdNiGf1ZXGPxf61UPjeX1oXCNV/Xc/Bi9pk61YpVM7fDHCkA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=d07nHhnz+E2r01WFGPEt/6CKVIk3TuRy1RIwAxzVZg8=;
 b=NZvFQIvZKZIPVK7wUXoqVgCJf3IxmM2cLNB8VZDkPZQByKcyfuKcNW6XJMIRSc5L/MJlS3Iu6q0Q9I/DG0Qn+vR+S4ErIZlMCfoGUsmuK8T5yNTGpiUtKxj3/oWzaeno9hfTrDGxacR93qGMs2mkZf4Q5Na6RQCFQblh9aJb/h1U6YKmArS+U+gmhuOwi8lpXgqvbfgFTbIrxFxlsr//Z2MQfspV24ZDFIa3FbWKlgs9gO/tFkLA1YUUCJCJ9op8HWTsI+RNuiEUIvXF12bQ/tK3SGiufPRYeGFHkRcQLHhQkga2QP9ixKjvGoBJ5YAitsA+UU58G1paTqvplPNjSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=d07nHhnz+E2r01WFGPEt/6CKVIk3TuRy1RIwAxzVZg8=;
 b=0agY+9cxmafoZu2T+FJeoPF8TsVf7XQ0Ruu8n39kIikZjOkbIWcfELta3NPO4Ok3nqSfXw9bnmcEGyG1xKWPIhL8PMaEVoAj0z/lnfISGvE6g9ZEMLwf7M1TUKTI6cWYX7FvO/a0iAqA5uCIytmYksR4WVgwWXYeYfa7WHTRcOo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be direct-mapped for direct-mapped domains
Date: Fri, 17 May 2024 11:21:53 +0800
Message-ID: <20240517032156.1490515-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240517032156.1490515-1-xin.wang2@amd.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF000252A1:EE_|CH3PR12MB9283:EE_
X-MS-Office365-Filtering-Correlation-Id: 1cc80ec2-fb9c-48ee-4b07-08dc7620b527
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?0DvRM6rtOlsUc+yQyPL4++Ekav5HIZAl2PAdG83t4QVeqdODU3IDqyYVQVRf?=
 =?us-ascii?Q?GdGRrkXADLwEVljkA7fDmVoM5W2BqdAbpa1VhbqRh0mxoJuRdHaleFAM+4yC?=
 =?us-ascii?Q?H66jOMTehHN+eThX0zJ8FJ39ZCL2cfa/tleae6iwK7qvscIZtZ3jKblPqlRG?=
 =?us-ascii?Q?HYZF3Do1vYD8Mta0XM5KFpU5D4yp6ohjOYbjP/fmmlwIWFtXIbs4N6YnE5G9?=
 =?us-ascii?Q?5PIk9hfGP73Y/zfNc7wDN1VVhYEN3fWtk/R18BdrdsPGEm0z0zWCvuoifSSm?=
 =?us-ascii?Q?kZnzVE97tgV8rfRDLy1ld5ojXgARLCqzZGcIXUvUag6k1q6AWlO+H5vGH7jS?=
 =?us-ascii?Q?MSmnuWx/2BvUWkkGE4cDOtxT8Uwsgz8+yyTXi/We6nI4MomLpHiQ8LdBZfuw?=
 =?us-ascii?Q?OZyOrKV2sDWYlcivbk2zgWxKC+4Ha9rdozepOGR5L9hXoSw2FQ3QshD6RNxT?=
 =?us-ascii?Q?vNNG4uuEdQr5TngAmkzdcQnS94IhDhx8OaqT6ru4v75GC0q+75qwOXXPRa0C?=
 =?us-ascii?Q?z32ehEjopzgisILgkaBg5JLPxCJOd//COJ3Fj+twPt+7esB5VqSx/yRp8M1L?=
 =?us-ascii?Q?AdErgP1Oj2v6PHDCqNvvmyJ4Hu3JkU0tCkRDgG7Vkj2/jIuD0xT3JKiYxyiP?=
 =?us-ascii?Q?cuGrvn+tNVLzAYCYm3sB9b+PgYPwITRnqJnGkn0ESxxMp0A+6g3z3TrcuYDq?=
 =?us-ascii?Q?HTppdo334EZKES0qeyivc54bgT9Ld2z4spZSdT2zhAnUHb4gSejdyuLmAdOY?=
 =?us-ascii?Q?FKHhgv0ifBy8peOs7W4nUTzvWTT0BdhHBjKJV3FQsa/qVu2OixGzRnvKUrnY?=
 =?us-ascii?Q?1WIvWBPeLWk+7A9xLjVb2hOHRklI0wucgwSXTZJaW1pC64HOSzHXTGOlYEba?=
 =?us-ascii?Q?Qx2Xk9w1+0NQeWZ+hY0yBTsMm3rykKhs2AqrhW8LLzAi4VTsMwSdzHUaJ1ya?=
 =?us-ascii?Q?gKDIDKOD44oY41v6Dzt0a7zkJTkx/W6M45VbOe8JfCMaV++4qYnqZjmlmDuO?=
 =?us-ascii?Q?ldousSKri38eqFOnie5awiZNkirshQNvVT2z8rXPlTeq2EZR18zFeMYqbOwu?=
 =?us-ascii?Q?iU0BS6YuwTWtmBCq+nln6fWr4F+OJeHHNhuGuUGHm+58G0/n+MbEsE8egRTN?=
 =?us-ascii?Q?vdYN/ZgEA/FJ2XN24Zzzc0z7f0y7MGhKdH2Vn5OsS/8pcb3UwOUtZzZLkCh0?=
 =?us-ascii?Q?dIKZEMLQv00RjjcT8DImvCUDfMQXbx8EENxbdNJKaKblsi7hMOANtSdQ9KGG?=
 =?us-ascii?Q?MXayQmDfnBzf9CvRRsndu0myXPt/Gnxc2kYEaKOjAZDlPOS7bSNzw46etBxl?=
 =?us-ascii?Q?KAe9G6od+U+/7/MbyOO9NRbQ5b3oUlnsZP9A1l6l77LVSJNtVp0YoSot8Cyo?=
 =?us-ascii?Q?5p9fR1gtOGfewaMOoiJy4PiW3wC8?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 03:23:22.9425
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1cc80ec2-fb9c-48ee-4b07-08dc7620b527
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF000252A1.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9283

Currently, users are allowed to map static shared memory in a
non-direct-mapped way for direct-mapped domains. This can lead to
clashing of guest memory spaces. Also, the current extended region
finding logic only removes the host physical addresses of the
static shared memory areas for direct-mapped domains, which may be
inconsistent with the guest memory map if users map the static
shared memory in a non-direct-mapped way. This will lead to incorrect
extended region calculation results.

To make things easier, add restriction that static shared memory
should also be direct-mapped for direct-mapped domains. Check the
host physical address to be matched with guest physical address when
parsing the device tree. Document this restriction in the doc.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- New patch.
---
 docs/misc/arm/device-tree/booting.txt | 3 +++
 xen/arch/arm/static-shmem.c           | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..c994e48391 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -591,6 +591,9 @@ communication.
     shared memory region in host physical address space, a size, and a guest
     physical address, as the target address of the mapping.
     e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
+    Note that if a domain is direct-mapped, i.e. the Dom0 and the Dom0less
+    DomUs with `direct-map` device tree property, the static shared memory
+    should also be direct-mapped (host physical address == guest address).
 
     It shall also meet the following criteria:
     1) If the SHM ID matches with an existing region, the address range of the
diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 78881dd1d3..b26fb69874 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -235,6 +235,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                    d, psize);
             return -EINVAL;
         }
+        if ( is_domain_direct_mapped(d) && (pbase != gbase) )
+        {
+            printk("%pd: physical address 0x%"PRIpaddr" and guest address 0x%"PRIpaddr" are not 1:1 direct-mapped.\n",
+                   d, pbase, gbase);
+            return -EINVAL;
+        }
 
         for ( i = 0; i < PFN_DOWN(psize); i++ )
             if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 03:24:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:24:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723789.1128874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCP-0007M4-1i; Fri, 17 May 2024 03:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723789.1128874; Fri, 17 May 2024 03:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCO-0007Lx-VK; Fri, 17 May 2024 03:24:00 +0000
Received: by outflank-mailman (input) for mailman id 723789;
 Fri, 17 May 2024 03:23:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7oCN-0007En-Mo
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 03:23:59 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e5f63868-13fc-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 05:23:58 +0200 (CEST)
Received: from CH0PR03CA0067.namprd03.prod.outlook.com (2603:10b6:610:cc::12)
 by CH3PR12MB7643.namprd12.prod.outlook.com (2603:10b6:610:152::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Fri, 17 May
 2024 03:23:55 +0000
Received: from CH2PEPF00000146.namprd02.prod.outlook.com
 (2603:10b6:610:cc:cafe::43) by CH0PR03CA0067.outlook.office365.com
 (2603:10b6:610:cc::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Fri, 17 May 2024 03:23:55 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF00000146.mail.protection.outlook.com (10.167.244.103) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 03:23:54 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:23:02 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 22:22:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5f63868-13fc-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cM3+el3unuMU10rud3w2f70OlJrNpEKvpi+Q+OTAguVkxqQy1EGeh/zPDVabehbt5UAYwVtDGzdoEVVs9f0pcAP3D1hiqLCsNp99Y95aQZCGNAoPLJngnq968MDZ8WtqUMGF8Dy36NhGvVgtDVLc2Fohu7KNgXgNDuvjxScGAQHP6e+3eYI66E/naS05C1kuGR0E9NcQ3+KQlU7v3F/3dA35TaQo3utrE+e9ck7aqNnU5Ro1UdTdFKAL9ZXwIe72WZ/mO/w+MKIL9IkfVX7gI67rZspCVlOPXLpUvBO1TflUNjSoODhOhDSd6Glj2pvhLP1NPYjipcglmkK9XXkxxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fG1F9XegT/sJSYG6Fu85jzQtPZmim9YqynPLlOCoLeE=;
 b=lgWrhr1X7NFphekaFNfrMFCNl6qK1tkr++mr4XDCxGz/A9km37nU5ngwIcy4HRirjmSGcR/82t8BwQO5yAm5Gi34Ixd8m70wBxb0eqgYZgFWME37+kjNhYnP6jA3qZ+PS5OvrI4LFBOCTJ748mBJIkbK8KyjVIpLP+th/bmGgaUZvlF//bFMQtGiGxXNI/vWQu0C2VJGdKfp2xNNr5ttIIylc6dQl6+xTRFMd3BGgvAuKY0TFN8xxl/SyR87G1UihkM/U0LqXQbFvaHEiL4K7QCT0fx1b01Ms5ypPxoROBPGODzpfhyAU/K3FQ9JHw/zMbeg//wymwnl/AWHf3a0VQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fG1F9XegT/sJSYG6Fu85jzQtPZmim9YqynPLlOCoLeE=;
 b=Ob/HghIMy4+7AuuOjEmwGtFEOFbtR+arb8ugrofvaFUDYX4Q7zHsZgp0ZD26xjdzy/d8tl6ZvG8kmokmblJrIErFGSl1toJi1mZr0/PUnVjqPal3KpA+D72UDpsrCuswIVWwlYo+lAY1FTU/0O3kK26juMiAVP3aHWScHWOcTnM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
Subject: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs from hypervisor
Date: Fri, 17 May 2024 11:21:54 +0800
Message-ID: <20240517032156.1490515-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240517032156.1490515-1-xin.wang2@amd.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000146:EE_|CH3PR12MB7643:EE_
X-MS-Office365-Filtering-Correlation-Id: 36fbe1af-e58e-4651-de76-08dc7620c831
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?6BiiqWCLmIwOmluOM5NDMDVRIN7urfuugWAcSQyiQ0feyxQjQz0D547Nrx+c?=
 =?us-ascii?Q?fDDrUjkgsaPtZLWsLfmRrJro4kedSrPWWvzZ7gmrPVvNRtDwf8SHJgGhufCA?=
 =?us-ascii?Q?bzhSF3snc4lHudZsCgAXIDKA9XlIVqQOOCdb4IZsmRUtKiXmFRG6mHXosxOn?=
 =?us-ascii?Q?RVGfnAM31COHLC374WOQI3aHQZP1hFHW0mIWDKbyKsxKsHpw6gDtqKxRzliK?=
 =?us-ascii?Q?3NgAonQs0coTkpQK2bCKlAHZgW7ppepzNkge7uqOcCkRqMmGdk45WYyj3HDe?=
 =?us-ascii?Q?fa3az5RZPG038L7hfXGubIYlbZB1F4gcDcrkIE07fAjUkoi/mPEIoPY1hEbb?=
 =?us-ascii?Q?Mi1b0c8ccUaVCFk8Kyi6wY0cEThitWh49/rD00nreMTNi+PVI5zYvKInWmMs?=
 =?us-ascii?Q?SWdpjMkymw22EcDEouqV9KTT5ZzQ6ZzF8JlfAvCrHvG9RxWnaXFaKhroPVri?=
 =?us-ascii?Q?9XlqpG4RKOdzNa5eUrA+zmgu/u7L5EsQ14ESPQUJPXk05wjiscZYCiG1t4kD?=
 =?us-ascii?Q?qQUp8GjxuDs3SrGRtwhhB0L8XYZrqpGVQDvzbC4+ZyKydD47hGaOA/BF4ouq?=
 =?us-ascii?Q?UjwwStoN++E0Mnqohm/9XSqXOI/SeLftAlCfNA5s4Is7klWNEZQsop6Mxhzq?=
 =?us-ascii?Q?zKsPDSX0TC01V2Q0muSDQdCLtpikVB8uL7Nk2YN/92FSwlVAdClET9fZF2p0?=
 =?us-ascii?Q?kC6Jd/SiFCcLItZbORi1fAS7aEjQFGfyTLqQL2hlLdevZAMwR8IFblCrpXfz?=
 =?us-ascii?Q?JiMEjNakq5sNJaN8lNbMaDMyrTUknxjj0p4IRciP9o2WP45DYman1h1j5T/O?=
 =?us-ascii?Q?Hj9yD6DcWlhGbQ5tmpu6lJb77YlEgQuBwAlu3OVG64uF/fT2gcdSpdV0jiWF?=
 =?us-ascii?Q?UarCr1botGRsIJI9nIwg/VfrF/k2is8WDKStDDtXRcqKPEuEWeZJtrGcKlO8?=
 =?us-ascii?Q?Mq3Ff3qAAibJd+1VEURPcGrDcCp3RlHmwMAlQpsfbQtoTlHtcRyYbxmVEonw?=
 =?us-ascii?Q?K1Lv6sfKoTa6obqoot/7BMGX3op7i08F7AC7WU+rHYC5M0z1uAKOcMj4EDgs?=
 =?us-ascii?Q?EYP4DG5wVwUWMUHGBJXrtifl7nhiDd95tyfVCy40qg8WB2+GYxegy+OvOF1W?=
 =?us-ascii?Q?ShNiHjHPe/qmgdrmu7SViAVT7dKFGGg9uOl6b5139+Q70hEKUkDMzmb+84XJ?=
 =?us-ascii?Q?5bx+sqxykFQAHw+ymxLIv3bL7xtmtKvxDF20KP8s+OvW1poTluKjG6HTOVy5?=
 =?us-ascii?Q?VuHmk+ro8sBgh7GZE/1E2ofPP9y5tu5FVGqEU5LzmZZMgKz6vJWCizv9H6zj?=
 =?us-ascii?Q?SyGmhL19W1KdxfbB5IqzxATi+gabUggCT/YXRv0XxEykT01ikJdCIh0Q7p+V?=
 =?us-ascii?Q?/mxshxz8mWkvQoYkmjNn2iHUw0U5?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 03:23:54.7635
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 36fbe1af-e58e-4651-de76-08dc7620c831
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000146.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7643

There are use cases (for example using the PV driver) in Dom0less
setup that require Dom0less DomUs start immediately with Dom0, but
initialize XenStore later after Dom0's successful boot and call to
the init-dom0less application.

An error message can seen from the init-dom0less application on
1:1 direct-mapped domains:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

The "magic page" is a terminology used in the toolstack as reserved
pages for the VM to have access to virtual platform capabilities.
Currently the magic pages for Dom0less DomUs are populated by the
init-dom0less app through populate_physmap(), and populate_physmap()
automatically assumes gfn == mfn for 1:1 direct mapped domains. This
cannot be true for the magic pages that are allocated later from the
init-dom0less application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list is
empty at that time.

Since for init-dom0less, the magic page region is only for XenStore.
To solve above issue, this commit allocates the XenStore page for
Dom0less DomUs at the domain construction time. The PFN will be
noted and communicated to the init-dom0less application executed
from Dom0. To keep the XenStore late init protocol, set the connection
status to XENSTORE_RECONNECT.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Only allocate XenStore page. (Julien)
- Set HVM_PARAM_STORE_PFN and the XenStore connection status directly
  from hypervisor. (Stefano)
v2:
- Reword the commit msg to explain what is "magic page" and use generic
  terminology "hypervisor reserved pages" in commit msg. (Daniel)
- Also move the offset definition of magic pages. (Michal)
- Extract the magic page allocation logic to a function. (Michal)
---
 xen/arch/arm/dom0less-build.c | 44 ++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..95c4fd1a2d 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 #include <xen/device_tree.h>
+#include <xen/domain_page.h>
 #include <xen/err.h>
 #include <xen/event.h>
 #include <xen/grant_table.h>
@@ -10,6 +11,8 @@
 #include <xen/sizes.h>
 #include <xen/vmap.h>
 
+#include <public/io/xs_wire.h>
+
 #include <asm/arm64/sve.h>
 #include <asm/dom0less-build.h>
 #include <asm/domain_build.h>
@@ -739,6 +742,42 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
     return 0;
 }
 
+#define XENSTORE_PFN_OFFSET 1
+static int __init alloc_xenstore_page(struct domain *d)
+{
+    struct page_info *xenstore_pg;
+    struct xenstore_domain_interface *interface;
+    mfn_t mfn;
+    gfn_t gfn;
+    int rc;
+
+    d->max_pages += 1;
+    xenstore_pg = alloc_domheap_page(d, 0);
+    if ( xenstore_pg == NULL )
+        return -ENOMEM;
+
+    mfn = page_to_mfn(xenstore_pg);
+    if ( !is_domain_direct_mapped(d) )
+        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE +
+                           (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
+    else
+        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
+
+    rc = guest_physmap_add_page(d, gfn, mfn, 0);
+    if ( rc )
+    {
+        free_domheap_page(xenstore_pg);
+        return rc;
+    }
+
+    d->arch.hvm.params[HVM_PARAM_STORE_PFN] = gfn_x(gfn);
+    interface = (struct xenstore_domain_interface *)map_domain_page(mfn);
+    interface->connection = XENSTORE_RECONNECT;
+    unmap_domain_page(interface);
+
+    return 0;
+}
+
 static int __init construct_domU(struct domain *d,
                                  const struct dt_device_node *node)
 {
@@ -839,7 +878,10 @@ static int __init construct_domU(struct domain *d,
         rc = alloc_xenstore_evtchn(d);
         if ( rc < 0 )
             return rc;
-        d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
+
+        rc = alloc_xenstore_page(d);
+        if ( rc < 0 )
+            return rc;
     }
 
     return rc;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 03:24:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:24:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723791.1128885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCa-0007it-Ag; Fri, 17 May 2024 03:24:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723791.1128885; Fri, 17 May 2024 03:24:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCa-0007ij-6B; Fri, 17 May 2024 03:24:12 +0000
Received: by outflank-mailman (input) for mailman id 723791;
 Fri, 17 May 2024 03:24:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7oCZ-0006l6-CG
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 03:24:11 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ec385ba3-13fc-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 05:24:09 +0200 (CEST)
Received: from CH2PR12CA0022.namprd12.prod.outlook.com (2603:10b6:610:57::32)
 by CYYPR12MB8701.namprd12.prod.outlook.com (2603:10b6:930:bf::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Fri, 17 May
 2024 03:24:07 +0000
Received: from CH2PEPF00000143.namprd02.prod.outlook.com
 (2603:10b6:610:57:cafe::c5) by CH2PR12CA0022.outlook.office365.com
 (2603:10b6:610:57::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30 via Frontend
 Transport; Fri, 17 May 2024 03:24:06 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF00000143.mail.protection.outlook.com (10.167.244.100) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 03:24:06 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:23:54 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 22:23:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec385ba3-13fc-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z1XYILrUxe22f+CO8p9a/Uy7Poo7DKsvyV4yI6p+LfGwfkcvdTz629vvLFTxr1DFF6TiKcTFOTJZtxCf6oyyB46z2sYsALR2xh8KbP1WeuPHzexdedS1dgZmMx9pYKGp96/q5Wq11teXco0VYDJ5m+F2pk7zqqx4nS7S2I6PhKZfQlxjqiFYXTyJDeG8n72xivvPXwvSjYOr0blm6B2fTtisdDOyawr918RW85ortydi/yLheM1Rhnmzz+B3L50shDv93LM0CVU2zatFp7z0RdtQOxA0ROyrinu1cz7XavlGlTOLpAPGF9FXUsfcHrD6NKqDeOp0LipiZY/1snYP9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GyKN4QF9fVjSu+AyjNRAOue0Obnrq0vCp9OkHGHQS+8=;
 b=Ug61W5mJiOquEseiazbkFBTgKoFK+ldvDyyduTys/YrB2fyr7lE6071VIb+AC/qvWz7WFQWfhhtJyNQtbZP5iRe/leSUyGTZcr2IPcktTMFl14NOqJyAh1+N78ONe102cVjBOSemqx71tYwE8dRqbKHRozqr9L8OV3w54YCro69/xoXx8Jk8+48h1ZgsOEAHoJTEePvhPVRDt69+UUE5QQSyH/Xb+l4ihFhQil+REdOL/hG8b/rgemRighaHDXbmtefLkoxg3onSu4x4XRoykuQXGO+sssHlJfKldXzlhDLqqCapS74Y7nGcoSbKrtHJ8WneiCWyqP6ijzAsh/NlcA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GyKN4QF9fVjSu+AyjNRAOue0Obnrq0vCp9OkHGHQS+8=;
 b=sYZi1WHYB2b6gqIn9RpP/IkualfsU+zpIyCDPSH48beLXbRbQD43nsRagh9nOiSmitR1e04kG2n/aywDZ9DmAxGpGnnCeZUjBAyi56F6TB5nflicCUKW/Y1EJeCC5ijsnbv3nbZvZwNBLYz7IiJPqqt4q/biwnW8E8TWt7LLoFk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Anthony PERARD <anthony@xenproject.org>,
	Alec Kwapis <alec.kwapis@medtronic.com>
Subject: [PATCH v3 3/4] tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
Date: Fri, 17 May 2024 11:21:55 +0800
Message-ID: <20240517032156.1490515-4-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240517032156.1490515-1-xin.wang2@amd.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000143:EE_|CYYPR12MB8701:EE_
X-MS-Office365-Filtering-Correlation-Id: 2116035e-4bcf-482a-0f45-08dc7620cef4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?NQHS6Ndpm0l2r0sHPBDXsEws8+31YWiTYLcnl8J9Clp6KfWkGctnT62XMdWa?=
 =?us-ascii?Q?IwuhBBtSkSGlwqfyl3aQHOan+O1NoX7XWNlQg+iU99pulfS+9rwMkbn9mqpP?=
 =?us-ascii?Q?zEOSCs8rUEtJOfbLlz0hqDwIbo4RCo+E79A4ERiH5SmqRQ/IahHmOTfTnsCf?=
 =?us-ascii?Q?7WIWjIrmoU+tQpl482hUG/x0HNEY/JywmtCKCJIwTGlDYothq++OS7E/q2PR?=
 =?us-ascii?Q?jAtLAwu1IHCHrh4tfaqvO/ZPxIIMnr12vnqvTRTlCPZ5T2kiI32x+scFXM0d?=
 =?us-ascii?Q?ajZauCGVbq2WWr7Ockv6mG3H24M2f3Tro82bIUHdNUR/okYuF0QOk0VDA2DG?=
 =?us-ascii?Q?DmMHYOKhMx2+4t/gQPdh0q5SHIR2b3x9wQXtBogvCYPyy55EkcR7jA4bnm4N?=
 =?us-ascii?Q?WHqCXI657Hp/Bbp6Cuf2UM2zKDGtA0JG1CxvP729vOGGb8zcJAzfsDlK/gRg?=
 =?us-ascii?Q?fO9mUzSs8eJI9H0JxBJCALtsP4bUeefUSGqdD8y313qu+k8KnxsjplDRhkfg?=
 =?us-ascii?Q?SY3iOgrdq4mOS4e0f80NTrYK1ffutG36H87uTsPQoBpbsvaISpjERprAMMp+?=
 =?us-ascii?Q?woxqR+owYpPWkqWlCbZCaQCVn8KSUjI9JHpMJPLM/6lXt115cPHbDKn4J/wu?=
 =?us-ascii?Q?2NJ7Yt9jl58f+BEnUbWuVzMpQzy/7n0dHg5NIcAHLvyXXWL2+jwAGDx5z8Is?=
 =?us-ascii?Q?JKcgxTvB1N3XcsRMExVaIDMEng3SDYULbrnC4m4Vs1aIx78onyAbv24GSQVH?=
 =?us-ascii?Q?vbh0XqI0SNJzdMQZ11pPOHIgLk3sYwciwkHQrkGH59HIpBlyIr3XNfoktood?=
 =?us-ascii?Q?StIqgVfai6X1ZR+Jx1fav38bJzwXIQrKG9rpx97q/AG06Jxrlx+YduqPL6Ko?=
 =?us-ascii?Q?W8fQoYMY5nYnrC0JMnUoA5wyh6A6YAcqbTz5tpdrzrUWu3kApmHrDVb1NOu9?=
 =?us-ascii?Q?FbRdcHQ4vYRJih16nnRcmbX83GiemIJOgh44R2WAbInoxoKPa104AZdT3H1Z?=
 =?us-ascii?Q?kQW79+bY7Ew+F/Lwgv87zm2tNtlMwRFQmSOAO/xk3R8UlScoJ4gzPavZl2+p?=
 =?us-ascii?Q?KHx0Y2AvdxkCu4QgHeIAe7QrNFtg1RmhEfw9JN1cCjOGlZUvgeZPscuPUHdt?=
 =?us-ascii?Q?wxP8/an7/qJOm3Kh6GvDrY09t7LqA4JOup1u5TeiQ77yE4b9YMwGgl7akhu8?=
 =?us-ascii?Q?zeNi3SO+XHsrmCWkbmpoWtQ75AOR84++yl0o7ZnOD7ZExJzhmmthWT5yzFe7?=
 =?us-ascii?Q?agtZZg8lgzHxnvaVkNrLjRl6oaHryNZtAdCIin60pLcdaxW17r+r2Pqt64e9?=
 =?us-ascii?Q?xVze7+DdsB2KZzYDRX4TvefDy+ra62cNAyPg9cZBKU6V3x1NAHEfLvcrtwGq?=
 =?us-ascii?Q?8B4zIJtapZlHnOY2J7XxCyT2Fs/3?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 03:24:06.2275
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2116035e-4bcf-482a-0f45-08dc7620cef4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000143.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8701

Currently the GUEST_MAGIC_BASE in the init-dom0less application is
hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
DomUs.

Since the guest magic region allocation from init-dom0less is for
XenStore, and the XenStore page is now allocated from the hypervisor,
instead of hardcoding the guest magic pages region, use
xc_hvm_param_get() to get the XenStore page PFN. Rename alloc_xs_page()
to get_xs_page() to reflect the changes.

With this change, some existing code is not needed anymore, including:
(1) The definition of the XenStore page offset.
(2) Call to xc_domain_setmaxmem() and xc_clear_domain_page() as we
    don't need to set the max mem and clear the page anymore.
(3) Foreign mapping of the XenStore page, setting of XenStore interface
    status and HVM_PARAM_STORE_PFN from init-dom0less, as they are set
    by the hypervisor.

Take the opportunity to do some coding style improvements when possible.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Only get the XenStore page.
- Drop the unneeded code.
v2:
- Update HVMOP keys name.
---
 tools/helpers/init-dom0less.c | 58 +++++++++--------------------------
 1 file changed, 14 insertions(+), 44 deletions(-)

diff --git a/tools/helpers/init-dom0less.c b/tools/helpers/init-dom0less.c
index fee93459c4..2b51965fa7 100644
--- a/tools/helpers/init-dom0less.c
+++ b/tools/helpers/init-dom0less.c
@@ -16,30 +16,18 @@
 
 #include "init-dom-json.h"
 
-#define XENSTORE_PFN_OFFSET 1
 #define STR_MAX_LENGTH 128
 
-static int alloc_xs_page(struct xc_interface_core *xch,
-                         libxl_dominfo *info,
-                         uint64_t *xenstore_pfn)
+static int get_xs_page(struct xc_interface_core *xch, libxl_dominfo *info,
+                       uint64_t *xenstore_pfn)
 {
     int rc;
-    const xen_pfn_t base = GUEST_MAGIC_BASE >> XC_PAGE_SHIFT;
-    xen_pfn_t p2m = (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET;
 
-    rc = xc_domain_setmaxmem(xch, info->domid,
-                             info->max_memkb + (XC_PAGE_SIZE/1024));
-    if (rc < 0)
-        return rc;
-
-    rc = xc_domain_populate_physmap_exact(xch, info->domid, 1, 0, 0, &p2m);
-    if (rc < 0)
-        return rc;
-
-    *xenstore_pfn = base + XENSTORE_PFN_OFFSET;
-    rc = xc_clear_domain_page(xch, info->domid, *xenstore_pfn);
-    if (rc < 0)
-        return rc;
+    rc = xc_hvm_param_get(xch, info->domid, HVM_PARAM_STORE_PFN, xenstore_pfn);
+    if (rc < 0) {
+        printf("Failed to get HVM_PARAM_STORE_PFN\n");
+        return 1;
+    }
 
     return 0;
 }
@@ -100,6 +88,7 @@ static bool do_xs_write_vm(struct xs_handle *xsh, xs_transaction_t t,
  */
 static int create_xenstore(struct xs_handle *xsh,
                            libxl_dominfo *info, libxl_uuid uuid,
+                           uint64_t xenstore_pfn,
                            evtchn_port_t xenstore_port)
 {
     domid_t domid;
@@ -145,8 +134,7 @@ static int create_xenstore(struct xs_handle *xsh,
     rc = snprintf(target_memkb_str, STR_MAX_LENGTH, "%"PRIu64, info->current_memkb);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
-    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%lld",
-                  (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET);
+    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%"PRIu64, xenstore_pfn);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
     rc = snprintf(xenstore_port_str, STR_MAX_LENGTH, "%u", xenstore_port);
@@ -230,7 +218,6 @@ static int init_domain(struct xs_handle *xsh,
     libxl_uuid uuid;
     uint64_t xenstore_evtchn, xenstore_pfn;
     int rc;
-    struct xenstore_domain_interface *intf;
 
     printf("Init dom0less domain: %u\n", info->domid);
 
@@ -245,20 +232,11 @@ static int init_domain(struct xs_handle *xsh,
     if (!xenstore_evtchn)
         return 0;
 
-    /* Alloc xenstore page */
-    if (alloc_xs_page(xch, info, &xenstore_pfn) != 0) {
-        printf("Error on alloc magic pages\n");
-        return 1;
-    }
-
-    intf = xenforeignmemory_map(xfh, info->domid, PROT_READ | PROT_WRITE, 1,
-                                &xenstore_pfn, NULL);
-    if (!intf) {
-        printf("Error mapping xenstore page\n");
+    /* Get xenstore page */
+    if (get_xs_page(xch, info, &xenstore_pfn) != 0) {
+        printf("Error on getting xenstore page\n");
         return 1;
     }
-    intf->connection = XENSTORE_RECONNECT;
-    xenforeignmemory_unmap(xfh, intf, 1);
 
     rc = xc_dom_gnttab_seed(xch, info->domid, true,
                             (xen_pfn_t)-1, xenstore_pfn, 0, 0);
@@ -272,19 +250,11 @@ static int init_domain(struct xs_handle *xsh,
     if (rc)
         err(1, "gen_stub_json_config");
 
-    /* Now everything is ready: set HVM_PARAM_STORE_PFN */
-    rc = xc_hvm_param_set(xch, info->domid, HVM_PARAM_STORE_PFN,
-                          xenstore_pfn);
-    if (rc < 0)
-        return rc;
-
-    rc = create_xenstore(xsh, info, uuid, xenstore_evtchn);
+    rc = create_xenstore(xsh, info, uuid, xenstore_pfn, xenstore_evtchn);
     if (rc)
         err(1, "writing to xenstore");
 
-    rc = xs_introduce_domain(xsh, info->domid,
-            (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET,
-            xenstore_evtchn);
+    rc = xs_introduce_domain(xsh, info->domid, xenstore_pfn, xenstore_evtchn);
     if (!rc)
         err(1, "xs_introduce_domain");
     return 0;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 03:24:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 03:24:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723792.1128895 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCe-000805-HK; Fri, 17 May 2024 03:24:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723792.1128895; Fri, 17 May 2024 03:24:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7oCe-0007zw-Dv; Fri, 17 May 2024 03:24:16 +0000
Received: by outflank-mailman (input) for mailman id 723792;
 Fri, 17 May 2024 03:24:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7oCc-0006l6-N9
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 03:24:14 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2408::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ee33ac71-13fc-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 05:24:13 +0200 (CEST)
Received: from CH2PR12CA0005.namprd12.prod.outlook.com (2603:10b6:610:57::15)
 by PH7PR12MB7209.namprd12.prod.outlook.com (2603:10b6:510:204::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 03:24:09 +0000
Received: from CH2PEPF00000143.namprd02.prod.outlook.com
 (2603:10b6:610:57:cafe::62) by CH2PR12CA0005.outlook.office365.com
 (2603:10b6:610:57::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Fri, 17 May 2024 03:24:08 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF00000143.mail.protection.outlook.com (10.167.244.100) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 03:24:08 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May
 2024 22:23:56 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 16 May 2024 22:23:55 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee33ac71-13fc-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n26oQiB3rqYQQhQY7AHV0S4Zl1cvg9yF0o0rMNei2YEX4kszHQevDgJy1nNlVNTBcFXhuCUH2OKzu7KVUd86vSRg/nn6BDu6PU8raGFGE//t3JgI5t06xjXABvIgSASX5WuXX8BqMQ0MqkMZdO+/L9Qou23lykiQB0jBwobPmvEbJfx/N/M/DIGsoipqNAthh2K6Q73PEJpjtCU9a54/CLR6NVjlBkUTgMKOWbFos8dSI7BTf481OjWXUpvAF9L6mi03aExjSrf5by/GwomtfFlLILwMa7xCu+qW0SDPNoGSgog24t3ICxD3iXQACzhXcNi9RNdAev/VG+zNQu0sVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=1u+M1MpJUGmCneiJ6tyWezkJk2s1bMCTX2NWTZTsw/0=;
 b=mjtzhKxESPc/Tdud9EX0lppgwWM6CqqXKDwKDB0o2rdOv5fLb+sGJ75NdpWPRkUEe/TuF62plOkMv5DQ9SdBvPn2IGbLEuOcsauOemei2OjMFU/aGzWVLMn21LwKXGFxqInS6SlM+eiQ5iycuPJInirGImmwHLqv4y/e5amchorSSRG8S2+ul+l+8JtowR2/2OceZFqca4wgKqHu0gu8Vollo8NZVzZiG1PSAwPL7WsMItbn6nyk9FuVGdOfFiq0MgMZW+7GiBKiQUpW6nTPKDJUnfvVY9lt1fr2tUQ/SQ92piJt/2bZn2V/NuWmLJioDqVApK7PNThifD0eirVwhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1u+M1MpJUGmCneiJ6tyWezkJk2s1bMCTX2NWTZTsw/0=;
 b=zgCaMAMQCRubaJCTtqToLscyBF7EEpGybk7Jihnxcq4ykznbPD+4sonFC7B86G0gm+t34qgPVeYJajPoTTSfMjCgj2os35BwlU2JJWRFZoKcR9wetvRuWdciz/0qMtsgWrF0GAh8KujWHrUQ9JdNEirZ4G1k9XdAmYvRkR77oE8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v3 4/4] docs/features/dom0less: Update the late XenStore init protocol
Date: Fri, 17 May 2024 11:21:56 +0800
Message-ID: <20240517032156.1490515-5-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240517032156.1490515-1-xin.wang2@amd.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000143:EE_|PH7PR12MB7209:EE_
X-MS-Office365-Filtering-Correlation-Id: dd70f450-ec54-4c0f-c236-08dc7620d06c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|82310400017|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?X2ampRtCEEh65iOCTCHgF9XZPwfHtYMd3/m7SyaVaJ8i/dguyoN3bVymRiOR?=
 =?us-ascii?Q?7T4NAeRVBegjjh81/fppiX2zczUy068D2Gn6UHwtPBUJ2cJLICFmKYqtYBw1?=
 =?us-ascii?Q?RSRTgn276VawDMoBb1qqbSDGMLOeBQD+n2X0XqCvMKh5UENsxV0l9HgweVru?=
 =?us-ascii?Q?1FGrNmSffQ2yPXw8RBkVbynrkwURoL+IpCV9Pj+dM90RMkMLmHzwN3z90Hy0?=
 =?us-ascii?Q?xEPPSDrG09B5B5M/GMMR8r0HrtmLdMdYXg87jcpo1KoDeY9CqlxNqjxr/Q5y?=
 =?us-ascii?Q?jtD6SrEOzfh0bn5XOi5PQepQuGZSB6rAuyfxa7oY09itSF/4kQ7uw1pouool?=
 =?us-ascii?Q?2ROU7zRJTXWTrmKj8HwRrlxK7TB++4Kde9VXsPcWPhV+GeRkJPToCqhp12a9?=
 =?us-ascii?Q?M3w6IK8vPZx452riFs8aUcaJUqtnwthlEHvWjS1WzHafi0EEjw3oDIX/eFes?=
 =?us-ascii?Q?q5X9zLrdxlwltAkNfrxwtgJoWHlk5CJAM72l2wpYgafsdu+9hfnwjeEMXlTe?=
 =?us-ascii?Q?b4JHtgGtdndK4RE9GuCUG7HjBwZYp29YiG5f9z+/ti0HH1z9PHMr9tmBAUup?=
 =?us-ascii?Q?GvfQsS+/mmTOw0Qhd2KanLaeSb2SduNbG2Q9EC4X7GLnq6IxMgrdwYZip7yB?=
 =?us-ascii?Q?AcwB+nKpY69Euk8dPbYaW6XKhFfe5tTsqUceH6Fy9C+r0Lp8vNjAfBLQw3D5?=
 =?us-ascii?Q?E/MZNhSUMFVNtDNluWavL8VgLBFfRScaD4ePE2MBTbHjTZyIyUzLKLovuhbr?=
 =?us-ascii?Q?7OwKLjgQ7ph0NJuOJystpV5O8yBve14FHJp5sFibHUXcQZ7Q271i5QoSYPA0?=
 =?us-ascii?Q?NVos41CE+AlAztRLAevkk0BPE+190ckI1phcEJfhIdrs55HZzxuv9zYd1AY/?=
 =?us-ascii?Q?K/Bs2cxBYk9m7KHIBeiBJJOlhlcMHlbR+frmf1K61O4DNWCukwGpJ4v3qp9P?=
 =?us-ascii?Q?tN2NEGOzKhem5ZJ5azWXnRYM4hqrLKFiAHvo2wm6MRuIDZjmUFWeJtNdYdxM?=
 =?us-ascii?Q?A1xw/ENu1jQBpq/sYEdn06uCEdEcU+StQuBI0rWc8WD5ayXs5xsRgY9XifWo?=
 =?us-ascii?Q?PUAMHa2HGlptdEVwIXritzPH0uogOiIrntmBWCRizQvg5ePC2Emnzcnlr+74?=
 =?us-ascii?Q?YSulqRHTbJXzE+wIYp5nKpLBagm2/Ltle7gRzvzhRpfWrL97/31stIXhLb+n?=
 =?us-ascii?Q?gkxwj5KJSZAAEwCMu5UhXJxWGohYE36d35WKhYBYS9A8Y5jNBvy0zAInUZyS?=
 =?us-ascii?Q?e98tfIYpJNy704yJUyLok75QCOk1PTyD47z/WI71J3JpqFgt59Ds7cG/m8sM?=
 =?us-ascii?Q?Gou/uGZhHEx5YvPjyO/3mP1+tqMj3eV3n29S5K5CE9S4TqLM1YgyVVq6Cfiu?=
 =?us-ascii?Q?34Y60s+tGwAraZ9IQ2G4OcQq16q4?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(82310400017)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 03:24:08.6806
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dd70f450-ec54-4c0f-c236-08dc7620d06c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000143.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7209

With the new allocation strategy of Dom0less DomUs XenStore page,
update the doc of the late XenStore init protocol accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Wording change.
v2:
- New patch.
---
 docs/features/dom0less.pandoc | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/docs/features/dom0less.pandoc b/docs/features/dom0less.pandoc
index 725afa0558..8b178edee0 100644
--- a/docs/features/dom0less.pandoc
+++ b/docs/features/dom0less.pandoc
@@ -110,9 +110,10 @@ hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.
 The implementation works as follows:
 - Xen allocates the xenstore event channel for each dom0less domU that
   has the "xen,enhanced" property, and sets HVM_PARAM_STORE_EVTCHN
-- Xen does *not* allocate the xenstore page and sets HVM_PARAM_STORE_PFN
-  to ~0ULL (invalid)
-- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid
+- Xen allocates the xenstore page and sets HVM_PARAM_STORE_PFN as well
+  as the connection status to XENSTORE_RECONNECT.
+- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to
+  ~0ULL (invalid) or the connection status is *not* XENSTORE_CONNECTED.
     - Old kernels will continue without xenstore support (Note: some old
       buggy kernels might crash because they don't check the validity of
       HVM_PARAM_STORE_PFN before using it! Disable "xen,enhanced" in
@@ -121,13 +122,14 @@ The implementation works as follows:
       channel (HVM_PARAM_STORE_EVTCHN) before continuing with the
       initialization
 - Once dom0 is booted, init-dom0less is executed:
-    - it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN
+    - it gets the xenstore shared page from HVM_PARAM_STORE_PFN
     - it calls xs_introduce_domain
 - Xenstored notices the new domain, initializes interfaces as usual, and
   sends an event channel notification to the domain using the xenstore
   event channel (HVM_PARAM_STORE_EVTCHN)
 - The Linux domU kernel receives the event channel notification, checks
-  HVM_PARAM_STORE_PFN again and continue with the initialization
+  HVM_PARAM_STORE_PFN and the connection status again and continue with
+  the initialization
 
 
 Limitations
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 06:03:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 06:03:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723822.1128904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7qgq-00030e-GZ; Fri, 17 May 2024 06:03:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723822.1128904; Fri, 17 May 2024 06:03:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7qgq-00030X-Dn; Fri, 17 May 2024 06:03:36 +0000
Received: by outflank-mailman (input) for mailman id 723822;
 Fri, 17 May 2024 06:03:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6mIz=MU=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s7qgo-00030R-KI
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 06:03:34 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2408::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ec51e20-1413-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 08:03:33 +0200 (CEST)
Received: from MN2PR05CA0008.namprd05.prod.outlook.com (2603:10b6:208:c0::21)
 by MW4PR12MB5603.namprd12.prod.outlook.com (2603:10b6:303:16a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Fri, 17 May
 2024 06:03:25 +0000
Received: from BL6PEPF0002256E.namprd02.prod.outlook.com
 (2603:10b6:208:c0:cafe::d7) by MN2PR05CA0008.outlook.office365.com
 (2603:10b6:208:c0::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.12 via Frontend
 Transport; Fri, 17 May 2024 06:03:25 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF0002256E.mail.protection.outlook.com (10.167.249.36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 06:03:24 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 01:03:24 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 01:03:24 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 01:03:22 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ec51e20-1413-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BFlxTWr36NYgGxGk79I7nvb+ssd4dylNKZXvQQmYgUlqcF9yu6oyJ0/eCMq1DmxxL+Ff8dt7Q3WUsg9t8qkGn5HoVDIpwaX5W3WDZzi66uN27Fbzak1XaiBUc7DbNfZrpy7qvkrCakHa8n36TgaYNJ8j3QxxE7X/iS7CSsM04CukPpd2zaeuGuCYqKifML/zAjz8b6iHgJ6w+4vJzToomSbLvRt4iXDoavToQcpDySvyJZ5XHmd8gIA2h2GdT3WHgOsHO5/QVBZty5N2OhzIdcX2yzkHW3Lc1ZJpIpOMrORVnq1zW54Hv1Q4E12rkVB2/iwbqUrksJcUx2oLQxTaEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EQSB1ASWz/oXaBldxTRIk2Q7eTJHYhI6okfoX6IssWk=;
 b=i3fcibowIv9WzAjghPvchLRmuErsRpm6+StI/YDPQ5GtszxW8AteHJJB3n6pnKUTccMKPk7S2Qh2ZWbmN4JpzQT1EWmx+jhMQkUl4VJvHenqFdrmQOJl4P2Czb7TAgB86nR9RB35EgO12XlXwyxqSqNWN8y38OuhqDJjsNrj/79Y62XMiyAhOFuno5NvV6vrWlUlPTwXyHtK5NtcXpf9hwkIUIXlUp5MyoS1+bGoclXYytHe1v6D//j09PQ9k67PvTj3g7D8jeJhFKNdEq6t2ipWdtylDqG5hPM4lN7A7SfVr0ikw7DcRWjC8jqGOXU5OUsmLa2E0EyOxoKsoSdHiw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EQSB1ASWz/oXaBldxTRIk2Q7eTJHYhI6okfoX6IssWk=;
 b=XJW1IUv/T7Xrsz8T2UyR1L2wJCqqUIYkGVUZTF1j8nwWH3iqVUTmatnrcDvjJlTQEQd7AdERXCFs4Ypt/v/G9gKpzABCOVRtRgH+Iv+QphiMASrV+3zCPjz0rmldNk6reHfmzEc8vak4zLPG6uQW8o1M7HYf7+HxUHn4+8624ZM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com>
Date: Fri, 17 May 2024 14:03:21 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-6-xin.wang2@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <20240516100330.1433265-6-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0002256E:EE_|MW4PR12MB5603:EE_
X-MS-Office365-Filtering-Correlation-Id: f9e1b45e-4504-4a11-8e38-08dc7637104b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QXRCY1pyazdGV1FMQ1ZRL3JRSlNoa2JLd3RKQTd6ajFzbHVhY0JhbittOHFT?=
 =?utf-8?B?cFdYN1hDK2RoUnZjQVJDcExYTkpVVDhoM2RMWUVJLzhmVnZUZzRhNzkwYzJ3?=
 =?utf-8?B?TVJBQk1RNFk2ZFJZbXVEbG90RXlSM1pHWHRxM0piN0g4QkJCV3RMS3lacXVt?=
 =?utf-8?B?RGU5OWgxRTFncTVJbVEwUlVUUnF0M1Bybm9PQkpiRVVELzlwbEJrNTEwUzVz?=
 =?utf-8?B?NStLamczbGdhOUFmbDZkdDk2Ti9MeFNhVVA5S1drTXZjak5vbWdTU2FMdXMz?=
 =?utf-8?B?K1N6cC82MDB5dVZRd0Rkdm5KSDBaZElaekY2R1hNaytlVHI1QklUbUZSeWdp?=
 =?utf-8?B?U3FYZUM5SnIzbzViYUpBdU9qRGU1Nld3Z1ZkZTJrS1hEb092VU5ndnNXN1Mz?=
 =?utf-8?B?Sml1Uk0rMzZOZmQvWTFqM29FTVNDY1hYdmFiTHprZDdxbm55YW1qN2dJSFdZ?=
 =?utf-8?B?K1J1T01XQTZEM0pRa3JEOGJ1a2J2UWdlY1ZNUUtFSWFDSm5SaExJN29TVks1?=
 =?utf-8?B?U1U3aU9IN2RYMEFrT0w5YzFGQUJFTHNIWHphOWxYbXl3dndibFczODNaeGN4?=
 =?utf-8?B?eDZiTnh4b2hsNmZYT1BCMEg0ZW15OFEyemlZQ2gzR2JkbkhOOUx6S09oOVIy?=
 =?utf-8?B?MHVsWGpvd2VFTFB4WC9wZkc5aDdDRDl3MFVkNFdyZVB3Mlh3Yk5wdEZYaTF4?=
 =?utf-8?B?L3RzOEdXczdWWkwrejAvSS8ybVBseXZmTi9GdEVhVlVkUm5xVDM3Z3d0R25P?=
 =?utf-8?B?eDRlQ2N1UjBTdEM4SllNWkFYRXpGaGoyZmhmZHRFd2VVV2Z4NnRFTkY2dCtn?=
 =?utf-8?B?Qjl4K0lsOUc2Q09jdWpyOVJ1RDZwcytibTNnVG9HYWlQTURvTlZkZGt1Si9y?=
 =?utf-8?B?Uk9aTy9Nc1pCRk9LQmE1c0xmbUlna2J2cmxUTEdmMjd4M3pyRnIyNzFJUnhB?=
 =?utf-8?B?OXJ6Y0tPbnd5QWhlMjlmQjVDbEc1emhZUndXek1xMFJWN21jSHVzYTdnRFd0?=
 =?utf-8?B?cUxQMGZ1RmIxSHpqazBLeHZMdDU5M3lKU3hBaG5PSmNDMHAzS2VYNkdJQTll?=
 =?utf-8?B?ZWdLNGcyQ1I2bXZJSmxNVC9yZzcxaW9xb3Z5UjZ2WC9yRS95Sll0aThsd25E?=
 =?utf-8?B?ZVQyUnF0VEZaNmRBVXhEc2FuNDhtaDc1a2tXQ2RNYUhpdGhnWHBJN1pscFRz?=
 =?utf-8?B?YkczS0xIR0tHTWZTaHB3Mi9WT2k3Tkt3WThubWYxUEdoWXNXSDEwRThpYkwr?=
 =?utf-8?B?Q0kwQ2NFaTdtSjcrV3ZDekt6RmVDbEk4R0RmbEZHOGdUTGNoTlhlS05tSmFt?=
 =?utf-8?B?bDZXdDB2QTRlVmdDQUdUYVVmSlJGSWg5UHpsK21XUnRlL1RtVFZpM1VGa0NX?=
 =?utf-8?B?bDNWNFk5Y0d0NGNCTnpRSExvK3ZyR2x4YzJDcG0vQlp0bXVOd3pnMHljRXZT?=
 =?utf-8?B?c3RMQWtIV25VdXRwS0lEMmV4TG85cHRtTkw1VXROSlc5dUplSTkyYmMwNkJn?=
 =?utf-8?B?YkVpQWlnQnN5VTN0MVBlMUxUcFczWGxNL0RGNnAyTjc5WDNvdEV0MDJvM3Ex?=
 =?utf-8?B?V0cxcStFTEt0SDdhNkdCQjNvY1E1aEJCazY4cHhaSEFVWDl4RTFvNm5PbVpa?=
 =?utf-8?B?RWI4SWxvenJtdnFXTDF6Y3drbUI4LytmeXVma3F6S0U5VVRKczEyaFBKZFVv?=
 =?utf-8?B?T3Q5NEhia2NDSytwenVyRFFvaUFGbENHTk8xSFE4MWlOTkRIOW92ZXlhQ2Zx?=
 =?utf-8?B?Z1VRY3JQRHNBWnFqOEQrSWFyZ01aZ0pKZjU5eEVleW1wN1dTaU1pb092YkEv?=
 =?utf-8?B?WmVNdFpmTHgyYVF0UkU3bXRBU283ejJuM2dyNyt0TEloU1RhOHJWamdCWUJJ?=
 =?utf-8?Q?tlQW163A2sS0k?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 06:03:24.8121
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f9e1b45e-4504-4a11-8e38-08dc7637104b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0002256E.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5603



On 5/16/2024 6:03 PM, Henry Wang wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
>
> Currently, routing/removing physical interrupts are only allowed at
> the domain creation/destroy time. For use cases such as dynamic device
> tree overlay adding/removing, the routing/removing of physical IRQ to
> running domains should be allowed.
>
> Removing the above-mentioned domain creation/dying check. Since this
> will introduce interrupt state unsync issues for cases when the
> interrupt is active or pending in the guest, therefore for these cases
> we simply reject the operation. Do it for both new and old vGIC
> implementations.
>
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - Reject the case where the IRQ is active or pending in guest.
> ---
>   xen/arch/arm/gic-vgic.c  |  8 ++++++--
>   xen/arch/arm/gic.c       | 15 ---------------
>   xen/arch/arm/vgic/vgic.c |  5 +++--
>   3 files changed, 9 insertions(+), 19 deletions(-)
>
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index 56490dbc43..d1608415f8 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>       {
>           /* The VIRQ should not be already enabled by the guest */
>           if ( !p->desc &&
> -             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> +             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> +             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
> +             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
>               p->desc = desc;
>           else
>               ret = -EBUSY;
>       }
>       else
>       {
> -        if ( desc && p->desc != desc )
> +        if ( desc && p->desc != desc &&
> +             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
> +              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )

This should be

+        if ( (desc && p->desc != desc) ||
+             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
+             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )

> @@ -887,7 +887,8 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
>       }
>       else                                /* remove a mapped IRQ */
>       {
> -        if ( desc && irq->hwintid != desc->irq )
> +        if ( desc && irq->hwintid != desc->irq &&
> +             (irq->active || irq->pending_latch) )

Same here, this should be

+        if ( (desc && irq->hwintid != desc->irq) ||
+             irq->active || irq->pending_latch )

Kind regards,
Henry



From xen-devel-bounces@lists.xenproject.org Fri May 17 06:58:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 06:58:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723828.1128914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rY4-0003BU-8n; Fri, 17 May 2024 06:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723828.1128914; Fri, 17 May 2024 06:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rY4-0003BN-5k; Fri, 17 May 2024 06:58:36 +0000
Received: by outflank-mailman (input) for mailman id 723828;
 Fri, 17 May 2024 06:58:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7rY2-0003BH-1G
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 06:58:34 +0000
Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com
 [2607:f8b0:4864:20::f32])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df5c15fd-141a-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 08:58:32 +0200 (CEST)
Received: by mail-qv1-xf32.google.com with SMTP id
 6a1803df08f44-6a05f376effso2737606d6.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 May 2024 23:58:32 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1851f7sm85036176d6.47.2024.05.16.23.58.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 May 2024 23:58:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df5c15fd-141a-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715929111; x=1716533911; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=atCC8Hh/k8kCgrNGdLCBtHARPiZ9rYxrpcX+dCOV98c=;
        b=l8xTBc7y44TAxi0jvIOXTTCiVVVI7RIJcTFFuPevT+pWMLFNDZOOHWQ22MNY2MgrVN
         1XJ7rQet4ENI1C7Womz0eyY1uCz8rcLA2kgSrFuPlGKxtZ07dC9ux+vnfk4IV5siyefD
         LVj2O9wp0dPW/8mxizxsD80IApTAhQ/3CyeSM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715929111; x=1716533911;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=atCC8Hh/k8kCgrNGdLCBtHARPiZ9rYxrpcX+dCOV98c=;
        b=U51iorqV3z/Bi7aKJEXhGY3N3E5erIjE4ILxeWRAnZo01P9b81F4Ewl5/0iGNMyck1
         RIHo7263vQMdOaFbexOGVuVccAIHBAW/N3ly6rBJrUL2gVNbLojiyPwtzP53S0he445K
         dmt8Bei6V5lAY2yna60co9lygV5XUNUJlHoz7rjoa216pupoGjwbSpUnqfYhkRQmu8SH
         F+WAhtrIizQwPx/5NcWdXCZKBYn/iYOGWwt8xkvpavJx0kz6FMvB36rK7xomGhCH58gD
         6sOUidcNKxnXAnNi1S3NN8hjx1Mb8okIq/ciw9GoquGdn94i4AASQSpkKhwcI6zeBtBG
         G/VQ==
X-Gm-Message-State: AOJu0Yxt7xdzHO8hFtNBBzcKMO/ggJMXPNyBvLjh0eZcLdyipBCBu5tq
	1zNM+JD6yiADvKvnFZQb+1hM3OjOk3Ff1qLjYKt7KWIg+NWgkydeEdjUZBI6yrLse00y3qeeWWp
	v
X-Google-Smtp-Source: AGHT+IESzcvBDK6hgZ9zAvEbRjpBX976oKsp+KFhklEGtv5CyOJYgVxAVojTv8ng6N8R/r9nLVL9Ug==
X-Received: by 2002:a05:6214:3f81:b0:6a0:caef:3c4b with SMTP id 6a1803df08f44-6a16822f259mr256488696d6.48.1715929111419;
        Thu, 16 May 2024 23:58:31 -0700 (PDT)
Date: Fri, 17 May 2024 08:58:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity
 masks
Message-ID: <ZkcAFfVpxH4iGFDZ@macbook>
References: <20240515152925.77197-1-roger.pau@citrix.com>
 <06844367-cc1d-42f7-b925-ba78b4532b11@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <06844367-cc1d-42f7-b925-ba78b4532b11@citrix.com>

On Thu, May 16, 2024 at 06:13:29PM +0100, Andrew Cooper wrote:
> On 15/05/2024 4:29 pm, Roger Pau Monne wrote:
> > Print the CPU affinity masks as numeric ranges instead of plain hexadecimal
> > bitfields.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  xen/arch/x86/irq.c | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
> > index 80ba8d9fe912..3b951d81bd6d 100644
> > --- a/xen/arch/x86/irq.c
> > +++ b/xen/arch/x86/irq.c
> > @@ -1934,10 +1934,10 @@ void do_IRQ(struct cpu_user_regs *regs)
> >                  if ( ~irq < nr_irqs && irq_desc_initialized(desc) )
> >                  {
> >                      spin_lock(&desc->lock);
> > -                    printk("IRQ%d a=%04lx[%04lx,%04lx] v=%02x[%02x] t=%s s=%08x\n",
> > -                           ~irq, *cpumask_bits(desc->affinity),
> > -                           *cpumask_bits(desc->arch.cpu_mask),
> > -                           *cpumask_bits(desc->arch.old_cpu_mask),
> > +                    printk("IRQ%d a={%*pbl}[{%*pbl},{%*pbl}] v=%02x[%02x] t=%s s=%08x\n",
> 
> Looking at this more closely, there's still some information obfuscation
> going on.
> 
> How about "... a={} o={} n={} v=..."
> 
> so affinity, old and new masks are all stated explicitly, instead of
> having to remember what the square brackets mean, and in particular that
> the masks are backwards?
> 
> Happy to adjust on commit.

Sure, I guess I got used to it and didn't think of adjusting the
format.  The only risk is anyone having an automated parser to consume
that information, but I think it's unlikely.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:01:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:01:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723831.1128924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7raz-0005JP-MI; Fri, 17 May 2024 07:01:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723831.1128924; Fri, 17 May 2024 07:01:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7raz-0005JI-JH; Fri, 17 May 2024 07:01:37 +0000
Received: by outflank-mailman (input) for mailman id 723831;
 Fri, 17 May 2024 07:01:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7ray-0005J7-VI
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 07:01:36 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4cd048c9-141b-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 09:01:35 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a1054cf61so473541366b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 00:01:35 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01598sm1091572566b.178.2024.05.17.00.01.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 00:01:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cd048c9-141b-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715929295; x=1716534095; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RBG75lcdtAlTkY8FLRz4q0hO7wLgWZckS6elinM+HvQ=;
        b=Jli8GZOrXky/s/O46dppWO6pMrVC7lFc7cVoDXEXPjs4oWmzA58rkJuZvfxCVn33bP
         efrNEvbVEwC7GjHh6Jk0iXBdK0PghUqc8H8MhMcd7wmICciFLqy1oRp3Z6eSX2yYQjnk
         +ujn3e/rwsbrksVYNnUcoD50a/4G3M+tyVLhI+8O+Ylo4HSu1MAU6OXh2EJA5QIFWCMq
         ZBEuqx7sTsmfvga8Ucgb00QzYhje+W0FTtcaotkvrHgo82iuoF7C6iVEHeWulBDo9PwG
         7Fe39hRS3fw+RW/zFmU/gugTwqbTHGwsYFDEUeilzk9wlf7r8BI3w0Jv6jp+stmtSvdk
         oMWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715929295; x=1716534095;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RBG75lcdtAlTkY8FLRz4q0hO7wLgWZckS6elinM+HvQ=;
        b=dHkBVHHnP51Aevw9P8Onpy/7PgKg+NAWE3mhuEF+10EPO2MRppYEdoswLez8fwzFcO
         yyz1DFgIkCCYISOgPKCpA63Wxr0uvdL1ZC3VqRPH9uaC1XqSeQo6gsUTeHrc5HKIzDbP
         CrZL2DV6aQB/yCwmDdeXwHsAA/0imMOWbMENhTMtFqSAtmcPAXk3N1XMU+hb0kRS176j
         CvTsq+5y0K32BRvBwFa8bWQiDOBT2Eyi5rquLzogsXW8Rp1t9F2WWEUgnpIXs8nrBHFj
         ysQ9gU9QO9f6FqsSS0IBpFAiu5MdhZOo2roxLYQfV2l4TGFh2q/kEjoG+D0+t1wUQgaQ
         gWTQ==
X-Forwarded-Encrypted: i=1; AJvYcCXp+PHOjEVfz1jgpeztasrTd0Olr1khhO3r/YQ0nWt0UnI+iSo556ID8LvuyiYBE6UnZz625OQFQKmMwneQQjvd9aUbH2EaY7egT17PWhI=
X-Gm-Message-State: AOJu0Yz8Aav+mhN6smjcEhfJLmh5ZERq1AZw+/edvFg2e0GqBk7LIAxJ
	SjQ3VcxBR/Q+4XFukdL3uiofHXAhxvvfWV0YpH5KfkxT0vaIo9ke8yuR6mwirA==
X-Google-Smtp-Source: AGHT+IEoIsORyWeGFwAyH+EOBDjJhJDuTDwtidrBdXnxnTZ+yPh9wAoHGMfUoU2k3zO376d/hz5Lsg==
X-Received: by 2002:a17:906:3c10:b0:a59:cc9b:d6f8 with SMTP id a640c23a62f3a-a5a2d5cca1cmr1942070966b.39.1715929295152;
        Fri, 17 May 2024 00:01:35 -0700 (PDT)
Message-ID: <afac644e-240f-47b6-be01-699b05e853dc@suse.com>
Date: Fri, 17 May 2024 09:01:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Daniel Smith <dpsmith@apertussolutions.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 "committers@xenproject.org" <committers@xenproject.org>,
 Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <1ba4d87f-48e2-4332-bcd8-ac891cee8917@citrix.com>
 <ccd70c8f-1381-43f7-8be8-edc1dd505ce2@suse.com>
 <9249bc7e-7ca9-4f6e-8f3e-f2962c14e306@xen.org>
 <66ac3d6aaf0e393ce76580f274c222d26c0aa0a1.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <66ac3d6aaf0e393ce76580f274c222d26c0aa0a1.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 21:15, Oleksii K. wrote:
> I am not deeply familiar with the technical details surrounding XSM,
> but if I understand Daniel's point correctly, the original change
> violates the access control framework. This suggests to me that the
> revert should be merged.
> 
> However, I have a question: if we merge this revert, does it mean that
> using channels a user ( domain ) will be able to get information about
> certain events such as EVTCHNSTAT_unbound, EVTCHNSTAT_interdomain,
> EVTCHNSTAT_pirq, EVTCHNSTAT_virq, and EVTCHNSTAT_IPI (based on the code
> of lseventch.c)? Is this information really so critical that it cannot
> be exposed for some time until a patch that changes the XSM policy
> consistently is provided and merged?
> 
> If this information is indeed critical and should not be exposed, I
> think we can consider Daniel's suggestion to add a check to the dummy
> XSM policy as a solution.

The question isn't whether it's critical. As pointed out elsewhere, my
view is that any exposure of information needs to come with a proof that
nothing undue can be derived from that information. I see Daniel has
responded there, so we'll continue the discussion there.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:05:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:05:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723837.1128945 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7reK-0006Bd-EO; Fri, 17 May 2024 07:05:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723837.1128945; Fri, 17 May 2024 07:05:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7reK-0006BT-BB; Fri, 17 May 2024 07:05:04 +0000
Received: by outflank-mailman (input) for mailman id 723837;
 Fri, 17 May 2024 07:05:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7reJ-0006AW-1U
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 07:05:03 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c784a105-141b-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 09:05:01 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-573061776e8so4400249a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 00:05:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c322c85sm11653972a12.84.2024.05.17.00.05.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 00:05:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c784a105-141b-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715929501; x=1716534301; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8PHNZgILOy1MQlCOuS7WwvW8i7CptPWYzUVwIn8u8o8=;
        b=J4GV4UEvQc3163k/oEFnow47zgqV7fBcH/zKl4jmx952h3VMSO/lj/XHVWVne5yTpq
         Z8ddArtmtd774LmQUtQF97uIGFKhIYrlgd+lAlyH4Kxq2/Yagt7aes4phDMMDNJstLp1
         /2fuFDDGvZT8usYXzY4B5Y7DpPmBbWPkmysaF3HMe2NyoaIwIcSq4W62oWUuCHAFGucI
         /VyJQCnfBiG1pw9YX+EKaA20KCjQQMJxQsA4hmlmbIuZN1N4NKLqkwmW5NrSnG7Mh6w9
         votfXGPDVQ/Oj3kMUOI002nrK/fxOKnXhAM2GjaX2GOiVCLb2sS1QU5bSWujnWlSagdM
         aGmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715929501; x=1716534301;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8PHNZgILOy1MQlCOuS7WwvW8i7CptPWYzUVwIn8u8o8=;
        b=rBvlhA5zU6ZucZ8Ew5OTxA8VFEz36Bxnrcgs0AOEYcxjxYeiniIbXG3ecvGw1DwkT+
         6FKyK0+U/rAZ0V5YarUv8Tpc0GI3pNCAsi8dpt7WbSoPPotOnnu/2dfCAV030rJPF2g8
         rz5/iP1lNhE7a73GQ24OqhdXiamXhb41UITHcpexhPCQtSCG8VsHlKA1AFXCRQ/HGOJq
         +9pFov1KpfBZFwjDRvwIzcgGkXtOBULNkOjZCqg6xzxobn6CEp1ZRs2Engr9zq7GOo5b
         Lhy+3hk+a6GuhpBwCdGH2GC9NOVHgVawh8JSOhdSNlSa07zmCcgHUS1jJpuIQHvwqu3N
         bi4Q==
X-Forwarded-Encrypted: i=1; AJvYcCVSnumxsCmHaSWgcJRsc2bzPsVb/vX2V9qcMPejG9XZc7aQbdkvvYlHlnRF50/ZestpNo17aKz/rObKgzWHpmA3jxdzr84IpLU8F58WsV8=
X-Gm-Message-State: AOJu0YzJD8Do8LM2VymOUYydhRfxDaURKI92puWdHfYKaS2/eflis4G2
	17Tc3/I7HJ3EWbe++rw1oRDDaVj/OWtCX0L5xpaIobLBch4XrllNfticx5P38w==
X-Google-Smtp-Source: AGHT+IF1VxgxyluUeMT7KnNXDwt9w+MqU/EYunaTGPdvIyK9Ic+rQmge7/v10YRTY5lnXt7dX5Lr7g==
X-Received: by 2002:a05:6402:3199:b0:574:d098:37ae with SMTP id 4fb4d7f45d1cf-574d0983832mr7287069a12.4.1715929501168;
        Fri, 17 May 2024 00:05:01 -0700 (PDT)
Message-ID: <e75ca24e-12fe-44ac-9c67-bcc222ac8752@suse.com>
Date: Fri, 17 May 2024 09:04:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Daniel Smith <dpsmith@apertussolutions.com>, Julien Grall <julien@xen.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
 <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 03:21, Stefano Stabellini wrote:
> On Thu, 16 May 2024, Jan Beulich wrote:
>> 1) In the discussion George claimed that exposing status information in
>> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
>> how a similar assumption by CPU designers has led to a flood of
>> vulnerabilities over the last 6+ years. Information exposure imo is never
>> okay, unless it can be _proven_ that absolutely nothing "useful" can be
>> inferred from it. (I'm having difficulty seeing how such a proof might
>> look like.)
> 
> Many would agree that it is better not to expose status information in
> an uncontrolled manner. Anyway, let's focus on the actionable.
> 
> 
>> 2) Me pointing out that the XSM hook might similarly get in the way of
>> debugging, Andrew suggested that this is not an issue because any sensible
>> XSM policy used in such an environment would grant sufficient privilege to
>> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
>> for Xen-internal event channels. The debugging argument then becomes weak,
>> as in that case the XSM hook is possibly going to get in the way.
>>
>> 3) In the discussion Andrew further gave the impression that evtchn_send()
>> had no XSM check. Yet it has; the difference to evtchn_status() is that
>> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
>> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
>> similarly useful to allow getting a stuck channel unstuck.)
>>
>> In summary I continue to think that an outright revert was inappropriate.
>> DomU-s should continue to be denied status information on Xen-internal
>> event channels, unconditionally and independent of whether dummy, silo, or
>> Flask is in use.
> 
> I think DomU-s should continue to be denied status information on
> Xen-internal event channels *based on the default dummy, silo, or Flask
> policy*. It is not up to us to decide the security policy, only to
> enforce it and provide sensible defaults.
> 
> In any case, the XSM_TARGET check in evtchn_status seems to do what we
> want?

No. XSM_TARGET permits the "owning" (not really, but it's its table) domain
access. See xsm_default_action() in xsm/dummy.h.

Jan

> evtchn_send uses XSM_HOOK, which is weaker, but it doesn't seem to be an
> issue because (ignoring the consumer_is_xen check) there is a if(!lchn)
> check that would fail on invalid event channels?



From xen-devel-bounces@lists.xenproject.org Fri May 17 07:05:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:05:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723834.1128935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7re9-0005tS-3q; Fri, 17 May 2024 07:04:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723834.1128935; Fri, 17 May 2024 07:04:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7re9-0005tL-0x; Fri, 17 May 2024 07:04:53 +0000
Received: by outflank-mailman (input) for mailman id 723834;
 Fri, 17 May 2024 07:04:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7re8-0005tB-0a; Fri, 17 May 2024 07:04:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7re7-0007dm-TF; Fri, 17 May 2024 07:04:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7re7-000287-JS; Fri, 17 May 2024 07:04:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7re7-0007f4-Ix; Fri, 17 May 2024 07:04:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uyHS+ewU+DBHcBCi74sNOXOwGsUFJxTsxBQ4L1PgsPk=; b=f33gznoCWFybCVjjz/GTkd6sP4
	1agG1wFRpnjtKeyY+DyZRTPWh+3WTTg8obukHp9tjqC7ow4ydPKTomBpK+1UX9SDQWSyGTWr+4xDT
	93wCObVtuIJ5FqIOPuAzNPs832rUEvo7tRBYZFU+aJ6ZKvj46w8/dmr8UFb2ZqR7VIEI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186021-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186021: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ae7584f63678cd9adc1c2f3a1e813b67a6b24544
X-Osstest-Versions-That:
    xen=762a848223e0fd61aa87b4e040c83dd2c490fb79
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 07:04:51 +0000

flight 186021 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186021/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186013
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186013
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186013
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186013
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186013
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186013
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ae7584f63678cd9adc1c2f3a1e813b67a6b24544
baseline version:
 xen                  762a848223e0fd61aa87b4e040c83dd2c490fb79

Last test of basis   186013  2024-05-16 08:20:07 Z    0 days
Testing same since   186021  2024-05-16 21:08:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Rahul Singh <rahul.singh@arm.com>
  Sergiy Kibrik <Sergiy_Kibrik@epam.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   762a848223..ae7584f636  ae7584f63678cd9adc1c2f3a1e813b67a6b24544 -> master


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:05:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:05:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723843.1128954 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7res-0006tn-LQ; Fri, 17 May 2024 07:05:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723843.1128954; Fri, 17 May 2024 07:05:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7res-0006tg-Iq; Fri, 17 May 2024 07:05:38 +0000
Received: by outflank-mailman (input) for mailman id 723843;
 Fri, 17 May 2024 07:05:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7rer-0006tS-22; Fri, 17 May 2024 07:05:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7rer-0007ed-00; Fri, 17 May 2024 07:05:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7req-0002AZ-Ml; Fri, 17 May 2024 07:05:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7req-0000Qh-MC; Fri, 17 May 2024 07:05:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9186pL3Ln7FaWUiwxcV0cvY5bTsG0Wku7XwT5bdVcjo=; b=S6nJ+z2RHq4nKApTzrzNm67Yhu
	P0LjfHY84bQS6lqQeHXfnBB7rrFZqLyRjqbRAItU82riOdXvas742Mm6AsDMO0Sq1QJHZgXowcH6s
	2ucBKwzuFJzWNP2+eSVEunpYdIFOdo32rgMQ0KprWRmq6R33545TuEWxCLuhR6dUl3+U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186026-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186026: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=284dbac43da752ee34825c8b3f6f9e8281cb5a19
X-Osstest-Versions-That:
    ovmf=558a25366d66e9ff96784ed3aab476f808531bf3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 07:05:36 +0000

flight 186026 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186026/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 284dbac43da752ee34825c8b3f6f9e8281cb5a19
baseline version:
 ovmf                 558a25366d66e9ff96784ed3aab476f808531bf3

Last test of basis   186000  2024-05-15 01:41:14 Z    2 days
Testing same since   186026  2024-05-17 05:43:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Pakkirisamy ShanmugavelX <shanmugavelx.pakkirisamy@intel.com>
  Shanmugavel Pakkirisamy <shanmugavelx.pakkirisamy@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   558a25366d..284dbac43d  284dbac43da752ee34825c8b3f6f9e8281cb5a19 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:11:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:11:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723860.1128989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rkw-00018s-Lr; Fri, 17 May 2024 07:11:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723860.1128989; Fri, 17 May 2024 07:11:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rkw-00018i-Ib; Fri, 17 May 2024 07:11:54 +0000
Received: by outflank-mailman (input) for mailman id 723860;
 Fri, 17 May 2024 07:11:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7rkv-00017n-US
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 07:11:53 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bcaac4a5-141c-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 09:11:53 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-574b3d6c0f3so4449237a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 00:11:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574ea5d755bsm4554916a12.51.2024.05.17.00.11.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 00:11:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcaac4a5-141c-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715929912; x=1716534712; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mq6i4DzkB4Dc/X42+OKA6whNauQKkseIsPXHgnpnpVo=;
        b=bYAfR+Ch6B5u47ZshPoOaEg+d3UY+3vemklcvVb8YKRm/Hiv0SO0mwi1pSczjmm4A8
         K6uK328gquA52QtFLvtbA9JTHcaftAgqQH7I6MDAjuXMsih1KXKTUbvwZkChUG5XKkPB
         fBMjG9/BZGMcRmluydSfBc0wo4prfGVFtSmkEOSIZogOr+SfjWbQye5DvMwzp9wxy9gz
         a7LEmQud44WG0dc0N/k/9ylqcEgW00ua3Iv60NiklWGhofW+Ze6I0tDyYPviLQBqyRIZ
         iOK4RjfmIcfSg8pbQ74aNmLrqWtRui8nHSM/91WwdIwY+IoJBUl/gEclVGpDR0Qu4h1l
         3uag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715929912; x=1716534712;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=mq6i4DzkB4Dc/X42+OKA6whNauQKkseIsPXHgnpnpVo=;
        b=wjKbVKvPAB28/zVMWNkyB+9khDkBzN8JA1hXF1BiXkjgti2ZR6D9BwgphoWEUlIq0x
         nP2sl0TgoLOlGeNIyHyREnv9ZZaBuXt6oNL9HVnKDz2PI6UyFrYrYkyIDUocmPPZYLt4
         ud+3fNNphrkKCQvo0zNIQ+CdFbweIKGQPASzieEuakhAAKugIfZiWz2Sh7Ru26+1k62L
         i2fW4JTh/xIDnabLfTRh/8BCSgRarB9LMQsSluNBlSbhGzg4tdd7vJ0PqGSRe91jEPM9
         HuYc1UJ6+2pI0Zvcc5iTnJV7JlSi6EE9LOAWSEAcwcL8sE57YukzNblhRH0ZiAM85i9a
         1NvQ==
X-Forwarded-Encrypted: i=1; AJvYcCXPW2dPLkcWWkC8EdTADc9X+6PX1khHJhqGZYUi1vKcpbA53el+duDInl3b+ZxMlyM58y827gLwzPoKTF4x26FHCNKLyLqTmC1S5iZIrx0=
X-Gm-Message-State: AOJu0YzpOQcTWE4UM9lJVQyhoalRNJI00mhZLCf14eq6UZyuI0BGrnIP
	VHf5J84Z17TYKsStBm16C/blwIPWVpxvD+GenBdIix5zwqYz99ihhKmvFpB5JA==
X-Google-Smtp-Source: AGHT+IEIUrzqOFbJxxYCYchsoxhDvpeHH+aiBmXSEt7Ii4MXzMMB8rJFNbYmT9TkGsPLqhAuXp1EiA==
X-Received: by 2002:a50:9b42:0:b0:568:c6a2:f427 with SMTP id 4fb4d7f45d1cf-5734d67aa84mr14606663a12.27.1715929912490;
        Fri, 17 May 2024 00:11:52 -0700 (PDT)
Message-ID: <5a979a5b-51c0-4359-813e-d9307932d8f9@suse.com>
Date: Fri, 17 May 2024 09:11:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and
 related operations
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, xen-devel@lists.xenproject.org
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-7-xin.wang2@amd.com>
 <28138e7d-26c7-4faa-86f1-a1c7b175114b@suse.com>
 <1f02615c-916c-4ab8-babc-17a117c56e87@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1f02615c-916c-4ab8-babc-17a117c56e87@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 17.05.2024 03:36, Henry Wang wrote:
> On 5/16/2024 8:31 PM, Jan Beulich wrote:
>> On 16.05.2024 12:03, Henry Wang wrote:
>>> --- a/xen/include/public/domctl.h
>>> +++ b/xen/include/public/domctl.h
>>> @@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op {
>>>   typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
>>>   DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
>>>   
>>> +#if defined(__arm__) || defined (__aarch64__)
>> Nit: Consistent use of blanks please (also again below).
> 
> Good catch. Will fix it.
> 
>>> +struct xen_domctl_dt_overlay {
>>> +    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
>>> +    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
>>> +#define XEN_DOMCTL_DT_OVERLAY_ATTACH                3
>>> +#define XEN_DOMCTL_DT_OVERLAY_DETACH                4
>> While the numbers don't really matter much, picking 3 and 4 rather than,
>> say, 1 and 2 still looks a little odd.
> 
> Well although I agree with you it is indeed a bit odd, the problem of 
> this is that, in current implementation I reused the libxl_dt_overlay() 
> (with proper backward compatible) to deliver the sysctl and domctl 
> depend on the op, and we have:
> #define LIBXL_DT_OVERLAY_ADD                   1
> #define LIBXL_DT_OVERLAY_REMOVE                2
> #define LIBXL_DT_OVERLAY_ATTACH                3
> #define LIBXL_DT_OVERLAY_DETACH                4
> 
> Then the op-number is passed from the toolstack to Xen, and checked in 
> dt_overlay_domctl(). So with this implementation the attach/detach op 
> number should be 3 and 4 since 1 and 2 have different meanings.
> 
> But I realized that I can also implement a similar API, say 
> libxl_dt_overlay_domain() and that way we can reuse 1 and 2 and there is 
> not even need to provide backward compatible of libxl_dt_overlay(). So 
> would you mind sharing your preference on which approach would you like 
> more? Thanks!

While I think tying together libxl and domctl values isn't a very good idea,
if you really want to do so, you'll want to add suitable checking somewhere,
alongside comments. The comments ought to keep people from changing the
values then, while the checks would need to be there for people not paying
attention.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:25:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:25:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723869.1129002 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rxV-0003dC-Ri; Fri, 17 May 2024 07:24:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723869.1129002; Fri, 17 May 2024 07:24:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7rxV-0003d5-Of; Fri, 17 May 2024 07:24:53 +0000
Received: by outflank-mailman (input) for mailman id 723869;
 Fri, 17 May 2024 07:24:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7rxU-0003cz-58
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 07:24:52 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8b5b2cb7-141e-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 09:24:49 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-56e1baf0380so4583308a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 00:24:49 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea6a36sm11579286a12.12.2024.05.17.00.24.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 00:24:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b5b2cb7-141e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715930689; x=1716535489; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JWTMGb4mIkptzaQIDdkzMZJueZLkF8jZhw5X5fOzFjA=;
        b=LmDFyGJ39YR+cCvSMsqorMdj3aLmUSnNMUV0oQsiwmt7UusOguknLpI0ZCs58H+YuV
         QYKmAVWFyUxzFA0C1jUq99wotXkplOk/z8pND1iyu8PI0+LUzHzt8oatWLhtOhMS6VDp
         hXv9asyDxh6Z/7pGfq4VTtu0Lp+ModJzDHIsy8ulSvKIfyjtZtTxwTSezhZHY+rYSLWh
         /5+ywoMnm2buRNtjU3j5ylsA0SeBRdDohM1FHFIHKlHrqfAEiRV9Wpus4sJPDzrJ8gme
         QzAEI74gNaAgMSsFVcr9I1E57ZKfl2rbb6pkHLyc0fJF7l5FVQNLJdde3c/LVrpAWP2a
         I2yg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715930689; x=1716535489;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JWTMGb4mIkptzaQIDdkzMZJueZLkF8jZhw5X5fOzFjA=;
        b=Xs2XI3khQZTDedWVpHQPB21bXZjTGAV8nuNtyTBhQmGGHJicMNAJeDadIrg2FsncKj
         kT9S+O5FDZpotdV2vJGb/QOPXt8YiooTPcbsSc/NyEXkkQ3ymhBO7uzWZBEHD9Wwu7w0
         0g0TnRjUDavO78iihQBA/L4MFKRMr5VojiUYRseu9aOCcSaKTN1X9J0AAqIEuD9UlWNd
         WJ/zYwnz9cfnxKcqn0JU2wjrzB8O0L2bRhnrEr4Ow7Sz/O4Mzcn2DFL7CcRk5DagCi2+
         VMN6xqt7AgpmfHrJDWiOTySW5JxuKH8RnzXZBYC0+NSyQeN0OMTn11CdvyJ15CO4vBKE
         ClZQ==
X-Forwarded-Encrypted: i=1; AJvYcCW/62G3KtZU3zxhe3JtGn9jwO31iKZkhKod6DkxQqMGaAS4CnEMFaP/XQET2mzf02g+jJOa/yDY1FKdok1FLxQCb/T/ozXnMj64HzvGo9g=
X-Gm-Message-State: AOJu0YwAJMqmlEs7d0sv3ifzMO94uL68GpEKTzYUFSxsyp2KmGww86Mk
	I4hJv2tmMLSsFOMYmco0wn7jlHmRJ4NFl70aSVqCSb1NHCRt0VOFCIWpj9Ca9g==
X-Google-Smtp-Source: AGHT+IEveJxLI6McjLFCEBGWi/iUSeAOGqdRvEF6QVYZx5DR0p2QEOdLkn6+dKzBkxhtj+hXC8qQSw==
X-Received: by 2002:a50:8a84:0:b0:56e:2c34:cfec with SMTP id 4fb4d7f45d1cf-5734d59043dmr16306577a12.7.1715930688941;
        Fri, 17 May 2024 00:24:48 -0700 (PDT)
Message-ID: <686e58b4-cced-46d4-9a88-5a3c00180268@suse.com>
Date: Fri, 17 May 2024 09:24:47 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <George.Dunlap@citrix.com>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
 <5059a536-2495-4121-8aba-5a2542ff9987@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <5059a536-2495-4121-8aba-5a2542ff9987@apertussolutions.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 03:22, Daniel P. Smith wrote:
> On 5/16/24 02:41, Jan Beulich wrote:
>> On 14.05.2024 11:25, Jan Beulich wrote:
>>> On 03.04.2024 08:16, Jan Beulich wrote:
>>>> On 02.04.2024 19:06, Andrew Cooper wrote:
>>>>> The commit makes a claim without any kind of justification.
>>>>
>>>> Well, what does "have no business" leave open?
>>>>
>>>>> The claim is false, and the commit broke lsevtchn in dom0.
>>>>
>>>> Or alternatively lsevtchn was doing something that was never meant to work
>>>> (from Xen's perspective).
>>>>
>>>>>   It is also quite
>>>>> obvious from XSM_TARGET that it has broken device model stubdoms too.
>>>>
>>>> Why would that be "obvious"? What business would a stubdom have to look at
>>>> Xen's side of an evtchn?
>>>>
>>>>> Whether to return information about a xen-owned evtchn is a matter of policy,
>>>>> and it's not acceptable to short circuit the XSM on the matter.
>>>>
>>>> I can certainly accept this as one possible view point. As in so many cases
>>>> I'm afraid I dislike you putting it as if it was the only possible one.
>>>>
>>>> In summary: The supposed justification you claim is missing in the original
>>>> change is imo also missing here then: What business would any entity in the
>>>> system have to look at Xen's side of an event channel? Back at the time, 3
>>>> people agreed that it's "none".
>>>
>>> You've never responded to this reply of mine, or its follow-up. You also
>>> didn't chime in on the discussion Daniel and I were having. I consider my
>>> objections unaddressed, and in fact I continue to consider the change to
>>> be wrong. Therefore it was inappropriate for you to commit it; it needs
>>> reverting asap. If you're not going to do so, I will.
>>
>> For the record: With Andrew having clarified that in his revert's
>> description:
>>
>> "The claim is false; it broke lsevtchn in dom0, a debugging utility which
>>   absolutely does care about all of the domain's event channels."
>>
>> "all of the domain's event channels" means to include event channels Xen
>> uses for its own, and merely puts in the domain's table for ease of
>> handling, I've agreed that - in the absence of any better debugging
>> means - the revert may stay in place. For context, my prior understanding
>> of the issue was that lsevtchn simply stops probing further channels when
>> getting back any kind of error from EVTCHNOP_status (which I continue to
>> think wants addressing there, by a future version of a patch that was
>> already sent). However, there are follow-on concerns I have:
>>
>> 1) In the discussion George claimed that exposing status information in
>> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
>> how a similar assumption by CPU designers has led to a flood of
>> vulnerabilities over the last 6+ years. Information exposure imo is never
>> okay, unless it can be _proven_ that absolutely nothing "useful" can be
>> inferred from it. (I'm having difficulty seeing how such a proof might
>> look like.)
> 
> Jan, I would agree with you that any resources exposed to the guest, 
> even if that resource is status information, should be behind an XSM 
> check. I would, however, have to disagree the position over proving 
> absolutely nothing "useful" can be inferred. Prior to spectre becoming 
> commonly aware, no one considered proving there needed to be protections 
> against out-of-order instruction execution being used to bypass branch 
> conditions.

Interesting perspective.

> Predicting the future will more often than not fail, but 
> with an XSM check in place, the dummy/default policy can just be updated 
> with the general safe decision and others can use FLASK to provide fine 
> grain access.

I have to admit I have difficulty seeing how one would adjust dummy to do
restrict things by default, at least as long as XSM_TARGET is used.

>> 2) Me pointing out that the XSM hook might similarly get in the way of
>> debugging, Andrew suggested that this is not an issue because any sensible
>> XSM policy used in such an environment would grant sufficient privilege to
>> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
>> for Xen-internal event channels. The debugging argument then becomes weak,
>> as in that case the XSM hook is possibly going to get in the way.
> 
> And this is where we have a fundamental difference. Event channels are 
> XSM labeled objects that are always connected to a guest. If they were 
> truly Xen-internal, then a guest would have no way to get 
> access/reference them.

And that's what I'd like to achieve. The fact that Xen puts these event
channels in the guest's table is a mere implementation detail. They could
as well be kept elsewhere, just that handling then would (likely) be more
cumbersome.

> Again, I never disagreed with whether the guest 
> should or should not be able to access them. I did ask for a better 
> explanation than, "has no business", which is a statement of opinion not 
> of fact or reason. The point is these event channels are a resource 
> attached to the guest

As per above - not really, they merely appear like that.

> and access control decisions to them should be 
> made under XSM. Injecting a hard-coded access control in front of the 
> XSM check subverted/invalidated rules in the existing FLASK policy.

I'd agree if these were truly guest resources. Elsewhere I pointed at
the equivalent in memory management: Xen-internal memory also isn't
protected by XSM checks. It's inaccessible for entirely different reasons.

>> 3) In the discussion Andrew further gave the impression that evtchn_send()
>> had no XSM check. Yet it has; the difference to evtchn_status() is that
>> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
>> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
>> similarly useful to allow getting a stuck channel unstuck.)
> 
> A more appropriate default might be XSM_OTHER with conditions with in 
> the hook  as to whether the policy should be XSM_HOOK, XSM_TARGET or 
> flat denial.

That you mean for send, status, or both?

>> In summary I continue to think that an outright revert was inappropriate.
>> DomU-s should continue to be denied status information on Xen-internal
>> event channels, unconditionally and independent of whether dummy, silo, or
>> Flask is in use.
> 
> Any guest facing resources, regardless if the backing end is the 
> hypervisor, should be protected with XSM. This allows the maintainers to 
> codify what they believe is a sane policy in the dummy policy and the 
> end user can use FLASK to enforce what they believe is acceptable.

I continue to be under the impression that either I don't look at things
correctly, or you don't. What's "guest facing" here? Xen-internal channels
aren't guest facing aiui. Their remote end is, in Dom0 or a stubdom. The
Xen side of it isn't guest facing at all; it's merely a vehicle to ease
handling that they're put in the guest's table.

>> Plus, as stated before, evtchn_send() would better remain in sync in this
>> regard with evtchn_status(). The situation is less clear for evtchn_close()
>> and evtchn_bind_vcpu(): Those indeed have no XSM checks while they do deny
>> operation on Xen-internal channels. It is likely more far fetched to
>> assume permitting them for debugging purposes might in fact help in rare
>> situations. Yet it may still be a matter of consistency to keep them in
>> sync with the other two. (Note that there's also evtchn_usable(), which
>> might then also need tweaking itself or at the use sites.)
> 
> Just because that is how it was, doesn't mean it was correct. I had a 
> discussion with one of the original authors of FLASK before taking up 
> the maintainership and he felt there were likely more XSM checks that 
> should have been in place originally. He considered it a first order 
> approximation of what should be protected.

I'm afraid it's not really clear to me what to take from this. Are you
suggesting that further XSM checks may be wanted? If so, where, and who
would take care of adding them?

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:39:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:39:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723876.1129013 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sBn-0006C0-5s; Fri, 17 May 2024 07:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723876.1129013; Fri, 17 May 2024 07:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sBn-0006Bt-3E; Fri, 17 May 2024 07:39:39 +0000
Received: by outflank-mailman (input) for mailman id 723876;
 Fri, 17 May 2024 07:39:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7sBl-0006Bn-QC
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 07:39:37 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9c563fb0-1420-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 09:39:36 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-51f0f6b613dso2178061e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 00:39:36 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bed72bbsm11643241a12.57.2024.05.17.00.39.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 00:39:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c563fb0-1420-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715931576; x=1716536376; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KEU05UX5DXF0EkY4DoKFBorUQ7NKoRlveCGyioriTzY=;
        b=UCXC+k+W7SQJUKfjMhLswKuNvMjFp8aC63fzVQ1Qh++Tl1639+O6/BNQ5FMR3FX14x
         SzFpv5oFLExNZ54GRwb9EHJN+f1WYy+WHzteSTsQP9AwetURJ1esPVwOWRcZ8vrI1Dbs
         MX3aDLd44mVUNpQjENa5ts0nuhOOpBbYQfaR+ibK9oIMxtUYkQyDCliCLRJg66Rbvx9K
         nzJDfbZVyjN9YJ8QhhtrnlPFXLXwAZZdk3OES9b/QO2xqZs3E8J4EhnFodUiKaR6XTgM
         ukNPXgFdYRVExBhCHJeIgn0lZOoXN4ZoJDsz8ioRPctS5i1HsoGOea+7GS1Gw3hZ/E/e
         zZ7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715931576; x=1716536376;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KEU05UX5DXF0EkY4DoKFBorUQ7NKoRlveCGyioriTzY=;
        b=quJ9GoV2QZLxuf5yqJUwmOuXBfQ+RmKYZ8YHVbgM8CAYWxN/Mj//oR0AZUJMtt8G71
         1jf/Gvbee4KUXHW9watOnSBnuINAgYXZwez33ZqIuzafduh0uLp75uyTMdF1rp79ErBz
         rLyc0y4lQtjXej7gIQiuHPFQmsc7jHkVFt0YAU3rTHX05+Gp/Mgu0R8/gmXmmtYElAqw
         vup8GpC/R1apnLKKO1Or03VUAXavr3mCXEyTh8+GFNBCEaVmC7q0C45khKBOZOlPPfmf
         YtBKfa764bi3RjrpAXXUmjS9DbQn2eGexGBbGeZ7llG6nylGGPh2QDRAfoPWRCFO0t1c
         XCOQ==
X-Forwarded-Encrypted: i=1; AJvYcCVVRaOMkcgCexZbyYbK2CS1dBcAGbB9sSgcatC2YAbSvyKWojEzkEgDu5xqVaMNRBW6jks4uPo3RKvcpBlDf217VyNV5zsrzIzpa8gOm9Q=
X-Gm-Message-State: AOJu0YxMgFUcUYtOhx8VuC59fkKSlsJGM6RrKh0OImvVOsnoSM31gmNW
	Op0NyvrecM6d3JCyzO3cgUrLOSK4i3Rcwihek0XpEEgjG7ymfXGOw0CaWp7uiA==
X-Google-Smtp-Source: AGHT+IG2NE19+Xx58x2IPIxxQmMj72AE+b4WoR5NLCOTTGDbkoXphyhU+1XDIxzGCHaFwCVS4z9YWw==
X-Received: by 2002:a05:6512:39c9:b0:522:33dd:dcd9 with SMTP id 2adb3069b0e04-52233dddd7amr14957725e87.33.1715931576142;
        Fri, 17 May 2024 00:39:36 -0700 (PDT)
Message-ID: <c943807b-f5dd-4e46-aac3-a519c4051f4f@suse.com>
Date: Fri, 17 May 2024 09:39:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19?] xen/x86: pretty print interrupt CPU affinity
 masks
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20240515152925.77197-1-roger.pau@citrix.com>
 <06844367-cc1d-42f7-b925-ba78b4532b11@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <06844367-cc1d-42f7-b925-ba78b4532b11@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 19:13, Andrew Cooper wrote:
> On 15/05/2024 4:29 pm, Roger Pau Monne wrote:
>> Print the CPU affinity masks as numeric ranges instead of plain hexadecimal
>> bitfields.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> ---
>>  xen/arch/x86/irq.c | 10 +++++-----
>>  1 file changed, 5 insertions(+), 5 deletions(-)
>>
>> diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
>> index 80ba8d9fe912..3b951d81bd6d 100644
>> --- a/xen/arch/x86/irq.c
>> +++ b/xen/arch/x86/irq.c
>> @@ -1934,10 +1934,10 @@ void do_IRQ(struct cpu_user_regs *regs)
>>                  if ( ~irq < nr_irqs && irq_desc_initialized(desc) )
>>                  {
>>                      spin_lock(&desc->lock);
>> -                    printk("IRQ%d a=%04lx[%04lx,%04lx] v=%02x[%02x] t=%s s=%08x\n",
>> -                           ~irq, *cpumask_bits(desc->affinity),
>> -                           *cpumask_bits(desc->arch.cpu_mask),
>> -                           *cpumask_bits(desc->arch.old_cpu_mask),
>> +                    printk("IRQ%d a={%*pbl}[{%*pbl},{%*pbl}] v=%02x[%02x] t=%s s=%08x\n",
> 
> Looking at this more closely, there's still some information obfuscation
> going on.
> 
> How about "... a={} o={} n={} v=..."
> 
> so affinity, old and new masks are all stated explicitly, instead of
> having to remember what the square brackets mean, and in particular that
> the masks are backwards?

Just one question: Why put old ahead of new? Aiui that's what you refer to
with "backwards", yet I don't see what's backwards about it. Old would
possibly matter only when the IRQ was recently moved, whereas new (actually:
Why "new"?) would matter at all times. I'd see "... a={} m={} o={} v=..."
as more appropriate.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 07:48:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 07:48:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723882.1129023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sKM-0008Mv-VZ; Fri, 17 May 2024 07:48:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723882.1129023; Fri, 17 May 2024 07:48:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sKM-0008Mo-St; Fri, 17 May 2024 07:48:30 +0000
Received: by outflank-mailman (input) for mailman id 723882;
 Fri, 17 May 2024 07:48:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7sKM-0008Me-0m; Fri, 17 May 2024 07:48:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7sKL-0008NW-U4; Fri, 17 May 2024 07:48:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7sKL-0003C0-MB; Fri, 17 May 2024 07:48:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7sKL-0004Lv-La; Fri, 17 May 2024 07:48:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jkWEZw3rF7SlhrKbzpOXlUmWW3Pd6qjsAAY1Qzm7Ji4=; b=TXwJNmLzS2ClVR4rY8epzOPDvu
	+74H9UULeKFyRTKT+E0zAl3jlt77YsMstwTys498brEoanQmoFCmZu3U1i93kl42UuasdV+RLFXhQ
	36TgpOr01c7iT0Pf8m8DzjAlkfrh6jQ5t1YASPCF+Gf/XcE2e7mPZdWZEbvjhhPrU5rQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186024-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186024: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
X-Osstest-Versions-That:
    xen=53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 07:48:29 +0000

flight 186024 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186024/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca
baseline version:
 xen                  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e

Last test of basis   186022  2024-05-16 23:00:22 Z    0 days
Testing same since   186024  2024-05-17 04:02:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   53dc37829c..21611c6870  21611c68702dae2e18cb519a6e166cdceeaff4ca -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 17 08:07:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723899.1129033 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sc1-0004Xi-OC; Fri, 17 May 2024 08:06:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723899.1129033; Fri, 17 May 2024 08:06:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sc1-0004Xb-Kp; Fri, 17 May 2024 08:06:45 +0000
Received: by outflank-mailman (input) for mailman id 723899;
 Fri, 17 May 2024 08:06:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=q70L=MU=cloud.com=christian.lindig@srs-se1.protection.inumbo.net>)
 id 1s7sc0-0004XV-NE
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:06:44 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 65ec4a14-1424-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 10:06:43 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52388d9ca98so639222e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 01:06:43 -0700 (PDT)
Received: from smtpclient.apple ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf91sm1087447766b.1.2024.05.17.01.06.41
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 17 May 2024 01:06:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65ec4a14-1424-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715933203; x=1716538003; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DxV1hJF49YePmCkWfVoIRzVzYu9CpcUHmXWTZF3b13c=;
        b=MJkO6fSbxNS0+PB5X2li0CrRsEKXwWSqcZoKRQF5rhGgmVLVuyBEuqwzdfqcHFIlBG
         5/Db2Dy2Ckne89w+CCYKrRXK8dnas1jVSTnYfZYbCFfM0FAp4F3YcMUxeAt8nkO8+OS0
         bCR9DSvhNi2lbpIeUC/0keLYE4XlY3MG2nbJs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715933203; x=1716538003;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DxV1hJF49YePmCkWfVoIRzVzYu9CpcUHmXWTZF3b13c=;
        b=Ze6ojv4mJcmgV62CmuU6HVj5hJ3CWwxax+spIebqj6UhBL5poYDr+5Y+DqTq5ZPiXj
         AEcy2xdw6v6twnN/1jIkg+onw36oDwSIhlSMVzhYHWEkd9rg4zbhy91G6FUOBZLZL1+M
         YPd0CsYIBO4qOeTTXAntcrfEcyikFVOAdev6EdBskFR+i+40L9RA+BUJwvgcoYIQUff/
         0KEE9konCh5FiES+IYFMAE+lVUmmB9JlGTq+iDTqTTdNVJ1Dr++VhvgAqDScwvaiR7ou
         dZ3bO8PFAA4j7bQ53XArsM+ShAgdLOFyTrOnBSBoNhMQfT1lBlwx6v552m2QKGymk0XH
         eFkg==
X-Gm-Message-State: AOJu0YyFd8c19kLanLqYTZZQm0GZs2vhngYeF2HxW1MU7IYraXYQScYk
	tgPQSi48EKchgz/hOfDQtdIeYTpLVktbSQac43TGLXmMnn0mTDTDgyfFt+/TAbg=
X-Google-Smtp-Source: AGHT+IGOfFUSvTUnVB89H81Wq5EJL4RINiDv7O/pKP6TVa0pl30QgRGD06Xsp+wwOxeR85ni4aN7iQ==
X-Received: by 2002:a05:6512:2310:b0:51b:9254:91e7 with SMTP id 2adb3069b0e04-5236e7185abmr10038677e87.61.1715933202396;
        Fri, 17 May 2024 01:06:42 -0700 (PDT)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\))
Subject: Re: [PATCH v2 for-4.19 0/4] Drop libsystemd
From: Christian Lindig <christian.lindig@cloud.com>
In-Reply-To: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
Date: Fri, 17 May 2024 09:06:30 +0100
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>,
 Juergen Gross <jgross@suse.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?utf-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
Content-Transfer-Encoding: 7bit
Message-Id: <C4B17A26-6959-43AC-8ABC-9F8BEF4EFCEC@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
X-Mailer: Apple Mail (2.3774.500.171.1.1)



> On 16 May 2024, at 19:58, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> 
> On advise from the systemd leadership.
> 
> v2:
> * Import the free-standing example and use that to retain existing
>   functionality.
> 
> Andrew Cooper (4):
>  LICENSES: Add MIT-0 (MIT No Attribution)
>  tools: Import standalone sd_notify() implementation from systemd
>  tools/{c,o}xenstored: Don't link against libsystemd
>  tools: Drop libsystemd as a dependency
> 
> LICENSES/MIT-0                                |  31 +++
> automation/build/archlinux/current.dockerfile |   1 +
> .../build/suse/opensuse-leap.dockerfile       |   1 +
> .../build/suse/opensuse-tumbleweed.dockerfile |   1 +
> automation/build/ubuntu/focal.dockerfile      |   1 +
> config/Tools.mk.in                            |   2 -
> m4/systemd.m4                                 |   9 -
> tools/configure                               | 256 ------------------
> tools/include/xen-sd-notify.h                 |  98 +++++++
> tools/ocaml/xenstored/Makefile                |   2 -
> tools/ocaml/xenstored/systemd_stubs.c         |   2 +-
> tools/xenstored/Makefile                      |   5 -
> tools/xenstored/posix.c                       |   4 +-
> 13 files changed, 136 insertions(+), 277 deletions(-)
> create mode 100644 LICENSES/MIT-0
> create mode 100644 tools/include/xen-sd-notify.h
> 
> 
> base-commit: 977d98e67c2e929c62aa1f495fc4c6341c45abb5
> -- 
> 2.30.2
> 

Acked-by: Christian Lindig <christian.lindig@cloud.com>



From xen-devel-bounces@lists.xenproject.org Fri May 17 08:07:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:07:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723902.1129042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7scv-00052Q-W1; Fri, 17 May 2024 08:07:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723902.1129042; Fri, 17 May 2024 08:07:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7scv-00052J-TF; Fri, 17 May 2024 08:07:41 +0000
Received: by outflank-mailman (input) for mailman id 723902;
 Fri, 17 May 2024 08:07:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=q70L=MU=cloud.com=christian.lindig@srs-se1.protection.inumbo.net>)
 id 1s7scu-0004xV-Nv
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:07:40 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 86b2c984-1424-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 10:07:38 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso25301011fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 01:07:38 -0700 (PDT)
Received: from smtpclient.apple ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cf91sm1087447766b.1.2024.05.17.01.07.37
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 17 May 2024 01:07:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86b2c984-1424-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715933258; x=1716538058; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iAhbvMNjNeI6uVfAm48uZmXWCz65I34c4tiky0xbTic=;
        b=M+bQi0OdGU3hbJ3bxK79FJ38LCo6FEw13OOWWPoccukiHGEIcTlpR7rsIs1R4bz9ku
         rfNCmcL5ZRv43nnUyPMlv8h5lgibMFCd2gtZi/p6Rb4XP9Eul4Xt49mga+QIB/kciFdw
         wUhRd7X/TMxTPzO5I0t/d9Dr9qIYfnkjVSYHk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715933258; x=1716538058;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iAhbvMNjNeI6uVfAm48uZmXWCz65I34c4tiky0xbTic=;
        b=vz3wpGR/Rw3hx7/dAMUfeAaAfynUmdd4UuHuRHBjUo/BYO9J3L40yVQmfkwYv8L1Bf
         uzih9e0XxYNlUpu+MY/mnZum19Dx7Xp7AZFX5eeknoBolNcnf+V2AZGqgNjFVsgua0Cz
         i2onjM4IY9zqbnUJm3xb7s7FymroXtidKurW3UhZEgk5ZkC9t5Z1mHYpbrmBTJwvFVPb
         ShlrmH+Ka+YPPvfPuK6bbGcS9E7ttNeufcrYPAo9TKowNQEBvXuHgR7fuBsWTR/ZTSph
         dgJm8ZnwKPYo3F6YqzgoOLumTNEBt4711BcbP7aRC/VNHWSfr5ifCU+QbhwL79cV5A6P
         KggQ==
X-Gm-Message-State: AOJu0YwBg8BVUyAFoI2xWV4Ah3++68Zsn7WM9FaCObwGFxpkXwyDuoBs
	68arL+RP3vOxObhz45aKpc3v92f1hjDpERJ2QoXNpYu34FBBCLVGbUI95HmtIl4=
X-Google-Smtp-Source: AGHT+IFzFN4piK6kzfPjplbXkTBQXObg1P7TV+4dXfLWKgWMj8bKzZibiPdzLmsJSKN3Oq3BomP4gA==
X-Received: by 2002:a2e:984b:0:b0:2e0:3ad2:b371 with SMTP id 38308e7fff4ca-2e51fe5875dmr167094401fa.25.1715933257871;
        Fri, 17 May 2024 01:07:37 -0700 (PDT)
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\))
Subject: Re: [PATCH v2 1/4] LICENSES: Add MIT-0 (MIT No Attribution)
From: Christian Lindig <christian.lindig@cloud.com>
In-Reply-To: <20240516185804.3309725-2-andrew.cooper3@citrix.com>
Date: Fri, 17 May 2024 09:07:26 +0100
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>,
 Juergen Gross <jgross@suse.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?utf-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <17B1DAA5-EBFB-47C2-95A3-057D5BD2AA60@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-2-andrew.cooper3@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
X-Mailer: Apple Mail (2.3774.500.171.1.1)



> On 16 May 2024, at 19:58, Andrew Cooper <andrew.cooper3@citrix.com> =
wrote:
>=20
> We are about to import code licensed under MIT-0.  It's compatible for =
us to
> use, so identify it as a permitted license.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin T=C3=B6r=C3=B6k <edwin.torok@cloud.com>


Acked-by: Christian Lindig <christian.lindig@cloud.com>



From xen-devel-bounces@lists.xenproject.org Fri May 17 08:08:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:08:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723904.1129053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sdl-0005ZC-80; Fri, 17 May 2024 08:08:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723904.1129053; Fri, 17 May 2024 08:08:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7sdl-0005Z5-53; Fri, 17 May 2024 08:08:33 +0000
Received: by outflank-mailman (input) for mailman id 723904;
 Fri, 17 May 2024 08:08:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7sdj-0005Yj-G3
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:08:31 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on20615.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::615])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a3f8068a-1424-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 10:08:28 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by MN2PR12MB4238.namprd12.prod.outlook.com (2603:10b6:208:199::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 08:08:25 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 08:08:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3f8068a-1424-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YTrJCF2pK2OBc4xZ5QacGEujfP5KgTOsHjkA3GX2IhV9mz3g/2FDQo6lac2nsyO+0U487gNiShG/3E8VNXiULYs9cA8TrVrZFhVDJNOobWC6f9SKXXDpX3IYBbr2D6faN4G2wOqTCmQ27lvfg3WyG0aaPEv3GbQYDts+bj++h23dfiL8OW1cEZV/U1pjwTUNrabYR6lDqUmmFIOqob6PQuDOC4mLgAinjIvpBoj3SW4VaDvk2m27U26QY9K0ysaBiY1e1cX+90grhEjt8jIYpCgsTpMk2mFYpmvIUnK0OPZQQ+W6/J0FL3DytoPMQCh/f7JqO5CUk2vnCMBw4O3Y7w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=iWW2coKYW1UsbNI5qTwHy6PHBT9XL55xGzNsMUsF/0o=;
 b=IpuG+DcMce/u6jhVNi00PGZVweO6p5wc9MOrdllBkEjV50i0JsskL2QaBaxG+Ifml9IZIKIB75yDd3+4H3ErLeXijnMR4s/ijCCOD/T8L56HAC7Ok5r4RCI5qQjPtLyGuT2N4kBnkWkjZFxVlMEAUGgGo/HYEr0JMMc3ZX+QChBeO6CrJOU2t9WJIvJdV2UNZMWMJbj5ywDsNpT86/FewXnqx6IWwo2iK/o1oef28keYLtWicNc+pbaxVCJB8VNj1xFBZ/IgUp35oHHDB8HljTSiZJvZS40EaStIfswXtTvmVBo3tv7gBJ5TZCTq7VpViHXDqmqhy5n7/MOgw/Cuqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iWW2coKYW1UsbNI5qTwHy6PHBT9XL55xGzNsMUsF/0o=;
 b=WsfXrfUSN36hhUQ9WWBsyjwQZw32L/AvTJtndfE1MUuDTHnAtjW74gQAMxt4DpdKrNh4O0sr7QS6xUNJmp5JuUk8CuD+QCF/rxv0FybSBvFL4i6igWx52nI/KcNxr0BVAEHg8HaoSJrWdOM10n4HOQpDR/ARgrr+iG+k3iTSMxs=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Chen,
 Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Index: AQHap3bYcGtlWtrcpEyRLTK5fjq5SLGZ1UMAgAFwSAA=
Date: Fri, 17 May 2024 08:08:25 +0000
Message-ID:
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
In-Reply-To: <fb708441-b302-4727-8131-62435a54d99c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|MN2PR12MB4238:EE_
x-ms-office365-filtering-correlation-id: aa255509-d0ae-48d2-474b-08dc764886cc
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|376005|7416005|1800799015|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?WmZUTzBhdjRuejI3b2l4Ty8wWUFLekhYMVlMNTZnT2doejJGYzQxdGJsMm1E?=
 =?utf-8?B?WmZaZDdNM29WRmdYekswbTEyS0xSL1U5Z2hlYTBJd1VnNkRpMDlpUlNkdlZW?=
 =?utf-8?B?TTJyK1lvbzVoQkFhYWpNZzNQSWZXZ1hHOG9ScTZTempMMVBHS2IycDBDTTE3?=
 =?utf-8?B?Q0U5enhjT3dRaHVqUGx5ank2VHMydzZGdy9ocnN2MW9ERnpNdVAvZlNINTJi?=
 =?utf-8?B?MU10UDQzS1NWNWNkNjRJN3BwamV3U216bGY0WFE0TzhkUGhHQlJHRzVNNUVo?=
 =?utf-8?B?THY0K0xNUzYzNndheXM5Vnd6QVNubTNJeUozUDJ5Qmt4Z01jVitZcUg4eVBC?=
 =?utf-8?B?RU9TOUZoRFJzTkxWUy9xY1dxNjc0V1BvR2Q0K21Ha0gzak5lN3g1cnhiZDNQ?=
 =?utf-8?B?eTJyMDFETGRKcjhUYlFtdGIzYmhRYmFMdFpKUWdlUldCd0ZidWtwekl0Tnhh?=
 =?utf-8?B?M0lQT01Ib09Ga1pBeFdDalYya3o0TDhTcm80c2NYRHFnR3BkaUwxMjg1a3lH?=
 =?utf-8?B?L28yQkxMaDR0aXplNjVmR0plVEtTUHY0Y25zVTBvVGVmUTVUelRQeW4xQTE3?=
 =?utf-8?B?SEpRZFYyVHhPSnQ3aVFpNDRZTmFQK1E3d055SEt5ZmJzSkVINzZ2dlFLZE9y?=
 =?utf-8?B?dTBNLzhsS1I1Z1l0WXV6OW9tZ0tHUlg4aDNnWUs2UkFPZUVIOXJGUkNaeGJY?=
 =?utf-8?B?WHArRG9aQTRldUl0Q1pucU51U1g2bytmbDdLeURMaWJoZGUzK0J1WEpQbFFr?=
 =?utf-8?B?blI3ZEZuS0RheHdlaFNvaW96T0RjZUhoQTQ0OERKTFhBVW9tT3VlN2ZhWEJP?=
 =?utf-8?B?OCtnaFFGQlJaNGdXVG9CSVFRazY4dDk3TTl1Nm1hR0F5M1ppOXVNZlRCaUUr?=
 =?utf-8?B?Y3VOY1dPcHpWdkN6c2pvME5NUFZZajFHM0x6dzJUTmU4TDBJczJDWFJ1ei90?=
 =?utf-8?B?S1RrMk9DUSthcjJyTkswejZ6T0xpU2tPVFJsTWQwdGlkVytEcWcrTXBqWFpE?=
 =?utf-8?B?aFZtVmxtaGNmUE1CS1FjOVBQdUd0Y01DS3NIQUgzUDg1QnY1cW4rT2JCS0Qy?=
 =?utf-8?B?K2RrQ3hFVDgwMkl4OUp2c0d0azc3RTRpY25SNEpDdy9CQWkvY1RwNzg3azc5?=
 =?utf-8?B?aHNnbG85UWRCdEpHc2ZXY3RybDdaYXdobGwxanhqblN1N1U3VFFOTjl1cFEz?=
 =?utf-8?B?eUJqbUhVVjNNVlRBRUpwQmZTQjlZU2RyUVpMZ1ltRzNwMmEwOEdjdk1nUHdx?=
 =?utf-8?B?ak5OR0ZXWDRTTkhmNU1kQnhYRi9RQ1FiOVptZGx1Z0dlczhha2d3ZmxDS0or?=
 =?utf-8?B?aUg0a3lVaGpRWU5sekFWZlFkN2xISHRVTk5IcFBoVTFTYlhwRFljM2ZGNG1O?=
 =?utf-8?B?T0p2RXgrU1Q1Z25tUmsxZ3p2WDd4bHNNUjZTNUM0UVFmRXE1MzdEcVRyTFJ0?=
 =?utf-8?B?ajVERkRMdUwweHZuY0Z3YkFKdUUxeWJmSWl6Z2RiUmlJOGNqNFJiQlRUemxn?=
 =?utf-8?B?TmFzWVdQYmxZR0grVG15WWNqaDRKaWVaVGFyZkpYWDQ3cWFUaGZkeWJZeTc1?=
 =?utf-8?B?UkQrZ2prQzVkRW9NUHl0ZjNCZU9uVWVndXU3QW9ZckR6YVBDWVB4MzQ3NEJo?=
 =?utf-8?B?TXZhdzVkbFkxMzk1QkdJNmVvTVp6NURlVTJ5bGZDYTNaRVBBR0YyRm1vdjNO?=
 =?utf-8?B?SFh1cG55L3UrdHNlTDMwV3pHb1MzYmZnK0FqUUxMNzZMVnUrZ01Cci9CajJj?=
 =?utf-8?B?a3JWTEJFc1FOT1FEcDhCVSthcUNIR2hVMm8vN1I2aHNpNlFweXBicjQ5ckhX?=
 =?utf-8?B?REpoYTZyOVFTQUVRbEhrQT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(7416005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?VWRjRnRTMC85aDF2K2ZCbTZkZTM2V1o1eWIwR2hPdkUxUlZGdVUzTzg4YmND?=
 =?utf-8?B?OFhRVG1qTnNqellrNnZHOEdZaytvSnBYbHpkZVAxZWJLQXNPeHU4UVBVVlFw?=
 =?utf-8?B?S21nbTluNytBbkk4VzBOdnpKQnp2Vlc4V0hWVFR2dTRQWVJhTHFpQnZnVTIy?=
 =?utf-8?B?WjlEb0ZLZ3ZpNHcxVGh5UHRkaUppajg5OThkS20vaTUwYkpRaWFSSkVhU0tC?=
 =?utf-8?B?bG1MN3pyWCtBdGxPQVB2MkttVkdLQkttNWFDdTlycWo2ZWg4RVJLRnZ2TE4v?=
 =?utf-8?B?OFM2NW1QZmV6TVlvTlJBc01JV2ozYms5Sm9VdVlCRXNLOFB5Mm1SNjBXTTNs?=
 =?utf-8?B?QXIrd3lyNUNickFpSE5NSkNxSTIvNDlVT3BUVUYyUDNMb1NmNlFoRVlTTjZK?=
 =?utf-8?B?ai9CanZtbmZCYURDRnRNRVBoM2NYdTcwMS9ZeE5nQjRIbnZwVmNCcTRvcjlY?=
 =?utf-8?B?QjlSbUxyeUVhUXJKa1JqUmwwVjVpQnVmZ2o2cmNNanBXUTJ3MXRrMXV2QVlt?=
 =?utf-8?B?RUg1Y1liWTIxOEY2YnZNRlRBVWduR0VHSWRTbFdXeVpMRmQzdElsQ2N5R253?=
 =?utf-8?B?SVd2NmppOGFpVjVIMUhXdkZGdllIQ09SRFoyVmlXbGIxYjlIaU41bDlzdXRJ?=
 =?utf-8?B?SlBFRUpBQ09xcFhTWmI5WkRtQVBEbWF1N0NseSt4ZUJIa0p4Q0dBMWc0TXp1?=
 =?utf-8?B?Z2JmeDhLMEoyb3dIL0IzTVZSK2N5TDNZL0M0aVFIb0JNZllHS3phVExDZnFv?=
 =?utf-8?B?S1hmN1ZLZ1NkNmJ5Y0xVbnp5YjBNR21GRU1vdTZtVVNJc01YSmNMd1lWa2c0?=
 =?utf-8?B?UHJDNjJpMWpUQnZEcWQvWGVCK05IWm1GRE1YdTZPem5uQjdocHVzcWVvZFE5?=
 =?utf-8?B?OEtKelJPc256NG5IdlFDejdHMFhueTBZVlpWb0RhZm94OVJ2ZG9rUVlsU1Vv?=
 =?utf-8?B?enZsUDlJa1hIakNDa0krUjBCT29peWtJLzNDSDZEWXRzeGJka1ExT29pemM5?=
 =?utf-8?B?TFRmUVVqd0c1UCtld0F5RHh6Q3R4aVl1NU1kQzZodkZVbjBSY2lzcTRwL1Zu?=
 =?utf-8?B?ZHZERFZRdzZNWW93UldTR2lGN0swSkdBcm40azV2OVJhRlI2aURDSStvMFV4?=
 =?utf-8?B?R2VySjFta1dFdE5HK2g4V3VvRVNlcEFxdzJYSHBrckZOaVEzcmNodUEvV0V3?=
 =?utf-8?B?QjhNR00wU0graTlhbkNnQUNyZWhCaUhCT3hYVzc5OEJTZmhxYkgzai9BVzdM?=
 =?utf-8?B?REEzbnF2b2ZFbFZneW52b3lqemltZ3hBeVlrSGhOdjNIcFlXLzZXdnUzbHNT?=
 =?utf-8?B?Z0hrS0MxYzJ2K2g0UjJDUDZQVUUyZ2tSVWtFcGVMRnFvYzdLSDE4Z0QxZWRy?=
 =?utf-8?B?cFlwWU1zQkprMVphS2hRcGdGajY0WmdhckxtcDh0Yk9ETTY5TkdNL1U5U2Zq?=
 =?utf-8?B?V0ZQaUhpcmUyQ3JCeW1pQWZaTktLNVA0ZlMvczBBcjJUUytyWXpNRGNjc2c3?=
 =?utf-8?B?K2VPUGRiMkUyVHJjdUlwY0NIb1VtY1MzSUxoQWFUTnBPT3ZwTUZhWHR2SnZm?=
 =?utf-8?B?eEhYSGxmbnlyejlhcTFjSDN0SlJzaFZnRFZzTkFmNlhpdGk0ZHRJYUF0OFZm?=
 =?utf-8?B?YTcxN2h5UTd1RUtZeXN6MmlMOGFHeHdaYTNpYVBXenFoem1JVVg3eU5QOUtS?=
 =?utf-8?B?NXlFSXlSbnR3N3kyNXEvVWRGWStXL2l6VnU4SEhkdi9RbktabUhLQ3I3OWlR?=
 =?utf-8?B?M3NBZ2hpV0dweW1WK1NlNlpYSFRRMWNRUTI3VTNyY0h6ZE9LVSthZ0xtcWcz?=
 =?utf-8?B?ZjR1NXJnbDJET3hoM3FSV20wa2wxVWZtVVFGYzMxYzdyc1BROEpONno5WWdX?=
 =?utf-8?B?Z2VGL1ZxeWFWRVlIRDBscEdxc2xOOEM5NTlja3JZYm1OMFhKWXBQWDBDeVBR?=
 =?utf-8?B?UWZzYk5lblZHbEVWUHJyYXdBN0VrK0FLZWo1Sm1veFNZeVVoSTdjOHdocmFa?=
 =?utf-8?B?Yk9MVjZxb3prdk5Mck9GbFFqZWpiSUVsTGxZMUlpYWRyaEJISi9MQlhoUGtG?=
 =?utf-8?B?RlpXVXU0emFyWlZQTnovUWZiTTZxNEFzUjdIUUl1RkRvd2NYaXhBTXRkQ054?=
 =?utf-8?Q?lsSY=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <FB7E314F8568AE4F9793F8E8C2CB5755@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aa255509-d0ae-48d2-474b-08dc764886cc
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 08:08:25.0862
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: wQzqzCHhehHjZaX+KaR8TvkPcf3Fm0lROg3VRiSG6miu41OKd3/ztpN+39gY4oln0O+ieN0473KnPbgOMLIOtQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4238

T24gMjAyNC81LzE2IDIxOjA4LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTYuMDUuMjAyNCAx
MTo1MiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBAQCAtNjcsNiArNjgsNDEgQEAgcmV0X3QgcGNp
X3BoeXNkZXZfb3AoaW50IGNtZCwgWEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh2b2lkKSBhcmcpDQo+
PiAgICAgICAgICBicmVhazsNCj4+ICAgICAgfQ0KPj4gIA0KPj4gKyAgICBjYXNlIFBIWVNERVZP
UF9wY2lfZGV2aWNlX3N0YXRlX3Jlc2V0OiB7DQo+PiArICAgICAgICBzdHJ1Y3QgcGh5c2Rldl9w
Y2lfZGV2aWNlIGRldjsNCj4+ICsgICAgICAgIHN0cnVjdCBwY2lfZGV2ICpwZGV2Ow0KPj4gKyAg
ICAgICAgcGNpX3NiZGZfdCBzYmRmOw0KPj4gKw0KPj4gKyAgICAgICAgaWYgKCAhaXNfcGNpX3Bh
c3N0aHJvdWdoX2VuYWJsZWQoKSApDQo+PiArICAgICAgICAgICAgcmV0dXJuIC1FT1BOT1RTVVBQ
Ow0KPj4gKw0KPj4gKyAgICAgICAgcmV0ID0gLUVGQVVMVDsNCj4+ICsgICAgICAgIGlmICggY29w
eV9mcm9tX2d1ZXN0KCZkZXYsIGFyZywgMSkgIT0gMCApDQo+PiArICAgICAgICAgICAgYnJlYWs7
DQo+PiArICAgICAgICBzYmRmID0gUENJX1NCREYoZGV2LnNlZywgZGV2LmJ1cywgZGV2LmRldmZu
KTsNCj4+ICsNCj4+ICsgICAgICAgIHJldCA9IHhzbV9yZXNvdXJjZV9zZXR1cF9wY2koWFNNX1BS
SVYsIHNiZGYuc2JkZik7DQo+PiArICAgICAgICBpZiAoIHJldCApDQo+PiArICAgICAgICAgICAg
YnJlYWs7DQo+IA0KPiBEYW5pZWwsIGlzIHJlLXVzZSBvZiB0aGlzIGhvb2sgYXBwcm9wcmlhdGUg
aGVyZT8NCkluIHRoZSB2MiBvZiB0aGlzIHNlcmllcywgRGFuaWVsIGFuZCBSb2dlciBoYXZlIGFn
cmVlZCB0aGF0IHJldXNpbmcgeHNtX3Jlc291cmNlX3NldHVwX3BjaSBpcyBlbm91Z2guDQoNCj4g
DQo+PiArICAgICAgICBwY2lkZXZzX2xvY2soKTsNCj4+ICsgICAgICAgIHBkZXYgPSBwY2lfZ2V0
X3BkZXYoTlVMTCwgc2JkZik7DQo+PiArICAgICAgICBpZiAoICFwZGV2ICkNCj4+ICsgICAgICAg
IHsNCj4+ICsgICAgICAgICAgICBwY2lkZXZzX3VubG9jaygpOw0KPj4gKyAgICAgICAgICAgIHJl
dCA9IC1FTk9ERVY7DQo+PiArICAgICAgICAgICAgYnJlYWs7DQo+PiArICAgICAgICB9DQo+PiAr
DQo+PiArICAgICAgICB3cml0ZV9sb2NrKCZwZGV2LT5kb21haW4tPnBjaV9sb2NrKTsNCj4+ICsg
ICAgICAgIHJldCA9IHZwY2lfcmVzZXRfZGV2aWNlX3N0YXRlKHBkZXYpOw0KPj4gKyAgICAgICAg
d3JpdGVfdW5sb2NrKCZwZGV2LT5kb21haW4tPnBjaV9sb2NrKTsNCj4+ICsgICAgICAgIHBjaWRl
dnNfdW5sb2NrKCk7DQo+IA0KPiBDYW4ndCB0aGlzIGJlIGRvbmUgcmlnaHQgYWZ0ZXIgdGhlIHdy
aXRlX2xvY2soKT8NCllvdSBhcmUgcmlnaHQsIEkgd2lsbCBtb3ZlIGl0IGluIG5leHQgdmVyc2lv
bi4NCg0KPiANCj4+ICsgICAgICAgIGlmICggcmV0ICkNCj4+ICsgICAgICAgICAgICBwcmludGso
WEVOTE9HX0VSUiAiJXBwOiBmYWlsZWQgdG8gcmVzZXQgUENJIGRldmljZSBzdGF0ZVxuIiwgJnNi
ZGYpOw0KPiANCj4gcy9QQ0kvdlBDSS8gKGF0IGxlYXN0IGFzIGxvbmcgYXMgbm90aGluZyBlbHNl
IGlzIGRvbmUgaGVyZSkNCk9LLCB3aWxsIGNoYW5nZSBpbiBuZXh0IHZlcnNpb24uDQoNCj4gDQo+
PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL3ZwY2kuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvdnBj
aS92cGNpLmMNCj4+IEBAIC0xMTUsNiArMTE1LDE2IEBAIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+PiAgDQo+PiAgICAgIHJldHVybiByYzsNCj4+ICB9DQo+
PiArDQo+PiAraW50IHZwY2lfcmVzZXRfZGV2aWNlX3N0YXRlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KQ0KPj4gK3sNCj4+ICsgICAgQVNTRVJUKHBjaWRldnNfbG9ja2VkKCkpOw0KPiANCj4gSXMgdGhp
cyBuZWNlc3NhcnkgZm9yIC4uLg0KPiANCj4+ICsgICAgQVNTRVJUKHJ3X2lzX3dyaXRlX2xvY2tl
ZCgmcGRldi0+ZG9tYWluLT5wY2lfbG9jaykpOw0KPj4gKw0KPj4gKyAgICB2cGNpX2RlYXNzaWdu
X2RldmljZShwZGV2KTsNCj4+ICsgICAgcmV0dXJuIHZwY2lfYXNzaWduX2RldmljZShwZGV2KTsN
Cj4gDQo+IC4uLiBlaXRoZXIgb2YgdGhlc2UgY2FsbHM/IEJvdGggZnVuY3Rpb25zIGRvIHRoZW1z
ZWx2ZXMgb25seSBoYXZlIHRoZQ0KPiAybmQgb2YgdGhlIGFzc2VydHMgeW91IGFkZC4NCkkgY2hl
Y2tlZCBjb2RlcyBhZ2FpbjsgSSB3aWxsIHJlbW92ZSB0aGUgdHdvIGFzc2VydHMgaGVyZSBpbiBu
ZXh0IHZlcnNpb24uDQoNCj4gDQo+PiAtLS0gYS94ZW4vaW5jbHVkZS9wdWJsaWMvcGh5c2Rldi5o
DQo+PiArKysgYi94ZW4vaW5jbHVkZS9wdWJsaWMvcGh5c2Rldi5oDQo+PiBAQCAtMjk2LDYgKzI5
NiwxMyBAQCBERUZJTkVfWEVOX0dVRVNUX0hBTkRMRShwaHlzZGV2X3BjaV9kZXZpY2VfYWRkX3Qp
Ow0KPj4gICAqLw0KPj4gICNkZWZpbmUgUEhZU0RFVk9QX3ByZXBhcmVfbXNpeCAgICAgICAgICAz
MA0KPj4gICNkZWZpbmUgUEhZU0RFVk9QX3JlbGVhc2VfbXNpeCAgICAgICAgICAzMQ0KPj4gKy8q
DQo+PiArICogTm90aWZ5IHRoZSBoeXBlcnZpc29yIHRoYXQgYSBQQ0kgZGV2aWNlIGhhcyBiZWVu
IHJlc2V0LCBzbyB0aGF0IGFueQ0KPj4gKyAqIGludGVybmFsbHkgY2FjaGVkIHN0YXRlIGlzIHJl
Z2VuZXJhdGVkLiAgU2hvdWxkIGJlIGNhbGxlZCBhZnRlciBhbnkNCj4+ICsgKiBkZXZpY2UgcmVz
ZXQgcGVyZm9ybWVkIGJ5IHRoZSBoYXJkd2FyZSBkb21haW4uDQo+PiArICovDQo+PiArI2RlZmlu
ZSBQSFlTREVWT1BfcGNpX2RldmljZV9zdGF0ZV9yZXNldCAgICAgMzINCj4gDQo+IE5pdDogSnVz
dCBhIHNpbmdsZSBibGFuayBhcyBhIHNlcGFyYXRvciB3aWxsIGRvIGhlcmUsIGZvciBnb2luZyBw
YXN0IHRoZQ0KPiBjb2x1bW5hciBhcnJhbmdlbWVudCBvZiBvdGhlciAjZGVmaW5lLXMgYW55d2F5
Lg0KT0suDQo+IA0KPj4gIHN0cnVjdCBwaHlzZGV2X3BjaV9kZXZpY2Ugew0KPj4gICAgICAvKiBJ
TiAqLw0KPj4gICAgICB1aW50MTZfdCBzZWc7DQo+IA0KPiBJcyByZS11c2luZyB0aGlzIHN0cnVj
dCBmb3IgdGhpcyBuZXcgc3ViLW9wIHN1ZmZpY2llbnQ/IElPVyBhcmUgYWxsDQo+IHBvc3NpYmxl
IHJlc2V0cyBlcXVhbCwgYW5kIGhlbmNlIGl0IGRvZXNuJ3QgbmVlZCBzcGVjaWZ5aW5nIHdoYXQg
a2luZCBvZg0KPiByZXNldCB3YXMgZG9uZT8gRm9yIGV4YW1wbGUsIG90aGVyIHRoYW4gRkxSIG1v
c3QgcmVzZXQgdmFyaWFudHMgcmVzZXQgYWxsDQo+IGZ1bmN0aW9ucyBpbiBvbmUgZ28gYWl1aS4g
SW1vIHRoYXQgd291bGQgYmV0dGVyIHJlcXVpcmUgb25seSBhIHNpbmdsZQ0KPiBoeXBlcmNhbGws
IGp1c3QgdG8gYXZvaWQgcG9zc2libGUgY29uZnVzaW9uLiBJdCBhbHNvIHJlYWRzIGFzIGlmIEZM
UiB3b3VsZA0KPiBub3QgcmVzZXQgYXMgbWFueSByZWdpc3RlcnMgYXMgb3RoZXIgcmVzZXQgdmFy
aWFudHMgd291bGQuDQpJZiBJIHVuZGVyc3Rvb2QgY29ycmVjdGx5IHRoYXQgeW91IG1lYW4gaW4g
dGhpcyBoeXBlcmNhbGwgaXQgbmVlZHMgdG8gc3VwcG9ydCByZXNldHRpbmcgYm90aCBvbmUgZnVu
Y3Rpb24gYW5kIGFsbCBmdW5jdGlvbnMgb2YgYSBzbG90KGRldik/DQpCdXQgaXQgY2FuIGJlIGRv
bmUgZm9yIGNhbGxlciB0byB1c2UgYSBjeWNsZSB0byBjYWxsIHRoaXMgcmVzZXQgaHlwZXJjYWxs
IGZvciBlYWNoIHNsb3QgZnVuY3Rpb24uDQoNCj4gDQo+IFRoaXMgbWF5IHNlZW0gdG8gbm90IG1h
dHRlciByaWdodCBub3csIGJ1dCB0aGlzIGlzIGEgc3RhYmxlIGludGVyZmFjZSB5b3UNCj4gYWRk
LCBhbmQgaGVuY2UgbW9kaWZ5aW5nIGl0IGxhdGVyIHdpbGwgYmUgY3VtYmVyc29tZSwgaWYgcG9z
c2libGUgYXQgYWxsLg0KPiANCj4gSmFuDQoNCi0tIA0KQmVzdCByZWdhcmRzLA0KSmlxaWFuIENo
ZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Fri May 17 08:20:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:20:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723913.1129063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7soz-0001F3-CP; Fri, 17 May 2024 08:20:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723913.1129063; Fri, 17 May 2024 08:20:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7soz-0001EM-8d; Fri, 17 May 2024 08:20:09 +0000
Received: by outflank-mailman (input) for mailman id 723913;
 Fri, 17 May 2024 08:20:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7soy-0001DB-8k
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:20:08 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4460541a-1426-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 10:20:06 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5a88339780so439743066b.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 01:20:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c63bsm1086219266b.51.2024.05.17.01.20.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 01:20:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4460541a-1426-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715934006; x=1716538806; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JwLRH2paqlhiXyenCAKfx1MFSjd2W6ThzY8ap3PFnW0=;
        b=OVsr72DqBmrftxffBfKAHsWEuZ/HirIhHu6Vjs/ssCLfchW1k0WNhEd7AP5FcNUe/a
         PmjtrRYuDzIia7DTpQeY6wLB+H/7rwZTw372v0OLMMD1O4p8ol4Hl404IlkrTtUOp2Qj
         njOausqCQyVHxgCzZWvThnxcRRqrgnQ3txGlNC/rDvfjV5fgy4FSiJRb3JceYoKoSXFa
         bvPfTYg6wkG3OUsZ005vCtxxi6N8BkWYT6Tsc/nXnMGpnrYYyKMTSry3viLEXXJsZ++j
         EWoZab8tIFzJVY6/X9mBrdUgwHM6ZNUeQi2KOpApQ3xTXI4UMPY8JINNHtA2KiO5XGgc
         xOtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715934006; x=1716538806;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JwLRH2paqlhiXyenCAKfx1MFSjd2W6ThzY8ap3PFnW0=;
        b=cVRuLmEmxykW24LtoTO+cpRBaE+XBrO0WuFVsOVErcO6CeEna6RMWRJfPmalWNDvSQ
         HleejNoNHwyPshpjno+SBm6N0KhYvaHtiODgT0twiuRyT8lIC4sCmDnaONAnypWdYPba
         8a37+JBSU/aeFMIsFP8ubXfCIEknnNoI4V6VuYnGoe3uOcoYW0gx29ZHaPPBwyOMUBIC
         1g5g092lX0z23aTKcuTKOx1WYOE4mgLg2MJVw/X8GS2XZVaIBnOQ2HZI7obfhoNGsNj4
         hGU3100je/mL7m5PkarH5WXPjzUXjGTiLFfLJaoiuXouYUC8FfCxyHd+uTdyu03lkE4V
         +q2A==
X-Forwarded-Encrypted: i=1; AJvYcCVJv4x+wddp+mUHACoipkmlojbIBlYQEA1NHf96a0PN16Ez9WiXuuWs4US669WRkJvAk/Eq6CA0DreP/7kH93J8EdM7vh6P0V1OKr3hgQs=
X-Gm-Message-State: AOJu0Yy1Nbf9zrSudgJoJ4lHNDvh3RvzSPoH3SfnFCiAlm5ISqxh48Es
	jrj4G7o9i8VPHhrZvJ6lOqAw+EO+Ds7VWBraef4wKK47GUZsd11hKOBXquu+mQ==
X-Google-Smtp-Source: AGHT+IEgcGeMLCHyj23o/brDq84GaXE2h0UHjH4ipgB3FWNuH0Gz5G9nOELe3WNhfITNBbb2ClMAZQ==
X-Received: by 2002:a17:906:57cf:b0:a59:c963:82b with SMTP id a640c23a62f3a-a5a2d5cffe2mr1276259266b.33.1715934005707;
        Fri, 17 May 2024 01:20:05 -0700 (PDT)
Message-ID: <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
Date: Fri, 17 May 2024 10:20:03 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Content-Language: en-US
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 10:08, Chen, Jiqian wrote:
> On 2024/5/16 21:08, Jan Beulich wrote:
>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>  struct physdev_pci_device {
>>>      /* IN */
>>>      uint16_t seg;
>>
>> Is re-using this struct for this new sub-op sufficient? IOW are all
>> possible resets equal, and hence it doesn't need specifying what kind of
>> reset was done? For example, other than FLR most reset variants reset all
>> functions in one go aiui. Imo that would better require only a single
>> hypercall, just to avoid possible confusion. It also reads as if FLR would
>> not reset as many registers as other reset variants would.
> If I understood correctly that you mean in this hypercall it needs to support resetting both one function and all functions of a slot(dev)?
> But it can be done for caller to use a cycle to call this reset hypercall for each slot function.

It could, yes, but since (aiui) there needs to be an indication of the
kind of reset anyway, we can as well avoid relying on the caller doing
so (and at the same time simplify what the caller needs to do).

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 08:42:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:42:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723924.1129073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tAr-0005sn-47; Fri, 17 May 2024 08:42:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723924.1129073; Fri, 17 May 2024 08:42:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tAr-0005sg-0y; Fri, 17 May 2024 08:42:45 +0000
Received: by outflank-mailman (input) for mailman id 723924;
 Fri, 17 May 2024 08:42:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7tAq-0005sa-8y
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:42:44 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d547940-1429-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 10:42:43 +0200 (CEST)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2e538a264f7so4655951fa.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 01:42:43 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0ce30e1sm26430461fa.38.2024.05.17.01.42.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 01:42:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d547940-1429-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715935363; x=1716540163; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=k/og926+VA8QfuRf/6fE44ANqz0xu3nonJ6pZThkEz4=;
        b=MV22jNf0F0AxCC4hjnXAFbH73lOH25Vq7ZY+s/evSOt8ENInsYwqD6W7mGyRkAF1Tj
         2uFGRllTZBsguhyV6VZwOKEDi7YJbNmy+ZPcOnCZoWi3RZUuQisa1vMnzyouNJNV/9ys
         OFI1ujIAe4f6PaexY4j4xz7F5lKjo5W+Fcas3h4vbpCFm2iuUqAf28p7YPGDRs1aXg1M
         LE8g0e8jsVHVPud8Bgsb6pHdnXN6YE/jIgizKinemE37YoQ5x1+EEfUQoRV2RcNzXIMU
         vtYvEL9Y3aP2SITTOfuEOEb8rcaOma+yApYVNHU2iP2GUTNCJhwHR6qd0lmC8rHkgES9
         hAVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715935363; x=1716540163;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=k/og926+VA8QfuRf/6fE44ANqz0xu3nonJ6pZThkEz4=;
        b=co+vQIXpEpJaTl1klqza2wv7ODzNbEM69aVWQbDg/eQUkqOgoZqnQpl6zdHJ34SHVM
         oHw4RV++njfroqlzdJFgiJfRjLoG1LbgZzEm6hzNk/RgjhOMfwJcHYZW2wRaRwJLB+jv
         uDKybF9HuKqNMRQ3UtseY36QHRxPJOCXsnvQw+fo0Mb2VsULCHzDpc6YDm3zjSaiHTtK
         sPKYS1RZxwz79J/gP0mSbyvekGTX6tlwB2/VP1TwY8MR98vUB1WGtfgWa4nj3+NnsoPe
         z+SrFnRT/SirkqOKtQ7OfTOK7mlKOiIfVhsaLycYWa8Xb/2bQUIO+fJFTSkjiFg03K9K
         EKkw==
X-Forwarded-Encrypted: i=1; AJvYcCUtAZxs4HDbEaudhZaGWsLzbEoj3+vnsHIPH/+wWFmYieB6xYOjy7Nmu8iCJEvp0q/Q5zfeHSUFuEnRyhBVL47I0G2DgsNobTPUeogaXWM=
X-Gm-Message-State: AOJu0YzKzp2Hfoh8LSKiC/6aW0KmNzZ4A+5q25o0PWxLGziCeZUQougy
	oWaqCK+0ATYauLxllX1o/L17keJHYbVyO/qYmY+xkIZuJHLawYP5
X-Google-Smtp-Source: AGHT+IFeGpw6tKWnJnQiJQG1rbX1/2C3Z/yzv1oPX8qOjvz1sGGdSz7yaYUhUyu9ge7ws/jpoAaq0w==
X-Received: by 2002:a2e:8907:0:b0:2e6:f1d3:9e7a with SMTP id 38308e7fff4ca-2e6f253dbc2mr58854271fa.19.1715935362421;
        Fri, 17 May 2024 01:42:42 -0700 (PDT)
Message-ID: <e13316f78234e92ac0b3a1a0d6fb9ba8c6c9bc0f.camel@gmail.com>
Subject: Re: [PATCH v9 02/15] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
Date: Fri, 17 May 2024 10:42:41 +0200
In-Reply-To: <5a121770-9bf3-40c6-ae91-8dc70174dad5@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <616e8be09f217a766b96c4f9060f6658636a4338.1714988096.git.oleksii.kurochko@gmail.com>
	 <2ac0ca46-6957-4d31-95cb-3016de7b11b8@suse.com>
	 <76a84e9e239e1338fc6f86d243b74fa239d8091d.camel@gmail.com>
	 <5174d73f-677d-4c5c-9d4c-e651a29e4366@suse.com>
	 <588867d6e054264551e0e27b534955fce9b6d6d5.camel@gmail.com>
	 <77c6e497-631d-4f92-bd38-8ab7dba4841d@suse.com>
	 <c8bd47e8f8558d88b4d5a4a09ea10728006fd4d3.camel@gmail.com>
	 <5a121770-9bf3-40c6-ae91-8dc70174dad5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-16 at 12:49 +0200, Jan Beulich wrote:
> > Anyway, I am not sure I understand which approach I should use in
> > this
> > patch. You mentioned that possibly test_and_() can't have a generic
> > form, meaning it won't be a set of arch_test_and_() functions.
> >=20
> > So, can I rename arch__test_() and generic__test_() to arch_test_()
> > and
> > generic_test_(), respectively, and use the renamed functions in
> > _test_and*() in xen/bitops.h? Is my understanding correct?
>=20
> You could. You could also stick to what you have now - as said, I can
> accept that with the worked out explanation. Or you could switch to
> using arch__test_bit() and generic__test_bit(), thus having the
> double
> inner underscores identify "internal to the implementation"
> functions.
> My preference would be in backwards order of what I have just
> enumerated
> as possible options. I wonder whether really no-one else has any
> opinion
> here ...

I see that __test_bit() doesn't exist now and perhaps won't exist at
all, but in this patch we are providing the generic for test_bit(), not
__test_bit(). Thereby according to provided by me naming for test_bit()
should be defined using {generic, arch}_test_bit().

~ Oleksii




From xen-devel-bounces@lists.xenproject.org Fri May 17 08:44:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 08:44:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723926.1129082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tCd-0006Ov-Ds; Fri, 17 May 2024 08:44:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723926.1129082; Fri, 17 May 2024 08:44:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tCd-0006Oo-Ap; Fri, 17 May 2024 08:44:35 +0000
Received: by outflank-mailman (input) for mailman id 723926;
 Fri, 17 May 2024 08:44:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7tCc-0006Oi-AR
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 08:44:34 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ad6d69e7-1429-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 10:44:31 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CY8PR12MB7220.namprd12.prod.outlook.com (2603:10b6:930:58::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 08:44:28 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 08:44:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad6d69e7-1429-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fENSODaGdX7y8JfBve+NxUutd0XOEYDXq1VxKgBVfyYC5kxQHaz5ZIiyweXNwZX8JQkBxBQq2K8j1NJPOtDLCmxFL5sHSF9IPxCbPV1U7uKXeMCvkeWmNmvHzpkF5asTvIJwcqafBb8kwHbX70BxaNreVnR7TDpR86rjz1Y+qXANY3zC1AH79/NnSpC/PHI/7MKy3e8k6mbUnDBagNLwyRma2LGnTcKt7WYR+iE3GKWr/WAcQwmTGNcRbDemRoBMjuwQ3NU1gF/Y67iNKt/MThbV2LGj4d9nRLbRUtWjwRqXnEmAcJ/uGGqdthG32adq5n2X+EIHNTjCErklm3d68w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DtLpXzztWnQFiZ/jQJdTjRq4vRU5Crq+vtVahnSG6r0=;
 b=QPuiO1SRRBQeBBZQDNmggxM09hXY8OIQdLoRP5wd94ygQFnxzaKEAGlIlBjfT0TdVCIy9qMH7le1M4MkdRnpvkJW1J4i3LZLhFTp2G7qnvGDDIfYbgrHYjNJkQpkgCvY5tJ1gbCleHTPXng1dQUIBRDrLKA4sw3bMj7pqmcODqC1mB6c2qrDo4xqQbwCWHQnSFrL06rJKWdmaxJBgd262QE16dbVlhsSpMlxGVIkPXc9J1sw5IvLjMCqsZjZ0tP25tIiOQFZzmIGcqpwyjvcD/aztroHW9O6ryyqRdalWqBCuKTbG0/kaGMULtjtTFUYXZCTQAF0Xk1/9JBoqEAjyw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DtLpXzztWnQFiZ/jQJdTjRq4vRU5Crq+vtVahnSG6r0=;
 b=u4e9+68JvqYaF1fMH8E+XsvPrX0am+NXdHgjJP65z7t22Dk0RZwyz6og9yorHJ58yLW7EVUPQHGzAvet5H0wILI2yedkjvGfS+s8j1DWwX6x3z4iDRYzjFw7lJtXLp0efTaQabNh0LH5Mwi8DJRwnBgJF92hZkHI0oPTweYnx0E=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>, "Hildebrand, Stewart"
	<Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Chen,
 Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 2/5] x86/pvh: Allow (un)map_pirq when dom0 is PVH
Thread-Topic: [XEN PATCH v8 2/5] x86/pvh: Allow (un)map_pirq when dom0 is PVH
Thread-Index: AQHap3bYQFs2wap0sUqpOSsc/BlazbGZ21MAgAG+yQA=
Date: Fri, 17 May 2024 08:44:27 +0000
Message-ID:
 <BL1PR12MB584994C1B2CBAD18E77A6D36E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-3-Jiqian.Chen@amd.com>
 <673505f6-2d0f-4ea7-ab9f-c132adce28c4@suse.com>
In-Reply-To: <673505f6-2d0f-4ea7-ab9f-c132adce28c4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CY8PR12MB7220:EE_
x-ms-office365-filtering-correlation-id: f80294f1-04f3-4f1f-c3cb-08dc764d8fa7
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|376005|366007|7416005|1800799015|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?Yy8xNVE2NlVscXRxUVYrWVVqKzBHeVduN1lsdHk3L0ZETFVtTG1RQXRVY2Qz?=
 =?utf-8?B?ZTZtZU5NTzJOeElmOGh4U3l3VnpPeDJnSlUxWXdYS1ZwMWlzZHdZdHNqRlk2?=
 =?utf-8?B?RnlOOUtyL01oRzIyVlQ1dE1yTHpvNk1nQnc2dzZ2YngyQjBiQm9WQjlaeHB0?=
 =?utf-8?B?Vm5NRDlGdUhSMy9oWHlHeXhBS2l5Y3RQQVhTeVRGWFNHSytWNzkwSGF2cmFL?=
 =?utf-8?B?aE1GcnBaUHQxZGYySGdSTGJNdm1ST0huL1NlNitMdWlUSVlKbWtvd1haZklF?=
 =?utf-8?B?cmNkbVN5aUJKWVNpTlVXZWZJckJnMWtaZjM1VWFuQTFzdWZLQytHYmZTRiti?=
 =?utf-8?B?VFNEQlY0V3piQUk2OXlkNnNLTFpIV2Y0VEhLTjRrTzVicW40U1FBTU45bDhu?=
 =?utf-8?B?OFVQdzRIT01sTE1QSElGdGVGdzdrVlB0a3ZZRytWSThoTnc0TTFpR1hGYU9Y?=
 =?utf-8?B?a0xmNnVmdlJyZWRBTkxQZTU5elhoZVhJWEFyWCtkSjhuRUhPWFlNdTNXb2ht?=
 =?utf-8?B?bWtjQ2ZGbGFUdVZZY0kyajhpQ0s1QjVDK2h1UDg3bWcrd3RIVC9wZyt3aHB4?=
 =?utf-8?B?eHJyakh2bUJoVFlUazFuR0Z0bmtDaklWSW1LY3ZpUkduWW5oMENMWlM5aUda?=
 =?utf-8?B?UzFlajNjblpobytnTDBXczB0M0U3WjB3bEsxNkV4NnVTWm0yZ3luL05OVHcx?=
 =?utf-8?B?c244ajVjUmd1Q2VzQWZSZlNCWThKMi90ZlJSR1ZnazNqYm8vZHdsMFVhTXNa?=
 =?utf-8?B?MjZWc0NvTkhpa2MveUUyUE9BVEVITVJWdmhha0g0WHBsZGFNVC9pVmpZY0o1?=
 =?utf-8?B?dDlJait2akJta1FUT1I4L21valRrdjBYV0FoaHpTSEt1aytmSkNiaEFQL3dm?=
 =?utf-8?B?aWhkUmg1dm9PZUlYN3lMdjI4UFNnM2xiUHlXV1MzTENWamlzdFZvRkQ0SWdy?=
 =?utf-8?B?SzdaZ3F6RGkzRHFwenJIOHd2b2RITSsxTnFlMVUxNGc5eXZndnBEcnV2QStq?=
 =?utf-8?B?Y1huaGw2MnpWQ29UVVZ0WlE4b1NDbTdqSWRoeHJCNW9yRFY3VzZIZEhuMGZh?=
 =?utf-8?B?dE1GeGVwRjlydTRPd2x3dGRieVM3T2NCeUw2VGFCaXFjY1MrMVlRRXlPNE1E?=
 =?utf-8?B?Rll5ZDNFNVFlQ0g1Wm5pNzBMUVFmanpOdkxZS2t4RjdhUXZqZDdPUDgzWmQw?=
 =?utf-8?B?eVVoU0N3NElBK3RUYzBwYW8rbG1zWUFRbUZld0xURXVVQ1VWUUFWcmFYOXJK?=
 =?utf-8?B?V0FOZzNBK1A3OGpmQU9TN3B6b1JiU2diNDM3d1R2d0tBV21hWEtMM29vWVJG?=
 =?utf-8?B?ZGRPTklzREFmUkJMNS9nQ2tJWFNRVGFkb1p1aWVrZW1nYlVhSUVxVm5DU1Mz?=
 =?utf-8?B?cXRNS2o2eWZTaTN4dTBtbk5DbWdoT0FZbnlscS8xaGxPRm1MK0FUdFl2L2JJ?=
 =?utf-8?B?R01nNGx5YzZXNWp0WnRGU3hEcTA5MVBUNVk5dy9YQk5mUHNjZlRjZ055TDJX?=
 =?utf-8?B?UU81NjNBbEtIZzB1c01uamdmM0RuZ2szbExUVWZqSnplUG9tYnVSNTl6TUZm?=
 =?utf-8?B?VHJ1UlFLWXE4QkR6cG9rYWhkYXMrdzhZTWtjcUIzY1p4eHNYckJrY0J5Vjhr?=
 =?utf-8?B?V1gwUVZmV1FUTGl1dWxMZmsyS1ptR3pRNHdMV05lNnJwZVVZangySExYWG5D?=
 =?utf-8?B?UlBMMHdjSW9BRmk0MjFCUFoyN3BpenovY1FvVUxYc2czdVhkT3JVcVpOVkZO?=
 =?utf-8?B?QVlOY25pTXoza0ZIaU1nZWQ0WFl6akI0Q2FhVENtc3ZmQUdoK3gvM1NRam8w?=
 =?utf-8?B?cUdjSTY0aSs5WGREaHFydz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(7416005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?TWw3UGMyQnJSbFdWZGdheDZVYlJOMDl5dlk5eHJUSWdGLzI5VU9DNXB4cytm?=
 =?utf-8?B?OTNCTmxodk9CdjZFWVNDZjE1STZrc1NyL0FkaVR2SC9hUFFya2RSNlV0ZmlR?=
 =?utf-8?B?UEllRVdXaG9iTGRlZkRVL2FVcEhsS0xnVlVVdVJzMVRXQzJDZUQ0SnZFN3hH?=
 =?utf-8?B?L3Vxd0o0RVIvTmR5eEM0NUd1RUN6WXh5TXdHeDU1dFAwaHhSc3NYUTdEUmtp?=
 =?utf-8?B?VHIzaWZvODB3VTRuRC9TK0VvQlpaWWtGM3FaZ2h0b3J0RDBFWVdSdWI4RTB6?=
 =?utf-8?B?YmNRdDFnOEtCcVhkcDNuRlZFMTRmSFJEME9RcjFPTVJLeU1qWmZadVV4bldY?=
 =?utf-8?B?YkoxUGtJMDNHK0FFV3JEN2lkNERHSVQ5NUxYa0xzamY4NDdFRzNHMUJ6Y245?=
 =?utf-8?B?TVVZWjVpMUNta0h6Qjg1dmdBdVZXZGpNR0ZrS2NNWENDYzR0TFhWdHpkREpm?=
 =?utf-8?B?dWkyUlgzUXVpaGt3bWRXbkJqWU9DcmhuME5VWFN4R2VYakpVOEtQVStYZ3N4?=
 =?utf-8?B?WkgyMUFwMkVvWHJhbGkydEovWUUxc3NjeThrS3JxYmZoYnFKVTNtRWhiT2Z2?=
 =?utf-8?B?STFEV3l2S1BaYkFWYVhLUVZ0aUNaSkt3d1lUYmxrU0hreTVaMG82YjNTMGJp?=
 =?utf-8?B?N1FHYjJUVUp1UG56eE0yaGdONmkxNEhkZW5kWVVzSDNQWWZnU1d5YnAvaDBT?=
 =?utf-8?B?bzJNSDdVME8xZ0xvSDkrNmVEaW10OC9PYlZCVGxNUVQzTHN3N3Rtb3ZLbHZk?=
 =?utf-8?B?U0JiOU45NE52ZU5xc2ZGM1FEY2lWenN4TkpiNDBkbk1hcDljTU1jOVlhRktO?=
 =?utf-8?B?eTFDZ1UyWEVITERjZjQ1N0J2Z0RmSHljb2pEQkxnc1lEUFEzU3I4VkJFaTNF?=
 =?utf-8?B?WVBHWm5LbkJiTG9QZ2NUWER6R3kzUDljMGdUajZZWXpJNlcvNENDcFdJUitY?=
 =?utf-8?B?blJsNFNDTG0vZnpuOFB3NGR0bzR6SUFFeS9OTW9rNGx5STlSbGdWNCticmRk?=
 =?utf-8?B?bmI0YVp3MnRmNE5UWWNnd3pCTFd3SVBpNTEySHBiV0pWQUJSVXVVWXppWDBj?=
 =?utf-8?B?OE9yZFVZOVF4YUx0N1dUdVE5dEt4T0ovelhuVkY1TllHSkpJdS9odENxWXpj?=
 =?utf-8?B?QnM5YXVRUjBMZGlXYUZHeFR3czRhWUJSeTcrVDdIRGd4L2krb1BFbFlnQ0h2?=
 =?utf-8?B?TXpmbEk2ZmpnbXFXZjhwT2RQbmxTeEZaNGFBYUs2eWN0U1FiMGpRcUp1YW1w?=
 =?utf-8?B?ekQvemdlbVdaVFZGNTFualdYL0FCbVduQlRjMURERjJZdVliaWF6T0toeE1x?=
 =?utf-8?B?emxJbGZOSUpWWnN1VDd6VWdiV21yMEJSZ0huRzBlRzBXTXh5cjBsRTdqeEQr?=
 =?utf-8?B?bEpad2hLWFlDMW13OVJGc2o4d3N3M01ZRnh6QVh3aGpBOXEvdUlBNGxUQWpR?=
 =?utf-8?B?ZHhTelBpek5hWjB5UlVucXlzK0M1SGN2NGZDcHFYaFFqSHRIaUViSEFuUFBp?=
 =?utf-8?B?MFJDcDFCYlV5R0k4VENKdGpuZGZxb2hZaFI1b3MwU1pWdlVRbFhaZWh0bXEz?=
 =?utf-8?B?SHB1QnUyT0VzMkUzTk5KSktxYTRpVGwxNUdESndTME5ydS9VUXZtSExYNnlF?=
 =?utf-8?B?bTRaRW93aW5JYS8yVFRheHlTMVIweUF6eG5oeXZ1b2plR3MvMnZ6Q1BPYUU4?=
 =?utf-8?B?ZUcvMXE5RmQ3eHl6Y25JYVB4UjNOWWxFRDI0R0p6Z0JBZHppaTc0ZExtcjBJ?=
 =?utf-8?B?M2lkNmhLMW5oRW9GcDk0TE9SbVBUSy9yYitaYlVMNktQNEVZSHg1OXFvZmN6?=
 =?utf-8?B?M1NibWJYNUZFaEF6NkxRcEVJQnRjOWRucHlrV3B4clFwVWpPNWJjT0ZkYmc2?=
 =?utf-8?B?NTVyYzRVa0FYTTVVbVVwZkFtbEtoK2dzNkh2c2l6VUE4REhpYkFQVHB6OGlh?=
 =?utf-8?B?UTBFQm4xWVcvNU0wUTBtSEFSOWh3bEdKTTZKN21YMkNOakQrVk5UcjEreDZO?=
 =?utf-8?B?azRjV1U3WlVoNmpDQnR5OEtJZkRKL0U1cEdsSG9wZ1hpbG80eFhtS2ZaeGs3?=
 =?utf-8?B?dVVDZXlycERobG56N3RHYS9PSTVMTlpGQ2N6OXBSaHh1Y0pVZWU0eVliTUNr?=
 =?utf-8?Q?detw=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <6B7A2270D5C38144891159313DFFA549@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f80294f1-04f3-4f1f-c3cb-08dc764d8fa7
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 08:44:27.4027
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Q6O7sIfP68bTrMvqJMXyStwPcoLdwmY6sGWTS9jBLy4pW908tTF9Cx61+b6ZN7R/CL2aU5tMPNNTxKmp4p0Hng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7220

T24gMjAyNC81LzE2IDIxOjI5LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTYuMDUuMjAyNCAx
MTo1MiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBJZiBydW4gWGVuIHdpdGggUFZIIGRvbTAgYW5k
IGh2bSBkb21VLCBodm0gd2lsbCBtYXAgYSBwaXJxIGZvcg0KPj4gYSBwYXNzdGhyb3VnaCBkZXZp
Y2UgYnkgdXNpbmcgZ3NpLCBzZWUNCj4+IHhlbl9wdF9yZWFsaXplLT54Y19waHlzZGV2X21hcF9w
aXJxIGFuZA0KPj4gcGNpX2FkZF9kbV9kb25lLT54Y19waHlzZGV2X21hcF9waXJxLg0KPiANCj4g
eGVuX3B0X3JlYWxpemUoKSBpcyBpbiBxZW11LCB3aGljaCBpbW8gd2FudHMgc2F5aW5nIGhlcmUg
KGZvciBiZWluZyBhIGRpZmZlcmVudA0KPiByZXBvKSwgdGhlIG1vcmUgdGhhdCBwY2lfYWRkX2Rt
X2RvbmUoKSBpcyBpbiBsaWJ4bC4NCk9LLCBJIHdpbGwgZGVzY3JpYmUgbW9yZSBoZXJlKGluIHFl
bXUgYW5kIGluIGxpYnhsKS4NCg0KPiANCj4+IC0tLSBhL3hlbi9hcmNoL3g4Ni9odm0vaHlwZXJj
YWxsLmMNCj4+ICsrKyBiL3hlbi9hcmNoL3g4Ni9odm0vaHlwZXJjYWxsLmMNCj4+IEBAIC03NCw2
ICs3NCw4IEBAIGxvbmcgaHZtX3BoeXNkZXZfb3AoaW50IGNtZCwgWEVOX0dVRVNUX0hBTkRMRV9Q
QVJBTSh2b2lkKSBhcmcpDQo+PiAgICAgIHsNCj4+ICAgICAgY2FzZSBQSFlTREVWT1BfbWFwX3Bp
cnE6DQo+PiAgICAgIGNhc2UgUEhZU0RFVk9QX3VubWFwX3BpcnE6DQo+PiArICAgICAgICBicmVh
azsNCj4gDQo+IEkgdGhpbmsgdGhpcyBjb3VsZCBkbyB3aXRoIGEgY29tbWVudCBhcyB0byB3aHkg
aXQncyBwZXJtaXR0ZWQgYXMgd2VsbCBhcyBnaXZpbmcNCj4gYSByZWZlcmVuY2UgdG8gd2hlcmUg
ZnVydGhlciByZXN0cmljdGlvbnMgYXJlIGVuZm9yY2VkIChvciBzaW1wbHkgbWVudGlvbmluZw0K
PiB0aGUgY29uc3RyYWludCBvZiB0aGlzIG9ubHkgYmVpbmcgcGVybWl0dGVkIGZvciBtYW5hZ2Vt
ZW50IG9mIG90aGVyIGRvbWFpbnMpLg0KVGhhbmtzLCB3aWxsIGFkZA0KLyogDQogICogT25seSBi
ZWluZyBwZXJtaXR0ZWQgZm9yIG1hbmFnZW1lbnQgb2Ygb3RoZXIgZG9tYWlucy4NCiAgKiBGdXJ0
aGVyIHJlc3RyaWN0aW9ucyBhcmUgZW5mb3JjZWQgaW4gZG9fcGh5c2Rldl9vcC4NCiAgKi8NCg0K
PiANCj4+IC0tLSBhL3hlbi9hcmNoL3g4Ni9waHlzZGV2LmMNCj4+ICsrKyBiL3hlbi9hcmNoL3g4
Ni9waHlzZGV2LmMNCj4+IEBAIC0zMDUsMTEgKzMwNSwyMyBAQCByZXRfdCBkb19waHlzZGV2X29w
KGludCBjbWQsIFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0odm9pZCkgYXJnKQ0KPj4gICAgICBjYXNl
IFBIWVNERVZPUF9tYXBfcGlycTogew0KPj4gICAgICAgICAgcGh5c2Rldl9tYXBfcGlycV90IG1h
cDsNCj4+ICAgICAgICAgIHN0cnVjdCBtc2lfaW5mbyBtc2k7DQo+PiArICAgICAgICBzdHJ1Y3Qg
ZG9tYWluICpkOw0KPj4gIA0KPj4gICAgICAgICAgcmV0ID0gLUVGQVVMVDsNCj4+ICAgICAgICAg
IGlmICggY29weV9mcm9tX2d1ZXN0KCZtYXAsIGFyZywgMSkgIT0gMCApDQo+PiAgICAgICAgICAg
ICAgYnJlYWs7DQo+PiAgDQo+PiArICAgICAgICBkID0gcmN1X2xvY2tfZG9tYWluX2J5X2FueV9p
ZChtYXAuZG9taWQpOw0KPj4gKyAgICAgICAgaWYgKCBkID09IE5VTEwgKQ0KPj4gKyAgICAgICAg
ICAgIHJldHVybiAtRVNSQ0g7DQo+PiArICAgICAgICAvKiBJZiBjYWxsZXIgaXMgdGhlIHNhbWUg
SFZNIGd1ZXN0IGFzIGN1cnJlbnQsIGNoZWNrIHBpcnEgZmxhZyAqLw0KPiANCj4gVGhlIGNhbGxl
ciBpcyBhbHdheXMgY3VycmVudC4gV2hhdCBJIHRoaW5rIHlvdSBtZWFuIGlzICJjYWxsZXIgaXMg
c2FtZSBhcw0KPiB0aGUgc3ViamVjdCBkb21haW4iLiANClllcywgSSB3YW50IHRvIHByZXZlbnQg
c2VsZi1tYXAgd2hlbiBzdWJqZWN0IGRvbWFpbihkb21VKSBkb2Vzbid0IGhhdmUgWDg2X0VNVV9V
U0VfUElSUSBmbGFnLg0KDQo+IEknbSBhbHNvIGhhdmluZyB0cm91YmxlIHdpdGggc2VlaW5nIHRo
ZSB1c2VmdWxuZXNzIG9mIHNheWluZyAiY2hlY2sgcGlycSBmbGFnIi4gSW5zdGVhZCBJIHRoaW5r
IHlvdSB3YW50IHRvIHN0YXRlIHRoZQ0KPiByZXN0cmljdGlvbiBoZXJlIHRoYXQgeW91IGFjdHVh
bGx5IG1lYW4gdG8gZW5mb3JjZSAod2hpY2ggd291bGQgYWxzbyBtZWFuDQo+IG1lbnRpb25pbmcg
UFZIIGluIHNvbWUgd2F5LCB0byBkaXN0aW5ndWlzaCBmcm9tIHRoZSAibm9ybWFsIEhWTSIgY2Fz
ZSkuDQpZZXMsIFBWSCBhbmQgdGhlIEhWTSB3aXRob3V0IFg4Nl9FTVVfVVNFX1BJUlEgZmxhZywN
CklmIGEgSFZNIGhhcyBYODZfRU1VX1VTRV9QSVJRIGZsYWcsIG1hcF9waXJxIHNob3VsZCBiZSBw
ZXJtaXR0ZWQuDQoNCkkgd2lsbCBjaGFuZ2UgdGhpcyBjb21tZW50IHRvIGJlOg0KLyogUHJldmVu
dCBzZWxmLW1hcCB3aGVuIGRvbWFpbiBoYXMgbm8gWDg2X0VNVV9VU0VfUElSUSBmbGFnICovDQoN
Cj4gDQo+PiArICAgICAgICBpZiAoICFpc19wdl9kb21haW4oZCkgJiYgIWhhc19waXJxKGQpICYm
IG1hcC5kb21pZCA9PSBET01JRF9TRUxGICkNCj4gDQo+IFlvdSBleGNsdWRlIERPTUlEX1NFTEYg
YnV0IG5vdCB0aGUgZG9tYWluJ3MgSUQ/IFdoeSBub3Qgc2ltcGx5IGNoZWNrIGQNCj4gYmVpbmcg
Y3VycmVudC0+ZG9tYWluLCB0aHVzIGNvdmVyaW5nIGJvdGggY2FzZXM/IA0KPiBQbHVzIHlvdSBj
b3VsZCB1c2UgcmN1X2xvY2tfZG9tYWluX2J5X2lkKCkgdG8gZXhjbHVkZSBET01JRF9TRUxGLCBh
bmQgeW91IGNvdWxkIHVzZQ0KPiByY3VfbG9ja19yZW1vdGVfZG9tYWluX2J5X2lkKCkgdG8gZXhj
bHVkZSB0aGUgbG9jYWwgZG9tYWluIGFsdG9nZXRoZXIuDQpCdXQgdGhlcmUgaXMgYSBjYXNlIHRo
YXQgaHZtIGhvbGQgUElSUSBmbGFnIGFuZCBET01JRF9TRUxGIGlkIHdpbGwgZG8gdGhpcyBwaXJx
X21hcCwgc2VlIGNvZGUNCnBoeXNkZXZfbWFwX3BpcnEuDQpJIHRoaW5rIGNoYW5nZSB0byBjaGVj
ayBkIGJlaW5nIGN1cnJlbnQtPmRvbWFpbiBpcyBtb3JlIHN1aXRhYmxlLg0KDQo+IEZpbmFsbHkg
SSdtIG5vdCBldmVuIHN1cmUgeW91IG5lZWQgdGhlIFJDVSBsb2NrIGhlcmUgKGVsc2UgeW91IGNv
dWxkDQo+IHVzZSBrbm93bmFsaXZlX2RvbWFpbl9mcm9tX2RvbWlkKCkpLiBCdXQgcGVyaGFwcyB0
aGF0J3MgYmV0dGVyIHRvIGNvdmVyDQo+IHRoZSBxZW11LWluLXN0dWJkb20gY2FzZSwgd2hpY2gg
d2UgaGF2ZSB0byBjb25zaWRlciBwb3RlbnRpYWxseSBtYWxpY2lvdXMuDQpZZXMsIGZvciBwb3Rl
bnRpYWwgc2FmZXR5IHJlYXNvbnMsIGxldCdzIGtlZXAgdGhlIFJDVSBsb2NrLg0KDQo+IA0KPiBJ
J20gYWxzbyBpbmNsaW5lZCB0byBzdWdnZXN0IHRvIHVzZSBpc19odm1fZG9tYWluKCkgaGVyZSBp
biBmYXZvciBvZg0KPiAhaXNfcHZfZG9tYWluKCkuDQpPSywgd2lsbCBjaGFuZ2UgdG8gaXNfaHZt
X2RvbWFpbiBpbiBuZXh0IHZlcnNpb24uDQoNCj4gDQo+IEphbg0KDQotLSANCkJlc3QgcmVnYXJk
cywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:00:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:00:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723938.1129093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tRo-0001kr-Pp; Fri, 17 May 2024 09:00:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723938.1129093; Fri, 17 May 2024 09:00:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tRo-0001kk-M1; Fri, 17 May 2024 09:00:16 +0000
Received: by outflank-mailman (input) for mailman id 723938;
 Fri, 17 May 2024 09:00:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7tRn-0001kd-BT
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:00:15 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dec7b3c7-142b-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 11:00:13 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CYXPR12MB9337.namprd12.prod.outlook.com (2603:10b6:930:d8::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 09:00:08 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 09:00:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dec7b3c7-142b-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gDT+hcHSiR0vmUhY6mkb5f/+KHLPoAglh1mYnt3uKw9q2v7NeQI53lHzCL9Kl+7FessOhwUvrX0xtGHg2i2QF9jgvADiXFJSQdbxqcWpiDVnUmAfyiLM19jRqffEyZW7D8DtVIl1zAWoxUxzmsgpOQrgVMsvTQOSxpC11u2otoZArWyYsovSbKVh1wgWkxkbGStGavivpdJGB4o1BCy1JZhxjD4UHEh3K4FonhdzCcQKqx57VfaeoQn45RoKx+f+PCgCGGmH27v9fwP4dgDjsHZ3thdCVJQsLtsG40joJgwS3JbNjJc3x5sM0SXnv3B/96YMxEbxOONpcJpPvP9Y/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=+R+nUC/jdNwPhayFfRejkS/Aq6B3U6J0pxAzdnK0lYQ=;
 b=YcCtZLenA3k0fWpn/x2g5F5k1Qc1T+U9910t6QeSF2yCkdziuWAHAqm/lcvCuZVBIbguauO5u3vUVH7ptkA2Bb9GMOr1tZWAxnGi03Lgg8WitpYZtugi8MJF4N30xA6RGRfUzHAZfwjxs8SX8oqz4Xv8rfmDPOg4CtakezEz//6foQ/wJm7dgk6PgoopmvUMD7JJXZz/YYlc9ZI1joBWamNUG+JY+YwVXWTZ8j1nx5jF2rs25P/4VVbSjI8rTwt5UAioQ9lTSqjbPZEPjp8KZbMrUTkMBaCJ+3bwp3tQT+0XwWxW5LE7taHpdOU0q9w5rG/EoA/h4WnQX02ctBI5qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+R+nUC/jdNwPhayFfRejkS/Aq6B3U6J0pxAzdnK0lYQ=;
 b=NgaBJE4KNYnNYmLqIjeLkY0j8LWLmgSEugrY5llCrxXjbQLqAYDSjjzEgF5Uz7hJ+2F1ekuckaQn9zDVfcsJtN72B0KfgO6/5iuBthrFcawHcYrgJNpL2HGolVn/c2Jyn/HjESRKd3AA5gY/IY9k3VDc+J64UT1j8qVyrz16qYc=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>, "Hildebrand, Stewart"
	<Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Chen,
 Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
Thread-Topic: [XEN PATCH v8 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
Thread-Index: AQHap3bbllnzPAJ6rEGPUd9rNZSctrGZ4LUAgAHFMQA=
Date: Fri, 17 May 2024 09:00:08 +0000
Message-ID:
 <BL1PR12MB5849A45EC6E19E53B074A3C0E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-4-Jiqian.Chen@amd.com>
 <34f6ac6d-0bad-478a-86c1-a124df5504c8@suse.com>
In-Reply-To: <34f6ac6d-0bad-478a-86c1-a124df5504c8@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CYXPR12MB9337:EE_
x-ms-office365-filtering-correlation-id: 4f47b596-cc63-420d-4ae7-08dc764fc0a7
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|7416005|1800799015|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?ZU5aT1k2bE1uSkl0MVJDclcyaXBac2gzcVl6NDFuU1BFZ2pUL2prYzZBMG1C?=
 =?utf-8?B?cXpjTzM1cjl2RDVaaVIyc0FmQUNIWS9MYUI0U3YvMlNqUVdaL3I4bzFNc2xN?=
 =?utf-8?B?WnJNREZrSFFJY0FpUHFjK01TK2xjY2tPdUZQR2JLL3BYUnRkQkgycTBBUEpv?=
 =?utf-8?B?c2FCaVN2djNKVlNIYzZHTW5sRWFkUGFtVzdoU0wzN3Z2SVhPZjAwd0F3Wk05?=
 =?utf-8?B?cGJsUnZsc2x2R1QzNDZpMGUzSWQ1ZktZWmE4bkhwWHBlSnJJaVI4S3llcmQ0?=
 =?utf-8?B?cDFMZEhFMVFtcG9temdNU2pyWHRPS0FpZnBWUkpnZW8xYW1pcWttUVVQZVRq?=
 =?utf-8?B?QXpqaU82MUE3blBoVXBUN0MyVE5ocUo1MTkwc1FiSWlSekh5MHp5TUxZN1F3?=
 =?utf-8?B?MlRIUzh5ay9TYkRJaGJha0k5UzNBZ1YxM2o2YVhvSjRlWUlZWFpRNG5jYWp0?=
 =?utf-8?B?S0M2T2ZJdld3Y0J0dnBGaWxnWGd6T2k1YjZvY21SK2JWZHVhWTZMYlFOZ0Fl?=
 =?utf-8?B?QXhZUE1ibnhRTHRqR3dGTUlxa1RMUnBqT2lMaGREcTF6VHBYOG8vekhMYUdo?=
 =?utf-8?B?WCsrL01RSU00d2tJVFJBbmpZV2E2aXJEOVM5aGZzWHNFd1pqWW45QlVzU09z?=
 =?utf-8?B?ZFR5SXU3ZHp1cTQvek9Wc3ZtT0VRbkFDNTc0MWxhWVJnNE9MQkVIVzI1d3Yx?=
 =?utf-8?B?RFRpWlQxVVJJSllTWUR5bmZBVm9FcmJoMUY3OG5sR1UxL3hJZmpvc3JucnJS?=
 =?utf-8?B?SkR1SkdwS0hLNFc3c3VVTHRiVzhQK05uQXFEMC9aeXQ5ckl4T0NYTEpsdlZX?=
 =?utf-8?B?WUhOVnc4QXdVdEFoTURJQ0pUM3A1RW9SeHZrQ2U5Y0RWZ0hnb1lKTjhLVWxz?=
 =?utf-8?B?MlcrKy8rRE1sRlVxMGRmejdRdFd1WWF2U0tmblU2elF1MEJzR3F3eTRzcE5D?=
 =?utf-8?B?UHRGM3Q3MkpaQUxVRzZ4UDBSenU3VTByYk1IQzlxd0pEUk1QTUprS3BQSkRN?=
 =?utf-8?B?VmdqNldJQm92NHBMVkY3RVZlR01QNGg1VXY3SGxCVEx4NFBDQjRQNmFaQXBP?=
 =?utf-8?B?QUEveU14czlIR2VMbXZSSUlFYzB5dnhVd2tZTis2dHF4NXFUNkNrN3ZsOHpo?=
 =?utf-8?B?ZEp5THBiaVZRWStoeVlvbTFFKzlUVXY4dHJ3WGoyRDlSZUJvL0paSUUzZmI3?=
 =?utf-8?B?RFE3NDBRNnBJZ2UxaWk2Z0xVK2drdGxrQzJJTTVTSmYxeHlqekZsR2Z5L2ZW?=
 =?utf-8?B?RlVsQVEyQVFObTNsbjZPQitqYWVudE04NTRIdGlML1JwMkhPQ3luWDRETW5I?=
 =?utf-8?B?eE9BNnkrYktmYXlCakl3VE45a1ozcGJUVkxybFZDTklMRFIzQ0YvSkdoREF1?=
 =?utf-8?B?L0RGQ1hDM0ZkYVR1bHlzRUFmUjRlU3dvSk9ZdG9rVzBXSTdWeFp5c2toWHIv?=
 =?utf-8?B?aTk4bmNJKytDczNJNTlSTDBIQytMK1hhbkNqRm1vRllTbjhaMFBvOGpQazFB?=
 =?utf-8?B?UDI2VDRZN3BmNE5JeXBTM0ZsRXd3bFJ6NTdRYjNEejlNY0VMOVpMUjVBR0dX?=
 =?utf-8?B?UzEycFVDT1EwRFJjem9DeWNhSzRFNTBzblRwajhIUno3dFVldWhEM01nZUZ4?=
 =?utf-8?B?cGhDcks4S2FLVzFaMlRPSlM2ZUJWQnc0dmxDcTRRaGRHVCszM0tYcm5PV25O?=
 =?utf-8?B?V09wQ0VNdGJyWjBUVEVUbkFUbkppZERlZ2JEUGhxWDRZN0t1NDViR2t3PT0=?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?cjlBRGowMlkybUVNbFM0d2ZneFhqTFI1S2RXOStRUVFtNW9JZDZWVmk1Z2xh?=
 =?utf-8?B?dlFVQ0tTRGRZeGtGTjZCUFJKYXBQVkF4bjQ0OHFOYkhYbWc5OHJtNUhzTXFU?=
 =?utf-8?B?Y3FuVXhjVGJZTHlOWkJyZENQdnZ1cEFqZTdKVU55c2ZoL2V6dlFQZFdQTTll?=
 =?utf-8?B?UHc1QktQQ2t0bUJBczYyOW1RTFV4cE52YXNYV0RBdUx4RE5aZDNkazlwOWl1?=
 =?utf-8?B?d2poOWJDNUI0cDJMNnlrTDZJanNCOC9OUzBzY2ZnSC9XNFhGUjdrTWRNN2Rl?=
 =?utf-8?B?MUl0OVNKUFFsSloyR0xVWTZ0K2dHamR2RGRwbWdaY2dEN24yc2FseHZ2UGdM?=
 =?utf-8?B?QWNBWTdGamhwMnl1czRHRE1oZmFuVXdsS3krZDArRDMxOFVqWDc3enNldzJr?=
 =?utf-8?B?enVnWUdlMVVTVmQ5ZzNyRG5JK2IvaHcveVUvMHl3Zm52cjZJbkpVRldkNy9j?=
 =?utf-8?B?MXZkclpzeStLNlJDKzZMMUZXeG9vUlE0bkJVZklXcWxXWk5yUWpuY3BPZ0py?=
 =?utf-8?B?TGVmNXduZCtFb0V5clhwQjZYTk9rbElSL3p0NHVmVHlGZlR4U0ZGbGFtK0lJ?=
 =?utf-8?B?NTEyTzRMRW9JSnRyKzBGblFuSThWZU9YK2lZMkNpcjhJc0hEbVF3bS9IWVZB?=
 =?utf-8?B?V3AxblNqVWx1ZXM3SCtkQTMvRTFOSW9DVTd5dFZzRCtYN0djb3FSajVIQ1hB?=
 =?utf-8?B?YktWbmRWWUI0bm9QNHhlNWlQVnNKakljVlFYdDdNSU5nTkxxME16ZU1xR2dn?=
 =?utf-8?B?Zk5QQ2ppSmYyVkp0MnVLZXFvcHBqNkFwMFFEbGZ5Z0FNNkJmNUdrZWZPWW4y?=
 =?utf-8?B?SndpYmJobjB5M2xQcFFnckp4aHlJU1drS05Da0JVbWR0NUozR2Nsa0IyTGJV?=
 =?utf-8?B?Z3NPRUliR0cvbWFvRElVSndwRnd3Y1Uzd2pLRXlIMGh4czFhV3E0MkJKamg2?=
 =?utf-8?B?NjVCS1Z3R1NDNjIveG9CUW5lUXlSSWUrTzV3MURLbzRzZTd2eFdLSGxNOHM3?=
 =?utf-8?B?MGNRK3ZuTVZUYTBFV3poWk96MHdVMVE3VHBhVnF6Vkg4L1NYRFZtYmxqRXNN?=
 =?utf-8?B?VFFZNVV1Mndmd3BKZ29YNm1NcWdJV2hlYWN2dVVOWXFCb0l1NnhRakRVZlN1?=
 =?utf-8?B?elhlY1A0QU55RFE3a2VsRlhVUkpJbjZORjFFaHJyYS9RL0VvRzhLZFRrN2lX?=
 =?utf-8?B?MWxvRFcweG1CZTQ2KzZJbXRoeDlLZFVqa2hGWGNqUTU0cnh2dHR4NWwrN2oy?=
 =?utf-8?B?blRnRDVISkNhdlN2RVMyOXdKL2EvK2E0bzhMeldXTElkbzRTMFNydXY3K2Vi?=
 =?utf-8?B?QTNZTWRhSGdJbUprZzZiaWxtS1kxbjdwRjJ6M0h1Z0pnRDBidDBYdXVhVTlh?=
 =?utf-8?B?QzFMSDNDQWpoVEFybkJtckRvdG1iQVRFbFZKVm1hM2xKQmNrSjFVK3JuOENC?=
 =?utf-8?B?ekVRN1dJZTBEaE11dWQrTDNPN0x1bUlXZUlpMDIyMFdIdjlZQTdMYXVMT0VB?=
 =?utf-8?B?djNSVVJiS1p0Q3hWT0pLSUpmOUhVb3N4WmtlTzdidmVGWkRhdDd2dE1zckF2?=
 =?utf-8?B?dTVkV2N1cUxwMlkrOVV0WDA3cmV3a2tNVXpxZ0d6dTJKN1FqZzZ1cDZrWUpJ?=
 =?utf-8?B?NTZuZ3dHMXoxQWVlYWJrcHA1MHRJWjdBMGZUaG1ZVFFycGR6UzNiNnRFR2Js?=
 =?utf-8?B?VlZSUzlaRWtOcnREdWZLSjROT09SSkZCRUFwbk5BVlpwd1dCK3luY1c2Kzh5?=
 =?utf-8?B?azM0VVc3S3dWTDJJbVg5SWZUbjNIOHlzUENQQVJoZ1g2OUpjazFDUXZCVzg5?=
 =?utf-8?B?ZU5sUWp0TXVBSy91NlBCYS9lbHBkaGtxOFF4ajQzaXZuMTF2NkluV3BVRzRw?=
 =?utf-8?B?NnY1Tnl6R3RWK05kanRmU1lxNFZzbEkyVXNteVFFbHJoMVZERThGendoV1Mz?=
 =?utf-8?B?OVZmaGhGTHNPMDJkZXY2R29CVVRabE5NOUpYSUM1dXRCM0dubnBIOHJ6bkRG?=
 =?utf-8?B?cVpvemRORjkwT3F1dFlIZmo4d1YzVHQ3V3k2Y2hNSXpLQTFFR0srK3ZDRmpY?=
 =?utf-8?B?eFRpMXN6d2pab3RjS2xGcWFoVGRxOEt2UFBYQy9OKzgrS2M5OW1XSFd3WGh0?=
 =?utf-8?Q?K0rE=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <152A4198D8A23041A70D24265DBA077A@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f47b596-cc63-420d-4ae7-08dc764fc0a7
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 09:00:08.6120
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 5F2vHtpGWyYHMTbOJ/9AKHvHZ6WA+6Lok495MTKu8JxObk4b65UcQwZHHXBvdq36OotoZY9eTSzSz1ouMfsmEw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR12MB9337

T24gMjAyNC81LzE2IDIxOjQ5LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTYuMDUuMjAyNCAx
MTo1MiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiAtLS0gYS94ZW4vYXJjaC94ODYvaHZtL2h5cGVy
Y2FsbC5jDQo+PiArKysgYi94ZW4vYXJjaC94ODYvaHZtL2h5cGVyY2FsbC5jDQo+PiBAQCAtNzYs
NiArNzYsMTEgQEAgbG9uZyBodm1fcGh5c2Rldl9vcChpbnQgY21kLCBYRU5fR1VFU1RfSEFORExF
X1BBUkFNKHZvaWQpIGFyZykNCj4+ICAgICAgY2FzZSBQSFlTREVWT1BfdW5tYXBfcGlycToNCj4+
ICAgICAgICAgIGJyZWFrOw0KPj4gIA0KPj4gKyAgICBjYXNlIFBIWVNERVZPUF9zZXR1cF9nc2k6
DQo+PiArICAgICAgICBpZiAoICFpc19oYXJkd2FyZV9kb21haW4oY3VycmQpICkNCj4+ICsgICAg
ICAgICAgICByZXR1cm4gLUVPUE5PVFNVUFA7DQo+PiArICAgICAgICBicmVhazsNCj4+ICsNCj4+
ICAgICAgY2FzZSBQSFlTREVWT1BfZW9pOg0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9pcnFfc3Rh
dHVzX3F1ZXJ5Og0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9nZXRfZnJlZV9waXJxOg0KPiANCj4g
QmVsb3cgaGVyZSB3ZSBoYXZlIGEgaGFyZHdhcmUtZG9tYWluLW9ubHkgYmxvY2sgYWxyZWFkeS4g
QW55IHJlYXNvbiBub3QNCj4gdG8gYWRkIHRvIHRoZXJlPyBZZXMsIHRoYXQgdXNlcyAtRU5PU1lT
LiBJbW8gdGhhdCB3YW50cyBjaGFuZ2luZyBhbnl3YXksDQo+IGJ1dCBldmVuIHdpdGhvdXQgdGhh
dCB0byBtZSBpdCB3b3VsZCBzZWVtIG1vcmUgY29uc2lzdGVudCBvdmVyYWxsIHRvIGhhdmUNCj4g
dGhlIG5ldyBjYXNlIHNpbXBseSBhZGRlZCB0aGVyZS4NCkFoIHllcywgSSByZW1lbWJlcmVkIHlv
dSBzdWdnZXN0IG1lIHRvIHVzZSBFT1BOT1RTVVBQIGluIHY0LCBpZiBjaGFuZ2UgdG8gRU5PU1lT
IGlzIGFsc28gZmluZSwgSSB3aWxsIG1vdmUgdG8gYmVsb3cgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+
IA0KPiBKdXN0IHRvIGRvdWJsZSBjaGVjazogSXMgdGhlcmUgYSByZXNwZWN0aXZlIExpbnV4IHBh
dGNoIGFscmVhZHkgKGlmIHNvLA0KPiBjcm9zcy1saW5raW5nIHRoZSBwYXRjaGVzIG1heSBiZSBo
ZWxwZnVsKT8NClllcywgbXkgY29ycmVzcG9uZGluZyBrZXJuZWwgcGF0Y2g6DQpodHRwczovL2xv
cmUua2VybmVsLm9yZy9sa21sLzIwMjQwNTE1MDY1MDExLjEzNzk3LTEtSmlxaWFuLkNoZW5AYW1k
LmNvbS9ULyNtYzU2YjExMTU2MmQ3YzY3ODg2ZGE5MDVlMzA2ZjEyYjNlZjgwNzZiNCANCkRvIHlv
dSBtZWFuIEkgbmVlZCB0byBhZGQgdGhpcyBsaW5rIGludG8gdGhlIGNvbW1pdCBtZXNzYWdlIG9u
Y2UgdGhlIGtlcm5lbCBwYXRjaCBpcyBhY2NlcHRlZD8NCj4gT3IgZG9lcyBQVkggTGludXggaW52
b2tlIHRoZSBzdWItb3AgYWxyZWFkeSBhbnl3YXksIGp1c3QgdGhhdCBzbyBmYXIgaXQgZmFpbHM/
IA0KPiANCj4gSmFuDQoNCi0tIA0KQmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:04:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:04:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723942.1129103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tVa-0002nc-Bm; Fri, 17 May 2024 09:04:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723942.1129103; Fri, 17 May 2024 09:04:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tVa-0002nV-8d; Fri, 17 May 2024 09:04:10 +0000
Received: by outflank-mailman (input) for mailman id 723942;
 Fri, 17 May 2024 09:04:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7tVY-0002nP-Vl
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:04:08 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ada59e5-142c-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 11:04:07 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59b49162aeso402418366b.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 02:04:07 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a5ecd37a3sm713726366b.79.2024.05.17.02.04.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 02:04:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ada59e5-142c-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715936647; x=1716541447; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OlHm4NYfhjgJBXbsD4SzErBSiDqj334BvJJYbaiMYhc=;
        b=WiAxpDiR+S59l4vtfSZCznjdPqysfAuUxPu1Q0tmVmnbCfmnSO22ahug4p0nR4q+mv
         XeNl5C8rSmZC705uNm/q7s83XFhD9/DuUttb3Nq/uM64xjrPz8xNQWrbAGwZt0S/1/cj
         yb8lqMq/xMiKha7LRG1Fm07i1krf/EmScvaTDnVld5chpEt20Bv9ugcLzQAWCfJkU5wq
         FrW2aTiQHp3yYAUp7pZludtO3dNrwsKxcLuJVKb9FDhY0tBqE1KUV9mXGLaiD4HsLoWz
         V/L1GyfdjlE2tgZ8dQiZuUVSGe1a6V1fiIdvn2rabPTsCNGUhl1NTV3AknjhSRnR17Xw
         lzBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715936647; x=1716541447;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OlHm4NYfhjgJBXbsD4SzErBSiDqj334BvJJYbaiMYhc=;
        b=CNTVjZquR2HlpM6CcF3BKo+9EvqyRyb/aDV53qxjYPTMLazojoA/3m8Fg9JfYfjQt6
         jJOZreBu5wutDfoM0PnOBqyddjQqnx7HXfbKydJjj7CKaZB6KUjHxr8dyRfsXeZtjNC7
         44wcY6fWaLu7dzlc13mNi9/Dg3++dFA2i00R1/b68+ENZJwUVoRvJP6ZYg8+ZU+m2NiC
         ls+jsno6J9fFehN0tluOp7TMzQk6yb9wfFaWrkNAKbKpQWdA0tcPOWMIK6beSKNtUUab
         yZB11nC8rC3OFaSod1rpn0XDBUXXXkMwYvB+zOlULmpYlJNokZfnrn0fLokZqLdBSSfv
         ZxAw==
X-Forwarded-Encrypted: i=1; AJvYcCVx6XQL+mhClhFt+ySmXRLrDqOWo0daundFR/ZRh/ixT0/TXfbDo7dujAW6b6lD8IXHk2N+woys4HDK5VQXxke3tainTsyC3Hx3i3pjv0M=
X-Gm-Message-State: AOJu0YyG9J6/IdyZ4YKCjsKt87l77LRxbJ4u96YpCZsG2bEnSRm++HkR
	2/G+3k3+EkeGudvMNo1QY2j9m+wnWjBRck2xeaOhYjJV9/3Pp8l+qieIWm57Cg==
X-Google-Smtp-Source: AGHT+IHc1AXfC2VvM63SODeaxhatqxfvtbGzFupNQ95M5xLb92uFNMP8YaLgCm7MO3Rc2Bkv30QbzQ==
X-Received: by 2002:a17:906:a190:b0:a59:a221:e2d4 with SMTP id a640c23a62f3a-a5a2d536d97mr1352690566b.8.1715936647091;
        Fri, 17 May 2024 02:04:07 -0700 (PDT)
Message-ID: <98ae62ad-a569-4db3-9914-b3b91f4a4370@suse.com>
Date: Fri, 17 May 2024 11:04:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0
Content-Language: en-US
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-4-Jiqian.Chen@amd.com>
 <34f6ac6d-0bad-478a-86c1-a124df5504c8@suse.com>
 <BL1PR12MB5849A45EC6E19E53B074A3C0E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849A45EC6E19E53B074A3C0E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 11:00, Chen, Jiqian wrote:
> On 2024/5/16 21:49, Jan Beulich wrote:
>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>> --- a/xen/arch/x86/hvm/hypercall.c
>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>> @@ -76,6 +76,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>      case PHYSDEVOP_unmap_pirq:
>>>          break;
>>>  
>>> +    case PHYSDEVOP_setup_gsi:
>>> +        if ( !is_hardware_domain(currd) )
>>> +            return -EOPNOTSUPP;
>>> +        break;
>>> +
>>>      case PHYSDEVOP_eoi:
>>>      case PHYSDEVOP_irq_status_query:
>>>      case PHYSDEVOP_get_free_pirq:
>>
>> Below here we have a hardware-domain-only block already. Any reason not
>> to add to there? Yes, that uses -ENOSYS. Imo that wants changing anyway,
>> but even without that to me it would seem more consistent overall to have
>> the new case simply added there.
> Ah yes, I remembered you suggest me to use EOPNOTSUPP in v4, if change to ENOSYS is also fine, I will move to below in next version.
> 
>>
>> Just to double check: Is there a respective Linux patch already (if so,
>> cross-linking the patches may be helpful)?
> Yes, my corresponding kernel patch:
> https://lore.kernel.org/lkml/20240515065011.13797-1-Jiqian.Chen@amd.com/T/#mc56b111562d7c67886da905e306f12b3ef8076b4 
> Do you mean I need to add this link into the commit message once the kernel patch is accepted?

Not necessarily the commit message itself, but below the --- marker.
If the kernel patch was accepted earlier than the Xen one (which imo it
better shouldn't be), you'd likely want to move to pointing at the
resulting commit.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:06:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:06:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723948.1129112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tXx-0003LV-Mg; Fri, 17 May 2024 09:06:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723948.1129112; Fri, 17 May 2024 09:06:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7tXx-0003LO-Jz; Fri, 17 May 2024 09:06:37 +0000
Received: by outflank-mailman (input) for mailman id 723948;
 Fri, 17 May 2024 09:06:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7tXw-0003KJ-TB
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:06:36 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c38a5394-142c-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 11:06:36 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51f1e8eaee5so1551915e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 02:06:36 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d36d3sm3232504e87.169.2024.05.17.02.06.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 02:06:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c38a5394-142c-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715936796; x=1716541596; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=9wWPg2RcvLbF3/ZiMfHO//CKV1nJJUaxYjJQloOp4Zk=;
        b=ELHtN5BYVTzWsZ1RdIVq4xlEnFT84oDg5aNPPBzF64UmEV+IE/nBCyr+FoaehmtmvS
         4sqxddyQEcyvvBO/H2SqjHNLlG2OzeBdsifJe3VX37bjpTnFVXtcsj/TcGIf12NFzB9V
         vD+q8Ec1yFOmZqAvO4XXqNDrmuFeakpYEHtPlKBL0q+9FYvF14X7BpfQDHd0Dn5Iizfu
         zyYhMm9rhcjXIGiixJ2MFIu+A4GG3uxAXfyLn9dZj5yTeGCxSqzqMrYq2vNHrPzz5Akn
         hXDpEhMA1u86J/ypCSV3htRgXq4z/t8RTEG3Q6oruFKWJV+gCrKvZufPBpGP6E3qGZ95
         oVjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715936796; x=1716541596;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9wWPg2RcvLbF3/ZiMfHO//CKV1nJJUaxYjJQloOp4Zk=;
        b=Ih/MfsNBqGKi2mLrJcLa8jBg9N91IqHqm+5oSJkBbkL0TsGyZP1cJtOmZnqS1I0SFj
         K9wSWdy1t3/alzWmrWMlL0LaXbA1v1zZYghwGLuJqnA43Pc4HadCi/838zmxgBw5mGpj
         D/YTE6uQ/jl5/pbUS1ohvf+3IBXzkKVjgnF8+ZPnEsY/QQys2sHFfzm11LDA50BzDBZG
         Gofqzn8aY948hy9JRCcYlSd8hl5YshYlDxmU51KlAlcT2pjS4fDW1PtOV5VH4Uy9gnGh
         +3qiklCeJDr4NJCDjKABGUOhDC2Acnt7Sj3csXcfFBYPuUwk/mNulloUfhAyNkqVfOZd
         hUYA==
X-Forwarded-Encrypted: i=1; AJvYcCWcMGiGUB43n6Wdag6T9OMUAOZPSyuGUkksvjj2Hiam7Tl4dFhCbvolF4eWsDWVJ6T0yphy9K17i/i4F39pAPT8D81oltu+gfPi6hkbYQ0=
X-Gm-Message-State: AOJu0YxhWMEbzSl/peuz+yS94OgFcC/jORuo7EuOxRSjEa2MI8TlD0eV
	2G7QjdM5x6sDjxTwqZT3c2IyAFMvNyeMhZtlTAXJFKS7zYUrEsy7
X-Google-Smtp-Source: AGHT+IE3yS7zTECqcemNQ4MmtlArLHFK3jnTiNKERP4VSX8hn+HQ9aet/+tj5NIUeTyaEn/Yaelfgg==
X-Received: by 2002:a05:6512:3083:b0:523:892c:44ff with SMTP id 2adb3069b0e04-523892c4573mr3463554e87.1.1715936795570;
        Fri, 17 May 2024 02:06:35 -0700 (PDT)
Message-ID: <9cee6531c9aa6650be803d004b06cc88a25df7f5.camel@gmail.com>
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Fri, 17 May 2024 11:06:34 +0200
In-Reply-To: <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
	 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
	 <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-15 at 11:09 +0200, Jan Beulich wrote:
> But this then needs carrying through to ...
>=20
> > --- a/xen/arch/arm/include/asm/arm64/bitops.h
> > +++ b/xen/arch/arm/include/asm/arm64/bitops.h
> > @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long
> > __ffs(unsigned long word)
> > =C2=A0=C2=A0 */
> > =C2=A0 #define ffz(x)=C2=A0 __ffs(~(x))
> > =C2=A0=20
> > -static inline int flsl(unsigned long x)
> > +static inline int arch_flsl(unsigned long x)
>=20
> ... e.g. here. You don't want to introduce signed/unsigned
> mismatches.
Is it critical for x86 to return int for flsl() and fls() or I can
update the return type for x86 too?

   static always_inline int arch_flsl(unsigned long x)
   {
       long r;
  =20
       asm ( "bsr %1,%0\n\t"
             "jnz 1f\n\t"
             "mov $-1,%0\n"
             "1:" : "=3Dr" (r) : "rm" (x));
       return (int)r+1;
   }
   #define arch_flsl arch_flsl
  =20
   static always_inline int arch_fls(unsigned int x)
   {
       int r;
  =20
       asm ( "bsr %1,%0\n\t"
             "jnz 1f\n\t"
             "mov $-1,%0\n"
             "1:" : "=3Dr" (r) : "rm" (x));
       return r + 1;
   }
   #define arch_fls arch_fls

Any specific reason why 'long' and 'int' types for r are used?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:29:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:29:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723960.1129123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ttc-0007hn-EC; Fri, 17 May 2024 09:29:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723960.1129123; Fri, 17 May 2024 09:29:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7ttc-0007hg-B7; Fri, 17 May 2024 09:29:00 +0000
Received: by outflank-mailman (input) for mailman id 723960;
 Fri, 17 May 2024 09:28:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7ttb-0007gt-Kj
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:28:59 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e2203afc-142f-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 11:28:57 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CH2PR12MB4133.namprd12.prod.outlook.com (2603:10b6:610:7a::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 09:28:51 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 09:28:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2203afc-142f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rs/9wyTk0qOi69I40OuS02bZkkCJbEw8x3fRnAm0Fp8wFmNVLP1x/Q2i7Kkt9zQucgQ5unxitEJ4AFuT9cw6VQCt0PuY0CUel9naT264X55w/wFfL7irAChA6ygiV21MZQEb0T6xKK2onITmlx610cuTSq6B9Q+pfvzjUKExpqlutv4djRHB/xsiwkWjiRp0lr+6iMY9RiChCHweh5OM5FwwBxu/ODpfiKCGAaeqXMMBvTdll39k9iY3209ey5QRTItZd1dafA/9nZUEzAcDat2+R5zRZIlF9i4tAqbWMvyK4fXzOS0KaKYmxFGygaXOe4eHBUMcqiCXKkDug2/e8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DRtjy8UEl8ifJ30kOfJdmlhWDEHW6vaTf9cB80QFjbI=;
 b=UMSMAlloqzEq5jD+lsG6LKY4HFUyjG2wotzYExKbYYbZAtMvvfFUWtH9Gozy/cLXQhymD/l1WA3G33e8opI69x52/4hS2jowUKuM/LwALFC67AI3Vi9F3k/2Sdd/g8pj6QCqcUbZZqR2U7gqoe6bln0SoA7lw7W5daULdgUzKsB+sBteWFF0Pl10aK3UrSmoB5TqMYraTqoJbFWxNMQZbKJUTCoVPpLiRUwQvll8+2JzwRCzYhAgExsKsgraZWvPUkFnmXtjmTFupIrmr4EaSe53SOe+4XYSTnm7xCMjyQaZu7Qwy5+Y4xO5KvNICUQJieu/o3l3W5pIr5QOiUGigw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DRtjy8UEl8ifJ30kOfJdmlhWDEHW6vaTf9cB80QFjbI=;
 b=3kARA8dcJgm46KcjWrrFhu5MsCwftoeH3Z0BmWjUyXqSwKTz5Hvz4Lt8FAnSc6B3WDA6g1xCSWE6mbwD/omoC+B6BN0aLCeopUnm8TnuevjSk+RLwA5MsPVwK4pf+rm2Li5NZRYpa5JKMIqVVGl4KSPKeCvHq3ljvBtBgT/cAYo=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Index: AQHap3bYcGtlWtrcpEyRLTK5fjq5SLGZ1UMAgAFwSAD//9GPgIAAlWQA
Date: Fri, 17 May 2024 09:28:51 +0000
Message-ID:
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
In-Reply-To: <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CH2PR12MB4133:EE_
x-ms-office365-filtering-correlation-id: d3dec301-32ec-46c8-cc32-08dc7653c353
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|376005|1800799015|366007|7416005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?ejd6WUZaL1FvNWEya2hrWHJaaFdnNCtnV1dvQjFpOW15NEQ5WEt1RFhYa3hl?=
 =?utf-8?B?cXZXNGZMSHlucG9FdFZONCsySEMzaktJYVA5SUZCVTlINjF1eDJpeXRZWEtB?=
 =?utf-8?B?Y2hzYisyUVBMR2ljVXlZZGdZeW4zVlVaK2FwMHI3NkZ3a0dYWG1hdFlMdHly?=
 =?utf-8?B?MWZWemc3Qk5TRWlBN3NzMkYrQ1hvS0VXdWRCNTFSazZxTmRqR2ZSaTFBTnZv?=
 =?utf-8?B?N2tLQkxwRlNkd2xnOFNDTE1yelRrQkduM2w3b3pLQUMzL2pjM2JmMGtrRVhC?=
 =?utf-8?B?Q2prYU5XeEhFT3pjTzJCdWlzd29GaFl4VElyeUd6WlhoRHU1VER5MEkwVUtU?=
 =?utf-8?B?MTdpM1VDNDJFejdRY1pBcHZHRStwWnAvM20xUTQyM2VsYjZ4bWtMelVhZlFX?=
 =?utf-8?B?bklGRHJlSGU5c1poRjJxUDMvcCsxVHVzU1VicXJrUk52ejVpMFpBWWlWZmk4?=
 =?utf-8?B?WTYwb09XQTcrcmxXMFhLVHN3aEJVRE5VREY3VTVDeDlheXNNRmEyT1FoaWF1?=
 =?utf-8?B?c2cvOVkrWVNXOFlpZjNnTnJUUFpvTW1HTU8vU092TGxadUoyNFFBMFg2UEIw?=
 =?utf-8?B?ZjBLMFZ0a3BleXl2NzNWMTJKK1dURkN6akRTazFuTlY2OW5Udzlrd2gwUnMw?=
 =?utf-8?B?Z2NuK3h0Y2F3cGRGQzRyZEtDUSs3QXphMDJCZHdBZTBETHpqK0dpdjl2ZGps?=
 =?utf-8?B?SzhXY3EvajV4ZkttTm9BSG90YlJJdUZ0Z2hHVWdPTmV0a3o2UTYzd1ZZUWIy?=
 =?utf-8?B?UWhEUDVDcHdSdlNNOUg1TzZBSzcwNnNIeTBkVStvQkZhR21xSnN6c1dhYnB4?=
 =?utf-8?B?dkF4cFg2aytndnFPTmNhaTJmeGJUTHFhRnNpQUxEcEdSbDdQcWJtVi91bXpP?=
 =?utf-8?B?OXMyVFFkZFdxa1Q1RmtxSGZaajI1SjIxTkpydnVubHV1ZDB3N0ZlNTA2dGZY?=
 =?utf-8?B?V0lRUVJzZHRpTEdjcHAzRk1kVWlwa0c1c3hFZXBLWURmYzF6b2RnVlRUWE9X?=
 =?utf-8?B?VWVvQjY1WUQva0VCMzBwSkVvcDA3KzJ2aFU1TTRFTXgwdkRZVWk5cmpscTh4?=
 =?utf-8?B?NTJGUWJqcUdUeGlXUFo3M3N0RFprMmlrV2hydnNiaUNqemJIQkZQcjNENFl1?=
 =?utf-8?B?OWN3UkF4QTJrVEdVYzQxMGxiOTRGcDlTMTdlR2N6TEpDLzNwUExBV2hUaldv?=
 =?utf-8?B?M2VLRmhTeG14cW4weWRxbGgxbk1aQ0VFNVF1dHlLWkM3NlkvL0pzakVXdWdw?=
 =?utf-8?B?OHZPNVVZT0hIZ01WbG5STGQrZVlPUFhHNGdpcCtHSXZwbUZReXlhSVpmWGdY?=
 =?utf-8?B?QXIvU3ZwSytwSVlyeWxMcGlHMGQ2eU1ObHIvUjMzN0dKRXpGNDVER3Rkalhl?=
 =?utf-8?B?eFlScjZncjBnQlhMMUJoZlozZFZVVHFUcldDRFBCbUE4d3c5Q2VjVktLOTc0?=
 =?utf-8?B?MGtadU9menlaQVV0UTJwWmw1aENxZkxna3kzVDA0dUFqbnZUNEdaN2MwTWlh?=
 =?utf-8?B?dmwzS0Z1SVhFK2hMR1RzeXJacDlvbkpxT2M2cnNIQllBcHNsU3Budjl4cEFX?=
 =?utf-8?B?NkhRWnMxN2VESUE4SVJGNUxJOC9hMzNpMGcreThBNDNjZEZSdWdwNCticUtU?=
 =?utf-8?B?blFvL1dNMWhNOXFGRVRhTWUzSGkzcHlhUGt4bnJQaFBjcGF4KzFJTCs3T3Bl?=
 =?utf-8?B?bndzVjJRWmRhZTZaVTZqSmJwdkJpL2VaNnE5a2Nlb1NCakJERGl0aVRRPT0=?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(7416005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?eVRSVDlQVERYTm0wankxblErdkNKOGozdkdxWnFUNDV0T2VncG9LdkdEazd1?=
 =?utf-8?B?dzl2MnhXc1NzNFpURzI4SEpWMnloSCtxZGRINTV5VVp3VEcybGducXprQ3Nx?=
 =?utf-8?B?OWFwTkVlSktJTXVMSDlveUJGaENlbGpJNWFIa2ZJQ0FOaS9pY2pNR1ptVHFX?=
 =?utf-8?B?bDJrTi9RaTlFWERvOFJrVmIvTXNobEdPRHFDbTllNXl2cE41TE1rQ2VqeDVS?=
 =?utf-8?B?U1RWbzdlaDVtNnJiRVlrQmhBams0MzBqanNDTmt4RVFSeFdrQlVIekRSNzdW?=
 =?utf-8?B?ZmM2TE1wY1JDdno0QmUwNUNvZHpoZHYwNkFvVFdsUmp5ZzgrMkpucGc1TSs4?=
 =?utf-8?B?RlkxNjBSTXhZaG5Oa3BUZnZMWnFWTW9tL0RDUXdjZkg5WVh5V2k3dmoxUmdU?=
 =?utf-8?B?SEhjWTZWcERCQm1zU3NlSCtNOEFDTWpPRmxlVTVER3FQcWVOZ2M2alZ0UzQ5?=
 =?utf-8?B?ejc5WlZrZE5BWVVEQ0RLWThvdGtrMVRjZUIzMVRBZDVXRlBRa1FBN3BLZ1lt?=
 =?utf-8?B?NDE4VE40OFNTMEVITm1yYTNKWTVFZ0hQWUhMQjlCR1NuVDNORm1naXh4M1Zt?=
 =?utf-8?B?WXNPRDd6QlNMZS8yRmcrdFRCeHU4V1NQaVI0bFpYWkRkRVB3bXh6eHBTSVl2?=
 =?utf-8?B?Y3NQWi9EME9nU09kVnVJMjUxbTZLSTlQU2hLbXk3Y0puaEZWTElhcGw3SXFj?=
 =?utf-8?B?dFdWY2d4dHNWL3JRWHQ0bVc4eU5KYStIL3A3bmU3WUJkNTFRcHZDUU9kR2ht?=
 =?utf-8?B?NlpWYmNFZHowQWhpVWpjOE5CWlUrZWt1RGd2TU1hWkZnUVUxNm9BQmRob1Ur?=
 =?utf-8?B?V01zVDVJeGlqTmdQNG1qSFB2T0pQSFBVTGxodTQ0cWVuYVVhS3NKM3E3cXNt?=
 =?utf-8?B?SEY4UlVXQ1lpTzJxQmJ5cllpa2F6VWxLMTU5WnRHZlh0L1E3VVpXOHFVOFRz?=
 =?utf-8?B?UFBWVXRIRnhNUkxTYnhMM3RWTjEwSmY0b00vSHppaE9WamtoTmZYS05jcHJK?=
 =?utf-8?B?QzV6cjJBQ0lvQmdVZ0owL01GOXg2MVowL0h1UmtDMENGNzZjRVVrMExJcEdl?=
 =?utf-8?B?dVM1ZUhOcmtHcUg0bmU2dDgzZW1MSG1CUUlacXlHMXF4WStVNXoreHZzbndW?=
 =?utf-8?B?Tm81NXk2V25iRlZSQ2N3dSs5NzBPSWJISTZJLzY1d3FvcDgvaHY3aFVxRWNQ?=
 =?utf-8?B?S0FzSjlONTFWVFVEUXk4RFR5TGkxUktxaHFJTlEyQVcxalY2WFdzd29YYVpa?=
 =?utf-8?B?dVROYmpWQ082Y2JRU3VLRVFmbklMQVhWYlFrQzY4OThmWVB5TzRDWk1lbmR1?=
 =?utf-8?B?dWFocGYzUXlxMHh0QWEzTVpQY3hjSDlQQVViamVUcHpvMVRUeHpUVGxycTJW?=
 =?utf-8?B?TUllWjU2NjJUODNYRFVEMEd6UUZzeVJlR2pOOHhmbWdaUExKOWczblNvcTQ2?=
 =?utf-8?B?SFcvTFJCTGhGd0pmVkwwUkN5V0ZqQTRsMVMzSU9ZT1grc0F1dU9mMThOR2NW?=
 =?utf-8?B?WWZsaWVwLzNHOVI5L0xrUUFhbjJpTWlwRktRNHJLcXh3UUNobFZGd1RIOVE2?=
 =?utf-8?B?ZW9ZLzVmOElVUkx3bktvaEx4VnNhOUlNMi8yWkJ5OG1OZWxNUGk5QW9zQVZM?=
 =?utf-8?B?dGhiZ1dlLzBOMXdhR2QrTE1FdzZGL1BjdlNsQjFtR3I4MkxwWVR6ZXI2Smlo?=
 =?utf-8?B?T2RISE5wNk9sODFpUDBOMUUxa1hmOWhvN3dQeldwVUpRK01GbkNFbE1KZVdv?=
 =?utf-8?B?NWZPY0Y2YmhtYUlzckhQVW96QW1qMStVRkg5aXB0OVMxV3V1TWhSOVNkMnpn?=
 =?utf-8?B?MUIzMU5zWVlBeU05bHJ2TTRxREhxSnpWUlplYlhrc3JJb3ZoekJibDQ5RGJn?=
 =?utf-8?B?YlVTVmVPOHhtZXhSV0NnbGpCOEU2S1Q3OUxFYW1SMlVWbUg4NlBQUWlvNnlE?=
 =?utf-8?B?SE1xdVlEdmEyRnp5djNmNWdGZEN5VFR4S0I5anZjeUo4ZW40V2hVejBubzcv?=
 =?utf-8?B?NGtFb015bFBlQVpVTTQvcFUxT3ppVTQ0T0Q2OFIvWk5wWmdWM2U4TlpCZ3JL?=
 =?utf-8?B?Z0t2NmxXMWEvam1DejVwWTdVUG1UaDMwbHpKOXU3V1FNNDNnZXgrT05BT0dS?=
 =?utf-8?Q?TX+E=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <780451B5A743C940808C50E7AD6E0BDF@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d3dec301-32ec-46c8-cc32-08dc7653c353
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 09:28:51.1072
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 5Js/3mxymdrcVweLPUoos8+nohpylDk/dNIchhElw8330lyalL7zLm4t6JkIkA7simWeUJt0WO5RNEGm9azjVA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4133

T24gMjAyNC81LzE3IDE2OjIwLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTcuMDUuMjAyNCAx
MDowOCwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzE2IDIxOjA4LCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAxNi4wNS4yMDI0IDExOjUyLCBKaXFpYW4gQ2hlbiB3cm90ZToN
Cj4+Pj4gIHN0cnVjdCBwaHlzZGV2X3BjaV9kZXZpY2Ugew0KPj4+PiAgICAgIC8qIElOICovDQo+
Pj4+ICAgICAgdWludDE2X3Qgc2VnOw0KPj4+DQo+Pj4gSXMgcmUtdXNpbmcgdGhpcyBzdHJ1Y3Qg
Zm9yIHRoaXMgbmV3IHN1Yi1vcCBzdWZmaWNpZW50PyBJT1cgYXJlIGFsbA0KPj4+IHBvc3NpYmxl
IHJlc2V0cyBlcXVhbCwgYW5kIGhlbmNlIGl0IGRvZXNuJ3QgbmVlZCBzcGVjaWZ5aW5nIHdoYXQg
a2luZCBvZg0KPj4+IHJlc2V0IHdhcyBkb25lPyBGb3IgZXhhbXBsZSwgb3RoZXIgdGhhbiBGTFIg
bW9zdCByZXNldCB2YXJpYW50cyByZXNldCBhbGwNCj4+PiBmdW5jdGlvbnMgaW4gb25lIGdvIGFp
dWkuIEltbyB0aGF0IHdvdWxkIGJldHRlciByZXF1aXJlIG9ubHkgYSBzaW5nbGUNCj4+PiBoeXBl
cmNhbGwsIGp1c3QgdG8gYXZvaWQgcG9zc2libGUgY29uZnVzaW9uLiBJdCBhbHNvIHJlYWRzIGFz
IGlmIEZMUiB3b3VsZA0KPj4+IG5vdCByZXNldCBhcyBtYW55IHJlZ2lzdGVycyBhcyBvdGhlciBy
ZXNldCB2YXJpYW50cyB3b3VsZC4NCj4+IElmIEkgdW5kZXJzdG9vZCBjb3JyZWN0bHkgdGhhdCB5
b3UgbWVhbiBpbiB0aGlzIGh5cGVyY2FsbCBpdCBuZWVkcyB0byBzdXBwb3J0IHJlc2V0dGluZyBi
b3RoIG9uZSBmdW5jdGlvbiBhbmQgYWxsIGZ1bmN0aW9ucyBvZiBhIHNsb3QoZGV2KT8NCj4+IEJ1
dCBpdCBjYW4gYmUgZG9uZSBmb3IgY2FsbGVyIHRvIHVzZSBhIGN5Y2xlIHRvIGNhbGwgdGhpcyBy
ZXNldCBoeXBlcmNhbGwgZm9yIGVhY2ggc2xvdCBmdW5jdGlvbi4NCj4gDQo+IEl0IGNvdWxkLCB5
ZXMsIGJ1dCBzaW5jZSAoYWl1aSkgdGhlcmUgbmVlZHMgdG8gYmUgYW4gaW5kaWNhdGlvbiBvZiB0
aGUNCj4ga2luZCBvZiByZXNldCBhbnl3YXksIHdlIGNhbiBhcyB3ZWxsIGF2b2lkIHJlbHlpbmcg
b24gdGhlIGNhbGxlciBkb2luZw0KPiBzbyAoYW5kIGF0IHRoZSBzYW1lIHRpbWUgc2ltcGxpZnkg
d2hhdCB0aGUgY2FsbGVyIG5lZWRzIHRvIGRvKS4NClNpbmNlIHRoZSBjb3JyZXNwb25kaW5nIGtl
cm5lbCBwYXRjaCBoYXMgYmVlbiBtZXJnZWQgaW50byBsaW51eF9uZXh0IGJyYW5jaA0KaHR0cHM6
Ly9naXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5lbC9naXQvbmV4dC9saW51eC1uZXh0
LmdpdC9jb21taXQvP2g9bmV4dC0yMDI0MDUxNSZpZD1iMjcyNzIyNTExZDVlOGFlNTgwZjAxODMw
Njg3YjhhNmIyNzE3ZjAxLA0KaWYgaXQncyBub3QgdmVyeSBtYW5kYXRvcnkgYW5kIG5lY2Vzc2Fy
eSwganVzdCBsZXQgdGhlIGNhbGxlciBoYW5kbGUgaXQgdGVtcG9yYXJpbHkuDQpPciBpdCBjYW4g
YWRkIGEgbmV3IGh5cGVyY2FsbCB0byByZXNldCBhbGwgZnVuY3Rpb25zIGluIG9uZSBnbyBpbiBm
dXR1cmUgcG90ZW50aWFsIHJlcXVpcmVtZW50LCBsaWtlIFBIWVNERVZPUF9wY2lfZGV2aWNlX3N0
YXRlX3Jlc2V0X2FsbF9mdW5jLg0KDQo+IA0KPiBKYW4NCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQpK
aXFpYW4gQ2hlbi4NCg==


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:50:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:50:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723967.1129132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uEF-00055t-1v; Fri, 17 May 2024 09:50:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723967.1129132; Fri, 17 May 2024 09:50:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uEE-00055m-VU; Fri, 17 May 2024 09:50:18 +0000
Received: by outflank-mailman (input) for mailman id 723967;
 Fri, 17 May 2024 09:50:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7uED-00055g-QK
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:50:17 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc9c1e96-1432-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 11:50:15 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso577774866b.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 02:50:15 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1789242csm1100467866b.66.2024.05.17.02.50.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 02:50:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc9c1e96-1432-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715939415; x=1716544215; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9oU72KYFlP0ia66lxCFd6gMiS9BxhVCy5XYqXbNsxaQ=;
        b=MRl+Q6CyoPBZnMsWGwgz8g7JGgXZcSj83h9CkF0HCYlrh9JEj0KCysz4vEsZiLlIxx
         f2yg13IKGphpQGc8c8E0omn3HCCIK8FzW6K7tobdicgTBB/UpWBPuCihQsxYpsGYiUMT
         C4xQvQQkuZnLGgjCKS1JIG33+IFt+RSHntMiES0QVqe2OtrGijcymbdV7TB0eTYBszDm
         KTIk7oBygbiYkYaFDdIm5lYLoypNuL0j33361xYl7Y5+frIDeOAabGx3x3zO4BD0IoV4
         WJXWn4DuIE+ODdCny/ipHXQUVAv6eETloU2ChVYzNv7rhZztu8AQjtPO8kayge0n+0yd
         EQMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715939415; x=1716544215;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9oU72KYFlP0ia66lxCFd6gMiS9BxhVCy5XYqXbNsxaQ=;
        b=unEj63c4XSoCG4Xq81B/IrccJ866jBTU5Tk90mzIF+FVGd+pJlRbRbe7s/a8vrMYII
         xWSPC3A9EykxC1XtRSTUYK2JxyZjl9IP6iWZOshKiJ/B36WTJ9VYgAhFebK5dUbSZPww
         qHRwbH2CAYrgTAc4o0IMCNtU5yHZ/z5qJ0PVmoUDcdJhQre4TyrJD/amw86ShNPi/jia
         FwtLqL7hrmlR7y+ZxHuiqTUet9mgzmaZPVjrl+uAZKND/jhG8mgIAiRU6slJhp3WiUbL
         JRX5utNKicDDZjlcCSXOBYVoCQhvEQRSe0q4AZJIyFAHji9y5wq/1mBDm2LCIi8Whiqs
         6RkQ==
X-Forwarded-Encrypted: i=1; AJvYcCW2OeOT6kS17gKD6ConQQkmxyJMwg1S0BslNHIa5nGQO3ki2DkfPj8DlK3XVfP4XL3t6zknqzP4lSTX6OVFhpW6X3tGo4Zu0yyG6gMTspk=
X-Gm-Message-State: AOJu0YxbO9QvKEKRPiRZkroQ2NidsSoeeuCjUHQMxxSzVmKZIwDh6/k1
	W29fXdD4oWSgXGCNqWWOLjDGQeXwqkLa6rwR2uBqwO5i2OQHqMWI68ScAmoFZg==
X-Google-Smtp-Source: AGHT+IH0lMkyWRlMcw2xg+WhPBOig0azlmgGfNZNdA5+hh3yf7dYnRNyGITnSAWx9OgcQxJacslFHg==
X-Received: by 2002:a17:906:3558:b0:a59:c345:17fb with SMTP id a640c23a62f3a-a5a11557164mr2111160166b.5.1715939415154;
        Fri, 17 May 2024 02:50:15 -0700 (PDT)
Message-ID: <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
Date: Fri, 17 May 2024 11:50:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Content-Language: en-US
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 11:28, Chen, Jiqian wrote:
> On 2024/5/17 16:20, Jan Beulich wrote:
>> On 17.05.2024 10:08, Chen, Jiqian wrote:
>>> On 2024/5/16 21:08, Jan Beulich wrote:
>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>  struct physdev_pci_device {
>>>>>      /* IN */
>>>>>      uint16_t seg;
>>>>
>>>> Is re-using this struct for this new sub-op sufficient? IOW are all
>>>> possible resets equal, and hence it doesn't need specifying what kind of
>>>> reset was done? For example, other than FLR most reset variants reset all
>>>> functions in one go aiui. Imo that would better require only a single
>>>> hypercall, just to avoid possible confusion. It also reads as if FLR would
>>>> not reset as many registers as other reset variants would.
>>> If I understood correctly that you mean in this hypercall it needs to support resetting both one function and all functions of a slot(dev)?
>>> But it can be done for caller to use a cycle to call this reset hypercall for each slot function.
>>
>> It could, yes, but since (aiui) there needs to be an indication of the
>> kind of reset anyway, we can as well avoid relying on the caller doing
>> so (and at the same time simplify what the caller needs to do).
> Since the corresponding kernel patch has been merged into linux_next branch
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240515&id=b272722511d5e8ae580f01830687b8a6b2717f01,
> if it's not very mandatory and necessary, just let the caller handle it temporarily.

As also mentioned for the other patch having a corresponding kernel one:
The kernel patch would imo better not be merged until the new sub-op is
actually finalized.

> Or it can add a new hypercall to reset all functions in one go in future potential requirement, like PHYSDEVOP_pci_device_state_reset_all_func.

I disagree. We shouldn't introduce incomplete sub-ops. At the very least,
if you want to stick to the present form, I'd expect you to supply reasons
why distinguishing different reset forms is not necessary (now or later).

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 09:52:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 09:52:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723971.1129142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uGf-0005kd-HH; Fri, 17 May 2024 09:52:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723971.1129142; Fri, 17 May 2024 09:52:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uGf-0005kW-Ee; Fri, 17 May 2024 09:52:49 +0000
Received: by outflank-mailman (input) for mailman id 723971;
 Fri, 17 May 2024 09:52:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7uGe-0005kQ-DO
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 09:52:48 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 367343c5-1433-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 11:52:46 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-5206a5854adso508749e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 02:52:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d6dsm1104664666b.71.2024.05.17.02.52.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 02:52:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 367343c5-1433-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715939566; x=1716544366; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TNSUXRoBwA2VsVvHORhtphWHohvt1BGxA6D18hjFb/Y=;
        b=cfMUgasuKxuqJQ7sKHUWy5iEKjVtG04RMZeW59AFA4citkBuMV/lOjLdlZyyOx5Oi+
         5GTK0SsrqUuWb99kJiiZGEXTxEM3SUMbl0KGapyLTz37SI80VekPqt4vJ7ih/TSBiw+e
         mxecCzOuvwISb0Z+w8tPxo9klg7xX+Jhf1qhDII3Y1jKuCbVXAcrj7DnbNz+fPigNHYm
         Svhv4RXLA9Ym5nV5LSi2P+VNeAysiYKYOJ3bZz8eRA82EJpmwfbBBX3fudaCM1V0TNCS
         ubJWLgPn+TTON+/j7qhOTmqEgJ3TeblqNkSJFGW+MSNYpTP+68KjkGPTDoX8GGIhK7JE
         R62Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715939566; x=1716544366;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TNSUXRoBwA2VsVvHORhtphWHohvt1BGxA6D18hjFb/Y=;
        b=S8ZMZg9zHJEq/Cwbkl/uGGCHcIYf0PSlj0iaBKrMu/pLWxoJcCRhtGtq+zKBAaCxNv
         qIb2sX/Y0IWzlEPgxiZvndbokcittO/DFg+/pZDVwugKkkEwQZ8pR4NoeirDxjSa0B/o
         o1ifnXLHFXNIdojnrQ4GJiyQabu0nSGmA01r508+VqtBLSyRz3KfhiA0Ld/XQLXlBgAd
         fSikz0dUyzO/ljw1zqYwASA2baD1kRxC689j//Q/aRDxhgZWc6Hna7vtmClme42CloAj
         l8C8z8GAj0HuHidbMoqKW40STbx81lUjY7mkq4ckqXC2vFKCi30BCq5pMTSz2ErbtMIf
         Mltw==
X-Forwarded-Encrypted: i=1; AJvYcCWnnkjReB2jxy8NDERFVL9+UQq4MzvjO8qDfuwdCFdxE0tEMDLeRtVn1ONkK92vQUesNxcNkouM/PooySuUUdaBTDr0waTzWvY308ganIo=
X-Gm-Message-State: AOJu0YxAHroo9+MZrfHoEwGULHzmgF6NDvFtVQrS5wDHQscBkwojmDmI
	gqv4lQ8wFS2RX7LFumHIsSSDxdcFuwTjG+zkbci2bI5wLILIDBzxwQwiGQButA==
X-Google-Smtp-Source: AGHT+IFKVJp8MbSOrgOCZ0oXEnRYT83kiqZNJb7m5dhMU9zhcaeXhLj9Zg0iMVGMEeM+QsksJXsZiw==
X-Received: by 2002:a19:f503:0:b0:51f:3fea:cbcf with SMTP id 2adb3069b0e04-5221016e864mr11831054e87.52.1715939565899;
        Fri, 17 May 2024 02:52:45 -0700 (PDT)
Message-ID: <d35ee29a-f6cf-4e3b-86f0-d77a84c63a3a@suse.com>
Date: Fri, 17 May 2024 11:52:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 03/15] xen/bitops: implement fls{l}() in common logic
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1714988096.git.oleksii.kurochko@gmail.com>
 <fe183f7e8ada7c3fb00ebf9b38f1fffffcc9c2d7.1714988096.git.oleksii.kurochko@gmail.com>
 <c16c0554-c376-41f4-bfe8-04c977702a04@suse.com>
 <9cee6531c9aa6650be803d004b06cc88a25df7f5.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9cee6531c9aa6650be803d004b06cc88a25df7f5.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 17.05.2024 11:06, Oleksii K. wrote:
> On Wed, 2024-05-15 at 11:09 +0200, Jan Beulich wrote:
>> But this then needs carrying through to ...
>>
>>> --- a/xen/arch/arm/include/asm/arm64/bitops.h
>>> +++ b/xen/arch/arm/include/asm/arm64/bitops.h
>>> @@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long
>>> __ffs(unsigned long word)
>>>    */
>>>   #define ffz(x)  __ffs(~(x))
>>>   
>>> -static inline int flsl(unsigned long x)
>>> +static inline int arch_flsl(unsigned long x)
>>
>> ... e.g. here. You don't want to introduce signed/unsigned
>> mismatches.
> Is it critical for x86 to return int for flsl() and fls() or I can
> update the return type for x86 too?

The return types ought to be changed imo, for everything to end up
consistent.

>    static always_inline int arch_flsl(unsigned long x)
>    {
>        long r;
>    
>        asm ( "bsr %1,%0\n\t"
>              "jnz 1f\n\t"
>              "mov $-1,%0\n"
>              "1:" : "=r" (r) : "rm" (x));
>        return (int)r+1;
>    }
>    #define arch_flsl arch_flsl
>    
>    static always_inline int arch_fls(unsigned int x)
>    {
>        int r;
>    
>        asm ( "bsr %1,%0\n\t"
>              "jnz 1f\n\t"
>              "mov $-1,%0\n"
>              "1:" : "=r" (r) : "rm" (x));
>        return r + 1;
>    }
>    #define arch_fls arch_fls
> 
> Any specific reason why 'long' and 'int' types for r are used?

I don't think so. I expect it merely fits with no-one having cared about
signedness back at the time.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:00:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:00:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723982.1129153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uOC-0008DC-8y; Fri, 17 May 2024 10:00:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723982.1129153; Fri, 17 May 2024 10:00:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uOC-0008D5-5X; Fri, 17 May 2024 10:00:36 +0000
Received: by outflank-mailman (input) for mailman id 723982;
 Fri, 17 May 2024 10:00:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7uOA-00084w-Ck
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:00:34 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4bd0db11-1434-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 12:00:33 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CH3PR12MB9121.namprd12.prod.outlook.com (2603:10b6:610:1a1::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Fri, 17 May
 2024 10:00:28 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 10:00:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bd0db11-1434-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VUOHT00a6iT92iBJBcfc4/nfl93AQLm7vMn9Eg385m74fV9Xu+pypv9vi9Xxd//1MrgTQhiVW5cN+7wJM1Mt6Jfn1Gx0SKDkUtNpxDa4ujQThrItOPUHcNuZZgjBBtT+cGcEBxMCiF6NbKO0Q+FkjvDqjJ1kx2NhJEevlHy09lHbYb+omlaTmf8izewNbtKrRNyTrZWAlZX0T3ySM8kEZSawgvNSK1v78O0ZnzAGwU6JpVVJO6NKWW42gdXCLqfZv8Fc0lk6GupeNEBnyhZ8H3RqXaDqUHSi9/kXwKs01KntVZKIr42Nsoy3mJCGlKUw/iS3IcYry+mcVCDq4OnVPA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qL8Zbd0+bVxFxeNkqFUC4+wUMs3F7xlhfO3YaNDWndA=;
 b=nSiF4wC8SpCwGdoPJe/BfrbGcshyUI6lSc4fepiJyuvzfcFti0qosPbMPJS5OEkXymutyTXMeIxbvzBgwDUhkXNy60qcZpC/KufLCLtCDI7cDlwQztgDB7S7sROJnvCsQ1nFVZ8D0TGxLQqi1A1AVW98xKMHTJ7uptMjGfF7Y1SPCsMAu73wh0Ky5Bq7oTw115rQ+chRnC8gaLsA/1ZjebMEF8uCFIOF1JIG2NUs9teFBnwCwW5XFMWWTEVlF+KNXM+WIg1tJSQRD6TmC/7bMg9e/vkKUaS1pGUFBtMLns9/HxMTpPDSYig2o16Emipm0z4sIhxeee+1HBxI9CVEiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qL8Zbd0+bVxFxeNkqFUC4+wUMs3F7xlhfO3YaNDWndA=;
 b=u8Pcj5jXB3pi3T8MA2p7XcsxNHumzQ/LIBUWZjOEjiSwDg8jmn5dEc7V1kJg02tf2bSCu2h/HK0Ix0LTTNsXiGpBsbBlN2vw4OT0ZtJcKviQLWlRMiW8ycmGZ3/mUXj9qeYRjAMBQVrKo4R3vUIpdn5qE1y4QNupBbohD+sSE5A=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Index:
 AQHap3bYcGtlWtrcpEyRLTK5fjq5SLGZ1UMAgAFwSAD//9GPgIAAlWQA//+DzoCAAIa3gA==
Date: Fri, 17 May 2024 10:00:28 +0000
Message-ID:
 <BL1PR12MB5849CCF9A890A7DA969F6474E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
In-Reply-To: <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CH3PR12MB9121:EE_
x-ms-office365-filtering-correlation-id: 7f2f6352-44cc-4e1a-954d-08dc76582e67
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|376005|1800799015|7416005|366007|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?VEptTEZzYkRydUg0TDJTZjQ4TGUxU2czeTJKbkJ3aTlLU1IvSjg4ajdQYVFx?=
 =?utf-8?B?UnlDOUdMM2lMdUNkR2E0VW1kd2tjQU0vVHZEWHBTYnRyOGhVOEZ0b3pJOXJW?=
 =?utf-8?B?WGJOYnNxUXZpNTA2WXowT0hjeFgramVZdHFwYU4xR0tpNVN1VU1MT25nSUda?=
 =?utf-8?B?Z2RHemRrM1VjOHdLaE8rb3VHZnV2ei9xaGl0MDJCZW5PSDUxSFZhSG9hMzF5?=
 =?utf-8?B?b25EZ1dYa0ZuOGZIajBvQTM3OENJSUtZRFBlZ24vK2xFaTFMRHpxeVRXRlk5?=
 =?utf-8?B?bklRdkxZcUNpdFF2SEpLOS9FMTRJN095Ulo1ZkNPS2w0cGRUSldIeHZLSWVY?=
 =?utf-8?B?TW1yTmJweGphbFljcE5qZURiNUd3RGJTRGx0RXFWMFZ4SDd0aUFwYnNXbDJO?=
 =?utf-8?B?YU4zQTBnaDhNOHZQWUZ0U2JVc3IvdWN1Z3UzZUhqMGs4YXVXU2Jtek9pbWJL?=
 =?utf-8?B?bzhJUTRnaWpIanp2N203ZkZTTXZ4VDdwRUdodkZaRkE5N21DOThyUFZuVkZ3?=
 =?utf-8?B?aFpCbitjZldGakFoYUlvL3VwN3lHTk53ejlkYmRXNU9Sa3VIckdzSWc1T3J4?=
 =?utf-8?B?allzZ05odEV5bEprMHYxZC9RTThtSlFUTjlRSHVrc2hUM0RlSDRjOFk4eFZE?=
 =?utf-8?B?RnZ1Tk1XdjU1YnJwbHJYNXVRQVNXd2dtbHFDMnlpV2pLNmttbjVMb1BaclVv?=
 =?utf-8?B?cUtUNU4xQ1QyOHdKN3lnUzJXVXJNajdUSkwrMzdXcmNrQ1JKN0ZVQnZGeXRN?=
 =?utf-8?B?Qy9LWjJsNS9wS0dybi81akRXRzdqcDV5N0cwVkFBa2dJT2JIbG1RV3pINjh2?=
 =?utf-8?B?RUU4Q3hvQmwxQkxwNkNUMkFEZ2dUa3UxbzVxb0FRdUw3L3Mzc1JFUWxURmR0?=
 =?utf-8?B?S2c5N1N0cFJVcm4yK2oxa3VlTkJtUEVET2lRN1dnRVhIOHpFNkludzFUajcv?=
 =?utf-8?B?NU81aGVqcFNybVJlZXFINTVXbUpiZ1BaSWVOanA5YkVsMzVVZFBYei9zUmJm?=
 =?utf-8?B?Nm1wRkNLU0x1QWY3TnpCS0Yyek5yL29veHRET1BoQWlyN0gzV2dnR1hXUkJT?=
 =?utf-8?B?cHFYRU12YXFUUXpzeWlZRWZLWlNsN0lTREEra2N0cGtOQTlMYTlEaXZ3YTlX?=
 =?utf-8?B?U1puNXBPUXBweVR3RWpLb2owKzRJYzVOUkY4ZCthSlp0TnhDcFdzNXZXazFS?=
 =?utf-8?B?aU1jQTY5ejNtVXdKd2dRKzdsYnMvakF4OGZrZnN4Qy93aVQ0TUZwTlhvU2sx?=
 =?utf-8?B?YkFPNkVLWWZYZFNVQVZxM1pzUVp1bnJVVHMwdHJaT0hCZ1RYSHRPYVlnTTJx?=
 =?utf-8?B?eTlpWmlud21UUHFGUUVYZml3ei8vekdFOW1aQUoxOHl0cVdzSHlkeXpnOUZS?=
 =?utf-8?B?eXFDWlhYb1dhZ0pOaEdTeG5kaU5kLzl5TUI4M09ncGdlK2MreklxK015ZnBD?=
 =?utf-8?B?OHI2SWRMUHhqZGlBdGNvK0ovVHVZM0p4VE9aa21lTnBTM0MzM1JZS2dpOFNL?=
 =?utf-8?B?TE5Xd1ByQllYazBySnBzeS9aQXN1MUo2Sy9WUVJYSE4yRkFJbmNTSWFjOWt1?=
 =?utf-8?B?UXlDL2YrTWUxclJ2ZDE0UXJoSDh4cXhCVXpUdzhHWFpGdG9rRlZJd013Yndw?=
 =?utf-8?B?Z2hUMTRhT3NGQ1R6aFdyaEgvaEZqaW45V0NYYUpaT01WVGk3MGdVMFFIUXBh?=
 =?utf-8?B?SVpzbi8vTXQvNGd3amZjOExDMTdhbk1ERjJKN1htK2FrNmxhekxSSjNnPT0=?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(7416005)(366007)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?emVUMXZSdk5wazFUZ2NUMzVjaE9MN09EeFZ2aDQ3aDRTMmpiSUx6aVRmczkz?=
 =?utf-8?B?T29uS2xTSlNwVXRaWlg5V0dZRU1UWm9xY25pcmt5NjYwMDlwcm9Cb3prVDMz?=
 =?utf-8?B?QzljemloYld3bVRid2pTbTY2QkdUSUZ2SmpqWWduMGxVbmRzNUxUU2RYbjdQ?=
 =?utf-8?B?SVJzYlptTkZBdXR3Yk5aalc0N3dNYzJ4UW5aWmRGWVZVUzQrSmFQTnFEWUdy?=
 =?utf-8?B?RFM4dHowTDFLU2JUTEpabVB5dmc2MEtpdFRIVXl1eTdMVHNSQXNJM3F0Q0Q3?=
 =?utf-8?B?ZTNJZmhPUWRZVW9aTEtDODk1ZVBKb2RGUFlTNk5oTGtpaWVocEJBZTkxK1Fj?=
 =?utf-8?B?MzRyeWlBRkRJQ2tKTHBSbnZXV3AwR09RaVVhMXlJSkZtSEtoRlpGREFyYklQ?=
 =?utf-8?B?Wit1U0pjYjBwZ1ozcUVSeThsZEJRaGpVbm9JcmpYMG40MGwvcEJXb0UyS0Jl?=
 =?utf-8?B?YTNtVkx2MVV0UkI4S2EvME9za2ZkQjZZUXU3ZkRRM0M0NDVpeVdkUHhaRURu?=
 =?utf-8?B?WXY2TWI4OU4vSzNSbUIyZ2dCNDkrZ0FkVCtnSmhobzFIQXVIZW9lbXZ3bkcw?=
 =?utf-8?B?dUNtRkdIQ0YrRzBIMlZndmtTU3RVQzZFWm41RkZ6cFBTTktkeFJ5cG93bnVx?=
 =?utf-8?B?KzN2SHVyei9lb2xKODUvanpzY0RjNkRsdVN6ZTVVbHZQN0VFcElMU0thMjlJ?=
 =?utf-8?B?RC9pQ1BTNVlTOW5NaDEyV3o1U0xpV0VLdnpnMktLcmo0RHpEMEJBRjJuWWZh?=
 =?utf-8?B?b1ZhZDRMaDM4c1pHWVE5aE5VSkJxMnhhR29tU2x5YzdFUWVnaU00ajk2NlFn?=
 =?utf-8?B?akJoN0NuNFRjMnpRZVVWMXZxTzRqWVlLUXJTZGJ1bndBUkhGVWJKQnB6RG04?=
 =?utf-8?B?d2dOQTc2LzRHUVlCY0pmS3ZxaU9aRnd6TmtrWG1CZ3VBejhkcUQ3bXBNa1pq?=
 =?utf-8?B?em1KeEJSaC92UjNrVTFlTS9xVitJQURoOVd6bXJyby9lUWQ4WFBCOEtIaUNP?=
 =?utf-8?B?am1sRlJjM3VVNkY1WWEwVHZ0cmZob3p0NThqL3p0ZWxtNExBcGJ2OUN0TCtS?=
 =?utf-8?B?RXBaWkU3S3VLQmFoZm5NblpnSzJlNGxjcmVWYU9JNGVvL1dna1YyeHV3UEdC?=
 =?utf-8?B?UjRJcDhnb3RuOFJTcG5LeUgvc25BMmc0YmQrUWJLczRFWkgxenhlaHlKTGNn?=
 =?utf-8?B?NU1obzVZU1h1Vzg5UTdoa1R5NldXaUl5ajYvZUk4M0lmRmFIcHNmK0d1QXRL?=
 =?utf-8?B?ZnBoR09QYWVNN25iRHlTd2R1TVRBQzh2enRzMVRVWmsyRHdMSUdaN3RuZGhZ?=
 =?utf-8?B?VkxkSUkvWDhPOHFCUXc2TmUxVndQM1k0ZjU0UmNodWNXWjVmZnIrZXlTaE9O?=
 =?utf-8?B?ZXFoMU1oWjJ0b2pFVVZYN2pDMVowV3UxZTFmZ3lPTEdQdG9CM3M5Rmx5NDhZ?=
 =?utf-8?B?VEJQbE4xaTNuZHNpa3d3bWZMeU05M3RseEptTmJKZFZjdXJCQ00rdC9hWFdV?=
 =?utf-8?B?N2hobWF4YlAxKzRoZEZaNzRyUmJIR1huZ0F0S3U5TUxrbGdUSHRjRmRTdHFu?=
 =?utf-8?B?SG1tUll1anVzYmlWdTZmb1QxeWszU3k1MGxYTmQxdzN2U1N1V1ZINzhiR0xn?=
 =?utf-8?B?U3dPMTJweHBiL1JkSmZUN1lIZXR4aThyZjZiNXhubmlabFlscXRYeUJ3emtC?=
 =?utf-8?B?MUttNWpFckZBTEZ1cmNJRmh6K2tzYmY2T0R2TFBYK0xuMFhVeXk0N014Vysw?=
 =?utf-8?B?U3JkV0R3cm9KQnI5WjVwVXBBTFNqdkJCbGJycFY3ekQ4L2wwNVRRTlFyUnVJ?=
 =?utf-8?B?cit4VUZNMUVsTi8xOGJURXlyOFUzalpZVDFEeWhsZ3VaREM0NzFuUTQyN3NI?=
 =?utf-8?B?TDFNYjE5QXoraXgrMDMvUWUzVFhZTG5QVThPWE1kclErVEUwbGxyVjVqU24v?=
 =?utf-8?B?TjcrUVAyUE5ycVZCWUQ4NUp3bS9pZTdVMVF2eDkxbXJmM0NFM1NiVEpPcDQ1?=
 =?utf-8?B?QjlnQkVRc1NQVnp0T2UwMXFUaFRLNG1PbFcxaFduVnI2aDBzay92L0s0eFhm?=
 =?utf-8?B?amk2cUhSUmt5YWozb1l5aENwazBXMFlUU2dKZGhIcWZEMnN0bHJQcEN6SE1Y?=
 =?utf-8?Q?QuhI=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <152754977115A44F88D04467606AADFE@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f2f6352-44cc-4e1a-954d-08dc76582e67
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 10:00:28.7084
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /8C7AnmekO31SqYgyh2UXqvS7Oh7TLdUYBB3f5Tl2mHt+J3qEBH/q3xVb0HzkfKGc3mSnenB1vPJIwjMm29bUQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9121

T24gMjAyNC81LzE3IDE3OjUwLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTcuMDUuMjAyNCAx
MToyOCwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzE3IDE2OjIwLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAxNy4wNS4yMDI0IDEwOjA4LCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IE9uIDIwMjQvNS8xNiAyMTowOCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+PiBPbiAx
Ni4wNS4yMDI0IDExOjUyLCBKaXFpYW4gQ2hlbiB3cm90ZToNCj4+Pj4+PiAgc3RydWN0IHBoeXNk
ZXZfcGNpX2RldmljZSB7DQo+Pj4+Pj4gICAgICAvKiBJTiAqLw0KPj4+Pj4+ICAgICAgdWludDE2
X3Qgc2VnOw0KPj4+Pj4NCj4+Pj4+IElzIHJlLXVzaW5nIHRoaXMgc3RydWN0IGZvciB0aGlzIG5l
dyBzdWItb3Agc3VmZmljaWVudD8gSU9XIGFyZSBhbGwNCj4+Pj4+IHBvc3NpYmxlIHJlc2V0cyBl
cXVhbCwgYW5kIGhlbmNlIGl0IGRvZXNuJ3QgbmVlZCBzcGVjaWZ5aW5nIHdoYXQga2luZCBvZg0K
Pj4+Pj4gcmVzZXQgd2FzIGRvbmU/IEZvciBleGFtcGxlLCBvdGhlciB0aGFuIEZMUiBtb3N0IHJl
c2V0IHZhcmlhbnRzIHJlc2V0IGFsbA0KPj4+Pj4gZnVuY3Rpb25zIGluIG9uZSBnbyBhaXVpLiBJ
bW8gdGhhdCB3b3VsZCBiZXR0ZXIgcmVxdWlyZSBvbmx5IGEgc2luZ2xlDQo+Pj4+PiBoeXBlcmNh
bGwsIGp1c3QgdG8gYXZvaWQgcG9zc2libGUgY29uZnVzaW9uLiBJdCBhbHNvIHJlYWRzIGFzIGlm
IEZMUiB3b3VsZA0KPj4+Pj4gbm90IHJlc2V0IGFzIG1hbnkgcmVnaXN0ZXJzIGFzIG90aGVyIHJl
c2V0IHZhcmlhbnRzIHdvdWxkLg0KPj4+PiBJZiBJIHVuZGVyc3Rvb2QgY29ycmVjdGx5IHRoYXQg
eW91IG1lYW4gaW4gdGhpcyBoeXBlcmNhbGwgaXQgbmVlZHMgdG8gc3VwcG9ydCByZXNldHRpbmcg
Ym90aCBvbmUgZnVuY3Rpb24gYW5kIGFsbCBmdW5jdGlvbnMgb2YgYSBzbG90KGRldik/DQo+Pj4+
IEJ1dCBpdCBjYW4gYmUgZG9uZSBmb3IgY2FsbGVyIHRvIHVzZSBhIGN5Y2xlIHRvIGNhbGwgdGhp
cyByZXNldCBoeXBlcmNhbGwgZm9yIGVhY2ggc2xvdCBmdW5jdGlvbi4NCj4+Pg0KPj4+IEl0IGNv
dWxkLCB5ZXMsIGJ1dCBzaW5jZSAoYWl1aSkgdGhlcmUgbmVlZHMgdG8gYmUgYW4gaW5kaWNhdGlv
biBvZiB0aGUNCj4+PiBraW5kIG9mIHJlc2V0IGFueXdheSwgd2UgY2FuIGFzIHdlbGwgYXZvaWQg
cmVseWluZyBvbiB0aGUgY2FsbGVyIGRvaW5nDQo+Pj4gc28gKGFuZCBhdCB0aGUgc2FtZSB0aW1l
IHNpbXBsaWZ5IHdoYXQgdGhlIGNhbGxlciBuZWVkcyB0byBkbykuDQo+PiBTaW5jZSB0aGUgY29y
cmVzcG9uZGluZyBrZXJuZWwgcGF0Y2ggaGFzIGJlZW4gbWVyZ2VkIGludG8gbGludXhfbmV4dCBi
cmFuY2gNCj4+IGh0dHBzOi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0
L25leHQvbGludXgtbmV4dC5naXQvY29tbWl0Lz9oPW5leHQtMjAyNDA1MTUmaWQ9YjI3MjcyMjUx
MWQ1ZThhZTU4MGYwMTgzMDY4N2I4YTZiMjcxN2YwMSwNCj4+IGlmIGl0J3Mgbm90IHZlcnkgbWFu
ZGF0b3J5IGFuZCBuZWNlc3NhcnksIGp1c3QgbGV0IHRoZSBjYWxsZXIgaGFuZGxlIGl0IHRlbXBv
cmFyaWx5Lg0KPiANCj4gQXMgYWxzbyBtZW50aW9uZWQgZm9yIHRoZSBvdGhlciBwYXRjaCBoYXZp
bmcgYSBjb3JyZXNwb25kaW5nIGtlcm5lbCBvbmU6DQo+IFRoZSBrZXJuZWwgcGF0Y2ggd291bGQg
aW1vIGJldHRlciBub3QgYmUgbWVyZ2VkIHVudGlsIHRoZSBuZXcgc3ViLW9wIGlzDQo+IGFjdHVh
bGx5IGZpbmFsaXplZC4NCk9LLCB3aGF0IHNob3VsZCBJIGRvIG5leHQgc3RlcD8NClVwc3RyZWFt
IGEgcGF0Y2ggdG8gcmV2ZXJ0IHRoZSBtZXJnZWQgcGF0Y2ggb24ga2VybmVsIHNpZGU/DQoNCj4g
DQo+PiBPciBpdCBjYW4gYWRkIGEgbmV3IGh5cGVyY2FsbCB0byByZXNldCBhbGwgZnVuY3Rpb25z
IGluIG9uZSBnbyBpbiBmdXR1cmUgcG90ZW50aWFsIHJlcXVpcmVtZW50LCBsaWtlIFBIWVNERVZP
UF9wY2lfZGV2aWNlX3N0YXRlX3Jlc2V0X2FsbF9mdW5jLg0KPiANCj4gSSBkaXNhZ3JlZS4gV2Ug
c2hvdWxkbid0IGludHJvZHVjZSBpbmNvbXBsZXRlIHN1Yi1vcHMuIEF0IHRoZSB2ZXJ5IGxlYXN0
LA0KPiBpZiB5b3Ugd2FudCB0byBzdGljayB0byB0aGUgcHJlc2VudCBmb3JtLCBJJ2QgZXhwZWN0
IHlvdSB0byBzdXBwbHkgcmVhc29ucw0KPiB3aHkgZGlzdGluZ3Vpc2hpbmcgZGlmZmVyZW50IHJl
c2V0IGZvcm1zIGlzIG5vdCBuZWNlc3NhcnkgKG5vdyBvciBsYXRlcikuDQpPSywgaWYgd2FudCB0
byBkaXN0aW5ndWlzaCBkaWZmZXJlbnQgcmVzZXQsIGlzIGl0IGFjY2VwdGFibGUgdG8gYWRkIGEg
cGFyYW1ldGVyLCBsaWtlICJ1OCBmbGFnIiwgYW5kIHJlc2V0IGV2ZXJ5IGZ1bmN0aW9uIGlmIGNv
cnJlc3BvbmRpbmcgYml0IGlzIDE/DQoNCj4gDQo+IEphbg0KDQotLSANCkJlc3QgcmVnYXJkcywN
CkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:03:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.723988.1129162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uRK-0000M1-Mp; Fri, 17 May 2024 10:03:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 723988.1129162; Fri, 17 May 2024 10:03:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uRK-0000Lu-KM; Fri, 17 May 2024 10:03:50 +0000
Received: by outflank-mailman (input) for mailman id 723988;
 Fri, 17 May 2024 10:03:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=g7K8=MU=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1s7uRJ-0000Lo-IL
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:03:49 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c03a100f-1434-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 12:03:46 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a599c55055dso419039166b.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 03:03:46 -0700 (PDT)
Received: from ?IPV6:2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c?
 (p200300e5873ca5006aafb7a77c29ae5c.dip0.t-ipconnect.de.
 [2003:e5:873c:a500:6aaf:b7a7:7c29:ae5c])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894dc3sm1099123766b.86.2024.05.17.03.03.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 03:03:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c03a100f-1434-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715940226; x=1716545026; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RCNMD6qSYt+oq7gnC2MrSoLrVZzbr9VCri2gbO7qRY8=;
        b=SL3cGw88oNIn0FehSKVq9Yc2A3i8wJVixJD1asx5QzuEGP69fT0WJwcjq7HdyXqLc3
         fSNTE+AJWF0RiHRwAftwXfel4KnbFBW3AsF3qTbk/XzGmG+wvGcz1xK3eH50NPWi/snP
         b9+SXHxz9i0hTG8+Kp6QEHQ8MYy6grvVc+BXcEGMdWsUWVrspyDeGCIDxJmlTiA3rPpG
         K8zR3WPShFI5bwK6oHV4QVn8BH39jMPLgprBjloAVXvKKhUmAp8ptLGA+0LMxQ4Y87hZ
         L1BPBl3T2MB6QHydKYNlD0kOs/1ww8YgZyeOZwGZFHzQvNUeZzwOTv85UFVQ2VhZmu34
         e+Kw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715940226; x=1716545026;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RCNMD6qSYt+oq7gnC2MrSoLrVZzbr9VCri2gbO7qRY8=;
        b=WATlWc9bUirVvYZvOvtHRApkq1EaR/GEKqTTDg0H2j2xVjCFCG3MAxxbo6trfJxrrj
         jnUNMftzDX3r4whhhH/jco6g5aKnbw+AtCzUi58P2hikSzWGohW9HwBxFaEVYL+Attbr
         CMTtNG9lYTLFWunDCvxSbCDaWqmvnzBj8+IQoQ+hO5O96QhRFswKoVWMIQfQgwT0vFHh
         Tp8RNHb1BfAHlHW8hm4V6G00nQtr+91Ei2fG5vBOxu4YF+QFybmdqX6kA4aXmvLxAo9o
         nnOnwCEUhemp5MdkFvBEVpoUuK8GnFBZWdHj4bqTwxYJN3UB1DFTMGcllmOqgMXbYWl1
         wJMA==
X-Forwarded-Encrypted: i=1; AJvYcCWBqlv066ej3JwdVhltxi+QI7TKrGS7ckpQvBjNV36bIA5yFNj6GJySOGTccV1OU6kUedw9UTRPM84EmHmr5of5Xc/MFpV0Pqm3DR8yUHw=
X-Gm-Message-State: AOJu0YwIHrsNdBHkhq5vjqmiXDQz9WK7tJehnDblaf8JXS4Aa6PfrSUi
	SmEcKAiZoQ4D3SgJ7l5T7nno1x7qVVt7ynO7Oq/ZSvmO3Rd2Fbkd6C7V9NDwMWA=
X-Google-Smtp-Source: AGHT+IFK3gjHgioDL2i0FIC4Wmg2sikK8o9RTszlH9Hh1fBVm4YaxyB+dUqhBorFN9L4fAn2a9GSlg==
X-Received: by 2002:a17:906:7004:b0:a59:bcfd:d950 with SMTP id a640c23a62f3a-a5a2d5cdbecmr1542300466b.46.1715940226416;
        Fri, 17 May 2024 03:03:46 -0700 (PDT)
Message-ID: <3ef32fec-53e7-403c-81d6-9f3e89966e2c@suse.com>
Date: Fri, 17 May 2024 12:03:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
To: Jan Beulich <jbeulich@suse.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 17.05.24 11:50, Jan Beulich wrote:
> On 17.05.2024 11:28, Chen, Jiqian wrote:
>> On 2024/5/17 16:20, Jan Beulich wrote:
>>> On 17.05.2024 10:08, Chen, Jiqian wrote:
>>>> On 2024/5/16 21:08, Jan Beulich wrote:
>>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>>   struct physdev_pci_device {
>>>>>>       /* IN */
>>>>>>       uint16_t seg;
>>>>>
>>>>> Is re-using this struct for this new sub-op sufficient? IOW are all
>>>>> possible resets equal, and hence it doesn't need specifying what kind of
>>>>> reset was done? For example, other than FLR most reset variants reset all
>>>>> functions in one go aiui. Imo that would better require only a single
>>>>> hypercall, just to avoid possible confusion. It also reads as if FLR would
>>>>> not reset as many registers as other reset variants would.
>>>> If I understood correctly that you mean in this hypercall it needs to support resetting both one function and all functions of a slot(dev)?
>>>> But it can be done for caller to use a cycle to call this reset hypercall for each slot function.
>>>
>>> It could, yes, but since (aiui) there needs to be an indication of the
>>> kind of reset anyway, we can as well avoid relying on the caller doing
>>> so (and at the same time simplify what the caller needs to do).
>> Since the corresponding kernel patch has been merged into linux_next branch
>> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240515&id=b272722511d5e8ae580f01830687b8a6b2717f01,
>> if it's not very mandatory and necessary, just let the caller handle it temporarily.
> 
> As also mentioned for the other patch having a corresponding kernel one:
> The kernel patch would imo better not be merged until the new sub-op is
> actually finalized.

Oh, sorry to have overlooked that the interfcae change isn't yet committed on
Xen side.

I'll drop the patch from my linux-next branch.


Juergen



From xen-devel-bounces@lists.xenproject.org Fri May 17 10:09:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:09:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724000.1129173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uX9-00016y-B2; Fri, 17 May 2024 10:09:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724000.1129173; Fri, 17 May 2024 10:09:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uX9-00016r-77; Fri, 17 May 2024 10:09:51 +0000
Received: by outflank-mailman (input) for mailman id 724000;
 Fri, 17 May 2024 10:09:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7uX8-00016k-Cm
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:09:50 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 97708af5-1435-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 12:09:49 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CH3PR12MB8483.namprd12.prod.outlook.com (2603:10b6:610:15c::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.54; Fri, 17 May
 2024 10:09:45 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 10:09:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97708af5-1435-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nf/m7rDMbVVcUcs0LEJZR3B6nGDp5aqSqkZAjYI+n22w+09P7JIyb9hiaCJsPvhWIYF/nwx66KLxtY1zpoDJ3dGIhXwDZAArK7PXkR6MhVSXLvmwDKwuWGD8egOz5SMUG3Q4bgt9QgpMfw15/ifNbUvNUhsx2Z/DjtMiBt3ZHYQPWNx9vdykAxwTJ9LNA++ylNaVAAI+2ex4xmu31PrG7dQXMRVtI2bvO5nLRsPh8c4kEtslwUFtESsNo2fesCMrGkNx8k5ow/9QqNis+XgEkZ9XOBUzIMHHWJORiLtYYXdafSZeQ4MPTWR2mFwIxrz7Ef/N0GM8mX+tiGv71AMZUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=VrBNDZm4NVDfTbuhqJBPUVkbLlFN+3+NAlARdffVE5I=;
 b=erZ+oGsvavOm12AWfMc/zOwC/8r0rtTBJS64mft+xl+EESgzGVrWM1xZjcWVTSS/q0OIkgR8AXRXWGZIHABJM261scTs5g4XwAvcYvVeM1l6CSbsi6YEjcOJ5zD4MgNWDy9PZRaJfzGMePqyFUfAydUBbIBep0Kl+7tfFAc02/6ILZHqjtKycgPWPn/sNeOq4g6J2MkTU+5Uct8mmnKfcXncLt8A6vYsncGEmzHTcDHKAu0du2MZUJvAtCfZL5tIg4ta2TS0xAKX3du8bfo1m62ytsm+Ai6+IaCgbUIk+7H80ZPppuhpZkC12E2K65ei2NEi6VHhHWS1BdKWcELHrw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VrBNDZm4NVDfTbuhqJBPUVkbLlFN+3+NAlARdffVE5I=;
 b=gdzRKasrknd2vjHgZ9o2PSvrph/WzM4znbJaC1GcmsN4dTI5AeRuAdlQ360lDP4R6tY/Fn5ooPFr/ZcCy99r0dL7y5eXabpp+gl6qD1Tq7IXYm6GXAiztRcfyKYuMaPmfcANGwZ3zr41umV93eJxc8r9WQqHt9x8u5RpIK5LETw=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>, Jan Beulich
	<jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Index:
 AQHap3bYcGtlWtrcpEyRLTK5fjq5SLGZ1UMAgAFwSAD//9GPgIAAlWQA//+DzoCAAAPIgIAAhyMA
Date: Fri, 17 May 2024 10:09:45 +0000
Message-ID:
 <BL1PR12MB58494E8AFFFC7ACE31C24DB0E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
 <3ef32fec-53e7-403c-81d6-9f3e89966e2c@suse.com>
In-Reply-To: <3ef32fec-53e7-403c-81d6-9f3e89966e2c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CH3PR12MB8483:EE_
x-ms-office365-filtering-correlation-id: d27bcfaf-9cbd-4af1-89a8-08dc76597a35
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|376005|1800799015|7416005|366007|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?YXh3QzRWL2JlS3hjY2VrdVNjU0Ywd2RlaEtPSUZta3ZtUUJqbXhnQjFVU25t?=
 =?utf-8?B?cXFFWVZHSHpwcjNxS0pyYjhZcGF1QTVRK3pwamU1YnNETGwweXZuRWRQVmRP?=
 =?utf-8?B?TDJFRS9sMTB1K2FoZEZzTjlJVWZmNWdmQ3QycFRndDBZQjNGZjhRdUlhKzR4?=
 =?utf-8?B?YThzOUpZS2tFTkRpb0ZXMElJRVNXK1k3VzZWMXdGcVYvM0h0T2lpODdjN0Jh?=
 =?utf-8?B?cHFZS0YveFd0d3I1UG9qMDF5VWpjL2ZvNGVPZnBqQU41N2FKcmZJSFBtN00y?=
 =?utf-8?B?QnlCUEZOK2JhNXNTVk5Vay9EZ2NkZ05YZGJmMFd1Y1p0OE8zU0tIdWNvVS9H?=
 =?utf-8?B?TnM5UU9DMkU5S29jeDdFemJpV3pIRldkdUNQNTVBK2R3Y1JqMEMwaHJka1o2?=
 =?utf-8?B?NGZ3MEk0Y2liU0xtQ1czaXhJY2dQQXBSSlM5cWpYZ2M2MDQ3L05FNytKUk9S?=
 =?utf-8?B?aXliOWpZdVdRaTJIOE1oSUQwM213S3BydEFFMFgvalJhcEN5UkpNakxOSDE3?=
 =?utf-8?B?TGxrdHVQcmxLKzRiWktmSnZ1YSt4c0JIbURrNkhYNFBOU2k1T1lXR3hTSnl2?=
 =?utf-8?B?U0htR29UUjR0eE1wVG1PQlU3Qm14SnhJOWV1ZG1zSjdWZkNyTjhxUXh2MXUz?=
 =?utf-8?B?d3dGYWQxTzBGMzcybHNac3lSNm5vcURpQzdYOGVlakczVURWcW55dnVqSGpQ?=
 =?utf-8?B?c3NWNHlXa2Vpc0tDbDg1VU5vTFZGeFpodGRaN2pydkIvS0N1RnBJS0dSY1ln?=
 =?utf-8?B?elJXUFZtd3FMbVRNNjhvdW9HUlAvYXFMOEVMbEdCSGgzNnI0K1g5Z3R2dmtj?=
 =?utf-8?B?bGdWVVNsMnJrQVpabFFLdm1xOEQzZnc4NUpQTklsL1ZjbXhGa2xpaFRrS2lx?=
 =?utf-8?B?M0lIYkNRRUhXeVNnMGUwcG5wa0lFU1hsMHZMTXQ1VlQremJUS3lhVm5RT2t4?=
 =?utf-8?B?NUpZTkJFWUEyaEtPbW1nSGZqSXR4cVFwNERRc0g5VXg3L1BBMWVvZGdGNndk?=
 =?utf-8?B?WkhoRVlKcEJvVnE3b3FXT3AvK09yaUEyUTdES1RIZWRtMis1eCtwV0J4Mi9G?=
 =?utf-8?B?cXhiY24wdUEvRmhPTzdrSmVHZFRnMCtUanpadFc1dWxlL29Qc2gyZkNYdW1v?=
 =?utf-8?B?M2JaMVpSY0pHSm1mWktla0xOc0VhMHN2R0ZLc2d4b0R4OGp4LytNbzBKNEhN?=
 =?utf-8?B?VWxWMy8zQUVLMVRCRTU3MUpmREFUT3VZQ2VxdHV6QXRSbGFTL1BVeXZ5a05u?=
 =?utf-8?B?RFVHV0JaWU92NjIycFltY2ViQjdUU2xjVzV4ZW9ic2lISS9aZFQ3V2s0SkFM?=
 =?utf-8?B?T0hBcW9ZV2l4VHdqV2FmWUZDZklabzBNRmdmMEZlTGtCdGJGM2d5cW9mNzNX?=
 =?utf-8?B?dmUvMFRVaGhLc1h1dmlLL3N2dVVhOXJnbWN4NW00Y2YxZmFWT0lYbjVXUDBt?=
 =?utf-8?B?aW9zbFRHRWpDeC9qQXAwcS9peHBBWUhEdWc4Z3hBV2ZYKzRaZW50bSt1ZGlB?=
 =?utf-8?B?ZnlhbGwycTZHOGo1alVFZkNlbXNYRTA2aU1EWWFhbUo0bGxQRmlNT2pGWkdz?=
 =?utf-8?B?eEU3Rno2d3NDZWtheVhHeUc2dTQ1NFg2bkgxVUZHMkp3a0t2MG9QbXdhaG9n?=
 =?utf-8?B?ME9HK04zd1ozTlg4dWJiclB0cXFLRE5WV2pwWitoV3ArSUthUzJ4K0NGREJq?=
 =?utf-8?B?NkVja0FFTUNhemwwOTJ5bXNsWVlYSDlZd25KVFZTajlORllkWmZ4SDVnPT0=?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(7416005)(366007)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?ZEtOZWplSmdSNktoajRMd0F2cVY4NG5qUXRXYUtjRUpRNjNHR0FOZkVmVmls?=
 =?utf-8?B?VlEwenNyU0o5bkxLQi9mUVNKOW9FS1pUNVRUOGtWTDFuOEwyZktPNWtoeGxC?=
 =?utf-8?B?MkZ3c2xOZW5jcGVjTzJnU0g3L3lsWFpBNXhRVW9sbXhZYzRhWUsrcE4veWp2?=
 =?utf-8?B?S0ZRNlRUUHpBalVlQUhtVDFTSGV4dHBGMnhpd2o3V0hWZDJjY1N5RHlKWURz?=
 =?utf-8?B?QnFQbUNnS2JQOXMyTjlHM1FNakE4aGVvcXg2QlJxdkFoQU1nZWdRVUE0bEtU?=
 =?utf-8?B?OXROa09hdWhsaVlXMll6a2lxUTV0TjdTaGxSQ050b1c3QWhROFpBblN5TStK?=
 =?utf-8?B?N29xbDZ2Vllxd3NuVFNjL2JnTnhoYW5DVWZQK0VHdklPMkpEMDcrMEc2NmxF?=
 =?utf-8?B?RXdqNzc4dDVpVnBLU0VCZmVCOGZhVVRkZWt2bzg1TW5NNkRnaktUeE9oTU03?=
 =?utf-8?B?Rmh1bTFIYUpEaEhFTkRZaVhvakFhOXFDRnI3UWJXc1l2ajJTSXpMZ0dGOFMx?=
 =?utf-8?B?MzNKQXl1U1hMcXNGNittdDluUFFScFlzQ1BmMWhhS3M0ZDc5aGZxcFJxakVh?=
 =?utf-8?B?b2NJdm4rVjJKaVRQQnZjUGlBMU9xNUFiN2hFZkMyTjlpVml2TjhpMUNLNkNP?=
 =?utf-8?B?Yno5eEduRnVvWWxTZGtrTVZZQThZUk9hdTU2cjk1TUQzUXJremlVWWV0MVV3?=
 =?utf-8?B?akNDdmM0RVg5RUJ4dUlaV3FZeEJ5SVJBaFo0cEE2a3h2Yk4wdFI2VGdocnAx?=
 =?utf-8?B?UXlvWjcvNEZORjU0MndRSTdxYlRYVCthS09DaTJyVmNMS2lOZk5RNWhYdEtO?=
 =?utf-8?B?YzVoa2VXVFlIVHViamY2SU9jS0ZhRStZOCtjOVpCS3BNUGNBZVV0RkNWeUxD?=
 =?utf-8?B?TnBYcWhDV2l4WFB6NEszSWx2dXJPeG53TWRxWTd3czhjVmJGWUZEWVl0T2ZY?=
 =?utf-8?B?bno3WktQeHk5Z0RvN21QQjVCVUxPOHVEV3ZRaUFRd0dNQTU3WXp0OVBSelNt?=
 =?utf-8?B?TXpsLzEvSEVSaVM2UjlOTk83QTJlZEtGekRLM0MwR1RRZVZScGErYUlIdTNw?=
 =?utf-8?B?ZExlZVlaR3o0ZnYvYkxKMDBrQkRwQ3hWdktmV1F3VnpmZWVlSzN5ZDB6bzYr?=
 =?utf-8?B?RVVNdlFrUTdzOXJwLzBFeThaeGRxVi84UktjOTlzLzdZUnl4VURhemN1QnBp?=
 =?utf-8?B?WUJEMDZkM1RTbWcrV0tKNkZSeGhvTHI5KzhSRWF1UzNKWW9mUWZlVTFWQzc2?=
 =?utf-8?B?Y2tEdHllV0dadGpYdG92c2VMdmlNK1l1WFpoSWN1b2lKa3NoS0FwUlY3MGs3?=
 =?utf-8?B?R3JLVmxzbE0rQjN0NlVoL0oxRWNxa0RFMEJrdDdxWWJhbWdsRU5CRE5YU0N1?=
 =?utf-8?B?Q20xZWFEbWFVUi9ndFdOQzZRMTlIbGZVOVNrc09DejN0WDZrMDhwcjkxbVRa?=
 =?utf-8?B?djRRb3NNUmVvYnlBbSsxWEhQS2ptaVlpRmUvRVVxd3ZIbi9SZ3VyNzVXdWVS?=
 =?utf-8?B?ckRCYUhDKzdUVWd5UDVncVJPZjRINHdCT2o0R3VPcmNKOXkwai8rUUdxOUkr?=
 =?utf-8?B?NmhoSVNaRk1RNG4rYVJwa1dwS3BtVWRBQW1Xc3hCL0lENzFtUTJ4UVdsSEE5?=
 =?utf-8?B?b1p0T3R3TUtQRVFNb0pkRTQ1aDJYSFFocXFtdlJoajFhZ2JOK2pKM20wdkVL?=
 =?utf-8?B?c0hUSm94RzdsRENzRFJyVXdWRkgrVjFuS1IwdFlLaDgzeWp5RG9vQ01qTTRD?=
 =?utf-8?B?SWZEbEIvcDlGTWpFSklFazdqK0VDU0M3MEJZUkZESW9XTHZtSG5jMzlIcXF0?=
 =?utf-8?B?c1RFN1U3eDRjYVVUNlFsL1pQK2ZLUC81REdZNEF4aitDalA4a1BtSGZLZGp0?=
 =?utf-8?B?TDlRbTlGVWp3ZkxSWFpHZWdwSlpDL0ZwWXovNy9LOW5CZk0vSFg1MDVrYm84?=
 =?utf-8?B?OVl2Z3RjNXlYRHVGWjBWTXJrK1BkZGRqRDFwaklVSXg3ZGdFcEUvdURPSDVh?=
 =?utf-8?B?VlhtRkt6ZnY4NmZaWXZOYzNBRy84QmZnZVVnYmdqTEtwdlYyditwVDk5UFkw?=
 =?utf-8?B?UndPaCtSK2NlUjZjZ0x6VzRFdzJzQURVa2lqQTBGVkZXMTRRR1VGS3Fzd0VL?=
 =?utf-8?Q?nOLQ=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <49A5F14578F7D345B3024562BC40F8C2@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d27bcfaf-9cbd-4af1-89a8-08dc76597a35
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 10:09:45.4174
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /PJoFsw0jMjWsbsMX8UWWQ+Bp913WiAFHnQXvoxQk0N3HFdrDO67z56SpkhlNKcmS5th1IP9io+zAT5ioipxMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8483

SGkgSnVlcmdlbjoNCg0KT24gMjAyNC81LzE3IDE4OjAzLCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0K
PiBPbiAxNy4wNS4yNCAxMTo1MCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+PiBPbiAxNy4wNS4yMDI0
IDExOjI4LCBDaGVuLCBKaXFpYW4gd3JvdGU6DQo+Pj4gT24gMjAyNC81LzE3IDE2OjIwLCBKYW4g
QmV1bGljaCB3cm90ZToNCj4+Pj4gT24gMTcuMDUuMjAyNCAxMDowOCwgQ2hlbiwgSmlxaWFuIHdy
b3RlOg0KPj4+Pj4gT24gMjAyNC81LzE2IDIxOjA4LCBKYW4gQmV1bGljaCB3cm90ZToNCj4+Pj4+
PiBPbiAxNi4wNS4yMDI0IDExOjUyLCBKaXFpYW4gQ2hlbiB3cm90ZToNCj4+Pj4+Pj4gwqAgc3Ry
dWN0IHBoeXNkZXZfcGNpX2RldmljZSB7DQo+Pj4+Pj4+IMKgwqDCoMKgwqAgLyogSU4gKi8NCj4+
Pj4+Pj4gwqDCoMKgwqDCoCB1aW50MTZfdCBzZWc7DQo+Pj4+Pj4NCj4+Pj4+PiBJcyByZS11c2lu
ZyB0aGlzIHN0cnVjdCBmb3IgdGhpcyBuZXcgc3ViLW9wIHN1ZmZpY2llbnQ/IElPVyBhcmUgYWxs
DQo+Pj4+Pj4gcG9zc2libGUgcmVzZXRzIGVxdWFsLCBhbmQgaGVuY2UgaXQgZG9lc24ndCBuZWVk
IHNwZWNpZnlpbmcgd2hhdCBraW5kIG9mDQo+Pj4+Pj4gcmVzZXQgd2FzIGRvbmU/IEZvciBleGFt
cGxlLCBvdGhlciB0aGFuIEZMUiBtb3N0IHJlc2V0IHZhcmlhbnRzIHJlc2V0IGFsbA0KPj4+Pj4+
IGZ1bmN0aW9ucyBpbiBvbmUgZ28gYWl1aS4gSW1vIHRoYXQgd291bGQgYmV0dGVyIHJlcXVpcmUg
b25seSBhIHNpbmdsZQ0KPj4+Pj4+IGh5cGVyY2FsbCwganVzdCB0byBhdm9pZCBwb3NzaWJsZSBj
b25mdXNpb24uIEl0IGFsc28gcmVhZHMgYXMgaWYgRkxSIHdvdWxkDQo+Pj4+Pj4gbm90IHJlc2V0
IGFzIG1hbnkgcmVnaXN0ZXJzIGFzIG90aGVyIHJlc2V0IHZhcmlhbnRzIHdvdWxkLg0KPj4+Pj4g
SWYgSSB1bmRlcnN0b29kIGNvcnJlY3RseSB0aGF0IHlvdSBtZWFuIGluIHRoaXMgaHlwZXJjYWxs
IGl0IG5lZWRzIHRvIHN1cHBvcnQgcmVzZXR0aW5nIGJvdGggb25lIGZ1bmN0aW9uIGFuZCBhbGwg
ZnVuY3Rpb25zIG9mIGEgc2xvdChkZXYpPw0KPj4+Pj4gQnV0IGl0IGNhbiBiZSBkb25lIGZvciBj
YWxsZXIgdG8gdXNlIGEgY3ljbGUgdG8gY2FsbCB0aGlzIHJlc2V0IGh5cGVyY2FsbCBmb3IgZWFj
aCBzbG90IGZ1bmN0aW9uLg0KPj4+Pg0KPj4+PiBJdCBjb3VsZCwgeWVzLCBidXQgc2luY2UgKGFp
dWkpIHRoZXJlIG5lZWRzIHRvIGJlIGFuIGluZGljYXRpb24gb2YgdGhlDQo+Pj4+IGtpbmQgb2Yg
cmVzZXQgYW55d2F5LCB3ZSBjYW4gYXMgd2VsbCBhdm9pZCByZWx5aW5nIG9uIHRoZSBjYWxsZXIg
ZG9pbmcNCj4+Pj4gc28gKGFuZCBhdCB0aGUgc2FtZSB0aW1lIHNpbXBsaWZ5IHdoYXQgdGhlIGNh
bGxlciBuZWVkcyB0byBkbykuDQo+Pj4gU2luY2UgdGhlIGNvcnJlc3BvbmRpbmcga2VybmVsIHBh
dGNoIGhhcyBiZWVuIG1lcmdlZCBpbnRvIGxpbnV4X25leHQgYnJhbmNoDQo+Pj4gaHR0cHM6Ly9n
aXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5lbC9naXQvbmV4dC9saW51eC1uZXh0Lmdp
dC9jb21taXQvP2g9bmV4dC0yMDI0MDUxNSZpZD1iMjcyNzIyNTExZDVlOGFlNTgwZjAxODMwNjg3
YjhhNmIyNzE3ZjAxLA0KPj4+IGlmIGl0J3Mgbm90IHZlcnkgbWFuZGF0b3J5IGFuZCBuZWNlc3Nh
cnksIGp1c3QgbGV0IHRoZSBjYWxsZXIgaGFuZGxlIGl0IHRlbXBvcmFyaWx5Lg0KPj4NCj4+IEFz
IGFsc28gbWVudGlvbmVkIGZvciB0aGUgb3RoZXIgcGF0Y2ggaGF2aW5nIGEgY29ycmVzcG9uZGlu
ZyBrZXJuZWwgb25lOg0KPj4gVGhlIGtlcm5lbCBwYXRjaCB3b3VsZCBpbW8gYmV0dGVyIG5vdCBi
ZSBtZXJnZWQgdW50aWwgdGhlIG5ldyBzdWItb3AgaXMNCj4+IGFjdHVhbGx5IGZpbmFsaXplZC4N
Cj4gDQo+IE9oLCBzb3JyeSB0byBoYXZlIG92ZXJsb29rZWQgdGhhdCB0aGUgaW50ZXJmY2FlIGNo
YW5nZSBpc24ndCB5ZXQgY29tbWl0dGVkIG9uDQo+IFhlbiBzaWRlLg0KPiANCj4gSSdsbCBkcm9w
IHRoZSBwYXRjaCBmcm9tIG15IGxpbnV4LW5leHQgYnJhbmNoLg0KVGhhbmtzLiBJIHdpbGwgbW9k
aWZ5IG15IGNvZGUgYWNjb3JkaW5nIHRvIEphbidzIHJlcXVpcmVtZW50cyBhbmQgc2VuZCBhIG5l
dyB2ZXJzaW9uIHNvb24uDQoNCj4gDQo+IA0KPiBKdWVyZ2VuDQo+IA0KDQotLSANCkJlc3QgcmVn
YXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:27:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:27:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724013.1129183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uo3-0005Vw-Qu; Fri, 17 May 2024 10:27:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724013.1129183; Fri, 17 May 2024 10:27:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uo3-0005Vp-Nq; Fri, 17 May 2024 10:27:19 +0000
Received: by outflank-mailman (input) for mailman id 724013;
 Fri, 17 May 2024 10:27:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DaPs=MU=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s7uo2-0005Vj-RS
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:27:18 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 084586e5-1438-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 12:27:16 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id DFAF44EE073C;
 Fri, 17 May 2024 12:27:14 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 084586e5-1438-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org,
	nicola.vetrini@bugseng.com
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH] automation/eclair_analysis: set MISRA C Rule 10.2 as clean
Date: Fri, 17 May 2024 12:27:10 +0200
Message-Id: <7ded6026fc8a02f53bdb841a4e4c5e3be45a15a0.1715930684.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This rule has no more violations in the codebase, so it can be
set as clean.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index acea15f486a1..b7a9f75b275b 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R10.2||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 10:32:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:32:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724018.1129192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7usZ-0007op-BS; Fri, 17 May 2024 10:31:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724018.1129192; Fri, 17 May 2024 10:31:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7usZ-0007oi-8u; Fri, 17 May 2024 10:31:59 +0000
Received: by outflank-mailman (input) for mailman id 724018;
 Fri, 17 May 2024 10:31:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7usY-0007oc-7u
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:31:58 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id af0c6883-1438-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 12:31:56 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-572b37afd73so5338118a12.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 03:31:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c323d3asm11728262a12.90.2024.05.17.03.31.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 03:31:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af0c6883-1438-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715941915; x=1716546715; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KBkYLZVPnkDZ3y9RmABTZ4P2h5x58O6bKi/zZ0eegZI=;
        b=g9fOiX5lIrj8Wd8PPyI9in8P/BU4Z8PqHKEz12Qwi7fU0Qdz/spfDzFrJjggYICR+D
         HuXAxvDWjFjUGxZunSSm0kjYLdJjAF2ytL7UKJ6pUUvDSHVXVfGjEffUxEKPUHZV+uFP
         q0lXb5+hsEGdF9pRcFdvOpH74QTiT9i1bF1CI11ppboJxt6Hfo8jMN3qMnydpg/UXzjZ
         ezNQY4kNensBg/FEzPmQ01WOzUYwF3hpGAKp81z8Q3sHwT2TTaH5qoRoOG3NW3aJz6Li
         gDorWfo1KH9QLtXhFyp3Vz0fRk30yE2TeKmJqG8IxD04RUC12Jsxa+MWxsoWiXRoIMxY
         S82w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715941915; x=1716546715;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KBkYLZVPnkDZ3y9RmABTZ4P2h5x58O6bKi/zZ0eegZI=;
        b=hQPH8i5L7WF1C9wVJzBbJygISgWUlpmKGlG/KtzOAte+OhYCAxSlOB2+EiPqepXlu0
         jUKmk98h6Z3V9lTp5TkxdDYDxGMrohGLBc5YK4Y3mB6hp7+dq6nCqvGzXY3s+XnJ2Pfp
         yweUokEXHYB2rzt3w1aweTdaV0eXq6mNhM8ExXLsSBH6Hcpk5bfosCby7PDn4hIx2Eew
         P5XUHyZGWHpnI5rLG6cbGZlCdlBKTcP8hJINGyuX+Rz1jOR9OG8VqiGmBgSNwGs820Jq
         L2xIlvVHEnpTNWyEHFFOkTmyiNQg2TXrbYnRWAN6cOh/CiJFGT7mh6watR0lRSE54D5j
         GwQw==
X-Forwarded-Encrypted: i=1; AJvYcCVNPtihAVR0X95c5Y8sVFjJtuIGwD/OaVs8vNWYGGMqmU2yc9Ww5hAQg/WD1jLEIfX0TNFIj+8nB+ygdgR+mdOOA6uEsHQQcNDo8S+Xf3Y=
X-Gm-Message-State: AOJu0Ywpi0LCW3JVn4uYgb9VLBZofegnlEIG+PrGB+LNvDulJAuDdDW8
	f2ReSvX7Uf8c8A2kzSJBs1RxnvaU/cqUCUd9GbIQ18lRGM+PCt2RquMLYt8jag==
X-Google-Smtp-Source: AGHT+IEMFH1AZ4RL7xyfe4Ew+k13sKOkJOBsWMUJD1XBPT5Az8Atl5BxuZf97o+ADFQHl5G/zz14Bw==
X-Received: by 2002:a50:8e4d:0:b0:572:4faf:ed67 with SMTP id 4fb4d7f45d1cf-5734d5cecb4mr13048243a12.24.1715941915555;
        Fri, 17 May 2024 03:31:55 -0700 (PDT)
Message-ID: <8d96e75d-5ca1-4276-9ddc-b46b11949fc6@suse.com>
Date: Fri, 17 May 2024 12:31:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Content-Language: en-US
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
 <BL1PR12MB5849CCF9A890A7DA969F6474E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849CCF9A890A7DA969F6474E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 12:00, Chen, Jiqian wrote:
> On 2024/5/17 17:50, Jan Beulich wrote:
>> On 17.05.2024 11:28, Chen, Jiqian wrote:
>>> On 2024/5/17 16:20, Jan Beulich wrote:
>>>> On 17.05.2024 10:08, Chen, Jiqian wrote:
>>>>> On 2024/5/16 21:08, Jan Beulich wrote:
>>>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>>>  struct physdev_pci_device {
>>>>>>>      /* IN */
>>>>>>>      uint16_t seg;
>>>>>>
>>>>>> Is re-using this struct for this new sub-op sufficient? IOW are all
>>>>>> possible resets equal, and hence it doesn't need specifying what kind of
>>>>>> reset was done? For example, other than FLR most reset variants reset all
>>>>>> functions in one go aiui. Imo that would better require only a single
>>>>>> hypercall, just to avoid possible confusion. It also reads as if FLR would
>>>>>> not reset as many registers as other reset variants would.
>>>>> If I understood correctly that you mean in this hypercall it needs to support resetting both one function and all functions of a slot(dev)?
>>>>> But it can be done for caller to use a cycle to call this reset hypercall for each slot function.
>>>>
>>>> It could, yes, but since (aiui) there needs to be an indication of the
>>>> kind of reset anyway, we can as well avoid relying on the caller doing
>>>> so (and at the same time simplify what the caller needs to do).
>>> Since the corresponding kernel patch has been merged into linux_next branch
>>> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240515&id=b272722511d5e8ae580f01830687b8a6b2717f01,
>>> if it's not very mandatory and necessary, just let the caller handle it temporarily.
>>
>> As also mentioned for the other patch having a corresponding kernel one:
>> The kernel patch would imo better not be merged until the new sub-op is
>> actually finalized.
> OK, what should I do next step?
> Upstream a patch to revert the merged patch on kernel side?
> 
>>
>>> Or it can add a new hypercall to reset all functions in one go in future potential requirement, like PHYSDEVOP_pci_device_state_reset_all_func.
>>
>> I disagree. We shouldn't introduce incomplete sub-ops. At the very least,
>> if you want to stick to the present form, I'd expect you to supply reasons
>> why distinguishing different reset forms is not necessary (now or later).
> OK, if want to distinguish different reset, is it acceptable to add a parameter, like "u8 flag", and reset every function if corresponding bit is 1?

I'm afraid a boolean won't do, at least not long term. I think it wants to
be an enumeration (i.e. a set of enumeration-like #define-s). And just to
stress it again: The extra argument is _not_ primarily for the looping over
all functions. It is to convey the kind of reset that was done. The single
vs all function(s) aspect is just a useful side effect this will have.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:38:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:38:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724023.1129202 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uyu-0008SK-0m; Fri, 17 May 2024 10:38:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724023.1129202; Fri, 17 May 2024 10:38:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7uyt-0008SD-U5; Fri, 17 May 2024 10:38:31 +0000
Received: by outflank-mailman (input) for mailman id 724023;
 Fri, 17 May 2024 10:38:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4QDV=MU=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7uys-0008S7-Cm
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:38:30 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 98f2a54d-1439-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 12:38:29 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id C26DE80078;
 Fri, 17 May 2024 11:38:27 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98f2a54d-1439-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715942307;
	bh=JbgDqEkKYwG8oYvQtlqZV0fdmMP03V4Gg1bYu38mNKU=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=iGlpphG1VruUs3TmxFGzJaolWTubYWOxySTcSSiGUB3iJDUBr0+PJw3NlIB4yeMdD
	 UwXtYKm4Yj4YkKGJkrW4een46sq12NY2d3USO5S3JhCRU1PHR+kLCBPjAVHtr+8Iwc
	 0d1uqv5wvs6g4+0HE7xM7Uc5GuRRqvhrCEEEQfas=
MIME-Version: 1.0
Date: Fri, 17 May 2024 11:38:27 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Anthony Perard <anthony.perard@citrix.com>
Subject: Re: [PATCH v3 3/4] tools/hotplug/Linux: Add bridge VLAN support
In-Reply-To: <CAKf6xpvX2T3xhx59b0_X3e+9BpYuMZS84P=ghisZFD3yWsFssw@mail.gmail.com>
References: <20240516105612.15306-1-leigh@solinno.co.uk>
 <20240516105612.15306-4-leigh@solinno.co.uk>
 <CAKf6xpvX2T3xhx59b0_X3e+9BpYuMZS84P=ghisZFD3yWsFssw@mail.gmail.com>
Message-ID: <18967904c69aa6d4975d1a80dd825aeb@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

Hi Jason,

On 2024-05-17 03:19, Jason Andryuk wrote:
> On Thu, May 16, 2024 at 6:56 AM Leigh Brown <leigh@solinno.co.uk> 
> wrote:
>> 
>> Update add_to_bridge shell function to read the vlan parameter from
>> xenstore and set the bridge VLAN configuration for the VID.
>> 
>> Add additional helper functions to parse the vlan specification,
>> which consists of one or more of the following:
>> 
>> a) single VLAN (e.g. 10).
>> b) contiguous range of VLANs (e.g. 10-15).
>> c) discontiguous range with base, increment and count
>>    (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).
>> 
>> A single VLAN can be suffixed with "p" to indicate the PVID, or
>> "u" to indicate untagged. A range of VLANs can be suffixed with
>> "u" to indicate untagged.  A complex example would be:
>> 
>>    vlan=1p/10-15/20-25u
>> 
>> This capability requires the iproute2 bridge command to be
>> installed.  An error will be generated if the vlan parameter is
>> set and the bridge command is not available.
>> 
>> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
>> 
>> ---
>>  tools/hotplug/Linux/xen-network-common.sh | 103 
>> ++++++++++++++++++++++
>>  1 file changed, 103 insertions(+)
>> 
>> diff --git a/tools/hotplug/Linux/xen-network-common.sh 
>> b/tools/hotplug/Linux/xen-network-common.sh
>> index 42fa704e8d..fa7615ce0f 100644
>> --- a/tools/hotplug/Linux/xen-network-common.sh
>> +++ b/tools/hotplug/Linux/xen-network-common.sh
> 
>> +_vif_vlan_setup() {
>> +    # References vlans and dev variable from the calling function
>> +    local vid cmd
>> +
>> +    bridge vlan del dev "$dev" vid 1
> 
> Is vid 1 always added, so you need to remove it before customizing?

Correct. I will add a comment to that effect.

>> +    for vid in ${!vlans[@]} ;do
>> +        cmd="bridge vlan add dev '$dev' vid $vid"
>> +        case ${vlans[$vid]} in
>> +             p) cmd="$cmd pvid untagged" ;;
>> +             u) cmd="$cmd untagged" ;;
>> +             t) ;;
>> +        esac
>> +        eval "$cmd"
> 
> Sorry if I missed this last time, but `eval` shouldn't be necessary.
> 
> 
>         local args
> 
>         case ${vlans[$vid]} in
>              p) args="pvid untagged" ;;
>              u) args="untagged" ;;
>              t) ;;
>         esac
>         bridge vlan add dev "$dev" vid "$vid" $args
> 
> args unquoted so it expands into maybe two args.  You could also make
> args an array and do "${args[@]}"

I will make that change, using an array.

>> +    done
>> +}
>> +
>> +_vif_vlan_membership() {
>> +    # The vlans, pvid and dev variables are used by sub-functions
>> +    local -A vlans=()
>> +    local -a terms=()
>> +    local -i i pvid=0
>> +    local dev=$1
>> +
>> +    # Split the vlan specification string into its terms
>> +    readarray -d / -t terms <<<$2
>> +    for (( i=0; i<${#terms[@]}; ++i )) ;do
>> +        _vif_vlan_parse_term ${terms[$i]%%[[:space:]]}
> 
> Because terms is not in double quotes, wouldn't it expand out?  Then
> any whitespace would be dropped when calling _vif_vlan_parse_term.
> Your %% only drops trailing spaces too.  Maybe you want
> "${terms//[[:space:]]/}" to remove all spaces?

That is a hack because readarray adds a newline at the end of the
last element (not sure why).  I will change the code just to fix up
the last element before the loop, and it will be clearer.

> Stylistically, you use (( )) more than I would.  I'd do:
> 
> local term
> for term in "${terms[@]}" ; do
>      _vif_vlan_parse_term "$term"
> 
> But you can keep it your way if you prefer.

You guessed correctly that like using (( )) but in this case your
suggestion is simpler, so I will make that change.

> Regards,
> Jason

I am making the changes then will test, after which I will send an
updated version.

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:45:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:45:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724027.1129213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7v5m-0002Pg-Md; Fri, 17 May 2024 10:45:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724027.1129213; Fri, 17 May 2024 10:45:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7v5m-0002PZ-K3; Fri, 17 May 2024 10:45:38 +0000
Received: by outflank-mailman (input) for mailman id 724027;
 Fri, 17 May 2024 10:45:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7v5l-0002PT-OL
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:45:37 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060b.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 96c0423a-143a-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 12:45:35 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by PH8PR12MB7280.namprd12.prod.outlook.com (2603:10b6:510:220::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Fri, 17 May
 2024 10:45:31 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 10:45:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96c0423a-143a-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kUuW+ytr87E0V+/Oi+Sk3PlMKGU3crhODDs07VlzSOB5ApqUV2xDtWZqtdede4etCfzvCnJDbr5Zwxzt1mIRlUsvBmtPTgv+W83l470EkLjnLsAREteb5EZvJ4lNWk3Puucv9mifhTh2vS/YUvgA3qbe2CiU4eHS5yBkOnCbagjr90ikiSG73F6mzrsRRv1d3Yt2jFh9g+mQiVdyj0JTfNvwX+/zfR0OiWbISikv89r9aLDOxDnrhFmAFm9zfD0hqz83dT2BGy5yl1yP+Fxi6eQXar0GK+h7jqCceoVHxii0rAaaqn+AWSTTIQxxXpAkORGyqYMzd3fzYRkIreQZ3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=KbFX44emBKD3eyLevuU6FiqZZ0xQP9oMRE4uPdIl4Ko=;
 b=kKPgoN+PKRbQj188Hz9SxSQsgubxYgjkDTKi70FkeRz6vS5s4bUL11M25k4qG++RL++ZT2YhONQx6yafKVZqz8zBa6W3FNVrkDFgAUzvdcsKHO/klOiVuJnwzcztyN/Ovusd4IRScBhRqxtXw5kiNHCUgDv7/CI2/Hp+2+Gq4SIAyJoqAOGljnVpqjm3t11wcXuH/UMKdEApBoTmMfgOnaVh6juU0MiQp5YaArNfwgfXc29M5Zd1mDQg0nl2aRzV7GKWURprY8C01ixpyH5MpBlTZ5TgUA0abelIgF5Qp/PCVGA96uaptqc4+SlGPzVZjjNxVbN6xVXq/+jHlHl8pw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KbFX44emBKD3eyLevuU6FiqZZ0xQP9oMRE4uPdIl4Ko=;
 b=LcYvv0FzLygmARHHHMZmyjFV4J3s+zaggVCXZI35mZJjGLko6H8j2c8HRKTjx5hCreIDTKqrepzfzI7Ol8EBxaUxwGSS9FvyzMxaPHACXNotENuZPNzvLsBl8BEMTS/tbDzjkdKvyzSCcUSmJ9cJCiGunwkiswMGUqKq4zyL88o=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Chen,
 Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index: AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4A=
Date: Fri, 17 May 2024 10:45:31 +0000
Message-ID:
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
In-Reply-To: <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|PH8PR12MB7280:EE_
x-ms-office365-filtering-correlation-id: 8dd0abd7-8a0c-4a83-5580-08dc765e794d
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|1800799015|7416005|366007|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?VVFEVncyZUtiU2hMK1ZJZVZTRm1yaEdDZVR5MFI4dUZ6NXJzRUx5OStHRkZM?=
 =?utf-8?B?U09qYXZGekJHbG5DL1k1elpRSFl5QzdlZUV3R2FiZVhZeDlFdzRyTWhEemMv?=
 =?utf-8?B?NlBwM3NjWFRTbGczRGdDdnlTZW1CeWt1VHZzMFBOYTlqSW9qUUF3Vml0UlF3?=
 =?utf-8?B?cFNDcHM1RjBXMHEvZ0NXTlNmcXk4S1JxQVdLOHBNY2wveStGdVZCMWlQSUNa?=
 =?utf-8?B?UHZXZHNiYm5QaGJVTEo3Ris4YkdhY0tHYTlLSS95K0MyOWg4U1ZGOXFLeFRF?=
 =?utf-8?B?WEpPTCt5SUhOUzFTS3RnZG1RQzBrRFhDdWNIY3JCb2tzQ0xHQzhGaGtSdFBj?=
 =?utf-8?B?cjlsNnZjQUN6TS94UVkveXpJbmtRYkc5ZGlNVUt4bElFb25qUGZyMGVzZFIr?=
 =?utf-8?B?WWE5MGZlY2M2bE0vSTB0R3ZlTW1jMUZna3I1TTB6M1ZiOXBQUWIrRHhGSkVE?=
 =?utf-8?B?UkRBYWFJaWlxcGlFZmMrQTM1NkV6TzVPU2M2Y1RMbXZUQ1psYnI5VTdJanRs?=
 =?utf-8?B?TkUrZStYdDkxZ3RrL1VXWkgxVGN6UmQxUExnNS9lZWJBTGtkeGo3eTArSU96?=
 =?utf-8?B?RGRlZ2NjUFRhZDkvRjYrcktGYU16TUNyN0lDZWEzaktiQ21ZSEVWUW4zbm40?=
 =?utf-8?B?UFIzRjdiU2JZbERXWFRZcGRaTWpyUFdZd0ZTMWt3clRSQnlqdVFPT29ucVNX?=
 =?utf-8?B?Vm1tYi8zRXFZN2VPaHJaNkVnTHN1S2dFdkExYTlHY1A1VlU3dFlydWpkYVpu?=
 =?utf-8?B?Y1N2d1RxU1EvTGJmaFFickJOcE1BdFdmOEo0NHlRVlFUTzl5bnVjMFNMeDFH?=
 =?utf-8?B?SGlFczFaVFN0ajBqSW5zenFXOXQwREdzeE9sYWhmcVdYclN2UEMxRmh6emg3?=
 =?utf-8?B?d1JXR2Z5bWlQbmw4RTdoNWlPcjc3QXdGc3NVK3Q2ZEh1LzJoWnBYcGhBcjRo?=
 =?utf-8?B?a3E2amJoM1oveHo0eVBuUTNySjlIQXRUNlMrQXl2MlRlS3FUM0JTa1BreUtJ?=
 =?utf-8?B?K25USWJYeVRlcHZ2OGkzZndHUVNCclZNWWY3ck9tZGZOTFQ5eXRiU1NUQXc5?=
 =?utf-8?B?NW54SlczVmlGTlJLN0VFbmhHWWpmNHBKWHZLRVFRVmt4RStzbm94VUd1OThU?=
 =?utf-8?B?OU5zSXZNVU5LckJhTEExeHlNTzkvdmpFQUpCRDRBZDR4OWc4L0wrc1VacjZ4?=
 =?utf-8?B?eTQ4QVBMYVFwYVBkWHJGM3RhdEZETzQ0dHp4Z0huNmN2Tk0wTk9DcSt2N1d0?=
 =?utf-8?B?Y05ZejJmYVZRdFk1VUpiTThNc284anltaDM1N3RXVWxGZ0hLWVpyY1BlZ0ht?=
 =?utf-8?B?QTBJckNTN2hscWNVdjdxWVNHb1JBQUxzT0ZhRWF4YjZGRXpoalBNa1FsTi9r?=
 =?utf-8?B?djdhRlB3UUZ2N0ZqZmRucG9LUkREemoyYzUyTmdCNU9YcVd3OWs4eWlyTmN0?=
 =?utf-8?B?RGdhbjBtZndLdm5RMVM5dWE5RWthVzdzTEFEVEtWb09wdHdwWmFSZnBGYzI1?=
 =?utf-8?B?N3p4SXg2NTRoU2drK2hoNEo4WjgwQ2x2dmVHYTg4UTVtcWNuUXdQQk1LQzVu?=
 =?utf-8?B?bW8yNmNFQ0J1VGwvRWgwRFRicEpqa0NNVTI4RVpGdlFmcU5XUzBrVWs0RU9Y?=
 =?utf-8?B?V2tSOEU2dGpJb0Nodmh5SXF5emsvd3Nyc280L0Zmd0lCMFlRaUprNkd6SXFE?=
 =?utf-8?B?T2prQWdIWGx5TjY3VjVlVWZhRUh4VTNGd0lLNmZLVU5hZTE1cmFqRGtFLzdV?=
 =?utf-8?B?aldUZE9vYktjSUlLR3o3Ums3alR6OXUwbkxyMTVLOTAwV052TkFIRVBJaEdp?=
 =?utf-8?B?NW1aTmcwNjF5d3Fwbmd6dz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(7416005)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?a1JkdDlKazBNQ3RjUTdOUGJ2M3ZMb3k4c2plbjJ2ajh6MmRwc0ZPbFpSUVZG?=
 =?utf-8?B?UDJOVGpCV0NLcFZ1K2dVam94MnBNdnJ4bXNKUXVHRWVaNG95dHlNejMvc2k3?=
 =?utf-8?B?WlMzVUpyeVZPVTFNNHdkUlRTSzlzQ1ZoL1BGQStiZ0JNUksvMFVJWk1RNlps?=
 =?utf-8?B?dGM4Y1F6QjcxQ2ZlbURodDRLeTFOaHB5ZzhEOWIzZXFWWDAzQ0FUL1FLd29K?=
 =?utf-8?B?RUhmMEd5KzhQd3ZjNHcxc0xLRi9odzc0VDdxYnhqbzNXM3RBaCtXQVh6RjFB?=
 =?utf-8?B?eHhqZHkySmFxRmpPMXN3ckRhR2FDbnYyTFRWL1A3QjY0Z1JxQ1NJR2tzREFW?=
 =?utf-8?B?ZTZ0Z2Fmdy9adUNMZWFWcWdTOE5lY3JIcENuL3grSXpzdTRhRFplYmVBQkp1?=
 =?utf-8?B?N0NxS0lJVnZ2T1YxOE53YlVEamYzbVpiUkl2SVM2WWJKUzJDY29YNy83aVZS?=
 =?utf-8?B?K3BDKzUxZVI2QTRuT2ZSMlcySGE3cjJKam9RMWFoenlCaEMrSXVvR2JCakFJ?=
 =?utf-8?B?bWd2Tm9mMm5FVUppTlhRV3FIUzVpaDJUNFVENkpmU1dyTjA3d1VUeGh2cEdG?=
 =?utf-8?B?WmZXWHpodmxCZm5vdTRNMTZaNDNlRzdSeGs3Sk1WSVdiS0ZFbkNZbUpsQm8v?=
 =?utf-8?B?ZGRzYjVwYkNURU5XYVA2dTFYTi85UWFSc1ZBbGFjcHFOdWxibzhPSGJIYi9q?=
 =?utf-8?B?UlhjbUhmZS9ubDFoOUl4cDJ6WEhJK0JZRFVIZ1RKbHpFcnNyUDdLYWM3OEor?=
 =?utf-8?B?Y2ZpQ1pId1RodVhWVklqcXM5aGJEcnhrZUFSSDZ2Qi9XRFNVclU5RmljMUNk?=
 =?utf-8?B?N01lZXFxUG5FTWdCbUE5UThNekpiRUFiUmJaQ05wS0NUUndIZE5XdjE4OU1H?=
 =?utf-8?B?cjF6R3VGVWdWUUZ5eVN1QUdrZ2ZSekVDUzZPbGZFZHozaVVwN0FFN0ZjWXQ0?=
 =?utf-8?B?d3FnVFRCOFdybGVwL3ZvNnNIb0pNcjRkSW5jV3F3ZWpFTVJURXIxS2REbVlH?=
 =?utf-8?B?VzI3Vk4wSm5wWVBiVjJsUHZ6L0pKblV6c29BcnpYUTU0YjI2Z2YrRVlqb1Uz?=
 =?utf-8?B?ZHlPdklETk9Gc3BwWjZMbDRDcktCcU5vazZaWGZ5L3kxN2s2bUN4d0dvTENu?=
 =?utf-8?B?SXdtRUlWUHlPYSsweGVaQ25iZjI2dzB6ZlYxYWowUXc5ZTE2S25WRXRCem8v?=
 =?utf-8?B?cU5La2VkSnB4YlNJMDlYb1FZTU1sSC8xRWpUelRPejRENm41VC9BM1lsUDNU?=
 =?utf-8?B?SVgyS0g3SndtOURXUkJXNXdLODZtdmpUR3kxby8vTDBzcmlrOTVUVHZSclVT?=
 =?utf-8?B?d3RFTmswMkZ6cWFzK3NzTUtvbERDeTdIN3NSUGNmcU9YYVhENnFwalRkNVVF?=
 =?utf-8?B?aXVsZlBUTkRUVU1wZUFqZDl1VGt4ZWdsZ0htUkVGVXAwVjhyQ3h1N05EVWI1?=
 =?utf-8?B?VTAwSjVpVTZrSGFmcUN5NERvYUN6VFErVjhyVUZNQUZ1MHgzNGcwblNtNGxh?=
 =?utf-8?B?UmU5K1dsVW5RdE1LVzhxSmpzVjRVdEJIOENKV2ZqNS9yVGEzMHZkdzVyY1Bw?=
 =?utf-8?B?NkI5SGpjU0FXaWNGUkpMNXFRcEovdW1EVnZSa0xUVTZhYXJyQzJOMVJsYjRN?=
 =?utf-8?B?blRjazRMM1dXV1Yrd1VxUVY3NkMrZTJPWGJRd1FPWksvZERKN0FXQnAvUTdQ?=
 =?utf-8?B?d1pSOUUyZmU0SXVEY1hHaGVEYjdlRWEzSE5BT0h5MjVXWUtzVFk3Q2xqRzZr?=
 =?utf-8?B?RGQwcThldjRIS2xlNWZudDI4aGFpRldPeDVZdW8rOTFaaWgxbnptYU1mblFr?=
 =?utf-8?B?WCtydkZFS2w0bW4weW8xK0ZPb1FOWHMrRzM4Q2pkYnNBWDdPaFQ5QzFydkZ6?=
 =?utf-8?B?NERsaGpxc3M0alMwUEF6MndBTE45WHFBenZMNE9QeC8veVppVjk3cWhWeDRK?=
 =?utf-8?B?Sy9kVjJQTThqQnpMd2YwUkUydTljM2dQZFY1V3JLRXVTTmFGTDAyZTJsM0hO?=
 =?utf-8?B?UFplR0UwK0pZaVVMTFhLaG1kRW1ZWE40S0hnY0RrN0k5KzkxTTRCZUNwbjdm?=
 =?utf-8?B?Um5oZE5Za2RNdmtMTFdvRDNKU2k3T3JxQ1lvdlNvNWNKT3ljSi9JbWZIeHZ5?=
 =?utf-8?Q?5B6o=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F6A69957F123574CA7D320559B46C539@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8dd0abd7-8a0c-4a83-5580-08dc765e794d
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 10:45:31.3747
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ua+UOsQ5afVTh/cMCJqn9TKWqTW6nDvNK9x4h38wCPCTaWsuxB9ux+G9+cKLAIGv80QUoh5irxc/4R5BAukf+Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7280

T24gMjAyNC81LzE2IDIyOjAxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTYuMDUuMjAyNCAx
MTo1MiwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBTb21lIHR5cGUgb2YgZG9tYWluIGRvbid0IGhh
dmUgUElSUSwgbGlrZSBQVkgsIHdoZW4NCj4+IHBhc3N0aHJvdWdoIGEgZGV2aWNlIHRvIGd1ZXN0
IG9uIFBWSCBkb20wLCBjYWxsc3RhY2sNCj4+IHBjaV9hZGRfZG1fZG9uZS0+WEVOX0RPTUNUTF9p
cnFfcGVybWlzc2lvbiB3aWxsIGZhaWxlZA0KPj4gYXQgZG9tYWluX3BpcnFfdG9faXJxLg0KPj4N
Cj4+IFNvLCBhZGQgYSBuZXcgaHlwZXJjYWxsIHRvIGdyYW50L3Jldm9rZSBnc2kgcGVybWlzc2lv
bg0KPj4gd2hlbiBkb20wIGlzIG5vdCBQViBvciBkb20wIGhhcyBub3QgUElSUSBmbGFnLg0KPiAN
Cj4gSG9uZXN0bHkgSSBmaW5kIHRoaXMgaGFyZCB0byBmb2xsb3csIGFuZCB0aHVzIG5vdCByZWFs
bHkgbWFraW5nIGNsZWFyIHdoeQ0KPiBubyBvdGhlciBleGlzdGluZyBtZWNoYW5pc20gY291bGQg
YmUgdXNlZC4NClNvcnJ5LCBJIHdpbGwgZGVzY3JpYmUgbW9yZSBjbGVhcmx5IGluIG5leHQgdmVy
c2lvbi4NCg0KPiANCj4+IFNpZ25lZC1vZmYtYnk6IEh1YW5nIFJ1aSA8cmF5Lmh1YW5nQGFtZC5j
b20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKaXFpYW4gQ2hlbiA8SmlxaWFuLkNoZW5AYW1kLmNvbT4N
Cj4+IC0tLQ0KPiANCj4gQmVsb3cgaGVyZSBpbiBhbiBSRkMgcGF0Y2ggeW91IHR5cGljYWxseSB3
b3VsZCB3YW50IHRvIHB1dCBzcGVjaWZpYyBpdGVtcw0KPiB5b3UncmUgc2Vla2luZyBmZWVkYmFj
ayBvbi4gV2l0aG91dCB0aGF0IGl0J3MgaGFyZCB0byB0ZWxsIHdoeSB0aGlzIGlzDQo+IG1hcmtl
ZCBSRkMuDQpPSywgSSB3aWxsIGFkZCAiIFJGQzogd2FpdCBmb3IgdGhlIHRoaXJkIHBhdGNoIG9u
IGtlcm5lbCBzaWRlIGlzIGFjY2VwdGVkLiIgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+IA0KPj4gLS0t
IGEveGVuL2FyY2gveDg2L2RvbWN0bC5jDQo+PiArKysgYi94ZW4vYXJjaC94ODYvZG9tY3RsLmMN
Cj4+IEBAIC0yMzcsNiArMjM3LDM3IEBAIGxvbmcgYXJjaF9kb19kb21jdGwoDQo+PiAgICAgICAg
ICBicmVhazsNCj4+ICAgICAgfQ0KPj4gIA0KPj4gKyAgICBjYXNlIFhFTl9ET01DVExfZ3NpX3Bl
cm1pc3Npb246DQo+PiArICAgIHsNCj4+ICsgICAgICAgIHVuc2lnbmVkIGludCBnc2kgPSBkb21j
dGwtPnUuZ3NpX3Blcm1pc3Npb24uZ3NpOw0KPj4gKyAgICAgICAgaW50IGFsbG93ID0gZG9tY3Rs
LT51LmdzaV9wZXJtaXNzaW9uLmFsbG93X2FjY2VzczsNCj4gDQo+IGJvb2w/DQpXaWxsIGNoYW5n
ZS4NCg0KPiANCj4+ICsgICAgICAgIGlmICggaXNfcHZfZG9tYWluKGN1cnJlbnQtPmRvbWFpbikg
fHwgaGFzX3BpcnEoY3VycmVudC0+ZG9tYWluKSApDQo+PiArICAgICAgICB7DQo+PiArICAgICAg
ICAgICAgcmV0ID0gLUVPUE5PVFNVUFA7DQo+PiArICAgICAgICAgICAgYnJlYWs7DQo+PiArICAg
ICAgICB9DQo+IA0KPiBTdWNoIGEgcmVzdHJpY3Rpb24gaW1vIHdhbnRzIGV4cGxhaW5pbmcgaW4g
YSBjb21tZW50Lg0KV2lsbCBhZGQgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+IA0KPj4gKyAgICAgICAg
aWYgKCBnc2kgPj0gbnJfaXJxc19nc2kgKQ0KPj4gKyAgICAgICAgew0KPj4gKyAgICAgICAgICAg
IHJldCA9IC1FSU5WQUw7DQo+PiArICAgICAgICAgICAgYnJlYWs7DQo+PiArICAgICAgICB9DQo+
PiArDQo+PiArICAgICAgICBpZiAoICFpcnFfYWNjZXNzX3Blcm1pdHRlZChjdXJyZW50LT5kb21h
aW4sIGdzaSkgfHwNCj4gDQo+IEkuZS4gYXNzdW1pbmcgSVJRID09IEdTST8gSXMgdGhhdCBhIHZh
bGlkIGFzc3VtcHRpb24gd2hlbiBhbnkgbnVtYmVyIG9mDQo+IHNvdXJjZSBvdmVycmlkZXMgbWF5
IGJlIHN1cmZhY2VkIGJ5IEFDUEk/DQpBbGwgaXJxcyBzbWFsbGVyIHRoYW4gbnJfaXJxc19nc2kg
YXJlIGdzaSwgYXJlbid0IHRoZXk/DQoNCj4gDQo+PiArICAgICAgICAgICAgIHhzbV9pcnFfcGVy
bWlzc2lvbihYU01fSE9PSywgZCwgZ3NpLCBhbGxvdykgKQ0KPiANCj4gSGVyZSBJJ20gcHJldHR5
IHN1cmUgeW91IGNhbid0IHZlcnkgd2VsbCByZS11c2UgYW4gZXhpc3RpbmcgaG9vaywgYXMgdGhl
DQo+IHZhbHVlIG9mIGludGVyZXN0IGlzIGluIGEgZGlmZmVyZW50IG51bWJlcmluZyBzcGFjZSwg
YW5kIGEgcG9zc2libGUgaG9vaw0KPiBmdW5jdGlvbiBoYXMgbm8gd2F5IG9mIGtub3dpbmcgd2hp
Y2ggb25lIGl0IGlzLiBEYW5pZWw/DQo+IA0KPj4gKyAgICAgICAgew0KPj4gKyAgICAgICAgICAg
IHJldCA9IC1FUEVSTTsNCj4+ICsgICAgICAgICAgICBicmVhazsNCj4+ICsgICAgICAgIH0NCj4+
ICsNCj4+ICsgICAgICAgIGlmICggYWxsb3cgKQ0KPj4gKyAgICAgICAgICAgIHJldCA9IGlycV9w
ZXJtaXRfYWNjZXNzKGQsIGdzaSk7DQo+PiArICAgICAgICBlbHNlDQo+PiArICAgICAgICAgICAg
cmV0ID0gaXJxX2RlbnlfYWNjZXNzKGQsIGdzaSk7DQo+IA0KPiBBcyBhYm92ZSBJJ20gYWZyYWlk
IHlvdSBjYW4ndCBhc3N1bWUgSVJRID09IEdTSS4NCj4gDQo+PiAtLS0gYS94ZW4vaW5jbHVkZS9w
dWJsaWMvZG9tY3RsLmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaA0KPj4g
QEAgLTQ0Nyw2ICs0NDcsMTMgQEAgc3RydWN0IHhlbl9kb21jdGxfaXJxX3Blcm1pc3Npb24gew0K
Pj4gIH07DQo+PiAgDQo+PiAgDQo+PiArLyogWEVOX0RPTUNUTF9nc2lfcGVybWlzc2lvbiAqLw0K
Pj4gK3N0cnVjdCB4ZW5fZG9tY3RsX2dzaV9wZXJtaXNzaW9uIHsNCj4+ICsgICAgdWludDMyX3Qg
Z3NpOw0KPj4gKyAgICB1aW50OF90IGFsbG93X2FjY2VzczsgICAgLyogZmxhZyB0byBzcGVjaWZ5
IGVuYWJsZS9kaXNhYmxlIG9mIHg4NiBnc2kgYWNjZXNzICovDQo+PiArfTsNCj4gDQo+IEV4cGxp
Y2l0IHBhZGRpbmcgcGxlYXNlLCBpbmNsdWRpbmcgYSBjaGVjayB0aGF0IGl0J3MgemVybyBvbiBp
bnB1dC4NClRoYW5rcywgSSB3aWxsIGFkZCBpbiBuZXh0IHZlcnNpb24uDQoNCj4gDQo+PiArDQo+
PiArDQo+PiAgLyogWEVOX0RPTUNUTF9pb21lbV9wZXJtaXNzaW9uICovDQo+IA0KPiBObyBkb3Vi
bGUgYmxhbmsgbGluZXMgcGxlYXNlLiBJbiBmYWN0IHlvdSB3aWxsIHdhbnQgdG8gYnJlYWsgdGhl
IGRvdWJsZSBibGFuaw0KPiBsaW5lcyBpbiBsZWFkaW5nIGNvbnRleHQsIGluc2VydGluZyBpbiB0
aGUgbWlkZGxlLg0KV2lsbCByZW1vdmUgb25lLg0KPiANCj4gSmFuDQoNCi0tIA0KQmVzdCByZWdh
cmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Fri May 17 10:51:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 10:51:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724035.1129222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vBS-0004pi-DF; Fri, 17 May 2024 10:51:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724035.1129222; Fri, 17 May 2024 10:51:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vBS-0004pb-AN; Fri, 17 May 2024 10:51:30 +0000
Received: by outflank-mailman (input) for mailman id 724035;
 Fri, 17 May 2024 10:51:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7vBR-0004ou-C3
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 10:51:29 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 69f2be96-143b-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 12:51:28 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-5231efd80f2so1955403e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 03:51:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5750d24c8c1sm1953881a12.72.2024.05.17.03.51.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 03:51:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69f2be96-143b-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715943088; x=1716547888; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=8aXRxHrsWURK+LT/SUII3cdbVAWn4GY6HQxQg8SYLM0=;
        b=De17KFxUp1T1YVJHp4OVGN9LZtO0UZf5zIR2yVuNPtN9EWQZQkH4oizM0tbW/vmBjA
         gW+dhGPar6spB+y8WPVV9C6L8f5vIVupEWnZDdvPtUvRaM7aewmlsYCvx7lHG66rFbvC
         RCSlXFJ4YiDfiVkHbDGX9A6qttU7/iSIpLox7S9AgyUnQfUFctxZNfeyrudXHl+T0P11
         9aJ3DmQn7IAzjq1NbqUxTk2sj6sQeisD05fU5gc8H+PoGsC/VtLEvFhfpMfNY3ezknDk
         X8a6PDz0vTtXrxOvx+JI8dIzulhWvUAXcq4ldxl+hs5A77lks71c5xFgZOpRbtiKRcPe
         DvTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715943088; x=1716547888;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8aXRxHrsWURK+LT/SUII3cdbVAWn4GY6HQxQg8SYLM0=;
        b=OMRIGj+D5f3V3v55WK1xfd0pizcrT0n3cTcBEzSjqkrHR1/3hae/ffcZOVT48WTeC4
         Tn0AsIQXu9TQ/xV+82v/Y7qU6/MlNRqr+8f9SEPXyEHWqbh77Be4iuCPf2emPKjObXnT
         ajOpRX1Z+u/i2bu69f/Ekd4JXSD0ncLOpuLV7jOFqclnzkkvU3tJd+t4i0j6GgakoBia
         BYbc0dkMtzJVwGUEPPWBA0vCmdSWTk9Gb8LR9cezEwN4s58YkQzE4X5ca4UPNYEmkXcu
         mRwijmnHj2z9FichjevRNZ6UiolLRlKyyHq2orAMGjfK33eoTqV31YDbtcm4zbg7WQ2x
         y3EA==
X-Forwarded-Encrypted: i=1; AJvYcCVw8LI9eD/omZhd49PqSctM/4hjsj/qXzQF/msJK6l3eZ7Vkb9QClqiLEra6/cq9aVimdUCpsfPbN3ZBztjllaUrQaLSOCwnwpALiH7wS0=
X-Gm-Message-State: AOJu0Yy5mTiGY+dqf5ZqRrf2uzlelqQTQuJ972BNkeaustcFXcr4WqgU
	17W11D45j4xIMyqgULJBncuhXD34tyvX9GBr+GO8MPxAg++4zXf0En/oRzcOrQ==
X-Google-Smtp-Source: AGHT+IGfZjbE6U2m/+wPZs6sFNF26OyW5wKHcGOalxDvshvoy2qJCMrLv7Yf5LTLZpxjSrzr2Rwxtg==
X-Received: by 2002:a05:6512:158d:b0:519:5df9:d945 with SMTP id 2adb3069b0e04-5220fb77429mr19477391e87.4.1715943088084;
        Fri, 17 May 2024 03:51:28 -0700 (PDT)
Message-ID: <b10e68e2-3279-471d-a089-c40934050737@suse.com>
Date: Fri, 17 May 2024 12:51:26 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 12:45, Chen, Jiqian wrote:
> On 2024/5/16 22:01, Jan Beulich wrote:
>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>> +        if ( gsi >= nr_irqs_gsi )
>>> +        {
>>> +            ret = -EINVAL;
>>> +            break;
>>> +        }
>>> +
>>> +        if ( !irq_access_permitted(current->domain, gsi) ||
>>
>> I.e. assuming IRQ == GSI? Is that a valid assumption when any number of
>> source overrides may be surfaced by ACPI?
> All irqs smaller than nr_irqs_gsi are gsi, aren't they?

They are, but there's not necessarily a 1:1 mapping.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:02:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:02:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724042.1129232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vLV-00072A-Ag; Fri, 17 May 2024 11:01:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724042.1129232; Fri, 17 May 2024 11:01:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vLV-000723-7W; Fri, 17 May 2024 11:01:53 +0000
Received: by outflank-mailman (input) for mailman id 724042;
 Fri, 17 May 2024 11:01:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7vLU-00071x-6Y
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 11:01:52 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060f.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dc049559-143c-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 13:01:50 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by SJ2PR12MB9242.namprd12.prod.outlook.com (2603:10b6:a03:56f::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 11:01:46 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 11:01:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc049559-143c-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WiC8dxKe+RhjJZSdM7ctyKM+TMP5RbT2UF3kXDBfeFT8E1Sp2f1CfEfPg0SWig9rset6gA9Ae0U4ymGpn6C++McjiBBHKUvwrH2WvBtvndf9CUgPSW3D1kSoAZ5m8kJEVjDwmSxM+NoyG+1g87Dk+9vZ1BMfBp3ggIZV9krA8v1eJV4me/UAgh+0Wr5cwZTK5sBRksSUMrYvZPjHtVCGg6Kvyv8MSqbRzzdIq4N9C8ULsjPCt82fj0mvC5MHVbVJzw/KH7Cmr2fDgf3eMsnKBVz6EumCEaDHKOkyqPA3h2BBRjbjVWrc44z7A8mF0zksA3diHrHOz11Fav9Upoz+tA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lXUspc/KmdqKgmU+4BNvCueklnSAmJrXNnaG2f1FFAA=;
 b=Gc1l/YMxc13wpZHl/835GVnnoVt80GN+wHha0F8+nkd6ty8u3ClbQ3nxkNyepPpugk9vfc9FqMzZTlwS6Rh82aM77DtWD+iOaBOst/9uLDtX1felTD+cznnI4iR7pJPjPKipwvj0cB+kUu+kM8fcr2nnCuQQIALe2zmI8Ao9P1Cn3For8klUdFuKZFb2/kw5aGouQ6JtBJHOcUtUJy8ev0Om7dPwGXEXSldI79wEiuJwFFRCsSvHxBPCZItwKp7i88C+KpDKtFymhd4RWhiPS21Z9+dduJ0UFlTUWZ0fqOX9fc38RufTZpnPiVlJK+xtn2tpTag0+ag6phjFP/Zsig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lXUspc/KmdqKgmU+4BNvCueklnSAmJrXNnaG2f1FFAA=;
 b=xeWw63UTCOwm5KPUWGqSvOdeLzhdcgg/x++VzJJGpw6JaqUGNblxZUKXSMlNdvJoZJKqy7UZzcMOOiWEnrFazdlF/ctPcDqnT/Huf9+f+6OjPTY6oc68rbY40h2gx2vmStjXYJCmL6VZRx67FdxFRs8x25rRXjdrS5MfqTn2sT8=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
Thread-Index:
 AQHap3bYcGtlWtrcpEyRLTK5fjq5SLGZ1UMAgAFwSAD//9GPgIAAlWQA//+DzoCAAIa3gP//hO2AABFvioA=
Date: Fri, 17 May 2024 11:01:46 +0000
Message-ID:
 <BL1PR12MB584920177282E2F65C7A3DD9E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
 <BL1PR12MB5849CCF9A890A7DA969F6474E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <8d96e75d-5ca1-4276-9ddc-b46b11949fc6@suse.com>
In-Reply-To: <8d96e75d-5ca1-4276-9ddc-b46b11949fc6@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|SJ2PR12MB9242:EE_
x-ms-office365-filtering-correlation-id: 0d3e26b7-763a-430f-6324-08dc7660be74
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|1800799015|7416005|366007|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?VXlrYWxxZmZhOS9URUx3aDF2QXhRc2h4L1p4a2RFTVgvUEEwMGkyMTZhUnRz?=
 =?utf-8?B?UnBwdzdXK3hhMEVCbXRoUmxWQXlGWjh4RUNxSy9oNm5QRTk3Yk94Qktxam95?=
 =?utf-8?B?eWlkRmRYNjZ4SXV3dW95SVVEa0ZiNzN1ZkViMWN2UWxIQS9mYVNhZWJHN2cy?=
 =?utf-8?B?eVUvYy8rbUZyUVBuNS9hRHNucHBLUFJBM1Z1OGhuNmx4c3FmUlhOV25MQWJo?=
 =?utf-8?B?MWFMUlA3S1RuRVliZW1CNlJCWWk5MjBOSGdFRE1ycHB4KzVvbkw4b2F1NnVZ?=
 =?utf-8?B?TzcwaU9zaklYZ0JIdlh4ZnlJNDNhOVpSQ095YjY0TllKc0cvdlNqNFg4bCtT?=
 =?utf-8?B?UUwzcjZaaEc1R3FSRThqVHdIRGYrVC9jWjJVYkU1RnNhUnNwMWRyMnlmcXor?=
 =?utf-8?B?NWo3M0FjTHNNaWVqQk11amNZTGxhZEpNN1JOb1ZyNDlmVmlnbGpEQkdLVFkr?=
 =?utf-8?B?YjkvMVpVUjhzY2d1TGU5QkpsSE1uS29aUG1QdzhrTzFBUFBZYVU4NzEzR1d1?=
 =?utf-8?B?bEc4OWxpYzJ1YU1SNklndG5JU2VFcFdpNWxSbTRIQ2hVZDlKU29FR2pncHcv?=
 =?utf-8?B?UXd1OXliaTRwK08vdmxTZXhHb0JoSmtsdzEreVNrTFBsODJiWGV4K2tubVk1?=
 =?utf-8?B?MW5NaURlWVA1T0ErY3hSb2FsV21Fc2hlMjdkVk12c2VxU05RUGtrSHdWS1RK?=
 =?utf-8?B?VTk2RnVTZUwxSGpUd1RkRGQ5djQ0ZXVDK0NsTXhxeTVPaUp6MkJ5M29SdU04?=
 =?utf-8?B?NjBXZ0ZCbDB4NHd3bU03ZXBQRUpzQXRkNktRdXBaWjNMc1Y3eHFER1VQWGRz?=
 =?utf-8?B?ejc0bFp2RHp0MEpvMlp6c1RiTTlPdEkvcjF1bERLanJLWDJ6dzBpL1lhSFRs?=
 =?utf-8?B?cFdzbG5MSHIreXhJc3dxbjZyR3dlTE04ZXh5MjhEWGtwd2lsNC83RTcwdkFz?=
 =?utf-8?B?SEwrcC8xOU1va1VXQUZubUJVazlTOFVackZxTmhNVkxTS1RaZFlOWkwvKzRn?=
 =?utf-8?B?eUVpcjVnSFlvbHdkR1NhZVZQdU4wU0dBV1FtdjJWQ0JxdUEzZDJUUWsvTmRi?=
 =?utf-8?B?NkloRHhWVFBIQmsxTFlSLzY5ZE5JZ3NJQkpscjcvMHJuTFhVc3MxcnloK1Fs?=
 =?utf-8?B?aVVQeWxLR2wxZTlTZkVOVkF1VDVpNCt6V0ZIRmNSSG4wT3U4YTYrOWxNZmgx?=
 =?utf-8?B?ZG5sQjdWVzMrMzlsMGJENnNqVDRtWlZpdVB4bEZwMmpmZWprSnR0Q0V4eFk5?=
 =?utf-8?B?TEdtQnRFV0QzWUdtVi9kTW9LcWtvU2czZ0RDZ1M5MlVvNHB1VnZpclVGdUtI?=
 =?utf-8?B?ak1MUEw4cTVpWm5HYVlxS2QvWEsrTlZCWWRrTllpWjh2VGp5MDIzYmFzK1M2?=
 =?utf-8?B?K0lUM0t6WHhHNHkrRStJanBQZFhtWVV1VXBFelNPQitlcGw0QmpIVU5uSXFa?=
 =?utf-8?B?Y1VxQWVYdCtORy9ETGdZRWtZaU9za3FrQjEyc1hheG10TmdjRDFTUUJ2Tytp?=
 =?utf-8?B?SGVTbHM0R2dNQ0wyb2dFcGQrbUZqUC9Uczh1Zy9TcVdNbXhnYnlqNWMvMjNC?=
 =?utf-8?B?TWpmMU9aWldRS3A0ZTR1ajdiYVpadXFsQk4wTEdlNnpjWXBJQktWUHYzR0RT?=
 =?utf-8?B?eXFrdDBEVEZlMm1jeVNBb2J2UVJseC92WXZGZC9QYTc0V2NZUm42R0d3WUJs?=
 =?utf-8?B?TEZGa3ZQSlJVQUVweGpGWlp2dldIMU5takNOVWR1NWcyazF6VmxkUC9RPT0=?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(7416005)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?T0RsWWNXaUU1a3M1YVdtNWtTcGYwdnRvNzBYVnZtQytLWGlYV3JmQ1ljbEoz?=
 =?utf-8?B?MnVVT08zSk5oc0ZDNE04azA4ZmZiMWliVkI2Q200RWhpblpxVTJhcHAxb2tD?=
 =?utf-8?B?N1g0bWs3Y2l4dzNrQVJib3BTUm4zTVZpZlhoY2FZWTNGeVFVNUI0WEZ5SVRS?=
 =?utf-8?B?cHpucnh6TXFJSzgrSVRISXJ1VkhWemk2SVNzT2NkQWJJcGRYSWYrRC8xMGJz?=
 =?utf-8?B?b2FDeVpPR3dGTGdYaTU4dmpESGFXWU04RDlMeTkwMXBZOHRnb2RHdGNpbmlx?=
 =?utf-8?B?MVVIQmtmQXdieXMxTXBoYlAwMk9xMW1wRDBEY3kvYWd2WGkzSFlCQUNkYjBT?=
 =?utf-8?B?QVhnQm1hdXNITFdBREUwdTNid0VFMVF6bUFXMDdJcFJBem4xTkYxY0JJYmVL?=
 =?utf-8?B?ZmJTZlVwcm9oUGg3N2wvRlIzbXltN0luc2VGK2J0ZnhyT01ydEN6bmF0OFZR?=
 =?utf-8?B?ZEFjVCtTTlRTVFFpc2ZMcm5Sd055cFphUzlleW1ISjJhSDQ3NmV3R1p2bVZy?=
 =?utf-8?B?RTk5YVg3NzdZWnBsRG1hK1N4bWFUaXBJZm5GdmJNbWFJTnBiNG9WZXExY3ZZ?=
 =?utf-8?B?UXkzMDJyUEQzM3BIT1RPaHJWb21BZ3JQTUZVUmIwQVBpd1hNNFEwUGU2V21l?=
 =?utf-8?B?VmNwZVNaWENNbExJVVdhZ3BRaldZZkFYK3NJOHN5cTZxR0dOTE5DclppWWFH?=
 =?utf-8?B?ZC9XSXl2ZHVTbmpiTjJaVVEweFQ4MGF4bTk2UmtQSG9POUlHSnIwQ0JVU1hR?=
 =?utf-8?B?RXRQV1FhNHdWYUI4elhTejVOTmhNQWdKUk9pcjdvUnNMdGYyKzRTMWlLQjd1?=
 =?utf-8?B?M2t0TUlBUHJ1eWRFdDlwRm9BK0RuUU1xOTl5a3FQOUVTRE1NMEc4SUV3ZlpQ?=
 =?utf-8?B?cXNFS05ERXEvS2NMM3hISmpyb0tRZFVFQ3BERFVCc2JVZWJPN1VpU0JTMzdY?=
 =?utf-8?B?aXdNd0NFZ2FPa3c4akdlbWFoQ0tpUnlwMk1wM1FubEJSWXpXMHNZVFlCdmsz?=
 =?utf-8?B?OGpLZS8zckpsYUlOdC9mcC9KbUNhcmRaL1dxSFYzRzllc2dwdlNiemRobkNl?=
 =?utf-8?B?UEl5NFl3UmVOUVNyWUtIODBNalp3WERFVFYxSEd0azloOTRmaXpTeWRZQks5?=
 =?utf-8?B?clByZ1BiNVRQWklJdjhpNVFiYjI3NnBVdWZ5VktYNEtLQVE0ZFZteFYxNWNm?=
 =?utf-8?B?U0dzaVhVWWZldUlHTHhzelNQL2IwZmlYczlXQTNCYkEyQ1Jja1YwOHFOUjU4?=
 =?utf-8?B?d0E3RDZhdXVackd3aGVpbWZtaUF6TDg3MWNXQmRNaERBKzg2ckJzWnhCbkI5?=
 =?utf-8?B?c2I0MWg5VTVJcnMrQXlzeDV4VVNSOHIxMlk5eGdvMkV4aTdGVFBLKzZqNDNo?=
 =?utf-8?B?eHE3ZFZ0d1VydUgzQlBMVFpsVkZvYkZpdnBCNTRQVmlqL2NJSVlUVnVQOUd1?=
 =?utf-8?B?TFRxdVhKOUNYdk0rWHZFTVVLNDc5dnVGU0RRYzd4N2ZBRm5oSmo1eTV1MDky?=
 =?utf-8?B?b3NvcHh6UkJFSk53M2M0MTluSFlSS3BydytxZ1BqbGd1SWpHbVJuK2h3N1ov?=
 =?utf-8?B?cGVqREZiU3VsQVpObm5NQXpvbkN1MzBjUHVic1AxV1RuQzRwc0Q4V0ZFcEdi?=
 =?utf-8?B?cStHN2NLYU53eEp4MThDS251SUludk9FTmt2ZE1pQ1Z4eUJ4ZGQwUHVkS0pk?=
 =?utf-8?B?V3c1eUpwZGZJRWtHUHgyaWFsY1lEelZKbXVQeFJGZXNvY25WSUhYV25qTGpV?=
 =?utf-8?B?MG5IQXdlbzZoR1RBSFByV3BVdnFWTVZoYjNOMDBseGRpcmFzZ3R0L3czVjlF?=
 =?utf-8?B?cEhLbFcrbUpSdllHaDhEVWc5RGVSREVNS3Q2b0VTREVIbXNkMi9PdG0yZGhz?=
 =?utf-8?B?LzZ2dnBjZm44aWI0SG0yWERaTzBDRUl4RDNVWVMreTZ5a0RJU1pjZkV2OUNZ?=
 =?utf-8?B?ZDJCWXZuNlRJaUdsZFlDbiszL2FvOXNSS1BGOVN2VCttSnhTREY2YU9Pbk5l?=
 =?utf-8?B?R1ZFRTROWXVXRDgwV1JReENZYWhJRzhXR2ZWOTYva3hQMVFGbEl0VjhWSnRp?=
 =?utf-8?B?TVcvVkUwcU5DYnIvZUE4NUJvQUVBNENBWlZDaFdYVlppeUtuVTZ3dk9IbFdu?=
 =?utf-8?Q?89q4=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <909C897AAAEB7545839542F468F90B2E@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0d3e26b7-763a-430f-6324-08dc7660be74
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 11:01:46.3587
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: v9ozi1jajwakRfHCc7Pt19vCXcHIQjs01W2Zv4efcU6Afkq4YuJKT9JqT+EGdtzEF+3D35g95lwvRYookZTpXQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9242

T24gMjAyNC81LzE3IDE4OjMxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTcuMDUuMjAyNCAx
MjowMCwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzE3IDE3OjUwLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAxNy4wNS4yMDI0IDExOjI4LCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IE9uIDIwMjQvNS8xNyAxNjoyMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+PiBPbiAx
Ny4wNS4yMDI0IDEwOjA4LCBDaGVuLCBKaXFpYW4gd3JvdGU6DQo+Pj4+Pj4gT24gMjAyNC81LzE2
IDIxOjA4LCBKYW4gQmV1bGljaCB3cm90ZToNCj4+Pj4+Pj4gT24gMTYuMDUuMjAyNCAxMTo1Miwg
SmlxaWFuIENoZW4gd3JvdGU6DQo+Pj4+Pj4+PiAgc3RydWN0IHBoeXNkZXZfcGNpX2RldmljZSB7
DQo+Pj4+Pj4+PiAgICAgIC8qIElOICovDQo+Pj4+Pj4+PiAgICAgIHVpbnQxNl90IHNlZzsNCj4+
Pj4+Pj4NCj4+Pj4+Pj4gSXMgcmUtdXNpbmcgdGhpcyBzdHJ1Y3QgZm9yIHRoaXMgbmV3IHN1Yi1v
cCBzdWZmaWNpZW50PyBJT1cgYXJlIGFsbA0KPj4+Pj4+PiBwb3NzaWJsZSByZXNldHMgZXF1YWws
IGFuZCBoZW5jZSBpdCBkb2Vzbid0IG5lZWQgc3BlY2lmeWluZyB3aGF0IGtpbmQgb2YNCj4+Pj4+
Pj4gcmVzZXQgd2FzIGRvbmU/IEZvciBleGFtcGxlLCBvdGhlciB0aGFuIEZMUiBtb3N0IHJlc2V0
IHZhcmlhbnRzIHJlc2V0IGFsbA0KPj4+Pj4+PiBmdW5jdGlvbnMgaW4gb25lIGdvIGFpdWkuIElt
byB0aGF0IHdvdWxkIGJldHRlciByZXF1aXJlIG9ubHkgYSBzaW5nbGUNCj4+Pj4+Pj4gaHlwZXJj
YWxsLCBqdXN0IHRvIGF2b2lkIHBvc3NpYmxlIGNvbmZ1c2lvbi4gSXQgYWxzbyByZWFkcyBhcyBp
ZiBGTFIgd291bGQNCj4+Pj4+Pj4gbm90IHJlc2V0IGFzIG1hbnkgcmVnaXN0ZXJzIGFzIG90aGVy
IHJlc2V0IHZhcmlhbnRzIHdvdWxkLg0KPj4+Pj4+IElmIEkgdW5kZXJzdG9vZCBjb3JyZWN0bHkg
dGhhdCB5b3UgbWVhbiBpbiB0aGlzIGh5cGVyY2FsbCBpdCBuZWVkcyB0byBzdXBwb3J0IHJlc2V0
dGluZyBib3RoIG9uZSBmdW5jdGlvbiBhbmQgYWxsIGZ1bmN0aW9ucyBvZiBhIHNsb3QoZGV2KT8N
Cj4+Pj4+PiBCdXQgaXQgY2FuIGJlIGRvbmUgZm9yIGNhbGxlciB0byB1c2UgYSBjeWNsZSB0byBj
YWxsIHRoaXMgcmVzZXQgaHlwZXJjYWxsIGZvciBlYWNoIHNsb3QgZnVuY3Rpb24uDQo+Pj4+Pg0K
Pj4+Pj4gSXQgY291bGQsIHllcywgYnV0IHNpbmNlIChhaXVpKSB0aGVyZSBuZWVkcyB0byBiZSBh
biBpbmRpY2F0aW9uIG9mIHRoZQ0KPj4+Pj4ga2luZCBvZiByZXNldCBhbnl3YXksIHdlIGNhbiBh
cyB3ZWxsIGF2b2lkIHJlbHlpbmcgb24gdGhlIGNhbGxlciBkb2luZw0KPj4+Pj4gc28gKGFuZCBh
dCB0aGUgc2FtZSB0aW1lIHNpbXBsaWZ5IHdoYXQgdGhlIGNhbGxlciBuZWVkcyB0byBkbykuDQo+
Pj4+IFNpbmNlIHRoZSBjb3JyZXNwb25kaW5nIGtlcm5lbCBwYXRjaCBoYXMgYmVlbiBtZXJnZWQg
aW50byBsaW51eF9uZXh0IGJyYW5jaA0KPj4+PiBodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9z
Y20vbGludXgva2VybmVsL2dpdC9uZXh0L2xpbnV4LW5leHQuZ2l0L2NvbW1pdC8/aD1uZXh0LTIw
MjQwNTE1JmlkPWIyNzI3MjI1MTFkNWU4YWU1ODBmMDE4MzA2ODdiOGE2YjI3MTdmMDEsDQo+Pj4+
IGlmIGl0J3Mgbm90IHZlcnkgbWFuZGF0b3J5IGFuZCBuZWNlc3NhcnksIGp1c3QgbGV0IHRoZSBj
YWxsZXIgaGFuZGxlIGl0IHRlbXBvcmFyaWx5Lg0KPj4+DQo+Pj4gQXMgYWxzbyBtZW50aW9uZWQg
Zm9yIHRoZSBvdGhlciBwYXRjaCBoYXZpbmcgYSBjb3JyZXNwb25kaW5nIGtlcm5lbCBvbmU6DQo+
Pj4gVGhlIGtlcm5lbCBwYXRjaCB3b3VsZCBpbW8gYmV0dGVyIG5vdCBiZSBtZXJnZWQgdW50aWwg
dGhlIG5ldyBzdWItb3AgaXMNCj4+PiBhY3R1YWxseSBmaW5hbGl6ZWQuDQo+PiBPSywgd2hhdCBz
aG91bGQgSSBkbyBuZXh0IHN0ZXA/DQo+PiBVcHN0cmVhbSBhIHBhdGNoIHRvIHJldmVydCB0aGUg
bWVyZ2VkIHBhdGNoIG9uIGtlcm5lbCBzaWRlPw0KPj4NCj4+Pg0KPj4+PiBPciBpdCBjYW4gYWRk
IGEgbmV3IGh5cGVyY2FsbCB0byByZXNldCBhbGwgZnVuY3Rpb25zIGluIG9uZSBnbyBpbiBmdXR1
cmUgcG90ZW50aWFsIHJlcXVpcmVtZW50LCBsaWtlIFBIWVNERVZPUF9wY2lfZGV2aWNlX3N0YXRl
X3Jlc2V0X2FsbF9mdW5jLg0KPj4+DQo+Pj4gSSBkaXNhZ3JlZS4gV2Ugc2hvdWxkbid0IGludHJv
ZHVjZSBpbmNvbXBsZXRlIHN1Yi1vcHMuIEF0IHRoZSB2ZXJ5IGxlYXN0LA0KPj4+IGlmIHlvdSB3
YW50IHRvIHN0aWNrIHRvIHRoZSBwcmVzZW50IGZvcm0sIEknZCBleHBlY3QgeW91IHRvIHN1cHBs
eSByZWFzb25zDQo+Pj4gd2h5IGRpc3Rpbmd1aXNoaW5nIGRpZmZlcmVudCByZXNldCBmb3JtcyBp
cyBub3QgbmVjZXNzYXJ5IChub3cgb3IgbGF0ZXIpLg0KPj4gT0ssIGlmIHdhbnQgdG8gZGlzdGlu
Z3Vpc2ggZGlmZmVyZW50IHJlc2V0LCBpcyBpdCBhY2NlcHRhYmxlIHRvIGFkZCBhIHBhcmFtZXRl
ciwgbGlrZSAidTggZmxhZyIsIGFuZCByZXNldCBldmVyeSBmdW5jdGlvbiBpZiBjb3JyZXNwb25k
aW5nIGJpdCBpcyAxPw0KPiANCj4gSSdtIGFmcmFpZCBhIGJvb2xlYW4gd29uJ3QgZG8sIGF0IGxl
YXN0IG5vdCBsb25nIHRlcm0uIEkgdGhpbmsgaXQgd2FudHMgdG8NCj4gYmUgYW4gZW51bWVyYXRp
b24gKGkuZS4gYSBzZXQgb2YgZW51bWVyYXRpb24tbGlrZSAjZGVmaW5lLXMpLiBBbmQganVzdCB0
bw0KPiBzdHJlc3MgaXQgYWdhaW46IFRoZSBleHRyYSBhcmd1bWVudCBpcyBfbm90XyBwcmltYXJp
bHkgZm9yIHRoZSBsb29waW5nIG92ZXINCj4gYWxsIGZ1bmN0aW9ucy4gSXQgaXMgdG8gY29udmV5
IHRoZSBraW5kIG9mIHJlc2V0IHRoYXQgd2FzIGRvbmUuIFRoZSBzaW5nbGUNCj4gdnMgYWxsIGZ1
bmN0aW9uKHMpIGFzcGVjdCBpcyBqdXN0IGEgdXNlZnVsIHNpZGUgZWZmZWN0IHRoaXMgd2lsbCBo
YXZlLg0KRG8geW91IG1lYW4sIGxpa2U6DQplbnVtIFJFU0VUX0RFVklDRV9TVEFURSB7DQoJUkVT
RVRfREVWSUNFX1NJTkdMRV9GVU5DLA0KCVJFU0VUX0RFVklDRV9BTExfRlVOQywNCglPdGhlcnMN
Cn07DQpJZiBjYWxsZXIgcGFzcyBpbiBSRVNFVF9ERVZJQ0VfU0lOR0xFX0ZVTkMsIEkgY2FsbCB3
aGF0IEkgYWRkIGluIHRoaXMgcGF0Y2gsIGFzIGZvciBvdGhlciB0eXBlcyBjYWxsIG90aGVyIGZ1
bmN0aW9ucyBpZiBhZGRlZCBpbiBmdXR1cmU/DQoNCj4gDQo+IEphbg0KDQotLSANCkJlc3QgcmVn
YXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:14:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:14:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724046.1129243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vXP-0001Qr-Bz; Fri, 17 May 2024 11:14:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724046.1129243; Fri, 17 May 2024 11:14:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vXP-0001Qk-8d; Fri, 17 May 2024 11:14:11 +0000
Received: by outflank-mailman (input) for mailman id 724046;
 Fri, 17 May 2024 11:14:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPUr=MU=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1s7vXN-0001Qe-GY
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 11:14:09 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 936b5fcb-143e-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 13:14:06 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by SJ2PR12MB7847.namprd12.prod.outlook.com (2603:10b6:a03:4d2::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Fri, 17 May
 2024 11:14:00 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7587.026; Fri, 17 May 2024
 11:14:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 936b5fcb-143e-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=A0jzGPCcEh6xapv2SNOpkO3vzirhlYomzovMcsj/0OoTHGP9rU5hcF2vO8ApLxWTLJlaLsoj/9BGQue5x6IeR4O53zVdhhWjX7XOMG+KjmLID/JljgFSEvcn9p9NYkwFURXih6on9fLD6aZ5X8fm6Zw/9+BWWJqDm+/ckxKZGQ0fZjMT9JUO8a+Pu5+sp87s/wMZF0qrXf1dO5yF7kRQKlMit1XcUFFHPajdEoBkOgY2kvsbxTylRINitTQM7eU8AFFZl2gLIeGa7+rugl6zUu5DYIz/xRvyhuGs1uXrrjQKsmw1loZ6s3/TKwUVZp+de2jpPRnJmfmDXPvw15dCMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=VQs12np3lOboD0HaA3Llz13Ng4nYVMZwoFGR1yxNgWg=;
 b=ir8PBJo8ars1YdblgPf1N1NyZiU9lKbjDbbMkGBOKk0DAPkM3j5Op4oO9V0jfURH7GyWrkr6+2m862BQnuCdVed6DfRcG954QX2sPtQ1w9maBBOMlY1y4jPydRs/njNwbAkPBHbiyG9oCtiONBZHVNQzWkTUkp0sAIgACZoM19wnYYEmMk/AvNRdFwLpPY3Pmp2mQ+ljmdhbiodbn4J/Le3aaWUrFeJ/sJx9DXE3Rgyl5k+lm8CXy0EDnIpMsR0xq34fqNvE9YRCukz7LVSz2MBZf5z/8cOhn7am9IB5+KeQgeQLMrvXJcfoV7VfQ9giyytlF8Rm8DE/ZBCGO9IJFA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VQs12np3lOboD0HaA3Llz13Ng4nYVMZwoFGR1yxNgWg=;
 b=33h7mZ6smtdCA0RWHrJcjCxRa9vezbCBkjz40ZoEpu6nofy9OcMiJ2S1VFxHv0ilm6GSX9TzzO8Ct45xLML6dAGYeFccmrQemEXsaCIi1g/g6mH0ANxS6ToqMrV9WrqomLj1qvHpgUR1BcGOCzmEtg2ZQtYb9/zincUJ6iOtt4w=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index: AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4D//4e1AIAAis+A
Date: Fri, 17 May 2024 11:14:00 +0000
Message-ID:
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
In-Reply-To: <b10e68e2-3279-471d-a089-c40934050737@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7587.026)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|SJ2PR12MB7847:EE_
x-ms-office365-filtering-correlation-id: b5dd9f43-c8a3-4ec0-ea86-08dc766273da
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|1800799015|7416005|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?Y2w5Kyt2RlZwdDJhS2V1VkwxVDA4TlVkRGpESHZJT0dJRVhoVWRQZHBWY3Bw?=
 =?utf-8?B?TDIwMXl4ZDd4NWFDZ1gvUUs5dkYxV3BaU0RHR0NEaWFpVzNxYytmVHozSER5?=
 =?utf-8?B?NTF4YVBNT08vdGFUUzEwWWFnTmh4SStCaVJxcXhiK1RNYUkwQTU4Y1NBWEVI?=
 =?utf-8?B?UkVqU3pZVE00OU1EV2lqbzI5NGhSM1V0dEdrdUt2dUxxc1hTcWVLTzlhRUJM?=
 =?utf-8?B?akVmWFU1NStkcjYxc1U1VjVSRWdxVUlCRUp1d2VjMXNHTDl4WUUzNS96cG11?=
 =?utf-8?B?OVVPazI3YmtVYk5NbVFXUkpWeUd5QVlCOFVGY3VLMTBmOTM0enlYU1lqWXJ2?=
 =?utf-8?B?amR4Y1piODJ1TC95R2dwTVJEZGY4UGZ4Vk52TU5MdG5QUFVqemdrMDh3aGk2?=
 =?utf-8?B?dGNaZWZoc0VxYStBRWgwcFZoRE40VEs0bDk5SVltT3BEOUQwUGdOZXZtOE5I?=
 =?utf-8?B?OWF4ZTgrMWJRSUF2VzdyRWdKcVhQYmtkWHV4aXNqVytic0psb0VIODNPTWxw?=
 =?utf-8?B?TXRFRWU5ZERQdFB6SVVFWi9xOGU5blE2eHVhb1JjOFlLcjViUjc1aC9oSUhz?=
 =?utf-8?B?Mmt3ekxJejhyNExiaS9PUHpkRlhnbUkyQ0dHQ1ZIUGRYbE9nQlVtRXA1eVo0?=
 =?utf-8?B?cVdyQ0dxZWxSRUtnQkk1QWZLSlBDUnlNN0pzQmRXekYvUXZBMVdldUlhZC9y?=
 =?utf-8?B?cEdYTnM2T3lBMVNpaVZsTVh2Ymc5RkF5SnBVSnJ4SG9CV2ZzeWhaNWFOWUhF?=
 =?utf-8?B?UjJwNVVmNUdRTmN3d2RSZmxYVGYwUk9iY29MZ0xKRklqcXV3a1h4TmJmbWxa?=
 =?utf-8?B?OTlCbllTaG1PcVFZdWt5eTU2QlhjNDBsdElhNnpza1hNMVZaRUZyUUh3Sit4?=
 =?utf-8?B?cEhZZzdMd3F6citGbHpCRXprZE5nYkNLeVNzdW1nVTBtOHNjYkhsaFBCbHJH?=
 =?utf-8?B?ZVU4Q2QrcUs2aFA4dkFDRjg3cXF3SzJLanYyNW1WVE9iRktKem4rcU1OUjVS?=
 =?utf-8?B?NUpOZmx0V214NDkzTm9WdjYvYTcwSlNNK1ZDTlZob3FrYTljMkhzZmdoOHdF?=
 =?utf-8?B?c0lsaHV3djd2MWlwUDVrYUIxZVdnUnRXNTY2cVZ6Mk5NVlR3SmdVUG5nRXo2?=
 =?utf-8?B?UGJHN3FLdWdtalpxM1NDNHRMTkR5V2NNbUFKNG8zejlFZDhkanVkZnErZ0F3?=
 =?utf-8?B?cXBBc1JJK0JZNlh3NElRQUltY3R0cXlmNGNsZmZTZ1lyZTF2UkEvSTVOSGw3?=
 =?utf-8?B?d3J5eUgrK1JVV1VCd0hMbnlHdmNHVTQwZHoyNmRpS2VwUTJJTWJiOXFrNjM0?=
 =?utf-8?B?Z1pUNy8wb0V1NVRqNlFUbWwyUlhsbGZCWTZzRHNoOWVKN3NjanUzcmZTSTVy?=
 =?utf-8?B?Q0xuWk5zaFhkdVNWaUMwdGZ4aWZNN3ZoSWY1WUNWSDdIbysyZzB4c0VMSC85?=
 =?utf-8?B?Zlp3SWJLSXJmeXk1blNQMUF3OGlWdTNlR3AyTkVreUJpVitTbWF4dzBlZ21M?=
 =?utf-8?B?OUhjK2pWS1FuRHJnMmZZNy8xQVZtNGVGcDBJVm5YRmwwZkxUVXV0QXN0Z0Yv?=
 =?utf-8?B?ZVlkWU9GNlhiWFNtM2dkWnpQVEtUNFp0T0JmK0J3L04vWnVUekwyczQwMWpL?=
 =?utf-8?B?MzF0azdLRVJaVGp3cDhySS9wK1IxM05DdWpRY3BYeFljdUlUL3dwOHR5aW5N?=
 =?utf-8?B?NzY3Z0ZyQVNNNGZpenhTVDliWkpybVJ2RHBtWFlHSTlmcU9ORUkwcExYQkdz?=
 =?utf-8?B?K2tTR2U5RDR5M2FsbDV0eG9LeGJiUytMTjF3ZE5JSFpwWEtJaFBPMm9Telk5?=
 =?utf-8?B?L0xQcVQ1dmxXdjBJSExSQT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(7416005)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?MTJuVmUzaFpOeGRmOVh1S3RaYWpGRVV6Wkk5d0ZST0pDWlpMRnJjTW5WK256?=
 =?utf-8?B?KzVnaHoyUG1tSDFIemx0K2NqY0RDWGcra2hCekk4aUhHRlVoYUlXalV3TUNi?=
 =?utf-8?B?dVhyWTJnd3NucGdLdjNla01jUmN6SnQ2dGhMUyswNVRob3JSSDdvMFJYeTZ4?=
 =?utf-8?B?VjltZTNSeHhyMXdycFErOEtaN0x2bmk0WVhDbjFEMytiQkJGUkU0bzA3a3E3?=
 =?utf-8?B?dWNGQmU5bnBXbmVDMVpLQTk1TlArZDV6cXhtNDVTeEFFRmdUTkRwM0E0SFR2?=
 =?utf-8?B?K2xDTTRwZTB0VEszVWdBK0MzWUVncWdyanFZZTlxVlFpKzFCdkFhSEdDWU5a?=
 =?utf-8?B?Zi9DaG43ZG8wQU5HNGVxKzFJNkJ3a3IxZWorK214Zmo2V0tOS0hzRHFXZENI?=
 =?utf-8?B?S3UvMDNldklOTU1QcTdkYmdyL09PM1EzdWhMODVKR1F0QkJPN3ZQVG53eHlZ?=
 =?utf-8?B?bi83TE1HV0Ntb0NDcSs5MkFBWmNjRlRzdlNYL1lSaG04RHdkcHovMjhZRHZM?=
 =?utf-8?B?aFVvZkpyTzRaMU5hQWdPQlpxNERTbXJTM3ZzZG1wU25oWTJvenJOQ1pKY3Vy?=
 =?utf-8?B?Y2ZNWlZPdHJjU1k3MEVtSU4xNUp1S0ZNWVZDQzVpbkptN3U2Y00xYUY4TUNm?=
 =?utf-8?B?T0ZUdklKdHZSSTNXUk1RdENEbE9PR0kxbmsyaHJHcWdpVzAyZEluNy9qakx4?=
 =?utf-8?B?S25wNGVCNE5pR2dEeVlhWkpLcEtUZFpsL3B3aVVHL1gyYUI1Y2Q2ZEViT3My?=
 =?utf-8?B?bjJ3MjlXR2krSndnZ1NuME5EclJ0RlBUQUx2MkxESG9RTk9DcDFqMjdEQzBD?=
 =?utf-8?B?TjAzaWlZRU5UTzBoZ2E5LzBFSmlDY3dBenptMG1Sc25OdFB5b1UvVG4zaDhh?=
 =?utf-8?B?aGtMTVAwSTFhb0dhLzUyQy9WRFpXQm5KVE5jd0NqN3QwZVNucmFxRnlkV09X?=
 =?utf-8?B?L1M5MGRqN2RsV0VWb2trN21LbGJPQ1V0TkNkYm5UNldLSE1PZ2wweUVSN3FL?=
 =?utf-8?B?Q2g0TjdmNjFKTWhuNDdzb2dBZjAxaGFIY2xvSkhJRjZpbWhQcUhiME8zQW85?=
 =?utf-8?B?YVBTdmsyak5ZNHhxU1RvVXJpRk0xL3VEejRlbW1MeUI0UzNDM0F5dnVjclBC?=
 =?utf-8?B?QWhLVGxLbWh0MUhFZnl1NW5sVHB6U1FyNGpSQjk5L2x6TjkyTlg5OGhuVnNY?=
 =?utf-8?B?Z0JISHE0ZnljV0NjdEF0cmVySjhEdEpRUmpzSWE3MnJVWHh6L09VVE1zRDBO?=
 =?utf-8?B?TGtaQkxST0tiTnRsdFFyK2ZPdWlvQjlsazFLdDlocTBPMS8xcUVKWEF2d0l0?=
 =?utf-8?B?YTk1NUJpZyt6Q0NsN3ViL2Noa3o4dHVuVHA2MXpGa0VNV3gyZFBwcVUyMFNq?=
 =?utf-8?B?UVJxSkxUN0Rwc2Y1blBOSXRNUUhUbHhucFRXZUFSU2ROTlp5Slk0TFpaaGZn?=
 =?utf-8?B?enNkYlM3WnpTYUM2OTVLR3pmb1JmSkEwOFRQYUdFSW5aOXkveDB2eUN3N3ha?=
 =?utf-8?B?TzdqSHdvcHYrbWZLT2lxSURieGVwdWpwQjJaUkUya3h3Q0dUQlBIL0RVUkJn?=
 =?utf-8?B?WUlJVlo1YXQweTdBVnJqMzVwWERyUytOVWNoaVMzUHlHUlJoeGJBbSt5d1Zi?=
 =?utf-8?B?MFNXSnQ4bHlpOEU2QlZwWlRidXA0MWlNN2tVL3FHcm5HOGd4T05DcWVxUzhJ?=
 =?utf-8?B?Y0orK0wraEthM053cXJleG9pM282QitNbEZzRWxGVjlJc3d6YlVoM0IxSEov?=
 =?utf-8?B?UEpZdnlSOVNyNXNzNXRoUURPSUdRM2RIRTdjd3l6R1JaZGplOUpzSlZGdGZw?=
 =?utf-8?B?L3UvL1g2c01oMDFwK0g5RDZrekpqKzlrNVRWeHBJU2FETmQvaXhJZFE3ODBC?=
 =?utf-8?B?WWVnMFVaS2ZxVThVOWhMWUoydHBkUWpiMi83eEQyTUo4cFNJU3FERWRNSVNa?=
 =?utf-8?B?T1lTTWlFbGZSYlVZZlJkcHYrYU9WLzhWQzZ5cXI1cWw4YVo3TlhsK1luNmZh?=
 =?utf-8?B?OUdQUVhZUDRkYVN4dEJkVzJnMDl1Qk84SXFHVFhkdHdhMHBJR2dHN2FUQTYz?=
 =?utf-8?B?QkZ6UU56UXJOVFpIZ1lON1R0QStEaTVhcktrbXFETmw0RUVoN2lkV1lpSWR0?=
 =?utf-8?Q?kvFQ=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <90A7216E25A1D5449BB29B33CA87ABA3@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5dd9f43-c8a3-4ec0-ea86-08dc766273da
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 11:14:00.1874
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: apk7hynPlDwbARtivwz2U4WLQFG2SKiXhiEzpKXPm4a1Uh5ZcycZVkUOAdUAiCO9/AhyJguxzmaKkEvdDLYRag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7847

T24gMjAyNC81LzE3IDE4OjUxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMTcuMDUuMjAyNCAx
Mjo0NSwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzE2IDIyOjAxLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAxNi4wNS4yMDI0IDExOjUyLCBKaXFpYW4gQ2hlbiB3cm90ZToN
Cj4+Pj4gKyAgICAgICAgaWYgKCBnc2kgPj0gbnJfaXJxc19nc2kgKQ0KPj4+PiArICAgICAgICB7
DQo+Pj4+ICsgICAgICAgICAgICByZXQgPSAtRUlOVkFMOw0KPj4+PiArICAgICAgICAgICAgYnJl
YWs7DQo+Pj4+ICsgICAgICAgIH0NCj4+Pj4gKw0KPj4+PiArICAgICAgICBpZiAoICFpcnFfYWNj
ZXNzX3Blcm1pdHRlZChjdXJyZW50LT5kb21haW4sIGdzaSkgfHwNCj4+Pg0KPj4+IEkuZS4gYXNz
dW1pbmcgSVJRID09IEdTST8gSXMgdGhhdCBhIHZhbGlkIGFzc3VtcHRpb24gd2hlbiBhbnkgbnVt
YmVyIG9mDQo+Pj4gc291cmNlIG92ZXJyaWRlcyBtYXkgYmUgc3VyZmFjZWQgYnkgQUNQST8NCj4+
IEFsbCBpcnFzIHNtYWxsZXIgdGhhbiBucl9pcnFzX2dzaSBhcmUgZ3NpLCBhcmVuJ3QgdGhleT8N
Cj4gDQo+IFRoZXkgYXJlLCBidXQgdGhlcmUncyBub3QgbmVjZXNzYXJpbHkgYSAxOjEgbWFwcGlu
Zy4NCk9oLCBzbyBkbyBJIG5lZWQgdG8gYWRkIGEgbmV3IGdzaV9jYXBzIHRvIHN0b3JlIGdyYW50
ZWQgZ3NpPw0KQW5kIERvbTAgZGVmYXVsdHMgdG8gb3duIGFsbCBnc2lzIHBlcm1pc3Npb24/DQoN
Cj4gDQo+IEphbg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:35:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:35:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724081.1129253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vrp-0005qy-3n; Fri, 17 May 2024 11:35:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724081.1129253; Fri, 17 May 2024 11:35:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7vrp-0005qr-19; Fri, 17 May 2024 11:35:17 +0000
Received: by outflank-mailman (input) for mailman id 724081;
 Fri, 17 May 2024 11:35:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7vrn-0005qh-V2; Fri, 17 May 2024 11:35:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7vrn-0004iW-Ms; Fri, 17 May 2024 11:35:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7vrn-00047Y-D3; Fri, 17 May 2024 11:35:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7vrn-00050M-CT; Fri, 17 May 2024 11:35:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TeFfm2Yp1Pfnb5fGbk8LJ7Y9kiynfBlgHweG/h6ivtg=; b=E/frm2hDwWmPvFJLHxUxrYTmH6
	uBoBm7hA3iqrw8x+FRYY1jRd0+XgIyxxVfXqlqKTnYzNi3IEHkzu3ZXUKmz4TJQkehRzJ/bFsJ+yN
	H/5TOoJHzcNapipzYVVBvgVnM4GyKbdRlK207phMTmuY60Slt2RtMUaYSXiNP05djNbs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186023-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186023: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit2:host-ping-check-xen:fail:regression
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ea5f6ad9ad9645733b72ab53a98e719b460d36a6
X-Osstest-Versions-That:
    linux=3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 11:35:15 +0000

flight 186023 linux-linus real [real]
flight 186029 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186023/
http://logs.test-lab.xenproject.org/osstest/logs/186029/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit2 10 host-ping-check-xen fail in 186029 REGR. vs. 186018

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 186029-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186029-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 186010
 test-armhf-armhf-xl-multivcpu  8 xen-boot                     fail like 186010
 test-armhf-armhf-xl           8 xen-boot                     fail  like 186010
 test-armhf-armhf-xl-raw       8 xen-boot                     fail  like 186010
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186018
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 186018
 test-armhf-armhf-examine      8 reboot                       fail  like 186018
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186018
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186018
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                ea5f6ad9ad9645733b72ab53a98e719b460d36a6
baseline version:
 linux                3c999d1ae3c75991902a1a7dad0cb62c2a3008b4

Last test of basis   186018  2024-05-16 15:33:02 Z    0 days
Testing same since   186023  2024-05-17 01:13:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aapo Vienamo <aapo.vienamo@linux.intel.com>
  AceLan Kao <acelan.kao@canonical.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Ai Chao <aichao@kylinos.cn>
  Aleksandr Burakov <a.burakov@rosalinux.ru>
  Alex Volkov <alex@bootes.sytes.net>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Andreas Helbech Kleist <andreaskleist@gmail.com>
  Andrzej Pietrasiewicz <andrzej.p@collabora.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Andy Shevchenko <andy.shevchenko@gmail.com>
  Angelo Dureghello <adureghello@baylibre.com>
  Arend van Spriel <arend.vanspriel@broadcom.com>
  Armin Wolf <W_Armin@gmx.de>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Arvid Norlander <lkml@vorpal.se>
  Avri Altman <avri.altman@wdc.com>
  Basavaraj Natikar <Basavaraj.Natikar@amd.com>
  Bastien Curutchet <bastien.curutchet@bootlin.com>
  Ben Fradella <bfradell@netapp.com>
  Benjamin Gaignard <benjamin.gaignard@collabora.com>
  Bingbu Cao <bingbu.cao@intel.com>
  Bjorn Helgaas <bhelgaas@google.com>
  Bryan O'Donoghue <bryan.odonoghue@linaro.org>
  Changhuang Liang <changhuang.liang@starfivetech.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Dave Airlie <airlied@redhat.com>
  Dave Stevenson <dave.stevenson@raspberrypi.com>
  David E. Box <david.e.box@linux.intel.com>
  Denis Arefev <arefev@swemel.ru>
  Devarsh Thakkar <devarsht@ti.com>
  Dongliang Mu <mudongliangabcd@gmail.com>
  Erick Archer <erick.archer@outlook.com>
  Eugen Hristev <eugen.hristev@collabora.com>
  Fabio Estevam <festevam@denx.de>
  Fei Shao <fshao@chromium.org>
  Felix Qin <xiaokeqinhealth@126.com>
  Fullway Wang <fullwaywang@outlook.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Gergo Koteles <soyer@irl.hu>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  hanchunchao <hanchunchao@inspur.com>
  Hans de Goede <hdegoede@redhat.com>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Hien Huynh <hien.huynh.px@renesas.com>
  Hongju Wang <hongju.wang@intel.com>
  Ihor Matushchak <ihor.matushchak@foobox.net>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  István Váradi <ivaradi@varadiistvan.hu>
  Ivor Wanders <ivor@iwanders.net>
  Jackson Lee <jackson.lee@chipsnmedia.com>
  Jacopo Mondi <jacopo.mondi@ideasonboard.com>
  Jai Luthra <j-luthra@ti.com>
  Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
  Jeff Johnson <quic_jjohnson@quicinc.com>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jisheng Zhang <jszhang@kernel.org>
  Jithu Joseph <jithu.joseph@intel.com>
  Johnson Wang <johnson.wang@mediatek.com>
  Jonathan Bergh <bergh.jonathan@gmail.com>
  Jonathan Woithe <jwoithe@just42.net>
  Judith Mendez <jm@ti.com>
  Julia Lawall <Julia.Lawall@inria.fr>
  Julien Massot <julien.massot@collabora.com>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Kalle Valo <kvalo@kernel.org>
  Kate Hsuan <hpa@redhat.com>
  Kieran Bingham <kieran.bingham@ideasonboard.com>
  Klara Modin <klarasmodin@gmail.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee@kernel.org>
  Li Zhijian <lizhijian@fujitsu.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Luke D. Jones <luke@ljones.dev>
  Mark Pearson <mpearson-lenovo@squebb.ca>
  Martin Tůma <martin.tuma@digiteqautomotive.com>
  Matthias Schwarzott <zzam@gentoo.org>
  Matthijs Kooijman <matthijs@stdin.nl>
  Maulik Shah <quic_mkshah@quicinc.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Maxim Korotkov <korotkov.maxim.s@gmail.com>
  Maximilian Luz <luzmaximilian@gmail.com>
  Michael Bunk <micha@freedict.org>
  Michael Walle <mwalle@kernel.org>
  Miguel Ojeda <ojeda@kernel.org>
  Mikhail Rudenko <mike.rudenko@gmail.com>
  Minghao Chi <chi.minghao@zte.com.cn>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Mirela Rabulea <mirela.rabulea@nxp.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nathan Chancellor <nathan@kernel.org>
  Naushir Patuck <naush@raspberrypi.com>
  Nikita Travkin <nikita@trvn.ru>
  Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
  Nitin Joshi <njoshi1@lenovo.com>
  Oliver Neukum <oneukum@suse.com>
  Paweł Anikiel <panikiel@google.com>
  Prasad Pandit <pjp@fedoraproject.org>
  Pratyush Yadav <p.yadav@ti.com>
  Pratyush Yadav <pratyush@kernel.org>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rajeshwar R Shinde <coolrrsh@gmail.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ricardo B. Marliere <ricardo@marliere.net>
  Ricardo Ribalda <ribalda@chromium.org>
  Roman Bogoyev <roman@computercheck.com.au>
  Rouven Czerwinski <r.czerwinski@pengutronix.de>
  Sakari Ailus <sakari.ailus@linux.intel.com>
  Sean Young <sean@mess.org>
  Sebastian Fricke <sebastian.fricke@collabora.com>
  Sergey Khimich <serghox@gmail.com>
  Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
  Simon Glass <sjg@chromium.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
  Stefan Herdler <herdler@nurfuerspam.de>
  Stefan Klug <stefan.klug@ideasonboard.com>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Szilard Fabian <szfabian@bluemarch.art>
  Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Ulf Hansson <ulf.hansson@linaro.org>
  Umang Jain <umang.jain@ideasonboard.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vishnu Sankar <vishnuocv@gmail.com>
  Víctor Gonzalo <victor.gonzalo@anddroptable.net>
  Wadim Mueller <wafgo01@gmail.com>
  Weifeng Liu <weifeng.liu.z@gmail.com>
  Xi Pardee <xi.pardee@intel.com>
  yunshui <jiangyunshui@kylinos.cn>
  Zheyu Ma <zheyuma97@gmail.com>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 9993 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724087.1129263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w0C-0008E1-Ug; Fri, 17 May 2024 11:43:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724087.1129263; Fri, 17 May 2024 11:43:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w0C-0008Du-Rh; Fri, 17 May 2024 11:43:56 +0000
Received: by outflank-mailman (input) for mailman id 724087;
 Fri, 17 May 2024 11:43:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=30RX=MU=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1s7w0B-0008Do-Td
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 11:43:55 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bc66d791-1442-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 13:43:53 +0200 (CEST)
Received: from S036ANL.actianordic.se (10.12.31.117) by S036ANL.actianordic.se
 (10.12.31.117) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 17 May
 2024 13:43:52 +0200
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.039; Fri, 17 May 2024 13:43:52 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc66d791-1442-11ef-b4bb-af5377834399
From: John Ernberg <john.ernberg@actia.se>
To: Juergen Gross <jgross@suse.com>, Greg Kroah-Hartman
	<gregkh@linuxfoundation.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-usb@vger.kernel.org" <linux-usb@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, John Ernberg
	<john.ernberg@actia.se>, "stable@vger.kernel.org" <stable@vger.kernel.org>
Subject: [PATCH] USB: xen-hcd: Traverse host/ when CONFIG_USB_XEN_HCD is
 selected
Thread-Topic: [PATCH] USB: xen-hcd: Traverse host/ when CONFIG_USB_XEN_HCD is
 selected
Thread-Index: AQHaqE99M70NH7RzN0K31Fykj7Mi3Q==
Date: Fri, 17 May 2024 11:43:52 +0000
Message-ID: <20240517114345.1190755-1-john.ernberg@actia.se>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: git-send-email 2.45.0
x-originating-ip: [10.12.12.11]
x-esetresult: clean, is OK
x-esetid: 37303A29059A2F57607765
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

If no other USB HCDs are selected when compiling a small pure virutal
machine, the Xen HCD driver cannot be built.

Fix it by traversing down host/ if CONFIG_USB_XEN_HCD is selected.

Fixes: 494ed3997d75 ("usb: Introduce Xen pvUSB frontend (xen hcd)")
Cc: stable@vger.kernel.org # v5.17+
Signed-off-by: John Ernberg <john.ernberg@actia.se>
---
 drivers/usb/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/usb/Makefile b/drivers/usb/Makefile
index 3a9a0dd4be70..949eca0adebe 100644
--- a/drivers/usb/Makefile
+++ b/drivers/usb/Makefile
@@ -35,6 +35,7 @@ obj-$(CONFIG_USB_R8A66597_HCD)	+=3D host/
 obj-$(CONFIG_USB_FSL_USB2)	+=3D host/
 obj-$(CONFIG_USB_FOTG210_HCD)	+=3D host/
 obj-$(CONFIG_USB_MAX3421_HCD)	+=3D host/
+obj-$(CONFIG_USB_XEN_HCD)	+=3D host/
=20
 obj-$(CONFIG_USB_C67X00_HCD)	+=3D c67x00/
=20
--=20
2.45.0


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:49:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:49:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724094.1129274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w58-0000Pq-Gq; Fri, 17 May 2024 11:49:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724094.1129274; Fri, 17 May 2024 11:49:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w58-0000Pj-Cm; Fri, 17 May 2024 11:49:02 +0000
Received: by outflank-mailman (input) for mailman id 724094;
 Fri, 17 May 2024 11:49:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7w57-0000Pd-Ko
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 11:49:01 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 72ba905e-1443-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 13:48:59 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51f99f9e0faso2152125e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 04:48:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfc2fsm11921323a12.39.2024.05.17.04.48.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 04:48:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72ba905e-1443-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715946539; x=1716551339; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=2hxd3kr1Tye406izSZ3XG4RVkVzjFpjePoPxuGJgKA8=;
        b=S/vCfZWuAyo+0FXC3uzmeZWbg+pms5WPBhGgVMxN9EBvmVmfkGHTBCpczkrvccdA3N
         bgIl+Yv+iXM6nvDG4xhmGzLMO90KQpv0iFZCYXVrm5PRws+yzeZrP3U42XoNafc3/WoS
         3fm7nHj6Aip2jXxj9KJKnKDQ3IENvrPFxXogpYufqYY4lmLbHNHdeTlUZBhN/7fJZ78Q
         1Qh8Eu+f4UnZEJ5fF5Six5PQosNIg1PZMsbPJMnUOn48OvMRY5+u5JLu28P8YY0pUG5q
         lJZihTsjxafGDssBBBnK34IimTBNZs7w7uBNCyyTozLJL9xyhpgMuwkLJkf9hkZj+IY6
         lKNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715946539; x=1716551339;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2hxd3kr1Tye406izSZ3XG4RVkVzjFpjePoPxuGJgKA8=;
        b=wnG7GJfJgdg2vJRBadZFrmJ+bf7kqdSp7bwXcL6we+v6XxSCXDr2/vBjpKzSvB+fMR
         SZZZLrqmI6VDeWKMBPPuQrg6mILYhw54PL9ZoK3tXwJafgtkFdohVsDG2Li/vd1qlnJm
         VoyfWC3fJJaSpt543Z30YuJA+6Sxf+g3VEGRtoHB8ARni6UPeIWOqel6k9mfiHblewOJ
         +iBHUtb90rP3ba1Tgfp1eqFaJ+AMIrn4YcjVDcmq75mmAG5h6P26en6NBX2KWHF6UwDe
         4fPEkoXo4HXLi3QWZpUEzPFgZ7Eh1cAODuROB6pWjQ4s8z5MPSmErr55saeQ3ucp6ebG
         /zBg==
X-Forwarded-Encrypted: i=1; AJvYcCVssQf8CLPI3KAkMRxG427t8lR1ki5EhCzQ5fRlKucZROvhlNNMOaegbnAsXbdkP2f4GBIBOCeARYDMX980FgKh+354m+8jdTAlCGEhcH0=
X-Gm-Message-State: AOJu0YyfmNo/q/CcuqREVtsl9317Z3vPMZ25a7jYS6E8OkkcAm9vii6l
	jGSm7ncFF6Uywb2ezREedfJ/NkShKf1MJ1zrEfYssnBiVi/qhGxVYAyezowIvQ==
X-Google-Smtp-Source: AGHT+IFEGPBKyE6k0AHMBYvNzd4mgAcSiH/hwvlZJo4oDjAPNAtlWf6g97ybINwFJy/c1YMd6QQfww==
X-Received: by 2002:a05:6512:3117:b0:51e:1bed:13b3 with SMTP id 2adb3069b0e04-52210174834mr14898912e87.32.1715946538774;
        Fri, 17 May 2024 04:48:58 -0700 (PDT)
Message-ID: <852fc4fd-0661-45d5-b0c2-6163c6d8984c@suse.com>
Date: Fri, 17 May 2024 13:48:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <cdcbb04d-72e8-4555-a67f-9fd28c5c7743@suse.com>
 <BL1PR12MB5849447757191C93B77D19E6E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <0cc1e273-2c42-40ea-a02a-ef8702efc030@suse.com>
 <BL1PR12MB5849CCF9A890A7DA969F6474E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <8d96e75d-5ca1-4276-9ddc-b46b11949fc6@suse.com>
 <BL1PR12MB584920177282E2F65C7A3DD9E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB584920177282E2F65C7A3DD9E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 13:01, Chen, Jiqian wrote:
> On 2024/5/17 18:31, Jan Beulich wrote:
>> On 17.05.2024 12:00, Chen, Jiqian wrote:
>>> On 2024/5/17 17:50, Jan Beulich wrote:
>>>> On 17.05.2024 11:28, Chen, Jiqian wrote:
>>>>> On 2024/5/17 16:20, Jan Beulich wrote:
>>>>>> On 17.05.2024 10:08, Chen, Jiqian wrote:
>>>>>>> On 2024/5/16 21:08, Jan Beulich wrote:
>>>>>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>>>>>  struct physdev_pci_device {
>>>>>>>>>      /* IN */
>>>>>>>>>      uint16_t seg;
>>>>>>>>
>>>>>>>> Is re-using this struct for this new sub-op sufficient? IOW are all
>>>>>>>> possible resets equal, and hence it doesn't need specifying what kind of
>>>>>>>> reset was done? For example, other than FLR most reset variants reset all
>>>>>>>> functions in one go aiui. Imo that would better require only a single
>>>>>>>> hypercall, just to avoid possible confusion. It also reads as if FLR would
>>>>>>>> not reset as many registers as other reset variants would.
>>>>>>> If I understood correctly that you mean in this hypercall it needs to support resetting both one function and all functions of a slot(dev)?
>>>>>>> But it can be done for caller to use a cycle to call this reset hypercall for each slot function.
>>>>>>
>>>>>> It could, yes, but since (aiui) there needs to be an indication of the
>>>>>> kind of reset anyway, we can as well avoid relying on the caller doing
>>>>>> so (and at the same time simplify what the caller needs to do).
>>>>> Since the corresponding kernel patch has been merged into linux_next branch
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240515&id=b272722511d5e8ae580f01830687b8a6b2717f01,
>>>>> if it's not very mandatory and necessary, just let the caller handle it temporarily.
>>>>
>>>> As also mentioned for the other patch having a corresponding kernel one:
>>>> The kernel patch would imo better not be merged until the new sub-op is
>>>> actually finalized.
>>> OK, what should I do next step?
>>> Upstream a patch to revert the merged patch on kernel side?
>>>
>>>>
>>>>> Or it can add a new hypercall to reset all functions in one go in future potential requirement, like PHYSDEVOP_pci_device_state_reset_all_func.
>>>>
>>>> I disagree. We shouldn't introduce incomplete sub-ops. At the very least,
>>>> if you want to stick to the present form, I'd expect you to supply reasons
>>>> why distinguishing different reset forms is not necessary (now or later).
>>> OK, if want to distinguish different reset, is it acceptable to add a parameter, like "u8 flag", and reset every function if corresponding bit is 1?
>>
>> I'm afraid a boolean won't do, at least not long term. I think it wants to
>> be an enumeration (i.e. a set of enumeration-like #define-s). And just to
>> stress it again: The extra argument is _not_ primarily for the looping over
>> all functions. It is to convey the kind of reset that was done. The single
>> vs all function(s) aspect is just a useful side effect this will have.
> Do you mean, like:
> enum RESET_DEVICE_STATE {
> 	RESET_DEVICE_SINGLE_FUNC,
> 	RESET_DEVICE_ALL_FUNC,
> 	Others
> };

No. What we need to be able to tell apart in the hypervisor is (at least)
FLR and Conventional Reset. I can't tell right away whether the sub-forms
of the latter also may need telling apart. I expect you to dive into that
and make a good proposal.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 11:50:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 11:50:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724098.1129283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w6I-0002GP-SO; Fri, 17 May 2024 11:50:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724098.1129283; Fri, 17 May 2024 11:50:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7w6I-0002GI-PJ; Fri, 17 May 2024 11:50:14 +0000
Received: by outflank-mailman (input) for mailman id 724098;
 Fri, 17 May 2024 11:50:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bw27=MU=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s7w6H-0002Dg-Ma
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 11:50:13 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9e777f7a-1443-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 13:50:12 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5727dc6d3edso4833443a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 04:50:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfc2fsm11921323a12.39.2024.05.17.04.50.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 May 2024 04:50:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e777f7a-1443-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1715946612; x=1716551412; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IyMP6548kp/KKX7EzB6pG8oTFh39aQ5ntVpckr2G3v0=;
        b=PJl7UYEdZ5OJnfx+fnwWyzUdG4R8ny4LPflcEH9OHa9x+Goovb+9xNRnM+xD2li2hH
         cX6t0LtrW8LSD/RMQd/sQJuZ7Na5NUc3i0+d9QTnkHqFQC59qiT3bQsoQ64BmI2mmh9N
         Ba9CZHfRyZcwAwNVdEjT8KrZ4fQuoRTrKVqm+eP5EFjDJ9Zd2gPzQt2oWvykuuotJe6+
         PW19+RLE1V0hGfCK0pxrR9XIk+cW8PlFmt4twfVZKdfqgtvxUKkCsrhAwGEHaCZRCTeu
         srven03VMZxsw/bcbbLTj9E+PcuibAe6lCnqkg+IVnH2Pc0sDCXhR40ObekNG/oCeIUQ
         DXwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715946612; x=1716551412;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IyMP6548kp/KKX7EzB6pG8oTFh39aQ5ntVpckr2G3v0=;
        b=mtFzHWcjZOWB2ioRCOxK4zoRHXkeihuh7cZfk4USkY4gokTiNkqc62bGO4figGcGja
         5Firpk1U34oSAiZr000DwfWG+dp5GVzFWOBp80Uhfofq3Wb0PNDdiZhCYiYfyhQ69SWJ
         JSIBscYvzrfB3DLKLkeMeKMqhwDZb2PMI88CsVeliw2s7+6LkftOK7kkiTRnJMOE5T+H
         nEIS46fyHc2CZrzyPBI1Yt0E+t0rD74gVgcme9UNBM5R/8vKEDprh4Wl1vH4yzbwIuwc
         jWaG41q3csvWks66erv6XtNepZIJT+HXYAccUu6aBScJa1ib/sz0IfB9rn79JVipRDGS
         0bjw==
X-Forwarded-Encrypted: i=1; AJvYcCVPX3lkmnQ2LSM5A/yOvVeUNJjaXAb8YRkjQt3pYG7CCYCW1eXFqCgJtei1dQcect+gs7h7+OyKg1XUoimroQzYYwTpF3Sw8NWb4hePtNc=
X-Gm-Message-State: AOJu0YzeblqnxNoVh2ir1GMV7P6BoWtnqTfJ+a7xZ8gqRoFEw0z7mCwz
	78gtJxuzxPFbrqRAbN9GSxRfEr3inPB+dLdbp2PQMqd+OJm+YBR6GfFvikySnA==
X-Google-Smtp-Source: AGHT+IGNA10RwLs/G/b4L042DFQgtEzoBVguDiaE5VpPciMK5mUSj+SEel/KPDeF9MHMevRIxfBfcA==
X-Received: by 2002:a05:6402:3449:b0:574:d0ae:568b with SMTP id 4fb4d7f45d1cf-574d0ae5745mr8897722a12.38.1715946612162;
        Fri, 17 May 2024 04:50:12 -0700 (PDT)
Message-ID: <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
Date: Fri, 17 May 2024 13:50:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Content-Language: en-US
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 13:14, Chen, Jiqian wrote:
> On 2024/5/17 18:51, Jan Beulich wrote:
>> On 17.05.2024 12:45, Chen, Jiqian wrote:
>>> On 2024/5/16 22:01, Jan Beulich wrote:
>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>> +        if ( gsi >= nr_irqs_gsi )
>>>>> +        {
>>>>> +            ret = -EINVAL;
>>>>> +            break;
>>>>> +        }
>>>>> +
>>>>> +        if ( !irq_access_permitted(current->domain, gsi) ||
>>>>
>>>> I.e. assuming IRQ == GSI? Is that a valid assumption when any number of
>>>> source overrides may be surfaced by ACPI?
>>> All irqs smaller than nr_irqs_gsi are gsi, aren't they?
>>
>> They are, but there's not necessarily a 1:1 mapping.
> Oh, so do I need to add a new gsi_caps to store granted gsi?

Probably not. You ought to be able to translate between GSI and IRQ,
and then continue to record in / check against IRQ permissions.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 17 13:16:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:16:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724130.1129292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xR9-0008Vb-Li; Fri, 17 May 2024 13:15:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724130.1129292; Fri, 17 May 2024 13:15:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xR9-0008VU-JA; Fri, 17 May 2024 13:15:51 +0000
Received: by outflank-mailman (input) for mailman id 724130;
 Fri, 17 May 2024 13:15:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s7xR8-0008VO-2D
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:15:50 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 925907ee-144f-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:15:48 +0200 (CEST)
Received: from CH2PR07CA0045.namprd07.prod.outlook.com (2603:10b6:610:5b::19)
 by IA0PR12MB8895.namprd12.prod.outlook.com (2603:10b6:208:491::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Fri, 17 May
 2024 13:15:43 +0000
Received: from DS3PEPF0000C380.namprd04.prod.outlook.com
 (2603:10b6:610:5b:cafe::7f) by CH2PR07CA0045.outlook.office365.com
 (2603:10b6:610:5b::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34 via Frontend
 Transport; Fri, 17 May 2024 13:15:43 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF0000C380.mail.protection.outlook.com (10.167.23.10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 13:15:42 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 08:15:42 -0500
Received: from [172.18.62.51] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 08:15:41 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 925907ee-144f-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kJK8/fj8iiF7XhBBX/FwaN4cyGcVwWO6DzH0kPWX6qdHPvfsmPX1xIITsLyAczrdJLci4JrKwfeG5J2yGL4OLXNPp1vCgc6gjTlnBRYVo4mDsMJHZ6FwzwVBI+nUhRQrJywrIg1NJBWo82LNoZ/bFQUpSZHQZy0++jYCmOXATsUviUVv/gO8B4grJN/jfEhjSC8yhFGoJSu+rG3a2WhHdR55qW7LmYNGC+YFD9NgN8txxj/xoHm89KrXb0uYpOpOgjrzdGUNTEJUFQIMLgraCBIt/+K82+4KIABYZ00A8myQPzyaWNUkFjdDku7rrPdkuXS8q5YjAAgydrRkG0WKvg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8TcHqf/2wKc5G2hwPJnRH1JFlS15hzlrRu4ufDXl3Gs=;
 b=HdQmliV1e/BF6uG23JYhW/cnkg61YeLkPf8ye/l365ddC+dRTx0ZawEKWdsIKJyRqoHyVha0+Nf5vQTPmFoxWCaaVrawneSc6oenHiEvtPx57PeaYGrDoakJyp/cpLqW9v/3mT+NntjdODrioJCEIkhg4VFNrSfi6ziyBiSm8Pw9b79eBtYwbp5X/mayBJzQ7oXHmN8E3fB3TiznsYm8mpHlVI83uodosipFwvTIUoE/XlhlTuJ+ztv2I/LAzsHE1jrUM6v0cp4BKiLSP5xi+aizxq6AXl11tSjXySdZf47UjpE0csQl24ZuHhAE4wpZ1pxk4KK1SThoJEVx/3Y5qQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8TcHqf/2wKc5G2hwPJnRH1JFlS15hzlrRu4ufDXl3Gs=;
 b=bzGlLU11ifJbIVkQMGBd5pmfA7EMolHh4lTm1To0nE2yzJ6iGtCClD/wW63eem4ppy7yUSqFYBVjO+7+Eoq/AcdBRNpjjTnZ2EWfbSZSoOwdjnnnn/p70esxK9U+UMnVNiezLL0F/S7j1Gng1XRmx46MwR0ljRo5GFVqkfPznyg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <48901bbd-40ed-4783-992d-9f9ec6a81a24@amd.com>
Date: Fri, 17 May 2024 09:15:35 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v8 1/5] xen/vpci: Clear all vpci status of device
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>, Jan Beulich <jbeulich@suse.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-2-Jiqian.Chen@amd.com>
 <fb708441-b302-4727-8131-62435a54d99c@suse.com>
 <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <BL1PR12MB5849FCB53BDAB6968536E5A5E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF0000C380:EE_|IA0PR12MB8895:EE_
X-MS-Office365-Filtering-Correlation-Id: 498efe6c-6220-49f4-047a-08dc767374a4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|7416005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?azVabXIxSm1XaVUzWGpCckptY0k3aVVNQURtdEZXQ3lXYlVpMG5pQTJIVkxW?=
 =?utf-8?B?eXBEUWV3LzNFY1hxWldZWEhGQ2dNODNyRGRHMGNYVDRnQjkxbzV4U1BiVThL?=
 =?utf-8?B?ZHJLTGhIYkgvVXBMR2RZRXBjZ1B3WlJnb1llbGNIcEZ5VVJzMlZBVk5rMUd2?=
 =?utf-8?B?S1crVStJTGJ2aGg1TWdBRjBvbzRxM0V3ZzJxem1RdDVlYVpRWi9uOXd2MUxC?=
 =?utf-8?B?bWlKRGkwSkNLVlJwZ0c2VHdvODNGQ3lGM1diUkVGUFF3bHJhUnpwYUNZb29x?=
 =?utf-8?B?b0VUM25kN0dBd0tSTUgyZE9YS3Q5OU0rc3JZa3J4aEpHdU9QOUhDSlc4NXJN?=
 =?utf-8?B?U3lrYzY1YWtJbnkrYVBTMVBZM0lsZmZaSFJBTCtCNmZseXdWcll6aVVyTWdo?=
 =?utf-8?B?MDNDdEhId2djZ2FKNUYyUHgzeGM1RTdPSWtSMUx5RDAzVFNleU5nS21mWDM4?=
 =?utf-8?B?UG5rcXhGMGIwdUYwU251QlEyWkdERFR0SXYxY25FM2VxQ0FjRUpvMWdhbUh0?=
 =?utf-8?B?enNUdW0xYmtqWFBSMDJ3NTlkN2gzVmxodU5wZ2tGaHZhMlFOS3pqaEI1elZK?=
 =?utf-8?B?d291WExmWGNNQy9sUE9iTDR1R0syVWtzTWJMMWZNRVVkb09JNEpLNlhmVmpm?=
 =?utf-8?B?dFNjQlFER0pXNE12UG5hV01DcGt4U1BRMEFsL3FUY1pEZUNRNkJlaFB4T2Jo?=
 =?utf-8?B?VXRsNFpQcExDeGhYVVkvYjlPVUNTUFdvaHE0dXB5MTM4OUFnMkt2TklCcDh2?=
 =?utf-8?B?eld0UEZCbW9CK0NqQlMrRHJCR1YyTVJuc3dCRmxqUUFaaDZ3STRMdXNwWis4?=
 =?utf-8?B?eEhpZEJ6a3dhTGdpZzNpTWtlc2ZkL01XUDNsVkZUWGdIby9xNWxVdTZtZWRz?=
 =?utf-8?B?V1Y0NVcrYllqamFGQ3V1YzVUQ1k5RGxkQ1F0SHdXdnEvcVRDWmw4cDRzcVgx?=
 =?utf-8?B?Sm9maW9pZktTRTU1Tlo3NmYzSFpYU2YwSTBERWVPc21UOWtwQ01uUVljTTNR?=
 =?utf-8?B?TWtMV2RRNVdiU1g0MzQwdlBza2lmUFZVOGg5TGIvNndlT0pWMkt6M2I3SG81?=
 =?utf-8?B?RnhSTitTWGNORkVNNHJjU1gxTUFUcStZSUR1K1lVZE1qTXZncHh1N2t1TVlG?=
 =?utf-8?B?cGZEZ2d3UENYOUVFU1JISDF1cFpHcmJSNzJYcDFDWXd5bDhPSDBEN1IyenhD?=
 =?utf-8?B?azV4K2lEaE5CM1lRZG1GZHBBTVZwRzF6VldxREhHMUhua1c0Smp5RUNYRCtW?=
 =?utf-8?B?aUk0Q1hBRTZaUFNlZS9aNnFWamxJWTJCSmhFOWplb3l2U0c2SG0vM3ZFeGtw?=
 =?utf-8?B?S3ZQK1Rhb2NmZkhqa3RsOFpBTzFkRU92bkdCcVVvbkk0SDEwTmFKd0VUeWRY?=
 =?utf-8?B?dkMzYlA3b0N5UkxDK0FwSW43V3pVd0E2NW9OSXR6RUtnOVBXMmt5UDJGQzgy?=
 =?utf-8?B?ZUhkQnAxOGQ1bzlDSnNPVWoxOEFSU1ZZdVBjSHhHMnFsNXNTWnZLU3hEZ3dJ?=
 =?utf-8?B?ZGFKM3JwRFRHOTVXaytHYjR4cWpzRlROMWxKKzRJT0pWTkJPd09CakxHTlZt?=
 =?utf-8?B?bDdTUXdDcXpxdncwQndlQzJCS25jTlFZdEtOQkVOOVliVlh3TXY3dUlOUmRr?=
 =?utf-8?B?cE1PbFBuRUdybnZVQ0k3aGlOdFV1bTEzNEJLTDBpWkdTb2k2bW05NUlRaVJ1?=
 =?utf-8?B?NXYrSGhGd1hZQ1BUb2poVnhGRkY4bERWd3JKbkEvR21oSzZhejlJR2dLcWxK?=
 =?utf-8?B?Q01QNy9FdVAxUnUvTGZRVXJ1THYvc1M4T2NqeGdYS3pRQ2VabUlnbU5oZFBH?=
 =?utf-8?B?dDQ3YStEdW1OMmtuYWVQTS9XK3JKWEc1T1NTYm5TTXJGa2pVckRqdmgyZkFK?=
 =?utf-8?Q?EyfAKHB5qVIjs?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(7416005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 13:15:42.9209
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 498efe6c-6220-49f4-047a-08dc767374a4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF0000C380.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8895

On 5/17/24 04:08, Chen, Jiqian wrote:
> On 2024/5/16 21:08, Jan Beulich wrote:
>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>> @@ -67,6 +68,41 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>> +        pcidevs_lock();
>>> +        pdev = pci_get_pdev(NULL, sbdf);
>>> +        if ( !pdev )
>>> +        {
>>> +            pcidevs_unlock();
>>> +            ret = -ENODEV;
>>> +            break;
>>> +        }
>>> +
>>> +        write_lock(&pdev->domain->pci_lock);
>>> +        ret = vpci_reset_device_state(pdev);
>>> +        write_unlock(&pdev->domain->pci_lock);
>>> +        pcidevs_unlock();
>>
>> Can't this be done right after the write_lock()?
> You are right, I will move it in next version.

So that we are clear on the proposed order of calls here:

+        write_lock(&pdev->domain->pci_lock);
+        pcidevs_unlock();
+        ret = vpci_reset_device_state(pdev);
+        write_unlock(&pdev->domain->pci_lock);

>>> --- a/xen/drivers/vpci/vpci.c
>>> +++ b/xen/drivers/vpci/vpci.c
>>> @@ -115,6 +115,16 @@ int vpci_assign_device(struct pci_dev *pdev)
>>>  
>>>      return rc;
>>>  }
>>> +
>>> +int vpci_reset_device_state(struct pci_dev *pdev)
>>> +{
>>> +    ASSERT(pcidevs_locked());
>>
>> Is this necessary for ...
>>
>>> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
>>> +
>>> +    vpci_deassign_device(pdev);
>>> +    return vpci_assign_device(pdev);
>>
>> ... either of these calls? Both functions do themselves only have the
>> 2nd of the asserts you add.
> I checked codes again; I will remove the two asserts here in next version.

Nit: I think it's okay to keep the
ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:34:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:34:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724136.1129303 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjO-0003lT-6s; Fri, 17 May 2024 13:34:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724136.1129303; Fri, 17 May 2024 13:34:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjO-0003lM-3c; Fri, 17 May 2024 13:34:42 +0000
Received: by outflank-mailman (input) for mailman id 724136;
 Fri, 17 May 2024 13:34:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7xjM-0003lB-8J
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:34:40 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3573f61c-1452-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:34:39 +0200 (CEST)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4202c1d19d5so984825e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:34:39 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fe1ab43casm161950035e9.1.2024.05.17.06.34.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:34:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3573f61c-1452-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715952878; x=1716557678; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yQiIH5EjccvKwupxVBx+YRsXJvqYn8GGNADJa6shblg=;
        b=lofsjYPFERb+66MY2PR+RTMnlLPxp2C5u0MbyuE26nv+4fMqbd0YBqrzB3FplcJEyS
         Fo4koxDS41uXPJLhzeLYfwI/rw7k+WuIhZoXDpIR6+YULfdSj07KVy50GPMgKlpS6aQO
         //2i/cOj8ciYfHYZ/XKKqMbZolM7s9rwzTCGs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715952878; x=1716557678;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=yQiIH5EjccvKwupxVBx+YRsXJvqYn8GGNADJa6shblg=;
        b=liVH9LxJCP+jBs2qijWzHv6lNrs9P5hhA0Xo6LStdwU/2yzUAvT/uyBU17mcZ+h8ee
         240i/61W6IpwmLa4wwik0hAEbo4pbOXv7eq8kbu/cj/hbKZKJ+M/x3KXuCmYTBPA+IvC
         01nizkGAP3J920x8Tg9LAVFl05Y04iba9djQsCBs0QzuLnCRFelLGI9AESHRj4qTTiwT
         eL58v94GWZ/Ht56c1s8UmK/cdEdf+O/yclBPGy3k22MNXH3lkHwFqwlILekabVnOzsu8
         cRiM9/s1eruxopGmC7Qs1aKkp/WXb3JlmqFrph+Lj9/RbarqZED6OLKPG7TVYqzxOohn
         lE0w==
X-Gm-Message-State: AOJu0YxsyYmzEWYTUebKY4tfvEdClMdUh5ptX2nehEPAHbsakIqQwhmh
	qqJSpGhUIJthY88XEnWa+C7hgfWlDapPhWI4FqmeNmCIBMHswY+Xdd8eVIMq0BqtkxLC7AdpznZ
	U
X-Google-Smtp-Source: AGHT+IG18ojI2l0jkzFxiCmjvzTqfP3VYfHPw8NO8LsBvdHo2eufXLp2SSvdtlyWOBfeoKxbhjjJYQ==
X-Received: by 2002:a05:600c:45cb:b0:41a:3b7e:2235 with SMTP id 5b1f17b1804b1-41fea9318cbmr203511785e9.5.1715952878102;
        Fri, 17 May 2024 06:34:38 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 v3 1/3] xen/x86: account number of foreign mappings in the p2m
Date: Fri, 17 May 2024 15:33:50 +0200
Message-ID: <20240517133352.94347-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240517133352.94347-1-roger.pau@citrix.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Such information will be needed in order to remove foreign mappings during
teardown for HVM guests.

Right now the introduced counter is not consumed.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes since v1:
 - Drop max_gfn accounting.
---
 xen/arch/x86/include/asm/p2m.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 111badf89a6e..107b9f260848 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -380,6 +380,9 @@ struct p2m_domain {
         unsigned int flags;
         unsigned long entry_count;
     } ioreq;
+
+    /* Number of foreign mappings. */
+    unsigned long      nr_foreign;
 #endif /* CONFIG_HVM */
 };
 
@@ -1049,6 +1052,8 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         if ( !page_get_owner_and_reference(mfn_to_page(nfn)) )
             return -EBUSY;
 
+        p2m->nr_foreign++;
+
         break;
 
     default:
@@ -1069,6 +1074,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
             return -EINVAL;
         }
         put_page(mfn_to_page(ofn));
+        p2m->nr_foreign--;
         break;
 
     default:
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:34:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:34:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724139.1129333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjS-0004Tv-46; Fri, 17 May 2024 13:34:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724139.1129333; Fri, 17 May 2024 13:34:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjS-0004To-0P; Fri, 17 May 2024 13:34:46 +0000
Received: by outflank-mailman (input) for mailman id 724139;
 Fri, 17 May 2024 13:34:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7xjQ-0003lA-VZ
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:34:44 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 34a18ed1-1452-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:34:37 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2e2c70f0c97so7731961fa.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:34:37 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b79bc8esm21606970f8f.15.2024.05.17.06.34.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:34:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34a18ed1-1452-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715952876; x=1716557676; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=otGmcHIqdbmt5QRbpQ0jxqZf5KUMuwh1Shw5luwAmdM=;
        b=G5CATHHC1xH3HR4Usdi85fAIwvzpN5hrpiirXzVxPKBN/GYPyPLSdCqaLa7j0YiXDf
         VjZVCxoPjXRqkbX6xDwb/l8zU61ysu5m6B31yGQtB27JCz3FeHydWPqEtRbHdM/Ds4bJ
         39+TGiLd+8MtYV3nd1GLgCqqqSsyGSYcKO2x4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715952876; x=1716557676;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=otGmcHIqdbmt5QRbpQ0jxqZf5KUMuwh1Shw5luwAmdM=;
        b=vYZnUZlLexVGrA3DCS7FvQO2Fzf1miQ/NIArRyMJVklIWfLLnEOjtc+xXAXqK5kNZX
         pdtnZgjzFuhFRtIHV6StS77YVUAB1OxS8YfM1kJSI/MGDtYuPR2gnOvlJ7PXaT913YtR
         k6+tP8iUFWY58O1itgiu4enGznadwqwseGefN+QqtTXraHT6zPc800nP/nxdsYDyslWD
         Tg9AexjiFmceDwc3syhVd1klxOcyGDIgVVfmlqdA86yBhxIXv2sX2cL5JcVbXrUVvCxu
         E/LyeHrPMv1pr4umpr2mBwBYFIhu0YxpR1uirY9K1SSVPo3K/a4KZ3w3EebU7p12kny6
         aGZg==
X-Gm-Message-State: AOJu0YzQIyX5aE56HBMjNs3Z/iIqSPYOKp+oiwqSDKvOFN6mY2ol3zFR
	WsO8yTBSk7DvhZZDc0Kfl1GoQMq6HQA4YwFCZM/U5Xr3157N0AuMvN32khScSxHSLW+TgvrWjRU
	n
X-Google-Smtp-Source: AGHT+IEwP/08fN7nqJ4O4SvNTBe0joZm5SNHfjWFdvR6WKRtn/JzA+wUi1hHN4HfcDkVdXYZj+uBxw==
X-Received: by 2002:a2e:96ca:0:b0:2e3:603e:469b with SMTP id 38308e7fff4ca-2e5204ccf5cmr141259241fa.40.1715952876275;
        Fri, 17 May 2024 06:34:36 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>
Subject: [PATCH for-4.19 v3 0/3] xen/x86: support foreign mappings for HVM/PVH
Date: Fri, 17 May 2024 15:33:49 +0200
Message-ID: <20240517133352.94347-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following series attempts to solve a shortcoming of HVM/PVH guests
with the lack of support for foreign mappings.  Such lack of support
prevents using PVH based guests as stubdomains for example.

Add support in a way similar to how it's done on Arm, by iterating over
the p2m based on the maximum gfn.

Patch 2 is not strictly needed.  Moving the enablement of altp2m from an
HVM param to a create domctl flag avoids any possible race with the HVM
param changing after it's been evaluated.  Note the param can only be
set by the control domain, and libxl currently sets it at domain
create.  Also altp2m enablement is different from activation, as
activation does happen during runtime of the domain.

Thanks, Roger.

Roger Pau Monne (3):
  xen/x86: account number of foreign mappings in the p2m
  xen/x86: enable altp2m at create domain domctl
  xen/x86: remove foreign mappings from the p2m on teardown

 CHANGELOG.md                        |  1 +
 tools/libs/light/libxl_create.c     | 23 +++++++++-
 tools/libs/light/libxl_x86.c        | 26 +++++------
 tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
 xen/arch/arm/domain.c               |  6 +++
 xen/arch/x86/domain.c               | 28 ++++++++++++
 xen/arch/x86/hvm/hvm.c              | 23 +++++++++-
 xen/arch/x86/include/asm/p2m.h      | 32 +++++++++-----
 xen/arch/x86/mm/p2m-basic.c         | 18 ++++++++
 xen/arch/x86/mm/p2m.c               | 68 +++++++++++++++++++++++++++--
 xen/include/public/domctl.h         | 20 ++++++++-
 xen/include/public/hvm/params.h     |  9 +---
 12 files changed, 215 insertions(+), 41 deletions(-)

-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:34:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:34:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724137.1129306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjO-0003oO-Cl; Fri, 17 May 2024 13:34:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724137.1129306; Fri, 17 May 2024 13:34:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjO-0003nK-A8; Fri, 17 May 2024 13:34:42 +0000
Received: by outflank-mailman (input) for mailman id 724137;
 Fri, 17 May 2024 13:34:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7xjN-0003lB-Cp
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:34:41 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 367aa102-1452-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:34:40 +0200 (CEST)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2db17e8767cso8917091fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:34:40 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41f88110f3esm343070175e9.29.2024.05.17.06.34.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:34:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 367aa102-1452-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715952879; x=1716557679; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=InYh5oMvLk+F01NISv6iHX9jFgL9SL6h0cc3g+aH2ug=;
        b=RL/SaNwE+93gf9w0hnt4rJxgaTY51WtC5bIs65iHdfjFYwkJhXRx2DEWsSX9WM0hTI
         nBverXOIEyVzSQwkAbS9Onc1wFGZ/EgEnytxinJ+eNFu7Y8GdPLZVvbpIcDu1WElGpBJ
         IsqUuWSPl49hHyAIL7jwRx7/f6CFNtY3KNPCU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715952879; x=1716557679;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=InYh5oMvLk+F01NISv6iHX9jFgL9SL6h0cc3g+aH2ug=;
        b=afofppW0j65988NM1H7ujvYqXFwb0d3rNFh62ykP0e8ADYrvhH8VvMUB5YfsehDDDC
         wZxDUQJc9eLfzDLQbRySz1zdXSd6224aOa8Slc/1pxlBPPKPjNQAUYbHKSlkfgDTJxAh
         TzIsncBqjRKDqECXQj0Bk5eYZFMBjckdmxaSgRcqvYzle65weZu3PDn3AOuD63RAQAm3
         7R1DOUI9FLWPjpliKIAn9AtWTef3x1uWHtSpF47gkV2LpjzKpwFgVhsbNTNHX+PAw1F6
         jk56/7BGJsBErtUgbgECmELexix5mbrcdE5onfnJbN+4rSaSk3oEmjUlb+9atbqaY00+
         HH0g==
X-Gm-Message-State: AOJu0YwqMfliseBYg4hzv4TKo/itVHkLu1vs9v+tbFkEjUNvOVsg4Vlq
	RjWVy/iW2JU4xjuJm2vW4siMTiG2/IuQAwkL6HEG1OD/LkyweYlv3j3B9ezQfpqCp5bQ8vyqyiU
	w
X-Google-Smtp-Source: AGHT+IGQFISxfnzNfTscn7hw2MgSEuiMarnI41ybpc/VCXw6LtDUEGVSUEHws/R+y9K0LGm392G+jA==
X-Received: by 2002:a05:651c:1542:b0:2de:7046:b8f8 with SMTP id 38308e7fff4ca-2e51fd2e6a8mr223879701fa.5.1715952879322;
        Fri, 17 May 2024 06:34:39 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain domctl
Date: Fri, 17 May 2024 15:33:51 +0200
Message-ID: <20240517133352.94347-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240517133352.94347-1-roger.pau@citrix.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Enabling it using an HVM param is fragile, and complicates the logic when
deciding whether options that interact with altp2m can also be enabled.

Leave the HVM param value for consumption by the guest, but prevent it from
being set.  Enabling is now done using and additional altp2m specific field in
xen_domctl_createdomain.

Note that albeit only currently implemented in x86, altp2m could be implemented
in other architectures, hence why the field is added to xen_domctl_createdomain
instead of xen_arch_domainconfig.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Introduce a new altp2m field in xen_domctl_createdomain.

Changes since v1:
 - New in this version.
---
 tools/libs/light/libxl_create.c     | 23 ++++++++++++++++++++++-
 tools/libs/light/libxl_x86.c        | 26 ++++++++++++--------------
 tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
 xen/arch/arm/domain.c               |  6 ++++++
 xen/arch/x86/domain.c               | 22 ++++++++++++++++++++++
 xen/arch/x86/hvm/hvm.c              | 23 ++++++++++++++++++++++-
 xen/include/public/domctl.h         | 20 +++++++++++++++++++-
 xen/include/public/hvm/params.h     |  9 ++-------
 8 files changed, 106 insertions(+), 25 deletions(-)

diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 41252ec55370..edeadd57ef5a 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -372,7 +372,6 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
         libxl_defbool_setdefault(&b_info->u.hvm.viridian,           false);
         libxl_defbool_setdefault(&b_info->u.hvm.hpet,               true);
         libxl_defbool_setdefault(&b_info->u.hvm.vpt_align,          true);
-        libxl_defbool_setdefault(&b_info->u.hvm.altp2m,             false);
         libxl_defbool_setdefault(&b_info->u.hvm.usb,                false);
         libxl_defbool_setdefault(&b_info->u.hvm.vkb_device,         true);
         libxl_defbool_setdefault(&b_info->u.hvm.xen_platform_pci,   true);
@@ -678,6 +677,28 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
             create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
 
+        LOG(DETAIL, "altp2m: %s", libxl_altp2m_mode_to_string(b_info->altp2m));
+        switch(b_info->altp2m) {
+        case LIBXL_ALTP2M_MODE_MIXED:
+            create.altp2m_opts |=
+                XEN_DOMCTL_ALTP2M_mode(XEN_DOMCTL_ALTP2M_mixed);
+            break;
+
+        case LIBXL_ALTP2M_MODE_EXTERNAL:
+            create.altp2m_opts |=
+                XEN_DOMCTL_ALTP2M_mode(XEN_DOMCTL_ALTP2M_external);
+            break;
+
+        case LIBXL_ALTP2M_MODE_LIMITED:
+            create.altp2m_opts |=
+                XEN_DOMCTL_ALTP2M_mode(XEN_DOMCTL_ALTP2M_limited);
+            break;
+
+        case LIBXL_ALTP2M_MODE_DISABLED:
+            /* Nothing to do - altp2m disabled is signaled as mode == 0. */
+            break;
+        }
+
         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
 
diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
index a50ec37eb3eb..60643d6f5376 100644
--- a/tools/libs/light/libxl_x86.c
+++ b/tools/libs/light/libxl_x86.c
@@ -407,19 +407,9 @@ static int hvm_set_conf_params(libxl__gc *gc, uint32_t domid,
     libxl_ctx *ctx = libxl__gc_owner(gc);
     xc_interface *xch = ctx->xch;
     int ret = ERROR_FAIL;
-    unsigned int altp2m = info->altp2m;
 
     switch(info->type) {
     case LIBXL_DOMAIN_TYPE_HVM:
-        /* The config parameter "altp2m" replaces the parameter "altp2mhvm". For
-         * legacy reasons, both parameters are accepted on x86 HVM guests.
-         *
-         * If the legacy field info->u.hvm.altp2m is set, activate altp2m.
-         * Otherwise set altp2m based on the field info->altp2m. */
-        if (info->altp2m == LIBXL_ALTP2M_MODE_DISABLED &&
-            libxl_defbool_val(info->u.hvm.altp2m))
-            altp2m = libxl_defbool_val(info->u.hvm.altp2m);
-
         if (xc_hvm_param_set(xch, domid, HVM_PARAM_HPET_ENABLED,
                              libxl_defbool_val(info->u.hvm.hpet))) {
             LOG(ERROR, "Couldn't set HVM_PARAM_HPET_ENABLED");
@@ -444,10 +434,6 @@ static int hvm_set_conf_params(libxl__gc *gc, uint32_t domid,
             LOG(ERROR, "Couldn't set HVM_PARAM_TIMER_MODE");
             goto out;
         }
-        if (xc_hvm_param_set(xch, domid, HVM_PARAM_ALTP2M, altp2m)) {
-            LOG(ERROR, "Couldn't set HVM_PARAM_ALTP2M");
-            goto out;
-        }
         break;
 
     default:
@@ -818,6 +804,18 @@ int libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
     libxl_defbool_setdefault(&b_info->acpi, true);
     libxl_defbool_setdefault(&b_info->arch_x86.msr_relaxed, false);
 
+    /*
+     * The config parameter "altp2m" replaces the parameter "altp2mhvm".
+     * For legacy reasons, both parameters are accepted on x86 HVM guests.
+     *
+     * If the legacy field info->u.hvm.altp2m is set, activate altp2m.
+     * Otherwise set altp2m based on the field info->altp2m.
+     */
+    libxl_defbool_setdefault(&b_info->u.hvm.altp2m, false);
+    if (b_info->altp2m == LIBXL_ALTP2M_MODE_DISABLED &&
+        libxl_defbool_val(b_info->u.hvm.altp2m))
+        b_info->altp2m = libxl_defbool_val(b_info->u.hvm.altp2m);
+
     return 0;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index 2b6d3c09dfa0..c6da9bb09137 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -257,7 +257,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #if defined(__i386__) || defined(__x86_64__)
 
 		/* Quick & dirty check for ABI changes. */
-		BUILD_BUG_ON(sizeof(cfg) != 64);
+		BUILD_BUG_ON(sizeof(cfg) != 68);
 
         /* Mnemonics for the named fields inside xen_x86_arch_domainconfig */
 #define VAL_EMUL_FLAGS          Field(arch_domconfig, 0)
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 34cbfe699a68..8bde2f730dfb 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -688,6 +688,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    if ( config->altp2m_opts )
+    {
+        dprintk(XENLOG_INFO, "Altp2m not supported\n");
+        return -EINVAL;
+    }
+
     return 0;
 }
 
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 20e83cf38bbd..a89ceff9ae93 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
     bool hap = config->flags & XEN_DOMCTL_CDF_hap;
     bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
     unsigned int max_vcpus;
+    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
+                                         XEN_DOMCTL_ALTP2M_mode_mask);
 
     if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
     {
@@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
+    {
+        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
+                config->flags);
+        return -EINVAL;
+    }
+
+    if ( altp2m_mode && nested_virt )
+    {
+        dprintk(XENLOG_INFO,
+                "Nested virt and altp2m are not supported together\n");
+        return -EINVAL;
+    }
+
+    if ( altp2m_mode && !hap )
+    {
+        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
+        return -EINVAL;
+    }
+
     return 0;
 }
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c530..31f64725b725 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -659,6 +659,22 @@ int hvm_domain_initialise(struct domain *d,
 
     d->arch.hvm.params[HVM_PARAM_TRIPLE_FAULT_REASON] = SHUTDOWN_reboot;
 
+    /* Set altp2m based on domctl flags. */
+    switch ( MASK_EXTR(config->altp2m_opts, XEN_DOMCTL_ALTP2M_mode_mask) )
+    {
+    case XEN_DOMCTL_ALTP2M_mixed:
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_mixed;
+        break;
+
+    case XEN_DOMCTL_ALTP2M_external:
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_external;
+        break;
+
+    case XEN_DOMCTL_ALTP2M_limited:
+        d->arch.hvm.params[HVM_PARAM_ALTP2M] = XEN_ALTP2M_limited;
+        break;
+    }
+
     vpic_init(d);
 
     rc = vioapic_init(d);
@@ -4169,6 +4185,12 @@ static int hvm_allow_set_param(struct domain *d,
     case HVM_PARAM_CONSOLE_EVTCHN:
     case HVM_PARAM_X87_FIP_WIDTH:
         break;
+
+    /* The following parameters are read-only. */
+    case HVM_PARAM_ALTP2M:
+        rc = -EEXIST;
+        break;
+
     /* The following parameters are deprecated. */
     case HVM_PARAM_PAE_ENABLED:
     case HVM_PARAM_DM_DOMAIN:
@@ -4210,7 +4232,6 @@ static int hvm_allow_set_param(struct domain *d,
     case HVM_PARAM_BUFIOREQ_PFN:
     case HVM_PARAM_IOREQ_SERVER_PFN:
     case HVM_PARAM_NR_IOREQ_SERVER_PAGES:
-    case HVM_PARAM_ALTP2M:
     case HVM_PARAM_MCA_CAP:
         if ( value != 0 && new_value != value )
             rc = -EEXIST;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b08..e72967717e67 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -21,7 +21,7 @@
 #include "hvm/save.h"
 #include "memory.h"
 
-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
 
 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -86,6 +86,24 @@ struct xen_domctl_createdomain {
 
     uint32_t grant_opts;
 
+/*
+ * Enable altp2m mixed mode.
+ *
+ * Note that 'mixed' mode has not been evaluated for safety from a security
+ * perspective.  Before using this mode in a security-critical environment,
+ * each subop should be evaluated for safety, with unsafe subops blacklisted in
+ * XSM.
+ */
+#define XEN_DOMCTL_ALTP2M_mixed      (1U)
+/* Enable altp2m external mode. */
+#define XEN_DOMCTL_ALTP2M_external   (2U)
+/* Enable altp2m limited mode. */
+#define XEN_DOMCTL_ALTP2M_limited    (3U)
+/* Altp2m mode signaling uses bits [0, 1]. */
+#define XEN_DOMCTL_ALTP2M_mode_mask  (0x3U)
+#define XEN_DOMCTL_ALTP2M_mode(m)    ((m) & XEN_DOMCTL_ALTP2M_mode_mask)
+    uint32_t altp2m_opts;
+
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;
 
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index a22b4ed45d2e..99c40b4287f1 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -246,16 +246,11 @@
 #define HVM_PARAM_VM_GENERATION_ID_ADDR 34
 
 /*
- * Set mode for altp2m:
- *  disabled: don't activate altp2m (default)
+ * Get mode for altp2m:
+ *  disabled: altp2m not active (default)
  *  mixed: allow access to all altp2m ops for both in-guest and external tools
  *  external: allow access to external privileged tools only
  *  limited: guest only has limited access (ie. control VMFUNC and #VE)
- *
- * Note that 'mixed' mode has not been evaluated for safety from a
- * security perspective.  Before using this mode in a
- * security-critical environment, each subop should be evaluated for
- * safety, with unsafe subops blacklisted in XSM.
  */
 #define HVM_PARAM_ALTP2M       35
 #define XEN_ALTP2M_disabled      0
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:34:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:34:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724138.1129323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjP-0004EA-S0; Fri, 17 May 2024 13:34:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724138.1129323; Fri, 17 May 2024 13:34:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7xjP-0004E1-OR; Fri, 17 May 2024 13:34:43 +0000
Received: by outflank-mailman (input) for mailman id 724138;
 Fri, 17 May 2024 13:34:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7xjO-0003lB-D0
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:34:42 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 37124836-1452-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:34:41 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-34e7a35d5d4so90308f8f.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:34:41 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-351daf8aacbsm4053618f8f.5.2024.05.17.06.34.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:34:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 37124836-1452-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715952881; x=1716557681; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=cz5G6tUTw2J7tgiMDU5XGAZ4Acz+mTvXsOD7q6Ij0fU=;
        b=ThBzQg0xYYAl02TtN/17aupi8m95kfSwqFbhA/k8phPtOiApZ+4hY8xwlbYS5sWLyb
         1MR923GyyVuuSBApOHjurW8HP9fFC3TNvZFkOYpmFSPHNHamfyJZcQlLbCTaVN+qaxsb
         dp2X/ZSvwcB5BMp5b7TwUfkqs38wCyctipT1o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715952881; x=1716557681;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=cz5G6tUTw2J7tgiMDU5XGAZ4Acz+mTvXsOD7q6Ij0fU=;
        b=vOT6DxGBS+5J7d5rRZ+BAXtFH7f05qOCbiHMU9kKJoCvrqch1aLp1ixiXNySLbgdFk
         2iV8vU98mrOaIUI/Gyj02AKcZ340BIQZ4Lfhl6h3BhTrM+VWCr6l+HDvhpkHDQWi1dJ/
         bgPcKHztNIFeF3Sg+b0xjixTn5JxDVnhoUG3ryLnEV/Sx53A/oQlvB/AYH1yEm7RbVNR
         z7hzb01taVUVaUJjhQERHc21lOInGyaF6GEB2kr155rDu7fyRuackE3+Lxe+4WyzR+GB
         Ux+XbJrc5muozbDWE7ndQS/FYUITTI/whFjIvCARgBoTzel+NoUMwleGvNJM8vPJaFBD
         66LA==
X-Gm-Message-State: AOJu0YzFA4b3ZFEXufTee9tnOyeeoDpWlU/6KrQkLFISyCdEG4rLF4BK
	urGSCc022a4vvy8+wIH0Pb7rtVkgU4+Xsjg/Xp8Bpddd0VqcTyrpjDDxSPrF+UdS628RKY+ZwAU
	L
X-Google-Smtp-Source: AGHT+IGKchGSk9hZuPXcPcoqBWvz2ekAFm+dIQcZpMZC1EqlpLiDeUADq3bBc7/yMAh3RtvynUJ8Lw==
X-Received: by 2002:adf:ec48:0:b0:34d:9467:d6d7 with SMTP id ffacd0b85a97d-3504a61d3c7mr14379536f8f.9.1715952880817;
        Fri, 17 May 2024 06:34:40 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Community Manager <community.manager@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH for-4.19 v3 3/3] xen/x86: remove foreign mappings from the p2m on teardown
Date: Fri, 17 May 2024 15:33:52 +0200
Message-ID: <20240517133352.94347-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240517133352.94347-1-roger.pau@citrix.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Iterate over the p2m up to the maximum recorded gfn and remove any foreign
mappings, in order to drop the underlying page references and thus don't keep
extra page references if a domain is destroyed while still having foreign
mappings on it's p2m.

The logic is similar to the one used on Arm.

Note that foreign mappings cannot be created by guests that have altp2m or
nested HVM enabled, as p2ms different than the host one are not currently
scrubbed when destroyed in order to drop references to any foreign maps.

It's unclear whether the right solution is to take an extra reference when
foreign maps are added to p2ms different than the host one, or just rely on the
host p2m already having a reference.  The mapping being removed from the host
p2m should cause it to be dropped on all domain p2ms.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes since v2:
 - Fix arch_acquire_resource_check() condition.

Changes since v1:
 - Use existing p2m max_mapped_pfn field.
 - Prevent creating foreign mappings by guests that have altp2m or nestedhvm
   enabled.
---
 CHANGELOG.md                   |  1 +
 xen/arch/x86/domain.c          |  6 +++
 xen/arch/x86/include/asm/p2m.h | 26 +++++++------
 xen/arch/x86/mm/p2m-basic.c    | 18 +++++++++
 xen/arch/x86/mm/p2m.c          | 68 ++++++++++++++++++++++++++++++++--
 5 files changed, 103 insertions(+), 16 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index c43c45d8d4bd..59aad04e9364 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
  - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
  - Increase the maximum number of CPUs Xen can be built for from 4095 to
    16383.
+ - Allow HVM/PVH domains to map foreign pages.
 
 ### Added
  - On x86:
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index a89ceff9ae93..de5f6b3212fb 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -2386,6 +2386,7 @@ int domain_relinquish_resources(struct domain *d)
         enum {
             PROG_iommu_pagetables = 1,
             PROG_shared,
+            PROG_mappings,
             PROG_paging,
             PROG_vcpu_pagetables,
             PROG_xen,
@@ -2434,6 +2435,11 @@ int domain_relinquish_resources(struct domain *d)
         }
 #endif
 
+    PROGRESS(mappings):
+        ret = relinquish_p2m_mapping(d);
+        if ( ret )
+            return ret;
+
     PROGRESS(paging):
 
         /* Tear down paging-assistance stuff. */
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 107b9f260848..c1478ffc3647 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -383,6 +383,8 @@ struct p2m_domain {
 
     /* Number of foreign mappings. */
     unsigned long      nr_foreign;
+    /* Cursor for iterating over the p2m on teardown. */
+    unsigned long      teardown_gfn;
 #endif /* CONFIG_HVM */
 };
 
@@ -395,16 +397,7 @@ struct p2m_domain {
 #endif
 #include <xen/p2m-common.h>
 
-static inline bool arch_acquire_resource_check(struct domain *d)
-{
-    /*
-     * FIXME: Until foreign pages inserted into the P2M are properly
-     * reference counted, it is unsafe to allow mapping of
-     * resource pages unless the caller is the hardware domain
-     * (see set_foreign_p2m_entry()).
-     */
-    return !paging_mode_translate(d) || is_hardware_domain(d);
-}
+bool arch_acquire_resource_check(const struct domain *d);
 
 /*
  * Updates vCPU's n2pm to match its np2m_base in VMCx12 and returns that np2m.
@@ -720,6 +713,10 @@ p2m_pod_offline_or_broken_hit(struct page_info *p);
 void
 p2m_pod_offline_or_broken_replace(struct page_info *p);
 
+/* Perform cleanup of p2m mappings ahead of teardown. */
+int
+relinquish_p2m_mapping(struct domain *d);
+
 #else
 
 static inline bool
@@ -748,6 +745,11 @@ static inline void p2m_pod_offline_or_broken_replace(struct page_info *p)
     ASSERT_UNREACHABLE();
 }
 
+static inline int relinquish_p2m_mapping(struct domain *d)
+{
+    return 0;
+}
+
 #endif
 
 
@@ -1043,7 +1045,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         break;
 
     case p2m_map_foreign:
-        if ( !mfn_valid(nfn) )
+        if ( !mfn_valid(nfn) || p2m != p2m_get_hostp2m(p2m->domain) )
         {
             ASSERT_UNREACHABLE();
             return -EINVAL;
@@ -1068,7 +1070,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
         break;
 
     case p2m_map_foreign:
-        if ( !mfn_valid(ofn) )
+        if ( !mfn_valid(ofn) || p2m != p2m_get_hostp2m(p2m->domain) )
         {
             ASSERT_UNREACHABLE();
             return -EINVAL;
diff --git a/xen/arch/x86/mm/p2m-basic.c b/xen/arch/x86/mm/p2m-basic.c
index 8599bd15c61a..25d27a0a9f56 100644
--- a/xen/arch/x86/mm/p2m-basic.c
+++ b/xen/arch/x86/mm/p2m-basic.c
@@ -13,6 +13,8 @@
 
 #include <xen/event.h>
 #include <xen/types.h>
+#include <asm/altp2m.h>
+#include <asm/hvm/nestedhvm.h>
 #include <asm/p2m.h>
 #include "mm-locks.h"
 #include "p2m.h"
@@ -207,6 +209,22 @@ void p2m_final_teardown(struct domain *d)
     p2m_teardown_hostp2m(d);
 }
 
+bool arch_acquire_resource_check(const struct domain *d)
+{
+    /*
+     * altp2m is not supported as we would otherwise also need to walk the
+     * altp2m tables and drop any foreign map entries in order to drop the page
+     * reference.
+     *
+     * The same applies to nestedhvm nested p2m tables, as the type from the L0
+     * p2m is replicated into the L1 p2m, and there's no filtering that
+     * prevents foreign mappings from being created in nestedp2m.
+     */
+    return is_pv_domain(d) ||
+           (d->arch.hvm.params[HVM_PARAM_ALTP2M] == XEN_ALTP2M_disabled &&
+            !nestedhvm_enabled(d));
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index ce742c12e0de..23c6c42af275 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2333,10 +2333,6 @@ static int p2m_add_foreign(struct domain *tdom, unsigned long fgfn,
     int rc;
     struct domain *fdom;
 
-    /*
-     * hvm fixme: until support is added to p2m teardown code to cleanup any
-     * foreign entries, limit this to hardware domain only.
-     */
     if ( !arch_acquire_resource_check(tdom) )
         return -EPERM;
 
@@ -2693,6 +2689,70 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
     return rc;
 }
 
+/*
+ * Remove foreign mappings from the p2m, as that drops the page reference taken
+ * when mapped.
+ */
+int relinquish_p2m_mapping(struct domain *d)
+{
+    struct p2m_domain *p2m = p2m_get_hostp2m(d);
+    unsigned long gfn, count = 0;
+    int rc = 0;
+
+    if ( !paging_mode_translate(d) )
+        return 0;
+
+    BUG_ON(!d->is_dying);
+
+    p2m_lock(p2m);
+
+    gfn = p2m->teardown_gfn;
+
+    /* Iterate over the whole p2m on debug builds to ensure correctness. */
+    while ( gfn <= p2m->max_mapped_pfn &&
+            (IS_ENABLED(CONFIG_DEBUG) || p2m->nr_foreign) )
+    {
+        unsigned int order;
+        p2m_type_t t;
+        p2m_access_t a;
+
+        _get_gfn_type_access(p2m, _gfn(gfn), &t, &a, 0, &order, 0);
+        ASSERT(IS_ALIGNED(gfn, 1UL << order));
+
+        if ( t == p2m_map_foreign )
+        {
+            ASSERT(p2m->nr_foreign);
+            ASSERT(order == 0);
+
+            rc = p2m_set_entry(p2m, _gfn(gfn), INVALID_MFN, order, p2m_invalid,
+                               p2m->default_access);
+            if ( rc )
+            {
+                printk(XENLOG_ERR
+                       "%pd: failed to unmap foreign page %" PRI_gfn " order %u error %d\n",
+                       d, gfn, order, rc);
+                ASSERT_UNREACHABLE();
+                break;
+            }
+        }
+
+        gfn += 1UL << order;
+
+        if ( !(++count & 0xff) && hypercall_preempt_check() )
+        {
+            rc = -ERESTART;
+            break;
+        }
+    }
+
+    ASSERT(gfn <= p2m->max_mapped_pfn || !p2m->nr_foreign);
+    p2m->teardown_gfn = gfn;
+
+    p2m_unlock(p2m);
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:54:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:54:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724176.1129342 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y2a-0001Ew-OP; Fri, 17 May 2024 13:54:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724176.1129342; Fri, 17 May 2024 13:54:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y2a-0001Ep-LC; Fri, 17 May 2024 13:54:32 +0000
Received: by outflank-mailman (input) for mailman id 724176;
 Fri, 17 May 2024 13:54:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=q70L=MU=cloud.com=christian.lindig@srs-se1.protection.inumbo.net>)
 id 1s7y2Y-0001EP-SX
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:54:30 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id faae33da-1454-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:54:28 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so4857876a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:54:28 -0700 (PDT)
Received: from smtpclient.apple ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574ec5fb906sm4685160a12.30.2024.05.17.06.54.27
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 17 May 2024 06:54:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: faae33da-1454-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715954068; x=1716558868; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zcCSs9okMCivkQhbeDBuMiXnfaxUqNGd4oaWoP83P/s=;
        b=LEBU8OH//kLsEzUZxCUo7Mo84S+1FobqxNn0QZj/U7G4w4kVvrVDST4xm/4rONcbI7
         TrydM5T0Z9GXeOtBMowaapWHgNbcGXLmQpc5BH5y9x5nXrpr9Q6gZQYiI4Kv2FwwSf4i
         uBCEbJlEL97oCZACVN4G2QNOS0iuZyLvKOF2U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954068; x=1716558868;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zcCSs9okMCivkQhbeDBuMiXnfaxUqNGd4oaWoP83P/s=;
        b=kLG4ahYJBr0AmVJEqbhwuJrM05xV4vvaOJY+Ix+695my9SdZhO2WIBbc2oRtFFyLCs
         YSPaZIZ2jLs3XYurUPndNvYvwm5xKNVL7qYMVVnAkihTxULz/JX9RebEVKY6h+QdS458
         /Z9hH/H2N+Zc+0b8KF0lG23fLYbZ1sp6S/PmqBsamTuqCOUaoGQPSlDSHmtxIgYN4+XO
         aMVyLAM9qm7/sGHGtbKM2RDkhAzgNHeTxMV4VM+BQOOHt9eX/wQBL4ckgFrsDvHDIvpy
         gN2GEn2mPtMB/7lsJrlyRnil4rWvux9Sd1NO0jiarAU66gkSBXDc4aFHNrzGv99eEiKt
         0/Eg==
X-Gm-Message-State: AOJu0YwPdAZfvPzP1a25vSU/bvEAa9tJW2cQbB6467ijlQZbDJX4N+pH
	yZXYrJpU2noWMMRrgZ5nAW3ZFPyIs0D1ryDopThzvnR4kq21Bo4njtj/GIcr7t4=
X-Google-Smtp-Source: AGHT+IHZY3wVRm39WlAI58oALla5n2J4PtGPIFbCfbjpZIZe28HeAoJV81ZYbhIe/gHxpomDPdNbPA==
X-Received: by 2002:a05:6402:40cd:b0:574:c473:294b with SMTP id 4fb4d7f45d1cf-574c47329camr12429489a12.4.1715954068378;
        Fri, 17 May 2024 06:54:28 -0700 (PDT)
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\))
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
From: Christian Lindig <christian.lindig@cloud.com>
In-Reply-To: <20240517133352.94347-3-roger.pau@citrix.com>
Date: Fri, 17 May 2024 14:54:16 +0100
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>,
 Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 David Scott <dave@recoil.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <CB1A21D1-3557-4A9D-898C-FF8238ADCB11@cloud.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>
X-Mailer: Apple Mail (2.3774.500.171.1.1)



> On 17 May 2024, at 14:33, Roger Pau Monne <roger.pau@citrix.com> =
wrote:
>=20
> Enabling it using an HVM param is fragile, and complicates the logic =
when
> deciding whether options that interact with altp2m can also be =
enabled.
>=20
> Leave the HVM param value for consumption by the guest, but prevent it =
from
> being set.  Enabling is now done using and additional altp2m specific =
field in
> xen_domctl_createdomain.
>=20
> Note that albeit only currently implemented in x86, altp2m could be =
implemented
> in other architectures, hence why the field is added to =
xen_domctl_createdomain
> instead of xen_arch_domainconfig.
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> Changes since v2:
> - Introduce a new altp2m field in xen_domctl_createdomain.
>=20
> Changes since v1:
> - New in this version.
> ---
> tools/libs/light/libxl_create.c     | 23 ++++++++++++++++++++++-
> tools/libs/light/libxl_x86.c        | 26 ++++++++++++--------------
> tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
> xen/arch/arm/domain.c               |  6 ++++++
> xen/arch/x86/domain.c               | 22 ++++++++++++++++++++++
> xen/arch/x86/hvm/hvm.c              | 23 ++++++++++++++++++++++-
> xen/include/public/domctl.h         | 20 +++++++++++++++++++-
> xen/include/public/hvm/params.h     |  9 ++-------
> 8 files changed, 106 insertions(+), 25 deletions(-)

>=20
> diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c =
b/tools/ocaml/libs/xc/xenctrl_stubs.c
> index 2b6d3c09dfa0..c6da9bb09137 100644
> --- a/tools/ocaml/libs/xc/xenctrl_stubs.c
> +++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
> @@ -257,7 +257,7 @@ CAMLprim value stub_xc_domain_create(value =
xch_val, value wanted_domid, value co
> #if defined(__i386__) || defined(__x86_64__)
>=20
> /* Quick & dirty check for ABI changes. */
> - BUILD_BUG_ON(sizeof(cfg) !=3D 64);
> + BUILD_BUG_ON(sizeof(cfg) !=3D 68);
>=20

Acked-by: Christian Lindig <christian.lindig@cloud.com>




From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724178.1129353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3C-0001fi-01; Fri, 17 May 2024 13:55:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724178.1129353; Fri, 17 May 2024 13:55:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3B-0001fb-Te; Fri, 17 May 2024 13:55:09 +0000
Received: by outflank-mailman (input) for mailman id 724178;
 Fri, 17 May 2024 13:55:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3B-0001cB-6C
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:09 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 12488c2d-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:08 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51f174e316eso741759e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:08 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12488c2d-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954107; x=1716558907; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=6Zat0qaq8czAsPhXnTr4M/ns8tt3lB7uw+7Hy5UjY58=;
        b=kEmKG0R4OsT5cvja9/IAe+ezVp1WiX0sYAHEf4snv/YH+PvEfaCXg1r1n4Xur8Hz1A
         V7xNlf+XdR11UkhF0SUvcvJPjUtYEuzyz8wAu8fjQsV3oWsikO5VYnx+KnAFl/so/ud6
         KqcQRRea5XeFddJnwolHEawHXPrKeJfkJppQveqOwerBo24WOx+HvYG3cLjorgaGOh18
         w9u+lQEEefWFkLsVXPv8HE2foUURdVEB1RC7ZIIZ5es8OC7O8/U027inI6mHqTGLnZVU
         V4F9Vzn9bsuNnNZUkMw0DuZzPHHCsh96gBukkF+W3oUqEDbV2ynKABLvK1P+R8Ti040u
         8Iaw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954107; x=1716558907;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=6Zat0qaq8czAsPhXnTr4M/ns8tt3lB7uw+7Hy5UjY58=;
        b=tCwEtqWx+61bGU0TtXW7XBfdD5l/+L3Sp3BAmyo7px6qjOg6ZL1W0ZL5yN+pRrYk56
         v3n43Fj5oXsO8lqoo5kTr+blTXJvbzV+iMnNSFcavhTf/wIK9cyzbAurK8SITKDlBXkE
         +RxDZctSMfmabcA4+kD2NTPJRBbJTvLPS+JoNpLQtHWkqZto7EH1JbnLQ1E90ui0vYz5
         4jIPrcW8BiVqmKnrebw5HVNVyL+oGxlBRkAThKmRc6orsTe/+20ayho9AM1l848pplfq
         EM9x8nPcJ5FFKP2LHK/anwg+mT2IjNnJTr5iBxJRVaE6iMZUju1/g30Z/kqFSgCTuHK/
         MPDw==
X-Gm-Message-State: AOJu0YwUVON6H1X/2fpsrPInAHRHv3/fJVQSdtkT6kGieMKlcUO/924L
	bUBr8s6GwpOUSUH3rZ1RwOCjtHt4d9YXWePFaRn6+xY8ET4tHr2fE2ZV/gdw
X-Google-Smtp-Source: AGHT+IH6zdXU24EUIAVRScxLXMjJ/UC/CV4kd7SskYpsZ4Kqe3loCkAnKfktM/2pzTcc5IBMF0ZLxw==
X-Received: by 2002:ac2:592f:0:b0:523:89b0:9b64 with SMTP id 2adb3069b0e04-52389b09bbdmr6058436e87.7.1715954107129;
        Fri, 17 May 2024 06:55:07 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v10 00/14] Enable build of full Xen for RISC-V
Date: Fri, 17 May 2024 15:54:49 +0200
Message-ID: <cover.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.
As an option, it can be considered to merge arch-specific patch and then just
rebase [2] and drop arch-specific changes.

[1] https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

---
Changes in V10:
  - Patch was merged to staging:
    - [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V9:
 - Patch was merged to staging:
    - [PATCH v8 07/17] xen/riscv: introduce io.h
  	- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V8:
 - Patch was merged to staging:
    - [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
    - [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
 - Update the commit message:
     - drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
     - Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (14):
  xen/riscv: disable unnecessary configs
  xen: introduce generic non-atomic test_*bit()
  xen/bitops: implement fls{l}() in common logic
  xen/riscv: introduce bitops.h
  xen/riscv: introduce cmpxchg.h
  xen/riscv: introduce atomic.h
  xen/riscv: introduce monitor.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add required things to current.h
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: introduce vm_event_*() functions
  xen/riscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                  |   4 +
 automation/gitlab-ci/build.yaml         |   4 +
 xen/arch/arm/arm64/livepatch.c          |   1 -
 xen/arch/arm/include/asm/arm32/bitops.h |   2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |   6 +-
 xen/arch/arm/include/asm/bitops.h       |  76 +----
 xen/arch/ppc/include/asm/bitops.h       |  55 ----
 xen/arch/ppc/include/asm/page.h         |   2 +-
 xen/arch/ppc/mm-radix.c                 |   2 +-
 xen/arch/riscv/Makefile                 |  18 +-
 xen/arch/riscv/arch.mk                  |   4 -
 xen/arch/riscv/configs/tiny64_defconfig |  12 +-
 xen/arch/riscv/early_printk.c           | 168 ----------
 xen/arch/riscv/include/asm/atomic.h     | 280 ++++++++++++++++
 xen/arch/riscv/include/asm/bitops.h     | 137 ++++++++
 xen/arch/riscv/include/asm/cache.h      |   2 +
 xen/arch/riscv/include/asm/cmpxchg.h    | 241 ++++++++++++++
 xen/arch/riscv/include/asm/config.h     |   2 +
 xen/arch/riscv/include/asm/current.h    |  19 ++
 xen/arch/riscv/include/asm/domain.h     |   2 +
 xen/arch/riscv/include/asm/mm.h         | 240 ++++++++++++++
 xen/arch/riscv/include/asm/monitor.h    |  26 ++
 xen/arch/riscv/include/asm/p2m.h        |   2 +
 xen/arch/riscv/mm.c                     |  52 ++-
 xen/arch/riscv/setup.c                  |  10 +-
 xen/arch/riscv/stubs.c                  | 415 ++++++++++++++++++++++++
 xen/arch/riscv/traps.c                  |  25 ++
 xen/arch/riscv/vm_event.c               |  19 ++
 xen/arch/x86/include/asm/bitops.h       |  43 +--
 xen/common/bitops.c                     |  22 ++
 xen/common/page_alloc.c                 |   4 +-
 xen/include/asm-generic/atomic-ops.h    |  97 ++++++
 xen/include/xen/bitops.h                | 155 +++++++++
 33 files changed, 1800 insertions(+), 347 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
 create mode 100644 xen/arch/riscv/include/asm/monitor.h
 create mode 100644 xen/arch/riscv/stubs.c
 create mode 100644 xen/arch/riscv/vm_event.c
 create mode 100644 xen/include/asm-generic/atomic-ops.h

-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724179.1129363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3E-0001wt-An; Fri, 17 May 2024 13:55:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724179.1129363; Fri, 17 May 2024 13:55:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3E-0001wk-6f; Fri, 17 May 2024 13:55:12 +0000
Received: by outflank-mailman (input) for mailman id 724179;
 Fri, 17 May 2024 13:55:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3D-0001sq-0f
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:11 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 129db0b8-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:09 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59a9d66a51so435757766b.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:09 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 129db0b8-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954108; x=1716558908; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Z/cIbtAQ9QV8Thdpou0mLeIkv3TrJg+eYRS5T1TORCI=;
        b=jXZFVLWZGn95Qq/VP6EPHPW2eP5253DhRsBvgTW9uSo5LS1GKx/52lNSn4v2VAFzvc
         gxoNcxQZfua9Thb0LaI6B9L/WzM1qs4643czbnYarK1oybWt3ZsF3hkDqQqebNKqlcUo
         AJvW0ioy6wXxhNTQ5+q81m3B3/EuwLQqa51qC+42KEnIcf07scsiqgdeAG30NzNc57mv
         RBudW5xCM3CcEoL1pWVgdg0Rwgpf3Xk9lGdAZwfz7Ok0PvFDAEPRftG40ngCknqZUbzV
         gVDp0mU2lImOBdI1MD9iwigLOZZq2dPMl2MzLL87Tz1wlSZbFoW2P9J3mjwpZ6ivtGaI
         dL6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954108; x=1716558908;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Z/cIbtAQ9QV8Thdpou0mLeIkv3TrJg+eYRS5T1TORCI=;
        b=tqUPfE5cMUwAApSzyJxD/vBnHGe+l5GWp/TySn4Pc0bAsS1d+GVm48IKbQ8TwDsinO
         4fzhXhgiUwQjFwk9FuDvptFtvofKogEKURuyxDTGQRvFaW7qmlLDWgUxUge3lIeBh7SF
         P3vycRG2QzpIieGGi2cIs1On5M5K6T5DIboEdZpRCETkuAnXpXrbNpurob0W7q3ZVmIr
         HUbWWh3XypIRXFmN+jZYBcpQ+qtBnEZ/c/olD0bnpuXagnqfC8iiM40AWvkY1fVTjFVg
         qBsexut97sccHbKRzoUhghXplZ5/3JPRLNxyJFDzh+ZvbiCahQPQLQg6ZqSw+oooVVnQ
         N6vw==
X-Gm-Message-State: AOJu0YzuLek2Lqc6VSiHt4wwfe0VyDykc2IxlC9REzFGQaiqL+wyMl+0
	5WmVFIH9LUIyWwqnqgVDKUUkAVcX3i/z7kphp3h1bH9jfR6TmhRd2zRKNVUt
X-Google-Smtp-Source: AGHT+IFcCxnAHbA5W6tbB/ONNqD9u6AsBJWsl4xVV8GKFS2M3nUZ8Wue5rqDReGSzSSBaN5Z2jFXog==
X-Received: by 2002:a17:906:5a5a:b0:a5a:34ae:10ea with SMTP id a640c23a62f3a-a5a34ae118emr1274240566b.76.1715954107915;
        Fri, 17 May 2024 06:55:07 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v10 01/14] xen/riscv: disable unnecessary configs
Date: Fri, 17 May 2024 15:54:50 +0200
Message-ID: <9924f75e0173813a654db4805301995a3af41abb.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Only configs which lead to compilation issues were disabled.

Remove lines related to disablement of configs which aren't affected
compilation:
 -# CONFIG_SCHED_CREDIT is not set
 -# CONFIG_SCHED_RTDS is not set
 -# CONFIG_SCHED_NULL is not set
 -# CONFIG_SCHED_ARINC653 is not set
 -# CONFIG_TRACEBUFFER is not set
 -# CONFIG_HYPFS is not set
 -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set

To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
check_get_page_from_gfn() from xen/p2m-common.h.

Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was
included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
aren't declared started to occur.

CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.

CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
acpi_disabled, so it is disabled for now.

PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
also disabled for now, as RISC-V hasn't introduced this headers yet.

LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V10:
 - Nothing changed. Only rebase.
---
Changes in V9:
 - update the commit message: add info about LIVEPATCH and PERF_COUNTERS.
---
Changes in V8:
 - disabled CPU_BOOT_TIME_CPUPOOLS as it requires an introduction of cpu_physical_id() and acpi_disabled.
 - leave XSM disabled, add explanation in the commit message.
 - drop HYPFS as the patch was provided to resolve compilation issue when this condif is enabled for RISC-V.
 - include asm/p2m.h to asm/domain.h, and xen/errno.h to asm/p2m.h to drop ARGO config from
   tiny64_defconfing and build.yaml.
 - update the commit message.
---
Changes in V7:
 - Disable only configs which cause compilation issues.
 - Update the commit message.
---
Changes in V6:
 - Nothing changed. Only rebase.
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         |  4 ++++
 xen/arch/riscv/configs/tiny64_defconfig | 12 +++++-------
 xen/arch/riscv/include/asm/domain.h     |  2 ++
 xen/arch/riscv/include/asm/p2m.h        |  2 ++
 4 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5..ff5c9055d1 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -494,10 +494,14 @@ alpine-3.18-gcc-debug-arm64-earlyprintk:
 .riscv-fixed-randconfig:
   variables: &riscv-fixed-randconfig
     EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_BOOT_TIME_CPUPOOLS=n
       CONFIG_COVERAGE=n
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..fc7a04872f 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -1,12 +1,10 @@
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
+# CONFIG_BOOT_TIME_CPUPOOLS is not set
 # CONFIG_GRANT_TABLE is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_XSM is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 027bfa8a93..16a9dd57aa 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -5,6 +5,8 @@
 #include <xen/xmalloc.h>
 #include <public/hvm/params.h>
 
+#include <asm/p2m.h>
+
 struct hvm_domain
 {
     uint64_t              params[HVM_NR_PARAMS];
diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
index 387f372b5d..26860c0ae7 100644
--- a/xen/arch/riscv/include/asm/p2m.h
+++ b/xen/arch/riscv/include/asm/p2m.h
@@ -2,6 +2,8 @@
 #ifndef __ASM_RISCV_P2M_H__
 #define __ASM_RISCV_P2M_H__
 
+#include <xen/errno.h>
+
 #include <asm/page-bits.h>
 
 #define paddr_bits PADDR_BITS
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724180.1129368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3E-0001zh-Kt; Fri, 17 May 2024 13:55:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724180.1129368; Fri, 17 May 2024 13:55:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3E-0001yl-Dk; Fri, 17 May 2024 13:55:12 +0000
Received: by outflank-mailman (input) for mailman id 724180;
 Fri, 17 May 2024 13:55:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3D-0001cB-3M
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:11 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1341fefd-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:10 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-52389c1308dso767289e87.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:10 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1341fefd-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954109; x=1716558909; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=54376BKXaMRj0bU8DqMJL73bnzloZBB1RI+aKenu2eI=;
        b=gdP/4CI5TOgI/sY/pNh+WFS4+XybeGcT1WuN6QrFkaQ4JFfbQuBsbhCeZertfIEhV3
         69LA4PywRqRIGyU3Cn8h8mSC5tgYJEG9SE0BloJoGtNVre7+LkqrJBAzLZoPonu8o9kZ
         XvDy96JtXu7gKGCLfSW65UblpDTvzUCGHCXj7+ou/D1z1Kb5IuUxR+0cNqWqxTPFOnhW
         XKIEI0Am24RD6pzMKN0c4kOFPB+TLyNpC4Sgpv/lvtaJnaP4+aAnuDgNfbyOYSB6ssJE
         5vDs0wfBh+OIAoGtyzi9zP5IArq2qPwP8YYtRug4B+4KhPt0XfqMOS7x4xndQXQpR+8E
         6TCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954109; x=1716558909;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=54376BKXaMRj0bU8DqMJL73bnzloZBB1RI+aKenu2eI=;
        b=Y/8Yp/59OpnUNW8pYT3D3QSE7fN7NNjLc9yXkUiZeYPeiCMari0sCEYLswm89steCB
         CBOfvGzPb8eTbAACd7Su08GrQ1D34WwMRUk/Ybreul9fCMGY4V4fJH0ov3lT62dpVrXi
         1N2KDbeKWedse3cf99pJyPcNCcbFzr/7kSQCHhl95gddbVnBF9PL+MEHEHJKUaZvDiax
         hQIuAMAY845ftd2RBA1P2miGzMukWjRe1j1EnJG8mTRtT51VuOLPQLAnuIcunIy7evIo
         /n/SuiyZkdWUQRJaO8p57SlTJAoYJHkhHf2wTD3PDnTcsF3pDOId4J1G8cg6T17c3esK
         X4hw==
X-Gm-Message-State: AOJu0YzBbBrhRZMSMP8OVOzAUBTZDM1cEy+MmcXAhfxL4tujukVvjlMj
	pR2YlZvy6FysaQ2xEB7wydPohuySEv8db5Xs27/apBlJ+JjgTiCmjg6RYTn7
X-Google-Smtp-Source: AGHT+IFTfWDRo3Tqa1PCPq1VbAzDPpfputSHVqpLpWWJ1xgDRzivP3CQXXKgxgm1Xq7uMSyW791i5A==
X-Received: by 2002:a05:6512:2254:b0:523:294f:7fe6 with SMTP id 2adb3069b0e04-523294f806bmr11415779e87.44.1715954109015;
        Fri, 17 May 2024 06:55:09 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Date: Fri, 17 May 2024 15:54:51 +0200
Message-ID: <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

The following approach was chosen for generic*() and arch*() bit
operation functions:
If the bit operation function that is going to be generic starts
with the prefix "__", then the corresponding generic/arch function
will also contain the "__" prefix. For example:
 * test_bit() will be defined using arch_test_bit() and
   generic_test_bit().
 * __test_and_set_bit() will be defined using
   arch__test_and_set_bit() and generic__test_and_set_bit().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
   The context ("* Find First Set bit.  Bits are labelled from 1." in xen/bitops.h )
   suggests there's a dependency on an uncommitted patch. It happens becuase the current patch
   series is based on Andrew's patch series ( https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t ),
   but if everything is okay with the current one patch it can be merged without Andrew's patch series being merged.
---
Changes in V10:
 - update the commit message. ( re-order paragraphs and add explanation usage of prefix "__" in bit
   operation function names )
 - add  parentheses around the whole expression of bitop_bad_size() macros.
 - move macros bitop_bad_size() above asm/bitops.h as it is not arch-specific anymore and there is
   no need for overriding it.
 - drop macros check_bitop_size() and use "if ( bitop_bad_size(addr) ) __bitop_bad_size();" implictly
   where it is needed.
 - in <xen/bitops.h> use 'int' as a first parameter for __test_and_*(), generic__test_and_*() to be
   consistent with how the mentioned functions were declared in the original per-arch functions.
 - add 'const' to p variable in generic_test_bit().
 - move definition of BITOP_BITS_PER_WORD and bitop_uint_t to xen/bitops.h as we don't allow for arch
   overrides these definitions anymore. 
---
Changes in V9:
  - move up xen/bitops.h in ppc/asm/page.h.
  - update defintion of arch_check_bitop_size.
    And drop correspondent macros from x86/asm/bitops.h
  - drop parentheses in generic__test_and_set_bit() for definition of
    local variable p.
  - fix indentation inside #ifndef BITOP_TYPE...#endif
  - update the commit message.
---
 Changes in V8:
  - drop __pure for function which uses volatile.
  - drop unnessary () in generic__test_and_change_bit() for addr casting.
  - update prototype of generic_test_bit() and test_bit(): now it returns bool
    instead of int.
  - update generic_test_bit() to use BITOP_MASK().
  - Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
  - add a footer with explanation of dependency on an uncommitted patch after Signed-off.
  - abstract bitop_size().
  - move BITOP_TYPE define to <xen/types.h>.
---
 Changes in V7:
  - move everything to xen/bitops.h to follow the same approach for all generic
    bit ops.
  - put together BITOP_BITS_PER_WORD and bitops_uint_t.
  - make BITOP_MASK more generic.
  - drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
    enough.
  - drop "_" for generic__{test_and_set_bit,...}().
  - drop " != 0" for functions which return bool.
  - add volatile during the cast for generic__{...}().
  - update the commit message.
  - update arch related code to follow the proposed generic approach.
---
 Changes in V6:
  - Nothing changed ( only rebase )
---
 Changes in V5:
   - new patch
---
 xen/arch/arm/arm64/livepatch.c    |   1 -
 xen/arch/arm/include/asm/bitops.h |  67 ---------------
 xen/arch/ppc/include/asm/bitops.h |  52 ------------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  31 ++-----
 xen/include/xen/bitops.h          | 131 ++++++++++++++++++++++++++++++
 7 files changed, 142 insertions(+), 144 deletions(-)

diff --git a/xen/arch/arm/arm64/livepatch.c b/xen/arch/arm/arm64/livepatch.c
index df2cebedde..4bc8ed9be5 100644
--- a/xen/arch/arm/arm64/livepatch.c
+++ b/xen/arch/arm/arm64/livepatch.c
@@ -10,7 +10,6 @@
 #include <xen/mm.h>
 #include <xen/vmap.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/insn.h>
 #include <asm/livepatch.h>
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 5104334e48..8e16335e76 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -22,9 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 #define ADDR (*(volatile int *) addr)
@@ -76,70 +73,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 /*
  * On ARMv5 and above those functions can be implemented around
  * the clz instruction for much better code efficiency.
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 20f7865358..049aa62b89 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -15,9 +15,6 @@
 #define __set_bit(n, p)         set_bit(n, p)
 #define __clear_bit(n, p)       clear_bit(n, p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 /* PPC bit number conversion */
@@ -69,17 +66,6 @@ static inline void clear_bit(int nr, volatile void *addr)
     clear_bits(BITOP_MASK(nr), (volatile unsigned int *)addr + BITOP_WORD(nr));
 }
 
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-    const volatile unsigned int *p = addr;
-    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
-}
-
 static inline unsigned int test_and_clear_bits(
     unsigned int mask,
     volatile unsigned int *p)
@@ -133,44 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old | mask;
-    return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old & ~mask;
-    return (old & mask) != 0;
-}
-
 #define flsl(x) generic_flsl(x)
 #define fls(x) generic_fls(x)
 
diff --git a/xen/arch/ppc/include/asm/page.h b/xen/arch/ppc/include/asm/page.h
index 890e285051..6d4cd2611c 100644
--- a/xen/arch/ppc/include/asm/page.h
+++ b/xen/arch/ppc/include/asm/page.h
@@ -2,9 +2,9 @@
 #ifndef _ASM_PPC_PAGE_H
 #define _ASM_PPC_PAGE_H
 
+#include <xen/bitops.h>
 #include <xen/types.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
diff --git a/xen/arch/ppc/mm-radix.c b/xen/arch/ppc/mm-radix.c
index ab5a10695c..9055730997 100644
--- a/xen/arch/ppc/mm-radix.c
+++ b/xen/arch/ppc/mm-radix.c
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index dd439b69a0..23f09fdb7a 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -19,9 +19,6 @@
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
-extern void __bitop_bad_size(void);
-#define bitop_bad_size(addr) (sizeof(*(addr)) < 4)
-
 /**
  * set_bit - Atomically set a bit in memory
  * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index f14ad0d33a..6eeeff0117 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned long x)
  * scope
  */
 
+#define BITOP_BITS_PER_WORD 32
+typedef uint32_t bitop_uint_t;
+
+#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
+
+#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
+
+extern void __bitop_bad_size(void);
+
+#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
+
 /* --------------------- Please tidy above here --------------------- */
 
 #include <asm/bitops.h>
 
+/**
+ * generic__test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_set_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask);
+}
+
+/**
+ * generic__test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_clear_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask);
+}
+
+/* WARNING: non atomic and it can be reordered! */
+static always_inline bool
+generic__test_and_change_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask);
+}
+/**
+ * generic_test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static always_inline bool generic_test_bit(int nr, const volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    const volatile bitop_uint_t *p =
+                        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+
+    return (*p & mask);
+}
+
+static always_inline bool
+__test_and_set_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_set_bit
+#define arch__test_and_set_bit generic__test_and_set_bit
+#endif
+
+    return arch__test_and_set_bit(nr, addr);
+}
+#define __test_and_set_bit(nr, addr) ({             \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_set_bit(nr, addr);                   \
+})
+
+static always_inline bool
+__test_and_clear_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_clear_bit
+#define arch__test_and_clear_bit generic__test_and_clear_bit
+#endif
+
+    return arch__test_and_clear_bit(nr, addr);
+}
+#define __test_and_clear_bit(nr, addr) ({           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_clear_bit(nr, addr);                 \
+})
+
+static always_inline bool
+__test_and_change_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_change_bit
+#define arch__test_and_change_bit generic__test_and_change_bit
+#endif
+
+    return arch__test_and_change_bit(nr, addr);
+}
+#define __test_and_change_bit(nr, addr) ({              \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    __test_and_change_bit(nr, addr);                    \
+})
+
+static always_inline bool test_bit(int nr, const volatile void *addr)
+{
+#ifndef arch_test_bit
+#define arch_test_bit generic_test_bit
+#endif
+
+    return arch_test_bit(nr, addr);
+}
+#define test_bit(nr, addr) ({                           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    test_bit(nr, addr);                                 \
+})
+
 /*
  * Find First Set bit.  Bits are labelled from 1.
  */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724181.1129383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3G-0002T9-UM; Fri, 17 May 2024 13:55:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724181.1129383; Fri, 17 May 2024 13:55:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3G-0002Sw-Qh; Fri, 17 May 2024 13:55:14 +0000
Received: by outflank-mailman (input) for mailman id 724181;
 Fri, 17 May 2024 13:55:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3E-0001sq-M9
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:12 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13be8d67-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:10 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59a0e4b773so458873266b.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:10 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13be8d67-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954110; x=1716558910; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=woatuozuOyseeL4cnN4KyDezDGVt7DBVcd4a7zZ/GoA=;
        b=WdzIylq215iprKTArZcnSdkcdpYaKOtSS+C2mhpMFtBrV+GcbPnPf3PMmmjxPeC2Yu
         4K+k8YXTpyYcjOAHkMMBNTKk9+Bk9PeKc1VXbgnkJYWIz7C+6lxzU+sOD8uuRZwOPI82
         R4eMkGtSybEpuBw+caCMR2UHyp2OVdNnTNbK45fuExWA33CEtiCDqISm7ABaj0PEe79S
         OMrb3N9BMZeAOqZTR9PyUlBzkcMBtFJpjPPPb4bwI1I7I358jmTXXHuuPtjNWP8SItwA
         rbfB0hq5EYoLnZl8ubnsTaA6+BAFefYeOWZVcu5uecM/zvJ5x15BRFsk4LD6Yc5yq7W4
         TdUQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954110; x=1716558910;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=woatuozuOyseeL4cnN4KyDezDGVt7DBVcd4a7zZ/GoA=;
        b=SNBogpROmET7WwsdeRG7rEQeQd1kR6BzS1vwOgRjrQ3cyUSBSNbIYoAvXg1X0RrTXj
         SzxBTWedRYrm7zVn4DuWgckYM3ExL/iWmKsz1TgaEDjOHrzg9MCHs2EuRvCBiQ4+mV4h
         l+7IN53BrKlSWBmiGbOcqY7uD6J+ugrTBc1Ub59k3JaGzkrC9AeM8EPHzCUGas76dPiD
         BR2GufdBlvR7L1U4SA5BBL06boHhIb5cOogJJmkGtVXd+deCwyZUcvhduLbC7qysjktg
         b75yV2Vw7gp5uu8Q0694JeS/xh3ow+xvasEthwiudA2I7CDaOx4ZG2s++fVhVnl66C11
         uilQ==
X-Gm-Message-State: AOJu0YxFDJs4oGgndC7ItssMbvnuWgefuENZJ5rplRH9Arz+6uLA4Z4e
	tbtkrLVjSZC4Nqc0gelrGFZqylqTXbCeMkIP/NqBs3Bo8ii/WI1p62VtU7cL
X-Google-Smtp-Source: AGHT+IHsehvx8pqw/g9Quv5gEW1QWWYdyQ7L1VBT233xZ+7+m0DFfBk+bO2yECVK8fjtIZ1/kLvG1A==
X-Received: by 2002:a17:906:2746:b0:a5a:84f9:df73 with SMTP id a640c23a62f3a-a5a84f9dfd2mr736182166b.38.1715954110155;
        Fri, 17 May 2024 06:55:10 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v10 03/14] xen/bitops: implement fls{l}() in common logic
Date: Fri, 17 May 2024 15:54:52 +0200
Message-ID: <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To avoid the compilation error below, it is needed to update to places
in common/page_alloc.c where flsl() is used as now flsl() returns unsigned int:

./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
       18 |         (void) (&_x == &_y);            \
          |                     ^~
    common/page_alloc.c:1843:34: note: in expansion of macro 'min'
     1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);

generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0,
the result in undefined.

The prototype of the per-architecture fls{l}() functions was changed to
return 'unsigned int' to align with the generic implementation of these
functions and avoid introducing signed/unsigned mismatches.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 The patch is almost independent from Andrew's patch series
 ( https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t)
 except test_fls() function which IMO can be merged as a separate patch after Andrew's patch
 will be fully ready.
---
Changes in V10:
 - update return type of arch_flsl() across arcitectures to 'unsigned int' to be aligned
   with return type of generic flsl() in xen/bitops.h.
 - switch inline to always_inline for arch_flsl() across architectures to be in sync
   with other similar changes.
 - define arch_flsl as arch_fls not just only fls.
 - update the commit message ( add information that per-arch fls{l)() protypes were
   changed ).
---
Changes in V9:
 - update return type of fls and flsl() to unsigned int to be aligned with other
   bit ops.
 - update places where return value of fls() and flsl() is compared with int.
 - update the commit message.
---
Changes in V8:
 - do proper rebase: back definition of fls{l} to the current patch.
 - drop the changes which removed ffz() in PPC. it should be done not
   in this patch.
 - add a message after Signed-off.
---
Changes in V7:
 - Code style fixes
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm32/bitops.h |  2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  6 ++----
 xen/arch/arm/include/asm/bitops.h       |  9 +++------
 xen/arch/ppc/include/asm/bitops.h       |  3 ---
 xen/arch/x86/include/asm/bitops.h       | 12 +++++++-----
 xen/common/bitops.c                     | 22 ++++++++++++++++++++++
 xen/common/page_alloc.c                 |  4 ++--
 xen/include/xen/bitops.h                | 24 ++++++++++++++++++++++++
 8 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/bitops.h b/xen/arch/arm/include/asm/arm32/bitops.h
index d0309d47c1..9ee96f568b 100644
--- a/xen/arch/arm/include/asm/arm32/bitops.h
+++ b/xen/arch/arm/include/asm/arm32/bitops.h
@@ -1,7 +1,7 @@
 #ifndef _ARM_ARM32_BITOPS_H
 #define _ARM_ARM32_BITOPS_H
 
-#define flsl fls
+#define arch_flsl arch_fls
 
 /*
  * Little endian assembly bitops.  nr = 0 -> byte 0 bit 0.
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 906d84e5f2..d942077392 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -1,17 +1,15 @@
 #ifndef _ARM_ARM64_BITOPS_H
 #define _ARM_ARM64_BITOPS_H
 
-static inline int flsl(unsigned long x)
+static always_inline unsigned int arch_flsl(unsigned long x)
 {
         uint64_t ret;
 
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
         asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
 
         return BITS_PER_LONG - ret;
 }
+#define arch_flsl arch_flsl
 
 /* Based on linux/include/asm-generic/bitops/find.h */
 
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 8e16335e76..f428cf8338 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -78,17 +78,14 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p,
  * the clz instruction for much better code efficiency.
  */
 
-static inline int fls(unsigned int x)
+static always_inline unsigned int arch_fls(unsigned int x)
 {
-        int ret;
-
-        if (__builtin_constant_p(x))
-               return generic_fls(x);
+        unsigned int ret;
 
         asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
         return 32 - ret;
 }
-
+#define arch_fls arch_fls
 
 #define arch_ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
 #define arch_ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 049aa62b89..46154fc957 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -119,9 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-#define flsl(x) generic_flsl(x)
-#define fls(x) generic_fls(x)
-
 /**
  * hweightN - returns the hamming weight of a N-bit word
  * @x: the word to weigh
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 23f09fdb7a..17849fae97 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -425,20 +425,21 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
  *
  * This is defined the same way as ffs.
  */
-static inline int flsl(unsigned long x)
+static always_inline unsigned int arch_flsl(unsigned long x)
 {
-    long r;
+    unsigned long r;
 
     asm ( "bsr %1,%0\n\t"
           "jnz 1f\n\t"
           "mov $-1,%0\n"
           "1:" : "=r" (r) : "rm" (x));
-    return (int)r+1;
+    return (unsigned int)r+1;
 }
+#define arch_flsl arch_flsl
 
-static inline int fls(unsigned int x)
+static always_inline unsigned int arch_fls(unsigned int x)
 {
-    int r;
+    unsigned int r;
 
     asm ( "bsr %1,%0\n\t"
           "jnz 1f\n\t"
@@ -446,6 +447,7 @@ static inline int fls(unsigned int x)
           "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_fls arch_fls
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index a8c32f6767..95bc47176b 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -62,9 +62,31 @@ static void test_ffs(void)
     CHECK(ffs64, (uint64_t)0x8000000000000000, 64);
 }
 
+static void test_fls(void)
+{
+    /* unsigned int ffs(unsigned int) */
+    CHECK(fls, 1, 1);
+    CHECK(fls, 3, 2);
+    CHECK(fls, 3U << 30, 32);
+
+    /* unsigned int flsl(unsigned long) */
+    CHECK(flsl, 1, 1);
+    CHECK(flsl, 1UL << (BITS_PER_LONG - 1), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    CHECK(flsl, 3UL << 32, 34);
+#endif
+
+    /* unsigned int fls64(uint64_t) */
+    CHECK(fls64, 1, 1);
+    CHECK(fls64, 0x00000000C0000000ULL, 32);
+    CHECK(fls64, 0x0000000180000000ULL, 33);
+    CHECK(fls64, 0xC000000000000000ULL, 64);
+}
+
 static int __init cf_check test_bitops(void)
 {
     test_ffs();
+    test_fls();
 
     return 0;
 }
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index be4ba3962a..eed6b2a901 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1842,7 +1842,7 @@ static void _init_heap_pages(const struct page_info *pg,
          * Note that the value of ffsl() and flsl() starts from 1 so we need
          * to decrement it by 1.
          */
-        unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
+        unsigned int inc_order = min(MAX_ORDER + 0U, flsl(e - s) - 1);
 
         if ( s )
             inc_order = min(inc_order, ffsl(s) - 1U);
@@ -2266,7 +2266,7 @@ void __init xenheap_max_mfn(unsigned long mfn)
     ASSERT(!first_node_initialised);
     ASSERT(!xenheap_bits);
     BUILD_BUG_ON((PADDR_BITS - PAGE_SHIFT) >= BITS_PER_LONG);
-    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS);
+    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS + 0U);
     printk(XENLOG_INFO "Xen heap: %u bits\n", xenheap_bits);
 }
 
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 6eeeff0117..59b8028894 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -200,6 +200,30 @@ static always_inline bool test_bit(int nr, const volatile void *addr)
     test_bit(nr, addr);                                 \
 })
 
+static always_inline __pure unsigned int fls(unsigned int x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_fls(x);
+
+#ifndef arch_fls
+#define arch_fls generic_fls
+#endif
+
+    return arch_fls(x);
+}
+
+static always_inline __pure unsigned int flsl(unsigned long x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_flsl(x);
+
+#ifndef arch_flsl
+#define arch_flsl generic_flsl
+#endif
+
+    return arch_flsl(x);
+}
+
 /*
  * Find First Set bit.  Bits are labelled from 1.
  */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724182.1129390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3H-0002X1-DC; Fri, 17 May 2024 13:55:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724182.1129390; Fri, 17 May 2024 13:55:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3H-0002W6-39; Fri, 17 May 2024 13:55:15 +0000
Received: by outflank-mailman (input) for mailman id 724182;
 Fri, 17 May 2024 13:55:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3F-0001sq-I5
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:13 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1446885c-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:11 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a5a7d28555bso515547366b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:11 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1446885c-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954111; x=1716558911; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Rh5FFSQR5hpsEecLRvPLMcmIow3q1eeW4spmciqITUA=;
        b=WuH1E1AVcmpm4+HGJdGSdthduKLfXQaLF8vD9WS5KOivUz+vrykhe10hzVQqjUj+mI
         7PJFIQt52JWr0knOhzuhKBF6iXHSnsRP46X2tJxd+e+M4EHaQAw3L30cB+xjE+867qkB
         3gESwMBqmNEwcZmOdeDG3e3m1Mzz1tPuifzG29uP0hEub9O1/QX626coZ+fX1ObK1iRs
         0j+RA1wxE6FlEs5c1vx9FEg0/0f9f1iedarDxzLSl236Fz/fTdJBnxqEpSO2vkw2L0CI
         kkUFFsrKvvXXhDSz5KuhFDrRjc2POah/MHRCjf2R/557lD742tGS6N/Xlc3Fs5BJ72nV
         V+wQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954111; x=1716558911;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Rh5FFSQR5hpsEecLRvPLMcmIow3q1eeW4spmciqITUA=;
        b=dQAoMmE57PZPgpvf3Hmv6Ba5bfb3EmGvOhoTmum8EyJGNx1ifXu6haVuCuvWAvfQD6
         +I64uA6+7zNbWzzqPCeHCr195exJcVATqTqkRnImxymy7oumrs6KaOHfj6vsZYCZ0hml
         7xn4XxTSSkoMO2OkAFh7ALiJ4DZaLsSs53I2R8rkS/Zbm01MHdtAdv7KZCXcv4ushpLe
         tmF0zmMWuWAm/5EDJXSIpfxtx5YML+r7mCCbh1wPhC/xZkb9xjZgOiNCs4/ZHsaTYlUx
         EU7/Jl0VhTqlb5eeWXxndQeoqF6vd8Leg3jha7oW3U+ycwl3SaZSEBhowPWezgUg2R2j
         VJpA==
X-Gm-Message-State: AOJu0YxIImeu8ZpZ1E2ix+Lim1vSjtYY9RdWe9v6yO2UCSqyccCsnuyG
	JhzUwCMjy2uIYNslkcJSWTQxSlwQWfz72NFG0GsP0+KQdf2niLtljw5o58Ag
X-Google-Smtp-Source: AGHT+IHNjIsZ33hbP/mwv+OD1FCavU7oLCFaO/9euqPJVY8ZS5AUcTgBa4Fspvr4AA9k9Tle0yAsUA==
X-Received: by 2002:a17:906:17c5:b0:a5a:1b57:426f with SMTP id a640c23a62f3a-a5a2d54c10fmr1418274466b.13.1715954111026;
        Fri, 17 May 2024 06:55:11 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 04/14] xen/riscv: introduce bitops.h
Date: Fri, 17 May 2024 15:54:53 +0200
Message-ID: <91a359cb03c150ca01b52bfcab696612a847d283.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V10:
 - update the error message BITS_PER_LONG -> BITOP_BITS_PER_WORD
---
Changes in V9:
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects
   to work with 32-bit quantities.
 - s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
 - define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of
   "introduce generic non-atomic test_*bit()".
 - drop duplicated __set_bit() and __clear_bit().
 - drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
 - update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
 - drop 1 padding blank in test_and_op_bit_ord().
 - update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
   change return type to bool.
 - change addr argument type of test_and_change_bit(): unsigned long * -> void *.
 - move test_and_change_bit() closer to other test_and-s function.
 - Code style fixes: tabs -> space.
 - s/#undef __op_bit/#undef op_bit.
 - update the commit message: delete information about generic-non-atomic.h changes as now
   it is a separate patch.
---
Changes in V7:
 - Update the commit message.
 - Drop "__" for __op_bit and __op_bit_ord as they are atomic.
 - add comment above __set_bit and __clear_bit about why they are defined as atomic.
 - align bitops_uint_t with __AMO().
 - make changes after  generic non-atomic test_*bit() were changed.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
 - rebase clean ups were done: drop unused asm-generic includes
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..7f7af3fda1
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline bool test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline bool test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline bool test_and_change_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724183.1129397 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3H-0002fU-Rk; Fri, 17 May 2024 13:55:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724183.1129397; Fri, 17 May 2024 13:55:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3H-0002bx-Gd; Fri, 17 May 2024 13:55:15 +0000
Received: by outflank-mailman (input) for mailman id 724183;
 Fri, 17 May 2024 13:55:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3G-0001cB-5R
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:14 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 150695c4-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:13 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-572f6ee87c1so4572333a12.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:13 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 150695c4-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954112; x=1716558912; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dmyFXVNnt+hxL3zbiepowjxUxD57Pi8IWfGlJVjPZ2c=;
        b=HNhUOPj1KLI1pWjId5P8dNr1Yh2P/HvbmXQwAxKnf68YZtdtHxaxl7F7z640eEuwSA
         /gI3I5xyyJwvmfHWTw8e2/ZtJ9Ct8puVz1DAHUraKTuFz8wxm+LEQaaynSghtpOowlZR
         OIFS6j3AJ538vrq6Mscl3kAVdfiJGr3DnK52ykxiLMAgH3bGr5eu0NsfqKp6srK3wpSR
         LVBjTK5Q7TIDk643qxatkp6lVzr6sHZTguwr/cgaOR/lpAmVVDmatSEcEdY+5pyHHsaS
         YX9Tp6hiAAJBJqBuYa7OrOkLXuGfONJ9XduqA+OYINC9/9SIJAw4tjoMLLkj/XiVe+E3
         ZfZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954112; x=1716558912;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dmyFXVNnt+hxL3zbiepowjxUxD57Pi8IWfGlJVjPZ2c=;
        b=YalY2mKMe1qNAoT6/L6w5x6Uazx3vxfgNWE6EqNUNPqQLH3ZHayLsmlVozwMFCBU8L
         2HgBel2Njs+b25i4aa3Jtwa56CfBR+qpzVhSEM4irf2wFVFsb5qm1EjAqoCqVglt8g0E
         Hju6WOr3syaxZ0h5QHC2BkDjHYcMWlnMKenNKx1JXbwJ3Q2vUGtA5GyCkkaOoUE35eSD
         bNAxntyytbSZ8UaYv3HTiYmUxSLNXBV7lDzmqTE6E44m4YMW01IBeMyrTrjQ/erbtosq
         yrhQ6Re42aa49n6sBvw3vsrk0qZ3AgN3eZUJV+WUGipTBRM1p5jkNG5Gnf5d72QbFHS3
         HZhQ==
X-Gm-Message-State: AOJu0Yy47P5Kj3wxMu61tt7MczzPsHZLlsknl6u9TZaUThPPTBs6xCOm
	ly65CFyWfprQBhP6dB4KUycH1rG7R/u+hRA5iopAg7+XwngwJUumfS7SgEv9
X-Google-Smtp-Source: AGHT+IGcyYgqw7vHA3eICyUNiTI1THyvqb+ALgLMhYDvqNtVqiPJ30pnF69tErln8kIe4xgeQcGPvg==
X-Received: by 2002:a17:906:c2c7:b0:a59:beb2:62d7 with SMTP id a640c23a62f3a-a5a2d665449mr1453259066b.56.1715954111845;
        Fri, 17 May 2024 06:55:11 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
Date: Fri, 17 May 2024 15:54:54 +0200
Message-ID: <79303bd6ba08b8c3bad96cb4e47d46f61a42d081.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header was taken from Linux kernl 6.4.0-rc1.

Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic
  access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use them
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )

Implementation of 4- and 8-byte cases were updated according to the spec:
```
              ....
Linux Construct         RVWMO AMO Mapping
    ...
atomic <op>             amo<op>.{w|d}.aqrl
Linux Construct         RVWMO LR/SC Mapping
    ...
atomic <op>             loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop

Table A.5: Mappings from Linux memory primitives to RISC-V primitives

```

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
          setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V10:
 - Code style fixes for __xchg, __cmpxchg ( the line was too long )
 - change type of size argument from int to unsigned int for
   __bad_{cmp}xchg(), __{cmp}xchg().
 - drop parentheses around ptr in {cmp}xchg() when ptr is an argument of
   __{cmp}xchg().
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V9:
 - update return type of __bad_xchg();
 - update the comment above __bad_cmpxchg();
 - update the default case inside __xchg() to be aligned with similar default
   case in __cmpxchg().
---
Changes in V8:
 - use __bad_{xchg,cmpxch}(ptr,size) insetead of STATIC_ASSERT_UNREACHABLE() to
   make this patch be independent from the macros that haven't been committed yet
   and may never be.
---
Changes in V7:
 - replace __*() -> _*() in cmpxchg.h
 - add () around ptr in _amoswap_generic(), emulate_xchg_1_2()
 - fix typos
 - code style fixes.
 - refactor emulate_xcgh_1_2():
   - add parentheses for new argument.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - s/CONFIG_32BIT/CONFIG_RISCV_32
 - drop unnecessary parentheses in xchg()
 - drop register in _generic_cmpxchg()
 - refactor and update prototype of _generic_cmpxchg():
   add named operands, return value instead of passing ret as an argument, drop %z and J
   constraints for mask operand as it can't be zero
 - refactor and code style fixes in emulate_cmpxchg_1_2():
   - add explanatory comment for emulate_cmpxchg_1_2().
   - add parentheses for old and new arguments.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - drop unnessary parenthesses in cmpxchg().
 - update the commit message.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
-  update the commit message? ( As before I don't understand this point. Can you give an example of what sort of opcode / instruction is missing?)
 - Code style fixes
 - change sizeof(*ptr) -> sizeof(*(ptr))
 - update operands names and some local variables for macros emulate_xchg_1_2() and emulate_cmpxchg_1_2()
 - drop {cmp}xchg_{relaxed,acquire,release) versions as they aren't needed for Xen
 - update __amoswap_generic() prototype and defintion: drop pre and post barries.
 - update emulate_xchg_1_2() prototype and definion: add lr_sfx, drop pre and post barries.
 - rename __xchg_generic to __xchg(), make __xchg as static inline function to be able to "#ifndef CONFIG_32BIT case 8:... "
---
Changes in V5:
 - update the commit message.
 - drop ALIGN_DOWN().
 - update the definition of emulate_xchg_1_2():
   - lr.d -> lr.w, sc.d -> sc.w.
   - drop ret argument.
   - code style fixes around asm volatile.
   - update prototype.
   - use asm named operands.
   - rename local variables.
   - add comment above the macros
 - update the definition of __xchg_generic:
   - rename to __xchg()
   - transform it to static inline
   - code style fixes around switch()
   - update prototype.
 - redefine cmpxchg()
 - update emulate_cmpxchg_1_2():
   - update prototype
   - update local variables names and usage of them
   - use name asm operands.
   - add comment above the macros
 - drop pre and post, and use .aq,.rl, .aqrl suffixes.
 - drop {cmp}xchg_{relaxed, aquire, release} as they are not used by Xen.
 - drop unnessary details in comment above emulate_cmpxchg_1_2()
---
Changes in V4:
 - Code style fixes.
 - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
   was removed at the end of asm instruction.
 - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
 - drop cmpxcg{32,64}_{local} as they aren't used.
 - introduce generic version of xchg_* and cmpxchg_*.
 - update the commit message.
---
Changes in V3:
 - update the commit message
 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
 - update the comment at the top of the header.
 - change xen/lib.h to xen/bug.h.
 - sort inclusion of headers properly.
---
 xen/arch/riscv/include/asm/cmpxchg.h | 241 +++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/config.h  |   2 +
 2 files changed, 243 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
new file mode 100644
index 0000000000..38e241a4e5
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -0,0 +1,241 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2014 Regents of the University of California */
+
+#ifndef _ASM_RISCV_CMPXCHG_H
+#define _ASM_RISCV_CMPXCHG_H
+
+#include <xen/compiler.h>
+#include <xen/lib.h>
+
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#define _amoswap_generic(ptr, new, ret, sfx) \
+    asm volatile ( \
+        " amoswap" sfx " %0, %2, %1" \
+        : "=r" (ret), "+A" (*(ptr)) \
+        : "r" (new) \
+        : "memory" );
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits.
+ * - for 2-byte xchg access the containing word by clearing bit 1.
+ *
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */
+#define emulate_xchg_1_2(ptr, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    unsigned int new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)(ptr) & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
+        "   andn  %[scratch], %[old], %[mask]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        : [old] "=&r" (old), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [new_] "rJ" (new_), [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((old & mask) >> new_val_bit); \
+})
+
+/*
+ * This function doesn't exist, so you'll get a linker error
+ * if something tries to do an invalid xchg().
+ */
+extern unsigned long __bad_xchg(volatile void *ptr, unsigned int size);
+
+static always_inline unsigned long __xchg(volatile void *ptr,
+                                          unsigned long new,
+                                          unsigned int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_xchg_1_2((volatile uint8_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_xchg_1_2((volatile uint16_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 4:
+        _amoswap_generic((volatile uint32_t *)ptr, new, ret, ".w.aqrl");
+        break;
+#ifndef CONFIG_RISCV_32
+    case 8:
+        _amoswap_generic((volatile uint64_t *)ptr, new, ret, ".d.aqrl");
+        break;
+#endif
+    default:
+        return __bad_xchg(ptr, size);
+    }
+
+    return ret;
+}
+
+#define xchg(ptr, x) \
+({ \
+    __typeof__(*(ptr)) n_ = (x); \
+    (__typeof__(*(ptr))) \
+        __xchg(ptr, (unsigned long)n_, sizeof(*(ptr))); \
+})
+
+#define _generic_cmpxchg(ptr, old, new, lr_sfx, sc_sfx) \
+ ({ \
+    unsigned int rc; \
+    unsigned long ret; \
+    unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0); \
+    asm volatile ( \
+        "0: lr" lr_sfx " %[ret], %[ptr_]\n" \
+        "   and  %[ret], %[ret], %[mask]\n" \
+        "   bne  %[ret], %z[old_], 1f\n" \
+        "   sc" sc_sfx " %[rc], %z[new_], %[ptr_]\n" \
+        "   bnez %[rc], 0b\n" \
+        "1:\n" \
+        : [ret] "=&r" (ret), [rc] "=&r" (rc), [ptr_] "+A" (*ptr) \
+        : [old_] "rJ" (old), [new_] "rJ" (new), [mask] "r" (mask)  \
+        : "memory" ); \
+    ret; \
+ })
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits
+ * - for 2-byte xchg ccess the containing word by clearing first bit.
+ * 
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ *
+ * old_val was casted to unsigned long for cmpxchgptr()
+ */
+#define emulate_cmpxchg_1_2(ptr, old, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    uint8_t new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int old_ = (old) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old_val; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)ptr & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[scratch], %[ptr_]\n" \
+        "   and  %[old_val], %[scratch], %[mask]\n" \
+        "   bne  %[old_val], %z[old_], 1f\n" \
+        /* the following line is an equivalent to: \
+         *     scratch = old_val & ~mask; \
+         * And to elimanate one ( likely register ) input it was decided \
+         * to use: \
+         *     scratch = old_val ^ scratch \
+         */ \
+        "   xor  %[scratch], %[old_val], %[scratch]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        "1:\n" \
+        : [old_val] "=&r" (old_val), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [old_] "rJ" (old_), [new_] "rJ" (new_), \
+          [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((unsigned long)old_val >> new_val_bit); \
+})
+
+/*
+ * This function doesn't exist, so you'll get a linker error
+ * if something tries to do an invalid cmpxchg().
+ */
+extern unsigned long __bad_cmpxchg(volatile void *ptr, unsigned int size);
+
+/*
+ * Atomic compare and exchange.  Compare OLD with MEM, if identical,
+ * store NEW in MEM.  Return the initial value in MEM.  Success is
+ * indicated by comparing RETURN with OLD.
+ */
+static always_inline unsigned long __cmpxchg(volatile void *ptr,
+                                             unsigned long old,
+                                             unsigned long new,
+                                             unsigned int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_cmpxchg_1_2((volatile uint8_t *)ptr, old, new,
+                                  ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_cmpxchg_1_2((volatile uint16_t *)ptr, old, new,
+                                   ".aq", ".aqrl");
+        break;
+    case 4:
+        ret = _generic_cmpxchg((volatile uint32_t *)ptr, old, new,
+                          ".w.aq", ".w.aqrl");
+        break;
+#ifndef CONFIG_32BIT
+    case 8:
+        ret = _generic_cmpxchg((volatile uint64_t *)ptr, old, new,
+                           ".d.aq", ".d.aqrl");
+        break;
+#endif
+    default:
+        return __bad_cmpxchg(ptr, size);
+    }
+
+    return ret;
+}
+
+#define cmpxchg(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr))) \
+    __cmpxchg(ptr, (unsigned long)o_, (unsigned long)n_, \
+              sizeof(*(ptr))); \
+})
+
+#endif /* _ASM_RISCV_CMPXCHG_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index c5f93e6a01..50583aafdc 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -119,6 +119,8 @@
 
 #define BITS_PER_LLONG 64
 
+#define BITS_PER_BYTE 8
+
 /* xen_ulong_t is always 64 bits */
 #define BITS_PER_XEN_ULONG 64
 
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724184.1129412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3K-0003HV-6F; Fri, 17 May 2024 13:55:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724184.1129412; Fri, 17 May 2024 13:55:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3K-0003HD-1n; Fri, 17 May 2024 13:55:18 +0000
Received: by outflank-mailman (input) for mailman id 724184;
 Fri, 17 May 2024 13:55:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3H-0001sq-PW
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:15 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15c412a7-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:14 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59b097b202so368001466b.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:14 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15c412a7-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954113; x=1716558913; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VaYdqVd0N5jds1aXT3FGFFYweAq5RLcPelehJE7+BNc=;
        b=PjT6iFQ0b9MePFbncqWjscHjBuDgXAWOgW9F0QzRzI8eQkhyti+29Io7o9wS+xvjfk
         Ns690r0Su2dvxNlJjqxxNt5bK8iS3B4oDyTpFk9+bJ5iGyMW8IOtaf9KsT09wulhza47
         77ez+alQGdmHP4NmKz0yb/LbdlaMIVEUItn8eT07pUmbA8LMUtGmURxcU8NrJbUsl6sV
         K0Q0DrFG5UJHeavj/ZuRaDSzkaAX7bDqJcSyN0mtRFQFngmf+k35qP3XvS7ChY3eJYHy
         G6PUYqreyQMr5fqpI7kedqsLGAIXE8LPeTE5Fyzvi3ld8SzFwSNgxzB2oggP5115re/0
         HfKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954113; x=1716558913;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VaYdqVd0N5jds1aXT3FGFFYweAq5RLcPelehJE7+BNc=;
        b=HxzpZXwSMGWyUEKV3AFUf7IkQHQ+bHUrcKWbnsphPpxwclp/cXxHNVsvLDstMMNGL6
         ozqv5+g1BJH8AoWYOwYGTazC3ODfNfx6sqKF05iMOFAZjDC9+Y1N7UPQm9y1a3EpVaBg
         jD6vbyh/VlSbVr2LN3ikzURR9a+zFSMzFeffJiSV1ga6aw3SHbKLv1YKmWcJMCsKCiYw
         BLF/IfMi02mHbPpAngw2Ifb7Fv0C4a6zFHVz3nscLGKHGLbIzjmB/Tp7dsLkcWGGH6Hv
         2dYNtxKtKF5jcdrH1zk5EkmjNtOK952z7obSsMAm6yezjcYXOpkoKANw6FXk9D4SY4xx
         vOMg==
X-Gm-Message-State: AOJu0Yz3lfbmuVI/j0FR9Gp3VjXwN1OeXfq+hF6povOmeCHHuq9blg1Y
	538xl/FVOzQPrmHaI/eFiW4vilNRN4c43+TgZ1u9jzoPsBtFynpSlJn17nqr
X-Google-Smtp-Source: AGHT+IGy29899FZzaOg6+XBBOUVhaI5Uf1BV0Etf4zSaNRlkTevfqHswC0eilVvkraM/HFGmuZ4Euw==
X-Received: by 2002:a17:907:7b8a:b0:a59:9b8e:aa61 with SMTP id a640c23a62f3a-a5a2d5c9303mr1980189966b.35.1715954113608;
        Fri, 17 May 2024 06:55:13 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v10 07/14] xen/riscv: introduce monitor.h
Date: Fri, 17 May 2024 15:54:56 +0200
Message-ID: <bb75ebc8bd75318759b450d398294eee8ed3ca55.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4-V10:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - new patch.
---
 xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/monitor.h

diff --git a/xen/arch/riscv/include/asm/monitor.h b/xen/arch/riscv/include/asm/monitor.h
new file mode 100644
index 0000000000..f4fe2c0690
--- /dev/null
+++ b/xen/arch/riscv/include/asm/monitor.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_MONITOR_H__
+#define __ASM_RISCV_MONITOR_H__
+
+#include <xen/bug.h>
+
+#include <asm-generic/monitor.h>
+
+struct domain;
+
+static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+#endif /* __ASM_RISCV_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724185.1129418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3K-0003Lh-MG; Fri, 17 May 2024 13:55:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724185.1129418; Fri, 17 May 2024 13:55:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3K-0003K5-Ea; Fri, 17 May 2024 13:55:18 +0000
Received: by outflank-mailman (input) for mailman id 724185;
 Fri, 17 May 2024 13:55:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3I-0001sq-Pp
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:16 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15822ea0-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:13 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a5a7d28555bso515552566b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:13 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15822ea0-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954113; x=1716558913; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8LVIzWnb4Mr13ThHiKzAiz89V4t6vZ0dpQ2obZh+rUk=;
        b=ITTxY2viO/+UKk3MDwEbJSSFZW0AR1oPHmvvWHHcc4DQhwt05gueE5P9e6EqNdESf1
         gbdPq/+tq2JYNvskErUM0OhPsk9xvxjnhWUW1LIZtfgRsyOy2dVJxk/3YcaWKB8uiqif
         QOnQf4D+o7GC8+SJsxUrXRFk6yRhFwHQEoYYaWu51cpJ8/03TJbNN4m7IVfI1y4nJ3O6
         DO/wfiAseNGV26fWQdehovZKVihfjnzkrPQnPj/j3qfo0An+ZS4SSXObD/5TNpk5ixY9
         oCEOZFxYytl60TA8Ecu7J3J1Y8hoQpfxuEqaH2pvFadzK3aIgcYR9QSLZyCV/WzZdFPj
         K5yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954113; x=1716558913;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8LVIzWnb4Mr13ThHiKzAiz89V4t6vZ0dpQ2obZh+rUk=;
        b=DgF62gi/KLPS4W2AGdLVZ04scAzuuhL5gKUKsT/Y99k4vF0tdrlICVRxRUjWXC47Xn
         BhbuHjFAGwLpQroIg2lL5oGq7Gfk5PJnmLFLyE36XfuH1dtjAFyt9Rz8fT7YP+U+VFc4
         v0qD/thnbhzsDtkqirLNj+az9DQa1WbTqv5ERw20rXTkEOFaA3y4hf2GADzk8TxnJgsW
         iXN8VvufIm5LywGBMwm8Y3aNlQsoouy3Ts9zznNZa0wQDpuhra6YWoofvPSnzWoKVy8A
         om+h3YupGhE3hAWe43YWa5k+cOBDo3erTXgEuzsBFovckw+jhUeKzBmxFOjMsoACe+ZV
         7eWQ==
X-Gm-Message-State: AOJu0Yy9ok8EAnQhOSO3Rb8Y0HuEzBmL94dcZqWLMKqJ9fSMNlDWuBXY
	KgFm5MQHGqCFY0VG1x4speDB3W07dZqjqR4Km4q1ibE0bqqQJkFwNcbJjXNc
X-Google-Smtp-Source: AGHT+IEqB4puYJmNpx00kOKICGOKy52EQdfcxH6ptgJHjislAjztsRVBNQ15+p5HIZKWgzGXhdY36A==
X-Received: by 2002:a17:907:130f:b0:a5c:df6b:a9b5 with SMTP id a640c23a62f3a-a5cdf6bade3mr382475466b.59.1715954112729;
        Fri, 17 May 2024 06:55:12 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 06/14] xen/riscv: introduce atomic.h
Date: Fri, 17 May 2024 15:54:55 +0200
Message-ID: <c54eeba8741d3ef4298b471133ce526cf3d3f4eb.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Initially the patch was introduced by Bobby, who takes the header from
Linux kernel.

The following changes were done on top of Bobby's changes:
 - atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
   to use__*xchg_generic()
 - drop casts in write_atomic() as they are unnecessary
 - drop introduction of WRITE_ONCE() and READ_ONCE().
   Xen provides ACCESS_ONCE()
 - remove zero-length array access in read_atomic()
 - drop defines similar to pattern:
   #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
 - drop  atomic##prefix##_{cmp}xchg_{release, aquire, release}() as they
   are not used in Xen.
 - update the defintion of  atomic##prefix##_{cmp}xchg according to
   {cmp}xchg() implementation in Xen.
 - some ATOMIC_OP() macros were updated:
   - drop size argument for ATOMIC_OP which defines atomic##prefix##_xchg()
     and atomic##prefix##_cmpxchg().
   - drop c_op argument for ATOMIC_OPS which defines ATOMIC_OPS(and, and),
     ATOMIC_OPS( or,  or), ATOMIC_OPS(xor, xor), ATOMIC_OPS(add, add, +),
     ATOMIC_OPS(sub, add, -) as c_op is always "+" for them.
   - drop "" from definition of __atomic_{acquire/release"}_fence.

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
              setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Bobby Eshleman <bobbyeshleman@gmail.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V10:
 - drop unnessary parentheses around p in add_sized()
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V9:
 - update the defintion of write_atomic macros:
   drop the return value as this macros isn't expeceted to return something
   drop unnessary parentheses around p.
 - drop casts inside _add_sized() for ptr variable as they aren't necessary.
---
Changes in V8:
 - ???? add the explanatory comment to _add_sized().
 - drop "" in __atomic_{acquire, release}_fence().
 - code style fixes in atomic##prefix##_##op##_return(): indentation.
 - drop an unary_op argument ("+") for ATOMIC_OPS(and, and), ATOMIC_OPS( or,  or), ATOMIC_OPS(xor, xor)
   and use "+" directly inside definition of ATOMIC_OPS().
 - drop c_op for ATOMIC_OPS(add, add, +) and ATOMIC_OPS(sub, add, -) as it is always "+" for now.
   Just use "+" inside definition of ATOMIC_OPS().
 - drop size argument for ATOMIC_OP() defintions of atomic##prefix##_{xchg,cmpxchg}()
 - update the commit message.
---
Changes in V7:
 - drop relaxed version of atomic ops as they are not used.
 - update the commit message
 - code style fixes
 - refactor functions write_atomic(), add_sized() to be able to use #ifdef CONFIG_RISCV_32 ... #endif
   for {write,read}q().
 - update ATOMIC_OPS to receive unary operator.
 - update the header on top of atomic-ops.h.
 - some minor movements of function inside atomic-ops.h header.
---
Changes in V6:
 - drop atomic##prefix##_{cmp}xchg_{release, aquire, relaxed} as they aren't used
   by Xen
 - code style fixes.
 - %s/__asm__ __volatile__/asm volatile
 - add explanational comments.
 - move inclusion of "#include <asm-generic/atomic-ops.h>" further down in atomic.h
   header.
---
Changes in V5:
 - fence.h changes were moved to separate patch as patches related to io.h and cmpxchg.h,
   which are dependecies for this patch, also needed changes in fence.h
 - remove accessing of zero-length array
 - drops cast in write_atomic()
 - drop introduction of WRITE_ONCE() and READ_ONCE().
 - drop defines similar to pattern #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - Xen code style fixes
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
---
Changes in V4:
 - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
 - drop casts in read_atomic_size(), write_atomic(), add_sized()
 - tabs -> spaces
 - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
   the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
  - update the commit message
  - add SPDX for fence.h
  - code style fixes
  - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
  - re-order the patch
  - merge to this patch fence.h
---
Changes in V2:
 - Change an author of commit. I got this header from Bobby's old repo.
---
 xen/arch/riscv/include/asm/atomic.h  | 280 +++++++++++++++++++++++++++
 xen/include/asm-generic/atomic-ops.h |  97 ++++++++++
 2 files changed, 377 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/include/asm-generic/atomic-ops.h

diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
new file mode 100644
index 0000000000..554bcc2087
--- /dev/null
+++ b/xen/arch/riscv/include/asm/atomic.h
@@ -0,0 +1,280 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Taken and modified from Linux.
+ *
+ * The following changes were done:
+ * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
+ *     to use__*xchg_generic()
+ * - drop casts in write_atomic() as they are unnecessary
+ * - drop introduction of WRITE_ONCE() and READ_ONCE().
+ *   Xen provides ACCESS_ONCE()
+ * - remove zero-length array access in read_atomic()
+ * - drop defines similar to pattern
+ *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
+ * - move not RISC-V specific functions to asm-generic/atomics-ops.h
+ * 
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2024 Vates SAS
+ */
+
+#ifndef _ASM_RISCV_ATOMIC_H
+#define _ASM_RISCV_ATOMIC_H
+
+#include <xen/atomic.h>
+
+#include <asm/cmpxchg.h>
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+void __bad_atomic_size(void);
+
+/*
+ * Legacy from Linux kernel. For some reason they wanted to have ordered
+ * read/write access. Thereby read* is used instead of read*_cpu()
+ */
+static always_inline void read_atomic_size(const volatile void *p,
+                                           void *res,
+                                           unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: *(uint8_t *)res = readb(p); break;
+    case 2: *(uint16_t *)res = readw(p); break;
+    case 4: *(uint32_t *)res = readl(p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: *(uint32_t *)res = readq(p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define read_atomic(p) ({                                   \
+    union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_;   \
+    read_atomic_size(p, x_.c, sizeof(*(p)));                \
+    x_.val;                                                 \
+})
+
+static always_inline void _write_atomic(volatile void *p,
+                                       unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: writeb(x, p); break;
+    case 2: writew(x, p); break;
+    case 4: writel(x, p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: writeq(x, p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define write_atomic(p, x)                              \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _write_atomic(p, x_, sizeof(*(p)));                 \
+})
+
+static always_inline void _add_sized(volatile void *p,
+                                     unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1:
+    {
+        volatile uint8_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+    case 2:
+    {
+        volatile uint16_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+    case 4:
+    {
+        volatile uint32_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+#ifndef CONFIG_RISCV_32
+    case 8:
+    {
+        volatile uint64_t *ptr = p;
+        write_atomic(ptr, read_atomic(ptr) + x);
+        break;
+    }
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define add_sized(p, x)                                 \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _add_sized(p, x_, sizeof(*(p)));                    \
+})
+
+#define __atomic_acquire_fence() \
+    asm volatile ( RISCV_ACQUIRE_BARRIER ::: "memory" )
+
+#define __atomic_release_fence() \
+    asm volatile ( RISCV_RELEASE_BARRIER ::: "memory" )
+
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set.  These don't return anything, so there's only
+ * one version to worry about.
+ */
+#define ATOMIC_OP(op, asm_op, unary_op, asm_type, c_type, prefix)  \
+static inline                                               \
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{                                                           \
+    asm volatile (                                          \
+        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
+        : "+A" (v->counter)                                 \
+        : "r" (unary_op i)                                  \
+        : "memory" );                                       \
+}                                                           \
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument for ATOMIC_OP isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, unary_op)                    \
+        ATOMIC_OP (op, asm_op, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +)
+ATOMIC_OPS(sub, add, -)
+ATOMIC_OPS(and, and, +)
+ATOMIC_OPS( or,  or, +)
+ATOMIC_OPS(xor, xor, +)
+
+#undef ATOMIC_OP
+#undef ATOMIC_OPS
+
+#include <asm-generic/atomic-ops.h>
+
+/*
+ * Atomic ops that have ordered variant.
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#define ATOMIC_FETCH_OP(op, asm_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{                                                                   \
+    register c_type ret;                                            \
+    asm volatile (                                                  \
+        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (unary_op i)                                          \
+        : "memory" );                                               \
+    return ret;                                                     \
+}
+
+#define ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
+{                                                                       \
+    return atomic##prefix##_fetch_##op(i, v) c_op (unary_op i);         \
+}
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, unary_op)                        \
+        ATOMIC_FETCH_OP( op, asm_op,    unary_op, w, int,   )   \
+        ATOMIC_OP_RETURN(op, asm_op, +, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +)
+ATOMIC_OPS(sub, add, -)
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, asm_op) \
+        ATOMIC_FETCH_OP(op, asm_op, +, w, int,   )
+
+ATOMIC_OPS(and, and)
+ATOMIC_OPS( or,  or)
+ATOMIC_OPS(xor, xor)
+
+#undef ATOMIC_OPS
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+
+/* This is required to provide a full barrier on success. */
+static inline int atomic_add_unless(atomic_t *v, int a, int u)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   beq      %[p],  %[u], 1f\n"
+        "   add      %[rc], %[p], %[a]\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [a] "r" (a), [u] "r" (u)
+        : "memory");
+    return prev;
+}
+
+static inline int atomic_sub_if_positive(atomic_t *v, int offset)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   sub      %[rc], %[p], %[o]\n"
+        "   bltz     %[rc], 1f\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [o] "r" (offset)
+        : "memory" );
+    return prev - offset;
+}
+
+/*
+ * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return.
+ */
+#define ATOMIC_OP(c_t, prefix)                                  \
+static inline                                                   \
+c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
+{                                                               \
+    return __xchg(&v->counter, n, sizeof(c_t));                 \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
+{                                                               \
+    return __cmpxchg(&v->counter, o, n, sizeof(c_t));           \
+}
+
+#define ATOMIC_OPS() \
+    ATOMIC_OP(int,   )
+
+ATOMIC_OPS()
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP
+
+#endif /* _ASM_RISCV_ATOMIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-generic/atomic-ops.h b/xen/include/asm-generic/atomic-ops.h
new file mode 100644
index 0000000000..98dd907942
--- /dev/null
+++ b/xen/include/asm-generic/atomic-ops.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * The header provides default implementations for every xen/atomic.h-provided
+ * forward inline declaration that can be synthesized from other atomic
+ * functions or being created from scratch.
+ */
+#ifndef _ASM_GENERIC_ATOMIC_OPS_H_
+#define _ASM_GENERIC_ATOMIC_OPS_H_
+
+#include <xen/atomic.h>
+#include <xen/lib.h>
+
+#ifndef ATOMIC_READ
+static inline int atomic_read(const atomic_t *v)
+{
+    return ACCESS_ONCE(v->counter);
+}
+#endif
+
+#ifndef _ATOMIC_READ
+static inline int _atomic_read(atomic_t v)
+{
+    return v.counter;
+}
+#endif
+
+#ifndef ATOMIC_SET
+static inline void atomic_set(atomic_t *v, int i)
+{
+    ACCESS_ONCE(v->counter) = i;
+}
+#endif
+
+#ifndef _ATOMIC_SET
+static inline void _atomic_set(atomic_t *v, int i)
+{
+    v->counter = i;
+}
+#endif
+
+#ifndef ATOMIC_SUB_AND_TEST
+static inline int atomic_sub_and_test(int i, atomic_t *v)
+{
+    return atomic_sub_return(i, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC_AND_TEST
+static inline int atomic_inc_and_test(atomic_t *v)
+{
+    return atomic_add_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC
+static inline void atomic_inc(atomic_t *v)
+{
+    atomic_add(1, v);
+}
+#endif
+
+#ifndef ATOMIC_INC_RETURN
+static inline int atomic_inc_return(atomic_t *v)
+{
+    return atomic_add_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC
+static inline void atomic_dec(atomic_t *v)
+{
+    atomic_sub(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_RETURN
+static inline int atomic_dec_return(atomic_t *v)
+{
+    return atomic_sub_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_AND_TEST
+static inline int atomic_dec_and_test(atomic_t *v)
+{
+    return atomic_sub_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_ADD_NEGATIVE
+static inline int atomic_add_negative(int i, atomic_t *v)
+{
+    return atomic_add_return(i, v) < 0;
+}
+#endif
+
+#endif /* _ASM_GENERIC_ATOMIC_OPS_H_ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724186.1129425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3L-0003Uc-6B; Fri, 17 May 2024 13:55:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724186.1129425; Fri, 17 May 2024 13:55:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3K-0003Tc-SO; Fri, 17 May 2024 13:55:18 +0000
Received: by outflank-mailman (input) for mailman id 724186;
 Fri, 17 May 2024 13:55:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3J-0001sq-Pp
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:17 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 163a8bba-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:15 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-574f7c0bab4so5527038a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:15 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 163a8bba-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954114; x=1716558914; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=IyBSZPlFHowbije2gbqJW9jX49ovLzWsyAVo7e+9BiA=;
        b=L/z7WBYDgtlz3Lq4W6ov/xA4GNcwQVp/Zjhg1t3/AkqFmrIRlQYKGepOR5nbkZSkrv
         qC/Hkw+79y8BWMrEBThMFFQZGdFJtggHDq8U52+sMrdivxRRCVAptk7w/v2kAjj47tef
         Fzb7cxpQNgHLGz7HiwtcyHkQuGi7AQNAiqLqmVnp/SCn7mfZKdv055ItSlIf9yapoKk5
         bLqqTbHMfZkG/UkVHyGTdBKh4R6vWcb7xTXmGbETWNQhKmAZXFjWOWkAGvHG9EM5q2Yn
         4qY7kQI/OPI0vlkgd1LNHKaHEnJJJ/xU1cs3ttbWIsmizrtWjLIFh7v7OYAvWtPGw7e+
         pyuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954114; x=1716558914;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=IyBSZPlFHowbije2gbqJW9jX49ovLzWsyAVo7e+9BiA=;
        b=ZgaqBM5ym7KUFoxAkmCb7tm6Xl3ZF59uh9xLW4jdZF6D6+Sw3l7vmlxbRYdC9bfwAC
         pZD4FdQGHhieJcv7NLHrlT6WX2bec75We9u9+0qLgjzCsSRaFIE29GrHMRrI60mLDZho
         vQEN8icbC4ypMRKn1w+Ly0PJX+vlArFss9UvVIRDK4YWXTCuoGHWnuizEtHYYVB5cQs0
         lGyW+sihN4Zoz1RddY6zJO8t0WGcRQhrXqJK6kfx6YQPFRQZ+csrIzrkGvVGtvoYoCJE
         w9yDsYDQ/12SocHYqAtVTpnZhlOY8E4QsVOHq6q2t6RNM5DIN2MHh/ajg8KLuRS4yU7X
         3X8g==
X-Gm-Message-State: AOJu0YwK6rEERgHvvAF40wmJ3qyGkwKzelZObnCcYq6S2k4suFoT6anl
	DanyIkZNtbDI6vqhhLsUgjB4HsDYdzAfSAjR8w/wMf+UCGA+iP5hMTw7xKKp
X-Google-Smtp-Source: AGHT+IEH4GixmiF/ZnYd2IrHPQ6pJGJyu1Ytdqrigw3Z/aFxvE19tc1zBfWvqXV4dA53oYRV9fQxng==
X-Received: by 2002:a17:906:240e:b0:a5c:dce0:9f4e with SMTP id a640c23a62f3a-a5cdce0a0camr589738166b.28.1715954114429;
        Fri, 17 May 2024 06:55:14 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 08/14] xen/riscv: add definition of __read_mostly
Date: Fri, 17 May 2024 15:54:57 +0200
Message-ID: <834486ff8e3233e067a0e4603f700182826eca94.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

Considering that there is still no still final decision regarding patch [2] my suggestion
is to merge RISC-V specific patch and just drop the changes in patch [2].

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V9-V10:
 - Only rebase was done.
---
Change in V8:
 - update the footer after Signed-off.
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724187.1129437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3M-0003u3-OH; Fri, 17 May 2024 13:55:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724187.1129437; Fri, 17 May 2024 13:55:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3M-0003sm-Fb; Fri, 17 May 2024 13:55:20 +0000
Received: by outflank-mailman (input) for mailman id 724187;
 Fri, 17 May 2024 13:55:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3J-0001cB-V2
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:18 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17726393-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:17 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a5d02b45262so123719866b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:17 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17726393-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954116; x=1716558916; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KU2J4ehhQREX9aohkUCxXxXEnfxAhv6f1eRMLAoyz2s=;
        b=bcWEAmwolFHOUgLezjT2aYjSLvByXK6P5lGpMrXdz234jNQlyduEgQBURmm71/Z0i6
         nkE8zbosgP/8NAl7m9b4X3AorpTPGejBV6oB2HfwFsAhP7D5bnYXAIPmgY/3lRkUZpDB
         7wlB+K5Bg9+Aj0YPUNSJjKh8ZF6N/8vgv3eoQo8MkCRx4HT0nGXeUWoUdlYz81e4ect3
         4iLL3ei0YSqTnPt4TF6nR4snOOpAlMCWz6iO5mvweI+NBrABQ6SJb1DQ2DG/Waxhb8eP
         fuxg3E1W1llLWbnrNPeTojCMjJT+EKXuNX2gG1ydkBRf6pJhYY45IglPEq7JcWGF1Ikv
         NFqg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954116; x=1716558916;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KU2J4ehhQREX9aohkUCxXxXEnfxAhv6f1eRMLAoyz2s=;
        b=nIM86d3EgE90lNOplA3t2KnR9NwveQh3cFNwP5gzIn/9Q/rlXA1Qsmvn8dmTfgSSjS
         cJOaJo/+TPkwhMJ80Zc1PBc8H/l9rkv0FYsguPmDjDzcu3bTaMrgxRtFg+PVc/m6Sah1
         S5HhJKUD4brzAGB1J3cC+u9mIg5EMHTU8rmfMpErVnhWdzMf0vqxhbAQUpFiWIRTMyfL
         FlTHAo2q38JvzrioEk9DmwlTLcQlspJfaK9juzWQaxeEdRPnRGjd60sjC4jtqpbiCURe
         ntynKTUBTeMi17tt2wgdy3NUUY3ihLnB5hW2jZ8QIbHH7lQUBSjZPHYy/+t343xQRi6a
         SRcg==
X-Gm-Message-State: AOJu0YyB+15O1p8vhHFDgtNXieBaxcPZ97ZaQfF2kYEHwCeB2cHl8KD6
	CfVto07+1BAOgEH/ltEDgxbJRAn6svwhxL5H8PG4Z68xTOINhqtL7F/m6+jF
X-Google-Smtp-Source: AGHT+IHmsoziuc4fyB4Cz+upZaGGb/0p3VflZy9DZeEXYmYAAXv0aK0Ua4LT0M+a5btqqD1Zjya7hQ==
X-Received: by 2002:a17:907:36f:b0:a52:6fcb:564a with SMTP id a640c23a62f3a-a5a2d53aa70mr1484720766b.9.1715954116246;
        Fri, 17 May 2024 06:55:16 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 10/14] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Fri, 17 May 2024 15:54:59 +0200
Message-ID: <c9b8a462e85cc11f60391b6eff2c7cb5db9903ad.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V8-V10:
 - Nothing changed only rebase.
---
Changes in V7:
 - update argument type of maddr_to_virt() function: unsigned long -> paddr_t
 - rename argument of PFN_ORDER(): pfn -> pg.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V6:
 - drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
 - parenthesize va in definition of vmap_to_mfn().
 - Code style fixes.
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 242 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..cc4a07a71c 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,246 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724188.1129441 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3N-00041H-5N; Fri, 17 May 2024 13:55:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724188.1129441; Fri, 17 May 2024 13:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3M-0003z6-Ug; Fri, 17 May 2024 13:55:20 +0000
Received: by outflank-mailman (input) for mailman id 724188;
 Fri, 17 May 2024 13:55:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3K-0001sq-QG
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:18 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 16e13c8c-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:16 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51fb14816f6so766853e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:16 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16e13c8c-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954115; x=1716558915; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=fTDNSGKQqAKoOtL7zFg1D0HVcpP6MEv+cyhDVsjtWjY=;
        b=JVTxYvK83JKiKdJ64CMxW7rdDY+W76aVxt1kTSkyehriTX6p72tuwgl1/FlppTQIBe
         lw8Y/SZ6SoVOh9NC1aYhrrh8fGCo3fbWEjIMWw6IAxEjVq0OEl5hTyQayvK8b5YYedij
         qzv/90mh1xYg30eEHe0lafBl4gzlHziqYbZoRgh6dJzjoI7g5JTlzoHY0Gwa1KW9LEpg
         hccpGq6buzx2wYZ3oWkFBmGTosbuZh4BaHAv4YNG8S/qSe+VDUGfF2g6h/x6yoYRgjcz
         yj8lDjjfi1GlnjkKwI0JDwhdMnWO5cavTXqN+HLaDNqKpcPiaLioFotepDpl2WCGTgw+
         WUJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954115; x=1716558915;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=fTDNSGKQqAKoOtL7zFg1D0HVcpP6MEv+cyhDVsjtWjY=;
        b=cqAYvDEQJ6BY9Hzhk9I9ApMwd9I1oTTgs7wmr3xfg+BUzJEl2ex5hSOJj3Mhs0biBy
         zfhjuKMxc8NkuiosiZ9bEggZJQqEroYTG5Ol2d+ra3WPVnZ3HdAnFT2XhR9iZlNZaPmG
         WHtdS/USuukxXGM50nb2eZEeTEF959oU+HLtZftsbhaH/yk1HmL7+K8gRJwfS2HNHFpA
         fZm0LNeX2IH/jaUcarbIhhmtJC7jY0sFsMfWDtPpZYXDi4haALCINIp5SDk3KrW4bJiK
         +jpTcUkWnYKgZMf/6EiJREDkLXSpFM0p6mLfh+m6GWwqEDl4KdLHEKIx3/CIp5BTn/QB
         QUGA==
X-Gm-Message-State: AOJu0YyZbZUIk2fR6YLumY4k7CM/hR8rvxbzFPQzjQGe2RxwNFw0+sg+
	kv66e5dEb9xi60Wv74Db/eYBiCS2NpnU31aUb0LDebtrQXk7UySHWFeuMIcV
X-Google-Smtp-Source: AGHT+IGRn6SScVZRY3a4dnKl4tKASVwrpUz5YO+fZnCSMnLLdTKOL18B0Ez524cvClsoV2oad3TtLg==
X-Received: by 2002:a05:6512:2356:b0:519:6953:2ffc with SMTP id 2adb3069b0e04-5221007027amr17339627e87.42.1715954115225;
        Fri, 17 May 2024 06:55:15 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 09/14] xen/riscv: add required things to current.h
Date: Fri, 17 May 2024 15:54:58 +0200
Message-ID: <6ba3a3f673e54c4235fe3040699d0ce5b52234f4.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add minimal requied things to be able to build full Xen.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V10:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - BUG() was changed to BUG_ON("unimplemented");
 - Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - drop a forward declaration of struct vcpu;
 - update guest_cpu_user_regs() macros
 - replace get_processor_id with smp_processor_id
 - update the commit message
 - code style fixes
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/current.h b/xen/arch/riscv/include/asm/current.h
index d84f15dc50..aedb6dc732 100644
--- a/xen/arch/riscv/include/asm/current.h
+++ b/xen/arch/riscv/include/asm/current.h
@@ -3,6 +3,21 @@
 #ifndef __ASM_CURRENT_H
 #define __ASM_CURRENT_H
 
+#include <xen/lib.h>
+#include <xen/percpu.h>
+#include <asm/processor.h>
+
+#ifndef __ASSEMBLY__
+
+/* Which VCPU is "current" on this PCPU. */
+DECLARE_PER_CPU(struct vcpu *, curr_vcpu);
+
+#define current            this_cpu(curr_vcpu)
+#define set_current(vcpu)  do { current = (vcpu); } while (0)
+#define get_cpu_current(cpu)  per_cpu(curr_vcpu, cpu)
+
+#define guest_cpu_user_regs() ({ BUG_ON("unimplemented"); NULL; })
+
 #define switch_stack_and_jump(stack, fn) do {               \
     asm volatile (                                          \
             "mv sp, %0\n"                                   \
@@ -10,4 +25,8 @@
     unreachable();                                          \
 } while ( false )
 
+#define get_per_cpu_offset() __per_cpu_offset[smp_processor_id()]
+
+#endif /* __ASSEMBLY__ */
+
 #endif /* __ASM_CURRENT_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724189.1129452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3O-0004Az-8p; Fri, 17 May 2024 13:55:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724189.1129452; Fri, 17 May 2024 13:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3N-000489-Je; Fri, 17 May 2024 13:55:21 +0000
Received: by outflank-mailman (input) for mailman id 724189;
 Fri, 17 May 2024 13:55:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3K-0001cB-VD
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:18 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17fcc428-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:18 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-572e48f91e9so5322423a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:18 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17fcc428-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954117; x=1716558917; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aDbZAe0TMDQ2eX40FFZW7nZOljZEfwxgNBXnUjMJMos=;
        b=DCS0HfEobkxMSLXlx1UeDm1ahF8wyQc5Cn4VeKlJ9VvEkdQi6MKlDmqzPjfKaHfuRr
         Sjdh4F44aNNWzzm5kIcXXHVp20mYkv+Abtra5QPOh898i8kJn15+39Z0BP0pOCZk50oB
         EukBXcF6BZcivUw0CfKt5v/JVBh78oC5vxf9dosA50N6PGee3th4bHZ1lU2zfeApejSf
         hITmi022HAF+Xwt5rziYx9/xX/pUnu51kQ+CjUdyJQ8o0t9qQtsbsP5JByQ9blGPJRsR
         zcT7Ac8yPdH2mCzICa5DogPaqza7PqHLHgL2w/fwejhhcutzOwyHADRqWdL6FRtGqjYc
         kxJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954117; x=1716558917;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=aDbZAe0TMDQ2eX40FFZW7nZOljZEfwxgNBXnUjMJMos=;
        b=AUeYd8DhdBkYvlEdn9YYbbfM7uyJ0aJFGbh4ZobVjjyidiy8qvl4rtpeZVjzhx8Bgk
         Uh/2kHNzMIbjtI2z8AN6NIVBOeZSM4lj9E6TgmV6ZrCCTJ7re0iOWLf3QcF1UGd9vNIi
         5jsDSFPC5RPh9A4KLsrtZMCfpNP3GmHtoQ5Fn7+h1UNhouW+I8XtDS8/GpKC4d/nNE+2
         rWfwNr7LvWB03jRYbKblOTYQ4siD4FF2QoAS7awpCN0nO7Z0pwDJ64GlOznZ70NNJulx
         ww0fGOlI3TH10pcVzRi/rfo06VAEynZrTKwhCumO17CM9XWuGfO9ZDD3P3/igGKYoStl
         4vRg==
X-Gm-Message-State: AOJu0Yze98gzQkpTx0PZqg430AdvHEUl1ncOyG4swxDpvGYLmcm6aUpg
	waeuPzqxeU6SCMkOsCpvdar+CKe8cyrAEQ/gtU5KPZzKNS9xag69r3Jdo5w6
X-Google-Smtp-Source: AGHT+IHeeI4nFYYD06k+JYMKCpQ7dw73V97VzJ+EOfcxd9+nEg0sGqkAj/A88VrrcHWIlPoLSXPgyg==
X-Received: by 2002:a17:906:5394:b0:a52:2441:99c with SMTP id a640c23a62f3a-a5a2d67f8c0mr1592579666b.69.1715954117256;
        Fri, 17 May 2024 06:55:17 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
Date: Fri, 17 May 2024 15:55:00 +0200
Message-ID: <050567a35805361edc0e222f4ddebfa91403cfb6.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V10:
 - Only rebase was done.
---
Changes in V4:
  - New patch.
---
 xen/arch/riscv/Makefile   |  1 +
 xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 xen/arch/riscv/vm_event.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 2fefe14e7c..1ed1a8369b 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
 obj-y += traps.o
+obj-y += vm_event.o
 
 $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
diff --git a/xen/arch/riscv/vm_event.c b/xen/arch/riscv/vm_event.c
new file mode 100644
index 0000000000..bb1fc73bc1
--- /dev/null
+++ b/xen/arch/riscv/vm_event.c
@@ -0,0 +1,19 @@
+#include <xen/bug.h>
+
+struct vm_event_st;
+struct vcpu;
+
+void vm_event_fill_regs(struct vm_event_st *req)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_set_registers(struct vcpu *v, struct vm_event_st *rsp)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_monitor_next_interrupt(struct vcpu *v)
+{
+    /* Not supported on RISCV. */
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724190.1129457 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3P-0004J9-02; Fri, 17 May 2024 13:55:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724190.1129457; Fri, 17 May 2024 13:55:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3O-0004G8-Az; Fri, 17 May 2024 13:55:22 +0000
Received: by outflank-mailman (input) for mailman id 724190;
 Fri, 17 May 2024 13:55:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3L-0001cB-VW
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:20 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1879612e-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:18 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51f3a49ff7dso2567301e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:18 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1879612e-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954118; x=1716558918; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VqZoxeXcvwG6qA/xb5N6kbA45uUp111KA2sPPRW9tSw=;
        b=YfjpJ3DD8d5yHl1Y7JGRtcoK5vvYrXqM4QhZ9++KywKJzyveL6gnP7ivCMyUKJBTUN
         oPFp34DbfRG98+5ePTZIybkxa9mRJGafcwfhVEfVFzrdAqwFCrEoxz7K/Bv0mfHRBn90
         xD68AdxYD7Yd/PgkTAeXyLWSYs+8INf2FIN6RPf1oDGALn1kvrqgXyZ5XtGbUEYPWH8F
         wjWIOza1fqpOW43Ppz3AcuFfzM/gZEpjiUNtWulVTDN9GUuBbszrN0n2lTroogPuehkH
         ZqAt2ojNN1CNa/wCazfRIvvsyQLqLdrJh0LJvAdADu2YGLaVF2ey305Wgli6wNYZHnNa
         a50A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954118; x=1716558918;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VqZoxeXcvwG6qA/xb5N6kbA45uUp111KA2sPPRW9tSw=;
        b=BPoB1zpvwczIOfpyMb6Vh7I23U3u5yux5bs9PKxGCPWiIgvgDfsGv69781njK4gtT/
         qeXBjoRJfPtONjJVeTpmsTGHUA0oJZy53Jw5P5RSIy8wecDrtfeTY147t48bVnIfqpEs
         EbDjKB6O5v5v0o2zVMVEQVmkpDtZF1Mty5nl9R14fZOApyhgkI+K8xvHYtiN5HCQwMUr
         VUpYh/9kjZdJTbvxcuRrMGFGW+gt1Zeyjh9YC6nf75WjHdEOlKk3fQZVE8/+fUtSgHG0
         ozUWJvS6H3Ew0jbRjFrFksK6MsdczSqBEUK4GPrmqjTDaAvJlq0mQm+W9KHvJiMxg0qB
         iJRw==
X-Gm-Message-State: AOJu0YyCV05hhOM1OMFYq85AIUDtS1mreJ6wRstETqMa+e//NjmguvO7
	HQJnMCTNzikdt9i0TFECG3HH0HcIhGc3LWVJwfo1WlhuJan+fg2+F9HSRc+w
X-Google-Smtp-Source: AGHT+IHxHmV3A0bcwpvmuR0omarL/kP79NpLAVNtxStRnNja9z9394gkxZ5NreQvFMSAtX5EU/ulbg==
X-Received: by 2002:a05:6512:688:b0:523:dab3:39 with SMTP id 2adb3069b0e04-523dab300c6mr2292690e87.55.1715954118080;
        Fri, 17 May 2024 06:55:18 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 12/14] xen/riscv: add minimal amount of stubs to build full Xen
Date: Fri, 17 May 2024 15:55:01 +0200
Message-ID: <7ecaf3a902db375eec5936ff98c98916e7bc8895.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7-V10:
 - Only rebase was done.
---
Changes in V6:
 - update the commit in stubs.c around /* ... common/irq.c ... */
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---

 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 439 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 523 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..8285bcffef
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, but common/irq.c isn't
+ * built for now. These changes will be removed there when common/irq.c is
+ * ready.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724192.1129468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3R-0004lt-1o; Fri, 17 May 2024 13:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724192.1129468; Fri, 17 May 2024 13:55:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3Q-0004h1-6t; Fri, 17 May 2024 13:55:24 +0000
Received: by outflank-mailman (input) for mailman id 724192;
 Fri, 17 May 2024 13:55:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3N-0001sq-KR
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:21 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1913a501-1455-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 15:55:19 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-571ba432477so5469260a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:19 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1913a501-1455-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954119; x=1716558919; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UdmrjijEC2ETr1rKEtwKxNjLUtWchNNAW7Shb3Uh3Pk=;
        b=FmfbrQDOXUAVP205ADDm0zHkwd0z1HGKQll7Fmt+CrBCNFCy0JJx4LDRcR/2hWmvSe
         4GDemVEUSfzoXCRg01fIZi2LieS36UjI8koO6zX9Acv0nJz7prdQAbkhqyCUoEEehi/f
         t/XopJz2y/89mncWwwYPy88IAJhm02onNvMXzGQddJ+Wg5pD9q/uxC3epkhdSgkiJ1NK
         xKgB3IA2ONN86sgc0o4KWtPh050/R2lknrLarGguzDoephZbHCjLa9nPPl8vS/8kBms5
         6m4RV9l0CN91BSkphw7HYRP+as28kBpmQWiMjqh9Fhfi7NUYbqWUXs/IKQiw0EQaDp8a
         bn3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954119; x=1716558919;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UdmrjijEC2ETr1rKEtwKxNjLUtWchNNAW7Shb3Uh3Pk=;
        b=LkBqNyLsUqhzUO9y64Zro+jhwwMMcnct/ajuy9xSrHx1Dn9LqZB+ESJ6dmUQvakX9k
         jc724cRa1n+sYt1zVV6KXvh/xAyHlaiQJfmVFIW1D4L8jfGg58w4UXvTbjwCqqN4WPgJ
         bF/ZYZXW0Xu9vi3QoS8iTSFDOaOdP2xu1dBAWqdrdW+M5rclCys5zxiZ9c1ESldC8spF
         no1H/s3E0i1X8SvsjYlvOEeVEl1z8gvU2QflBGrvxYSxVdgMGRrt0zSqroJLJNVQssWk
         CtfnS0dl3lZmvL2GBCsgOLwLmDkMGJIBn88sF0wWZF9IsdxQe0lsf+cZiE3JmlDoNxxf
         0cog==
X-Gm-Message-State: AOJu0Yx8TckBoUErKjNe2vST6/zvFcLCALGw/fQ2BzCqqGYSLzYGzA5T
	Ar1iXzlumTDPG+Kpvk5/6XBaZMseVtbLqL+YV9VOoGrqHCW4NtrERE+DTSu9
X-Google-Smtp-Source: AGHT+IHq3shkhryXGz9XXJbte1obJO0HoEvZW7GV+zdjdgt1z8nh7+hS9zDS82ax8/7rp4mc1dsXZQ==
X-Received: by 2002:a17:906:9e08:b0:a5a:8b17:d851 with SMTP id a640c23a62f3a-a5a8b17db00mr585229966b.20.1715954119103;
        Fri, 17 May 2024 06:55:19 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 13/14] xen/riscv: enable full Xen build
Date: Fri, 17 May 2024 15:55:02 +0200
Message-ID: <a406b22d3d53167c22e5fe607c4defb71b4ce1d5.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V10:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  24 -----
 4 files changed, 15 insertions(+), 197 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8c071aff65..17827c302c 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 8285bcffef..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, but common/irq.c isn't
- * built for now. These changes will be removed there when common/irq.c is
- * ready.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:55:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:55:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724193.1129474 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3R-00052t-Tm; Fri, 17 May 2024 13:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724193.1129474; Fri, 17 May 2024 13:55:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y3R-0004x6-9O; Fri, 17 May 2024 13:55:25 +0000
Received: by outflank-mailman (input) for mailman id 724193;
 Fri, 17 May 2024 13:55:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Ubw=MU=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s7y3N-0001cB-W4
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:55:22 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19807cb0-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:55:20 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a5a88339780so485649266b.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:55:20 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c81b4sm1117456466b.113.2024.05.17.06.55.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:55:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19807cb0-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1715954120; x=1716558920; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=mXA5k6NQLwnOQ/WhnpaidF2ChH86fmHaLKiU9LUuBe8=;
        b=Ny2NfuAP0b+VTowOJHcipcbTCVUJ61xXsSivrVxuKUTavxM7ysV4jHVRs6JvvwEKsc
         ME3oLmKCCV75UtBFm4NjtV6h9+N5jGMEvgnVCwsGh9DO5S8gX63bLm946V9c7dcfelOQ
         0ZX27WqdANfuyLHp04z9dQy+bxGjoh3Fo9gXkfpH/eG/5M9fPo/F07wu7gTir07AAHxA
         rhyIFp5eoufHyjQAjTlUNThUWUtRA57Ycd06GLphJx2ntp67tRNBiiYsm7Uj5crCieUy
         8XpeQNvb+wA/3eG76IHWMw68lt6DBbRiE7ePeBu61B8lE2TM4nb1dfmwBvOd2ntXIHPy
         d6dg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954120; x=1716558920;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=mXA5k6NQLwnOQ/WhnpaidF2ChH86fmHaLKiU9LUuBe8=;
        b=l6f8XEMRO231x/Nfy2L66O1lRLFYO8W3C5i5hY/7j1znlIiXJnLBWU5kgohlrBB8Js
         DOER3eJJ5iVNQ2uMpIwKGdPW0x5OjVXFgyQTDVp6x0zcNidvW95O+HlJNBdm6haOD+Kz
         3BcM1pony+aB52+uxz4sKsNrhbKLWR9faZWh7GIKKdVOmvDWdZNbNsVucZvnd9DIkkK3
         QLZ3ZNxCNzWwkT7g09ArdfKqhrT9QpYXkxtUlkESRkuU7IsBlOGv2G21AyLG9y9aaaMd
         tKPXQeTMsHjJWBUlMtFAJI8qvR+4GiE/y5k74BoIQOvFpOb90eEpmkGPjZ73uwK43Xwj
         1b4w==
X-Gm-Message-State: AOJu0Yx5Xu9wpOKCbWchFQf5MIfpFDHvMH5g/LnLlxq/EeK6Bd4On/ZR
	EYlFSNtNCCdzx+Zn48+Hm+PqG4aRV02XRXcp+6QKNpGrit8MFdckagHUPnp7
X-Google-Smtp-Source: AGHT+IFlHn4riCAoAVLptslFcHSUF8D7d/Mhu9kIX5sx5rf7ZnlKWhXQQk9IqqGNWbsXrS264cWyuw==
X-Received: by 2002:a17:906:57cf:b0:a59:c963:82b with SMTP id a640c23a62f3a-a5a2d5cffe2mr1330690966b.33.1715954119886;
        Fri, 17 May 2024 06:55:19 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v10 14/14] xen/README: add compiler and binutils versions for RISC-V64
Date: Fri, 17 May 2024 15:55:03 +0200
Message-ID: <e5f2bd44fa1c516ccce22d89053192f85301095e.1715952103.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1715952103.git.oleksii.kurochko@gmail.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V10:
 - Nothing changed. Only rebase.
---
 Changes in V6:
  - update the message in README.
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README b/README
index c8a108449e..30da5ff9c0 100644
--- a/README
+++ b/README
@@ -48,6 +48,10 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+          Older GCC and GNU Binutils would work, but this is not a guarantee.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 13:57:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:57:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724197.1129504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y5W-0001Sp-8Z; Fri, 17 May 2024 13:57:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724197.1129504; Fri, 17 May 2024 13:57:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y5W-0001Si-3S; Fri, 17 May 2024 13:57:34 +0000
Received: by outflank-mailman (input) for mailman id 724197;
 Fri, 17 May 2024 13:57:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7y5U-0001SY-R3; Fri, 17 May 2024 13:57:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7y5U-0007HP-Od; Fri, 17 May 2024 13:57:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s7y5U-0004Cq-FH; Fri, 17 May 2024 13:57:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s7y5U-0000yh-El; Fri, 17 May 2024 13:57:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=bpBitsrVjYsqBLjJxQVx2P5vjyhQJm4lyBQJDLEj21c=; b=HR3T01GA4lS33REibtGBLRIDqW
	rw4cq5tpQePoP00aUM5iXtdC2W1YL3Wpeb7pjORju8cIsrFomUQhsZlhOjqIYfoYQPTcChaMyxBJg
	ZsWFVYRw0S5dfj1RlHaUJD1XZrZmE5r6YyNZjPy5mWhysSDTy1Y508OlWAcQdEBkwcc4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186025-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186025: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=34f52aec286af47066484e8a96ecba0ef8e79451
X-Osstest-Versions-That:
    libvirt=8b133e82fc62188b04dcdb2ddfc5589a48222a2c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 13:57:32 +0000

flight 186025 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186025/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186011
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              34f52aec286af47066484e8a96ecba0ef8e79451
baseline version:
 libvirt              8b133e82fc62188b04dcdb2ddfc5589a48222a2c

Last test of basis   186011  2024-05-16 04:20:35 Z    1 days
Testing same since   186025  2024-05-17 04:20:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Rayhan Faizel <rayhan.faizel@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   8b133e82fc..34f52aec28  34f52aec286af47066484e8a96ecba0ef8e79451 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 17 13:59:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 13:59:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724202.1129514 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y7V-00024Z-Jq; Fri, 17 May 2024 13:59:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724202.1129514; Fri, 17 May 2024 13:59:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7y7V-00024S-Fy; Fri, 17 May 2024 13:59:37 +0000
Received: by outflank-mailman (input) for mailman id 724202;
 Fri, 17 May 2024 13:59:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJM7=MU=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s7y4W-0001cB-1H
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 13:56:32 +0000
Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com
 [2607:f8b0:4864:20::f2f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 433cb533-1455-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 15:56:31 +0200 (CEST)
Received: by mail-qv1-xf2f.google.com with SMTP id
 6a1803df08f44-6a0b68733f5so4932196d6.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 06:56:31 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f205d88sm85754546d6.144.2024.05.17.06.56.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 06:56:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 433cb533-1455-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1715954189; x=1716558989; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=M+QOPgKgaP14grMwcOnBETVr6+/OzicFLEDWvdXBbb8=;
        b=DfNHB3P4ZvVodkxRHlTglvtcNH2rH+3QSQw2FBxj2ekAhQoFTMutxFQG0sZ3UkpuXT
         yWZDUhJBxzFh8hPnPPCh/E6X5+zWQASLF9Eq56MVP21ulcS2R/iY1Db/jCTz+HMFVgk3
         PcqvTuT2ETVo8JwNMVR9G7P2NCptFuS+KvcOo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715954189; x=1716558989;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=M+QOPgKgaP14grMwcOnBETVr6+/OzicFLEDWvdXBbb8=;
        b=iTaKA3kprUw6rTk7+kueIwAh7T7VuCxuZeD9c5FrfRQ4tMwo1uFbHg/inQyJ6BwTvN
         wVZ7g7y4Dj5NXsXlb2niAg08eVZBoSsnIIqtvyuGxINfWB9M/gpHMbW2SSOzOln5imHh
         QR/w2A+lfxAxxP1CH6MQm1HOMRy3N/bKpVf4u9JTQzVqQ7MjTrm9n1Ec+ikUQ43qcQYo
         U+KZS1+gLKioqwwrAAhQHJXiDEKrORkga/40uzHHauFmDnnhxYDuzuIDCjgrVmqFYZug
         3uVXD+hoG3UbrsXwZa2TjGEuIsqAEJ9S8u5HnJAtzVVBcpHkYmsZYwp0yCxgCergJjjn
         k53Q==
X-Gm-Message-State: AOJu0YweNm3hFbyAYsM5rJJ6uu2GIGt4ic2yxnhkql6wUqKRCUBfeZU/
	WlfbWuIw+RaKKsF9RtsPHDh8xMnfsYPqV7fYPFYnhNekhMRZSYKTUCdo44UB5C2hEacbAFKn/8P
	f
X-Google-Smtp-Source: AGHT+IHgrQTPsapkmh3V9ID2fxskDwsofaTuANyIWY+R6O9TdkN1TKqI3uGLThAa5+sDl2A9eaE7DA==
X-Received: by 2002:a05:6214:2f08:b0:6a0:bb97:9470 with SMTP id 6a1803df08f44-6a1682588d5mr283883386d6.41.1715954189511;
        Fri, 17 May 2024 06:56:29 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19] x86/msi: prevent watchdog triggering when dumping MSI state
Date: Fri, 17 May 2024 15:56:05 +0200
Message-ID: <20240517135605.94781-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the same check that's used in dump_irqs().

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/msi.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 19830528b65a..0c97fbb3fc03 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -17,6 +17,7 @@
 #include <xen/param.h>
 #include <xen/pci.h>
 #include <xen/pci_regs.h>
+#include <xen/softirq.h>
 #include <xen/iocap.h>
 #include <xen/keyhandler.h>
 #include <xen/pfn.h>
@@ -1451,6 +1452,9 @@ static void cf_check dump_msi(unsigned char key)
         unsigned long flags;
         const char *type = "???";
 
+        if ( !(irq & 0x1f) )
+            process_pending_softirqs();
+
         if ( !irq_desc_initialized(desc) )
             continue;
 
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Fri May 17 14:05:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 14:05:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724258.1129523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCw-00059k-9B; Fri, 17 May 2024 14:05:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724258.1129523; Fri, 17 May 2024 14:05:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCw-00059d-6Z; Fri, 17 May 2024 14:05:14 +0000
Received: by outflank-mailman (input) for mailman id 724258;
 Fri, 17 May 2024 14:05:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4QDV=MU=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7yCu-000588-7G
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 14:05:12 +0000
Received: from doppler.solinno.uk
 (8.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.9.a.d.0.6.f.d.1.0.b.8.0.1.0.0.2.ip6.arpa
 [2001:8b0:1df6:da9::18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 78a2b814-1456-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 16:05:10 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 2E9C580078;
 Fri, 17 May 2024 15:05:09 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 0B1B42016C; Fri, 17 May 2024 15:05:09 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78a2b814-1456-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715954709;
	bh=+qr5UyGF5YNpfNX83c8/e3HXP91//GFe1wX1VW+GKEs=;
	h=From:To:Cc:Subject:Date:From;
	b=zs8Onk7phf+jj5p9jQIag1gJa7zOHNAyTN2FURnCxILN+OE6/TmSsRua6x6bAaGfF
	 R6qZ1JSdTaFC6kmeKG8UYaOw9+MEjZB/FbC27gManqm0oRvJ4GyBDHnLg9HBSBan+R
	 wSQRya7kmp58WG/jRv98nVQ9Iwa49wkJmgmBtd3Y=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v4 0/2] Finalise bridge VLAN support
Date: Fri, 17 May 2024 15:05:04 +0100
Message-Id: <20240517140506.8460-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hello,

The first two patches have been merged so this version has the remaining
two. Only the first of these two patches has changed since v3.

Summary of changes from v3 to v4:
- Drop merged patches.
- _vif_vlan_setup: Add comment that we delete vid 1 due to it being
  automatically added.
- _vif_vlan_setup: Restructure to avoid using eval and simplify.
- _vif_vlan_membership: Add comment that we remove trailing spaces due
  to readarray behaviour and change the code to do it only on the last
  element of the array, prior to entering the loop.
- _vif_vlan_membership: Simplify the loop iterating through terms.

Regards,

Leigh.

---

Leigh Brown (2):
  tools/hotplug/Linux: Add bridge VLAN support
  tools/examples: Example Linux bridge VLAN config

 docs/misc/linux-bridge-vlan/README         |  68 +++++++++++++
 docs/misc/linux-bridge-vlan/br0.netdev     |   7 ++
 docs/misc/linux-bridge-vlan/br0.network    |   8 ++
 docs/misc/linux-bridge-vlan/enp0s0.network |  16 +++
 tools/hotplug/Linux/xen-network-common.sh  | 109 +++++++++++++++++++++
 5 files changed, 208 insertions(+)
 create mode 100644 docs/misc/linux-bridge-vlan/README
 create mode 100644 docs/misc/linux-bridge-vlan/br0.netdev
 create mode 100644 docs/misc/linux-bridge-vlan/br0.network
 create mode 100644 docs/misc/linux-bridge-vlan/enp0s0.network

-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 17 14:05:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 14:05:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724260.1129537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCz-0005RE-PM; Fri, 17 May 2024 14:05:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724260.1129537; Fri, 17 May 2024 14:05:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCz-0005QU-JS; Fri, 17 May 2024 14:05:17 +0000
Received: by outflank-mailman (input) for mailman id 724260;
 Fri, 17 May 2024 14:05:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4QDV=MU=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7yCy-0005Nn-5R
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 14:05:16 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7b1f8177-1456-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 16:05:13 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 973458009E;
 Fri, 17 May 2024 15:05:13 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 73F292016C; Fri, 17 May 2024 15:05:13 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b1f8177-1456-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715954713;
	bh=/lU9yTgpboEAtXceuLIDss8GmkMWmz9roeXA4WqyFzY=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=LeKEDUoIMtzl6EmCLfeTsvLFR1NFFx/nn7+0tUmg9kju+CjdEemopvyXefryc4FMl
	 su7fEiYrJwChL/JFX0lWVeUIKsHNZ5wANkxs231xIFyMDEm6kc4MIp+AsFEWMaZXly
	 +U3HhyoLjmizKmeW90iB0eVdtkpKTqMmwUMgnAN0=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v4 2/2] tools/examples: Example Linux bridge VLAN config
Date: Fri, 17 May 2024 15:05:06 +0100
Message-Id: <20240517140506.8460-3-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240517140506.8460-1-leigh@solinno.co.uk>
References: <20240517140506.8460-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a new directory linux-bridge-vlan with example files showing
how to configure systemd-networkd to support a bridge VLAN
configuration.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 docs/misc/linux-bridge-vlan/README         | 68 ++++++++++++++++++++++
 docs/misc/linux-bridge-vlan/br0.netdev     |  7 +++
 docs/misc/linux-bridge-vlan/br0.network    |  8 +++
 docs/misc/linux-bridge-vlan/enp0s0.network | 16 +++++
 4 files changed, 99 insertions(+)
 create mode 100644 docs/misc/linux-bridge-vlan/README
 create mode 100644 docs/misc/linux-bridge-vlan/br0.netdev
 create mode 100644 docs/misc/linux-bridge-vlan/br0.network
 create mode 100644 docs/misc/linux-bridge-vlan/enp0s0.network

diff --git a/docs/misc/linux-bridge-vlan/README b/docs/misc/linux-bridge-vlan/README
new file mode 100644
index 0000000000..9a048bca39
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/README
@@ -0,0 +1,68 @@
+Linux Xen Dom0 single bridge multiple VLAN configuration with systemd
+=====================================================================
+
+Introduction
+------------
+
+This directory contains example files to be placed in /etc/systemd/network
+to enable a single bridge with multiple VLAN support.
+
+The example is to support the scenario where the Xen host network interface
+is connected to an Ethernet switch configured as a trunk port. Each domain
+VIF can then be configured with one or more VLAN IDs, one of which will be
+the PVID.
+
+The example files create a bridge device called br0, with a physical interface 
+called enp0s0. You will need to update this with your system's device name.
+
+Key points of the configuration are:
+
+1. In br0.netdev, VLANFiltering=on is set. This is required to ensure the
+   VLAN tags are handled correctly.  If it is not set then the packets
+   from the VIF interfaces will not have the correct VLAN tags set.
+
+2. In br0.network, a system IPv4 address is configured that can be updated
+   according to your local network settings.
+
+3. In enp0s0.network, Bridge=br0 sets the bridge device to connect to. There
+   is also a [BridgeVLAN] section for each VLAN allowed on the external
+   interface. Note, if you want to create an internal VLAN private to the
+   host, do not include its VLAN ID in this file.
+
+
+Domain configuration
+--------------------
+
+Add the vlan= keyword to the vif definition in the domain. The simplest
+and most common example is a domain that wishes to connect to a single VLAN:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10' ]
+
+If you wish to configure a domain to route between two VLANs, you have two
+options. Option 1 is to create multiple interfaces on different VLANs:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10',
+        'max=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=20' ]
+
+Alternatively, you can create single interface:
+
+vif = [ 'mac=xx:xx:xx:xx:xx:xx, bridge=br0, vlan=10p/20' ]
+
+In the domain, you would, for example, use enX0 for VLAN 10 and enX0.20 for 
+VLAN 20.
+
+
+Hints and tips
+--------------
+
+You can run the following commands on dom0 or a driver domain:
+
+1. To check if vlan_filtering is enabled:
+   # cat /sys/devices/virtual/net/<name>/bridge/vlan_filtering
+
+2. To check the bridge port VLAN assignments:
+   # bridge vlan
+
+3. To check the vlan setting in the xenstore (dom0 only):
+   # xenstore-ls -f | grep 'vlan ='
+
diff --git a/docs/misc/linux-bridge-vlan/br0.netdev b/docs/misc/linux-bridge-vlan/br0.netdev
new file mode 100644
index 0000000000..ae1fe487c3
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/br0.netdev
@@ -0,0 +1,7 @@
+[NetDev]
+Name=br0
+Kind=bridge
+MACAddress=xx:xx:xx:xx:xx:xx
+
+[Bridge]
+VLANFiltering=on
diff --git a/docs/misc/linux-bridge-vlan/br0.network b/docs/misc/linux-bridge-vlan/br0.network
new file mode 100644
index 0000000000..b56203b66a
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/br0.network
@@ -0,0 +1,8 @@
+[Match]
+Name=br0
+
+[Network]
+DNS=8.8.8.8
+#Domains=example.com
+Address=10.1.1.10/24
+Gateway=10.1.1.1
diff --git a/docs/misc/linux-bridge-vlan/enp0s0.network b/docs/misc/linux-bridge-vlan/enp0s0.network
new file mode 100644
index 0000000000..6ee3154dfc
--- /dev/null
+++ b/docs/misc/linux-bridge-vlan/enp0s0.network
@@ -0,0 +1,16 @@
+[Match]
+Name=enp0s0
+
+[Network]
+Bridge=br0
+
+# If Jumbo frames are required
+#[Link]
+#MTUBytes=9000
+
+[BridgeVLAN]
+VLAN=10
+
+[BridgeVLAN]
+VLAN=20
+
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 17 14:05:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 14:05:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724259.1129533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCz-0005OY-GF; Fri, 17 May 2024 14:05:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724259.1129533; Fri, 17 May 2024 14:05:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s7yCz-0005OQ-Cl; Fri, 17 May 2024 14:05:17 +0000
Received: by outflank-mailman (input) for mailman id 724259;
 Fri, 17 May 2024 14:05:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4QDV=MU=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s7yCx-0005Nn-FJ
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 14:05:15 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7a6930c2-1456-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 16:05:12 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 6AB3A80078;
 Fri, 17 May 2024 15:05:12 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id 486B02016C; Fri, 17 May 2024 15:05:12 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a6930c2-1456-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1715954712;
	bh=zsHXVKI9xcbrElKiMA1O6RG7/JYFBSB09X+Lmvdpy1w=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=XWu3UlR9zsIzuiqzQzVaaCSR31zaHU66zWlTZ1DhWb3jDwdDHgAOEQfRtDAnr9aZM
	 5mf4vodeDhiFaIe7uX2sXfnj/BVYEZZm8//l7Nu/F4/xl1EDFqtWslHdxmFdfW0URg
	 Px2nsaF0X1rXeVpQrqACojeZCwAeTdZri6WEKRdg=
From: Leigh Brown <leigh@solinno.co.uk>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH v4 1/2] tools/hotplug/Linux: Add bridge VLAN support
Date: Fri, 17 May 2024 15:05:05 +0100
Message-Id: <20240517140506.8460-2-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240517140506.8460-1-leigh@solinno.co.uk>
References: <20240517140506.8460-1-leigh@solinno.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update add_to_bridge shell function to read the vlan parameter from
xenstore and set the bridge VLAN configuration for the VID.

Add additional helper functions to parse the vlan specification,
which consists of one or more of the following:

a) single VLAN (e.g. 10).
b) contiguous range of VLANs (e.g. 10-15).
c) discontiguous range with base, increment and count
   (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).

A single VLAN can be suffixed with "p" to indicate the PVID, or
"u" to indicate untagged. A range of VLANs can be suffixed with
"u" to indicate untagged.  A complex example would be:

   vlan=1p/10-15/20-25u

This capability requires the iproute2 bridge command to be
installed.  An error will be generated if the vlan parameter is
set and the bridge command is not available.

Signed-off-by: Leigh Brown <leigh@solinno.co.uk>
---
 tools/hotplug/Linux/xen-network-common.sh | 109 ++++++++++++++++++++++
 1 file changed, 109 insertions(+)

diff --git a/tools/hotplug/Linux/xen-network-common.sh b/tools/hotplug/Linux/xen-network-common.sh
index 42fa704e8d..31d359b83c 100644
--- a/tools/hotplug/Linux/xen-network-common.sh
+++ b/tools/hotplug/Linux/xen-network-common.sh
@@ -121,10 +121,111 @@ create_bridge () {
     fi
 }
 
+_vif_vlan_add() {
+    # References vlans and pvid variables from the calling function
+    local -i vid=$1
+    local flag=${2:-}
+
+    if (( vid < 1 || vid > 4094 )) ;then
+        fatal "vlan id $vid not between 1 and 4094"
+    fi
+    if [[ -n "${vlans[$vid]}" ]] ;then
+        fatal "vlan id $vid specified more than once"
+    fi
+    case $flag in
+     p) if (( pvid != 0 )) ;then
+            fatal "more than one pvid specified ($vid and $pvid)"
+        fi
+        pvid=$vid
+        vlans[$vid]=p ;;
+     u) vlans[$vid]=u ;;
+     *) vlans[$vid]=t ;;
+    esac
+}
+
+_vif_vlan_parse_term() {
+    local vid incr last term=${1:-}
+
+    if [[ $term =~ ^([0-9]+)([pu])?$ ]] ;then
+        _vif_vlan_add ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}
+    elif [[ $term =~ ^([0-9]+)-([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        last=${BASH_REMATCH[2]}
+        if (( last >= vid )) ;then
+            for (( ; vid<=last; vid++ )) ;do
+                _vif_vlan_add $vid ${BASH_REMATCH[3]}
+            done
+        else
+            fatal "invalid vlan id range: $term"
+        fi
+    elif [[ $term =~ ^([0-9]+)\+([0-9]+)x([0-9]+)(u)?$ ]] ;then
+        vid=${BASH_REMATCH[1]}
+        incr=${BASH_REMATCH[2]}
+        for (( j=${BASH_REMATCH[3]}; j>0; --j, vid+=incr ))
+        do
+            _vif_vlan_add $vid ${BASH_REMATCH[4]}
+        done
+    else
+        fatal "invalid vlan specification: $term"
+    fi
+}
+
+_vif_vlan_validate_pvid() {
+    # References vlans and pvid variables from the calling function
+    if (( pvid == 0 )) ;then
+        if (( ${#vlans[@]} == 1 )) ;then
+            vlans[${!vlans[*]}]=p
+        else
+            fatal "pvid required when using multiple vlan ids"
+        fi
+    fi
+}
+
+_vif_vlan_setup() {
+    # References vlans and dev variable from the calling function
+    local -i vid
+    local -a args
+
+    # Remove the default vlan id automatically added to the vif
+    bridge vlan del dev $dev vid 1
+
+    # Add the required vlans
+    for vid in ${!vlans[@]} ;do
+        case ${vlans[$vid]} in
+             p) args=(pvid untagged) ;;
+             u) args=(untagged) ;;
+             t) args=() ;;
+        esac
+        bridge vlan add dev $dev vid $vid ${args[@]}
+    done
+}
+
+_vif_vlan_membership() {
+    # The vlans, pvid and dev variables are used by sub-functions
+    local -A vlans=()
+    local -a terms=()
+    local -i i pvid=0
+    local dev=$1 term
+
+    # Split the vlan specification string into its terms, removing the newline
+    # that readarray adds to the last element
+    readarray -d / -t terms <<<$2
+    terms[-1]=${terms[-1]%%[[:space:]]}
+
+    for term in ${terms[@]} ;do
+        _vif_vlan_parse_term $term
+    done
+
+    _vif_vlan_validate_pvid
+    _vif_vlan_setup
+    return 0
+}
+
 # Usage: add_to_bridge bridge dev
 add_to_bridge () {
     local bridge=$1
     local dev=$2
+    local vlan=$(xenstore_read_default "$XENBUS_PATH/vlan" "")
 
     # Don't add $dev to $bridge if it's already on the bridge.
     if [ ! -e "/sys/class/net/${bridge}/brif/${dev}" ]; then
@@ -134,6 +235,14 @@ add_to_bridge () {
         else
             ip link set ${dev} master ${bridge}
         fi
+        if [ -n "${vlan}" ] ;then
+            log debug "configuring vlans for ${dev} on ${bridge}"
+            if which bridge >&/dev/null; then
+                _vif_vlan_membership "${dev}" "${vlan}"
+            else
+                fatal "vlan configuration failed: bridge command not found"
+            fi
+        fi
     else
         log debug "$dev already on bridge $bridge"
     fi
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Fri May 17 16:08:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 16:08:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724317.1129563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808U-00029l-Ee; Fri, 17 May 2024 16:08:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724317.1129563; Fri, 17 May 2024 16:08:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808U-00029a-A7; Fri, 17 May 2024 16:08:46 +0000
Received: by outflank-mailman (input) for mailman id 724317;
 Fri, 17 May 2024 16:08:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jVtU=MU=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s808T-0001vE-D5
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 16:08:45 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bb6cc059-1467-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 18:08:43 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a5a1054cf61so555671566b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 09:08:43 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a60eabd3csm739104366b.108.2024.05.17.09.08.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 09:08:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb6cc059-1467-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715962122; x=1716566922; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NRfRLYM2uaIOUr5UnCxOVm7BN+wzkpxkUR6EABqZEIM=;
        b=Pov1zbFFcTy/bPVEPAoC1lbBrD60fkrDjgoEK4sjxvAP4haZU5WpwR1MUzl92lrwBy
         cbAtZ9h4hDKqQjmiygCb3SJcaOfNcmRFp9eJAT1RrcJ0P20KZ0aOFaP+YbolGTUIUq+d
         EwBqH8tuzH2BIS+hmX+cFPKunnxeuKUP8ojyw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715962122; x=1716566922;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=NRfRLYM2uaIOUr5UnCxOVm7BN+wzkpxkUR6EABqZEIM=;
        b=RzjPc8iVrgohLzfur3LITxwK6rO/rcaX7gElSpbK1Ib/Xd/kJbC8sxfELu3xXAK0Vx
         H1eO5xdLvSCIHZWjgljuv2CIFeMzjbqCdHz5RGH5p8Ekry9f9E2XAT21QzoDwEw/pFFE
         teAj8tsiFVYJhEWnYjZEkEgyMnhHY/PhwRaa1IP0e7de9qKKf+Wm5LwvBNrR4UxrSbA2
         qXrLZgvxBy1krOeUd3LN/tb8Ov+4esfOkRWz/NT9CSAUa7tl8KZWEqhQYTOfJCBichII
         7aIPOZOUY+27gWiNliz5AZb3CKSd9rLACUIlyYVzhM/4/SzCIjLWV21XRgYqWV6IsPSZ
         ZZpQ==
X-Gm-Message-State: AOJu0Yz/wAiighj4RgohPOQmFshxiN7EedssCgUwAAzwK8fN0Akr1VfU
	84V3yyxWtuHns+hfnQu7Nw5GnlBWMwsmOEQVTnHPZc1wUrqKGQMFm13UwtapKrKahfo8d41N+Ji
	+
X-Google-Smtp-Source: AGHT+IGYuJX1kmCNe86D53+81GF3VMyvUiycUA9rCHz8TKzTSBHr6hVAAowzwLbNpyRoc4hNwxWmLw==
X-Received: by 2002:a17:906:f883:b0:a5a:8d0f:3ad1 with SMTP id a640c23a62f3a-a5a8d0f4cc1mr965870366b.17.1715962122098;
        Fri, 17 May 2024 09:08:42 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 1/2] tools/xg: Streamline cpu policy serialise/deserialise calls
Date: Fri, 17 May 2024 17:08:34 +0100
Message-Id: <c6c89012d189a2c3077b3e224942b33028e29336.1715954111.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715954111.git.alejandro.vallejo@cloud.com>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The idea is to use xc_cpu_policy_t as a single object containing both the
serialised and deserialised forms of the policy. Note that we need lengths
for the arrays, as the serialised policies may be shorter than the array
capacities.

* Add the serialised lengths to the struct so we can distinguish
  between length and capacity of the serialisation buffers.
* Remove explicit buffer+lengths in serialise/deserialise calls
  and use the internal buffer inside xc_cpu_policy_t instead.
* Refactor everything to use the new serialisation functions.
* Remove redundant serialization calls and avoid allocating dynamic
  memory aside from the policy objects in xen-cpuid. Also minor cleanup
  in the policy print call sites.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * Removed v1/patch1.
  * Added the accessors suggested in feedback.
---
 tools/include/xenguest.h            |  8 ++-
 tools/libs/guest/xg_cpuid_x86.c     | 98 ++++++++++++++++++++---------
 tools/libs/guest/xg_private.h       |  2 +
 tools/libs/guest/xg_sr_common_x86.c | 54 ++++++----------
 tools/misc/xen-cpuid.c              | 43 ++++---------
 5 files changed, 104 insertions(+), 101 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index e01f494b772a..563811cd8dde 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -799,14 +799,16 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy);
 
 /* Manipulate a policy via architectural representations. */
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
 int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                                const xen_cpuid_leaf_t *leaves,
                                uint32_t nr);
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
                               const xen_msr_entry_t *msrs, uint32_t nr);
+int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
+int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs, uint32_t *nr);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..4f4b86b59470 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
-                              unsigned int nr_leaves, unsigned int nr_entries)
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
     int rc;
 
     rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    nr_leaves, &err_leaf, &err_subleaf);
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
     if ( rc )
     {
         if ( err_leaf != -1 )
@@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
     }
 
     rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  nr_entries, &err_msr);
+                                  policy->nr_msrs, &err_msr);
     if ( rc )
     {
         if ( err_msr != -1 )
@@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
-    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
     int rc;
 
-    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
-                                 policy->msrs, &nr_msrs);
+    rc = xc_cpu_policy_serialise(xch, policy);
     if ( rc )
         return rc;
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
-                                  nr_msrs, policy->msrs,
+    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
+                                  policy->nr_msrs, policy->msrs,
                                   &err_leaf, &err_subleaf, &err_msr);
     if ( rc )
     {
@@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
     return rc;
 }
 
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
 {
+    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
+    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
     int rc;
 
-    if ( leaves )
+    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
+    if ( rc )
     {
-        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
-        if ( rc )
-        {
-            ERROR("Failed to serialize CPUID policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize CPUID policy");
+        errno = -rc;
+        return -1;
     }
 
-    if ( msrs )
+    p->nr_leaves = nr_leaves;
+
+    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &nr_msrs);
+    if ( rc )
     {
-        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
-        if ( rc )
-        {
-            ERROR("Failed to serialize MSR policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize MSR policy");
+        errno = -rc;
+        return -1;
     }
 
+    p->nr_msrs = nr_msrs;
+
     errno = 0;
     return 0;
 }
@@ -1012,6 +1012,42 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
     return rc;
 }
 
+int xc_cpu_policy_get_leaves(xc_interface *xch,
+                             const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves,
+                             uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch CPUID leaves from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *leaves = policy->leaves;
+    *nr = policy->nr_leaves;
+
+    return 0;
+}
+
+int xc_cpu_policy_get_msrs(xc_interface *xch,
+                           const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs,
+                           uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch MSRs from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *msrs = policy->msrs;
+    *nr = policy->nr_msrs;
+
+    return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest)
 {
diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
index d73947094f2e..a65dae818f3d 100644
--- a/tools/libs/guest/xg_private.h
+++ b/tools/libs/guest/xg_private.h
@@ -177,6 +177,8 @@ struct xc_cpu_policy {
     struct cpu_policy policy;
     xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
     xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
+    uint32_t nr_leaves;
+    uint32_t nr_msrs;
 };
 #endif /* x86 */
 
diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
index 563b4f016877..832047756e58 100644
--- a/tools/libs/guest/xg_sr_common_x86.c
+++ b/tools/libs/guest/xg_sr_common_x86.c
@@ -1,4 +1,5 @@
 #include "xg_sr_common_x86.h"
+#include "xg_sr_stream_format.h"
 
 int write_x86_tsc_info(struct xc_sr_context *ctx)
 {
@@ -45,54 +46,37 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
-    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
-    uint32_t nr_leaves = 0, nr_msrs = 0;
-    xc_cpu_policy_t *policy = NULL;
+    struct xc_sr_record record;
+    xc_cpu_policy_t *policy = xc_cpu_policy_init();
     int rc;
 
-    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
-    {
-        PERROR("Unable to get CPU Policy size");
-        return -1;
-    }
-
-    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
-    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
-    policy = xc_cpu_policy_init();
-    if ( !cpuid.data || !msrs.data || !policy )
-    {
-        ERROR("Cannot allocate memory for CPU Policy");
-        rc = -1;
-        goto out;
-    }
-
-    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
+    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
     {
         PERROR("Unable to get d%d CPU Policy", ctx->domid);
         rc = -1;
         goto out;
     }
-    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
-                                 msrs.data, &nr_msrs) )
-    {
-        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
-        rc = -1;
-        goto out;
-    }
 
-    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
-    if ( cpuid.length )
+    record = (struct xc_sr_record) {
+        .type = REC_TYPE_X86_CPUID_POLICY,
+        .data = policy->leaves,
+        .length = policy->nr_leaves * sizeof(*policy->leaves),
+    };
+    if ( record.length )
     {
-        rc = write_record(ctx, &cpuid);
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
 
-    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
-    if ( msrs.length )
+    record = (struct xc_sr_record) {
+        .type = REC_TYPE_X86_MSR_POLICY,
+        .data = policy->msrs,
+        .length = policy->nr_msrs * sizeof(*policy->msrs),
+    };
+    if ( record.length )
     {
-        rc = write_record(ctx, &msrs);
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
@@ -100,8 +84,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     rc = 0;
 
  out:
-    free(cpuid.data);
-    free(msrs.data);
     xc_cpu_policy_destroy(policy);
 
     return rc;
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 8893547bebce..1c9ba6d32060 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -409,17 +409,21 @@ static void dump_info(xc_interface *xch, bool detail)
     free(fs);
 }
 
-static void print_policy(const char *name,
-                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
-                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
+static void print_policy(xc_interface *xch, const char *name, const xc_cpu_policy_t *policy)
 {
-    unsigned int l;
+    const xen_cpuid_leaf_t *leaves;
+    const xen_msr_entry_t *msrs;
+    uint32_t nr_leaves, nr_msrs;
+
+    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) ||
+         xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
+        err(1, "print_policy()");
 
     printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
     printf(" CPUID:\n");
     printf("  %-8s %-8s -> %-8s %-8s %-8s %-8s\n",
            "leaf", "subleaf", "eax", "ebx", "ecx", "edx");
-    for ( l = 0; l < nr_leaves; ++l )
+    for ( uint32_t l = 0; l < nr_leaves; ++l )
     {
         /* Skip empty leaves. */
         if ( !leaves[l].a && !leaves[l].b && !leaves[l].c && !leaves[l].d )
@@ -432,7 +436,7 @@ static void print_policy(const char *name,
 
     printf(" MSRs:\n");
     printf("  %-8s -> %-16s\n", "index", "value");
-    for ( l = 0; l < nr_msrs; ++l )
+    for ( uint32_t l = 0; l < nr_msrs; ++l )
         printf("  %08x -> %016"PRIx64"\n",
                msrs[l].idx, msrs[l].val);
 }
@@ -539,8 +543,6 @@ int main(int argc, char **argv)
             [ XEN_SYSCTL_cpu_policy_pv_default ]   = "PV Default",
             [ XEN_SYSCTL_cpu_policy_hvm_default ]  = "HVM Default",
         };
-        xen_cpuid_leaf_t *leaves;
-        xen_msr_entry_t *msrs;
         uint32_t i, max_leaves, max_msrs;
 
         xc_interface *xch = xc_interface_open(0, 0, 0);
@@ -557,36 +559,21 @@ int main(int argc, char **argv)
             printf("Xen reports there are maximum %u leaves and %u MSRs\n",
                    max_leaves, max_msrs);
 
-        leaves = calloc(max_leaves, sizeof(xen_cpuid_leaf_t));
-        if ( !leaves )
-            err(1, "calloc(max_leaves)");
-        msrs = calloc(max_msrs, sizeof(xen_msr_entry_t));
-        if ( !msrs )
-            err(1, "calloc(max_msrs)");
-
         if ( domid != -1 )
         {
             char name[20];
-            uint32_t nr_leaves = max_leaves;
-            uint32_t nr_msrs = max_msrs;
 
             if ( xc_cpu_policy_get_domain(xch, domid, policy) )
                 err(1, "xc_cpu_policy_get_domain(, %d, )", domid);
-            if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                         msrs, &nr_msrs) )
-                err(1, "xc_cpu_policy_serialise");
 
             snprintf(name, sizeof(name), "Domain %d", domid);
-            print_policy(name, leaves, nr_leaves, msrs, nr_msrs);
+            print_policy(xch, name, policy);
         }
         else
         {
             /* Get system policies */
             for ( i = 0; i < ARRAY_SIZE(sys_policies); ++i )
             {
-                uint32_t nr_leaves = max_leaves;
-                uint32_t nr_msrs = max_msrs;
-
                 if ( xc_cpu_policy_get_system(xch, i, policy) )
                 {
                     if ( errno == EOPNOTSUPP )
@@ -598,18 +585,12 @@ int main(int argc, char **argv)
 
                     err(1, "xc_cpu_policy_get_system(, %s, )", sys_policies[i]);
                 }
-                if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                             msrs, &nr_msrs) )
-                    err(1, "xc_cpu_policy_serialise");
 
-                print_policy(sys_policies[i], leaves, nr_leaves,
-                             msrs, nr_msrs);
+                print_policy(xch, sys_policies[i], policy);
             }
         }
 
         xc_cpu_policy_destroy(policy);
-        free(leaves);
-        free(msrs);
         xc_interface_close(xch);
     }
     else if ( mode == MODE_INFO || mode == MODE_DETAIL )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 16:08:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 16:08:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724318.1129572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808V-0002OH-Ny; Fri, 17 May 2024 16:08:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724318.1129572; Fri, 17 May 2024 16:08:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808V-0002OA-L4; Fri, 17 May 2024 16:08:47 +0000
Received: by outflank-mailman (input) for mailman id 724318;
 Fri, 17 May 2024 16:08:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jVtU=MU=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s808U-0001vE-63
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 16:08:46 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bbc50613-1467-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 18:08:43 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-5723edf0ae5so4808011a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 09:08:43 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a60eabd3csm739104366b.108.2024.05.17.09.08.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 09:08:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbc50613-1467-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715962123; x=1716566923; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=47z4kOfo9ItA11lex/Eyo+IH6kKUzEGtdLK3F1DkmDk=;
        b=HkimycW2o+/Z7pT/KZErLCpSCN/w2C5R3mZXIe7+8G3j1TMn7ot26BMeaRRER81mK7
         4xjskPpVsSDfg479n6scw175ZyMKLdHKLyw6NjDo+ig1FP4Fm6/fJUCxQiaPl77k9IFP
         NNl64wT/eEOYbUzzoZrSiVAZUT7tM5Y8mw5hY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715962123; x=1716566923;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=47z4kOfo9ItA11lex/Eyo+IH6kKUzEGtdLK3F1DkmDk=;
        b=NjiYK8fNYc8XdqZ/HlsJ2IHYq3QAAiD9AY5+WnYQMaI/gvii5EpdOzw98kMqYr6l45
         w4FpfURTAG4P5NUhpna8mjxA/umPN8OQSIgQhMd70SKFGMgGFUD7ad1F6xKtjKwme44u
         /Xw81V8NK5Ss7QO0AuST0qoS3tPgVfA4gDYl37BGjnOsvjMyef6RQBigScFSigmOhuEx
         UCGfpZUVeBZWRIUjrNUvqVo9I9JVY6gPdXJB0QKcCGqG4gQewylDGuC+oXUpfADa0PWM
         Rz2sK3F1f31HlLSDMzE1eqWUVTkECcqG+x5x8io7fNXRJQ5kooKMhZq6F/h/aRhem4R1
         4isQ==
X-Gm-Message-State: AOJu0YxmTlrw6hft8VUXT3vpgAihWJGtBdiIL5n+Yu9/WhyCOsyTiEBN
	rwFQChnAm4Z0p54CMz5DUUEd6CGdS7F8XaIoHjwITbOL5y1DB4AtjBOJov6JMgz+YDpuJRXW2xX
	I
X-Google-Smtp-Source: AGHT+IFgiQRDPF0oHWJ8+7Wpfmo+yOeD6YdIfm4YPI93QPYfCMxj1b2B8mv9T+MnBIZG1pYqEL4RMg==
X-Received: by 2002:a17:907:3f0d:b0:a55:8f2a:950d with SMTP id a640c23a62f3a-a5a2d53b28amr1991370966b.16.1715962122854;
        Fri, 17 May 2024 09:08:42 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 2/2] tools/xg: Clean up xend-style overrides for CPU policies
Date: Fri, 17 May 2024 17:08:35 +0100
Message-Id: <d397ec0de4138e32feeb910f3401a6568a75035e.1715954111.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1715954111.git.alejandro.vallejo@cloud.com>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Factor out policy getters/setters from both (CPUID and MSR) policy override
functions. Additionally, use host policy rather than featureset when
preparing the cur policy, saving one hypercall and several lines of
boilerplate.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v2:
  * Cosmetic change to comment (// into /**/).
  * Added missing null pointer check to MSR override function.
---
 tools/libs/guest/xg_cpuid_x86.c | 445 ++++++++++----------------------
 1 file changed, 130 insertions(+), 315 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4f4b86b59470..74bca0e65b69 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -36,6 +36,34 @@ enum {
 #define bitmaskof(idx)      (1u << ((idx) & 31))
 #define featureword_of(idx) ((idx) >> 5)
 
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
+{
+    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    int rc;
+
+    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
+    if ( rc )
+    {
+        if ( err_leaf != -1 )
+            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
+                  err_leaf, err_subleaf, -rc, strerror(-rc));
+        return rc;
+    }
+
+    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
+                                  policy->nr_msrs, &err_msr);
+    if ( rc )
+    {
+        if ( err_msr != -1 )
+            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
+                  err_msr, -rc, strerror(-rc));
+        return rc;
+    }
+
+    return 0;
+}
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
 {
     struct xen_sysctl sysctl = {};
@@ -260,102 +288,34 @@ static int compare_leaves(const void *l, const void *r)
     return 0;
 }
 
-static xen_cpuid_leaf_t *find_leaf(
-    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-    const struct xc_xend_cpuid *xend)
+static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
+                                   const struct xc_xend_cpuid *xend)
 {
     const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
 
-    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
+    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),
+                   sizeof(*p->leaves), compare_leaves);
 }
 
-static int xc_cpuid_xend_policy(
-    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
+static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
+                                const struct xc_xend_cpuid *xend,
+                                xc_cpu_policy_t *host,
+                                xc_cpu_policy_t *def,
+                                xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto fail;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -ENOMEM;
-    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        goto fail;
-    }
-
-    /* Get the domain's current policy. */
-    nr_msrs = 0;
-    nr_cur = nr_leaves;
-    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_msrs = 0;
-    nr_def = nr_leaves;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_def, def, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Get the host policy. */
-    nr_msrs = 0;
-    nr_host = nr_leaves;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_host, host, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto fail;
-    }
+    if ( !xend )
+        return 0;
 
-    rc = -EINVAL;
     for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
     {
-        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
-        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
-        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
+        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
+        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
+        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
 
         if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
         {
             ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
-            goto fail;
+            return -EINVAL;
         }
 
         for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
@@ -384,7 +344,7 @@ static int xc_cpuid_xend_policy(
                 {
                     ERROR("Bad character '%c' in policy[%d] string '%s'",
                           xend->policy[i][j], i, xend->policy[i]);
-                    goto fail;
+                    return -EINVAL;
                 }
 
                 clear_bit(31 - j, cur_reg);
@@ -394,25 +354,7 @@ static int xc_cpuid_xend_policy(
         }
     }
 
-    /* Feed the transformed currrent policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Success! */
-
- fail:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 static int compare_msr(const void *l, const void *r)
@@ -426,107 +368,37 @@ static int compare_msr(const void *l, const void *r)
     return lhs->idx < rhs->idx ? -1 : 1;
 }
 
-static xen_msr_entry_t *find_msr(
-    xen_msr_entry_t *msrs, unsigned int nr_msrs,
-    uint32_t index)
+static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
+                                 uint32_t index)
 {
     const xen_msr_entry_t key = { .idx = index };
 
-    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
+    return bsearch(&key, p->msrs, ARRAY_SIZE(p->msrs),
+                   sizeof(*p->msrs), compare_msr);
 }
 
-
 static int xc_msr_policy(xc_interface *xch, domid_t domid,
-                         const struct xc_msr *msr)
+                         const struct xc_msr *msr,
+                         xc_cpu_policy_t *host,
+                         xc_cpu_policy_t *def,
+                         xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto out;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto out;
-    }
-
-    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        rc = -ENOMEM;
-        goto out;
-    }
-
-    /* Get the domain's current policy. */
-    nr_leaves = 0;
-    nr_cur = nr_msrs;
-    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_leaves = 0;
-    nr_def = nr_msrs;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, NULL, &nr_def, def);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the host policy. */
-    nr_leaves = 0;
-    nr_host = nr_msrs;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_leaves, NULL, &nr_host, host);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto out;
-    }
+    if ( !msrs )
+        return 0;
 
     for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
     {
-        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
-        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
-        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
-        unsigned int i;
+        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
+        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
+        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
 
         if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
         {
             ERROR("Missing MSR %#x", msr->index);
-            rc = -ENOENT;
-            goto out;
+            return -ENOENT;
         }
 
-        for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
+        for ( size_t i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
         {
             bool val;
 
@@ -542,8 +414,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
             {
                 ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
                       msr->index, msr->policy[i], msr->policy);
-                rc = -EINVAL;
-                goto out;
+                return -EINVAL;
             }
 
             if ( val )
@@ -553,25 +424,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
         }
     }
 
-    /* Feed the transformed policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Success! */
-
- out:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
@@ -583,14 +436,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     int rc;
     bool hvm;
     xc_domaininfo_t di;
-    struct xc_cpu_policy *p = xc_cpu_policy_init();
-    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-    uint32_t len = ARRAY_SIZE(host_featureset);
 
-    if ( !p )
-        return -ENOMEM;
+    struct xc_cpu_policy *host = xc_cpu_policy_init();
+    struct xc_cpu_policy *def = xc_cpu_policy_init();
+    struct xc_cpu_policy *cur = xc_cpu_policy_init();
+
+    if ( !host || !def || !cur )
+    {
+        rc = -ENOMEM;
+        goto out;
+    }
 
     if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
     {
@@ -600,21 +455,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-    /* Get the host policy. */
-    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-                               &len, host_featureset);
-    /* Tolerate "buffer too small", as we've got the bits we need. */
-    if ( rc && errno != ENOBUFS )
+    /* Get the raw host policy */
+    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+    if ( rc )
     {
-        PERROR("Failed to obtain host featureset");
+        PERROR("Failed to obtain host policy");
         rc = -errno;
         goto out;
     }
 
     /* Get the domain's default policy. */
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, p->leaves, &nr_msrs, NULL);
+    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                                  def);
     if ( rc )
     {
         PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
@@ -622,14 +475,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
-                                    &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-              err_leaf, err_subleaf, -rc, strerror(-rc));
-        goto out;
-    }
+    /* Copy the deserialised default policy to modify it */
+    memcpy(cur, def, sizeof(*cur));
 
     if ( restore )
     {
@@ -647,18 +494,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * - Re-enable features which have become (possibly) off by default.
          */
 
-        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+        cur->policy.basic.rdrand = host->policy.basic.rdrand;
+        cur->policy.feat.hle = host->policy.feat.hle;
+        cur->policy.feat.rtm = host->policy.feat.rtm;
 
         if ( hvm )
         {
-            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+            cur->policy.feat.mpx = host->policy.feat.mpx;
         }
 
-        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
-        p->policy.feat.max_subleaf = 0;
-        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
+        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
+        cur->policy.feat.max_subleaf = 0;
+        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
     }
 
     if ( featureset )
@@ -666,7 +513,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         uint32_t disabled_features[FEATURESET_NR_ENTRIES],
             feat[FEATURESET_NR_ENTRIES] = {};
         static const uint32_t deep_features[] = INIT_DEEP_FEATURES;
-        unsigned int i, b;
+        unsigned int b;
 
         /*
          * The user supplied featureset may be shorter or longer than
@@ -677,14 +524,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
 
         /* Check for truncated set bits. */
         rc = -EOPNOTSUPP;
-        for ( i = user_len; i < nr_features; ++i )
+        for ( size_t i = user_len; i < nr_features; ++i )
             if ( featureset[i] != 0 )
                 goto out;
 
         memcpy(feat, featureset, sizeof(*featureset) * user_len);
 
         /* Disable deep dependencies of disabled features. */
-        for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
+        for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
             disabled_features[i] = ~feat[i] & deep_features[i];
 
         for ( b = 0; b < sizeof(disabled_features) * CHAR_BIT; ++b )
@@ -695,24 +542,24 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
                  !(dfs = x86_cpu_policy_lookup_deep_deps(b)) )
                 continue;
 
-            for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
+            for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
             {
                 feat[i] &= ~dfs[i];
                 disabled_features[i] &= ~dfs[i];
             }
         }
 
-        x86_cpu_featureset_to_policy(feat, &p->policy);
+        x86_cpu_featureset_to_policy(feat, &cur->policy);
     }
     else
     {
-        p->policy.extd.itsc = itsc;
+        cur->policy.extd.itsc = itsc;
 
         if ( hvm )
         {
-            p->policy.basic.pae = pae;
-            p->policy.basic.vmx = nested_virt;
-            p->policy.extd.svm = nested_virt;
+            cur->policy.basic.pae = pae;
+            cur->policy.basic.vmx = nested_virt;
+            cur->policy.extd.svm = nested_virt;
         }
     }
 
@@ -722,8 +569,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * On hardware without CPUID Faulting, PV guests see real topology.
          * As a consequence, they also need to see the host htt/cmp fields.
          */
-        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
-        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+        cur->policy.basic.htt       = host->policy.basic.htt;
+        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
     }
     else
     {
@@ -731,28 +578,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * Topology for HVM guests is entirely controlled by Xen.  For now, we
          * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
          */
-        p->policy.basic.htt = true;
-        p->policy.extd.cmp_legacy = false;
+        cur->policy.basic.htt = true;
+        cur->policy.extd.cmp_legacy = false;
 
         /*
          * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
          * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
          * overflow.
          */
-        if ( !p->policy.basic.lppp )
-            p->policy.basic.lppp = 2;
-        else if ( !(p->policy.basic.lppp & 0x80) )
-            p->policy.basic.lppp *= 2;
+        if ( !cur->policy.basic.lppp )
+            cur->policy.basic.lppp = 2;
+        else if ( !(cur->policy.basic.lppp & 0x80) )
+            cur->policy.basic.lppp *= 2;
 
-        switch ( p->policy.x86_vendor )
+        switch ( cur->policy.x86_vendor )
         {
         case X86_VENDOR_INTEL:
-            for ( i = 0; (p->policy.cache.subleaf[i].type &&
-                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
+            for ( size_t i = 0; (cur->policy.cache.subleaf[i].type &&
+                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )
             {
-                p->policy.cache.subleaf[i].cores_per_package =
-                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
-                p->policy.cache.subleaf[i].threads_per_cache = 0;
+                cur->policy.cache.subleaf[i].cores_per_package =
+                    (cur->policy.cache.subleaf[i].cores_per_package << 1) | 1;
+                cur->policy.cache.subleaf[i].threads_per_cache = 0;
             }
             break;
 
@@ -772,49 +619,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
              * apic_id_size values greater than 7.  Limit the value to
              * 7 for now.
              */
-            if ( p->policy.extd.nc < 0x7f )
+            if ( cur->policy.extd.nc < 0x7f )
             {
-                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
-                    p->policy.extd.apic_id_size++;
+                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )
+                    cur->policy.extd.apic_id_size++;
 
-                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
+                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
             }
             break;
         }
     }
 
-    nr_leaves = ARRAY_SIZE(p->leaves);
-    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
-    if ( rc )
+    if ( xend || msr )
     {
-        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-        goto out;
+        /* The overrides are over the serialised form of the policy */
+        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
+            goto out;
+
+        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
+            goto out;
+        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
+            goto out;
+
+        if ( (rc = deserialize_policy(xch, cur)) )
+            goto out;
     }
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, p->leaves, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
+    rc = xc_cpu_policy_set_domain(xch, domid, cur);
     if ( rc )
     {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
         rc = -errno;
         goto out;
     }
 
-    if ( xend && (rc = xc_cpuid_xend_policy(xch, domid, xend)) )
-        goto out;
-
-    if ( msr )
-    {
-        rc = xc_msr_policy(xch, domid, msr);
-        if ( rc )
-            goto out;
-    }
-
     rc = 0;
 
 out:
-    xc_cpu_policy_destroy(p);
+    xc_cpu_policy_destroy(def);
+    xc_cpu_policy_destroy(host);
+    xc_cpu_policy_destroy(cur);
 
     return rc;
 }
@@ -834,34 +677,6 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
-{
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    int rc;
-
-    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    policy->nr_leaves, &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        if ( err_leaf != -1 )
-            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-                  err_leaf, err_subleaf, -rc, strerror(-rc));
-        return rc;
-    }
-
-    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  policy->nr_msrs, &err_msr);
-    if ( rc )
-    {
-        if ( err_msr != -1 )
-            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
-                  err_msr, -rc, strerror(-rc));
-        return rc;
-    }
-
-    return 0;
-}
-
 int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t *policy)
 {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 16:08:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 16:08:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724316.1129553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808T-0001vT-6y; Fri, 17 May 2024 16:08:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724316.1129553; Fri, 17 May 2024 16:08:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s808T-0001vM-4M; Fri, 17 May 2024 16:08:45 +0000
Received: by outflank-mailman (input) for mailman id 724316;
 Fri, 17 May 2024 16:08:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jVtU=MU=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s808S-0001vE-6b
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 16:08:44 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id babdd507-1467-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 18:08:42 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59a934ad50so467794266b.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 09:08:42 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a60eabd3csm739104366b.108.2024.05.17.09.08.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 May 2024 09:08:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: babdd507-1467-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1715962121; x=1716566921; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+CtUn48a+twwOLd6c/CfhFyRomRswzCE/hkLUGpxuFA=;
        b=DL7E5cHNhqYdZ2NKoqBPoidL/W4SSrnhihnODW874JRv8amdh9JlwfF85OtGE3BF1P
         8GzpRlyslA3ecbxu6A/LF6wSVWayz2klPHr0nMAMO+BhaK9p7yrJtbfUZYWxeR1KUBG4
         n5UNSzwcMwOKdrt4E/WmzL5EIJOdtpg4SI41E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1715962121; x=1716566921;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=+CtUn48a+twwOLd6c/CfhFyRomRswzCE/hkLUGpxuFA=;
        b=mg7T54iyEQowAbjTDPbUa2MCLP5J8uD3JpNnzuAHm+sy5Hf8Ix4ZWUFe+MELoH/Tei
         wtr2QFfell/cMV9x2QY43OIXP0UsUiPJgHl6qFal2jRGKg8sqdiSr0NSJ3nOV9xGnJp2
         E482Scs92AyhUCPixobIHgtl3K4Vlsy0IE3kdetpQtxRX7pCniJL7nDHkO11lA1IbnWb
         XJ5AVpFUFwg9JwWW9i1G4wNkvvHoYpKuBQI8in4RqaKnNN6WIJE02VmHdfcazSngHCmP
         7Bb6/GOconkCB3v/CHYIhbJLb7Y8o3xqP63L3qWyIgrJQf2jFmkLPyaaVVxyNK16+X69
         8wow==
X-Gm-Message-State: AOJu0Yw4/huBqKKD6HOhezlpLNovvQCIM+QYp2ZMrgEjEt0Pb3wpwTnw
	C7NiHll0SivGpJ7LL7346/bSXYtKNFhLH5iVR41+31wwEYib9cfu0XQB8WgCnCVSyW4CUZyCURT
	+
X-Google-Smtp-Source: AGHT+IGIvdq22S0y8DPMeTrCuoecZpdwQYUI2ac29ClGm0mlKZhV/SjKt2/zTJa7ry8D/culQ95ICw==
X-Received: by 2002:a17:907:7711:b0:a5c:dd3f:9151 with SMTP id a640c23a62f3a-a5cdd3f933cmr460592066b.35.1715962121227;
        Fri, 17 May 2024 09:08:41 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 0/3] Clean the policy manipulation path in domain creation
Date: Fri, 17 May 2024 17:08:33 +0100
Message-Id: <cover.1715954111.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

v2:
  * Removed xc_cpu_policy from xenguest.h
  * Added accessors for xc_cpu_policy so the serialised form can be extracted.
  * Modified xen-cpuid to use accessors.

==== Original cover letter ====

In the context of creating a domain, we currently issue a lot of hypercalls
redundantly while populating its CPU policy; likely a side effect of
organic growth more than anything else.

However, the worst part is not the overhead (this is a glacially cold
path), but the insane amounts of boilerplate that make it really hard to
pick apart what's going on. One major contributor to this situation is the
fact that what's effectively "setup" and "teardown" phases in policy
manipulation are not factored out from the functions that perform said
manipulations, leading to the same getters and setter being invoked many
times, when once each would do.

Another big contributor is the code being unaware of when a policy is
serialised and when it's not.

This patch attempts to alleviate this situation, yielding over 200 LoC
reduction.

Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
         from clients of libxc/libxg.
Patch 2: Changes the (de)serialization wrappers in xenguest so they always
         serialise to/from the internal buffers of xc_cpu_policy_t. The
         struct is suitably expanded to hold extra information required.
Patch 3: Performs the refactor of the policy manipulation code so that it
         follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.

Alejandro Vallejo (2):
  tools/xg: Streamline cpu policy serialise/deserialise calls
  tools/xg: Clean up xend-style overrides for CPU policies

 tools/include/xenguest.h            |   8 +-
 tools/libs/guest/xg_cpuid_x86.c     | 537 ++++++++++------------------
 tools/libs/guest/xg_private.h       |   2 +
 tools/libs/guest/xg_sr_common_x86.c |  54 +--
 tools/misc/xen-cpuid.c              |  43 +--
 5 files changed, 231 insertions(+), 413 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:06:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:06:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724354.1129583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812Y-0005ci-Pq; Fri, 17 May 2024 17:06:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724354.1129583; Fri, 17 May 2024 17:06:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812Y-0005cb-ND; Fri, 17 May 2024 17:06:42 +0000
Received: by outflank-mailman (input) for mailman id 724354;
 Fri, 17 May 2024 17:06:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s812X-0005cV-AE
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:06:41 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d2aa98b3-146f-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 19:06:39 +0200 (CEST)
Received: from DM6PR13CA0056.namprd13.prod.outlook.com (2603:10b6:5:134::33)
 by PH7PR12MB5973.namprd12.prod.outlook.com (2603:10b6:510:1d8::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Fri, 17 May
 2024 17:06:32 +0000
Received: from CH2PEPF00000099.namprd02.prod.outlook.com
 (2603:10b6:5:134:cafe::b5) by DM6PR13CA0056.outlook.office365.com
 (2603:10b6:5:134::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.12 via Frontend
 Transport; Fri, 17 May 2024 17:06:32 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF00000099.mail.protection.outlook.com (10.167.244.20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:06:31 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:31 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:06:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2aa98b3-146f-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CU3UhsL4EhL5lNaAHlC+trYba1yCRB+AuPvunvrBYe5kMPehpN5rpR6hY1olfUakwkFsJIbAlwBeZlJch/BOPAWqsoIhrB1SOD6iskCN3uU6f1Qa987lTbqGBwGjS5cAAIxegONrUeUSEM2GP/5R8517M8FhX+ebKVtixvzBvGgqhWF0hNOn4GJiD9+BrS4UD/9bIhrM+ExhYcMIrAXG3sZYEE+QpbPbWs0EZ96pTbKfm6ro3XVsfZ2S3+7ieVjIjanBnrTkmYAQQhgjbxU4aAancK+o44H/B1gdVLScvz2rlezapv14xbbJdfEilGFXXsth2ek3c3fD6quQtJPoSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ts3rSIFwmlBSfKyriMqkstTx6/HEQLoHhTNACy/CIpk=;
 b=fXnvFUkdzxRvAmg6AnypZeQFq/eHNIhKN8N1izwEbyu5rDsY5WESwo1YJyvUpaGHQ0Dk77WHzDfxtXEubM+IgcJA4PJcAJwPPc2r6dX5ZWt7vJmjr1LVNR+r13DIbsm5IQp1MuI4vyp+f8mAd7+CR4iHimV4PdTaMnC0mOHsro5LVV6C5I6vneXGCV9kiqm+/KuaHrITnR3+JDq7cCiBRZ7Qnz9QPP3KtVHO36rwx4jeULvCmevYlud1fnxEp7EpG3Dm/g1fiHFOo7k7Qbhn0poddTKF7SpKNkfKnKJuVHTFxxSUYiB+FKnuSozmq3pDTfv6OUmmp17S9ABMFFuQCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ts3rSIFwmlBSfKyriMqkstTx6/HEQLoHhTNACy/CIpk=;
 b=XG0QoieIBVKQavW2Vf5zVhGdqaG4GKjfNJlVYbjKSjdfwxo8ATLzqIfVNbMnFKX/Dsqyj6N8g8qa+r8brWfryXYmHtF+ilQFBnQWnXQJ98e2kZ6ToRhosrrsdqpEg/p+FZHrqCllV2q0sAM8kIYxi7QuF6ZppO+6hWaWpD/n8bY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stewart Hildebrand <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v15 0/5] PCI devices passthrough on Arm, part 3
Date: Fri, 17 May 2024 13:06:10 -0400
Message-ID: <20240517170619.45088-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000099:EE_|PH7PR12MB5973:EE_
X-MS-Office365-Filtering-Correlation-Id: f914b54e-5342-46f4-e521-08dc7693b33a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?LJJIsOtYYUbaaHzzeninox/ve41NG42U245Ka5qyklHDVEbhsTyLWWy6wjZ3?=
 =?us-ascii?Q?0E2aLN8K9LmhKBzw7qzOizbIt0TJint+3PZiCOmCOl4ddooQioNC8StMKDuB?=
 =?us-ascii?Q?TCZD+12XI8rmOdNEloEu1GrQabhMaH5ZXRgOYJMOImC0loTnOA1A1vH9fySG?=
 =?us-ascii?Q?iE6ytamSVJSUaQnkTr7CD7FItbD3LWbPOXFxI42i32lXwUkebMhUnsbz7cEh?=
 =?us-ascii?Q?XYfmuZq8Cf19U++J+AuIznza5hiMFQZlGfFkH4vMoU1gqa1hc4v7Y5ORXezh?=
 =?us-ascii?Q?9cawcG3U54mPBJpZuGhBV+qzxppX38KZrMfmoiaubFrbnnLeJmYacJkjsFIL?=
 =?us-ascii?Q?Xn04SO8qzelMwrKBAEz9V52gZn0TN3gYtXUkUlWvp42WIieRa+4CkFcm2q3I?=
 =?us-ascii?Q?yWKFy83gcGfmDp3fTS9h9JEDJFLFKCdc98GHUxTWr0gCQ8jGf7c+ZB8GAt2F?=
 =?us-ascii?Q?ooy+QWpslyKcsJH2l7R5FCaBW+kXPe3rxJdcshBMJYvJX0rvppZifu7ce/Gc?=
 =?us-ascii?Q?Q+jfF1ZJHyM7hso+gJCGMYWqRrVHIVhEjj/3tlqlvQnO40zdxFmp+jk1trM2?=
 =?us-ascii?Q?tbsvm1+qSTNLLt/o1avR8uSnjhtzWnL9rzxfNpoj50Fv08z5wBNKZ2AcQ3rK?=
 =?us-ascii?Q?A5j+qq3raIUyTII5S1WKKJN26SJnL8KOVnPzmwobIemO2KdxN1eeyyH6fUeR?=
 =?us-ascii?Q?0cShChb+21GCfxL9d4MSOgRbsKB7Eo2jZRDqj7yLN8ElSkr3NZpI7nPv52CK?=
 =?us-ascii?Q?tRt0FyfYaWZsolHUvfj0bvgsRZdeifaoXCr7YXFZaN+uiJhoJYwpL4sSeLWu?=
 =?us-ascii?Q?FFAiFdjQrERr3uTZ0vOaa93e7NRwMI9GCpNtuD6QD8vf67ACeGeMkv/V2Q4R?=
 =?us-ascii?Q?FzLKwI0qLX0z9q6xnohMy5208xVUycKbXrppCePex7WKzGsVhZAtYGsEDhYQ?=
 =?us-ascii?Q?YlxuQQ8uJpawvtJH02kg4kdNtV1HIt7l19Uiy9gJxsxE5nSHKhLGc9gkzD/E?=
 =?us-ascii?Q?ieHVZ3IM1P9LARndJUBVt/vZop9x9I7yxlaDscjKW9QwYU1/jQ4m2btinOwl?=
 =?us-ascii?Q?3i6wmAfaFBzyQP4B7u+99DRLG0yhrLMVl/HcnwGY492dDmeu4Kgj6g2vRKX2?=
 =?us-ascii?Q?orda676gH0ZqLiiECgfmXcl0bSeDulYrpheZyfMZz5c4GfiXS9wJI36gFRpi?=
 =?us-ascii?Q?ute82BUdj25jWxvxKXIsrbepdZVSvVH2QtKhq7JksB1B8TFlL9X+pVlw5Xkt?=
 =?us-ascii?Q?iuNGfMYeFy4t0BoqS33lQ43ast72gGP+qYI7F8+x2OUTgy0GUQjwEs6LT/zl?=
 =?us-ascii?Q?me9TtV4YPrGm7ldkLABWsJWctQXZ3cxyV0U2bFVqxybBnObBJxicuqCqLZLp?=
 =?us-ascii?Q?OTtn7aCyybQiG9P9aIROQsAto5ph?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:06:31.8394
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f914b54e-5342-46f4-e521-08dc7693b33a
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000099.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5973

This is next version of vPCI rework. Aim of this series is to prepare
ground for introducing PCI support on ARM platform.

in v15:
 - reorder so ("arm/vpci: honor access size when returning an error")
   comes first

in v14:
 - drop first 9 patches as they were committed
 - updated ("vpci/header: emulate PCI_COMMAND register for guests")

in v13:
 - drop ("xen/arm: vpci: permit access to guest vpci space") as it was
   unnecessary

in v12:
 - I (Stewart) coordinated with Volodomyr to send this whole series. So,
   add my (Stewart) Signed-off-by to all patches.
 - The biggest change is to re-work the PCI_COMMAND register patch.
   Additional feedback has also been addressed - see individual patches.
 - Drop ("pci: msi: pass pdev to pci_enable_msi() function") and
   ("pci: introduce per-domain PCI rwlock") as they were committed
 - Rename ("rangeset: add rangeset_empty() function")
       to ("rangeset: add rangeset_purge() function")
 - Rename ("vpci/header: rework exit path in init_bars")
       to ("vpci/header: rework exit path in init_header()")

in v11:
 - Added my (Volodymyr) Signed-off-by tag to all patches
 - Patch "vpci/header: emulate PCI_COMMAND register for guests" is in
   intermediate state, because it was agreed to rework it once Stewart's
   series on register handling are in.
 - Addressed comments, please see patch descriptions for details.

in v10:

 - Removed patch ("xen/arm: vpci: check guest range"), proper fix
   for the issue is part of ("vpci/header: emulate PCI_COMMAND
   register for guests")
 - Removed patch ("pci/header: reset the command register when adding
   devices")
 - Added patch ("rangeset: add rangeset_empty() function") because
   this function is needed in ("vpci/header: handle p2m range sets
   per BAR")
 - Added ("vpci/header: handle p2m range sets per BAR") which addressed
   an issue discovered by Andrii Chepurnyi during virtio integration
 - Added ("pci: msi: pass pdev to pci_enable_msi() function"), which is
   prereq for ("pci: introduce per-domain PCI rwlock")
 - Fixed "Since v9/v8/... " comments in changelogs to reduce confusion.
   I left "Since" entries for older versions, because they were added
   by original author of the patches.

in v9:

v9 includes addressed commentes from a previous one. Also it
introduces a couple patches from Stewart. This patches are related to
vPCI use on ARM. Patch "vpci/header: rework exit path in init_bars"
was factored-out from "vpci/header: handle p2m range sets per BAR".

in v8:

The biggest change from previous, mistakenly named, v7 series is how
locking is implemented. Instead of d->vpci_rwlock we introduce
d->pci_lock which has broader scope, as it protects not only domain's
vpci state, but domain's list of PCI devices as well.

As we discussed in IRC with Roger, it is not feasible to rework all
the existing code to use the new lock right away. It was agreed that
any write access to d->pdev_list will be protected by **both**
d->pci_lock in write mode and pcidevs_lock(). Read access on other
hand should be protected by either d->pci_lock in read mode or
pcidevs_lock(). It is expected that existing code will use
pcidevs_lock() and new users will use new rw lock. Of course, this
does not mean that new users shall not use pcidevs_lock() when it is
appropriate.

Changes from previous versions are described in each separate patch.

Oleksandr Andrushchenko (4):
  vpci/header: emulate PCI_COMMAND register for guests
  vpci: add initial support for virtual PCI bus topology
  xen/arm: translate virtual PCI bus topology for guests
  xen/arm: account IO handlers for emulated PCI MSI-X

Volodymyr Babchuk (1):
  arm/vpci: honor access size when returning an error

 xen/arch/arm/vpci.c        | 63 +++++++++++++++++++++++------
 xen/drivers/Kconfig        |  4 ++
 xen/drivers/vpci/header.c  | 60 +++++++++++++++++++++++++---
 xen/drivers/vpci/msi.c     |  9 +++++
 xen/drivers/vpci/msix.c    |  7 ++++
 xen/drivers/vpci/vpci.c    | 81 ++++++++++++++++++++++++++++++++++++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/sched.h    | 10 ++++-
 xen/include/xen/vpci.h     | 27 +++++++++++++
 9 files changed, 243 insertions(+), 19 deletions(-)


base-commit: 21611c68702dae2e18cb519a6e166cdceeaff4ca
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:06:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:06:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724355.1129592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812i-0005ts-4k; Fri, 17 May 2024 17:06:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724355.1129592; Fri, 17 May 2024 17:06:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812i-0005tf-24; Fri, 17 May 2024 17:06:52 +0000
Received: by outflank-mailman (input) for mailman id 724355;
 Fri, 17 May 2024 17:06:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s812h-0005cV-AO
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:06:51 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d964181a-146f-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 19:06:50 +0200 (CEST)
Received: from CH2PR14CA0060.namprd14.prod.outlook.com (2603:10b6:610:56::40)
 by DM3PR12MB9326.namprd12.prod.outlook.com (2603:10b6:0:3d::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 17:06:44 +0000
Received: from CH1PEPF0000A346.namprd04.prod.outlook.com
 (2603:10b6:610:56:cafe::27) by CH2PR14CA0060.outlook.office365.com
 (2603:10b6:610:56::40) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend
 Transport; Fri, 17 May 2024 17:06:44 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000A346.mail.protection.outlook.com (10.167.244.11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:06:43 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:43 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:43 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:06:37 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d964181a-146f-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P01g5uxPvru8qmcnL+KhXnJeJOJPzjku6XF3xFGF+Ophw8Wh2fGi5ZsaY6wMzAv4KW6LTsgu4sqpio53l18dbuFzRDYmxgMzBRbdxdzwEb9DCy2g7A/2B10N6gbfJKvJxVN7GpiS+4PFNXNJ8KSu+1jewe4/zPSGm27Wr4Q3Qs3VOn3IEmmu714XillsA8Q1fGLNdHvTGC9+5cI5rOQjf9pDah8wpS8F+nUmRggiMLrXMspuhBUwOaOqO+VZgkvDp4WmJ4yqmF81HTAJr5k4tgvKu72C1v9tNCfEGDJGM2VUyOaKzp5cC84CklMhMoh+yjrDOhUGeWtznNFIhA/u8Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=k8+/xJpZRZsK6+RKr2c/38/ePPkO2ljT1mguCOagC4g=;
 b=lCkUIbMsxmPmVPwA8WQfZHMciSU1fQJRq3/I+G5ti8Z77a0tbNPCFdLkwt5zi2ps31jV8yXrgrm1T0lCobVyYzGFK2cdN7xIA47UVlr/nR4cr8U1dyR5PyEHew50zlyP/0N9X3lC8cge15Q/gpPY1wV1mAAscHN7uCrBHXrmEnXuDDiWpd8hf6fuYJlov4yXI9uBOj+fk1dlQJJ1vprLO7v1q4q6wPBtH3K4Hq3B6iFBxiH8NPoq/ff0N2Q9268HWqpNUc7e59cfhkuI9iGDZnJov/JJNm+5Fs+rDneCl0iOe5kXkBUYRmR7RFc+An3UzXcPOSd+c3wXUTyPf8GHwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=k8+/xJpZRZsK6+RKr2c/38/ePPkO2ljT1mguCOagC4g=;
 b=p9EvjbHziTFxIK4yn069lzHVP/bSaJOUT945UxW09zLG8uHMFsPz/ttdU19+wEVtZm6jw7TS6b0AiD20pEOoJD5NIs7poLq4j8IiCxmiV30Y7cl40H26OqxWLRblxFLnxy9vCxO9ufXkc3DATtGn2xTEqoSiG2874Oi45jyuV+c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v15 1/5] arm/vpci: honor access size when returning an error
Date: Fri, 17 May 2024 13:06:11 -0400
Message-ID: <20240517170619.45088-2-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240517170619.45088-1-stewart.hildebrand@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000A346:EE_|DM3PR12MB9326:EE_
X-MS-Office365-Filtering-Correlation-Id: ea2952d8-bd52-48b8-ce1b-08dc7693ba57
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?nvzXThYk1gFpvov+RcY0hREqCRWQgPhOFrmAsPWXl3o0uLtyFP6GJmIlbtjZ?=
 =?us-ascii?Q?tukrzbVEA0eIF8OYcSp6L9hi6KaDNl4RsbxXe8sH780ZXmbKod6qrdhJHaWh?=
 =?us-ascii?Q?sobOYwAl6yWaNoKNMJFjMHR2LINJmmd8JeDhw7lY4hSGjw+LUHW0N2so6gw+?=
 =?us-ascii?Q?wHniDCEUniD+5obrGSpOB7Vk+42Mc0F/mqZy1PdDPr7r+slQrcKpYNtLON4Q?=
 =?us-ascii?Q?DLS2SBQ5t+ih2T4PCalCpGUzeg/tqnk2q2IkOwYtCVMwJimHzU0OgR2944/c?=
 =?us-ascii?Q?3E2q3zTgK0OnzFNIheBHD3Ely3V0umsG7kfIMViDp9wM4DtEunLQwhDIPjaU?=
 =?us-ascii?Q?1uf71Vb0kxd5531IjP4yFXh2/OTUQ7tCTeNzuME2QzI/gv67CPYIbq6A2FWI?=
 =?us-ascii?Q?oR8UkaAGo+UYXdCHgDoDm9z7DJU+dME76TapgQq6wvfzjyBTH8OWQexMGT5X?=
 =?us-ascii?Q?ZXNMQxhJIfAsPoH+kawSiQCefQZ0KCEyvc1YuGINDdiEzDy/Q/+YpG/c2gee?=
 =?us-ascii?Q?MsNExf+oqvcDWlNezAkR4s21L9jqg8ntohH/vhVrX5CIuGDkJh4zp6e/Ey4Y?=
 =?us-ascii?Q?rV4jBgzxku1WE5m0DCxNIXQCltt8mtConmJJXcNZm7q6OMAH+mihPoeO1gCR?=
 =?us-ascii?Q?K781H9tyAzw9PrsXvepZWdrZKyvnXHH2FXNHI3OB0QIyRpZ8hFulstgKtAVh?=
 =?us-ascii?Q?UhIgiY5VjxfVkHkbpWCg1Nty9t4tD8HfEF/+AeEOoBuSV6gkDxdp853LH2qN?=
 =?us-ascii?Q?h9vsAwv12xpc71TXHOAYKvWyJj8yW6xvhm+8PEjisFZeY/qdek6lTimAZq5x?=
 =?us-ascii?Q?qefJ99Oyz+fXybUrGOew1y3E9JnpHFZn2lsoy6O/3XB42RxlaszuvH0wiHE5?=
 =?us-ascii?Q?U1ll9O8jpcPRUo4Txw3w26wX1lnRos4H6b6FKiDVFaqiNzOhsVAKHyQcLiqt?=
 =?us-ascii?Q?6PhpXQ1bGiozPZYNzAj5py42S1SUOyPDCcLgqUtIQX4AarOHKcYV62AHdy8N?=
 =?us-ascii?Q?UiEeJzbmlENAdYSUlK/lhOaqmVi+1VcwMxFP7pyF0wn6HG6fIJBJ7Cz8svlz?=
 =?us-ascii?Q?ALoJMJ7sWDXlUh0RsLb6BeeWw1d3K0516X4jaAWDnqCmB3aInJ9mCrOZTvl8?=
 =?us-ascii?Q?Iqgf1y7H5W/jCxAmATpA86ye0nwhc+Kt9Nbo+ppS+z7MOWbdz6m9PQyFPtox?=
 =?us-ascii?Q?L3E9wa23v01USnvK7BdULkl5lO6Y3qfF4sgxkzPByDwqJ+qr/wTDoBUcqj2w?=
 =?us-ascii?Q?FLQuTaZBevCqw+GpIgjS9QXUK8HvYwufGR2T+K8uzTOrZb+crAwXSMz73sjG?=
 =?us-ascii?Q?RWSTfg6m+bksv82Y600H+D/XL2XtXsJFrt3A5igXSwAJ0APGH0cDmMHtzPOs?=
 =?us-ascii?Q?yIaQQ4FSAkw7GB/GGbFTpzVjhthX?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:06:43.7729
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ea2952d8-bd52-48b8-ce1b-08dc7693ba57
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000A346.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9326

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

Guest can try to read config space using different access sizes: 8,
16, 32, 64 bits. We need to take this into account when we are
returning an error back to MMIO handler, otherwise it is possible to
provide more data than requested: i.e. guest issues LDRB instruction
to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
register.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
v14->v15:
* re-order so this patch comes before ("xen/arm: translate virtual PCI
  bus topology for guests")
* s/access_mask/invalid/
* add U suffix to 1
* s/uint8_t/unsigned int/
* s/uint64_t/register_t/
* although Julien gave an Acked-by on v14, I omitted it due to the
  changes made in v15

v9->10:
* New patch in v10.
---
 xen/arch/arm/vpci.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 3bc4bb55082a..b63a356bb4a8 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 {
     struct pci_host_bridge *bridge = p;
     pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    const unsigned int access_size = (1U << info->dabt.size) * 8;
+    const register_t invalid = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
@@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0ul;
+    *r = invalid;
 
     return 0;
 }
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:06:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:06:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724356.1129603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812p-0006DK-Fd; Fri, 17 May 2024 17:06:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724356.1129603; Fri, 17 May 2024 17:06:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s812p-0006DB-B7; Fri, 17 May 2024 17:06:59 +0000
Received: by outflank-mailman (input) for mailman id 724356;
 Fri, 17 May 2024 17:06:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s812n-0006C6-Rw
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:06:57 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc2c6de2-146f-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 19:06:55 +0200 (CEST)
Received: from CH0PR03CA0197.namprd03.prod.outlook.com (2603:10b6:610:e4::22)
 by SA0PR12MB4430.namprd12.prod.outlook.com (2603:10b6:806:70::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 17:06:51 +0000
Received: from CH2PEPF00000099.namprd02.prod.outlook.com
 (2603:10b6:610:e4:cafe::af) by CH0PR03CA0197.outlook.office365.com
 (2603:10b6:610:e4::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30 via Frontend
 Transport; Fri, 17 May 2024 17:06:51 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF00000099.mail.protection.outlook.com (10.167.244.20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:06:51 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:50 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:50 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:06:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc2c6de2-146f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dcbDXjw5RSNAYliHXmmL/eqO5RfXY9IbNuSe8qlKjtSoYq0y2fsiMvDOojx7O5RcPSeoD/wg8vmrA1qelY6AfgAevW8i/uaUXPK/GmYsSoYRwm6R+XH/mwKb/naLoBT74UKvcnKn8bd+fmSB97dpUXzHOzAT1nCgjY6ycnRTUYBoKzPFUMlx4mOJuzii4EYVlzw/z9ULkKHtYNuEQKoqbPk9Ne1gSQhgQXRGyRzb/fK39JAejuGTgcSC/JBsJOSuRhI82SIbgTWK2nWSmu5beSIVDXvrcpJoeTJ/eYbbBT5xC7SuDv0zX+Izhk8oiGanVfd810lAe6MMINgDgkU2Fg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hUlhftlOCNqSzK4Q7JMuj42oNyZSiQtLeJCRs2LCTBA=;
 b=MWhKriMgg5Qwmjoe+zPyTTvDpvDlNxCfqaa2qjOvmA1oiDesjxZtLGttVbLQJaqN6SQ0j8YfSAnElcWv9FqYYPw4JjiSU2sRZ2C81fzF3dxUWzXMYsTqYcoaPZWry/h3qB8zHaNU2BUrvhU1DI6OJ5KCw/2zWQBTZ+A77fKCBTgo0odLAZ7wcPLeJegc88uFXSvp78Mm6ANfv2vC7gwcue7j+yBIiqF9aInTDefdxPXazATiKlt46RLllbmuxt7s37k1UJLAHKPS2jq/WfSQ0WuuMrjlUgBolPe/qGpyDnb48poCK6qnBnUX2uJGHit8r1iA0/74xapPvNUKaSs+yg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hUlhftlOCNqSzK4Q7JMuj42oNyZSiQtLeJCRs2LCTBA=;
 b=R42IKjJRy4Duzqziy8v6jDlUawoVM9bh6g7XCK0KKlBju54DUDbmNGuiFTklTqU3psmC+MubvW0d5ZSrIbcCZ3iiq31nAkn6BADEh2VZ5sktVgsb4x1jas3dYj5Q/uBf5aRahAbdhWS1nRqUHj9mVyTLmvauXz+as0jBOvuWwgQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v15 2/5] vpci/header: emulate PCI_COMMAND register for guests
Date: Fri, 17 May 2024 13:06:12 -0400
Message-ID: <20240517170619.45088-3-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240517170619.45088-1-stewart.hildebrand@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000099:EE_|SA0PR12MB4430:EE_
X-MS-Office365-Filtering-Correlation-Id: cc549463-26a4-4999-a970-08dc7693bee8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?0rRgFlFW7quJWwFnx8/5onH2wBWgvczo2wc3R79RSU+m/X2pMZsXWpfNpTMQ?=
 =?us-ascii?Q?+zKKahfxiwL90yV0nm8TPegP2Hl2Q2fNYWdfPpYXkAPLXyx1D5+cl/8ysMSW?=
 =?us-ascii?Q?xLOzPRHrnwM2719vdv1RiQXrsssfZXxGfYBR5ZCQjdyhrFt3eplOFhl19upE?=
 =?us-ascii?Q?1EyFqUwfT+4l+o/uzNxM9IqRHv/t93cKcnt7SwXcMBwMZmbAZ2+B8smcXHf9?=
 =?us-ascii?Q?1FhVK0rNpcs/9/AiehMby1FPuUpfSizo32YRS8o7TWlo2d3xI0FhEM2m+9de?=
 =?us-ascii?Q?0gGEzfIPmJFjwoVkTCvn7HHDVUyf7/u7bv7h7jSbtxv2Rdls/dJ0gHN4cHLW?=
 =?us-ascii?Q?sv0KjPPvFS/SYTqTamK3BD/V56MmMT/dR93dnNRnUisyrVUyPN7oiRpnowZ5?=
 =?us-ascii?Q?YJW0TK7mbTqvEmgTQjag+Ly8saEI40DnrPnoDwmCtgux99oR2wtz6l7UjIIx?=
 =?us-ascii?Q?ReHzuxPQYM+iPssrmfit1PivByOWuLqn3jiIzadFkFyJIc44rGWVbaZbfvhe?=
 =?us-ascii?Q?GX6JA4iYTwIr3tBJmrb1k+stZsvQW056BpWgZjs4wyH/xqBpKtPLUFN+KSqe?=
 =?us-ascii?Q?k+l5vcMhZhDQ9ph8t6gJKwjkRLSRdPUGtywuSoqExz/2O6RK8IQJqeMDqX4o?=
 =?us-ascii?Q?js7WOEfAy99Ac28sxcCsWdag0XBI06epNFtATXlPC1rwuqVISTUmhoCBP6oi?=
 =?us-ascii?Q?bAcKmSm5ADM0S3ygLVe1TI0S96EkSeCQvCJckUGeHlI4E59ymoh2MYwWpsm5?=
 =?us-ascii?Q?LiQ/se9GhtP11jB0m9JgaKPjHOkUm4fDYAXZy3bokWvqclsjMykIjvvEB2ZR?=
 =?us-ascii?Q?1g7LuqZe5Aq0LhtIv4mqTivBgsFKTHDiOoWAvwtW1dyaf6IICxaURIgs+HV0?=
 =?us-ascii?Q?GOtoxx0g9j7q5EJx1LdDUtpke/MG9z1pnF8xITfVBsqykYSCl2B8NSNwwqFM?=
 =?us-ascii?Q?hz384Ebx8srC+RZYtCDs4o5XlPAPp1UL890IIcgI6TPXFLlb1eiwUIWspumq?=
 =?us-ascii?Q?WvmstMYde7/0zSQZ6KxYq3Rllsn4MUuGWrX8pA5KgOvc+/UkTw4ej0Bfeoni?=
 =?us-ascii?Q?pTSDBIjSSN9XO7Rhi8zk7cu0yK76k9gJib557lAQBpTUfejc/r6F73fmlWE3?=
 =?us-ascii?Q?Akb41n+ewzE5renkL7+wdQXBd8kaXdDjiZbmVDA/ZBL4EKy0Y29OHZ6O5Z9t?=
 =?us-ascii?Q?PUppmBQIwhgdvabKPvzoFS+utXDr04W8zlb39/NN1jfuCWJlUo3Av0biqpem?=
 =?us-ascii?Q?nAldOgz6pLUkmJzmipWeMWLF2kILty9AAhAoT7b+z7l4ph5+IGcca0EbH/TX?=
 =?us-ascii?Q?BiQS3NzsBJcPZZtBj4ZmYaw0?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:06:51.4331
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: cc549463-26a4-4999-a970-08dc7693bee8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000099.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4430

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
guest's (domU) view of this will want to be zero (for now), the host
having set it to 1 should be preserved, or else we'd effectively be
giving the domU control of the bit. Thus, PCI_COMMAND register needs
proper emulation in order to honor host's settings.

According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
Device Control" the reset state of the command register is typically 0,
so when assigning a PCI device use 0 as the initial state for the
guest's (domU) view of the command register.

Here is the full list of command register bits with notes about
PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
also documented here since that is our current reference implementation
for PCI passthrough.

PCI_COMMAND_IO (bit 0)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if an I/O BAR is exposed to the guest.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
    don't yet support I/O BARs for domUs.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_MEMORY (bit 1)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if a Memory BAR is exposed to the guest.
  Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
    regions.
  Xen domU: For devices assigned to DomUs, memory decoding will be
    disabled at the time of initialization.

PCI_COMMAND_MASTER (bit 2)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_SPECIAL (bit 3)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_INVALIDATE (bit 4)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_VGA_PALETTE (bit 5)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_PARITY (bit 6)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_WAIT (bit 7)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: hardwire to 0
  QEMU: res_mask
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_SERR (bit 8)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_FAST_BACK (bit 9)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_INTX_DISABLE (bit 10)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU checks if INTx was mapped
    for a device. If it is not, then guest can't control
    PCI_COMMAND_INTX_DISABLE bit.
  Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
  Xen dom0: We allow dom0 to control this bit freely.

Bits 11-15
  PCIe 6.1: RsvdP
  PCI LB 3.0: Reserved
  QEMU: res_mask
  Xen domU/dom0: rsvdp_mask

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
RFC: There is an unaddressed question for Roger: should we update the
     guest view of the PCI_COMMAND_INTX_DISABLE bit in
     msi.c/msix.c:control_write()? See prior discussion at [1].
     In my opinion, I think we should make sure that hardware state and
     the guest view are consistent (i.e. don't lie to the guest).

[1] https://lore.kernel.org/xen-devel/86b25777-788c-4b9a-8166-a6f8174bedc9@suse.com/

In v15:
- add Jan's R-b
- add blank line after declaration in msi.c:control_write()

In v14:
- check for 0->1 transition in INTX_DISABLE-setting logic in
  msi.c:control_write() to match msix.c:control_write()
- clear domU-controllable bits in header.c:init_header()

In v13:
- Update right away (don't defer) PCI_COMMAND_MEMORY bit in guest_cmd
  variable in cmd_write()
- Make comment single line in xen/drivers/vpci/msi.c:control_write()
- Rearrange memory decoding disabling snippet in init_header()

In v12:
- Rework patch using vpci_add_register_mask()
- Add bitmask #define in pci_regs.h according to PCIe 6.1 spec, except
  don't add the RO bits because they were RW in PCI LB 3.0 spec.
- Move and expand TODO comment about properly emulating bits
- Update guest_cmd in msi.c/msix.c:control_write()
- Simplify cmd_write(), thanks to rsvdp_mask
- Update commit description

In v11:
- Fix copy-paste mistake: vpci->msi should be vpci->msix
- Handle PCI_COMMAND_IO
- Fix condition for disabling INTx in the MSI-X code
- Show domU changes to only allowed bits
- Show PCI_COMMAND_MEMORY write only after P2M was altered
- Update comments in the code
In v10:
- Added cf_check attribute to guest_cmd_read
- Removed warning about non-zero cmd
- Updated comment MSI code regarding disabling INTX
- Used ternary operator in vpci_add_register() call
- Disable memory decoding for DomUs in init_bars()
In v9:
- Reworked guest_cmd_read
- Added handling for more bits
Since v6:
- fold guest's logic into cmd_write
- implement cmd_read, so we can report emulated INTx state to guests
- introduce header->guest_cmd to hold the emulated state of the
  PCI_COMMAND register for guests
Since v5:
- add additional check for MSI-X enabled while altering INTX bit
- make sure INTx disabled while guests enable MSI/MSI-X
Since v3:
- gate more code on CONFIG_HAS_MSI
- removed logic for the case when MSI/MSI-X not enabled
---
 xen/drivers/vpci/header.c  | 60 ++++++++++++++++++++++++++++++++++----
 xen/drivers/vpci/msi.c     |  9 ++++++
 xen/drivers/vpci/msix.c    |  7 +++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/vpci.h     |  3 ++
 5 files changed, 74 insertions(+), 6 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 47648c395132..2491dbae8901 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -524,9 +524,21 @@ static void cf_check cmd_write(
 {
     struct vpci_header *header = data;
 
+    if ( !is_hardware_domain(pdev->domain) )
+    {
+        const struct vpci *vpci = pdev->vpci;
+
+        if ( (vpci->msi && vpci->msi->enabled) ||
+             (vpci->msix && vpci->msix->enabled) )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+
+        header->guest_cmd = cmd;
+    }
+
     /*
      * Let Dom0 play with all the bits directly except for the memory
-     * decoding one.
+     * decoding one. Bits that are not allowed for DomU are already
+     * handled above and by the rsvdp_mask.
      */
     if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) )
         /*
@@ -540,6 +552,14 @@ static void cf_check cmd_write(
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static uint32_t cf_check guest_cmd_read(
+    const struct pci_dev *pdev, unsigned int reg, void *data)
+{
+    const struct vpci_header *header = data;
+
+    return header->guest_cmd;
+}
+
 static void cf_check bar_write(
     const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
 {
@@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    /*
+     * Setup a handler for the command register.
+     *
+     * TODO: If support for emulated bits is added, re-visit how to handle
+     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
+     */
+    rc = vpci_add_register_mask(pdev->vpci,
+                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
+                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
+                                PCI_COMMAND_RSVDP_MASK |
+                                    (is_hwdom ? 0
+                                              : PCI_COMMAND_IO |
+                                                PCI_COMMAND_PARITY |
+                                                PCI_COMMAND_WAIT |
+                                                PCI_COMMAND_SERR |
+                                                PCI_COMMAND_FAST_BACK),
+                                0);
     if ( rc )
         return rc;
 
@@ -836,9 +870,23 @@ static int cf_check init_header(struct pci_dev *pdev)
     if ( pdev->ignore_bars )
         return 0;
 
-    /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
-    if ( cmd & PCI_COMMAND_MEMORY )
+
+    /*
+     * For DomUs, clear PCI_COMMAND_{MASTER,MEMORY,IO} and other
+     * DomU-controllable bits in PCI_COMMAND. Devices assigned to DomUs will
+     * start with memory decoding disabled, and modify_bars() will not be called
+     * at the end of this function.
+     */
+    if ( !is_hwdom )
+        cmd &= ~(PCI_COMMAND_VGA_PALETTE | PCI_COMMAND_INVALIDATE |
+                 PCI_COMMAND_SPECIAL | PCI_COMMAND_MASTER | PCI_COMMAND_MEMORY |
+                 PCI_COMMAND_IO);
+
+    header->guest_cmd = cmd;
+
+    /* Disable memory decoding before sizing. */
+    if ( !is_hwdom || (cmd & PCI_COMMAND_MEMORY) )
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY);
 
     for ( i = 0; i < num_bars; i++ )
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index 30adcf7df05d..dd6620ec5674 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -57,6 +57,8 @@ static void cf_check control_write(
 
     if ( new_enabled )
     {
+        bool old_enabled = msi->enabled;
+
         /*
          * If the device is already enabled it means the number of
          * enabled messages has changed. Disable and re-enable the
@@ -70,6 +72,13 @@ static void cf_check control_write(
 
         if ( vpci_msi_arch_enable(msi, pdev, vectors) )
             return;
+
+        /* Make sure domU doesn't enable INTx while enabling MSI. */
+        if ( !old_enabled && !is_hardware_domain(pdev->domain) )
+        {
+            pci_intx(pdev, false);
+            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
     }
     else
         vpci_msi_arch_disable(msi, pdev);
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index 58c16ebdf283..fbe710ab92ef 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -135,6 +135,13 @@ static void cf_check control_write(
         }
     }
 
+    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
+    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
+    {
+        pci_intx(pdev, false);
+        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+    }
+
     msix->masked = new_masked;
     msix->enabled = new_enabled;
 
diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h
index 0bc18efabb74..250ba106dbd3 100644
--- a/xen/include/xen/pci_regs.h
+++ b/xen/include/xen/pci_regs.h
@@ -48,6 +48,7 @@
 #define  PCI_COMMAND_SERR	0x100	/* Enable SERR */
 #define  PCI_COMMAND_FAST_BACK	0x200	/* Enable back-to-back writes */
 #define  PCI_COMMAND_INTX_DISABLE 0x400 /* INTx Emulation Disable */
+#define  PCI_COMMAND_RSVDP_MASK	0xf800
 
 #define PCI_STATUS		0x06	/* 16 bits */
 #define  PCI_STATUS_IMM_READY	0x01	/* Immediate Readiness */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 6e4c972f35ed..2064d9354f5b 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -107,6 +107,9 @@ struct vpci {
         } bars[PCI_HEADER_NORMAL_NR_BARS + 1];
         /* At most 6 BARS + 1 expansion ROM BAR. */
 
+        /* Guest (domU only) view of the PCI_COMMAND register. */
+        uint16_t guest_cmd;
+
         /*
          * Store whether the ROM enable bit is set (doesn't imply ROM BAR
          * is mapped into guest p2m) if there's a ROM BAR on the device.
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:07:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:07:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724363.1129612 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813A-00075f-R4; Fri, 17 May 2024 17:07:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724363.1129612; Fri, 17 May 2024 17:07:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813A-00075U-OX; Fri, 17 May 2024 17:07:20 +0000
Received: by outflank-mailman (input) for mailman id 724363;
 Fri, 17 May 2024 17:07:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s8139-0005cV-Ez
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:07:19 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea290aca-146f-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 19:07:18 +0200 (CEST)
Received: from CH0PR07CA0014.namprd07.prod.outlook.com (2603:10b6:610:32::19)
 by SJ0PR12MB6781.namprd12.prod.outlook.com (2603:10b6:a03:44b::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Fri, 17 May
 2024 17:07:13 +0000
Received: from CH2PEPF000000A0.namprd02.prod.outlook.com
 (2603:10b6:610:32:cafe::d6) by CH0PR07CA0014.outlook.office365.com
 (2603:10b6:610:32::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30 via Frontend
 Transport; Fri, 17 May 2024 17:07:13 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF000000A0.mail.protection.outlook.com (10.167.244.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:07:12 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:07:12 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:07:12 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:07:05 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea290aca-146f-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f4zeHa5+mYcNejHDgbSebek8KjYBpkQK2u5AlNgP+8Ippj2O3xRRv7FciQrlt4yX2LMXjJPAEh6DByroDztdhjLbCGwNsGMM4WgeTlO75rSg4TzqDGA3mJu+oGBf4+ipBaf8IgTk3HQkV6CDToaxQJk9iWeJAzE6z1OCyKUXkEzyZ6PW+MDigjWw9is/76yeX5LSXkwJcbKzVBhc33N9E/ve4ZAgiAdz3vS8M5yoATI439kNYgtUmxcMvN0K0LReAd1KfIiHCe0+UHiYMwzP7UhAvlp8E1USR+5nqtL6Ywt+RXlqO0+8B9mAEVC0DQL/FAKlbyVOVDCtUxZOYO9Rsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zUM6dVyovYpjq8KjZD0l2PS8q/TOLp/Wit92KUplaCw=;
 b=JN00j14+XaNx7V/ype4xKqhW6VbtqyqIYd9MOe6QeHdpZIIykewkc+Jljghvc/BV3sJ+wLbpxaYhhd0CGmgo2KEP3qF2Iezn5lAEBtcPA/GFvfHcZ+c3hYt11WsoLxzXgs4ye4OqpUjIP0c+CvMTnIORJEJmAVcj6N541Xt/2c/rx5O/f+Lp8qtQNPoCoBI0CULrvT0W59uFqhRZ/DYgq3t+DcnOosKz59ag9pwuSgg3Yq1UpvZPiERXDTWt8PSLM1WGmORW9Le2FVnaBE8+EF7P9fCnb6mdRsPsbv2KrO2rbo9ZLdCpMFtlrYc5/gIzBiWyHGHP7sB1uZ9bMlr36Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zUM6dVyovYpjq8KjZD0l2PS8q/TOLp/Wit92KUplaCw=;
 b=gnayb4iCZKRb2sZgjkEyZQ8C4m3nAFCZz01hI92Ijyx0b5yX5A9/lBK5/Z6Aa+vmU9bCWGVbiHmV2xoDeWwSEkisGlhjBnO+vX6zuuv5asU4AArpxceNv8Jnsii2CZ7QCGXpPiiprFzLGoICldpZLcaJlDBKlDbAalEinqA88c8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v15 4/5] xen/arm: translate virtual PCI bus topology for guests
Date: Fri, 17 May 2024 13:06:14 -0400
Message-ID: <20240517170619.45088-5-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240517170619.45088-1-stewart.hildebrand@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF000000A0:EE_|SJ0PR12MB6781:EE_
X-MS-Office365-Filtering-Correlation-Id: f5c0efbd-de52-42a2-50cc-08dc7693cba1
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?CTwG2Q13s/um5srL3kk8yIvX9sf9IhUslUJiU0HBvoAqJu3S25dnTFvbPvSO?=
 =?us-ascii?Q?TTwCJznSa8zFuftEpNXI+n19tRjUs/8BeEFcsacOVp89LG8QRrAL87W/1xga?=
 =?us-ascii?Q?vodKMFDV9HJ+ajQt675qVgdhAfnZE0Qh+oQ3awMpxOv8iwIi5VqS6vCL2atD?=
 =?us-ascii?Q?iYgMGRx/Yd/t6PC5VYRtiWVprcQNV77yWwBRP+kHS0RvLFOHgZBK1isElQcZ?=
 =?us-ascii?Q?pxLBT+7F2J2zoxNFaAJBGAO4dfLRD3pl46cNkF+oqn2VK65jCCZOtffndTcl?=
 =?us-ascii?Q?5ZYa5E6Ae64uQMIsf4mE+g+Zz7YUjvAGVmuOveYWEMUhOaDlDk+SRrxlA2ts?=
 =?us-ascii?Q?+FW6Hq9zTTTfcaZ39N+7vs0pMU6pxWAZmGHZzGsB7DMBc1hZR7cLCLaClDlO?=
 =?us-ascii?Q?dCjYZE5tRX62bkvV0oMJNZ/JR0lU3tN9vDURC/A6EMmAufVlXgafhIRGWIqi?=
 =?us-ascii?Q?XqkZYMSyLlgqE63UL7LqItik2CcNshy/4ud+Hxflv+0eXhgx0WtlxFy/a7Q6?=
 =?us-ascii?Q?R3aEXyzk8cxozLmtI3XGhoOQIc53CIBBPSolnIffxzuWd2DiaT2xTLd8AsIo?=
 =?us-ascii?Q?W2jcfDL/ZIjTaDYnAZ5tsyudr83kgGZo+n3prhgguMeYSm2NMAn9HNxy1TIP?=
 =?us-ascii?Q?Q/Gzfi0AXDWaMRj7gS06Gp/jm+c7JSsBMkwUucMZfW9yBsKvNa/8AC81jog4?=
 =?us-ascii?Q?SAGAvMFe6asI9Z0yzhQd0LQLR+vuPxJo4BqixKy9blkDIXm0w44w4Ou08P6O?=
 =?us-ascii?Q?ODHR5jLEsOI54uy2ykJjYcI/Z024H/uo87kSqWvt2BalyRRRaNAHe9UX3Hz0?=
 =?us-ascii?Q?RhXG/OWQ8yqWBEYMD4VHfpmbqSO+rOgntjaujbyG6uggG7s5RljCKP0mC5N/?=
 =?us-ascii?Q?LbXQ2WSXoeYKfhtj+7bKqjt795E7Ftvnn/zTas6SwJS4k12qWszxMap5sfmh?=
 =?us-ascii?Q?C9IBaPmaTs001GgMknX/nd676pD7+GokngWNqcGKcq3HUQrseMHIWj+n/97y?=
 =?us-ascii?Q?Kp4xe6bQlzoIsRexd6UGlRIV2CFKhWopJ4YUgdu9oL+me1i092eT7nEfv7Rs?=
 =?us-ascii?Q?/JZnSx5x+Hh7zXgnkLZi9oPyhTaxfX4PkqK6un87/9WnZ+3maoZLi9aGoVjm?=
 =?us-ascii?Q?oDS67mylUJELWMFX3WvXS76yYfz3nF2QJYjtRbLr+q4Oi3vlKyHh73nkdUKp?=
 =?us-ascii?Q?8ESC0hTCctC/tSkgqF4q9vmJc4FdWwSKSYal0aVK8QfFe7YJPChu/gFzK9Dd?=
 =?us-ascii?Q?7gUcWOYFrO3jGAxCiDiVmCtprBzPlF7Kbl6a6FNuf1DWhK69Ma7LraqHjfnn?=
 =?us-ascii?Q?jXBTF8XgZhTiSD+xHzyrakZQ5tw4Uo5E+8BaCWSyV+ERkVCWGIsuQZSL1mnO?=
 =?us-ascii?Q?fh51sqVr4Ws3mhficJ2CcJ+oxSb+?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:07:12.7759
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f5c0efbd-de52-42a2-50cc-08dc7693cba1
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF000000A0.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6781

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v15:
- base on top of ("arm/vpci: honor access size when returning an error")
In v11:
- Fixed format issues
- Added ASSERT_UNREACHABLE() to the dummy implementation of
vpci_translate_virtual_device()
- Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
the logic in the function
Since v9:
- Commend about required lock replaced with ASSERT()
- Style fixes
- call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
Since v8:
- locks moved out of vpci_translate_virtual_device()
Since v6:
- add pcidevs locking to vpci_translate_virtual_device
- update wrt to the new locking scheme
Since v5:
- add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
  case to simplify ifdefery
- add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
- reset output register on failed virtual SBDF translation
Since v4:
- indentation fixes
- constify struct domain
- updated commit message
- updates to the new locking scheme (pdev->vpci_lock)
Since v3:
- revisit locking
- move code to vpci.c
Since v2:
 - pass struct domain instead of struct vcpu
 - constify arguments where possible
 - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/arch/arm/vpci.c     | 45 ++++++++++++++++++++++++++++++++---------
 xen/drivers/vpci/vpci.c | 24 ++++++++++++++++++++++
 xen/include/xen/vpci.h  | 12 +++++++++++
 3 files changed, 71 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index b63a356bb4a8..516933bebfb3 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -7,33 +7,53 @@
 
 #include <asm/mmio.h>
 
-static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
-                                     paddr_t gpa)
+static bool vpci_sbdf_from_gpa(struct domain *d,
+                               const struct pci_host_bridge *bridge,
+                               paddr_t gpa, pci_sbdf_t *sbdf)
 {
-    pci_sbdf_t sbdf;
+    bool translated = true;
+
+    ASSERT(sbdf);
 
     if ( bridge )
     {
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
-        sbdf.seg = bridge->segment;
-        sbdf.bus += bridge->cfg->busn_start;
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
+        sbdf->seg = bridge->segment;
+        sbdf->bus += bridge->cfg->busn_start;
     }
     else
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+    {
+        /*
+         * For the passed through devices we need to map their virtual SBDF
+         * to the physical PCI device being passed through.
+         */
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+        read_lock(&d->pci_lock);
+        translated = vpci_translate_virtual_device(d, sbdf);
+        read_unlock(&d->pci_lock);
+    }
 
-    return sbdf;
+    return translated;
 }
 
 static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
                           register_t *r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
     const unsigned int access_size = (1U << info->dabt.size) * 8;
     const register_t invalid = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+    {
+        *r = invalid;
+        return 1;
+    }
+
     if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa),
                         1U << info->dabt.size, &data) )
     {
@@ -50,7 +70,12 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
                            register_t r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
+
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+        return 1;
 
     return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
                            1U << info->dabt.size, r);
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 23722634d50b..98b294f09688 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -81,6 +81,30 @@ static int add_virtual_device(struct pci_dev *pdev)
     return 0;
 }
 
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
+{
+    const struct pci_dev *pdev;
+
+    ASSERT(!is_hardware_domain(d));
+    ASSERT(rw_is_locked(&d->pci_lock));
+
+    for_each_pdev ( d, pdev )
+    {
+        if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) )
+        {
+            /* Replace guest SBDF with the physical one. */
+            *sbdf = pdev->sbdf;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
 void vpci_deassign_device(struct pci_dev *pdev)
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 980aded26fc1..7e5a0f0c50c1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -303,6 +303,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v)
 }
 #endif
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
+#else
+static inline bool vpci_translate_virtual_device(const struct domain *d,
+                                                 pci_sbdf_t *sbdf)
+{
+    ASSERT_UNREACHABLE();
+
+    return false;
+}
+#endif
+
 #endif
 
 /*
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:07:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:07:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724364.1129623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813C-0007Le-4f; Fri, 17 May 2024 17:07:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724364.1129623; Fri, 17 May 2024 17:07:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813C-0007LT-1A; Fri, 17 May 2024 17:07:22 +0000
Received: by outflank-mailman (input) for mailman id 724364;
 Fri, 17 May 2024 17:07:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s813A-0006C6-OQ
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:07:20 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea53e26b-146f-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 19:07:19 +0200 (CEST)
Received: from DM6PR13CA0061.namprd13.prod.outlook.com (2603:10b6:5:134::38)
 by PH0PR12MB7887.namprd12.prod.outlook.com (2603:10b6:510:26d::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Fri, 17 May
 2024 17:07:13 +0000
Received: from CH1PEPF0000A349.namprd04.prod.outlook.com
 (2603:10b6:5:134:cafe::d3) by DM6PR13CA0061.outlook.office365.com
 (2603:10b6:5:134::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.13 via Frontend
 Transport; Fri, 17 May 2024 17:07:13 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000A349.mail.protection.outlook.com (10.167.244.9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:07:12 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:59 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:06:58 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:06:57 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea53e26b-146f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FXlIt6aRLa/lQKTfM+WWsVar9wDtckkBisIw8/U4Arw0/P7gsnMk+eY6LuuouvOQc3nrUENeMy8QWbhF3BQnGHV7qeSuB6FALovQF70pnPgjkmTJY1Uhkk/4XT/6oTUmJ+p03Lm9EGHRQA0Lrt05E8RaRUgSx9hRDKfMDvFeQDlSH1O8DQVKjDqpK8Qvh45NJBM7ekphsKSwoNGHlj/u9kVRNx+lkZW67QHK0hmgEjAJd7oQ4SaqiJjy21xIE+/cIsaxm+yxOAFY01pl3zdcYnhEpytBECtJ5YWqYIlK3D+sKVz/XLYY+0tw1kLUTNxDSWDxNW9s+rYtYqQu8rAEQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=bv7+nDlnnhjXbpYGAA9ivNDf2bKqLCBADTr28OUk9Vg=;
 b=aETuViYGb5htggsb9glyy2sZzlELADBNsd26btEK+QcbgRUYQ9UOK9HHyLlOW0s3xzazArgUcn279N4aeK1dOZCwS9Xel8QLfegtMJ7LOrAVT9GsFDvCJL/XlMKzRnqv4Zt2LDtB7IOWacfuiM9D1OWM7bdo8RQSK7YCT3hJJQGJ+pTSLp0lOtPSKkMdEkAZbmbivPq/5UjwUhtsYAL53TwBP0loOc3Kudy1PD64PQ8xZeN8UoSSUBUgsfWnjKpD5BnQzDhM6zJxpQuf59b0aRhQVjrkeTZ7O8/CkcAUmMS5/fjZmMroo5BYLTeDUF69IEyR3mvRwJ/2IyjZ7mBsgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bv7+nDlnnhjXbpYGAA9ivNDf2bKqLCBADTr28OUk9Vg=;
 b=AJ5xnYte03j+kOZ4Is6aWcOEyWtVqVhYOxflDYeXctbkZKM+CU7lnsQMJ262LiZhV5ldrO4+80kkvSWZkzais8O3eL3dqNyjDA6btvFATPnj0iuJq03HUJcvKlNe0Iv2S5H2P2LrYZowul/E3uuUVz1aSYm+oAM3ejJRPnngT+M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v15 3/5] vpci: add initial support for virtual PCI bus topology
Date: Fri, 17 May 2024 13:06:13 -0400
Message-ID: <20240517170619.45088-4-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240517170619.45088-1-stewart.hildebrand@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000A349:EE_|PH0PR12MB7887:EE_
X-MS-Office365-Filtering-Correlation-Id: 472dc026-7d09-452b-9f67-08dc7693cb8a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?/i+5S5SGy2bG6UpvQvWOsdrdytOEUQj/mqE+G1xy11UV6daVf3zfRrNg9qzf?=
 =?us-ascii?Q?o7N/UVYNaMao2QpONcgAhgcP8AVfztautVjXy9Kl65EIUpIci7Pew3Sjtxgx?=
 =?us-ascii?Q?aKG9LMmyE9toE7COZUf9UDp/gGgVNABJV8vmz9Ji28Vg1a1uFDh/PTmEEIar?=
 =?us-ascii?Q?ijYyR1V9TDMOg+CL3e51w1x7Dwbcp4zp3Zul5YyNzX5qGVseQ/8ulb+JjiYl?=
 =?us-ascii?Q?MSGy/i+WLqNLg3I15L/r3JG+C/uQIS9eCAO7JL2CLUe3bKQ8/Cdr1kRjKBza?=
 =?us-ascii?Q?tG+4X61Zdw6VNoi9/Gv46xaGUk7Cn1q/ACJdYsgI+uS7sQAXWkXYi0VW2Sjk?=
 =?us-ascii?Q?Vx/O4fndl6+JeC2+CYCippRk5wXfjRrGJFJE3l8Ns5+5Zza8AjjQ4s4kVbWm?=
 =?us-ascii?Q?gprBgHqpDCssdi52Veq72sJFu1HyY7nQyig+uvpCLoIqUoK/QHEOfV/NIOM0?=
 =?us-ascii?Q?5qHqgb8iaHn0fRwNbDuCAJrA28yQ5h66YgepwM/84pswG5/SrXTCFF4Jn99n?=
 =?us-ascii?Q?36lsfTPXqWonCIfYnX/48gFq7m25TSaHf3BQYNPudIq0XJY8qz+Eq/bfnGyy?=
 =?us-ascii?Q?e3RVfjMCnaOdPriIhD6iSQldTOnhJ0EeN2EoQP+PODQBWqAt/KDly/8L2dRT?=
 =?us-ascii?Q?Qw3rab3uanExQFuEHIB2hpSNsFcsBObRL/Kx6SG/Dp93F1nUJZ/LUAgMr5v3?=
 =?us-ascii?Q?XwFU4hnPWhIiSuQczQA+t2AKnGvclLSYzwy8Pa09y3bKlQmTp8u0S5Y3R4RF?=
 =?us-ascii?Q?DPAi06gAelGBrb10iwenSpsMXyfi0H7guJI71ivHN+LECEs5UUKBibdl3kDG?=
 =?us-ascii?Q?Fa6o90i2uPaO4HO2jIyRERoP+NiT1IvVJ7AgOAedYvy/uHqiF8LzkTrG+b4u?=
 =?us-ascii?Q?c5MV4SpO2IvRlqQKNTa7vnGb9FPJ25e5CUc5BauayLF256LrHNhiUb+IPDgN?=
 =?us-ascii?Q?M/JwU+MhwVybQp7qn0LfStz0AyLouIomBWnVbmQqfoAiDJ7ApY8seKS3fEqJ?=
 =?us-ascii?Q?4D9mRKa8smVPErQKtrC5WUc1NT2wZsviFHkgShUihpUYtLZmgCNitwGwbI0/?=
 =?us-ascii?Q?dw6x1M6bvvVi1p22l+oghRjVv4fLt7c8MupoxlrXHRBi+RWwA4KZtae1DdA+?=
 =?us-ascii?Q?B2bft58yiptwdOr1d4I7h3rn4T1eGFaKddHvKR1JFyOSuJEa+Zf4qKOxsHLv?=
 =?us-ascii?Q?V9zy2BjoEOeHruL1Lyh3N/teBlyBD/BRy2ddYRHjWrB9Zf+Zy2+8i+v3SdBz?=
 =?us-ascii?Q?RJr1DPVooJ8FO7NAoL+rCVs/V2qoRKctAB7Qv24nP8NPNN3ElMu9X9Y92+vI?=
 =?us-ascii?Q?3V6ZY7JKvBQ9D6BXrWDe0Ve/MTuhtn7QXcVIHYrH/AfDyzRzPi5NQOqLRjd4?=
 =?us-ascii?Q?thDA/og=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:07:12.6272
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 472dc026-7d09-452b-9f67-08dc7693cb8a
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000A349.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7887

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Please note, that at the moment only function 0 of a multifunction
device can be passed through.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
In v15:
- add Jan's A-b
In v13:
- s/depends on/select/ in Kconfig
- check pdev->sbdf.fn instead of two booleans in add_virtual_device()
- comment #endifs in sched.h
- clarify comment about limits in vpci.h with seg/bus limit
In v11:
- Fixed code formatting
- Removed bogus write_unlock() call
- Fixed type for new_dev_number
In v10:
- Removed ASSERT(pcidevs_locked())
- Removed redundant code (local sbdf variable, clearing sbdf during
device removal, etc)
- Added __maybe_unused attribute to "out:" label
- Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the
  first patch where it is used (previously was in "vpci: add hooks for
  PCI device assign/de-assign")
In v9:
- Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart)
In v8:
- Added write lock in add_virtual_device
Since v6:
- re-work wrt new locking scheme
- OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device()
Since v5:
- s/vpci_add_virtual_device/add_virtual_device and make it static
- call add_virtual_device from vpci_assign_device and do not use
  REGISTER_VPCI_INIT machinery
- add pcidevs_locked ASSERT
- use DECLARE_BITMAP for vpci_dev_assigned_map
Since v4:
- moved and re-worked guest sbdf initializers
- s/set_bit/__set_bit
- s/clear_bit/__clear_bit
- minor comment fix s/Virtual/Guest/
- added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used
  later for counting the number of MMIO handlers required for a guest
  (Julien)
Since v3:
 - make use of VPCI_INIT
 - moved all new code to vpci.c which belongs to it
 - changed open-coded 31 to PCI_SLOT(~0)
 - added comments and code to reject multifunction devices with
   functions other than 0
 - updated comment about vpci_dev_next and made it unsigned int
 - implement roll back in case of error while assigning/deassigning devices
 - s/dom%pd/%pd
Since v2:
 - remove casts that are (a) malformed and (b) unnecessary
 - add new line for better readability
 - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
    functions are now completely gated with this config
 - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/drivers/Kconfig     |  4 +++
 xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched.h | 10 +++++++-
 xen/include/xen/vpci.h  | 12 +++++++++
 4 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..20050e9bb8b3 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	select HAS_VPCI
+
 endmenu
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 97e115dc5798..23722634d50b 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+static int add_virtual_device(struct pci_dev *pdev)
+{
+    struct domain *d = pdev->domain;
+    unsigned int new_dev_number;
+
+    if ( is_hardware_domain(d) )
+        return 0;
+
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
+    /*
+     * Each PCI bus supports 32 devices/slots at max or up to 256 when
+     * there are multi-function ones which are not yet supported.
+     */
+    if ( pdev->sbdf.fn )
+    {
+        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
+                 &pdev->sbdf);
+        return -EOPNOTSUPP;
+    }
+    new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map,
+                                         VPCI_MAX_VIRT_DEV);
+    if ( new_dev_number == VPCI_MAX_VIRT_DEV )
+        return -ENOSPC;
+
+    __set_bit(new_dev_number, &d->vpci_dev_assigned_map);
+
+    /*
+     * Both segment and bus number are 0:
+     *  - we emulate a single host bridge for the guest, e.g. segment 0
+     *  - with bus 0 the virtual devices are seen as embedded
+     *    endpoints behind the root complex
+     *
+     * TODO: add support for multi-function devices.
+     */
+    pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0);
+
+    return 0;
+}
+
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 void vpci_deassign_device(struct pci_dev *pdev)
 {
     unsigned int i;
@@ -49,6 +92,12 @@ void vpci_deassign_device(struct pci_dev *pdev)
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    if ( pdev->vpci->guest_sbdf.sbdf != ~0 )
+        __clear_bit(pdev->vpci->guest_sbdf.dev,
+                    &pdev->domain->vpci_dev_assigned_map);
+#endif
+
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
     {
@@ -103,6 +152,13 @@ int vpci_assign_device(struct pci_dev *pdev)
     INIT_LIST_HEAD(&pdev->vpci->handlers);
     spin_lock_init(&pdev->vpci->lock);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    pdev->vpci->guest_sbdf.sbdf = ~0;
+    rc = add_virtual_device(pdev);
+    if ( rc )
+        goto out;
+#endif
+
     for ( i = 0; i < NUM_VPCI_INIT; i++ )
     {
         rc = __start_vpci_array[i](pdev);
@@ -110,6 +166,7 @@ int vpci_assign_device(struct pci_dev *pdev)
             break;
     }
 
+ out: __maybe_unused;
     if ( rc )
         vpci_deassign_device(pdev);
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b84199598..2dcd1d1a4f8a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -486,7 +486,15 @@ struct domain
      * 2. pdev->vpci->lock
      */
     rwlock_t pci_lock;
-#endif
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /*
+     * The bitmap which shows which device numbers are already used by the
+     * virtual PCI bus topology and is used to assign a unique SBDF to the
+     * next passed through virtual PCI device.
+     */
+    DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV);
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+#endif /* CONFIG_HAS_PCI */
 
 #ifdef CONFIG_HAS_PASSTHROUGH
     struct domain_iommu iommu;
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 2064d9354f5b..980aded26fc1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -21,6 +21,14 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 
 #define VPCI_ECAM_BDF(addr)     (((addr) & 0x0ffff000) >> 12)
 
+/*
+ * Maximum number of devices supported by the virtual bus topology:
+ * each PCI bus supports 32 devices/slots at max or up to 256 when
+ * there are multi-function ones which are not yet supported.
+ * This limit implies only segment 0, bus 0 is supported.
+ */
+#define VPCI_MAX_VIRT_DEV       (PCI_SLOT(~0) + 1)
+
 #define REGISTER_VPCI_INIT(x, p)                \
   static vpci_register_init_t *const x##_entry  \
                __used_section(".data.vpci." p) = (x)
@@ -175,6 +183,10 @@ struct vpci {
             struct vpci_arch_msix_entry arch;
         } entries[];
     } *msix;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /* Guest SBDF of the device. */
+    pci_sbdf_t guest_sbdf;
+#endif
 #endif
 };
 
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 17:07:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 17:07:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724367.1129632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813P-0007rn-I0; Fri, 17 May 2024 17:07:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724367.1129632; Fri, 17 May 2024 17:07:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s813P-0007rg-F6; Fri, 17 May 2024 17:07:35 +0000
Received: by outflank-mailman (input) for mailman id 724367;
 Fri, 17 May 2024 17:07:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KIA2=MU=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s813O-0006C6-Ag
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 17:07:34 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2977c4c-146f-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 19:07:32 +0200 (CEST)
Received: from CH0PR04CA0018.namprd04.prod.outlook.com (2603:10b6:610:76::23)
 by BL1PR12MB5921.namprd12.prod.outlook.com (2603:10b6:208:398::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May
 2024 17:07:26 +0000
Received: from CH2PEPF0000009C.namprd02.prod.outlook.com
 (2603:10b6:610:76:cafe::19) by CH0PR04CA0018.outlook.office365.com
 (2603:10b6:610:76::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.31 via Frontend
 Transport; Fri, 17 May 2024 17:07:26 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF0000009C.mail.protection.outlook.com (10.167.244.24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7587.21 via Frontend Transport; Fri, 17 May 2024 17:07:25 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 17 May
 2024 12:07:25 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 17 May 2024 12:07:19 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2977c4c-146f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mu3r6ORU84PuX3HDQ7SwzWqlPti1SO9+IBG+NK7Lc6MXrIJH47Owd+6a7a9HtlhdifYm3RMsdUZV3CTt5NGAjjZghczZKzhg5bb5+E8LN1N1iaejzmsq7QPe8pFuFS7D0a//EFswBJ0N1cCndia4KN8n3rpr81N7dUkEugc4+Gg3519U3XvgxzkQkBz6SEtRlkppjhaIL08Y+6qiHsPg4NMtGyH7wQP+IDc9q44XT+SSH4sMuZ7xqK7FKE3ryLe9tXofjLQxI36skZ/70cqhciT7+BF68W6jMegb6ay6RqKCE3nMXIEzdS6RY1Nsf9hHha13I3iMBBU5ohIBsMrN9Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wg1lIYFsxCkrbMSdoPQQCZQ7ozFKfKSY7zA6PHJCXCc=;
 b=c2QhoeBJ6EdKJXNGkRQGfCTo0BnA+L575RBbWFhAzsr2GuKMbAdQue7Noy5D/+AlH/HjDpr0Bh9bfGv6rIwU5WKcRNgmguTiZqKsnx+aZnArpYRycko5ZrFlc4/Lh5Gcv8RsxEmA/sJwz5C5DQVn8AoUwAbPu6lwt8/BE9sVYYhbYJzp5nXVv0Du0E3aolO7k7pqhJE6RBmIzoNdgFB05Roun1NW29M7R2XHuNfsbcDUAVc8lUqCEUktZs990uOWnuSMmC3Zsf0KASMnXbB4pMnepb5xO6yweIoWxdW7kdn7jPiio6Yvm7+q5Hg4109UxMrPC9Wkda+C0pePnVhx3A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wg1lIYFsxCkrbMSdoPQQCZQ7ozFKfKSY7zA6PHJCXCc=;
 b=DNQ3978As1te2Q0tl4Hxt8zwQoa3PtLIAPQJE6z9JczMc0Zvm8MH0gAJ8H3OXYgPCEdAmNSGx5t0N1qwtgqXAxAeMvH/dPiVbxN/jXiebG+OAyt9Yv/dKbeN7ZLJt38ZK+Xud550jbx8qLLHBhJOwHmTRTwgP98uBREp+Lv7vC0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Julien Grall <jgrall@amazon.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v15 5/5] xen/arm: account IO handlers for emulated PCI MSI-X
Date: Fri, 17 May 2024 13:06:15 -0400
Message-ID: <20240517170619.45088-6-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240517170619.45088-1-stewart.hildebrand@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF0000009C:EE_|BL1PR12MB5921:EE_
X-MS-Office365-Filtering-Correlation-Id: b40d0cd0-3024-40b6-df5d-08dc7693d386
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?Y3kuB04l1s5iff60H3ERnvdonr9dL1NtYAX1rH6weVScjRX+6rAUKlRg08Su?=
 =?us-ascii?Q?S+rBsVd1QigYKbhZ3DfXGO+59RNofwDwFPlPAkhEXpHhppXigKRDAGy0H1kW?=
 =?us-ascii?Q?dnjDwzMHcRr+qVqUWAQ5B0/lINZFxPsr0BkMr+vgAyyunsDiA8pXYk0MSOZ2?=
 =?us-ascii?Q?vi29aZbcjYj3G1bZ30GTlcfKZlamiRwutiIpWcMEBKF/ltmjJ2Wa5Sw8JE+l?=
 =?us-ascii?Q?8GRkyjZrOKhpWFUcM/lDM64hRqo5jM46Aj89KTTpiljKcM0nwj1pihYxrtxh?=
 =?us-ascii?Q?TMlQosszQveMhEEy6TlJR/fHmmoCJvQY8H/LAu01CNsTwPdWTSP8dNf+hQuT?=
 =?us-ascii?Q?fvG9+yhXUOvZjKEBRYqbgp1Gv1lk4QtNw0ul8or9ZWp/Z8meW7UWuzfLctTs?=
 =?us-ascii?Q?6LsP7uu1CJcVw8zcY6rx9TT9/yJUaiULz/C7tYc0M9m7DmoFJG/QHJyVxDki?=
 =?us-ascii?Q?+c5G6Yl7VW0/am9jBpefRAR439Q4DmLXFBnjFJ1i9bUvpmpLYjXiOjyDT8aw?=
 =?us-ascii?Q?WlrkT504dKpC+cIgtkHfhgztMYA2dDR2PWkAOclBPGALQX3jVD+uaWM1sLN/?=
 =?us-ascii?Q?99vSkrsJoT+1xKwcCrGq3YfjqaIV+xjfiQdW3pTkjoksPM6fFf66NwZji4j1?=
 =?us-ascii?Q?iDMHzQ2cXOsHk5Tx9yPWH70LaLw0kZOuYDTlUC8yg9r/BGhk03LaaE5KkXVq?=
 =?us-ascii?Q?HlMHwQRaZ6JaxRVPAl9loM4mJhhr4JENqvdIoMNiUAwEJlpgJOBC9GbJtxAc?=
 =?us-ascii?Q?idClDpV61WA1QF57qfhf9SEnK2V69TL1nkdL2+RrhCI18I0rkIKjHi1RqktD?=
 =?us-ascii?Q?sV2dimnfzuRXbLlEiZomqVDZvd2pFi4vj695JW6CJpacfaonMJUMHdcUxpE1?=
 =?us-ascii?Q?iBi/Mp2lMr7ZgGiMhToHRuAejfVoOksGycdOERgx6yqloBnMmjSIwdq0D5HS?=
 =?us-ascii?Q?maWpKZmn5FKQMgj46DTCOK5xfEAJfGg0fUKUkrVfOSnnKpkiYgmaXZ+LQzxw?=
 =?us-ascii?Q?xm3iztD/1p3UOozblh7k40PjP2Z0AnY134a55DGGaPwQYU7Rbr9n3PRPwJy4?=
 =?us-ascii?Q?Ycf/JpWD2+FYBEH2dmbxH86WXMpVvuNfP03QWboVQxJjb9MHewxGUdwX8yF0?=
 =?us-ascii?Q?eDkI7RGP6/HGTGqPmpQwNT3whh8Tp794uwsITNR5iQe79iOyeYz7lcexGqcF?=
 =?us-ascii?Q?thNhfeRK5t8zSHk0eMZ/XmHyCCy3TFxvKYL5Vrd1gVTgHzxbeHdBoN8qJywa?=
 =?us-ascii?Q?weQChhoynw6cKdWaxBnv1YQBDL3BdUur9tHiPFlpBHQHl5VjSbHCz00y9WG8?=
 =?us-ascii?Q?jUBOSKVoCrpZG3qAvxAjtSwJwrLX3AOVozzQuGc/IHXrbdgKC1vjScBD8KQJ?=
 =?us-ascii?Q?c9HMfQXBIWEDPK9qkwKaVmEQudi2?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:07:25.8838
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b40d0cd0-3024-40b6-df5d-08dc7693d386
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF0000009C.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5921

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

At the moment, we always allocate an extra 16 slots for IO handlers
(see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated
MSI-X registers we need to explicitly tell that we have additional IO
handlers, so those are accounted.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
This depends on a constant defined in ("vpci: add initial support for
virtual PCI bus topology"), so cannot be committed without the
dependency.

Since v5:
- optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is
  defined unconditionally
New in v5
---
 xen/arch/arm/vpci.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 516933bebfb3..4779bbfa9be3 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -132,6 +132,8 @@ static int vpci_get_num_handlers_cb(struct domain *d,
 
 unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
+    unsigned int count;
+
     if ( !has_vpci(d) )
         return 0;
 
@@ -152,7 +154,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
      * For guests each host bridge requires one region to cover the
      * configuration space. At the moment, we only expose a single host bridge.
      */
-    return 1;
+    count = 1;
+
+    /*
+     * There's a single MSI-X MMIO handler that deals with both PBA
+     * and MSI-X tables per each PCI device being passed through.
+     * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV.
+     */
+    if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) )
+        count += VPCI_MAX_VIRT_DEV;
+
+    return count;
 }
 
 /*
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Fri May 17 18:23:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 18:23:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724403.1129675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s82Ee-0007ao-Ao; Fri, 17 May 2024 18:23:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724403.1129675; Fri, 17 May 2024 18:23:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s82Ee-0007ah-7Z; Fri, 17 May 2024 18:23:16 +0000
Received: by outflank-mailman (input) for mailman id 724403;
 Fri, 17 May 2024 18:23:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s82Ed-0007aM-5P; Fri, 17 May 2024 18:23:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s82Ed-0004Ca-3v; Fri, 17 May 2024 18:23:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s82Ec-0004vA-Ov; Fri, 17 May 2024 18:23:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s82Ec-0000eu-OQ; Fri, 17 May 2024 18:23:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=npKKYmXxRgTeLQ8ueKq1PoylY39FGBxLWa8TYDSyLoA=; b=0NAh+t2/BW3bJ3ciIvgKMKFy9C
	HkpUEAvjhjuIjqjbAAg5HpKUgK0BewDWChe3MSLIKmSdh6v8O9AB3IZ4U8MJNr0UopgXtnfu5vuX+
	9V4DwJ7NgB2gDxAbSgGdlHzUk6DPkZO8ptJxXsPCWRQNEvKwB2UUzMsDSiWpoUfUNW6c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186027-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186027: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
X-Osstest-Versions-That:
    xen=ae7584f63678cd9adc1c2f3a1e813b67a6b24544
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 18:23:14 +0000

flight 186027 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186027/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186021
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186021
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186021
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186021
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186021
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186021
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
baseline version:
 xen                  ae7584f63678cd9adc1c2f3a1e813b67a6b24544

Last test of basis   186021  2024-05-16 21:08:45 Z    0 days
Testing same since   186027  2024-05-17 07:07:37 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Edgar E. Iglesias <edgar.iglesias@amd.com>
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ae7584f636..53dc37829c  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e -> master


From xen-devel-bounces@lists.xenproject.org Fri May 17 20:29:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 20:29:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724432.1129697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84CL-0001ie-4w; Fri, 17 May 2024 20:29:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724432.1129697; Fri, 17 May 2024 20:29:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84CL-0001iX-1j; Fri, 17 May 2024 20:29:01 +0000
Received: by outflank-mailman (input) for mailman id 724432;
 Fri, 17 May 2024 20:28:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s84CJ-0001iR-Tk
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 20:28:59 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 14bf1afa-148c-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 22:28:57 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 828E7CE19DF;
 Fri, 17 May 2024 20:28:50 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E81FC2BD10;
 Fri, 17 May 2024 20:28:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14bf1afa-148c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715977729;
	bh=sgyR0ItSUktpZeq8+TJLDrrZo9msa4hhF4TyojkEJ4o=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=CrvSvXGEO2Ptui9B6bf8XRrlVUxPlt3s756t47WRYB/yoQkRw6bAohNoLFg8i9IVr
	 arl6LrswhdvvPfjTtrNFg5YCNhvPsl3Z5YRUFklyYPn1jav5QYlc2tGlCNILlnj07o
	 H2wmB8rLNyVE5XEfFFMyLS1Zi9gaacvXI0AKtucwXBNb+txmmZ9WojqDecouets3bW
	 U97QHdsSoLoVC6StthAQoX3wAXRfvxofqY1tkDVm2jY2CeRskcS66nr4HSGG5c502V
	 C7OrqEWtepAYuYKOtfZszdSe0dbGny+oR3SCuLnINL3o/tXGkWiEu5lkovEiMicffC
	 Xaxw989jhJmbA==
Date: Fri, 17 May 2024 13:28:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <George.Dunlap@citrix.com>, 
    Daniel Smith <dpsmith@apertussolutions.com>, Julien Grall <julien@xen.org>, 
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
In-Reply-To: <e75ca24e-12fe-44ac-9c67-bcc222ac8752@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405171328250.1052252@ubuntu-linux-20-04-desktop>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com> <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com> <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com> <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
 <e75ca24e-12fe-44ac-9c67-bcc222ac8752@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 May 2024, Jan Beulich wrote:
> On 17.05.2024 03:21, Stefano Stabellini wrote:
> > On Thu, 16 May 2024, Jan Beulich wrote:
> >> 1) In the discussion George claimed that exposing status information in
> >> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
> >> how a similar assumption by CPU designers has led to a flood of
> >> vulnerabilities over the last 6+ years. Information exposure imo is never
> >> okay, unless it can be _proven_ that absolutely nothing "useful" can be
> >> inferred from it. (I'm having difficulty seeing how such a proof might
> >> look like.)
> > 
> > Many would agree that it is better not to expose status information in
> > an uncontrolled manner. Anyway, let's focus on the actionable.
> > 
> > 
> >> 2) Me pointing out that the XSM hook might similarly get in the way of
> >> debugging, Andrew suggested that this is not an issue because any sensible
> >> XSM policy used in such an environment would grant sufficient privilege to
> >> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
> >> for Xen-internal event channels. The debugging argument then becomes weak,
> >> as in that case the XSM hook is possibly going to get in the way.
> >>
> >> 3) In the discussion Andrew further gave the impression that evtchn_send()
> >> had no XSM check. Yet it has; the difference to evtchn_status() is that
> >> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
> >> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
> >> similarly useful to allow getting a stuck channel unstuck.)
> >>
> >> In summary I continue to think that an outright revert was inappropriate.
> >> DomU-s should continue to be denied status information on Xen-internal
> >> event channels, unconditionally and independent of whether dummy, silo, or
> >> Flask is in use.
> > 
> > I think DomU-s should continue to be denied status information on
> > Xen-internal event channels *based on the default dummy, silo, or Flask
> > policy*. It is not up to us to decide the security policy, only to
> > enforce it and provide sensible defaults.
> > 
> > In any case, the XSM_TARGET check in evtchn_status seems to do what we
> > want?
> 
> No. XSM_TARGET permits the "owning" (not really, but it's its table) domain
> access. See xsm_default_action() in xsm/dummy.h.

Sorry I still don't understand. Why is that a problem? It looks like the
wanted default behavior?


From xen-devel-bounces@lists.xenproject.org Fri May 17 20:30:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 20:30:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724436.1129706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84E1-0003j7-H0; Fri, 17 May 2024 20:30:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724436.1129706; Fri, 17 May 2024 20:30:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84E1-0003j0-EK; Fri, 17 May 2024 20:30:45 +0000
Received: by outflank-mailman (input) for mailman id 724436;
 Fri, 17 May 2024 20:30:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s84E0-0003is-5z
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 20:30:44 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 52bef38a-148c-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 22:30:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 1B0DDCE1A70;
 Fri, 17 May 2024 20:30:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36F83C2BD10;
 Fri, 17 May 2024 20:30:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52bef38a-148c-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715977837;
	bh=l1bHbBdaAZNckLc8f+mmN8ZM6MqCAhEGHsfsqTqXtx4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=leVSUTgrMH+EnknipHKE5aEspA0xTI37Os52hk3hXRBRZktLMVsBg6EUhWQZnQfRA
	 yXnQ/GR7l4uqPPth2YeL1A/PcCR6Tm+cnfD4TIgRzPzZfcW+CKKUpMuK61ZdOyHeFE
	 vmZ0ik28D0BqsmnphlVQttD3wvIGFISJp1wj1zUyDp06ERuEIyhZgCWPSC/lMAIq9l
	 FdiRgQND3EHkmvWKR74cpZeWRvQFRdTDb+L/dleEzKYPKVnTni4FpqTlWxrjfH38cf
	 ZT9uxfzEKX5YBvYsk/Jk3yBUhW26YZigzdFXhHZ43kQi2m6wFkticjnC1PTAWw6EwC
	 5vyIkqrmxKDWA==
Date: Fri, 17 May 2024 13:30:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [XEN PATCH] automation/eclair_analysis: set MISRA C Rule 10.2
 as clean
In-Reply-To: <7ded6026fc8a02f53bdb841a4e4c5e3be45a15a0.1715930684.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405171330251.1052252@ubuntu-linux-20-04-desktop>
References: <7ded6026fc8a02f53bdb841a4e4c5e3be45a15a0.1715930684.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 May 2024, Nicola Vetrini wrote:
> This rule has no more violations in the codebase, so it can be
> set as clean.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



> ---
>  automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
> index acea15f486a1..b7a9f75b275b 100644
> --- a/automation/eclair_analysis/ECLAIR/tagging.ecl
> +++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
> @@ -19,7 +19,7 @@
>  
>  -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
>  
> --service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
> +-service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R10.2||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
>  }
>  
>  -setq=target,getenv("XEN_TARGET_ARCH")
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri May 17 20:57:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 20:57:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724455.1129717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84e1-0007mY-Hq; Fri, 17 May 2024 20:57:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724455.1129717; Fri, 17 May 2024 20:57:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84e1-0007mR-Eg; Fri, 17 May 2024 20:57:37 +0000
Received: by outflank-mailman (input) for mailman id 724455;
 Fri, 17 May 2024 20:57:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s84dz-0007m5-I0
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 20:57:35 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 152df9e8-1490-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 22:57:34 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B163461DDA;
 Fri, 17 May 2024 20:57:32 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB600C2BD10;
 Fri, 17 May 2024 20:57:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 152df9e8-1490-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715979452;
	bh=sx9LlUQgWX7kme8U3/TEYamKVLPj2vBDROCcWlcAPmc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OK0ehEm2pz0AJJzgSMp9tutPwYYt4VJnbBzkDQLMzP3N5ymENvOMhhR7phUEKQ+OO
	 q6q40p2DMg6DwPk0h8PeTVinOknMtZkeEo3Z4N5izi0kL/cqA78AHxq9rIaQoLlEi1
	 /Wsj+34xLfkPVhMy33FACH8TW8BBfREwww9tdpwPTDbaNfuSId09R5JmhWbadvTK+Z
	 q6lIhjoZIR6zlDxN9CS3ZbdsRbxbt2rbDiELnirDiqwWGQVm3ers4HfvLfPm2A25lZ
	 X40BTKNmz31M4ngTvqcb1Y1uaUQPbYCooZwQx6gYeuz956V50xx/1ZLg/XGUgpwn7t
	 VW+JvzQlEvJZA==
Date: Fri, 17 May 2024 13:57:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 01/12] automation: include domU kernel messages in the
 console output log
In-Reply-To: <eb081e91f8c82d3293d6935c0567df96903d0b6c.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171357190.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <eb081e91f8c82d3293d6935c0567df96903d0b6c.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-242755061-1715979452=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-242755061-1715979452=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  automation/scripts/qubes-x86-64.sh | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index d81ed7b931cf..4beeff17d31b 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -131,6 +131,8 @@ mkdir sys
>  rm var/run
>  echo "#!/bin/sh
>  
> +echo 8 > /proc/sys/kernel/printk
> +
>  ${domU_check}
>  " > etc/local.d/xen.start
>  chmod +x etc/local.d/xen.start
> -- 
> git-series 0.9.1
> 
--8323329-242755061-1715979452=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 20:58:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 20:58:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724456.1129727 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84eW-0008Fa-QO; Fri, 17 May 2024 20:58:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724456.1129727; Fri, 17 May 2024 20:58:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s84eW-0008FT-N2; Fri, 17 May 2024 20:58:08 +0000
Received: by outflank-mailman (input) for mailman id 724456;
 Fri, 17 May 2024 20:58:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s84eV-0008Cs-Bc
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 20:58:07 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2651e691-1490-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 22:58:04 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id BC8C0CE1ACF;
 Fri, 17 May 2024 20:57:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75EBCC2BD10;
 Fri, 17 May 2024 20:57:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2651e691-1490-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715979477;
	bh=V0jJpL7gnRBXfBTZqPkn7Wu//yGS1QTNfkj3Eh7zTR4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=E6CZ1WkrVrrtl0SZ0o5Xs07PEaiIRH438iVZlKTFtdof8nUGFF9klEJKXrCxNdkly
	 TYzSr7JW6AtpKwvAgj+avZCGFuk8tqCnCx8iVhR3Fa5kJsRIpEQFgKA58dY9m68wWX
	 1LUlNDQTv+A5TZycNWcAPgJkZpqF0JPg6MohtIRij2DuTuVJy99AOPwzkD7x3ztI2r
	 ZLZU+sRHlSenqDTh+2LCsR2B6mEifLpGlwi6xh1TmOoEDw73wA/5ZiOVF8fHiBq9wV
	 L3Ag/Q16WdCVQYlBM1jdc6q5hF0B3vh2NB1I19AfnP+1IGGMqSKBFKI3LkGF5DFGhl
	 HoxbMWb3hljWw==
Date: Fri, 17 May 2024 13:57:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 02/12] automation: update fedora build to F39
In-Reply-To: <ae23eeb2e08c2fa95d975bfad64b818870e370e6.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171357490.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <ae23eeb2e08c2fa95d975bfad64b818870e370e6.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1335279025-1715979476=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1335279025-1715979476=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Fedora 29 is long EOL
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  automation/build/fedora/29.dockerfile | 46 +----------------------------
>  automation/build/fedora/39.dockerfile | 46 ++++++++++++++++++++++++++++-
>  automation/gitlab-ci/build.yaml       |  4 +-
>  3 files changed, 48 insertions(+), 48 deletions(-)
>  delete mode 100644 automation/build/fedora/29.dockerfile
>  create mode 100644 automation/build/fedora/39.dockerfile
> 
> diff --git a/automation/build/fedora/29.dockerfile b/automation/build/fedora/29.dockerfile
> deleted file mode 100644
> index f473ae13e7c1..000000000000
> --- a/automation/build/fedora/29.dockerfile
> +++ /dev/null
> @@ -1,46 +0,0 @@
> -FROM --platform=linux/amd64 fedora:29
> -LABEL maintainer.name="The Xen Project" \
> -      maintainer.email="xen-devel@lists.xenproject.org"
> -
> -# install Xen depends
> -RUN dnf -y install \
> -        clang \
> -        gcc \
> -        gcc-c++ \
> -        ncurses-devel \
> -        zlib-devel \
> -        openssl-devel \
> -        python-devel \
> -        python3-devel \
> -        libuuid-devel \
> -        pkgconfig \
> -        flex \
> -        bison \
> -        libaio-devel \
> -        glib2-devel \
> -        yajl-devel \
> -        pixman-devel \
> -        glibc-devel \
> -        make \
> -        binutils \
> -        git \
> -        wget \
> -        acpica-tools \
> -        python-markdown \
> -        patch \
> -        checkpolicy \
> -        dev86 \
> -        xz-devel \
> -        bzip2 \
> -        nasm \
> -        ocaml \
> -        ocaml-findlib \
> -        golang \
> -        # QEMU
> -        ninja-build \
> -    && dnf clean all && \
> -    rm -rf /var/cache/dnf
> -
> -RUN useradd --create-home user
> -USER user
> -WORKDIR /build
> diff --git a/automation/build/fedora/39.dockerfile b/automation/build/fedora/39.dockerfile
> new file mode 100644
> index 000000000000..054f73444060
> --- /dev/null
> +++ b/automation/build/fedora/39.dockerfile
> @@ -0,0 +1,46 @@
> +FROM --platform=linux/amd64 fedora:39
> +LABEL maintainer.name="The Xen Project" \
> +      maintainer.email="xen-devel@lists.xenproject.org"
> +
> +# install Xen depends
> +RUN dnf -y install \
> +        clang \
> +        gcc \
> +        gcc-c++ \
> +        ncurses-devel \
> +        zlib-devel \
> +        openssl-devel \
> +        python-devel \
> +        python3-devel \
> +        libuuid-devel \
> +        pkgconfig \
> +        flex \
> +        bison \
> +        libaio-devel \
> +        glib2-devel \
> +        yajl-devel \
> +        pixman-devel \
> +        glibc-devel \
> +        make \
> +        binutils \
> +        git \
> +        wget \
> +        acpica-tools \
> +        python-markdown \
> +        patch \
> +        checkpolicy \
> +        dev86 \
> +        xz-devel \
> +        bzip2 \
> +        nasm \
> +        ocaml \
> +        ocaml-findlib \
> +        golang \
> +        # QEMU
> +        ninja-build \
> +    && dnf clean all && \
> +    rm -rf /var/cache/dnf
> +
> +RUN useradd --create-home user
> +USER user
> +WORKDIR /build
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 49d6265ad5b4..69665ec5b11f 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -691,12 +691,12 @@ debian-bookworm-32-gcc-debug:
>  fedora-gcc:
>    extends: .gcc-x86-64-build
>    variables:
> -    CONTAINER: fedora:29
> +    CONTAINER: fedora:39
>  
>  fedora-gcc-debug:
>    extends: .gcc-x86-64-build-debug
>    variables:
> -    CONTAINER: fedora:29
> +    CONTAINER: fedora:39
>  
>  # Ubuntu Trusty's Clang is 3.4 while Xen requires 3.5
>  
> -- 
> git-series 0.9.1
> 
--8323329-1335279025-1715979476=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 21:51:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 21:51:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724476.1129736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85Tn-0002d9-G2; Fri, 17 May 2024 21:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724476.1129736; Fri, 17 May 2024 21:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85Tn-0002d2-DW; Fri, 17 May 2024 21:51:07 +0000
Received: by outflank-mailman (input) for mailman id 724476;
 Fri, 17 May 2024 21:51:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s85Tm-0002cv-8r
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 21:51:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8cd35686-1497-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 23:51:01 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2213061E0F;
 Fri, 17 May 2024 21:51:00 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3E973C2BD10;
 Fri, 17 May 2024 21:50:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8cd35686-1497-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715982659;
	bh=DuPDxR+/wnXLaLSayYn+Lc6ZaqKuqoSweToKrjwmE1Q=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hs6ZeIAvOY3S3laj2qvhNPQYTkYxPeqwxFDF19BnR8y17Niqf0wfNCux5XOhM++nu
	 wJGrdBBNw0vPxaSnqueWU8yWOUvd6vNZKK4SYT3JL475v2ySC5BTbckhKQFY7E9qzk
	 7vnN0S2t5UF1QxFXyA76oUvopSPZKwPbNBYt8xKusCPtDk4FbtTVBKpFf2hnV3SkQ6
	 g2YJp007oOPnNDJ0e1eSgPVB5W9OnlCH98UimJbydi0PgbANiSVZmYeojIWmExOiM9
	 n+L43jCJGkHv7urXLVmWbRuTIRNOVAyZryP6cFGnDwv24heerz0QbjVwwToc/XPrg3
	 4f8OJZW+lH6rw==
Date: Fri, 17 May 2024 14:50:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 04/12] automation: increase verbosity of starting a
 domain
In-Reply-To: <cebd305832fab02532fba4b63d2b3c64801753ef.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171450280.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <cebd305832fab02532fba4b63d2b3c64801753ef.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1165086875-1715982659=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1165086875-1715982659=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> And start collecting qemu log earlier, so it isn't lost in case of a
> timeout during domain startup.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/scripts/qemu-alpine-x86_64.sh    | 2 +-
>  automation/scripts/qemu-smoke-dom0-arm32.sh | 2 +-
>  automation/scripts/qemu-smoke-dom0-arm64.sh | 2 +-
>  automation/scripts/qubes-x86-64.sh          | 4 ++--
>  4 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> index 8e398dcea34b..a188d60ea6f3 100755
> --- a/automation/scripts/qemu-alpine-x86_64.sh
> +++ b/automation/scripts/qemu-alpine-x86_64.sh
> @@ -56,7 +56,7 @@ bash /etc/init.d/xencommons start
>  
>  xl list
>  
> -xl create -c /root/test.cfg
> +xl -vvv create -c /root/test.cfg
>  
>  " > etc/local.d/xen.start
>  chmod +x etc/local.d/xen.start
> diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
> index d91648905669..3d208cd55bfa 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm32.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
> @@ -21,7 +21,7 @@ echo "#!/bin/bash
>  
>  xl list
>  
> -xl create -c /root/test.cfg
> +xl -vvv create -c /root/test.cfg
>  
>  " > ./root/xen.start
>  echo "bash /root/xen.start" >> ./etc/init.d/xen-watchdog
> diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
> index e0bb37af3610..afc24074eef8 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm64.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
> @@ -52,7 +52,7 @@ bash /etc/init.d/xencommons start
>  
>  xl list
>  
> -xl create -c /root/test.cfg
> +xl -vvv create -c /root/test.cfg
>  
>  " > etc/local.d/xen.start
>  chmod +x etc/local.d/xen.start
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index 4beeff17d31b..bd620b0d9273 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -112,7 +112,6 @@ echo \"${passed}\"
>  "
>  
>      dom0_check="
> -tail -F /var/log/xen/qemu-dm-domU.log &
>  until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.log; do
>      sleep 1
>  done
> @@ -167,7 +166,8 @@ ifconfig xenbr0 192.168.0.1
>  
>  # get domU console content into test log
>  tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
> -xl create /etc/xen/domU.cfg
> +tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
> +xl -vvv create /etc/xen/domU.cfg
>  ${dom0_check}
>  " > etc/local.d/xen.start
>  chmod +x etc/local.d/xen.start
> -- 
> git-series 0.9.1
> 
--8323329-1165086875-1715982659=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 21:51:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 21:51:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724477.1129748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85UA-00030E-Po; Fri, 17 May 2024 21:51:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724477.1129748; Fri, 17 May 2024 21:51:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85UA-000307-KY; Fri, 17 May 2024 21:51:30 +0000
Received: by outflank-mailman (input) for mailman id 724477;
 Fri, 17 May 2024 21:51:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s85U9-0002cv-4z
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 21:51:29 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9c23de29-1497-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 23:51:28 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 07AC8CE1B7B;
 Fri, 17 May 2024 21:51:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B32E6C2BD10;
 Fri, 17 May 2024 21:51:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c23de29-1497-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715982681;
	bh=nZ60/NuZAh1uGxhlIEgxY5lAvOezDc7KKvn+y8uvDR8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=W46sa7WVXc64xl0/LeZOiQ7TBE71x1KrmjTgoP9jq51vDw4uSoEDoTXF3PcIkoBIn
	 aWJd/jIrOA/KlWxIttIVhs79rGhV8zr9ckzLmPFqQcRrhrRfCfR5DujgQK8+7pETtE
	 ISE7/lIhOMRg20wKm7OXHAvJcxuOjpdC5Hl00IquuXlrHvH/QjyEP3ySm94bwhFZor
	 sLzaEMTjNxokZcHYkY/NmlFvVkYH2BcEyrIacFM3YOw+qROixLBryg0J6CHwgrGPiF
	 hU4huJw0jKXPJmtNEKeAC/kWdxEs2tvCbJgLu/oCfqS4dhOblXGxMXHuFH3f74bXs9
	 o8ZjMQfUGTxKQ==
Date: Fri, 17 May 2024 14:51:18 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 05/12] automation: prevent grub unpacking initramfs
In-Reply-To: <f56b08c94bacb493651f0121660aeaece34f3df7.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171451040.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <f56b08c94bacb493651f0121660aeaece34f3df7.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1651299005-1715982680=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1651299005-1715982680=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> It fails on larger initramfs (~250MB one), let Linux do it.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/scripts/qubes-x86-64.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index bd620b0d9273..77cb0d45815d 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -189,7 +189,7 @@ CONTROLLER=control@thor.testnet
>  echo "
>  multiboot2 (http)/gitlab-ci/xen $CONSOLE_OPTS loglvl=all guest_loglvl=all dom0_mem=4G console_timestamps=boot $extra_xen_opts
>  module2 (http)/gitlab-ci/vmlinuz console=hvc0 root=/dev/ram0 earlyprintk=xen
> -module2 (http)/gitlab-ci/initrd-dom0
> +module2 --nounzip (http)/gitlab-ci/initrd-dom0
>  " > $TFTP/grub.cfg
>  
>  cp -f binaries/xen $TFTP/xen
> -- 
> git-series 0.9.1
> 
--8323329-1651299005-1715982680=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 21:53:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 21:53:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724482.1129756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85Vv-0003hh-1P; Fri, 17 May 2024 21:53:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724482.1129756; Fri, 17 May 2024 21:53:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85Vu-0003ha-V1; Fri, 17 May 2024 21:53:18 +0000
Received: by outflank-mailman (input) for mailman id 724482;
 Fri, 17 May 2024 21:53:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s85Vt-0003hO-Hq
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 21:53:17 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dbfaef92-1497-11ef-909e-e314d9c70b13;
 Fri, 17 May 2024 23:53:15 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id C61A6CE1B82;
 Fri, 17 May 2024 21:53:12 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B8556C2BD10;
 Fri, 17 May 2024 21:53:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbfaef92-1497-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715982792;
	bh=BlHaVUWJ8kAYphWXt7I3+BGdUW3wkmdbXEAlEub1lYo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kah0ZXepH2CeO0cSNAAA6Di4MgKbAjgp1QBqsewrMYCJxVqgvz+XJDw2vABd2xjMU
	 F2ug00a/GKEt8n0Ymzvs5wfQ2ZaQLe4BUJuXcbaR24P0EOPRFfmNRgJwwFvJp+eYRp
	 EWs/cT00gtuvpj5nrya70+98DEKJqoGVFkyiD5IejPlN5PDrAudtU1UmHvLnwNUMkH
	 XJHF+ipl8WVBMwSCj193xeN2T+dk/S9JXbUISRrxujGYr9n5mZdQVxYTz1wJOLw9ZC
	 qX6veD1Q+l1WCWzSdMbYRwgVrsVd0j56Poz9vGe43PWpnftosiah8yRa9vDvnpTqd+
	 LHKHa55vARQPQ==
Date: Fri, 17 May 2024 14:53:08 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 08/12] automation: update kernel for x86 tests
In-Reply-To: <2e6dc425056db1267768fd1bd0a20c444b1f1610.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171452570.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <2e6dc425056db1267768fd1bd0a20c444b1f1610.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1717371359-1715982790=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1717371359-1715982790=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Update 6.1.x kernel to the latest version in this branch. This is
> especially needed to include MSI-X related fixes for stubdomain
> ("xen-pciback: Consider INTx disabled when MSI/MSI-X is enabled").
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/gitlab-ci/build.yaml                     |  4 +-
>  automation/gitlab-ci/test.yaml                      |  2 +-
>  automation/tests-artifacts/kernel/6.1.19.dockerfile | 40 +--------------
>  automation/tests-artifacts/kernel/6.1.90.dockerfile | 40 ++++++++++++++-
>  4 files changed, 43 insertions(+), 43 deletions(-)
>  delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
>  create mode 100644 automation/tests-artifacts/kernel/6.1.90.dockerfile
> 
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 783a0687ba34..9b9e5464f179 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -331,9 +331,9 @@ alpine-3.19-rootfs-export:
>    tags:
>      - x86_64
>  
> -kernel-6.1.19-export:
> +kernel-6.1.90-export:
>    extends: .test-jobs-artifact-common
> -  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
> +  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.90
>    script:
>      - mkdir binaries && cp /bzImage binaries/bzImage
>    artifacts:
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 80d10eb7f476..e3910f4c1a9f 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -12,7 +12,7 @@
>  
>  .x86-64-test-needs: &x86-64-test-needs
>    - alpine-3.19-rootfs-export
> -  - kernel-6.1.19-export
> +  - kernel-6.1.90-export
>  
>  .qemu-arm64:
>    extends: .test-jobs-common
> diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile
> deleted file mode 100644
> index 3a4096780d20..000000000000
> --- a/automation/tests-artifacts/kernel/6.1.19.dockerfile
> +++ /dev/null
> @@ -1,40 +0,0 @@
> -FROM --platform=linux/amd64 debian:bookworm
> -LABEL maintainer.name="The Xen Project" \
> -      maintainer.email="xen-devel@lists.xenproject.org"
> -
> -ENV DEBIAN_FRONTEND=noninteractive
> -ENV LINUX_VERSION=6.1.19
> -ENV USER root
> -
> -RUN mkdir /build
> -WORKDIR /build
> -
> -# build depends
> -RUN apt-get update && \
> -    apt-get --quiet --yes install \
> -        build-essential \
> -        libssl-dev \
> -        bc \
> -        curl \
> -        flex \
> -        bison \
> -        libelf-dev \
> -        && \
> -    apt-get autoremove -y && \
> -    apt-get clean && \
> -    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> -
> -# Build the kernel
> -RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
> -    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> -    cd linux-"$LINUX_VERSION" && \
> -    make defconfig && \
> -    make xen.config && \
> -    scripts/config --enable BRIDGE && \
> -    scripts/config --enable IGC && \
> -    cp .config .config.orig && \
> -    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
> -    make -j$(nproc) bzImage && \
> -    cp arch/x86/boot/bzImage / && \
> -    cd /build && \
> -    rm -rf linux-"$LINUX_VERSION"*
> diff --git a/automation/tests-artifacts/kernel/6.1.90.dockerfile b/automation/tests-artifacts/kernel/6.1.90.dockerfile
> new file mode 100644
> index 000000000000..46cadf02ca78
> --- /dev/null
> +++ b/automation/tests-artifacts/kernel/6.1.90.dockerfile
> @@ -0,0 +1,40 @@
> +FROM --platform=linux/amd64 debian:bookworm
> +LABEL maintainer.name="The Xen Project" \
> +      maintainer.email="xen-devel@lists.xenproject.org"
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV LINUX_VERSION=6.1.90
> +ENV USER root
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +# build depends
> +RUN apt-get update && \
> +    apt-get --quiet --yes install \
> +        build-essential \
> +        libssl-dev \
> +        bc \
> +        curl \
> +        flex \
> +        bison \
> +        libelf-dev \
> +        && \
> +    apt-get autoremove -y && \
> +    apt-get clean && \
> +    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> +
> +# Build the kernel
> +RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> +    cd linux-"$LINUX_VERSION" && \
> +    make defconfig && \
> +    make xen.config && \
> +    scripts/config --enable BRIDGE && \
> +    scripts/config --enable IGC && \
> +    cp .config .config.orig && \
> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
> +    make -j$(nproc) bzImage && \
> +    cp arch/x86/boot/bzImage / && \
> +    cd /build && \
> +    rm -rf linux-"$LINUX_VERSION"*
> -- 
> git-series 0.9.1
> 
--8323329-1717371359-1715982790=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 21:58:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 21:58:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724489.1129766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85b4-0004MJ-OT; Fri, 17 May 2024 21:58:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724489.1129766; Fri, 17 May 2024 21:58:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s85b4-0004MC-Ln; Fri, 17 May 2024 21:58:38 +0000
Received: by outflank-mailman (input) for mailman id 724489;
 Fri, 17 May 2024 21:58:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ijk8=MU=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s85b2-0004M6-Mp
 for xen-devel@lists.xenproject.org; Fri, 17 May 2024 21:58:36 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 929e9a9a-1498-11ef-b4bb-af5377834399;
 Fri, 17 May 2024 23:58:22 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0D654CE1B67;
 Fri, 17 May 2024 21:58:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B89B9C2BD10;
 Fri, 17 May 2024 21:58:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 929e9a9a-1498-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715983096;
	bh=rUqFai1gUXokCmftMmeJg2QnclhtWE9JTxxkYCtBU6U=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UWA3HEzZi7lgbMMw1rLzE/vtbNbWNJkl4lwnmOM+PPsb2iooxjWGzNFCHuSChfvYW
	 1A+W13GiDbEkce8mmYZU0lxHrVGNAenGB8U42PeNNXb8Ad6cn16JgC3JyEm08YZ0ld
	 OZEqx73vmeelUJtZFR6tZ/cnSkFGZ1A3YLskoFiUMRZz2h5QufLoFijl0mOg9RbR38
	 7EIt25/Ql4Ks6Z96hYqmRWaKXGDVdPxyNgNB7S9o8tBh7Ba8KVR+71BwF74dgZ/lmG
	 1xQzttV/gnpNPgyZ4pK+tSEC3gi60OWoeudkY2XQX2ObaKJLEOrrZLfccloa0HKXT8
	 NJFIGscVIB1hQ==
Date: Fri, 17 May 2024 14:58:13 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 03/12] automation: switch to alpine:3.19
In-Reply-To: <a681e1d414c0735128cdde70d6b521d8c96e6e45.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171458040.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <a681e1d414c0735128cdde70d6b521d8c96e6e45.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-529455756-1715983095=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-529455756-1715983095=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Alpine 3.19 is needed for upcoming stubdomain tests, as linux stubdomain
> build requires dracut-core package (dracut-install tool specifically)
> which isn't available in 3.18. While technically it will be needed only
> in the x86_64 builds, switch Alpine version everywhere for uniformity.
> Note this bumps kernel version requirement on docker runners -
> dracut-install uses faccessat2() syscall which was introduced in Linux
> 5.8.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

--8323329-529455756-1715983095=:1052252--


From xen-devel-bounces@lists.xenproject.org Fri May 17 22:27:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 May 2024 22:27:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724496.1129776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s862r-0002Dg-Rp; Fri, 17 May 2024 22:27:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724496.1129776; Fri, 17 May 2024 22:27:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s862r-0002DZ-Ox; Fri, 17 May 2024 22:27:21 +0000
Received: by outflank-mailman (input) for mailman id 724496;
 Fri, 17 May 2024 22:27:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s862r-0002DP-6A; Fri, 17 May 2024 22:27:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s862r-0008LK-0F; Fri, 17 May 2024 22:27:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s862q-0006Bw-Oc; Fri, 17 May 2024 22:27:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s862q-0002lX-OA; Fri, 17 May 2024 22:27:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cVdy2naBceOuUp3It3wTtxCPCF/XljMuCp20O5kEwRY=; b=fsZtqjARwuWSROiPhYapS4JebD
	qvjrN/2zqVrhCy32DYNo/TjaBYgIS3YycYdX84FgNk3TGPooxScPaai2anqUWeBm5RTp/fhJZRfma
	9xFDvbc0iQOMV58lLC3lbTRHhqKOvWT8fighFOVWcmr3uCdmvKKVOJhe2MgDhnZ6AbMs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186028-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 186028: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4078fa637fcd80c8487680ec2e4ef7c58308e9aa
X-Osstest-Versions-That:
    linux=909ba1f1b4146de529469910c1bd0b1248964536
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 May 2024 22:27:20 +0000

flight 186028 linux-6.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186028/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185901
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185901
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185901
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185901
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185901
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185901
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4078fa637fcd80c8487680ec2e4ef7c58308e9aa
baseline version:
 linux                909ba1f1b4146de529469910c1bd0b1248964536

Last test of basis   185901  2024-05-02 15:12:21 Z   15 days
Testing same since   186028  2024-05-17 10:16:47 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Goldman <adamg@pobox.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alan Stern <stern@rowland.harvard.edu>
  Aleksa Savic <savicaleksa83@gmail.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Potapenko <glider@google.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Usyskin <alexander.usyskin@intel.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Allen Pais <apais@linux.microsoft.com>
  Aman Dhoot <amandhoot12@gmail.com>
  Amit Sunil Dhamne <amitsd@google.com>
  Anand Jain <anand.jain@oracle.com>
  Andrea Righi <andrea.righi@canonical.com>
  Andrea Righi <andrea.righi@canonical.com> # non-hostprogs
  Andreas Gruenbacher <agruenba@redhat.com>
  Andrei Matei <andreimatei1@gmail.com>
  Andrew Donnellan <ajd@linux.ibm.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrew Price <anprice@redhat.com>
  Andrey Ryabinin <ryabinin.a.a@gmail.com>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anton Protopopov <aspsk@isovalent.com>
  Arjan van de Ven <arjan@linux.intel.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Asahi Lina <lina@asahilina.net>
  Asbjørn Sloth Tønnesen <ast@fiberby.net>
  Aurabindo Pillai <aurabindo.pillai@amd.com>
  Badhri Jagan Sridharan <badhri@google.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Benno Lossin <benno.lossin@proton.me>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Boris Burkov <boris@bur.io>
  Borislav Petkov (AMD) <bp@alien8.de>
  Boy Wu <boy.wu@mediatek.com>
  Boy.Wu <boy.wu@mediatek.com>
  Bui Quang Minh <minhquangbui99@gmail.com>
  Bumyong Lee <bumyong.lee@samsung.com>
  Chad Wagner <wagnerch42@gmail.com>
  Chen Ni <nichen@iscas.ac.cn>
  Chen-Yu Tsai <wenst@chromium.org>
  Chris Wulff <chris.wulff@biamp.com>
  Christian A. Ehrhardt <lk@c--e.de>
  Christian König <christian.koenig@amd.com>
  Christoph Paasch <cpaasch@apple.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Conor Dooley <conor.dooley@microchip.com>
  Damien Le Moal <dlemoal@kernel.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Okazaki <dtokazaki@google.com>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Daniel Xu <dxu@dxuuu.xyz>
  David Bauer <mail@david-bauer.net>
  David Lechner <dlechner@baylibre.com>
  David Rientjes <rientjes@google.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Devyn Liu <liudingyuan@huawei.com>
  Dmitry Antipov <dmantipov@yandex.ru>
  Dominique Martinet <dominique.martinet@atmark-techno.com>
  Doug Berger <opendmb@gmail.com>
  Doug Smythies <dsmythies@telus.net>
  Douglas Anderson <dianders@chromium.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Eric Curtin <ecurtin@redhat.com>
  Eric Dumazet <edumazet@google.com>
  Eric Van Hensbergen <ericvh@kernel.org>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <f.fainelli@gmail.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gabe Teeger <gabe.teeger@amd.com>
  Gaurav Batra <gbatra@linux.ibm.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  George Shen <george.shen@amd.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guillaume Nault <gnault@redhat.com>
  Hans de Goede <hdegoede@redhat.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiner Kallweit <hkallweit1@gmail.com>
  Igor Artemiev <Igor.A.Artemiev@mcst.ru>
  Ivan Babrou <ivan@cloudflare.com>
  Jakub Kicinski <kuba@kernel.org>
  James Chapman <jchapman@katalix.com>
  Jan Dakinevich <jan.dakinevich@salutedevices.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jason Xing <kernelxing@tencent.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jeff Johnson <quic_jjohnson@quicinc.com>
  Jeff Layton <jlayton@kernel.org>
  Jens Axboe <axboe@kernel.dk>
  Jens Remus <jremus@linux.ibm.com>
  Jeremy Bongio <jbongio@google.com>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jerome Brunet <jbrunet@baylibre.com>
  Jesper Dangaard Brouer <hawk@kernel.org>
  Jian Shen <shenjian15@huawei.com>
  Jiaxun Yang <jiaxun.yang@flygoat.com>
  Jijie Shao <shaojijie@huawei.com>
  Jim Cromie <jim.cromie@gmail.com>
  Joakim Sindholt <opensource@zhasha.com>
  Joao Paulo Goncalves <joao.goncalves@toradex.com>
  Joerg Roedel <jroedel@suse.de>
  Johan Hovold <johan+linaro@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  John Stultz <jstultz@google.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Josef Bacik <josef@toxicpanda.com>
  Julian Taylor <julian.taylor@1und1.de>
  Justin Ernst <justin.ernst@hpe.com>
  Justin Tee <justin.tee@broadcom.com>
  Karthikeyan Ramasubramanian <kramasub@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Keith Busch <kbusch@kernel.org>
  Kent Gibson <warthog618@gmail.com>
  kernelci.org bot <bot@kernelci.org>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Lakshmi Yadlapati <lakshmiy@us.ibm.com>
  Leah Rumancik <leah.rumancik@gmail.com>
  Len Brown <len.brown@intel.com>
  Li Nan <linan122@huawei.com>
  Li Zetao <lizetao1@huawei.com>
  Lijo Lazar <lijo.lazar@amd.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
  Lukasz Majewski <lukma@denx.de>
  Lyude Paul <lyude@redhat.com>
  Maarten Vanraes <maarten@rmail.be>
  Mans Rullgard <mans@mansr.com>
  Marc Zyngier <maz@kernel.org>
  Marek Behún <kabel@kernel.org>
  Marek Vasut <marex@denx.de>
  Mario Limonciello <mario.limonciello@amd.com>
  Marius Zachmann <mail@mariuszachmann.de>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <martin.lau@kernel.org>
  Martin Rodriguez Reboredo <yakoyoku@gmail.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Maurizio Lombardi <mlombard@redhat.com>
  Maxime Ripard <mripard@kernel.org>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Kelley <mhklinux@outlook.com>
  Miguel Ojeda <ojeda@kernel.org>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Mike Kravetz <mike.kravetz@oracle.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Nageswara R Sastry <rnsastry@linux.ibm.com>
  Namhyung Kim <namhyung@kernel.org>
  Namjae Jeon <linkinjeon@kernel.org>
  Nayna Jain <nayna@linux.ibm.com>
  Neal Cardwell <ncardwell@google.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-QRD
  Nikhil Rao <nikhil.rao@intel.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Nilesh Javali <njavali@marvell.com>
  Oliver Upton <oliver.upton@linux.dev>
  Oscar Salvador <osalvador@suse.de>
  Paolo Abeni <pabeni@redhat.com>
  Paul Davey <paul.davey@alliedtelesis.co.nz>
  Paul Menzel <pmenzel@molgen.mpg.de> # Dell XPS 13
  Pavel Machek (CIP) <pavel@denx.de>
  Peiyang Wang <wangpeiyang1@huawei.com>
  Peng Liu <liupeng17@lenovo.com>
  Peter Korsgaard <peter@korsgaard.com>
  Peter Ujfalusi <peter.ujfalusi@gmail.com>
  Peter Wang <peter.wang@mediatek.com>
  Peter Xu <peterx@redhat.com>
  Phil Elwell <phil@raspberrypi.com>
  Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
  Qu Wenruo <wqu@suse.com>
  Ramona Gradinariu <ramona.bolboaca13@gmail.com>
  Raphael Nestler <raphael.nestler@gmail.com> # non-hostprogs
  Richard Gobert <richardbgobert@gmail.com>
  Rick Edgecombe <rick.p.edgecombe@intel.com>
  Rik van Riel <riel@surriel.com>
  Roded Zats <rzats@paloaltonetworks.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Ron Economos <re@w6rz.net>
  Ronald Wahl <ronald.wahl@raritan.com> # KS8851 SPI
  Russell Currey <ruscur@russell.cc>
  Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
  Rémi Denis-Courmont <courmisch@gmail.com>
  Salvatore Bonaccorso <carnil@debian.org>
  Sameer Pujar <spujar@nvidia.com>
  Sasha Levin <sashal@kernel.org>
  Saurav Kashyap <skashyap@marvell.com>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  SeongJae Park <sj@kernel.org>
  Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Shyam Prasad N <sprasad@microsoft.com>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Silvio Gissi <sifonsec@amazon.com>
  Song Liu <song@kernel.org>
  Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
  Stanislav Fomichev <sdf@google.com>
  Steffen Klassert <steffen.klassert@secunet.com>
  Stephen Boyd <sboyd@kernel.org>
  Steve French <stfrench@microsoft.com>
  Sungwoo Kim <iam@sung-woo.kim>
  Takashi Iwai <tiwai@suse.de>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tejun Heo <tj@kernel.org>
  Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
  Thanassis Avgerinos <thanassis.avgerinos@gmail.com>
  Thierry Reding <treding@nvidia.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Bertschinger <tahbertschinger@gmail.com>
  Thomas Bogendoerfer <tsbogend@alpha.franken.de>
  Thomas Gleixner <tglx@linutronix.de>
  Tim Jiang <quic_tjiang@quicinc.com>
  Toke Høiland-Jørgensen <toke@redhat.com>
  Tomas Winkler <tomas.winkler@intel.com>
  Tony Luck <tony.luck@intel.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vanillan Wang <vanillanwang@163.com>
  Viken Dadhaniya <quic_vdadhani@quicinc.com>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Vinod Koul <vkoul@kernel.org>
  Vlastimil Babka <vbabka@suse.cz>
  Wedson Almeida Filho <walmeida@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  Wei Yang <richard.weiyang@gmail.com>
  Wen Gu <guwen@linux.alibaba.com>
  Wyes Karny <wyes.karny@amd.com>
  Xin Long <lucien.xin@gmail.com>
  Xu Kuohai <xukuohai@huawei.com>
  Yang Yingliang <yangyingliang@huawei.com>
  Yann Sionneau <ysionneau@kalrayinc.com>
  Yann Sionneau<ysionneau@kalrayinc.com>
  Yi Zhang <yi.zhang@redhat.com>
  Yonglong Liu <liuyonglong@huawei.com>
  Zack Rusin <zack.rusin@broadcom.com>
  Zeng Heng <zengheng4@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   909ba1f1b414..4078fa637fcd  4078fa637fcd80c8487680ec2e4ef7c58308e9aa -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Sat May 18 00:41:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 00:41:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724532.1129787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s888D-0000vo-3e; Sat, 18 May 2024 00:41:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724532.1129787; Sat, 18 May 2024 00:41:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s888D-0000vh-11; Sat, 18 May 2024 00:41:01 +0000
Received: by outflank-mailman (input) for mailman id 724532;
 Sat, 18 May 2024 00:40:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I+vq=MV=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s888B-0000vb-9K
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 00:40:59 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 49218828-14af-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 02:40:56 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id A163D61E1E;
 Sat, 18 May 2024 00:40:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D36BC2BD10;
 Sat, 18 May 2024 00:40:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49218828-14af-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715992854;
	bh=Qkdx1sDq5eH4FJtNtg1vyUmnEccgj9muqvELlebJdiQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UjUeuYXaSYVg4mqdNIQrApjBsFxHWzLhzgQ/mqyBfYoTZlgp/mNpeuo0p86eP67yg
	 tcJT+jQt0DkJzpf/PAHRO8SaZot+6Lfh8800MIGhXrUel4j/DKIbnwHN326QrOrLXi
	 /iw70/+1gk3R4P+85hgV9e5N7NDJTu40QTrySN1Mlq513S6zTg9WJJOwpO1Rvt+Q31
	 EBxWhhmKeRPDHxFTnG/3M3h8BzHyG43+sb1O/zU3TB2ID/vOSNIIzNc68L46/SXGRS
	 fNgelOz55aoJBh7EbJRXcWqbUBBB6sme5yGRVeUWhWLu1vsZQir2yE4vg74dVYkDUY
	 GF5CBzEj+WfRw==
Date: Fri, 17 May 2024 17:40:52 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 06/12] RFC: automation: Add linux stubdom build and smoke
 test
In-Reply-To: <c11feff249928f7e06a91c9b60633e1f74bbd6fd.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171718580.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <c11feff249928f7e06a91c9b60633e1f74bbd6fd.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1214147340-1715992830=:1052252"
Content-ID: <alpine.DEB.2.22.394.2405171740320.1052252@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1214147340-1715992830=:1052252
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2405171740321.1052252@ubuntu-linux-20-04-desktop>

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Add minimal linux-stubdom smoke test. It starts a simple HVM with
> linux-stubdom. The actual stubdom implementation is taken from Qubes OS
> and then stripped off Qubes-specific code. In particular, the remaining
> code does _not_ support:
>  - direct kernel boot (implemented by relaying on specific guest disk
>    laying in Qubes OS)
>  - graphical console (used Qubes GUI agent injected into
>    stubdomain's qemu)
>  - audio input/output (used Qubes audio agent inside stubdomain)
>  - USB passthrough (used qrexec <-> usbip proxy inside stubdomain)
>  - setting up DHCP server (assumes guest addressing used in Qubes OS)
> 
> For this smoke test, the relevant part is missing direct kernel boot, as
> that's used in other smoke tests. Solve this by preparing disk image
> with proper bootloader (grub) installed. Since the test script is
> running on arm64 to control x86_64 box, it cannot (easily) install grub
> directly. For this reason, prepare bootsector as part of the Xen build
> (which runs on x86_64) and then prepend do the disk image during the
> test (and adjust partitions table afterwards).

I am not an expert on this, but do you think it would be possible to use
network boot and tftp instead of grub on emulated disk? That would not
require us to build neither /grub-core.img nor build_domU_disk().

I am trying to avoid grub-core.img and disk.img because I think direct
kernel boot or network boot are easier to maintain and more similar to
the other tests. If you see the ARM tests, they all use tftp boot.


> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> The test is implemented using hardware runner, because some of the
> further tests will require it (for example PCI passthrough with
> stubdomain). But if there is strong desire to have stubdomain tested
> inside qemu tests (to be included in patchew runs), it is probably an
> option for this basic smoke test.

Thanks for this amazing work. This is a great start, we can see how to
create more tests after merging this one.


> For now I'm keeping stubdomain code (build and glue scripts) in separate
> repository on my github account. This is far from ideal. What would be
> preferred option? New repository on xenbits? Or add directly into
> xen.git (stubdom directory)? Honestly, I'd rather avoid the latter, as
> from packager point of view those are mostly separate beings (similar to
> qemu, where many use distribution-provide one instead of the one bundled
> with Xen) and it's convenient to not need to rebuild stubdomain on every
> hypervisor change (like a security patch).

My suggestion is to create repositories under gitlab.com/xen-project


> Another topic is QEMU version inside stubdomain. It needs to be a
> separate build due to vastly different configure options, so I cannot
> reuse the qemu binary built for dom0 (or distribution-provided one if
> Xen is configured to use it). But also, at this moment qemu for
> stubdomain needs few extra patches that are not upstream yet.
> What should be the proper solution here (after upstreaming all the
> patches)?

It is fine to use a different QEMU version. For now, I would suggest to
keep the QEMU patches as patches to be applied, in the new repositories
under gitlab.com/xen-project


> Generally, I try to add tests early, even though there is still some
> work to do for proper stubdomain integration into upstream Xen, so any
> cleanups and future changes (like the CDROM libxl patches by Jason
> Andryuk) can be made with more confidence and reduce risk of
> regressions.
> 
> The patch is RFC only because of the stubdom repository location.
> ---
>  automation/build/alpine/3.19-arm64v8.dockerfile   |  2 +-
>  automation/build/alpine/3.19.dockerfile           |  9 ++-
>  automation/gitlab-ci/build.yaml                   |  3 +-
>  automation/gitlab-ci/test.yaml                    |  8 +-
>  automation/scripts/build                          | 12 ++-
>  automation/scripts/qubes-x86-64.sh                | 87 +++++++++++++++-
>  automation/tests-artifacts/alpine/3.19.dockerfile |  6 +-
>  7 files changed, 123 insertions(+), 4 deletions(-)
> 
> diff --git a/automation/build/alpine/3.19-arm64v8.dockerfile b/automation/build/alpine/3.19-arm64v8.dockerfile
> index 158cf465a9ff..12810f87ecc6 100644
> --- a/automation/build/alpine/3.19-arm64v8.dockerfile
> +++ b/automation/build/alpine/3.19-arm64v8.dockerfile
> @@ -47,3 +47,5 @@ RUN apk --no-cache add \
>    # qubes test deps
>    openssh-client \
>    fakeroot \
> +  sfdisk \
> +  e2fsprogs \
> diff --git a/automation/build/alpine/3.19.dockerfile b/automation/build/alpine/3.19.dockerfile
> index 0be6d7c85fe7..108284613987 100644
> --- a/automation/build/alpine/3.19.dockerfile
> +++ b/automation/build/alpine/3.19.dockerfile
> @@ -49,3 +49,12 @@ RUN apk --no-cache add \
>    pixman-dev \
>    # livepatch-tools deps
>    elfutils-dev \
> +  # stubdom deps
> +  dracut-core \
> +  quilt \
> +  gnupg \
> +  libseccomp-dev \
> +  glib-static \
> +  gmp-dev \
> +  mpc1-dev \
> +  mpfr-dev \
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index b186289bbd82..783a0687ba34 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -323,9 +323,11 @@ alpine-3.19-rootfs-export:
>    image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.19
>    script:
>      - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
> +    - cp /grub-core.img binaries/grub-core.img
>    artifacts:
>      paths:
>        - binaries/initrd.tar.gz
> +      - binaries/grub-core.img
>    tags:
>      - x86_64
>  
> @@ -353,6 +355,7 @@ alpine-3.19-gcc-debug:
>    extends: .gcc-x86-64-build-debug
>    variables:
>      CONTAINER: alpine:3.19
> +    STUBDOM_LINUX: y
>  
>  debian-stretch-gcc-debug:
>    extends: .gcc-x86-64-build-debug
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index f62d426a8d34..80d10eb7f476 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -199,6 +199,14 @@ adl-pci-hvm-x86-64-gcc-debug:
>      - *x86-64-test-needs
>      - alpine-3.19-gcc-debug
>  
> +adl-stubdom-hvm-x86-64-gcc-debug:
> +  extends: .adl-x86-64
> +  script:
> +    - ./automation/scripts/qubes-x86-64.sh stubdom-hvm 2>&1 | tee ${LOGFILE}
> +  needs:
> +    - *x86-64-test-needs
> +    - alpine-3.19-gcc-debug
> +
>  zen3p-smoke-x86-64-gcc-debug:
>    extends: .zen3p-x86-64
>    script:
> diff --git a/automation/scripts/build b/automation/scripts/build
> index b3c71fb6fb60..f7c85b07a8d0 100755
> --- a/automation/scripts/build
> +++ b/automation/scripts/build
> @@ -102,4 +102,16 @@ else
>      # build Xen
>      cp -r dist binaries/
>      if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fi
> +
> +    if [[ "${STUBDOM_LINUX}" == "y" ]]; then
> +        git clone https://github.com/marmarek/qubes-vmm-xen-stubdom-linux -b for-upstream2
> +        make -j$(nproc) -C qubes-vmm-xen-stubdom-linux get-sources
> +        CFLAGS="-I${PWD}/dist/install/usr/local/include" \
> +            LDFLAGS="-L${PWD}/dist/install/usr/local/lib" \
> +            LD_LIBRARY_PATH="${PWD}/dist/install/usr/local/lib" \
> +            PATH="${PWD}/dist/install/usr/local/bin:$PATH" \
> +            make -j$(nproc) -C qubes-vmm-xen-stubdom-linux -f Makefile.stubdom all
> +        cp qubes-vmm-xen-stubdom-linux/build/linux/arch/x86/boot/bzImage binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-kernel
> +        cp qubes-vmm-xen-stubdom-linux/build/rootfs/stubdom-linux-rootfs binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-rootfs
> +    fi

I think qemu-stubdom-linux-kernel and qemu-stubdom-linux-rootfs should
be built as test-artifacts if they are not strictly dependent on the Xen
version.

Otherwise, please create a separate new build job for linux stubdoms (do
not reuse alpine-3.19-gcc-debug). Also I would prefer if these build
instructions were on a separate file, separate from "build".


>  fi
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index 77cb0d45815d..fc73403dbadf 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -18,6 +18,37 @@ memory = 512
>  vif = [ "bridge=xenbr0", ]
>  disk = [ ]
>  '
> +domU_disk_path=
> +
> +### helper functions
> +
> +build_domU_disk() {
> +    local kernel="$1"
> +    local initrd="$2"
> +    local rootfs="$3"
> +    local output="$4"
> +    local grubcfg="$rootfs/boot/grub2/grub.cfg"
> +    local kernel_cmdline="root=/dev/xvda1 console=hvc0 earlyprintk=xen"
> +
> +    mkdir -p "$rootfs/boot/grub2"
> +    cp "$kernel" "$rootfs/boot/vmlinuz"
> +    echo "linux /boot/vmlinuz $kernel_cmdline" >> "$grubcfg"
> +    if [ -n "$initrd" ]; then
> +        cp "$initrd" "$rootfs/boot/initrd.img"
> +        echo "initrd /boot/initrd.img" >> "$grubcfg"
> +    fi
> +    echo "boot" >> "$grubcfg"
> +    size=$(du -sm "$rootfs")
> +    size=${size%%	*}
> +    # add 5M margin
> +    size=$(( size + 5 ))
> +    mke2fs -d "$rootfs" "$output.part1" ${size}m
> +    cat "$rootfs/usr/lib/grub/i386-pc/boot_hybrid.img" binaries/grub-core.img > "$output"
> +    # align for the partition 1 start (2048 sectors)
> +    truncate -s $((2048 * 512)) "$output"
> +    cat "$output.part1" >> "$output"
> +    echo ",,linux,*" | sfdisk "$output"
> +}
>  
>  ### test: smoke test & smoke test PVH
>  if [ -z "${test_variant}" ] || [ "${test_variant}" = "dom0pvh" ]; then
> @@ -116,6 +147,41 @@ until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.
>      sleep 1
>  done
>  "
> +
> +### test: stubdom-hvm
> +elif [ "${test_variant}" = "stubdom-hvm" ]; then
> +    passed="ping test passed"
> +
> +    domU_config='
> +type = "hvm"
> +name = "domU"
> +memory = 512
> +vif = [ "bridge=xenbr0", ]
> +disk = [ "/srv/disk.img,format=raw,vdev=xvda" ]
> +device_model_version = "qemu-xen"
> +device_model_stubdomain_override = 1
> +on_reboot = "destroy"
> +# libxl configures vkb backend to be dom0 instead of the stubdomain, defer
> +# changing that until there is consensus what to do about VGA output (VNC)
> +vkb_device = 0
> +'
> +    domU_check="
> +ifconfig eth0 192.168.0.2
> +until ping -c 10 192.168.0.1; do
> +    sleep 1
> +done
> +echo \"${passed}\"
> +"
> +    dom0_check="
> +set +x
> +until grep -q \"${passed}\" /var/log/xen/console/guest-domU.log; do
> +    sleep 1
> +done
> +set -x
> +echo \"${passed}\"
> +"
> +
> +    domU_disk_path=/srv/disk.img
>  fi
>  
>  # DomU
> @@ -137,8 +203,17 @@ ${domU_check}
>  chmod +x etc/local.d/xen.start
>  echo "rc_verbose=yes" >> etc/rc.conf
>  sed -i -e 's/^Welcome/domU \0/' etc/issue
> -find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
>  cd ..
> +if [ -n "$domU_disk_path" ]; then
> +    build_domU_disk \
> +        "binaries/bzImage" \
> +        "" \
> +        "rootfs" \
> +        "binaries/disk.img"
> +else
> +    (cd rootfs; find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz)
> +fi
> +
>  rm -rf rootfs
>  
>  # DOM0 rootfs
> @@ -152,6 +227,9 @@ mkdir srv
>  mkdir sys
>  rm var/run
>  cp -ar ../binaries/dist/install/* .
> +if [ -n "$domU_disk_path" ]; then
> +    cp ../binaries/disk.img "./$domU_disk_path"
> +fi
>  
>  echo "#!/bin/bash
>  
> @@ -164,8 +242,9 @@ ifconfig eth0 up
>  ifconfig xenbr0 up
>  ifconfig xenbr0 192.168.0.1
>  
> -# get domU console content into test log
> +# get domU (and possibly its stubdom) console content into test log
>  tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
> +tail -F /var/log/xen/console/guest-domU-dm.log 2>/dev/null | sed -e \"s/^/(domU-dm) /\" &
>  tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
>  xl -vvv create /etc/xen/domU.cfg
>  ${dom0_check}
> @@ -178,7 +257,9 @@ echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
>  echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
>  mkdir -p var/log/xen/console
>  cp ../binaries/bzImage boot/vmlinuz
> -cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
> +if [ -r ../binaries/domU-rootfs.cpio.gz ]; then
> +    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
> +fi
>  find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
>  cd ..
>  
> diff --git a/automation/tests-artifacts/alpine/3.19.dockerfile b/automation/tests-artifacts/alpine/3.19.dockerfile
> index 6d665daedfa4..cfb2cb30fb30 100644
> --- a/automation/tests-artifacts/alpine/3.19.dockerfile
> +++ b/automation/tests-artifacts/alpine/3.19.dockerfile
> @@ -35,6 +35,8 @@ RUN \
>    apk add pciutils && \
>    apk add libelf && \
>    apk add libdw && \
> +  apk add grub-bios && \
> +  apk add libseccomp && \
>    \
>    # Xen
>    cd / && \
> @@ -64,4 +66,6 @@ RUN \
>    \
>    # Create rootfs
>    cd / && \
> -  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
> +  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var && \
> +  # Prepare boot sector for HVM disk
> +  grub-mkimage -o /grub-core.img -O i386-pc -p '(hd0,msdos1)/boot/grub2' boot part_msdos ext2 linux biosdisk configfile normal
> -- 
> git-series 0.9.1
> 
--8323329-1214147340-1715992830=:1052252--


From xen-devel-bounces@lists.xenproject.org Sat May 18 00:41:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 00:41:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724536.1129797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8895-0001PD-Ci; Sat, 18 May 2024 00:41:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724536.1129797; Sat, 18 May 2024 00:41:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8895-0001P6-9f; Sat, 18 May 2024 00:41:55 +0000
Received: by outflank-mailman (input) for mailman id 724536;
 Sat, 18 May 2024 00:41:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I+vq=MV=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s8894-0001Ja-PF
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 00:41:54 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6b8d2fae-14af-11ef-909e-e314d9c70b13;
 Sat, 18 May 2024 02:41:53 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 809B76163C;
 Sat, 18 May 2024 00:41:52 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8FEE2C2BD10;
 Sat, 18 May 2024 00:41:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b8d2fae-14af-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1715992912;
	bh=V8Q5MWxedopOTHvDYQ3KV3TQEh3uHjrguKJ0NZRj8fk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Y960RWJ1oDVK3TKob1KcAOHoVZHdue6UheYYR9+Ad8RiUIbPYbdyrvCW0eM+uO4wv
	 pPgDT7F+uoiDUeXZe5zxLKLXUCh+Go+gjvkdQAnSD6kkA/nyrif09bj88CpDBKXwOl
	 UgSb4RyuijzpikHP91GS25NxoTnFVzfDJZ2sdH9eR7P3ZQYtVTX/WZVbCruD9GF5KX
	 lFDT+15sj5+Itdk4MnfM5jajWK2ldZCbV/8YDqoLOt2sG5hz5CDxiU/I/3fEd4Nc4c
	 RMNIf4XPt+roAtuDoBTIwXf34uNJhd3k+2mHcFkWxGGxt0RzaRVRk8TdWCpLlmfyXF
	 OAQx+ak3sTA1A==
Date: Fri, 17 May 2024 17:41:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH 10/12] automation: stubdom test with PCI passthrough
In-Reply-To: <16c7001cfec1cb7a601414b745655656ea63fe29.1715867907.git-series.marmarek@invisiblethingslab.com>
Message-ID: <alpine.DEB.2.22.394.2405171741160.1052252@ubuntu-linux-20-04-desktop>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com> <16c7001cfec1cb7a601414b745655656ea63fe29.1715867907.git-series.marmarek@invisiblethingslab.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1808204904-1715992912=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1808204904-1715992912=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 May 2024, Marek Marczykowski-Górecki wrote:
> Based on the initial stubdomain test and existing PCI passthrough tests,
> add one that combines both.
> Schedule it on the AMD runner, as it has less tests right now.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

With the caveat that if we do tftp boot this might have to change:

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/gitlab-ci/test.yaml     |  8 ++++++++
>  automation/scripts/qubes-x86-64.sh | 30 +++++++++++++++++++++++++-----
>  2 files changed, 33 insertions(+), 5 deletions(-)
> 
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index e3910f4c1a9f..76cc430ae00f 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -231,6 +231,14 @@ zen3p-pci-hvm-x86-64-gcc-debug:
>      - *x86-64-test-needs
>      - alpine-3.19-gcc-debug
>  
> +zen3p-pci-stubdom-x86-64-gcc-debug:
> +  extends: .zen3p-x86-64
> +  script:
> +    - ./automation/scripts/qubes-x86-64.sh pci-stubdom 2>&1 | tee ${LOGFILE}
> +  needs:
> +    - *x86-64-test-needs
> +    - alpine-3.19-gcc-debug
> +
>  qemu-smoke-dom0-arm64-gcc:
>    extends: .qemu-arm64
>    script:
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index fc73403dbadf..816c16fbab3e 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -98,8 +98,8 @@ ping -c 10 192.168.0.2 || exit 1
>  echo \"${passed}\"
>  "
>  
> -### test: pci-pv, pci-hvm
> -elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ]; then
> +### test: pci-pv, pci-hvm, pci-stubdom
> +elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ] || [ "${test_variant}" = "pci-stubdom" ]; then
>  
>      if [ -z "$PCIDEV" ]; then
>          echo "Please set 'PCIDEV' variable with BDF of test network adapter" >&2
> @@ -109,15 +109,35 @@ elif [ "${test_variant}" = "pci-pv" ] || [ "${test_variant}" = "pci-hvm" ]; then
>  
>      passed="pci test passed"
>  
> -    domU_config='
> +    domain_type="${test_variant#pci-}"
> +    if [ "$test_variant" = "pci-stubdom" ]; then
> +        domain_type="hvm"
> +        domU_config='
> +type = "hvm"
> +disk = [ "/srv/disk.img,format=raw,vdev=xvda" ]
> +device_model_version = "qemu-xen"
> +device_model_stubdomain_override = 1
> +on_reboot = "destroy"
> +# libxl configures vkb backend to be dom0 instead of the stubdomain, defer
> +# changing that until there is consensus what to do about VGA output (VNC)
> +vkb_device = 0
> +'
> +        domU_disk_path=/srv/disk.img
> +    else
> +        domU_config='
>  type = "'${test_variant#pci-}'"
> -name = "domU"
>  kernel = "/boot/vmlinuz"
>  ramdisk = "/boot/initrd-domU"
>  extra = "root=/dev/ram0 console=hvc0 earlyprintk=xen"
> +disk = [ ]
> +'
> +    fi
> +
> +    # common part
> +    domU_config="$domU_config"'
> +name = "domU"
>  memory = 512
>  vif = [ ]
> -disk = [ ]
>  pci = [ "'$PCIDEV',seize=1" ]
>  on_reboot = "destroy"
>  '
> -- 
> git-series 0.9.1
> 
--8323329-1808204904-1715992912=:1052252--


From xen-devel-bounces@lists.xenproject.org Sat May 18 01:08:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 01:08:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724551.1129807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88YJ-0004R3-9x; Sat, 18 May 2024 01:07:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724551.1129807; Sat, 18 May 2024 01:07:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88YJ-0004Qw-6L; Sat, 18 May 2024 01:07:59 +0000
Received: by outflank-mailman (input) for mailman id 724551;
 Sat, 18 May 2024 01:07:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=clH7=MV=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s88YH-0004Qq-Nl
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 01:07:57 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0e06a6e8-14b3-11ef-909e-e314d9c70b13;
 Sat, 18 May 2024 03:07:55 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715994470841306.1623133554082;
 Fri, 17 May 2024 18:07:50 -0700 (PDT)
Received: by mail-yb1-f181.google.com with SMTP id
 3f1490d57ef6-de5b1e6beceso861053276.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 18:07:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e06a6e8-14b3-11ef-909e-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; t=1715994473; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=StgLXpea2Vwb99hH4wGwDDfXJQ/KprFl2VvfawaP4pvsbgxHL4SGTntaTlOlLLGf0f4ieMsLm27RrBEKovcPMM7MjUW9WNecDFW3uQkuc3DpshF9aC5qOX8kg/ghl36ZlPMFldymtf8L0yIgx1E4wRJ37mQ9mh3a3TMFd9pDHUs=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715994473; h=Content-Type:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=xkR8+wmKEUf0ZWHZQWY59zOICyXkQMKCcMF/DSMIUx8=; 
	b=avSXi+xLyrW7h2mANVpuAiLcJ+i0QTXP1ufBtHEusYkcHBfMJL622gbtrx6zDbKFL9ZTsFtwcHumUbqjKEZe5Z+cNgW4n+2dm8mRWupIcQwIMO4Ia7GYjhr4s2Fmm2rUx6inBjwGOd6xqipIekakBcjk6hLiBHZtISIIJ5sxnPs=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715994473;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Message-Id:Reply-To;
	bh=xkR8+wmKEUf0ZWHZQWY59zOICyXkQMKCcMF/DSMIUx8=;
	b=f6Mczlx3pc0MsnsOKLnXmpCfYuvj8H9hvet6nWT7IO2ARJ/QLlcVTL4Neazy7PDH
	9svhtJx8HuG9wbFCsAFk3s9o+s2cYhvy1HNhRF81jor9JGrL2D30U5BSB7ahjMq8MjJ
	NfYrxq9r4ddS9ktjuRqXJSPaDS1hP1B1z0g3Vxd8=
X-Gm-Message-State: AOJu0YxZcbQQaz1KA5abHvMfYWfF3OtQJBl1+buJXfVirCsr9dK2i3x4
	Lt2U2FdFrdYX1rOlfteFQk92oT2D55Bu0NAKhfsJ18VA5OE8uZAb8UfW4aM8Fucj2M93CEx334s
	ToCoE515oH9X2fhdghQkdvvbQN1U=
X-Google-Smtp-Source: AGHT+IEbOH6FPRcVpXlX5Kaqv5pt/u50WWfytF6UNtO9dMJVzZqae7DlEA/ICyj/fw7jYdBlWxxQvgbMtfGWKudxNUY=
X-Received: by 2002:a25:c791:0:b0:de5:53c0:b9e5 with SMTP id
 3f1490d57ef6-dee4f4b9abdmr23902838276.40.1715994469859; Fri, 17 May 2024
 18:07:49 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
In-Reply-To: <14a8c523b24c87959941e905bd60933a91144bc7.1715761386.git.Sergiy_Kibrik@epam.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 17 May 2024 21:07:13 -0400
X-Gmail-Original-Message-ID: <CABfawhkFG5waaP8A9KTtOT1nn4GS74Oi+yRBK-+YtagDLTbNJA@mail.gmail.com>
Message-ID: <CABfawhkFG5waaP8A9KTtOT1nn4GS74Oi+yRBK-+YtagDLTbNJA@mail.gmail.com>
Subject: Re: [XEN PATCH v2 05/15] x86: introduce CONFIG_ALTP2M Kconfig option
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xenia Ragiadakou <xenia.ragiadakou@amd.com>
Content-Type: text/plain; charset="UTF-8"

> Currently altp2m support provided for VT-d only, so option is dependant on VMX.

No clue what is meant by "support provided for VT-d only". Altp2m has
nothing to do with VT-d. It would be more accurate to say it's only
implemented for Intel EPT.

Tamas


From xen-devel-bounces@lists.xenproject.org Sat May 18 01:12:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 01:12:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724554.1129817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88cD-0006a0-PL; Sat, 18 May 2024 01:12:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724554.1129817; Sat, 18 May 2024 01:12:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88cD-0006Zt-MK; Sat, 18 May 2024 01:12:01 +0000
Received: by outflank-mailman (input) for mailman id 724554;
 Sat, 18 May 2024 01:12:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=clH7=MV=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s88cC-0006Zn-RQ
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 01:12:00 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9ebfffa7-14b3-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 03:11:58 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715994712030331.834854596682;
 Fri, 17 May 2024 18:11:52 -0700 (PDT)
Received: by mail-yb1-f171.google.com with SMTP id
 3f1490d57ef6-dcc71031680so734173276.2
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 18:11:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ebfffa7-14b3-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; t=1715994714; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=MafmZQ56iE2vTtS830ntl9aALvlbyjGDbi4HNBICasSZaH0WgQcNFC2ngEhjEqWGl7Hjox0JROFufOegFfSZ7xYayXKXdLPxgjHG7YRcnvLuRYebc7GkOXz9nROfF+Hx0zap2n45lbNDDsbB/9m1HZo9UNb6QRD17zks76ywuvQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715994714; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=H9DuXgbTdtu+6poQIgNvq5HDYC+0Cz6HDnPJImUWZ8I=; 
	b=V0/leUf9hSWpBmHct6reFaEs8K5kjemNtCzodt4YLDZCh9prmEOsBrtfU/CtpRg+ANOh2T+dhwgwe9MahYM4HDI3dzkpD0MQe/M0Qufz9MhxjzOD4eKtVCj3GuEpxY64bQ0jQGz2lHCtjSkCC8cm6x7/YI/W33RAL/RR6jf/I1s=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715994714;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=H9DuXgbTdtu+6poQIgNvq5HDYC+0Cz6HDnPJImUWZ8I=;
	b=DSk69ltxdgIEmMtAXiygoH8qpViZYWLC8Hr3jOmDQGNq0jM0tM278IGZARm9IpHz
	7b8QxZ1W/uzNClmkhrN+Ygr1Lef3yI5MQ/lEwuuXy6F+hRZQHGLVFaip+jKZYKMbG1S
	dH1bBpmiqR5Wdpfl8T30Cs4HbIQS1MaAyficzeWY=
X-Gm-Message-State: AOJu0YwZvClSc/n7esfStQcxEE682CantLeIG6jyrQRxUYmra8YeD9tT
	sZdJ4B5klsmjZ+KWFc9ahjqO26+D/qopemduVFY2iWxpM+ZAsWHVWcKFmFytRiAYpeaLHR78PFe
	rW+h+YLQsHYT8LbQGtNWC8kLpUiE=
X-Google-Smtp-Source: AGHT+IG/y2hcDgj03cc4X2SwUPHqTOI+D137mCQ6cxzcZmGEocA0YVsrmCugh6EJpcDU09T03pBa2l+O/6RtBY7OQTw=
X-Received: by 2002:a5b:5cc:0:b0:dbd:be40:2191 with SMTP id
 3f1490d57ef6-dee4f3a9dc0mr20842037276.42.1715994711241; Fri, 17 May 2024
 18:11:51 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1715952103.git.oleksii.kurochko@gmail.com> <bb75ebc8bd75318759b450d398294eee8ed3ca55.1715952103.git.oleksii.kurochko@gmail.com>
In-Reply-To: <bb75ebc8bd75318759b450d398294eee8ed3ca55.1715952103.git.oleksii.kurochko@gmail.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 17 May 2024 21:11:14 -0400
X-Gmail-Original-Message-ID: <CABfawh=o0pUSYACUzH4-De2mDm5_vm7Hbf5bEt6Vo8+zyU2L0A@mail.gmail.com>
Message-ID: <CABfawh=o0pUSYACUzH4-De2mDm5_vm7Hbf5bEt6Vo8+zyU2L0A@mail.gmail.com>
Subject: Re: [PATCH v10 07/14] xen/riscv: introduce monitor.h
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>, Alistair Francis <alistair.francis@wdc.com>, 
	Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 17, 2024 at 9:55=E2=80=AFAM Oleksii Kurochko
<oleksii.kurochko@gmail.com> wrote:
>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Sat May 18 01:13:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 01:13:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724557.1129826 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88dI-0007BA-4d; Sat, 18 May 2024 01:13:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724557.1129826; Sat, 18 May 2024 01:13:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88dI-0007B3-1u; Sat, 18 May 2024 01:13:08 +0000
Received: by outflank-mailman (input) for mailman id 724557;
 Sat, 18 May 2024 01:13:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=clH7=MV=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s88dG-0007Av-SK
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 01:13:06 +0000
Received: from sender3-op-o19.zoho.com (sender3-op-o19.zoho.com
 [136.143.184.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c63eb351-14b3-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 03:13:04 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715994780280138.69004225141668;
 Fri, 17 May 2024 18:13:00 -0700 (PDT)
Received: by mail-yb1-f176.google.com with SMTP id
 3f1490d57ef6-dcd7c526cc0so887655276.1
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 18:13:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c63eb351-14b3-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; t=1715994780; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=NvDjzQUWpbm/0e6GYrnb35cNeS+KNN1f7UsJpezvg3SKRPJQg1N0mFuwE01OyTx1CGzHQib08kJdtCSBQc+BqZkvKH6fI9L6+fo9kol4NHvWDTxMUsRDS0Fg7ECaIbPWdcDZRXm98xTKsUE6iIguhu+BH6kvWr7Qt8YpENF4uJA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715994780; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=H9DuXgbTdtu+6poQIgNvq5HDYC+0Cz6HDnPJImUWZ8I=; 
	b=G8ZzAlaovf7YDmr/stUpWKdJf8PRCXdHR39bG+IgSCuWuc/f/d0Hh0CPDjNsSMwpeDtJ8GZoKVy4TVH5bSQWcU9/BF9ch/r2XerGLseUO4B/tetgH2cXxyd6g07Uhz6h1MJI9HGIgYkSSSmJ9L10ZKalsgf0xX65ix/4kYcXRLs=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715994780;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=H9DuXgbTdtu+6poQIgNvq5HDYC+0Cz6HDnPJImUWZ8I=;
	b=EZM4glpB42MtXDBBOVh+xHthmOWMqkKXKogUgGUQsMx2a+o8tBNZ/TqiXmzor9Eq
	W8uVS0SK5T3x5lQ08WpIyIkZJknsCyXItZQQLYHSPRqMRss4yp5R4NRv8a2HLrWBZ9+
	c+hxqMkHK8QvjvSzDF21N/XlvSXmErO6Kn4Le02g=
X-Gm-Message-State: AOJu0YxCAxCDbLH94MRzQxikMQEY9ae7rXWeJNWjWEPcEF4+bsO9TJRz
	e/z/8hqyCgQ/+yXVar9J162oxAQyqD7E8wL1YMXRCLE/H/yOxLpK4b6Sj/1jaorHOL4jfywrTWD
	NEw+p5VVJHbqLctDtSrABnWxBXh4=
X-Google-Smtp-Source: AGHT+IFD77zm8A7spUf4++zueAy+CMjlNTF2jnf7ZxaQRc9L+vcmVBwIP1OhWBHrKGzrjYn58FGuiWqSNqagjgIJAik=
X-Received: by 2002:a25:b101:0:b0:df1:cdfa:8e46 with SMTP id
 3f1490d57ef6-df1cdfa9222mr10006736276.41.1715994779365; Fri, 17 May 2024
 18:12:59 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1715952103.git.oleksii.kurochko@gmail.com> <050567a35805361edc0e222f4ddebfa91403cfb6.1715952103.git.oleksii.kurochko@gmail.com>
In-Reply-To: <050567a35805361edc0e222f4ddebfa91403cfb6.1715952103.git.oleksii.kurochko@gmail.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 17 May 2024 21:12:23 -0400
X-Gmail-Original-Message-ID: <CABfawhmGmn-bSpfL81-yxmt0EMDw73bsibND0nGpAPecJjBc_g@mail.gmail.com>
Message-ID: <CABfawhmGmn-bSpfL81-yxmt0EMDw73bsibND0nGpAPecJjBc_g@mail.gmail.com>
Subject: Re: [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: xen-devel@lists.xenproject.org, 
	Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, 
	Connor Davis <connojdavis@gmail.com>, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 17, 2024 at 9:55=E2=80=AFAM Oleksii Kurochko
<oleksii.kurochko@gmail.com> wrote:
>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Sat May 18 01:18:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 01:18:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724563.1129837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88if-0007ns-O6; Sat, 18 May 2024 01:18:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724563.1129837; Sat, 18 May 2024 01:18:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88if-0007nl-K6; Sat, 18 May 2024 01:18:41 +0000
Received: by outflank-mailman (input) for mailman id 724563;
 Sat, 18 May 2024 01:18:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=clH7=MV=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s88id-0007nf-ST
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 01:18:39 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8cf7975e-14b4-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 03:18:37 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1715995113574541.5799814820792;
 Fri, 17 May 2024 18:18:33 -0700 (PDT)
Received: by mail-yb1-f171.google.com with SMTP id
 3f1490d57ef6-de462979e00so810350276.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 May 2024 18:18:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8cf7975e-14b4-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; t=1715995114; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=KrTXJ/6x92WjMR1IqMfMy2vfgaZSFnrgdKyKzL6lXhHUQe6Gr4oDWLpMqAubaEi79u2I2ig29BOCorKUFJmvV47HUJDj/KKVm9Ctv1O/s23CFNP9dj89Z/u9CBJ5Ug6wJCQuLQICC71w+D7MZ9yC3+91xZ93W47HDGmUfo4uqMY=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1715995114; h=Content-Type:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=RvRG5y7QgTMlvDcAZl1+oKr+gk4YK/iWfKJdCgb0dc0=; 
	b=EXotpwKtAHEbYoU+AWlL0z6vVjNbgdfmU0R3xif4zV6jAO7dL2STThQu1UXOnbnsIfYs06VZuaB+Aqumxt6jtMG2d+XTajenGp/2kcb+y/mXjyrHpgZSM7J5ocA3u+zB6tadioHmTAj5rs+JcVnrK98ImaDpCQl5Ut1CSzcSM00=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1715995114;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Message-Id:Reply-To;
	bh=RvRG5y7QgTMlvDcAZl1+oKr+gk4YK/iWfKJdCgb0dc0=;
	b=I0/FmhNXxCnFoi4eLCzbZRmurDaubzCb3d4EV+ZdM9YxJgOcJOZez4SC/7Gff20p
	bLeomvHDy/kcE9Yw27VdQtftcdYh0glXYfJBscEqIr7ldi3jNA6V1dAPiH7dbgh8udW
	THn5xqIBN48UGOuGa2N4V/YfZHa8Lwpchzc14X24=
X-Gm-Message-State: AOJu0YztKK7C0jQ8Y8xakCgqa33MRNegKa6fZ1IPALNqdO//EuYJJryU
	Fvlw/FDgCUEJN/Bj6UHZ6oSdkP+TRgdgVZ42DHXP4XFry7ZjffQv4pCaCayiavcK7H6/wVxrisg
	DnWaNp1ETwqsJ3u2Z7IfP672BmjQ=
X-Google-Smtp-Source: AGHT+IGwtRcwnGqskymcoQCIk/2RbEkK0LJy5Pg5+ApPvd1YKrzkvW5fi1Ny0CNN7Bt7byX1JosHKTXDkp6XZlhTdqk=
X-Received: by 2002:a25:8c86:0:b0:dee:797e:a325 with SMTP id
 3f1490d57ef6-dee797ea442mr16388088276.10.1715995112787; Fri, 17 May 2024
 18:18:32 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1715858136.git.w1benny@gmail.com> <b6bf7e5ba0deab4550b9d15c327015fe7d94d463.1715858136.git.w1benny@gmail.com>
In-Reply-To: <b6bf7e5ba0deab4550b9d15c327015fe7d94d463.1715858136.git.w1benny@gmail.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 17 May 2024 21:17:56 -0400
X-Gmail-Original-Message-ID: <CABfawhmB+XND75WPJ_MjORyjUiV_h_NNmJQx2BaZ5WRmzkX79A@mail.gmail.com>
Message-ID: <CABfawhmB+XND75WPJ_MjORyjUiV_h_NNmJQx2BaZ5WRmzkX79A@mail.gmail.com>
Subject: Re: [PATCH for-4.19? v3 4/6] x86: Make the maximum number of altp2m
 views configurable
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>
Content-Type: text/plain; charset="UTF-8"

> -        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +        ap2m = d->arch.altp2m_p2m[altp2m_idx];

Why is it no longer necessary to use array_access_nospec?

Tamas


From xen-devel-bounces@lists.xenproject.org Sat May 18 01:28:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 01:28:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724570.1129846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88sR-0001sG-JV; Sat, 18 May 2024 01:28:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724570.1129846; Sat, 18 May 2024 01:28:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s88sR-0001s9-H1; Sat, 18 May 2024 01:28:47 +0000
Received: by outflank-mailman (input) for mailman id 724570;
 Sat, 18 May 2024 01:28:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9G4V=MV=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s88sQ-0001s1-KF
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 01:28:46 +0000
Received: from wfout8-smtp.messagingengine.com
 (wfout8-smtp.messagingengine.com [64.147.123.151])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f5864e5c-14b5-11ef-909e-e314d9c70b13;
 Sat, 18 May 2024 03:28:43 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.west.internal (Postfix) with ESMTP id D45471C000C5;
 Fri, 17 May 2024 21:28:39 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute5.internal (MEProxy); Fri, 17 May 2024 21:28:40 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 17 May 2024 21:28:38 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5864e5c-14b5-11ef-909e-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1715995719;
	 x=1716082119; bh=zd22xuUubCFGRKxRtahgPrZdVvCDuTzfTyBsoeycvAk=; b=
	B0ixvmidpWNed+fJYIGwYeAGlJw2klLz6RXFHhToLu6bEBayv8saLYHogiIZqSGn
	NGCmUQUvR5hotBCyu5cAilFhoVuT/croNw7gyWun0aZwMFV7lAbmWupCCtAaKIOl
	61Kiy3vVjrgjau1/crWvkUNEjiTBdQqEbqWNhDPMaF+SZ+Ftkd5XAKY2zRSCEQQf
	UTTwcDNUiAspiN+N0s2NMLgxT+t1wroEhdgW5qNdUlvk4W4wEECyPE2/DENE8Yd0
	SE/cNhFfFCCcAW5sgq2daT71MR+pzuZAo3VfRszQsaGNDVCGOhw2B2uolzAKyIrN
	CrttZ//Yn+4DIZDurCmmyA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1715995719; x=1716082119; bh=zd22xuUubCFGRKxRtahgPrZdVvCD
	uTzfTyBsoeycvAk=; b=KBJKyJwlyftVL9jzFNiRYGdUXN57Ueq1RixPnt6ZL2jc
	HY8uWj27kUrTW11BYFwnsu9rFjRYfSoq6P9Qxneh+XQgGJeLnZGaApXfZzd4m2jt
	za+HargUjafECQXL0T7r4CGUVq8x7J1eQ1sTNnQsRSPLcVwzXJbd7R3+liaZuWAK
	X3dPJRWiK/7qdygnDKp7mXBWCyoAaKksd9vjX3+XtTRw2bhXGM+n8VIp60YcL7PW
	2tt9+5jBH8f8ZwHdNaEJ5Cs/nmW6wTuXJUn/RRZdeBUMoYPoKqx5ywCI+OKXNrm+
	oCS7ne2OamamUxnzWUkrFl8wX9oBIaLKgXL20yu4Pg==
X-ME-Sender: <xms:RwRIZgHfT7GjU9xvnxv1p5nqtG9hlRdVkK9emnMRxMjxygpncqMOaQ>
    <xme:RwRIZpX1PvzxW7XHzvBLrqjz-IyN3llQf-YFna42JtT1eX-fM0iT-JUj_zE0w4NHB
    d0P1I2Jdi8xOQ>
X-ME-Received: <xmr:RwRIZqLGLopJGr-EMgcs36LY_ROiknbX8qZqRKlfL-L0kDD_9f4PLVMeHIJJhId9GX3XKfYDXGQpJs_jMeStgUONln8w7nMN3g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehhedggeeiucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepkefg
    vefhveekffeivedvgfelvddvhefhfeeukeefudfhheeuvddutdekleefkeejnecuffhomh
    grihhnpehgihhtlhgrsgdrtghomhdpghhithhhuhgsrdgtohhmnecuvehluhhsthgvrhfu
    ihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvih
    hsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:RwRIZiGLm7ppr6GHGLv_VnyEDHiRGCEk307YiViRD7rUZRyIlosqng>
    <xmx:RwRIZmWn6U97eQUNHreeOBBKW_K8xaxeeHe8yvan-NAa3RWg0o4NkA>
    <xmx:RwRIZlOg41qb-7Fw5XJNgtzN6jkCnhvdBiORgcijIuRNwUcNWQ2nEw>
    <xmx:RwRIZt1mF25k526qH9-9sz8-chSGA1NoOWEZXejEZ1DNOTkGuXNT8w>
    <xmx:RwRIZjhcehU_ihHglc_PUktLr7KJmEbC9mXslmKLf9_AXXekE7zJI_WT>
Feedback-ID: i1568416f:Fastmail
Date: Sat, 18 May 2024 03:28:35 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [PATCH 06/12] RFC: automation: Add linux stubdom build and smoke
 test
Message-ID: <ZkgERD6pejvJs2gg@mail-itl>
References: <cover.f23e316613d8f82981f99b5fb90e36255c4adc63.1715867907.git-series.marmarek@invisiblethingslab.com>
 <c11feff249928f7e06a91c9b60633e1f74bbd6fd.1715867907.git-series.marmarek@invisiblethingslab.com>
 <alpine.DEB.2.22.394.2405171718580.1052252@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="/jbizhnpfyenlI1g"
Content-Disposition: inline
In-Reply-To: <alpine.DEB.2.22.394.2405171718580.1052252@ubuntu-linux-20-04-desktop>


--/jbizhnpfyenlI1g
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sat, 18 May 2024 03:28:35 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [PATCH 06/12] RFC: automation: Add linux stubdom build and smoke
 test

On Fri, May 17, 2024 at 05:40:52PM -0700, Stefano Stabellini wrote:
> On Thu, 16 May 2024, Marek Marczykowski-G=C3=B3recki wrote:
> > Add minimal linux-stubdom smoke test. It starts a simple HVM with
> > linux-stubdom. The actual stubdom implementation is taken from Qubes OS
> > and then stripped off Qubes-specific code. In particular, the remaining
> > code does _not_ support:
> >  - direct kernel boot (implemented by relaying on specific guest disk
> >    laying in Qubes OS)
> >  - graphical console (used Qubes GUI agent injected into
> >    stubdomain's qemu)
> >  - audio input/output (used Qubes audio agent inside stubdomain)
> >  - USB passthrough (used qrexec <-> usbip proxy inside stubdomain)
> >  - setting up DHCP server (assumes guest addressing used in Qubes OS)
> >=20
> > For this smoke test, the relevant part is missing direct kernel boot, as
> > that's used in other smoke tests. Solve this by preparing disk image
> > with proper bootloader (grub) installed. Since the test script is
> > running on arm64 to control x86_64 box, it cannot (easily) install grub
> > directly. For this reason, prepare bootsector as part of the Xen build
> > (which runs on x86_64) and then prepend do the disk image during the
> > test (and adjust partitions table afterwards).
>=20
> I am not an expert on this, but do you think it would be possible to use
> network boot and tftp instead of grub on emulated disk? That would not
> require us to build neither /grub-core.img nor build_domU_disk().

Honestly, I don't know. I guess I'd need at least dnsmasq in dom0, and
also iPXE for the domU (if not built already?). I can try for this test.
But a later test (the PCI one) connects a network card and dom0 can't
really setup own DHCP on that network. Additionally combining this with
vif network for PXE might be confusing down the road.

> I am trying to avoid grub-core.img and disk.img because I think direct
> kernel boot or network boot are easier to maintain and more similar to
> the other tests. If you see the ARM tests, they all use tftp boot.

The ARM ones boot as dom0less, where there is only one boot mode for the
system to start in. Here, we have two: xen+dom0 (which already
does network boot), and then domU started by dom0. The latter would
need either a separate DHCP server on a separate network (vif interface
in dom0 should be fine), or some other way to separate dom0/domU boot
mode.

That said, the stubdom used in Qubes does support direct kernel boot. It
is removed from this version, because it relies on specific disk layout
(it reserves xvdd for this purpose). But I do want to bring this
capability to the upstream version too at some point.

> > Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblething=
slab.com>
> > ---
> > The test is implemented using hardware runner, because some of the
> > further tests will require it (for example PCI passthrough with
> > stubdomain). But if there is strong desire to have stubdomain tested
> > inside qemu tests (to be included in patchew runs), it is probably an
> > option for this basic smoke test.
>=20
> Thanks for this amazing work. This is a great start, we can see how to
> create more tests after merging this one.
>=20
>=20
> > For now I'm keeping stubdomain code (build and glue scripts) in separate
> > repository on my github account. This is far from ideal. What would be
> > preferred option? New repository on xenbits? Or add directly into
> > xen.git (stubdom directory)? Honestly, I'd rather avoid the latter, as
> > from packager point of view those are mostly separate beings (similar to
> > qemu, where many use distribution-provide one instead of the one bundled
> > with Xen) and it's convenient to not need to rebuild stubdomain on every
> > hypervisor change (like a security patch).
>=20
> My suggestion is to create repositories under gitlab.com/xen-project

gitlab.com/xen-project/stubdom-dm-linux ?
Initially I can create the repository under people/marmarek/.

Is there any preference regarding git history? I see two options:
1. Preserve the current history, where there is a lot of qubes-specific
work and on top a bunch of commits making it not qubes-specific (this is
what is there now).
2. Start with fresh history and reference original repository (and the
commit id) in the initial commit message.

> > Another topic is QEMU version inside stubdomain. It needs to be a
> > separate build due to vastly different configure options, so I cannot
> > reuse the qemu binary built for dom0 (or distribution-provided one if
> > Xen is configured to use it). But also, at this moment qemu for
> > stubdomain needs few extra patches that are not upstream yet.
> > What should be the proper solution here (after upstreaming all the
> > patches)?
>=20
> It is fine to use a different QEMU version. For now, I would suggest to
> keep the QEMU patches as patches to be applied, in the new repositories
> under gitlab.com/xen-project

Ok, makes sense.

> > Generally, I try to add tests early, even though there is still some
> > work to do for proper stubdomain integration into upstream Xen, so any
> > cleanups and future changes (like the CDROM libxl patches by Jason
> > Andryuk) can be made with more confidence and reduce risk of
> > regressions.
> >=20
> > The patch is RFC only because of the stubdom repository location.
> > ---
> >  automation/build/alpine/3.19-arm64v8.dockerfile   |  2 +-
> >  automation/build/alpine/3.19.dockerfile           |  9 ++-
> >  automation/gitlab-ci/build.yaml                   |  3 +-
> >  automation/gitlab-ci/test.yaml                    |  8 +-
> >  automation/scripts/build                          | 12 ++-
> >  automation/scripts/qubes-x86-64.sh                | 87 +++++++++++++++-
> >  automation/tests-artifacts/alpine/3.19.dockerfile |  6 +-
> >  7 files changed, 123 insertions(+), 4 deletions(-)
> >=20
> > diff --git a/automation/build/alpine/3.19-arm64v8.dockerfile b/automati=
on/build/alpine/3.19-arm64v8.dockerfile
> > index 158cf465a9ff..12810f87ecc6 100644
> > --- a/automation/build/alpine/3.19-arm64v8.dockerfile
> > +++ b/automation/build/alpine/3.19-arm64v8.dockerfile
> > @@ -47,3 +47,5 @@ RUN apk --no-cache add \
> >    # qubes test deps
> >    openssh-client \
> >    fakeroot \
> > +  sfdisk \
> > +  e2fsprogs \
> > diff --git a/automation/build/alpine/3.19.dockerfile b/automation/build=
/alpine/3.19.dockerfile
> > index 0be6d7c85fe7..108284613987 100644
> > --- a/automation/build/alpine/3.19.dockerfile
> > +++ b/automation/build/alpine/3.19.dockerfile
> > @@ -49,3 +49,12 @@ RUN apk --no-cache add \
> >    pixman-dev \
> >    # livepatch-tools deps
> >    elfutils-dev \
> > +  # stubdom deps
> > +  dracut-core \
> > +  quilt \
> > +  gnupg \
> > +  libseccomp-dev \
> > +  glib-static \
> > +  gmp-dev \
> > +  mpc1-dev \
> > +  mpfr-dev \
> > diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/bui=
ld.yaml
> > index b186289bbd82..783a0687ba34 100644
> > --- a/automation/gitlab-ci/build.yaml
> > +++ b/automation/gitlab-ci/build.yaml
> > @@ -323,9 +323,11 @@ alpine-3.19-rootfs-export:
> >    image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.=
19
> >    script:
> >      - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
> > +    - cp /grub-core.img binaries/grub-core.img
> >    artifacts:
> >      paths:
> >        - binaries/initrd.tar.gz
> > +      - binaries/grub-core.img
> >    tags:
> >      - x86_64
> > =20
> > @@ -353,6 +355,7 @@ alpine-3.19-gcc-debug:
> >    extends: .gcc-x86-64-build-debug
> >    variables:
> >      CONTAINER: alpine:3.19
> > +    STUBDOM_LINUX: y
> > =20
> >  debian-stretch-gcc-debug:
> >    extends: .gcc-x86-64-build-debug
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test=
=2Eyaml
> > index f62d426a8d34..80d10eb7f476 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -199,6 +199,14 @@ adl-pci-hvm-x86-64-gcc-debug:
> >      - *x86-64-test-needs
> >      - alpine-3.19-gcc-debug
> > =20
> > +adl-stubdom-hvm-x86-64-gcc-debug:
> > +  extends: .adl-x86-64
> > +  script:
> > +    - ./automation/scripts/qubes-x86-64.sh stubdom-hvm 2>&1 | tee ${LO=
GFILE}
> > +  needs:
> > +    - *x86-64-test-needs
> > +    - alpine-3.19-gcc-debug
> > +
> >  zen3p-smoke-x86-64-gcc-debug:
> >    extends: .zen3p-x86-64
> >    script:
> > diff --git a/automation/scripts/build b/automation/scripts/build
> > index b3c71fb6fb60..f7c85b07a8d0 100755
> > --- a/automation/scripts/build
> > +++ b/automation/scripts/build
> > @@ -102,4 +102,16 @@ else
> >      # build Xen
> >      cp -r dist binaries/
> >      if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fi
> > +
> > +    if [[ "${STUBDOM_LINUX}" =3D=3D "y" ]]; then
> > +        git clone https://github.com/marmarek/qubes-vmm-xen-stubdom-li=
nux -b for-upstream2
> > +        make -j$(nproc) -C qubes-vmm-xen-stubdom-linux get-sources
> > +        CFLAGS=3D"-I${PWD}/dist/install/usr/local/include" \
> > +            LDFLAGS=3D"-L${PWD}/dist/install/usr/local/lib" \
> > +            LD_LIBRARY_PATH=3D"${PWD}/dist/install/usr/local/lib" \
> > +            PATH=3D"${PWD}/dist/install/usr/local/bin:$PATH" \
> > +            make -j$(nproc) -C qubes-vmm-xen-stubdom-linux -f Makefile=
=2Estubdom all
> > +        cp qubes-vmm-xen-stubdom-linux/build/linux/arch/x86/boot/bzIma=
ge binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-kernel
> > +        cp qubes-vmm-xen-stubdom-linux/build/rootfs/stubdom-linux-root=
fs binaries/dist/install/usr/local/lib/xen/boot/qemu-stubdom-linux-rootfs
> > +    fi
>=20
> I think qemu-stubdom-linux-kernel and qemu-stubdom-linux-rootfs should
> be built as test-artifacts if they are not strictly dependent on the Xen
> version.

This is almost true - QEMU needs to be built for the specific Xen version,
since it still uses some of the unstable ABIs... In practice, it might
be possible to have test-artifact for each Xen version separately, but
it will be PITA about the time of bumping the interface version (like
XEN_DOMCTL_INTERFACE_VERSION).

> Otherwise, please create a separate new build job for linux stubdoms (do
> not reuse alpine-3.19-gcc-debug). Also I would prefer if these build
> instructions were on a separate file, separate from "build".

Ok.

> >  fi
> > diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qu=
bes-x86-64.sh
> > index 77cb0d45815d..fc73403dbadf 100755
> > --- a/automation/scripts/qubes-x86-64.sh
> > +++ b/automation/scripts/qubes-x86-64.sh
> > @@ -18,6 +18,37 @@ memory =3D 512
> >  vif =3D [ "bridge=3Dxenbr0", ]
> >  disk =3D [ ]
> >  '
> > +domU_disk_path=3D
> > +
> > +### helper functions
> > +
> > +build_domU_disk() {
> > +    local kernel=3D"$1"
> > +    local initrd=3D"$2"
> > +    local rootfs=3D"$3"
> > +    local output=3D"$4"
> > +    local grubcfg=3D"$rootfs/boot/grub2/grub.cfg"
> > +    local kernel_cmdline=3D"root=3D/dev/xvda1 console=3Dhvc0 earlyprin=
tk=3Dxen"
> > +
> > +    mkdir -p "$rootfs/boot/grub2"
> > +    cp "$kernel" "$rootfs/boot/vmlinuz"
> > +    echo "linux /boot/vmlinuz $kernel_cmdline" >> "$grubcfg"
> > +    if [ -n "$initrd" ]; then
> > +        cp "$initrd" "$rootfs/boot/initrd.img"
> > +        echo "initrd /boot/initrd.img" >> "$grubcfg"
> > +    fi
> > +    echo "boot" >> "$grubcfg"
> > +    size=3D$(du -sm "$rootfs")
> > +    size=3D${size%%	*}
> > +    # add 5M margin
> > +    size=3D$(( size + 5 ))
> > +    mke2fs -d "$rootfs" "$output.part1" ${size}m
> > +    cat "$rootfs/usr/lib/grub/i386-pc/boot_hybrid.img" binaries/grub-c=
ore.img > "$output"
> > +    # align for the partition 1 start (2048 sectors)
> > +    truncate -s $((2048 * 512)) "$output"
> > +    cat "$output.part1" >> "$output"
> > +    echo ",,linux,*" | sfdisk "$output"
> > +}
> > =20
> >  ### test: smoke test & smoke test PVH
> >  if [ -z "${test_variant}" ] || [ "${test_variant}" =3D "dom0pvh" ]; th=
en
> > @@ -116,6 +147,41 @@ until grep -q \"^domU Welcome to Alpine Linux\" /v=
ar/log/xen/console/guest-domU.
> >      sleep 1
> >  done
> >  "
> > +
> > +### test: stubdom-hvm
> > +elif [ "${test_variant}" =3D "stubdom-hvm" ]; then
> > +    passed=3D"ping test passed"
> > +
> > +    domU_config=3D'
> > +type =3D "hvm"
> > +name =3D "domU"
> > +memory =3D 512
> > +vif =3D [ "bridge=3Dxenbr0", ]
> > +disk =3D [ "/srv/disk.img,format=3Draw,vdev=3Dxvda" ]
> > +device_model_version =3D "qemu-xen"
> > +device_model_stubdomain_override =3D 1
> > +on_reboot =3D "destroy"
> > +# libxl configures vkb backend to be dom0 instead of the stubdomain, d=
efer
> > +# changing that until there is consensus what to do about VGA output (=
VNC)
> > +vkb_device =3D 0
> > +'
> > +    domU_check=3D"
> > +ifconfig eth0 192.168.0.2
> > +until ping -c 10 192.168.0.1; do
> > +    sleep 1
> > +done
> > +echo \"${passed}\"
> > +"
> > +    dom0_check=3D"
> > +set +x
> > +until grep -q \"${passed}\" /var/log/xen/console/guest-domU.log; do
> > +    sleep 1
> > +done
> > +set -x
> > +echo \"${passed}\"
> > +"
> > +
> > +    domU_disk_path=3D/srv/disk.img
> >  fi
> > =20
> >  # DomU
> > @@ -137,8 +203,17 @@ ${domU_check}
> >  chmod +x etc/local.d/xen.start
> >  echo "rc_verbose=3Dyes" >> etc/rc.conf
> >  sed -i -e 's/^Welcome/domU \0/' etc/issue
> > -find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../bina=
ries/domU-rootfs.cpio.gz
> >  cd ..
> > +if [ -n "$domU_disk_path" ]; then
> > +    build_domU_disk \
> > +        "binaries/bzImage" \
> > +        "" \
> > +        "rootfs" \
> > +        "binaries/disk.img"
> > +else
> > +    (cd rootfs; find . | fakeroot -i ../fakeroot-save cpio -H newc -o =
| gzip > ../binaries/domU-rootfs.cpio.gz)
> > +fi
> > +
> >  rm -rf rootfs
> > =20
> >  # DOM0 rootfs
> > @@ -152,6 +227,9 @@ mkdir srv
> >  mkdir sys
> >  rm var/run
> >  cp -ar ../binaries/dist/install/* .
> > +if [ -n "$domU_disk_path" ]; then
> > +    cp ../binaries/disk.img "./$domU_disk_path"
> > +fi
> > =20
> >  echo "#!/bin/bash
> > =20
> > @@ -164,8 +242,9 @@ ifconfig eth0 up
> >  ifconfig xenbr0 up
> >  ifconfig xenbr0 192.168.0.1
> > =20
> > -# get domU console content into test log
> > +# get domU (and possibly its stubdom) console content into test log
> >  tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^=
/(domU) /\" &
> > +tail -F /var/log/xen/console/guest-domU-dm.log 2>/dev/null | sed -e \"=
s/^/(domU-dm) /\" &
> >  tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu=
-dm) /\" &
> >  xl -vvv create /etc/xen/domU.cfg
> >  ${dom0_check}
> > @@ -178,7 +257,9 @@ echo "XENCONSOLED_TRACE=3Dall" >> etc/default/xenco=
mmons
> >  echo "QEMU_XEN=3D/bin/false" >> etc/default/xencommons
> >  mkdir -p var/log/xen/console
> >  cp ../binaries/bzImage boot/vmlinuz
> > -cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
> > +if [ -r ../binaries/domU-rootfs.cpio.gz ]; then
> > +    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
> > +fi
> >  find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../bina=
ries/dom0-rootfs.cpio.gz
> >  cd ..
> > =20
> > diff --git a/automation/tests-artifacts/alpine/3.19.dockerfile b/automa=
tion/tests-artifacts/alpine/3.19.dockerfile
> > index 6d665daedfa4..cfb2cb30fb30 100644
> > --- a/automation/tests-artifacts/alpine/3.19.dockerfile
> > +++ b/automation/tests-artifacts/alpine/3.19.dockerfile
> > @@ -35,6 +35,8 @@ RUN \
> >    apk add pciutils && \
> >    apk add libelf && \
> >    apk add libdw && \
> > +  apk add grub-bios && \
> > +  apk add libseccomp && \
> >    \
> >    # Xen
> >    cd / && \
> > @@ -64,4 +66,6 @@ RUN \
> >    \
> >    # Create rootfs
> >    cd / && \
> > -  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin =
usr var
> > +  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin =
usr var && \
> > +  # Prepare boot sector for HVM disk
> > +  grub-mkimage -o /grub-core.img -O i386-pc -p '(hd0,msdos1)/boot/grub=
2' boot part_msdos ext2 linux biosdisk configfile normal
> > --=20
> > git-series 0.9.1
> >=20


--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--/jbizhnpfyenlI1g
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZIBEQACgkQ24/THMrX
1yzF9Qf9GeFsU+7R7GJrRSdePb6LtqorKVJBaoq+4MHw7odCNjkqNuGIdI08Wx/b
hfH+h4+6qV3xE8MrgRrUHMdLSafCAhyn3wG306iNtoXXqzy0klNLfvv04s189Bry
9tkA1iJ9IqGiuSbTICQ0MGGCa7hUZPDD5aN67XZPng8lNCGEtDYQUFTGFaQ/M80G
sqSf0QdL0u9tna+4rEwZs970h/YZuGQbj1SitHGF1K95HCQJLs1NYxSTHUxoMMHZ
76Z0+Gjgm1bdc1tjnE+KeH6y75fu7M88G27wD7kMonqd/HBt1loLSBsHTKWOTO0e
84vmi0MIiOyt2ejJ9VYcnVQPsljCYQ==
=5/El
-----END PGP SIGNATURE-----

--/jbizhnpfyenlI1g--


From xen-devel-bounces@lists.xenproject.org Sat May 18 02:38:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 02:38:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724581.1129857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s89xN-0007Oe-Nm; Sat, 18 May 2024 02:37:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724581.1129857; Sat, 18 May 2024 02:37:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s89xN-0007OX-Ks; Sat, 18 May 2024 02:37:57 +0000
Received: by outflank-mailman (input) for mailman id 724581;
 Sat, 18 May 2024 02:37:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s89xM-0007ON-NX; Sat, 18 May 2024 02:37:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s89xM-00045l-Jz; Sat, 18 May 2024 02:37:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s89xM-0004B6-62; Sat, 18 May 2024 02:37:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s89xM-0007cV-5Y; Sat, 18 May 2024 02:37:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/VWctQSNF9yRo26yUi6HnQ9mnMFBLLPgnTJzCkmt4h0=; b=qO7uDj4IWEGLdFufKdXXHCQtrJ
	5KOkG1/9fa6z2VoBqtC+u5BzeH71MyaC1WGh7Y2DF3XMw4Elgd+xS0ZsfAfyw1X5E0BrrFKRopV9B
	p1AEG8N7GpBhMydy6pajyCdA4SawBg7/vCwtUE9S19G1eTjcvBDgRRcHTCcrMLeQm1A8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186030-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186030: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ea5f6ad9ad9645733b72ab53a98e719b460d36a6
X-Osstest-Versions-That:
    linux=3c999d1ae3c75991902a1a7dad0cb62c2a3008b4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 May 2024 02:37:56 +0000

flight 186030 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186030/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot         fail in 186023 pass in 186030
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 186023 pass in 186030
 test-armhf-armhf-xl-qcow2     8 xen-boot                   fail pass in 186023

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale   8 xen-boot            fail in 186023 like 186010
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail in 186023 like 186010
 test-armhf-armhf-xl           8 xen-boot            fail in 186023 like 186010
 test-armhf-armhf-xl-raw       8 xen-boot            fail in 186023 like 186010
 test-armhf-armhf-examine      8 reboot              fail in 186023 like 186018
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 186023 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 186023 never pass
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186023 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186023 never pass
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186010
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186018
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 186018
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186018
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186018
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186018
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                ea5f6ad9ad9645733b72ab53a98e719b460d36a6
baseline version:
 linux                3c999d1ae3c75991902a1a7dad0cb62c2a3008b4

Last test of basis   186018  2024-05-16 15:33:02 Z    1 days
Testing same since   186023  2024-05-17 01:13:59 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aapo Vienamo <aapo.vienamo@linux.intel.com>
  AceLan Kao <acelan.kao@canonical.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Ai Chao <aichao@kylinos.cn>
  Aleksandr Burakov <a.burakov@rosalinux.ru>
  Alex Volkov <alex@bootes.sytes.net>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Andreas Helbech Kleist <andreaskleist@gmail.com>
  Andrzej Pietrasiewicz <andrzej.p@collabora.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Andy Shevchenko <andy.shevchenko@gmail.com>
  Angelo Dureghello <adureghello@baylibre.com>
  Arend van Spriel <arend.vanspriel@broadcom.com>
  Armin Wolf <W_Armin@gmx.de>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Arvid Norlander <lkml@vorpal.se>
  Avri Altman <avri.altman@wdc.com>
  Basavaraj Natikar <Basavaraj.Natikar@amd.com>
  Bastien Curutchet <bastien.curutchet@bootlin.com>
  Ben Fradella <bfradell@netapp.com>
  Benjamin Gaignard <benjamin.gaignard@collabora.com>
  Bingbu Cao <bingbu.cao@intel.com>
  Bjorn Helgaas <bhelgaas@google.com>
  Bryan O'Donoghue <bryan.odonoghue@linaro.org>
  Changhuang Liang <changhuang.liang@starfivetech.com>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Dave Airlie <airlied@redhat.com>
  Dave Stevenson <dave.stevenson@raspberrypi.com>
  David E. Box <david.e.box@linux.intel.com>
  Denis Arefev <arefev@swemel.ru>
  Devarsh Thakkar <devarsht@ti.com>
  Dongliang Mu <mudongliangabcd@gmail.com>
  Erick Archer <erick.archer@outlook.com>
  Eugen Hristev <eugen.hristev@collabora.com>
  Fabio Estevam <festevam@denx.de>
  Fei Shao <fshao@chromium.org>
  Felix Qin <xiaokeqinhealth@126.com>
  Fullway Wang <fullwaywang@outlook.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Gergo Koteles <soyer@irl.hu>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  hanchunchao <hanchunchao@inspur.com>
  Hans de Goede <hdegoede@redhat.com>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Hien Huynh <hien.huynh.px@renesas.com>
  Hongju Wang <hongju.wang@intel.com>
  Ihor Matushchak <ihor.matushchak@foobox.net>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  István Váradi <ivaradi@varadiistvan.hu>
  Ivor Wanders <ivor@iwanders.net>
  Jackson Lee <jackson.lee@chipsnmedia.com>
  Jacopo Mondi <jacopo.mondi@ideasonboard.com>
  Jai Luthra <j-luthra@ti.com>
  Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
  Jeff Johnson <quic_jjohnson@quicinc.com>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jisheng Zhang <jszhang@kernel.org>
  Jithu Joseph <jithu.joseph@intel.com>
  Johnson Wang <johnson.wang@mediatek.com>
  Jonathan Bergh <bergh.jonathan@gmail.com>
  Jonathan Woithe <jwoithe@just42.net>
  Judith Mendez <jm@ti.com>
  Julia Lawall <Julia.Lawall@inria.fr>
  Julien Massot <julien.massot@collabora.com>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Kalle Valo <kvalo@kernel.org>
  Kate Hsuan <hpa@redhat.com>
  Kieran Bingham <kieran.bingham@ideasonboard.com>
  Klara Modin <klarasmodin@gmail.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee@kernel.org>
  Li Zhijian <lizhijian@fujitsu.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Luke D. Jones <luke@ljones.dev>
  Mark Pearson <mpearson-lenovo@squebb.ca>
  Martin Tůma <martin.tuma@digiteqautomotive.com>
  Matthias Schwarzott <zzam@gentoo.org>
  Matthijs Kooijman <matthijs@stdin.nl>
  Maulik Shah <quic_mkshah@quicinc.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Maxim Korotkov <korotkov.maxim.s@gmail.com>
  Maximilian Luz <luzmaximilian@gmail.com>
  Michael Bunk <micha@freedict.org>
  Michael Walle <mwalle@kernel.org>
  Miguel Ojeda <ojeda@kernel.org>
  Mikhail Rudenko <mike.rudenko@gmail.com>
  Minghao Chi <chi.minghao@zte.com.cn>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Mirela Rabulea <mirela.rabulea@nxp.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nathan Chancellor <nathan@kernel.org>
  Naushir Patuck <naush@raspberrypi.com>
  Nikita Travkin <nikita@trvn.ru>
  Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
  Nitin Joshi <njoshi1@lenovo.com>
  Oliver Neukum <oneukum@suse.com>
  Paweł Anikiel <panikiel@google.com>
  Prasad Pandit <pjp@fedoraproject.org>
  Pratyush Yadav <p.yadav@ti.com>
  Pratyush Yadav <pratyush@kernel.org>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rajeshwar R Shinde <coolrrsh@gmail.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ricardo B. Marliere <ricardo@marliere.net>
  Ricardo Ribalda <ribalda@chromium.org>
  Roman Bogoyev <roman@computercheck.com.au>
  Rouven Czerwinski <r.czerwinski@pengutronix.de>
  Sakari Ailus <sakari.ailus@linux.intel.com>
  Sean Young <sean@mess.org>
  Sebastian Fricke <sebastian.fricke@collabora.com>
  Sergey Khimich <serghox@gmail.com>
  Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
  Simon Glass <sjg@chromium.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
  Stefan Herdler <herdler@nurfuerspam.de>
  Stefan Klug <stefan.klug@ideasonboard.com>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Szilard Fabian <szfabian@bluemarch.art>
  Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Ulf Hansson <ulf.hansson@linaro.org>
  Umang Jain <umang.jain@ideasonboard.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vishnu Sankar <vishnuocv@gmail.com>
  Víctor Gonzalo <victor.gonzalo@anddroptable.net>
  Wadim Mueller <wafgo01@gmail.com>
  Weifeng Liu <weifeng.liu.z@gmail.com>
  Xi Pardee <xi.pardee@intel.com>
  yunshui <jiangyunshui@kylinos.cn>
  Zheyu Ma <zheyuma97@gmail.com>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   3c999d1ae3c7..ea5f6ad9ad96  ea5f6ad9ad9645733b72ab53a98e719b460d36a6 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat May 18 06:55:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 06:55:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724625.1129867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Dy3-0007Ed-7j; Sat, 18 May 2024 06:54:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724625.1129867; Sat, 18 May 2024 06:54:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Dy3-0007EW-4C; Sat, 18 May 2024 06:54:55 +0000
Received: by outflank-mailman (input) for mailman id 724625;
 Sat, 18 May 2024 06:54:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8Dy1-0007EM-M3; Sat, 18 May 2024 06:54:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8Dy1-0001M2-Im; Sat, 18 May 2024 06:54:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8Dy1-0005QE-6s; Sat, 18 May 2024 06:54:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8Dy1-0006fO-6P; Sat, 18 May 2024 06:54:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nCiZzV2pv9xyd5/klurkCdNdmDsvaTFdPRrdvwE5Jqs=; b=q3veCqcJH6QZj3EOHElRDaMR6F
	XuU0Cr/eBiaQFI9Z2uuBtvLvAXPam8ilhJL119xKfc3w3rmKHqU+gdrDzAqQuo2Vqi+DQvhxIc7zw
	61dWN74PF0XCSRes7MaX2LIF20I1TFxzp3rxqqvt/cHNFlN/gLe/Z3MtwqpQge1YIag4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186031-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186031: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
X-Osstest-Versions-That:
    xen=53dc37829c31edf02e8fc56aeccca8d60f6e2f5e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 May 2024 06:54:53 +0000

flight 186031 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186031/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186027
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186027
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186027
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186027
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186027
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186027
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca
baseline version:
 xen                  53dc37829c31edf02e8fc56aeccca8d60f6e2f5e

Last test of basis   186027  2024-05-17 07:07:37 Z    0 days
Testing same since   186031  2024-05-17 18:40:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   53dc37829c..21611c6870  21611c68702dae2e18cb519a6e166cdceeaff4ca -> master


From xen-devel-bounces@lists.xenproject.org Sat May 18 10:18:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 10:18:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724728.1129880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8H8f-0005yE-2S; Sat, 18 May 2024 10:18:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724728.1129880; Sat, 18 May 2024 10:18:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8H8e-0005y7-WB; Sat, 18 May 2024 10:18:04 +0000
Received: by outflank-mailman (input) for mailman id 724728;
 Sat, 18 May 2024 10:18:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8H8d-0005y1-Pm
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 10:18:03 +0000
Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com
 [2607:f8b0:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e8183941-14ff-11ef-909f-e314d9c70b13;
 Sat, 18 May 2024 12:18:02 +0200 (CEST)
Received: by mail-ot1-x330.google.com with SMTP id
 46e09a7af769-6f0ef6bee72so590521a34.0
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 03:18:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8183941-14ff-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716027481; x=1716632281; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=R1MdbXUXeDvckJAbagFPWhg2rYINBHeuo1GegK8vYnw=;
        b=lTiJPtmRWylMa3ZiQmzxWrSQaNN7uFNEzLBWbST6wJQjyIdT0AMst0NLNlgfl1ucc0
         ZBbUzCqylI58rA53ghRIc49rW/dZDdDaAEyqOqsRzcOaCppZ4iazxAcu/UzcfkOLchT0
         keKELxapqQ5fbXbK+MGQTGK8B99ZX/M6O0atm5Ccj45DZQCzg5cqzOKWFoiGKibgxedr
         yW7SrMlennAbEghdvK6VXXg2dryiEj9cDi4TCRm8lHYnLHeC8ljAmiIp0hYFpXFaMoa6
         prIZEoRxFOzwDwerKdWGTmDTLcpAZlypWJLk1QFLGRDaiarvBb3AdYV9wAHxAsvY7nfy
         m74A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716027481; x=1716632281;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=R1MdbXUXeDvckJAbagFPWhg2rYINBHeuo1GegK8vYnw=;
        b=qLPiLNiDLOiiK4hcFs5dzBtXReLuu9VPeuA08ScgHiIFlI4vzQJ/sx6CGFg7YL36jY
         sXNMLh177AigSr75QqTq44IiyRGN8Yg4eD3AtHTgJ1n0nJHjB7kvfLpiS5TWl9jzxvSL
         bYXnlRyWohnlBfOqGcDrhGYhSffR4v/B2gLqwvioc91FseClI8JrA8ovo4BOX1g3S+yF
         DWKIXgz4Ud/xxS3qyqQLi4epcyz9Lz1UUiVWYexAM3N4+WT5cDAFHycxtvbVtMaZ418r
         NyBr2GR49zz5noEgsLs9RTHAmSwnKjdVez/cX35JK6gFa1w+AZyQbbBur9NDvwkU3M+u
         4EWw==
X-Gm-Message-State: AOJu0YyJ8PEiMldaJzH0ju+ajj1XFJueh8PNIstrxk2ZNqLljZhlRoOV
	ml9eHL83rcGUrPMLICIowCSC3aOn5bimal3MxPhquOWLsFxAncWkUdv8E9nwBwsR6Wq/hkn/ZAv
	3X+XMsXLgPHJV1WwWjG6/F7XnmP0=
X-Google-Smtp-Source: AGHT+IHSpKUbM3Pc0cE7hQveNZdZbphdVrAuIJP9IQ1rBQuopVnrYRTYxVcI8uh3Dxn6FNHyUv4c6aAJ5vU23XyfI0U=
X-Received: by 2002:a05:6871:7246:b0:245:407f:96a6 with SMTP id
 586e51a60fabf-245408edb80mr8272524fac.36.1716027480867; Sat, 18 May 2024
 03:18:00 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1715858136.git.w1benny@gmail.com> <b6bf7e5ba0deab4550b9d15c327015fe7d94d463.1715858136.git.w1benny@gmail.com>
 <CABfawhmB+XND75WPJ_MjORyjUiV_h_NNmJQx2BaZ5WRmzkX79A@mail.gmail.com>
In-Reply-To: <CABfawhmB+XND75WPJ_MjORyjUiV_h_NNmJQx2BaZ5WRmzkX79A@mail.gmail.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Sat, 18 May 2024 12:17:49 +0200
Message-ID: <CAKBKdXh7rQ1x8vRZkXw1wC8M64Qk82mmz66XEa2gjjCZ+m7Arg@mail.gmail.com>
Subject: Re: [PATCH for-4.19? v3 4/6] x86: Make the maximum number of altp2m
 views configurable
To: Tamas K Lengyel <tamas@tklengyel.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, May 18, 2024 at 3:18=E2=80=AFAM Tamas K Lengyel <tamas@tklengyel.co=
m> wrote:
>
> > -        ap2m =3D array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> > +        ap2m =3D d->arch.altp2m_p2m[altp2m_idx];
>
> Why is it no longer necessary to use array_access_nospec?
>
> Tamas

I was under the impression that when the domain_create guarantees the
length of the array, and before each access to the altp2m_p2m there is
an explicit check if idx is within bounds of nr_altp2m, that the
array_access_nospec is not needed anymore. But apparently I
misunderstood how speculative execution works, when I now read about
it.

I'll put it back.

P.


From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724745.1129900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpf-0006DN-I7; Sat, 18 May 2024 11:02:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724745.1129900; Sat, 18 May 2024 11:02:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpf-0006DG-FZ; Sat, 18 May 2024 11:02:31 +0000
Received: by outflank-mailman (input) for mailman id 724745;
 Sat, 18 May 2024 11:02:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hpe-0005yr-GJ
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:30 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1e8082f4-1506-11ef-909f-e314d9c70b13;
 Sat, 18 May 2024 13:02:29 +0200 (CEST)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-4202cea9a2fso8231935e9.3
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:29 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e8082f4-1506-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030149; x=1716634949; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aWD/Qk3szSfEZBTtbpcC8O4b8u+9LVoa4Hx6fikLkdc=;
        b=X1CNpWfCnOd83w1vZKQTTdRnA5VbHtPh44DRpSWomUqxOkP6m9GaKnoeMjJM6HlvfJ
         mYguuaMxW5YyimRPhjvc/dN/esu7PlRLPyKxKO2Aom3ys2jTpsMYXMBBfJKF8f+hZn05
         +4QtZEpKYOfgnKPhFZch8ZiwSSlE70Ddm98W5HvRvbY76nNyV3qn2e53u+MxYf7nYt85
         3mxyhiwBeoTGywZMSHGnEGczNR7ZOq7CRMem6hVQmyBwbSoi+xYnfNZiSeCs8lFBKEF9
         TkafBtJHWM1sWAwON+ml5oiRG0g4mcNIH66SlD6Vtm6xOWwJ/Ntbju3ZzvFaHjFGTQlL
         LiKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030149; x=1716634949;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=aWD/Qk3szSfEZBTtbpcC8O4b8u+9LVoa4Hx6fikLkdc=;
        b=xUzQNnFpdMLzkOtbrj0LZv3+qhCAo1NhKWv0eLXmCvp2cPBpbGJhmXBiHMNQ3totdO
         EMJJ48pvM+L9GiU0yaZdK3GZ3k/L2d9nnkSVtyzUKZQJbTyjunTTNs7QBYWOSyS7vLGm
         sezK+0FoSVtAtAKiAiOJ4Hm0sk9heS2iqMQVZchnX3su70xpqSs3sg6M6xv36P4HJYfe
         Y0Xb7s5qhTXDtuf5HCYnc1v9p8PzN1+rtWOUxH6J0gmi3mAohXA1QZHKVT68YcA57Rlv
         5MJLGUjAwNtlymwl6AfurJazzFf9usm+SPACsN9sM+faf1vE9c8IQ+GARiFbGoxnWwoP
         CT7w==
X-Gm-Message-State: AOJu0YycUp0AUrVESppsweA41gtTCLdAkj82MAMRj8TrygI0g81azs+X
	tJ2W3gujR+bDLcVWZ+/keSIgG+18H3yPHaNdTOhMj2KyUWWAcxDSHjI3WQ==
X-Google-Smtp-Source: AGHT+IG7398q/5Qx6x7u3mnCKt2quXYSkSeMQG8C4wVwNPsRYjrqo+PixDsdfPG9NxBEx+6TwAKxZw==
X-Received: by 2002:a5d:53c5:0:b0:34d:a29c:b8b6 with SMTP id ffacd0b85a97d-3504a6311f6mr16471163f8f.24.1716030149141;
        Sat, 18 May 2024 04:02:29 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH for-4.19? v4 2/6] tools/xl: Add altp2m_count parameter
Date: Sat, 18 May 2024 11:02:13 +0000
Message-Id: <b4b7754bad90f234778b41f8227b50d1fb03f8f3.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Introduce a new altp2m_count parameter to control the maximum number of altp2m
views a domain can use. By default, if altp2m_count is unspecified and altp2m
is enabled, the value is set to 10, reflecting the legacy behavior.

This change is preparatory; it establishes the groundwork for the feature but
does not activate it.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/golang/xenlight/helpers.gen.go | 2 ++
 tools/golang/xenlight/types.gen.go   | 1 +
 tools/include/libxl.h                | 8 ++++++++
 tools/libs/light/libxl_create.c      | 9 +++++++++
 tools/libs/light/libxl_types.idl     | 1 +
 tools/xl/xl_parse.c                  | 4 ++++
 6 files changed, 25 insertions(+)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 78bdb08b15..40c247a0d0 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1158,6 +1158,7 @@ if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
 x.Altp2M = Altp2MMode(xc.altp2m)
+x.Altp2MCount = uint32(xc.altp2m_count)
 x.VmtraceBufKb = int(xc.vmtrace_buf_kb)
 if err := x.Vpmu.fromC(&xc.vpmu);err != nil {
 return fmt.Errorf("converting field Vpmu: %v", err)
@@ -1674,6 +1675,7 @@ if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
 xc.altp2m = C.libxl_altp2m_mode(x.Altp2M)
+xc.altp2m_count = C.uint32_t(x.Altp2MCount)
 xc.vmtrace_buf_kb = C.int(x.VmtraceBufKb)
 if err := x.Vpmu.toC(&xc.vpmu); err != nil {
 return fmt.Errorf("converting field Vpmu: %v", err)
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index ccfe18019e..a3ae8ef35a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -602,6 +602,7 @@ ArchX86 struct {
 MsrRelaxed Defbool
 }
 Altp2M Altp2MMode
+Altp2MCount uint32
 VmtraceBufKb int
 Vpmu Defbool
 }
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..1f149a8015 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1239,6 +1239,14 @@ typedef struct libxl__ctx libxl_ctx;
  */
 #define LIBXL_HAVE_ALTP2M 1

+/*
+ * LIBXL_HAVE_ALTP2M_COUNT
+ * If this is defined, then libxl supports setting the maximum number of
+ * alternate p2m tables.
+ */
+#define LIBXL_HAVE_ALTP2M_COUNT 1
+#define LIBXL_ALTP2M_COUNT_DEFAULT (~(uint32_t)0)
+
 /*
  * LIBXL_HAVE_REMUS
  * If this is defined, then libxl supports remus.
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 41252ec553..236b8c1965 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -483,6 +483,15 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
         return -ERROR_INVAL;
     }

+    if (b_info->altp2m_count == LIBXL_ALTP2M_COUNT_DEFAULT) {
+        if ((libxl_defbool_val(b_info->u.hvm.altp2m) ||
+            b_info->altp2m != LIBXL_ALTP2M_MODE_DISABLED))
+            /* Reflect the default legacy count */
+            b_info->altp2m_count = 10;
+        else
+            b_info->altp2m_count = 0;
+    }
+
     /* Assume that providing a bootloader user implies enabling restrict. */
     libxl_defbool_setdefault(&b_info->bootloader_restrict,
                              !!b_info->bootloader_user);
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..eb306fedf5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -728,6 +728,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     # Alternate p2m is not bound to any architecture or guest type, as it is
     # supported by x86 HVM and ARM support is planned.
     ("altp2m", libxl_altp2m_mode),
+    ("altp2m_count", uint32, {'init_val': 'LIBXL_ALTP2M_COUNT_DEFAULT'}),

     # Size of preallocated vmtrace trace buffers (in KBYTES).
     # Use zero value to disable this feature.
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..048ab6be0d 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2063,6 +2063,10 @@ void parse_config_data(const char *config_source,
         }
     }

+    if (!xlu_cfg_get_long(config, "altp2m_count", &l, 1)) {
+        b_info->altp2m_count = l;
+    }
+
     if (!xlu_cfg_get_long(config, "vmtrace_buf_kb", &l, 1) && l) {
         b_info->vmtrace_buf_kb = l;
     }
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724744.1129891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpe-0005z8-Ci; Sat, 18 May 2024 11:02:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724744.1129891; Sat, 18 May 2024 11:02:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpe-0005z1-8m; Sat, 18 May 2024 11:02:30 +0000
Received: by outflank-mailman (input) for mailman id 724744;
 Sat, 18 May 2024 11:02:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hpc-0005yr-Ua
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:28 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1d4542e5-1506-11ef-909f-e314d9c70b13;
 Sat, 18 May 2024 13:02:28 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-34d7d04808bso663375f8f.0
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:27 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d4542e5-1506-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030147; x=1716634947; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Ny2FsrepfKJ2y1XQxnkhlj9O8cTMpSVhjHMJ5QpUVps=;
        b=d/Amrz5IjOCZAGCZS17u7BIIYDTbQUoXuDkMVyP8cIeoyuXG5WHXB7a6Q1urZyLcYs
         fyd4qnYlriVbRWzDVq2TfVvLMf1rPa9YxqLmH3qEV3kF4l0hhDnF3T7f5RAXMIpxSMGn
         g3twKEbh3nlqWgCSuKQ72ADMtryciUwWzynXy6HUKwQkxDFvA1OsgatzGt7o94qnRpjh
         Kmbs1IquR+JzW0g2+5IVp4G5IDrCrDJCv3H5cRqjODGPgXbH6hWvFReSNbdpgaFpk19N
         amdWUDdFJaZM7wl3PrvzkXLibFX9g8c/EkyIf+Yfdjz7Bt1akbo1vfe1urDmspBypUQg
         qAvA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030147; x=1716634947;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Ny2FsrepfKJ2y1XQxnkhlj9O8cTMpSVhjHMJ5QpUVps=;
        b=OITzh5t3/0ziOSKEeBy81naG+1A/vch+A+1/xIOIK7nGPLKP66umfTCjsRDw2G9el1
         heK/SZVEabc4YOuFOBLoVV0vyjYcA0xEXd18I6T5r7Wt2V69Xkvo6btJjAMii3/1hXNe
         EgSB0q0mj3CtORWIKe8TgEFNAVzpieMEaTcKxd8wy8K/ZJW/mNHlTt+4MN/fMOke9quW
         0wlBJwgYjBKDx4CiueyQKTS2exMeCNcxD23rMFlycou6wuyaCWBMi1NXFNkRjub4bkMH
         +hj5wle+A8v3u3egz5/Zcj/bztjtCJI6NEtdmBN0zQDkYOWZ/Q1QW/f6bqo/9IBjhTKh
         c0Wg==
X-Gm-Message-State: AOJu0YyEvLbZNRaKu6/00Fau55dxEmK3DgjEfHKrqLv0QGtdDijMw+vY
	/2xk8NqU86++gRxH2OVKGy1aB5fQHWkMlWKe27YDw6WYi6iq1w2VWW+vqtPS
X-Google-Smtp-Source: AGHT+IHGIpNZMwjM0oinNCY7ljwWVnBRwhz7LmwvNHu6KdFVMr7iO+9BT/4+0cuCFkkcuFfmg/B3Qw==
X-Received: by 2002:adf:f8c4:0:b0:34f:5d07:ebda with SMTP id ffacd0b85a97d-3504a63111fmr25333962f8f.23.1716030146662;
        Sat, 18 May 2024 04:02:26 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>
Subject: [PATCH for-4.19? v4 0/6] x86: Make MAX_ALTP2M configurable
Date: Sat, 18 May 2024 11:02:11 +0000
Message-Id: <cover.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This series introduces the ability to configure the maximum number of altp2m
tables during domain creation. Previously, the limits were hardcoded to a
maximum of 10. This change allows for greater flexibility in environments that
require more or fewer altp2m views.

Adjustments include:
- "Prepare" commits with style changes.
- Adding a new `altp2m_count` parameter to xl.
- Adding a new `altp2m_count` parameter to the OCaml bindings.
- Adding a new `altp2m_count` parameter to the xl.cfg manual.
- Adding a new `nr_altp2m` field into the `xen_domctl_createdomain`, which,
  after sanity checks, is stored in newly introduced `nr_altp2m` field of
  `struct domain` - leaving room for other architectures to implement the
  altp2m feature.
- Replacing MAX_ALTP2M macro occurrences with `domain->nr_altp2m`.

This enhancement is particularly relevant for users leveraging Xen's features
for virtual machine introspection.

Changes since v2:
- Changed max_altp2m to nr_altp2m.
- Moved arch-dependent check from xen/common/domain.c to xen/arch/x86/domain.c.
- Replaced min(d->nr_altp2m, MAX_EPTP) occurences for just d->nr_altp2m.
- Replaced array_index_nospec(altp2m_idx, ...) for just altp2m_idx.
- Shortened long lines.
- Removed unnecessary comments in altp2m_vcpu_initialise/destroy.
- Moved nr_altp2m field after max_ fields in xen_domctl_createdomain.
- Removed the commit that adjusted the initial allocation of pages from 256
  to 1024. This means that after these patches, technically, the nr_altp2m will
  be capped to (256 - 1 - vcpus - MAX_NESTEDP2M) instead of MAX_EPTP (512).
  Future work will be needed to fix this.

Changes since v3:
- Rebased on top of staging (some functions were moved to altp2m.c).
- Re-added the array_index_nospec() where it was removed.

Petr Beneš (6):
  x86/p2m: Add braces for better code clarity
  tools/xl: Add altp2m_count parameter
  docs/man: Add altp2m_count parameter to the xl.cfg manual
  x86: Make the maximum number of altp2m views configurable
  tools/libxl: Activate the altp2m_count feature
  tools/ocaml: Add altp2m_count parameter

 docs/man/xl.cfg.5.pod.in             | 14 +++++
 tools/golang/xenlight/helpers.gen.go |  2 +
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/include/libxl.h                |  8 +++
 tools/libs/light/libxl_create.c      | 10 +++
 tools/libs/light/libxl_types.idl     |  1 +
 tools/ocaml/libs/xc/xenctrl.ml       |  1 +
 tools/ocaml/libs/xc/xenctrl.mli      |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c  | 11 ++--
 tools/xl/xl_parse.c                  |  4 ++
 xen/arch/x86/domain.c                | 12 ++++
 xen/arch/x86/hvm/hvm.c               |  8 ++-
 xen/arch/x86/hvm/vmx/vmx.c           |  2 +-
 xen/arch/x86/include/asm/domain.h    |  7 +--
 xen/arch/x86/include/asm/p2m.h       |  6 +-
 xen/arch/x86/mm/altp2m.c             | 91 +++++++++++++++++-----------
 xen/arch/x86/mm/hap/hap.c            |  6 +-
 xen/arch/x86/mm/mem_access.c         | 24 ++++----
 xen/arch/x86/mm/mem_sharing.c        |  2 +-
 xen/arch/x86/mm/p2m-ept.c            | 12 ++--
 xen/arch/x86/mm/p2m.c                | 12 ++--
 xen/common/domain.c                  |  1 +
 xen/include/public/domctl.h          |  5 +-
 xen/include/xen/sched.h              |  2 +
 24 files changed, 169 insertions(+), 74 deletions(-)

--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724746.1129911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpi-0006Sk-Qw; Sat, 18 May 2024 11:02:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724746.1129911; Sat, 18 May 2024 11:02:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpi-0006Sd-OA; Sat, 18 May 2024 11:02:34 +0000
Received: by outflank-mailman (input) for mailman id 724746;
 Sat, 18 May 2024 11:02:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hpg-0006PW-L8
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:32 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1dfccfdf-1506-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 13:02:30 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-351ae94323aso759108f8f.0
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:29 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1dfccfdf-1506-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030148; x=1716634948; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gyu7N26K83JM6reFihsC9/TI9yx+pCoPayJ3wbx78AU=;
        b=F6sP0e23rHsdYO0kY9SABSvwScPX0t11n6RRu6qJxZRmcou9W5PRZ/XUYz8Q+x6rqj
         cW+70kij1F3Wc1NWz/jwbNHwqt/ZAfAI42S3c8rUAj4te9hidu02XdqKVzneYe8I7XLm
         NHZgypm/dkd7DkWYies6hENXyh4kHRXFJfzoQ22k/hKRiv4K8kfQLjvX8fIee/CSAZCS
         sBosLzZ+W3i5seF1RY0iY0ElToxpuWlAJ3L2s1+cijFB5iXMFRQQQtp15BC6qqyNVjOs
         1+HnnZTNs3Isz8VpE8X/Iv7CZt0FzTT1I5NkhvbQIcwrxCBu4CQYxbWLcXnNoECY3Al6
         OF+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030148; x=1716634948;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gyu7N26K83JM6reFihsC9/TI9yx+pCoPayJ3wbx78AU=;
        b=m4iBH5wPhh4n1HlOW0TS0CrvURNrsyEfF0XXz3kBlM5KPhoo89ZcTvDLp5xQTWvCNv
         1eI2PSW+CCa3Y6tSzS7FRuPhIIcdp6t/isKZ3vifQcj5INEjJWu24XfjrQV9/fZ8FSWg
         0SBywn3c05n+/xB4JpA5eeIENSU0YSQHgb+2E08U2+PD7wlRxtVnXeRoC2VpyD4szB4/
         hJQRpsf6zqR9z/UzKJT04nqt9l/HnwvU8VlNuuljbQ9+SspUUpscFIMH5QUrEuSpIYoa
         TuBJmRhuqTo479lAGuhXNYpFCsqfByvHHZ1HY1htd7pM5qUX4rVMjfcI4ia/UycX/8p4
         /WYA==
X-Gm-Message-State: AOJu0Yz01+G6UmVULjMaOwq1dwL8lY7E9wfcpeycu8Txw11bodJDGBd3
	ZRfQFQb/S1u12pHqMOSBgnI0Ldg8gTcDNDVSzJqkm2sLOFIBGJ6yP5u8G7F1
X-Google-Smtp-Source: AGHT+IGe35sLDZcm+WO6wUrfzATgQqu1v8ygaKKDXUdhBEi8r8o+WYGxnnwQfFBbIh2r8glPt7nrKQ==
X-Received: by 2002:adf:ebc2:0:b0:34c:925e:6c2c with SMTP id ffacd0b85a97d-3504a61c6c2mr19059826f8f.6.1716030148024;
        Sat, 18 May 2024 04:02:28 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH for-4.19? v4 1/6] x86/p2m: Add braces for better code clarity
Date: Sat, 18 May 2024 11:02:12 +0000
Message-Id: <36d1488857a595fd6a286ab2de26cd672df3f387.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

No functional change.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/x86/mm/p2m.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 4a4620e870..db5d9b6c2a 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -106,6 +106,7 @@ void p2m_change_entry_type_global(struct domain *d,
         unsigned int i;

         for ( i = 0; i < MAX_ALTP2M; i++ )
+        {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -114,6 +115,7 @@ void p2m_change_entry_type_global(struct domain *d,
                 change_entry_type_global(altp2m, ot, nt);
                 p2m_unlock(altp2m);
             }
+        }
     }

     p2m_unlock(hostp2m);
@@ -139,6 +141,7 @@ void p2m_memory_type_changed(struct domain *d)
         unsigned int i;

         for ( i = 0; i < MAX_ALTP2M; i++ )
+        {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -147,6 +150,7 @@ void p2m_memory_type_changed(struct domain *d)
                 _memory_type_changed(altp2m);
                 p2m_unlock(altp2m);
             }
+        }
     }

     p2m_unlock(hostp2m);
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724748.1129922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpj-0006g6-I7; Sat, 18 May 2024 11:02:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724748.1129922; Sat, 18 May 2024 11:02:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpj-0006cP-Cr; Sat, 18 May 2024 11:02:35 +0000
Received: by outflank-mailman (input) for mailman id 724748;
 Sat, 18 May 2024 11:02:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hph-0005yr-ML
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:33 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2074f509-1506-11ef-909f-e314d9c70b13;
 Sat, 18 May 2024 13:02:33 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4202959b060so8032425e9.2
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:33 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2074f509-1506-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030152; x=1716634952; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lcXsJFjS9m5+2bzRAoHvtJjIOqkMU5GtwndPgJCwhX4=;
        b=GudoogxHMvdn4w0cqD5eTYm6th1gwGqv4gs9gLQ0mrO/5R7z0FK1O8tgF9pLN2rqYY
         1foSI5c7Su2FqiCj7Ee0xeV76j0alPFhUQS5em1KWrXTjPQBYtet64ycX1U3KiSDpSMQ
         wx4zEUAQSLvTlPSniIpfmTMAmyptDslAAMd+zqdyFkpMYkU8orb5jDGpxZaiEd/IKzzu
         zSdrX8n+I//YEa5P4GsdX20ZzQXaIbUGPzjjgTqYHlZRxavOHzLNX2PiLdafsRnsNkGr
         lRTIHsgj2WhZChYNvB1iQgzK15G5Qw3SSsIxsFc81ai6ps7d5SMyV3LzaLagSuzHixQk
         LWlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030152; x=1716634952;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=lcXsJFjS9m5+2bzRAoHvtJjIOqkMU5GtwndPgJCwhX4=;
        b=K+KBs7P3a2cOK3dwrgqjZTICWF5x2Wn5XSy0Wz2iutxak8D6ag18c2cGv2ugsja3VU
         26303llDZR2kjrhaPKCBDpDH84MogP3jqRHx2GSjYcUzJtu4SwNBsYmg9SYtJVTHpoma
         wMK7jMCqXzbVr1mbnyeHT8u/dgPSf/pzFTtMdf6ADVur15QZeF3QG1d2j2l9O5vRZd9d
         A5HmLlWHKsEc+jbbj6aHHXQqgVV9UtMuTVrOKsAwp0uVPM2D2nNw3GkAn5rJ80JuRvJn
         TAqKQ9ILhg4iAtH60P0CC7W1mfnKItkZdB/sfj0TL/v3ADT70J1q0asJz8as2eKhbZu7
         Cctw==
X-Gm-Message-State: AOJu0Yxh1unlnQGp9pTxbTZB2ebDkTBYbkuTEZbzUjLxKkkIp1p5tZND
	xaZ5Ivs8ip8A7sq4lqn15X0bvP57bNS5QZc7ZZO1g0KDlZ9K9sIQNoESXA==
X-Google-Smtp-Source: AGHT+IECOdi8/7PFZQliYUQDOmQMOAewH/Lgc3HHGhX0HkDS8B59my93i6l669Pnaf3F5kA5yxREEQ==
X-Received: by 2002:a05:600c:1f92:b0:420:29dd:84dd with SMTP id 5b1f17b1804b1-42029dd8865mr50068795e9.3.1716030152348;
        Sat, 18 May 2024 04:02:32 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH for-4.19? v4 5/6] tools/libxl: Activate the altp2m_count feature
Date: Sat, 18 May 2024 11:02:16 +0000
Message-Id: <b4ed18f65fb1b5d38a5ce4061cd8f4990598167a.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This commit activates the previously introduced altp2m_count parameter,
establishing the connection between libxl and Xen.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/libs/light/libxl_create.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 236b8c1965..f5eb16d0dc 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -657,6 +657,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
             .max_grant_frames = b_info->max_grant_frames,
             .max_maptrack_frames = b_info->max_maptrack_frames,
             .grant_opts = XEN_DOMCTL_GRANT_version(b_info->max_grant_version),
+            .nr_altp2m = b_info->altp2m_count,
             .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SHIFT),
             .cpupool_id = info->poolid,
         };
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724749.1129941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpk-0007AY-Sr; Sat, 18 May 2024 11:02:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724749.1129941; Sat, 18 May 2024 11:02:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpk-0007A6-NL; Sat, 18 May 2024 11:02:36 +0000
Received: by outflank-mailman (input) for mailman id 724749;
 Sat, 18 May 2024 11:02:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hpi-0006PW-Sr
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:35 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 202ab4ab-1506-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 13:02:32 +0200 (CEST)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-34da4d6f543so985408f8f.3
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:32 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 202ab4ab-1506-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030152; x=1716634952; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Y2wCtNW3QUhP9N3J/Fy1fQGx/Dx3sFuXd5cXXTLYbzg=;
        b=AqrQf68aw1lqxNfWfc2vdfltk9KFoGg9aWrLRM7Jkdn//+dk0FJHe09soCc2OoZite
         uUu3BOtwgPPo4BGwrW7h8OVtDFGipkFT/mUVOkBhSpappHs0TwZST6l9Ng1XOHiPsyxq
         TTbGvzz/AGhmKMZkuVZYh5Zr6uYPLMgL3UZo4/yxaiquBYwxKerhY8d226yNxhoY2nRR
         SRWbtdIsWknpIVWaPnvp/7QVyIjo6vM3FZ4xvdpKSYxECTfYCjvSR00K8xvNClfVzT3S
         BI09bz3ZuwRTjpf87OBXFcRnzTmn0EJXZ1yQ9OYkctWmfQhpDHRGWgRakLP6KNlFd4eh
         EFcA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030152; x=1716634952;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Y2wCtNW3QUhP9N3J/Fy1fQGx/Dx3sFuXd5cXXTLYbzg=;
        b=KRlCRj7rxjQRQXynWwzhCxR7zd8XFoPoTwPe1GjI+WIQXXFSdqkw7Opf+iXkZvUXUS
         48vpxVExlhXrBDQPFf4Ltnca4MGWHflWB5ZV4os3eYvbNFzA0Qiib1t2O+l0XL2nIHDR
         9uBgbtb773PJH0VYZsmuMXMzUZ3UnmyRfhNS/SFC+LroLerRs6y2TuOdW5viGQQrU7+G
         79zJktXtyIIi3VitcjQdjIMJj1KeJHTYsurpkL0TFp2TOHteY9KMgyQ53b3DiQ35LgEo
         TonME8n1Lhb17BjQfcY+T8R9zboGZj9xcFekFHdgKIMbw5326Tpku4eBfOy9w4Fd96/8
         fJGw==
X-Gm-Message-State: AOJu0YwdAH+Rx+CVbRamKMUnk5f4wsRoFCb8gui407NwxRUYlTscAWem
	zOCFQd0JB7Sk+JoDw9Ic/gNQL7OorlLaqvsDWIUTcBsLiIY66JBCLb/D0pry
X-Google-Smtp-Source: AGHT+IFWsbLQJAImnuNEzei7CHLPmV4kF3inVd/ch4Ctm8/zyEUhVkbFcmFZGwMuX75ux5aE3z34Ng==
X-Received: by 2002:adf:ecc9:0:b0:352:2700:3f34 with SMTP id ffacd0b85a97d-3522700401emr4171544f8f.32.1716030151388;
        Sat, 18 May 2024 04:02:31 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m views configurable
Date: Sat, 18 May 2024 11:02:15 +0000
Message-Id: <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This commit introduces the ability to configure the maximum number of altp2m
views for the domain during its creation. Previously, the limits were hardcoded
to a maximum of 10. This change allows for greater flexibility in environments
that require more or fewer altp2m views.

The maximum configurable limit for max_altp2m on x86 is now set to MAX_EPTP
(512). This cap is linked to the architectural limit of the EPTP-switching
VMFUNC, which supports up to 512 entries. Despite there being no inherent need
for limiting max_altp2m in scenarios not utilizing VMFUNC, decoupling these
components would necessitate substantial code changes.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 xen/arch/x86/domain.c             | 12 ++++
 xen/arch/x86/hvm/hvm.c            |  8 ++-
 xen/arch/x86/hvm/vmx/vmx.c        |  2 +-
 xen/arch/x86/include/asm/domain.h |  7 +--
 xen/arch/x86/include/asm/p2m.h    |  6 +-
 xen/arch/x86/mm/altp2m.c          | 91 +++++++++++++++++++------------
 xen/arch/x86/mm/hap/hap.c         |  6 +-
 xen/arch/x86/mm/mem_access.c      | 24 ++++----
 xen/arch/x86/mm/mem_sharing.c     |  2 +-
 xen/arch/x86/mm/p2m-ept.c         | 12 ++--
 xen/arch/x86/mm/p2m.c             |  8 +--
 xen/common/domain.c               |  1 +
 xen/include/public/domctl.h       |  5 +-
 xen/include/xen/sched.h           |  2 +
 14 files changed, 116 insertions(+), 70 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 00a3aaa576..3bd18cb2d0 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -685,6 +685,18 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }

+    if ( config->nr_altp2m && !hvm_altp2m_supported() )
+    {
+        dprintk(XENLOG_INFO, "altp2m requested but not available\n");
+        return -EINVAL;
+    }
+
+    if ( config->nr_altp2m > MAX_EPTP )
+    {
+        dprintk(XENLOG_INFO, "nr_altp2m must be <= %lu\n", MAX_EPTP);
+        return -EINVAL;
+    }
+
     if ( config->vmtrace_size )
     {
         unsigned int size = config->vmtrace_size;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c5..77e4016bdb 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4639,6 +4639,12 @@ static int do_altp2m_op(
         goto out;
     }

+    if ( d->nr_altp2m == 0 )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
     if ( (rc = xsm_hvm_altp2mhvm_op(XSM_OTHER, d, mode, a.cmd)) )
         goto out;

@@ -5228,7 +5234,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)
     if ( !hvm_is_singlestep_supported() )
         return;

-    if ( p2midx >= MAX_ALTP2M )
+    if ( p2midx >= v->domain->nr_altp2m )
         return;

     v->arch.hvm.single_step = true;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 5f67a48592..76ee09b701 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -4888,7 +4888,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
         {
             unsigned int i;

-            for ( i = 0; i < MAX_ALTP2M; ++i )
+            for ( i = 0; i < currd->nr_altp2m; ++i )
             {
                 if ( currd->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
                     continue;
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index f5daeb182b..3935328781 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -258,11 +258,10 @@ struct paging_vcpu {
     struct shadow_vcpu shadow;
 };

-#define MAX_NESTEDP2M 10
-
-#define MAX_ALTP2M      10 /* arbitrary */
 #define INVALID_ALTP2M  0xffff
 #define MAX_EPTP        (PAGE_SIZE / sizeof(uint64_t))
+#define MAX_NESTEDP2M   10
+
 struct p2m_domain;
 struct time_scale {
     int shift;
@@ -353,7 +352,7 @@ struct arch_domain

     /* altp2m: allow multiple copies of host p2m */
     bool altp2m_active;
-    struct p2m_domain *altp2m_p2m[MAX_ALTP2M];
+    struct p2m_domain **altp2m_p2m;
     mm_lock_t altp2m_list_lock;
     uint64_t *altp2m_eptp;
     uint64_t *altp2m_visible_eptp;
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 111badf89a..e66c081149 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -881,7 +881,7 @@ static inline struct p2m_domain *p2m_get_altp2m(struct vcpu *v)
     if ( index == INVALID_ALTP2M )
         return NULL;

-    BUG_ON(index >= MAX_ALTP2M);
+    BUG_ON(index >= v->domain->nr_altp2m);

     return v->domain->arch.altp2m_p2m[index];
 }
@@ -891,7 +891,7 @@ static inline bool p2m_set_altp2m(struct vcpu *v, unsigned int idx)
 {
     struct p2m_domain *orig;

-    BUG_ON(idx >= MAX_ALTP2M);
+    BUG_ON(idx >= v->domain->nr_altp2m);

     if ( idx == vcpu_altp2m(v).p2midx )
         return false;
@@ -943,7 +943,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
                                 p2m_type_t p2mt, p2m_access_t p2ma);

 /* Set a specific p2m view visibility */
-int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
+int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int idx,
                                    uint8_t visible);
 #else /* !CONFIG_HVM */
 struct p2m_domain *p2m_get_altp2m(struct vcpu *v);
diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
index 6fe1e9ed6b..5cb71c8b8e 100644
--- a/xen/arch/x86/mm/altp2m.c
+++ b/xen/arch/x86/mm/altp2m.c
@@ -15,6 +15,11 @@
 void
 altp2m_vcpu_initialise(struct vcpu *v)
 {
+    struct domain *d = v->domain;
+
+    if ( d->nr_altp2m == 0 )
+        return;
+
     if ( v != current )
         vcpu_pause(v);

@@ -30,8 +35,12 @@ altp2m_vcpu_initialise(struct vcpu *v)
 void
 altp2m_vcpu_destroy(struct vcpu *v)
 {
+    struct domain *d = v->domain;
     struct p2m_domain *p2m;

+    if ( d->nr_altp2m == 0 )
+        return;
+
     if ( v != current )
         vcpu_pause(v);

@@ -122,7 +131,12 @@ int p2m_init_altp2m(struct domain *d)
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);

     mm_lock_init(&d->arch.altp2m_list_lock);
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, d->nr_altp2m);
+
+    if ( !d->arch.altp2m_p2m )
+        return -ENOMEM;
+
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         d->arch.altp2m_p2m[i] = p2m = p2m_init_one(d);
         if ( p2m == NULL )
@@ -143,7 +157,10 @@ void p2m_teardown_altp2m(struct domain *d)
     unsigned int i;
     struct p2m_domain *p2m;

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    if ( !d->arch.altp2m_p2m )
+        return;
+
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( !d->arch.altp2m_p2m[i] )
             continue;
@@ -151,6 +168,8 @@ void p2m_teardown_altp2m(struct domain *d)
         d->arch.altp2m_p2m[i] = NULL;
         p2m_free_one(p2m);
     }
+
+    XFREE(d->arch.altp2m_p2m);
 }

 int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t *mfn,
@@ -200,7 +219,7 @@ bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
     struct domain *d = v->domain;
     bool rc = false;

-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= d->nr_altp2m )
         return rc;

     altp2m_list_lock(d);
@@ -306,8 +325,8 @@ static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
 {
     struct p2m_domain *p2m;

-    ASSERT(idx < MAX_ALTP2M);
-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ASSERT(idx < d->nr_altp2m);
+    p2m = d->arch.altp2m_p2m[idx];

     p2m_lock(p2m);

@@ -332,7 +351,7 @@ void p2m_flush_altp2m(struct domain *d)

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
         d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
@@ -348,9 +367,9 @@ static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
     struct p2m_domain *hostp2m, *p2m;
     int rc;

-    ASSERT(idx < MAX_ALTP2M);
+    ASSERT(idx < d->nr_altp2m);

-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    p2m = d->arch.altp2m_p2m[idx];
     hostp2m = p2m_get_hostp2m(d);

     p2m_lock(p2m);
@@ -388,12 +407,12 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
     int rc = -EINVAL;
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);

-    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( idx >= d->nr_altp2m )
         return rc;

     altp2m_list_lock(d);

-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
+    if ( d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
          mfn_x(INVALID_MFN) )
         rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);

@@ -415,7 +434,7 @@ int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             continue;
@@ -437,7 +456,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     struct p2m_domain *p2m;
     int rc = -EBUSY;

-    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( !idx || idx >= d->nr_altp2m )
         return rc;

     rc = domain_pause_except_self(d);
@@ -447,17 +466,17 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     rc = -EBUSY;
     altp2m_list_lock(d);

-    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
+    if ( d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] !=
          mfn_x(INVALID_MFN) )
     {
-        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+        p2m = d->arch.altp2m_p2m[array_index_nospec(idx, d->nr_altp2m)];

         if ( !_atomic_read(p2m->active_vcpus) )
         {
             p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
-            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =
+            d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] =
                 mfn_x(INVALID_MFN);
-            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP)] =
+            d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
                 mfn_x(INVALID_MFN);
             rc = 0;
         }
@@ -475,7 +494,7 @@ int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
     struct vcpu *v;
     int rc = -EINVAL;

-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= d->nr_altp2m )
         return rc;

     rc = domain_pause_except_self(d);
@@ -510,13 +529,13 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
     mfn_t mfn;
     int rc = -EINVAL;

-    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
+    if ( idx >= d->nr_altp2m ||
+         d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
          mfn_x(INVALID_MFN) )
         return rc;

     hp2m = p2m_get_hostp2m(d);
-    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ap2m = d->arch.altp2m_p2m[array_index_nospec(idx, d->nr_altp2m)];

     p2m_lock(hp2m);
     p2m_lock(ap2m);
@@ -572,7 +591,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         p2m_type_t t;
         p2m_access_t a;
@@ -595,7 +614,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
             else
             {
                 /* At least 2 altp2m's impacted, so reset everything */
-                for ( i = 0; i < MAX_ALTP2M; i++ )
+                for ( i = 0; i < d->nr_altp2m; i++ )
                 {
                     if ( i == last_reset_idx ||
                          d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
@@ -659,12 +678,13 @@ int p2m_set_suppress_ve_multi(struct domain *d,

     if ( sve->view > 0 )
     {
-        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
+        if ( sve->view >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[array_index_nospec(sve->view, d->nr_altp2m)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
+        p2m = ap2m =
+            d->arch.altp2m_p2m[array_index_nospec(sve->view, d->nr_altp2m)];
     }

     p2m_lock(host_p2m);
@@ -727,12 +747,13 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,

     if ( altp2m_idx > 0 )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = ap2m =
+            d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
     }
     else
         p2m = host_p2m;
@@ -754,7 +775,7 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
     return rc;
 }

-int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
+int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int idx,
                                    uint8_t visible)
 {
     int rc = 0;
@@ -763,17 +784,17 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,

     /*
      * Eptp index is correlated with altp2m index and should not exceed
-     * min(MAX_ALTP2M, MAX_EPTP).
+     * d->nr_altp2m.
      */
-    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+    if ( idx >= d->nr_altp2m ||
+         d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
          mfn_x(INVALID_MFN) )
         rc = -EINVAL;
     else if ( visible )
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
-            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)];
+        d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
+            d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)];
     else
-        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
+        d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
             mfn_x(INVALID_MFN);

     altp2m_list_unlock(d);
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index d2011fde24..501fd9848b 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -515,7 +515,7 @@ int hap_enable(struct domain *d, u32 mode)
             d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
         }

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             rv = p2m_alloc_table(d->arch.altp2m_p2m[i]);
             if ( rv != 0 )
@@ -538,7 +538,7 @@ void hap_final_teardown(struct domain *d)
     unsigned int i;

     if ( hvm_altp2m_supported() )
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
             p2m_teardown(d->arch.altp2m_p2m[i], true, NULL);

     /* Destroy nestedp2m's first */
@@ -590,7 +590,7 @@ void hap_teardown(struct domain *d, bool *preempted)
         FREE_XENHEAP_PAGE(d->arch.altp2m_eptp);
         FREE_XENHEAP_PAGE(d->arch.altp2m_visible_eptp);

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             p2m_teardown(d->arch.altp2m_p2m[i], false, preempted);
             if ( preempted && *preempted )
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 60a0cce68a..63bb2e10ed 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -347,12 +347,12 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, uint32_t nr,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;

-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
     }

     if ( !xenmem_access_to_p2m_access(p2m, access, &a) )
@@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
+             == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
     }

     p2m_lock(p2m);
@@ -466,12 +466,12 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access,
     }
     else if ( altp2m_idx ) /* altp2m view 0 is treated as the hostp2m */
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
-             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
-             mfn_x(INVALID_MFN) )
+        if ( altp2m_idx >= d->nr_altp2m ||
+             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
+             == mfn_x(INVALID_MFN) )
             return -EINVAL;

-        p2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
     }

     return _p2m_get_mem_access(p2m, gfn, access);
@@ -486,7 +486,7 @@ void arch_p2m_set_access_required(struct domain *d, bool access_required)
     if ( altp2m_active(d) )
     {
         unsigned int i;
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             struct p2m_domain *p2m = d->arch.altp2m_p2m[i];

diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index da28266ef0..83bb9dd5df 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -912,7 +912,7 @@ static int nominate_page(struct domain *d, gfn_t gfn,

         altp2m_list_lock(d);

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             ap2m = d->arch.altp2m_p2m[i];
             if ( !ap2m )
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index f83610cb8c..42b868ca45 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1293,7 +1293,7 @@ static void ept_set_ad_sync(struct domain *d, bool value)
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             struct p2m_domain *p2m;

@@ -1500,15 +1500,17 @@ void setup_ept_dump(void)

 void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
 {
-    struct p2m_domain *p2m = array_access_nospec(d->arch.altp2m_p2m, i);
+    struct p2m_domain *p2m =
+        d->arch.altp2m_p2m[array_index_nospec(i, d->nr_altp2m)];
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
     struct ept_data *ept;

     p2m->ept.ad = hostp2m->ept.ad;
     ept = &p2m->ept;
     ept->mfn = pagetable_get_pfn(p2m_get_pagetable(p2m));
-    d->arch.altp2m_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
-    d->arch.altp2m_visible_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
+    d->arch.altp2m_eptp[array_index_nospec(i, d->nr_altp2m)] = ept->eptp;
+    d->arch.altp2m_visible_eptp[array_index_nospec(i, d->nr_altp2m)] =
+        ept->eptp;
 }

 unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
@@ -1519,7 +1521,7 @@ unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)

     altp2m_list_lock(d);

-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < d->nr_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
             continue;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index db5d9b6c2a..549aec8d6b 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -105,7 +105,7 @@ void p2m_change_entry_type_global(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -140,7 +140,7 @@ void p2m_memory_type_changed(struct domain *d)
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -913,7 +913,7 @@ void p2m_change_type_range(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
@@ -986,7 +986,7 @@ int p2m_finish_type_change(struct domain *d,
     {
         unsigned int i;

-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < d->nr_altp2m; i++ )
         {
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6773f7fb90..a10a70e9d4 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -610,6 +610,7 @@ struct domain *domain_create(domid_t domid,
     if ( config )
     {
         d->options = config->flags;
+        d->nr_altp2m = config->nr_altp2m;
         d->vmtrace_size = config->vmtrace_size;
     }

diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..60a871f123 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -21,7 +21,7 @@
 #include "hvm/save.h"
 #include "memory.h"

-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017

 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -86,6 +86,9 @@ struct xen_domctl_createdomain {

     uint32_t grant_opts;

+    /* Number of altp2ms to allocate. */
+    uint32_t nr_altp2m;
+
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b841995..18cc0748a1 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -602,6 +602,8 @@ struct domain
         unsigned int guest_request_sync          : 1;
     } monitor;

+    unsigned int nr_altp2m;    /* Number of altp2m tables */
+
     unsigned int vmtrace_size; /* Buffer size in bytes, or 0 to disable. */

 #ifdef CONFIG_ARGO
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724750.1129947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpl-0007Jj-Fo; Sat, 18 May 2024 11:02:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724750.1129947; Sat, 18 May 2024 11:02:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpl-0007Ha-8C; Sat, 18 May 2024 11:02:37 +0000
Received: by outflank-mailman (input) for mailman id 724750;
 Sat, 18 May 2024 11:02:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hpj-0006PW-SN
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:35 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2123da9d-1506-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 13:02:34 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-420104e5336so5201145e9.1
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:34 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2123da9d-1506-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030153; x=1716634953; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CZx7e4I8f24AmBOZ73vCNLclq5iufdJNqbBIJFjDxwY=;
        b=MpZ2gj2nWq0O6DAu4jX0D5Y2ONcuKIHJdvbOoT7OAR1FtOgqJt1rYjzi4B3Xqc1yB6
         ayubHyvZbJUdFQQPzTw/sR63pwdz5S1tpSycvgknGIxgw6rCAOZnOy6PTyAbCEQKFiOn
         RvRIpzI0D+kJc8dkgphYfHnNDCr+q3fh0ss7xfFFrIHsJOY1AUXwQdF2V0wRebyg1Bdw
         E4CQANVArrsgkXGAEFKrnU/E4W7DuwmkpovyydON0PlN9QIrLqi2e2ktOlMvvMuaY3M4
         q0FKQ3Xq7uMGjr2fAsreOGP5HIEQZmUToHmbxCRVVh577rY9zN1VSd1u3dJ203opbUgH
         Z3qQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030153; x=1716634953;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=CZx7e4I8f24AmBOZ73vCNLclq5iufdJNqbBIJFjDxwY=;
        b=PiW1AkU29a61ho/LQ9a5mH/hC7oS3o5k24vTM9LoE4HWVGZmUMC0Ld8JfVA0TYi8Hp
         G9iDGZJncF74fvaiZqwaa4lCWbS8FIDqENvs5eEnYk0Bhiozc5ZKf3u0MlYemcgt5hfJ
         LhtvOgKbYdGAa/3PiBplHLWy48Kw/gMeQCY0kUw9ln/QvN7ju1Wm76gJMCaC3Js2dAq/
         +iSpEjVuKguruy3XhToEGUhN30QiUp/9b/T1eFA8QYA4njuNQxuCclyQhk/ZiLFfbP4G
         n/da1g6BuWpeIzOsExw/3hldI6zHipUbXYxjRJ8YmjR0BBb6sVxYcz4V30FAFmBA8tm5
         xWpA==
X-Gm-Message-State: AOJu0Yx1FUbQKrlPt60U0/qi9KXLwvyrvu8NkWtpEXiuEQP8NiJas2tf
	EhQ/BdwOBaYIdPMPOqXFMTTaHDLt+zZ/FZGkAKMHqU37F41pgzR+5n8Hfw==
X-Google-Smtp-Source: AGHT+IHsRTfJKAJsoyxs2Lhvbc+XDn+/W7DQC6ZXTI8u/8PQVwkGsEuGgUjdy6Bw0awD8RmAgnXg6w==
X-Received: by 2002:a05:600c:1c16:b0:418:bdcd:e59b with SMTP id 5b1f17b1804b1-420e19d5fa3mr13856515e9.7.1716030153510;
        Sat, 18 May 2024 04:02:33 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Christian Lindig <christian.lindig@cloud.com>
Subject: [PATCH for-4.19? v4 6/6] tools/ocaml: Add altp2m_count parameter
Date: Sat, 18 May 2024 11:02:17 +0000
Message-Id: <9ae1893beea2bb043413c014bd4adf6cbdab0aa7.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Allow developers using the OCaml bindings to set the altp2m_count parameter.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
---
 tools/ocaml/libs/xc/xenctrl.ml      |  1 +
 tools/ocaml/libs/xc/xenctrl.mli     |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c | 11 +++++++----
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index 55923857ec..dfb3d331c9 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -85,6 +85,7 @@ type domctl_create_config =
     max_grant_frames: int;
     max_maptrack_frames: int;
     max_grant_version: int;
+    altp2m_count: int;
     vmtrace_buf_kb: int32;
     cpupool_id: int32;
     arch: arch_domainconfig;
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 9b4b45db3a..ff0e309c56 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -77,6 +77,7 @@ type domctl_create_config = {
   max_grant_frames: int;
   max_maptrack_frames: int;
   max_grant_version: int;
+  altp2m_count: int;
   vmtrace_buf_kb: int32;
   cpupool_id: int32;
   arch: arch_domainconfig;
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index 2b6d3c09df..1f544cd2e4 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -210,9 +210,10 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
 #define VAL_MAX_GRANT_VERSION   Field(config, 8)
-#define VAL_VMTRACE_BUF_KB      Field(config, 9)
-#define VAL_CPUPOOL_ID          Field(config, 10)
-#define VAL_ARCH                Field(config, 11)
+#define VAL_ALTP2M_COUNT        Field(config, 9)
+#define VAL_VMTRACE_BUF_KB      Field(config, 10)
+#define VAL_CPUPOOL_ID          Field(config, 11)
+#define VAL_ARCH                Field(config, 12)

 	uint32_t domid = Int_val(wanted_domid);
 	uint64_t vmtrace_size = Int32_val(VAL_VMTRACE_BUF_KB);
@@ -230,6 +231,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 		.max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES),
 		.grant_opts =
 		    XEN_DOMCTL_GRANT_version(Int_val(VAL_MAX_GRANT_VERSION)),
+		.nr_altp2m = Int_val(VAL_ALTP2M_COUNT),
 		.vmtrace_size = vmtrace_size,
 		.cpupool_id = Int32_val(VAL_CPUPOOL_ID),
 	};
@@ -257,7 +259,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #if defined(__i386__) || defined(__x86_64__)

 		/* Quick & dirty check for ABI changes. */
-		BUILD_BUG_ON(sizeof(cfg) != 64);
+		BUILD_BUG_ON(sizeof(cfg) != 68);

         /* Mnemonics for the named fields inside xen_x86_arch_domainconfig */
 #define VAL_EMUL_FLAGS          Field(arch_domconfig, 0)
@@ -288,6 +290,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #undef VAL_ARCH
 #undef VAL_CPUPOOL_ID
 #undef VAL_VMTRACE_BUF_KB
+#undef VAL_ALTP2M_COUNT
 #undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 11:02:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 11:02:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724747.1129917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpj-0006W3-4s; Sat, 18 May 2024 11:02:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724747.1129917; Sat, 18 May 2024 11:02:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Hpi-0006VV-V1; Sat, 18 May 2024 11:02:34 +0000
Received: by outflank-mailman (input) for mailman id 724747;
 Sat, 18 May 2024 11:02:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v3Z6=MV=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1s8Hph-0006PW-9X
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 11:02:33 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1f11fff8-1506-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 13:02:30 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-3504ad82a64so739484f8f.3
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 04:02:30 -0700 (PDT)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b8969fbsm23777282f8f.37.2024.05.18.04.02.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 18 May 2024 04:02:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f11fff8-1506-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716030150; x=1716634950; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BlMJnuVPgAuWx5mzHbOSNLc3FPf2ltoLWAdUx3vSNgk=;
        b=gLTjy9Ns0nqu8PTPCDvsCJKYBBslocmjcdAYLbmwi4X7FfMVA7HDdlbZIxUU4uXKZn
         6HyBJ/Po6UPTRp90DXOpQ2Qjs39sXS1bUg6pEGh71WEC7YOIJxzkgw/5yIsO2ni/K9Hf
         pzf8SQ7fXHm5xtpf1T1mwjhozvhaKj2Q98frzIS+oh2L20tBge0mMJ0Iyxf0Us1bu7Ht
         JSFVSAVodCNFG2Bw9NfgEZraWxeQvqkHHVuwL9OMj25EkpBEVP95DVFJH7SE9fh3v6CB
         KXIWq4LrUZgQXoJzVV58QckXSMPeQ5SAZI07jTXgdcDk2MrzBB0RK3mKJfAlMPJ3zS6b
         kS4A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716030150; x=1716634950;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BlMJnuVPgAuWx5mzHbOSNLc3FPf2ltoLWAdUx3vSNgk=;
        b=ozckDJzIesOTB2XymCkxnKOENn8kbXGggmvU7JdZP1GvWsf1pT45P39pZPJa7QnkEQ
         /7N/FCLG/+Z07ly822P/D4TCOv3bljx59y8R/WSN9b/CgThYoHwHiyLW2Gnu3V9WfmTI
         k43qkvx4UP9FVOzNgelQaf0Ch8YnAiq34+uR3KWhCe9+5MQGcnXpARE3lT6E1XlsDbU6
         BgLDJKLDQSoigmnft4RU21/XjSVi5TQoQ6tZF0KPpSr/nPGNugSNA8AoeZjDqdyVMVwD
         eYlDNllO4U6g9GlzXaSBAM9XUKWOBCGYFpZjgV4x0F2AoYsFlXI8peAlWBHPfBuyThpe
         SGgw==
X-Gm-Message-State: AOJu0YwNQv0OtF2QrsTfuYHTcYl8rIa5wPzIMG2Bp69M8ItRKLurxVEd
	D6nZkb2pDkmyi7ybqWufAMKbIRDAJ5XMNAxY+UlltccGPETLfk5Gavbi1w==
X-Google-Smtp-Source: AGHT+IF3sugIpO35EWvY9cNYgcjkJaN2OWSjqd2R9sn79jz003ogS9QhQXo8F4pZmOtiVvARJnHCyw==
X-Received: by 2002:adf:e501:0:b0:34d:9e54:11ec with SMTP id ffacd0b85a97d-3504a61d24bmr16398583f8f.5.1716030150086;
        Sat, 18 May 2024 04:02:30 -0700 (PDT)
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH for-4.19? v4 3/6] docs/man: Add altp2m_count parameter to the xl.cfg manual
Date: Sat, 18 May 2024 11:02:14 +0000
Message-Id: <35afc0e34aae2672a924103f77a8fe62b791b45e.1716029860.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716029860.git.w1benny@gmail.com>
References: <cover.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Update manual pages to include detailed information about the altp2m_count
configuration parameter.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 docs/man/xl.cfg.5.pod.in | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..5c09610cf4 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2039,6 +2039,20 @@ a single guest HVM domain. B<This option is deprecated, use the option
 B<Note>: While the option "altp2mhvm" is deprecated, legacy applications for
 x86 systems will continue to work using it.

+=item B<altp2m_count=NUMBER>
+
+Specifies the maximum number of alternate-p2m views available to the guest.
+This setting is crucial in domain introspection scenarios that require
+multiple physical-to-machine (p2m) memory mappings to be established
+simultaneously.
+
+Enabling multiple p2m views may increase memory usage. It is advisable to
+review and adjust the B<shadow_memory> setting as necessary to accommodate
+the additional memory requirements.
+
+B<Note>: This option is ignored if B<altp2m> is disabled. The default value
+is 10.
+
 =item B<nestedhvm=BOOLEAN>

 Enable or disables guest access to hardware virtualisation features,
--
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat May 18 12:03:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 12:03:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724788.1129961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Imf-0004XH-Vv; Sat, 18 May 2024 12:03:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724788.1129961; Sat, 18 May 2024 12:03:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8Imf-0004XA-SL; Sat, 18 May 2024 12:03:29 +0000
Received: by outflank-mailman (input) for mailman id 724788;
 Sat, 18 May 2024 12:03:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=e6GZ=MV=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s8Ime-0004X2-DY
 for xen-devel@lists.xenproject.org; Sat, 18 May 2024 12:03:28 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a1dce1cf-150e-11ef-b4bb-af5377834399;
 Sat, 18 May 2024 14:03:26 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-571be483ccaso5800556a12.2
 for <xen-devel@lists.xenproject.org>; Sat, 18 May 2024 05:03:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1dce1cf-150e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716033806; x=1716638606; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gDgUUedlq+wHsurHoYWZ+h5FGofkrPEY4psJ90oQjvM=;
        b=m3xSkOfxG1sWL1FyTZR62FO6g6E2lrRakRFcU5BOzuzdpSmp5pVAjMez3b5wTyOFoN
         23E4+hyt0ZCo7KqKf/WDNp95rCBKLRCgR1b4BkFQOsRM715yWfoOCMeajDw6pzcdo8Rp
         +3UMz4O4tL7W21bp01saL04IqWKOpoUEOaJbSnS6Sd/6Lyh3Zei27sVb14DzDT4YJHIz
         AaldyDPkiqtzAqHwZcm8yLGVPmHbHrJVJ5RpEXUerUxB2/bWTMzjMnbF+kD2UNp4U5jf
         bnzq8Ik4mcSlAxJPk0CvuVXwhfgB4SqxC2AaSYJM9YGRlDxvsockVjPoPPo9vnTsXrCQ
         +2MA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716033806; x=1716638606;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gDgUUedlq+wHsurHoYWZ+h5FGofkrPEY4psJ90oQjvM=;
        b=wjq0fdG5V0UxBbae/Ko6QU2vX3p0ZYXGMcYCPpDkaULJiQZUjqL+2smX77rcTFo9hq
         yHZOeOK0l59j58RfbLTHcskbYPEKPDU2/45sfIzkeYOcsPnPyHozJLLFmbZgf3b/yxyz
         i4MLVzskeh274FUkgVimekHN42kK1bbQjtmiBBDUnndrX2h7wr5JV0tX5NWdO9aIcevb
         3KHttBM6prprJe1h1WsFVRBoI1T0BZtjISFJ00SZs5efnTAM81qkPSIo9ciZWe4b5OHA
         Gc1X/J1jsP0p2rVQvA4cNqbotZiroIgoc6LRgb/RuKYM0fBzKf2yIBfKXR9EH2iItAXy
         zHIQ==
X-Gm-Message-State: AOJu0YxN/UbpTzJml8ECNVW5Rm/LOr0s5f727oGYAtYWYT8aaPow7xDo
	j+GPSPJn+hOaPl+IsQRvjp/R0FEpwkglY+9Q9LqUtRn1IpdFq8Li9l0JDkDsN4DlAefcuIlujhd
	IobLrWNrUIHU0Lez5/gaYzIPv92uk4A==
X-Google-Smtp-Source: AGHT+IHaTJoKJIJ8/GHVEkMkHqTQxfgtLCCaacZHTEAmxDhn1JYC/8rUT+fuiknbF9yX8kKLX84YABTlF1KzE9hkvg0=
X-Received: by 2002:aa7:d74b:0:b0:574:b113:6677 with SMTP id
 4fb4d7f45d1cf-574b11366ecmr14327163a12.17.1716033805527; Sat, 18 May 2024
 05:03:25 -0700 (PDT)
MIME-Version: 1.0
References: <20240517140506.8460-1-leigh@solinno.co.uk> <20240517140506.8460-2-leigh@solinno.co.uk>
In-Reply-To: <20240517140506.8460-2-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sat, 18 May 2024 08:03:13 -0400
Message-ID: <CAKf6xpvprAScrAKN_s1K7EXc+mNXyvS7-3iaqhK8Cu=BKCYrEw@mail.gmail.com>
Subject: Re: [PATCH v4 1/2] tools/hotplug/Linux: Add bridge VLAN support
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Anthony Perard <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 17, 2024 at 10:05=E2=80=AFAM Leigh Brown <leigh@solinno.co.uk> =
wrote:
>
> Update add_to_bridge shell function to read the vlan parameter from
> xenstore and set the bridge VLAN configuration for the VID.
>
> Add additional helper functions to parse the vlan specification,
> which consists of one or more of the following:
>
> a) single VLAN (e.g. 10).
> b) contiguous range of VLANs (e.g. 10-15).
> c) discontiguous range with base, increment and count
>    (e.g. 100+10x9 which gives VLAN IDs 100, 110, ... 190).
>
> A single VLAN can be suffixed with "p" to indicate the PVID, or
> "u" to indicate untagged. A range of VLANs can be suffixed with
> "u" to indicate untagged.  A complex example would be:
>
>    vlan=3D1p/10-15/20-25u
>
> This capability requires the iproute2 bridge command to be
> installed.  An error will be generated if the vlan parameter is
> set and the bridge command is not available.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Sat May 18 13:11:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 13:11:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724807.1129971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8JqO-00007Y-RQ; Sat, 18 May 2024 13:11:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724807.1129971; Sat, 18 May 2024 13:11:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8JqO-00007R-OM; Sat, 18 May 2024 13:11:24 +0000
Received: by outflank-mailman (input) for mailman id 724807;
 Sat, 18 May 2024 13:11:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8JqN-00007H-I8; Sat, 18 May 2024 13:11:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8JqN-0008OU-F3; Sat, 18 May 2024 13:11:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8JqN-0001op-46; Sat, 18 May 2024 13:11:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8JqN-0006Zd-3Z; Sat, 18 May 2024 13:11:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Rc4Z650koDoYgpfRtthPj6ehV0FLskYUkf9R67qwZGE=; b=NkEuDsnB/CMCvGwt+xYPpF9MFP
	BEXxKr4uN4vkJzW7gKh+O8AdNmGD1tiZDixEVp+0ZXIy5ksU6HtJyu1Gd+EC5Vt3yYDioXvMz9VAA
	G280jiGQ54rijS+3oj+YBWj/RteUszQY+OrlTS6R5g2r6Cy5G+cRkrmPZnYJvTnscueU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186032-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186032: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b377b4868ef17b040065bd468668c707d2477a5
X-Osstest-Versions-That:
    linux=ea5f6ad9ad9645733b72ab53a98e719b460d36a6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 May 2024 13:11:23 +0000

flight 186032 linux-linus real [real]
flight 186034 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186032/
http://logs.test-lab.xenproject.org/osstest/logs/186034/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 186030

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 186023
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 186023
 test-armhf-armhf-xl-multivcpu  8 xen-boot                     fail like 186023
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186030
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186030
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186030
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186030
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4b377b4868ef17b040065bd468668c707d2477a5
baseline version:
 linux                ea5f6ad9ad9645733b72ab53a98e719b460d36a6

Last test of basis   186030  2024-05-17 11:41:19 Z    1 days
Testing same since   186032  2024-05-18 02:43:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexei Starovoitov <ast@kernel.org>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Donnellan <ajd@linux.ibm.com>
  Andrew Jeffery <andrew@codeconstruct.com.au>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Anup Patel <apatel@ventanamicro.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arun Ramadoss <arun.ramadoss@microchip.com>
  Baoquan He <bhe@redhat.com>
  Beau Belgrave <beaub@linux.microsoft.com>
  Benjamin Gray <bgray@linux.ibm.com>
  Bjorn Helgaas <bhelgaas@google.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chris Lew <quic_clew@quicinc.com>
  Chris Morgan <macromorgan@hotmail.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Christophe Leroy <christophe.leroy@csgroup.eu>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Bristot de Oliveira <bristot@kernel.org>
  Daniel Jurgens <danielj@nvidia.com>
  Dave Thaler <dthaler1968@gmail.com>
  David Hildenbrand <david@redhat.com>
  David S. Miller <davem@davemloft.net>
  Dharma Balasubiramani <dharma.b@microchip.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Dr. David Alan Gilbert <linux@treblig.org>
  Eric Dumazet <edumazet@google.com>
  Fan Yu <fan.yu9@zte.com.cn>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Frank Li <Frank.Li@nxp.com>
  Ganesh Goudar <ganeshgr@linux.ibm.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Geoff Levand <geoff@infradead.org>
  Ghanshyam Agrawal <ghanshyam1898@gmail.com>
  Greg Kurz <groug@kaod.org>
  Guo Ren <guoren@kernel.org>
  GUO Zihua <guozihua@huawei.com>
  Hangbin Liu <liuhangbin@gmail.com>
  Hao Ge <gehao@kylinos.cn>
  Hari Bathini <hbathini@linux.ibm.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Herve Codina <herve.codina@bootlin.com>
  Jakub Kicinski <kuba@kernel.org>
  Jason Wang <jasowang@redhat.com>
  Jay Vosburgh <jay.vosburgh@canonical.com>
  Jeffrey Hugo <quic_jhugo@quicinc.com>
  Jiawen Wu <jiawenwu@trustnetic.com>
  Jiri Olsa <jolsa@kernel.org>
  Joel Granados <j.granados@samsung.com>
  Joel Savitz <jsavitz@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John David Anglin <dave.anglin@bell.net>
  John Fastabend <john.fastabend@gmail.com>
  John Kacur <jkacur@redhat.com>
  Jonathan Haslam <jonathan.haslam@gmail.com>
  Kees Cook <keescook@chromium.org>
  Krzysztof Kozlowski <k.kozlowski@samsung.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kui-Feng Lee <thinker.li@gmail.com>
  Kumar Kartikeya Dwivedi <memxor@gmail.com>
  Kunwu Chan <chentao@kylinos.cn>
  Li Yang <leoyang.li@nxp.com>
  Lidong Zhong <lidong.zhong@suse.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Madhavan Srinivasan <maddy@linux.ibm.com>
  Mahesh Salgaonkar <mahesh@linux.ibm.com>
  Mark Brown <broonie@kernel.org>
  Martin KaFai Lau <martin.lau@kernel.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael S. Tsirkin <mst@redhat.com>
  Michal Schmidt <mschmidt@redhat.com>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nathan Chancellor <nathan@kernel.org>
  Nathan Lynch <nathanl@linux.ibm.com>
  Naveen N Rao <naveen@kernel.org>
  Nicholas Miehlbradt <nicholas@linux.ibm.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Oreoluwa Babatunde <quic_obabatun@quicinc.com>
  Paul E. McKenney <paulmck@kernel.org>
  Paul Mackerras <paulus@ozlabs.org>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Puranjay Mohan <puranjay@kernel.org>
  Rafał Miłecki <rafal@milecki.pl>
  Ran Wang <ran.wang_1@nxp.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rasmus Villemoes <linux@rasmusvillemoes.dk>
  Ravi Gunasekaran <r-gunasekaran@ti.com>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Roger Quadros <rogerq@kernel.org>
  Ronald Wahl <ronald.wahl@raritan.com>
  Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
  Sachin Sant <sachinp@linux.ibm.com>
  Sagar Cheluvegowda <quic_scheluve@quicinc.com>
  Saravana Kannan <saravanak@google.com>
  Sergey Shtylyov <s.shtylyov@omp.ru>
  Shirisha Ganta <shirisha@linux.ibm.com>
  Shresth Prasad <shresthprasad7@gmail.com>
  Shrikanth Hegde <sshegde@linux.ibm.com>
  Sourabh Jain <sourabhjain@linux.ibm.com>
  Stephen Brennan <stephen.s.brennan@oracle.com>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  sundar <prosunofficial@gmail.com>
  Suzuki K Poulose <suzuki.poulose@arm.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Blum <thorsten.blum@toblux.com>
  Tom Parkin <tparkin@katalix.com>
  Tony Battersby <tonyb@cybernetics.com>
  Vaibhav Jain <vaibhav@linux.ibm.com>
  Valentin Schneider <vschneid@redhat.com>
  Valentina Fernandez <valentina.fernandezalanis@microchip.com>
  Vincent Donnefort <vdonnefort@google.com>
  Vinod Koul <vkoul@kernel.org>
  Wadim Mueller <wafgo01@gmail.com>
  Xiaowei Bao <xiaowei.bao@nxp.com>
  xu xin <xu.xin16@zte.com.cn>
  Yang Li <yang.lee@linux.alibaba.com>
  Ye Bin <yebin10@huawei.com>
  Yuntao Wang <ytcoode@gmail.com>
  Yuran Pereira <yuran.pereira@hotmail.com>
  Zhao Chenhui <chenhui.zhao@freescale.com>
  Zheng Yejian <zhengyejian1@huawei.com>
  Zi Shen Lim <zlim.lnx@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 6589 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat May 18 15:56:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 15:56:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.724871.1129980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8MPt-0001rR-5S; Sat, 18 May 2024 15:56:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 724871.1129980; Sat, 18 May 2024 15:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8MPt-0001rK-2Z; Sat, 18 May 2024 15:56:13 +0000
Received: by outflank-mailman (input) for mailman id 724871;
 Sat, 18 May 2024 15:56:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8MPs-0001r6-28; Sat, 18 May 2024 15:56:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8MPr-0002rY-TZ; Sat, 18 May 2024 15:56:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8MPr-0008MD-Dm; Sat, 18 May 2024 15:56:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8MPr-0003ZQ-DL; Sat, 18 May 2024 15:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9gnk4GWRB11U1r2y+mpW9oL/pqtgrpK5nCGHfS7FTDw=; b=K1XYIOKIVjQXpuWGJkr18Col0j
	yXn3DXXWhCDuWv0K2Z/cIGMUmxNswtbJoqXks4d26JqLAeGdFhwnx+3cuDJnlpoMfJvaMGVnEj53k
	qGrBdyMiLLjEGuhBnl/LAVGikt7XN8SJP0n16bssW2Dh9tAo/9fU7T3k0ihUhDQ0MH08=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186033-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186033: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
X-Osstest-Versions-That:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 May 2024 15:56:11 +0000

flight 186033 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186033/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186031
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186031
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186031
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186031
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186031
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186031
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca
baseline version:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca

Last test of basis   186033  2024-05-18 06:59:05 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sat May 18 20:59:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 May 2024 20:59:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725424.1129990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8R8u-0001qT-Qx; Sat, 18 May 2024 20:59:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725424.1129990; Sat, 18 May 2024 20:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8R8u-0001qM-Nk; Sat, 18 May 2024 20:59:00 +0000
Received: by outflank-mailman (input) for mailman id 725424;
 Sat, 18 May 2024 20:59:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8R8u-0001qA-5y; Sat, 18 May 2024 20:59:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8R8u-0000Ip-0h; Sat, 18 May 2024 20:59:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8R8t-00082e-HT; Sat, 18 May 2024 20:58:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8R8t-0003jP-Gz; Sat, 18 May 2024 20:58:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4H6989aQieXF+nrsWtzEFFcAb9LwQSq9pMnbrbYNIZo=; b=xQyyOoTHcT/D4u1hlXd4V/+/ih
	8Lugpa+I7UYGoQHuv4AAOrHh4qSnfjjPK04cOh7FriTGI9L9ieYBgyRilqHTaEVNpCeLAZkxhbwwu
	epoNEdmVh3trmP3kxyWrcyetYK01lp2/jAGh5I5xoj9RrwyAFyOUwkhvJDhIDdT1aiXw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186035-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186035: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b377b4868ef17b040065bd468668c707d2477a5
X-Osstest-Versions-That:
    linux=ea5f6ad9ad9645733b72ab53a98e719b460d36a6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 May 2024 20:58:59 +0000

flight 186035 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186035/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186030
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186030
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186030
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186030
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186030
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186030
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4b377b4868ef17b040065bd468668c707d2477a5
baseline version:
 linux                ea5f6ad9ad9645733b72ab53a98e719b460d36a6

Last test of basis   186030  2024-05-17 11:41:19 Z    1 days
Testing same since   186032  2024-05-18 02:43:32 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexei Starovoitov <ast@kernel.org>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Donnellan <ajd@linux.ibm.com>
  Andrew Jeffery <andrew@codeconstruct.com.au>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Anup Patel <apatel@ventanamicro.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arun Ramadoss <arun.ramadoss@microchip.com>
  Baoquan He <bhe@redhat.com>
  Beau Belgrave <beaub@linux.microsoft.com>
  Benjamin Gray <bgray@linux.ibm.com>
  Bjorn Helgaas <bhelgaas@google.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chris Lew <quic_clew@quicinc.com>
  Chris Morgan <macromorgan@hotmail.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Christophe Leroy <christophe.leroy@csgroup.eu>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Bristot de Oliveira <bristot@kernel.org>
  Daniel Jurgens <danielj@nvidia.com>
  Dave Thaler <dthaler1968@gmail.com>
  David Hildenbrand <david@redhat.com>
  David S. Miller <davem@davemloft.net>
  Dharma Balasubiramani <dharma.b@microchip.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Dr. David Alan Gilbert <linux@treblig.org>
  Eric Dumazet <edumazet@google.com>
  Fan Yu <fan.yu9@zte.com.cn>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Frank Li <Frank.Li@nxp.com>
  Ganesh Goudar <ganeshgr@linux.ibm.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Geoff Levand <geoff@infradead.org>
  Ghanshyam Agrawal <ghanshyam1898@gmail.com>
  Greg Kurz <groug@kaod.org>
  Guo Ren <guoren@kernel.org>
  GUO Zihua <guozihua@huawei.com>
  Hangbin Liu <liuhangbin@gmail.com>
  Hao Ge <gehao@kylinos.cn>
  Hari Bathini <hbathini@linux.ibm.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Herve Codina <herve.codina@bootlin.com>
  Jakub Kicinski <kuba@kernel.org>
  Jason Wang <jasowang@redhat.com>
  Jay Vosburgh <jay.vosburgh@canonical.com>
  Jeffrey Hugo <quic_jhugo@quicinc.com>
  Jiawen Wu <jiawenwu@trustnetic.com>
  Jiri Olsa <jolsa@kernel.org>
  Joel Granados <j.granados@samsung.com>
  Joel Savitz <jsavitz@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John David Anglin <dave.anglin@bell.net>
  John Fastabend <john.fastabend@gmail.com>
  John Kacur <jkacur@redhat.com>
  Jonathan Haslam <jonathan.haslam@gmail.com>
  Kees Cook <keescook@chromium.org>
  Krzysztof Kozlowski <k.kozlowski@samsung.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kui-Feng Lee <thinker.li@gmail.com>
  Kumar Kartikeya Dwivedi <memxor@gmail.com>
  Kunwu Chan <chentao@kylinos.cn>
  Li Yang <leoyang.li@nxp.com>
  Lidong Zhong <lidong.zhong@suse.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Madhavan Srinivasan <maddy@linux.ibm.com>
  Mahesh Salgaonkar <mahesh@linux.ibm.com>
  Mark Brown <broonie@kernel.org>
  Martin KaFai Lau <martin.lau@kernel.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael S. Tsirkin <mst@redhat.com>
  Michal Schmidt <mschmidt@redhat.com>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nathan Chancellor <nathan@kernel.org>
  Nathan Lynch <nathanl@linux.ibm.com>
  Naveen N Rao <naveen@kernel.org>
  Nicholas Miehlbradt <nicholas@linux.ibm.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Oreoluwa Babatunde <quic_obabatun@quicinc.com>
  Paul E. McKenney <paulmck@kernel.org>
  Paul Mackerras <paulus@ozlabs.org>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Puranjay Mohan <puranjay@kernel.org>
  Rafał Miłecki <rafal@milecki.pl>
  Ran Wang <ran.wang_1@nxp.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rasmus Villemoes <linux@rasmusvillemoes.dk>
  Ravi Gunasekaran <r-gunasekaran@ti.com>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Roger Quadros <rogerq@kernel.org>
  Ronald Wahl <ronald.wahl@raritan.com>
  Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
  Sachin Sant <sachinp@linux.ibm.com>
  Sagar Cheluvegowda <quic_scheluve@quicinc.com>
  Saravana Kannan <saravanak@google.com>
  Sergey Shtylyov <s.shtylyov@omp.ru>
  Shirisha Ganta <shirisha@linux.ibm.com>
  Shresth Prasad <shresthprasad7@gmail.com>
  Shrikanth Hegde <sshegde@linux.ibm.com>
  Sourabh Jain <sourabhjain@linux.ibm.com>
  Stephen Brennan <stephen.s.brennan@oracle.com>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  sundar <prosunofficial@gmail.com>
  Suzuki K Poulose <suzuki.poulose@arm.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Blum <thorsten.blum@toblux.com>
  Tom Parkin <tparkin@katalix.com>
  Tony Battersby <tonyb@cybernetics.com>
  Vaibhav Jain <vaibhav@linux.ibm.com>
  Valentin Schneider <vschneid@redhat.com>
  Valentina Fernandez <valentina.fernandezalanis@microchip.com>
  Vincent Donnefort <vdonnefort@google.com>
  Vinod Koul <vkoul@kernel.org>
  Wadim Mueller <wafgo01@gmail.com>
  Xiaowei Bao <xiaowei.bao@nxp.com>
  xu xin <xu.xin16@zte.com.cn>
  Yang Li <yang.lee@linux.alibaba.com>
  Ye Bin <yebin10@huawei.com>
  Yuntao Wang <ytcoode@gmail.com>
  Yuran Pereira <yuran.pereira@hotmail.com>
  Zhao Chenhui <chenhui.zhao@freescale.com>
  Zheng Yejian <zhengyejian1@huawei.com>
  Zi Shen Lim <zlim.lnx@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   ea5f6ad9ad96..4b377b4868ef  4b377b4868ef17b040065bd468668c707d2477a5 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun May 19 03:48:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 03:48:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725475.1130001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8XWu-0002mu-HT; Sun, 19 May 2024 03:48:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725475.1130001; Sun, 19 May 2024 03:48:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8XWu-0002mn-EU; Sun, 19 May 2024 03:48:12 +0000
Received: by outflank-mailman (input) for mailman id 725475;
 Sun, 19 May 2024 03:48:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8XWt-0002md-Lv; Sun, 19 May 2024 03:48:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8XWt-00071t-HO; Sun, 19 May 2024 03:48:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8XWt-0005Or-8x; Sun, 19 May 2024 03:48:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8XWt-0002uR-8X; Sun, 19 May 2024 03:48:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TQGtvF3l0vOMFJ4D1X+WNzClK8hzBRgAHr6JVi3m+Vk=; b=YkLj+WL5mLVGf395eHUuNgX6fq
	D9FopGPKtwVEroi97mQRUuAVSHVzJw81/x1/uWHtCGcTBziAPQdIxFnvTMVLaOVEiqqcnJXAYhwi0
	YCwlOy6uUUjR8OfjakmfKa8MnXM7f3ovMh0OiGXKI/ZjyEeWMJFRuQnfX7nLaIBuuHjk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186036-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186036: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
X-Osstest-Versions-That:
    linux=4b377b4868ef17b040065bd468668c707d2477a5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 May 2024 03:48:11 +0000

flight 186036 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186036/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186035
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186035
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186035
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186035
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186035
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186035
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f
baseline version:
 linux                4b377b4868ef17b040065bd468668c707d2477a5

Last test of basis   186035  2024-05-18 13:41:54 Z    0 days
Testing same since   186036  2024-05-18 21:13:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abel Vesa <abel.vesa@linaro.org>
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Alexandre Mergnat <amergnat@baylibre.com>
  Andrew Davis <afd@ti.com>
  André Draszik <andre.draszik@linaro.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Babis Chalios <bchalios@amazon.es>
  Bagas Sanjaya <bagasdotme@gmail.com>
  Binbin Zhou <zhoubinbin@loongson.cn>
  Bjorn Andersson <andersson@kernel.org>
  Bob Pearson <rpearsonhpe@gmail.com>
  Boshi Yu <boshiyu@linux.alibaba.com>
  Breno Leitao <leitao@debian.org>
  Catalin Marinas <catalin.marinas@arm.com>
  Catalin Popescu <catalin.popescu@leica-geosystems.com>
  Chengchang Tang <tangchengchang@huawei.com>
  Chiara Meiohas <cmeiohas@nvidia.com>
  Christian Marangi <ansuelsmth@gmail.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
  Colin Ian King <colin.i.king@gmail.com>
  Cong Dang <cong.dang.xn@renesas.com>
  Conor Dooley <conor.dooley@microchip.com>
  Cristian Marussi <cristian.marussi@arm.com>
  David Jander <david@protonic.nl>
  David Rientjes <rientjes@google.com>
  Dimitri Sivanich <sivanich@hpe.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Rokosov <ddrokosov@salutedevices.com>
  Emil Renner Berthing <emil.renner.berthing@canonical.com>
  Erick Archer <erick.archer@outlook.com>
  Fabio Estevam <festevam@denx.de>
  Frank Oltmanns <frank@oltmanns.dev>
  Gabor Juhos <j4g8y7@gmail.com>
  Gabriel Fernandez <gabriel.fernandez@foss.st.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Georgi Djakov <quic_c_gdjako@quicinc.com>
  Günther Noack <gnoack@google.com>
  Hanjun Guo <guohanjun@huawei.com>
  Heiko Stuebner <heiko@sntech.de>
  Helge Deller <deller@gmx.de>
  Herman van Hazendonk <github.com@herrie.org>
  Huacai Chen <chenhuacai@loongson.cn>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Inochi Amaoto <inochiama@outlook.com>
  Ivanov Mikhail <ivanov.mikhail1@huawei-partners.com>
  Jaewon Kim <jaewon02.kim@samsung.com>
  Jakub Kicinski <kuba@kernel.org>
  Jason A. Donenfeld <Jason@zx2c4.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Jens Axboe <axboe@kernel.dk>
  Jernej Skrabec <jernej.skrabec@gmail.com>
  Jerome Brunet <jbrunet@baylibre.com>
  Jingqi Liu <Jingqi.liu@intel.com>
  Joerg Roedel <jroedel@suse.de>
  John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
  Jules Irenge <jbi.octave@gmail.com>
  Junxian Huang <huangjunxian6@hisilicon.com>
  Kathiravan Thirumoorthy <quic_kathirav@quicinc.com>
  Klara Modin <klarasmodin@gmail.com>
  Konstantin Taranov <kotaranov@microsoft.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Leon Romanovsky <leon@kernel.org>
  Leon Romanovsky <leonro@nvidia.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Lu Baolu <baolu.lu@linux.intel.com>
  Luca Weiss <luca.weiss@fairphone.com>
  Luca Weiss <luca@z3ntu.xyz>
  Lucas Stach <l.stach@pengutronix.de>
  Marc Gonzalez <mgonzalez@freebox.fr>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Michael Guralnik <michaelgur@nvidia.com>
  Michael Margolin <mrgolin@amazon.com>
  Michael Shavit <mshavit@google.com>
  Michal Schmidt <mschmidt@redhat.com>
  Mickaël Salaün <mic@digikod.net>
  Nathan Chancellor <nathan@kernel.org>
  Neil Armstrong <neil.armstrong@linaro.org>
  Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-HDK
  Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-HDK
  Nick Desaulniers <ndesaulniers@google.com> # Boot
  Nicolin Chen <nicolinc@nvidia.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Or Har-Toov <ohartoov@nvidia.com>
  Pasha Tatashin <pasha.tatashin@soleen.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Peng Fan <peng.fan@nxp.com>
  Peter Griffin <peter.griffin@linaro.org>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Roberto Sassu <roberto.sassu@huawei.com>
  Robin Murphy <robin.murphy@arm.com>
  Sam Protsenko <semen.protsenko@linaro.org>
  Sascha Hauer <s.hauer@pengutronix.de>
  Satya Priya Kakitapalli <quic_skakitap@quicinc.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Selvin Xavier <selvin.xavier@broadcom.com>
  Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
  Shengjiu Wang <shengjiu.wang@nxp.com>
  Shreeya Patel <shreeya.patel@collabora.com>
  Stephen Boyd <sboyd@kernel.org>
  Sudan Landge <sudanl@amazon.com>
  Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
  Thanh Le <thanh.le.xv@renesas.com>
  Thanh Quan <thanh.quan.xn@renesas.com>
  Thierry Reding <treding@nvidia.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Thorsten Leemhuis <linux@leemhuis.info>
  Tina Zhang <tina.zhang@intel.com>
  Tudor Ambarus <tudor.ambarus@linaro.org>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uros Bizjak <ubizjak@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vasant Hegde <vasant.hegde@amd.com>
  Vladimir Zapolskiy <vz@mleia.com>
  Wang Yao <wangyao@lemote.com>
  Wei Huang <wei.huang2@amd.com>
  Wei Lei <quic_leiwei@quicinc.com>
  Wenchao Hao <haowenchao2@huawei.com>
  wenglianfa <wenglianfa@huawei.com>
  Will Deacon <will@kernel.org>
  Yangyang Li <liyangyang20@huawei.com>
  Yi Liu <yi.l.liu@intel.com>
  Zhengchao Shao <shaozhengchao@huawei.com>
  Zhengping Zhang <zhengping.zhang@airoha.com>
  Zhu Yanjun <yanjun.zhu@linux.dev>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4b377b4868ef..25f4874662fb  25f4874662fb0d43fc1d934dd7802b740ed2ab5f -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun May 19 08:09:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 08:09:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725511.1130010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8bbx-00034k-QR; Sun, 19 May 2024 08:09:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725511.1130010; Sun, 19 May 2024 08:09:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8bbx-00034d-Ng; Sun, 19 May 2024 08:09:41 +0000
Received: by outflank-mailman (input) for mailman id 725511;
 Sun, 19 May 2024 08:09:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8bbw-00034T-88; Sun, 19 May 2024 08:09:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8bbw-0003xW-4Y; Sun, 19 May 2024 08:09:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8bbv-0001GE-OE; Sun, 19 May 2024 08:09:39 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8bbv-0007BS-Nc; Sun, 19 May 2024 08:09:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SyoB1JXTFYwEA0US5+mzSpdXNCFKLYxu8nrtwo/lg5k=; b=qdljeIMaZps2F/N1qwPBxreUtm
	/hp4g2yNhTSUa63Ir3FPDA6IaaWM3Ygn6fPvusZhWsnjyRXRLylbbAv03H3tWjOk+NEc8b8yinX0l
	ySJY9+9jkYGyx/g/XRSuin9HNPpxBBwAoNz+yOteU236AGUblIu2uWihFih/Yce6snIE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186037-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186037: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
X-Osstest-Versions-That:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 May 2024 08:09:39 +0000

flight 186037 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186037/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1   8 xen-boot                   fail pass in 186033

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186033 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186033 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186033
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186033
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186033
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186033
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186033
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186033
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca
baseline version:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca

Last test of basis   186037  2024-05-19 01:53:41 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun May 19 10:04:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 10:04:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725551.1130021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8dOj-0002ne-H9; Sun, 19 May 2024 10:04:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725551.1130021; Sun, 19 May 2024 10:04:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8dOj-0002nX-E2; Sun, 19 May 2024 10:04:09 +0000
Received: by outflank-mailman (input) for mailman id 725551;
 Sun, 19 May 2024 10:04:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s8dOi-0002nR-AV
 for xen-devel@lists.xenproject.org; Sun, 19 May 2024 10:04:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8dOi-0005qo-4i; Sun, 19 May 2024 10:04:08 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8dOh-00055q-Vm; Sun, 19 May 2024 10:04:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=17Umg97djKNIGcrRa6KjlmrflguEWj93dNOJwvfqHIk=; b=bbCnZVmjCZ4f1Czf1F/qenyL3l
	JWB6pKN5S+1vCVeEqUhht2IB1CsYM9uPyndsVxT2i8Zji1qdRdh6fflJk04E+0RKJnVza4RWNoFwm
	Qq47dF1q1T5NZ68t/XvVmNFA6aL0WqDlqJQOsW/VIa7+iWkkK7PdEq/WkSr9bTABiJgQ=;
Message-ID: <c1ff249f-bd00-4b15-8e25-2ead4fc5e852@xen.org>
Date: Sun, 19 May 2024 11:04:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/8] xen/common/dt-overlay: Fix lock issue when
 add/remove the device
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-2-xin.wang2@amd.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240516100330.1433265-2-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 16/05/2024 11:03, Henry Wang wrote:
> If CONFIG_DEBUG=y, below assertion will be triggered:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
> [...]
> (XEN) Xen call trace:
> (XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
> (XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
> (XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
> (XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
> (XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
> (XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
> (XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
> (XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
> (XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
> (XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
> (XEN) ****************************************
> 
> This is because iommu_remove_dt_device() is called without taking the
> dt_host_lock. dt_host_lock is meant to ensure that the DT node will not
> disappear behind back. So fix the issue by taking the lock as soon as
> getting hold of overlay_node.
> 
> Similar issue will be observed in adding the dtbo:
> (XEN) Assertion 'system_state < SYS_STATE_active || rw_is_locked(&dt_host_lock)'
> failed at xen-source/xen/drivers/passthrough/device_tree.c:192
> (XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
> [...]
> (XEN) Xen call trace:
> (XEN)    [<00000a00002594f4>] iommu_add_dt_device+0x7c/0x17c (PC)
> (XEN)    [<00000a0000259494>] iommu_add_dt_device+0x1c/0x17c (LR)
> (XEN)    [<00000a0000267db4>] handle_device+0x68/0x1e8
> (XEN)    [<00000a0000208ba8>] dt_overlay_sysctl+0x9d4/0xb84
> (XEN)    [<00000a000027342c>] arch_do_sysctl+0x24/0x38
> (XEN)    [<00000a0000231ac8>] do_sysctl+0x9ac/0xa34
> (XEN)    [<00000a0000274b70>] traps.c#do_trap_hypercall+0x230/0x2dc
> (XEN)    [<00000a0000276330>] do_trap_guest_sync+0x478/0x688
> (XEN)    [<00000a000025e480>] entry.o#guest_sync_slowpath+0xa8/0xd8
> 
> This is because the lock is released too early. So fix the issue by
> releasing the lock after handle_device().
> 
> Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun May 19 10:17:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 10:17:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725561.1130032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8dc2-0004ev-L6; Sun, 19 May 2024 10:17:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725561.1130032; Sun, 19 May 2024 10:17:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8dc2-0004eo-H0; Sun, 19 May 2024 10:17:54 +0000
Received: by outflank-mailman (input) for mailman id 725561;
 Sun, 19 May 2024 10:17:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s8dc1-0004ei-CH
 for xen-devel@lists.xenproject.org; Sun, 19 May 2024 10:17:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8dc1-00063B-39; Sun, 19 May 2024 10:17:53 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8dc0-0005kW-TI; Sun, 19 May 2024 10:17:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=PYc/LpTwVSpraHnYT7Wlk5KHA645IoK/9ZUMvQHzb2g=; b=R/C+cxggq4O8UAV6bIoXH+LcQ3
	iItQg44EVm/DohbcRJBAplY3eNxV33JTFCnltqr15j9id4xZgimQCqSm0VCHOyfzg5lHgm5NJb/Ue
	TlF1TGWG5xg98LeAqaNRiUrev+oeVIU4TzdS9GD3BwHp3akF/W+JDXxXX4vbXvNZDe24=;
Message-ID: <01825e63-3cf1-4539-85e4-d87fecbcfcf4@xen.org>
Date: Sun, 19 May 2024 11:17:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU
 for Dom0less domUs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-4-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240516100330.1433265-4-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 16/05/2024 11:03, Henry Wang wrote:
> There are some use cases in which the dom0less domUs need to have
> the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
> example, the dynamic dtbo feature allows the domain to be assigned
> a device that is behind the IOMMU at runtime. For these use cases,
> we need to have a way to specify the domain will need the IOMMU
> mapping at domain construction time.
> 
> Introduce a "passthrough" DT property for Dom0less DomUs following
> the same entry as the xl.cfg. Currently only provide two options,
> i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
> construction time based on the property.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - New patch to replace the original patch in v1:
>    "[PATCH 03/15] xen/arm: Always enable IOMMU"
> ---
>   docs/misc/arm/device-tree/booting.txt | 13 +++++++++++++
>   xen/arch/arm/dom0less-build.c         |  7 +++++--
>   2 files changed, 18 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index bbd955e9c2..61f9082553 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -260,6 +260,19 @@ with the following properties:
>       value specified by Xen command line parameter gnttab_max_maptrack_frames
>       (or its default value if unspecified, i.e. 1024) is used.
>   
> +- passthrough
> +
> +    A string property specifying whether IOMMU mappings are enabled for the
> +    domain and hence whether it will be enabled for passthrough hardware.
> +    Possible property values are:
> +
> +    - "enabled"
> +    IOMMU mappings are enabled for the domain.
> +
> +    - "disabled"
> +    IOMMU mappings are disabled for the domain and so hardware may not be
> +    passed through. This option is the default if this property is missing.

Looking at the code below, it seems like the default will depend on 
whether the partial device-tree is present. Did I misunderstand?

> +
>   Under the "xen,domain" compatible node, one or more sub-nodes are present
>   for the DomU kernel and ramdisk.
>   
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..1396a102e1 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -848,6 +848,7 @@ static int __init construct_domU(struct domain *d,
>   void __init create_domUs(void)
>   {
>       struct dt_device_node *node;
> +    const char *dom0less_iommu;
>       const struct dt_device_node *cpupool_node,
>                                   *chosen = dt_find_node_by_path("/chosen");
>   
> @@ -895,8 +896,10 @@ void __init create_domUs(void)
>               panic("Missing property 'cpus' for domain %s\n",
>                     dt_node_name(node));
>   
> -        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
> -             iommu_enabled )
> +        if ( iommu_enabled &&
> +             ((!dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
> +               !strcmp(dom0less_iommu, "enabled")) ||
> +              dt_find_compatible_node(node, NULL, "multiboot,device-tree")) )

This condition is getting a little bit harder to read. Can we cache the 
"passthrough" flag separately?

Also, shouldn't we throw a panic if passthrough = "enabled" but the 
IOMMU is enabled?

>               d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>   
>           if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun May 19 11:08:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 11:08:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725577.1130041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8eOi-0003PE-Uw; Sun, 19 May 2024 11:08:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725577.1130041; Sun, 19 May 2024 11:08:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8eOi-0003P7-S8; Sun, 19 May 2024 11:08:12 +0000
Received: by outflank-mailman (input) for mailman id 725577;
 Sun, 19 May 2024 11:08:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s8eOh-0003P1-D0
 for xen-devel@lists.xenproject.org; Sun, 19 May 2024 11:08:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8eOh-0006rh-2W; Sun, 19 May 2024 11:08:11 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8eOg-0008QL-Ry; Sun, 19 May 2024 11:08:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=nvX8Y+OXmTeJ95NkoYRik+bu86UoFfz7aKvwtb4C8XI=; b=3uKHlZrBbVqQIVTUhhoOHjxH2j
	LaiGpBgjUAxsWiYST6p9T30E+XVc625oEtlbAXjm5WdLie4Xo/lPGmPI76HX36IukH7ZP+bjtsdc9
	+PqL0hcRnahNhrjXzWN75Zc6PtKRJcsQWDDhNgtJHnSoLZakGaYMLLDg+99x/MZy58j0=;
Message-ID: <9f086470-a17f-482a-ad98-814424da0ee5@xen.org>
Date: Sun, 19 May 2024 12:08:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-6-xin.wang2@amd.com>
 <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 17/05/2024 07:03, Henry Wang wrote:
> 
> 
> On 5/16/2024 6:03 PM, Henry Wang wrote:
>> From: Vikram Garhwal <fnu.vikram@xilinx.com>
>>
>> Currently, routing/removing physical interrupts are only allowed at
>> the domain creation/destroy time. For use cases such as dynamic device
>> tree overlay adding/removing, the routing/removing of physical IRQ to
>> running domains should be allowed.
>>
>> Removing the above-mentioned domain creation/dying check. Since this
>> will introduce interrupt state unsync issues for cases when the
>> interrupt is active or pending in the guest, therefore for these cases
>> we simply reject the operation. Do it for both new and old vGIC
>> implementations.
>>
>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> ---
>> v2:
>> - Reject the case where the IRQ is active or pending in guest.
>> ---
>>   xen/arch/arm/gic-vgic.c  |  8 ++++++--
>>   xen/arch/arm/gic.c       | 15 ---------------
>>   xen/arch/arm/vgic/vgic.c |  5 +++--
>>   3 files changed, 9 insertions(+), 19 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>> index 56490dbc43..d1608415f8 100644
>> --- a/xen/arch/arm/gic-vgic.c
>> +++ b/xen/arch/arm/gic-vgic.c
>> @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, struct 
>> vcpu *v, unsigned int virq,
>>       {
>>           /* The VIRQ should not be already enabled by the guest */

This comment needs to be updated.

>>           if ( !p->desc &&
>> -             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>> +             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
>> +             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
>> +             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
>>               p->desc = desc;
>>           else
>>               ret = -EBUSY;
>>       }
>>       else
>>       {
>> -        if ( desc && p->desc != desc )
>> +        if ( desc && p->desc != desc &&
>> +             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
>> +              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )
> 
> This should be
> 
> +        if ( (desc && p->desc != desc) ||
> +             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
> +             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
Looking at gic_set_lr(), we first check p->desc, before setting 
IRQ_GUEST_VISIBLE.

I can't find a common lock, so what would guarantee that p->desc is not 
going to be used or IRQ_GUEST_VISIBLE set afterwards?



>> @@ -887,7 +887,8 @@ int vgic_connect_hw_irq(struct domain *d, struct 
>> vcpu *vcpu,
>>       }
>>       else                                /* remove a mapped IRQ */
>>       {
>> -        if ( desc && irq->hwintid != desc->irq )
>> +        if ( desc && irq->hwintid != desc->irq &&
>> +             (irq->active || irq->pending_latch) )
> 
> Same here, this should be
> 
> +        if ( (desc && irq->hwintid != desc->irq) ||
> +             irq->active || irq->pending_latch )

I think the new vGIC doesn't have the same problem because it looks like 
the IRQ lock will be taken for any access to 'irq'.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun May 19 11:38:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 11:38:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725591.1130050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8ert-00086m-97; Sun, 19 May 2024 11:38:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725591.1130050; Sun, 19 May 2024 11:38:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8ert-00086f-68; Sun, 19 May 2024 11:38:21 +0000
Received: by outflank-mailman (input) for mailman id 725591;
 Sun, 19 May 2024 11:38:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s8err-00086Y-Pf
 for xen-devel@lists.xenproject.org; Sun, 19 May 2024 11:38:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8err-0007KV-Ci; Sun, 19 May 2024 11:38:19 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8err-0001Sa-5L; Sun, 19 May 2024 11:38:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EiDRjJxzgRWuRqpy1Hv/AAFEHRo2iP6ZmpPuMy+bNjk=; b=Ad7WzhkCE8R2+pJHV+zh7GDfVb
	anPUfIbsagYETTU1iutR8ZxZHEF9HV128t+CjinwP0MXQj3xxFWeLmp2gPd9zxAfEDAIfJA94cOoF
	1+NZTOn4KDIoRhh1t9ITNzERfQ1tSJY50W4En2xEoLjeMKEbaWoZt/AVF6SK6/1ps3TI=;
Message-ID: <10e1ec4f-d15f-4d24-9542-b3fd8b642b79@xen.org>
Date: Sun, 19 May 2024 12:38:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH v1] xen: arm: Set state to SYS_STATE_smp_boot before
 smp initialization
Content-Language: en-GB
To: Ayan Kumar Halder <ayan.kumar.halder@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com,
 Volodymyr_Babchuk@epam.com, dpsmith@apertussolutions.com
References: <20240320185455.3409182-1-ayan.kumar.halder@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240320185455.3409182-1-ayan.kumar.halder@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Ayan,

Sorry this patch fell through the cracks.

On 20/03/2024 18:54, Ayan Kumar Halder wrote:
> This is similar to what is being done on x86.


OOI, how did you find this issue?

> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>


> ---
>   xen/arch/arm/setup.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 424744ad5e..3d110685bc 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -787,6 +787,8 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset+
>   
>       processor_id();
>   
> +    system_state = SYS_STATE_smp_boot;
> +

This is not quite the samem position as x86. The system state will be 
updated just before do_presmp_initcalls().

Can you explain the difference?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun May 19 13:38:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 13:38:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725625.1130061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8gjl-0006yc-EB; Sun, 19 May 2024 13:38:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725625.1130061; Sun, 19 May 2024 13:38:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8gjl-0006yV-Bd; Sun, 19 May 2024 13:38:05 +0000
Received: by outflank-mailman (input) for mailman id 725625;
 Sun, 19 May 2024 13:38:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8gjk-0006yL-IT; Sun, 19 May 2024 13:38:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8gjk-00010w-GC; Sun, 19 May 2024 13:38:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8gjk-0000jn-3E; Sun, 19 May 2024 13:38:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8gjk-0001be-2o; Sun, 19 May 2024 13:38:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xzM5mLJKPjUTiWwPKsm8HBQPY8t6drNFoIjBeFNaUpg=; b=hxu0Yyyh7JGmqJY7FFsWq8qVdo
	+qEblLFsIStOWtfWxw3FahHuXdw+jqi4zWOTpuyUR0GwXvKWNO+njEfzzVNToaK7OoZs5XfX1QaR2
	ANHMtsuAEl25NVM+j6Z4cxIND2NdMHNYTdI7Y6r0Rp2EzC0Pq7f/zEtJvUu/pSPMqi7Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186038-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186038: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-qcow2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0450d2083be6bdcd18c9535ac50c55266499b2df
X-Osstest-Versions-That:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 May 2024 13:38:04 +0000

flight 186038 linux-linus real [real]
flight 186039 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186038/
http://logs.test-lab.xenproject.org/osstest/logs/186039/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   8 xen-boot                 fail REGR. vs. 186036

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qcow2 21 guest-start/debian.repeat fail pass in 186039-retest
 test-armhf-armhf-xl           8 xen-boot            fail pass in 186039-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 186039-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 186036

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 186039 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 186039 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186039 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186039 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186036
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186036
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186036
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0450d2083be6bdcd18c9535ac50c55266499b2df
baseline version:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f

Last test of basis   186036  2024-05-18 21:13:52 Z    0 days
Testing same since   186038  2024-05-19 03:51:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Baokun Li <libaokun1@huawei.com>
  Christoph Hellwig <hch@lst.de>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Disha Goel <disgoel@linux.ibm.com>
  Guenter Roeck <linux@roeck-us.net>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  Jan Kara <jack@suse.cz>
  Jeff Layton <jlayton@kernel.org>
  Justin Stitt <justinstitt@google.com>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Li kunyu <kunyu@nfschina.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Luis Henriques <lhenriques@suse.de>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Michael Forney <mforney@mforney.org>
  NeilBrown <neilb@suse.de>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Stephen Smalley <stephen.smalley.work@gmail.com>
  Steve French <stfrench@microsoft.com>
  Theodore Ts'o <tytso@mit.edu>
  Thorsten Blum <thorsten.blum@toblux.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Vladimir Benes <vbenes@redhat.com>
  Ye Bin <yebin10@huawei.com>
  Zhang Yi <yi.zhang@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    fail    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1600 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun May 19 14:54:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 14:54:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725666.1130070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8hvs-0000Nj-Pt; Sun, 19 May 2024 14:54:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725666.1130070; Sun, 19 May 2024 14:54:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8hvs-0000Nc-ND; Sun, 19 May 2024 14:54:40 +0000
Received: by outflank-mailman (input) for mailman id 725666;
 Sun, 19 May 2024 14:54:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2wXv=MW=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1s8hvr-0000NU-Ud
 for xen-devel@lists.xenproject.org; Sun, 19 May 2024 14:54:39 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b6c58a25-15ef-11ef-909f-e314d9c70b13;
 Sun, 19 May 2024 16:54:38 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-571ba432477so7217463a12.1
 for <xen-devel@lists.xenproject.org>; Sun, 19 May 2024 07:54:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6c58a25-15ef-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716130477; x=1716735277; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Y1IbXT6s+TDvE5Z/rvchiR3mdtk/BquEIjBA3qg8iRs=;
        b=Qfs7EQZFGCn6Ms7QiFD4Bl9voW35iA1qFYUNsEiTjS5aJ/GniyWIkUlICKP4jxb6H3
         GcldIlhQbv/C+yQeJrygqDAHHTPgikSSLiafNH/buQZBPdRcY+6gRuA7gvP2tQ+06iOK
         H/aMoGatSTHuui6Avu5ZI45SZsA83k53WlMBvl1PaIV1JlfLDeUBP+hv0KuDxLamK/R/
         iX1Jcr0eTLAZkWAA/dw2GNrBDI/5bvKaBS7LiDN6zXj6F4XpGZeDwRr3zknm/aypdxsu
         kGDqLXhZXQWv71DFqY26pZx7sgtbHDa0BlaPRWkIgHgbdffzvOlyEx3aNsObniuJrgOh
         uAUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716130477; x=1716735277;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Y1IbXT6s+TDvE5Z/rvchiR3mdtk/BquEIjBA3qg8iRs=;
        b=iUehUYfP989h4Onov9C7KF4lPO7gV6odIbVzN4x0otIM0iuafaSc0DAUjXayGHT+CS
         J7qy9cUt+JFrCI8z+vy12WZOgPpw3ik0pQbC75JqZEik5R5/Uyt+tfUt3IlHvgmzSOCK
         CotFkvouY4yCbMdqrnJd3Tl28TzO5pFhI8ALF0nZyy6blo5ppWSFAmBWw5GYc1eREV8x
         cakBhsvV2fEiKx+VXZJpYT+tP7jmVZxxNC5veVpInqQeHPYXg92ItK8ercakW5lidObs
         Y54OlkbTMRPSlEuw3rvgynbS/9JxxDpzyTXwTkaeeseAv519rP9WOq91k06617qNx/K/
         QI6g==
X-Gm-Message-State: AOJu0Yxdf9JVXeZgyhp1PjJGr3NyXPLCdI4LEp8hrAiQmraP4eRacz7R
	+maNeEO8zqsbNuWlToBvYZGNfVnXn6+QTJYK/U2PVnJhgT+9L0vE/hm9U6/eGC6BsBCQCaDBfP4
	KDK5MDnx1NBlIcjxMFyeqdkzhyVc=
X-Google-Smtp-Source: AGHT+IF6SVaVB8uwtc+vUaOu52y3ePi8uNN3pRmTrw7uUZ0C5e6Lxu9ZcS62VS+ukK9f05SJNv8dd3AfxKp2PkaQA3Q=
X-Received: by 2002:a50:9ec5:0:b0:568:8e22:4eff with SMTP id
 4fb4d7f45d1cf-5734d6df0d5mr17248259a12.37.1716130477350; Sun, 19 May 2024
 07:54:37 -0700 (PDT)
MIME-Version: 1.0
References: <20240517140506.8460-1-leigh@solinno.co.uk> <20240517140506.8460-3-leigh@solinno.co.uk>
In-Reply-To: <20240517140506.8460-3-leigh@solinno.co.uk>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 19 May 2024 10:54:25 -0400
Message-ID: <CAKf6xpsHhe1v1evHOaRE_EBqAnC3vN2URsXkUwTu1dzWXP6BHQ@mail.gmail.com>
Subject: Re: [PATCH v4 2/2] tools/examples: Example Linux bridge VLAN config
To: Leigh Brown <leigh@solinno.co.uk>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Anthony PERARD <anthony.perard@cloud.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 17, 2024 at 10:05=E2=80=AFAM Leigh Brown <leigh@solinno.co.uk> =
wrote:
>
> Add a new directory linux-bridge-vlan with example files showing
> how to configure systemd-networkd to support a bridge VLAN
> configuration.
>
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

The systemd network config looks like it matches the the systemd man pages,=
 so:
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Sun May 19 19:51:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 May 2024 19:51:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725707.1130080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8mZ3-0003rV-DS; Sun, 19 May 2024 19:51:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725707.1130080; Sun, 19 May 2024 19:51:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8mZ3-0003rO-Af; Sun, 19 May 2024 19:51:25 +0000
Received: by outflank-mailman (input) for mailman id 725707;
 Sun, 19 May 2024 19:51:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8mZ2-0003rE-UE; Sun, 19 May 2024 19:51:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8mZ2-0007gq-Qt; Sun, 19 May 2024 19:51:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8mZ2-0004cs-Gd; Sun, 19 May 2024 19:51:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8mZ2-0005K9-GI; Sun, 19 May 2024 19:51:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SnIhNulCIbaVLYjRaELdEXkR3J+KM/tonODZN6dfYj0=; b=tj/ALj2rkRUTWPC56MboOLEoyJ
	4zTg6xG+ucKPbztOjnX5oWA4lQ74SXivRaxshVZoAfZK7vwaThTV8wzFOxUW+Ya2hAf865UXytpzW
	HyCPwW4+rYU4yihODWP10vAthLM22qJAK+LbNkCO/WpwfovnlyxLL06I8xv2n0LPuVNY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186040-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186040: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qcow2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0450d2083be6bdcd18c9535ac50c55266499b2df
X-Osstest-Versions-That:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 May 2024 19:51:24 +0000

flight 186040 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186040/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl           8 xen-boot                 fail REGR. vs. 186036

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot         fail in 186038 pass in 186040
 test-amd64-amd64-xl-qcow2 21 guest-start/debian.repeat fail in 186038 pass in 186040
 test-armhf-armhf-xl-credit1   8 xen-boot         fail in 186038 pass in 186040
 test-armhf-armhf-xl-qcow2     8 xen-boot                   fail pass in 186038
 test-armhf-armhf-libvirt-vhd  8 xen-boot                   fail pass in 186038

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 186036

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check fail in 186038 never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check fail in 186038 never pass
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186038 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186038 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186036
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186036
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186036
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0450d2083be6bdcd18c9535ac50c55266499b2df
baseline version:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f

Last test of basis   186036  2024-05-18 21:13:52 Z    0 days
Testing same since   186038  2024-05-19 03:51:42 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Baokun Li <libaokun1@huawei.com>
  Christoph Hellwig <hch@lst.de>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Disha Goel <disgoel@linux.ibm.com>
  Guenter Roeck <linux@roeck-us.net>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  Jan Kara <jack@suse.cz>
  Jeff Layton <jlayton@kernel.org>
  Justin Stitt <justinstitt@google.com>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Li kunyu <kunyu@nfschina.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Luis Henriques <lhenriques@suse.de>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Michael Forney <mforney@mforney.org>
  NeilBrown <neilb@suse.de>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Stephen Smalley <stephen.smalley.work@gmail.com>
  Steve French <stfrench@microsoft.com>
  Theodore Ts'o <tytso@mit.edu>
  Thorsten Blum <thorsten.blum@toblux.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Vladimir Benes <vbenes@redhat.com>
  Ye Bin <yebin10@huawei.com>
  Zhang Yi <yi.zhang@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1600 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 20 00:41:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 00:41:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725756.1130091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8r5Z-0005XJ-5r; Mon, 20 May 2024 00:41:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725756.1130091; Mon, 20 May 2024 00:41:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8r5Z-0005XC-2L; Mon, 20 May 2024 00:41:17 +0000
Received: by outflank-mailman (input) for mailman id 725756;
 Mon, 20 May 2024 00:41:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8r5Y-0005X6-LR
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 00:41:16 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a64c5bb1-1641-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 02:41:09 +0200 (CEST)
Received: from BLAPR03CA0150.namprd03.prod.outlook.com (2603:10b6:208:32e::35)
 by PH7PR12MB5760.namprd12.prod.outlook.com (2603:10b6:510:1d3::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.34; Mon, 20 May
 2024 00:41:05 +0000
Received: from BN2PEPF000044A8.namprd04.prod.outlook.com
 (2603:10b6:208:32e:cafe::b8) by BLAPR03CA0150.outlook.office365.com
 (2603:10b6:208:32e::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30 via Frontend
 Transport; Mon, 20 May 2024 00:41:05 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN2PEPF000044A8.mail.protection.outlook.com (10.167.243.102) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 00:41:05 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 19:41:04 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 19 May 2024 19:41:03 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a64c5bb1-1641-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FaUveT7HkVTse8/kaqjtfWkHT+lBWd/2s25tDq6BzdezW3yzl77z8vErKbhuY74kyUZH33uqRIkqzgG7uip1vS4+5eEmuv9ukkwzBVCVpW6a5i1FC4c2Xo9x2XgaNnMTcO3vs/RCUSKHksFMq6RaqUn4iQ49VTA7tOGjV1OPDCNHyqATUhu5UZO2DPw39ToCS7auTQ1uwVCaG+NiDHeGO9VsDj3Z1Vvh/36P8O8u3GwUU70a4T5VDgdwRkR8uA+U87+piPJIjHjJpzJMnNbD5u1uGnpo3vbusK/9MDDr4E6/FzXLWkQEZykKz1fSnEKXvdJmF9Ylh02yEFUEMkhk+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2xpNV1czfTjyWsHWrjwN22EkhMlaADqWr442PdeSCqQ=;
 b=lswjHEhTJmt3UIjA8GoRcPiNahi+FXodpInEYxfNLW6KTJ5x0AQBv4MltnBXeXNluYyOPzLEDIzvxAe+Rb1NT7zDFmn8jP2xywW3yJU3lYDzIhJ/4LVp1kiI69zaT59xLwh6bMroPn3oo5j9Xp2ZwMoKuu634FUZ5xYHavlXDxkL15EcJDBrFvChijZjKMlYWvkdIH6D5djGfHzC+wzAbSoYgzWbBaBbaSGsu3K1cYKMLMhCDm2FFTDMwuoqVde8wOOBJNCNiov1EqOhC8BrxUUZ+28FpC9nrW90+8Ie1BsD6cjlr+4nCVHQ8n3Vqd54w1iYi0D6GQjufqCxoLCQLQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2xpNV1czfTjyWsHWrjwN22EkhMlaADqWr442PdeSCqQ=;
 b=bdDJv1stzbVxrxzdFF+htaooBCtsDJrDDvIHoloWvXChJTRDK5XAwDJVS9TgU0HfAu7qdDh+VYiBYoJS/ngdkcPDAactoMhVyk06lYWxJNO/z+rLNSMaPFu9YryNdhCFAlp1tBo4PGcqr/DU9o+XX9EC7xJhlG04bU2cb8gxhtg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <d208a4d6-8d3c-4bee-a77c-a3f4758ac9ae@amd.com>
Date: Mon, 20 May 2024 08:41:02 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU
 for Dom0less domUs
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-4-xin.wang2@amd.com>
 <01825e63-3cf1-4539-85e4-d87fecbcfcf4@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <01825e63-3cf1-4539-85e4-d87fecbcfcf4@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A8:EE_|PH7PR12MB5760:EE_
X-MS-Office365-Filtering-Correlation-Id: 111171a0-ebd4-4a90-c39c-08dc78658847
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eERzanArM1BQdFIxeGRyYzNIZjNrR1RFOGthbmxkSks3K01maUVlZkhrL3gy?=
 =?utf-8?B?ZER3L25PWG1zREZ3c0x5K1lMNjFLeXdHSUErTUw3dDZDd09UNzNZekdFcUor?=
 =?utf-8?B?ZG8wRlNMRVhrdFF3bkhrWWpnakhoS1hrUElYemZQVFlsenNSRHI1MkJGbnhV?=
 =?utf-8?B?ZDZZUm9Hd1VBL0oxdW1YQ3d4SWQ1Q2grSG9iQUJ2TU1LdXA5VDVIU2orNnB1?=
 =?utf-8?B?dGkyT1ArNU1WbzFReGdPUndSZXlIRktHMVdId24rWUM3MUtMVlU4cHBGRmFV?=
 =?utf-8?B?RXphZ1VlTS8wUnplMlQ2UE9LVk0wTkZSU3RSWlY0Z2d1aFpYZDRqdWxhUjVG?=
 =?utf-8?B?RlFsL3RvelhKemU2TFNaNFpaTk1BbEtJTnNqOFQ5WFFWTWhjNnFXZGN3dVpt?=
 =?utf-8?B?WVNzWjB2YXp4UmwzVXhwM1dneUlDZ3F2TlNyR3FtbVpUay84b1RYZjBybXc4?=
 =?utf-8?B?WjdnSjZST0RMU2YzT0pPR0NWM21oK2RPTi91T3N5aUM2bDRNYURvMm5GWGVt?=
 =?utf-8?B?WTVkLzI3cTBYKythaWUxakMwM2FkTjRjT3lOKzV1L0tmcTJJUnhrUUkxZTlp?=
 =?utf-8?B?WEFtNkNlejBhWC8zT2xQSmxvaE5lZzlSZ3hmeXBSUVc5NksxTWtYMmp0STIx?=
 =?utf-8?B?L0lOSXdrZVAraTZyd0diazdLVVFMcHpHV2RWS2llOG9KTTF4M2V3VGlCcTg4?=
 =?utf-8?B?VHZoOWtvbk1EblRWcGlJejhPTDZFSEJnU1dPRy9UdXJwV0dTNEQ2Rjh5UElj?=
 =?utf-8?B?NTlCN3JKenh0UjliRnF5Q05QalYyTThMUTVnck5sSk1mdVZpQkRoOWJ6WUg1?=
 =?utf-8?B?WWZyMzFKSTNTekozVDAyM0lQS2VUSENUTEV3VE1HVkM3aW43ZWJzSmNpRTg2?=
 =?utf-8?B?OXFZUFNuaVBKTHhicTFuUnU1OVdFUzFMcC9YZ1l4elZmc1pEeUp4NGZ1c2VC?=
 =?utf-8?B?bklwVG5KMFJmSU0yeWp6cEtjSDFjTnkzblVaYkhzZ3NwM3hFY1g5cVNYZFI0?=
 =?utf-8?B?djBQRGtyUGowbUk5bllLV09kZWhPUnhZd3dxZ1FnMzZmYTBQRWQreHIzVlZS?=
 =?utf-8?B?UWRvOHd1cCttR0VoQzRQbXd6VmpJTVRPWUxLSHNuRm1RN2NLbEFTMVNtYmIv?=
 =?utf-8?B?TnFVZUQ4RVJIdDlOaXhZZzZSK0QrdUhNRjY3aDNjMXMzd2NkbFAraDJRR2tC?=
 =?utf-8?B?UGUrM1c0YjFLLzBRQ0kzTTZZM05UdHFPSTlxSytla3ZLZEtOYTZVazBWYTJs?=
 =?utf-8?B?eHlHb0hhaytISVdwYkpxT2xyQklQOUF2L2hNZk0xRHRFYlVUT2JCVEhReVIz?=
 =?utf-8?B?TDBkL0tPemk3amJHa2x6aEQzSnlPUllpbTNuMlY5N3ZCUHJuMGtZTlBRSHMv?=
 =?utf-8?B?RnUxT0hLSHJJdlRXVk50T2xXeHpTY1AvZFRFc3NCWnhzVkwxZDNQYVVMYW9j?=
 =?utf-8?B?bDVuVkE1OTgrQjFGV2lLSUhHY0hDMjBjeWN0Qm1QeXluRnVxWXZDYjZ0QktD?=
 =?utf-8?B?TFpMZTAraFJHeU5oeU5Ndy9DalN6aUNzcXRWL21NMysvV2RzMUxGQWQ2WUoz?=
 =?utf-8?B?ZWQwVVZpSHRCM2s1ajJKS0poSEtXVTBIRVhvODEvUFd3eTZQaUVja0F6cFc3?=
 =?utf-8?B?RWpZZDFkczlaNWFOT1ZpYndMTlM3TlZ0cDdLNyttdVlqRlBPcGF1N0E5ZERO?=
 =?utf-8?B?RWpZa0tiYVFsbW00REVJNndGa1c3d2NJZTN2cXp4T1d5NVpwTElOVnRwTER4?=
 =?utf-8?B?QllmU2pwKy9ENGdqcHc3ckJVNmFwZENMV0hFMG1HcmFLTHF2bjBHaDhOTU5l?=
 =?utf-8?B?RmVhd1EwMmRSNTdkdkxXU0svZ0lpQ2hVaU5pb29FYnJQVkg4MSt1ZlptMnRx?=
 =?utf-8?Q?kSgxpGFOQAiKX?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 00:41:05.2733
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 111171a0-ebd4-4a90-c39c-08dc78658847
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A8.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5760

Hi Julien,

Thanks for spending time on the review!

On 5/19/2024 6:17 PM, Julien Grall wrote:
> Hi Henry,
>
> On 16/05/2024 11:03, Henry Wang wrote:
>> diff --git a/docs/misc/arm/device-tree/booting.txt 
>> b/docs/misc/arm/device-tree/booting.txt
>> index bbd955e9c2..61f9082553 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -260,6 +260,19 @@ with the following properties:
>>       value specified by Xen command line parameter 
>> gnttab_max_maptrack_frames
>>       (or its default value if unspecified, i.e. 1024) is used.
>>   +- passthrough
>> +
>> +    A string property specifying whether IOMMU mappings are enabled 
>> for the
>> +    domain and hence whether it will be enabled for passthrough 
>> hardware.
>> +    Possible property values are:
>> +
>> +    - "enabled"
>> +    IOMMU mappings are enabled for the domain.
>> +
>> +    - "disabled"
>> +    IOMMU mappings are disabled for the domain and so hardware may 
>> not be
>> +    passed through. This option is the default if this property is 
>> missing.
>
> Looking at the code below, it seems like the default will depend on 
> whether the partial device-tree is present. Did I misunderstand?

I am not sure if I understand the "partial device tree" in above comment 
correctly. The "passthrough" property is supposed to be placed in the 
dom0less domU domain node exactly the same way as the other dom0less 
domU properties (such as "direct-map" etc.). This way we can control the 
XEN_DOMCTL_CDF_iommu is set or not for each dom0less domU separately.

>> +
>>   Under the "xen,domain" compatible node, one or more sub-nodes are 
>> present
>>   for the DomU kernel and ramdisk.
>>   diff --git a/xen/arch/arm/dom0less-build.c 
>> b/xen/arch/arm/dom0less-build.c
>> index 74f053c242..1396a102e1 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -848,6 +848,7 @@ static int __init construct_domU(struct domain *d,
>>   void __init create_domUs(void)
>>   {
>>       struct dt_device_node *node;
>> +    const char *dom0less_iommu;
>>       const struct dt_device_node *cpupool_node,
>>                                   *chosen = 
>> dt_find_node_by_path("/chosen");
>>   @@ -895,8 +896,10 @@ void __init create_domUs(void)
>>               panic("Missing property 'cpus' for domain %s\n",
>>                     dt_node_name(node));
>>   -        if ( dt_find_compatible_node(node, NULL, 
>> "multiboot,device-tree") &&
>> -             iommu_enabled )
>> +        if ( iommu_enabled &&
>> +             ((!dt_property_read_string(node, "passthrough", 
>> &dom0less_iommu) &&
>> +               !strcmp(dom0less_iommu, "enabled")) ||
>> +              dt_find_compatible_node(node, NULL, 
>> "multiboot,device-tree")) )
>
> This condition is getting a little bit harder to read. Can we cache 
> the "passthrough" flag separately?

Yes sure. Will do this in v3.

> Also, shouldn't we throw a panic if passthrough = "enabled" but the 
> IOMMU is enabled?

I take the above "enabled" should be "disabled"? Actually we already 
have several checks to do that: Firstly, the above if condition checks 
the "iommu_enabled", so if IOMMU is disabled, the XEN_DOMCTL_CDF_iommu 
is never set. Also, in later on domain config sanitising process, i.e. 
domain_create() -> sanitise_domain_config(), there is also a check and 
panic to check if XEN_DOMCTL_CDF_iommu is somehow set but IOMMU is 
disabled. So I think these are sufficient for us. Did I understand your 
comment correctly?

Kind regards,
Henry

>>               d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>>             if ( !dt_property_read_u32(node, "nr_spis", 
>> &d_cfg.arch.nr_spis) )
>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Mon May 20 01:01:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 01:01:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725766.1130101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8rPJ-0006Wt-NH; Mon, 20 May 2024 01:01:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725766.1130101; Mon, 20 May 2024 01:01:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8rPJ-0006Wc-K5; Mon, 20 May 2024 01:01:41 +0000
Received: by outflank-mailman (input) for mailman id 725766;
 Mon, 20 May 2024 01:01:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8rPI-00067F-Je
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 01:01:40 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on2061b.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::61b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8343b2c0-1644-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 03:01:38 +0200 (CEST)
Received: from BLAPR03CA0049.namprd03.prod.outlook.com (2603:10b6:208:32d::24)
 by SA3PR12MB8048.namprd12.prod.outlook.com (2603:10b6:806:31e::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 01:01:33 +0000
Received: from BN3PEPF0000B071.namprd04.prod.outlook.com
 (2603:10b6:208:32d:cafe::99) by BLAPR03CA0049.outlook.office365.com
 (2603:10b6:208:32d::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 01:01:33 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN3PEPF0000B071.mail.protection.outlook.com (10.167.243.116) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 01:01:33 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 20:01:32 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 19 May 2024 20:01:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8343b2c0-1644-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ise0+4BTQSr6FGJCLw61MTRng3sQl8txOHZpJR1LCx6nQbI7NLdjrn3M0HTN5AQvFqkwPoR7QU+z6eEzxj0slS42XUFExsN/Dt/ggNJJOWvktOoxSWxcoo1jbX649pQlYNjz7QkE9oxZMwGdpsx0VYreIM85VhNPWCq29rJGUzXpQZXyMHGRKaNyhM+lP87+bluilQyAE6Dt4kDGfvsXc5JW4CNx+63p20OZ/thugPwhJfdmaNGOGecj8lqtKpE09ycjauW0w0HdbAnvTEX7fDvf1hBwj2+j+RZ12dztrXxzGJynQ0XcXcWM0tCWw/C3arGQGOCl5EDaW6fZgwrQbw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=jAEzMUEC39/7LxxheMN8lvN9AclDqhjSvZZfjpOUyIg=;
 b=bI30MG9lTltSKUfJ3yVCGhJDVCoBXPtgcRIS2MIQ0dl0wqWScuBzaMCUdCfpXRCepsfPErAGe9TY/a3hs4yULsszQ9aOu/uQabQ1q2fFM1HAThVQifNze1OBDfy1MpuNkC9Z0boFwJ7alXPGujb25QtHfqoGTX3Dl47NqYgHYXF+DVjEh3YvKDy8v5+sPW599adkatZ7wIleXhJkJoe0W+WNghTYexyqNzm39pCTUuPY3mw2q8JrzUVeOH6Qi3A9EheOaZKmDVTrjozC1NBWzPLj9zyqKhnRdu/BbW3aa3YACRDfEK7aFqKm04Lh/J6Kqd+P3ZsBtLYMxvP4HSTvwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jAEzMUEC39/7LxxheMN8lvN9AclDqhjSvZZfjpOUyIg=;
 b=BjTF6XMZbc2ipF++gD9L0A/UG0ty4+2VB9RgQpmxiz3/qRnEdQa7cov7C4wVe76FnUfPqQrvudG+u5SwSI//hUR17xrIqIXuRMMLu814MQfjWl1HJImlN7r/UHpxd1UUSc8gZonOR6p8bM3dokyUOaFJlI/zh8sbr5TFRFwg0CY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <c475fc50-3d30-49d4-b12d-31535cfe1306@amd.com>
Date: Mon, 20 May 2024 09:01:29 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-6-xin.wang2@amd.com>
 <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com>
 <9f086470-a17f-482a-ad98-814424da0ee5@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <9f086470-a17f-482a-ad98-814424da0ee5@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B071:EE_|SA3PR12MB8048:EE_
X-MS-Office365-Filtering-Correlation-Id: 01e48cf5-fcad-4a96-da6e-08dc7868641e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?KzBUMmREMkpMTTFWZ25NTUhUdHlBYTlDdGs5dWE4eG9nUUUyTW40eWpudXho?=
 =?utf-8?B?MlVMMzlSK2w5Kyt3SHZjNGtnVGowdGM5VjRkYWZzSW1YMEdJSGFIWk1WQnNR?=
 =?utf-8?B?MFluNEVzR3JoVyt0Wnp0cnBHLzNORHFEUllXakhGanhINmpwZVh3eDNpdjJI?=
 =?utf-8?B?TDN3ODY1bnhVaG1ZbTNUczduRHVNMzVkY0x1c2ZOVENhRkxEK3BUTk9HOEhi?=
 =?utf-8?B?SUFzK2pORCt5ZTFQK05ablR3SUNpalF0Sm1KZW9naWR4TWZOZ3FVQkJxeTAz?=
 =?utf-8?B?ZDVvVjU2TzJvbTBDQXpzUTNHbUdpck5FT2k2WE81UmFDeU5WQWtISGZZRHda?=
 =?utf-8?B?VU15bERQaGFXb2lXaGxWUDdzS1ppUjVTamZEckdOMjFwM3ZoWVM1cHdZTnpz?=
 =?utf-8?B?ZEl3bURtS0xBdFdhQXczUVp6OEQwS0paR0JJeUxUUjV5OXhSQ2oxS0g1bVpO?=
 =?utf-8?B?ZG1oc0NaRnh2VTZTN3JHV3A4aE8wOFlnYTlabWd0QldvVzBLWTBkRGV5YkxR?=
 =?utf-8?B?QWtRTGRnTWFtQ1YzTFpLZnVlbnlOQU5YaC9mWEs3MnZsbWxzTTVzZjBPbzZi?=
 =?utf-8?B?NDBJVXhON05IVVhtOUlsbHMrK2VkdjA0Uk5VOW91eWZ4eTNGOUFybjFDT3lj?=
 =?utf-8?B?eDZMUXZoUVBRUlV5QWt0bXVQWkhzaENHR1FIWGRKcjhIaDRGOVBHQk5pR21a?=
 =?utf-8?B?My91M2NqTmo4aU9kdnBIbzZTaGMzeHJONkxqRHBJV2VZaEFMaE1xQmkxMWtH?=
 =?utf-8?B?TDdtTXdaVkhhcmVTWjVIcER0OGNCOThoV2xlb3ZyNVJOR1F6enp1bEJhUE8v?=
 =?utf-8?B?RkFTMDdXeStqMGVOV1Qvd2krMmx4TE1kaVVWZWFpbVNkRDN0QTcwbHBhSzVG?=
 =?utf-8?B?aC8rci9EaC9LUmViWVR2enVvc05WWTdCVjZCbnZMb0JobmVPVk9PcE9jUnMv?=
 =?utf-8?B?M1Z2amZySWhBbTBFUWxVRnhVcmNJQjVjZjdHVGoyamgxOEJtanFnNks5eGdy?=
 =?utf-8?B?K0p0UUMvdVhoS0VGaHFDWUJSR2twTW9BZkdPWWZsdzBoemdWRWxORXN0bnpl?=
 =?utf-8?B?eEhsRTl2M0R1a0t2U2U3STJPQjVCT3F2c21JeVVXbVViNVRDUytrNXJkK05o?=
 =?utf-8?B?RitqVFNHL0VSNFprc05FbW9XbC9vRUpRQlE3emlFaHlBMGtIcGhSblUxNHBN?=
 =?utf-8?B?OEo3Y0t2WitueEd2Y1N2Zzl2NWYzU082ZjExR21IYTIyNXV2TGF5dkhrMWRB?=
 =?utf-8?B?LzQzV3FSV1Y5KzhOa2RDUi93dTR2QkFieHJlZ3dhelJtZU9yWkJQbEttTThW?=
 =?utf-8?B?ZlJjc1JFaVR0ZU9aaERNUXB5VFl1RDZBNFBHWWpvTSt3bjZtZkxFOS9QTXJV?=
 =?utf-8?B?am93NVFXVG93aVBlUTZxTmkyYWFTVWxsRWJwVlVvTmh5aVY5b0l3dWh3TTRY?=
 =?utf-8?B?MkFDcTVvMk9KSjZKYU1hTUVLR0NoY1pNNEZmb242cDUrVm55b2p4VGorNG02?=
 =?utf-8?B?Uys0c203VWZMZjRXOEV2RHJZa3VhU2lzWElpSFhSQzNKSGdjVjBjS1FWS0cw?=
 =?utf-8?B?cE5nZE44Q3dQWTBNL1VtTk1sS0RjaWcwWVFOREJXeFE0M2EvTXRDV3hkU0VU?=
 =?utf-8?B?S3pDSzdWVHJxenc5TDVVNENUeFBCQ2hhdmR1Q2U0V1VNUllwbTh6QWJTQ2x2?=
 =?utf-8?B?QVVQRFhUWFpXNmJleXZ5RENKVDBxNU1UcU5pWEpwQUNmV3ArOFVPUTJkeCs0?=
 =?utf-8?B?ZnE5TzMyZ3Y1SGc1TURaVEdid0t5K0VvMVJDd2dHMmFiS0hmdGlKNGpFU1ZJ?=
 =?utf-8?B?NFZqOHBpTk9MVjZxd3hLYjJCMytLc2ZaRGErMXk2TUk2NEMxck9JWGdKT3pL?=
 =?utf-8?Q?xMpEnQh2pCALW?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 01:01:33.0974
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 01e48cf5-fcad-4a96-da6e-08dc7868641e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B071.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB8048

Hi Julien,

On 5/19/2024 7:08 PM, Julien Grall wrote:
> Hi,
>
> On 17/05/2024 07:03, Henry Wang wrote:
>>> @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, 
>>> struct vcpu *v, unsigned int virq,
>>>       {
>>>           /* The VIRQ should not be already enabled by the guest */
>
> This comment needs to be updated.

Yes, sorry. I will update this and the one in the new vGIC in v3.

>>>           if ( !p->desc &&
>>> -             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>>> +             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
>>> +             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
>>> +             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
>>>               p->desc = desc;
>>>           else
>>>               ret = -EBUSY;
>>>       }
>>>       else
>>>       {
>>> -        if ( desc && p->desc != desc )
>>> +        if ( desc && p->desc != desc &&
>>> +             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
>>> +              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )
>>
>> This should be
>>
>> +        if ( (desc && p->desc != desc) ||
>> +             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
>> +             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
> Looking at gic_set_lr(), we first check p->desc, before setting 
> IRQ_GUEST_VISIBLE.
>
> I can't find a common lock, so what would guarantee that p->desc is 
> not going to be used or IRQ_GUEST_VISIBLE set afterwards?

I think the gic_set_lr() is supposed to be called with v->arch.vgic.lock 
taken, at least the current two callers (gic_raise_guest_irq() and 
gic_restore_pending_irqs()) are doing it this way. Would this address 
your concern? Thanks.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Mon May 20 01:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 01:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725783.1130112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8rzM-0003N9-C0; Mon, 20 May 2024 01:38:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725783.1130112; Mon, 20 May 2024 01:38:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8rzM-0003N2-7d; Mon, 20 May 2024 01:38:56 +0000
Received: by outflank-mailman (input) for mailman id 725783;
 Mon, 20 May 2024 01:38:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8rzK-0003Mw-Oc
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 01:38:54 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b624c2fd-1649-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 03:38:53 +0200 (CEST)
Received: from SN7PR04CA0198.namprd04.prod.outlook.com (2603:10b6:806:126::23)
 by MW6PR12MB8735.namprd12.prod.outlook.com (2603:10b6:303:245::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 01:38:48 +0000
Received: from SA2PEPF0000150A.namprd04.prod.outlook.com
 (2603:10b6:806:126:cafe::ff) by SN7PR04CA0198.outlook.office365.com
 (2603:10b6:806:126::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 01:38:48 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF0000150A.mail.protection.outlook.com (10.167.242.42) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 01:38:47 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 20:38:46 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 19 May 2024 20:38:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b624c2fd-1649-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xq3jdPSv91GKzyaci/FZlBKluSxitY4CZBjdcxUGBqErVLFhCRoL4SD5y9axFNtLLST4cJZaxoEDgHikA7OTNF14DRzxmnvyXpb8hX21qxWR2iIows7LNDfzZMNsZ9ARPYPjBTKW9h/OyjXaLpqx2+drk3fBLPPmPGfBVTVSIMAJ7Jbw3ywgMx/U3ISlsSm9wAGa8Zf9aYF8RZBSh9YET6JxaoAUDew1ELF5HlzYsFYdQ4m+J7lza/xnRdtPjtTOymIg4tCQsZlWQz8EitOflBZKCGduMYGHJQuX32QlWNKThf7jRpFgOqhxMk//DP+w+5PtkW8fRGW8w+8crsI+dw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=E0YAPpV586Sp1/JSE7Lx2DUZD0CE4GcSsEhWCW/Pcxk=;
 b=JwRyhw3F+m48C3nvvnJVIHL9Wxt0y+4KD3GEbqmQ5N3eP7zPPaKbyy8LQ6nDC/Q0QCyR/8KDDn7nBTaL6XWUzm8nOxelBYAd/xNNOa/eaIHmSXU7cp6BLNOrnbeHFjCgLIqzZiFxZuLiycy7vGwLtySOOQEhF9b94b9DeWIuOEngkhsm1PPU1KWKZpXGWsKN6/ylNqxN+UCyzSNm/I6y2+xnT8GFpLndrojDAi6tNH6vwuSBcUucF2qWGQfpJHpQIRyG/STvWV0r7bPfwqST8v6Nf0jOai9UYfSA1Naby+X9m4Sje7B499HVCU5RSeD6gRawmlv85P3HAxh8GJOicQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=E0YAPpV586Sp1/JSE7Lx2DUZD0CE4GcSsEhWCW/Pcxk=;
 b=v4NHffNZAwEjHY+pn7Aqrw/5zOusMuaZquxK9YRohbEFPOWuTkuLsKKJVK1Jb0jdJ95c6/WIsbThuqfRtPEeUMMBHpxvVdB6M/fX3Q5Aegp569KKPGYQIXqVtGberlV2m+QffTv/w4kQoOCVxtLksmwgtaSI6bef7rsM0YTqtIk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <72303595-0b15-414b-8249-0d8021312356@amd.com>
Date: Mon, 20 May 2024 09:38:44 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU
 for Dom0less domUs
To: <xen-devel@lists.xenproject.org>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-4-xin.wang2@amd.com>
 <01825e63-3cf1-4539-85e4-d87fecbcfcf4@xen.org>
 <d208a4d6-8d3c-4bee-a77c-a3f4758ac9ae@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <d208a4d6-8d3c-4bee-a77c-a3f4758ac9ae@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF0000150A:EE_|MW6PR12MB8735:EE_
X-MS-Office365-Filtering-Correlation-Id: 7b6a3f06-c8d5-4143-b41a-08dc786d97f7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?cWU1NDZSRlF5Y25CR0FNbnpIYmZVd1pZUjFQUENWYmVVNTVHY2ptMFN2TFps?=
 =?utf-8?B?TFRYOWRsZlUwTUVjRnJtRHVkMDRPY0Jxa2NpVGtvVzZ4UzZGSmNsakZqNHdj?=
 =?utf-8?B?dkI1d2xnMzI0b1ZZUVQxdkh2a1gyKzNHZTlFV1FwVUZvNEVPeGloV3dSU1ZX?=
 =?utf-8?B?Y2tzMzI3RVR6QWFaOHhoUGJPUlZxaTR5bTJ1NTN0cGNmb1N1YXZkbkV5NnNl?=
 =?utf-8?B?SDlBYjlucmFiUGowcXpsamVjR3EvOWxzNDE4UXYyTXFESXFMTmNqbDcveHcx?=
 =?utf-8?B?dkE1V3FlcWNueFB3SGlXM3A2S2VJQiszOXdKK2htQXl3TitlN081enJKSDcv?=
 =?utf-8?B?M0w4aDBTNi9CcWU5dHY3RkVuWk12QmRDWU1EUk9HQTArQ0I4QXdzdWdaUXEr?=
 =?utf-8?B?NkhxcHpwanhaTi80SThucVp5WDJiREE5RGlyWVlkMkZoVE9xNFRFMExnWGZp?=
 =?utf-8?B?RFppd2FlQnQ3QksxdFpLSlZoeHVTK0hRbllxTHJHNjNWeEhFc28rWGRWdkpN?=
 =?utf-8?B?b2c5a1RkSjhYUlhSbm5hOERZdzl6blZFR1RFTXdhS0JLbHIvNVRnY01yR1ph?=
 =?utf-8?B?ZjAyT3NEMExQSkNBRWp6c0Z5UHdsYXVveTRvZzIzdE5KamFoNEFrMldtV3hv?=
 =?utf-8?B?eVVZcWV3WGYrOUVpVWhMaExmai9ycFRWRjNTeUpsRzdsTGpXWXNqUWo3WlhJ?=
 =?utf-8?B?dzBQU3lMVHNjUXA0dEdMaStLcDdSSVc3WjgrbjZjdW1HTUl6K29vY2lha1c1?=
 =?utf-8?B?dkxCM2tURVl6SmtFRFFzYmpET0VTamU2R1pvUUtSdGZwbjdDNHVBWWxuZm1R?=
 =?utf-8?B?R1FZNlhoUjMxZ2xOckkraGQxLzRVMW9yMUVZREdiL0Z2QzcxT0lZWmJlcjZn?=
 =?utf-8?B?NHo4MC9oQmVmbWF0QXNFV0dEdnRFYUJMOGdtOXpNdG1PRzFoaHU3cHdLWlJz?=
 =?utf-8?B?M25WeEJmMTJWQ1BNaEZGM29qdTgyK09yQjVZT1BzbVNJWEEzNHZ6Q25rNDFV?=
 =?utf-8?B?K0NZM3c5bDJPT3BzMFJ6TDNncExtMHVSdHJYdUtESXlZTUV4MnV4dTNxSGlB?=
 =?utf-8?B?cW5KWnRTNGlmV0xud2lUY1d4M2U3R1hUaGFUOEFFZTIvdkZGSzU1ZlJIdHV4?=
 =?utf-8?B?elplZmtiYXBybTlCVzNjM1dvV3FBc0JFSDNpT1c5N1Z4d3BIbVFOakdhZUdJ?=
 =?utf-8?B?NG9pRVNaYnUxUVN4SVQ5a29kQlhzT0lxM0ZMaDQweVZ1Tyt6cU1LNmwvck90?=
 =?utf-8?B?VWdMWnpUTFBhQlBHWG9CenVPMHVUMkN3SEtPS1Z5cllCTEpTaC9pd2J4bE1u?=
 =?utf-8?B?OXNxME5yN2FHako0SDkrbDJaZWpxSGlKcFMxbGxrNWFNVFBrdDdpVE0wZSt4?=
 =?utf-8?B?c0ZFK1M5QW8zUy9LenJvempMRmp3UzY0N0d2NytZN0w3UkRRcFB3WVJIMW9U?=
 =?utf-8?B?NDBNRk5QeXJzSHVGNWJjY1dXY2JLS0h1TC9sUWprbWVWTFZIS3pzQUlYK3ha?=
 =?utf-8?B?cHZpdXJrVndqWDVlSjVURUFpc0NMRVN6NE9raThjcmxmQ3kzQXFMWHhJcWtK?=
 =?utf-8?B?M0xaVDNyT1hBbmFIL21tcjkvd2N3UHp4Zmw0YVMzQWZ6RWl1aXNPalhZd3gr?=
 =?utf-8?B?NFhJTUZUcUpxcHdUdnZRcEF2RFpLOGt1SkVDMXh4L2dtNFNJNGtvcnpCY2VH?=
 =?utf-8?B?cXZRNWlyZmZ0Vi8wUXpUWVAxWXV5TmphbTRwWWNMalVTNUR5YkRGeFB2c1RP?=
 =?utf-8?B?NkxyaUh3TER5ZmhYaUtneWtvT3VmK08zMDRtSXdWWlU1OGVwYVA1WklOZUN6?=
 =?utf-8?B?UmI1WGtXYkpXS0lFK0V4ZkQ0M2YyUDRlNS96Y2dNZVFDZ2QrVUQ3akdTaEJl?=
 =?utf-8?Q?GkJE4XgBYiB6S?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 01:38:47.5504
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b6a3f06-c8d5-4143-b41a-08dc786d97f7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF0000150A.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8735

Hi Julien,

On 5/20/2024 8:41 AM, Henry Wang wrote:
> Hi Julien,
>
> Thanks for spending time on the review!
>
> On 5/19/2024 6:17 PM, Julien Grall wrote:
>> Hi Henry,
>>
>> On 16/05/2024 11:03, Henry Wang wrote:
>>> diff --git a/docs/misc/arm/device-tree/booting.txt 
>>> b/docs/misc/arm/device-tree/booting.txt
>>> index bbd955e9c2..61f9082553 100644
>>> --- a/docs/misc/arm/device-tree/booting.txt
>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>> @@ -260,6 +260,19 @@ with the following properties:
>>>       value specified by Xen command line parameter 
>>> gnttab_max_maptrack_frames
>>>       (or its default value if unspecified, i.e. 1024) is used.
>>>   +- passthrough
>>> +
>>> +    A string property specifying whether IOMMU mappings are enabled 
>>> for the
>>> +    domain and hence whether it will be enabled for passthrough 
>>> hardware.
>>> +    Possible property values are:
>>> +
>>> +    - "enabled"
>>> +    IOMMU mappings are enabled for the domain.
>>> +
>>> +    - "disabled"
>>> +    IOMMU mappings are disabled for the domain and so hardware may 
>>> not be
>>> +    passed through. This option is the default if this property is 
>>> missing.
>>
>> Looking at the code below, it seems like the default will depend on 
>> whether the partial device-tree is present. Did I misunderstand?
>
> I am not sure if I understand the "partial device tree" in above 
> comment correctly. The "passthrough" property is supposed to be placed 
> in the dom0less domU domain node exactly the same way as the other 
> dom0less domU properties (such as "direct-map" etc.). This way we can 
> control the XEN_DOMCTL_CDF_iommu is set or not for each dom0less domU 
> separately.

Oh I think I get your points, you meant the XEN_DOMCTL_CDF_iommu will 
still be set if the passthrough dt property is "disabled", but user 
provides a partial device tree. Yes you are correct. I will update the 
doc to explain a bit more details as below. Thanks for pointing it out.

  - "enabled"
     IOMMU mappings are enabled for the domain. Note that this option is the
     default if the user provides the device partial passthrough device tree
     for the domain.

  - "disabled"
     IOMMU mappings are disabled for the domain and so hardware may not be
     passed through. This option is the default if this property is missing
     and the user does not provide the device partial device tree for 
the domain.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Mon May 20 01:40:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 01:40:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725788.1130120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8s1A-00057q-Nv; Mon, 20 May 2024 01:40:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725788.1130120; Mon, 20 May 2024 01:40:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8s1A-00057j-LM; Mon, 20 May 2024 01:40:48 +0000
Received: by outflank-mailman (input) for mailman id 725788;
 Mon, 20 May 2024 01:40:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8s19-00057d-NG
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 01:40:47 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f8c0ef78-1649-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 03:40:45 +0200 (CEST)
Received: from DS7PR03CA0091.namprd03.prod.outlook.com (2603:10b6:5:3b7::6) by
 SA1PR12MB6847.namprd12.prod.outlook.com (2603:10b6:806:25e::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 01:40:41 +0000
Received: from DS1PEPF00017098.namprd05.prod.outlook.com
 (2603:10b6:5:3b7:cafe::fe) by DS7PR03CA0091.outlook.office365.com
 (2603:10b6:5:3b7::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.34 via Frontend
 Transport; Mon, 20 May 2024 01:40:40 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS1PEPF00017098.mail.protection.outlook.com (10.167.18.102) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 01:40:40 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 20:40:38 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 19 May 2024 20:40:37 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8c0ef78-1649-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=llM6NhCelw5DeDW6oxQ5PHDXdNaf8GRd77YDtTv9oipksUGue8s8KTAgrnn2xeJB8GbBioyUY7xpB0XT93krw459AY+Z1eboC5mWCBzXD/3cmgw2P2svanXwFw/ILZ4RqUbPAoMPJHNwLuR7GEFM91MYiVqHcEekVPXsgU/JDeo9fy6vks7mAbLdXqTwx2I8BSDKTpf96v7knvorqMsCodxxuxP28LH5zfUP/tMKvFKiPbA/BaSZj0dgsowNVWOyhWiwjRozzMvSjYZYAdqWVUOjFcjQPOmUiJ+TeAOWWYJ2hApFpJijqE4qVylGyZauropxEFwWCEE0vyjrFYcjSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WEUp9WTVQyswgeb6eSjJSLyIAOn7Qu+yXZBdvP93TQk=;
 b=G5QCBSq2CnOu4iQlOJFsEg+W1DBPpIVbIYNRe/z+ykQqhBajWEk52uTGV+hU2kFZnfahRypIK2lpjk/75083LpaxPe8+AeW5w+7nJOcckAAE4Awn7tcwWlcpN5Tmj2M+5cb6BGhdBW3Kxd3SYbb6gneuvJWdiiyJNTeavOKA7yHELmUcElxEHLsYFgw2ln6iwZiU7f2uzxyISRwxKHGta2YWtmql0cGCXmOs2Euo+8mx/vNk+PFMATKi9gK/yrk+QGcqdaQdrOnRLVOE3BNPObNrnoNMIm6oT25y1vJGccyYAkOMInqDzfoi4T7hF9Ace606ykSxuT6Ubv+UOi1d2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WEUp9WTVQyswgeb6eSjJSLyIAOn7Qu+yXZBdvP93TQk=;
 b=iAZeGcbT3xwmyaQug2vs87yNwq/WYuuuxPisphYnLW1U5rIK6wvsfgwrvysylKIw4ohxmfJ4Ax4e+6ngXYO4SKaMyyOUECXM6jtMb+vRcK+R3CCZ9NtayYHg1g75CJytu/eq6NwuPXe8Ldue6Cqyi+lnjmL6ZCRxVhi9bBYUW5Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <e6c9c37f-1ecd-4b76-ab97-b3649503f0ae@amd.com>
Date: Mon, 20 May 2024 09:40:36 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU
 for Dom0less domUs
From: Henry Wang <xin.wang2@amd.com>
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-4-xin.wang2@amd.com>
 <01825e63-3cf1-4539-85e4-d87fecbcfcf4@xen.org>
 <d208a4d6-8d3c-4bee-a77c-a3f4758ac9ae@amd.com>
Content-Language: en-US
In-Reply-To: <d208a4d6-8d3c-4bee-a77c-a3f4758ac9ae@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF00017098:EE_|SA1PR12MB6847:EE_
X-MS-Office365-Filtering-Correlation-Id: 87a023ac-b53f-48e2-b986-08dc786ddb47
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QTFsanJNN3RpNWdkRkI1cyswOTVpZ21JZXQ4RFFwMENhVjRueTZ4UjFlMzh5?=
 =?utf-8?B?Q05xZmswaG5SVkphKzlFTEdxRGpER0JKTEd3bkEvTlpMbUc2V2ZBY2xRYms0?=
 =?utf-8?B?K2ZMUTJmQ1RXNlFjZGNQYWJxUG9VVVMxZXFhcUdJNzNpRjBlNkMzeE5tUEsx?=
 =?utf-8?B?NUxnU0hUZjdkMGVWazFnUW1wazVjcTd0VWhkMTdXaG1xM053RGxScEVvdk45?=
 =?utf-8?B?eXoxcGd6cjVNbWFLNWxTL0xBUDQ2dENoZWtnUzU2V2lCU0pUcUxRVlVKN0ZO?=
 =?utf-8?B?ZmhvbUJtaTYxTmMyRHVPS0lRTktSVVpXODUrSEM1bjcyZlBNNis0bWRyNjNo?=
 =?utf-8?B?OXFRaWxXTkxJUWdrU2xaSHdjc0ZsM1RhMFQ4OWJSQ0JwMEFERHB3WXZyYmRn?=
 =?utf-8?B?bjBFT0NidzA2ckVwbCtnM0lnNnBzVEhsMng2MEhLR2Zuc29QVCsvSXpVUlBa?=
 =?utf-8?B?VlBrN05uRExIUWxWUzR6ajFMZW5PcFF1eVlkUEdIYWRmNTV2Zk5YNVFtNitZ?=
 =?utf-8?B?blZ0RCtHbEt2Ri9nZG9ISHhPME9aTTVVRkJnQ0kyYUtURW9ScHZlc29mc2Q0?=
 =?utf-8?B?Q2xRckh5d3Z6NXZKUlgzOGhJK0c4Q3RDMUF0aHJhNWlYSG4rei9vOU5QaG5p?=
 =?utf-8?B?dDZSMlY1Z3FwQWR2K0p1ZmJxd2VGc3dZcnF3eDN0RTU2L0ZOZHNwVW5TSWM5?=
 =?utf-8?B?NFltYm4rR3hQdWhnTTlsa2ZBQnVHR1JTcmxJWEdFVy9ST05HajNqVWw5YlFa?=
 =?utf-8?B?NnN3NEdxRk1mbWszQ2hTWE9VTGMrVU4vSG5uUlV1OVBMTTRzcWQ4Y0d3cGNo?=
 =?utf-8?B?bk9MK3VOVHpkR21qaEl4VGFWOGJEYkpQZldVSFVJVUtrTW9HRC9yQlZCbmhC?=
 =?utf-8?B?VXhJZ21mREl6Y1NnTFQ0aUZnYjJxTlRrc09OTHRGRUN0cG1LY2tQZmlHT3Zj?=
 =?utf-8?B?TGZPbkNQbDJNRUZLMFQrSk1FaHN0MWEyZVFXa1ZjaXIxUmovbEljY3BaYUd3?=
 =?utf-8?B?eE9rV09BcEpIZ0VhK1l6Z0tqVHhwbEl6bmE1NGtVVEViL1NPR2E0ZllnSVlE?=
 =?utf-8?B?S0gvajNTT1NKQysrcUVZQkt3elZBeUhTRDZPdU1raUc5bmJuUHFEZ3ZUc3Np?=
 =?utf-8?B?cTFxNmtZQ1dhZTArMzdPTUdYMVhDK0xOWTIxaHJFd1lPN3JlT0VvalF2ZTZK?=
 =?utf-8?B?RlBuTjF6MjJ6SEFvbXA0dGMycE9kbGhlV1dMcG41ZlhSOThzMlNpZnl0ZElt?=
 =?utf-8?B?bzRlNG1zdkxBa3ZXZUplYmFmVFUxNzQ2bnJrbkpZcXBFeVFDVHRiSzUwYlpI?=
 =?utf-8?B?aEoyM0tJSjBxQzkxempHWUNRWTlBbDBHVXdXdlNVRmdEQk1hb0NEbk8xVlZu?=
 =?utf-8?B?U25RWDdPMUhVT0xjaG04REFEWjNpQ1NTLzZCNmNiemd5MHRSR0pTVStZQ3Bn?=
 =?utf-8?B?emtXTmcvVDNwOXlQM1gzc0lCNE9hZzgxVTRxUkpLMVQ0ckVZaUJaNEY2dE5Y?=
 =?utf-8?B?WmJQR3BrL1ZMMFpUckZOOHRib3paSWlxWVdENTFRQ2RuZkE0a0RzNWVmYjFo?=
 =?utf-8?B?c0RXb01TMnU3QlEyR1h0bGtZWXMydXAyeXFYb0xBRGlTWkwrNy9TY0tsQUpE?=
 =?utf-8?B?ajV4UEpGZkR4K0tVeWNFMlpDZW9vSU16NUJnN0E3MW9OTTZHWlpWLzZXWWh4?=
 =?utf-8?B?ckdzUDFOT3dLdURRSm9TbDdUelducjhNTWF5OW5vQjVTNE9UUWEzYUhWaTY4?=
 =?utf-8?B?Y05uTmVtbnpLWGM0L2V6NG1UK2RwV2NQQklaUXVuTytVS0hMalNmaStJc0hV?=
 =?utf-8?B?dXI2b3l0dWNLSUtpcmRhS09qNGVXZm9JdkRiTlY3d2VtYVVrNGU2ak13VWFy?=
 =?utf-8?Q?TBfb95FQ4fQA0?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 01:40:40.2322
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 87a023ac-b53f-48e2-b986-08dc786ddb47
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF00017098.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6847

Hi Julien,

On 5/20/2024 8:41 AM, Henry Wang wrote:
> Hi Julien,
>
> Thanks for spending time on the review!
>
> On 5/19/2024 6:17 PM, Julien Grall wrote:
>> Hi Henry,
>>
>> On 16/05/2024 11:03, Henry Wang wrote:
>>> diff --git a/docs/misc/arm/device-tree/booting.txt 
>>> b/docs/misc/arm/device-tree/booting.txt
>>> index bbd955e9c2..61f9082553 100644
>>> --- a/docs/misc/arm/device-tree/booting.txt
>>> +++ b/docs/misc/arm/device-tree/booting.txt
>>> @@ -260,6 +260,19 @@ with the following properties:
>>>       value specified by Xen command line parameter 
>>> gnttab_max_maptrack_frames
>>>       (or its default value if unspecified, i.e. 1024) is used.
>>>   +- passthrough
>>> +
>>> +    A string property specifying whether IOMMU mappings are enabled 
>>> for the
>>> +    domain and hence whether it will be enabled for passthrough 
>>> hardware.
>>> +    Possible property values are:
>>> +
>>> +    - "enabled"
>>> +    IOMMU mappings are enabled for the domain.
>>> +
>>> +    - "disabled"
>>> +    IOMMU mappings are disabled for the domain and so hardware may 
>>> not be
>>> +    passed through. This option is the default if this property is 
>>> missing.
>>
>> Looking at the code below, it seems like the default will depend on 
>> whether the partial device-tree is present. Did I misunderstand?
>
> I am not sure if I understand the "partial device tree" in above 
> comment correctly. The "passthrough" property is supposed to be placed 
> in the dom0less domU domain node exactly the same way as the other 
> dom0less domU properties (such as "direct-map" etc.). This way we can 
> control the XEN_DOMCTL_CDF_iommu is set or not for each dom0less domU 
> separately.

Oh I think I get your points, you meant the XEN_DOMCTL_CDF_iommu will 
still be set if the passthrough dt property is "disabled", but user 
provides a partial device tree. Yes you are correct. I will update the 
doc to explain a bit more details as below. Thanks for pointing it out.

  - "enabled"
     IOMMU mappings are enabled for the domain. Note that this option is 
the
     default if the user provides the device partial passthrough device 
tree
     for the domain.

  - "disabled"
     IOMMU mappings are disabled for the domain and so hardware may not be
     passed through. This option is the default if this property is missing
     and the user does not provide the device partial device tree for 
the domain.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Mon May 20 02:58:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 02:58:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725800.1130131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8tDi-0006Sy-2H; Mon, 20 May 2024 02:57:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725800.1130131; Mon, 20 May 2024 02:57:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8tDh-0006Sr-Vi; Mon, 20 May 2024 02:57:49 +0000
Received: by outflank-mailman (input) for mailman id 725800;
 Mon, 20 May 2024 02:57:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8tDg-0006Sl-K9
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 02:57:48 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ba2177f3-1654-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 04:57:43 +0200 (CEST)
Received: from MW4PR04CA0245.namprd04.prod.outlook.com (2603:10b6:303:88::10)
 by IA1PR12MB6137.namprd12.prod.outlook.com (2603:10b6:208:3eb::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29; Mon, 20 May
 2024 02:57:40 +0000
Received: from CO1PEPF000044FA.namprd21.prod.outlook.com
 (2603:10b6:303:88:cafe::67) by MW4PR04CA0245.outlook.office365.com
 (2603:10b6:303:88::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.31 via Frontend
 Transport; Mon, 20 May 2024 02:57:40 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7633.0 via Frontend Transport; Mon, 20 May 2024 02:57:39 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 21:57:38 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 19 May
 2024 21:57:38 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 19 May 2024 21:57:36 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba2177f3-1654-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gB83k6Zp9lrH/EEKcJM33MSk5dM14P4VbHlKsl1oJVlwtXqN2/hbSUjLL+NiV5Q4N9IIABy66XnbPbV2QN6t4wzdiBp6R1Qo8FSGHtYg67iXhQu08V8dI4HPuZvPQfFw3GrHDc4+OEGFe+8FHVxhxoCGDj473QyTFaHmLYYD9eaKfsjiA8uyc6KAXsA5i6e9WYauqJUvwdyjmxZe80D+hTWHzr//xkD01oYqleAD0xpeuciRkucLLuWVEGAWnMhnsqxv9iYZDAnSkMLFNgRa7EpGFsOt0fmEBFjIWa+ejuOxU3NaMo9mCTQeMr+24tspGUM3CJf5RM+DO4UanfVBXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=gF50HH6PrPfTAjlX9fenu08DnzjytRDXcilNX+Rd5HQ=;
 b=c0rac++A2sIBx/52m4xsyw7ItqyF+gES+25SaGvZWvleaUEPhToHA1NB49PV/aT7EYe1xCQLUEK40pr+Cb/UJks+AMp5VJMvnIJvgrtHzauyWHphdcsjBn1PEosTmlyq1tXE0CrYMXUsKvc9wzxASs15NmU6Fq3R8g9sbaO7Ab3E8IQtHBDxLZutnEG7RWtUqqsSHG2dha5QV6hZ+5B3UemDx+UlTVSGwOh+8xaC2JOdzWMHxCCyBfMq3kPC9hXB0lhjPMbNofgkYFM/oyLxY1mi4jQHTB7pS1VC+w+6sc5AixVi5G2+/5bOzliHH4ESH/gtqPshREBY/J7kDCVR/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gF50HH6PrPfTAjlX9fenu08DnzjytRDXcilNX+Rd5HQ=;
 b=CInDUVa69L65fOybHkoBIcKb7iVYh52wnDMTGRoLJu2LowaqDZjMJUXoSCtvSTO1Bg0ynQAkrhXPITkTNnHH499PQLht1sQB6E8M6QNAfu/dRulO51hbLjFohMwYAjQxYtQg2PDpAGoKNnTpdzjI+AJihQzgss+iKWP7Gq2uOG0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
Date: Mon, 20 May 2024 10:57:35 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] xen/arm: Set correct per-cpu cpu_core_mask
To: <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal Orzel
	<michal.orzel@amd.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>
References: <20240321035706.165253-1-xin.wang2@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <20240321035706.165253-1-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|IA1PR12MB6137:EE_
X-MS-Office365-Filtering-Correlation-Id: f7612c30-13d8-4096-15fe-08dc78789c71
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Q0FLMlBrb1VCSTI5Y24wUFVZd1ZPTlZOZUJ4clFJUS9Ja1M0aXFhRjYvVXBp?=
 =?utf-8?B?Y2xvSGRaM3VMUk4xemhQSXR6U3R4NlNISFEyWGJKNCtaV0tIUHR1WGVkUUJz?=
 =?utf-8?B?a3VyanNQWTFPRHFacjJLZzEyMjVOcmJjZUFEUUZqTzVLUHBnUUpWUGVhR0cy?=
 =?utf-8?B?OG1TMHJoU3hxVEdrQVBadmQ1dXphVDhZSVVCODQ3QVRDVEdJWnN1STZ1UGtN?=
 =?utf-8?B?VlFsNXgxcnArS2FzUlViTnd2TURjZ3lkTWJaMVZsdjM4alZvQzZlRHc4aFRP?=
 =?utf-8?B?UDl6ejFZenpCZnJvSjd0dDl3NDFxbzZicFhaZEk5RXVaQVRpenNPOXpkdlBJ?=
 =?utf-8?B?SjdZclE5SG1pa05DMWVBcWdwQms0dERDQmM1cXdqamI4b1ZZRDRMMjFuQ3Iv?=
 =?utf-8?B?am1tZXZ2N0VPb2t1Q2YyRmpyM09YNmp0cjVEU2Jvay95VEpUZll4T2RobkV1?=
 =?utf-8?B?SzFMTVpiQUdaUlpUcGhFdmdQWld1b0VwcmdsSjdGQUU2SUJJb3hCUXgyZXlK?=
 =?utf-8?B?NHJoZDdQTWo2STVrM3RWZmFCWUhoaW5sam1HK3ZlVGtKY0xTa25HUDRGNDJk?=
 =?utf-8?B?S0liYklIS0lqUS9ReDJkdm42c0RiNjlGSk41NkhFbnowNk1vUXpjdnFKWTJr?=
 =?utf-8?B?UVFFK0RxcnM3VndHOGExNUtveTNKL25QYmI3ckdBdk1NSEQ1by9NL2RLZlpa?=
 =?utf-8?B?MXVibmZjU0ExR09YZVdHZWpkbW16Y1ZpWUJaYkIwRUtGZDdtRTduV1FWWHgx?=
 =?utf-8?B?S3V5WjVNOHNDaTZudTBMSWFMRWE2MnNFdlZZbzdPRzNYVjk3WHUvRXlXNkdp?=
 =?utf-8?B?MHpXb21udHVpNWw5TVBaM09VZ2NJNGxKZGJ3b29hN3k0UXUvLzh3NzhaeGNX?=
 =?utf-8?B?ZTVrVkVJaU80bmNXR1FWcXA0ckZFdkdhKzVXOExEeThFZTJFelhOZEFXdHlq?=
 =?utf-8?B?ck9xN1ZRYjRiYUU1QjhHS2F4cnQvQ3NTQTlnZ3l5SEoxeUZseG9XdUNuMmRH?=
 =?utf-8?B?SHN5NU1lN2NwZlNZcllxWGs5cFV5YkNpSWUxSGJiOTdyL1cvcStQdUNWUjhy?=
 =?utf-8?B?RlhGaFNTc0cyTmFqNWVGUzgwQzVQTnUyMTZJYm5KQzNoM2RveFZvbDE5RjhX?=
 =?utf-8?B?NU9GTlNEdFVtUHozc2pCUHJnU2dubXhpSDczRW05eS9nbFY4WDF4M0lPeTg5?=
 =?utf-8?B?emRnR21ZTGhSQUI1QktmQytGbTFWSlk3VTBWb2g0WnZkaXJ1RFRmWWw3MWli?=
 =?utf-8?B?Tm4zbk9CRmR2dXl5MVk0clN6cUx5S1o3cE1tc0VHbjFoMEpRazJnUE92VlZt?=
 =?utf-8?B?Q0RXVi91NlVzTzQ1KzNFODZXeFBXaDVoYzFrSlhoeUdXdVNqVUJGdWozOGRv?=
 =?utf-8?B?bXBKajk2bklPcGpydEpZOVZ6dFc0eGFPT0JEcXNaRkE3TVJvQ3NkRk14SVA1?=
 =?utf-8?B?Si8rbWlOWUdWTFNFeFFEVW5lc09vTVAyNFNVNTN1aTZMQTJxL0hYZm03RVlO?=
 =?utf-8?B?UVhTYnVQcXBMbDBRNHFPN3o4bVQzWFA5eXFsWTh5OFV0dUNVSUFJU09pQjlo?=
 =?utf-8?B?MWVMOC9GRlFCZ0RFTXFFemdEYktVUjd5ZHVwTUhabFY5L1NGZzVCM2c0N0xs?=
 =?utf-8?B?ZXBiUUJHSHFXS2RVY0xYdTFWTkI3VnY2NzJveE1xenpZekN5M3FZdnpJOURJ?=
 =?utf-8?B?THZHbUxXNERrVzFZYllEWWF2TzlLRURhTU04VkU2d1JxZDFDcEEzRnlLdys2?=
 =?utf-8?B?R25kZ0V3QWV1dzY1TnZyb2IrdnZGa3FiNllxaDh3eVYxaEtGbnVxSWY4eXk4?=
 =?utf-8?B?Vkh5dWR2Zzk5UElZbjE5RGQ1V3ZaUVo2TEJCWFlhWXppNEJBZnNibkw1TTdn?=
 =?utf-8?Q?4NzA/UeUvTDDK?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 02:57:39.4427
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f7612c30-13d8-4096-15fe-08dc78789c71
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044FA.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6137

Hi All,

Gentle ping since it has been a couple of months, any comments on this 
updated patch? Thanks!

Kind regards,
Henry

On 3/21/2024 11:57 AM, Henry Wang wrote:
> In the common sysctl command XEN_SYSCTL_physinfo, the value of
> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
> Currently on Arm this is a fixed value 1 (can be checked via xl info),
> which is not correct. This is because during the Arm CPU online
> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
> cpu_core_mask for itself.
>
> cores_per_socket refers to the number of cores that belong to the same
> socket (NUMA node). Currently Xen on Arm does not support physical
> CPU hotplug and NUMA, also we assume there is no multithread. Therefore
> cores_per_socket means all possible CPUs detected from the device
> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
> accordingly. Modify the in-code comment which seems to be outdated. Add
> a warning to users if Xen is running on processors with multithread
> support.
>
> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v3:
> - Use cpumask_copy() to set cpu_core_mask and drop the unnecessary
>    cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu)).
> - In-code comment adjustments.
> - Add a warning for multithread.
> v2:
> - Do not do the multithread check.
> ---
>   xen/arch/arm/smpboot.c | 18 +++++++++++++++---
>   1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a84e706d77..b6268be27a 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -66,7 +66,11 @@ static bool cpu_is_dead;
>   
>   /* ID of the PCPU we're running on */
>   DEFINE_PER_CPU(unsigned int, cpu_id);
> -/* XXX these seem awfully x86ish... */
> +/*
> + * Although multithread is part of the Arm spec, there are not many
> + * processors support multithread and current Xen on Arm assumes there
> + * is no multithread.
> + */
>   /* representing HT siblings of each logical CPU */
>   DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
>   /* representing HT and core siblings of each logical CPU */
> @@ -85,9 +89,13 @@ static int setup_cpu_sibling_map(int cpu)
>            !zalloc_cpumask_var(&per_cpu(cpu_core_mask, cpu)) )
>           return -ENOMEM;
>   
> -    /* A CPU is a sibling with itself and is always on its own core. */
> +    /*
> +     * Currently we assume there is no multithread and NUMA, so
> +     * a CPU is a sibling with itself, and the all possible CPUs
> +     * are supposed to belong to the same socket (NUMA node).
> +     */
>       cpumask_set_cpu(cpu, per_cpu(cpu_sibling_mask, cpu));
> -    cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu));
> +    cpumask_copy(per_cpu(cpu_core_mask, cpu), &cpu_possible_map);
>   
>       return 0;
>   }
> @@ -277,6 +285,10 @@ void __init smp_init_cpus(void)
>           warning_add("WARNING: HMP COMPUTING HAS BEEN ENABLED.\n"
>                       "It has implications on the security and stability of the system,\n"
>                       "unless the cpu affinity of all domains is specified.\n");
> +
> +    if ( system_cpuinfo.mpidr.mt == 1 )
> +        warning_add("WARNING: MULTITHREADING HAS BEEN DETECTED ON THE PROCESSOR.\n"
> +                    "It might impact the security of the system.\n");
>   }
>   
>   unsigned int __init smp_get_max_cpus(void)



From xen-devel-bounces@lists.xenproject.org Mon May 20 05:11:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 05:11:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725814.1130141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8vIl-0007fO-8M; Mon, 20 May 2024 05:11:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725814.1130141; Mon, 20 May 2024 05:11:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8vIl-0007fH-5n; Mon, 20 May 2024 05:11:11 +0000
Received: by outflank-mailman (input) for mailman id 725814;
 Mon, 20 May 2024 05:11:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8vIk-0007f7-3p; Mon, 20 May 2024 05:11:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8vIj-0001Xw-Vh; Mon, 20 May 2024 05:11:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8vIj-00085Y-MQ; Mon, 20 May 2024 05:11:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8vIj-0002bQ-Lu; Mon, 20 May 2024 05:11:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=A8OoqmCg+vpQj79FiC5mtUPuygxz2SpDjeqbaqp0JmE=; b=vmSjxT6EStMHpdq1OjZ7jJw8Ak
	BLwzACkx8+LJC42pDRK1nEHMF7aG1g/ND5bpUuy+0uBxUtsjG+a+86cy5UY1OKSCeCkLGOicCFAcn
	WPxp6pROstHsyS3NYdsFmFkjPMksFjEu/WB/QhtdVw0bHuIG8YlaXg56jKstgB97ot48=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186041-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186041: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a90f1cd105c6c5c246f07ca371d873d35b78c7d9
X-Osstest-Versions-That:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 May 2024 05:11:09 +0000

flight 186041 linux-linus real [real]
flight 186043 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186041/
http://logs.test-lab.xenproject.org/osstest/logs/186043/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt      8 xen-boot                 fail REGR. vs. 186036
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 186036

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 186043-retest
 test-armhf-armhf-xl           8 xen-boot            fail pass in 186043-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 186043 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 186043 never pass
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186043 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186043 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186036
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a90f1cd105c6c5c246f07ca371d873d35b78c7d9
baseline version:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f

Last test of basis   186036  2024-05-18 21:13:52 Z    1 days
Failing since        186038  2024-05-19 03:51:42 Z    1 days    3 attempts
Testing same since   186041  2024-05-19 20:13:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Chen, Tim C" <tim.c.chen@intel.com>
  "Huang, Ying" <ying.huang@intel.com>
  "King, Colin" <colin.king@intel.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Alex Rusuf <yorha.op@gmail.com>
  Alex Shi (tencent) <alexs@kernel.org>
  Alex Shi <alexs@kernel.org>
  Alexandre Ghiti <alexghiti@rivosinc.com
  Alexei Starovoitov <ast@kernel.org>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Axel Rasmussen <axelrasmussen@google.com>
  Baokun Li <libaokun1@huawei.com>
  Baolin Wang <baolin.wang@linux.alibaba.com>
  Baoquan He <bhe@redhat.com>
  Barry Song <v-songbaohua@oppo.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Borislav Petkov <bp@suse.de>
  Breno Leitao <leitao@debian.org>
  Brian Geffon <bgeffon@google.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chang S. Bae <chang.seok.bae@intel.com>
  Cheng Yu <serein.chengyu@huawei.com>
  Chris Li <chrisl@kernel.org>
  Christian Loehle <christian.loehle@arm.com>
  Christoph Hellwig <hch@lst.de>
  Christophe Leroy <christophe.leroy@csgroup.eu>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Thompson <daniel.thompson@linaro.org>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Hildenbrand <david@redhat.com>
  Dawei Li <daweilics@gmail.com>
  Dennis Zhou <dennis@kernel.org>
  Dev Jain <dev.jain@arm.com>
  Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Disha Goel <disgoel@linux.ibm.com>
  Donet Tom <donettom@linux.ibm.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Frank van der Linden <fvdl@google.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  Hao Ge <gehao@kylinos.cn>
  Hao Xiang <hao.xiang@bytedance.com>
  Hariom Panthi <hariom1.p@samsung.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@gmx.de> [parisc]
  Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
  Honggyu Kim <honggyu.kim@sk.com>
  Huang Ying <ying.huang@intel.com>
  Ingo Molnar <mingo@kernel.org>
  James Houghton <jthoughton@google.com>
  Jan Kara <jack@suse.cz>
  Jan Kara <jack@suse.cz>               [jbd2]
  Jeff Layton <jlayton@kernel.org>
  Jinjiang Tu <tujinjiang@huawei.com>
  Joel Granados <j.granados@samsung.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Hubbard <jhubbard@nvidia.com>
  Justin Stitt <justinstitt@google.com>
  K Prateek Nayak <kprateek.nayak@amd.com>
  Kairui Song <kasong@tencent.com>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Klara Modin <klarasmodin@gmail.com>
  Konrad Dybcio <konrad.dybcio@linaro.org> # QC SM8550 QRD
  Lance Yang <ioworker0@gmail.com>
  Len Brown <len.brown@intel.com>
  Li kunyu <kunyu@nfschina.com>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Liu Shixin <liushixin2@huawei.com>
  Long Li <leo.lilong@huawei.com>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Lucas Stach <l.stach@pengutronix.de>
  Luis Henriques <lhenriques@suse.de>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Matthew Wilcox <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Mel Gorman <mgorman@techsingularity.net>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Ellerman <mpe@ellerman.id.au> (powerpc)
  Michael Forney <mforney@mforney.org>
  Michal Hocko <mhocko@suse.com>
  Miguel Ojeda <ojeda@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Nhat Pham <nphamcs@gmail.com>
  Oscar Salvador <osalvador@suse.de>
  Pankaj Raghav <p.raghav@samsung.com>
  Patryk Wlazlyn <patryk.wlazlyn@linux.intel.com>
  Peter Xu <peterx@redhat.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rick Edgecombe <rick.p.edgecombe@intel.com>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Roman Gushchin <roman.gushchin@linux.dev>
  rulinhuang <rulin.huang@intel.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Saurav Shah <sauravshah.31@gmail.com>
  Sean Christopherson <seanjc@google.com>       [KVM]
  SeongJae Park <sj@kernel.org>
  Sergey Senozhatsky <senozhatsky@chromium.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Stephen Smalley <stephen.smalley.work@gmail.com>
  Steve French <stfrench@microsoft.com>
  Sudeep Holla <sudeep.holla@arm.com>
  Suren Baghdasaryan <surenb@google.com>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thorsten Blum <thorsten.blum@toblux.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Usama Arif <usamaarif642@gmail.com>
  Vincent Guittot <vincent.guittot@linaro.org>
  Vishal Moola (Oracle) <vishal.moola@gmail.com>
  Vishal Verma <vishal.l.verma@intel.com>
  Vitalii Bursov <vitaly@bursov.com>
  Vladimir Benes <vbenes@redhat.com>
  Vlastimil Babka <vbabka@suse.cz>
  Waiman Long <longman@redhat.com>
  Wei Yang <richard.weiyang@gmail.com>
  Xiu Jianfeng <xiujianfeng@huawei.com>
  Yajun Deng <yajun.deng@linux.dev>
  Ye Bin <yebin10@huawei.com>
  York Jasper Niebuhr <yjnworkstation@gmail.com>
  Yosry Ahmed <yosryahmed@google.com>
  Yu Zhao <yuzhao@google.com>
  Zhang Qiao <zhangqiao22@huawei.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhang Yi <yi.zhang@huawei.com>
  ZhangPeng <zhangpeng362@huawei.com>
  Zi Yan <ziy@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 17098 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 20 07:46:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 07:46:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725840.1130151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8xiv-0000O4-0Y; Mon, 20 May 2024 07:46:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725840.1130151; Mon, 20 May 2024 07:46:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8xiu-0000Nx-TJ; Mon, 20 May 2024 07:46:20 +0000
Received: by outflank-mailman (input) for mailman id 725840;
 Mon, 20 May 2024 07:46:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ovh5=MX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s8xit-0000Nq-Cl
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 07:46:19 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0af8c875-167d-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 09:46:18 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5766cd9ca1bso1475019a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 00:46:18 -0700 (PDT)
Received: from [192.168.226.248] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c323887sm14838455a12.89.2024.05.20.00.46.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 00:46:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0af8c875-167d-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716191178; x=1716795978; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=iV+0g0/GQ8wUbJqz6bMOVhIdbF+12LObjVNNzEiUTgU=;
        b=Hl2OW1bj6Zt+L+OIPEhVIMvloN00bVOaDJsKlsd25hHHMici/QxhO/dsmrr/0GmTeB
         2keOOpsSVocZgNl3XTcy6VyJt9vMuaxhf8ycFZ0THDOPOk5spgzxfQ/4HULHs4+vrORO
         8lDqSyr7mrXa9ghw4ifbNfeDUEce5JHjHwn2EkbL70pEw6tp7ctQM8x+ZMC/GbxNL3N2
         93/f6JinXFbLdMrlZCX7pOYmGSvgJL01PZCY4UwWLfPBqvU5Ns6npKU7T6v1i0uah9Fw
         cLaX1q4lTC9HqkqSlTj2qDxMcGWcAsdTgkOKxzpK1eWAivXKtnr3b0HR8LnuR06TRrUI
         xonw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716191178; x=1716795978;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=iV+0g0/GQ8wUbJqz6bMOVhIdbF+12LObjVNNzEiUTgU=;
        b=XyhA0U3nF8PJ8EDwjnPKQSE3EpbzQ7rurzb6aOsjgHutXeeR+4AMyCEVzCBIilBagv
         EVv1p63qXEbo7WbvgZ1J4vunUnXQEEk79S/AKiNjbtVjGyggM86EaLCCfS+aaqvjZRoL
         IWzHIBR6uDt0fnqbpkHPoJ23AyOozCvWgliSHJfxEvz/BNjQxS/URYibhWsU1VpAJRYL
         aKRPQnw3i48DNaQ0DE1GT18SrArNcvQKCQiLQyVl5xLZVGomKyj+W65Yy44vQ4Kz8vrA
         dR+ijWgj9wP0W0YVOfh1RY3LwO+jpA7vdugktCidsT+/h4cTBoogjOioP8gZ+D9h2Pyo
         vXQw==
X-Forwarded-Encrypted: i=1; AJvYcCVFZTH14KMZJ5LU1/LB/RPbDloeFGkF+3fqbreN1RIVa1fQ9aWlwCqUNsVh66yD9dn4tCkjJArCimkaKCxkn4tmpTB8IXzBZOPrjq5QWEY=
X-Gm-Message-State: AOJu0YwrR42uDk6luYy1TH6uQP0c3uvWjN3f6DWKA4da0l4ppCmhy84J
	JNoLMUor7U25ZPF2uYfWwnxfZ5wvnlJsUkBTyu/1tzmKWM9zweF/
X-Google-Smtp-Source: AGHT+IHdO+7a9m7ODEgZ87P2NlJggK6m3Ogi2+lwWBOHo3sR7ZRkleEvVifd1jG7Yikt4Y+v225UjQ==
X-Received: by 2002:a50:a456:0:b0:56f:e585:2388 with SMTP id 4fb4d7f45d1cf-5734d6de9a1mr19332993a12.36.1716191177624;
        Mon, 20 May 2024 00:46:17 -0700 (PDT)
Message-ID: <4a5ac8daf5d16662a0233771a625b8313f0fecc7.camel@gmail.com>
Subject: Re: [PATCH for-4.19] x86/msi: prevent watchdog triggering when
 dumping MSI state
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>
Date: Mon, 20 May 2024 09:46:16 +0200
In-Reply-To: <20240517135605.94781-1-roger.pau@citrix.com>
References: <20240517135605.94781-1-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Fri, 2024-05-17 at 15:56 +0200, Roger Pau Monne wrote:
> Use the same check that's used in dump_irqs().
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> =C2=A0xen/arch/x86/msi.c | 4 ++++
> =C2=A01 file changed, 4 insertions(+)
>=20
> diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> index 19830528b65a..0c97fbb3fc03 100644
> --- a/xen/arch/x86/msi.c
> +++ b/xen/arch/x86/msi.c
> @@ -17,6 +17,7 @@
> =C2=A0#include <xen/param.h>
> =C2=A0#include <xen/pci.h>
> =C2=A0#include <xen/pci_regs.h>
> +#include <xen/softirq.h>
> =C2=A0#include <xen/iocap.h>
> =C2=A0#include <xen/keyhandler.h>
> =C2=A0#include <xen/pfn.h>
> @@ -1451,6 +1452,9 @@ static void cf_check dump_msi(unsigned char
> key)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long flags;
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const char *type =3D "??=
?";
> =C2=A0
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( !(irq & 0x1f) )
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 proce=
ss_pending_softirqs();
> +
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( !irq_desc_initializ=
ed(desc) )
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
continue;
> =C2=A0

Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon May 20 08:22:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 08:22:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725863.1130165 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yHP-0006J3-NL; Mon, 20 May 2024 08:21:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725863.1130165; Mon, 20 May 2024 08:21:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yHP-0006Iw-Ke; Mon, 20 May 2024 08:21:59 +0000
Received: by outflank-mailman (input) for mailman id 725863;
 Mon, 20 May 2024 08:21:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s8yHN-0006HY-Vl
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 08:21:58 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 052d786d-1682-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 10:21:57 +0200 (CEST)
Received: from CY8PR12CA0054.namprd12.prod.outlook.com (2603:10b6:930:4c::17)
 by CY8PR12MB7292.namprd12.prod.outlook.com (2603:10b6:930:53::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 08:21:54 +0000
Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com
 (2603:10b6:930:4c:cafe::c) by CY8PR12CA0054.outlook.office365.com
 (2603:10b6:930:4c::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 08:21:54 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 08:21:53 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 03:21:53 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 03:21:52 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 03:21:50 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 052d786d-1682-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KckXvErkpYY9uKTniKW7Mva3zDb5luNOrBf3gN4UuwaLqE089usYYWB3MtIwimbvCsE64yo/DqzL6azrzwYA0s3zREuzfVBmeBH2BDWZ8Ekr/s0INqNfKneAobTpDZfRHxOvC6/JBNaZpG/hW3kEYaI6Wmf46eqtlpJB/S9BF0Ev8XQO4b+JzBqjiWPdAZfkalbzA7ppExGU5W7kw6pyzjSknvCf8SGpLOoquOJabZif39LUsXE7sE4pxKbX3Cz6fbehsMWBExxXsQsiHI/qo3cSc0YrTJmCKnve54eoMnYm6gofDskoQoh9dQRLKtiWbdSEphIComVa6xjCtSYcsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UbqrXnoQt1tDj70V69JT90VeEkTrc+nsLeY+/Ia23sc=;
 b=nvGt2SkNCGctfcdfTq5zn+GeK/cAQSL7UlURPCT2brL305KUGDQ0khMRjs8dHT2h1nRSZyUnuesdJGrnO3XdTBXK6tQczHSOQ/H/lz9lbttKVLNQGgFqPnZlZZgz3fGwZMIN8wyAeu+RWPXbkpKPMk02emw8jJmkL44ctjuiPYwYjPAQcZuFvKrucZpeKTRle7PiWYBlehSAZyDnQoEAYwLYPo3buV4x7UtSVFg9Y3tzgQXDfX54oGByEzM7gzPctkdapfA+2PGRuS1sEAnXonCc3lEcx5V6yORtz5fZlQNc3iE74TCzrI+Azh6O+GiZ7vQ7yjPRErPYlmWHuakJXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UbqrXnoQt1tDj70V69JT90VeEkTrc+nsLeY+/Ia23sc=;
 b=3ehfNfV3HNWVRpMdQ4QoMOzpLOHDO3ExHaslwDfFQSPKRHx1GjXOwx+5fH0mVxgiRlsq6KSUO44CAG+07wlfAKH3HsDBx6FRZXIQzzlTAkga+6Ppz7WTEMwoKQMVUFSxgysbW+hptnr/8t46Da5qRrC9ZbYQTFNSHA5FETh4mEM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>, Anthony PERARD <anthony@xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH] tools/golang: Add missing golang bindings for vlan
Date: Mon, 20 May 2024 16:21:45 +0800
Message-ID: <20240520082145.1571451-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|CY8PR12MB7292:EE_
X-MS-Office365-Filtering-Correlation-Id: 5cf4bfd3-2c01-4f76-ed01-08dc78a5e816
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|82310400017|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?GaTe1HXb79Vh//ZEBx2k7DGXtHENqlGOHhPsg7CldIJRFhmNwS7KOncVSTCQ?=
 =?us-ascii?Q?/o9cRntLb+qfIWPyiozKyxbChfdjVex6Czd+l2JQ8Z7kvvNUmi5B+90xzfqh?=
 =?us-ascii?Q?2S6aTn74BVrpWI+j8JzVjXMYXy0EmelA59XQwou6JhnxM8GNpG3SqTLAXtTk?=
 =?us-ascii?Q?iJqtGqPzHJlYMojL9sqJvffYczSOgCp5GmB+XGJcyW5fvXczhnDY1EgkrMvL?=
 =?us-ascii?Q?NFOfLuUl6ZZ5vjQQ8ZJ0QuLV3IfC0LXGwxPSRWu1jgmvPTKkyRRmNwi8BGY4?=
 =?us-ascii?Q?fvA/959ijXU5p78VxA6TqDv7bLMsHw7FV8xblpHDXyiliGZr6IKTXMj8SV1C?=
 =?us-ascii?Q?Tb+Xcwunht67AfDgG8Ag9LA/Pzf1ZSYn/y98sSNzF5LFopSIAMZeCNztDKmo?=
 =?us-ascii?Q?W1DZ6Lvu685+JA2C53t7IG/JhqJDL9nAjocsQaylRFl5J1vLWqsOmv86R9IV?=
 =?us-ascii?Q?HmH7JwvraZ95HdmuENz8elKJx5G3AGAhJCz+aa2etoFU3WiWlC8zqds3IvtF?=
 =?us-ascii?Q?G25c9MHrFH8GQukkg0y/Lb1seQNCKU2pCM4SGaEAokc0YXjP1LChpYe5E68y?=
 =?us-ascii?Q?SJRQJv5r0et5CNpFO8s9gfAGenmV07d4se5uDr8SSjibcpK3RHq+RoK+LHys?=
 =?us-ascii?Q?xkc3SzImvwKPlH4IFdOvXqp4cTxtpuxnlk/psrs3opXB4A+AjHy8FD4idBap?=
 =?us-ascii?Q?Nf4Hd0v8/FOiSQVChUZvTnoPvGYsgYZvoMZeyWeG6lcP6mMQuU+vm7P2vYak?=
 =?us-ascii?Q?12mqfQu4h+AS678R8iWVZBEFJCF5eARedczbJ062xINRTZLc4vqzXZ1UXGZ1?=
 =?us-ascii?Q?jJ3YoYxtbJs21sxL+POW+avXTP04f95zr1d37vVxsHgU7YkCkJk5en7//R4F?=
 =?us-ascii?Q?XnQPIEnOpDrJU33HVYD+9euf3jIvzIJ0gRdqO7tDLcQ2Xf6fxrsgbnacbSa2?=
 =?us-ascii?Q?fSDtcs9Uic2I7EeId3gANDYu33W6wFCfQ7v/KhQjkXuTFKqBD3CS1Sbn/M4+?=
 =?us-ascii?Q?BifAq6h8Y9EghXjDAQ7YZqeYCHm6DjmRhVHRH3ElFS7/zBPxP7REAJ+2sw6r?=
 =?us-ascii?Q?lIu96+twxEwJRL+k1j4I6zp2hD8lqIxrZAavQ2RN4s6KjbHV3FxU8DKwcK9o?=
 =?us-ascii?Q?A49rHl1OXHtaN8ZN+Jktstm47CImb7HIHAvri29THynrYlnO6pvSm89FuwYs?=
 =?us-ascii?Q?6rTtzLoLLVeFmJqjnVp2ZmGOmvVq0fyLqO3NR71yGUj8enFtuKWYZVpbL2VZ?=
 =?us-ascii?Q?cwqEgENMN1PZaTUkTrA7WTe/3MPI6wkmYqRBsLvtP9c1AIGtcXR8Zcd6xLdc?=
 =?us-ascii?Q?M9fcudN+kLhTHzeFKhWsP93keGF/JDt+2FPrnuGd9+YfVVuxq7byIx6eWnSN?=
 =?us-ascii?Q?XFHHvG8=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(82310400017)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 08:21:53.7583
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5cf4bfd3-2c01-4f76-ed01-08dc78a5e816
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD5.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7292

It is noticed that commit:
3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
introduces a new "vlan" string field to libxl_device_nic. But the
golang bindings are missing. Add it in this patch.

Fixes: 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
The code is automatically generated by:
```
./configure
make tools
```
---
 tools/golang/xenlight/helpers.gen.go | 3 +++
 tools/golang/xenlight/types.gen.go   | 1 +
 2 files changed, 4 insertions(+)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 78bdb08b15..b9cb5b33c7 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1963,6 +1963,7 @@ func (x *DeviceNic) fromC(xc *C.libxl_device_nic) error {
 x.BackendDomname = C.GoString(xc.backend_domname)
 x.Devid = Devid(xc.devid)
 x.Mtu = int(xc.mtu)
+x.Vlan = C.GoString(xc.vlan)
 x.Model = C.GoString(xc.model)
 if err := x.Mac.fromC(&xc.mac);err != nil {
 return fmt.Errorf("converting field Mac: %v", err)
@@ -2040,6 +2041,8 @@ if x.BackendDomname != "" {
 xc.backend_domname = C.CString(x.BackendDomname)}
 xc.devid = C.libxl_devid(x.Devid)
 xc.mtu = C.int(x.Mtu)
+if x.Vlan != "" {
+xc.vlan = C.CString(x.Vlan)}
 if x.Model != "" {
 xc.model = C.CString(x.Model)}
 if err := x.Mac.toC(&xc.mac); err != nil {
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index ccfe18019e..5b293755d7 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -756,6 +756,7 @@ BackendDomid Domid
 BackendDomname string
 Devid Devid
 Mtu int
+Vlan string
 Model string
 Mac Mac
 Ip string
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 20 08:25:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 08:25:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725879.1130178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yKh-0006x4-6A; Mon, 20 May 2024 08:25:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725879.1130178; Mon, 20 May 2024 08:25:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yKh-0006wx-3N; Mon, 20 May 2024 08:25:23 +0000
Received: by outflank-mailman (input) for mailman id 725879;
 Mon, 20 May 2024 08:25:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s8yKf-0006wr-MH
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 08:25:21 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7d102f0e-1682-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 10:25:17 +0200 (CEST)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-42016c8db2aso17795995e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 01:25:17 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41ff063d8cesm378652455e9.46.2024.05.20.01.25.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 01:25:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d102f0e-1682-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716193517; x=1716798317; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=qMroFsj3THTwl/uzIoBxBtxkfAtMnfitk+pNuz5SD4Y=;
        b=Wa1kNDPUcFMqP+trZH8mM4vAA2B5gP1+8H6Za2Yy8Zt4JLpYSVRg0+x0ymwPXEB5ZR
         +1TsxHd3/PFXndTItmZ9omtLM5oHuT+u78r3QP5cwQuuaELaRbnbp2Epe6nUpKLYzZQg
         gchZOXRJCMj6AMdjRssTvm8C/tgN61gncavIg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716193517; x=1716798317;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qMroFsj3THTwl/uzIoBxBtxkfAtMnfitk+pNuz5SD4Y=;
        b=P4FyD8103R+slLirxSyfWFi+Th+xS7173maHn5g4bOBp92IVRFbALL51Q85b18Tc0j
         pnxYXoXsUl0cQowOKfWtuUcUMrmo4KDLZk99HnpwW6RxdfirnxNGlglRMjHkitbRVh8K
         GTLzK5HHmobR2UOrHJZviLgdVz82Gxx3kb2IYB/EAEHQV25qzrIeC/vouNBr2AcE5lt1
         yVC6MbnXKQVpKNevQJxbh8xyIGHRCc5PF/Vi46jN+cUhBFfSINE1uNBFQ4EKJxH/A68k
         8s7pcavSwHbAFPmx1we3CT28oEOsKR09UFviDTK3bUCd5r4CeWvgn0YqnobCthai/9mC
         SlqA==
X-Gm-Message-State: AOJu0YwsLzbIXQvGM4+UhHP1bXdGy24Tr+UpJ4f3PU5duDPR8/cQMiHZ
	WkX4JzUeekT3JCPfze5+McFj+4hwXNWcYLTpdJcqcgrvA19A/8Kpe5ruIGZh55I=
X-Google-Smtp-Source: AGHT+IEC+9LeHqdNTW17YsDJoY7pHS+AegyGnzYE0m+DOIwcNJPDgt+jBcDdNmpYhv65H9d4c6UGUA==
X-Received: by 2002:a05:600c:1d99:b0:420:16b2:67f4 with SMTP id 5b1f17b1804b1-42016b26a4emr183060635e9.12.1716193516449;
        Mon, 20 May 2024 01:25:16 -0700 (PDT)
Date: Mon, 20 May 2024 10:25:15 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Petr =?utf-8?B?QmVuZcWh?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: Re: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m
 views configurable
Message-ID: <ZksI6zJz53EdSmYp@macbook>
References: <cover.1716029860.git.w1benny@gmail.com>
 <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>

On Sat, May 18, 2024 at 11:02:15AM +0000, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
> 
> This commit introduces the ability to configure the maximum number of altp2m
> views for the domain during its creation. Previously, the limits were hardcoded
> to a maximum of 10. This change allows for greater flexibility in environments
> that require more or fewer altp2m views.
> 
> The maximum configurable limit for max_altp2m on x86 is now set to MAX_EPTP
> (512). This cap is linked to the architectural limit of the EPTP-switching
> VMFUNC, which supports up to 512 entries. Despite there being no inherent need
> for limiting max_altp2m in scenarios not utilizing VMFUNC, decoupling these
> components would necessitate substantial code changes.
> 
> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> ---
>  xen/arch/x86/domain.c             | 12 ++++
>  xen/arch/x86/hvm/hvm.c            |  8 ++-
>  xen/arch/x86/hvm/vmx/vmx.c        |  2 +-
>  xen/arch/x86/include/asm/domain.h |  7 +--
>  xen/arch/x86/include/asm/p2m.h    |  6 +-
>  xen/arch/x86/mm/altp2m.c          | 91 +++++++++++++++++++------------
>  xen/arch/x86/mm/hap/hap.c         |  6 +-
>  xen/arch/x86/mm/mem_access.c      | 24 ++++----
>  xen/arch/x86/mm/mem_sharing.c     |  2 +-
>  xen/arch/x86/mm/p2m-ept.c         | 12 ++--
>  xen/arch/x86/mm/p2m.c             |  8 +--
>  xen/common/domain.c               |  1 +
>  xen/include/public/domctl.h       |  5 +-
>  xen/include/xen/sched.h           |  2 +
>  14 files changed, 116 insertions(+), 70 deletions(-)
> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index 00a3aaa576..3bd18cb2d0 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -685,6 +685,18 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }

You also need to adjust arch_sanitise_domain_config() in ARM to return
an error if nr_altp2m is set, as there's no support for altp2m on ARM
yet.

> 
> +    if ( config->nr_altp2m && !hvm_altp2m_supported() )
> +    {
> +        dprintk(XENLOG_INFO, "altp2m requested but not available\n");
> +        return -EINVAL;
> +    }
> +
> +    if ( config->nr_altp2m > MAX_EPTP )
> +    {
> +        dprintk(XENLOG_INFO, "nr_altp2m must be <= %lu\n", MAX_EPTP);
> +        return -EINVAL;
> +    }
> +
>      if ( config->vmtrace_size )
>      {
>          unsigned int size = config->vmtrace_size;
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 9594e0a5c5..77e4016bdb 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -4639,6 +4639,12 @@ static int do_altp2m_op(
>          goto out;
>      }
> 
> +    if ( d->nr_altp2m == 0 )

I would merge this with the previous check, which also returns
-EINVAL.

> +    {
> +        rc = -EINVAL;
> +        goto out;
> +    }
> +
>      if ( (rc = xsm_hvm_altp2mhvm_op(XSM_OTHER, d, mode, a.cmd)) )
>          goto out;
> 
> @@ -5228,7 +5234,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)
>      if ( !hvm_is_singlestep_supported() )
>          return;
> 
> -    if ( p2midx >= MAX_ALTP2M )
> +    if ( p2midx >= v->domain->nr_altp2m )
>          return;
> 
>      v->arch.hvm.single_step = true;
> diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
> index 5f67a48592..76ee09b701 100644
> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -4888,7 +4888,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
>          {
>              unsigned int i;
> 
> -            for ( i = 0; i < MAX_ALTP2M; ++i )
> +            for ( i = 0; i < currd->nr_altp2m; ++i )
>              {
>                  if ( currd->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
>                      continue;
> diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
> index f5daeb182b..3935328781 100644
> --- a/xen/arch/x86/include/asm/domain.h
> +++ b/xen/arch/x86/include/asm/domain.h
> @@ -258,11 +258,10 @@ struct paging_vcpu {
>      struct shadow_vcpu shadow;
>  };
> 
> -#define MAX_NESTEDP2M 10
> -
> -#define MAX_ALTP2M      10 /* arbitrary */
>  #define INVALID_ALTP2M  0xffff
>  #define MAX_EPTP        (PAGE_SIZE / sizeof(uint64_t))
> +#define MAX_NESTEDP2M   10
> +
>  struct p2m_domain;
>  struct time_scale {
>      int shift;
> @@ -353,7 +352,7 @@ struct arch_domain
> 
>      /* altp2m: allow multiple copies of host p2m */
>      bool altp2m_active;
> -    struct p2m_domain *altp2m_p2m[MAX_ALTP2M];
> +    struct p2m_domain **altp2m_p2m;
>      mm_lock_t altp2m_list_lock;
>      uint64_t *altp2m_eptp;
>      uint64_t *altp2m_visible_eptp;
> diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
> index 111badf89a..e66c081149 100644
> --- a/xen/arch/x86/include/asm/p2m.h
> +++ b/xen/arch/x86/include/asm/p2m.h
> @@ -881,7 +881,7 @@ static inline struct p2m_domain *p2m_get_altp2m(struct vcpu *v)
>      if ( index == INVALID_ALTP2M )
>          return NULL;
> 
> -    BUG_ON(index >= MAX_ALTP2M);
> +    BUG_ON(index >= v->domain->nr_altp2m);
> 
>      return v->domain->arch.altp2m_p2m[index];
>  }
> @@ -891,7 +891,7 @@ static inline bool p2m_set_altp2m(struct vcpu *v, unsigned int idx)
>  {
>      struct p2m_domain *orig;
> 
> -    BUG_ON(idx >= MAX_ALTP2M);
> +    BUG_ON(idx >= v->domain->nr_altp2m);
> 
>      if ( idx == vcpu_altp2m(v).p2midx )
>          return false;
> @@ -943,7 +943,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
>                                  p2m_type_t p2mt, p2m_access_t p2ma);
> 
>  /* Set a specific p2m view visibility */
> -int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> +int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int idx,
>                                     uint8_t visible);
>  #else /* !CONFIG_HVM */
>  struct p2m_domain *p2m_get_altp2m(struct vcpu *v);
> diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
> index 6fe1e9ed6b..5cb71c8b8e 100644
> --- a/xen/arch/x86/mm/altp2m.c
> +++ b/xen/arch/x86/mm/altp2m.c
> @@ -15,6 +15,11 @@
>  void
>  altp2m_vcpu_initialise(struct vcpu *v)
>  {
> +    struct domain *d = v->domain;
> +
> +    if ( d->nr_altp2m == 0 )
> +        return;
> +
>      if ( v != current )
>          vcpu_pause(v);
> 
> @@ -30,8 +35,12 @@ altp2m_vcpu_initialise(struct vcpu *v)
>  void
>  altp2m_vcpu_destroy(struct vcpu *v)
>  {
> +    struct domain *d = v->domain;
>      struct p2m_domain *p2m;
> 
> +    if ( d->nr_altp2m == 0 )
> +        return;
> +
>      if ( v != current )
>          vcpu_pause(v);
> 
> @@ -122,7 +131,12 @@ int p2m_init_altp2m(struct domain *d)
>      struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> 
>      mm_lock_init(&d->arch.altp2m_list_lock);
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, d->nr_altp2m);
> +
> +    if ( !d->arch.altp2m_p2m )
> +        return -ENOMEM;
> +
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          d->arch.altp2m_p2m[i] = p2m = p2m_init_one(d);
>          if ( p2m == NULL )
> @@ -143,7 +157,10 @@ void p2m_teardown_altp2m(struct domain *d)
>      unsigned int i;
>      struct p2m_domain *p2m;
> 
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    if ( !d->arch.altp2m_p2m )
> +        return;
> +
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          if ( !d->arch.altp2m_p2m[i] )
>              continue;
> @@ -151,6 +168,8 @@ void p2m_teardown_altp2m(struct domain *d)
>          d->arch.altp2m_p2m[i] = NULL;
>          p2m_free_one(p2m);
>      }
> +
> +    XFREE(d->arch.altp2m_p2m);
>  }
> 
>  int altp2m_get_effective_entry(struct p2m_domain *ap2m, gfn_t gfn, mfn_t *mfn,
> @@ -200,7 +219,7 @@ bool p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
>      struct domain *d = v->domain;
>      bool rc = false;
> 
> -    if ( idx >= MAX_ALTP2M )
> +    if ( idx >= d->nr_altp2m )
>          return rc;
> 
>      altp2m_list_lock(d);
> @@ -306,8 +325,8 @@ static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
>  {
>      struct p2m_domain *p2m;
> 
> -    ASSERT(idx < MAX_ALTP2M);
> -    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +    ASSERT(idx < d->nr_altp2m);
> +    p2m = d->arch.altp2m_p2m[idx];

If the array_access_nospec() is not required removing it should be a
separate commit.  Also there's no mention of why removing the nospec
is safe in the commit message.

> 
>      p2m_lock(p2m);
> 
> @@ -332,7 +351,7 @@ void p2m_flush_altp2m(struct domain *d)
> 
>      altp2m_list_lock(d);
> 
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
>          d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
> @@ -348,9 +367,9 @@ static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
>      struct p2m_domain *hostp2m, *p2m;
>      int rc;
> 
> -    ASSERT(idx < MAX_ALTP2M);
> +    ASSERT(idx < d->nr_altp2m);
> 
> -    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +    p2m = d->arch.altp2m_p2m[idx];

Same here.

>      hostp2m = p2m_get_hostp2m(d);
> 
>      p2m_lock(p2m);
> @@ -388,12 +407,12 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
>      int rc = -EINVAL;
>      struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
> 
> -    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> +    if ( idx >= d->nr_altp2m )
>          return rc;
> 
>      altp2m_list_lock(d);
> 
> -    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> +    if ( d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
>           mfn_x(INVALID_MFN) )
>          rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);
> 
> @@ -415,7 +434,7 @@ int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
> 
>      altp2m_list_lock(d);
> 
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
>              continue;
> @@ -437,7 +456,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
>      struct p2m_domain *p2m;
>      int rc = -EBUSY;
> 
> -    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
> +    if ( !idx || idx >= d->nr_altp2m )
>          return rc;
> 
>      rc = domain_pause_except_self(d);
> @@ -447,17 +466,17 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
>      rc = -EBUSY;
>      altp2m_list_lock(d);
> 
> -    if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
> +    if ( d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] !=
>           mfn_x(INVALID_MFN) )
>      {
> -        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +        p2m = d->arch.altp2m_p2m[array_index_nospec(idx, d->nr_altp2m)];
> 
>          if ( !_atomic_read(p2m->active_vcpus) )
>          {
>              p2m_reset_altp2m(d, idx, ALTP2M_DEACTIVATE);
> -            d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] =
> +            d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] =
>                  mfn_x(INVALID_MFN);
> -            d->arch.altp2m_visible_eptp[array_index_nospec(idx, MAX_EPTP)] =
> +            d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
>                  mfn_x(INVALID_MFN);
>              rc = 0;
>          }
> @@ -475,7 +494,7 @@ int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
>      struct vcpu *v;
>      int rc = -EINVAL;
> 
> -    if ( idx >= MAX_ALTP2M )
> +    if ( idx >= d->nr_altp2m )
>          return rc;
> 
>      rc = domain_pause_except_self(d);
> @@ -510,13 +529,13 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
>      mfn_t mfn;
>      int rc = -EINVAL;
> 
> -    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -         d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
> +    if ( idx >= d->nr_altp2m ||
> +         d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
>           mfn_x(INVALID_MFN) )
>          return rc;
> 
>      hp2m = p2m_get_hostp2m(d);
> -    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
> +    ap2m = d->arch.altp2m_p2m[array_index_nospec(idx, d->nr_altp2m)];
> 
>      p2m_lock(hp2m);
>      p2m_lock(ap2m);
> @@ -572,7 +591,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
> 
>      altp2m_list_lock(d);
> 
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          p2m_type_t t;
>          p2m_access_t a;
> @@ -595,7 +614,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
>              else
>              {
>                  /* At least 2 altp2m's impacted, so reset everything */
> -                for ( i = 0; i < MAX_ALTP2M; i++ )
> +                for ( i = 0; i < d->nr_altp2m; i++ )
>                  {
>                      if ( i == last_reset_idx ||
>                           d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
> @@ -659,12 +678,13 @@ int p2m_set_suppress_ve_multi(struct domain *d,
> 
>      if ( sve->view > 0 )
>      {
> -        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
> +        if ( sve->view >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(sve->view, d->nr_altp2m)] ==
>               mfn_x(INVALID_MFN) )
>              return -EINVAL;
> 
> -        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
> +        p2m = ap2m =
> +            d->arch.altp2m_p2m[array_index_nospec(sve->view, d->nr_altp2m)];

This expression is so common that it might be helpful to introduce a
helper: altp2m_get_p2m() or similar that encapsulates it (in a
separate patch).

>      }
> 
>      p2m_lock(host_p2m);
> @@ -727,12 +747,13 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
> 
>      if ( altp2m_idx > 0 )
>      {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> +        if ( altp2m_idx >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)] ==
>               mfn_x(INVALID_MFN) )
>              return -EINVAL;
> 
> -        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +        p2m = ap2m =
> +            d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
>      }
>      else
>          p2m = host_p2m;
> @@ -754,7 +775,7 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
>      return rc;
>  }
> 
> -int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> +int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int idx,
>                                     uint8_t visible)
>  {
>      int rc = 0;
> @@ -763,17 +784,17 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
> 
>      /*
>       * Eptp index is correlated with altp2m index and should not exceed
> -     * min(MAX_ALTP2M, MAX_EPTP).
> +     * d->nr_altp2m.
>       */
> -    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -         d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> +    if ( idx >= d->nr_altp2m ||
> +         d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)] ==
>           mfn_x(INVALID_MFN) )
>          rc = -EINVAL;
>      else if ( visible )
> -        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> -            d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)];
> +        d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
> +            d->arch.altp2m_eptp[array_index_nospec(idx, d->nr_altp2m)];
>      else
> -        d->arch.altp2m_visible_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] =
> +        d->arch.altp2m_visible_eptp[array_index_nospec(idx, d->nr_altp2m)] =
>              mfn_x(INVALID_MFN);
> 
>      altp2m_list_unlock(d);
> diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
> index d2011fde24..501fd9848b 100644
> --- a/xen/arch/x86/mm/hap/hap.c
> +++ b/xen/arch/x86/mm/hap/hap.c
> @@ -515,7 +515,7 @@ int hap_enable(struct domain *d, u32 mode)
>              d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
>          }
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              rv = p2m_alloc_table(d->arch.altp2m_p2m[i]);
>              if ( rv != 0 )
> @@ -538,7 +538,7 @@ void hap_final_teardown(struct domain *d)
>      unsigned int i;
> 
>      if ( hvm_altp2m_supported() )
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>              p2m_teardown(d->arch.altp2m_p2m[i], true, NULL);
> 
>      /* Destroy nestedp2m's first */
> @@ -590,7 +590,7 @@ void hap_teardown(struct domain *d, bool *preempted)
>          FREE_XENHEAP_PAGE(d->arch.altp2m_eptp);
>          FREE_XENHEAP_PAGE(d->arch.altp2m_visible_eptp);
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              p2m_teardown(d->arch.altp2m_p2m[i], false, preempted);
>              if ( preempted && *preempted )
> diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
> index 60a0cce68a..63bb2e10ed 100644
> --- a/xen/arch/x86/mm/mem_access.c
> +++ b/xen/arch/x86/mm/mem_access.c
> @@ -347,12 +347,12 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, uint32_t nr,
>      /* altp2m view 0 is treated as the hostp2m */
>      if ( altp2m_idx )
>      {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> +        if ( altp2m_idx >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)] ==
>               mfn_x(INVALID_MFN) )
>              return -EINVAL;
> 
> -        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
>      }
> 
>      if ( !xenmem_access_to_p2m_access(p2m, access, &a) )
> @@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
>      /* altp2m view 0 is treated as the hostp2m */
>      if ( altp2m_idx )
>      {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> -             mfn_x(INVALID_MFN) )
> +        if ( altp2m_idx >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
> +             == mfn_x(INVALID_MFN) )
>              return -EINVAL;
> 
> -        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
>      }
> 
>      p2m_lock(p2m);
> @@ -466,12 +466,12 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access,
>      }
>      else if ( altp2m_idx ) /* altp2m view 0 is treated as the hostp2m */
>      {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> -             mfn_x(INVALID_MFN) )
> +        if ( altp2m_idx >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
> +             == mfn_x(INVALID_MFN) )
>              return -EINVAL;
> 
> -        p2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
> +        p2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, d->nr_altp2m)];
>      }
> 
>      return _p2m_get_mem_access(p2m, gfn, access);
> @@ -486,7 +486,7 @@ void arch_p2m_set_access_required(struct domain *d, bool access_required)
>      if ( altp2m_active(d) )
>      {
>          unsigned int i;
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
> 
> diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
> index da28266ef0..83bb9dd5df 100644
> --- a/xen/arch/x86/mm/mem_sharing.c
> +++ b/xen/arch/x86/mm/mem_sharing.c
> @@ -912,7 +912,7 @@ static int nominate_page(struct domain *d, gfn_t gfn,
> 
>          altp2m_list_lock(d);
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              ap2m = d->arch.altp2m_p2m[i];
>              if ( !ap2m )
> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
> index f83610cb8c..42b868ca45 100644
> --- a/xen/arch/x86/mm/p2m-ept.c
> +++ b/xen/arch/x86/mm/p2m-ept.c
> @@ -1293,7 +1293,7 @@ static void ept_set_ad_sync(struct domain *d, bool value)
>      {
>          unsigned int i;
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              struct p2m_domain *p2m;
> 
> @@ -1500,15 +1500,17 @@ void setup_ept_dump(void)
> 
>  void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
>  {
> -    struct p2m_domain *p2m = array_access_nospec(d->arch.altp2m_p2m, i);
> +    struct p2m_domain *p2m =
> +        d->arch.altp2m_p2m[array_index_nospec(i, d->nr_altp2m)];
>      struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
>      struct ept_data *ept;
> 
>      p2m->ept.ad = hostp2m->ept.ad;
>      ept = &p2m->ept;
>      ept->mfn = pagetable_get_pfn(p2m_get_pagetable(p2m));
> -    d->arch.altp2m_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
> -    d->arch.altp2m_visible_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
> +    d->arch.altp2m_eptp[array_index_nospec(i, d->nr_altp2m)] = ept->eptp;
> +    d->arch.altp2m_visible_eptp[array_index_nospec(i, d->nr_altp2m)] =
> +        ept->eptp;
>  }
> 
>  unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
> @@ -1519,7 +1521,7 @@ unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
> 
>      altp2m_list_lock(d);
> 
> -    for ( i = 0; i < MAX_ALTP2M; i++ )
> +    for ( i = 0; i < d->nr_altp2m; i++ )
>      {
>          if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
>              continue;
> diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
> index db5d9b6c2a..549aec8d6b 100644
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -105,7 +105,7 @@ void p2m_change_entry_type_global(struct domain *d,
>      {
>          unsigned int i;
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
>              {
> @@ -140,7 +140,7 @@ void p2m_memory_type_changed(struct domain *d)
>      {
>          unsigned int i;
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
>              {
> @@ -913,7 +913,7 @@ void p2m_change_type_range(struct domain *d,
>      {
>          unsigned int i;
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
>              {
> @@ -986,7 +986,7 @@ int p2m_finish_type_change(struct domain *d,
>      {
>          unsigned int i;
> 
> -        for ( i = 0; i < MAX_ALTP2M; i++ )
> +        for ( i = 0; i < d->nr_altp2m; i++ )
>          {
>              if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
>              {
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6773f7fb90..a10a70e9d4 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -610,6 +610,7 @@ struct domain *domain_create(domid_t domid,
>      if ( config )
>      {
>          d->options = config->flags;
> +        d->nr_altp2m = config->nr_altp2m;
>          d->vmtrace_size = config->vmtrace_size;
>      }
> 
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index a33f9ec32b..60a871f123 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -21,7 +21,7 @@
>  #include "hvm/save.h"
>  #include "memory.h"
> 
> -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
> +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
> 
>  /*
>   * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
> @@ -86,6 +86,9 @@ struct xen_domctl_createdomain {
> 
>      uint32_t grant_opts;
> 
> +    /* Number of altp2ms to allocate. */
> +    uint32_t nr_altp2m;

I've got a colliding patch that introduces altp2m_opts, and uses the
bottom 2 bits to signal the altp2m mode.  On EPT this is limited to
512 altp2m, so using 16bits (from the high part of altp2m_opts) should
be enough, hopefully also for other architectures.

> +
>      /* Per-vCPU buffer size in bytes.  0 to disable. */
>      uint32_t vmtrace_size;
> 
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 132b841995..18cc0748a1 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -602,6 +602,8 @@ struct domain
>          unsigned int guest_request_sync          : 1;
>      } monitor;
> 
> +    unsigned int nr_altp2m;    /* Number of altp2m tables */

Shouldn't this new field be placed in arch_domain with the rest of the
altp2m fields?  I know that number of altp2m is an option that's
suitable to be shared between all architectures that have altp2m
implementations, but it's IMO better if it's grouped together with the
rest of the altp2m fields for consistency.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 08:31:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 08:31:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725887.1130190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yQE-0000NB-T5; Mon, 20 May 2024 08:31:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725887.1130190; Mon, 20 May 2024 08:31:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8yQE-0000N4-Oh; Mon, 20 May 2024 08:31:06 +0000
Received: by outflank-mailman (input) for mailman id 725887;
 Mon, 20 May 2024 08:31:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s8yQD-0000Mx-SZ
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 08:31:05 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4bc7e707-1683-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 10:31:04 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e538a264f7so33063841fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 01:31:04 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41ff6fd7e73sm371847995e9.27.2024.05.20.01.31.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 01:31:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bc7e707-1683-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716193863; x=1716798663; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=yqEjJIT9mlNdqu0zBGWobRlv+C0QGKeWgUxXWYwC3Cg=;
        b=eqgR+/r7+EKl2xkwyBdpHUWMHgUrHrQwjzSA5vws0cly6sQe0PGg/B9bFK20tKp8EW
         1J4dLMrOCyFYM2zXa1Nt1Edcw+IpEfRC21+oJfMnGUVlss9fNq5wKxz6/LMplUdmkhNm
         aKaW9j+abF8cmqSP0bZKK+YWz5pID8KX77cVk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716193863; x=1716798663;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yqEjJIT9mlNdqu0zBGWobRlv+C0QGKeWgUxXWYwC3Cg=;
        b=hC44NcNVPVmXVFvymYrwFnzd61S7+dvA34LtMwPmcfbSR65ohIlORgmMj7QeXcOY10
         WXFIVPqXz6mz2l2g0+o2O4zb2Qd+Mp4h/NFoOmZDwvVk80oRYkGRHUqWJScwbCkZ39Hc
         FbyJOVMyA5XoBA79Cyw5NoUl9fqVY2DX/9IMIGqHWTp2BhxeIzczrTSSIbqT5EDCzBGg
         igZ7jOy0CqGXi6taSYZYn5NQEJi/IWtsaND8HJa56p/arHy/JsePZ4xu3pxnaWmT7PW/
         sOW9cmllQhN8HqrqVWIbpQVbdLGaPOfxV9jUm0mas3KMeeE8GuNadmrIjihMIOs3L3l9
         jTJg==
X-Gm-Message-State: AOJu0YytHpZCt6LxiNGNWLxI7yuMbztJrd4Hd5PLjrxXBGbL6d8TTVZP
	BnaYQq1yeIGBD/lP13MPuL3HCN69oDZRuhghIfbzFCrs8BwkMFWim3fmNUkzjVU=
X-Google-Smtp-Source: AGHT+IFy/aIeAjbwW3Y208Wq5B5ZbEUsQ9yZw2tAqtYxnlgG6wKiWlE2muw5Wne31jOdoeS7L6ctPw==
X-Received: by 2002:a2e:9e51:0:b0:2e7:1588:7255 with SMTP id 38308e7fff4ca-2e71588729dmr39982421fa.32.1716193863325;
        Mon, 20 May 2024 01:31:03 -0700 (PDT)
Date: Mon, 20 May 2024 10:31:02 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Petr =?utf-8?B?QmVuZcWh?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH for-4.19? v4 1/6] x86/p2m: Add braces for better code
 clarity
Message-ID: <ZksKRrr5aeWnkK1l@macbook>
References: <cover.1716029860.git.w1benny@gmail.com>
 <36d1488857a595fd6a286ab2de26cd672df3f387.1716029860.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <36d1488857a595fd6a286ab2de26cd672df3f387.1716029860.git.w1benny@gmail.com>

On Sat, May 18, 2024 at 11:02:12AM +0000, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
> 
> No functional change.
> 
> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

TBH I'm fine without the braces, but if lack of them can cause
confusion:

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

CODING_STYLE states that braces can be omitted for blocks with single
statements, I guess we should clarify whether multi-line statements
are accepted, as the example contains a single-line statement.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 08:52:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 08:52:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725903.1130203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8ykm-0003tS-Ii; Mon, 20 May 2024 08:52:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725903.1130203; Mon, 20 May 2024 08:52:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8ykm-0003tL-Ey; Mon, 20 May 2024 08:52:20 +0000
Received: by outflank-mailman (input) for mailman id 725903;
 Mon, 20 May 2024 08:52:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8ykl-0003tB-8f; Mon, 20 May 2024 08:52:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8ykl-00064Y-3d; Mon, 20 May 2024 08:52:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s8ykk-0000M2-Q6; Mon, 20 May 2024 08:52:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s8ykk-0005IW-Pc; Mon, 20 May 2024 08:52:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hNPleHqqc4X5HOs8cNaOGCp0TWNtxCFI3MGbYD2xSak=; b=WEuNFnOtN4lfkCpw7VdBxdSS4A
	mQCN1sZ0MA8aO5/x5e3ikhWp3ig+5K3n2TuGz7DLouD0M+1uwlWbhaJGGgJubo8ljNi6uQnz9jYp5
	Utby8BPzptO3/scvDOc79mRYG8EB/FH7l41VRT8bPzDP2AF41UuFdXwCRgSJSuwIxC+E=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186042-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186042: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
X-Osstest-Versions-That:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 May 2024 08:52:18 +0000

flight 186042 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186042/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1   8 xen-boot         fail in 186037 pass in 186042
 test-armhf-armhf-xl-arndale   8 xen-boot                   fail pass in 186037

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 186037 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 186037 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186037
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186037
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186037
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186037
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186037
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186037
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca
baseline version:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca

Last test of basis   186042  2024-05-20 01:53:34 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon May 20 09:33:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 09:33:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725925.1130217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zOF-0001Vg-Lr; Mon, 20 May 2024 09:33:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725925.1130217; Mon, 20 May 2024 09:33:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zOF-0001VZ-I0; Mon, 20 May 2024 09:33:07 +0000
Received: by outflank-mailman (input) for mailman id 725925;
 Mon, 20 May 2024 09:33:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H0dS=MX=epam.com=prvs=187012ae9b=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s8zOE-0001VT-CU
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 09:33:06 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3f1bf85-168b-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 11:33:03 +0200 (CEST)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44K8xleQ006339;
 Mon, 20 May 2024 09:32:52 GMT
Received: from eur02-am0-obe.outbound.protection.outlook.com
 (mail-am0eur02lp2233.outbound.protection.outlook.com [104.47.11.233])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3y83gc84ww-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 20 May 2024 09:32:52 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by GV1PR03MB10519.eurprd03.prod.outlook.com (2603:10a6:150:162::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.10; Mon, 20 May
 2024 09:32:49 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Mon, 20 May 2024
 09:32:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3f1bf85-168b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Q0LC6vNfEKRauUm2Q08ZWTdsy9SwmSmy29i8A7bG8j7/6dohlpZym1E3rxKu9QUs0ZIrhZqDO2QNqjCWDHQY05XW+QNxQMJFcqk3jLMo5EkaDzGzcjsq6ek5KtHeiSVZLFgFXXUL5cc5Keu/nD4GqOptKztjMgbWZdqzIMHIr1N6p+bC6Wd/UFnJpbTE+EFiWPODFiS169wkAmwVHY+BN2lziXKU240J4KyRU6XGEI33UwGlAIaG+QCepho579NxGUOubEPoTuAsq0rYCKE72ZiFf7HQWP8/bHvVM/9Kj21g919Ie4yZeuM+IW565wNipmxCvXyA0Za1RH4jtCFTlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5Dobb6zopaOwVYR/Hww3nW/UPjZbNglrak70RDzBrCk=;
 b=L5iU1s8qHOOVcj8VnwroaFxvC9K1uxeVYxzgUgJ4TLrGyHntYaCO87rM64Ill9MENQWqdrVDqTHcGRjjhedLh0IgdILTA7nU4gODFYCSU2O2NeYuMKJwYc3qZy5PhSwZkP/pY4+Dl3sg+8K1iBMmwZgKuBtThQ+7hSTBH0HlOtSVjxGYjgu8p3thZhlEnDBKRAxmL2VuFf5ySsS94a3TiwwpgfbQroELw6k0u+WxQ+iOe8fI0QJsj9GKDYKzA0dEt21LkjcqX3EbZZ8LbBierXQ4Ulm5qeNFNfbioszmU7ETbKa2dYjmfg2JA8Ido3TWcLlep1HZjbB0HmEtUPv/Vg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Dobb6zopaOwVYR/Hww3nW/UPjZbNglrak70RDzBrCk=;
 b=fkWHSZjGEHRV5BZiE+M8kNrfZXV3UN71fEmY6gKfqyAZ9cVlTA8xVWESFwYrhvywzrtlDPpXNj288rtyWtJxxyxvFvlpPtyLD704yoinIVEX9/FRXfV/ID87DebN3Hh164whIDZYIJI8I8jOFoN6Cf6j6X0mwnxPx0sMI5zpMoTlU8q0BwDT8LuzPrADq4arz+b2zD3R80Z9Q9mPr8VNcrXrza7GZR+SyMfTWGBBLIQlqwzkSYiMGuNrV+gKBUKUWiyKDiwjVpTRddzUWmR6N78JhqXlCD0z9JY2sy5b1j07h7OX2l9Nh5hZj5vF3ia32jFJ0trRXnMzMQj25Rd+fA==
Message-ID: <68e2b3b5-649f-4e53-8700-350bdb8e5b09@epam.com>
Date: Mon, 20 May 2024 12:32:47 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to
 header
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
 <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA2P291CA0002.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1e::22) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|GV1PR03MB10519:EE_
X-MS-Office365-Filtering-Correlation-Id: 95e6a470-593f-4389-5f75-08dc78afd083
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|366007;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?WEMrRnhMemhvbFdEQlp6cmlJN082WFRFUzUzV3V2UmVRSXhHMWhoeCtGbFFu?=
 =?utf-8?B?UTIxQWJvOHk1MFRBL1NDZ2kyRHZzT1RKVjVGTmZkNUNjSVJINE5FbUFGNXNy?=
 =?utf-8?B?d0dZYVhXbGxiOEtoSUlrTjFES2IrMUcvZXpoVXhyaW5LenhvZS9US0htZTFT?=
 =?utf-8?B?bllzNzl0K05PRHY3SlZWL2xzRUx6VHVzNXpKNDRrNzVSMWVULzBFSkFpcjhj?=
 =?utf-8?B?Q0UvMkI3ZDhtWm84OWhoK0w3eERlY00wTVRJaHV3bVpKYzFzQnViZTBCb2M2?=
 =?utf-8?B?aWc0OU1IZWJ4cGg2UXlCZWcweHIxbmtiZXJOeEc3ZDVrcW13QXVua2JUdFpx?=
 =?utf-8?B?VzFwZlB6b3JRR014VXcza1E5STFpOUQ0YjV5b0hJS2t1Snh3dElDckdJVDhW?=
 =?utf-8?B?TUgzbFRPVE5NWEk0a2UvNVVEZEpPQlRuV3liM2F0SkRvYkNRYTNxcm9rQVpW?=
 =?utf-8?B?V2JHM21KVFpYZTFDb21ybXpSM2VoUElCT0RLaGxEQjZtbHlLYkdrYnlGZ3Ry?=
 =?utf-8?B?ZWhOZzc0QzM0cEs0ZXlHL2wwejA1R25uNFdQcDB0d3BLMFUwaHdpT0Ivb1U4?=
 =?utf-8?B?RVNPM3gzUmUzK0RPOUN1dG9RTSt1d2JMWEowUDhkMUV2TUFjWHQwVi8vdVNS?=
 =?utf-8?B?SThUM2lsekJkWVNDSTZvUHBBOEdPKzZhWGcyM1pkSURCbWNwNGVvK2E5OVJZ?=
 =?utf-8?B?K2Fya3RHZFYxR2NGL1VYeXB4QUlQd0V5cGRlZUg4MVJXRWNXZUNJUDFyQWFv?=
 =?utf-8?B?aW01aW5MNnZBTElPTDB5OFg5VjJ5RWpnSGtLanJnSWZUUmJKc21RbHBEZlV6?=
 =?utf-8?B?RE5DNHk0dnhTdkRlVUZDYWtIcWlVQWswYnUvMml3bzVaRnpFbFdtbUJHcmdi?=
 =?utf-8?B?dG9XQXZCYyt6TTZzUkJ0UDN5VzZWQ3NRbys2WUtEZktFVklXbzhTV1ZBeS8x?=
 =?utf-8?B?ODcwQ3JGSEk5SE4zTExhaytQUHBiVVJEMzBLQ3pVT2oyNzF3a2FFMlgxNmxp?=
 =?utf-8?B?dFVLaElPLytWNTVRRU1ad3pZNm9uRjZscXNpUURWNVhXcldVNjlyUzQrRmY0?=
 =?utf-8?B?Y0Ercjd4dXlyb1RGTnFJWWtnT3VEMDdVVlkyQnRGZ2Nzc1M2bWZBRTh4L1JM?=
 =?utf-8?B?K3NHc21tMlBmSk9UR1U3Ti83ZTBFaHlyR2QrdVlMNGdyTVlyUnI0SFhaWUxn?=
 =?utf-8?B?OEc1eFREdkRRUHVmbzlRZ25SQ2w0dXZNcDNhR3V3NjR2cE9nSzdZb2VYa09D?=
 =?utf-8?B?SklJNm5GVm1FNjFrQVR4ZGU4d0JycHA3aUR1aXBBSXp1VHVRN0RpeG01aHNF?=
 =?utf-8?B?SjQweUkzMk5PVVhWa0RRRUE3VGQ1Z1FRZVVNUjMyRENSTThxSVpybGE4Mmc2?=
 =?utf-8?B?cGNoK09HNWt6UFAzbEd2T3F6SmNIUkpmVXpxVFJhNXZtS1IrVHpPZXJ2bWw3?=
 =?utf-8?B?SWFWZUQvNTlsR25LM05NSWNlbEozRm5SUURHaHhBdnVJSmxLVVM0cHJNc0Qw?=
 =?utf-8?B?WElxV1pTbHFzdUJHelRZVDBzb3lSeE1yWnlRbW1WRDEvWHpMOTQwbnluVGh3?=
 =?utf-8?B?N2RnbDdoOXhFUld6USswRk5qQldmczlkaVNjeVZXSnFvbTRJOTlXY00rTVhP?=
 =?utf-8?B?Q1hyS2drdjUydXRUZW1ZWFU1dTFLL3hoNWNPNStVbFcyVVdtMWJjdHl0V29Y?=
 =?utf-8?B?cUVoRnMzMTYzRndoOG9YU2lkdTZPcHMrSTdGdDlEVUhqTU5pSWRQYWtnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?TE9laDAyRUk4MjVrU1plWGRGTjdSdi9LM2hDVFZNYmV1YWtldkcyYmdDSW5S?=
 =?utf-8?B?WlFwR0RVTUF6c3FJQ0RNNVRFWDNIQTdBUlNjYnhtai9LMWY0QzRya0QwQWFt?=
 =?utf-8?B?WWFoOW94elZnY2RaNWNOWDdwRmN5eGhQUWFzaGVGampObk9FUkFiUDJ4V1ZY?=
 =?utf-8?B?M051Rno4bEthZEpqeXpKdUh2S3pIbVBzbW9QT1hhME9ERlNvMEZ6YVVaVjFt?=
 =?utf-8?B?dlV0eGViUVJ4RzUrWEp1bThhVnRmWFpKTUpNUWpvTFhXODBZSURNdk8wbzlW?=
 =?utf-8?B?UGJtMWllalJJREVXdVBvM0JIemR2Q3B3YkFORmJMeXZRL0tISlgxUlRVTmtk?=
 =?utf-8?B?cThnMjdrNlMwMG51ejg4OFZ4S2QxWC82Q21vdmw2amcyTCtVVTJOcVF3bWwz?=
 =?utf-8?B?bGVvKzNzVXJJbmV3YUt2NndhdTZYQkZZNXA4bU5Xa3A0Y3kva01YdW9iQ2pF?=
 =?utf-8?B?MjBLZmxEUEFrR0VxK3NtaE9XZUd3MnRoVDhqcXZRd1RSR0hDeEttQ3FsSHBB?=
 =?utf-8?B?VVAwbEJoMk9FVkc3Zk0zRm1TTzBoNEI2dEovL1d6cE5qcTRHOEFyQ3A5TXJS?=
 =?utf-8?B?SituQ2JLN3FOTDVzQXBUNVRRbEduWVVQZjNkOEc4RWl3SWY5MjJKT3NzYmZZ?=
 =?utf-8?B?NUtIV0FPZ2NsaE41cnh5ZFZYcldsMGh3b2JaSHYrVXptNitqRWdsOU0xQ1pr?=
 =?utf-8?B?dUdMbjUwN2txVUp2ZlcwNkVKWWxsN3FrbTdEdTA2UTd0SVo0ZWdibyt1NnVo?=
 =?utf-8?B?VHhMS2t1dWZ2NmduRk9mUVdGN1lGMU1ERUxHM3JrWjN5MHNJaUJ1T2N5aDEy?=
 =?utf-8?B?dFBsUTJxL01ML005RC92SU5iNUc0Q3JHYVdyM1JmMTRnSlJYV0Z6V25yQVA0?=
 =?utf-8?B?YXQ3MmpPdU1JWDlJeStRbzR0cEhsTWlWVWNUZWx1R3gzQmgzbVJiU3BOUmpR?=
 =?utf-8?B?b0N6V2R5SlQwUW1HNUtOWHBFaDQxRVdUV25oN3V0NGE3OFVQTmpsd3FpZ213?=
 =?utf-8?B?N2NSU3krVDFTUEM4RTVZNGJFNENudjBvMWFKckdRRUtwYTFPTlpHdVJIVmNz?=
 =?utf-8?B?bk0raDBtQ1ZaZFQ4ekh3TWtKSmZqL2hYeURtOWl1c3p2eUNIUnl3NFMvY2Zq?=
 =?utf-8?B?eThpTElnRVVZa1hFd1FvUjdqRCt5dzRzc0VDODNmelJIenFnK3haVW81NzVB?=
 =?utf-8?B?cGM0bDNJaUM3UW1YTXlzOVJsNThwNDVBaXpwV2wzSmJ4YWgxaHd4ZUk3MFBN?=
 =?utf-8?B?U1pFS1FBRXdSV3ZPQ2hJN2hFb29FMjFpZXNBWTFkekxkZ2pMcDNKMytHT1ls?=
 =?utf-8?B?alpoOHk2d1E2SDdhTFgxL1pSMWZmK3BSRU9wOUI3OE9LMXR0MGxoU1piWFZx?=
 =?utf-8?B?T3JDRi9LSWZha2wrLzRCczZORVRVOWhtUHI2ZDhHVTZwK1R0Q1d3dGphOHFO?=
 =?utf-8?B?eGU5UWhXenQ1ZnpDY0V3dExySDZGU20xazBqR3MrUkJtMXU4OXpDZ1V3d2gr?=
 =?utf-8?B?OTNPajB6VDNsQk5wUWljV1p0UlBVZUVuS1FmL0tNYUU3NEhTQ2tlVWwxYmdx?=
 =?utf-8?B?cFZpU0xBTWtud2FlQWFZMFdzblN3M2JlV1prR1lzZS9lWEp5WHBpc2t6NGkw?=
 =?utf-8?B?MUVqZkFHcjVaTVlCcnR1VklveHNsaWFpMDBaU2NVM1B0TEE0MmxxK2xVSWFr?=
 =?utf-8?B?Nnc0TmorYTgyYk5teFUzcExkbFYrbStVeWtpQjlqQlUzcCtHdVd3Z0theFJw?=
 =?utf-8?B?U3dVZzhqZ0pnaVpVWERUNzFsOHNlYkJMU2EvR0ZkZ1d1VmtDMVZMSUJER1I5?=
 =?utf-8?B?OXY5WXFYckNVRGhSenhBTmVpcGtTN3Yxa3U4SW00c1ZYYXNwZWR1OTNXT2h6?=
 =?utf-8?B?QzgzSjR4ck5IK01SK1VlVUlEd2Y2bko2UDg1V1d5dU1BNlhVTzI3SlhXSU1U?=
 =?utf-8?B?S3BhTVd2MUgvUHZlcUVHdThyQ1dyZXl3SVAzdkJTYnhuUVRuUmdUWHZNL242?=
 =?utf-8?B?Z2JkZmpHcWVMRHNYSGRIcE5EN3ZkZHZObE9JWGVHajMyOHNRM0JpL1dzYWpT?=
 =?utf-8?B?OHgzT0cxQTk5eGtIa2N5T1hwcEdWV3VodnZtajh3WlpObjJ4K2tzcTJTZXQ2?=
 =?utf-8?Q?y09X2T3+xGH1VJbn0lctZlNR2?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 95e6a470-593f-4389-5f75-08dc78afd083
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 09:32:49.3666
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oCz3K6Iz1L2vPHVn0t0MUzAO4w9Ta6w4+Ni1LnIGca2iay0RzrnF9C3+r6sg8FsG6zO1t+VeFTcOLJiHkGrSeg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR03MB10519
X-Proofpoint-ORIG-GUID: xqmlm5qnsGBqaayqgf-VzK9TjICdj3Jy
X-Proofpoint-GUID: xqmlm5qnsGBqaayqgf-VzK9TjICdj3Jy
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-20_05,2024-05-17_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 adultscore=0 impostorscore=0 malwarescore=0 clxscore=1015 phishscore=0
 bulkscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 priorityscore=1501
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405010000 definitions=main-2405200079

16.05.24 12:39, Jan Beulich:
> On 14.05.2024 10:20, Sergiy Kibrik wrote:
>> Moving this function out of mce_intel.c would make it possible to disable
>> build of Intel MCE code later on, because the function gets called from
>> common x86 code.
> 
> Why "would"? "Will" or "is going to" would seem more to the point to me.

yes, sure

> But anyway.
> 
>> --- a/xen/arch/x86/cpu/mcheck/mce.h
>> +++ b/xen/arch/x86/cpu/mcheck/mce.h
>> @@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v, uint32_t msr)
>>       return 0;
>>   }
>>   
>> +static inline bool vmce_has_lmce(const struct vcpu *v)
>> +{
>> +    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
>> +}
> 
> Is there a particular reason this is placed here, rather than ...
> 
>> --- a/xen/arch/x86/include/asm/mce.h
>> +++ b/xen/arch/x86/include/asm/mce.h
>> @@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
>>   extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu *ctxt);
>>   extern int vmce_wrmsr(uint32_t msr, uint64_t val);
>>   extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
>> -extern bool vmce_has_lmce(const struct vcpu *v);
>>   extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);
> 
> ... in the file the declaration was in, thus avoiding ...
> 
>> --- a/xen/arch/x86/msr.c
>> +++ b/xen/arch/x86/msr.c
>> @@ -24,6 +24,8 @@
>>   
>>   #include <public/hvm/params.h>
>>   
>> +#include "cpu/mcheck/mce.h"
> 
> ... the need for such a non-standard, cross-directory #include?
> 


This is because MCG_LMCE_P is defined in arch/x86/cpu/mcheck/x86_mca.h 
-- so either MCG_LMCE_P (+ a bunch of MCG_* declarations) has to be 
moved to common header to be accessible, or local x86_mca.h got to be 
included from common arch/x86/include/asm/mce.h.

As for the MCG_* declarations movement I didn't think there's a good 
enough reason to do it; as for the inclusion of x86_mca.h it didn't look 
nice at all.

Are there any more options?

  -Sergiy


From xen-devel-bounces@lists.xenproject.org Mon May 20 09:34:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 09:34:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725930.1130227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zPM-00024Z-42; Mon, 20 May 2024 09:34:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725930.1130227; Mon, 20 May 2024 09:34:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zPM-00024S-0E; Mon, 20 May 2024 09:34:16 +0000
Received: by outflank-mailman (input) for mailman id 725930;
 Mon, 20 May 2024 09:34:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s8zPL-00024H-1n
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 09:34:15 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1e049fb8-168c-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 11:34:13 +0200 (CEST)
Received: from PH7PR03CA0010.namprd03.prod.outlook.com (2603:10b6:510:339::27)
 by CYYPR12MB8962.namprd12.prod.outlook.com (2603:10b6:930:c4::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 09:34:10 +0000
Received: from CY4PEPF0000EDD1.namprd03.prod.outlook.com
 (2603:10b6:510:339:cafe::86) by PH7PR03CA0010.outlook.office365.com
 (2603:10b6:510:339::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 09:34:09 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD1.mail.protection.outlook.com (10.167.241.205) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 09:34:09 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 04:34:09 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 04:34:08 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 04:34:07 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e049fb8-168c-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nXdyfhaiQSbCTHxXkezrk8yl0V9AUuU0gQgZLDejXTIUOFjY5gApSdz1dQIxcvotlHQht5n++kP/WBPLHMUHgBqrlc+SvHrMBsbg1FtVgZbCyisNWLKktSV/hutgjQJPBhiYU3cAM87UOonSCM3RPJvYWmLJNAmb+mPF48Kn0c6Gbw8Ukbm07ojJFk+4EqMNA/8EJ87RSgNqRxE3XdEYIRZEM8GuzDVe1RjlLkZvCBCbEWsMFplYpVxuoCwCS5AQv5Nr2lLPR7A1LL1Fp1ALH4Ixxu0fb2r9n8AvBdALkUSmvHd23R/6qUhzDLHOGWSZiS9H1opCN68vwYefoXt7hA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FRPgyXaN7jDhKyEUc9RkqWThbOrdnnjunqUkZ37rzbw=;
 b=mdg0KW+S72sQEPn0uG6xfiTKEDpPanutwSA3KewvZR1g9zMfQUD1to+E7O7+tuuU+aEbb5c5za9V3t4FDOokUm1Bx8QM5BS32RE175LT6DHWgwsxfzhfXzFH9v/3bt1tQDhS9cKI7fE5zSMJItrRVZkagfsMtgVG4t4MShvSRyIhPBDvQcureTUkjqi+eXXQ56f0pEbP3w7VTuMd0eqd8B2mlL0HUPLV/Q8CgR/xwcKZo3+TRoZltPoSfX1hBuFikLQb4cshC2JkCQMiGp1ggdSAsPfBZUrBq8EiDgREMgl9pD5OW6ya2/W7FRlyCZVu3kdoFmNBB5kRUAvl1ZKzcg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FRPgyXaN7jDhKyEUc9RkqWThbOrdnnjunqUkZ37rzbw=;
 b=yC/iBOCfyogkxNQTlnLjsfNxsPF653VwvR6gcFdCCVpZTCEZICAG0B/DBmGVNzia3bVvII9g87AFwfXubCaGwWQMhTZXWMrbazlJvgrJAYN5EaeRNNq4VW0GlCJcD+hfxLMHJo6eNwEQmMdcc66/E0tx+Mz2QEz9LFK7inIZqm8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <f2a5d401-86f0-46bb-9f3f-87200d4107d1@amd.com>
Date: Mon, 20 May 2024 11:34:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/7] xen/arm: Parse xen,shared-mem when host phys
 address is not provided
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-5-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-5-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD1:EE_|CYYPR12MB8962:EE_
X-MS-Office365-Filtering-Correlation-Id: d9b6d9bc-7696-4f1f-83e1-08dc78b0007a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?WXlwZVFGMlFxenFpNzNQS2kzS3BLZEFySEo4SjZ0VXlpS3o5czcyVzBTVGdx?=
 =?utf-8?B?MXcwMEtxelR3VEFUeW9KclhzQi9wdjBmQXdDdHNWd3FRd1N5TTNzc3ZpUngx?=
 =?utf-8?B?a3BTRzQwMDdNQUVTNkdlYXNlMEIxaFlKcjcvZytBUEFNYnRreUdnV1hocTQx?=
 =?utf-8?B?Z1BnS1ZPcUZIOW1HblJib3FyQU5ldHFEZFgxUXhNZlhCcUVNRzBiOFZnYXpL?=
 =?utf-8?B?TzdGaUJEZW1odnFLNDRRalVWQWtCd05vSjVXWWdlalovQUNCT2VFWm5GTEwv?=
 =?utf-8?B?bms1aS93T01PazJkZGl4WFVaMXlxQ2JwNUxLcjU3dTFaSUtpOUdjTnVFMW03?=
 =?utf-8?B?c1FwT0paTVFUYVBEa2x1ckxpeVgvMzZrdnJTYkpmdDk1aU1DT2lTbzVadGVQ?=
 =?utf-8?B?UmNmalQwaDhuSXZLNzQrUllNanQ2elM2NkIwYmNWcUdQaitoVWVRZloxWCtX?=
 =?utf-8?B?RXlzMHFpckFXQVBIUEI0b1ZSQy9ad2VKYVBKZThjSTlpcmZFK2Izb2t3VUxY?=
 =?utf-8?B?RzVVYnZuOGZPT0hNSHYwOEMwZXpHOTZib2JJUVNSOXhOWG9zRUszSmE5TlZ5?=
 =?utf-8?B?M1F1N3huZ2MxdCtFTGdOUjJYWkZQYlliUzFKamEvTCtZSXRZSnhjOG90SzFu?=
 =?utf-8?B?SHRldkg3bSs5WVJ1c01LZWI0SkU3QzZ0d2N5UzhOUGtNZ085ZkI4bmp5SGNC?=
 =?utf-8?B?dCtMcG1OQ0JkL21WSDRYOTNKazJ4NDkxMWx6b2VIcHZudm5zTjdHcWxLQXdl?=
 =?utf-8?B?bnVKK0FINXFXS0VhMG5hd0RTb2JiTUkrSzk4cHE4R2NQUkpIU09ycUFZbnRU?=
 =?utf-8?B?aW12SSs0TzVWVUVsM0E4cHB5dzhkcjc0Qy9XQzBiMGJmVDI0ZG8zT1Uvb0I5?=
 =?utf-8?B?RW8weG4vdUZkcllkUGlPeHM4OWtYR0w4NGlvci9lcmo5dkFSQXRMSjliK3Vr?=
 =?utf-8?B?K1pnbkhGZERVVXo0OHQ3VWJlbFlhSHBaaittK3dSdGVpU21QVkNaNXBLSk9V?=
 =?utf-8?B?UEJzR3VKb2c3REtvVElnTHFMTzBqSm1KRGpPL3MvSHdxZGV6aUNQQllPNHlu?=
 =?utf-8?B?WGJKV1k5R2lQT1BxUjQ4Y1Q5cHRhNzVobDJkM3A0ekc5dmpnNzhidkZqUVpU?=
 =?utf-8?B?TWNvbmNlR01yLzFhQmFJUlg0OGNmM05tR0ZORUJSaE9kQUJVcER0QkNSb2Fr?=
 =?utf-8?B?cEFWUFBWNHZNaDNyeFdlc1Rmbk9BN2FESEtsWWYrdkw4SWMzWUxOWGNEcWRR?=
 =?utf-8?B?cXdtRjJ4RU5pVUY2Q3FCRi85TVlxOWVZSmdVb25LN0pFdExzUkJBdGtLNFVV?=
 =?utf-8?B?TC8xMzFSeC9valM4VTF6RWRESi9lZHg4UGwzd0N0WkhsMDlvZnRCOW1qbUt5?=
 =?utf-8?B?c1FUWkh5aXZSM3dnZ09DdGZjdGxUUE9PTUlETUdMS20yL0tKR3o2cXZEdG9k?=
 =?utf-8?B?NnlFdWhwQXlxRzI5VDZFdG1YS044Y1lTdFUzUnpmTWJpMGh0K01uME1uU3Av?=
 =?utf-8?B?NHZyTjh0ZTV6SXd6S3ZzbHA2WEdtMlljQW5mSll1RzRtSEZ6N0RpSU5Xdm9a?=
 =?utf-8?B?bFdUM091a1d5T0NLOFdRaklHSFRMSzhMRGRDUksveWkwd01ic1FFVmFaSlI0?=
 =?utf-8?B?ZFhZUCtsWjVad09GK0Y2RC92czFIMG5sZnZKZUVXaHVkbGNRbklKajdxaUFI?=
 =?utf-8?B?clpnMkFjaHByL3JKNkdMR0R3MmJEMVJlQmlZaDhteUM4T0RUd0hJb2NiVmpC?=
 =?utf-8?B?bWhOZFozMWRvZUNQZWNrL05pY3lKZkV0TCthS0lpVlVCN1JhRjVvRnJuK05H?=
 =?utf-8?B?ZE4wZEhUYlB0U2k1M1pEbkxnMHArUlB4bmNaYmlqOWVNR0Y1WndZTHl1bW5s?=
 =?utf-8?Q?zyiBnRHZnCD0C?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 09:34:09.6280
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d9b6d9bc-7696-4f1f-83e1-08dc78b0007a
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD1.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8962

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> Handle the parsing of the 'xen,shared-mem' property when the host physical
> address is not provided, this commit is introducing the logic to parse it,
> but the functionality is still not implemented and will be part of future
> commits.
> 
> Rework the logic inside process_shm_node to check the shm_id before doing
> the other checks, because it ease the logic itself, add more comment on
> the logic.
> Now when the host physical address is not provided, the value
> INVALID_PADDR is chosen to signal this condition and it is stored as
> start of the bank, due to that change also early_print_info_shmem and
> init_sharedmem_pages are changed, to don't handle banks with start equal
s/don't/not/

> to INVALID_PADDR.
> 
> Another change is done inside meminfo_overlap_check, to skip banks that
> are starting with the start address INVALID_PADDR, that function is used
> to check banks from reserved memory, shared memory and ACPI and since
> the comment above the function states that wrapping around is not handled,
> it's unlikely for these bank to have the start address as INVALID_PADDR.
> Same change is done inside consider_modules, find_unallocated_memory and
> dt_unreserved_regions functions, in order to skip banks that starts with
> INVALID_PADDR from any computation.
> The changes above holds because of this consideration.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Note:
This patch will need rebasing since it does not apply cleanly on staging.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 09:41:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 09:41:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725942.1130237 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zWQ-0003iP-QL; Mon, 20 May 2024 09:41:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725942.1130237; Mon, 20 May 2024 09:41:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zWQ-0003iI-Ma; Mon, 20 May 2024 09:41:34 +0000
Received: by outflank-mailman (input) for mailman id 725942;
 Mon, 20 May 2024 09:41:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H0dS=MX=epam.com=prvs=187012ae9b=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s8zWP-0003hq-Oc
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 09:41:33 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 237a7322-168d-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 11:41:32 +0200 (CEST)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44K8lF9J012123;
 Mon, 20 May 2024 09:41:27 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3y6n1e5d9s-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 20 May 2024 09:41:27 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by AS8PR03MB9217.eurprd03.prod.outlook.com (2603:10a6:20b:5b1::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.16; Mon, 20 May
 2024 09:41:24 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Mon, 20 May 2024
 09:41:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 237a7322-168d-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lxhm+6vaNKYwHefwNWxRrc65D0f1dqGz3mLkHDDK0jUE50eaNkZ0bkk4tzMCQ42/gHJohalCYXOKWvNgjuuzCt0Glv9bAd0x0AoP8UZvoqrHyLosGpizKgNkV4FygpZvraegnfvmEbncolI6QycPcRDjrTreUGbYzwT0EGKFp/OhyhlJEGxkhfUmNs6tQ/rvXh01Re2IKyT0ZZhsBv2MgA40ak2IWo/WFrujF06m/gSmS56SHIdCk2VYFlwow8j2TFRF6xz7NwLvxveAlRSu102+9xltgoMgMQ/HBygefjHtFWjKtyiBIRQA8J8R579kBIWVSOs/cx+aHiKZGQDNPw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qKKzr/TNqZsZXKYVtC1RvRlyBxT4+FNoFnSIEiFpoWk=;
 b=b/Empw2ivvld5dgTqEd0cLsHviE2GlcVX/Tnc9Tdxj26ljsdE5L2bYBNC2thoNwwaybCE+ModwiNZQABDgkbrBq61H27wjgpmmG/Z914/72C02e6Wv/cV09CtqGzX0xuk5KrOYcARH7aHtzLr6RdoTgRKqkNroStegGhDEUcYvA8zDVn4ngtAjRHiBXRQ9S/BgZkUP56TeXuHo3YmMYLtmAuxRv7/AVa2r8wp65xPL0Lgn5ZF5lmD9FrxZJk/7uEBfeb1stkzQRRkW1tTpARSsSzK0GXxWuUMrJSo6vMwPqbw8vZe5HbEu2V6JCd8+pmA149FhkqJnmnlgcHnpZlCA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qKKzr/TNqZsZXKYVtC1RvRlyBxT4+FNoFnSIEiFpoWk=;
 b=mk8l9iAZBTOE4l7tIWlkGvEHora71OW0Iox4yPxU0yjOpF9/VLH7Nw3HcjHB0x5Akg2KBtlAYw3qm78kusHxiN/YaqwvSdXWONLLnq23gRQgjVW17DY5xfOyUGw7KzbA0fA/Uei1AO4rmn4F/aYwAr3ZgIMn97VuYwTWLYZMiyIlUj+Y40gLsZCNfbAI+J6yNqefVbA1IJc4iGUf81ud2FDGdfqiawE6X4BAYeCqVsb39Ia7tt7NZnu+c8MdBj2LktaIPqN7Eebs1suJIZtuR6YF3k+ACLiz8rUgYAcIsHIphXE2i5bOvkXpyW/aRhZ2UR+FBNHha3riePcQIDbi2Q==
Message-ID: <4b82462c-34cc-44f2-9c3d-c441dae63bd7@epam.com>
Date: Mon, 20 May 2024 12:41:22 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 5/6] x86/MCE: add default switch case in
 init_nonfatal_mce_checker()
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <01aad263ad60f37ed74138b5abf2733361bb7d25.1715673586.git.Sergiy_Kibrik@epam.com>
 <7cffa5fc-7e23-4acf-98c3-56929025bbd9@suse.com>
 <ea51dc26-e098-457f-a44c-262742b317d5@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <ea51dc26-e098-457f-a44c-262742b317d5@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA2P291CA0024.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1e::15) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|AS8PR03MB9217:EE_
X-MS-Office365-Filtering-Correlation-Id: 4e1d672e-c61f-4fb6-b090-08dc78b1038e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?TnVxYTdvZ3lySGlFUUZkREhZYXVDY0VZa2xZUHZ0M05yOUc2R1NHMFJrUllq?=
 =?utf-8?B?QlM5TG41eEFJcUpNV3JGV0dUdytRK3BFL0VWK3d4SW9VejRUdzJWYjAwRHVE?=
 =?utf-8?B?RFllUVZTem9IQ3ZML1crM3ZUWGdCcUNzR3U2SnJRMFdVeXIvL055M2RGcEhO?=
 =?utf-8?B?SC9acVRtVU5xa1BIM3FzWnFNOHNnemZiSEViRWVhV3gyVWpEa2VMMlN2SG90?=
 =?utf-8?B?Nm0yRXRiTkdxeVBZSkdGbS82SzlaMDRFdTBVbWxHQjNIWWR6NzUxeWdaR3Fw?=
 =?utf-8?B?YUIvK2crRVg2QjhmN0ZNaHlDYnhJdWFad3VWUlhublgyMFRJOENXdzZuMGE4?=
 =?utf-8?B?NmZpR0pHdFVBdHdOZVNyRmVlL1U2QlU3SzMzOUFDbEtOeG55a3NvV1dUZlJB?=
 =?utf-8?B?MDdOKzZUcFNUSDIxOVZNUDlmR21lR2VlYzZ5dmZKY2cyUGJSRHYvRjVRTkhk?=
 =?utf-8?B?MCsrMi9seGdCcGN3Qzh4aWlkNkwySEdjeWEwYUE2TWI5RmNoZ0ZrYU9MQkRQ?=
 =?utf-8?B?SFprUVN1UXBTZHB2bXJKVXFERVVqZ2pMU0JOR04vQkRTOGh1aHZCNTJqNVFL?=
 =?utf-8?B?d3JWbTZXWVRRVmNXT3FBRHN6dW54MW9kVjRQWVRZcFJzZVJucWVFZWliTzZ2?=
 =?utf-8?B?cG5ZcEsvdzVrYUFXUUpIdGdWRGZhNjd5S2tDNWtOSTgvK3dYV0U5dlVKODAy?=
 =?utf-8?B?ekdRbndCVjdKdW9MMjFWRUJwNEJpWHpES01raStPSkl5UmEzM2NLMTNTcm41?=
 =?utf-8?B?L29DUjM4WEFaMVFjSjFra2VTWEdoVW4vWFo1RE1FT0JGQ1Y1REpKV0xlQUhh?=
 =?utf-8?B?NHlocjVCNGtUNzA3QWtlQndrNU4wRlo1WmRJNjZJbjJmMUcydm83VHNLUllz?=
 =?utf-8?B?aFJrYVBiSVlPY2NzYkluQ09yUmVJOVhsc0NqamF6NDRoZjBlR1FuK2g1QzJG?=
 =?utf-8?B?QnkraWNvWGNRTjc2VVpYbm9PQm4vcDVha1NSOTZ5aEtTQ0gvOGFvbzRKaWJw?=
 =?utf-8?B?MXJIdlJ2WTBseENva1QvQ0JBT3k5dW9Eb1ExcFBqN0NyY05wSFF6YlRvQ1Nu?=
 =?utf-8?B?M3dFNnBTdFFjWGcwNE4rak1lMXJVTkdBVFdqMWNDL1F4MFp4QWZ1M0lTQ1hT?=
 =?utf-8?B?a1lPZjVESjVGNEVWdFl2dytFWVRyUnM0OEZPZjdrUGtPYUR2clM1T242S0lC?=
 =?utf-8?B?UU5GcjRSelVEaFc4T1AwTm1MN2VWdXh0d0dIZURmTkdZWk93N0RmSUU2RzJt?=
 =?utf-8?B?VmJvODdnVVA4TUprYUpqQS8yZUN1cUpTaWtsV3RjWmJJRjZZZVI2THRVc2tU?=
 =?utf-8?B?SExWVWhqZVJFUEF3cmVxL1ZqMGhxMlNWTENoaGtWeDJobStKdXhTQlFnM2NK?=
 =?utf-8?B?MW51Y2w0eVUvcW9USHFtWnlISElpOTdwNXRESmRUdW9NR1ZaNWVkRmJwbnFC?=
 =?utf-8?B?dnJxRE56c3doUlQvUHJhOHFQak5pMWVaWUpUNERHZnFXV0FOYjFVS3FzZExa?=
 =?utf-8?B?bkkxUHI2U3NWZjFmYTZVSDZFUGhFek96eXYzcm4xUlNzejZHbklneW5NZHBu?=
 =?utf-8?B?ajcyYWlIRkNmZUV4QXlSbU4xUVgxVm03S0dWN3ZPUnVCcGdpYjF0VEd5bDdP?=
 =?utf-8?B?ZEFNSEEvY2xPK2xEdnZsOGlMM2VGcVg3RWxGNkJFdHZBT2loZjNUZW9mejgy?=
 =?utf-8?B?NGF2RFdENXhiZ3h0SjNBTDJxSWxFcjhUd1VpQjhkeW8vSnZWWnVKS1BRPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?WksrNXZCMUJxdzJrZjluRldlL2NVZklCc2M3VzVrTkhXYldUWUJYS3Bva29V?=
 =?utf-8?B?dzVQaVgxZklPaEFqckNmb1p3SGN1L0NlcUpDYWI0OUpBYW1vSmlkNlVmQlZz?=
 =?utf-8?B?NG1mUDYrVHE1Nm9PTXcwMUpoZFNxV3lVYXprYjBRZnF4QWJvZEY4aktKTzU2?=
 =?utf-8?B?YXE5eHM4b1dFTHBxZmxlN1VRSlE4b204a09UNnFoLzVrQ0lmdGNkcGZ5a0N4?=
 =?utf-8?B?RVhPVDJ1MU9LM1lPT2N0aENsV2xSRmMwT1RabWhNY3hNMlR6UzRUWjNaQnVY?=
 =?utf-8?B?MmNNZ1lzQkJYc3l6MFJFNVVKem9zLytaRFI0dmdxM0dqNEQ3UlJIQ2ZBRytz?=
 =?utf-8?B?ejVqUEd4SnhCdHlLeGFRbU1oRjZXVGJMSC9DVVptT0o3cEpBTjRjNDdnSkVM?=
 =?utf-8?B?dlpMTDRyVVYzUGQxZmQ5UjhLb3pIRytkNEdtWC8zdVVYeFYyUitidDFZMHZF?=
 =?utf-8?B?aXd5MzlndVBqT3VNREIvWXF0dHA0aHR5OEFqdFZLTzV4cVduNnZHUlhDOUF4?=
 =?utf-8?B?WUQ3djVWRDhXcHJnZjZFbENpQVpHcXRJajNwTW1FZFZVYjR4TG1tQVkvRUxk?=
 =?utf-8?B?WUVmczZwYmRucnlzZ0JJUTRXL1drNTQ2MkdzME5IK3l1QnVEeDY3MWhoZWly?=
 =?utf-8?B?clpHR1c1ODIrQkx3aG5rSGlvY28xZnAxOFF1RVd1dDUyOThEbkVYd013akll?=
 =?utf-8?B?bFBMbm1CNnVJUFNRTlNhM09PckVPa2xmbzBPL3EwSnBzTC91UVlkVkdQaWhZ?=
 =?utf-8?B?dDBRaUdLdjBwS00xOTZ4aURSYlFBTVJVREVMY3ZjQjdneWhIZWZHd2VZa0pV?=
 =?utf-8?B?WUZyMTJreVcvR2RINWIzQ0JSelJsR3R5ZXExY1YyRUsxWVdXQmtYY0JKT3Vv?=
 =?utf-8?B?ZjJEYXNQcWtTVGQ2RFpJemk5VEFIckZmUk14N2paV1pDK2EvUnE3dm83ZlJK?=
 =?utf-8?B?S25tM3YvaHdhdkhobUNWZnJML0owa2ZXZ2dwWHpjUWNGSGhWbXA0RXdEZ0pL?=
 =?utf-8?B?Mms4Rm8zaFBWRnFNU2pOMjI1Um5DMTZTZzd0MG1pVUdQZ2p0M2N6dTM0S3Nv?=
 =?utf-8?B?eHUxYmZnVnJHUy9GdXBtSzdkai90ZFU0ZTFCeno2RXdGVjJhODRGNjlkSUpM?=
 =?utf-8?B?bGZRT0hqRHptUjMrSDJDR2VoL2VVVU0rWllZOGxjWFE0VTYvU1BHRUR4cUln?=
 =?utf-8?B?enFzMlJPNDFxMFlzdytqbFM5Q2Y2WWVBZ0p2SWFUR3oxNHBhbm1QZ09qcWxm?=
 =?utf-8?B?ZmJuVHN5azFRd1RxQmFVZk5RR1lnVWpXTVZPcVYrQmcyanVPYkZsdS9tV2N2?=
 =?utf-8?B?REUvbW5oU3lvUmVwRVpESFBEbHFSYTBlVUtZeld5dC9GNUpnYUlqZlVyRXgv?=
 =?utf-8?B?NHpiR1J4V0ppVGpYVkZtK2RtczJqVWpjejErTDNuVGpVblhjRXBHU2NobnE4?=
 =?utf-8?B?aHN3WkV1ejZDdWszU3hIeXZyY3JJRFJrdkFYME0rZTRuVkZUdnBBOHEyalN6?=
 =?utf-8?B?KzZHa1dPby84MERMWDltWkhhdjNxWkVwWWI5UXZaWVFXYkpwWk9ZTk53VHVC?=
 =?utf-8?B?YVVZR1ErckRzUmhsWk9leGNuSXJEN1ZuZzNaamFSMi9VaGJiN2V5V1JrUUpo?=
 =?utf-8?B?UDgwYXI1Wm50clhoNlAwb2ZNZ2gwK3JXSE1sVm03enByNi9ma0FVMDRLNkhT?=
 =?utf-8?B?cVR3K1BpM1pFeVBPWnYvSU9yU2IvT2NNQ0FwT1EvRUgxL1lCSytObjhVdjFj?=
 =?utf-8?B?cWxiYk44SjFxK3NVWCtPUXhWNy9aWDhjSjRYNlovTDUvOUNlZ1BUNFJNNVRI?=
 =?utf-8?B?MmJUalRIdzVXQlYxTFFzYTZZeXdRazJHbmpibkh0RmNVdDNieVF2bCtYSGdh?=
 =?utf-8?B?eXNNM281YW5nMGFydC9rV1lLT2REYXhkcldOdGdOMFcxRzVzdFdlTHJRRTRT?=
 =?utf-8?B?eVEvWWM3RUFCdTRPTTdvbUc2MWdPdUs3QlVBZDhEY1ZzN2dIdEhsK3hXN0hC?=
 =?utf-8?B?aGlRNFhOT1pWeFFEWldtRzhRRjNYb3hDcGVVLy9PTzFmazNhbWZ4T3FLbHF2?=
 =?utf-8?B?bkFrSDYwNk1udzJldWFEMXk4NENwaVBGcjc4Q3J1RWdtSkFRNWtsRm5MV0N0?=
 =?utf-8?Q?0k8wGi2yPfRgU0FfQZeT3Hci/?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e1d672e-c61f-4fb6-b090-08dc78b1038e
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 09:41:24.4818
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: F/qeVjtISyYyDyPGugP3yr94RRjRQxGovCIC6bdVaGvImAtDy00m1pOC/yrZgc3Mec4w//6I5sXmAZcrHtlqFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB9217
X-Proofpoint-GUID: 6VyASyw_ZpWK4H4LG8ivvoNfIGSPlsz2
X-Proofpoint-ORIG-GUID: 6VyASyw_ZpWK4H4LG8ivvoNfIGSPlsz2
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-20_05,2024-05-17_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 mlxlogscore=880
 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 malwarescore=0
 clxscore=1015 suspectscore=0 phishscore=0 spamscore=0 priorityscore=1501
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405010000
 definitions=main-2405200080

16.05.24 12:46, Jan Beulich:
>>> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
>>> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
>>> @@ -29,9 +29,13 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>>>   		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>>>   		amd_nonfatal_mcheck_init(c);
>>>   		break;
>>> +
>>>   	case X86_VENDOR_INTEL:
>>>   		intel_nonfatal_mcheck_init(c);
>>>   		break;
>>> +
>>> +	default:
>>> +		return -ENODEV;
>>>   	}
>>>   	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>>>   	return 0;
>> ... converting this to an error return still isn't justified. On one hand it's
>> benign because we still don't check return values from initcalls. Yet otoh it
>> isn't really an error, is it?
> I realize earlier in the function there are error returns, too. I question at
> least the former of them as well. And the latter shouldn't be an error at least
> when the vendor isn't going to be handled in the switch().

I'll just return 0 then, and maybe leave a comment, as this code will 
start to look a bit unnatural to me.

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Mon May 20 09:42:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 09:42:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725947.1130247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zXi-0004dc-2L; Mon, 20 May 2024 09:42:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725947.1130247; Mon, 20 May 2024 09:42:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zXh-0004dV-Vm; Mon, 20 May 2024 09:42:53 +0000
Received: by outflank-mailman (input) for mailman id 725947;
 Mon, 20 May 2024 09:42:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s8zXg-0004dN-5J
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 09:42:52 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 527abfc8-168d-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 11:42:51 +0200 (CEST)
Received: from SA1P222CA0188.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:3c4::16)
 by SA0PR12MB4416.namprd12.prod.outlook.com (2603:10b6:806:99::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 09:42:48 +0000
Received: from SA2PEPF00001506.namprd04.prod.outlook.com
 (2603:10b6:806:3c4:cafe::22) by SA1P222CA0188.outlook.office365.com
 (2603:10b6:806:3c4::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 09:42:48 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001506.mail.protection.outlook.com (10.167.242.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 09:42:48 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 04:42:47 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 04:42:46 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 527abfc8-168d-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bYZqGMdB1jVe/5Nn4thBiKsxaADMj1Jo5+dBo2+5zLAyo5MsOC+AkSehSJgDo/d7nw6sNP2neQ/UGjSUxz0zpVntneuHspBgne8pU8VWdTW6/GshkNMCHKuan0UMxA9tDaJ/GfIEzAdapSVOC0VFQ6cC2FLp3cW4GUbJKglQGwHjZ+xR7zwm0WOeQjA42iW98vU+1w/VkmHlGQJMO7mTvb262wslZQ8im4oM0x1xWdnp85q4ELi/feEVFN63wIg6RtuApl3KyKat91wmdCXxwTmW5IIQMK1SanP+RjDKvPmRlqn77vrjWtwzJKqAVU1zWHb54T6/+bp6Q9sn1U5Z6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WPU8hD81Pm9J++nH/iyRK9mWmWSLUCEydeiBUSqW8qg=;
 b=mob5SFvSEO7Q8/QRUGkUgtTS5VNynRM7j+u1AJ/1YwRekgJvbVQwUrJw+2RaFKRAfRamGmWQ1Gi2GRaUNIcmgeh9HBDm1mmUStfV3RgGCpVp9q1QhnhsRzROg0mZNnx9oYh7ygLyTFAs3HN57qTyNFnCy+q+8Aw49aKXaeT+Arpyau/5YnmXVB0ohDZhokBVLH5tcKZz1TVgN4lvDu5GOsZkOHkzugZVLzaZVW36jPl22tYs3MSQCVZRc6N/2q7FMreV0ezerMpLIeBfuosQrRboIZQ0vCpF6b83Jmicet3IV0CyArfksWD4OVARbr8tirvvvUazzWZvbHkOQ+Mckw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WPU8hD81Pm9J++nH/iyRK9mWmWSLUCEydeiBUSqW8qg=;
 b=Shv6rXXzKs4tsYyGBbPWsFyPsU44JUrYUaHJk0HFwsAjdK0R4HRRk1dkg8Fx766nDzyrNlifsSWSwrdyX0XEL6aBDHsRAz4MfhlIKUEXtQ8f0hulXNVap0xwFEOluu+qQM7VBykGAdI5O1k6xhDQxReuBvxty4EvDNvt7YN13dg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <67750577-4014-4815-b6e4-e88bf1f5b098@amd.com>
Date: Mon, 20 May 2024 11:42:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/7] xen/arm: Rework heap page allocation outside
 allocate_bank_memory
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-6-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-6-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001506:EE_|SA0PR12MB4416:EE_
X-MS-Office365-Filtering-Correlation-Id: dd7fce0d-68e5-4581-3d3a-08dc78b135aa
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?VUs1Q3dnL2tPWjRySjNYSWZ4bU9ZMlBBRHNoVW9xRzVpZi9OQ1QvRXZWeWMv?=
 =?utf-8?B?ZGpiQ0dmalY1dm9mVFN3WWRtVkRIVU1nV28zcXhiaGFKRUNWa3cvU1Ryb2NQ?=
 =?utf-8?B?bk9yZHhOb2NXeGtpcGp1WDZiRFl6dTRDOCt6T3VqMjNNTmhySWl1L3lLWEs3?=
 =?utf-8?B?NnMwbHduRVVoNnEyNnR6dStWcHZHcXZPNFhkellQUzhUeHdNWElVQkJDL2tS?=
 =?utf-8?B?K2NJdzNRQ0ltMThLV1AvU0tKa1Mwa2dMa0xEbFFWMktteUo0amJkT1QyZ3Rz?=
 =?utf-8?B?M0tIMVE2S0h1R3hpVHpDVlZOdElUNTBzOU5EYi81YTl4MFFRZEtYRm1mM05H?=
 =?utf-8?B?azQ0SXVjN05FaGYvY1gvMFBIWENNNFQ4QlkzZXV6dmQvUEV5TGtlbS92ZnVV?=
 =?utf-8?B?d0FFKy93ZzJYVmFLMUJxa1dBYVloQzAyYTB2VXlWRTB0MHhKdGd4a1hiZCt1?=
 =?utf-8?B?UmxZZFdSYWxxU05JUTRML2FuRWVaWlowR0ExYmxaSjBTN1h1S3ZTRndMMGlP?=
 =?utf-8?B?NXJZT2EzR1AzTDcrSWVZZEx3RytDRFZPYUlzS2JIT0lONTdiNG16WVR1YjNo?=
 =?utf-8?B?bVFFMEhpc3hZeXIxOXFqd3BubzlOL3AzRFJXUXNlU2FFcHh5bTE5dFdLSXhS?=
 =?utf-8?B?SXdwc1N1QkhYdWZtQVZhV0g4WWxlSWhYV0FOdXRGUlA5TTBLV2l0ZVRoK2ov?=
 =?utf-8?B?Q2NXTUJJc3RWWS9sam9OSEV0aGhDdUJwSU1uRjRNMFA5TjZ1cnB6T21LVk9R?=
 =?utf-8?B?ZEc1T1NscFd4bUdkcWxDdk5HSVRUbVdtaGZQNWZLb0dPdUxFZnNMQzJyaG85?=
 =?utf-8?B?TmR6YWJDZGlkTUtBSHNBbitsU3VENkFvMExyaENmQUJJUHhJSUlFc1JkTmdP?=
 =?utf-8?B?LzgyY2xNRVhqMzBYYjFZdkEzL0hLa0RCVGhMQWlyTDFhbmprQnNaa1lBWXVp?=
 =?utf-8?B?R1Z5UlpNSCtZV3k3UituL3JKRjBBZjQ3WktaZDdrZ1oxU2thSEZuNkxnQ1dk?=
 =?utf-8?B?Vmw2S2dsWmN6VlZicVE5dGNQd0hTb1E1Y0pLUlQxZHR1N2wwbW9lUWdoU0t0?=
 =?utf-8?B?WkluYllzN295Wjg1MS9yeDh3VUx2WnQ4SE1NTTRHRkFDc082bHpVM2w5eWR0?=
 =?utf-8?B?ZDVCOG90N29GTzJOUk91S0g4a0FOREZQMW9kZEhzaVovOWY0UmdoMU5MaWx4?=
 =?utf-8?B?RDk5K0lrd3RIYW55WkJjaVJMeXQxVVhDSmtXUjZRaWFVODRWRHcvNzlJWDdX?=
 =?utf-8?B?bkFXREpWRHM4NFpGR3oxZlk0N25CRngzc2svY2xFWmRCbUxxNHhrQ0JublJJ?=
 =?utf-8?B?c294V1hKWHM0bS9jaDRHV2NpV29EdXJvd3F6L21XbmRxOGd4aWhhNHo0NzZr?=
 =?utf-8?B?WXVKYWhGOTAyWmJNTWhsNStZSjltR25pOWdUUGdyekJYaUphL0Q3MVRkOXN3?=
 =?utf-8?B?b3VKVmY0U1dQalBLUGhnWlhWcDhWbU1JUkR6Tis2OVJUWjdtRmE3dEVPQ0Rq?=
 =?utf-8?B?S3hBY204T0JMT1RsbXdVV3NmdExZVnEyWHVOaDB4WXJUVjM3NTFYaWxYV1Nj?=
 =?utf-8?B?VlVJOTMrbmVHVE9ORjh4M0lnWVE4Vm5YQi9sT2lEUVJnMVpVRjFFZmFNSWdv?=
 =?utf-8?B?dDc5bW9hMXBHVjBrMnQxQUsrR3lOZWhQMUxzNzkybnZ3bzZxdmE1TGF2RWNN?=
 =?utf-8?B?M1cyOElwK1dQcVkzaUc3dzZMOUliOFpLZTg1ZTJ6c1F3aDA5Um81YUVKREF6?=
 =?utf-8?B?S01tSnhJNG42S1NDR0xPR0tNSzNMcUwvMlNTYjVSeXMvWitCVkVlVDVsWmtZ?=
 =?utf-8?B?UXB0UzhWRThzREpUVTBSb0NRN1A5ZVZGb09aWlNkY0NlenNWTzNZSHYxRWMr?=
 =?utf-8?Q?Pz2Y5CuADxXA5?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 09:42:48.3880
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dd7fce0d-68e5-4581-3d3a-08dc78b135aa
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001506.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4416

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> The function allocate_bank_memory allocates pages from the heap and
> maps them to the guest using guest_physmap_add_page.
> 
> As a preparation work to support static shared memory bank when the
> host physical address is not provided, Xen needs to allocate memory
> from the heap, so rework allocate_bank_memory moving out the page
> allocation in a new function called allocate_domheap_memory.
> 
> The function allocate_domheap_memory takes a callback function and
> a pointer to some extra information passed to the callback and this
> function will be called for every region, until a defined size is
> reached.
> 
> In order to keep allocate_bank_memory functionality, the callback
> passed to allocate_domheap_memory is a wrapper for
> guest_physmap_add_page.
> 
> Let allocate_domheap_memory be externally visible, in order to use
> it in the future from the static shared memory module.
> 
> Take the opportunity to change the signature of allocate_bank_memory
> and remove the 'struct domain' parameter, which can be retrieved from
> 'struct kernel_info'.
> 
> No functional changes is intended.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Mon May 20 09:50:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 09:50:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725955.1130258 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zfL-0006Yl-Uw; Mon, 20 May 2024 09:50:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725955.1130258; Mon, 20 May 2024 09:50:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s8zfL-0006Ye-Qm; Mon, 20 May 2024 09:50:47 +0000
Received: by outflank-mailman (input) for mailman id 725955;
 Mon, 20 May 2024 09:50:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s8zfL-0006YY-Ao
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 09:50:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8zfK-00073H-ML; Mon, 20 May 2024 09:50:46 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s8zfK-00017j-Gv; Mon, 20 May 2024 09:50:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=S+jEu/Ft9jn5gtKVP+wjkqZmA/ELJVEKPQS6jnL1OjE=; b=I+V6akTgC97xLa17Xu7d9sCfg/
	/5jO7V6C2fang3TTwyyDCCPZgMU/0WzUUrjWMpL9eHUvK8dxytB11LQ8/4yrM1jWznfCUQRGAtKRC
	ox39PryQm3rKUjLoKC6ZQuQbOD8NF41+1dp3e8zlr9BS8bMB6vuaLV/FHwaMQaEsIOCI=;
Message-ID: <30c42949-bf67-416d-9f6f-e476fb1da436@xen.org>
Date: Mon, 20 May 2024 10:50:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-6-xin.wang2@amd.com>
 <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com>
 <9f086470-a17f-482a-ad98-814424da0ee5@xen.org>
 <c475fc50-3d30-49d4-b12d-31535cfe1306@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <c475fc50-3d30-49d4-b12d-31535cfe1306@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 20/05/2024 02:01, Henry Wang wrote:
> Hi Julien,
> 
> On 5/19/2024 7:08 PM, Julien Grall wrote:
>> Hi,
>>
>> On 17/05/2024 07:03, Henry Wang wrote:
>>>> @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, 
>>>> struct vcpu *v, unsigned int virq,
>>>>       {
>>>>           /* The VIRQ should not be already enabled by the guest */
>>
>> This comment needs to be updated.
> 
> Yes, sorry. I will update this and the one in the new vGIC in v3.
> 
>>>>           if ( !p->desc &&
>>>> -             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
>>>> +             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
>>>> +             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
>>>> +             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
>>>>               p->desc = desc;
>>>>           else
>>>>               ret = -EBUSY;
>>>>       }
>>>>       else
>>>>       {
>>>> -        if ( desc && p->desc != desc )
>>>> +        if ( desc && p->desc != desc &&
>>>> +             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
>>>> +              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )
>>>
>>> This should be
>>>
>>> +        if ( (desc && p->desc != desc) ||
>>> +             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
>>> +             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
>> Looking at gic_set_lr(), we first check p->desc, before setting 
>> IRQ_GUEST_VISIBLE.
>>
>> I can't find a common lock, so what would guarantee that p->desc is 
>> not going to be used or IRQ_GUEST_VISIBLE set afterwards?
> 
> I think the gic_set_lr() is supposed to be called with v->arch.vgic.lock 
> taken, at least the current two callers (gic_raise_guest_irq() and 
> gic_restore_pending_irqs()) are doing it this way. Would this address 
> your concern? Thanks.

I don't think it would address my concern. AFAICT, the lock is not taken 
by vgic_connect_hw_irq().

I also haven't touched the vGIC for quite a while and didn't have much 
time to dig into the code. Hence why I didn't propose a fix.

The vGIC code was mainly written by Stefano, so maybe he will have an 
idea how this could be fixed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon May 20 10:30:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 10:30:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.725991.1130268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s90HL-0003rg-RR; Mon, 20 May 2024 10:30:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 725991.1130268; Mon, 20 May 2024 10:30:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s90HL-0003rF-MQ; Mon, 20 May 2024 10:30:03 +0000
Received: by outflank-mailman (input) for mailman id 725991;
 Mon, 20 May 2024 10:30:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s90HK-0003ei-Ud
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 10:30:02 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e96197d9-1693-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 12:30:00 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-4202cea9a2fso18392875e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 03:30:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fa9dbab53sm428386215e9.13.2024.05.20.03.29.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 03:29:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e96197d9-1693-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716201000; x=1716805800; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=IfozEeetqNhwyMYjRgF4bQazCh9I8baeHjAw3Njqlew=;
        b=QsQcU6k18pwwgyOYLZmprrxylhKP4bsXubnP0kQ59capMhCJxKxm0J/ffnoFSElL9a
         pMn9jZbL6RTwmcTTwnMhDBHBHqfmAvPkKt8xLjpKR5Dfp/qt+Ij3JmiTRo3YjXrilDLs
         crE8HuIXf8E9jklIYK4U5VxiEQ8baLO0/cz38=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716201000; x=1716805800;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IfozEeetqNhwyMYjRgF4bQazCh9I8baeHjAw3Njqlew=;
        b=tMMMRTC0c53TDBPDL8EwWVFyPEVP0yxpO6MTM3TyI0s/cDLc8RN559FyqdRImCAAIY
         IkWBvxKrZVOFokFBuKBa4kUfusCk7Wn8fm5i4cZNzsknY/N0RQbKngeKDDiV87fov/QH
         u1wGxPGav97JGeZZLK4fe0+koRX+HW6j4Ue8HTqo0cVuVVNbyusYApc1CtMDNaMmRmf1
         f2PSQ9NigqUi3UXvGYScNxYoH5tG+A0WT8T7Ofb4uussU6TAiI3zn3yCwsqEAgOQT1N4
         eV+HSvaeO6C9aZPR4qhSy7Hy7dBSz9MT77denltpH5TPRiIF8I9rrUfnC6ZtXVcuIzZi
         Zu0Q==
X-Gm-Message-State: AOJu0YxS4nsS4gKb8cvgWGbTXVKvNK21M5QZmeSo/EZviit6eCPcX0fi
	bQXcKm/WsA1rEU9aw4EWU4B2MT254wJ25Oh75Xjeanr8icUEOTWdi+Ru6nuqknTRxu6RvU4kJbO
	b
X-Google-Smtp-Source: AGHT+IEFZHuRncKNMui18rlGOP3kAFWruMSa8pThjPgELR6Tv23bt+N8WbUTLcfC21VZiIImg2mm/Q==
X-Received: by 2002:a05:600c:1396:b0:418:ee30:3f92 with SMTP id 5b1f17b1804b1-41feac633cdmr222317915e9.25.1716200999664;
        Mon, 20 May 2024 03:29:59 -0700 (PDT)
Date: Mon, 20 May 2024 12:29:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Message-ID: <ZksmJp5JnQoBYZ6U@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
 <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com>
 <Zjjg2ueqgjmn-MS3@macbook>
 <44af358a-9510-4056-826c-3be99dc25830@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <44af358a-9510-4056-826c-3be99dc25830@suse.com>

On Wed, May 15, 2024 at 12:07:50PM +0200, Jan Beulich wrote:
> On 06.05.2024 15:53, Roger Pau Monné wrote:
> > On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
> >> On 06.05.2024 14:42, Roger Pau Monné wrote:
> >>> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
> >>>> Make the variable a tristate, with (as done elsewhere) a negative value
> >>>> meaning "default". Since all use sites need looking at, also rename it
> >>>> to match our usual "opt_*" pattern. While touching it, also move it to
> >>>> .data.ro_after_init.
> >>>
> >>> I guess I need to look at further patches, as given the feedback on
> >>> the past version I think we agreed we want to set ATS unconditionally
> >>> disabled by default, and hence I'm not sure I see the point of the
> >>> tri-state if enabling ATS will require an explicit opt-in on the
> >>> command line (ats=1).
> >>
> >> With the present wording in the VT-d spec (which we've now had vague
> >> indication that it may not be meant that way) there needs to be
> >> tristate behavior:
> >> - With "ats=0" ATS won't be used.
> >> - With "ats=1" ATS will be used for all ATS-capable devices.
> >> - Without either option ATS will be used for devices where firmware
> >>   mandates its use.
> > 
> > I'm afraid I don't agree to this behavior.  Regardless of what the
> > firmware requests ATS must only be enabled on user-request (iow: when
> > the ats=1 command line option is passed).  Otherwise ATS must remain
> > disabled for all devices.  It's not fine for firmware to trigger the
> > enabling of a feature that's not supported on Xen.
> 
> Well. On one hand I can see your point. Otoh with the spec still being the
> way it is, on systems mandating ATS use for at least one device we'd then
> simply need to deem Xen unsupported there altogether. The goal of the
> series, though, is to make things work as mandated by the spec on such
> systems, which to me implies we need to consider use of ATS supported in
> such cases (and only for those specific devices, i.e. still without
> considering use of "ats" on the command line supported).

I'm in general hesitant of ATS because I think it undermines the
security of PCI passthrough.  However this would still be acceptable
for dom0 because it's (usually?) part of the trusted base of a Xen
host.

If we want to make use of ATS for devices assigned to dom0 we should
clarify the warning in xen-command-line.pandoc.

We should also consider that dom0 usually does a lot of p2m
manipulations (by having to map grants and foreign pages).  Those will
result in p2m flushes that will lead to IOMMU flushes, and when using
ATS that will require device TLB flushes.  I wonder how much of an
overhead this will add to normal dom0 operations (plus the added risk
of those device TLB flushes stalling the IOMMU queue).

I would be much more comfortable with making the ats= command line
option a tri-state:

ats={0,1,mandatory}

Where the 'mandatory' option or equivalent enables ATS only for
devices that mandate it.  However I still think the default option
should be disabled for all devices.  If devices that require ATS are
found on the system I would use `warning_add()` to notify the user
of the need to consider adding ats=mandatory to the command line.

> If and when the spec was changed to clarify the flag is a performance hint,
> not a functional requirement, then we could do as you suggest. At which
> point, as mentioned before, opt_ats may be possible to become a plain
> boolean variable.

It's a complex situation, and I'm kind of surprised by the
introduction of this mandatory ATS requirement by Intel in a
non-backwards compatible way (as the specification claims the device
won't be functional without ATS enabled if required).

> >>>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
> >>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
> >>>>  
> >>>>          if ( use_ats(pdev, iommu, ivrs_dev) )
> >>>> -            dte->i = ats_enabled;
> >>>> +            dte->i = true;
> >>>
> >>> Might be easier to just use:
> >>>
> >>> dte->i = use_ats(pdev, iommu, ivrs_dev);
> >>
> >> I'm hesitant here, as in principle we might be overwriting a "true" by
> >> "false" then.
> > 
> > Hm, but that would be fine, what's the point in enabling the IOMMU to
> > reply to ATS requests if ATS is not enabled on the device?
> > 
> > IOW: overwriting a "true" with a "false" seem like the correct
> > behavior if it's based on the output of use_ats().
> 
> I don't think so, unless there were flow guarantees excluding the possibility
> of taking this path twice without intermediately disabling the device again.
> Down from here the enabling of ATS is gated on use_ats(). Hence if, in an
> earlier invocation, we enabled ATS (and set dte->i), we wouldn't turn off ATS
> below (there's only code to turn it on), yet with what you suggest we'd clear
> dte->i.

Please bear with me, I think I'm confused, why would use_ats(), and if
that's the case, don't we want to update dte->i so that it matches the
ATS state?

Otherwise we would fail to disable IOMMU device address translation
support if ATS was disabled?

> Thinking about it: Maybe your comment roots in you meaning to leverage here
> that use_ats() is not supposed to return different values for the same device,
> when invoked multiple times. If so, I'm afraid I'm hesitant to make use of
> such a property when I can easily avoid it.
> 
> >>>> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
> >>>>  
> >>>>  static inline int pci_ats_device(int seg, int bus, int devfn)
> >>>>  {
> >>>> -    if ( !ats_enabled )
> >>>> +    if ( !opt_ats )
> >>>>          return 0;
> >>>
> >>> Can't you remove that check altogether now, since you are adding an
> >>> opt_ats check to use_ats()?
> >>
> >> Two reasons why not: For one this isn't AMD-specific code, and hence
> >> shouldn't be tied to the AMD-specific use_ats(). In principle VT-d
> >> code should be okay to call here, too. And then
> >> amd_iommu_disable_domain_device() doesn't use use_ats(), but does call
> >> here.
> > 
> > Oh, that's confusing, I didn't realize use_ats was AMD specific code.
> > It should have some kind of prefix to avoid this kind of confusion.
> 
> Hmm, the function being static in an AMD-only file, I would have thought that
> makes it clear enough that it's AMD-specific.

Yes, sure, I guess the name looked generic enough to be something that
could be shared across vendor implementations.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 11:17:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 11:17:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726015.1130276 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s910X-0001wW-1n; Mon, 20 May 2024 11:16:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726015.1130276; Mon, 20 May 2024 11:16:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s910W-0001wP-VF; Mon, 20 May 2024 11:16:44 +0000
Received: by outflank-mailman (input) for mailman id 726015;
 Mon, 20 May 2024 11:16:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s910V-0001wJ-Qe
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 11:16:43 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6eab678e-169a-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 13:16:42 +0200 (CEST)
Received: from SJ0PR05CA0046.namprd05.prod.outlook.com (2603:10b6:a03:33f::21)
 by PH8PR12MB7159.namprd12.prod.outlook.com (2603:10b6:510:229::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 11:16:38 +0000
Received: from CO1PEPF000066EB.namprd05.prod.outlook.com
 (2603:10b6:a03:33f:cafe::9a) by SJ0PR05CA0046.outlook.office365.com
 (2603:10b6:a03:33f::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.16 via Frontend
 Transport; Mon, 20 May 2024 11:16:38 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066EB.mail.protection.outlook.com (10.167.249.7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 11:16:38 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 06:16:37 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 06:16:37 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 06:16:35 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6eab678e-169a-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EAv34O+/UY2Xn4eus+0r/5nPZesnP3wc7vk+zWpSDyKBcLD6GSDLjO8DZfYjGJzVWNtpLDAacnq9HpwdQjYQ4kYsLoMcY0aYS0CPkubD6YTsTFgsQyY/HaAT9vQ3AWYM6hqprsM3Wik5xO/O926TZ/jtBYzrOxW3T0emDn9+nwVjZ9BpQycCOo2YWPnpn8FEx1Mm8/22rIDI2hIP/iTBHz13EihR6Zel3MVu8p8KnBP9gOd/+kj4xZCCKmr0DZgj0wC6QBlLn3N+IsR8vc+GZBQepM2Ht/4QtryFNvLxDepP/oi0EzbJFUaxad0Y9DV+kEDOtdkK4IDUUuZuLxP5hw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=TDJ3msYQKWjf/JZ1Aqz1pUVl4Q5XyW+mlcb8u/ab4Io=;
 b=cMg2ceiMHEOhwB97Wm3YecpVmvvQ9CPDLHR7fj48eP1oTRTJC2fpbIw2q+8lHbReuo0ljn3ozzN8ukRaaTxxnwiRFqbXQNfUZ77XRsnQiIYGtXkhVSTjj5IV3D+0lRiaeoxgI3+ryJTurgy9eSFIYNIu0uFXoyFEBdDTyGtvRqpOo3EXp6KXOlauTXVPVWsXD4s16xYq04DYhCkMlqv7Ue9sS6UhPHYNFow7HWW2+7Mgm5CXwEr/pdGjR0nWQ5SgzLrRkKmmNvRyj6v4FBY4HXYuWYhASCVoJYP11k/pOkjkFW0ZXtZxusyQFwEzW4aAYg6ZX0A+ZBYq5hYnccbrdg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TDJ3msYQKWjf/JZ1Aqz1pUVl4Q5XyW+mlcb8u/ab4Io=;
 b=cx1xVznDnZaP8xscjbuRJkpxFRAmA7dKhiQ1gTRl9LDMVDWEfzpdBekwUDj59BCqm4fF0BTUywtKAuCALQy/axkrNUEc0Lt2wpv9VmCF2Ua2sKUW05e/6NIXtpRXfeEyCWb4S3lav1ZDxVBp+YD5y8RnuWrFb+7l/im9+LVtWcc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
Date: Mon, 20 May 2024 13:16:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-7-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240515142626.3480640-7-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EB:EE_|PH8PR12MB7159:EE_
X-MS-Office365-Filtering-Correlation-Id: 61159ad9-1055-40e1-43ce-08dc78be5155
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?SlJyTTlXUjVnM1lPODh1ck9HM2NGdTBVUDh5aCtWVGk0Q1NVd3NCOVdHbG40?=
 =?utf-8?B?VmFDY2p1TGVDYjNDUW9aK0tvZERMRjJFVDlOeUN2bDNmM3R2VjhiV3BGanhq?=
 =?utf-8?B?WTFuUUFTRlBKbk11elFpOVhjWFI1RXdoWUd1NG04eEJ0SENJYytLRWRkZG9E?=
 =?utf-8?B?TmdnY09nTjBsek53eEg5ZjJRRVVzd3RpSStnZmNmSlQrZWxBcDNWUHlORFFq?=
 =?utf-8?B?K2I0aGJsRWtoWEZKOGV0cEJCanUyN1JiL2svdmNyTTlKc3hxVHg3WHVkNUpO?=
 =?utf-8?B?eVZPVmg5REk2VVp3ZkM5bWtDd200SzQxRklmZXpBNHNLVFJ6aENXNURlMEdB?=
 =?utf-8?B?bjRhWlp6Q2oyOXJ6cXBFeDNBTzJ5VnBFbklEZnE2YXMvU1JibXRzdlpMSG56?=
 =?utf-8?B?MkhCQzR5Y3pZK3Z0aENDL0E0aXJMREFleDZyV2ZNcEtFTmxhd013aDM2TzFl?=
 =?utf-8?B?UWt3V2s0N1A2ZzNBKzZobWNBLzY5UW8wVDFSb1BKSlNZZFM3WW5kNlBvNlpj?=
 =?utf-8?B?bWdQQnk2VE5NNUE5L1ZEMlpyeE1RSk52aHNZMGVDcGllUEh4MTc0WWxBaU5B?=
 =?utf-8?B?V2hFaDVpa3Nhdk5oRVZIZWZDM2sreGtFOGhrcFFPOEdjaDZ3NHVJNE1oWGhk?=
 =?utf-8?B?UEs1YSs3YU01YmNkNG84Y29aN0gxWmJsMnNrVzIvK1J6ci9WR0NpUjNFeENH?=
 =?utf-8?B?cjVFZU8wQUJrRjRrcVk3QkNESTJ1a0t3anZNV0V2cHJteFpJTFY2SGtvRTRq?=
 =?utf-8?B?NExXNWwrUjZHTENzdXFkeHFhZVlua3oxWjlwRVNENWgvZmFKNG9EdWgxemVy?=
 =?utf-8?B?dFJlcllTS1lDUzRiSEFhNnNDWk4yQU8rKzR0NS9UU0lYS2tNT2hZMmNQcWg3?=
 =?utf-8?B?YmpmdVdnZHVYRFFaeGNBcXlNZEtqQk54VTVvRGZUQjVQVm55K3dIMnpic3hM?=
 =?utf-8?B?d3RaVHFYRG84Vmd0ZGdiVjRpLy9vRUhzVytlRitNSVFnQjFWdjlSM1plOTkx?=
 =?utf-8?B?UGpPdkxzcWNWc013WXhDeFBUNnZYaGhGSStGOU1MT24zc1Q5bWM0dFZyNkVl?=
 =?utf-8?B?UlI4cWNuZXFML1hLbGN6T3pHOU55RVVQb0RIb3A2NDMzWTh6NWlUWTRRU3Ft?=
 =?utf-8?B?NURaRlIzczJSY1dqdDFXMGt3bklpM0RYQXpQUGFFMzlHR2Y1aTRmT00yNkVY?=
 =?utf-8?B?MWxNbEtXSCtkZWZIS05WK201WVo5cE8yc21nRStZZ2svaStOVlRGSTh5Nk9i?=
 =?utf-8?B?eWZoUFpPMU93MmJPbjFxdlpXY1UvcEVUZlUzQjN2OTJ5NkZEenFCcjZTRVhz?=
 =?utf-8?B?MGF3MDZmM0RNUlZJYmZ3K3Avem94citOQWZsZnRZRTBxSTdIUTZsN29ySjhP?=
 =?utf-8?B?ckZLWXdicWJjZVA5TmVTemJRQjc3QVFlZHVGRTZlUHFmczBWQXR3SlltZjVS?=
 =?utf-8?B?QnFlVUo2em96SG9PVE4xWXkzaEtMSzV3bEhaZUFoR0U0S2E3UHorcUQ1Ymtm?=
 =?utf-8?B?TFlBdkhBNHdhSzhQVVVKUWlSUG10OHlWbmZKZGljWERyRVhNVlJMcHZTaVRZ?=
 =?utf-8?B?RjViYWZ5ZnlCRk10cS96ckJESkRFYTV4eDFhSlRWKy9GaTI2MFZwT283dWox?=
 =?utf-8?B?YStMZkIwOWRZQzZLemtiVzJLY3NINW1ia1Vkclp2NldGN0RHUFJ3SWtHV0hB?=
 =?utf-8?B?TVZvTG5Dd3hCS0VYWFR5NFZ0NmZXK24xWm9sSk5IZlRHWmY4dnl0U0gvdnFF?=
 =?utf-8?B?c1VPbEhPVW5KMWpmWXRJM0NWTCtmZ2JhZlM2RU4wV202WTNsZElJVDZhU3F1?=
 =?utf-8?Q?2UjSn/15PLikfR7ApMi2PdBANtWTz2LqGR3yU=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 11:16:38.1710
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 61159ad9-1055-40e1-43ce-08dc78be5155
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7159

Hi Luca,

On 15/05/2024 16:26, Luca Fancellu wrote:
> 
> 
> This commit implements the logic to have the static shared memory banks
> from the Xen heap instead of having the host physical address passed from
> the user.
> 
> When the host physical address is not supplied, the physical memory is
> taken from the Xen heap using allocate_domheap_memory, the allocation
> needs to occur at the first handled DT node and the allocated banks
> need to be saved somewhere, so introduce the 'shm_heap_banks' static
> global variable of type 'struct meminfo' that will hold the banks
> allocated from the heap, its field .shmem_extra will be used to point
> to the bootinfo shared memory banks .shmem_extra space, so that there
> is not further allocation of memory and every bank in shm_heap_banks
> can be safely identified by the shm_id to reconstruct its traceability
> and if it was allocated or not.
NIT for the future: it's better to split 10 lines long sentence into multiple ones.
Otherwise it reads difficult.

> 
> A search into 'shm_heap_banks' will reveal if the banks were allocated
> or not, in case the host address is not passed, and the callback given
> to allocate_domheap_memory will store the banks in the structure and
> map them to the current domain, to do that, some changes to
> acquire_shared_memory_bank are made to let it differentiate if the bank
> is from the heap and if it is, then assign_pages is called for every
> bank.
> 
> When the bank is already allocated, for every bank allocated with the
> corresponding shm_id, handle_shared_mem_bank is called and the mapping
> are done.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v2 changes:
>  - add static inline get_shmem_heap_banks(), given the changes to the
>    struct membanks interface. Rebase changes due to removal of
>    owner_dom_io arg from handle_shared_mem_bank.
>    Change save_map_heap_pages return type given the changes to the
>    allocate_domheap_memory callback type.
> ---
>  xen/arch/arm/static-shmem.c | 186 ++++++++++++++++++++++++++++++------
>  1 file changed, 155 insertions(+), 31 deletions(-)
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index ddaacbc77740..9c3a83042d8b 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -9,6 +9,22 @@
>  #include <asm/static-memory.h>
>  #include <asm/static-shmem.h>
> 
> +typedef struct {
> +    struct domain *d;
> +    paddr_t gbase;
> +    const char *role_str;
You could swap role_str and gbase to avoid a 4B hole on arm32

> +    struct shmem_membank_extra *bank_extra_info;
> +} alloc_heap_pages_cb_extra;
> +
> +static struct meminfo __initdata shm_heap_banks = {
> +    .common.max_banks = NR_MEM_BANKS
Do we expect that many banks?

> +};
> +
> +static inline struct membanks *get_shmem_heap_banks(void)
> +{
> +    return container_of(&shm_heap_banks.common, struct membanks, common);
> +}
> +
>  static void __init __maybe_unused build_assertions(void)
>  {
>      /*
> @@ -64,7 +80,8 @@ static bool __init is_shm_allocated_to_domio(paddr_t pbase)
>  }
> 
>  static mfn_t __init acquire_shared_memory_bank(struct domain *d,
> -                                               paddr_t pbase, paddr_t psize)
> +                                               paddr_t pbase, paddr_t psize,
> +                                               bool bank_from_heap)
>  {
>      mfn_t smfn;
>      unsigned long nr_pfns;
> @@ -84,19 +101,31 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
>      d->max_pages += nr_pfns;
> 
>      smfn = maddr_to_mfn(pbase);
> -    res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
> +    if ( bank_from_heap )
> +        /*
> +         * When host address is not provided, static shared memory is
> +         * allocated from heap and shall be assigned to owner domain.
> +         */
> +        res = assign_pages(maddr_to_page(pbase), nr_pfns, d, 0);
> +    else
> +        res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
> +
>      if ( res )
>      {
> -        printk(XENLOG_ERR
> -               "%pd: failed to acquire static memory: %d.\n", d, res);
> -        d->max_pages -= nr_pfns;
> -        return INVALID_MFN;
> +        printk(XENLOG_ERR "%pd: failed to %s static memory: %d.\n", d,
> +               bank_from_heap ? "assign" : "acquire", res);
> +        goto fail;
>      }
> 
>      return smfn;
> +
> + fail:
> +    d->max_pages -= nr_pfns;
> +    return INVALID_MFN;
>  }
> 
>  static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
> +                                       bool bank_from_heap,
>                                         const struct membank *shm_bank)
>  {
>      mfn_t smfn;
> @@ -109,10 +138,7 @@ static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
>      psize = shm_bank->size;
>      nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
> 
> -    printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
> -           d, pbase, pbase + psize);
> -
> -    smfn = acquire_shared_memory_bank(d, pbase, psize);
> +    smfn = acquire_shared_memory_bank(d, pbase, psize, bank_from_heap);
>      if ( mfn_eq(smfn, INVALID_MFN) )
>          return -EINVAL;
> 
> @@ -183,6 +209,7 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
> 
>  static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>                                           const char *role_str,
> +                                         bool bank_from_heap,
>                                           const struct membank *shm_bank)
>  {
>      bool owner_dom_io = true;
> @@ -192,6 +219,9 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>      pbase = shm_bank->start;
>      psize = shm_bank->size;
> 
> +    printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
> +           d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
This looks more like a debug print since I don't expect user to want to see a machine address.

> +
>      /*
>       * "role" property is optional and if it is defined explicitly,
>       * then the owner domain is not the default "dom_io" domain.
> @@ -211,7 +241,8 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>           * We found the first borrower of the region, the owner was not
>           * specified, so they should be assigned to dom_io.
>           */
> -        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
> +        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
> +                                   bank_from_heap, shm_bank);
>          if ( ret )
>              return ret;
>      }
> @@ -228,6 +259,39 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>      return 0;
>  }
> 
> +static bool __init save_map_heap_pages(struct domain *d, struct page_info *pg,
> +                                       unsigned int order, void *extra)
> +{
> +    alloc_heap_pages_cb_extra *b_extra = (alloc_heap_pages_cb_extra *)extra;
> +    int idx = shm_heap_banks.common.nr_banks;
> +    int ret = -ENOSPC;
> +
> +    BUG_ON(!b_extra);
> +
> +    if ( idx < shm_heap_banks.common.max_banks )
> +    {
> +        shm_heap_banks.bank[idx].start = page_to_maddr(pg);
> +        shm_heap_banks.bank[idx].size = (1ULL << (PAGE_SHIFT + order));
> +        shm_heap_banks.bank[idx].shmem_extra = b_extra->bank_extra_info;
> +        shm_heap_banks.common.nr_banks++;
> +
> +        ret = handle_shared_mem_bank(b_extra->d, b_extra->gbase,
> +                                     b_extra->role_str, true,
> +                                     &shm_heap_banks.bank[idx]);
> +        if ( !ret )
> +        {
> +            /* Increment guest physical address for next mapping */
> +            b_extra->gbase += shm_heap_banks.bank[idx].size;
> +            return true;
> +        }
> +    }
> +
> +    printk("Failed to allocate static shared memory from Xen heap: (%d)\n",
> +           ret);
> +
> +    return false;
> +}
> +
>  int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>                         const struct dt_device_node *node)
>  {
> @@ -265,37 +329,97 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          pbase = boot_shm_bank->start;
>          psize = boot_shm_bank->size;
> 
> -        if ( INVALID_PADDR == pbase )
> -        {
> -            printk("%pd: host physical address must be chosen by users at the moment", d);
> -            return -EINVAL;
> -        }
> +        /* "role" property is optional */
> +        dt_property_read_string(shm_node, "role", &role_str);
This function returns a value but you seem to ignore it

> 
>          /*
> -         * xen,shared-mem = <pbase, gbase, size>;
> -         * TODO: pbase is optional.
> +         * xen,shared-mem = <[pbase,] gbase, size>;
> +         * pbase is optional.
>           */
>          addr_cells = dt_n_addr_cells(shm_node);
>          size_cells = dt_n_size_cells(shm_node);
>          prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
>          BUG_ON(!prop);
>          cells = (const __be32 *)prop->value;
> -        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
> 
> -        for ( i = 0; i < PFN_DOWN(psize); i++ )
> -            if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
> -            {
> -                printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
> -                       d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
> -                return -EINVAL;
> -            }
> +        if ( pbase != INVALID_PADDR )
> +        {
> +            /* guest phys address is after host phys address */
> +            gbase = dt_read_paddr(cells + addr_cells, addr_cells);
> +
> +            for ( i = 0; i < PFN_DOWN(psize); i++ )
> +                if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
> +                {
> +                    printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
> +                        d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
> +                    return -EINVAL;
> +                }
> +
> +            /* The host physical address is supplied by the user */
> +            ret = handle_shared_mem_bank(d, gbase, role_str, false,
> +                                         boot_shm_bank);
> +            if ( ret )
> +                return ret;
> +        }
> +        else
> +        {
> +            /*
> +             * The host physical address is not supplied by the user, so it
> +             * means that the banks needs to be allocated from the Xen heap,
> +             * look into the already allocated banks from the heap.
> +             */
> +            const struct membank *alloc_bank =
> +                find_shm_bank_by_id(get_shmem_heap_banks(), shm_id);
> 
> -        /* "role" property is optional */
> -        dt_property_read_string(shm_node, "role", &role_str);
> +            /* guest phys address is right at the beginning */
> +            gbase = dt_read_paddr(cells, addr_cells);
> 
> -        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
> -        if ( ret )
> -            return ret;
> +            if ( !alloc_bank )
> +            {
> +                alloc_heap_pages_cb_extra cb_arg = { d, gbase, role_str,
> +                    boot_shm_bank->shmem_extra };
> +
> +                /* shm_id identified bank is not yet allocated */
> +                if ( !allocate_domheap_memory(NULL, psize, save_map_heap_pages,
> +                                              &cb_arg) )
> +                {
> +                    printk(XENLOG_ERR
> +                           "Failed to allocate (%"PRIpaddr"MB) pages as static shared memory from heap\n",
Why limiting to MB?

> +                           psize >> 20);
> +                    return -EINVAL;
> +                }
> +            }
> +            else
> +            {
> +                /* shm_id identified bank is already allocated */
> +                const struct membank *end_bank =
> +                        &shm_heap_banks.bank[shm_heap_banks.common.nr_banks];
> +                paddr_t gbase_bank = gbase;
> +
> +                /*
> +                 * Static shared memory banks that are taken from the Xen heap
> +                 * are allocated sequentially in shm_heap_banks, so starting
> +                 * from the first bank found identified by shm_id, the code can
> +                 * just advance by one bank at the time until it reaches the end
> +                 * of the array or it finds another bank NOT identified by
> +                 * shm_id
> +                 */
> +                for ( ; alloc_bank < end_bank; alloc_bank++ )
> +                {
> +                    if ( strncmp(shm_id, alloc_bank->shmem_extra->shm_id,
> +                                 MAX_SHM_ID_LENGTH) != 0 )
shm_id has been already validated above, hence no need for a safe version of strcmp

> +                        break;
> +
> +                    ret = handle_shared_mem_bank(d, gbase_bank, role_str, true,
> +                                                 alloc_bank);
> +                    if ( ret )
> +                        return ret;
> +
> +                    /* Increment guest physical address for next mapping */
> +                    gbase_bank += alloc_bank->size;
> +                }
> +            }
> +        }
> 
>          /*
>           * Record static shared memory region info for later setting
> --
> 2.34.1
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 11:24:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 11:24:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726023.1130287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s918L-0003m8-U2; Mon, 20 May 2024 11:24:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726023.1130287; Mon, 20 May 2024 11:24:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s918L-0003m1-QH; Mon, 20 May 2024 11:24:49 +0000
Received: by outflank-mailman (input) for mailman id 726023;
 Mon, 20 May 2024 11:24:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s918K-0003lv-1M
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 11:24:48 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8f0c30ef-169b-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 13:24:45 +0200 (CEST)
Received: from MN2PR16CA0032.namprd16.prod.outlook.com (2603:10b6:208:134::45)
 by CY5PR12MB6227.namprd12.prod.outlook.com (2603:10b6:930:21::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.34; Mon, 20 May
 2024 11:24:41 +0000
Received: from BN2PEPF000044AA.namprd04.prod.outlook.com
 (2603:10b6:208:134:cafe::7d) by MN2PR16CA0032.outlook.office365.com
 (2603:10b6:208:134::45) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 11:24:41 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN2PEPF000044AA.mail.protection.outlook.com (10.167.243.105) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 11:24:41 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 06:24:40 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 06:24:38 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f0c30ef-169b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CDC38WT///F81ifqxvumWRqcv1WUVcdtykY1mIC7NOkeKEhsGt+C9XarYicnMovVjBOi9c2hW8FQNm6925dYhBKV+xP4sPTst+QYFIOYSzUyq88h+Br1woR8NO6UEA5OG9v9aO/eFmRJkT9dpz1AwI8R2i9zk0fdlTawXGK2EkbUrz6zyE9iitu1I1UWbFDIW2wfT4ocgMSCXJqVC/DwyRmOzA0jhe7NfCliCnr/83i7Dgb4+q16PRQvaTnGh2k3cMPS9ecSe1HdWkNIZTONYuq1i660pzBxHLkSYn7ALahANTuXewwfhArMjzcAEJdnQHXig3JGFIXXuljgLg0YAg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Q6Dn2mNakz3TW3eerJbC6D7zuX+vojhzUPqjLfrzLPQ=;
 b=C8wAwcxX1hWBMpAYh3RYiXlJEEEb992YQ8lMJVbHHBaxDdhxza8fidOpb8toaaGM5r+18QktRqnVN14kYW7OrQIaWMqXJIXDoqcCBpvZx6Sm7IbP280v+k2M/1em6n1OglZ0EWZJOLqg0m3BiK5CyM2GM4d6nE0svHG0UXw989knVeWttvpK+ICFWv9+M2yr1tjgLfMWmwSX499FLIJZEwTVWOKKBJdTeOTwYpXuhrNXNRFe6tDiq6Rwl00G9Zy4xyiGx8NJ6sT7IgusdK2iAkcL5hcqsCfyYclA+xhw9toK4jUAsnVnGN/iGJRMD1b2+jdjqc1FDYoaVHqCDMhQsQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q6Dn2mNakz3TW3eerJbC6D7zuX+vojhzUPqjLfrzLPQ=;
 b=lXMFrRYccM9udbqi+DHgiPM4SIytz3B0D/jcPLyLKlMlKmSEJhNBRXbfBO0xIjlMiwlwxBpip+z6uBXPNaJJZkYy/eWz9TeJP+6AMaiumTfLfyne5whPrp2g3/6Yh0qQomUT50XtNK11dZuauL7aeXhYkYWlka7ZVzLWouhwbXg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
Date: Mon, 20 May 2024 13:24:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Luca Fancellu <Luca.Fancellu@arm.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240517032156.1490515-2-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044AA:EE_|CY5PR12MB6227:EE_
X-MS-Office365-Filtering-Correlation-Id: 78716d5b-137e-44c8-f665-08dc78bf7118
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RE1KMk05MEVmZGY3RVNQUEMvSE1TTEVWdjhkU0ZpR2RXZnMzeEZnb2JkUkdF?=
 =?utf-8?B?L3RnY3piR0dnN0gxeFpqUXlIS2VmdVFXRWcyY2ZNOExnWFNlR0Vad3N5amVU?=
 =?utf-8?B?ZFNveTNJeDMzb0NwSjQ4K2pFWUhhNk56UEdlYXljRDBBamNnN0twLzZ2MkxX?=
 =?utf-8?B?TlhEZmR1OWVPUDlNN2h3d0pPMjZsd0F3WEpRL1JLV3ZGa3JQMjNPMktTb3Fy?=
 =?utf-8?B?M25oV0pkS21aNk9mcHNrZ2ZVU0h6bzRTNndkUmIyb0RMRUdRZFoySEs3QVVE?=
 =?utf-8?B?MG1FOHFoM0VrdnhZVkhmSzQrOEQ0Tm1CclFuYU1FSjJ6bU5RYzJVd0RXd053?=
 =?utf-8?B?SWxQbzAzVE0xQzJSWXVFL2pTRXZ6OVN5WGg0RXkzTFU1WVdmRExuMU1mYU9U?=
 =?utf-8?B?aGpqMUl0T2d1M2x2NzZBTjhPUW5LV0xyWm5PK3VTbU91U3hLaTlSd2dPVjda?=
 =?utf-8?B?OUlKSUYzRWhJOVpEVHVPWUFaRU5lVHJ4SDNkbWkzQXRCSDdhSFNEc2VFVHhx?=
 =?utf-8?B?cE0xMnhMM3JidEVyWVRhRmJUcDc2KzFkSVVnVnpFd2htejdURUVxdGEwYm5Z?=
 =?utf-8?B?U0NLRzhUbjBMbHZ4ZUdkUWFWazl0MjFvcmNiUzk2amxJbUpZUGxLam9CZXdI?=
 =?utf-8?B?MUE3RlUwUG1YZHRMUUZWalZiTXBFOFhrRUlCZklza0xMT2dnbmRPVnMyb0RG?=
 =?utf-8?B?MXdzQkVnQXgvbmtGeVdyMlQxbXgyTmNVcjMrUVRLYWJMT0h3SWMvZzViT2lL?=
 =?utf-8?B?bTVySEY5WmVOTmFPeEdmc3BqU2lZSHFsUHpWTFpWeGlKL2dzM0d5MXA3V1RK?=
 =?utf-8?B?NDZqRmhaNkxBNjk3VWdxcVNLUUF2UEVhN1pIZVZsQlBOVFczc2dhT2MzNlU0?=
 =?utf-8?B?RUVtdndOeGFtSlplZFRLTERIMm1Tc3RYQ3hVZU0wZ1BJcm1zVVFZLzgvN0xC?=
 =?utf-8?B?REh4UGpnYW5adDJobTVHajc4YmUvQ3oySi9HRlJHb1VBcWF4TmxQK0pLQTFj?=
 =?utf-8?B?bXZsejMrblBsU2cyYTJIV21TMWl6cE53bXVNT3NuckZpbDMxL3IzbTlHSzNo?=
 =?utf-8?B?TzJOZ2o0c2FmSXpSYkI2bllpQmJMUmVoWWZiTW1paWdQaWJtK0t6d2Nld3Bl?=
 =?utf-8?B?QmhmRnpLMDA2UkFSc3RzR0xQbmJLSWFYdkNZYmtYbjA2UWhrR0w5QXFqVVBU?=
 =?utf-8?B?amVpKzV2OVpNQStzd08razZQVVJBdG9XSVUvbkNEd2dRSHczM0lDRGUvWXhN?=
 =?utf-8?B?OGlid3Z0ZlF0alZmNld0S1VUamVyM0l6cWJ0Q21RZjdFMzdQSjE1UWV3dFZl?=
 =?utf-8?B?Q1RkRDAwc2gxZnVaS3AxbDljUmRQTlJiSVJHVXk0M2lyMURjTWo3N2V6RHVL?=
 =?utf-8?B?QURvZjY5Z3BETy9VMnMxczNGTTNOMzRKMjRYNmdBaXU3NjlHSGZwMW5ST1Vj?=
 =?utf-8?B?OGtacHlKeUgyUVB5YlJZN2FUY1pjVFBvdDRDdkU1R1pmeWlvdVF1RS9tejA4?=
 =?utf-8?B?RjhaeVI0N0pGbUdDcHIwYjg0cmp5dFE2c3o0RWtBcXE1c2pBNm5oM0k5bUha?=
 =?utf-8?B?a1lWejNKdnpCYXBST3RGQmFBeHUwNGU3MVlCU1NhV2V2Tk9NelR3ekZ3Z3hT?=
 =?utf-8?B?WDYwVy9Nd095VVZ3ZTFVR25rOVpaeFMvUXR0NVNSbGc2VDRJMTRDNkFVKyts?=
 =?utf-8?B?NjliamZ6azBVcTV4SGpjUFdTWHJZTzY3YVdmSHBUWEZ6RDZqMmIyVC9oVmdV?=
 =?utf-8?B?Z1B1ZnJwTmRnQWJVbnpLV0tvWXgzUktvQ1dMY0FNTUwvMFd5WFRlVjVwWTNs?=
 =?utf-8?B?b3FFcnQwS3hKbk93citRaUVXUHZra0dkN3N0aStxd3hBcnRKekRpWi9GUnBO?=
 =?utf-8?Q?0D6bLjqxr9vIn?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 11:24:41.0847
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 78716d5b-137e-44c8-f665-08dc78bf7118
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044AA.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6227

Hi Henry,

+CC: Luca

On 17/05/2024 05:21, Henry Wang wrote:
> 
> 
> Currently, users are allowed to map static shared memory in a
> non-direct-mapped way for direct-mapped domains. This can lead to
> clashing of guest memory spaces. Also, the current extended region
> finding logic only removes the host physical addresses of the
> static shared memory areas for direct-mapped domains, which may be
> inconsistent with the guest memory map if users map the static
> shared memory in a non-direct-mapped way. This will lead to incorrect
> extended region calculation results.
> 
> To make things easier, add restriction that static shared memory
> should also be direct-mapped for direct-mapped domains. Check the
> host physical address to be matched with guest physical address when
> parsing the device tree. Document this restriction in the doc.
I'm ok with this restriction.

@Luca, do you have any use case preventing us from making this restriction?

This patch clashes with Luca series so depending on which goes first,
Acked-by: Michal Orzel <michal.orzel@amd.com>

> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v3:
> - New patch.
> ---
>  docs/misc/arm/device-tree/booting.txt | 3 +++
>  xen/arch/arm/static-shmem.c           | 6 ++++++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index bbd955e9c2..c994e48391 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -591,6 +591,9 @@ communication.
>      shared memory region in host physical address space, a size, and a guest
>      physical address, as the target address of the mapping.
>      e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
> +    Note that if a domain is direct-mapped, i.e. the Dom0 and the Dom0less
> +    DomUs with `direct-map` device tree property, the static shared memory
> +    should also be direct-mapped (host physical address == guest address).
> 
>      It shall also meet the following criteria:
>      1) If the SHM ID matches with an existing region, the address range of the
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 78881dd1d3..b26fb69874 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -235,6 +235,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>                     d, psize);
>              return -EINVAL;
>          }
> +        if ( is_domain_direct_mapped(d) && (pbase != gbase) )
> +        {
> +            printk("%pd: physical address 0x%"PRIpaddr" and guest address 0x%"PRIpaddr" are not 1:1 direct-mapped.\n",
NIT: 1:1 and direct-mapped means the same so no need to place them next to each other

> +                   d, pbase, gbase);
> +            return -EINVAL;
> +        }
> 
>          for ( i = 0; i < PFN_DOWN(psize); i++ )
>              if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
> --
> 2.34.1
> 
> 

~Michal



From xen-devel-bounces@lists.xenproject.org Mon May 20 11:36:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 11:36:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726039.1130297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s91JN-0005f4-SK; Mon, 20 May 2024 11:36:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726039.1130297; Mon, 20 May 2024 11:36:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s91JN-0005ex-Pd; Mon, 20 May 2024 11:36:13 +0000
Received: by outflank-mailman (input) for mailman id 726039;
 Mon, 20 May 2024 11:36:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s91JM-0005er-Hy
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 11:36:12 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 27900ea0-169d-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 13:36:10 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-34e28e32ea4so1918506f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 04:36:10 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbbde1sm29036195f8f.97.2024.05.20.04.36.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 04:36:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27900ea0-169d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716204969; x=1716809769; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=7G1UNVHntPqOsTFZ4pYMGCFJ1i/Q1Hr1oP7bRCRszRQ=;
        b=teJZy/8URf1mD4tyfdfiLpreIu7MHdnj1MihjdYsGHbcB/lh1pYHRg9agFQD7RXMx2
         HVNCHEA5HFDlJIqCpMOaZ3i9yi+1hD0/PknQ3s6LG3Pz9dcnxWv7yx0FkHtif5BWokdz
         i+SQ6zIh2rlrRJCHxcd9TmPPcjZ/JkAsnugNE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716204969; x=1716809769;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7G1UNVHntPqOsTFZ4pYMGCFJ1i/Q1Hr1oP7bRCRszRQ=;
        b=dtFJ+SdMjmCaR2HJVB5MZY20CQVW1DSBc0cp9mEbGzsjGSCpudI1UZdGIcu2oJIzpd
         2CaSbmbR/lRL+3QsXwY3jk5KOuyLkI+BrM3NkRhfCC8rO8qGVnjeyISOStO0GkoX1jOP
         dSigj98/nS4UBD77G+D4ycLrL+lfSwra6Ue3s1u+1+jAQE8Vljc5/YY7boaBa5gqRYNb
         oeo1wEJfAI4lXjuR+j9LpbyQ58oSUc5SX53zSWP11R5V75ApfDe3e04HiScR/sbk1ksx
         c+l+YzlYTMlvl7QzJicf9gs/A2/6Eb/RLPIlpmH5fK9Lj9ECS7mGVnmQyjafTwSzT1nK
         Cezw==
X-Gm-Message-State: AOJu0YxQbnzaBRSfpRbO3w7jvY7UWHrjIB7BEgs5YTTBHiYcB7VluVee
	EbPOiqlrtE7p5Ko3OnKoUfwLypATTF7ucp/H14pUff3UuggBt4hNJPuQ4TW6p3y0TD2z+Nbymb2
	i
X-Google-Smtp-Source: AGHT+IH13V6X2KGF/5xDJMvod4udLcJjR1HHieaKFavy7bcm2rYnRymHLlxVM4sRJVEW0CLFT6Rcbw==
X-Received: by 2002:adf:ecc9:0:b0:352:2700:3f34 with SMTP id ffacd0b85a97d-3522700401emr7076984f8f.32.1716204969516;
        Mon, 20 May 2024 04:36:09 -0700 (PDT)
Date: Mon, 20 May 2024 13:36:08 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 06/12] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Message-ID: <Zks1qFNsQdLkOCAs@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com>
 <ZjjdZRPluS0YIazc@macbook>
 <e9281523-a807-4889-80f0-a13804188af9@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <e9281523-a807-4889-80f0-a13804188af9@suse.com>

On Wed, May 15, 2024 at 12:42:40PM +0200, Jan Beulich wrote:
> On 06.05.2024 15:38, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 11:16:11AM +0100, Jan Beulich wrote:
> >> When the flag is set, permit Dom0 to control the device (no worse than
> >> what we had before and in line with other "best effort" behavior we use
> >> when it comes to Dom0),
> > 
> > I think we should somehow be able to signal dom0 that this device
> > might not operate as expected, otherwise dom0 might use it and the
> > device could silently malfunction due to ATS not being enabled.
> 
> Whatever signaling we invented, no Dom0 would be required to respect it,
> and for (perhaps quite) some time no Dom0 kernel would even exist to query
> that property.
> 
> > Otherwise we should just hide the device from dom0.
> 
> This would feel wrong to me, almost like a regression from what we had
> before.

Exposing a device to dom0 that won't be functional doesn't seem like a
very wise choice from Xen TBH.

> > I assume setting the IOMMU context entry to passthrough mode would
> > also be fine for such devices that require ATS?
> 
> I'm afraid I'm lacking the connection of the question to what is being
> done here. Can you perhaps provide some more context? To provide some
> context from my side: Using pass-through mode would be excluded when Dom0
> is PVH. Hence why I'm not getting why we would want to even just consider
> doing so.
> 
> Yet, looking at the spec, in pass-through mode translation requests are
> treated as UR. So maybe your question was towards there needing to be
> handling (whichever way) for the case where pass-through mode was
> requested for PV Dom0? The only half-way sensible thing to do in that case
> that I can think of right now would be to ignore that command line option,

Hm, maybe I'm confused, but if the IOMMU device context entry is set
in pass-through mode ATS won't be enabled and hence no translation
requests would be send from the device?

IOW, devices listed in the SATC can only mandate ATS enabled when the
IOMMU is enforcing translation.   IF the IOMMU is not enabled or if
the device is in passthrough mode then the requirement for having ATS
enabled no longer applies.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 12:18:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 12:18:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726060.1130308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s91yJ-0003Ia-4j; Mon, 20 May 2024 12:18:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726060.1130308; Mon, 20 May 2024 12:18:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s91yJ-0003IT-0D; Mon, 20 May 2024 12:18:31 +0000
Received: by outflank-mailman (input) for mailman id 726060;
 Mon, 20 May 2024 12:18:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2329=MX=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s91yH-0003IN-G9
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 12:18:29 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05on20600.outbound.protection.outlook.com
 [2a01:111:f403:2613::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f07cd15-16a3-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 14:18:26 +0200 (CEST)
Received: from DU2PR04CA0154.eurprd04.prod.outlook.com (2603:10a6:10:2b0::9)
 by VI0PR08MB11041.eurprd08.prod.outlook.com (2603:10a6:800:256::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.34; Mon, 20 May
 2024 12:18:18 +0000
Received: from DB5PEPF00014B8E.eurprd02.prod.outlook.com
 (2603:10a6:10:2b0:cafe::12) by DU2PR04CA0154.outlook.office365.com
 (2603:10a6:10:2b0::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 12:18:18 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5PEPF00014B8E.mail.protection.outlook.com (10.167.8.202) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Mon, 20 May 2024 12:18:17 +0000
Received: ("Tessian outbound 082664cc04c1:v315");
 Mon, 20 May 2024 12:18:17 +0000
Received: from a6b9332de319.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 56771730-563E-4566-8D7C-27B2ED429720.1; 
 Mon, 20 May 2024 12:18:11 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a6b9332de319.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 20 May 2024 12:18:11 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DU0PR08MB8043.eurprd08.prod.outlook.com (2603:10a6:10:3e7::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 12:18:08 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.030; Mon, 20 May 2024
 12:18:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f07cd15-16a3-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=eZJj6MsSFELARtQleGXiDxOa1EUEjN2ZC9grRkWN6CFeQAwI7ecWhhoycFb8va1uLeWV8BT5mfAyqChbNVDKYFtIQ3OUM4Ec/pYeGEhJJYbPGqO7aQOKlvCkRNKLLZ4bla8tla+MzsYaSIAsFjs1qPVIG4pI1eUNWQFS+qCvOl7dhXW5GViEktBSaifAoKJfygHusJWA0tXr6ilhbAxkzCwyPFSUEv2f2D5zCHoMsGzidzPCRu9ZNBv6hf6Qzhi7v6sA7ckfWhuEyRpdf7bHG8zZJ0htf6hEDnbypC5SNcrmBy1SJbORvyivMY0+gPr+Kq18MbZq/NZqiR4DsYD5Jg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=IUh//qIrrLC50lj3hr8be+GwwZhhSOoLCMvWd+b23UM=;
 b=Gr7aq9svVz5/W2iso9BBorj41tXPOE89xzp3SpS/Rc4Oh0nnoyn+mZD8q7++fwsTnzgg4UHN89s+RH8349USrRJ4YjcG5r9nDEwi8+NqJTOG0j8TUSXZZzgjlOpjvTyWUydYYuhCiYTweyuPpbYejXzSMMwmKYbj4NtXoOmtCsL77VEuqxDyTn/lqhVhzzyPfjyPkS51fIf+7DNi0eiEQZWGiTAQZwyBLj58AmyBRzy3ERnFWcbYQkHHpBAydAhqg4eU0ZZ/eoxfhBo8Z13AwzEhZl5OKZaYlR31l33h2dvTiZ7M59I1hwBPxzzMboLRKyWYY7DDt+H0R5+ste3KXg==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IUh//qIrrLC50lj3hr8be+GwwZhhSOoLCMvWd+b23UM=;
 b=FRhvvvSbn8hjbEVU68jude7ZLpKjGvwKvfUuTNfAtoCo/vV8kePUQJ18aARhApmOEnrTIahy7t3gtvW3eNlTaeyew5ZKVqk9MJBukrj8OTnzNjxNR3pwKBHwB+NZ6tw0lusq+iDqGdwOamfeomN8gmxTCqpeQTyoJXxdrNKxgYE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: c810c46770452edc
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cnr4he1qs8KlnENWz+pDG9L0+ymC+0yqYspNQeeB5PUef7h3RtpPG82J+CIwMoufaC36UJ5bjX3543yBzaDr3CKEGTHpW/v8bUBzp+TMk57TdIH/O+9MeiK5WKZvxV26NkYhKzlRPj+pmfDNRGgQdmfaZDx3QAMOufzA6UjHQLpnoOsni4FUezVi47ly+drZSlvQmytKiewBjV2HkHNfcAC9ulfd/n8VwjHHIppIsWhZZxw/0y1VJoY+K+dQC4CY/GphvmMfCsxMZFfKP2a7rmi7KlI3xQ9jXM7uVAVgl+F2P8CxEIFvLA6XFqwsgFs2fYEQuwNGgGWC7cjugccVZQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=IUh//qIrrLC50lj3hr8be+GwwZhhSOoLCMvWd+b23UM=;
 b=IV9PHOatfMUVM64zzSE1Fe7h2rRbxnuDvx1GChjBrEfGiXu7zOazV3M5kIf//zbYkeF4a1PcPUEF3cYZ3Zg2yi5BL7e/wfDh4MnUVM/K7scuBtw4W6zdwYryVD9oisRVVqYtEBIF4UemKYllN0HAO8ZAeq1LfQzRDWJNkE+0u9/mwM9TK6UdhhZ/N7plqVeLKF3QmLz2iZdreFreHjBiUKv3sdq37Lbz1LK5HGQ4hoNXswjfyhOszJ3c+A8J7Vzrm+YxRSelQtt5e2zsC2Gj8KewpcPiS+qWXlyIFcbJOHtAqoMzUl/D2AcSDc9ksImezNVw4WjjQoQ2Z8JzSSH3oQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IUh//qIrrLC50lj3hr8be+GwwZhhSOoLCMvWd+b23UM=;
 b=FRhvvvSbn8hjbEVU68jude7ZLpKjGvwKvfUuTNfAtoCo/vV8kePUQJ18aARhApmOEnrTIahy7t3gtvW3eNlTaeyew5ZKVqk9MJBukrj8OTnzNjxNR3pwKBHwB+NZ6tw0lusq+iDqGdwOamfeomN8gmxTCqpeQTyoJXxdrNKxgYE=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Henry Wang <xin.wang2@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
Thread-Topic: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
Thread-Index: AQHaqqhXub4l7BSWRU+Wa81RtopbbLGgCi6A
Date: Mon, 20 May 2024 12:18:08 +0000
Message-ID: <CC1164A5-D7C4-4455-95BC-A688C0FF38BB@arm.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
In-Reply-To: <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DU0PR08MB8043:EE_|DB5PEPF00014B8E:EE_|VI0PR08MB11041:EE_
X-MS-Office365-Filtering-Correlation-Id: 3110ba9f-94ba-4c40-c008-08dc78c6ee68
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?dU5aVnczUTBxZkRLdFUwYytzM3Byb1B1MUg4ZDdDcE1rKzF6ZU9CSm1MN3Vp?=
 =?utf-8?B?Q1dURm5GWUp1M1BWK3YvME1qcUtadDdJUEVVWFN3dTdHcmRMVEJzSjRhVFBh?=
 =?utf-8?B?VmFpdlcvSERBY0o1LzhiVGZSMmdib0lZTUdySW5LRmNjRG1DN3Qva1pMTERa?=
 =?utf-8?B?Z0NBVmhIVU9NR3VXeGZHQXcvUWlMRXpGMmJLS1psOXpaSmNyR1NqU0l3SU9G?=
 =?utf-8?B?cjB5eFZzZnAyRVk0bXdBNTk1VHg5Y3VCNi9ZdHZxN0VSMkpOdU83cUlYUFhN?=
 =?utf-8?B?NVhiajYxTTNSUUVQbmY1d3EyS0dtK1g1TmZmYm5WVjdpSGVHbWhxZm1VOEZs?=
 =?utf-8?B?M01LQTYvTVJjWG9OSGRFUms2V2hmemZQRGd1WFpaNTdCcURKQUdRK2RWT2tX?=
 =?utf-8?B?eU9SbllsRHVMeGVLNTNJTnBlRWQ2TnZKTWN0ZHYrWUpkWXRwT3hxNjZBOWQy?=
 =?utf-8?B?ei8rMmxnL0NQQ2FMYTdodDRuUnN0N1NLUEJlVU83L3Z1ekdrakIxUGxVa0hF?=
 =?utf-8?B?TWpRTGo3aEU1eFZsU29qMlBERVZPZGJ4ejBDVzAvdzNLQ0dRSGs2VkVmWTdU?=
 =?utf-8?B?c210dXNjMjdUT0swSUxxN01JbStqRjlqR25EK0VlbW8zTFQ1L1lPWXdLNnFV?=
 =?utf-8?B?RWY0eW1mTHBCN1pZanhTckJ5UUdGekcraWVkcTlPRHUySXkvMHNVOE8yL0NE?=
 =?utf-8?B?Zm1xSFRHZHh0SG04RFhZbVRNcmg4aFFTcFBmODlaMDJuRnRTTmZieHhycU5m?=
 =?utf-8?B?K2tlRHI0aHpwRmpnWTdNR01lVWVYZEs0cGdVT3RjL3JYV1RrSTNWTyszNU9R?=
 =?utf-8?B?aEpGOFBsb0NBOXlXK1FKNU9kVWN0dmhlSGN5aXJVNERiY3NQUDdDL01ub0FC?=
 =?utf-8?B?VGxEODIwcUVXOEZ2UWlDUlFzOGcrcTN3a1Bsa3pMc0s2UlBYODN4WVEwM1FB?=
 =?utf-8?B?cG15ZXkxYW5ncVJKUTd4NGxvdzNqcFk2ZW53dkY5T2NsYTJxWHZENVA3cXd3?=
 =?utf-8?B?QndtN0hMcUFNRzZCOWphSGQvdWJ0VmltYjYrMk42NkJIenFtQi9VY2Z1N3Ja?=
 =?utf-8?B?THZWUHk0VFh3eUUyTnFNM2gwOWlrSEVzZTdDV1JwVGVsUVlveEtaVklZSHZy?=
 =?utf-8?B?aCtNeUpPRm9PSnl4VUlGamJHTzFLdFpzV1ZYQ0JyZHM1UGh0d1o4Y0RIbUNq?=
 =?utf-8?B?VmwyaVBWV3l0OVFvTjhFTDhaMTZrNlUxWTBYaHZjd3l6MDZUOWdyVEs4cFIv?=
 =?utf-8?B?MGJmUmV3MFdrTWtFaHhsV05zSDNpQTQ5dGVxbStzcWJhTGY5VURKWEhxaVJv?=
 =?utf-8?B?N3Q3aDViWk9SSVpFeWZ6USs1T3Y2d21xNkRRUS9xY1BPY3JYYlpjazJOSDZa?=
 =?utf-8?B?bUh0WGZkUkowSFFQeXlteHUreEJldmVDand4bXczOUlZejBhQjJ3V0RPU21h?=
 =?utf-8?B?b1UxcHJOYWJ0OHRSQzI4eWhuWU5TS2kxbmRkRExqSUFmRUxsUmxtVFJYSWJs?=
 =?utf-8?B?WHVCZUlGeFlRSDhuS1EwQ0pXa3A1d1NNZkN2d2V5dGY1by9nT2RNVE9kczlB?=
 =?utf-8?B?Yy9KVFk3QSszR3R1eCthalI5Vk5LSXpqTDBOS2JYZEpqQjh4d2FXZ0JlWWFH?=
 =?utf-8?B?SzlXa3ZnL2JJZlg2azIvWm85TFgyV2ZVRDFwVnN4L0d1byt3Unh6OEF4VUFY?=
 =?utf-8?B?dW1KWlVEWlBobWtzKy9uM1IwQUVydUtVRE1HU2prWGlTQll0eHErWVlCUUR2?=
 =?utf-8?B?Yy9BM2FsdUswWGRVcVZlZjYrbWNnRzd4clhIbHo5b1gxZkdkczBld0toZjdz?=
 =?utf-8?B?MUR6eS8ra3lVSHFkczZIUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <F5664407148F56429AA4E84FEB2178C7@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8043
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5PEPF00014B8E.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bb3c3ed3-ce0e-4e92-0e2b-08dc78c6e8a9
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RGIwZ1BhbUVRWlhJeTdWRkdINmJvakdsMjRoV3ZXQlRFckJhL2JRcVE0VllN?=
 =?utf-8?B?c0ZFWGY3eUo5d3NJWGdEMzh4M2FOa0pXbHVldTZFVjRiYVJFc3hEeXZXNGYr?=
 =?utf-8?B?dnIzTldDYkV1a0VwYTdQek1Iek1EaU1yUUNSeEIwYk1raEhORlFFUjJ2TE9E?=
 =?utf-8?B?UXZZVDZqNWI2N3lmZ21aWFZBRHlKWEcyTkI1d3NQWVFOelU5S2c0RGNMUlFK?=
 =?utf-8?B?ZTlFUytSYjhkNnRMVXBPaXVtNkhidDZaaDVVR3crOWtPREJ5ZXRuUFFNKzVL?=
 =?utf-8?B?TnhsQ0M3Njg5cW9wQlg2SmxnRFZaRFB4N1NheFI3aldUZVNOSm5vempqMWRp?=
 =?utf-8?B?SnBvLzR2Mm1iSU1pOU1LU3lrUFlTTlBiamJMRUEvWVo1ZFhVWVh3L1d4MHlt?=
 =?utf-8?B?VFJBRGN0TUVQeVNMM25NNk0rS05lQ0o4c0VuOWs5NXdmakhuYWduKzlITlM5?=
 =?utf-8?B?ZGlHeS9XNFBVNHVkZFNERGFUNkxUbFFCQU0yWTVlWkw5c1p4RFZJcnJGcGpG?=
 =?utf-8?B?MUdOc1NYRU5sanlCcUNOM3RZQzVHTnl6Tm14U21qSHhLVkRKeVR0ZFpzbGNv?=
 =?utf-8?B?YlRoSE9UdkpaUzI3WU96OHFjUVdSQXlCN0x5L0Ywc3lTZ3FkdHowWWh5MTVa?=
 =?utf-8?B?M1dLUDlwS2VRQXhBbHVSQWF3dHowU3pWUWxKNVZVMkRDK0V3bmp4ZDFmcFFJ?=
 =?utf-8?B?S2ZTWnVRSnhaRFgrUlVNRWVWWk45K0pDWkMwanhldEFURlNFd2FjV1hUNDlk?=
 =?utf-8?B?SFpLY1k5VUhXVVB1a2swbWZ6RzduYisrbDc4R3NndXNGc0h3dTIzc3FLVXJn?=
 =?utf-8?B?QTlVMDhDbmlqNXpCaHVxZFF2cXFSWGp1UmNhZGJEUWtiV0FKdUVIWnNVSmFO?=
 =?utf-8?B?NFl6REJ3YTVmdys4b2pqeDlOL0hvR1RpQXNCV2RoTHFleXQ0K3J2RHRIMWhl?=
 =?utf-8?B?S0I1RlFBeG1QdDUzdE1NU2UyR1hjSzllcDdReUc4dk9KWHlFY0pMTksydmhM?=
 =?utf-8?B?MlJyQnBEN3hFTGQwOUswL2hFaUZueHZDQ3I5eHFwZ05FeXRseXEva08vWllp?=
 =?utf-8?B?dGl5YWIzVWtCSE1GcytKWDY1WFl5amtVaXV5eGpEbHJvcFF4QmtTdkk1RTRq?=
 =?utf-8?B?UEZTL3E1NWd5aFozWVNFNzZJUmRPcHU4dEZrK09iNjkvWUY4d1U3NGxjMFBh?=
 =?utf-8?B?UndoRlAza1dwcFUxaGFlMzZWUkd0bzd6ZDJvR0Y3ZXZ2OVlTc2FrU1ZIeVZu?=
 =?utf-8?B?ZHowRmppUmJOL3lvQ1VEdFJBVVBnNTdzT2xIaFc5dTVvWlZLNzZoNGo2cTVu?=
 =?utf-8?B?blZkT0hSMmd5V2R2UlIvUE1sNFBrYzlSUjAyYkJRZ3BNQ09vWkt3d2kyVDRC?=
 =?utf-8?B?YlJ1NjJ3MHBUZkt6VnlRemxyYUxpMGNZRENyMy8zd0plYW1lZXhMVExqdEFi?=
 =?utf-8?B?cXk5K250YlpRbHVVd2xUUklHaVcvUEJ4T084a1VZUk5NRDhYbTRyVWJ0YnFr?=
 =?utf-8?B?S0IwSmNUa2pUYjVuaG85U0wwNjg3ek9YV2x6YzVxZDFkL3ZuVHZaWEdKdkdL?=
 =?utf-8?B?MjhzSjFWU2licWJxNXVScjhUUWJ5OXdGQTB3elFxWTVmME5yMUJsRHYyTmNS?=
 =?utf-8?B?Q3k3bmEwRzh3K3M2a2NNSVBiWS94dzJXOWVKeFJyakwxMmd3bGM0dDhzNTVW?=
 =?utf-8?B?S3FpZ3V2MXhHaTRIZFIwdGFnU1B3T1FwS0UyaFV3UTVwTTZ4V0dMb2pCSkVQ?=
 =?utf-8?B?eXUwNUxvRjQrWHlYREtVYTRDTUlYQnF5YXNqVlAvSHR1WkI3dUJLMUhabHV6?=
 =?utf-8?B?aGhxSkNkL3pwdUFHNXRGcnhYQVVaOE15a005S25kb0owd1U2ay96Rjd3WFJa?=
 =?utf-8?Q?aRQCUV80jF4JL?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 12:18:17.8311
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3110ba9f-94ba-4c40-c008-08dc78c6ee68
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5PEPF00014B8E.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI0PR08MB11041

DQoNCj4gT24gMjAgTWF5IDIwMjQsIGF0IDEyOjI0LCBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnpl
bEBhbWQuY29tPiB3cm90ZToNCj4gDQo+IEhpIEhlbnJ5LA0KPiANCj4gK0NDOiBMdWNhDQo+IA0K
PiBPbiAxNy8wNS8yMDI0IDA1OjIxLCBIZW5yeSBXYW5nIHdyb3RlOg0KPj4gDQo+PiANCj4+IEN1
cnJlbnRseSwgdXNlcnMgYXJlIGFsbG93ZWQgdG8gbWFwIHN0YXRpYyBzaGFyZWQgbWVtb3J5IGlu
IGENCj4+IG5vbi1kaXJlY3QtbWFwcGVkIHdheSBmb3IgZGlyZWN0LW1hcHBlZCBkb21haW5zLiBU
aGlzIGNhbiBsZWFkIHRvDQo+PiBjbGFzaGluZyBvZiBndWVzdCBtZW1vcnkgc3BhY2VzLiBBbHNv
LCB0aGUgY3VycmVudCBleHRlbmRlZCByZWdpb24NCj4+IGZpbmRpbmcgbG9naWMgb25seSByZW1v
dmVzIHRoZSBob3N0IHBoeXNpY2FsIGFkZHJlc3NlcyBvZiB0aGUNCj4+IHN0YXRpYyBzaGFyZWQg
bWVtb3J5IGFyZWFzIGZvciBkaXJlY3QtbWFwcGVkIGRvbWFpbnMsIHdoaWNoIG1heSBiZQ0KPj4g
aW5jb25zaXN0ZW50IHdpdGggdGhlIGd1ZXN0IG1lbW9yeSBtYXAgaWYgdXNlcnMgbWFwIHRoZSBz
dGF0aWMNCj4+IHNoYXJlZCBtZW1vcnkgaW4gYSBub24tZGlyZWN0LW1hcHBlZCB3YXkuIFRoaXMg
d2lsbCBsZWFkIHRvIGluY29ycmVjdA0KPj4gZXh0ZW5kZWQgcmVnaW9uIGNhbGN1bGF0aW9uIHJl
c3VsdHMuDQo+PiANCj4+IFRvIG1ha2UgdGhpbmdzIGVhc2llciwgYWRkIHJlc3RyaWN0aW9uIHRo
YXQgc3RhdGljIHNoYXJlZCBtZW1vcnkNCj4+IHNob3VsZCBhbHNvIGJlIGRpcmVjdC1tYXBwZWQg
Zm9yIGRpcmVjdC1tYXBwZWQgZG9tYWlucy4gQ2hlY2sgdGhlDQo+PiBob3N0IHBoeXNpY2FsIGFk
ZHJlc3MgdG8gYmUgbWF0Y2hlZCB3aXRoIGd1ZXN0IHBoeXNpY2FsIGFkZHJlc3Mgd2hlbg0KPj4g
cGFyc2luZyB0aGUgZGV2aWNlIHRyZWUuIERvY3VtZW50IHRoaXMgcmVzdHJpY3Rpb24gaW4gdGhl
IGRvYy4NCj4gSSdtIG9rIHdpdGggdGhpcyByZXN0cmljdGlvbi4NCj4gDQo+IEBMdWNhLCBkbyB5
b3UgaGF2ZSBhbnkgdXNlIGNhc2UgcHJldmVudGluZyB1cyBmcm9tIG1ha2luZyB0aGlzIHJlc3Ry
aWN0aW9uPw0KDQpIaSBNaWNoYWwsIEhlbnJ5LA0KDQpJIHRoaW5rIGl04oCZcyBzZW5zaWJsZSwg
SSBkb27igJl0IHRoaW5rIHdlIGhhdmUgYW55IHVzZSBjYXNlIGZvciBkaXJlY3QtbWFwcGVkIGRv
bWFpbnMgdXNpbmcNCm5vbiBkaXJlY3QgbWFwcGVkIHN0YXRpYyBzaGFyZWQgbWVtb3J5Lg0KDQpD
aGVlcnMsDQpMdWNhDQoNCg==


From xen-devel-bounces@lists.xenproject.org Mon May 20 12:45:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 12:45:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726075.1130316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92Nl-0007iO-3T; Mon, 20 May 2024 12:44:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726075.1130316; Mon, 20 May 2024 12:44:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92Nl-0007iH-0n; Mon, 20 May 2024 12:44:49 +0000
Received: by outflank-mailman (input) for mailman id 726075;
 Mon, 20 May 2024 12:44:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2329=MX=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s92Nj-0007iB-RH
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 12:44:47 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05on20600.outbound.protection.outlook.com
 [2a01:111:f403:2613::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bc21915f-16a6-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 14:44:45 +0200 (CEST)
Received: from AS4P191CA0022.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:5d9::13)
 by DU4PR08MB10983.eurprd08.prod.outlook.com (2603:10a6:10:570::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 12:44:41 +0000
Received: from AMS0EPF000001B3.eurprd05.prod.outlook.com
 (2603:10a6:20b:5d9:cafe::c7) by AS4P191CA0022.outlook.office365.com
 (2603:10a6:20b:5d9::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 12:44:41 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF000001B3.mail.protection.outlook.com (10.167.16.167) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Mon, 20 May 2024 12:44:40 +0000
Received: ("Tessian outbound 85e363419cbe:v315");
 Mon, 20 May 2024 12:44:40 +0000
Received: from 280908b22e00.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 24188BB8-684A-4B7D-9C90-63DA1DD54A31.1; 
 Mon, 20 May 2024 12:44:29 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 280908b22e00.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 20 May 2024 12:44:29 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by GVXPR08MB10574.eurprd08.prod.outlook.com (2603:10a6:150:14d::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 12:44:26 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.030; Mon, 20 May 2024
 12:44:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc21915f-16a6-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=a6y8fbZYiCOUvXhxbXdWr1XVlQsPIDmrzv97k9EiEYFrxJe0J4Fgj/KpwmA0YKEegBFb8O6fDaVpuwjMi5FTxGXJXk8or4Gx4/J9e6rx3PWRvr4GOxhf++hm/SNnn1zpHdsB4/aGS6bIQCA7juMTOdzBMmV1ZiS5ICP/porkRBosTJm48OLjbt0eHk4uC5wxozXZ/2DRiidSYzx5zNZ99EK4hCEundIROs0506wF05EuzUpk6KikvjnxkFG/9nECZB27WsJhHA6sHYYOVkeCTC4ZgIM7m3JVWlFo/f0Bh/JWm1aRDSUMSgnkygcyEwidFvJowXQ8fK2q/XCBp34hLQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FUVo4RFHYXH7qAZMy3ECxtKeGZ0y4iTDJ3FSuBnpHuo=;
 b=CT0oZYBxYlWmIVkBD7ytwGUnesexO5uWwG7xO+uCSAgDFISKM9vYBx7zWKx8nYpH99/Uola3TfMgjyU78PyQRPZTT14CIDuIsQ9BrXaW1YSdLtfzlNx/bs043ErT9hYEgaIyIWkhSxM5eBOKEfiGDdVizKdKmsHgmpD3FWztzCKgtKyiDYreW8aQLVuQzlgwfKtqgZ8bOibwHoUB40G8nNuJxZqTsUsUmOnIdbizq+1o5LnEfAda8uxbADz2sUdXyAmbC1MHIJnn4995XUMRe4Fy0V8oGseu26XnecBhK8HlreJe5JdpF2uVdPE8hpklsaf2F+jeZ9m5LFSCbVCOpA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FUVo4RFHYXH7qAZMy3ECxtKeGZ0y4iTDJ3FSuBnpHuo=;
 b=m7SYazop0R2aprhDzjexhAaNoCZNqbJcGuZBwzpuwoyeGAg3den2vJKBQk1u6MNZ1vRl2ZS2e600vXG6h+pJuIH1t8tiGtwEYmqDfCYS2HR9pFGnD36u4w5ObLbOdIb0cjwe2tEfutLFEZTOlXz8AFF5qBFxrn24TN2Kn4OOMN4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8203847e01a4b48b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SPYUyasNMx5xecrhXXHvhvnIbU4ahWX+rqQ/PE3NnkswH1PKc5O6ezmEhN6K07Bx9gs38HwOgrYnij0oTt+2AliOnzvDhR2LGfvGYlrogstcfTMN3ZgVNwrbo3WmitD1gB4RXysGaj+G4Q1rX8spaJn/cyU5cl42ON3dytnwFNb1fFmI0F/vRftAT5x2njHK8pKX7XTczBkzit9lAsl2qKskaUSRfdz2C1ixGf9azGwb0GZfONKMZQmeuMqIn3eLvs2Qk7FafOfjpYCVba6fO4A6iXrKQ/BCDXBrUssthBtU99jv6T0JIMkO0tiUBYPukK6KjSRLKYgyGcmJjkK7bQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FUVo4RFHYXH7qAZMy3ECxtKeGZ0y4iTDJ3FSuBnpHuo=;
 b=RdiY2XmCBRqpFmCl9gSqQjfLyyvceYmEnHG3N/KnPQSObDEBdSjvh9t7LYnVol0KreA89CkLZYupR8+mnu/Jt5gMHsiLTTzeYDQBYVMtpWaQk403i7FMyW3kmCCmeDd1L8bJA7JI2moWPWZhZkwxiAbW1lS8O9Zju8ftp13GP7awYprh35zyc3q2O2InSpg292OEtNrpq/8ZEfpCrXB6OhCjRF7wJbxn9yBZ7UgWdtlZY+pbERpn9uNTRkAs+3LNLmGw/e3yoQm7HccpIfuAvY9KL2H/c1HHKknsLlJ1DP02eMoXgs4HnvfPDOrAa+xnWrkb/CjUMeq+ezVlB7auTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FUVo4RFHYXH7qAZMy3ECxtKeGZ0y4iTDJ3FSuBnpHuo=;
 b=m7SYazop0R2aprhDzjexhAaNoCZNqbJcGuZBwzpuwoyeGAg3den2vJKBQk1u6MNZ1vRl2ZS2e600vXG6h+pJuIH1t8tiGtwEYmqDfCYS2HR9pFGnD36u4w5ObLbOdIb0cjwe2tEfutLFEZTOlXz8AFF5qBFxrn24TN2Kn4OOMN4=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Topic: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Index: AQHaptP4l1WkDlPwC0KOp+ZDl+nOHbGgALAAgAAYgIA=
Date: Mon, 20 May 2024 12:44:26 +0000
Message-ID: <03C2DB73-2B91-4E74-9CBE-ACA21CDA0783@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-7-luca.fancellu@arm.com>
 <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
In-Reply-To: <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|GVXPR08MB10574:EE_|AMS0EPF000001B3:EE_|DU4PR08MB10983:EE_
X-MS-Office365-Filtering-Correlation-Id: 5ee7593a-0343-40d8-7b43-08dc78ca9dfe
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?ME96NmZvNnpZeTA1OUFLdk13Q0owR0J4SHF3TjRMUnZhM3hQMkp4eDJlWGx2?=
 =?utf-8?B?OFRhUzBHUGdsRFdaMWFNUE96b1JhSG95UTk4ZHNDT213WUdYWUpoWHJYR2oy?=
 =?utf-8?B?MWE2ZjNVUDNJR08yUjQxT0lnZkptd01Vc1B3eE1XNHpqSFM3aG5pekRheDEw?=
 =?utf-8?B?T1VHbCtSRE4zMlJQSGhYTmttbk9EUndlSFAyY2U1dnV0RGRndVNVT1Y4S0M0?=
 =?utf-8?B?UDZEV3kwemUrNWVTWlRjalczUStwcEZVTm1VbW12U01UbXFOSWpHd2NhblVv?=
 =?utf-8?B?am9ZQVV1dDdpMzNYaGZUU05rT0tmL0N2UHZST2hUanRBVHRIMndDWnJIaUFt?=
 =?utf-8?B?SkNUMkZBWGRmZThZUUlFbmFLUy9UMFVjZHlCaGw5SzVNNzQyV3REVmQyb0Zp?=
 =?utf-8?B?SG0ycFJlaWZMTFRhb3VjcTN1TitsTHNqM1VpNVBIb201SXh1ZlVGSnlDbGw2?=
 =?utf-8?B?aldXcm9zUUtNdTF0THRpVGxQTTJtUkFpVnlINy9CM21waWJJVENRalpCRVlT?=
 =?utf-8?B?WFVEeGdDdnlxL0xJMTBFZUtCaTIvNUVtV01ZQzZUZFpaS2Y2Zzl0S2ZXeXIr?=
 =?utf-8?B?S2tPVFE2UER1Z252enpDZ0RBVTFTVi96YzZtOEM3YzJJaldUV1RYSDYyWmRY?=
 =?utf-8?B?NkNpaHpJYTR3bm9sZnIyK29tU3F1WXpuRmRTQkxIRzErTm9ITDY3aTZIRyt1?=
 =?utf-8?B?b01WVEM1S240MUkwVTFqS0dlYk0yNWRXZWVOMUtzd1hDRkMyd1g0TlY3Tjhh?=
 =?utf-8?B?Y3EwOFdYTE5MeEFXMFZTM0t5TTIxTmRqaTFHVXdYSCtVVWRhNEVXQ2ZYanc5?=
 =?utf-8?B?NGYrWDBNTWJ6akdqbGVTcGJqSHFWaGsyNDZUN2JGTytwRmw4c0ljMVJMYVlo?=
 =?utf-8?B?UkRTKzRxVUNqNjJIRGx4dTUyNlZvV3A3QnJnSW5aMWZUNjZDazRXTHUzNHNy?=
 =?utf-8?B?eVNZSkFEbllid09LUlB6UFYwbHpidUZ3YzJ6T3Z2WG45L1Z2WkpUR0E3SUpQ?=
 =?utf-8?B?M0tDV2lwSDEyL2lUV3hteU51eUZHbTBiVjZVWDdad21iakFRNDAvMlJlQ3J3?=
 =?utf-8?B?ZUhxOHJOdS9pNFhyQWd4WnhuYTI0Nk9PcnZZNmRrRkVsQzNZSFRpdTJsM3Q2?=
 =?utf-8?B?RFlFTzNWUUFZa2YxZGlYamFpU2x0SnVTenFaOHZTcmhUNUUzMHduenJVci9R?=
 =?utf-8?B?eFZWUGZwR01USnQwZFpLdUhsWE1MMFVrenRhai9tUVVJWHAwUnhuZXJWbTZE?=
 =?utf-8?B?ckF0N2lHd0ZvRUVmNGdwK045Y0k0NUlDdWdUOTE1dUlVZk95REQrVkpYSEhM?=
 =?utf-8?B?R3R6OWNZYzFMVC9VdWoyRTZpWllkK1NYWXpxdG5HUGV2K0dLYWxWV1piczBj?=
 =?utf-8?B?ZzloRDZsOTQzeEtHTU9waHE5bjM5MjFUWmZQNm9TQVBlZEZtdHVhTlp6YTJR?=
 =?utf-8?B?MndYVDA4ODZXbFBnTVczc3NpWkF0ZHBNaEUxWkMzRmw0OGNlVGhrR0dDVHAz?=
 =?utf-8?B?VTZMeklSbk9LOUhjRWg0Y1pEWWZ4aGNOVGFoS3FyencxR3loZEl6M1hQSzc5?=
 =?utf-8?B?YVR4ajlodmh2cjhReUE2NFhIc29aNTEyUFNpRDBaek5QRm9JUzlJQi9FWjdo?=
 =?utf-8?B?WW1IekJOK0tBRldhT3FiYXdIcnM0MjV2S3hQR2pVbGU4dlhEcTlZQ3RnZ09Z?=
 =?utf-8?B?aTEydGYwUG9Md2JjRHFGd256YUx2MHJ0eCsycC9CelBtU1hoMDlkMnRVNThm?=
 =?utf-8?B?RnE2SG52eDRYbGNjWU5BWmNhNTdlamliNXo4RTJ2bW9KaTUzc3hiWnhBSTdr?=
 =?utf-8?B?QWJXRmtrQjF4WEU2VGJIUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <4600DF257EBD3A4282B653F7AE6B0A4F@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB10574
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF000001B3.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d01edcfc-30db-48a5-a978-08dc78ca954d
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|82310400017|35042699013|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?QnJob0FPVnQ0QjJ3Rll0NW9vcExha3FQdVhjQVVVMXdwYjBMcmhBSUEyd0p2?=
 =?utf-8?B?UTYwOThVOVpUQVJLKzByWU93WHQ5d0lnV0tEMHIxekEyaFhMeFAvbE5NZm1p?=
 =?utf-8?B?N0dVRFZ0K3JPd3hDQVFaUk1rUnA0bFI4L1FjSTNkQ2trUmR5bGNoTDJLZjNa?=
 =?utf-8?B?TkMrMnFjd0h5Z01rN2k4aWRNMCtBZGp5aFpnanZkYlJYcG9GUzN3OUE5OFVw?=
 =?utf-8?B?aXZCcFl5OG5UaFlvS0xrVVRPMXhGUk4yWFVrNFVmeHBMV09rek1NQmYySVpM?=
 =?utf-8?B?dk1wYnZyN1l2UEcwSFd2bFZnVElEZmlVMk1QbHRWVXNsQ3AxeTFNRHl4VC90?=
 =?utf-8?B?ODFGeUZCbkRpRnBLaFFuSjNSQnBJMW9ydXlISGUxZXByR1lsSHVuUVdBWitt?=
 =?utf-8?B?RDIwRVN5UlJwNVgzbmp4TXBtOFo4UnI5cnRPTWIyQVZYNWUvVE9BTCtIdy81?=
 =?utf-8?B?eDVSRkdLMTMvc1F1SlJTbC85azRPQWxjaHpTemo1OEticEFVZFJweFpTQ0ZU?=
 =?utf-8?B?UzdDanhZNmxCSHM5OXpyZHNCSnpCTU9yZzF3b08vY2FCd2RjLzNqaXBvUGRY?=
 =?utf-8?B?RFRGOEZxaGkrVENmd1o0NGpjK200ZTZBZlJjSXpHSys2ZldrVGpqM2x3NjhH?=
 =?utf-8?B?MlBKYWl1dWV5eGZ0Z0lLVmF0c0ZLdUduOXlzWU1YNkNrV21pS0FCYndhcExW?=
 =?utf-8?B?SUNVVzQ2R01VMDBDS3VLdGdBRjZUd3AwN24xVlh2VGdLT3NSWHV2U2NUdEV1?=
 =?utf-8?B?aXFDeTZYcUx0V21MYnhKU0RrbHhBN1I2SURuaUltUGhTbDIxdnpOWWJrRjRn?=
 =?utf-8?B?cnYyZEJ2MjBwNlJrOVprWlcwUkQ5VmloQ29CclJEN1JjTUZCRVc2L1F2b2Js?=
 =?utf-8?B?cytJK24zcTgybG5OZU9oMGxXTzZnTTMzakYrWU9kaFhxNEoyOEUxbEpIR1Qz?=
 =?utf-8?B?TlgweUlRc0ZCR3dNQWlWQ2FrN2pRL0l0bGRpL2FZSjFkMHBKYjViWVdRZi81?=
 =?utf-8?B?S2RyUjBSdmVha2dXQm9NbCtZM01EVUt3YWhWNFFMdEF0SmdxY3p6ZzFtNTNO?=
 =?utf-8?B?RUxhQXAvOVRKSVg2dlZnWTdWTGRvYXNWZXZaV3liU3d0QVV4R2lzUm1oQnZU?=
 =?utf-8?B?VWptcTUwRVN0ZjZTZStWR0tkdE5LWjd4U2FPNnA4WndMMWdNek9YdXp2MVRR?=
 =?utf-8?B?aHNoVHAyeFk3Q1QvWWMrVFd1YmxNVHhpY3JkOEJMbHl4QVJqRVk4NGIrWDJ6?=
 =?utf-8?B?c0RVbGYvck1xRC9NYU1PT3NucGRLdkY2alY3S1I5UTByNXh0cG52eDRzNzl5?=
 =?utf-8?B?cFNyZkNkTkxMRVY0OUdiWnhaUlFyTlFWMXpTZElsQlAvTm9WK2RsZVZad21p?=
 =?utf-8?B?Z251VzN6dElZbTdHckNISkJETEV6Mkp1ejQ3SzFTcnpwcXRzT3VOUnpkWkdU?=
 =?utf-8?B?RHhGaGFScWRJNlVhVm9UdHAyT2lsNytWTG42TlA0bGFQMU5lVEFidGx6MmJO?=
 =?utf-8?B?Yndnc3ZyeEhBU2tJUVVuYWRNK1Z6VjFBeGFGTW1HMS9pSHhoYVhUZHFuSWNX?=
 =?utf-8?B?SzByYmpLREt3WUJyL2ZKcVJTRDI0U3o0bGw4Mm5Gb0hUajlzbHN3NDBMbDl2?=
 =?utf-8?B?VFI2RkZza0dhdmVTK05WMTRxMTNsNjBKcG1zbkl6QVlmSTB0NGV2SGxBVGZZ?=
 =?utf-8?B?WUFMUDJTQ1JSZmRBWU1YUjdKeEdxeXlHdVVtSGxhMVMvWE1IekcyZnBXMk1Q?=
 =?utf-8?B?MDVPR1YzRU9aSlhOVjkzODFUSzFvNXk5RVZKMi9aMVB1YngzekpYSFA4cWQx?=
 =?utf-8?Q?Sv7sa+FVtz87DLbEVulcSoAugxrEiSbq4weZc=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(1800799015)(82310400017)(35042699013)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 12:44:40.8414
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ee7593a-0343-40d8-7b43-08dc78ca9dfe
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF000001B3.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR08MB10983

SGkgTWljaGFsLA0KDQo+IE9uIDIwIE1heSAyMDI0LCBhdCAxMjoxNiwgTWljaGFsIE9yemVsIDxt
aWNoYWwub3J6ZWxAYW1kLmNvbT4gd3JvdGU6DQo+IA0KPiBIaSBMdWNhLA0KPiANCj4gT24gMTUv
MDUvMjAyNCAxNjoyNiwgTHVjYSBGYW5jZWxsdSB3cm90ZToNCj4+IA0KPj4gDQo+PiBUaGlzIGNv
bW1pdCBpbXBsZW1lbnRzIHRoZSBsb2dpYyB0byBoYXZlIHRoZSBzdGF0aWMgc2hhcmVkIG1lbW9y
eSBiYW5rcw0KPj4gZnJvbSB0aGUgWGVuIGhlYXAgaW5zdGVhZCBvZiBoYXZpbmcgdGhlIGhvc3Qg
cGh5c2ljYWwgYWRkcmVzcyBwYXNzZWQgZnJvbQ0KPj4gdGhlIHVzZXIuDQo+PiANCj4+IFdoZW4g
dGhlIGhvc3QgcGh5c2ljYWwgYWRkcmVzcyBpcyBub3Qgc3VwcGxpZWQsIHRoZSBwaHlzaWNhbCBt
ZW1vcnkgaXMNCj4+IHRha2VuIGZyb20gdGhlIFhlbiBoZWFwIHVzaW5nIGFsbG9jYXRlX2RvbWhl
YXBfbWVtb3J5LCB0aGUgYWxsb2NhdGlvbg0KPj4gbmVlZHMgdG8gb2NjdXIgYXQgdGhlIGZpcnN0
IGhhbmRsZWQgRFQgbm9kZSBhbmQgdGhlIGFsbG9jYXRlZCBiYW5rcw0KPj4gbmVlZCB0byBiZSBz
YXZlZCBzb21ld2hlcmUsIHNvIGludHJvZHVjZSB0aGUgJ3NobV9oZWFwX2JhbmtzJyBzdGF0aWMN
Cj4+IGdsb2JhbCB2YXJpYWJsZSBvZiB0eXBlICdzdHJ1Y3QgbWVtaW5mbycgdGhhdCB3aWxsIGhv
bGQgdGhlIGJhbmtzDQo+PiBhbGxvY2F0ZWQgZnJvbSB0aGUgaGVhcCwgaXRzIGZpZWxkIC5zaG1l
bV9leHRyYSB3aWxsIGJlIHVzZWQgdG8gcG9pbnQNCj4+IHRvIHRoZSBib290aW5mbyBzaGFyZWQg
bWVtb3J5IGJhbmtzIC5zaG1lbV9leHRyYSBzcGFjZSwgc28gdGhhdCB0aGVyZQ0KPj4gaXMgbm90
IGZ1cnRoZXIgYWxsb2NhdGlvbiBvZiBtZW1vcnkgYW5kIGV2ZXJ5IGJhbmsgaW4gc2htX2hlYXBf
YmFua3MNCj4+IGNhbiBiZSBzYWZlbHkgaWRlbnRpZmllZCBieSB0aGUgc2htX2lkIHRvIHJlY29u
c3RydWN0IGl0cyB0cmFjZWFiaWxpdHkNCj4+IGFuZCBpZiBpdCB3YXMgYWxsb2NhdGVkIG9yIG5v
dC4NCj4gTklUIGZvciB0aGUgZnV0dXJlOiBpdCdzIGJldHRlciB0byBzcGxpdCAxMCBsaW5lcyBs
b25nIHNlbnRlbmNlIGludG8gbXVsdGlwbGUgb25lcy4NCj4gT3RoZXJ3aXNlIGl0IHJlYWRzIGRp
ZmZpY3VsdC4NCg0KSeKAmWxsIGRvLA0KDQo+PiANCj4+IHhlbi9hcmNoL2FybS9zdGF0aWMtc2ht
ZW0uYyB8IDE4NiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0NCj4+IDEgZmls
ZSBjaGFuZ2VkLCAxNTUgaW5zZXJ0aW9ucygrKSwgMzEgZGVsZXRpb25zKC0pDQo+PiANCj4+IGRp
ZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vc3RhdGljLXNobWVtLmMgYi94ZW4vYXJjaC9hcm0vc3Rh
dGljLXNobWVtLmMNCj4+IGluZGV4IGRkYWFjYmM3Nzc0MC4uOWMzYTgzMDQyZDhiIDEwMDY0NA0K
Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3N0YXRpYy1zaG1lbS5jDQo+PiArKysgYi94ZW4vYXJjaC9h
cm0vc3RhdGljLXNobWVtLmMNCj4+IEBAIC05LDYgKzksMjIgQEANCj4+ICNpbmNsdWRlIDxhc20v
c3RhdGljLW1lbW9yeS5oPg0KPj4gI2luY2x1ZGUgPGFzbS9zdGF0aWMtc2htZW0uaD4NCj4+IA0K
Pj4gK3R5cGVkZWYgc3RydWN0IHsNCj4+ICsgICAgc3RydWN0IGRvbWFpbiAqZDsNCj4+ICsgICAg
cGFkZHJfdCBnYmFzZTsNCj4+ICsgICAgY29uc3QgY2hhciAqcm9sZV9zdHI7DQo+IFlvdSBjb3Vs
ZCBzd2FwIHJvbGVfc3RyIGFuZCBnYmFzZSB0byBhdm9pZCBhIDRCIGhvbGUgb24gYXJtMzINCg0K
U3VyZSBJIHdpbGwsDQoNCj4gDQo+PiArICAgIHN0cnVjdCBzaG1lbV9tZW1iYW5rX2V4dHJhICpi
YW5rX2V4dHJhX2luZm87DQo+PiArfSBhbGxvY19oZWFwX3BhZ2VzX2NiX2V4dHJhOw0KPj4gKw0K
Pj4gK3N0YXRpYyBzdHJ1Y3QgbWVtaW5mbyBfX2luaXRkYXRhIHNobV9oZWFwX2JhbmtzID0gew0K
Pj4gKyAgICAuY29tbW9uLm1heF9iYW5rcyA9IE5SX01FTV9CQU5LUw0KPiBEbyB3ZSBleHBlY3Qg
dGhhdCBtYW55IGJhbmtzPw0KDQpOb3QgcmVhbGx5LCBidXQgSSB3YXMgdHJ5aW5nIHRvIGRvbuKA
mXQgaW50cm9kdWNlIGFub3RoZXIgdHlwZSwgZG8geW91IHRoaW5rIGl04oCZcyBiZXR0ZXIgaW5z
dGVhZCB0bw0KaW50cm9kdWNlIGEgbmV3IHR5cGUgb25seSBoZXJlLCB3aXRoIGEgbG93ZXIgYW1v
dW50IG9mIGJhbmtzPw0KDQpCZWNhdXNlIGlmIHdlIHRha2Ugc3RydWN0IHNoYXJlZF9tZW1pbmZv
LCB3ZSB3b3VsZCB3YXN0ZSBtZW0gZm9yIGl0cyDigJhleHRyYeKAmSBtZW1iZXIuDQoNCj4+IA0K
Pj4gc3RhdGljIGludCBfX2luaXQgYXNzaWduX3NoYXJlZF9tZW1vcnkoc3RydWN0IGRvbWFpbiAq
ZCwgcGFkZHJfdCBnYmFzZSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBib29sIGJhbmtfZnJvbV9oZWFwLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IG1lbWJhbmsgKnNobV9iYW5rKQ0KPj4gew0KPj4gICAg
IG1mbl90IHNtZm47DQo+PiBAQCAtMTA5LDEwICsxMzgsNyBAQCBzdGF0aWMgaW50IF9faW5pdCBh
c3NpZ25fc2hhcmVkX21lbW9yeShzdHJ1Y3QgZG9tYWluICpkLCBwYWRkcl90IGdiYXNlLA0KPj4g
ICAgIHBzaXplID0gc2htX2JhbmstPnNpemU7DQo+PiAgICAgbnJfYm9ycm93ZXJzID0gc2htX2Jh
bmstPnNobWVtX2V4dHJhLT5ucl9zaG1fYm9ycm93ZXJzOw0KPj4gDQo+PiAtICAgIHByaW50aygi
JXBkOiBhbGxvY2F0ZSBzdGF0aWMgc2hhcmVkIG1lbW9yeSBCQU5LICUjIlBSSXBhZGRyIi0lIyJQ
UklwYWRkciIuXG4iLA0KPj4gLSAgICAgICAgICAgZCwgcGJhc2UsIHBiYXNlICsgcHNpemUpOw0K
Pj4gLQ0KPj4gLSAgICBzbWZuID0gYWNxdWlyZV9zaGFyZWRfbWVtb3J5X2JhbmsoZCwgcGJhc2Us
IHBzaXplKTsNCj4+ICsgICAgc21mbiA9IGFjcXVpcmVfc2hhcmVkX21lbW9yeV9iYW5rKGQsIHBi
YXNlLCBwc2l6ZSwgYmFua19mcm9tX2hlYXApOw0KPj4gICAgIGlmICggbWZuX2VxKHNtZm4sIElO
VkFMSURfTUZOKSApDQo+PiAgICAgICAgIHJldHVybiAtRUlOVkFMOw0KPj4gDQo+PiBAQCAtMTgz
LDYgKzIwOSw3IEBAIGFwcGVuZF9zaG1fYmFua190b19kb21haW4oc3RydWN0IGtlcm5lbF9pbmZv
ICpraW5mbywgcGFkZHJfdCBzdGFydCwNCj4+IA0KPj4gc3RhdGljIGludCBfX2luaXQgaGFuZGxl
X3NoYXJlZF9tZW1fYmFuayhzdHJ1Y3QgZG9tYWluICpkLCBwYWRkcl90IGdiYXNlLA0KPj4gICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBjaGFyICpyb2xlX3N0
ciwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJvb2wgYmFu
a19mcm9tX2hlYXAsDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBtZW1iYW5rICpzaG1fYmFuaykNCj4+IHsNCj4+ICAgICBib29sIG93bmVy
X2RvbV9pbyA9IHRydWU7DQo+PiBAQCAtMTkyLDYgKzIxOSw5IEBAIHN0YXRpYyBpbnQgX19pbml0
IGhhbmRsZV9zaGFyZWRfbWVtX2Jhbmsoc3RydWN0IGRvbWFpbiAqZCwgcGFkZHJfdCBnYmFzZSwN
Cj4+ICAgICBwYmFzZSA9IHNobV9iYW5rLT5zdGFydDsNCj4+ICAgICBwc2l6ZSA9IHNobV9iYW5r
LT5zaXplOw0KPj4gDQo+PiArICAgIHByaW50aygiJXBkOiBTSE1FTSBtYXAgZnJvbSAlczogbXBo
eXMgMHglIlBSSXBhZGRyIiAtPiBncGh5cyAweCUiUFJJcGFkZHIiLCBzaXplIDB4JSJQUklwYWRk
ciJcbiIsDQo+PiArICAgICAgICAgICBkLCBiYW5rX2Zyb21faGVhcCA/ICJYZW4gaGVhcCIgOiAi
SG9zdCIsIHBiYXNlLCBnYmFzZSwgcHNpemUpOw0KPiBUaGlzIGxvb2tzIG1vcmUgbGlrZSBhIGRl
YnVnIHByaW50IHNpbmNlIEkgZG9uJ3QgZXhwZWN0IHVzZXIgdG8gd2FudCB0byBzZWUgYSBtYWNo
aW5lIGFkZHJlc3MuDQoNCnByaW50ayhYRU5MT0dfREVCVUcgPyANCg0KDQo+PiANCj4+IGludCBf
X2luaXQgcHJvY2Vzc19zaG0oc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGtlcm5lbF9pbmZvICpr
aW5mbywNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGR0X2RldmljZV9u
b2RlICpub2RlKQ0KPj4gew0KPj4gQEAgLTI2NSwzNyArMzI5LDk3IEBAIGludCBfX2luaXQgcHJv
Y2Vzc19zaG0oc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGtlcm5lbF9pbmZvICpraW5mbywNCj4+
ICAgICAgICAgcGJhc2UgPSBib290X3NobV9iYW5rLT5zdGFydDsNCj4+ICAgICAgICAgcHNpemUg
PSBib290X3NobV9iYW5rLT5zaXplOw0KPj4gDQo+PiAtICAgICAgICBpZiAoIElOVkFMSURfUEFE
RFIgPT0gcGJhc2UgKQ0KPj4gLSAgICAgICAgew0KPj4gLSAgICAgICAgICAgIHByaW50aygiJXBk
OiBob3N0IHBoeXNpY2FsIGFkZHJlc3MgbXVzdCBiZSBjaG9zZW4gYnkgdXNlcnMgYXQgdGhlIG1v
bWVudCIsIGQpOw0KPj4gLSAgICAgICAgICAgIHJldHVybiAtRUlOVkFMOw0KPj4gLSAgICAgICAg
fQ0KPj4gKyAgICAgICAgLyogInJvbGUiIHByb3BlcnR5IGlzIG9wdGlvbmFsICovDQo+PiArICAg
ICAgICBkdF9wcm9wZXJ0eV9yZWFkX3N0cmluZyhzaG1fbm9kZSwgInJvbGUiLCAmcm9sZV9zdHIp
Ow0KPiBUaGlzIGZ1bmN0aW9uIHJldHVybnMgYSB2YWx1ZSBidXQgeW91IHNlZW0gdG8gaWdub3Jl
IGl0DQoNClN1cmUsIEnigJlsbCBoYW5kbGUgdGhhdA0KDQo+PiANCj4+IC0gICAgICAgIHJldCA9
IGhhbmRsZV9zaGFyZWRfbWVtX2JhbmsoZCwgZ2Jhc2UsIHJvbGVfc3RyLCBib290X3NobV9iYW5r
KTsNCj4+IC0gICAgICAgIGlmICggcmV0ICkNCj4+IC0gICAgICAgICAgICByZXR1cm4gcmV0Ow0K
Pj4gKyAgICAgICAgICAgIGlmICggIWFsbG9jX2JhbmsgKQ0KPj4gKyAgICAgICAgICAgIHsNCj4+
ICsgICAgICAgICAgICAgICAgYWxsb2NfaGVhcF9wYWdlc19jYl9leHRyYSBjYl9hcmcgPSB7IGQs
IGdiYXNlLCByb2xlX3N0ciwNCj4+ICsgICAgICAgICAgICAgICAgICAgIGJvb3Rfc2htX2Jhbmst
PnNobWVtX2V4dHJhIH07DQo+PiArDQo+PiArICAgICAgICAgICAgICAgIC8qIHNobV9pZCBpZGVu
dGlmaWVkIGJhbmsgaXMgbm90IHlldCBhbGxvY2F0ZWQgKi8NCj4+ICsgICAgICAgICAgICAgICAg
aWYgKCAhYWxsb2NhdGVfZG9taGVhcF9tZW1vcnkoTlVMTCwgcHNpemUsIHNhdmVfbWFwX2hlYXBf
cGFnZXMsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICZjYl9hcmcpICkNCj4+ICsgICAgICAgICAgICAgICAgew0KPj4gKyAgICAgICAgICAgICAgICAg
ICAgcHJpbnRrKFhFTkxPR19FUlINCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAiRmFp
bGVkIHRvIGFsbG9jYXRlICglIlBSSXBhZGRyIk1CKSBwYWdlcyBhcyBzdGF0aWMgc2hhcmVkIG1l
bW9yeSBmcm9tIGhlYXBcbiIsDQo+IFdoeSBsaW1pdGluZyB0byBNQj8NCg0KSSB0aGluayBJIHVz
ZWQgaXQgZnJvbSBkb21haW5fYnVpbGQuYywgZG8geW91IHRoaW5rIGl04oCZcyBiZXR0ZXIgdG8g
bGltaXQgaXQgb24gS0IgaW5zdGVhZD8NCg0KPj4gDQo+PiArICAgICAgICAgICAgICAgIGZvciAo
IDsgYWxsb2NfYmFuayA8IGVuZF9iYW5rOyBhbGxvY19iYW5rKysgKQ0KPj4gKyAgICAgICAgICAg
ICAgICB7DQo+PiArICAgICAgICAgICAgICAgICAgICBpZiAoIHN0cm5jbXAoc2htX2lkLCBhbGxv
Y19iYW5rLT5zaG1lbV9leHRyYS0+c2htX2lkLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIE1BWF9TSE1fSURfTEVOR1RIKSAhPSAwICkNCj4gc2htX2lkIGhhcyBiZWVuIGFs
cmVhZHkgdmFsaWRhdGVkIGFib3ZlLCBoZW5jZSBubyBuZWVkIGZvciBhIHNhZmUgdmVyc2lvbiBv
ZiBzdHJjbXANCj4gDQoNCkkgYWx3YXlzIHRyeSB0byB1c2UgdGhlIHNhZmUgdmVyc2lvbiwgZXZl
biB3aGVuIHJlZHVuZGFudCwgSSBmZWVsIHRoYXQgaWYgc29tZW9uZSBpcyBjb3B5aW5nIHBhcnQg
b2YgdGhlIGNvZGUsDQphdCBsZWFzdCBpdCB3b3VsZCBjb3B5IGEgc2FmZSB2ZXJzaW9uLiBBbnl3
YXkgSSB3aWxsIGNoYW5nZSBpdCBpZiBpdOKAmXMgbm90IGRlc2lyYWJsZS4NCg0KQ2hlZXJzLA0K
THVjYQ0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Mon May 20 12:46:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 12:46:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726080.1130327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92P4-0008GN-Go; Mon, 20 May 2024 12:46:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726080.1130327; Mon, 20 May 2024 12:46:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92P4-0008GG-EC; Mon, 20 May 2024 12:46:10 +0000
Received: by outflank-mailman (input) for mailman id 726080;
 Mon, 20 May 2024 12:46:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s92P3-00083S-0M
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 12:46:09 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eada3d34-16a6-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 14:46:03 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-34e0d8b737eso1304643f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 05:46:03 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354c9bcaa98sm1671106f8f.4.2024.05.20.05.46.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 05:46:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eada3d34-16a6-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716209163; x=1716813963; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=jxUyZpN1suFKeFoxu2f1Gn72DbRiYZT0++YwB29/0cE=;
        b=oBn/CDdug4DnaWDkHWuo8akoASPhwRhixVH8k+01bIpAl2yn/KxKGBsCsX16VUcFGR
         Ou6OQLKthMqN4HrypwSdSoENY+ldaf+M+M90STYfkyNG3NHfUFgnZEtDmag/wIc/MI4l
         SqmwXhGcEUL92AmtjmwYWKp+Pup7w8vbjWDJY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716209163; x=1716813963;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jxUyZpN1suFKeFoxu2f1Gn72DbRiYZT0++YwB29/0cE=;
        b=lZtUZ2Zq383NecQZqkNN9UVyCF3lzw6StzHRxyCyYt/f1vZZZQflKnJJfgGZtsV0as
         pOfP1fOWJR8d6XHjr4aVaFcGBoKHfgE7X0Kjg1Mu1ERfXdIQ83x8VHQniVbn+3nXvFSR
         WHbDDSebVyBO7cixEoVwD6hYIWkythlYeA9D5PNkkxDojPftQ+QR9ibC5hxsMPlz6rTl
         /MJH9X5H0fBlV8Gb6KtEdgwPJa0McBbDqjLE2sKcdjtr/44PCVGEQ1HZ4AJS75Bzwt1L
         CqgoY20yEXp/IvWAUozO4r9jqV7ZLbcJ9RJ16ugodE0lT4Wqc8sPPtKhc/PdGz+gT8Id
         QAXg==
X-Gm-Message-State: AOJu0Yzu92mJ154ERL2qPxzBf06dTNqXJgpStyAty85A03M0Cz/DmekZ
	S5z+ojCdQ96259KpmH+FyWPc0xvetOCtzfqZQTmCKWGP/DBb/SxXdvJO8UZWUxQ=
X-Google-Smtp-Source: AGHT+IHb8piXZ+JowYCnYQ3dkHmRp6lGCgfHkOv/6sKzFT+fUMotKMhg6umOk/jrHxWZNhxFxjtKuA==
X-Received: by 2002:adf:9797:0:b0:34d:10a9:3a22 with SMTP id ffacd0b85a97d-354b8e377f5mr4745377f8f.32.1716209162779;
        Mon, 20 May 2024 05:46:02 -0700 (PDT)
Date: Mon, 20 May 2024 14:45:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 0/3] Clean the policy manipulation path in domain
 creation
Message-ID: <ZktGBkfwxEzrbojw@macbook>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <cover.1715954111.git.alejandro.vallejo@cloud.com>

On Fri, May 17, 2024 at 05:08:33PM +0100, Alejandro Vallejo wrote:
> v2:
>   * Removed xc_cpu_policy from xenguest.h
>   * Added accessors for xc_cpu_policy so the serialised form can be extracted.
>   * Modified xen-cpuid to use accessors.
> 
> ==== Original cover letter ====
> 
> In the context of creating a domain, we currently issue a lot of hypercalls
> redundantly while populating its CPU policy; likely a side effect of
> organic growth more than anything else.
> 
> However, the worst part is not the overhead (this is a glacially cold
> path), but the insane amounts of boilerplate that make it really hard to
> pick apart what's going on. One major contributor to this situation is the
> fact that what's effectively "setup" and "teardown" phases in policy
> manipulation are not factored out from the functions that perform said
> manipulations, leading to the same getters and setter being invoked many
> times, when once each would do.
> 
> Another big contributor is the code being unaware of when a policy is
> serialised and when it's not.
> 
> This patch attempts to alleviate this situation, yielding over 200 LoC
> reduction.
> 
> Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
>          from clients of libxc/libxg.
> Patch 2: Changes the (de)serialization wrappers in xenguest so they always
>          serialise to/from the internal buffers of xc_cpu_policy_t. The
>          struct is suitably expanded to hold extra information required.
> Patch 3: Performs the refactor of the policy manipulation code so that it
>          follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.

I think patch 3 is no longer part of the set?  I don't see anything
in the review of v1 that suggests patch 3 was not going to be part of
the next submission?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:02:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:02:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726090.1130336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92fA-0003Bj-Rx; Mon, 20 May 2024 13:02:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726090.1130336; Mon, 20 May 2024 13:02:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92fA-0003Bc-PO; Mon, 20 May 2024 13:02:48 +0000
Received: by outflank-mailman (input) for mailman id 726090;
 Mon, 20 May 2024 13:02:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s92f8-0003BW-LB
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:02:46 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3dfbcbcb-16a9-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 15:02:43 +0200 (CEST)
Received: from MN2PR18CA0014.namprd18.prod.outlook.com (2603:10b6:208:23c::19)
 by SJ1PR12MB6123.namprd12.prod.outlook.com (2603:10b6:a03:45a::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 13:01:20 +0000
Received: from BL6PEPF0001AB72.namprd02.prod.outlook.com
 (2603:10b6:208:23c:cafe::7f) by MN2PR18CA0014.outlook.office365.com
 (2603:10b6:208:23c::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 13:01:20 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB72.mail.protection.outlook.com (10.167.242.165) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 13:01:20 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 08:01:11 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 08:01:10 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 08:01:09 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dfbcbcb-16a9-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z0tzbxXwxG1idJr/9pbmsO2xqbrau4AXN0l//jJ240wiV9DaOaAu9PFwpBPiTqcZoF/+oWt8ubcyW5caQEfw8XEfzIFME9lBU9i7bfYdsX9oG5O2hU6UuvtihYhPKH51YzFq6zWQ89aFrgcA2AbQVTwJVtcd22e7bcs8b/rg9ruY8rNBDyzc3xv5gxZ31h7hMC78Sn9RZ4ykh0G7TfsULhpIT/K1KJIvuVVhsF8hFiuTcLDyd8uxIuIg5DXnp6jqyn0H2+XyEqRe/35N3X7IfamrkYVu+KZTqaicnpdAOtGxCzpjhn5krzW+EN6MQ7j0ZcXsY1cxyZVtsWJQeR2QUA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WXhwJ/lAdqqPpa2P2dJzsd2jdipQVKlLWeMEXgJnyTg=;
 b=akc1NAZYMjWvWUUnaqYaXfe4EI+c0WYaOYgGKSLen6Kch/yB/wJZPZu4K5/Pkfc/lWP+JgN6NiflglG9lGLtT0BAVDXazWpmQVyt5YtEBXht0ekV53okg0pCFbrXyDqctn0xPGu+45wwr7VzwVnsaKTU/omH62ftG9LhQCLu1iQSJ61l33KiUi/v1dMfQ7XyZu1XDNDVlxuNjwORt6Kyby6OFmwZ9V3fISSyKS4ebOpgKoO1+H31TNyIbC3OakqbfunRKfnzSSkdl8miShEkA8N/SwKB0gtaC1UKT4Si2iykYAyUU081DN7190DgZzuXxfLF2jjjZSQ/nn3q3UYLsg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WXhwJ/lAdqqPpa2P2dJzsd2jdipQVKlLWeMEXgJnyTg=;
 b=WIBKCFzZv3PqeaY7MryfP2nrnLyl/5SkR/nptE16Sc0Vvz8n9QAznNqyq35AUb3VkivtMkd+cEONnPr7JdbipxMtGPEKQgBk4fwdt/L6iwxZudnXXzv6L7Bdr8jBkhQJzPQHtGygUH5ivF9wRuT5F3RS6DHvnsf00lwmsdbruhg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e9fe1ee4-15ea-402f-8418-0c308afbf1db@amd.com>
Date: Mon, 20 May 2024 15:01:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-7-luca.fancellu@arm.com>
 <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
 <03C2DB73-2B91-4E74-9CBE-ACA21CDA0783@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <03C2DB73-2B91-4E74-9CBE-ACA21CDA0783@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB72:EE_|SJ1PR12MB6123:EE_
X-MS-Office365-Filtering-Correlation-Id: 43991d18-db56-4050-a3fe-08dc78ccf1be
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NTkrZEM5MktnSTd3SnFQYjRZNkJrTWZtL3J2S1lMTjQ2SjJmQStaRFJmbHV1?=
 =?utf-8?B?OXVuY2JTSCsrZDRRRTdTdW9jZGpGRzU0YWZyZVg0M0VmcXg2by9vUXNtZWJo?=
 =?utf-8?B?T0tCWWZxUjdaNzRPREtSY3NMUU0zSUR0RFhWZmxuK2VPM3g4TUxuOUVSMjZ5?=
 =?utf-8?B?cUVGVHVBVEdOS3RqelBxbzdWKzZtcEV0VmptVWxuRWNuTDFtcDBSOTVxK3BB?=
 =?utf-8?B?TGRheTNJeEhvNTA3S3JLMUF5OXFQUmxJT21yU3FyRWQyb0xRM3Ixd3dQbTIv?=
 =?utf-8?B?ZUVhWkt3bTIwcTZKWVBUUWJuU01kL05Ib3dRbE9ocW5pVFg1RnNTSkpjL1c4?=
 =?utf-8?B?QTZtV0hiNkhrV21DMnFtUXozckdQaVJsLy9Sbkk2cDhISmUxTFNEOW9uR25i?=
 =?utf-8?B?aDR5aVgvY3pObWhEb1NqQ3hQUU43YlhSd3ZqczF0UHNwRXpDT0RxclhuS0hE?=
 =?utf-8?B?d3NlSU1NSWZQeVZoZXQ3ejArTU9IbTRFcEd3ejBzSm85VkdYUmFpOEZ5eUFN?=
 =?utf-8?B?NzY4SE1WcW9qa3BwMnkxZ0x4bEJTaHlYNU9kQW5WQWR0MzdvQjFvZFJHeXNW?=
 =?utf-8?B?OURjcnF3cDhYUVJVVlJOU2E4ZnE5ek03OGpaSjFHMm5WNXVyeEJxcUplZUZj?=
 =?utf-8?B?MStMekFrVVNyWXg1RGRVR2NZbmxOSlozUFNwa05VRWMrV1BLck5xQURmeEJG?=
 =?utf-8?B?QXA5Wi94d0xMVWY1Z3BjS2VwZkZSSm1TaW9FWkw0ejNqWUxuaStDTXh5Vjlh?=
 =?utf-8?B?dUVyUGJaUDNwdTZvcjFPNXJYTEpwOVRwUGhXc0U4SlUzQnFVdWpNanVrWERV?=
 =?utf-8?B?dDkvZ3R4UWJqNnNDcFp3UFRXbmVOeGhvNlVLYnYrRy9IQkdNVnc4WDcza0l1?=
 =?utf-8?B?Z2FOUXhPTkNPbEtsN2RpbE45K29KWTZqYlczRDBsd0RjM043RFhXWXhlY3g3?=
 =?utf-8?B?RWdIY3J5bWl4SFBXKzVPdXJLSHk3bDk3VXBjdC9mN3FBOTZBRGphQ2N3SENF?=
 =?utf-8?B?d0NvTEpJa2cyNDNtTFczVjdyb000NGdjcG1OeG5JeW5XZDhrbndzWlNRWFVx?=
 =?utf-8?B?bFVmRTB2cTQ3dHBDaFVlSGV1d3lwbWdXTUZOWEc3WWhvdkN4QnVwcC84L0NZ?=
 =?utf-8?B?VUtUWkZONFFVOXhtT2loVVQyZEQ5SzVsK01SclZBcnVQUDhZWkFEbS8xdUQ2?=
 =?utf-8?B?WEMzWDJSQTRHN2ZDV3VRc0xGYlRrbzNBM2kzYStWdFQvNHhubWZ4Z2FhUlFa?=
 =?utf-8?B?VzNIOE0vS1JRTldVODJhQk5BMnhFV0xpN2Q1MERsbFQvY1I1R2hnODRYNVNM?=
 =?utf-8?B?WTY3RFB3VkZNS0g2bS96RzByTm5oRUc3OW9aQ3RleUhHbVRLQjBIUGVYY1hY?=
 =?utf-8?B?bHhLdlRxMER2NlNiUE5lNXVlVy9Edmxia3o1WG9WVkFpVE1FeTRjTGlieUU3?=
 =?utf-8?B?SjZORlpYb3puNExPdHlPTFZhRFBNK3hEand6bzF6RnUvV0g4cjZUNkM5MjNH?=
 =?utf-8?B?KzEwYzd0dEpkTlZ6R2l1WnUwY3l3Um5WZno4UlFxSzZyT29DK2d5VXhmZDEy?=
 =?utf-8?B?b1JJZWNFT2p5RGRKNEhVSzdwbHhVZlMvTFRLU3Faanl4KzAySmRDM1ZhcWxK?=
 =?utf-8?B?OFNQeFZHaEVYeTFUeDNnbDlRek5rbXNHM3RxQWw0dGdYeWRJUlR4Z3Z1dFV1?=
 =?utf-8?B?UDRNOTI3SU1oWE0zZFJHZjdiRnVrbExkRFJhbkgySWVvZDAzNUV2VFNyQWpL?=
 =?utf-8?B?R1JEcUl5WlN0RHlyemJsV2lQMzNDR0E0R2t5NWNZMWxlZGxFbTlSK1lxKzZu?=
 =?utf-8?B?VkxWcU9JUjZrZkg4aGJwUiswK3YrbHlha0RBeVBINFVvZXNRZUZHSENCYWRm?=
 =?utf-8?Q?wXM4Io3r/y6aj?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 13:01:20.3607
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 43991d18-db56-4050-a3fe-08dc78ccf1be
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB72.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6123



On 20/05/2024 14:44, Luca Fancellu wrote:
> 
> 
> Hi Michal,
> 
>> On 20 May 2024, at 12:16, Michal Orzel <michal.orzel@amd.com> wrote:
>>
>> Hi Luca,
>>
>> On 15/05/2024 16:26, Luca Fancellu wrote:
>>>
>>>
>>> This commit implements the logic to have the static shared memory banks
>>> from the Xen heap instead of having the host physical address passed from
>>> the user.
>>>
>>> When the host physical address is not supplied, the physical memory is
>>> taken from the Xen heap using allocate_domheap_memory, the allocation
>>> needs to occur at the first handled DT node and the allocated banks
>>> need to be saved somewhere, so introduce the 'shm_heap_banks' static
>>> global variable of type 'struct meminfo' that will hold the banks
>>> allocated from the heap, its field .shmem_extra will be used to point
>>> to the bootinfo shared memory banks .shmem_extra space, so that there
>>> is not further allocation of memory and every bank in shm_heap_banks
>>> can be safely identified by the shm_id to reconstruct its traceability
>>> and if it was allocated or not.
>> NIT for the future: it's better to split 10 lines long sentence into multiple ones.
>> Otherwise it reads difficult.
> 
> I’ll do,
> 
>>>
>>> xen/arch/arm/static-shmem.c | 186 ++++++++++++++++++++++++++++++------
>>> 1 file changed, 155 insertions(+), 31 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
>>> index ddaacbc77740..9c3a83042d8b 100644
>>> --- a/xen/arch/arm/static-shmem.c
>>> +++ b/xen/arch/arm/static-shmem.c
>>> @@ -9,6 +9,22 @@
>>> #include <asm/static-memory.h>
>>> #include <asm/static-shmem.h>
>>>
>>> +typedef struct {
>>> +    struct domain *d;
>>> +    paddr_t gbase;
>>> +    const char *role_str;
>> You could swap role_str and gbase to avoid a 4B hole on arm32
> 
> Sure I will,
> 
>>
>>> +    struct shmem_membank_extra *bank_extra_info;
>>> +} alloc_heap_pages_cb_extra;
>>> +
>>> +static struct meminfo __initdata shm_heap_banks = {
>>> +    .common.max_banks = NR_MEM_BANKS
>> Do we expect that many banks?
> 
> Not really, but I was trying to don’t introduce another type, do you think it’s better instead to
> introduce a new type only here, with a lower amount of banks?
I'd be ok with meminfo provided you add a reasoning behind this being meminfo and not shared_meminfo.

> 
> Because if we take struct shared_meminfo, we would waste mem for its ‘extra’ member.
Would it result in a smaller footprint overall?

> 
>>>
>>> static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
>>> +                                       bool bank_from_heap,
>>>                                        const struct membank *shm_bank)
>>> {
>>>     mfn_t smfn;
>>> @@ -109,10 +138,7 @@ static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
>>>     psize = shm_bank->size;
>>>     nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
>>>
>>> -    printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
>>> -           d, pbase, pbase + psize);
>>> -
>>> -    smfn = acquire_shared_memory_bank(d, pbase, psize);
>>> +    smfn = acquire_shared_memory_bank(d, pbase, psize, bank_from_heap);
>>>     if ( mfn_eq(smfn, INVALID_MFN) )
>>>         return -EINVAL;
>>>
>>> @@ -183,6 +209,7 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
>>>
>>> static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>>>                                          const char *role_str,
>>> +                                         bool bank_from_heap,
>>>                                          const struct membank *shm_bank)
>>> {
>>>     bool owner_dom_io = true;
>>> @@ -192,6 +219,9 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
>>>     pbase = shm_bank->start;
>>>     psize = shm_bank->size;
>>>
>>> +    printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
>>> +           d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
>> This looks more like a debug print since I don't expect user to want to see a machine address.
> 
> printk(XENLOG_DEBUG ?
Why would you want user to know the machine physical address? It's a heap address.

> 
> 
>>>
>>> int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>                        const struct dt_device_node *node)
>>> {
>>> @@ -265,37 +329,97 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>>>         pbase = boot_shm_bank->start;
>>>         psize = boot_shm_bank->size;
>>>
>>> -        if ( INVALID_PADDR == pbase )
>>> -        {
>>> -            printk("%pd: host physical address must be chosen by users at the moment", d);
>>> -            return -EINVAL;
>>> -        }
>>> +        /* "role" property is optional */
>>> +        dt_property_read_string(shm_node, "role", &role_str);
>> This function returns a value but you seem to ignore it
> 
> Sure, I’ll handle that
> 
>>>
>>> -        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
>>> -        if ( ret )
>>> -            return ret;
>>> +            if ( !alloc_bank )
>>> +            {
>>> +                alloc_heap_pages_cb_extra cb_arg = { d, gbase, role_str,
>>> +                    boot_shm_bank->shmem_extra };
>>> +
>>> +                /* shm_id identified bank is not yet allocated */
>>> +                if ( !allocate_domheap_memory(NULL, psize, save_map_heap_pages,
>>> +                                              &cb_arg) )
>>> +                {
>>> +                    printk(XENLOG_ERR
>>> +                           "Failed to allocate (%"PRIpaddr"MB) pages as static shared memory from heap\n",
>> Why limiting to MB?
> 
> I think I used it from domain_build.c, do you think it’s better to limit it on KB instead?
User can request static shmem region of as little as 1 page.

> 
>>>
>>> +                for ( ; alloc_bank < end_bank; alloc_bank++ )
>>> +                {
>>> +                    if ( strncmp(shm_id, alloc_bank->shmem_extra->shm_id,
>>> +                                 MAX_SHM_ID_LENGTH) != 0 )
>> shm_id has been already validated above, hence no need for a safe version of strcmp
>>
> 
> I always try to use the safe version, even when redundant, I feel that if someone is copying part of the code,
> at least it would copy a safe version. Anyway I will change it if it’s not desirable.
> 
> Cheers,
> Luca
> 
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:08:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:08:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726096.1130347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92kF-0003m2-G7; Mon, 20 May 2024 13:08:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726096.1130347; Mon, 20 May 2024 13:08:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92kF-0003lv-C7; Mon, 20 May 2024 13:08:03 +0000
Received: by outflank-mailman (input) for mailman id 726096;
 Mon, 20 May 2024 13:08:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cRrK=MX=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1s92kE-0003lp-6n
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:08:02 +0000
Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com
 [2001:4860:4864:20::29])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbd9045f-16a9-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 15:08:01 +0200 (CEST)
Received: by mail-oa1-x29.google.com with SMTP id
 586e51a60fabf-2451da9b4feso1263423fac.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 06:08:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbd9045f-16a9-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716210479; x=1716815279; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TO5UrABCDtztqQoaLvBeUtfIOS9kDOaSnyrlFQpHDAA=;
        b=WYA5i8oBKVXkw09ZHOBDOoG84zb2Ho3m+dyielVY2ZMLofA3jTnLRwxi3ucxFSdYpF
         wMAuNztb5gAaPcf2tRFi+1eG/HGb+WTEMldFOX9uZDdE41xMsj+x9zNioW2sP0eiz3Jx
         tounpS0NV4rWJFuWEO6ETLGRPEmZ5h2HQwXF8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716210479; x=1716815279;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=TO5UrABCDtztqQoaLvBeUtfIOS9kDOaSnyrlFQpHDAA=;
        b=OKI1As7DkgEtuvr9Pjdo/LPo6OUziglZ46uODt4qDWm/0i1ny39sjwRvZsSksC2jG/
         YdtfozunY9zqCw9dbowapjqZRdQMl1p0UYvw160p/AcTWfKSR03ZPe2EAYEmU8aEaJnI
         3DRytkMS/0Age2xOwkiHV0fzXz4bidYxfhjV65H88ah7A3SEIEQH6d5pf7rA5nnQdZqe
         5LLlITsqTvShg93RiOmHOz/eiUDXkkSXSOafV0DImOQ05tjcp9dTDT3a6OCQG9HscSkA
         6fwy+s8pRGRuuM/56sqzLTIcXq/LtrqdR1yDX1AGyMep8p3iudV5iifbLYFcm5izAepn
         njlw==
X-Gm-Message-State: AOJu0YyFAdQZJasoM7drh7Xw2wJKYP45KkqDKDOL22rg1nSsuV1kSVpr
	rN+BF4bgANCtClAgWIWsMidObcrHulcOSDmLXtqdcQ/y4w88H6/te9S3m5EJwENQJb4YfeMqKm0
	E1ovdXOEkGjxR81lLc5Hc/Odi8pfr3zBwwBciTA==
X-Google-Smtp-Source: AGHT+IF4niWJMdcwDp17C9EaQg3LWFhfY/G9LB9CWEpn7pjIgPCX7YnW5qUgA6jXHWduMcSTp7zhKQjODQ6vYhMdeaA=
X-Received: by 2002:a05:6870:7246:b0:241:98e4:5590 with SMTP id
 586e51a60fabf-24198e4a118mr35530792fac.5.1716210479677; Mon, 20 May 2024
 06:07:59 -0700 (PDT)
MIME-Version: 1.0
References: <20240520082145.1571451-1-xin.wang2@amd.com>
In-Reply-To: <20240520082145.1571451-1-xin.wang2@amd.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Mon, 20 May 2024 14:07:48 +0100
Message-ID: <CA+zSX=bRO-GvH6pa9QAnHq9UBKHNx1d3jzd+-_FjTyXvADSJAw@mail.gmail.com>
Subject: Re: [PATCH] tools/golang: Add missing golang bindings for vlan
To: Henry Wang <xin.wang2@amd.com>
Cc: xen-devel@lists.xenproject.org, Nick Rosbrook <rosbrookn@gmail.com>, 
	Anthony PERARD <anthony@xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Leigh Brown <leigh@solinno.co.uk>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 20, 2024 at 9:21=E2=80=AFAM Henry Wang <xin.wang2@amd.com> wrot=
e:
>
> It is noticed that commit:
> 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
> introduces a new "vlan" string field to libxl_device_nic. But the
> golang bindings are missing. Add it in this patch.
>
> Fixes: 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_ni=
c")
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Acked-by: George Dunlap <george.dunlap@cloud.com>


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:11:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:11:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726100.1130357 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92nR-0005G8-UF; Mon, 20 May 2024 13:11:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726100.1130357; Mon, 20 May 2024 13:11:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92nR-0005G1-Qe; Mon, 20 May 2024 13:11:21 +0000
Received: by outflank-mailman (input) for mailman id 726100;
 Mon, 20 May 2024 13:11:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2329=MX=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s92nQ-0005Fq-68
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:11:20 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01on062f.outbound.protection.outlook.com
 [2a01:111:f400:fe02::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7145dff8-16aa-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 15:11:17 +0200 (CEST)
Received: from AM5PR1001CA0042.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::19) by PAVPR08MB8991.eurprd08.prod.outlook.com
 (2603:10a6:102:32f::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 13:11:15 +0000
Received: from AMS0EPF000001AA.eurprd05.prod.outlook.com
 (2603:10a6:206:15:cafe::c0) by AM5PR1001CA0042.outlook.office365.com
 (2603:10a6:206:15::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 13:11:15 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF000001AA.mail.protection.outlook.com (10.167.16.150) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Mon, 20 May 2024 13:11:15 +0000
Received: ("Tessian outbound 9d9bf1c5d85a:v315");
 Mon, 20 May 2024 13:11:14 +0000
Received: from 94c9b09d5798.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 50818A00-B6D1-4413-9AC9-F3E143171A7A.1; 
 Mon, 20 May 2024 13:11:08 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 94c9b09d5798.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 20 May 2024 13:11:08 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DU5PR08MB10514.eurprd08.prod.outlook.com (2603:10a6:10:527::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 13:11:06 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.030; Mon, 20 May 2024
 13:11:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7145dff8-16aa-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=Mm/O9+H6YxgMI/CpF5dCTgtObRIrF81eGOzZ/Ld5VqK0xcktspXCAgo1A5+UzAxwNdzfAysOnLkjjl3cOG1ulDbDdizjRGBX6FZz5zLVdEzCIg7KK5OkCzW6f/kkjzNYWDPdze2sp/wQ257N2OnVqQaHtzU6XLTqNia4x3tKbEM3s0gSyv/T9uEcdDSBP0xnNSub8x4ZoNh6PPLeBS2oW3qMdHVRabI22kaiekcbNXkgWt9/AJuugUP/2ZqR6CcWKM4/EmG6GWg2+ZKTJaqjU7/49Lt3oXdNcs082qFzU8XZ89f2G10Po0HVMfsOu0HfS7V2ha3cRe3EU0pp9G03Tw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lp1DqpcJayaK20Ky8dRfa8nfdAjnU/mEShlYoSbGCJ4=;
 b=lkOvmo8jUkhZbQWkUDS3ArVfSpufDSrGrbxHiH/s479cqiEbvh8pQLduerR9QWTmqUbRcEE0AZUpN286kOSsmbapBaQWSQjMvl1bP/rLIDN1QAVXm0smz9ZxbN7R7Zi82yUCrWyvh0LgTIK0Q7cYJ1silbPH1wBaPAOulseIisGSMiTXlJu0B7332U9VgdWGuuSF8BXCaT+2c2Pm0KkSZkfwX4gRR9WLA5eYctkWM8U9BGVElr+/ix2Nf9n1szctM36W1V4k+n3Tu6rVDDCcuCPpg2hIroPH7w2csFmb68wWWVbuh/HQh3++No0BqTY7p1OrWrptDR3n/jX6Ql2QRQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lp1DqpcJayaK20Ky8dRfa8nfdAjnU/mEShlYoSbGCJ4=;
 b=mY0ftl9KF9F4vPTGpRc+I1iPU6wnJp8IScG8kbWICDzuCW2cNRIjxO/CkT+xVE3D8XN6I9S+Jgjuulgvo3RgrigqP9d+oYjnBZCBzJgeB9aOG1kl6XFHLkHZgViaSYazyIn+2zkzjmI+xLlOyT/arv+robIPBDYEmrqxB7uq5yE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: e31f63481a28dfb0
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gZB9ioESLKFMRHxSQTcFKgJ0jNB7rZfXHEQINCrxYQ1hcH/lfr7dRsY8pOOJOB0F9NZemNEWr7axN7AJnXyHcX+56ibYP6N6M5pIaHEf5Fwt9IuGlykcFbmexwFv8rNxzae4VQ0Q9xmqCOKQvDf4KiDeRiv6LzuBBked5/+QI7ah6oasfQQX4DEOYhsem4DAojPI+VcxrdIM6qKxSIs/L5T5r8EpQ9PeiG86W4wK60DPzZ0OUZ2IAXJVQQ/sktaebviXaAcmVboOFLZMNY+jIql4A/T56nJ0zbRKmmBnsqAh/794Fm9+E9aPJ93MVxDy4d9w449xk16FRkwV2U3ofg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lp1DqpcJayaK20Ky8dRfa8nfdAjnU/mEShlYoSbGCJ4=;
 b=UrWbugpAiJOkgdHyKK15d4Ol/UYOCs10r+fOk/R5qgqRlnyco1CwzDCi00HH4UN+gsRwxBGNDZHEM1o/Jy9yicxUNTRTRMt2CFup7SmIAutKszJ1GcXnewGXfSLKli2CKHK3z7dFjyC6ZXiUhBuHPnz7rMFjTOsvUoYtn8E//r90FBeJkv+nrqHoTSkH1+pgOfPd/Zj9/7CwyDCS1LFluVPT8d36Taih4fwLHiq8f7rzinTavRYuOP4o4bJL77mG5edAJlsOxNwpy8mMsWz6EfqUUY970iZJOoNUfYq8UnnsB2T/fcKCz7+evrDZhtC2nDjTxz23ixLSEFkF4vElwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lp1DqpcJayaK20Ky8dRfa8nfdAjnU/mEShlYoSbGCJ4=;
 b=mY0ftl9KF9F4vPTGpRc+I1iPU6wnJp8IScG8kbWICDzuCW2cNRIjxO/CkT+xVE3D8XN6I9S+Jgjuulgvo3RgrigqP9d+oYjnBZCBzJgeB9aOG1kl6XFHLkHZgViaSYazyIn+2zkzjmI+xLlOyT/arv+robIPBDYEmrqxB7uq5yE=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Topic: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
Thread-Index: AQHaptP4l1WkDlPwC0KOp+ZDl+nOHbGgALAAgAAYgICAAASzAIAAAsEA
Date: Mon, 20 May 2024 13:11:06 +0000
Message-ID: <6AC54D50-7277-44ED-8C69-9E4984C3531C@arm.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-7-luca.fancellu@arm.com>
 <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
 <03C2DB73-2B91-4E74-9CBE-ACA21CDA0783@arm.com>
 <e9fe1ee4-15ea-402f-8418-0c308afbf1db@amd.com>
In-Reply-To: <e9fe1ee4-15ea-402f-8418-0c308afbf1db@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DU5PR08MB10514:EE_|AMS0EPF000001AA:EE_|PAVPR08MB8991:EE_
X-MS-Office365-Filtering-Correlation-Id: c7c46a7d-a2c5-4d2d-7d0b-08dc78ce5453
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|366007|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?RDZpR2wzS2h2TzFKMUg1V1FaRlpOOW9WdGVhMWlQNFVqb2ZZb1h2WnltN3Yr?=
 =?utf-8?B?M2xXZmhOQWpMSTFORmtWTzBaMHdnNGtRQUdmMmtlL2hWanhkZmhBbXhvdmVj?=
 =?utf-8?B?Wk1veHRxM0h3V05QczFxU3ZhL3IzbSsvVFA5cTcvaTY4a0F1QVpveG9uMXRj?=
 =?utf-8?B?MGNxVURjR1l0VVJCNGU0b1pmWHpZZVZtemRyckFJT29zbDNjczJXNFhYWEk1?=
 =?utf-8?B?bDEyQStZb3NqS21jbitvdHpPZnd3RnRXRFVzK1JXMHNhUDVUM0Nham9iYUho?=
 =?utf-8?B?dU1mRk51WExaUXUyMElNdnM3d0JyMFlWei9JN1NYbzRBTk4ySU11Ti83MG42?=
 =?utf-8?B?MlBCRUpRdk9xSkltRCtYN3lBNE1EYkJ1UzByYWNSbTF5cFpsVWdJbVphR1Y2?=
 =?utf-8?B?SXM1clJLYVFsUS9TczJIRjdGV0FDdXp4RFNXSnRNeDd6bmwxQTIrRVM5cHZn?=
 =?utf-8?B?cU9yekFrdjBoTW9LWGQ5Z3NsZkpPVmNNajJLeWVHVXNra0xMVnhYUGdDakJQ?=
 =?utf-8?B?NWFySUFGbUVyb05ERDJDMFR1RlR3dmJTcnJjdUl2d2J2SDBQT3k5M0JmUi85?=
 =?utf-8?B?QkZMaVl6eFFKc3Y2TEhaWmJtZHRlb2Q0a3lJUndwejJlMUM2NUJkZGhXb3BQ?=
 =?utf-8?B?cm4xSFltUnN3QUNIV0hScW1RN05nOVpUL3B4UWZOaTJCdzlNaGsweDZObDVS?=
 =?utf-8?B?ajFDenBoR3kzT1RVRG9uQTB2aEZtQ00xTjRvU1l0c3FreGl4akV0YlRpYjlT?=
 =?utf-8?B?cXk5R3l2VjdMeDROa3c2cHlWelVTRElkYXdvV2RkVFA3L3NWVWh1V3kwOC9t?=
 =?utf-8?B?VXhwWEcvZXZBc1lJZktRV0JvdlQ0c0Z5bytJSk15OUJ4Y0c4VGVlWHFkNFNz?=
 =?utf-8?B?c0IwRXFVaFZBckgyM1BSMUMxVXJ5K1JoZSttcCtOYWhOVFRhdDRQblNQL1NW?=
 =?utf-8?B?Y1hhSHJkaDYrNkl1Nmg4NTBXWVFqSlF2dW9LNFoxWS9OOHFUY1dyM1BRS2F3?=
 =?utf-8?B?NlltOFJkTzVnaDB1VHFVV2NITTF3b2lpQkF6dEMvK2ZiSFI5VzgyWFdBa09m?=
 =?utf-8?B?ZVZMb2ZjbEc5U2tJRlNUWXI5bnU4UmNQaWZOQlFZbVE3Nk8rVHdId0hGVWxV?=
 =?utf-8?B?VzVJR1pic1llajFmUGNGeFM1Smt4RjR5UjlVbVF2MUtIWHFSVXhjbTlWajRI?=
 =?utf-8?B?Z2JBcm94WU94MkE0NmRqK3paRG9lTGlVVmJVS0ZFTEhMOC90dzl2a2VIdXdt?=
 =?utf-8?B?R2lBb25DQWJhV3dWMjc4TWR5VktUOGsydFFqSDBBM1UyM0lGTnpCMTBLSVUx?=
 =?utf-8?B?Ym1nYXcyc0g4N1RCMEJvRnJDY3dHMVZOWU9PSUpQc0xrbjhzR2pyR1kyUVk5?=
 =?utf-8?B?YjlFVGhKMDZKYVFOTjVUUVFMbERHYmg3V3VHdk5GMWVTQ2dRTmIwYkpJT3F4?=
 =?utf-8?B?Z21ValNNVFp4Zit1cXF2czh3bzR5dEhFOEN2VmtNY1c5QTBpWDJIMFVzTEFJ?=
 =?utf-8?B?OEZLVEllUmVpUjlBd3hSaldzWldtdWFZaUw4b1p1WWoxd1RjcVAwR3lRaHJI?=
 =?utf-8?B?YWIxQVF2SjhXMG9vcEk5WFU5bDMyMFdpbVJ0aHB0b0VvTFREc29kMURqaStQ?=
 =?utf-8?B?a2plYk1NaFljTi9xcmJDbFJGOTg2TXl3T1BUS21OYjYzSXJlL01ES0FSYlp2?=
 =?utf-8?B?a2ZhOXBhZThCd1lHbjVtdko2ZnZQeXp2UFdxWlJYY0d4VnFkOGJJUWc3cmxo?=
 =?utf-8?B?dlJOUXhPWHZxdUVZUjd1RVV4Wm1WVzUyaWNTd0RhT0dhUXNIbUkwM2V3VW8r?=
 =?utf-8?B?VStXWE9zWDVjMXNQOWhwQT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <AAF75BF59172814195B37D3E25583A6B@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU5PR08MB10514
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF000001AA.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	32dc8a4e-d276-486f-55e6-08dc78ce4f3c
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|35042699013|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RHViSkp0TVo5QytTd0RrK2R2TnBXU0ZjOVJtdzc1ZzFSWGMwOXB6eTdUVmdn?=
 =?utf-8?B?bmEzZ1VkUlhZbC9tM0RiNzEvd2pIQ3oydVpidlRHOTY0dUZDVy9tYlArUE9O?=
 =?utf-8?B?T0NRb3EzQ0ZpUVVvZ1pyL09iMGtIMVQxaU5vMTZVN2x2M01YTERqSlRxTTM3?=
 =?utf-8?B?VFlkV25ldW1yYzJHVUVpL1NsSVlzeU1PQ2tyazBrcDQ3WGh1V2pMdmJ0TFhl?=
 =?utf-8?B?bG01azlSQmtwb3pQd0x1dytOdEFwU3NORlBIYUFnMTB6enJLNGcxdkxqbXZL?=
 =?utf-8?B?VXRNcEZKcEw3cVJnWDRZajl4Rk9DQ3dtVm1iRHNCS1pnb2JVUndjZ0l3NnFL?=
 =?utf-8?B?WE12L2NrRDJWNEJQSFVnMWVPZkxBWXY4dGg3RHlPSmRKak5MUTVJOEw0Uk1v?=
 =?utf-8?B?YldRT2NaOWNBSmtUa296cVlrcHgvOUcyLzJnUENKZFZYL1ZHWDdMNlB4ZFNm?=
 =?utf-8?B?b0pYVFIyQzNhRWoyZ1FGK3BLM3JPWmNUUU11VHRjd0xVRVdJc3lFc3FEMDBG?=
 =?utf-8?B?MzE2WTZlWlFwaHFqVlY2OGlSZklWeVk2QWdMc0UraUhlMmVTUXBHM09rN0pq?=
 =?utf-8?B?eHBtUXFLSEVVanF6QWNYU2tWZXJ0ZjliZ2hQR2o1NnAvNUJvZ2wxZVp6WVI3?=
 =?utf-8?B?MUJJb1pxVFRQSmFqZVM3UDM2aE4wQmxoNUEwYWovaFRTQUdYdTY0RVdSVjRI?=
 =?utf-8?B?WHk4dEp3MlVmRk9Pbm9XUWRwd3YrOHZ0Z0ptU1FuMUNtdktoQlBxYnpIdERt?=
 =?utf-8?B?T0htT3pVczA5RWFXT2FDVmZ3cWpibUJHejY5KzZOWXRBVUxSSVl3VnY1NUJm?=
 =?utf-8?B?amdManNkOFFOaTYwRitOc2RTbUg2ZThDMW1DWk1acWRER3pxeCtLa2JSM1dy?=
 =?utf-8?B?MzdBUzB6V2FLcnFMVEsxZXRjNDduODF5d056Y0ErR1BWcXpsNXpXYnd6NjNC?=
 =?utf-8?B?R05jREdFQzlLNERSTUYrRkIwUHMraDRobXB3VWdPM2ZZazByZStVYWQzOWs4?=
 =?utf-8?B?alRzZXJJS1NwMHAzOWlacVBtUDVnU2s2cC9TTkNqbFIyRU5JazhHRVhHYnpr?=
 =?utf-8?B?MW9ERE9VWDhtb1dSVE5kV0JNTXlMUXYwN2tqL3loZHMyL004VEswZm1vdFFr?=
 =?utf-8?B?cU8vMitQZmdIZjBtaWcwMjI2SUF4dlArZTNWVEE2eU42djZwSFJGMmdYRzJL?=
 =?utf-8?B?TDh0dGxPREFuZnFPVk50MFJ6dWdsUnhDeDlZSmhUa3RIczZPNEtVU3lLZ21w?=
 =?utf-8?B?cmFpTVJxT2h6L0hhZkNBWnd4VVRGSGNEVFRuWk12Q3NlcXpaNk1UNjFKK2hw?=
 =?utf-8?B?VnZEeE1HZzN4VzM5Tk5qQ0VJUTVRVFkxbDdCaFZOMTJneGkxMnFJVlN3dEVa?=
 =?utf-8?B?WmdlOUFvVmdES3h0L1NJZGs4dzlROUMyTnljaE5iQTZ5ZGsrSlUzMFFhdldo?=
 =?utf-8?B?QlZIb2lreDd5VmRnZFN5UkZDQnlLZHlMQzhPVFZ3bnpUSDV5MHU1RUtJYUhC?=
 =?utf-8?B?UjNjKy9Bb1JDMWdSVEJXWVRKQlMzSjcwdjd6bmhHRFY2MFVBaXF4TUlic3M4?=
 =?utf-8?B?bEtTUFk1VVlxdWhMS1BIK2wzVlNpdUtxZGxYMDgzT1JsQVFFK1lqQ25KVUZj?=
 =?utf-8?B?TWNxbVBCZnBOa09hak91Zy9rRUhhVUtOSnEvYlNCa0lmamQzSU9OdzloSldx?=
 =?utf-8?B?R3RnbkhUSXhRdW81UnhSS25YSUNKeHBZRXlPSzRQWUNOWEdHb0Z6aG1tTlNq?=
 =?utf-8?B?YUZzU0NzZEtnbnZKWVYzeUhOSTFpUDZqSXRHSnRRcDVrMXdWUFVLaTVEL2Nm?=
 =?utf-8?B?ektiNWRwa0MxQit5OGpHbXFodktTb1RIdTdSVksrbWVnclZuMFhaV0t0Y1lq?=
 =?utf-8?Q?GG2GlMBunmFOQ?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(35042699013)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 13:11:15.2340
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c7c46a7d-a2c5-4d2d-7d0b-08dc78ce5453
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF000001AA.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB8991

Pj4gDQo+Pj4gDQo+Pj4+ICsgICAgc3RydWN0IHNobWVtX21lbWJhbmtfZXh0cmEgKmJhbmtfZXh0
cmFfaW5mbzsNCj4+Pj4gK30gYWxsb2NfaGVhcF9wYWdlc19jYl9leHRyYTsNCj4+Pj4gKw0KPj4+
PiArc3RhdGljIHN0cnVjdCBtZW1pbmZvIF9faW5pdGRhdGEgc2htX2hlYXBfYmFua3MgPSB7DQo+
Pj4+ICsgICAgLmNvbW1vbi5tYXhfYmFua3MgPSBOUl9NRU1fQkFOS1MNCj4+PiBEbyB3ZSBleHBl
Y3QgdGhhdCBtYW55IGJhbmtzPw0KPj4gDQo+PiBOb3QgcmVhbGx5LCBidXQgSSB3YXMgdHJ5aW5n
IHRvIGRvbuKAmXQgaW50cm9kdWNlIGFub3RoZXIgdHlwZSwgZG8geW91IHRoaW5rIGl04oCZcyBi
ZXR0ZXIgaW5zdGVhZCB0bw0KPj4gaW50cm9kdWNlIGEgbmV3IHR5cGUgb25seSBoZXJlLCB3aXRo
IGEgbG93ZXIgYW1vdW50IG9mIGJhbmtzPw0KPiBJJ2QgYmUgb2sgd2l0aCBtZW1pbmZvIHByb3Zp
ZGVkIHlvdSBhZGQgYSByZWFzb25pbmcgYmVoaW5kIHRoaXMgYmVpbmcgbWVtaW5mbyBhbmQgbm90
IHNoYXJlZF9tZW1pbmZvLg0KPiANCj4+IA0KPj4gQmVjYXVzZSBpZiB3ZSB0YWtlIHN0cnVjdCBz
aGFyZWRfbWVtaW5mbywgd2Ugd291bGQgd2FzdGUgbWVtIGZvciBpdHMg4oCYZXh0cmHigJkgbWVt
YmVyLg0KPiBXb3VsZCBpdCByZXN1bHQgaW4gYSBzbWFsbGVyIGZvb3RwcmludCBvdmVyYWxsPw0K
DQpXZWxsIG92ZXJhbGwgeWVzLCBtZW1pbmZvIG5vdyBpcyAyNTUgYmFua3MsIHNoYXJlZF9tZW1p
bmZvIGlzIDY0IGluIHRvdGFsLCBldmVuIGlmIHdlIHVzZSAzMiBvZiB0aGVtIGFuZA0KMzIgYXJl
IHdhc3RlZC4NCg0KT3RoZXJ3aXNlLCBhcyBJIHNhaWQsIEkgY291bGQgZG8gc29tZXRoaW5nIGxp
a2UgdGhpcyBpbiB0aGlzIG1vZHVsZToNCg0Kc3RhdGljIHN0cnVjdCBzaGFyZWRfaGVhcF9tZW1p
bmZvIHsNCiAgICBzdHJ1Y3QgbWVtYmFua3NfaGRyIGNvbW1vbjsNCiAgICBzdHJ1Y3QgbWVtYmFu
ayBiYW5rW05SX1NITUVNX0JBTktTXTsNCn0gX19pbml0ZGF0YSBzaG1faGVhcF9iYW5rcyA9IHsN
CiAgICAuY29tbW9uLm1heF9iYW5rcyA9IE5SX1NITUVNX0JBTktTDQp9Ow0KDQo+Pj4+IA0KPj4+
PiAgICBib29sIG93bmVyX2RvbV9pbyA9IHRydWU7DQo+Pj4+IEBAIC0xOTIsNiArMjE5LDkgQEAg
c3RhdGljIGludCBfX2luaXQgaGFuZGxlX3NoYXJlZF9tZW1fYmFuayhzdHJ1Y3QgZG9tYWluICpk
LCBwYWRkcl90IGdiYXNlLA0KPj4+PiAgICBwYmFzZSA9IHNobV9iYW5rLT5zdGFydDsNCj4+Pj4g
ICAgcHNpemUgPSBzaG1fYmFuay0+c2l6ZTsNCj4+Pj4gDQo+Pj4+ICsgICAgcHJpbnRrKCIlcGQ6
IFNITUVNIG1hcCBmcm9tICVzOiBtcGh5cyAweCUiUFJJcGFkZHIiIC0+IGdwaHlzIDB4JSJQUklw
YWRkciIsIHNpemUgMHglIlBSSXBhZGRyIlxuIiwNCj4+Pj4gKyAgICAgICAgICAgZCwgYmFua19m
cm9tX2hlYXAgPyAiWGVuIGhlYXAiIDogIkhvc3QiLCBwYmFzZSwgZ2Jhc2UsIHBzaXplKTsNCj4+
PiBUaGlzIGxvb2tzIG1vcmUgbGlrZSBhIGRlYnVnIHByaW50IHNpbmNlIEkgZG9uJ3QgZXhwZWN0
IHVzZXIgdG8gd2FudCB0byBzZWUgYSBtYWNoaW5lIGFkZHJlc3MuDQo+PiANCj4+IHByaW50ayhY
RU5MT0dfREVCVUcgPw0KPiBXaHkgd291bGQgeW91IHdhbnQgdXNlciB0byBrbm93IHRoZSBtYWNo
aW5lIHBoeXNpY2FsIGFkZHJlc3M/IEl0J3MgYSBoZWFwIGFkZHJlc3MuDQoNCk9oIG9rIHlvdXIg
Y29tbWVudCB3YXMgYWJvdXQgcmVtb3ZpbmcgaXQsIG9rIEkgZG9u4oCZdCBoYXZlIHN0cm9uZyBv
YmplY3Rpb25zIHRvIHRoYXQuDQoNCg0KPj4gDQo+PiANCj4+Pj4gDQo+Pj4+IC0gICAgICAgIHJl
dCA9IGhhbmRsZV9zaGFyZWRfbWVtX2JhbmsoZCwgZ2Jhc2UsIHJvbGVfc3RyLCBib290X3NobV9i
YW5rKTsNCj4+Pj4gLSAgICAgICAgaWYgKCByZXQgKQ0KPj4+PiAtICAgICAgICAgICAgcmV0dXJu
IHJldDsNCj4+Pj4gKyAgICAgICAgICAgIGlmICggIWFsbG9jX2JhbmsgKQ0KPj4+PiArICAgICAg
ICAgICAgew0KPj4+PiArICAgICAgICAgICAgICAgIGFsbG9jX2hlYXBfcGFnZXNfY2JfZXh0cmEg
Y2JfYXJnID0geyBkLCBnYmFzZSwgcm9sZV9zdHIsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAg
IGJvb3Rfc2htX2JhbmstPnNobWVtX2V4dHJhIH07DQo+Pj4+ICsNCj4+Pj4gKyAgICAgICAgICAg
ICAgICAvKiBzaG1faWQgaWRlbnRpZmllZCBiYW5rIGlzIG5vdCB5ZXQgYWxsb2NhdGVkICovDQo+
Pj4+ICsgICAgICAgICAgICAgICAgaWYgKCAhYWxsb2NhdGVfZG9taGVhcF9tZW1vcnkoTlVMTCwg
cHNpemUsIHNhdmVfbWFwX2hlYXBfcGFnZXMsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgJmNiX2FyZykgKQ0KPj4+PiArICAgICAgICAgICAgICAg
IHsNCj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19FUlINCj4+Pj4gKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICJGYWlsZWQgdG8gYWxsb2NhdGUgKCUiUFJJcGFkZHIi
TUIpIHBhZ2VzIGFzIHN0YXRpYyBzaGFyZWQgbWVtb3J5IGZyb20gaGVhcFxuIiwNCj4+PiBXaHkg
bGltaXRpbmcgdG8gTUI/DQo+PiANCj4+IEkgdGhpbmsgSSB1c2VkIGl0IGZyb20gZG9tYWluX2J1
aWxkLmMsIGRvIHlvdSB0aGluayBpdOKAmXMgYmV0dGVyIHRvIGxpbWl0IGl0IG9uIEtCIGluc3Rl
YWQ/DQo+IFVzZXIgY2FuIHJlcXVlc3Qgc3RhdGljIHNobWVtIHJlZ2lvbiBvZiBhcyBsaXR0bGUg
YXMgMSBwYWdlLg0KDQpPayBJ4oCZbGwgY2hhbmdlIHRvIEtCDQoNCj4gDQo+PiANCj4+Pj4gDQo+
Pj4+ICsgICAgICAgICAgICAgICAgZm9yICggOyBhbGxvY19iYW5rIDwgZW5kX2Jhbms7IGFsbG9j
X2JhbmsrKyApDQo+Pj4+ICsgICAgICAgICAgICAgICAgew0KPj4+PiArICAgICAgICAgICAgICAg
ICAgICBpZiAoIHN0cm5jbXAoc2htX2lkLCBhbGxvY19iYW5rLT5zaG1lbV9leHRyYS0+c2htX2lk
LA0KPj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgTUFYX1NITV9JRF9MRU5H
VEgpICE9IDAgKQ0KPj4+IHNobV9pZCBoYXMgYmVlbiBhbHJlYWR5IHZhbGlkYXRlZCBhYm92ZSwg
aGVuY2Ugbm8gbmVlZCBmb3IgYSBzYWZlIHZlcnNpb24gb2Ygc3RyY21wDQo+Pj4gDQo+PiANCj4+
IEkgYWx3YXlzIHRyeSB0byB1c2UgdGhlIHNhZmUgdmVyc2lvbiwgZXZlbiB3aGVuIHJlZHVuZGFu
dCwgSSBmZWVsIHRoYXQgaWYgc29tZW9uZSBpcyBjb3B5aW5nIHBhcnQgb2YgdGhlIGNvZGUsDQo+
PiBhdCBsZWFzdCBpdCB3b3VsZCBjb3B5IGEgc2FmZSB2ZXJzaW9uLiBBbnl3YXkgSSB3aWxsIGNo
YW5nZSBpdCBpZiBpdOKAmXMgbm90IGRlc2lyYWJsZS4NCj4+IA0KPj4gQ2hlZXJzLA0KPj4gTHVj
YQ0KPj4gDQo+PiANCj4gDQo+IH5NaWNoYWwNCg0KQ2hlZXJzLA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:13:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:13:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726107.1130367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92pp-0006CU-Cr; Mon, 20 May 2024 13:13:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726107.1130367; Mon, 20 May 2024 13:13:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92pp-0006CN-AE; Mon, 20 May 2024 13:13:49 +0000
Received: by outflank-mailman (input) for mailman id 726107;
 Mon, 20 May 2024 13:13:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s92pn-0006CH-Nq
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:13:47 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c8e5dbf7-16aa-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 15:13:45 +0200 (CEST)
Received: from CH2PR04CA0008.namprd04.prod.outlook.com (2603:10b6:610:52::18)
 by DS7PR12MB8346.namprd12.prod.outlook.com (2603:10b6:8:e5::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 13:13:42 +0000
Received: from CH1PEPF0000AD78.namprd04.prod.outlook.com
 (2603:10b6:610:52:cafe::9f) by CH2PR04CA0008.outlook.office365.com
 (2603:10b6:610:52::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 13:13:42 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000AD78.mail.protection.outlook.com (10.167.244.56) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 13:13:39 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 08:13:35 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 08:13:34 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8e5dbf7-16aa-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TSkZakAmVIB7plj/gRBeK3tDwRmKL5x6ww7/DsPLJWKV0bH8yUcEfxCq/4yKdVgtCglzzElbA8KIIG6L4mBKoyvGsdL0vqfudkUhb12ItmvNzb8xemxT7FvO2P/h8Wz1Ul8zCdNXygSXfx7hYMMpqqzOeeN1XH/2OPEJoq12KougqTgdYUZjrNhpvTDy8c5eKGpLB3ohx4Yrs5UgnngZZ7JDfRBqwKhAsRSJid3VXwCfTLw7s9C80PFaLzDPmFuTf2uu0PQrL6R+9s1RZEQwjaC052eYX95jFLR+mzYw7AwHSR4FbLx/9fahi05xwZZYYhALMJImSmqmMNXykiapkQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Bjj6UaZejEdBGuyqu/WcFnlnU49Zl3j4wbcK49I1K68=;
 b=Bv7a3ZgruAwnsK1fp73UQHGRV3wHckRtsw99DTJCOdKmfHDDTKvGZOKhx1yXpvpdxBOktMgEm72JolWm+JCPqVqZn0UadG61mNFoTasMcX257poCBRl+5wuNTUHbOd3weKuo7Zqgk+sdaWvqTPKb6el5pm6e9UfmaK+lfUL9nxNF8PDm9Ya5PFOjLHW3b9/QwsJQ5mgkIfQaS2yGQg81bC/VL9Rg1rWQ3P6sItYmvDPbGzvvkT42zjGJF/82VycjdxLd3qHyBcKaIpBfbgbHYLEr2ZHrxW0MmIyGsiI5pBRudTt1NxwHMYtpPV5EMpMFz+ON6hacMm6qKntwLAs/dQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bjj6UaZejEdBGuyqu/WcFnlnU49Zl3j4wbcK49I1K68=;
 b=PEkg8wtr0xk18wNbVxdQNI7bMde5okGQEhAOpv1rsA+Po9CffkeMF21d/76H128YcNo77B0J1tpskerHLStMeuA7IsvWnQiY1aLt22SEajWlP09YRT3jr8OeXVvsLYeyBYA9+UGAZJSbi0Hy6BuDXYH/vROE8iKdjGeyMQrYGHs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <07db0c52-5063-4f2d-82e9-26ca8167524c@amd.com>
Date: Mon, 20 May 2024 15:13:34 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240515142626.3480640-1-luca.fancellu@arm.com>
 <20240515142626.3480640-7-luca.fancellu@arm.com>
 <cbe1fb4a-9c2b-48eb-acb0-6726aecdfe85@amd.com>
 <03C2DB73-2B91-4E74-9CBE-ACA21CDA0783@arm.com>
 <e9fe1ee4-15ea-402f-8418-0c308afbf1db@amd.com>
 <6AC54D50-7277-44ED-8C69-9E4984C3531C@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <6AC54D50-7277-44ED-8C69-9E4984C3531C@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD78:EE_|DS7PR12MB8346:EE_
X-MS-Office365-Filtering-Correlation-Id: 3baf3c13-52a5-4c38-797d-08dc78ceab63
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Z1VMUTRLZml0a0NyRnNkeWhiN2hHREpRazZURFVGVDdqUWEydC9IdEVjRmdT?=
 =?utf-8?B?T3M4enZrTVc5Z09rbDVFVjY4YVU0QlV6Yk54QmFEYmFtUjQ0djlIODJFVTN1?=
 =?utf-8?B?TzQvNzNaWE1KZEo4Q0E4M0FqSWxORGFxZFVyU1IvVjM1NTd6Q3NvS3REc3ls?=
 =?utf-8?B?UnZzUzZVbEVIMGJTaDF4eHQ0TUxYZG52YzIzaE5yS0hSdWRTL1RsRnZJOTB4?=
 =?utf-8?B?ZXcvdXV3dktGQUEvREZKekpiWmVSczFJNjZOakFrVkkrdEpSYUhBN1Fwb05p?=
 =?utf-8?B?NGlZM0M0VTVpTklsM2praEU4bC9IZjR1Kys5OXhqb2pmLy9oQkVaSVJjcGth?=
 =?utf-8?B?ODhLRDlmZERyUk4rVzF4ZEp3YldKUzY3N2NCQS9VU2dQU3IxektXcmZFU1du?=
 =?utf-8?B?aklwM25ONXR5RXJQOHN6bEpTaGpkdHh3eVdGa0NJQXFJWVB3UjUweGdsYWYr?=
 =?utf-8?B?V0ZFcUFsaisrcmE4aGJ5cUV4dDlQenhHeG5OY1RkcGNOTFRqSkFxNDFYYTNV?=
 =?utf-8?B?aEIvdDdVcko5VTgvRkROdWNuS3NFQ1BuUXptanVDWWlFSHlxd3BHTDVBMlZ2?=
 =?utf-8?B?d0RmYTh6MVpzS2k4bnd0YmhsYzIzdVllcDNjVGhtZUYvaGk4RnZNWUlYM2RI?=
 =?utf-8?B?WGNVTVE2eEtET3BZVENEMVMrS2hpMEh5T283d2dpQW5WOG9MZTZPRzZMTmJk?=
 =?utf-8?B?RVp1bk5NaHJtSVQ4OU9NM0NOU3hGUmtVbEFCQnVJbnVhdEZQU0ZIR2dKS3Ns?=
 =?utf-8?B?Z3BkSUJHbVFaRWZHUjB3VEtHUjE4TnBZMEtLeWdGRXQzR3FXL2E1aWw0NEdo?=
 =?utf-8?B?Z1JSWkhLbXBkUjhGbjhYRSt6dzErVlJzNmdLZFNyOGpkQ2lGS3RiVnRacmV3?=
 =?utf-8?B?ZFE1N0ZEWXdIMHVYRUhRVi9FZVUvb0o3cFhYLzh3eG9BVE1SSjBtTjBKSjYz?=
 =?utf-8?B?K0Nsd3RiTExDcmU0M0c5Z0tlbThpdkV3MndQZWMvMGhFOW9HdjhhUW5VQW9H?=
 =?utf-8?B?ZDVnUTYzQk9aSkNVS1dqQmRhUXFpVzMxSlAyUHF1ZGVMamFFS3YyRE5EYXNU?=
 =?utf-8?B?VEtFVjFvQUJmTFFqQk1pbmFlTEg0RlRQQldWWFNRbit5czNmL040UnVGWlQ0?=
 =?utf-8?B?NG9SRytya096UythU0k4SGRobU1rUnhmMi9FN2VBK0EwQ0lQRy9rZTNhZzdQ?=
 =?utf-8?B?dFNHbkI5a2NId2drV0E3VlltLzA0czF3NzNGNEJ1aWtSdmY5QkVsMkJtZDQr?=
 =?utf-8?B?dStXQ3lYWVlkZUt6TG15UDJycFpTOC9JWEQ1U21wbVp4cVNiMEpTcXU2cnlR?=
 =?utf-8?B?QXNkT2N1aCtqU3dxRjdrc0ZWajgrOW1MOEF5bS9GL3JNalFkeDNJVGtESEl4?=
 =?utf-8?B?cWtqby9FZ3RVZzQ2SlROb1RBN08wTm1sTmNTMjY1VjN2aEsvVXlJS2tiaGFu?=
 =?utf-8?B?alk4WkZhZityeUtlbXlLbStUWlpYc2JIZjZsMk9rZEpRQnQyOHFjMU54ODBL?=
 =?utf-8?B?NE9Yam1WTGJUWkJRbllreWR1YkdCUU5TVFpMVG02M0NTVnp6RHNRaFVmNDdY?=
 =?utf-8?B?dHNqdTFMaGw2d3lFSStwam1TV3AxaTViaEhyL1ZXTW9EM3lSbHNXQ0s5ejRY?=
 =?utf-8?B?Q3YySjFsdk1wU2VZK0RBM0pnblordU4yNFRMVExPUHRWNnAxRFFrTjFKRFlN?=
 =?utf-8?B?QWxGRmRxMmh2djFXUkFlUzREL3Vhb3Y3Q0VkM0JGMzVXMmlxcC9mVFBTdnQz?=
 =?utf-8?B?bWJlVVJUUjZuSkNiTmZLZUhaVy9ldmRERFY2UGQyU3ZUandNTEszY09yOVRh?=
 =?utf-8?B?QmQ4WUF3MjkwNlJTK2djZCtvSUFEeHp5QXErdXg1b0xYZlVaQmZGRkxRRHR0?=
 =?utf-8?Q?jqEOm30t0z0O9?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 13:13:39.2224
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3baf3c13-52a5-4c38-797d-08dc78ceab63
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD78.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8346



On 20/05/2024 15:11, Luca Fancellu wrote:
> 
> 
>>>
>>>>
>>>>> +    struct shmem_membank_extra *bank_extra_info;
>>>>> +} alloc_heap_pages_cb_extra;
>>>>> +
>>>>> +static struct meminfo __initdata shm_heap_banks = {
>>>>> +    .common.max_banks = NR_MEM_BANKS
>>>> Do we expect that many banks?
>>>
>>> Not really, but I was trying to don’t introduce another type, do you think it’s better instead to
>>> introduce a new type only here, with a lower amount of banks?
>> I'd be ok with meminfo provided you add a reasoning behind this being meminfo and not shared_meminfo.
>>
>>>
>>> Because if we take struct shared_meminfo, we would waste mem for its ‘extra’ member.
>> Would it result in a smaller footprint overall?
> 
> Well overall yes, meminfo now is 255 banks, shared_meminfo is 64 in total, even if we use 32 of them and
> 32 are wasted.
> 
> Otherwise, as I said, I could do something like this in this module:
> 
> static struct shared_heap_meminfo {
>     struct membanks_hdr common;
>     struct membank bank[NR_SHMEM_BANKS];
> } __initdata shm_heap_banks = {
>     .common.max_banks = NR_SHMEM_BANKS
> };
If that's all it takes to avoid defining unnecessarily big variable, then I'd be ok with it.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:15:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:15:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726111.1130377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92r9-0006iO-PA; Mon, 20 May 2024 13:15:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726111.1130377; Mon, 20 May 2024 13:15:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s92r9-0006iH-LC; Mon, 20 May 2024 13:15:11 +0000
Received: by outflank-mailman (input) for mailman id 726111;
 Mon, 20 May 2024 13:15:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gPfm=MX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s92r8-0006iB-98
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:15:10 +0000
Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com
 [2607:f8b0:4864:20::72d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa77e651-16aa-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 15:15:08 +0200 (CEST)
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-792b9790f6bso227050785a.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 06:15:08 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a36423a92csm41678996d6.112.2024.05.20.06.15.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 06:15:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa77e651-16aa-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716210907; x=1716815707; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=kUypuKFd8Vm8ZmOySG+b6uiZ/7i/EML1EjSCQNuHnj8=;
        b=Kf23oeE4FHN4LfeHu1ntQibYPf3aAZP2Ylu3FW821de3E4jXEjenkixsUL7fMX2/pc
         TUWOuKARyFyypi8tdpapqKudr2OIXuzN7j0W/fvINuP9Wj3Mx/qVpeTZ83DXVwx2BIc2
         vfPF4QRMpZDehUVpPe9+jYee9mif8eTwVjj1I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716210907; x=1716815707;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kUypuKFd8Vm8ZmOySG+b6uiZ/7i/EML1EjSCQNuHnj8=;
        b=la3EXuIRxXBElOfQEzSbO5GMuVTov7mWRP9VPCTNudV32toXylktrvIg07sreQlD+m
         HCk6nxNL3VQ7dyCA62nrtCjT4TDbExFIGZqycsWgsgIPPtJ5WVk/cz3xitZWjMCMDaOu
         koZY5X8ebNTrFrjq8e3s+3bRVPaM12SR75eguc0gsfKfQrvqCRO69jUEV9U8dzktFXvP
         KyxrXiVzMFAggkpmJsy5uuAyuxqrynQmTtFCHW5JK/qSiKtNYv9Pokq8SP4SBHtCrcpm
         VWKfJktloTyI9nw5a6PR0wx9/vnIFP6PupGjifrrY8L524zyXqmdHYOJSRZy2sdc/DhV
         /w1w==
X-Forwarded-Encrypted: i=1; AJvYcCUbVsUfzqbX2z8e41kPs/awmBYNepAX2Dm1rHBuA1bDMxkfll6aW6jFrCkKGl0T1Wzc9lrF5yebWIUxO8ESamN26dwWnlbEzT7tsAdfo64=
X-Gm-Message-State: AOJu0YxYDn5uA5ZP6X4OQwNzLvsPwSBU3c+UcieQez/mXgAM7wjodvdY
	fSB+ixYSOWv7J0WoJ7QuCHsADszA+lMR2UU6q3t5shAQyUrzBfqk3hjqE+HiXG8=
X-Google-Smtp-Source: AGHT+IFeBVUrqXWJUTJPmIt5xAGMOI1Bj2AMe8U0gCCIvHDRO6J44tw7GjU/j+L8mAFiWvgvtlsx2g==
X-Received: by 2002:a05:6214:5f11:b0:6a0:b262:db1b with SMTP id 6a1803df08f44-6a168199df9mr373080186d6.14.1716210907075;
        Mon, 20 May 2024 06:15:07 -0700 (PDT)
Message-ID: <ea446501-e4fe-49bb-8888-d44d0292cbcf@citrix.com>
Date: Mon, 20 May 2024 14:15:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] x86/msi: prevent watchdog triggering when
 dumping MSI state
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240517135605.94781-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240517135605.94781-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 17/05/2024 2:56 pm, Roger Pau Monne wrote:
> Use the same check that's used in dump_irqs().
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon May 20 13:48:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 13:48:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726139.1130395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s93Mi-0003Xy-7y; Mon, 20 May 2024 13:47:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726139.1130395; Mon, 20 May 2024 13:47:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s93Mi-0003Xr-59; Mon, 20 May 2024 13:47:48 +0000
Received: by outflank-mailman (input) for mailman id 726139;
 Mon, 20 May 2024 13:47:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s93Mh-0003Xl-3u
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 13:47:47 +0000
Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com
 [2607:f8b0:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 892bd2ad-16af-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 15:47:45 +0200 (CEST)
Received: by mail-ot1-x32c.google.com with SMTP id
 46e09a7af769-6f054c567e2so2072135a34.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 06:47:45 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1d7040sm111823476d6.107.2024.05.20.06.47.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 06:47:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 892bd2ad-16af-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716212864; x=1716817664; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=XbOtBl8kjp1FlMIjczDv0NWOu6HXg6n8obSgX1DIgCQ=;
        b=XJPO5DtgjFGZkXs3CAsvv/fpx7LEkamzjqVQ8SG9Qgsx3aHRx2iJ744mqAMt9pZbWq
         19O0fqeJma078bOyfQnN4isDNy26DD6xyglcYAMsb4j/fW0+8ZqIq2Y5AJ+yC7I0+AWN
         EiBV3ATzPFn3NRt6QYYHzJNyk8MzgFw9Y/IRg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716212864; x=1716817664;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XbOtBl8kjp1FlMIjczDv0NWOu6HXg6n8obSgX1DIgCQ=;
        b=twLzEiWAPQ6d7D5HURosgE+fYbROjoWEONrW9tTrwy+nnJtDmBy+vr8lApnCUPmWB4
         Mky15aZwyhJgAYe3HnjlKIOFt3atSsO4WEapgofXQhGNQAYtmrWYW6JP/8OeHZqRpQmN
         I/Rx78WT64IqyXFuH47TMYu10yp1Qa96inRv6NyQGnphXF/A0rBYnjyB18PDxh0xZ70V
         aN+83vln0iafTq+lxye4StFG67e/UwpNsIG62D/4hvk4qpu/gop/b69KA8FSuLiYN9SH
         g4qnYiK6CJvIl+nC2vyWpW7scw/WOrErph6WDkIEy97+oYre14v8Z92uzZDztNmONSQ6
         UnHw==
X-Gm-Message-State: AOJu0YwYGZCQLZ0mQh3ra3IkmQIwe5mxNl9/7fKXiRMeQkcgr6Exv55u
	H0akLNhIBicHfZA7tpcCIxciaB5W7wflz6J1U0iZxDWR7D9ec3v3wy5Z5nJEtcn32GlD2srNXqA
	x
X-Google-Smtp-Source: AGHT+IGgmnJtSPXrZR+Yd7Xuagma0e7UBjML7B7DHQRqNtNBzfWMyG3td+P5ohyP1mgB0VYdJfeDdQ==
X-Received: by 2002:a05:6830:1e57:b0:6f1:2ca5:4d08 with SMTP id 46e09a7af769-6f12ca55035mr12907677a34.24.1716212864195;
        Mon, 20 May 2024 06:47:44 -0700 (PDT)
Date: Mon, 20 May 2024 15:47:41 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
Message-ID: <ZktUfWZD67zkvaeJ@macbook>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
 <c6c89012d189a2c3077b3e224942b33028e29336.1715954111.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <c6c89012d189a2c3077b3e224942b33028e29336.1715954111.git.alejandro.vallejo@cloud.com>

On Fri, May 17, 2024 at 05:08:34PM +0100, Alejandro Vallejo wrote:
> The idea is to use xc_cpu_policy_t as a single object containing both the
> serialised and deserialised forms of the policy. Note that we need lengths
> for the arrays, as the serialised policies may be shorter than the array
> capacities.
> 
> * Add the serialised lengths to the struct so we can distinguish
>   between length and capacity of the serialisation buffers.
> * Remove explicit buffer+lengths in serialise/deserialise calls
>   and use the internal buffer inside xc_cpu_policy_t instead.
> * Refactor everything to use the new serialisation functions.
> * Remove redundant serialization calls and avoid allocating dynamic
>   memory aside from the policy objects in xen-cpuid. Also minor cleanup
>   in the policy print call sites.
> 
> No functional change intended.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * Removed v1/patch1.
>   * Added the accessors suggested in feedback.
> ---
>  tools/include/xenguest.h            |  8 ++-
>  tools/libs/guest/xg_cpuid_x86.c     | 98 ++++++++++++++++++++---------
>  tools/libs/guest/xg_private.h       |  2 +
>  tools/libs/guest/xg_sr_common_x86.c | 54 ++++++----------
>  tools/misc/xen-cpuid.c              | 43 ++++---------
>  5 files changed, 104 insertions(+), 101 deletions(-)
> 
> diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
> index e01f494b772a..563811cd8dde 100644
> --- a/tools/include/xenguest.h
> +++ b/tools/include/xenguest.h
> @@ -799,14 +799,16 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy);
>  
>  /* Manipulate a policy via architectural representations. */
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
>  int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
>                                 const xen_cpuid_leaf_t *leaves,
>                                 uint32_t nr);
>  int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>                                const xen_msr_entry_t *msrs, uint32_t nr);
> +int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
> +int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                           const xen_msr_entry_t **msrs, uint32_t *nr);
>  
>  /* Compatibility calculations. */
>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4453178100ad..4f4b86b59470 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
>      }
>  }
>  
> -static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
> -                              unsigned int nr_leaves, unsigned int nr_entries)
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>      int rc;
>  
>      rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> -                                    nr_leaves, &err_leaf, &err_subleaf);
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
>      if ( rc )
>      {
>          if ( err_leaf != -1 )
> @@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
>      }
>  
>      rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> -                                  nr_entries, &err_msr);
> +                                  policy->nr_msrs, &err_msr);
>      if ( rc )
>      {
>          if ( err_msr != -1 )
> @@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);
>      if ( rc )
>      {
>          errno = -rc;
> @@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);
>      if ( rc )
>      {
>          errno = -rc;
> @@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
> -    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
>      int rc;
>  
> -    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
> -                                 policy->msrs, &nr_msrs);
> +    rc = xc_cpu_policy_serialise(xch, policy);
>      if ( rc )
>          return rc;
>  
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
> -                                  nr_msrs, policy->msrs,
> +    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
> +                                  policy->nr_msrs, policy->msrs,

I would be tempted to just pass the policy to
xc_set_domain_cpu_policy() and get rid of the separate cpuid and msrs
serialized arrays, but that hides (or makes it less obvious) that the
policy needs to be serialized before providing to
xc_set_domain_cpu_policy().  Just a rant, no need to change it here.

>                                    &err_leaf, &err_subleaf, &err_msr);
>      if ( rc )
>      {
> @@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>      return rc;
>  }
>  
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
>  {
> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
> +    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
>      int rc;
>  
> -    if ( leaves )
> +    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
> +    if ( rc )
>      {
> -        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
> -        if ( rc )
> -        {
> -            ERROR("Failed to serialize CPUID policy");
> -            errno = -rc;
> -            return -1;
> -        }
> +        ERROR("Failed to serialize CPUID policy");
> +        errno = -rc;
> +        return -1;
>      }
>  
> -    if ( msrs )
> +    p->nr_leaves = nr_leaves;
> +
> +    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &nr_msrs);
> +    if ( rc )
>      {
> -        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
> -        if ( rc )
> -        {
> -            ERROR("Failed to serialize MSR policy");
> -            errno = -rc;
> -            return -1;
> -        }
> +        ERROR("Failed to serialize MSR policy");
> +        errno = -rc;
> +        return -1;
>      }
>  
> +    p->nr_msrs = nr_msrs;
> +
>      errno = 0;
>      return 0;
>  }
> @@ -1012,6 +1012,42 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>      return rc;
>  }
>  
> +int xc_cpu_policy_get_leaves(xc_interface *xch,
> +                             const xc_cpu_policy_t *policy,
> +                             const xen_cpuid_leaf_t **leaves,
> +                             uint32_t *nr)
> +{
> +    if ( !policy )
> +    {
> +        ERROR("Failed to fetch CPUID leaves from policy object");
> +        errno = -EINVAL;
> +        return -1;
> +    }
> +
> +    *leaves = policy->leaves;
> +    *nr = policy->nr_leaves;
> +
> +    return 0;
> +}
> +
> +int xc_cpu_policy_get_msrs(xc_interface *xch,
> +                           const xc_cpu_policy_t *policy,
> +                           const xen_msr_entry_t **msrs,
> +                           uint32_t *nr)
> +{
> +    if ( !policy )
> +    {
> +        ERROR("Failed to fetch MSRs from policy object");
> +        errno = -EINVAL;
> +        return -1;
> +    }
> +
> +    *msrs = policy->msrs;
> +    *nr = policy->nr_msrs;
> +
> +    return 0;
> +}

My preference would probably be to return NULL or
xen_{leaf,msr}_entry_t * from those, as we can then avoid an extra
leaves/msrs parameter.  Again I'm fine with leaving it like this.

> +
>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
>                                   xc_cpu_policy_t *guest)
>  {
> diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
> index d73947094f2e..a65dae818f3d 100644
> --- a/tools/libs/guest/xg_private.h
> +++ b/tools/libs/guest/xg_private.h
> @@ -177,6 +177,8 @@ struct xc_cpu_policy {
>      struct cpu_policy policy;
>      xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
>      xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
> +    uint32_t nr_leaves;
> +    uint32_t nr_msrs;
>  };
>  #endif /* x86 */
>  
> diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
> index 563b4f016877..832047756e58 100644
> --- a/tools/libs/guest/xg_sr_common_x86.c
> +++ b/tools/libs/guest/xg_sr_common_x86.c
> @@ -1,4 +1,5 @@
>  #include "xg_sr_common_x86.h"
> +#include "xg_sr_stream_format.h"
>  
>  int write_x86_tsc_info(struct xc_sr_context *ctx)
>  {
> @@ -45,54 +46,37 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
>  int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
>  {
>      xc_interface *xch = ctx->xch;
> -    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
> -    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
> -    uint32_t nr_leaves = 0, nr_msrs = 0;
> -    xc_cpu_policy_t *policy = NULL;
> +    struct xc_sr_record record;
> +    xc_cpu_policy_t *policy = xc_cpu_policy_init();
>      int rc;
>  
> -    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
> -    {
> -        PERROR("Unable to get CPU Policy size");
> -        return -1;
> -    }
> -
> -    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
> -    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
> -    policy = xc_cpu_policy_init();
> -    if ( !cpuid.data || !msrs.data || !policy )
> -    {
> -        ERROR("Cannot allocate memory for CPU Policy");
> -        rc = -1;
> -        goto out;
> -    }
> -
> -    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
> +    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
>      {
>          PERROR("Unable to get d%d CPU Policy", ctx->domid);
>          rc = -1;
>          goto out;
>      }
> -    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
> -                                 msrs.data, &nr_msrs) )
> -    {
> -        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
> -        rc = -1;
> -        goto out;
> -    }
>  
> -    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
> -    if ( cpuid.length )
> +    record = (struct xc_sr_record) {
> +        .type = REC_TYPE_X86_CPUID_POLICY,
> +        .data = policy->leaves,
> +        .length = policy->nr_leaves * sizeof(*policy->leaves),
> +    };
> +    if ( record.length )
>      {
> -        rc = write_record(ctx, &cpuid);
> +        rc = write_record(ctx, &record);
>          if ( rc )
>              goto out;
>      }


You could maybe write this as:

if ( policy->nr_leaves )
{
    const struct xc_sr_record r = {
        .type = REC_TYPE_X86_CPUID_POLICY,
        .data = policy->leaves,
        .length = policy->nr_leaves * sizeof(*policy->leaves),
    };

    rc = write_record(ctx, &record);
}

(same for the msr record)

>  
> -    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
> -    if ( msrs.length )
> +    record = (struct xc_sr_record) {
> +        .type = REC_TYPE_X86_MSR_POLICY,
> +        .data = policy->msrs,
> +        .length = policy->nr_msrs * sizeof(*policy->msrs),
> +    };
> +    if ( record.length )
>      {
> -        rc = write_record(ctx, &msrs);
> +        rc = write_record(ctx, &record);
>          if ( rc )
>              goto out;
>      }
> @@ -100,8 +84,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
>      rc = 0;
>  
>   out:
> -    free(cpuid.data);
> -    free(msrs.data);
>      xc_cpu_policy_destroy(policy);
>  
>      return rc;
> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> index 8893547bebce..1c9ba6d32060 100644
> --- a/tools/misc/xen-cpuid.c
> +++ b/tools/misc/xen-cpuid.c
> @@ -409,17 +409,21 @@ static void dump_info(xc_interface *xch, bool detail)
>      free(fs);
>  }
>  
> -static void print_policy(const char *name,
> -                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
> -                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
> +static void print_policy(xc_interface *xch, const char *name, const xc_cpu_policy_t *policy)

Line length.

>  {
> -    unsigned int l;
> +    const xen_cpuid_leaf_t *leaves;
> +    const xen_msr_entry_t *msrs;
> +    uint32_t nr_leaves, nr_msrs;
> +
> +    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) ||
> +         xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
> +        err(1, "print_policy()");

Shouldn't the error message be "xc_cpu_policy_get_{leaves,msrs}()"
instead, as one of those is the cause of the error?

Other err() usages do print the function triggering the error, not the
function context name.

>  
>      printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
>      printf(" CPUID:\n");
>      printf("  %-8s %-8s -> %-8s %-8s %-8s %-8s\n",
>             "leaf", "subleaf", "eax", "ebx", "ecx", "edx");
> -    for ( l = 0; l < nr_leaves; ++l )
> +    for ( uint32_t l = 0; l < nr_leaves; ++l )
>      {
>          /* Skip empty leaves. */
>          if ( !leaves[l].a && !leaves[l].b && !leaves[l].c && !leaves[l].d )
> @@ -432,7 +436,7 @@ static void print_policy(const char *name,
>  
>      printf(" MSRs:\n");
>      printf("  %-8s -> %-16s\n", "index", "value");
> -    for ( l = 0; l < nr_msrs; ++l )
> +    for ( uint32_t l = 0; l < nr_msrs; ++l )

I would be tempted to leave `l` as-is, seeing as there's no real need
to modify it in the patch context, and the patch is already fairly
long.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 14:17:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 14:17:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726166.1130405 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s93pF-0008LJ-KZ; Mon, 20 May 2024 14:17:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726166.1130405; Mon, 20 May 2024 14:17:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s93pF-0008LC-GS; Mon, 20 May 2024 14:17:17 +0000
Received: by outflank-mailman (input) for mailman id 726166;
 Mon, 20 May 2024 14:17:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gPfm=MX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s93pD-0008L6-V0
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 14:17:15 +0000
Received: from mail-oo1-xc2b.google.com (mail-oo1-xc2b.google.com
 [2607:f8b0:4864:20::c2b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a6fc3a87-16b3-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 16:17:13 +0200 (CEST)
Received: by mail-oo1-xc2b.google.com with SMTP id
 006d021491bc7-5b277e17e15so2489938eaf.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 07:17:13 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2971c5sm1180724485a.51.2024.05.20.07.17.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 07:17:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6fc3a87-16b3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716214632; x=1716819432; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=UhnDU8sUL5y+X8ol6Kjrz9AYZn8Ic1bHjj3hFBtZoXA=;
        b=B6S/9q0WUqtk4QP/l/18ES8sTJNWuVf1pRk7nlHIK8lSAqMSSAFYH8klfOpePYFku0
         XbQDQW45tE9pdYIhatOtnWz4RLZeSKNBKpEwCYEsusQwC9RVMtzAA6dl+Z3MtDS2v/al
         wWfCR6dJEeVlxa9ypXkWScm53h+jFt+0b4ZLk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716214632; x=1716819432;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UhnDU8sUL5y+X8ol6Kjrz9AYZn8Ic1bHjj3hFBtZoXA=;
        b=XB1fgl1A4vKGXm5F5XlAjiVYnnJwoBzNbZslCFh/vXCvDkiWuLA7PTsAN/HBUSUayT
         +eNnTGXpjk0pefJeIWIRQzlevT+uehVFw2PavJcEJw2zc1QAjaUEOVe4HaYTTUrMoSv3
         Cslsn8KXKmVG6ktxXN/Vyyneo/8jmCovPJFjl+uE45rzlIXMC+WAXvj9/gR6SSmXL2zQ
         CzFqH+aSvHfwKFqmbRo0zIa4YfDL1bx3rjTrFiabo7j4w/Dvh9b4w27TK0RBWtG2tFAS
         6bTophkENznGw4Mrq2mQpDF8H7t01WttAX4lr6d+XVDc07qRIpsb//R9/jxsYF+Apvii
         +B0g==
X-Gm-Message-State: AOJu0YyBKjFYQ0abmtIr2IxpvIXFlBW/b4rz/2LNdgIqenPQJTU0emXA
	7QMfgzDF3CGzPfcfpQp+NsV9skpMoVboRTZNQcxoKsXIRKpl3GZXmQybSR/zaAY=
X-Google-Smtp-Source: AGHT+IGx9Ht91H+O8exJywiqP5d+wMgeIlRYdoGWm40YSv6Xt6FusoIaExbGcHxv/o+P4EocPvP6bw==
X-Received: by 2002:a05:6870:3b05:b0:248:e519:2185 with SMTP id 586e51a60fabf-248e51943c6mr8134032fac.26.1716214631980;
        Mon, 20 May 2024 07:17:11 -0700 (PDT)
Message-ID: <165c385b-a688-4e87-9e3a-e87ada724595@citrix.com>
Date: Mon, 20 May 2024 15:17:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/4] x86/gen-cpuid: Minor cleanup
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240510224002.2324578-1-andrew.cooper3@citrix.com>
 <20240510224002.2324578-2-andrew.cooper3@citrix.com>
 <ZkMPOxF2aR6wp1Vj@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZkMPOxF2aR6wp1Vj@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/05/2024 8:14 am, Roger Pau Monné wrote:
> On Fri, May 10, 2024 at 11:39:59PM +0100, Andrew Cooper wrote:
>> Rename INIT_FEATURE_NAMES to INIT_FEATURE_NAME_TO_VAL as we're about to gain a
>> inverse mapping of the same thing.
>>
>> Use dict.items() unconditionally.  iteritems() is a marginal perf optimsiation
>> for Python2 only, and simply not worth the effort on a script this small.
> My understanding is that what used to be iteritems() in Python 2 is
> the behavior of items() in Python 3 (return a generator instead of a
> copy of the dictionary list).

Yes-ish.  They're actually now view() objects following the official
stabilisation of the internal format, which are more-efficient-still in
the common case.

>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon May 20 14:34:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 14:34:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726173.1130425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s945T-0003UZ-5m; Mon, 20 May 2024 14:34:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726173.1130425; Mon, 20 May 2024 14:34:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s945T-0003US-1d; Mon, 20 May 2024 14:34:03 +0000
Received: by outflank-mailman (input) for mailman id 726173;
 Mon, 20 May 2024 14:34:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gPfm=MX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s945S-0003Nh-IG
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 14:34:02 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00023732-16b6-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 16:34:01 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-571ba432477so8320133a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 07:34:01 -0700 (PDT)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733beb89e7sm15485718a12.21.2024.05.20.07.34.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 07:34:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00023732-16b6-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716215641; x=1716820441; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Md//2DDasxVOHTchjm5rVEUtH/pkUFEDxQJCwSHBhkE=;
        b=jAMXymFkLFEK1wCRr5+pc89ocaXgU7j6nL59SMu/rjEDfzKN0CLY0+iT3Ddzu0RKQh
         ATlhmnGq68ytnQvgEDhVdJnrso3/oVWQItYS4kx0jUYEywHCVyj5c/weHAlRwPE2jGhU
         lWV+slSpcVvQLzbv7CsRnDCgc3O/WTJtrWGeU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716215641; x=1716820441;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Md//2DDasxVOHTchjm5rVEUtH/pkUFEDxQJCwSHBhkE=;
        b=uzAyJ28GQiEAtB33EqXfAGR6GsKWRFeq9dAx0o8cE+BgKwj9akMtj4mmg7C/3WCRUm
         PZV2sgEqlhaAAR5eSBeRtY0Q3ApuXd+A2H3JLQX2UD5fKCsjnm9B7VKz3/g1QG3iSyH8
         iYRkuEkWYT1GbV6rxn5TsIrz/+1puuLhVAy/ta5h59lisFs8KgQtDHuyDESdudQgWt3R
         VUQlB7x0+juGAPosZ8++yh0EN60q8dVjSsl8VjtKokiyj8UpU3+NvCV3Kua7UEN+F99i
         wAAzYBtZUH1Wqws9CbatkxYtt0QCVYP+irT4QRxlpKOWwNBbMjfatzdy+n6AttVgIBsG
         yNCg==
X-Gm-Message-State: AOJu0YwlmunHd611M88yk0vj/gez3I0HJP55ygV8OmnwhnlTAA5BJAms
	iFmsv3B0W4X4w86P43ZGHKNz06G4sa0vlOv1WyylW9YxN13EDCQ+XUGHBAszDDHlqDAe3UPIY60
	R
X-Google-Smtp-Source: AGHT+IENdmG5RR1qRY/y+xXcpkCZHOGG9Vy+036FJKk8wEigHETxhcEJa7kbq3MdbBETqzSPctwZRw==
X-Received: by 2002:a50:9f88:0:b0:572:a7fb:b6ad with SMTP id 4fb4d7f45d1cf-5734d590353mr21323474a12.5.1716215640765;
        Mon, 20 May 2024 07:34:00 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>
Subject: [PATCH v3.5 3/4] tools/xen-cpuid: Use automatically generated feature names
Date: Mon, 20 May 2024 15:33:59 +0100
Message-Id: <20240520143359.3376849-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
References: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Roger Pau Monné <roger.pau@citrix.com>

Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
are needed.

In xen-cpuid.c, sanity check at build time that leaf_info[] and
feature_names[] are of sensible length.

As dump_leaf() rendered missing names as numbers, always dump leaves even if
we don't have the leaf name.  This conversion was argumably missed in commit
59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

Differences in names are:

 sysenter    -> sep
 tm          -> tm1
 ds-cpl      -> dscpl
 est         -> eist
 sse41       -> sse4-1
 sse42       -> sse4-2
 movebe      -> movbe
 tsc-dl      -> tsc-deadline
 rdrnd       -> rdrand
 hyper       -> hypervisor
 mmx+        -> mmext
 fxsr+       -> ffxsr
 pg1g        -> page1gb
 3dnow+      -> 3dnowext
 cmp         -> cmp-legacy
 cr8d        -> cr8-legacy
 lzcnt       -> abm
 msse        -> misalignsse
 3dnowpf     -> 3dnowprefetch
 nodeid      -> nodeid-msr
 dbx         -> dbext
 tsc-adj     -> tsc-adjust
 fdp-exn     -> fdp-excp-only
 deffp       -> no-fpu-sel
 <24>        -> bld
 ppin        -> amd-ppin
 lfence+     -> lfence-dispatch
 ppin        -> intel-ppin
 energy-ctrl -> energy-filtering

Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
one.  Several of the + names would be nice to keep as were, but doing so isn't
nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
but we intentionally don't list them, so I'm not worried.

Thoughts?

v3:
 * Rework somewhat.
 * Insert aliases of common_1d.

v4:
 * Pad at the gen stage.  I don't like this, but I'm clearly outvoted on the matter.
---
 tools/misc/xen-cpuid.c | 16 ++++++++--------
 xen/tools/gen-cpuid.py | 29 +++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 6ee835b22949..51009683da1b 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -11,6 +11,7 @@
 #include <xenguest.h>
 
 #include <xen-tools/common-macros.h>
+#include <xen/lib/x86/cpuid-autogen.h>
 
 static uint32_t nr_features;
 
@@ -291,6 +292,8 @@ static const struct {
 
 #define COL_ALIGN "24"
 
+static const char *const feature_names[] = INIT_FEATURE_VAL_TO_NAME;
+
 static const char *const fs_names[] = {
     [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
     [XEN_SYSCTL_cpu_featureset_host]    = "Host",
@@ -304,12 +307,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
 {
     unsigned i;
 
-    if ( !strs )
-    {
-        printf(" ???");
-        return;
-    }
-
     for ( i = 0; i < 32; ++i )
         if ( leaf & (1u << i) )
         {
@@ -327,6 +324,10 @@ static void decode_featureset(const uint32_t *features,
 {
     unsigned int i;
 
+    /* If this trips, you probably need to extend leaf_info[] above. */
+    BUILD_BUG_ON(ARRAY_SIZE(leaf_info) != FEATURESET_NR_ENTRIES);
+    BUILD_BUG_ON(ARRAY_SIZE(feature_names) != FEATURESET_NR_ENTRIES * 32);
+
     printf("%-"COL_ALIGN"s        ", name);
     for ( i = 0; i < length; ++i )
         printf("%08x%c", features[i],
@@ -338,8 +339,7 @@ static void decode_featureset(const uint32_t *features,
     for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
     {
         printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
-        if ( leaf_info[i].name )
-            dump_leaf(features[i], leaf_info[i].strs);
+        dump_leaf(features[i], &feature_names[i * 32]);
         printf("\n");
     }
 }
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 79d7f5c8e1c9..601eec608983 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -470,6 +470,35 @@ def write_results(state):
     state.output.write(
 """}
 
+""")
+
+    state.output.write(
+"""
+#define INIT_FEATURE_VAL_TO_NAME { \\
+""")
+
+    for name, bit in sorted(state.values.items()):
+        state.output.write(
+            '    [%s] = "%s",\\\n' % (bit, name)
+            )
+
+        # Add the other alias for 1d/e1d common bits.  64 is the difference
+        # between 1d and e1d.
+        if bit in state.common_1d:
+            state.output.write(
+                '    [%s] = "%s",\\\n' % (64 + bit, name)
+            )
+
+    # Pad to an exact multiple of FEATURESET_SIZE if necessary
+    pad_feat = state.nr_entries * 32 - 1
+    if not state.names.get(pad_feat):
+        state.output.write(
+            '    [%s] = NULL,\\\n' % (pad_feat, )
+        )
+
+    state.output.write(
+"""}
+
 """)
 
     for idx, text in enumerate(state.bitfields):
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon May 20 14:34:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 14:34:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726172.1130414 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s945R-0003GK-Uw; Mon, 20 May 2024 14:34:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726172.1130414; Mon, 20 May 2024 14:34:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s945R-0003GC-Re; Mon, 20 May 2024 14:34:01 +0000
Received: by outflank-mailman (input) for mailman id 726172;
 Mon, 20 May 2024 14:34:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+3cx=MX=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s945Q-0003G6-1O
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 14:34:00 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fd81f8d0-16b5-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 16:33:57 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 922B38006F;
 Mon, 20 May 2024 15:33:56 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd81f8d0-16b5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1716215636;
	bh=umNSVV7br+LWKfpTsCOsdtnYlHjj/kq7PBIz0k7y5gM=;
	h=Date:From:To:Cc:Subject:From;
	b=g0q7NgJmLs1wkcowodrDQ9+3ZDBeSjCIzLXEkKc49Lijx2vkEr9VMOerwi3p3u2wc
	 AbETm2LaYC+JFh9UzVEf2QNUr0o8WjOZs6bQE5K3ZBD/kV1dpKE3fRnYEtFweuEU8M
	 Td3W12JjWyp/Vbz7JQtdD9JRuY+cDxjLan0SLUhY=
MIME-Version: 1.0
Date: Mon, 20 May 2024 15:33:56 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Xen Devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Jason Andryuk
 <jandryuk@gmail.com>
Subject: xl list -l aborts with double free error with vlan patches
Message-ID: <0d8fbde01bedf97dcd7540ceea6c42ab@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hello,

When running xl list -l with my VLAN patches applied, a double free 
error is raised. I'm unable to determine why.

broken: 0cc01c603f4287233715a526b056bc20e0e97412 (HEAD) tools/xl: add 
vlan keyword to vif option
okay:   3bc14e4fa4b9832888710759a7dbe5f0d239f33b tools/libs/light: Add 
vlan field to libxl_device_nic
okay:   e27fc7d15eab79e604e8b8728778594accc23cf1 tools/xentop: Fix cpu% 
sort order

Any suggestions appreciated...

Debug run and backtrace:

(gdb) run -vvvv list -l
Starting program: /usr/sbin/xl -vvvv list -l
[Thread debugging using libthread_db enabled]
Using host libthread_db library 
"/lib/x86_64-linux-gnu/libthread_db.so.1".
libxl: debug: libxl_domain.c:2295:libxl_retrieve_domain_configuration: 
ao 0x555555592f50: create: how=(nil) callback=(nil) 
poller=0x55555559dde0
[Detaching after fork from child process 1044]
libxl: debug: libxl_domain.c:2311:libxl_retrieve_domain_configuration: 
ao 0x555555592f50: inprogress: poller=0x55555559dde0, flags=i
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vbd from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vif from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vtpm from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vusb from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vusb from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
pci from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vdispl from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: No 
vsnd from xenstore
libxl: debug: libxl_qmp.c:1920:libxl__ev_qmp_dispose:  ev 0x5555555a0240
libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x555555592f50: 
complete, rc=0
libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x555555592f50: 
destroy
libxl: debug: libxl_domain.c:2295:libxl_retrieve_domain_configuration: 
Domain 1:ao 0x55555559f830: create: how=(nil) callback=(nil) 
poller=0x55555559dde0
[Detaching after fork from child process 1045]
libxl: debug: libxl_domain.c:2311:libxl_retrieve_domain_configuration: 
Domain 1:ao 0x55555559f830: inprogress: poller=0x55555559dde0, flags=i
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No vtpm from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No vusb from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No vusb from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No pci from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No vdispl from xenstore
libxl: debug: libxl_domain.c:2587:retrieve_domain_configuration_end: 
Domain 1:No vsnd from xenstore
libxl: debug: libxl_qmp.c:1920:libxl__ev_qmp_dispose: Domain 1: ev 
0x5555555a2820
libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x55555559f830: 
complete, rc=0
libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x55555559f830: 
destroy
free(): double free detected in tcache 2

Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (threadid=<optimized out>, 
signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, 
signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007ffff7d5de8f in __pthread_kill_internal (signo=6, 
threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x00007ffff7d0efb2 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/posix/raise.c:26
#3  0x00007ffff7cf9472 in __GI_abort () at ./stdlib/abort.c:79
#4  0x00007ffff7d52430 in __libc_message (action=action@entry=do_abort, 
fmt=fmt@entry=0x7ffff7e6c459 "%s\n") at 
../sysdeps/posix/libc_fatal.c:155
#5  0x00007ffff7d677aa in malloc_printerr (str=str@entry=0x7ffff7e6f098 
"free(): double free detected in tcache 2") at ./malloc/malloc.c:5660
#6  0x00007ffff7d69a36 in _int_free (av=0x7ffff7ea5c60 <main_arena>, 
p=0x5555555a0f90, have_lock=have_lock@entry=0) at ./malloc/malloc.c:4469
#7  0x00007ffff7d6be8f in __GI___libc_free (mem=<optimized out>) at 
./malloc/malloc.c:3385
#8  0x00007ffff7f160a3 in libxl__free_all (gc=gc@entry=0x55555559f870) 
at libxl_internal.c:86
#9  0x00007ffff7f27c11 in libxl__ao__destroy (ctx=0x555555591850, 
ao=ao@entry=0x55555559f830) at libxl_event.c:2039
#10 0x00007ffff7f27c3e in ao__check_destroy (ctx=<optimized out>, 
ao=ao@entry=0x55555559f830) at libxl_event.c:2028
#11 0x00007ffff7f27c76 in ao__manip_leave (ctx=<optimized out>, 
ao=ao@entry=0x55555559f830) at libxl_event.c:2021
#12 0x00007ffff7f2948f in libxl__ao_inprogress 
(ao=ao@entry=0x55555559f830, file=file@entry=0x7ffff7f80866 
"libxl_domain.c", line=line@entry=2311,
     func=func@entry=0x7ffff7f814a0 <__func__.3> 
"libxl_retrieve_domain_configuration") at libxl_event.c:2235
#13 0x00007ffff7f46add in libxl_retrieve_domain_configuration 
(ctx=<optimized out>, domid=<optimized out>, 
d_config=d_config@entry=0x7fffffffe510, ao_how=ao_how@entry=0x0)
     at libxl_domain.c:2311
#14 0x00005555555744f2 in list_domains_details 
(info=info@entry=0x5555555aa1e0, nb_domain=2) at xl_info.c:479
#15 0x0000555555575e21 in main_list (argc=<optimized out>, 
argv=<optimized out>) at xl_info.c:582
#16 0x0000555555561b55 in main (argc=2, argv=0x7fffffffecd8) at xl.c:451

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Mon May 20 14:52:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 14:52:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726185.1130434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94NU-00077X-Go; Mon, 20 May 2024 14:52:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726185.1130434; Mon, 20 May 2024 14:52:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94NU-00077Q-E0; Mon, 20 May 2024 14:52:40 +0000
Received: by outflank-mailman (input) for mailman id 726185;
 Mon, 20 May 2024 14:52:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s94NT-00077K-AB
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 14:52:39 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 98e3604c-16b8-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 16:52:37 +0200 (CEST)
Received: from SA9PR10CA0010.namprd10.prod.outlook.com (2603:10b6:806:a7::15)
 by CY8PR12MB7265.namprd12.prod.outlook.com (2603:10b6:930:57::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 14:52:31 +0000
Received: from SA2PEPF00003F66.namprd04.prod.outlook.com
 (2603:10b6:806:a7:cafe::a2) by SA9PR10CA0010.outlook.office365.com
 (2603:10b6:806:a7::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 14:52:31 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00003F66.mail.protection.outlook.com (10.167.248.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 14:52:30 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 09:52:28 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 09:52:26 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98e3604c-16b8-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RjwTV1r2oOKd9W230ry/OCXCdEHzeoV3ktYpD5DPQXWBMj+FiEx0/LFPryhWHkjebjovBqYKI+BhRIXiGgtUxtqC34HXspI9/jJv1Nf6RCrLb4s4yHVlU6ibygf/+zxI8RSVVrElrZ0dHUCYmXzB0Sp/mdnY2uNmA9vt6Rz6vMNGcUPxFYRmpPLJwILGThGBDdcu/9CGH0nRIVMSLCp43RZOxB2gnwxxG7kuKM/xszGYw0lp7wmX9NcsI/EqH0BfuwJkZ2VkqeQPdoR44Ve5t0dEn21ZQ0lsUjTdWBOhdz/6hgMgEuOGXVf5dIr4DXDzP3iYQaTLyHqJtnglCwVjwA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EUZTRuRZpTJvKvnJre3jqWAN7n3G4U+8+LDtO2FmSXo=;
 b=JwbcwpY0yNV41wWCK8W10352XKsymvanbe42ZMXjC5iJKO3S+AJBkjTbACBHxZusGBa3HHrlWzwvtfnNHKgPaNTO5le/OysR7fc09inSr+Mxm6TVgIFyG2UWr7dc/Qhj/Uv8iUVP4p+CR/Hp152H34hPHIIILnIz5o/D4eguS7j32SBoFSULCOi9G0o2VbR69lr40ypwK6+hsDNhYzL5wA/MQZhYEX1aEh442wLr80ZtZVN7kAaRLOkIAIiomQzrnahPZzdrTCdMOV+YT03sbTyZuPffy2dkzlBqLo1nB3uRotfEbiQtVPdn/L8BCkDcX8A7nJP3HMnV2iCzD8vPkw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EUZTRuRZpTJvKvnJre3jqWAN7n3G4U+8+LDtO2FmSXo=;
 b=bSiKOKgOoYaHwLN0CPcfUYQC3Jn+tafi7Q6wRothYCSdYkaYx8a2t38WmyMycJun45k2nTtj559a6iavZFaGkV9IVerhP1U09KePWLiZJ6TaeprSyk4RWNMGhyG0BEprKqBmEM5xF2MPPwe0o8MVyaA/cfz2sB+aSDUvDNoR5Fs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
Date: Mon, 20 May 2024 22:52:25 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>,
	Luca Fancellu <Luca.Fancellu@arm.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00003F66:EE_|CY8PR12MB7265:EE_
X-MS-Office365-Filtering-Correlation-Id: d5f25c36-c727-45b3-0d66-08dc78dc798d
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eFE0bDlmUUo0ajlzSFd5SWd4VUxTeE0vWkFMd0pLWW8veVlCazQzYkVmUTVG?=
 =?utf-8?B?K2x2dWNROTlZZE8zak9DM1daSUtSYkpZOHlGUEorWkJOZzhhT3hBWDRrQ3lz?=
 =?utf-8?B?ZGhxcExEQlNmRklIakdQc3d2V090T1dsNWd6YklaR1ZPMXlrSUloSkF1MFg5?=
 =?utf-8?B?aVZTWTNZN3A2a1lpV1JsTWt2bDFPWHpFM0svZWZBbVlvN250c3d1ekdLUEpZ?=
 =?utf-8?B?Ynh0U2NxWEZtRWhPYkc2RjRQTnFseVlhQmtIMUdWNEYrVUZaYkFqbEk1U1BQ?=
 =?utf-8?B?UjZrRlg4SnJjQnVYR2hIL3JZS0t1NktXcDhRV3JsaUtTL3VURWdXYUgwbG5W?=
 =?utf-8?B?TlpnU1ZzajVXN3hqMUtLNXRmRFVOdHNrMGgvU01UN0Q4R05SUmc3Q3g4ZTcv?=
 =?utf-8?B?cXhLMzQraEllZm1ocWxOQWtrTTB3RjgvQWowVEpQakFzMGYrWkVXbkJlNE45?=
 =?utf-8?B?K3orc2o2LzVKQUhFUko5Rzc5amh5ckhQbUNzOWszRzJYRjVCdXdrK1JrQXpa?=
 =?utf-8?B?T25UZjFyWWVWRnhDOU9BZmJ6OUtaS0FEOCtLT0NUbXh6TFprSkFsSTRWWTVS?=
 =?utf-8?B?a0FQcTVCK29TRmt1eUdyL1A2RWVBa1FMYW5oR0N2aUxXMVgyb3NFYkY5dkd2?=
 =?utf-8?B?SnVVcGNDR0lVRjZ5Yk9SMUJhQ2FMSUxaUWgrYVU1UDcwZGhRb1JodWxhSTlS?=
 =?utf-8?B?OUpiUmlpTmhmTDR2dzBsY050Yyt4OGdqUm9WSEVHbzIydWttS3RUYXNQM1dB?=
 =?utf-8?B?OHAxNUdvQ2tyUU1BbjlXY0ZCNzBUNXpHc3ROMUZSbUN5Y1h6OEJudWc2VGJo?=
 =?utf-8?B?OHo4Tmg4b3U3L3E2bGlZVnAzdEpzMXRSTHpnZ0NGcnhjb1EwbW1hR3BKbGc3?=
 =?utf-8?B?OTczWjFWSjZiM3BNYVcyRElzYzhYbUY5YVBtZXNoeEJTOWw1Mk9lSkxvNllt?=
 =?utf-8?B?dDZqZ0hIbnFJY3VKdlJKclNSVG9ZZStLRS9jZ0tMSVd5VUt2a0lGZncxcnU4?=
 =?utf-8?B?OHVuZUxqOFRxb01wNFlSV0R6eE0xUVhETVZ2S2tCbG1meGxDQTYvTWRJZm1F?=
 =?utf-8?B?QXd0NnNXdFVJc0NMTjJOWEc4K2VqMlhxMmMzNnJTOW1FNUd0RUdrUXpiMWNF?=
 =?utf-8?B?TXBveXgwQW90cFFVSW1xeFMwUktoSzJuNCtOR3Job3h0NTdhNWF3VkkrRGRF?=
 =?utf-8?B?MldZSVB4NEhNb1JiaHpPYU1IcmJKTkkzOEZYZ1A2Yzd6dmZ0TjE0QXRqQzVD?=
 =?utf-8?B?V3JndHVZeWRDV0kwbTlub0hzakQraTdCR3BBcklDbjVlZHFVbjFRT3d2QW16?=
 =?utf-8?B?Y25PTHlLMHRCK0MwQVN1SnZlQnNzTTA1QTkrcGJxQ01MZVZ3QmpHUmZEbXRH?=
 =?utf-8?B?amZaeEd3bmQvbzgzcktDZmdoTGgybnV4WkRCY2xvdlZKc2d0cmFHOVE5OEtJ?=
 =?utf-8?B?bzR1M3pLamtkdHRiNTBZSm1pcjNlT2I2UDJBN0I0U3NXVVZ3ZXRCRml5Nlpp?=
 =?utf-8?B?WG5ZRkRiSmExTEE5SUFTTHRnZmRvRWk1L25QZVdqMldPUlMrMU9jY29zclJn?=
 =?utf-8?B?ejU5cmRmSTl5RUY3VEppankrSlRuci94Zkw1bWk2U3VCdUwyKzJWRitMSzE4?=
 =?utf-8?B?UE5wREg1clIwUHhSTlRKbmVtZzBmUWtSVUNOalhzQU1jRGZGTnpkd2pHRHBI?=
 =?utf-8?B?L3kzaWpnWjBZL0xMdjBoVy9YUnc0L3owKzhFSjZhVmN1bXV4bXJwbGxzUEVq?=
 =?utf-8?B?N2pJVVZ1SFdEaHlTVVVEaFRBdVBTOEJsV2treVVXSTJkUERkcHBVdkt3T01s?=
 =?utf-8?B?aXlrNDJreUFBaVk0ZldyYW9uODJQeFEwQ0VsUkpWeG9GMUhPYXk0ck1aNVVL?=
 =?utf-8?Q?C3himXi2Kj2bi?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 14:52:30.6411
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d5f25c36-c727-45b3-0d66-08dc78dc798d
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00003F66.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7265

Hi Michal, Luca,

On 5/20/2024 7:24 PM, Michal Orzel wrote:
> Hi Henry,
>
> +CC: Luca
>
> On 17/05/2024 05:21, Henry Wang wrote:
>> To make things easier, add restriction that static shared memory
>> should also be direct-mapped for direct-mapped domains. Check the
>> host physical address to be matched with guest physical address when
>> parsing the device tree. Document this restriction in the doc.
> I'm ok with this restriction.
>
> @Luca, do you have any use case preventing us from making this restriction?
>
> This patch clashes with Luca series so depending on which goes first,

I agree that there will be some conflicts between the two series. To 
avoid back and forth, if Luca's series goes in first, would it be ok for 
you if I place the same check from this patch in 
handle_shared_mem_bank() like below?

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 9c3a83042d..2d23fa4917 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -219,6 +219,13 @@ static int __init handle_shared_mem_bank(struct 
domain *d, paddr_t gbase,
      pbase = shm_bank->start;
      psize = shm_bank->size;

+    if ( is_domain_direct_mapped(d) && (pbase != gbase) )
+    {
+        printk("%pd: physical address 0x%"PRIpaddr" and guest address 
0x%"PRIpaddr" are not direct-mapped.\n",
+               d, pbase, gbase);
+        return -EINVAL;
+    }
+
      printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys 
0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
             d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);

> Acked-by: Michal Orzel <michal.orzel@amd.com>

Thanks. I will take the tag if you are ok with above diff (for the case 
if this series goes in later than Luca's).

>>           }
>> +        if ( is_domain_direct_mapped(d) && (pbase != gbase) )
>> +        {
>> +            printk("%pd: physical address 0x%"PRIpaddr" and guest address 0x%"PRIpaddr" are not 1:1 direct-mapped.\n",
> NIT: 1:1 and direct-mapped means the same so no need to place them next to each other

Ok. I will drop the "1:1" in the next version. Thanks.

Kind regards,
Henry

> ~Michal



From xen-devel-bounces@lists.xenproject.org Mon May 20 15:00:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:00:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726192.1130445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94UW-0007ri-BI; Mon, 20 May 2024 14:59:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726192.1130445; Mon, 20 May 2024 14:59:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94UW-0007rb-8Z; Mon, 20 May 2024 14:59:56 +0000
Received: by outflank-mailman (input) for mailman id 726192;
 Mon, 20 May 2024 14:59:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s94UV-0007rR-G4; Mon, 20 May 2024 14:59:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s94UV-0004OP-Ai; Mon, 20 May 2024 14:59:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s94UU-0000fd-WE; Mon, 20 May 2024 14:59:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s94UU-0005JB-Vh; Mon, 20 May 2024 14:59:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=bptfkd+CZuWLLdIaDHbHs65+aU5RefCGIW1Amqv0oAE=; b=pL1cIbxBj7QYdwodHlaMeXtc2X
	IZ+uhLGsGYwcJ4WgzQP2ovtGBScrXzYMvGO3AMA6geIK/oSB+vBiAtt3tRPICDVdB0TIBzwDjbnho
	CTmfLF3NBC5qQKGGhl2q0+dvFMrL6J28S6dSJzBaEAD5eLzAZKOWegFNMaVxHFTLq8vU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186044-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186044: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:debian-install:fail:allowable
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c
X-Osstest-Versions-That:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 May 2024 14:59:54 +0000

flight 186044 linux-linus real [real]
flight 186045 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186044/
http://logs.test-lab.xenproject.org/osstest/logs/186045/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1   8 xen-boot                 fail REGR. vs. 186036
 test-armhf-armhf-libvirt      8 xen-boot                 fail REGR. vs. 186036

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   8 xen-boot            fail pass in 186045-retest
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 186045-retest
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 186045-retest
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail pass in 186045-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     12 debian-install           fail REGR. vs. 186036

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 186045 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 186045 never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 186045 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 186045 never pass
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186045 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186045 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 186045 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 186045 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186036
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c
baseline version:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f

Last test of basis   186036  2024-05-18 21:13:52 Z    1 days
Failing since        186038  2024-05-19 03:51:42 Z    1 days    4 attempts
Testing same since   186044  2024-05-20 05:14:37 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Chen, Tim C" <tim.c.chen@intel.com>
  "Huang, Ying" <ying.huang@intel.com>
  "King, Colin" <colin.king@intel.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Alex Rusuf <yorha.op@gmail.com>
  Alex Shi (tencent) <alexs@kernel.org>
  Alex Shi <alexs@kernel.org>
  Alexander Potapenko <glider@google.com>
  Alexandre Ghiti <alexghiti@rivosinc.com
  Alexei Starovoitov <ast@kernel.org>
  Alexey Dobriyan <adobriyan@gmail.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Axel Rasmussen <axelrasmussen@google.com>
  Baokun Li <libaokun1@huawei.com>
  Baolin Wang <baolin.wang@linux.alibaba.com>
  Baoquan He <bhe@redhat.com>
  Barry Song <v-songbaohua@oppo.com>
  Bart Van Assche <bvanassche@acm.org>
  Borislav Petkov (AMD) <bp@alien8.de>
  Borislav Petkov <bp@suse.de>
  Breno Leitao <leitao@debian.org>
  Brian Foster <bfoster@redhat.com>
  Brian Geffon <bgeffon@google.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chang S. Bae <chang.seok.bae@intel.com>
  Cheng Yu <serein.chengyu@huawei.com>
  Chris Li <chrisl@kernel.org>
  Christian Loehle <christian.loehle@arm.com>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Christophe Leroy <christophe.leroy@csgroup.eu>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Hill <daniel@gluo.nz>
  Daniel Thompson <daniel.thompson@linaro.org>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Hildenbrand <david@redhat.com>
  Dawei Li <daweilics@gmail.com>
  Dennis Zhou <dennis@kernel.org>
  Dev Jain <dev.jain@arm.com>
  Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Disha Goel <disgoel@linux.ibm.com>
  Donet Tom <donettom@linux.ibm.com>
  Douglas Anderson <dianders@chromium.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Edward Liaw <edliaw@google.com>
  Eric Sandeen <sandeen@redhat.com>
  Florian Rommel <mail@florommel.de>
  Frank van der Linden <fvdl@google.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  Hao Ge <gehao@kylinos.cn>
  Hao Xiang <hao.xiang@bytedance.com>
  Hariom Panthi <hariom1.p@samsung.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@gmx.de> [parisc]
  Heming Zhao <heming.zhao@suse.com>
  Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
  Hongbo Li <lihongbo22@huawei.com>
  Honggyu Kim <honggyu.kim@sk.com>
  Huang Shijie <shijie@os.amperecomputing.com>
  Huang Ying <ying.huang@intel.com>
  Ingo Molnar <mingo@kernel.org>
  James Houghton <jthoughton@google.com>
  Jan Kara <jack@suse.cz>
  Jan Kara <jack@suse.cz>               [jbd2]
  Jeff Layton <jlayton@kernel.org>
  Jinjiang Tu <tujinjiang@huawei.com>
  Joe Perches <joe@perches.com>
  Joel Granados <j.granados@samsung.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Hubbard <jhubbard@nvidia.com>
  Joseph Qi <joseph.qi@linux.alibaba.com>
  Justin Stitt <justinstitt@google.com>
  K Prateek Nayak <kprateek.nayak@amd.com>
  Kairui Song <kasong@tencent.com>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Klara Modin <klarasmodin@gmail.com>
  Konrad Dybcio <konrad.dybcio@linaro.org> # QC SM8550 QRD
  Kuan-Wei Chiu <visitorckw@gmail.com>
  Lance Yang <ioworker0@gmail.com>
  Len Brown <len.brown@intel.com>
  Li kunyu <kunyu@nfschina.com>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Liu Shixin <liushixin2@huawei.com>
  Long Li <leo.lilong@huawei.com>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Lucas Stach <l.stach@pengutronix.de>
  Luis Henriques <lhenriques@suse.de>
  Lukas Bulwahn <lbulwahn@redhat.com>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Matthew Wilcox <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Mel Gorman <mgorman@techsingularity.net>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Ellerman <mpe@ellerman.id.au> (powerpc)
  Michael Forney <mforney@mforney.org>
  Michal Hocko <mhocko@suse.com>
  Miguel Ojeda <ojeda@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Nhat Pham <nphamcs@gmail.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
  Oscar Salvador <osalvador@suse.de>
  Palmer Dabbelt <palmer@rivosinc.com>  [RISC-V]
  Pankaj Raghav <p.raghav@samsung.com>
  Patryk Wlazlyn <patryk.wlazlyn@linux.intel.com>
  Peter Xu <peterx@redhat.com>
  Petr Vorel <pvorel@suse.cz>
  Phillip Lougher <phillip@squashfs.org.uk>
  Randy Dunlap <rdunlap@infradead.org>
  Rasmus Villemoes <linux@rasmusvillemoes.dk>
  Ricardo B. Marliere <ricardo@marliere.net>
  Rick Edgecombe <rick.p.edgecombe@intel.com>
  Rik van Riel <riel@surriel.com>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Roman Gushchin <roman.gushchin@linux.dev>
  rulinhuang <rulin.huang@intel.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Saurav Shah <sauravshah.31@gmail.com>
  Sean Christopherson <seanjc@google.com>       [KVM]
  SeongJae Park <sj@kernel.org>
  Sergey Senozhatsky <senozhatsky@chromium.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Song Liu <song@kernel.org>
  Stephen Smalley <stephen.smalley.work@gmail.com>
  Steve French <stfrench@microsoft.com>
  Su Yue <glass.su@suse.com>
  Sudeep Holla <sudeep.holla@arm.com>
  Suren Baghdasaryan <surenb@google.com>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Bertschinger <tahbertschinger@gmail.com>
  Thorsten Blum <thorsten.blum@toblux.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Usama Arif <usamaarif642@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vincent Guittot <vincent.guittot@linaro.org>
  Vishal Moola (Oracle) <vishal.moola@gmail.com>
  Vishal Verma <vishal.l.verma@intel.com>
  Vitalii Bursov <vitaly@bursov.com>
  Vladimir Benes <vbenes@redhat.com>
  Vlastimil Babka <vbabka@suse.cz>
  Waiman Long <longman@redhat.com>
  Wei Yang <richard.weiyang@gmail.com>
  Xining Xu <mac.xxn@outlook.com>
  Xiu Jianfeng <xiujianfeng@huawei.com>
  Yajun Deng <yajun.deng@linux.dev>
  Yang Li <yang.lee@linux.alibaba.com>
  Ye Bin <yebin10@huawei.com>
  York Jasper Niebuhr <yjnworkstation@gmail.com>
  Yosry Ahmed <yosryahmed@google.com>
  Youling Tang <tangyouling@kylinos.cn>
  Yu Zhao <yuzhao@google.com>
  Zhang Qiao <zhangqiao22@huawei.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhang Yi <yi.zhang@huawei.com>
  ZhangPeng <zhangpeng362@huawei.com>
  Zi Yan <ziy@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 20845 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 20 15:02:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:02:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726198.1130455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94Wr-0001AO-PN; Mon, 20 May 2024 15:02:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726198.1130455; Mon, 20 May 2024 15:02:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94Wr-0001AH-LR; Mon, 20 May 2024 15:02:21 +0000
Received: by outflank-mailman (input) for mailman id 726198;
 Mon, 20 May 2024 15:02:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s94Wp-0001A7-PI
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 15:02:19 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f1ea07f8-16b9-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 17:02:16 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-792b8c9046bso280073485a.3
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 08:02:16 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf3106cfsm1188444485a.108.2024.05.20.08.02.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 08:02:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1ea07f8-16b9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716217335; x=1716822135; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=rPjE5Qnnk/P6B3pQj3ASZ+iZi+YzWQO3+vJ709b2zt4=;
        b=WbrBhKFNKyEKmLgZjMkmqP1ju4jk29Xdh4R0ciIPpgcqLG3seHcQ7A4+m5+kdd+Iea
         M+BDzF2BaXwq4pUkAr+cwloN7IBqcA5aciddw7cClSIderrsgLrkeu7zbxRDucWGxUkw
         fY09aA8c0N4GMT2Yg0E8bXEl5so/1sBG8eY3g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716217335; x=1716822135;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rPjE5Qnnk/P6B3pQj3ASZ+iZi+YzWQO3+vJ709b2zt4=;
        b=ZM0vheHv2lOY+QnklcjzIfON/pHL5cmfTyF/vSVX2wE8vzWza/jQCOkahMflUhHFw7
         23s9+1V5ga0cvE7hdxMItG8hjNKXkp1sIX4rTJ90MyhptQfBMO1kVUd3r7k8+1n0OuFp
         Vmcj2XGG7x2hhL9RxkOi2y3xh3Uu/cgqLSY2OUrj8ZoPrEE7JTxjcyq2eAUoZaaeGliM
         MebIfoX2OIe+YDQM4sDVJ+7ikimqNnv5m/VyrykQwRNSu5oBS29lI3FKlmdn0QBuKGVu
         CHoL7JEJOSgpxCJlDSDTrDMyTst1I9lzjpLnUud3nSTel7KY71PxkxPM33SrwHOpp4N4
         FVzQ==
X-Gm-Message-State: AOJu0YzJx1g4/ey2FzTzant8D8wJhOd6XzoMg2Pq7aebJYI8RBnenvZl
	tOyQG86OArsOQVR7meUNNPHBnFlBuvg8xwqqj38lqibD5NJKRIDKq4VNxgJOcVU=
X-Google-Smtp-Source: AGHT+IEjty7b6dbecHwPLHZurTqtJ1GT77VDpv90pkW89EfqrlwikV4DJ04MxJ+sDD9qTUcoqgrPxA==
X-Received: by 2002:ae9:e014:0:b0:792:8c06:cd8 with SMTP id af79cd13be357-792c7610199mr3075354185a.71.1716217334352;
        Mon, 20 May 2024 08:02:14 -0700 (PDT)
Date: Mon, 20 May 2024 17:02:11 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 2/2] tools/xg: Clean up xend-style overrides for CPU
 policies
Message-ID: <Zktl8zRt1kue5vA6@macbook>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
 <d397ec0de4138e32feeb910f3401a6568a75035e.1715954111.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <d397ec0de4138e32feeb910f3401a6568a75035e.1715954111.git.alejandro.vallejo@cloud.com>

On Fri, May 17, 2024 at 05:08:35PM +0100, Alejandro Vallejo wrote:
> Factor out policy getters/setters from both (CPUID and MSR) policy override
> functions. Additionally, use host policy rather than featureset when
> preparing the cur policy, saving one hypercall and several lines of
> boilerplate.
> 
> No functional change intended.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * Cosmetic change to comment (// into /**/).
>   * Added missing null pointer check to MSR override function.
> ---
>  tools/libs/guest/xg_cpuid_x86.c | 445 ++++++++++----------------------
>  1 file changed, 130 insertions(+), 315 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4f4b86b59470..74bca0e65b69 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -36,6 +36,34 @@ enum {
>  #define bitmaskof(idx)      (1u << ((idx) & 31))
>  #define featureword_of(idx) ((idx) >> 5)
>  
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
> +{
> +    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> +    int rc;
> +
> +    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
> +    if ( rc )
> +    {
> +        if ( err_leaf != -1 )
> +            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> +                  err_leaf, err_subleaf, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> +                                  policy->nr_msrs, &err_msr);
> +    if ( rc )
> +    {
> +        if ( err_msr != -1 )
> +            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
> +                  err_msr, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
>  int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
>  {
>      struct xen_sysctl sysctl = {};
> @@ -260,102 +288,34 @@ static int compare_leaves(const void *l, const void *r)
>      return 0;
>  }
>  
> -static xen_cpuid_leaf_t *find_leaf(
> -    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
> -    const struct xc_xend_cpuid *xend)
> +static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
> +                                   const struct xc_xend_cpuid *xend)
>  {
>      const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
>  
> -    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
> +    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),
> +                   sizeof(*p->leaves), compare_leaves);
>  }
>  
> -static int xc_cpuid_xend_policy(
> -    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
> +static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
> +                                const struct xc_xend_cpuid *xend,
> +                                xc_cpu_policy_t *host,
> +                                xc_cpu_policy_t *def,
> +                                xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    rc = -ENOMEM;
> -    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        goto fail;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_msrs = 0;
> -    nr_cur = nr_leaves;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    /* Get the domain type's default policy. */
> -    nr_msrs = 0;
> -    nr_def = nr_leaves;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_def, def, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    /* Get the host policy. */
> -    nr_msrs = 0;
> -    nr_host = nr_leaves;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_host, host, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto fail;
> -    }
> +    if ( !xend )
> +        return 0;
>  
> -    rc = -EINVAL;
>      for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
>      {
> -        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
> -        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
> -        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
> +        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
> +        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
> +        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
>  
>          if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
>          {
>              ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
> -            goto fail;
> +            return -EINVAL;
>          }
>  
>          for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
> @@ -384,7 +344,7 @@ static int xc_cpuid_xend_policy(
>                  {
>                      ERROR("Bad character '%c' in policy[%d] string '%s'",
>                            xend->policy[i][j], i, xend->policy[i]);
> -                    goto fail;
> +                    return -EINVAL;
>                  }
>  
>                  clear_bit(31 - j, cur_reg);
> @@ -394,25 +354,7 @@ static int xc_cpuid_xend_policy(
>          }
>      }
>  
> -    /* Feed the transformed currrent policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    /* Success! */
> -
> - fail:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  static int compare_msr(const void *l, const void *r)
> @@ -426,107 +368,37 @@ static int compare_msr(const void *l, const void *r)
>      return lhs->idx < rhs->idx ? -1 : 1;
>  }
>  
> -static xen_msr_entry_t *find_msr(
> -    xen_msr_entry_t *msrs, unsigned int nr_msrs,
> -    uint32_t index)
> +static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
> +                                 uint32_t index)
>  {
>      const xen_msr_entry_t key = { .idx = index };
>  
> -    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
> +    return bsearch(&key, p->msrs, ARRAY_SIZE(p->msrs),
> +                   sizeof(*p->msrs), compare_msr);
>  }
>  
> -
>  static int xc_msr_policy(xc_interface *xch, domid_t domid,
> -                         const struct xc_msr *msr)
> +                         const struct xc_msr *msr,
> +                         xc_cpu_policy_t *host,
> +                         xc_cpu_policy_t *def,

host and def should likely be const?

> +                         xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        rc = -ENOMEM;
> -        goto out;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_leaves = 0;
> -    nr_cur = nr_msrs;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Get the domain type's default policy. */
> -    nr_leaves = 0;
> -    nr_def = nr_msrs;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, NULL, &nr_def, def);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Get the host policy. */
> -    nr_leaves = 0;
> -    nr_host = nr_msrs;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_leaves, NULL, &nr_host, host);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto out;
> -    }
> +    if ( !msrs )

Does this build?  Where is 'msrs' defined in this context?  The
function parameter is 'msr' AFAICT.

> +        return 0;

Should we also check for host, def, cur != NULL also?

>  
>      for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
>      {
> -        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
> -        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
> -        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
> -        unsigned int i;
> +        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
> +        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
> +        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
>  
>          if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
>          {
>              ERROR("Missing MSR %#x", msr->index);
> -            rc = -ENOENT;
> -            goto out;
> +            return -ENOENT;
>          }
>  
> -        for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
> +        for ( size_t i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
>          {
>              bool val;
>  
> @@ -542,8 +414,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>              {
>                  ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
>                        msr->index, msr->policy[i], msr->policy);
> -                rc = -EINVAL;
> -                goto out;
> +                return -EINVAL;
>              }
>  
>              if ( val )
> @@ -553,25 +424,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>          }
>      }
>  
> -    /* Feed the transformed policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Success! */
> -
> - out:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
> @@ -583,14 +436,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      int rc;
>      bool hvm;
>      xc_domaininfo_t di;
> -    struct xc_cpu_policy *p = xc_cpu_policy_init();
> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
> -    uint32_t len = ARRAY_SIZE(host_featureset);
>  
> -    if ( !p )
> -        return -ENOMEM;
> +    struct xc_cpu_policy *host = xc_cpu_policy_init();
> +    struct xc_cpu_policy *def = xc_cpu_policy_init();

I would be helpful to have some kind of mechanism to allocate + init a
policy at the same time, so that the resulting object could be made
const here.  (Not that you need to do it in this patch).

> +    struct xc_cpu_policy *cur = xc_cpu_policy_init();
> +
> +    if ( !host || !def || !cur )
> +    {
> +        rc = -ENOMEM;
> +        goto out;
> +    }
>  
>      if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
>      {
> @@ -600,21 +455,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      }
>      hvm = di.flags & XEN_DOMINF_hvm_guest;
>  
> -    /* Get the host policy. */
> -    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
> -                               &len, host_featureset);
> -    /* Tolerate "buffer too small", as we've got the bits we need. */
> -    if ( rc && errno != ENOBUFS )
> +    /* Get the raw host policy */
> +    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
> +    if ( rc )
>      {
> -        PERROR("Failed to obtain host featureset");
> +        PERROR("Failed to obtain host policy");
>          rc = -errno;
>          goto out;
>      }
>  
>      /* Get the domain's default policy. */
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, p->leaves, &nr_msrs, NULL);
> +    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> +                                           : XEN_SYSCTL_cpu_policy_pv_default,
> +                                  def);
>      if ( rc )
>      {
>          PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
> @@ -622,14 +475,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>          goto out;
>      }
>  
> -    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
> -                                    &err_leaf, &err_subleaf);
> -    if ( rc )
> -    {
> -        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> -              err_leaf, err_subleaf, -rc, strerror(-rc));
> -        goto out;
> -    }
> +    /* Copy the deserialised default policy to modify it */
> +    memcpy(cur, def, sizeof(*cur));
>  
>      if ( restore )
>      {
> @@ -647,18 +494,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * - Re-enable features which have become (possibly) off by default.
>           */
>  
> -        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
> -        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
> -        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
> +        cur->policy.basic.rdrand = host->policy.basic.rdrand;
> +        cur->policy.feat.hle = host->policy.feat.hle;
> +        cur->policy.feat.rtm = host->policy.feat.rtm;
>  
>          if ( hvm )
>          {
> -            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
> +            cur->policy.feat.mpx = host->policy.feat.mpx;
>          }
>  
> -        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
> -        p->policy.feat.max_subleaf = 0;
> -        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
> +        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
> +        cur->policy.feat.max_subleaf = 0;
> +        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
>      }
>  
>      if ( featureset )
> @@ -666,7 +513,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>          uint32_t disabled_features[FEATURESET_NR_ENTRIES],
>              feat[FEATURESET_NR_ENTRIES] = {};
>          static const uint32_t deep_features[] = INIT_DEEP_FEATURES;
> -        unsigned int i, b;
> +        unsigned int b;
>  
>          /*
>           * The user supplied featureset may be shorter or longer than
> @@ -677,14 +524,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>  
>          /* Check for truncated set bits. */
>          rc = -EOPNOTSUPP;
> -        for ( i = user_len; i < nr_features; ++i )
> +        for ( size_t i = user_len; i < nr_features; ++i )
>              if ( featureset[i] != 0 )
>                  goto out;
>  
>          memcpy(feat, featureset, sizeof(*featureset) * user_len);
>  
>          /* Disable deep dependencies of disabled features. */
> -        for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
> +        for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
>              disabled_features[i] = ~feat[i] & deep_features[i];
>  
>          for ( b = 0; b < sizeof(disabled_features) * CHAR_BIT; ++b )
> @@ -695,24 +542,24 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>                   !(dfs = x86_cpu_policy_lookup_deep_deps(b)) )
>                  continue;
>  
> -            for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
> +            for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )

All this loop index type changes could be done as a separate patch,
you are not even touching the surrounding lines.  It adds a lot of
churn to this patch for no reason IMO.

>              {
>                  feat[i] &= ~dfs[i];
>                  disabled_features[i] &= ~dfs[i];
>              }
>          }
>  
> -        x86_cpu_featureset_to_policy(feat, &p->policy);
> +        x86_cpu_featureset_to_policy(feat, &cur->policy);
>      }
>      else
>      {
> -        p->policy.extd.itsc = itsc;
> +        cur->policy.extd.itsc = itsc;
>  
>          if ( hvm )
>          {
> -            p->policy.basic.pae = pae;
> -            p->policy.basic.vmx = nested_virt;
> -            p->policy.extd.svm = nested_virt;
> +            cur->policy.basic.pae = pae;
> +            cur->policy.basic.vmx = nested_virt;
> +            cur->policy.extd.svm = nested_virt;
>          }
>      }
>  
> @@ -722,8 +569,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * On hardware without CPUID Faulting, PV guests see real topology.
>           * As a consequence, they also need to see the host htt/cmp fields.
>           */
> -        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
> -        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
> +        cur->policy.basic.htt       = host->policy.basic.htt;
> +        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
>      }
>      else
>      {
> @@ -731,28 +578,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * Topology for HVM guests is entirely controlled by Xen.  For now, we
>           * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
>           */
> -        p->policy.basic.htt = true;
> -        p->policy.extd.cmp_legacy = false;
> +        cur->policy.basic.htt = true;
> +        cur->policy.extd.cmp_legacy = false;
>  
>          /*
>           * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
>           * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
>           * overflow.
>           */
> -        if ( !p->policy.basic.lppp )
> -            p->policy.basic.lppp = 2;
> -        else if ( !(p->policy.basic.lppp & 0x80) )
> -            p->policy.basic.lppp *= 2;
> +        if ( !cur->policy.basic.lppp )
> +            cur->policy.basic.lppp = 2;
> +        else if ( !(cur->policy.basic.lppp & 0x80) )
> +            cur->policy.basic.lppp *= 2;
>  
> -        switch ( p->policy.x86_vendor )
> +        switch ( cur->policy.x86_vendor )
>          {
>          case X86_VENDOR_INTEL:
> -            for ( i = 0; (p->policy.cache.subleaf[i].type &&
> -                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
> +            for ( size_t i = 0; (cur->policy.cache.subleaf[i].type &&
> +                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )
>              {
> -                p->policy.cache.subleaf[i].cores_per_package =
> -                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
> -                p->policy.cache.subleaf[i].threads_per_cache = 0;
> +                cur->policy.cache.subleaf[i].cores_per_package =
> +                    (cur->policy.cache.subleaf[i].cores_per_package << 1) | 1;
> +                cur->policy.cache.subleaf[i].threads_per_cache = 0;
>              }
>              break;
>  
> @@ -772,49 +619,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>               * apic_id_size values greater than 7.  Limit the value to
>               * 7 for now.
>               */
> -            if ( p->policy.extd.nc < 0x7f )
> +            if ( cur->policy.extd.nc < 0x7f )
>              {
> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
> -                    p->policy.extd.apic_id_size++;
> +                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )

I would split the line while there, it's overly long.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 15:07:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726207.1130465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94bm-0001qk-Ei; Mon, 20 May 2024 15:07:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726207.1130465; Mon, 20 May 2024 15:07:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94bm-0001qd-Aq; Mon, 20 May 2024 15:07:26 +0000
Received: by outflank-mailman (input) for mailman id 726207;
 Mon, 20 May 2024 15:07:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s94bl-0001qX-5O
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 15:07:25 +0000
Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com
 [2607:f8b0:4864:20::f2d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8a17065-16ba-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 17:07:23 +0200 (CEST)
Received: by mail-qv1-xf2d.google.com with SMTP id
 6a1803df08f44-69cb4a046dfso13415436d6.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 08:07:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1ff4c1sm112571356d6.142.2024.05.20.08.07.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 08:07:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8a17065-16ba-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716217642; x=1716822442; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=hAvSQH9QO0M3arfgY7SM0FYdcrJGZGLBtAnh4EZNkbw=;
        b=fra2TzHnf+tx8/f4XycKTc50SnjnoAxkQkQtAGvcJQSvdnT04nqZE9nR9PApXP4sNa
         wu2Gq8sKgjLUifeQOvK+4SWfvuv9X1DA8gCE1ITJr8rMDF4JTeMZtu/3v1IPowGqeOoT
         dXbDtgCnR8TkMJqYC5n5/D9zWVXPv2OH4W26g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716217642; x=1716822442;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hAvSQH9QO0M3arfgY7SM0FYdcrJGZGLBtAnh4EZNkbw=;
        b=PZekkW+PlmjXOqV3mzefH8jdTvSwPGoU9taEqTw9/hlMMaE2rPRFMAVouKrfL6s5cj
         Dg1RTdMyFfcUM1LJnD0Il1y0DsLbiuoNvTRDK0VIdl5YKcubhrAk5HjwndGaAUoiPepL
         UaKBVjU1OPFUKWKGwlM/x7LQzhsqoSTNK8zFPHZ+oMAYRRqJ9/VUYyymngxWf1LjMLPy
         /D56dISwqa3RlPEVDa85XHyKfsLDNLhp7Y+PSMXrZqdrxZ9OSolVBEutR82cMizDMOus
         +TrsBvVK2pGr5u7KoQL9LI9yu+Afmu69UxpBC+6Ip5JZpraJJ3i+ieZ4T9ne9NdSydNR
         0/Zw==
X-Gm-Message-State: AOJu0Ywmy9ShaEY0IZZlNYW44Wyzwycmyu5zkQXSHZL6Tl6cHK92nCJS
	LytMg0b/he+qG1QF55mOz0oPiStOs27eC+56wmSze19e3sn5p2D/iwIiJzC7KOo=
X-Google-Smtp-Source: AGHT+IEM69dVQgUOuHhtNLSnf09CH0jdhMaPrPVb+OBYIjt74e8PkC+iwKf9065OttWuw0lSkWLGpA==
X-Received: by 2002:a05:6214:3a85:b0:6a0:936e:4f93 with SMTP id 6a1803df08f44-6a168243ff1mr362365606d6.60.1716217641435;
        Mon, 20 May 2024 08:07:21 -0700 (PDT)
Date: Mon, 20 May 2024 17:07:19 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH v3.5 3/4] tools/xen-cpuid: Use automatically generated
 feature names
Message-ID: <ZktnJ7XNxhdLOxdk@macbook>
References: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <20240520143359.3376849-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240520143359.3376849-1-andrew.cooper3@citrix.com>

On Mon, May 20, 2024 at 03:33:59PM +0100, Andrew Cooper wrote:
> From: Roger Pau Monné <roger.pau@citrix.com>
> 
> Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
> data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
> are needed.
> 
> In xen-cpuid.c, sanity check at build time that leaf_info[] and
> feature_names[] are of sensible length.
> 
> As dump_leaf() rendered missing names as numbers, always dump leaves even if
> we don't have the leaf name.  This conversion was argumably missed in commit
> 59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Just one question below.

> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> 
> Differences in names are:
> 
>  sysenter    -> sep
>  tm          -> tm1
>  ds-cpl      -> dscpl
>  est         -> eist
>  sse41       -> sse4-1
>  sse42       -> sse4-2
>  movebe      -> movbe
>  tsc-dl      -> tsc-deadline
>  rdrnd       -> rdrand
>  hyper       -> hypervisor
>  mmx+        -> mmext
>  fxsr+       -> ffxsr
>  pg1g        -> page1gb
>  3dnow+      -> 3dnowext
>  cmp         -> cmp-legacy
>  cr8d        -> cr8-legacy
>  lzcnt       -> abm
>  msse        -> misalignsse
>  3dnowpf     -> 3dnowprefetch
>  nodeid      -> nodeid-msr
>  dbx         -> dbext
>  tsc-adj     -> tsc-adjust
>  fdp-exn     -> fdp-excp-only
>  deffp       -> no-fpu-sel
>  <24>        -> bld
>  ppin        -> amd-ppin
>  lfence+     -> lfence-dispatch
>  ppin        -> intel-ppin
>  energy-ctrl -> energy-filtering
> 
> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
> one.  Several of the + names would be nice to keep as were, but doing so isn't
> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
> but we intentionally don't list them, so I'm not worried.
> 
> Thoughts?
> 
> v3:
>  * Rework somewhat.
>  * Insert aliases of common_1d.
> 
> v4:
>  * Pad at the gen stage.  I don't like this, but I'm clearly outvoted on the matter.
> ---
>  tools/misc/xen-cpuid.c | 16 ++++++++--------
>  xen/tools/gen-cpuid.py | 29 +++++++++++++++++++++++++++++
>  2 files changed, 37 insertions(+), 8 deletions(-)
> 
> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> index 6ee835b22949..51009683da1b 100644
> --- a/tools/misc/xen-cpuid.c
> +++ b/tools/misc/xen-cpuid.c
> @@ -11,6 +11,7 @@
>  #include <xenguest.h>
>  
>  #include <xen-tools/common-macros.h>
> +#include <xen/lib/x86/cpuid-autogen.h>
>  
>  static uint32_t nr_features;
>  
> @@ -291,6 +292,8 @@ static const struct {
>  
>  #define COL_ALIGN "24"
>  
> +static const char *const feature_names[] = INIT_FEATURE_VAL_TO_NAME;
> +
>  static const char *const fs_names[] = {
>      [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
>      [XEN_SYSCTL_cpu_featureset_host]    = "Host",
> @@ -304,12 +307,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
>  {
>      unsigned i;
>  
> -    if ( !strs )
> -    {
> -        printf(" ???");
> -        return;
> -    }
> -
>      for ( i = 0; i < 32; ++i )
>          if ( leaf & (1u << i) )
>          {
> @@ -327,6 +324,10 @@ static void decode_featureset(const uint32_t *features,
>  {
>      unsigned int i;
>  
> +    /* If this trips, you probably need to extend leaf_info[] above. */
> +    BUILD_BUG_ON(ARRAY_SIZE(leaf_info) != FEATURESET_NR_ENTRIES);
> +    BUILD_BUG_ON(ARRAY_SIZE(feature_names) != FEATURESET_NR_ENTRIES * 32);
> +
>      printf("%-"COL_ALIGN"s        ", name);
>      for ( i = 0; i < length; ++i )
>          printf("%08x%c", features[i],
> @@ -338,8 +339,7 @@ static void decode_featureset(const uint32_t *features,
>      for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
>      {
>          printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
> -        if ( leaf_info[i].name )
> -            dump_leaf(features[i], leaf_info[i].strs);
> +        dump_leaf(features[i], &feature_names[i * 32]);
>          printf("\n");
>      }
>  }
> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
> index 79d7f5c8e1c9..601eec608983 100755
> --- a/xen/tools/gen-cpuid.py
> +++ b/xen/tools/gen-cpuid.py
> @@ -470,6 +470,35 @@ def write_results(state):
>      state.output.write(
>  """}
>  
> +""")
> +
> +    state.output.write(
> +"""
> +#define INIT_FEATURE_VAL_TO_NAME { \\
> +""")
> +
> +    for name, bit in sorted(state.values.items()):
> +        state.output.write(
> +            '    [%s] = "%s",\\\n' % (bit, name)
> +            )
> +
> +        # Add the other alias for 1d/e1d common bits.  64 is the difference
> +        # between 1d and e1d.
> +        if bit in state.common_1d:
> +            state.output.write(
> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
> +            )
> +
> +    # Pad to an exact multiple of FEATURESET_SIZE if necessary
> +    pad_feat = state.nr_entries * 32 - 1
> +    if not state.names.get(pad_feat):
> +        state.output.write(
> +            '    [%s] = NULL,\\\n' % (pad_feat, )

One likely stupid question, but since my understanding of Python is
very limited, why do you add the comma after pad_feat?  There's no
other parameter to print.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 15:20:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:20:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726214.1130475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94oe-0004u0-HC; Mon, 20 May 2024 15:20:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726214.1130475; Mon, 20 May 2024 15:20:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s94oe-0004tt-EP; Mon, 20 May 2024 15:20:44 +0000
Received: by outflank-mailman (input) for mailman id 726214;
 Mon, 20 May 2024 15:20:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gPfm=MX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s94od-0004tk-5C
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 15:20:43 +0000
Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com
 [2607:f8b0:4864:20::72c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8463c5f4-16bc-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 17:20:40 +0200 (CEST)
Received: by mail-qk1-x72c.google.com with SMTP id
 af79cd13be357-792b934de39so234066285a.3
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 08:20:40 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2fca1dsm1193313985a.92.2024.05.20.08.20.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 08:20:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8463c5f4-16bc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716218440; x=1716823240; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=j5yr6RF+mfbUu+tusE/e6Lc1Z8TDEpJfUPHupQq4vWU=;
        b=Y70l/Wcle724NXm76a+6GYPuNKYPR5R5l77BdyBJ30DkYUuBcHfTLLG21Q53EzqyE+
         uzrP1t55/RQRgRdm+xOEgABeK8B1u0lPYCcPoAZ0cdy7KKug5I7yCPrk+c1Ni3efwy4L
         +Ol8TfDcA1hM1nyruvnxKPDsC8Bh2W7vD8S+I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716218440; x=1716823240;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=j5yr6RF+mfbUu+tusE/e6Lc1Z8TDEpJfUPHupQq4vWU=;
        b=FvO1/+B2oylxoRURyOchI0gLGf176UUqDg8GVjOjWvRxCmqlkdA4dHKy9Z/yGY5VcE
         gFfMPrdcyLxz1E8athnTQP+xbE6UYemSBppdQNSULCFN7bhdoE6LZLA/OpiY/4Sxi8wT
         kDwv1GeIrf8cuHW+R9YMUGbizlya/foVYmcerNiO4z1bUl0XD59iPUMxDyCQSIfrAkMC
         Z5QalJIyp2QZSEu1KIv8Pd4BFa3KVPeWb6/xgh+68WOQACZf3Onnvf+UsABif60/kCiU
         FhSpcA+nSSs/eN9O+0yexASMmyhqiZTnFz3BZ7fmSvF84NNTZOrsxC59jOed3Ckxhy+U
         ZYzA==
X-Gm-Message-State: AOJu0Yw5JE21i2vx8iWLd/tFNUfEYkvDEC0zkigWipR4s1b8p0CSPwcf
	6X9vvqT0e8yfkxaAiJOnmrPExhHtc2pbjiecd76jaRxe1Ng594tZwLtCWhPdttg=
X-Google-Smtp-Source: AGHT+IF0RMlUvxAgDq1KPXVZiKCMcnuETfLF4/MLke8NcfLOvXtreXo+812+fzSo2yKt9z/67/XmYA==
X-Received: by 2002:a05:620a:2993:b0:792:9527:c162 with SMTP id af79cd13be357-792c75f1f79mr3694937985a.50.1716218439739;
        Mon, 20 May 2024 08:20:39 -0700 (PDT)
Message-ID: <32c1b1b0-6372-4c15-bf0a-cb946df236f5@citrix.com>
Date: Mon, 20 May 2024 16:20:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3.5 3/4] tools/xen-cpuid: Use automatically generated
 feature names
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <20240520143359.3376849-1-andrew.cooper3@citrix.com>
 <ZktnJ7XNxhdLOxdk@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZktnJ7XNxhdLOxdk@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20/05/2024 4:07 pm, Roger Pau Monné wrote:
> On Mon, May 20, 2024 at 03:33:59PM +0100, Andrew Cooper wrote:
>> From: Roger Pau Monné <roger.pau@citrix.com>
>>
>> Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
>> data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
>> are needed.
>>
>> In xen-cpuid.c, sanity check at build time that leaf_info[] and
>> feature_names[] are of sensible length.
>>
>> As dump_leaf() rendered missing names as numbers, always dump leaves even if
>> we don't have the leaf name.  This conversion was argumably missed in commit
>> 59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

>
> Just one question below.
>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>>
>> Differences in names are:
>>
>>  sysenter    -> sep
>>  tm          -> tm1
>>  ds-cpl      -> dscpl
>>  est         -> eist
>>  sse41       -> sse4-1
>>  sse42       -> sse4-2
>>  movebe      -> movbe
>>  tsc-dl      -> tsc-deadline
>>  rdrnd       -> rdrand
>>  hyper       -> hypervisor
>>  mmx+        -> mmext
>>  fxsr+       -> ffxsr
>>  pg1g        -> page1gb
>>  3dnow+      -> 3dnowext
>>  cmp         -> cmp-legacy
>>  cr8d        -> cr8-legacy
>>  lzcnt       -> abm
>>  msse        -> misalignsse
>>  3dnowpf     -> 3dnowprefetch
>>  nodeid      -> nodeid-msr
>>  dbx         -> dbext
>>  tsc-adj     -> tsc-adjust
>>  fdp-exn     -> fdp-excp-only
>>  deffp       -> no-fpu-sel
>>  <24>        -> bld
>>  ppin        -> amd-ppin
>>  lfence+     -> lfence-dispatch
>>  ppin        -> intel-ppin
>>  energy-ctrl -> energy-filtering
>>
>> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
>> one.  Several of the + names would be nice to keep as were, but doing so isn't
>> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
>> but we intentionally don't list them, so I'm not worried.
>>
>> Thoughts?
>>
>> v3:
>>  * Rework somewhat.
>>  * Insert aliases of common_1d.
>>
>> v4:
>>  * Pad at the gen stage.  I don't like this, but I'm clearly outvoted on the matter.
>> ---
>>  tools/misc/xen-cpuid.c | 16 ++++++++--------
>>  xen/tools/gen-cpuid.py | 29 +++++++++++++++++++++++++++++
>>  2 files changed, 37 insertions(+), 8 deletions(-)
>>
>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>> index 6ee835b22949..51009683da1b 100644
>> --- a/tools/misc/xen-cpuid.c
>> +++ b/tools/misc/xen-cpuid.c
>> @@ -11,6 +11,7 @@
>>  #include <xenguest.h>
>>  
>>  #include <xen-tools/common-macros.h>
>> +#include <xen/lib/x86/cpuid-autogen.h>
>>  
>>  static uint32_t nr_features;
>>  
>> @@ -291,6 +292,8 @@ static const struct {
>>  
>>  #define COL_ALIGN "24"
>>  
>> +static const char *const feature_names[] = INIT_FEATURE_VAL_TO_NAME;
>> +
>>  static const char *const fs_names[] = {
>>      [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
>>      [XEN_SYSCTL_cpu_featureset_host]    = "Host",
>> @@ -304,12 +307,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
>>  {
>>      unsigned i;
>>  
>> -    if ( !strs )
>> -    {
>> -        printf(" ???");
>> -        return;
>> -    }
>> -
>>      for ( i = 0; i < 32; ++i )
>>          if ( leaf & (1u << i) )
>>          {
>> @@ -327,6 +324,10 @@ static void decode_featureset(const uint32_t *features,
>>  {
>>      unsigned int i;
>>  
>> +    /* If this trips, you probably need to extend leaf_info[] above. */
>> +    BUILD_BUG_ON(ARRAY_SIZE(leaf_info) != FEATURESET_NR_ENTRIES);
>> +    BUILD_BUG_ON(ARRAY_SIZE(feature_names) != FEATURESET_NR_ENTRIES * 32);
>> +
>>      printf("%-"COL_ALIGN"s        ", name);
>>      for ( i = 0; i < length; ++i )
>>          printf("%08x%c", features[i],
>> @@ -338,8 +339,7 @@ static void decode_featureset(const uint32_t *features,
>>      for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
>>      {
>>          printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
>> -        if ( leaf_info[i].name )
>> -            dump_leaf(features[i], leaf_info[i].strs);
>> +        dump_leaf(features[i], &feature_names[i * 32]);
>>          printf("\n");
>>      }
>>  }
>> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
>> index 79d7f5c8e1c9..601eec608983 100755
>> --- a/xen/tools/gen-cpuid.py
>> +++ b/xen/tools/gen-cpuid.py
>> @@ -470,6 +470,35 @@ def write_results(state):
>>      state.output.write(
>>  """}
>>  
>> +""")
>> +
>> +    state.output.write(
>> +"""
>> +#define INIT_FEATURE_VAL_TO_NAME { \\
>> +""")
>> +
>> +    for name, bit in sorted(state.values.items()):
>> +        state.output.write(
>> +            '    [%s] = "%s",\\\n' % (bit, name)
>> +            )
>> +
>> +        # Add the other alias for 1d/e1d common bits.  64 is the difference
>> +        # between 1d and e1d.
>> +        if bit in state.common_1d:
>> +            state.output.write(
>> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
>> +            )
>> +
>> +    # Pad to an exact multiple of FEATURESET_SIZE if necessary
>> +    pad_feat = state.nr_entries * 32 - 1
>> +    if not state.names.get(pad_feat):
>> +        state.output.write(
>> +            '    [%s] = NULL,\\\n' % (pad_feat, )
> One likely stupid question, but since my understanding of Python is
> very limited, why do you add the comma after pad_feat?  There's no
> other parameter to print.

It's a common python gotcha with %.

>>> a = (1, 2)
>>> "%s" % a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting
>>> "%s" % (a, )
'(1, 2)'

You should always pass % a tuple, even a 1-element tuple, so it does the
right thing when you're not sure of the type of the thing being printed.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon May 20 15:46:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:46:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726225.1130485 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95Do-00006P-HE; Mon, 20 May 2024 15:46:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726225.1130485; Mon, 20 May 2024 15:46:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95Do-00006I-EQ; Mon, 20 May 2024 15:46:44 +0000
Received: by outflank-mailman (input) for mailman id 726225;
 Mon, 20 May 2024 15:46:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s95Dm-00006C-TE
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 15:46:42 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2626d5e6-16c0-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 17:46:40 +0200 (CEST)
Received: from PH7PR10CA0002.namprd10.prod.outlook.com (2603:10b6:510:23d::28)
 by CH3PR12MB7523.namprd12.prod.outlook.com (2603:10b6:610:148::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 15:46:35 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:510:23d:cafe::b6) by PH7PR10CA0002.outlook.office365.com
 (2603:10b6:510:23d::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Mon, 20 May 2024 15:46:35 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 15:46:33 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 10:46:33 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 10:46:32 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 10:46:31 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2626d5e6-16c0-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QDJksvdHtqPzvIHxXHhZBYm6XqBcEuG853wPl74kKEZnyBZ+qKvArv9rBrffAZvfBoa1vvescD/Tgq3wBmUXnkZCemC19+voYbPasrXx37YRsVz0zwKLDAUmUCAjDfmT0KFd0afP1ZNULwRzSJbVI++xtarQ+0QaNh73JI04hG9F1iq/XHCE7noru5xj9Tza/ayg1AfyHWI9Psb+xZG3ohpdF3T0Mp3T/uWyFMvwgaXtjiZRwr60KddXhJhoM1sf6AOmSCbMSlGgg44vZ9lJ1Q95tG+JgtlFtCmzXKHwuaB0oSQCdhpS//pcEy/5k4GaCA+wo+uhxhon7G2B0UAxqQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=1pGhMeX1ES8SvaJ9i2ibwt+ui1KvMWBcqi0tw9y4FOE=;
 b=e8wjAbK0fhSZwsWkl+Q3JHFv0etSa1DzoAgvXTLgCpDe9G2hb6rQ8DegkU/+wDj9po9n8E6MH7EMkPK7SLvv91sQ4J26gM8XC1iix8v6bBMKgnPrr+AuR554nVWaOyHsS3aAAN0JgHdcgv4xyF88xv1QFT7xO0OHwSEr56P9Su1HUGPMgYJTPzB0Hl+EiLBoOrCPIjBymphn8r2TwDeGi8zvinemCnf5vX/OP2VJRKQ0D2v8M1q2NNrHLJ56O7MRMPIXJi10ZmvHJeKAqbv1OCg20/V6/lssYff0f0im/BYPPArk/hSgPsrzFS0dsfHeTD15AF5fWLtdwv/Nw5/bxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1pGhMeX1ES8SvaJ9i2ibwt+ui1KvMWBcqi0tw9y4FOE=;
 b=KMpwdg9QPIxyaMXM0VP95T4mbDWR7q5kvzilpM1dFub2qeCpdLi+U3am8CWxZxAaufOnm5DEq1n1a2ekawnx/AKAmxJsMFaHeU8gnZ0QAvbtR39uk8GYmlJgZYHMzrDAOEihBLZJjG0K7q1C0Bqi+pjtL03jMx9++5YeLM24BWg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
Date: Mon, 20 May 2024 17:46:31 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>, "Luca
 Fancellu" <Luca.Fancellu@arm.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
 <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|CH3PR12MB7523:EE_
X-MS-Office365-Filtering-Correlation-Id: 0f036d56-2020-4dec-46c4-08dc78e406aa
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NFhvaDdRcDVYSGRkRkRTdnZOYXh1NTZQMTN5em1aNmQ4VFZ0TllONUZQcTlI?=
 =?utf-8?B?KzArY0w0Uk1adzhRZW9IcUNndEJPU1Y5anMvTTVhMFgvVGZnTnY3MnNtQkFV?=
 =?utf-8?B?cjB0Z3pPaElxNjhmRGtSTTBFdGF6NTh0RmJVN3ZIU1Eyc2VjVVZtY3l2cWVG?=
 =?utf-8?B?TlFFemxHTWVTWmRMcDEyRUcxVGR1d3BsQ0xoWEU0cyt6K1p0S2h5UXk0Y0s1?=
 =?utf-8?B?M080dWdyMVZWbzZTdUdjaVR2Y29KSkRzMmFRL29hNS8xZjlMQXNMUkJXcFlU?=
 =?utf-8?B?UWVGMStPMVFvNk9CTEpYMXNWSzdHLzRDMU1uakp5KzBFdzhpMm9qa0ZGUld1?=
 =?utf-8?B?bUdQOEowYnhrVG4zb2MwekJCanlsVjg4M0pGcE5CNW94enc4RmZINFQzbHhk?=
 =?utf-8?B?QjcvUGM1VVBMcFlEZTAzQ0p6c3RNQUFxSUttZ2hlcENjdkI1Z3BJT0N1dmV3?=
 =?utf-8?B?MzZOZ0QvWGFmUytvWW9idGJJZFNsWXhlbm96VjlObWU4NWw4N040cWtGVDhN?=
 =?utf-8?B?YURmZkRJZEtMQVhyWnB4WXRlY3ZTRzh4OEdEczZITlMwUTBVZFVBSUtPamJ2?=
 =?utf-8?B?eTBaNFFKMVZOcGFrNFM2ZEs2TzhSd0FiUktEQnBMRzJnRlVoMk9xb0RrQ1RB?=
 =?utf-8?B?V2FwSjJqQ1AyMGw2a1ppUzBtV01OZ1FYRFZMSHhaRlFYVk1FVkN4R3ZuWUFK?=
 =?utf-8?B?RE9GMkN3UTdUbUVWdUFUSVdsWWhsRUVrYTFzMko2NUJFaWtNV3hGYWkwa0s2?=
 =?utf-8?B?MTZkeWhYR3crRytjcXIxOGFzd0JVU0Vkc1BUR1lPYkEzeS8zNC9BMXB0OEVt?=
 =?utf-8?B?a0FvN0txMGw2NzB3OFVZWDhkZ2p1c0dLdlc4aEFxQzlHVlFVUDZPR3RXNGs2?=
 =?utf-8?B?Y2ovc1NvdUZaZUs3Q2IwZnRzUDdBY3JRNXE4MWxTdFhTK3REWHZrZ3pyRTkx?=
 =?utf-8?B?S1VkLytWNE1tUXlFdEluMFZLZW9OZForcFJydHVuU0EyZmhhTTd6VmJiamM0?=
 =?utf-8?B?M3liRUpFV3RkcDJacGxKVll6TE9aTXoxeWFBY09SQlZOR3pJQkF5aTZOUksr?=
 =?utf-8?B?dGNqd1ZpZHUzZHFieUdwQ0JqYjJYYlhaaUFZc0dJV3JOc0RQdzRwVWtvRnM4?=
 =?utf-8?B?d09kNDljVVlvdk1vS2lHN0o3cGpvRWt5MHROanNGaCtVR255NStEMkdqcDdP?=
 =?utf-8?B?b1NFUkdvcVBUSWVsNlRmR2FEcVlQbEREc2h5UGZ4ZE45WW5aa0pqdlErZm9a?=
 =?utf-8?B?emlVS0pzRGl1Q0R0ZlVPQStSVS95SHg4UXRNSGh3ZlNEWVV0LzFLN0xoWnZX?=
 =?utf-8?B?dVpiamQ3M2JkM2JEYzl2TVZmOENiTDg0SkFCdjZTeXhlbjNldGNLcWY0bDhL?=
 =?utf-8?B?bUZPbGZVQTBURnk4L0ZrSFE5ck1Kczk1TERQVDhVVFhXM2tKTmlCSmw4MVMv?=
 =?utf-8?B?RWdkVkpYL1ZPbzViUXhuUFJ2bVByeHMzVGE0SnZpTHJpZmRDSWMwaDZ3eTRN?=
 =?utf-8?B?a1lGSEpDMDFaSitWRU9tN1pEWGVleTFtcFNpTUU1RWRnREVSaUZlM1o2ZWQr?=
 =?utf-8?B?TlRpWUwrZXo2dXdkVnBLV1IxajNCbjVTQ0tEbXE5Z25aTHFCakp2K29VMjhC?=
 =?utf-8?B?clh3b2JoM1dsSzRVVDMzR2NhUGJ1ZklWL2w1RG9ndGhrQTNtNHFWaEEyN1Y5?=
 =?utf-8?B?SHJ3SmQ2ai96TnNYZnFaSHpqQitUcUpub0hIZzRNWjRQVmNBMk5PTGRTOFR5?=
 =?utf-8?B?NTZoYzNuaTZDNlB2VnlXdkNmYkhRS3Fjb1VMZFBnU05mem43SjdUdldEalg5?=
 =?utf-8?B?NlZEcWkvcmtsREE0UkxvOVpTa1lLZVptWFlVZlpSS0t1bG9lVlVzQ29zVVR1?=
 =?utf-8?Q?GfoiKsBClQ3IK?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 15:46:33.8373
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f036d56-2020-4dec-46c4-08dc78e406aa
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7523

Hi Henry,

On 20/05/2024 16:52, Henry Wang wrote:
> Hi Michal, Luca,
> 
> On 5/20/2024 7:24 PM, Michal Orzel wrote:
>> Hi Henry,
>>
>> +CC: Luca
>>
>> On 17/05/2024 05:21, Henry Wang wrote:
>>> To make things easier, add restriction that static shared memory
>>> should also be direct-mapped for direct-mapped domains. Check the
>>> host physical address to be matched with guest physical address when
>>> parsing the device tree. Document this restriction in the doc.
>> I'm ok with this restriction.
>>
>> @Luca, do you have any use case preventing us from making this restriction?
>>
>> This patch clashes with Luca series so depending on which goes first,
> 
> I agree that there will be some conflicts between the two series. To 
> avoid back and forth, if Luca's series goes in first, would it be ok for 
> you if I place the same check from this patch in 
> handle_shared_mem_bank() like below?
> 
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index 9c3a83042d..2d23fa4917 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -219,6 +219,13 @@ static int __init handle_shared_mem_bank(struct 
> domain *d, paddr_t gbase,
>       pbase = shm_bank->start;
>       psize = shm_bank->size;
> 
> +    if ( is_domain_direct_mapped(d) && (pbase != gbase) )
> +    {
> +        printk("%pd: physical address 0x%"PRIpaddr" and guest address 
> 0x%"PRIpaddr" are not direct-mapped.\n",
> +               d, pbase, gbase);
> +        return -EINVAL;
> +    }
> +
>       printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys 
> 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
>              d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
> 
>> Acked-by: Michal Orzel <michal.orzel@amd.com>
> 
> Thanks. I will take the tag if you are ok with above diff (for the case 
> if this series goes in later than Luca's).
I would move this check to process_shm() right after "gbase = dt_read_paddr" setting.
This would be the most natural placement for such a check.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 15:51:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 15:51:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726235.1130495 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95If-00021f-7J; Mon, 20 May 2024 15:51:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726235.1130495; Mon, 20 May 2024 15:51:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95If-00021Y-42; Mon, 20 May 2024 15:51:45 +0000
Received: by outflank-mailman (input) for mailman id 726235;
 Mon, 20 May 2024 15:51:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s95Id-00021S-LU
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 15:51:43 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d92632f7-16c0-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 17:51:41 +0200 (CEST)
Received: from MW4PR04CA0270.namprd04.prod.outlook.com (2603:10b6:303:88::35)
 by SN7PR12MB8603.namprd12.prod.outlook.com (2603:10b6:806:260::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 15:51:36 +0000
Received: from CO1PEPF000066ED.namprd05.prod.outlook.com
 (2603:10b6:303:88:cafe::82) by MW4PR04CA0270.outlook.office365.com
 (2603:10b6:303:88::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.33 via Frontend
 Transport; Mon, 20 May 2024 15:51:36 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 15:51:35 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 10:51:34 -0500
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 10:51:32 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d92632f7-16c0-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IMqax1w0bLnuSwUCJfyquMm+MLFBlK+mXlytLO4AcotDdllcDP/iW6IYOkfqqyBjvT+26Fx2thsb4wP5osTcD37MUnD6a6/Gj2ZOVixAIINJSMvAwIHad6p29Yl6VkPSXGgZEjjw2LAubsnx3C2C55wn96joGm2UsoOcIP1Nb5dst0z01Y8m0rVhYY6bzr+dxSVXLzyB1h/61QPwnbTv9d0ShdoE7miM/cxuIHwnXEYPN+p6ueX7qwQ9WZ76JP2dDbtM071R4S/GHfC3hPkNmPoVAF0Vy6GdnpWJaDOff/x4kRGZISIQXx6YhEPH8PB8bt7xbZJFkci146wH/zV9KQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=eyUrQ23iDzW2/ijRs7j6gY/PKXIw77SJywLxUp/9efU=;
 b=Fg06qe4JB0XRKLf7W+VaZi9e2BtPvXF1Eut4Awd2J9dLlxKGb6xXGwLvLO/h96x11uAFMwzSjcSBKCZiNZzvGYVn5OC+GN2Igq6CN4ImTqqt9Dss+K4Nd2lAqSfPUMXNWgLwGDhOe042ILL8tmk4bFnc2Panpw+EHGNV4GlceF7EM8iuIOp356Te9tbJbkQHrBcyFMcIXBEZnRxtWw4MW914klF8vFWafeNgIKUvNm6UAP4G8xxXA23nkGDu8KYKsvU68Pmny/eGsXFvrLFFabhPYz7vYzQ/EAd2kFZqHioqTc73T1e9xudhfm3Z2F2bQKlDfjec88nryisJv2tcqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eyUrQ23iDzW2/ijRs7j6gY/PKXIw77SJywLxUp/9efU=;
 b=VrNmVOcsLV7V3qUo8BaPZr995xH13rGj0zzisybW1oh7dnBDij+nfx4SWTAPTvvotGoQcinMQdbEHQr6DMtqGmCH9suHqGFUSO8RB3lbFvqfi1tSjgkqNK8kPC1Zt3KtiWQHnRfSuqVquG/2XCBNwiYtgE4UeoW0r+mbFyNAkto=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <c0b59ccd-b3b0-4495-bbb4-2855fa2d7d16@amd.com>
Date: Mon, 20 May 2024 23:51:26 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>,
	Luca Fancellu <Luca.Fancellu@arm.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
 <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
 <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|SN7PR12MB8603:EE_
X-MS-Office365-Filtering-Correlation-Id: ec7cabfd-d623-4f75-dbe9-08dc78e4ba82
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?K1FhemM1cm0rWDUvUGpEZ1UyMlc2aGNsN1o3S21DbE9EQ1JvNG03RGR4QjVK?=
 =?utf-8?B?ZXVGdVVyYlRqL1UwYXZYMHhXVWxUTVpkUWxHdXZMbU9sRGFSUzJSRFV1YW9s?=
 =?utf-8?B?bXQxMklhYTZOVGRzSVVVaVNuRXFZNGg3Nm82SGdKMGZiZHJFdXB5bnkwcmds?=
 =?utf-8?B?RGdEYzhRTGRGeDRjdXQxWUdoU0ZOSU0rRE1lN3ZCK0V1eVpIUE5iQ2RvUTFD?=
 =?utf-8?B?am9FL1RmNkpBZE9oZ05hWlk0Q00xMERycXRxK2pmQmc1eXZRRGhKclBoZGcv?=
 =?utf-8?B?KzRjTHFNTTBia1l6RTR2bDU5STdaUHo0aDJIdjkvUG5wMms0bnBIa0R3TWw1?=
 =?utf-8?B?TDRXaGJ4WDJWTC83aEZsc0h2dFY5VThzeXFlcEUvOFoyRCtJOFFkbEUrY2Ez?=
 =?utf-8?B?VG8wS3FBMG1vM1JZaG42R1ZuM1lkaDNNOUJhbUdtT1AvWFRPU0ZMVms5MzJl?=
 =?utf-8?B?TldrRytBcWhMNTJjZ2x5WmVjcXRlcGw4MlkxcFBaRHBobnNKVFZPUkp1OS9I?=
 =?utf-8?B?anVlQlp0bnFTTmtGTnRpdGpVWlhYZm5IMG0zVk1BQjNYZnk2VmRxcFdjTzRC?=
 =?utf-8?B?NHM5Uzl6SUVwSjFDcnhDZGZQOUFMM1UzQ3ZRSmNsZnUyYmExeFlub0JoOHVQ?=
 =?utf-8?B?RVkvdVMyazVNc0xlbkdJTzd3KzRud1ZNczVzeUtUQXVNSmJSa2VPRUlMMDg4?=
 =?utf-8?B?Q0dyZlBNOGkyNXlsSnpwdHQ2KzlFVHZPMFVDNml6T2o5dXRreHp0TVE5QlFV?=
 =?utf-8?B?YWF5cVRycHVVc2hoNjBwYVBGRTRIaWxVeFJRUUpWVHpRVWQ2RzhRZWRLTmZl?=
 =?utf-8?B?NUV5eENuUDhvTDRnckx0RGtVNGtWWUIyUm1lNGc0b2pCMFNtK3RJdVRTaXJL?=
 =?utf-8?B?SU9STjF4Um5Rc1d0WUxDV1BhaWx2QmxGSS9vdUtxS2NYQlh1L0d3UlY3M1RX?=
 =?utf-8?B?K3RVOUVMMWQvMEd1L29CMEpCUUlUVC9MeFBuNUFqbHdIVC9Yd2tUWHluQXFq?=
 =?utf-8?B?bUxoeTRCWHU5bTVqd3MyUUs0K1VzZVlwNzZxMjJSci9PcEpVWTVPRG82MlRI?=
 =?utf-8?B?OGxoUWNlYjYwV1hneHVpMXFXcDdvRU5LQk9rSnNoQ081Wk1GbEpLYjdOaVJa?=
 =?utf-8?B?M2N6elFpaTBRVzVyTjR2MHJxOFQ0VzNGR2owc2dEVjhMWmF1cmxhL2ttMVlp?=
 =?utf-8?B?ZUVGQm02ejN0YWJ1SjViVGdjaGZGakZVcGNxNnMzRlV5L3MwYnUwejVhMFgv?=
 =?utf-8?B?bkI1blBLTUlxaEQ2N05IM3F5Q08xM21idE90UGlWU0R3LzFkNFMwdTM3d3o4?=
 =?utf-8?B?UHdySHBualJVR0d6cDVRZTAzTWh5bWFtMXBNVjJlNWVseVhLYndvcGovZWJH?=
 =?utf-8?B?M1ZiWHlsVE9ISVBtZXgyOHlyMkt5OTZqVEFqSEtKejRMeUFaWjBSRGpueWVM?=
 =?utf-8?B?WVc2WU4xc21wc0NRVDZOaUhzOTFENHZSdUJIYld1U2NNVG5RSWtTNEsvNkZw?=
 =?utf-8?B?WVc3WUtHbmI5cDN2WHF0WXhzRDNyZDJQMjRFYXJJTVBZSi9Wb0JCMkxxbWxN?=
 =?utf-8?B?YWhmTlBYMWN6Q2ZhWC9RSGF5Y0hsVXpMYTkwSDhLelJtRWFPMmFwQ25GVEIy?=
 =?utf-8?B?b0h3UEhvWWtrWTl3NWZVYjJMV1VyelhpWVVFVi96N2xFL05BR2txb092a1VG?=
 =?utf-8?B?czNQSUpuMDNMV2EvaUhjcmhrdXdpV0l1bzNlY010R3lDZmpvL29peEdodThx?=
 =?utf-8?B?dHdvMGkySnhzK2wxV3RKb2ErNTNDR0hyd1NscGFNUzE4cURuYnV5MmxoeGVM?=
 =?utf-8?B?WUk2c2ptZnZUTHc5UUpMS09RYUFJQ0xrNjVIZW5YU3hLNTVvQWJXU09WeEVJ?=
 =?utf-8?Q?8nbHs+e6Unina?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 15:51:35.5376
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ec7cabfd-d623-4f75-dbe9-08dc78e4ba82
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066ED.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8603

Hi Michal,

On 5/20/2024 11:46 PM, Michal Orzel wrote:
> Hi Henry,
>
> On 20/05/2024 16:52, Henry Wang wrote:
>> Hi Michal, Luca,
>>
>> On 5/20/2024 7:24 PM, Michal Orzel wrote:
>>> Hi Henry,
>>>
>>> +CC: Luca
>>>
>>> On 17/05/2024 05:21, Henry Wang wrote:
>>>> To make things easier, add restriction that static shared memory
>>>> should also be direct-mapped for direct-mapped domains. Check the
>>>> host physical address to be matched with guest physical address when
>>>> parsing the device tree. Document this restriction in the doc.
>>> I'm ok with this restriction.
>>>
>>> @Luca, do you have any use case preventing us from making this restriction?
>>>
>>> This patch clashes with Luca series so depending on which goes first,
>> I agree that there will be some conflicts between the two series. To
>> avoid back and forth, if Luca's series goes in first, would it be ok for
>> you if I place the same check from this patch in
>> handle_shared_mem_bank() like below?
>>
>> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
>> index 9c3a83042d..2d23fa4917 100644
>> --- a/xen/arch/arm/static-shmem.c
>> +++ b/xen/arch/arm/static-shmem.c
>> @@ -219,6 +219,13 @@ static int __init handle_shared_mem_bank(struct
>> domain *d, paddr_t gbase,
>>        pbase = shm_bank->start;
>>        psize = shm_bank->size;
>>
>> +    if ( is_domain_direct_mapped(d) && (pbase != gbase) )
>> +    {
>> +        printk("%pd: physical address 0x%"PRIpaddr" and guest address
>> 0x%"PRIpaddr" are not direct-mapped.\n",
>> +               d, pbase, gbase);
>> +        return -EINVAL;
>> +    }
>> +
>>        printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys
>> 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
>>               d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
>>
>>> Acked-by: Michal Orzel <michal.orzel@amd.com>
>> Thanks. I will take the tag if you are ok with above diff (for the case
>> if this series goes in later than Luca's).
> I would move this check to process_shm() right after "gbase = dt_read_paddr" setting.
> This would be the most natural placement for such a check.

That sounds good. Thanks! IIUC we only need to add the check for the 
pbase != INVALID_PADDR case correct?

Kind regards,
Henry

> ~Michal



From xen-devel-bounces@lists.xenproject.org Mon May 20 16:10:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 16:10:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726248.1130505 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95aK-0004ef-Je; Mon, 20 May 2024 16:10:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726248.1130505; Mon, 20 May 2024 16:10:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95aK-0004eY-Ge; Mon, 20 May 2024 16:10:00 +0000
Received: by outflank-mailman (input) for mailman id 726248;
 Mon, 20 May 2024 16:09:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nMKL=MX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s95aI-0004d5-FJ
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 16:09:58 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 63e3bd2a-16c3-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 18:09:55 +0200 (CEST)
Received: from SN4PR0501CA0061.namprd05.prod.outlook.com
 (2603:10b6:803:41::38) by DM4PR12MB5820.namprd12.prod.outlook.com
 (2603:10b6:8:64::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 16:09:49 +0000
Received: from SN1PEPF000397AE.namprd05.prod.outlook.com
 (2603:10b6:803:41:cafe::ca) by SN4PR0501CA0061.outlook.office365.com
 (2603:10b6:803:41::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.16 via Frontend
 Transport; Mon, 20 May 2024 16:09:49 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF000397AE.mail.protection.outlook.com (10.167.248.52) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 16:09:48 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 11:09:47 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 11:09:46 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63e3bd2a-16c3-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DHHn9PNBiNqi7zYGh8JcwBswzHaehcZ6FjN/ES+J8EjtQjg9H9f24LTmdVJrMpCpgO7QyuC9489ebn5oy7CnI8wYUB7tLuNjDrfyYoE12bXJKpGdw5fAlmElFGxqhFvxigjLsqhLrE+qOnW+z89VzG8vT22NOao6v9PQ1hTbCSptubWwJfA7CZi7xZY3zL3ND1ueCm+wBN/lp3x7Z1Grjp0zSBA0w+UiQV/KHoCmVu6pFvVZP0AgO/K2Sqar13b6j/AuMvtRxc0Zdl1Au1cz7AK41oV+NCbDLKDqlVR13Wr45y2m2xlIZwHZW8U1Xq8lS0JD77o6x28VtEoX38MDPg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=r01wKV+EZkO49GWXKN3y+QUk+8tw/NOThTl1UerYC4A=;
 b=Vg9f40DSenIaDkFK4KSC4P+Pz4OayEsXFFPVR5vMCQpVHUl1lzdHGG8ysHWIW+/ljIrYd5F8sRcsRKkma05P7AMfBYR3w2y8zHKC50ZHGBRpBDF/zppGBlD6Yf5cKU4kn9aQ0l9wKVEHf6rJQbj+NCNivYkKX9MHGL28Rd+TSPR/ZdENCKjJPdB3iTtmSkWpCsMSm1A4eFPGU1ubuqdrxGTJnp0+q7WdQP/HRKEB89LAeeGlQXlZsdShPYprDqu7KTLPSL/ZeN/FRJEzG6n0la1dAD1iWhzIgdY8Eoq9Re6/ONy5ukNJkAGzk3u/qOoJ/NzcTw7gekX6eEuqLUlC0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r01wKV+EZkO49GWXKN3y+QUk+8tw/NOThTl1UerYC4A=;
 b=PMalGxU51FY3exVmKX2V3ZdNkBS0SyW1qtmiOL18a3TqrYUblix7WInD9a9EoekaDuQSBFaBp2Dou7bUajTFr/BbBkdCXaZ8J2ZMgZ7S5uhMtv4v3+vdOUKVtCunhndHvFcdYvt4AOUpAWqEgB/vvjGGbKnoZLvw2n8a+KainMQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <93336435-309d-4aa5-9977-9ce0d4725811@amd.com>
Date: Mon, 20 May 2024 18:09:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>, "Luca
 Fancellu" <Luca.Fancellu@arm.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
 <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
 <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
 <c0b59ccd-b3b0-4495-bbb4-2855fa2d7d16@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <c0b59ccd-b3b0-4495-bbb4-2855fa2d7d16@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF000397AE:EE_|DM4PR12MB5820:EE_
X-MS-Office365-Filtering-Correlation-Id: ed22a0d3-4903-4078-ec9c-08dc78e7461d
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bGZRT3lPaWxxYUU2YjNvN1IzTERqN3VOblduclNBTEpuZzQyZ1hKMnVab29C?=
 =?utf-8?B?K1FiWFdnVzJ6Qkd1UUpSdXc0SzVrSndiRE1WSzhlZVYydWI0dDdkOEpMcmVR?=
 =?utf-8?B?SkhqT0RlaEcyY3BwYmhCMVpZbUxpeDNZMGhDU1NQZ2NvMEhaOW9FaW1jblRa?=
 =?utf-8?B?S0xjUHArTGZVeEptaGtVSFlrMHhhRVdsakg3bzd2MEFBRjY4STNpMTJMcHZV?=
 =?utf-8?B?L3JFSkNSL1dJLzJFSDJ5R1NWdURrYlRqbkZVM0NPeTJERVhEdWQ0R0NLVjE4?=
 =?utf-8?B?L3FOUGRkcGp2VFhvcDFKWFNlTWNrTGpxaVo4NnlHTXY0UVJBd1RMb1hGVzV1?=
 =?utf-8?B?VnliNnlnM0F0NHlCZVVkOEtGL04vMzd0cXExSjk2cUI0RUhvRGNON29GdFd3?=
 =?utf-8?B?VGZ3R0swZ2EyZjBDTVZsL1dCZkNuNzFXSHh1Wm9UWkxWdHRQanhOcU9DeWR1?=
 =?utf-8?B?R1k1WHRnSDUzN0ZMdXpiZ3RieTlPdThndU1MNUhwWkZHUGU4WFJtZHU1QWZ3?=
 =?utf-8?B?bXpDMWdoekgrOHBDVkx4VzNtV0JJZks3dU4vczEweE8xYWNqMFg2Z2dvUWxN?=
 =?utf-8?B?YXlkTnBNd2ZTUnVIaEFuaVlJRDZRdlBTa2VEMjBNeTBzS0pjMTdDa1kxY09w?=
 =?utf-8?B?TkRYMGFSMzhlREgyOGpkSmVlRXdlRjZYbCs4MHpnNVUyaVp6eU5USVI4N0RY?=
 =?utf-8?B?T1YwUFZDZ2p3eElYMzFJZkJVVzR3Q0VqdVZkbExEYXBINU5tMXVXVnV5bVFX?=
 =?utf-8?B?WDJ2K1pBSzdPS3lnMVVQT0N5clBtRHBXeXZFWXpwS3ZaSUVnUjVpUkZ0QTdR?=
 =?utf-8?B?OGNnTDhqUTB4amhEN2VnYUgwUlo2SjlFVGNXbzkzVmlZM3dIdW41TGptODd0?=
 =?utf-8?B?cTVpcUF2MnpwSWhFYmVUT3FyQjZvSXAzbWtPZXFseFZvVGVhR2ZLRnNLWndI?=
 =?utf-8?B?d1VHUlFMNDZuSjhjMysyaE9weFNsNDhia2hYWWFPY2hZWWJOemtDUkdXRE40?=
 =?utf-8?B?TEhaSUxLU3puemRzeDA5VllSUTZ6VkJOMVU3VVZLWWllODJSQml6RWJEUFJq?=
 =?utf-8?B?SzRZUkFoMmV0MTNWNWc2aTA0bVgrbjhMMnhocXJucGd2Mm9rNFlEUlg1eDdT?=
 =?utf-8?B?cmthOXlPT0NLeHBQTzI3aktXQ1F1eE5mN1VSbDRXOSt6RUhwTUd5dmRGMTk3?=
 =?utf-8?B?UWVlY1BRUmd4aHRQczRmbVFLTmxMdkp3OW1IU09yNDI5ckpnQVVhSGRpK0dj?=
 =?utf-8?B?V1piVmxFU2E1dUt2a3ZkVkdiMGhWYnJTSDBqcW9yeWRpQSt4dVJ0Q254K3lC?=
 =?utf-8?B?djUyVWI4MkF6MlIxbkFiSlpZckQ4SU5pWEtRNnJpdTZpbkhzMUxIakFtYnU2?=
 =?utf-8?B?dHNtZ0ZOd2x1NlJoSjFnUEhMb21za2JrSU0ySWtWNlQvRThhYVFTNURlMHh0?=
 =?utf-8?B?M3YrTjdVYldudTNhODV0WXUwQmR1UURwN3BhaHNjaVBONklNdllNNXBQeU1x?=
 =?utf-8?B?YzlVczROQkp0cTE1ZmZDaVhrL0RXazdpZHFpUWtjVFBkZCtVWXRKQ3M4QXpY?=
 =?utf-8?B?Nng1U3ZLR3p5dWJSZWNOYVFOMzNUa3pXT1RPbTZ4WlY1aG04eXNrN01ZYksv?=
 =?utf-8?B?OHJLUTBLTitwRDdiRlQ1Z2l0cDZSdFBIWWxreGZBVmNPUjJzWTU3dEpta1Nx?=
 =?utf-8?B?ZndjdlZBNEQ4cDl3WWU3NHc2blZ6ejNFQ2xnQy9OZ0luR0tiMGxCVzM1Ry9G?=
 =?utf-8?B?YjlJaGpNZE5jRjl2NzNqNmZ4KzVKSE9TNmxrTTNDUnhmRFhGOVRiV2dOVmJW?=
 =?utf-8?B?WmpLU1V1MnJKdlpReXFHT01oWmxETW1ZZEFlSW5VQzJMZG53RnV5ZFBhRHJD?=
 =?utf-8?Q?6N/qJaKlQKk8V?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 16:09:48.8131
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ed22a0d3-4903-4078-ec9c-08dc78e7461d
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF000397AE.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5820



On 20/05/2024 17:51, Henry Wang wrote:
> Hi Michal,
> 
> On 5/20/2024 11:46 PM, Michal Orzel wrote:
>> Hi Henry,
>>
>> On 20/05/2024 16:52, Henry Wang wrote:
>>> Hi Michal, Luca,
>>>
>>> On 5/20/2024 7:24 PM, Michal Orzel wrote:
>>>> Hi Henry,
>>>>
>>>> +CC: Luca
>>>>
>>>> On 17/05/2024 05:21, Henry Wang wrote:
>>>>> To make things easier, add restriction that static shared memory
>>>>> should also be direct-mapped for direct-mapped domains. Check the
>>>>> host physical address to be matched with guest physical address when
>>>>> parsing the device tree. Document this restriction in the doc.
>>>> I'm ok with this restriction.
>>>>
>>>> @Luca, do you have any use case preventing us from making this restriction?
>>>>
>>>> This patch clashes with Luca series so depending on which goes first,
>>> I agree that there will be some conflicts between the two series. To
>>> avoid back and forth, if Luca's series goes in first, would it be ok for
>>> you if I place the same check from this patch in
>>> handle_shared_mem_bank() like below?
>>>
>>> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
>>> index 9c3a83042d..2d23fa4917 100644
>>> --- a/xen/arch/arm/static-shmem.c
>>> +++ b/xen/arch/arm/static-shmem.c
>>> @@ -219,6 +219,13 @@ static int __init handle_shared_mem_bank(struct
>>> domain *d, paddr_t gbase,
>>>        pbase = shm_bank->start;
>>>        psize = shm_bank->size;
>>>
>>> +    if ( is_domain_direct_mapped(d) && (pbase != gbase) )
>>> +    {
>>> +        printk("%pd: physical address 0x%"PRIpaddr" and guest address
>>> 0x%"PRIpaddr" are not direct-mapped.\n",
>>> +               d, pbase, gbase);
>>> +        return -EINVAL;
>>> +    }
>>> +
>>>        printk("%pd: SHMEM map from %s: mphys 0x%"PRIpaddr" -> gphys
>>> 0x%"PRIpaddr", size 0x%"PRIpaddr"\n",
>>>               d, bank_from_heap ? "Xen heap" : "Host", pbase, gbase, psize);
>>>
>>>> Acked-by: Michal Orzel <michal.orzel@amd.com>
>>> Thanks. I will take the tag if you are ok with above diff (for the case
>>> if this series goes in later than Luca's).
>> I would move this check to process_shm() right after "gbase = dt_read_paddr" setting.
>> This would be the most natural placement for such a check.
> 
> That sounds good. Thanks! IIUC we only need to add the check for the 
> pbase != INVALID_PADDR case correct?
Yes, but at the same time I wonder whether we should also return error if a user omits pbase
for direct mapped domain.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon May 20 16:30:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 16:30:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726269.1130531 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95tZ-0000T1-Sx; Mon, 20 May 2024 16:29:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726269.1130531; Mon, 20 May 2024 16:29:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s95tZ-0000St-Bi; Mon, 20 May 2024 16:29:53 +0000
Received: by outflank-mailman (input) for mailman id 726269;
 Mon, 20 May 2024 16:29:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ccuw=MX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s95tW-0000RK-Dp
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 16:29:50 +0000
Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com
 [2607:f8b0:4864:20::832])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c445b1c-16c6-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 18:29:48 +0200 (CEST)
Received: by mail-qt1-x832.google.com with SMTP id
 d75a77b69052e-43e0d564136so20261491cf.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 09:29:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df56d1a08sm147928111cf.88.2024.05.20.09.29.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 20 May 2024 09:29:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c445b1c-16c6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716222587; x=1716827387; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=q8tpl4lSMojgasGzV7D0LbOdJ5uTqQcYaymY/DPHWB8=;
        b=MMTs3YnjU8u95k35ZmzBtn+Dq/r9shIGAm3yhC/B2brwzSwcHoUSZlQEwfSj6Yi1/G
         FpgyDEDJtC6wjubDvFKxF+ctuhuAlk+Tc8okwnsKqwLrhTIX0JoNjlooOUfIANViF/3D
         8GjjX5WC4JJFD6hopK7umtSWvRNkaQ5DTCiBU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716222587; x=1716827387;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q8tpl4lSMojgasGzV7D0LbOdJ5uTqQcYaymY/DPHWB8=;
        b=kFGKg/8jGLEnIIFeUu6K1bCnPDbKhb+MbwfPx5UXt7vXWqnpZPPLZqerzNJO8D3I3g
         ByxnqrGAfxbBoLJyh56UieLzzkgGIwXM0qOwPuo44Hfv84xV2RJJ95NgXt99IqhF4IY9
         gUbFOm/m8V9dIQsIK6Tw5J8fKMCXV/HUHUbI9EUL/f696l9RZzOSSVzy3SPBgaF7vzB5
         pK7EbPYTp7u5+XwwZ8ZirGTRt8One1uJ60lYiruNxpVsEd6Mq/kIib0BajIeZOQnpHr5
         rw4sMDrbRThGFt/bSFskEUa2kZ6J21xiEFndj8TZZYUxr5KrAEe4tIjYfMWuCfFgs/WB
         3cAg==
X-Gm-Message-State: AOJu0Ywa2KS8aptRiFhI5WjWMgEl6QTb/GU0vXKR3SWSb0w9xqqp8nWm
	yUP3Qi3edQ9gH2kne5cVxBAKE0ke29Nbnbmf9IOVQQjhFrs1dqiRLGd+xUmDbH8=
X-Google-Smtp-Source: AGHT+IHEVtbC6Rmlv4/Z9agFhnsFmtIBCLzfdGXNbF/Q+f6aVQyS+8/JZxYdg2Eyf8e/oWMk0W78jg==
X-Received: by 2002:a05:622a:13ce:b0:43e:1ae6:dcec with SMTP id d75a77b69052e-43e1ae6def1mr268556121cf.55.1716222586735;
        Mon, 20 May 2024 09:29:46 -0700 (PDT)
Date: Mon, 20 May 2024 18:29:44 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>
Subject: Re: [PATCH v3.5 3/4] tools/xen-cpuid: Use automatically generated
 feature names
Message-ID: <Zkt6eHDYqNCOdgud@macbook>
References: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <20240520143359.3376849-1-andrew.cooper3@citrix.com>
 <ZktnJ7XNxhdLOxdk@macbook>
 <32c1b1b0-6372-4c15-bf0a-cb946df236f5@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <32c1b1b0-6372-4c15-bf0a-cb946df236f5@citrix.com>

On Mon, May 20, 2024 at 04:20:37PM +0100, Andrew Cooper wrote:
> On 20/05/2024 4:07 pm, Roger Pau Monné wrote:
> > On Mon, May 20, 2024 at 03:33:59PM +0100, Andrew Cooper wrote:
> >> From: Roger Pau Monné <roger.pau@citrix.com>
> >>
> >> Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
> >> data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
> >> are needed.
> >>
> >> In xen-cpuid.c, sanity check at build time that leaf_info[] and
> >> feature_names[] are of sensible length.
> >>
> >> As dump_leaf() rendered missing names as numbers, always dump leaves even if
> >> we don't have the leaf name.  This conversion was argumably missed in commit
> >> 59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").
> >>
> >> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.
> 
> >
> > Just one question below.
> >
> >> ---
> >> CC: Jan Beulich <JBeulich@suse.com>
> >> CC: Roger Pau Monné <roger.pau@citrix.com>
> >>
> >> Differences in names are:
> >>
> >>  sysenter    -> sep
> >>  tm          -> tm1
> >>  ds-cpl      -> dscpl
> >>  est         -> eist
> >>  sse41       -> sse4-1
> >>  sse42       -> sse4-2
> >>  movebe      -> movbe
> >>  tsc-dl      -> tsc-deadline
> >>  rdrnd       -> rdrand
> >>  hyper       -> hypervisor
> >>  mmx+        -> mmext
> >>  fxsr+       -> ffxsr
> >>  pg1g        -> page1gb
> >>  3dnow+      -> 3dnowext
> >>  cmp         -> cmp-legacy
> >>  cr8d        -> cr8-legacy
> >>  lzcnt       -> abm
> >>  msse        -> misalignsse
> >>  3dnowpf     -> 3dnowprefetch
> >>  nodeid      -> nodeid-msr
> >>  dbx         -> dbext
> >>  tsc-adj     -> tsc-adjust
> >>  fdp-exn     -> fdp-excp-only
> >>  deffp       -> no-fpu-sel
> >>  <24>        -> bld
> >>  ppin        -> amd-ppin
> >>  lfence+     -> lfence-dispatch
> >>  ppin        -> intel-ppin
> >>  energy-ctrl -> energy-filtering
> >>
> >> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
> >> one.  Several of the + names would be nice to keep as were, but doing so isn't
> >> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
> >> but we intentionally don't list them, so I'm not worried.
> >>
> >> Thoughts?
> >>
> >> v3:
> >>  * Rework somewhat.
> >>  * Insert aliases of common_1d.
> >>
> >> v4:
> >>  * Pad at the gen stage.  I don't like this, but I'm clearly outvoted on the matter.
> >> ---
> >>  tools/misc/xen-cpuid.c | 16 ++++++++--------
> >>  xen/tools/gen-cpuid.py | 29 +++++++++++++++++++++++++++++
> >>  2 files changed, 37 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> >> index 6ee835b22949..51009683da1b 100644
> >> --- a/tools/misc/xen-cpuid.c
> >> +++ b/tools/misc/xen-cpuid.c
> >> @@ -11,6 +11,7 @@
> >>  #include <xenguest.h>
> >>  
> >>  #include <xen-tools/common-macros.h>
> >> +#include <xen/lib/x86/cpuid-autogen.h>
> >>  
> >>  static uint32_t nr_features;
> >>  
> >> @@ -291,6 +292,8 @@ static const struct {
> >>  
> >>  #define COL_ALIGN "24"
> >>  
> >> +static const char *const feature_names[] = INIT_FEATURE_VAL_TO_NAME;
> >> +
> >>  static const char *const fs_names[] = {
> >>      [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
> >>      [XEN_SYSCTL_cpu_featureset_host]    = "Host",
> >> @@ -304,12 +307,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
> >>  {
> >>      unsigned i;
> >>  
> >> -    if ( !strs )
> >> -    {
> >> -        printf(" ???");
> >> -        return;
> >> -    }
> >> -
> >>      for ( i = 0; i < 32; ++i )
> >>          if ( leaf & (1u << i) )
> >>          {
> >> @@ -327,6 +324,10 @@ static void decode_featureset(const uint32_t *features,
> >>  {
> >>      unsigned int i;
> >>  
> >> +    /* If this trips, you probably need to extend leaf_info[] above. */
> >> +    BUILD_BUG_ON(ARRAY_SIZE(leaf_info) != FEATURESET_NR_ENTRIES);
> >> +    BUILD_BUG_ON(ARRAY_SIZE(feature_names) != FEATURESET_NR_ENTRIES * 32);
> >> +
> >>      printf("%-"COL_ALIGN"s        ", name);
> >>      for ( i = 0; i < length; ++i )
> >>          printf("%08x%c", features[i],
> >> @@ -338,8 +339,7 @@ static void decode_featureset(const uint32_t *features,
> >>      for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
> >>      {
> >>          printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
> >> -        if ( leaf_info[i].name )
> >> -            dump_leaf(features[i], leaf_info[i].strs);
> >> +        dump_leaf(features[i], &feature_names[i * 32]);
> >>          printf("\n");
> >>      }
> >>  }
> >> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
> >> index 79d7f5c8e1c9..601eec608983 100755
> >> --- a/xen/tools/gen-cpuid.py
> >> +++ b/xen/tools/gen-cpuid.py
> >> @@ -470,6 +470,35 @@ def write_results(state):
> >>      state.output.write(
> >>  """}
> >>  
> >> +""")
> >> +
> >> +    state.output.write(
> >> +"""
> >> +#define INIT_FEATURE_VAL_TO_NAME { \\
> >> +""")
> >> +
> >> +    for name, bit in sorted(state.values.items()):
> >> +        state.output.write(
> >> +            '    [%s] = "%s",\\\n' % (bit, name)
> >> +            )
> >> +
> >> +        # Add the other alias for 1d/e1d common bits.  64 is the difference
> >> +        # between 1d and e1d.
> >> +        if bit in state.common_1d:
> >> +            state.output.write(
> >> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
> >> +            )
> >> +
> >> +    # Pad to an exact multiple of FEATURESET_SIZE if necessary
> >> +    pad_feat = state.nr_entries * 32 - 1
> >> +    if not state.names.get(pad_feat):
> >> +        state.output.write(
> >> +            '    [%s] = NULL,\\\n' % (pad_feat, )
> > One likely stupid question, but since my understanding of Python is
> > very limited, why do you add the comma after pad_feat?  There's no
> > other parameter to print.
> 
> It's a common python gotcha with %.
> 
> >>> a = (1, 2)
> >>> "%s" % a
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: not all arguments converted during string formatting
> >>> "%s" % (a, )
> '(1, 2)'

Right, but just using:

>>> "%s" % (a)

Should still be fine?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 20 16:42:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 16:42:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726278.1130541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s965N-0003cn-9D; Mon, 20 May 2024 16:42:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726278.1130541; Mon, 20 May 2024 16:42:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s965N-0003cg-6R; Mon, 20 May 2024 16:42:05 +0000
Received: by outflank-mailman (input) for mailman id 726278;
 Mon, 20 May 2024 16:42:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+3cx=MX=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s965M-0003ca-P5
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 16:42:04 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2a1e559-16c7-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 18:42:03 +0200 (CEST)
Received: from mail.solinno.uk (localhost [127.0.0.1])
 by doppler.solinno.uk (Postfix) with ESMTP id 490198006F;
 Mon, 20 May 2024 17:42:02 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2a1e559-16c7-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1716223322;
	bh=/ZnkWziLBVnC6JxsP8Loi42TbdfzlAQwNmRa/ZBHliE=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=lqLjlwIvccCUHA345PG9ltJGu4grOHFJ9CIlY4Q9TKCgmbk7lVOddr9eoeG6QaMeV
	 /VoXVa4fSgt60WBySNqMabq1wCkFRJXDr/SeXQVcZCCBAtq+1WD198avqDRrya+xAW
	 RlKIQ+MsRqjGG0DQ9+rGHNaSeBFGr3yysTHOyp4E=
MIME-Version: 1.0
Date: Mon, 20 May 2024 17:42:02 +0100
From: Leigh Brown <leigh@solinno.co.uk>
To: Xen Devel <xen-devel@lists.xenproject.org>
Cc: Xen Devel <xen-devel@lists.xenproject.org>, Andrew Cooper
 <andrew.cooper3@citrix.com>, Jason Andryuk <jandryuk@gmail.com>
Subject: Re: xl list -l aborts with double free error with vlan patches
In-Reply-To: <0d8fbde01bedf97dcd7540ceea6c42ab@solinno.co.uk>
References: <0d8fbde01bedf97dcd7540ceea6c42ab@solinno.co.uk>
Message-ID: <98be4595598cddcd7456a77794992aa1@solinno.co.uk>
X-Sender: leigh@solinno.co.uk
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 2024-05-20 15:33, Leigh Brown wrote:
> Hello,
> 
> When running xl list -l with my VLAN patches applied, a double free 
> error is raised. I'm unable to determine why.
> 
> broken: 0cc01c603f4287233715a526b056bc20e0e97412 (HEAD) tools/xl: add 
> vlan keyword to vif option
> okay:   3bc14e4fa4b9832888710759a7dbe5f0d239f33b tools/libs/light: Add 
> vlan field to libxl_device_nic
> okay:   e27fc7d15eab79e604e8b8728778594accc23cf1 tools/xentop: Fix cpu% 
> sort order
> 
> Any suggestions appreciated...
> 
> Debug run and backtrace:
> [snip]

Thanks for Jason for pointing out my error - patch incoming.

Regards,

Leigh.


From xen-devel-bounces@lists.xenproject.org Mon May 20 16:44:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 16:44:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726281.1130550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s967O-00049F-Jb; Mon, 20 May 2024 16:44:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726281.1130550; Mon, 20 May 2024 16:44:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s967O-000498-H9; Mon, 20 May 2024 16:44:10 +0000
Received: by outflank-mailman (input) for mailman id 726281;
 Mon, 20 May 2024 16:44:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+3cx=MX=solinno.co.uk=leigh@srs-se1.protection.inumbo.net>)
 id 1s967N-000492-7e
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 16:44:09 +0000
Received: from doppler.solinno.uk (doppler.solinno.uk [81.2.106.178])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2d5d7a95-16c8-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 18:44:08 +0200 (CEST)
Received: from folly.solinno.co.uk (folly.dyn.solinno.co.uk [192.168.2.135])
 by doppler.solinno.uk (Postfix) with ESMTPSA id 0A9908006F;
 Mon, 20 May 2024 17:44:08 +0100 (BST)
Received: by folly.solinno.co.uk (Postfix, from userid 1000)
 id DCEED2018A; Mon, 20 May 2024 17:44:07 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d5d7a95-16c8-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=solinno.co.uk;
	s=mail; t=1716223448;
	bh=wrswVYnquATIjozyd9IVNy44jE4XjEZ7Z1r+nDydqro=;
	h=From:To:Cc:Subject:Date:From;
	b=gVf8ulRSpwWkA8K+H1sRfQhr4iz6PkZTRyPRPjQl54t14HtJ81BXy/U4F36Yy8ouE
	 i//OM4sw3da49PWQSDGrkcEb+QV9e9cuEAVMOXYC1nEvC3GOtiPHWcTtdmRS6fvO1I
	 yFLg/ReUm5FIDd4wl1NneEfoVa5wBxNzcFZrRkSw=
From: Leigh Brown <leigh@solinno.co.uk>
To: Xen Devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Leigh Brown <leigh@solinno.co.uk>
Subject: [PATCH] tools/libs/light: Fix nic->vlan memory allocation
Date: Mon, 20 May 2024 17:44:00 +0100
Message-Id: <20240520164400.15740-1-leigh@solinno.co.uk>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

After the following commit:
3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
xl list -l aborts with a double free error if a domain has at least
one vif defined:

  $ sudo xl list -l
  free(): double free detected in tcache 2
  Aborted

Orginally, the vlan field was called vid and was defined as an integer.
It was appropriate to call libxl__xs_read_checked() with gc passed as
the string data was copied to a different variable.  However, the final
version uses a string data type and the call should have been changed
to use NOGC instead of gc to allow that data to live past the gc
controlled lifetime, in line with the other string fields.

This patch makes the change to pass NOGC instead of gc and moves the
new code to be next to the other string fields (fixing a couple of
errant tabs along the way), as recommended by Jason.

Fixes: 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

---
 tools/libs/light/libxl_nic.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_nic.c b/tools/libs/light/libxl_nic.c
index d861e3726d..300a96a8b1 100644
--- a/tools/libs/light/libxl_nic.c
+++ b/tools/libs/light/libxl_nic.c
@@ -318,11 +318,6 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const char *libxl_path,
         nic->mtu = LIBXL_DEVICE_NIC_MTU_DEFAULT;
     }
 
-    rc = libxl__xs_read_checked(gc, XBT_NULL,
-                                GCSPRINTF("%s/vlan", libxl_path),
-				(const char **)(&nic->vlan));
-    if (rc) goto out;
-
     rc = libxl__xs_read_checked(gc, XBT_NULL,
                                 GCSPRINTF("%s/mac", libxl_path), &tmp);
     if (rc) goto out;
@@ -345,6 +340,10 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const char *libxl_path,
                                 GCSPRINTF("%s/script", libxl_path),
                                 (const char **)(&nic->script));
     if (rc) goto out;
+    rc = libxl__xs_read_checked(NOGC, XBT_NULL,
+                                GCSPRINTF("%s/vlan", libxl_path),
+                                (const char **)(&nic->vlan));
+    if (rc) goto out;
     rc = libxl__xs_read_checked(NOGC, XBT_NULL,
                                 GCSPRINTF("%s/forwarddev", libxl_path),
                                 (const char **)(&nic->coloft_forwarddev));
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Mon May 20 17:08:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 17:08:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726291.1130560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s96Uh-0007ic-FG; Mon, 20 May 2024 17:08:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726291.1130560; Mon, 20 May 2024 17:08:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s96Uh-0007iV-Cj; Mon, 20 May 2024 17:08:15 +0000
Received: by outflank-mailman (input) for mailman id 726291;
 Mon, 20 May 2024 17:08:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s96Ug-0007iF-1r
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 17:08:14 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 892e8f18-16cb-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 19:08:11 +0200 (CEST)
Received: from SN6PR08CA0031.namprd08.prod.outlook.com (2603:10b6:805:66::44)
 by DM3PR12MB9390.namprd12.prod.outlook.com (2603:10b6:0:42::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 17:08:06 +0000
Received: from SN1PEPF00036F3D.namprd05.prod.outlook.com
 (2603:10b6:805:66:cafe::43) by SN6PR08CA0031.outlook.office365.com
 (2603:10b6:805:66::44) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 17:08:06 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00036F3D.mail.protection.outlook.com (10.167.248.21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 17:08:05 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 12:08:04 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 12:08:03 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 892e8f18-16cb-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aPx+vZqd5fh6a+fn0pMfmqQlSjbXCirP6jgTFM6CAb0zwsFNWub957mVPVFyDl10G4f040e5Rtghhjjmjtm1L1KMKS/l0ZXITuoO4zO4JDRXvn9UfSCE24A28DZWH3qHNm0F0CBitMxBDE8IhV0NM9gXzcJNXMeRL7F/W5PC8baebBkiG3R+3WFKWMNWNOEkpC5IZzZtR2+m89jSq5Bvk9gNxEe0Fzzgzh4LWMsl+hkV2dLE5y4gyF1lk9arHiVrCvQpEYnX58APunEhR+PnuM/NxyxJCOUQBFbL7FtsW41Wn9Aspyo+iyMRHp9h4iGyWqQRItTDTDoeZHjXwyUuLQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UCxy5wlf43fGF5cch2iu/1Mi+Qa0MXpy3LB2q4FujXo=;
 b=Amejcxp+2Pd6iJnXk6vW85O+WC/yOJbIl4EIs39//dSGbXM1E/TWrWJLnFHORUY9nBSq5ehRO4Isq4whTCuzouYiQVzVaYC6+AK/0uUDxr4EJSZYwpdvpPk6XKV5T04C0pHtxDhm5HD3YFG7SeXo5bBgJ384dJdGna9a1O5noyhq1JM6+5PhtLIuafqAzsHDRuwQkCnFQ/LNytjgsfPzCjvUGNQLplTH2ktX0gdtd6nG+8DD1CFp+SteD9J4bYX9j2FU8fBFPRnXdmjNbHm5y7ru74WLWE6FqMqfOVlMm6NI1o0YPQBl2e08dQfvyTgJRT8nJBzbo8pOfFAONHukbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=solinno.co.uk smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UCxy5wlf43fGF5cch2iu/1Mi+Qa0MXpy3LB2q4FujXo=;
 b=VaHAmdfGg6ftujwiri4cyVumXSLOPxGQaCFnEgvt+u5W7iOQjE9244YkDu+AaS1YOYtPmEWy+URRqxg9R6mXFpz99+oMg5vckRUbxgATk3S8oT3CGxglPhBUOudHyFTeTD8bZDcAmR1Ripl969T2aLR2l7eSPmVTVSB9jECI49k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <c600e5e8-d169-417c-bc02-d33e84dca0fb@amd.com>
Date: Mon, 20 May 2024 13:08:03 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/libs/light: Fix nic->vlan memory allocation
To: Leigh Brown <leigh@solinno.co.uk>, Xen Devel
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jason Andryuk
	<jandryuk@gmail.com>
References: <20240520164400.15740-1-leigh@solinno.co.uk>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240520164400.15740-1-leigh@solinno.co.uk>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00036F3D:EE_|DM3PR12MB9390:EE_
X-MS-Office365-Filtering-Correlation-Id: 5ea4b879-c83a-45bb-6799-08dc78ef6a4f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?b3cwZGlpSDJEbGorVlBZeW1Gc3hTZEowQVZhV2xaL2NDT0VZZldqYVNXT3pO?=
 =?utf-8?B?SklhdnRqSm1CVDQ5MStLM1pTMTkxRjUzNGJ2cExRR1VTc0M0WnpQQWp0OENK?=
 =?utf-8?B?RkNibnZYWWFJQjNYdVpKOGxMdGFkR2JzcThoRXlHZVJ1QkRVNDUyMTNTSzdP?=
 =?utf-8?B?TkNEZmNZaXhyWVV3Zko1UG9xeW1LRXVnNEhSdUNmTWQvSDRINWV3YlNWZzla?=
 =?utf-8?B?Y3VxTkVXZFBGY2pVQm5ydVFDODV4M0xlMG1OcTY5YXBPbCt2TjdwdnR1OG1i?=
 =?utf-8?B?bENCcGpUelNPSWh5Nm5CSStDOWdCS3dtbU1sZUpaenJSZTRDMS9TYjJKTkZ5?=
 =?utf-8?B?N3BKSDc5Q3d6a044RU1kQ29uVjhLUzN4aFdMeDlpY1dKTkw3WUdEYzNJeTNq?=
 =?utf-8?B?ZXZBcFQwR0ZSOXpUM3ZTT2ZoT1NiTWpiTEpud2J6MVlqM0xpeUVsREp3a2Zw?=
 =?utf-8?B?Q1pFdVI4UTk1Rm1rd3VQMTFmeFBUSVFOajJCeDZqLzhQOFV2Q2RRbmU3c0FH?=
 =?utf-8?B?K2xDY1hUSi90TFh4eDNNeWdMNjRjSHMyeVk5b3I2MG83Wlh5YS9FczM2ZS8r?=
 =?utf-8?B?alIxeXlZS2F5c1VaMGZJV1d1ZllVVzRZcllpV0lxS2Y0R0lDRXl0NmtTRzFz?=
 =?utf-8?B?dldFejZCRHhwd0JZYzJIZVpSQnhWbFhhbU9uK0xmMjdaRktvaHo0MkVpTCt1?=
 =?utf-8?B?VjljSjg1QWtlbW9SZmpvc2pZM0tZUDhNVnBZN0FOTWc0OWZNVU4zb2I5SkJZ?=
 =?utf-8?B?NXp0b0xEcFprSC9iTmRpc0VuZUV2MzRDQjRkaUFTamxkK1pBdlJzOExXaGpW?=
 =?utf-8?B?a0NKZWNRem5hSkE1ajIyQXRKT3FZSitOU2NmalhJVVdiMzZvU2VmNVhXbkJz?=
 =?utf-8?B?YkI1UkpKSjFLNmFtQ1d3MituTDVjYjFIWUExL3diQ21sd3JaTm1YWnR3NERm?=
 =?utf-8?B?RGhTZWhVaXd2c1dSVnZDS0hCc0NpVXBvbWVwNXZQM0VsbDJETGJNK1Z6YnVQ?=
 =?utf-8?B?RG8vWktHRFZ0TkNteVhQcWNDdmorcVNxV0dianhxb1kwd1czbmlEcjZpSktQ?=
 =?utf-8?B?TnNvWGtFNU9ic0Nxa2hIc3JLSUowS0EyeGRLT0FUYW9mSUlxUndlY2lKa2E3?=
 =?utf-8?B?bWJoSm8rcjJzUk9FKytDNjQvRzdIdndpbFhITEU1VFJuOVF4ZXBzYVNvZXl5?=
 =?utf-8?B?NWlBYm0rQnB0bTR3SVd6ZW1lSjBSYXZQeUlEUEZhTm5sWlRDRGdwY3N2amRN?=
 =?utf-8?B?ODFJYU9PSU5LbGNRS3IrUFVwNCsvQUlNV3RYaWMybkphNEhrTUt2VWVCQXNE?=
 =?utf-8?B?TGFSTFVMaUtJRi9wS2RkOHlMNFdUdjhwVWdrMXBGSmR3cG91NFFvaGtKc0FL?=
 =?utf-8?B?RHRhM0tMR1cyei9WT2xNejdyYVRaalRTM3dEVXUwMTFFRzlpMEtCQzdnMVI5?=
 =?utf-8?B?ZkRlUVpyajIxZDY2YmxpejVIWFFoZWFSM01USzNGVS9rOGEzbVNFbXVycDBE?=
 =?utf-8?B?Y25YcmdML0pqWFV6Sm1uMkp3SVRwNGQyRmV2a2pBQVpyb2FpdmxMTEdRUEoz?=
 =?utf-8?B?VGhDSVJ4clN4NTE3MEFOdXdtak1aeTVrT1hsaUtzL1NOa1VUZCtEUHZaNEpr?=
 =?utf-8?B?L203bndmMnRFSzRNOC9Gcmo3amJEVDZMeVNWS21WQ2RxSFlCU0gwQVBFUG5o?=
 =?utf-8?B?QmtHaGxweXVvcWhXMzZ5VDhzVzg5VjhaMUdQbk5DdlA4Q2ZuMVVIZm10VHpH?=
 =?utf-8?B?UnZKakJvTTl0ZXBYeWszUy8xNTlBQXNjc01oazdzOUF1WDNLUmNYTXlYdGo5?=
 =?utf-8?B?MUhLLzFjZ0dRUjl3TDNiRlBVaHBqVU9HeE95aGhYTXlTZU5JMmEzRTlLaEY2?=
 =?utf-8?Q?caVsl6x8ELPEq?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 17:08:05.5248
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ea4b879-c83a-45bb-6799-08dc78ef6a4f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00036F3D.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9390

On 2024-05-20 12:44, Leigh Brown wrote:
> After the following commit:
> 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
> xl list -l aborts with a double free error if a domain has at least
> one vif defined:
> 
>    $ sudo xl list -l
>    free(): double free detected in tcache 2
>    Aborted
> 
> Orginally, the vlan field was called vid and was defined as an integer.
> It was appropriate to call libxl__xs_read_checked() with gc passed as
> the string data was copied to a different variable.  However, the final
> version uses a string data type and the call should have been changed
> to use NOGC instead of gc to allow that data to live past the gc
> controlled lifetime, in line with the other string fields.
> 
> This patch makes the change to pass NOGC instead of gc and moves the
> new code to be next to the other string fields (fixing a couple of
> errant tabs along the way), as recommended by Jason.
> 
> Fixes: 3bc14e4fa4b9 ("tools/libs/light: Add vlan field to libxl_device_nic")
> Signed-off-by: Leigh Brown <leigh@solinno.co.uk>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 17:38:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 17:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726299.1130571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s96xR-0003s5-Q8; Mon, 20 May 2024 17:37:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726299.1130571; Mon, 20 May 2024 17:37:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s96xR-0003ry-My; Mon, 20 May 2024 17:37:57 +0000
Received: by outflank-mailman (input) for mailman id 726299;
 Mon, 20 May 2024 17:37:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gPfm=MX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s96xP-0003rs-T3
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 17:37:55 +0000
Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com
 [2607:f8b0:4864:20::f36])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aff1e87e-16cf-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 19:37:54 +0200 (CEST)
Received: by mail-qv1-xf36.google.com with SMTP id
 6a1803df08f44-69b6d36b71cso11412536d6.3
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 10:37:54 -0700 (PDT)
Received: from [10.80.67.140] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e1583185bsm110426471cf.64.2024.05.20.10.37.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 10:37:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aff1e87e-16cf-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716226673; x=1716831473; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=NhBRA5aHy07gR2CD1mjlnPwsLGPOdFrWBuhG0u80I+g=;
        b=qUW2cbp2urhWCxyFAJA9ai9s1Idmu7BzvYjXnrAUT760+zarPThFxEAAyoYea6MyUE
         9yijpynZ5NEWSGBvsZU+VVwFEHjyyYV+BNgS+m+6DqHBAdzDG2fZht+FuQdH50GPNXcL
         smaAoXfiPBylPzWugYa0EnIEtFOV12GikRuZo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716226673; x=1716831473;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NhBRA5aHy07gR2CD1mjlnPwsLGPOdFrWBuhG0u80I+g=;
        b=RYkqm0M3UukWPDHsbcz00FdDLngDF7IHg0xchifFBCf1JousBm8V0Mzmp5XsqKulHi
         Mg3iMyn74kTX7IgaeusvCF8FyYHGJz4fgQXE7HzJKHuakn5sSP7nqikNKypM8upQ1mSq
         B40x5gTvPTBeDtFVTF4Ekyt/qZ6fxYw3Ru3r6nDcK2vMgKojo2vt/ud8hE8FrWzaGLsJ
         UwHDHsTII8JIeqm5kQY+GFpyy5f/PfcjtjRP8AGing910/oJYIBtEeQefKCifJumaHDP
         O3THqitsVdrk6q9HKXYKycjWSG7yTt1mOsdIipXDw2O6yQP2A69hhvY6nBJ49hC5HNpc
         qjIQ==
X-Gm-Message-State: AOJu0YyzTE/6V1UMqW5Obk4BpsGThLK7I/jE5ruG2pSQHOr8wou7/jIJ
	orHHxBfvzu2A5jQ+032F4HYAwlnTlsu4UginvoMp5sP/ReEZ5XG2Cf1Sn9uByxM=
X-Google-Smtp-Source: AGHT+IEHdOtA92CH5/utYKNNSfoyTmKrRFgNo1WC5LSmQYC5/nyNdlS8OifaRwHyFiCKuu54j93WIg==
X-Received: by 2002:a05:6214:5d8d:b0:6aa:834:db10 with SMTP id 6a1803df08f44-6aa0834dcaamr46508486d6.5.1716226673193;
        Mon, 20 May 2024 10:37:53 -0700 (PDT)
Message-ID: <37ffccd2-41f2-4574-bf65-41ad4445347a@citrix.com>
Date: Mon, 20 May 2024 18:37:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3.5 3/4] tools/xen-cpuid: Use automatically generated
 feature names
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>
References: <20240510224002.2324578-4-andrew.cooper3@citrix.com>
 <20240520143359.3376849-1-andrew.cooper3@citrix.com>
 <ZktnJ7XNxhdLOxdk@macbook> <32c1b1b0-6372-4c15-bf0a-cb946df236f5@citrix.com>
 <Zkt6eHDYqNCOdgud@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zkt6eHDYqNCOdgud@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20/05/2024 5:29 pm, Roger Pau Monné wrote:
> On Mon, May 20, 2024 at 04:20:37PM +0100, Andrew Cooper wrote:
>> On 20/05/2024 4:07 pm, Roger Pau Monné wrote:
>>> On Mon, May 20, 2024 at 03:33:59PM +0100, Andrew Cooper wrote:
>>>> From: Roger Pau Monné <roger.pau@citrix.com>
>>>>
>>>> Have gen-cpuid.py write out INIT_FEATURE_VAL_TO_NAME, derived from the same
>>>> data source as INIT_FEATURE_NAME_TO_VAL, although both aliases of common_1d
>>>> are needed.
>>>>
>>>> In xen-cpuid.c, sanity check at build time that leaf_info[] and
>>>> feature_names[] are of sensible length.
>>>>
>>>> As dump_leaf() rendered missing names as numbers, always dump leaves even if
>>>> we don't have the leaf name.  This conversion was argumably missed in commit
>>>> 59afdb8a81d6 ("tools/misc: Tweak reserved bit handling for xen-cpuid").
>>>>
>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
>> Thanks.
>>
>>> Just one question below.
>>>
>>>> ---
>>>> CC: Jan Beulich <JBeulich@suse.com>
>>>> CC: Roger Pau Monné <roger.pau@citrix.com>
>>>>
>>>> Differences in names are:
>>>>
>>>>  sysenter    -> sep
>>>>  tm          -> tm1
>>>>  ds-cpl      -> dscpl
>>>>  est         -> eist
>>>>  sse41       -> sse4-1
>>>>  sse42       -> sse4-2
>>>>  movebe      -> movbe
>>>>  tsc-dl      -> tsc-deadline
>>>>  rdrnd       -> rdrand
>>>>  hyper       -> hypervisor
>>>>  mmx+        -> mmext
>>>>  fxsr+       -> ffxsr
>>>>  pg1g        -> page1gb
>>>>  3dnow+      -> 3dnowext
>>>>  cmp         -> cmp-legacy
>>>>  cr8d        -> cr8-legacy
>>>>  lzcnt       -> abm
>>>>  msse        -> misalignsse
>>>>  3dnowpf     -> 3dnowprefetch
>>>>  nodeid      -> nodeid-msr
>>>>  dbx         -> dbext
>>>>  tsc-adj     -> tsc-adjust
>>>>  fdp-exn     -> fdp-excp-only
>>>>  deffp       -> no-fpu-sel
>>>>  <24>        -> bld
>>>>  ppin        -> amd-ppin
>>>>  lfence+     -> lfence-dispatch
>>>>  ppin        -> intel-ppin
>>>>  energy-ctrl -> energy-filtering
>>>>
>>>> Apparently BLD missed the update to xen-cpuid.c.  It appears to be the only
>>>> one.  Several of the + names would be nice to keep as were, but doing so isn't
>>>> nice in gen-cpuid.  Any changes would alter the {dom0-}cpuid= cmdline options,
>>>> but we intentionally don't list them, so I'm not worried.
>>>>
>>>> Thoughts?
>>>>
>>>> v3:
>>>>  * Rework somewhat.
>>>>  * Insert aliases of common_1d.
>>>>
>>>> v4:
>>>>  * Pad at the gen stage.  I don't like this, but I'm clearly outvoted on the matter.
>>>> ---
>>>>  tools/misc/xen-cpuid.c | 16 ++++++++--------
>>>>  xen/tools/gen-cpuid.py | 29 +++++++++++++++++++++++++++++
>>>>  2 files changed, 37 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>>>> index 6ee835b22949..51009683da1b 100644
>>>> --- a/tools/misc/xen-cpuid.c
>>>> +++ b/tools/misc/xen-cpuid.c
>>>> @@ -11,6 +11,7 @@
>>>>  #include <xenguest.h>
>>>>  
>>>>  #include <xen-tools/common-macros.h>
>>>> +#include <xen/lib/x86/cpuid-autogen.h>
>>>>  
>>>>  static uint32_t nr_features;
>>>>  
>>>> @@ -291,6 +292,8 @@ static const struct {
>>>>  
>>>>  #define COL_ALIGN "24"
>>>>  
>>>> +static const char *const feature_names[] = INIT_FEATURE_VAL_TO_NAME;
>>>> +
>>>>  static const char *const fs_names[] = {
>>>>      [XEN_SYSCTL_cpu_featureset_raw]     = "Raw",
>>>>      [XEN_SYSCTL_cpu_featureset_host]    = "Host",
>>>> @@ -304,12 +307,6 @@ static void dump_leaf(uint32_t leaf, const char *const *strs)
>>>>  {
>>>>      unsigned i;
>>>>  
>>>> -    if ( !strs )
>>>> -    {
>>>> -        printf(" ???");
>>>> -        return;
>>>> -    }
>>>> -
>>>>      for ( i = 0; i < 32; ++i )
>>>>          if ( leaf & (1u << i) )
>>>>          {
>>>> @@ -327,6 +324,10 @@ static void decode_featureset(const uint32_t *features,
>>>>  {
>>>>      unsigned int i;
>>>>  
>>>> +    /* If this trips, you probably need to extend leaf_info[] above. */
>>>> +    BUILD_BUG_ON(ARRAY_SIZE(leaf_info) != FEATURESET_NR_ENTRIES);
>>>> +    BUILD_BUG_ON(ARRAY_SIZE(feature_names) != FEATURESET_NR_ENTRIES * 32);
>>>> +
>>>>      printf("%-"COL_ALIGN"s        ", name);
>>>>      for ( i = 0; i < length; ++i )
>>>>          printf("%08x%c", features[i],
>>>> @@ -338,8 +339,7 @@ static void decode_featureset(const uint32_t *features,
>>>>      for ( i = 0; i < length && i < ARRAY_SIZE(leaf_info); ++i )
>>>>      {
>>>>          printf("  [%02u] %-"COL_ALIGN"s", i, leaf_info[i].name ?: "<UNKNOWN>");
>>>> -        if ( leaf_info[i].name )
>>>> -            dump_leaf(features[i], leaf_info[i].strs);
>>>> +        dump_leaf(features[i], &feature_names[i * 32]);
>>>>          printf("\n");
>>>>      }
>>>>  }
>>>> diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
>>>> index 79d7f5c8e1c9..601eec608983 100755
>>>> --- a/xen/tools/gen-cpuid.py
>>>> +++ b/xen/tools/gen-cpuid.py
>>>> @@ -470,6 +470,35 @@ def write_results(state):
>>>>      state.output.write(
>>>>  """}
>>>>  
>>>> +""")
>>>> +
>>>> +    state.output.write(
>>>> +"""
>>>> +#define INIT_FEATURE_VAL_TO_NAME { \\
>>>> +""")
>>>> +
>>>> +    for name, bit in sorted(state.values.items()):
>>>> +        state.output.write(
>>>> +            '    [%s] = "%s",\\\n' % (bit, name)
>>>> +            )
>>>> +
>>>> +        # Add the other alias for 1d/e1d common bits.  64 is the difference
>>>> +        # between 1d and e1d.
>>>> +        if bit in state.common_1d:
>>>> +            state.output.write(
>>>> +                '    [%s] = "%s",\\\n' % (64 + bit, name)
>>>> +            )
>>>> +
>>>> +    # Pad to an exact multiple of FEATURESET_SIZE if necessary
>>>> +    pad_feat = state.nr_entries * 32 - 1
>>>> +    if not state.names.get(pad_feat):
>>>> +        state.output.write(
>>>> +            '    [%s] = NULL,\\\n' % (pad_feat, )
>>> One likely stupid question, but since my understanding of Python is
>>> very limited, why do you add the comma after pad_feat?  There's no
>>> other parameter to print.
>> It's a common python gotcha with %.
>>
>>>>> a = (1, 2)
>>>>> "%s" % a
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: not all arguments converted during string formatting
>>>>> "%s" % (a, )
>> '(1, 2)'
> Right, but just using:
>
>>>> "%s" % (a)
> Should still be fine?

(a) is just a set of brackets around an expression.

(a, ) is a 1-element tuple containing a as it's only element.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon May 20 17:45:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 17:45:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726305.1130582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9750-0005lR-O1; Mon, 20 May 2024 17:45:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726305.1130582; Mon, 20 May 2024 17:45:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9750-0005lK-JT; Mon, 20 May 2024 17:45:46 +0000
Received: by outflank-mailman (input) for mailman id 726305;
 Mon, 20 May 2024 17:45:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s974z-0005lA-LV; Mon, 20 May 2024 17:45:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s974z-0007eu-H4; Mon, 20 May 2024 17:45:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s974z-00068Y-85; Mon, 20 May 2024 17:45:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s974z-00057P-7W; Mon, 20 May 2024 17:45:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Zj0EgvfiTsn1yrpPLcXKMpVDAKwz7Je0ge4SGXisqNQ=; b=KkcZVHIkL73U2CtWOOKIxWYDUu
	LreZ8Ldf9IfcowiEA756kuweAkLoPXvfJQOIfWAtFmdGX3CSlafALLGESNc0l+7R8bvNm4hOESMHf
	oBP+UpDBHISWMHdUYuQUgYKJyftg3XuBxr44baPJJmI7QYX0JJUQLYORvpueuUkG/0rc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186046-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186046: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=54aa34fc89151c943550541e0af47664fb6e8676
X-Osstest-Versions-That:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 May 2024 17:45:45 +0000

flight 186046 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186046/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  54aa34fc89151c943550541e0af47664fb6e8676
baseline version:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca

Last test of basis   186024  2024-05-17 04:02:10 Z    3 days
Testing same since   186046  2024-05-20 14:00:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Henry Wang <xin.wang2@amd.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   21611c6870..54aa34fc89  54aa34fc89151c943550541e0af47664fb6e8676 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon May 20 18:24:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 18:24:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726319.1130591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s97g3-0002pt-C6; Mon, 20 May 2024 18:24:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726319.1130591; Mon, 20 May 2024 18:24:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s97g3-0002pm-8d; Mon, 20 May 2024 18:24:03 +0000
Received: by outflank-mailman (input) for mailman id 726319;
 Mon, 20 May 2024 18:24:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s97g2-0002pg-5F
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 18:24:02 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 200fb62f-16d6-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 20:23:59 +0200 (CEST)
Received: from BN1PR10CA0018.namprd10.prod.outlook.com (2603:10b6:408:e0::23)
 by MN0PR12MB6002.namprd12.prod.outlook.com (2603:10b6:208:37e::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 18:23:56 +0000
Received: from BN2PEPF00004FBE.namprd04.prod.outlook.com
 (2603:10b6:408:e0:cafe::c0) by BN1PR10CA0018.outlook.office365.com
 (2603:10b6:408:e0::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 18:23:56 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN2PEPF00004FBE.mail.protection.outlook.com (10.167.243.184) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 18:23:55 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 13:23:55 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 13:23:55 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 13:23:54 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 200fb62f-16d6-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fOS6Br0hJH6xewZ16FFHrA1Sg7hvksSKWcWGf43XIM1s/tUxepTNB7qfAEn5f7RnPB6tu6KSnWFAzSLMw6G/w9glGEN5H4UI47UfQouZ73yXszAMhJ/69h/taOPAlnaKdRjs2Ppz0b7flpkkxUHPF1V85bOVakHgbp9ZIVb8da03ywYqRWaIlH/ozTNumJo8xh8WcSIeVttH3pix2WIeunUN+IQlFPW/Zu2z1V0S62XPulmpe+Zv0ZuMPc6NmSBs5t6BeDUb9oL/S0RkDB/3CEBwkb2W8e978kCytr7GczmZD2S02EwMhL8H1vLjljXsWilP+YGW3SqEID3+TVMncQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uTrotKv/YbafHCsyE2adcCRyYL0l3sH9ww95U5n1sEQ=;
 b=kgpEsC5MxNU9gui0wLsH8R7WD9le3KIfGEcnXpgQaI+Wsyr2r/4cIPP21Vm1M9xHF/JPM3fqR/2BI7dNnIDLDhRkLglwkHHrwkZTW/K8w+XyV3ISEDs7fcGhUIdL8NF1hce1rgnaXD/DlVh0DX32vbxBfuuYt3tH3000T3i7auUNlZz4ngkJwusVGkVpn2RtjfP7Z0YSoKpJgYPdGsO4qvMw1URadaRxGxC6x8NGW9DKKcsdkcftdfhqFq3tP9LmO2oz+0cwj4p9Z5yGkVtTHy1CaW6fm2/yPYH31o+UR9lWrG7TXbbaMDPf9DSN3AMzDVu29VFjKVpOtMngdADLng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uTrotKv/YbafHCsyE2adcCRyYL0l3sH9ww95U5n1sEQ=;
 b=suoJPD1bEoObGXce+L/0C6jW5KKX6c48eBhugSh2fxMqqYgkQlaK/NyudZj8ni5u/NhdVPo807aDGpxFEKBFlJxj2zbPCyUyaxa/uwuhQzGHqWd+odADMRhoMfY88QObaQ6pTYxPOt+pqjjl4mgs1xzrobrdYoH71xNVy+Q3YVM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <46e1d5f3-20e5-47ea-a1cd-10d120211f0d@amd.com>
Date: Mon, 20 May 2024 14:23:52 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/4] tools/init-dom0less: Avoid hardcoding
 GUEST_MAGIC_BASE
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Alec Kwapis
	<alec.kwapis@medtronic.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-4-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240517032156.1490515-4-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF00004FBE:EE_|MN0PR12MB6002:EE_
X-MS-Office365-Filtering-Correlation-Id: f61064fd-8e72-4b9b-e1af-08dc78fa0279
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eE00UmpiMUpkaHkwVUJsTG1jcGhGMUhoUER2cVAydUs2R1pHZWdaNkdhd1Zv?=
 =?utf-8?B?NS8xaW12VW16T1QwKzVwbGVzVS90SHhpTm9jaVMzOTBnaXd6dE5aMEp4TzBJ?=
 =?utf-8?B?MUpmMGZkcTg4aXF5VEVyMFpDaGI5eVg2L01HeGJkRExyTDVFbld0RnV1N0Z2?=
 =?utf-8?B?clRpeENtVkRzdmNPQ2xiRlVXMGg4M3pLdkVGanV4bTJ1NWt0aUlydzdRbW5D?=
 =?utf-8?B?czdvQVE5YkVWOVlCaW92ZkhJYmlGZFp0YlU5V1YwVmMxK3RXQmUwS3hLM080?=
 =?utf-8?B?OXpBaE40SE9GREowbVJxVG1QNHBVMzE0dWtQdXFydFNmclcxMmpqMHM1WGE1?=
 =?utf-8?B?aExXcUR4ZWp3dEpibldHaTJTME93WVIvS2hKY3I4QVgrZFVHeHdET2kzL2dt?=
 =?utf-8?B?NXNKVlgvaUdNdDdCeTBIbXFhVERvSVMxSFZCMWZHNWl4WkZxNWFKQ2V2a1o1?=
 =?utf-8?B?bC9XMU1CellXRS9yWm9sWmN0a3ZvbjBaWDBBT2JoeTQ0R05tNHgxU045RlRr?=
 =?utf-8?B?aU1nYVExODJYbmNVZ3ZYQnVIZ0N5VDN1ckRrTkRYc2tqMS9YN09FTFBLMHdE?=
 =?utf-8?B?YzdKMW82OExlRXJlQ21FdGVWVTYrQ0VvSzRKcTRBS1hyQlFKNUFHSjVJMGZL?=
 =?utf-8?B?UjNQaEFDQldJYmZSMkl2RW05bU1aUk9lSDF6V0pYaTI0eG13eTZIRWQ4S2ZY?=
 =?utf-8?B?UXZIZkhHY3VuR2Yya0ZPNENuQnRvZFpWYTRUY21DU0JwMUdON2RNdlNnNFNI?=
 =?utf-8?B?WWhObkc1UllQZGNkQWxDWnN2czd6bUNVU2RSTjR5b0VrUFBMRnplTU94dEVN?=
 =?utf-8?B?TFQ1NnZXd0dMRi9Cazg4clZ5S1ZBb3JCS2xrUDFsN05wWUpwZlp5TW5WSzRv?=
 =?utf-8?B?UjZuUGVwdm5IM0psc0YvTm90cWxRRjZ6WGhVMzZoY0U2NlByeG9DWHNDWW9I?=
 =?utf-8?B?S210YkpGdFdZRnRNbDJjVzRoRnJZc3Fmd1EzMWNxLy9xZ0VIUWowMFB4cjBH?=
 =?utf-8?B?K0dPVFNsZXI5VzVzY0hDNzdyZVZjcjgrcHZNbjV2d0VRQzNMNjlZTnd3TlFj?=
 =?utf-8?B?RUhGalM5WW5vT1JoQjhhQW1ueURnVDNoZVpmSHU4UWNNSUk4ZTQrcG9GcUFx?=
 =?utf-8?B?Q1VzNnJxYyt0eHh0aTJiaDVhK1JKN0lNVnJUU3pqQmtrMGg5bnVkRDFvMG1Q?=
 =?utf-8?B?VkMwankrRTZ0QVNXSlVyWGUxWkNicWwyaFNrSEl6UUNXWUF0ZGdMT3hqRVQ5?=
 =?utf-8?B?bWpNaWRHcUJZM0E0RU9MRlp1SzZ2R2Y0d0JDK3phdCtuR3YzZGhZNjluNUZR?=
 =?utf-8?B?aGdWQkc5Ykt0QllmOGJ1U0hFNnMwcDA3Y01oNnJlWW40MzJzTVRWakFnRC94?=
 =?utf-8?B?M3lNTlA2ZjRtVEZmNmJGbkFqY2liTHRhbXhRM2FROVQ3WnE3R1UvcGZGS3h3?=
 =?utf-8?B?UkVUckpQbzVxT0J0UFQrK1poUlE2clREaXV1L0ZIK1NLN1hBT0hlUDlOQUtL?=
 =?utf-8?B?WlJZcVRvMDlUTW5QZEFRZzZUMTZUQ0FzeFh5M3ZtbDRKNmxrOVljWmQ3UzRt?=
 =?utf-8?B?UHRsbElJamZ4S3ZSVGY0WXJlVUM3bHBnZzdjZjJmQlY1QjgrUnlHT016Ynp0?=
 =?utf-8?B?MzFqclY5TS9CN3hQSmtEL1hXeDFHc0F5MHVCMjZwR2Z2YTFSSktwcC9FS29F?=
 =?utf-8?B?UnlraFFydit4VmIwSExXbGhrUjNWV3UrMjFDbE8rbmwvSytFNjJmSlU1UHBh?=
 =?utf-8?B?VnhnZ2x3ckhZcHcwcnVXZW1GYnhFOVpqd3YweFhFUHY0QUNlSkxyc0YrVmx6?=
 =?utf-8?B?aEMzb1RWcmNsMVZIb0NvNUFHWXNsaGVWTnd0RTlvRDZ0M1FwR2JyZHdCU2lm?=
 =?utf-8?Q?6aCsgilwSWEfi?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 18:23:55.7998
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f61064fd-8e72-4b9b-e1af-08dc78fa0279
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF00004FBE.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6002

On 2024-05-16 23:21, Henry Wang wrote:
> Currently the GUEST_MAGIC_BASE in the init-dom0less application is
> hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
> DomUs.
> 
> Since the guest magic region allocation from init-dom0less is for
> XenStore, and the XenStore page is now allocated from the hypervisor,
> instead of hardcoding the guest magic pages region, use
> xc_hvm_param_get() to get the XenStore page PFN. Rename alloc_xs_page()
> to get_xs_page() to reflect the changes.
> 
> With this change, some existing code is not needed anymore, including:
> (1) The definition of the XenStore page offset.
> (2) Call to xc_domain_setmaxmem() and xc_clear_domain_page() as we
>      don't need to set the max mem and clear the page anymore.
> (3) Foreign mapping of the XenStore page, setting of XenStore interface
>      status and HVM_PARAM_STORE_PFN from init-dom0less, as they are set
>      by the hypervisor.
> 
> Take the opportunity to do some coding style improvements when possible.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 18:48:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 18:48:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726330.1130601 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s983b-0005pF-7U; Mon, 20 May 2024 18:48:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726330.1130601; Mon, 20 May 2024 18:48:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s983b-0005p8-41; Mon, 20 May 2024 18:48:23 +0000
Received: by outflank-mailman (input) for mailman id 726330;
 Mon, 20 May 2024 18:48:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s983Z-0005om-Nh
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 18:48:21 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 85907d1b-16d9-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 20:48:19 +0200 (CEST)
Received: from CH0PR03CA0378.namprd03.prod.outlook.com (2603:10b6:610:119::33)
 by LV8PR12MB9262.namprd12.prod.outlook.com (2603:10b6:408:1e7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 18:48:13 +0000
Received: from DS3PEPF000099D7.namprd04.prod.outlook.com
 (2603:10b6:610:119:cafe::e7) by CH0PR03CA0378.outlook.office365.com
 (2603:10b6:610:119::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 18:48:13 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF000099D7.mail.protection.outlook.com (10.167.17.8) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 18:48:13 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 13:48:12 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 13:48:12 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85907d1b-16d9-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DCf06PXOAa+LnBg6IusSn7sG7tXNuZ2uTT/PkOnAGZa5ix7Rj+iYTsIw/0+Jwty3iV9DOXgI4HVGnp8GnuL3ChPO8m3rCzpL1lnpzeJpDj4hZtY5F0CgP7pbvEuhqo0RGHNXauQ121Ae8lQtKecuvErMGV9tB2oZnwMx8LwBLRfpBwNVNYlfbnkYIEwlvG3t/fK7CNxOvHIYIIUZwTG3Q2X7sYX03QAENmKR8DxaMGreLDzNkYtNf0JUW8uElMPDu4Yp/OTnRvsTtcoobYjZvfeuX3dqnUeS1R4+VrxFdUkmQVB9vZyTeEQkMWBs3DCvwDcwOrswK2ffPYpDpI1sEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Ypj/b6oBZ1LDZx4wW75mCIvDToozVqoLBcL8se35bug=;
 b=SH08hLr6uHEFPuIqQ+zblX9NA2Qcl3F2PhwXZJ1nGT8SSY0hBIxr5h2se8FQhoxQJnyGa5FvydmS1wilewDa2oMxZ9MOutKD7P6i8gMHTp4JeAhQjm419pdnafzEl4zA4rj9FP3K+9y5t2a4yf9vCNdtBvcQ0dtssKReFT2vMtqwkJSFkYe59K39nImC5vN0BjM891Rlb2MJoafKrat5CQfKH323xreEINfZipTJByzJK1FrHFMnheEFY6G3VJ5PSJXYlA6pwE1qWyeXh9budGSTQr+390QKT8pEVs7fcVSviw6OAuYvFZF6uioRccbQSwytd+2BElih3pNefWYZ8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ypj/b6oBZ1LDZx4wW75mCIvDToozVqoLBcL8se35bug=;
 b=pI1BruSTp8awO42YVsmGITPV0Et5Co34Cvt/p5KO/AZ6+Jy3V1UGuGht+vdwTKfD6SK3vTUfARgOt6fi3crNHdqUmh2asKCL80Wv52o85jpI3E2X27+OEPBOiNsNM7mu1zlwZ37BLbdKDIIcLC+AfNWR3zBvCbg2mwQd9lJDyIg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <05fc6a62-4e88-4771-8662-368cb8b63721@amd.com>
Date: Mon, 20 May 2024 14:48:11 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/8] tools/xl: Correct the help information and exit
 code of the dt-overlay command
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Anthony PERARD
	<anthony.perard@cloud.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-3-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240516100330.1433265-3-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099D7:EE_|LV8PR12MB9262:EE_
X-MS-Office365-Filtering-Correlation-Id: d6e51242-4977-4c26-a7b6-08dc78fd6722
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?by9Pb0w3anJLTUhsZFJQUVVHbjZ2Vk85bDBMTEhjblQxcUNaWGNsaUcxdVNV?=
 =?utf-8?B?NzlpNEZleTNCT3puRG03ekx3enJmTnJNbnYwSlBXSVh1azYrQ0I4SFBiVHFX?=
 =?utf-8?B?d1EwZEtqWmpjbEFFTm1RRktkQnZTeEJ0QTNyNzFoeUJOZ1pQdWpsRExQdFZh?=
 =?utf-8?B?VllZbUtMMW42ankzT24xMXhHOGJEWWliaCtCSU5XL1oyYW1MMFFHeVR6OEdR?=
 =?utf-8?B?Rzc1THY0RWxLc2V4OFk3UE9sVjBGR2piWUlIS2dvR2Y4NmhMdytWYmFqa1JR?=
 =?utf-8?B?QmJ1NkpqMnRMWlJyUmZ5eGZhNDVSclgxRFRYOFRvYzU5NHVTeUFYYXNJUnh0?=
 =?utf-8?B?ejFmOUtwcDhyUG9TdGE0Z20xRFZFMzArbGlRWnhPUHJESHNqTFR6SEZzK0d0?=
 =?utf-8?B?UE1hTnBEVGJwTC95ODlmS1VWUHZwbWRTUDh1d3Bhb0R1aW5QZGFZZ3dwVXZ0?=
 =?utf-8?B?ZmFLTjd2dndsZFFlaDBHYmFlbUt6L2pNYUZZMU1EcGhxbDgyVTlhM3hnbzMv?=
 =?utf-8?B?Y1JyT05uZXBKN2QzQ0szMkFVVHo4M01saG96ZHN1Yno1bk9WVlpvcHVmcVZR?=
 =?utf-8?B?T2U1RmdSanA5YVhmKzZHdkhYY01CbGJRU3FMZjJZTThrN0dUSFV4bHBId1Mx?=
 =?utf-8?B?bGpNVEUwUDVXQi9xdHBDbmhybUVNeFNCMFNBcnIrMTJrWEpCK0hPTFFZTCtN?=
 =?utf-8?B?SHFzOWVyWlEraG1LelBaaGd4cHU4bnI4VmlmSlI5eTlZOG9CQ1VNS3hJVVNr?=
 =?utf-8?B?R2tXMitCQ1pMQ2hRVVZvclphT1NQZElZYTRvQmF3UzdhTE5jVlZCcXZWMEJm?=
 =?utf-8?B?NlVmNDIzbXZJcW53QTBrZWsxQ2RoRDhObHNVbXVoaUJqZXNobXRCUk1Rd2Vh?=
 =?utf-8?B?a3NSTzd4emhYeFZFam9yUW8zOGFlTXNuQU83QnRXSEJnc1hmeDVqZWRiMVBo?=
 =?utf-8?B?WFowSmlRMUpseW5OSk5lNUFmZUhiWkIyU2tUWVFBa0hlTzNjdHFkTUY1M05k?=
 =?utf-8?B?MFBORTEwOGhvVXJRMGM4YWxNMVZ2QWpNSDNjdzYwTlFFOUtLSFFVaTRvL1RP?=
 =?utf-8?B?TnZ3ZzlnaUNESncySENFWkEyYjk1WUdpa1hBUTRMeHNGQUlIOWMzUWh2RlJj?=
 =?utf-8?B?NjBtcG1XeDRZTVY5blFVb1h1N1FWVmlENThmSzErajhqR3Zrc3F0MFpFZGdv?=
 =?utf-8?B?TjhGNVJnMzJ3TE9VREtvOHBNZzFZRjlYNThIOE0zWW9RN2loZnNXZkxjYWxo?=
 =?utf-8?B?MitQT2p4Vk1xTHloQUVIUGlHRXVIT3F2V2FKd1VrUnJpTDNscnZtcm8yWkJo?=
 =?utf-8?B?WU1QRkpDekU4dXh3MTdEb0wxNGQwNFdsSnNrb0UyTkUyUTVySXN1RnNKOGk5?=
 =?utf-8?B?bEw0R1FkS01MQTI0YlA3aHJiU0twTEZGTExnZlVPeGVaQ0tmMU1DRUpmbUc1?=
 =?utf-8?B?RGRMeWZHY1NzM0h5OUxrK0hnYzNOZ0NDUXJReUY0QWw2Nkg1MDF5RkxLMUlC?=
 =?utf-8?B?N3dFOHdVbWxGRDdIZGxmbHFQeTlqdkRWck9xaHBOanJFL04xZEpkaXhGdS9T?=
 =?utf-8?B?ZFh4WldhOThjei8rOWhNbjVUclk4VlFJVHU5OVUydTJhUXpMc2t5SmtTeWZs?=
 =?utf-8?B?V2JSdmR0NmN4cEZXSzFGYWpHait4TFErdWd5Nm43Wm1CU0NXc21ZYUkwVDV4?=
 =?utf-8?B?Y0dTZ3IwbGs0K0g2QXJDT3FIMHFzRm1vUGpkUDB0WUxrUkFWYi9vaTFOWnNH?=
 =?utf-8?B?VGpjbEtvNFVNeXM3NHUzSXRLUmozVENnRWgycE9TWjdRYTJxRTNGdzljNjlv?=
 =?utf-8?Q?AkbH6Z285Zfv1gjwCBQrPwgi54HXRrGosMg7U=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 18:48:13.1369
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d6e51242-4977-4c26-a7b6-08dc78fd6722
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099D7.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9262

On 2024-05-16 06:03, Henry Wang wrote:
> Fix the name mismatch in the xl dt-overlay command, the
> command name should be "dt-overlay" instead of "dt_overlay".
> Add the missing "," in the cmdtable.
> 
> Fix the exit code of the dt-overlay command, use EXIT_FAILURE
> instead of ERROR_FAIL.
> 
> Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
> Suggested-by: Anthony PERARD <anthony.perard@cloud.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 19:13:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 19:13:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726340.1130611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s98S9-0001e0-Qk; Mon, 20 May 2024 19:13:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726340.1130611; Mon, 20 May 2024 19:13:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s98S9-0001dt-O2; Mon, 20 May 2024 19:13:45 +0000
Received: by outflank-mailman (input) for mailman id 726340;
 Mon, 20 May 2024 19:13:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s98S8-0001dl-Dw
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 19:13:44 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 120150e0-16dd-11ef-909f-e314d9c70b13;
 Mon, 20 May 2024 21:13:42 +0200 (CEST)
Received: from CH2PR12CA0023.namprd12.prod.outlook.com (2603:10b6:610:57::33)
 by IA1PR12MB6188.namprd12.prod.outlook.com (2603:10b6:208:3e4::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 19:13:34 +0000
Received: from CH1PEPF0000AD76.namprd04.prod.outlook.com
 (2603:10b6:610:57:cafe::d4) by CH2PR12CA0023.outlook.office365.com
 (2603:10b6:610:57::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 19:13:34 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH1PEPF0000AD76.mail.protection.outlook.com (10.167.244.53) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 19:13:34 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 14:13:33 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 14:13:33 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 14:13:32 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 120150e0-16dd-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CRC8qGBWc36axJGli4EB9px/eJ8j/7cy5yWnrJKG9C6rDX0pcdiOPVecwK5L0wv+4qTA9OTIZDKfTEU2uplQGXCJ09E1lqLiHTUHtBMi9KNmngGR6W4SkdFNTPQ+rNGTRhOcfPICKcVa0PJyKXlmzi0sr4xyfcmk8rFPJqnUZXz4W3S/4IiGaEzGk0qvq/mfbzILWaShQk8y+TlNQ115RFI6IRJujHZQuYpYCOXcv+psCtnDhoRqvnAG8Q1xEuFQcz6UH2uSiSx29HYdHdbK1X2z61C0ddhhyA3ozm8How2LrYCeT8+HkoVJ4k4nh3JqAZ9i7rQ1dZzMmNZj+jkFyQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FSHdhrWZj36bM4yAP74lblruAnRUdBDhdc4mQG0lj0w=;
 b=b8h+K1CHvp/btt5eHscxyG98dhSNu3oCAz4FlxMFES07DAb8znebm7k+pLUf6/+EswNg5qLL7xKtn1updwUW1kYnXC8vdmkpV2V/SVfBL7gpxe8cmpv2Ge4Da4RlkuPN/LUaT/QakowXjRGo6IAmsnbSot3cq85iu4qiy4DHNFxB7JezjeVEXGr0v0uUb2MTSLgX2PONbsCfbId8rtiJ9/4RUm3yZbvAL0/SbrmyaJlR5wH3qtVBwLkSNrb3isLv8wvJ9k/j/GAkvgNshZvqoOS/q+tH8amSdQeahIp6mesM0liJi/yWnUSiGQFhc481dEvst9vM8PhaYLcVwkqHXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FSHdhrWZj36bM4yAP74lblruAnRUdBDhdc4mQG0lj0w=;
 b=Xe0BD/5Mlq9IXsNGrWXKiPusAuGX5Vig3FCX8WIeZK0oF4K04+dAaadgTgENrb/9P50cWB5JwuV7HdKHM97nHEAIQ8gsqZYOXMz6ZWH4XnWLkCciQul+tztZCVNcQN/yMZSTwppRBw8FvHoSoPQ6C0t+7YGKaTkY096pqEpAGpk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <0a87ccc8-4262-4122-8142-2f64a174bab2@amd.com>
Date: Mon, 20 May 2024 15:13:32 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/8] tools/arm: Introduce the "nr_spis" xl config entry
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-5-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240516100330.1433265-5-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD76:EE_|IA1PR12MB6188:EE_
X-MS-Office365-Filtering-Correlation-Id: 1bba8075-8f5f-4773-0c40-08dc7900f1a2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?dFZRaTNYbExZMlUvdkYwWVkzWWRJTWZMZlBxWkhQMXVUWHFId0kveDZkd2lI?=
 =?utf-8?B?cjhTUXA0T1piOFJQUVc2VWR0SGNlNW5odk1Rd2QvNWFacDBidzc3amcreU54?=
 =?utf-8?B?SE5RS2NFbEJyVVFFQjErT1M4Rm5meXpKMUtxMkF1MDZQQlVUalpwNEcyM1gx?=
 =?utf-8?B?NWl4dC9sc3hkZ00zQ3ozRy8zdzVqU3Z4NkVSVGpwU25ORDBPSkkycXBZQTY1?=
 =?utf-8?B?QjZCclhXMVlmMDVZS3owZ3orNFNWTGxnUVpGU29kL1pUckl2S0NsbjZJMUhr?=
 =?utf-8?B?WDAwWHVjYWFmVU1uRlRHdm81N2ZPdnZhUHlvUjVoN1pmdU5YUlh4SEQvRFFo?=
 =?utf-8?B?cmVJZmdiT3hUL1kzRDJOalQ5OW1pY0N2T2tGckFCakpUY0hHUWZYdWZURDBG?=
 =?utf-8?B?NWZPSUN3VlJOTXE2T1YxTjVmV3dFcHcrcnh3N0pOa0Uyb1R4a1VOTk1rM25K?=
 =?utf-8?B?SGVZMlNFMm9PZ3FuQmg5c0dIOFJwUU1oRyt6bEh6endpZFNTbVJ3SjJKUEdQ?=
 =?utf-8?B?NUM1QjhlL2Z6d0ZPWWdWYnVHcExWRUptR2tHbHJxTnBSbTFlM0tNT3g5cENj?=
 =?utf-8?B?ZWJlNDJwWFZwbTIrcGN0TjhnVXBEN21lVnMvbzdHa1RGUDZSeG51MjNjOVl1?=
 =?utf-8?B?dnZuVE9hcDNuWmpDWDRQa0FxbENEUERzWHZaUndvZDN2MFJCYzFiNXhrYVRt?=
 =?utf-8?B?WjZwdno4NnVVSlV4QnJ4UDBBRmVET08wazJocmlZMEJrVTJKc2VIUzFZQVJq?=
 =?utf-8?B?VVB5akViYldqSllrL29udHltMzNmN3JZZStYWFV0N1o0SDlNWXRhVzY5d1d5?=
 =?utf-8?B?UVRzZDJMYWVJbDg1ckRTZU1iOENNaVRpVFR1eThrcTZQMS9iMVdlYjdXekxQ?=
 =?utf-8?B?bGNER2t0ZnVMeUpHYUdZb2RpcVpFeHl5VHFMTEJQTUFvVldMcUVMTUNONDdW?=
 =?utf-8?B?SWhZajNlVHpnQ292YjEzaGdXeGQ1eis2NjRINWdqWHlnbzZlT0gxeTl4T2g4?=
 =?utf-8?B?WFRFd3l3d0FsVFEzT3pPS3RKZ2RwZVdGTVFYZWVQdE51djdiRzlzVjJ1OGs3?=
 =?utf-8?B?dG5IczdJR0FUMlV3NytYYVM1R3FkcmMxazBwSy9Nc0FmbU40aHUvTWR6T1Az?=
 =?utf-8?B?Z3JVVXViT2dCVndwU3NCL0U2alo2bDVnSEwrQXFSajdCQjZvL2dMTHJHMlhS?=
 =?utf-8?B?LzZsYTJwWXBRK0tpTFRrYVpiejNoV0hsdU54emZMdTg5dGRiaFQ4bWdRbTQw?=
 =?utf-8?B?Z1VIZlNrU0taMnpDY0lJcVpmZ2E3U1pBWituLzduOXVGTDVDRG5hMEd5UzFj?=
 =?utf-8?B?YTFFeWpNOCtndU01WVBFMjVmRW1vWEtRWk5xd0c1U1lUakRUbHMycm1KNUVL?=
 =?utf-8?B?cTd6cmJIMERFTnM0bGJwTm44aDV2L00rdEZzaCtUUUs1dk1RYkhQWUUxcW9Y?=
 =?utf-8?B?ZWR4Vm5FUXd2Vm4ySnQzOGtLSDVSRnc5cTM2TEhQVXFodjNEelV2dTZJd2Yr?=
 =?utf-8?B?Z2RNTlpmZFFWWmtXV0NVRVRRalQ4alNGSENVRitNYlJBbHAyRU5sTUx0dGYz?=
 =?utf-8?B?NHJwVWUrODR3TVdrQXVsaVR5RUVTZjZJaGtHTHlqNHFKNjBrNUs1ZHlYdStl?=
 =?utf-8?B?bXFtdVVnTUt0UWtKTGZ2WVd3dzY4OWtNeEVld1N0S1RkbVR4T1BhSTEycW0x?=
 =?utf-8?B?c1VrWGFXbjhRRFU2dVovRnVpY2FwZnptNERkdnovdUM3Z2tkMWxKNHNXOS84?=
 =?utf-8?B?VmZoTFB5ZklLcitzSmhIdFl1cDVkbFlHOUNxVFd2MnZTdjB3elJDcFhsVUZM?=
 =?utf-8?B?ZnQzNmU0aEFNQkx1TXVYSlVZZEVmSllwN00wNms5SitrOWlIZmp5M2owbUFs?=
 =?utf-8?Q?k68g1k6hUi0Ww?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 19:13:34.0032
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1bba8075-8f5f-4773-0c40-08dc7900f1a2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD76.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6188

On 2024-05-16 06:03, Henry Wang wrote:
> Currently, the number of SPIs allocated to the domain is only
> configurable for Dom0less DomUs. Xen domains are supposed to be
> platform agnostics and therefore the numbers of SPIs for libxl
> guests should not be based on the hardware.
> 
> Introduce a new xl config entry for Arm to provide a method for
> user to decide the number of SPIs. This would help to avoid
> bumping the `config->arch.nr_spis` in libxl everytime there is a
> new platform with increased SPI numbers.
> 
> Update the doc and the golang bindings accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - New patch to replace the original patch in v1:
>    "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
> ---
>   docs/man/xl.cfg.5.pod.in             | 11 +++++++++++
>   tools/golang/xenlight/helpers.gen.go |  2 ++
>   tools/golang/xenlight/types.gen.go   |  1 +
>   tools/libs/light/libxl_arm.c         |  4 ++--
>   tools/libs/light/libxl_types.idl     |  1 +
>   tools/xl/xl_parse.c                  |  3 +++
>   6 files changed, 20 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 8f2b375ce9..6a2d86065e 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3072,6 +3072,17 @@ raised.
>   
>   =back
>   
> +=over 4
> +
> +=item B<nr_spis="NR_SPIS">
> +
> +A 32-bit optional integer parameter specifying the number of SPIs (Shared

I'd phrase it "An optional 32-but integer"

> +Peripheral Interrupts) to allocate for the domain. If the `nr_spis` parameter
> +is missing, the max number of SPIs calculated by the toolstack based on the
> +devices allocated for the domain will be used.

This text says the maximum only applies if xl.cfg nr_spis is not setup.

> +
> +=back
> +
>   =head3 x86
>   
>   =over 4

> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 1cb89fa584..a4029e3ac8 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>   
>       LOG(DEBUG, "Configure the domain");
>   
> -    config->arch.nr_spis = nr_spis;
> -    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
> +    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
> +    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);

But this is always taking the max.  Should it instead be:

config->arch.nr_spis = d_config->b_info.arch_arm.nr_spis ?: nr_spis;

However, I don't know if that makes sense for ARM.  Does the hardware 
nr_spis need to be a minimum for a domain?

Really, we just want the documentation to match the code.

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 19:41:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 19:41:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726349.1130621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s98t2-0005u5-UW; Mon, 20 May 2024 19:41:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726349.1130621; Mon, 20 May 2024 19:41:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s98t2-0005ty-Ru; Mon, 20 May 2024 19:41:32 +0000
Received: by outflank-mailman (input) for mailman id 726349;
 Mon, 20 May 2024 19:41:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s98t1-0005ts-6E
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 19:41:31 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2f71861-16e0-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 21:41:28 +0200 (CEST)
Received: from BN9PR03CA0298.namprd03.prod.outlook.com (2603:10b6:408:f5::33)
 by SA1PR12MB6800.namprd12.prod.outlook.com (2603:10b6:806:25c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 19:41:24 +0000
Received: from BN2PEPF00004FBF.namprd04.prod.outlook.com
 (2603:10b6:408:f5:cafe::5) by BN9PR03CA0298.outlook.office365.com
 (2603:10b6:408:f5::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 19:41:24 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN2PEPF00004FBF.mail.protection.outlook.com (10.167.243.185) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 19:41:24 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 14:41:23 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 14:41:23 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 14:41:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2f71861-16e0-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KO9xZjlJUUrGjV65aWvIEgQhVmPDSW9u69t0phjRLDUl4M8JZxvapQUNC8NK4w2p5DXl1tCaDInvz8QwGaS79E3+BllY7/LlwWq67Oa6D5hOQjrSLqQgqNIqm5VEb2STDdm5/GEA1I+Uw3+vVW6oFu6CnJNd2YVu8MgIeV2gC0keQezTVD4O4907IrLUamB8bCApgGq6j8swy4WQM+qgBwLeCs9re80lCAlBw+JGszZ/lRhWtvYovMAOsm4gy8P83jafaxO44jQHATzi1klNuCtNOm9ETLwXlTOdhOETp5rWVo3ivBnk+kQAHPcTMKfEshcas6R9g1Q3akNt78N0Yw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HpVi/t7xur6IAV/kp/Bqzy90KKAo4YAaYUnQfTVA5d4=;
 b=h+lREDkXJDghQVRkoULhJCzjM6ExmugOBfT5mJJgMjNTD0GRNjqEpZ4ZSmJIH52sZ4GG9gSeNYKB9Nt2LYIw2UZXm4kjG3my/PpeQQPt44YVdh+laV8xUqvIWZjZd46e1DZ/MCVMPiJq2oXFW36rEJTo43PtUB9oIe3J5ONQoXf4iVNeYqM4EYAQBi4g+0ISFqSdvXt4Oq0qpBz0CPpgOZ8j8T3v8N757ZdtnB7qW4k/DRDgFUm0x2uKuQA3febXiegc2Dg52f7IUYUkEPOiQ5E8ivb4cEXB5aMr49wHESLVGJhUJgEe1XJJhULb31wwGo0FTCVEZk+WrbknFqn8qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HpVi/t7xur6IAV/kp/Bqzy90KKAo4YAaYUnQfTVA5d4=;
 b=RfQ1fDhR+4yg0YMkfffhatYUny0jIZW1sVG0nywN62Zx9nz8miZ8l7aPdoIw5vyhKTV91eF5TUQkCdpg0iEHmC2s3qQJp5VRtf2sERhqYzqCTYle1PXOTGqEJ9ROuXUWpf6zzFL6C46hnpU9oElpZ5CfcQOkhoQscF9Cen4x+EQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <6a04ef17-2580-4f62-b377-aad2ebbcdb5b@amd.com>
Date: Mon, 20 May 2024 15:41:21 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 7/8] tools: Introduce the "xl dt-overlay
 {attach,detach}" commands
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-8-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240516100330.1433265-8-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF00004FBF:EE_|SA1PR12MB6800:EE_
X-MS-Office365-Filtering-Correlation-Id: b3409caa-32cd-4db4-db97-08dc7904d505
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?cFdZdmJsUmtVS3dDNFA2ajJYamF3LzFmTmtiamp3M3hBMUhNTndJSGpkOGdL?=
 =?utf-8?B?UWxFemIzVzdjYk10dFNTM2pmSmJsRmI0cmcrWkFHNTJhVmpya0JxOFp5R1hH?=
 =?utf-8?B?UHphL2x6eHg4R0dkeUFCT2NtQ1JLM0d5d2w4VjJQdEpGdExpU0FRWGt5YmY0?=
 =?utf-8?B?eTl0U09kTncyODlOTElscEFEZzNZUHVhR3NOWTc5NWFYS0JITWoxdnpIbzcw?=
 =?utf-8?B?RWh3YzB2bkJkVkg2WEIydTVTcTJOd2xlZkE2VkYxK2huWndTNi9HSlVETWZ5?=
 =?utf-8?B?VFpQZ2Q0aHNIQ00wV2h4N1NkUjNZekNSaC8rN29RelRnbGJoc2VpY3pIWmpI?=
 =?utf-8?B?dTB5SlYzdkwybGtvVk1qTmFBcUNEU3N1aGI3ZWtlV1RDWlJFdGZoa3dGelpT?=
 =?utf-8?B?d0tHNUI3aUhkcFpJbk5SdXpXVktiNEJueGxvWXBtcU50VlRLM2FSZm9tNjR3?=
 =?utf-8?B?aDg1VlJZajUwdTRibzZCZlhzakEvUVBQSjROTUFEMmVCOTBoNXlQQmdycDdm?=
 =?utf-8?B?c2lsTXI0SnFGbUptSjlVMjRvb25LTXJwRnloaGZZVGg3cE9KTEsySTYzeWZE?=
 =?utf-8?B?ZUV4RmF6SStORnNwZXJiTzZxTEMrdS9kV2pIbWFjTDBzQUpwdkl5eWsxSjda?=
 =?utf-8?B?Qm02SE5HdG8xb3hCREphVW1PWm1pL29UQlFFeFU1UjBPcFFza0x6alVEaEZR?=
 =?utf-8?B?S2ovZlB1enREeTNBTlc0L0FnRFZpRE1Ta052cUxCNDZoNjg1ZXVBdXdsdmdF?=
 =?utf-8?B?RDVtbC80QmZCU0daWHNlWnJuNC9CZ3F5c3NNaW5jdmFoQWtCK1VwTmRGVlFX?=
 =?utf-8?B?L0Jvb0hRZTlQYnZwUll2bXBveFk1aFdiZWhSWFBEaXZHN1AyRlMvY0YyWUt4?=
 =?utf-8?B?bGMwakNpakc4NXVscW9kcVNmTVpJY1ZFdzRpakUyOXVQK3JTenNZYWpXOXRk?=
 =?utf-8?B?UE9SRURnM0ZoYmIwZGhGOGhOaTc4ZWlpdUllYWs0aUhTTCtQVTFiK1M5SVQz?=
 =?utf-8?B?aGNWMDdJQlltdkJyTThOeDhBVVhOaXMyeWJqeU1LMHFxTXJ1Y1lhYlNOVGRX?=
 =?utf-8?B?MWJ3V29xcVh1TU5LN05MK3dFNmhiK2tYb1BEcTNxQmdnTTBWNnZ2TG0xQnRO?=
 =?utf-8?B?OTRwLzVkb0xneWR6RFhJT2JSUXhjeSsxYzNNSXFidVJjdXBsR1ZlbkNDd285?=
 =?utf-8?B?c2k5Nkhwc0wrcVh5UExQN2xGVlBOanNxSmowM3F0cHpMZFUyQlhFbFZEUld1?=
 =?utf-8?B?R0lIRURZZTl6TllIWGVWYWRSR0N5KytNMXRWc0xLTFZYcFN2UHhrVEZpQmNm?=
 =?utf-8?B?Vm9YOGFDaXViakpQTmxvd3RYalhaTUlmQkhuUG0vZlNOWkoxY2pJSEgwTlZ2?=
 =?utf-8?B?bm8yckZEY3dZdmpPZDVzRWV3YjN6SFB1elhxMVJKQnVENVpUdkZCMndMekV0?=
 =?utf-8?B?YUpRT0QrY1orcXMyVENnZ0duZDhVQ0krMDJBZUtEVHd3dUhzbmFSczhwNjk2?=
 =?utf-8?B?cS9DT3l0emVwRFBWVWhpS2tTL3hkQi8xd29heDRnYzR5azVWTU5NMWZIRlVW?=
 =?utf-8?B?WU1EQzZnVW5sd2hrMmN0Q2w3SHorelh0N3R0eWJObldwbTQ2OTJYYUFWWEZ3?=
 =?utf-8?B?bEhweEtTMnhVOXd6TFJZeFA4QVBIeVVQeEJRNHFYL1hVNkNLZkZmZFUrdGV3?=
 =?utf-8?B?dTR5RDA2R3hFRXBiMktaM054K0NXQmVHemhaZEhCdjI0T3RqNGk4REY4T0w1?=
 =?utf-8?B?L29SdVlYd2tJcHJhU0s1a3J1Z3cyQVIvUk0wWW9XUXVJSk5QT0g4bS9lMVRI?=
 =?utf-8?B?VTg0YkZXc1ZUVE5tZUNmRnVTSElIeG9hd2JZQzRxdkRiN3gzVVNhN1dMWlk5?=
 =?utf-8?Q?qMEyP5jLbD8yP?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 19:41:24.0216
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b3409caa-32cd-4db4-db97-08dc7904d505
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF00004FBF.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6800

On 2024-05-16 06:03, Henry Wang wrote:
> With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> attach/detach devices from the provided DT overlay to domains.
> Support this by introducing a new set of "xl dt-overlay" commands and
> related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
> rework the command option parsing logic.
> 
> Since the addition of these two commands modifies the existing libxl
> API libxl_dt_overlay(), also provide the backward compatible for it.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v2:
> - New patch.

Mostly looks good.  One small thing below.

> diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
> index 02575d5d36..53d1fa3655 100644
> --- a/tools/xl/xl_vmcontrol.c
> +++ b/tools/xl/xl_vmcontrol.c
> @@ -1268,32 +1268,43 @@ int main_create(int argc, char **argv)
>   #ifdef LIBXL_HAVE_DT_OVERLAY
>   int main_dt_overlay(int argc, char **argv)
>   {
> -    const char *overlay_ops = NULL;
>       const char *overlay_config_file = NULL;
>       void *overlay_dtb = NULL;
>       int rc;
>       uint8_t op;
>       int overlay_dtb_size = 0;
> -    const int overlay_add_op = 1;
> -    const int overlay_remove_op = 2;
> +    uint32_t domain_id = 0;
>   
>       if (argc < 2) {
>           help("dt-overlay");
>           return EXIT_FAILURE;
>       }
>   
> -    overlay_ops = argv[1];
> -    overlay_config_file = argv[2];
> -
> -    if (strcmp(overlay_ops, "add") == 0)
> -        op = overlay_add_op;
> -    else if (strcmp(overlay_ops, "remove") == 0)
> -        op = overlay_remove_op;
> +    if (strcmp(argv[optind], "add") == 0)
> +        op = LIBXL_DT_OVERLAY_ADD;
> +    else if (strcmp(argv[optind], "remove") == 0)
> +        op = LIBXL_DT_OVERLAY_REMOVE;
> +    else if (strcmp(argv[optind], "attach") == 0)
> +        op = LIBXL_DT_OVERLAY_ATTACH;
> +    else if (strcmp(argv[optind], "detach") == 0)
> +        op = LIBXL_DT_OVERLAY_DETACH;
>       else {
>           fprintf(stderr, "Invalid dt overlay operation\n");
>           return EXIT_FAILURE;
>       }
>   
> +    overlay_config_file = argv[optind+1];
> +
> +    if (op == LIBXL_DT_OVERLAY_ATTACH || op == LIBXL_DT_OVERLAY_DETACH) {
> +        if (argc <= optind + 2) {
> +            fprintf(stderr, "Missing domain ID\n");
> +            help("dt-overlay");
> +            return EXIT_FAILURE;
> +        } else {
> +            domain_id = strtol(argv[optind+2], NULL, 10);

domain_id = find_domain(argv[optind+2]);

And you'll get name resolution, too.

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 21:06:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 21:06:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726361.1130630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ACh-0007Yc-QB; Mon, 20 May 2024 21:05:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726361.1130630; Mon, 20 May 2024 21:05:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ACh-0007YV-NK; Mon, 20 May 2024 21:05:55 +0000
Received: by outflank-mailman (input) for mailman id 726361;
 Mon, 20 May 2024 21:05:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=l4Ei=MX=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s9ACg-0007YP-Th
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 21:05:54 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2405::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bcf79b9c-16ec-11ef-b4bb-af5377834399;
 Mon, 20 May 2024 23:05:52 +0200 (CEST)
Received: from SJ0PR13CA0138.namprd13.prod.outlook.com (2603:10b6:a03:2c6::23)
 by LV3PR12MB9258.namprd12.prod.outlook.com (2603:10b6:408:1bb::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 21:05:47 +0000
Received: from CO1PEPF000066E9.namprd05.prod.outlook.com
 (2603:10b6:a03:2c6:cafe::27) by SJ0PR13CA0138.outlook.office365.com
 (2603:10b6:a03:2c6::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.16 via Frontend
 Transport; Mon, 20 May 2024 21:05:47 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066E9.mail.protection.outlook.com (10.167.249.11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 21:05:46 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 16:05:45 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 16:05:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcf79b9c-16ec-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ERN8D9mU+kVbO/1d2c+Vz7TEWRAnhj/etc7AmtIkbTyV/b88ojE8QW6jvbQQciRqh/RTz+WGKQuHw47DO6W17aVgfes3zhUin1ZuwKBUN8bZDeSUTBGX0UMHLGw2i2k84ilZv60UvSIIcrq621tEE9KLANxRnkyprbGCG3Vm9rscXovY40wSj/LdCf6cbhwzviU4iGUX+yOClq03g5uVRIovKI3jcs3XeXP6eQv2TTnv+ucdwvYku8qOerbgICd5suOSWLnENWMe0dy0QMO6FJxaE9IeEiU6UaP+cE/Tk/3/ivVyo44JJVfNFwkW94yZdMKwn2Ss7QYOtLM9TaVZIw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=a1Dh8SCd+uOTZ2/DoKJPBehNZMUCC4Bg5NtdqbgxHj4=;
 b=i4YHsLAhs+FIx85IE6n1IaGjyDMLuYVzpiPtOh8v+t2v2YzYtFbVXxJunGdUFeQ8zuZ0NjW//tLoiETwkVva/xWKI1Ioppg3FNESYRUml88uqhq73nf8fvatOfTZS2AvzkpixrR/oSBHeayDmdac7InEhsLgguYZeLuDSUNpHgnw8/tq3Dz2FlRukOazT8SLRhjeS2WkpwpfyA8WnHUahS9O1XuRAAGWjvo6Qtvj5SsszxjF+7562MiYvMbAix41I45WTOwTeVaI0CmOgNhFWQJ2pqEb9bdQ40VzYZK/HFlsvJrVH0rZBItGUcipyTC40OIsM1lYrKnv+QqgBAZThA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=rabbit.lu smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=a1Dh8SCd+uOTZ2/DoKJPBehNZMUCC4Bg5NtdqbgxHj4=;
 b=kxWVcElWhlizgrEem+JbHT1Wo270N3RJmeEn6Gxe0fyR0UE2ZIp4P+b7S2F+9rNIEBXGLbDeISmo60aTZMc4+kGTjVcizPtU2EPDpN66NIpLY6bThGa7RJplNAUhpwFZZK1oAKcX83spgGOsZkTrxwLrsyWpJ6z0I+Ngs50GMf4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <c23b7593-0ead-4ff7-8032-052269fe82e5@amd.com>
Date: Mon, 20 May 2024 17:05:44 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xl: add suspend-to-ram and resume subcommands
From: Jason Andryuk <jason.andryuk@amd.com>
To: =?UTF-8?Q?zithro_/_Cyril_R=C3=A9bert?= <slack@rabbit.lu>,
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
	<marmarek@invisiblethingslab.com>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>
References: <4e9f3590be9f19edb87e05c7b7c2efeae8863109.1709190010.git.slack@rabbit.lu>
 <d3c9ea82-ff33-4211-9e76-f2e599f21a8b@amd.com>
Content-Language: en-US
In-Reply-To: <d3c9ea82-ff33-4211-9e76-f2e599f21a8b@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066E9:EE_|LV3PR12MB9258:EE_
X-MS-Office365-Filtering-Correlation-Id: 16d09114-d41d-49ca-186b-08dc79109ed5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NWFneXZYLy9oUGduMlJaUUoxN0ZqZnZJUmFuRTFHWnNVYXhjWTZxNXdsOHpw?=
 =?utf-8?B?OWVMSHJxUnlTcW9TdmxxRWVHc0VsZkozM01YWm1IVjZmdEJhWjlodERzNWo0?=
 =?utf-8?B?WnBJbFgvNnZQUWRXK2pkeVo1NXY4dEJGbDhqblpTb3B1eVErWW0wZnZTWjhk?=
 =?utf-8?B?U1RaVkZ5Uldhdmhoc0tWMzltMkozdUJBRlEwbXJpTmgrRWNKbGoySWZrRjlv?=
 =?utf-8?B?Skg0dUZOa0ZUbVJUYkQzcitySUo1SHpMU0dwdENrV3pzT3NpaUswNkZ2VmVZ?=
 =?utf-8?B?T1lRM2UrR1YwQlBBZmw1b2c5TFo0ZjFsN3pzRzhrNkhpVi9KbEdZYm9xOEVP?=
 =?utf-8?B?S2dIWGpkam9CUXhvdkJNZ1UxSHVyUTZ6M0hrZ2twL3VuZEtsMUxNMFVqQStV?=
 =?utf-8?B?bWV5MWlZSXZFYjl3UXZaWEQrSDQ0SU1qblAzSzhlU28zYmVVVW96YkFzRHky?=
 =?utf-8?B?U2I3bEdsaW5xVlVoS3dTWkF1MEZGNWRZOHBCU1h4Wmg4QWhiSHRTL3ZJUWFD?=
 =?utf-8?B?dzIxWmN1N2R6WG9HMkVhamJ2YnZUbHJoYVNJZHRrQ3hKQXFJVExWNUJkL0FU?=
 =?utf-8?B?TjRSRTI5NG9WbXNtZnQxcEsxRHY4MTY3TktRZDVmQ2U5bWlYTXZYOXJieVZt?=
 =?utf-8?B?aW5mMlZsTEdORjl0aWdsMElqV25uL3lDTU13YVhoNkgwbXAzVjUrZGpqdEor?=
 =?utf-8?B?ZmY0WkNyQ285cFhTazh3cnJiU1VxVFNMWjVSZ2RQTzIvQUlXQVR1Zk9XU2NX?=
 =?utf-8?B?MzAzb3JpUEdnKzlFb211MzB0aTR5dTk0YXEwOFMxZUt6Yys4M3kyenpwbWtU?=
 =?utf-8?B?b2Yvams0YkJrTUwwQzZjaTVnZ3l5ZkdVMDBBV1RyT2tWRStsaHAxVnY2L1By?=
 =?utf-8?B?RGdTdzRnNy9XZzBjL2tDSGE3aEl1R005YkFocmZGaFlIQWhSNWFJKzlGeFUz?=
 =?utf-8?B?aW95MGZJeHhsZDhsOC9hbDI5OEFETFNXOU8vVlVLdlY2R0NKUTA0UGd4SWd5?=
 =?utf-8?B?K0pkRFZrWllyWXRYbTlscVpJREVFTUd4RDlGUHl4YXZvSTBaQ04xNGd5TmpI?=
 =?utf-8?B?TEYwSzZMVFNIQWd4OUg3d3BkaW1WUUNhVm12SkRJTzlKaXZHNUJ0ZU5DZmlr?=
 =?utf-8?B?LzNKM3lKMG9uOGR0cmRHanU3RDQ0TG1mR2ZINldFOEM3TVdiSUNZdWNCODE5?=
 =?utf-8?B?Nm1hV1NtcWtoU0Fqc2xKUUhhM2doUlJ4K0xlV1Z5NVVreHhDN0I4Q0dGeHp3?=
 =?utf-8?B?aitzaXlGazMwRUVVVzRRRVdtOXBDdlJFR0JuUTRwdmNSeVBBZURIaGF1UTNW?=
 =?utf-8?B?VDE5TC9yR3l0ZWs4ckJ2N1dESFV1bDFVSzZQalNDOHQrMTRsVWpaUFdib3hD?=
 =?utf-8?B?RERDQTVnWkh4bzhxV1lGK3VyTFVMd2VSVGVUeFFZeU4yL0RMa1h5Uis2Rlla?=
 =?utf-8?B?c2dXWW5HdEhxR01Kc0tYVGVCa1JMcERkWGlLdGZMTVFpMVlqNXlqSFp1OWpZ?=
 =?utf-8?B?MU9NZW9VbG5CN0VyakFZTHBXVE5aZ0tZck1RNUVsWWQvSTJGcHhENVVBZ3VM?=
 =?utf-8?B?dXcvb0hFQ2ZRcnZHcWNQR1k0NTM1ZWV0ZUVzdDRVL2pjb20wZWJaWnJTWkoy?=
 =?utf-8?B?N0xMYmNOV2h1bU51ME5Hc3RBandtUlY3eTZkakMvZXpFc25uR2t0TW5vTGVJ?=
 =?utf-8?B?a3BPbnlJVzd6Y1Nxd3lMWGdzV05DNG5hdCt6ZlRsbHJHQlg3cXpIS291Nm5v?=
 =?utf-8?B?WGFNc3JKVy9GN0hOSEJtR20vaHc0TzNaS0hWZHc0M2dvRStlbGJHSEpNTTV5?=
 =?utf-8?B?UTBYbTdZd3lDS1kvNFBCb1BsL1RjWWVEZmZ3Qno3UlhoZWlvMUY2VmdSYnNy?=
 =?utf-8?Q?svWHz6to0/saQ?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 21:05:46.9564
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 16d09114-d41d-49ca-186b-08dc79109ed5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066E9.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9258

Hi Zithro,

On 2024-04-24 10:03, Jason Andryuk wrote:
> On 2024-02-29 02:00, zithro / Cyril Rébert wrote:
>> The xl command doesn't provide suspend/resume, so add them :
>>    xl suspend-to-ram <Domain>
>>    xl resume <Domain>
>>
>> This patch follows a discussion on XenDevel: when you want the
>> virtualized equivalent of "sleep"-ing a host, it's better to
>> suspend/resume than to pause/unpause a domain.
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Suggested-by: Marek Marczykowski-Górecki 
>> <marmarek@invisiblethingslab.com>
> 
> You should include you S-o-B here to certify your patch under the 
> Developer's Certificate of Origin.  You can read what that means in 
> CONTRIBUTING.  tl;dr: You are stating you can make the open source 
> contribution.

I'd like to re-submit this while retaining your authorship.  Are you 
able to provide your S-o-B?

I'll do all the follow ups, but I need your S-o-B for me to make the 
submission.

> I tested this with a PVH and HVM guest.  suspend-to-ram and resume seem 
> to function properly.  The VCPUs stop, but the domain & qemu remain. 
> Resume works - the VCPUs start running again.
> 
> However, the domain destruction seems to hang on poweroff.  The VM 
> transitions to powered off - state shutdown - but the domain and QEMU 
> instance are not cleaned up.
> 
> If I power off without a suspend-to-ram, everything is cleaned up properly.
> 
> Have you seen this?  It's not your code, but I guess something with 
> libxl or qemu.

I've found the issue.  xl exits when the domain suspends.  When the 
domain finally shuts down, xl isn't there to perform the remaining cleanup.

>> ---
>> - Tested on v4.17, x86
>> - the function "libxl_domain_resume" is called like libvirt does, so
>>    using a "co-operative resume", but I don't know what it means (:
>> - there may be a problem with the words resume <-> restore, like
>>    for "LIBXL_HAVE_NO_SUSPEND_RESUME"
>> - for the docs, I only slightly adapted a copy/paste from xl pause ...
>> ---
> 
>> diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
>> index 98f6bd2e76..ba45f89c5a 100644
>> --- a/tools/xl/xl_vmcontrol.c
>> +++ b/tools/xl/xl_vmcontrol.c
>> @@ -42,6 +42,16 @@ static void unpause_domain(uint32_t domid)
>>       libxl_domain_unpause(ctx, domid, NULL);
>>   }
>> +static void suspend_domain_toram(uint32_t domid)
>> +{
>> +    libxl_domain_suspend_only(ctx, domid, NULL);
>> +}
>> +
>> +static void resume_domain(uint32_t domid)
>> +{
>> +    libxl_domain_resume(ctx, domid, 1, NULL);
>> +}
>> +
> 
> I would just inline these functions below.

I see you were following the existing style, so this may remain as you 
wrote it.

>>   static void destroy_domain(uint32_t domid, int force)
>>   {
>>       int rc;
>> @@ -82,6 +92,32 @@ int main_unpause(int argc, char **argv)
>>       return EXIT_SUCCESS;
>>   }
>> +int main_suspendtoram(int argc, char **argv)
> 
> Maybe main_suspend_to_ram to be closer to the command line suspend-to-ram.

I'm thinking we may want to just use "suspend" for the command name and 
all the functions.

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Mon May 20 21:46:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 21:46:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726390.1130668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Apr-0004hW-30; Mon, 20 May 2024 21:46:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726390.1130668; Mon, 20 May 2024 21:46:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Apr-0004hP-0A; Mon, 20 May 2024 21:46:23 +0000
Received: by outflank-mailman (input) for mailman id 726390;
 Mon, 20 May 2024 21:46:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9App-0004hF-HV; Mon, 20 May 2024 21:46:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9App-0003Ri-Cg; Mon, 20 May 2024 21:46:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9App-0008W2-4B; Mon, 20 May 2024 21:46:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9App-0007nP-3m; Mon, 20 May 2024 21:46:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qBwVaR+4VXSynZxNGRXu/Wn9XYP6x2w6y9kyzeTsmDo=; b=RTbOjZZXcZiiQIjBPHec91it5M
	LlgVWBI382m0eoRhtH1Krqe4v5PI6IBaBFec0ZWP1qH+EIJdsu/LtkNtTZ5Rd5l7bHWn+DTQG21uB
	MtcqRHZOjjUoTm/T3Wxmat0oZdVMaku2SNDJyeOiAV7YU7ZlxA1mWQwKanC9I2YlA1N4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186048-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186048: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=26b122e3bf8f3921d87312fbf5e7e13872ae92b0
X-Osstest-Versions-That:
    xen=54aa34fc89151c943550541e0af47664fb6e8676
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 May 2024 21:46:21 +0000

flight 186048 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186048/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  26b122e3bf8f3921d87312fbf5e7e13872ae92b0
baseline version:
 xen                  54aa34fc89151c943550541e0af47664fb6e8676

Last test of basis   186046  2024-05-20 14:00:23 Z    0 days
Testing same since   186048  2024-05-20 18:02:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   54aa34fc89..26b122e3bf  26b122e3bf8f3921d87312fbf5e7e13872ae92b0 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon May 20 23:17:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 23:17:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726405.1130679 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CFe-0006gi-6D; Mon, 20 May 2024 23:17:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726405.1130679; Mon, 20 May 2024 23:17:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CFe-0006gb-3Z; Mon, 20 May 2024 23:17:06 +0000
Received: by outflank-mailman (input) for mailman id 726405;
 Mon, 20 May 2024 23:17:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9CFc-0006fj-2n
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 23:17:04 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0f156b19-16ff-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 01:17:01 +0200 (CEST)
Received: from MN2PR11CA0030.namprd11.prod.outlook.com (2603:10b6:208:23b::35)
 by DM4PR12MB6544.namprd12.prod.outlook.com (2603:10b6:8:8d::6) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.27; Mon, 20 May 2024 23:16:50 +0000
Received: from BL02EPF0001A107.namprd05.prod.outlook.com
 (2603:10b6:208:23b:cafe::f5) by MN2PR11CA0030.outlook.office365.com
 (2603:10b6:208:23b::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 23:16:50 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL02EPF0001A107.mail.protection.outlook.com (10.167.241.136) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 23:16:49 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 18:16:49 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 18:16:47 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f156b19-16ff-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=USfY+x4hKEmhAlpZihXdUije6QIq+gcJPe943VpabUmnAbjZit1xsLH7xudTOk9oQnQtLgud2yZImbHwOtINROmVwtgTeb1dztIO6zUIeK+vH5/fb90VbFZQK3GbY+zoykhXrrEcl1veMxqA0zCNvUVPb2X2/LuA0rJF4ZAHHhtWyb6dwmlIu0Wx0yD/zZMUtsAPvu8FVrIMbxvTgwuuUUqSJkiT5CI74wYhPblUOA54pAyHCK8Oh+O1GgfI8lC9kQd7Il4aei0I/Nn99XGQ/U9nRFJINxIO7Xp3+Qklhj1roGdFE+04zDxERYnwklH40RPuOkyFEq5+sU5DlUQBLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=akZ4jp/WdYcgcYndzHzWDNr2zMLdJ+e/qXQx01Jh8X4=;
 b=CsQIar/n6r2w5gVKtsfp+LmHGw37+SMiHVUL4za9hau1jTZF9iy+3VZLx+wxLVofWGmGtnFSP+RZDK64mbKQfBErf6NBETSH3sxvf3ZYU/vOPYe2AULKqda1f43dYh4wwtSOyXyHgSt93D5dMz0antRUzjQ78wtQWz6Gfjf085JwJqBBN6cBDUnTvH/TtYRTTaSKwQtsPMGQ9aNn/aSFfJubg4/CLykrraQrbyX6+9HmGecoq0aiRuT7xbAfdrBn2aenPnKe3+mjzJeeo1v6i0/SkXRwjdtQHD1wC9yG78xDYhq5WY63vV7pjMEUhHud7/5kziLVLHsRGt/vW6shYQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=akZ4jp/WdYcgcYndzHzWDNr2zMLdJ+e/qXQx01Jh8X4=;
 b=DeucdnaHwRsc4s6iRlNdTsEMOoEDTOvYxJI2mOKfrtA0gOfE/rI5jl1JHFEptgBtPJUSSFHLXSiEf8WwNe9ac2OBjrS1mJEQwl6zxU4vjdNVb6E8oO6/LNO9s7ukbtyBc7XH5vpczf47uqmHIasQIDqszDIc96YgSCMusYwl6MY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e53fbcba-f7dd-48a5-9481-a69bfb501cee@amd.com>
Date: Tue, 21 May 2024 07:16:42 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>,
	Luca Fancellu <Luca.Fancellu@arm.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-2-xin.wang2@amd.com>
 <ee20798f-a459-492a-a7a1-308472878fe1@amd.com>
 <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com>
 <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
 <c0b59ccd-b3b0-4495-bbb4-2855fa2d7d16@amd.com>
 <93336435-309d-4aa5-9977-9ce0d4725811@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <93336435-309d-4aa5-9977-9ce0d4725811@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A107:EE_|DM4PR12MB6544:EE_
X-MS-Office365-Filtering-Correlation-Id: 3d0540ef-305b-4980-eead-08dc7922ed57
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZHQ3eWtueWVZaTd0bGxYMmIza2VwUHNHNFovd2ZoQVlrdnMxbXZQc1dGVDRO?=
 =?utf-8?B?Wm4vOENXY2xDaGRDeDFlY3lENG1WQS93Q2dOSHd3WlFsekduWTJXRkZzblBD?=
 =?utf-8?B?MHVpR3hvWDdqUTFUQXRORmZQQ1RFQVBDUGgzU0VSVTMxQVNFUkdrKzRsV1kz?=
 =?utf-8?B?Y0l3Wm1kTURQblFoUTd5STA1ajBCbW56dmh5U1BudXZvT21LOVBNNTd4US9u?=
 =?utf-8?B?WHVqTGp6eWpkM3NRa3JhWkdyQ3E5YVFOd1B1cUVlYk51NkwxU1JQbExDT2dT?=
 =?utf-8?B?ZTYyUnBoeFA4M1E0OCtEWHpJWm9GcUIyNGNnclY3WnFOY1NrbE8rTGIzQk5W?=
 =?utf-8?B?eWV5MjRFbkZiNWIvSzFya3RBeHZqU3ZJWGw4aU1CcFBYYzlCc1hCNHo3QVpw?=
 =?utf-8?B?YXI3V0hxRzZSUXNPMzB5OTRBenA3ZXRxb2MzblRtMitaMVlpc3MzQ2dOalZs?=
 =?utf-8?B?NVVkVFVLRjdnbDgyckhETzAvOWdqL2UzZFhkVVdoTUlsbERIUTdNZUo1WENm?=
 =?utf-8?B?R295bEsrb2FwOENJUkRmQ2dpeVFiRFdLeWN0djNHTEUyRjByZU5uWXZ6bFJJ?=
 =?utf-8?B?ZUJxL1JZcXB4SUdhaEFoa0EvdS9aSVp6bmZUYmdTWHZhVkpES0JKeUY4bTJy?=
 =?utf-8?B?MEFuckRoK1NZOFZBNnZHc1NUSXlWWExiLzBJZTg1RXM5akVGMzZWc2ZrYW5i?=
 =?utf-8?B?bnlGWEdGWi9yOUFHYVJZTE9zOW5EMlhPMjFONGVkQlVLYTFtL3V2dXBpSXJ1?=
 =?utf-8?B?L2svWGZuWjlncUQyc0xzMFZKa3pTSFdDdTQwS212eGZ1NnV0Qk5veWlRaVZE?=
 =?utf-8?B?U3VvZ2VmamR4ampQL0UydzFMT3J0MFRMdzdXVUlKWERHSTF0WGpCcHUzN01O?=
 =?utf-8?B?RE9KVWR4R0JsdS9GYXhacnl0alEyU1ZuMlZycE13QlZ3RzgyUUJVQSswYjNY?=
 =?utf-8?B?Y3hrS2dwSzFCcnB3akpZeEsrMTRSSnE0SU1EbG5lYW9nMEtzVEdsMW1xM3pF?=
 =?utf-8?B?dUlEeFM4d2grRDF5RFpHOEN6RjM5Tlc4Nk90cFBxWkJWYkNqbldMTDdldnY3?=
 =?utf-8?B?OUZoSWJHWkdRekZyQ05uTlBIbkdOU3pMdk12VjM4dTkyWnA5eWc2REJKSnYv?=
 =?utf-8?B?NU11c1lXdXdNNklqUmd6SVh1aFhMM09oTlFpRThRdE1ZZHhBODdPOGlITk90?=
 =?utf-8?B?TUdIZlljNkVJRWFOSy9wdTFmM0luUU5Hb2Q4Tll3bmtJQjBjS3grUUI0bTBJ?=
 =?utf-8?B?bjB0VHpOSGgxcm40Mk42MHpZYW12K0VQaHM5ZnVSdEVYVXRWSG9tWjhwRFFs?=
 =?utf-8?B?MUZ1bVdaY3hmVU1ONVVpN1FWMVlHTGhoU050bjQ5Zi9TeVF2Z1dMUTdTL3Q5?=
 =?utf-8?B?cHZHbEZ5c01MajFhUXlvQWczSUhRaTl3dGJ6a25JUHdJZFJlN0dOTnB4YzI1?=
 =?utf-8?B?dnM3YWcwMm9EZjV1VlI5TURyVGczV2FiVXpOUk12SXduRzdWSjFnc1ZhV2Nk?=
 =?utf-8?B?bTJGbWNHcXhVbFcyVnF0RU4yZ0R1T2NjRlBvZDhyaFRGVFNiTGtia0VZcXpm?=
 =?utf-8?B?REF5UFoyc1ZTVXdJQkVHRTVtWWtXckFkSys5YXd6dTRoUzVFcGN1ckRtN3Vq?=
 =?utf-8?B?UlJ2em9KTnVXREs4YmFnZEt3MVJrZnFFdlZ4SDU0My9Gc0lidnBLWWplQ1hn?=
 =?utf-8?B?VHo0WnJiM0JOT3YvcDcwT3ZNcExNNGVDWWhjVVVLMUQ1SkhIZHUrdU9NNFp3?=
 =?utf-8?B?YXlPL1JHRHRsSHNsTEYxNWs3Z0JVZ082akdDYkNkN21wMkNQR2xodExlQWVD?=
 =?utf-8?B?aGl3eSs5MVhVOWpNRFFSMFozdC8zOHBuK0FtQ3J4STlCUFRVbis4VTVxMmlL?=
 =?utf-8?Q?pnHcyYhmAixf1?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 23:16:49.7110
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d0540ef-305b-4980-eead-08dc7922ed57
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A107.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6544

Hi Michal,

On 5/21/2024 12:09 AM, Michal Orzel wrote:
>>>> Thanks. I will take the tag if you are ok with above diff (for the case
>>>> if this series goes in later than Luca's).
>>> I would move this check to process_shm() right after "gbase = dt_read_paddr" setting.
>>> This would be the most natural placement for such a check.
>> That sounds good. Thanks! IIUC we only need to add the check for the
>> pbase != INVALID_PADDR case correct?
> Yes, but at the same time I wonder whether we should also return error if a user omits pbase
> for direct mapped domain.

I think this makes sense. So I will add also a check for the case if 
users omit pbase in the device tree for the direct mapped domain.

Kind regards,
Henry

>
> ~Michal



From xen-devel-bounces@lists.xenproject.org Mon May 20 23:20:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 23:20:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726410.1130689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CIR-0007Ol-Nb; Mon, 20 May 2024 23:19:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726410.1130689; Mon, 20 May 2024 23:19:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CIR-0007Oe-JU; Mon, 20 May 2024 23:19:59 +0000
Received: by outflank-mailman (input) for mailman id 726410;
 Mon, 20 May 2024 23:19:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9CIQ-0007OY-U3
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 23:19:58 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7915e3e9-16ff-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 01:19:57 +0200 (CEST)
Received: from BN8PR04CA0065.namprd04.prod.outlook.com (2603:10b6:408:d4::39)
 by LV3PR12MB9141.namprd12.prod.outlook.com (2603:10b6:408:1a7::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 20 May
 2024 23:19:53 +0000
Received: from BN2PEPF000044A1.namprd02.prod.outlook.com
 (2603:10b6:408:d4:cafe::ae) by BN8PR04CA0065.outlook.office365.com
 (2603:10b6:408:d4::39) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 23:19:53 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF000044A1.mail.protection.outlook.com (10.167.243.152) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 23:19:53 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 18:19:52 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 18:19:51 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7915e3e9-16ff-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nMvGLiV16EisY5V/scVTZ6cvRWJmGM8qr6LJF4tu+lgKCXmFTXAy70whLrg/h6PG1qh5f8Y+6fRs+Mp+Y0crAVkftuyi6bQilqyTyB2iTDzqTnhbACcXclm4MDEgNfzRE150b91U4IDETq9awaWZ9wdvfMvO/ySVGls4eJMkpAoPeH6zV1FM/V305c+9XM2hzR7rRmEazdihrZFcVzAubU2G399jHUYFRsW/qcNBD1ATGckS0EuK5gjPdqyWdhaOyCCIhqcroeKaAeB5CnyIhVwqfJs6By3X8Wd0qP6YyRgLLc+PUQ2voyrI5e58E5n3Lm92Vgskeb/Epc+EG0abxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=L7ZttqjF7bjrJVSr5FUkjnklGVSPmyTjxCFezK3B7A4=;
 b=gUmJzHenHCWXoNkOLB+VFOJkT7bVD9UxdtW5C2HXgSgoPo4EwJkq4JY3yr8zAlncYsk7jENYui1JaSO4hGYKqN/4DgnNPxQm2hBoVnPyksHBZeEwfhTvfyJlM4HRbUx2w8r4tcFMYIvj8Oke4CO3FK4hFnC189HtmvgxcVOajAZ1wSuywzeqa6FQggfmFCUprQXSxG6+GOxDpfD+5hKhmX8iLtTL8gqTUgxGWzGNT1ot6ficrEFQWrEPstkeflGlC2nUQTyRbVvqoBJL5beWxEmkexzPbhO9hLnZ1wa2R2QkqBqknDtBxADMeFILL8IHOkM3+b0W9zipqAp+NKsX7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L7ZttqjF7bjrJVSr5FUkjnklGVSPmyTjxCFezK3B7A4=;
 b=r1hbhMkoZgF/hINvFRS/7nKOB27snjbUsBt7gd1NQzubZYjravCqI/AuJRYZInT9XCgIiYz2vUFi0VY5oQjzLXjQ41nOryRawFpsQrfMhXs9IEqn6ohn7qzzIvtRvisRlG3E7cSjqXy7E128KJJJut61NGdqRGMJZkCJJSm5Dos=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <db6c9b30-e63b-4979-976c-9171a2942e81@amd.com>
Date: Tue, 21 May 2024 07:19:50 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/8] tools/arm: Introduce the "nr_spis" xl config entry
To: Jason Andryuk <jason.andryuk@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-5-xin.wang2@amd.com>
 <0a87ccc8-4262-4122-8142-2f64a174bab2@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <0a87ccc8-4262-4122-8142-2f64a174bab2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A1:EE_|LV3PR12MB9141:EE_
X-MS-Office365-Filtering-Correlation-Id: e73761d7-6620-46cc-88d8-08dc79235ab6
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?YW1DQ216TU1xRjBDQm5ZZG12MGhjaUhpNk9LV3FDY0NlckxSV1NWUTB0K0Np?=
 =?utf-8?B?TURMM09vcnhMVlYwQjRVSGQySzVxQVJQVDM5T0NqeGRNenR1OEVvM2pnV3Vw?=
 =?utf-8?B?aTRHZDVGM3ZxK3gyb2tZU0RKTFpjRkE4MnlCUzhMVyt3VzV5Y0oyaTZsZXQ3?=
 =?utf-8?B?bXJHdlBVSDZpR0lkN1RrT29MWkYrSHNWalpnT3QwdGJwbEVXeXJiMjJ1eXB4?=
 =?utf-8?B?Uzdac2pia3FTbldmaFhOWlZuVUNKSVVyYmIxUEVBWnlDZmNTVm03Ky8rMUNL?=
 =?utf-8?B?VWZJcTNXNEtDSXl5NW01WnlZRE5HQ2xhWnZsZWMyYTluYytqcjltRFpQaWtH?=
 =?utf-8?B?WjJEYjNMY1BWd3kvaEFROHdiYWNPdHk5Yy8wdFh4cWFDSkNRNHVPVzZaeGRS?=
 =?utf-8?B?bmpZMWJCZlA2TmdpVmlPTEN2amh6UTU1QldPcEtkNlZVVzl0Mld3Sk1JQ1hJ?=
 =?utf-8?B?bHBxWEZmYTcyb0MxUWhUb0xyeS9RRUtRbWRVbXY4M3EvNXQ1U0JsV0lwZ0Nu?=
 =?utf-8?B?UGhDdWZYZGR2RDc5aDlEUSs3dEx0ZlRDUDNaYWhnaFlsbnZMZjEvTHFPL0I3?=
 =?utf-8?B?aDJSZjRWanlLQ1l0UUY1WFo4N2g3QnBaMkhWVksrSUZCSUY2ZnhiTnBvWEoy?=
 =?utf-8?B?REFnWmxOM0lPU2hGK3hCUThiY3o3S0Y0Mjg3d1BET2hYcWV2TTZ1SXFGaC9h?=
 =?utf-8?B?U3kybGFrMml4N1dSekF5RGNWQ0ZoemhoMEU1ZlJ0eUI1V3Iyb2VZaTA1TXVl?=
 =?utf-8?B?ODJtSHNiWDVsR0V6cWJUbXIrazAzcnNTSXhTZUVuc0hHSzduNU9qSlBYaUo4?=
 =?utf-8?B?cXJ4MUx2VVdwN1Rham5WUlRZUm93Z2RiS2R0Z2FhbHpuM0JVWXpMbVV1RVEy?=
 =?utf-8?B?dFVOK3dLK01ENldtOVcwOEYwZlU0TzV4OWlyUGV3ZEhsNk0veWd4QVdIa0Mx?=
 =?utf-8?B?cTR0anM5ZG1QeXlwa1dZaitFWEkrRnB6NkJlYTJSRG5IKzBJNVpUYW1hQk5G?=
 =?utf-8?B?NEZiWStDc25WTUJEaVNmUTB0SVhVTEFtL2daRDJnRHdTWmZRSFpyaVdjTWdR?=
 =?utf-8?B?NlExemxTbkhrNGtXWDFlK3lIVUdGanlvUEszWTFwUjk3YWFIMWFZMUFvUmt5?=
 =?utf-8?B?MnFocUNHZ2p3bzdyeWcrK3hPak9KS0hmZ1ZuT0M0OFJMMThsUDY5emM5bG5R?=
 =?utf-8?B?U2p1TXB5Tkt1RHBLTmRKZWkwNUtsVE5RdUZPc2V0WGNtV3F4TG5EWjRjNC9U?=
 =?utf-8?B?Z0RTS1Nxelo2Zk4vZUlabTZDTE84c2xhbENmZHRHMGRRVS8rY1lkcDNyeDc1?=
 =?utf-8?B?VWVvTzFNSC9lelRyUkdEckEyd0RJQWsya0hHYjkxY0ZwV0hiazdVMDBjQ2FD?=
 =?utf-8?B?RkNycjRiL1liOTEyd1FQWDA4MUtQckNoSThzLzFRZ0lUbkdQM0tpVDN6RU02?=
 =?utf-8?B?R2llK3JhUkJQNEYxU212bnlZR0tGeUdwbW80YmdQM2JSZDdCTjFxbGJPYUxv?=
 =?utf-8?B?c0ovcm1YVm0wLzVNZjhwbVpvYlJsYWZCeGs1bjFFQjVjNk9UTnVFZWhNb05C?=
 =?utf-8?B?bEFkTmxUWUY1WWRXcnBWbGhpanI3eC95NzMzM21NN2kxMTlLMUpVRlVtZUJm?=
 =?utf-8?B?a3huN1JzYUpsdVZLeUpJYmluMUFOVVBSVktnbHJ2dlFKUkpNT0FoUjB0K2c3?=
 =?utf-8?B?d3pKcnU3L1F1SitVMzY5aDVYdmVTUFJtaG4wcnpoOWZvM0o2aWNjZHdTcW0x?=
 =?utf-8?B?TStyN1dpOGM0TXowWFJKc1RGNmN2eU1FK1NaS3p0bWhDSGFsTjJqcjZ0Q3Qy?=
 =?utf-8?B?bUFVS2hjY0dzMGhOMUpDMHRpRFNIU1lUSkxQcUMvYW9vOVVhU09FSmlOeHd6?=
 =?utf-8?Q?CFjudLmAFXzdU?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 23:19:53.2027
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e73761d7-6620-46cc-88d8-08dc79235ab6
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A1.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9141

Hi Jason,

On 5/21/2024 3:13 AM, Jason Andryuk wrote:
>> +
>> +=item B<nr_spis="NR_SPIS">
>> +
>> +A 32-bit optional integer parameter specifying the number of SPIs 
>> (Shared
>
> I'd phrase it "An optional 32-but integer"
>
>> +Peripheral Interrupts) to allocate for the domain. If the `nr_spis` 
>> parameter
>> +is missing, the max number of SPIs calculated by the toolstack based 
>> on the
>> +devices allocated for the domain will be used.
>
> This text says the maximum only applies if xl.cfg nr_spis is not setup.
>
>> +
>> +=back
>> +
>>   =head3 x86
>>     =over 4
>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index 1cb89fa584..a4029e3ac8 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>         LOG(DEBUG, "Configure the domain");
>>   -    config->arch.nr_spis = nr_spis;
>> -    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
>> +    config->arch.nr_spis = max(nr_spis, 
>> d_config->b_info.arch_arm.nr_spis);
>> +    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
>
> But this is always taking the max.  Should it instead be:
>
> config->arch.nr_spis = d_config->b_info.arch_arm.nr_spis ?: nr_spis;
>
> However, I don't know if that makes sense for ARM.  Does the hardware 
> nr_spis need to be a minimum for a domain?
>
> Really, we just want the documentation to match the code.

Before you pointed this out, I didn't realize the ambiguity in the doc 
about the "max". The "max" in the doc have different meanings compared 
to the "max()" in the code. I will drop the "max" in the doc and reword 
the doc to "If the `nr_spis` parameter is missing, the number of SPIs 
calculated by the toolstack based on the  devices allocated for the 
domain will be used.". Thanks for pointing it out.

Kind regards,
Henry

>
> Thanks,
> Jason



From xen-devel-bounces@lists.xenproject.org Mon May 20 23:21:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 May 2024 23:21:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726414.1130698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CJh-0000Ll-Vl; Mon, 20 May 2024 23:21:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726414.1130698; Mon, 20 May 2024 23:21:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9CJh-0000Le-Sn; Mon, 20 May 2024 23:21:17 +0000
Received: by outflank-mailman (input) for mailman id 726414;
 Mon, 20 May 2024 23:21:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jwTB=MX=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9CJg-0000LW-QV
 for xen-devel@lists.xenproject.org; Mon, 20 May 2024 23:21:16 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a610f54b-16ff-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 01:21:14 +0200 (CEST)
Received: from BN8PR12CA0024.namprd12.prod.outlook.com (2603:10b6:408:60::37)
 by PH7PR12MB6467.namprd12.prod.outlook.com (2603:10b6:510:1f5::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May
 2024 23:21:09 +0000
Received: from BN2PEPF000044A3.namprd02.prod.outlook.com
 (2603:10b6:408:60:cafe::91) by BN8PR12CA0024.outlook.office365.com
 (2603:10b6:408:60::37) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Mon, 20 May 2024 23:21:08 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF000044A3.mail.protection.outlook.com (10.167.243.154) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Mon, 20 May 2024 23:21:06 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 18:21:06 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 18:21:06 -0500
Received: from [172.31.124.47] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 18:21:04 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a610f54b-16ff-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SMqVKxYyb2f2ZHgg9VRsR3ANCwpOjnsTDHWucpnzn6x3jDYojO6Qs4SIP/fEPIeSJHayvI+H2GMQN1R4mvc822Tj/nnoGAtZLEpk2TcoIKFb0e/VvgvMz7NRc1mxb34Loeg4xOJLc9jQWdOwcMfiLEeXkII9LJizLzt2KaD7Q9e5GFbgflFGqnlGPxCQqbbYuMz8t+D2Tw0jElJp/ZVHqhIXjE7IS/xyB1z3P9jxSW2Q60FzLARd5hrNqdUzltM1DOdDcP36/9RU0raAnnxaLpOCSyj1zN6RZ7AbX2vNGiF7eZA/9iCfoCdCnSTNL/ofPkGupP2ufMLMwwe8xCDsnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EN3S4OosfqwRG9b51l+yCzFsbjpm/Dor86nu+LAVHcQ=;
 b=OLJGzgVbTgTZOwgnciE3yQz59YiA4g/H78uBMn7iGGdQwQK7uWWWdCKMxbrQXMXepRs3H+1Q5GprywlHydgg90c7mdpitFEIZdJrrdB98DvImqhBt9e9O2yTVD7wYtTg/el4mpEY90wiENaUkMpAIW2wByfeRr93Xa0/Xb6bfNXVwyj0FvDCxOqcNSJ1lrqL4LLqgCz4I71xlp4V9Ht6uihLwvYVwpvXEuLEhP6Sl5gtiBDgupUad7hHnZJvrO63cM7dU2zuUQE++0mWpsf4UMFyYqbgHKNCZ947fqQdL1CUvpNvcez2Q0LXXWAUjRiHNdq6V3jGIfb+lbke1/P0nA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EN3S4OosfqwRG9b51l+yCzFsbjpm/Dor86nu+LAVHcQ=;
 b=K0TNfFFKEysyiKuLLxKk6droWutxAbjw5KkExQAyZPFKHqUdqfUhLwJDpAdC1DzI9SCYzkkvylnEc4LKXam6NXzCE92LQzZOUCY3MVUh4+Wy1h9LKegQTOtDtQV+B+hOVJeQD16iWHqewxE4JOTfGFiETijjHyK4slMZq1EQsIY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <d7bb6db0-e27b-491c-ab52-0294d59b26eb@amd.com>
Date: Tue, 21 May 2024 07:21:04 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 7/8] tools: Introduce the "xl dt-overlay
 {attach,detach}" commands
To: Jason Andryuk <jason.andryuk@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240516100330.1433265-1-xin.wang2@amd.com>
 <20240516100330.1433265-8-xin.wang2@amd.com>
 <6a04ef17-2580-4f62-b377-aad2ebbcdb5b@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <6a04ef17-2580-4f62-b377-aad2ebbcdb5b@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A3:EE_|PH7PR12MB6467:EE_
X-MS-Office365-Filtering-Correlation-Id: 80e6291a-0712-4f10-caf0-08dc7923868b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?Mlp6UG44QU02U2I3OXNyZUE5QUVIWkVpY1d2cjNDZXZ0dWMxM05scHBoalVv?=
 =?utf-8?B?WDEva3lvbTBlb2dPU0poL1lYVDc4K2VwVXp1SDFTUkpUcDgwUS92ZU05Zms0?=
 =?utf-8?B?QUxkTFNGYmU5bXEwSm95WlQ2eWN6OEtPMzFjSU9NWG5sL1VNTGwwNEFSRzIx?=
 =?utf-8?B?VzZKT1dVc2J4WmNFNmJWUzhJck1KL1pDNTgwUWJFVy9uaEJoVmxIOVpYTm1y?=
 =?utf-8?B?ZVdEVVlINVVqMVZFYjdGQmVoMEFFUUdKTlNjYWRwSXZ2U2V2T2NWdng3V1hB?=
 =?utf-8?B?S1hxWWl2WEZtdG9KQzdTeUx0ZzJJb1hZd1pFQUg1NHBqN25vR0ZVMlBzK3Aw?=
 =?utf-8?B?VWxuelJiUFFtZ2Qycis5WGhBdzRxZlJPRDBqRksybXFHUzArUC9oY3RBNTlM?=
 =?utf-8?B?UTlvd2dJWHdSQm42WFlMdHpPWHBQdkxiTGJodFBzY1FSanlRT242UmpqN0dU?=
 =?utf-8?B?TEJIUDkvYkxMa2pMQ2N1VFBabWZTZmQzU3pJTWppdXROZ3l0VFN5eUVQTVBw?=
 =?utf-8?B?UGVqV1NPRkVweXJZMGpwL01WWWZUbldBS1kxRWs2ZHAzOXRPempCVzF2VG1v?=
 =?utf-8?B?T1pEcFFHWFByQlNNd3JxM0JSNnl4VzJZcU1PTDN5T0xPY0YxcnJnM1R6VzZz?=
 =?utf-8?B?Tk5vVk5ud1VueUNHNVRVSXBEYXlrVUlNMEdra1NTUGo5SVFaVXg3WUpCQ2RY?=
 =?utf-8?B?Z2Z2Wml1b1RvS04wZERHdjdscVlWU0hCdVJxQTR3eCtRTUtjM0hhNElsWUV1?=
 =?utf-8?B?U2EzVlpPZTFSN0FoQ3RvUU5CbjB2Z0xBbEY3c1NZMW12ZzF0Wno0cFhCSUVJ?=
 =?utf-8?B?R2RTSGNMOW81bW02bGgzMTJ4Q3R3TmhHY1BjejEydXViRThQV0x0OXdBdkNo?=
 =?utf-8?B?VXRUSm9hdFJMeS9hdXhGclFqMmMweXBMNmNFTU5SclJpbWozOWtBM3prWWVp?=
 =?utf-8?B?RWdqWVFCM2lzZ3hVamFoRVRsb3Qxd1MrVlBPQ1kyaytzRCsxQzh1NDVhMkZz?=
 =?utf-8?B?UEdJYmJwMHQxL1BWaFhiYlVlVkpwSkhPRnZ2TTBMQmlhSEkvcnBoQi9JZlA1?=
 =?utf-8?B?QXhZWmN4TnAzQWhzRGIrMkE4dEprNkpQVldvcjZod1NZOUczZElaMmFJNzV6?=
 =?utf-8?B?Zkw4Y1pXWFN1QlBveGtXWThGQnJUcnFwRk9HRHFsNDE5SGRRNEFHQ0ZEMmll?=
 =?utf-8?B?a2RKdERlY0N0QXgzcVVLelJYVVg3RUozdlgrNjJmMXB3aVNKNHZSV0diTFBF?=
 =?utf-8?B?RURNOVk3elNVWGRaSU9aNG9DdWpsL3BZNFliZCtNYW5iMGZZY3lFWFJwK2Ex?=
 =?utf-8?B?cTkybFpSZG8zQnV4M3prdk12cEpxa0JlYmlMWXBJMlJGeHNpM0FNTXo0U2Vm?=
 =?utf-8?B?SFlHeUtEWWJEdW5KYjhHdWdwdGk1RjVjcVMzVTJLY2g3dWdLakh1TGxqZDdY?=
 =?utf-8?B?c3gwbjlWS1pQa2xLQ24xMW1USUh5QzZqRjZUalVkeFpXUlEvMFBMWjZCYnVt?=
 =?utf-8?B?dkV3eHZveDQwNmI0WHlOWEQ4V0NsRU44ZFJHb3N3djJ4WlZ2SHpWYzEzSXhp?=
 =?utf-8?B?RXdpQnE1OWUrYnR2MWxnZ3dvTE0yWUJuWnJIckJDNDdWTHFZRHZpUEtEOTlY?=
 =?utf-8?B?dkNsMUllcndkbEROcVdBS0FnYW9oSjM2S3FydmNPR0RVRTkyazJEWEtNa0JF?=
 =?utf-8?B?NGdmWkJVRU94Q2pvbXNhbEs2MGcyTU84T3Z1dm4vRkJuSFZVQ2psemI5RkNB?=
 =?utf-8?B?bXkyclJUMlVLSE5mQjdNeXhWQmpIZFVLZkxZTkorUSszNUx4bWdYM0o3ck4x?=
 =?utf-8?Q?a1gA7w3JqH94verFT7eOEGZGRERFWdSWtk7Eg=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 23:21:06.7423
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 80e6291a-0712-4f10-caf0-08dc7923868b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A3.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6467

Hi Jason,

On 5/21/2024 3:41 AM, Jason Andryuk wrote:
> On 2024-05-16 06:03, Henry Wang wrote:
>> +            domain_id = strtol(argv[optind+2], NULL, 10);
>
> domain_id = find_domain(argv[optind+2]);

Good point, thanks. I will use the find_domain() in the next version.

Kind regards,
Henry

>
> And you'll get name resolution, too.
>
> Thanks,
> Jason



From xen-devel-bounces@lists.xenproject.org Tue May 21 01:20:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 01:20:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726434.1130709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9EAU-00052G-EV; Tue, 21 May 2024 01:19:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726434.1130709; Tue, 21 May 2024 01:19:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9EAU-000529-Ap; Tue, 21 May 2024 01:19:54 +0000
Received: by outflank-mailman (input) for mailman id 726434;
 Tue, 21 May 2024 01:19:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9EAT-00051z-5g; Tue, 21 May 2024 01:19:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9EAT-0006tB-2H; Tue, 21 May 2024 01:19:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9EAS-0006Mh-Pa; Tue, 21 May 2024 01:19:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9EAS-0006wF-Oz; Tue, 21 May 2024 01:19:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pmtskKEs/Cfl3l52xlu4Z5ZQM0g2hJkL+9sm3zLXOCs=; b=lbSMoiyu81ODdSxM76GWZG/I0H
	wTj14GTnkgZrIIF2ohdfTHGBvFMOvqX7TfBOS0tPj8m3zre/roHhR9Y7aT/mpijyAWWod0sK060Dp
	/YaWqjo3ADAh/WLjaC1UxeNr3S76zJ1HZnbFrBdC3t/aydOi4Tf7E1+x9o5NaBZVagUw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186047-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186047: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c
X-Osstest-Versions-That:
    linux=25f4874662fb0d43fc1d934dd7802b740ed2ab5f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 01:19:52 +0000

flight 186047 linux-linus real [real]
flight 186051 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186047/
http://logs.test-lab.xenproject.org/osstest/logs/186051/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 186051-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186051 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186051 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186036
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186036
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186036
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186036
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c
baseline version:
 linux                25f4874662fb0d43fc1d934dd7802b740ed2ab5f

Last test of basis   186036  2024-05-18 21:13:52 Z    2 days
Failing since        186038  2024-05-19 03:51:42 Z    1 days    5 attempts
Testing same since   186044  2024-05-20 05:14:37 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Chen, Tim C" <tim.c.chen@intel.com>
  "Huang, Ying" <ying.huang@intel.com>
  "King, Colin" <colin.king@intel.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Aleksandr Aprelkov <aaprelkov@usergate.com>
  Alex Rusuf <yorha.op@gmail.com>
  Alex Shi (tencent) <alexs@kernel.org>
  Alex Shi <alexs@kernel.org>
  Alexander Potapenko <glider@google.com>
  Alexandre Ghiti <alexghiti@rivosinc.com
  Alexei Starovoitov <ast@kernel.org>
  Alexey Dobriyan <adobriyan@gmail.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Axel Rasmussen <axelrasmussen@google.com>
  Baokun Li <libaokun1@huawei.com>
  Baolin Wang <baolin.wang@linux.alibaba.com>
  Baoquan He <bhe@redhat.com>
  Barry Song <v-songbaohua@oppo.com>
  Bart Van Assche <bvanassche@acm.org>
  Borislav Petkov (AMD) <bp@alien8.de>
  Borislav Petkov <bp@suse.de>
  Breno Leitao <leitao@debian.org>
  Brian Foster <bfoster@redhat.com>
  Brian Geffon <bgeffon@google.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chang S. Bae <chang.seok.bae@intel.com>
  Cheng Yu <serein.chengyu@huawei.com>
  Chris Li <chrisl@kernel.org>
  Christian Loehle <christian.loehle@arm.com>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Christophe Leroy <christophe.leroy@csgroup.eu>
  Chuck Lever <chuck.lever@oracle.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Hill <daniel@gluo.nz>
  Daniel Thompson <daniel.thompson@linaro.org>
  Dave Hansen <dave.hansen@linux.intel.com>
  David Hildenbrand <david@redhat.com>
  Dawei Li <daweilics@gmail.com>
  Dennis Zhou <dennis@kernel.org>
  Dev Jain <dev.jain@arm.com>
  Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Disha Goel <disgoel@linux.ibm.com>
  Donet Tom <donettom@linux.ibm.com>
  Douglas Anderson <dianders@chromium.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Edward Liaw <edliaw@google.com>
  Eric Sandeen <sandeen@redhat.com>
  Florian Rommel <mail@florommel.de>
  Frank van der Linden <fvdl@google.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  Hao Ge <gehao@kylinos.cn>
  Hao Xiang <hao.xiang@bytedance.com>
  Hariom Panthi <hariom1.p@samsung.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@gmx.de> [parisc]
  Heming Zhao <heming.zhao@suse.com>
  Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
  Hongbo Li <lihongbo22@huawei.com>
  Honggyu Kim <honggyu.kim@sk.com>
  Huang Shijie <shijie@os.amperecomputing.com>
  Huang Ying <ying.huang@intel.com>
  Ingo Molnar <mingo@kernel.org>
  James Houghton <jthoughton@google.com>
  Jan Kara <jack@suse.cz>
  Jan Kara <jack@suse.cz>               [jbd2]
  Jeff Layton <jlayton@kernel.org>
  Jinjiang Tu <tujinjiang@huawei.com>
  Joe Perches <joe@perches.com>
  Joel Granados <j.granados@samsung.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Hubbard <jhubbard@nvidia.com>
  Joseph Qi <joseph.qi@linux.alibaba.com>
  Justin Stitt <justinstitt@google.com>
  K Prateek Nayak <kprateek.nayak@amd.com>
  Kairui Song <kasong@tencent.com>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Klara Modin <klarasmodin@gmail.com>
  Konrad Dybcio <konrad.dybcio@linaro.org> # QC SM8550 QRD
  Kuan-Wei Chiu <visitorckw@gmail.com>
  Lance Yang <ioworker0@gmail.com>
  Len Brown <len.brown@intel.com>
  Li kunyu <kunyu@nfschina.com>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  linke li <lilinke99@qq.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Liu Shixin <liushixin2@huawei.com>
  Long Li <leo.lilong@huawei.com>
  Lorenzo Bianconi <lorenzo@kernel.org>
  Lucas Stach <l.stach@pengutronix.de>
  Luis Henriques <lhenriques@suse.de>
  Lukas Bulwahn <lbulwahn@redhat.com>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Matthew Wilcox <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Mel Gorman <mgorman@techsingularity.net>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Ellerman <mpe@ellerman.id.au> (powerpc)
  Michael Forney <mforney@mforney.org>
  Michal Hocko <mhocko@suse.com>
  Miguel Ojeda <ojeda@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Nhat Pham <nphamcs@gmail.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
  Oscar Salvador <osalvador@suse.de>
  Palmer Dabbelt <palmer@rivosinc.com>  [RISC-V]
  Pankaj Raghav <p.raghav@samsung.com>
  Patryk Wlazlyn <patryk.wlazlyn@linux.intel.com>
  Peter Xu <peterx@redhat.com>
  Petr Vorel <pvorel@suse.cz>
  Phillip Lougher <phillip@squashfs.org.uk>
  Randy Dunlap <rdunlap@infradead.org>
  Rasmus Villemoes <linux@rasmusvillemoes.dk>
  Ricardo B. Marliere <ricardo@marliere.net>
  Rick Edgecombe <rick.p.edgecombe@intel.com>
  Rik van Riel <riel@surriel.com>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Roman Gushchin <roman.gushchin@linux.dev>
  rulinhuang <rulin.huang@intel.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Saurav Shah <sauravshah.31@gmail.com>
  Sean Christopherson <seanjc@google.com>       [KVM]
  SeongJae Park <sj@kernel.org>
  Sergey Senozhatsky <senozhatsky@chromium.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Sidhartha Kumar <sidhartha.kumar@oracle.com>
  Song Liu <song@kernel.org>
  Stephen Smalley <stephen.smalley.work@gmail.com>
  Steve French <stfrench@microsoft.com>
  Su Yue <glass.su@suse.com>
  Sudeep Holla <sudeep.holla@arm.com>
  Suren Baghdasaryan <surenb@google.com>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Bertschinger <tahbertschinger@gmail.com>
  Thorsten Blum <thorsten.blum@toblux.com>
  Trond Myklebust <trond.myklebust@hammerspace.com>
  Usama Arif <usamaarif642@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vincent Guittot <vincent.guittot@linaro.org>
  Vishal Moola (Oracle) <vishal.moola@gmail.com>
  Vishal Verma <vishal.l.verma@intel.com>
  Vitalii Bursov <vitaly@bursov.com>
  Vladimir Benes <vbenes@redhat.com>
  Vlastimil Babka <vbabka@suse.cz>
  Waiman Long <longman@redhat.com>
  Wei Yang <richard.weiyang@gmail.com>
  Xining Xu <mac.xxn@outlook.com>
  Xiu Jianfeng <xiujianfeng@huawei.com>
  Yajun Deng <yajun.deng@linux.dev>
  Yang Li <yang.lee@linux.alibaba.com>
  Ye Bin <yebin10@huawei.com>
  York Jasper Niebuhr <yjnworkstation@gmail.com>
  Yosry Ahmed <yosryahmed@google.com>
  Youling Tang <tangyouling@kylinos.cn>
  Yu Zhao <yuzhao@google.com>
  Zhang Qiao <zhangqiao22@huawei.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhang Yi <yi.zhang@huawei.com>
  ZhangPeng <zhangpeng362@huawei.com>
  Zi Yan <ziy@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   25f4874662fb..eb6a9339efeb  eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 21 02:29:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 02:29:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726449.1130719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FFh-0004zy-GH; Tue, 21 May 2024 02:29:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726449.1130719; Tue, 21 May 2024 02:29:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FFh-0004zr-D4; Tue, 21 May 2024 02:29:21 +0000
Received: by outflank-mailman (input) for mailman id 726449;
 Tue, 21 May 2024 02:29:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wS2T=MY=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9FFf-0004zk-N6
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 02:29:19 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea34d8f4-1719-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 04:29:16 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 990C5CE0D88;
 Tue, 21 May 2024 02:29:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 437B5C32786;
 Tue, 21 May 2024 02:29:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea34d8f4-1719-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716258550;
	bh=P67AskWNTrdJ4dOXuRas5XfbseatRzLMeypvoV7gxow=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rf9VJ7CZtIZPF5kIdSo8B5lQe58RG2jITrphNs4frhsSaalr0MNaMSWL8/v2R7YBl
	 AHct3lU5JBtCfNq+ziiC7udR8+hV1k9hd2lkA49emiH0vAwVlbfrzMwt3pY1jzJcLf
	 ycFddWlUaCcZMB2LKMsy5UpegF1LQO/6u9we/2J/MEHMyhYwa5hGZmWABzwPhlTD9z
	 OLSwu/2XqGIY91EXHttDLBJuhBOMfn1VIIkJsdKxhtu8B1HGpcWDf/jHMdx7tFmBah
	 EeRbuGNuQTexH8hF7JguLWI8opKhLcRWTyDLj/VRAhYSgb216KmninsNsY3qcv3XBt
	 zspw9sBpCK04Q==
Date: Mon, 20 May 2024 19:29:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
In-Reply-To: <30c42949-bf67-416d-9f6f-e476fb1da436@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405201927050.1052252@ubuntu-linux-20-04-desktop>
References: <20240516100330.1433265-1-xin.wang2@amd.com> <20240516100330.1433265-6-xin.wang2@amd.com> <59e72623-00a7-4b19-9240-fb8c4982a381@amd.com> <9f086470-a17f-482a-ad98-814424da0ee5@xen.org> <c475fc50-3d30-49d4-b12d-31535cfe1306@amd.com>
 <30c42949-bf67-416d-9f6f-e476fb1da436@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1368579412-1716258550=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1368579412-1716258550=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 20 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> On 20/05/2024 02:01, Henry Wang wrote:
> > Hi Julien,
> > 
> > On 5/19/2024 7:08 PM, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 17/05/2024 07:03, Henry Wang wrote:
> > > > > @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, struct
> > > > > vcpu *v, unsigned int virq,
> > > > >       {
> > > > >           /* The VIRQ should not be already enabled by the guest */
> > > 
> > > This comment needs to be updated.
> > 
> > Yes, sorry. I will update this and the one in the new vGIC in v3.
> > 
> > > > >           if ( !p->desc &&
> > > > > -             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
> > > > > +             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> > > > > +             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
> > > > > +             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
> > > > >               p->desc = desc;
> > > > >           else
> > > > >               ret = -EBUSY;
> > > > >       }
> > > > >       else
> > > > >       {
> > > > > -        if ( desc && p->desc != desc )
> > > > > +        if ( desc && p->desc != desc &&
> > > > > +             (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
> > > > > +              test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) )
> > > > 
> > > > This should be
> > > > 
> > > > +        if ( (desc && p->desc != desc) ||
> > > > +             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
> > > > +             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
> > > Looking at gic_set_lr(), we first check p->desc, before setting
> > > IRQ_GUEST_VISIBLE.
> > > 
> > > I can't find a common lock, so what would guarantee that p->desc is not
> > > going to be used or IRQ_GUEST_VISIBLE set afterwards?
> > 
> > I think the gic_set_lr() is supposed to be called with v->arch.vgic.lock
> > taken, at least the current two callers (gic_raise_guest_irq() and
> > gic_restore_pending_irqs()) are doing it this way. Would this address your
> > concern? Thanks.
> 
> I don't think it would address my concern. AFAICT, the lock is not taken by
> vgic_connect_hw_irq().
> 
> I also haven't touched the vGIC for quite a while and didn't have much time to
> dig into the code. Hence why I didn't propose a fix.
> 
> The vGIC code was mainly written by Stefano, so maybe he will have an idea how
> this could be fixed.

I think we need to take the v->arch.vgic.lock just after the rank lock
in vgic_connect_hw_irq():

  vgic_lock_rank(v_target, rank, flags);
  spin_lock(&v_target->arch.vgic.lock);
--8323329-1368579412-1716258550=:1052252--


From xen-devel-bounces@lists.xenproject.org Tue May 21 02:58:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 02:58:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726458.1130735 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhL-0000WC-Ty; Tue, 21 May 2024 02:57:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726458.1130735; Tue, 21 May 2024 02:57:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhL-0000Vz-Ob; Tue, 21 May 2024 02:57:55 +0000
Received: by outflank-mailman (input) for mailman id 726458;
 Tue, 21 May 2024 02:57:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TC+7=MY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9FhK-0000Sf-JT
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 02:57:54 +0000
Received: from wfout6-smtp.messagingengine.com
 (wfout6-smtp.messagingengine.com [64.147.123.149])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e9d2f07a-171d-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 04:57:52 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id E21881C00115;
 Mon, 20 May 2024 22:57:50 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Mon, 20 May 2024 22:57:51 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 20 May 2024 22:57:49 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9d2f07a-171d-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1716260270; x=1716346670; bh=N0jz6LIRA8
	G2AVDWcwPsElyhd6eufRVEJ6uYH7nmGas=; b=BMtFBDKpfI8yZ+Y/FaPt+Y8hYu
	wod6qJ2D1R+iYSoUmvL+V9jBdYEO0lPBeHnKXYJeUb3L4XxR0ZiU/CuzjNAA2url
	8WYQFz5VPJFYTQzzE9mW31W0Metfm9mnFZCg/RJMssPAvk12TfvGnHljQBCH45H5
	nFe4jwP5l24Vw5A52jQ/OyaUvVkCBmrjsR0DdGVLzbdKzWc9hJxu38KN1pdnHXp1
	NcpZrsqjBC61Udtrv6/uQt0eJlqGiQR/ucP1vQDX5nvArPCYnqBsPn91JUFC2pOi
	pnRIaF+V++soJf+XUaxHz220NZD+9HKA/dn5UnwY0VoMLLDCYNSZtzSRA+NQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1716260270; x=
	1716346670; bh=N0jz6LIRA8G2AVDWcwPsElyhd6eufRVEJ6uYH7nmGas=; b=P
	w0dgrJe4NnxTIGVfPD6+nypqPFxbbdMlxkOK7lYqB1GuiT/CT9Z5d57WlVzPYjFM
	0XwdXH+ogM6xpKapz9AoxKoy6uk9c0ep6qc5bKWahlr0QbDBl5U7UWNn/xS+dYJa
	qqlFwCToflp9NggLfkD0FnUW8ffwJZoik1QwR/fbEIkQSglKlNRVaS2Aa0rSDQdj
	U9wlVJ+Wy5D7ebZ9rlgYTbYXChPpGDS67MyVYRfzeaqdZ26IpY1/VIdcA2JgPJzW
	UiesWNVvKmBz0OHR3nK6cVvv5MTdKMmRFNxpcvk/IgO4ius5uqLHMmEY/6z1L5Xp
	hV9yMc3HLQoW1Bi8j1sHg==
X-ME-Sender: <xms:rg1MZuqyk8rjyyEBaKmNOJZJU8_IsE0D5Pr1QUrc4_9BvW09miNLFA>
    <xme:rg1MZsqFTxdSobyuVRg7vGmt2O7IQqPZF9CWq021XSEFgp3MVS4xd2ON_oTzVfSd9
    wTMSFC3Eo9-WQ>
X-ME-Received: <xmr:rg1MZjPLp9g8PBfoSHzTtMppVihn7cVW80u9iO-AekZ8LkiH8jHEq0tBcDFlRBvUNGE-D7UxxNlYml9sJvLPlX1y9F1ZEITgjaIAumx3Pto3rks9cOE>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeiuddgieegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:rg1MZt7C6NHrKd54HbA5-WIMDhuHweeVBFKQXSlRfiZr8A7_WIvDzQ>
    <xmx:rg1MZt4wGuQNnb_M4FsCW8cbUtPlT0JvRLmYUxbsJQS8ed4T305ruQ>
    <xmx:rg1MZtjenLh5F_LvwkvjrpUnn6IbqD0yK4qktaLjBxEVWeCEl8PRjA>
    <xmx:rg1MZn6Dvrxg5u4TTcrSeKXl8KBEAOLeKxLjeKtAMN8MT4TuFm5DcA>
    <xmx:rg1MZg3EYUD_VMkD1z-2wR5Fbao7oaR9XCwRQ_tZwgfAqPY_VqELK0zn>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO page read only
Date: Tue, 21 May 2024 04:54:32 +0200
Message-ID: <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In some cases, only few registers on a page needs to be write-protected.
Examples include USB3 console (64 bytes worth of registers) or MSI-X's
PBA table (which doesn't need to span the whole table either), although
in the latter case the spec forbids placing other registers on the same
page. Current API allows only marking whole pages pages read-only,
which sometimes may cover other registers that guest may need to
write into.

Currently, when a guest tries to write to an MMIO page on the
mmio_ro_ranges, it's either immediately crashed on EPT violation - if
that's HVM, or if PV, it gets #PF. In case of Linux PV, if access was
from userspace (like, /dev/mem), it will try to fixup by updating page
tables (that Xen again will force to read-only) and will hit that #PF
again (looping endlessly). Both behaviors are undesirable if guest could
actually be allowed the write.

Introduce an API that allows marking part of a page read-only. Since
sub-page permissions are not a thing in page tables (they are in EPT,
but not granular enough), do this via emulation (or simply page fault
handler for PV) that handles writes that are supposed to be allowed.
The new subpage_mmio_ro_add() takes a start physical address and the
region size in bytes. Both start address and the size need to be 8-byte
aligned, as a practical simplification (allows using smaller bitmask,
and a smaller granularity isn't really necessary right now).
It will internally add relevant pages to mmio_ro_ranges, but if either
start or end address is not page-aligned, it additionally adds that page
to a list for sub-page R/O handling. The list holds a bitmask which
qwords are supposed to be read-only and an address where page is mapped
for write emulation - this mapping is done only on the first access. A
plain list is used instead of more efficient structure, because there
isn't supposed to be many pages needing this precise r/o control.

The mechanism this API is plugged in is slightly different for PV and
HVM. For both paths, it's plugged into mmio_ro_emulated_write(). For PV,
it's already called for #PF on read-only MMIO page. For HVM however, EPT
violation on p2m_mmio_direct page results in a direct domain_crash().
To reach mmio_ro_emulated_write(), change how write violations for
p2m_mmio_direct are handled - specifically, check if they relate to such
partially protected page via subpage_mmio_write_accept() and if so, call
hvm_emulate_one_mmio() for them too. This decodes what guest is trying
write and finally calls mmio_ro_emulated_write(). The EPT write
violation is detected as npfec.write_access and npfec.present both being
true (similar to other places), which may cover some other (future?)
cases - if that happens, emulator might get involved unnecessarily, but
since it's limited to pages marked with subpage_mmio_ro_add() only, the
impact is minimal.
Both of those paths need an MFN to which guest tried to write (to check
which part of the page is supposed to be read-only, and where
the page is mapped for writes). This information currently isn't
available directly in mmio_ro_emulated_write(), but in both cases it is
already resolved somewhere higher in the call tree. Pass it down to
mmio_ro_emulated_write() via new mmio_ro_emulate_ctxt.mfn field.

This may give a bit more access to the instruction emulator to HVM
guests, but only for pages explicitly marked with subpage_mmio_ro_add().
As of the next patch, it applies only configuration explicitly
documented as not security supported.

The subpage_mmio_ro_add() function cannot be called with overlapping
ranges, and on pages already added to mmio_ro_ranges separately.
Successful calls would result in correct handling, but error paths may
result in incorrect state (like pages removed from mmio_ro_ranges too
early). Debug build has asserts for relevant cases.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Shadow mode is not tested, but I don't expect it to work differently than
HAP in areas related to this patch.

Changes in v3:
- use unsigned int for loop iterators
- use __set_bit/__clear_bit when under spinlock
- avoid ioremap() under spinlock
- do not cast away const
- handle unaligned parameters in release build
- comment fixes
- remove RCU - the add functions are __init and actual usage is only
  much later after domains are running
- add checks overlapping ranges in debug build and document the
  limitations
- change subpage_mmio_ro_add() so the error path doesn't potentially
  remove pages from mmio_ro_ranges
- move printing message to avoid one goto in
  subpage_mmio_write_emulate()
Changes in v2:
- Simplify subpage_mmio_ro_add() parameters
- add to mmio_ro_ranges from within subpage_mmio_ro_add()
- use ioremap() instead of caller-provided fixmap
- use 8-bytes granularity (largest supported single write) and a bitmap
  instead of a rangeset
- clarify commit message
- change how it's plugged in for HVM domain, to not change the behavior for
  read-only parts (keep it hitting domain_crash(), instead of ignoring
  write)
- remove unused subpage_mmio_ro_remove()
---
 xen/arch/x86/hvm/emulate.c      |   2 +-
 xen/arch/x86/hvm/hvm.c          |   8 +-
 xen/arch/x86/include/asm/mm.h   |  18 ++-
 xen/arch/x86/mm.c               | 268 +++++++++++++++++++++++++++++++++-
 xen/arch/x86/pv/ro-page-fault.c |   1 +-
 5 files changed, 296 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index ab1bc516839a..e98513afc69b 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -2735,7 +2735,7 @@ int hvm_emulate_one_mmio(unsigned long mfn, unsigned long gla)
         .write      = mmio_ro_emulated_write,
         .validate   = hvmemul_validate,
     };
-    struct mmio_ro_emulate_ctxt mmio_ro_ctxt = { .cr2 = gla };
+    struct mmio_ro_emulate_ctxt mmio_ro_ctxt = { .cr2 = gla, .mfn = _mfn(mfn) };
     struct hvm_emulate_ctxt ctxt;
     const struct x86_emulate_ops *ops;
     unsigned int seg, bdf;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c530..9898e3fdb9ec 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -2009,6 +2009,14 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,
         goto out_put_gfn;
     }
 
+    if ( (p2mt == p2m_mmio_direct) && npfec.write_access && npfec.present &&
+         subpage_mmio_write_accept(mfn, gla) &&
+         (hvm_emulate_one_mmio(mfn_x(mfn), gla) == X86EMUL_OKAY) )
+    {
+        rc = 1;
+        goto out_put_gfn;
+    }
+
     /* If we fell through, the vcpu will retry now that access restrictions have
      * been removed. It may fault again if the p2m entry type still requires so.
      * Otherwise, this is an error condition. */
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 98b66edaca5e..a8bbed71013e 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -522,9 +522,27 @@ extern struct rangeset *mmio_ro_ranges;
 void memguard_guard_stack(void *p);
 void memguard_unguard_stack(void *p);
 
+/*
+ * Add more precise r/o marking for a MMIO page. Range specified here
+ * will still be R/O, but the rest of the page (not marked as R/O via another
+ * call) will have writes passed through.
+ * The start address and the size must be aligned to SUBPAGE_MMIO_RO_ALIGN.
+ *
+ * This API cannot be used for overlapping ranges, nor for pages already added
+ * to mmio_ro_ranges separately.
+ *
+ * Return values:
+ *  - negative: error
+ *  - 0: success
+ */
+#define SUBPAGE_MMIO_RO_ALIGN 8
+int subpage_mmio_ro_add(paddr_t start, size_t size);
+bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla);
+
 struct mmio_ro_emulate_ctxt {
         unsigned long cr2;
         unsigned int seg, bdf;
+        mfn_t mfn;
 };
 
 int cf_check mmio_ro_emulated_write(
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc7315..e601c8c98665 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -150,6 +150,17 @@ bool __read_mostly machine_to_phys_mapping_valid;
 
 struct rangeset *__read_mostly mmio_ro_ranges;
 
+/* Handling sub-page read-only MMIO regions */
+struct subpage_ro_range {
+    struct list_head list;
+    mfn_t mfn;
+    void __iomem *mapped;
+    DECLARE_BITMAP(ro_qwords, PAGE_SIZE / SUBPAGE_MMIO_RO_ALIGN);
+};
+
+static LIST_HEAD(subpage_ro_ranges);
+static DEFINE_SPINLOCK(subpage_ro_lock);
+
 static uint32_t base_disallow_mask;
 /* Global bit is allowed to be set on L1 PTEs. Intended for user mappings. */
 #define L1_DISALLOW_MASK ((base_disallow_mask | _PAGE_GNTTAB) & ~_PAGE_GLOBAL)
@@ -4910,6 +4921,260 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     return rc;
 }
 
+/*
+ * Mark part of the page as R/O.
+ * Returns:
+ * - 0 on success - first range in the page
+ * - 1 on success - subsequent range in the page
+ * - <0 on error
+ *
+ * This needs subpage_ro_lock already taken */
+static int __init subpage_mmio_ro_add_page(
+    mfn_t mfn, unsigned int offset_s, unsigned int offset_e)
+{
+    struct subpage_ro_range *entry = NULL, *iter;
+    unsigned int i;
+
+    list_for_each_entry(iter, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(iter->mfn, mfn) )
+        {
+            entry = iter;
+            break;
+        }
+    }
+    if ( !entry )
+    {
+        /* iter == NULL marks it was a newly allocated entry */
+        iter = NULL;
+        entry = xzalloc(struct subpage_ro_range);
+        if ( !entry )
+            return -ENOMEM;
+        entry->mfn = mfn;
+    }
+
+    for ( i = offset_s; i <= offset_e; i += SUBPAGE_MMIO_RO_ALIGN )
+    {
+        int oldbit = __test_and_set_bit(i / SUBPAGE_MMIO_RO_ALIGN,
+                                        entry->ro_qwords);
+        ASSERT(!oldbit);
+    }
+
+    if ( !iter )
+        list_add(&entry->list, &subpage_ro_ranges);
+
+    return iter ? 1 : 0;
+}
+
+/* This needs subpage_ro_lock already taken */
+static void __init subpage_mmio_ro_remove_page(
+    mfn_t mfn,
+    int offset_s,
+    int offset_e)
+{
+    struct subpage_ro_range *entry = NULL, *iter;
+    unsigned int i;
+
+    list_for_each_entry(iter, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(iter->mfn, mfn) )
+        {
+            entry = iter;
+            break;
+        }
+    }
+    if ( !entry )
+        return;
+
+    for ( i = offset_s; i <= offset_e; i += SUBPAGE_MMIO_RO_ALIGN )
+        __clear_bit(i / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords);
+
+    if ( !bitmap_empty(entry->ro_qwords, PAGE_SIZE / SUBPAGE_MMIO_RO_ALIGN) )
+        return;
+
+    list_del(&entry->list);
+    if ( entry->mapped )
+        iounmap(entry->mapped);
+    xfree(entry);
+}
+
+int __init subpage_mmio_ro_add(
+    paddr_t start,
+    size_t size)
+{
+    mfn_t mfn_start = maddr_to_mfn(start);
+    paddr_t end = start + size - 1;
+    mfn_t mfn_end = maddr_to_mfn(end);
+    unsigned int offset_end = 0;
+    int rc;
+    bool subpage_start, subpage_end;
+
+    ASSERT(IS_ALIGNED(start, SUBPAGE_MMIO_RO_ALIGN));
+    ASSERT(IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN));
+    if ( !IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN) )
+        size = ROUNDUP(size, SUBPAGE_MMIO_RO_ALIGN);
+
+    if ( !size )
+        return 0;
+
+    if ( mfn_eq(mfn_start, mfn_end) )
+    {
+        /* Both starting and ending parts handled at once */
+        subpage_start = PAGE_OFFSET(start) || PAGE_OFFSET(end) != PAGE_SIZE - 1;
+        subpage_end = false;
+    }
+    else
+    {
+        subpage_start = PAGE_OFFSET(start);
+        subpage_end = PAGE_OFFSET(end) != PAGE_SIZE - 1;
+    }
+
+    spin_lock(&subpage_ro_lock);
+
+    if ( subpage_start )
+    {
+        offset_end = mfn_eq(mfn_start, mfn_end) ?
+                     PAGE_OFFSET(end) :
+                     (PAGE_SIZE - 1);
+        rc = subpage_mmio_ro_add_page(mfn_start,
+                                      PAGE_OFFSET(start),
+                                      offset_end);
+        if ( rc < 0 )
+            goto err_unlock;
+        /* Check if not marking R/W part of a page intended to be fully R/O */
+        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
+                                                  mfn_x(mfn_start)));
+    }
+
+    if ( subpage_end )
+    {
+        rc = subpage_mmio_ro_add_page(mfn_end, 0, PAGE_OFFSET(end));
+        if ( rc < 0 )
+            goto err_unlock_remove;
+        /* Check if not marking R/W part of a page intended to be fully R/O */
+        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
+                                                  mfn_x(mfn_end)));
+    }
+
+    spin_unlock(&subpage_ro_lock);
+
+    rc = rangeset_add_range(mmio_ro_ranges, mfn_x(mfn_start), mfn_x(mfn_end));
+    if ( rc )
+        goto err_remove;
+
+    return 0;
+
+ err_remove:
+    spin_lock(&subpage_ro_lock);
+    if ( subpage_end )
+        subpage_mmio_ro_remove_page(mfn_end, 0, PAGE_OFFSET(end));
+ err_unlock_remove:
+    if ( subpage_start )
+        subpage_mmio_ro_remove_page(mfn_start, PAGE_OFFSET(start), offset_end);
+ err_unlock:
+    spin_unlock(&subpage_ro_lock);
+    return rc;
+}
+
+static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *entry)
+{
+    void __iomem *mapped_page;
+
+    if ( entry->mapped )
+        return entry->mapped;
+
+    mapped_page = ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE);
+
+    spin_lock(&subpage_ro_lock);
+    /* Re-check under the lock */
+    if ( entry->mapped )
+    {
+        spin_unlock(&subpage_ro_lock);
+        iounmap(mapped_page);
+        return entry->mapped;
+    }
+
+    entry->mapped = mapped_page;
+    spin_unlock(&subpage_ro_lock);
+    return entry->mapped;
+}
+
+static void subpage_mmio_write_emulate(
+    mfn_t mfn,
+    unsigned int offset,
+    const void *data,
+    unsigned int len)
+{
+    struct subpage_ro_range *entry;
+    void __iomem *addr;
+
+    list_for_each_entry(entry, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(entry->mfn, mfn) )
+        {
+            if ( test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords) )
+            {
+ write_ignored:
+                gprintk(XENLOG_WARNING,
+                        "ignoring write to R/O MMIO 0x%"PRI_mfn"%03x len %u\n",
+                        mfn_x(mfn), offset, len);
+                return;
+            }
+
+            addr = subpage_mmio_get_page(entry);
+            if ( !addr )
+            {
+                gprintk(XENLOG_ERR,
+                        "Failed to map page for MMIO write at 0x%"PRI_mfn"%03x\n",
+                        mfn_x(mfn), offset);
+                return;
+            }
+
+            switch ( len )
+            {
+            case 1:
+                writeb(*(const uint8_t*)data, addr);
+                break;
+            case 2:
+                writew(*(const uint16_t*)data, addr);
+                break;
+            case 4:
+                writel(*(const uint32_t*)data, addr);
+                break;
+            case 8:
+                writeq(*(const uint64_t*)data, addr);
+                break;
+            default:
+                /* mmio_ro_emulated_write() already validated the size */
+                ASSERT_UNREACHABLE();
+                goto write_ignored;
+            }
+            return;
+        }
+    }
+    /* Do not print message for pages without any writable parts. */
+}
+
+bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla)
+{
+    unsigned int offset = PAGE_OFFSET(gla);
+    const struct subpage_ro_range *entry;
+
+    list_for_each_entry_rcu(entry, &subpage_ro_ranges, list)
+        if ( mfn_eq(entry->mfn, mfn) &&
+             !test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords) )
+        {
+            /*
+             * We don't know the write size at this point yet, so it could be
+             * an unaligned write, but accept it here anyway and deal with it
+             * later.
+             */
+            return true;
+        }
+
+    return false;
+}
+
 int cf_check mmio_ro_emulated_write(
     enum x86_segment seg,
     unsigned long offset,
@@ -4928,6 +5193,9 @@ int cf_check mmio_ro_emulated_write(
         return X86EMUL_UNHANDLEABLE;
     }
 
+    subpage_mmio_write_emulate(mmio_ro_ctxt->mfn, PAGE_OFFSET(offset),
+                               p_data, bytes);
+
     return X86EMUL_OKAY;
 }
 
diff --git a/xen/arch/x86/pv/ro-page-fault.c b/xen/arch/x86/pv/ro-page-fault.c
index cad28ef928ad..578bb4caaf0b 100644
--- a/xen/arch/x86/pv/ro-page-fault.c
+++ b/xen/arch/x86/pv/ro-page-fault.c
@@ -330,6 +330,7 @@ static int mmio_ro_do_page_fault(struct x86_emulate_ctxt *ctxt,
             return X86EMUL_UNHANDLEABLE;
     }
 
+    mmio_ro_ctxt.mfn = mfn;
     ctxt->data = &mmio_ro_ctxt;
     if ( pci_ro_mmcfg_decode(mfn_x(mfn), &mmio_ro_ctxt.seg, &mmio_ro_ctxt.bdf) )
         return x86_emulate(ctxt, &mmcfg_intercept_ops);
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 21 02:58:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 02:58:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726459.1130748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhR-0000xo-7V; Tue, 21 May 2024 02:58:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726459.1130748; Tue, 21 May 2024 02:58:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhR-0000xh-4f; Tue, 21 May 2024 02:58:01 +0000
Received: by outflank-mailman (input) for mailman id 726459;
 Tue, 21 May 2024 02:58:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TC+7=MY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9FhQ-0000wX-FO
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 02:58:00 +0000
Received: from wfhigh1-smtp.messagingengine.com
 (wfhigh1-smtp.messagingengine.com [64.147.123.152])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eb5950c2-171d-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 04:57:57 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfhigh.west.internal (Postfix) with ESMTP id 59201180013A;
 Mon, 20 May 2024 22:57:53 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Mon, 20 May 2024 22:57:53 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 20 May 2024 22:57:51 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb5950c2-171d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1716260272; x=1716346672; bh=bXn4ATxgHs
	YUiQuSECyUBykp11qbPlmHmV2/8rOqouw=; b=eiy9AMiPteWokdkWidguJh47nh
	0B4klAge7QqkMwZGtvWYI2mxcZBCq85onYXASK+KITbnWw0XM/Oae/pUxPJ9oL74
	XpgnSzPuImdVvyjb7O7nc9zUvS6fUhxxv8EX5ejidnDloNEKfQYTL9B0uPBSfvNh
	t25hyAoQFt4upZIojddfR0Cdn5RNVKf5MzsZmy4xgusmdw94yzUp1htEfAkfAWpa
	/ehIgpQe+VHzmNb5V4gxWR+9QL2crQ3zJH+yZt0Naf0rTKJpyT7l5j091HLtnYXp
	7bqbA013DADGtqvwAFzbxIIpqXj07m3PJFnOXfTZ0E3PaICWFCEqow1Rd/zQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1716260272; x=
	1716346672; bh=bXn4ATxgHsYUiQuSECyUBykp11qbPlmHmV2/8rOqouw=; b=g
	6H2KkhfhzGK6xTenSbqWlqkf9jH8hG5TpaiQNVk9kZcSd9Re5IKYn3qY+hmcaA0n
	H46HiVSRRJVGCXhtH+Bi8UlDE+1gpWOX6sa2YLusjI8Q+ztn+m5lSRXslMFl8i0K
	MusamoSWPq4j+29ssWt2OpIEUwYRxO4lfYbDFlPk0OajEgfbUh20/d2cB3DY6+A3
	jymsn305ayHnIGZoXJK3VxPAXu878Z5+mSHfTzlv0Yu8xhkkVR5y4siQX299JgQR
	bxGDHMdg3WqzEC1TkKrfx2kMP3QePM4qgtSG4/OgVqB1PhSG+ZszYWytGmodk/xQ
	CdlIRex5M8lPIQUcxmNWw==
X-ME-Sender: <xms:sA1MZsyP7820szCgCD2Ip9Lr9VzDBUtO4KkS9ApTjNiZDZR3Zld6dg>
    <xme:sA1MZgSeN6pmyIzRRnF7XzRgdIL5iFONFwR8lhfsix-P45pLpTfkbcj4imxNTl68K
    EhH4AYJ9o44PQ>
X-ME-Received: <xmr:sA1MZuU3towJlYiXPD_El92gzLgxOtPZWgtui52gnb7AojphsD2CvAHGwh4qShsef5-a3xb_g1EToWqMZyZSja3PzKLbHzXjV7CPXOIkL0bNPvzZx4I>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeiuddgieegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:sA1MZqj2zAFcEbj2rne_xzsQ-2TQ_2tMOnPNbhIZPH7Ko6knj8V4SQ>
    <xmx:sA1MZuC5AzlXGZ_q0mdSnWA9KFW8kNbMADp27sSwoc1Kcb8zLcoSxA>
    <xmx:sA1MZrIA_LqDle6tWZN9EtyeuyIpD_Ii_X_Qw4vutsSeZ82notGxZw>
    <xmx:sA1MZlDJoXiyjG3EL-pQ6llBlaMUcfm0oPVLsbvgEXq5ZIiwXnHfYA>
    <xmx:sA1MZg2A0Hxks6a9kL6j2QP0QycvvJeth59q8xwhBpW1wwXyrZkENThg>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v3 2/2] drivers/char: Use sub-page ro API to make just xhci dbc cap RO
Date: Tue, 21 May 2024 04:54:33 +0200
Message-ID: <7682bbf3a24045ae1fcbdf651a6dd8e609ec3a20.1716260066.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Not the whole page, which may contain other registers too. The XHCI
specification describes DbC as designed to be controlled by a different
driver, but does not mandate placing registers on a separate page. In fact
on Tiger Lake and newer (at least), this page do contain other registers
that Linux tries to use. And with share=yes, a domU would use them too.
Without this patch, PV dom0 would fail to initialize the controller,
while HVM would be killed on EPT violation.

With `share=yes`, this patch gives domU more access to the emulator
(although a HVM with any emulated device already has plenty of it). This
configuration is already documented as unsafe with untrusted guests and
not security supported.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v3:
- indentation fix
- remove stale comment
- fallback to pci_ro_device() if subpage_mmio_ro_add() fails
- extend commit message
Changes in v2:
 - adjust for simplified subpage_mmio_ro_add() API
---
 xen/drivers/char/xhci-dbc.c | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 8e2037f1a5f7..cac9d90d3e39 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1216,20 +1216,19 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port)
         break;
     }
 #ifdef CONFIG_X86
-    /*
-     * This marks the whole page as R/O, which may include other registers
-     * unrelated to DbC. Xen needs only DbC area protected, but it seems
-     * Linux's XHCI driver (as of 5.18) works without writting to the whole
-     * page, so keep it simple.
-     */
-    if ( rangeset_add_range(mmio_ro_ranges,
-                PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
-                         uart->dbc.xhc_dbc_offset),
-                PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
-                       uart->dbc.xhc_dbc_offset +
-                sizeof(*uart->dbc.dbc_reg)) - 1) )
-        printk(XENLOG_INFO
-               "Error while adding MMIO range of device to mmio_ro_ranges\n");
+    if ( subpage_mmio_ro_add(
+             (uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+              uart->dbc.xhc_dbc_offset,
+             sizeof(*uart->dbc.dbc_reg)) )
+    {
+        printk(XENLOG_WARNING
+               "Error while marking MMIO range of XHCI console as R/O, "
+               "making the whole device R/O (share=no)\n");
+        if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
+            printk(XENLOG_WARNING
+                   "Failed to mark read-only %pp used for XHCI console\n",
+                   &uart->dbc.sbdf);
+    }
 #endif
 }
 
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 21 02:58:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 02:58:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726457.1130729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhL-0000Sx-KQ; Tue, 21 May 2024 02:57:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726457.1130729; Tue, 21 May 2024 02:57:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9FhL-0000Sq-Hg; Tue, 21 May 2024 02:57:55 +0000
Received: by outflank-mailman (input) for mailman id 726457;
 Tue, 21 May 2024 02:57:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TC+7=MY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9FhJ-0000Sf-UZ
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 02:57:54 +0000
Received: from wfout6-smtp.messagingengine.com
 (wfout6-smtp.messagingengine.com [64.147.123.149])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e8e86ebd-171d-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 04:57:51 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id AC5841C00122;
 Mon, 20 May 2024 22:57:48 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Mon, 20 May 2024 22:57:48 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 20 May 2024 22:57:47 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8e86ebd-171d-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1716260268; x=1716346668; bh=5iRq2U5WdpuhpARvdUR3L+8oSCsXiVO/
	kcVN8kDxkv4=; b=eTW30/trzT+rX9HtCzIDgPrIBxTAmxBMIaN9A9mgUmvEFN9x
	04+EMaOKsCPT4oaLWj1K8lQl/E+s7oxPWChDkmJpTaJpUidj9APL3658uneuCU3g
	CKlX2+5QfTI0PUbZVht1hmPLZvVgh9HGimv5w/OArwNZZIed5X/5lEyOvErdGruQ
	Fj8EmcI+yyeLlP9ata9YKh9zGhjHvZ7QNYcPzQ33vUs4yBOvfjpQpZbcCVhWdoTw
	YJ23a2WR42ZUAWm1Fq7tsPRnAKbVh5ATZ5gkQotYUbRK9TYsnqde+r4pKyd7cuQQ
	WgyyGo9QuC7oG8Gx8QWA/1M/r5Kwmo+VzwmFFQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1716260268; x=1716346668; bh=5iRq2U5Wdpuhp
	ARvdUR3L+8oSCsXiVO/kcVN8kDxkv4=; b=WZPdqBHfy5LVEUDp4kutm3OXo4eOI
	5xJr6OnHO1BI1QZcLu1q6nTOaZy0MHRXNYDUlR7HYvN0qbates0oSZckADsKKpnp
	iOScjF5ky2B5RlvYeIXrJlBcqK01dOkoznlV8ElafXVOnESRqNrWse1KQK1tkHGG
	gs64570AUMBJ48L4TCKUa6+96Pb3fByVWohkdRCfyrBQwROtiWPcw2y7vbBMizMx
	CYLoarKuSf6Cx1f4aOOKTlqdIiAKFlmei9g3OSHwO1RODTWBsts0vvBdalD9t7DX
	TILJEy2E979zDq8nvDYguRyZqBrsisA3StNn+ByUgB0Oh6CK8SacxHKpg==
X-ME-Sender: <xms:rA1MZo844NTgK4gHaUCo_zwU-9XBJHzbD8VPQhiEXvp0YFNubf6i7A>
    <xme:rA1MZgurEoRZm1hTw1Hh5TmJbK2ARZcNtEbKrqGebnsWMqXqLlwILGovXF-ZxI2mp
    Ql9-Z5RcBtVOA>
X-ME-Received: <xmr:rA1MZuANIzE7GsaL2Af5qixqqKqA9ba5_b-afDbA26hae7VhcdmbcvIBrIJL1BetYDZlUVlgHM9_onP5rwLe4uEL_0tdX2JHkhJesA98HSCDiiDWBnI>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeiuddgieegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggt
    khhiuceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
    eqnecuggftrfgrthhtvghrnhepleekhfduleetleelleetteevfeefteffkeetteejheel
    gfegkeelgeehhfdthedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg
    hilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdr
    tghomh
X-ME-Proxy: <xmx:rA1MZoeUx7mroKiafaipkIYRxA3XvhGxXj8jMGgfn3wl8aK60nvoEw>
    <xmx:rA1MZtNLI6TEQ-z9PRMu1F27b4hv19tlj6FwJDAnOTzTE28F19WorQ>
    <xmx:rA1MZil3kfqQfkildmfESZMrJfqprjVEBK8q5I46-6ybCvl5g1wHvQ>
    <xmx:rA1MZvsjaDsrgKCVBG_sdcnCi7HVUUVS4iZSkUHuUo3tzi2g53SRLg>
    <xmx:rA1MZr0J1FMN8Wz0bdrHuZ73AiQdEaH58kHXTWuoYnHj2yMxluRPr4kP>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH v3 0/2] Add API for making parts of a MMIO page R/O and use it in XHCI console
Date: Tue, 21 May 2024 04:54:31 +0200
Message-ID: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On older systems, XHCI xcap had a layout that no other (interesting) registers
were placed on the same page as the debug capability, so Linux was fine with
making the whole page R/O. But at least on Tiger Lake and Alder Lake, Linux
needs to write to some other registers on the same page too.

Add a generic API for making just parts of an MMIO page R/O and use it to fix
USB3 console with share=yes or share=hwdom options. More details in commit
messages.

Technically it may still qualify for 4.19, since v1 was sent well before
last posting date. But I realize it's quite late and it isn't top
priority series, so if it won't hit 4.19, it's okay with me too.

Marek Marczykowski-Górecki (2):
  x86/mm: add API for marking only part of a MMIO page read only
  drivers/char: Use sub-page ro API to make just xhci dbc cap RO

 xen/arch/x86/hvm/emulate.c      |   2 +-
 xen/arch/x86/hvm/hvm.c          |   8 +-
 xen/arch/x86/include/asm/mm.h   |  18 ++-
 xen/arch/x86/mm.c               | 268 +++++++++++++++++++++++++++++++++-
 xen/arch/x86/pv/ro-page-fault.c |   1 +-
 xen/drivers/char/xhci-dbc.c     |  27 +--
 6 files changed, 309 insertions(+), 15 deletions(-)

base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Tue May 21 03:49:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 03:49:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726482.1130758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9GV0-0008Uu-06; Tue, 21 May 2024 03:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726482.1130758; Tue, 21 May 2024 03:49:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9GUz-0008Un-Tm; Tue, 21 May 2024 03:49:13 +0000
Received: by outflank-mailman (input) for mailman id 726482;
 Tue, 21 May 2024 03:49:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9GUy-0008Ud-BY; Tue, 21 May 2024 03:49:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9GUy-0001JT-62; Tue, 21 May 2024 03:49:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9GUx-0003fF-SO; Tue, 21 May 2024 03:49:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9GUx-00028n-Rx; Tue, 21 May 2024 03:49:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UW9T21fOHhSL4xmDBcuSEio2u4DulyZTDdbaTl6QsQw=; b=IZIqeNugImrs6WqbhzXNxzEbBf
	W/Y1ENrw90vdRskogvofwwVwtfFoGgpu1FAY4ZlAdjxLSNFu2O3MeHiHpK4XkVSnYwLrZAlhaA8be
	snw14KfbguMv8j/9mK4GZtS9RS8NrTxwvnv9iqTIekN7p1S3K061Tj+YnOqGt30sWE54=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186050-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186050: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:test-armhf-armhf-xl:xen-boot:fail:regression
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9c5444b01ad51369bc09197a442a93d87b4b76f2
X-Osstest-Versions-That:
    xen=26b122e3bf8f3921d87312fbf5e7e13872ae92b0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 03:49:11 +0000

flight 186050 xen-unstable-smoke real [real]
flight 186053 xen-unstable-smoke real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186050/
http://logs.test-lab.xenproject.org/osstest/logs/186053/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl           8 xen-boot                 fail REGR. vs. 186048

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9c5444b01ad51369bc09197a442a93d87b4b76f2
baseline version:
 xen                  26b122e3bf8f3921d87312fbf5e7e13872ae92b0

Last test of basis   186048  2024-05-20 18:02:09 Z    0 days
Testing same since   186050  2024-05-20 22:02:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 9c5444b01ad51369bc09197a442a93d87b4b76f2
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Mon Sep 20 13:40:21 2021 +0100

    xen/trace: Drop old trace API
    
    With all users updated to the new API, drop the old API.  This includes all of
    asm/hvm/trace.h, which allows us to drop some includes.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>

commit aa60520cd37f0f81cec543af37b89c342e915dbd
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Tue Sep 21 19:55:47 2021 +0100

    xen/trace: Removal final {__,}trace_var() users in favour of the new API
    
    The cycles parameter (which gets removed as a consequence) determines whether
    trace() or trace_time() is used.
    
    No functional change.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>

commit 26da08059eef35e6ce1a308dbe276e8ef6a6c5b1
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Sat Sep 18 00:31:27 2021 +0100

    xen: Switch to new TRACE() API
    
    (Almost) no functional change.
    
     * In irq_move_cleanup_interrupt(), use the 'me' local variable rather than
       calling smp_processor_id() again.  This manifests as a minor code
       improvement.
     * In vlapic_update_timer() and lapic_rearm(), introduce a new 'timer_period'
       local variable to simplify the expressions used for both the trace and
       create_periodic_time() calls.
    
    All other differences in the compiled binary are to do with line numbers
    changing.
    
    Some conversion notes:
     * HVMTRACE_LONG_[234]D() and TRACE_2_LONG_[234]D() were latently buggy.  They
       blindly discard extra parameters, but luckily no users are impacted.  They
       are also obfuscated wrappers, depending on exactly one or two parameters
       being TRC_PAR_LONG() to compile successfully.
     * HVMTRACE_LONG_1D() behaves unlike its named companions, and takes exactly
       one 64bit parameter which it splits manually.  It's one user,
       vmx_cr_access()'s LMSW path, is gets adjusted.
     * TRACE_?D() and TRACE_2_LONG_*() change to TRACE_TIME() as cycles is always
       enabled.
     * HVMTRACE_ND() is opencoded for VMENTRY/VMEXIT records to include cycles.
       These are converted to TRACE_TIME(), with the old modifier parameter
       expressed as an OR at the callsite.  One callsite, svm_vmenter_helper() had
       a nested tb_init_done check, which is dropped.  (The optimiser also spotted
       this, which is why it doesn't manifest as a binary difference.)
     * All uses of *LONG() are either opencoded or swapped to using a struct, to
       avoid MISRA issues.
     * All HVMTRACE_?D() change to TRACE() as cycles is explicitly skipped.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>

commit d64693df1ab98f5278cf32ab3e0c12295237c42b
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Mon Sep 20 14:07:43 2021 +0100

    xen/sched: Clean up trace handling
    
    There is no need for bitfields anywhere - use more sensible types.  There is
    also no need to cast 'd' to (unsigned char *) before passing it to a function
    taking void *.  Switch to new trace_time() API.
    
    No functional change.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: Dario Faggioli <dfaggioli@suse.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>

commit df50666449a4ef0bc28f3118dc8adf1ecf5ba4a1
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 17 16:28:19 2021 +0100

    xen/rt: Clean up trace handling
    
    Most uses of bitfields and __packed are unnecessary.  There is also no need to
    cast 'd' to (unsigned char *) before passing it to a function taking void *.
    Switch to new trace_time() API.
    
    No functional change.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: Dario Faggioli <dfaggioli@suse.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>

commit ab8bc198507de47429e0b06dfd4e31c0d9461a34
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Wed Sep 15 17:01:43 2021 +0100

    xen/credit2: Clean up trace handling
    
    There is no need for bitfields anywhere - use types with an explicit width
    instead.  There is also no need to cast 'd' to (unsigned char *) before
    passing it to a function taking void *.  Switch to new trace_time() API.
    
    No functional change.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

commit e978b462ec9a74ec90788fca0b9d37707e1f79c4
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Mon Sep 20 13:36:12 2021 +0100

    xen/trace: Introduce new API
    
    trace() and trace_time(), in function form for struct arguments, and macro
    form for simple uint32_t list arguments.
    
    This will be used to clean up the mess of macros which exists throughout the
    codebase, as well as eventually dropping __trace_var().
    
    There is intentionally no macro to split a 64-bit parameter in the new API,
    for MISRA reasons.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: George Dunlap <george.dunlap@cloud.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue May 21 04:35:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:35:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726492.1130771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HDq-0006E3-9U; Tue, 21 May 2024 04:35:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726492.1130771; Tue, 21 May 2024 04:35:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HDq-0006Dw-6Q; Tue, 21 May 2024 04:35:34 +0000
Received: by outflank-mailman (input) for mailman id 726492;
 Tue, 21 May 2024 04:35:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9HDo-0006Dm-LE; Tue, 21 May 2024 04:35:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9HDo-0002PM-JG; Tue, 21 May 2024 04:35:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9HDo-0005wS-8V; Tue, 21 May 2024 04:35:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9HDo-00085m-89; Tue, 21 May 2024 04:35:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0ekCUrFjnAwt2wPQ4FTzVgXGEYFFTflLLhg5gr8mLqI=; b=hSpMlQq2YGdQY0oc668A0FQyJV
	HrxQCrqwtpILBZyvgaah9r4cpEmAEvWEVYYoogLC5XIool67ZTOHBBGu8HLdPB2U2WAPQEqvF8TJe
	bIt5S0k+hoVWguiaksMZcHhiw6aTH0+2qIoxb71EvazG8i0PjmuNxtzQSh793Iks+PyY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186054-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186054: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=7142e648416ff5d3eac6c6d607874805f5de0ca8
X-Osstest-Versions-That:
    ovmf=284dbac43da752ee34825c8b3f6f9e8281cb5a19
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 04:35:32 +0000

flight 186054 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186054/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 7142e648416ff5d3eac6c6d607874805f5de0ca8
baseline version:
 ovmf                 284dbac43da752ee34825c8b3f6f9e8281cb5a19

Last test of basis   186026  2024-05-17 05:43:05 Z    3 days
Testing same since   186054  2024-05-21 02:43:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   284dbac43d..7142e64841  7142e648416ff5d3eac6c6d607874805f5de0ca8 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726497.1130782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HES-0006iG-I0; Tue, 21 May 2024 04:36:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726497.1130782; Tue, 21 May 2024 04:36:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HES-0006i9-FN; Tue, 21 May 2024 04:36:12 +0000
Received: by outflank-mailman (input) for mailman id 726497;
 Tue, 21 May 2024 04:36:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HER-0006eS-MK
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:11 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1b8813a-172b-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 06:36:05 +0200 (CEST)
Received: from SA1PR03CA0024.namprd03.prod.outlook.com (2603:10b6:806:2d3::25)
 by SA3PR12MB9131.namprd12.prod.outlook.com (2603:10b6:806:395::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 04:36:01 +0000
Received: from SA2PEPF0000150A.namprd04.prod.outlook.com
 (2603:10b6:806:2d3:cafe::1) by SA1PR03CA0024.outlook.office365.com
 (2603:10b6:806:2d3::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Tue, 21 May 2024 04:36:01 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF0000150A.mail.protection.outlook.com (10.167.242.42) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:01 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:00 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:35:58 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1b8813a-172b-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Myx5ijFldV5n+x68EZsN/wlpKmjooLyCxovle+spapZz0vh8DS5+ZmLNweZBgKz7NilMXVmTHJmhS1qOE3wjU/kLKkNSNybBqfupDgi5jQYJByeXHSa1WWRaqsh1y9XMDQEbKk4oOqGOLvjxwGibN+lPsc6AAPy+33KA1J2h7RqvHfILyUQbHj6SR5zpxhehNcueCqj2nIShJOPzN+D6GB3eMbv2sq4hOP21ghzkOhUqq4YaeNBQRPVvuGLiUW8RsVW06K/VuFLq3YCB6xxDhJxXCzO6ZVo84x5cW4MsUA++mHEkQRuPP7DsENNkX3hdY0ZxhlLhUIulZ8a+8cvMyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=QIYn95b97cEL4fKyfXhO9HNIpzTSdg19jXxuBX8v++o=;
 b=fqbX2mLlchUdVFwqEc44At9yVDyBa/4BnCV/QZ/anGOPhfeLJC2A4r+2P/zZDSrdZcXkfPDTMJp57UAFyvwvhikAbKADmXhf8K2UlTwnxw7EbLB5xkFkk0fr8RLUybhco+ui36lm4KOfCx4D8PkegoB9dFv0ieHeRrVnXoDLomXcbrzh5Z222L/pCt+RvWgKth82lyYHkun1ja8m0fi+fN3g4CRcqkjqcoffYzuqTJZlTSOFfgnlMjNWP1J5fNbEsyfKVrrdGlGxLBzxanGnCaeI3JeL7BfK+XTZwnpvxg4NO30Jn50pp1DyvJl3Feh31yxyKMoPqMBw0FqapNL/DA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QIYn95b97cEL4fKyfXhO9HNIpzTSdg19jXxuBX8v++o=;
 b=juT00ZDuRl4w9BbPPBU4fSnx7VnUAo03E4qgBBQDI5H3sqHiy6UNQK89qrxaL2m0fEr9/zV+NFkNActSZY9oDJn99sunz4vc/1hIkbX3VzLKtWS6cjnLsH7Ft30xMM7VhfnzlWMiuBQ//ZL3X1WASw/HMyNRJYPRb+pVLHT8Few=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Anthony PERARD <anthony@xenproject.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH v3 0/8] Remaining patches for dynamic node programming using overlay dtbo
Date: Tue, 21 May 2024 12:35:49 +0800
Message-ID: <20240521043557.1580753-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF0000150A:EE_|SA3PR12MB9131:EE_
X-MS-Office365-Filtering-Correlation-Id: 02e6fc97-21fb-4db0-a85b-08dc794f8483
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015|7416005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?BIpanMFS4WIWKlK6B1hQhbYBGqw+kLn1eWt8KNATmXK5dwlgvrVwX4Pq1ovs?=
 =?us-ascii?Q?BXN5Vkuka6vrXA3lGjEg9PKuTwKRQlz2fCRY1zqlbX+rYABqJieLIqtv4oBA?=
 =?us-ascii?Q?9/0rY1H934XJptuYCEvh3L38KYWdEAD/807LsuQQfNoT1rpKPJew8uK+CXgx?=
 =?us-ascii?Q?81IYjhViEB/2A5CzUIyk3k0/kARo5e7Xry4b1tUZf686hs7CcunQ91guC0EY?=
 =?us-ascii?Q?SerhZKAlstvIYDXUW5Tx8jv4O3nbJxHE3q+Thk32sQ452FbHNsVstcqIosrR?=
 =?us-ascii?Q?GdAqVjNIWb1rnbsbZvvyB875SMI4rltZrJ6jPy0LgcVGdyeVV9aSVmq/hqHS?=
 =?us-ascii?Q?aAze7XkCA1KwfwsrV5ys5JGhHqJPWB2KfGTPYc7BL9Fm6tkW3yzXwf6m9JcU?=
 =?us-ascii?Q?NjR2MIGl2TvExYqpsnAVnRXIlqXSuYkjADD23ZUFAks5EA0BbaN+4uR898ZB?=
 =?us-ascii?Q?NZw5AZDOE0ylfOj+aCOl13w4OsEJwo2Z/coWoPT4kbHQ90x6Iz5PIf6aI/T7?=
 =?us-ascii?Q?MQRSTGtskk6QLxVNMV2zdCpWar5IxAenodtQ4V4X2fwUHhe4QbQ3BLrtce5w?=
 =?us-ascii?Q?nDF3xAdU+2iL/8Bcd0wCwVtC5fIVFdfCNEbqGenkS0Pbml2C1NLxdQ4Pwfn9?=
 =?us-ascii?Q?pt/6K8GaBmgkkd2CAWjfGF3/6WXyrv7dPyA92a7lvS3sYJZDhJU1UfUw3DLz?=
 =?us-ascii?Q?9tK0aCxQYsgXWzU7Bvb1uBcnn60AVF8TU201p6/RoACEs2Z0/e9om7L7Q1y5?=
 =?us-ascii?Q?GaokhNbEz78i99tCH7ozfEJnBn/oe12few+4MG75WceRejcsd+n2/2jAwXBV?=
 =?us-ascii?Q?thSDVDXlPyMFcNSJA696u1qDps+mckQm3LETNP1yY/cP6F6/G62cYMuN9GIv?=
 =?us-ascii?Q?ES0OhGxkQGcHI6Cgt1UihT1PJNnW0G/UXi9NLiO6Jl3UjC7WoCrMPyNycE9W?=
 =?us-ascii?Q?scSQqyyzy09JrwxVE5uHiyYMQhhl9GOFaraREF2BMk6Yu1T9giVHWN0ypeU1?=
 =?us-ascii?Q?2ANBG7i7wmOIaDy4dHKCsJO1wYxdfYk6XbpDsN+Tv6O3mvcySBoYAq+fY7qB?=
 =?us-ascii?Q?DOp+tYNrrkA8nDgc8WMXfLEe6bhoBeEAC2y7mZJMG8fBr0nRy5TVTHG8Ul9I?=
 =?us-ascii?Q?l0lZqkSDcwIiBsVKjbeMovwiDc0aSmVL9HyRSErLoSz3swuxwIAoW+DLnzrl?=
 =?us-ascii?Q?rSzj88qourp857L3Fyymlq19jhRrPsvjzi9ro5yqc6BvSjR/rgPGaALMHCQc?=
 =?us-ascii?Q?y7NFjW17SuN2lcBdOGwQ2S3ETjxGhib7WOmiFYlJ+0x+ZX2mcv37KpCD6vpZ?=
 =?us-ascii?Q?EnZYA+iv920C5T+2j3iUItIQm3F7FFRSP4XSFXSkucg/2g=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:01.1740
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 02e6fc97-21fb-4db0-a85b-08dc794f8483
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF0000150A.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB9131

Hi all,

This is the remaining series for the full functional "dynamic node
programming using overlay dtbo" feature. The first part [1] has
already been merged.

Quoting from the original series, the first part has already made
Xen aware of new device tree node which means updating the dt_host
with overlay node information, and in this series, the goal is to
map IRQ and IOMMU during runtime, where we will do the actual IOMMU
and IRQ mapping and unmapping to a running domain. Also, documentation
of the "dynamic node programming using overlay dtbo" feature is added.

Patch 1 and 2 are fixes of the existing code which is noticed during
my local tests, details please see the commit message.

Gitlab CI for this series can be found in [2].

[1] https://lore.kernel.org/xen-devel/20230906011631.30310-1-vikram.garhwal@amd.com/
[2] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1298425517

Henry Wang (6):
  xen/common/dt-overlay: Fix lock issue when add/remove the device
  tools/xl: Correct the help information and exit code of the dt-overlay
    command
  xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
  tools/arm: Introduce the "nr_spis" xl config entry
  xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations
  tools: Introduce the "xl dt-overlay {attach,detach}" commands

Vikram Garhwal (2):
  xen/arm/gic: Allow routing/removing interrupt to running VMs
  docs: Add device tree overlay documentation

 docs/man/xl.cfg.5.pod.in              |  14 +
 docs/misc/arm/device-tree/booting.txt |  16 +
 docs/misc/arm/overlay.txt             |  99 ++++++
 tools/golang/xenlight/helpers.gen.go  |   2 +
 tools/golang/xenlight/types.gen.go    |   1 +
 tools/include/libxl.h                 |  10 +
 tools/include/xenctrl.h               |   3 +
 tools/libs/ctrl/xc_dt_overlay.c       |  31 ++
 tools/libs/light/libxl_arm.c          |   4 +-
 tools/libs/light/libxl_dt_overlay.c   |  28 ++
 tools/libs/light/libxl_types.idl      |   1 +
 tools/xl/xl_cmdtable.c                |   4 +-
 tools/xl/xl_parse.c                   |   3 +
 tools/xl/xl_vmcontrol.c               |  48 ++-
 xen/arch/arm/dom0less-build.c         |  11 +-
 xen/arch/arm/domctl.c                 |   3 +
 xen/arch/arm/gic-vgic.c               |  15 +-
 xen/arch/arm/gic.c                    |  15 -
 xen/arch/arm/vgic/vgic.c              |  10 +-
 xen/common/dt-overlay.c               | 441 ++++++++++++++++++++------
 xen/include/public/domctl.h           |  15 +
 xen/include/public/sysctl.h           |  11 +-
 xen/include/xen/dt-overlay.h          |   7 +
 23 files changed, 644 insertions(+), 148 deletions(-)
 create mode 100644 docs/misc/arm/overlay.txt

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726498.1130788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HES-0006lZ-UE; Tue, 21 May 2024 04:36:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726498.1130788; Tue, 21 May 2024 04:36:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HES-0006l6-MJ; Tue, 21 May 2024 04:36:12 +0000
Received: by outflank-mailman (input) for mailman id 726498;
 Tue, 21 May 2024 04:36:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HES-0006hu-3W
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:12 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4385c6f-172b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 06:36:08 +0200 (CEST)
Received: from SA9PR13CA0057.namprd13.prod.outlook.com (2603:10b6:806:22::32)
 by DM6PR12MB4252.namprd12.prod.outlook.com (2603:10b6:5:211::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May
 2024 04:36:06 +0000
Received: from SA2PEPF00001507.namprd04.prod.outlook.com
 (2603:10b6:806:22:cafe::46) by SA9PR13CA0057.outlook.office365.com
 (2603:10b6:806:22::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 04:36:06 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:06 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:03 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:02 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4385c6f-172b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gBWTSlySG9p9Ae6RjL0C+Kr51gF7ITt0cBlVY003oDwljuC3e9lwBPuxooeRQdBOWr3H94GjZkD41AkAb+rjvf46Eic8gferqRxss5IujoWAnoqf56bpwxcAckFBOq8f8lP/wa08KDAk6BQxP48bDJBpAjJHffOg6FwFnRYnadvEkwdpjYOgKc17RrP3ekfw6IiHIRoGMEyZxdMKJgUCLX7J/8rc3vJCX+l0RuygBaMx+i+KABiQMoa+vmhfeCQgLRZ71Q/nT8Lf4P9VzFa4MZTl19nm5Ssy3SoeewJNnqxTPMPoUiZu2B3vGX+13OSI4k5gn3YA3GC3cPRh5nCUmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6MgtBjlO1A8ilW5oFOin/8HiTsVKeBou6t2YnvlwNys=;
 b=oeHMEoOkkpR1cbiCvXZL4B/zvvoKPa3pGw6i4hVo0NuA7I1pOQSHJyWqia0FcCiYpzdzErUvp/phEaUtZ/ps8uz3bC2V0T47KX7FZgFcH8Szsf4dnq9zBdNf/rcOk0ZGKBSz7uGBfnj0//PzUr/bCDms0MNPbu+cs0+XGwNt1Yvqz39STI2f77t7QXfGv4cdnRiUUU0Kx+X/0iubBUFla7/MtHzeW3P0d2HtVnV/PSd0l1aDfCzPv1IWxRmTsfb3App75fGTZ6qtj13ToDdkpSXMQzXWwWa4/gmqhHQmIHmc0zSukJgDeBrD8EyVGmzTS+2AAJtFGyPZZ0HkJSfzyw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6MgtBjlO1A8ilW5oFOin/8HiTsVKeBou6t2YnvlwNys=;
 b=oRW02LtyEF6Ox24cpEq+YTKbIq6Hr82X3gFFA2uJZ33twsZLH+coinuNIRl/FdLFnNaz/hUnKGvf9RjBcQksFHZi3u5a4RjkHC5dwyr2NfnRWlR6q0Mt/DB3iT2ppJjcyFZtzW/5Zy6Rx76OM0pGS8COg3AtiF94FH15zRTWq94=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Henry Wang <xin.wang2@amd.com>,
	Jason Andryuk <jason.andryuk@amd.com>
Subject: [PATCH v3 2/8] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Tue, 21 May 2024 12:35:51 +0800
Message-ID: <20240521043557.1580753-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|DM6PR12MB4252:EE_
X-MS-Office365-Filtering-Correlation-Id: effbd565-7baf-44a3-f460-08dc794f8791
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?cz8HrDpXqiDvLVcZTeBBGTQdK1ZTcd9yoVmYA4g/cC/7cYlKEJ9SqPkw7lUY?=
 =?us-ascii?Q?jbHTO6IsWNsBf3/8pj1kStE+2s7Ri5acSonboWaveYOvpFpSOUJpqe+DpAfx?=
 =?us-ascii?Q?sxTyRxmdto+5UZuJKFYZJ/NneEUDwSecZLmGYlWrKR5ToCTex4mWLCmO9ddf?=
 =?us-ascii?Q?s4VKu1FJLpNHbPxihnD4idiyrTCdUWiOagJtY/75L4uwK0sSp1HVdbFDbJtA?=
 =?us-ascii?Q?vC8tLImRNNUcHDlpleKU+3pnzfvkTH9sxn7nuREh1rF5otmLh2zZBhpY8MpD?=
 =?us-ascii?Q?FJ6lYdmrzjI3w/HNqs3imEpkBzGoNLDI7Ep66RFCQOhNmCrboS+IBNDO5Xcr?=
 =?us-ascii?Q?C0jOILIaV3P3xa2TDGRmxHE8NbNg91HrsKcx/KxuN6AI8adwOkpwCoN0pCOI?=
 =?us-ascii?Q?NnNVu0SOxeeWy0TkeEG9XYg+kPtR+7s3iU6nwaedY3ub8IzGBHpCDBBB4daE?=
 =?us-ascii?Q?06ZvYXA69i1ta7HRRzZ8EX5fZ/089ywT1Omx5ZwZ3FHOX3eXaFPtOGxdssTL?=
 =?us-ascii?Q?zGwuQ3sKphsFWLuhjM7cN83grRw6x/khiTEn/EjGkYgt82TE479GTE0A0EiW?=
 =?us-ascii?Q?myLjHqlX0ZY6nXEyfEQCfHoQMPIvUPigxPNt17r6RVn07QrqoCOSLwhT/im6?=
 =?us-ascii?Q?lGnq814NIVHQoWpEtPbXWsrBON9NWLFTHGw1qd5gdDpAjKGDyORLleLLHmQs?=
 =?us-ascii?Q?H2sNb0yr+Mf4LBcoR4tl+MSBVSpF/nMTweJ4TLHFF0pnIznZkLKurCmRCKAf?=
 =?us-ascii?Q?XFlyrezmGGB1QomkVFwDnAL4Gyj5hKhLi8CWlDaq6obW9Y2/EVlfJHeQEPXN?=
 =?us-ascii?Q?DjWH3no1Nf7Uzg9l7icKHqtGWA3HbhIAS6MBUu1wy1ZWmAIdyLFWOsA8K1Tv?=
 =?us-ascii?Q?iw3ORBEvh2XlU7SsN8+fPI/KFUJLLBosfq15wWS/sXBVTBewJGOuv8pFOnvx?=
 =?us-ascii?Q?4q1turwr9w5t4mIFTDNpTIeicvNkqUUFYtQ0hz+SQiZgxrhookwDGRHS5S3l?=
 =?us-ascii?Q?B8BLjcj1nrJu13u4HK4WUoP/s2D+YAZruEUfiayJrJRMnKINdwxkPzCAyaQk?=
 =?us-ascii?Q?BSojwVmKen6lBc2GeOidLYN/EhDXfWW/jNbCb5sMY4r2Z1Meby0ZK065GNws?=
 =?us-ascii?Q?52rNr8sOWaNBhh9FMxygQa8jSxi6vsy4EviTh0RufkRR7d8VCNVHSV1ZWGHK?=
 =?us-ascii?Q?NGbko7XOnbc0jCYVqrTM8gR8QCTMw84slNTIUgVV5uJw2KWLOQ4RREZs7O5h?=
 =?us-ascii?Q?DnoilTjtjVLO75GO7kHZ7YL6RknP/2b3WVC7uDf9RVTFb9JC24FrDklPUfI9?=
 =?us-ascii?Q?wnohQ2cKC1OXteqrb53bmnNLunLCt1/ufxDoBrgkRfkpX41YPcnRbHYqps3X?=
 =?us-ascii?Q?cxfIYmY=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:06.3009
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: effbd565-7baf-44a3-f460-08dc794f8791
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001507.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4252

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".
Add the missing "," in the cmdtable.

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony@xenproject.org>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
- Add Jason's Reviewed-by tag.
v2:
- New patch
---
 tools/xl/xl_cmdtable.c  | 2 +-
 tools/xl/xl_vmcontrol.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1f3c6b5897 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
       "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>"
+      "add/remove <.dtbo>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726499.1130802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEU-0007CE-8J; Tue, 21 May 2024 04:36:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726499.1130802; Tue, 21 May 2024 04:36:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEU-0007C3-4I; Tue, 21 May 2024 04:36:14 +0000
Received: by outflank-mailman (input) for mailman id 726499;
 Tue, 21 May 2024 04:36:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HES-0006eS-MX
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:12 +0000
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2407::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a52348c6-172b-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 06:36:10 +0200 (CEST)
Received: from SA9PR13CA0045.namprd13.prod.outlook.com (2603:10b6:806:22::20)
 by CH3PR12MB7499.namprd12.prod.outlook.com (2603:10b6:610:142::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May
 2024 04:36:05 +0000
Received: from SA2PEPF00001507.namprd04.prod.outlook.com
 (2603:10b6:806:22:cafe::22) by SA9PR13CA0045.outlook.office365.com
 (2603:10b6:806:22::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 04:36:05 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:04 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:02 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:01 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a52348c6-172b-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZbHT8CJrUUQY5hs5537SOck+tLTRvrprlKSawMwfvSCMiwpbiKfn4KsvUGkEpLx2W1oIbUuZ5AI2j8ISgrJB/an5qViiNdtVfH6smVhqZg3ai4jEGiXcIQU+0yuwLX7B4TmT/RF7gh0CiT1Z9bNuYwJvaovfKt8wOIbugPkfP5KHAzeyuPhFvo6iXZ5bYGUXMi2JEzSfIjMcANzygWeWtobzrqzt7aYjVkN0EV6Ucv3nX2Xat/vot1DWcmjdfEUw1PBSR4xvhVvPyuYrnxS3kQrOAdUpcYTULPxc82Lp2bS+C6o9iEcbnCL4RCts4ymVn6XYMJr67meK1SO7sE2ZTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kBS9p4N9tMcJAJGNweW/eDIIQQk4+w4cUu2XcNfNYwg=;
 b=TQjlVF8XqVN7gQWYuaMlIM1P77lTW2lHcGKf0XbJPvH4jXv/fj4gatbk+VED5CKR0+2y5icmSeMx1MOLBv5JG8802TAizCgomcwrIjw1EYJtlHhqFcFDHoYrshWGbNFFGfZUbeaTlfp7MY/KO7XCycRbivGYUduvRvIHX0Gsta4HeV2m3IvaoTpSOALCL6y7OnYkgMnCzgEikMxv4qxks2ji8X5sbeDLNtveC6h64h37rcqzxFQfBBW4UdhtUiEnYpQSGBzJcmM8pTAbB8VEemTNPNM0xhIH/DMeArrASVXvsxLAPG8sjHb46B0GOx3SrokwlqpD12KONnsPgQKdRA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kBS9p4N9tMcJAJGNweW/eDIIQQk4+w4cUu2XcNfNYwg=;
 b=m6XNOcaiydaSRylGgFvLSPAujP/G2QNPXWxNeL3XajMYzEgztIjAJH8Kyhbs+xQQdl+K2KRwNycmM2n4Afsa/hBk9nbepNG2ATUcgiqW1oJNUt3egpyRd8MY3/fLp71qEyZxkQJejQA9yrwf6YogMrVgP13ENgDFDhqCi4UIJbQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Henry Wang <xin.wang2@amd.com>, Julien Grall
	<jgrall@amazon.com>
Subject: [PATCH v3 1/8] xen/common/dt-overlay: Fix lock issue when add/remove the device
Date: Tue, 21 May 2024 12:35:50 +0800
Message-ID: <20240521043557.1580753-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|CH3PR12MB7499:EE_
X-MS-Office365-Filtering-Correlation-Id: 227e150c-a10e-4550-9bf4-08dc794f86b6
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?sQqQ87UkyLKxgaGUSQWaQotMsMysHeA3ERrkt1ttq82ARRub86jbMIa+TVab?=
 =?us-ascii?Q?KFatSTw5BrVfMfr9n0Ltym/gB0tTd+Gs7INIkUKy5bQ8pglJlQoxqK/h+e8v?=
 =?us-ascii?Q?PrA+ilqa2/pVqjlbaecjvHKZDxf04RqvYJLvEEwYegxob7UrJaWHNlNvu7b1?=
 =?us-ascii?Q?cZ+bIhGPLYWjFNJmwDscAqZ44lHVehD0Qg5u6faqtPa6DoFG229+xpjuHQ7I?=
 =?us-ascii?Q?DRUp3oKaJiy9/hNEHQ2Yar5ee88HEMndUAC/gNMAVtglv9dYRU9ZDrGIO+D3?=
 =?us-ascii?Q?KYEjlk1UDoczh03+P+YGs2agmLqd7rb2JWRCk90/zxWZ7ROBLk0vk7vvdu95?=
 =?us-ascii?Q?UQhpyUUP2KICR0FSpZfRPDCCDs/9xEvXBHwmwmj+v6fzVeIFy+LZ3aEZIJ8g?=
 =?us-ascii?Q?xBKWkFryIKECPWiGN0+LREGja6FCEojX5EX0VZ8klQWCfBIUYv1F9HB6WV7B?=
 =?us-ascii?Q?/SaS7bnjZBGrnt0ETeAQ0ScNYO53TXD/lFQtrW7EXCeV0kgVzwAmCrSTtp2y?=
 =?us-ascii?Q?tLDIV9TcxAMZCigcY7BufVol5IEhw+VDwQIYZcHtDQPCPsE7/gOQ5Wo0pm9b?=
 =?us-ascii?Q?JHFZ/wjKCAS+SiQtwIyQhWfXQ58ji0Xx2EEc7bKxQZfrVyMKh0ECcUDw1XP3?=
 =?us-ascii?Q?3ijGP+UBJ2TqpIqbSPM5Yyvn+VL356p1HRx60EL4DRlNbDg6ZA7H9g6n31yW?=
 =?us-ascii?Q?QI3mxEnH+R5zJJiLXSjXLN4kXbSURda/dCeFyosOsW7yQmohT7Fo+67IeK5/?=
 =?us-ascii?Q?oH2GMdpSpEMFqJlv1K3k5CYvKNvoW6Okse6w5fG17Z+I09MXkgmxKVayftjK?=
 =?us-ascii?Q?V96xrYbiaUjmYUWLcoA+uiQG0s1CbBr8PHcFYeH7FxtCxRD8/Wd2nJssHtgS?=
 =?us-ascii?Q?s9oUpHJarssSVQztRxSBLhtEF1QNT+KgDwgei6XUxCSIRCX5rGAt6TFdTq31?=
 =?us-ascii?Q?H+ihqMGN8e01g/bLIPNRtg7OSWcsn8ec5FVesqJPfqcJPobQleSBBoUOXIzu?=
 =?us-ascii?Q?yhj3hAq+yU9hdeaEp5MUrTPUNJpw8yHbSZefevJ9RLNS6ToAXL1aqCHrjU+5?=
 =?us-ascii?Q?tFIyzHBE7kHK5O1PCDnOvvGKyGJsCDgkFed+9BpcgVkILPHpTiK0Qq7EoAaA?=
 =?us-ascii?Q?+fL3+M9VCS1wnotFL3oSSRSd9fj0nAxgQE1/1KXUkWW7FbH9ZGSKL6CS+4Ee?=
 =?us-ascii?Q?MzilOnwp8CLZkTujlMrIrHa0umRlK3wsfu91Qn2FYfT7B9yl/J6lrcmSukZr?=
 =?us-ascii?Q?KnTaZh7ddQiVtmMsJakV+wwXDDTi0IoDP+xdFAhTA8zL/wZ5n4pI0UQkCePz?=
 =?us-ascii?Q?JON7qmwpkjOnDXj8lROUSrrMm3tlZEnX2xx1nawnK9ZPR9K5IKYe6SPcvOWJ?=
 =?us-ascii?Q?ejlapFo=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:04.8790
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 227e150c-a10e-4550-9bf4-08dc794f86b6
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001507.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7499

If CONFIG_DEBUG=y, below assertion will be triggered:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
[...]
(XEN) Xen call trace:
(XEN)    [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC)
(XEN)    [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR)
(XEN)    [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90
(XEN)    [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648
(XEN)    [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68
(XEN)    [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c
(XEN)    [<00000a0000230b40>] do_sysctl+0x96c/0x9ec
(XEN)    [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288
(XEN)    [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c
(XEN)    [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146
(XEN) ****************************************

This is because iommu_remove_dt_device() is called without taking the
dt_host_lock. dt_host_lock is meant to ensure that the DT node will not
disappear behind back. So fix the issue by taking the lock as soon as
getting hold of overlay_node.

Similar issue will be observed in adding the dtbo:
(XEN) Assertion 'system_state < SYS_STATE_active || rw_is_locked(&dt_host_lock)'
failed at xen-source/xen/drivers/passthrough/device_tree.c:192
(XEN) ----[ Xen-4.19-unstable  arm64  debug=y  Not tainted ]----
[...]
(XEN) Xen call trace:
(XEN)    [<00000a00002594f4>] iommu_add_dt_device+0x7c/0x17c (PC)
(XEN)    [<00000a0000259494>] iommu_add_dt_device+0x1c/0x17c (LR)
(XEN)    [<00000a0000267db4>] handle_device+0x68/0x1e8
(XEN)    [<00000a0000208ba8>] dt_overlay_sysctl+0x9d4/0xb84
(XEN)    [<00000a000027342c>] arch_do_sysctl+0x24/0x38
(XEN)    [<00000a0000231ac8>] do_sysctl+0x9ac/0xa34
(XEN)    [<00000a0000274b70>] traps.c#do_trap_hypercall+0x230/0x2dc
(XEN)    [<00000a0000276330>] do_trap_guest_sync+0x478/0x688
(XEN)    [<00000a000025e480>] entry.o#guest_sync_slowpath+0xa8/0xd8

This is because the lock is released too early. So fix the issue by
releasing the lock after handle_device().

Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities")
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
v3:
- Add Julien's Reviewed-by tag.
v2:
- Take the lock as soon as getting hold of overlay_node. Also
  release the lock after handle_device() when adding dtbo.
v1.1:
- Move the unlock position before the check of rc.
---
 xen/common/dt-overlay.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 1b197381f6..9cece79067 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -429,18 +429,24 @@ static int remove_nodes(const struct overlay_track *tracker)
         if ( overlay_node == NULL )
             return -EINVAL;
 
+        write_lock(&dt_host_lock);
+
         rc = remove_descendant_nodes_resources(overlay_node);
         if ( rc )
+        {
+            write_unlock(&dt_host_lock);
             return rc;
+        }
 
         rc = remove_node_resources(overlay_node);
         if ( rc )
+        {
+            write_unlock(&dt_host_lock);
             return rc;
+        }
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
 
-        write_lock(&dt_host_lock);
-
         rc = dt_overlay_remove_node(overlay_node);
         if ( rc )
         {
@@ -604,8 +610,6 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return rc;
         }
 
-        write_unlock(&dt_host_lock);
-
         prev_node->allnext = next_node;
 
         overlay_node = dt_find_node_by_path(overlay_node->full_name);
@@ -619,6 +623,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
         rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
                            tr->iomem_ranges,
                            tr->irq_ranges);
+        write_unlock(&dt_host_lock);
         if ( rc )
         {
             printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726500.1130811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEV-0007Ru-GJ; Tue, 21 May 2024 04:36:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726500.1130811; Tue, 21 May 2024 04:36:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEV-0007Rg-Ce; Tue, 21 May 2024 04:36:15 +0000
Received: by outflank-mailman (input) for mailman id 726500;
 Tue, 21 May 2024 04:36:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HET-0006hu-G0
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:13 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a60a278c-172b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 06:36:11 +0200 (CEST)
Received: from SN6PR08CA0013.namprd08.prod.outlook.com (2603:10b6:805:66::26)
 by MN2PR12MB4255.namprd12.prod.outlook.com (2603:10b6:208:198::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May
 2024 04:36:09 +0000
Received: from SA2PEPF00001508.namprd04.prod.outlook.com
 (2603:10b6:805:66:cafe::2e) by SN6PR08CA0013.outlook.office365.com
 (2603:10b6:805:66::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Tue, 21 May 2024 04:36:08 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001508.mail.protection.outlook.com (10.167.242.40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:08 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:07 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:06 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a60a278c-172b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gDQjP1EqJXQPNeWNjdHFD9UehBjTc+UWgjTa9vgYXI5ZQCbOtn+60VAWSKe9LxxenbyeBXZ4ztO0My6g1GkHh7Dy3xtltz9v2nc7bPcWmnwGw0jWR7Lz/ciW83WYrESMZ8Gkf8X3w3IYTVogJjQxqSXcj3amd3fQzuiLTPewRBidgiCLSy6mQ7Mp7wSNdsn2Mm0h6U8nYb5cgHtho3vp4nLhv6+EM1RjSgoojqHA6KqzdRLPVwTekZ5wFmh50mIM8BQ5itvWDKCbF7/Yo/qQX9fmATX/x6MOiKNi7MmeCVy7/unqfu9i2W9fijRGBmClMndmBhyDg/kUEoe++zKVNg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Wmo94nWRzmCLwFNiswsI5NwrIKUvcZBhuU9kpPno8cI=;
 b=MuTKnAyNK2o8xX7o6SDp6A891mMXTG44akktQgjKKRmyd3MFHO2qAHd+KF5M7HlpJJUICPEwh/sGFqwNzy8sdju06CoRLISMvW/T2XxRn09voqtuGreKhAjyNiG6LkPP0G+DHfA6QB1M4jyfU6tMDZjEtcNNDul4lMJn8sKnI/Hd8xPuNHkd6a/NNo8pcTGBecuL2+Cdag9aSgKc2Y0fZVd1GkVAJkMfSelmaMQcFegEqwHLuOTh3QstkZP3pFHWDgh3FvIF8gJOZpf366ghIpqtm31QqqOWtWu4USk6g9cPAynizE+T/YQxUjh9kOVS7dCjm5lBIIlSmYr1gVulSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Wmo94nWRzmCLwFNiswsI5NwrIKUvcZBhuU9kpPno8cI=;
 b=VtzGigXWO1TXy7+LJWzdx/zB5q8KIXX3ilObFOh7Y64la3MLf2C5wgA75ZDrQCjK/8/q0DmT4Nu1kq4uO32j5OOGUdWts4wm1t7eQmUhKF4giGY52vQedkOV4jIwzLu3ZA250d65e9dCwq5JYEv2TUhF+wZP+4gf11rfKAl3XxU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Henry Wang <xin.wang2@amd.com>
Subject: [PATCH v3 4/8] tools/arm: Introduce the "nr_spis" xl config entry
Date: Tue, 21 May 2024 12:35:53 +0800
Message-ID: <20240521043557.1580753-5-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001508:EE_|MN2PR12MB4255:EE_
X-MS-Office365-Filtering-Correlation-Id: b5279e61-7b56-48d6-445c-08dc794f88da
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?aeCy5ecntFfY3oyUBQxWx8biEGZ0cvsquN4WWY6TASVCBEwTeb33K4l/+GSO?=
 =?us-ascii?Q?hxZvWSd3Eqd9f9+tgXLrK/fKaYSKfWoYJSET52GuFHKq/FL1UsIAOEi2HFLY?=
 =?us-ascii?Q?QNCJsJ7V8HghPC0E11SEhYKA2kIWJ/vc3sv8vBcvlijeIiqG27C6HXsfGw+U?=
 =?us-ascii?Q?rzwoQwJ5yordmPMya18GB6QEpN9qapYeG6/GIyJANmEBCv2OkIov2h6lctlG?=
 =?us-ascii?Q?K96bcH5UlGVAk7gQZUoLRxT57XPwVMx9GwY1rATUWUgI5KhZ1FpR+qZCstXM?=
 =?us-ascii?Q?WhClPcfGIpsU3hjIaxX4YqDlVNbByrmj4OK730wqyKntdephWtfDYTHok5q6?=
 =?us-ascii?Q?Bu+Bp2eFzq05ibOYtS9G5urFJrowdhjBgxH9SWy3MUW3xz9Ik02be76RCjHH?=
 =?us-ascii?Q?5YOCfTvZnpbzihWWFI6oIFct28UQlD+r2qZq0ngj3va17cCFoENuTV0hNSQX?=
 =?us-ascii?Q?4e7vTw97xI8d2GGSgzZP7r3aaTmcdIs4Wd9XefPQMkEBUh8+Ao2/Tf4aEDN5?=
 =?us-ascii?Q?8SbiLz1DaYlfma0HifwOeUDQ8Q+OKEBhuKmt8FUEruyZrtiAT0wDOweYmvjs?=
 =?us-ascii?Q?J2qvNAYblK8/8aR6RmJE5d7HsUaA5xT5bC9zo00PS3GlD0ikGgWUw2/wBJey?=
 =?us-ascii?Q?xPXVo0gvV55SwenAAOEsQbqwaJu118E+w7DfVbQVfD/5oKeWHT3M0/r5kKoM?=
 =?us-ascii?Q?kl9bySgiFPL3OTPbcMChG4SOBcO8LR1cYcZ6XE+O4rTL/mWcAYR4sP1RycZk?=
 =?us-ascii?Q?70QKhF/HWfpDAzk0sZETUuF1Ge3XV+wtCAYF01MwGPJ2MdvsGCDWqQuloR1u?=
 =?us-ascii?Q?kLcggf3eg7KPmzcv/QBncENRN4KdQkE8+tZhGBucDE7ltt5pNjQ2T9biCVgt?=
 =?us-ascii?Q?b0n8Xrvy1fJsLcjvT4IcdPrrEdfV6HxLabI2JhEA58ZK7rfi2YcKaVrvGLfe?=
 =?us-ascii?Q?03rzuA2qC0wAk63vPcP16ZX7yxMRtnm0h+0XAm8HYxyWMriN8Qf32f6FnKxo?=
 =?us-ascii?Q?JJiVoAY3E+ZHg5fQWZYPqrqop98P76fzqo51i91xsME33XnI20QXZ/IaazhP?=
 =?us-ascii?Q?terScFBDMTvy7YiorjU5Q9XzdEMk5/++MIgy4HooFhnjs0F3pmEY2x+UOS+J?=
 =?us-ascii?Q?39NvyMFoCy0sq+uEvU0wAfry3sEPtTOEUgsHFKi31/cga5I9rJemX7wDZaAU?=
 =?us-ascii?Q?DAQy5h8lmdgZh/uGhMU0kz9DL0384zv1mDxli4xKGN0OF3FkvUkGXV/eFNho?=
 =?us-ascii?Q?kwFwVx0VPimtfaae5XLy4U1SJ/Id2jlTF22Y9DHTWLlCIoDkVQP3Ot0y+IOp?=
 =?us-ascii?Q?QMsf+Pyk8o7wBeRd5hYrBvfVGy/neT7N/074CK58wN7mU7DcgP171BOrSA4R?=
 =?us-ascii?Q?o7XbjG2lj6hiU81hKRsFbO3fp+gB?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:08.4682
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b5279e61-7b56-48d6-445c-08dc794f88da
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001508.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4255

Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.

Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.

Update the doc and the golang bindings accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Reword documentation to avoid ambiguity.
v2:
- New patch to replace the original patch in v1:
  "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
---
 docs/man/xl.cfg.5.pod.in             | 14 ++++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/libs/light/libxl_arm.c         |  4 ++--
 tools/libs/light/libxl_types.idl     |  1 +
 tools/xl/xl_parse.c                  |  3 +++
 6 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..416d582844 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,20 @@ raised.
 
 =back
 
+=over 4
+
+=item B<nr_spis="NR_SPIS">
+
+An optional 32-bit integer parameter specifying the number of SPIs (Shared
+Peripheral Interrupts) to allocate for the domain. If the value specified by
+the `nr_spis` parameter is smaller than the number of SPIs calculated by the
+toolstack based on the devices allocated for the domain, or the `nr_spis`
+parameter is not specified, the value calculated by the toolstack will be used
+for the domain. Otherwise, the value specified by the `nr_spis` parameter will
+be used.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index b9cb5b33c7..fe5110474d 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
+x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
+xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 5b293755d7..c9e45b306f 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -597,6 +597,7 @@ ArchArm struct {
 GicVersion GicVersion
 Vuart VuartType
 SveVl SveType
+NrSpis uint32
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1cb89fa584..a4029e3ac8 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 
     LOG(DEBUG, "Configure the domain");
 
-    config->arch.nr_spis = nr_spis;
-    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
+    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
+    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
 
     switch (d_config->b_info.arch_arm.gic_version) {
     case LIBXL_GIC_VERSION_DEFAULT:
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..4e65e6fda5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
                                ("sve_vl", libxl_sve_type),
+                               ("nr_spis", uint32),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..e3a4800f6e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2935,6 +2935,9 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
+        b_info->arch_arm.nr_spis = l;
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726501.1130817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEV-0007Uv-SS; Tue, 21 May 2024 04:36:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726501.1130817; Tue, 21 May 2024 04:36:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEV-0007Tw-MX; Tue, 21 May 2024 04:36:15 +0000
Received: by outflank-mailman (input) for mailman id 726501;
 Tue, 21 May 2024 04:36:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HET-0006eS-Ms
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:13 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a5d4fe5f-172b-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 06:36:11 +0200 (CEST)
Received: from SA9PR13CA0045.namprd13.prod.outlook.com (2603:10b6:806:22::20)
 by SA1PR12MB6775.namprd12.prod.outlook.com (2603:10b6:806:25a::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 04:36:07 +0000
Received: from SA2PEPF00001507.namprd04.prod.outlook.com
 (2603:10b6:806:22:cafe::23) by SA9PR13CA0045.outlook.office365.com
 (2603:10b6:806:22::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 04:36:07 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:07 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:05 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:04 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5d4fe5f-172b-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=C17Pi7tJGLw4DxXwSM9ESWg+rU7/DL+BNHCHdzx108xL3y0u1SAhsITSJG/V+hZ6e2Mw83HLbVmKSf19vVmx5Dfwd2qmvf6PJE+z52vOzDBDCxdXDijE6a8F6rOzeaAcPXlDNKUA69J27tHRj+k1vQn8hng/rJiykmgbs4dRHvvFeS3mj06/6T2uuxSLVEQxlHWyuilGtbngxRogVBKH+17igtCWP6/pbhN+JgPCDFOK5q5yBttJiTmkTXvn0QZ64+dcLoIJDoJLO1hCtUDMpTJ4cPPah0bWKjgRoHuQB21f+LBpfS6xqD1X19o7mgdMHGbobn1uIm3KU59HOWe2ow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FuT4kemK+kjHrj50JMbp96wCovrhPDOGjTghim1ZSdM=;
 b=bjAXEwE2weqPzRyAftaegUnj/bQU5S9W+wpTYVqmqVnEGNrt3dC3BvezD/7WbBfg7Ya8Dy+Bt21s0NC0jUGbc+hJiL61lzIt5j40ODG5U3Kz2sW99tzqmbZRZhE/+Oq8aVgAm4Tg3b9Pcabh6M8Y5R1gU8qOtJDLee5z5ShoanKPr2Aqp4qgu7lpk8Nox6MVkk1iVCLUM+zjyKaoftFmHaxYwy234Xgbe+srYG94RwPL/eqX8k8CCLooFtOYfbFXdnrWrlGzUN5CRGWi1nUv58SkLCEyyY38JWNNzcl8COGHcOxtB4HPWZfKUkSPmNA6+A5U5UODmg8JCOxwQU741Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FuT4kemK+kjHrj50JMbp96wCovrhPDOGjTghim1ZSdM=;
 b=OyewpQFMLsLzwHRRGnuZZR+b9WXn9PLZ7dv9WAkvDvye5aTGKcelpFrl24Uiub4fZPFuJiA828qlNfOL4N3+Gn2BAZ2g67P2StzAmJ7jGcZoj9Rit+Pgn1Ogo0WVegm8eCaoiGZxdxGoBED/sXBjl9LgqJgH7e3nvCdViME6z+k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v3 3/8] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
Date: Tue, 21 May 2024 12:35:52 +0800
Message-ID: <20240521043557.1580753-4-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|SA1PR12MB6775:EE_
X-MS-Office365-Filtering-Correlation-Id: e98ba37c-bd5c-4a95-9f03-08dc794f881e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?OjUZlyRjaxo9UymNdsxuU/VMmz908IooKJ5Hq41wYYPUxw0jvP/J1SH0Dgzg?=
 =?us-ascii?Q?9Qw6vUhe77juLSOdALoUTJ7MPOQ2NGQX6Oaj/zLaT+nY+J3cFE2LZCd7N50b?=
 =?us-ascii?Q?oG8Ftl0iVGrIYTH/jYcWQf20eWXBh6wj2yFECT+obqpij9ApHlPJ4piJxhmP?=
 =?us-ascii?Q?zIU9AlVjcq/3E0QacQqHvjewJWMI1BcHeZEBjBUuB1aB9ayCQFlW62SGH4ug?=
 =?us-ascii?Q?a3qCMyvT+3ejtZOvRdVdJF/laNLdJ7hYtvyVPnc7oHyjChDvXivAUKrQASIM?=
 =?us-ascii?Q?mShsFZEDZeWEHcqSs6qRebPdjkzCIRrpRfh4BeZkoRI8JC7CoxF3hxCrl9gi?=
 =?us-ascii?Q?5GYKZrqrM7W0Uq2/4c+rOeE2VKZYKTioXmNUOu9QRaZSG2DnwVB1UqqLaKhB?=
 =?us-ascii?Q?rGPeswld3orys+i+IffK34ObGxA06R+WODQvVmQyTmy55HaxVA2hvVIE/Cy8?=
 =?us-ascii?Q?X6cqfR2aIcmlPvboh0C4zW4HN1f8SPtJ/EFGgHQrtcB76E4aYFPiemGGiktF?=
 =?us-ascii?Q?jpSLye93VpyMgo6UFUl257Mt9jeF5TC4D87vv7JZZXpmzfN5CS+3QorHRaJ4?=
 =?us-ascii?Q?hqn181uz+HALBFmeLw8N8faFLmV7pBlySY09CGOMxAriS3BiU6ubzPgzVQAT?=
 =?us-ascii?Q?n9tfVefES7qLSyAcVUZJTDlUEgTbF1NgpvOhe+Baxui5rCA3WcttrhGJmC9i?=
 =?us-ascii?Q?G4cZgpdK6ec3dIzi2QwaLymNxwbpJVCiTPbE4nNncham2PJOWBJrVU3m9RdZ?=
 =?us-ascii?Q?UEzXzzV/XSlWwM49E0TedO8VAzJzB1ZcYH8J3Fm72wfXdURKiykfqw64Scaq?=
 =?us-ascii?Q?67tFYvtE6L2l6iMRdgP6WlmZ/FTR6dUsNwewZkOeLi3324QGNga9hQUWT+2U?=
 =?us-ascii?Q?pavGXQxrk0hvlwped46TBMVAYpnyIxKKz04Y2A00serVA/tjlu0NCZHi6k33?=
 =?us-ascii?Q?AY7EHYA1YmwHZSWyQ0VaKU1GYFzR+p7DsOMwS4rcih1ncND5r7ymh4gks3QY?=
 =?us-ascii?Q?cUOJPlYE66v/pMU8W5fyHST/IQoHhEdLd1+eAonoNgrZEpnOlIqsKxPYJSXq?=
 =?us-ascii?Q?68N5DdBoJrT0SdS6oBpPVEnRCCvciINCgmzhGv3ejDNPTn2AMk1gjyU303g9?=
 =?us-ascii?Q?Kzo37LiGvRNKPoyC7DiD8H3+3srzKGIrArFbR5bPLsJvnN77/DObSTNzSN6b?=
 =?us-ascii?Q?axwYVR1dfbwbPZDkt/2uDRKsvjPw5jSGgaattr1ThGyioCP1498OyTnNmFue?=
 =?us-ascii?Q?pnqRvxQSHUeCzvagMwNOqrdYetjMLT9A3PYyYk4VZC4dYXepr92uVHatVjuE?=
 =?us-ascii?Q?36Dm8MlP5dUf+gukoZCvTw/PjcWYli7ApBV5AaOFrXOgag4xKvD9tngxtwBh?=
 =?us-ascii?Q?G3jUZOFQp2O/kuR4lgAwSCZa+pXo?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:07.2384
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e98ba37c-bd5c-4a95-9f03-08dc794f881e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001507.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6775

There are some use cases in which the dom0less domUs need to have
the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
example, the dynamic dtbo feature allows the domain to be assigned
a device that is behind the IOMMU at runtime. For these use cases,
we need to have a way to specify the domain will need the IOMMU
mapping at domain construction time.

Introduce a "passthrough" DT property for Dom0less DomUs following
the same entry as the xl.cfg. Currently only provide two options,
i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
construction time based on the property.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Use a separate variable to cache the condition from the "passthrough"
  flag separately to improve readability.
- Update the doc to explain the default condition more clearly.
v2:
- New patch to replace the original patch in v1:
  "[PATCH 03/15] xen/arm: Always enable IOMMU"
---
 docs/misc/arm/device-tree/booting.txt | 16 ++++++++++++++++
 xen/arch/arm/dom0less-build.c         | 11 +++++++++--
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..f1fd069c87 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -260,6 +260,22 @@ with the following properties:
     value specified by Xen command line parameter gnttab_max_maptrack_frames
     (or its default value if unspecified, i.e. 1024) is used.
 
+- passthrough
+
+    A string property specifying whether IOMMU mappings are enabled for the
+    domain and hence whether it will be enabled for passthrough hardware.
+    Possible property values are:
+
+    - "enabled"
+    IOMMU mappings are enabled for the domain. Note that this option is the
+    default if the user provides the device partial passthrough device tree
+    for the domain.
+
+    - "disabled"
+    IOMMU mappings are disabled for the domain and so hardware may not be
+    passed through. This option is the default if this property is missing
+    and the user does not provide the device partial device tree for the domain.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..5830a7051d 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -848,6 +848,8 @@ static int __init construct_domU(struct domain *d,
 void __init create_domUs(void)
 {
     struct dt_device_node *node;
+    const char *dom0less_iommu;
+    bool iommu = false;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
 
@@ -895,8 +897,13 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
-             iommu_enabled )
+        if ( !dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
+             !strcmp(dom0less_iommu, "enabled") )
+            iommu = true;
+
+        if ( iommu_enabled &&
+             (iommu || dt_find_compatible_node(node, NULL,
+                                               "multiboot,device-tree")) )
             d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726503.1130832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEZ-00081t-Bg; Tue, 21 May 2024 04:36:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726503.1130832; Tue, 21 May 2024 04:36:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEZ-00081l-7k; Tue, 21 May 2024 04:36:19 +0000
Received: by outflank-mailman (input) for mailman id 726503;
 Tue, 21 May 2024 04:36:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HEY-0006hu-8Q
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:18 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a857611f-172b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 06:36:15 +0200 (CEST)
Received: from SA1P222CA0046.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:2d0::14)
 by MW6PR12MB8957.namprd12.prod.outlook.com (2603:10b6:303:23a::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 04:36:12 +0000
Received: from SA2PEPF00001506.namprd04.prod.outlook.com
 (2603:10b6:806:2d0:cafe::c3) by SA1P222CA0046.outlook.office365.com
 (2603:10b6:806:2d0::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 21 May 2024 04:36:11 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001506.mail.protection.outlook.com (10.167.242.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:10 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:10 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:08 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a857611f-172b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lZgPU7G1H+hhtuaWEuG/cMMmBo/QQ49SAG+AZvvFieqW6nQcvTRxqXdKrDBK2oWwTIgqAYyXW2HJawQa13miNOxFqlYK/NcLU9BNnnDtbAnhbktwlBWXwyWWzccQUWlaczfX8Xe+S4bDs7sE9OceNUvQyVQmYZNddeSWI9MsSzMa6tnKnLHsW7xjrHa23FIAgq5KXRB/r4O7+4KG/2TISHL4pSdKMxIJEDbKI3vS9DAJO2FpZJviUDn98kll0mW7Un4FNF4nx0bH+B1ILui13JtcZYmuWHkL7AkkQeIk74XG7xdAogJs/P1spFDp9PnHCKxfOoYtekCTdmWMp+wfnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EOJtF38SHzbKHHCO6HcbBP/3k8UEkTSkkoNvwkF1QIM=;
 b=QBtl44x1G/czAYBYL4NEKW+rBKOl7GF0UAaPXDJO83C9zkgJZzZwFcV5iz1OL4hQ6jiR0fVfLZyqAK4RpIOAwoxof1OYBH3T5pv/qK6SlbxCAL9lyon2lTGztVWG9zN1aO7nR96FJenXJwNDoUA3rZHfPbu/Gg1CRcCJtAp9L+8pH+Tq+ZzCjUzj0RUL4nFYfHDil3+B5y3vpnRuC6TivZLSoeine2eK7ypgBjECuYl1d+MR4n9b8KPyfhT+/7UlKXiokTsDq0KtaaoCaUTHDVlQf5z2GpH92BOPoacxQXbDGdVXMYfWe9RdMvXqNmHk5rhGMnvTL0zG1YgUzll7yg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EOJtF38SHzbKHHCO6HcbBP/3k8UEkTSkkoNvwkF1QIM=;
 b=S8Sm22uAopwOkW6WFnp9X+UOmz1gKQmZeaERmqNckV/Ei1KDdKMH7fKKmqLOWWEB+5H24Os1gsNfWYIAEJImkk50W1cb5htRRPGFeYD7W7/VwPNeWz0cAufpNhmAl3Znjc1tQ3LRg6NjhA/01+IaVAJbwKhgneLsIMudVo1j03U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to running VMs
Date: Tue, 21 May 2024 12:35:54 +0800
Message-ID: <20240521043557.1580753-6-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001506:EE_|MW6PR12MB8957:EE_
X-MS-Office365-Filtering-Correlation-Id: 556d5ffa-525c-4836-1cb7-08dc794f8a1b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?+Qql7oBpC7z9QsDGgKoq0TXcutywO3ibPMhx5jjDz4u/TIaGO7etjfTbZsGy?=
 =?us-ascii?Q?LDgMF64CKDEOrduo2EqUd0O3amflBPJzXguY/tmBp6uJrvukT6SNIfmPp3Zp?=
 =?us-ascii?Q?SONcEMkqFLhEqwyVAKCkpimVhdOZ9MjiXqL9AEgr5Dei4jS4jW6hbeC06+Bp?=
 =?us-ascii?Q?LOhClgJ/Br1ngGSJ4vCeGABNV0SDYRNAFfmK+qHOq+M8dQYzS0n+0b60ieIj?=
 =?us-ascii?Q?Nmg0rn6VA5es5XHPomXD7eZqMUW4gobcuwoXgJRjPfL0tyaaKNTjeQMQ6IcC?=
 =?us-ascii?Q?J+s+M3XG42t51JbL4K8vbhmxnaWPtfqpbDwVycqRfbYzSFYIpOJcVD0LGW2z?=
 =?us-ascii?Q?YKQlz3E3JeDzw5Ubn81UE1AKxiCsVKGzywraP7DSuVFf2NC8lMNnes40b11i?=
 =?us-ascii?Q?ZB9Uy32A5F+soo6bTzuizlRa13FBrI6of4mKSASUyePIm2SNmdTTSJItJTsg?=
 =?us-ascii?Q?TrjGJqILLjX5YvsjneTvM8OJPcR20QCC3nv5TroNS53/KOdfcyttZya3sIfX?=
 =?us-ascii?Q?YlFhduTiJVttzQwz9HocLEJFULDlxjyp8zifNTBMk3vrOAOdNeo+rgkQ96Te?=
 =?us-ascii?Q?b97jGy+K54tNnriXFjg84+hi9l1vjyY8ceqV38223vtvh6ZMnJVfQ5TuFony?=
 =?us-ascii?Q?WJlwiYgcJ92Bt0ULXmkxbdZdetiN0q/J2wUM+bBVpNRrEGKnvdEDxovArEre?=
 =?us-ascii?Q?RNktqaKpCibXxyq4KqFdXhZCOhVtNVzw5d145FEWl7PX+LG+AbaO9Suv60Hx?=
 =?us-ascii?Q?ehgwww3VWirIjY20UZjbpMjR7s10pBddbSJcahtTk+zwD9cXpNoWj0UelRcQ?=
 =?us-ascii?Q?A/R54EhsXavqzrUIqq2g+sk2gc2C1GA3fPvxSyBIVGBgrMNuvVXoXFUUzHXV?=
 =?us-ascii?Q?oRGbgRhSJxXQUW/qhJLlesL0MSdjjbnv/hxxcyyALPhsj36RoxwOCPJsxzd5?=
 =?us-ascii?Q?2+kKOcP+U9K/14WHLbMEH6EhYaCLklqHyrdKOkJTG4hlMvKftswV6qpNcUzb?=
 =?us-ascii?Q?0mScb+vCZWFUr3lKLdBfcvJI5SMjQY/ls9tbm0HmPAoPjcEXHFSGjuxUGF9b?=
 =?us-ascii?Q?fS+CuJc8Ffipda4ivAM9c1AbYdT2TkS+vk+H/TiHiL/t9sqwo1smfpuUJnZo?=
 =?us-ascii?Q?Zchvu0pAoUetsdiXiRBoMt567GbHaUbj6240XYljz7EsSJYs9sA22HLh9WFP?=
 =?us-ascii?Q?UUu6pL980WbgDzkeNYjEexdX+0FL4yY0fZ0pdqvCvxwHcBdSxrtJrEdrT0rX?=
 =?us-ascii?Q?ssaB1sFUR99TYl4yHi77POeDjw+4KCWrNGnDe/bXdJWAFg/zwnz7peq8ToKm?=
 =?us-ascii?Q?ebMJO8GuPynAuL9xf7OIfX0Y33FNIEofX9PybYAkmQPSYZOUoE1Sp350KCvv?=
 =?us-ascii?Q?DeLKjjytWc6zdcBgwuOb3HYNJGYP?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:10.5552
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 556d5ffa-525c-4836-1cb7-08dc794f8a1b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001506.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8957

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Currently, routing/removing physical interrupts are only allowed at
the domain creation/destroy time. For use cases such as dynamic device
tree overlay adding/removing, the routing/removing of physical IRQ to
running domains should be allowed.

Removing the above-mentioned domain creation/dying check. Since this
will introduce interrupt state unsync issues for cases when the
interrupt is active or pending in the guest, therefore for these cases
we simply reject the operation. Do it for both new and old vGIC
implementations.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Update in-code comments.
- Correct the if conditions.
- Add taking/releasing the vgic lock of the vcpu.
v2:
- Reject the case where the IRQ is active or pending in guest.
---
 xen/arch/arm/gic-vgic.c  | 15 ++++++++++++---
 xen/arch/arm/gic.c       | 15 ---------------
 xen/arch/arm/vgic/vgic.c | 10 +++++++---
 3 files changed, 19 insertions(+), 21 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 56490dbc43..956c11ba13 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
 
     /* We are taking to rank lock to prevent parallel connections. */
     vgic_lock_rank(v_target, rank, flags);
+    spin_lock(&v_target->arch.vgic.lock);
 
     if ( connect )
     {
-        /* The VIRQ should not be already enabled by the guest */
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
         if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             p->desc = desc;
         else
             ret = -EBUSY;
     }
     else
     {
-        if ( desc && p->desc != desc )
+        if ( (desc && p->desc != desc) ||
+             test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
+             test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             ret = -EINVAL;
         else
             p->desc = NULL;
     }
 
+    spin_unlock(&v_target->arch.vgic.lock);
     vgic_unlock_rank(v_target, rank, flags);
 
     return ret;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86de..3ebd89940a 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
     ASSERT(virq < vgic_num_irqs(d));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-    if ( d->creation_finished )
-        return -EBUSY;
-
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
     if ( ret )
         return ret;
@@ -167,13 +159,6 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
     ASSERT(test_bit(_IRQ_GUEST, &desc->status));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * Removing an interrupt while the domain is running may have
-     * undesirable effect on the vGIC emulation.
-     */
-    if ( !d->is_dying )
-        return -EBUSY;
-
     desc->handler->shutdown(desc);
 
     /* EOI the IRQ if it has not been done by the guest */
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b9463a5f27..78554c11e2 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
 
     if ( connect )                      /* assign a mapped IRQ */
     {
-        /* The VIRQ should not be already enabled by the guest */
-        if ( !irq->hw && !irq->enabled )
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest
+         */
+        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
         {
             irq->hw = true;
             irq->hwintid = desc->irq;
@@ -887,7 +890,8 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
     }
     else                                /* remove a mapped IRQ */
     {
-        if ( desc && irq->hwintid != desc->irq )
+        if ( (desc && irq->hwintid != desc->irq) ||
+             irq->active || irq->pending_latch )
         {
             ret = -EINVAL;
         }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726504.1130842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEb-0008Lb-L8; Tue, 21 May 2024 04:36:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726504.1130842; Tue, 21 May 2024 04:36:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEb-0008LO-HZ; Tue, 21 May 2024 04:36:21 +0000
Received: by outflank-mailman (input) for mailman id 726504;
 Tue, 21 May 2024 04:36:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HEZ-0006hu-OB
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:19 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a91c120f-172b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 06:36:17 +0200 (CEST)
Received: from SA1PR04CA0016.namprd04.prod.outlook.com (2603:10b6:806:2ce::21)
 by DS0PR12MB8504.namprd12.prod.outlook.com (2603:10b6:8:155::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 04:36:15 +0000
Received: from SA2PEPF00001509.namprd04.prod.outlook.com
 (2603:10b6:806:2ce:cafe::3f) by SA1PR04CA0016.outlook.office365.com
 (2603:10b6:806:2ce::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Tue, 21 May 2024 04:36:15 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001509.mail.protection.outlook.com (10.167.242.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:15 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:14 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:13 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a91c120f-172b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NHWCQdml5dsyd4yFmEEqFcnqeCsYYa5zf+yjVwVhDiX5MSfRcJxncv90F+LduLO9bvmyQsHmiUri+z/Kb2xBX7vLPPJUyBKRcujNLDyMO9/JHEAd/m1EJeuov9awrm+ol3BCx7WfI1aY3Shv+q5smepX1ehEbXkwl9cEtOMtGKCi8DL8ePpW8O25Hkvh360xHP5XjV51Tk1PsyKtj8fnDqYWFVLsvqXmEoszriGnT75x+btEwizXBwgo6ZGn/m0TIAlD+QBgrQWpV4cwVO34zkx3pJCKhAZgZNXskNPyYCgl0KVnk0T3rqStQ+pPkCawPhH5vyni9QpaDzIF0LUIWw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BE7z9HeLzOE9JPXCOJkwYcS8CjTT2Ru5y5EpVZBDA6c=;
 b=i3ZuKH//kSg9znsMnvoRrbQSZyIP5KMKXf3CPo+uCxrRFIcqnodr9PW3VX98kx0la017YA0nlSaI2yQrP7iPmPPbjN4tPjVu6rgUyqXlIm1TgV/qmIKh32SMfpTVrlM/tB47h9P0MAydbuzvli9T+J6BvQzDrEbNs3zGFdZ1Owo5lyRZkP02GgG44ZdFVrl1E7s6sTuNxED2QfUGoEysvQiGNO7/HXUqvZIho99mSDo4KtV+hbBN82c7dWM4qEnhRyXW+f5jGyfBajpfYQCR0/iRjqQAuWvq/iqzuWdTxYjFzZ4paK95VIiwy9zYujf0hlFS3AG6n02Zg6zKiFPR4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BE7z9HeLzOE9JPXCOJkwYcS8CjTT2Ru5y5EpVZBDA6c=;
 b=fDcBnNDMgjgGpw58GfvPp6nkvXvcuoNk2zHHI/ETgxZB+/dR5nmkh3/01WtQ7bIq7cN4K1HXjjrQAHQYrcK5Lo5zvC9/Tukq728sEtWEElUX91wAoZcBezD1h7pQyA4AJH1OMlfRa11N36FvmORZgetYhmnsaK3CA9ACi0pqQqw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
	Henry Wang <xin.wang2@amd.com>
Subject: [PATCH v3 7/8] tools: Introduce the "xl dt-overlay {attach,detach}" commands
Date: Tue, 21 May 2024 12:35:56 +0800
Message-ID: <20240521043557.1580753-8-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001509:EE_|DS0PR12MB8504:EE_
X-MS-Office365-Filtering-Correlation-Id: 4b9fe93c-9dbc-4460-2658-08dc794f8cc2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?Fzb3V6t8baIST7lGd/94pmH7QN5oUjDSEYhvlfpybNneUdap1+G/ESLH9Ilk?=
 =?us-ascii?Q?lOPHq1t/2P/HUSEYhmy263qZdy+O++CuiXaO+NGdKF5OhsixsVACXWj3cJGB?=
 =?us-ascii?Q?d4vTMHNnFTEeT7uliTlqaGjyFaJZfIqVYNH6mXbL5GedV6ltCNNUKB9wI1Lh?=
 =?us-ascii?Q?Br4YfH6ei3l2ExajslKGVDezuthZ7dM5vJ9OaqZFIPiN3xFwaZUUKw2OreQ3?=
 =?us-ascii?Q?IoyduyL/dF3bhQCtazN+Tlxbf8ttpO9ZKyhxWC9Pt5PWUgpZeIpMdZXGI3Kq?=
 =?us-ascii?Q?NKjsRacfbBRKUDc39fTl5vI8aUmBODFpYtz6TQCOvP1BM4X7F5BOzho2vD2V?=
 =?us-ascii?Q?DnrFUD70k+eYAUkKiikJQuP2dRNgCQvpWa6Vrq85h8kr4xBMgZrf9An/j/zl?=
 =?us-ascii?Q?YGa2Hs+v5gChKdbOM4J51UW4BkcKil/LG1m1YeFdeuqlyEXT35fcTJiKWwYB?=
 =?us-ascii?Q?oXAtOJU1Ctz6u0CetiVATZZAkfxxcIyRfaELdtkq09EHI5usewpQYbpKzBcF?=
 =?us-ascii?Q?sYLHjP5XxIGYx44SE89KFyw1cjb+C6jId87kACSCaD6l8cqmuPbXFpxhXNt8?=
 =?us-ascii?Q?VjH8jwDcLkyMVd8OemML4+m7htmAyYCfTAw//bRmUd016JFfGpNGYg25jueL?=
 =?us-ascii?Q?nIOvsd1BHS7UfNuRHzneNH4mBShJd20kPwLh/p7lxIaNxfVD5J8QMcX2ipbj?=
 =?us-ascii?Q?UgCdhJso2yWVZi0vVfvRdiwPuxv7Xby4fYGBUhjPUnWXsXveWSiq0db5q9Gy?=
 =?us-ascii?Q?Pxv3L9fnOLOrS6U8i9pwnDMk8Mnw0z6GX7piI185MoyBu3FbOnlEnpVRA4kf?=
 =?us-ascii?Q?U7bz7cHJXK+KGNB7NaUGaCkcZSJv8TJfGbfjJoWDO5/4Y14xsmwCu5aPwtcX?=
 =?us-ascii?Q?JJaiXW4bSrRz0QSWjLvOM63UVjP/exc9C8DwjTJ73R+nE5v1Mo5ZCka0DHyb?=
 =?us-ascii?Q?kAiOf36SKDy4neU6wIQquOJYJc1D68OuNOOoOdx1H8EL0hoibt0mJ8GmWI/Y?=
 =?us-ascii?Q?AeFAMS9fX5Ygu4cZbLUTA9dEfyU5M4KH9/BdfSXnCEA6FfDoo4O1+pLbQM0L?=
 =?us-ascii?Q?NXnsBDV+LNFm5uWyiN/rgsDyNij1SEFi9mKXHmx6k/+HeHLBGRnIQnMCBwNR?=
 =?us-ascii?Q?fMOb4z3yLXDYVnrjmAWBb4FxGRkHyTcZwrrF1EsHne4MQ5ftU6vf/9jjdzUA?=
 =?us-ascii?Q?fIsmSDfmAmL7MWk+eqxyHr48ROXG1K+c7nTS0l5gMctDbzHAIzvEOMgw6+2Y?=
 =?us-ascii?Q?AS0edmW9HzgBwzzhsUDCgYMoxd2Yi1zyzFcQiQjVO77dn33dNFpZRWbdbDR5?=
 =?us-ascii?Q?gKt0v3wkW8Uj5SKUjKK2Gqj1VtEXuk4uhpezehvxM1pDQPp10iwbz51ao9X/?=
 =?us-ascii?Q?szKXkltQbNkmNnczlkndUYpnXgTL?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:15.0054
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b9fe93c-9dbc-4460-2658-08dc794f8cc2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001509.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8504

With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach/detach devices from the provided DT overlay to domains.
Support this by introducing a new set of "xl dt-overlay" commands and
related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
rework the command option parsing logic.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- Introduce new API libxl_dt_overlay_domain() and co., instead of
  reusing existing API libxl_dt_overlay().
- Add in-code comments for the LIBXL_DT_OVERLAY_* macros.
- Use find_domain() to avoid getting domain_id from strtol().
v2:
- New patch.
---
 tools/include/libxl.h               | 10 +++++++
 tools/include/xenctrl.h             |  3 +++
 tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++
 tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++
 tools/xl/xl_cmdtable.c              |  4 +--
 tools/xl/xl_vmcontrol.c             | 42 ++++++++++++++++++++---------
 6 files changed, 104 insertions(+), 14 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..6cc6d6bf6a 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -2549,8 +2549,18 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 #if defined(__arm__) || defined(__aarch64__)
+/* Values should keep consistent with the op from XEN_SYSCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_ADD                   1
+#define LIBXL_DT_OVERLAY_REMOVE                2
 int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
                      uint32_t overlay_size, uint8_t overlay_op);
+
+/* Values should keep consistent with the op from XEN_DOMCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_DOMAIN_ATTACH         1
+#define LIBXL_DT_OVERLAY_DOMAIN_DETACH         2
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op);
 #endif
 
 /*
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..9ceca0cffc 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id);
 #endif
 
 /* Compat shims */
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
index c2224c4d15..ea1da522d1 100644
--- a/tools/libs/ctrl/xc_dt_overlay.c
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -48,3 +48,34 @@ err:
 
     return err;
 }
+
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id)
+{
+    int err;
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_dt_overlay,
+        .domain = domain_id,
+        .u.dt_overlay = {
+            .overlay_op = overlay_op,
+            .overlay_fdt_size = overlay_fdt_size,
+        }
+    };
+
+    DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+        goto err;
+
+    set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, overlay_fdt);
+
+    return err;
+}
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
index a6c709a6dc..00503b76bd 100644
--- a/tools/libs/light/libxl_dt_overlay.c
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -69,3 +69,31 @@ out:
     return rc;
 }
 
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op)
+{
+    int rc;
+    int r;
+    GC_INIT(ctx);
+
+    if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+        LOG(ERROR, "Overlay DTB check failed");
+        rc = ERROR_FAIL;
+        goto out;
+    } else {
+        LOG(DEBUG, "Overlay DTB check passed");
+        rc = 0;
+    }
+
+    r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, overlay_op,
+                             domain_id);
+    if (r) {
+        LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__);
+        rc = ERROR_FAIL;
+    }
+
+out:
+    GC_FREE;
+    return rc;
+}
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 1f3c6b5897..37770b20e3 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = {
 #ifdef LIBXL_HAVE_DT_OVERLAY
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
-      "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>",
+      "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain",
+      "<operation=add|remove> <.dtbo> OR <operation=attach|detach> <.dtbo> <Domain>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 02575d5d36..ccfeec39f1 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1268,32 +1268,46 @@ int main_create(int argc, char **argv)
 #ifdef LIBXL_HAVE_DT_OVERLAY
 int main_dt_overlay(int argc, char **argv)
 {
-    const char *overlay_ops = NULL;
     const char *overlay_config_file = NULL;
     void *overlay_dtb = NULL;
     int rc;
     uint8_t op;
     int overlay_dtb_size = 0;
-    const int overlay_add_op = 1;
-    const int overlay_remove_op = 2;
+    uint32_t domain_id = 0;
+    bool domain_op = false;
 
     if (argc < 2) {
         help("dt-overlay");
         return EXIT_FAILURE;
     }
 
-    overlay_ops = argv[1];
-    overlay_config_file = argv[2];
-
-    if (strcmp(overlay_ops, "add") == 0)
-        op = overlay_add_op;
-    else if (strcmp(overlay_ops, "remove") == 0)
-        op = overlay_remove_op;
-    else {
+    if (strcmp(argv[optind], "add") == 0)
+        op = LIBXL_DT_OVERLAY_ADD;
+    else if (strcmp(argv[optind], "remove") == 0)
+        op = LIBXL_DT_OVERLAY_REMOVE;
+    else if (strcmp(argv[optind], "attach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_ATTACH;
+        domain_op = true;
+    } else if (strcmp(argv[optind], "detach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_DETACH;
+        domain_op = true;
+    } else {
         fprintf(stderr, "Invalid dt overlay operation\n");
         return EXIT_FAILURE;
     }
 
+    overlay_config_file = argv[optind+1];
+
+    if (domain_op) {
+        if (argc <= optind + 2) {
+            fprintf(stderr, "Missing domain ID\n");
+            help("dt-overlay");
+            return EXIT_FAILURE;
+        } else {
+            domain_id = find_domain(argv[optind+2]);
+        }
+    }
+
     if (overlay_config_file) {
         rc = libxl_read_file_contents(ctx, overlay_config_file,
                                       &overlay_dtb, &overlay_dtb_size);
@@ -1309,7 +1323,11 @@ int main_dt_overlay(int argc, char **argv)
         return EXIT_FAILURE;
     }
 
-    rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    if (!domain_op)
+        rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    else
+        rc = libxl_dt_overlay_domain(ctx, domain_id, overlay_dtb,
+                                     overlay_dtb_size, op);
 
     free(overlay_dtb);
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726505.1130848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEc-0008Sc-Cy; Tue, 21 May 2024 04:36:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726505.1130848; Tue, 21 May 2024 04:36:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEc-0008QW-2g; Tue, 21 May 2024 04:36:22 +0000
Received: by outflank-mailman (input) for mailman id 726505;
 Tue, 21 May 2024 04:36:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HEa-0006eS-Bo
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:20 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20601.outbound.protection.outlook.com
 [2a01:111:f400:7e88::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aa4ee97c-172b-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 06:36:19 +0200 (CEST)
Received: from SA1P222CA0035.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:2d0::10)
 by SA1PR12MB9247.namprd12.prod.outlook.com (2603:10b6:806:3af::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May
 2024 04:36:14 +0000
Received: from SA2PEPF00001506.namprd04.prod.outlook.com
 (2603:10b6:806:2d0:cafe::ed) by SA1P222CA0035.outlook.office365.com
 (2603:10b6:806:2d0::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Tue, 21 May 2024 04:36:14 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001506.mail.protection.outlook.com (10.167.242.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:14 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:12 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:10 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa4ee97c-172b-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BAa3lpafPZQvr1YDjSdauNNXobXxJcpOnNsPWh5+1icXRlADKKLU1MlwpI6v5jurDAEluiO4pHZYIyEkR5Wsd4hGK+gsviZEieVVdOjFBPwqo7T7DZqAsqTvqRcs51C/waS5m6f5nrLGfgw17hkSXq8Q2G0L/NX6v46KwJ6e0ZcArZTE36WOul1aLX/hRv9LlWOIVVKXgTQzhc5S6Psy7+TsmSn7v8mYMqx7yWrcKnYSeFFAeiY371glACcOZoV60MMaY8n7/13vxzVPTJaXmX/uHZvBRu1TpFZt/+MFOarEX2J9525ExL65PxCoEcdbSx24C/Bklr2KtKv1Y7CLgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=c5asE2vIShV06rhyQR/L2eZ3PEF3qss1h6tMuTFdt4g=;
 b=epZ9eySRQp5IaVic1R+4fyc4tk8BzeT7norjD66eu4nZU24HfOfh6zS7oq4JjTkPZ0Km1KrGL+pjwriUtF11pejDKzdByuA4KaAG00JmgaZelzWKhMqyqx3Hs69Tn8stfd3tKonbQlHHt9M86q7ONO/TOvkqJsGmsZ0PFN8rxukgX64uGk5lFGZI6XxMJqoWXyovaST3ATxkgi+yCMTp/3VkfZCO5t7QH+F636MCwNOa5flrdVtSYPhj9hhMz0DzKahWdsGtAvyDNyyVulkOpClxr3OcsAqBTsv3w/sxdQ1zMiUcK81Haz5m48VhvUxLWbRHdTVLvCLumAsBgyePgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=c5asE2vIShV06rhyQR/L2eZ3PEF3qss1h6tMuTFdt4g=;
 b=sqwYR0Mg4WjfZYWlz0NbAuuoxG9kCZT2BKUAbh4wZDAsn4pUp8kaVlCmQ7ZYpWPivvMKjUu01pkTNSvzQzGdQAFggFaDxAXZkSdyIgfZ/QkzxafRlNjN06IlStiwh2zjR2N2YacR8BEnJAJNqj1mzYqzdgCxolGaUZJ9mgjZPcY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v3 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations
Date: Tue, 21 May 2024 12:35:55 +0800
Message-ID: <20240521043557.1580753-7-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001506:EE_|SA1PR12MB9247:EE_
X-MS-Office365-Filtering-Correlation-Id: b7df2670-00aa-46e9-dbc3-08dc794f8c60
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?iDPD4qsEn3+NtRctDhK14FfHQI854Qi/ehp2yAH0A0tgAd9z237TVYEWYurA?=
 =?us-ascii?Q?PYj/LEsIBAI2XiIacSWG0BWVqZZ+LAy50vhNQ/4PR3fRqvYWAAhZRXFmhM9+?=
 =?us-ascii?Q?DM38fOmgWyF3ItOIUx6WmwSXrP0tDJxDkAclKbYyt4i1UFMumcYFuI6w5oPW?=
 =?us-ascii?Q?zTryYyY+6/HVoBSCkakwgcAFRWoSaRJVSn+wlwiEVo7mgEejUAbRipvW07iM?=
 =?us-ascii?Q?i8zFabfooD5yM+8YeRAkQct5E47g+f6p43lgnIPwxQEgpCVlA4iShwcEzHFo?=
 =?us-ascii?Q?0+UyrcPBIuJFMHZZvRCXpEWlFL275Ile00Xzbe6MaBnjCtunvzswLSLTtlbL?=
 =?us-ascii?Q?pq4NKBxoLN0FvNUueKCOc5AwH51rv7x0amwlrDLsEHHKCTgVoBP0ctnA8sxI?=
 =?us-ascii?Q?+8G90CCAvYxVvlcIZwDYQAdF8A2z+6MJFrTTgc+N9cZToilfKs7ZNgp7+wxX?=
 =?us-ascii?Q?kfoZP9a2Ban4P8X213w5jlq9kSn+YqBnEdT8LpQdGXbQKEiJAW2NYGPzwRww?=
 =?us-ascii?Q?/674yAQ1AmrWSGGzTOW5Bvked1iKN7x5T8Qx4roDocWXzObx/Hbi6drKNf0R?=
 =?us-ascii?Q?YNZzlPDldmE8xP2twOCpdNX7+W0j/gZIiYZQla0bDXeE3DCdoakk2aHO3REG?=
 =?us-ascii?Q?Rbl52H4WQDvUqcw6gChUvj8z6nDjgvl9TbbaNaAgNonxNUAIzlOkgSpcGHDP?=
 =?us-ascii?Q?ekEKU32LJRSsbho9pPnQ0kVOnm5r2rWOROJGWMJloLRtr9Z/j9Z3NDCtkc6q?=
 =?us-ascii?Q?CsxHNoSijKL5Iu4lGttDS0nWqd1FeKkkDK06emAjuUBIAcvaKZacEuObdOfR?=
 =?us-ascii?Q?EikaLO7HytKsGT/2g1HVSiWLDr3w81Bib5HjhWz0SREWufMzhn3/F45Ely3/?=
 =?us-ascii?Q?3WpLOJq0qtmRZj09o7eIjcZxBUQwYg9KO/2SSS86Eoum7VSX15FuazMktsAE?=
 =?us-ascii?Q?Td1ROk2WHIRTchkzv5Uat1C9Wghe1GYyHD6nJDrM8xPI3QltJjIzq7xSK2ng?=
 =?us-ascii?Q?pEGvbFGfxfw/ib3x/VMqQk790M9F828bQWCHxk4n/SJFfURP1LiCpjlSAA8y?=
 =?us-ascii?Q?ODaR9ldL+eG0CvDdXEOpED3NHN6tMYhx1EAx4tIDyE+8BeLn//UMLY8UyR/w?=
 =?us-ascii?Q?2xRojwf1DCop1saZgWgh8cbE61amzsiwC0ky2mxNHBY9pDf6uFByweIR1iH/?=
 =?us-ascii?Q?pTPikLsUaT9J4p91cIaaLI0RFuGrkDqAbsW6nBFMqNLSM/oTqm1oTGfJqTz8?=
 =?us-ascii?Q?hux/OWbFxe8XtycR08pBvpL44PpUfLwBt/EOm7qTawVg6NhpIuquMIdE++//?=
 =?us-ascii?Q?K1DTO2iI4w4arKJlyGUY8OsJCfna29wlND7U009N5WrQnbgjoElJ21tI8UvZ?=
 =?us-ascii?Q?tENyE7H8knDq72XMc1pP4V/C84zw?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:14.3678
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b7df2670-00aa-46e9-dbc3-08dc794f8c60
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001506.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9247

In order to support the dynamic dtbo device assignment to a running
VM, the add/remove of the DT overlay and the attach/detach of the
device from the DT overlay should happen separately. Therefore,
repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
overlay to Xen device tree, instead of assigning the device to the
hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with
operations XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH} to do/undo the
device assignment to the domain.

The hypervisor firstly checks the DT overlay passed from the toolstack
is valid. Then the device nodes are retrieved from the overlay tracker
based on the DT overlay. The attach/detach of the device is implemented
by map/unmap the IRQ and IOMMU resources. Note that with these changes,
the device de-registration from the IOMMU driver should only happen at
the time when the DT overlay is removed from the Xen device tree.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
v3:
- Style fixes for arch-selection #ifdefs.
- Do not include public/domctl.h, only add a forward declaration of
  struct xen_domctl_dt_overlay.
- Extract the overlay track entry finding logic to a function, drop
  the unused variables.
- Use op code 1&2 for XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH}.
v2:
- New patch.
---
 xen/arch/arm/domctl.c        |   3 +
 xen/common/dt-overlay.c      | 438 +++++++++++++++++++++++++++--------
 xen/include/public/domctl.h  |  15 ++
 xen/include/public/sysctl.h  |  11 +-
 xen/include/xen/dt-overlay.h |   7 +
 5 files changed, 367 insertions(+), 107 deletions(-)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad56efb0f5..12a12ee781 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -5,6 +5,7 @@
  * Copyright (c) 2012, Citrix Systems
  */
 
+#include <xen/dt-overlay.h>
 #include <xen/errno.h>
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
@@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+    case XEN_DOMCTL_dt_overlay:
+        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 9cece79067..693b6e4777 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -356,24 +356,136 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
     return 0;
 }
 
+/* This function should be called with the overlay_lock taken */
+static struct overlay_track *
+find_track_entry_from_tracker(const void *overlay_fdt,
+                              uint32_t overlay_fdt_size)
+{
+    struct overlay_track *entry, *temp;
+    bool found_entry = false;
+
+    ASSERT(spin_is_locked(&overlay_lock));
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Operation is supported only for prior added dtbo.\n");
+        return NULL;
+    }
+
+    return entry;
+}
+
+static int remove_irq(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+
+    /*
+     * IRQ should always have access unless there are duplication of
+     * of irqs in device tree. There are few cases of xen device tree
+     * where there are duplicate interrupts for the same node.
+     */
+    if (!irq_access_permitted(d, s))
+        return 0;
+    /*
+     * TODO: We don't handle shared IRQs for now. So, it is assumed that
+     * the IRQs was not shared with another domain.
+     */
+    rc = irq_deny_access(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s);
+        return rc;
+    }
+
+    rc = release_guest_irq(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to release irq %ld\n", s);
+        return rc;
+    }
+
+    return rc;
+}
+
+static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d);
+}
+
+static int remove_iomem(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+    p2m_type_t t;
+    mfn_t mfn;
+
+    mfn = p2m_lookup(d, _gfn(s), &t);
+    if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL )
+        return -EINVAL;
+
+    rc = iomem_deny_access(d, s, e);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n",
+               d, s, e);
+        return rc;
+    }
+
+    rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s));
+    if ( rc )
+        return rc;
+
+    return rc;
+}
+
+static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d);
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
-static int remove_node_resources(struct dt_device_node *device_node)
+static int remove_node_resources(struct dt_device_node *device_node,
+                                 struct domain *d)
 {
     int rc = 0;
     unsigned int len;
     domid_t domid;
 
-    domid = dt_device_used_by(device_node);
+    if ( !d )
+    {
+        domid = dt_device_used_by(device_node);
 
-    dt_dprintk("Checking if node %s is used by any domain\n",
-               device_node->full_name);
+        dt_dprintk("Checking if node %s is used by any domain\n",
+                   device_node->full_name);
 
-    /* Remove the node if only it's assigned to hardware domain or domain io. */
-    if ( domid != hardware_domain->domain_id && domid != DOMID_IO )
-    {
-        printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n",
-               device_node->full_name, domid);
-        return -EINVAL;
+        /*
+         * We also check if device is assigned to DOMID_IO as when a domain
+         * is destroyed device is assigned to DOMID_IO.
+         */
+        if ( domid != DOMID_IO )
+        {
+            printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n",
+                   device_node->full_name, DOMID_IO, domid);
+            return -EINVAL;
+        }
     }
 
     /* Check if iommu property exists. */
@@ -381,9 +493,12 @@ static int remove_node_resources(struct dt_device_node *device_node)
     {
         if ( dt_device_is_protected(device_node) )
         {
-            rc = iommu_remove_dt_device(device_node);
-            if ( rc < 0 )
-                return rc;
+            if ( !list_empty(&device_node->domain_list) )
+            {
+                rc = iommu_deassign_dt_device(d, device_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
     }
 
@@ -392,7 +507,8 @@ static int remove_node_resources(struct dt_device_node *device_node)
 
 /* Remove all descendants from IOMMU. */
 static int
-remove_descendant_nodes_resources(const struct dt_device_node *device_node)
+remove_descendant_nodes_resources(const struct dt_device_node *device_node,
+                                  struct domain *d)
 {
     int rc = 0;
     struct dt_device_node *child_node;
@@ -402,12 +518,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node)
     {
         if ( child_node->child )
         {
-            rc = remove_descendant_nodes_resources(child_node);
+            rc = remove_descendant_nodes_resources(child_node, d);
             if ( rc )
                 return rc;
         }
 
-        rc = remove_node_resources(child_node);
+        rc = remove_node_resources(child_node, d);
         if ( rc )
             return rc;
     }
@@ -420,8 +536,7 @@ static int remove_nodes(const struct overlay_track *tracker)
 {
     int rc = 0;
     struct dt_device_node *overlay_node;
-    unsigned int j;
-    struct domain *d = hardware_domain;
+    unsigned int j, len;
 
     for ( j = 0; j < tracker->num_nodes; j++ )
     {
@@ -431,18 +546,15 @@ static int remove_nodes(const struct overlay_track *tracker)
 
         write_lock(&dt_host_lock);
 
-        rc = remove_descendant_nodes_resources(overlay_node);
-        if ( rc )
+        /* Check if iommu property exists. */
+        if ( dt_get_property(overlay_node, "iommus", &len) )
         {
-            write_unlock(&dt_host_lock);
-            return rc;
-        }
-
-        rc = remove_node_resources(overlay_node);
-        if ( rc )
-        {
-            write_unlock(&dt_host_lock);
-            return rc;
+            if ( dt_device_is_protected(overlay_node) )
+            {
+                rc = iommu_remove_dt_device(overlay_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
@@ -457,22 +569,6 @@ static int remove_nodes(const struct overlay_track *tracker)
         write_unlock(&dt_host_lock);
     }
 
-    /* Remove IRQ access. */
-    if ( tracker->irq_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
-   /* Remove mmio access. */
-    if ( tracker->iomem_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
     return rc;
 }
 
@@ -485,8 +581,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
                                         uint32_t overlay_fdt_size)
 {
     int rc;
-    struct overlay_track *entry, *temp, *track;
-    bool found_entry = false;
+    struct overlay_track *entry;
 
     rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
     if ( rc )
@@ -494,31 +589,11 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     spin_lock(&overlay_lock);
 
-    /*
-     * First check if dtbo is correct i.e. it should one of the dtbo which was
-     * used when dynamically adding the node.
-     * Limitation: Cases with same node names but different property are not
-     * supported currently. We are relying on user to provide the same dtbo
-     * as it was used when adding the nodes.
-     */
-    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
-    {
-        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
-        {
-            track = entry;
-            found_entry = true;
-            break;
-        }
-    }
-
-    if ( !found_entry )
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
     {
         rc = -EINVAL;
-
-        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
-               " Removing nodes is supported only for prior added dtbo.\n");
         goto out;
-
     }
 
     rc = remove_nodes(entry);
@@ -536,9 +611,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     xfree(entry->nodes_address);
 
-    rangeset_destroy(entry->irq_ranges);
-    rangeset_destroy(entry->iomem_ranges);
-
     xfree(entry);
 
  out:
@@ -620,15 +692,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return -EFAULT;
         }
 
-        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
-                           tr->iomem_ranges,
-                           tr->irq_ranges);
         write_unlock(&dt_host_lock);
-        if ( rc )
-        {
-            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-            return rc;
-        }
 
         /* Keep overlay_node address in tracker. */
         tr->nodes_address[j] = (unsigned long)overlay_node;
@@ -638,9 +702,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
 }
 /*
  * Adds device tree nodes under target node.
- * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
- * is done to avoid the removal of device_tree generation, iomem regions mapping
- * to hardware domain done by handle_node().
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
  */
 static long handle_add_overlay_nodes(void *overlay_fdt,
                                      uint32_t overlay_fdt_size)
@@ -774,20 +836,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
         goto err;
     }
 
-    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
-    if (tr->irq_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IRQ rangeset failed");
-        goto err;
-    }
-
-    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
-    if (tr->iomem_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
-        goto err;
-    }
-
     rc = add_nodes(tr, nodes_full_path);
     if ( rc )
     {
@@ -843,14 +891,165 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     xfree(tr->nodes_address);
     xfree(tr->fdt);
 
-    rangeset_destroy(tr->irq_ranges);
-    rangeset_destroy(tr->iomem_ranges);
-
     xfree(tr);
 
     return rc;
 }
 
+static long handle_detach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = remove_descendant_nodes_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+
+        rc = remove_node_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+        write_unlock(&dt_host_lock);
+
+        rc = remove_all_irqs(entry->irq_ranges, d);
+        if ( rc )
+            goto out;
+
+        rc = remove_all_iomems(entry->iomem_ranges, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove IRQ access. */
+    if ( entry->irq_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove mmio access. */
+    if ( entry->iomem_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    return rc;
+}
+
+static long handle_attach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
+    if (entry->irq_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IRQ rangeset failed");
+        goto out;
+    }
+
+    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
+    if (entry->iomem_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
+                           entry->iomem_ranges, entry->irq_ranges);
+        write_unlock(&dt_host_lock);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+            goto out;
+        }
+    }
+
+    spin_unlock(&overlay_lock);
+
+    return 0;
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+    return rc;
+}
+
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     long ret;
@@ -890,6 +1089,45 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
     return ret;
 }
 
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
+{
+    long ret;
+    void *overlay_fdt;
+
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH &&
+         op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH )
+        return -EOPNOTSUPP;
+
+    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
+        return -EINVAL;
+
+    if ( op->pad[0] || op->pad[1] || op->pad[2] )
+        return -EINVAL;
+
+    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+    if ( overlay_fdt == NULL )
+        return -ENOMEM;
+
+    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+    if ( ret )
+    {
+        gprintk(XENLOG_ERR, "copy from guest failed\n");
+        xfree(overlay_fdt);
+
+        return -EFAULT;
+    }
+
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH )
+        ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+    else
+        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+
+    xfree(overlay_fdt);
+
+    return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..81efb39014 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+#if defined(__arm__) || defined(__aarch64__)
+struct xen_domctl_dt_overlay {
+    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
+    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
+#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
+#define XEN_DOMCTL_DT_OVERLAY_DETACH                2
+    uint8_t overlay_op;                     /* IN: Attach or detach. */
+    uint8_t pad[3];                         /* IN: Must be zero. */
+};
+#endif
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1288,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_dt_overlay                    87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1351,9 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+#if defined(__arm__) || defined(__aarch64__)
+        struct xen_domctl_dt_overlay        dt_overlay;
+#endif
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index febaa4b16a..3a6e7d48f0 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
 DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
 /*
  * XEN_SYSCTL_dt_overlay
- * Performs addition/removal of device tree nodes under parent node using dtbo.
- * This does in three steps:
- *  - Adds/Removes the nodes from dt_host.
- *  - Adds/Removes IRQ permission for the nodes.
- *  - Adds/Removes MMIO accesses.
+ * Performs addition/removal of device tree nodes under parent node using dtbo
+ * from dt_host.
  */
 struct xen_sysctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
@@ -1265,7 +1262,7 @@ struct xen_sysctl {
         struct xen_sysctl_cpu_policy        cpu_policy;
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
         struct xen_sysctl_dt_overlay        dt_overlay;
 #endif
         uint8_t                             pad[128];
diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
index c0567741ee..7f0f3741b5 100644
--- a/xen/include/xen/dt-overlay.h
+++ b/xen/include/xen/dt-overlay.h
@@ -39,15 +39,22 @@ struct overlay_track {
 };
 
 struct xen_sysctl_dt_overlay;
+struct xen_domctl_dt_overlay;
 
 #ifdef CONFIG_OVERLAY_DTB
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
 #else
 #include <xen/errno.h>
 static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     return -EOPNOTSUPP;
 }
+static inline long dt_overlay_domctl(struct domain *d,
+                                     struct xen_domctl_dt_overlay *op)
+{
+    return -EOPNOTSUPP;
+}
 #endif
 
 #endif /* __XEN_DT_OVERLAY_H__ */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 04:36:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 04:36:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726506.1130854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEd-0000BB-5d; Tue, 21 May 2024 04:36:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726506.1130854; Tue, 21 May 2024 04:36:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9HEc-000091-Te; Tue, 21 May 2024 04:36:22 +0000
Received: by outflank-mailman (input) for mailman id 726506;
 Tue, 21 May 2024 04:36:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9HEb-0006hu-QU
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:21 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aaab29f0-172b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 06:36:19 +0200 (CEST)
Received: from SA9PR13CA0038.namprd13.prod.outlook.com (2603:10b6:806:22::13)
 by LV8PR12MB9184.namprd12.prod.outlook.com (2603:10b6:408:18f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 04:36:17 +0000
Received: from SA2PEPF00001507.namprd04.prod.outlook.com
 (2603:10b6:806:22:cafe::c3) by SA9PR13CA0038.outlook.office365.com
 (2603:10b6:806:22::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 04:36:17 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:17 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May
 2024 23:36:16 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 20 May 2024 23:36:14 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aaab29f0-172b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fywDq+F8ZGMrfUxKPqSqqF3oAolIRpEEbBFbzBbAa8KXVzudey5rsgsIMbNRln5vrQHdRLJ9ab5T7pk215gEzEvj0OsmDfhE1f8optPrabs0UyZOvnWuJf42OMe5FZ4qY0ovnWcH4D23pXFa0mUa4XuR2B+HMp7YC8+Q45sJqq0Mi3hUq9FeJ9HH2/UQe1qKq/dw6P4b8/JEI1kGeEp8Fu/jeWaeu9/moU+vo09wA+yWHLlvSAMMaMK93PEW80sUbWEgkZoUtJ45OyNrlU8iwhfeYaPSwuShMpH2WAUL27clBZPq8S0/OckiRJeYlDnASdMpd03zChjsImOe0+kOVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=XLkeA8mEKzf4yLdB3J64v/O8UY7F2ncd41W/P3+Us/g=;
 b=UUub2UTTVSpd/hDLNf1Id5Kw4FolJSjfqMNtapVVOJr0CNMcYp4XXmDzQiz84aapscx3VbC0ybGwWqvDLhIKkC6woFYbNOfCdPGh4TCCq6t3L18JbWcXgtri1/g5VaVd5xq2rHrpDrayXC1RGXxnx8mO1QyFa4bBClO5NPBrpRpA6+An2BX1tr5U0W0ehABmiPadlouqzF74Vxl0pXiUyrdabfPynS5j4LbZLXSdfE7GYygv2M0EYjfhAHZcnqe0g6sKXhn2X6DpqPYXQrlJIr5bGO2moTryTgPpm1j2BBUdDZsCzudJc19YU9PPryCoFSjnQ0zvxJQnIX3yxgjN3w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XLkeA8mEKzf4yLdB3J64v/O8UY7F2ncd41W/P3+Us/g=;
 b=JDTEYIVXoeRRJgh+spDH81Yzkfx9TPDdd7H346WpzdFVni27wvGUP/7dE5HUaZsWJBFCG2HnpRO/FI4YAibEySljg+P7qfVlNeUlBseFSmA+LjZgcH1uZ9Byw5tniT8KGp/+OLHbj71/mfFnQLGtJOOFC2/77YDBx1dtaWUBAZU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v3 8/8] docs: Add device tree overlay documentation
Date: Tue, 21 May 2024 12:35:57 +0800
Message-ID: <20240521043557.1580753-9-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|LV8PR12MB9184:EE_
X-MS-Office365-Filtering-Correlation-Id: 6fc71ecd-dcde-4d0a-93da-08dc794f8e08
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?TBe6ZPZElNPFoIHFtmMPTFQQ9IZdZDGoI6LfPDW8BdhsaoX3Uy1pdWfPHH05?=
 =?us-ascii?Q?aidU7lt3oXxEXHQjtrBCxlbQq5KOnisUzbbhiQolp1BAM3RjxFiDfFAgmhmg?=
 =?us-ascii?Q?dL1VuA/Upb0OfbwtU+Uqs9cnOyT4UNxBuM0SIaDo5rmZqxqGqynnZzJaGt9s?=
 =?us-ascii?Q?2kAUQ7FFoQ+n1HJmElJR7b37QqyjXhm3UE9kVsXYL7p1Yi+RZMasUSYfp/np?=
 =?us-ascii?Q?1BtP/LgVgKhWHcsxiYGWNRgzOKTfDIXBR3lV4NUeL3c0c9uXtZORXjNw0fE5?=
 =?us-ascii?Q?smxIfxB4LPHyyDVFrjRJ02em5GH7K31i7uxoW8awXiGFepv3jdlNn3zC8LJc?=
 =?us-ascii?Q?GBGlPcEwHNb9IDif/kWw5PaWXKBj7XIo6eTQE0fs+vnHiKUjoGGDIo13mjg+?=
 =?us-ascii?Q?OWlJEJxkblHavDEwq0yfIcEU3n6X0vhxQAkk1sNtmw2fe4ZNP4bwICahgWM0?=
 =?us-ascii?Q?/bGbKGZwbWLbCHxcVbXerDMx8Dqgt1J1sOi+BjWH72GhaauoqTY4niLCxA9Z?=
 =?us-ascii?Q?0oWbSNu5GE1o+Sv5rlnWOkcWflKFr6TA3UQDaX7/rlhIRd7vNGP+avEL/Jc/?=
 =?us-ascii?Q?xP9GjhiaObiBey8O79FDb28sDntQwC9kGth8+gpY4o+Hc0dElZm0H62aifQ2?=
 =?us-ascii?Q?MGmeZBhyKxR1fvw2DIG3UwwXlGQmEogDYPfxKIecucdUsidVQ1d4MdTKmiE6?=
 =?us-ascii?Q?ohXXbIoqp6a0bpCeIucUCrbofKAgVZ/u4KtR+/1g/u1wSnr6zpc0UXfZhx3K?=
 =?us-ascii?Q?wkVQ6X3FG1+2uFlocRz3cPjlwfbbKe+CfFJI8kJPzIXvojV3t7xseqSw5stb?=
 =?us-ascii?Q?QuD3saALNrTWej5pI4yP3uo3OHX4bMthi+lfFFjv5WjNI5fg6CG+I8AxfHs9?=
 =?us-ascii?Q?whFINvUnol8u9XvRc7k4UyPH+ofMRoPydf/9YnerRb2w0g9xumXvMa6aWIKj?=
 =?us-ascii?Q?70hAIPUrp2QMyTo/AMa+gOcfcaS7rSI7/NcL8VPD+6kHbwTW9UL7LgbMHu0T?=
 =?us-ascii?Q?feUusEEaQKclleAMAMLw1n+VX8QWmeiduGPEB34LOE8pnQYOGHBPrKhXPe+m?=
 =?us-ascii?Q?Bxjk8OdJWSwlG9jcvvAp1iHLQ7yrbhcjXIYeY2xvyItdn4MEGKpNclZ6sTmS?=
 =?us-ascii?Q?iNxpnO9CowvtGiLtCRgs/RH0qVSOoDg11KOjFdtPmllmxRZ0Kxg6GWK/19D1?=
 =?us-ascii?Q?h3F4OL9iFN5WMk3aFZ9CBdcXfDJygGon7LpzZNA4oSERyvCkZhWmJzlQUDek?=
 =?us-ascii?Q?0AVZ5hVgx+FtL+x08FshCcysfgtwZJHAzEVv/Ux9TKW4k7wzOxwqu4rl1iq9?=
 =?us-ascii?Q?LmYmrDcjdxIdyzElOozVliSj?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:17.1446
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6fc71ecd-dcde-4d0a-93da-08dc794f8e08
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001507.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9184

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v3:
- No change.
v2:
- Update the content based on the changes in this version.
---
 docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
 create mode 100644 docs/misc/arm/overlay.txt

diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
new file mode 100644
index 0000000000..811a6de369
--- /dev/null
+++ b/docs/misc/arm/overlay.txt
@@ -0,0 +1,99 @@
+# Device Tree Overlays support in Xen
+
+Xen now supports dynamic device assignment to running domains,
+i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and
+attaching/detaching them to/from a running domain with given $domid.
+
+Dynamic node assignment works in two steps:
+
+## Add/Remove device tree overlay to/from Xen device tree
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
+
+## Attach/Detach device from the DT overlay to/from domain
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor attach/detach the device to/from the user-provided $domid by
+   mapping/unmapping node resources in the DT overlay.
+
+# Examples
+
+Here are a few examples on how to use it.
+
+## Dom0 device add
+
+For assigning a device tree overlay to Dom0, user should firstly properly
+prepare the DT overlay. More information about device tree overlays can be
+found in [1]. Then, in Dom0, enter the following:
+
+    (dom0) xl dt-overlay add overlay.dtbo
+
+This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
+
+To assign the newly added device from the dtbo to Dom0:
+
+    (dom0) xl dt-overlay attach overlay.dtbo 0
+
+Next, if the user wants to add the same device tree overlay to dom0
+Linux, execute the following:
+
+    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally if needed, the relevant Linux kernel drive can be loaded using:
+
+    (dom0) modprobe module_name.ko
+
+## Dom0 device remove
+
+For removing the device from Dom0, first detach the device from Dom0:
+
+    (dom0) xl dt-overlay detach overlay.dtbo 0
+
+NOTE: The user is expected to unload any Linux kernel modules which
+might be accessing the devices in overlay.dtbo before detach the device.
+Detaching devices without unloading the modules might result in a crash.
+
+Then remove the overlay from Xen device tree:
+
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+## DomU device add/remove
+
+All the nodes in dtbo will be assigned to a domain; the user will need
+to prepare the dtb for the domU. For example, the `interrupt-parent` property
+of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`.
+Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
+
+User will need to create the DomU with below properties properly configured
+in the xl config file:
+- `iomem`
+- `passthrough` (if IOMMU is needed)
+
+User will also need to modprobe the relevant drivers.
+
+Example for domU device add:
+
+    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
+    (dom0) xl dt-overlay attach overlay.dtbo $domid
+    (dom0) xl console $domid                         # To access $domid console
+
+Next, if the user needs to modify/prepare the overlay.dtbo suitable for
+the domU:
+
+    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally, if needed, the relevant Linux kernel drive can be probed:
+
+    (domU) modprobe module_name.ko
+
+Example for domU overlay remove:
+
+    (dom0) xl dt-overlay detach overlay.dtbo $domid
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 05:06:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 05:06:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726558.1130872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Hhv-0000Ci-EC; Tue, 21 May 2024 05:06:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726558.1130872; Tue, 21 May 2024 05:06:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Hhv-0000Cb-BS; Tue, 21 May 2024 05:06:39 +0000
Received: by outflank-mailman (input) for mailman id 726558;
 Tue, 21 May 2024 05:06:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Hhu-0000CR-Ki; Tue, 21 May 2024 05:06:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Hhu-0003MD-Gp; Tue, 21 May 2024 05:06:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Hhu-0006yo-4m; Tue, 21 May 2024 05:06:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Hhu-0005xt-4P; Tue, 21 May 2024 05:06:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OpqRdH1+z5hVE21htFc1ATsrLh1VS2QgOke+9E1kOh4=; b=vH9bfK0cHpPgfXAHAnS3FRz44I
	hKV1wkKyondy87lsN5+eJIxBNWDDxiLXS8tBF+9oH+peWWDM9WvneRGIEXrInTNINlWf8Z/noH2NI
	PCR4WLBV6UBhZZWG+L5Wn8kBcTyjSmzARbz0m18oYnO6ttdPD4DWdT3sqBJp6JvkjYA4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186049-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186049: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=54aa34fc89151c943550541e0af47664fb6e8676
X-Osstest-Versions-That:
    xen=21611c68702dae2e18cb519a6e166cdceeaff4ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 05:06:38 +0000

flight 186049 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186049/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186042
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186042
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186042
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186042
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186042
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186042
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  54aa34fc89151c943550541e0af47664fb6e8676
baseline version:
 xen                  21611c68702dae2e18cb519a6e166cdceeaff4ca

Last test of basis   186042  2024-05-20 01:53:34 Z    1 days
Testing same since   186049  2024-05-20 18:07:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Henry Wang <xin.wang2@amd.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   21611c6870..54aa34fc89  54aa34fc89151c943550541e0af47664fb6e8676 -> master


From xen-devel-bounces@lists.xenproject.org Tue May 21 06:06:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 06:06:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726573.1130890 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9IdD-00016L-Tr; Tue, 21 May 2024 06:05:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726573.1130890; Tue, 21 May 2024 06:05:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9IdD-00016E-PN; Tue, 21 May 2024 06:05:51 +0000
Received: by outflank-mailman (input) for mailman id 726573;
 Tue, 21 May 2024 06:05:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9IdC-000168-46
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 06:05:50 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b5547ec-1738-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 08:05:48 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-571be483ccaso8061002a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 23:05:48 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574ec5fb906sm8725595a12.30.2024.05.20.23.05.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 23:05:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b5547ec-1738-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716271548; x=1716876348; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8a67SusQOOo6uuZnruLFLJSizKvhZntR0mjj5FUY0FA=;
        b=JXfM/X3planU0mwk6KFZnzextjG0zFOK+4dguU929fXCxD5sidyRtTrPFPEhP3Ut5s
         XCbNDc7Lxl5s8+4MXOS0pvyIocJwVaQTBDIIUxOvrPBoRjGaRTAhfxVqfxOLcJZar0ve
         VaTmlDiRbOqHuqUEyC+KG8RJzmlwOGO73ngR6iYkRaiD9t3w4BCOdLqP02ktilpiRYsq
         dZ7SFb5eTxf+sLjmzdJi4DpSW1Oparij59qHTtDx88nn5+u+1MxsmOR49NIZ1FDTMae5
         XYAWtQ95cFfFNAGSarhCACiWDLX+oX07JpSGF//Vg6VJr2TRS+LdtKLCff5IX/G+pCyN
         kaFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716271548; x=1716876348;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8a67SusQOOo6uuZnruLFLJSizKvhZntR0mjj5FUY0FA=;
        b=Gzq5kQPDZlreqkVFUEJQpTmohF2dtbAkKQQUQi6Ys2ovS2JHFouJUgXlkqs0N32qDX
         k4GVwAJpUPlfksuWWglKq0ZuVD5FBTQznih9XdkEKYH9Ti1LYJr+9kUxs7ci1kem170z
         iZp+d7rZk6UYTnaN+x7xXROeEWDk8lvG8q8E5nEj9/azFim5ydso+DOTDX0J657gfJ3m
         L4oEfLg/bYojodZ+SLK8j29rDm0TuomEDiNV3o0m5nUXkMFK/oW3bcXkL/TR4aOdEnK1
         LhtDpx6q4u4NCn+yWaj87XVP3rKCni+QbHDr9WNySL2GdBlORH8R8YcSdIAmSEuPpqR2
         5U4w==
X-Forwarded-Encrypted: i=1; AJvYcCXpMyK6Smk4SlZ6tdzEjVFTbjBvtmqRYOUsjHLnIpQecbXUESDWAG/s7RFaJt3xyQ6Y6KuTo66Rw3CSAypQ1C5sNnSgnjdf4t0g55mj35w=
X-Gm-Message-State: AOJu0YyPgLVkwGWGv7ti4EwpLNGpZAD7UIAuphYsXC6urXekjLkWxjc+
	cM6/8jzz5ZPHAAsF3LQy9WeC++TVDgSSyGGY1A6z4IVXoWLwyZY7Xon1h0r0MA==
X-Google-Smtp-Source: AGHT+IH8vuWsyWqODV3gdg/udFHQjdc0SH8kFNPRhT/eczjbN0opEPH9hZ9z4M1T6yC61/VsiGnGYg==
X-Received: by 2002:a50:8ac1:0:b0:578:258c:1091 with SMTP id 4fb4d7f45d1cf-578258c11d6mr152477a12.4.1716271548038;
        Mon, 20 May 2024 23:05:48 -0700 (PDT)
Message-ID: <276192d0-9119-467c-ba8d-b4d808863364@suse.com>
Date: Tue, 21 May 2024 08:05:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to
 header
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
 <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
 <68e2b3b5-649f-4e53-8700-350bdb8e5b09@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <68e2b3b5-649f-4e53-8700-350bdb8e5b09@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.05.2024 11:32, Sergiy Kibrik wrote:
> 16.05.24 12:39, Jan Beulich:
>> On 14.05.2024 10:20, Sergiy Kibrik wrote:
>>> Moving this function out of mce_intel.c would make it possible to disable
>>> build of Intel MCE code later on, because the function gets called from
>>> common x86 code.
>>
>> Why "would"? "Will" or "is going to" would seem more to the point to me.
> 
> yes, sure
> 
>> But anyway.
>>
>>> --- a/xen/arch/x86/cpu/mcheck/mce.h
>>> +++ b/xen/arch/x86/cpu/mcheck/mce.h
>>> @@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v, uint32_t msr)
>>>       return 0;
>>>   }
>>>   
>>> +static inline bool vmce_has_lmce(const struct vcpu *v)
>>> +{
>>> +    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
>>> +}
>>
>> Is there a particular reason this is placed here, rather than ...
>>
>>> --- a/xen/arch/x86/include/asm/mce.h
>>> +++ b/xen/arch/x86/include/asm/mce.h
>>> @@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
>>>   extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu *ctxt);
>>>   extern int vmce_wrmsr(uint32_t msr, uint64_t val);
>>>   extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
>>> -extern bool vmce_has_lmce(const struct vcpu *v);
>>>   extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);
>>
>> ... in the file the declaration was in, thus avoiding ...
>>
>>> --- a/xen/arch/x86/msr.c
>>> +++ b/xen/arch/x86/msr.c
>>> @@ -24,6 +24,8 @@
>>>   
>>>   #include <public/hvm/params.h>
>>>   
>>> +#include "cpu/mcheck/mce.h"
>>
>> ... the need for such a non-standard, cross-directory #include?
>>
> 
> 
> This is because MCG_LMCE_P is defined in arch/x86/cpu/mcheck/x86_mca.h 
> -- so either MCG_LMCE_P (+ a bunch of MCG_* declarations) has to be 
> moved to common header to be accessible, or local x86_mca.h got to be 
> included from common arch/x86/include/asm/mce.h.
> 
> As for the MCG_* declarations movement I didn't think there's a good 
> enough reason to do it; as for the inclusion of x86_mca.h it didn't look 
> nice at all.

I'm afraid I don't follow the latter: Why's including x86_mca.h any worse
than what you do right now?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 06:13:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 06:13:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726580.1130901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Ikn-0002qb-Lp; Tue, 21 May 2024 06:13:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726580.1130901; Tue, 21 May 2024 06:13:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Ikn-0002qU-HP; Tue, 21 May 2024 06:13:41 +0000
Received: by outflank-mailman (input) for mailman id 726580;
 Tue, 21 May 2024 06:13:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Ikl-0002qO-CN
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 06:13:39 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4362e1df-1739-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 08:13:38 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-574f7c0bab4so10581910a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 23:13:38 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a614224cb3csm249449566b.13.2024.05.20.23.13.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 23:13:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4362e1df-1739-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716272018; x=1716876818; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4Py0J/8XG38Oeg/B/OwG3ASmwChchTQyQzCMIKm8DEk=;
        b=MwyUGyGeruF2dxDpYsaisyLPwSLXUg9RqsX3feyNXSZYvKeN9RdPC5PCFiARY4qXpF
         INa6ABu+2Z5e5RsAbxRjnyVrGq5bbRkNuqII0Vh+M4i3JQqRVVRYgrFjxs+s6iBrHZdV
         ndhvLBg6+3dJIek9J4Hyp7yo+6tf8uFNS9eonxei/Hk0WZwOd4tDzJ0ssqxiMgj4jqcn
         atGjkTclqmYyAZUm++kO7roHGkXdZ9QL16rKQn6nvjQ4Pg9W5GPeYM7cXSVOLG3snFXC
         xoXS1MJ5SzkdabB0N5qY7RM96Pi+7yHIwYAI6j2C34WK8ZZ72mvAkemKZ/FfkZSS984H
         vlOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716272018; x=1716876818;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4Py0J/8XG38Oeg/B/OwG3ASmwChchTQyQzCMIKm8DEk=;
        b=jImjvm42qD+qZ/EuzPMBm4CUDPFbcZ31m2mV1WQfkCPaTHpMzx4QUHXdRXSA2gEybr
         nTlhyG7WE4RC5AHn/akYm29ml6FWIn0AusDIx011DDd0HWl4XnZnoAG4RH+V37tMhxql
         TsdsaFkDVfbLgA+A8bXb9mEti4EQGLr0rTise8kBOnb2NEF9AgUKAq8W6RuRJ1shsndq
         0iIZcVqw0vdxKUf6UFo4sA8ZmyWM6HUhPuRB2bud13RDV4fBNlWX2tq5Qa1Od/bBNZ5l
         9nPzG23sWeNjsGPpcaJ4EIXMntxBC+3ML3F8c7HClUfP1UYThXaFZjWsvlMioM/DHK2Q
         UqjQ==
X-Gm-Message-State: AOJu0Yzgihe57lOCmwXN6jYMxVf8VvNFJfFyCg/Qp1Kji4pmJ1PXHq6E
	IqN9xbGsGL3F/tEuRpbdGd5gPmu29OxFHSc3U6h6YsXxw9Q2teZB3XWua38FDEX6LGspRyrFi/U
	=
X-Google-Smtp-Source: AGHT+IGikqzXrDPgTGwZ1UqAgpXY/STcOmJJlk2+dLaMbdWw2LRCr3xUdZj26NLThTj1PfGdl7UNqw==
X-Received: by 2002:a17:906:cf83:b0:a5a:5c8f:120d with SMTP id a640c23a62f3a-a5d5fd58885mr659250066b.36.1716272017749;
        Mon, 20 May 2024 23:13:37 -0700 (PDT)
Message-ID: <1bc341cb-7620-4597-9e35-187758dc28ce@suse.com>
Date: Tue, 21 May 2024 08:13:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: Add install/uninstall targets to
 tests/x86_emulator
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240516110710.3446-1-alejandro.vallejo@cloud.com>
 <ZkXvdWf2mVkPW458@macbook> <7965ba74-9177-44b7-9253-7ab5a77fd88f@cloud.com>
 <ce573d24-1d52-48a8-b3f0-8636dacd5c9b@suse.com>
 <26f1627c-4325-42f0-bbb1-0bd9be35bbce@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <26f1627c-4325-42f0-bbb1-0bd9be35bbce@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 16:46, Alejandro Vallejo wrote:
> Hi,
> 
> On 16/05/2024 13:37, Jan Beulich wrote:
>> On 16.05.2024 14:29, Alejandro Vallejo wrote:
>>> On 16/05/2024 12:35, Roger Pau Monné wrote:
>>>> On Thu, May 16, 2024 at 12:07:10PM +0100, Alejandro Vallejo wrote:
>>>>> Bring test_x86_emulator in line with other tests by adding
>>>>> install/uninstall rules.
>>>>>
>>>>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>>>>> ---
>>>>>  tools/tests/x86_emulator/Makefile | 11 +++++++++--
>>>>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/tools/tests/x86_emulator/Makefile b/tools/tests/x86_emulator/Makefile
>>>>> index 834b2112e7fe..30edf7e0185d 100644
>>>>> --- a/tools/tests/x86_emulator/Makefile
>>>>> +++ b/tools/tests/x86_emulator/Makefile
>>>>> @@ -269,8 +269,15 @@ clean:
>>>>>  .PHONY: distclean
>>>>>  distclean: clean
>>>>>  
>>>>> -.PHONY: install uninstall
>>>>> -install uninstall:
>>>>> +.PHONY: install
>>>>> +install: all
>>>>> +	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
>>>>> +	$(if $(TARGET-y),$(INSTALL_PROG) $(TARGET-y) $(DESTDIR)$(LIBEXEC_BIN))
>>>>> +
>>>>> +.PHONY: uninstall
>>>>> +uninstall:
>>>>> +	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGET-y))
>>>>> +
>>>>
>>>> FWIW, should you check that HOSTCC == CC before installing?  Otherwise
>>>> I'm unsure of the result in cross-compiled builds, as the x86_emulator
>>>> is built with HOSTCC, not CC.
>>>>
>>>> Thanks, Roger.
>>>
>>> Right...
>>>
>>> More generally, should we do s/CC/HOSTCC/ on all compiler checks? I see
>>> no particular reason to do them on $(CC) rather than the actual compiler
>>> used during build.
>>
>> No. There really is a mix here, intentionally. Anything built through testcase.mk
>> is using CC, and hence respective checking needs to use CC, too. That said, I
>> don't think the split is done quite correctly just yet, which may raise the
>> question of whether having the split is actually worth it.
> 
> I'm a bit puzzled by this. Why do we compile pieces of the test binary
> with different toolchains?
> 
> At a glance it seems inconsequential in the native case and
> fully broken on the cross-compiled case (which I guess is what Roger was
> hinting at and I failed to notice).
> 
> Why the distinction? What am I missing?

It's convoluted and not fully consistent, yes. Consider for a moment that the
emulator truly was what its name says, i.e. not merely re-playing insns. In
such a case it could be run on non-x86, while still emulating x86 code. Thus
code being emulated and code doing the emulation would necessarily need to be
built with different compilers.

It being (in most cases) merely replaying, the boundary has been fuzzy for a
very long time: While for most parts it's clear what group they fall into,
test_x86_emulator.c itself is (has become? even 3.2.3 already has at least
one instance) a hybrid. Yet in principle this file should also be buildable
with the (x86 or non-x86) host compiler.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 06:17:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 06:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726585.1130909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Io9-0003Rf-2N; Tue, 21 May 2024 06:17:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726585.1130909; Tue, 21 May 2024 06:17:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Io8-0003RY-W7; Tue, 21 May 2024 06:17:08 +0000
Received: by outflank-mailman (input) for mailman id 726585;
 Tue, 21 May 2024 06:17:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Io7-0003RS-OC
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 06:17:07 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bef42d9f-1739-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 08:17:05 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a599c55055dso774640466b.0
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 23:17:05 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7fdasm1558866466b.128.2024.05.20.23.17.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 23:17:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bef42d9f-1739-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716272225; x=1716877025; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7fbuPSgCj+Zx3hOeIkqBZTYf9SaBHpm0cpXaZth3lHM=;
        b=a43ed6n+Rt1zUgJHjWTRh09p84+HqxTo7XdSej1OL1v3p6knUjaeSGKoFLywfeJCB2
         5iD9ONnIP8cNzy3aSSucw+QwaHVE2x9z17Qq9fKgtuwwh1XRWxYDEDfhhsTOE9ye5T5p
         DS5q5Sh5qCXyEakL5+Qzs4uymP/Kvp+PFMsWnwH/76r+kPMedVMo8VSAZ95SzRmXh1qQ
         wgYjSBjdVd7GhBLtWmlo/W/U8lZZIASGI6veyqPLTqdllCKK7QDpWVamQ1yw/Vs3uPIp
         5jrT/bHNvTh7dVYMo9yvGmbZLu54m4JAPHUSl6qczwzoUzlKGkLOXsJOt8gSN7y8CT3f
         u5xA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716272225; x=1716877025;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7fbuPSgCj+Zx3hOeIkqBZTYf9SaBHpm0cpXaZth3lHM=;
        b=cvEec81n9WQMlgXR10O+bkxAn5BqjKlgFAjbs7X44L+YiZ/a9Xfdok+LoJJYbr3phY
         lQHoBfGWsPeQTZqYse8LMPI/Z+Zk1CZnEXzSP7d4v9/1pdRD9yh+GMwfIGSxiwf3ic8p
         moIfLNSuuoEuYOfV6Co3GAMT55NVErQGfLMFCZ6MjmF/M0i5VpaL9XFyiPGvVAI/DxVp
         8Ke8wh5Lubuh98rl2T6BlfsR20vmKUd7wK2N9Ovx08ZZEMDUQZdcwIj6UIdIjgEdQNeI
         y/q/IoxqyIEN4UrKjuT9MwDJuAnUWou3+JRWBNykYEOB5HAR1h5LIDcrs3oSYFCYkr7o
         hpJg==
X-Forwarded-Encrypted: i=1; AJvYcCU9OdWUeA4SfZHb+wfCodIaRZt1tUjTEohsD+eiGU+3WJ7JWNKK1Z7MLyy6+AC6xrE1/kSRO7hEYicmAXaNzERJmlypsON5Yy9sDUrVFWg=
X-Gm-Message-State: AOJu0YxLvQU6dyjVD4HPaNBJSIwbsUYEgdMiOklV1dtG2wGn7ctmO2wM
	HFZUnXMWYOZoSOmThToEvjlw31eVwrKGVf+JNMQmcZhfkgDwhzH86eDuNhmk+Q==
X-Google-Smtp-Source: AGHT+IH0kheLRJErklJaJgVykhVGrbQIdujwZOwUq/+KL+wnwNUFEWCbglvzqGF9VO774F5RHeS0zw==
X-Received: by 2002:a17:906:4083:b0:a62:c5f:e34a with SMTP id a640c23a62f3a-a620c5fe404mr103671766b.39.1716272225195;
        Mon, 20 May 2024 23:17:05 -0700 (PDT)
Message-ID: <3cfee629-b9bc-4b3d-9e4e-ff7e54e701d1@suse.com>
Date: Tue, 21 May 2024 08:17:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <George.Dunlap@citrix.com>,
 Daniel Smith <dpsmith@apertussolutions.com>, Julien Grall <julien@xen.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com>
 <11957460-0b2b-432d-ad92-38350306c9ff@suse.com>
 <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com>
 <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com>
 <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
 <e75ca24e-12fe-44ac-9c67-bcc222ac8752@suse.com>
 <alpine.DEB.2.22.394.2405171328250.1052252@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405171328250.1052252@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 22:28, Stefano Stabellini wrote:
> On Fri, 17 May 2024, Jan Beulich wrote:
>> On 17.05.2024 03:21, Stefano Stabellini wrote:
>>> On Thu, 16 May 2024, Jan Beulich wrote:
>>>> 1) In the discussion George claimed that exposing status information in
>>>> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
>>>> how a similar assumption by CPU designers has led to a flood of
>>>> vulnerabilities over the last 6+ years. Information exposure imo is never
>>>> okay, unless it can be _proven_ that absolutely nothing "useful" can be
>>>> inferred from it. (I'm having difficulty seeing how such a proof might
>>>> look like.)
>>>
>>> Many would agree that it is better not to expose status information in
>>> an uncontrolled manner. Anyway, let's focus on the actionable.
>>>
>>>
>>>> 2) Me pointing out that the XSM hook might similarly get in the way of
>>>> debugging, Andrew suggested that this is not an issue because any sensible
>>>> XSM policy used in such an environment would grant sufficient privilege to
>>>> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
>>>> for Xen-internal event channels. The debugging argument then becomes weak,
>>>> as in that case the XSM hook is possibly going to get in the way.
>>>>
>>>> 3) In the discussion Andrew further gave the impression that evtchn_send()
>>>> had no XSM check. Yet it has; the difference to evtchn_status() is that
>>>> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
>>>> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
>>>> similarly useful to allow getting a stuck channel unstuck.)
>>>>
>>>> In summary I continue to think that an outright revert was inappropriate.
>>>> DomU-s should continue to be denied status information on Xen-internal
>>>> event channels, unconditionally and independent of whether dummy, silo, or
>>>> Flask is in use.
>>>
>>> I think DomU-s should continue to be denied status information on
>>> Xen-internal event channels *based on the default dummy, silo, or Flask
>>> policy*. It is not up to us to decide the security policy, only to
>>> enforce it and provide sensible defaults.
>>>
>>> In any case, the XSM_TARGET check in evtchn_status seems to do what we
>>> want?
>>
>> No. XSM_TARGET permits the "owning" (not really, but it's its table) domain
>> access. See xsm_default_action() in xsm/dummy.h.
> 
> Sorry I still don't understand. Why is that a problem? It looks like the
> wanted default behavior?

For ordinary event channels - yes. But not for Xen-internal ones, at least
from my pov.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 06:21:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 06:21:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726592.1130919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9IsR-0005Au-IU; Tue, 21 May 2024 06:21:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726592.1130919; Tue, 21 May 2024 06:21:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9IsR-0005An-G1; Tue, 21 May 2024 06:21:35 +0000
Received: by outflank-mailman (input) for mailman id 726592;
 Tue, 21 May 2024 06:21:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9IsQ-00059w-Hx
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 06:21:34 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5e16b469-173a-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 08:21:32 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-5751bcb3139so5150499a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 23:21:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdd7f8ea1sm721719866b.109.2024.05.20.23.21.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 23:21:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e16b469-173a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716272492; x=1716877292; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RSMjAPAr7SehfQnq3F5P7aKJo2Vu8WmPbIChaMO1iIE=;
        b=fO/6bxqDN3lWK07rjYuulSTsUlphaUIaaXHp652pM4cLwVOeRAzQLD0Lt7O4wIlCM7
         mNiCNoTWwS5BWDbzehL9nq2W553jhVB4uTdGrybhPq/qYzid2E2rcJCwSynTuHnBaKTi
         ij2KxjMDvzvvkm5kbMu85jnpVh9KaiBvjVKYBgjlcA7dMrz8pJu+uFK3vWN9d9o/IW2y
         sN8cToDPlVPtA4VMPD8PwWuiUPxJ+Xq38eTTaRdqhATcKFm4FGVcd2mnieEjYujx/i8m
         U4sktQk7u3oQihctkZhXT0mj4V4mX5TteaM5j8Whjfs94v082OzAbLIlmq3eiWfYZNwv
         7flw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716272492; x=1716877292;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RSMjAPAr7SehfQnq3F5P7aKJo2Vu8WmPbIChaMO1iIE=;
        b=Ov5N9Ft7r1S4eqqZA6czfMVX9G03zP4+zbDgA3KBPVGopMZdlmiboPJwBPX0HK0qpg
         LRNSF/A4bqEeThkRP38MqQbp6P2PCwevy0UHa7ib2hyGFjGHVaaVTVCw1+DwVYYQVUvY
         YvtdH4Z/qvaxr2D26uFbGE5+RiQdjNDaaNczmYL6g4V9jzvCiLDz4AgWkyh+ZKDGy06P
         a3LexCgungsS/QOw3cq/uOyyvB39Bn+HXQMX9/LdnSU/uoZAGOqmG+8zOTp8GQs7rQLG
         Sx1fmcUIP9oZJXJH3gbnKqkC6jr7et12ECRXmEXRDKYUR52FjGpA38rjx9lUzAGWDavY
         ahXQ==
X-Gm-Message-State: AOJu0YxXpgh4NE2d1pF80Q0ACJg5FZJ0d51VKKyIH53pqUZoCnNknuf3
	ohxnnimhM+YogKC4NUZzkdIrJg5vxpF3vO2IRFWi79GuwLehsrSz1zZcBlDqTA==
X-Google-Smtp-Source: AGHT+IGEhYwDlKSRaScqOjniRCwXW1yPntKTRFj3hMx/TZZxoYq8XHE2WnyV22Q7mE3PYbCFNdbSow==
X-Received: by 2002:a17:906:528b:b0:a59:cf0a:4e4d with SMTP id a640c23a62f3a-a5a2d55a730mr2056669966b.12.1716272492285;
        Mon, 20 May 2024 23:21:32 -0700 (PDT)
Message-ID: <d1267c40-e2d6-4919-81ac-54ad217e4702@suse.com>
Date: Tue, 21 May 2024 08:21:35 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com> <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com> <Zjjg2ueqgjmn-MS3@macbook>
 <44af358a-9510-4056-826c-3be99dc25830@suse.com> <ZksmJp5JnQoBYZ6U@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZksmJp5JnQoBYZ6U@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20.05.2024 12:29, Roger Pau Monné wrote:
> On Wed, May 15, 2024 at 12:07:50PM +0200, Jan Beulich wrote:
>> On 06.05.2024 15:53, Roger Pau Monné wrote:
>>> On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
>>>> On 06.05.2024 14:42, Roger Pau Monné wrote:
>>>>> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
>>>>>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>>>>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>>>>>  
>>>>>>          if ( use_ats(pdev, iommu, ivrs_dev) )
>>>>>> -            dte->i = ats_enabled;
>>>>>> +            dte->i = true;
>>>>>
>>>>> Might be easier to just use:
>>>>>
>>>>> dte->i = use_ats(pdev, iommu, ivrs_dev);
>>>>
>>>> I'm hesitant here, as in principle we might be overwriting a "true" by
>>>> "false" then.
>>>
>>> Hm, but that would be fine, what's the point in enabling the IOMMU to
>>> reply to ATS requests if ATS is not enabled on the device?
>>>
>>> IOW: overwriting a "true" with a "false" seem like the correct
>>> behavior if it's based on the output of use_ats().
>>
>> I don't think so, unless there were flow guarantees excluding the possibility
>> of taking this path twice without intermediately disabling the device again.
>> Down from here the enabling of ATS is gated on use_ats(). Hence if, in an
>> earlier invocation, we enabled ATS (and set dte->i), we wouldn't turn off ATS
>> below (there's only code to turn it on), yet with what you suggest we'd clear
>> dte->i.
> 
> Please bear with me, I think I'm confused, why would use_ats(), and if
> that's the case, don't we want to update dte->i so that it matches the
> ATS state?

I'm afraid I can't parse this. Maybe a result of incomplete editing? The
topic is complex enough that I don't want to even try to guess what you
may have meant to ask ...

> Otherwise we would fail to disable IOMMU device address translation
> support if ATS was disabled?

I think the answer here is "no", but with the above I'm not really sure
here, either.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 06:25:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 06:25:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726599.1130931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Iw5-0005or-5g; Tue, 21 May 2024 06:25:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726599.1130931; Tue, 21 May 2024 06:25:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Iw5-0005ok-1W; Tue, 21 May 2024 06:25:21 +0000
Received: by outflank-mailman (input) for mailman id 726599;
 Tue, 21 May 2024 06:25:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Iw4-0005oe-6x
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 06:25:20 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e45bf144-173a-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 08:25:18 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a59c0a6415fso854627266b.1
 for <xen-devel@lists.xenproject.org>; Mon, 20 May 2024 23:25:18 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781d350sm1564244766b.23.2024.05.20.23.25.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 May 2024 23:25:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e45bf144-173a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716272717; x=1716877517; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jWAMIsl7OjzznvuQYy7NRnhhpXRz3kn+3K3LTHeRdGw=;
        b=fLJiXtVE4DPdg+rBMAVWg1cq04+mOjbwB+iLqIPsRz2wNpfvkrjbog/mQCm7dAxAet
         KENCMngvQCSN0SRiOBtWYS8KKPGhFccdVQn18bQHtvwcRaPlZz5eUlz56EqnTajYWIDo
         LmYxw9T4rxGe7HmvLWT0MW+gPIhFWBkeFCW+RukEPnm9hrkV+pA/NS1cSO2muJK9S6Uu
         pbdew+uz9hbzTH6FSCtJFuavGQ2qI3orbLpoy0j6eJeTCxMzYcpmfRr4UHUrZWUPGbJW
         d77dAUP4k3RdM57y+1KHwHzeBJGlyBJBxbCgypAvV5hoUtnnAKrFcJEwiL5BRiO9mdE+
         W8/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716272717; x=1716877517;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jWAMIsl7OjzznvuQYy7NRnhhpXRz3kn+3K3LTHeRdGw=;
        b=XNK2QlA1nf4hjcDkAaL245J17iaqCdOxWeKVlP4KL7veAO45pXZVlmINridUryoyX8
         F6GO7v0hvcoKa0zH264J/shK6Du/D9Yb2+CQU70SJpDoRinI0Ag3xEG99ierywgmoEtq
         wKWC7lEA4akuDGgvdX8gWrZS+mS+SjhOTbC3W/OzS+Ma9zv8n4MgB5W9wE9VqjxAqkyz
         PX2BX6mHaiwUZ/rBO7cA6qEOzkwtDvJ6Sx0/K8sQeOOcUbnkTbk1OksLCiJUl4QTxn8X
         ArRTJ+fSHA09N6EyWHk145SG/uN5vXhySm9+IGTaw6NvjBN/NFDEgMITpOsMq4u/E67W
         J+bg==
X-Gm-Message-State: AOJu0Yw3/YT1KuTi+79nIBe+jG02bWBAGK/DAYpYS19+C69SBfcMa34z
	BRDzlVH4e9hVpHN4vC3aEWtnNeu1nrX6x2D38213nt5j0EwrbgBeFt99yYxIeg==
X-Google-Smtp-Source: AGHT+IGoxVyobR84bSt31y4HWaxgJJ2uJL3R8jBQ17KG3/avDB18+dYliQcbQf3ur6RffFwKxHHHaw==
X-Received: by 2002:a17:906:9e08:b0:a5a:8b17:d851 with SMTP id a640c23a62f3a-a5a8b17db00mr1048558566b.20.1716272717523;
        Mon, 20 May 2024 23:25:17 -0700 (PDT)
Message-ID: <fd5334e0-d54d-48fa-bf7d-c34f9f1a78ab@suse.com>
Date: Tue, 21 May 2024 08:25:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 06/12] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com> <ZjjdZRPluS0YIazc@macbook>
 <e9281523-a807-4889-80f0-a13804188af9@suse.com> <Zks1qFNsQdLkOCAs@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zks1qFNsQdLkOCAs@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20.05.2024 13:36, Roger Pau Monné wrote:
> On Wed, May 15, 2024 at 12:42:40PM +0200, Jan Beulich wrote:
>> On 06.05.2024 15:38, Roger Pau Monné wrote:
>>> On Thu, Feb 15, 2024 at 11:16:11AM +0100, Jan Beulich wrote:
>>>> When the flag is set, permit Dom0 to control the device (no worse than
>>>> what we had before and in line with other "best effort" behavior we use
>>>> when it comes to Dom0),
>>>
>>> I think we should somehow be able to signal dom0 that this device
>>> might not operate as expected, otherwise dom0 might use it and the
>>> device could silently malfunction due to ATS not being enabled.
>>
>> Whatever signaling we invented, no Dom0 would be required to respect it,
>> and for (perhaps quite) some time no Dom0 kernel would even exist to query
>> that property.
>>
>>> Otherwise we should just hide the device from dom0.
>>
>> This would feel wrong to me, almost like a regression from what we had
>> before.
> 
> Exposing a device to dom0 that won't be functional doesn't seem like a
> very wise choice from Xen TBH.

Yes but. That's what we're doing right now, after all.

>>> I assume setting the IOMMU context entry to passthrough mode would
>>> also be fine for such devices that require ATS?
>>
>> I'm afraid I'm lacking the connection of the question to what is being
>> done here. Can you perhaps provide some more context? To provide some
>> context from my side: Using pass-through mode would be excluded when Dom0
>> is PVH. Hence why I'm not getting why we would want to even just consider
>> doing so.
>>
>> Yet, looking at the spec, in pass-through mode translation requests are
>> treated as UR. So maybe your question was towards there needing to be
>> handling (whichever way) for the case where pass-through mode was
>> requested for PV Dom0? The only half-way sensible thing to do in that case
>> that I can think of right now would be to ignore that command line option,
> 
> Hm, maybe I'm confused, but if the IOMMU device context entry is set
> in pass-through mode ATS won't be enabled and hence no translation
> requests would be send from the device?
> 
> IOW, devices listed in the SATC can only mandate ATS enabled when the
> IOMMU is enforcing translation.   IF the IOMMU is not enabled or if
> the device is in passthrough mode then the requirement for having ATS
> enabled no longer applies.

Oh, I think I now get what your original question was about: Instead of
enabling ATS on such devices, we might run them in pass-through mode.
For PV that would appear to be an option, yes. But with PVH (presumably)
being the future I'd be rather hesitant to go that route.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 07:47:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 07:47:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726629.1130963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KDj-00005E-3H; Tue, 21 May 2024 07:47:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726629.1130963; Tue, 21 May 2024 07:47:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KDi-000054-UN; Tue, 21 May 2024 07:47:38 +0000
Received: by outflank-mailman (input) for mailman id 726629;
 Tue, 21 May 2024 07:47:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sbJz=MY=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s9KDh-00004w-MB
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 07:47:37 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 630f97de-1746-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 09:47:35 +0200 (CEST)
Received: from MN2PR12CA0002.namprd12.prod.outlook.com (2603:10b6:208:a8::15)
 by MN0PR12MB6247.namprd12.prod.outlook.com (2603:10b6:208:3c1::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 07:47:32 +0000
Received: from BL6PEPF0001AB77.namprd02.prod.outlook.com
 (2603:10b6:208:a8:cafe::75) by MN2PR12CA0002.outlook.office365.com
 (2603:10b6:208:a8::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Tue, 21 May 2024 07:47:32 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF0001AB77.mail.protection.outlook.com (10.167.242.170) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 07:47:32 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 02:47:31 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 02:47:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 630f97de-1746-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gLkCHtDzi7q3Fg/04gX0NJR3rM0mVDa17etC8rugbRLtgwRIwDYKm7QRsp1Nqb3ic7tDF//+vvqjT9m54suYoJMshqAOJ7HTVE84C5urfoTgboWb7UY4bESnMjds9nFQw3ITUM7Z+Hh+jOWDx81QwOUyH8Fu8DnXNDaNuBrYjYVgAJScOwI/lTrYuKviktsHOEqMsZSnivlou1KcidJOLO2HEN3tufkoscUnqZ4rkGAquubxuFQ2+VsX2NyqXpsdyfKLl6TTeHUuNW0P13zCO7SsjKMfI12DdAQ+76xex0HDP2oY7O1N1bJEXkvRWg7cw9JRv0F4gYe6NUsDMg5oeg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=QjoqvSVRSR9NDC874BgKN3oOL36ifj5ne8qhLOku9b0=;
 b=WDmamD3eGbTsunOWAUY50qky5mj6LkScyEnE1NeHKeVkenJf6teF/GhlapztPkY5PusghhGhMu4KmBbG/6+LB3IwPJhpsNiLIjZjhXBAzSsJuYpJcQ8UPx9+8udlTN1K11+aHw9yXhF6aa3hV1zlpwaPIXd0n+l+qJxGF5A4BQ1oq5mLZXUwAlsIQvvK29FQtw5lB43ijmc8wW/btOexGBxC3nHppZew+ljyCOWbWR3cFpy2N/FIqPVuaEuDBXgGGgZoPYLSmgDpYOts1ZV3FZxXmKhOliQGXuRXwoNTA/d4gDWpNik4CKjPLZmEIGtWvxrcU3dGNv3yxVDoK2IBfw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QjoqvSVRSR9NDC874BgKN3oOL36ifj5ne8qhLOku9b0=;
 b=cDniG10Xnn2uEoch2o5uGDXtJIto8byDK6SLs/iyIcjQoXZwrGRlbGmwLh0kQrnLM01TxRDCUxdITWwkX0MFpCsFKjk4ZW9QTsoFwLpnWNSqXfuppqpAxDdelAuspSwsEgn4cb1iRSN+5bDImOQ1sJYqkxE2NhrxBtMvHKnXIR4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <e724dd97-03fc-4136-9a53-94bbddce160f@amd.com>
Date: Tue, 21 May 2024 09:47:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] xen/arm: Set correct per-cpu cpu_core_mask
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>
References: <20240321035706.165253-1-xin.wang2@amd.com>
 <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB77:EE_|MN0PR12MB6247:EE_
X-MS-Office365-Filtering-Correlation-Id: ce45eb0e-7af5-4dd7-76fe-08dc796a4599
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?bE53c0Z4bWQrblNrdUhETm9EUXBCTzYrbnVubWdlWGdHSmpudUlsMzFBVXl1?=
 =?utf-8?B?bTl4RDByYXUvMUM3NnFUNWZhaDRjMDRVWEw2eUdsL1VkM3lWenhzY3pWNGtK?=
 =?utf-8?B?OThpQU53THZGK1ZtMUM5cm55S1YwOG1sQnBleDBQUXVxNTNFNHNtVkZpTVdL?=
 =?utf-8?B?eFpMc0ZJR0p5R3NVUXNGbkpsV1hHOExUTVlVeE5kVnRVbThYS0tJeVpkVFNC?=
 =?utf-8?B?YzgwWTBTbGszUmRpTmNMZ3Zjc1Q2U3JJY3VaODYxeHM5ZDVHZlYvaWsxY3Bm?=
 =?utf-8?B?WHpma2RIZXo3NmlsNEo1YWtHT29FMmYxQWdRU0Yxb09rYkJaWUtvaEI2SnA1?=
 =?utf-8?B?cm8wVHNaTXpRb09nKzgwYVZVOXZvL05aTWc4UFh2bVVSdVBpTmVFRm13cG5C?=
 =?utf-8?B?bFNjbjRmL0l1M0hySGJnVHRDeklBNzZkWW9wSVRIM0FVbi9xVFVuckY5RFda?=
 =?utf-8?B?ZENSMEp5S2QyZXBRTXREWjdhTjM3eWl4SlhtbFJxYzRicHJJYUlGQzE3TmRK?=
 =?utf-8?B?WWVHNHEwRUZ4ZmZCc2ZacDJaZ2Q0dmROS0NkOGRVenZXQUQwRkVlRnVUTFdQ?=
 =?utf-8?B?OFRsR3hReGJDMGNvQmdZWCtIWjlYdzVoQ0tEMTUvZDdkSHNONVA5UVh5VzRl?=
 =?utf-8?B?ZlhvcDFaeHVjTms3MGtpT2M2U0VDQmtmR05jNGIwemlmdzQ0Y2dEVktvYXE2?=
 =?utf-8?B?QVRaQWNDSWdoYktvYzZYTFYrUiswN3I5c2hDNUttOEc2elJLQk5Pd1VscGRp?=
 =?utf-8?B?b2ZFcE5tRXR5QWtJSVVmV00xekdkZlBGRkhCQTVTVE5QVTd3a2owMzRkdFRi?=
 =?utf-8?B?c1hWcXREdXNicFpXZkg4SWpCQTRIRVoyMHN4SjJabWQvU0xDbDFySEV4SVE5?=
 =?utf-8?B?YjZic0pqbzQ5cklTQk5Ma201ZFV0YlhjdHZZWElTZVBtRjZFWi9vOUYvRCsv?=
 =?utf-8?B?TkxYRkZkQmpBeWt4QkxCVk13WUpEU3VKSDVWZU1McnFCZW1KZEZPejZ2enh0?=
 =?utf-8?B?SEhxVDF3elZUT2dIS2pZL1pVbkMraSs5L0RISTJMTUVXbStLUi9aalprd3Bn?=
 =?utf-8?B?ZWdPTzlHNVhpaGo1L2N2NXI3RXdTMzZPRHg0bE9vSWoxMkp1a0dkU2owZkVh?=
 =?utf-8?B?dHVUaGJxS0tuSVpTMUFXTUJnUHY4SDVEa0VNaHREaFhsZlZ2aGFLTkFwVHZy?=
 =?utf-8?B?YTdubEFWWkh5RXd1YUgrS2JWMkxJdnF1OFVKb3dTNkZNTGE2ck5qTmF6WXZ1?=
 =?utf-8?B?Q2tFZnBzSjlsWTBlTURta2lTOFBHYytiM21qUFJRZGhnTWVsV05xZkdkTmxh?=
 =?utf-8?B?N0w5TGo3Zzl0TkczTnZKdnBZeUIybklTUkd2NWN5cW1CNHV4UCtjRmZvcHoy?=
 =?utf-8?B?N1cvZG9uVTArZzluNlFnZ21kdXZOak1pUitWOGNYNjNaakZDYjYwcWtGNG15?=
 =?utf-8?B?YndEa25JdlpuRzVodzQ3Vm1rVUplWmowL2ZnWXArSHhySUxQVEwrMVRrbGlB?=
 =?utf-8?B?MU9iSjV2UjZZL0R1Uy8vUnBDQ0V1Y1VLbDE5anhqanp2enJqVGRxbXFVWlli?=
 =?utf-8?B?a3Z6NHZidElYK0k5SG1OcFZLSzhmRWJ4enlLK1lQaVh6ZEhwSEcybUVVNlhO?=
 =?utf-8?B?WGRTZXJ5K3lmaWNwYTZubFJQdWYzcEZydUtTK0RyeU55L1FxeU5jVnpjTFVq?=
 =?utf-8?B?V1BsTGlTazdZTmd5VnIvQ21NVnRkOFBSNkRTMy8yZnRob1RHRk1YckNxaFYv?=
 =?utf-8?B?SGZhbUNGYWJMQW9IZVdoSW1xRjk2a3E0bkczYVNwRWU0dU1CT2lKZGNyK2x3?=
 =?utf-8?B?QTVVcVVYYnhXWXl5S2NnYU1pdkxMM3hXOFlSSFdlQ3dOK0pyN0NtSEcwdmxI?=
 =?utf-8?Q?MjyXhqbBmVyk0?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 07:47:32.0478
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ce45eb0e-7af5-4dd7-76fe-08dc796a4599
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB77.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6247

Hi Henry.

On 20/05/2024 04:57, Henry Wang wrote:
> Hi All,
> 
> Gentle ping since it has been a couple of months, any comments on this 
> updated patch? Thanks!
Sorry for the late reply.

> 
> Kind regards,
> Henry
> 
> On 3/21/2024 11:57 AM, Henry Wang wrote:
>> In the common sysctl command XEN_SYSCTL_physinfo, the value of
>> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
>> Currently on Arm this is a fixed value 1 (can be checked via xl info),
>> which is not correct. This is because during the Arm CPU online
>> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
>> cpu_core_mask for itself.
>>
>> cores_per_socket refers to the number of cores that belong to the same
>> socket (NUMA node). Currently Xen on Arm does not support physical
>> CPU hotplug and NUMA, also we assume there is no multithread. Therefore
>> cores_per_socket means all possible CPUs detected from the device
>> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
>> accordingly. Modify the in-code comment which seems to be outdated. Add
>> a warning to users if Xen is running on processors with multithread
>> support.
>>
>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

>> ---
>> v3:
>> - Use cpumask_copy() to set cpu_core_mask and drop the unnecessary
>>    cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu)).
>> - In-code comment adjustments.
>> - Add a warning for multithread.
>> v2:
>> - Do not do the multithread check.
>> ---
>>   xen/arch/arm/smpboot.c | 18 +++++++++++++++---
>>   1 file changed, 15 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
>> index a84e706d77..b6268be27a 100644
>> --- a/xen/arch/arm/smpboot.c
>> +++ b/xen/arch/arm/smpboot.c
>> @@ -66,7 +66,11 @@ static bool cpu_is_dead;
>>   
>>   /* ID of the PCPU we're running on */
>>   DEFINE_PER_CPU(unsigned int, cpu_id);
>> -/* XXX these seem awfully x86ish... */
>> +/*
>> + * Although multithread is part of the Arm spec, there are not many
>> + * processors support multithread and current Xen on Arm assumes there
NIT: s/support/supporting

>> + * is no multithread.
>> + */
>>   /* representing HT siblings of each logical CPU */
>>   DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
>>   /* representing HT and core siblings of each logical CPU */
>> @@ -85,9 +89,13 @@ static int setup_cpu_sibling_map(int cpu)
>>            !zalloc_cpumask_var(&per_cpu(cpu_core_mask, cpu)) )
>>           return -ENOMEM;
>>   
>> -    /* A CPU is a sibling with itself and is always on its own core. */
>> +    /*
>> +     * Currently we assume there is no multithread and NUMA, so
>> +     * a CPU is a sibling with itself, and the all possible CPUs
>> +     * are supposed to belong to the same socket (NUMA node).
>> +     */
>>       cpumask_set_cpu(cpu, per_cpu(cpu_sibling_mask, cpu));
>> -    cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu));
>> +    cpumask_copy(per_cpu(cpu_core_mask, cpu), &cpu_possible_map);
>>   
>>       return 0;
>>   }
>> @@ -277,6 +285,10 @@ void __init smp_init_cpus(void)
>>           warning_add("WARNING: HMP COMPUTING HAS BEEN ENABLED.\n"
>>                       "It has implications on the security and stability of the system,\n"
>>                       "unless the cpu affinity of all domains is specified.\n");
>> +
>> +    if ( system_cpuinfo.mpidr.mt == 1 )
>> +        warning_add("WARNING: MULTITHREADING HAS BEEN DETECTED ON THE PROCESSOR.\n"
>> +                    "It might impact the security of the system.\n");
>>   }
>>   
>>   unsigned int __init smp_get_max_cpus(void)
> 

~Michal


From xen-devel-bounces@lists.xenproject.org Tue May 21 07:51:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 07:51:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726635.1130972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KHq-0001g7-HT; Tue, 21 May 2024 07:51:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726635.1130972; Tue, 21 May 2024 07:51:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KHq-0001g0-EY; Tue, 21 May 2024 07:51:54 +0000
Received: by outflank-mailman (input) for mailman id 726635;
 Tue, 21 May 2024 07:51:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=H+om=MY=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9KHp-0001fu-6s
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 07:51:53 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on2060f.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::60f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb14a6fc-1746-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 09:51:51 +0200 (CEST)
Received: from CH0PR03CA0275.namprd03.prod.outlook.com (2603:10b6:610:e6::10)
 by DS0PR12MB7581.namprd12.prod.outlook.com (2603:10b6:8:13d::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May
 2024 07:51:47 +0000
Received: from CH2PEPF0000014A.namprd02.prod.outlook.com
 (2603:10b6:610:e6:cafe::47) by CH0PR03CA0275.outlook.office365.com
 (2603:10b6:610:e6::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Tue, 21 May 2024 07:51:47 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH2PEPF0000014A.mail.protection.outlook.com (10.167.244.107) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 07:51:47 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 02:51:46 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 02:51:45 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb14a6fc-1746-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bkqlFgee5kmZ4hnm1GgjBfpXxQshRE3GzOZHl8WdfWfxcltXuUUXD8DtWDA6A4VtT4y7+2tO33yksUQrngvMYty9fDfPHBZe/G5uLbZYrgldCfdf96iBnIdQxBWqvm7qeeugldf9VQH/XnkXioF14eteOjCJYhJ96TZzM7ZW0DRtaspEft5jSmTGeLMCkK7f1JqqOf4EZSFVDZDTkJPUivqe8DY1Ouw3MOk5EPW668DCYI9fwKucsCTWrOU6mCzuWXWQ3xAYMZhzNGI8+x2Yo/DZS2tCumR60y4g76wAHW4dZ7caFW3PEzesoMyRpMcgpHueksv5PqRjKEVwczmQzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Ww3zOaVoMdczEdesHSI60+yUBAIUVaUNzvvA/uGMmdY=;
 b=CHLCfyQxHzcRcQF3wK41u4UJkJ/KmFJ+O2UONRtgiBg/UYJv6sRRJvGIYNDMHqFZfhpU7k63Kepg5vf45hXdOAeruDPPK0G9lJ8DgWMHi0sFz8KBggHQTqQmjP7YBBZl0UiB+utivlPZNDbrsZjTqlWkWYtUO+4KURME+jOgjAhk4cGnewReVUaIMTugNRIXJQSVVlSbsBEWpePn70bBTWZgQq1Km3bVN/J1I9ibiQSjRkzNvbs1rYrK0PLqTtgOeZuCKAgopcv0uH9WCpVOYOPtIQgWREiE9Q6RtsxGMl7C/eGaPIcgmBSoThKG4Xo4zYwXrK3SuByNFBYSaTkc8A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ww3zOaVoMdczEdesHSI60+yUBAIUVaUNzvvA/uGMmdY=;
 b=iN/w6jaM94s9D9OhyBi50B63T3ZVSJhuTQg6ur5IschEx08IRWSfKfTgo+uu5NpgT9hl1Iv3bGJUcRKCaHE1/Mtn7BQGSAmV8IhoGwf1g9JY56i2BYew6AeQcFWn+zH4xcTlA2B51xS2BcvUv4PhDVbWwLT+m52c+8QFuXAOtMU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <e81356e7-fcf2-4fdb-8585-e544ac4a15df@amd.com>
Date: Tue, 21 May 2024 15:51:44 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] xen/arm: Set correct per-cpu cpu_core_mask
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>
References: <20240321035706.165253-1-xin.wang2@amd.com>
 <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
 <e724dd97-03fc-4136-9a53-94bbddce160f@amd.com>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <e724dd97-03fc-4136-9a53-94bbddce160f@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF0000014A:EE_|DS0PR12MB7581:EE_
X-MS-Office365-Filtering-Correlation-Id: 4e2637b3-ff28-4925-9472-08dc796adda2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?TnJLSVp0UXZSdy9STldXbEY4cFNNbWFUKzhHODlraEZmalpVN2ZKVXZWaW05?=
 =?utf-8?B?QktMNFJvWnVzdFczUGc5MWt2QlVRS3ppSmNQQVJoSGFVUGhuaDA2MEdVSEth?=
 =?utf-8?B?alR6Z2tVdHJPZGd1SUVoYlZ4b3UzajJJdlprYW5mZEtzcUg5S0FmeW94M00y?=
 =?utf-8?B?cmd6YzNuOGVUNnJLZEcwakJRdFJKanBDWktYK1R1VmFWSEpoanh3NWR5Y0FL?=
 =?utf-8?B?bkJyY0JzbW5DY2V0cG9lRTZURHBwUHpiUjNScndkV0JidVFNcE1mK1lGY0Ux?=
 =?utf-8?B?bkNwT1U2dUlNZ0d1MFBDcWxCSzNFL1AvVjZ2am9mQ0luYzkrdUVhcU8vSVNv?=
 =?utf-8?B?YWU0UThHY0M2TlZsaTN6VUhzZFh0SldYYkRwQVErUXdzdUNrTjBmL3RqMzRD?=
 =?utf-8?B?eG9ldXdoT3pDMEc5VTlqbXBPd0JCQ2VoaEFEYXRCQm5pMUF3R0U0Yy9FRUdy?=
 =?utf-8?B?NVdydlhIQTM3RmxNZFVSaTNzSFV4QjUxM2JwQ1hPTmtlSWRpUnl3VDMrRmUx?=
 =?utf-8?B?aWF5WitGNFNXb2NyQVdaazZlWityMloxYmpUVXB6K3JndXJCaWo3NUs3MitE?=
 =?utf-8?B?VUtEUlFhYnZsSUt0MzUrSUhPNFQ4RXh6OTR1dGNWa1V3M2VjcEZNaXc2bWgr?=
 =?utf-8?B?MFVTTE9nejhHMlpBY0VIVmRGRWNZZUVUaXVKazc2YmV4MS9GZmR2YzFDUWtO?=
 =?utf-8?B?eE9ZTjhRUnRXRzFQdjJuUWZKWEdJYmI1QWw2SEIwNGl5MVQ0MkdBVGd5ck56?=
 =?utf-8?B?VW93Umh3S0Z4RWFrQmx1bzVSNDQ3eXJuSTJDNSs1Q21CV3JNZXU3NHNlRFZ1?=
 =?utf-8?B?Q1oxM1FXUTEyNloweWJBeTQwT2gvMkxtdndjb0lBOUVORDlRa3lFMDV2c3Uv?=
 =?utf-8?B?Qjhyb3JNVUN5U1JqNXdNZUpHUWtqb3k1eDA0UExQRnFTZGhNOU1QS3dHaUtL?=
 =?utf-8?B?MGJlNnprbWY1RjloUVlWdkw3cXk0a1M0R1J5TU9sRHp1VThWLzRWV0tPaFVh?=
 =?utf-8?B?OW9rcmVvTlcrV3h1azh2L3N6NEF0bHhaTk11TW53ZDNFMUhWMzl1QUFXbFpi?=
 =?utf-8?B?MnVuUHNtVHZNdSs3ZVJqMlJ0TndqTzZrTkR6UlRtNU9hVXB2UW15WktlY25B?=
 =?utf-8?B?SFVqL0gwbUF5T2FwS1U3UWc4SE5Qa0F6ZHU1Sk9NYzJ5RzFINjY3SURTOG9u?=
 =?utf-8?B?dVBCTlRWM1ROR3o4U3NOeFJ1amhCQWdHOVVySFFJTzRoYlFlNW1PeUlRWStN?=
 =?utf-8?B?QzQxVzZ5UFp6azBYNzIzS3Zlei85UjRBbTMzaWJaNElqaFI3c2tHcmZ0WG5H?=
 =?utf-8?B?SW0rL1dqTzUxUVRrUk1yNXhlZFhpOTlVUGNFODNJbjl1QlZJMGF5S3dDbElX?=
 =?utf-8?B?dXoyU3VHUjcxZEdYQUxnSW5YT09XVFdxekRBU3I1YnJRclB3ejlBZzk5NkFH?=
 =?utf-8?B?cGlQYmwxOFNWZnBxdHVCYzFYU0ZtVEFUOUtuMlVaRWFyWE1FNHRZUzNWK1JP?=
 =?utf-8?B?MkR5ZEQyNWp1YmxFenRvKzlTZTJvRDdBYU0wSE1VOGJ6d1JiWldwSlVYNmxV?=
 =?utf-8?B?K0x2NFZKaWo1UGNPcElmVmpVTDhvdkdXQnU2UGZxTVpqQWNKWkVjWVJLckZn?=
 =?utf-8?B?WHFIdU45bUprU3lFZ3NseU16RU40YndEdXhDL2xpaUd2V3IvakxBZ1FZQ0Zo?=
 =?utf-8?B?NjNzVUhUTWZHTXJjY1djekoyTW9vbWRhbGxSZ2VTaUJ3eFBrQVYyMGxDZTBx?=
 =?utf-8?B?OVlFTGpFcVRRVGtBNkhlZU0rL0pVb0J6VEJ3MlVldENYQVUzWUtEOTlhQVN3?=
 =?utf-8?Q?7mEpwvrts+8zp1lYDzAmSJ3H5PyAEnTDwJqPw=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 07:51:47.1056
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e2637b3-ff28-4925-9472-08dc796adda2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF0000014A.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7581

Hi Michal,

On 5/21/2024 3:47 PM, Michal Orzel wrote:
> Hi Henry.
>
> On 3/21/2024 11:57 AM, Henry Wang wrote:
>>> In the common sysctl command XEN_SYSCTL_physinfo, the value of
>>> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
>>> Currently on Arm this is a fixed value 1 (can be checked via xl info),
>>> which is not correct. This is because during the Arm CPU online
>>> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
>>> cpu_core_mask for itself.
>>>
>>> cores_per_socket refers to the number of cores that belong to the same
>>> socket (NUMA node). Currently Xen on Arm does not support physical
>>> CPU hotplug and NUMA, also we assume there is no multithread. Therefore
>>> cores_per_socket means all possible CPUs detected from the device
>>> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
>>> accordingly. Modify the in-code comment which seems to be outdated. Add
>>> a warning to users if Xen is running on processors with multithread
>>> support.
>>>
>>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Thanks.

>>>    /* ID of the PCPU we're running on */
>>>    DEFINE_PER_CPU(unsigned int, cpu_id);
>>> -/* XXX these seem awfully x86ish... */
>>> +/*
>>> + * Although multithread is part of the Arm spec, there are not many
>>> + * processors support multithread and current Xen on Arm assumes there
> NIT: s/support/supporting

Sorry, it should have been spotted locally before sending. Anyway, I 
will correct this in v4 with your Reviewed-by tag taken. Thanks for 
pointing this out.

Kind regards,
Henry

>>> __init smp_get_max_cpus(void)
> ~Michal



From xen-devel-bounces@lists.xenproject.org Tue May 21 07:57:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 07:57:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726642.1130982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KN6-0002PI-5k; Tue, 21 May 2024 07:57:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726642.1130982; Tue, 21 May 2024 07:57:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9KN6-0002PB-2X; Tue, 21 May 2024 07:57:20 +0000
Received: by outflank-mailman (input) for mailman id 726642;
 Tue, 21 May 2024 07:57:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sbJz=MY=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s9KN5-0002P5-1C
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 07:57:19 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bdf79c77-1747-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 09:57:18 +0200 (CEST)
Received: from DS7PR05CA0050.namprd05.prod.outlook.com (2603:10b6:8:2f::9) by
 MW4PR12MB8611.namprd12.prod.outlook.com (2603:10b6:303:1ed::8) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7587.35; Tue, 21 May 2024 07:57:14 +0000
Received: from DS1PEPF00017098.namprd05.prod.outlook.com
 (2603:10b6:8:2f:cafe::49) by DS7PR05CA0050.outlook.office365.com
 (2603:10b6:8:2f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 07:57:14 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS1PEPF00017098.mail.protection.outlook.com (10.167.18.102) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 07:57:14 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 02:57:13 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 02:57:12 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdf79c77-1747-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H5tjaY+JkNAboZ1C+WZ89Vz32d0tcsRvGBILO2FMS74HznJDMkbSZYxo9ZYn1FR8uZrES2Y7K8V08VSB/CF+Xoi0SCyUkThWbgv0vSH67nowMrZ0QY9wIlw9XixaDRfxwFR/+5I2R5ltsNpxWGfdRfG08Ptqd9cZq7jU3wfc8oAYfrAQnHQWAvLIG1rrZATzJ/Sg00/dkg1QMyL0bzpm1A+5GZ8/VPFNFwcdrK1zl9GNVx9jtfuzHFu8VzOR4TG1D49lvsHIUFAA84ckzJ7IsGAI2/gwcRBcABOOhgowDl8Nf27vpTKhrQPt+tLxQjyq4533lk53DRWM7OIx553RsQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=rSBbL6llymM5kwICXoSUKun7pq9ChUTp2++7iY5p2uI=;
 b=Bxq9gOTlJ1+/HodywRXf8c8LHlzkLRMxV31eskFz2y+MLaPx0tfGROyGOMeZVoZ2BwwnWfALFuUn2+UH4NL5SuqMi3O/5WYoUJYea/wt8ib5Aryz3rzT5dMQFxL23/BJmKLgN3Cg95K3vl7OIMLfs6g7bvrrPQalXZB37TQ95/lK0qVxb0BZ8WUoj418+7+pAgm6lAXnIZkjwnUrbX7aXt5xQYhcZGrQrkyamI1YtoWzVfMHvlWnJjBLa7t/HPJBRj38lGteHO0DNDm0DfOhPzg3tgtbFTlCD3vNblqIFFojQnsBxI8Aura2ixPmrkWJ6ifPJ/Ef3LnMV/DDuoOL0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rSBbL6llymM5kwICXoSUKun7pq9ChUTp2++7iY5p2uI=;
 b=gyucHzBBnoNgFWNxA75geKcpPWTXn3St3lzbKLWwFt1qZUDL3jynPeMP1U/tZ7HdJ/es9YU2v7VBB69oVB2QhKZtmIXtxk/IKf1hU4pq775qF6xbg6yAfM0GSWVAbyV9jJtJayyp8TdCgnxM+8mAJ9VnhEEPbK24VA1AV1NeJ9s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <e9e188ef-6de5-4cc7-8edd-dd7d004670de@amd.com>
Date: Tue, 21 May 2024 09:57:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] xen/arm: Set correct per-cpu cpu_core_mask
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>
References: <20240321035706.165253-1-xin.wang2@amd.com>
 <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
 <e724dd97-03fc-4136-9a53-94bbddce160f@amd.com>
 <e81356e7-fcf2-4fdb-8585-e544ac4a15df@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <e81356e7-fcf2-4fdb-8585-e544ac4a15df@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF00017098:EE_|MW4PR12MB8611:EE_
X-MS-Office365-Filtering-Correlation-Id: 6f294874-f4b8-4ae6-44e7-08dc796ba098
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?MHNQOFd3OCtuWW4ydXJkN1ZIUzRWYlRoSU1ickF6cEIzbDlKZldlT0lydnVu?=
 =?utf-8?B?ZWFrZ1MvNFdPc0w3d0VaM3ZqVFFxUlZiUnZnLzQ3V1ZySGorakVvSTRVekNh?=
 =?utf-8?B?V3EwMmhxaitzNUo3Y0ZCZmdoUWRZU1ZmNTBmbUQ1dU9yVm1VamxmRGZZWU9E?=
 =?utf-8?B?NURnS0pQYzhhMXRnNnNIaEJPRTRydTVaNHJyNmpTRldvVzJ6S1dWNGRPcW5a?=
 =?utf-8?B?OHZwbitCNGFHeG9nc1MvclhSTTBtWXhxT2Z0d1daUzNtdnFrekthQzRlZUpv?=
 =?utf-8?B?Uno2VG5HcEJObUxHK2NZSGd4dXExS0ZTMG8vTTZmWUlTb2VOZFIrcS95MTZN?=
 =?utf-8?B?SW1URDBpOFBPZUNpZzRKVEtqS2p3a3AvSTcwdWxtejVUeXd0YTdWUFltNjFa?=
 =?utf-8?B?LzQ0V1VZWFdQWXF2bTFxajRpaVJOMjFpK0hUZG41bG9KR1lLMDBvUm1VQU1Y?=
 =?utf-8?B?eEJuQmZxaU1BQjhXS04rREQwbDZtY2ZpWHE5bWZNSnpLUlVrQUNOMGNObXYx?=
 =?utf-8?B?N2FqUU5TY3BkOUZwcnFsQU1seEFIdHdTRU9XZTRweEtWRS9ZMU9jSVZ5Zmty?=
 =?utf-8?B?RUkxUkJMZEJtQ0JrZTJWdFZEbWtTTDIrRmh1SnI0U3FrbWd3b3ZLQy9Tck1D?=
 =?utf-8?B?bWV5Q1dsSjJ2MkZrenhjNytHTGI0RjVnSzNTVE9tdmZkcXFVYThEUW1xS1FY?=
 =?utf-8?B?N2EwenljNGIzTFYyYjAwTVd1YU45emZReVo0R2hjTTF0emU1SFliYVJUVVVv?=
 =?utf-8?B?SGtwMnVpTHdESUgwRElBS1ZDZjRIVXBrYldYMHJ0bFM5TUNmTFB3a25mWVRI?=
 =?utf-8?B?Q2hIVVFmUTQwUDBURFlNU2QrclJNZkJ1RVNjYmZHOURwZzFuYnJoSFpBWTlu?=
 =?utf-8?B?WXBHam1VaElHMExpcGpKa0RxdUF5KzZQemJucTltS1VIdFNRNHQzSmhoUHdR?=
 =?utf-8?B?MmpaL1kyUGFwem1CMy80L0dzQU9MNWdmWStxd1pTakpUNEU0RG1ycmsxZlVT?=
 =?utf-8?B?UEdzS1oyK3BZbTdieGxoMjAvMDhhRy9Ra3dTbURsSnlJa2Uyc0ZNZlc1N3Vl?=
 =?utf-8?B?RlF1b1k3eklPejVBSmNJT3JhaW1GeDU3R1c0SU9WV3JxNUdSUy9MVk45Q3pF?=
 =?utf-8?B?NWh6bWpxM1lqWkhrTFpoWkJFaGxVNlNuQ2V4VlBQMGNJeWFDQVBpbnJ5c1dR?=
 =?utf-8?B?MUQ0ODkwMm1ZTHBkd01TUUtQd3BzNUIxVWxUMVF6VVNHV21peG9RZk04T0ta?=
 =?utf-8?B?ZlpzOGk3QUcyL1JveC9Rd0xOdVR5QkhIRHV0Z3FBVzl4dXZITkRja3FHZ3Fn?=
 =?utf-8?B?ZDRkb0pmOWdFTDErSEl1SFRReTdhVUN1YkNyMlNtUHpTYmtCSzEwUUc0MUM4?=
 =?utf-8?B?YktOZ01wcHBXYkdhVnBRclFTWVBQMnJLRkhNWlBHQzlHZU9YYWhJemFYckdw?=
 =?utf-8?B?OG9WRTYvdnZMTVYvWlU4aDBwZlVocVJwMytjeE80SVhUa1FKaFZpYlp0a3JR?=
 =?utf-8?B?VjEzZTZBTDllekZ6MTl1YmVhRGt3VDFWRExuVFNpMEtISlBFSXM0czNvajYv?=
 =?utf-8?B?aUh1aURlVkU2SHFjc3NQcm0yT0pDY2UyelhHVHZvR2RrQTlFWFZSaVI2WUVQ?=
 =?utf-8?B?eTU1YnJWWWFMVlJyNDFwN0hZL2tTaEwrWVpyYzYwSFJiRHFLb0JURktCZ3ND?=
 =?utf-8?B?bVZ3bkpYQ0RCVWUyU2Z1Tk1EeDRxV3VzQUxBTmhSU0IrSXpaSW9vUktrUWxC?=
 =?utf-8?B?Qm41bUJsUkpXTTBuSDlJNnNtenNWdEVDMTNYRnlFREFVN2pZOUlua2Z4aTk2?=
 =?utf-8?Q?oQLGlIOKNlw+3IV0jrOCF1/Y9gLRatVJF+PMA=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 07:57:14.1781
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6f294874-f4b8-4ae6-44e7-08dc796ba098
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF00017098.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB8611



On 21/05/2024 09:51, Henry Wang wrote:
> Hi Michal,
> 
> On 5/21/2024 3:47 PM, Michal Orzel wrote:
>> Hi Henry.
>>
>> On 3/21/2024 11:57 AM, Henry Wang wrote:
>>>> In the common sysctl command XEN_SYSCTL_physinfo, the value of
>>>> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
>>>> Currently on Arm this is a fixed value 1 (can be checked via xl info),
>>>> which is not correct. This is because during the Arm CPU online
>>>> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
>>>> cpu_core_mask for itself.
>>>>
>>>> cores_per_socket refers to the number of cores that belong to the same
>>>> socket (NUMA node). Currently Xen on Arm does not support physical
>>>> CPU hotplug and NUMA, also we assume there is no multithread. Therefore
>>>> cores_per_socket means all possible CPUs detected from the device
>>>> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
>>>> accordingly. Modify the in-code comment which seems to be outdated. Add
>>>> a warning to users if Xen is running on processors with multithread
>>>> support.
>>>>
>>>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
> 
> Thanks.
> 
>>>>    /* ID of the PCPU we're running on */
>>>>    DEFINE_PER_CPU(unsigned int, cpu_id);
>>>> -/* XXX these seem awfully x86ish... */
>>>> +/*
>>>> + * Although multithread is part of the Arm spec, there are not many
>>>> + * processors support multithread and current Xen on Arm assumes there
>> NIT: s/support/supporting
> 
> Sorry, it should have been spotted locally before sending. Anyway, I 
> will correct this in v4 with your Reviewed-by tag taken. Thanks for 
> pointing this out.
I don't think there is a need to resend a patch just for fixing this typo. It can be done on commit.

~Michal



From xen-devel-bounces@lists.xenproject.org Tue May 21 09:37:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 09:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726703.1131040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Lvg-00089G-UC; Tue, 21 May 2024 09:37:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726703.1131040; Tue, 21 May 2024 09:37:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Lvg-000899-RJ; Tue, 21 May 2024 09:37:08 +0000
Received: by outflank-mailman (input) for mailman id 726703;
 Tue, 21 May 2024 09:37:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Lvf-000893-V2
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 09:37:07 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id af7d74c7-1755-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 11:37:05 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so8748851a12.3
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 02:37:05 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b16669sm1578644466b.188.2024.05.21.02.37.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 02:37:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af7d74c7-1755-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716284225; x=1716889025; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DynbRU0A7GPBquUvob9jhkjzzQdPSGZLT+Ekxx6uq6E=;
        b=UiQZmuHjTmwgHE2FLdUukyQ2sszv7HYEHF8MXosPar/YtW1rRkUHjfvNMmJRoeRJti
         g5l7x+iUU9sLN874EvkaZNiGkGRKQFjfPaDfh/WmX7MuwAWUWY0DxSqjIMXQn+WArv8F
         sSAVSwhGrVhtzdRVFtCnpOgR4qtGNBHBFbmVD4OSmyEpRUVjxaH+kTOtpg/6YFQckhee
         TW/qR1HqxuyjFCNVHaxiM8YN5Da/v8jsD/80nXvzGqQiMthkpKmYI8A9Sd6QBl94cz5y
         XbZ1Fgfb0so15zCFumznmbWYXkYNLvluMWJjaghKEu6boiUjN9ZP0WU93Z1fu8fHB6nY
         eNQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716284225; x=1716889025;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DynbRU0A7GPBquUvob9jhkjzzQdPSGZLT+Ekxx6uq6E=;
        b=txRTn7u246m679UYQGiBY+STbj9uyWpZzhducmBDtxEqC1I/0BLKVievre7i8P/tgx
         J7P2+fslpO/9LwjkuOm2jZolo7XOdiHJyJyeFyzO5yVlpwo846jgdFLRHguQBtww+8jS
         cu2N2zvusKGWU5Xarth/k1JOf6e22mnd3CPvD5hrvrWCSla71G8tWj4pYNMeKU/K359j
         7BIPB8DVQOdn9sQZNOJHEH7HS/4AP3Vg9dLz4VsWoRn+IHI2Cgc4ASfupQpS7cul5Xdv
         4Z+7MURMOxtvnvOwnCivbRN1uKj1AXvotDLgIOHNShsBOes0eoeaiWDFkZtKUF6Nf1XQ
         KVHw==
X-Forwarded-Encrypted: i=1; AJvYcCVU8F6VeGMdVNiQVi1JjO6ZpKvbRkTP+cVkb4XJrjthqpRDZi+5ztWiWUNYVzPDD3Kbva+fHDpwN7DKT7X2NJvJW3JiOqJn5EOWnTRP9IM=
X-Gm-Message-State: AOJu0YwGhSGJtAjdbLdADGs0IlqdTYR+i08ZQIy2oVMLrSgH+E8fmgmK
	UCC/nxyYwENlrMlkMsQlWrv0pt9xI/Xb0utQvSeGGNTpxIns4RDOqsJnZ1QhbQ==
X-Google-Smtp-Source: AGHT+IFHxY18K3TKJNVnsoZCtml+EAAUwmKWmK45uy6YaEpFg8+cPJpYF404pc4d8kDW2g6Xjol6kA==
X-Received: by 2002:a17:906:da8b:b0:a59:bdb7:73f8 with SMTP id a640c23a62f3a-a5a2d66a3b4mr2738871266b.47.1716284225136;
        Tue, 21 May 2024 02:37:05 -0700 (PDT)
Message-ID: <eef0f9f6-daee-4430-a73a-9b7e161e8731@suse.com>
Date: Tue, 21 May 2024 11:37:03 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240516132224.86005-1-roger.pau@citrix.com>
 <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com> <ZkYstvGkdffWMn2z@macbook>
 <b77790c6-a5b5-40d9-9387-5101ace14a4c@suse.com> <ZkYzGZ3tDg9dRbnF@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkYzGZ3tDg9dRbnF@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 18:23, Roger Pau Monné wrote:
> On Thu, May 16, 2024 at 06:04:22PM +0200, Jan Beulich wrote:
>> On 16.05.2024 17:56, Roger Pau Monné wrote:
>>> On Thu, May 16, 2024 at 05:00:54PM +0200, Jan Beulich wrote:
>>>> On 16.05.2024 15:22, Roger Pau Monne wrote:
>>>>> @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>>>>>                  release_old_vec(desc);
>>>>>          }
>>>>>  
>>>>> -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
>>>>> +        /*
>>>>> +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
>>>>> +         * that's all covered by the requested affinity mask.
>>>>> +         */
>>>>> +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
>>>>> +        if ( !desc->action || cpumask_subset(affinity, mask) )
>>>>>          {
>>>>>              spin_unlock(&desc->lock);
>>>>>              continue;
>>>> [...]
>>>> In
>>>> which case cpumask_subset() is going to always return true with your
>>>> change in place, if I'm not mistaken. That seems to make your change
>>>> questionable. Yet with that I guess I'm overlooking something.)
>>>
>>> I might we wrong, but I think you are missing that the to be offlined
>>> CPU has been removed from cpu_online_map by the time it gets passed
>>> to fixup_irqs().
>>
>> Just on this part (I'll need to take more time to reply to other parts):
>> No, I've specifically paid attention to that fact. Yet for this particular
>> observation of mine is doesn't matter. If mask == &cpu_online_map, then
>> no matter what is in cpu_online_map
>>
>>         cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
>>
>> will mask things down to a subset of cpu_online_map, and hence
>>
>>         if ( !desc->action || cpumask_subset(affinity, mask) )
>>
>> (effectively being
>>
>>         if ( !desc->action || cpumask_subset(affinity, &cpu_online_map) )
>>
>> ) is nothing else than
>>
>>         if ( !desc->action || true )
>>
>> . Yet that doesn't feel quite right.
> 
> Oh, I get it now.  Ideally we would use cpu_online_map with the to be
> removed CPU set, but that's complicated in this context.
> 
> For the purposes here we might as well avoid the AND of
> ->arch.cpu_mask with cpu_online_map and just check:
> 
> if ( !desc->action || cpumask_subset(desc->arch.cpu_mask, mask) )

Right, just that I wouldn't say "as well" - we simply may not mask with
cpu_online_map, for the reason stated in the earlier reply.

However, I remain unconvinced that we can outright drop the check of
->affinity. While I doubt cpumask_subset() was correct before, if there's
no intersection with cpu_online_map we still need to update ->affinity
too, to avoid it becoming an "impossible" setting. So I continue to think
that the logic as we have it right now may need splitting into two parts,
one dealing with IRQ movement and the other with ->affinity.

> As even if ->arch.cpu_mask has non-online CPUs set aside from the to
> be offlined CPU, it would just mean that we might be shuffling more
> than strictly necessary.

Limiting the overall benefit of your change, but yes.

>  Note this will only be an issue with cluster
> mode, physical mode must always have a single online CPU set in
> ->arch.cpu_mask.

Sure.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 09:46:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 09:46:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726710.1131049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9M4u-0001Q8-PM; Tue, 21 May 2024 09:46:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726710.1131049; Tue, 21 May 2024 09:46:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9M4u-0001Q1-Mn; Tue, 21 May 2024 09:46:40 +0000
Received: by outflank-mailman (input) for mailman id 726710;
 Tue, 21 May 2024 09:46:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9M4t-0001PK-EI
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 09:46:39 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 05080783-1757-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 11:46:38 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-572a93890d1so9354719a12.3
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 02:46:38 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5ce2eabad1sm704481566b.202.2024.05.21.02.46.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 02:46:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05080783-1757-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716284798; x=1716889598; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=R96BCP96vO92VytP1iz3Eh8vWoYjjxBtd2vJWaEMbis=;
        b=HHhPr+gWcRtypbzsqWtp4nYU14mGqNJ59BkyoiUC0it3yzqLFpwp7ThcZSk9EucFM8
         s6sQ7t9+ap6A9Rm6en3whsDWhhebFm/4MMV5C4ja1Phqnq4VrPG8vD8G2T2uWC0cMTrC
         zGiMQIGsveRDWKdW4eFmoQYAGfvz2lFE5N1Sx5u2UV27EpGmDlWiPeQtadKpRRVniZAp
         l/c74yKSmXtbmhDuklIFirxv6VPjfThaXynfu9AoNTqtQvdZGCaCldp/eehAx4yVYHsI
         bLDEL5f2c9Q36C585ozGviEyCj9zgi6h4qiqe7D361EJudNgPEJzYHQqz9F+pQvEemVy
         AkJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716284798; x=1716889598;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=R96BCP96vO92VytP1iz3Eh8vWoYjjxBtd2vJWaEMbis=;
        b=UQ5qWa9kt3i4uBGq26Of3g8dHuwU+Mt2mTNMZkSXsasaOBvl4mUhGK9+YYJ/oVTcoe
         yswrQYtdriBVXOOeOBHhHAjPi1qVGitngUJ1ZSxK1C4pX9TipP5YemXWrSKbyjdM3TNR
         LanIZpfOEUCzrDnesUOp1lH5sRI1yg34da+W/qONsSJeip7wQq2r4tRNGmutpTdbtES7
         cbYYZxIes1WP4h/j1abZl1q/yMU3dwNVjsiWc99nbAC2T50euibAZAQz8Pt6xv5PHEsz
         XajX9lTGKUFGUuqJdi3xlWXmJb4JwBz+DmLUsZRE8vePWvGDZM2SyR13FrrfI0HrLLMw
         Uz3g==
X-Forwarded-Encrypted: i=1; AJvYcCUJzBCNZaJXaEqwM/NgJV/tppzABrhtfDmy+d9Njc/YpStym2wLOPzDuxJP1he3rNkeMPffAqDNnFuLoLQ40H705FHkOW8oeTAo09ZZWcI=
X-Gm-Message-State: AOJu0Yw26lQKgPxW4qsXLolnZqJrLYRVNzCtoCG1FH25o6D+2IevL4+/
	xsO3aUNEWrbwp8ZSdSrdiKjxTrrNudWMyf5JIC6szoQllCG/hd2BcFYAEBNGbA==
X-Google-Smtp-Source: AGHT+IGJB5qyQ7Mihd9HkcvgJJztPSgMI2ahnLqs+4l9UlSRyQVcdAe5qF2uyEJvdmw2GXRPVpt4xQ==
X-Received: by 2002:a17:907:747:b0:a59:a3ef:21f5 with SMTP id a640c23a62f3a-a5a2d65ed16mr3935973566b.57.1716284798003;
        Tue, 21 May 2024 02:46:38 -0700 (PDT)
Message-ID: <43000c85-169e-4a39-8559-de2eada6bcbb@suse.com>
Date: Tue, 21 May 2024 11:46:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19] xen/x86: limit interrupt movement done by
 fixup_irqs()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <20240516132224.86005-1-roger.pau@citrix.com>
 <449c7562-18a5-4f5b-a7a6-36fbf0383517@suse.com> <ZkYstvGkdffWMn2z@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkYstvGkdffWMn2z@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.05.2024 17:56, Roger Pau Monné wrote:
> On Thu, May 16, 2024 at 05:00:54PM +0200, Jan Beulich wrote:
>> On 16.05.2024 15:22, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/irq.c
>>> +++ b/xen/arch/x86/irq.c
>>> @@ -2527,7 +2527,7 @@ static int __init cf_check setup_dump_irqs(void)
>>>  }
>>>  __initcall(setup_dump_irqs);
>>>  
>>> -/* Reset irq affinities to match the given CPU mask. */
>>> +/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
>>>  void fixup_irqs(const cpumask_t *mask, bool verbose)
>>>  {
>>
>> Evacuating is one purpose. Updating affinity, if need be, is another. I've
>> been wondering more than once though whether it is actually correct /
>> necessary for ->affinity to be updated by the function. As it stands you
>> don't remove the respective code, though.
> 
> Yeah, I didn't want to get into updating ->affinity in this patch, so
> decided to leave that as-is.
> 
> Note however that if we shuffle the interrupt around we should update
> ->affinity, so that the new destination is part of ->affinity?

I would put it differently: If we shuffle the IRQ around, we want to
respect ->affinity if at all possible. Only if that's impossible (all CPUs
in ->affinity offline) we may need to update ->affinity as well. Issue is
that ...

> Otherwise we could end up with the interrupt assigned to CPU(s) that
> are not part of the ->affinity mask.  Maybe that's OK, TBH I'm not
> sure I understand the purpose of the ->affinity mask, hence why I've
> decided to leave it alone in this patch.

..., as you say, it's not entirely clear what ->affinity's purpose is, and
hence whether it might be okay(ish) to leave it without any intersection
with online CPUs. If we were to permit that, I'm relatively sure though
that then other code may need updating (it'll at least need auditing).

>>> @@ -2576,7 +2576,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>>>                  release_old_vec(desc);
>>>          }
>>>  
>>> -        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
>>> +        /*
>>> +         * Avoid shuffling the interrupt around if it's assigned to a CPU set
>>> +         * that's all covered by the requested affinity mask.
>>> +         */
>>> +        cpumask_and(affinity, desc->arch.cpu_mask, &cpu_online_map);
>>> +        if ( !desc->action || cpumask_subset(affinity, mask) )
>>>          {
>>>              spin_unlock(&desc->lock);
>>>              continue;
>>
>> First my understanding of how the two CPU sets are used: ->affinity is
>> merely a representation of where the IRQ is permitted to be handled.
>> ->arch.cpu_mask describes all CPUs where the assigned vector is valid
>> (and would thus need cleaning up when a new vector is assigned). Neither
>> of the two needs to be a strict subset of the other.
> 
> Oh, so it's allowed to have the interrupt target a CPU
> (->arch.cpu_mask) that's not set in the affinity mask?

To be honest I'm not quite sure whether it's "allowed" or merely "happens
to".

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 09:58:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 09:58:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726719.1131064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MGT-0003Py-Sc; Tue, 21 May 2024 09:58:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726719.1131064; Tue, 21 May 2024 09:58:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MGT-0003Pr-Q1; Tue, 21 May 2024 09:58:37 +0000
Received: by outflank-mailman (input) for mailman id 726719;
 Tue, 21 May 2024 09:58:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9MGT-0003Ph-0y; Tue, 21 May 2024 09:58:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9MGS-0000ye-UK; Tue, 21 May 2024 09:58:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9MGS-0001OP-KB; Tue, 21 May 2024 09:58:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9MGS-0000hH-Jf; Tue, 21 May 2024 09:58:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PmMFlSgdavixawH749lfSCuxFNYk0szsHcbYdRUcF3c=; b=O47bg4vcsEx9JKqyoQjsVCLTYM
	6BdX1n7Ao22KMf+kNoLiVvzdXjdOe3FAo1pmqTEP9UW86X8Sgt2KFEBRQKLh0LpKgZva2nKRW3uIv
	EuwwPg4UfGlfWQbXCqkFiRfn1oIchEUEPj5WexgLfLqfxvHPnOZkydAtzcWMgYkT1UJo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186056-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186056: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9c5444b01ad51369bc09197a442a93d87b4b76f2
X-Osstest-Versions-That:
    xen=26b122e3bf8f3921d87312fbf5e7e13872ae92b0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 09:58:36 +0000

flight 186056 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186056/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9c5444b01ad51369bc09197a442a93d87b4b76f2
baseline version:
 xen                  26b122e3bf8f3921d87312fbf5e7e13872ae92b0

Last test of basis   186048  2024-05-20 18:02:09 Z    0 days
Testing same since   186050  2024-05-20 22:02:07 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   26b122e3bf..9c5444b01a  9c5444b01ad51369bc09197a442a93d87b4b76f2 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 21 10:00:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 10:00:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726729.1131075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MIV-0005Bd-Bq; Tue, 21 May 2024 10:00:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726729.1131075; Tue, 21 May 2024 10:00:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MIV-0005BW-8z; Tue, 21 May 2024 10:00:43 +0000
Received: by outflank-mailman (input) for mailman id 726729;
 Tue, 21 May 2024 10:00:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fgXu=MY=epam.com=prvs=1871a78dee=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1s9MIT-0005BQ-JB
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:00:41 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fa1e2b6b-1758-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 12:00:40 +0200 (CEST)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44L9WXU8008463;
 Tue, 21 May 2024 10:00:30 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3y8s2u83jj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 May 2024 10:00:29 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by DBBPR03MB10534.eurprd03.prod.outlook.com (2603:10a6:10:53d::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17; Tue, 21 May
 2024 10:00:23 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Tue, 21 May 2024
 10:00:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa1e2b6b-1758-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=calF6uL2o/WyOWr66RKhGSqSkqtHPpWvMJDqK42gsqGgw2QCYLNAJ8K+2b+nCSNKUZk01GOGRVRXCcSzgwN8JZwUSeTG8lNXnukPZQRKLeok/QFk+xNSSglpp6jCBUwgOrakWwqmLcueuSDF6pQ065orhEmgpPceS6gjaQ5iIP70wWDRiuCS7Xa708qICKQ5x4BoLPfg3ICjDRsk+6uJUUuC4rvDMjBoSqFMxMl5WJ81URng8yHW7iX/SHNZmJhrkHLYBJ6tTVLgLm9Y1eA0I5w7MjTzS/gospyiOltVrNe1lWJf8IN67kmTdWW0TcLR8adOnwNVQQFU6P/q4d+hYA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=doe6Hhp9+/08HcLKIHTqxCv1sK8tl+uf27JKsHXvQ14=;
 b=IujxGpXBWEJnuUgbFyeJjLo50sxTTqUg8MsjYANO3BEctK8vrJwnJ5CmAYNbMqe+xV8DRjz+NanyZtQ3K7O1tKVpsouzpD2NZiqO57NyFg0V9XUDvFNpb9TveaO9VQdIJeeCCYXw+gVSBfo4ubPaK8SXTURb/mw8pcRAR9C08gDAF8cqekBQa+qioV78uOALYOO8VFQ8LXUGBP1DgDetaGdtXxLnFPMHMy6BHcLE+ZtoozRH9AfL9pjUZ78RT7rrQXN2ltIczQpHiYU5L19dS3DoVlm8czZXTFnO9EBDzgIa3nMOnDbx/vXXhA2XyzB2NVMt9Oxj6iYYin0x3/iTag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=doe6Hhp9+/08HcLKIHTqxCv1sK8tl+uf27JKsHXvQ14=;
 b=uLxD/eGMJnzLypZiw0wII7SjtV3Yixio0rXdHUbH2Hfkvko0LcBcI2bKc8g82Dfi3DuqRtflAVoCOxUMzalS6bsAapkwwVxE6fxmGpsINdLFxCXzirqA+zfR84ElON+s7//GslfcC92UvR1LjAQNx4yff8/O+A/3gvQbKNkU0YkjQDiEv9ZRkTglCQ6u1VUydpY1M4KQk9Ab4tPtxbhcQHQgt2AQYMV+vF2ffbwR76XxFpjPJqIE8KV7t1jHdhaEFJNHdYGh7bS9fq2ThvSvwiZSeyg5zaSDxYEZpyPbgaWS212glm69ZMcHwN75fJSXfWOhVn3s1JzAjgd24zY9kg==
Message-ID: <19b1896d-459f-4963-9199-50a87d910605@epam.com>
Date: Tue, 21 May 2024 13:00:21 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to
 header
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
 <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
 <68e2b3b5-649f-4e53-8700-350bdb8e5b09@epam.com>
 <276192d0-9119-467c-ba8d-b4d808863364@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <276192d0-9119-467c-ba8d-b4d808863364@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0006.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::14) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|DBBPR03MB10534:EE_
X-MS-Office365-Filtering-Correlation-Id: 597de5e9-2335-4224-93ee-08dc797cd4cf
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|376005|366007;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?NTA2TExHa2tCUy8vMWNYQ0J4NEpkWVJLQTVSbjJ5bGlLNWVqenpZY0lubGZU?=
 =?utf-8?B?b0txQmdvZVc4aWFhMmFOL3JHR1ZBOVUwRUVXMllXRTFzWHp3WkM3Uk5kMjJX?=
 =?utf-8?B?Myt2T1VOK2ZhL0dUNWFEckNDbFhPeTU0OFJ0QmlONEJ3STZ0ZDMrSUxpODBo?=
 =?utf-8?B?UE1WWXpxTjNlc0ZkaGJscVE3MTBKdlZyeWpub2hpM25vZzB0azFJNEJ3TlFh?=
 =?utf-8?B?S1BLTWZTWkRPOXJsUHBlY2QvYzlTdyt2VklzTlBqSUhjRjVlSkRwTlZkN0pE?=
 =?utf-8?B?UTFDQzFtU3VsekxMcmhPby93NTEvd1l0aEZuU1I0cko4L0RIUW9BU1Y2dXF1?=
 =?utf-8?B?QytUS1VRdGo5ZXcyQVB1VlN6NFZobHhLbE53eHBkdkhSUjRuV3BIZElnbFNk?=
 =?utf-8?B?aDdVK2ExTGVGMEpNZDBjYjZIcU9RbHE5bXlUT01hZWdUaDMrV2VnQ3g3UFha?=
 =?utf-8?B?Ukx4NmdiR0VYTENCNVA1Tk5IdkJVb2xCZGkwc3Jjdlh5SHd3Mk9SWWtjTHg4?=
 =?utf-8?B?V2MweFVqQTl5MHU3UUtmTjJpMjZCa1lxUDVDbTZBNGhtMC80RUkza3ZNaEpw?=
 =?utf-8?B?N2JZdkh3TUw5TFlUNHJGdTQ5TEQwekVWR3lpbTdnS1JjWnFJUGRNMXJhTlFU?=
 =?utf-8?B?WTZMVDhVTUpzYjZORUk2WGFyUjdkZHVGMThzQUVQMkx3ZUVnWWdsSXlsbUVN?=
 =?utf-8?B?M0hLd0JaeEVKbkdVUEZtT2piTnQ0ZXM2dVMzQWQvellMZFg5V1V4c0NRRXhj?=
 =?utf-8?B?VDdEalJicktRZlRSdjhSMzBRbXhQQS9VR1NvUmozR1JZQmszVzNNSDNGVTZk?=
 =?utf-8?B?a2ZBQVdHREJZd0F6S3M2QXdZVjFyRGJTQnZNdHRnMGxkR0hQSkRZSUVLWnBn?=
 =?utf-8?B?QmhBaDlsYldORit2SVRMdzJKb2NFTG50WWRpZUh2QmduRjhuTThFTmo0NUpu?=
 =?utf-8?B?V3UvNjFvMTFOYWhPYktOSkZTZEpGL2tCdGtGL3pKRjZxSmNLYlZWaEJaTkFE?=
 =?utf-8?B?TFVEeVhyMHdFUTZyN0ZPSXl5RS8zQzZ3azJLVUlQekl0Z1VPN29BS2Vwa2Yx?=
 =?utf-8?B?SGxWVHdaMnd5WEdKamtlaVM5SWhHR1Bwa1ZmZDJJN3pGT3VrRVA1M0tnbldh?=
 =?utf-8?B?ajZBdzVTUVNjT3NSWmdKY1h0V1g1d1Z3T2lSUmt0dFV6LzZqWE5qU21kR1h2?=
 =?utf-8?B?V0xyUHZ5V1dLREcyakRNWHNSaGVrdkI3K2YwMlZ0bXBzdS91NWl5cjdqQllI?=
 =?utf-8?B?akJ6bk5ScjRqWkVNa0hHMDE0TTgyOGVqZjNPcFlLZWhmdVR6UC9Tc2JvemVS?=
 =?utf-8?B?Sy9yRFRJeHBDOG9SZEFOVm12SG1BVzJ4VU9NMVVnWUpNTG1zajc1RnBmZzBp?=
 =?utf-8?B?RmFWcWRCYkhGZ3dGQ0dIczk3Q3pjc0pRam9VNVh0OHBlK0psTjBUWmZ1U0Zw?=
 =?utf-8?B?S2hkUVRZOGVyblM5RXJ4RWRBUXFsdUUwYXpabGszUHpoM3RHTCtFbEE5WnRG?=
 =?utf-8?B?VzJNNERUOG1uK1Bja3ZrZmtFbkRFRlp1OUpSRzFMYVI1RWVXTld3U3VYK2Rz?=
 =?utf-8?B?YzYya0pwUFgyWjE2bm5qTjJQTE9mQUtVa3NtbUZJRTVZWUVuNC9kZmJTWjdN?=
 =?utf-8?B?NVVOcURsbzcxMi8xbUZMK0RrcThCTTQxQmQvTng1R1ZOdmkwSlBlN2RtcmhF?=
 =?utf-8?B?NWlrVXMwVnZSQmhrYVVGakUxMkJpZ2VwMWNidlpmQVpWNFd1N1dXaVdnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?VDdBWjl1c1BvbFlvSWNaR0lsRGp5MWVwRVQrdDNkQ1YrVkEzRmErd1FpY3d3?=
 =?utf-8?B?ZUZEWVozWWxyeWhpSTVEb0cxMmNGcXFuR3ZYdjBFVU1lZEtiR3VSL2lqZUpr?=
 =?utf-8?B?YVR3MkRnK0s4ZTM0cVRPMDk4SmZxMytockRBMzh0RlhWK3FYL281YktnZHYx?=
 =?utf-8?B?MEdzS1ZMSkx3b0NBcXlBcGYrZnN0UGFhMTBIMFNuZlUyL3l2TWdncUtjRFlw?=
 =?utf-8?B?Q0hKTzEzVEtpQVNSbnVHZ0FmdHhNd2tqL0ZHcjMvUkVQMWhXelpvRTB1bXdK?=
 =?utf-8?B?RzB0YTFXVFJkeUpoSGlRT2hlSDd0dCtKQmxSUjU2WFV5KzVpQzYzK0pWY3NT?=
 =?utf-8?B?VzZYWUszOWJnOEFVQUhqdkUvajVzRm9kMnlKZW1EQzRrRmZkeFpHMUo0QlhU?=
 =?utf-8?B?K1pQK2hCUlJ1dC9XdXRLZWR3d0xoVVhvaWhaT1hMY213cUZkWis5VHQwbjZE?=
 =?utf-8?B?MjB5c1dud1VSWFFsYlErZkFFaVVMSHJJMDJRcU1kaG13NllEa1pZRXNsQVhx?=
 =?utf-8?B?aXN2OXFRQmxCZTl5bHVuUGYxbkw5djkzNDR2SUtLeUg1NXE2MllWV2ZoY0xw?=
 =?utf-8?B?cUJua0xZNjVNaU4zcnNHS3JTT0xQbHhnSDFNK0Zpb2l5SCtERUlFUFEzc3Za?=
 =?utf-8?B?Tkl3NzU4RHl5TU0xWTQrT25Vai9rS3VvcW5NNnllUG51Y2ZqWndleVFTQ2Jr?=
 =?utf-8?B?b1lUQzI1NmQzNzJCRThlbkRFVVhpWEJlNTR1U3dIZUg2Qjh2N1lZUFpDNCtj?=
 =?utf-8?B?T3lkV2JQMHV4bnZEMXdxRUIycjJEVTU5ZkhmWFRTcHhJVDhJMnR3TWwwbHdG?=
 =?utf-8?B?RVAzYVVUODMrWHlTUnd3UnNibnNEYTFzWWVlWktzR2tPUFg1eXN5YkFrWVY3?=
 =?utf-8?B?cmpuVGMxaHRpTVZkSUMxa2dkbzhGRnFlSjk5MmovZTJ0TS93b2JURndwdHk5?=
 =?utf-8?B?YW9iaDZrVHRqWEFyV0p2bm13bmtvQ0Yyc2pKOFhGcGs5cVd1SnJqano1MWcx?=
 =?utf-8?B?U2VnMzNZNkJVek9xcnlrT2tUNW5INklJVlZ6YStOUmxlUVVqUXB2cW12YVB4?=
 =?utf-8?B?RVFnUU8rMDV3dDZXdWUzS1hJZjNPeUhveUtoSGdBYkNZN2tmYU12bjUrME96?=
 =?utf-8?B?Ti9BdDgrMm9IcjBnbjlMeWpxUnpMbGZLU1dBYVVJaE4zaVBhUkVjeVZIdFFG?=
 =?utf-8?B?Q2Q3WjRFOHVnQk5XelV2VkZ2bzBwckdaM3JNMjFyUWpjNkdBV2VZL0EycTVP?=
 =?utf-8?B?dGJqLzdWSUJVVXY1YmlDSUQ5NmhaYzlGOVBPMm51ZE5VMkY1WVpqbExQSWs0?=
 =?utf-8?B?S0RhV1gxbXdOS0tMaGVuRGRuZ0M1TXpNRjlONXJsUS8xYVJ2eHRwVnFoTlZI?=
 =?utf-8?B?NVhrT2syS3dxeTlDRmV0dFlkTWhPM3NtYytTdGlKanpnODBMbm9vN05IZkxw?=
 =?utf-8?B?L0JNcytkNXZ2MWdwbmZTUHozT1B4bkVCWVRXbUN4L3IyR29HN1U0eGJEdnJ5?=
 =?utf-8?B?UG9lOW5LejhqTGFkb2tvcGZ0MGV4VWlFVWRWeVlOSnNnck9WbWR5NkRnQ0Zs?=
 =?utf-8?B?OUMvZ3dqNmZkaUlvaDRsbEp6TmN6MVMyNW9aZkg1dUQ3OE9NdnZmV3RZWlpm?=
 =?utf-8?B?UXVOczZwWm5CS1dmVHFVUVlpem5rSnREc3hiZE83eHhxeDlvYjFVQk1Yd0d2?=
 =?utf-8?B?TFRzRndiQ05zSGhDalFkWGxyTXJGdTMrY2JsUXNHeUxpdy9YYzdWUTd2MzFB?=
 =?utf-8?B?VE1jM0JQcW1YRkdvL3E0QllRRm41RzVLN0RUaGdPa2tQdmxKSFBSaVk1M2dH?=
 =?utf-8?B?Ulgvd3o5bGxCMXZPTmE5c0hRYkNoWkgyRU5KQkkwUEdsUVBlSnZaVGNpV0px?=
 =?utf-8?B?M1VPVVFXU1NkN1JuclpadGFjMHEveGdqMEg5NEk1eDdRU21PUDdrRkR2R2dL?=
 =?utf-8?B?WjNnOGoxUzhMRjhFODFhc1hPczdZYURUTFVXSDNwNnVKendFMVVIT0V0ZWJY?=
 =?utf-8?B?S3FXU1B3MTRVU2NDRlh5bWU3MUNnMDZybER2a1QxbXloL2JDVXJCVitnN05D?=
 =?utf-8?B?c3M1TGV3V1RaL0w3V2l5MkV1RFNXaWdFSkRuRDQ3YVRqcThxUkdHQ2FVVTV2?=
 =?utf-8?B?TW1FSHVLSGxFTG5meUpqNkFLZExkeVo5WTkrNHFWVzFWbjI5NzVadzN6QXlD?=
 =?utf-8?B?SXc9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 597de5e9-2335-4224-93ee-08dc797cd4cf
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 10:00:23.4268
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 61ZrRX7VfWVmNu/YS8mrWpwPHj/n5/5MePkx7qtL/2n2IXgErsY8OYVWrnbiqfFSb+8A14Z00XOol+dIQVKKyg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR03MB10534
X-Proofpoint-GUID: lXTMMjOF-VN0LolND_boSOiP7rZqz7UO
X-Proofpoint-ORIG-GUID: lXTMMjOF-VN0LolND_boSOiP7rZqz7UO
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-21_06,2024-05-21_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 mlxlogscore=768 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0
 impostorscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0
 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.19.0-2405010000 definitions=main-2405210074

21.05.24 09:05, Jan Beulich:
>> This is because MCG_LMCE_P is defined in arch/x86/cpu/mcheck/x86_mca.h
>> -- so either MCG_LMCE_P (+ a bunch of MCG_* declarations) has to be
>> moved to common header to be accessible, or local x86_mca.h got to be
>> included from common arch/x86/include/asm/mce.h.
>>
>> As for the MCG_* declarations movement I didn't think there's a good
>> enough reason to do it; as for the inclusion of x86_mca.h it didn't look
>> nice at all.
> I'm afraid I don't follow the latter: Why's including x86_mca.h any worse
> than what you do right now?

To include x86_mca.h from asm/mce.h something like this line would be 
needed:

#include "../../cpu/mcheck/x86_mca.h"

I've found only two include-s of such kind, so I presume they're not common.
Besides xen/sched.h includes asm/mce.h before declaration of struct 
vcpu, so body of vmce_has_lmce(const struct vcpu *v) can't really be 
compiled in asm/mce.h

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Tue May 21 10:03:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 10:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726735.1131085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MKy-0005nC-Pm; Tue, 21 May 2024 10:03:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726735.1131085; Tue, 21 May 2024 10:03:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MKy-0005n5-Lg; Tue, 21 May 2024 10:03:16 +0000
Received: by outflank-mailman (input) for mailman id 726735;
 Tue, 21 May 2024 10:03:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=R9BQ=MY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9MKx-0005mz-56
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:03:15 +0000
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com
 [2607:f8b0:4864:20::f35])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 55416051-1759-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 12:03:12 +0200 (CEST)
Received: by mail-qv1-xf35.google.com with SMTP id
 6a1803df08f44-69b5ece41dfso7730656d6.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 03:03:12 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f194da1sm120076446d6.67.2024.05.21.03.03.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 May 2024 03:03:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55416051-1759-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716285792; x=1716890592; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=JCodHXALEwX3rcjPsEqWu+TfABQYk2zNMzXaKsxlTiI=;
        b=HomM6hWVAeWVoHEYtdiTFSDFblZm3KZ332JBGVY6JivkFQo7CE5B0hZUHgTii+r9cN
         YrMvhRAf9wHvN48sIozVXOu1nIF5KLYTH4g2j2lPeyAwmnK2Op2FyGP1di9nxInx9rQ5
         wV5DsVKpP6Q2YWJS6U3Y7GHomE+htpAl5Xxtg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716285792; x=1716890592;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JCodHXALEwX3rcjPsEqWu+TfABQYk2zNMzXaKsxlTiI=;
        b=RLhwWpN6/yx+mDWPPmLGklIAWoCDXyGMJnCpW3QCjDclGtw49jnzjQJ8gmafLjtkUv
         zz0FJlzDIMb/PbLvokRXU2iqgPjiPVIwjINRESOWq3D3YGFJA4RCoGrgIDFgRtHk5mjC
         E77Je5VCWgq6MtlkdLkANaFMmelbET8t4WjOPaZrkwoiSVUZutL4DilQZcoKBIthVwEG
         qzlPH6t1FWekjJomI+yVL9ovVm1Fw/HUO7Sjj9Po9APlCnHbRUDFdf6m7MQ+gG4Z2Mkq
         dHRlzppeN5mbGlAMy0y7/tAnX3MBDfvEhz2mIgpDiRkcB6YlbbYLxLsIYXrvDGWfeylN
         3kgg==
X-Gm-Message-State: AOJu0Yw7yRg+g+5AAJInlh8+ttNfRUqpo+jwGM9Eb/zHSmcakBDu4OmX
	EgwsjzwnomuYo2VR1vJcaKC0L7ViwhRlOx29Z95VuUu8zT05C+FVD+Szg/zth557dOpvguJLwzS
	5
X-Google-Smtp-Source: AGHT+IHyAs9wkukllqxUeu+6zXvC9UjriusAK0/vPemcSXU+ppSaw+I3IphkeL+VRyDkCvvnwT/TlA==
X-Received: by 2002:a05:6214:320a:b0:6a9:2fc9:f539 with SMTP id 6a1803df08f44-6a92fc9f733mr76254016d6.9.1716285791529;
        Tue, 21 May 2024 03:03:11 -0700 (PDT)
Date: Tue, 21 May 2024 12:03:08 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Message-ID: <ZkxxXNbOJ48Bl_Sk@macbook>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
 <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com>
 <Zjjg2ueqgjmn-MS3@macbook>
 <44af358a-9510-4056-826c-3be99dc25830@suse.com>
 <ZksmJp5JnQoBYZ6U@macbook>
 <d1267c40-e2d6-4919-81ac-54ad217e4702@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d1267c40-e2d6-4919-81ac-54ad217e4702@suse.com>

On Tue, May 21, 2024 at 08:21:35AM +0200, Jan Beulich wrote:
> On 20.05.2024 12:29, Roger Pau Monné wrote:
> > On Wed, May 15, 2024 at 12:07:50PM +0200, Jan Beulich wrote:
> >> On 06.05.2024 15:53, Roger Pau Monné wrote:
> >>> On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
> >>>> On 06.05.2024 14:42, Roger Pau Monné wrote:
> >>>>> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
> >>>>>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
> >>>>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
> >>>>>>  
> >>>>>>          if ( use_ats(pdev, iommu, ivrs_dev) )
> >>>>>> -            dte->i = ats_enabled;
> >>>>>> +            dte->i = true;
> >>>>>
> >>>>> Might be easier to just use:
> >>>>>
> >>>>> dte->i = use_ats(pdev, iommu, ivrs_dev);
> >>>>
> >>>> I'm hesitant here, as in principle we might be overwriting a "true" by
> >>>> "false" then.
> >>>
> >>> Hm, but that would be fine, what's the point in enabling the IOMMU to
> >>> reply to ATS requests if ATS is not enabled on the device?
> >>>
> >>> IOW: overwriting a "true" with a "false" seem like the correct
> >>> behavior if it's based on the output of use_ats().
> >>
> >> I don't think so, unless there were flow guarantees excluding the possibility
> >> of taking this path twice without intermediately disabling the device again.
> >> Down from here the enabling of ATS is gated on use_ats(). Hence if, in an
> >> earlier invocation, we enabled ATS (and set dte->i), we wouldn't turn off ATS
> >> below (there's only code to turn it on), yet with what you suggest we'd clear
> >> dte->i.
> > 
> > Please bear with me, I think I'm confused, why would use_ats(), and if
> > that's the case, don't we want to update dte->i so that it matches the
> > ATS state?
> 
> I'm afraid I can't parse this. Maybe a result of incomplete editing? The
> topic is complex enough that I don't want to even try to guess what you
> may have meant to ask ...

Oh, indeed, sorry, the full sentences should have been:

Please bear with me, I think I'm confused, why would use_ats() return
different values for the same device?

And if that's the case, don't we want to update dte->i so that it
matches the ATS state signaled by use_ats()?

> > Otherwise we would fail to disable IOMMU device address translation
> > support if ATS was disabled?
> 
> I think the answer here is "no", but with the above I'm not really sure
> here, either.

Given the current logic in use_ats() AFAICT the return value of that
function should not change for a given device?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 21 10:19:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 10:19:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726800.1131167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Maa-00009u-RS; Tue, 21 May 2024 10:19:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726800.1131167; Tue, 21 May 2024 10:19:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Maa-00009n-O9; Tue, 21 May 2024 10:19:24 +0000
Received: by outflank-mailman (input) for mailman id 726800;
 Tue, 21 May 2024 10:19:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9MaY-00009R-QH
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:19:22 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 965662cc-175b-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 12:19:20 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a620a28e95cso127818966b.3
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 03:19:20 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6214655b23sm62857266b.165.2024.05.21.03.19.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 03:19:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 965662cc-175b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716286760; x=1716891560; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EKVWzKv/ad9gW/9G47W+be3C1C1SddbZeJ0Z08V47f4=;
        b=Ecz0OKNkZHx4S4G/gs9IUDlfphciCw+WwI2JifODOubLxQ1tOFNk+6rlx+45dYfd7F
         Xt5/6cYzBNfZ/D7/tWM7vKt1Ai/htb9WDi6bkbx6bPi3nZJap6h6jrwIEqzPvCkHoQEE
         vjq7yqB7DwCicxJ+Lb1IeghtXkZ8tpuPabEYWuNNDwiPzuN/JM7Oh0fyiG7QCuq9YQAn
         JkueCs+fTutt+9/wQmJWV0LmGq382bDOwPBVTitZfP7ZLk7MeSGB+MqYcKjMkhCjjpcz
         UD89WpV91ygZ/3s8nFhumwIQoHLD3+RLHQxVuRkaMJyfHA9ieMXvxYal3Lo6bNbRO7Gq
         WJGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716286760; x=1716891560;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=EKVWzKv/ad9gW/9G47W+be3C1C1SddbZeJ0Z08V47f4=;
        b=VPmXQFgpTXuen44GC1VnH5KaAPOHN08q85NUu8dMkQuMMx9275NUBKMSUKRTxXYVyr
         FuJNxTgb5Hh1P8LDLL2FXDjEkQPJb/RA6SyU6PlhdbSOSyFyRBeJccZ5lytTFzo+t56g
         Ak5BWZ3N6BV6cicHGxE59PRjQFvJRWQHwGQOGfID6lJsIq1CaiZEpZ/KJioDjqV4ycXk
         TSXFFpkiDUBu6OBFqKm6+H4xDoKdNu3t5qBjgv3aqhbs/cqWozN9Kdg1o2GVywPX6zS6
         QhA/8LO3Sto/lPUMiSBAaD/7IlFsh4+3yISBA3gvLvtFjcujtGNNSYiyRd+XdC2ryel5
         +QMA==
X-Forwarded-Encrypted: i=1; AJvYcCXjlil8ySMOKVe1Wlu31kxyrMn9wpcKDE4f/JPQIVsd9lPdbJj05kLgtrJEw3nDjVGsoulpBtHZRf5f59ahqXMuw+eKCaXnfd5wF4WRC8Q=
X-Gm-Message-State: AOJu0YxML6Wkq2t7sSdVKKKqBagIQWafwSKNEPagWgoQ7oD1z4zE3Fgr
	PT/ds7hpeb4yJf5fBPkzUDhPeSobv3x8x+RDdYCpnQPQpN6qSjVVZ+eHTnErKg==
X-Google-Smtp-Source: AGHT+IGT7FQBMIpsgPKSLJ+EftdaStyRm7Q7Hx0yM8s8RMvxmUykZmlg5kx3goqkF6y6NybAocPwcw==
X-Received: by 2002:a17:906:1797:b0:a5a:8d07:6a00 with SMTP id a640c23a62f3a-a5a8d076e2cmr1223586566b.64.1716286759998;
        Tue, 21 May 2024 03:19:19 -0700 (PDT)
Message-ID: <9cf8e78c-7ce8-4c65-b312-c89b1061a1a7@suse.com>
Date: Tue, 21 May 2024 12:19:19 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v3 2/6] x86/intel: move vmce_has_lmce() routine to
 header
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1715673586.git.Sergiy_Kibrik@epam.com>
 <77bc29d74cdc43539a060bca26495a4115171f6e.1715673586.git.Sergiy_Kibrik@epam.com>
 <a21691b7-224e-446b-a2db-761ffa48107e@suse.com>
 <68e2b3b5-649f-4e53-8700-350bdb8e5b09@epam.com>
 <276192d0-9119-467c-ba8d-b4d808863364@suse.com>
 <19b1896d-459f-4963-9199-50a87d910605@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <19b1896d-459f-4963-9199-50a87d910605@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.05.2024 12:00, Sergiy Kibrik wrote:
> 21.05.24 09:05, Jan Beulich:
>>> This is because MCG_LMCE_P is defined in arch/x86/cpu/mcheck/x86_mca.h
>>> -- so either MCG_LMCE_P (+ a bunch of MCG_* declarations) has to be
>>> moved to common header to be accessible, or local x86_mca.h got to be
>>> included from common arch/x86/include/asm/mce.h.
>>>
>>> As for the MCG_* declarations movement I didn't think there's a good
>>> enough reason to do it; as for the inclusion of x86_mca.h it didn't look
>>> nice at all.
>> I'm afraid I don't follow the latter: Why's including x86_mca.h any worse
>> than what you do right now?
> 
> To include x86_mca.h from asm/mce.h something like this line would be 
> needed:
> 
> #include "../../cpu/mcheck/x86_mca.h"
> 
> I've found only two include-s of such kind, so I presume they're not common.

Indeed, and I have to apologize for not reading your earlier reply quite
right.

Jan

> Besides xen/sched.h includes asm/mce.h before declaration of struct 
> vcpu, so body of vmce_has_lmce(const struct vcpu *v) can't really be 
> compiled in asm/mce.h
> 
>    -Sergiy



From xen-devel-bounces@lists.xenproject.org Tue May 21 10:23:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 10:23:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726805.1131176 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MeF-0001wL-AK; Tue, 21 May 2024 10:23:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726805.1131176; Tue, 21 May 2024 10:23:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MeF-0001wE-7k; Tue, 21 May 2024 10:23:11 +0000
Received: by outflank-mailman (input) for mailman id 726805;
 Tue, 21 May 2024 10:23:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9MeD-0001w8-Lj
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:23:09 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1e328073-175c-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 12:23:08 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-574b3d6c0f3so9337062a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 03:23:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfde7sm16390746a12.48.2024.05.21.03.23.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 03:23:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e328073-175c-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716286988; x=1716891788; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9O6uATdDQLsSAHSsAZ+wK+aVoX+VXl4XXz85O4tGkbA=;
        b=coRY/HRPJmOhkTX23+5bHKwGAvG3RYddyQ1D60Au5UfpCYWkk1O0b7/o/TWW/VLY0k
         1UrxIR6kyvzWSzoGvktAG2rmx3/Oal6WSuTgZrZNNwrWYK883JJwBcLd1t0zUhnb1PpN
         dfNgXsNaB/sS1OoUXyhuBRcW51LqoRIljafCmoH5FNBWSYadtARYWxBQtsbcKAzEvNGL
         ex5+//pDLXsxiCGTQd/tBXS068luZTrpTsP1j4SEe+sGlZzALSC4Nz7S4ESPraAZfMbs
         yZyJcAzvDBr+CM8DgtR+iCvLhBZPkJtg6ctOa8r55caY8LAqnb3mF0ky+i2hvojFrz1k
         Wr+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716286988; x=1716891788;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9O6uATdDQLsSAHSsAZ+wK+aVoX+VXl4XXz85O4tGkbA=;
        b=enDrGMRe2qzqDnlWny4j4cvPPk5zYpjzfkpjSpfWSmih3IooazCqtDn6rNysOKNcmJ
         VkN4wMKRJsezQ1RhFVyUxEYr2XiSRsa7wkzGuC90B+Gmu7vndV2UOrA68yUjb40Adqqb
         BQGzwrUCN6mzAVVK72HtFI5esMNfSmRbjTJJaBqkqucx0TfYtlcydhkdDWcn2jUtPLNM
         AbG4CifVT1/0yPkJFn3Smnk7Uk8kSPIrLDd+2Vo/eWEZuV6Gg2tfC5c69TsgDjwNce0r
         67GAakBhuHvuDGIWM2ysWvZaPr1oc1mI1djND3WbcaMlcqSJkE3CiTL4DzxIBEFV92PW
         q+8A==
X-Gm-Message-State: AOJu0YxaKwnxFT9fPEmmpC5v8bTQT1R4V8/Nfd4J89SatssmIQbYlCGd
	AEDcb/LAvyMQ7eQLty9knWDyeolZv48+qedhHDOavKmNcn5mgoU6TTVBf3Gy8VI/WfbGtSBfjOc
	=
X-Google-Smtp-Source: AGHT+IHcoIl3Jyd3o0SkfB2YTSpNGcIiUNyPR4E0ZKUE7rjo2YKrfbLp3mnWwMFjxhXUumdUDKWaiw==
X-Received: by 2002:aa7:d450:0:b0:572:72ff:da34 with SMTP id 4fb4d7f45d1cf-5735a1c68e9mr20716392a12.14.1716286987920;
        Tue, 21 May 2024 03:23:07 -0700 (PDT)
Message-ID: <b2cc7bd7-d1a8-45eb-8fd2-cdb95824d1f2@suse.com>
Date: Tue, 21 May 2024 12:23:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com> <ZjjQIaxEwS6b-swj@macbook>
 <24d52bbb-1329-4f8a-81be-505a35969875@suse.com> <Zjjg2ueqgjmn-MS3@macbook>
 <44af358a-9510-4056-826c-3be99dc25830@suse.com> <ZksmJp5JnQoBYZ6U@macbook>
 <d1267c40-e2d6-4919-81ac-54ad217e4702@suse.com> <ZkxxXNbOJ48Bl_Sk@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZkxxXNbOJ48Bl_Sk@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 12:03, Roger Pau Monné wrote:
> On Tue, May 21, 2024 at 08:21:35AM +0200, Jan Beulich wrote:
>> On 20.05.2024 12:29, Roger Pau Monné wrote:
>>> On Wed, May 15, 2024 at 12:07:50PM +0200, Jan Beulich wrote:
>>>> On 06.05.2024 15:53, Roger Pau Monné wrote:
>>>>> On Mon, May 06, 2024 at 03:20:38PM +0200, Jan Beulich wrote:
>>>>>> On 06.05.2024 14:42, Roger Pau Monné wrote:
>>>>>>> On Thu, Feb 15, 2024 at 11:15:39AM +0100, Jan Beulich wrote:
>>>>>>>> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>>>>>>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>>>>>>>  
>>>>>>>>          if ( use_ats(pdev, iommu, ivrs_dev) )
>>>>>>>> -            dte->i = ats_enabled;
>>>>>>>> +            dte->i = true;
>>>>>>>
>>>>>>> Might be easier to just use:
>>>>>>>
>>>>>>> dte->i = use_ats(pdev, iommu, ivrs_dev);
>>>>>>
>>>>>> I'm hesitant here, as in principle we might be overwriting a "true" by
>>>>>> "false" then.
>>>>>
>>>>> Hm, but that would be fine, what's the point in enabling the IOMMU to
>>>>> reply to ATS requests if ATS is not enabled on the device?
>>>>>
>>>>> IOW: overwriting a "true" with a "false" seem like the correct
>>>>> behavior if it's based on the output of use_ats().
>>>>
>>>> I don't think so, unless there were flow guarantees excluding the possibility
>>>> of taking this path twice without intermediately disabling the device again.
>>>> Down from here the enabling of ATS is gated on use_ats(). Hence if, in an
>>>> earlier invocation, we enabled ATS (and set dte->i), we wouldn't turn off ATS
>>>> below (there's only code to turn it on), yet with what you suggest we'd clear
>>>> dte->i.
>>>
>>> Please bear with me, I think I'm confused, why would use_ats(), and if
>>> that's the case, don't we want to update dte->i so that it matches the
>>> ATS state?
>>
>> I'm afraid I can't parse this. Maybe a result of incomplete editing? The
>> topic is complex enough that I don't want to even try to guess what you
>> may have meant to ask ...
> 
> Oh, indeed, sorry, the full sentences should have been:
> 
> Please bear with me, I think I'm confused, why would use_ats() return
> different values for the same device?

Right now it can't, yet in principle opt_ats could change value when
wired up accordingly via hypfs.

> And if that's the case, don't we want to update dte->i so that it
> matches the ATS state signaled by use_ats()?

That depends on what adjustment would be done besides changing the
variable value, if that was to become runtime changeable.

>>> Otherwise we would fail to disable IOMMU device address translation
>>> support if ATS was disabled?
>>
>> I think the answer here is "no", but with the above I'm not really sure
>> here, either.
> 
> Given the current logic in use_ats() AFAICT the return value of that
> function should not change for a given device?

Right now it shouldn't, yes. I'm still a little hesitant to have callers
make implications from that.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 10:30:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 10:30:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726815.1131187 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MlS-0003xz-3F; Tue, 21 May 2024 10:30:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726815.1131187; Tue, 21 May 2024 10:30:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9MlS-0003xs-0d; Tue, 21 May 2024 10:30:38 +0000
Received: by outflank-mailman (input) for mailman id 726815;
 Tue, 21 May 2024 10:30:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9MlR-0003xm-2o
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:30:37 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2842443a-175d-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 12:30:34 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59b58fe083so592331366b.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 03:30:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17f0dsm1588205966b.211.2024.05.21.03.30.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 03:30:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2842443a-175d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716287434; x=1716892234; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=en/AzRk/WEfRzuawz7Kqg4Hvj7RCwJmtNwU8noXrYUo=;
        b=QfIZyMtAORIH4jnFEBoAKe2kjBrFnPrbbz9Hrbg/bblXmOWPF568vDxLckDy/tdw08
         Anrx/L8ok4varTJ6ClwzM3hWpV0f5BhtqiKfeT68Awe/0MyAg1tGMS6Rbq7KC0E3LjU0
         uvPmb6MH/t7T5UzXgeyaL1gmRiiQhOKS2bDFB562M6CAVOiEL2MC5RPJKeXG1Pw6fbcR
         8Kkl/5J90jNv4DsAuaxyBC0/Ecio4Geam2rS/AqNDsbzgcNpwFr7/l4uQdh0hCyiq1Qe
         +3GMbj1jtMOFZFszTDlt3GyuQV+5F1ZwFsrE8Shi+M3gx6yyeg4wBYFpqaXAFe5sgsaR
         iXRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716287434; x=1716892234;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=en/AzRk/WEfRzuawz7Kqg4Hvj7RCwJmtNwU8noXrYUo=;
        b=TakUjPoLjUCgYMcm5cstnw9aXJdgPTZOUdrGXkbcL4CUKIZ4uPtsWjuJbK+XWBAe1K
         jo8RfeyltA1L2tbQ2WMNu6X9m6VPrj/xdLfpKsj0Y+0m9RwW0ms20WO3J0NWGkHTIQX2
         tfVh4hTsE1+F2HXINu1mX/0syWJxBDGoJDSeaLKWuj0tHFOT6UKKSZezlxUgazYENDfS
         uCwYb2U457u/B0Lu0U+fym0qPVVIdiqLpPd/I4narEkZfCfhH/Z7O3KnPQqBS/RuLOIY
         Acd3XLkB4hraCVbregMCBwLMQ0NiChI4qAfdfyipwpHVegDF7MZ2tWTPOOQR04jm0cub
         PejQ==
X-Forwarded-Encrypted: i=1; AJvYcCWHnhUtmKAjrHwoWbuNsXf+pIEMakUwxrBEysCWNj8LQn7SjpAoWLhM268KuzeRF/p+d23P75iNvjeSo2ydnwwukorUszmhjQmrPOCUsWw=
X-Gm-Message-State: AOJu0Yy7QgSPyXZN4qXQ09PhIIHe8F3xtr1x2IouusBeJVhvupjiH5QN
	6GdcNt0OJP+sOlTZB2ksnjeok1CPt2R48JUXEbDCyF5nvbRXB7J/7ydg29nWAg==
X-Google-Smtp-Source: AGHT+IH7/j/Th8iA6++dCnnalHKSsnZe2z57dYjHKt5K30zsTiwegAibQFGvYk6WtFfGBOrfqYxvGQ==
X-Received: by 2002:a17:907:d22:b0:a59:aa9d:3142 with SMTP id a640c23a62f3a-a5a2d5cb794mr2572004266b.37.1716287434331;
        Tue, 21 May 2024 03:30:34 -0700 (PDT)
Message-ID: <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com>
Date: Tue, 21 May 2024 12:30:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240517133352.94347-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 17.05.2024 15:33, Roger Pau Monne wrote:
> Enabling it using an HVM param is fragile, and complicates the logic when
> deciding whether options that interact with altp2m can also be enabled.
> 
> Leave the HVM param value for consumption by the guest, but prevent it from
> being set.  Enabling is now done using and additional altp2m specific field in
> xen_domctl_createdomain.
> 
> Note that albeit only currently implemented in x86, altp2m could be implemented
> in other architectures, hence why the field is added to xen_domctl_createdomain
> instead of xen_arch_domainconfig.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com> # hypervisor
albeit with one question:

> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>      bool hap = config->flags & XEN_DOMCTL_CDF_hap;
>      bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
>      unsigned int max_vcpus;
> +    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
> +                                         XEN_DOMCTL_ALTP2M_mode_mask);
>  
>      if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
>      {
> @@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }
>  
> +    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
> +    {
> +        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
> +                config->flags);
> +        return -EINVAL;
> +    }
> +
> +    if ( altp2m_mode && nested_virt )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "Nested virt and altp2m are not supported together\n");
> +        return -EINVAL;
> +    }
> +
> +    if ( altp2m_mode && !hap )
> +    {
> +        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
> +        return -EINVAL;
> +    }

Should this last one perhaps be further extended to permit altp2m with EPT
only?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 11:00:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:00:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726824.1131197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NDc-0007Y6-9F; Tue, 21 May 2024 10:59:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726824.1131197; Tue, 21 May 2024 10:59:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NDc-0007Xz-6N; Tue, 21 May 2024 10:59:44 +0000
Received: by outflank-mailman (input) for mailman id 726824;
 Tue, 21 May 2024 10:59:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9NDb-0007Wj-1E
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 10:59:43 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 38eea05e-1761-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 12:59:40 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so1193829966b.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 03:59:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d40sm1583862266b.75.2024.05.21.03.59.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 03:59:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38eea05e-1761-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716289180; x=1716893980; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E4P+xMRlYpy264pZ1nWx3xHxFBAFbv6+a31JFtHGv5w=;
        b=gCafwuSAbebqK93HryDYEyfBv63JZ7qUeYjfrAx1S3thPpBqQXDk2/no0jDFO94n53
         leeuQI0eVOqm+uOL4cCg3+x5QQTAONMpW5/d2x94tb5YN4iESlliaACCnHPon5NSGDJD
         DQewkuYa8ja/yGJiyU9qKo4B+yjPLfjHzaVh7huoPTekrM5BEzU3tlT+I3hzJVByvMEc
         TD8xqzuMXtY1hxfcmY4gIMkPyLwpZqGR2QfkFGK2XsX59QNbKJVPQU6hbVHlfoBJ8Xnk
         F1izJzpOIUzqSJkkH3lJCkg3LA3+20EAeePSZYeI4GZA+W9+kZHMLgAICv0nZ2oIOg1e
         MXQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716289180; x=1716893980;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=E4P+xMRlYpy264pZ1nWx3xHxFBAFbv6+a31JFtHGv5w=;
        b=GZgY+Lw7H8fNmpop0n4Vq7KF0wnOhzbPQy0IeB7Mw60+OcXq5z03lSyn2w+Ww55Boe
         NCCNrhswhxo1SdazTwPsuY+7W9Mc/zupqpFO8QfK3nnRQMTA5xdp+k2u7s5iHqjEp7wJ
         6JOV3Faqp/s3UhgxwJY+sWpjFdrCHFaHRvlvAWKsemXGzTVLNqzEI1hlGu9dXoBuT2R6
         17UVmKqObbTQPuOyzr4IgaM5ip7xB1UG5jPZcj0W2ytw5LZy3EhEi04v9frG4NNJJYHP
         9iofUIxZcEZ58n9CU0h+BgH3uOXNtbjfcWR34NQUAXvYw9fw61Df4L0bymahV8NEc769
         xA7w==
X-Forwarded-Encrypted: i=1; AJvYcCUJxlHA3GUyX+SLMTyRvFnJJJHGQ3nrtyyhhYpvl/6xDveclBuooMm52568Ddzc4jTVf7I1sXp3PLQog/nyNUQm/xd2MKdcwoDweFyx7T4=
X-Gm-Message-State: AOJu0YxsKxqcdPIsP6WQtjA/uMKXqQp5S4zMEyZbO8HjTcXUGKWw9Tce
	yl0xqmdeTnCB2pp9HfSHsB1kQ7L4TDzbAmFdii6L0Fzr3IkX1/Q474k4iqhb7A==
X-Google-Smtp-Source: AGHT+IHRyBvXRWfVhwcjl1pf9tzSU8YXLqNtxVdOePb9YckUXCx2f76jo8TpdnycEgi61MykfWY0AQ==
X-Received: by 2002:a17:907:a4e:b0:a61:a705:56f3 with SMTP id a640c23a62f3a-a61a705582fmr420057166b.35.1716289180357;
        Tue, 21 May 2024 03:59:40 -0700 (PDT)
Message-ID: <21196eb3-82ca-470b-8dc9-1a95b427f3b9@suse.com>
Date: Tue, 21 May 2024 12:59:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m
 views configurable
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <cover.1716029860.git.w1benny@gmail.com>
 <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 18.05.2024 13:02, Petr Beneš wrote:
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -685,6 +685,18 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }
> 
> +    if ( config->nr_altp2m && !hvm_altp2m_supported() )
> +    {
> +        dprintk(XENLOG_INFO, "altp2m requested but not available\n");
> +        return -EINVAL;
> +    }
> +
> +    if ( config->nr_altp2m > MAX_EPTP )

The compared entities don't really fit together. I think we want a new
MAX_NR_ALTP2M, which - for the time being - could simply be

#define MAX_NR_ALTP2M MAX_EPTP

in the header. That would then be a suitable replacement for the
min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) that you're adjusting
elsewhere. Which however raises the question whether in EPT-specific
code the min() wouldn't better survive, as min(d->nr_altp2m, MAX_EPTP).

> @@ -5228,7 +5234,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)
>      if ( !hvm_is_singlestep_supported() )
>          return;
> 
> -    if ( p2midx >= MAX_ALTP2M )
> +    if ( p2midx >= v->domain->nr_altp2m )
>          return;

You don't introduce a new local variable here. I'd like to ask that you also
don't ...

> @@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
>      /* altp2m view 0 is treated as the hostp2m */
>      if ( altp2m_idx )
>      {
> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
> -             mfn_x(INVALID_MFN) )
> +        if ( altp2m_idx >= d->nr_altp2m ||
> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
> +             == mfn_x(INVALID_MFN) )

Please don't break previously correct style: Binary operators (here: == )
belong onto the end of the earlier line. That'll render the line too long
again, but you want to deal with that e.g. thus:

             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx,
                                                    d->nr_altp2m)] ==
             mfn_x(INVALID_MFN) )

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 11:10:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:10:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726834.1131207 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NOH-00025S-97; Tue, 21 May 2024 11:10:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726834.1131207; Tue, 21 May 2024 11:10:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NOH-00025L-5h; Tue, 21 May 2024 11:10:45 +0000
Received: by outflank-mailman (input) for mailman id 726834;
 Tue, 21 May 2024 11:10:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9NOG-00025F-3O
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:10:44 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c2d94469-1762-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:10:41 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-572a93890d1so9472623a12.3
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:10:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d14sm1599411566b.130.2024.05.21.04.10.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:10:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2d94469-1762-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716289841; x=1716894641; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RowLZQ7QNS+dIfDO2vpYQHhf0HXk41FQllbMwLSDpVs=;
        b=DevbnG0B4yqq1PjIf+w4Y0qWBzHwrqu97jwExxsNyfb/4GrKZiG/hqVTavkj/qyEy+
         FuXbh7Dv0sQ7F3ogV3r7Hm8Oy+3NtmfncWPN0qzKEu2HmpSpLvSqLTUio0nZrsKIsmJv
         lPcQvPmupeGFBOkL6l7iz8IwnKYTTjX8kt0CONH+3ayNX2gpuUfj2ZXPeYqiOl4ENo2K
         hSQu0v1mdCo/Pzvv9dg+zoOqfXsW3OKiHHut8b4LsQIgg9mvK95n9D+8vXAOBp2ZvPr4
         I/0s/waj9spcC6/b3bTMhk6uv24HdchWQQUxU0WxONUXbtBJZ+X6ymncc4N3rXctxJ8H
         UEow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716289841; x=1716894641;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RowLZQ7QNS+dIfDO2vpYQHhf0HXk41FQllbMwLSDpVs=;
        b=CEpr9l7dG+NoKVSnY5tkPocLHidGBMqwMrTE9O+yxD+xrmc/ZwBrfNqttXIcQAfnqb
         X16pbLxqNvGyXTwtOx/b6/YMfWoZizCgPXifqjYwKwxXfQ/H1SCO3La/nSqjsk2O3RsH
         4U64YJm87k0y9TPiCoFtfU3YPmE9DUcaXeFC0yareUzF5pzCx/cbbmYqUecfGVynSyJh
         DjoBPo4Pq4Yj+uGaEBs0sYvGk+4CEd8T42iLwZ7ZwsTVD+1nlMary9zVATp12RiouaAC
         i/T84Z2nFPxXjHpUdI/4Zel7FBphllr16DMXBUDTyMvE1uXYnUyBgaNLf3AUn2iBjr+T
         fZlw==
X-Forwarded-Encrypted: i=1; AJvYcCXEO8rkhwYdGJfhMSuZgUnM8HMIGUz+yZK6cA/512r4q5n3Af/P4lD5KiRAc8c2806B7qymwR2HmWY4jZNFhNzyqWIVSPjPJ2TDyQmN5LQ=
X-Gm-Message-State: AOJu0YzMofsH5aaFXLwx/FWml08jXlSibhv6rj0YEpD1ar/zcBevxfEX
	oXp3zEOXaSgJr/1e9puMi7wGo81Lw97i0rGZOfOrbkkMQfQlj4jKoqS4zd7ZfQ==
X-Google-Smtp-Source: AGHT+IHHH7xGkHcklLxFeb2TpeBRJtaQOXTq50OzQ6JOZ2PMqfEbLDri4KYAJF7kMC81GrbX947Kzw==
X-Received: by 2002:a17:906:ae50:b0:a59:c3a5:4df0 with SMTP id a640c23a62f3a-a5a2d53b0bbmr3215910366b.4.1716289841147;
        Tue, 21 May 2024 04:10:41 -0700 (PDT)
Message-ID: <54821721-0d29-487d-b346-9dda52f339dd@suse.com>
Date: Tue, 21 May 2024 13:10:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 15:54, Oleksii Kurochko wrote:
> The following generic functions were introduced:
> * test_bit
> * generic__test_and_set_bit
> * generic__test_and_clear_bit
> * generic__test_and_change_bit
> 
> These functions and macros can be useful for architectures
> that don't have corresponding arch-specific instructions.
> 
> Also, the patch introduces the following generics which are
> used by the functions mentioned above:
> * BITOP_BITS_PER_WORD
> * BITOP_MASK
> * BITOP_WORD
> * BITOP_TYPE
> 
> The following approach was chosen for generic*() and arch*() bit
> operation functions:
> If the bit operation function that is going to be generic starts
> with the prefix "__", then the corresponding generic/arch function
> will also contain the "__" prefix. For example:
>  * test_bit() will be defined using arch_test_bit() and
>    generic_test_bit().
>  * __test_and_set_bit() will be defined using
>    arch__test_and_set_bit() and generic__test_and_set_bit().
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one remaining nit (can be adjusted while committing, provided other
necessary acks arrive):

> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned long x)
>   * scope
>   */
>  
> +#define BITOP_BITS_PER_WORD 32
> +typedef uint32_t bitop_uint_t;
> +
> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
> +
> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> +
> +extern void __bitop_bad_size(void);
> +
> +#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
> +
>  /* --------------------- Please tidy above here --------------------- */
>  
>  #include <asm/bitops.h>
>  
> +/**
> + * generic__test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline bool
> +generic__test_and_set_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old | mask;
> +    return (old & mask);
> +}
> +
> +/**
> + * generic__test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline bool
> +generic__test_and_clear_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old & ~mask;
> +    return (old & mask);
> +}
> +
> +/* WARNING: non atomic and it can be reordered! */
> +static always_inline bool
> +generic__test_and_change_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old ^ mask;
> +    return (old & mask);
> +}
> +/**
> + * generic_test_bit - Determine whether a bit is set
> + * @nr: bit number to test
> + * @addr: Address to start counting from
> + */
> +static always_inline bool generic_test_bit(int nr, const volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    const volatile bitop_uint_t *p =
> +                        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);

Indentation is odd here. Seeing that the line needs wrapping here, it would
imo want to be

    const volatile bitop_uint_t *p =
        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);

(i.e. one indentation level further from what the start of the decl has).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 11:19:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:19:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726841.1131217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NW5-0002iW-1g; Tue, 21 May 2024 11:18:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726841.1131217; Tue, 21 May 2024 11:18:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NW4-0002iP-UT; Tue, 21 May 2024 11:18:48 +0000
Received: by outflank-mailman (input) for mailman id 726841;
 Tue, 21 May 2024 11:18:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9NW3-0002hu-Io
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:18:47 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e369d7eb-1763-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:18:45 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a5a5c930cf6so846174466b.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:18:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cde6f8c89sm744409266b.70.2024.05.21.04.18.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:18:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e369d7eb-1763-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716290325; x=1716895125; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rxPl5N/kM0RaLefHbtKfZV/XFQxzSJfrE1XvgFFM9S0=;
        b=C3eXzaGinmzAXenJdsj9CZsZPoqxDOb03F5pRGlee07jiOeshMUC9AqwLeSGRX0d12
         T6vy+BFp56essRl6td+94CZIjZCl2SWzeKlstqA2W2wIUCppDBaRQkjzjREOjvT3cAGO
         uSwKEiEEE7uRfmJck1oh8M2fBmRPoOX5r0i8I/FZh0jPBxUGuQ2nfGF/OpGi7EobtU1j
         kIWzK4D8V32CoZkFWGWoZuHTUOG1Ap9Hv9PbxexxNZN0feMM88f5MEchOLTSKwnWbHwW
         mTa7NnFQ4RUR3hbHyRMky+yMvw6Ok5IbCMRPpBG5HA+vOYHA+YHNa0C4Deg+9LAPZqoX
         zQQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716290325; x=1716895125;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rxPl5N/kM0RaLefHbtKfZV/XFQxzSJfrE1XvgFFM9S0=;
        b=Tz7U9Cu8EePBE/e2T+YYw79NqlO6/FPOT7sCXVJSK8qyj/CXWmq2Ow4ajqgUhLUAHt
         QJXixa3s9QPJaKtswna0RRIf2NW6BXxAxYneWS4PZkzS3oOiu6jKiFgIR+io3Ji+cJB8
         jw2+r6SgDFqv4geX8o0ACzFERsIe5C8HwIZk+MaG++B9dbqFhzzmvP+fi+NtigyhscXf
         9CQqmYD9O0sBoA4UMFj44gPfpT1WQXWKPHp+JyGSvrXhHmHs5GqUwSh4c9n0Y8JW7qwT
         Qyd9MStGl53ExJQoEJLWMqy8849PMHMhdyBSrSKM2t46WYXeGVA16zI945leDerXmydq
         hMSg==
X-Forwarded-Encrypted: i=1; AJvYcCVg5F6bW/AnkmUKWH3ojlNE+lIyT1ppxf5VNYdxnyyeNyHzdnm1/7XLGpeHsOqeF/shyZokxaiOsV8yWjaqJlGj/S14vglURj2IXyVPEbM=
X-Gm-Message-State: AOJu0Yw9/01VM1C0YWe6wtFpGVvbhXazyhPeayVuwGonJZWiFi6MRzmz
	cv7qWNNQeaYg2eu7QJj1rosmC1CvK0UQCBgYp2EFHuyCVw0IAeVDzIX3bv7q8Q==
X-Google-Smtp-Source: AGHT+IFKKKf5Optn3FbgOwxAjmyZL4ynzOyX221FSYXTHI93WThfv52xlvb2Q349rcz7CNrhjYWoaA==
X-Received: by 2002:a17:907:8687:b0:a59:cb29:3fb3 with SMTP id a640c23a62f3a-a5a2d666ad7mr2526071866b.53.1716290325331;
        Tue, 21 May 2024 04:18:45 -0700 (PDT)
Message-ID: <d26a4033-8cd1-4a9c-8549-49a36b7d52fb@suse.com>
Date: Tue, 21 May 2024 13:18:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 03/14] xen/bitops: implement fls{l}() in common logic
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 15:54, Oleksii Kurochko wrote:
> To avoid the compilation error below, it is needed to update to places
> in common/page_alloc.c where flsl() is used as now flsl() returns unsigned int:
> 
> ./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
>        18 |         (void) (&_x == &_y);            \
>           |                     ^~
>     common/page_alloc.c:1843:34: note: in expansion of macro 'min'
>      1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
> 
> generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0,
> the result in undefined.
> 
> The prototype of the per-architecture fls{l}() functions was changed to
> return 'unsigned int' to align with the generic implementation of these
> functions and avoid introducing signed/unsigned mismatches.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  The patch is almost independent from Andrew's patch series
>  ( https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t)
>  except test_fls() function which IMO can be merged as a separate patch after Andrew's patch
>  will be fully ready.

If there wasn't this dependency (I don't think it's "almost independent"),
I'd be offering R-b with again one nit below.

> --- a/xen/arch/x86/include/asm/bitops.h
> +++ b/xen/arch/x86/include/asm/bitops.h
> @@ -425,20 +425,21 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
>   *
>   * This is defined the same way as ffs.
>   */
> -static inline int flsl(unsigned long x)
> +static always_inline unsigned int arch_flsl(unsigned long x)
>  {
> -    long r;
> +    unsigned long r;
>  
>      asm ( "bsr %1,%0\n\t"
>            "jnz 1f\n\t"
>            "mov $-1,%0\n"
>            "1:" : "=r" (r) : "rm" (x));
> -    return (int)r+1;
> +    return (unsigned int)r+1;

Since you now touch this, you'd better tidy it at the same time:

    return r + 1;

(i.e. style and no need for a cast).

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 11:20:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:20:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726847.1131226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NY0-0004QV-Ei; Tue, 21 May 2024 11:20:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726847.1131226; Tue, 21 May 2024 11:20:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NY0-0004QO-By; Tue, 21 May 2024 11:20:48 +0000
Received: by outflank-mailman (input) for mailman id 726847;
 Tue, 21 May 2024 11:20:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9NY0-0004QG-2Z
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:20:48 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2afb98ab-1764-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:20:45 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56e37503115so6034542a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:20:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cebb3cd33sm676964566b.34.2024.05.21.04.20.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:20:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2afb98ab-1764-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716290445; x=1716895245; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=iclHLejdVsdq+hBjiyZTbl5Uhef9O0yZKOK9QwDNKkc=;
        b=QXcm5yECrHHQuCyWRGdufXE9uk100TdW44Y0+J4c+Onj3D/GRfnU6Jxjund/u9W4zM
         l2udztIAlbeylekwU/LafBi/D8DOjHOubWpEvx2UeMGrmpSLaEeojdsGbazwQ2T+YIh2
         z+PhLEhr2YUcnHcUU6FoHBkFBrddLZPZ11M/PEyIMuLSJEiPIcUP9Xew4fDAtsNNY9OT
         HZrLDfsph+KhM7OC6oJzgCoJ7VSn42Q4RV/sWozUJlzeKlXbxnKj84hPsizoW2bvojX1
         ZC8ZJAxZixrGz15VhZ3ZVK0oHuHS7KK114A1yu/oeMASfPQMKqwxj5et5sSg653cGxRn
         BiiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716290445; x=1716895245;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=iclHLejdVsdq+hBjiyZTbl5Uhef9O0yZKOK9QwDNKkc=;
        b=ilAlJSuAAxMFEYB29eJ6XK2Af8vPAr5GVfLyHektd3wa7xvndFepFwiXfuoTxtBbcd
         ALbrNgannZpvbDcLxm2YfWiVkAwD4HASxUyQmlRXAQE7Cz48BJ7DJHuSPWFnmQy7fqqc
         ffDfz8QwKtYH4Ti7vd7SE15PCn7CVebRe+Z5zSZ38GDQTuh/+pmjNeQ1OJTegIyqegoN
         lYngyPiC4zt0OzmVQ17N3bBHwGBk1v6h3Gd0p0yuzvIKnCchdJCZESyyZFzWG/bYve5G
         MKXFSPdlwk5tjhYHjwiBmbe9KM6faxacs0FpQyOMWoCg4zLQOb4rC33CSoW34N+mn6Kp
         IvvQ==
X-Gm-Message-State: AOJu0Yw9yNvhFcW+GbknKtNjh3oLuyW6hrBGyXqi9VJcEhCgcHETA9Vo
	4gyqSy2Bf1smWMD0OSc6CpXk4i9YXJerPYX0S09wDwM5UN6k82taRMV2WPIXEg==
X-Google-Smtp-Source: AGHT+IH8vRu13EL93AfMjfUhzQ0GR87tQfa7IHQmgio0Bw4IGOx8gQ2EzKuLuNJiXyLUKJbYwWwRJg==
X-Received: by 2002:a17:906:c07:b0:a5c:f349:da7c with SMTP id a640c23a62f3a-a5cfe08c9ddmr846154366b.56.1716290445365;
        Tue, 21 May 2024 04:20:45 -0700 (PDT)
Message-ID: <d5fbea94-ae3d-4818-8947-a3f509218959@suse.com>
Date: Tue, 21 May 2024 13:20:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/4] x86/vpmu: address violations of MISRA C Rule 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <ecba64bb8295fa27f0ddbb0905b0983a3572b1ae.1715757982.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405151617240.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151617240.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 01:17, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Tue May 21 11:24:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:24:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726852.1131236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Nba-000569-RL; Tue, 21 May 2024 11:24:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726852.1131236; Tue, 21 May 2024 11:24:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Nba-000562-Ol; Tue, 21 May 2024 11:24:30 +0000
Received: by outflank-mailman (input) for mailman id 726852;
 Tue, 21 May 2024 11:24:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Nba-00055w-6s
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:24:30 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id af686151-1764-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:24:28 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so738501166b.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:24:28 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5ce94e51basm688875266b.103.2024.05.21.04.24.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:24:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af686151-1764-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716290667; x=1716895467; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OvBsnKckcv8siwHttjPOLwSVhuZ7ny3vrc/c/QGK/tw=;
        b=goc+Padq+6E5bigVUlS0ttaarV91toNYHz2oXDC5IyYZnRuf4Q5t8mj1mkpfuXPShm
         xaVPs+sSDFuP92ljELFZDHO29WjNZdRTtzi6qLKhUcNcIuX0ZFJ6c6KwOmsb+VfkX2Mr
         If5yt/JTGKngIN3iJzixtAQO+RbRdlAqkJkjoDGkRtXzdaFnNtRu1YSCe1BHkU2n8hPH
         IF05Bm7eG+M0pbzuGOl1ccRKGvwmxQdJBBC8IUu7YCltbuSW/6SRCnIG/yCF/fFEnomO
         pQZXhmLmXRbyzoY9KwW7UIveLqXDx+mbUfQ7OpyaRWkA9J/tC2c0q9rSS+6qIkaQIum+
         BVCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716290667; x=1716895467;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OvBsnKckcv8siwHttjPOLwSVhuZ7ny3vrc/c/QGK/tw=;
        b=lZWIVx/uifpQVzmJwxWfHY5j8hM9FzcxyZqM8LCTeWPv8jXjs+VaOQt6jXXeP5N7qH
         Jlf9H42RTDcgeo0fXDmXMCEPTuDSuYs/Dov6WOV/FHANzpplvyUrs32iQm2VCwF8jmhR
         me6jTnR7GcuK/jI/h+GAWFvUSBN+Mbyvjs2/r0Ije4HLxFaUJvjSjLsdHnhi5/FaN5YB
         ouVE429qKt61297sLbAaXmkxbUaSI005oOQQDTdiLeUzIENN3umLBeTTwJrvvdc+YEhP
         B330NYNu/C45ELAFWis6wvgV0hV+mkNi1XL/EQaip7T79W5eIPcifg3XJHWyyOEfv+5v
         JlQQ==
X-Gm-Message-State: AOJu0YzsHn/NSI5IILFvMyfDw2rRhAVoStVU8ATvFaSejCM8wj2QmE+g
	6c2llOh8yCDmeuEGfVOrYnGdSBM8Y0y0dwUcbfrLhCa3zIsoID+bGUL7nnmIRA==
X-Google-Smtp-Source: AGHT+IEOfwEdWknvuu05DaO8dJ0CQzlIbqA7lR55qfnLCykl8YMm0TCLwXwr5xNM2Zj4Kp9JyC8nkA==
X-Received: by 2002:a17:906:4882:b0:a59:a727:1e81 with SMTP id a640c23a62f3a-a5a2d57a381mr3103859166b.20.1716290667595;
        Tue, 21 May 2024 04:24:27 -0700 (PDT)
Message-ID: <d3710ec6-63fd-4387-af7d-7f8f79c522e3@suse.com>
Date: Tue, 21 May 2024 13:24:26 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/4] x86/hvm: address violations of MISRA C Rule 20.7
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <6d14b3283005cf1a30c4fa24f9841586a41e2b1b.1715757982.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405151618170.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151618170.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 01:18, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> 
>> ---
>>  xen/arch/x86/hvm/mtrr.c             | 2 +-
>>  xen/arch/x86/hvm/rtc.c              | 2 +-
>>  xen/arch/x86/include/asm/hvm/save.h | 2 +-
>>  3 files changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
>> index 32f74c1db03b..1079851f70ed 100644
>> --- a/xen/arch/x86/hvm/mtrr.c
>> +++ b/xen/arch/x86/hvm/mtrr.c
>> @@ -16,7 +16,7 @@
>>  #include <public/hvm/e820.h>
>>  
>>  /* Get page attribute fields (PAn) from PAT MSR. */
>> -#define pat_cr_2_paf(pat_cr,n)  ((((uint64_t)pat_cr) >> ((n)<<3)) & 0xff)
>> +#define pat_cr_2_paf(pat_cr, n)  ((((uint64_t)(pat_cr)) >> ((n) << 3)) & 0xff)
> 
> just a cosmetic change

Not really, as Nicola already pointed out. However, there's an excess pair of
parentheses which better would have been re-arranged rather than adding yet
another pair:

#define pat_cr_2_paf(pat_cr, n)  (((uint64_t)(pat_cr) >> ((n) << 3)) & 0xff)

Preferably with that (which I'll likely take the liberty of adjusting while
committing)
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 11:27:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:27:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726861.1131250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Nei-0005gl-98; Tue, 21 May 2024 11:27:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726861.1131250; Tue, 21 May 2024 11:27:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Nei-0005ge-5z; Tue, 21 May 2024 11:27:44 +0000
Received: by outflank-mailman (input) for mailman id 726861;
 Tue, 21 May 2024 11:27:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9Neg-0005gY-Np
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:27:42 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 224b9586-1765-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:27:40 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-5206a5854adso4306954e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:27:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c6b1sm1600780466b.57.2024.05.21.04.27.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:27:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 224b9586-1765-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716290860; x=1716895660; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xmc5d0J0AlvJbOD8urO/lbTRDEzhDbKc0+/1IT9h7NQ=;
        b=LZU88CQNsJApMR5u/99eS6UVoA33xq/I2Hde6sCUX8g7Taum0qGNkP/hgdZqAgO/GU
         Qg5llaOeR6mK8l0NwWpjY72OLzBFM7n64xDbaoPDCNSwwlRXEzu9B/rBot3sTH89hZ9u
         E7RExxDI3YPBgA6aqgcUXum2rFf4lwq7TOgxc+oLD0uB70nnpENb0SXzdaQhNyOl8lOo
         6tBw5m4Q0Gbud7E1cONhIkV7F0jqVPnyqVM+GVqeCKBxkRIBgDzxlyyshgLbS/7kZ/C+
         LMuRDn5iw7KDKC4vYdNOQNqNftbZBDjKamz3H5s4MSgZhkM/BUi6Oe9RB2X3XsIUwxgr
         2HNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716290860; x=1716895660;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xmc5d0J0AlvJbOD8urO/lbTRDEzhDbKc0+/1IT9h7NQ=;
        b=fB28383EXjy7sbTePRZX8GwMC8UAMH1NZCHCO9I+BZofl8trKnThFyp6YNRBjpPFKR
         5bGnkhI5AMHETi180Kl9i+LHYIkY2SCv+KFgebUVR1Gt+npYaiFxgoL5xCXQEPKMnQHb
         Cg+0BZ9Rk+ybA+V407DPTJaFFbaJB/s1eedLgc2Drm5cjiCV/TnhjeHiATxRRq1xBBj4
         zYQM18k0xPkJYsyA6tnbnnr+Pm74kkBCOkyVNNjARkhVMIlAHRIwksvjLv80H2ykAtpR
         AJP05/JkW0Ze8dK3SdRE5oG3ZcR62kSwGoux7Dv93P1tBiTPR0P4Z/FvIAFP0F6L2QNE
         VNFw==
X-Gm-Message-State: AOJu0Yw+gp31nVq4FJfI3qID93aU07jkekD5vp9P1VVe8eGot8FUyQmS
	2QwJQ7GWxykiVowgqxl50yTO8lWJJ3bMX7il35hYtNWDZFhCZnFso3f9TwvAdw==
X-Google-Smtp-Source: AGHT+IHfw9s0/d7+7zoLR1V4OxsIDjUKhEA1cYJ7LcK3DhbKmQFwf2kObKKIz3BSwWGxY63LY/B+oQ==
X-Received: by 2002:a05:6512:280c:b0:523:ef21:8e2c with SMTP id 2adb3069b0e04-523ef218e56mr8786116e87.51.1716290860276;
        Tue, 21 May 2024 04:27:40 -0700 (PDT)
Message-ID: <8f5a4a61-b985-4381-a53d-bea5b8429d75@suse.com>
Date: Tue, 21 May 2024 13:27:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/4] x86_64/uaccess: address violations of MISRA C
 Rule 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <b131427dc03c4b89141bab7648523b81bf5186a6.1715757982.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405151619200.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151619200.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 01:19, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>>
>> xlat_malloc_init is touched for consistency, despite the construct
>> being already deviated.
>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Tue May 21 11:28:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 11:28:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726864.1131260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NfE-0006Ab-H9; Tue, 21 May 2024 11:28:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726864.1131260; Tue, 21 May 2024 11:28:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9NfE-0006AM-E6; Tue, 21 May 2024 11:28:16 +0000
Received: by outflank-mailman (input) for mailman id 726864;
 Tue, 21 May 2024 11:28:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9NfD-0006A9-4U
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 11:28:15 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35937591-1765-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 13:28:13 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so738931866b.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 04:28:13 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c6b1sm1600780466b.57.2024.05.21.04.28.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 04:28:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35937591-1765-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716290893; x=1716895693; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JvNaK8MD3i0AfYF97GM7SS5tHhq3J9SrugfKg1Wn3dM=;
        b=a8Ihd99oBmjWTUsLa3Bq+JO1/rpA9JIWbMK5OA3/fmxFTYhUPpFcgZ1WOcLLOeDFD9
         PC8XJscmLmH1XX74N4oQ/FEW6PSKnoD3fkuuJCYMfIF7IJlhzAkHLd2mf017UUj7gOU6
         EsLelOBxWpv6FSct1ZfhznTVragXWZFqIEIOEml9VdWOzFne/aWiCjtklAU5NXZYCMt3
         ScvE+Sc6peF5XoBJyfQr+njDavIeSaiWarAA6hmXqC+UNnLqwUFJVW27ZJVNfyer+QJU
         Z+cVmHvfebBgXtJLNsqWlhP5pO2jkTk4sUygp6S+rdV8ZeDRQ/Vc+/U0Zbpb49HfW9mE
         wQdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716290893; x=1716895693;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JvNaK8MD3i0AfYF97GM7SS5tHhq3J9SrugfKg1Wn3dM=;
        b=dqnjXslmtMK3jCkitb3kXXmzvyvpk4WuqkTS8lgcuWF1Oz0gqVYw8t/b2ip7lHmpze
         E+NTiuhx7e5ufdcHr09Gju76ifJT1hEA75ys6N65Z2PmVrnSvCbpr3vQiWgI4fvWyrKp
         aHWKfgNpAIt+A4N41utUjCsfsp8dbLeGUrimp5yW1Cx2OmLcLyJykoV4WNW9VEPW9p6D
         UC29YsVkRVMfJnFGqMTWlkEAsCIp7o5QhtGtowjaw7eqhU/mBOs2DC5gEz2/LDtTWnKG
         dlWIVp5u1OPL7FWJOXnuCvSeJIe7AOGncVPSA8Msw8EYv1GPwFn/J8er/DvR7FHY9xYO
         k3SQ==
X-Gm-Message-State: AOJu0YypJv9XXvHaAxXc3maqMqF/nUqgakBuCiXwz5HTJN6RGK2qPw6V
	CoZu4OWgO1YqXThscJPKDyg3T+L+/U23xSCfg7PCZ9WZuApjOfeR817qAKj76g==
X-Google-Smtp-Source: AGHT+IGZtPQ06uGP7qQfEEIzoGQ7y6rDYNuaKr0uQs8NQVuiptnvunBDxKN2yiEI/3HBrBzeYQprcg==
X-Received: by 2002:a17:907:7d8e:b0:a60:c514:31d9 with SMTP id a640c23a62f3a-a60c514327bmr693315566b.6.1716290892690;
        Tue, 21 May 2024 04:28:12 -0700 (PDT)
Message-ID: <1dc3b47d-b1c5-4ca6-9dd5-097e5e873f9e@suse.com>
Date: Tue, 21 May 2024 13:28:12 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/4] x86_64/cpu_idle: address violations of MISRA C
 Rule 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1715757982.git.nicola.vetrini@bugseng.com>
 <6c84f6f0bf93a00ab1acda2d39d31b1f3d5bf8aa.1715757982.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2405151619410.2544314@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405151619410.2544314@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.05.2024 01:19, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Nicola Vetrini wrote:
>> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
>> of macro parameters shall be enclosed in parentheses". Therefore, some
>> macro definitions should gain additional parentheses to ensure that all
>> current and future users will be safe with respect to expansions that
>> can possibly alter the semantics of the passed-in macro parameter.
>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Tue May 21 12:30:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 12:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726883.1131273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OdE-0008Da-61; Tue, 21 May 2024 12:30:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726883.1131273; Tue, 21 May 2024 12:30:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OdE-0008DT-3U; Tue, 21 May 2024 12:30:16 +0000
Received: by outflank-mailman (input) for mailman id 726883;
 Tue, 21 May 2024 12:30:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9OdC-0008DN-IN
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 12:30:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9OdC-0003j9-7I; Tue, 21 May 2024 12:30:14 +0000
Received: from [15.248.2.238] (helo=[10.24.67.30])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9OdB-0004VN-WC; Tue, 21 May 2024 12:30:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=W3U6witmGjEe3b9StaN0T26uzhn81FhZugRqwJNnlyU=; b=gKV8aTNr/56t0OrLSIixd0s408
	HxH/nnfhYAa66Fk8tnad5q4pzLLmD4gkbZdMY3F7+FKbN4xLu6Q27gAXkgtvClW+d3eEjQoGR+cCj
	dNi+0PHIezwIHTowscuJy1XEvY/0b7TebkuaF3+DScU2H9pV2/brABitGFXpre8I3ouo=;
Message-ID: <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
Date: Tue, 21 May 2024 13:30:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-6-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240521043557.1580753-6-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 21/05/2024 05:35, Henry Wang wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
> 
> Currently, routing/removing physical interrupts are only allowed at
> the domain creation/destroy time. For use cases such as dynamic device
> tree overlay adding/removing, the routing/removing of physical IRQ to
> running domains should be allowed.
> 
> Removing the above-mentioned domain creation/dying check. Since this
> will introduce interrupt state unsync issues for cases when the
> interrupt is active or pending in the guest, therefore for these cases
> we simply reject the operation. Do it for both new and old vGIC
> implementations.
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v3:
> - Update in-code comments.
> - Correct the if conditions.
> - Add taking/releasing the vgic lock of the vcpu.
> v2:
> - Reject the case where the IRQ is active or pending in guest.
> ---
>   xen/arch/arm/gic-vgic.c  | 15 ++++++++++++---
>   xen/arch/arm/gic.c       | 15 ---------------
>   xen/arch/arm/vgic/vgic.c | 10 +++++++---
>   3 files changed, 19 insertions(+), 21 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index 56490dbc43..956c11ba13 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>   
>       /* We are taking to rank lock to prevent parallel connections. */
>       vgic_lock_rank(v_target, rank, flags);
> +    spin_lock(&v_target->arch.vgic.lock);

I know this is what Stefano suggested, but v_target would point to the 
current affinity whereas the interrupt may be pending/active on the 
"previous" vCPU. So it is a little unclear whether v_target is the 
correct lock. Do you have more pointer to show this is correct?

Also, while looking at the locking, I noticed that we are not doing 
anything with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem 
to assume that if the flag is set, then p->desc cannot be NULL.

Can we reach vgic_connect_hw_irq() with the flag set?

What about the other flags? Is this going to be a concern if we don't 
reset them?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 21 12:42:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 12:42:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726889.1131284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OpA-0001lF-6f; Tue, 21 May 2024 12:42:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726889.1131284; Tue, 21 May 2024 12:42:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OpA-0001l8-3h; Tue, 21 May 2024 12:42:36 +0000
Received: by outflank-mailman (input) for mailman id 726889;
 Tue, 21 May 2024 12:42:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Bu/9=MY=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s9Op8-0001l2-TI
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 12:42:34 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 973dc5cd-176f-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 14:42:33 +0200 (CEST)
Received: from SJ0PR05CA0180.namprd05.prod.outlook.com (2603:10b6:a03:339::35)
 by BL3PR12MB6524.namprd12.prod.outlook.com (2603:10b6:208:38c::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May
 2024 12:42:29 +0000
Received: from CO1PEPF000044F0.namprd05.prod.outlook.com
 (2603:10b6:a03:339:cafe::b5) by SJ0PR05CA0180.outlook.office365.com
 (2603:10b6:a03:339::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend
 Transport; Tue, 21 May 2024 12:42:28 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000044F0.mail.protection.outlook.com (10.167.241.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 12:42:28 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 07:42:27 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 07:42:27 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 07:42:26 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 973dc5cd-176f-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LoDQNKrZmS0HoXP8Mb1qiOu6WKB4Ilp93Y8Tjtwcjx+WlYhoE6TPOXDwXG5jYZ7asQMlXBwI2iAaL0eSRSyXghIm3G/t6twCY9W5rDKEImBVfrzzoOyI1uPIEL7SxQNEiG6bKp0XcAfpT4cEqAsodzZxNnN5KBvLGmTKiyWPv/Q3HQcj5VUKQNncthsL1bSW68CYT96AOsugQ37x3uVj3Wm3coJ3uKke1mlhMQRjRvn/vqvMHFz8tTsW6sk/r5C/RDO4y0aNaXF1tcaUkS8/VY4iui5oX7xJ6Kb7nOyORHX/F/aJ4C2PgbGHckw0J28jI3sjDM/AyZpzNwo47xzzTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=O7ay0SUHqpEYLSQP/3icBteOkCZ0riAwLWh2u6rvs2w=;
 b=CzamKhMxCIUA0sNQg05Vbcg16cNWeB5Pw10O+VpqsDa4Vh44OffJT2nXImE/byLzpd0FqWox3QjAdfgwh7Yy14guD8x2uvxys6pa7qas4EpAkrj+LBw3WgBqeIKFdiqRC03Pqaf6PhZjX2Va+IWTNjXjgcnsVzbL33L8ootNsX4OJjTAx+0xKzGEojPk3EQj6/JAwzA0Q04cBpTcUQesn76AViXQk+ZpY9J6UqwzrIRvaBFpr8USWn1gITpBBY+yU1bD07qxStQMr5ZY6NcfQqvocr4LBIjQsglno79THHsasJC5jhe8jfxK+AnCQcln41BszSearXgroxoYD5iucQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O7ay0SUHqpEYLSQP/3icBteOkCZ0riAwLWh2u6rvs2w=;
 b=krmGyWUWz5G1qhi5uF6B7relDSrTAonexH8FDU6akZy+nPbDIUHgyw4HIQx2MyAc2RrH9nX7/SsAAqd+5SUDmVKdc7LX9oHNWaROyna8A9ZCO0i51/xsup2Qt4VXAaQR+n9YEdrxMaP6//iSmKeYjfFT0jqjqICbnOcdZgFJ5u0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <8fe9c039-820f-451f-a680-c04f3c7e44cb@amd.com>
Date: Tue, 21 May 2024 08:42:28 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/8] tools/arm: Introduce the "nr_spis" xl config entry
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-5-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240521043557.1580753-5-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F0:EE_|BL3PR12MB6524:EE_
X-MS-Office365-Filtering-Correlation-Id: a329b301-3183-463a-46d9-08dc79937987
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?ZkRDblE4T0h4bHBiZStVOFNpcjMzMHNRZDVWMzB2bnQzOFQwZzRkdml2YXJR?=
 =?utf-8?B?OGxueUJ5NmptVFZKZ1ZYc3NRUmRVcTVtVTlWSmxNSE0vVFB1Z1RtaWY3cUlF?=
 =?utf-8?B?WEZGMWZkMzhUeUhzenYrL3lLUW1YRnRMUmhTTllTdEx1dHBtb0NRTit6dmx5?=
 =?utf-8?B?SU9JY2VnM3N3MUFNeUpaOVFnU3NKcEd3UjQ3NmJDanQ4UGwwR0diSGRsaXdU?=
 =?utf-8?B?ZzJ6L2s2dU56ZzRBQWNLT2trWmI4U2NMVUU5UkpCb0dQbGQvem14eUlGQmcv?=
 =?utf-8?B?eGxEQnp6d0lycDdBcDFnajBkdjg3ZlZYck5jYWdEcHhQR2ovY25uMW1ERWVp?=
 =?utf-8?B?bjdNcGtreWxicmJFVDZiQ3pta0lVSUp2T3h5SGVzOVdnU1dzWU5ZdklRMTdQ?=
 =?utf-8?B?dThJT3dtR2tQNkY1cEY4d0VVTGNkazlxaHQzMDNaaExacVJoVEpuZUNhZmda?=
 =?utf-8?B?RjRFaG1YdGFucm14S3N3T2orL0NDS012QXhsTFlFUExhUU1zTmVmRy9IZEN1?=
 =?utf-8?B?MGVjZzN2ejVyRGExSUdwZ3VvN01BQk1yTmc2N3FKbUU0emwvOTBMZzVOZ3Fv?=
 =?utf-8?B?clFIQkE4ZkVmazJBZTk0dmNmdmVueUxjQkZmSmVSSUVJamJkbjVEemxEQ0N6?=
 =?utf-8?B?dkZOM241Qm1Id1lFb1VQa05oTU82WURVOGdaM0U2NEliaU8yUS9xTFZDZ2Zw?=
 =?utf-8?B?NFVnZFlESDRsRGZjTm0yRTRid2x5MEVIY0k2ZVp1cXlpT3g1Ky9wK1Y4VWdE?=
 =?utf-8?B?c1BPdWxVdm4xNFJsRmFvL2kyN244M3lTYkkydUNIYk4xT3RLWXVrbm4xaWhG?=
 =?utf-8?B?RGhQWmMveWFwYkF1SG9EL25MZFJ1VDl3OWNkbmR4MTJyeDkyOHNSeXJGcXhn?=
 =?utf-8?B?OHB3K2dNRXZXUmk4NHNMOGlIMjJwcUcwS1ZVQk5vNE12Y1NuVHVDbGtRNks0?=
 =?utf-8?B?LzM4U1kvanZYcUlXSkIrSHo2c0krQUVxNGkxVW1yeWlxK3k5Tk1vMzgwQUNx?=
 =?utf-8?B?T3VKdGpqUmFwUlRzS1hKSC9lYmdrSVU3Z2dCeWVrYTliL1Y3dG9tQW1ZTDZM?=
 =?utf-8?B?eGtsMmVjZm9LWnBQVy95ekVFUytMVHU5ckYrZzhhamExUEdTR3AyR3l2Lzlk?=
 =?utf-8?B?YU1pNzhIOHg5UmpUcGlVaDl3b1hYUG9LV01MOFVzdFo0RTUzcGdUY0lCVE8r?=
 =?utf-8?B?bWhZK2dMMFdJQXY1Sy9OOVNWUkYrb1oyU0p2bVVCSG5SMk1jVVBBekU0aHoz?=
 =?utf-8?B?RnY3SjFVUnlsQ2tvekZyNGEvb2J1YkVJbE5oMklDc0FJNjVKNEFVc1dIVnU1?=
 =?utf-8?B?YjM2dUk1R1JMT0lFTmJxNFppR2pLYnlaNTFLOWk1VkdacUVNZjV6eWF5QU81?=
 =?utf-8?B?UElsbmRpaGxOMjlCSTJvMG9WUncvRG5DWnZ6TWVCaGR5ZEZrU1FLbU5BeE1y?=
 =?utf-8?B?Wmt4c1E3WUF2SVBDeHcrTjVvUGM4MnJLSUtYcmN6SC9aZnpSY3ByY3hzT0U5?=
 =?utf-8?B?eERFcU11UjZ2SUgreStFVmVqQzhMUzhlSXBlQlJBYUJYQkRrbitTWXNXYUVW?=
 =?utf-8?B?UFV5dDdiZVB1VzI2d0l4Z2F1WitGbFp0NU92RHlEaEN5WnY5bHZWK1EwZkZy?=
 =?utf-8?B?cXBFZHlKNjdJSHpXejgwamVaTFRrRHhESDc5WDRzOEZZV3lYenhWRGVsZ2Fh?=
 =?utf-8?B?UDVGY3ZlZEJJZm1XVk5pNXdkNXA5Zno4YnlJN3ptbUh4bXRIQ3hHNm55dllO?=
 =?utf-8?B?aTEzdXJiKzNkeDZ1WXZzWUgyalFlMTltb1B1TmUvMDVKS3g0K2ZkUlFCVnpG?=
 =?utf-8?Q?2H4HlPe547Zwu2F7T8kKFL3F1kKEjQYL9xe3M=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 12:42:28.3798
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a329b301-3183-463a-46d9-08dc79937987
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6524

On 2024-05-21 00:35, Henry Wang wrote:
> Currently, the number of SPIs allocated to the domain is only
> configurable for Dom0less DomUs. Xen domains are supposed to be
> platform agnostics and therefore the numbers of SPIs for libxl
> guests should not be based on the hardware.
> 
> Introduce a new xl config entry for Arm to provide a method for
> user to decide the number of SPIs. This would help to avoid
> bumping the `config->arch.nr_spis` in libxl everytime there is a
> new platform with increased SPI numbers.
> 
> Update the doc and the golang bindings accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Tue May 21 12:53:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 12:53:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726895.1131293 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OzG-0003a8-3K; Tue, 21 May 2024 12:53:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726895.1131293; Tue, 21 May 2024 12:53:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9OzG-0003a1-0m; Tue, 21 May 2024 12:53:02 +0000
Received: by outflank-mailman (input) for mailman id 726895;
 Tue, 21 May 2024 12:53:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9OzF-0003Zr-4F; Tue, 21 May 2024 12:53:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9OzE-000471-Vg; Tue, 21 May 2024 12:53:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9OzE-0000xx-JI; Tue, 21 May 2024 12:53:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9OzE-0003eS-Ir; Tue, 21 May 2024 12:53:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SoNKZYf/aVoaqTIWYq++0S5YvWTtSESAnUsZPmu8ezw=; b=Fg9Y+M3LJgCQGJDIn78/51ozJF
	wUkhyqlKGgGMtBKiZwohCz6zqKkq9blWN1tNhmi+czR9YPxe42i/GINi/I850SuGvERu5b6+kPBHB
	kr5PZzLMBMxvOFaqeltr5HxdCefRhLjokGt41X81ZJzmfKsZoSq+6cUKnEG8BbYFVC40=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186052-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186052: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
X-Osstest-Versions-That:
    linux=eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 12:53:00 +0000

flight 186052 linux-linus real [real]
flight 186061 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186052/
http://logs.test-lab.xenproject.org/osstest/logs/186061/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-thunderx  8 xen-boot            fail pass in 186061-retest
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 186061-retest

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx 15 migrate-support-check fail in 186061 never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check fail in 186061 never pass
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 186061 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 186061 never pass
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186044
 test-armhf-armhf-libvirt      8 xen-boot                     fail  like 186044
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186047
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186047
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186047
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 186047
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186047
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186047
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
baseline version:
 linux                eb6a9339efeb6f3d2b5c86fdf2382cdc293eca2c

Last test of basis   186047  2024-05-20 15:13:33 Z    0 days
Testing same since   186052  2024-05-21 01:42:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Darrick J. Wong" <djwong@kernel.org>
  Abhinav Jain <jain.abhinav177@gmail.com>
  Alex Williamson <alex.williamson@redhat.com>
  Alexander Lobakin <aleksander.lobakin@intel.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Allison Henderson <allison.henderson@oracle.com>
  Amir Goldstein <amir73il@gmail.com>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Geissler <geissonator@yahoo.com>
  Andrew Jeffery <andrew@codeconstruct.com.au>
  André Draszik <andre.draszik@linaro.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Andy Shevchenko <andy.shevchenko@gmail.com>
  Animesh Agarwal <animeshagarwal28@gmail.com>
  Anjelique Melendez <quic_amelende@quicinc.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Arnd Bergmann <arnd@kernel.org>
  Baruch Siach <baruch@tkos.co.il>
  Bryan O'Donoghue <bryan.odonoghue@linaro.org>
  Chandan Babu R <chandanbabu@kernel.org>
  Chao Yu <chao@kernel.org>
  Chen Ni <nichen@iscas.ac.cn>
  Chen-Yu Tsai <wens@csie.org>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Daeho Jeong <daehojeong@google.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Danila Tikhonov <danila@jiaxyga.com>
  Darrick J. Wong <djwong@kernel.org>
  Dave Chinner <dchinner@redhat.com>
  David Collins <quic_collinsd@quicinc.com>
  David Ober <dober@lenovo.com>
  Dawei Li <set_pte_at@outlook.com>
  Delphine CC Chiu <Delphine_CC_Chiu@wiwynn.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Eddie James <eajames@linux.ibm.com>
  Emil Renner Berthing <emil.renner.berthing@canonical.com>
  Eng Lee Teh <englee.teh@starfivetech.com>
  Eric Dumazet <edumazet@google.com>
  Eric Sandeen <sandeen@redhat.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  foryun.ma <foryun.ma@jaguarmicro.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guenter Roeck <linux@roeck-us.net>
  Gustavo A. R. Silva <gustavoars@kernel.org>
  Hamish Martin <hamish.martin@alliedtelesis.co.nz>
  Hannah Peuckmann <hannah.peuckmann@canonical.com>
  Hans Hu <hanshu-oc@zhaoxin.com>
  Heiko Stuebner <heiko@sntech.de>
  Heiner Kallweit <hkallweit1@gmail.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Herman van Hazendonk <github.com@herrie.org>
  Jaegeuk Kim <jaegeuk@kernel.org>
  Jan Kara <jack@suse.cz>
  Jarkko Nikula <jarkko.nikula@linux.intel.com>
  Jean Delvare <jdelvare@suse.de>
  Ji Sheng Teoh <jisheng.teoh@starfivetech.com>
  Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
  Jiaxun Yang <jiaxun.yang@flygoat.com>
  Jisheng Zhang <jszhang@kernel.org>
  Joel Stanley <joel@jms.id.au>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Judith Mendez <jm@ti.com>
  Julia Lawall <Julia.Lawall@inria.fr>
  Justin Stitt <justinstitt@google.com>
  Justin Swartz <justin.swartz@risingedge.co.za>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kelly Hung <ppighouse@gmail.com>
  Kelvin Zhang <kelvin.zhang@amlogic.com>
  Krzystof Kozlowski <krzystof.kozlowski@linaro.org>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Liao Yuanhong <liaoyuanhong@vivo.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Luca Weiss <luca@z3ntu.xyz>
  Lukas Bulwahn <lbulwahn@redhat.com>
  Lukas Bulwahn <lukas.bulwahn@redhat.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Pearson <mpearson-lenovo@squebb.ca>
  Mark Tinguely <mark.tinguely@oracle.com>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Matthijs Kooijman <matthijs@stdin.nl>
  Matti Vaittinen <mazziesaccount@gmail.com>
  Michael Kelley <mhklinux@outlook.com>
  Michal Simek <michal.simek@amd.com>
  Nathan Chancellor <nathan@kernel.org>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nikita Kiryushin <kiryushin@ancud.ru>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Ninad Palsule <ninad@linux.ibm.com>
  Nipun Gupta <nipun.gupta@amd.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Paul Cercueil <paul@crapouillou.net>
  Peng Fan <peng.fan@nxp.com>
  Peter Korsgaard <peter@korsgaard.com>
  Peter Rosin <peda@axentia.se>
  Peter Yin <peteryin.openbmc@gmail.com>
  Piyush Malgujar <pmalgujar@marvell.com>
  Rafał Miłecki <rafal@milecki.pl>
  Reinette Chatre <reinette.chatre@intel.com>
  Renze Nicolai <renze@rnplus.nl>
  Ricardo Ribalda <ribalda@chromium.org>
  Ritesh Harjani (IBM) <ritesh.list@gmail.com>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Robin Murphy <robin.murphy@arm.com>
  Roman Smirnov <r.smirnov@omp.ru>
  Sai Pavan Boddu <sai.pavan.boddu@amd.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Serge Semin <fancer.lancer@gmail.com>
  Shanth Murthy <shanth.murthy@intel.com>
  Shengyu Qu <wiagn233@outlook.com>
  Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Siarhei Volkau <lis8215@gmail.com>
  Songyang Li <leesongyang@outlook.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Wahren <wahrenst@gmx.net>
  Steven Price <steven.price@arm.com>
  Suneel Garapati <sgarapati@marvell.com>
  Tao Ren <rentao.bupt@gmail.com>
  Tengfei Fan <quic_tengfan@quicinc.com>
  Thomas Bogendoerfer <tsbogend@alpha.franken.de>
  Thomas Bonnefille <thomas.bonnefille@bootlin.com>
  Thomas Richard <thomas.richard@bootlin.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Thorsten Blum <thorsten.blum@toblux.com>
  Wenjie Qi <qwjhust@gmail.com>
  Wim Van Sebroeck <wim@linux-watchdog.org>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Wu Bo <bo.wu@vivo.com>
  Xianwei Zhao <xianwei.zhao@amlogic.com>
  Xin Zeng <xin.zeng@intel.com>
  Yahui Cao <yahui.cao@intel.com>
  Yang Chen <yangchen.openbmc@gmail.com>
  Ye Bin <yebin10@huawei.com>
  Yeongjin Gil <youngjin.gil@samsung.com>
  Yi Zhang <yi.zhang@redhat.com>
  Yifan Zhao <zhaoyifan@sjtu.edu.cn>
  Yongzhen Zhang <zhangyongzhen@kylinos.cn>
  Yunlei He <heyunlei@oppo.com>
  Yury Norov <yury.norov@gmail.com>
  Zelong Dong <zelong.dong@amlogic.com>
  Zev Weiss <zev@bewilderbeest.net>
  Zhang Yi <yi.zhang@huawei.com>
  Zhiguo Niu <zhiguo.niu@unisoc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   eb6a9339efeb..8f6a15f095a6  8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 21 13:34:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 13:34:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726907.1131304 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9PdO-0000aw-8Y; Tue, 21 May 2024 13:34:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726907.1131304; Tue, 21 May 2024 13:34:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9PdO-0000ap-55; Tue, 21 May 2024 13:34:30 +0000
Received: by outflank-mailman (input) for mailman id 726907;
 Tue, 21 May 2024 13:34:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Bu/9=MY=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s9PdM-0000aj-F9
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 13:34:28 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d7bca5c2-1776-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 15:34:26 +0200 (CEST)
Received: from MN2PR14CA0007.namprd14.prod.outlook.com (2603:10b6:208:23e::12)
 by CY8PR12MB7218.namprd12.prod.outlook.com (2603:10b6:930:5a::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Tue, 21 May
 2024 13:34:22 +0000
Received: from BL6PEPF00022572.namprd02.prod.outlook.com
 (2603:10b6:208:23e:cafe::7a) by MN2PR14CA0007.outlook.office365.com
 (2603:10b6:208:23e::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Tue, 21 May 2024 13:34:22 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF00022572.mail.protection.outlook.com (10.167.249.40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 13:34:21 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 08:34:21 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 08:34:20 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 08:34:20 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7bca5c2-1776-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Zg6KaBJ3kba66vWJjodjFX6d3bLJE+zMRv4ulhiim2GRDGVg9Bg4qWykxT6T/x4K+qHYk1e/78MgrCSYA7Y/nNyDFAOfHmHg6aOizs8nhjMMyfdwzKr+bjALDgAneHzYwD0eqhnlAe4IQXk+WBhx9bXK1e/we1ox1vV384E1eweqJ7ogK3WdMuz7+LNCf5vu0o5iL6tsrCoAyz+RHVt3vbrQ/YBmKfLpywbGgpgCR8zMR4hChyc/EPI0t0jdpAbF5dJnGWVFEamxkhoKf66C8QpykXCEkHUXHMGRAaMTJDmztnKBFmslMNE6qMMIR68LRFhjXpy1vSEz6zpgW2RvJQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OGGqlmKWPzIrMlF+uoMxnufPGILvr9T9DtT3ZXKA5Yg=;
 b=E2/jabjXbvbvkTFculmgj6BMVykm6rjrxJFjj5UdxNI8Khf6OnF1amEvGlNpsAmLxRJjaQdIWtqMP5lUyA+KrEGZIcfmBraBba929kGERZfcsYFoB8v7ZpWLoDw3LS8GGWpSLk5theZVnl7uUKvrI3HSsmLm/eou1VS6yOUWXT/Q3GrS1TEUaNQEi2+odDSPunP2tMvgSD/s30ROP3kdM4rPnUQiwTYIbgtnJACCCdXxksoWEDufppLr4w2UUVt0mUxmrpgWiFSx8Slvwn9elNXkHtdq+Q2mn3mnsMxiOEK+4O1z1KiSjLdlgvOpO1YyCPEtWBdyQRLR8LNLlr0tkg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OGGqlmKWPzIrMlF+uoMxnufPGILvr9T9DtT3ZXKA5Yg=;
 b=hcsMFFUmZqZqPYNEENnWfzrMDweiBura7TMWlLK8vZY9myCpy6x+iF/RHDM0z24BpP6+JPFmTMUEydzjEArOCJuEoG6V5SFwY91ai70xuzi6Y+kADClNgrkF4VuC3JCCd7Eiq7EkZXO/ggJUHLA6SRhbLAfnTXu0Ppi/QQE4mhM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <ff8c8936-8150-45dd-b6b3-0f5fef5deec1@amd.com>
Date: Tue, 21 May 2024 09:34:21 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 7/8] tools: Introduce the "xl dt-overlay
 {attach,detach}" commands
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-8-xin.wang2@amd.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <20240521043557.1580753-8-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF00022572:EE_|CY8PR12MB7218:EE_
X-MS-Office365-Filtering-Correlation-Id: 415d1c85-dcad-4165-85e7-08dc799ab901
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?RGtvVXM3N3NFV3NOY3c5aXFzUEFqZUpQeFM3bHlpS0p1WTVkQmZLcnlOTzdv?=
 =?utf-8?B?amRCWnQ4bVNjb1JwV3pZUzdEVTdIMThzR3g1S3ptM0ZjNkMxdTZXOWovcmFi?=
 =?utf-8?B?UG5iaFFMTlphY2ZBQTR1a0RTajBGRzU0NkpEVzlrOFhHVDFhRll2OHIxS3Zr?=
 =?utf-8?B?b2JRVHRvVmZKOEZSUitJa0JaVC9OMm84SXk4dm85bWVvdnNKTUU2YnNqcUNu?=
 =?utf-8?B?TE9VY3VFdEt0U0ZtTDlrSWQ0K0hXU0VMK3FWRU14RS9rWi9Tb0lFQlN0MlFC?=
 =?utf-8?B?Mk91NXJlQ3A4K1h0cHdHOFZlSDVHZHcxYzJYU0FiQXNyYTFsczA0UjQ3L2lh?=
 =?utf-8?B?VVBGbDRBOUZjeEcxVkIwSzVXcVRVM0dyZGZSTjMxREFKL1JpQ3U3VlFPcitr?=
 =?utf-8?B?Skx4Q1B3eU9oOUF1Qjh5a1EzZnJPM0pYSUlkc1A4K2VJWEV6SFVwbmx0RnBL?=
 =?utf-8?B?WVM3TnNwYUkrK2g4U0ovdWQvM0ZoOEhyYSs4a0tCSHpubEV0SWpoa1FuMllt?=
 =?utf-8?B?VnU5ZnhTaVFwcUk2ZTRDaUJ3WDJ5bWRuUWNZMk9WNHdoQWROb3o3MmMyaWFD?=
 =?utf-8?B?cFdDL3AweWJFZFhxaG0rVGllZ1p6Z2F2SUI3OHZrRmt6aUwzYjJESStHK2d1?=
 =?utf-8?B?TzFTNVVaWnlteHR6dDVBWGpvbVNKdWVoZXRnekpnTWYrQVZaT21aSG42dndy?=
 =?utf-8?B?YkxEN2UwVmNiNTVjdWN1T1VIZy9XN3NIanJMZ0YxajBKT2pVNSt6dzFqYVpn?=
 =?utf-8?B?Qno4VHNhU3F3OENGb0l3YlpVU3lzR0gyNVlQT0tYMGIwVnhXZnZDSUN4RHor?=
 =?utf-8?B?MnhhL0F0dkUyTUFLSzR3SGlSNVk5ZDV1enFUanpLdmtSSmxPUFhTK3o3WWJn?=
 =?utf-8?B?aUpMRW0vUDU3RCt0M014TFptN0RUZzAvWUVLc0YybUxjcElnYTdEWDd3dXM0?=
 =?utf-8?B?OGlYQU96Vml5TXB0c1d6R0lkblRobjByNW94TkJoaS96ZVZyMGFoVTVtUC90?=
 =?utf-8?B?ejAxUml0YUgxeVlRVVl3eGYxbEdkMG9oc3pIbVFRRWJjNUxzYmUzSldCQzQv?=
 =?utf-8?B?ZG54ZXZtNXFQVllIb0VvaG9HTFdkd0xxUDRRUFJFazJCRHVFbGlyNC9SSG5x?=
 =?utf-8?B?S3dlcWtCRkhubC9Ka2c2ZGYxS2xaSitGaVUvUzJlM0RxWmplVy92dHkrK2sw?=
 =?utf-8?B?eUtZaWp3SW56ZVUybWtCZXZFQjFXUXp4d1BsUzNaellQdWYreHcxbWJhVVYv?=
 =?utf-8?B?S0NxYkhHaDNJVjRnV1M0Z3VGeTNFTGFTS29wWkZaak1vdVZlQXd4a3lQZFNa?=
 =?utf-8?B?UlBVekNja3hyRHA3VFFCS1k4bytoNjZnSVNNUGRYM2dLMGVxSUg0YVRxajdD?=
 =?utf-8?B?MDFkQWNURHN1TjA0dE9sZW5OUENZWGlJd2czWWFVWEowOTA2SFZwaUhxcXZp?=
 =?utf-8?B?QUFZRFM0YTNOYi9YTFBjS2hsKys5K05ZNk9nTkliU21XaHJKMlFIbHFoRUhs?=
 =?utf-8?B?a21xTFg5Ni9JS1JVcTFBT1lWVlZUemlHZFp1ejdUSlMrZDhoQVlYa1FsV0RE?=
 =?utf-8?B?RDhBbTN2OVo4MlpVeVBZK0hyNGZGM3NPTFBHS1lyQ09oR0tza0NjRVZFQXdL?=
 =?utf-8?B?U1EyQWhOb25DQkNlMTJrK1NzQ2tXQzFVSHo4U2pJN0FUWUluMG43NVRqcDBY?=
 =?utf-8?B?MVg2RTUwYzhxeDRzbVh5WVBXb1A4bXYwYmVmNXFPK1Z3bDJaakdjL3ZtUmh2?=
 =?utf-8?B?RlluTy9DRDFnV2c1emNrM1oweXNnYmgyTTJTUC9YYkxTMlA2RzdwWjMvbWdr?=
 =?utf-8?B?UGRTb3dRVk5rYk9QWndwUlNvSUhYOEtNTmNkbjA1SXgvL1ZXdUdhV0ZvV2lq?=
 =?utf-8?Q?CX0+VCkyM5Pew?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 13:34:21.5108
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 415d1c85-dcad-4165-85e7-08dc799ab901
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF00022572.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7218

On 2024-05-21 00:35, Henry Wang wrote:
> With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> attach/detach devices from the provided DT overlay to domains.
> Support this by introducing a new set of "xl dt-overlay" commands and
> related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
> rework the command option parsing logic.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Tue May 21 13:45:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 13:45:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726916.1131314 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Po3-0002Jh-7l; Tue, 21 May 2024 13:45:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726916.1131314; Tue, 21 May 2024 13:45:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Po3-0002Ja-3k; Tue, 21 May 2024 13:45:31 +0000
Received: by outflank-mailman (input) for mailman id 726916;
 Tue, 21 May 2024 13:45:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9Po1-0002JU-7J
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 13:45:29 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9Po0-0004yH-Pq; Tue, 21 May 2024 13:45:28 +0000
Received: from [15.248.2.238] (helo=[10.24.67.30])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9Po0-0000Z2-Hh; Tue, 21 May 2024 13:45:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=SfXYgXJFvjs2JIPRGPh2HOmFDfPwtOxBmItpoSJCn2A=; b=5z7wz0eIiQ7hP8PE2ngkis5r/t
	xpB6JklFL5qGRiDLAN8sr+l+YH7p8r5eIhEm3tST8IPK/UlLstxneNDVPaA5d9vboEFLr6e92g4Hg
	4CZGsdRQj4S7a3aDyXAtB4COPG88zeepna2MzEh51eIkDNrx7+GUEZV7s2T18v92p36M=;
Message-ID: <8fa02e06-d8dc-4e73-a58e-e4d84b090ea8@xen.org>
Date: Tue, 21 May 2024 14:45:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v15 1/5] arm/vpci: honor access size when returning an
 error
Content-Language: en-GB
To: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
 <20240517170619.45088-2-stewart.hildebrand@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240517170619.45088-2-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stewart,

On 17/05/2024 18:06, Stewart Hildebrand wrote:
> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> 
> Guest can try to read config space using different access sizes: 8,
> 16, 32, 64 bits. We need to take this into account when we are
> returning an error back to MMIO handler, otherwise it is possible to
> provide more data than requested: i.e. guest issues LDRB instruction
> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
> register.
> 
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 21 13:53:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 13:53:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726923.1131323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9PvQ-00046M-Te; Tue, 21 May 2024 13:53:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726923.1131323; Tue, 21 May 2024 13:53:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9PvQ-00046F-R5; Tue, 21 May 2024 13:53:08 +0000
Received: by outflank-mailman (input) for mailman id 726923;
 Tue, 21 May 2024 13:53:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9PvP-000465-KA; Tue, 21 May 2024 13:53:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9PvP-00056c-Hl; Tue, 21 May 2024 13:53:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9PvP-0004f3-6W; Tue, 21 May 2024 13:53:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9PvP-0007UU-5u; Tue, 21 May 2024 13:53:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wtokMn8vydX/vohbsLEsxtnXWk5QPVRyKLMDWm7XPLE=; b=Z1lCKt2vjThh4Ghhjaw71UVMKr
	nIObaSwOabKNFiEX/Sx+JAmIXQMxXHwFDaaHXOquEK+fvwgbj4AhCk6tyPhUqejjVZJbUWVy/BO0p
	EVNZ7pcHum+ZrbAFlYQn5OQmr9PsdRZqOAQKc6nxk/d1jn++69N83VIag7hxTWgi1RSw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186057-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186057: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=7c8e606b64c73ca56d7134cb16d01257f39c53ef
X-Osstest-Versions-That:
    libvirt=34f52aec286af47066484e8a96ecba0ef8e79451
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 13:53:07 +0000

flight 186057 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186057/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186025
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              7c8e606b64c73ca56d7134cb16d01257f39c53ef
baseline version:
 libvirt              34f52aec286af47066484e8a96ecba0ef8e79451

Last test of basis   186025  2024-05-17 04:20:41 Z    4 days
Testing same since   186057  2024-05-21 04:18:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Göran Uddeborg <goeran@uddeborg.se>
  Jonathon Jongsma <jjongsma@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   34f52aec28..7c8e606b64  7c8e606b64c73ca56d7134cb16d01257f39c53ef -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 21 14:21:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 14:21:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726967.1131354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9QMi-0008Sn-8N; Tue, 21 May 2024 14:21:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726967.1131354; Tue, 21 May 2024 14:21:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9QMi-0008Sg-4d; Tue, 21 May 2024 14:21:20 +0000
Received: by outflank-mailman (input) for mailman id 726967;
 Tue, 21 May 2024 14:21:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9QMh-0008SW-5z; Tue, 21 May 2024 14:21:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9QMh-0005eB-2o; Tue, 21 May 2024 14:21:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9QMg-0005nR-Q3; Tue, 21 May 2024 14:21:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9QMg-0001l3-Pd; Tue, 21 May 2024 14:21:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aBQkd82Oa3w1xN+2ck0MGNPk0xaaimt53A4Rl2eGKzw=; b=Y8Y7mGYCVSO6SYmwXXpua9MAth
	Am/+a+ixuPQ9KcIBq0++hskHMDXj+dWV3fkGwLS8DMvO0dH+OkAf/9m/lIDcUucbRvKdUGKzF8TqQ
	+gaWCwiGj+ldOIgVhoEZKL576qn3p+eOvwuqkGvv7kY/U3l5sggk0MEZBkZe4VZIyS/4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186062-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186062: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=24dbf5bd03ff11dbcebb4d15e34fbba8eb34936b
X-Osstest-Versions-That:
    xen=9c5444b01ad51369bc09197a442a93d87b4b76f2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 14:21:18 +0000

flight 186062 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186062/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  24dbf5bd03ff11dbcebb4d15e34fbba8eb34936b
baseline version:
 xen                  9c5444b01ad51369bc09197a442a93d87b4b76f2

Last test of basis   186056  2024-05-21 04:09:17 Z    0 days
Testing same since   186062  2024-05-21 10:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9c5444b01a..24dbf5bd03  24dbf5bd03ff11dbcebb4d15e34fbba8eb34936b -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 21 15:17:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 15:17:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726977.1131364 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9REe-0006Pf-4h; Tue, 21 May 2024 15:17:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726977.1131364; Tue, 21 May 2024 15:17:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9REe-0006PP-1e; Tue, 21 May 2024 15:17:04 +0000
Received: by outflank-mailman (input) for mailman id 726977;
 Tue, 21 May 2024 15:17:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TLq1=MY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9REd-0006OJ-6z
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 15:17:03 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b6181d3-1785-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 17:17:01 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5dcb5a0db4so387913966b.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 08:16:59 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17cc5sm1616303866b.205.2024.05.21.08.16.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 08:16:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b6181d3-1785-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716304619; x=1716909419; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aqDWoDoB9/P1jqXa+vy9bE5aV3gSEt16LrMoswx/xD0=;
        b=D1h+O3pNkPh1aMCRDCgwO2dmzCiixb/zvnF3K4HRVSVYWQSQQ4nUeQutnQm4YWg8YO
         AzybpYtFBJQu/ruvUdgOdcgxkJscmEutwJkiwjGEnol34XzBrzxUTOzNVOyGQdzhfYLH
         DcDGo8zZeYVfeVoGzlC94etu1SfcvBwROFFaYpf93nl0CP3bheZlB/GXibOXkXUGV7Ip
         w91Hp2gZtHxiUOlQhf2FISH0GVHjTqDzX6OhizPrh2LsJEIXEo4KPG7W4PKve5P2NZxQ
         BkHyaGg7oZydcQ7aXNbqj2keDeRpl8lBqgpcwxsWo/R6GJEoDOZQYcQVuyEk76CjUUg8
         vlkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716304619; x=1716909419;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aqDWoDoB9/P1jqXa+vy9bE5aV3gSEt16LrMoswx/xD0=;
        b=SZyDubLU9NIkbdGD6q0BnRADgwp+5iIaqbGf7aNq8kEVIYnnEgyVl0aSgEru3a2G/c
         vu9eqLiod9CVlSU5ulRuqoOFklqmJuL9FefBsBRknHIHIWHtNHk9qA2cg02HcLC/wwFL
         BnM5zIrPahZMKfwVz2sP2+18AjK4YstRW/4JKFDYwPs0Jbeyh7VBqatuLygg44RzCJMr
         FiKNJ/Ek4Y/Bmriod1VurSjqzeIj1k/jLeq6kMZ3p8kQDhni4qQp7VEm8RRpcY+Nx91d
         WcSRbtyGuyZiBWYSIm3mgpMxqOHDFNRgizo/DvvVEOeI6IHWXB749PGON9lWLaTBFkUN
         6M8A==
X-Forwarded-Encrypted: i=1; AJvYcCUJr/XVazAyQLB14Drn3H9R500hXBF9rCyCKcVqVVSz7iDHb4/1jrg7rc/oZuDWkUcENCks+C5ahT3uY+Z0U6oPrJjsnQKacetyKZea2ZU=
X-Gm-Message-State: AOJu0YzgIoR+MKn2tFRuhCb/slescrxDi4lg7aYXVVimrGjiWyFCbchX
	7OZxf8ktVB/8xZNBL8e5t3nF1F7Ela2VS74z5EgzbK5coKHpeARglrW2cAjJnRfD7jNNTFYpfNk
	=
X-Google-Smtp-Source: AGHT+IEwq48Jgw2CfTXIM1p6tKBcXU8w8YTVw9PxOxzR2rgCaB/xgaMzW6uokku+ziJf88PfZ7VWhw==
X-Received: by 2002:a17:906:97c8:b0:a5a:a2b6:ba8b with SMTP id a640c23a62f3a-a5aa2b6bf6fmr1182722466b.0.1716304619365;
        Tue, 21 May 2024 08:16:59 -0700 (PDT)
Message-ID: <245e535a-5b23-4d3e-83e5-dc797068652c@suse.com>
Date: Tue, 21 May 2024 17:16:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
> --- a/xen/arch/x86/include/asm/mm.h
> +++ b/xen/arch/x86/include/asm/mm.h
> @@ -522,9 +522,27 @@ extern struct rangeset *mmio_ro_ranges;
>  void memguard_guard_stack(void *p);
>  void memguard_unguard_stack(void *p);
>  
> +/*
> + * Add more precise r/o marking for a MMIO page. Range specified here
> + * will still be R/O, but the rest of the page (not marked as R/O via another
> + * call) will have writes passed through.
> + * The start address and the size must be aligned to SUBPAGE_MMIO_RO_ALIGN.
> + *
> + * This API cannot be used for overlapping ranges, nor for pages already added
> + * to mmio_ro_ranges separately.
> + *
> + * Return values:
> + *  - negative: error
> + *  - 0: success
> + */
> +#define SUBPAGE_MMIO_RO_ALIGN 8

This isn't just alignment, but also (and perhaps more importantly) granularity.
I think the name wants to express this.

> @@ -4910,6 +4921,260 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      return rc;
>  }
>  
> +/*
> + * Mark part of the page as R/O.
> + * Returns:
> + * - 0 on success - first range in the page
> + * - 1 on success - subsequent range in the page
> + * - <0 on error
> + *
> + * This needs subpage_ro_lock already taken */

Nit: Comment style (full stop and */ on its own line).

> +static int __init subpage_mmio_ro_add_page(
> +    mfn_t mfn, unsigned int offset_s, unsigned int offset_e)
> +{
> +    struct subpage_ro_range *entry = NULL, *iter;
> +    unsigned int i;
> +
> +    list_for_each_entry(iter, &subpage_ro_ranges, list)
> +    {
> +        if ( mfn_eq(iter->mfn, mfn) )
> +        {
> +            entry = iter;
> +            break;
> +        }
> +    }
> +    if ( !entry )
> +    {
> +        /* iter == NULL marks it was a newly allocated entry */
> +        iter = NULL;
> +        entry = xzalloc(struct subpage_ro_range);
> +        if ( !entry )
> +            return -ENOMEM;
> +        entry->mfn = mfn;
> +    }
> +
> +    for ( i = offset_s; i <= offset_e; i += SUBPAGE_MMIO_RO_ALIGN )
> +    {
> +        int oldbit = __test_and_set_bit(i / SUBPAGE_MMIO_RO_ALIGN,
> +                                        entry->ro_qwords);

Why int, not bool?

> +        ASSERT(!oldbit);
> +    }
> +
> +    if ( !iter )
> +        list_add(&entry->list, &subpage_ro_ranges);
> +
> +    return iter ? 1 : 0;
> +}
> +
> +/* This needs subpage_ro_lock already taken */
> +static void __init subpage_mmio_ro_remove_page(
> +    mfn_t mfn,
> +    int offset_s,
> +    int offset_e)

Can either of these be negative? The more that ...

> +{
> +    struct subpage_ro_range *entry = NULL, *iter;
> +    unsigned int i;

... this is used ...

> +    list_for_each_entry(iter, &subpage_ro_ranges, list)
> +    {
> +        if ( mfn_eq(iter->mfn, mfn) )
> +        {
> +            entry = iter;
> +            break;
> +        }
> +    }
> +    if ( !entry )
> +        return;
> +
> +    for ( i = offset_s; i <= offset_e; i += SUBPAGE_MMIO_RO_ALIGN )

... with both of them?

> +        __clear_bit(i / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords);
> +
> +    if ( !bitmap_empty(entry->ro_qwords, PAGE_SIZE / SUBPAGE_MMIO_RO_ALIGN) )
> +        return;
> +
> +    list_del(&entry->list);
> +    if ( entry->mapped )
> +        iounmap(entry->mapped);
> +    xfree(entry);
> +}
> +
> +int __init subpage_mmio_ro_add(
> +    paddr_t start,
> +    size_t size)
> +{
> +    mfn_t mfn_start = maddr_to_mfn(start);
> +    paddr_t end = start + size - 1;
> +    mfn_t mfn_end = maddr_to_mfn(end);
> +    unsigned int offset_end = 0;
> +    int rc;
> +    bool subpage_start, subpage_end;
> +
> +    ASSERT(IS_ALIGNED(start, SUBPAGE_MMIO_RO_ALIGN));
> +    ASSERT(IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN));
> +    if ( !IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN) )
> +        size = ROUNDUP(size, SUBPAGE_MMIO_RO_ALIGN);
> +
> +    if ( !size )
> +        return 0;
> +
> +    if ( mfn_eq(mfn_start, mfn_end) )
> +    {
> +        /* Both starting and ending parts handled at once */
> +        subpage_start = PAGE_OFFSET(start) || PAGE_OFFSET(end) != PAGE_SIZE - 1;
> +        subpage_end = false;
> +    }
> +    else
> +    {
> +        subpage_start = PAGE_OFFSET(start);
> +        subpage_end = PAGE_OFFSET(end) != PAGE_SIZE - 1;
> +    }
> +
> +    spin_lock(&subpage_ro_lock);
> +
> +    if ( subpage_start )
> +    {
> +        offset_end = mfn_eq(mfn_start, mfn_end) ?
> +                     PAGE_OFFSET(end) :
> +                     (PAGE_SIZE - 1);
> +        rc = subpage_mmio_ro_add_page(mfn_start,
> +                                      PAGE_OFFSET(start),
> +                                      offset_end);
> +        if ( rc < 0 )
> +            goto err_unlock;
> +        /* Check if not marking R/W part of a page intended to be fully R/O */
> +        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
> +                                                  mfn_x(mfn_start)));
> +    }
> +
> +    if ( subpage_end )
> +    {
> +        rc = subpage_mmio_ro_add_page(mfn_end, 0, PAGE_OFFSET(end));
> +        if ( rc < 0 )
> +            goto err_unlock_remove;
> +        /* Check if not marking R/W part of a page intended to be fully R/O */
> +        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
> +                                                  mfn_x(mfn_end)));
> +    }
> +
> +    spin_unlock(&subpage_ro_lock);
> +
> +    rc = rangeset_add_range(mmio_ro_ranges, mfn_x(mfn_start), mfn_x(mfn_end));
> +    if ( rc )
> +        goto err_remove;
> +
> +    return 0;
> +
> + err_remove:
> +    spin_lock(&subpage_ro_lock);
> +    if ( subpage_end )
> +        subpage_mmio_ro_remove_page(mfn_end, 0, PAGE_OFFSET(end));
> + err_unlock_remove:
> +    if ( subpage_start )
> +        subpage_mmio_ro_remove_page(mfn_start, PAGE_OFFSET(start), offset_end);
> + err_unlock:
> +    spin_unlock(&subpage_ro_lock);
> +    return rc;
> +}
> +
> +static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *entry)
> +{
> +    void __iomem *mapped_page;
> +
> +    if ( entry->mapped )
> +        return entry->mapped;
> +
> +    mapped_page = ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE);

mfn_to_maddr() or some such?

> +    spin_lock(&subpage_ro_lock);
> +    /* Re-check under the lock */
> +    if ( entry->mapped )
> +    {
> +        spin_unlock(&subpage_ro_lock);
> +        iounmap(mapped_page);

What if ioremap() failed?

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 21 15:34:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 15:34:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.726984.1131374 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9RV3-0000pr-Kg; Tue, 21 May 2024 15:34:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 726984.1131374; Tue, 21 May 2024 15:34:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9RV3-0000pk-I2; Tue, 21 May 2024 15:34:01 +0000
Received: by outflank-mailman (input) for mailman id 726984;
 Tue, 21 May 2024 15:33:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TC+7=MY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9RV1-0000pe-Du
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 15:33:59 +0000
Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com
 [103.168.172.146]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 897633fa-1787-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 17:33:57 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.nyi.internal (Postfix) with ESMTP id 09C0F138151B;
 Tue, 21 May 2024 11:33:56 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute5.internal (MEProxy); Tue, 21 May 2024 11:33:56 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 21 May 2024 11:33:54 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 897633fa-1787-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716305636;
	 x=1716392036; bh=vFl4+lxzr6JyOt+gEQtfeTNSYfxkYgu2WIZ0W6tKu7c=; b=
	Av1VKWxmXnSg/WET12THzERdDEappmOO+kKJKEftWV3uR9yNMle/LvaA9/ulTWL1
	/j96WeG/Zo2Fy8xurTuMFv5v70ntxC/ahGJ9HLnKZj39forRiubtK2kq94e32mnz
	Xvgb6qJyWtskRNWUnKoZX1Wp1dg5Y3bkSGJFwoqq9JzZUQUOxK/oHmT0G+gFQw2r
	76GvHGyFhHG+s1l6rV0nmtHtDb9JPcVIqgD+fN7oARrcKHQeOCNCGHjcSnsEK5wI
	53CuMyGsPbKf8I8knUn6UltBptQhNfFTaqKDYOkDUgYKC3Uh2hI+5ym0Da/DYHWs
	ykVQCRHVwn0wvszTVAMr3w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716305636; x=1716392036; bh=vFl4+lxzr6JyOt+gEQtfeTNSYfxk
	Ygu2WIZ0W6tKu7c=; b=U1UAUMIjB+nXpeM8KKjqa+DmkihEoM2c0D2NXXVKoLFF
	THFSmEsyzRbkAGRe8/87icL1Szpxc7Zg95VV9leRZDB+VDo3Yca+/bX7HpxYhmZy
	43ueOLxNLQvvJ3McLJqsr+OJ6YdAxA+TRiOoH/TbDN7cHVkMiu8LO4R6hT1ulYQ4
	UO3i1BaAjWbBrW527VzKVPrMJky6J53CUZoi0CRUrAGmofF7Ijq8d49e5VR7V3nl
	G0AtNhoS4iY6iQb02Keb8f4wdy8iVHmuyF4t4yuJTeqNLyQyoqFiH0sxaL8RbbLC
	OB73C6xMMRJ+rSXJqINWQ5IH8V9NVYdzCdqEJQr2WQ==
X-ME-Sender: <xms:475MZt4B1NqAv7sJ5Q28mVuPtQapSdIEIGwHob1nj-kMr7P0LdHaUw>
    <xme:475MZq5bkCrS31ZjdFwyyVGIYq1h-Bkp71AQBO0MgAGQ2XMxMDufMuN8b6WNFShM7
    Q-n6iPRcsuUHg>
X-ME-Received: <xmr:475MZke1nTG757zOjtr0RsMotyp7vQmFVYb6zZMZEvTFluppfq1n_wM>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeivddgkeekucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:475MZmJlW8XG51eIo-1s_b_yxSbZ3qTtEeTBbXpX59G6bmNvMm5yeA>
    <xmx:475MZhL7m9MWKu7itdXo3ZLausi-QWm7M2PqjQBLbHGGpXNsfx7vDg>
    <xmx:475MZvyues5bMS1GzJvjzSoDMvgQMNcVv7GkX0lciHV998Oemlrz5w>
    <xmx:475MZtJ6YG7NJfXUll5uU9fn_hZCvAcAlPI6s5M8QJxuazfWRTZCJQ>
    <xmx:475MZh14k4RHIxZvw4v75FwRNQuBhGRabCNeRJvEQ1kbGDSOFkuu-a33>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 21 May 2024 17:33:51 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Message-ID: <Zky-4C2WOTboJFLb@mail-itl>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <245e535a-5b23-4d3e-83e5-dc797068652c@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="zkEtAoOBRXaQqDHo"
Content-Disposition: inline
In-Reply-To: <245e535a-5b23-4d3e-83e5-dc797068652c@suse.com>


--zkEtAoOBRXaQqDHo
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 21 May 2024 17:33:51 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only

On Tue, May 21, 2024 at 05:16:58PM +0200, Jan Beulich wrote:
> On 21.05.2024 04:54, Marek Marczykowski-G=C3=B3recki wrote:
> > --- a/xen/arch/x86/include/asm/mm.h
> > +++ b/xen/arch/x86/include/asm/mm.h
> > @@ -522,9 +522,27 @@ extern struct rangeset *mmio_ro_ranges;
> >  void memguard_guard_stack(void *p);
> >  void memguard_unguard_stack(void *p);
> > =20
> > +/*
> > + * Add more precise r/o marking for a MMIO page. Range specified here
> > + * will still be R/O, but the rest of the page (not marked as R/O via =
another
> > + * call) will have writes passed through.
> > + * The start address and the size must be aligned to SUBPAGE_MMIO_RO_A=
LIGN.
> > + *
> > + * This API cannot be used for overlapping ranges, nor for pages alrea=
dy added
> > + * to mmio_ro_ranges separately.
> > + *
> > + * Return values:
> > + *  - negative: error
> > + *  - 0: success
> > + */
> > +#define SUBPAGE_MMIO_RO_ALIGN 8
>=20
> This isn't just alignment, but also (and perhaps more importantly) granul=
arity.
> I think the name wants to express this.

SUBPAGE_MMIO_RO_GRANULARITY? Sounds a bit long...

>=20
> > @@ -4910,6 +4921,260 @@ long arch_memory_op(unsigned long cmd, XEN_GUES=
T_HANDLE_PARAM(void) arg)
> >      return rc;
> >  }
> > =20
> > +/*
> > + * Mark part of the page as R/O.
> > + * Returns:
> > + * - 0 on success - first range in the page
> > + * - 1 on success - subsequent range in the page
> > + * - <0 on error
> > + *
> > + * This needs subpage_ro_lock already taken */
>=20
> Nit: Comment style (full stop and */ on its own line).
>=20
> > +static int __init subpage_mmio_ro_add_page(
> > +    mfn_t mfn, unsigned int offset_s, unsigned int offset_e)
> > +{
> > +    struct subpage_ro_range *entry =3D NULL, *iter;
> > +    unsigned int i;
> > +
> > +    list_for_each_entry(iter, &subpage_ro_ranges, list)
> > +    {
> > +        if ( mfn_eq(iter->mfn, mfn) )
> > +        {
> > +            entry =3D iter;
> > +            break;
> > +        }
> > +    }
> > +    if ( !entry )
> > +    {
> > +        /* iter =3D=3D NULL marks it was a newly allocated entry */
> > +        iter =3D NULL;
> > +        entry =3D xzalloc(struct subpage_ro_range);
> > +        if ( !entry )
> > +            return -ENOMEM;
> > +        entry->mfn =3D mfn;
> > +    }
> > +
> > +    for ( i =3D offset_s; i <=3D offset_e; i +=3D SUBPAGE_MMIO_RO_ALIG=
N )
> > +    {
> > +        int oldbit =3D __test_and_set_bit(i / SUBPAGE_MMIO_RO_ALIGN,
> > +                                        entry->ro_qwords);
>=20
> Why int, not bool?

Because __test_and_set_bit returns int. But I can change to bool if you
prefer.

> > +        ASSERT(!oldbit);
> > +    }
> > +
> > +    if ( !iter )
> > +        list_add(&entry->list, &subpage_ro_ranges);
> > +
> > +    return iter ? 1 : 0;
> > +}
> > +
> > +/* This needs subpage_ro_lock already taken */
> > +static void __init subpage_mmio_ro_remove_page(
> > +    mfn_t mfn,
> > +    int offset_s,
> > +    int offset_e)
>=20
> Can either of these be negative? The more that ...

Right, I can change them to unsigned. They are unsigned already in
subpage_mmio_ro_add_page.

> > +{
> > +    struct subpage_ro_range *entry =3D NULL, *iter;
> > +    unsigned int i;
>=20
> ... this is used ...
>=20
> > +    list_for_each_entry(iter, &subpage_ro_ranges, list)
> > +    {
> > +        if ( mfn_eq(iter->mfn, mfn) )
> > +        {
> > +            entry =3D iter;
> > +            break;
> > +        }
> > +    }
> > +    if ( !entry )
> > +        return;
> > +
> > +    for ( i =3D offset_s; i <=3D offset_e; i +=3D SUBPAGE_MMIO_RO_ALIG=
N )
>=20
> ... with both of them?
>=20
> > +        __clear_bit(i / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords);
> > +
> > +    if ( !bitmap_empty(entry->ro_qwords, PAGE_SIZE / SUBPAGE_MMIO_RO_A=
LIGN) )
> > +        return;
> > +
> > +    list_del(&entry->list);
> > +    if ( entry->mapped )
> > +        iounmap(entry->mapped);
> > +    xfree(entry);
> > +}
> > +
> > +int __init subpage_mmio_ro_add(
> > +    paddr_t start,
> > +    size_t size)
> > +{
> > +    mfn_t mfn_start =3D maddr_to_mfn(start);
> > +    paddr_t end =3D start + size - 1;
> > +    mfn_t mfn_end =3D maddr_to_mfn(end);
> > +    unsigned int offset_end =3D 0;
> > +    int rc;
> > +    bool subpage_start, subpage_end;
> > +
> > +    ASSERT(IS_ALIGNED(start, SUBPAGE_MMIO_RO_ALIGN));
> > +    ASSERT(IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN));
> > +    if ( !IS_ALIGNED(size, SUBPAGE_MMIO_RO_ALIGN) )
> > +        size =3D ROUNDUP(size, SUBPAGE_MMIO_RO_ALIGN);
> > +
> > +    if ( !size )
> > +        return 0;
> > +
> > +    if ( mfn_eq(mfn_start, mfn_end) )
> > +    {
> > +        /* Both starting and ending parts handled at once */
> > +        subpage_start =3D PAGE_OFFSET(start) || PAGE_OFFSET(end) !=3D =
PAGE_SIZE - 1;
> > +        subpage_end =3D false;
> > +    }
> > +    else
> > +    {
> > +        subpage_start =3D PAGE_OFFSET(start);
> > +        subpage_end =3D PAGE_OFFSET(end) !=3D PAGE_SIZE - 1;
> > +    }
> > +
> > +    spin_lock(&subpage_ro_lock);
> > +
> > +    if ( subpage_start )
> > +    {
> > +        offset_end =3D mfn_eq(mfn_start, mfn_end) ?
> > +                     PAGE_OFFSET(end) :
> > +                     (PAGE_SIZE - 1);
> > +        rc =3D subpage_mmio_ro_add_page(mfn_start,
> > +                                      PAGE_OFFSET(start),
> > +                                      offset_end);
> > +        if ( rc < 0 )
> > +            goto err_unlock;
> > +        /* Check if not marking R/W part of a page intended to be full=
y R/O */
> > +        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
> > +                                                  mfn_x(mfn_start)));
> > +    }
> > +
> > +    if ( subpage_end )
> > +    {
> > +        rc =3D subpage_mmio_ro_add_page(mfn_end, 0, PAGE_OFFSET(end));
> > +        if ( rc < 0 )
> > +            goto err_unlock_remove;
> > +        /* Check if not marking R/W part of a page intended to be full=
y R/O */
> > +        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
> > +                                                  mfn_x(mfn_end)));
> > +    }
> > +
> > +    spin_unlock(&subpage_ro_lock);
> > +
> > +    rc =3D rangeset_add_range(mmio_ro_ranges, mfn_x(mfn_start), mfn_x(=
mfn_end));
> > +    if ( rc )
> > +        goto err_remove;
> > +
> > +    return 0;
> > +
> > + err_remove:
> > +    spin_lock(&subpage_ro_lock);
> > +    if ( subpage_end )
> > +        subpage_mmio_ro_remove_page(mfn_end, 0, PAGE_OFFSET(end));
> > + err_unlock_remove:
> > +    if ( subpage_start )
> > +        subpage_mmio_ro_remove_page(mfn_start, PAGE_OFFSET(start), off=
set_end);
> > + err_unlock:
> > +    spin_unlock(&subpage_ro_lock);
> > +    return rc;
> > +}
> > +
> > +static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *en=
try)
> > +{
> > +    void __iomem *mapped_page;
> > +
> > +    if ( entry->mapped )
> > +        return entry->mapped;
> > +
> > +    mapped_page =3D ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE=
);
>=20
> mfn_to_maddr() or some such?

Makes sense.

> > +    spin_lock(&subpage_ro_lock);
> > +    /* Re-check under the lock */
> > +    if ( entry->mapped )
> > +    {
> > +        spin_unlock(&subpage_ro_lock);
> > +        iounmap(mapped_page);
>=20
> What if ioremap() failed?

Good point.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--zkEtAoOBRXaQqDHo
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZMvuAACgkQ24/THMrX
1yy3pwf+JOHHFBUzbLrpHdZu4MsujlnqKVQSnQOHAD/W/8GA1z/nwuYb0QQalO7w
Z/08c8t3FfRcRyVdvbPO/X5AkymdHTSdpKQoeyxaGtKn8pONJNlNS8quE5qqaQG/
BVP1E9hSuRzNtDxzaHAZzFyIVB/GB8MdbSa8dYSFiNTUzL1ek+PPKJg1QSF5cX3w
e9hcdKco5ymuVJfBeKF2I+k28IZM3B0v9pkQgwRAiRECG+03t0YrLJqSXj0pJ37S
PR6OCsUMqxD6FBJbDJRec0Vba5F8Ko3u9tPp7bOg9Wv87E4KjA3I1JOj0P1RyuTc
yBfAIYoHPe4+tOKCU+8tR1bX4LO5og==
=uuat
-----END PGP SIGNATURE-----

--zkEtAoOBRXaQqDHo--


From xen-devel-bounces@lists.xenproject.org Tue May 21 17:16:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 17:16:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727007.1131404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5g-000505-5N; Tue, 21 May 2024 17:15:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727007.1131404; Tue, 21 May 2024 17:15:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5g-0004zw-1K; Tue, 21 May 2024 17:15:56 +0000
Received: by outflank-mailman (input) for mailman id 727007;
 Tue, 21 May 2024 17:15:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ryb1=MY=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9T5e-0004kx-Uh
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 17:15:54 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c6cc0968-1795-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 19:15:52 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so914803766b.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 10:15:52 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01a2esm1633701766b.185.2024.05.21.10.15.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 May 2024 10:15:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6cc0968-1795-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716311752; x=1716916552; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5B2c9g1y7aKzjwGqZJNQwIvT3KveJvpY8WkxZZKikdI=;
        b=ntoYAngRptf6uQPSoPC5WUrVq1bnhlzt4WZpb5J9f2CykjF17levd1gENElY1PD6va
         ZWOCHkUhtJhJSo3y4sG2+JCqx7L9iPOR1Y8FRqeKp8qeAK4NozQJ/qZfUE+PDlZE9y4i
         +qQ/evlIngJtjucblxthV0IgI9SLQkYXHWTDo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716311752; x=1716916552;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5B2c9g1y7aKzjwGqZJNQwIvT3KveJvpY8WkxZZKikdI=;
        b=d9ufg7LN2KrHB/Ia1flrUr7ZFaEe1QLksA3ppi2k+wzOHeJ+LkRrK+aRpZy0KcaiNX
         Y58i/0PkuZPbAsgslWrQVAZD35IzWDvPIlBB+9zycfnU7TsxOgDK8dCp1luLWs/3466D
         uyOzyMEQfC679X+64J4ylOAL2rRCRkPUcFdGQVx+MToFT4tLFRUC5Y4ya6be23chNHOP
         zpDFbIqqlP5hajoGTC/SDTklZDnkC9Qm2DwiBu+5SLYB8uQsJb/FjMd224EFFi/62tzG
         JjytInpabPFbayKGFhfAMrukAWmuCA7zwsBG7yTDnR6evXMnytHiWM63ZuZFOFeyBc9g
         YEHQ==
X-Gm-Message-State: AOJu0Yz2Wr5ajnkii1cCTpUEZqr6JSWLQ5OUQSEeHfPLwPHgswkXBW1d
	qa2xnEr8fxEBHWuTMFM4qKPrIMr82ytxSDbOjucnKpQ3v0fytyt0/AAOBxQ9wSiKkbC286U8a8H
	NjQc=
X-Google-Smtp-Source: AGHT+IGKBlnC4aUlAtSStcXUBX/m7c79FkGO43xaEXt+WhXGcyap3nd6jcMgUWmIeKjp87TzozmoFw==
X-Received: by 2002:a17:906:8315:b0:a59:a977:a154 with SMTP id a640c23a62f3a-a5a2d6786b5mr2132997566b.64.1716311751813;
        Tue, 21 May 2024 10:15:51 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Roberto Bagnara <roberto.bagnara@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 2/3] xen/x86: Drop useless non-Kconfig CONFIG_* variables
Date: Tue, 21 May 2024 18:15:45 +0100
Message-Id: <20240521171546.3650664-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These are all either completely unused, or do nothing useful.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/x86/include/asm/config.h |  4 ----
 xen/include/xen/acpi.h            |  9 ---------
 xen/include/xen/watchdog.h        | 11 -----------
 3 files changed, 24 deletions(-)

diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
index ab7288cb3682..24b005ba1ff7 100644
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -20,7 +20,6 @@
 #define BITS_PER_XEN_ULONG BITS_PER_LONG
 
 #define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS 1
-#define CONFIG_DISCONTIGMEM 1
 #define CONFIG_NUMA_EMU 1
 
 #define CONFIG_PAGEALLOC_MAX_ORDER (2 * PAGETABLE_ORDER)
@@ -30,11 +29,8 @@
 /* Intel P4 currently has largest cache line (L2 line size is 128 bytes). */
 #define CONFIG_X86_L1_CACHE_SHIFT 7
 
-#define CONFIG_ACPI_SRAT 1
 #define CONFIG_ACPI_CSTATE 1
 
-#define CONFIG_WATCHDOG 1
-
 #define CONFIG_MULTIBOOT 1
 
 #define HZ 100
diff --git a/xen/include/xen/acpi.h b/xen/include/xen/acpi.h
index e78e7e785252..bc4818c9430a 100644
--- a/xen/include/xen/acpi.h
+++ b/xen/include/xen/acpi.h
@@ -140,15 +140,6 @@ int get_cpu_id(u32 acpi_id);
 unsigned int acpi_register_gsi (u32 gsi, int edge_level, int active_high_low);
 int acpi_gsi_to_irq (u32 gsi, unsigned int *irq);
 
-/*
- * This function undoes the effect of one call to acpi_register_gsi().
- * If this matches the last registration, any IRQ resources for gsi
- * are freed.
- */
-#ifdef CONFIG_ACPI_DEALLOCATE_IRQ
-void acpi_unregister_gsi (u32 gsi);
-#endif
-
 #ifdef	CONFIG_ACPI_CSTATE
 /*
  * max_cstate sets the highest legal C-state.
diff --git a/xen/include/xen/watchdog.h b/xen/include/xen/watchdog.h
index 86fde0884ae7..434fcbdd2b11 100644
--- a/xen/include/xen/watchdog.h
+++ b/xen/include/xen/watchdog.h
@@ -9,8 +9,6 @@
 
 #include <xen/types.h>
 
-#ifdef CONFIG_WATCHDOG
-
 /* Try to set up a watchdog. */
 int watchdog_setup(void);
 
@@ -23,13 +21,4 @@ void watchdog_disable(void);
 /* Is the watchdog currently enabled. */
 bool watchdog_enabled(void);
 
-#else
-
-#define watchdog_setup() ((void)0)
-#define watchdog_enable() ((void)0)
-#define watchdog_disable() ((void)0)
-#define watchdog_enabled() ((void)0)
-
-#endif
-
 #endif /* __XEN_WATCHDOG_H__ */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 21 17:16:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 17:16:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727008.1131413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5h-0005FO-EJ; Tue, 21 May 2024 17:15:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727008.1131413; Tue, 21 May 2024 17:15:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5h-0005FF-Bd; Tue, 21 May 2024 17:15:57 +0000
Received: by outflank-mailman (input) for mailman id 727008;
 Tue, 21 May 2024 17:15:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ryb1=MY=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9T5f-0004kx-Uk
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 17:15:55 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c750c4e0-1795-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 19:15:53 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso881811266b.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 10:15:53 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01a2esm1633701766b.185.2024.05.21.10.15.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 May 2024 10:15:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c750c4e0-1795-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716311753; x=1716916553; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8g3u7OwfcnXYNWIVYdAYd5uW4iG/kfNNCLqG31h0UaY=;
        b=PCzXBupEebowze8u8FKDQAZ+qFh8tZu/VBT/gzTXbYgV2o6CFoGH9VHtcSYFKa20xR
         69FGQ5GaXyGGIrvwnVOsBV6UsW6+jgtXeJzR0YL7VzL9Y6tGRFpiz8hYhmmJuekGhkxo
         YPBUmiqYZW78QruSlsVg8rFxYEX5ve1+nKmyo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716311753; x=1716916553;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8g3u7OwfcnXYNWIVYdAYd5uW4iG/kfNNCLqG31h0UaY=;
        b=HkLcpULQo9Vyl+6PQsIHCKZ6I2Inv5MdP93nUIOCbGPL4vcAc2hvwf83Dp3LiqS3Sg
         PC1i3ei/HGglgI3HGekWJtYIVJU6hkSv5jSNHWI+t35qPUE2Hm/Xo+m4njrd/6P26bjQ
         1FYh6oNeLpWY5AgFLnc09bd3OKjgKxHa/C7MJqBVEAgZC5GmlRu6q79MPdrUz/3PrUcI
         sKX2K+7uVJR4zyNzRqycwH4rl+zJDivmWdg+nrnXcC3n2AlbUZbvJT84lEYO8oMlQP5A
         bECjc93Ath2lUK1xcTfYxH1EfK+eSQ06dDtKwsdN/CeRJ+Rp8Nlw07N4mpgj5pw8Kcck
         o4hg==
X-Gm-Message-State: AOJu0YzYCTABFnkVevP3YUEEGnrQxQdYe0w7BF5v38XHKNM2Opyg03/C
	mI2QDxFRB7tU1mPs1zOWzi1D5jbd+5nZ33EBBItGTAOHu8akwYK/Hrv1Gbu2+xh7vw0/6YNieZr
	DbUw=
X-Google-Smtp-Source: AGHT+IHi8WUDniYaK0qJBVO/ZXZhtE6ocfJ6wJ8+lH9JrASZvQuMyMVqddjClPEi+/rLDz+2zDlgVw==
X-Received: by 2002:a17:906:d935:b0:a59:d243:945 with SMTP id a640c23a62f3a-a5a2d66a888mr1940746666b.57.1716311752774;
        Tue, 21 May 2024 10:15:52 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Roberto Bagnara <roberto.bagnara@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 3/3] xen/x86: Address two misc MISRA 17.7 violations
Date: Tue, 21 May 2024 18:15:46 +0100
Message-Id: <20240521171546.3650664-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Neither text_poke() nor watchdog_setup() have return value consulted.  Switch
them to being void.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/x86/alternative.c | 4 ++--
 xen/arch/x86/nmi.c         | 5 ++---
 xen/include/xen/watchdog.h | 2 +-
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c
index 2e7ba6e0b833..7824053c9d33 100644
--- a/xen/arch/x86/alternative.c
+++ b/xen/arch/x86/alternative.c
@@ -155,10 +155,10 @@ void init_or_livepatch add_nops(void *insns, unsigned int len)
  * "noinline" to cause control flow change and thus invalidate I$ and
  * cause refetch after modification.
  */
-static void *init_or_livepatch noinline
+static void init_or_livepatch noinline
 text_poke(void *addr, const void *opcode, size_t len)
 {
-    return memcpy(addr, opcode, len);
+    memcpy(addr, opcode, len);
 }
 
 extern void *const __initdata_cf_clobber_start[];
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index f6329cb0270e..9793fa23168d 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -464,12 +464,12 @@ bool watchdog_enabled(void)
     return !atomic_read(&watchdog_disable_count);
 }
 
-int __init watchdog_setup(void)
+void __init watchdog_setup(void)
 {
     unsigned int cpu;
 
     /*
-     * Activate periodic heartbeats. We cannot do this earlier during 
+     * Activate periodic heartbeats. We cannot do this earlier during
      * setup because the timer infrastructure is not available.
      */
     for_each_online_cpu ( cpu )
@@ -477,7 +477,6 @@ int __init watchdog_setup(void)
     register_cpu_notifier(&cpu_nmi_nfb);
 
     watchdog_enable();
-    return 0;
 }
 
 /* Returns false if this was not a watchdog NMI, true otherwise */
diff --git a/xen/include/xen/watchdog.h b/xen/include/xen/watchdog.h
index 434fcbdd2b11..40c21bca823f 100644
--- a/xen/include/xen/watchdog.h
+++ b/xen/include/xen/watchdog.h
@@ -10,7 +10,7 @@
 #include <xen/types.h>
 
 /* Try to set up a watchdog. */
-int watchdog_setup(void);
+void watchdog_setup(void);
 
 /* Enable the watchdog. */
 void watchdog_enable(void);
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 21 17:16:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 17:16:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727005.1131383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5c-0004Wv-OG; Tue, 21 May 2024 17:15:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727005.1131383; Tue, 21 May 2024 17:15:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5c-0004Wo-Lk; Tue, 21 May 2024 17:15:52 +0000
Received: by outflank-mailman (input) for mailman id 727005;
 Tue, 21 May 2024 17:15:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ryb1=MY=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9T5b-0004Wi-NL
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 17:15:51 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5c0deab-1795-11ef-909f-e314d9c70b13;
 Tue, 21 May 2024 19:15:50 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5dcb5a0db4so404610666b.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 10:15:50 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01a2esm1633701766b.185.2024.05.21.10.15.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 May 2024 10:15:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5c0deab-1795-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716311750; x=1716916550; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=3xre9RWfCvvSejwmBrpN5O2Au/2lcj4lXEzwU2TKx8s=;
        b=NHl8XBNs+NpkbvW2OSivpcXX7oobqX1jIFBj7MtmCEwkfVvaU9JGefIL3S9IYENjIr
         lZ21RBTk6KOygtWUh9i27fTTVE48pQ8m6E7US8LSHcCg3uZsOpA5flbK0USaQIHEKu5J
         WtndthGjRMGzIVMG6bWI72gS0nMnWL1hNGmdY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716311750; x=1716916550;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=3xre9RWfCvvSejwmBrpN5O2Au/2lcj4lXEzwU2TKx8s=;
        b=Z0MP9U4Xdbfp5f3A5CqrlABc3byh1kOLjL6tKmj4vvRB+CgHlB6BfrTzRy9QKWnj/W
         c05p0nCagOhZVwbhPRF3FPfW+EhgkpLdWiLZVVb5NZ+yYhkBxd01frf+Oi7iTCmciXOs
         xbInfIJuLgXRqp2+t7ZbjEVInzrFsD5Hxir3vjJgGS3V0ThHLe1XJXooHts/HueBaV+h
         q0vrvdkO+C4OUJ+s8mxCxbPOc3ZSU4zOC2bi+GSL6VY8tRNsL4n1pEIPPvn5gznnJKBp
         PyMJ5ARKKmanrL36Ipne46QK7OO9jpWB/gogLPeWF2quB0gsw+B8OtgAGUMWIoB+OZzg
         M5Vg==
X-Gm-Message-State: AOJu0YyBjngGl03hAa/Bttjwsn3CsovQpDLC/+Exgw/EzbU9tfCRsFQO
	eEV0psyJwYn5GcFA76KE7mN8c6cDv82/SkWh2lOdMfHcljPXxRdtTXwUkhVzMmQuPGUQLHyBQgP
	k/6s=
X-Google-Smtp-Source: AGHT+IEhtyMTNMWgnZK6GROuUOFWw3ho9OlQvrwrcHF4ju1TyXX42MZdTruLhkvfEeP74xu7vxz3Lg==
X-Received: by 2002:a17:907:d22:b0:a59:aa9d:3142 with SMTP id a640c23a62f3a-a5a2d5cb794mr2650121966b.37.1716311750021;
        Tue, 21 May 2024 10:15:50 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Roberto Bagnara <roberto.bagnara@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 0/3] xen: Misc MISRA changes
Date: Tue, 21 May 2024 18:15:43 +0100
Message-Id: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Misc fixes collected during today's call.

Andrew Cooper (3):
  xen/lzo: Implement COPY{4,8} using memcpy()
  xen/x86: Drop useless non-Kconfig CONFIG_* variables
  xen/x86: Address two misc MISRA 17.7 violations

 xen/arch/x86/alternative.c        |  4 ++--
 xen/arch/x86/include/asm/config.h |  4 ----
 xen/arch/x86/nmi.c                |  5 ++---
 xen/common/lzo.c                  | 11 ++---------
 xen/include/xen/acpi.h            |  9 ---------
 xen/include/xen/watchdog.h        | 13 +------------
 6 files changed, 7 insertions(+), 39 deletions(-)


base-commit: 26b122e3bf8f3921d87312fbf5e7e13872ae92b0
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 21 17:16:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 17:16:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727006.1131394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5e-0004lC-Uo; Tue, 21 May 2024 17:15:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727006.1131394; Tue, 21 May 2024 17:15:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9T5e-0004l3-Ru; Tue, 21 May 2024 17:15:54 +0000
Received: by outflank-mailman (input) for mailman id 727006;
 Tue, 21 May 2024 17:15:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ryb1=MY=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9T5e-0004kx-8t
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 17:15:54 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c6a1317b-1795-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 19:15:52 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso83555081fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 10:15:52 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01a2esm1633701766b.185.2024.05.21.10.15.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 May 2024 10:15:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6a1317b-1795-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716311751; x=1716916551; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=n8KPCauD1h/QxTPyhyMp5VxJ7O+vNU2tVT5i7N7FwDg=;
        b=bqNs0vlyGD9nO3m7MdjXfQa48aWEQOmCBp8QG77zT3D31uipM3VqdgCVZkiFDijQBj
         /KaXaOa24wMEgd+OVZEa2xRhv+bNqPgc3lntih6LQPMq6VWW98GnuSlAmXEP3vPQYOn9
         Pk7RkQf8ZYSAhSktIJJB1th4ROICnIOkmQd4g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716311751; x=1716916551;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=n8KPCauD1h/QxTPyhyMp5VxJ7O+vNU2tVT5i7N7FwDg=;
        b=MkbsFLWSUV8O5hWQed1FsBo+NuOvJM6YOVlDKazJbfjjbV9ogAFv0D/Zh/djCScxEH
         Cd05l9QM0u084uNqQlespRzaYXDOrNCSq10YzaahMkIdWTCFT4SKxPCX6PnOE5UXXTzg
         IR9noBksa1RC+WAaigyIN7lSkXogdkTL3ZF8q6QpXXizjMu2Gq6/GPkPxaY3512LYPIy
         wiqyfZYOL6PRLRJ1y7LcxEGvPBSVS680egVcrmkXMuWMuGav3lbXwGDaDSQwD1v6107A
         ip988Oi/1eJGf67m8ndPdXR+YKFanfy7JsnzG2CRtgg5Ogg355hOluMgQ7v0SLccDfWY
         ck0w==
X-Gm-Message-State: AOJu0YzWqg7zOcB9XbdsaUe2DgMk6hF6l0FnOiPfFkKL3pJbilEMVstK
	0mx3lctyo0MngxiMUkQmuDu/xW1p9cEedMvsKDVHxZNbDfi/b8+Jsv/XDtuseEoJ9J16/KASWMD
	x1jo=
X-Google-Smtp-Source: AGHT+IGwSfiFmsPP2j0uUZoXchMnqqeT1sZn4y+yf1F1qYyLp3rJi6sjSSAbgO/rbThPohw3cpuxeQ==
X-Received: by 2002:a2e:7a0e:0:b0:2e2:466b:1a56 with SMTP id 38308e7fff4ca-2e52039e2c3mr254345661fa.53.1716311751043;
        Tue, 21 May 2024 10:15:51 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Roberto Bagnara <roberto.bagnara@bugseng.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 1/3] xen/lzo: Implement COPY{4,8} using memcpy()
Date: Tue, 21 May 2024 18:15:44 +0100
Message-Id: <20240521171546.3650664-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is simpler and easier for both humans and compilers to read.

It also addresses 6 instances of MISRA R5.3 violation (shadowing of the ptr_
local variable inside both {put,get}_unaligned()).

No change, not even in the compiled binary.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/common/lzo.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/xen/common/lzo.c b/xen/common/lzo.c
index cc03f0f5547f..3454ce4a7e24 100644
--- a/xen/common/lzo.c
+++ b/xen/common/lzo.c
@@ -25,15 +25,8 @@
  */
 
 
-#define COPY4(dst, src)    \
-        put_unaligned(get_unaligned((const u32 *)(src)), (u32 *)(dst))
-#if defined(__x86_64__)
-#define COPY8(dst, src)    \
-        put_unaligned(get_unaligned((const u64 *)(src)), (u64 *)(dst))
-#else
-#define COPY8(dst, src)    \
-        COPY4(dst, src); COPY4((dst) + 4, (src) + 4)
-#endif
+#define COPY4(dst, src) memcpy(dst, src, 4)
+#define COPY8(dst, src) memcpy(dst, src, 8)
 
 #ifdef __MINIOS__
 # include <lib.h>
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 21 18:00:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 18:00:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727030.1131423 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Tmx-0004Te-Lo; Tue, 21 May 2024 18:00:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727030.1131423; Tue, 21 May 2024 18:00:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9Tmx-0004TX-J7; Tue, 21 May 2024 18:00:39 +0000
Received: by outflank-mailman (input) for mailman id 727030;
 Tue, 21 May 2024 18:00:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Tmw-0004TN-Fj; Tue, 21 May 2024 18:00:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Tmw-0001TD-Bt; Tue, 21 May 2024 18:00:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Tmw-0005g6-04; Tue, 21 May 2024 18:00:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9Tmv-0003gK-Vk; Tue, 21 May 2024 18:00:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HregG1UcmSiXtcBODM9pDeyv3vQkmT6ML1SOg8U9kSE=; b=zJBFWd//459iFAMdWpjsAlZ6+J
	c6dL+Kp23dICOvkh6SCQ9btIFEc57AVychUe4MImWyze1MEAtOxWSkgqao4OtZyt7eM0HjsAhAFNd
	x344fF+/iEG1U4tG8qx0/iV/2/zHN3yr5oper0Z3l7/KHZgTJfPb03yYk6vnQ6akDewM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186064-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186064: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
X-Osstest-Versions-That:
    xen=24dbf5bd03ff11dbcebb4d15e34fbba8eb34936b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 18:00:37 +0000

flight 186064 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186064/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c
baseline version:
 xen                  24dbf5bd03ff11dbcebb4d15e34fbba8eb34936b

Last test of basis   186062  2024-05-21 10:00:22 Z    0 days
Testing same since   186064  2024-05-21 15:04:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Henry Wang <xin.wang2@amd.com>
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   24dbf5bd03..ced21fbb28  ced21fbb2842ac4655048bdee56232974ff9ff9c -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 21 19:20:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 19:20:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727041.1131434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9V1v-0005Bn-9N; Tue, 21 May 2024 19:20:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727041.1131434; Tue, 21 May 2024 19:20:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9V1v-0005Bg-62; Tue, 21 May 2024 19:20:11 +0000
Received: by outflank-mailman (input) for mailman id 727041;
 Tue, 21 May 2024 19:20:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9V1t-0005BW-Rx; Tue, 21 May 2024 19:20:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9V1t-0002lS-OS; Tue, 21 May 2024 19:20:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9V1t-000092-B9; Tue, 21 May 2024 19:20:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9V1t-00035F-Ab; Tue, 21 May 2024 19:20:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2awRNg5PSZw6bNbLi2GfKPAUdceBsE1Lfpsu6bZkLhc=; b=cp4HYlw8GCxqmIQ0q9WMsMp1Kw
	iEnZoo3Z7LdL4ufCRR+P/ldsnRA7z9yltM9oCr/tQ9O7nc/RZ5xbpirEt3n5uIddyy7btITn/95yv
	Orjcz5Nc0BgHzmfo15zR8N475oVR/DnbGP7hCyfJa7RQcHFEvtdcbIKiMq9NQ2lD8Cm4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186058-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186058: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=26b122e3bf8f3921d87312fbf5e7e13872ae92b0
X-Osstest-Versions-That:
    xen=54aa34fc89151c943550541e0af47664fb6e8676
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 19:20:09 +0000

flight 186058 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186058/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186049
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186049
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186049
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186049
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186049
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186049
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  26b122e3bf8f3921d87312fbf5e7e13872ae92b0
baseline version:
 xen                  54aa34fc89151c943550541e0af47664fb6e8676

Last test of basis   186049  2024-05-20 18:07:01 Z    1 days
Testing same since   186058  2024-05-21 05:10:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   54aa34fc89..26b122e3bf  26b122e3bf8f3921d87312fbf5e7e13872ae92b0 -> master


From xen-devel-bounces@lists.xenproject.org Tue May 21 19:34:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 19:34:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727064.1131460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9VFm-000706-Mg; Tue, 21 May 2024 19:34:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727064.1131460; Tue, 21 May 2024 19:34:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9VFm-0006zz-J3; Tue, 21 May 2024 19:34:30 +0000
Received: by outflank-mailman (input) for mailman id 727064;
 Tue, 21 May 2024 19:34:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Qq=MY=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s9VFl-0006zc-6H
 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 19:34:29 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 224a2c1b-17a9-11ef-b4bb-af5377834399;
 Tue, 21 May 2024 21:34:27 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.202])
 by support.bugseng.com (Postfix) with ESMTPSA id 2414F4EE0738;
 Tue, 21 May 2024 21:34:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 224a2c1b-17a9-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH] automation/eclair_analysis: add already clean rules to the analysis
Date: Tue, 21 May 2024 21:34:21 +0200
Message-Id: <b54857c4bd0ac6e29042a55c0692178bb54ac307.1716319970.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Some MISRA C rules already have no violations in Xen, so they can be
set as clean.

Reorder the rules in tagging.ecl according to version ordering
(i.e. sort -V) and split the configuration on multiple lines for
readability.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 .../eclair_analysis/ECLAIR/monitored.ecl      | 17 ++++
 automation/eclair_analysis/ECLAIR/tagging.ecl | 78 ++++++++++++++++++-
 2 files changed, 94 insertions(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/monitored.ecl b/automation/eclair_analysis/ECLAIR/monitored.ecl
index 9da709dc889c..4daecb0c838f 100644
--- a/automation/eclair_analysis/ECLAIR/monitored.ecl
+++ b/automation/eclair_analysis/ECLAIR/monitored.ecl
@@ -79,4 +79,21 @@
 -enable=MC3R1.R9.3
 -enable=MC3R1.R9.4
 -enable=MC3R1.R9.5
+-enable=MC3R1.R18.8
+-enable=MC3R1.R20.2
+-enable=MC3R1.R20.3
+-enable=MC3R1.R20.6
+-enable=MC3R1.R20.11
+-enable=MC3R1.R21.3
+-enable=MC3R1.R21.4
+-enable=MC3R1.R21.5
+-enable=MC3R1.R21.7
+-enable=MC3R1.R21.8
+-enable=MC3R1.R21.12
+-enable=MC3R1.R22.1
+-enable=MC3R1.R22.3
+-enable=MC3R1.R22.7
+-enable=MC3R1.R22.8
+-enable=MC3R1.R22.9
+-enable=MC3R1.R22.10
 -doc_end
diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index b7a9f75b275b..a354ff322e03 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -19,7 +19,83 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R10.2||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common,
+"MC3R1.D1.1||
+MC3R1.D2.1||
+MC3R1.D4.1||
+MC3R1.D4.11||
+MC3R1.D4.14||
+MC3R1.R1.1||
+MC3R1.R1.3||
+MC3R1.R1.4||
+MC3R1.R2.2||
+MC3R1.R2.6||
+MC3R1.R3.1||
+MC3R1.R3.2||
+MC3R1.R4.1||
+MC3R1.R4.2||
+MC3R1.R5.1||
+MC3R1.R5.2||
+MC3R1.R5.4||
+MC3R1.R5.6||
+MC3R1.R6.1||
+MC3R1.R6.2||
+MC3R1.R7.1||
+MC3R1.R7.2||
+MC3R1.R7.4||
+MC3R1.R8.1||
+MC3R1.R8.2||
+MC3R1.R8.5||
+MC3R1.R8.6||
+MC3R1.R8.8||
+MC3R1.R8.10||
+MC3R1.R8.12||
+MC3R1.R8.14||
+MC3R1.R9.2||
+MC3R1.R9.3||
+MC3R1.R9.4||
+MC3R1.R9.5||
+MC3R1.R10.2||
+MC3R1.R11.7||
+MC3R1.R11.9||
+MC3R1.R12.5||
+MC3R1.R14.1||
+MC3R1.R16.7||
+MC3R1.R17.1||
+MC3R1.R17.3||
+MC3R1.R17.4||
+MC3R1.R17.5||
+MC3R1.R17.6||
+MC3R1.R18.8||
+MC3R1.R20.2||
+MC3R1.R20.3||
+MC3R1.R20.4||
+MC3R1.R20.6||
+MC3R1.R20.9||
+MC3R1.R20.11||
+MC3R1.R20.13||
+MC3R1.R20.14||
+MC3R1.R21.3||
+MC3R1.R21.4||
+MC3R1.R21.5||
+MC3R1.R21.7||
+MC3R1.R21.8||
+MC3R1.R21.9||
+MC3R1.R21.10||
+MC3R1.R21.12||
+MC3R1.R21.13||
+MC3R1.R21.19||
+MC3R1.R21.21||
+MC3R1.R22.1||
+MC3R1.R22.2||
+MC3R1.R22.3||
+MC3R1.R22.4||
+MC3R1.R22.5||
+MC3R1.R22.6||
+MC3R1.R22.7||
+MC3R1.R22.8||
+MC3R1.R22.9||
+MC3R1.R22.10"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 21 23:07:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 May 2024 23:07:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727098.1131472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9YZs-0003uf-51; Tue, 21 May 2024 23:07:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727098.1131472; Tue, 21 May 2024 23:07:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9YZs-0003uY-2F; Tue, 21 May 2024 23:07:28 +0000
Received: by outflank-mailman (input) for mailman id 727098;
 Tue, 21 May 2024 23:07:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9YZq-0003uO-LV; Tue, 21 May 2024 23:07:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9YZq-0006Wu-I0; Tue, 21 May 2024 23:07:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9YZq-00017g-A8; Tue, 21 May 2024 23:07:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9YZq-000552-9i; Tue, 21 May 2024 23:07:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YJiJd7L8iFHYrvZpCZm41jxO+827AQ13qphtzl/ylwE=; b=7NoBlLeLc4XNBqgrECw296j/ka
	xnyoVH6aeJMI5asuL+BXr9tG4bVr3xjBkBWPv2wvVhdSTF2SHFCWqcQ+2feIi7K6rY6jWvS7MMjQM
	b4NbZTrWpkPFLSf0nIACHNWtL9qbNSNsJjEH2GLZVDFOeIMnEdUSk4+oRw28jWg8MhPk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186060-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.18-testing test] 186060: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.18-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=7cdb1fa2ab0b5e11f66cada0370770404153c824
X-Osstest-Versions-That:
    xen=f0ff1d9cb96041a84a24857a6464628240deed4f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 May 2024 23:07:26 +0000

flight 186060 xen-4.18-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186060/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185865
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185865
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185865
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185865
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185865
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185865
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  7cdb1fa2ab0b5e11f66cada0370770404153c824
baseline version:
 xen                  f0ff1d9cb96041a84a24857a6464628240deed4f

Last test of basis   185865  2024-04-29 08:08:54 Z   22 days
Testing same since   186060  2024-05-21 08:38:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f0ff1d9cb9..7cdb1fa2ab  7cdb1fa2ab0b5e11f66cada0370770404153c824 -> stable-4.18


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:07:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:07:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727137.1131531 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aRI-0008Cg-QM; Wed, 22 May 2024 01:06:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727137.1131531; Wed, 22 May 2024 01:06:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aRI-0008CZ-Na; Wed, 22 May 2024 01:06:44 +0000
Received: by outflank-mailman (input) for mailman id 727137;
 Wed, 22 May 2024 01:06:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9aRG-0008CD-EF
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:06:42 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 893cf9c1-17d7-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 03:06:38 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 6AF87CE118D;
 Wed, 22 May 2024 01:06:31 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58010C2BD11;
 Wed, 22 May 2024 01:06:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 893cf9c1-17d7-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716339990;
	bh=TTOoVNOjd3mf81omwkGQsgjBkQ/aPgXm6jABF/oZqKI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mlojxMnydV5m/ffmRtXV9PAzTtjpGA9kwwf90vjZgCcNs7JBMIQKXHfTR101oTlPF
	 QCCKgXWN2k4kBupIhjZxFyxSIFXvb0mf8u6muveg8VNsdZlwctuEI8RUAo81st8HrM
	 WGXPHFHcVHYgqdxJuo/7h+uEVdFxwWrbYKlUG+XkYSfCmel9wC1i/4Zu3OOjfvaLVE
	 f86bEfiw5OBASJDr4M+Wez8r7AUk6OjCKPVu1zlcjOc62IxRyVTp5hH0081oDypIge
	 /F8QGU2tXPOmJj6uO9Ok9Bd5FNLuxfQhkmMOlFagp0VOiz6KvlWD/cTvRDT2JqDTJM
	 ez+dH4SUF3WOA==
Date: Tue, 21 May 2024 18:06:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [XEN PATCH] automation/eclair_analysis: add already clean rules
 to the analysis
In-Reply-To: <b54857c4bd0ac6e29042a55c0692178bb54ac307.1716319970.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2405211806190.1052252@ubuntu-linux-20-04-desktop>
References: <b54857c4bd0ac6e29042a55c0692178bb54ac307.1716319970.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Nicola Vetrini wrote:
> Some MISRA C rules already have no violations in Xen, so they can be
> set as clean.
> 
> Reorder the rules in tagging.ecl according to version ordering
> (i.e. sort -V) and split the configuration on multiple lines for
> readability.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  .../eclair_analysis/ECLAIR/monitored.ecl      | 17 ++++
>  automation/eclair_analysis/ECLAIR/tagging.ecl | 78 ++++++++++++++++++-
>  2 files changed, 94 insertions(+), 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/monitored.ecl b/automation/eclair_analysis/ECLAIR/monitored.ecl
> index 9da709dc889c..4daecb0c838f 100644
> --- a/automation/eclair_analysis/ECLAIR/monitored.ecl
> +++ b/automation/eclair_analysis/ECLAIR/monitored.ecl
> @@ -79,4 +79,21 @@
>  -enable=MC3R1.R9.3
>  -enable=MC3R1.R9.4
>  -enable=MC3R1.R9.5
> +-enable=MC3R1.R18.8
> +-enable=MC3R1.R20.2
> +-enable=MC3R1.R20.3
> +-enable=MC3R1.R20.6
> +-enable=MC3R1.R20.11
> +-enable=MC3R1.R21.3
> +-enable=MC3R1.R21.4
> +-enable=MC3R1.R21.5
> +-enable=MC3R1.R21.7
> +-enable=MC3R1.R21.8
> +-enable=MC3R1.R21.12
> +-enable=MC3R1.R22.1
> +-enable=MC3R1.R22.3
> +-enable=MC3R1.R22.7
> +-enable=MC3R1.R22.8
> +-enable=MC3R1.R22.9
> +-enable=MC3R1.R22.10
>  -doc_end
> diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
> index b7a9f75b275b..a354ff322e03 100644
> --- a/automation/eclair_analysis/ECLAIR/tagging.ecl
> +++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
> @@ -19,7 +19,83 @@
>  
>  -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
>  
> --service_selector={clean_guidelines_common, "MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R10.2||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.10||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R21.9||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
> +-service_selector={clean_guidelines_common,
> +"MC3R1.D1.1||
> +MC3R1.D2.1||
> +MC3R1.D4.1||
> +MC3R1.D4.11||
> +MC3R1.D4.14||
> +MC3R1.R1.1||
> +MC3R1.R1.3||
> +MC3R1.R1.4||
> +MC3R1.R2.2||
> +MC3R1.R2.6||
> +MC3R1.R3.1||
> +MC3R1.R3.2||
> +MC3R1.R4.1||
> +MC3R1.R4.2||
> +MC3R1.R5.1||
> +MC3R1.R5.2||
> +MC3R1.R5.4||
> +MC3R1.R5.6||
> +MC3R1.R6.1||
> +MC3R1.R6.2||
> +MC3R1.R7.1||
> +MC3R1.R7.2||
> +MC3R1.R7.4||
> +MC3R1.R8.1||
> +MC3R1.R8.2||
> +MC3R1.R8.5||
> +MC3R1.R8.6||
> +MC3R1.R8.8||
> +MC3R1.R8.10||
> +MC3R1.R8.12||
> +MC3R1.R8.14||
> +MC3R1.R9.2||
> +MC3R1.R9.3||
> +MC3R1.R9.4||
> +MC3R1.R9.5||
> +MC3R1.R10.2||
> +MC3R1.R11.7||
> +MC3R1.R11.9||
> +MC3R1.R12.5||
> +MC3R1.R14.1||
> +MC3R1.R16.7||
> +MC3R1.R17.1||
> +MC3R1.R17.3||
> +MC3R1.R17.4||
> +MC3R1.R17.5||
> +MC3R1.R17.6||
> +MC3R1.R18.8||
> +MC3R1.R20.2||
> +MC3R1.R20.3||
> +MC3R1.R20.4||
> +MC3R1.R20.6||
> +MC3R1.R20.9||
> +MC3R1.R20.11||
> +MC3R1.R20.13||
> +MC3R1.R20.14||
> +MC3R1.R21.3||
> +MC3R1.R21.4||
> +MC3R1.R21.5||
> +MC3R1.R21.7||
> +MC3R1.R21.8||
> +MC3R1.R21.9||
> +MC3R1.R21.10||
> +MC3R1.R21.12||
> +MC3R1.R21.13||
> +MC3R1.R21.19||
> +MC3R1.R21.21||
> +MC3R1.R22.1||
> +MC3R1.R22.2||
> +MC3R1.R22.3||
> +MC3R1.R22.4||
> +MC3R1.R22.5||
> +MC3R1.R22.6||
> +MC3R1.R22.7||
> +MC3R1.R22.8||
> +MC3R1.R22.9||
> +MC3R1.R22.10"
>  }
>  
>  -setq=target,getenv("XEN_TARGET_ARCH")
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:08:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:08:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727141.1131541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aSZ-0000GN-43; Wed, 22 May 2024 01:08:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727141.1131541; Wed, 22 May 2024 01:08:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aSZ-0000GG-18; Wed, 22 May 2024 01:08:03 +0000
Received: by outflank-mailman (input) for mailman id 727141;
 Wed, 22 May 2024 01:08:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=X7YA=MZ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9aSX-0000G5-RG
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:08:01 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ba88343c-17d7-11ef-909f-e314d9c70b13;
 Wed, 22 May 2024 03:08:00 +0200 (CEST)
Received: from BL1PR13CA0401.namprd13.prod.outlook.com (2603:10b6:208:2c2::16)
 by DM4PR12MB7574.namprd12.prod.outlook.com (2603:10b6:8:10e::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Wed, 22 May
 2024 01:07:55 +0000
Received: from BL6PEPF00022573.namprd02.prod.outlook.com
 (2603:10b6:208:2c2:cafe::46) by BL1PR13CA0401.outlook.office365.com
 (2603:10b6:208:2c2::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.18 via Frontend
 Transport; Wed, 22 May 2024 01:07:53 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF00022573.mail.protection.outlook.com (10.167.249.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 01:07:53 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 20:07:51 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 20:07:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba88343c-17d7-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U82u3ZDtxZzXBsbEzl2ox4DVPupL5M2ixAgpPJgBaiyyMPof2bRCWITVkwwQ6uK2rnjoZQzNusKxgHiPJ94vMVs9+4DHJAXL2NnRjCl4cKRdMn/NBkLfVUPpJjq4dKwJLHy+DVwMxKtw1XBaacm7s/0YQDAIqllzJOvOIjKcTcXEIl2lx3u6V74eXNf0X9kty8WntbdqPns2BlazLzvRuw5w42JBg28dqIjEsiwRJDzYc5V7TnVcNkOdoQsymqXjEMP7D/z6nf0iS08PzDYgH1Y0BSdCZoCKDPkDt64HQ+SBdqi4+6cZwID95uUCbnGgsCN6W98yrwgl++z7qd+KHA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hlfI7zv2QHpWmE4HOs84OGGn0X8XL8jHHn0FUak/OOE=;
 b=WAkY5En4vMB70pnY6fA4ZT+bRut5FhytJ9pdtgAndTmlMmrRVqGKbKLbgUCYGNPq7KVbe3y4U3/mmGo/Wb0oFntGlnxoq0AQjGEe7ffeQU6MwWvrwq0rKplSwllibX1wT3Fbid+ayRnOXae6XFCleQ9JtJxHMM6s3o4lp96hbRY1pWewmImyLAkJb0iCmUY9n7s2RAekHzvqjxfHVDaRg5CIgGoD9woXAoLOEFyGNvY+N9+UdtAdyL77uYS7FowmKNEsrjuO3+uuwyYmYnzGKxgls5s68UYTKrECadghQxRMqS7kiHdJacMUKKGeXUHlxQv5/okuqaeBTF9Ua0Mgxg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hlfI7zv2QHpWmE4HOs84OGGn0X8XL8jHHn0FUak/OOE=;
 b=bTPpraY3lrDXrJwgDS43vT4M88S6ihSB6wFkg6wjUQVhebDAhiLZ/G7U1hGQqPoHYJfq8ohh49ishdiwcKZ3Tq6SoiclRGYXLmpyeB7RoiJjLfiH0iXycDW8jnqqAvSwHlVs/KE28dExztG8SPbmUcCERehuEmJ+A4DhiaIsmqY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
Date: Wed, 22 May 2024 09:07:30 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-6-xin.wang2@amd.com>
 <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF00022573:EE_|DM4PR12MB7574:EE_
X-MS-Office365-Filtering-Correlation-Id: de370f08-3898-4525-4d1c-08dc79fb9b92
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?d0Zpa2V5cys1ZXg2ZnJnOWNwN0xSZFE1Y2IwSTdENHZING5DenNLQ3dXeDE3?=
 =?utf-8?B?SHFVL3FxUzlnL2xYekhKWE5zWkhHOXU0OVlNRE9KUlh2RnU5SXdIaTFZYk5J?=
 =?utf-8?B?RWxaeEFtMTlySDZadFRuMGNiQVNEdFdlSk10N0VJL3VlRG1FZWo2M200NUtP?=
 =?utf-8?B?ak9DOWxBOVNPcURuM3c1NmkwdTcwb1J0MTc4VnY1citIcG0wWWVmZFNxVi96?=
 =?utf-8?B?bnQzYkNVaDlOQm5OL3JCeGNOT1JITDkxbnl5N0hnVDhEWlFXQUpMVFIycFpX?=
 =?utf-8?B?VTBrZHN3NDdZUDZ5MEI3c0RHVS9keEQzWml3NW10NEloa0JzZ3NrTHJaTUd3?=
 =?utf-8?B?Y0lpVlVUWE9VNVVUbjgrSFRHeWpxUGFsL2pUblZ4Vk9oYjNnU0lJc2NudDhj?=
 =?utf-8?B?V0hJRFFEdG9MTG5iSUVvTFRMN0liU2x4TjBhNkY3aUdiM0xQWVRrcVhqNEMr?=
 =?utf-8?B?N1ozT3JvV09BNk55c0xDdzYvYzJGdkRiemRTUElxUnVpR1k2eVBCdjhaT0Rh?=
 =?utf-8?B?T1hWNGlxV2RuNkVRV3ZMK3JrS09CbTBCWmdYNXpGcGxYRWlpVzQ5ckJFY0NL?=
 =?utf-8?B?UFNLanJDUmZ6NGRFd3VrUzVReXdQWXpLQUpiK2swOHZWdGpmSElzTmNPdkQv?=
 =?utf-8?B?cnNRWi9pYlVqdHMrV3pFOHl6RjBqbEliMmxEb0hrcGpVa1YzdUNvbzVqNjlC?=
 =?utf-8?B?c1d3bFRnOXdrMk5xTlgxOWdmV2dLSzFSbGIyYW81VlNQcVlKVWx2eTg3dkM1?=
 =?utf-8?B?RXNIRHd0WkZRRHB1Z1FvT3JrZDQ0MVoxaVk5c013dmROYkpGU2hGK0c5Y3V2?=
 =?utf-8?B?cnRmT0pqZ29CdjV2cEpLditMWXJUYnZwUHNqdHdVcVl2NVFDZTVCcHp0bVlG?=
 =?utf-8?B?ZUI3d1RNOWpSd0FsVEtFV2w3L2ZzTjFiOTFIL3NYVTlvR1BueVF3dGdEV2NQ?=
 =?utf-8?B?eWZaRnRuTy9JMGJJUWEwb1VyalZFSXh4Y2h6a25JTTlMZi9FWlhiZkk0cmZw?=
 =?utf-8?B?ZGxMd1VCSmhWclZPRFFOOUVwd1J6dkJhYzVQcVo2M09KQkMwSVlSYVdVY0NR?=
 =?utf-8?B?NHVEZmw5Wlo1SHcrSXRmWUwvODZHTWlKY0xpTjZ3TC93RFA4em5ZdHQzMEto?=
 =?utf-8?B?YTlraVpvVWhOcUI1VEJydy94WkZIaG9Wbm05SnRsZTRYRFN6dHRibXMzb2JP?=
 =?utf-8?B?TXdxS1NPaTFMRTRrTmNQTGNQckFGTXFneVdNVURyRUhlRXB3RGFMMnJHT3dr?=
 =?utf-8?B?bWVZTVVDM0lsVmswRFFBdTlWOW9hS1htQVFnTjVHbHFMcUN4UEUrTXgrczZ6?=
 =?utf-8?B?R2I4MitzZU1jUittMHBnNXZ2REUyYmUwaFdWRkVlcHpFL3krRUNnY0dQN2ph?=
 =?utf-8?B?ekM2S2g3WDRVdWdmQTMrRmpGK1JKQW9GMkwzUzNlZ2txelk5bDNlSTBhM3Bo?=
 =?utf-8?B?MmdUaU5VOFdmUS93cHpUWGdNcGZWTEI1T3FlRXBHVnBSZVp0UmcrZlJBM1Nv?=
 =?utf-8?B?a0lpZ1dNbmRoWGk2dTlBeEdQakNXMTBFc3IrcmxkbDFSdVJpdDBZcFlraG1T?=
 =?utf-8?B?NitYQklCbUVrbnFxM0hxOTJhUjhsbmhZVnB1ejh4VGdzOTV2OWMzWlNEUWtp?=
 =?utf-8?B?eTBaM3IySVJjMWt0WGprZGY3b0R3RTNnTFAwL2VITkZJeXlDdXp5amVYWjJw?=
 =?utf-8?B?VXJ5Y0EvYmdqZTlYTkFjVXVkMzlNM2F2SG1kUGFzQk16a01TeUgvSDR4c3Y5?=
 =?utf-8?B?dE90L3ZRQTZKa1FISytsRGZUb24wU1BvSnIrMnBucGNvQ1ZPSWhLdU5VYksx?=
 =?utf-8?Q?AWG3pbRvqzO3pEXWQ4TREbmt6Z39HBFrBUXb8=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 01:07:53.3107
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: de370f08-3898-4525-4d1c-08dc79fb9b92
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF00022573.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7574

Hi Julien,

On 5/21/2024 8:30 PM, Julien Grall wrote:
> Hi,
>
> On 21/05/2024 05:35, Henry Wang wrote:
>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>> index 56490dbc43..956c11ba13 100644
>> --- a/xen/arch/arm/gic-vgic.c
>> +++ b/xen/arch/arm/gic-vgic.c
>> @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, 
>> struct vcpu *v, unsigned int virq,
>>         /* We are taking to rank lock to prevent parallel 
>> connections. */
>>       vgic_lock_rank(v_target, rank, flags);
>> +    spin_lock(&v_target->arch.vgic.lock);
>
> I know this is what Stefano suggested, but v_target would point to the 
> current affinity whereas the interrupt may be pending/active on the 
> "previous" vCPU. So it is a little unclear whether v_target is the 
> correct lock. Do you have more pointer to show this is correct?

No I think you are correct, we have discussed this in the initial 
version of this patch. Sorry.

I followed the way from that discussion to note down the vcpu ID and 
retrieve here, below is the diff, would this make sense to you?

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 956c11ba13..134ed4e107 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -439,7 +439,7 @@ int vgic_connect_hw_irq(struct domain *d, struct 
vcpu *v, unsigned int virq,

      /* We are taking to rank lock to prevent parallel connections. */
      vgic_lock_rank(v_target, rank, flags);
-    spin_lock(&v_target->arch.vgic.lock);
+ spin_lock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);

      if ( connect )
      {
@@ -465,7 +465,7 @@ int vgic_connect_hw_irq(struct domain *d, struct 
vcpu *v, unsigned int virq,
              p->desc = NULL;
      }

-    spin_unlock(&v_target->arch.vgic.lock);
+ spin_unlock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
      vgic_unlock_rank(v_target, rank, flags);

      return ret;
diff --git a/xen/arch/arm/include/asm/vgic.h 
b/xen/arch/arm/include/asm/vgic.h
index 79b73a0dbb..f4075d3e75 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -85,6 +85,7 @@ struct pending_irq
      uint8_t priority;
      uint8_t lpi_priority;       /* Caches the priority if this is an 
LPI. */
      uint8_t lpi_vcpu_id;        /* The VCPU for an LPI. */
+    uint8_t spi_vcpu_id;        /* The VCPU for an SPI. */
      /* inflight is used to append instances of pending_irq to
       * vgic.inflight_irqs */
      struct list_head inflight;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index c04fc4f83f..e852479f13 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -632,6 +632,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu 
*v, unsigned int virq,
      }
      list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
  out:
+    n->spi_vcpu_id = v->vcpu_id;
      spin_unlock_irqrestore(&v->arch.vgic.lock, flags);

      /* we have a new higher priority irq, inject it into the guest */
      vcpu_kick(v);


> Also, while looking at the locking, I noticed that we are not doing 
> anything with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem 
> to assume that if the flag is set, then p->desc cannot be NULL.
>
> Can we reach vgic_connect_hw_irq() with the flag set?

I think even from the perspective of making the code extra safe, we 
should also check GIC_IRQ_GUEST_MIGRATING as the LR is allocated for 
this case. I will also add the check of GIC_IRQ_GUEST_MIGRATING here.

> What about the other flags? Is this going to be a concern if we don't 
> reset them?

I don't think so, if I am not mistaken, no LR will be allocated with 
other flags set.

Kind regards,
Henry

>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Wed May 22 01:16:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:16:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727148.1131550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aat-00022s-05; Wed, 22 May 2024 01:16:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727148.1131550; Wed, 22 May 2024 01:16:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aas-00022l-Tf; Wed, 22 May 2024 01:16:38 +0000
Received: by outflank-mailman (input) for mailman id 727148;
 Wed, 22 May 2024 01:16:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9aar-00022c-Ct
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:16:37 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ec3c6965-17d8-11ef-909f-e314d9c70b13;
 Wed, 22 May 2024 03:16:34 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D596ECE127D;
 Wed, 22 May 2024 01:16:28 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBE02C32786;
 Wed, 22 May 2024 01:16:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec3c6965-17d8-11ef-909f-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716340588;
	bh=kUn+DDiRT0c0kx5drYEsDBfZBZ6NJAIQieGv72PxdXU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=eh0LMNl3kq1JzzRuqHAZ10CuUbwb4wN+6p3XuD8bsV7JDyOmZOIg1UEd5TzZs+YpD
	 wKJeeB5k2fNKxL2TGlypf4Af4jGAyNFwhZvK1yAL7eoDcB9ocS+Om2EUWLszLNCS2S
	 JBkudZcLw1AVFDpoe4Jp1Kmrfh+ie2efWriHgutUX95BonVohRFSEELGKLqVqzIZjH
	 Y8rokZjYZqrEXDQska9YpPvZMcEFXZkPitUNlzakE4uOQdxyVnqiaq1r4rA8hlDhL0
	 b1GkgKcK+rLdssirdW8XTZt8NBg4B6omCxnp4u1tiFr1gbu3ljt2tFjMu40NGkRHQn
	 PBuFJwmK2SeYA==
Date: Tue, 21 May 2024 18:16:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
In-Reply-To: <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405211815040.1052252@ubuntu-linux-20-04-desktop>
References: <20240521043557.1580753-1-xin.wang2@amd.com> <20240521043557.1580753-6-xin.wang2@amd.com> <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org> <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1156148855-1716340587=:1052252"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1156148855-1716340587=:1052252
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 May 2024, Henry Wang wrote:
> Hi Julien,
> 
> On 5/21/2024 8:30 PM, Julien Grall wrote:
> > Hi,
> > 
> > On 21/05/2024 05:35, Henry Wang wrote:
> > > diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> > > index 56490dbc43..956c11ba13 100644
> > > --- a/xen/arch/arm/gic-vgic.c
> > > +++ b/xen/arch/arm/gic-vgic.c
> > > @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct
> > > vcpu *v, unsigned int virq,
> > >         /* We are taking to rank lock to prevent parallel connections. */
> > >       vgic_lock_rank(v_target, rank, flags);
> > > +    spin_lock(&v_target->arch.vgic.lock);
> > 
> > I know this is what Stefano suggested, but v_target would point to the
> > current affinity whereas the interrupt may be pending/active on the
> > "previous" vCPU. So it is a little unclear whether v_target is the correct
> > lock. Do you have more pointer to show this is correct?
> 
> No I think you are correct, we have discussed this in the initial version of
> this patch. Sorry.
> 
> I followed the way from that discussion to note down the vcpu ID and retrieve
> here, below is the diff, would this make sense to you?
> 
> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
> index 956c11ba13..134ed4e107 100644
> --- a/xen/arch/arm/gic-vgic.c
> +++ b/xen/arch/arm/gic-vgic.c
> @@ -439,7 +439,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v,
> unsigned int virq,
> 
>      /* We are taking to rank lock to prevent parallel connections. */
>      vgic_lock_rank(v_target, rank, flags);
> -    spin_lock(&v_target->arch.vgic.lock);
> + spin_lock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
> 
>      if ( connect )
>      {
> @@ -465,7 +465,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v,
> unsigned int virq,
>              p->desc = NULL;
>      }
> 
> -    spin_unlock(&v_target->arch.vgic.lock);
> + spin_unlock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
>      vgic_unlock_rank(v_target, rank, flags);
> 
>      return ret;
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 79b73a0dbb..f4075d3e75 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -85,6 +85,7 @@ struct pending_irq
>      uint8_t priority;
>      uint8_t lpi_priority;       /* Caches the priority if this is an LPI. */
>      uint8_t lpi_vcpu_id;        /* The VCPU for an LPI. */
> +    uint8_t spi_vcpu_id;        /* The VCPU for an SPI. */
>      /* inflight is used to append instances of pending_irq to
>       * vgic.inflight_irqs */
>      struct list_head inflight;
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index c04fc4f83f..e852479f13 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -632,6 +632,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v,
> unsigned int virq,
>      }
>      list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>  out:
> +    n->spi_vcpu_id = v->vcpu_id;
>      spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> 
>      /* we have a new higher priority irq, inject it into the guest */
>      vcpu_kick(v);
> 
> 
> > Also, while looking at the locking, I noticed that we are not doing anything
> > with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem to assume that
> > if the flag is set, then p->desc cannot be NULL.
> > 
> > Can we reach vgic_connect_hw_irq() with the flag set?
> 
> I think even from the perspective of making the code extra safe, we should
> also check GIC_IRQ_GUEST_MIGRATING as the LR is allocated for this case. I
> will also add the check of GIC_IRQ_GUEST_MIGRATING here.

Yes. I think it might be easier to check for GIC_IRQ_GUEST_MIGRATING
early and return error immediately in that case. Otherwise, we can
continue and take spin_lock(&v_target->arch.vgic.lock) because no
migration is in progress




> > What about the other flags? Is this going to be a concern if we don't reset
> > them?
> 
> I don't think so, if I am not mistaken, no LR will be allocated with other
> flags set.
> 
> Kind regards,
> Henry

--8323329-1156148855-1716340587=:1052252--


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:20:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:20:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727157.1131560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aes-0003oA-Fu; Wed, 22 May 2024 01:20:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727157.1131560; Wed, 22 May 2024 01:20:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aes-0003o3-DA; Wed, 22 May 2024 01:20:46 +0000
Received: by outflank-mailman (input) for mailman id 727157;
 Wed, 22 May 2024 01:20:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9aer-0003nx-3P
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:20:45 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f8fad40-17d9-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 03:20:41 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0393CCE104A;
 Wed, 22 May 2024 01:20:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 029F0C2BD11;
 Wed, 22 May 2024 01:20:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f8fad40-17d9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716340837;
	bh=usutB9kPWnf7yH5amBRs8apuxDPP3FOX4HkS9wiYDt4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=toiKTwUlVz7v3j4HqRj5tLETv5GhQGYDP8KZwP07g66Aa1Gs7X/YGqqT1GOrPYIVD
	 lRsc1TEYG9eTZf61cUwDwcr33GnMkkBaHR25S7mYLwIx5ThRIpul/WttCTnFKHHXA+
	 IssliekBlOkUC53+2/Unq32AtSCnYqAtAiivAtI7GNMHwLLsXDQ4tenTCwp2QbBdYD
	 mBJ4C9u/7Chhl/8k9IhSJNIKeSuDVfRIr/Ni4R3nNkSFgc2t3GKyDwq8YojqY+2nbm
	 aMoOpKHb+sMjk+Npu00vsU7AyYRVFy594wQnWifc4OlYT77Zl6XQLBxTRCze7Fwyj6
	 dpSJkHDQOgxmA==
Date: Tue, 21 May 2024 18:20:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Roberto Bagnara <roberto.bagnara@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] xen/lzo: Implement COPY{4,8} using memcpy()
In-Reply-To: <20240521171546.3650664-2-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405211819560.1052252@ubuntu-linux-20-04-desktop>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com> <20240521171546.3650664-2-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Andrew Cooper wrote:
> This is simpler and easier for both humans and compilers to read.
> 
> It also addresses 6 instances of MISRA R5.3 violation (shadowing of the ptr_
> local variable inside both {put,get}_unaligned()).
> 
> No change, not even in the compiled binary.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  xen/common/lzo.c | 11 ++---------
>  1 file changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/common/lzo.c b/xen/common/lzo.c
> index cc03f0f5547f..3454ce4a7e24 100644
> --- a/xen/common/lzo.c
> +++ b/xen/common/lzo.c
> @@ -25,15 +25,8 @@
>   */
>  
>  
> -#define COPY4(dst, src)    \
> -        put_unaligned(get_unaligned((const u32 *)(src)), (u32 *)(dst))
> -#if defined(__x86_64__)
> -#define COPY8(dst, src)    \
> -        put_unaligned(get_unaligned((const u64 *)(src)), (u64 *)(dst))
> -#else
> -#define COPY8(dst, src)    \
> -        COPY4(dst, src); COPY4((dst) + 4, (src) + 4)
> -#endif
> +#define COPY4(dst, src) memcpy(dst, src, 4)
> +#define COPY8(dst, src) memcpy(dst, src, 8)
>  
>  #ifdef __MINIOS__
>  # include <lib.h>
> -- 
> 2.30.2
> 


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:22:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:22:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727161.1131570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9agd-0004Lo-QI; Wed, 22 May 2024 01:22:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727161.1131570; Wed, 22 May 2024 01:22:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9agd-0004Lh-Ng; Wed, 22 May 2024 01:22:35 +0000
Received: by outflank-mailman (input) for mailman id 727161;
 Wed, 22 May 2024 01:22:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=X7YA=MZ=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9agc-0004LZ-Ab
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:22:34 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c2f41552-17d9-11ef-909f-e314d9c70b13;
 Wed, 22 May 2024 03:22:33 +0200 (CEST)
Received: from SJ0P220CA0017.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:41b::27)
 by PH7PR12MB7186.namprd12.prod.outlook.com (2603:10b6:510:202::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 01:22:29 +0000
Received: from MWH0EPF000971E2.namprd02.prod.outlook.com
 (2603:10b6:a03:41b:cafe::a) by SJ0P220CA0017.outlook.office365.com
 (2603:10b6:a03:41b::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Wed, 22 May 2024 01:22:29 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MWH0EPF000971E2.mail.protection.outlook.com (10.167.243.69) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 01:22:29 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 20:22:26 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 21 May
 2024 20:22:26 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 21 May 2024 20:22:25 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2f41552-17d9-11ef-909f-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EMQElXQrTCWwyvHEX4i9mvjYriTEFcYgH0RW3LMEwPdlG3lm992m0/zDj6WrO/uakFlNlzCMtjZF9fzLHVHBSUo6E5+l1iNRzCuX8z0Ns769O7nr+RczcNnQsPr3FOQWA/a88bvxOfKneR+WWPUSns6drwFNb9S9WMJajKPgPeJwyrbo4euXTSu1T/yXSWcRUxhgBU9ikOrsLkK7WsTCD5VJ3bTnkrVdjMkd5mg1E/wWSBKIYUr4750brud3D4s0Ah7hslB0sG56Tox5DREwrwJzVIitTaOwa2N0+2hjq5jzGosnJ6qrJpCiW5D9nRlwsSmNC01V14m/zKS83XZhLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ECnBaYTW+9NhBUuNxED0nph6VbLqiyh5JVId249zVRE=;
 b=NHJIMWh4PLCnlBMhs5DmnIY1X9Uc6XFKnwoTH1OW8AhhhBuIs/ZiYDjwTD2+6AD6CNJcS7oTm5HwB7bNz+hGIBpHjkeQJiOKqLcMlJDk6Dft9wxn+pYv8qZNcLaumGnMqFYwBPLhxVrIwz0ncPWvgVsNu0qgRGRtaIZlxv/D7Pni9Pn5a55dJu5+qoLn/Jug3SPAg3sfACOamaLaTrl7BYVP5/j2jYgwvBV87iKwLm74P9oIWVFLo5/adY5SM1EYRxysLkvxds9S4m8BVNeAwpH/AeUN/ZHYrzjSQuVDxDL2wJu9/Qn5nc1RCx26CMjsiMrCjGrXyIZLdDO//yEkog==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ECnBaYTW+9NhBUuNxED0nph6VbLqiyh5JVId249zVRE=;
 b=pJK4mBVolFzHn0FGYcbWDzQzqgWAYEt1J8o5eMmdwurIdPVTPf2OxZJQmyyFf1Tw/qroR6qrU3RcmqTL3RvxGLal6M//3Ko8uPLmVkYH9AB65O0hZV0o4wkrmTwrAN7X+BnLC5mKpTs3UebMPIVav54UzGKEw9d8vB/YVlALsos=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <29073700-673d-40aa-8759-efc60af3b7c0@amd.com>
Date: Wed, 22 May 2024 09:22:02 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-6-xin.wang2@amd.com>
 <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
 <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
 <alpine.DEB.2.22.394.2405211815040.1052252@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2405211815040.1052252@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E2:EE_|PH7PR12MB7186:EE_
X-MS-Office365-Filtering-Correlation-Id: 330c5ee1-21f2-4646-0e92-08dc79fda5a9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?NGpLVFNIQ3hzd3JaazlzM3UxMkI4dmFmV0ljeXhCYmNWbFozYVBSTys2SjI4?=
 =?utf-8?B?d2RrVDVKQWgxd3J6MVQ2YmtpSkVYYnhTUGVQem5KaWswZHN0blJ3T1JOUERw?=
 =?utf-8?B?VVNoK2RXRGFReGxkSS9MRkY4VzQwaThBZ0tMcUs4S2NlS3FQNUU1QWhuUnVB?=
 =?utf-8?B?K3U5azdBRnMwZmszRzdMWHdNRWxBMnBCWmgwNm81eENJZTdzK3BrMTFreVNV?=
 =?utf-8?B?RGtJVnd1bUxsRE1LWHhmcXhyTGlPQ2NUVkQ0R2ZmcHBLWHVEbkQ2MTVNMWpP?=
 =?utf-8?B?NzVIczkzTXZCVm0xK2R6NmMyd3k5WWVTMlkwd2sxUG9MMllaUWpSTlRsNllP?=
 =?utf-8?B?MDNpTzBkME9yYkRkdUdsaDY2N0l5RHgxSDFhL200MmxhV0RoZmhvNTNJbVFw?=
 =?utf-8?B?V1RLdjg0ODZtWUJvVzNRWVRYU2tRd3FHanZsanVUbzJWM0VjV0JpaFluQjVk?=
 =?utf-8?B?ZTltRkZjVGRYVXY2VldkdEQ3MnpnREJXMUZBYUFSQUlKZGo5UW9OVjZoNzhv?=
 =?utf-8?B?NXA5L3J2bnZLUDVMZnZOcGY1VENaZUJUSlJDUFpoT0wxanJ0RUFYcVdobis2?=
 =?utf-8?B?V3FEUlFDWkNJTjJWWHlMQTB5ajlDV3NWYXJxdEE4Q0FRNG8wTWZNYm9TZjM4?=
 =?utf-8?B?L1ovY1lpUFVDL3pQejlaMUx3R1ZUcnFQSWFGM2FINTRIdUd1SHZtQW8zdHEx?=
 =?utf-8?B?dHNoc0t1elNmYzlhUEljalZFSjNHM1pBL3RGQ2ptbE82amF1UURML1FUZWZO?=
 =?utf-8?B?TjRuVys1eThyaWhIdEc5aTdWUEhWcXNxUVN1cTRIUGlra3dPbkVLS3dzVnFz?=
 =?utf-8?B?Y2hvNmVTUHlDZ0tNQ2tqbVFLZlFWQjQ1bm45TTlQSFlwaTRUQmxoTVZiMWsr?=
 =?utf-8?B?WWlBMFl5M25PQVIwTjhHakYzRUVRcjVIWEwrRFNLenpqSFh4QWpvN1U3R29F?=
 =?utf-8?B?TzlQVnhIcWprWkYzYklqRzJOMnZ2SVl2ajdYcWVLaXN3eTFxVHkyeTR5RDRy?=
 =?utf-8?B?YnZjVmMzUEN3WGpQM3JPVlNiVUJ4NXJhdU9WVk5MMThLU3ZCNk1QU2JLWjg3?=
 =?utf-8?B?bEk5a1ZRd0d3bmRxVnZDUjU1TC9xMndwUDF0Y0l5NFpYRGxUMG4zalIvVnl6?=
 =?utf-8?B?SlpTT1dQS3RqeUZPRjErekgrdmllc3lmWTlzZDdPWmd0T2JHc3lpcXd5bzJZ?=
 =?utf-8?B?M1d5cTN4RU1WOHMwS0daN0FUa0pKRytWZVl6anc3NjEzR2FSTEF6TW5tbGJ2?=
 =?utf-8?B?bTY0d1VVUU9TaHhCK09nNlM4MkN2aTEvMlN6MlduNHZZYmNNYlp4blNBZnNp?=
 =?utf-8?B?R2JmMmtFQittbmlXS0RRcFlqUXQrYlY2UzBwdEhHSkFGK3MxWGN6eElSUCs1?=
 =?utf-8?B?dzRYTk51V24zS3RhbDhkb24zQURiZ1Q1b0s3R1VBYXRqWW9LWXFSWHZpRWor?=
 =?utf-8?B?U0lPaHZJejNZZWZIQ2g3OTFHc0J0YjVId1VOa2trM09QN3JyN293N3dZbksv?=
 =?utf-8?B?c0p5dE9RTk5TdDRrUW04alFqMlBNWjJ2ZW5WMzRscXFydzd6MWZYd2RqKyti?=
 =?utf-8?B?dFpzeFZ0MXhzWkorMW82ZGR6eUg2dGluOWgwL0F1KzhpcVlnZG1xTkVDOExp?=
 =?utf-8?B?QmpBaXMrQm5nc2taQXBNQVBPS1VWVGtPZWV1by9IMVFwek5OWVd3UUlYeHFj?=
 =?utf-8?B?WXIvcG5NZ1NkUUZWMklMamdFK1FBek92Y0lxNzI3eVJ0VGFldWd0Y0dwcVVV?=
 =?utf-8?B?Z0UxZWpsQlVRaGo4TDJnajJubU5Cb2k4eHZ2RHI4NStwMnJYK25DYWhVaDhs?=
 =?utf-8?Q?fUGyGvz6XWjXGXd/RQP4LKoYRVzDyOIXuImBI=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 01:22:29.1368
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 330c5ee1-21f2-4646-0e92-08dc79fda5a9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E2.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7186

Hi Stefano,

On 5/22/2024 9:16 AM, Stefano Stabellini wrote:
> On Wed, 22 May 2024, Henry Wang wrote:
>> Hi Julien,
>>
>> On 5/21/2024 8:30 PM, Julien Grall wrote:
>>> Hi,
>>>
>>> On 21/05/2024 05:35, Henry Wang wrote:
>>>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>>>> index 56490dbc43..956c11ba13 100644
>>>> --- a/xen/arch/arm/gic-vgic.c
>>>> +++ b/xen/arch/arm/gic-vgic.c
>>>> @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct
>>>> vcpu *v, unsigned int virq,
>>>>          /* We are taking to rank lock to prevent parallel connections. */
>>>>        vgic_lock_rank(v_target, rank, flags);
>>>> +    spin_lock(&v_target->arch.vgic.lock);
>>> I know this is what Stefano suggested, but v_target would point to the
>>> current affinity whereas the interrupt may be pending/active on the
>>> "previous" vCPU. So it is a little unclear whether v_target is the correct
>>> lock. Do you have more pointer to show this is correct?
>> No I think you are correct, we have discussed this in the initial version of
>> this patch. Sorry.
>>
>> I followed the way from that discussion to note down the vcpu ID and retrieve
>> here, below is the diff, would this make sense to you?
>>
>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>> index 956c11ba13..134ed4e107 100644
>> --- a/xen/arch/arm/gic-vgic.c
>> +++ b/xen/arch/arm/gic-vgic.c
>> @@ -439,7 +439,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v,
>> unsigned int virq,
>>
>>       /* We are taking to rank lock to prevent parallel connections. */
>>       vgic_lock_rank(v_target, rank, flags);
>> -    spin_lock(&v_target->arch.vgic.lock);
>> + spin_lock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
>>
>>       if ( connect )
>>       {
>> @@ -465,7 +465,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v,
>> unsigned int virq,
>>               p->desc = NULL;
>>       }
>>
>> -    spin_unlock(&v_target->arch.vgic.lock);
>> + spin_unlock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
>>       vgic_unlock_rank(v_target, rank, flags);
>>
>>       return ret;
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>> index 79b73a0dbb..f4075d3e75 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -85,6 +85,7 @@ struct pending_irq
>>       uint8_t priority;
>>       uint8_t lpi_priority;       /* Caches the priority if this is an LPI. */
>>       uint8_t lpi_vcpu_id;        /* The VCPU for an LPI. */
>> +    uint8_t spi_vcpu_id;        /* The VCPU for an SPI. */
>>       /* inflight is used to append instances of pending_irq to
>>        * vgic.inflight_irqs */
>>       struct list_head inflight;
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index c04fc4f83f..e852479f13 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -632,6 +632,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v,
>> unsigned int virq,
>>       }
>>       list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>>   out:
>> +    n->spi_vcpu_id = v->vcpu_id;
>>       spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>>
>>       /* we have a new higher priority irq, inject it into the guest */
>>       vcpu_kick(v);
>>
>>
>>> Also, while looking at the locking, I noticed that we are not doing anything
>>> with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem to assume that
>>> if the flag is set, then p->desc cannot be NULL.
>>>
>>> Can we reach vgic_connect_hw_irq() with the flag set?
>> I think even from the perspective of making the code extra safe, we should
>> also check GIC_IRQ_GUEST_MIGRATING as the LR is allocated for this case. I
>> will also add the check of GIC_IRQ_GUEST_MIGRATING here.
> Yes. I think it might be easier to check for GIC_IRQ_GUEST_MIGRATING
> early and return error immediately in that case. Otherwise, we can
> continue and take spin_lock(&v_target->arch.vgic.lock) because no
> migration is in progress

Ok, this makes sense to me, I will add

     if( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
     {
         vgic_unlock_rank(v_target, rank, flags);
         return -EBUSY;
     }

right after taking the vgic rank lock.

Kind regards,
Henry


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:24:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:24:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727166.1131580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aiF-0004ss-53; Wed, 22 May 2024 01:24:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727166.1131580; Wed, 22 May 2024 01:24:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9aiF-0004sl-2B; Wed, 22 May 2024 01:24:15 +0000
Received: by outflank-mailman (input) for mailman id 727166;
 Wed, 22 May 2024 01:24:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9aiD-0004sf-Qe
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:24:13 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fcca2845-17d9-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 03:24:11 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 03E96CE12AE;
 Wed, 22 May 2024 01:24:05 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95240C2BD11;
 Wed, 22 May 2024 01:24:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcca2845-17d9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716341044;
	bh=AT/tYDDNkLHfc9e7JNd+ZKDqKmkYDThuOaWKNlBw/a4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ELi8m9r1lumX00sDOH5OIPe0gQsBzrBlUDy5Hiq8q7EDZ2yJVn4NuO26UJnyMJOD8
	 nD0AfEqAEihcZe5f3UuniL1DYX/11EicxYtm+rKPZX8lPr0pfS4S+Jvqt1yPdhQyzm
	 npyhpp1vkx8/VUcNwZVAjPMm3dTtspzhwT2wyteiPhfxjoxus8NeYKyZ12bV4S9wgS
	 dUTOCunaoL3edH2oqnH7LxmdoVLwqprbNoZs4p+8S4TTLlcLx5IFkNiaPMw4ulJIt4
	 CkvIFMvyDYyKx+3Caiabq7uqAYyOEVVoDeMqnASjDUFRxNznyEfyZaBijkbLWimcyy
	 LbjzuniqYBxOg==
Date: Tue, 21 May 2024 18:24:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Roberto Bagnara <roberto.bagnara@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 2/3] xen/x86: Drop useless non-Kconfig CONFIG_*
 variables
In-Reply-To: <20240521171546.3650664-3-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405211823540.1052252@ubuntu-linux-20-04-desktop>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com> <20240521171546.3650664-3-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Andrew Cooper wrote:
> These are all either completely unused, or do nothing useful.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  xen/arch/x86/include/asm/config.h |  4 ----
>  xen/include/xen/acpi.h            |  9 ---------
>  xen/include/xen/watchdog.h        | 11 -----------
>  3 files changed, 24 deletions(-)
> 
> diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
> index ab7288cb3682..24b005ba1ff7 100644
> --- a/xen/arch/x86/include/asm/config.h
> +++ b/xen/arch/x86/include/asm/config.h
> @@ -20,7 +20,6 @@
>  #define BITS_PER_XEN_ULONG BITS_PER_LONG
>  
>  #define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS 1
> -#define CONFIG_DISCONTIGMEM 1
>  #define CONFIG_NUMA_EMU 1
>  
>  #define CONFIG_PAGEALLOC_MAX_ORDER (2 * PAGETABLE_ORDER)
> @@ -30,11 +29,8 @@
>  /* Intel P4 currently has largest cache line (L2 line size is 128 bytes). */
>  #define CONFIG_X86_L1_CACHE_SHIFT 7
>  
> -#define CONFIG_ACPI_SRAT 1
>  #define CONFIG_ACPI_CSTATE 1
>  
> -#define CONFIG_WATCHDOG 1
> -
>  #define CONFIG_MULTIBOOT 1
>  
>  #define HZ 100
> diff --git a/xen/include/xen/acpi.h b/xen/include/xen/acpi.h
> index e78e7e785252..bc4818c9430a 100644
> --- a/xen/include/xen/acpi.h
> +++ b/xen/include/xen/acpi.h
> @@ -140,15 +140,6 @@ int get_cpu_id(u32 acpi_id);
>  unsigned int acpi_register_gsi (u32 gsi, int edge_level, int active_high_low);
>  int acpi_gsi_to_irq (u32 gsi, unsigned int *irq);
>  
> -/*
> - * This function undoes the effect of one call to acpi_register_gsi().
> - * If this matches the last registration, any IRQ resources for gsi
> - * are freed.
> - */
> -#ifdef CONFIG_ACPI_DEALLOCATE_IRQ
> -void acpi_unregister_gsi (u32 gsi);
> -#endif
> -
>  #ifdef	CONFIG_ACPI_CSTATE
>  /*
>   * max_cstate sets the highest legal C-state.
> diff --git a/xen/include/xen/watchdog.h b/xen/include/xen/watchdog.h
> index 86fde0884ae7..434fcbdd2b11 100644
> --- a/xen/include/xen/watchdog.h
> +++ b/xen/include/xen/watchdog.h
> @@ -9,8 +9,6 @@
>  
>  #include <xen/types.h>
>  
> -#ifdef CONFIG_WATCHDOG
> -
>  /* Try to set up a watchdog. */
>  int watchdog_setup(void);
>  
> @@ -23,13 +21,4 @@ void watchdog_disable(void);
>  /* Is the watchdog currently enabled. */
>  bool watchdog_enabled(void);
>  
> -#else
> -
> -#define watchdog_setup() ((void)0)
> -#define watchdog_enable() ((void)0)
> -#define watchdog_disable() ((void)0)
> -#define watchdog_enabled() ((void)0)
> -
> -#endif
> -
>  #endif /* __XEN_WATCHDOG_H__ */
> -- 
> 2.30.2
> 


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:27:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:27:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727173.1131591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9al3-0005Wx-Mn; Wed, 22 May 2024 01:27:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727173.1131591; Wed, 22 May 2024 01:27:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9al3-0005Wq-Ix; Wed, 22 May 2024 01:27:09 +0000
Received: by outflank-mailman (input) for mailman id 727173;
 Wed, 22 May 2024 01:27:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9al2-0005Wk-B1
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:27:08 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 65b05b36-17da-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 03:27:05 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 38269623C9;
 Wed, 22 May 2024 01:27:04 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AC74C2BD11;
 Wed, 22 May 2024 01:27:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65b05b36-17da-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716341223;
	bh=TXldyTLK3Cf+rlFk9PEM4RWOf3bbRQQ10lumRKf+fxI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=D16D0ZhuBkylC0MJKZRPVTcDlcRqlBRq35ROAI8JyxQSQ4147ZWe0lg3xaOrHfk1O
	 TvBbdKGKLWPcbIL0VtpQFZNNIiLO9SIhhKXqfaAQ8BpjUHFX16CqmmnCcm/zyb9Vvs
	 4wCRJ5X714+JwVlXcwlzOYbZtRReNW9Io9irSilfF9Fqcs9SfXIXfCMNI9tRXoU/yz
	 exiJ0TVR4GFGjtszgEwrPksQPesElWY9BYF0eeGPiHjxDAAmL/CefX2/xIxWBCV9SR
	 MK0SwH1/Ik1kxozTPYo3iI45PkspsBKu3c80FMK5GrU0Bud1cph0JS4594JZc/fvyv
	 XnkfhrMZykK7A==
Date: Tue, 21 May 2024 18:26:59 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Roberto Bagnara <roberto.bagnara@bugseng.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 3/3] xen/x86: Address two misc MISRA 17.7 violations
In-Reply-To: <20240521171546.3650664-4-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405211826520.1052252@ubuntu-linux-20-04-desktop>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com> <20240521171546.3650664-4-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Andrew Cooper wrote:
> Neither text_poke() nor watchdog_setup() have return value consulted.  Switch
> them to being void.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Roberto Bagnara <roberto.bagnara@bugseng.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  xen/arch/x86/alternative.c | 4 ++--
>  xen/arch/x86/nmi.c         | 5 ++---
>  xen/include/xen/watchdog.h | 2 +-
>  3 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c
> index 2e7ba6e0b833..7824053c9d33 100644
> --- a/xen/arch/x86/alternative.c
> +++ b/xen/arch/x86/alternative.c
> @@ -155,10 +155,10 @@ void init_or_livepatch add_nops(void *insns, unsigned int len)
>   * "noinline" to cause control flow change and thus invalidate I$ and
>   * cause refetch after modification.
>   */
> -static void *init_or_livepatch noinline
> +static void init_or_livepatch noinline
>  text_poke(void *addr, const void *opcode, size_t len)
>  {
> -    return memcpy(addr, opcode, len);
> +    memcpy(addr, opcode, len);
>  }
>  
>  extern void *const __initdata_cf_clobber_start[];
> diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
> index f6329cb0270e..9793fa23168d 100644
> --- a/xen/arch/x86/nmi.c
> +++ b/xen/arch/x86/nmi.c
> @@ -464,12 +464,12 @@ bool watchdog_enabled(void)
>      return !atomic_read(&watchdog_disable_count);
>  }
>  
> -int __init watchdog_setup(void)
> +void __init watchdog_setup(void)
>  {
>      unsigned int cpu;
>  
>      /*
> -     * Activate periodic heartbeats. We cannot do this earlier during 
> +     * Activate periodic heartbeats. We cannot do this earlier during
>       * setup because the timer infrastructure is not available.
>       */
>      for_each_online_cpu ( cpu )
> @@ -477,7 +477,6 @@ int __init watchdog_setup(void)
>      register_cpu_notifier(&cpu_nmi_nfb);
>  
>      watchdog_enable();
> -    return 0;
>  }
>  
>  /* Returns false if this was not a watchdog NMI, true otherwise */
> diff --git a/xen/include/xen/watchdog.h b/xen/include/xen/watchdog.h
> index 434fcbdd2b11..40c21bca823f 100644
> --- a/xen/include/xen/watchdog.h
> +++ b/xen/include/xen/watchdog.h
> @@ -10,7 +10,7 @@
>  #include <xen/types.h>
>  
>  /* Try to set up a watchdog. */
> -int watchdog_setup(void);
> +void watchdog_setup(void);
>  
>  /* Enable the watchdog. */
>  void watchdog_enable(void);
> -- 
> 2.30.2
> 


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:33:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:33:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727183.1131601 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ara-0007Yk-D8; Wed, 22 May 2024 01:33:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727183.1131601; Wed, 22 May 2024 01:33:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ara-0007Yc-8a; Wed, 22 May 2024 01:33:54 +0000
Received: by outflank-mailman (input) for mailman id 727183;
 Wed, 22 May 2024 01:33:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9arZ-0007YW-F9
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 01:33:53 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 558b7ffe-17db-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 03:33:49 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 1CE2BCE1130;
 Wed, 22 May 2024 01:33:43 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 57BA7C2BD11;
 Wed, 22 May 2024 01:33:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 558b7ffe-17db-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716341622;
	bh=N55FJc/Lz+jmrcQsDAsZ7pnxvDkaU5y21y6LJZoUAuA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ahUEqemRMBsz59wS/Dj3QbN9nUm0pHe13pVfPgsW/7DQTSLDVTYGnmquZ3n2wkpX5
	 hyP78Ad5sFJpiYSYqOW83eVlGNwTj+i30jyfDeGMoLETCqpYiGiS8BeTz9HB7rvvW2
	 DGSp1z3F2nfk8EkC7V1eJVQAc2y+LnzmS5iqRSrAmJlQYtpAbdhOugyipIM9V9LkNJ
	 o6XhkYaWKzQG5ob8ylF6medXTb1DCGUebUJDvOz1dJ7JjFcS2XTWIYHtZdgMYZBFz3
	 fFJnqfkwXHaeegsSv2XHLp0+EJ5WtEX1a9biAwSsFT7BEc6v8wnI2+kdqAyqeGSu7B
	 nSwXwXoZmE7zQ==
Date: Tue, 21 May 2024 18:33:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <George.Dunlap@citrix.com>, 
    Daniel Smith <dpsmith@apertussolutions.com>, Julien Grall <julien@xen.org>, 
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] Revert "evtchn: refuse EVTCHNOP_status for Xen-bound
 event channels"
In-Reply-To: <3cfee629-b9bc-4b3d-9e4e-ff7e54e701d1@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405211832410.1052252@ubuntu-linux-20-04-desktop>
References: <20240402170612.2477791-1-andrew.cooper3@citrix.com> <11957460-0b2b-432d-ad92-38350306c9ff@suse.com> <b14acc58-5d3c-46ed-a300-61f422501d4c@suse.com> <b3695ec1-b9e6-4db6-a242-5dfa11bc00b6@suse.com> <alpine.DEB.2.22.394.2405161806560.2544314@ubuntu-linux-20-04-desktop>
 <e75ca24e-12fe-44ac-9c67-bcc222ac8752@suse.com> <alpine.DEB.2.22.394.2405171328250.1052252@ubuntu-linux-20-04-desktop> <3cfee629-b9bc-4b3d-9e4e-ff7e54e701d1@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Jan Beulich wrote:
> On 17.05.2024 22:28, Stefano Stabellini wrote:
> > On Fri, 17 May 2024, Jan Beulich wrote:
> >> On 17.05.2024 03:21, Stefano Stabellini wrote:
> >>> On Thu, 16 May 2024, Jan Beulich wrote:
> >>>> 1) In the discussion George claimed that exposing status information in
> >>>> an uncontrolled manner is okay. I'm afraid I have to disagree, seeing
> >>>> how a similar assumption by CPU designers has led to a flood of
> >>>> vulnerabilities over the last 6+ years. Information exposure imo is never
> >>>> okay, unless it can be _proven_ that absolutely nothing "useful" can be
> >>>> inferred from it. (I'm having difficulty seeing how such a proof might
> >>>> look like.)
> >>>
> >>> Many would agree that it is better not to expose status information in
> >>> an uncontrolled manner. Anyway, let's focus on the actionable.
> >>>
> >>>
> >>>> 2) Me pointing out that the XSM hook might similarly get in the way of
> >>>> debugging, Andrew suggested that this is not an issue because any sensible
> >>>> XSM policy used in such an environment would grant sufficient privilege to
> >>>> Dom0. Yet that then still doesn't cover why DomU-s also can obtain status
> >>>> for Xen-internal event channels. The debugging argument then becomes weak,
> >>>> as in that case the XSM hook is possibly going to get in the way.
> >>>>
> >>>> 3) In the discussion Andrew further gave the impression that evtchn_send()
> >>>> had no XSM check. Yet it has; the difference to evtchn_status() is that
> >>>> the latter uses XSM_TARGET while the former uses XSM_HOOK. (Much like
> >>>> evtchn_status() may indeed be useful for debugging, evtchn_send() may be
> >>>> similarly useful to allow getting a stuck channel unstuck.)
> >>>>
> >>>> In summary I continue to think that an outright revert was inappropriate.
> >>>> DomU-s should continue to be denied status information on Xen-internal
> >>>> event channels, unconditionally and independent of whether dummy, silo, or
> >>>> Flask is in use.
> >>>
> >>> I think DomU-s should continue to be denied status information on
> >>> Xen-internal event channels *based on the default dummy, silo, or Flask
> >>> policy*. It is not up to us to decide the security policy, only to
> >>> enforce it and provide sensible defaults.
> >>>
> >>> In any case, the XSM_TARGET check in evtchn_status seems to do what we
> >>> want?
> >>
> >> No. XSM_TARGET permits the "owning" (not really, but it's its table) domain
> >> access. See xsm_default_action() in xsm/dummy.h.
> > 
> > Sorry I still don't understand. Why is that a problem? It looks like the
> > wanted default behavior?
> 
> For ordinary event channels - yes. But not for Xen-internal ones, at least
> from my pov.

I understand your point of view, but in my opinion it is OK


From xen-devel-bounces@lists.xenproject.org Wed May 22 01:54:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 01:54:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727192.1131611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9bBK-0002dj-TH; Wed, 22 May 2024 01:54:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727192.1131611; Wed, 22 May 2024 01:54:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9bBK-0002dc-Q7; Wed, 22 May 2024 01:54:18 +0000
Received: by outflank-mailman (input) for mailman id 727192;
 Wed, 22 May 2024 01:54:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9bBJ-0002dS-Sw; Wed, 22 May 2024 01:54:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9bBJ-0000az-Pf; Wed, 22 May 2024 01:54:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9bBJ-0005TM-BA; Wed, 22 May 2024 01:54:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9bBJ-00080e-Ae; Wed, 22 May 2024 01:54:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FlllyXzHms2KhV6+u81rnkBkjsnkwaAv4am7FAsKjvY=; b=EYR0g55aaG+Yu9mLX6DypRxd0G
	Ryqvl58ffYUjc1A7mPZbG7E9WdEgnQImgCMG61TJ5Sh4AxKkxHqi+cRnEgfyUWv1z+o44pFMXVsfo
	quVjjyX3LR9U5FE2dlH9/A6TBD+RLafzV342eZn0Yh/v+BulsAjj+Z/BAh33/3LrLjx8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186063-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186063: regressions - FAIL
X-Osstest-Failures:
    xen-4.17-testing:build-arm64-xsm:xen-build:fail:regression
    xen-4.17-testing:build-amd64-xsm:xen-build:fail:regression
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=afcce3cfc2458efbd3ff9320153c534a82a108c5
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 May 2024 01:54:17 +0000

flight 186063 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186063/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm               6 xen-build                fail REGR. vs. 185864
 build-amd64-xsm               6 xen-build                fail REGR. vs. 185864

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185864
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185864
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  afcce3cfc2458efbd3ff9320153c534a82a108c5
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   22 days
Testing same since   186063  2024-05-21 10:06:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              fail    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 429 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed May 22 05:56:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 05:56:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727212.1131623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9exB-000576-NW; Wed, 22 May 2024 05:55:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727212.1131623; Wed, 22 May 2024 05:55:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9exB-00056z-Ko; Wed, 22 May 2024 05:55:57 +0000
Received: by outflank-mailman (input) for mailman id 727212;
 Wed, 22 May 2024 05:55:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9exA-00056r-D7
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 05:55:56 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f3aacc1e-17ff-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 07:55:54 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso2111453a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 22:55:54 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17f37sm1718809566b.224.2024.05.21.22.55.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 22:55:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3aacc1e-17ff-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716357354; x=1716962154; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LePKoXr0FLblhlhUauuhiDI0SW2kKNtKN3dtQv1Flfo=;
        b=BVjbZyUtDpJ9nnfYj2yQdMgG51swxb8HdqOHZGrwxB0ALKrubJW9NaykqWjVp8B5jL
         WCaVPzOFl6T7OIuZzYtZMiDNMIky/ixZvklavBmqAUkBwQpFBm5rIAXVLMgMpz9OfdpM
         WLDppAGj12xpzZhFvDQABcsG1t80bvgYiHaIKIkwi/cbCHjMjcDNlGvC8D8ZjZOdwRBu
         Ox92baWmlIdXIDFvTan3J/UYY9Z3h7p6SyxHGGeE3Gpb/u5WyysKzzLwvl2cnXxNP1zU
         07N1XlvT8DndRGOxLpVeuVCSws+CvmAGuhmOpT8Y5tS3oGBoni34N4qt/8V4lRe4/L/5
         zljw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716357354; x=1716962154;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LePKoXr0FLblhlhUauuhiDI0SW2kKNtKN3dtQv1Flfo=;
        b=JmMRMnCCfPp4lyt+fYA0zQ6nPQ0VxGw+VfSHfd2cT0esLglY27/P1Q7H3QxCs9yPHN
         VEy8bJiHqRZ/YGbwBeyH06Xo3EO1ykrzUqnKqTf2XfdgBQ5xiPJ0wIDH289VCd3d+3dR
         MTczdxurym9F2Slc3Z4HGddRY8P+B8kvCNdoEtmaOYFimN6Lao9NR5B4r/3BPI6luW8/
         wQXEz3f6xvevVjI29CJIiGoDpasN/nmgCtwo2HEfpEBj/Uh4CuWTtrrtlL15MQwVnBSG
         HHERHa/WBPiF2N/+mHDZ+OncJZG0XnJqiSID2eYKNPN7pcXqhEwsJKP63YSWwkFUYl1+
         lh9w==
X-Forwarded-Encrypted: i=1; AJvYcCWnzizW47wEpuIogNryTTqEyhg9STi0JkBE79uawOUomcnTrQ1nY+5U9Ro2QRGpG+HVgNJbhl/ZHyZd6VzUeIUQ1jwFPzvsJ1oCRwp3xV8=
X-Gm-Message-State: AOJu0YwNJ4S1mOn525vx8g5sEkLLqka3Bq2vDsGYTAIMI4NmmeDUOJfj
	tXHnd+W3GUGBSdy1tmpdLrgPcEtiQevxWGuRyTwQ7WRVeVdc2U8x1K/JzSujcA==
X-Google-Smtp-Source: AGHT+IEcx7Ep/6O5ryEQJBBnbjc5YGxzDB+UpytZQa/jmueNtrdtr8NSLOzRDrok5Fe7PNE1eDf5Hg==
X-Received: by 2002:a17:906:374e:b0:a59:ba2b:590e with SMTP id a640c23a62f3a-a62281cb3e8mr45481266b.48.1716357353843;
        Tue, 21 May 2024 22:55:53 -0700 (PDT)
Message-ID: <6115ebe4-0554-48b6-bf4f-8370e9a9f539@suse.com>
Date: Wed, 22 May 2024 07:55:52 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <245e535a-5b23-4d3e-83e5-dc797068652c@suse.com> <Zky-4C2WOTboJFLb@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zky-4C2WOTboJFLb@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 17:33, Marek Marczykowski-Górecki wrote:
> On Tue, May 21, 2024 at 05:16:58PM +0200, Jan Beulich wrote:
>> On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
>>> --- a/xen/arch/x86/include/asm/mm.h
>>> +++ b/xen/arch/x86/include/asm/mm.h
>>> @@ -522,9 +522,27 @@ extern struct rangeset *mmio_ro_ranges;
>>>  void memguard_guard_stack(void *p);
>>>  void memguard_unguard_stack(void *p);
>>>  
>>> +/*
>>> + * Add more precise r/o marking for a MMIO page. Range specified here
>>> + * will still be R/O, but the rest of the page (not marked as R/O via another
>>> + * call) will have writes passed through.
>>> + * The start address and the size must be aligned to SUBPAGE_MMIO_RO_ALIGN.
>>> + *
>>> + * This API cannot be used for overlapping ranges, nor for pages already added
>>> + * to mmio_ro_ranges separately.
>>> + *
>>> + * Return values:
>>> + *  - negative: error
>>> + *  - 0: success
>>> + */
>>> +#define SUBPAGE_MMIO_RO_ALIGN 8
>>
>> This isn't just alignment, but also (and perhaps more importantly) granularity.
>> I think the name wants to express this.
> 
> SUBPAGE_MMIO_RO_GRANULARITY? Sounds a bit long...

..._GRAN? ..._CHUNK? ..._UNIT? (Perhaps also want to have MMIO_RO_ first.)

>>> +static int __init subpage_mmio_ro_add_page(
>>> +    mfn_t mfn, unsigned int offset_s, unsigned int offset_e)
>>> +{
>>> +    struct subpage_ro_range *entry = NULL, *iter;
>>> +    unsigned int i;
>>> +
>>> +    list_for_each_entry(iter, &subpage_ro_ranges, list)
>>> +    {
>>> +        if ( mfn_eq(iter->mfn, mfn) )
>>> +        {
>>> +            entry = iter;
>>> +            break;
>>> +        }
>>> +    }
>>> +    if ( !entry )
>>> +    {
>>> +        /* iter == NULL marks it was a newly allocated entry */
>>> +        iter = NULL;
>>> +        entry = xzalloc(struct subpage_ro_range);
>>> +        if ( !entry )
>>> +            return -ENOMEM;
>>> +        entry->mfn = mfn;
>>> +    }
>>> +
>>> +    for ( i = offset_s; i <= offset_e; i += SUBPAGE_MMIO_RO_ALIGN )
>>> +    {
>>> +        int oldbit = __test_and_set_bit(i / SUBPAGE_MMIO_RO_ALIGN,
>>> +                                        entry->ro_qwords);
>>
>> Why int, not bool?
> 
> Because __test_and_set_bit returns int. But I can change to bool if you
> prefer.

test_bit() and the test-and set of functions should eventually all change
to be returning bool. One less use site to then also take care of.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 06:03:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 06:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727217.1131634 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9f4m-0006ld-FR; Wed, 22 May 2024 06:03:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727217.1131634; Wed, 22 May 2024 06:03:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9f4m-0006lW-CK; Wed, 22 May 2024 06:03:48 +0000
Received: by outflank-mailman (input) for mailman id 727217;
 Wed, 22 May 2024 06:03:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9f4k-0006lO-PG
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 06:03:46 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0c74b417-1801-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 08:03:45 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-572baf393ddso839414a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 May 2024 23:03:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c323d3asm17657662a12.90.2024.05.21.23.03.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 May 2024 23:03:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c74b417-1801-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716357825; x=1716962625; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=owceRJhufTWT1Cdxp0koOg0CpKeWWSqtqNgL/ikKK/U=;
        b=bxc7Q1ErL2OOC5lsVYvYNH5pPy8sM4fnJvFG3a1etu/loxNcUmP0v/TbUifpSb0hUh
         w7k1HZAv/Yr2qM7WhfMHuueiTYF0XXYJNM+RaOmoUgkf5n2FQoDV67VZ5NGOkCWA0dJO
         zurea0qGukDSYKcs9yhteYXey7TZkjinJESo2De6inggbEHZTck9yXqB2AGrGi0is1KJ
         Einrw/Fu9qTJp4NcGStzvvlypIbcbVz2CNWpSkoPg5+6lvWKXYL3uR5Ae++DYgqfQ7hN
         ckNjg9/OFeW8ljmm3LmxfyuZsHf7BJHoXvv7fd61aPEihzhxVBGxc4fyCeG1zf8LBYol
         0Rkw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716357825; x=1716962625;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=owceRJhufTWT1Cdxp0koOg0CpKeWWSqtqNgL/ikKK/U=;
        b=S459ijuuzD5KlT6N2kGs26PCyABHitYTV4/5w2zGDLSQxo/b1I98Zd/AwIXa/bw9uz
         y2j9S6g2lHSOK2n2lRh5D5/MAAIlkk6C6a9MZDKL1lPx7M1snGxYg0p7+nM2m7MbdU9z
         ms51AUcR1H7hHlbD+orJpCJpH1Dll/iXHN+F++TaBrp3VW5vYWs7we7xFVqUPjV4/iTZ
         cCN5HssjM3Lynp1QbQMQJRnk91jzdDZD2f2nKRIZnhxFwTfvLSLPCQhAD/azrwZ8z4Kd
         GuA/BihKLs3DToVD0X3wbgdi596aBlTZN5c24vNQbZMabp9grMIF26reufpAQGg+4bQv
         2rrg==
X-Forwarded-Encrypted: i=1; AJvYcCUCDgWlAZ/Cw0BLQpCOo79mZyojcwTI+jOqSreLfQtWoIfXtcRlnOYvzv1OGgxf++qG+39fZpM1uJ6DWRY2GTNY65V5j+FnB/kf+LlYsDI=
X-Gm-Message-State: AOJu0YyyQXk9u0+Y6kWiQxcDNeg9MA4wHvAHpIRyPqual10iLHliqd2W
	cfcOYcdMNH+IOc7nLrvgEHUy6/ZAj669b9gGiIXgbnqTL/Vd5B6GMaTvNAfARg==
X-Google-Smtp-Source: AGHT+IGdxbsnZf2VU8XWuzwBw0Q0tZpRj6bZBG/9eBNTgPMW+6AUBUya80OXf7lHGdBzcab3MrPC+Q==
X-Received: by 2002:a50:8d5e:0:b0:572:2fdf:b965 with SMTP id 4fb4d7f45d1cf-5752b432b79mr10704205a12.7.1716357824777;
        Tue, 21 May 2024 23:03:44 -0700 (PDT)
Message-ID: <bd0fe385-73a4-4f8f-b7b0-28acc9078f40@suse.com>
Date: Wed, 22 May 2024 08:03:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] xen/x86: Drop useless non-Kconfig CONFIG_* variables
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Roberto Bagnara <roberto.bagnara@bugseng.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
 <20240521171546.3650664-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240521171546.3650664-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.05.2024 19:15, Andrew Cooper wrote:
> These are all either completely unused, or do nothing useful.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Not an objection, i.e. you're fine to commit as is with Stefano's R-b, yet
still a question:

> @@ -30,11 +29,8 @@
>  /* Intel P4 currently has largest cache line (L2 line size is 128 bytes). */
>  #define CONFIG_X86_L1_CACHE_SHIFT 7
>  
> -#define CONFIG_ACPI_SRAT 1
>  #define CONFIG_ACPI_CSTATE 1
>  
> -#define CONFIG_WATCHDOG 1

I wonder if this wouldn't make sense to become a proper Kconfig setting,
thus ...

> --- a/xen/include/xen/watchdog.h
> +++ b/xen/include/xen/watchdog.h
> @@ -9,8 +9,6 @@
>  
>  #include <xen/types.h>
>  
> -#ifdef CONFIG_WATCHDOG
> -
>  /* Try to set up a watchdog. */
>  int watchdog_setup(void);
>  
> @@ -23,13 +21,4 @@ void watchdog_disable(void);
>  /* Is the watchdog currently enabled. */
>  bool watchdog_enabled(void);
>  
> -#else
> -
> -#define watchdog_setup() ((void)0)
> -#define watchdog_enable() ((void)0)
> -#define watchdog_disable() ((void)0)
> -#define watchdog_enabled() ((void)0)
> -
> -#endif

... assigning purpose to these stubs.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 06:27:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 06:27:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727225.1131644 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9fRP-0001Ca-6X; Wed, 22 May 2024 06:27:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727225.1131644; Wed, 22 May 2024 06:27:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9fRP-0001CT-36; Wed, 22 May 2024 06:27:11 +0000
Received: by outflank-mailman (input) for mailman id 727225;
 Wed, 22 May 2024 06:27:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Dzh3=MZ=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1s9fRN-0001CN-VZ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 06:27:10 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01on062c.outbound.protection.outlook.com
 [2a01:111:f400:fe1f::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4fe5b250-1804-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 08:27:07 +0200 (CEST)
Received: from AM0PR02CA0013.eurprd02.prod.outlook.com (2603:10a6:208:3e::26)
 by DB9PR08MB6633.eurprd08.prod.outlook.com (2603:10a6:10:23d::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 06:27:04 +0000
Received: from AMS1EPF0000004B.eurprd04.prod.outlook.com
 (2603:10a6:208:3e:cafe::78) by AM0PR02CA0013.outlook.office365.com
 (2603:10a6:208:3e::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 06:27:04 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS1EPF0000004B.mail.protection.outlook.com (10.167.16.136) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Wed, 22 May 2024 06:27:04 +0000
Received: ("Tessian outbound ba75727f6dca:v315");
 Wed, 22 May 2024 06:27:03 +0000
Received: from d881921c12bc.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E55CA698-EF52-4777-8797-632A1869EC93.1; 
 Wed, 22 May 2024 06:26:57 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d881921c12bc.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 May 2024 06:26:57 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by PAXPR08MB6478.eurprd08.prod.outlook.com (2603:10a6:102:159::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.18; Wed, 22 May
 2024 06:26:52 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7587.035; Wed, 22 May 2024
 06:26:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fe5b250-1804-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=Tj+ICPQDdeOcvfW0ROMbV+YmydkT56QD1hc/xW32T3ISNX2Vc6wRh/jb4uoEnzDuADSzqQap6oS+8anh2LXokEwyG9UuMEu+cyr+3vjrbMC2XJPeWpW65AYDkBxOATlgGaYMaPWHllGAEeJOATKu3G3zN1Wr0UxsDfexlSBh/SjPjE7bo6j/gXDfCMXINrZ/jKGC70UWgHz5mcm5jKlkQGpavfyBAHS7Cxvvf/EMwMIWtZLkcO8tt76TqoI093ZnkztgZVMaUNBatT+dQyASPIFS1drLoub77uPnZtm529L3FYwHcNy4HwF9I49A3KWSfYcBHY8h8KBRsqy+LhGhPw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RZEpCOmpNmuIPw/Yy9KgxrXyWOZlbKX61cWq1sUfZMg=;
 b=hsRo5tKeR+QPCynKL1L7bi9B2FPbbMMXCMbDt61LEDB3RE7To/qy3BxD/HYGuPI1SxkchAtdN26yKenJ7XLXlz/cN13auRg3kI3iUiUXH4ujzfHwzjo72vSMZlRdVBR2+noJ1IpjiySZ3foBp/IINYoZdNHWaIa365KWzxpX29JICu+psSazMZhTyMd01CGN1Xsww8tbkX2TkJ+bcbQ3cxNMlHivfx03SJV5xCKPJGHg2tvFcbgOxc6ooyMb9GtF2uzEzfEQcjqwy5lMJGz6l+dBfBzm6WKiMlhQx7vGow5cix1iKihTrbilrQ7UNfCsBnKa+zspGMJ7DTdsL+g3yg==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RZEpCOmpNmuIPw/Yy9KgxrXyWOZlbKX61cWq1sUfZMg=;
 b=S8lqTr1nc7Ya5oBMP/mWgsdqgOMsyGnEmi1mRBFWFGy5OLoZi2/CzVn5GJwvlzztclUzrpbJHQjx9HnbG0hERDDngDpK8WcH6lXaoBhBV59ln0yTgSeEJRLp64iXlLlUmOlctyaKouRX6Xffw85/PvXh0W3qkzNgfowOfTxSaAE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 29cf0c3dd70de103
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VU/QvnW39OfSIjRtlk+4jdvnAicFKjQQN2iIqgmqJMMaKVP8Pwt+I3FJI3ss3oBD66RCbW818GbTsOoHQvZXgU5Qa1eIDt2c91QvDp8mCCKovXI/JkkjTOHx1hQjgFdT8925DyoLZAmOb+3UvYO3HyeX9jZMiALAkKsKqHjF3PQCFNQ4RI2NKM0wzBWQMK/RpC4uG4W6VmMNWVOgZumiC1qRNY26xaJ6Eo1kJhu2ZVvSsHAU9A07v5ejRlC8UN9F/LKh8m9Y54fUM8G1KO2ZJN/WvujX0b/AOFfTurHAzGGCFDr3zczr5N86F+VgXVIKCDHE0BpJSmb9MaAdMbPDkA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RZEpCOmpNmuIPw/Yy9KgxrXyWOZlbKX61cWq1sUfZMg=;
 b=ml0phyu0EjRnxlIF0hPt7rGQ3L81ek47F+LRbNIc6SUVReY+naXjL+DWymep8V1OTWgMvm7IThyiJ1D/Fv5id3i/obz2YzvR3nfgf7szl94Cn12gW0UfgaIlzK1YO6867BMimLcpzYDgJBQ+bxru0kOTWdw3/r48K7GzE86Quj0fQs/aRMRnPzSL19BszXcce7keGCE2veYWZF5CTEFR5qr0M2NxTVudNkqwCggBFOOVYeXCd8GNHJJ0ev5b+TWgHBLKnyluBmcNcJez72Ek5USNIXiAKmeQCoIfQ0nkTkXZlYHZsx2tUwZRauQbUc86bWpuh5/5LEv2IXWuOsjukA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RZEpCOmpNmuIPw/Yy9KgxrXyWOZlbKX61cWq1sUfZMg=;
 b=S8lqTr1nc7Ya5oBMP/mWgsdqgOMsyGnEmi1mRBFWFGy5OLoZi2/CzVn5GJwvlzztclUzrpbJHQjx9HnbG0hERDDngDpK8WcH6lXaoBhBV59ln0yTgSeEJRLp64iXlLlUmOlctyaKouRX6Xffw85/PvXh0W3qkzNgfowOfTxSaAE=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>,
	=?iso-8859-1?Q?Roger_Pau_Monn=E9?= <roger.pau@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall
	<julien@xen.org>, Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH v3] docs/misra: add R21.6 R21.9 R21.10 R21.14 R21.15
 R21.16
Thread-Topic: [PATCH v3] docs/misra: add R21.6 R21.9 R21.10 R21.14 R21.15
 R21.16
Thread-Index: AQHamCHZQDAooijVSkmDWqQKzv2AALGi8cCA
Date: Wed, 22 May 2024 06:26:51 +0000
Message-ID: <92270667-FB5D-4275-94C5-7D6D3C54FED8@arm.com>
References:
 <alpine.DEB.2.22.394.2404261435010.3940@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2404261435010.3940@ubuntu-linux-20-04-desktop>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|PAXPR08MB6478:EE_|AMS1EPF0000004B:EE_|DB9PR08MB6633:EE_
X-MS-Office365-Filtering-Correlation-Id: 1bc546c9-db43-43fa-9935-08dc7a28325d
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?iso-8859-1?Q?ZFX9lSslO41hZgWOGYb23Hr0VX+Vd62PIEH1N+ez4h0IcUHrXx6AHruEe5?=
 =?iso-8859-1?Q?TkbtnIP23Bdk7OeEcY/Fxgl0FUz1v29lF1koIEjetXRXeFUzeQThAk6I8u?=
 =?iso-8859-1?Q?IckgELw3lhvBiO4XYhSWCwGv6zGSRTEN59sOoK0LAAXm6cbe4Lu61r67/d?=
 =?iso-8859-1?Q?mMTh9NJUCiiEtHnu16DSebk86Gdo56+5OAZcqOBsXNOjrXvnrzBRL6a9kA?=
 =?iso-8859-1?Q?BJLpLVO7/ZIABu2ls/r6sA3AoCRH+YiFcuKUKTrxg2Iv/Pp0EkWsJjKp9R?=
 =?iso-8859-1?Q?FXpn16Eyh0jAP8rfuphsI/5qKmwf+UJQtgQL3v7zUu06L7xmbjsx7iOXS4?=
 =?iso-8859-1?Q?aeWbDzZ1NEbM5dQyIRlhQBqh5MBJDECY8Z/QXKqApclTp25Vdgmkj88yt2?=
 =?iso-8859-1?Q?/74c+ebACQFxzsIr6aUTp04+d1/428Cqb2ZnXswFl6QpA3lRpNH691Apj0?=
 =?iso-8859-1?Q?GQxcUvkMjovO1CiGnY1plhNm2SrAwEOmfNaKbI09R45e6CilVXkibU9kzA?=
 =?iso-8859-1?Q?zqMarMBWJRBXO3jdCDpAEH/iKEiAbvmgN70VTyf3bn0eboim3NhnF28P+J?=
 =?iso-8859-1?Q?yCcAopqtrfH/GUPhf+NNEjFd7zbAwgvgLsIPQjoJrlikrtD8jA5nBz+cow?=
 =?iso-8859-1?Q?sp/YSHXA9CDiwKNXs6DDw7El2uiY86qsAfDtbjpFdu3BY1oZbOVPRtx34/?=
 =?iso-8859-1?Q?nT1PSS26zcgYYyAlGJ/RfU2bGiWPPxuCjijhAkA2UELEluohCYP906l1b4?=
 =?iso-8859-1?Q?510NSwxa7yFs+KDRWnhySFugErXEuz+xrqlagwdnq1mvG7E5126Czo3ULu?=
 =?iso-8859-1?Q?qDS1KJLJGXSOrevS7H61lgats6+UCLGjE/cz7t/P00feUAPBud46Yk3jCK?=
 =?iso-8859-1?Q?w3Bmt5bAF4gofEbNyHBQnzMnBbQ9e74DmdM9XO50/m8AG/p/8Z7nChzPKc?=
 =?iso-8859-1?Q?OaE822l2vUQo33s4cFXAhq5j9bGcH4b12CkGBEq9Uvrlhr8o3W+JTujBhf?=
 =?iso-8859-1?Q?xVdM1J90HB0Cg6ZL2yzD7PTBLyR0s8JsxpqytpwMOQZEdqnQMzDxfjeHO/?=
 =?iso-8859-1?Q?rviStwqeToAyzaFE07lGdquaEBl1E2hwlpkzmSZ4hxgWA+HX3UafTkmM8W?=
 =?iso-8859-1?Q?qp+V7Q+ku7TltDmQkVoXCmkfr9XwqWHHpJPXOQkLC5+VFby+QN0kI8osBQ?=
 =?iso-8859-1?Q?TBReo/ssvDflHZHzSS6g/TY+S3wDYSIdLX6jhrJ1MaQIqLlHj30dtyjCwG?=
 =?iso-8859-1?Q?QoMMG2e6cxCnf2xMq7qTow4hK35p44Ls5CwtmfCMmpaT3kLq3HWtxZkvys?=
 =?iso-8859-1?Q?D5RBTo9XnZbRSgtVm32MXowoSWhd/Rxx/FFfGQAou6FvcN0=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <CC3461F9018E684CBFB1F8F9DAE8F7E9@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6478
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS1EPF0000004B.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1a8560f8-c439-4ac8-542a-08dc7a282ae9
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?iso-8859-1?Q?+Hk8RigbvTUqNhbCuE6zjCKDeJiNScCe6H4yUCpndsk/T5cto2qn36eFqg?=
 =?iso-8859-1?Q?CG+7paZZvFaAh7KBwiB7Mzo1636vP/p5Xqg7ROkQtHFH/P9cLkXgWbXApk?=
 =?iso-8859-1?Q?vYNsapkTbYZ1t9IvB0rAB3s5GqxAJSG696Xs+YEXSGZgkth34yr9WGF8+D?=
 =?iso-8859-1?Q?QTTqO19YNMQYM2TNDo/JC91+XfK03bSMXZMsJNbtQenQX0mouNmW8avxHu?=
 =?iso-8859-1?Q?DHmp/RfG9f9S8B4upIKaeljJVevlYkQVYTy9z7G2uzczX9VD9iwyvsTnX6?=
 =?iso-8859-1?Q?ZY8browknTjnh1HcLeRDwL4OxC9Iij76DS+OFRaKGZzcy4OHAABnmCKamU?=
 =?iso-8859-1?Q?PjP1nVplW9Xf0zyHkKh2X4e6+Ts8VkW3XubVoPrP+s/UIKCVBzpQl+fAY9?=
 =?iso-8859-1?Q?2Rp+UfnN1pWr686shdITN8jER8lQ1AyBNSHT9ay6hpd80FOCYxK3u7X2ow?=
 =?iso-8859-1?Q?ZXUsfquLafQfngHAN3lJpaGkp+iitSgXMVthNcQOrJSbhHTOqSOktRkEeL?=
 =?iso-8859-1?Q?/GxPMmRSiV53YS4wqE8clbaCO+Xawneqsrke4Vfer9WCaglp6JqJbiZsVu?=
 =?iso-8859-1?Q?d5ggN7prV8hXPtyyxp+3+3c4LdMeLeh8cnd7u0AEmQ9ZJAN/txxAcT8QzP?=
 =?iso-8859-1?Q?3c6eXDJ98mAuImvhNLHLLhztK2zPE1yHTJNii68M08YpC/Wa4NflAeQO9b?=
 =?iso-8859-1?Q?uMS4u4izBKIAY37Ulhwvvbe5fHKFzHdZEy7ifBg9Cf5Pbwj5NDyVycxAZ/?=
 =?iso-8859-1?Q?R/oiNQmNmQJGLaFN9FJQg+32BHgwHP27Chih47roL0v8qwdsxs4Wh8n/QM?=
 =?iso-8859-1?Q?RkDcMhG6v7bxOin8u3DYRRvgIlmmGCUD1tfrzc+jSaHdH9A/8JwCgGpwE4?=
 =?iso-8859-1?Q?bw1mAbADyHlbu+9AnCuNdAl32nlbo5HJ6jeb9CE0TQ7rvjndRq6XHFx+BJ?=
 =?iso-8859-1?Q?ITyZdz6LFaQ8nv2qo+QkSWUmSl9F134MuqsqTtrtkzA4G4b+5YNn6QG8PC?=
 =?iso-8859-1?Q?7ChPwZir+1/pqKF4VIVa2ioHun+MBYEAVXuqqwoiGQwv+XTaVMGR7IyTG4?=
 =?iso-8859-1?Q?GibFgpL6pGVI8bPmVMa7U48Gm9OGNIG+Y5lZPq/6D26Ao55e79fXnpTJ3n?=
 =?iso-8859-1?Q?xtYnvXVvpCoezDjiQjgDkv23b/698JRmX5OVCAyWyazkXvG/dWyiH79tp2?=
 =?iso-8859-1?Q?D8PnvOob9srsHrQPjhLgjReAespH1pfIuGniMzlbP23UZlgoDL8p6FircN?=
 =?iso-8859-1?Q?o/8vXogxwIYJsM4ZriaHzHQrUABlaWq1XCi+iMMMjVydSin3ogqRnoV4xi?=
 =?iso-8859-1?Q?ikHLtX0f5CU2zJTlzgZqKoZCLsAclmng33DotxH7yKGt/60jfYT++AB2mP?=
 =?iso-8859-1?Q?lY1rDhPK9QTAd5T1f1B9/l9zZkOJv62A=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 06:27:04.1574
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1bc546c9-db43-43fa-9935-08dc7a28325d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS1EPF0000004B.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6633

Hi Stefano,


> On 26 Apr 2024, at 23:36, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

You might want to answer to Andrew on the footnote and agree or not on his =
way to fix.

Appart from that:
Acked-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand


> ---
>=20
> Changes in v3:
> - add explanation in footnote
> - remove comment from 21.14, 21.15, 21.16
>=20
> docs/misra/rules.rst | 42 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
>=20
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index b7b447e152..5ba7394f05 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -652,12 +652,48 @@ maintainers if you want to suggest a change.
>        declared
>      - See comment for Rule 21.1
>=20
> +   * - `Rule 21.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example=
-Suite/-/blob/master/R_21_06.c>`_
> +     - Required
> +     - The Standard Library input/output routines shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
> +   * - `Rule 21.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example=
-Suite/-/blob/master/R_21_09.c>`_
> +     - Required
> +     - The library functions bsearch and qsort of <stdlib.h> shall not b=
e used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
> +   * - `Rule 21.10 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Exampl=
e-Suite/-/blob/master/R_21_10.c>`_
> +     - Required
> +     - The Standard Library time and date routines shall not be used
> +     - Xen doesn't provide, use, or link against a Standard Library [1]_
> +
>    * - `Rule 21.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example=
-Suite/-/blob/master/R_21_13.c>`_
>      - Mandatory
>      - Any value passed to a function in <ctype.h> shall be representable=
 as an
>        unsigned char or be the value EOF
>      -
>=20
> +   * - `Rule 21.14 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Exampl=
e-Suite/-/blob/master/R_21_14.c>`_
> +     - Required
> +     - The Standard Library function memcmp shall not be used to compare
> +       null terminated strings
> +     -
> +
> +   * - `Rule 21.15 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Exampl=
e-Suite/-/blob/master/R_21_15.c>`_
> +     - Required
> +     - The pointer arguments to the Standard Library functions memcpy,
> +       memmove and memcmp shall be pointers to qualified or unqualified
> +       versions of compatible types
> +     -
> +
> +   * - `Rule 21.16 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Exampl=
e-Suite/-/blob/master/R_21_16.c>`_
> +     - Required
> +     - The pointer arguments to the Standard Library function memcmp
> +       shall point to either a pointer type, an essentially signed type,
> +       an essentially unsigned type, an essentially Boolean type or an
> +       essentially enum type
> +     - void* arguments are allowed
> +
>    * - `Rule 21.17 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example=
-Suite/-/blob/master/R_21_17.c>`_
>      - Mandatory
>      - Use of the string handling functions from <string.h> shall not res=
ult in
> @@ -712,3 +748,9 @@ maintainers if you want to suggest a change.
>      - The value of a pointer to a FILE shall not be used after the assoc=
iated
>        stream has been closed
>      -
> +
> +
> +.. [1] Xen implements itself a few functions with names that match the
> +       corresponding function names of the Standard Library for develope=
rs'
> +       convenience. These functions are part of the Xen code and subject=
 to
> +       analysis.
> --=20
> 2.25.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:31:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:31:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727242.1131653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gRV-0001VT-QE; Wed, 22 May 2024 07:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727242.1131653; Wed, 22 May 2024 07:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gRV-0001VM-NW; Wed, 22 May 2024 07:31:21 +0000
Received: by outflank-mailman (input) for mailman id 727242;
 Wed, 22 May 2024 07:31:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+y9w=MZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s9gRU-0001VB-73
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:31:20 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 47a55c8a-180d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:31:18 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso91434301fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 00:31:19 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0ce3113sm39310941fa.36.2024.05.22.00.31.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 00:31:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47a55c8a-180d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716363078; x=1716967878; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=w4KNNMhufpSW6ROKgnE3tNTc24uwhMZ97GXhgr5wGvU=;
        b=mh0sOFeY/6jgfy6F7bW8i/hAU9HNzqgfT9RagvTV7oGBjkcqHENiHsl2aAnmybwfEc
         UESKhVZeRl1MbJaYw5ylRM4fZL2q0AKxisPj+qWXzZYBbrC9/k5jCIdi+/2DHwn++pIH
         obUC3+w0mtG/Rraj22udRwmeAHPy+Hiz8tWeMRsVcn5yinLDm5rprbDUIgOoAtOcdsWU
         9+jrcffJNK5dlhri17K3Yikut+HUYrzBT0qZljQjwrS6cynJVDuDsTRoPpANb+tBsMae
         atwK7/CZf0UIu6d0IeFeqC0EDsp76DNeKu8RIVu8u6CUr70vpYTQiweb7HusAleO3z6Q
         IXZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716363078; x=1716967878;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=w4KNNMhufpSW6ROKgnE3tNTc24uwhMZ97GXhgr5wGvU=;
        b=N/aRWjLFO+SROhXF9IurazN7y4GLFPdGD4wFIwJkFNVW/JXVvzyrpPbAgPoOIWthbG
         QNRz9vhVS4dz9CJt9dp4pNtcWbcqyQgHeq6rmp4CIT1KakIAKFJN0xFsfZJ2BP1sa48w
         Q4uNQn9q5O3z4ePSE5eIDKm9qEkL6dTvPyamS8gjPJISGHhMuujxWasiVRUPenbScHbk
         oPUnyijJ07D41KSe85sfFWuAmF3jj4B/Njl1WlWhBhe9EvL+S4TOc16Jkuo1L4pt/Iaz
         oHnS4FZLCUNY+UogNxF8+3pbK6Np4YpIN6+4OT8OUiEeE6scyXcTGogloQnsdDnjaZKX
         y6ZQ==
X-Forwarded-Encrypted: i=1; AJvYcCXyG1XmPryQBgCA3rZzg2jYQCsfrjLG+mTuHO3WQxKnVNSlROZPQB9t75Z929WIzQNYxSx8DbohNIAopbj/X3QUjtYh4pjJtfyxMd0agDg=
X-Gm-Message-State: AOJu0YyNKF58KHvmAnKyTA32pAuOXcRobidD2pbZyKzzWQ0kdscTg7Ck
	rdKaFMu2aFHThkqXbJDoqAUFapbkQHJLlPM1bNeMlfiZJU+x1her
X-Google-Smtp-Source: AGHT+IEXEYRZuxEhaQS/WwajAfA+BStnLuNtBHjiIr9Aw9ncr/sAMCJ7HxvcTmMO1oiX4hbRQyExoA==
X-Received: by 2002:a2e:b70e:0:b0:2e5:6957:187a with SMTP id 38308e7fff4ca-2e949431ec5mr8658581fa.4.1716363078258;
        Wed, 22 May 2024 00:31:18 -0700 (PDT)
Message-ID: <d5e5b0dbae9270a1dcbb45b87c33a58bbbbfffe4.camel@gmail.com>
Subject: Re: [PATCH for-4.19 0/3] xen: Misc MISRA changes
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>,  Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Roberto Bagnara <roberto.bagnara@bugseng.com>, 
 "consulting @ bugseng . com" <consulting@bugseng.com>
Date: Wed, 22 May 2024 09:31:17 +0200
In-Reply-To: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hi Andrew,

We can consider this patch series to be in Xen 4.19:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
On Tue, 2024-05-21 at 18:15 +0100, Andrew Cooper wrote:
> Misc fixes collected during today's call.
>=20
> Andrew Cooper (3):
> =C2=A0 xen/lzo: Implement COPY{4,8} using memcpy()
> =C2=A0 xen/x86: Drop useless non-Kconfig CONFIG_* variables
> =C2=A0 xen/x86: Address two misc MISRA 17.7 violations
>=20
> =C2=A0xen/arch/x86/alternative.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 |=C2=A0 4 ++--
> =C2=A0xen/arch/x86/include/asm/config.h |=C2=A0 4 ----
> =C2=A0xen/arch/x86/nmi.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 5 ++---
> =C2=A0xen/common/lzo.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 11 ++---------
> =C2=A0xen/include/xen/acpi.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 9 ---------
> =C2=A0xen/include/xen/watchdog.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 | 13 +------------
> =C2=A06 files changed, 7 insertions(+), 39 deletions(-)
>=20
>=20
> base-commit: 26b122e3bf8f3921d87312fbf5e7e13872ae92b0



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:37:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:37:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727247.1131664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gXP-00027d-Bd; Wed, 22 May 2024 07:37:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727247.1131664; Wed, 22 May 2024 07:37:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gXP-00027W-7r; Wed, 22 May 2024 07:37:27 +0000
Received: by outflank-mailman (input) for mailman id 727247;
 Wed, 22 May 2024 07:37:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+y9w=MZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1s9gXO-00027Q-1g
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:37:26 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 20e19630-180e-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 09:37:23 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e8a8ff4083so5616291fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 00:37:23 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d18344desm38641271fa.138.2024.05.22.00.37.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 00:37:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20e19630-180e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716363443; x=1716968243; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=DKRNFZzvIbL0pORYIxOHD/ZDQvIm7q+yrchKCOPsIjE=;
        b=LjJlq0a9jGxB/ajVdGbSpt4jsujjaSU1juTrLflNh9y0Oq8rfVLxyEA37DXjNrldaw
         f9mqrUSRF5r6SuSlikxo29bYCIybl7KMHuI6LbnkYgtMlSfIFWwdjVPGjTljnIaJfKv4
         WxD0aj59doHtIOutb1+ZuJ3xOO7k5kieDXgtCw8kgc2mCbzCNbz8FRl/nyqO/Fgiinwv
         c7ZP9WZLtGpiSR15GpELVXylblhE/R5C+WRtG9Xc53/AIAio9iPM7CLFxrWnbgg+UDS0
         n4nxKrpCowng9fZwNNnt3Ydjztld6YHg8PpsozqOXeqbuNUp2UaPYSTG4xbcoHtpn1C7
         cu0Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716363443; x=1716968243;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DKRNFZzvIbL0pORYIxOHD/ZDQvIm7q+yrchKCOPsIjE=;
        b=OxSTrCtKK705aXO6d99sjLBLMMLeuGZrpQCsOIa8mI/g8EpwDrlZxdQeXTosmrAYYS
         JoE40UZAIml9WaxKKC06kAvJFyYTkMgtykzzacnmMxcx+zbgfD9LbqUzRBhBlB4e+6Qd
         auvPS9YeYrZHfejLHpcrsGRXLTi83xPuIG46SyrELXr7HlIubKxOG5VZITwPytMCwl+7
         rvBQOk2QMTbVLffduaH9xM34ty2kADPnBNC8CycisCmPlQqYiglMI2AiFDAVJSg4SSqL
         7jQojZtgPa6U+Y6sfIQuK90lXYnu69tUkpNCvOvExTjFR7Kop4v+sGaubMitC43XStmy
         tAEg==
X-Forwarded-Encrypted: i=1; AJvYcCXvg022U2MXRrZTJAIqvyLCEDt7+NFfsrdCKmwROzgUVqU9V601H9VnIaChXpMoLQs2BkVXaIDF5SlQIPSHVX+JgCMxTUsWiTExcKT9XUk=
X-Gm-Message-State: AOJu0YzW7Li53wjBnGMppbJZLpclCp7GsNsoQQq3y7Lr+TXgH+Q56q6M
	kMCLIAKwJ1uiGykN8gTndpRYVpW8qtq6DndvwwaiItAWPh4V5fjQ
X-Google-Smtp-Source: AGHT+IESgKnIK9J8eqTpzAs/87mX/VdvKKQvGMvWmTeZvHGLDkK00OOWjhDFVddxJ15so5DzNeJnbA==
X-Received: by 2002:a2e:8911:0:b0:2dc:f188:9945 with SMTP id 38308e7fff4ca-2e7171cc6demr31057841fa.25.1716363442785;
        Wed, 22 May 2024 00:37:22 -0700 (PDT)
Message-ID: <9e59da99eb87a7d3fcd8cc6a3c6d6ef2753736af.camel@gmail.com>
Subject: Re: [PATCH v10 03/14] xen/bitops: implement fls{l}() in common logic
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Wed, 22 May 2024 09:37:21 +0200
In-Reply-To: <d26a4033-8cd1-4a9c-8549-49a36b7d52fb@suse.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
	 <d26a4033-8cd1-4a9c-8549-49a36b7d52fb@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-21 at 13:18 +0200, Jan Beulich wrote:
> On 17.05.2024 15:54, Oleksii Kurochko wrote:
> > To avoid the compilation error below, it is needed to update to
> > places
> > in common/page_alloc.c where flsl() is used as now flsl() returns
> > unsigned int:
> >=20
> > ./include/xen/kernel.h:18:21: error: comparison of distinct pointer
> > types lacks a cast [-Werror]
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 18 |=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 (void) (&_x =3D=3D &_y);=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ^~
> > =C2=A0=C2=A0=C2=A0 common/page_alloc.c:1843:34: note: in expansion of m=
acro 'min'
> > =C2=A0=C2=A0=C2=A0=C2=A0 1843 |=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 unsigned int inc_order =3D min(MAX_ORDER, flsl(e
> > - s) - 1);
> >=20
> > generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is
> > 0,
> > the result in undefined.
> >=20
> > The prototype of the per-architecture fls{l}() functions was
> > changed to
> > return 'unsigned int' to align with the generic implementation of
> > these
> > functions and avoid introducing signed/unsigned mismatches.
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > =C2=A0The patch is almost independent from Andrew's patch series
> > =C2=A0(
> > https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.coope=
r3@citrix.com/T/#t
> > )
> > =C2=A0except test_fls() function which IMO can be merged as a separate
> > patch after Andrew's patch
> > =C2=A0will be fully ready.
>=20
> If there wasn't this dependency (I don't think it's "almost
> independent"),
> I'd be offering R-b with again one nit below.

Aren't all changes, except those in xen/common/bitops.c, independent? I
could move these changes in xen/common/bitops.c to a separate commit. I
think it is safe to commit them ( an introduction of common logic for
fls{l}() and tests ) separately since the CI tests have passed.

~ Oleksii

>=20
> > --- a/xen/arch/x86/include/asm/bitops.h
> > +++ b/xen/arch/x86/include/asm/bitops.h
> > @@ -425,20 +425,21 @@ static always_inline unsigned int
> > arch_ffsl(unsigned long x)
> > =C2=A0 *
> > =C2=A0 * This is defined the same way as ffs.
> > =C2=A0 */
> > -static inline int flsl(unsigned long x)
> > +static always_inline unsigned int arch_flsl(unsigned long x)
> > =C2=A0{
> > -=C2=A0=C2=A0=C2=A0 long r;
> > +=C2=A0=C2=A0=C2=A0 unsigned long r;
> > =C2=A0
> > =C2=A0=C2=A0=C2=A0=C2=A0 asm ( "bsr %1,%0\n\t"
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "jnz 1f\n\=
t"
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "mov $-1,%=
0\n"
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "1:" : "=
=3Dr" (r) : "rm" (x));
> > -=C2=A0=C2=A0=C2=A0 return (int)r+1;
> > +=C2=A0=C2=A0=C2=A0 return (unsigned int)r+1;
>=20
> Since you now touch this, you'd better tidy it at the same time:
>=20
> =C2=A0=C2=A0=C2=A0 return r + 1;
>=20
> (i.e. style and no need for a cast).
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727254.1131674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9giW-0003sl-B6; Wed, 22 May 2024 07:48:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727254.1131674; Wed, 22 May 2024 07:48:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9giW-0003se-7x; Wed, 22 May 2024 07:48:56 +0000
Received: by outflank-mailman (input) for mailman id 727254;
 Wed, 22 May 2024 07:48:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9giU-0003sU-TR; Wed, 22 May 2024 07:48:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9giU-0007XK-Pi; Wed, 22 May 2024 07:48:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9giU-0004Pb-D3; Wed, 22 May 2024 07:48:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9giU-0004vi-Cb; Wed, 22 May 2024 07:48:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=K91Via/UOZWzTXAsa54I0XBHCFR0QDEulnh6sDdrb80=; b=5jGM4SoOgzvGs8pTETHh7Oac41
	clzAmmcON0SCnUpMw+mzMFrGKMQ1/Vwd/2kMU30QkRDZoJ+j7nOKj/v2sAzwvWNwAiPHE01Bnpyhy
	d6k4FtCcPn4NQ8i5BOP4f9VzlhKwwiAA+y4HVmcEv46v5znQd96SOe9Kzj5WFdFo6Iz8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186065-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186065: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=5ad8b6ad9a08abdbc8c57a51a5faaf2ef1afc547
X-Osstest-Versions-That:
    linux=8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 May 2024 07:48:54 +0000

flight 186065 linux-linus real [real]
flight 186071 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186065/
http://logs.test-lab.xenproject.org/osstest/logs/186071/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 186052

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 186052

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 186052
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186052
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186052
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 186052
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186052
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                5ad8b6ad9a08abdbc8c57a51a5faaf2ef1afc547
baseline version:
 linux                8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6

Last test of basis   186052  2024-05-21 01:42:42 Z    1 days
Testing same since   186065  2024-05-21 16:10:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Christian Brauner <brauner@kernel.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Linus Torvalds <torvalds@linux-foundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 5ad8b6ad9a08abdbc8c57a51a5faaf2ef1afc547
Merge: db3d841ac9ed d18a8679581e
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Tue May 21 08:34:51 2024 -0700

    Merge tag 'pull-set_blocksize' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
    
    Pull vfs blocksize updates from Al Viro:
     "This gets rid of bogus set_blocksize() uses, switches it over
      to be based on a 'struct file *' and verifies that the caller
      has the device opened exclusively"
    
    * tag 'pull-set_blocksize' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs:
      make set_blocksize() fail unless block device is opened exclusive
      set_blocksize(): switch to passing struct file *
      btrfs_get_bdev_and_sb(): call set_blocksize() only for exclusive opens
      swsusp: don't bother with setting block size
      zram: don't bother with reopening - just use O_EXCL for open
      swapon(2): open swap with O_EXCL
      swapon(2)/swapoff(2): don't bother with block size
      pktcdvd: sort set_blocksize() calls out
      bcache_register(): don't bother with set_blocksize()

commit db3d841ac9edb0b98cc002e3b27c0b266ecfe5ba
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Tue May 21 14:34:43 2024 +0200

    fs/pidfs: make 'lsof' happy with our inode changes
    
    pidfs started using much saner inodes in commit b28ddcc32d8f ("pidfs:
    convert to path_from_stashed() helper"), but that exposed the fact that
    lsof had some knowledge of just how odd our old anon_inode usage was.
    
    For example, legacy anon_inodes hadn't even initialized the inode type
    in the inode mode, so everything had a type of zero.
    
    So sane tools like 'stat' would report these files as "weird file", but
    'lsof' instead used that (together with the name of the link in proc) to
    notice that it's an anonymous inode, and used it to detect pidfd files.
    
    Let's keep our internal new sane inode model, but mask the file type
    bits at 'stat()' time in the getattr() function we already have, and by
    making the dentry name match what lsof expects too.
    
    This keeps our internal models sane, but should make user space see the
    same old odd behavior.
    
    Reported-by: Jiri Slaby <jirislaby@kernel.org>
    Link: https://lore.kernel.org/all/a15b1050-4b52-4740-a122-a4d055c17f11@kernel.org/
    Link: https://github.com/lsof-org/lsof/issues/317
    Cc: Alexander Viro <viro@zeniv.linux.org.uk>
    Cc: Seth Forshee <sforshee@kernel.org>
    Cc: Tycho Andersen <tycho@tycho.pizza>
    Signed-off-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit d18a8679581e8d1166b68e211d16c5349ae8c38c
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Thu May 2 17:36:32 2024 -0400

    make set_blocksize() fail unless block device is opened exclusive
    
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit ead083aeeed9df44fab9227e47688f7305c3a233
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Thu Apr 18 00:34:31 2024 -0400

    set_blocksize(): switch to passing struct file *
    
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit b85c42981ac4abeeb15d16437c40f52a8a64787d
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Thu Apr 18 00:21:25 2024 -0400

    btrfs_get_bdev_and_sb(): call set_blocksize() only for exclusive opens
    
    btrfs_get_bdev_and_sb() has two callers - btrfs_open_one_device(),
    which asks for open to be exclusive and btrfs_get_dev_args_from_path(),
    which doesn't.  Currently it does set_blocksize() in all cases.
    
    I'm rather dubious about the need to do set_blocksize() anywhere in btrfs,
    to be honest - there's some access to page cache of underlying block
    devices in there, but it's nowhere near the hot paths, AFAICT.
    
    In any case, btrfs_get_dev_args_from_path() only needs to read
    the on-disk superblock and copy several fields out of it; all
    callers are only interested in devices that are already opened
    and brought into per-filesystem set, so setting the block size
    is redundant for those and actively harmful if we are given
    a pathname of unrelated device.
    
    So we only need btrfs_get_bdev_and_sb() to call set_blocksize()
    when it's asked to open exclusive.
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit b1439b179d351977641a1df9745a24d08693f9d4
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Wed Apr 17 18:38:38 2024 -0400

    swsusp: don't bother with setting block size
    
    same as with the swap...
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit ebb0173df2015187bacf704d16a95119d4bc306d
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Sun Apr 28 18:55:47 2024 -0400

    zram: don't bother with reopening - just use O_EXCL for open
    
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit 51d908b3db0e588aeb2d06df37e4df3fb1754bb5
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Wed Apr 17 18:33:34 2024 -0400

    swapon(2): open swap with O_EXCL
    
    ... eliminating the need to reopen block devices so they could be
    exclusively held.
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit 798cb7f9aec35460c383eab57b9fa474d999a2eb
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Wed Apr 17 18:26:54 2024 -0400

    swapon(2)/swapoff(2): don't bother with block size
    
    once upon a time that used to matter; these days we do swap IO for
    swap devices at the level that doesn't give a damn about block size,
    buffer_head or anything of that sort - just attach the page to
    bio, set the location and size (the latter to PAGE_SIZE) and feed
    into queue.
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit 3a52c03d1ece8f480d6a6c35d92f7c1c6215d2a6
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Wed Apr 17 00:28:03 2024 -0400

    pktcdvd: sort set_blocksize() calls out
    
    1) it doesn't make any sense to have ->open() call set_blocksize() on the
    device being opened - the caller will override that anyway.
    
    2) setting block size on underlying device, OTOH, ought to be done when
    we are opening it exclusive - i.e. as part of pkt_open_dev().  Having
    it done at setup time doesn't guarantee us anything about the state
    at the time we start talking to it.  Worse, if you happen to have
    the underlying device containing e.g. ext2 with 4Kb blocks that
    is currently mounted r/o, that set_blocksize() will confuse the hell
    out of filesystem.
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

commit af63dd715a5c6b66bbd1485c2189b92c1a3fba41
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Tue Apr 16 20:12:47 2024 -0400

    bcache_register(): don't bother with set_blocksize()
    
    We are not using __bread() anymore and read_cache_page_gfp() doesn't
    care about block size.  Moreover, we should *not* change block
    size on a device that is currently held exclusive - filesystems
    that use buffer cache expect the block numbers to be interpreted
    in units set by filesystem.
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Christian Brauner <brauner@kernel.org>
    ACKed-by: Kent Overstreet <kent.overstreet@linux.dev>
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>


From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727263.1131689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glZ-0005dO-4A; Wed, 22 May 2024 07:52:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727263.1131689; Wed, 22 May 2024 07:52:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glZ-0005cP-08; Wed, 22 May 2024 07:52:05 +0000
Received: by outflank-mailman (input) for mailman id 727263;
 Wed, 22 May 2024 07:52:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glX-0005YY-M3
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:03 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2c9faaab-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:02 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CF224DA7;
 Wed, 22 May 2024 00:52:25 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E8DDB3F766;
 Wed, 22 May 2024 00:52:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c9faaab-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Date: Wed, 22 May 2024 08:51:45 +0100
Message-Id: <20240522075151.3373899-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The current static shared memory code is using bootinfo banks when it
needs to find the number of borrowers, so every time assign_shared_memory
is called, the bank is searched in the bootinfo.shmem structure.

There is nothing wrong with it, however the bank can be used also to
retrieve the start address and size and also to pass less argument to
assign_shared_memory. When retrieving the information from the bootinfo
bank, it's also possible to move the checks on alignment to
process_shm_node in the early stages.

So create a new function find_shm_bank_by_id() which takes a
'struct shared_meminfo' structure and the shared memory ID, to look for a
bank with a matching ID, take the physical host address and size from the
bank, pass the bank to assign_shared_memory() removing the now unnecessary
arguments and finally remove the acquire_nr_borrower_domain() function
since now the information can be extracted from the passed bank.
Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
case of errors (unlikely), as said above, move the checks on alignment
to process_shm_node.

Drawback of this change is that now the bootinfo are used also when the
bank doesn't need to be allocated, however it will be convenient later
to use it as an argument for assign_shared_memory when dealing with
the use case where the Host physical address is not supplied by the user.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - switch strncmp with strcmp in find_shm_bank_by_id, fix commit msg typo,
   add R-by Michal.
v2 changes:
 - fix typo commit msg, renamed find_shm() to find_shm_bank_by_id(),
   swap region size check different from zero and size alignment, remove
   not necessary BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 100 +++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 47 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 78881dd1d3f7..0a1c327e90ea 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -19,29 +19,21 @@ static void __init __maybe_unused build_assertions(void)
                  offsetof(struct shared_meminfo, bank)));
 }
 
-static int __init acquire_nr_borrower_domain(struct domain *d,
-                                             paddr_t pbase, paddr_t psize,
-                                             unsigned long *nr_borrowers)
+static const struct membank __init *
+find_shm_bank_by_id(const struct membanks *shmem, const char *shm_id)
 {
-    const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
 
-    /* Iterate reserved memory to find requested shm bank. */
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
     {
-        paddr_t bank_start = shmem->bank[bank].start;
-        paddr_t bank_size = shmem->bank[bank].size;
-
-        if ( (pbase == bank_start) && (psize == bank_size) )
+        if ( strcmp(shm_id, shmem->bank[bank].shmem_extra->shm_id) == 0 )
             break;
     }
 
     if ( bank == shmem->nr_banks )
-        return -ENOENT;
+        return NULL;
 
-    *nr_borrowers = shmem->bank[bank].shmem_extra->nr_shm_borrowers;
-
-    return 0;
+    return &shmem->bank[bank];
 }
 
 /*
@@ -103,14 +95,18 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     return smfn;
 }
 
-static int __init assign_shared_memory(struct domain *d,
-                                       paddr_t pbase, paddr_t psize,
-                                       paddr_t gbase)
+static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       const struct membank *shm_bank)
 {
     mfn_t smfn;
     int ret = 0;
     unsigned long nr_pages, nr_borrowers, i;
     struct page_info *page;
+    paddr_t pbase, psize;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+    nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
     printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
            d, pbase, pbase + psize);
@@ -135,14 +131,6 @@ static int __init assign_shared_memory(struct domain *d,
         }
     }
 
-    /*
-     * Get the right amount of references per page, which is the number of
-     * borrower domains.
-     */
-    ret = acquire_nr_borrower_domain(d, pbase, psize, &nr_borrowers);
-    if ( ret )
-        return ret;
-
     /*
      * Instead of letting borrower domain get a page ref, we add as many
      * additional reference as the number of borrowers when the owner
@@ -199,6 +187,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
 
     dt_for_each_child_node(node, shm_node)
     {
+        const struct membank *boot_shm_bank;
         const struct dt_property *prop;
         const __be32 *cells;
         uint32_t addr_cells, size_cells;
@@ -212,6 +201,23 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
 
+        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
+        {
+            printk("%pd: invalid \"xen,shm-id\" property", d);
+            return -EINVAL;
+        }
+        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
+
+        boot_shm_bank = find_shm_bank_by_id(bootinfo_get_shmem(), shm_id);
+        if ( !boot_shm_bank )
+        {
+            printk("%pd: static shared memory bank not found: '%s'", d, shm_id);
+            return -ENOENT;
+        }
+
+        pbase = boot_shm_bank->start;
+        psize = boot_shm_bank->size;
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -221,20 +227,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
-        psize = dt_read_paddr(cells, size_cells);
-        if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
-        {
-            printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
-                   d, pbase, gbase);
-            return -EINVAL;
-        }
-        if ( !IS_ALIGNED(psize, PAGE_SIZE) )
-        {
-            printk("%pd: size 0x%"PRIpaddr" is not suitably aligned\n",
-                   d, psize);
-            return -EINVAL;
-        }
+        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
         for ( i = 0; i < PFN_DOWN(psize); i++ )
             if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
@@ -251,13 +244,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
             owner_dom_io = false;
 
-        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
-        {
-            printk("%pd: invalid \"xen,shm-id\" property", d);
-            return -EINVAL;
-        }
-        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
-
         /*
          * DOMID_IO is a fake domain and is not described in the Device-Tree.
          * Therefore when the owner of the shared region is DOMID_IO, we will
@@ -270,8 +256,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
              * We found the first borrower of the region, the owner was not
              * specified, so they should be assigned to dom_io.
              */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d,
-                                       pbase, psize, gbase);
+            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                       boot_shm_bank);
             if ( ret )
                 return ret;
         }
@@ -439,12 +425,32 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
     size = dt_next_cell(size_cells, &cell);
 
+    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+    {
+        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+               paddr);
+        return -EINVAL;
+    }
+
+    if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
+    {
+        printk("fdt: guest address 0x%"PRIpaddr" is not suitably aligned.\n",
+               gaddr);
+        return -EINVAL;
+    }
+
     if ( !size )
     {
         printk("fdt: the size for static shared memory region can not be zero\n");
         return -EINVAL;
     }
 
+    if ( !IS_ALIGNED(size, PAGE_SIZE) )
+    {
+        printk("fdt: size 0x%"PRIpaddr" is not suitably aligned\n", size);
+        return -EINVAL;
+    }
+
     end = paddr + size;
     if ( end <= paddr )
     {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727264.1131693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glZ-0005gU-Bj; Wed, 22 May 2024 07:52:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727264.1131693; Wed, 22 May 2024 07:52:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glZ-0005fq-65; Wed, 22 May 2024 07:52:05 +0000
Received: by outflank-mailman (input) for mailman id 727264;
 Wed, 22 May 2024 07:52:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glY-0005YY-MP
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:04 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2d3c4e7b-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:03 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8BB0FEC;
 Wed, 22 May 2024 00:52:26 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 080133F766;
 Wed, 22 May 2024 00:52:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d3c4e7b-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 2/7] xen/arm: Wrap shared memory mapping code in one function
Date: Wed, 22 May 2024 08:51:46 +0100
Message-Id: <20240522075151.3373899-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Wrap the code and logic that is calling assign_shared_memory
and map_regions_p2mt into a new function 'handle_shared_mem_bank',
it will become useful later when the code will allow the user to
don't pass the host physical address.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - check return value of dt_property_read_string, add R-by Michal
v2 changes:
 - add blank line, move owner_dom_io computation inside
   handle_shared_mem_bank in order to reduce args count, remove
   not needed BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 86 +++++++++++++++++++++++--------------
 1 file changed, 53 insertions(+), 33 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 0a1c327e90ea..c15a65130659 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -180,6 +180,53 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
     return 0;
 }
 
+static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
+                                         const char *role_str,
+                                         const struct membank *shm_bank)
+{
+    bool owner_dom_io = true;
+    paddr_t pbase, psize;
+    int ret;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+
+    /*
+     * "role" property is optional and if it is defined explicitly,
+     * then the owner domain is not the default "dom_io" domain.
+     */
+    if ( role_str != NULL )
+        owner_dom_io = false;
+
+    /*
+     * DOMID_IO is a fake domain and is not described in the Device-Tree.
+     * Therefore when the owner of the shared region is DOMID_IO, we will
+     * only find the borrowers.
+     */
+    if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
+         (!owner_dom_io && strcmp(role_str, "owner") == 0) )
+    {
+        /*
+         * We found the first borrower of the region, the owner was not
+         * specified, so they should be assigned to dom_io.
+         */
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        if ( ret )
+            return ret;
+    }
+
+    if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
+    {
+        /* Set up P2M foreign mapping for borrower domain. */
+        ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
+                               _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
+        if ( ret )
+            return ret;
+    }
+
+    return 0;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -196,7 +243,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         unsigned int i;
         const char *role_str;
         const char *shm_id;
-        bool owner_dom_io = true;
 
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
@@ -237,39 +283,13 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                 return -EINVAL;
             }
 
-        /*
-         * "role" property is optional and if it is defined explicitly,
-         * then the owner domain is not the default "dom_io" domain.
-         */
-        if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
-            owner_dom_io = false;
+        /* "role" property is optional */
+        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
+            role_str = NULL;
 
-        /*
-         * DOMID_IO is a fake domain and is not described in the Device-Tree.
-         * Therefore when the owner of the shared region is DOMID_IO, we will
-         * only find the borrowers.
-         */
-        if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
-             (!owner_dom_io && strcmp(role_str, "owner") == 0) )
-        {
-            /*
-             * We found the first borrower of the region, the owner was not
-             * specified, so they should be assigned to dom_io.
-             */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
-                                       boot_shm_bank);
-            if ( ret )
-                return ret;
-        }
-
-        if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
-        {
-            /* Set up P2M foreign mapping for borrower domain. */
-            ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
-                                   _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
-            if ( ret )
-                return ret;
-        }
+        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
+        if ( ret )
+            return ret;
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727262.1131683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glY-0005aR-SX; Wed, 22 May 2024 07:52:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727262.1131683; Wed, 22 May 2024 07:52:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glY-0005aK-Pi; Wed, 22 May 2024 07:52:04 +0000
Received: by outflank-mailman (input) for mailman id 727262;
 Wed, 22 May 2024 07:52:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glX-0005YY-08
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:03 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2bf9491e-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:01 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C0254339;
 Wed, 22 May 2024 00:52:24 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D50B23F766;
 Wed, 22 May 2024 00:51:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2bf9491e-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 0/7] Static shared memory followup v2 - pt2
Date: Wed, 22 May 2024 08:51:44 +0100
Message-Id: <20240522075151.3373899-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This serie is a partial rework of this other serie:
https://patchwork.kernel.org/project/xen-devel/cover/20231206090623.1932275-1-Penny.Zheng@arm.com/

The original serie is addressing an issue of the static shared memory feature
that impacts the memory footprint of other component when the feature is
enabled, another issue impacts the device tree generation for the guests when
the feature is enabled and used and the last one is a missing feature that is
the option to have a static shared memory region that is not from the host
address space.

This serie is handling some comment on the original serie and it is splitting
the rework in two part, this first part is addressing the memory footprint issue
and the device tree generation and currently is fully merged
(https://patchwork.kernel.org/project/xen-devel/cover/20240418073652.3622828-1-luca.fancellu@arm.com/),
this serie is addressing the static shared memory allocation from the Xen heap.

Luca Fancellu (5):
  xen/arm: Lookup bootinfo shm bank during the mapping
  xen/arm: Wrap shared memory mapping code in one function
  xen/arm: Parse xen,shared-mem when host phys address is not provided
  xen/arm: Rework heap page allocation outside allocate_bank_memory
  xen/arm: Implement the logic for static shared memory from Xen heap

Penny Zheng (2):
  xen/p2m: put reference for level 2 superpage
  xen/docs: Describe static shared memory when host address is not
    provided

 docs/misc/arm/device-tree/booting.txt   |  52 ++-
 xen/arch/arm/arm32/mmu/mm.c             |  11 +-
 xen/arch/arm/dom0less-build.c           |   4 +-
 xen/arch/arm/domain_build.c             |  84 +++--
 xen/arch/arm/include/asm/domain_build.h |   9 +-
 xen/arch/arm/mmu/p2m.c                  |  63 +++-
 xen/arch/arm/setup.c                    |  14 +-
 xen/arch/arm/static-shmem.c             | 432 +++++++++++++++++-------
 8 files changed, 486 insertions(+), 183 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727265.1131715 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gla-0006Eh-KJ; Wed, 22 May 2024 07:52:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727265.1131715; Wed, 22 May 2024 07:52:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gla-0006Cm-D3; Wed, 22 May 2024 07:52:06 +0000
Received: by outflank-mailman (input) for mailman id 727265;
 Wed, 22 May 2024 07:52:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glZ-0005YY-MQ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:05 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2dfeae50-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:04 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 35DC31474;
 Wed, 22 May 2024 00:52:28 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1BA2B3F766;
 Wed, 22 May 2024 00:52:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2dfeae50-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
Date: Wed, 22 May 2024 08:51:47 +0100
Message-Id: <20240522075151.3373899-4-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

We are doing foreign memory mapping for static shared memory, and
there is a great possibility that it could be super mapped.
But today, p2m_put_l3_page could not handle superpages.

This commits implements a new function p2m_put_l2_superpage to handle
2MB superpages, specifically for helping put extra references for
foreign superpages.

Modify relinquish_p2m_mapping as well to take into account preemption
when type is foreign memory and order is above 9 (2MB).

Currently 1GB superpages are not handled because Xen is not preemptible
and therefore some work is needed to handle such superpages, for which
at some point Xen might end up freeing memory and therefore for such a
big mapping it could end up in a very long operation.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v3:
 - Add reasoning why we don't support now 1GB superpage, remove level_order
   variable from p2m_put_l2_superpage, update TODO comment inside
   p2m_free_entry, use XEN_PT_LEVEL_ORDER(2) instead of value 9 inside
   relinquish_p2m_mapping. (Michal)
v2:
 - Do not handle 1GB super page as there might be some issue where
   a lot of calls to put_page(...) might be issued which could lead
   to free memory that is a long operation.
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
---
 xen/arch/arm/mmu/p2m.c | 63 ++++++++++++++++++++++++++++++------------
 1 file changed, 46 insertions(+), 17 deletions(-)

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 41fcca011cf4..b496266deef6 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
     return rc;
 }
 
-/*
- * Put any references on the single 4K page referenced by pte.
- * TODO: Handle superpages, for now we only take special references for leaf
- * pages (specifically foreign ones, which can't be super mapped today).
- */
-static void p2m_put_l3_page(const lpae_t pte)
+/* Put any references on the single 4K page referenced by mfn. */
+static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
 {
-    mfn_t mfn = lpae_get_mfn(pte);
-
-    ASSERT(p2m_is_valid(pte));
-
     /*
      * TODO: Handle other p2m types
      *
@@ -771,16 +763,43 @@ static void p2m_put_l3_page(const lpae_t pte)
      * flush the TLBs if the page is reallocated before the end of
      * this loop.
      */
-    if ( p2m_is_foreign(pte.p2m.type) )
+    if ( p2m_is_foreign(type) )
     {
         ASSERT(mfn_valid(mfn));
         put_page(mfn_to_page(mfn));
     }
     /* Detect the xenheap page and mark the stored GFN as invalid. */
-    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
+    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
         page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
 }
 
+/* Put any references on the superpage referenced by mfn. */
+static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
+{
+    unsigned int i;
+
+    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
+    {
+        p2m_put_l3_page(mfn, type);
+
+        mfn = mfn_add(mfn, 1);
+    }
+}
+
+/* Put any references on the page referenced by pte. */
+static void p2m_put_page(const lpae_t pte, unsigned int level)
+{
+    mfn_t mfn = lpae_get_mfn(pte);
+
+    ASSERT(p2m_is_valid(pte));
+
+    /* We have a second level 2M superpage */
+    if ( p2m_is_superpage(pte, level) && (level == 2) )
+        return p2m_put_l2_superpage(mfn, pte.p2m.type);
+    else if ( level == 3 )
+        return p2m_put_l3_page(mfn, pte.p2m.type);
+}
+
 /* Free lpae sub-tree behind an entry */
 static void p2m_free_entry(struct p2m_domain *p2m,
                            lpae_t entry, unsigned int level)
@@ -809,9 +828,16 @@ static void p2m_free_entry(struct p2m_domain *p2m,
 #endif
 
         p2m->stats.mappings[level]--;
-        /* Nothing to do if the entry is a super-page. */
-        if ( level == 3 )
-            p2m_put_l3_page(entry);
+        /*
+         * TODO: Currently we don't handle 1GB super-page, Xen is not
+         * preemptible and therefore some work is needed to handle such
+         * superpages, for which at some point Xen might end up freeing memory
+         * and therefore for such a big mapping it could end up in a very long
+         * operation.
+         */
+        if ( level >= 2 )
+            p2m_put_page(entry, level);
+
         return;
     }
 
@@ -1558,9 +1584,12 @@ int relinquish_p2m_mapping(struct domain *d)
 
         count++;
         /*
-         * Arbitrarily preempt every 512 iterations.
+         * Arbitrarily preempt every 512 iterations or when type is foreign
+         * mapping and the order is above 9 (2MB).
          */
-        if ( !(count % 512) && hypercall_preempt_check() )
+        if ( (!(count % 512) ||
+              (p2m_is_foreign(t) && (order > XEN_PT_LEVEL_ORDER(2)))) &&
+             hypercall_preempt_check() )
         {
             rc = -ERESTART;
             break;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727266.1131724 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glc-0006aT-Qa; Wed, 22 May 2024 07:52:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727266.1131724; Wed, 22 May 2024 07:52:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glc-0006aK-NA; Wed, 22 May 2024 07:52:08 +0000
Received: by outflank-mailman (input) for mailman id 727266;
 Wed, 22 May 2024 07:52:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glb-0005YY-HV
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:07 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2f414d23-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:06 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7716B14BF;
 Wed, 22 May 2024 00:52:30 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 909813F766;
 Wed, 22 May 2024 00:52:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f414d23-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 5/7] xen/arm: Rework heap page allocation outside allocate_bank_memory
Date: Wed, 22 May 2024 08:51:49 +0100
Message-Id: <20240522075151.3373899-6-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The function allocate_bank_memory allocates pages from the heap and
maps them to the guest using guest_physmap_add_page.

As a preparation work to support static shared memory bank when the
host physical address is not provided, Xen needs to allocate memory
from the heap, so rework allocate_bank_memory moving out the page
allocation in a new function called allocate_domheap_memory.

The function allocate_domheap_memory takes a callback function and
a pointer to some extra information passed to the callback and this
function will be called for every region, until a defined size is
reached.

In order to keep allocate_bank_memory functionality, the callback
passed to allocate_domheap_memory is a wrapper for
guest_physmap_add_page.

Let allocate_domheap_memory be externally visible, in order to use
it in the future from the static shared memory module.

Take the opportunity to change the signature of allocate_bank_memory
and remove the 'struct domain' parameter, which can be retrieved from
'struct kernel_info'.

No functional changes is intended.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - Add R-by Michal
v2:
 - Reduced scope of pg var in allocate_domheap_memory, removed not
   necessary BUG_ON(), changed callback to return bool and fix
   comment. (Michal)
---
 xen/arch/arm/dom0less-build.c           |  4 +-
 xen/arch/arm/domain_build.c             | 79 +++++++++++++++++--------
 xen/arch/arm/include/asm/domain_build.h |  9 ++-
 3 files changed, 62 insertions(+), 30 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242f4..20ddf6f8f250 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -60,12 +60,12 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
 
     mem->nr_banks = 0;
     bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
                                bank_size) )
         goto fail;
 
     bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
                                bank_size) )
         goto fail;
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 02e741685102..669970c86fd5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -417,30 +417,15 @@ static void __init allocate_memory_11(struct domain *d,
 }
 
 #ifdef CONFIG_DOM0LESS_BOOT
-bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                                 gfn_t sgfn, paddr_t tot_size)
+bool __init allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                                    alloc_domheap_mem_cb cb, void *extra)
 {
-    struct membanks *mem = kernel_info_get_mem(kinfo);
-    int res;
-    struct page_info *pg;
-    struct membank *bank;
-    unsigned int max_order = ~0;
-
-    /*
-     * allocate_bank_memory can be called with a tot_size of zero for
-     * the second memory bank. It is not an error and we can safely
-     * avoid creating a zero-size memory bank.
-     */
-    if ( tot_size == 0 )
-        return true;
-
-    bank = &mem->bank[mem->nr_banks];
-    bank->start = gfn_to_gaddr(sgfn);
-    bank->size = tot_size;
+    unsigned int max_order = UINT_MAX;
 
     while ( tot_size > 0 )
     {
         unsigned int order = get_allocation_size(tot_size);
+        struct page_info *pg;
 
         order = min(max_order, order);
 
@@ -463,17 +448,61 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
             continue;
         }
 
-        res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
-        if ( res )
-        {
-            dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        if ( !cb(d, pg, order, extra) )
             return false;
-        }
 
-        sgfn = gfn_add(sgfn, 1UL << order);
         tot_size -= (1ULL << (PAGE_SHIFT + order));
     }
 
+    return true;
+}
+
+static bool __init guest_map_pages(struct domain *d, struct page_info *pg,
+                                   unsigned int order, void *extra)
+{
+    gfn_t *sgfn = (gfn_t *)extra;
+    int res;
+
+    BUG_ON(!sgfn);
+    res = guest_physmap_add_page(d, *sgfn, page_to_mfn(pg), order);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        return false;
+    }
+
+    *sgfn = gfn_add(*sgfn, 1UL << order);
+
+    return true;
+}
+
+bool __init allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                                 paddr_t tot_size)
+{
+    struct membanks *mem = kernel_info_get_mem(kinfo);
+    struct domain *d = kinfo->d;
+    struct membank *bank;
+
+    /*
+     * allocate_bank_memory can be called with a tot_size of zero for
+     * the second memory bank. It is not an error and we can safely
+     * avoid creating a zero-size memory bank.
+     */
+    if ( tot_size == 0 )
+        return true;
+
+    bank = &mem->bank[mem->nr_banks];
+    bank->start = gfn_to_gaddr(sgfn);
+    bank->size = tot_size;
+
+    /*
+     * Allocate pages from the heap until tot_size is zero and map them to the
+     * guest using guest_map_pages, passing the starting gfn as extra parameter
+     * for the map operation.
+     */
+    if ( !allocate_domheap_memory(d, tot_size, guest_map_pages, &sgfn) )
+        return false;
+
     mem->nr_banks++;
     kinfo->unassigned_mem -= bank->size;
 
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 45936212ca21..e712afbc7fbf 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -5,9 +5,12 @@
 #include <asm/kernel.h>
 
 typedef __be32 gic_interrupt_t[3];
-
-bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                          gfn_t sgfn, paddr_t tot_size);
+typedef bool (*alloc_domheap_mem_cb)(struct domain *d, struct page_info *pg,
+                                     unsigned int order, void *extra);
+bool allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                             alloc_domheap_mem_cb cb, void *extra);
+bool allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                          paddr_t tot_size);
 int construct_domain(struct domain *d, struct kernel_info *kinfo);
 int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit);
 int make_chosen_node(const struct kernel_info *kinfo);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727267.1131730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gld-0006g4-B8; Wed, 22 May 2024 07:52:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727267.1131730; Wed, 22 May 2024 07:52:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gld-0006fS-6g; Wed, 22 May 2024 07:52:09 +0000
Received: by outflank-mailman (input) for mailman id 727267;
 Wed, 22 May 2024 07:52:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glc-0006Z1-9a
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:08 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id 2eade77b-1810-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 09:52:05 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 637E11480;
 Wed, 22 May 2024 00:52:29 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 635903F766;
 Wed, 22 May 2024 00:52:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2eade77b-1810-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 4/7] xen/arm: Parse xen,shared-mem when host phys address is not provided
Date: Wed, 22 May 2024 08:51:48 +0100
Message-Id: <20240522075151.3373899-5-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Handle the parsing of the 'xen,shared-mem' property when the host physical
address is not provided, this commit is introducing the logic to parse it,
but the functionality is still not implemented and will be part of future
commits.

Rework the logic inside process_shm_node to check the shm_id before doing
the other checks, because it ease the logic itself, add more comment on
the logic.
Now when the host physical address is not provided, the value
INVALID_PADDR is chosen to signal this condition and it is stored as
start of the bank, due to that change also early_print_info_shmem and
init_sharedmem_pages are changed, to not handle banks with start equal
to INVALID_PADDR.

Another change is done inside meminfo_overlap_check, to skip banks that
are starting with the start address INVALID_PADDR, that function is used
to check banks from reserved memory, shared memory and ACPI and since
the comment above the function states that wrapping around is not handled,
it's unlikely for these bank to have the start address as INVALID_PADDR.
Same change is done inside consider_modules, find_unallocated_memory and
dt_unreserved_regions functions, in order to skip banks that starts with
INVALID_PADDR from any computation.
The changes above holds because of this consideration.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - fix typo in commit msg, add R-by Michal
v2 changes:
 - fix comments, add parenthesis to some conditions, remove unneeded
   variables, remove else branch, increment counter in the for loop,
   skip INVALID_PADDR start banks from also consider_modules,
   find_unallocated_memory and dt_unreserved_regions. (Michal)
---
 xen/arch/arm/arm32/mmu/mm.c |  11 +++-
 xen/arch/arm/domain_build.c |   5 ++
 xen/arch/arm/setup.c        |  14 +++-
 xen/arch/arm/static-shmem.c | 125 +++++++++++++++++++++++++-----------
 4 files changed, 111 insertions(+), 44 deletions(-)

diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c
index be480c31ea05..30a7aa1e8e51 100644
--- a/xen/arch/arm/arm32/mmu/mm.c
+++ b/xen/arch/arm/arm32/mmu/mm.c
@@ -101,8 +101,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
     nr += reserved_mem->nr_banks;
     for ( ; i - nr < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - nr].start;
-        paddr_t r_e = r_s + shmem->bank[i - nr].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - nr].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - nr].size;
 
         if ( s < r_e && r_s < e )
         {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 968c497efc78..02e741685102 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -927,6 +927,11 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
         for ( j = 0; j < mem_banks[i]->nr_banks; j++ )
         {
             start = mem_banks[i]->bank[j].start;
+
+            /* Shared memory banks can contain INVALID_PADDR as start */
+            if ( INVALID_PADDR == start )
+                continue;
+
             end = mem_banks[i]->bank[j].start + mem_banks[i]->bank[j].size;
             res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
                                         PFN_DOWN(end - 1));
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index c4e5c19b11d6..0c2fdaceaf21 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -240,8 +240,15 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
     offset = reserved_mem->nr_banks;
     for ( ; i - offset < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - offset].start;
-        paddr_t r_e = r_s + shmem->bank[i - offset].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - offset].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - offset].size;
 
         if ( s < r_e && r_s < e )
         {
@@ -272,7 +279,8 @@ static bool __init meminfo_overlap_check(const struct membanks *mem,
         bank_start = mem->bank[i].start;
         bank_end = bank_start + mem->bank[i].size;
 
-        if ( region_end <= bank_start || region_start >= bank_end )
+        if ( INVALID_PADDR == bank_start || region_end <= bank_start ||
+             region_start >= bank_end )
             continue;
         else
         {
diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index c15a65130659..74c81904b8a4 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -264,6 +264,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
+        if ( INVALID_PADDR == pbase )
+        {
+            printk("%pd: host physical address must be chosen by users at the moment", d);
+            return -EINVAL;
+        }
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -377,7 +383,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
 {
     const struct fdt_property *prop, *prop_id, *prop_role;
     const __be32 *cell;
-    paddr_t paddr, gaddr, size, end;
+    paddr_t paddr = INVALID_PADDR;
+    paddr_t gaddr, size, end;
     struct membanks *mem = bootinfo_get_shmem();
     struct shmem_membank_extra *shmem_extra = bootinfo_get_shmem_extra();
     unsigned int i;
@@ -432,24 +439,37 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     if ( !prop )
         return -ENOENT;
 
+    cell = (const __be32 *)prop->data;
     if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) )
     {
-        if ( len == dt_cells_to_size(size_cells + address_cells) )
-            printk("fdt: host physical address must be chosen by users at the moment.\n");
-
-        printk("fdt: invalid `xen,shared-mem` property.\n");
-        return -EINVAL;
+        if ( len == dt_cells_to_size(address_cells + size_cells) )
+            device_tree_get_reg(&cell, address_cells, size_cells, &gaddr,
+                                &size);
+        else
+        {
+            printk("fdt: invalid `xen,shared-mem` property.\n");
+            return -EINVAL;
+        }
     }
+    else
+    {
+        device_tree_get_reg(&cell, address_cells, address_cells, &paddr,
+                            &gaddr);
+        size = dt_next_cell(size_cells, &cell);
 
-    cell = (const __be32 *)prop->data;
-    device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
-    size = dt_next_cell(size_cells, &cell);
+        if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+        {
+            printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+                paddr);
+            return -EINVAL;
+        }
 
-    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
-    {
-        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
-               paddr);
-        return -EINVAL;
+        end = paddr + size;
+        if ( end <= paddr )
+        {
+            printk("fdt: static shared memory region %s overflow\n", shm_id);
+            return -EINVAL;
+        }
     }
 
     if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
@@ -471,39 +491,64 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
         return -EINVAL;
     }
 
-    end = paddr + size;
-    if ( end <= paddr )
-    {
-        printk("fdt: static shared memory region %s overflow\n", shm_id);
-        return -EINVAL;
-    }
-
     for ( i = 0; i < mem->nr_banks; i++ )
     {
         /*
          * Meet the following check:
-         * 1) The shm ID matches and the region exactly match
-         * 2) The shm ID doesn't match and the region doesn't overlap
-         * with an existing one
+         * - when host address is provided:
+         *   1) The shm ID matches and the region exactly match
+         *   2) The shm ID doesn't match and the region doesn't overlap
+         *      with an existing one
+         * - when host address is not provided:
+         *   1) The shm ID matches and the region size exactly match
          */
-        if ( paddr == mem->bank[i].start && size == mem->bank[i].size )
+        bool paddr_assigned = (INVALID_PADDR == paddr);
+
+        if ( strncmp(shm_id, shmem_extra[i].shm_id, MAX_SHM_ID_LENGTH) == 0 )
         {
-            if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                         MAX_SHM_ID_LENGTH) == 0  )
+            /*
+             * Regions have same shm_id (cases):
+             * 1) physical host address is supplied:
+             *    - OK:   paddr is equal and size is equal (same region)
+             *    - Fail: paddr doesn't match or size doesn't match (there
+             *            cannot exists two shmem regions with same shm_id)
+             * 2) physical host address is NOT supplied:
+             *    - OK:   size is equal (same region)
+             *    - Fail: size is not equal (same shm_id must identify only one
+             *            region, there can't be two different regions with same
+             *            shm_id)
+             */
+            bool start_match = paddr_assigned ? (paddr == mem->bank[i].start) :
+                                                true;
+
+            if ( start_match && (size == mem->bank[i].size) )
                 break;
             else
             {
-                printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n",
+                printk("fdt: different shared memory region could not share the same shm ID %s\n",
                        shm_id);
                 return -EINVAL;
             }
         }
-        else if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                          MAX_SHM_ID_LENGTH) != 0 )
+
+        /*
+         * Regions have different shm_id (cases):
+         * 1) physical host address is supplied:
+         *    - OK:   paddr different, or size different (case where paddr
+         *            is equal but psize is different are wrong, but they
+         *            are handled later when checking for overlapping)
+         *    - Fail: paddr equal and size equal (the same region can't be
+         *            identified with different shm_id)
+         * 2) physical host address is NOT supplied:
+         *    - OK:   Both have different shm_id so even with same size they
+         *            can exists
+         */
+        if ( !paddr_assigned || (paddr != mem->bank[i].start) ||
+             (size != mem->bank[i].size) )
             continue;
         else
         {
-            printk("fdt: different shared memory region could not share the same shm ID %s\n",
+            printk("fdt: xen,shm-id %s does not match for all the nodes using the same region\n",
                    shm_id);
             return -EINVAL;
         }
@@ -513,7 +558,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     {
         if (i < mem->max_banks)
         {
-            if ( check_reserved_regions_overlap(paddr, size) )
+            if ( (paddr != INVALID_PADDR) &&
+                 check_reserved_regions_overlap(paddr, size) )
                 return -EINVAL;
 
             /* Static shared memory shall be reserved from any other use. */
@@ -583,13 +629,13 @@ void __init early_print_info_shmem(void)
 {
     const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
+    unsigned int printed = 0;
 
-    for ( bank = 0; bank < shmem->nr_banks; bank++ )
-    {
-        printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", bank,
-               shmem->bank[bank].start,
-               shmem->bank[bank].start + shmem->bank[bank].size - 1);
-    }
+    for ( bank = 0; bank < shmem->nr_banks; bank++, printed++ )
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", printed,
+                shmem->bank[bank].start,
+                shmem->bank[bank].start + shmem->bank[bank].size - 1);
 }
 
 void __init init_sharedmem_pages(void)
@@ -598,7 +644,8 @@ void __init init_sharedmem_pages(void)
     unsigned int bank;
 
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
-        init_staticmem_bank(&shmem->bank[bank]);
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            init_staticmem_bank(&shmem->bank[bank]);
 }
 
 int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727268.1131736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gld-0006mb-RT; Wed, 22 May 2024 07:52:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727268.1131736; Wed, 22 May 2024 07:52:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gld-0006lk-JE; Wed, 22 May 2024 07:52:09 +0000
Received: by outflank-mailman (input) for mailman id 727268;
 Wed, 22 May 2024 07:52:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9glc-0005YY-No
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:08 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 2fe6d436-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:07 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8A861150C;
 Wed, 22 May 2024 00:52:31 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A431B3F766;
 Wed, 22 May 2024 00:52:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fe6d436-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 6/7] xen/arm: Implement the logic for static shared memory from Xen heap
Date: Wed, 22 May 2024 08:51:50 +0100
Message-Id: <20240522075151.3373899-7-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This commit implements the logic to have the static shared memory banks
from the Xen heap instead of having the host physical address passed from
the user.

When the host physical address is not supplied, the physical memory is
taken from the Xen heap using allocate_domheap_memory, the allocation
needs to occur at the first handled DT node and the allocated banks
need to be saved somewhere.

Introduce the 'shm_heap_banks' for that reason, a struct that will hold
the banks allocated from the heap, its field bank[].shmem_extra will be
used to point to the bootinfo shared memory banks .shmem_extra space, so
that there is not further allocation of memory and every bank in
shm_heap_banks can be safely identified by the shm_id to reconstruct its
traceability and if it was allocated or not.

A search into 'shm_heap_banks' will reveal if the banks were allocated
or not, in case the host address is not passed, and the callback given
to allocate_domheap_memory will store the banks in the structure and
map them to the current domain, to do that, some changes to
acquire_shared_memory_bank are made to let it differentiate if the bank
is from the heap and if it is, then assign_pages is called for every
bank.

When the bank is already allocated, for every bank allocated with the
corresponding shm_id, handle_shared_mem_bank is called and the mapping
are done.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v3 changes:
 - reworded commit msg section, swap role_str and gbase in
   alloc_heap_pages_cb_extra to avoid padding hole in arm32, remove
   not needed printk, modify printk to print KB instead of MB, swap
   strncmp for strcmp, reduced memory footprint for shm_heap_banks.
   (Michal)
v2 changes:
 - add static inline get_shmem_heap_banks(), given the changes to the
   struct membanks interface. Rebase changes due to removal of
   owner_dom_io arg from handle_shared_mem_bank.
   Change save_map_heap_pages return type given the changes to the
   allocate_domheap_memory callback type.
---
 xen/arch/arm/static-shmem.c | 187 ++++++++++++++++++++++++++++++------
 1 file changed, 155 insertions(+), 32 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 74c81904b8a4..53e8d3ecf030 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -9,6 +9,25 @@
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
 
+typedef struct {
+    struct domain *d;
+    const char *role_str;
+    paddr_t gbase;
+    struct shmem_membank_extra *bank_extra_info;
+} alloc_heap_pages_cb_extra;
+
+static struct {
+    struct membanks_hdr common;
+    struct membank bank[NR_SHMEM_BANKS];
+} shm_heap_banks __initdata = {
+    .common.max_banks = NR_SHMEM_BANKS
+};
+
+static inline struct membanks *get_shmem_heap_banks(void)
+{
+    return container_of(&shm_heap_banks.common, struct membanks, common);
+}
+
 static void __init __maybe_unused build_assertions(void)
 {
     /*
@@ -63,7 +82,8 @@ static bool __init is_shm_allocated_to_domio(paddr_t pbase)
 }
 
 static mfn_t __init acquire_shared_memory_bank(struct domain *d,
-                                               paddr_t pbase, paddr_t psize)
+                                               paddr_t pbase, paddr_t psize,
+                                               bool bank_from_heap)
 {
     mfn_t smfn;
     unsigned long nr_pfns;
@@ -83,19 +103,31 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     d->max_pages += nr_pfns;
 
     smfn = maddr_to_mfn(pbase);
-    res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+    if ( bank_from_heap )
+        /*
+         * When host address is not provided, static shared memory is
+         * allocated from heap and shall be assigned to owner domain.
+         */
+        res = assign_pages(maddr_to_page(pbase), nr_pfns, d, 0);
+    else
+        res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+
     if ( res )
     {
-        printk(XENLOG_ERR
-               "%pd: failed to acquire static memory: %d.\n", d, res);
-        d->max_pages -= nr_pfns;
-        return INVALID_MFN;
+        printk(XENLOG_ERR "%pd: failed to %s static memory: %d.\n", d,
+               bank_from_heap ? "assign" : "acquire", res);
+        goto fail;
     }
 
     return smfn;
+
+ fail:
+    d->max_pages -= nr_pfns;
+    return INVALID_MFN;
 }
 
 static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       bool bank_from_heap,
                                        const struct membank *shm_bank)
 {
     mfn_t smfn;
@@ -108,10 +140,7 @@ static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
     psize = shm_bank->size;
     nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
-    printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
-           d, pbase, pbase + psize);
-
-    smfn = acquire_shared_memory_bank(d, pbase, psize);
+    smfn = acquire_shared_memory_bank(d, pbase, psize, bank_from_heap);
     if ( mfn_eq(smfn, INVALID_MFN) )
         return -EINVAL;
 
@@ -182,6 +211,7 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
 
 static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
                                          const char *role_str,
+                                         bool bank_from_heap,
                                          const struct membank *shm_bank)
 {
     bool owner_dom_io = true;
@@ -210,7 +240,8 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
          * We found the first borrower of the region, the owner was not
          * specified, so they should be assigned to dom_io.
          */
-        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                   bank_from_heap, shm_bank);
         if ( ret )
             return ret;
     }
@@ -227,6 +258,39 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
     return 0;
 }
 
+static bool __init save_map_heap_pages(struct domain *d, struct page_info *pg,
+                                       unsigned int order, void *extra)
+{
+    alloc_heap_pages_cb_extra *b_extra = (alloc_heap_pages_cb_extra *)extra;
+    int idx = shm_heap_banks.common.nr_banks;
+    int ret = -ENOSPC;
+
+    BUG_ON(!b_extra);
+
+    if ( idx < shm_heap_banks.common.max_banks )
+    {
+        shm_heap_banks.bank[idx].start = page_to_maddr(pg);
+        shm_heap_banks.bank[idx].size = (1ULL << (PAGE_SHIFT + order));
+        shm_heap_banks.bank[idx].shmem_extra = b_extra->bank_extra_info;
+        shm_heap_banks.common.nr_banks++;
+
+        ret = handle_shared_mem_bank(b_extra->d, b_extra->gbase,
+                                     b_extra->role_str, true,
+                                     &shm_heap_banks.bank[idx]);
+        if ( !ret )
+        {
+            /* Increment guest physical address for next mapping */
+            b_extra->gbase += shm_heap_banks.bank[idx].size;
+            return true;
+        }
+    }
+
+    printk("Failed to allocate static shared memory from Xen heap: (%d)\n",
+           ret);
+
+    return false;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -264,38 +328,97 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
-        if ( INVALID_PADDR == pbase )
-        {
-            printk("%pd: host physical address must be chosen by users at the moment", d);
-            return -EINVAL;
-        }
+        /* "role" property is optional */
+        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
+            role_str = NULL;
 
         /*
-         * xen,shared-mem = <pbase, gbase, size>;
-         * TODO: pbase is optional.
+         * xen,shared-mem = <[pbase,] gbase, size>;
+         * pbase is optional.
          */
         addr_cells = dt_n_addr_cells(shm_node);
         size_cells = dt_n_size_cells(shm_node);
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
-        for ( i = 0; i < PFN_DOWN(psize); i++ )
-            if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
-            {
-                printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
-                       d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
-                return -EINVAL;
-            }
+        if ( pbase != INVALID_PADDR )
+        {
+            /* guest phys address is after host phys address */
+            gbase = dt_read_paddr(cells + addr_cells, addr_cells);
+
+            for ( i = 0; i < PFN_DOWN(psize); i++ )
+                if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
+                {
+                    printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
+                        d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
+                    return -EINVAL;
+                }
+
+            /* The host physical address is supplied by the user */
+            ret = handle_shared_mem_bank(d, gbase, role_str, false,
+                                         boot_shm_bank);
+            if ( ret )
+                return ret;
+        }
+        else
+        {
+            /*
+             * The host physical address is not supplied by the user, so it
+             * means that the banks needs to be allocated from the Xen heap,
+             * look into the already allocated banks from the heap.
+             */
+            const struct membank *alloc_bank =
+                find_shm_bank_by_id(get_shmem_heap_banks(), shm_id);
 
-        /* "role" property is optional */
-        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
-            role_str = NULL;
+            /* guest phys address is right at the beginning */
+            gbase = dt_read_paddr(cells, addr_cells);
 
-        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
-        if ( ret )
-            return ret;
+            if ( !alloc_bank )
+            {
+                alloc_heap_pages_cb_extra cb_arg = { d, role_str, gbase,
+                    boot_shm_bank->shmem_extra };
+
+                /* shm_id identified bank is not yet allocated */
+                if ( !allocate_domheap_memory(NULL, psize, save_map_heap_pages,
+                                              &cb_arg) )
+                {
+                    printk(XENLOG_ERR
+                           "Failed to allocate (%"PRIpaddr"KB) pages as static shared memory from heap\n",
+                           psize >> 10);
+                    return -EINVAL;
+                }
+            }
+            else
+            {
+                /* shm_id identified bank is already allocated */
+                const struct membank *end_bank =
+                        &shm_heap_banks.bank[shm_heap_banks.common.nr_banks];
+                paddr_t gbase_bank = gbase;
+
+                /*
+                 * Static shared memory banks that are taken from the Xen heap
+                 * are allocated sequentially in shm_heap_banks, so starting
+                 * from the first bank found identified by shm_id, the code can
+                 * just advance by one bank at the time until it reaches the end
+                 * of the array or it finds another bank NOT identified by
+                 * shm_id
+                 */
+                for ( ; alloc_bank < end_bank; alloc_bank++ )
+                {
+                    if ( strcmp(shm_id, alloc_bank->shmem_extra->shm_id) != 0 )
+                        break;
+
+                    ret = handle_shared_mem_bank(d, gbase_bank, role_str, true,
+                                                 alloc_bank);
+                    if ( ret )
+                        return ret;
+
+                    /* Increment guest physical address for next mapping */
+                    gbase_bank += alloc_bank->size;
+                }
+            }
+        }
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727269.1131754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glg-0007Pm-0l; Wed, 22 May 2024 07:52:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727269.1131754; Wed, 22 May 2024 07:52:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9glf-0007Pb-Tw; Wed, 22 May 2024 07:52:11 +0000
Received: by outflank-mailman (input) for mailman id 727269;
 Wed, 22 May 2024 07:52:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1s9gle-0005YY-5U
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:10 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id 30c1425d-1810-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 09:52:09 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D252E1515;
 Wed, 22 May 2024 00:52:32 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B7E3B3F766;
 Wed, 22 May 2024 00:52:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30c1425d-1810-11ef-90a0-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v3 7/7] xen/docs: Describe static shared memory when host address is not provided
Date: Wed, 22 May 2024 08:51:51 +0100
Message-Id: <20240522075151.3373899-8-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522075151.3373899-1-luca.fancellu@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

This commit describe the new scenario where host address is not provided
in "xen,shared-mem" property and a new example is added to the page to
explain in details.

Take the occasion to fix some typos in the page.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v2:
 - Add Michal R-by
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-10-Penny.Zheng@arm.com/
   with some changes in the commit message.
---
 docs/misc/arm/device-tree/booting.txt | 52 ++++++++++++++++++++-------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2f6..ac4bad6fe5e0 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -590,7 +590,7 @@ communication.
     An array takes a physical address, which is the base address of the
     shared memory region in host physical address space, a size, and a guest
     physical address, as the target address of the mapping.
-    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
+    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >;
 
     It shall also meet the following criteria:
     1) If the SHM ID matches with an existing region, the address range of the
@@ -601,8 +601,8 @@ communication.
     The number of cells for the host address (and size) is the same as the
     guest pseudo-physical address and they are inherited from the parent node.
 
-    Host physical address is optional, when missing Xen decides the location
-    (currently unimplemented).
+    Host physical address is optional, when missing Xen decides the location.
+    e.g. xen,shared-mem = < [guest address] [size] >;
 
 - role (Optional)
 
@@ -629,7 +629,7 @@ chosen {
         role = "owner";
         xen,shm-id = "my-shared-mem-0";
         xen,shared-mem = <0x10000000 0x10000000 0x10000000>;
-    }
+    };
 
     domU1 {
         compatible = "xen,domain";
@@ -640,25 +640,36 @@ chosen {
         vpl011;
 
         /*
-         * shared memory region identified as 0x0(xen,shm-id = <0x0>)
-         * is shared between Dom0 and DomU1.
+         * shared memory region "my-shared-mem-0" is shared
+         * between Dom0 and DomU1.
          */
         domU1-shared-mem@10000000 {
             compatible = "xen,domain-shared-memory-v1";
             role = "borrower";
             xen,shm-id = "my-shared-mem-0";
             xen,shared-mem = <0x10000000 0x50000000 0x10000000>;
-        }
+        };
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between DomU1 and DomU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * DomU1 and DomU2.
          */
         domU1-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x60000000 0x20000000>;
-        }
+        };
+
+        /*
+         * shared memory region "my-shared-mem-2" is shared between
+         * DomU1 and DomU2.
+         */
+        domU1-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "owner";
+            xen,shared-mem = <0x80000000 0x20000000>;
+        };
 
         ......
 
@@ -672,14 +683,21 @@ chosen {
         cpus = <1>;
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between domU1 and domU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * domU1 and domU2.
          */
         domU2-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x70000000 0x20000000>;
-        }
+        };
+
+        domU2-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "borrower";
+            xen,shared-mem = <0x90000000 0x20000000>;
+        };
 
         ......
     };
@@ -699,3 +717,11 @@ shared between DomU1 and DomU2. It will get mapped at 0x60000000 in DomU1 guest
 physical address space, and at 0x70000000 in DomU2 guest physical address space.
 DomU1 and DomU2 are both the borrower domain, the owner domain is the default
 owner domain DOMID_IO.
+
+For the static shared memory region "my-shared-mem-2", since host physical
+address is not provided by user, Xen will automatically allocate 512MB
+from heap as static shared memory to be shared between DomU1 and DomU2.
+The automatically allocated static shared memory will get mapped at
+0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
+guest physical address space. DomU1 is explicitly defined as the owner domain,
+and DomU2 is the borrower domain.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 07:52:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 07:52:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727288.1131764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gmH-0001AL-GC; Wed, 22 May 2024 07:52:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727288.1131764; Wed, 22 May 2024 07:52:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gmH-0001AE-Cq; Wed, 22 May 2024 07:52:49 +0000
Received: by outflank-mailman (input) for mailman id 727288;
 Wed, 22 May 2024 07:52:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9gmG-0006Z1-Eg
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 07:52:48 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 46f3f3cd-1810-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 09:52:46 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5238b7d0494so6243102e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 00:52:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a893a91b9sm1059832166b.7.2024.05.22.00.52.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 00:52:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46f3f3cd-1810-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716364366; x=1716969166; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GCOB4JeNG5xQU2yj2tSqSJzbGGZIC4RzJ677ijFSXRA=;
        b=V+EuCGw9ZqwGgqbiqAB8UPiYcoMGy/L5LrcpcjLitGApB/MxC7ZdcQwwxA5uv4y7LW
         L7dU4qQbuWDsAQOSboyIMx7PeN4dpVwmiYeekqXPO9jIBkGsWuMRdW5TEwR7gdIu08fW
         iFg6vmzjNrxlyWaFguR5TzjPJhbWUjLbtFGfwreChaaD6iI5l0OOFD0Hd44yPbM1dp5S
         EiykNh4z0yBhsBzz7AGDOb7ngFzyW8nZB0JVuzJ/YZ9UsRrapVC53mS7hKa/Nuo5GELS
         vHWshRKYKZuBV2Tpf69tTUfhw0QUFNV6ai6Hh/iymhpaQEHfzFzzqV2BQoRhc2y6U2va
         klDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716364366; x=1716969166;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GCOB4JeNG5xQU2yj2tSqSJzbGGZIC4RzJ677ijFSXRA=;
        b=s6uw2IZjxJeXsjkmsGhBS0mKPiURhNUmnzGxzPpn80qLKF/N3DZMCg8NOOtvb3zyNJ
         dy179aQ1+srRhEnqmaSUgn/csKTNEv8+5MK9GAIugR7D0l7KbngwhU/CwKsYCq43gr8c
         hvQlPh8ygX3GmMkBLjnJua3hCaMHsOGu/nb9flksJXeT1U3isYeo/xvB8npfvlqqE/Fl
         tmdexurSeBZpc0cyEKgKImxdhVbrxrjUc4c9ONh9vA1AkCpRqCnr3xxIqSNctZy4eNdv
         dCu+GawzmB/2KaGr5df6nHXU4h8upPCsyY/iTiF754IDJ1+6sZNbAzZ/0K8JK8tSFrs4
         FGLA==
X-Forwarded-Encrypted: i=1; AJvYcCVHzWErVRXnqP8MlQNvJg9j21FETPNeiBvYjT+3oNrEOA8KChl+7zRg3hbm4/cJDJKv/iSGbIwOkrfA5MlwOl2Ickvf6PEMRGpLFYqTutk=
X-Gm-Message-State: AOJu0Yy5qeO/jXrnO543hqXejwQahpanQSNgqz5bkYOgDyWt454kkybL
	6du8dywqx3B9CMQ6ZQnKo+5RhkZ6YSx0H9RLBz5Sn5VvhWW679OS2IxtcVgqTg==
X-Google-Smtp-Source: AGHT+IGAM6NRvz0OIvbVysBcvW+HGDRWtViIpJYmOkWlHHJ/bX7slkm3WJxsUjr96TO60yKEwDV/LQ==
X-Received: by 2002:ac2:5e3b:0:b0:51d:5ffd:efe4 with SMTP id 2adb3069b0e04-526beca95f8mr1089686e87.1.1716364365778;
        Wed, 22 May 2024 00:52:45 -0700 (PDT)
Message-ID: <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>
Date: Wed, 22 May 2024 09:52:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -2009,6 +2009,14 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,
>          goto out_put_gfn;
>      }
>  
> +    if ( (p2mt == p2m_mmio_direct) && npfec.write_access && npfec.present &&
> +         subpage_mmio_write_accept(mfn, gla) &&

Afaics subpage_mmio_write_accept() is unreachable then when CONFIG_HVM=n?

> +         (hvm_emulate_one_mmio(mfn_x(mfn), gla) == X86EMUL_OKAY) )
> +    {
> +        rc = 1;
> +        goto out_put_gfn;
> +    }

Overall this new if() is pretty similar to the immediate preceding one.
So similar that I wonder whether the two shouldn't be folded. In fact
it looks as if the new one is needed only for the case where you'd pass
through (to a DomU) a device partially used by Xen. That could certainly
do with mentioning explicitly.

> +static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *entry)

Considering what the function does and what it returns, perhaps better
s/get/map/? The "get_page" part of the name generally has a different
meaning in Xen's memory management.

> +{
> +    void __iomem *mapped_page;
> +
> +    if ( entry->mapped )
> +        return entry->mapped;
> +
> +    mapped_page = ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE);
> +
> +    spin_lock(&subpage_ro_lock);
> +    /* Re-check under the lock */
> +    if ( entry->mapped )
> +    {
> +        spin_unlock(&subpage_ro_lock);
> +        iounmap(mapped_page);

The only unmap is on an error path here and on another error path elsewhere.
IOW it looks as if devices with such marked pages are meant to never be hot
unplugged. I can see that being intentional for the XHCI console, but imo
such a restriction also needs prominently calling out in a comment next to
e.g. the function declaration.

> +        return entry->mapped;
> +    }
> +
> +    entry->mapped = mapped_page;
> +    spin_unlock(&subpage_ro_lock);
> +    return entry->mapped;
> +}
> +
> +static void subpage_mmio_write_emulate(
> +    mfn_t mfn,
> +    unsigned int offset,
> +    const void *data,
> +    unsigned int len)
> +{
> +    struct subpage_ro_range *entry;
> +    void __iomem *addr;

Wouldn't this better be pointer-to-volatile, with ...

> +    list_for_each_entry(entry, &subpage_ro_ranges, list)
> +    {
> +        if ( mfn_eq(entry->mfn, mfn) )
> +        {
> +            if ( test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords) )
> +            {
> + write_ignored:
> +                gprintk(XENLOG_WARNING,
> +                        "ignoring write to R/O MMIO 0x%"PRI_mfn"%03x len %u\n",
> +                        mfn_x(mfn), offset, len);
> +                return;
> +            }
> +
> +            addr = subpage_mmio_get_page(entry);
> +            if ( !addr )
> +            {
> +                gprintk(XENLOG_ERR,
> +                        "Failed to map page for MMIO write at 0x%"PRI_mfn"%03x\n",
> +                        mfn_x(mfn), offset);
> +                return;
> +            }
> +
> +            switch ( len )
> +            {
> +            case 1:
> +                writeb(*(const uint8_t*)data, addr);
> +                break;
> +            case 2:
> +                writew(*(const uint16_t*)data, addr);
> +                break;
> +            case 4:
> +                writel(*(const uint32_t*)data, addr);
> +                break;
> +            case 8:
> +                writeq(*(const uint64_t*)data, addr);
> +                break;

... this being how it's written? (If so, volatile suitably carried through to
other places as well.)

> +            default:
> +                /* mmio_ro_emulated_write() already validated the size */
> +                ASSERT_UNREACHABLE();
> +                goto write_ignored;
> +            }
> +            return;
> +        }
> +    }
> +    /* Do not print message for pages without any writable parts. */
> +}
> +
> +bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla)
> +{
> +    unsigned int offset = PAGE_OFFSET(gla);
> +    const struct subpage_ro_range *entry;
> +
> +    list_for_each_entry_rcu(entry, &subpage_ro_ranges, list)

Considering the other remark about respective devices impossible to go
away, is the RCU form here really needed? Its use gives the (false)
impression of entry removal being possible.

> +        if ( mfn_eq(entry->mfn, mfn) &&
> +             !test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qwords) )

Btw, "qwords" in the field name is kind of odd when SUBPAGE_MMIO_RO_ALIGN
in principle suggests that changing granularity ought to be possible by
simply adjusting that #define. Maybe "->ro_elems"?

> --- a/xen/arch/x86/pv/ro-page-fault.c
> +++ b/xen/arch/x86/pv/ro-page-fault.c
> @@ -330,6 +330,7 @@ static int mmio_ro_do_page_fault(struct x86_emulate_ctxt *ctxt,
>              return X86EMUL_UNHANDLEABLE;
>      }
>  
> +    mmio_ro_ctxt.mfn = mfn;
>      ctxt->data = &mmio_ro_ctxt;
>      if ( pci_ro_mmcfg_decode(mfn_x(mfn), &mmio_ro_ctxt.seg, &mmio_ro_ctxt.bdf) )
>          return x86_emulate(ctxt, &mmcfg_intercept_ops);

Wouldn't you better set .mfn only on the "else" path, just out of context?
Suggesting that the new field in the struct could actually overlay the
(seg,bdf) tuple (being of relevance only to MMCFG intercept handling).
This would be more for documentation purposes than to actually save space.
(If so, perhaps the "else" itself would also better be dropped while making
the adjustment.)

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 08:05:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:05:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727315.1131774 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gyC-0003oI-Oz; Wed, 22 May 2024 08:05:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727315.1131774; Wed, 22 May 2024 08:05:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9gyC-0003oB-Lj; Wed, 22 May 2024 08:05:08 +0000
Received: by outflank-mailman (input) for mailman id 727315;
 Wed, 22 May 2024 08:05:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9gyB-0003o5-Ql
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:05:07 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00519e2e-1812-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:05:06 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-57825ff689bso1717054a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 01:05:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea65e2sm17909588a12.19.2024.05.22.01.05.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 01:05:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00519e2e-1812-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716365106; x=1716969906; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0oWIspwXriEJHqp2br2tUh4rUaOxLFO8PrsC0tRtBIo=;
        b=LR8+uo9wM0a0Ub0qU/qfme4z9DvToYuTnMy+20fAPhV9h46g4JP6ArbmTBxFtzZaKJ
         TIJzlq6s+5cdLEQGWWR8KSAg7qtGVTS4bNhick0pS2LNqH3QmzYY15vUgZIHXyfY9Wn+
         oLW42sULap/Om+AxAV/FaT1DC8beASdyWzPRRbHLLeos9DNXZ4neWXwSvb5b6mXPnUm6
         L8BaNaxVtWpCMlNoPWIYlVKPZBOvuHhAMoQ2KNvN1i4MmbmL28p7YPU3Wf0HOfK4bcxR
         NsfxGsAN95ORhiPbEAz4n3C7V8GArnmGwKWzvTXRfS/dMua+Hj9k3xq1dZu2Tc1/fbwS
         7mHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716365106; x=1716969906;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0oWIspwXriEJHqp2br2tUh4rUaOxLFO8PrsC0tRtBIo=;
        b=MFwPJ0Ypg5sAjsCjFnOO+KWfAyzJlpPSUGo/6eFusSNH/sPI2Xw7M1jrT/vWt3v/TA
         J+FHBd5h6pO+4EZo8o+zE9dpEnCDF0sCgTOjjm6l63ce8d4SCr1YOqx4hcRoUIZxwOf7
         ze0rqiCCPRb6hWbmx8PqZnmSMDJzINFNE3yYFHqNSUM2xx41QUiyvUInTu83LMN/16L1
         kgMgptF0HaZtigGe8UrMBRAflDBt0t/xc4wdA+4KU8fc6XYT69I8TgHWX9j1XDuyA5PS
         eh4tzNuHr9xwpIXj9PTzc/fNM4+a0goPJz8ZcOXVMC5x8L4qMWwCfWsgJKnnbns4K49K
         WERw==
X-Forwarded-Encrypted: i=1; AJvYcCUj3pZ8eVEV/5IbJYs777X7lz9H9XPQ+4qEUclz2pT0kJjvHsRMbZx9qlU80Npvj5qzcXG5psUO5rSYvcKUkHecQP+wi9EkfNwhOyH3R3U=
X-Gm-Message-State: AOJu0YzBF72y53mz2azBj9BUKT4SvDTueTf7ak+Et9tNm7fMC6QSyYxX
	41ddekj6e3IEeB8F0QWrf7eDGJojKAICEfZ1Qr+te6u7at864ePnF3PrcWfHZA==
X-Google-Smtp-Source: AGHT+IF/PreU14Xh9SXa+H0Ho1Li59vJe6rA4TV8PweRe4sPsb6aOujFKFJkm1oJ1pk4kFSTJAdiEw==
X-Received: by 2002:a50:9301:0:b0:573:1e9e:bb54 with SMTP id 4fb4d7f45d1cf-578329ec5a5mr717277a12.8.1716365106384;
        Wed, 22 May 2024 01:05:06 -0700 (PDT)
Message-ID: <0dbc701d-d8e6-4cbf-b7bf-27d9e05b5491@suse.com>
Date: Wed, 22 May 2024 10:05:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/2] drivers/char: Use sub-page ro API to make just
 xhci dbc cap RO
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <7682bbf3a24045ae1fcbdf651a6dd8e609ec3a20.1716260066.git-series.marmarek@invisiblethingslab.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7682bbf3a24045ae1fcbdf651a6dd8e609ec3a20.1716260066.git-series.marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
> --- a/xen/drivers/char/xhci-dbc.c
> +++ b/xen/drivers/char/xhci-dbc.c
> @@ -1216,20 +1216,19 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port)
>          break;
>      }
>  #ifdef CONFIG_X86
> -    /*
> -     * This marks the whole page as R/O, which may include other registers
> -     * unrelated to DbC. Xen needs only DbC area protected, but it seems
> -     * Linux's XHCI driver (as of 5.18) works without writting to the whole
> -     * page, so keep it simple.
> -     */
> -    if ( rangeset_add_range(mmio_ro_ranges,
> -                PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
> -                         uart->dbc.xhc_dbc_offset),
> -                PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
> -                       uart->dbc.xhc_dbc_offset +
> -                sizeof(*uart->dbc.dbc_reg)) - 1) )
> -        printk(XENLOG_INFO
> -               "Error while adding MMIO range of device to mmio_ro_ranges\n");
> +    if ( subpage_mmio_ro_add(
> +             (uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
> +              uart->dbc.xhc_dbc_offset,
> +             sizeof(*uart->dbc.dbc_reg)) )
> +    {
> +        printk(XENLOG_WARNING
> +               "Error while marking MMIO range of XHCI console as R/O, "
> +               "making the whole device R/O (share=no)\n");

Since you mention "share=no" here, wouldn't you then better also update the
respective struct field, even if (right now) there may be nothing subsequently
using that? Except that dbc_ensure_running() actually is looking at it, and
that's not an __init function.

> +        if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
> +            printk(XENLOG_WARNING
> +                   "Failed to mark read-only %pp used for XHCI console\n",
> +                   &uart->dbc.sbdf);
> +    }
>  #endif
>  }

It's been a long time since v2 and the description doesn't say anything in
this regard: Is there a reason not to retain the rangeset addition alongside
the pci_ro_device() on the fallback path?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 08:08:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:08:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727320.1131783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9h12-0004N5-5F; Wed, 22 May 2024 08:08:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727320.1131783; Wed, 22 May 2024 08:08:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9h12-0004My-2i; Wed, 22 May 2024 08:08:04 +0000
Received: by outflank-mailman (input) for mailman id 727320;
 Wed, 22 May 2024 08:08:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9h10-0004Ms-H4
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:08:02 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 67d52215-1812-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 10:08:00 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-523b017a5c6so9206280e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 01:08:00 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdd4a2793sm895867266b.0.2024.05.22.01.07.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 01:07:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67d52215-1812-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716365280; x=1716970080; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FPL+Uni1yJqIWR95mE7XmWQsFKJtpoYLHfuEejgmN9w=;
        b=f2qYG7pWiOXltp3vcldX158hMvkoPD8ZpEizv4YNgq2xpKx3uFdXl5LflHclPFNTNG
         RZpU1rpGwVGHhnUhyxQNFyhpy5sq9RnTqOUQyHwqb+1KIr37W0cQTUD279kqJIRD3pXS
         m5gS4lePz31H4g0VQGUC57dbHmDFn74u6haK2blf8LILeZ7NJt9lxrEAe5/47K7e2+WZ
         bXc7dVsDMhZnSyHV0lUaYNQMBM7FTD7egkCOE9ZTW59qQcfbF3yEF8QoAYIcV3qD46RE
         VtC3AvdWFszaPspP3sghC977fw01MapM2rPmh5u4yQDQL6sb5Q223FrOiW1zarY88r4O
         FUnA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716365280; x=1716970080;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FPL+Uni1yJqIWR95mE7XmWQsFKJtpoYLHfuEejgmN9w=;
        b=VclO1y/Yo64b/cFfp2l/Httf0kladWEJVvhc3ekXncp56ICrZ8U2GuvoIGPW9d40H7
         Z/nRH10Ssq8Wo9H1SfURLbazkq0mLgBWItE0meFzgogrdECyC/5u91poMv499Tm9phrV
         UEiZAkNW61S8bhoy4WJDdO+gSagfigShLlK4eplzFktt9n9KxEXLkHgJ6y6ieHMTevub
         8psl85eBF3LIpFlut99dYS9HGNAUQ+OwxfONt7EgYM9gA09qTWijSe/GOCP9jQ/Gw/6e
         +Upn0SkBVEVF+eF9nByyGkhsI/38BSgWJvK8tcgLYYLfBxs6IH3E32bZwTzoeNvakNZP
         S6IQ==
X-Gm-Message-State: AOJu0YwpVwWLslFVlMvdk/xR+VyaTC4diQvA7Aole8hcop8HA2rWDGbZ
	8VOzLJGHMzoofvk9bihpeRlks3dUodsiT1ymM8qHPNm1OGcRyvEP5jCFuaAs5gS/vibtANicxP4
	=
X-Google-Smtp-Source: AGHT+IEGOKobW5DaUipjVA/f1L7V7p2phujRxc45B43HOYccZlPgCH9TLh0Izo0/ebSMNSxKDRMafg==
X-Received: by 2002:a05:6512:2011:b0:523:772c:2c1c with SMTP id 2adb3069b0e04-526bf82cc11mr789486e87.38.1716365278742;
        Wed, 22 May 2024 01:07:58 -0700 (PDT)
Message-ID: <7655e401-b927-4250-ae63-05361a5ee71d@suse.com>
Date: Wed, 22 May 2024 10:07:57 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 0/2] Add API for making parts of a MMIO page R/O and
 use it in XHCI console
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
> On older systems, XHCI xcap had a layout that no other (interesting) registers
> were placed on the same page as the debug capability, so Linux was fine with
> making the whole page R/O. But at least on Tiger Lake and Alder Lake, Linux
> needs to write to some other registers on the same page too.
> 
> Add a generic API for making just parts of an MMIO page R/O and use it to fix
> USB3 console with share=yes or share=hwdom options. More details in commit
> messages.
> 
> Technically it may still qualify for 4.19, since v1 was sent well before
> last posting date. But I realize it's quite late and it isn't top
> priority series, so if it won't hit 4.19, it's okay with me too.
> 
> Marek Marczykowski-Górecki (2):
>   x86/mm: add API for marking only part of a MMIO page read only
>   drivers/char: Use sub-page ro API to make just xhci dbc cap RO
> 
>  xen/arch/x86/hvm/emulate.c      |   2 +-
>  xen/arch/x86/hvm/hvm.c          |   8 +-
>  xen/arch/x86/include/asm/mm.h   |  18 ++-
>  xen/arch/x86/mm.c               | 268 +++++++++++++++++++++++++++++++++-
>  xen/arch/x86/pv/ro-page-fault.c |   1 +-
>  xen/drivers/char/xhci-dbc.c     |  27 +--
>  6 files changed, 309 insertions(+), 15 deletions(-)

Just to mention it here again, with v2 having been quite some time ago:
Like for that other work of yours I'm not really convinced the complexity
is worth the gain. Ultimately this may once again mean that I'll demand
a 2nd maintainer's ack, once technically I may be okay to offer R-b.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 08:15:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:15:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727324.1131794 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9h8I-0006Ej-TJ; Wed, 22 May 2024 08:15:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727324.1131794; Wed, 22 May 2024 08:15:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9h8I-0006Ec-QY; Wed, 22 May 2024 08:15:34 +0000
Received: by outflank-mailman (input) for mailman id 727324;
 Wed, 22 May 2024 08:15:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9h8G-0006EU-Um
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:15:32 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 74f4dbcb-1813-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:15:31 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a387fbc9so979434066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 01:15:31 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8a55f2e8sm1054428466b.126.2024.05.22.01.15.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 01:15:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74f4dbcb-1813-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716365731; x=1716970531; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=I07GPXVjk/wNjF5w8+JeEg5eMFs3z+5p0J67OkCSbm8=;
        b=OUriiPb7Qblw97E4UFZFTvvx8Iu/PeQg+NiVnUKs9DitjmRFjpyRdQHeX4VRvrWOKT
         mm4ttd3jzdSxAgAeU9x0Ua0UEauy9Ks+YWVlpefZUKWwpA8XTv8+UcO1/WeDSKOWIEHc
         G5GTa1xKILYx07Y/auBFQZcFXWSJCTXPg4aU+TkDE2rSrgz4P805WcdxDDPog++VMzdR
         bkLWGsLj4+i+DOPur2YrosFqv55FoRXsQOF0/IKb9a2DQl1LAt4HcG3K5ujjcfa0msq6
         SIltmd7T7aqJ5Hc3alHDW9ETd9LBVbNQP20hi13YMzJ0tGy1sVuGSlrbedujLD7VMpLv
         NP0w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716365731; x=1716970531;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=I07GPXVjk/wNjF5w8+JeEg5eMFs3z+5p0J67OkCSbm8=;
        b=EKsmEEQ5cmjF9yivolHnrSd4y+IB+HRrnouLjUBZWwNrO/EUULZF6rg6Ux14epCmBn
         2BYHDAt2is7325KgQRKTJ7oDFiXL4EoPHxhTS4AQ07Nv5XhhiW5+kW+JqpYocsrY0mEW
         PUJSIjGyVLAJgaglsv8A7ZG6XCp2CAxKoqPYZRK/uXil9Oin4gCs3D/yBKahb+tWtJn5
         VoOK27/2/LJ55BPPzJgvBBVe5CMDl4Lf2kDaA0AQxko/uOON97/Qu6FrGHv2vvkTJ7xz
         ZTWh+kVa1nxiOPtrIr/ef1BdVDMlkIs5iiGGVa7dGSeWo5CSOoU5CpGgDu4b93EUmIxP
         O7lA==
X-Forwarded-Encrypted: i=1; AJvYcCUzLR5QGeOHjOqMzgJbKX6o4xfauLJVW6Kqezl+S9+rSQuHvINj06jslN7VrwaA582kq/EcMyyHRVdcKm16SGPI/D7Voua8Q+FoFkBkHm8=
X-Gm-Message-State: AOJu0YzSL5Xi8fO3Um3TiFLHjZWaWv0KHsXl5+YL3AJrd/iu/cvSnK2L
	+3ceIg4thNy9Pt1M0fVcLg5MrYopXgfdmG/F4eCdEttui1f+iSRPx0bMSmXdLg==
X-Google-Smtp-Source: AGHT+IEq9vhuF6IA/pC+hMVG6JtiF5y6XQnWVQDsV7lejdlkTWk+JfCp/li40vC7FB0+k4P0zExyWA==
X-Received: by 2002:a17:906:c302:b0:a59:b8e2:a0c5 with SMTP id a640c23a62f3a-a6228143abfmr61026666b.51.1716365731340;
        Wed, 22 May 2024 01:15:31 -0700 (PDT)
Message-ID: <df910cf3-b784-48a6-a0af-32527302fb9f@suse.com>
Date: Wed, 22 May 2024 10:15:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 03/14] xen/bitops: implement fls{l}() in common logic
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
 <d26a4033-8cd1-4a9c-8549-49a36b7d52fb@suse.com>
 <9e59da99eb87a7d3fcd8cc6a3c6d6ef2753736af.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9e59da99eb87a7d3fcd8cc6a3c6d6ef2753736af.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 09:37, Oleksii K. wrote:
> On Tue, 2024-05-21 at 13:18 +0200, Jan Beulich wrote:
>> On 17.05.2024 15:54, Oleksii Kurochko wrote:
>>> To avoid the compilation error below, it is needed to update to
>>> places
>>> in common/page_alloc.c where flsl() is used as now flsl() returns
>>> unsigned int:
>>>
>>> ./include/xen/kernel.h:18:21: error: comparison of distinct pointer
>>> types lacks a cast [-Werror]
>>>        18 |         (void) (&_x == &_y);            \
>>>           |                     ^~
>>>     common/page_alloc.c:1843:34: note: in expansion of macro 'min'
>>>      1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e
>>> - s) - 1);
>>>
>>> generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is
>>> 0,
>>> the result in undefined.
>>>
>>> The prototype of the per-architecture fls{l}() functions was
>>> changed to
>>> return 'unsigned int' to align with the generic implementation of
>>> these
>>> functions and avoid introducing signed/unsigned mismatches.
>>>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>>  The patch is almost independent from Andrew's patch series
>>>  (
>>> https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
>>> )
>>>  except test_fls() function which IMO can be merged as a separate
>>> patch after Andrew's patch
>>>  will be fully ready.
>>
>> If there wasn't this dependency (I don't think it's "almost
>> independent"),
>> I'd be offering R-b with again one nit below.
> 
> Aren't all changes, except those in xen/common/bitops.c, independent? I
> could move these changes in xen/common/bitops.c to a separate commit. I
> think it is safe to commit them ( an introduction of common logic for
> fls{l}() and tests ) separately since the CI tests have passed.

Technically they might be, but contextually there are further conflicts.
Just try "patch --dry-run" on top of a plain staging tree. You really
need to settle, perhaps consulting Andrew, whether you want to go on top
of his change, or ahead of it. I'm not willing to approve a patch that's
presented one way but then is (kind of) expected to go in the other way.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 08:38:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:38:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727336.1131803 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hUB-0001F1-PZ; Wed, 22 May 2024 08:38:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727336.1131803; Wed, 22 May 2024 08:38:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hUB-0001Eu-Mz; Wed, 22 May 2024 08:38:11 +0000
Received: by outflank-mailman (input) for mailman id 727336;
 Wed, 22 May 2024 08:38:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=V5RZ=MZ=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s9hUA-0001Eo-Th
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:38:10 +0000
Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9cf9d6f7-1816-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:38:09 +0200 (CEST)
Received: from pb-smtp21.pobox.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 3439D3575F;
 Wed, 22 May 2024 04:38:06 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1])
 by pb-smtp21.pobox.com (Postfix) with ESMTP id 2BA023575E;
 Wed, 22 May 2024 04:38:06 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.97])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 3BDF13575D;
 Wed, 22 May 2024 04:38:03 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cf9d6f7-1816-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:mime-version:content-transfer-encoding;
	 s=sasl; bh=cyvIiPqvM/7nZVlXaYF1EcDqHBNiPDg8MTnalFi55SY=; b=osH6
	QGqPLjIRfxya/+3drkSECNMz2C7FTIws5l09b5xPKfEa1ieJJirP0NhDxKg1xCI0
	wUJSn++r8Am6Jxuvxx+aj3rIwxK12Q1cwNP1/j6X7D8pz6o5aeQY16njk0EzTrSZ
	b+CyJbpMg2TvE3E9rmNIotzoqYWN85iYFzKnWVg=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v4 0/3] x86: make Intel/AMD vPMU & MCE support configurable
Date: Wed, 22 May 2024 11:37:59 +0300
Message-Id: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
X-Pobox-Relay-ID:
 9A8663D4-1816-11EF-830E-A19503B9AAD1-90055647!pb-smtp21.pobox.com
Content-Transfer-Encoding: quoted-printable

Three remaining patches to separate support of Intel & AMD CPUs in Xen bu=
ild.
Most of related patches from previous series had already been merged.
Specific changes since v3 are provided per-patch.

v3 series here:
https://lore.kernel.org/xen-devel/cover.1715673586.git.Sergiy_Kibrik@epam=
.com/

  -Sergiy

Sergiy Kibrik (3):
  x86/intel: move vmce_has_lmce() routine to header
  x86/MCE: add default switch case in init_nonfatal_mce_checker()
  x86/MCE: optional build of AMD/Intel MCE code

 xen/arch/x86/cpu/mcheck/Makefile    | 8 ++++----
 xen/arch/x86/cpu/mcheck/mce.c       | 4 ++++
 xen/arch/x86/cpu/mcheck/mce.h       | 5 +++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 8 --------
 xen/arch/x86/cpu/mcheck/non-fatal.c | 9 +++++++++
 xen/arch/x86/cpu/mcheck/vmce.c      | 5 ++---
 xen/arch/x86/include/asm/mce.h      | 1 -
 xen/arch/x86/msr.c                  | 2 ++
 8 files changed, 26 insertions(+), 16 deletions(-)

--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 08:40:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:40:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727339.1131814 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hWA-0002mI-4T; Wed, 22 May 2024 08:40:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727339.1131814; Wed, 22 May 2024 08:40:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hWA-0002mB-1Z; Wed, 22 May 2024 08:40:14 +0000
Received: by outflank-mailman (input) for mailman id 727339;
 Wed, 22 May 2024 08:40:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=V5RZ=MZ=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s9hW8-0002m1-SO
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:40:12 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e684e0a9-1816-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:40:11 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 21F1F35779;
 Wed, 22 May 2024 04:40:10 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 1919535777;
 Wed, 22 May 2024 04:40:10 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.97])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 9684035776;
 Wed, 22 May 2024 04:40:08 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e684e0a9-1816-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=/X53LE/dz4N5HVUlkrUINn3qO
	TL2CeuDunQfSqMtggo=; b=onfZYvPkj8IbEJkz9sPXJpYcpY8exJiNsIIYl8wde
	QMDG5tjDXTz8isSiP7tlytN2ZrHu/ik7pMG7FUZY7kwGPKClpL3RD0GG1J280xVZ
	yZwTXRZ1BoUyjpoXxpIi6uPiFHuQBSN8D/cfk3SvWD9ax1bqHeHEWRCOe8ddN1bx
	KA=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v4 1/3] x86/intel: move vmce_has_lmce() routine to header
Date: Wed, 22 May 2024 11:40:06 +0300
Message-Id: <db547833fa3887e2e36c8418d3f8a142bebb847d.1716366581.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 E5427ACA-1816-11EF-B91C-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

Moving this function out of mce_intel.c will make it possible to disable
build of Intel MCE code later on, because the function gets called from
common x86 code.

Also replace boilerplate code that checks for MCG_LMCE_P flag with
vmce_has_lmce(), which might contribute to readability a bit.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v4:
 - changed description a bit
changes in v3:
 - do not check for CONFIG_INTEL
 - remove CONFIG_INTEL from patch description
changes in v2:
 - move vmce_has_lmce() to cpu/mcheck/mce.h
 - move IS_ENABLED(CONFIG_INTEL) check inside vmce_has_lmce()
 - changed description
---
 xen/arch/x86/cpu/mcheck/mce.h       | 5 +++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ----
 xen/arch/x86/cpu/mcheck/vmce.c      | 5 ++---
 xen/arch/x86/include/asm/mce.h      | 1 -
 xen/arch/x86/msr.c                  | 2 ++
 5 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/mce.h b/xen/arch/x86/cpu/mcheck/mce.=
h
index 4806405f96..eba4b536c7 100644
--- a/xen/arch/x86/cpu/mcheck/mce.h
+++ b/xen/arch/x86/cpu/mcheck/mce.h
@@ -170,6 +170,11 @@ static inline int mce_bank_msr(const struct vcpu *v,=
 uint32_t msr)
     return 0;
 }
=20
+static inline bool vmce_has_lmce(const struct vcpu *v)
+{
+    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
+}
+
 struct mce_callbacks {
     void (*handler)(const struct cpu_user_regs *regs);
     bool (*check_addr)(uint64_t status, uint64_t misc, int addr_type);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mchec=
k/mce_intel.c
index 3f5199b531..af43281cc6 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -1050,7 +1050,3 @@ int vmce_intel_rdmsr(const struct vcpu *v, uint32_t=
 msr, uint64_t *val)
     return 1;
 }
=20
-bool vmce_has_lmce(const struct vcpu *v)
-{
-    return v->arch.vmce.mcg_cap & MCG_LMCE_P;
-}
diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmc=
e.c
index 4da6f4a3e4..5abdf4cb5f 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -203,7 +203,7 @@ int vmce_rdmsr(uint32_t msr, uint64_t *val)
          * bits are always set in guest MSR_IA32_FEATURE_CONTROL by Xen,=
 so it
          * does not need to check them here.
          */
-        if ( cur->arch.vmce.mcg_cap & MCG_LMCE_P )
+        if ( vmce_has_lmce(cur) )
         {
             *val =3D cur->arch.vmce.mcg_ext_ctl;
             mce_printk(MCE_VERBOSE, "MCE: %pv: rd MCG_EXT_CTL %#"PRIx64"=
\n",
@@ -332,8 +332,7 @@ int vmce_wrmsr(uint32_t msr, uint64_t val)
         break;
=20
     case MSR_IA32_MCG_EXT_CTL:
-        if ( (cur->arch.vmce.mcg_cap & MCG_LMCE_P) &&
-             !(val & ~MCG_EXT_CTL_LMCE_EN) )
+        if ( vmce_has_lmce(cur) && !(val & ~MCG_EXT_CTL_LMCE_EN) )
             cur->arch.vmce.mcg_ext_ctl =3D val;
         else
             ret =3D -1;
diff --git a/xen/arch/x86/include/asm/mce.h b/xen/arch/x86/include/asm/mc=
e.h
index 6ce56b5b85..2ec47a71ae 100644
--- a/xen/arch/x86/include/asm/mce.h
+++ b/xen/arch/x86/include/asm/mce.h
@@ -41,7 +41,6 @@ extern void vmce_init_vcpu(struct vcpu *v);
 extern int vmce_restore_vcpu(struct vcpu *v, const struct hvm_vmce_vcpu =
*ctxt);
 extern int vmce_wrmsr(uint32_t msr, uint64_t val);
 extern int vmce_rdmsr(uint32_t msr, uint64_t *val);
-extern bool vmce_has_lmce(const struct vcpu *v);
 extern int vmce_enable_mca_cap(struct domain *d, uint64_t cap);
=20
 DECLARE_PER_CPU(unsigned int, nr_mce_banks);
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 9babd441f9..b0ec96f021 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -24,6 +24,8 @@
=20
 #include <public/hvm/params.h>
=20
+#include "cpu/mcheck/mce.h"
+
 DEFINE_PER_CPU(uint32_t, tsc_aux);
=20
 int init_vcpu_msr_policy(struct vcpu *v)
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 08:42:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:42:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727345.1131824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hY7-0003cB-FG; Wed, 22 May 2024 08:42:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727345.1131824; Wed, 22 May 2024 08:42:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9hY7-0003c4-CN; Wed, 22 May 2024 08:42:15 +0000
Received: by outflank-mailman (input) for mailman id 727345;
 Wed, 22 May 2024 08:42:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=V5RZ=MZ=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s9hY5-0003br-Nw
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:42:13 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2efde923-1817-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:42:12 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 243D835794;
 Wed, 22 May 2024 04:42:12 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 1DDC335793;
 Wed, 22 May 2024 04:42:12 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.97])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 5811F35792;
 Wed, 22 May 2024 04:42:11 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2efde923-1817-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=b2BmrhjIaVjxHWLhcCazmWsg7
	7Ut75EvzLcrNm5R+Dc=; b=SyosGijh/FEKLb6xHf9QCa8Sn9Fc2Xda9OfbIcTso
	mxo9nBAN+DZUlI4DtvsGoxtN8gbaoq31C8hwp8wFSU8aCnmLdAbyoUqvxQYtifBN
	ldafFpWUqvW0MGik1tP3nY8zd5osL/tVtM7iY60D6z7Ku5EvoCWqz1Mk4P1bkYYN
	eU=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v4 2/3] x86/MCE: add default switch case in init_nonfatal_mce_checker()
Date: Wed, 22 May 2024 11:42:09 +0300
Message-Id: <9ec888c37a261c86a52f360146cbbb3ad437ad30.1716366581.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 2E697050-1817-11EF-ADA8-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

The default switch case block is wanted here, to handle situation
e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, bu=
t
misleading message still gets logged anyway.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Jan Beulich <jbeulich@suse.com>
---
changes in v4:
 - return 0 instead of -ENODEV and put a comment
 - update description a bit
---
 xen/arch/x86/cpu/mcheck/non-fatal.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mchec=
k/non-fatal.c
index 33cacd15c2..5a53bcd0b7 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -29,9 +29,14 @@ static int __init cf_check init_nonfatal_mce_checker(v=
oid)
 		/* Assume we are on K8 or newer AMD or Hygon CPU here */
 		amd_nonfatal_mcheck_init(c);
 		break;
+
 	case X86_VENDOR_INTEL:
 		intel_nonfatal_mcheck_init(c);
 		break;
+
+	default:
+		/* unhandled vendor isn't really an error */
+		return 0;
 	}
 	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n")=
;
 	return 0;
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 08:44:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 08:44:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727352.1131833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ha7-00049h-P0; Wed, 22 May 2024 08:44:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727352.1131833; Wed, 22 May 2024 08:44:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ha7-00049a-MG; Wed, 22 May 2024 08:44:19 +0000
Received: by outflank-mailman (input) for mailman id 727352;
 Wed, 22 May 2024 08:44:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=V5RZ=MZ=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1s9ha6-00049U-Cc
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 08:44:18 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7926cbdf-1817-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 10:44:17 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 357263579F;
 Wed, 22 May 2024 04:44:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 1E8333579E;
 Wed, 22 May 2024 04:44:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.97])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 8621E3579D;
 Wed, 22 May 2024 04:44:14 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7926cbdf-1817-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=7TGP+W8jkZr2yV0hnrertRwq7
	O7r+p4IX9AQgP14leA=; b=YPbKPtoPohuFUMfxlv3ra2i8mxjUzTgwmnmJC4pCp
	SvSHenornZ/YLsyNdaYv3/CEgmR8xPw2I+PkVyXTDA6zW5TPayzqllRnWIa1oPrr
	SsmpZfnH5BCEO1un5V2JaS92k+Nyao8ee+yU7/nBppi9513li9luorCB35UgJDN7
	JY=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: xen-devel@lists.xenproject.org
Cc: Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH v4 3/3] x86/MCE: optional build of AMD/Intel MCE code
Date: Wed, 22 May 2024 11:44:12 +0300
Message-Id: <ceed2e3492037f198c407b10e41e0e086f8b0e1c.1716366581.git.Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 77D6EBFA-1817-11EF-B26D-78DCEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

Separate Intel/AMD-specific MCE code using CONFIG_{INTEL,AMD} config opti=
ons.
Now we can avoid build of mcheck code if support for specific platform is
intentionally disabled by configuration.

Also global variables lmce_support & cmci_support from Intel-specific
mce_intel.c have to moved to common mce.c, as they get checked in common =
code.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
changes in v4:
 - attribute {lmce_support,cmci_support} with __ro_after_init
changes in v3:
 - default return value of init_nonfatal_mce_checker() done in separate p=
atch
 - move lmce_support & cmci_support to common mce.c code
 - changed patch description
changes in v2:
 - fallback to original ordering in Makefile
 - redefine lmce_support & cmci_support global vars to false when !INTEL
 - changed patch description
---
 xen/arch/x86/cpu/mcheck/Makefile    | 8 ++++----
 xen/arch/x86/cpu/mcheck/mce.c       | 4 ++++
 xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ----
 xen/arch/x86/cpu/mcheck/non-fatal.c | 4 ++++
 4 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/Makefile b/xen/arch/x86/cpu/mcheck/M=
akefile
index f927f10b4d..e6cb4dd503 100644
--- a/xen/arch/x86/cpu/mcheck/Makefile
+++ b/xen/arch/x86/cpu/mcheck/Makefile
@@ -1,12 +1,12 @@
-obj-y +=3D amd_nonfatal.o
-obj-y +=3D mce_amd.o
+obj-$(CONFIG_AMD) +=3D amd_nonfatal.o
+obj-$(CONFIG_AMD) +=3D mce_amd.o
 obj-y +=3D mcaction.o
 obj-y +=3D barrier.o
-obj-y +=3D intel-nonfatal.o
+obj-$(CONFIG_INTEL) +=3D intel-nonfatal.o
 obj-y +=3D mctelem.o
 obj-y +=3D mce.o
 obj-y +=3D mce-apei.o
-obj-y +=3D mce_intel.o
+obj-$(CONFIG_INTEL) +=3D mce_intel.o
 obj-y +=3D non-fatal.o
 obj-y +=3D util.o
 obj-y +=3D vmce.o
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.=
c
index fb9dec5b89..1664ca6412 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -38,6 +38,10 @@ DEFINE_PER_CPU_READ_MOSTLY(unsigned int, nr_mce_banks)=
;
 unsigned int __read_mostly firstbank;
 unsigned int __read_mostly ppin_msr;
 uint8_t __read_mostly cmci_apic_vector;
+bool __ro_after_init cmci_support;
+
+/* If mce_force_broadcast =3D=3D 1, lmce_support will be disabled forcib=
ly. */
+bool __ro_after_init lmce_support;
=20
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, poll_bankmask);
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, no_cmci_banks);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mchec=
k/mce_intel.c
index af43281cc6..dd812f4b8a 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -26,16 +26,12 @@
 #include "mcaction.h"
=20
 static DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, mce_banks_owned);
-bool __read_mostly cmci_support;
 static bool __read_mostly ser_support;
 static bool __read_mostly mce_force_broadcast;
 boolean_param("mce_fb", mce_force_broadcast);
=20
 static int __read_mostly nr_intel_ext_msrs;
=20
-/* If mce_force_broadcast =3D=3D 1, lmce_support will be disabled forcib=
ly. */
-bool __read_mostly lmce_support;
-
 /* Intel SDM define bit15~bit0 of IA32_MCi_STATUS as the MC error code *=
/
 #define INTEL_MCCOD_MASK 0xFFFF
=20
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mchec=
k/non-fatal.c
index 5a53bcd0b7..a9ee9bb94f 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -24,15 +24,19 @@ static int __init cf_check init_nonfatal_mce_checker(=
void)
 	 * Check for non-fatal errors every MCE_RATE s
 	 */
 	switch (c->x86_vendor) {
+#ifdef CONFIG_AMD
 	case X86_VENDOR_AMD:
 	case X86_VENDOR_HYGON:
 		/* Assume we are on K8 or newer AMD or Hygon CPU here */
 		amd_nonfatal_mcheck_init(c);
 		break;
+#endif
=20
+#ifdef CONFIG_INTEL
 	case X86_VENDOR_INTEL:
 		intel_nonfatal_mcheck_init(c);
 		break;
+#endif
=20
 	default:
 		/* unhandled vendor isn't really an error */
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 09:29:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 09:29:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727360.1131845 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iHF-0002Ji-Tr; Wed, 22 May 2024 09:28:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727360.1131845; Wed, 22 May 2024 09:28:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iHF-0002Jb-Pc; Wed, 22 May 2024 09:28:53 +0000
Received: by outflank-mailman (input) for mailman id 727360;
 Wed, 22 May 2024 09:28:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ieL=MZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9iHE-0002JV-9S
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 09:28:52 +0000
Received: from mail-qv1-xf33.google.com (mail-qv1-xf33.google.com
 [2607:f8b0:4864:20::f33])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b271d8b1-181d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 11:28:50 +0200 (CEST)
Received: by mail-qv1-xf33.google.com with SMTP id
 6a1803df08f44-69b730fc89bso17701206d6.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 02:28:50 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f195018sm132971176d6.74.2024.05.22.02.28.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 02:28:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b271d8b1-181d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716370129; x=1716974929; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=gz1YsCbI1a4oUPtYa4QDaeqCIURr7ry/9yejUUOBEsk=;
        b=EyrpkNdCC+9S7WRtHiBc6Wr/HDQ611MGbK/MgrtVx9pTw59lZoR1HK9QdHUl0TcMBX
         w8t2ByA8GgCIO2Js4cv79IonBVvhXJlL6081I/gi7hNJN4EThhS29fHdMf4/xsAv52wW
         AdlFKyyI5+FENpdEjqKdk/qwK7wZDP/ubTzWU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716370129; x=1716974929;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gz1YsCbI1a4oUPtYa4QDaeqCIURr7ry/9yejUUOBEsk=;
        b=dgUN3kf6HmxkD9Y91IKpxVEheFBRABUq/6V08rZkoQ/WeKQiHhK2V4tSiHx+0MLQgO
         AjB3DU3PWwV2JnAP9QbdKOLIsd5FahChq0HwjrLEDI4U+0Bx9YD/vop0JDovp/d+mThE
         6aZ/QFxUlDkSG7/Hk1SJ260Ewgx9weLSR47QuvEqZWnhG8kzcg142pIzmmKWm4yrJ+qO
         KyAv32sAfSPehX/un+2xAhhAfn3I4bB50eJYlIMVCT4sFnqsY/v+OEavUEbc+cusA7/w
         Rqa+RujvMpM5y25rNUQy09zLwI9mo7y7dtNORzQrW2t1/9vJlthjvhJ5Xyb8NoCBMtks
         0T2w==
X-Gm-Message-State: AOJu0YwgALXyOrWy7iedihju4b/yHOIRk8c9krBdBtbv/ZPWAG02hln4
	Tun6OMvNJDtDM0Cs5GxhqEyp2KO70NVQ9RuN5FM9WnCVI95aNx7WPXEv6J5kplE=
X-Google-Smtp-Source: AGHT+IGrDQm9FAtR51aBACkT2Di0gxvalTT3yo3WfSsIvhZLPZoxUdssYMxqUSfHsMtaa0OKsRGiNw==
X-Received: by 2002:a05:6214:498f:b0:6ab:6f32:1a66 with SMTP id 6a1803df08f44-6ab7f361273mr19532236d6.26.1716370129124;
        Wed, 22 May 2024 02:28:49 -0700 (PDT)
Date: Wed, 22 May 2024 11:28:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v15 2/5] vpci/header: emulate PCI_COMMAND register for
 guests
Message-ID: <Zk26zphhCQ40rn2B@macbook>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
 <20240517170619.45088-3-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240517170619.45088-3-stewart.hildebrand@amd.com>

On Fri, May 17, 2024 at 01:06:12PM -0400, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
> to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
> guest's (domU) view of this will want to be zero (for now), the host
> having set it to 1 should be preserved, or else we'd effectively be
> giving the domU control of the bit. Thus, PCI_COMMAND register needs
> proper emulation in order to honor host's settings.
> 
> According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
> Device Control" the reset state of the command register is typically 0,
> so when assigning a PCI device use 0 as the initial state for the
> guest's (domU) view of the command register.
> 
> Here is the full list of command register bits with notes about
> PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
> also documented here since that is our current reference implementation
> for PCI passthrough.
> 
> PCI_COMMAND_IO (bit 0)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU sets this bit to 1 in
>     hardware if an I/O BAR is exposed to the guest.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
>     don't yet support I/O BARs for domUs.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_MEMORY (bit 1)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU sets this bit to 1 in
>     hardware if a Memory BAR is exposed to the guest.
>   Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
>     regions.
>   Xen domU: For devices assigned to DomUs, memory decoding will be
>     disabled at the time of initialization.
> 
> PCI_COMMAND_MASTER (bit 2)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_SPECIAL (bit 3)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_INVALIDATE (bit 4)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_VGA_PALETTE (bit 5)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: Pass through writes to hardware.
>   Xen domU/dom0: Pass through writes to hardware.
> 
> PCI_COMMAND_PARITY (bit 6)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_WAIT (bit 7)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: hardwire to 0
>   QEMU: res_mask
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_SERR (bit 8)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_FAST_BACK (bit 9)
>   PCIe 6.1: RO, hardwire to 0
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware.
>   Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> PCI_COMMAND_INTX_DISABLE (bit 10)
>   PCIe 6.1: RW
>   PCI LB 3.0: RW
>   QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
>     writes do not propagate to hardware. QEMU checks if INTx was mapped
>     for a device. If it is not, then guest can't control
>     PCI_COMMAND_INTX_DISABLE bit.
>   Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
>   Xen dom0: We allow dom0 to control this bit freely.
> 
> Bits 11-15
>   PCIe 6.1: RsvdP
>   PCI LB 3.0: Reserved
>   QEMU: res_mask
>   Xen domU/dom0: rsvdp_mask
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> RFC: There is an unaddressed question for Roger: should we update the
>      guest view of the PCI_COMMAND_INTX_DISABLE bit in
>      msi.c/msix.c:control_write()? See prior discussion at [1].
>      In my opinion, I think we should make sure that hardware state and
>      the guest view are consistent (i.e. don't lie to the guest).
> 
> [1] https://lore.kernel.org/xen-devel/86b25777-788c-4b9a-8166-a6f8174bedc9@suse.com/

I think updating the guest view is helpful in case we need to debug
issues in the guest.

> 
> In v15:
> - add Jan's R-b
> - add blank line after declaration in msi.c:control_write()
> 
> In v14:
> - check for 0->1 transition in INTX_DISABLE-setting logic in
>   msi.c:control_write() to match msix.c:control_write()
> - clear domU-controllable bits in header.c:init_header()
> 
> In v13:
> - Update right away (don't defer) PCI_COMMAND_MEMORY bit in guest_cmd
>   variable in cmd_write()
> - Make comment single line in xen/drivers/vpci/msi.c:control_write()
> - Rearrange memory decoding disabling snippet in init_header()
> 
> In v12:
> - Rework patch using vpci_add_register_mask()
> - Add bitmask #define in pci_regs.h according to PCIe 6.1 spec, except
>   don't add the RO bits because they were RW in PCI LB 3.0 spec.
> - Move and expand TODO comment about properly emulating bits
> - Update guest_cmd in msi.c/msix.c:control_write()
> - Simplify cmd_write(), thanks to rsvdp_mask
> - Update commit description
> 
> In v11:
> - Fix copy-paste mistake: vpci->msi should be vpci->msix
> - Handle PCI_COMMAND_IO
> - Fix condition for disabling INTx in the MSI-X code
> - Show domU changes to only allowed bits
> - Show PCI_COMMAND_MEMORY write only after P2M was altered
> - Update comments in the code
> In v10:
> - Added cf_check attribute to guest_cmd_read
> - Removed warning about non-zero cmd
> - Updated comment MSI code regarding disabling INTX
> - Used ternary operator in vpci_add_register() call
> - Disable memory decoding for DomUs in init_bars()
> In v9:
> - Reworked guest_cmd_read
> - Added handling for more bits
> Since v6:
> - fold guest's logic into cmd_write
> - implement cmd_read, so we can report emulated INTx state to guests
> - introduce header->guest_cmd to hold the emulated state of the
>   PCI_COMMAND register for guests
> Since v5:
> - add additional check for MSI-X enabled while altering INTX bit
> - make sure INTx disabled while guests enable MSI/MSI-X
> Since v3:
> - gate more code on CONFIG_HAS_MSI
> - removed logic for the case when MSI/MSI-X not enabled
> ---
>  xen/drivers/vpci/header.c  | 60 ++++++++++++++++++++++++++++++++++----
>  xen/drivers/vpci/msi.c     |  9 ++++++
>  xen/drivers/vpci/msix.c    |  7 +++++
>  xen/include/xen/pci_regs.h |  1 +
>  xen/include/xen/vpci.h     |  3 ++
>  5 files changed, 74 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index 47648c395132..2491dbae8901 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -524,9 +524,21 @@ static void cf_check cmd_write(
>  {
>      struct vpci_header *header = data;
>  
> +    if ( !is_hardware_domain(pdev->domain) )
> +    {
> +        const struct vpci *vpci = pdev->vpci;
> +
> +        if ( (vpci->msi && vpci->msi->enabled) ||
> +             (vpci->msix && vpci->msix->enabled) )
> +            cmd |= PCI_COMMAND_INTX_DISABLE;
> +
> +        header->guest_cmd = cmd;
> +    }
> +
>      /*
>       * Let Dom0 play with all the bits directly except for the memory
> -     * decoding one.
> +     * decoding one. Bits that are not allowed for DomU are already
> +     * handled above and by the rsvdp_mask.
>       */
>      if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) )
>          /*
> @@ -540,6 +552,14 @@ static void cf_check cmd_write(
>          pci_conf_write16(pdev->sbdf, reg, cmd);
>  }
>  
> +static uint32_t cf_check guest_cmd_read(
> +    const struct pci_dev *pdev, unsigned int reg, void *data)
> +{
> +    const struct vpci_header *header = data;
> +
> +    return header->guest_cmd;
> +}
> +
>  static void cf_check bar_write(
>      const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
>  {
> @@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
>          return -EOPNOTSUPP;
>      }
>  
> -    /* Setup a handler for the command register. */
> -    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
> -                           2, header);
> +    /*
> +     * Setup a handler for the command register.
> +     *
> +     * TODO: If support for emulated bits is added, re-visit how to handle
> +     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
> +     */
> +    rc = vpci_add_register_mask(pdev->vpci,
> +                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
> +                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
> +                                PCI_COMMAND_RSVDP_MASK |
> +                                    (is_hwdom ? 0
> +                                              : PCI_COMMAND_IO |
> +                                                PCI_COMMAND_PARITY |
> +                                                PCI_COMMAND_WAIT |
> +                                                PCI_COMMAND_SERR |
> +                                                PCI_COMMAND_FAST_BACK),

We want to allow full access to the hw domain and only apply the
PCI_COMMAND_RSVDP_MASK when !is_hwdom in order to keep the current
behavior for dom0.

I don't think it makes a difference in practice, but we are very lax
in explicitly not applying any of such restrictions to dom0.

With that fixed:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 22 09:33:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 09:33:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727366.1131854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iLe-0004Fs-HP; Wed, 22 May 2024 09:33:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727366.1131854; Wed, 22 May 2024 09:33:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iLe-0004Fl-Ei; Wed, 22 May 2024 09:33:26 +0000
Received: by outflank-mailman (input) for mailman id 727366;
 Wed, 22 May 2024 09:33:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9iLd-0004Ff-7B
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 09:33:25 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 552f732d-181e-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 11:33:23 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a621cb07d8fso99550766b.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 02:33:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1781cdb5sm1744216066b.29.2024.05.22.02.33.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 02:33:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 552f732d-181e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716370402; x=1716975202; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8AYXYZ50OpjNgv0sGHRw4nmKedbD2eMhNqcKvs7HDDg=;
        b=egUcPrjoY23e6OznqPNkkq1i3GPdnPzYwxJbaxFTsjDrpQfJwzhdUYcJKJveMvSIcH
         AV4mF9syslYNLZnuWx8/+XH8ON27vzNxz5YqvZ+sxo83MJFwkUArOm6R0B/1+HDvP/xa
         bfjVwPns2QaqexPb7lOKiqwn5j8uWm13KwJxzhk9ISta8AGFOU11ETbcD3TO56FDwEcG
         cNjx7MZGSKOF+Bq1QA2rEqKktPoipPQbRsthhyyBernXuy6DKrHEUkBpO2D+MAMw5asC
         f0xKiLGNfJidNRjenTb5mxje+UbQ/x2O6ewrUugjqBlzPGk5zg6ZXCFJuNoI4vC/FJbf
         AERw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716370402; x=1716975202;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8AYXYZ50OpjNgv0sGHRw4nmKedbD2eMhNqcKvs7HDDg=;
        b=Yelmq1Mgcy31HLJw8PTueh0LKanVwoxTJL5tofkITLhQ5z/L19iC1wWPto18iFdAHq
         rqccfCBH4lfS7O8aeiN2NLVUiVShQEGO9hzVP2s/hbQewQXQ7HKIS4+l/HrQC5/sp9LO
         s9GjO4jV28v39AXY7yFekdY5U2uS16ncvAQobxO6J2vHqVtbWbSdqwucW8af7FsSNd4v
         efvDR17BSvxkRpX4qNx0VbaVDnD3uJ7kSS/34i0LQGds8QKXXjv+TjIjGxhfdJjih9ar
         zhWyTgUWFf8Y2A5A0r8XylDrqCBpLlz+pAZoaTKSpiakBX59tSUSErXyXXgidDmRWx31
         oQ+w==
X-Forwarded-Encrypted: i=1; AJvYcCXR6cSHgdYBu+LoVR4VPtIP1B5h/EIDg64YaklAScbS0lm0Al0woj9l2ipSN6Pm0BbKmicqLANvaFUmoad6BRJY+qAfhll1mHvGguuebQs=
X-Gm-Message-State: AOJu0YwIT94XM95vVcieGKJTr2WJaBY7Q5+UJxpCrnvA8b+gNKsZLg4G
	IH9ypGT/Hc62gk25Z3K3Ce78bANbDnHFhdfYajbxNxv7CASxHaGirb4JlBbW4A==
X-Google-Smtp-Source: AGHT+IFNtCiKQv/ifn+lJiyDvHuCAERl6wrIquH5hTkisUcsOxkKe7P58e75n1lTulvkz9dHfDPhNQ==
X-Received: by 2002:a17:906:9c41:b0:a5a:84c8:7710 with SMTP id a640c23a62f3a-a62281253a4mr82541066b.55.1716370402468;
        Wed, 22 May 2024 02:33:22 -0700 (PDT)
Message-ID: <6a21a57f-e0fa-43af-a7ea-d813e4b2976d@suse.com>
Date: Wed, 22 May 2024 11:33:21 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
Content-Language: en-US
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.05.2024 14:39, Alejandro Vallejo wrote:
> Otherwise it's not possible to call functions described in hvm/vlapic.h from the
> inline functions of hvm/hvm.h.
> 
> This is because a static inline in vlapic.h depends on hvm.h, and pulls it
> transitively through vpt.h. The ultimate cause is having hvm.h included in any
> of the "v*.h" headers, so break the cycle moving the guilty inline into hvm.h.
> 
> No functional change.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

In principle:
Reviewed-by: Jan Beulich <jbeulich@suse.com>
But see below for one possible adjustment.

> ---
> v2:
>   * New patch. Prereq to moving vlapic_cpu_policy_changed() onto hvm.h

That hook invocation living outside of hvm/hvm.h was an outlier anyway,
so even without the planned further work this is probably a good move.

> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
>          alternative_vcall(hvm_funcs.update_vlapic_mode, v);
>  }
>  
> +static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
> +{
> +    if ( hvm_funcs.sync_pir_to_irr )
> +        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
> +}

The hook doesn't have "vlapic" in its name. Therefore instead or prepending
hvm_ to the original name or the wrapper, how about replacing the vlapic_
that was there. That would then also fit better with the naming scheme used
for other hooks and their wrappers. Happy to adjust while committing, so
long as you don't disagree.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 09:35:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 09:35:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727370.1131865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iNi-0004mo-Uu; Wed, 22 May 2024 09:35:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727370.1131865; Wed, 22 May 2024 09:35:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iNi-0004mh-Qi; Wed, 22 May 2024 09:35:34 +0000
Received: by outflank-mailman (input) for mailman id 727370;
 Wed, 22 May 2024 09:35:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9iNh-0004mb-RZ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 09:35:33 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a28c0b29-181e-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 11:35:32 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59b097b202so733000866b.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 02:35:32 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1789240asm1752346166b.48.2024.05.22.02.35.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 02:35:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a28c0b29-181e-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716370532; x=1716975332; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ImI3BBKzBbW+mD8DrnXzbVQSijdQ7O2nEuqn52AOb4Q=;
        b=RBYqQh230JLeQcyDCDmGcEiyeR0p3jozzBoVbpvKL4CodLEOItdPcmSD0Ze/oXFWEc
         B2NdJ4YkBuuWX5OOQv9QUqndDq44e7SSHGzLTFcmmRDyg4Yi35PnDXbTJc9u0BaRZ0X6
         36vtmm/D8z0onhjycEwpD6g1OPXP58Yg6Kz+fc23mIEvbPZtAJS5y9ETJNyiSI9lXgtO
         aD+eW7ITFI20NN5ZceXhoNYRTWy3z41DhKOOdEDzH0mbaV8bJF9XMNo8SKsDBom5iCoU
         jT2zFDVxI5QmN2QRF8G/Vqjd3b8YDgQcks8X34AZXgD3+3FxzJRORTNizCs+neb3fran
         C+Tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716370532; x=1716975332;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ImI3BBKzBbW+mD8DrnXzbVQSijdQ7O2nEuqn52AOb4Q=;
        b=agvLbBYFabHJUr9hDBdLHdiXzHBGxr3c+gRXPC3KQhFN1Vml+9Sha0unUJY6AiOqmR
         cflIO6Na2vTb+kpLDpSqyaXJdvL3uchz5JgyrcHIMpkwlu6ioBsb/3adv6jqSrwWSW1W
         mWht064n1iDq9b5l2cRK+JV/oPDSI771MgjHekC26pGgSOSqG/JbZbwurU0AlK1hIkkC
         lh5Zu0zhwPiniuj7BXuT1/Q+fDQyQoKTN3EQSHRRd0k67GiQ86M2ITh2+rRefQd/4bK5
         aA58k+kk4D6RhiDz6TsyKR+QK8KdO7arnlRbARG55u8a0AT7It+8bUMSlbxScJvPM8pC
         Deyw==
X-Gm-Message-State: AOJu0YzUxaItjL3nt64KZTvC2zZhaMyAc4bGp0LpyXOSorn1zDyZtJkk
	rjFBEPMHPIlNSWyI6SgBUWt6+SiT0PhL6Xdd6G+yNTSu2k8gxSuKSGec22B8/w==
X-Google-Smtp-Source: AGHT+IHvb74j6QekF+SATOi1Li7vssxxV2cE8EWP0kY1kUyaIOH4Zw+fxOkqE/s9hZPOSMx4VEz/xg==
X-Received: by 2002:a17:906:2b05:b0:a59:c28a:d350 with SMTP id a640c23a62f3a-a6228084310mr106564666b.24.1716370532290;
        Wed, 22 May 2024 02:35:32 -0700 (PDT)
Message-ID: <981ede8f-b9e0-4827-a0aa-8665e0254ed7@suse.com>
Date: Wed, 22 May 2024 11:35:31 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v15 2/5] vpci/header: emulate PCI_COMMAND register for
 guests
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
 <20240517170619.45088-3-stewart.hildebrand@amd.com>
 <Zk26zphhCQ40rn2B@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk26zphhCQ40rn2B@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 11:28, Roger Pau Monné wrote:
> On Fri, May 17, 2024 at 01:06:12PM -0400, Stewart Hildebrand wrote:
>> @@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
>>          return -EOPNOTSUPP;
>>      }
>>  
>> -    /* Setup a handler for the command register. */
>> -    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
>> -                           2, header);
>> +    /*
>> +     * Setup a handler for the command register.
>> +     *
>> +     * TODO: If support for emulated bits is added, re-visit how to handle
>> +     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
>> +     */
>> +    rc = vpci_add_register_mask(pdev->vpci,
>> +                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
>> +                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
>> +                                PCI_COMMAND_RSVDP_MASK |
>> +                                    (is_hwdom ? 0
>> +                                              : PCI_COMMAND_IO |
>> +                                                PCI_COMMAND_PARITY |
>> +                                                PCI_COMMAND_WAIT |
>> +                                                PCI_COMMAND_SERR |
>> +                                                PCI_COMMAND_FAST_BACK),
> 
> We want to allow full access to the hw domain and only apply the
> PCI_COMMAND_RSVDP_MASK when !is_hwdom in order to keep the current
> behavior for dom0.
> 
> I don't think it makes a difference in practice, but we are very lax
> in explicitly not applying any of such restrictions to dom0.
> 
> With that fixed:
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Makes sense to me, so please feel free to retain my R-b with that adjustment.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 09:54:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 09:54:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727381.1131874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9igG-00089I-BY; Wed, 22 May 2024 09:54:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727381.1131874; Wed, 22 May 2024 09:54:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9igG-00089B-8e; Wed, 22 May 2024 09:54:44 +0000
Received: by outflank-mailman (input) for mailman id 727381;
 Wed, 22 May 2024 09:54:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xn/7=MZ=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1s9igF-00088u-5Q
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 09:54:43 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4f03518a-1821-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 11:54:41 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-57825ff689bso1865197a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 02:54:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f03518a-1821-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716371680; x=1716976480; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=q8bHjw9cpzVbGFuolJD8PptND+oSuFhb4SNUO1OU8X4=;
        b=fFjJ6e46LG43dhi6vzSEXXnbCcG6BxF365ZCs3QwkT2wRqZS8gRBLhzmZrpBC2vhub
         mPnFM1d+QaAkkJAnyUI74y6S0rRdxjNx7D7jpSZT6SjYlVa1xcQmYCT6E35MDRDjvnWj
         tmrcNT4le4wupkE+UmvHXMcl1o7Vo+hGB4l1A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716371680; x=1716976480;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=q8bHjw9cpzVbGFuolJD8PptND+oSuFhb4SNUO1OU8X4=;
        b=t4Eq1KMWg5g0pU/7Rk4Jmt6g4fukSZsgY1jBaIP+swqiBN9TT5pab1MHHI5BSELNQ4
         10bhGBcO0mVv5gzwqGYTvEj0zY8NpLvZ4vqq89NLvqzKfhX2L8WUGfLxinuX/BcHNtMX
         fcVqFWHAdEdU6pCAEN0iC2gxQLqvrfOV381RYLnVT7N+hHJgDQZjYMqyxirjuyL7cQ68
         ijBu/LPY1xnKXrahr4lLhhFcfD7Xts4PEX9rUV9bbh4Xt83ucmP6uCw3vqEAhRq87rOp
         Sfeix7vpqY05lbLko/jnuEpuidgY3HaNtrORhd8R+oPbT9d6rl3/4vm9wE1g/Iv36edn
         oX7A==
X-Gm-Message-State: AOJu0YwCU36YOZmBGSssobMzSyfRFTJBB4zpDgFC5q7k4QDCp5/Fo7qs
	eXorYGD84g2slreYzeDYCQuQxQX3GI9I1XcsFyBgbv4gRbwZ/SfBxTwSa1dW7eF0WrXj6MAQ1uC
	XlnPqMkrcokE6UhWnm6BdbX6/oZqfhQXYyrRN2DXEMN0x23b+exALY18O
X-Google-Smtp-Source: AGHT+IF7GIkd9MfU4UpemzoVhlxsdnDKAYE6i3kug+g5d5UV9PjwPb83QN3Wm8wWgaadY0s6+kuLnZel+OCQ/yZ+c7g=
X-Received: by 2002:a50:d74e:0:b0:56e:99e:1fac with SMTP id
 4fb4d7f45d1cf-57832c4b0a0mr949525a12.39.1716371680219; Wed, 22 May 2024
 02:54:40 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 22 May 2024 10:54:04 +0100
Message-ID: <CAO-mL=yTtZeRuOMHXitaHLi184ghXQqJqRfSMgFsH7mrTKiR-Q@mail.gmail.com>
Subject: Virtual Xen Summit 2024
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="0000000000004d78bd061907ec6a"

--0000000000004d78bd061907ec6a
Content-Type: text/plain; charset="UTF-8"

Hi all,

To make sure we encourage as many people to attend Xen Summit 2024
<https://events.linuxfoundation.org/xen-project-summit/>, the community
will be helping to host the event virtually via Jitsi. This will be free to
join!

I will send the links and instructions to join our event, early next week.
Keep your eyes peeled on the mailing list for the announcement.

Please note there will be no professional AV or video equipment and this is
an effort to encourage participation.

As a reminder, virtual attendees can still get involved in design sessions,
so please register and propose a session today!
https://design-sessions.xenproject.org/

Many thanks,
Kelly Choi

Community Manager
Xen Project

--0000000000004d78bd061907ec6a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,<div><br></div><div>To make sure we encourage as ma=
ny people to attend=C2=A0<a href=3D"https://events.linuxfoundation.org/xen-=
project-summit/">Xen Summit 2024</a>, the community will be helping to=C2=
=A0host the event virtually via Jitsi. This will be free to join!</div><div=
><br></div><div>I will send the links and instructions to join our event, e=
arly next week. Keep your eyes peeled on the mailing list for the announcem=
ent.=C2=A0</div><div><br></div><div>Please note there will be no profession=
al AV or video equipment and this is an effort to encourage participation.=
=C2=A0</div><div><br></div><div>As a reminder, virtual attendees can still =
get involved in design sessions, so please register and propose a session t=
oday! <a href=3D"https://design-sessions.xenproject.org/">https://design-se=
ssions.xenproject.org/</a>=C2=A0</div><div><br></div><div><div><div dir=3D"=
ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=
=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></div><div><d=
iv style=3D"color:rgb(136,136,136)">Community Manager</div><div style=3D"co=
lor:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></div></div></div></=
div></div>

--0000000000004d78bd061907ec6a--


From xen-devel-bounces@lists.xenproject.org Wed May 22 10:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 10:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727443.1131912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iqX-0003ho-1x; Wed, 22 May 2024 10:05:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727443.1131912; Wed, 22 May 2024 10:05:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9iqW-0003hh-VN; Wed, 22 May 2024 10:05:20 +0000
Received: by outflank-mailman (input) for mailman id 727443;
 Wed, 22 May 2024 10:05:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9iqW-0003hb-6A
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 10:05:20 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cb53c015-1822-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 12:05:19 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-52742fdd363so242618e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 03:05:19 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c814dsm1745026166b.118.2024.05.22.03.05.18
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 03:05:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb53c015-1822-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716372319; x=1716977119; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=44FK3+ZDN+NKpxUSF4D8wmATvQF1ajJL4qiJab6qpH0=;
        b=EcFk24fZeV6fLD0bzSlQ7ASh+ElhDbC7lX+5DIK7AYFWSTx3GFtUGP1VlVM7tm6lem
         LHqPFuiHgUQhhEHs78gXs4iirPV1U5m0PKX8t5A1IF+opO/W3A7eEWtwafCVcyb6P5uf
         HrUWSQs6I7htV4cvT3Fk+j9pD8UvmxJh+Kqp5JcM3CudgBPKPUbOY7hAvapR2V/GjPVg
         xaxEviCKKPYVGdmuEL80Bd8WdqGdg9z+kSmRXXJqHKKEKNPRyDs9fC2+WW+g9m2kqLob
         W70FzPUKs0K1Sb0fIdplJ9cn1pRHzgKsfL/+Vnh+0CHwV8x4wmnPbUYu69LG4VSq6656
         Nmhw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716372319; x=1716977119;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=44FK3+ZDN+NKpxUSF4D8wmATvQF1ajJL4qiJab6qpH0=;
        b=XboyjCVir5HTcxPL8LQTs9tiV+pedwxn6dMhpZFpfiddfRTsMCpLD0vVCyI1LJvjMT
         lPIR84CcRT98r+q9u+xgLfJZjQyMyYVYAxQrAkA+AfoAI+qWGzYADdu1FF30vDncEXMf
         NLU5GDARzMSkOOnsykxt3IEFnY4RXwO8R+MwHk821wsFA/ceQYHSYBKAPFQ3yiO2N6dw
         2HkS3euGFDrEFpVM23McCuZKwi2J8PiJ+RcLR2G2s5qc1Q5WQlfpL7kHJ2zjyaxUYghQ
         Y+XM02e0LWpX3IFZc8PwxU2NgdcvnFrFl6TCcS/qHHMmRpQULOavmfMqM5iVA9OOiRtv
         Mx/w==
X-Gm-Message-State: AOJu0Yz3n+rI2m7HrPlL0XycNjnws/S41klreWIJCycmAZaGcc2DAQcL
	vtNjuyGa+xvSS7o/LwacC9nlJeJLAhEGl3GUt4ultgJ/9KaMo7u0pXhVYRSC019agvZkSRh4sAY
	=
X-Google-Smtp-Source: AGHT+IEe/OjtgVdqneqvwLczqW7tRhjXM/vYH+P+kj/aHzrRtWv1QXljEVsA2UeFeh757XpXWyZiLA==
X-Received: by 2002:a19:4311:0:b0:51c:1d6b:5171 with SMTP id 2adb3069b0e04-526bfc01ae0mr828472e87.56.1716372318609;
        Wed, 22 May 2024 03:05:18 -0700 (PDT)
Message-ID: <1b607d3b-429e-4841-a3b0-ed3a39e12ed7@suse.com>
Date: Wed, 22 May 2024 12:05:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: New Defects reported by Coverity Scan for XenProject
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <664dc165759df_5e9362b92d249399c762@prd-scan-dashboard-0.mail>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <664dc165759df_5e9362b92d249399c762@prd-scan-dashboard-0.mail>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 11:56, scan-admin@coverity.com wrote:
> ** CID 1598431:  Memory - corruptions  (OVERRUN)
> 
> 
> ________________________________________________________________________________________________________
> *** CID 1598431:  Memory - corruptions  (OVERRUN)
> /xen/common/trace.c: 798 in trace()
> 792         }
> 793     
> 794         if ( rec_size > bytes_to_wrap )
> 795             insert_wrap_record(buf, rec_size);
> 796     
> 797         /* Write the original record */
>>>>     CID 1598431:  Memory - corruptions  (OVERRUN)
>>>>     Overrunning callee's array of size 28 by passing argument "extra" (which evaluates to 31) in call to "__insert_record".
> 798         __insert_record(buf, event, extra, cycles, rec_size, extra_data);
> 799     
> 800     unlock:
> 801         spin_unlock_irqrestore(&this_cpu(t_lock), flags);
> 802     
> 803         /* Notify trace buffer consumer that we've crossed the high water mark. */

How does the tool conclude "extra" evaluating to 31, when at the top of
the function it is clearly checked to be less than 28?

> ** CID 1598430:  Uninitialized variables  (UNINIT)
> 
> 
> ________________________________________________________________________________________________________
> *** CID 1598430:  Uninitialized variables  (UNINIT)
> /xen/arch/x86/mm/shadow/multi.c: 2109 in trace_shadow_emulate()
> 2103             d.va = va;
> 2104     #if GUEST_PAGING_LEVELS == 3
> 2105             d.emulation_count = this_cpu(trace_extra_emulation_count);
> 2106     #endif
> 2107             d.flags = this_cpu(trace_shadow_path_flags);
> 2108     
>>>>     CID 1598430:  Uninitialized variables  (UNINIT)
>>>>     Using uninitialized value "d". Field "d.emulation_count" is uninitialized when calling "trace".
> 2109             trace(event, sizeof(d), &d);
> 2110         }
> 2111     }
> 2112     #endif /* CONFIG_HVM */
> 2113     
> 2114     /**************************************************************************/

This, otoh, looks to be a valid (but long-standing) issue, which I'll make
a patch for.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 10:17:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 10:17:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727457.1131922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9j2M-00061g-5K; Wed, 22 May 2024 10:17:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727457.1131922; Wed, 22 May 2024 10:17:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9j2M-00061Z-2o; Wed, 22 May 2024 10:17:34 +0000
Received: by outflank-mailman (input) for mailman id 727457;
 Wed, 22 May 2024 10:17:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9j2K-00061T-Rp
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 10:17:32 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8002fb06-1824-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 12:17:31 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-56e69888a36so10667124a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 03:17:31 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b017a5sm1758048466b.180.2024.05.22.03.17.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 03:17:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8002fb06-1824-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716373051; x=1716977851; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=LsEGTnCgHx262Iq6DzYer+wCgYB5sQXE9o6NLnCsuBo=;
        b=RDlfUaW8o6s9axQS1/E7R4ThBvm4CyvFuZrjpYyejdlLH0qf2UEeVqjGBj7+MSf6TV
         oxxSAGKtvIMHrWsO/oS9x1DJyt1wVHT1UP9rVbXWT/ShnhlhAlh4zwXg/7MHgCjlRaZN
         diPzNACPus2yFflav3XRBjQgJEAaV7yoMseuKQCrIekd6TQqyUDwJ9o9blQpFdzoOntv
         MW2CWp8Nj2rehQBa8MAl5W8y2hCUKlQ8RKZlB+KG8rSh3wGvTlvyqnHDF7F4HQxL/Jb7
         aVz6BAB4NR4+NhB6JZ8N1u1t2j4nyyXeniqqMDSEW3KIszZEQi7c7EtN2HOsRv2WQZ7u
         VTnw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716373051; x=1716977851;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LsEGTnCgHx262Iq6DzYer+wCgYB5sQXE9o6NLnCsuBo=;
        b=Z4wl0QtjtXi+bDc51NHc0MsqS+sQU1pLL+DSSz2ZR+ZXBvuznYSBxOnrOVNvadbkXw
         bhDuLV29FmVzPyyu0GtQlae27yvHlU0/YFJtdHlafTB4knePV9tKrfIKFDFY+TYETyan
         6bvGIFWRLFEyeQB57zvXfyrlaPQTCfmpDdec51irVVq7MOg4Np9Y10KUL/0+ADqmo8/2
         iP9uRh5VZED3kE5PMyRnJ9nHE7wuW2iMv5FopzjMz2x3e6pF6DUMByLgJB+UwLUQRYuj
         s0qdr08fmiHBwbE99yMFv/ZKuySGKTDuQL+f59qlLKewTL3WbBRqJoZoRhnTAZmCLz5M
         W9UA==
X-Gm-Message-State: AOJu0Yy2jf0cLLFcIPnL2MuuPUlSaxFOw/yAkYQzopvSVuZsVgai2ET5
	HXQaOUVQvcQb5IlH20/+6td+TUEOHY8CHbqIL56KaWTg+sv6nk6P3yShP4Zn1d53U6GsafYDvXk
	=
X-Google-Smtp-Source: AGHT+IEerRNk/qnKg7CM/QXot6V5imyKJxpO6/hp2cu+eLXdmcVWzjdtokqAlPIXo++nXmDoIUabzA==
X-Received: by 2002:a17:906:16c9:b0:a59:b136:e40f with SMTP id a640c23a62f3a-a62280b6fb3mr97414266b.38.1716373051238;
        Wed, 22 May 2024 03:17:31 -0700 (PDT)
Message-ID: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
Date: Wed, 22 May 2024 12:17:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tim Deegan <tim@xen.org>, George Dunlap <george.dunlap@cloud.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/shadow: don't leave trace record field uninitialized
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The emulation_count field is set only conditionally right now. Convert
all field setting to an initializer, thus guaranteeing that field to be
set to 0 (default initialized) when GUEST_PAGING_LEVELS != 3.

While there also drop the "event" local variable, thus eliminating an
instance of the being phased out u32 type.

Coverity ID: 1598430
Fixes: 9a86ac1aa3d2 ("xentrace 5/7: Additional tracing for the shadow code")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2093,20 +2093,18 @@ static inline void trace_shadow_emulate(
             guest_l1e_t gl1e, write_val;
             guest_va_t va;
             uint32_t flags:29, emulation_count:3;
-        } d;
-        u32 event;
-
-        event = TRC_SHADOW_EMULATE | ((GUEST_PAGING_LEVELS-2)<<8);
-
-        d.gl1e = gl1e;
-        d.write_val.l1 = this_cpu(trace_emulate_write_val);
-        d.va = va;
+        } d = {
+            .gl1e = gl1e,
+            .write_val.l1 = this_cpu(trace_emulate_write_val),
+            .va = va,
 #if GUEST_PAGING_LEVELS == 3
-        d.emulation_count = this_cpu(trace_extra_emulation_count);
+            .emulation_count = this_cpu(trace_extra_emulation_count),
 #endif
-        d.flags = this_cpu(trace_shadow_path_flags);
+            .flags = this_cpu(trace_shadow_path_flags),
+        };
 
-        trace(event, sizeof(d), &d);
+        trace(TRC_SHADOW_EMULATE | ((GUEST_PAGING_LEVELS - 2) << 8),
+              sizeof(d), &d);
     }
 }
 #endif /* CONFIG_HVM */


From xen-devel-bounces@lists.xenproject.org Wed May 22 10:34:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 10:34:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727474.1131938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jIG-0001M9-KB; Wed, 22 May 2024 10:34:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727474.1131938; Wed, 22 May 2024 10:34:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jIG-0001M2-HX; Wed, 22 May 2024 10:34:00 +0000
Received: by outflank-mailman (input) for mailman id 727474;
 Wed, 22 May 2024 10:33:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ieL=MZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9jIE-0001Lt-PQ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 10:33:58 +0000
Received: from mail-vk1-xa2e.google.com (mail-vk1-xa2e.google.com
 [2607:f8b0:4864:20::a2e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ca9f1ca2-1826-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 12:33:56 +0200 (CEST)
Received: by mail-vk1-xa2e.google.com with SMTP id
 71dfb90a1353d-4df3f7b93a7so238994e0c.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 03:33:56 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792dc0a5bb2sm1063747185a.42.2024.05.22.03.33.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 03:33:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca9f1ca2-1826-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716374035; x=1716978835; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=IgajI+gTSooT6QyaQjrxAm9fg0R7qgPUodTv4dLfUDk=;
        b=kvYYHXR4xP9VSlkk7y6ee032DTv4aDpmwZS2sBV8MwUyv/YF4wmTvEZO6a685Jpc20
         0rrvc4hDebG3qCcoFIwXgyJUukxYGPPhTvk06O7db9eFP2nnhfB6FW+KdoFqt5Fr9FDn
         SXgGzN8TaQmBADBwIjnYhplNyh4t7o0QfaJlg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716374035; x=1716978835;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=IgajI+gTSooT6QyaQjrxAm9fg0R7qgPUodTv4dLfUDk=;
        b=Y/0KD8W4S0ICKolO/vo+CKeVaf6pggguIA6hHpEYaqd4gqn3iOS1DL+U/rhpZqD+IZ
         lFvzsNi5r6jtqHgtJQt8z12EWQJK3fbsyEZk3nVu9lg9o3KdBXr+BXDBNbfoID5IvPob
         ZBdZhfX5q9avqyDOkrvYjoVRPnL/4yi6jy6hBfbvIIGtdFj2u3wtpZpPMPVc5mgAXyx8
         zloDOmyWH5mAuimHgAo0DczxcFwXY2A0Dm4SWCvJxmTvzt+Oh0GTh2Kl6hcREqlwpIqR
         pImK6isTxk0iBcClz35uPNVANHNEtYOzztrn7RWljhbwe+GYAAU+vVE8x9s8I+sGQIHh
         TjDg==
X-Gm-Message-State: AOJu0YzJcFRHlFXLmaCNRDm3lwe/kZbEUsRxGhsvo6PKRyoM7oqOjMhV
	w55Sw4x5+0yX40lhew2P4XKJ4E2jARPXo6cCuLBh7c7C7o/OrfZPcX3Bs1MFABA=
X-Google-Smtp-Source: AGHT+IGIJx7Q5C9fIuBMt7z5SOvQVv+nyedzlS3WpPdjf5OU5LMSZxPQO/F1ApjtlbUIRwld87XQJw==
X-Received: by 2002:a05:6122:31a8:b0:4d8:7222:b6da with SMTP id 71dfb90a1353d-4e2184ed83amr1520531e0c.6.1716374035190;
        Wed, 22 May 2024 03:33:55 -0700 (PDT)
Date: Wed, 22 May 2024 12:33:52 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v15 3/5] vpci: add initial support for virtual PCI bus
 topology
Message-ID: <Zk3KEKSb5ZsDhFBR@macbook>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
 <20240517170619.45088-4-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240517170619.45088-4-stewart.hildebrand@amd.com>

On Fri, May 17, 2024 at 01:06:13PM -0400, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Assign SBDF to the PCI devices being passed through with bus 0.
> The resulting topology is where PCIe devices reside on the bus 0 of the
> root complex itself (embedded endpoints).
> This implementation is limited to 32 devices which are allowed on
> a single PCI bus.
> 
> Please note, that at the moment only function 0 of a multifunction
> device can be passed through.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> In v15:
> - add Jan's A-b
> In v13:
> - s/depends on/select/ in Kconfig
> - check pdev->sbdf.fn instead of two booleans in add_virtual_device()
> - comment #endifs in sched.h
> - clarify comment about limits in vpci.h with seg/bus limit
> In v11:
> - Fixed code formatting
> - Removed bogus write_unlock() call
> - Fixed type for new_dev_number
> In v10:
> - Removed ASSERT(pcidevs_locked())
> - Removed redundant code (local sbdf variable, clearing sbdf during
> device removal, etc)
> - Added __maybe_unused attribute to "out:" label
> - Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the
>   first patch where it is used (previously was in "vpci: add hooks for
>   PCI device assign/de-assign")
> In v9:
> - Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart)
> In v8:
> - Added write lock in add_virtual_device
> Since v6:
> - re-work wrt new locking scheme
> - OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device()
> Since v5:
> - s/vpci_add_virtual_device/add_virtual_device and make it static
> - call add_virtual_device from vpci_assign_device and do not use
>   REGISTER_VPCI_INIT machinery
> - add pcidevs_locked ASSERT
> - use DECLARE_BITMAP for vpci_dev_assigned_map
> Since v4:
> - moved and re-worked guest sbdf initializers
> - s/set_bit/__set_bit
> - s/clear_bit/__clear_bit
> - minor comment fix s/Virtual/Guest/
> - added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used
>   later for counting the number of MMIO handlers required for a guest
>   (Julien)
> Since v3:
>  - make use of VPCI_INIT
>  - moved all new code to vpci.c which belongs to it
>  - changed open-coded 31 to PCI_SLOT(~0)
>  - added comments and code to reject multifunction devices with
>    functions other than 0
>  - updated comment about vpci_dev_next and made it unsigned int
>  - implement roll back in case of error while assigning/deassigning devices
>  - s/dom%pd/%pd
> Since v2:
>  - remove casts that are (a) malformed and (b) unnecessary
>  - add new line for better readability
>  - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
>     functions are now completely gated with this config
>  - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
> New in v2
> ---
>  xen/drivers/Kconfig     |  4 +++
>  xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++
>  xen/include/xen/sched.h | 10 +++++++-
>  xen/include/xen/vpci.h  | 12 +++++++++
>  4 files changed, 82 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
> index db94393f47a6..20050e9bb8b3 100644
> --- a/xen/drivers/Kconfig
> +++ b/xen/drivers/Kconfig
> @@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
>  config HAS_VPCI
>  	bool
>  
> +config HAS_VPCI_GUEST_SUPPORT
> +	bool
> +	select HAS_VPCI
> +
>  endmenu
> diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
> index 97e115dc5798..23722634d50b 100644
> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
>  extern vpci_register_init_t *const __end_vpci_array[];
>  #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
>  
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +static int add_virtual_device(struct pci_dev *pdev)

This seems quite generic, IMO it would better named
`assign_{guest,virtual}_sbdf()` or similar, unless there are plans to
add more code here that's not strictly only about setting the guest
SBDF.

> +{
> +    struct domain *d = pdev->domain;
> +    unsigned int new_dev_number;
> +
> +    if ( is_hardware_domain(d) )
> +        return 0;
> +
> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));

Shouldn't the assert be done before the is_hardware_domain() check, so
that we assert that all possible paths (even those from dom0) have
taken the correct lock?

> +
> +    /*
> +     * Each PCI bus supports 32 devices/slots at max or up to 256 when
> +     * there are multi-function ones which are not yet supported.
> +     */
> +    if ( pdev->sbdf.fn )
> +    {
> +        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
> +                 &pdev->sbdf);
> +        return -EOPNOTSUPP;
> +    }
> +    new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map,
> +                                         VPCI_MAX_VIRT_DEV);
> +    if ( new_dev_number == VPCI_MAX_VIRT_DEV )
> +        return -ENOSPC;
> +
> +    __set_bit(new_dev_number, &d->vpci_dev_assigned_map);
> +
> +    /*
> +     * Both segment and bus number are 0:
> +     *  - we emulate a single host bridge for the guest, e.g. segment 0
> +     *  - with bus 0 the virtual devices are seen as embedded
> +     *    endpoints behind the root complex
> +     *
> +     * TODO: add support for multi-function devices.
> +     */
> +    pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0);
> +
> +    return 0;
> +}
> +
> +#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
> +
>  void vpci_deassign_device(struct pci_dev *pdev)
>  {
>      unsigned int i;
> @@ -49,6 +92,12 @@ void vpci_deassign_device(struct pci_dev *pdev)
>      if ( !has_vpci(pdev->domain) || !pdev->vpci )
>          return;
>  
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +    if ( pdev->vpci->guest_sbdf.sbdf != ~0 )
> +        __clear_bit(pdev->vpci->guest_sbdf.dev,
> +                    &pdev->domain->vpci_dev_assigned_map);
> +#endif
> +
>      spin_lock(&pdev->vpci->lock);
>      while ( !list_empty(&pdev->vpci->handlers) )
>      {
> @@ -103,6 +152,13 @@ int vpci_assign_device(struct pci_dev *pdev)
>      INIT_LIST_HEAD(&pdev->vpci->handlers);
>      spin_lock_init(&pdev->vpci->lock);
>  
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +    pdev->vpci->guest_sbdf.sbdf = ~0;

I think ~0 wants to be in a define here:

#define INVALID_GUEST_SBDF ~0

Or similar.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 22 10:37:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 10:37:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727478.1131948 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jLC-0001tq-1f; Wed, 22 May 2024 10:37:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727478.1131948; Wed, 22 May 2024 10:37:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jLB-0001tj-Ur; Wed, 22 May 2024 10:37:01 +0000
Received: by outflank-mailman (input) for mailman id 727478;
 Wed, 22 May 2024 10:37:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9jLA-0001tb-CF
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 10:37:00 +0000
Received: from wfhigh5-smtp.messagingengine.com
 (wfhigh5-smtp.messagingengine.com [64.147.123.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35ead437-1827-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 12:36:57 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfhigh.west.internal (Postfix) with ESMTP id 797021800152;
 Wed, 22 May 2024 06:36:54 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Wed, 22 May 2024 06:36:54 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 06:36:52 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35ead437-1827-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716374214;
	 x=1716460614; bh=GjZWerAElwAk/mgSGVDAYJhDseHE+UnF9rG9aG0r/tU=; b=
	vVOQ37cqz/PByJF6x22DcaiHfVJ2DzR9HA6i7IqT9KPL/yGikdYd3EsmPcBDVxVb
	rhXb3xtAHdiprGmpv5MXinRc+I+fn8XvxVPn4A9MZLCy30Moj1ZGVA0vEPyoP49M
	0BNz4UbzNF/xKnlrLBcc6uKGhK931DFenP1QeSFiI2JpudJ9m2setEoeuZ4Jgea8
	BfRWtz412960EclOdNh0pbvaxy7x0ErOj5MhIe+bBc8Vs2fE8flxFbxZxwXDS04j
	Pkl+JPeuCeVdSq8KYv+d7jHZyhsKlzWGwAfB3OQy0YRGZDHDrAvgP4mUo/cK+VxS
	XbVDrhqo7I6uqKIu3FtG0g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716374214; x=1716460614; bh=GjZWerAElwAk/mgSGVDAYJhDseHE
	+UnF9rG9aG0r/tU=; b=itNbxUhgYHt0J9NUHQXH4MMNykaV1lEAGaYnSrTbBnVV
	r7S8yFL11pfXf/+jCDBqv1kCDfD0vnHAkvZlLMg/+KhQ4yXo4FxbPlQf0X1rqCxD
	oJA/sg6zd2wOrbRcxcjZ+KVvQtoPmMWICJIR6t9m/9Gz3uo06GtXk1zJgEyXkUvU
	DNxd2+wPoUmYUg7GBzs2xmiwQ5ymxPd/DhFjGVtyp3uhi2VeSegMMq1VmtA1iUst
	jC1JmfbQfHbPCXDDlip7H8A9pwc8fEQycjIaDqNwfGm4yz256eVbsV9dnEIwHx8W
	NsEzVjsAWpt7aM2q17+g6BwqR7QGxfiN0os/vNaTrw==
X-ME-Sender: <xms:xcpNZjY2bdfxLZTweqGNup3zqKHvtm0P6K_qSlKo7E-6tSk6iz9U4w>
    <xme:xcpNZia18u2LV4CG9s9z20pxOyBVNPOWKedcFRcTVUS3x3tsm-3KzNdOBIKCx4MyH
    U0aGFsXLtSZLg>
X-ME-Received: <xmr:xcpNZl-df0mrDpxknMrbz-P60-DSDO9JOWMMsbhRzw6YS_94Vqk6g8Heb1L4mMUsKMbOu0JkJVhvz1IgqUXebjRJwUem8dAuNg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgtdduucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:xcpNZprHMK05BIIiNU4f44x6tzC2loYUQA7_oZW7Rav54ifR7HqKjQ>
    <xmx:xcpNZuocVVs6n_tOckJNRmVmkmrv_TkRMMe_MMhJ3AEpsJvuh6lshw>
    <xmx:xcpNZvQn1m-j_cZUilpWIE9WOBrK2KsoTyceQgcSgjZociPXThRzDQ>
    <xmx:xcpNZmrMoeUQcDkmUl53gN_EjahZvJ-dkb0rJn6ymmm6mNTF_5-LCQ>
    <xmx:xspNZhUYWKVKB0nkYWZeRL8ePMtHSJ6NPgQbAbRx3aZwmxmSZ7eV8zUZ>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 22 May 2024 12:36:51 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Message-ID: <Zk3Kw2WX8r8qpjQj@mail-itl>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="BEvgsup/CxFtuvLg"
Content-Disposition: inline
In-Reply-To: <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>


--BEvgsup/CxFtuvLg
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 22 May 2024 12:36:51 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only

On Wed, May 22, 2024 at 09:52:44AM +0200, Jan Beulich wrote:
> On 21.05.2024 04:54, Marek Marczykowski-G=C3=B3recki wrote:
> > --- a/xen/arch/x86/hvm/hvm.c
> > +++ b/xen/arch/x86/hvm/hvm.c
> > @@ -2009,6 +2009,14 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsig=
ned long gla,
> >          goto out_put_gfn;
> >      }
> > =20
> > +    if ( (p2mt =3D=3D p2m_mmio_direct) && npfec.write_access && npfec.=
present &&
> > +         subpage_mmio_write_accept(mfn, gla) &&
>=20
> Afaics subpage_mmio_write_accept() is unreachable then when CONFIG_HVM=3D=
n?

Right, the PV path hits mmio_ro_emulated_write() without my changes
already.
Do you suggest to make subpage_mmio_write_accept() under #ifdef
CONFIG_HVM?

> > +         (hvm_emulate_one_mmio(mfn_x(mfn), gla) =3D=3D X86EMUL_OKAY) )
> > +    {
> > +        rc =3D 1;
> > +        goto out_put_gfn;
> > +    }
>=20
> Overall this new if() is pretty similar to the immediate preceding one.
> So similar that I wonder whether the two shouldn't be folded.=20

I can do that if you prefer.

> In fact
> it looks as if the new one is needed only for the case where you'd pass
> through (to a DomU) a device partially used by Xen. That could certainly
> do with mentioning explicitly.

Well, the change in mmio_ro_emulated_write() is relevant to both dom0
and domU. It simply wasn't reachable (in this case) for HVM domU before
(but was for PV already).

> > +static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *en=
try)
>=20
> Considering what the function does and what it returns, perhaps better
> s/get/map/? The "get_page" part of the name generally has a different
> meaning in Xen's memory management.

Ok.

> > +{
> > +    void __iomem *mapped_page;
> > +
> > +    if ( entry->mapped )
> > +        return entry->mapped;
> > +
> > +    mapped_page =3D ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE=
);
> > +
> > +    spin_lock(&subpage_ro_lock);
> > +    /* Re-check under the lock */
> > +    if ( entry->mapped )
> > +    {
> > +        spin_unlock(&subpage_ro_lock);
> > +        iounmap(mapped_page);
>=20
> The only unmap is on an error path here and on another error path elsewhe=
re.
> IOW it looks as if devices with such marked pages are meant to never be h=
ot
> unplugged. I can see that being intentional for the XHCI console, but imo
> such a restriction also needs prominently calling out in a comment next to
> e.g. the function declaration.

The v1 included subpage_mmio_ro_remove() function (which would need to
be used in case of hot-unplug of such device, if desirable), but since
this series doesn't introduce any use of it (as you say, it isn't
desirable for XHCI console specifically), you asked me to remove it...

Should I add an explicit comment about the limitation, instead of having
it implicit by not having subpage_mmio_ro_remove() there?

> > +        return entry->mapped;
> > +    }
> > +
> > +    entry->mapped =3D mapped_page;
> > +    spin_unlock(&subpage_ro_lock);
> > +    return entry->mapped;
> > +}
> > +
> > +static void subpage_mmio_write_emulate(
> > +    mfn_t mfn,
> > +    unsigned int offset,
> > +    const void *data,
> > +    unsigned int len)
> > +{
> > +    struct subpage_ro_range *entry;
> > +    void __iomem *addr;
>=20
> Wouldn't this better be pointer-to-volatile, with ...
>=20
> > +    list_for_each_entry(entry, &subpage_ro_ranges, list)
> > +    {
> > +        if ( mfn_eq(entry->mfn, mfn) )
> > +        {
> > +            if ( test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qw=
ords) )
> > +            {
> > + write_ignored:
> > +                gprintk(XENLOG_WARNING,
> > +                        "ignoring write to R/O MMIO 0x%"PRI_mfn"%03x l=
en %u\n",
> > +                        mfn_x(mfn), offset, len);
> > +                return;
> > +            }
> > +
> > +            addr =3D subpage_mmio_get_page(entry);
> > +            if ( !addr )
> > +            {
> > +                gprintk(XENLOG_ERR,
> > +                        "Failed to map page for MMIO write at 0x%"PRI_=
mfn"%03x\n",
> > +                        mfn_x(mfn), offset);
> > +                return;
> > +            }
> > +
> > +            switch ( len )
> > +            {
> > +            case 1:
> > +                writeb(*(const uint8_t*)data, addr);
> > +                break;
> > +            case 2:
> > +                writew(*(const uint16_t*)data, addr);
> > +                break;
> > +            case 4:
> > +                writel(*(const uint32_t*)data, addr);
> > +                break;
> > +            case 8:
> > +                writeq(*(const uint64_t*)data, addr);
> > +                break;
>=20
> ... this being how it's written? (If so, volatile suitably carried throug=
h to
> other places as well.)
>=20
> > +            default:
> > +                /* mmio_ro_emulated_write() already validated the size=
 */
> > +                ASSERT_UNREACHABLE();
> > +                goto write_ignored;
> > +            }
> > +            return;
> > +        }
> > +    }
> > +    /* Do not print message for pages without any writable parts. */
> > +}
> > +
> > +bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla)
> > +{
> > +    unsigned int offset =3D PAGE_OFFSET(gla);
> > +    const struct subpage_ro_range *entry;
> > +
> > +    list_for_each_entry_rcu(entry, &subpage_ro_ranges, list)
>=20
> Considering the other remark about respective devices impossible to go
> away, is the RCU form here really needed? Its use gives the (false)
> impression of entry removal being possible.

Right, I forgot to change this one.

> > +        if ( mfn_eq(entry->mfn, mfn) &&
> > +             !test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qword=
s) )
>=20
> Btw, "qwords" in the field name is kind of odd when SUBPAGE_MMIO_RO_ALIGN
> in principle suggests that changing granularity ought to be possible by
> simply adjusting that #define. Maybe "->ro_elems"?

Makes sense.

> > --- a/xen/arch/x86/pv/ro-page-fault.c
> > +++ b/xen/arch/x86/pv/ro-page-fault.c
> > @@ -330,6 +330,7 @@ static int mmio_ro_do_page_fault(struct x86_emulate=
_ctxt *ctxt,
> >              return X86EMUL_UNHANDLEABLE;
> >      }
> > =20
> > +    mmio_ro_ctxt.mfn =3D mfn;
> >      ctxt->data =3D &mmio_ro_ctxt;
> >      if ( pci_ro_mmcfg_decode(mfn_x(mfn), &mmio_ro_ctxt.seg, &mmio_ro_c=
txt.bdf) )
> >          return x86_emulate(ctxt, &mmcfg_intercept_ops);
>=20
> Wouldn't you better set .mfn only on the "else" path, just out of context?
> Suggesting that the new field in the struct could actually overlay the
> (seg,bdf) tuple (being of relevance only to MMCFG intercept handling).
> This would be more for documentation purposes than to actually save space.
> (If so, perhaps the "else" itself would also better be dropped while maki=
ng
> the adjustment.)

I can do that if you prefer. But personally, I find such such use of an
union risky (without some means for a compiler to actually enforce their
proper use) - while for correct code it may save some space, it makes
the impact of a type confusion bug potentially worse - now that the
unexpected value would be potentially attacker controlled.
For a documentation purpose I can simply add a comment.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--BEvgsup/CxFtuvLg
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZNysMACgkQ24/THMrX
1yzjFQgAhcdVJfnkBUGUd4ZW0buOVuIZrw2zsqciwgLcscAwy4WzvqvwbDPJ6bJs
HpWExRpCmkfgnhsucnylpj5WPcWeWjQu7ctnlAKvVEjc/YdnJLmQTf9UAdbXLE/C
Olk6DSQW9VUnao7kez6+Er5I1FnI9hKBpI2gzS3yUYiNq7XJPagI9PF6JoY/fOLR
ooClh1AINqkmsB+WsRwXAWhsRU66y0uZKu/1MgpYDzmXoVbdi9zxh99RFd8D3ykF
kqRELR5jgoBmLGwHWk8yrIgu1DsVccDMp3y7sinL8RDwpSnLIaS/zqqE9WLPQODU
Msk2CGK7Dblh29KLNKOYxgg+E0QbMQ==
=pwKK
-----END PGP SIGNATURE-----

--BEvgsup/CxFtuvLg--


From xen-devel-bounces@lists.xenproject.org Wed May 22 10:50:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 10:50:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727487.1131963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jXt-00053v-6Z; Wed, 22 May 2024 10:50:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727487.1131963; Wed, 22 May 2024 10:50:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jXt-00053o-3h; Wed, 22 May 2024 10:50:09 +0000
Received: by outflank-mailman (input) for mailman id 727487;
 Wed, 22 May 2024 10:50:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9jXs-00053i-7M
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 10:50:08 +0000
Received: from wfout2-smtp.messagingengine.com
 (wfout2-smtp.messagingengine.com [64.147.123.145])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0b7e6862-1829-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 12:50:06 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.west.internal (Postfix) with ESMTP id 5EA9B1C000A9;
 Wed, 22 May 2024 06:50:02 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Wed, 22 May 2024 06:50:02 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 06:50:00 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b7e6862-1829-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716375002;
	 x=1716461402; bh=T2OJF4JeRo6DywaHzV/H7q42Y/iQEwfR5qEXWTKvIGk=; b=
	QNtaVr5zwgZH1Yq+pu8CWRw5gg2yjlXk0mFZnt7aNb/jNa1MT/wR98E1CcWAa/8A
	6PjbO9yIOSZruaVz62ZVzVPfQO4Yt6n8EFpbyDoxgnNg35WYfo4iD5KLvaGgrOet
	DMfhc5uKWzaaldm9kp1bw2H1f4MuNLqbsTy90SUweR/H09lpjdmE2I7FyMVdMpRv
	2S8ir7JUX1cA19bBBWYosTILVVO9pv8mNKDqyymK9YDKK270ITKOR9+R5aszfbdl
	Z9eHGODfl/lnC2x1xVlB/P2Kv/zmJ8EonyNro5grdE1yxaQNFSAd/99Fzunnst1B
	QgTEv0VLsRqhKD477UZN1A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716375002; x=1716461402; bh=T2OJF4JeRo6DywaHzV/H7q42Y/iQ
	EwfR5qEXWTKvIGk=; b=euAXoKnhRTrWrFmwH8F2KpW1YC+7kP3QmqhYJ62y9nwW
	yCAsbSLuJiIZzGFIBXjVN9FZVIGpp6j8LTUZqWdh88uqktqRZT93YB/2f+6FArQN
	LEtahhnjfFjJohErvqeoTftQjtQ4Rr0MP7Q+T41aSvnOIl0pbkyV82eBoGA1HmED
	meSIjMAVoa6oLoQ/Kga4TknqmA16k2oMCJB2ksXlCh8JOljttcwv2LLIP2MYTJmX
	M584JL4/xd+Ab6sAUqLTexvitLf8lHgJjf17xHToBPIAiL6nm7i+E0sImldtltJw
	XiZwgznexaN2/Et3Plk3sVzKTK++uteeTe2qGKt/dg==
X-ME-Sender: <xms:2c1NZtroFsOxe5-T6MHX5zCMNzbXhbLYkhxHk07N0nRc5NaSgAiVJw>
    <xme:2c1NZvoc46Jqpq905JMPgroDp3EC5crgbL6IA3HX7vB1nL-VNRugHswRyrsqaTzxh
    lx3g_K0XDSP5A>
X-ME-Received: <xmr:2c1NZqNKlOCLfKigPE7tuZbxV1sQxPukQy4QAfag743dz0ImxhaOBjvoMrnjA9qiei9F3XMYBY9wV6AoB8XlAbOSYSny2FPbYw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgtddvucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:2c1NZo6lWUdggVK7JBN6bEyAYnhz2UDEwy3QT24FyS4oZ9SArgp6JA>
    <xmx:2c1NZs7VFOspiqfs3VX7jMLrpLIXxQVDbzo2IcSphyUrZ8k2v0_Fnw>
    <xmx:2c1NZgi5xA6wLN3jJ0xogjZJ-SXC-3pYIVZm0zXSvmSTZKW2q2iJ-g>
    <xmx:2c1NZu6J_f1KkGscKtcyLmLXGAyXML4XrhB_661OH1W9yqV46I-bvQ>
    <xmx:2s1NZmSBU7-rhn8-OkHvp9VTFl_kryf58FHLgbZ4i7CcDXWsVQ8VdrMU>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 22 May 2024 12:49:58 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 2/2] drivers/char: Use sub-page ro API to make just
 xhci dbc cap RO
Message-ID: <Zk3N1peuxXthOnwy@mail-itl>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <7682bbf3a24045ae1fcbdf651a6dd8e609ec3a20.1716260066.git-series.marmarek@invisiblethingslab.com>
 <0dbc701d-d8e6-4cbf-b7bf-27d9e05b5491@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="RfZcqG+IrPjM0llv"
Content-Disposition: inline
In-Reply-To: <0dbc701d-d8e6-4cbf-b7bf-27d9e05b5491@suse.com>


--RfZcqG+IrPjM0llv
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 22 May 2024 12:49:58 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 2/2] drivers/char: Use sub-page ro API to make just
 xhci dbc cap RO

On Wed, May 22, 2024 at 10:05:05AM +0200, Jan Beulich wrote:
> On 21.05.2024 04:54, Marek Marczykowski-G=C3=B3recki wrote:
> > --- a/xen/drivers/char/xhci-dbc.c
> > +++ b/xen/drivers/char/xhci-dbc.c
> > @@ -1216,20 +1216,19 @@ static void __init cf_check dbc_uart_init_posti=
rq(struct serial_port *port)
> >          break;
> >      }
> >  #ifdef CONFIG_X86
> > -    /*
> > -     * This marks the whole page as R/O, which may include other regis=
ters
> > -     * unrelated to DbC. Xen needs only DbC area protected, but it see=
ms
> > -     * Linux's XHCI driver (as of 5.18) works without writting to the =
whole
> > -     * page, so keep it simple.
> > -     */
> > -    if ( rangeset_add_range(mmio_ro_ranges,
> > -                PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MAS=
K) +
> > -                         uart->dbc.xhc_dbc_offset),
> > -                PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK)=
 +
> > -                       uart->dbc.xhc_dbc_offset +
> > -                sizeof(*uart->dbc.dbc_reg)) - 1) )
> > -        printk(XENLOG_INFO
> > -               "Error while adding MMIO range of device to mmio_ro_ran=
ges\n");
> > +    if ( subpage_mmio_ro_add(
> > +             (uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
> > +              uart->dbc.xhc_dbc_offset,
> > +             sizeof(*uart->dbc.dbc_reg)) )
> > +    {
> > +        printk(XENLOG_WARNING
> > +               "Error while marking MMIO range of XHCI console as R/O,=
 "
> > +               "making the whole device R/O (share=3Dno)\n");
>=20
> Since you mention "share=3Dno" here, wouldn't you then better also update=
 the
> respective struct field, even if (right now) there may be nothing subsequ=
ently
> using that? Except that dbc_ensure_running() actually is looking at it, a=
nd
> that's not an __init function.

That case is just an optimization - if pci_ro_device() is used, nobody
else could write to PCI_COMMAND behind the driver backs, so there is no
point checking. Anyway, yes, makes sense to adjust dbc->share too.

> > +        if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn=
) )
> > +            printk(XENLOG_WARNING
> > +                   "Failed to mark read-only %pp used for XHCI console=
\n",
> > +                   &uart->dbc.sbdf);
> > +    }
> >  #endif
> >  }
>=20
> It's been a long time since v2 and the description doesn't say anything in
> this regard: Is there a reason not to retain the rangeset addition alongs=
ide
> the pci_ro_device() on the fallback path?

pci_ro_device() prevents device from being assigned to domU at all, so
that case is covered already. Dom0 would fail to load any driver (if
nothing else - because it can't size the BARs with R/O config space), so
a _well behaving_ Dom0 would also not touch the device in this case.
But otherwise, yes, it makes sense keep adding to mmio_ro_ranges in the
fallback path.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--RfZcqG+IrPjM0llv
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZNzdYACgkQ24/THMrX
1yzulwf/fLSbA75UWeJwt5Wks4vp5zyGXbvFK1zmkA3jBBNG5l4Xy4fSXgUD1mht
J0DY/S11sBo8poVJijba7tKTPHQl2rj/EOEFCQGFUi3+SJKGvYYcWLFQtOcZIZvV
FZ2zdctRlX/UYfBRRsX6TljKQnnqbKWx23cY3mXxgrOrki0VdOnVkocTxbX2tU7w
a28bJOqc8/9HM0J5iZA2Vmub1hBsRl4Kyo9HlUy5t+iMn6bIg1ov8gYfVrYnnSjC
YzgEvtsHvvQcXkfBLbt4jyg6gVB7xKKwVed23G89W9w0VAVc3kLynIdMZZBlCl0i
xNs+KgSGr/UQ9NKqNKWU15LSbVeRmQ==
=q+Pw
-----END PGP SIGNATURE-----

--RfZcqG+IrPjM0llv--


From xen-devel-bounces@lists.xenproject.org Wed May 22 11:12:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 11:12:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727506.1131987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jsn-0000zv-1O; Wed, 22 May 2024 11:11:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727506.1131987; Wed, 22 May 2024 11:11:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9jsm-0000zo-Tb; Wed, 22 May 2024 11:11:44 +0000
Received: by outflank-mailman (input) for mailman id 727506;
 Wed, 22 May 2024 11:11:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ieL=MZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9jsl-0000zi-KL
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 11:11:43 +0000
Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com
 [2607:f8b0:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0e29335f-182c-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 13:11:37 +0200 (CEST)
Received: by mail-oi1-x232.google.com with SMTP id
 5614622812f47-3c9c67c059bso2317914b6e.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 04:11:37 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf27f847sm1384224385a.29.2024.05.22.04.11.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 04:11:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e29335f-182c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716376296; x=1716981096; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=5TGGydkdUDAa2iDRyDX9I68XDOJ0g6xmW8BLIK0f/dU=;
        b=PLAuUWDikjaOtUWDLLj0nJsb8mVqWlWptB3pKFTw0PZOr5hUPuBs8qeCc7UdExdQn8
         rUoC17qSH/1mbJycGGCd5Lc+WysXVPZmEOs8PIGtupLSk8JUvSReM3SX8gcMQZvUGxhV
         3mSdDt7oy5PC33tlE9wWdg1WELl/HUbt5lCDI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716376296; x=1716981096;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5TGGydkdUDAa2iDRyDX9I68XDOJ0g6xmW8BLIK0f/dU=;
        b=nH+HbbFZCeZaj/PXH3IWZZFc3KUgOqvwpVw+uMGYPCT5n09n/bSJcZ+bVGZ5+oTJsI
         Z//YkNpqLPkyLdMFZgk2lYhyf3YzY2m4F8B434172E7Z3WOfebWefFdOdi4zA+QEriZO
         0kX+EiTfR8W2PLWS5v4VNoKKXgb8Lq+UhEX5uGDx+vB4pOTA6+8QYX2bCUWtrL3ge72V
         jWSIHBn4/cwbZx08XFqBFb/hgjgfD9mGKrTuAcAE6s/LitRnnw9nllXgc9iCY8jWeWwf
         TOuNn7Nz9BIm59C1ZMV9ywsAlmgUCXH+zEG5dCAiHFirvQy+lWhJJAdA3mYgG8UZc4hT
         KLgQ==
X-Gm-Message-State: AOJu0Yw2n5cyVrB8U13oBTw/U9IFG/qgvnAgiRUDxK8gZtOQEcAsYO4S
	g3cB5y1WD4qWGiYEMn72oHBx5ch3Txpo8RBOomKbIG7VjGAb/JP9EOUmVJpo2yU=
X-Google-Smtp-Source: AGHT+IEAsOkFas2LpR9RdVRT4Yfp/4g3CfgAqp5COztGmmByyhedJzET8Z28W4YqVot/waccbXaZZg==
X-Received: by 2002:a05:6808:211a:b0:3c9:956b:c738 with SMTP id 5614622812f47-3cdb9d39676mr2856569b6e.43.1716376296055;
        Wed, 22 May 2024 04:11:36 -0700 (PDT)
Date: Wed, 22 May 2024 13:11:33 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Tim Deegan <tim@xen.org>,
	George Dunlap <george.dunlap@cloud.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH] x86/shadow: don't leave trace record field uninitialized
Message-ID: <Zk3S5SBC0fp-gk4G@macbook>
References: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5133a01d-346d-4779-b319-a156ef944669@suse.com>

On Wed, May 22, 2024 at 12:17:30PM +0200, Jan Beulich wrote:
> The emulation_count field is set only conditionally right now. Convert
> all field setting to an initializer, thus guaranteeing that field to be
> set to 0 (default initialized) when GUEST_PAGING_LEVELS != 3.
> 
> While there also drop the "event" local variable, thus eliminating an
> instance of the being phased out u32 type.
> 
> Coverity ID: 1598430
> Fixes: 9a86ac1aa3d2 ("xentrace 5/7: Additional tracing for the shadow code")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 22 11:13:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 11:13:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727509.1131997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ju7-0001WO-BP; Wed, 22 May 2024 11:13:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727509.1131997; Wed, 22 May 2024 11:13:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ju7-0001WH-7F; Wed, 22 May 2024 11:13:07 +0000
Received: by outflank-mailman (input) for mailman id 727509;
 Wed, 22 May 2024 11:13:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9ju6-0001W9-5x
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 11:13:06 +0000
Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com
 [2607:f8b0:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4278de74-182c-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 13:13:05 +0200 (CEST)
Received: by mail-oi1-x22d.google.com with SMTP id
 5614622812f47-3c9c36db8eeso2513658b6e.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 04:13:05 -0700 (PDT)
Received: from [10.80.67.140] ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf2ffde5sm1381988885a.79.2024.05.22.04.13.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 04:13:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4278de74-182c-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716376384; x=1716981184; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=OeNoFbEVVU41I/gwYiUiW1qHxDGWrTcL9nGwAaFs8DU=;
        b=WJz1XpP7Ntf9YMXok34scQk7guZR5IBApwoephZWijUxGv5mfMhA/wuf+AZl+JkOJp
         OqQR8RqekgtgYAgerREERhAylMrrqBA0+TMAu6CC/08VIEh1epRp8cjn5eVd1obPOSe4
         86WttOk8V1mEvpBcMWgbxXJf68tvzZcWcS1l4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716376384; x=1716981184;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OeNoFbEVVU41I/gwYiUiW1qHxDGWrTcL9nGwAaFs8DU=;
        b=E9ce2mM4NqdIQ0o4RdRNBLfsUt3Kq0pE+oGiqSkjfE6rz08kzKkuGfxis9fAoFqM52
         IPDI3mCS2lyIFe4/az2cAJnWVUmfGsP7J+AmjbEtLjump1FLBt3fzXke9AwQfdYHP7XR
         3O7ZSeyFLqZDxzUKqubqsNd8aca0dGNJBL2Q7MSU5AmPVuVU6V1/XDKH0kItReVlqMLS
         n2aqPBCoJY4BL2MP4qI+bNOMdDJnCQl8FR13oD5hPvgQ+PWm0Lzd7sfCGvF3VNciZJEv
         PyPC+ja+JUgmQIjAHobdQvMFsVvY0rNsDi+ng7CqWx4arIgspAVGx9+pvnp0R1KOB422
         +pJw==
X-Forwarded-Encrypted: i=1; AJvYcCU/eY3ANxvauXi6fDgSYr4fWaAeqLTDbJW5BV0GOcC/baYIk8R/3oHVgE869JnovyUFz67+RhmvjS4rgIoTl2BPKRQMVK4M9qoSri0RnBo=
X-Gm-Message-State: AOJu0YxmwNiEooLGisxjuLx9FF+ZKqGohhb4xT5zbJxZhShCNjOQzBpF
	GjG9uhBxUaVP/kl9Kyhs05Fpaeb1gwX8rfBW5JJJG16GE8W9vXbzzOxuwrHyJQT2KXJQmD4gV6M
	F3Js=
X-Google-Smtp-Source: AGHT+IHAqKGqZnni416nD1SNpzrMYdm+omPfRt32HC6SizrxpqIbBN/8jMHXt93SIR7Aux7v22z6TA==
X-Received: by 2002:a05:6808:1383:b0:3c9:63d1:6fde with SMTP id 5614622812f47-3cdba10c880mr2053049b6e.37.1716376383123;
        Wed, 22 May 2024 04:13:03 -0700 (PDT)
Message-ID: <a8651117-d07f-403e-b433-7ebae1a1d886@citrix.com>
Date: Wed, 22 May 2024 12:13:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/shadow: don't leave trace record field uninitialized
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tim Deegan <tim@xen.org>, George Dunlap <george.dunlap@cloud.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22/05/2024 11:17 am, Jan Beulich wrote:
> The emulation_count field is set only conditionally right now. Convert
> all field setting to an initializer, thus guaranteeing that field to be
> set to 0 (default initialized) when GUEST_PAGING_LEVELS != 3.
>
> While there also drop the "event" local variable, thus eliminating an
> instance of the being phased out u32 type.
>
> Coverity ID: 1598430
> Fixes: 9a86ac1aa3d2 ("xentrace 5/7: Additional tracing for the shadow code")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

This is an improvement, but there's a related mess right next to it.

I think this would be a whole lot better with a couple of tweaks, if
you're willing to wait a little for me to try.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 22 12:49:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 12:49:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727538.1132006 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lPL-0006yi-Fb; Wed, 22 May 2024 12:49:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727538.1132006; Wed, 22 May 2024 12:49:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lPL-0006yb-CV; Wed, 22 May 2024 12:49:27 +0000
Received: by outflank-mailman (input) for mailman id 727538;
 Wed, 22 May 2024 12:49:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9lPK-0006yR-En; Wed, 22 May 2024 12:49:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9lPK-0004gP-C8; Wed, 22 May 2024 12:49:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9lPK-0003Po-0n; Wed, 22 May 2024 12:49:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9lPK-0007Iw-0O; Wed, 22 May 2024 12:49:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hIwtaOP0um8vnlD5dWGoPODBjLXZ/HqE5iAovTX1v0Q=; b=rhmTE6zw5sXroJMwX53GaaB3P2
	4opJBFMJdbiKj5pfU8np4uDcGrxug4jOFSdR3YCfKUxIeb74tCr2KbxuEuONDK1N31lU9aHmFL3Lp
	FuKkJogZevWJEBjvR011NuUqJzjY0XWEuaNIK/RIOT+aExigmEGuX3iteY1gRO6ppWGU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186066-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186066: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
X-Osstest-Versions-That:
    xen=26b122e3bf8f3921d87312fbf5e7e13872ae92b0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 May 2024 12:49:26 +0000

flight 186066 xen-unstable real [real]
flight 186075 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186066/
http://logs.test-lab.xenproject.org/osstest/logs/186075/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot            fail pass in 186075-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-qcow2   14 migrate-support-check fail in 186075 never pass
 test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail in 186075 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186058
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186058
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186058
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186058
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186058
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186058
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c
baseline version:
 xen                  26b122e3bf8f3921d87312fbf5e7e13872ae92b0

Last test of basis   186058  2024-05-21 05:10:55 Z    1 days
Testing same since   186066  2024-05-21 19:40:37 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Henry Wang <xin.wang2@amd.com>
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   26b122e3bf..ced21fbb28  ced21fbb2842ac4655048bdee56232974ff9ff9c -> master


From xen-devel-bounces@lists.xenproject.org Wed May 22 12:50:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 12:50:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727544.1132016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lQd-0008Vl-PR; Wed, 22 May 2024 12:50:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727544.1132016; Wed, 22 May 2024 12:50:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lQd-0008Ve-Mp; Wed, 22 May 2024 12:50:47 +0000
Received: by outflank-mailman (input) for mailman id 727544;
 Wed, 22 May 2024 12:50:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9lQc-0008VS-KQ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 12:50:46 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e787069a-1839-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 14:50:45 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59ad344f7dso805911966b.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 05:50:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a61a00e96c1sm421187166b.151.2024.05.22.05.50.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 05:50:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e787069a-1839-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716382244; x=1716987044; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=To6RFsbStQNcF2OM1F7hlOKwvz4Avk1FcrDGPsfmwao=;
        b=Zg0dtiwrga3VZgNXylWFKOgudcU/2L/1dxt9NENw9Md/1hg4iTTG8N9/D9Z+J2Gpsx
         t10kvbvu/LFPhJBXDYQppIH5zasPkXhRF5yh4o0QZ3Mndd2PlMxD4/Z3W5bQpqNCqgKc
         zEDdAPjY8Thf5pV3sWstXap0X1MXTaMjrnlkLAmB0LYdP7I4dUhgjWUDmAJiqTQ/qNtJ
         W8tcPuw6K3vYc3ZUVAt++1QO5OPgAjarsHDBj9TMxBy0KVpbpQ8azo8BgP6cBF0LXGb2
         091Xrlafklv095NaamGgBHuMZvxs7TwtnWLe6pcQNSRAAoGkCRJfp4qo3IhC92t/tFGX
         rwlg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716382244; x=1716987044;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=To6RFsbStQNcF2OM1F7hlOKwvz4Avk1FcrDGPsfmwao=;
        b=CEUQn8ZYXuxodOuienl6MwMz5ywrnHdpTBjaPeJePctItjmXKem5VVPgIIxZMuHHow
         B45otf43cj5q+1py7jgzaJXew6TGA0dCN9d3iSHsJygEkHjWDtM4reiHWE8EuBFxlqc8
         LvSZF1B8HL0KbHcahHDpDHRtM+mJppWa8UYOJuAkd6fQDCC5ZXOVm6xDVW0STwfP/Sfb
         L67x2azpZ9ovqC2x7yFdtPTGnW0OsapQlAccB1ne3/1cOYfbIf0dMhtpWpUZy0eyulSJ
         KCXJIl99b++lkvzllgB3RVQtvZVsf27kMIeSrd1VktBna7a4wgLwcNn4i5pg2IP+30zL
         d24g==
X-Forwarded-Encrypted: i=1; AJvYcCUyfCvgEaOeujkl/v9RYtQFvspYlzAsn86GO4/41o9mmBg/Ll59XXYx9SyyAZ0MNdPupF02S7HbhxNuaLrVjSq0/e/bw3CbzC5KRt+hm5U=
X-Gm-Message-State: AOJu0Yw42Kmn5TAKsqrOj7Izqvx8COuqPYdOYRvnevKS2aIQIJjZ21ph
	tBRwdEwDgjB4+a68f1b10r9aXMBN858CcFPYYs1RJTgy66sQqjyhNApZh2z25A==
X-Google-Smtp-Source: AGHT+IHWUJ179WKtW0ez1S05YqC/5+srEduaIZvIIVUZRn31nhlHQoJzMuTVFlrFP7K/H0cHGVXF2A==
X-Received: by 2002:a17:906:404:b0:a5a:377e:4c4a with SMTP id a640c23a62f3a-a622816dd71mr117854666b.63.1716382244377;
        Wed, 22 May 2024 05:50:44 -0700 (PDT)
Message-ID: <d14a8c61-7f29-49be-97a8-0d8fd8719655@suse.com>
Date: Wed, 22 May 2024 14:50:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2708dbbb-f913-4920-a008-beef12cd1265@suse.com> <Zk3Kw2WX8r8qpjQj@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk3Kw2WX8r8qpjQj@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 12:36, Marek Marczykowski-Górecki wrote:
> On Wed, May 22, 2024 at 09:52:44AM +0200, Jan Beulich wrote:
>> On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
>>> --- a/xen/arch/x86/hvm/hvm.c
>>> +++ b/xen/arch/x86/hvm/hvm.c
>>> @@ -2009,6 +2009,14 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,
>>>          goto out_put_gfn;
>>>      }
>>>  
>>> +    if ( (p2mt == p2m_mmio_direct) && npfec.write_access && npfec.present &&
>>> +         subpage_mmio_write_accept(mfn, gla) &&
>>
>> Afaics subpage_mmio_write_accept() is unreachable then when CONFIG_HVM=n?
> 
> Right, the PV path hits mmio_ro_emulated_write() without my changes
> already.
> Do you suggest to make subpage_mmio_write_accept() under #ifdef
> CONFIG_HVM?

That's not just me, but also Misra.

>>> +         (hvm_emulate_one_mmio(mfn_x(mfn), gla) == X86EMUL_OKAY) )
>>> +    {
>>> +        rc = 1;
>>> +        goto out_put_gfn;
>>> +    }
>>
>> Overall this new if() is pretty similar to the immediate preceding one.
>> So similar that I wonder whether the two shouldn't be folded. 
> 
> I can do that if you prefer.
> 
>> In fact
>> it looks as if the new one is needed only for the case where you'd pass
>> through (to a DomU) a device partially used by Xen. That could certainly
>> do with mentioning explicitly.
> 
> Well, the change in mmio_ro_emulated_write() is relevant to both dom0
> and domU. It simply wasn't reachable (in this case) for HVM domU before
> (but was for PV already).

The remark was about the code here only. Of course that other change you
talk about is needed for both, and I wasn't meaning to suggest Dom0 had
worked (in this regard) prior to your change.

>>> +static void __iomem *subpage_mmio_get_page(struct subpage_ro_range *entry)
>>> +{
>>> +    void __iomem *mapped_page;
>>> +
>>> +    if ( entry->mapped )
>>> +        return entry->mapped;
>>> +
>>> +    mapped_page = ioremap(mfn_x(entry->mfn) << PAGE_SHIFT, PAGE_SIZE);
>>> +
>>> +    spin_lock(&subpage_ro_lock);
>>> +    /* Re-check under the lock */
>>> +    if ( entry->mapped )
>>> +    {
>>> +        spin_unlock(&subpage_ro_lock);
>>> +        iounmap(mapped_page);
>>
>> The only unmap is on an error path here and on another error path elsewhere.
>> IOW it looks as if devices with such marked pages are meant to never be hot
>> unplugged. I can see that being intentional for the XHCI console, but imo
>> such a restriction also needs prominently calling out in a comment next to
>> e.g. the function declaration.
> 
> The v1 included subpage_mmio_ro_remove() function (which would need to
> be used in case of hot-unplug of such device, if desirable), but since
> this series doesn't introduce any use of it (as you say, it isn't
> desirable for XHCI console specifically), you asked me to remove it...
> 
> Should I add an explicit comment about the limitation, instead of having
> it implicit by not having subpage_mmio_ro_remove() there?

That's what I was asking for in my earlier comment, yes.

>>> --- a/xen/arch/x86/pv/ro-page-fault.c
>>> +++ b/xen/arch/x86/pv/ro-page-fault.c
>>> @@ -330,6 +330,7 @@ static int mmio_ro_do_page_fault(struct x86_emulate_ctxt *ctxt,
>>>              return X86EMUL_UNHANDLEABLE;
>>>      }
>>>  
>>> +    mmio_ro_ctxt.mfn = mfn;
>>>      ctxt->data = &mmio_ro_ctxt;
>>>      if ( pci_ro_mmcfg_decode(mfn_x(mfn), &mmio_ro_ctxt.seg, &mmio_ro_ctxt.bdf) )
>>>          return x86_emulate(ctxt, &mmcfg_intercept_ops);
>>
>> Wouldn't you better set .mfn only on the "else" path, just out of context?
>> Suggesting that the new field in the struct could actually overlay the
>> (seg,bdf) tuple (being of relevance only to MMCFG intercept handling).
>> This would be more for documentation purposes than to actually save space.
>> (If so, perhaps the "else" itself would also better be dropped while making
>> the adjustment.)
> 
> I can do that if you prefer. But personally, I find such such use of an
> union risky (without some means for a compiler to actually enforce their
> proper use) - while for correct code it may save some space, it makes
> the impact of a type confusion bug potentially worse - now that the
> unexpected value would be potentially attacker controlled.
> For a documentation purpose I can simply add a comment.

Well, I'm not going to insist on using a union. But I am pretty firm on
expecting the setting of .mfn to move down. Not using a union will then
mean static analysis tools may point out that .mfn is left uninitialized
for the above visible 1st invocation of x86_emulate().

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 12:58:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 12:58:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727583.1132090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lXg-0001WH-8x; Wed, 22 May 2024 12:58:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727583.1132090; Wed, 22 May 2024 12:58:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lXg-0001WA-69; Wed, 22 May 2024 12:58:04 +0000
Received: by outflank-mailman (input) for mailman id 727583;
 Wed, 22 May 2024 12:58:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9lXf-0001W4-6G
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 12:58:03 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eb7eace8-183a-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 14:58:01 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso958822266b.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 05:58:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a5c033afasm1433727366b.156.2024.05.22.05.58.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 05:58:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb7eace8-183a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716382680; x=1716987480; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ejHjN77I6205aI9N+tr1vAfoGHfyt+oSRb0cin8tAFI=;
        b=dY3qUnx1gu3j2vMbD28fbsyjm+Pm6K8dNy3/SsNmcVK4KespojRFVT+s/7qoRuLY38
         /a03Uzr6Wkzwh2UbpmI4Q8Kc6iVgITPZjEs2zOUV5MNTw7Sko6G8zFF2VqfejRp44czB
         Ht4ltHvf6JsS11uCEN08vBNC31r+90PFgiwHsLEc+msqDFteJiy6R5q5+LfPz1GAd1pJ
         xMzhi9dK8deckBIFAYRUiPPSR8fx5S745fu/MUz5E0/HSIRwr49SnEvovRe6IhOkZbP6
         SxYHw6juMwGmafDKlLjB0kXuCJQomZQVIJCMtu55tGJTbjSeBd6VSaFH45RgZacQnSga
         fGJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716382680; x=1716987480;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ejHjN77I6205aI9N+tr1vAfoGHfyt+oSRb0cin8tAFI=;
        b=iv4qJJiJV6KB4WmuIBjtoBtfpTnfkefRZZkTUIXLu3evPLZA7PwSY3gs2koQ0ceRIZ
         PBFsNkZY+OYgXX89i3TSHYx0+FsLThZepfxzH0GioCdBJpUiAvqjf5+EZ1G8cFQ9Mx3E
         nG+6Pr0Mc+upIy9h2TfYba7/DiYSiD6vRYB+6zKl/Rfe8XUUgCkwPB25YEC4v6+wAByI
         x6xibUTPy7kEOtVDzw28S5ryArxQ81r8iIiLBCFK6Bd5X9NzDS3lVvNQ1rMy+68yZ+bJ
         fgikLA9C9mVv4czsU1imAdXTUApqWRe2s3HDrhJB58qyJ/zLyK4orzwP9q3ByrZFP/LX
         M2BA==
X-Gm-Message-State: AOJu0Ywhf9cVvHY/ubSWNIzc5sK/CDx6g1jL3nab48GUaG2SfzTJE2mD
	Yi9cEJrVCKO/s9rGm93/CL8OObC8+T4tV/FUvP0KR5bvL7nWceNfQlYwN9VNuV+PPLeTSR2JLsQ
	=
X-Google-Smtp-Source: AGHT+IGyfMEOvJdPNDLFJjA7oHtW9RZok+yOaZGxf7dJCdQHGStGpZiS50mD25kIFWpOw8QilXzVQQ==
X-Received: by 2002:a17:906:a1d4:b0:a59:a033:3e2 with SMTP id a640c23a62f3a-a622820b853mr176348566b.74.1716382680584;
        Wed, 22 May 2024 05:58:00 -0700 (PDT)
Message-ID: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
Date: Wed, 22 May 2024 14:57:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jason Andryuk <jason.andryuk@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v3 0/2] x86: detect PIT aliasing on ports other than 0x4[0-3]
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

1: PIT: supply and use #define-s
2: detect PIT aliasing on ports other than 0x4[0-3]

No functional change from v2, just the introduction of the new prereq
patch to help overall readability.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 12:59:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 12:59:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727588.1132099 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lZ3-00027W-MZ; Wed, 22 May 2024 12:59:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727588.1132099; Wed, 22 May 2024 12:59:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lZ3-00027P-Jx; Wed, 22 May 2024 12:59:29 +0000
Received: by outflank-mailman (input) for mailman id 727588;
 Wed, 22 May 2024 12:59:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9lZ2-00025w-04
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 12:59:28 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e0d845f-183b-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 14:59:25 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59ad344f7dso806760666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 05:59:25 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7e02sm1775829466b.105.2024.05.22.05.59.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 05:59:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e0d845f-183b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716382765; x=1716987565; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Ot36334SXNG/2Q+xhXO1nt7KGyGP0a+FespoVK2pges=;
        b=f+2OcwC9CSj58JJK5x+RfOLcMLTZ3+ga85Wjo3kJr+kTT8rIxGaivJBce8Tc1QjFZC
         YaklO1mWV5CqZswhziLSgRT8FNTICDvH95FI4BxMdBZ4RW1YofH1Itf3fq4daMrVL9p7
         VIHxMWFBndXJg76UNXjE/tZiiyKWO08GSFdgOrLBXLFu4gcshSKVceGwDZUS5WwDB642
         FpxQ64RkwaSXzIxnh2WdpS14zqsVLdbvZHJPhkTCvuY6D2Mnr/jnvin7Q8dwP1rYYAi+
         Jsf0pAhIXWU7N6oDUtrHknmFMcC0Y7qNl9GGTH5ZpYWaVwIP6ebum0MjZJ4Smy8VM04F
         bqCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716382765; x=1716987565;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Ot36334SXNG/2Q+xhXO1nt7KGyGP0a+FespoVK2pges=;
        b=GGBwXrN/z5isVONSVELJ+61YD/Qs27tz9zHuJZNM5IK216h67PkB2HjdTH85zBgZrI
         gfVB5txRMme6TWQV48Q98hG8G0pv3eG3gT3CjAdAIjdbCKTaCeWawxvhLfnHsqr4fO1y
         ER/XNl9jaiuxjYijcc3Mebjo3/IykPm3ABMDGD+2uB+QNxeN2rvlNLTtB3M8hx+n8Fkm
         v2a4eH3l0tZ1woJfnjGb7tjFw2ZXxYhtmEpkTX5yNBnnnXwx2PfhGav+BXW7B6H/QoW7
         mRo0+b7gwjT1PMYnu6JfFMmaZC6awDnf9nG7Y4zCGP1+7KLZ53FW1RjFWQymkfzkV+ob
         7rBQ==
X-Gm-Message-State: AOJu0Yw8T+tHkoHP4z3/4W+ukt9cJ3xpJY8QQZPvFxeW9CuFgq4xnt8a
	TpHACuM3RXKi/BoscUTpd6swz5fFrmXDJNE1D81Ocw+3KXa62EGzneau4Om9y91T0MkV4Ehr42Q
	=
X-Google-Smtp-Source: AGHT+IEVYMYlfCgC5O1dvLV7XE9cbvi0a7DkNm9/U+lwVpGbgblJx0gDuYb9X/n1dsRhkK43Vk9qZA==
X-Received: by 2002:a17:906:404:b0:a5a:377e:4c4a with SMTP id a640c23a62f3a-a622816dd71mr119023766b.63.1716382765327;
        Wed, 22 May 2024 05:59:25 -0700 (PDT)
Message-ID: <fb87a1db-4d8f-4e33-8858-71de4de6e276@suse.com>
Date: Wed, 22 May 2024 14:59:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 1/2] x86/PIT: supply and use #define-s
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jason Andryuk <jason.andryuk@amd.com>
References: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Help reading of code programming the PIT by introducing constants for
control word, read back and latch commands, as well as status.

Requested-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: New.

--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -983,7 +983,7 @@ static unsigned int __init get_8254_time
 
     /*spin_lock_irqsave(&i8253_lock, flags);*/
 
-    outb_p(0x00, PIT_MODE);
+    outb_p(PIT_LTCH_CH(0), PIT_MODE);
     count = inb_p(PIT_CH0);
     count |= inb_p(PIT_CH0) << 8;
 
--- a/xen/arch/x86/include/asm/time.h
+++ b/xen/arch/x86/include/asm/time.h
@@ -58,4 +58,38 @@ struct time_scale;
 void set_time_scale(struct time_scale *ts, u64 ticks_per_sec);
 u64 scale_delta(u64 delta, const struct time_scale *scale);
 
+/* Programmable Interval Timer (8254) */
+
+/* Timer Control Word */
+#define PIT_TCW_CH(n)         ((n) << 6)
+/* Lower bits also Timer Status. */
+#define PIT_RW_MSB            (1 << 5)
+#define PIT_RW_LSB            (1 << 4)
+#define PIT_RW_LSB_MSB        (PIT_RW_LSB | PIT_RW_MSB)
+#define PIT_MODE_EOC          (0 << 1)
+#define PIT_MODE_ONESHOT      (1 << 1)
+#define PIT_MODE_RATE_GEN     (2 << 1)
+#define PIT_MODE_SQUARE_WAVE  (3 << 1)
+#define PIT_MODE_SW_STROBE    (4 << 1)
+#define PIT_MODE_HW_STROBE    (5 << 1)
+#define PIT_BINARY            (0 << 0)
+#define PIT_BCD               (1 << 0)
+
+/* Read Back Command */
+#define PIT_RDB               PIT_TCW_CH(3)
+#define PIT_RDB_NO_COUNT      (1 << 5)
+#define PIT_RDB_NO_STATUS     (1 << 4)
+#define PIT_RDB_CH2           (1 << 3)
+#define PIT_RDB_CH1           (1 << 2)
+#define PIT_RDB_CH0           (1 << 1)
+#define PIT_RDB_RSVD          (1 << 0)
+
+/* Counter Latch Command */
+#define PIT_LTCH_CH(n)        PIT_TCW_CH(n)
+
+/* Timer Status */
+#define PIT_STATUS_OUT_PIN    (1 << 7)
+#define PIT_STATUS_NULL_COUNT (1 << 6)
+/* Lower bits match Timer Control Word. */
+
 #endif /* __X86_TIME_H__ */
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -222,7 +222,7 @@ static void cf_check timer_interrupt(int
 
         spin_lock_irq(&pit_lock);
 
-        outb(0x80, PIT_MODE);
+        outb(PIT_LTCH_CH(2), PIT_MODE);
         count  = inb(PIT_CH2);
         count |= inb(PIT_CH2) << 8;
 
@@ -245,7 +245,8 @@ static void preinit_pit(void)
 {
     /* Set PIT channel 0 to HZ Hz. */
 #define LATCH (((CLOCK_TICK_RATE)+(HZ/2))/HZ)
-    outb_p(0x34, PIT_MODE);        /* binary, mode 2, LSB/MSB, ch 0 */
+    outb_p(PIT_TCW_CH(0) | PIT_RW_LSB_MSB | PIT_MODE_RATE_GEN | PIT_BINARY,
+           PIT_MODE);
     outb_p(LATCH & 0xff, PIT_CH0); /* LSB */
     outb(LATCH >> 8, PIT_CH0);     /* MSB */
 #undef LATCH
@@ -356,7 +357,7 @@ static u64 cf_check read_pit_count(void)
 
     spin_lock_irqsave(&pit_lock, flags);
 
-    outb(0x80, PIT_MODE);
+    outb(PIT_LTCH_CH(2), PIT_MODE);
     count16  = inb(PIT_CH2);
     count16 |= inb(PIT_CH2) << 8;
 
@@ -383,7 +384,8 @@ static s64 __init cf_check init_pit(stru
      */
 #define CALIBRATE_LATCH CALIBRATE_VALUE(CLOCK_TICK_RATE)
     BUILD_BUG_ON(CALIBRATE_LATCH >> 16);
-    outb(0xb0, PIT_MODE);                  /* binary, mode 0, LSB/MSB, Ch 2 */
+    outb(PIT_TCW_CH(2) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
+         PIT_MODE);
     outb(CALIBRATE_LATCH & 0xff, PIT_CH2); /* LSB of count */
     outb(CALIBRATE_LATCH >> 8, PIT_CH2);   /* MSB of count */
 #undef CALIBRATE_LATCH
@@ -408,7 +410,8 @@ static s64 __init cf_check init_pit(stru
 static void cf_check resume_pit(struct platform_timesource *pts)
 {
     /* Set CTC channel 2 to mode 0 again; initial value does not matter. */
-    outb(0xb0, PIT_MODE); /* binary, mode 0, LSB/MSB, Ch 2 */
+    outb(PIT_TCW_CH(2) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
+         PIT_MODE);
     outb(0, PIT_CH2);     /* LSB of count */
     outb(0, PIT_CH2);     /* MSB of count */
 }
@@ -2456,7 +2459,8 @@ static int _disable_pit_irq(bool init)
     }
 
     /* Disable PIT CH0 timer interrupt. */
-    outb_p(0x30, PIT_MODE);
+    outb_p(PIT_TCW_CH(0) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
+           PIT_MODE);
     outb_p(0, PIT_CH0);
     outb_p(0, PIT_CH0);
 
@@ -2562,17 +2566,18 @@ int hwdom_pit_access(struct ioreq *ioreq
     case PIT_MODE:
         if ( ioreq->dir == IOREQ_READ )
             return 0; /* urk! */
-        switch ( ioreq->data & 0xc0 )
+        switch ( ioreq->data & PIT_TCW_CH(3) )
         {
-        case 0xc0: /* Read Back */
-            if ( ioreq->data & 0x08 )    /* Select Channel 2? */
-                outb(ioreq->data & 0xf8, PIT_MODE);
-            if ( !(ioreq->data & 0x06) ) /* Select Channel 0/1? */
+        case PIT_RDB: /* Read Back */
+            if ( ioreq->data & PIT_RDB_CH2 )
+                outb(ioreq->data & ~(PIT_RDB_CH1 | PIT_RDB_CH0 | PIT_RDB_RSVD),
+                     PIT_MODE);
+            if ( !(ioreq->data & (PIT_RDB_CH0 | PIT_RDB_CH1)) )
                 return 1; /* no - we're done */
             /* Filter Channel 2 and reserved bit 0. */
-            ioreq->data &= ~0x09;
+            ioreq->data &= ~(PIT_RDB_CH2 | PIT_RDB_RSVD);
             return 0; /* emulate ch0/1 readback */
-        case 0x80: /* Select Counter 2 */
+        case PIT_TCW_CH(2):
             outb(ioreq->data, PIT_MODE);
             return 1;
         }



From xen-devel-bounces@lists.xenproject.org Wed May 22 12:59:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 12:59:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727591.1132110 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lZT-0002p9-UY; Wed, 22 May 2024 12:59:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727591.1132110; Wed, 22 May 2024 12:59:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lZT-0002p2-Rw; Wed, 22 May 2024 12:59:55 +0000
Received: by outflank-mailman (input) for mailman id 727591;
 Wed, 22 May 2024 12:59:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9lZS-00025w-Ky
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 12:59:54 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2e1d14d2-183b-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 14:59:52 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so815171566b.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 05:59:52 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7e02sm1775829466b.105.2024.05.22.05.59.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 05:59:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e1d14d2-183b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716382792; x=1716987592; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=4YzomDhXXm/vFcq+Eo68VE4aq0MfzRMkOmfWSyEBToo=;
        b=XiTeH/AMKeoFvUO3S2FhER3hpsyFZudjKHisGBdY6Zsb7CWDWIjZ3BKTz5XROc9pyg
         TlbICZJiM5GqgmDhgiHZEGJYTppXk/WiWKs4ej/vD88bekqlXKxhQ7ar89b6VgsOSFKK
         UekXDhfZwvWTi+Z0xuO5h49Tj4bgpIA+3o+xM4ds47rxf1+q7LWg6/erJB6qYGQGpwD9
         gqL/wOpIkWJmQFw9uPUpRN8Y2tBEpWDY/k8S0SlpxTx7yh5A7MNGqH641V3/ik2y+UA+
         anVm190gxQoe1SefDNiVett3CQUk+5rjSXbbJ81plofPPa4Cq+RHXQnW7nwloFVWxCds
         9q1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716382792; x=1716987592;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=4YzomDhXXm/vFcq+Eo68VE4aq0MfzRMkOmfWSyEBToo=;
        b=JjR3eqDRtSmDjR3ArqHjbA0jsXnSLDYUujdAqaQhvT8vr7tIujdYHgoTt8mmmGvL3x
         xveI6Fyd3jLxbTw9JPgQmIqp2gQoCYZDc7WuzsW8W3tweB4H9+l6GpgTV8+rVklsk1nG
         EPhn1d8W4CmKHwskK0MDlpSBFpG8M5Z19DPTzaR6xF5QcOYWOiZrpPxH9jV7bBGYi2RM
         InOVL5fZN1Vnp1/4HQfii+8qUghHvRm4mJ3bdtQc0GfiWVP6nEWutmJyE87ktdlxryTw
         V6WLlFMH+GnfvpjFeJV6MUTt9YNvHyZY06EPSdnW9YK0DU6IT/9O4qIUx8ifBd6ngKYR
         s1MA==
X-Gm-Message-State: AOJu0Yw1bsFjcw6SNXwqfo9CACcn7dpif1LC9kBsDU2xEPyetkbC5pnE
	foSFbffFKV4B3xm1PWSFHpS32On11qxG99oqAKHfScBun6I+XGPruzL4z+75tdUQyAxQMS6Ruas
	=
X-Google-Smtp-Source: AGHT+IF4mcbHXyyQIF7CzBK3CMoIN3r/xzogk7aUVq111GiUwOhUlZ7d20pjVYsARTy/cvZwrqu82A==
X-Received: by 2002:a17:906:3715:b0:a55:a895:46ae with SMTP id a640c23a62f3a-a62281c48b7mr128549666b.63.1716382792354;
        Wed, 22 May 2024 05:59:52 -0700 (PDT)
Message-ID: <df53dd77-9341-4a72-a4ce-33654b2e877d@suse.com>
Date: Wed, 22 May 2024 14:59:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 2/2] x86: detect PIT aliasing on ports other than 0x4[0-3]
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jason Andryuk <jason.andryuk@amd.com>
References: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... in order to also deny Dom0 access through the alias ports (commonly
observed on Intel chipsets). Without this it is only giving the
impression of denying access to PIT. Unlike for CMOS/RTC, do detection
pretty early, to avoid disturbing normal operation later on (even if
typically we won't use much of the PIT).

Like for CMOS/RTC a fundamental assumption of the probing is that reads
from the probed alias port won't have side effects (beyond such that PIT
reads have anyway) in case it does not alias the PIT's.

As to the port 0x61 accesses: Unlike other accesses we do, this masks
off the top four bits (in addition to the bottom two ones), following
Intel chipset documentation saying that these (read-only) bits should
only be written with zero.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
If Xen was running on top of another instance of itself (in HVM mode,
not PVH, i.e. not as a shim), prior to 14f42af3f52d ('x86/vPIT: account
for "counter stopped" time') I'm afraid our vPIT logic would not have
allowed the "Try to further make sure ..." check to pass in the Xen
running on top: We don't respect the gate bit being clear when handling
counter reads. (There are more unhandled [and unmentioned as being so]
aspects of PIT behavior though, yet it's unclear in how far addressing
at least some of them would be useful.)
---
v3: Use PIT_* in dom0_setup_permissions(). Use #define-s introduced by
    new earlier patch.
v2: Use new command line option. Re-base over changes to earlier
    patches. Use ISOLATE_LSB().

--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -20,6 +20,7 @@
 #include <asm/p2m.h>
 #include <asm/setup.h>
 #include <asm/spec_ctrl.h>
+#include <io_ports.h>
 
 struct memsize {
     long nr_pages;
@@ -495,7 +496,11 @@ int __init dom0_setup_permissions(struct
     rc |= ioports_deny_access(d, 0x4D0, 0x4D1);
 
     /* Interval Timer (PIT). */
-    rc |= ioports_deny_access(d, 0x40, 0x43);
+    for ( offs = 0, i = ISOLATE_LSB(pit_alias_mask) ?: 4;
+          offs <= pit_alias_mask; offs += i )
+        if ( !(offs & ~pit_alias_mask) )
+            rc |= ioports_deny_access(d, PIT_CH0 + offs, PIT_MODE + offs);
+
     /* PIT Channel 2 / PC Speaker Control. */
     rc |= ioports_deny_access(d, 0x61, 0x61);
 
--- a/xen/arch/x86/include/asm/setup.h
+++ b/xen/arch/x86/include/asm/setup.h
@@ -49,6 +49,7 @@ extern unsigned long highmem_start;
 #endif
 
 extern unsigned int i8259A_alias_mask;
+extern unsigned int pit_alias_mask;
 
 extern int8_t opt_smt;
 extern int8_t opt_probe_port_aliases;
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -427,6 +427,74 @@ static struct platform_timesource __init
     .resume = resume_pit,
 };
 
+unsigned int __initdata pit_alias_mask;
+
+static void __init probe_pit_alias(void)
+{
+    unsigned int mask = 0x1c;
+    uint8_t val = 0;
+
+    if ( !opt_probe_port_aliases )
+        return;
+
+    /*
+     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
+     * count is loaded independent of counting being / becoming enabled.  Thus
+     * we have a 16-bit value fully under our control, to write and then check
+     * whether we can also read it back unaltered.
+     */
+
+    /* Turn off speaker output and disable channel 2 counting. */
+    outb(inb(0x61) & 0x0c, 0x61);
+
+    outb(PIT_LTCH_CH(2) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
+         PIT_MODE);
+
+    do {
+        uint8_t val2;
+        unsigned int offs;
+
+        outb(val, PIT_CH2);
+        outb(val ^ 0xff, PIT_CH2);
+
+        /* Wait for the Null Count bit to clear. */
+        do {
+            /* Latch status. */
+            outb(PIT_RDB | PIT_RDB_NO_COUNT | PIT_RDB_CH2, PIT_MODE);
+
+            /* Try to make sure we're actually having a PIT here. */
+            val2 = inb(PIT_CH2);
+            if ( (val2 & ~(PIT_STATUS_OUT_PIN | PIT_STATUS_NULL_COUNT)) !=
+                 (PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY) )
+                return;
+        } while ( val2 & (1 << 6) );
+
+        /*
+         * Try to further make sure we're actually having a PIT here.
+         *
+         * NB: Deliberately |, not ||, as we always want both reads.
+         */
+        val2 = inb(PIT_CH2);
+        if ( (val2 ^ val) | (inb(PIT_CH2) ^ val ^ 0xff) )
+            return;
+
+        for ( offs = ISOLATE_LSB(mask); offs <= mask; offs <<= 1 )
+        {
+            if ( !(mask & offs) )
+                continue;
+            val2 = inb(PIT_CH2 + offs);
+            if ( (val2 ^ val) | (inb(PIT_CH2 + offs) ^ val ^ 0xff) )
+                mask &= ~offs;
+        }
+    } while ( mask && (val += 0x0b) );  /* Arbitrary uneven number. */
+
+    if ( mask )
+    {
+        dprintk(XENLOG_INFO, "PIT aliasing mask: %02x\n", mask);
+        pit_alias_mask = mask;
+    }
+}
+
 /************************************************************
  * PLATFORM TIMER 2: HIGH PRECISION EVENT TIMER (HPET)
  */
@@ -2416,6 +2484,8 @@ void __init early_time_init(void)
     }
 
     preinit_pit();
+    probe_pit_alias();
+
     tmp = init_platform_timer();
     plt_tsc.frequency = tmp;
 



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727601.1132125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ldC-0004T7-ER; Wed, 22 May 2024 13:03:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727601.1132125; Wed, 22 May 2024 13:03:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ldC-0004T0-Bs; Wed, 22 May 2024 13:03:46 +0000
Received: by outflank-mailman (input) for mailman id 727601;
 Wed, 22 May 2024 13:03:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9ldA-0004Ss-FK
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:03:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9ldA-0004z9-5B; Wed, 22 May 2024 13:03:44 +0000
Received: from [15.248.2.235] (helo=[10.24.67.23])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9ld9-0007LV-VI; Wed, 22 May 2024 13:03:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=i8TmxNEgU3AOS6pP//wlBVKSGvFnWCKtg1OZBAIjH4I=; b=fwJpHJR3BU4M5jX0n7l+IEspfW
	X/ga8s+6BifKcySyHRiXpmhbob7SY/J8cDrcs6ZRof6iT5b9d6gGR2f0xrE3tNAEL7l7/5r4Wh5Pv
	QLK4ir4wEB35FqfEWVk0fOyb7W7VleSxPq1n925kAnFl/ugAwEc/I8eL+Kwc6LZLSf4Q=;
Message-ID: <1754e63c-00a5-4d66-b151-8fa436f5c844@xen.org>
Date: Wed, 22 May 2024 14:03:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-6-xin.wang2@amd.com>
 <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
 <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
 <alpine.DEB.2.22.394.2405211815040.1052252@ubuntu-linux-20-04-desktop>
 <29073700-673d-40aa-8759-efc60af3b7c0@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <29073700-673d-40aa-8759-efc60af3b7c0@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 22/05/2024 02:22, Henry Wang wrote:
> On 5/22/2024 9:16 AM, Stefano Stabellini wrote:
>> On Wed, 22 May 2024, Henry Wang wrote:
>>> Hi Julien,
>>>
>>> On 5/21/2024 8:30 PM, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 21/05/2024 05:35, Henry Wang wrote:
>>>>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>>>>> index 56490dbc43..956c11ba13 100644
>>>>> --- a/xen/arch/arm/gic-vgic.c
>>>>> +++ b/xen/arch/arm/gic-vgic.c
>>>>> @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct
>>>>> vcpu *v, unsigned int virq,
>>>>>          /* We are taking to rank lock to prevent parallel 
>>>>> connections. */
>>>>>        vgic_lock_rank(v_target, rank, flags);
>>>>> +    spin_lock(&v_target->arch.vgic.lock);
>>>> I know this is what Stefano suggested, but v_target would point to the
>>>> current affinity whereas the interrupt may be pending/active on the
>>>> "previous" vCPU. So it is a little unclear whether v_target is the 
>>>> correct
>>>> lock. Do you have more pointer to show this is correct?
>>> No I think you are correct, we have discussed this in the initial 
>>> version of
>>> this patch. Sorry.
>>>
>>> I followed the way from that discussion to note down the vcpu ID and 
>>> retrieve
>>> here, below is the diff, would this make sense to you?
>>>
>>> diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
>>> index 956c11ba13..134ed4e107 100644
>>> --- a/xen/arch/arm/gic-vgic.c
>>> +++ b/xen/arch/arm/gic-vgic.c
>>> @@ -439,7 +439,7 @@ int vgic_connect_hw_irq(struct domain *d, struct 
>>> vcpu *v,
>>> unsigned int virq,
>>>
>>>       /* We are taking to rank lock to prevent parallel connections. */
>>>       vgic_lock_rank(v_target, rank, flags);
>>> -    spin_lock(&v_target->arch.vgic.lock);
>>> + spin_lock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
>>>
>>>       if ( connect )
>>>       {
>>> @@ -465,7 +465,7 @@ int vgic_connect_hw_irq(struct domain *d, struct 
>>> vcpu *v,
>>> unsigned int virq,
>>>               p->desc = NULL;
>>>       }
>>>
>>> -    spin_unlock(&v_target->arch.vgic.lock);
>>> + spin_unlock(&d->vcpu[p->spi_vcpu_id]->arch.vgic.lock);
>>>       vgic_unlock_rank(v_target, rank, flags);
>>>
>>>       return ret;
>>> diff --git a/xen/arch/arm/include/asm/vgic.h 
>>> b/xen/arch/arm/include/asm/vgic.h
>>> index 79b73a0dbb..f4075d3e75 100644
>>> --- a/xen/arch/arm/include/asm/vgic.h
>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>> @@ -85,6 +85,7 @@ struct pending_irq
>>>       uint8_t priority;
>>>       uint8_t lpi_priority;       /* Caches the priority if this is 
>>> an LPI. */
>>>       uint8_t lpi_vcpu_id;        /* The VCPU for an LPI. */
>>> +    uint8_t spi_vcpu_id;        /* The VCPU for an SPI. */
>>>       /* inflight is used to append instances of pending_irq to
>>>        * vgic.inflight_irqs */
>>>       struct list_head inflight;
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index c04fc4f83f..e852479f13 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -632,6 +632,7 @@ void vgic_inject_irq(struct domain *d, struct 
>>> vcpu *v,
>>> unsigned int virq,
>>>       }
>>>       list_add_tail(&n->inflight, &v->arch.vgic.inflight_irqs);
>>>   out:
>>> +    n->spi_vcpu_id = v->vcpu_id;
>>>       spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>>>
>>>       /* we have a new higher priority irq, inject it into the guest */
>>>       vcpu_kick(v);
>>>
>>>
>>>> Also, while looking at the locking, I noticed that we are not doing 
>>>> anything
>>>> with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem to 
>>>> assume that
>>>> if the flag is set, then p->desc cannot be NULL.
>>>>
>>>> Can we reach vgic_connect_hw_irq() with the flag set?
>>> I think even from the perspective of making the code extra safe, we 
>>> should
>>> also check GIC_IRQ_GUEST_MIGRATING as the LR is allocated for this 
>>> case. I
>>> will also add the check of GIC_IRQ_GUEST_MIGRATING here.
>> Yes. I think it might be easier to check for GIC_IRQ_GUEST_MIGRATING
>> early and return error immediately in that case. Otherwise, we can
>> continue and take spin_lock(&v_target->arch.vgic.lock) because no
>> migration is in progress
> 
> Ok, this makes sense to me, I will add
> 
>      if( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>      {
>          vgic_unlock_rank(v_target, rank, flags);
>          return -EBUSY;
>      }
> 
> right after taking the vgic rank lock.

I think that would be ok. I have to admit, I am still a bit wary about 
allowing to remove interrupts when the domain is running.

I am less concerned about the add part. Do you need the remove part now? 
If not, I would suggest to split in two so we can get the most of this 
series merged for 4.19 and continue to deal with the remove path in the 
background.

I will answer here to the other reply:

 > I don't think so, if I am not mistaken, no LR will be allocated with 
other flags set.

I wasn't necessarily thinking about the LR allocation. I was more 
thinking whether there are any flags that could still be set.

IOW, will the vIRQ like new once vgic_connect_hw_irq() is succesful?

Also, while looking at the flags, I noticed we clear _IRQ_INPROGRESS 
before vgic_connect_hw_irq(). Shouldn't we only clear *after*?

This brings to another question. You don't special case a dying domain. 
If the domain is crashing, wouldn't this mean it wouldn't be possible to 
destroy it?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:16:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:16:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727608.1132136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lpf-0006IT-LB; Wed, 22 May 2024 13:16:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727608.1132136; Wed, 22 May 2024 13:16:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lpf-0006IM-Gu; Wed, 22 May 2024 13:16:39 +0000
Received: by outflank-mailman (input) for mailman id 727608;
 Wed, 22 May 2024 13:16:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ieL=MZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9lpe-0006IG-AZ
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:16:38 +0000
Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com
 [2607:f8b0:4864:20::734])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 84687780-183d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:16:37 +0200 (CEST)
Received: by mail-qk1-x734.google.com with SMTP id
 af79cd13be357-7948b50225bso96965385a.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:16:37 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf1cd143sm1392849085a.0.2024.05.22.06.16.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:16:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84687780-183d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383796; x=1716988596; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=PX7IYW6ySnMaVJwRwj+MatcXvq1r3mjVOL6i9GRyM8c=;
        b=k3ZdO9NQQQkPUQX4nn3RI8q0726MbS6C/XAn2DwcRgvso7ijh/lViepLi/M6cEzSwp
         aESKJoPqyTRqajHER5FdxEsfTqYhMpWvZFlg5SMNWqNWZRQnHoZ+Bs3HG+dVpjw6e+oy
         5yMb9iM/okehhVK69yEKSFCAKwppFO0M3UJQQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383796; x=1716988596;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PX7IYW6ySnMaVJwRwj+MatcXvq1r3mjVOL6i9GRyM8c=;
        b=cpg6CtRkmzvTtO4qD4lINwT3/4PMqBW+ZPwtC/yhC5xA8QpkSfBrCdvLYOVVDaVfrS
         jp9z+etJtygu29sUVgwY9AskwcWbw8SVftv32qgsQVBQmw82GDGD46eRlYjn48hthQlZ
         q1tjTsWikNXO4zx+1XmDup/tkbxif28m7GSN5W2/J3tFR2/Vdn43ZPJiYOE5DvPreA+I
         9Z5rB+Ur6RtxlE1qiK9MKAU9xyWPk4wHXTr9AFfUqpiS/1AoSLdmmr0/zs3UIq1yWBwU
         j/vpyAiKDryIME8F5m22xrwYEiCGYDCV4tnuArbUwHNRtxgIIlBh+IHMhK5wmdkKz8U5
         ZNBw==
X-Forwarded-Encrypted: i=1; AJvYcCW3D54CQind+qo6vyqV9pKhZhY+p6wENJxGRfEZQkHuJQctXOWa2ar+YPWO7HYiVk2tiUR53SJd+0VrIyC7vVHArgZIMbD7e0rn9VZDw94=
X-Gm-Message-State: AOJu0YxLS3ZX0MOm3nXSsfp/lf4HRkNE0/sb/KoOqBe/P2d6Fkd8PurN
	bT0j6SFAlawW5rz/p6K8GgCFoQg3gXP+xMzbN3a2gEahdAF+vaVTdjf9OmQY6Cw=
X-Google-Smtp-Source: AGHT+IFUpH1CAyUj9e41ym92kgDMh5S70/nICkehPf1CbJCNGH26LyaU0G5cqsvidOpe85aLrOIb/Q==
X-Received: by 2002:a05:620a:2942:b0:794:9def:9691 with SMTP id af79cd13be357-7949def9d06mr57332585a.35.1716383795947;
        Wed, 22 May 2024 06:16:35 -0700 (PDT)
Date: Wed, 22 May 2024 15:16:33 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Message-ID: <Zk3wMevPye8Rfjm4@macbook>
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
 <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com>

On Tue, May 21, 2024 at 12:30:32PM +0200, Jan Beulich wrote:
> On 17.05.2024 15:33, Roger Pau Monne wrote:
> > Enabling it using an HVM param is fragile, and complicates the logic when
> > deciding whether options that interact with altp2m can also be enabled.
> > 
> > Leave the HVM param value for consumption by the guest, but prevent it from
> > being set.  Enabling is now done using and additional altp2m specific field in
> > xen_domctl_createdomain.
> > 
> > Note that albeit only currently implemented in x86, altp2m could be implemented
> > in other architectures, hence why the field is added to xen_domctl_createdomain
> > instead of xen_arch_domainconfig.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com> # hypervisor
> albeit with one question:
> 
> > --- a/xen/arch/x86/domain.c
> > +++ b/xen/arch/x86/domain.c
> > @@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> >      bool hap = config->flags & XEN_DOMCTL_CDF_hap;
> >      bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
> >      unsigned int max_vcpus;
> > +    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
> > +                                         XEN_DOMCTL_ALTP2M_mode_mask);
> >  
> >      if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
> >      {
> > @@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> >          return -EINVAL;
> >      }
> >  
> > +    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
> > +    {
> > +        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
> > +                config->flags);
> > +        return -EINVAL;
> > +    }
> > +
> > +    if ( altp2m_mode && nested_virt )
> > +    {
> > +        dprintk(XENLOG_INFO,
> > +                "Nested virt and altp2m are not supported together\n");
> > +        return -EINVAL;
> > +    }
> > +
> > +    if ( altp2m_mode && !hap )
> > +    {
> > +        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
> > +        return -EINVAL;
> > +    }
> 
> Should this last one perhaps be further extended to permit altp2m with EPT
> only?

Hm, yes, that would be more accurate as:

if ( altp2m_mode && (!hap || !hvm_altp2m_supported()) )

Would you be fine adjusting at commit, or would you prefer me to send
an updated version?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:16:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:16:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727609.1132145 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lpr-0006b7-R8; Wed, 22 May 2024 13:16:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727609.1132145; Wed, 22 May 2024 13:16:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lpr-0006aw-OR; Wed, 22 May 2024 13:16:51 +0000
Received: by outflank-mailman (input) for mailman id 727609;
 Wed, 22 May 2024 13:16:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DzQV=MZ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s9lpq-0006aV-Li
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:16:50 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8a65dcf0-183d-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:16:48 +0200 (CEST)
Received: from BY3PR04CA0021.namprd04.prod.outlook.com (2603:10b6:a03:217::26)
 by BL3PR12MB6449.namprd12.prod.outlook.com (2603:10b6:208:3b8::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 13:16:42 +0000
Received: from MWH0EPF000971E8.namprd02.prod.outlook.com
 (2603:10b6:a03:217:cafe::9a) by BY3PR04CA0021.outlook.office365.com
 (2603:10b6:a03:217::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Wed, 22 May 2024 13:16:41 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MWH0EPF000971E8.mail.protection.outlook.com (10.167.243.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 13:16:41 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:16:39 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 08:16:38 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a65dcf0-183d-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VpDEF+CSzSe27gFEpZPk8Q4+mkWZ5+VpkrV87T9qcPbeI7ze3n8WSmmM71cpJV6eaQsCZLZmEkom80iFJdiPzYC6FoMyiFx/Bm7yUjOECBR4BtgTIcuhx4LaBSK6Keyf32Mp1w+HS9/BUsHn9th1zCN2StrZXK8TBLSGoTGpL/8ypvuMRz7brRqTz2o42QOZqy1enP9ZT//WaBfyXe6uRB64RDBqjiKdQaBvHlnF24nv3nDmbub85xEHYEEBc6lhKXiSLphyYnKxO4aNqLGPGh85L5sJdAnHeOU+C1CmtPGfmhYptB4uqXYBRbM35SOhN6T5DDT0l1CNOkVIU/gRSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=qezSWhI+vIIqlcOsVgRs306/jm1ivIoR5Jn5pb1EKJQ=;
 b=Z3dnwR/GlL3UAjWNIfZ3tOjr74sN0QGy6oNED0xeKmCzpq02fDfzLA1RhSRwTyoUyTnl7y4d4FV+m/x4QttOwZh40In3z0Ho9z9BA9kKDpa9D94QEly2FbD77SVwYlk0R5jBh8Br6QpIhm0UWvhPjDuGFj64VIHx9+zCU2wcueLt/VjKG8qfbfFLkRYTVBlSF9032RO9G1HBh1dcA7OhMsuC//2ZWHZ6qzHc4hDz+GKun2/CSDllljAJmhBsItJCHEdAOnSKljyB8UFoi6KGGC8BuANanph1xLtxJQ4LTecfF6fe5qZh+wo7U6z0FEfUJBTeiosSMKvtzuiqkkkZIA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qezSWhI+vIIqlcOsVgRs306/jm1ivIoR5Jn5pb1EKJQ=;
 b=hrZxdG2f6QH0IFYyvvaXtqjrZsfFIrNh9sB1Uy1g5LhxOpqeUrbcCwqNt+XuZwkl5L3YdpuwPITvXSCiveEGyLvGHP7w9jW2UG0m2U0Ol+BwlQUeIbLmBvG5KSRd6wzZkN9eApQeb6+jTsEf9fJzq3FGrowZEbyPoiw9WBysL68=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <309bdbda-a720-4891-a082-ee749f4c9440@amd.com>
Date: Wed, 22 May 2024 15:16:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Penny Zheng <Penny.Zheng@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-4-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240522075151.3373899-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E8:EE_|BL3PR12MB6449:EE_
X-MS-Office365-Filtering-Correlation-Id: d64b647f-d9aa-401a-e767-08dc7a616b90
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?SVBPeU9tclNxcGVsMW95V0greXdiL1lmTEpPYVpJR0lvUFJuNEwvTlEyU0RL?=
 =?utf-8?B?aUZvQ2l1UVJJSUI5TG5Ec2dIQk9uL3o0TjNhcG1JNUYyamdnTmZ2K1hjTEwx?=
 =?utf-8?B?R1pkZHQwbVpURFQxT0h6amZXY3MvbmlkR0lwaWFjekJiQ1BuRjBEU3dmVVda?=
 =?utf-8?B?VXU0UUt3VDQyanJjWDRia00vSWw1MFNhMUZpSm9HUUVrUXBOdjhkaC9abTJz?=
 =?utf-8?B?MHBDY0RhbHdtVFE1UlUvOXAyRm9wd2lHYjhEQkZqVnBWYTRPKzFwaHZhVjR3?=
 =?utf-8?B?eUgxWkJmdWo5MVZURWF4MDMxbHpycTlLZzNKYTJXVDhVcjhYdVYrelF1RXhE?=
 =?utf-8?B?NUlVa1pYY2ZDN1doTzJ1QWcxazBYYytPTVg2Z2xWNXlRR2JvWVh4K0F4QXZx?=
 =?utf-8?B?TlpwUVhhSzBBT3J0OEo2Wmc5c0ZTdDZlbHlGa1dteWFjVC9SVlkwQUZZSlJk?=
 =?utf-8?B?T21YdnVyMEZoNmVPck5VTllJNlRiSnN2eDFTcVNDcElLM0dTSGFvb3l4MXlR?=
 =?utf-8?B?Rk5WdnNjRjB3VEljdkd5c0ZDdm1XNnZHR09UZE4zbVg2N3pQeWphQUVKc0xj?=
 =?utf-8?B?bUp3UTFzQ1pTY2ZyQUVLdHlIODg1K09tVzQ3M1VRcnV5UmgwZEQ5enB6K201?=
 =?utf-8?B?YlhNSFRkeEF1ei9ud3BrMEtLR3pMeVNNU29NTlR3c0JrQ0xENW9XbzdXUjFZ?=
 =?utf-8?B?SVd1WU1GRUowQXB5NEhjN3g5RHBSeVgzaGY2bmNYeEZFTHllY2NQOFZrSjlO?=
 =?utf-8?B?cnZ5cUswV0NCczd3NGkrQkNzTTdraExwNXVDbTM2S0ZyNndjMzRScjh5UjFv?=
 =?utf-8?B?clJPQkUyN09YaWJHL0daZzRJZUVaeVNBeFBYbzVPcktYUndUR0ZkM202dXh3?=
 =?utf-8?B?b2JYenBHdTFCNzdUTnBPYmVzNEZMa1NoblR4N2d5bkZmcUpyU0ErRVRCQ3Aw?=
 =?utf-8?B?Si9TQzhiSmMwemQrcWVndCtXSlRvMlBDeU12c2p1blRudTIyclY3a1krbE5a?=
 =?utf-8?B?MVVRZUdpU21EOUFtWVFCZE5oa3c1K2Uydm5BQXJpZTUxVklGLzlSaDR3ck9N?=
 =?utf-8?B?NWUyakpYQWdHak4weXlsbGJLZ0tRQ3h0aG85VmtRU1U0R0ZaQmlNZ0xjRFp2?=
 =?utf-8?B?VjAyYnF2SEF5dVpGbWt0WGhqelJOUm13VTl0YU9Za3hJSFVSM1VyS3pkSXRw?=
 =?utf-8?B?OUZHenZlaTVPaVpkMWliZnJNenU1cmFrQzNjeUNUT1NJWmhEbHZwR0llMW0r?=
 =?utf-8?B?cXRkNmJtQW5yVlpHMFN3V0p1ekw3TU9lUFBMZ0Z5dnJXWkprV0ZqS2Zsazht?=
 =?utf-8?B?eHZFU0gzYWFaRVBwN1FVSm12THgzREIvRlBQZmY3bGJ6TnBLVENYTHE3eU1C?=
 =?utf-8?B?RW02Ymc0dDM4UG8wOU80VFB5ZXNHUHZVRDJHZ3FyaVhJbXhIRHdrNUUrSy9z?=
 =?utf-8?B?U3JVWGgybGMwV2RIQTJjQkRtTmFuSDNNZ01vS2lkaHNVbWRtbzVWck5wOHI3?=
 =?utf-8?B?UE40S1dVSklVcm0xci8wRGJGbHhOTGRvN21qbzRDcnlHek9kbEhLYWRFZ3FZ?=
 =?utf-8?B?UXl6VXlCUndLdTJnelJYWDlvTjJtTzNKM2RXMmpxT1FOZGJzNEVuUlVRcTh5?=
 =?utf-8?B?SHc3MTV1T3F0ME1ESStnMEFJcGdKZnRONFA0cVk4a3hmbzBZY01LdUFjRDJ3?=
 =?utf-8?B?MjZSbnJLbUNOcVQvOTlpVS9IdGJqeFZuTnVldy85Z0IvMW1Ib2Noa0VtM0pk?=
 =?utf-8?B?NnhGM3VXN2t5aytQR2ZOREhGV2ZsWlMyS1VyTklIVm9NVEN6SjBURTV1d3hi?=
 =?utf-8?Q?oXj91unr1QsSknBGYJ2P1N6ksTrc06EOkJV0s=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:16:41.3123
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d64b647f-d9aa-401a-e767-08dc7a616b90
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E8.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6449

Hi Luca,

On 22/05/2024 09:51, Luca Fancellu wrote:
> 
> 
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
> 
> This commits implements a new function p2m_put_l2_superpage to handle
> 2MB superpages, specifically for helping put extra references for
> foreign superpages.
> 
> Modify relinquish_p2m_mapping as well to take into account preemption
> when type is foreign memory and order is above 9 (2MB).
> 
> Currently 1GB superpages are not handled because Xen is not preemptible
> and therefore some work is needed to handle such superpages, for which
> at some point Xen might end up freeing memory and therefore for such a
> big mapping it could end up in a very long operation.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:17:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:17:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727614.1132156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqC-00078e-3T; Wed, 22 May 2024 13:17:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727614.1132156; Wed, 22 May 2024 13:17:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqB-00078T-W4; Wed, 22 May 2024 13:17:11 +0000
Received: by outflank-mailman (input) for mailman id 727614;
 Wed, 22 May 2024 13:17:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9lqB-0006aV-2w
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:17:11 +0000
Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com
 [2607:f8b0:4864:20::829])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97a69ef7-183d-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:17:09 +0200 (CEST)
Received: by mail-qt1-x829.google.com with SMTP id
 d75a77b69052e-43f91aac792so5822141cf.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:17:09 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df8308580sm168782291cf.66.2024.05.22.06.17.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:17:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97a69ef7-183d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383828; x=1716988628; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VYqG2PwvCby3wswfH3oW/bedkYG965aa/Kv7xf2kuwk=;
        b=X/VCmprySlTtYlIp7ojMaQIvp8JxJCjjbwZqu2cpJcT4y8lA5TPttjB1vdnrcEhwDG
         NYljR18N66fJKzUFoYc4PAzePngSSrd12MHL2AOwJukKX3nOvgbHQIvgxDtutO/sZ+GQ
         0ntw5mhoxV1jYwigBZ2G0QW3sLnkZH6yiHM9M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383828; x=1716988628;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VYqG2PwvCby3wswfH3oW/bedkYG965aa/Kv7xf2kuwk=;
        b=kGHhNAWP2P69iLgQAaHHqfkZjHgyOnnJP7oOBu6IQUNlK22PFok7HYfRlZ8WX58/yX
         z3oP3GlcG7hJJU3qTUTNvN/lXgku1Nq3XP2WYEOBvBWKCc2Y0wpFdXohIH4u1Dtt8q5V
         qhmu/LhubCM/v7bdaR450mPHUBh5fijc4YIdtQaMyjP3tBSKYD75WNk7kv4djA0Ol3A6
         2IhRd/wiNtNlTUbaFIBn5Fzo6F32MlGfp5+cRLU0mGiDXS1nU1BCthZi6olacRoIW8xZ
         nzK8MKzSilwrk+G9EkwgYdqDLEnFHT13FTooltWa0fb4VR1u2KdPAqkZxr3u8vOoQMiE
         6LiQ==
X-Gm-Message-State: AOJu0Yy8tO1oQ1fjOwzDMvd8A1mLZtft+eNfZrMv4T5ccFV1n+wzrRoe
	UDxSuZ+d9t7NYgU3JDivQUK0gcQ7upMEj/F3tgU510LwwWKzs7g1nIRAUT2prHCzBq7T8YTLUZi
	ijv0=
X-Google-Smtp-Source: AGHT+IG7SzZjtJqMXc/0TJMsERyvpNvdjs1EKGm7srM0lFiRQ6FylYoUa4ZubQkO02MjAatJ0EJ8CA==
X-Received: by 2002:ac8:57c5:0:b0:43c:7755:961c with SMTP id d75a77b69052e-43f7a28f061mr206279971cf.5.1716383827910;
        Wed, 22 May 2024 06:17:07 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v2 0/4] x86/shadow: Trace fixes and cleanup
Date: Wed, 22 May 2024 14:16:59 +0100
Message-Id: <20240522131703.30839-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: 5133a01d-346d-4779-b319-a156ef944669@suse.com
References: 
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Patches 1-3 new, following reivew of Jan's bugfix (patch 4)

Andrew Cooper (3):
  x86/shadow: Rework trace_shadow_gen() into sh_trace_va()
  x86/shadow: Introduce sh_trace_gl1e_va()
  x86/shadow: Rework trace_shadow_emulate_other() as sh_trace_gfn_va()

Jan Beulich (1):
  x86/shadow: Don't leave trace record field uninitialized

 xen/arch/x86/mm/shadow/multi.c | 150 ++++++++++++++-------------------
 1 file changed, 63 insertions(+), 87 deletions(-)


base-commit: 9c5444b01ad51369bc09197a442a93d87b4b76f2
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:17:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:17:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727615.1132166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqD-0007Oy-9r; Wed, 22 May 2024 13:17:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727615.1132166; Wed, 22 May 2024 13:17:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqD-0007Op-6W; Wed, 22 May 2024 13:17:13 +0000
Received: by outflank-mailman (input) for mailman id 727615;
 Wed, 22 May 2024 13:17:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9lqB-00078O-W5
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:17:11 +0000
Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com
 [2607:f8b0:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 989a55e4-183d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:17:11 +0200 (CEST)
Received: by mail-il1-x12c.google.com with SMTP id
 e9e14a558f8ab-36dc410c051so23575535ab.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:17:11 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df8308580sm168782291cf.66.2024.05.22.06.17.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:17:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 989a55e4-183d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383829; x=1716988629; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3W9egxiDkbfNPkqbiYstXvWaWDLlMru2YaLPYeMLJk4=;
        b=Eey/lI36tR/Hw9I3ReTw348HbWmFKf1OAoYmRsRv6C+KDRmywg9x93q7jAsPiUyC/3
         WBSdx/e9ulXeTDRNxdcefLuIChiHYSCXKcyEoVp4LjSP3GZTAEWYYjV78UXrpwZN5YWP
         3QbGdTeD7PtUzUyuYU2I4yh9EABQMLtFuTJPY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383829; x=1716988629;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3W9egxiDkbfNPkqbiYstXvWaWDLlMru2YaLPYeMLJk4=;
        b=wks4rv/mVW2yeCZu6H2407IQs4iCy1JtoBeqW2RjI3aG3kKieEhipY9d7Gmmn75Ols
         tcIFEVmmJH87b1KnVW48LycShx6+f7mxr+T+hC7Od2AKBmVqb4W2KeeFBun4UbgW0S90
         A8ZTTVAM3EK4JSN5jhuLlv7vn/I5C7qLzciYXGgGWzVsy9Jsg4Kro0Q3xzGsf4MwI7wH
         yqoDFMSC03Ebe/mYGHmlWvTttquO+veiMMG9ucxRIL9hfTBNp42zbqBSIYXfb11En2kA
         rUCIB1XCxxEyVU898fqkhHIPgGI3WwQs30s0dLCgFaHzN68QD8Y1/3zRcYBzMxP7lMDr
         MBjw==
X-Gm-Message-State: AOJu0YyYyHHyNydMfJu5+NqVxJPnOYs9YpjANC4kPKQMhInWY7fYKVW7
	hraWGk0Xs9Nx/DfqLMn1nnmhRqrU6RVE+hTB2Trz/BPnsF7qQMJEn+cTjDWGxVcYYagx/FG4VCO
	EkXc=
X-Google-Smtp-Source: AGHT+IFOsCZnn3ppdQFR01k/LArnvWpuwcO8Lm9Aw5tWCalNjdTDf2rT+KGQTsBatgJy6F4Dp1WTHg==
X-Received: by 2002:a05:6e02:1a2a:b0:36b:1e1:552f with SMTP id e9e14a558f8ab-371faf56c05mr22422715ab.23.1716383829497;
        Wed, 22 May 2024 06:17:09 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH v2 1/4] x86/shadow: Rework trace_shadow_gen() into sh_trace_va()
Date: Wed, 22 May 2024 14:17:00 +0100
Message-Id: <20240522131703.30839-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522131703.30839-1-andrew.cooper3@citrix.com>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The ((GUEST_PAGING_LEVELS - 2) << 8) expression in the event field is common
to all shadow trace events, so introduce sh_trace() as a very thin wrapper
around trace().

Then, rename trace_shadow_gen() to sh_trace_va() to better describe what it is
doing, and to be more consistent with later cleanup.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>

v2:
 * New
---
 xen/arch/x86/mm/shadow/multi.c | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index bcd02b2d0037..1775952d7e18 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -1974,13 +1974,17 @@ typedef u32 guest_va_t;
 typedef u32 guest_pa_t;
 #endif
 
-static inline void trace_shadow_gen(u32 event, guest_va_t va)
+/* Shadow trace event with GUEST_PAGING_LEVELS folded into the event field. */
+static void sh_trace(uint32_t event, unsigned int extra, const void *extra_data)
+{
+    trace(event | ((GUEST_PAGING_LEVELS - 2) << 8), extra, extra_data);
+}
+
+/* Shadow trace event with the guest's linear address. */
+static void sh_trace_va(uint32_t event, guest_va_t va)
 {
     if ( tb_init_done )
-    {
-        event |= (GUEST_PAGING_LEVELS-2)<<8;
-        trace(event, sizeof(va), &va);
-    }
+        sh_trace(event, sizeof(va), &va);
 }
 
 static inline void trace_shadow_fixup(guest_l1e_t gl1e,
@@ -2239,7 +2243,7 @@ static int cf_check sh_page_fault(
                 sh_reset_early_unshadow(v);
                 perfc_incr(shadow_fault_fast_gnp);
                 SHADOW_PRINTK("fast path not-present\n");
-                trace_shadow_gen(TRC_SHADOW_FAST_PROPAGATE, va);
+                sh_trace_va(TRC_SHADOW_FAST_PROPAGATE, va);
                 return 0;
             }
 #ifdef CONFIG_HVM
@@ -2250,7 +2254,7 @@ static int cf_check sh_page_fault(
             perfc_incr(shadow_fault_fast_mmio);
             SHADOW_PRINTK("fast path mmio %#"PRIpaddr"\n", gpa);
             sh_reset_early_unshadow(v);
-            trace_shadow_gen(TRC_SHADOW_FAST_MMIO, va);
+            sh_trace_va(TRC_SHADOW_FAST_MMIO, va);
             return handle_mmio_with_translation(va, gpa >> PAGE_SHIFT, access)
                    ? EXCRET_fault_fixed : 0;
 #else
@@ -2265,7 +2269,7 @@ static int cf_check sh_page_fault(
              * Retry and let the hardware give us the right fault next time. */
             perfc_incr(shadow_fault_fast_fail);
             SHADOW_PRINTK("fast path false alarm!\n");
-            trace_shadow_gen(TRC_SHADOW_FALSE_FAST_PATH, va);
+            sh_trace_va(TRC_SHADOW_FALSE_FAST_PATH, va);
             return EXCRET_fault_fixed;
         }
     }
@@ -2481,7 +2485,7 @@ static int cf_check sh_page_fault(
 #endif
         paging_unlock(d);
         put_gfn(d, gfn_x(gfn));
-        trace_shadow_gen(TRC_SHADOW_DOMF_DYING, va);
+        sh_trace_va(TRC_SHADOW_DOMF_DYING, va);
         return 0;
     }
 
@@ -2569,7 +2573,7 @@ static int cf_check sh_page_fault(
         put_gfn(d, gfn_x(gfn));
 
         perfc_incr(shadow_fault_mmio);
-        trace_shadow_gen(TRC_SHADOW_MMIO, va);
+        sh_trace_va(TRC_SHADOW_MMIO, va);
 
         return handle_mmio_with_translation(va, gpa >> PAGE_SHIFT, access)
                ? EXCRET_fault_fixed : 0;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:17:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:17:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727616.1132176 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqF-0007iM-I7; Wed, 22 May 2024 13:17:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727616.1132176; Wed, 22 May 2024 13:17:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqF-0007iF-FE; Wed, 22 May 2024 13:17:15 +0000
Received: by outflank-mailman (input) for mailman id 727616;
 Wed, 22 May 2024 13:17:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9lqD-00078O-9R
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:17:13 +0000
Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com
 [2607:f8b0:4864:20::832])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 999cc691-183d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:17:12 +0200 (CEST)
Received: by mail-qt1-x832.google.com with SMTP id
 d75a77b69052e-43e0d564136so7661601cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:17:12 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df8308580sm168782291cf.66.2024.05.22.06.17.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:17:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 999cc691-183d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383831; x=1716988631; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dUgjIaJwGOveqC8f3Hnlrm+bkXv3Lg6iNL+SZwb8N6Q=;
        b=V4h8tgLYHMop/DKO1BRIOprCKj0pyhJ5pK7ffsWc5tf/JHY2v7rdIWQ3xd+v2UZ8Nd
         DzwqZmm+cFayVRF8VjUqi4xDzImgKRyZYswdkl0qWTEO537U4lXsipalOlScHQqNh9GX
         QGwyj4QFS56aHw+jyXKrbstop3ZKJPWiAxQlA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383831; x=1716988631;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dUgjIaJwGOveqC8f3Hnlrm+bkXv3Lg6iNL+SZwb8N6Q=;
        b=XBP6DtBHMk+nXSOP1lqtJ0KlbCRIE6KVRbVAfZGRHkynvpHgdXKAAsQqg2r+to54X1
         ccdzROXwWL711OkGGeM0Sv/FVYHbVw07spC09XSWuCWoH5n530cEquJQo504+3ljO9q2
         heoA3qpK9+eRWZ80YFU2lhUF9LGTOwy7qjQWEDaK7YaVqlknuviG32eqP06A8ofXZ6wC
         U/YsXU9CUdqRwKDiUEAl4mH7Hr2e8zfqFdaapoDadqHiNerXWPeJ6nqRc9wINtn/YnZk
         o4qV7CtaqQWHLANJfU1sZWBNo/q1buS1hOhT19N1ZNFDhNHsNR8PgUQggdLUgqjrRoIz
         WWHg==
X-Gm-Message-State: AOJu0YwR9EXav6bA+min5XC4udOac5RFYKjra0uzpOpgYrQMNzvjaQ1y
	+9A3don/AuRM+IqjUQSrhBDrB02FC+f2yVk3XL6WYdpMeOFf7weDPDTJGvVFIgh3CHcCN9A8w6Z
	dntY=
X-Google-Smtp-Source: AGHT+IE058XLU1M59pKmCtXLyedictXtR2d3bLBkWZ1fCtR64oB4lj/5qgOU1e77TQNhYbhybjhYnA==
X-Received: by 2002:ac8:5e49:0:b0:43a:ecfb:8a5c with SMTP id d75a77b69052e-43f9e124359mr28617561cf.40.1716383831009;
        Wed, 22 May 2024 06:17:11 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH v2 2/4] x86/shadow: Introduce sh_trace_gl1e_va()
Date: Wed, 22 May 2024 14:17:01 +0100
Message-Id: <20240522131703.30839-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522131703.30839-1-andrew.cooper3@citrix.com>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

trace_shadow_fixup() and trace_not_shadow_fault() both write out identical
trace records.  Reimplement them in terms of a common sh_trace_gl1e_va().

There's no need to pack the trace record, even in the case of PAE paging.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>

v2:
 * New
---
 xen/arch/x86/mm/shadow/multi.c | 59 ++++++++++------------------------
 1 file changed, 17 insertions(+), 42 deletions(-)

diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 1775952d7e18..f0a9cc527c0b 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -1987,51 +1987,26 @@ static void sh_trace_va(uint32_t event, guest_va_t va)
         sh_trace(event, sizeof(va), &va);
 }
 
-static inline void trace_shadow_fixup(guest_l1e_t gl1e,
-                                      guest_va_t va)
+/* Shadow trace event with a gl1e, linear address and flags. */
+static void sh_trace_gl1e_va(uint32_t event, guest_l1e_t gl1e, guest_va_t va)
 {
     if ( tb_init_done )
     {
-        struct __packed {
-            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
-               so put it first for alignment sake. */
-            guest_l1e_t gl1e;
-            guest_va_t va;
-            u32 flags;
-        } d;
-        u32 event;
-
-        event = TRC_SHADOW_FIXUP | ((GUEST_PAGING_LEVELS-2)<<8);
-
-        d.gl1e = gl1e;
-        d.va = va;
-        d.flags = this_cpu(trace_shadow_path_flags);
-
-        trace(event, sizeof(d), &d);
-    }
-}
-
-static inline void trace_not_shadow_fault(guest_l1e_t gl1e,
-                                          guest_va_t va)
-{
-    if ( tb_init_done )
-    {
-        struct __packed {
-            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
-               so put it first for alignment sake. */
+        struct {
+            /*
+             * For GUEST_PAGING_LEVELS=3 (PAE paging), guest_l1e is 64 while
+             * guest_va is 32.  Put it first to avoid padding.
+             */
             guest_l1e_t gl1e;
             guest_va_t va;
-            u32 flags;
-        } d;
-        u32 event;
-
-        event = TRC_SHADOW_NOT_SHADOW | ((GUEST_PAGING_LEVELS-2)<<8);
-
-        d.gl1e = gl1e;
-        d.va = va;
-        d.flags = this_cpu(trace_shadow_path_flags);
-
-        trace(event, sizeof(d), &d);
+            uint32_t flags;
+        } d = {
+            .gl1e = gl1e,
+            .va = va,
+            .flags = this_cpu(trace_shadow_path_flags),
+        };
+
+        sh_trace(event, sizeof(d), &d);
     }
 }
 
@@ -2603,7 +2578,7 @@ static int cf_check sh_page_fault(
     d->arch.paging.log_dirty.fault_count++;
     sh_reset_early_unshadow(v);
 
-    trace_shadow_fixup(gw.l1e, va);
+    sh_trace_gl1e_va(TRC_SHADOW_FIXUP, gw.l1e, va);
  done: __maybe_unused;
     sh_audit_gw(v, &gw);
     SHADOW_PRINTK("fixed\n");
@@ -2857,7 +2832,7 @@ static int cf_check sh_page_fault(
     put_gfn(d, gfn_x(gfn));
 
 propagate:
-    trace_not_shadow_fault(gw.l1e, va);
+    sh_trace_gl1e_va(TRC_SHADOW_NOT_SHADOW, gw.l1e, va);
 
     return 0;
 }
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:17:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:17:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727617.1132186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqG-000804-Vt; Wed, 22 May 2024 13:17:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727617.1132186; Wed, 22 May 2024 13:17:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqG-0007zt-SG; Wed, 22 May 2024 13:17:16 +0000
Received: by outflank-mailman (input) for mailman id 727617;
 Wed, 22 May 2024 13:17:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9lqF-0006aV-NS
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:17:15 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9a689107-183d-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:17:14 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-43f91aac792so5822751cf.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:17:14 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df8308580sm168782291cf.66.2024.05.22.06.17.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:17:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a689107-183d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383832; x=1716988632; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oUlYslBBFz8SJFw2HFYr2U4IW4XAKwPKEvjfJVvwvsQ=;
        b=j1fHsoszGxka524cwGAFbgs2MRpdTBw+DZlVvfJw1OJT+j+y612ZV9xcYZJyn+ixyP
         Kg7ryvhWPeB6EHsNzltpY6qoVMKw/r5gw2Q6DVU/o70NsM6VSzdce2c4TS3m8qrlZMSm
         Yf0/+wgEKzHI9YRLln9ZHLNrTThNQiApp6Ggc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383832; x=1716988632;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oUlYslBBFz8SJFw2HFYr2U4IW4XAKwPKEvjfJVvwvsQ=;
        b=mxlzf9j7L/CUYb9ZXxENmtYvxSmw7qh5Kh+aIH8Ko9F1R0n3AtNpNJDpNrUXBRpWhd
         eixcjpWjnNZC3Bw6/9R3kwd6OPOlAGHMbn8kIjgepnAPg+ena7+xl21BU8OT/yOHd95V
         D25j0CQNmcYdrQOp7Koj0EMZ/okT13AMOqY8kyn8xUC6ZB4ka+IBzsWKhTOLVr+JZs+i
         +RT+KA1IQEMAVAcxu2eZxmmCQulTOSXDvtNF/RCOtU0C8gNxSVYtTjq/ZltWOdtdk0jI
         sQWv/Kbd3dNkVUl9rZthZx9CbQGKrIWUXY2a7c59s/izbKyuQDe9KacaYZJ80sYhisZS
         Azeg==
X-Gm-Message-State: AOJu0Yze0GC9Ufr8GIgwi2IpOT6sIcfmuupiQsl5QTA1tKuDSEuAW6WL
	l1IO5Zlbg9xoPhBDzIddVqMDjBv/r3FN6ETZ7QNwLHmPWlUuzz0iVm6jeHXPQxaJy06g/wbmetc
	/UT8=
X-Google-Smtp-Source: AGHT+IHgcNFz0cHXAnovqybxYLcIlXH3BYwKJQGW2bzSQcH9gB41flGGy3toNyocqY82GQ5X3vsOTw==
X-Received: by 2002:ac8:598e:0:b0:43a:3fce:32c2 with SMTP id d75a77b69052e-43f7a28dfd5mr195215021cf.3.1716383832534;
        Wed, 22 May 2024 06:17:12 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH v2 3/4] x86/shadow: Rework trace_shadow_emulate_other() as sh_trace_gfn_va()
Date: Wed, 22 May 2024 14:17:02 +0100
Message-Id: <20240522131703.30839-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522131703.30839-1-andrew.cooper3@citrix.com>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

sh_trace_gfn_va() is very similar to sh_trace_gl1e_va(), and a rather shorter
name than trace_shadow_emulate_other().  Like sh_trace_gl1e_va(), there is no
need to pack the trace record.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>

v2:
 * New
---
 xen/arch/x86/mm/shadow/multi.c | 40 ++++++++++++++++------------------
 1 file changed, 19 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index f0a9cc527c0b..d2fe4e148fe0 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2010,29 +2010,30 @@ static void sh_trace_gl1e_va(uint32_t event, guest_l1e_t gl1e, guest_va_t va)
     }
 }
 
-static inline void trace_shadow_emulate_other(u32 event,
-                                                 guest_va_t va,
-                                                 gfn_t gfn)
+/* Shadow trace event with a gfn, linear address and flags. */
+static void sh_trace_gfn_va(uint32_t event, gfn_t gfn, guest_va_t va)
 {
     if ( tb_init_done )
     {
-        struct __packed {
-            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
-               so put it first for alignment sake. */
+        struct {
+            /*
+             * For GUEST_PAGING_LEVELS=3 (PAE paging), gfn is 64 while
+             * guest_va is 32.  Put it first to avoid padding.
+             */
 #if GUEST_PAGING_LEVELS == 2
-            u32 gfn;
+            uint32_t gfn;
 #else
-            u64 gfn;
+            uint64_t gfn;
 #endif
             guest_va_t va;
-        } d;
-
-        event |= ((GUEST_PAGING_LEVELS-2)<<8);
-
-        d.gfn=gfn_x(gfn);
-        d.va = va;
+            uint32_t flags;
+        } d = {
+            .gfn = gfn_x(gfn),
+            .va = va,
+            .flags = this_cpu(trace_shadow_path_flags),
+        };
 
-        trace(event, sizeof(d), &d);
+        sh_trace(event, sizeof(d), &d);
     }
 }
 
@@ -2603,8 +2604,7 @@ static int cf_check sh_page_fault(
                       mfn_x(gmfn));
         perfc_incr(shadow_fault_emulate_failed);
         shadow_remove_all_shadows(d, gmfn);
-        trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_USER,
-                                      va, gfn);
+        sh_trace_gfn_va(TRC_SHADOW_EMULATE_UNSHADOW_USER, gfn, va);
         goto done;
     }
 
@@ -2683,8 +2683,7 @@ static int cf_check sh_page_fault(
         }
 #endif
         shadow_remove_all_shadows(d, gmfn);
-        trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_EVTINJ,
-                                   va, gfn);
+        sh_trace_gfn_va(TRC_SHADOW_EMULATE_UNSHADOW_EVTINJ, gfn, va);
         return EXCRET_fault_fixed;
     }
 
@@ -2739,8 +2738,7 @@ static int cf_check sh_page_fault(
          * though, this is a hint that this page should not be shadowed. */
         shadow_remove_all_shadows(d, gmfn);
 
-        trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_UNHANDLED,
-                                   va, gfn);
+        sh_trace_gfn_va(TRC_SHADOW_EMULATE_UNSHADOW_UNHANDLED, gfn, va);
         goto emulate_done;
     }
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:17:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:17:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727619.1132196 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqI-0008HN-8l; Wed, 22 May 2024 13:17:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727619.1132196; Wed, 22 May 2024 13:17:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lqI-0008HG-4Y; Wed, 22 May 2024 13:17:18 +0000
Received: by outflank-mailman (input) for mailman id 727619;
 Wed, 22 May 2024 13:17:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9lqG-00078O-GN
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:17:16 +0000
Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com
 [2607:f8b0:4864:20::82b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b8a0971-183d-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:17:16 +0200 (CEST)
Received: by mail-qt1-x82b.google.com with SMTP id
 d75a77b69052e-43df23e034cso8450601cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:17:15 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43df8308580sm168782291cf.66.2024.05.22.06.17.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 06:17:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b8a0971-183d-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716383834; x=1716988634; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SKZ+hrKbmIX0HLzcZm1/QsXM9UY491uFgrKkXcJYF8Y=;
        b=b5ixs+yXSJvIAMvV6U0JtXbWd4UiuYeQCP2XjgRiHSV93S910EES0Sh4LO917RwJeK
         gXsfXq005oziqc4nEa4zm6GEUlR30Kwj3ZpY0huD3WjN++jRREGUJMIM33/6CX9ZUdm1
         Lm1x/rGiSxj/V21kVaI7lp+QtwM1vVbjQQ/A8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716383834; x=1716988634;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SKZ+hrKbmIX0HLzcZm1/QsXM9UY491uFgrKkXcJYF8Y=;
        b=QeE7x+0n6OSsxH9fxj6pNtTHXHp7ThJGm84Ua6t8GY106uq7bBeLk+pmtC/si51nGx
         JIIojxRJK5H6d9N5jMCv6YFD9JdsxUljq1zYfpIebBMR2D/jc5A30JiqTPuJXmaaJvbr
         c8HOg5RKcGJay83DrjOQ8xTQppXQ73vLdaOW2LLX1e8ZAyF8ZGDM6TxrucYxPnqnwoAJ
         F3KDoHd0ZwsIJ9UIZ414DiPAQ1MwvEnyMH/RUcsYxUbAH5eiBMBzInwKQToN329vQ28K
         UfUSTnBts3CImt8ra04RKJeVnJ/CNST/HyoNInimox2sbtvdRMbnlTuW7CVjYcb/bKwX
         OZiw==
X-Gm-Message-State: AOJu0YyEZfy5vr+ed1XvOIFeOmyFBaz7yFbTW1cgU77FHE21AK5VYq5R
	4qpmv/mgpwpLXYRiZ84JR/8JmvfIqnbd35arqW4TDnv0eHL39UipczhsDHahmwLzeaxSy46SPJB
	8Awc=
X-Google-Smtp-Source: AGHT+IElu8z/Eu8w8c3oL9QCc/0cb9N1XmB+dL6SmIf139Vn5/gJPiYk1pYs35eiOA29ZFkoAO+0Tw==
X-Received: by 2002:a05:622a:4:b0:43a:7b64:2fcc with SMTP id d75a77b69052e-43f7a29127dmr242246561cf.7.1716383834375;
        Wed, 22 May 2024 06:17:14 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: [PATCH v2 4/4] x86/shadow: Don't leave trace record field uninitialized
Date: Wed, 22 May 2024 14:17:03 +0100
Message-Id: <20240522131703.30839-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240522131703.30839-1-andrew.cooper3@citrix.com>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

The emulation_count field is set only conditionally right now. Convert
all field setting to an initializer, thus guaranteeing that field to be
set to 0 (default initialized) when GUEST_PAGING_LEVELS != 3.

Rework trace_shadow_emulate() to be consistent with the other trace helpers.

Coverity-ID: 1598430
Fixes: 9a86ac1aa3d2 ("xentrace 5/7: Additional tracing for the shadow code")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>

v2:
 * Rebase over packing/sh_trace() cleanup.
---
 xen/arch/x86/mm/shadow/multi.c | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index d2fe4e148fe0..47dd1cc626b2 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2063,30 +2063,29 @@ static void cf_check trace_emulate_write_val(
 #endif
 }
 
-static inline void trace_shadow_emulate(guest_l1e_t gl1e, unsigned long va)
+static inline void sh_trace_emulate(guest_l1e_t gl1e, unsigned long va)
 {
     if ( tb_init_done )
     {
-        struct __packed {
-            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
-               so put it first for alignment sake. */
+        struct {
+            /*
+             * For GUEST_PAGING_LEVELS=3 (PAE paging), guest_l1e is 64 while
+             * guest_va is 32.  Put it first to avoid padding.
+             */
             guest_l1e_t gl1e, write_val;
             guest_va_t va;
             uint32_t flags:29, emulation_count:3;
-        } d;
-        u32 event;
-
-        event = TRC_SHADOW_EMULATE | ((GUEST_PAGING_LEVELS-2)<<8);
-
-        d.gl1e = gl1e;
-        d.write_val.l1 = this_cpu(trace_emulate_write_val);
-        d.va = va;
+        } d = {
+            .gl1e = gl1e,
+            .write_val.l1 = this_cpu(trace_emulate_write_val),
+            .va = va,
 #if GUEST_PAGING_LEVELS == 3
-        d.emulation_count = this_cpu(trace_extra_emulation_count);
+            .emulation_count = this_cpu(trace_extra_emulation_count),
 #endif
-        d.flags = this_cpu(trace_shadow_path_flags);
+            .flags = this_cpu(trace_shadow_path_flags),
+        };
 
-        trace(event, sizeof(d), &d);
+        sh_trace(TRC_SHADOW_EMULATE, sizeof(d), &d);
     }
 }
 #endif /* CONFIG_HVM */
@@ -2815,7 +2814,7 @@ static int cf_check sh_page_fault(
     }
 #endif /* PAE guest */
 
-    trace_shadow_emulate(gw.l1e, va);
+    sh_trace_emulate(gw.l1e, va);
  emulate_done:
     SHADOW_PRINTK("emulated\n");
     return EXCRET_fault_fixed;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:20:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:20:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727635.1132206 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lt6-0002uD-N5; Wed, 22 May 2024 13:20:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727635.1132206; Wed, 22 May 2024 13:20:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lt6-0002u6-KC; Wed, 22 May 2024 13:20:12 +0000
Received: by outflank-mailman (input) for mailman id 727635;
 Wed, 22 May 2024 13:20:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DzQV=MZ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s9lt5-0002sc-10
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:20:11 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02de5ed5-183e-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:20:09 +0200 (CEST)
Received: from DM6PR18CA0025.namprd18.prod.outlook.com (2603:10b6:5:15b::38)
 by MW4PR12MB5642.namprd12.prod.outlook.com (2603:10b6:303:187::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 13:20:05 +0000
Received: from CY4PEPF0000E9D6.namprd05.prod.outlook.com
 (2603:10b6:5:15b:cafe::4b) by DM6PR18CA0025.outlook.office365.com
 (2603:10b6:5:15b::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Wed, 22 May 2024 13:20:04 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D6.mail.protection.outlook.com (10.167.241.69) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 13:20:04 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:20:04 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:20:03 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 08:20:02 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02de5ed5-183e-11ef-90a0-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T++K8TuLf8+kKlPCIGYoDDpxLlk0txDOpSrH6tkWvSsoxjUUDiOlwLTYkJEyNBlYk0jm9kYQMJMu9I5ZVPlVVsm/8kW8OKPFXgbiAb1lt12oNJNlH9RLpYJQdg94wW3dQHlPbStsZpF+WH2pmT/K1sX/F9RsITJpVgCSB5lJ4SDwAD2MLNDx+vt4IppD+Bt07hTUdsgjcxtlrDSBIgylyoVHEIl/mQK8H0nT0C5MmFzSUldoDt77ONchLC1y+VHD+FppxoluWKNM06Ck9iiP7QLCWcD6LBpjEnEZRPDqZ8C27LOqyNmkwErKIkFUUvX3y9/abQqlS7Q7COiqfc4kpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EO6KUCu3JGuzHIVuVIM4KDT+40PD1yt4ONaC+056r/Q=;
 b=awrdrld6ja5ztkDiI5Y0TcTAKW/kN0AFEYFwBUTYeZd4yy470twnlCRiPoucsltVBwDfSTifMAJMzv7KE0WME6+HyNj2IyugZwQH+lO1bg++wvsBdV8HhZhel+6gMOes47Y9J5fjT2N8uOzLl61uBOK5X4WTkBkIcqdAWd1WHAF/6CPfXRchUGjX/XWkBDolrFo6vnyC7B46oHmF3jk2wrkohYkEy+jvDfmt0pMZGzeUnND941yUHvQi7CSRWuZ5J4PGc1bO/P8P0f7HmIJlHPf/gFyWoxvi8Ye3fRcNpRKdxOmgu2Oowzfymxtpv0BTjZ8N0E4WpdHc4Lkytu8GSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EO6KUCu3JGuzHIVuVIM4KDT+40PD1yt4ONaC+056r/Q=;
 b=n30hk+LbIc0yBUqisXM61wOtU5CkBP4nQZ5zf1qhuTj6TKk263+gH1V8izz9REYAGeIe6vowQvBBrdQcR4Ww6wHJtkdYC9a/7FTGm+JD/TK7hQ0QkPWxkeuSS13gLe4/L7RWssnR6D0GTPbfysulhxojku94pPfQVMnlhDVQ/EU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <6ea06251-4e47-426a-aa7a-ef951100e8cf@amd.com>
Date: Wed, 22 May 2024 15:19:57 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 6/7] xen/arm: Implement the logic for static shared
 memory from Xen heap
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-7-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240522075151.3373899-7-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D6:EE_|MW4PR12MB5642:EE_
X-MS-Office365-Filtering-Correlation-Id: e3ff5888-8a79-496b-9557-08dc7a61e4a6
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?djVvcFpTTVBhYytaQmFQTUg3L1pDWjl0SDM0SnRGTW43RzdYbUh1VXhjRlQ5?=
 =?utf-8?B?b29nWTRDU1BUVTR6R09oc2NOMndLVGV6RHVQRmN1YVNBR1hxZGRqcVUxcEc1?=
 =?utf-8?B?a1NvWmViRnZ6TEdiNXFYQjc0ZVV6dGRRMlRuSDdobERrODVGRVpmNGg4K3ha?=
 =?utf-8?B?YUFwVERQZ3VJYy8zOTY4bzd2KzhHcU9BZC9oSkZJNTRFTncyL254NGJZaUw1?=
 =?utf-8?B?c0M3cTlmdVBIdU5tbkpPYmw2RXBSdzFQWURPSDNVTWNmU2ZjUnF3empyekxv?=
 =?utf-8?B?QUxpNGNqU1doUDR0VUJPbkZsNFV0Wlp6NGZzc0JkN0tjbTRtZXZnOUpWQlpN?=
 =?utf-8?B?M0g1L25RSTJMVEdGTDNaaWVNSHUwVUwzUHppcHpJS3FCSS9jYVgrOTg5djVq?=
 =?utf-8?B?VzNyQ25BSWdETkhSdXpEbHpPenAzZlFFbU5BTUJFalV2a2RQU01CUnBiRTd3?=
 =?utf-8?B?ckNwcVg0NlJqMWRRN0JZc1FZWmVFb0ZOR3dLVEhzdmwyNnFsNkZVZ09xVXBh?=
 =?utf-8?B?VXkvd3B5M3NWVVY1NlR6WjM5UDZiNHJRY2Y1aGZ6M2V0dndVT0p6RlV2cmhy?=
 =?utf-8?B?ZU5NelVMalBVdnRSR3FHOWQ1ZC9sc1lmdFZwQzh4QjNSTk1BVkFFTk5DNGFI?=
 =?utf-8?B?d1ZyZHVqR1pvQUxieUNUZGhLb0dnS2VkdERrcGF4TEpNTXNKcEpoYld5Si9O?=
 =?utf-8?B?WDhzeHNWUENLZHZCcjhGZlY3TTQzSTBXKzZUaDBOTHhxeXF4ZUI2YVF6VVJL?=
 =?utf-8?B?RjJGVCtGSm9sdW9EREtoeDdMczAzdC9MTVhIeGVqUXNNcTVxbkY1aGVHMDJH?=
 =?utf-8?B?ZjJEak01WlpsSWpNTncvRndxY1dWOElKS1dFR09PbEk5QkhiZmZKcFQrL004?=
 =?utf-8?B?Z1JaZmR2RGh6aS9ZVGVkdURuWkJKb1FITExEVHdNNW5ncTEyY0FjeGNwU25r?=
 =?utf-8?B?TmxYZ1dVU2wyUnF0U3RCc3ExeENUTGloUDNFYkdRL2MxRHpubUxTZHZhbzZV?=
 =?utf-8?B?MjBqOWsyL3Rhais5MThHRVowSnJIVGkvaUdrSHVOakhSODRERXR2OTF6K0Np?=
 =?utf-8?B?RFE4emNHbG9hejg5eWQyR3ozN1FjK3RZRmtEbHBkckFOMkZiWnlUUmRXRjZk?=
 =?utf-8?B?amthZW5LZ3oxaGpHUGJ6SUNSaU9Fc3VkN0pjeTBEUnlGeThvR1BCU256aU9M?=
 =?utf-8?B?VnlHeWJ6VU1UUmYwRWR6a0gyQVNOZzZmNk5YM011QWw3cXQra1lZRDhLNGpz?=
 =?utf-8?B?blgwWjNEYVhBNFlhWTAzNlF6NkNPNnJsTjFpSWQ5Ujd1MWxHakhtcU9yY0U2?=
 =?utf-8?B?dXlhUzdJRkdqdm9FWDVBdm1wTExkb3pBcktTbDI3OUlMa3JKRXFQbm0rdXdp?=
 =?utf-8?B?c3drQ2JISHJqZXJsOWk5SThRMVFnUmE2dVZCOExOL0xzR3NDWWh2aEVMT0hx?=
 =?utf-8?B?ajZ2WWdWSDJMbFY2TkNYZ1AzVFdYd0QvcnROTEJaVlhVOUNqRzdpcjFHODVr?=
 =?utf-8?B?OWY3TTJNVEJTdUdWcmd6N0Z4T2hncGsxbnBRVHhLM3c1UTZNKzNlaTFNSHBo?=
 =?utf-8?B?a3RYeUFzbXVxSGF1WGlNMGNpYTlWQitCd1Y3eUlQNUtkTnJjSktkdnNaWXJX?=
 =?utf-8?B?NnlyK3FyL1dPVGZ1M2xQNEc3MCtvOWNKMHdiTmdub3NMVVBYdXJCME10OTZm?=
 =?utf-8?B?Uy9DWXlSdkNqZnRtMWFnSTVmeUp5S0UyNEhQcjBzRnZkVm50NkRBMlRMYTdY?=
 =?utf-8?B?cVhFRnV0a2toNmxtZjNJZ1Z5b3l0Y3pnVGVVVGF0Y3lkYk1yYll4eGJ5bE1k?=
 =?utf-8?Q?n4QUV7mP9V2YCPP0uQoae0sX0bNhM54RQN5HU=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:20:04.5369
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e3ff5888-8a79-496b-9557-08dc7a61e4a6
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D6.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5642

Hi Luca,

On 22/05/2024 09:51, Luca Fancellu wrote:
> 
> 
> This commit implements the logic to have the static shared memory banks
> from the Xen heap instead of having the host physical address passed from
> the user.
> 
> When the host physical address is not supplied, the physical memory is
> taken from the Xen heap using allocate_domheap_memory, the allocation
> needs to occur at the first handled DT node and the allocated banks
> need to be saved somewhere.
> 
> Introduce the 'shm_heap_banks' for that reason, a struct that will hold
> the banks allocated from the heap, its field bank[].shmem_extra will be
> used to point to the bootinfo shared memory banks .shmem_extra space, so
> that there is not further allocation of memory and every bank in
> shm_heap_banks can be safely identified by the shm_id to reconstruct its
> traceability and if it was allocated or not.
> 
> A search into 'shm_heap_banks' will reveal if the banks were allocated
> or not, in case the host address is not passed, and the callback given
> to allocate_domheap_memory will store the banks in the structure and
> map them to the current domain, to do that, some changes to
> acquire_shared_memory_bank are made to let it differentiate if the bank
> is from the heap and if it is, then assign_pages is called for every
> bank.
> 
> When the bank is already allocated, for every bank allocated with the
> corresponding shm_id, handle_shared_mem_bank is called and the mapping
> are done.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:23:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:23:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727644.1132216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lw0-00042a-43; Wed, 22 May 2024 13:23:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727644.1132216; Wed, 22 May 2024 13:23:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lw0-00042T-0v; Wed, 22 May 2024 13:23:12 +0000
Received: by outflank-mailman (input) for mailman id 727644;
 Wed, 22 May 2024 13:23:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9lvy-00042N-OK
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:23:11 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c1eacf1-183e-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:23:07 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfout.west.internal (Postfix) with ESMTP id BDFB61C000A9;
 Wed, 22 May 2024 09:23:03 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Wed, 22 May 2024 09:23:04 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 09:23:02 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c1eacf1-183e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716384183;
	 x=1716470583; bh=FLnl9Hofz7FZpFMb5Naycrq/oqV73WrcuNoNEEYE35s=; b=
	wLFWTTeVdk8/0rA5pvA7DFeMkhK1c0lH04Usye8ZVNPTRemJlrEZcxcUo2kBvRlO
	MYQwYD+1bQDRexzvEBBpL4fwQP+2oqSfuQyrLfrXympu6RBpj1g4+A1es6jC0yNW
	y+VIaqPW5cSIt9PDGHeJGD/ALZ1Ieoe8M8r7Lj7FwwGYu73CwqyIT86anywZQJfP
	BgtiFMw3jklLQ5brhf0WYp+/R6LNkkntYMiNu9cZuocIuheblEV8KGc0ZgGqsslX
	RnQkyZ6/JU/N19WxUAWIjS3rlT00i03BkwoVxBB0KpsnX2EAFDSj/t4vM1bi7j6y
	IncYh7ICKKhKWDWaCK8r1g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716384183; x=1716470583; bh=FLnl9Hofz7FZpFMb5Naycrq/oqV7
	3WrcuNoNEEYE35s=; b=bHKsdIu9LmhswPoTOEhgcIlWpKpV6yjNjOryLGO0qHQa
	NYg9ea5v/m/LX/D+Xlo+VcCtizodeTAKLl9iSwCXAnux4PCvChYT4UiCbsR+at1+
	H8SNAwifEhIyPC0a38EnhX+Os42bbYhbm+odZ2rUxdg0GmGzQxdMY3v2klk4+540
	yb/+BWhFWCIpbKewpVO7+2RLWnEjHTbCSBhNcYGZnE5MwLW6BiZmhO6DO7V4dYD6
	V54QNN1XJj73x5Zs/isSJG5jC8wKLwXEQKi2kfP1PqzMbEz5N6ge813bag+Q1JR9
	nR2p81qLT6mkVu/Z++lzINE095xcRfcbdMMzL9r1zw==
X-ME-Sender: <xms:t_FNZn1d-d7SKFTcnWqbCjcwlCtzi-HmICUbwjMufNyQwFh_aE88aQ>
    <xme:t_FNZmE4XddeSjLm9QlVsCVyMyA7nrZg7vXYyluk8Ocv38frdst3IoBJl7qwqY2P0
    sARY6iv92A-EQ>
X-ME-Received: <xmr:t_FNZn5_JIIryDpqmVlIMrD_bWuVoeXYqF4eD6ZB_jkerwmT38ulgMfCrfym9RjAjK7yvtxTaoolAorRp_U6EZZ3BhIGJc_jRQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgudegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:t_FNZs3oIEVaXLAcm-CApcrRww_2sCHm6zmrUbiaKYbjaNY4Zy11aw>
    <xmx:t_FNZqERa58TdtM0VQs5Jt81_6bf54iPblfdr20QPiuq5qyyL3GFfg>
    <xmx:t_FNZt_Dv3FGNCKx46ariRAM1jEcdQJCwCCM0W4hg0LLgPjaumbLrw>
    <xmx:t_FNZnlR0h4HAQ5f6g_KphZLqQ8LWeAJwT-Seu8h52BOvxYAYCdNPA>
    <xmx:t_FNZmAmQ3rTuUodGVsK9sKSza7xlSE4REA2I_Won7O1GSpdesjkVYT7>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 22 May 2024 15:22:59 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Message-ID: <Zk3xtK0SaaqEadXY@mail-itl>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="zsucm9wcRX2e8Q2A"
Content-Disposition: inline
In-Reply-To: <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>


--zsucm9wcRX2e8Q2A
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 22 May 2024 15:22:59 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only

On Wed, May 22, 2024 at 09:52:44AM +0200, Jan Beulich wrote:
> On 21.05.2024 04:54, Marek Marczykowski-G=C3=B3recki wrote:
> > +static void subpage_mmio_write_emulate(
> > +    mfn_t mfn,
> > +    unsigned int offset,
> > +    const void *data,
> > +    unsigned int len)
> > +{
> > +    struct subpage_ro_range *entry;
> > +    void __iomem *addr;
>=20
> Wouldn't this better be pointer-to-volatile, with ...

Shouldn't then most other uses of __iomem in the code base be this way
too? I see volatile only in few places...

> > +    list_for_each_entry(entry, &subpage_ro_ranges, list)
> > +    {
> > +        if ( mfn_eq(entry->mfn, mfn) )
> > +        {
> > +            if ( test_bit(offset / SUBPAGE_MMIO_RO_ALIGN, entry->ro_qw=
ords) )
> > +            {
> > + write_ignored:
> > +                gprintk(XENLOG_WARNING,
> > +                        "ignoring write to R/O MMIO 0x%"PRI_mfn"%03x l=
en %u\n",
> > +                        mfn_x(mfn), offset, len);
> > +                return;
> > +            }
> > +
> > +            addr =3D subpage_mmio_get_page(entry);
> > +            if ( !addr )
> > +            {
> > +                gprintk(XENLOG_ERR,
> > +                        "Failed to map page for MMIO write at 0x%"PRI_=
mfn"%03x\n",
> > +                        mfn_x(mfn), offset);
> > +                return;
> > +            }
> > +
> > +            switch ( len )
> > +            {
> > +            case 1:
> > +                writeb(*(const uint8_t*)data, addr);
> > +                break;
> > +            case 2:
> > +                writew(*(const uint16_t*)data, addr);
> > +                break;
> > +            case 4:
> > +                writel(*(const uint32_t*)data, addr);
> > +                break;
> > +            case 8:
> > +                writeq(*(const uint64_t*)data, addr);
> > +                break;
>=20
> ... this being how it's written? (If so, volatile suitably carried throug=
h to
> other places as well.)

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--zsucm9wcRX2e8Q2A
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZN8bQACgkQ24/THMrX
1yzSeAf/UwyibLPwFQSpLOLjZ9m8YwaWWkgtYKFyIftZSb9/QTEBg1Q0nXTS9V1G
Ugflun7zINy8H5vtVXe64xw3A+/tQnSW0hejn8Oi52e9oUv0qc2qawSSlfAUpM6B
gvjZlyEMU7JUk5l6QpsTUUceTOTs8Ac4BHvh1KaFTzZJzGwGzQCgdsYTGteJSijT
4cwvKY+gRgRNgpOYNdVEG8pXiFue4huZumGvIDfoDYD9vZT60idMsGKWNIcEv0as
WTMVMOXaV9NBy8C1sAL/XMFA05lIFx5fC4g9izz8soBk7hUe3zTs98z8LRqZ+SXD
bDXtlkPDWHe5+93Axc1DtaHvWNrRYg==
=pan3
-----END PGP SIGNATURE-----

--zsucm9wcRX2e8Q2A--


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:25:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:25:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727649.1132226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lye-0004fE-Jj; Wed, 22 May 2024 13:25:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727649.1132226; Wed, 22 May 2024 13:25:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9lye-0004f7-Gj; Wed, 22 May 2024 13:25:56 +0000
Received: by outflank-mailman (input) for mailman id 727649;
 Wed, 22 May 2024 13:25:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9lyd-0004f1-2C
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:25:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9lyc-0005N7-KQ; Wed, 22 May 2024 13:25:54 +0000
Received: from [15.248.2.235] (helo=[10.24.67.23])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9lyc-0001Sq-Dw; Wed, 22 May 2024 13:25:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VDKSEBUOclnWGgyKYib2zSQlrQbuicW7+yxJnId/su0=; b=deD4V1vGpE0wMVVeHmbcg94zHR
	btOpJkUxSgpikADmV3XnIZeO4I0U8JBwgFiLGx5gtiUDIDnWlUjjts/+ZZgnrZqftFliKQ7uxIbrx
	JcpiAvUjmMBPdqOfQbOlNJqeOuqzmOm8PbxLShKG17Pa1hsvfqkWZmgRWYwEUY96ldj0=;
Message-ID: <c10b2513-b0f1-4042-a538-2115e92408d1@xen.org>
Date: Wed, 22 May 2024 14:25:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
Content-Language: en-GB
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-4-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240522075151.3373899-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Luca,

On 22/05/2024 08:51, Luca Fancellu wrote:
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
> 
> This commits implements a new function p2m_put_l2_superpage to handle
> 2MB superpages, specifically for helping put extra references for
> foreign superpages.
> 
> Modify relinquish_p2m_mapping as well to take into account preemption
> when type is foreign memory and order is above 9 (2MB).
> 
> Currently 1GB superpages are not handled because Xen is not preemptible
> and therefore some work is needed to handle such superpages, for which
> at some point Xen might end up freeing memory and therefore for such a
> big mapping it could end up in a very long operation.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v3:
>   - Add reasoning why we don't support now 1GB superpage, remove level_order
>     variable from p2m_put_l2_superpage, update TODO comment inside
>     p2m_free_entry, use XEN_PT_LEVEL_ORDER(2) instead of value 9 inside
>     relinquish_p2m_mapping. (Michal)
> v2:
>   - Do not handle 1GB super page as there might be some issue where
>     a lot of calls to put_page(...) might be issued which could lead
>     to free memory that is a long operation.
> v1:
>   - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
> ---
>   xen/arch/arm/mmu/p2m.c | 63 ++++++++++++++++++++++++++++++------------
>   1 file changed, 46 insertions(+), 17 deletions(-)
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index 41fcca011cf4..b496266deef6 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>       return rc;
>   }
>   
> -/*
> - * Put any references on the single 4K page referenced by pte.
> - * TODO: Handle superpages, for now we only take special references for leaf
> - * pages (specifically foreign ones, which can't be super mapped today).
> - */
> -static void p2m_put_l3_page(const lpae_t pte)
> +/* Put any references on the single 4K page referenced by mfn. */
> +static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
>   {
> -    mfn_t mfn = lpae_get_mfn(pte);
> -
> -    ASSERT(p2m_is_valid(pte));
> -
>       /*
>        * TODO: Handle other p2m types
>        *
> @@ -771,16 +763,43 @@ static void p2m_put_l3_page(const lpae_t pte)
>        * flush the TLBs if the page is reallocated before the end of
>        * this loop.
>        */
> -    if ( p2m_is_foreign(pte.p2m.type) )
> +    if ( p2m_is_foreign(type) )
>       {
>           ASSERT(mfn_valid(mfn));
>           put_page(mfn_to_page(mfn));
>       }
>       /* Detect the xenheap page and mark the stored GFN as invalid. */
> -    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
> +    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
>           page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
>   }

All the pages within a 2MB mapping should be the same type. So...

>   
> +/* Put any references on the superpage referenced by mfn. */
> +static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
> +{
> +    unsigned int i;
> +
> +    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
> +    {
> +        p2m_put_l3_page(mfn, type);
> +
> +        mfn = mfn_add(mfn, 1);
> +    }

... this solution is a bit wasteful as we will now call 
p2m_put_l3_page() 512 times even though there is nothing to do.

So instead can we move the checks outside to optimize the path a bit? 
Otherwise...

> +}
> +
> +/* Put any references on the page referenced by pte. */
> +static void p2m_put_page(const lpae_t pte, unsigned int level)
> +{
> +    mfn_t mfn = lpae_get_mfn(pte);
> +
> +    ASSERT(p2m_is_valid(pte));
> +
> +    /* We have a second level 2M superpage */
> +    if ( p2m_is_superpage(pte, level) && (level == 2) )
> +        return p2m_put_l2_superpage(mfn, pte.p2m.type);
> +    else if ( level == 3 )
> +        return p2m_put_l3_page(mfn, pte.p2m.type);
> +}
> +
>   /* Free lpae sub-tree behind an entry */
>   static void p2m_free_entry(struct p2m_domain *p2m,
>                              lpae_t entry, unsigned int level)
> @@ -809,9 +828,16 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>   #endif
>   
>           p2m->stats.mappings[level]--;
> -        /* Nothing to do if the entry is a super-page. */
> -        if ( level == 3 )
> -            p2m_put_l3_page(entry);
> +        /*
> +         * TODO: Currently we don't handle 1GB super-page, Xen is not
> +         * preemptible and therefore some work is needed to handle such
> +         * superpages, for which at some point Xen might end up freeing memory
> +         * and therefore for such a big mapping it could end up in a very long
> +         * operation.
> +         */
> +        if ( level >= 2 )
> +            p2m_put_page(entry, level);
> +
>           return;
>       }
>   
> @@ -1558,9 +1584,12 @@ int relinquish_p2m_mapping(struct domain *d)
>   
>           count++;
>           /*
> -         * Arbitrarily preempt every 512 iterations.
> +         * Arbitrarily preempt every 512 iterations or when type is foreign
> +         * mapping and the order is above 9 (2MB).
>            */
> -        if ( !(count % 512) && hypercall_preempt_check() )
> +        if ( (!(count % 512) ||
> +              (p2m_is_foreign(t) && (order > XEN_PT_LEVEL_ORDER(2)))) &&

... we would need to preempt for every 2MB rather than just for the 
p2m_is_foreign().

BTW, p2m_put_l3_page() has also another case. Should we consider to 
handle preemption for it too?

> +             hypercall_preempt_check() )
>           {
>               rc = -ERESTART;
>               break;

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:27:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:27:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727655.1132235 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m09-0005DP-TP; Wed, 22 May 2024 13:27:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727655.1132235; Wed, 22 May 2024 13:27:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m09-0005DI-Qk; Wed, 22 May 2024 13:27:29 +0000
Received: by outflank-mailman (input) for mailman id 727655;
 Wed, 22 May 2024 13:27:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KudH=MZ=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1s9m08-0005DA-QU
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:27:28 +0000
Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com
 [2607:f8b0:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 077a223c-183f-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:27:26 +0200 (CEST)
Received: by mail-pl1-x635.google.com with SMTP id
 d9443c01a7336-1eeb1a4c10aso9670035ad.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:27:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 077a223c-183f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716384445; x=1716989245; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EwB3RcBisi76IVmLC/UT43L475zdqnUfkVo4QkSZkpc=;
        b=OxRqsVrvqPmZHfR2nRAUvCbiwtWx2soi5UYEI7cNW7aWKPjbuO2DBEWIZLAqXIMnOm
         HQUAQI/2KhNbA65S5P3VRRWkjJlqVxfauKWH7rIrcIOLRaQjwtIbfsCdS/H8scaxd1Ki
         tPJgxF2+5MPa8FNIj8IRq8Z/BVUJJVLQTHeiw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716384445; x=1716989245;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=EwB3RcBisi76IVmLC/UT43L475zdqnUfkVo4QkSZkpc=;
        b=BoVfj7KSfoliz56Xsk7RJu+Bu1q259Awrww7+zFGLJJZsbbzVyejiypmZ0/WZcMEl6
         V8OBJJICvZauoZPlR6ChOfwe/wR07/FH4T9h3+wR89Ok9aFNkWN4ogXAm4fZ4uvHZrEs
         xQZAFkJBy9Q94lD8+UDK9C1/iRZUcQjCMR6TZ4MrG1aQ06M872Ahu4Y/MvGz8kJhrnGs
         0AVt9ZFwKHzQ/p5DBFkbLOtOL7wstHcu6EDSIvcUEmY+LTu2GkIj+Qiog/BvKF+eHTUd
         EuxtOQBTrOisfxj3P9ffxJZSDLfDYemRd0dTZJDxu2+IF7GsUuTt+Trc6QuZwwGvvN3c
         RcUw==
X-Gm-Message-State: AOJu0YxHNqp5wcAhHJR5vz99rxzU/GuIxrq4O2NY3DmLfbWIvvBhda05
	l2OwxWWEudxcq7dE/YqlYH5n+FMSMFKi0B41kc8R6AhGqfNTt2khdZFWGl7deJlFUKOS9rGRyHO
	kWuRKv1mHu1FSzCIOmFgWbpDZiQrN5qEZ8X8A
X-Google-Smtp-Source: AGHT+IEPsZO4wTSTvCCWbqkQjo3rpj0QNpwikohXeENbBm/GcVY3/qwcND9IKst8HidOA5jp2xgin3nRz3GJRNuLlko=
X-Received: by 2002:a17:902:e54e:b0:1f2:f9de:c9e8 with SMTP id
 d9443c01a7336-1f31c95d498mr21594415ad.2.1716384445238; Wed, 22 May 2024
 06:27:25 -0700 (PDT)
MIME-Version: 1.0
References: <20231201101641.51198-1-roger.pau@citrix.com>
In-Reply-To: <20231201101641.51198-1-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Wed, 22 May 2024 14:27:14 +0100
Message-ID: <CAG7k0EoQDXiK4TL4Ew3RxE8s=tM5zvSVPZ3_TBmfn=6w-GBqpg@mail.gmail.com>
Subject: Re: [PATCH] xen/livepatch: make .livepatch.funcs read-only for
 in-tree tests
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Dec 1, 2023 at 10:16=E2=80=AFAM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> This matches the flags of the .livepatch.funcs section when generated usi=
ng
> livepatch-build-tools, which only sets the SHT_ALLOC flag.
>
> Also constify the definitions of the livepatch_func variables in the test=
s
> themselves, in order to better match the resulting output.  Note that jus=
t
> making those variables constant is not enough to force the generated sect=
ions
> to be read-only.
>
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:30:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:30:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727659.1132246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m2X-0005vr-9b; Wed, 22 May 2024 13:29:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727659.1132246; Wed, 22 May 2024 13:29:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m2X-0005vk-6D; Wed, 22 May 2024 13:29:57 +0000
Received: by outflank-mailman (input) for mailman id 727659;
 Wed, 22 May 2024 13:29:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9m2V-0005uB-IU
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:29:55 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5f4ec0a9-183f-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:29:53 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-56e6a1edecfso14127946a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:29:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17555sm1773890266b.189.2024.05.22.06.29.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:29:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f4ec0a9-183f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716384593; x=1716989393; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TBQwF2MTC44b/9TtLPH2MX0VuDwOJrgwU5fmaM9VTQw=;
        b=VySFlEjRjzqRIdw9F2BLsQbPKqXpsIuD7I0GtwyTn5pc8nT2E6aqh+VVFWYf7OKno5
         Fv06PiW80d5dlnckI1BO9k9Et4tHmClNtGkkaxmuVtMMmaZCm6sQEShyGtN5OUdtu89s
         mNxT/6h898bM0ZTKUBiDF2kuPwQ2xT/l72xRzEh3mHcOU9HMrX0a0lfkjK77qFAuIWVg
         KvRtfeiX207yc70TEV1fn8Dfw7I9Eyc1EfU8TZbMfWqMjvw3oSwbSIPP/AvGOtzWyD+R
         06HzMjTdORchWnUkzx6T/KJrbDoS4zpSdm2V4ndHspdz2mB1CiRbzR6VKBtHjCRIzLXZ
         Rkhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716384593; x=1716989393;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TBQwF2MTC44b/9TtLPH2MX0VuDwOJrgwU5fmaM9VTQw=;
        b=i9njNCtg+aSXjTwtiS3cHytEg2DZJpCDwd9QF1yeINR9UJcX+dD+BdAya/iYaXkPNu
         RuEBS6ZJmuK5K1sjrrY69cx3jps58m9olXhEPRwg3KLUlDW3Goa9QMfh/L0WpxGSNSvY
         ZvS+nfHC/DxTJe/FLq99W8P0fGBvQVUb7rYomc+KvW/QvC3JprNmkiokJUAR0wSQbEf8
         hIlt5Xopg3zr7FOfytVZBpG6bfy1tUHKYsdCZnDgSmS5procvYwVMiEAipSAT2cilaPO
         MnDm6NpQaoSOr91z5Tqd168u797OYcAqtcbC1BHnGBXXs/XC81xILcmQ33Wjrlziiy/4
         m9pQ==
X-Forwarded-Encrypted: i=1; AJvYcCVldbPwv4/7uKlmPxWQh4gH5dOmxaL/Dpx84p668MaRmV4rLbtCifUM/OPT2x60nEHNALzNwjdqUYd0YesZ4NEs/7e8m2Ih1cLFg0A588g=
X-Gm-Message-State: AOJu0Yz8HaKkJYEP5TVeKwEXxWssRocZlbgaGS5TjBXuvWAVzbWDCD24
	tyGYCYenNIqyTpWemFHxPJrXtfnPU7H1idX4bukgdhkWbbng0AyFpR9HCnxqwA==
X-Google-Smtp-Source: AGHT+IGX9DUTMg58NzGkz/C5hDlFYHSaWaR9BMHaC2XzSJw57H/UxAqRbwZ2cNoNhtS4rFCpFeJ2Wg==
X-Received: by 2002:a17:906:3594:b0:a59:b784:ced4 with SMTP id a640c23a62f3a-a622818ebc3mr125008766b.75.1716384592843;
        Wed, 22 May 2024 06:29:52 -0700 (PDT)
Message-ID: <93c7230a-bd2a-4763-a90a-89e02a169b1d@suse.com>
Date: Wed, 22 May 2024 15:29:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2708dbbb-f913-4920-a008-beef12cd1265@suse.com> <Zk3xtK0SaaqEadXY@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk3xtK0SaaqEadXY@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 15:22, Marek Marczykowski-Górecki wrote:
> On Wed, May 22, 2024 at 09:52:44AM +0200, Jan Beulich wrote:
>> On 21.05.2024 04:54, Marek Marczykowski-Górecki wrote:
>>> +static void subpage_mmio_write_emulate(
>>> +    mfn_t mfn,
>>> +    unsigned int offset,
>>> +    const void *data,
>>> +    unsigned int len)
>>> +{
>>> +    struct subpage_ro_range *entry;
>>> +    void __iomem *addr;
>>
>> Wouldn't this better be pointer-to-volatile, with ...
> 
> Shouldn't then most other uses of __iomem in the code base be this way
> too? I see volatile only in few places...

Quite likely, yet being consistent at least in new code is going to be
at least desirable.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:34:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:34:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727666.1132256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m70-00088U-Q3; Wed, 22 May 2024 13:34:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727666.1132256; Wed, 22 May 2024 13:34:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9m70-00088N-My; Wed, 22 May 2024 13:34:34 +0000
Received: by outflank-mailman (input) for mailman id 727666;
 Wed, 22 May 2024 13:34:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9m6z-00088H-2J
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:34:33 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 04b5b287-1840-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:34:30 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-571ba432477so11578263a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:34:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfde7sm18385336a12.48.2024.05.22.06.34.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:34:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04b5b287-1840-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716384870; x=1716989670; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Cf9soWzvYdTy8NjbCQLAIORudY8WanUALOqCMgKvsfw=;
        b=dXq0OY5o6ToVd/MQi4LJmvTzfyVHSorDQycLFMev2C88yWGGWABkpnBgGWmFZNKu5T
         uGwQnF5vB6YemESCTVJKE+9ANziSWGW/+rqpuYRYGrLqCjM0DLSoVL0b2Az96F3Iv63A
         KabEJQLXjexwdZmo3vJH6oUABMLqcLKaMHtxyleYcY6BEYjjuqN9VmhfMSehNzkxu78/
         Xtj5NmyV7ymfP6CAdwtozjpxvo35AOf9prdS5UxiNoByQaNwR9mmJUQJAhQEEQ9bklUO
         Kb6hkZGRqkjeAs1GmkVA9tBxXanZtFAcyaHwVnGCB4J0cGR7lcVhIIT2SWLaJUtum0JX
         jF6A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716384870; x=1716989670;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Cf9soWzvYdTy8NjbCQLAIORudY8WanUALOqCMgKvsfw=;
        b=Lpry4xg+gjVDpdNUQdx6molR0I3cHuz0XAh+Bq+RbtaoFyVHndUJAm7J7YoBVo7e3E
         +6Sb8sNmUIMg0GJJoxG2MIb30BqeEkarvelh7rlK0fMx1+eLWe67zwKJgn4Dr1lBCqi4
         r7SZt0Qj/peN9KDTdhPmCJyf7ZJVDOkSaHurWoNRi32hi+xD1owoGvMuA6le0NqIDbo/
         IJXp6vCT4McPTgFwM/qKq58dUPjjVtUTttt/z8rOwHl21uDUtpB63aoL4XR7xKvKnPfS
         Sgt2Qec9HLLjFGMK+bBz66twi5O7PfOOMPdjuz1pQn4B8ZbW3acjBTFisob3KwtzhZC8
         DcRA==
X-Forwarded-Encrypted: i=1; AJvYcCUbHlQclTTBzRRcm9lsRtHvv3bVtqkn12F88FayGQm5+5rf3/OUhWBwKr4bJwM32E881hoheFI3+ThJo0gDyAMbfmoXqNCS7g1+Qp7as3Y=
X-Gm-Message-State: AOJu0YzN5luOsXESoWsUa3or5TiPm79jHJj6liMZ6okF+VC4f3rUBWSd
	oEASM/KYyVt8MjdfnTIa87D9/oUtlBwrrBolexQOm0pJT5S71MPNNs79Itkxrg==
X-Google-Smtp-Source: AGHT+IGMlS5esugqUatklBs0q8eioDDVcGA7Cmj8xEvRAHh7Y6iHpTCBS9GnM5QzijmRcNopQqUCdw==
X-Received: by 2002:a50:954e:0:b0:56e:2bf4:fe0a with SMTP id 4fb4d7f45d1cf-57832c06250mr1636496a12.35.1716384870426;
        Wed, 22 May 2024 06:34:30 -0700 (PDT)
Message-ID: <5484aab7-b15d-46c5-9ff1-9a5b54640acc@suse.com>
Date: Wed, 22 May 2024 15:34:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
 <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com> <Zk3wMevPye8Rfjm4@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk3wMevPye8Rfjm4@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 15:16, Roger Pau Monné wrote:
> On Tue, May 21, 2024 at 12:30:32PM +0200, Jan Beulich wrote:
>> On 17.05.2024 15:33, Roger Pau Monne wrote:
>>> Enabling it using an HVM param is fragile, and complicates the logic when
>>> deciding whether options that interact with altp2m can also be enabled.
>>>
>>> Leave the HVM param value for consumption by the guest, but prevent it from
>>> being set.  Enabling is now done using and additional altp2m specific field in
>>> xen_domctl_createdomain.
>>>
>>> Note that albeit only currently implemented in x86, altp2m could be implemented
>>> in other architectures, hence why the field is added to xen_domctl_createdomain
>>> instead of xen_arch_domainconfig.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com> # hypervisor
>> albeit with one question:
>>
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>      bool hap = config->flags & XEN_DOMCTL_CDF_hap;
>>>      bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
>>>      unsigned int max_vcpus;
>>> +    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
>>> +                                         XEN_DOMCTL_ALTP2M_mode_mask);
>>>  
>>>      if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
>>>      {
>>> @@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>          return -EINVAL;
>>>      }
>>>  
>>> +    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
>>> +    {
>>> +        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
>>> +                config->flags);
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if ( altp2m_mode && nested_virt )
>>> +    {
>>> +        dprintk(XENLOG_INFO,
>>> +                "Nested virt and altp2m are not supported together\n");
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if ( altp2m_mode && !hap )
>>> +    {
>>> +        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
>>> +        return -EINVAL;
>>> +    }
>>
>> Should this last one perhaps be further extended to permit altp2m with EPT
>> only?
> 
> Hm, yes, that would be more accurate as:
> 
> if ( altp2m_mode && (!hap || !hvm_altp2m_supported()) )

Wouldn't

   if ( altp2m_mode && !hvm_altp2m_supported() )

suffice? hvm_funcs.caps.altp2m is not supposed to be set when no HAP,
as long as HAP continues to be a pre-condition?

> Would you be fine adjusting at commit, or would you prefer me to send
> an updated version?

I'd be happy to fold in whatever we settle on.

Jan



From xen-devel-bounces@lists.xenproject.org Wed May 22 13:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727674.1132265 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mD8-0001TB-E0; Wed, 22 May 2024 13:40:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727674.1132265; Wed, 22 May 2024 13:40:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mD8-0001T4-BO; Wed, 22 May 2024 13:40:54 +0000
Received: by outflank-mailman (input) for mailman id 727674;
 Wed, 22 May 2024 13:40:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mD7-0001Rk-GP
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:40:53 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e78fe5d5-1840-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:40:51 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5231efd80f2so7170000e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:40:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cdd7f8ea1sm939739066b.109.2024.05.22.06.40.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:40:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e78fe5d5-1840-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716385251; x=1716990051; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Pev6S84hbabPRG5sK0VuSlB/JOoBHk5GMSJ0GUMmO64=;
        b=P+hGP9nyuC/YWzmE7+nCkL+mDUgud/jJu01Mm4t/1wFauF5nw88Xsba3HSS4N+toqj
         2oxEVznbsMi+y8g7nnayjDGfv7O+9J8iMerEbyJrqoqsxQMqqBHSRMrGAju3Kiy8XNKI
         oEj8MeCifg+AsYU9GCg9+7ERQjh33ruR+WmRn6OS3pephlKTrYEsaHhN29FnS3lQmyBR
         fjp7D7ib33rs/CCnA+C/iHJVm7SotaNhWXT7xeAzZ7f/DbhakheJ4IscF9/YlSy/80eD
         djJPstBs9TeSxGZ4DWeKbD1cSnGSFUCTMcyoyAGnl9GYdGY8DAaomAgOJHgNL3y8ine8
         ZMFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716385251; x=1716990051;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Pev6S84hbabPRG5sK0VuSlB/JOoBHk5GMSJ0GUMmO64=;
        b=k5wATlNiUgXyjAnm5BMYyqcC+6lSTztGh2EPrFaeUSawrECK2QGM4bJ67KRbn7EiJr
         ngng3JRyY+TLxQonHAT1jYKyfceHul6LbVRYpFqjEtL+kqKnmhrejJ1s5M1wCfrc8hUm
         8AFe59j36w6mYRUWIOzp5fapd6zNx5AoXJCZnr/qzePP3iQkD7YtgET8kmAy5LeWe//z
         BSjKybe2bgQZCQn7Vc5ZjNuHfpUznTnfUJm01nbHktObtq/4AuA4/sryfdhmjKs5BBr3
         N341AeuUKVMLM4J/W9uALS4zEhdqTl+BQzMBv2hFj5e1peJU+V/RDbhYbQEruS+hTqbc
         Z5bQ==
X-Forwarded-Encrypted: i=1; AJvYcCXTA+zMb4dxQygEFBiyDlWzugyeYL1Ow9aL7M7n9+reKkpuFqdZwDJZlAEVQ/Gxz+VMP/KBHANVvYMh/SBnBRwi/4eJCiRqJBQ7sY/ykTA=
X-Gm-Message-State: AOJu0YwmjiLtJs8sepdeHZHUNxlu5+gOW0ABYC1IjEmoWEKwzJqcORum
	RMkgDNvZmmFUsxdjnrojM8Wi6zPbsb1rl9x+lYp7zzTt7XPBdmbJcMKXvoDiRw==
X-Google-Smtp-Source: AGHT+IH/CVoPvduXRkd20bv5YoPTQ4G0HM5cNIh/wHlEbtSq5olm9UtJFaG717pRgYhv5vn8ofSAHA==
X-Received: by 2002:a05:6512:1042:b0:51d:44a3:6cc9 with SMTP id 2adb3069b0e04-526c130a249mr1302533e87.58.1716385250912;
        Wed, 22 May 2024 06:40:50 -0700 (PDT)
Message-ID: <1140d40f-2f5c-402d-a0fc-1c6598a7a424@suse.com>
Date: Wed, 22 May 2024 15:40:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] x86/shadow: Rework trace_shadow_gen() into
 sh_trace_va()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-2-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240522131703.30839-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 15:17, Andrew Cooper wrote:
> --- a/xen/arch/x86/mm/shadow/multi.c
> +++ b/xen/arch/x86/mm/shadow/multi.c
> @@ -1974,13 +1974,17 @@ typedef u32 guest_va_t;
>  typedef u32 guest_pa_t;
>  #endif
>  
> -static inline void trace_shadow_gen(u32 event, guest_va_t va)
> +/* Shadow trace event with GUEST_PAGING_LEVELS folded into the event field. */
> +static void sh_trace(uint32_t event, unsigned int extra, const void *extra_data)
> +{
> +    trace(event | ((GUEST_PAGING_LEVELS - 2) << 8), extra, extra_data);
> +}
> +
> +/* Shadow trace event with the guest's linear address. */
> +static void sh_trace_va(uint32_t event, guest_va_t va)
>  {
>      if ( tb_init_done )
> -    {
> -        event |= (GUEST_PAGING_LEVELS-2)<<8;
> -        trace(event, sizeof(va), &va);
> -    }
> +        sh_trace(event, sizeof(va), &va);
>  }

If any tb_init_done check, then perhaps rather in sh_trace()? With that
(and provided you agree)
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:47:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:47:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727680.1132276 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJ0-0002ll-5d; Wed, 22 May 2024 13:46:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727680.1132276; Wed, 22 May 2024 13:46:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJ0-0002le-2S; Wed, 22 May 2024 13:46:58 +0000
Received: by outflank-mailman (input) for mailman id 727680;
 Wed, 22 May 2024 13:46:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8RWL=MZ=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s9mIy-0002lX-IS
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:46:56 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd3732ab-1841-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:46:50 +0200 (CEST)
Received: from PH8PR02CA0052.namprd02.prod.outlook.com (2603:10b6:510:2da::32)
 by CH3PR12MB8458.namprd12.prod.outlook.com (2603:10b6:610:155::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 13:46:47 +0000
Received: from CY4PEPF0000E9D0.namprd03.prod.outlook.com
 (2603:10b6:510:2da:cafe::33) by PH8PR02CA0052.outlook.office365.com
 (2603:10b6:510:2da::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 13:46:46 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000E9D0.mail.protection.outlook.com (10.167.241.135) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 13:46:45 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:46:44 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:46:44 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 08:46:44 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd3732ab-1841-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=azE6+E7+AEx3XdJcDmlPa4PrMr8OJjXMI1gz4KVnEFiTJ+CYMHCnF1pWKoA6I4lv0OXOUjPOPQOxjLzij2lcu7VfCsSiDivsD0Colh+QLvqTxnBAabW427Ls4wBoSkK1dcI8yrcQGYUjLXKFUanSpuA4Kc8ZstwmrCuiAXLyfS7mKoaFyQ2GdAkZvGDzPBb3TMfQJXiZXGnsnKBMZOpC4y2kGKa8An+d5KE1MCO4aTXAgfRVmcg1AMNYft9urU6BV7R/J4UlzDjiNQWtgCcN+7HrcHfY3TLOkp45PwMvrHLaoFGse3WMByzBlQ90BDKo8bSYZOD8N678wECX6xipkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=vs5X4rkFtjieBtrzO5z+2i3QZv8W7hE5OOmvDjlmVVo=;
 b=Q0cASRYFlnKDqqDTFSfc6I0V8rr7YPI8zauuqAmUc4nW/Om+cRqduhITU7ziZYH16pdYSxvmqOocOBz7/sVrIgztIRjG82C8jWdkjirrYXffUr/TRtgjzFgYDCW84snO3IuBsvu5jb0+jL0RdiZ+Nb8GmrQ/lBdPm8nJkgv/OoE2ZeBd0515Cdxwa68EnIeYfU3uQbOQ+e2b1410KQGoPR/88V4a2BY6mAjiu9cVSO+Pf6Oty9X+N7nKT3PIcfWFxGIM549xOy3QKuJDqOYZkNQK5w3JzP5WUdXINMyZYQzwUhcA9QoyV86QYwfUVzF7uciQ1F4UMThNRN87RO0dBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vs5X4rkFtjieBtrzO5z+2i3QZv8W7hE5OOmvDjlmVVo=;
 b=hUUAjFnDaxo2CrvyZsw5U0XFoMyk5tWYldPvLo0Rwpc2ZDP4S0heetnLh/T3YwLcE8S3F0f29eR9FpDdbbukut3SfU2nlNnks1vVTqPqUAi4OXLQXqDXoV+IjP4pA+HdGRMpQTtgxG59iuvupczDuQ+BYKyJ67t2l2iM8Y1J4fw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <883ab811-1cde-406e-8a55-13cc9c0944ef@amd.com>
Date: Wed, 22 May 2024 09:46:46 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] x86/PIT: supply and use #define-s
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
 <fb87a1db-4d8f-4e33-8858-71de4de6e276@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <fb87a1db-4d8f-4e33-8858-71de4de6e276@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D0:EE_|CH3PR12MB8458:EE_
X-MS-Office365-Filtering-Correlation-Id: 556dac09-c645-46c5-b8a3-08dc7a659ee5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eWJ1NzlZQ2lDY3Zrem16dmg5cXd5SXhJRGNwRmgvS0tXNUpsRE8vdWdMK0FO?=
 =?utf-8?B?WHFFSzVDQWlBdjlSblRSbkRZcUxBSDQyKzUydjFsdmpoR1dDQ2xxUk4yWUtO?=
 =?utf-8?B?VS91aWJ0ODRXTFhvV0tnV0Q4Rm9CTGZMZDIvTUN4aGs5U0pHa01KMVJtOXo1?=
 =?utf-8?B?a08wVFAwRlYzUkppaTg3SDMrQWlUcTBRaDVsK0s3clEyMlRhdTljRW9LNzFX?=
 =?utf-8?B?bTFmOGtWUHpjalJlN0FHaDlFcjNPTWEwbDRmOXMwRzR2YVhMQU5zZWJwaSt4?=
 =?utf-8?B?UGNVdmZlMnNVSGpIOUNub2dXNjR5MUw2T1RQWHJTYU9KeXhqK1JNWXpTWW1U?=
 =?utf-8?B?RXprS2t0SXhBNW0wYjBvMXI3M2NuamZmUVk5VU1ZNGVLWHpGUHdlRDQ1RlJk?=
 =?utf-8?B?L3ArOXRocmRtZ3JEZ0dTbDIrWm5JVW9DcWRWL0hHUVNmRSs0bVBsUGNnbW9y?=
 =?utf-8?B?TFNIYXZJVm0yb3hLQ2xsTUZTN0FJbHVrV3BCb0gzdXpkQmhXQ3pTWmxHRFEw?=
 =?utf-8?B?dUtvV0xneXJKa0E5Nm1zT0ZtTHZReEFhaG90T0M2cnVpQkM3ZmZrMmJwQWQ4?=
 =?utf-8?B?QS9DS3VsaHRZU2dTaFpxVC9lc3V6N0ZUdHZsOFRBL0FaQVAyYm5hSHgwdzZa?=
 =?utf-8?B?VmNuZEdHM09zQWRVM3RDcGQybTlvd3JlUitGN1pLRU05NklOQUNyOGRydWph?=
 =?utf-8?B?d3F0Z0FpcGkrVm5GZ3hjaTh5eDV0NTNCcXB1bGVwRTdENHdiLzMzTHlmU3p4?=
 =?utf-8?B?Z3ZDWUJjTjkrdzY4a2ZpS1VOWFhMUk1KZWxkTllLSnQ4T3U5a3VDUDAzb1dh?=
 =?utf-8?B?SVNOTDUxRjBLZmcrVGVjU05Mek03TExDVTJOUGVFTWl2VTZnM094N2RNVktJ?=
 =?utf-8?B?WjBxeUtoYWtUdU5KTjZWdFlvWERMVzl6WVpteTJLdGRuTm1KZS92bTk1VlJO?=
 =?utf-8?B?bGNVSnI2MEd3WmUzT2FBL3lva1dPeFFPUldZMS85WGVqMHE5UUU5d1pRVEMr?=
 =?utf-8?B?MUFyTFVlaEEzZitRcWZMQ2ZDUzBtNmdrV2k0T0V4MDhWRWlWNU9sRlNHWFZM?=
 =?utf-8?B?ZCtSWWVzY1ZrRndscVVvelgwMVBQdDNrS3k0R1R2dXpPKysxdWVZeUtieUZa?=
 =?utf-8?B?TEp6RERDR2l6dlpEbWRPWWtKeDVMc2ZBR01XTlhZKzhiNnp6QTlDbThSQyti?=
 =?utf-8?B?YkJJaTM3bkYrRFh0K3Q3NEdhczgxYTBZUGR2Z1lHQVFXclc4M3pUTElhZEtx?=
 =?utf-8?B?Ync0V0pYelVxanlOaFRUV05RRFNmeDVhWUp0NjJwajQrZ2hPUlRUNnl4QjRl?=
 =?utf-8?B?N1I2cVU1OG44RWZ4bG5NamJ4Yy9VSm9nendBOEpCRDlMRUhpQ0RqcUx6am1H?=
 =?utf-8?B?OVRsbUY1VExWNG9FRFZzQm84RHQ4Y0FSaGhqNkp6WGh3R25vOU1WVDhkL1BR?=
 =?utf-8?B?ZGVMcFljemVNcUh3Q0EwRUlkS1UrRmRCMWF6RUUzRTgyaG4zTXdxS1EwQ283?=
 =?utf-8?B?dnBsYVpYT2ZYZTBLVDMzTlFTaDNLWHF3dG1jbDdXaUVuRmdkcmlqRm1VSmYw?=
 =?utf-8?B?YUhUb1czWkYxUTBGRU4vVGNuc2ZLNXdTTXJ5bE9JbHgxK0RVRENoejFzRkJm?=
 =?utf-8?B?QnZnMHJ0d2xQL21tUnBqekdGZXZuZ3lxM3ZybjF5WHk5bzdQUzhNN0k4aFhh?=
 =?utf-8?B?cVl3NzV1UXN5dE1MVW1XYlhWNWU3THluRWV0WW1KZTR2WGdxNkVXSHZ5MDls?=
 =?utf-8?B?RGlmTXZPc3B5RFk4OUZZSUxJTDRlb3VGSEZ0c3hPRWh2akJjellLK21ZRVht?=
 =?utf-8?B?SStjUWwxN2ZVY01PYy9GaUtOVGE0Wm5ER3JSSEFUbFJyS0xSbytXSHVRY0pv?=
 =?utf-8?Q?8FsTjoQOxspxX?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:46:45.4814
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 556dac09-c645-46c5-b8a3-08dc7a659ee5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D0.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8458

On 2024-05-22 08:59, Jan Beulich wrote:
> Help reading of code programming the PIT by introducing constants for
> control word, read back and latch commands, as well as status.
> 
> Requested-by: Jason Andryuk <jason.andryuk@amd.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks for making the switch.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:47:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:47:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727681.1132286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJD-00038r-EJ; Wed, 22 May 2024 13:47:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727681.1132286; Wed, 22 May 2024 13:47:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJD-00038k-9h; Wed, 22 May 2024 13:47:11 +0000
Received: by outflank-mailman (input) for mailman id 727681;
 Wed, 22 May 2024 13:47:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9mJB-00036q-Mr
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:47:09 +0000
Received: from mail-vs1-xe2b.google.com (mail-vs1-xe2b.google.com
 [2607:f8b0:4864:20::e2b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c82451df-1841-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:47:08 +0200 (CEST)
Received: by mail-vs1-xe2b.google.com with SMTP id
 ada2fe7eead31-47ef7e85cf0so1554675137.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:47:08 -0700 (PDT)
Received: from [10.80.67.140] ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43e2c8d3d45sm106303711cf.47.2024.05.22.06.47.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:47:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c82451df-1841-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716385627; x=1716990427; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=mrpfkd+3vqtkVZQuW0YaayVQGhiM1Dn/yVCYsI8ziV8=;
        b=l3cxJzwSnsWpniJEpWvBNI6T7HjeWHQhS5p0c4srlIG+5g9DAZvGJbsltxNp93lRmG
         SGj/3fJ/dhKw20UGbn/UjHAHsPn9BgLg5w9OZ+DB4UOi8nicvuMV3qxuCvRn5gXeaLnK
         1SBknSLRRqWR6FXcPsjbLPAu0EjZjAxEb2XY4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716385627; x=1716990427;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=mrpfkd+3vqtkVZQuW0YaayVQGhiM1Dn/yVCYsI8ziV8=;
        b=Pu6QL9neWASN7eT8/4JgLzYpnqe0q/lfn0CuFyVKQFUNFinqmGmbGIzNeXfIM8uwtm
         6bFLC4mNax9fguudUbLMwBYhRyDA/NMy/7Xh5ScMMV/DAgp/m/5JuLppFekVU+BdEOPw
         k5JJvvrZoovrSss5fa0+pxmVXSs0bo5Y3ijh3YL+f7Xe3X7BXh1tF/RrtizTm9H+jmhE
         XTWBl51OQQ2G6zZWBBEjkIhDgogeNrrimk9l8U/XaDrx0phVGU5QJjHaCl7bdQuOPsH/
         D+at3eGZwltisV5Qhc6xvNwk/K1D/gvl7eqdlujQzuKV88nbcGa0RjRlWCINlR8dILPl
         jeBg==
X-Forwarded-Encrypted: i=1; AJvYcCXjfkjFgYbWhJEUQwIkwelY+4Qv2QtNNagjY6kMj11gqNhUgvVeh9svuzWG2CFVNRfI2TaWMUY38LH5TsxyNF53TTh9KMp1Jt+0wlcaLCs=
X-Gm-Message-State: AOJu0Ywe/0LOMngcaZuiwsf9UEdK+Q3gmg1P32KKXQWH9FjrF03+akIx
	6Vk+y57okMT7HqcfkX+RaxjNJlfoASuOHgEWc5lnsNfjnzzU+JdJerngKaZDGTA=
X-Google-Smtp-Source: AGHT+IHVjDiuiZbV8LCzVyuB36HFHCptEEBcHvRt7SvxdwwEu1oDKgNoKsmlzv2BMFEvv5u9NTbTKg==
X-Received: by 2002:a05:6102:3f0f:b0:47e:f1f1:d052 with SMTP id ada2fe7eead31-4890a2c6ab7mr2119170137.33.1716385627688;
        Wed, 22 May 2024 06:47:07 -0700 (PDT)
Message-ID: <b3b4f2d0-fb42-44be-bb24-a22c8c237f13@citrix.com>
Date: Wed, 22 May 2024 14:47:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] x86/shadow: Rework trace_shadow_gen() into
 sh_trace_va()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-2-andrew.cooper3@citrix.com>
 <1140d40f-2f5c-402d-a0fc-1c6598a7a424@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <1140d40f-2f5c-402d-a0fc-1c6598a7a424@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22/05/2024 2:40 pm, Jan Beulich wrote:
> On 22.05.2024 15:17, Andrew Cooper wrote:
>> --- a/xen/arch/x86/mm/shadow/multi.c
>> +++ b/xen/arch/x86/mm/shadow/multi.c
>> @@ -1974,13 +1974,17 @@ typedef u32 guest_va_t;
>>  typedef u32 guest_pa_t;
>>  #endif
>>  
>> -static inline void trace_shadow_gen(u32 event, guest_va_t va)
>> +/* Shadow trace event with GUEST_PAGING_LEVELS folded into the event field. */
>> +static void sh_trace(uint32_t event, unsigned int extra, const void *extra_data)
>> +{
>> +    trace(event | ((GUEST_PAGING_LEVELS - 2) << 8), extra, extra_data);
>> +}
>> +
>> +/* Shadow trace event with the guest's linear address. */
>> +static void sh_trace_va(uint32_t event, guest_va_t va)
>>  {
>>      if ( tb_init_done )
>> -    {
>> -        event |= (GUEST_PAGING_LEVELS-2)<<8;
>> -        trace(event, sizeof(va), &va);
>> -    }
>> +        sh_trace(event, sizeof(va), &va);
>>  }
> If any tb_init_done check, then perhaps rather in sh_trace()? With that
> (and provided you agree)
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Sadly not.  That leads to double reads of tb_init_done when tracing is
compiled in.

When GCC can't fully inline the structure initialisation, it can't prove
that a function call modified tb_init_done.  This is why I arranged all
the trace cleanup in this way.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:47:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:47:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727683.1132296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJM-0003VC-Kr; Wed, 22 May 2024 13:47:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727683.1132296; Wed, 22 May 2024 13:47:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mJM-0003V5-I2; Wed, 22 May 2024 13:47:20 +0000
Received: by outflank-mailman (input) for mailman id 727683;
 Wed, 22 May 2024 13:47:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mJL-0002lX-KE
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:47:19 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cde0ea1c-1841-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:47:17 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-56e47843cc7so9432688a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:47:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17f0dsm1777980866b.211.2024.05.22.06.47.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:47:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cde0ea1c-1841-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716385637; x=1716990437; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XJYP15VwKFL3djmvZ3a+FhRBTqtbBevvFkBpnASPt7I=;
        b=T5Caaqo+23kvmFCvFH/gSgXi+NAgECsMTjJaFVH0v5gfIGgSH7S1svFKTYX2Vn+qdl
         o5udiWoSxfaxuwNgpFk/TiQK6AW1Rlyai4isYLXd6RZ7YOr90eh8iHMrmOvrekGPuKFT
         gxrhH/zVZqzr08YufO2dMxSlopz/1CQbP7iJOCevz660vWF11D1lcjSDA3jddhxpZ4F8
         ++XxLYz3FAPFtiGBqYk5vmaC8qnWM75/bfRIKvgaFj44geL+gQLnICgEXB6JMfY1XMwD
         m69mIzuU5IsP7HoTAwUuDwTi5MUHGpRuJB7BoIa4qJ6aeNV1AyTbMHyks9rRsvbJO0l0
         9swg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716385637; x=1716990437;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XJYP15VwKFL3djmvZ3a+FhRBTqtbBevvFkBpnASPt7I=;
        b=bRIJrdT192iHlqhoXvdANB+mtztZs7Ja0uMIiWmjniQxqhe78gmI7C3XySFrWsT10b
         rMVtUn9C/ycdLtlMCS9MrSgzbZkp1NYlE33JX+xh7O519DT+chxnygJYjvsAiZds4DhV
         ViXyjxegzW8wiTkZkzGdhY1cofJ/0NAT8wFT9XFmSGsww9r+j9IlX0tgeBQXmI/uCdOz
         Di8mNdrvuS1/MSUhWk2bEUlzRpK7Ohqlm5DcZM0XCIPrPHwfsL2P87nzVlSiwxHyrMW+
         2KeLWUwGz2y+pQRT8xjjyhLsSRMlE+OVZaWJ5f7bUd0sYIB14UX86PBcM0BG0uubWuJl
         zMwA==
X-Forwarded-Encrypted: i=1; AJvYcCU9LaMgXna8Z+haKEubg+IAsSMgOQGY7lF2v/tNu4KJdTmA5u+Bbyk++0dJCxtBS32VqLdrjkcUNPU/U8E92m9/svNn4/BGy0d5jOai0dE=
X-Gm-Message-State: AOJu0YyHafxiSLA+UtBqcYFvcW8F6MyKQJZSmuxHIL8XuV18/+Ck3qDv
	hv52H9CM/ToTZXccoxcrh71X6OgF22kQymhMAJmCdkFTq5o8M2TCBxJS8lrctw==
X-Google-Smtp-Source: AGHT+IEAZIHPi8DuEOcfh6XzYpQ2v59n46gbHda5F/OuWK3hniG4xTK9/FATj27/PHTNdZbA6XmKSA==
X-Received: by 2002:a17:906:3ad8:b0:a59:bd78:34ca with SMTP id a640c23a62f3a-a62280a1976mr146335366b.36.1716385637400;
        Wed, 22 May 2024 06:47:17 -0700 (PDT)
Message-ID: <e0f0d9e7-ab53-414a-b689-dde9a9089f5d@suse.com>
Date: Wed, 22 May 2024 15:47:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] x86/shadow: Introduce sh_trace_gl1e_va()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240522131703.30839-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 15:17, Andrew Cooper wrote:
> trace_shadow_fixup() and trace_not_shadow_fault() both write out identical
> trace records.  Reimplement them in terms of a common sh_trace_gl1e_va().
> 
> There's no need to pack the trace record, even in the case of PAE paging.

Isn't this altering the generated trace record for the 4-level case, in
size changing from 20 to 24 bytes?

> --- a/xen/arch/x86/mm/shadow/multi.c
> +++ b/xen/arch/x86/mm/shadow/multi.c
> @@ -1987,51 +1987,26 @@ static void sh_trace_va(uint32_t event, guest_va_t va)
>          sh_trace(event, sizeof(va), &va);
>  }
>  
> -static inline void trace_shadow_fixup(guest_l1e_t gl1e,
> -                                      guest_va_t va)
> +/* Shadow trace event with a gl1e, linear address and flags. */
> +static void sh_trace_gl1e_va(uint32_t event, guest_l1e_t gl1e, guest_va_t va)
>  {
>      if ( tb_init_done )
>      {
> -        struct __packed {
> -            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
> -               so put it first for alignment sake. */
> -            guest_l1e_t gl1e;
> -            guest_va_t va;
> -            u32 flags;
> -        } d;
> -        u32 event;
> -
> -        event = TRC_SHADOW_FIXUP | ((GUEST_PAGING_LEVELS-2)<<8);
> -
> -        d.gl1e = gl1e;
> -        d.va = va;
> -        d.flags = this_cpu(trace_shadow_path_flags);
> -
> -        trace(event, sizeof(d), &d);
> -    }
> -}
> -
> -static inline void trace_not_shadow_fault(guest_l1e_t gl1e,
> -                                          guest_va_t va)
> -{
> -    if ( tb_init_done )
> -    {
> -        struct __packed {
> -            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
> -               so put it first for alignment sake. */
> +        struct {
> +            /*
> +             * For GUEST_PAGING_LEVELS=3 (PAE paging), guest_l1e is 64 while
> +             * guest_va is 32.  Put it first to avoid padding.
> +             */
>              guest_l1e_t gl1e;
>              guest_va_t va;
> -            u32 flags;
> -        } d;
> -        u32 event;
> -
> -        event = TRC_SHADOW_NOT_SHADOW | ((GUEST_PAGING_LEVELS-2)<<8);
> -
> -        d.gl1e = gl1e;
> -        d.va = va;
> -        d.flags = this_cpu(trace_shadow_path_flags);
> -
> -        trace(event, sizeof(d), &d);
> +            uint32_t flags;
> +        } d = {
> +            .gl1e = gl1e,
> +            .va = va,
> +            .flags = this_cpu(trace_shadow_path_flags),
> +        };
> +
> +        sh_trace(event, sizeof(d), &d);
>      }
>  }

Unlike in patch 1, it's less clear here whether leaving the tb_init_done
check is actually better to keep where it is. In principle the compiler
should be able to re-arrange code enough to make it identical no matter
which way it's written, at which point it might again be more desirable
to have the check solely in sh_trace().

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:48:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:48:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727691.1132305 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mKD-0004N3-UD; Wed, 22 May 2024 13:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727691.1132305; Wed, 22 May 2024 13:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mKD-0004Mw-RL; Wed, 22 May 2024 13:48:13 +0000
Received: by outflank-mailman (input) for mailman id 727691;
 Wed, 22 May 2024 13:48:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s9mKC-00036q-Dk
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:48:12 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on0604.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::604])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ed72933b-1841-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:48:11 +0200 (CEST)
Received: from AS4P192CA0029.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:5e1::14)
 by DB5PR08MB10213.eurprd08.prod.outlook.com (2603:10a6:10:48d::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 13:48:06 +0000
Received: from AM3PEPF0000A79C.eurprd04.prod.outlook.com
 (2603:10a6:20b:5e1:cafe::91) by AS4P192CA0029.outlook.office365.com
 (2603:10a6:20b:5e1::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.34 via Frontend
 Transport; Wed, 22 May 2024 13:48:06 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF0000A79C.mail.protection.outlook.com (10.167.16.107) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Wed, 22 May 2024 13:48:06 +0000
Received: ("Tessian outbound 57d5075de207:v327");
 Wed, 22 May 2024 13:48:05 +0000
Received: from 3b5107f893e7.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A68587BA-DEE6-4534-9782-EC73DB0329ED.1; 
 Wed, 22 May 2024 13:47:59 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3b5107f893e7.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 May 2024 13:47:59 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS1PR08MB7540.eurprd08.prod.outlook.com (2603:10a6:20b:470::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 13:47:57 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.035; Wed, 22 May 2024
 13:47:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed72933b-1841-11ef-90a0-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=DQPuuRPtwgizhbR2BXWi5OzM1s/TQohkzRG8+prfvZhqJpyoGohT0KP5GplrS7MQx3S9+bO5s5qbx7am9yUL6CT+agaXpRY/+hHxuPvpvvicLefJn15icuW7MQXIazx7gJ2VwOks7/MJcN/54ILx7y1CcumomNouP4hvq6dJ1RpMwBxsUA8M3TNTMzeohAWISxq45RVesLrRTwLHHTPMlB1UoYbz62ssV+8c0XRRkxVOx9GwK7K8Oq7NNbZWzlewtSoHRXZvOwUzqY31E6+uWzTmM3dr1NHZQBRXbl6Xh6qUV046jG8+BYEhJHvRmGPn6A1fLxiIA5rLvbHIdJpXig==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8inm99bsJSwaZum7vHP1UHCxY0hjtJOUxkxPN83wO1w=;
 b=ZNASMT/IpPsKZ63b0ikjcVQur3IVupzfcFWY3GiugcyHMzAjAVwxm6y0spAiP677Tb3XkQp2pSkFAVnAuVOF2W0tLPOv8hLIsgKnamf4lYjwzARhx8Hk0PVnkAYeDLqZJYjI5ugDLgDGh4us6/xHWwJTUwyOZ1gpeoz04qk7hX2Ws3Gp1WzMEE4ILaQ7x+SpIIREL9UxfVPsObKCez3BQVTpvvxpEHA+I5gCYcE/2hVfbmi+EosNDAeNMTzklkIDTqD3YzxoGUAzWhgTLsaZul/FUK/pLO9CP9rEhF+qDpS+fAaix4pq7KLeSnzVNKYUCp4/UrjdCMlpio5446sDFw==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8inm99bsJSwaZum7vHP1UHCxY0hjtJOUxkxPN83wO1w=;
 b=EM0fVuEg9rQY9nn6wES150KRMpAryqbMTT+L41jOe4PMN+Hot8LrINI/5MrIR05gNCiNtmhRMayAj2ns8xqnvDAfcGQbABG70BGDwAT5iTicKukW3LPDGaKJhBOnj0KOL8Vh6K6TjTFJDt/hqaXYLdv1lnlPYW9OVcL/a9Jv7X0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5cf9bc6dca38adb6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PxKwuGKbSUbpyFbjjzEdb3zomNruKfNVn0ctO0yS8TDi9r7seSo+c7kjnHN2hwv30ZEND5VnPyFYu3w9Nf6Okri97Xdw/5DI+88G4RzlKywB1gmFZdE45RolNSElwMCwN147+VhOy8lh9Oml7QjTS8XUPBjO9HEMRESMhpBQ7LU5lbLYLQAnX0vHI3HoMnPK1PmDfU1XW5va+gIa+jzHL3EWh/Lxu6YybCAQtTxXFmu1WHMPmq3B2/HlZu7WE9L9GIB4tHjmBgqShIDiDSxPVCbKucqdQlMf+YySniJSDTuyzNZJy+6y892IZCnWDQ47wFld6hVb8YBTohzAjrskww==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8inm99bsJSwaZum7vHP1UHCxY0hjtJOUxkxPN83wO1w=;
 b=b0cnezilGMBVWZKYFS8PukRDrerUqiyZ3reU/uGiIntBIV3R21BgH2LD/awh5vqh/1NuA9Idxmm+wuFXMpgG/EYHTu0KHCpJcqNy/YXCyREhrSVGnJp6+y1XbmB98+1FL5zKA6usoyiGqVeANsbD+FMjMoVCr3tYQ1bqaABH6AhudqYuMHlmu4nU8o9Hxcp3x6vBRUE74OtvacFfjd9pbdjXMoEc+olHj9O4lieAYbr4LJ//SZ3BC6HOnS+9QIJbvuO4ACiH+Spqdtf51sjWADwC5COEeTgn4+eYouv/PIdwt0Al7+E5VK05PkFTanGVVvY03L2KBmyqz+BbDC71Tg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8inm99bsJSwaZum7vHP1UHCxY0hjtJOUxkxPN83wO1w=;
 b=EM0fVuEg9rQY9nn6wES150KRMpAryqbMTT+L41jOe4PMN+Hot8LrINI/5MrIR05gNCiNtmhRMayAj2ns8xqnvDAfcGQbABG70BGDwAT5iTicKukW3LPDGaKJhBOnj0KOL8Vh6K6TjTFJDt/hqaXYLdv1lnlPYW9OVcL/a9Jv7X0=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
	<Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Bertrand
 Marquis <Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
Thread-Topic: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
Thread-Index: AQHarBz8P+fwY/NKMEm727VASI/i97GjPugAgAAGHgA=
Date: Wed, 22 May 2024 13:47:57 +0000
Message-ID: <CD6149B6-EC9C-4313-AA21-F0F0156A9E1C@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-4-luca.fancellu@arm.com>
 <c10b2513-b0f1-4042-a538-2115e92408d1@xen.org>
In-Reply-To: <c10b2513-b0f1-4042-a538-2115e92408d1@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.600.62)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS1PR08MB7540:EE_|AM3PEPF0000A79C:EE_|DB5PR08MB10213:EE_
X-MS-Office365-Filtering-Correlation-Id: c891ddab-30a0-4429-2d7d-08dc7a65cee8
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?NUYzc3JQQVFFUkdFeG9NS3VQaFB5aVhlV2swOFhZczZCeU4yMlFmYXFSYXMz?=
 =?utf-8?B?Z3BKZUI2WGJlVm03M0xGZ0R3ZlBuYTR3L2NMVGN3VUF3Uk5VMjYrR09sZ0tU?=
 =?utf-8?B?R0Y4VjVlYUZISXpMUGd1Wkd2WmFwazJiNzByc1pudHFIWHNsSU1JaFp0dGVh?=
 =?utf-8?B?UFRsY0NpbDBQMVZOWmRDbkpGeFFnZkhUQjJpeTYxQ2xyTU9EQmZPVTdLem42?=
 =?utf-8?B?KzdNNlpFcVRRbFErYWFEWldiZUc4bGZ1L2YzRW1xMHVid1VxZ2pFenRna0Fp?=
 =?utf-8?B?OG5MUDEwSVZwMFRsVzgxSHBES2JQcVhad2xtci9SV0pBYWdZZE0wb1I0SGE4?=
 =?utf-8?B?Y0lMdS9vb05EdzF0OXp5Z0xiU3MvWGJkNjRNcmVhSlQwWmNiRG40T3lpQTlp?=
 =?utf-8?B?bDFnanFqWlNPVWtNMFFnTUtMcHBVWW1KaGxCUGwwaUNzV3VoTXZwNEFiL3Zo?=
 =?utf-8?B?a0JwZVI5d09oWGRQaWVyYTFNcEtMNllKTVYxcVIxcUlTTFFlWnBvVEZtRS82?=
 =?utf-8?B?WEh5aC9YR2FOZFh5Ym1IdTJhRzQ5V0JhZk1TeC9Rc1pWR3JNWVVRNUliYW5S?=
 =?utf-8?B?U2orTjhwUkVGSG9nNWErSnl0RUYrazV6RjFCallVaXJuREFDTWwydmNibHU0?=
 =?utf-8?B?WDJZekVUZzJ6UCtuenA1NUg0YzFiclFIYVdoL1FIM3gzUG1hTHNFd0FsZndX?=
 =?utf-8?B?QkNkZlpxZk5RUHhrWFJRb1hGUmNkdkVZWkE5SkJlWlVvWU9YaVdXdm1DbHJT?=
 =?utf-8?B?dEhYNGRrM215bDBpWis4T3V0ZGN5U0FsVkh5TXladWIrVVFqZHptbDY5d1Zv?=
 =?utf-8?B?aklqK0NNMGRoLzZDR0k1N0MvR0VvYWcvanNMQS9KbVB5L1dNeUh3T3MyZ1Uw?=
 =?utf-8?B?V1BGNll1STRQSlhWMy82NHRraG9wWE9Od2V6bFdkYkhqdklqdXN5TmtlM0hE?=
 =?utf-8?B?NDhMK2l3eks2Z3d1TjlZQm5ZV0NLR01qR1k0MkVTQzdXU3NHZHJtdjBGR2wr?=
 =?utf-8?B?dkMrRitpMHd6UGYzVUVJNk9HRm81ZUVLc055NmdOMEc0aVNUS1d3d2VZd2ZZ?=
 =?utf-8?B?RjdUR1BJazVTM3dDdTZMUmx6c1RaNHBpSVNzWENkN1I5WjhXU1RjQ3pYMmhy?=
 =?utf-8?B?WW54K1RiRktSNlkzUkN1RkdRSG5tZFVOWmlDNUcrUjlldVFTMzJUODduU0Zw?=
 =?utf-8?B?TFU0cWdxYTZYczcxMld2OXdibm5rak10NkdsS1hzeDB0NGxCS2RsNzh3RTRy?=
 =?utf-8?B?dkxZOHZyOE9BbEJaakgrd3ZKUUI4SDNIRVJuVGZoRnNBL25hWHg0ektwWTdn?=
 =?utf-8?B?emxTNkZyVUVTUkpTUWFEYjU2VkZxUlRkSDFqNDVKR3hwWVZXRTQwc2NNb216?=
 =?utf-8?B?dzBvZDNqL0VyK2V1M1EydHVyWXdTVWo2cWdHbHpLSTB5ZHV0cXBBZlJBcCtL?=
 =?utf-8?B?TkZWekNSOFl5eDJJZEVGQzdIMkNQRThCd1lxWS9sZHhRMVVzZ0hLMWRBMXlK?=
 =?utf-8?B?Z2kzVWc0NHRUNmRZRU1vcGIvMktTNFNjQnZ1cGorb3lMZzQ0T3ZGQlppekQy?=
 =?utf-8?B?NW0ySzhjQU5OR1h2WDNBOTdnM21pWDREVDBnSVRhWGg4S29KaUhNcllDakJj?=
 =?utf-8?B?V2pDRmtWNW1vNUVDcWFrVkVPdGdRdXVHaDVQRElXQ3RkbE1TUGhuR2pPRVpq?=
 =?utf-8?B?U1ppVVdFTC85SFpnMmlhVC9FZzA3dEIrL0JDUHB1QlNYMlJsMmFabFZpYXVO?=
 =?utf-8?B?cnNJS2ZVckN1Wk5Wc1Z1cXNmbi9STER4KzBhN0d0TUR6aFVvLzI4SXhhbFZY?=
 =?utf-8?B?NFhOZy9jN2xvU1ZVT0ZzUT09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <6651BF4B12F53E449BC875744E7EF26A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS1PR08MB7540
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF0000A79C.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0f89821e-5d33-4e3c-c318-08dc7a65c9aa
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?a3hIdWkxU1FMdFBlY0FSa3AvTE00OVFWdEpjSGhPbTViMEh0Vm5RUTdsQnJY?=
 =?utf-8?B?NUc0YnNVaUxtWVNTUHVTQzZKYXlENnZ3VktlYW5HbGpQYW5aSnIwTzdEbzhC?=
 =?utf-8?B?UWNJZG5KRlJ5ekhnZ0F1U2ovZm1ib3hjOFJkU3FKU1hpVWQ2bC9wM3NGdUt6?=
 =?utf-8?B?VEgxa0k0MmN2eks4eGxkaDNNZHc3SWs3UUdlbXU3cEsrR0VTNXBEdlJmN2Qw?=
 =?utf-8?B?QnhZMlZGTG5QN0VBanhWa05lSXlOM0RNaExwL29WclFrZWxzMUVVeVllL0dX?=
 =?utf-8?B?YUc4VVFpZmFraXRVVU1JSlcyUmJ6WkFJNklpSDkwWVZFaTc0RnR2MHNRN25P?=
 =?utf-8?B?bndrZHpzb1V5bzhmVE91Y0E3Y0p6ZG5pUjRPUnlKQUF3d25MZ3pPY21mNUd6?=
 =?utf-8?B?N1Qwb0dRWVZTcWpnaHl3VmxZUWlzS1F2eExmV3o2cTlXWnF1SnNiVEc1SlVR?=
 =?utf-8?B?KzlXUDN0SDE5SlF6eHU2RzNhTjlUK2NRVWVvRG15UWY1MFNIN3NlM3NWN2t3?=
 =?utf-8?B?QVEzZExpMmRLVG1Ea3JmUEY1V0lqVjZRVFQ3emhWTkttbUpGaGFkK3BlVXNJ?=
 =?utf-8?B?NnNvWjBCNWlxZVVNR1BxR3pkTy9QbjR6Q29Ibjl1QUF3NVNxc0RTWmtBbXBk?=
 =?utf-8?B?ck1GNG9SODVCTVNpQlNXQ1NwWXV0SnRLdDBldU85QUZXUnR0NVVybVEyWE1y?=
 =?utf-8?B?WlE2SW1YYXdCT0Nma3hwLzMvVlFMc3VENWtrcDE0Y09OR3Rpb1FCWkpXUm93?=
 =?utf-8?B?OG4vTTl4dGJqTEJGbFpCL0t6U2VsdGVzd0FYV1crckZ6RnR2M2RRdDRUSEJl?=
 =?utf-8?B?SitZRmRiQSt0V3pFNlVxL2NEWkpoWkdPMDlLR0JNbFJZUjY1RlAzaWFSM3Vw?=
 =?utf-8?B?dkN5MXRHMmJGUjE5OXhVcEVhVEVqbnU5ellML3pwOWkyY2VZVEY2NzQyRHdv?=
 =?utf-8?B?VzB0SzFKaEVKM2lhMUxuMm9mSy9MWDlVTHovVTJkQ2tFVExBUWtjME4vcWlz?=
 =?utf-8?B?akRUQ1NETWRzbEwyNVcxZ1U3OTgvUlBYRHZTUUxvNU1pc1MyMnpCTDlKWEpN?=
 =?utf-8?B?SjJFRXF6c0ZMR3J1TmlaM01wWVJxdUlSUGZoT3R6V3BDaVVRb2N4dVN1V2R6?=
 =?utf-8?B?bWUxRDdGLzYwQ3FnL2x5a1R3S2RzK3Q0ejR3dUpjZUNMMTgwQVNJY2ZwZGhQ?=
 =?utf-8?B?bkQzVHcvS2c1VkNpUlMzaCswQ0wwTmxpTTVRZHZKaDRuMUtHQ09ZSGxLV1V6?=
 =?utf-8?B?YUdxM0tMbjRCdE05alNOeUl2Qm1yNGs5L3RZWmN4bjdRL3p2dDBIQXJHSGlk?=
 =?utf-8?B?Zy9Jd291TmFtMFJMQksxNjRjVGVtSFNNdC9nNHlKWHUxVHpSNHE3Y2dHNTQz?=
 =?utf-8?B?a1RibUt0dUszYVRrZmNUdEZPb3VVSktlRjdueURjdzBaY20zc0UvbjAzLzlH?=
 =?utf-8?B?TlBzQ0FldHJuSFlyQ0dDbFdXMC94UVNrdlhjbHlyaGJGU3lCRlp4OXRJWVZ1?=
 =?utf-8?B?emxlRmdPT2sxVlg3em9zKzZRM0JkSVpXNE1uL0c2RXAwK0h0RDg3aDRNWFZI?=
 =?utf-8?B?WnN6V0ZIbmJYM3FzWUdubFVDdWJ4RWNjTWNmNTdkdmVVT1hDbkE0RmRaTzVR?=
 =?utf-8?B?aUhQUEV1ZkxCWU5hZ0IxR1lxS0NIRkVhdWdsZ2RKVWJ2Z0N1N2FkTFcveklh?=
 =?utf-8?B?SHFZTjhFcDBpallMNm9GcmZCcDFDSUdnMEh6aS9iUkJ6TzUwa1JueStYNDNq?=
 =?utf-8?B?WllBUGdVUHRRUVp3NnFJNUFPV0RGTytEaXYyWGNEOXRWdmdWRVl5VW52aTJv?=
 =?utf-8?B?cjVmRzNrMXZmQ1YzREl6emViaVpoK1RJMUlEN0p6dlFPc0hhWGlNWi9wSFVo?=
 =?utf-8?Q?/NRAFL6bA3LiQ?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:48:06.0594
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c891ddab-30a0-4429-2d7d-08dc7a65cee8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF0000A79C.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR08MB10213

SGkgSnVsaWVuLA0KDQo+IE9uIDIyIE1heSAyMDI0LCBhdCAxNDoyNSwgSnVsaWVuIEdyYWxsIDxq
dWxpZW5AeGVuLm9yZz4gd3JvdGU6DQo+IA0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9t
bXUvcDJtLmMgYi94ZW4vYXJjaC9hcm0vbW11L3AybS5jDQo+PiBpbmRleCA0MWZjY2EwMTFjZjQu
LmI0OTYyNjZkZWVmNiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9tbXUvcDJtLmMNCj4+
ICsrKyBiL3hlbi9hcmNoL2FybS9tbXUvcDJtLmMNCj4+IEBAIC03NTMsMTcgKzc1Myw5IEBAIHN0
YXRpYyBpbnQgcDJtX21lbV9hY2Nlc3NfcmFkaXhfc2V0KHN0cnVjdCBwMm1fZG9tYWluICpwMm0s
IGdmbl90IGdmbiwNCj4+ICAgICAgcmV0dXJuIHJjOw0KPj4gIH0NCj4+ICAtLyoNCj4+IC0gKiBQ
dXQgYW55IHJlZmVyZW5jZXMgb24gdGhlIHNpbmdsZSA0SyBwYWdlIHJlZmVyZW5jZWQgYnkgcHRl
Lg0KPj4gLSAqIFRPRE86IEhhbmRsZSBzdXBlcnBhZ2VzLCBmb3Igbm93IHdlIG9ubHkgdGFrZSBz
cGVjaWFsIHJlZmVyZW5jZXMgZm9yIGxlYWYNCj4+IC0gKiBwYWdlcyAoc3BlY2lmaWNhbGx5IGZv
cmVpZ24gb25lcywgd2hpY2ggY2FuJ3QgYmUgc3VwZXIgbWFwcGVkIHRvZGF5KS4NCj4+IC0gKi8N
Cj4+IC1zdGF0aWMgdm9pZCBwMm1fcHV0X2wzX3BhZ2UoY29uc3QgbHBhZV90IHB0ZSkNCj4+ICsv
KiBQdXQgYW55IHJlZmVyZW5jZXMgb24gdGhlIHNpbmdsZSA0SyBwYWdlIHJlZmVyZW5jZWQgYnkg
bWZuLiAqLw0KPj4gK3N0YXRpYyB2b2lkIHAybV9wdXRfbDNfcGFnZShtZm5fdCBtZm4sIHAybV90
eXBlX3QgdHlwZSkNCj4+ICB7DQo+PiAtICAgIG1mbl90IG1mbiA9IGxwYWVfZ2V0X21mbihwdGUp
Ow0KPj4gLQ0KPj4gLSAgICBBU1NFUlQocDJtX2lzX3ZhbGlkKHB0ZSkpOw0KPj4gLQ0KPj4gICAg
ICAvKg0KPj4gICAgICAgKiBUT0RPOiBIYW5kbGUgb3RoZXIgcDJtIHR5cGVzDQo+PiAgICAgICAq
DQo+PiBAQCAtNzcxLDE2ICs3NjMsNDMgQEAgc3RhdGljIHZvaWQgcDJtX3B1dF9sM19wYWdlKGNv
bnN0IGxwYWVfdCBwdGUpDQo+PiAgICAgICAqIGZsdXNoIHRoZSBUTEJzIGlmIHRoZSBwYWdlIGlz
IHJlYWxsb2NhdGVkIGJlZm9yZSB0aGUgZW5kIG9mDQo+PiAgICAgICAqIHRoaXMgbG9vcC4NCj4+
ICAgICAgICovDQo+PiAtICAgIGlmICggcDJtX2lzX2ZvcmVpZ24ocHRlLnAybS50eXBlKSApDQo+
PiArICAgIGlmICggcDJtX2lzX2ZvcmVpZ24odHlwZSkgKQ0KPj4gICAgICB7DQo+PiAgICAgICAg
ICBBU1NFUlQobWZuX3ZhbGlkKG1mbikpOw0KPj4gICAgICAgICAgcHV0X3BhZ2UobWZuX3RvX3Bh
Z2UobWZuKSk7DQo+PiAgICAgIH0NCj4+ICAgICAgLyogRGV0ZWN0IHRoZSB4ZW5oZWFwIHBhZ2Ug
YW5kIG1hcmsgdGhlIHN0b3JlZCBHRk4gYXMgaW52YWxpZC4gKi8NCj4+IC0gICAgZWxzZSBpZiAo
IHAybV9pc19yYW0ocHRlLnAybS50eXBlKSAmJiBpc194ZW5faGVhcF9tZm4obWZuKSApDQo+PiAr
ICAgIGVsc2UgaWYgKCBwMm1faXNfcmFtKHR5cGUpICYmIGlzX3hlbl9oZWFwX21mbihtZm4pICkN
Cj4+ICAgICAgICAgIHBhZ2Vfc2V0X3hlbmhlYXBfZ2ZuKG1mbl90b19wYWdlKG1mbiksIElOVkFM
SURfR0ZOKTsNCj4+ICB9DQo+IA0KPiBBbGwgdGhlIHBhZ2VzIHdpdGhpbiBhIDJNQiBtYXBwaW5n
IHNob3VsZCBiZSB0aGUgc2FtZSB0eXBlLiBTby4uLg0KPiANCj4+ICArLyogUHV0IGFueSByZWZl
cmVuY2VzIG9uIHRoZSBzdXBlcnBhZ2UgcmVmZXJlbmNlZCBieSBtZm4uICovDQo+PiArc3RhdGlj
IHZvaWQgcDJtX3B1dF9sMl9zdXBlcnBhZ2UobWZuX3QgbWZuLCBwMm1fdHlwZV90IHR5cGUpDQo+
PiArew0KPj4gKyAgICB1bnNpZ25lZCBpbnQgaTsNCj4+ICsNCj4+ICsgICAgZm9yICggaSA9IDA7
IGkgPCBYRU5fUFRfTFBBRV9FTlRSSUVTOyBpKysgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBw
Mm1fcHV0X2wzX3BhZ2UobWZuLCB0eXBlKTsNCj4+ICsNCj4+ICsgICAgICAgIG1mbiA9IG1mbl9h
ZGQobWZuLCAxKTsNCj4+ICsgICAgfQ0KPiANCj4gLi4uIHRoaXMgc29sdXRpb24gaXMgYSBiaXQg
d2FzdGVmdWwgYXMgd2Ugd2lsbCBub3cgY2FsbCBwMm1fcHV0X2wzX3BhZ2UoKSA1MTIgdGltZXMg
ZXZlbiB0aG91Z2ggdGhlcmUgaXMgbm90aGluZyB0byBkby4NCj4gDQo+IFNvIGluc3RlYWQgY2Fu
IHdlIG1vdmUgdGhlIGNoZWNrcyBvdXRzaWRlIHRvIG9wdGltaXplIHRoZSBwYXRoIGEgYml0Pw0K
DQpZb3UgbWVhbiB0aGlzPw0KDQpkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL21tdS9wMm0uYyBi
L3hlbi9hcmNoL2FybS9tbXUvcDJtLmMNCmluZGV4IGI0OTYyNjZkZWVmNi4uZDQwY2RkZGE0OGYz
IDEwMDY0NA0KLS0tIGEveGVuL2FyY2gvYXJtL21tdS9wMm0uYw0KKysrIGIveGVuL2FyY2gvYXJt
L21tdS9wMm0uYw0KQEAgLTc5NCw3ICs3OTQsOCBAQCBzdGF0aWMgdm9pZCBwMm1fcHV0X3BhZ2Uo
Y29uc3QgbHBhZV90IHB0ZSwgdW5zaWduZWQgaW50IGxldmVsKQ0KICAgICBBU1NFUlQocDJtX2lz
X3ZhbGlkKHB0ZSkpOw0KIA0KICAgICAvKiBXZSBoYXZlIGEgc2Vjb25kIGxldmVsIDJNIHN1cGVy
cGFnZSAqLw0KLSAgICBpZiAoIHAybV9pc19zdXBlcnBhZ2UocHRlLCBsZXZlbCkgJiYgKGxldmVs
ID09IDIpICkNCisgICAgaWYgKCBwMm1faXNfc3VwZXJwYWdlKHB0ZSwgbGV2ZWwpICYmIChsZXZl
bCA9PSAyKSAmJg0KKyAgICAgICAgIHAybV9pc19mb3JlaWduKHB0ZS5wMm0udHlwZSkgKQ0KICAg
ICAgICAgcmV0dXJuIHAybV9wdXRfbDJfc3VwZXJwYWdlKG1mbiwgcHRlLnAybS50eXBlKTsNCiAg
ICAgZWxzZSBpZiAoIGxldmVsID09IDMgKQ0KICAgICAgICAgcmV0dXJuIHAybV9wdXRfbDNfcGFn
ZShtZm4sIHB0ZS5wMm0udHlwZSk7DQoNCg0KPiBPdGhlcndpc2UuLi4NCj4gDQo+PiArfQ0KPj4g
Kw0KPj4gKy8qIFB1dCBhbnkgcmVmZXJlbmNlcyBvbiB0aGUgcGFnZSByZWZlcmVuY2VkIGJ5IHB0
ZS4gKi8NCj4+ICtzdGF0aWMgdm9pZCBwMm1fcHV0X3BhZ2UoY29uc3QgbHBhZV90IHB0ZSwgdW5z
aWduZWQgaW50IGxldmVsKQ0KPj4gK3sNCj4+ICsgICAgbWZuX3QgbWZuID0gbHBhZV9nZXRfbWZu
KHB0ZSk7DQo+PiArDQo+PiArICAgIEFTU0VSVChwMm1faXNfdmFsaWQocHRlKSk7DQo+PiArDQo+
PiArICAgIC8qIFdlIGhhdmUgYSBzZWNvbmQgbGV2ZWwgMk0gc3VwZXJwYWdlICovDQo+PiArICAg
IGlmICggcDJtX2lzX3N1cGVycGFnZShwdGUsIGxldmVsKSAmJiAobGV2ZWwgPT0gMikgKQ0KPj4g
KyAgICAgICAgcmV0dXJuIHAybV9wdXRfbDJfc3VwZXJwYWdlKG1mbiwgcHRlLnAybS50eXBlKTsN
Cj4+ICsgICAgZWxzZSBpZiAoIGxldmVsID09IDMgKQ0KPj4gKyAgICAgICAgcmV0dXJuIHAybV9w
dXRfbDNfcGFnZShtZm4sIHB0ZS5wMm0udHlwZSk7DQo+PiArfQ0KPj4gKw0KPj4gIC8qIEZyZWUg
bHBhZSBzdWItdHJlZSBiZWhpbmQgYW4gZW50cnkgKi8NCj4+ICBzdGF0aWMgdm9pZCBwMm1fZnJl
ZV9lbnRyeShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtLA0KPj4gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGxwYWVfdCBlbnRyeSwgdW5zaWduZWQgaW50IGxldmVsKQ0KPj4gQEAgLTgwOSw5ICs4
MjgsMTYgQEAgc3RhdGljIHZvaWQgcDJtX2ZyZWVfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAy
bSwNCj4+ICAjZW5kaWYNCj4+ICAgICAgICAgICAgcDJtLT5zdGF0cy5tYXBwaW5nc1tsZXZlbF0t
LTsNCj4+IC0gICAgICAgIC8qIE5vdGhpbmcgdG8gZG8gaWYgdGhlIGVudHJ5IGlzIGEgc3VwZXIt
cGFnZS4gKi8NCj4+IC0gICAgICAgIGlmICggbGV2ZWwgPT0gMyApDQo+PiAtICAgICAgICAgICAg
cDJtX3B1dF9sM19wYWdlKGVudHJ5KTsNCj4+ICsgICAgICAgIC8qDQo+PiArICAgICAgICAgKiBU
T0RPOiBDdXJyZW50bHkgd2UgZG9uJ3QgaGFuZGxlIDFHQiBzdXBlci1wYWdlLCBYZW4gaXMgbm90
DQo+PiArICAgICAgICAgKiBwcmVlbXB0aWJsZSBhbmQgdGhlcmVmb3JlIHNvbWUgd29yayBpcyBu
ZWVkZWQgdG8gaGFuZGxlIHN1Y2gNCj4+ICsgICAgICAgICAqIHN1cGVycGFnZXMsIGZvciB3aGlj
aCBhdCBzb21lIHBvaW50IFhlbiBtaWdodCBlbmQgdXAgZnJlZWluZyBtZW1vcnkNCj4+ICsgICAg
ICAgICAqIGFuZCB0aGVyZWZvcmUgZm9yIHN1Y2ggYSBiaWcgbWFwcGluZyBpdCBjb3VsZCBlbmQg
dXAgaW4gYSB2ZXJ5IGxvbmcNCj4+ICsgICAgICAgICAqIG9wZXJhdGlvbi4NCj4+ICsgICAgICAg
ICAqLw0KPj4gKyAgICAgICAgaWYgKCBsZXZlbCA+PSAyICkNCj4+ICsgICAgICAgICAgICBwMm1f
cHV0X3BhZ2UoZW50cnksIGxldmVsKTsNCj4+ICsNCj4+ICAgICAgICAgIHJldHVybjsNCj4+ICAg
ICAgfQ0KPj4gIEBAIC0xNTU4LDkgKzE1ODQsMTIgQEAgaW50IHJlbGlucXVpc2hfcDJtX21hcHBp
bmcoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICAgICAgICAgICAgY291bnQrKzsNCj4+ICAgICAgICAg
IC8qDQo+PiAtICAgICAgICAgKiBBcmJpdHJhcmlseSBwcmVlbXB0IGV2ZXJ5IDUxMiBpdGVyYXRp
b25zLg0KPj4gKyAgICAgICAgICogQXJiaXRyYXJpbHkgcHJlZW1wdCBldmVyeSA1MTIgaXRlcmF0
aW9ucyBvciB3aGVuIHR5cGUgaXMgZm9yZWlnbg0KPj4gKyAgICAgICAgICogbWFwcGluZyBhbmQg
dGhlIG9yZGVyIGlzIGFib3ZlIDkgKDJNQikuDQo+PiAgICAgICAgICAgKi8NCj4+IC0gICAgICAg
IGlmICggIShjb3VudCAlIDUxMikgJiYgaHlwZXJjYWxsX3ByZWVtcHRfY2hlY2soKSApDQo+PiAr
ICAgICAgICBpZiAoICghKGNvdW50ICUgNTEyKSB8fA0KPj4gKyAgICAgICAgICAgICAgKHAybV9p
c19mb3JlaWduKHQpICYmIChvcmRlciA+IFhFTl9QVF9MRVZFTF9PUkRFUigyKSkpKSAmJg0KPiAN
Cj4gLi4uIHdlIHdvdWxkIG5lZWQgdG8gcHJlZW1wdCBmb3IgZXZlcnkgMk1CIHJhdGhlciB0aGFu
IGp1c3QgZm9yIHRoZSBwMm1faXNfZm9yZWlnbigpLg0KDQpPayBvdGhlcndpc2UgeW91IGFyZSBz
dWdnZXN0aW5nIHRoYXQgaWYgd2UgZG9u4oCZdCBnbyBmb3IgdGhlIHNvbHV0aW9uIGFib3ZlIHdl
IGRyb3AgcDJtX2lzX2ZvcmVpZ24odCkgZnJvbQ0KdGhlIGNvbmRpdGlvbiBoZXJlLCBhbSBJIHJp
Z2h0Pw0KDQo+IA0KPiBCVFcsIHAybV9wdXRfbDNfcGFnZSgpIGhhcyBhbHNvIGFub3RoZXIgY2Fz
ZS4gU2hvdWxkIHdlIGNvbnNpZGVyIHRvIGhhbmRsZSBwcmVlbXB0aW9uIGZvciBpdCB0b28/DQoN
CllvdSBtZWFuIGNoZWNraW5nIGZvciA1MTIgaXRlcmF0aW9ucywgb3IgZm9yZWlnbiBtYXBwaW5n
IHdoZW4gb3JkZXIgaXMgPiA5LCBvcg0KcDJtX2lzX3JhbSh0eXBlKSAmJiBpc194ZW5faGVhcF9t
Zm4obWZuKSA/DQoNCkp1c3Qgd2FudCB0byBiZSBzdXJlIEkgZnVsbHkgdW5kZXJzdGFuZCB5b3Vy
IGNvbW1lbnRzIGhlcmUuDQoNCkNoZWVycywNCkx1Y2ENCg0K


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:49:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:49:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727696.1132316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mLD-00053Q-Bi; Wed, 22 May 2024 13:49:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727696.1132316; Wed, 22 May 2024 13:49:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mLD-00053J-7x; Wed, 22 May 2024 13:49:15 +0000
Received: by outflank-mailman (input) for mailman id 727696;
 Wed, 22 May 2024 13:49:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9mLC-00036q-NW
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:49:14 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 12e1da4d-1842-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:49:14 +0200 (CEST)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-6a077a861e7so37343446d6.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:49:14 -0700 (PDT)
Received: from [10.80.67.140] ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1cd1d1sm133129086d6.90.2024.05.22.06.49.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:49:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12e1da4d-1842-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716385753; x=1716990553; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=wIhu189B01nDbbjYkoJQBwtv69wN4Gpws0k4pLKdsVI=;
        b=a8tpKI4gYtzyNuJfh0HZoqZshyjpylRlrfil2IUrV2N4USsb0lCEil3eRbrWh4J8uf
         ChMKcdvlQuAvobBxEYMMkycEt5jEapGEaVkzDodWxgmMbxKU/LWjwHxfa11ZgFKrSDc2
         wuC6EbtH1GZXoRNcnUVrBhldpVfvTZ8FoYz04=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716385753; x=1716990553;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=wIhu189B01nDbbjYkoJQBwtv69wN4Gpws0k4pLKdsVI=;
        b=QMxwvjvZp9YCDuFOjCQPIywID9X3KLmM43oaIlma2lq9F91sBA5tWtdzbPW4UEqoU+
         vXTe+i1xTnsFSIsn4ihwk81xoC2EhQW8CXHnGaBCe1YiIn4Vmv5dOr9w7jZIrBBGcvSk
         Yoakh6DLEA+aPCyVCIMz65+5pYZIR2/rkARyq/f6PYjGXRWmxjoLTzorNyJLzLXF9j6h
         IaDChiZdMqEfkmcwC0HE19+YuaZ7jEkz28FIwvmc9XPXZNsQotenx+4GSA01nSlfTxK5
         RgKmqx/EvM9xQRdE2DbEVcabt18I1K4tatyddHYdnFbY6qzr+s4eXpY1duz/zXAMO0KN
         u02w==
X-Forwarded-Encrypted: i=1; AJvYcCUtt2MwEvBySlpujpNKaHWx554bZPOzSV3+hI/A5APzt4mRNPk8apeEo2zQBdrlFyZ+kPDdxcGHXYgobdpt6g7ZH57eTztc8772GMhsE+g=
X-Gm-Message-State: AOJu0YwJ27vsMGz52LKT1kK+19r/D/vROkLahorIOPMAnrx6+Qdb6hDN
	eWEemKlnk+jRtH5JhUREcY2Qa5KkX2DOA65IfZa1LM4xu8q96LHlsdhljVRW5n0m/KBu1VPbD5K
	jew4=
X-Google-Smtp-Source: AGHT+IFA0cSXGztNyHfP10CZCoe+4yNmjJtuJ6PmejheNzVxFkJOzSdzf4+KFdAc8eJS42q+dyNfJA==
X-Received: by 2002:a05:6214:2f11:b0:6ab:8bb0:4f05 with SMTP id 6a1803df08f44-6ab8bb05022mr4623436d6.34.1716385752988;
        Wed, 22 May 2024 06:49:12 -0700 (PDT)
Message-ID: <f8eb1c08-c168-4259-a221-4719c91afb3b@citrix.com>
Date: Wed, 22 May 2024 14:49:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: New Defects reported by Coverity Scan for XenProject
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <664dc165759df_5e9362b92d249399c762@prd-scan-dashboard-0.mail>
 <1b607d3b-429e-4841-a3b0-ed3a39e12ed7@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <1b607d3b-429e-4841-a3b0-ed3a39e12ed7@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22/05/2024 11:05 am, Jan Beulich wrote:
> On 22.05.2024 11:56, scan-admin@coverity.com wrote:
>> ** CID 1598431:  Memory - corruptions  (OVERRUN)
>>
>>
>> ________________________________________________________________________________________________________
>> *** CID 1598431:  Memory - corruptions  (OVERRUN)
>> /xen/common/trace.c: 798 in trace()
>> 792         }
>> 793     
>> 794         if ( rec_size > bytes_to_wrap )
>> 795             insert_wrap_record(buf, rec_size);
>> 796     
>> 797         /* Write the original record */
>>>>>     CID 1598431:  Memory - corruptions  (OVERRUN)
>>>>>     Overrunning callee's array of size 28 by passing argument "extra" (which evaluates to 31) in call to "__insert_record".
>> 798         __insert_record(buf, event, extra, cycles, rec_size, extra_data);
>> 799     
>> 800     unlock:
>> 801         spin_unlock_irqrestore(&this_cpu(t_lock), flags);
>> 802     
>> 803         /* Notify trace buffer consumer that we've crossed the high water mark. */
> How does the tool conclude "extra" evaluating to 31, when at the top of
> the function it is clearly checked to be less than 28?

Which "top" ?

The reasoning is:

 2. Condition extra % 4UL /* sizeof (uint32_t) */, taking false branch.
 3. Condition extra / 4UL /* sizeof (uint32_t) */ > 7, taking false branch.
 4. cond_at_most: Checking extra / 4UL > 7UL implies that extra may be
up to 31 on the false branch.

which is where 31 comes from.

What Coverity hasn't done is equated "<31 && multiple of 4" to mean
"<28".  I don't think this is unreasonable; analysis has to prune the
reasoning somewhere...

This is (fundamentally) a dumb-ABI problem where we're passing a byte
count but only ever wanting to use it as a unit-of-uint32_t's count.

But it's also problem that we're passing both extra and rec_size into
__insert_record() when one is calculated from the other.

I had decided to leave this alone for now, but maybe it could do with
some improvements (simplifications) to the code.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:50:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:50:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727700.1132326 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mM9-0006Zn-Jz; Wed, 22 May 2024 13:50:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727700.1132326; Wed, 22 May 2024 13:50:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mM9-0006Zg-HB; Wed, 22 May 2024 13:50:13 +0000
Received: by outflank-mailman (input) for mailman id 727700;
 Wed, 22 May 2024 13:50:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DzQV=MZ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1s9mM8-0005ZK-PA
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:50:12 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20622.outbound.protection.outlook.com
 [2a01:111:f400:7e88::622])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3422bd5e-1842-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:50:10 +0200 (CEST)
Received: from SJ0P220CA0027.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:41b::18)
 by IA1PR12MB8467.namprd12.prod.outlook.com (2603:10b6:208:448::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Wed, 22 May
 2024 13:50:06 +0000
Received: from CO1PEPF000042AE.namprd03.prod.outlook.com
 (2603:10b6:a03:41b:cafe::4a) by SJ0P220CA0027.outlook.office365.com
 (2603:10b6:a03:41b::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 13:50:06 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000042AE.mail.protection.outlook.com (10.167.243.43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 13:50:05 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:50:03 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 08:50:01 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3422bd5e-1842-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=amuv4wdsOd0sQpS08Rb1xjpPkiy2ChXfbTPI2Z+9am+dtdjwGD7h2S3xp22oQUOFeWb7GNbxSyW+ThABmS+lv/0vmfjqSRh/TFsThFUlSOSopq2Y7UCYc1CQE7rdrPAnAyDHGh6zgCc/C57KFeqcKzOFwXW/l6LhdShzEcX649OVs5L5x7X7mjoNJSFlk1AaltlI0KXCKR9cNOZN3VNWowpUZFjUYzBolIgtRju1nR5XE13vn0hQOSMxxofnNEO9s5O+1Hn4yecVDTeI8JyZkoy6cQ8BdmMQGwAa98dWKdtGGLQjY5B46MW9z6lQqJuRdChaMo5EhGRWnZa4QL7X7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hpin80pWJf11sF4u2kqwI0Qs5OSeNVouPFEGB027mxw=;
 b=cjuNP/0yLYobKyTZ7DUaGHCnxJUinLiCu0ar5Ix5dQ/VhO71EqwFw7HMfH+Kx2zDRXBAhco4qvohnM6R8rq50HhAIrzBK8A+ZQWz3WCjS+Op1olXNUuwBh0Yq3Xdh6fssWVq9AVJj8yPGl0Ce8xOuZZyBWZDPfORo9Vecigr/tQ/lGIf96vKwl3sqIBOiz3AT1owbEJE8sTAy8YYlVBiPK/AMYVgmi6r43cK5Pb8onAT1QqO8yCDsS0k1o1L0UXqs94kTbFDo3SZ/snJjdq67G3xulbzOJ2DK2y33SUYJVbaNErGS4TmIehA2eeSd+hSwwhfVHBCOHjckkv6cpK34w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hpin80pWJf11sF4u2kqwI0Qs5OSeNVouPFEGB027mxw=;
 b=HaJpHjuxvlDW8mLazGpZqxtEjauTjyc5G+pbSGx4xXr1hTpzZymnR3djt/dfuZZp2o6uExNBMJkCFveFifZqJi0wRTnwUD3LBBPnwlUjDdYg9o0W/SVbVaSJNOAwr8OxXTM1RftcXVCG1V3GhFSkJCZXr1cxckv4jdDPhvW2xoY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <5a339b77-0ce6-4670-9e79-99be5ac48bb2@amd.com>
Date: Wed, 22 May 2024 15:50:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/7] xen/arm: Parse xen,shared-mem when host phys
 address is not provided
To: Luca Fancellu <luca.fancellu@arm.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-5-luca.fancellu@arm.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240522075151.3373899-5-luca.fancellu@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000042AE:EE_|IA1PR12MB8467:EE_
X-MS-Office365-Filtering-Correlation-Id: 49b2ef37-de84-4956-f930-08dc7a661659
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eU9jZFdhQ0wyc2h3cW1VbFJsYVB4eTdLdnZ4Y3ZadUtBRUh6M1NwQnpzZEZn?=
 =?utf-8?B?OUphQjdINUluSTBoTnFJSnJLWHIwSW1EdktYWFhMSUp1RUJJWkVUaGZBdjVZ?=
 =?utf-8?B?cWNORzgvdDZnQjB3SjQ0cmdQNzVMdENBeDJTMXRLU25wbzlVRVh5RzFEKzhW?=
 =?utf-8?B?QWdZdWhROGpZT0l4Zk5GYkNuWnNWYm9tUWJxT3BOZVN0eGM2VUIxSUlsK3hB?=
 =?utf-8?B?aVo3RlJMcXNrUGlaVkNVekNlbGZQWWZHNmxsbzhBakFNUnFLRFF2VzF2OERL?=
 =?utf-8?B?ajVUTE9jWElCMENGYzVTYWVNVnhEWjAyRExiRGkvK21vRnpQam1sb1NoaGQv?=
 =?utf-8?B?S0pHKzJJejhueHRCbXBVS3RiSG9YR1kweGpuQmdsdnNkaE84SmJJZytVU0tt?=
 =?utf-8?B?ZzJFVUZ2NFNVUHYwR3pSVW1hVkgzNHJuZlBzR25oNGFIZmE3bk9jb2ZjbFJp?=
 =?utf-8?B?dWVOb1hXUnM3a1MxM21NbHF6b0tsY2pCYVp5Y05lTDQ0cnZocDdNOTdieExa?=
 =?utf-8?B?dW11MFp6cFVlUHlmZWw4WjVSZUN5ZmRTZEk1U0pVZGU0UVMzd0M1VWkzaWhX?=
 =?utf-8?B?UXNlVUI3aUo4R0RyYkFrVzJFS0hVQmNxVW9rN0JmMVkxdndEdWFWeXpwdEV5?=
 =?utf-8?B?VElmaXV2MFhocDVScDBxV2ZqT01pYlhhdTZpK2ZMZHg4aTdKdWZDcFNWemkx?=
 =?utf-8?B?SjVFQS9YQ2cyVlVvVFVJemFFTXFtTi9FUzJTT3hGdXVacy9LSWkxS3J6V2xl?=
 =?utf-8?B?MVpOcG1OVEhIdnkrWEkvTEpxYU1uOC8yaGd2S0FLazY4VElsdkRtNVFod2Ja?=
 =?utf-8?B?QUlLMDFFc1hWdTRvNVkwVkprdGRZeU55OThFbmdubzFOY0hCcWNaTUJ6cG1v?=
 =?utf-8?B?V3o0anZZR2VobW00KzFFVnAyaUk4RGNUYmNHbEFFTVhBZm5ZMytiM1owYlFL?=
 =?utf-8?B?czg0VzlrQlVKdFQ1K2tBNlkxREdTWmNya0dPalRWcUw0VWNjS0hWNUtZSmNZ?=
 =?utf-8?B?ZXUydWtNaDdXblZCbHRBMzVIQkdWMGNyMGlZUEJtY3lDcllyOWdObmhyaVpP?=
 =?utf-8?B?RGhqYml1S1VCOFV2bnoxMDJyVXdwQW9GK01pMGp1ZVZZTCs5RWdGUWVjVjBU?=
 =?utf-8?B?amRhL05TL1hBYXMxNHQwOUxlK2lTRHk2V3E5UUdINE9ZakN0QnRzRWNKQlE5?=
 =?utf-8?B?eWNSUVdmSTNERUJpdWkydkZ5c3F6c2VuVXZSVVNFQnZRWGRUay8yRDNPUkla?=
 =?utf-8?B?bmdQbThDUzU0ZG9OWk10cllEaVJLV3lCOWRIaTIwS203VGtDaU1Ua0hrbURQ?=
 =?utf-8?B?dWd5amwxckw2MVZWbGkrWVJwNWtXOEV1Q1owaXRZbXVDeFlob3NsSlNVckJ4?=
 =?utf-8?B?ZXllY1R2SlY0UGUxa3hMN2kvYVFpYng0STBtRkM0clp1Z05LdlFFeU5TTEh5?=
 =?utf-8?B?b3FUdkFXLzBZMVJWSjQzNVdRY1VtOC8wQ1E2aW0zbEdaaHVGVWo4OXBnZ2ky?=
 =?utf-8?B?aE1SMVNjNmhNVHA1QmJDd0RZZ1NISlA3bkZYZU43QWtrdTIrSXRrZUpxMU0r?=
 =?utf-8?B?WkgvU3ZYL1lkRHFiSDE0WWRSa0RvRDFnY3RXYWprajBNMlpqREo4ZkFtdEpi?=
 =?utf-8?B?VDJXTDNqem5zVFYxaWlISW1JVXpTbjdsU0FQaFF3NnZiNHhZWDFhWWU3OHE1?=
 =?utf-8?B?NDNGdGpLaXBPTjZEN0RwbXVuWUFFWlZzcGp1Z05RSHY1QUJyU0R5dkc5OGJv?=
 =?utf-8?B?eWhmOUhzcGxwZ2RsQjdaSEs0VmJzSGZZMWhPS1VmalhtL2thU1FMT083clRO?=
 =?utf-8?B?dXp6ejh3S3JOemhhL0VQcGE0OTJlcGJxRjJqZ2M4UjJnenhxZ3huQXdxRHVq?=
 =?utf-8?Q?k/cZXylDZOx5b?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:50:05.8140
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 49b2ef37-de84-4956-f930-08dc7a661659
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000042AE.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8467

Hi Luca,

On 22/05/2024 09:51, Luca Fancellu wrote:
> 
> 
> Handle the parsing of the 'xen,shared-mem' property when the host physical
> address is not provided, this commit is introducing the logic to parse it,
> but the functionality is still not implemented and will be part of future
> commits.
> 
> Rework the logic inside process_shm_node to check the shm_id before doing
> the other checks, because it ease the logic itself, add more comment on
> the logic.
> Now when the host physical address is not provided, the value
> INVALID_PADDR is chosen to signal this condition and it is stored as
> start of the bank, due to that change also early_print_info_shmem and
> init_sharedmem_pages are changed, to not handle banks with start equal
> to INVALID_PADDR.
> 
> Another change is done inside meminfo_overlap_check, to skip banks that
> are starting with the start address INVALID_PADDR, that function is used
> to check banks from reserved memory, shared memory and ACPI and since
> the comment above the function states that wrapping around is not handled,
> it's unlikely for these bank to have the start address as INVALID_PADDR.
> Same change is done inside consider_modules, find_unallocated_memory and
> dt_unreserved_regions functions, in order to skip banks that starts with
> INVALID_PADDR from any computation.
> The changes above holds because of this consideration.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
> ---
> v3 changes:
>  - fix typo in commit msg, add R-by Michal
> v2 changes:
>  - fix comments, add parenthesis to some conditions, remove unneeded
>    variables, remove else branch, increment counter in the for loop,
>    skip INVALID_PADDR start banks from also consider_modules,
>    find_unallocated_memory and dt_unreserved_regions. (Michal)
> ---
>  xen/arch/arm/arm32/mmu/mm.c |  11 +++-
>  xen/arch/arm/domain_build.c |   5 ++
>  xen/arch/arm/setup.c        |  14 +++-
>  xen/arch/arm/static-shmem.c | 125 +++++++++++++++++++++++++-----------
>  4 files changed, 111 insertions(+), 44 deletions(-)
> 
> diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c
> index be480c31ea05..30a7aa1e8e51 100644
> --- a/xen/arch/arm/arm32/mmu/mm.c
> +++ b/xen/arch/arm/arm32/mmu/mm.c
> @@ -101,8 +101,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
>      nr += reserved_mem->nr_banks;
>      for ( ; i - nr < shmem->nr_banks; i++ )
>      {
> -        paddr_t r_s = shmem->bank[i - nr].start;
> -        paddr_t r_e = r_s + shmem->bank[i - nr].size;
> +        paddr_t r_s, r_e;
> +
> +        r_s = shmem->bank[i - nr].start;
> +
> +        /* Shared memory banks can contain INVALID_PADDR as start */
> +        if ( INVALID_PADDR == r_s )
> +            continue;
> +
> +        r_e = r_s + shmem->bank[i - nr].size;
> 
>          if ( s < r_e && r_s < e )
>          {
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 968c497efc78..02e741685102 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -927,6 +927,11 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
>          for ( j = 0; j < mem_banks[i]->nr_banks; j++ )
>          {
>              start = mem_banks[i]->bank[j].start;
> +
> +            /* Shared memory banks can contain INVALID_PADDR as start */
> +            if ( INVALID_PADDR == start )
> +                continue;
> +
>              end = mem_banks[i]->bank[j].start + mem_banks[i]->bank[j].size;
>              res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
>                                          PFN_DOWN(end - 1));
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index c4e5c19b11d6..0c2fdaceaf21 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -240,8 +240,15 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
>      offset = reserved_mem->nr_banks;
>      for ( ; i - offset < shmem->nr_banks; i++ )
>      {
> -        paddr_t r_s = shmem->bank[i - offset].start;
> -        paddr_t r_e = r_s + shmem->bank[i - offset].size;
> +        paddr_t r_s, r_e;
> +
> +        r_s = shmem->bank[i - offset].start;
> +
> +        /* Shared memory banks can contain INVALID_PADDR as start */
> +        if ( INVALID_PADDR == r_s )
> +            continue;
> +
> +        r_e = r_s + shmem->bank[i - offset].size;
> 
>          if ( s < r_e && r_s < e )
>          {
> @@ -272,7 +279,8 @@ static bool __init meminfo_overlap_check(const struct membanks *mem,
>          bank_start = mem->bank[i].start;
>          bank_end = bank_start + mem->bank[i].size;
> 
> -        if ( region_end <= bank_start || region_start >= bank_end )
> +        if ( INVALID_PADDR == bank_start || region_end <= bank_start ||
> +             region_start >= bank_end )
>              continue;
>          else
>          {
> diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
> index c15a65130659..74c81904b8a4 100644
> --- a/xen/arch/arm/static-shmem.c
> +++ b/xen/arch/arm/static-shmem.c
> @@ -264,6 +264,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          pbase = boot_shm_bank->start;
>          psize = boot_shm_bank->size;
> 
> +        if ( INVALID_PADDR == pbase )
> +        {
> +            printk("%pd: host physical address must be chosen by users at the moment", d);
> +            return -EINVAL;
> +        }
> +
>          /*
>           * xen,shared-mem = <pbase, gbase, size>;
>           * TODO: pbase is optional.
> @@ -377,7 +383,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>  {
>      const struct fdt_property *prop, *prop_id, *prop_role;
>      const __be32 *cell;
> -    paddr_t paddr, gaddr, size, end;
> +    paddr_t paddr = INVALID_PADDR;
> +    paddr_t gaddr, size, end;
>      struct membanks *mem = bootinfo_get_shmem();
>      struct shmem_membank_extra *shmem_extra = bootinfo_get_shmem_extra();
>      unsigned int i;
> @@ -432,24 +439,37 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>      if ( !prop )
>          return -ENOENT;
> 
> +    cell = (const __be32 *)prop->data;
>      if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) )
>      {
> -        if ( len == dt_cells_to_size(size_cells + address_cells) )
> -            printk("fdt: host physical address must be chosen by users at the moment.\n");
> -
> -        printk("fdt: invalid `xen,shared-mem` property.\n");
> -        return -EINVAL;
> +        if ( len == dt_cells_to_size(address_cells + size_cells) )
> +            device_tree_get_reg(&cell, address_cells, size_cells, &gaddr,
> +                                &size);
> +        else
> +        {
> +            printk("fdt: invalid `xen,shared-mem` property.\n");
> +            return -EINVAL;
> +        }
>      }
> +    else
> +    {
> +        device_tree_get_reg(&cell, address_cells, address_cells, &paddr,
> +                            &gaddr);
> +        size = dt_next_cell(size_cells, &cell);
> 
> -    cell = (const __be32 *)prop->data;
> -    device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
> -    size = dt_next_cell(size_cells, &cell);
> +        if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
> +        {
> +            printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
> +                paddr);
> +            return -EINVAL;
> +        }
> 
> -    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
> -    {
> -        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
> -               paddr);
> -        return -EINVAL;
> +        end = paddr + size;
> +        if ( end <= paddr )
> +        {
> +            printk("fdt: static shared memory region %s overflow\n", shm_id);
> +            return -EINVAL;
> +        }
>      }
> 
>      if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
> @@ -471,39 +491,64 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
>          return -EINVAL;
>      }
> 
> -    end = paddr + size;
> -    if ( end <= paddr )
> -    {
> -        printk("fdt: static shared memory region %s overflow\n", shm_id);
> -        return -EINVAL;
> -    }
> -
>      for ( i = 0; i < mem->nr_banks; i++ )
>      {
>          /*
>           * Meet the following check:
> -         * 1) The shm ID matches and the region exactly match
> -         * 2) The shm ID doesn't match and the region doesn't overlap
> -         * with an existing one
> +         * - when host address is provided:
> +         *   1) The shm ID matches and the region exactly match
> +         *   2) The shm ID doesn't match and the region doesn't overlap
> +         *      with an existing one
> +         * - when host address is not provided:
> +         *   1) The shm ID matches and the region size exactly match
>           */
> -        if ( paddr == mem->bank[i].start && size == mem->bank[i].size )
> +        bool paddr_assigned = (INVALID_PADDR == paddr);
Shouldn't it be INVALID_PADDR != paddr to indicate that paddr was assigned? Otherwise, looking at the
code belowe you would allow a configuration where the shm_id matches but the phys addresses don't.

~Michal


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:51:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:51:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727704.1132336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mMt-00079s-UU; Wed, 22 May 2024 13:50:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727704.1132336; Wed, 22 May 2024 13:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mMt-00079l-QA; Wed, 22 May 2024 13:50:59 +0000
Received: by outflank-mailman (input) for mailman id 727704;
 Wed, 22 May 2024 13:50:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mMs-00072Y-Ph
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:50:58 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 512b9aa3-1842-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:50:58 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51f174e316eso6024873e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:50:58 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57509d459desm8637328a12.61.2024.05.22.06.50.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:50:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 512b9aa3-1842-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716385857; x=1716990657; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IKgX9Mpo9DjdhPN7OV82GrHh7vZqzpG+Te61T+wmyLM=;
        b=atw3XIiu55n6rtVfR4HCX1qD39Gdy7tw95CtSOmomcCYAf4v26VAFnFgElGdql3GPC
         j7irfIeSArKsy3bwPZnXNYOQb7x3W62nqyKHKrfduNmFe1Z21u5cRHVUUpuR7gd3/KAA
         2mewrAt5Z80+K7WcIPB/qLFyE67T8b60K1RQ8ggbxLWW4u+3aIpteK/DrhVxyrOvAl9A
         4HXML+Fov/Mw9S5PM522Tpmh0MwY8JTvB5IWnnQjdcZq/36yhRoGZzwDoDkWPFZh/M4z
         +U0wvw1elxMPx0soWHvRW15wilKNX3iuhnMnG1xivy/n51ont4zMZlHYeXw2WmjCY4kU
         jr7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716385857; x=1716990657;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IKgX9Mpo9DjdhPN7OV82GrHh7vZqzpG+Te61T+wmyLM=;
        b=kayVld6sQq/FSYF7n/6kXy9iVW1BaKV6YkLpdL91v2gSePeCLtdJJywzq1yCBk2cMJ
         KO8sHjRxsqj8E1/wX9bDfcVBiitqjlDAeJK2H80HPsXIt1bwJ9G1cWHpM4BL74cMwksU
         SGbQ9wg5/HqsoSdvoKjR8YF/CMPSrvA9o2RQCNOfdnFjivYM7g6SGNzqUXRuXMTOtpqg
         qRecGZZrL9u9G3yIR2JYpsDuIVTny8mGtJ4f1UXGhtxS3CunkXFz2ZUs0xAl1TKW4tAE
         scb1v9NI3hy21WpKT13MM3qmQbmewTCrnK/FOkd3ofC9jOC6si7d7p4VUKykDNilgzKi
         o89A==
X-Forwarded-Encrypted: i=1; AJvYcCVZP3c2nQenzIwspDlgVNVM+mVvMzxbNUJkE0ZKBjb+GqJF+FHnXGELouGG2DUvqg6LfyeLM72c4VwPIBgzpNlaD85oQXlaDPxTDm33WHs=
X-Gm-Message-State: AOJu0YwFYf4CTkaaLzEfSvjbkCcfbKhogFyqj4qPOtZSsALkb3tFfvvA
	VtVTHClL11rksAJMa96DrbmjziToWyOMmMkuGVa2V+sHJHCL0+guE2+QamOviA==
X-Google-Smtp-Source: AGHT+IGKGDp1At1YEWITjU3d257z4ELkQl5xU5bWEz5rEEUXUmZiVgJPGhOScljpSzMgsWDYUVDEtQ==
X-Received: by 2002:ac2:4c8a:0:b0:523:8c7a:5f7 with SMTP id 2adb3069b0e04-526bebb452amr1221525e87.6.1716385857636;
        Wed, 22 May 2024 06:50:57 -0700 (PDT)
Message-ID: <cdff3b27-a10c-4579-9128-a9a4c8bd96a1@suse.com>
Date: Wed, 22 May 2024 15:50:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/4] x86/shadow: Rework trace_shadow_gen() into
 sh_trace_va()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-2-andrew.cooper3@citrix.com>
 <1140d40f-2f5c-402d-a0fc-1c6598a7a424@suse.com>
 <b3b4f2d0-fb42-44be-bb24-a22c8c237f13@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b3b4f2d0-fb42-44be-bb24-a22c8c237f13@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 15:47, Andrew Cooper wrote:
> On 22/05/2024 2:40 pm, Jan Beulich wrote:
>> On 22.05.2024 15:17, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/mm/shadow/multi.c
>>> +++ b/xen/arch/x86/mm/shadow/multi.c
>>> @@ -1974,13 +1974,17 @@ typedef u32 guest_va_t;
>>>  typedef u32 guest_pa_t;
>>>  #endif
>>>  
>>> -static inline void trace_shadow_gen(u32 event, guest_va_t va)
>>> +/* Shadow trace event with GUEST_PAGING_LEVELS folded into the event field. */
>>> +static void sh_trace(uint32_t event, unsigned int extra, const void *extra_data)
>>> +{
>>> +    trace(event | ((GUEST_PAGING_LEVELS - 2) << 8), extra, extra_data);
>>> +}
>>> +
>>> +/* Shadow trace event with the guest's linear address. */
>>> +static void sh_trace_va(uint32_t event, guest_va_t va)
>>>  {
>>>      if ( tb_init_done )
>>> -    {
>>> -        event |= (GUEST_PAGING_LEVELS-2)<<8;
>>> -        trace(event, sizeof(va), &va);
>>> -    }
>>> +        sh_trace(event, sizeof(va), &va);
>>>  }
>> If any tb_init_done check, then perhaps rather in sh_trace()? With that
>> (and provided you agree)
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Sadly not.  That leads to double reads of tb_init_done when tracing is
> compiled in.

Not here, but I can see how that could happen in principle, when ...

> When GCC can't fully inline the structure initialisation, it can't prove
> that a function call modified tb_init_done.  This is why I arranged all
> the trace cleanup in this way.

... inlining indeed doesn't happen. Patch 2 fits the one here in this regard
(no function calls); I have yet to look at patch 3, though.

But anyway, the present placement, while likely a little redundant, is not
the end of the world, so my R-b holds either way.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:55:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:55:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727715.1132346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mQy-0008Cu-Hp; Wed, 22 May 2024 13:55:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727715.1132346; Wed, 22 May 2024 13:55:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mQy-0008Cn-Eo; Wed, 22 May 2024 13:55:12 +0000
Received: by outflank-mailman (input) for mailman id 727715;
 Wed, 22 May 2024 13:55:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mQx-0008Ch-9e
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:55:11 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e709f4e9-1842-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:55:09 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso2789788a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:55:09 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574eb5ec296sm11232001a12.35.2024.05.22.06.55.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:55:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e709f4e9-1842-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716386109; x=1716990909; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5VfCPRgty1hG9lABhHk5bd6u9p+3qZJD+Cfif/QAzxw=;
        b=dzWonH18rtCsOuLYpoS6yFYy96MjGwK2AimcjPRtRYpYfC07q60w9ZywB5LIjsxz5x
         47jVQFvTv72Cp3QWESCD5wJnm1U88KRWHHEEwiEyRX0FIiem4wNe5ST05RvHU88aLVOx
         FZ64cX9pL1S8Ice0lX+eoaWzDrKmRhsiQGXnMDQ9PPdUgoE+NRbOjzgUdKyyGfBdUqrU
         0mg2QYkhx4oZXizs3CYPedHgpGzt334SwqtE8BBpUL0PL5qnwGbxHEsQxODvcI/VTbSw
         8OotUu8LzeezsbbMQ3U6no0G8w4lD/V0cMEVj2Nu/c4uLZ8fxPlqkw2OVvXQwlSNnWiS
         2C+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716386109; x=1716990909;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5VfCPRgty1hG9lABhHk5bd6u9p+3qZJD+Cfif/QAzxw=;
        b=pQHWvyCTqW1Ns0YRR7QRF/gFsL5MLzP/z90QzpVRpXht9cc3NxJ+L94kTR4aILAPQh
         faaW/K49jmm+G1npwTx+gg/ckrezVO/V3lZU4kD84N8ZHOd+I1owDw9WQTapcc09YbLd
         UrgMCfvypWSX5dOcWDklON1F4S/88x94PfSvhjyQm74XXC0+ERw6y5zfRcF91TPfpLan
         YBYafRiEZkNjlQUWivq4/yk9WCUu8RkuZ7FZAJbSUfK8U3bFJqy7xkZ3DV31+68r/J2L
         H6YQBnFNumLFVaVSi3iAvHBHXIjopGuZC/6E6rPkGqruJiBnUt1eknTSt2UYrqpsYmgh
         p6Cw==
X-Forwarded-Encrypted: i=1; AJvYcCU46GHSg1WFH5ToRRqsxU8f4o2RryN6O++BuR9oAb8sFFa5NKgxjncOcfKWOiRM63QyrdsahgvFYwr16ZoFPxG8gn4qMpRoqvEzoM/Cz5M=
X-Gm-Message-State: AOJu0YzESwYMFsuAB68i7XSgJ9PyiY+tknvJ7L7l4XorchFxzQ6TvM9g
	MyLp7AQmRrjUNCws1mYQJA3mchg42MoeMlYrthocW+tQxopCfLTmUEl49j4n9g==
X-Google-Smtp-Source: AGHT+IGcSAMotn3gImB5JJ5L4r0S+ZOKG7PU0LH1lDQu7hHe2JXX5HrrhOeIiNqTWJzM4vOxzFYJSQ==
X-Received: by 2002:a50:baa2:0:b0:572:a049:c4bb with SMTP id 4fb4d7f45d1cf-57832a847d9mr1316839a12.20.1716386109120;
        Wed, 22 May 2024 06:55:09 -0700 (PDT)
Message-ID: <fa825c2d-28c9-4164-828e-566caa8efc56@suse.com>
Date: Wed, 22 May 2024 15:55:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] x86/shadow: Rework trace_shadow_emulate_other() as
 sh_trace_gfn_va()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-4-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240522131703.30839-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 15:17, Andrew Cooper wrote:
> sh_trace_gfn_va() is very similar to sh_trace_gl1e_va(), and a rather shorter
> name than trace_shadow_emulate_other().  Like sh_trace_gl1e_va(), there is no
> need to pack the trace record.

Provided record size can freely change (here for the 3-level case) without
breaking consumers, i.e. similar to patch 2.

> --- a/xen/arch/x86/mm/shadow/multi.c
> +++ b/xen/arch/x86/mm/shadow/multi.c
> @@ -2010,29 +2010,30 @@ static void sh_trace_gl1e_va(uint32_t event, guest_l1e_t gl1e, guest_va_t va)
>      }
>  }
>  
> -static inline void trace_shadow_emulate_other(u32 event,
> -                                                 guest_va_t va,
> -                                                 gfn_t gfn)
> +/* Shadow trace event with a gfn, linear address and flags. */
> +static void sh_trace_gfn_va(uint32_t event, gfn_t gfn, guest_va_t va)
>  {
>      if ( tb_init_done )
>      {
> -        struct __packed {
> -            /* for PAE, guest_l1e may be 64 while guest_va may be 32;
> -               so put it first for alignment sake. */
> +        struct {
> +            /*
> +             * For GUEST_PAGING_LEVELS=3 (PAE paging), gfn is 64 while
> +             * guest_va is 32.  Put it first to avoid padding.
> +             */
>  #if GUEST_PAGING_LEVELS == 2
> -            u32 gfn;
> +            uint32_t gfn;
>  #else
> -            u64 gfn;
> +            uint64_t gfn;
>  #endif
>              guest_va_t va;
> -        } d;
> -
> -        event |= ((GUEST_PAGING_LEVELS-2)<<8);
> -
> -        d.gfn=gfn_x(gfn);
> -        d.va = va;
> +            uint32_t flags;
> +        } d = {
> +            .gfn = gfn_x(gfn),
> +            .va = va,
> +            .flags = this_cpu(trace_shadow_path_flags),
> +        };

There's again no function call involved here, so having tb_init_done checked
only in (inlined) sh_trace() ought to again be enough?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:57:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:57:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727721.1132355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mT0-0000b4-Rp; Wed, 22 May 2024 13:57:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727721.1132355; Wed, 22 May 2024 13:57:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mT0-0000ax-PI; Wed, 22 May 2024 13:57:18 +0000
Received: by outflank-mailman (input) for mailman id 727721;
 Wed, 22 May 2024 13:57:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mT0-0000ap-78
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:57:18 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3323db46-1843-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 15:57:17 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5755fafa5a7so5379760a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 06:57:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bebb57asm18261602a12.26.2024.05.22.06.57.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 06:57:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3323db46-1843-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716386237; x=1716991037; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ngfdMziJ+Dx9wHG0ZYXBe50zw9kBBFnqoLdJEWbjR/g=;
        b=IE+Cgk+7b3wWkgG0ftip+hg2hRWW7qtYFcLln3oGaSNUuljY1olUhd9JD/i7J5KLLc
         fSGFX9G13wJcu/z40nNSiRn482MmyacccYjrUEcovYjabkg8JJ96X1si54iS2mHFXyuP
         O2GFk6Y/CBEiWyeMXRYpvhuYj3aVzORdlO5M1Z3dujyGv3r316ihiMTld0bF2jbpnUzZ
         4fqpaZWZE6cokrZNpkZPSp9QamRZMES2NzHztdGH931lDabz8Yk8Qcxeeo2m3llT6Ke5
         1gyl1+IujfnbZxnFuAmCtKBpI0pJ5GhMgAjxFKiNiQXEHlLjzNmJoEs2NpY/MHRxDEXr
         rp9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716386237; x=1716991037;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ngfdMziJ+Dx9wHG0ZYXBe50zw9kBBFnqoLdJEWbjR/g=;
        b=SzCWFDScNWVIWvX2iOvaO7f+5VUgh9Tmd9Fxu2MjDJamhLCLWbehojdbLuoEcFslMZ
         2gRLPi9M/k5HKnOgwueDWAfEGCJsr8Q2VlRTjGON5WkIkeGidIeGgWay9sY2huvh/M72
         C96CBC4xgkLSllWiYxzO++Cgx+p9PFXDjjQ3rrrjHJZZEU1ZVXKi/Hi2YRwaT3z4xBUT
         LmlygTQM3Yb/MOBd58nHxyIacfu4N8QDpEmMo++eVynkDWxNARqdOgLAFgYE5Xav7wCe
         Yy9yG3EFBySZdUjLvbrJFBqTfl+G2XX9G1Dxitqzkn8bbfrCNjgI+bLqpjpKu24dxr6m
         f7dA==
X-Forwarded-Encrypted: i=1; AJvYcCVMTOYq2aCvzWNrxbYErCkWagTzS4bbog5WFl2081iWcCKjYJ37pwbqhSVbaf6gq2lzJE4U3nj/hkPlzqHfmmfeO5fSvDRxw6inruUZ7H4=
X-Gm-Message-State: AOJu0Yyp5Gre99FSaZ5jld6/6ss5zVa3fSKO+AcfCXIz/46M9UnqjSks
	nWJynlc5zcYTDAMl7eY8GMWPXnrOe8bcmDc5oAMSRxosr391V+5VBEwTocSsbA==
X-Google-Smtp-Source: AGHT+IGeHZiI15urXAyHaMK8vbQw9kySUZV5oEAnLNvgSeCIoBAdhNDe5rLkLFxQzzkbUCdA44vktw==
X-Received: by 2002:a50:d55c:0:b0:572:9bd2:9a90 with SMTP id 4fb4d7f45d1cf-57832c5112dmr1349307a12.31.1716386236720;
        Wed, 22 May 2024 06:57:16 -0700 (PDT)
Message-ID: <5aab89b4-1171-42e9-aefb-02f914eb982f@suse.com>
Date: Wed, 22 May 2024 15:57:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] x86/shadow: Don't leave trace record field
 uninitialized
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240522131703.30839-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 15:17, Andrew Cooper wrote:
> From: Jan Beulich <jbeulich@suse.com>
> 
> The emulation_count field is set only conditionally right now. Convert
> all field setting to an initializer, thus guaranteeing that field to be
> set to 0 (default initialized) when GUEST_PAGING_LEVELS != 3.
> 
> Rework trace_shadow_emulate() to be consistent with the other trace helpers.
> 
> Coverity-ID: 1598430
> Fixes: 9a86ac1aa3d2 ("xentrace 5/7: Additional tracing for the shadow code")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Your additional changes look pretty much independent of what my original
patch did. I don't mind the folding though, yet I think you need to add
your own S-o-b as well. Then in turn
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 13:57:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 13:57:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727722.1132365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mTE-0000xc-2e; Wed, 22 May 2024 13:57:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727722.1132365; Wed, 22 May 2024 13:57:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mTD-0000xV-Vy; Wed, 22 May 2024 13:57:31 +0000
Received: by outflank-mailman (input) for mailman id 727722;
 Wed, 22 May 2024 13:57:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8RWL=MZ=amd.com=Jason.Andryuk@srs-se1.protection.inumbo.net>)
 id 1s9mTD-0000wD-60
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 13:57:31 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39889712-1843-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 15:57:29 +0200 (CEST)
Received: from BY5PR20CA0016.namprd20.prod.outlook.com (2603:10b6:a03:1f4::29)
 by BY5PR12MB4257.namprd12.prod.outlook.com (2603:10b6:a03:20f::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 13:57:24 +0000
Received: from SJ5PEPF000001D1.namprd05.prod.outlook.com
 (2603:10b6:a03:1f4:cafe::10) by BY5PR20CA0016.outlook.office365.com
 (2603:10b6:a03:1f4::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Wed, 22 May 2024 13:57:24 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ5PEPF000001D1.mail.protection.outlook.com (10.167.242.53) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 13:57:24 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:57:23 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 08:57:23 -0500
Received: from [172.21.89.32] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 08:57:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39889712-1843-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CqTs30b683Llb2qdtzxD8zJ0hon4k8WMc7NCXyklAMPj4fzfGPu4gVF6v9DXOKdjHMjyL2XOjeIyX+S0+ZZGFQxJgVeo1AesTMIrImQU80AQex9WB695TbwJ2YJzbV0IV6xPaa+1fsaIMEO6Fll6sWH+NIN/KYwf/NTL4f5SWCqGrjZe5D8iYips/KcLb/jfUAoSnhkq9GHZf6MBkYGNHVaJzMEHzG0WvpoJn+QKeNlYhBFccmoY42JRPlvNLA5cLCggs5K3evbUfoj8/wMp0nIXTRmm9Re7NiKUL1eQPbVrNft+4rpwBsF3coMVPcAvXwiDdbuOlQf59PJCIQYxVg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JHQ7yRqcmXj9s0rkz0OK7BakOpjTzOKqOpjDy3dS5/0=;
 b=fVxwc1mODReGAeEx9ZFdXBSJduiQS4Iy6vKXRdaHXKbE/wJg57tzW3bKsSOyQO0BAK3gRUqG/pgy2gX14nXmZPk6t80SYnaQHUSYa0qpfXB7rVEJ2Yu5xycPHvwZLj2YdTqfVzDZBQAcESUJdmVEkpZIPH0f/OwURBUhx+UstH1RwXdPB/5PBulpt38fN1zfehPoAWuGFK1PYmZcyvRBQmGSaZsjYhjRnjOR2Wz3FuN0NhhBf9KO0psZ+fMv6QoPvWIoKSSUS84OPaZDvkFzw1RQtBoz3xF2eNuyRO5efSaVTxWMlEo/MVGgZHmeOhLNL2RQocyFpyTO4XsCpHG08Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JHQ7yRqcmXj9s0rkz0OK7BakOpjTzOKqOpjDy3dS5/0=;
 b=m6U7L3We9GbvySGVZQxzUEdBTuBR4OkkqHansyqdybECtnpB8Mqae9qXn5NImUMF9s/97ekyZX8Xp0TAF8rdn6e8+xVTZVyfoocpHIBeojd8dHC+V+f5XPuicm9njMxNyu9aoOreRR9zhz7YR52ufCxPrNn58nb4l/PJxV0P8to=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <aee2a6f1-c94c-4265-af24-ae54e1ceca96@amd.com>
Date: Wed, 22 May 2024 09:57:25 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/2] x86: detect PIT aliasing on ports other than
 0x4[0-3]
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
 <df53dd77-9341-4a72-a4ce-33654b2e877d@suse.com>
Content-Language: en-US
From: Jason Andryuk <jason.andryuk@amd.com>
In-Reply-To: <df53dd77-9341-4a72-a4ce-33654b2e877d@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: jason.andryuk@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D1:EE_|BY5PR12MB4257:EE_
X-MS-Office365-Filtering-Correlation-Id: 68d0045b-11ac-443e-a849-08dc7a671bbd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?c1hLYmhMQnBFaktCQm1KcTZSU2NFWTRJcDZBeXYwMS9BamNLYUtkMmFhdjZr?=
 =?utf-8?B?ZDVCUTJDREY5UHhLRzRXMTIwYzFCWmVIUzRFZjVoVk1ZdXJ0c21iWFhpSXh1?=
 =?utf-8?B?dXNqc3FRRlFWZDR4NGRwVjJDKzZjTGV2ZSs4U3BJRXpyQUlXYkRRUllzUE8x?=
 =?utf-8?B?Y0xwUTlPQzVTMDc2dUNvQ3BkVmxMSXhGM3BWTWlsRGkrenlhcVBNWVBOTVMy?=
 =?utf-8?B?dk45TE00WGovRVVqMm41NU9oM1dJVUpHbVV1Q2JTMGNDWVk1NlJiRC9HSXAy?=
 =?utf-8?B?dnVNeHg5cmViTjJYODNuNUptTFMvSkEyV0hOcDFsTzRqU1RqMjgyOVNyN2pn?=
 =?utf-8?B?b0hsdWJFUGtOTThFVVQ0ZHdjbnVBNWF4bzhUbWV2c3d0T2E4SzBPaWpURE0r?=
 =?utf-8?B?TmVnUXRFWFFqYVcyelVUWWFVUUhYMFQxTS9zV2VJUktBLytZdjd0WVVDTXls?=
 =?utf-8?B?RHdTT3VpelRUN2wrcXJ6bGUrSXdhb2ZIUVVDdjh0cUt2N2VvVjB2MnNvZnVs?=
 =?utf-8?B?N2paQy8yTkJHYitZUmlsWG1YRzJTK2g2ZFhPTXFVTDZ5eWx6d3NhSkg3MHl5?=
 =?utf-8?B?bHJ1K1d5S0NCdVdQVG44czNUN1ZkZk1JOTI3WlNiYzRWWEVuT2FCdEFrRkQy?=
 =?utf-8?B?VnFQLzkzZkFtYk9xM1JrbWJJSk9VSnFkS2hqOWpJeXYyM0ptRG4wMFVvU1hF?=
 =?utf-8?B?SUZxNmlkaTNjK1Fva01DWFA0YjVrcmdNSjV6VWFBbnE5OXNuUjZGTHh6VzRB?=
 =?utf-8?B?c2xDakd3Zy80TWVIY0hEOW9BTUhqdk1lMGNlLzFNNFhTMXg0NS9DTFB5RWRU?=
 =?utf-8?B?dzh0QnFaam8zZ1pDTlNsTEJPb2dyL25FVTFTaCtOc09zV1VLUzd3cC9pUnpV?=
 =?utf-8?B?MmtUSXgzZ0pYRWlMelFsYXhLMzlSa3dlb3lwempzY0grMU9MTHVTcmZ6Yks1?=
 =?utf-8?B?MmcrMXAwdlhkN1dCajFLbEZlMllxQTd1ZGNoU0ZSaUZEVmswNkw3VXRIcy9M?=
 =?utf-8?B?YVcwc08rMWlnY2FEbEdJVFNBaENFTmNnOHQybFhVLzlrNWh6dUpzVUxkSS9O?=
 =?utf-8?B?eGtBVXZwcEVEZFowQ29abk5wUjZTYlp1SnNrUlA0cGdSOGNCc3hEaWZ2bDBl?=
 =?utf-8?B?RWVRNjFIMFI5WG4xZi9jK1o5OGwxSjVmMmttbDcvR2NsSmFvVHN5L1k1bXlP?=
 =?utf-8?B?VTUxYmNLREd2aFIzMnN3akNIZ1A0b2JWN1BJVHVvYjJtd1Z5Mnh1eG1WeG1N?=
 =?utf-8?B?Tm5aQTBHNUxEUjFGUkdzV2VTaUVEQ1VqR2RVTG9RNUVnamx1cll5ajlGZFZQ?=
 =?utf-8?B?T2Fhb01ueWd4ZmlOK3N0OGpOUUQ4Uk84bEd2QXVYbk41NkhpczV1VnArenJO?=
 =?utf-8?B?NkxlM2ZDREJvNVAvZ3gzWGpzZEQxMUsyc2xCbHpYQkJQWm11TTkzOHBzR056?=
 =?utf-8?B?UmxxbXVtVTFNVzY1M3AvQTNHWktLS1Q3dHk4UExMVTZ0d2JTeUhDVzBiZnk0?=
 =?utf-8?B?OTdFWGZvenNJbUpkTm1tWWlYcGRSbncwMUVQQlk1Z1Zva3RFWVhMaEFXWjQ1?=
 =?utf-8?B?bTFESDRpb1ZJRnZORlI3MmFtWktMbmFYVmpqekQ4U0pkbHlVYTI2OFlGRzdr?=
 =?utf-8?B?YjJwdk9aSldyaHdtYmt2aStFdGlBdjVhTWlVRjlqVDBoYklEMlg0SEZrWDRo?=
 =?utf-8?B?b2ZCTStxeHlBUUgvUDVMSm0yZXNJVnZlQk43dmJTV2ZQRTVOOFU0RGpJaE9H?=
 =?utf-8?B?ZFpkNFBCcXhKMWN1NDhDaGd6RzlUeDR2S0N6YXBtMGg0bDAxeUIxNkxIRmYz?=
 =?utf-8?B?VWtYbDdKODllc0RxQTJ2R1Y5UUc0Rnh5K1FqTjlic0ZrcHdXa1FOYjZtOHlC?=
 =?utf-8?Q?svWH7+0Al5xpI?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 13:57:24.4152
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 68d0045b-11ac-443e-a849-08dc7a671bbd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001D1.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4257

On 2024-05-22 08:59, Jan Beulich wrote:
> ... in order to also deny Dom0 access through the alias ports (commonly
> observed on Intel chipsets). Without this it is only giving the
> impression of denying access to PIT. Unlike for CMOS/RTC, do detection
> pretty early, to avoid disturbing normal operation later on (even if
> typically we won't use much of the PIT).
> 
> Like for CMOS/RTC a fundamental assumption of the probing is that reads
> from the probed alias port won't have side effects (beyond such that PIT
> reads have anyway) in case it does not alias the PIT's.
> 
> As to the port 0x61 accesses: Unlike other accesses we do, this masks
> off the top four bits (in addition to the bottom two ones), following
> Intel chipset documentation saying that these (read-only) bits should
> only be written with zero.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/arch/x86/time.c
> +++ b/xen/arch/x86/time.c
> @@ -427,6 +427,74 @@ static struct platform_timesource __init
>       .resume = resume_pit,
>   };
>   
> +unsigned int __initdata pit_alias_mask;
> +
> +static void __init probe_pit_alias(void)
> +{
> +    unsigned int mask = 0x1c;
> +    uint8_t val = 0;
> +
> +    if ( !opt_probe_port_aliases )
> +        return;
> +
> +    /*
> +     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
> +     * count is loaded independent of counting being / becoming enabled.  Thus
> +     * we have a 16-bit value fully under our control, to write and then check
> +     * whether we can also read it back unaltered.
> +     */
> +
> +    /* Turn off speaker output and disable channel 2 counting. */
> +    outb(inb(0x61) & 0x0c, 0x61);
> +
> +    outb(PIT_LTCH_CH(2) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
> +         PIT_MODE);
> +
> +    do {
> +        uint8_t val2;
> +        unsigned int offs;
> +
> +        outb(val, PIT_CH2);
> +        outb(val ^ 0xff, PIT_CH2);
> +
> +        /* Wait for the Null Count bit to clear. */
> +        do {
> +            /* Latch status. */
> +            outb(PIT_RDB | PIT_RDB_NO_COUNT | PIT_RDB_CH2, PIT_MODE);
> +
> +            /* Try to make sure we're actually having a PIT here. */
> +            val2 = inb(PIT_CH2);
> +            if ( (val2 & ~(PIT_STATUS_OUT_PIN | PIT_STATUS_NULL_COUNT)) !=
> +                 (PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY) )
> +                return;
> +        } while ( val2 & (1 << 6) );

You can use PIT_STATUS_NULL_COUNT here.

With that:
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Wed May 22 14:02:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 14:02:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727730.1132376 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mXV-0003ky-Kq; Wed, 22 May 2024 14:01:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727730.1132376; Wed, 22 May 2024 14:01:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mXV-0003kr-HM; Wed, 22 May 2024 14:01:57 +0000
Received: by outflank-mailman (input) for mailman id 727730;
 Wed, 22 May 2024 14:01:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=AJ0E=MZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1s9mXT-0003jS-VK
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 14:01:55 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d7e74376-1843-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 16:01:53 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso969281666b.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 07:01:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c70asm1780655866b.56.2024.05.22.07.01.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 07:01:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7e74376-1843-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716386513; x=1716991313; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9e/6q4LVEqR3l1oMUflAuDP7qSLzmBssTUhZHefql1Q=;
        b=Z7W0NzR1Il98/n+/R8uffi4VhjDqWY81rfwfXgUOESCsAHse/gXpQYcUW1PVhNAPYJ
         YyRrf6RuYp8S4CQMER4wZY2HH4NUoQj6zOXS8jMCXAz3LS0S1+6bRxEQOic5rbG8VwgR
         ZK5oarqBmPlHPUFf16rgMIu0p8dLUXbl7MNGkDCe9AUyL1HobK+X18uK4CvtRJmZ//X9
         jY90aPNjxxy3bsvF/T57QClO6LdVH8q4xhAJFC1GgpGhWfM5loHA39H4FvowdZpAvrxT
         PHb+bbudS8ULnYuNaf3dCdG6/UfrYtaJvlDtNWLnCNJBhWeVeWNYaVNHc2hb/Tv2r/Pi
         0CzA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716386513; x=1716991313;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9e/6q4LVEqR3l1oMUflAuDP7qSLzmBssTUhZHefql1Q=;
        b=qCu6g0E/L4+Pq5fsG3U2oJU0GskFkzglAqVr1EL5YPmunHxqKFZAnoYSD9zQUDQk54
         AM/4XOL+rdaObUyzBNlsorOdY6EDDtcGrsV+SYzCMgeFszZqLijkVJB9fBj5XWG5QRlR
         EjiQ92Lgex3Sq+zM+bbNB/YHLxTnk5rBiwAVJrlfQ50kNSkZw221B4ezhZeRKBRtj16i
         lXT9V6CyfbKIenWldmatxykqrj/iZ59U/iDpEgWkuHfSWbLHqzKNYY2mrv71rxt8gGvU
         aqdIrhnVS60prGZHYMiPMvtmpiJHhwnU5yxQx9RzMBBjvnep16IesBnTJJ8bzxuNfYwJ
         E2MA==
X-Forwarded-Encrypted: i=1; AJvYcCXLEbhzRnw08Qrm/NmcixgPqvSXv3bKAB2oMcIbMgz1X+bnZ56qJV/HIamAVr+zv3l5D2IqPxd91uyIZPSz0LR+DOZBF48fNoKklezHqqI=
X-Gm-Message-State: AOJu0Yzp2/AMgA4UQ5dpRboXtP+CqHxeByX8IG+sNENQUXdrBi+rfiHZ
	BO1Qzf2bCYFmQzvC/78q2X+E6fdnlLTRH9ushfxw88ShCz7dWA5D2p6rKOF8EA==
X-Google-Smtp-Source: AGHT+IE/UHt6PbQ9yaDqY3FWicYb/NW+enBGyQdA40YLDPXv7O4f4ocjNAtEfs729l3T+c9NzSzPWw==
X-Received: by 2002:a17:906:d28d:b0:a59:cdf4:f948 with SMTP id a640c23a62f3a-a6228207bdamr207207766b.65.1716386512957;
        Wed, 22 May 2024 07:01:52 -0700 (PDT)
Message-ID: <6d77bd71-400b-4786-ab8c-5ea2b734a8da@suse.com>
Date: Wed, 22 May 2024 16:01:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/2] x86: detect PIT aliasing on ports other than
 0x4[0-3]
Content-Language: en-US
To: Jason Andryuk <jason.andryuk@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <14d35449-fc65-4dcf-95db-8d94dd3455fb@suse.com>
 <df53dd77-9341-4a72-a4ce-33654b2e877d@suse.com>
 <aee2a6f1-c94c-4265-af24-ae54e1ceca96@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <aee2a6f1-c94c-4265-af24-ae54e1ceca96@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 15:57, Jason Andryuk wrote:
> On 2024-05-22 08:59, Jan Beulich wrote:
>> --- a/xen/arch/x86/time.c
>> +++ b/xen/arch/x86/time.c
>> @@ -427,6 +427,74 @@ static struct platform_timesource __init
>>       .resume = resume_pit,
>>   };
>>   
>> +unsigned int __initdata pit_alias_mask;
>> +
>> +static void __init probe_pit_alias(void)
>> +{
>> +    unsigned int mask = 0x1c;
>> +    uint8_t val = 0;
>> +
>> +    if ( !opt_probe_port_aliases )
>> +        return;
>> +
>> +    /*
>> +     * Use channel 2 in mode 0 for probing.  In this mode even a non-initial
>> +     * count is loaded independent of counting being / becoming enabled.  Thus
>> +     * we have a 16-bit value fully under our control, to write and then check
>> +     * whether we can also read it back unaltered.
>> +     */
>> +
>> +    /* Turn off speaker output and disable channel 2 counting. */
>> +    outb(inb(0x61) & 0x0c, 0x61);
>> +
>> +    outb(PIT_LTCH_CH(2) | PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY,
>> +         PIT_MODE);
>> +
>> +    do {
>> +        uint8_t val2;
>> +        unsigned int offs;
>> +
>> +        outb(val, PIT_CH2);
>> +        outb(val ^ 0xff, PIT_CH2);
>> +
>> +        /* Wait for the Null Count bit to clear. */
>> +        do {
>> +            /* Latch status. */
>> +            outb(PIT_RDB | PIT_RDB_NO_COUNT | PIT_RDB_CH2, PIT_MODE);
>> +
>> +            /* Try to make sure we're actually having a PIT here. */
>> +            val2 = inb(PIT_CH2);
>> +            if ( (val2 & ~(PIT_STATUS_OUT_PIN | PIT_STATUS_NULL_COUNT)) !=
>> +                 (PIT_RW_LSB_MSB | PIT_MODE_EOC | PIT_BINARY) )
>> +                return;
>> +        } while ( val2 & (1 << 6) );
> 
> You can use PIT_STATUS_NULL_COUNT here.

Indeed, and I meant to but then forgot. Thanks for noticing.

> With that:
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Thanks.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 22 14:07:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 14:07:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727736.1132386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mdE-00057C-A6; Wed, 22 May 2024 14:07:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727736.1132386; Wed, 22 May 2024 14:07:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mdE-000575-70; Wed, 22 May 2024 14:07:52 +0000
Received: by outflank-mailman (input) for mailman id 727736;
 Wed, 22 May 2024 14:07:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfTZ=MZ=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1s9mdC-00056z-Gn
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 14:07:50 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 (mail-am0eur02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2606::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ac09e281-1844-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 16:07:49 +0200 (CEST)
Received: from AM8P189CA0026.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:218::31)
 by PA4PR08MB6110.eurprd08.prod.outlook.com (2603:10a6:102:e1::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 14:07:42 +0000
Received: from AM3PEPF0000A790.eurprd04.prod.outlook.com
 (2603:10a6:20b:218:cafe::44) by AM8P189CA0026.outlook.office365.com
 (2603:10a6:20b:218::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Wed, 22 May 2024 14:07:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF0000A790.mail.protection.outlook.com (10.167.16.119) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7611.14
 via Frontend Transport; Wed, 22 May 2024 14:07:41 +0000
Received: ("Tessian outbound f97ed8ca492d:v327");
 Wed, 22 May 2024 14:07:41 +0000
Received: from 440a50f4b897.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 ABC966A6-7134-471D-BEA6-CC2BC388EFD9.1; 
 Wed, 22 May 2024 14:07:34 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 440a50f4b897.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 May 2024 14:07:34 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB9PR08MB9636.eurprd08.prod.outlook.com (2603:10a6:10:45c::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Wed, 22 May
 2024 14:07:32 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7587.035; Wed, 22 May 2024
 14:07:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac09e281-1844-11ef-90a0-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=MTai4aRKHb6mTEqi18wNfICwSXX/NAR96XS7xxbSpWDzyyizwrJlBrNAOgUtD+DZ3/FjIxmQKpJGBJSyIbqWuuhoH/mJV+bd3RL1z+stafoVWoBiblZ8JCEyaqe6OgjJlRLxu/IXlOXjeAfJU9YzVQRHaeMwokCGSKGqfuFHkMfcKN7XS/J210qcqoEucj/7U6qrfTJ3GiNpkdD2O5b1FOWM32+OzKVHE+CGKyvosRg+xD6JnFMCMLkQQxGFlW2wzlO5ndeNTopkV5GiuiOSW1wt+ekXgPrUBf61dUkL5FZyNYWuMy7W1cjJB6fqOw7T4Os7yCDPZm4UjGqpgOI9fw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OddZpErMz4b2RPmJ4CFrNZ+f5/3Au+Xg5aIfEAhcO6w=;
 b=dFobLSfwMxpi07eUIrbsbFG2WZamZhkBZGQpBwYBkKgybvejMriLFk1m6nRk64xj8QubKV9jaLA+nCOoZGwZZZxIZf+evsuQYaaOQk/VLVeprCnzig/+SOmtfkB/DZQG1jL+p3DrHiXLToTj06aWO12V8wbJgsghX3bo6x9QZl3NXI+duDy/wSwjbW4N9iCjEQnF6o2ZmW726RA8x3X4HitCdep4INkl1VP40jfGiLWt4vzfuwK3tl9JnN2ASkSPnu8mJHQKBqpvQOv79uy5N4JQ/UYJb2mJ7PHfXVJ6ZS5w8hMvE125I5MVXW9wGp+Vft280/KAvCiX6pSpesY7LA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OddZpErMz4b2RPmJ4CFrNZ+f5/3Au+Xg5aIfEAhcO6w=;
 b=R330+A0iX0qeGP0vvmwnAh1NecfNpliaA3rMCK99/h2gniwsQjGlVh0eB7GL5ioB7ZRADnN/EgXELBbjY50k2ETew8YONWqHF2W8fz4wdS6T6JYx6UsCbn94fXrbgHADgeqwiS5cGPrl4LuoCPhNBikruadz7/yyj/dGXbjJ/qI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: d8df9f1b6dd19dd5
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AAtvNWadDqjC1cBN/2E8UXm4pm33p+6CPhzxPDRChvyWDIgIyjLHsQR0W4O+TNN/9M2R2IfD/d2vE3DooAEE2mP+YIKUYZTpFJ5krQyBud/YpSTirefXn4cMUB59dg1+GmkW0C60tgnDAoSrni9vokhoRcBLxMJLAnigEQ7Wic2yxJCNCiswaV+DpjkCJslCG3Setv+7uogUZlDwiOI6chErDtBEJliV55TQSxgXh7IthimRD7elfFa/O5rtRw1vjjXXkOGB7tzdf8kgXNluFUQYNf6j72aIdddrbfOec17MWyW3XGe2eNVFa/Zm73eDw0nZD5CCRFESImimkqP8ig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OddZpErMz4b2RPmJ4CFrNZ+f5/3Au+Xg5aIfEAhcO6w=;
 b=ZN+QAjCzpmdMmguO3/hkdTjycUYi1UgWQ+IHYzlbka/hxYUXYQRuXLismYgcUDTInxOskTP7FVfcUk0yxtR/nqWh9hIJBFmHNuSyTNDXnnSM/Q4QhZohE3NUm+FZnQapNLtHmOY1fQgKJqRFnYZMXKxuZYWG7dFXZNXQNJCBOOsR9dZRMri870cB3uItKqzPlMC256S+QPQtk4ElnqekojuFi1texovnJ2soTA7nlZqzI6+RTNUgw88jlPnxcPnf9CWS8PLphbcRjmGn3qEoEIgt1WW8dzopVUuefzoqHi+5fTH9M6H+yj0gIOFwVndjD3rbdx8oKAG56Xo8N9d2Xw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OddZpErMz4b2RPmJ4CFrNZ+f5/3Au+Xg5aIfEAhcO6w=;
 b=R330+A0iX0qeGP0vvmwnAh1NecfNpliaA3rMCK99/h2gniwsQjGlVh0eB7GL5ioB7ZRADnN/EgXELBbjY50k2ETew8YONWqHF2W8fz4wdS6T6JYx6UsCbn94fXrbgHADgeqwiS5cGPrl4LuoCPhNBikruadz7/yyj/dGXbjJ/qI=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 4/7] xen/arm: Parse xen,shared-mem when host phys
 address is not provided
Thread-Topic: [PATCH v3 4/7] xen/arm: Parse xen,shared-mem when host phys
 address is not provided
Thread-Index: AQHarB0DEGh17L7PW0qcXAGhEqHugbGjRaeAgAAE2QA=
Date: Wed, 22 May 2024 14:07:32 +0000
Message-ID: <0B5D10EA-B93F-45C5-81D1-3908EA1ED50D@arm.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-5-luca.fancellu@arm.com>
 <5a339b77-0ce6-4670-9e79-99be5ac48bb2@amd.com>
In-Reply-To: <5a339b77-0ce6-4670-9e79-99be5ac48bb2@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.600.62)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB9PR08MB9636:EE_|AM3PEPF0000A790:EE_|PA4PR08MB6110:EE_
X-MS-Office365-Filtering-Correlation-Id: 8280391f-0d62-4bdd-6f90-08dc7a688bc1
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?dFhWWE9RZjYxcDd2dzZiekNqYkxLdEFuWGwwWGdWQmM2dCtyQ2YxOWdoeGtK?=
 =?utf-8?B?aENxSmx1NlR3ZDZoQ3RWL0xtaGV0OTFydndZcTQ4Q1F0cDJUaTFUUzBHWUly?=
 =?utf-8?B?ZUExNzFFUC9HTjBrUTIvSUVhbDBjUXd4cTlpZmhha0pRdHZxMUxLazdGejRO?=
 =?utf-8?B?bWF1OElxOW1tNDJHcVkvUnNhWmptWEVxMS94b01ZTW9lRXhGQ2lVOWF3amlW?=
 =?utf-8?B?N2tTTlRGS0Q3bkoxSTdSd01WMUJaYjBQM1JzVzFCOURxdFlMcC9SQ0ZDejRM?=
 =?utf-8?B?TncrOURmK1Y4ZENlQXFjU3lzRDRJTWlIT0MzV0hMUTZJUlFnN2dwSUZSckh0?=
 =?utf-8?B?amJkR3diVzBGVnVDdElia3FCNXZNdUE0Q1BXL2huSzVJa1gyZ1FBdGZ2SE5j?=
 =?utf-8?B?ZlY4OUdYRUpRNW1ESXgvQUhFTnhTY2cvYm1VQWNLOEsycm1XRWExZVViZjFW?=
 =?utf-8?B?R3dHNExRLzVZa2FBVVF5Q1VvaS9hd20yMnN0STc1R2FQUFduOERCUEhHSzg1?=
 =?utf-8?B?WnZWSlcvKzQvOXdWdENURTgrMGJwUWtYVWVweDAxNmpsRFBldk1OblJNd01l?=
 =?utf-8?B?L2ZJM3l2dlFzRnhXbVJ1NjVJbkZqb3kweUdFWkxTd1dvbUY4T1J2ODh2UjJJ?=
 =?utf-8?B?clVMd1ZJRnNZbXY2YUM0YVhoRWduNm0vMG5qcTBTMWQyMEtYalY0MmJFTEQ5?=
 =?utf-8?B?cmtxTDk5YkVmTSttVkIwZXpyek02WWh0aUdpd2xrcitob3FLZ3NOc1NEUlJG?=
 =?utf-8?B?eStCbnVnOGZvSUFxcE1wT2l2QUNLSkVzdFdsOW5ZTGlVY3ZzMndyQ0I5Skp5?=
 =?utf-8?B?K1RkT1JJRStScHV1MnZYUXFtM2gxMkduSnZaRTN3SFhGcHF6T1BBV0J1ZzVz?=
 =?utf-8?B?azBUVGk1K2ZDaU1SK1M5cW1taklieEhNVE1NUEF4UFhPdGtaWkR3MllMV3Rp?=
 =?utf-8?B?S29pWnhDNFBYTjAwRVlqQkpOdExoZU8rWFZMQlVGNlQ4d0c1S1B4dEtOaFds?=
 =?utf-8?B?K2tmV2QxYmRuNUdFSHh3WFY1NFFyNGdTb2x2QzdzVVlaWG5rNEcrQWxEUWJ2?=
 =?utf-8?B?YnJiVGtyU3Q1cE4yK1ZhYnRMUE1hUVhYWFppemxHNnBybm8yZWsrWmtWSXBS?=
 =?utf-8?B?R3EwQTIrczNneHdKc2htQkZ6ck1NVkFyNTA1ZGZHOElxQVl6QURqejVjaGtw?=
 =?utf-8?B?eEovWUpFVjA3TTl4Qzl0RW5qOHlyeW1JdDBEajV1Kzg1TGJZUkVkc2t6Z1pN?=
 =?utf-8?B?RHhEUTA5V0psMmlNd1g5MjVYa3pJK3hyOEs1VnBJY2VFQlVJeVAyMHBJajhY?=
 =?utf-8?B?WUlTVG42MjBzR2dXenpLbkVubGxXSk80RHJvM1phRG9IZGdaZW96OVp2UXVt?=
 =?utf-8?B?MERhZ21EM1ZhdUNUSzQyV2JCem1yemVVMkpHSlNLQTI2ZXN5Ykw5U3phdFhU?=
 =?utf-8?B?ejN4S29hbHBGYnNuNDd6YllpREQvMUZMQXM4WXorRXMrNTZTU2hnRFRxME9s?=
 =?utf-8?B?YlpDQVVUUmMzUXlpSzBnK3ZEU3dYUzliYWxMUUtiOXBjZ1FFOEhOcEVMYmUr?=
 =?utf-8?B?NWlBZzZPYTVmR21JYlJTVzJ4cU9qZ01mMmFrQlQxSVhlSFcvRm5IaStxQTVu?=
 =?utf-8?B?SVRSQTJlUWUwd3Q2azdZQjFBM0ptcW1ZeXRnWXk4MXliZFowK2M1ZlRST2VW?=
 =?utf-8?B?eHZpNGVubWozYTRFQytBWjhOTmErelhYU2pJdzk2K0l1Nk81d29adEdIMFBt?=
 =?utf-8?B?dWFVOUFHNWU3RjZqR3BPWkpKSGI4VjZFOVRnRUkwclNDTkx1UGdjTHp4elVa?=
 =?utf-8?B?YlRWVFh5SnJWM1kwSXF6dz09?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <37A64EC1F03868488192042A020D04B9@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9636
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF0000A790.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	299bd493-0458-41cf-4408-08dc7a688651
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|35042699013|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?djZFZXorZFBSYmpzUXFNNGRpZDdvQnlZaHZSMlQ0aU5tNG9YbXBNTk9qMjgz?=
 =?utf-8?B?V0dwcGRGcjdlY1RZL0tXVndrbkozbjNNeFM0RklUYTBEU21RaHpsaHM1cE8x?=
 =?utf-8?B?REZyaUh0MytqVnplL2ltYXZVa0VaM2c5U2t1UGtzQlZRd2pwczh2SVZ3SWhw?=
 =?utf-8?B?Uko0RjBzNU1kSG5QZmNQdlIvTzJVdERSV2JOSjcwRXFWOFk0dkdXUytmVFVO?=
 =?utf-8?B?Nk9XbzR6S1kwbXlRVE5ZV3VqbjA4VTN1TTVxYXpmSC8xZ3ptbitFUEpUUkl2?=
 =?utf-8?B?cUJuSTlrR3E4UjdsK3l6Q0dhamtuMXhjZ0ZBcWJrUCttOVk3aFEwT0VYUkdR?=
 =?utf-8?B?ZGc1OEtSYmtQajV1blpnZnVXNXJBdzg4VjYvZ2hwckRtWnJtNnV4VWtRVTBJ?=
 =?utf-8?B?NjFPaTNSRTFrd0Vyei83d09MUHJBaFVRckxVNE8zeEZmQ0h3NHNaZXpObUR3?=
 =?utf-8?B?K09GdGVQQ3RWaGZtTXk0ZDUzOHJoa1R4bUUwejhEUS9XZjU1ZlVWVG1rK2hM?=
 =?utf-8?B?QmZ5WHZjRVpvMDZwQkpzZTZJMGJIbHJoSkh2U3Fjci9sVXFkWm9ZWlNlU2pH?=
 =?utf-8?B?Q3JkL1pydzZjdjlRNUtOd3hHTGprRnhralJtanVyczA1aXpOTWNsbmhzK2lN?=
 =?utf-8?B?MlNhdE1jRTIyK08vN2VtZ1R2OXBXYjJ5cHVvTWpXV0Rzc1V3ajVJS2lpSTNy?=
 =?utf-8?B?S2oxOGpUVGdwdlFmVlp1a2lXc3RGOWxwZENKZG5JV2tISHhNRjdESm96ZUx5?=
 =?utf-8?B?VW9yN0NoUlFTT1FyQ2tUa3NTYUxDUHcwNTR6UU1udDhjTGJLcUVZdGR6T045?=
 =?utf-8?B?V3lEclJYSjNvWEc3dzRIN25JMnJGSFJVM1BqcHgzanVrbGE3Q1BtaURUNmlJ?=
 =?utf-8?B?OHAzOXJ3YTFhVkNzL00zOVpncGFPOHk3MlRKNlk4dGVHYXZYR0FCd2xBQWJo?=
 =?utf-8?B?TE05RjJpbHBvZGZHR2N4M1dLSHQ5M20yVDd4WWU1d21rVVdsb1M1N0FJMU9y?=
 =?utf-8?B?UGxrbFRhUTNoUG1MSitndmU4V2dXbVNkN3lwVEJHbUh3aWVXOXVjNkpvREp6?=
 =?utf-8?B?RDRQdDdscjROanBQSG9XSHJtVDZaOG5RZTlyeHY1aWNRckcxc1hPNnRtRzds?=
 =?utf-8?B?S2hselhTMEZ5aldIbnF4dGF4d1Rqc1dzOHFxeVlrVDdKemNzdElaSEtYeEZ6?=
 =?utf-8?B?OUQveW4rNWhDT2JLM3pLZ3ZzNWh1NU82cWdXUEt4SUVKOURRMTBhQ1lpZUpY?=
 =?utf-8?B?dmg0bWM3R004VmxKbnA3Ty9UMnpOMURObndmdmlvM0JYZmRVeFJMMmVISVAw?=
 =?utf-8?B?SmZwQmhyYktLR293SmxLczJPYnRZZzhXMzZnaU05eE1jRGJpaFdsT3dxTkx2?=
 =?utf-8?B?SjJ6SE9XU2g5V3lPSFBYbnUrMjkrblVCbXErTjd5QWJDc1VrdTFZRVFGeG9J?=
 =?utf-8?B?OEVNWHpndGtYYUl5UzN2ay9WNC9xMWZKeGJnSEY4N1pLVTJUZW4zTmxGZHNy?=
 =?utf-8?B?bS9LeWtBditSRnkySW40TnF3SWdYOEZVRTdEc1NTN3cwa1hoQ2FpUU1MUVBI?=
 =?utf-8?B?Z2Fyaldjekt1dzNyN2Fwc3NFalRmajZyOUxmWDlQVk5FUmd0Nkl2RDh0NDlm?=
 =?utf-8?B?dmtaZ08yN1dCZnpvOEFSUmJPL3BPYWZDcDU3QURXZkZXN0MvcWhoMkttNUpI?=
 =?utf-8?B?d2RXZk55VURvN09IMWVBSUhOSTQzOExnOGk2YVpSbUNDMmkxTEJ3dEtDV1Q3?=
 =?utf-8?B?Kzc1MW5pYStFd3hXTzJ4NE13czdMeFlvblc1K2pyN2hObVZKcDNQZkl1c0FJ?=
 =?utf-8?B?U0NlUjJpcHo0WXVqamFWMlpyd0hBMGo5dFo0a3dudHMyeDM0NHNZaWVrUkpL?=
 =?utf-8?Q?XtmL7wyasObqW?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(35042699013)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 14:07:41.9040
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8280391f-0d62-4bdd-6f90-08dc7a688bc1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF0000A790.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6110

SGkgTWljaGFsLA0KDQo+PiAgICAgZm9yICggaSA9IDA7IGkgPCBtZW0tPm5yX2JhbmtzOyBpKysg
KQ0KPj4gICAgIHsNCj4+ICAgICAgICAgLyoNCj4+ICAgICAgICAgICogTWVldCB0aGUgZm9sbG93
aW5nIGNoZWNrOg0KPj4gLSAgICAgICAgICogMSkgVGhlIHNobSBJRCBtYXRjaGVzIGFuZCB0aGUg
cmVnaW9uIGV4YWN0bHkgbWF0Y2gNCj4+IC0gICAgICAgICAqIDIpIFRoZSBzaG0gSUQgZG9lc24n
dCBtYXRjaCBhbmQgdGhlIHJlZ2lvbiBkb2Vzbid0IG92ZXJsYXANCj4+IC0gICAgICAgICAqIHdp
dGggYW4gZXhpc3Rpbmcgb25lDQo+PiArICAgICAgICAgKiAtIHdoZW4gaG9zdCBhZGRyZXNzIGlz
IHByb3ZpZGVkOg0KPj4gKyAgICAgICAgICogICAxKSBUaGUgc2htIElEIG1hdGNoZXMgYW5kIHRo
ZSByZWdpb24gZXhhY3RseSBtYXRjaA0KPj4gKyAgICAgICAgICogICAyKSBUaGUgc2htIElEIGRv
ZXNuJ3QgbWF0Y2ggYW5kIHRoZSByZWdpb24gZG9lc24ndCBvdmVybGFwDQo+PiArICAgICAgICAg
KiAgICAgIHdpdGggYW4gZXhpc3Rpbmcgb25lDQo+PiArICAgICAgICAgKiAtIHdoZW4gaG9zdCBh
ZGRyZXNzIGlzIG5vdCBwcm92aWRlZDoNCj4+ICsgICAgICAgICAqICAgMSkgVGhlIHNobSBJRCBt
YXRjaGVzIGFuZCB0aGUgcmVnaW9uIHNpemUgZXhhY3RseSBtYXRjaA0KPj4gICAgICAgICAgKi8N
Cj4+IC0gICAgICAgIGlmICggcGFkZHIgPT0gbWVtLT5iYW5rW2ldLnN0YXJ0ICYmIHNpemUgPT0g
bWVtLT5iYW5rW2ldLnNpemUgKQ0KPj4gKyAgICAgICAgYm9vbCBwYWRkcl9hc3NpZ25lZCA9IChJ
TlZBTElEX1BBRERSID09IHBhZGRyKTsNCj4gU2hvdWxkbid0IGl0IGJlIElOVkFMSURfUEFERFIg
IT0gcGFkZHIgdG8gaW5kaWNhdGUgdGhhdCBwYWRkciB3YXMgYXNzaWduZWQ/IE90aGVyd2lzZSwg
bG9va2luZyBhdCB0aGUNCj4gY29kZSBiZWxvd2UgeW91IHdvdWxkIGFsbG93IGEgY29uZmlndXJh
dGlvbiB3aGVyZSB0aGUgc2htX2lkIG1hdGNoZXMgYnV0IHRoZSBwaHlzIGFkZHJlc3NlcyBkb24n
dC4NCg0KWW91IGFyZSByaWdodCwgZ29vZCBjYXRjaCwgc29tZWhvdyBpdCBlc2NhcGVkIHRlc3Rp
bmcsIEnigJlsbCBmaXggaW4gdGhlIG5leHQgcHVzaA0KDQo+IA0KPiB+TWljaGFsDQoNCg==


From xen-devel-bounces@lists.xenproject.org Wed May 22 14:20:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 14:20:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727745.1132395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mou-0008WU-8e; Wed, 22 May 2024 14:19:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727745.1132395; Wed, 22 May 2024 14:19:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9mou-0008WN-6B; Wed, 22 May 2024 14:19:56 +0000
Received: by outflank-mailman (input) for mailman id 727745;
 Wed, 22 May 2024 14:19:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vsR+=MZ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1s9mos-0008V1-IT
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 14:19:54 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b43b329-1846-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 16:19:53 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so95170366b.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 07:19:53 -0700 (PDT)
Received: from [10.80.67.140] ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b017b1sm1778456166b.181.2024.05.22.07.19.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 07:19:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b43b329-1846-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716387592; x=1716992392; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=wA0WKAeMXM6MNE0c7IGAxHxlLxAMIdxdodB10bvF13Y=;
        b=Zb4iuUAH0imfWxBpG86722UvfIG9HUT/RduA+Ruff4uDaXbXUW7HY7MFPbp8eyARRF
         hInBkq8VesTp7AZORxQDhVM8G2XVnvedMWSDzot4R8e+yY+nksIu/h7xCXWVs/VGVcDv
         PpnBF3kSWoRBM6TyzZVD02FrWYqrDPYQyl3bA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716387592; x=1716992392;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wA0WKAeMXM6MNE0c7IGAxHxlLxAMIdxdodB10bvF13Y=;
        b=gR3zMTz/Q+hP3kf/HZD1xQ6/s4pdBI5Va7EGvGq7oiuhufSLij8xm+P2obQe4sG47m
         KFdejkq/Xz2//nSg+YQ85wNXDhL1oNqqNKCJ5xPbAAfn0+Tp9WY6tN2EpmCbD8OEZwJu
         XlOEWoxDakXNHHSU10jDKjo6jD5aI2OICBmmcQg2Wn8K05DWqc9BcZmeX7I9B14Wk3V0
         RCyzVxuRFlvNP6cpJDo91TyhYypmvACN924jMyQOHn7Yvvqt22E87Wh4yannAvuABUax
         R1AyTuHbcop5jwQ9FlkvnLeCB0LGMRhRdOx78aV4pQRYlYQHZg3dAHpTlLXG8fdVqp1t
         c/jw==
X-Forwarded-Encrypted: i=1; AJvYcCXHqKXpucinjFMb+PRQM+NjDvWqXklCFzQ08pXDDY+/p8g3DdKhsouwybtrhrL/x0uhLoBZJ0KE3ZWLjrHd6/e6XOi4qR0U5l8EhCH+Rgc=
X-Gm-Message-State: AOJu0YyKmyzqhoySmWIPgfy6d1wn1533C8Y5xwMgCFec21zybbBA1rTj
	fn9RS7Y73LF7EwJosby89trylPdktEIMlj9Jo4LI8fGpb8oIdOPqDDK8+n73h0c=
X-Google-Smtp-Source: AGHT+IF2W59fLOBQrB5AgIOPWN6eg0HzdlLZ987J3lhiMfFXPCVcn6KcMU+jH+RQCtOnB72BmxXiAw==
X-Received: by 2002:a17:906:cec5:b0:a59:d458:8355 with SMTP id a640c23a62f3a-a6228191821mr126180566b.77.1716387592552;
        Wed, 22 May 2024 07:19:52 -0700 (PDT)
Message-ID: <bdefdd57-74b9-4bcc-a8b5-03e8eeee3569@citrix.com>
Date: Wed, 22 May 2024 15:19:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] x86/shadow: Introduce sh_trace_gl1e_va()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240522131703.30839-1-andrew.cooper3@citrix.com>
 <20240522131703.30839-3-andrew.cooper3@citrix.com>
 <e0f0d9e7-ab53-414a-b689-dde9a9089f5d@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <e0f0d9e7-ab53-414a-b689-dde9a9089f5d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22/05/2024 2:47 pm, Jan Beulich wrote:
> On 22.05.2024 15:17, Andrew Cooper wrote:
>> trace_shadow_fixup() and trace_not_shadow_fault() both write out identical
>> trace records.  Reimplement them in terms of a common sh_trace_gl1e_va().
>>
>> There's no need to pack the trace record, even in the case of PAE paging.
> Isn't this altering the generated trace record for the 4-level case, in
> size changing from 20 to 24 bytes?

Oh, eww.  Yes it does.

I'll need to rework with __packed still in place.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:15:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:15:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727759.1132412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ngE-0003xB-9e; Wed, 22 May 2024 15:15:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727759.1132412; Wed, 22 May 2024 15:15:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ngE-0003x4-6Y; Wed, 22 May 2024 15:15:02 +0000
Received: by outflank-mailman (input) for mailman id 727759;
 Wed, 22 May 2024 15:15:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eEN6=MZ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s9ngD-0003wy-31
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:15:01 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0db1c897-184e-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 17:14:58 +0200 (CEST)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-351b683f2d8so751139f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 08:14:58 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502bbbca98sm34581412f8f.112.2024.05.22.08.14.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 08:14:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0db1c897-184e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716390898; x=1716995698; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9kKdNh+rZiON7ptffywUHyNxYVSfVBM2j7BRr65fCRY=;
        b=FofK+Vh46ZvyCEvkozPgALswji7pM6FxkzGih1AjuVT48tBc4BhQ/JFYTseFq/lUs7
         rvaEvRkBim4LnyHQRKOPGRmolpButDGV9xrbhm1CUwwyA9GrPaOrK1l1PiX6wrhj1DXv
         QAE5MO3II/qHwIF8cdqhTN2ED+LnVMyejppC0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716390898; x=1716995698;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9kKdNh+rZiON7ptffywUHyNxYVSfVBM2j7BRr65fCRY=;
        b=LZdjjkV+sG23t86W59gL+BU40ywDE39UOMCiOzdj1fwVCc3ZTXrz4+yLg3bMPkZrGe
         iBmPmUcrZtkrUBX+TPp2GKcbwgBJWGbVQ/UrKPFRATLSR8t8meqIv+DBViZGPFzchHav
         terhKQJkwuKt8dv01KL8Ye73kI7gS4txIT9rv7Wha/L+lcLRujPAAyYGXO4xhswfTkrA
         sAqOfu3mASl0yatMKeNGKClHSNidRnMjSXz1F+PUrwgjjpyEWfqBx6MecxEzaRxpY7sS
         I+3MdNzeO+3mo+KnNdyUP1r/ZtY0Ml2a8m9KkQLE8s0ZzvFynvJHxasJLFjtiNn1Xxc0
         BpOw==
X-Gm-Message-State: AOJu0YwVWN9TJoYqVZP4jHtIcG24VjvLFurhp2xpyb4p0cU7cba0yOH2
	mvX2AZhsyWSK9xdHHSfYhVmBedDSZRZ5/nfCdN88uldRnOmOssWUjqb9lejIoes=
X-Google-Smtp-Source: AGHT+IG6oWA2Y3q/5TeHJdjPaFvHHHarkj3yId8ntGvmVgxu9DGx66VHuREBWLFhVf8wR2arvc6VdQ==
X-Received: by 2002:adf:e745:0:b0:354:f1de:33e7 with SMTP id ffacd0b85a97d-354f1de3617mr1051687f8f.2.1716390898327;
        Wed, 22 May 2024 08:14:58 -0700 (PDT)
Message-ID: <f0efee18-54f5-4caa-a4ab-d82ceeea64bb@cloud.com>
Date: Wed, 22 May 2024 16:14:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
 <c6c89012d189a2c3077b3e224942b33028e29336.1715954111.git.alejandro.vallejo@cloud.com>
 <ZktUfWZD67zkvaeJ@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZktUfWZD67zkvaeJ@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20/05/2024 14:47, Roger Pau Monné wrote:
>> @@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>>                               xc_cpu_policy_t *policy)
>>  {
>>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>> -    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
>> -    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
>>      int rc;
>>  
>> -    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
>> -                                 policy->msrs, &nr_msrs);
>> +    rc = xc_cpu_policy_serialise(xch, policy);
>>      if ( rc )
>>          return rc;
>>  
>> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
>> -                                  nr_msrs, policy->msrs,
>> +    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
>> +                                  policy->nr_msrs, policy->msrs,
> 
> I would be tempted to just pass the policy to
> xc_set_domain_cpu_policy() and get rid of the separate cpuid and msrs
> serialized arrays, but that hides (or makes it less obvious) that the
> policy needs to be serialized before providing to
> xc_set_domain_cpu_policy().  Just a rant, no need to change it here.

I'm still pondering what to do about that. I'd like to refactor all that
faff away as well, but I'm not sure how to do it cleanly yet. The
biggest danger I see is modifying one side of the policy and then wiping
those changes by mistake reserializing or deserializing at the wrong time.

Not for this series, I reckon.


>> +int xc_cpu_policy_get_msrs(xc_interface *xch,
>> +                           const xc_cpu_policy_t *policy,
>> +                           const xen_msr_entry_t **msrs,
>> +                           uint32_t *nr)
>> +{
>> +    if ( !policy )
>> +    {
>> +        ERROR("Failed to fetch MSRs from policy object");
>> +        errno = -EINVAL;
>> +        return -1;
>> +    }
>> +
>> +    *msrs = policy->msrs;
>> +    *nr = policy->nr_msrs;
>> +
>> +    return 0;
>> +}
> 
> My preference would probably be to return NULL or
> xen_{leaf,msr}_entry_t * from those, as we can then avoid an extra
> leaves/msrs parameter.  Again I'm fine with leaving it like this.
> 

It didn't feel right to have an output parameter as the return value
doubling as status code when another output is in the parameter list. I
can perfectly imagine someone grabbing "nr" and ignoring "msrs" because
"msrs" doesn't happen to be needed for them.

I think there's extra safety in making it harder to ignore the error.

>> -    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
>> -    if ( cpuid.length )
>> +    record = (struct xc_sr_record) {
>> +        .type = REC_TYPE_X86_CPUID_POLICY,
>> +        .data = policy->leaves,
>> +        .length = policy->nr_leaves * sizeof(*policy->leaves),
>> +    };
>> +    if ( record.length )
>>      {
>> -        rc = write_record(ctx, &cpuid);
>> +        rc = write_record(ctx, &record);
>>          if ( rc )
>>              goto out;
>>      }
> 
> 
> You could maybe write this as:
> 
> if ( policy->nr_leaves )
> {
>     const struct xc_sr_record r = {
>         .type = REC_TYPE_X86_CPUID_POLICY,
>         .data = policy->leaves,
>         .length = policy->nr_leaves * sizeof(*policy->leaves),
>     };
> 
>     rc = write_record(ctx, &record);
> }
> 
> (same for the msr record)
> 

Ack. Looks nicer that way.

>>  
>> -    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
>> -    if ( msrs.length )
>> +    record = (struct xc_sr_record) {
>> +        .type = REC_TYPE_X86_MSR_POLICY,
>> +        .data = policy->msrs,
>> +        .length = policy->nr_msrs * sizeof(*policy->msrs),
>> +    };
>> +    if ( record.length )
>>      {
>> -        rc = write_record(ctx, &msrs);
>> +        rc = write_record(ctx, &record);
>>          if ( rc )
>>              goto out;
>>      }
>> @@ -100,8 +84,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
>>      rc = 0;
>>  
>>   out:
>> -    free(cpuid.data);
>> -    free(msrs.data);
>>      xc_cpu_policy_destroy(policy);
>>  
>>      return rc;
>> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
>> index 8893547bebce..1c9ba6d32060 100644
>> --- a/tools/misc/xen-cpuid.c
>> +++ b/tools/misc/xen-cpuid.c
>> @@ -409,17 +409,21 @@ static void dump_info(xc_interface *xch, bool detail)
>>      free(fs);
>>  }
>>  
>> -static void print_policy(const char *name,
>> -                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
>> -                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
>> +static void print_policy(xc_interface *xch, const char *name, const xc_cpu_policy_t *policy)
> 
> Line length.

Ack

> 
>>  {
>> -    unsigned int l;
>> +    const xen_cpuid_leaf_t *leaves;
>> +    const xen_msr_entry_t *msrs;
>> +    uint32_t nr_leaves, nr_msrs;
>> +
>> +    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) ||
>> +         xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
>> +        err(1, "print_policy()");
> 
> Shouldn't the error message be "xc_cpu_policy_get_{leaves,msrs}()"
> instead, as one of those is the cause of the error?
> 
> Other err() usages do print the function triggering the error, not the
> function context name.
> 

Sure. I've demultiplexed them into separate conditionals as well now.

>>  
>>      printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
>>      printf(" CPUID:\n");
>>      printf("  %-8s %-8s -> %-8s %-8s %-8s %-8s\n",
>>             "leaf", "subleaf", "eax", "ebx", "ecx", "edx");
>> -    for ( l = 0; l < nr_leaves; ++l )
>> +    for ( uint32_t l = 0; l < nr_leaves; ++l )
>>      {
>>          /* Skip empty leaves. */
>>          if ( !leaves[l].a && !leaves[l].b && !leaves[l].c && !leaves[l].d )
>> @@ -432,7 +436,7 @@ static void print_policy(const char *name,
>>  
>>      printf(" MSRs:\n");
>>      printf("  %-8s -> %-16s\n", "index", "value");
>> -    for ( l = 0; l < nr_msrs; ++l )
>> +    for ( uint32_t l = 0; l < nr_msrs; ++l )
> 
> I would be tempted to leave `l` as-is, seeing as there's no real need
> to modify it in the patch context, and the patch is already fairly
> long.

done

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:15:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:15:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727760.1132422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ngQ-0004E6-LR; Wed, 22 May 2024 15:15:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727760.1132422; Wed, 22 May 2024 15:15:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ngQ-0004Dv-HI; Wed, 22 May 2024 15:15:14 +0000
Received: by outflank-mailman (input) for mailman id 727760;
 Wed, 22 May 2024 15:15:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eEN6=MZ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s9ngP-0003wy-8r
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:15:13 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15410651-184e-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 17:15:11 +0200 (CEST)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-34f0e55787aso4013910f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 08:15:11 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354db76acafsm1864576f8f.49.2024.05.22.08.15.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 08:15:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15410651-184e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716390911; x=1716995711; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kK3dRrXrwVNPXE4+0A/1KWYbPmGy2JdF5u4c4yDy9s4=;
        b=F5+Fies9vkolU8Q7W7A0cXxkxZaFFM1LOlH73VoqFVytxPtEk3NeO6Aa8eGCuv1Oj3
         KLzj1JNV8EkZ9+IE2RiLWXTHfc00fzzb/I+W1rHvXfjrafKJ2mh2DzWiWUaP3kjsGbTX
         v522lZUZh+nPB9xZoC2fmUgYd6QZtFXGQoZak=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716390911; x=1716995711;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kK3dRrXrwVNPXE4+0A/1KWYbPmGy2JdF5u4c4yDy9s4=;
        b=hoLOchSaUwjZxnDadbIhChEmeoYybx84DdDUzD3k25EbmrwzOUpmoFo/c9vbkJEqGA
         r/ELbiiB0qz4eiS3jhROpL+7nxJ+JeYQ9USmnCkZPm4VlT/WzU0Er73+lwT+b6+VATlO
         OUftw4IKgL4EpVW3kl2nVUzGvPSDcZ6DQflAU4rSM4nl/OyB0vx5P5DLTlkFjZL8+t3X
         lgP/mdLYDEvSDuM1mXFT9MmJPcBUea3iCvVutOB7d6m+GNfudVCzsW4D1xzjMLkRdJPs
         fk3Yjclik4SIUlunr6at66eyjYRkA/2KFciCHBBKnaxmG8FmH1m3M/Gm/NrhE9O7KL6q
         02gQ==
X-Gm-Message-State: AOJu0Yynm2fFcgqWfI9uFe59ORPh7Z+zsEq8AFOZhtT5g7rjb4JKSWuT
	6KEEECKs7nohiLsxFnAhujMpVPkv4NTSxXZpza+inSFf9QA2OwZi1dlyyYLt91k=
X-Google-Smtp-Source: AGHT+IG9GddBwM5c5CCKKh8vIp9biDptjqXlZAk9WQ+OyCsyroo+12gATRP4E15F30O4WDlyURbPxA==
X-Received: by 2002:adf:f104:0:b0:34d:8d88:7c53 with SMTP id ffacd0b85a97d-354d8bd1408mr1705577f8f.0.1716390910943;
        Wed, 22 May 2024 08:15:10 -0700 (PDT)
Message-ID: <974bfbc6-97f6-4afa-978c-562448042d4a@cloud.com>
Date: Wed, 22 May 2024 16:15:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/2] tools/xg: Clean up xend-style overrides for CPU
 policies
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
 <d397ec0de4138e32feeb910f3401a6568a75035e.1715954111.git.alejandro.vallejo@cloud.com>
 <Zktl8zRt1kue5vA6@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zktl8zRt1kue5vA6@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20/05/2024 16:02, Roger Pau Monné wrote:
>> -
>>  static int xc_msr_policy(xc_interface *xch, domid_t domid,
>> -                         const struct xc_msr *msr)
>> +                         const struct xc_msr *msr,
>> +                         xc_cpu_policy_t *host,
>> +                         xc_cpu_policy_t *def,
> 
> host and def should likely be const?

I tried, but I can't. All policies go through find_msr(), which takes a
non-const policy, and must be non-const because it's also used for the
cur policy.

I did the next best thing (I think) by const-ifying the result of
find_msr inside the loop for host and def. Same thing on the cpuid function.

>> -    if ( rc )
>> -    {
>> -        PERROR("Failed to obtain host policy");
>> -        rc = -errno;
>> -        goto out;
>> -    }
>> +    if ( !msrs )
> 
> Does this build?  Where is 'msrs' defined in this context?  The
> function parameter is 'msr' AFAICT.

Ugh. I fixed that while adjusting it for testing within XenServer and
then neglected to make the change in the actual for-upstream patches.

You're right.

> 
>> +        return 0;
> 
> Should we also check for host, def, cur != NULL also?

It's already done by the caller, but can do out of paranoia; returning
-EINVAL.

>> @@ -583,14 +436,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>      int rc;
>>      bool hvm;
>>      xc_domaininfo_t di;
>> -    struct xc_cpu_policy *p = xc_cpu_policy_init();
>> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
>> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>> -    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
>> -    uint32_t len = ARRAY_SIZE(host_featureset);
>>  
>> -    if ( !p )
>> -        return -ENOMEM;
>> +    struct xc_cpu_policy *host = xc_cpu_policy_init();
>> +    struct xc_cpu_policy *def = xc_cpu_policy_init();
> 
> I would be helpful to have some kind of mechanism to allocate + init a
> policy at the same time, so that the resulting object could be made
> const here.  (Not that you need to do it in this patch).

That would seem sensible, but we'd also need a way to clone it to avoid
repeating hypercalls when they aren't required. I had a patch that did
that, but was quite complicated for other reasons. I might get back to
it at some point now that per-vCPU policies don't seem to be required.

>> @@ -695,24 +542,24 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>                   !(dfs = x86_cpu_policy_lookup_deep_deps(b)) )
>>                  continue;
>>  
>> -            for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
>> +            for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
> 
> All this loop index type changes could be done as a separate patch,
> you are not even touching the surrounding lines.  It adds a lot of
> churn to this patch for no reason IMO.

I got carried away. Let me revert that. I still want to get rid of all
those overscoped indices, but this is not the patch for it.

>> @@ -772,49 +619,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>               * apic_id_size values greater than 7.  Limit the value to
>>               * 7 for now.
>>               */
>> -            if ( p->policy.extd.nc < 0x7f )
>> +            if ( cur->policy.extd.nc < 0x7f )
>>              {
>> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
>> -                    p->policy.extd.apic_id_size++;
>> +                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )
> 
> I would split the line while there, it's overly long.

Ack

> 
> Thanks, Roger.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:22:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:22:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727771.1132432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9nnX-0006Ta-Ar; Wed, 22 May 2024 15:22:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727771.1132432; Wed, 22 May 2024 15:22:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9nnX-0006TT-7x; Wed, 22 May 2024 15:22:35 +0000
Received: by outflank-mailman (input) for mailman id 727771;
 Wed, 22 May 2024 15:22:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9nnV-0006TN-KE
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:22:33 +0000
Received: from wfhigh8-smtp.messagingengine.com
 (wfhigh8-smtp.messagingengine.com [64.147.123.159])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1a3a7317-184f-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 17:22:31 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.west.internal (Postfix) with ESMTP id E16181800132;
 Wed, 22 May 2024 11:22:27 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Wed, 22 May 2024 11:22:28 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 11:22:26 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a3a7317-184f-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716391347;
	 x=1716477747; bh=cKcXdNX7CaZv/PzfX4FoVyVEW4o/dLnJS3xNLqLVInQ=; b=
	N/Yz3/HRkaaxjFQ8+83RXSS7Ue9U6kkL9TcsQ8Tqmqn6+m9X1JPx0HQ9IJEvKXX5
	qGKQX1+ns3kWeFwKZq4sYlaFvLzpdOSWNdHBzAmlm6PTLkj3iZWIs0LQn+dpqxu4
	r9jlSK9fPCOqXg9+HlSBUURxl30CFNhOunwP81JtHvnmsWTltR4jx3KnvgoXeT++
	PqrHmiV0gUjpuc9Nfnnx0LWybstyCpQm3DxtfMYGUahlo3ynkCtIw94zVfgCsqcz
	7QRTIaFo31XdhOxbsXbUHSnqBTWsnUFbYzq7v9He6w3ghK/3bbO3UNo7HANcylyw
	oBYdSUPIdUQ8hIDMsGcjIQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716391347; x=1716477747; bh=cKcXdNX7CaZv/PzfX4FoVyVEW4o/
	dLnJS3xNLqLVInQ=; b=f3y6WehXNNtGPRG8+tvjSQgeNy8aDu42LkIJiI+2lvOa
	sWhvcSoOCg6cZuLAbvqC87Ix83fTTsCTSfxFYHc1QzkwSwYTwc8ml3Bwwy6OXG7q
	k6j8bsI+aKRRjEn2N0as8a6kIFqXaR/iPgdxsUcExCIfx8B+yQCV8CErGOjux3Yg
	CV1EqrZcm5YmF9wjXqCnpk0+KgQPu7Gb+AmneQmucTyX7ZIHxeytywG5ToTO/mzR
	CHk4nYovfOEDw3LdpOCfIqmt6Sjc3zs9yAwLMwPsrbxk6Vx0XP9bm91hQmISa5J2
	giLYV9c7GyaXkbHs31ukazW9uPb6vTjOuSN4ejf/QQ==
X-ME-Sender: <xms:sw1OZpxJk7SQ403wVFVHXahfJ8vpQRv9KYyOufKhzEqH6OFQDbj_Hg>
    <xme:sw1OZpSbnqLTPvNwrDLU3XPqfs890dihM3P35oxG0epBExLwLfqWpl1S98lOn27jF
    Tcdd1nFbBpowQ>
X-ME-Received: <xmr:sw1OZjWOH3mpWIOfW8TQ7-hJB1gDInLRMo7SerpQugUDUdqCaYgD24DxIp93_Nlr4yefdugA8HkT7Y5UPPiedNS5bW7OPpiR_A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgvddvucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:sw1OZriW8j82jm6R5EjZU-viUeLv-cU8KOS2pAtxpE2RsmD-RciQMg>
    <xmx:sw1OZrBAl-sFy2txbIuMo8q5awSz6btRSBCZwqxfq2XSUdCoZP_-wA>
    <xmx:sw1OZkJHq_bpzdBIupdXSLwMlpw_JP30WrF6ixMioX810w7xEMjbcw>
    <xmx:sw1OZqA0aEgO16Xl1V5VFuUUkveXX8_MZGTmDmBZcsBlRcC2qYZ3HQ>
    <xmx:sw1OZiOTiF2mHV8XKxytQ0bgfsHW9jRbJYDmCBgTGWaAk4OYSjKKS6_V>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 22 May 2024 17:22:23 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only
Message-ID: <Zk4NsM_Df0futo-Z@mail-itl>
References: <cover.1b2f25e20efaa1fe1e6a2da565731becd27e7ead.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2596964a55f710a300143aa5ae195566bacb4e00.1716260066.git-series.marmarek@invisiblethingslab.com>
 <2708dbbb-f913-4920-a008-beef12cd1265@suse.com>
 <Zk3xtK0SaaqEadXY@mail-itl>
 <93c7230a-bd2a-4763-a90a-89e02a169b1d@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="uurQj1kwzf6TNO7b"
Content-Disposition: inline
In-Reply-To: <93c7230a-bd2a-4763-a90a-89e02a169b1d@suse.com>


--uurQj1kwzf6TNO7b
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 22 May 2024 17:22:23 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/2] x86/mm: add API for marking only part of a MMIO
 page read only

On Wed, May 22, 2024 at 03:29:51PM +0200, Jan Beulich wrote:
> On 22.05.2024 15:22, Marek Marczykowski-G=C3=B3recki wrote:
> > On Wed, May 22, 2024 at 09:52:44AM +0200, Jan Beulich wrote:
> >> On 21.05.2024 04:54, Marek Marczykowski-G=C3=B3recki wrote:
> >>> +static void subpage_mmio_write_emulate(
> >>> +    mfn_t mfn,
> >>> +    unsigned int offset,
> >>> +    const void *data,
> >>> +    unsigned int len)
> >>> +{
> >>> +    struct subpage_ro_range *entry;
> >>> +    void __iomem *addr;
> >>
> >> Wouldn't this better be pointer-to-volatile, with ...
> >=20
> > Shouldn't then most other uses of __iomem in the code base be this way
> > too? I see volatile only in few places...
>=20
> Quite likely, yet being consistent at least in new code is going to be
> at least desirable.

I tried. Build fails because iounmap() doesn't declare its argument as
volatile, so it triggers -Werror=3Ddiscarded-qualifiers...

I'll change it just in subpage_mmio_write_emulate(), but leave
subpage_mmio_map_page() (was _get_page) without volatile.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--uurQj1kwzf6TNO7b
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZODbAACgkQ24/THMrX
1yyN5Af+OMhdFfQA18mjMg7vv7WCWIURPTVXlCsHUvpCnnXxlrkrhT6NCUwMimgW
M1hS+jkbUHogu9rCxwad6hQ6Ex+OKat4qIVGep03NYoXFk+CyZJ0Ebv/QaLKTG27
8J4bqN3WHcaQxl6mTNB8S82iK1kpLdEaRWJJ4K8uhWdnL7Wg9FU6Rkce+wrqSH8m
QF6xaGGrGo2MEHBKSQfi2KFsnu/gfSZlV8u5jm/FeUQJLK1WacrnYpG2GU6AmxoL
x6J25dmVHG0Lr4z3v0KL0OrZX++pHcpW4ZQ14KQFv6zjA42IOSbAxSAi5zmrJOvj
f2sydZ0SiiisCaCt3GC38i8EN5oZfA==
=C0BQ
-----END PGP SIGNATURE-----

--uurQj1kwzf6TNO7b--


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:24:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:24:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727775.1132441 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9np8-0007A8-LG; Wed, 22 May 2024 15:24:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727775.1132441; Wed, 22 May 2024 15:24:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9np8-0007A1-IZ; Wed, 22 May 2024 15:24:14 +0000
Received: by outflank-mailman (input) for mailman id 727775;
 Wed, 22 May 2024 15:24:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eEN6=MZ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s9np6-00079t-UK
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:24:12 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57444265-184f-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 17:24:11 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-4202c1d19d5so41020775e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 08:24:11 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42012d08bb9sm396493725e9.48.2024.05.22.08.24.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 08:24:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57444265-184f-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716391451; x=1716996251; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Rf6xDJDdpe8kGsUFH2ndT9iUcGdRT3pfNMBT0dZE4Ao=;
        b=iwlp+Ziz2qb3MF0xUR2FO5GbG0zGWETpOxOqM9QXFbQBF6/R9tEeUhjQAFWLubWNkj
         xkWB0sEXoY+Y8gnX4t0vkMjpkGCjqDs2iv8LvvF/8eL0m5I5v3rsi1KF4UFfZ+nj5cZ5
         oFtq7lPdNdWoEznsBg7Xrb3xki0AkLdyUwNto=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716391451; x=1716996251;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Rf6xDJDdpe8kGsUFH2ndT9iUcGdRT3pfNMBT0dZE4Ao=;
        b=Inzbr1AyX9ef4UmLmPsMNNKSq10g7M1zPhKgEswaB6ujM76zS0iMcPHhpHQGngWHCC
         mahVc7xpzpWhy1mlyTITr5gWSx4rFAi7E/YN34O8iTeoqbx+BYn4OM9bCYuSq5nOSFf6
         COHnIC2ZGpAIiOxaBLgToDSmENCbSp64NtnUMzcnGSa545X2sAE3birVm5mrUg6rU8qK
         67QHOzahITdge/w/gJzZodHPY8iAaw30EKGW2GoO5EUtPD7v3Yk/GggHMJ9RrXjWN0Ka
         xqJBP4POOB7f7vr/7ml0FKYetO3y8EoQowhnHRsCmgyeDPfmzNtvgVVUiWq9Xy6QrY3R
         Bi7A==
X-Forwarded-Encrypted: i=1; AJvYcCXb4RNG+UPg8a9bTwtSV+nQ72tDqo+XcN83KX2izmBY39Xsd4wg19vAsOEFI3MSYukaFZK5BBvtHs5oPO4vnGsrM1ou1X1HCO4WY11lEsk=
X-Gm-Message-State: AOJu0Ywx1KOpJmTJmXUSIn9C7YlZAq9fbU505vY/NorEBhdGMTwCp3oa
	Kz4K/BoFRgdT3uzwOw65s6QS90O0aHE2/6ko2By+sMBZJj6+JC9rJO4DfAU7WJU=
X-Google-Smtp-Source: AGHT+IGntyWFU40ojQ5knSm1Bci7f+tmdftoursU7vfJURfGcaN01SPt1bpaXkRtT/LoTrIj/rvCNA==
X-Received: by 2002:a7b:cb42:0:b0:420:2986:ccee with SMTP id 5b1f17b1804b1-420fd34f1eemr21738355e9.30.1716391451329;
        Wed, 22 May 2024 08:24:11 -0700 (PDT)
Message-ID: <48f1b4e2-015b-4a2e-a17e-567aa442a455@cloud.com>
Date: Wed, 22 May 2024 16:24:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
 <6a21a57f-e0fa-43af-a7ea-d813e4b2976d@suse.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <6a21a57f-e0fa-43af-a7ea-d813e4b2976d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22/05/2024 10:33, Jan Beulich wrote:
> On 08.05.2024 14:39, Alejandro Vallejo wrote:
>> Otherwise it's not possible to call functions described in hvm/vlapic.h from the
>> inline functions of hvm/hvm.h.
>>
>> This is because a static inline in vlapic.h depends on hvm.h, and pulls it
>> transitively through vpt.h. The ultimate cause is having hvm.h included in any
>> of the "v*.h" headers, so break the cycle moving the guilty inline into hvm.h.
>>
>> No functional change.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> 
> In principle:
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> But see below for one possible adjustment.
> 
>> ---
>> v2:
>>   * New patch. Prereq to moving vlapic_cpu_policy_changed() onto hvm.h
> 
> That hook invocation living outside of hvm/hvm.h was an outlier anyway,
> so even without the planned further work this is probably a good move.
> 
>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>> @@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
>>          alternative_vcall(hvm_funcs.update_vlapic_mode, v);
>>  }
>>  
>> +static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
>> +{
>> +    if ( hvm_funcs.sync_pir_to_irr )
>> +        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
>> +}
> 
> The hook doesn't have "vlapic" in its name. Therefore instead or prepending
> hvm_ to the original name or the wrapper, how about replacing the vlapic_
> that was there. That would then also fit better with the naming scheme used
> for other hooks and their wrappers. Happy to adjust while committing, so
> long as you don't disagree.
> 
> Jan

Sounds reasonable. I wasn't sure whether vlapic was adding anything more
than a namespace prefix to the function name. Are you happy to adjust
that on commit?

If so, I'm good with it in the form you propose.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:34:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:34:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727782.1132452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9nzK-0000lw-I4; Wed, 22 May 2024 15:34:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727782.1132452; Wed, 22 May 2024 15:34:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9nzK-0000lp-FU; Wed, 22 May 2024 15:34:46 +0000
Received: by outflank-mailman (input) for mailman id 727782;
 Wed, 22 May 2024 15:34:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eEN6=MZ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1s9nzJ-0000lj-S0
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:34:45 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cf411ce6-1850-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 17:34:42 +0200 (CEST)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2e3b1b6e9d1so78526081fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 08:34:42 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-41fccce1912sm505463745e9.11.2024.05.22.08.34.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 08:34:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf411ce6-1850-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716392082; x=1716996882; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=hmR4Bg5ThxDHXEMpbzWp/r8/uiTssBYthxCmdU0/VQQ=;
        b=e9Ie3xOwBUkAhuuO30lFd9stKOCNutf16eduO67YxGQcTVNGR4Z5onqc+hP2F2Tz9W
         zOM/VoTQ65uKTspMKN8UZH8Vi1D0R3wFn+eTtwT8T9r9Tzp7PEN5p+scdxxaQW6B6POu
         RPPekst+pyQqXLzekjPEv4KoSJCT1nhALuMpQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716392082; x=1716996882;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hmR4Bg5ThxDHXEMpbzWp/r8/uiTssBYthxCmdU0/VQQ=;
        b=ESUPWzgp1HYupy+VPvvREdCiumckbyhhjqupMTYHn/ISfIyFW4Muiy6gBoB6EZPpau
         oDuVbgmdUrNt1j8DXyz1Yj33EbaDSVN994CeIvuMMoilpA/0RWrCOso1xo9vLzyzMXIG
         +dDG9dmraNDCSpkbl4KG7oJ+cuzR54l3qtxeyOPjUCbcHzzaEFO0TzplhIwj9BC2obxC
         aXQwFfg6olYLkKEmkIXTWCqCmroOl4EhpM1tuDXzeS2xoLD4rYRY07HKQOcZHPKQkRLF
         Ri45FG4mbPLk5v7+fSByCK2cVONuD9UMDwDtKNN6JIraQsHx+Xxmxu+Wxfj5i8H6lawI
         DPZQ==
X-Gm-Message-State: AOJu0YxwQHfKWua1vcv6FKVy4S7vhrKHztmSWkzXMeVhM+QVHBpWFvAt
	RLQbZl5t7/O2yQIJI25qDVuWhSqWb/n9aRiRTcfao75sC0vuvKZ5QaD+sfr8i28=
X-Google-Smtp-Source: AGHT+IE6Pzj4pRgo9ZEsm6G/fs7U33sy8gDSLZeTFJJtdCLxrfXNJaayQiGkITVzjyYksqdzkmqWcw==
X-Received: by 2002:a2e:9b54:0:b0:2de:d48a:953b with SMTP id 38308e7fff4ca-2e9494942f0mr15652851fa.23.1716392082110;
        Wed, 22 May 2024 08:34:42 -0700 (PDT)
Message-ID: <9641be72-76e4-4960-9e45-19c9d329d6cf@cloud.com>
Date: Wed, 22 May 2024 16:34:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 0/3] Clean the policy manipulation path in domain
 creation
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715954111.git.alejandro.vallejo@cloud.com>
 <ZktGBkfwxEzrbojw@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZktGBkfwxEzrbojw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20/05/2024 13:45, Roger Pau Monné wrote:
> On Fri, May 17, 2024 at 05:08:33PM +0100, Alejandro Vallejo wrote:
>> v2:
>>   * Removed xc_cpu_policy from xenguest.h
>>   * Added accessors for xc_cpu_policy so the serialised form can be extracted.
>>   * Modified xen-cpuid to use accessors.
>>
>> ==== Original cover letter ====
>>
>> In the context of creating a domain, we currently issue a lot of hypercalls
>> redundantly while populating its CPU policy; likely a side effect of
>> organic growth more than anything else.
>>
>> However, the worst part is not the overhead (this is a glacially cold
>> path), but the insane amounts of boilerplate that make it really hard to
>> pick apart what's going on. One major contributor to this situation is the
>> fact that what's effectively "setup" and "teardown" phases in policy
>> manipulation are not factored out from the functions that perform said
>> manipulations, leading to the same getters and setter being invoked many
>> times, when once each would do.
>>
>> Another big contributor is the code being unaware of when a policy is
>> serialised and when it's not.
>>
>> This patch attempts to alleviate this situation, yielding over 200 LoC
>> reduction.
>>
>> Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
>>          from clients of libxc/libxg.
>> Patch 2: Changes the (de)serialization wrappers in xenguest so they always
>>          serialise to/from the internal buffers of xc_cpu_policy_t. The
>>          struct is suitably expanded to hold extra information required.
>> Patch 3: Performs the refactor of the policy manipulation code so that it
>>          follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.
> 
> I think patch 3 is no longer part of the set?  I don't see anything
> in the review of v1 that suggests patch 3 was not going to be part of
> the next submission?
> 
> Thanks, Roger.

It's there, there was just a shift. The implication of the first line of
the changelog is that v1/patch1 was dropped. Sorry, should've been clearer.

    v1/patch1 => dropped
    v1/patch2 => v2/patch1
    v1/patch3 => v2/patch2

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:39:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:39:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727788.1132461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o3x-0001ME-3z; Wed, 22 May 2024 15:39:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727788.1132461; Wed, 22 May 2024 15:39:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o3x-0001M7-18; Wed, 22 May 2024 15:39:33 +0000
Received: by outflank-mailman (input) for mailman id 727788;
 Wed, 22 May 2024 15:39:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9o3w-0001M1-Ay
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:39:32 +0000
Received: from wfhigh8-smtp.messagingengine.com
 (wfhigh8-smtp.messagingengine.com [64.147.123.159])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 799956f0-1851-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 17:39:30 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.west.internal (Postfix) with ESMTP id 3A12E180012F;
 Wed, 22 May 2024 11:39:27 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Wed, 22 May 2024 11:39:27 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 11:39:25 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 799956f0-1851-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1716392366; x=1716478766; bh=lGY7DJZBxL+CSFvhqJjMHTmBfik4rcS3
	gTRhssuYaZ8=; b=Iiw+AOXlGdl+c+f10NI2bno8nJk3UoFxp+ZLCzlT4jYtZs56
	LnpbBU8OAILj2Kv35QJg5LDVd1seZlen8d+22v9MuzINNEBWe9gQgGk0/Y60GZ+w
	h29kgVthnjDu53YNg6YC/8MQ7u31kpx2H8BTiq7rtq003HtViLNEcHzdYxwQLWvO
	QJYvgRGHCd8jiV/EO0GkAiiJ0JCOlLRH6JTwtfRSjHQiSdPS+/JwLueTVwDqLuo5
	TM9piVT3tRQgwxnxO7V50rQXeB3LZ/Tkxz4X+sZ8E8lz5zpLzUVg1Y0b6rBn0N/c
	zxxK+Jiy85qcnG+pIg6VvhZBlsGikx76aSm93A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1716392366; x=1716478766; bh=lGY7DJZBxL+CS
	FvhqJjMHTmBfik4rcS3gTRhssuYaZ8=; b=C/k5DkhTKriR7aZ7MKknPLu9wLF/c
	DG9BgmSCi0+ENRN0JeuF0tf0W1S21musTBg0NlDkaAeGotiv9lYawX6HyfgfO1Pn
	AlfYHMwH1P6v34XwHk6cYDs08zC7uNKQkEHOpsyP2ZjDxNNXXa5WKhEgx9DaBxIh
	GhymfGChl8GWPUbDMVObidO0/8MR2n+lGY0aLDftaT6D0mIA+DS1CoMmcMnM4k1q
	un97wDfDSi3tTzJ8qkndC1x83VlmO/6FKI9BMZTJ3qaQIQRNmleRfCk40u+40K+T
	U7Sfoj3c2BpnLqiKLAENWsKrw4ZIWAO3I94sm6v/fQjSwFjj9cUj8ibTg==
X-ME-Sender: <xms:rhFOZtmPWjXta8VwIZ4QK1U3rziyRgdrs2htBraE7CuthOBvB3a1aA>
    <xme:rhFOZo1RziaNV8Ji3tyUg8gy9rqIoHJ-WQ02DQIj5zFWuM6LfM7P5ExOtuLjRxV1N
    c5EBbdBgbolwA>
X-ME-Received: <xmr:rhFOZjq_eU78GL7Rk65hDESpea8uuY_wfNOBjDc5-Ptlu3INKxxQ-V8JspFzxDDnqrBfS3Y7ZGMHQY4KQ5AIF6ozflYWdV0m1Iz7WsislS8RCls0DoE>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgvdegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggt
    khhiuceomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
    eqnecuggftrfgrthhtvghrnhepleekhfduleetleelleetteevfeefteffkeetteejheel
    gfegkeelgeehhfdthedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg
    hilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdr
    tghomh
X-ME-Proxy: <xmx:rhFOZtnz_6Xwc25J7oZ1zfM3stvjB1v80qt9iitZSf-r1JBdaZxdsA>
    <xmx:rhFOZr2NX9npgq-QKGuZnoFlJxdPGrU4e_esQ9eeXncXO1TYmy0R3g>
    <xmx:rhFOZsvDKXgxy9jzR6YF3NBAPx5FmpYKzyHoRacwj5BFXsur97p-HA>
    <xmx:rhFOZvVQcqt4XIa2zcP3F3ilV8WWE3_Uf53uevHIcUE571OreNSgTA>
    <xmx:rhFOZk9BJn8b2LG-sATIkFQ_DzBqboYKxZkSFNxahxHZP5O9je6gMRP6>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH v4 0/2] Add API for making parts of a MMIO page R/O and use it in XHCI console
Date: Wed, 22 May 2024 17:39:02 +0200
Message-ID: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On older systems, XHCI xcap had a layout that no other (interesting) registers
were placed on the same page as the debug capability, so Linux was fine with
making the whole page R/O. But at least on Tiger Lake and Alder Lake, Linux
needs to write to some other registers on the same page too.

Add a generic API for making just parts of an MMIO page R/O and use it to fix
USB3 console with share=yes or share=hwdom options. More details in commit
messages.

Marek Marczykowski-Górecki (2):
  x86/mm: add API for marking only part of a MMIO page read only
  drivers/char: Use sub-page ro API to make just xhci dbc cap RO

 xen/arch/x86/hvm/emulate.c      |   2 +-
 xen/arch/x86/hvm/hvm.c          |   4 +-
 xen/arch/x86/include/asm/mm.h   |  25 +++-
 xen/arch/x86/mm.c               | 273 +++++++++++++++++++++++++++++++++-
 xen/arch/x86/pv/ro-page-fault.c |   6 +-
 xen/drivers/char/xhci-dbc.c     |  36 ++--
 6 files changed, 327 insertions(+), 19 deletions(-)

base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:39:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:39:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727789.1132471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o40-0001bq-DO; Wed, 22 May 2024 15:39:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727789.1132471; Wed, 22 May 2024 15:39:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o40-0001bj-Aj; Wed, 22 May 2024 15:39:36 +0000
Received: by outflank-mailman (input) for mailman id 727789;
 Wed, 22 May 2024 15:39:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9o3z-0001b1-3h
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:39:35 +0000
Received: from wfhigh8-smtp.messagingengine.com
 (wfhigh8-smtp.messagingengine.com [64.147.123.159])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7c03ffad-1851-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 17:39:33 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.west.internal (Postfix) with ESMTP id A5E7B180013E;
 Wed, 22 May 2024 11:39:31 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Wed, 22 May 2024 11:39:32 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 11:39:29 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c03ffad-1851-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1716392371; x=1716478771; bh=y+sSs98gGM
	+UCF1S1MrkA/IvESQ3dOFmWnvEWkSSLzI=; b=e9B4gvJbSv9Sf9CXXGnS36UnHD
	kY4BgHm0Gk7eIIrNpAYLkvL8kFuyA5bkCvKhr02VqtRV9hyvwmGHdh2gDXr+kktw
	pXQHgjxASuzuwNxUmIUsTcEsKTJOWUdxrUQY32uh+i+WYAaYREW22wGar1jdsIIW
	fqq1FHbJNrFTZZAQDesvIlviRxlspo4ax8gqLD19Z8P3pgd/pfN6lpshZHCBHlEV
	lyRaC2UVX254MT1jgylMrcpujG4BWq2TANq7yiKGleKc7oVdEaUFqjEgx0XVf7r1
	W3lSGwVJ0KQIZkT9CzTJ8FR130F5Nvi3wbOtqDK5JbUQ14AvbbGQ7nx42iZQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1716392371; x=
	1716478771; bh=y+sSs98gGM+UCF1S1MrkA/IvESQ3dOFmWnvEWkSSLzI=; b=L
	57UiV40DjM/rz79neWqQk5ov/hE9Ra9xoONNrXU22l3xCaHo1hDFbnbdezRanV4v
	lQa0MZd2aV+gm01t564vREzQTgs+HO2E/OUIFjzeEZtdjuhT43hM5k9fHMiBpj7r
	xld+Tt6RQIhVNEkQfoDEl7YGz74px6Q8jhOc1Xku/z8k/2BF9UyGYefe6sbtnhJ8
	zwZLmVZVuVIjOlen/2sQFbzO81vYUY6lAAI9fwpE/3LAnTav1ub8SGRJtko0Xoie
	MIxFwLnPxa3xbUe6YQOvcAQSa6I0ANOvgnAnXbrSs5HymYLgrKVPAFMiNy0o5bbQ
	EnnKNE+Cmcc5LoBUz2JCA==
X-ME-Sender: <xms:sxFOZpCsinPyw8KO6x29XiJh54zNM_qZX17xvM7Cp3lWr3oLjYDOIg>
    <xme:sxFOZnj1ygU6Tu3A3J7f7Wu6cG4x30VaoNgirzb6jVBypB3ifIATF9i7T8oj7PqVU
    hiyZ8yliXWUkQ>
X-ME-Received: <xmr:sxFOZkms9CHdXbIm_EH64S6mQrt_1PULu4D6_OJ8xJrtOAcW1gBFhuWzgdPXCf1c8wvfJS32WIDSxIqTV5spl9rcTxCqsKDQ14vWfnRfCXo1X7-MAco>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgvdegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:sxFOZjxdJq7pZXMsd8ixStGOJfASL6h6dGuCCtb8w2j7QSf7I8fM9A>
    <xmx:sxFOZuRerdV-vAnOE7yP8d92w2VtbJpJTnag_SdovkewFyInRLDXgg>
    <xmx:sxFOZmbbKHGGRIHGowGWxmA7930awCrsshuUryCGO0DwGvEu7ZZomg>
    <xmx:sxFOZvSfKAw3Jwih3_jiRDeObqtPQl4NE21Tyr-wQN4YcJiCj6uKzA>
    <xmx:sxFOZkO5ysu5PRdcKXBC0WmtYrF9sUlNJi-piowwyYjltZuUkdT1La2x>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v4 1/2] x86/mm: add API for marking only part of a MMIO page read only
Date: Wed, 22 May 2024 17:39:03 +0200
Message-ID: <30562c807ff2e434731a76d7110d48614a58884b.1716392340.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
References: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In some cases, only few registers on a page needs to be write-protected.
Examples include USB3 console (64 bytes worth of registers) or MSI-X's
PBA table (which doesn't need to span the whole table either), although
in the latter case the spec forbids placing other registers on the same
page. Current API allows only marking whole pages pages read-only,
which sometimes may cover other registers that guest may need to
write into.

Currently, when a guest tries to write to an MMIO page on the
mmio_ro_ranges, it's either immediately crashed on EPT violation - if
that's HVM, or if PV, it gets #PF. In case of Linux PV, if access was
from userspace (like, /dev/mem), it will try to fixup by updating page
tables (that Xen again will force to read-only) and will hit that #PF
again (looping endlessly). Both behaviors are undesirable if guest could
actually be allowed the write.

Introduce an API that allows marking part of a page read-only. Since
sub-page permissions are not a thing in page tables (they are in EPT,
but not granular enough), do this via emulation (or simply page fault
handler for PV) that handles writes that are supposed to be allowed.
The new subpage_mmio_ro_add() takes a start physical address and the
region size in bytes. Both start address and the size need to be 8-byte
aligned, as a practical simplification (allows using smaller bitmask,
and a smaller granularity isn't really necessary right now).
It will internally add relevant pages to mmio_ro_ranges, but if either
start or end address is not page-aligned, it additionally adds that page
to a list for sub-page R/O handling. The list holds a bitmask which
qwords are supposed to be read-only and an address where page is mapped
for write emulation - this mapping is done only on the first access. A
plain list is used instead of more efficient structure, because there
isn't supposed to be many pages needing this precise r/o control.

The mechanism this API is plugged in is slightly different for PV and
HVM. For both paths, it's plugged into mmio_ro_emulated_write(). For PV,
it's already called for #PF on read-only MMIO page. For HVM however, EPT
violation on p2m_mmio_direct page results in a direct domain_crash() for
non hardware domains.  To reach mmio_ro_emulated_write(), change how
write violations for p2m_mmio_direct are handled - specifically, check
if they relate to such partially protected page via
subpage_mmio_write_accept() and if so, call hvm_emulate_one_mmio() for
them too. This decodes what guest is trying write and finally calls
mmio_ro_emulated_write(). The EPT write violation is detected as
npfec.write_access and npfec.present both being true (similar to other
places), which may cover some other (future?) cases - if that happens,
emulator might get involved unnecessarily, but since it's limited to
pages marked with subpage_mmio_ro_add() only, the impact is minimal.
Both of those paths need an MFN to which guest tried to write (to check
which part of the page is supposed to be read-only, and where
the page is mapped for writes). This information currently isn't
available directly in mmio_ro_emulated_write(), but in both cases it is
already resolved somewhere higher in the call tree. Pass it down to
mmio_ro_emulated_write() via new mmio_ro_emulate_ctxt.mfn field.

This may give a bit more access to the instruction emulator to HVM
guests (the change in hvm_hap_nested_page_fault()), but only for pages
explicitly marked with subpage_mmio_ro_add() - so, if the guest has a
passed through a device partially used by Xen.
As of the next patch, it applies only configuration explicitly
documented as not security supported.

The subpage_mmio_ro_add() function cannot be called with overlapping
ranges, and on pages already added to mmio_ro_ranges separately.
Successful calls would result in correct handling, but error paths may
result in incorrect state (like pages removed from mmio_ro_ranges too
early). Debug build has asserts for relevant cases.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Shadow mode is not tested, but I don't expect it to work differently than
HAP in areas related to this patch.

Changes in v4:
- rename SUBPAGE_MMIO_RO_ALIGN to MMIO_RO_SUBPAGE_GRAN
- guard subpage_mmio_write_accept with CONFIG_HVM, as it's used only
  there
- rename ro_qwords to ro_elems
- use unsigned arguments for subpage_mmio_ro_remove_page()
- use volatile for __iomem
- do not set mmio_ro_ctxt.mfn for mmcfg case
- comment where fields of mmio_ro_ctxt are used
- use bool for result of __test_and_set_bit
- do not open-code mfn_to_maddr()
- remove leftover RCU
- mention hvm_hap_nested_page_fault() explicitly in the commit message
Changes in v3:
- use unsigned int for loop iterators
- use __set_bit/__clear_bit when under spinlock
- avoid ioremap() under spinlock
- do not cast away const
- handle unaligned parameters in release build
- comment fixes
- remove RCU - the add functions are __init and actual usage is only
  much later after domains are running
- add checks overlapping ranges in debug build and document the
  limitations
- change subpage_mmio_ro_add() so the error path doesn't potentially
  remove pages from mmio_ro_ranges
- move printing message to avoid one goto in
  subpage_mmio_write_emulate()
Changes in v2:
- Simplify subpage_mmio_ro_add() parameters
- add to mmio_ro_ranges from within subpage_mmio_ro_add()
- use ioremap() instead of caller-provided fixmap
- use 8-bytes granularity (largest supported single write) and a bitmap
  instead of a rangeset
- clarify commit message
- change how it's plugged in for HVM domain, to not change the behavior for
  read-only parts (keep it hitting domain_crash(), instead of ignoring
  write)
- remove unused subpage_mmio_ro_remove()
---
 xen/arch/x86/hvm/emulate.c      |   2 +-
 xen/arch/x86/hvm/hvm.c          |   4 +-
 xen/arch/x86/include/asm/mm.h   |  25 +++-
 xen/arch/x86/mm.c               | 273 +++++++++++++++++++++++++++++++++-
 xen/arch/x86/pv/ro-page-fault.c |   6 +-
 5 files changed, 305 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index ab1bc516839a..e98513afc69b 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -2735,7 +2735,7 @@ int hvm_emulate_one_mmio(unsigned long mfn, unsigned long gla)
         .write      = mmio_ro_emulated_write,
         .validate   = hvmemul_validate,
     };
-    struct mmio_ro_emulate_ctxt mmio_ro_ctxt = { .cr2 = gla };
+    struct mmio_ro_emulate_ctxt mmio_ro_ctxt = { .cr2 = gla, .mfn = _mfn(mfn) };
     struct hvm_emulate_ctxt ctxt;
     const struct x86_emulate_ops *ops;
     unsigned int seg, bdf;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9594e0a5c530..73bbfe2bdc99 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -2001,8 +2001,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,
         goto out_put_gfn;
     }
 
-    if ( (p2mt == p2m_mmio_direct) && is_hardware_domain(currd) &&
-         npfec.write_access && npfec.present &&
+    if ( (p2mt == p2m_mmio_direct) && npfec.write_access && npfec.present &&
+         (is_hardware_domain(currd) || subpage_mmio_write_accept(mfn, gla)) &&
          (hvm_emulate_one_mmio(mfn_x(mfn), gla) == X86EMUL_OKAY) )
     {
         rc = 1;
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 98b66edaca5e..d04cf2c4165e 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -522,9 +522,34 @@ extern struct rangeset *mmio_ro_ranges;
 void memguard_guard_stack(void *p);
 void memguard_unguard_stack(void *p);
 
+/*
+ * Add more precise r/o marking for a MMIO page. Range specified here
+ * will still be R/O, but the rest of the page (not marked as R/O via another
+ * call) will have writes passed through.
+ * The start address and the size must be aligned to MMIO_RO_SUBPAGE_GRAN.
+ *
+ * This API cannot be used for overlapping ranges, nor for pages already added
+ * to mmio_ro_ranges separately.
+ *
+ * Since there is currently no subpage_mmio_ro_remove(), relevant device should
+ * not be hot-unplugged.
+ *
+ * Return values:
+ *  - negative: error
+ *  - 0: success
+ */
+#define MMIO_RO_SUBPAGE_GRAN 8
+int subpage_mmio_ro_add(paddr_t start, size_t size);
+#ifdef CONFIG_HVM
+bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla);
+#endif
+
 struct mmio_ro_emulate_ctxt {
         unsigned long cr2;
+        /* Used only for mmcfg case */
         unsigned int seg, bdf;
+        /* Used only for non-mmcfg case */
+        mfn_t mfn;
 };
 
 int cf_check mmio_ro_emulated_write(
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc7315..dab7cc018c3f 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -150,6 +150,17 @@ bool __read_mostly machine_to_phys_mapping_valid;
 
 struct rangeset *__read_mostly mmio_ro_ranges;
 
+/* Handling sub-page read-only MMIO regions */
+struct subpage_ro_range {
+    struct list_head list;
+    mfn_t mfn;
+    void __iomem *mapped;
+    DECLARE_BITMAP(ro_elems, PAGE_SIZE / MMIO_RO_SUBPAGE_GRAN);
+};
+
+static LIST_HEAD(subpage_ro_ranges);
+static DEFINE_SPINLOCK(subpage_ro_lock);
+
 static uint32_t base_disallow_mask;
 /* Global bit is allowed to be set on L1 PTEs. Intended for user mappings. */
 #define L1_DISALLOW_MASK ((base_disallow_mask | _PAGE_GNTTAB) & ~_PAGE_GLOBAL)
@@ -4910,6 +4921,265 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     return rc;
 }
 
+/*
+ * Mark part of the page as R/O.
+ * Returns:
+ * - 0 on success - first range in the page
+ * - 1 on success - subsequent range in the page
+ * - <0 on error
+ *
+ * This needs subpage_ro_lock already taken.
+ */
+static int __init subpage_mmio_ro_add_page(
+    mfn_t mfn, unsigned int offset_s, unsigned int offset_e)
+{
+    struct subpage_ro_range *entry = NULL, *iter;
+    unsigned int i;
+
+    list_for_each_entry(iter, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(iter->mfn, mfn) )
+        {
+            entry = iter;
+            break;
+        }
+    }
+    if ( !entry )
+    {
+        /* iter == NULL marks it was a newly allocated entry */
+        iter = NULL;
+        entry = xzalloc(struct subpage_ro_range);
+        if ( !entry )
+            return -ENOMEM;
+        entry->mfn = mfn;
+    }
+
+    for ( i = offset_s; i <= offset_e; i += MMIO_RO_SUBPAGE_GRAN )
+    {
+        bool oldbit = __test_and_set_bit(i / MMIO_RO_SUBPAGE_GRAN,
+                                        entry->ro_elems);
+        ASSERT(!oldbit);
+    }
+
+    if ( !iter )
+        list_add(&entry->list, &subpage_ro_ranges);
+
+    return iter ? 1 : 0;
+}
+
+/* This needs subpage_ro_lock already taken */
+static void __init subpage_mmio_ro_remove_page(
+    mfn_t mfn,
+    unsigned int offset_s,
+    unsigned int offset_e)
+{
+    struct subpage_ro_range *entry = NULL, *iter;
+    unsigned int i;
+
+    list_for_each_entry(iter, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(iter->mfn, mfn) )
+        {
+            entry = iter;
+            break;
+        }
+    }
+    if ( !entry )
+        return;
+
+    for ( i = offset_s; i <= offset_e; i += MMIO_RO_SUBPAGE_GRAN )
+        __clear_bit(i / MMIO_RO_SUBPAGE_GRAN, entry->ro_elems);
+
+    if ( !bitmap_empty(entry->ro_elems, PAGE_SIZE / MMIO_RO_SUBPAGE_GRAN) )
+        return;
+
+    list_del(&entry->list);
+    if ( entry->mapped )
+        iounmap(entry->mapped);
+    xfree(entry);
+}
+
+int __init subpage_mmio_ro_add(
+    paddr_t start,
+    size_t size)
+{
+    mfn_t mfn_start = maddr_to_mfn(start);
+    paddr_t end = start + size - 1;
+    mfn_t mfn_end = maddr_to_mfn(end);
+    unsigned int offset_end = 0;
+    int rc;
+    bool subpage_start, subpage_end;
+
+    ASSERT(IS_ALIGNED(start, MMIO_RO_SUBPAGE_GRAN));
+    ASSERT(IS_ALIGNED(size, MMIO_RO_SUBPAGE_GRAN));
+    if ( !IS_ALIGNED(size, MMIO_RO_SUBPAGE_GRAN) )
+        size = ROUNDUP(size, MMIO_RO_SUBPAGE_GRAN);
+
+    if ( !size )
+        return 0;
+
+    if ( mfn_eq(mfn_start, mfn_end) )
+    {
+        /* Both starting and ending parts handled at once */
+        subpage_start = PAGE_OFFSET(start) || PAGE_OFFSET(end) != PAGE_SIZE - 1;
+        subpage_end = false;
+    }
+    else
+    {
+        subpage_start = PAGE_OFFSET(start);
+        subpage_end = PAGE_OFFSET(end) != PAGE_SIZE - 1;
+    }
+
+    spin_lock(&subpage_ro_lock);
+
+    if ( subpage_start )
+    {
+        offset_end = mfn_eq(mfn_start, mfn_end) ?
+                     PAGE_OFFSET(end) :
+                     (PAGE_SIZE - 1);
+        rc = subpage_mmio_ro_add_page(mfn_start,
+                                      PAGE_OFFSET(start),
+                                      offset_end);
+        if ( rc < 0 )
+            goto err_unlock;
+        /* Check if not marking R/W part of a page intended to be fully R/O */
+        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
+                                                  mfn_x(mfn_start)));
+    }
+
+    if ( subpage_end )
+    {
+        rc = subpage_mmio_ro_add_page(mfn_end, 0, PAGE_OFFSET(end));
+        if ( rc < 0 )
+            goto err_unlock_remove;
+        /* Check if not marking R/W part of a page intended to be fully R/O */
+        ASSERT(rc || !rangeset_contains_singleton(mmio_ro_ranges,
+                                                  mfn_x(mfn_end)));
+    }
+
+    spin_unlock(&subpage_ro_lock);
+
+    rc = rangeset_add_range(mmio_ro_ranges, mfn_x(mfn_start), mfn_x(mfn_end));
+    if ( rc )
+        goto err_remove;
+
+    return 0;
+
+ err_remove:
+    spin_lock(&subpage_ro_lock);
+    if ( subpage_end )
+        subpage_mmio_ro_remove_page(mfn_end, 0, PAGE_OFFSET(end));
+ err_unlock_remove:
+    if ( subpage_start )
+        subpage_mmio_ro_remove_page(mfn_start, PAGE_OFFSET(start), offset_end);
+ err_unlock:
+    spin_unlock(&subpage_ro_lock);
+    return rc;
+}
+
+static void __iomem *subpage_mmio_map_page(
+    struct subpage_ro_range *entry)
+{
+    void __iomem *mapped_page;
+
+    if ( entry->mapped )
+        return entry->mapped;
+
+    mapped_page = ioremap(mfn_to_maddr(entry->mfn), PAGE_SIZE);
+
+    spin_lock(&subpage_ro_lock);
+    /* Re-check under the lock */
+    if ( entry->mapped )
+    {
+        spin_unlock(&subpage_ro_lock);
+        if ( mapped_page )
+            iounmap(mapped_page);
+        return entry->mapped;
+    }
+
+    entry->mapped = mapped_page;
+    spin_unlock(&subpage_ro_lock);
+    return entry->mapped;
+}
+
+static void subpage_mmio_write_emulate(
+    mfn_t mfn,
+    unsigned int offset,
+    const void *data,
+    unsigned int len)
+{
+    struct subpage_ro_range *entry;
+    volatile void __iomem *addr;
+
+    list_for_each_entry(entry, &subpage_ro_ranges, list)
+    {
+        if ( mfn_eq(entry->mfn, mfn) )
+        {
+            if ( test_bit(offset / MMIO_RO_SUBPAGE_GRAN, entry->ro_elems) )
+            {
+ write_ignored:
+                gprintk(XENLOG_WARNING,
+                        "ignoring write to R/O MMIO 0x%"PRI_mfn"%03x len %u\n",
+                        mfn_x(mfn), offset, len);
+                return;
+            }
+
+            addr = subpage_mmio_map_page(entry);
+            if ( !addr )
+            {
+                gprintk(XENLOG_ERR,
+                        "Failed to map page for MMIO write at 0x%"PRI_mfn"%03x\n",
+                        mfn_x(mfn), offset);
+                return;
+            }
+
+            switch ( len )
+            {
+            case 1:
+                writeb(*(const uint8_t*)data, addr);
+                break;
+            case 2:
+                writew(*(const uint16_t*)data, addr);
+                break;
+            case 4:
+                writel(*(const uint32_t*)data, addr);
+                break;
+            case 8:
+                writeq(*(const uint64_t*)data, addr);
+                break;
+            default:
+                /* mmio_ro_emulated_write() already validated the size */
+                ASSERT_UNREACHABLE();
+                goto write_ignored;
+            }
+            return;
+        }
+    }
+    /* Do not print message for pages without any writable parts. */
+}
+
+#ifdef CONFIG_HVM
+bool subpage_mmio_write_accept(mfn_t mfn, unsigned long gla)
+{
+    unsigned int offset = PAGE_OFFSET(gla);
+    const struct subpage_ro_range *entry;
+
+    list_for_each_entry(entry, &subpage_ro_ranges, list)
+        if ( mfn_eq(entry->mfn, mfn) &&
+             !test_bit(offset / MMIO_RO_SUBPAGE_GRAN, entry->ro_elems) )
+        {
+            /*
+             * We don't know the write size at this point yet, so it could be
+             * an unaligned write, but accept it here anyway and deal with it
+             * later.
+             */
+            return true;
+        }
+
+    return false;
+}
+#endif
+
 int cf_check mmio_ro_emulated_write(
     enum x86_segment seg,
     unsigned long offset,
@@ -4928,6 +5198,9 @@ int cf_check mmio_ro_emulated_write(
         return X86EMUL_UNHANDLEABLE;
     }
 
+    subpage_mmio_write_emulate(mmio_ro_ctxt->mfn, PAGE_OFFSET(offset),
+                               p_data, bytes);
+
     return X86EMUL_OKAY;
 }
 
diff --git a/xen/arch/x86/pv/ro-page-fault.c b/xen/arch/x86/pv/ro-page-fault.c
index cad28ef928ad..2ea1a6ad489c 100644
--- a/xen/arch/x86/pv/ro-page-fault.c
+++ b/xen/arch/x86/pv/ro-page-fault.c
@@ -333,8 +333,10 @@ static int mmio_ro_do_page_fault(struct x86_emulate_ctxt *ctxt,
     ctxt->data = &mmio_ro_ctxt;
     if ( pci_ro_mmcfg_decode(mfn_x(mfn), &mmio_ro_ctxt.seg, &mmio_ro_ctxt.bdf) )
         return x86_emulate(ctxt, &mmcfg_intercept_ops);
-    else
-        return x86_emulate(ctxt, &mmio_ro_emulate_ops);
+
+    mmio_ro_ctxt.mfn = mfn;
+
+    return x86_emulate(ctxt, &mmio_ro_emulate_ops);
 }
 
 int pv_ro_page_fault(unsigned long addr, struct cpu_user_regs *regs)
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Wed May 22 15:39:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 15:39:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727790.1132482 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o43-0001tq-Lp; Wed, 22 May 2024 15:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727790.1132482; Wed, 22 May 2024 15:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9o43-0001tj-Ii; Wed, 22 May 2024 15:39:39 +0000
Received: by outflank-mailman (input) for mailman id 727790;
 Wed, 22 May 2024 15:39:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKXr=MZ=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1s9o41-0001b1-OC
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 15:39:37 +0000
Received: from wfout5-smtp.messagingengine.com
 (wfout5-smtp.messagingengine.com [64.147.123.148])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7dd19db8-1851-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 17:39:36 +0200 (CEST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id 8D4FD1C0007C;
 Wed, 22 May 2024 11:39:34 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Wed, 22 May 2024 11:39:35 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 22 May 2024 11:39:32 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7dd19db8-1851-11ef-90a0-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1716392374; x=1716478774; bh=ODec38YfsT
	YTR4Z7j+bHVMoCfopdSqSbknuPJ8JX1zs=; b=R/oN/2foLIWImGj1tGXmwiHBcJ
	/Pv37jWMpvJhveqwVI97TFGbHPmrkPuwJzHHGnmAYuu3GKVAfdzPGmI8Iyf2Kt7I
	1kmdIcTLKJbI8vg0hQ/FiMEVXvEwSRgsDBMFDeZPoylbSSkxomxrv6cdWkkBOKKw
	8Hof+Ql0pFj29dqxILCtAWu2Nx1nQPOxeRFDKY1QlBN7QPJrU0jLwbjvdztsXEkk
	Vk/jXN62UvmWEW+NuKvYsMPajyf889zp4VhFn7cW9iCH1yWv0yretg+hT6j9Ll4e
	hDcnsWr6JlJdQOmtPfR/cuNd2NNYg3pfnWfN+Uwr9wT7RwsBD85lUvcST2hw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1716392374; x=
	1716478774; bh=ODec38YfsTYTR4Z7j+bHVMoCfopdSqSbknuPJ8JX1zs=; b=f
	duPTgPeZfPO66YAWnEsq4+cY7cBVOEe4Yjt2BjZLh2RUkmkZEigCNZmE0FexmCqk
	SJaiJsl1HyiROaCM5qv5oTBvSE3s/+eoRlfQcu8ZibSX7vBvNYPEN5b9eCWYxCt3
	Me9kO+OwCMgJp8gIPtgOpO0xboL0k0r3EqwJHFzG07BMadzFigWFS6OR//olZ6dF
	UU5pzSJmFFZ36X/QELveKv0wA22wtiLdC34lpfIwYOjqA2KTxWCWGBJl9lZlj9ap
	4VjzH4wZrAlm2sTbPN2AZDgbPJ/anyHqqQkwQbXltBxdP1D/EJNun917wUtt/jk+
	LbKXTJQ0qhp0BNPqiGzow==
X-ME-Sender: <xms:tRFOZllzAhVyKVLpsXZP5CjYCi4BkbboPc-vxqw8LdrZyTOyTREOlA>
    <xme:tRFOZg0vr_Endk2nWvSeVn1LTyhcRQc8qxL6Ne6EjLmNmN0oeDgyzfjZjAQ7M9Ssv
    RUlKvyawKet_w>
X-ME-Received: <xmr:tRFOZroOOem-oZ_IjHkCBclgrt3M3v9e0ghTkdVvaO8V1VKyrI6pHGbAB_Rl8zg0VvI81G3N8g3FZEhOc2_CDXeL6QHuIZGjwTZ02trdisi3t9EQlBg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeigedgvdegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:tRFOZlnN4bTrNy9_-JpdXerlQ66K8TLWH1yeLylECKgT_M1RkkvMpQ>
    <xmx:tRFOZj0p8ppcahqz-J_98q7z0RGUWzqmTiLdYsECr-yLg3xQ-PyuaQ>
    <xmx:tRFOZkv5SvWxmN0261HFOOupLO_Ol2OrK_j0zipdfrRvBPluQw1e6Q>
    <xmx:tRFOZnUYpxpNF8cxTZ43pFKpUUDPT4HH7RJ3_g-dFLv05PB9we0Klw>
    <xmx:thFOZtpgR6p7y0BDuhmU6oeilzRW5eMIf3mHvM_kZ9hQSoHt2R4xASRE>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v4 2/2] drivers/char: Use sub-page ro API to make just xhci dbc cap RO
Date: Wed, 22 May 2024 17:39:04 +0200
Message-ID: <83dfaaf5cbd056ffbe381cbd39f0230348257642.1716392340.git-series.marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
References: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Not the whole page, which may contain other registers too. The XHCI
specification describes DbC as designed to be controlled by a different
driver, but does not mandate placing registers on a separate page. In fact
on Tiger Lake and newer (at least), this page do contain other registers
that Linux tries to use. And with share=yes, a domU would use them too.
Without this patch, PV dom0 would fail to initialize the controller,
while HVM would be killed on EPT violation.

With `share=yes`, this patch gives domU more access to the emulator
(although a HVM with any emulated device already has plenty of it). This
configuration is already documented as unsafe with untrusted guests and
not security supported.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v4:
- restore mmio_ro_ranges in the fallback case
- set XHCI_SHARE_NONE in the fallback case
Changes in v3:
- indentation fix
- remove stale comment
- fallback to pci_ro_device() if subpage_mmio_ro_add() fails
- extend commit message
Changes in v2:
 - adjust for simplified subpage_mmio_ro_add() API
---
 xen/drivers/char/xhci-dbc.c | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 8e2037f1a5f7..c45e4b6825cc 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1216,20 +1216,28 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port)
         break;
     }
 #ifdef CONFIG_X86
-    /*
-     * This marks the whole page as R/O, which may include other registers
-     * unrelated to DbC. Xen needs only DbC area protected, but it seems
-     * Linux's XHCI driver (as of 5.18) works without writting to the whole
-     * page, so keep it simple.
-     */
-    if ( rangeset_add_range(mmio_ro_ranges,
-                PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
-                         uart->dbc.xhc_dbc_offset),
-                PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
-                       uart->dbc.xhc_dbc_offset +
-                sizeof(*uart->dbc.dbc_reg)) - 1) )
-        printk(XENLOG_INFO
-               "Error while adding MMIO range of device to mmio_ro_ranges\n");
+    if ( subpage_mmio_ro_add(
+             (uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+              uart->dbc.xhc_dbc_offset,
+             sizeof(*uart->dbc.dbc_reg)) )
+    {
+        printk(XENLOG_WARNING
+               "Error while marking MMIO range of XHCI console as R/O, "
+               "making the whole device R/O (share=no)\n");
+        uart->dbc.share = XHCI_SHARE_NONE;
+        if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
+            printk(XENLOG_WARNING
+                   "Failed to mark read-only %pp used for XHCI console\n",
+                   &uart->dbc.sbdf);
+        if ( rangeset_add_range(mmio_ro_ranges,
+                 PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+                          uart->dbc.xhc_dbc_offset),
+                 PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+                        uart->dbc.xhc_dbc_offset +
+                        sizeof(*uart->dbc.dbc_reg)) - 1) )
+            printk(XENLOG_INFO
+                   "Error while adding MMIO range of device to mmio_ro_ranges\n");
+    }
 #endif
 }
 
-- 
git-series 0.9.1


From xen-devel-bounces@lists.xenproject.org Wed May 22 16:22:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 16:22:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727816.1132499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9oiq-0002iE-RY; Wed, 22 May 2024 16:21:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727816.1132499; Wed, 22 May 2024 16:21:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9oiq-0002i7-Na; Wed, 22 May 2024 16:21:48 +0000
Received: by outflank-mailman (input) for mailman id 727816;
 Wed, 22 May 2024 16:21:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ieL=MZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1s9oip-0002i1-NS
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 16:21:47 +0000
Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com
 [2607:f8b0:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6113ac63-1857-11ef-b4bb-af5377834399;
 Wed, 22 May 2024 18:21:44 +0200 (CEST)
Received: by mail-ot1-x32e.google.com with SMTP id
 46e09a7af769-6f1016f41e2so2476220a34.1
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 09:21:45 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a8d595809esm42940246d6.56.2024.05.22.09.21.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 May 2024 09:21:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6113ac63-1857-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716394904; x=1716999704; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=N4+qUAE3RinRnCNLSdqj+tsXo//3UZJt8DNMpzP2Ibs=;
        b=PCGafyMTVgBpdvwV7QAX0uS/pEMWyhfzZXQ2Vzj25HLL3XP335AQQUISRIwfdSlewQ
         JTU/fw9zSjSb5T604hrngDlT/qUn1izZ5CZiQNWOGDIbRN5hUkvNyzvGdaMDSBpkVS3Q
         8m2woGCS4pfIMWLOCxQjAU5C3x+niq8muyKjk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716394904; x=1716999704;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=N4+qUAE3RinRnCNLSdqj+tsXo//3UZJt8DNMpzP2Ibs=;
        b=BsOFI2HoimmtZA+fK2xfvJCMrtm7a5ic9kkHcXvxE6ARlSfQWlDQfmryt6u50zfkbp
         h+hCcj8F8O9bIIXbuDmo/jhAeu7qcc5J/bmEcYPKn9hUaICcIeEfbPLpI8K5mpf+Ogbd
         f+OQMuvGczyeQQW3LZVNuqVneBlbTLH8iqFpFpo1/ys4fdsXHWGl/2qeCk9mc8uly38g
         cxlQRU9cL0kFtFd4Opjm8gso76jgi761Nndtm9U+puqcpcDRT7KFZbHkSHXXMz6MiI8U
         z9rVwr9lcbOFSug9bzEbgKu/GKhagDQjeCSndSMR97K9Adp99uJ4AbWP8LnCTcK+NtKc
         ERBg==
X-Forwarded-Encrypted: i=1; AJvYcCUR9uBk/cm7ETSrlQMWZuivy3S8uCE+lq3BjTTkbpz2lXU1K4ULhAiJCfXza8ReCaWdnlsysCMaf+fb0G6kDc4osFmK6ke/3CFXl7clgaY=
X-Gm-Message-State: AOJu0Yyxkl68JXCMHmSKzIhj0FZeO3cE+ut6kHqW0/lrFN6NYywBjpsI
	8Y1eiGDB8icYcj3c4wxLM8M3G2veZ4PnAWeqH/LTTLtSi2/PBzqPpqG77lFE3Uo=
X-Google-Smtp-Source: AGHT+IEkxH0PYLn/hDKBcNbrdXS8OyxHsQk7PQAVHZ1VuY6DaMH0yE7e+t1B597bI2RYqSkzIaY6BA==
X-Received: by 2002:a05:6358:a59:b0:192:47fd:8bf9 with SMTP id e5c5f4694b2df-19791cce683mr224927455d.3.1716394903628;
        Wed, 22 May 2024 09:21:43 -0700 (PDT)
Date: Wed, 22 May 2024 18:21:41 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Message-ID: <Zk4blZ9z6cn-ORrv@macbook>
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
 <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com>
 <Zk3wMevPye8Rfjm4@macbook>
 <5484aab7-b15d-46c5-9ff1-9a5b54640acc@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5484aab7-b15d-46c5-9ff1-9a5b54640acc@suse.com>

On Wed, May 22, 2024 at 03:34:29PM +0200, Jan Beulich wrote:
> On 22.05.2024 15:16, Roger Pau Monné wrote:
> > On Tue, May 21, 2024 at 12:30:32PM +0200, Jan Beulich wrote:
> >> On 17.05.2024 15:33, Roger Pau Monne wrote:
> >>> Enabling it using an HVM param is fragile, and complicates the logic when
> >>> deciding whether options that interact with altp2m can also be enabled.
> >>>
> >>> Leave the HVM param value for consumption by the guest, but prevent it from
> >>> being set.  Enabling is now done using and additional altp2m specific field in
> >>> xen_domctl_createdomain.
> >>>
> >>> Note that albeit only currently implemented in x86, altp2m could be implemented
> >>> in other architectures, hence why the field is added to xen_domctl_createdomain
> >>> instead of xen_arch_domainconfig.
> >>>
> >>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >>
> >> Reviewed-by: Jan Beulich <jbeulich@suse.com> # hypervisor
> >> albeit with one question:
> >>
> >>> --- a/xen/arch/x86/domain.c
> >>> +++ b/xen/arch/x86/domain.c
> >>> @@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> >>>      bool hap = config->flags & XEN_DOMCTL_CDF_hap;
> >>>      bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
> >>>      unsigned int max_vcpus;
> >>> +    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
> >>> +                                         XEN_DOMCTL_ALTP2M_mode_mask);
> >>>  
> >>>      if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
> >>>      {
> >>> @@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> >>>          return -EINVAL;
> >>>      }
> >>>  
> >>> +    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
> >>> +    {
> >>> +        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
> >>> +                config->flags);
> >>> +        return -EINVAL;
> >>> +    }
> >>> +
> >>> +    if ( altp2m_mode && nested_virt )
> >>> +    {
> >>> +        dprintk(XENLOG_INFO,
> >>> +                "Nested virt and altp2m are not supported together\n");
> >>> +        return -EINVAL;
> >>> +    }
> >>> +
> >>> +    if ( altp2m_mode && !hap )
> >>> +    {
> >>> +        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
> >>> +        return -EINVAL;
> >>> +    }
> >>
> >> Should this last one perhaps be further extended to permit altp2m with EPT
> >> only?
> > 
> > Hm, yes, that would be more accurate as:
> > 
> > if ( altp2m_mode && (!hap || !hvm_altp2m_supported()) )
> 
> Wouldn't
> 
>    if ( altp2m_mode && !hvm_altp2m_supported() )
> 
> suffice? hvm_funcs.caps.altp2m is not supposed to be set when no HAP,
> as long as HAP continues to be a pre-condition?

No, `hap` here signals whether the domain is using HAP, and we need to
take this int account, otherwise we would allow enabling altp2m for
domains using shadow.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 22 18:05:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 18:05:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727831.1132508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9qKz-0006hO-SW; Wed, 22 May 2024 18:05:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727831.1132508; Wed, 22 May 2024 18:05:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9qKz-0006hH-Ov; Wed, 22 May 2024 18:05:17 +0000
Received: by outflank-mailman (input) for mailman id 727831;
 Wed, 22 May 2024 18:05:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MVRZ=MZ=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1s9qKy-0006hB-Iy
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 18:05:16 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d6468b06-1865-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 20:05:14 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 484994EE0738;
 Wed, 22 May 2024 20:05:13 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6468b06-1865-11ef-90a0-e314d9c70b13
MIME-Version: 1.0
Date: Wed, 22 May 2024 20:05:13 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>,
 xen-devel@lists.xenproject.org, consulting@bugseng.com, Stefano Stabellini
 <sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu
 <ppircalabu@bitdefender.com>
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of
 mem_access.c
In-Reply-To: <d659e73f-762a-48c9-8936-94bbc771218c@xen.org>
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
 <d659e73f-762a-48c9-8936-94bbc771218c@xen.org>
Message-ID: <6b7fee28e8f54799fe8ec7fdc6d4001d@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-10 22:59, Julien Grall wrote:
> Hi,

Hi,

> 
> On 10/05/2024 13:32, Alessandro Zucchelli wrote:
>> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
>> changes are done:
>> revert preprocessor conditional changes to xen/mem_access.h which
>> had it build unconditionally, add conditional build for 
>> xen/mem_access.c
> 
> I am afraid, I don't understand this one as you don't seem to modify 
> xen/mem_access.h. Is this meant to be part of the changelog?
> 
> You also don't seem to mention the change in Makefile. This is the one 
> I was asking for in the previous version. So what about:
> 
> "xen/arm: mem_access: Conditionally compile mem_access.c
> 
> Commit 634cfc8beb ("Make MEM_ACCESS configurable") intended to make 
> MEM_ACCESS configurable on Arm to reduce the code size when the user 
> doesn't need it.
> 
> However, this didn't cover the arch specific code. None of the code in 
> arm/mem_access.c is necessary when MEM_ACCESS=n, so it can be compiled 
> out. This will require to provide some stub for functions called by the 
> common code.
> 
> This is also fixing violation of the MISRA C:2012 Rule 8.4 reported by 
> ECLAIR.
> "
> 
> The patch itself loks good so once we agree on the commit message, then 
> I am happy to update it on commit.
> 
> Cheers,

since Julien is ok with the patch, with the commit message he proposed, 
I think this needs an R-by or an A-by in order to commit for 4.19.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed May 22 18:28:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 18:28:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727844.1132524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9qhL-0000tP-NJ; Wed, 22 May 2024 18:28:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727844.1132524; Wed, 22 May 2024 18:28:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9qhL-0000tI-KP; Wed, 22 May 2024 18:28:23 +0000
Received: by outflank-mailman (input) for mailman id 727844;
 Wed, 22 May 2024 18:28:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Goh5=MZ=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1s9qhJ-0000tC-Mw
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 18:28:22 +0000
Received: from sender4-op-o15.zoho.com (sender4-op-o15.zoho.com
 [136.143.188.15]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0f5fe6df-1869-11ef-90a0-e314d9c70b13;
 Wed, 22 May 2024 20:28:19 +0200 (CEST)
Received: by mx.zohomail.com with SMTPS id 1716402495482663.7254359392649;
 Wed, 22 May 2024 11:28:15 -0700 (PDT)
Received: by mail-yb1-f169.google.com with SMTP id
 3f1490d57ef6-df4d60c59f7so1292030276.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 11:28:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f5fe6df-1869-11ef-90a0-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; t=1716402497; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=YcC+xRHHlYqCrwVwr0HaZD0jjMa3VfN7cJtgIRNIBZyZAu5vow3sM9lULxBN12iMZy2B/gi7Bum+nwv2ssJDAvvq17P6/fgblTD3mpjafLFYTHAtwL/YarZNIhOI+ro2WVT5I8RLIwZ9vpn7ZWSDiT7Df9k7pZ3i1V6qMtl8fHs=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1716402497; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=A9vs60JC2BOykRHL9q/qT34y99SvAiU/CBGf1fQR2ug=; 
	b=dqxPthekxSnHIHnhlK4W68YDnRs8TE3lGd4GIxsMK7juMFseLY/kgVWM/2lCfefRwyMDc8pKNMUtxA1rwaAubY5QV9SQrPOSS9F1nyKoeGTt4Vw3QiOsmZ24f4TdVW0pf5FUxRqKUybMlgg/cPymKzBKoAzdS5HO0/4Ukh5JU2Y=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1716402497;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=A9vs60JC2BOykRHL9q/qT34y99SvAiU/CBGf1fQR2ug=;
	b=jDCzhutMZYFn3/cgf8eoKpKU3r2Lc6pLrtlxWzBbHTtLHQQKWAHGNH3f8bGxAJo5
	wzzxo1B/iPvh07iyAgQ+cHzcjAzCfNTeowcBugUq2AL6T0rIpZKSxtWkHPIhgd1KL/M
	qh0/k8RWH7hsmaJYtflOG66z1Xh/PEiPkhhYNyHM=
X-Gm-Message-State: AOJu0Yz/TGZKRPEZLSqnfNAW0T6yUsKlgTS7Z+kZAJ+HvnNfwmLgdFf2
	tm4s+g1SrBEY8qL73hZP7/tSYGnGmjNKOZZYz+9xeC5xPl47MN/DC7qEXkq30KnRSDL8CSMMWMF
	03+fegdwjBMvdMdacWKl9BIbRr2M=
X-Google-Smtp-Source: AGHT+IHJtee1Lbl8B6OKTqccYOCN6SKccUMGVWjaMkztUTHljNjvTA2a68eXgAkpxANicbrjwZTYIYzMjrGsaq3hq9A=
X-Received: by 2002:a25:d60a:0:b0:df4:ebd2:661c with SMTP id
 3f1490d57ef6-df4ebd26784mr1561597276.63.1716402494390; Wed, 22 May 2024
 11:28:14 -0700 (PDT)
MIME-Version: 1.0
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
In-Reply-To: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Wed, 22 May 2024 14:27:38 -0400
X-Gmail-Original-Message-ID: <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
Message-ID: <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
Subject: Re: [XEN PATCH v3] arm/mem_access: add conditional build of mem_access.c
To: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 10, 2024 at 8:32=E2=80=AFAM Alessandro Zucchelli
<alessandro.zucchelli@bugseng.com> wrote:
>
> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
> changes are done:
> revert preprocessor conditional changes to xen/mem_access.h which
> had it build unconditionally, add conditional build for xen/mem_access.c
> as well and provide stubs in asm/mem_access.h for the users of this
> header.
>
> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>

Acked-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Wed May 22 20:27:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 20:27:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727866.1132540 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9sY1-0005md-U4; Wed, 22 May 2024 20:26:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727866.1132540; Wed, 22 May 2024 20:26:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9sY1-0005mW-Qd; Wed, 22 May 2024 20:26:53 +0000
Received: by outflank-mailman (input) for mailman id 727866;
 Wed, 22 May 2024 20:26:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9sXz-0005mM-UX; Wed, 22 May 2024 20:26:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9sXz-0004ww-Qm; Wed, 22 May 2024 20:26:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9sXz-0001U7-FU; Wed, 22 May 2024 20:26:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9sXz-0005hs-F0; Wed, 22 May 2024 20:26:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ZMEdNzjLkme+pKQlR77WSvBdU4CirYRDKfsDvOuotFk=; b=24pyoPTEi5kNwVIugqU7axS0Gw
	dCmemV6KvEC0Q529kouymXuj5ZMgeAa5uBGHBfVraNV9cUp/bvWcPpf6DZc+F2ZRwui7+2Ssp8Dnt
	7wZ7vW5GRx6gOpCUaj4CN99J4uD7chyqvjFEIJbCVZVCPPZhQKzRE3WBfoo6UdxSrBXs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186067-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.18-testing test] 186067: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:host-ping-check-xen:fail:heisenbug
    xen-4.18-testing:test-armhf-armhf-xl-rtds:host-ping-check-xen:fail:allowable
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=01f7a3c792241d348a4e454a30afdf6c0d6cd71c
X-Osstest-Versions-That:
    xen=7cdb1fa2ab0b5e11f66cada0370770404153c824
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 May 2024 20:26:51 +0000

flight 186067 xen-4.18-testing real [real]
flight 186083 xen-4.18-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186067/
http://logs.test-lab.xenproject.org/osstest/logs/186083/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu 10 host-ping-check-xen fail pass in 186083-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     10 host-ping-check-xen      fail REGR. vs. 186060

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 186083 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 186083 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186060
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186060
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186060
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186060
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186060
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186060
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  01f7a3c792241d348a4e454a30afdf6c0d6cd71c
baseline version:
 xen                  7cdb1fa2ab0b5e11f66cada0370770404153c824

Last test of basis   186060  2024-05-21 08:38:31 Z    1 days
Testing same since   186067  2024-05-21 23:40:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   7cdb1fa2ab..01f7a3c792  01f7a3c792241d348a4e454a30afdf6c0d6cd71c -> stable-4.18


From xen-devel-bounces@lists.xenproject.org Wed May 22 20:48:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 20:48:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727876.1132554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ssO-0008SW-Nn; Wed, 22 May 2024 20:47:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727876.1132554; Wed, 22 May 2024 20:47:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9ssO-0008SP-Ku; Wed, 22 May 2024 20:47:56 +0000
Received: by outflank-mailman (input) for mailman id 727876;
 Wed, 22 May 2024 20:47:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9ssO-0008SJ-1p
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 20:47:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9ssN-0005H8-Ou; Wed, 22 May 2024 20:47:55 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9ssN-00057a-Hm; Wed, 22 May 2024 20:47:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=BdR/yNSjH2i6t85xzLFAfqntVHQKlvGxk2BDo3UY0rc=; b=oejDbQYmJ0SXcu4nmGtTyX4CVp
	yRlUCrMg6oEYshVTs/EFkk9ON5LGfcaKoVrJSQEk4rGnkSZB7dBr+FtmhZK/dfcwCU/CN/3/yUCi9
	waMGjUcjSf057MvTWooGAkVTj6ZrvNlbykWXum4PVMUK/IgaK3ap3TFF1rKnTIByBjdg=;
Message-ID: <774cdc8a-d815-4330-a17a-558e3c0a49b5@xen.org>
Date: Wed, 22 May 2024 21:47:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/7] xen/p2m: put reference for level 2 superpage
Content-Language: en-GB
To: Luca Fancellu <Luca.Fancellu@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240522075151.3373899-1-luca.fancellu@arm.com>
 <20240522075151.3373899-4-luca.fancellu@arm.com>
 <c10b2513-b0f1-4042-a538-2115e92408d1@xen.org>
 <CD6149B6-EC9C-4313-AA21-F0F0156A9E1C@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CD6149B6-EC9C-4313-AA21-F0F0156A9E1C@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 22/05/2024 14:47, Luca Fancellu wrote:
> Hi Julien,

Hi Luca,

>> On 22 May 2024, at 14:25, Julien Grall <julien@xen.org> wrote:
>>
>>> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
>>> index 41fcca011cf4..b496266deef6 100644
>>> --- a/xen/arch/arm/mmu/p2m.c
>>> +++ b/xen/arch/arm/mmu/p2m.c
>>> @@ -753,17 +753,9 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>>>       return rc;
>>>   }
>>>   -/*
>>> - * Put any references on the single 4K page referenced by pte.
>>> - * TODO: Handle superpages, for now we only take special references for leaf
>>> - * pages (specifically foreign ones, which can't be super mapped today).
>>> - */
>>> -static void p2m_put_l3_page(const lpae_t pte)
>>> +/* Put any references on the single 4K page referenced by mfn. */
>>> +static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
>>>   {
>>> -    mfn_t mfn = lpae_get_mfn(pte);
>>> -
>>> -    ASSERT(p2m_is_valid(pte));
>>> -
>>>       /*
>>>        * TODO: Handle other p2m types
>>>        *
>>> @@ -771,16 +763,43 @@ static void p2m_put_l3_page(const lpae_t pte)
>>>        * flush the TLBs if the page is reallocated before the end of
>>>        * this loop.
>>>        */
>>> -    if ( p2m_is_foreign(pte.p2m.type) )
>>> +    if ( p2m_is_foreign(type) )
>>>       {
>>>           ASSERT(mfn_valid(mfn));
>>>           put_page(mfn_to_page(mfn));
>>>       }
>>>       /* Detect the xenheap page and mark the stored GFN as invalid. */
>>> -    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>>> +    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
>>>           page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
>>>   }
>>
>> All the pages within a 2MB mapping should be the same type. So...
>>
>>>   +/* Put any references on the superpage referenced by mfn. */
>>> +static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
>>> +{
>>> +    unsigned int i;
>>> +
>>> +    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
>>> +    {
>>> +        p2m_put_l3_page(mfn, type);
>>> +
>>> +        mfn = mfn_add(mfn, 1);
>>> +    }
>>
>> ... this solution is a bit wasteful as we will now call p2m_put_l3_page() 512 times even though there is nothing to do.
>>
>> So instead can we move the checks outside to optimize the path a bit?
> 
> You mean this?
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index b496266deef6..d40cddda48f3 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -794,7 +794,8 @@ static void p2m_put_page(const lpae_t pte, unsigned int level)
>       ASSERT(p2m_is_valid(pte));
>   
>       /* We have a second level 2M superpage */
> -    if ( p2m_is_superpage(pte, level) && (level == 2) )
> +    if ( p2m_is_superpage(pte, level) && (level == 2) &&
> +         p2m_is_foreign(pte.p2m.type) )
>           return p2m_put_l2_superpage(mfn, pte.p2m.type);
>       else if ( level == 3 )
>           return p2m_put_l3_page(mfn, pte.p2m.type);

I meant something like below. This is untested and to apply on top of 
this patch:

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index b496266deef6..60c4d680b417 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -753,20 +753,27 @@ static int p2m_mem_access_radix_set(struct 
p2m_domain *p2m, gfn_t gfn,
      return rc;
  }

+static void p2m_put_foreign_page(struct page_info *pg)
+{
+    /*
+     * It's safe to do the put_page here because page_alloc will
+     * flush the TLBs if the page is reallocated before the end of
+     * this loop.
+     */
+    put_page(pg)
+}
+
  /* Put any references on the single 4K page referenced by mfn. */
  static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
  {
      /*
       * TODO: Handle other p2m types
       *
-     * It's safe to do the put_page here because page_alloc will
-     * flush the TLBs if the page is reallocated before the end of
-     * this loop.
       */
      if ( p2m_is_foreign(type) )
      {
          ASSERT(mfn_valid(mfn));
-        put_page(mfn_to_page(mfn));
+        p2m_put_foreign_page(mfn_to_page(mfn));
      }
      /* Detect the xenheap page and mark the stored GFN as invalid. */
      else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
@@ -777,13 +784,18 @@ static void p2m_put_l3_page(mfn_t mfn, p2m_type_t 
type)
  static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
  {
      unsigned int i;
+    struct page_info *pg;

-    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
-    {
-        p2m_put_l3_page(mfn, type);
+    /* TODO: Handle other p2m types */
+    if ( p2m_is_foreign(type) )
+        return;

-        mfn = mfn_add(mfn, 1);
-    }
+    ASSERT(mfn_valid(mfn));
+
+    pg = mfn_to_page(mfn);
+
+    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++, pg++ )
+        p2m_put_foreign_page(pg);
  }

  /* Put any references on the page referenced by pte. */

The type check only happens once. Also, I moved mfn_to_page(...) outside 
of the loop because the operation is expensive. Yet, if the MFNs are 
contiguous, then the page_info structures will be too.
	
> 
> 
>> Otherwise...
>>
>>> +}
>>> +
>>> +/* Put any references on the page referenced by pte. */
>>> +static void p2m_put_page(const lpae_t pte, unsigned int level)
>>> +{
>>> +    mfn_t mfn = lpae_get_mfn(pte);
>>> +
>>> +    ASSERT(p2m_is_valid(pte));
>>> +
>>> +    /* We have a second level 2M superpage */
>>> +    if ( p2m_is_superpage(pte, level) && (level == 2) )
>>> +        return p2m_put_l2_superpage(mfn, pte.p2m.type);
>>> +    else if ( level == 3 )
>>> +        return p2m_put_l3_page(mfn, pte.p2m.type);
>>> +}
>>> +
>>>   /* Free lpae sub-tree behind an entry */
>>>   static void p2m_free_entry(struct p2m_domain *p2m,
>>>                              lpae_t entry, unsigned int level)
>>> @@ -809,9 +828,16 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>>>   #endif
>>>             p2m->stats.mappings[level]--;
>>> -        /* Nothing to do if the entry is a super-page. */
>>> -        if ( level == 3 )
>>> -            p2m_put_l3_page(entry);
>>> +        /*
>>> +         * TODO: Currently we don't handle 1GB super-page, Xen is not
>>> +         * preemptible and therefore some work is needed to handle such
>>> +         * superpages, for which at some point Xen might end up freeing memory
>>> +         * and therefore for such a big mapping it could end up in a very long
>>> +         * operation.
>>> +         */
>>> +        if ( level >= 2 )
>>> +            p2m_put_page(entry, level);
>>> +
>>>           return;
>>>       }
>>>   @@ -1558,9 +1584,12 @@ int relinquish_p2m_mapping(struct domain *d)
>>>             count++;
>>>           /*
>>> -         * Arbitrarily preempt every 512 iterations.
>>> +         * Arbitrarily preempt every 512 iterations or when type is foreign
>>> +         * mapping and the order is above 9 (2MB).
>>>            */
>>> -        if ( !(count % 512) && hypercall_preempt_check() )
>>> +        if ( (!(count % 512) ||
>>> +              (p2m_is_foreign(t) && (order > XEN_PT_LEVEL_ORDER(2)))) &&
>>
>> ... we would need to preempt for every 2MB rather than just for the p2m_is_foreign().
> 
> Ok otherwise you are suggesting that if we don’t go for the solution above we drop p2m_is_foreign(t) from
> the condition here, am I right?

That's correct.

> 
>>
>> BTW, p2m_put_l3_page() has also another case. Should we consider to handle preemption for it too?
> 
> You mean checking for 512 iterations, or foreign mapping when order is > 9, or
> p2m_is_ram(type) && is_xen_heap_mfn(mfn) ?

Looking at your proposal, your intent is to only handle foreign mapping 
for superpage. Is that right? If so, I think "(p2m_is_foreign(t) && 
(order > XEN_PT_LEVEL_ORDER(2))))" is ok. But I would suggest to 
document in p2m_put_l2_superpage() that any change in handling would 
also require to update the relinquish code.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 22 20:50:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 20:50:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727880.1132563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9svB-0001Xl-4E; Wed, 22 May 2024 20:50:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727880.1132563; Wed, 22 May 2024 20:50:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9svB-0001Xe-0x; Wed, 22 May 2024 20:50:49 +0000
Received: by outflank-mailman (input) for mailman id 727880;
 Wed, 22 May 2024 20:50:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1s9svA-0001XY-5w
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 20:50:48 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9sv9-0005Kl-BX; Wed, 22 May 2024 20:50:47 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1s9sv9-0005CC-5L; Wed, 22 May 2024 20:50:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=1b2wjNCaQ2OsGn5kNNZLj+IsWpe90RhjcKkH3BspCCc=; b=Gg3r/7UG5gdJr8ruNZOGmYwWTF
	Y7s+6bUpHmhF4UubPOn/FDsTkLDz+nS+Mn5e5c0ppHsdWMeJiv9mYVZhlmnfcrOCJCBRnlP2THSS+
	2nXCvOQFmVVItW+kuOB/vV0qCxm9PH+RMgibjxvhpQEobhLKcpedLaNWT+LELyZs6jpQ=;
Message-ID: <fb426cc8-fda9-4e21-983d-37649499e0e4@xen.org>
Date: Wed, 22 May 2024 21:50:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [for-4.19] Re: [XEN PATCH v3] arm/mem_access: add conditional build
 of mem_access.c
Content-Language: en-GB
To: Tamas K Lengyel <tamas@tklengyel.com>,
 Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
 <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

Adding Oleksii as the release manager.

On 22/05/2024 19:27, Tamas K Lengyel wrote:
> On Fri, May 10, 2024 at 8:32 AM Alessandro Zucchelli
> <alessandro.zucchelli@bugseng.com> wrote:
>>
>> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
>> changes are done:
>> revert preprocessor conditional changes to xen/mem_access.h which
>> had it build unconditionally, add conditional build for xen/mem_access.c
>> as well and provide stubs in asm/mem_access.h for the users of this
>> header.
>>
>> Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
> 
> Acked-by: Tamas K Lengyel <tamas@tklengyel.com>

Oleksii, would you be happy if this patch is committed for 4.19?

BTW, do you want to be release-ack every bug until the hard code freeze? 
Or would you be fine to levea the decision to the maintainers?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 22 22:30:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 22:30:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727896.1132579 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uTc-0004Tb-37; Wed, 22 May 2024 22:30:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727896.1132579; Wed, 22 May 2024 22:30:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uTc-0004TU-0h; Wed, 22 May 2024 22:30:28 +0000
Received: by outflank-mailman (input) for mailman id 727896;
 Wed, 22 May 2024 22:30:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujjH=MZ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9uTb-0004TN-CK
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 22:30:27 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0dea8a1-188a-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 00:30:25 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 13991CE1318;
 Wed, 22 May 2024 22:30:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 689BEC2BBFC;
 Wed, 22 May 2024 22:30:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0dea8a1-188a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716417019;
	bh=6AmxGm/usXVZKBFgyu9AglYwlVCoutuuiYzVJmO1rWI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rdf3Yl4q0T69wr97dTFxwdnwqUe+ogXPP4SP/M1Zm+SFBuzjcTfGByD7wdEY4Mn5I
	 wZOrriCcKC8eirE/4Ezmu3sS5aPsqzQ0HwehVSi2rjBK+mQtJ6NNm38mRVwarqSCTT
	 /EHn4oVbCxuXBs/ej+4LER6SO31LfHAMZk7anW5K1axFjc1qvHc2HG9mY+J1V0y9g9
	 SmQq7FVbt986VgM3+BTW6djml+RVkR+t0Rd1jB1g2mfJNkK8A1hIipLTW/jQqGq27m
	 WGcDkLIQjpxLr5pPKDY8hmjEXq/ZBCsp1UqQpnivB1bZTBqKbT/WCSYAj7UyLJL2VG
	 B5TghjTxiH9fA==
Date: Wed, 22 May 2024 15:30:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH v4 2/3] x86/MCE: add default switch case in
 init_nonfatal_mce_checker()
In-Reply-To: <9ec888c37a261c86a52f360146cbbb3ad437ad30.1716366581.git.Sergiy_Kibrik@epam.com>
Message-ID: <alpine.DEB.2.22.394.2405221529390.1052252@ubuntu-linux-20-04-desktop>
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com> <9ec888c37a261c86a52f360146cbbb3ad437ad30.1716366581.git.Sergiy_Kibrik@epam.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 May 2024, Sergiy Kibrik wrote:
> The default switch case block is wanted here, to handle situation
> e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, but
> misleading message still gets logged anyway.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> changes in v4:
>  - return 0 instead of -ENODEV and put a comment
>  - update description a bit
> ---
>  xen/arch/x86/cpu/mcheck/non-fatal.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c
> index 33cacd15c2..5a53bcd0b7 100644
> --- a/xen/arch/x86/cpu/mcheck/non-fatal.c
> +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
> @@ -29,9 +29,14 @@ static int __init cf_check init_nonfatal_mce_checker(void)
>  		/* Assume we are on K8 or newer AMD or Hygon CPU here */
>  		amd_nonfatal_mcheck_init(c);
>  		break;
> +
>  	case X86_VENDOR_INTEL:
>  		intel_nonfatal_mcheck_init(c);
>  		break;
> +
> +	default:
> +		/* unhandled vendor isn't really an error */
> +		return 0;
>  	}
>  	printk(KERN_INFO "mcheck_poll: Machine check polling timer started.\n");
>  	return 0;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed May 22 22:59:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 22:59:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727903.1132590 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uvq-0007HW-91; Wed, 22 May 2024 22:59:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727903.1132590; Wed, 22 May 2024 22:59:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uvq-0007HP-5q; Wed, 22 May 2024 22:59:38 +0000
Received: by outflank-mailman (input) for mailman id 727903;
 Wed, 22 May 2024 22:59:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uvp-0007HJ-At
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 22:59:37 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f475f494-188e-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 00:59:35 +0200 (CEST)
Received: from MN2PR01CA0060.prod.exchangelabs.com (2603:10b6:208:23f::29) by
 SJ1PR12MB6265.namprd12.prod.outlook.com (2603:10b6:a03:458::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20; Wed, 22 May
 2024 22:59:31 +0000
Received: from BL6PEPF0001AB53.namprd02.prod.outlook.com
 (2603:10b6:208:23f:cafe::a8) by MN2PR01CA0060.outlook.office365.com
 (2603:10b6:208:23f::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Wed, 22 May 2024 22:59:30 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB53.mail.protection.outlook.com (10.167.241.5) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 22:59:30 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 17:59:30 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 17:59:29 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 17:59:28 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f475f494-188e-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kDQVcd/ZyW2nzp4S214UvjpaiC8CuSdExpQblx7tVbT8odBLTxxwLobBYbo2EtjjQdgVhQ103CQIxVTAc3G7USnvvG0Vj6cK/25LqBAGSxebACCnEBhC2MoShrWyqWEXVlrnhBecsABTtt2+Y4ziMnth09/LoPKVHCXvo1iDNSUgchmBOU2qJxcNsdyPkmRa5KaxRe8V6W6CU4rNAULfpEOnTQohiOUXBhUi8TDQ7Ii1QcUljMhHhWLkkM2PloNFDpQevEK1owDY0kCUeBgGU95VdgsWbbxtUi5I/GhGSkU+uPz58eXmML67mm+L2lWc1TSeKXEtG8HVqRlQwexJ0g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zTMLUqBpPTGO1t5D5YY5Wm1WqI3sQqiLCW/DYasj35w=;
 b=OZEZxrrg9LKrtnbG6BeGzcbPcQrZNdWnHpo3CgS6TAdSl7uLZ0uTEfwhz5z9RtSeGZFJWtL6JOJP5sxaD4eOkd9K6rYZ9FJHApp9yAhO8ybU5VDNOA8ab5YfgxM/S71/PDvgAX9KfKExTTqMwPUZzF21PUJ0BJ1+uxqVTrwqQ06rsoZPctv40piY0u4yMgmOVw+0b7BeQ0aeMwimZO8Oi8jVytsBQZ2bisv1jFFHuRpStSfB0JlGuDxu1Ws0ACww4Cv7lZxdr1xDRbE8bc1w57wBp+PFkogarVgN8DtqeHZIIm7lBaiDHcl0j9hXrHTyD8ccR0inRHSmnHizW4lWyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zTMLUqBpPTGO1t5D5YY5Wm1WqI3sQqiLCW/DYasj35w=;
 b=b5dPT2VsiUTDKuP3JTZmOt0DigkafYWXDNlZSn/fvRKLbBMrRHTPSIuSpNBM80OpY0AkI+vIuQwRzttCz5pN34yJdpNo1S5wogp3YAFgVR+Mdtde8E8MSWd/5VWdBm5mGeSWIC4t9AJh4fgBt2f1WSYjE16Bu1bp9I0LSvcY+mg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stewart Hildebrand <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v16 0/5] PCI devices passthrough on Arm, part 3
Date: Wed, 22 May 2024 18:59:19 -0400
Message-ID: <20240522225927.77398-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB53:EE_|SJ1PR12MB6265:EE_
X-MS-Office365-Filtering-Correlation-Id: 686bc822-8494-40b5-f345-08dc7ab2d6be
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?SXJneXF6SmxEaG03OW1qMEdWM05wRS9OSkVTQm1NT051Z2FsZHl5NmwrZ1Az?=
 =?utf-8?B?a2sxODd5cE14VU9XU0IyRFpjS1NQYTZzd24vSHYxN2g3TThRNW84aUZ3NGZ0?=
 =?utf-8?B?SHZhU0o4dnNwN2pTZVJGRSs0WDBmQ281cHVXY2IyNFg2R1cvN3ZsRy9WK1Rt?=
 =?utf-8?B?M2FhUzFNYnArTm83cVU0SisvQ1k0YU1EWjJJZGgzZ2txRFNpRWRhNnl1bklH?=
 =?utf-8?B?TnB0aXU5UFd3Q0RENGZSdVNkMm9MWFVxR2FtckVBcFhteGZBeXE5dmx2RkdK?=
 =?utf-8?B?ZythUExCc1hjRmQ4dzE3cG9SWmdoYTBZd21RMkFlL3dERG1lcG9ndTFjZi9t?=
 =?utf-8?B?VURjMVN0VVFCNC83bjhYMTN3WlNVYWdEK3daRGlLU1hjQ2FCL04rcysremFT?=
 =?utf-8?B?NGtXZ21XQXBPemx3dG5qdXdXdzlwRytYa2xURnd1L2lJV1hFMjdrSXloZzNN?=
 =?utf-8?B?bUhBSnY2bTRoZjhyOHRkeitkTENiRHZacmtSY0FQQjIzYnN2d3lJY2pMdGQy?=
 =?utf-8?B?RVFsMXE1Y0U1SHM1bjEydm8yZUU2aGYrdlVYNS82d2JySVp1VVhoc0Rxallz?=
 =?utf-8?B?NFBzbVhzZjVVbW1yaUlweGxZbnMxL2xLdGphMW5NSFVNYURZN0hKMmplamsv?=
 =?utf-8?B?eHlHcEFUTTJiR1J5MVU5SnRDWEl1d2lmbW5pZnpURXpvSmNnRUJ5d1BGNGx4?=
 =?utf-8?B?SjVPRFJkMm0rK1pIMThPMGRqVVJpQ2t5V3VrWHEzNTl3ZWRQMktJOGpUT1gr?=
 =?utf-8?B?blFQdDVvdlVVTzhWQkVSa2kvcFR4UGZqT0lSeDcrRURQWWZla3IwOEdVTmht?=
 =?utf-8?B?L1h3b1JYWm4wOXN6U0lyWmFyQkc0Z0lPMDBYcGgwZmVic1p5OFlDbmozZU5w?=
 =?utf-8?B?VVY1Z2xIYkdoRnRnaklrbURYK3RBcjFrMmd5RlRPN294SVlnbEFFU0ZwV2dh?=
 =?utf-8?B?emVVUnhDRDM3Q1docjFjOGsvTUxXOWJKN3FVWVRZbVNsZHpYZlVUMEZFaEN4?=
 =?utf-8?B?Z29GeDZzSVpYaFlZMTgvTnk2RkRnaHhSN2xLRWJDWnNwUzN0MHlod29pNnpt?=
 =?utf-8?B?QkdFYllHNmRVR2duYWZxNG4zWHN0NWZ4ZnFUY3JLaUY3UHZzak5sWVF4UDQw?=
 =?utf-8?B?b3hCaXUxNGlxZHV2RE1rMHQvUXRZYkJxbXlwTjhJZjlYL0VhbWFpNVp6cmhw?=
 =?utf-8?B?elRaQk80UlpoMXJmTDMwOVN0aVJTbHM2NGtBeEhtUTZzempxOFNtN0d0R09O?=
 =?utf-8?B?UkJxYmh2VHZDNHNVQUFrWjJFamx5cFlpVWt2dHZXUU84V0ZEWUdmVnBXczVn?=
 =?utf-8?B?QTRDTTdGUTRQN1puTTJuL3hmQVRXVS9hOFlURWtHNDZxQkZ0azR4aWM2QXc1?=
 =?utf-8?B?RmpGYnpKUTQ1RHROK00vanBhTFlZZlBHYm1KNG1NVEdLYkdmSTYwek9CL3RW?=
 =?utf-8?B?SHo4Sm5tUXpRbDBKY3BoVVVVV0l1eFlKMnNoZU0xN1pOR1hoSzJ1Q0lyOU1h?=
 =?utf-8?B?V1puMGp3Qlh4b3V6dlZHc2hDMUQxbGNITS9xTkRqbmtxZm1TNFplajg5MTZo?=
 =?utf-8?B?aFZTUnI1NzMzNTBYUXVxMThFbEJGMGlNQUdoUnZLdmRrSlRXSFNxMTZKNFYy?=
 =?utf-8?B?dDQ5TWo2QlF1UWVGSmMwMkcwV2ZURS9uTk4yRWVYQnRTR1pNb3ZLZStLVmdR?=
 =?utf-8?B?Yjg1Yk5wdGlJbS82WVRyUEZ0QnFlSWUxTFgrd1JuSDBsU05rV0NlQndzS2sv?=
 =?utf-8?B?TjVXa3RJdnFwOEpEdENxRFJKNzQxTkc1dmlJeEFsZ0Z4SXh0Sk9QeGtEckpH?=
 =?utf-8?B?MHJWM25QU0JrTmlFcEJQNFNXK01MTy94UjFtNjk2RGZjdHBGZnMyd29BZWNY?=
 =?utf-8?Q?bqCyfYE37wT51?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 22:59:30.5060
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 686bc822-8494-40b5-f345-08dc7ab2d6be
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB53.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6265

This is next version of vPCI rework. Aim of this series is to prepare
ground for introducing PCI support on ARM platform.

in v16:
 - minor updates - see individual patches

in v15:
 - reorder so ("arm/vpci: honor access size when returning an error")
   comes first

in v14:
 - drop first 9 patches as they were committed
 - updated ("vpci/header: emulate PCI_COMMAND register for guests")

in v13:
 - drop ("xen/arm: vpci: permit access to guest vpci space") as it was
   unnecessary

in v12:
 - I (Stewart) coordinated with Volodomyr to send this whole series. So,
   add my (Stewart) Signed-off-by to all patches.
 - The biggest change is to re-work the PCI_COMMAND register patch.
   Additional feedback has also been addressed - see individual patches.
 - Drop ("pci: msi: pass pdev to pci_enable_msi() function") and
   ("pci: introduce per-domain PCI rwlock") as they were committed
 - Rename ("rangeset: add rangeset_empty() function")
       to ("rangeset: add rangeset_purge() function")
 - Rename ("vpci/header: rework exit path in init_bars")
       to ("vpci/header: rework exit path in init_header()")

in v11:
 - Added my (Volodymyr) Signed-off-by tag to all patches
 - Patch "vpci/header: emulate PCI_COMMAND register for guests" is in
   intermediate state, because it was agreed to rework it once Stewart's
   series on register handling are in.
 - Addressed comments, please see patch descriptions for details.

in v10:

 - Removed patch ("xen/arm: vpci: check guest range"), proper fix
   for the issue is part of ("vpci/header: emulate PCI_COMMAND
   register for guests")
 - Removed patch ("pci/header: reset the command register when adding
   devices")
 - Added patch ("rangeset: add rangeset_empty() function") because
   this function is needed in ("vpci/header: handle p2m range sets
   per BAR")
 - Added ("vpci/header: handle p2m range sets per BAR") which addressed
   an issue discovered by Andrii Chepurnyi during virtio integration
 - Added ("pci: msi: pass pdev to pci_enable_msi() function"), which is
   prereq for ("pci: introduce per-domain PCI rwlock")
 - Fixed "Since v9/v8/... " comments in changelogs to reduce confusion.
   I left "Since" entries for older versions, because they were added
   by original author of the patches.

in v9:

v9 includes addressed commentes from a previous one. Also it
introduces a couple patches from Stewart. This patches are related to
vPCI use on ARM. Patch "vpci/header: rework exit path in init_bars"
was factored-out from "vpci/header: handle p2m range sets per BAR".

in v8:

The biggest change from previous, mistakenly named, v7 series is how
locking is implemented. Instead of d->vpci_rwlock we introduce
d->pci_lock which has broader scope, as it protects not only domain's
vpci state, but domain's list of PCI devices as well.

As we discussed in IRC with Roger, it is not feasible to rework all
the existing code to use the new lock right away. It was agreed that
any write access to d->pdev_list will be protected by **both**
d->pci_lock in write mode and pcidevs_lock(). Read access on other
hand should be protected by either d->pci_lock in read mode or
pcidevs_lock(). It is expected that existing code will use
pcidevs_lock() and new users will use new rw lock. Of course, this
does not mean that new users shall not use pcidevs_lock() when it is
appropriate.

Changes from previous versions are described in each separate patch.

Oleksandr Andrushchenko (4):
  vpci/header: emulate PCI_COMMAND register for guests
  vpci: add initial support for virtual PCI bus topology
  xen/arm: translate virtual PCI bus topology for guests
  xen/arm: account IO handlers for emulated PCI MSI-X

Volodymyr Babchuk (1):
  arm/vpci: honor access size when returning an error

 xen/arch/arm/vpci.c        | 63 +++++++++++++++++++++++------
 xen/drivers/Kconfig        |  4 ++
 xen/drivers/vpci/header.c  | 60 +++++++++++++++++++++++++---
 xen/drivers/vpci/msi.c     |  9 +++++
 xen/drivers/vpci/msix.c    |  7 ++++
 xen/drivers/vpci/vpci.c    | 81 ++++++++++++++++++++++++++++++++++++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/sched.h    | 10 ++++-
 xen/include/xen/vpci.h     | 28 +++++++++++++
 9 files changed, 244 insertions(+), 19 deletions(-)


base-commit: ced21fbb2842ac4655048bdee56232974ff9ff9c
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 22:59:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 22:59:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727904.1132600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uw5-0007ea-JD; Wed, 22 May 2024 22:59:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727904.1132600; Wed, 22 May 2024 22:59:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uw5-0007eT-GT; Wed, 22 May 2024 22:59:53 +0000
Received: by outflank-mailman (input) for mailman id 727904;
 Wed, 22 May 2024 22:59:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uw4-0007HJ-7G
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 22:59:52 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe6007c1-188e-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 00:59:51 +0200 (CEST)
Received: from MN2PR04CA0035.namprd04.prod.outlook.com (2603:10b6:208:d4::48)
 by IA1PR12MB6649.namprd12.prod.outlook.com (2603:10b6:208:3a2::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Wed, 22 May
 2024 22:59:45 +0000
Received: from BL6PEPF0001AB56.namprd02.prod.outlook.com
 (2603:10b6:208:d4:cafe::11) by MN2PR04CA0035.outlook.office365.com
 (2603:10b6:208:d4::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Wed, 22 May 2024 22:59:45 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB56.mail.protection.outlook.com (10.167.241.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 22:59:45 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 17:59:44 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 17:59:44 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 17:59:38 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe6007c1-188e-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kSWqwj34lnSTz24C5X/+EHE/FPg2TljQtm89RzfiE1VKHedCnedBCw8ut4lHusSeYPRK6Ps/vDPnTm3xQAe9pEQqpe5tSD+NI5KGYK8re6VP9FbEiJNEZGxVGdIIueDqcyGow7oi1l0WNwxb3MIuA6X0F4RVXbvhmGbFrnPOFzhVMeKTibKg0Pal1Xy4IOnp69nVrIclj3mFdGEfz/cgenj7hvMnUyvamCjh2XuNpZZ9Ge0JuUGXsT6CLmf8d5Jr4oT4dWV9O8rcwlFetzIrPIWXDUKcyMt8mCBd8lgawi8+nYgfBZs2ymNSjO0I7So1Bc8Le1yC5XX7TUBOezYw3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=k8+/xJpZRZsK6+RKr2c/38/ePPkO2ljT1mguCOagC4g=;
 b=Z9ZfhUPdEEEFkmrZExIslnLi2/EYu5qEpVyaQoxlo/UwEEBUFO+oz13p6lExGxOo2j1rORaU1MAdqFBK5jqpI0X6ZWNVQEB8oL2EZWmcagnkpm2GgAUTJG+xEIjWeeG8qEhdC9Y2U/J8/WQaUPkaFj+fXjUIvZqWl134mVvwM/BVE1Wn+tl69RQN2Upyfkh5GGjCyInRvjYt8PT8ZliaSJywl1UsO+hMTVKBCA+km/FJ9OIrmofegy+UwuhZV69seuHFbIaFAhCs8zwbKwQg586X7OOb8ft0yB6pVYI77YqP0jlPzHuC1gBOQBti6RRFlFBj5hZbCevzJvwWJuUOwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=k8+/xJpZRZsK6+RKr2c/38/ePPkO2ljT1mguCOagC4g=;
 b=F8xSekaFfvRxCBuE4BX5RQahAK8ZR/fBn72L/nuol54TDUUfQbOAwSQVtpWKa9CDOr8kRAjmqoieCMsHSQhjVJYfVn2TUt+Q0GIwqogx7bPZBqwIKV8yFroHHyf5o6E5YXmDZlkjQ0+V639DFe4VnzHuXAHLpuBESWJ0WSuoB9o=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v16 1/5] arm/vpci: honor access size when returning an error
Date: Wed, 22 May 2024 18:59:20 -0400
Message-ID: <20240522225927.77398-2-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240522225927.77398-1-stewart.hildebrand@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB56:EE_|IA1PR12MB6649:EE_
X-MS-Office365-Filtering-Correlation-Id: 9ffd48ec-c467-4d07-6ea5-08dc7ab2df95
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?ngdG/B6/V0+aQxwTnZmyampS69xAgxf0RITFUzCdtvnbpqKPFLIB5KOYHe6B?=
 =?us-ascii?Q?c6BOJTM3bL4z5wN3ern9RKVPRTkXSEwQbx/klzQdmCMUWlkkUgsajFhMGR0g?=
 =?us-ascii?Q?Y1JfwYi4Tqd0jTTclXR53b33XPHx0R87MluOoGG8D1M7WUWnI5LCxfaHMKMO?=
 =?us-ascii?Q?LtLET4D7ixKK8DwoRodnYCAXcRZccesxIHTohZgESW6c/oOI/IvrtXNhQFS1?=
 =?us-ascii?Q?5WMLssChU0r39DcVgFs8+xDuMLiclMrKxwOHI+Ct3gNg46pXvWMQpaK35i/C?=
 =?us-ascii?Q?9OsSR+dZ5NG563QXty2AQUcD0peU29brq7c2vpzZMV0K3RKavw8lhSD7xjKk?=
 =?us-ascii?Q?iog/fvGnngTBfZqV7gKiZSx87r9JzcDgeAAt8bsMgxfZUYXNeZ3Djl5HEjgH?=
 =?us-ascii?Q?0yrR8DglBlTg82w3SO0P8/SSgc4iu9/LSbkf7bYNaEgS+VHiLS5s5pglT73Y?=
 =?us-ascii?Q?dbw4sTKg0pmIXUYj/1kPantHg/BqFYIBXHbZ/sNH4/XP43t7Ntu7uDwcr/qw?=
 =?us-ascii?Q?iqeYlfCnIov83B4DgVKJuGY03sKBGi+c+QxiYpQN8QdEQpFBmuXR21TWqSlx?=
 =?us-ascii?Q?q16UC8J9lUogorgtsKYlswA+vhsdORtuyMeSmcHgbvWioebscMxn0LoQFHoe?=
 =?us-ascii?Q?HSnLkpAsIZ/HlXy6m8zm/XnCgOfMZvoZ1G+i5mkDXqXv4UW0+55Aoesx9lfi?=
 =?us-ascii?Q?IApk10yBAFj4m9dzZ3+gsWwGQVRE29gISLamqxHJMbwYI9giDAJJyAoVLE/h?=
 =?us-ascii?Q?0najPYzC7+ku1icdugIvtHMIMStGuMPSkIWcg6GnoQD2jOzG9hVSswbqw0oz?=
 =?us-ascii?Q?uNX8UM/HnyXTM4aEnFr9hCpTw6GU0qG06+3NmSn5E0DWMbhInyxC/R3flQdX?=
 =?us-ascii?Q?mkkD/3C1cL2djnb1QwJoyNRsYh5AV/IPKVAqjLVnI4LrhLw9f/xO3yxUq/e2?=
 =?us-ascii?Q?fjVGq4BvB2TIhvDnBtMPqAiXyBzZAzxlAQi5G6EeB1IB4QXf3Fu9ifwkFfwZ?=
 =?us-ascii?Q?YlCftC+6eviuQ1AOTuQpXDJcHZftJ9sOyDArAYfe9SR28dSaE1uDwJSef1PU?=
 =?us-ascii?Q?E2PBzBya8xKLuIgYAxqeqAL4E+SKd0eaflIJJdu39ih1oKJiApIRpLEijz7f?=
 =?us-ascii?Q?VZmsG/csNGVSzFnronDM0/qlz8IBiEkSxW94/2x4T1K69ZyDvdPRFl087VZ2?=
 =?us-ascii?Q?18oy/j33F7fjstDzoOoYNjK/dlaEg4W1Mc6DG/ACnxh3KqzxKSQ+2flLmlvU?=
 =?us-ascii?Q?owxq5YSuYrMKzV5AU2tPXkxYhsSbYlwjxwsD2G055QWlD2vsRmRXGc0qMlpn?=
 =?us-ascii?Q?Q7UFE6HepFOsnDe7ZFAjZvLweF4BOZQsu9h1G6OUYXhXL8kVpyJJey/z59YX?=
 =?us-ascii?Q?By3opQ/fK9u7RbSYVLhP7iZglIit?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 22:59:45.3178
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ffd48ec-c467-4d07-6ea5-08dc7ab2df95
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB56.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6649

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

Guest can try to read config space using different access sizes: 8,
16, 32, 64 bits. We need to take this into account when we are
returning an error back to MMIO handler, otherwise it is possible to
provide more data than requested: i.e. guest issues LDRB instruction
to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
register.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
v14->v15:
* re-order so this patch comes before ("xen/arm: translate virtual PCI
  bus topology for guests")
* s/access_mask/invalid/
* add U suffix to 1
* s/uint8_t/unsigned int/
* s/uint64_t/register_t/
* although Julien gave an Acked-by on v14, I omitted it due to the
  changes made in v15

v9->10:
* New patch in v10.
---
 xen/arch/arm/vpci.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 3bc4bb55082a..b63a356bb4a8 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 {
     struct pci_host_bridge *bridge = p;
     pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    const unsigned int access_size = (1U << info->dabt.size) * 8;
+    const register_t invalid = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
@@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0ul;
+    *r = invalid;
 
     return 0;
 }
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:00:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:00:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727908.1132611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwF-0008Sv-VT; Wed, 22 May 2024 23:00:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727908.1132611; Wed, 22 May 2024 23:00:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwF-0008Sl-Pv; Wed, 22 May 2024 23:00:03 +0000
Received: by outflank-mailman (input) for mailman id 727908;
 Wed, 22 May 2024 23:00:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uwD-00085F-UR
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 23:00:02 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02b82ec6-188f-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 00:59:59 +0200 (CEST)
Received: from SA9PR13CA0006.namprd13.prod.outlook.com (2603:10b6:806:21::11)
 by LV8PR12MB9133.namprd12.prod.outlook.com (2603:10b6:408:188::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 22:59:54 +0000
Received: from SN1PEPF0002BA4C.namprd03.prod.outlook.com
 (2603:10b6:806:21:cafe::57) by SA9PR13CA0006.outlook.office365.com
 (2603:10b6:806:21::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 22:59:54 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF0002BA4C.mail.protection.outlook.com (10.167.242.69) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 22:59:53 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 17:59:53 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 17:59:51 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02b82ec6-188f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OLwmS6ciPK4yUCw0wRhDKnGjjhzZjz0hp0ifIJLcg9rUv02fF0DTiadQNkJIwEbdTSCDpWKCsUbwkcmYIBh6/cpV81zgSmcNMMyCisp69TJUMDnJpGZmr5jJxKqmbSYLySlyga3dQsjipwPgSI1T1QPKwaT2QuJ+NOqMju4nxqZxyk28FXpJSfMs/Wg3wI03/6+01j5xfgTvd5c/RCGsQedApk+9hSQaXNPRCLSNTJnLFWENjcOciOt0yI/x5MausQ8jO5aQpghuYEDcxijcno9uFUT4WqYoVoIRcipEpDezuyyBfPYxgaJqZMI5jh1jyVlmxdfV2JP/VtoZnCRDmQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pOnJWNfrhJ3Efv0BMkAnX4PfL/TY+zGAVYbpOc2ZfC8=;
 b=AKGmZa1jlzyD0w9esu1cSMwMrEEfdpGU3RZF0IgBCfjQ1u0w1sBWG0NM+7dAvdejPFDPsu4TOUCNMWW6QGREFyWmdxnw4hJXZ5xwQRk6ey+aLPZz5n0SbdexrLPVFuFkW/gPrAUJYS0fYYDzD1Zw775A7KldQHDzrGEJT3nEHZ8euj1gnXPG+dqxXQdLrwxywtpcS95hLCfBQZZBxOkdwoHirJYlDMePlvqGwcSFGR8Ybp/nwpaVL4Wrwfe26Zot/p6ibPVyHRJyccjKA2qYAr0ZUlNykOFiPyQjQKiZm9VbugdLsLS+8Ht9RLIr8p3uTYIk1c6k+YeZAC6to3lgTw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pOnJWNfrhJ3Efv0BMkAnX4PfL/TY+zGAVYbpOc2ZfC8=;
 b=jV8F2kLA7ntBg3SHg5q2WDOmhI4OWfp47BHLr2dX1e0ogNVGZFxFr+R6I/3yMVeTG0k45XdRPWa3Z6cwiXAS1Vw0QK2cVOj2u5qtev+ON2oWl9JvrHFYn0XUxJZuFhbnCu3WcRz5qTxNCS1riEQljvre4GD3jetVUPY2NwlBkY8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
Subject: [PATCH v16 2/5] vpci/header: emulate PCI_COMMAND register for guests
Date: Wed, 22 May 2024 18:59:21 -0400
Message-ID: <20240522225927.77398-3-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240522225927.77398-1-stewart.hildebrand@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4C:EE_|LV8PR12MB9133:EE_
X-MS-Office365-Filtering-Correlation-Id: 151bd5f9-2278-4276-0408-08dc7ab2e48a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?SmF6M2czSU9YZU54QW9COVNzZFFjYUFRS0lTN3U4TlNRN2RuVUNoNW9zVjhT?=
 =?utf-8?B?dmt3NndwY0tmTmRxU0JmdDZGeW9sMDFYMGJZeFZmb2N3UWFEZ290S2M2dTJQ?=
 =?utf-8?B?azVqSmJxa2V1Z0UzNmlFK2IzOTZ0L2NnNXBDMCs3OEZZUmJHVk5mWVJhRUhN?=
 =?utf-8?B?WmRPOS9sTktGd1FMUXU1aU80ckhEbzl0ZkxTM0lTOUFmZ01rWVJ4azZYRFdq?=
 =?utf-8?B?eURVOGsxWFR6bk9EaStZalNsRFREUFNXandQY0lOT2d4bnVxUW1GQWpob3Mv?=
 =?utf-8?B?cFlXdnIvbU9tZG9Nd2JZTEFUZ3locGFVSnRhNWl6c2dxVUUwalNpeGJVOHVu?=
 =?utf-8?B?cEVIM0UxSEE1V2sxUi9pRHJaSHVwcnRCN1g0c05rZUg5K1gwdFV4UWROMW5N?=
 =?utf-8?B?dFF2T0ZCdGljbENpRlNhRmFhTTh1QjlpT0c2ZkpzMUpJNFp6RHlFMzJKTWZK?=
 =?utf-8?B?QkV0Q3R3azZ1V25sUG1hT3NEVFdrV0RSZ3lDQWZjenhTazhIb3R5VHdnOUw3?=
 =?utf-8?B?L01aU2FCbzhYN0JtTHJBUVkvNjNRaytwckNxOXUwdldpWklpNEd2TFN0a0pV?=
 =?utf-8?B?ZnQ0MisrcTc2bHdkbjVFTUpOT29naWdkalNZQjIvMmVmSDVVaSt4ckM5a3gr?=
 =?utf-8?B?RTdmbk9mN3JVaTdkZXJUQzlXRmd2ejN0NHhlSjlMQ25Kb2d4OUdndmkyT2Zk?=
 =?utf-8?B?aEdCaFpkdW9VWnNJVVd5VTJvRmU1blR0Sm4rc2N4NTc1TmFhMCt0ckU0MnV3?=
 =?utf-8?B?NHJhMzRvSUhnVitqMjlNVXdMVVJ2UnNLb0RIaU1lTjI5ckpmSG9Fc2hhNUFJ?=
 =?utf-8?B?cmtGSkNDWTVYSGdldXdJUDBwUXBOZWlSVVJ2MEhyVGZRWTlJd1cwQ1VrZFBm?=
 =?utf-8?B?d1ZnU1NVZXhqU0Z0d3NvdTlIYjVUUXdRMk5RQmRYVm04bXQxVC8ralZpUERO?=
 =?utf-8?B?Z2h1aHcyYnZrcWszNmE2c3lVaFZ2OVZaSGxsYnowOFlVSjRHV09wcmdxWUk5?=
 =?utf-8?B?aU5pVWJ2UjZreEhSSWc2V0pVSmpXdnp0VnR1T1VMZjhrc2hRcWE5WEx1YzA4?=
 =?utf-8?B?Um0yN0x1eHVVTmZja3V5cjVCblMxd1VZUXgxKzVHSCs3UzlNYWpWbW1WYng4?=
 =?utf-8?B?eDhabVplMlMvejViem9SUVhzNTRnRGhSOXBvY2dBZUJKSnM2VWpNeFFudmlx?=
 =?utf-8?B?Yy9NSXVVMHpXMWNvOHI0bnlKcUVmWkdjTDBYUklvSTZVNDI2bG1UekNoYVM5?=
 =?utf-8?B?NFRGeDV1a0ZJMEQ4UmR4cElCTGF5ZWlrVXo3cjFGT2NsY09UTmlZT3F1K0pl?=
 =?utf-8?B?V1NqUGM0SFdlM2k1L09xbDEvOFJ5UVFta2hyYjdZbHRRdmFvbFE1ek9HekxV?=
 =?utf-8?B?SXNXRHUwU3pOUmZ1K0lIRXhRQkJ0VVNJY09uS1ZJU3c5UWdTZktJOCtZTGV5?=
 =?utf-8?B?ZEZ2T1ExMnFMZzFmU3dSZURHKy9QT0twMzl3M2ZCbnRKTktFZzJSK0JETW56?=
 =?utf-8?B?aWhINnJqMS9nY1YyaStyQnB1eEp6Ym1oM01KallJZm9XWk9QNG5tUlVMOXZO?=
 =?utf-8?B?bkpQVDVJUFdWTDd2Uk14bDllYnFBUkhkd2d1cDM3QnlTYWRoV3lMOHlJS2J3?=
 =?utf-8?B?UThYbElJRWp6RVo3aWNMODdJNE5Dc0tSdlFPMHlMczNkMnRWYUFENm9nMmRu?=
 =?utf-8?B?U1J6dWR5Tm1lWXA2cDlyQ0JFVnZ2ZWxkcUI3R0xQMzF6dkpnSklYK0U5bG5i?=
 =?utf-8?B?QWVlMGZEcVl3Q1NDL2g2TFJ3MVZrNHUvbVkxLzVwWGFkcXZqU2ZnQTRJejBk?=
 =?utf-8?B?Y0xoRlZvU2Jrc2J6MnFhL1oxTGI2dUNTekR2VDloVExNOUUza3Y5OUZHZHVD?=
 =?utf-8?Q?9ZatuV2f1fTlU?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 22:59:53.6160
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 151bd5f9-2278-4276-0408-08dc7ab2e48a
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002BA4C.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9133

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
guest's (domU) view of this will want to be zero (for now), the host
having set it to 1 should be preserved, or else we'd effectively be
giving the domU control of the bit. Thus, PCI_COMMAND register needs
proper emulation in order to honor host's settings.

According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
Device Control" the reset state of the command register is typically 0,
so when assigning a PCI device use 0 as the initial state for the
guest's (domU) view of the command register.

Here is the full list of command register bits with notes about
PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
also documented here since that is our current reference implementation
for PCI passthrough.

PCI_COMMAND_IO (bit 0)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if an I/O BAR is exposed to the guest.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
    don't yet support I/O BARs for domUs.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_MEMORY (bit 1)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if a Memory BAR is exposed to the guest.
  Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
    regions.
  Xen domU: For devices assigned to DomUs, memory decoding will be
    disabled at the time of initialization.

PCI_COMMAND_MASTER (bit 2)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_SPECIAL (bit 3)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_INVALIDATE (bit 4)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_VGA_PALETTE (bit 5)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_PARITY (bit 6)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_WAIT (bit 7)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: hardwire to 0
  QEMU: res_mask
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_SERR (bit 8)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_FAST_BACK (bit 9)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_INTX_DISABLE (bit 10)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU checks if INTx was mapped
    for a device. If it is not, then guest can't control
    PCI_COMMAND_INTX_DISABLE bit.
  Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
  Xen dom0: We allow dom0 to control this bit freely.

Bits 11-15
  PCIe 6.1: RsvdP
  PCI LB 3.0: Reserved
  QEMU: res_mask
  Xen domU: rsvdp_mask
  Xen dom0: We allow dom0 to control these bits freely.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
---
In v16:
- allow dom0 to freely control RsvdP bits (11-15)
- add Roger's R-b

In v15:
- add Jan's R-b
- add blank line after declaration in msi.c:control_write()

In v14:
- check for 0->1 transition in INTX_DISABLE-setting logic in
  msi.c:control_write() to match msix.c:control_write()
- clear domU-controllable bits in header.c:init_header()

In v13:
- Update right away (don't defer) PCI_COMMAND_MEMORY bit in guest_cmd
  variable in cmd_write()
- Make comment single line in xen/drivers/vpci/msi.c:control_write()
- Rearrange memory decoding disabling snippet in init_header()

In v12:
- Rework patch using vpci_add_register_mask()
- Add bitmask #define in pci_regs.h according to PCIe 6.1 spec, except
  don't add the RO bits because they were RW in PCI LB 3.0 spec.
- Move and expand TODO comment about properly emulating bits
- Update guest_cmd in msi.c/msix.c:control_write()
- Simplify cmd_write(), thanks to rsvdp_mask
- Update commit description

In v11:
- Fix copy-paste mistake: vpci->msi should be vpci->msix
- Handle PCI_COMMAND_IO
- Fix condition for disabling INTx in the MSI-X code
- Show domU changes to only allowed bits
- Show PCI_COMMAND_MEMORY write only after P2M was altered
- Update comments in the code
In v10:
- Added cf_check attribute to guest_cmd_read
- Removed warning about non-zero cmd
- Updated comment MSI code regarding disabling INTX
- Used ternary operator in vpci_add_register() call
- Disable memory decoding for DomUs in init_bars()
In v9:
- Reworked guest_cmd_read
- Added handling for more bits
Since v6:
- fold guest's logic into cmd_write
- implement cmd_read, so we can report emulated INTx state to guests
- introduce header->guest_cmd to hold the emulated state of the
  PCI_COMMAND register for guests
Since v5:
- add additional check for MSI-X enabled while altering INTX bit
- make sure INTx disabled while guests enable MSI/MSI-X
Since v3:
- gate more code on CONFIG_HAS_MSI
- removed logic for the case when MSI/MSI-X not enabled
---
 xen/drivers/vpci/header.c  | 60 ++++++++++++++++++++++++++++++++++----
 xen/drivers/vpci/msi.c     |  9 ++++++
 xen/drivers/vpci/msix.c    |  7 +++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/vpci.h     |  3 ++
 5 files changed, 74 insertions(+), 6 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 47648c395132..ef6c965c081c 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -524,9 +524,21 @@ static void cf_check cmd_write(
 {
     struct vpci_header *header = data;
 
+    if ( !is_hardware_domain(pdev->domain) )
+    {
+        const struct vpci *vpci = pdev->vpci;
+
+        if ( (vpci->msi && vpci->msi->enabled) ||
+             (vpci->msix && vpci->msix->enabled) )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+
+        header->guest_cmd = cmd;
+    }
+
     /*
      * Let Dom0 play with all the bits directly except for the memory
-     * decoding one.
+     * decoding one. Bits that are not allowed for DomU are already
+     * handled above and by the rsvdp_mask.
      */
     if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) )
         /*
@@ -540,6 +552,14 @@ static void cf_check cmd_write(
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static uint32_t cf_check guest_cmd_read(
+    const struct pci_dev *pdev, unsigned int reg, void *data)
+{
+    const struct vpci_header *header = data;
+
+    return header->guest_cmd;
+}
+
 static void cf_check bar_write(
     const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
 {
@@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    /*
+     * Setup a handler for the command register.
+     *
+     * TODO: If support for emulated bits is added, re-visit how to handle
+     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
+     */
+    rc = vpci_add_register_mask(pdev->vpci,
+                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
+                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
+                                is_hwdom ? 0
+                                         : PCI_COMMAND_RSVDP_MASK |
+                                           PCI_COMMAND_IO |
+                                           PCI_COMMAND_PARITY |
+                                           PCI_COMMAND_WAIT |
+                                           PCI_COMMAND_SERR |
+                                           PCI_COMMAND_FAST_BACK,
+                                0);
     if ( rc )
         return rc;
 
@@ -836,9 +870,23 @@ static int cf_check init_header(struct pci_dev *pdev)
     if ( pdev->ignore_bars )
         return 0;
 
-    /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
-    if ( cmd & PCI_COMMAND_MEMORY )
+
+    /*
+     * For DomUs, clear PCI_COMMAND_{MASTER,MEMORY,IO} and other
+     * DomU-controllable bits in PCI_COMMAND. Devices assigned to DomUs will
+     * start with memory decoding disabled, and modify_bars() will not be called
+     * at the end of this function.
+     */
+    if ( !is_hwdom )
+        cmd &= ~(PCI_COMMAND_VGA_PALETTE | PCI_COMMAND_INVALIDATE |
+                 PCI_COMMAND_SPECIAL | PCI_COMMAND_MASTER | PCI_COMMAND_MEMORY |
+                 PCI_COMMAND_IO);
+
+    header->guest_cmd = cmd;
+
+    /* Disable memory decoding before sizing. */
+    if ( !is_hwdom || (cmd & PCI_COMMAND_MEMORY) )
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY);
 
     for ( i = 0; i < num_bars; i++ )
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index 30adcf7df05d..dd6620ec5674 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -57,6 +57,8 @@ static void cf_check control_write(
 
     if ( new_enabled )
     {
+        bool old_enabled = msi->enabled;
+
         /*
          * If the device is already enabled it means the number of
          * enabled messages has changed. Disable and re-enable the
@@ -70,6 +72,13 @@ static void cf_check control_write(
 
         if ( vpci_msi_arch_enable(msi, pdev, vectors) )
             return;
+
+        /* Make sure domU doesn't enable INTx while enabling MSI. */
+        if ( !old_enabled && !is_hardware_domain(pdev->domain) )
+        {
+            pci_intx(pdev, false);
+            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
     }
     else
         vpci_msi_arch_disable(msi, pdev);
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index 58c16ebdf283..fbe710ab92ef 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -135,6 +135,13 @@ static void cf_check control_write(
         }
     }
 
+    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
+    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
+    {
+        pci_intx(pdev, false);
+        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+    }
+
     msix->masked = new_masked;
     msix->enabled = new_enabled;
 
diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h
index 0bc18efabb74..250ba106dbd3 100644
--- a/xen/include/xen/pci_regs.h
+++ b/xen/include/xen/pci_regs.h
@@ -48,6 +48,7 @@
 #define  PCI_COMMAND_SERR	0x100	/* Enable SERR */
 #define  PCI_COMMAND_FAST_BACK	0x200	/* Enable back-to-back writes */
 #define  PCI_COMMAND_INTX_DISABLE 0x400 /* INTx Emulation Disable */
+#define  PCI_COMMAND_RSVDP_MASK	0xf800
 
 #define PCI_STATUS		0x06	/* 16 bits */
 #define  PCI_STATUS_IMM_READY	0x01	/* Immediate Readiness */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 6e4c972f35ed..2064d9354f5b 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -107,6 +107,9 @@ struct vpci {
         } bars[PCI_HEADER_NORMAL_NR_BARS + 1];
         /* At most 6 BARS + 1 expansion ROM BAR. */
 
+        /* Guest (domU only) view of the PCI_COMMAND register. */
+        uint16_t guest_cmd;
+
         /*
          * Store whether the ROM enable bit is set (doesn't imply ROM BAR
          * is mapped into guest p2m) if there's a ROM BAR on the device.
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:00:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:00:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727912.1132620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwM-0000wN-BW; Wed, 22 May 2024 23:00:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727912.1132620; Wed, 22 May 2024 23:00:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwM-0000w5-8B; Wed, 22 May 2024 23:00:10 +0000
Received: by outflank-mailman (input) for mailman id 727912;
 Wed, 22 May 2024 23:00:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uwL-0007HJ-Om
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 23:00:09 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 089cad80-188f-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 01:00:09 +0200 (CEST)
Received: from PH8PR07CA0015.namprd07.prod.outlook.com (2603:10b6:510:2cd::20)
 by DS7PR12MB5959.namprd12.prod.outlook.com (2603:10b6:8:7e::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Wed, 22 May
 2024 23:00:04 +0000
Received: from SN1PEPF0002BA4B.namprd03.prod.outlook.com
 (2603:10b6:510:2cd:cafe::69) by PH8PR07CA0015.outlook.office365.com
 (2603:10b6:510:2cd::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.21 via Frontend
 Transport; Wed, 22 May 2024 23:00:03 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF0002BA4B.mail.protection.outlook.com (10.167.242.68) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 23:00:03 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:00:03 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:00:02 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 18:00:01 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 089cad80-188f-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VCb1Jg5weSlF7ymph+4mnXbVHH52neQ8HxIphgiqeUC1nTyEzXEBdb2IEGrUJr3OrGT4nERAU9SOvLFs+u2de8U0tSeVlpQo5kWpfiKThUEURakGQmAQpypxjPXmIYkvVqCqtb9bSg3TgTgPhMTBqw//zY7+DE+cF9MybQLTQx5dIWxLtwms9PitwjBJJhWG9vsNjpZ17UBxkCEC2P8XJ+cl5EFwbHSV+lAbm1RnzAL7UC7/MITtPQGSWkXEslhak5hcZJwX47e/IKMNStdwCWHbjlrocIlXTm4okiwbGvpazRBn2DDm7ecvJ13jJHLy04gsEwbfF0jemfelstkQrA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=91z7nZP+Kt+sNbijuSvMEIgPYCMJ9j1zgXB7s2O6sEQ=;
 b=og2OV92dLpYsp9J64GHz2H2Q96S3uwXHDfmxxpXxVAgTs8CVGQKYhjAJwI5uI9oURm16wJIBCdvKFLCpVjoBqS3kkppBzbdBFaM7iv46uZZZp0HKAZkhXzaWquW77PAc0J9i4pA1Dy/l2aI8HlGcJe5dBK2JXE5/zmyiZkZf8UgLt3h7a+puXWmzti10lCkyvnKecn4W3Q/hlB7pCl9EykcZv9NbXCtTRDdkM6L1Rqs99OG2co4AkpmXhpFyys7KBc/r6lEQw/UgN3EBAizMAQx5pfiUxqJkPf8oXiootOqKyBoBiKEPUBRt3BVf/CoZq8aTn8fwrDIcMlfM683deQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=91z7nZP+Kt+sNbijuSvMEIgPYCMJ9j1zgXB7s2O6sEQ=;
 b=RNZFhMhSaBudPjYAYP1zM0OkfmvRZoms4Sr+kRv94c0R0dMRzm4P8euQnwdD3StHsrLoc5r9FCVywhqoTb75seWCcgta5lvJu7ojUP9h82B/cAnQxRZjc+ewiqwaIyDCQYsbXXYogXPxOiFe+y/t3KHNp9rYtpHsHFFM3zln5pU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v16 3/5] vpci: add initial support for virtual PCI bus topology
Date: Wed, 22 May 2024 18:59:22 -0400
Message-ID: <20240522225927.77398-4-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240522225927.77398-1-stewart.hildebrand@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4B:EE_|DS7PR12MB5959:EE_
X-MS-Office365-Filtering-Correlation-Id: 3882d1c7-f221-4ec9-c588-08dc7ab2ea80
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?Fcj013H94GzpQGmy8lHng1XcpRo2lzRsJ6evaN1+TWJ81nxgt14dWo50z6zV?=
 =?us-ascii?Q?mavcrcnSBVCG3qhIjzJv9lusbvsM8mRmy+nKNg8vvMC/ymOUX80ZOnJ3qIkS?=
 =?us-ascii?Q?AVLUfiLqcOkeOGNNU76099uJa8Oc6i8ZAeGW50+0qADE51V5zBqQfvWjtrnK?=
 =?us-ascii?Q?ig459zMPHsajQl8F15YcKIWi9QkZ7wiVsOgZlKNcyVPr1+6V+Uo5YgVIRSEt?=
 =?us-ascii?Q?+CkunmPX0K967Qik9NKFmCHQH4hB06P1O7Eud/TfBbKg1Golt7Y38TuXeoxO?=
 =?us-ascii?Q?iXCraScHuBYSOv6G5sj7ybodC5JfBplNwTNzbWPvLLoBkJayf/BJmoAAdAil?=
 =?us-ascii?Q?z0ROZZdB9k2lGOzVmSk24ck/r3ihS5ybGfoJgHNZtu1Ae1oaIax8YazBkxR4?=
 =?us-ascii?Q?Wcbli7SkM7Ym1D4aKBxSy3RNk5xXgjr+eexz0PkwGpUovCpU3m6gkOY8ayq4?=
 =?us-ascii?Q?IeEyBIxk7XGFMiw0RbhmYw6M3BFwOPj2rwg90sl06h63w2w/ie0+8zmfrWmM?=
 =?us-ascii?Q?zzM5+bdi7LabiRLfqzhr7ncxmaI7XIz+v5R48M9QpmEJq59Wz3FH0Cj1YNmB?=
 =?us-ascii?Q?k/FKX36JLsDtiimIMI6EANM+UCERYq8zU/OSXE1I4luiuC8Emr+57w8+lIS4?=
 =?us-ascii?Q?LdZZuJTcA9NngD+j5Pm+R38222UvwPcWhI2cal4Rn0vSy4QhAMeN1EzaHt8c?=
 =?us-ascii?Q?naBLPXQ64YcvtAQPUxzbhpylGBKLTZ1Q9POvLuMbKstHiOjaB7Cs73nU1Q/a?=
 =?us-ascii?Q?9Egv2vNV4534K3MFVN1JEhZn91bXpugwO3vm3kpIVKBtcn1r3u3FwQ7hk/aK?=
 =?us-ascii?Q?jx5U5VxvepK8spUZhxCllr4ShYFCgO0O0zyv1cLAw1yJpKpdqP4DbkZb9eoG?=
 =?us-ascii?Q?ZXwjrelUqKWb6DgqrsF3SJyyBhVyLJZ64AbTric0RK5Bv6l28XReoQyx+VpQ?=
 =?us-ascii?Q?A9jC+n3jwj+5uTZMdA/g5aK7iSdOrcl1HWZBi/qg9ywaxVsdjKaefkME0SFq?=
 =?us-ascii?Q?QwXixwhOh+O5LVCgDlkKu+CfgFNLPCVogcKv+7nfe5qJdjy7lFf6m3WT/lK1?=
 =?us-ascii?Q?Drv5LI8YVXsemg9vyWUEDXXcLXbqLqjwPfa/SgWf4Q+jXD+3Y6BgQJGqi8q0?=
 =?us-ascii?Q?1kqRES1WB2s+At6J81moqlljy8sKAsUFfHUZ0zXYkZNEpMJvZUnECwEmMRjo?=
 =?us-ascii?Q?sAFIyKaLE6pZbt0hSP3lEyK4miIagSm8nVCOPc5nq+GI1k8QxDJqGO5wQHis?=
 =?us-ascii?Q?EcCoKEqnl/XNOPVFnXI1QaT4xgdNm0COd/2u1YgOgCnJyDNwjv3VeO5MlWN7?=
 =?us-ascii?Q?RUhb1qkR9tVPWbbd0AXcUQsYpkBFgHZZsk6cacJmCehZJT5NNGXBAYqe4WHL?=
 =?us-ascii?Q?3WkGzXA=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 23:00:03.6217
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3882d1c7-f221-4ec9-c588-08dc7ab2ea80
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002BA4B.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5959

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Please note, that at the moment only function 0 of a multifunction
device can be passed through.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
In v16:
- s/add_virtual_device/assign_virtual_sbdf/
- move ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)) earlier
- add #define INVALID_GUEST_SBDF
In v15:
- add Jan's A-b
In v13:
- s/depends on/select/ in Kconfig
- check pdev->sbdf.fn instead of two booleans in add_virtual_device()
- comment #endifs in sched.h
- clarify comment about limits in vpci.h with seg/bus limit
In v11:
- Fixed code formatting
- Removed bogus write_unlock() call
- Fixed type for new_dev_number
In v10:
- Removed ASSERT(pcidevs_locked())
- Removed redundant code (local sbdf variable, clearing sbdf during
device removal, etc)
- Added __maybe_unused attribute to "out:" label
- Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the
  first patch where it is used (previously was in "vpci: add hooks for
  PCI device assign/de-assign")
In v9:
- Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart)
In v8:
- Added write lock in add_virtual_device
Since v6:
- re-work wrt new locking scheme
- OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device()
Since v5:
- s/vpci_add_virtual_device/add_virtual_device and make it static
- call add_virtual_device from vpci_assign_device and do not use
  REGISTER_VPCI_INIT machinery
- add pcidevs_locked ASSERT
- use DECLARE_BITMAP for vpci_dev_assigned_map
Since v4:
- moved and re-worked guest sbdf initializers
- s/set_bit/__set_bit
- s/clear_bit/__clear_bit
- minor comment fix s/Virtual/Guest/
- added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used
  later for counting the number of MMIO handlers required for a guest
  (Julien)
Since v3:
 - make use of VPCI_INIT
 - moved all new code to vpci.c which belongs to it
 - changed open-coded 31 to PCI_SLOT(~0)
 - added comments and code to reject multifunction devices with
   functions other than 0
 - updated comment about vpci_dev_next and made it unsigned int
 - implement roll back in case of error while assigning/deassigning devices
 - s/dom%pd/%pd
Since v2:
 - remove casts that are (a) malformed and (b) unnecessary
 - add new line for better readability
 - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
    functions are now completely gated with this config
 - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/drivers/Kconfig     |  4 +++
 xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched.h | 10 +++++++-
 xen/include/xen/vpci.h  | 13 ++++++++++
 4 files changed, 83 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..20050e9bb8b3 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	select HAS_VPCI
+
 endmenu
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 97e115dc5798..1e6aa5d799b9 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+static int assign_virtual_sbdf(struct pci_dev *pdev)
+{
+    struct domain *d = pdev->domain;
+    unsigned int new_dev_number;
+
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
+    if ( is_hardware_domain(d) )
+        return 0;
+
+    /*
+     * Each PCI bus supports 32 devices/slots at max or up to 256 when
+     * there are multi-function ones which are not yet supported.
+     */
+    if ( pdev->sbdf.fn )
+    {
+        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
+                 &pdev->sbdf);
+        return -EOPNOTSUPP;
+    }
+    new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map,
+                                         VPCI_MAX_VIRT_DEV);
+    if ( new_dev_number == VPCI_MAX_VIRT_DEV )
+        return -ENOSPC;
+
+    __set_bit(new_dev_number, &d->vpci_dev_assigned_map);
+
+    /*
+     * Both segment and bus number are 0:
+     *  - we emulate a single host bridge for the guest, e.g. segment 0
+     *  - with bus 0 the virtual devices are seen as embedded
+     *    endpoints behind the root complex
+     *
+     * TODO: add support for multi-function devices.
+     */
+    pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0);
+
+    return 0;
+}
+
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 void vpci_deassign_device(struct pci_dev *pdev)
 {
     unsigned int i;
@@ -49,6 +92,12 @@ void vpci_deassign_device(struct pci_dev *pdev)
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    if ( pdev->vpci->guest_sbdf.sbdf != ~0 )
+        __clear_bit(pdev->vpci->guest_sbdf.dev,
+                    &pdev->domain->vpci_dev_assigned_map);
+#endif
+
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
     {
@@ -103,6 +152,13 @@ int vpci_assign_device(struct pci_dev *pdev)
     INIT_LIST_HEAD(&pdev->vpci->handlers);
     spin_lock_init(&pdev->vpci->lock);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    pdev->vpci->guest_sbdf = INVALID_GUEST_SBDF;
+    rc = assign_virtual_sbdf(pdev);
+    if ( rc )
+        goto out;
+#endif
+
     for ( i = 0; i < NUM_VPCI_INIT; i++ )
     {
         rc = __start_vpci_array[i](pdev);
@@ -110,6 +166,7 @@ int vpci_assign_device(struct pci_dev *pdev)
             break;
     }
 
+ out: __maybe_unused;
     if ( rc )
         vpci_deassign_device(pdev);
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 132b84199598..2dcd1d1a4f8a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -486,7 +486,15 @@ struct domain
      * 2. pdev->vpci->lock
      */
     rwlock_t pci_lock;
-#endif
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /*
+     * The bitmap which shows which device numbers are already used by the
+     * virtual PCI bus topology and is used to assign a unique SBDF to the
+     * next passed through virtual PCI device.
+     */
+    DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV);
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+#endif /* CONFIG_HAS_PCI */
 
 #ifdef CONFIG_HAS_PASSTHROUGH
     struct domain_iommu iommu;
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 2064d9354f5b..da8d0f41e6f4 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -21,6 +21,14 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 
 #define VPCI_ECAM_BDF(addr)     (((addr) & 0x0ffff000) >> 12)
 
+/*
+ * Maximum number of devices supported by the virtual bus topology:
+ * each PCI bus supports 32 devices/slots at max or up to 256 when
+ * there are multi-function ones which are not yet supported.
+ * This limit implies only segment 0, bus 0 is supported.
+ */
+#define VPCI_MAX_VIRT_DEV       (PCI_SLOT(~0) + 1)
+
 #define REGISTER_VPCI_INIT(x, p)                \
   static vpci_register_init_t *const x##_entry  \
                __used_section(".data.vpci." p) = (x)
@@ -175,6 +183,11 @@ struct vpci {
             struct vpci_arch_msix_entry arch;
         } entries[];
     } *msix;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /* Guest SBDF of the device. */
+#define INVALID_GUEST_SBDF ((pci_sbdf_t){ .sbdf = ~0U })
+    pci_sbdf_t guest_sbdf;
+#endif
 #endif
 };
 
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:00:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:00:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727916.1132630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwg-0001cl-LY; Wed, 22 May 2024 23:00:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727916.1132630; Wed, 22 May 2024 23:00:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwg-0001ce-HR; Wed, 22 May 2024 23:00:30 +0000
Received: by outflank-mailman (input) for mailman id 727916;
 Wed, 22 May 2024 23:00:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uwf-00085F-FS
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 23:00:29 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13f89e88-188f-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 01:00:27 +0200 (CEST)
Received: from CH2PR19CA0009.namprd19.prod.outlook.com (2603:10b6:610:4d::19)
 by DM4PR12MB5866.namprd12.prod.outlook.com (2603:10b6:8:65::6) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7611.20; Wed, 22 May 2024 23:00:18 +0000
Received: from DS3PEPF000099DC.namprd04.prod.outlook.com
 (2603:10b6:610:4d:cafe::30) by CH2PR19CA0009.outlook.office365.com
 (2603:10b6:610:4d::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35 via Frontend
 Transport; Wed, 22 May 2024 23:00:18 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF000099DC.mail.protection.outlook.com (10.167.17.198) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 23:00:18 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:00:17 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 18:00:10 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13f89e88-188f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Jt1/NlIeAv8OkqunZeZ/Wscjincc4KRRgJInrNPAUejv4X6DMD3TLTWMiOE9a+Ay0NySuA0oXUtgHsx8CEoZr5ex++Rrd+3v3YoonDhHg7lxCPFngzq/8YvkQoG44GHkRyCPDjmUmjwo9jQ+0HYQD3jfKru9EXQ3/pR/Bf4v0jfPHppqMt0ck8xSqLE8A1wNB7BUQyAjFQTN9s11hR7mcK9OdYn6Xn81djzx01sj37QTIV3lsQvNR2Gt5P0ThjcO79PPT78W3XG+2N/4JUG6SuWa+qsq6lHqAVnE9ywrZAUcqaCrKs9pxAAwThqXQ/qSWQPB8IKOM9NLk0fW9gbYDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uF7c5zavGjbbV/lkf4o6jybYZmZIylb5HcpXyWKE0tU=;
 b=MYpVIT0k9nyTpewe3mehTh6ec7igXTVVw9ksR4WGvv34qTMEHEMeP/+XdF/NlLNatf/fmihIY2XrgL1U7n70uBbMfgwragZDglnHc7dURCqbxHIDrZMqY15wCIApCgCb/6R2Oda7WTHhJUyA9Qo45db4WsVgYpjzh4L2gIq0IarIigFvCjyaraB3P9xn/IDBxUoLSE01keczMGRp1COSMczcTHLAgANgyGNNBXCfMdZsQ6NaXCasSxZ3IFovE4MOYcTRHYxTcuOyLesl/7EtxWIAiqMOirc29MUpODODqGpp3qawGk81+tKW/XvYXBbNxDPGUtbgtIkYIGhhf8y8hQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uF7c5zavGjbbV/lkf4o6jybYZmZIylb5HcpXyWKE0tU=;
 b=zKO+HHUqN7nTUyl43gWACKGXwoDhtm+2MCYOLBu3jow+IgB5f4mQVLnNRLwG5AGrWeCI/4rA0HPqZOK/GcjVuuc9kAyQ+nEBeIDmiIk8eqNKE5hc1d/C3Pv4eSZ1pQD0fxrmcpkrUyiTISkXMgzAaYqTXJauJ7pnxzwgPdoh9w0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v16 4/5] xen/arm: translate virtual PCI bus topology for guests
Date: Wed, 22 May 2024 18:59:23 -0400
Message-ID: <20240522225927.77398-5-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240522225927.77398-1-stewart.hildebrand@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099DC:EE_|DM4PR12MB5866:EE_
X-MS-Office365-Filtering-Correlation-Id: 2e0fa9ef-0ac7-4e54-32a9-08dc7ab2f337
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?fnZLKv+SGJt4vX2RMtcs9tiwjbyCahC6JTuuxvTAgkdpjbAsShlbm2pV6t9b?=
 =?us-ascii?Q?qcFm0uIXmjIFo3TwKM5+j5IIzoh0GwI4GqmV7QRC0qglEmsNkD0X8KSP6ch4?=
 =?us-ascii?Q?DQkAEC3fiP8RExYNNY0+DoEn1Wb1pwq14DxfHfucR2mKEdccizIgHbETCv5k?=
 =?us-ascii?Q?SoEDe/pL8WXo5VFE/U6ZxUEFZSyDTfkP/yqNe8NApZP1mf0KX6DblgTGdsrt?=
 =?us-ascii?Q?JXRwcTh/CW7IUq3GFwQsKk9I3ldsG258cdjlbsGuGpPof6JHYPfDWDTh65dj?=
 =?us-ascii?Q?7Y1HMEHBA0chpSDmnw3Lh+BSknn/RkER5CTvkmnoXw1CuEvJzjp0Ux9rie9P?=
 =?us-ascii?Q?Nt/lC0qMCfUexD1xj0KiCygIZAzyjgC6L6xeGq/+JneN4jZbdXVfLr8uKWE4?=
 =?us-ascii?Q?Hxr2jjI3eIE6mz4jkSRZvLpwVCI9Op63lx09jP6g8O3cnYG8tBFffrvGMaAt?=
 =?us-ascii?Q?wBMM1Mk2HKg0VlzblV0jpr2CNWQWVOQ4nyYH2JzQcYeG6Z4ynlv90VGfjCE4?=
 =?us-ascii?Q?8F/0nlQLEOOQTlCNDqT9OOzsnSRKO/yNgHm8NKqsLReMHxUU7NfC5qLiHd+V?=
 =?us-ascii?Q?SLfbGipts4gYgJDO/a0eTWaJ6ubeh3KyF1o4ohiBW4ZjNDz89dY30+m4q/ak?=
 =?us-ascii?Q?FfpENwYmT83Gg7CgI1VRfD/uq2KWivyh1OxPgvsHDxWL5b+MEfOS4HW7teIg?=
 =?us-ascii?Q?oPAIx4+ZLA0dTMUsydaYfHnmJ0hz722MYKKEKx/uA9M28UP8snvET5h3HVnG?=
 =?us-ascii?Q?PsWRoE6wtewOavtBBfV6ob7QWydj7VRJBUQODGkTJHDvCsTnCGjip+vJptXu?=
 =?us-ascii?Q?I7tpyqVt7WQiAb8GIKBWdkyY+SB0GapM+AqzXE5WwgNytlK7WnVeXCZeRWh0?=
 =?us-ascii?Q?DTIEIgBV/gaSI3SF8baSKDSWe5x4izWqV293gvR03XLJY8XqRoInioQwd31T?=
 =?us-ascii?Q?B1I/ZRnNdetzpsZpi3TgrnZR5tm2h2j+dXQOshUC1orGdLA7REYAQvqxL3/J?=
 =?us-ascii?Q?gC4w72Mix82EVp5t2MeoT5pRhzVszh4OALCpfYsD5DhaR8egojIuLPmyG/ta?=
 =?us-ascii?Q?gktwVQXX6L2MFiNYze2Nq/hOQBbiC7G7bRrzuFQVtL0o2x842baNPb+I9GYA?=
 =?us-ascii?Q?KqVcYL4F2LqA/BAmGYNAWVD1Nqs5JJNGo4bkJrEMcwQQk65IG7POQs/RG+AY?=
 =?us-ascii?Q?fHzAF0Zk4Rf6Y27farg/sBDSZ8cfv8UddOq0SygLxQ/JhTL2AJdSlAq5aJyr?=
 =?us-ascii?Q?+GBbzWIv78U7ZMPP13RN8S8/r7IOYnXmpcjRxumk/xtRfd6lFDa+faAewO69?=
 =?us-ascii?Q?Em4wCWut1UXKPQxBKGyblQ9S2WbR4ARxc59SiriOMjlSgbNRk1UCC9CvZhpe?=
 =?us-ascii?Q?olZFSEU=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 23:00:18.2078
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e0fa9ef-0ac7-4e54-32a9-08dc7ab2f337
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099DC.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5866

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v15:
- base on top of ("arm/vpci: honor access size when returning an error")
In v11:
- Fixed format issues
- Added ASSERT_UNREACHABLE() to the dummy implementation of
vpci_translate_virtual_device()
- Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
the logic in the function
Since v9:
- Commend about required lock replaced with ASSERT()
- Style fixes
- call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
Since v8:
- locks moved out of vpci_translate_virtual_device()
Since v6:
- add pcidevs locking to vpci_translate_virtual_device
- update wrt to the new locking scheme
Since v5:
- add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
  case to simplify ifdefery
- add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
- reset output register on failed virtual SBDF translation
Since v4:
- indentation fixes
- constify struct domain
- updated commit message
- updates to the new locking scheme (pdev->vpci_lock)
Since v3:
- revisit locking
- move code to vpci.c
Since v2:
 - pass struct domain instead of struct vcpu
 - constify arguments where possible
 - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/arch/arm/vpci.c     | 45 ++++++++++++++++++++++++++++++++---------
 xen/drivers/vpci/vpci.c | 24 ++++++++++++++++++++++
 xen/include/xen/vpci.h  | 12 +++++++++++
 3 files changed, 71 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index b63a356bb4a8..516933bebfb3 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -7,33 +7,53 @@
 
 #include <asm/mmio.h>
 
-static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
-                                     paddr_t gpa)
+static bool vpci_sbdf_from_gpa(struct domain *d,
+                               const struct pci_host_bridge *bridge,
+                               paddr_t gpa, pci_sbdf_t *sbdf)
 {
-    pci_sbdf_t sbdf;
+    bool translated = true;
+
+    ASSERT(sbdf);
 
     if ( bridge )
     {
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
-        sbdf.seg = bridge->segment;
-        sbdf.bus += bridge->cfg->busn_start;
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
+        sbdf->seg = bridge->segment;
+        sbdf->bus += bridge->cfg->busn_start;
     }
     else
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+    {
+        /*
+         * For the passed through devices we need to map their virtual SBDF
+         * to the physical PCI device being passed through.
+         */
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+        read_lock(&d->pci_lock);
+        translated = vpci_translate_virtual_device(d, sbdf);
+        read_unlock(&d->pci_lock);
+    }
 
-    return sbdf;
+    return translated;
 }
 
 static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
                           register_t *r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
     const unsigned int access_size = (1U << info->dabt.size) * 8;
     const register_t invalid = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+    {
+        *r = invalid;
+        return 1;
+    }
+
     if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa),
                         1U << info->dabt.size, &data) )
     {
@@ -50,7 +70,12 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
                            register_t r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
+
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+        return 1;
 
     return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
                            1U << info->dabt.size, r);
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 1e6aa5d799b9..86a89adb63b8 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -81,6 +81,30 @@ static int assign_virtual_sbdf(struct pci_dev *pdev)
     return 0;
 }
 
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
+{
+    const struct pci_dev *pdev;
+
+    ASSERT(!is_hardware_domain(d));
+    ASSERT(rw_is_locked(&d->pci_lock));
+
+    for_each_pdev ( d, pdev )
+    {
+        if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) )
+        {
+            /* Replace guest SBDF with the physical one. */
+            *sbdf = pdev->sbdf;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
 void vpci_deassign_device(struct pci_dev *pdev)
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index da8d0f41e6f4..533a319406b1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -304,6 +304,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v)
 }
 #endif
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
+#else
+static inline bool vpci_translate_virtual_device(const struct domain *d,
+                                                 pci_sbdf_t *sbdf)
+{
+    ASSERT_UNREACHABLE();
+
+    return false;
+}
+#endif
+
 #endif
 
 /*
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:00:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:00:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727922.1132640 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwq-00022O-0b; Wed, 22 May 2024 23:00:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727922.1132640; Wed, 22 May 2024 23:00:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9uwp-00022H-TT; Wed, 22 May 2024 23:00:39 +0000
Received: by outflank-mailman (input) for mailman id 727922;
 Wed, 22 May 2024 23:00:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9uwo-0007HJ-Uu
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 23:00:38 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 18087c7d-188f-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 01:00:38 +0200 (CEST)
Received: from DM6PR03CA0057.namprd03.prod.outlook.com (2603:10b6:5:100::34)
 by SA0PR12MB7461.namprd12.prod.outlook.com (2603:10b6:806:24b::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 23:00:31 +0000
Received: from DS3PEPF000099E2.namprd04.prod.outlook.com
 (2603:10b6:5:100:cafe::59) by DM6PR03CA0057.outlook.office365.com
 (2603:10b6:5:100::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 23:00:31 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF000099E2.mail.protection.outlook.com (10.167.17.201) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 23:00:31 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:00:30 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:00:30 -0500
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 18:00:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18087c7d-188f-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gd2Ybzmvg/1MIYTk7SOkvkxrXKIFu4+2/cgqFm2VEqs/WTypJ+XGibpbbeoFzTVLCUtaCbbICcEmm0l2V45kOYn3saYaWG+MQAFMcIgvrHRom8ZY98QXG3+CV9ES1dyvqYBy5mpUzfTvf3Wuxui7bLmKy0g2sQRWdgQmJjWO2ruGpef96OYVAcUGtxx2Nhib55HKDM5u3wkH9XmntzEKQkQcMp87CGrwY1SszGRmqh1IlZERfCEkaWjjeVf6Jh0jN9OkY6M1XnxgpBdMRdu7gRQc59sKwRM/qzlyJSxDvQp+TFhm7Uv8HwPrY9tVVAWGfUG5bNDjRxJrjLDXSmdJBQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wg1lIYFsxCkrbMSdoPQQCZQ7ozFKfKSY7zA6PHJCXCc=;
 b=Ei4v0t+5iSUYlVRq4f0kyy5WD+V7a2sl6gJrvwnVSW+je3AGLIkbIbYnzJ/qRidXf+8ETA9R2dUibX81jsyyrWtJzyVUoNYQrUTjU+z+hecz30XWA5d0AShgVeIYkkkpN3rWSNsIJ+Oy4KJkg9I1nWqwZEZVhby4G9Jjo3YDywHJ+ewDDOCHe7eihqc14AGksxoz9WwZwJeprjJKxgetNcbB+/w67SgqQjhy2GBeSSQ219riyjTDvAE2WOoMND8itqJ8rI/tzb39fFhCNviryHBKlJArNkI/UFJlRUNnOozWQVMxXuU3VhFkaL/d9bNGk/yOOpIWcoUcBzKynyF4bw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wg1lIYFsxCkrbMSdoPQQCZQ7ozFKfKSY7zA6PHJCXCc=;
 b=tlY1/bVaMY4/LpD5YxM+o85FZB1KQroOYHN/F3Bq7bi6WhPtwspq++/Cr3FSJP0anZWmjhxRZRcbaB7w3iabJHDkCJFd4KqeBcChiOPqO8zSKZpsvsGE2ZwOY71t8WVYuhw3UFON6w13FratC0+n+QVO19pU9NGm18tGf1r4eCw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Julien Grall <jgrall@amazon.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: [PATCH v16 5/5] xen/arm: account IO handlers for emulated PCI MSI-X
Date: Wed, 22 May 2024 18:59:24 -0400
Message-ID: <20240522225927.77398-6-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.45.1
In-Reply-To: <20240522225927.77398-1-stewart.hildebrand@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099E2:EE_|SA0PR12MB7461:EE_
X-MS-Office365-Filtering-Correlation-Id: 3400ac04-c933-4df0-d5d3-08dc7ab2fafe
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?TLhkC9k/W5n731QommvrKtoFE5n2rWV9SKXCaE6U3kQE/0LjRzWJy2Wy+DYm?=
 =?us-ascii?Q?kfnHEhe5g6m/9h5rpeI9jesP+07czC74A/ruFqHrVcRQX7G0Ovb3mwooVg67?=
 =?us-ascii?Q?UbbbGHN11eQBgpH2ju3QM3l61xoXsqxNzdULEkEBtDHTj9JirczEdNHbvTuG?=
 =?us-ascii?Q?pakw53v3D21X1J921dZCpybK9QmSyZwov2+FA59zCCOZJ3SDXdg1ffa+QHo3?=
 =?us-ascii?Q?uGJqzF9O/aUEsBdcBHHEeVPzjp/yfpIOVoWmSqLHudN5UiQVYRQ9BAtDQAY5?=
 =?us-ascii?Q?T7k4EuOG24gSUqUqQ8jBn1UJX5bWSyYzGuOsOUnwQ+M38aSF87gI11fVnESv?=
 =?us-ascii?Q?mgnuiGV9+MMNkVrayt+eDcsmpYHTvHUTPlaunNoOnDU9rGZLo6T1+6F0rt4G?=
 =?us-ascii?Q?yOk4DLz0gDh2bzKVvLrMNnsQajhrWbF0QKhUdqZNvaKa1Eeb5WWiuFJSAtWJ?=
 =?us-ascii?Q?GqKpxxeigLSaR76mPZmH0cHGukHp+CUzfWyqTjqmzijZwxQStjW71EMHVePZ?=
 =?us-ascii?Q?a1aIgjutX05G/OPsxYUZs9O0YdC3D7OYMN0aJdA+FMGhfBuV9i+0Cn91ICzF?=
 =?us-ascii?Q?b2ZAUbHFAiE6j77OsUtI/ex3rvIiog2blcsLuAecvho79XQv1QugGuVsAgzd?=
 =?us-ascii?Q?1LRmqHr59MDu4YeIJnm1sa4cFPv6MJDrAsUQNax8viXtWg5UaX5YTbQKxSto?=
 =?us-ascii?Q?oA7AH4Z5H8ny/gok3gzZmyYaWCcWEQ2dMeMNSmLyglXevlVkXoNrEj+UZ2Es?=
 =?us-ascii?Q?LTF8solADg6tmtr5/NtoF3z65rRNv7o5p5ZwXORiUVucroU034h2i6+Z2O0v?=
 =?us-ascii?Q?gc7ayVVbBbeGMRlF/+76VVbKTbWghVowyZsL7ANcuidkvlsI/cBwfgjeW8Jv?=
 =?us-ascii?Q?A57+tWgsFKlBwQx9G8ObJ2zVvVAf2eTPuNNmZ5rORi4YaG9yZZQFIMYrv8NN?=
 =?us-ascii?Q?XtYqwuEC9YQCdtHioox6wtcy4Wh5Ka3x7eB+Y6BVPwosu7gEOsLNenqp2JNi?=
 =?us-ascii?Q?vFqWv5MWe/VUgg6hc/v6GBQbNK6pKt2wLwFJT9NJizmabMP7NvJ/ZIsT+/4W?=
 =?us-ascii?Q?Vu55rAP1otbaEItcwLvjYIwCUqge+cuJgRfFnTGyPjEnD9fZ8GDEYFzubBMI?=
 =?us-ascii?Q?LouvOwQPQ5ltR0fi10xlMC3/Jvp2MUKaLKixde4KhzxVtjw+UFFpDikzlp8/?=
 =?us-ascii?Q?sib7aZfWJh2/r6JAEKLjts9EnbNDgL25Z+ieg3LN0yd3E5hB0t1De32Lm4go?=
 =?us-ascii?Q?viXrt0BgL5o9+0zqbTTKaDuWpOSjpLaBq7t++kKRUMp+LmEYxoRxgX87uBuT?=
 =?us-ascii?Q?qwPjTY/bMKHVojk72gqlV5ueO4Pe/mfBnncWMRGWvQMBZatPLN55P9V+IcNQ?=
 =?us-ascii?Q?qVD6i6bEe6HRXyLRkZ6xlprHd7tf?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 23:00:31.2602
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3400ac04-c933-4df0-d5d3-08dc7ab2fafe
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099E2.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB7461

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

At the moment, we always allocate an extra 16 slots for IO handlers
(see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated
MSI-X registers we need to explicitly tell that we have additional IO
handlers, so those are accounted.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
This depends on a constant defined in ("vpci: add initial support for
virtual PCI bus topology"), so cannot be committed without the
dependency.

Since v5:
- optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is
  defined unconditionally
New in v5
---
 xen/arch/arm/vpci.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 516933bebfb3..4779bbfa9be3 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -132,6 +132,8 @@ static int vpci_get_num_handlers_cb(struct domain *d,
 
 unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
+    unsigned int count;
+
     if ( !has_vpci(d) )
         return 0;
 
@@ -152,7 +154,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
      * For guests each host bridge requires one region to cover the
      * configuration space. At the moment, we only expose a single host bridge.
      */
-    return 1;
+    count = 1;
+
+    /*
+     * There's a single MSI-X MMIO handler that deals with both PBA
+     * and MSI-X tables per each PCI device being passed through.
+     * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV.
+     */
+    if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) )
+        count += VPCI_MAX_VIRT_DEV;
+
+    return count;
 }
 
 /*
-- 
2.45.1



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:05:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:05:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727943.1132656 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9v10-0003Fy-LP; Wed, 22 May 2024 23:04:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727943.1132656; Wed, 22 May 2024 23:04:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9v10-0003Fr-Hp; Wed, 22 May 2024 23:04:58 +0000
Received: by outflank-mailman (input) for mailman id 727943;
 Wed, 22 May 2024 23:04:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=31Ro=MZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1s9v0y-0003Fk-EN
 for xen-devel@lists.xenproject.org; Wed, 22 May 2024 23:04:56 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b2f73331-188f-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 01:04:54 +0200 (CEST)
Received: from BL0PR1501CA0001.namprd15.prod.outlook.com
 (2603:10b6:207:17::14) by DS7PR12MB6240.namprd12.prod.outlook.com
 (2603:10b6:8:94::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Wed, 22 May
 2024 23:04:51 +0000
Received: from BN2PEPF000044A5.namprd04.prod.outlook.com
 (2603:10b6:207:17:cafe::26) by BL0PR1501CA0001.outlook.office365.com
 (2603:10b6:207:17::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Wed, 22 May 2024 23:04:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF000044A5.mail.protection.outlook.com (10.167.243.104) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Wed, 22 May 2024 23:04:50 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:04:49 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 18:04:49 -0500
Received: from [172.30.116.145] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 18:04:48 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2f73331-188f-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ItF2QukBqKduBfq5POk2PKKhD/mdapQ2ocl+HtQ7ymO0PBrNpSdgUcyFByEtATmCtOGKJDvaWh8cOlxrUf/BOcbshR1rC3bd6laeuhkk3jHZWNO2kS9qgMSFUag1rBHrP/5aXLYFMRWsYlIV0NhG3tv65B8pWF58LYCyz/EL6UMcyv2Yf7RmgMubvo/h0YD0IpAhkdunJFaH7GNmoiSIkvFnN8nRfcnHetFr2zencNAzdc3U1BrXWJ68LM6ygOu9X1bwf4hN3fcSxPqf3t+WU6wWHrPy2i8Du90iRWHhD9tJAOcUjYwI/WhxPZGXGljHTb82+DIPN6S1czYm4AlEdA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=bsUpyP7kn8eJLu3oRss/4HmR2ID1rVQETHF+4Cg0ngM=;
 b=P43QVttNmWrJtAYcft59UTzzubcoirQvHunvU/I7kY26475AGWUMiRheUuStv9SzOIV3jIdrrO2jPUhylWXYLr3dQv3isid9XLN5f/ObifxoKAXCFxVAlBz7MfJ1BvnOnx9IlbGYCqn9QUdzzRgOB8gHqFE+7mq8Nq5pi6eBnONED6mCTFT88Sb25hPP0lFQJNJRSeAXyIWkOX9VV+l+mv0tACBFnmh1B3bYZ8Qs1vE6aQiqpaqdyIeY4CuHpgr7d9E6tddUX9XiH583DyOtfjkXWO/DxIT5AkH+9XkFWQkVs+/38u9JKXrWWxhTNBr9fX8bBYfLGLuAxMXPL3Mciw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bsUpyP7kn8eJLu3oRss/4HmR2ID1rVQETHF+4Cg0ngM=;
 b=YZvObtKJ6VucK0PzBJvJyYEvenI6ugR1ilrjZZmDsdz9V2LjMGuOVPOY+83iPioCgeuX+afxuRicl8jdk9aUNq9DMzF9B3jguBkGyNHev9jHYDYoiTOrW1udwPLogPWEu4f8LUZ/jgRQgwv6T3E2uNnf8LTjiqS6QAnxeI7KaJE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <7072184a-20bd-4ce4-956e-c7c424eb097e@amd.com>
Date: Wed, 22 May 2024 19:04:47 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v16 1/5] arm/vpci: honor access size when returning an
 error
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-2-stewart.hildebrand@amd.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <20240522225927.77398-2-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A5:EE_|DS7PR12MB6240:EE_
X-MS-Office365-Filtering-Correlation-Id: f1d1e53f-6abb-43ec-1c9f-08dc7ab39580
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?YmlFZXVFRFhET3JIdC9RSHFrOWV0b0hKWDNlTEhoSnFPVTJqdE45UFV3M2xQ?=
 =?utf-8?B?Yk5jQU96bXRDVHR3cHFYMFE1Rks2SlBZWlVET1I3K25OaWtkM2dTejk2QkFl?=
 =?utf-8?B?S1FxUUMxQkpiRWxnMkFxRVNpUkpMVWt0eUcxcUJ6Yi9RSHR1TFZwdUxNMHZq?=
 =?utf-8?B?aUVSMjN3UzJzcVd6UWM0OWRnOHhrM0k5VFBEMzJTZXJtdE5rWXBXN2t5S0E3?=
 =?utf-8?B?dHBxd1hNKzlRS1h3eUV0dW56YzlqbTZRd0c4aC9yZ1dBVGxnVGo1cjJzZDEy?=
 =?utf-8?B?Q3I3OFc5SklpMGZsQUlSSkM2cDB0SVZHNXkzSDM4ZktONklFNXBGKzJRcXFU?=
 =?utf-8?B?K3gvZUhTS2RIcEJHRkxWN25wS1FYN2JJeVJLUTRaUU5NTllMR3VzVTYzTy9z?=
 =?utf-8?B?b0F6VGcxNmtvMVp2K3ZSUGZpeTMrVE5KTjR6aTYxMmh4bHNEVWdnaGVRa0lq?=
 =?utf-8?B?MWIwZEhmZEEzd1hYTm9uaHl0bnR3d2NvZGJySDlNV0hIbFpBN3JDVEMyaHdo?=
 =?utf-8?B?YXF2KzI5ZDdNRkdreXRHVjhsZ0hrcXA0dlFWTFVSSHg5U3o5YjU3dUtWZm1Q?=
 =?utf-8?B?cVdCWTlSWFN5Smo2Mm9xWkZjczgwMVBldUxBTmkvVnFncDJTd3NqT1N4ZGIx?=
 =?utf-8?B?Rm9HS0ZwOFUzY3ZTUysrL2pDdkVvZVZ3SzhWWkgrMTVkTXZSQ2pCUm1KUDQw?=
 =?utf-8?B?cmZqN0lseVFQNlNCU0tJNXF2ZnowYU5tTkF6b2RQd0NXcVhSU1VaemJUQnB2?=
 =?utf-8?B?WlE3NU5qZnpWNE8wbm9EQ1J3aGNCNHUyUHFSL2dlSmIvVzkxQzYxZHFPNkti?=
 =?utf-8?B?TXRqb3ZrVTRWTDFOMzNmYk9FYUltekdaVFRUN1NGY050ZmVYWC9kY0syZ2Fy?=
 =?utf-8?B?ZFZsZ3N0ajFTYzRDZW0ranNIVEJDWDQ3bkQwRlFjSFN4SGFialNEVEVTSHBq?=
 =?utf-8?B?bnBZd2NlWTU3Q1hJcVFQM0d2MFJpTVBxZWJoZ0NWR2J0UU93QnhpeVBYeGt5?=
 =?utf-8?B?dDJpK09RQzZNTGFBbnc3aFRLMklPc25kWTVpQWd0NHh3L2JRZ2lmR3diS3lj?=
 =?utf-8?B?M3AyK29naitrRHgxSWx4MjZkU0piU1o4MGtYMEYwNU52UDhlL2psYXB2QUVY?=
 =?utf-8?B?dUF3bVF6SlFZbkZnYUd2WkVOOWozaTFjQ1BRZ3NTVHpvU3ZERUpHNlhxRHh6?=
 =?utf-8?B?NkxYYjlsSVo4S2ZIcC9DUEJmMUgwbmxjc3lBTXNJSDRxUkQ0M01ydkVrSTV4?=
 =?utf-8?B?NUlCODBrS1dGR0FPL094dlAzNk04YTlPbjBwbEtzb0hvNEpCNlRsUGEvRGxX?=
 =?utf-8?B?Z3BTWmNxeXd1Q2dkNnZWUWg2dy9aY1FXMUJIbHdRQVRUUHpkL08wNVhwWnB0?=
 =?utf-8?B?MEtpdHBadWhHYmFXd3JyREpiSzZ3bWJYOWE0RldxVTl3dm55VlppREhHV0pu?=
 =?utf-8?B?dFFUUmJmaEZaWmppckhucTROS3RJZ1JNRURYMzQva08zTEpQTVZ4aXdiVXlX?=
 =?utf-8?B?MGgxcVAxKzRjSE4vcmwrREE4SVhsYUJMN0txakJWQkt5WWFMTXBlNHk0d3ZC?=
 =?utf-8?B?dkVmM3kxOU5ZT2YzOE1iQ1Mrb0k3N3lsUFFQK2JVb0orL1dIdW1zUmdiampB?=
 =?utf-8?B?U0xpU3E0SkVGMlhkckhWMTlibkF5UjJuZXhMSitkUk5Jc3Y1Sks4Z1hjNkJ6?=
 =?utf-8?B?amF4aU9lL3J2ek11V1V0NUhpQThZSHFXeVhYYWFOdG41MlBnaGowclJULzdw?=
 =?utf-8?Q?Gy3Z72v6L7ucB34mzhmHpM/UNCy63ilQiuPaCCP?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 23:04:50.5234
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f1d1e53f-6abb-43ec-1c9f-08dc7ab39580
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A5.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6240

On 5/22/24 18:59, Stewart Hildebrand wrote:
> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> 
> Guest can try to read config space using different access sizes: 8,
> 16, 32, 64 bits. We need to take this into account when we are
> returning an error back to MMIO handler, otherwise it is possible to
> provide more data than requested: i.e. guest issues LDRB instruction
> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
> register.
> 
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

I forgot to pick up Julien's ack [0].

[0] https://lore.kernel.org/xen-devel/8fa02e06-d8dc-4e73-a58e-e4d84b090ea8@xen.org/

> ---
> v14->v15:
> * re-order so this patch comes before ("xen/arm: translate virtual PCI
>   bus topology for guests")
> * s/access_mask/invalid/
> * add U suffix to 1
> * s/uint8_t/unsigned int/
> * s/uint64_t/register_t/
> * although Julien gave an Acked-by on v14, I omitted it due to the
>   changes made in v15
> 
> v9->10:
> * New patch in v10.
> ---
>  xen/arch/arm/vpci.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 3bc4bb55082a..b63a356bb4a8 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -29,6 +29,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>  {
>      struct pci_host_bridge *bridge = p;
>      pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> +    const unsigned int access_size = (1U << info->dabt.size) * 8;
> +    const register_t invalid = GENMASK_ULL(access_size - 1, 0);
>      /* data is needed to prevent a pointer cast on 32bit */
>      unsigned long data;
>  
> @@ -39,7 +41,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>          return 1;
>      }
>  
> -    *r = ~0ul;
> +    *r = invalid;
>  
>      return 0;
>  }



From xen-devel-bounces@lists.xenproject.org Wed May 22 23:26:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 May 2024 23:26:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727957.1132666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9vLN-0006uY-7E; Wed, 22 May 2024 23:26:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727957.1132666; Wed, 22 May 2024 23:26:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9vLN-0006uR-4d; Wed, 22 May 2024 23:26:01 +0000
Received: by outflank-mailman (input) for mailman id 727957;
 Wed, 22 May 2024 23:26:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vLM-0006uH-Df; Wed, 22 May 2024 23:26:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vLM-0007sA-CZ; Wed, 22 May 2024 23:26:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vLM-0005oF-1z; Wed, 22 May 2024 23:26:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vLM-00069h-1d; Wed, 22 May 2024 23:26:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mILyFumYBhxYW20l9Gt+Uz9wIM8/XumIGWwmLvi0PTs=; b=MBEr+QG0TBC/FIUyFUSQT9QROo
	E2VUt94CrYIgkzxYFTFMFQPbouID03Pf7uR/aJuEOkiyVkcx2TIp07bzHJlIqGFgcBhr7Zv9DkLt2
	2rtV07FwhdVxhy5T6JOqR8+UOaRA4/llQBXv/k0nX3eanU8cE8kX5tYOqFa2KumVHrgI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186070-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186070: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129
X-Osstest-Versions-That:
    libvirt=7c8e606b64c73ca56d7134cb16d01257f39c53ef
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 May 2024 23:26:00 +0000

flight 186070 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186070/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186057
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129
baseline version:
 libvirt              7c8e606b64c73ca56d7134cb16d01257f39c53ef

Last test of basis   186057  2024-05-21 04:18:53 Z    1 days
Testing same since   186070  2024-05-22 04:20:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Han Han <hhan@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   7c8e606b64..7dda4a03ac  7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 23 00:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 00:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727969.1132675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9vuT-0005Cy-Rz; Thu, 23 May 2024 00:02:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727969.1132675; Thu, 23 May 2024 00:02:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9vuT-0005Cr-PJ; Thu, 23 May 2024 00:02:17 +0000
Received: by outflank-mailman (input) for mailman id 727969;
 Thu, 23 May 2024 00:02:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vuR-0005Ch-Tj; Thu, 23 May 2024 00:02:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vuR-0000ge-NU; Thu, 23 May 2024 00:02:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vuR-0006cg-Br; Thu, 23 May 2024 00:02:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1s9vuR-0007X3-BW; Thu, 23 May 2024 00:02:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Gn4Q8uNeXCUGIYSPqjPIUT3GF3r5WBpMZ7+d2TR2yXc=; b=BKaHHAiY/b0KcNgO79jvnoh9SZ
	anm6QY9Esql9vEvcYSNMXj5KBNq2lIy54yyNsJ7lyLUUUAsnQfQW7en53QJNI06JxtqhMFswdgzEw
	ywUNzyUqNl2JKIlipppKnnN3OFJptlzMs5IMgeqbJl1rzWXezC13Z8yybWmLkp9c5rXU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186069-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186069: regressions - FAIL
X-Osstest-Failures:
    xen-4.17-testing:build-i386-xsm:xen-build:fail:regression
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 00:02:15 +0000

flight 186069 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186069/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-xsm                6 xen-build                fail REGR. vs. 185864

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185864
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   23 days
Failing since        186063  2024-05-21 10:06:36 Z    1 days    2 attempts
Testing same since   186069  2024-05-22 01:58:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 438 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 00:51:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 00:51:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727984.1132692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9wgF-0003A6-HS; Thu, 23 May 2024 00:51:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727984.1132692; Thu, 23 May 2024 00:51:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9wgF-00039z-Ep; Thu, 23 May 2024 00:51:39 +0000
Received: by outflank-mailman (input) for mailman id 727984;
 Thu, 23 May 2024 00:51:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1s9wgE-00039t-SF
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 00:51:38 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9ab9a44d-189e-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 02:51:36 +0200 (CEST)
Received: from SJ0PR05CA0118.namprd05.prod.outlook.com (2603:10b6:a03:334::33)
 by CH3PR12MB7668.namprd12.prod.outlook.com (2603:10b6:610:14d::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Thu, 23 May
 2024 00:51:29 +0000
Received: from MWH0EPF000971E4.namprd02.prod.outlook.com
 (2603:10b6:a03:334:cafe::f7) by SJ0PR05CA0118.outlook.office365.com
 (2603:10b6:a03:334::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Thu, 23 May 2024 00:51:29 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MWH0EPF000971E4.mail.protection.outlook.com (10.167.243.72) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 00:51:28 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 19:51:27 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 22 May
 2024 19:51:27 -0500
Received: from [10.69.48.49] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 22 May 2024 19:51:26 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ab9a44d-189e-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SiXRzlYkyZ/xVGlWZF1TqLMbu6ONRREKiwFEqEgJaGQfgknbxVg0bhq22caiUqJZHnhw01stDo2XrJQYQdgCETpGgQp06xtIWlvdA/NTZFsRKxx1gWlck9Tf5lBf3o14DYyLBnAkcT0XGtJPHq1f2Df5byMIiidQnhlpDjfwkxrMtQqSnnTMvfWE4LcEqteUDrIaNY003nKwKEqbE9iWIsEtzjSbMNqLZWvq7TsmGTaDIC7dcp6fboaSbsRez5Cw053K34WLn7VxdRq2mzFUWXTmS0X21o9ty3rl0cUks0Ob6MDZH9wSALO5rIOu/toxf8zgzEg311zZGtnJ5mUqAw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=tqzL2ILy27IRr3fbs6qUU0Gh1DeGGGjERPmLLBR117w=;
 b=PO51PbfRDWSRDY3fyBngnTzQMjpKWbDxbzlb0y0eiZSgrgSjlgkMm0Gw0Ltp2k4lqhFwwkvFkxBVUge0LCoCB5TwHe2tMT8+EOmXjbwT5ZAEbw9ck0bDVAtmu8pl5GWs42Sy1oVFfM7G+Uk1fCAJt3Xt0qvTnXkNhBFINWCbJ2rIH9UaHiJ0Q11P7+NnSj+aYP+B5WH37P0AqAhUvsLluk1Rnjkixs+pXVndBTMU7h+75SLDQa8wdLvIWURW1a/UBbn0WUAa+kN/H44llPRYchL+9a180GY6fQvNoZ9uuxpN93NTFOG+hWTmmZSibM+t9EE9deXlVgyhajDLzhypVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tqzL2ILy27IRr3fbs6qUU0Gh1DeGGGjERPmLLBR117w=;
 b=chHgyXRI1JaXoqzkq2TIEAikb3vI0/Bkfz/p8OtB+CMD+QNa1cYgqPeAbT/57dtnunS6O3EZkaVgHjhSOSZfbCpWBYCeqP5vjfWiUxrSlq5RUsGLYO0ZlR98DFjYtYJxv46lGeZA1KfYoJxTYeCF9iAHZqJ2NJm4DuluzV0CSzo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e743d3d2-5884-4e55-8627-85985ba339f9@amd.com>
Date: Thu, 23 May 2024 08:51:20 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to
 running VMs
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240521043557.1580753-1-xin.wang2@amd.com>
 <20240521043557.1580753-6-xin.wang2@amd.com>
 <3f6fa7e7-c02c-4fa5-b4aa-5e4c2efed65f@xen.org>
 <4f6a24b5-5e7b-41e0-9314-496bbcaa8888@amd.com>
 <alpine.DEB.2.22.394.2405211815040.1052252@ubuntu-linux-20-04-desktop>
 <29073700-673d-40aa-8759-efc60af3b7c0@amd.com>
 <1754e63c-00a5-4d66-b151-8fa436f5c844@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <1754e63c-00a5-4d66-b151-8fa436f5c844@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E4:EE_|CH3PR12MB7668:EE_
X-MS-Office365-Filtering-Correlation-Id: 569c1b7e-0172-493c-a948-08dc7ac27b19
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?OWcyTklOVCtEcHREaG9PNlk3clFzY0Z5WW1pZHd5QjMyank3M1R1aW1SME5C?=
 =?utf-8?B?dE9QUjRLZkw2a3JqRUl1YnFGa2M0S25nRVEvT0IyeE5XdW44VXB2N2hTVmJN?=
 =?utf-8?B?VzE1TVo3bEZDcnY2RTNMYnVoRFZYaUtRSWcxYXZ2bTlQYW81K0d3d1RLd1Nw?=
 =?utf-8?B?UG96K0l0bEJ3RnY3ekZPL05JMlNWYVdBcGJPY1lIbGs2aWtUNHdNZkFKbjB0?=
 =?utf-8?B?QUljNGRwdUFZWFpoNENSSXZZeTNVU3ArbHdWekFvcUN2ZjBLWmNTeEVhK0gx?=
 =?utf-8?B?c2FoME9TckxzRnk3MksyY0hkRGxCZHhpVEFYMVJmWjAyZkJPR2djc3U0WHJM?=
 =?utf-8?B?UWpDQ0N3cjBzNWJwa25yNzIwSW4zODBhZTBLQ3BKbkxReXBQSWx5ZEtycU9S?=
 =?utf-8?B?dXA0bkJ6K2dHYWQ5TUQyeTNURjJrN1V3Ky90MnppMTZ0RDZFNWxvTlZKeitM?=
 =?utf-8?B?OEJHWW1KQUlZaGtoK0M0Z1VyTHFFK2RvZERTVnJxaVBsQjI1RW5sQmUyR09y?=
 =?utf-8?B?T2EwUkJ4Zm5EVjdrVXFJOVVwdVhZYXRjTU96NDVZTytJbTE5eEwyZVlRSWRq?=
 =?utf-8?B?bUVNVkNBNU5TQ1FmZkZEelQ4WG15ZlVhWnU2aVNYWlN6MEc0eXZaVVZCaTE2?=
 =?utf-8?B?dkFLM0p5SzdXZVd2T0lhbmcvRTVGM1UxUnJDbTNWblhPN0pDYnFMdXB5WElN?=
 =?utf-8?B?enFqY0x3R0x5bGRmMDlBQkpRblN6eFBKVDNnZEJXV29mbnJHRU1UUWluU25U?=
 =?utf-8?B?Vi92TFJRWmpPZjBPdTFyWG9GUUhQNjNyQ1VmSWp5UGIvTy9NTVlJYnNyWkFP?=
 =?utf-8?B?WXJBMWttS21IVnFKbFM3Z1Z6T2lwaHJzaXFKRDUxMTZ2b1JacVJ5SU1HWFdn?=
 =?utf-8?B?V2pnckJRbExweDNiU1IvYlI2OTJJSWpVYkR0K1NId0FOcGkxeGRFeTQ5T3py?=
 =?utf-8?B?S2VzcUNacThKK0oxekVSUkRnNmNxUHNmV1ovUGF1bzc3U1JYaGE5Ym1XT2I0?=
 =?utf-8?B?U2dPbDVvYjgwVm5JMkRlanBKdytrRG02TnFTUkNHSUxjQ0REWjRUdTVPSm8r?=
 =?utf-8?B?YlpOVWRsVTJTdGptVkJNdkVaRncvTkJGbjUwLzFpSWtEY3RUODYzaVhkTnBt?=
 =?utf-8?B?T01NWjBiN2laakgrWUgzVkFReCt1T0NLSkhNSUY5SE4rYWp6WitJTmFLTFJM?=
 =?utf-8?B?ZHMrdzk1dUJQSjR0UmVrSmZiT256THBnMHQ0aVZ6M3drVEV3YlJwYlUvTVhm?=
 =?utf-8?B?R213TG9uWm9oczRwS3pnUzlZVzdXdXZjYTMxWGtmWUk0eG9Tc3FFRytqYnpP?=
 =?utf-8?B?bmIzdlFIL3htWmVzSi9NaFo1aysrL3AxZ0hqVWNrTk9rcFNuekRLNmtiajFX?=
 =?utf-8?B?VUZuWnRzbFRMcFJYMk5RM05wU2VzYWlXM1poRXhrWmh4eEJrZTVTOGY3anh6?=
 =?utf-8?B?SHFjQWYzOVVrNmFwOHZJVVc0dVdaZXVPQzFpcXNaZnFVcWZzcUx2eXBoRmdm?=
 =?utf-8?B?UGd6aThJbC9yc2VXN21oWUlPOGpmejY3eHRoTGRrM1FvelB2TFkvR3BMMW45?=
 =?utf-8?B?Mm1ZdXRCUjhSWS85d0dVbjI2U1oxQkEyYzBMV09vZnRlUlVSbnh3YVVsaHZR?=
 =?utf-8?B?YnpsR0YxUGU2dFlIdDkyUjFidlFOeG0rMFdFbWVrQk1PZTBIc00vZU52aWhH?=
 =?utf-8?B?QlFFY0lndURXV3prMHZkWjR1NTE2b2FSdXFSZ2x6anh0dGRnMElSaVRPb3R3?=
 =?utf-8?B?ZFJ5NldCbzFOZTdRdkErVWE1L29sM096clFveXFEM04vMW5QaFFKaWZSYkZD?=
 =?utf-8?Q?/vabD+Cv1Oizlmsb6qGkHwQlh9avY+CjYgzDA=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 00:51:28.5728
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 569c1b7e-0172-493c-a948-08dc7ac27b19
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E4.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7668

Hi Julien, Stefano,

On 5/22/2024 9:03 PM, Julien Grall wrote:
> Hi Henry,
>
> On 22/05/2024 02:22, Henry Wang wrote:
>>>>> Also, while looking at the locking, I noticed that we are not 
>>>>> doing anything
>>>>> with GIC_IRQ_GUEST_MIGRATING. In gic_update_one_lr(), we seem to 
>>>>> assume that
>>>>> if the flag is set, then p->desc cannot be NULL.
>>>>>
>>>>> Can we reach vgic_connect_hw_irq() with the flag set?
>>>> I think even from the perspective of making the code extra safe, we 
>>>> should
>>>> also check GIC_IRQ_GUEST_MIGRATING as the LR is allocated for this 
>>>> case. I
>>>> will also add the check of GIC_IRQ_GUEST_MIGRATING here.
>>> Yes. I think it might be easier to check for GIC_IRQ_GUEST_MIGRATING
>>> early and return error immediately in that case. Otherwise, we can
>>> continue and take spin_lock(&v_target->arch.vgic.lock) because no
>>> migration is in progress
>>
>> Ok, this makes sense to me, I will add
>>
>>      if( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>>      {
>>          vgic_unlock_rank(v_target, rank, flags);
>>          return -EBUSY;
>>      }
>>
>> right after taking the vgic rank lock.

Summary of our yesterday's discussion on Matrix:
For the split of patch mentioned in...

> I think that would be ok. I have to admit, I am still a bit wary about 
> allowing to remove interrupts when the domain is running.
>
> I am less concerned about the add part. Do you need the remove part 
> now? If not, I would suggest to split in two so we can get the most of 
> this series merged for 4.19 and continue to deal with the remove path 
> in the background.

...here, I will do that in the next version.

> I will answer here to the other reply:
>
> > I don't think so, if I am not mistaken, no LR will be allocated with 
> other flags set.
>
> I wasn't necessarily thinking about the LR allocation. I was more 
> thinking whether there are any flags that could still be set.
>
> IOW, will the vIRQ like new once vgic_connect_hw_irq() is succesful?
>
> Also, while looking at the flags, I noticed we clear _IRQ_INPROGRESS 
> before vgic_connect_hw_irq(). Shouldn't we only clear *after*?

This is a good catch, with the logic of vgic_connect_hw_irq() extended 
to reject the invalid cases, it is indeed safer to clear the 
_IRQ_INPROGRESS  after the successful vgic_connect_hw_irq(). I will move 
it after.

> This brings to another question. You don't special case a dying 
> domain. If the domain is crashing, wouldn't this mean it wouldn't be 
> possible to destroy it?

Another good point, thanks. I will try to make a special case of the 
dying domain.

Kind regards,
Henry


>
> Cheers,
>



From xen-devel-bounces@lists.xenproject.org Thu May 23 01:22:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 01:22:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.727993.1132701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9xAB-0005yc-UJ; Thu, 23 May 2024 01:22:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 727993.1132701; Thu, 23 May 2024 01:22:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9xAB-0005yV-Rf; Thu, 23 May 2024 01:22:35 +0000
Received: by outflank-mailman (input) for mailman id 727993;
 Thu, 23 May 2024 01:20:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Vuyn=M2=linux.intel.com=pawan.kumar.gupta@srs-se1.protection.inumbo.net>)
 id 1s9x8B-0005vM-8b
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 01:20:31 +0000
Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a26fe070-18a2-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 03:20:27 +0200 (CEST)
Received: from orviesa008.jf.intel.com ([10.64.159.148])
 by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 22 May 2024 18:20:25 -0700
Received: from jjsopko-mobl1.amr.corp.intel.com (HELO desk) ([10.212.211.181])
 by orviesa008-auth.jf.intel.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 May 2024 18:20:24 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a26fe070-18a2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1716427228; x=1747963228;
  h=date:from:to:cc:subject:message-id:mime-version;
  bh=OBJ6NI3y8qebJ5AK/thr6FfcOgOdvK+rfZRhsCXNxII=;
  b=dBeTss7czqN2vZVEUPBuc0hEELDLdFnbRcvXwRzvQitTmsTYE+6OBXlN
   Ccf+9Opw6S7FRQdips+OLyU3/hOmnpvkc+NOu4YwYvRqqxCPw2mIZhkWo
   ZqgI1CTgT57FrTffovWqjafS2sXH33tqOdXKlk+35+W+Lebh6Zde8/ioA
   QdNyHrqbKRineIXVYKWiO8aqLHi1zgmVY+cOl4MgXy6WplRJADSFHrSYz
   hG/ou0dmZdYzleNfKBS6dqWZcUHmsdDrtpWO86mVqTF/enloO3/EPQOJy
   NtKynCq0dDaMHkB7neJpI9zgoqRWK5dUYJj/wiMGTpHiHdb/l0a2aEoRY
   A==;
X-CSE-ConnectionGUID: Z+PC3ZFARVGaJgCD0HlOlw==
X-CSE-MsgGUID: FsPLxJmMQQmp8rxse5hefw==
X-IronPort-AV: E=McAfee;i="6600,9927,11080"; a="12551042"
X-IronPort-AV: E=Sophos;i="6.08,181,1712646000"; 
   d="scan'208";a="12551042"
X-CSE-ConnectionGUID: OiVUy4GeSKWtjX+5/NuI4A==
X-CSE-MsgGUID: Pscf/n4xQnuzctPt7iKQSg==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="6.08,181,1712646000"; 
   d="scan'208";a="34026442"
Date: Wed, 22 May 2024 18:20:15 -0700
From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
To: stable@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org, dave.hansen@linux.intel.com,
	Daniel Sneddon <daniel.sneddon@linux.intel.com>,
	xen-devel@lists.xenproject.org, security@debian.org,
	Salvatore Bonaccorso <carnil@debian.org>, benh@debian.org,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andy Lutomirski <luto@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, Juergen Gross <jgross@suse.com>,
	Deep Shah <sdeep@vmware.com>,
	"VMware, Inc." <pv-drivers@vmware.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	David Woodhouse <dwmw2@infradead.org>,
	Nikolay Borisov <nik.borisov@suse.com>,
	Josh Poimboeuf <jpoimboe@kernel.org>, Borislav Petkov <bp@suse.de>
Subject: [PATCH 5.10] x86/xen: Drop USERGS_SYSRET64 paravirt call
Message-ID: <20240522-verw-xen-pv-fix-v1-1-45add699c0e4@linux.intel.com>
X-B4-Tracking: v=1; b=H4sIAHmZTmYC/x2MywqAIBAAfyX23Iat2etXooPYVnuxULAg+vek4
 wzMPBA5CEcYiwcCJ4ly+Ax1WYDbrd8YZckMpKhRhggThwtv9ngmXOVGbnXf0WCddhZydQbO+j9
 OYKpawfy+H0Rh7bhnAAAA
X-Mailer: b4 0.12.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

From: Juergen Gross <jgross@suse.com>

commit afd30525a659ac0ae0904f0cb4a2ca75522c3123 upstream.

USERGS_SYSRET64 is used to return from a syscall via SYSRET, but
a Xen PV guest will nevertheless use the IRET hypercall, as there
is no sysret PV hypercall defined.

So instead of testing all the prerequisites for doing a sysret and
then mangling the stack for Xen PV again for doing an iret just use
the iret exit from the beginning.

This can easily be done via an ALTERNATIVE like it is done for the
sysenter compat case already.

It should be noted that this drops the optimization in Xen for not
restoring a few registers when returning to user mode, but it seems
as if the saved instructions in the kernel more than compensate for
this drop (a kernel build in a Xen PV guest was slightly faster with
this patch applied).

While at it remove the stale sysret32 remnants.

  [ pawan: Brad Spengler and Salvatore Bonaccorso <carnil@debian.org>
	   reported a problem with the 5.10 backport commit edc702b4a820
	   ("x86/entry_64: Add VERW just before userspace transition").

	   When CONFIG_PARAVIRT_XXL=y, CLEAR_CPU_BUFFERS is not executed in
	   syscall_return_via_sysret path as USERGS_SYSRET64 is runtime
	   patched to:

	.cpu_usergs_sysret64    = { 0x0f, 0x01, 0xf8,
				    0x48, 0x0f, 0x07 }, // swapgs; sysretq

	   which is missing CLEAR_CPU_BUFFERS. It turns out dropping
	   USERGS_SYSRET64 simplifies the code, allowing CLEAR_CPU_BUFFERS
	   to be explicitly added to syscall_return_via_sysret path. Below
	   is with CONFIG_PARAVIRT_XXL=y and this patch applied:

	   syscall_return_via_sysret:
	   ...
	   <+342>:   swapgs
	   <+345>:   xchg   %ax,%ax
	   <+347>:   verw   -0x1a2(%rip)  <------
	   <+354>:   sysretq
  ]

Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Link: https://lkml.kernel.org/r/20210120135555.32594-6-jgross@suse.com
---
 arch/x86/entry/entry_64.S             | 17 ++++++++---------
 arch/x86/include/asm/irqflags.h       |  7 -------
 arch/x86/include/asm/paravirt.h       |  5 -----
 arch/x86/include/asm/paravirt_types.h |  8 --------
 arch/x86/kernel/asm-offsets_64.c      |  2 --
 arch/x86/kernel/paravirt.c            |  5 +----
 arch/x86/kernel/paravirt_patch.c      |  4 ----
 arch/x86/xen/enlighten_pv.c           |  1 -
 arch/x86/xen/xen-asm.S                | 21 ---------------------
 arch/x86/xen/xen-ops.h                |  2 --
 10 files changed, 9 insertions(+), 63 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 1631a9a1566e..bd785386d629 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -46,14 +46,6 @@
 .code64
 .section .entry.text, "ax"
 
-#ifdef CONFIG_PARAVIRT_XXL
-SYM_CODE_START(native_usergs_sysret64)
-	UNWIND_HINT_EMPTY
-	swapgs
-	sysretq
-SYM_CODE_END(native_usergs_sysret64)
-#endif /* CONFIG_PARAVIRT_XXL */
-
 /*
  * 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
  *
@@ -128,7 +120,12 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
 	 * Try to use SYSRET instead of IRET if we're returning to
 	 * a completely clean 64-bit userspace context.  If we're not,
 	 * go to the slow exit path.
+	 * In the Xen PV case we must use iret anyway.
 	 */
+
+	ALTERNATIVE "", "jmp	swapgs_restore_regs_and_return_to_usermode", \
+		X86_FEATURE_XENPV
+
 	movq	RCX(%rsp), %rcx
 	movq	RIP(%rsp), %r11
 
@@ -220,7 +217,9 @@ syscall_return_via_sysret:
 
 	popq	%rdi
 	popq	%rsp
-	USERGS_SYSRET64
+	swapgs
+	CLEAR_CPU_BUFFERS
+	sysretq
 SYM_CODE_END(entry_SYSCALL_64)
 
 /*
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index f40dea50dfbf..e585a4705b8d 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -132,13 +132,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
 #endif
 
 #define INTERRUPT_RETURN	jmp native_iret
-#define USERGS_SYSRET64				\
-	swapgs;					\
-	CLEAR_CPU_BUFFERS;			\
-	sysretq;
-#define USERGS_SYSRET32				\
-	swapgs;					\
-	sysretl
 
 #else
 #define INTERRUPT_RETURN		iret
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 4a32b0d34376..3c89c1f64871 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -776,11 +776,6 @@ extern void default_banner(void);
 
 #ifdef CONFIG_X86_64
 #ifdef CONFIG_PARAVIRT_XXL
-#define USERGS_SYSRET64							\
-	PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64),			\
-		  ANNOTATE_RETPOLINE_SAFE;				\
-		  jmp PARA_INDIRECT(pv_ops+PV_CPU_usergs_sysret64);)
-
 #ifdef CONFIG_DEBUG_ENTRY
 #define SAVE_FLAGS(clobbers)                                        \
 	PARA_SITE(PARA_PATCH(PV_IRQ_save_fl),			    \
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 903d71884fa2..55d8b7950e61 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -157,14 +157,6 @@ struct pv_cpu_ops {
 
 	u64 (*read_pmc)(int counter);
 
-	/*
-	 * Switch to usermode gs and return to 64-bit usermode using
-	 * sysret.  Only used in 64-bit kernels to return to 64-bit
-	 * processes.  Usermode register state, including %rsp, must
-	 * already be restored.
-	 */
-	void (*usergs_sysret64)(void);
-
 	/* Normal iret.  Jump to this with the standard iret stack
 	   frame set up. */
 	void (*iret)(void);
diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c
index 1354bc30614d..b14533af7676 100644
--- a/arch/x86/kernel/asm-offsets_64.c
+++ b/arch/x86/kernel/asm-offsets_64.c
@@ -13,8 +13,6 @@ int main(void)
 {
 #ifdef CONFIG_PARAVIRT
 #ifdef CONFIG_PARAVIRT_XXL
-	OFFSET(PV_CPU_usergs_sysret64, paravirt_patch_template,
-	       cpu.usergs_sysret64);
 #ifdef CONFIG_DEBUG_ENTRY
 	OFFSET(PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);
 #endif
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index f0e4ad8595ca..9d91061b862c 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -124,8 +124,7 @@ unsigned paravirt_patch_default(u8 type, void *insn_buff,
 	else if (opfunc == _paravirt_ident_64)
 		ret = paravirt_patch_ident_64(insn_buff, len);
 
-	else if (type == PARAVIRT_PATCH(cpu.iret) ||
-		 type == PARAVIRT_PATCH(cpu.usergs_sysret64))
+	else if (type == PARAVIRT_PATCH(cpu.iret))
 		/* If operation requires a jmp, then jmp */
 		ret = paravirt_patch_jmp(insn_buff, opfunc, addr, len);
 #endif
@@ -159,7 +158,6 @@ static u64 native_steal_clock(int cpu)
 
 /* These are in entry.S */
 extern void native_iret(void);
-extern void native_usergs_sysret64(void);
 
 static struct resource reserve_ioports = {
 	.start = 0,
@@ -299,7 +297,6 @@ struct paravirt_patch_template pv_ops = {
 
 	.cpu.load_sp0		= native_load_sp0,
 
-	.cpu.usergs_sysret64	= native_usergs_sysret64,
 	.cpu.iret		= native_iret,
 
 #ifdef CONFIG_X86_IOPL_IOPERM
diff --git a/arch/x86/kernel/paravirt_patch.c b/arch/x86/kernel/paravirt_patch.c
index 7c518b08aa3c..2fada2c347c9 100644
--- a/arch/x86/kernel/paravirt_patch.c
+++ b/arch/x86/kernel/paravirt_patch.c
@@ -27,7 +27,6 @@ struct patch_xxl {
 	const unsigned char	mmu_write_cr3[3];
 	const unsigned char	irq_restore_fl[2];
 	const unsigned char	cpu_wbinvd[2];
-	const unsigned char	cpu_usergs_sysret64[6];
 	const unsigned char	mov64[3];
 };
 
@@ -40,8 +39,6 @@ static const struct patch_xxl patch_data_xxl = {
 	.mmu_write_cr3		= { 0x0f, 0x22, 0xdf },	// mov %rdi, %cr3
 	.irq_restore_fl		= { 0x57, 0x9d },	// push %rdi; popfq
 	.cpu_wbinvd		= { 0x0f, 0x09 },	// wbinvd
-	.cpu_usergs_sysret64	= { 0x0f, 0x01, 0xf8,
-				    0x48, 0x0f, 0x07 },	// swapgs; sysretq
 	.mov64			= { 0x48, 0x89, 0xf8 },	// mov %rdi, %rax
 };
 
@@ -83,7 +80,6 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr,
 	PATCH_CASE(mmu, read_cr3, xxl, insn_buff, len);
 	PATCH_CASE(mmu, write_cr3, xxl, insn_buff, len);
 
-	PATCH_CASE(cpu, usergs_sysret64, xxl, insn_buff, len);
 	PATCH_CASE(cpu, wbinvd, xxl, insn_buff, len);
 #endif
 
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 94804670caab..b1efc4b4f42a 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1059,7 +1059,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
 	.read_pmc = xen_read_pmc,
 
 	.iret = xen_iret,
-	.usergs_sysret64 = xen_sysret64,
 
 	.load_tr_desc = paravirt_nop,
 	.set_ldt = xen_set_ldt,
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index e3031afcb103..3a33713cf449 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -220,27 +220,6 @@ SYM_CODE_START(xen_iret)
 	jmp hypercall_iret
 SYM_CODE_END(xen_iret)
 
-SYM_CODE_START(xen_sysret64)
-	UNWIND_HINT_EMPTY
-	/*
-	 * We're already on the usermode stack at this point, but
-	 * still with the kernel gs, so we can easily switch back.
-	 *
-	 * tss.sp2 is scratch space.
-	 */
-	movq %rsp, PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
-	movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp
-
-	pushq $__USER_DS
-	pushq PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
-	pushq %r11
-	pushq $__USER_CS
-	pushq %rcx
-
-	pushq $VGCF_in_syscall
-	jmp hypercall_iret
-SYM_CODE_END(xen_sysret64)
-
 /*
  * XEN pv doesn't use trampoline stack, PER_CPU_VAR(cpu_tss_rw + TSS_sp0) is
  * also the kernel stack.  Reusing swapgs_restore_regs_and_return_to_usermode()
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 8695809b88f0..98242430d07e 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -138,8 +138,6 @@ __visible unsigned long xen_read_cr2_direct(void);
 
 /* These are not functions, and cannot be called normally */
 __visible void xen_iret(void);
-__visible void xen_sysret32(void);
-__visible void xen_sysret64(void);
 
 extern int xen_panic_handler_init(void);
 

---
base-commit: ce3838dbefdccfb95a63f81fe6cf77592ae9138c
change-id: 20240522-verw-xen-pv-fix-e638729ac3ca

Best regards,
-- 
Thanks,
Pawan



From xen-devel-bounces@lists.xenproject.org Thu May 23 01:26:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 01:26:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728001.1132712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9xDj-0006aA-Fu; Thu, 23 May 2024 01:26:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728001.1132712; Thu, 23 May 2024 01:26:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1s9xDj-0006a3-Ck; Thu, 23 May 2024 01:26:15 +0000
Received: by outflank-mailman (input) for mailman id 728001;
 Thu, 23 May 2024 01:26:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GulG=M2=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1s9xDi-0006Zx-AL
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 01:26:14 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6f8d74eb-18a3-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 03:26:11 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 7F1E462C1B;
 Thu, 23 May 2024 01:26:09 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11049C2BBFC;
 Thu, 23 May 2024 01:26:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f8d74eb-18a3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716427569;
	bh=YR3KJ3TmhDushqp+LY+DYIddrHRhTOOrKUSPZFMBa8I=;
	h=Date:From:To:cc:Subject:From;
	b=DgDb/QOwpW21LsdJspqBdV9yPP9Wn9O6dIKR7OjphMV7/b3BVG/JBlNRxSDlGrz0p
	 ZGQVvihkRRqdsff0CZFWd3P4jJcJ7jjHm/9U/ixlLhRs1mqSb6YUZlylKbN4Qgvch8
	 CzbPEgA4zVZInkDohYJ9na777Zzd7caGjmZw5qk3OuaWtXiRfQ8t8F8/zpVLDAYMM9
	 ysn5Mh/neZjmd3/Y72eSVhsw7qg+vmIbImFOgepXA1N7Oy1Mpb66wq8rIb9GkEGL41
	 2SX70LDKwE+RXjIWtFeDXiFECClbEYgIOTPb6SpGFqKIEy+zF/rJpcfsMFLqGsJfKD
	 rLJ0UxXii5GSw==
Date: Wed, 22 May 2024 18:26:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, 
    julien@xen.org, michal.orzel@amd.com, bertrand.marquis@arm.com, 
    roger.pau@citrix.com
Subject: [PATCH] docs/misra: rules for mass adoption
Message-ID: <alpine.DEB.2.22.394.2405221822500.1052252@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

This patch adds a bunch of rules to rules.rst that are uncontroversial
and have zero violations in Xen. As such, they have been approved for
adoption.

All the ones that regard the standard library have the link to the
existing footnote in the notes.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 80e5e972ad..d67c74a083 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -580,6 +580,11 @@ maintainers if you want to suggest a change.
      - The relational operators > >= < and <= shall not be applied to objects of pointer type except where they point into the same object
      -
 
+   * - `Rule 18.8 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_18_08.c>`_
+     - Required
+     - Variable-length array types shall not be used
+     -
+
    * - `Rule 19.1 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_19_01.c>`_
      - Mandatory
      - An object shall not be assigned or copied to an overlapping
@@ -589,11 +594,29 @@ maintainers if you want to suggest a change.
        instances where Eclair is unable to verify that the code is valid
        in regard to Rule 19.1. Caution reports are not violations.
 
+   * - `Rule 20.2 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_02.c>`_
+     - Required
+     - The ', " or \ characters and the /* or // character sequences
+       shall not occur in a header file name
+     -
+
+   * - `Rule 20.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_03.c>`_
+     - Required
+     - The #include directive shall be followed by either a <filename>
+       or "filename" sequence
+     -
+
    * - `Rule 20.4 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_04.c>`_
      - Required
      - A macro shall not be defined with the same name as a keyword
      -
 
+   * - `Rule 20.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_06.c>`_
+     - Required
+     - Tokens that look like a preprocessing directive shall not occur
+       within a macro argument
+     -
+
    * - `Rule 20.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_07.c>`_
      - Required
      - Expressions resulting from the expansion of macro parameters
@@ -609,6 +632,12 @@ maintainers if you want to suggest a change.
        evaluation
      -
 
+   * - `Rule 20.11 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_11.c>`_
+     - Required
+     - A macro parameter immediately following a # operator shall not
+       immediately be followed by a ## operator
+     -
+
    * - `Rule 20.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_12.c>`_
      - Required
      - A macro parameter used as an operand to the # or ## operators,
@@ -651,11 +680,39 @@ maintainers if you want to suggest a change.
        declared
      - See comment for Rule 21.1
 
+   * - `Rule 21.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_03.c>`_
+     - Required
+     - The memory allocation and deallocation functions of <stdlib.h>
+       shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
+   * - `Rule 21.4 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_04.c>`_
+     - Required
+     - The standard header file <setjmp.h> shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
+   * - `Rule 21.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_05.c>`_
+     - Required
+     - The standard header file <signal.h> shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Rule 21.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_06.c>`_
      - Required
      - The Standard Library input/output routines shall not be used
      - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
 
+   * - `Rule 21.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_07.c>`_
+     - Required
+     - The Standard Library functions atof, atoi, atol and atoll of
+       <stdlib.h> shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
+   * - `Rule 21.8 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_08.c>`_
+     - Required
+     - The Standard Library functions abort, exit and system of
+       <stdlib.h> shall not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Rule 21.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_09.c>`_
      - Required
      - The library functions bsearch and qsort of <stdlib.h> shall not be used
@@ -666,6 +723,11 @@ maintainers if you want to suggest a change.
      - The Standard Library time and date routines shall not be used
      - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
 
+   * - `Rule 21.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_12.c>`_
+     - Advisory
+     - The exception handling features of <fenv.h> should not be used
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Rule 21.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_21_13.c>`_
      - Mandatory
      - Any value passed to a function in <ctype.h> shall be representable as an
@@ -725,12 +787,25 @@ maintainers if you want to suggest a change.
      - The Standard Library function system of <stdlib.h> shall not be used
      -
 
+   * - `Rule 22.1 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_01.c>`_
+     - Required
+     - All resources obtained dynamically by means of Standard Library
+       functions shall be explicitly released
+     -
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Rule 22.2 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_02.c>`_
      - Mandatory
      - A block of memory shall only be freed if it was allocated by means of a
        Standard Library function
      -
 
+   * - `Rule 22.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_03.c>`_
+     - Required
+     - The same file shall not be open for read and write access at the
+       same time on different streams 
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
    * - `Rule 22.4 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_04.c>`_
      - Mandatory
      - There shall be no attempt to write to a stream which has been opened as
@@ -748,6 +823,31 @@ maintainers if you want to suggest a change.
        stream has been closed
      -
 
+   * - `Rule 22.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_07.c>`_
+     - Required
+     - The macro EOF shall only be compared with the unmodified return
+       value from any Standard Library function capable of returning EOF
+     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
+
+   * - `Rule 22.8 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_08.c>`_
+     - Required
+     - The value of errno shall be set to zero prior to a call to an
+       errno-setting-function
+     -
+
+   * - `Rule 22.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_09.c>`_
+     - Required
+     - The value of errno shall be tested against zero after calling an
+       errno-setting-function
+     -
+
+   * - `Rule 22.10 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_10.c>`_
+     - Required
+     - The value of errno shall only be tested when the last function to
+       be called was an errno-setting-function
+     -
+
+
 Terms & Definitions
 -------------------
 


From xen-devel-bounces@lists.xenproject.org Thu May 23 04:57:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 04:57:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728051.1132799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA0Vd-0005CS-5U; Thu, 23 May 2024 04:56:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728051.1132799; Thu, 23 May 2024 04:56:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA0Vd-0005CL-2W; Thu, 23 May 2024 04:56:57 +0000
Received: by outflank-mailman (input) for mailman id 728051;
 Thu, 23 May 2024 04:56:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0Vc-0005CB-6z; Thu, 23 May 2024 04:56:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0Vc-0005NR-3r; Thu, 23 May 2024 04:56:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0Vb-0005FX-TG; Thu, 23 May 2024 04:56:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0Vb-0001cg-Sv; Thu, 23 May 2024 04:56:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=7XVcxiA8qnJCNB9qTVj1Hkrt3slRRf93Bgs40K0eLrA=; b=whZohE/pknNLCWDOMWSFgaPLTS
	+rgW5oA+t4I6tgQ/NrCdOGHi4wjKVK3d67jQdfG8qB76teIKfiS2/iALtJjsYdQltjbz0AFulgZow
	OCJUezg0FDXKiq011syCBAox9H6Kz6qvopSmVZDe53eWhnrzTKSxenMem3TXCFjqCj9s=;
To: xen-devel@lists.xenproject.org
Subject: [xen-4.17-testing bisection] complete build-i386-xsm
Message-Id: <E1sA0Vb-0001cg-Sv@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 04:56:55 +0000

branch xen-4.17-testing
xenbranch xen-4.17-testing
job build-i386-xsm
testid xen-build

Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf git://xenbits.xen.org/osstest/ovmf.git
  Bug introduced:  750d763623fd1ff4a69d2e350310333dcbc19d4f
  Bug not present: c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/186102/


  commit 750d763623fd1ff4a69d2e350310333dcbc19d4f
  Author: Wenxing Hou <wenxing.hou@intel.com>
  Date:   Thu Apr 18 17:28:15 2024 +0800
  
      SecurityPkg: add DeviceSecurity support
      
      This patch implement the SpdmSecurityLib,
      which is the core of DeviceSecurity.
      And the SpdmSecurityLib include Device Authentication and Measurement.
      The other library is to support SpdmSecurityLib.
      
      Cc: Jiewen Yao <jiewen.yao@intel.com>
      Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
      Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-4.17-testing/build-i386-xsm.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/xen-4.17-testing/build-i386-xsm.xen-build --summary-out=tmp/186102.bisection-summary --basis-template=185864 --blessings=real,real-bisect,real-retry xen-4.17-testing build-i386-xsm xen-build
Searching for failure / basis pass:
 186069 fail [host=pinot0] / 186063 [host=albana0] 185864 [host=pinot1] 185711 [host=nobling0] 185446 [host=huxelrebe0] 185400 [host=italia0] 185318 [host=huxelrebe0] 185300 [host=italia0] 185284 [host=huxelrebe0] 185217 ok.
Failure / basis pass flights: 186069 / 185217
(tree with no url: minios)
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 7142e648416ff5d3eac6c6d607874805f5de0ca8 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
Basis pass 8f698f0a646124ede518d3e255ef725de1239639 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 1588fd1437960d94cadc30c42243671e8c0f1281 9bc40dbcf9eafccc1923b2555286bf6a2af03b7a
Generating revisions with ./adhoc-revtuple-generator  git://xenbits.xen.org/osstest/ovmf.git#8f698f0a646124ede518d3e255ef725de1239639-7142e648416ff5d3eac6c6d607874805f5de0ca8 git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#ffb451126550b22b43b62fb8731a0d78e3376c03-ffb451126550b22b43b62fb8731a0d78e3376c03 git://xenbits.xen.org/osstest/seabios.git#1588fd1437960d94cadc30c42243671e8c0f1\
 281-e5f2e4c69643bc3cd385306a9e5d29e11578148c git://xenbits.xen.org/xen.git#9bc40dbcf9eafccc1923b2555286bf6a2af03b7a-3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
Loaded 12696 nodes in revision graph
Searching for test results:
 185217 pass 8f698f0a646124ede518d3e255ef725de1239639 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 1588fd1437960d94cadc30c42243671e8c0f1281 9bc40dbcf9eafccc1923b2555286bf6a2af03b7a
 185284 [host=huxelrebe0]
 185300 [host=italia0]
 185318 [host=huxelrebe0]
 185400 [host=italia0]
 185446 [host=huxelrebe0]
 185494 [host=huxelrebe0]
 185514 [host=huxelrebe0]
 185536 [host=fiano0]
 185711 [host=nobling0]
 185864 [host=pinot1]
 186063 [host=albana0]
 186069 fail 7142e648416ff5d3eac6c6d607874805f5de0ca8 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
 186088 pass 8f698f0a646124ede518d3e255ef725de1239639 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 1588fd1437960d94cadc30c42243671e8c0f1281 9bc40dbcf9eafccc1923b2555286bf6a2af03b7a
 186089 fail 7142e648416ff5d3eac6c6d607874805f5de0ca8 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
 186090 pass 2b330b57dbe8014c5fa9f10d4cf4ae5923e3b143 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 5d9a931fe2c1310dbfd946bbc1e22a177add4f5c
 186091 fail 5f783827bbaa1552edf4386bb71d8d8f471340f5 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186092 pass 680030a6ec1b16126acfb3616c5c8047b70df69f 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 5d9a931fe2c1310dbfd946bbc1e22a177add4f5c
 186093 pass c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186094 fail 5e31c5666d95de55b9a910944c83ecddb19334ab 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186095 fail 54a4fd9b35cac79e7c54b750d3dddd0fbb43dca1 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186096 fail cf3b34c0b8fe6408859734f24c6dd39361df4173 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186097 fail 750d763623fd1ff4a69d2e350310333dcbc19d4f 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186098 pass c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186100 fail 750d763623fd1ff4a69d2e350310333dcbc19d4f 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186101 pass c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
 186102 fail 750d763623fd1ff4a69d2e350310333dcbc19d4f 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
Searching for interesting versions
 Result found: flight 185217 (pass), for basis pass
 For basis failure, parent search stopping at c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda, results HASH(0x5585a82e12a0) HASH(0x5585a82a2968) HASH(0x5585a82945e0) For basis failure, parent search stopping at 680030a6ec1b16126acfb3616c5c8047b70df69f 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69\
 643bc3cd385306a9e5d29e11578148c 5d9a931fe2c1310dbfd946bbc1e22a177add4f5c, results HASH(0x5585a82dcc68) For basis failure, parent search stopping at 2b330b57dbe8014c5fa9f10d4cf4ae5923e3b143 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c 5d9a931fe2c1310dbfd946bbc1e22a177add4f5c, results HASH(0x5585a82c89e0) For basis failure, parent search stopping at 8f698f0a646124ede518d3e255ef725de1239639 3d273dd05e51e5a1ffba3d98c7437ee\
 84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 1588fd1437960d94cadc30c42243671e8c0f1281 9bc40dbcf9eafccc1923b2555286bf6a2af03b7a, results HASH(0x5585a82b3118) HASH(0x5585a829af20) Result found: flight 186069 (fail), for basis failure (at ancestor ~1329)
 Repro found: flight 186088 (pass), for basis pass
 Repro found: flight 186089 (fail), for basis failure
 0 revisions at c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 ffb451126550b22b43b62fb8731a0d78e3376c03 e5f2e4c69643bc3cd385306a9e5d29e11578148c effcf70f020ff12d34c80e2abde0ecb00ce92bda
No revisions left to test, checking graph state.
 Result found: flight 186093 (pass), for last pass
 Result found: flight 186097 (fail), for first failure
 Repro found: flight 186098 (pass), for last pass
 Repro found: flight 186100 (fail), for first failure
 Repro found: flight 186101 (pass), for last pass
 Repro found: flight 186102 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf git://xenbits.xen.org/osstest/ovmf.git
  Bug introduced:  750d763623fd1ff4a69d2e350310333dcbc19d4f
  Bug not present: c3f615a1bd7d64f42e7962f5a4d53f1f1a4423e6
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/186102/


  commit 750d763623fd1ff4a69d2e350310333dcbc19d4f
  Author: Wenxing Hou <wenxing.hou@intel.com>
  Date:   Thu Apr 18 17:28:15 2024 +0800
  
      SecurityPkg: add DeviceSecurity support
      
      This patch implement the SpdmSecurityLib,
      which is the core of DeviceSecurity.
      And the SpdmSecurityLib include Device Authentication and Measurement.
      The other library is to support SpdmSecurityLib.
      
      Cc: Jiewen Yao <jiewen.yao@intel.com>
      Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
      Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>

pnmtopng: 174 colors found
Revision graph left in /home/logs/results/bisect/xen-4.17-testing/build-i386-xsm.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
186102: tolerable ALL FAIL

flight 186102 xen-4.17-testing real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/186102/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-i386-xsm                6 xen-build               fail baseline untested


jobs:
 build-i386-xsm                                               fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Thu May 23 05:24:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 05:24:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728058.1132810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA0wb-0000cN-5b; Thu, 23 May 2024 05:24:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728058.1132810; Thu, 23 May 2024 05:24:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA0wb-0000cG-2t; Thu, 23 May 2024 05:24:49 +0000
Received: by outflank-mailman (input) for mailman id 728058;
 Thu, 23 May 2024 05:24:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0wZ-0000c6-Vi; Thu, 23 May 2024 05:24:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0wZ-0006A0-Rk; Thu, 23 May 2024 05:24:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0wZ-0005sI-H2; Thu, 23 May 2024 05:24:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA0wZ-0008Jf-Gi; Thu, 23 May 2024 05:24:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3rP/tPnms7Ju16XCN7dRvBA02MfIgkfvLbjfHTq3ODk=; b=mXoh2lYXyWfwINGJxGX9JxcaRo
	P8MO+ypEeyn0UXVI7Ds7WEGIipM/Qvlu2eY1DvVWj72yhhH8LzDf/g0hqtq5Q2Ydxp2qNA9jSkYDq
	XkiN1keoRpW8tZp6N/6bM6BzM77bTpMBlwQPLFqkjbjUycP/+M4hXenOuB1e51tbVjaI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186072-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186072: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-amd64-xsm:xen-build:fail:regression
    linux-linus:build-amd64:xen-build:fail:regression
    linux-linus:build-i386:xen-build:fail:regression
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:regression
    linux-linus:build-i386-xsm:xen-build:fail:regression
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:build-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:build-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-bios:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-uefi:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=29c73fc794c83505066ee6db893b2a83ac5fac63
X-Osstest-Versions-That:
    linux=8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 05:24:47 +0000

flight 186072 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186072/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 186052
 build-amd64                   6 xen-build                fail REGR. vs. 186052
 build-i386                    6 xen-build                fail REGR. vs. 186052
 test-armhf-armhf-xl           8 xen-boot                 fail REGR. vs. 186052
 build-i386-xsm                6 xen-build                fail REGR. vs. 186052
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 186052

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-vhd       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine      1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-bios  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 186052
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186052
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                29c73fc794c83505066ee6db893b2a83ac5fac63
baseline version:
 linux                8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6

Last test of basis   186052  2024-05-21 01:42:42 Z    2 days
Failing since        186065  2024-05-21 16:10:24 Z    1 days    2 attempts
Testing same since   186072  2024-05-22 07:52:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Liang, Kan" <kan.liang@linux.intel.com>
  "Ricardo B. Marliere" <ricardo@marliere.net>
  Abel Vesa <abel.vesa@linaro.org>
  Adam Ford <aford173@gmail.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Egorenkov <egorenar@linux.ibm.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexey Kardashevskiy <aik@amd.com>
  Allen Pais <allen.lkml@gmail.com>
  Allen Pais <apais@linux.microsoft.com>
  Amrit Anand <quic_amrianan@quicinc.com>
  Andi Kleen <ak@linux.intel.com>
  Andrew Davis <afd@ti.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Anne Macedo <retpolanne@posteo.net>
  Anshuman Khandual <anshuman.khandual@arm.com>
  Apurva Nandan <a-nandan@ti.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Athira Rajeev <atrajeev@linux.vnet.ibm.com>
  Atish Patra <atishp@rivosinc.com>
  Bard Liao <yung-chuan.liao@linux.intel.com>
  Baruch Siach <baruch@tkos.co.il>
  Beleswar Padhi <b-padhi@ti.com>
  Ben Levinsky <ben.levinsky@amd.com>
  Benjamin Tissoires <bentiss@kernel.org>
  Bhaskar Chowdhury <unixbhaskar@gmail.com>
  Bjorn Andersson <andersson@kernel.org>
  Bjorn Helgaas <bhelgaas@google.com>
  Chaitanya S Prakash <chaitanyas.prakash@arm.com>
  Chen Ni <nichen@iscas.ac.cn>
  Chen Pei <cp0613@linux.alibaba.com>
  Christian Brauner <brauner@kernel.org>
  Clark Wang <xiaoning.wang@nxp.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Cristian Marussi <cristian.marussi@arm.com>
  Damien Le Moal <dlemoal@kernel.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Dan Williams <dan.j.williams@intel.com>
  Daniel Golle <daniel@makrotopia.org>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  Danila Tikhonov <danila@jiaxyga.com>
  Dave Jiang <dave.jiang@intel.com>
  Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
  Devyn Liu <liudingyuan@huawei.com>
  Dima Kogan <dima@secretsauce.net>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dominique Martinet <asmadeus@codewreck.org>
  Duoming Zhou <duoming@zju.edu.cn>
  Eric Heintzmann <heintzmann.eric@free.fr>
  Erick Archer <erick.archer@outlook.com>
  Ethan Adams <j.ethan.adams@gmail.com>
  Fenghua Yu <fenghua.yu@intel.com>
  Frank Li <Frank.Li@nxp.com>
  Frank Wang <frank.wang@rock-chips.com>
  Frieder Schrempf <frieder.schrempf@kontron.de> # Kontron BL
  Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Greg Ungerer <gerg@linux-m68k.org>
  Hans de Goede <hdegoede@redhat.com>
  Harald Freudenberger <freude@linux.ibm.com>
  He Zhe <zhe.he@windriver.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko@sntech.de>
  Heiner Kallweit <hkallweit1@gmail.com>
  Holger Dengler <dengler@linux.ibm.com>
  Howard Chu <howardchu95@gmail.com>
  Ian Rogers <irogers@google.com>
  Ilkka Koskinen <ilkka@os.amperecomputing.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Iuliana Prodan <iuliana.prodan@nxp.com>
  James Clark <james.clark@arm.com>
  Jan Kiszka <jan.kiszka@siemens.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jason-JH.Lin <jason-jh.lin@mediatek.com>
  Jassi Brar <jassisinghbrar@gmail.com>
  Jens Remus <jremus@linux.ibm.com>
  Jerry Snitselaar <jsnitsel@redhat.com>
  Jiri Olsa <jolsa@kernel.org>
  Joao Pinto <Joao.Pinto@synopsys.com>
  Joao Pinto <jpinto@synopsys.com>
  Joy Zou <joy.zou@nxp.com>
  Jules Irenge <jbi.octave@gmail.com>
  Julien Panis <jpanis@baylibre.com>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Kan Liang <kan.liang@linux.intel.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Komal Bajaj <quic_kbajaj@quicinc.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Konstantin Kharlamov <Hi-Angel@yandex.ru>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Krzysztof Wilczyński <kwilczynski@kernel.org>
  Kuan-Wei Chiu <visitorckw@gmail.com>
  Kunwu Chan <chentao@kylinos.cn>
  Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
  Kyle Meyer <kyle.meyer@hpe.com>
  Laura Nao <laura.nao@collabora.com>
  Leonard Crestez <cdleonard@gmail.com>
  Lijun Pan <lijun.pan@intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Luca Ceresoli <luca.ceresoli@bootlin.com>
  Luca Weiss <luca@z3ntu.xyz>
  Lucas Stach <l.stach@pengutronix.de>
  Madadi Vineeth Reddy <vineethr@linux.ibm.com>
  Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marco Felsch <m.felsch@pengutronix.de>
  Marek Vasut <marex@denx.de>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Masami Hiramatsu <mhiramat@kernel.org>
  Mateusz Kaduk <mateusz.kaduk@gmail.com>
  Mathieu Poirier <mathieu.poirier@linaro.org>
  Michal Simek <michal.simek@amd.com>
  Miguel Ojeda <ojeda@kernel.org>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Mike Snitzer <snitzer@kernel.org>
  Miklos Szeredi <mszeredi@redhat.com>
  Mikulas Patocka <mpatocka@redhat.com>
  Mrinmay Sarkar <quic_msarkar@quicinc.com>
  Nam Cao <namcao@linutronix.de>
  Namhyung Kim <namhyung@kernel.org>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nicolin Chen <b42378@freescale.com>
  Niklas Cassel <cassel@kernel.org>
  Nuno Sa <nuno.sa@analog.com>
  Olivia Wen <olivia.wen@mediatek.com>
  Peng Ma <andypma@tencent.com>
  Peter Griffin <peter.griffin@linaro.org>
  Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
  Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
  Ricardo B. Marliere <ricardo@marliere.net>
  Richard Leitner <richard.leitner@skidata.com>
  Rick Wertenbroek <rick.wertenbroek@gmail.com>
  Rob Herring (Arm) <robh@kernel.org>
  Rob Herring <robh@kernel.org>
  Robin Gong <yibin.gong@nxp.com>
  Rohit Agarwal <quic_rohiagar@quicinc.com>
  Saeed Nowshadi <saeed.nowshadi@amd.com>
  Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
  Sandipan Das <sandipan.das@amd.com>
  Satish Thatchanamurthy <Satish.Thatchanamurt@Dell.com> # one platform
  Sean Anderson <sean.anderson@linux.dev>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Sergio Paracuellos <sergio.paracuellos@gmail.com>
  Shengjiu Wang <shengjiu.wang@nxp.com>
  Siddharth Vadapalli <s-vadapalli@ti.com>
  Srikanth Krishnakar <Srikanth_Krishnakar@mentor.com>
  Stephen Rothwell <sfr@canb.auug.org.au> # PowerPC le incermental build
  Sumanth Korikkar <sumanthk@linux.ibm.com>
  Sven Schnelle <svens@linux.ibm.com>
  Tan Chun Hau <chunhau.tan@starfivetech.com>
  Tanmay Shah <tanmay.shah@amd.com>
  Thomas Huth <thuth@redhat.com>
  Thomas Richter <tmricht@linux.ibm.com>
  Tommaso Merciai <tomm.merciai@gmail.com>
  Tycho Andersen <tycho@tycho.pizza>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vidya Sagar <vidyas@nvidia.com>
  Vijendar Mukunda <Vijendar.Mukunda@amd.com>
  Vincent Guittot <vincent.guittot@linaro.org>
  Vinod Koul <vkoul@kernel.org>
  Vipul Kumar <vipul_kumar@mentor.com>
  Viresh Kumar <viresh.kumar@linaro.org>
  Weilin Wang <weilin.wang@intel.com>
  Wendy Wang <wendy.wang@intel.com>
  Yang Jihong <yangjihong@bytedance.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Yu Kuai <yukuai3@huawei.com>
  Yury Norov <yury.norov@gmail.com>
  Zhang Yubing <yubing.zhang@rock-chips.com>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                blocked 
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-amd64-examine                                     blocked 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 blocked 
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      blocked 
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      blocked 
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 17610 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 06:35:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 06:35:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728072.1132820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA22K-0008Jc-89; Thu, 23 May 2024 06:34:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728072.1132820; Thu, 23 May 2024 06:34:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA22K-0008JV-51; Thu, 23 May 2024 06:34:48 +0000
Received: by outflank-mailman (input) for mailman id 728072;
 Thu, 23 May 2024 06:34:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QTNb=M2=linuxfoundation.org=gregkh@srs-se1.protection.inumbo.net>)
 id 1sA22I-0008JN-L6
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 06:34:46 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8adeec37-18ce-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 08:34:45 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 10D4862C85;
 Thu, 23 May 2024 06:34:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33804C2BD10;
 Thu, 23 May 2024 06:34:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8adeec37-18ce-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1716446083;
	bh=XxVSYbHZ+H/p/wEVBPhFkeHIAS4bvPHUHG/AfTbTDl0=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=qj4PLV3HfkRjRQvaCMLX5FImOPh0tTMCsbpqbwyjmQ9PuQCJMuRKK4DRfMAdT+H64
	 /pyVc6TFgQ1Ff0VO5zYnQCh7NihLZ5ubHevsFD67z9IPgAHJ09ei56lxzlYf5BLgaN
	 tDPpQVlYjfjJ5j3Xrko/muiRpAeisWvFzfXLZ7Ss=
Date: Thu, 23 May 2024 08:34:40 +0200
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Cc: stable@vger.kernel.org, virtualization@lists.linux-foundation.org,
	dave.hansen@linux.intel.com,
	Daniel Sneddon <daniel.sneddon@linux.intel.com>,
	xen-devel@lists.xenproject.org, security@debian.org,
	Salvatore Bonaccorso <carnil@debian.org>, benh@debian.org,
	Andy Lutomirski <luto@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, Juergen Gross <jgross@suse.com>,
	Deep Shah <sdeep@vmware.com>,
	"VMware, Inc." <pv-drivers@vmware.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	David Woodhouse <dwmw2@infradead.org>,
	Nikolay Borisov <nik.borisov@suse.com>,
	Josh Poimboeuf <jpoimboe@kernel.org>, Borislav Petkov <bp@suse.de>
Subject: Re: [PATCH 5.10] x86/xen: Drop USERGS_SYSRET64 paravirt call
Message-ID: <2024052333-squishy-phony-d2e2@gregkh>
References: <20240522-verw-xen-pv-fix-v1-1-45add699c0e4@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240522-verw-xen-pv-fix-v1-1-45add699c0e4@linux.intel.com>

On Wed, May 22, 2024 at 06:20:15PM -0700, Pawan Gupta wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> commit afd30525a659ac0ae0904f0cb4a2ca75522c3123 upstream.

Now queued up, thanks.

greg k-h


From xen-devel-bounces@lists.xenproject.org Thu May 23 06:55:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 06:55:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728077.1132829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2Ll-0002a0-Mt; Thu, 23 May 2024 06:54:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728077.1132829; Thu, 23 May 2024 06:54:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2Ll-0002Zt-KP; Thu, 23 May 2024 06:54:53 +0000
Received: by outflank-mailman (input) for mailman id 728077;
 Thu, 23 May 2024 06:54:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA2Lk-0002Zn-Rq
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 06:54:52 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5a32c9ea-18d1-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 08:54:51 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso3683772a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 May 2024 23:54:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733bea66f9sm19259539a12.15.2024.05.22.23.54.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 May 2024 23:54:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a32c9ea-18d1-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716447291; x=1717052091; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=oLfL0xWJbsfChY8rKWUYHVyUtWt3t7ZkmOe/WaKW2nA=;
        b=SgQSGXEWqSJtViQ/3H9A+pCIqq/xnPpW6B4C+kzY1jU5p1nydObFKb9z1+0HpqjV9Q
         bxgGFstqKYQdhSazAdXLKfW7z8OtNWnjdZBihqwi13Jlb2pa8FPj0xZSQ8uZlVhEuBYd
         dsf5xhkSZioxYJ0bJxWc7hoZEF1FHVUGsjEzT3x1aOJn9XLtDovEFSoUUTeAKLC3RrKE
         FLYv0GnPnLrLtRi8fzkHFLA529lbsG50lHOWPYDyHm5RxVdUacQmqnZv1OTH0FPjLtHG
         O1fymqhAi+rcf+q+qttYERwRjiy0a1ZyXvbRUIYVdz1pOUnJkYzsgjAYWNRpJh3wrftB
         4Czg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716447291; x=1717052091;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=oLfL0xWJbsfChY8rKWUYHVyUtWt3t7ZkmOe/WaKW2nA=;
        b=kXxdEN4bh+3/oTJIcUmwI9JHhJRAUu0cWGOvXs4bRj8GkS9OQxziHpV4JwScp57P0Y
         NWmadWea9w47FMEWM829jd85eDaiYwxDNWwz9k0svrgwyexQo+jhDOSTge0HE4FzHcie
         EXX1aTOmEpxqUU2LL/kv6OTqXvnaIUPuvdh8yDl9OPIUXSOU75rCHwf63O2DPZDKH3jr
         jDquecvdxtMT1d1JCPac2uA7f6hWPc6vAiGv336Xtu51jpmaRFp460Ht0+axF99Iw/EA
         imzLgwJdGQkiOVFCkBbIVkUZSuveYFxRfjE/Cr7dBVKDcySNnEPSxnhfh/6kCsRHjvaO
         DM2A==
X-Forwarded-Encrypted: i=1; AJvYcCWCviaubZIBdKep7qRV2dHFRHK1juzz9cpwQ2gOZMnhqvcLcDtcnK4pnMaghpnP89QpGnFkrFkZC/Ff0VHgq0hZ55ZdLisFsIuoAd1+XJo=
X-Gm-Message-State: AOJu0YysDiiMNWUJLNQHHMF8+3MZXlWASALI0O1mreSC13iddeXw4w1I
	RRjuFabzOrh7TnUd7llgEKHb15BG+KxcdRUT15Jfyi9baciSN32asuaXykARkA==
X-Google-Smtp-Source: AGHT+IEAvnQa3b97gEH9UWIBTfsfaCNgjp+twUtPOTgQ5kjkYvsKaTX8tD50yL80ujx7B1KH6X2wiQ==
X-Received: by 2002:a50:baa2:0:b0:572:a049:c4bb with SMTP id 4fb4d7f45d1cf-57832a847d9mr2461568a12.20.1716447290704;
        Wed, 22 May 2024 23:54:50 -0700 (PDT)
Message-ID: <14059212-3704-47e9-b758-6294f220cb83@suse.com>
Date: Thu, 23 May 2024 08:54:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
 <cd1bf7b0-4a0a-4a38-9129-53822cb8e037@suse.com> <Zk3wMevPye8Rfjm4@macbook>
 <5484aab7-b15d-46c5-9ff1-9a5b54640acc@suse.com> <Zk4blZ9z6cn-ORrv@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk4blZ9z6cn-ORrv@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 18:21, Roger Pau Monné wrote:
> On Wed, May 22, 2024 at 03:34:29PM +0200, Jan Beulich wrote:
>> On 22.05.2024 15:16, Roger Pau Monné wrote:
>>> On Tue, May 21, 2024 at 12:30:32PM +0200, Jan Beulich wrote:
>>>> On 17.05.2024 15:33, Roger Pau Monne wrote:
>>>>> Enabling it using an HVM param is fragile, and complicates the logic when
>>>>> deciding whether options that interact with altp2m can also be enabled.
>>>>>
>>>>> Leave the HVM param value for consumption by the guest, but prevent it from
>>>>> being set.  Enabling is now done using and additional altp2m specific field in
>>>>> xen_domctl_createdomain.
>>>>>
>>>>> Note that albeit only currently implemented in x86, altp2m could be implemented
>>>>> in other architectures, hence why the field is added to xen_domctl_createdomain
>>>>> instead of xen_arch_domainconfig.
>>>>>
>>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>>
>>>> Reviewed-by: Jan Beulich <jbeulich@suse.com> # hypervisor
>>>> albeit with one question:
>>>>
>>>>> --- a/xen/arch/x86/domain.c
>>>>> +++ b/xen/arch/x86/domain.c
>>>>> @@ -637,6 +637,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>>>      bool hap = config->flags & XEN_DOMCTL_CDF_hap;
>>>>>      bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
>>>>>      unsigned int max_vcpus;
>>>>> +    unsigned int altp2m_mode = MASK_EXTR(config->altp2m_opts,
>>>>> +                                         XEN_DOMCTL_ALTP2M_mode_mask);
>>>>>  
>>>>>      if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
>>>>>      {
>>>>> @@ -715,6 +717,26 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>>>>          return -EINVAL;
>>>>>      }
>>>>>  
>>>>> +    if ( config->altp2m_opts & ~XEN_DOMCTL_ALTP2M_mode_mask )
>>>>> +    {
>>>>> +        dprintk(XENLOG_INFO, "Invalid altp2m options selected: %#x\n",
>>>>> +                config->flags);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if ( altp2m_mode && nested_virt )
>>>>> +    {
>>>>> +        dprintk(XENLOG_INFO,
>>>>> +                "Nested virt and altp2m are not supported together\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if ( altp2m_mode && !hap )
>>>>> +    {
>>>>> +        dprintk(XENLOG_INFO, "altp2m is only supported with HAP\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>
>>>> Should this last one perhaps be further extended to permit altp2m with EPT
>>>> only?
>>>
>>> Hm, yes, that would be more accurate as:
>>>
>>> if ( altp2m_mode && (!hap || !hvm_altp2m_supported()) )
>>
>> Wouldn't
>>
>>    if ( altp2m_mode && !hvm_altp2m_supported() )
>>
>> suffice? hvm_funcs.caps.altp2m is not supposed to be set when no HAP,
>> as long as HAP continues to be a pre-condition?
> 
> No, `hap` here signals whether the domain is using HAP, and we need to
> take this int account, otherwise we would allow enabling altp2m for
> domains using shadow.

Oh, right. But then the original for is good enough HAP-wise, as a request
to use HAP when HAP isn't available is deal with elsewhere. The
!hvm_altp2m_supported() is still wanted imo (for there potentially being
other restrictions), but then in a separate check, not resulting in a HAP-
specific log message. I'll commit the patch in its original form, and that
further addition can then be an incremental change.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:20:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:20:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728083.1132840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2jw-0005Xh-Ka; Thu, 23 May 2024 07:19:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728083.1132840; Thu, 23 May 2024 07:19:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2jw-0005Xa-HU; Thu, 23 May 2024 07:19:52 +0000
Received: by outflank-mailman (input) for mailman id 728083;
 Thu, 23 May 2024 07:19:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA2jv-0005XU-Gm
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:19:51 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d704cb42-18d4-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:19:49 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so177962266b.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:19:49 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5df00490cfsm731905266b.159.2024.05.23.00.19.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 00:19:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d704cb42-18d4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716448788; x=1717053588; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EkA1IoKHeNHQ8JzFiCMQbdV8RlpbiTWROT1aS2MrU7c=;
        b=H78r9szOyyhsW37ezA/jONtXn8fTqwAp9YWvnr+YII91sFbq27L7CDE7KKkDn89/K8
         EfJKwTb9+YaItnj10gUgJOYv3iDXYGkOWPitkkANLQstjSOR0OkDmSLQwSuI9xaowt/N
         BlIV49awg6UVe8YDK3T+92b7fNdZ0/Rgdzd44lkdgiQ4z9zOCLOaabXrEdMjsPrXpY9i
         druuC0EEeZeJZP1GaTcHU4vCoDQoDla6lfvoF5QJEAd0csnDfrFh8SzKU1YqNa7rjpR2
         oFES5izSOiEnPYpBKG8m0kglt87MuB5BKIcf7Icy/1/76LyWSMO/O/b3KX8BkAFG8swJ
         185A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716448788; x=1717053588;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=EkA1IoKHeNHQ8JzFiCMQbdV8RlpbiTWROT1aS2MrU7c=;
        b=U31zWMEWEN3V8CexEkFvQBEP471TpCK+qTpIHVt9PQlbhDJNxu6bYsTLhr9aKXgj86
         D8wuz7cRqszAwy8qEwZ4D88fWfU71p4Kwwd0kHqKna7qsCAHs2RdswLCXQe55FBwke4d
         idX8suOGwTR7bmGsJe4TAf8P+AVxQSwv8wsn6L8Qgo2ex7YYpOF4L04gXxf4SyjnWDWj
         fja+cejc3Mc/hIJ4SHbjGiAg7HkFpjloejgC8hTViUUG9hN4FVVHufqYmRP1p0giYkfw
         yt/EiIVUPX17fq9yM4zXLbIFKYDvdXt7GVjeeNFARPPZcb1mOJVMLCSV5EfFPkoppFhx
         Kq8A==
X-Forwarded-Encrypted: i=1; AJvYcCWs2cenD7n6Ue+sIpt/TPtJLwdx+Wmmeh2ovIHm9hJIJnU6NFq7U2vNlveOpAcjxt7URH3K9v+Bn1TBUDMo/w8HXLJmbsreeauI8GGedVA=
X-Gm-Message-State: AOJu0YxjVru7K6wRKBIMtgubnffUdYufMpqr+kL8p5c+c+Rai7OomHCu
	TClqpJGDMmAF0oeDgcCLj87MiQqbvBLKGyzvMBfdep4ILVLeH81C8/MGM8LRNA==
X-Google-Smtp-Source: AGHT+IGOVbZnFkYKTFkg0eWf1vzGgFjD+A29pp2VSFQraBEwKEwhiXncV7I8e6Jx7JTABjzZajP7HQ==
X-Received: by 2002:a17:906:2b53:b0:a59:a977:a154 with SMTP id a640c23a62f3a-a622818fa42mr246364966b.64.1716448788664;
        Thu, 23 May 2024 00:19:48 -0700 (PDT)
Message-ID: <19073c21-d878-4d8d-95d8-90f567688ed5@suse.com>
Date: Thu, 23 May 2024 09:19:47 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/misra: rules for mass adoption
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 michal.orzel@amd.com, bertrand.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2405221822500.1052252@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405221822500.1052252@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 03:26, Stefano Stabellini wrote:
> @@ -725,12 +787,25 @@ maintainers if you want to suggest a change.
>       - The Standard Library function system of <stdlib.h> shall not be used
>       -
>  
> +   * - `Rule 22.1 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_01.c>`_
> +     - Required
> +     - All resources obtained dynamically by means of Standard Library
> +       functions shall be explicitly released
> +     -
> +     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_

The empty sub-bullet-point looks stray here.

> @@ -748,6 +823,31 @@ maintainers if you want to suggest a change.
>         stream has been closed
>       -
>  
> +   * - `Rule 22.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_07.c>`_
> +     - Required
> +     - The macro EOF shall only be compared with the unmodified return
> +       value from any Standard Library function capable of returning EOF
> +     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_

Shouldn't this remark also be replicated ...

> +   * - `Rule 22.8 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_08.c>`_
> +     - Required
> +     - The value of errno shall be set to zero prior to a call to an
> +       errno-setting-function
> +     -
> +
> +   * - `Rule 22.9 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_09.c>`_
> +     - Required
> +     - The value of errno shall be tested against zero after calling an
> +       errno-setting-function
> +     -
> +
> +   * - `Rule 22.10 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_22_10.c>`_
> +     - Required
> +     - The value of errno shall only be tested when the last function to
> +       be called was an errno-setting-function
> +     -

... for all three of these, seeing that errno is something a (standard) library
would provide? Or alternatively should remarks here say that we simply have no
errno?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:31:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:31:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728088.1132849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2ug-0008C4-Gm; Thu, 23 May 2024 07:30:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728088.1132849; Thu, 23 May 2024 07:30:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA2ug-0008Bx-EB; Thu, 23 May 2024 07:30:58 +0000
Received: by outflank-mailman (input) for mailman id 728088;
 Thu, 23 May 2024 07:30:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA2uf-0008BO-3y
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:30:57 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6474cd23-18d6-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:30:56 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4202ca70287so10515835e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:30:56 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100fb9840sm16671255e9.42.2024.05.23.00.30.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 00:30:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6474cd23-18d6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716449455; x=1717054255; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=KJIuqIYDJT5oOIvrbW+DQrke5YDIdSv4a9t/Q3ZFSuQ=;
        b=XlgEu8pHZXmvsFpQi1eKBb/KRZ3oDoKWLKZMcv1i59qFoMBocLfpcz8YkoruAo1SnT
         j0gedZYv9wqPnEYPOrHAFuEbLNe0c/Ziwp7ggbGLypB8cIwilcVwwGdf3ozLy9ASgiUE
         a3W6eYEjbC3oxLKFKxGIsDFmBgccLpXu8+7M8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716449455; x=1717054255;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KJIuqIYDJT5oOIvrbW+DQrke5YDIdSv4a9t/Q3ZFSuQ=;
        b=ksQu0u3d8tcXFU8afGUgzr0e2eKKRPdPCpQ3LqnwbiSX7Lvae3H7d9kOiZEGw0UwXX
         1SdP2+6Pr2SXSSaKIZYfvoUGxHAB7h02LIC3P3hN3GaDusHtzAL48/LPMsVIrVrMwHib
         67m8h6Z8xa/g5aRC4MusdIaUuBhHAcFXKOFT6DS9kWKGj2k99ulGTl76uNa5MPm8SEYL
         Hq9YARRIPscl1+Ix/UCsYH6t7u/PoDH53AmJauPAOFxwmtHqu3S/47+prGY1GYiqBfbf
         I9x+Za9pG40EvhrZ1Cvtdh3qiLUBpng9f8ckghqULBrJ9mea1b3my6QNVpFc5qmTv0vN
         blEQ==
X-Gm-Message-State: AOJu0Yy22jlRgLmg4nQchrqcv49AbRN2wXNdiDSSVNblXWKHMHAn+rLH
	Ar8J0vW+2S63a7B9D0EH5arfh2/iTilJq0z9npnBF4OWpMfu2GujfKNsJ5CULYo=
X-Google-Smtp-Source: AGHT+IHQCQSYp4Z81XvVBIY+IyRiFx0DBqXAXNZ4RvRg6SyFQOIvJSOdWDkhC8UEEKdjztc3f4DWPw==
X-Received: by 2002:a05:600c:4f4a:b0:421:20d:e3a4 with SMTP id 5b1f17b1804b1-421020de48dmr9873995e9.17.1716449455337;
        Thu, 23 May 2024 00:30:55 -0700 (PDT)
Date: Thu, 23 May 2024 09:30:54 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v16 3/5] vpci: add initial support for virtual PCI bus
 topology
Message-ID: <Zk7wrlMJjlrBJx6p@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-4-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240522225927.77398-4-stewart.hildebrand@amd.com>

On Wed, May 22, 2024 at 06:59:22PM -0400, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Assign SBDF to the PCI devices being passed through with bus 0.
> The resulting topology is where PCIe devices reside on the bus 0 of the
> root complex itself (embedded endpoints).
> This implementation is limited to 32 devices which are allowed on
> a single PCI bus.
> 
> Please note, that at the moment only function 0 of a multifunction
> device can be passed through.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728095.1132880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34J-00020W-VF; Thu, 23 May 2024 07:40:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728095.1132880; Thu, 23 May 2024 07:40:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34J-00020L-S1; Thu, 23 May 2024 07:40:55 +0000
Received: by outflank-mailman (input) for mailman id 728095;
 Thu, 23 May 2024 07:40:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34J-0001WY-5o
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:40:55 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c8a1b593-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:40:54 +0200 (CEST)
Received: from CH0PR03CA0317.namprd03.prod.outlook.com (2603:10b6:610:118::12)
 by DS7PR12MB5718.namprd12.prod.outlook.com (2603:10b6:8:71::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Thu, 23 May
 2024 07:40:49 +0000
Received: from CH1PEPF0000AD74.namprd04.prod.outlook.com
 (2603:10b6:610:118:cafe::9b) by CH0PR03CA0317.outlook.office365.com
 (2603:10b6:610:118::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Thu, 23 May 2024 07:40:49 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD74.mail.protection.outlook.com (10.167.244.52) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:49 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:47 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:46 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8a1b593-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BENyBqWYcm6Pq/xf854269+YaTfuU2GK/+p0Jnp2PKwUYsar2Nz83urLx3jCEj/L5ueXm1O5bTu0EAnIfASZBaRWjtE2RQowQqBMaV2rTC+MxlF38pTt5w9B4FEhhcSuovXgYdg+28i2a5rOLDeYWk3+FlBuQl3MTcGTxEROSFKSp8+ZXMPp42m75LnB4QLA6D1HthnMAWk9sU04CNsrAQgdOJgATTDE0n+D4ARNBfzzLSywBYE7HJp2N6f3jxGEN1+vwteRxZNjy1wxsKuH5k8YyRfIMUVvNXF5Qr6LUFE2ggRGlUQOXEj/6lfKoVBXDNdWFZyOa4GXjnzoTEio9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=1PvDoXhNpQSv0AS8DYv80AMDNw8eCiT+reRuwP+9FhI=;
 b=hbyVjSNL2HvF+bDJjnSDNSm4BtfimDM57P7OMhiL/VzPvDsYffq1kUjLm0+YOrCnTuEqelgJRq/3sdx8zxpyxxL9irtgAHRD1V6rdRYLNMKQGqwM9pb4dlwTjIlKlUixLKN7fjGa7KfGNKfzZzoR4XYq4Rq1nx/bWwDiLckRgtWmhVNAI1l68rm3ZaxbDVLrN+yTQFKVMByiRgIL0f9Lc8yHsDM7JXVooR71Kxgkf7nz/X03jNeiol6FjpEIOObsQjv9dOQIUbdklfawbNJ/qS/KCqpxx2o+KNOMtuk5GDx7CB6DZcWXpfUDmMJw4DhsDBQoCSnnne6kN8Lc/fqu2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1PvDoXhNpQSv0AS8DYv80AMDNw8eCiT+reRuwP+9FhI=;
 b=qC4v2zWovjYwQYxqAG6rfZZkoNEEk79LhoDTTOaW1l/azRRqFDmG2QZ3ISdtF4lKmiQNG5Z6ZKnfq0U0oR/YQjOjCTeIskaCWLoOcabEhsCRE4qOSs2gL7C1YdrhVsdA8PccVZfbR5h+WYKsRq4xcxhKiEwr+/XHJ1fb+HEt4xY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v4 2/9] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
Date: Thu, 23 May 2024 15:40:33 +0800
Message-ID: <20240523074040.1611264-3-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD74:EE_|DS7PR12MB5718:EE_
X-MS-Office365-Filtering-Correlation-Id: 857a9ee6-1009-4bda-0eb6-08dc7afbaa47
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?fFIyXY5nh2IaKIY5JtE4otIB+TL2D7QyIMTc6XQhKjU1adF76KFbQXL7Bd4o?=
 =?us-ascii?Q?jsFp1ThFKMlFIh6R2v0cnjhwFNXcfg3UDwgrZdCOwyzyQZszt7HrZpKZ/HKl?=
 =?us-ascii?Q?gTAKwcadTSjEjhRSOzUn7kWQXonrqLdONgNpBoA1SB42vU+uaERhIj41qdMl?=
 =?us-ascii?Q?wc7JCnaXe+Dl65CNAHQQ4EvQAcQxFsaCXenh+AAY3xjDiAgP422tD9te0kdS?=
 =?us-ascii?Q?oCnmhuI5E+vgJlqu3Pqx9ri4waOpeUZW5yi8ymU01Tk8FWTmfv8TrTYOvE9L?=
 =?us-ascii?Q?vmPj2IBZe9y0K+jQfl1e7GNzjt2Pb+iEOFMgIc10hnq1cdHWk/MQoNojL4vf?=
 =?us-ascii?Q?iQlqn7xUwUZ3Gw7964X55sAkGxZvrK+auGL7rjY4F7RJ0AssayHr4v6yH1Fy?=
 =?us-ascii?Q?sqvNrPSLN2kFm9VB4rVIeSHA4cVlhNBHC8gysgkDAmzkncjEzUzOd+NkOrO3?=
 =?us-ascii?Q?YIVowO/vDEx0qk9ToCA6HRMm5BJOFze/1N63TU8C9qeXwwgdq4sFzhmBFkXN?=
 =?us-ascii?Q?kQIKyu2w/y0nuU/XmiYKXpWi9eLSpCVtFSWr9pphQDfuNRWQbPIZQjexSWYT?=
 =?us-ascii?Q?mYHKOfRPsPKwiRBXxUoYtIeF8QvyOACNuOk9SZ2YLQ5+Fcr1A7Pl9W1FYHeA?=
 =?us-ascii?Q?yP1AfIXKu7fcssmdQQ64QejV7+dicu3iSsgDYwDCaER4oGSNJqBalqix2U7I?=
 =?us-ascii?Q?Sn2Wbiqs4KB7xe4X6N1PhyiLNGlZfPGiz6SfPVw85BgHbpb3A88yLoIMAGXN?=
 =?us-ascii?Q?IVY66NYfrh4wUyspjBLCF6cwPVRYPFLwAxGuBc7xIseDZv8yWd2LJTqp0lZK?=
 =?us-ascii?Q?4b8or9EmSQSTAXOAb5EGsZO6bqe1LJ1DT1B/TA0N66jZkXn1i5AMe7+ptH+i?=
 =?us-ascii?Q?wkymHX9jnsp75uKbx9+XAkbLmif3VfjOPeeo70ssCZqgwHGyxtAqEA1BDS66?=
 =?us-ascii?Q?5zr0dYiKqhxkiCfmTBsMZTu/CZqVvAVq/nxz0PWNrel1SA61cMvfB+vCtH9Z?=
 =?us-ascii?Q?UgishP6mhlzVUY4qVRIeH9fLag3ZGLld8Ct+gsyHOy4eIOnN6ltKkpmtEiSn?=
 =?us-ascii?Q?m4fs6pxbApFin2gZRF/IJxbFF0dnN5dmd6lQXl2u3dDjU+zk478H36pdXO04?=
 =?us-ascii?Q?b6jBiJJ1GjBClWuJsuUdIbSFOscGXk4NPJHj90xniQ45r1hi2ucuIt7kBYrW?=
 =?us-ascii?Q?M7sBNZ7Je3oFFNGsQuvqOS43+nCe5mKInfdyZ7xLO1o9dZzFPIxC27dZEKZs?=
 =?us-ascii?Q?9OJFydYElttwLsO/MOtwT0RI426znyPFwFWlXlpNHJNTlIw9xSsxeTchdXdL?=
 =?us-ascii?Q?7bsQPQQyeb4Q7q50WHI0s3SZb501MLa3qCmTeTz+Pr0HK2pNXUKPm0JwJzVg?=
 =?us-ascii?Q?r7yISToomKmTX2UNRiuX47stH1FS?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:49.1150
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 857a9ee6-1009-4bda-0eb6-08dc7afbaa47
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD74.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5718

There are some use cases in which the dom0less domUs need to have
the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
example, the dynamic dtbo feature allows the domain to be assigned
a device that is behind the IOMMU at runtime. For these use cases,
we need to have a way to specify the domain will need the IOMMU
mapping at domain construction time.

Introduce a "passthrough" DT property for Dom0less DomUs following
the same entry as the xl.cfg. Currently only provide two options,
i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
construction time based on the property.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v4:
- No change.
v3:
- Use a separate variable to cache the condition from the "passthrough"
  flag separately to improve readability.
- Update the doc to explain the default condition more clearly.
v2:
- New patch to replace the original patch in v1:
  "[PATCH 03/15] xen/arm: Always enable IOMMU"
---
 docs/misc/arm/device-tree/booting.txt | 16 ++++++++++++++++
 xen/arch/arm/dom0less-build.c         | 11 +++++++++--
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..f1fd069c87 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -260,6 +260,22 @@ with the following properties:
     value specified by Xen command line parameter gnttab_max_maptrack_frames
     (or its default value if unspecified, i.e. 1024) is used.
 
+- passthrough
+
+    A string property specifying whether IOMMU mappings are enabled for the
+    domain and hence whether it will be enabled for passthrough hardware.
+    Possible property values are:
+
+    - "enabled"
+    IOMMU mappings are enabled for the domain. Note that this option is the
+    default if the user provides the device partial passthrough device tree
+    for the domain.
+
+    - "disabled"
+    IOMMU mappings are disabled for the domain and so hardware may not be
+    passed through. This option is the default if this property is missing
+    and the user does not provide the device partial device tree for the domain.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..5830a7051d 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -848,6 +848,8 @@ static int __init construct_domU(struct domain *d,
 void __init create_domUs(void)
 {
     struct dt_device_node *node;
+    const char *dom0less_iommu;
+    bool iommu = false;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
 
@@ -895,8 +897,13 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
-             iommu_enabled )
+        if ( !dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
+             !strcmp(dom0less_iommu, "enabled") )
+            iommu = true;
+
+        if ( iommu_enabled &&
+             (iommu || dt_find_compatible_node(node, NULL,
+                                               "multiboot,device-tree")) )
             d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728096.1132889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34L-0002Fy-6t; Thu, 23 May 2024 07:40:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728096.1132889; Thu, 23 May 2024 07:40:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34L-0002Fn-3n; Thu, 23 May 2024 07:40:57 +0000
Received: by outflank-mailman (input) for mailman id 728096;
 Thu, 23 May 2024 07:40:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34K-0001WY-60
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:40:56 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c93c26aa-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:40:54 +0200 (CEST)
Received: from CH0PR03CA0328.namprd03.prod.outlook.com (2603:10b6:610:118::14)
 by PH7PR12MB6882.namprd12.prod.outlook.com (2603:10b6:510:1b8::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Thu, 23 May
 2024 07:40:51 +0000
Received: from CH1PEPF0000AD74.namprd04.prod.outlook.com
 (2603:10b6:610:118:cafe::c8) by CH0PR03CA0328.outlook.office365.com
 (2603:10b6:610:118::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Thu, 23 May 2024 07:40:50 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD74.mail.protection.outlook.com (10.167.244.52) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:50 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:50 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:48 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c93c26aa-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GASANB7wRfDiBw1ZYy5MobrfNCEfRYjgsrg/RI9Os1jl+FYX8BtBBlsH2i6cm0Zx5rhfpcNYvoon0qGoYlky02pMMaAG7C7BlMh8+c2uOkoZh+fogGwJxBT18uvpXXqhqLRL/p70C75TwqHAKykez7Zzefbun15eQntzJm98y8yoT64NwCb+Am8oBuTqczJTcXXWq6T9vqHcxU1d3r/YNySJprxtD/pSXPrvfehaE7LxaZcacR/KCFCPMjiwxQabclPXi0DsOlMOrsVvmdiQ4wXVzEjAK5LZzyFiwf5RqNkIUQQMcnIWmJ/NuDA7dwWDPhR5WEtrcYh0pQkYOCg45A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=KLxhi2kA+DFIhfmJBOruoj4kiXc2RPsQrpkdr7sIoWE=;
 b=GVmSctfn71A0m1gonkGAMKOeFxZbt20ohEYTYxQCIPp1oP3L7lW8xv3S3Rm9dqRJrb6E8W/IImqt/oAlmrKTjJUShz7PAmfQcBPK2a4uz58cPPo1ifLksBODz1hmwbZ68L9l7VTGCoaYTekavu3/QGmy5cr4Ck4WxQN8Bj5R2GGDmqKh4TroRksRb3kVj78kRL+opHIleJ0Gl5uybTBZfud/tPYgkZytNeAI0BOKqM+84dSNp5h3Rh0HGEKplaRXnlJie6ZzOhKFDl5q+fv3waqRC2SAffuy9cYUPfxR9NaXIecEvL8rzQCI0sPsEbb7UOREOxv0euKyolsoSnMlPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KLxhi2kA+DFIhfmJBOruoj4kiXc2RPsQrpkdr7sIoWE=;
 b=aRguojq0DNlj0lcnHRcOhH6ezOWHKdVvcV/CDw4KPrD1YP5uaohbpDqT26/qwbSr5Jmn3St02CrnbPhrwAKjDTRO6fdGNk4DMET2i+xzmpr+Vei14dfgb4M0B+1ZVnWd2UCIQ2otTCapcJaaSRpKmRUcExijgJ0lLrwVB0oPymg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Henry Wang <xin.wang2@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v4 3/9] tools/arm: Introduce the "nr_spis" xl config entry
Date: Thu, 23 May 2024 15:40:34 +0800
Message-ID: <20240523074040.1611264-4-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD74:EE_|PH7PR12MB6882:EE_
X-MS-Office365-Filtering-Correlation-Id: ef06980a-a40e-4bda-512a-08dc7afbab4d
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?DQVn/2bkLGWMeex8Bl0oFAwDENUj5lUQVM7JhtUDpuqti28h65sxsB0XEmcW?=
 =?us-ascii?Q?vu9Ll/jnDvQt2y8gPiOSDM+kqrLse/yY9l4shPu3u0tavdDdN0ny0Fovzz3P?=
 =?us-ascii?Q?XO4vYyEFHd4COKWGtGJIY/vXuiFIlvdNwY+etuJYDnbshwKdKr/FwHz+d79E?=
 =?us-ascii?Q?k0XQstTzeJhaAtOANh72wnufn7Y55/SpfRPtOIZjj8El4mAreYGQRUR2/Vuf?=
 =?us-ascii?Q?N9ZMG/wcoNsXu9f9Wk6JQ72EG6dy3oExY0XYaDy/U7DEZu9n0ohqtSo2+fNo?=
 =?us-ascii?Q?St/QUCzYNbvTWwM/xThDEUpxi5+HTv8cX0a8EvVj0XaH3MrC3qmsXfhdDo8y?=
 =?us-ascii?Q?JE1q48EyV6jJYwFGqEcRomaccr1wLMWOW7rycs/ZuTc5Ee22l+GOerac/fUx?=
 =?us-ascii?Q?rq0gm1t6eEr1ESi0x2xWji5qBG+FOqlt+cBbmaX51/QKi0TtvoMdE31nBm3/?=
 =?us-ascii?Q?hmHZ5xcSC4jnE+z7HRiHpgQ1wc4Dy+J7DLh+/hKpuNYGJK/hBL4ew8LcNS2Y?=
 =?us-ascii?Q?J16lCobsWuMKWuhorTcL8hCQiAGiKG2xp90g4ArdCTPIuB36vLrDLF8FFB3b?=
 =?us-ascii?Q?BLAFjECIsS8qPv7RJw9STMKErZ3tjSQGZb5t9r4KweC4dd0GNBm2vn5qNK6z?=
 =?us-ascii?Q?rV+hzCMqvkgDepTCbjrLzVMShw8k37hSM34lMNV2CEEadENZfUIp7OHXGsZD?=
 =?us-ascii?Q?l/pbGiYaj9jPOi6RGX8O8MNgIaD9/mOJyHO6okRNPAnQqEdM2qTT8Bx6Uz6O?=
 =?us-ascii?Q?m8Y4f9/1zpTq9BpNAqb8FZ9EGUHzNRJkKzX/B4vneUSKUkHggvr99QXjIzjS?=
 =?us-ascii?Q?nrYoFBiBXdf/6cdl9uFLiI5mzqNzQhzeP9RR8v7DF99eOQFHOM8j94M1JFr/?=
 =?us-ascii?Q?VKHscBBu1HrkYTNc0unvXPzsI7QQWu2etFHUSx7A65q1AFt395qgd2Emgo+X?=
 =?us-ascii?Q?Ld9cGnRBeCt2HK5dFTfI5uv+PCygWP4OPLsYH/qf5515ANqP9FTmVZVAR7N0?=
 =?us-ascii?Q?UVgwlScJskf9F6cavBKbnGSznmxfN1OKEiq0lg4/BuAVnKQPu7yaIEC4ycxq?=
 =?us-ascii?Q?8mXrXtLm2Ctrgxu01DFziwjXHpQldO7b4y3pOjSd5rVEnFd/mu5wIfG/GdIV?=
 =?us-ascii?Q?w7ydpqv6YMonOKAiHtMQIW/OYz8fyJm3RU8DsswCWOYUGX7sXfa74orrLJlJ?=
 =?us-ascii?Q?FG7yn8CcHYj32AFNJY4W59+aXgH+NSPacbLBggyQCuf59WT7a5HSP1tFH34o?=
 =?us-ascii?Q?D8j+W52GJqVgG0gLPR8sG14DWplVOnvaJ9oXxzknj1EM+9d+gSfsp6SpmJnW?=
 =?us-ascii?Q?IpRVXokmg06Iq3R09mQ+Xx8DPHYKdCe5zm26dM2Tlpvtc6S7i7dMyuhuLyhs?=
 =?us-ascii?Q?DqcrNmBSJiGCwkGht0ZC078NUjjs?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:50.8337
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ef06980a-a40e-4bda-512a-08dc7afbab4d
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD74.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6882

Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.

Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.

Update the doc and the golang bindings accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v4:
- Add Jason's Reviewed-by tag.
v3:
- Reword documentation to avoid ambiguity.
v2:
- New patch to replace the original patch in v1:
  "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
---
 docs/man/xl.cfg.5.pod.in             | 14 ++++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/libs/light/libxl_arm.c         |  4 ++--
 tools/libs/light/libxl_types.idl     |  1 +
 tools/xl/xl_parse.c                  |  3 +++
 6 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..416d582844 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,20 @@ raised.
 
 =back
 
+=over 4
+
+=item B<nr_spis="NR_SPIS">
+
+An optional 32-bit integer parameter specifying the number of SPIs (Shared
+Peripheral Interrupts) to allocate for the domain. If the value specified by
+the `nr_spis` parameter is smaller than the number of SPIs calculated by the
+toolstack based on the devices allocated for the domain, or the `nr_spis`
+parameter is not specified, the value calculated by the toolstack will be used
+for the domain. Otherwise, the value specified by the `nr_spis` parameter will
+be used.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index b9cb5b33c7..fe5110474d 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
+x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
+xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 5b293755d7..c9e45b306f 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -597,6 +597,7 @@ ArchArm struct {
 GicVersion GicVersion
 Vuart VuartType
 SveVl SveType
+NrSpis uint32
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1cb89fa584..a4029e3ac8 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 
     LOG(DEBUG, "Configure the domain");
 
-    config->arch.nr_spis = nr_spis;
-    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
+    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
+    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
 
     switch (d_config->b_info.arch_arm.gic_version) {
     case LIBXL_GIC_VERSION_DEFAULT:
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..4e65e6fda5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
                                ("sve_vl", libxl_sve_type),
+                               ("nr_spis", uint32),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..e3a4800f6e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2935,6 +2935,9 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
+        b_info->arch_arm.nr_spis = l;
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728093.1132860 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34G-0001Wl-Bm; Thu, 23 May 2024 07:40:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728093.1132860; Thu, 23 May 2024 07:40:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34G-0001We-8o; Thu, 23 May 2024 07:40:52 +0000
Received: by outflank-mailman (input) for mailman id 728093;
 Thu, 23 May 2024 07:40:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34E-0001WY-HU
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:40:50 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20624.outbound.protection.outlook.com
 [2a01:111:f400:7e88::624])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5d858ae-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:40:49 +0200 (CEST)
Received: from CH0PR03CA0210.namprd03.prod.outlook.com (2603:10b6:610:e4::35)
 by SN7PR12MB6672.namprd12.prod.outlook.com (2603:10b6:806:26c::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Thu, 23 May
 2024 07:40:45 +0000
Received: from CH1PEPF0000AD76.namprd04.prod.outlook.com
 (2603:10b6:610:e4:cafe::fd) by CH0PR03CA0210.outlook.office365.com
 (2603:10b6:610:e4::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Thu, 23 May 2024 07:40:45 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD76.mail.protection.outlook.com (10.167.244.53) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:44 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:44 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:41 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5d858ae-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eBUFhwhuQjb7+8X5X6R4kmafl+OL6+/T+s7/7QRLM8LsMPe7aAnQYQce5Uceckjnm4ulN7kPX90a87WaTQg3YIpy2IZ0fkW1m+b0yAjfRtxlGUrIwx53tWq6G7C0/HVKwD+rQF0vkz2BXkAGpYVPtI89FYg9ewNIFnqANypiqMgzLOedgXgbtxGZED90lYmpJVzoPBKf6qfBpG3Gjr7JohMBdQ+DhZA7ai+Sylo+N6gg108yNo0fxISl2O7bVpzjivIxMsqblngHYBZ8Lpf4RP8EgA1PSL6/x1emsTyH6+w5sMJOFEjiMH7CHR/i6lIC6f6fX7uVEhxtfHlrBV/UGg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=s6GY8LYUMBjl94igM9pP1TWRNhnTdORD7Nqir12syYE=;
 b=GD0yVgQJvqm2A6K/aRhqKWts4b8KpkSQIe1W6qG2WFmQGYlviHmqhyFwhxeh1KQUMVS83InoXJOBN18dMk0AXlc1y1HG0DjqfXomIfEXbjytlXX7kDeds6vS38Lc46zFu7Ygt5K1OjDsGVRoW7IldNZ3RnooEa7KTSrjfnQAflGPOqkillkMzd+MCqfrNo1jFHj5gQQpH0MQh5GbO2eH9N8kkHSQCmblknYFiI1UUH5wIabOb4ztLxfjar9Z3E6+eR3bOG1RuhrsX2ZSgNKlpJ7gKuLifuf9auQ1K+zhrldHAu9kU9PEoU+eJWMrvVjyNESO+iLCdUbhi977ahg6lw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=s6GY8LYUMBjl94igM9pP1TWRNhnTdORD7Nqir12syYE=;
 b=2/xt8J4PMWfwWRhXHPSqdOsc9VTic6XmrzYqH29ztZa7YUXdNSwdzfZjesVPQwBIZhGnzRUFpL/nV121VS64xace/RAgkaUabUyDLhbZI2m+IkXujzCa53Sgh5I9ZPpJszbUxm99E5g1Rgq2tWk849FIejbouwhu6dfcVcwwWKY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH v4 0/9] Remaining patches for dynamic node programming using overlay dtbo
Date: Thu, 23 May 2024 15:40:31 +0800
Message-ID: <20240523074040.1611264-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD76:EE_|SN7PR12MB6672:EE_
X-MS-Office365-Filtering-Correlation-Id: 7468e249-a0b4-4744-f97f-08dc7afba794
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004|7416005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?NBmC6hn3LQhREzPCW69Tq4ZqhcLA27uy65HuAVw65egaUcvJbqJgrntnyCRk?=
 =?us-ascii?Q?HsJzFsHdOD7aCDOThMmvuxJtJmjpVQngcjU32KuNJPws3hp8gPyDZl6oj5V6?=
 =?us-ascii?Q?N1OXRGJXYVxE4ZAoVBRSkwaB+PP8/fZc391NxIht8ha6T+y6/geYT6uma02M?=
 =?us-ascii?Q?Fy9jXd8Qw+6qNpgU49dlDC8Vrd6Kicoj898WdLPV/p/K5w3hxVw1skA6olSO?=
 =?us-ascii?Q?dSk/HPrPJrPIwj2dRTLGI0vxMG8PR0XEDEnLfswEctraVmE0SzawC4vz0784?=
 =?us-ascii?Q?nmseRuqq+qcMJmfQHOvuvDX14sGrlHgl64ECA5K1NcRnipVk+jTDUfU3Km1p?=
 =?us-ascii?Q?6bYYsHFBga6vPuj4iqFiwM1Je94bykjj4enWGf+7cY433Qa3uzVDx+cQLhk6?=
 =?us-ascii?Q?z4Nbx+fQMgQGsHuAtBFceUl4i5aW4yXIILeUraAQaIiB7mvMuByTJp06yO+m?=
 =?us-ascii?Q?oJCEYH3+6rMLtqZWITKTrn5ucp6QwcvweOXbfUz1HMDuWGmg9w/HEebVfjbm?=
 =?us-ascii?Q?LIlJW6W6ld7E+5EtPLB/Tc4fK6CvGlC05TEskK5ci33O5sGVVtTO1YTer5he?=
 =?us-ascii?Q?bNSuFewWuf9PTZHL8yQcCO7ERbCSpE9Ftnn3mk/R9SvMYrTTYtPcgHTMJntu?=
 =?us-ascii?Q?ij+G840Vzl+WRv1xEwOyjZCTFXrpFA+O1dlwoTyk7i5gLiqowOSTIBrbNR8H?=
 =?us-ascii?Q?37o4GbzdBkMmGsEFGLZKjgwJQgmp8ciwZyP0jvEaqlNxTm4mG7GViwUoLIdx?=
 =?us-ascii?Q?c/ueOkNJAbr3kxgl0O0nkR0fwNoMyn7JHLROK7lFjfAefe6u/pSoQRcvOL+F?=
 =?us-ascii?Q?dGx0f4J0LP40imynIOVELoU99lF2lXAm+eTOmGZIzfGXVz1Xb7+2XL/6IrlH?=
 =?us-ascii?Q?5B05esIR7fEcpmwQbvruKA9qZpTYH+mpSajFGHnS49fTxlmtc2GCgmGtZahn?=
 =?us-ascii?Q?w59RZWd3Ky2/EWUZ6EkaA0QNxxPtoirF7yjPLdC2faeZAOwfFfpy9SrY1UBA?=
 =?us-ascii?Q?o/ajC+Xn5Pl6TU/I0RgG0FgAoKuId7PlCtf8rN/RlTGFeg1FBdJMMT7cfJ03?=
 =?us-ascii?Q?VKxKU+39alBlxqAW6NkSFy1D9inRTSAjkHLUr3Poqm2IITT/7lWxczY8Qt1+?=
 =?us-ascii?Q?ud76zRIpk2IWfRRrtCbDVnhl4plrU3CyCSeOYnAadbXtZVJQCcgPaAgOIrEd?=
 =?us-ascii?Q?Xbq/8PizAW5YiwPC8HbfmT8k+TSbnUju5lQC05u/N36sG2luCW3Fau9a7IT3?=
 =?us-ascii?Q?rJ83A3qH2saH6Jq5d756Q/NLZDg4SFKNeeMRnw4Cep/bTFiUp31Mb+fXY6xW?=
 =?us-ascii?Q?bdtAzLXaDbFiF3x/2C+VnVHtIOA9iZhf57UM5kflcMqibA=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:44.5924
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7468e249-a0b4-4744-f97f-08dc7afba794
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD76.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6672

Hi all,

This is the remaining series for the full functional "dynamic node
programming using overlay dtbo" feature. The first part [1] has
already been merged.

Quoting from the original series, the first part has already made
Xen aware of new device tree node which means updating the dt_host
with overlay node information, and in this series, the goal is to
map IRQ and IOMMU during runtime, where we will do the actual IOMMU
and IRQ mapping and unmapping to a running domain. Also, documentation
of the "dynamic node programming using overlay dtbo" feature is added.

During the discussion in v3, I was recommended to split the overlay
devices attach/detach to/from running domains to separated patches [3].
But I decided to only expose the xl user interfaces together to the
users after device attach/detach is fully functional, so I didn't
split the toolstack patch (#8).

Patch 1 is a fix of the existing code which is noticed during my local
tests, details please see the commit message.

Gitlab CI for this series can be found in [2].

[1] https://lore.kernel.org/xen-devel/20230906011631.30310-1-vikram.garhwal@amd.com/
[2] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1301720278
[3] https://lore.kernel.org/xen-devel/e743d3d2-5884-4e55-8627-85985ba339f9@amd.com/

Henry Wang (7):
  tools/xl: Correct the help information and exit code of the dt-overlay
    command
  xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
  tools/arm: Introduce the "nr_spis" xl config entry
  xen/arm/gic: Allow adding interrupt to running VMs
  xen/arm: Add XEN_DOMCTL_dt_overlay and device attachment to domains
  xen/arm: Support device detachment from domains
  tools: Introduce the "xl dt-overlay {attach,detach}" commands

Vikram Garhwal (2):
  xen/arm/gic: Allow removing interrupt to running VMs
  docs: Add device tree overlay documentation

 docs/man/xl.cfg.5.pod.in              |  14 +
 docs/misc/arm/device-tree/booting.txt |  16 +
 docs/misc/arm/overlay.txt             |  99 ++++++
 tools/golang/xenlight/helpers.gen.go  |   2 +
 tools/golang/xenlight/types.gen.go    |   1 +
 tools/include/libxl.h                 |  10 +
 tools/include/xenctrl.h               |   3 +
 tools/libs/ctrl/xc_dt_overlay.c       |  31 ++
 tools/libs/light/libxl_arm.c          |   4 +-
 tools/libs/light/libxl_dt_overlay.c   |  28 ++
 tools/libs/light/libxl_types.idl      |   1 +
 tools/xl/xl_cmdtable.c                |   4 +-
 tools/xl/xl_parse.c                   |   3 +
 tools/xl/xl_vmcontrol.c               |  48 ++-
 xen/arch/arm/dom0less-build.c         |  11 +-
 xen/arch/arm/domctl.c                 |   3 +
 xen/arch/arm/gic-vgic.c               |  36 ++-
 xen/arch/arm/gic.c                    |  17 +-
 xen/arch/arm/vgic/vgic.c              |  31 +-
 xen/common/dt-overlay.c               | 438 ++++++++++++++++++++------
 xen/include/public/domctl.h           |  15 +
 xen/include/public/sysctl.h           |  11 +-
 xen/include/xen/dt-overlay.h          |   7 +
 23 files changed, 678 insertions(+), 155 deletions(-)
 create mode 100644 docs/misc/arm/overlay.txt

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728094.1132869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34H-0001lE-NG; Thu, 23 May 2024 07:40:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728094.1132869; Thu, 23 May 2024 07:40:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34H-0001l7-KE; Thu, 23 May 2024 07:40:53 +0000
Received: by outflank-mailman (input) for mailman id 728094;
 Thu, 23 May 2024 07:40:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34G-0001WY-PD
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:40:52 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c73155d3-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:40:52 +0200 (CEST)
Received: from SJ0PR03CA0272.namprd03.prod.outlook.com (2603:10b6:a03:39e::7)
 by DS0PR12MB7509.namprd12.prod.outlook.com (2603:10b6:8:137::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Thu, 23 May
 2024 07:40:47 +0000
Received: from CO1PEPF000066E7.namprd05.prod.outlook.com
 (2603:10b6:a03:39e:cafe::d0) by SJ0PR03CA0272.outlook.office365.com
 (2603:10b6:a03:39e::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Thu, 23 May 2024 07:40:47 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066E7.mail.protection.outlook.com (10.167.249.9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:46 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:45 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:44 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c73155d3-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TXdzLE+kKH46u9Sng84dpWlCTRQJI357cr0AvkengcQnhlsrf5Iwemeh7WjtNCl+oVmvyaOl+qRo056poVrfNWXIYqMuDkUyZNhExvVXeiGUK0NlJ+szO1eTUux0rxa1nrSuFTBhixg8+ls7qMOtkMPs0fkXS2Uny0J19MlrVILVjVAPMI3vp+to+wGDURkbvXSbl3BScWM3vpuOCr7azFdDV7Xg2OBIBeh9x4/93YjS6OKYXFWLAGMMSW8cDIBLF9m7y4KdPa/op6HPrPd+2ni4N4dh2OUfsgNYJHgO179kwWPMfBZR4BtlniN1+bT1AyOFT6Q+UFu6CID2JuMPvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=r/W168QEvQVYTC09iGsQQFU2oTMKOq6gSMJXAiUxem4=;
 b=NMaeuc0ruls78XaqMFJNAgAvtcN6bNBfxoJjmuiCbICes0/+QLC2obkCdpPQoVBhQL0ENQfje4cRmfZFR9KgqniUMpebFrkOOz3cFsZrqlmcF31v1qWJ5Rs0Ll3JbQypn8xlwTwasDpzPBUxDLNiwvc5y5Yej4wNOlhb9oYIcEuUVTSIlWLDMICcgpBs8uGxpfN2cYo335BptrilvzGoDOBd0yMxzEbWl0MlhPL1GU/fM7VMJviBrWDKUtfm9AtIS01DhceT3L80FxcC2t133Lz6riFYjAbvLyc6LS1EqgJ4WRhMDqh1CvGnwLuuxNo0t5LUrztUNWrKLCEPqUA/mQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r/W168QEvQVYTC09iGsQQFU2oTMKOq6gSMJXAiUxem4=;
 b=TXD8tE0v34Mn88tGEFcaqrfSKdZLPHpYn6pij5jvSEULaSUd5NfphWjyhYC4Wk8RFLc6Cjdgs0+Z9W0yFOh1XcOvIvj/168jlMKJmYT9dfPb/Fa6rEtzuSsdLyE+tntQWY84HDSRhDRTpYEVhelkQq3AVSwuDUbD/3tXMgFEHXA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Henry Wang <xin.wang2@amd.com>,
	Jason Andryuk <jason.andryuk@amd.com>
Subject: [PATCH v4 1/9] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Thu, 23 May 2024 15:40:32 +0800
Message-ID: <20240523074040.1611264-2-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066E7:EE_|DS0PR12MB7509:EE_
X-MS-Office365-Filtering-Correlation-Id: d377181f-5b2f-445f-d9d5-08dc7afba8e7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?aBxdqU1m92P565gZjrLCwUTUZ7iEh2pilmYFgRhCAGqcNpKQ2StZPIxzzdVd?=
 =?us-ascii?Q?96H/MB6e+5lA6zQaY1Ygk0zcz0wvfAuc4PudQL6b7spKqWEZZ84fHVWih+/f?=
 =?us-ascii?Q?76YGFzs14idDEeONJXl8x/+TFsqcViwcVpUoEHIQz4n8qwccKhdCxNPLhmKh?=
 =?us-ascii?Q?qvVaLAC4GEHySL3+qKLA7uAara81nnO+rPGhzJa4kKrTBlWfz6/g8Nz2JsMl?=
 =?us-ascii?Q?lz2HLILzJlxEHYGtZyfvoZLF9Z1sb6c91J4jUCy7SmpCRcKF42HkxmISBx0u?=
 =?us-ascii?Q?XDoUkJd3LaHZm3mpQRnQsQVKpb6zNez9gKi2HSLipryEgnyr4sEh802JgkBV?=
 =?us-ascii?Q?wYWnCrGZNF1jozBkvdFXGsZAFPM2duvy8tdWzQluoIfdQw7P7Tr9+njnFoXM?=
 =?us-ascii?Q?aLdlY45m51BjgXsjZAtPBixECkcedvnTfQxuuFgXmRAz+lgnefSYOJ84Se1h?=
 =?us-ascii?Q?477hGdGmXXoocAS4FQTeuoUJ9RxRI3xIbeVPf6oC+weWCzOqfnrH9zjiBqB8?=
 =?us-ascii?Q?9rv3I5PkWUJVOuk0jX7c3JyJ2KRvra0s09OGoi6YnTQ/OzWJ0OcHSn31kL9K?=
 =?us-ascii?Q?q2mzNIM65dbeMa2D2BDvBasOLyjFoKH1bIVFPoYmyK5bCpUaHlALXct3Ck9+?=
 =?us-ascii?Q?8UMfhnixOiX/EpU45+P/NwvqQXjg4/VET6KCcPTpvKtWFOHqVnKiN8E9r3Lo?=
 =?us-ascii?Q?prWwb6iuuXQkww4Y2mPmpve4yndQyXNp3IvN6JtFADjGskdtnZraZqmuEAbU?=
 =?us-ascii?Q?w7y6Gwo3QFTpvGVirVQa4ba13obncpsabH/5DEugocn4eyjegc8YB9rk9xRk?=
 =?us-ascii?Q?IhOnIyeOiSI5X1eWxN7mYThjkanxeUadYfqpKSu+99l8tlw+UW+Eci97Yr/L?=
 =?us-ascii?Q?+S+LrCEEdfbAD/a+MW9f4Q9Xe0qYo3LOn8WEG48DbsW3623Bk8y7yVawC5Nf?=
 =?us-ascii?Q?VCFGnTlnsC5eNIhqPbCAXxXlqiwVXgDfh9kcJFJWEYijHjJUS8sWG1pZbhiN?=
 =?us-ascii?Q?OH/5e/Sim4eQkCHRWCdmRSRhE89HnvPlnBXVtAbnkr2DAqcyWvjIiZc8WC84?=
 =?us-ascii?Q?gHjbM/ud/tf9RPk5RN8RJ5tOR7mWC+b+DM0O75iYebIuysx6GMv2nSIgQdpA?=
 =?us-ascii?Q?RYzFoFkm94etGITtytCUNlhjcU4g1bMqcqRKeYqUnOgtrhhpmfHEP523D9P4?=
 =?us-ascii?Q?nniGe3UZDHH/Pj4tyhKOAnJIl1FSgSZoe/TsnIww0SRQ+WHVwfdwz44P+zvm?=
 =?us-ascii?Q?jaGNIDUiA6+D0923XI/AlDwjBNneGhUhl6XUIUUgR7hvfpci/YNRdvG96VDr?=
 =?us-ascii?Q?ma2RNU9wq+unOYC9luT8f9w05OpSDAxXMp0gcpcCywDKtRaLhZ3tAeckhHTc?=
 =?us-ascii?Q?Dm5ZfbE=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:46.6883
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d377181f-5b2f-445f-d9d5-08dc7afba8e7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066E7.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7509

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".
Add the missing "," in the cmdtable.

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony@xenproject.org>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v4:
- No change.
v3:
- Add Jason's Reviewed-by tag.
v2:
- New patch
---
 tools/xl/xl_cmdtable.c  | 2 +-
 tools/xl/xl_vmcontrol.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1f3c6b5897 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
       "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>"
+      "add/remove <.dtbo>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728097.1132900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34P-0002aR-Ht; Thu, 23 May 2024 07:41:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728097.1132900; Thu, 23 May 2024 07:41:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34P-0002a2-EB; Thu, 23 May 2024 07:41:01 +0000
Received: by outflank-mailman (input) for mailman id 728097;
 Thu, 23 May 2024 07:40:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34N-0001WY-GQ
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:40:59 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cb2aae42-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:40:58 +0200 (CEST)
Received: from MW4PR03CA0214.namprd03.prod.outlook.com (2603:10b6:303:b9::9)
 by BL3PR12MB6643.namprd12.prod.outlook.com (2603:10b6:208:38f::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Thu, 23 May
 2024 07:40:54 +0000
Received: from CO1PEPF000066ED.namprd05.prod.outlook.com
 (2603:10b6:303:b9:cafe::2f) by MW4PR03CA0214.outlook.office365.com
 (2603:10b6:303:b9::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Thu, 23 May 2024 07:40:54 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:53 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:52 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:50 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb2aae42-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BAjQNZm/ATs7jCaRlEhbS7Nw5C78kdhdWkKY+KmIhV0cX4NtiF4QPSxtc/Xt1IlVT1SuyzkvX7TGHzG53LrMNaaViHLRUpU0mlKeP7YbF6oj22yNuCYk3lVoSdcdbB1AapQAuNQTXK179rfvCPqaErhQEprirOG/RWw6HohNeA1MK0p9q2wTIh13kGu5rRM05QlhqM+jIULWq/lfgO+lQJKhdiGClRpc95ExraEtCE28rf60mUBprBpGB0/ib1wEF6Qu/onkct1rtdSHwbf5ZAlEAzB4ukVqTs5VLUYvFQmfTVcV7Mw/UEx4SHxtfFFztdAnxPfyStUCdWvz0Q3eyA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=bJ4rFJutdXy/qfPZPTvRAO+qEyMuCy4lZffUC1ijz5Y=;
 b=TXoLpKg7Xbd+z/nzO7Ngb4nWAkG/qN4DWA0a215/cp+RLLFtwSTWOMnYRjjIx4GuM+azFIJGNW5IEXZiCmDyClIZpHUgyxn4mkfOcTZhV+DksfkBZujGbr+t4QBogaGdN8QNUFKoBUlHd51bIpQY0ZbDDqWuPovPzSkC1jff6RVAJvD4zyeCbo9nm+Rb4SSWA+V/Cch8/6tz8Fo7xYjH09ytMKx3Iq2+9gVEHj5I93gEwvMJLoYBYFRcAS4in+toRkm2x8DkypnNCEheX8qvdNJ9u7Q0+5SYuOACBBDtgQYPQXjZvxl5y4lpI7AniOGneFgaE9NiFpwtVJGydDuyBw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bJ4rFJutdXy/qfPZPTvRAO+qEyMuCy4lZffUC1ijz5Y=;
 b=uiXBZP2Tg6MRcvKJnEBlCwVRNe4gcODnm8nXD4nDZveMFnwiD38u0jS3p83CTVGJABLAbe0NGgAe5N8JAGF3SQ/JduAOVXvoxK2GIWQLsqhE3LF0IxATU7Hr0DNE4Bc7kozfff1biJeCyYebL5xv28ReFbBjK+8QdJL24u7lCXw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v4 4/9] xen/arm/gic: Allow adding interrupt to running VMs
Date: Thu, 23 May 2024 15:40:35 +0800
Message-ID: <20240523074040.1611264-5-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|BL3PR12MB6643:EE_
X-MS-Office365-Filtering-Correlation-Id: 7161e61b-6859-4b92-08d0-08dc7afbace7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?LYeGWke6n4bX49lwvxP85Ozwipg4qx2/aH0d7Un85TTkALAGeAp4Uj5QfGTD?=
 =?us-ascii?Q?BfXAIfeb2c8gHu8XC7YPqQ9qZG8KX/EawG9bDJQMLgqZ4KPkKPappLluKt5C?=
 =?us-ascii?Q?AgerW39/7TUGf8t5Y+fXeV4VXQQO5/JZSIFm25T/Y/ucrZF38YVNdMNkuLZe?=
 =?us-ascii?Q?gOov3WIXNDTblsPBoPfD6d/2eO6jgSjacGhALoZF3TX9v9xc5AFInBXp4Zj8?=
 =?us-ascii?Q?GJDwdGBm+uqYi6CcZU9KhKWpTo6hevZ9GTpBe4Tuge42MuyNTlyueZ9TEmYO?=
 =?us-ascii?Q?iYkGzRLajFg6Bh8G/Yszul9MU7U8Bi0H0tdnCZEsGg9oxpWHlsJK4DzKzOPq?=
 =?us-ascii?Q?7vm/S1HchCdNgZRrZMsCUHtdU5AicYbqGSLNia8F+xpd3VWvXMlAQijetkmv?=
 =?us-ascii?Q?YunXb5pXNH6uGgnNEU7T7jcc65xyIg0m3YyKiDHwgAbLTNDkEkEG9PHjRHjR?=
 =?us-ascii?Q?9wazh9bVULpB5nhlKkZEk5WZ51hHO/QkLrpLAGzXavYS/SbxjcT1mrpydE/1?=
 =?us-ascii?Q?EDM5Xx1fkOj1qPbF69+KqWGGShYaPH8s8BdNydApGnL5ntdnAPA4fH1Npe7K?=
 =?us-ascii?Q?LXGQvAlgTMZuOWzvpcIZn5GW1OzR0NP71FOYZgzp21ZDUDM9JODeHjofqkhm?=
 =?us-ascii?Q?UnHit6AYprwRXKvVz25deIdjPh1gMWndRMPwY77O5P4BdiHg1dpfXxWJ55O4?=
 =?us-ascii?Q?npal4ZNXr6odCKF9ZMzLtSGOMaha28QZ0eQgeQzldViIUypkwGffNrMbYmka?=
 =?us-ascii?Q?zvHqjnjgLclZna94RcdqD6UUOp+utj6igelleEyeynOz6qwVy0uoJeWd9Txc?=
 =?us-ascii?Q?KDAzY0onyLS4eOF3M9qpYHTFBCYaeBe2Y7FIOlOeSyEk/trjOJ78tNjJbkbZ?=
 =?us-ascii?Q?tpJ85VDvnk+QV6Zl1gJgtpjnpHRg8XzjW9rcmy6YD4DEZuXRQZ8CfEl5JbRb?=
 =?us-ascii?Q?nCiRH9ri3kjhcfmzXyBj6blllFFKlBe5Hh0lumyVJMFZ/+RFb87FNs0r2qcD?=
 =?us-ascii?Q?oV5r6FFjPKfI4mt6ZZGFUVw5oh3SWKahKKc0pRbyk1YIsgTWuJ6R6nbrNGb/?=
 =?us-ascii?Q?Oz3RC7GeMjRjewoHrCqzbgi7JhFI/vFVszGqgEfZ+64BymI2ET0nnui8bLKo?=
 =?us-ascii?Q?hML2YMb7Vq1mvqvUa4X0+AGeR70eZqM7pi0ANYWxF8DJ4Zmjg93W2/RsLlwW?=
 =?us-ascii?Q?xG5qNhAqTiCstsqIiIkrMjDFc5BvuSDfZ9i6cgMTPg4Mj7ClG/wPJHGRJISK?=
 =?us-ascii?Q?OVylkfwkqk1hud1MsyWXnAYvYqhn6eoIbds+e75u4aW4VWwIylncgJpH9sii?=
 =?us-ascii?Q?L4qerWCg1U2NjuMY2nkFg3UaWqate6R1U90Idcib8KwhocfbFjzFzCCs2LcM?=
 =?us-ascii?Q?zxpfzHbbPSX1KkPmpZYCjuDtb0Kl?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:53.4407
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7161e61b-6859-4b92-08d0-08dc7afbace7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066ED.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6643

Currently, adding physical interrupts are only allowed at
the domain creation time. For use cases such as dynamic device
tree overlay addition, the adding of physical IRQ to
running domains should be allowed.

Drop the above-mentioned domain creation check. Since this
will introduce interrupt state unsync issues for cases when the
interrupt is active or pending in the guest, therefore for these
cases we simply reject the operation. Do it for both new and old
vGIC implementations.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v4:
- Split the original patch, only do the adding IRQ stuff in this
  patch.
v3:
- Update in-code comments.
- Correct the if conditions.
- Add taking/releasing the vgic lock of the vcpu.
v2:
- Reject the case where the IRQ is active or pending in guest.
---
 xen/arch/arm/gic-vgic.c  | 9 +++++++--
 xen/arch/arm/gic.c       | 8 --------
 xen/arch/arm/vgic/vgic.c | 7 +++++--
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 56490dbc43..b99e287224 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -442,9 +442,14 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
 
     if ( connect )
     {
-        /* The VIRQ should not be already enabled by the guest */
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
         if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             p->desc = desc;
         else
             ret = -EBUSY;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86de..b3467a76ae 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
     ASSERT(virq < vgic_num_irqs(d));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-    if ( d->creation_finished )
-        return -EBUSY;
-
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
     if ( ret )
         return ret;
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b9463a5f27..048e12c562 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
 
     if ( connect )                      /* assign a mapped IRQ */
     {
-        /* The VIRQ should not be already enabled by the guest */
-        if ( !irq->hw && !irq->enabled )
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest
+         */
+        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
         {
             irq->hw = true;
             irq->hwintid = desc->irq;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728098.1132910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34R-0002tB-Ro; Thu, 23 May 2024 07:41:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728098.1132910; Thu, 23 May 2024 07:41:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34R-0002sv-Mm; Thu, 23 May 2024 07:41:03 +0000
Received: by outflank-mailman (input) for mailman id 728098;
 Thu, 23 May 2024 07:41:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34P-0001WY-GT
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:41:01 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cbe9361c-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:41:00 +0200 (CEST)
Received: from SJ0PR03CA0271.namprd03.prod.outlook.com (2603:10b6:a03:39e::6)
 by CY5PR12MB6059.namprd12.prod.outlook.com (2603:10b6:930:2c::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Thu, 23 May
 2024 07:40:57 +0000
Received: from CO1PEPF000066E7.namprd05.prod.outlook.com
 (2603:10b6:a03:39e:cafe::7) by SJ0PR03CA0271.outlook.office365.com
 (2603:10b6:a03:39e::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Thu, 23 May 2024 07:40:56 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066E7.mail.protection.outlook.com (10.167.249.9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:56 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:55 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:55 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:52 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbe9361c-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YgKPuDp7h7POl4wtY17dpSFPp8a6+4Lys+iwRMefruh2Udblq2blnNdDZi2gOiLTJUITY4Pt30iHICpvdu/IY45SLJxaonu2k56Nvtlcc2zkyH4fR7+F2SV2lm4pu1iQ0bAGDkbYbLp4KQCBL2pBRKC1MQ5j/hbMnaJX9OZivvlyyD4QIwrRKYg6VdSW624ucIJCCvaQ/E3Nfq2O/sIBjAebwwvhlXO/YjW+3HO4zenxkD3pN7vnBlXlyGeHyhY4jntT/HSyH9K2I1VT6bF4Dja1tXBj2pVOYYqvghCUZFXvc6doUtbVK3InJ7Xi7NdyaJ8M+146exxfVDx/GcgcJg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6cGqHcXBoZTzyZBk/4TEkkSbTo+NOg9YMZd/62zSuww=;
 b=RBlHBzNTiHNVbAyJR/b4bLst0YADu4Ycino6UouZKWc+a32S6KnktaRy+HkL+wBuyLkQU44MdC0YXbpn7Jw0hhz5RBhs2zuzrZI2cG1blozJcZr0Te6yqw/3NUIk8vEP6mfpJDQUyjbmzrQVrYCjFWyG8HtgyVYH6hhhwJd5in2zvU+1ZwGbtZQy5DH0nrQFOYrLhPWJzzaVIoY+hTyn4DPYtcC+q3/WJdymv1+D9XHU9yos9RfxvUXiXUJ9EnRYOt+jPBCgqv5LA92mAWzhfLDk4+IPAGy8m8iJCZl1sdw7xKVUdCcSZ+hdT3DiK7To42ANjoRCkoDkvuYN1jiujA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6cGqHcXBoZTzyZBk/4TEkkSbTo+NOg9YMZd/62zSuww=;
 b=xJskQFAZddZKAwcQxfx5spyajeQxOAUZ5CLC36EHTuG3H+tvT9g5A8R8BMNF7673A55ScT+DapVPdqY95hX84GlS1FcxFOdX3tbmIETcnb91ATT50UnPRkfKbGn9joKxDy/hbTtNDfXEw8PRM3KHl+ZS/DKxxHpZ1QwpgcJyPo4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v4 5/9] xen/arm: Add XEN_DOMCTL_dt_overlay and device attachment to domains
Date: Thu, 23 May 2024 15:40:36 +0800
Message-ID: <20240523074040.1611264-6-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066E7:EE_|CY5PR12MB6059:EE_
X-MS-Office365-Filtering-Correlation-Id: 45c5f866-43a4-4ba9-97dc-08dc7afbaeb7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?lecmbsste4sgjSXM7CrLlaewWO9a0Gm4oDdXUun5hp1CNdD9qi9oXY/NcOhP?=
 =?us-ascii?Q?b5Aoly6URh3mwihLbtCIQmfwlpGGROZiV8qVCm01V9KDF/+8IjJGjGuDuecS?=
 =?us-ascii?Q?7+hLjf+xuKlOO7yj3eP4wMqgb3LxaR6j0JVSUO9bPa33vfjWa8zmS9E7pcrB?=
 =?us-ascii?Q?U0NY0a+0yXVsvVRR996oR0rCUp6Pky9x7oc7S6JkZfplifNdNbxdEz6zMt+D?=
 =?us-ascii?Q?/7gJg2cNkFXmGsHsp4NYq4Ji3Oa37H5Bb7Y3dfBh/dWjPt2Yl393SUZSoIyW?=
 =?us-ascii?Q?4MHU3Wz3lgvseYlLVizKeNWGdM2nUTgsBcxrGVlEuiywpIXShYZdoG+6qZxv?=
 =?us-ascii?Q?4FJwADCHHz7ofWzVDnbxcVg7iFcoabqASPITLiWwvkIWCARk9yk3VEJVNPjY?=
 =?us-ascii?Q?6Ne3w2KWdHvXswNje9US0aKIVjGcqigM0vM+mZyydpKzKBgRyMKyvmgl+zID?=
 =?us-ascii?Q?OpuarVZ8GHxTPn1nGtZwG5qDsWjVPbRoLy6wZQkrbwq+z1zOSzDWink23W28?=
 =?us-ascii?Q?Nu2Uy3Vzgw9FCxDMDwkazl9GbGzrdD+5WP0VslJB4osAP7/n99Y5m3EzIkqY?=
 =?us-ascii?Q?DAmCADFh1Qmzus9qwR7fwU5zDpYtC3s7MsEESyArq7eRLxQ5OKd7pq4jYq0I?=
 =?us-ascii?Q?/ZLKCz/y0FQMqiLxhDEqDqWzt1YDp/v0CHQp45NTDPBHLxCATlpKeki+QaId?=
 =?us-ascii?Q?ppgMBCQjKFeD2Rgel67RJyB6XRUALiud4f9ZrBIEkA6Zb1OdeSPcyomr03X1?=
 =?us-ascii?Q?9fD+Q3PuoIfi4W3GBVVIy14CYL+F1a1CCMjXVlTLLH4O0mQ3hR2FWerjAW+c?=
 =?us-ascii?Q?UkfFGyNQ0gNwPR6H/wIU9FBuLJ8p9C0wk6n8OXiPb5HpAXBUBEAdKcD4EmM6?=
 =?us-ascii?Q?lAAEpiiUTsIApv3jwBYeqtRT+04Yxj1ahBd3LJERH7yrzNs8rRYOG4hSzSZl?=
 =?us-ascii?Q?TwousILBfu/w9gmMuKmpQTPgvB3Lj6pEJI0USgDcDNs6j2ce2oFZ9/GcYotp?=
 =?us-ascii?Q?o3HG9ZIdMkL7zBnuFj1O8YwSw3LL7ok9Baq7HfOMcKCth25XCp/AaKMStMpj?=
 =?us-ascii?Q?HWKLF66fPgmw2alA+ng3GbgHuqpfIu3w/SKoJmk6QkNPaR1FWYGxNqBJHyIZ?=
 =?us-ascii?Q?Q7yH/2KFiG9RxgzvWa0NU2gZY27kgBzjjcHxv6sj9rFcJmro2GuyZ5I4cjtL?=
 =?us-ascii?Q?vOmqcZOam3DKCPRPD6cDMy3DBHqh7BeVLu5NkbdS+xy3q0Ogc5jJqnkBdpi4?=
 =?us-ascii?Q?y3ZsqzuQBa1P1JQVxoLWdwsNBqmhk027kiIZ+Iife+WUP2fOe3jGp+/QzM00?=
 =?us-ascii?Q?zk1pr49lswhj/DBugS9kIiAglfKOthZRkcgQoLCQ9ik3NLZht1itzJ7XNvbQ?=
 =?us-ascii?Q?Delnns7ksV2btW6C7aA4UHlTjAUd?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:56.4696
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 45c5f866-43a4-4ba9-97dc-08dc7afbaeb7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066E7.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6059

In order to support the dynamic dtbo device assignment to a running
VM, the add/remove of the DT overlay and the attach/detach of the
device from the DT overlay should happen separately. Therefore,
repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
overlay to Xen device tree, instead of assigning the device to the
hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with
operations XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment
to the domain.

The hypervisor firstly checks the DT overlay passed from the toolstack
is valid. Then the device nodes are retrieved from the overlay tracker
based on the DT overlay. The attach of the device is implemented by
mapping the IRQ and IOMMU resources.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
v4:
- Split the original patch, only do the device attachment.
v3:
- Style fixes for arch-selection #ifdefs.
- Do not include public/domctl.h, only add a forward declaration of
  struct xen_domctl_dt_overlay.
- Extract the overlay track entry finding logic to a function, drop
  the unused variables.
- Use op code 1&2 for XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH}.
v2:
- New patch.
---
 xen/arch/arm/domctl.c        |   3 +
 xen/common/dt-overlay.c      | 199 ++++++++++++++++++++++++++---------
 xen/include/public/domctl.h  |  14 +++
 xen/include/public/sysctl.h  |  11 +-
 xen/include/xen/dt-overlay.h |   7 ++
 5 files changed, 176 insertions(+), 58 deletions(-)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad56efb0f5..12a12ee781 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -5,6 +5,7 @@
  * Copyright (c) 2012, Citrix Systems
  */
 
+#include <xen/dt-overlay.h>
 #include <xen/errno.h>
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
@@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+    case XEN_DOMCTL_dt_overlay:
+        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 9cece79067..1087f9b502 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -356,6 +356,42 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
     return 0;
 }
 
+/* This function should be called with the overlay_lock taken */
+static struct overlay_track *
+find_track_entry_from_tracker(const void *overlay_fdt,
+                              uint32_t overlay_fdt_size)
+{
+    struct overlay_track *entry, *temp;
+    bool found_entry = false;
+
+    ASSERT(spin_is_locked(&overlay_lock));
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Operation is supported only for prior added dtbo.\n");
+        return NULL;
+    }
+
+    return entry;
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
 static int remove_node_resources(struct dt_device_node *device_node)
 {
@@ -485,8 +521,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
                                         uint32_t overlay_fdt_size)
 {
     int rc;
-    struct overlay_track *entry, *temp, *track;
-    bool found_entry = false;
+    struct overlay_track *entry;
 
     rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
     if ( rc )
@@ -494,29 +529,10 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     spin_lock(&overlay_lock);
 
-    /*
-     * First check if dtbo is correct i.e. it should one of the dtbo which was
-     * used when dynamically adding the node.
-     * Limitation: Cases with same node names but different property are not
-     * supported currently. We are relying on user to provide the same dtbo
-     * as it was used when adding the nodes.
-     */
-    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
-    {
-        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
-        {
-            track = entry;
-            found_entry = true;
-            break;
-        }
-    }
-
-    if ( !found_entry )
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
     {
         rc = -EINVAL;
-
-        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
-               " Removing nodes is supported only for prior added dtbo.\n");
         goto out;
 
     }
@@ -620,15 +636,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return -EFAULT;
         }
 
-        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
-                           tr->iomem_ranges,
-                           tr->irq_ranges);
         write_unlock(&dt_host_lock);
-        if ( rc )
-        {
-            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-            return rc;
-        }
 
         /* Keep overlay_node address in tracker. */
         tr->nodes_address[j] = (unsigned long)overlay_node;
@@ -638,9 +646,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
 }
 /*
  * Adds device tree nodes under target node.
- * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
- * is done to avoid the removal of device_tree generation, iomem regions mapping
- * to hardware domain done by handle_node().
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
  */
 static long handle_add_overlay_nodes(void *overlay_fdt,
                                      uint32_t overlay_fdt_size)
@@ -774,20 +780,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
         goto err;
     }
 
-    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
-    if (tr->irq_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IRQ rangeset failed");
-        goto err;
-    }
-
-    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
-    if (tr->iomem_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
-        goto err;
-    }
-
     rc = add_nodes(tr, nodes_full_path);
     if ( rc )
     {
@@ -843,14 +835,83 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     xfree(tr->nodes_address);
     xfree(tr->fdt);
 
-    rangeset_destroy(tr->irq_ranges);
-    rangeset_destroy(tr->iomem_ranges);
-
     xfree(tr);
 
     return rc;
 }
 
+static long handle_attach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
+    if (entry->irq_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IRQ rangeset failed");
+        goto out;
+    }
+
+    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
+    if (entry->iomem_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
+                           entry->iomem_ranges, entry->irq_ranges);
+        write_unlock(&dt_host_lock);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+            goto out;
+        }
+    }
+
+    spin_unlock(&overlay_lock);
+
+    return 0;
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+    return rc;
+}
+
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     long ret;
@@ -890,6 +951,42 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
     return ret;
 }
 
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
+{
+    long ret;
+    void *overlay_fdt;
+
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        return -EOPNOTSUPP;
+
+    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
+        return -EINVAL;
+
+    if ( op->pad[0] || op->pad[1] || op->pad[2] )
+        return -EINVAL;
+
+    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+    if ( overlay_fdt == NULL )
+        return -ENOMEM;
+
+    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+    if ( ret )
+    {
+        gprintk(XENLOG_ERR, "copy from guest failed\n");
+        xfree(overlay_fdt);
+
+        return -EFAULT;
+    }
+
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+
+    xfree(overlay_fdt);
+
+    return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..ac3c2a7c4c 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1190,6 +1190,16 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+#if defined(__arm__) || defined(__aarch64__)
+struct xen_domctl_dt_overlay {
+    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
+    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
+#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
+    uint8_t overlay_op;                     /* IN: Attach. */
+    uint8_t pad[3];                         /* IN: Must be zero. */
+};
+#endif
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1287,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_dt_overlay                    87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1350,9 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+#if defined(__arm__) || defined(__aarch64__)
+        struct xen_domctl_dt_overlay        dt_overlay;
+#endif
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index febaa4b16a..3a6e7d48f0 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
 DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
 /*
  * XEN_SYSCTL_dt_overlay
- * Performs addition/removal of device tree nodes under parent node using dtbo.
- * This does in three steps:
- *  - Adds/Removes the nodes from dt_host.
- *  - Adds/Removes IRQ permission for the nodes.
- *  - Adds/Removes MMIO accesses.
+ * Performs addition/removal of device tree nodes under parent node using dtbo
+ * from dt_host.
  */
 struct xen_sysctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
@@ -1265,7 +1262,7 @@ struct xen_sysctl {
         struct xen_sysctl_cpu_policy        cpu_policy;
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
         struct xen_sysctl_dt_overlay        dt_overlay;
 #endif
         uint8_t                             pad[128];
diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
index c0567741ee..7f0f3741b5 100644
--- a/xen/include/xen/dt-overlay.h
+++ b/xen/include/xen/dt-overlay.h
@@ -39,15 +39,22 @@ struct overlay_track {
 };
 
 struct xen_sysctl_dt_overlay;
+struct xen_domctl_dt_overlay;
 
 #ifdef CONFIG_OVERLAY_DTB
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
 #else
 #include <xen/errno.h>
 static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     return -EOPNOTSUPP;
 }
+static inline long dt_overlay_domctl(struct domain *d,
+                                     struct xen_domctl_dt_overlay *op)
+{
+    return -EOPNOTSUPP;
+}
 #endif
 
 #endif /* __XEN_DT_OVERLAY_H__ */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728100.1132919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34X-0003M5-A0; Thu, 23 May 2024 07:41:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728100.1132919; Thu, 23 May 2024 07:41:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34X-0003Lu-6R; Thu, 23 May 2024 07:41:09 +0000
Received: by outflank-mailman (input) for mailman id 728100;
 Thu, 23 May 2024 07:41:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34W-0001WY-Bw
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:41:08 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2408::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d0c5d2ec-18d7-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:41:07 +0200 (CEST)
Received: from BY5PR20CA0024.namprd20.prod.outlook.com (2603:10b6:a03:1f4::37)
 by PH7PR12MB6738.namprd12.prod.outlook.com (2603:10b6:510:1a9::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Thu, 23 May
 2024 07:41:03 +0000
Received: from CO1PEPF000066EC.namprd05.prod.outlook.com
 (2603:10b6:a03:1f4:cafe::ef) by BY5PR20CA0024.outlook.office365.com
 (2603:10b6:a03:1f4::37) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Thu, 23 May 2024 07:41:03 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066EC.mail.protection.outlook.com (10.167.249.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:41:02 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:59 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:58 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0c5d2ec-18d7-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZawFVej4XbPQ44lzG6OFWPH5F7gp/ntM/F0lJ6b4W2SDVG6DjqzJ0SNvVOIxD3r53kSxc5L6fD+Kr52Cnmtm6LaZbg0w8iAfiPChYlbhglrjpL/lPR2l0Fi4yTMANNn48FdBez2h92eTfP2Nq7kr2MUfBA/5RrcNpmUjgQbb4Vx9T3sQOxOglATWbMnQNNAuO+rzsKk4po+8/k7zrBCMpeyNU2qXQ3dbnuFF2gy0Om5o1Mv9Zd3ufqUnGHiZWxd/NCH8E6q+mi/y9t7dzxHyhEerFhI7NhijOyi8EEhqYQFTYuOL8SlU03Hfz25vIBXG62z6TYJkQGwZxfANnE6vwA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2WjCV+UJTAIwa98i99pj6e9RnE3+oYy74bW5zI2VriQ=;
 b=n0nw5r70dTN9/IUIRZjlqTnDYsguiiKMq2UY84uPM9Krskv+CqFHFH0MWX77PGRKWu70gPA1O2RTg73EJCvn0kfeu0WfRybXdqTk53Xdic15frL7i72kusIMEI4lJSI2Pap4xaEuZgBlR7FG46PP2jj/4AM8Uh2T5OCXYrGgIBHEZjrQylsDS5wLBSX4hzC4/N2LEBJDpglYTRRt8pN3PGTo/1hawzq65eY8FD4Y430XRTYH7+2+v/970VwUzVZ1P2Lorxs/3ijI95QW4TD3q+oVw4FnwoeoYfkgsLV05B15JQCIf+qk+gs31soVDzagNE54iRJ/r7TsZMLEbt71qw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2WjCV+UJTAIwa98i99pj6e9RnE3+oYy74bW5zI2VriQ=;
 b=05HiC3DdPgRXfDroX4eQlYRO5tbdoPbfu8wuUuR+jEopWi0wsgN2dwBI49/WQp8ffYVDGw/wvlcer1w7ymZNXR/tu6qsUQ4NXfrmVSa+xzYIc+Uf98VdNmdNW2Fds0lMh9ULejxDsVlNd8cHjPdds8syvPQajJlm1d8kyH2/OpQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v4 7/9] xen/arm: Support device detachment from domains
Date: Thu, 23 May 2024 15:40:38 +0800
Message-ID: <20240523074040.1611264-8-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EC:EE_|PH7PR12MB6738:EE_
X-MS-Office365-Filtering-Correlation-Id: 4004e741-a608-4b1d-3e75-08dc7afbb262
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?HEj4UA5zBGDBgQ72s0ZLScb306lbL6WBiFpXCeolHkM2EqPOR/yIraiT8MBx?=
 =?us-ascii?Q?qJVupoEd3MiBwopBNCVv61Kxnm9FkwhMdvnrIO3V4FEDxtp0uSUFUjgiIeG2?=
 =?us-ascii?Q?rQNKK9MWcy3Twjvp0ojYy2XfnZia2Qeqlf0wcWA/vJK2T2Y4H2bGWWdUk6XC?=
 =?us-ascii?Q?18mWzpeBaG5mZh8VG0Wmk0YVB42emsoK/FEW8Fa4dfWFHPBPDg9rvyRY4TtR?=
 =?us-ascii?Q?q2HZpkc3Um44+VDwkKpet5svQLFOJSVZPE8qy3DS5v6Blq96W4+eLpSyhhdt?=
 =?us-ascii?Q?lIKRW0l+jr68AACveMoXFMuTgJbZm3WM8B8vmjPfKv4sIlR9KMEsb0/0Ul7m?=
 =?us-ascii?Q?nkAHL6XVU6ObtXBkzkEuTY0oe6jbdxrgT/xkLrDauvLiThgQTK6DrSYZCX1h?=
 =?us-ascii?Q?1RMExcCl52WXoIjvmr4gVVzAx3mbAHG2VRKjDXktiuGQxEjOC2e3gq8rTpB0?=
 =?us-ascii?Q?fhLQ+vfRFzm8xXIcY78nYzSTwKgfapaqeqcrl51894dIgowz+0Tm9tJl1YDl?=
 =?us-ascii?Q?HHFkd8pBIrvPD/nL1h92a0ZRSJam6lYADZaN1DmgbYM3Ejc71up7BGVylEWJ?=
 =?us-ascii?Q?GlRmss5qzNux5Ezo+BMez0jrLiP8IRKyBB0GPB5eCmiT5GjtkxNvxLreDRUo?=
 =?us-ascii?Q?IzOW5AceSnB1fMkgz/QdFR1WYnwqf4BFAzLGe86kl00neq9Fl1TqiH8ybmTc?=
 =?us-ascii?Q?LLXq1euz1E71zoSaqvfwPjztlcZeW0QF+dXLCAw+NMYeYnzbFRi7sJHZrC/8?=
 =?us-ascii?Q?e1CG8gFXf1ZBzDVbrMbQalRZzPUNpiTWJk/GWdN4TExm7dTGWecgLjFy1282?=
 =?us-ascii?Q?bUrPGAURmHhA2Z/sxf0nOWrSV8VHcXOU6TdYf2Drwn8r/Sh9nKuvEfSEOC5H?=
 =?us-ascii?Q?7QMuyqwahKvmk1eojH/J05VCMurl5YjCXu2Yb6PyLGfrieUuWHZaZg/8h349?=
 =?us-ascii?Q?YpAeJQp7LGp1XyF2Oztx4jMGtvZeto3Xu2EokhY46dqw+q3KjC4C9nek97ck?=
 =?us-ascii?Q?DiwZkNuPHfo6qIuzum79astagPxeGW3IN6XmkiKN0kEOrf+4jqhmbzlK2/yR?=
 =?us-ascii?Q?JsU3ffDkSnTkHO5Sp0lvHJcoUD9ROJSOldq2u6l0iGLd5jEu+34WDH7NnKzc?=
 =?us-ascii?Q?5k1TvFl2ncGLEtPgOEBiekDTqqTlrsa8eRfO3c4e9YPD5aZnhlU7oghHaDdM?=
 =?us-ascii?Q?MNM4nMUcAVos/F3M4/yErrqej72ZyTOJNkgs/wq+wtp6pVqrkWSjxX7eC4B4?=
 =?us-ascii?Q?GF7KJh2ks6gr5etccn4liRphAW/wHwtZPLEcpidz3tht6Y7/vUdGWvwftKIG?=
 =?us-ascii?Q?Vko78hCn72zq0JEwqE5zMqNofBodj5k0AwcgqJMGzv09w67qtWr42bhrjp/x?=
 =?us-ascii?Q?KvqCkdk39/qRB1TfODGYZv5VH/xi?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:41:02.6368
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4004e741-a608-4b1d-3e75-08dc7afbb262
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EC.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6738

Similarly as the device attachment from DT overlay to domain, this
commit implements the device detachment from domain. The DOMCTL
XEN_DOMCTL_dt_overlay op is extended to have the operation
XEN_DOMCTL_DT_OVERLAY_DETACH. The detachment of the device is
implemented by unmapping the IRQ and IOMMU resources. Note that with
these changes, the device de-registration from the IOMMU driver should
only happen at the time when the DT overlay is removed from the Xen
device tree.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
v4:
- Split the original patch, only do device detachment from domain.
---
 xen/common/dt-overlay.c     | 243 ++++++++++++++++++++++++++++--------
 xen/include/public/domctl.h |   3 +-
 2 files changed, 194 insertions(+), 52 deletions(-)

diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 1087f9b502..693b6e4777 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -392,24 +392,100 @@ find_track_entry_from_tracker(const void *overlay_fdt,
     return entry;
 }
 
+static int remove_irq(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+
+    /*
+     * IRQ should always have access unless there are duplication of
+     * of irqs in device tree. There are few cases of xen device tree
+     * where there are duplicate interrupts for the same node.
+     */
+    if (!irq_access_permitted(d, s))
+        return 0;
+    /*
+     * TODO: We don't handle shared IRQs for now. So, it is assumed that
+     * the IRQs was not shared with another domain.
+     */
+    rc = irq_deny_access(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s);
+        return rc;
+    }
+
+    rc = release_guest_irq(d, s);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "unable to release irq %ld\n", s);
+        return rc;
+    }
+
+    return rc;
+}
+
+static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d);
+}
+
+static int remove_iomem(unsigned long s, unsigned long e, void *data)
+{
+    struct domain *d = data;
+    int rc = 0;
+    p2m_type_t t;
+    mfn_t mfn;
+
+    mfn = p2m_lookup(d, _gfn(s), &t);
+    if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL )
+        return -EINVAL;
+
+    rc = iomem_deny_access(d, s, e);
+    if ( rc )
+    {
+        printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n",
+               d, s, e);
+        return rc;
+    }
+
+    rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s));
+    if ( rc )
+        return rc;
+
+    return rc;
+}
+
+static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d)
+{
+    return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d);
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
-static int remove_node_resources(struct dt_device_node *device_node)
+static int remove_node_resources(struct dt_device_node *device_node,
+                                 struct domain *d)
 {
     int rc = 0;
     unsigned int len;
     domid_t domid;
 
-    domid = dt_device_used_by(device_node);
+    if ( !d )
+    {
+        domid = dt_device_used_by(device_node);
 
-    dt_dprintk("Checking if node %s is used by any domain\n",
-               device_node->full_name);
+        dt_dprintk("Checking if node %s is used by any domain\n",
+                   device_node->full_name);
 
-    /* Remove the node if only it's assigned to hardware domain or domain io. */
-    if ( domid != hardware_domain->domain_id && domid != DOMID_IO )
-    {
-        printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n",
-               device_node->full_name, domid);
-        return -EINVAL;
+        /*
+         * We also check if device is assigned to DOMID_IO as when a domain
+         * is destroyed device is assigned to DOMID_IO.
+         */
+        if ( domid != DOMID_IO )
+        {
+            printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n",
+                   device_node->full_name, DOMID_IO, domid);
+            return -EINVAL;
+        }
     }
 
     /* Check if iommu property exists. */
@@ -417,9 +493,12 @@ static int remove_node_resources(struct dt_device_node *device_node)
     {
         if ( dt_device_is_protected(device_node) )
         {
-            rc = iommu_remove_dt_device(device_node);
-            if ( rc < 0 )
-                return rc;
+            if ( !list_empty(&device_node->domain_list) )
+            {
+                rc = iommu_deassign_dt_device(d, device_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
     }
 
@@ -428,7 +507,8 @@ static int remove_node_resources(struct dt_device_node *device_node)
 
 /* Remove all descendants from IOMMU. */
 static int
-remove_descendant_nodes_resources(const struct dt_device_node *device_node)
+remove_descendant_nodes_resources(const struct dt_device_node *device_node,
+                                  struct domain *d)
 {
     int rc = 0;
     struct dt_device_node *child_node;
@@ -438,12 +518,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node)
     {
         if ( child_node->child )
         {
-            rc = remove_descendant_nodes_resources(child_node);
+            rc = remove_descendant_nodes_resources(child_node, d);
             if ( rc )
                 return rc;
         }
 
-        rc = remove_node_resources(child_node);
+        rc = remove_node_resources(child_node, d);
         if ( rc )
             return rc;
     }
@@ -456,8 +536,7 @@ static int remove_nodes(const struct overlay_track *tracker)
 {
     int rc = 0;
     struct dt_device_node *overlay_node;
-    unsigned int j;
-    struct domain *d = hardware_domain;
+    unsigned int j, len;
 
     for ( j = 0; j < tracker->num_nodes; j++ )
     {
@@ -467,18 +546,15 @@ static int remove_nodes(const struct overlay_track *tracker)
 
         write_lock(&dt_host_lock);
 
-        rc = remove_descendant_nodes_resources(overlay_node);
-        if ( rc )
+        /* Check if iommu property exists. */
+        if ( dt_get_property(overlay_node, "iommus", &len) )
         {
-            write_unlock(&dt_host_lock);
-            return rc;
-        }
-
-        rc = remove_node_resources(overlay_node);
-        if ( rc )
-        {
-            write_unlock(&dt_host_lock);
-            return rc;
+            if ( dt_device_is_protected(overlay_node) )
+            {
+                rc = iommu_remove_dt_device(overlay_node);
+                if ( rc < 0 )
+                    return rc;
+            }
         }
 
         dt_dprintk("Removing node: %s\n", overlay_node->full_name);
@@ -493,22 +569,6 @@ static int remove_nodes(const struct overlay_track *tracker)
         write_unlock(&dt_host_lock);
     }
 
-    /* Remove IRQ access. */
-    if ( tracker->irq_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
-   /* Remove mmio access. */
-    if ( tracker->iomem_ranges )
-    {
-        rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d);
-        if ( rc )
-            return rc;
-    }
-
     return rc;
 }
 
@@ -534,7 +594,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
     {
         rc = -EINVAL;
         goto out;
-
     }
 
     rc = remove_nodes(entry);
@@ -552,9 +611,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     xfree(entry->nodes_address);
 
-    rangeset_destroy(entry->irq_ranges);
-    rangeset_destroy(entry->iomem_ranges);
-
     xfree(entry);
 
  out:
@@ -840,6 +896,88 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     return rc;
 }
 
+static long handle_detach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = remove_descendant_nodes_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+
+        rc = remove_node_resources(overlay_node, d);
+        if ( rc )
+        {
+            write_unlock(&dt_host_lock);
+            goto out;
+        }
+        write_unlock(&dt_host_lock);
+
+        rc = remove_all_irqs(entry->irq_ranges, d);
+        if ( rc )
+            goto out;
+
+        rc = remove_all_iomems(entry->iomem_ranges, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove IRQ access. */
+    if ( entry->irq_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    /* Remove mmio access. */
+    if ( entry->iomem_ranges )
+    {
+        rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d);
+        if ( rc )
+            goto out;
+    }
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    return rc;
+}
+
 static long handle_attach_overlay_nodes(struct domain *d,
                                         const void *overlay_fdt,
                                         uint32_t overlay_fdt_size)
@@ -956,7 +1094,8 @@ long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
     long ret;
     void *overlay_fdt;
 
-    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH &&
+         op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH )
         return -EOPNOTSUPP;
 
     if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
@@ -979,7 +1118,9 @@ long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
         return -EFAULT;
     }
 
-    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH )
+        ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+    else
         ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
 
     xfree(overlay_fdt);
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index ac3c2a7c4c..e91f743fa4 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1195,7 +1195,8 @@ struct xen_domctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
     uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
 #define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
-    uint8_t overlay_op;                     /* IN: Attach. */
+#define XEN_DOMCTL_DT_OVERLAY_DETACH                2
+    uint8_t overlay_op;                     /* IN: Attach/Detach. */
     uint8_t pad[3];                         /* IN: Must be zero. */
 };
 #endif
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728101.1132930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34Y-0003hP-N9; Thu, 23 May 2024 07:41:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728101.1132930; Thu, 23 May 2024 07:41:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34Y-0003gE-Gq; Thu, 23 May 2024 07:41:10 +0000
Received: by outflank-mailman (input) for mailman id 728101;
 Thu, 23 May 2024 07:41:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34W-0003CF-VT
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:41:08 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdf34002-18d7-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:41:03 +0200 (CEST)
Received: from CH0PR03CA0441.namprd03.prod.outlook.com (2603:10b6:610:10e::34)
 by LV3PR12MB9215.namprd12.prod.outlook.com (2603:10b6:408:1a0::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.18; Thu, 23 May
 2024 07:40:59 +0000
Received: from CH1PEPF0000AD77.namprd04.prod.outlook.com
 (2603:10b6:610:10e:cafe::99) by CH0PR03CA0441.outlook.office365.com
 (2603:10b6:610:10e::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.18 via Frontend
 Transport; Thu, 23 May 2024 07:40:59 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD77.mail.protection.outlook.com (10.167.244.55) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:40:58 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:40:57 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:40:55 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf34002-18d7-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RBUWCnqg9UeHYedJVl/R9pfYRSQPbqjaNn7+kPZ1VzeYDdGsRAp6XO47j2WTc9R8hM4UNNlWKMNw9MswuMqsmKEnjdMfgGqrE4wyf3xQ5L0TofVKYiwN/gtb9hkGr8TqbdvBzej0OAR6vc5DxmMLvGN0qg4FskHGYTkKxlhZ63v0cofGYvrswBoEhfCeZ8RopH0nWDa1LGDZsA/ec4cxH9Zj+CN4JI0T4+i/Q7sIZD1vJQ/3sudVXdI5ooolnAkTZcHZi2yu8B501MN90jnYqJ6+zAOnPs4lfH8osgQlkR2uMslfZYvy/hi8DC71mal6htn6GKUl6Sy27xa20e76Yw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=L8dKwKh5Agkr6/Sz2gAOhHiky/a7zfL5mmaZ+kO5RLw=;
 b=n1oH2n8ilKylm7HfqTfqeCDmNiQtZwqTCQcKFooiKb/aHe0ze2d4w91fG5jwTa5/Ewa6XB40tDtz+8hCSkuEIqIfOyufbswZ7vWKRIkSI+OmcaI2TH+9GepKfewK2djNlRVg1uBg0F0Q6uftP6/mJQ0cSDpICce7ZkXgFJziWxsChmFYBhpk8vTg2lrUJvskgp9invOzTcyybXEf4uopUDYYm4Gdm6N4BJYZEd+i9WI4mv89/JiCoCu5VuOfrZ8amP9G3upRy8TnMQmepNchdZiJcpL0wxXcbc8DHoa6v6l9VmJouTLy1ba52svo04rMRpS5NXJUTMhsSXsX5u8oXw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L8dKwKh5Agkr6/Sz2gAOhHiky/a7zfL5mmaZ+kO5RLw=;
 b=qu60VsrHqx7x0QzeoqjN1uswiuS4Q1T48lpY2u313+IUphvS2ILN1y95o22X8mXIlwIIRO/lLjU03QmQ+WB7xdH93inSSXHU55FDY6au3ea7lxuRv1I7M4lnt34nX9KtrG1WnBE3q4ESb9ADol4AxaplKS1KRoggPbLZzYm8kmQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Henry Wang" <xin.wang2@amd.com>
Subject: [PATCH v4 6/9] xen/arm/gic: Allow removing interrupt to running VMs
Date: Thu, 23 May 2024 15:40:37 +0800
Message-ID: <20240523074040.1611264-7-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD77:EE_|LV3PR12MB9215:EE_
X-MS-Office365-Filtering-Correlation-Id: 26087eb1-bd6e-4dcb-36f2-08dc7afbb00a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?589QVo1rUR6jtYI+hepZXhQS/2JuFhESRrE1KFczuIeIYQSskO+IhDLWCaOl?=
 =?us-ascii?Q?UHGvySCx19xxCsbhFfvDxlEYvOmKeT8tUgDhd6i6eGKqvGTlfBDq5zP+wCyV?=
 =?us-ascii?Q?gsGJbAP+G9sH6anfv2ucG5DNXxVZbqvkomx/ClEJHK1XMj+6s9BZxydxLttU?=
 =?us-ascii?Q?5ejamBYAhW8iDb8Pdgvn3NtIK6YaSY/B0HxvWftP7f0CDPeUeJf7uMX5cnE+?=
 =?us-ascii?Q?MABpDff/pLQDcKV6kRJaTp3vw8/07bZ+uEZgO3NbCyj/MH8peeG6167VzW9v?=
 =?us-ascii?Q?cdPeiM2QdG/3PU2qY2pk2b8PGd7yQvWp9/qHwlvNKYvIYaiIHFRvHzGwuM3v?=
 =?us-ascii?Q?+ECDZH2KsqqJ2WIQwqDdANA40bK7jsRe3Ajw1/clkgMXzbDOTEGaCfKJPT9e?=
 =?us-ascii?Q?EVgS3XXjolOepU2d3oOJojTuh0jK+NVYIJUB0gb7Yh41QTKxMXrKZWwIF3Dh?=
 =?us-ascii?Q?Ngf0qdpiqncx0la5AOMtFF/QOc1qlbWTnDn8hBiigpfhzqKGceSqJ8/gyWKU?=
 =?us-ascii?Q?KzXMBYpXwQzcG+j4A/fGi8WZwnjrxo4cmeCjEfUf5BefQLSwlwoCvelkAPDR?=
 =?us-ascii?Q?mp+Rrep0s8YKMOiOJneQK+7daTpCmFHKLxk1xPJ7HbGc1uZOHVAtnK0wEvYq?=
 =?us-ascii?Q?5r6pQdd1qV7F6zsGcR8Os60w2KG4PRc9s+LcT7yjki5S/uzSpUplqRiCMCI/?=
 =?us-ascii?Q?3AUFpoR5ZOz0c8S1j9BSGXigW6P6z5dcKzVWlH/8GfJbHybE2TKuzZep87Bk?=
 =?us-ascii?Q?PtsQcQUa9RG2aEPAHWZaGa0Wb5HBGoA9BE/vNzVLUtxeHFEirjvHV8fBKerQ?=
 =?us-ascii?Q?b9m1UKd/0BSeBMEvvtMMSUOdBi+UVu2qVaW9EDpHvxvN0XMK35Jm/mgo2DR9?=
 =?us-ascii?Q?hvUqWEoI0EqVGCxWZUy4H/6xtm54Lh8BR9ZLbAuX1Fmp0asdNgpXH2RwbaTx?=
 =?us-ascii?Q?TSLD/VXpJsPdxdfG5i62/EHK77LaRZnhmxtMZ++FcodvkTeUOdn6Sii9kTjx?=
 =?us-ascii?Q?Cw8CcwlL4L7dEIDmOT1TNgMIT1K6T0Kp1nY+CdbmS89ieHB8eFqu03mfGLoA?=
 =?us-ascii?Q?Hyg77CFxjbjs/pI/bG+0hN+jl54Fbsnm7stFkUUoiwnYcI+nyhL4L/VfLTIo?=
 =?us-ascii?Q?Ugy8oA/sCeZZOY6nDb4dIlipa+MqEUY6+j9RKnI+/ZZhiNV+PtC5ou5FZuvu?=
 =?us-ascii?Q?qMVYsFcWFnThJiHQMdu/yQjOFY/o1a5xYH5H3R8SphjTcPMGA0yh7jnB4ry9?=
 =?us-ascii?Q?W8P3JRGRfTY06Ca5swh4EeACv1QLdXY+G4llyZN2iBzyQV+jLZDmRkSqjkP+?=
 =?us-ascii?Q?3FQHLoxRyEecKZET/bsAAEXtd746LkaBI2a7eyW9Y71YsTtKrJPV8SPbQzI+?=
 =?us-ascii?Q?8P978YdRzxirDX2UX1v+FcpyI7C5?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:40:58.7826
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 26087eb1-bd6e-4dcb-36f2-08dc7afbb00a
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD77.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9215

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Currently, removing physical interrupts are only allowed at
the domain destroy time. For use cases such as dynamic device
tree overlay removing, the removing of physical IRQ to
running domains should be allowed.

Move the above-mentioned domain dying check to vgic_connect_hw_irq().
Similarly as the routing interrupt to running domains, reject the
operation if the IRQ is active or pending in the guest. Do it for
both new and old vGIC implementations. Since now vgic_connect_hw_irq()
may reject the invalid operation case, move the clear of
_IRQ_INPROGRESS flag in gic_remove_irq_from_guest() to after the
successful execution of vgic_connect_hw_irq().

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v4:
- Split the original patch, only do the removing IRQ stuff in this
  patch.
- Move the clear of _IRQ_INPROGRESS flag in gic_remove_irq_from_guest()
  to after the successful execution of vgic_connect_hw_irq().
- Special case the d->is_dying check.
---
 xen/arch/arm/gic-vgic.c  | 27 ++++++++++++++++++++++++---
 xen/arch/arm/gic.c       |  9 +--------
 xen/arch/arm/vgic/vgic.c | 24 ++++++++++++++++++++----
 3 files changed, 45 insertions(+), 15 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index b99e287224..56b6a3d5b0 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -439,6 +439,14 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
 
     /* We are taking to rank lock to prevent parallel connections. */
     vgic_lock_rank(v_target, rank, flags);
+    /* Return with error if the IRQ is being migrated. */
+    if( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
+    {
+        vgic_unlock_rank(v_target, rank, flags);
+        return -EBUSY;
+    }
+
+    spin_lock(&v_target->arch.vgic.lock);
 
     if ( connect )
     {
@@ -456,12 +464,25 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     }
     else
     {
-        if ( desc && p->desc != desc )
-            ret = -EINVAL;
+        if ( d->is_dying )
+        {
+            if ( desc && p->desc != desc )
+                ret = -EINVAL;
+            else
+                p->desc = NULL;
+        }
         else
-            p->desc = NULL;
+        {
+            if ( (desc && p->desc != desc) ||
+                 test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ||
+                 test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
+                ret = -EINVAL;
+            else
+                p->desc = NULL;
+        }
     }
 
+    spin_unlock(&v_target->arch.vgic.lock);
     vgic_unlock_rank(v_target, rank, flags);
 
     return ret;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index b3467a76ae..8633f14bdd 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -159,24 +159,17 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
     ASSERT(test_bit(_IRQ_GUEST, &desc->status));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * Removing an interrupt while the domain is running may have
-     * undesirable effect on the vGIC emulation.
-     */
-    if ( !d->is_dying )
-        return -EBUSY;
-
     desc->handler->shutdown(desc);
 
     /* EOI the IRQ if it has not been done by the guest */
     if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
         gic_hw_ops->deactivate_irq(desc);
-    clear_bit(_IRQ_INPROGRESS, &desc->status);
 
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, false);
     if ( ret )
         return ret;
 
+    clear_bit(_IRQ_INPROGRESS, &desc->status);
     clear_bit(_IRQ_GUEST, &desc->status);
     desc->handler = &no_irq_type;
 
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 048e12c562..0c324b58f7 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -890,14 +890,30 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
     }
     else                                /* remove a mapped IRQ */
     {
-        if ( desc && irq->hwintid != desc->irq )
+        if ( d->is_dying )
         {
-            ret = -EINVAL;
+            if ( desc && irq->hwintid != desc->irq )
+            {
+                ret = -EINVAL;
+            }
+            else
+            {
+                irq->hw = false;
+                irq->hwintid = 0;
+            }
         }
         else
         {
-            irq->hw = false;
-            irq->hwintid = 0;
+            if ( (desc && irq->hwintid != desc->irq) ||
+                 irq->active || irq->pending_latch )
+            {
+                ret = -EINVAL;
+            }
+            else
+            {
+                irq->hw = false;
+                irq->hwintid = 0;
+            }
         }
     }
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728103.1132935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34Z-0003tQ-Bb; Thu, 23 May 2024 07:41:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728103.1132935; Thu, 23 May 2024 07:41:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34Z-0003qF-6V; Thu, 23 May 2024 07:41:11 +0000
Received: by outflank-mailman (input) for mailman id 728103;
 Thu, 23 May 2024 07:41:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34Y-0003CF-A9
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:41:10 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0c96977-18d7-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:41:08 +0200 (CEST)
Received: from CH3P220CA0028.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:1e8::6)
 by SN7PR12MB6839.namprd12.prod.outlook.com (2603:10b6:806:265::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20; Thu, 23 May
 2024 07:41:05 +0000
Received: from CH1PEPF0000AD78.namprd04.prod.outlook.com
 (2603:10b6:610:1e8:cafe::18) by CH3P220CA0028.outlook.office365.com
 (2603:10b6:610:1e8::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend
 Transport; Thu, 23 May 2024 07:41:05 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH1PEPF0000AD78.mail.protection.outlook.com (10.167.244.56) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:41:04 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:41:04 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:41:03 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:41:02 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0c96977-18d7-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UmbpJ0HsSwwe+NJUpclGR6c3RxCh0P/ps3q7wVmhSEmnerM6SjhVj4T8DmBqzUIOxdZCaFGy/JxMm5TMJKuD8mfFc8vuFqdG1gBPTvsRhuXntrSxXCQDmoZt93I9QZP3U9eDC+5IRtYS54gjXlucnPt7US4lj8wAp2EkvZZqwHcxxN529JgwfRGDUW+jgt7KhmQKO2UwcSMPDdDG0wFRO7eyK//VSWBBEAW4Lar472c3dCDy99V4Z8brmIjKtrteYzTke1PYNrpRwSu8Hpkk6NLgatfk+7GisFvpbh0PX1OGKCQowsyUzHVgEGvPd3XW0hyThy3ncy1sTCqzMxGXDA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=j/FY1olHM2o9Q7bHo9+Dh+AEIc21S3KHBajG96rQ8uk=;
 b=GPy/cb2Ftw3Aaur26qyqNTFgeN/2u3/9MbDFiK8zqiTGDu8z/U3UXiy+xE156kpxoww6NtAXwGjSYzHpaGnVFih5/AM3N5qAmQxri2sp/sMawJ70kQlZZOITfeJjcsgfL9x+17ZmbCqDUe+bRNNfG+Bxlg0f5SMM5ycOrF2dXi5Z+lcYgqKYdgoaIX1nqNLhep5icI569r9zgJtfy8QIiN97v9p9ny6swpaNc5plh/Vyt88HZTzjYLR8pTTYfRmK6bSFcE/mNVw38ZZsBN2pO+MAC1Rlc8tluB1uHB3U9ifaSRas4mr1oVNtwXgeBI0sioKpR8uoXcl8VJyhaBct0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=j/FY1olHM2o9Q7bHo9+Dh+AEIc21S3KHBajG96rQ8uk=;
 b=x0HF11Ut5J5qvqv8o8lEU9slHgbH7ylvjLio4m13+rUqwcw5DQOaMpbwJuuD2ekcKieCMX1FFGUIGm3rGm2GehLf1h+a8gbdtTbQPA07GGgH+NeozFcnEizPLZbROMy1obq8hJaxfGk39w5UH0RZGM9MiMBgCfSOaeuPceWy6RQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Henry
 Wang <xin.wang2@amd.com>
Subject: [PATCH v4 9/9] docs: Add device tree overlay documentation
Date: Thu, 23 May 2024 15:40:40 +0800
Message-ID: <20240523074040.1611264-10-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD78:EE_|SN7PR12MB6839:EE_
X-MS-Office365-Filtering-Correlation-Id: c8cfb63d-fa69-4f2c-610d-08dc7afbb381
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?x8MX7l7cJJXwKSNdmYB957qHQ8qn5KZnsPnXe83e9gpHl7382t3HE5Mc6QG/?=
 =?us-ascii?Q?gjk0ygz8Htq/J6xGJ6xR25aawR97fF04ipzWqCGogE9XKPLtcW9R65DOgcNS?=
 =?us-ascii?Q?Ng6kZiLd0NBfWPE1nOgNhYv4Acn+LQ30X/BZDdpDxR1N43GW2UJ/V4VM2WBr?=
 =?us-ascii?Q?7pexlZu1VnJ7Nwt8RRF4ebpSaiyBQ/sBJSaMXP98iChBsTADivoRHBqewBd0?=
 =?us-ascii?Q?f8IGq3JStb/bmhjKvvW1Byh1j1paIm/gNSQPXkrLYd8VuhoFjFzQ8U3vUBuX?=
 =?us-ascii?Q?wotU1G4DCHYK8i9VOcAGQbKPibC5keQHs4e/vI1Jk2t7NSHw22bIqf3yBrns?=
 =?us-ascii?Q?brdnnlbZONunHGV2WXw1FJ2gdymVnGNhg0i2S2S8MlgF3ybw0imC70Jp4AU9?=
 =?us-ascii?Q?fZ4PMvgGFYpHSg8HlgdOrN00L1LFOIRPm81XpzhOvI5hB9HJkomb2G+KrpSO?=
 =?us-ascii?Q?vaqQ4nHMBwsHhVSNxS+XJqwjOVFfWvbgiGGmXI0DlMUoyntQ084HHmOgyzPj?=
 =?us-ascii?Q?XeGHRYKakXMO1it4xLJQ/JAAQd8CgkH4/UDmVXKJlWoR+vNE3ZoCkDIXGQcU?=
 =?us-ascii?Q?u/XGefwmO9r7JWtkGuoVF1J+lvvi7/8JlwuHW7YCmigvhzUSIvBr/hauNDe4?=
 =?us-ascii?Q?fl8IA8uGlYCOI4q23OuyXOmmbG9Jfp8eUNKqk6+0w5XnONGKIIjZcKU5CZIJ?=
 =?us-ascii?Q?sobwK6OHVmq0K5GWPVnSqvpC5MD+CSHwDVsTH25Ygy73NTTELS1By8xmkdly?=
 =?us-ascii?Q?PqkopfgGLT4QXrPqkfEwySxJLEaouqCnQHN4rdrxfgwbmK7IcTm01xRtveWN?=
 =?us-ascii?Q?JlT2+e7MvtyU/IA34TM/OjLoLjD3dgnLmmgq77jlEOd8oaLBQ+DI7CjIMHIj?=
 =?us-ascii?Q?B7lq+sezpMfkttFNnJ2fF3zrbaR1Ny8JVV41ybcBYEy0nHXcgE1hhDSyHrIc?=
 =?us-ascii?Q?mclND3WqtrSbLE/teyjU2GvJ6h8rBgvQDki9pYQ9sIq/0x+0lTmBp28M3UVT?=
 =?us-ascii?Q?XP5QIfTvtnTC+VKGRpz0t582Z8+JZPmkvjoYbUvC3x9VRHlATxYhC6sCXZFA?=
 =?us-ascii?Q?JUWLdg6/sE/gqrku53hmK+WQrZTh85M1B5SRbjHdG02Kb7OM4Z0XRkgkxHnE?=
 =?us-ascii?Q?oEkyajwl3I3dQFyT681KgFK0zR1y1VwVKjmvQeArUqE2SAH56ibvfT/kO6tr?=
 =?us-ascii?Q?qSObuDRJwNQutbmhGWTEMBEbLt9EChCZo+nxBPISv//bX4VXANDhF1j3CVvg?=
 =?us-ascii?Q?F2Wi8jvPQQVXI68iLl395GobR9qF4F9n7E1FHDqw+mZ/8llJtmVwz/f0XDnn?=
 =?us-ascii?Q?qUNwoTfnCTotvjD0bxCuUg5i?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:41:04.5967
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c8cfb63d-fa69-4f2c-610d-08dc7afbb381
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH1PEPF0000AD78.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6839

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v4:
- No change.
v3:
- No change.
v2:
- Update the content based on the changes in this version.
---
 docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
 create mode 100644 docs/misc/arm/overlay.txt

diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
new file mode 100644
index 0000000000..811a6de369
--- /dev/null
+++ b/docs/misc/arm/overlay.txt
@@ -0,0 +1,99 @@
+# Device Tree Overlays support in Xen
+
+Xen now supports dynamic device assignment to running domains,
+i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and
+attaching/detaching them to/from a running domain with given $domid.
+
+Dynamic node assignment works in two steps:
+
+## Add/Remove device tree overlay to/from Xen device tree
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
+
+## Attach/Detach device from the DT overlay to/from domain
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor attach/detach the device to/from the user-provided $domid by
+   mapping/unmapping node resources in the DT overlay.
+
+# Examples
+
+Here are a few examples on how to use it.
+
+## Dom0 device add
+
+For assigning a device tree overlay to Dom0, user should firstly properly
+prepare the DT overlay. More information about device tree overlays can be
+found in [1]. Then, in Dom0, enter the following:
+
+    (dom0) xl dt-overlay add overlay.dtbo
+
+This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
+
+To assign the newly added device from the dtbo to Dom0:
+
+    (dom0) xl dt-overlay attach overlay.dtbo 0
+
+Next, if the user wants to add the same device tree overlay to dom0
+Linux, execute the following:
+
+    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally if needed, the relevant Linux kernel drive can be loaded using:
+
+    (dom0) modprobe module_name.ko
+
+## Dom0 device remove
+
+For removing the device from Dom0, first detach the device from Dom0:
+
+    (dom0) xl dt-overlay detach overlay.dtbo 0
+
+NOTE: The user is expected to unload any Linux kernel modules which
+might be accessing the devices in overlay.dtbo before detach the device.
+Detaching devices without unloading the modules might result in a crash.
+
+Then remove the overlay from Xen device tree:
+
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+## DomU device add/remove
+
+All the nodes in dtbo will be assigned to a domain; the user will need
+to prepare the dtb for the domU. For example, the `interrupt-parent` property
+of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`.
+Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
+
+User will need to create the DomU with below properties properly configured
+in the xl config file:
+- `iomem`
+- `passthrough` (if IOMMU is needed)
+
+User will also need to modprobe the relevant drivers.
+
+Example for domU device add:
+
+    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
+    (dom0) xl dt-overlay attach overlay.dtbo $domid
+    (dom0) xl console $domid                         # To access $domid console
+
+Next, if the user needs to modify/prepare the overlay.dtbo suitable for
+the domU:
+
+    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally, if needed, the relevant Linux kernel drive can be probed:
+
+    (domU) modprobe module_name.ko
+
+Example for domU overlay remove:
+
+    (dom0) xl dt-overlay detach overlay.dtbo $domid
+    (dom0) xl dt-overlay remove overlay.dtbo
+
+[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:41:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:41:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728109.1132950 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34c-0004YO-OQ; Thu, 23 May 2024 07:41:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728109.1132950; Thu, 23 May 2024 07:41:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA34c-0004Y0-JH; Thu, 23 May 2024 07:41:14 +0000
Received: by outflank-mailman (input) for mailman id 728109;
 Thu, 23 May 2024 07:41:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Po+=M2=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1sA34b-0003CF-Hg
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:41:13 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d34da501-18d7-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:41:11 +0200 (CEST)
Received: from BY5PR20CA0013.namprd20.prod.outlook.com (2603:10b6:a03:1f4::26)
 by BL1PR12MB5804.namprd12.prod.outlook.com (2603:10b6:208:394::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Thu, 23 May
 2024 07:41:08 +0000
Received: from CO1PEPF000066EC.namprd05.prod.outlook.com
 (2603:10b6:a03:1f4:cafe::2e) by BY5PR20CA0013.outlook.office365.com
 (2603:10b6:a03:1f4::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Thu, 23 May 2024 07:41:07 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CO1PEPF000066EC.mail.protection.outlook.com (10.167.249.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Thu, 23 May 2024 07:41:07 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:41:01 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 02:41:01 -0500
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 02:41:00 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d34da501-18d7-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X8xx40pf5Keh1b42FvaLrBPfygRwS+VlBg+4pTsdKe3/5zk4nHES1+k9oi3u2mVTJPO2rld5GoqAAHfBPBCZqCihVjTf8Ec3MQZuWIxrxlUfD/IrXGfrp2f3as9Q6SBfOB58run/nINP7IvutIjKp4NwkAdoSHWQj+q4pWFqBBU5gigCfobNgeWjdJ6oFvsNHi1jImBZAwgPrx6L+ELYrHnrDuOgH4ZxJPDjFH5dkrSpZnAYXcb5C4s6DNjutiejZSAcAfKr5oLmLQbz6gbYPblppVavrKc1ZakViO+w0BcJX6icO1nYeNZBIAPo4fuovImMA8RqWBkr8bRu3SSfNg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=da2DEjMTTPOqHs5JEHft5f2zNU/TKUOCNbSUhm9LjB0=;
 b=mJe/hsJJNhF9SfQulWbnW8YkZ51UUl+LS+LW9zlfazjX7EEK13nvmIsnqlw3dLjR+xF1bCUe9tO0BzNDt8svJckLgY9qdecfsN7B0Zot6c3I5HchUj5XsnaDybLHtb5hxBYLubL4r6wDCnMv4LXXZPYGEFZBIMtNVoi+R9szpFc1PbV1PulgxkT6u7hPEguLMKnkWpcemo8EU7pA6qE47jdIZDcfU7oXDPf3oQqXYVsIHmPimbtL/dkGsPnD7K/4qSBkBbCz/CJ5TLty9fTlctAGArW2gE0cl0MVXDeMjuR+LoXt62aKLrkiMozVxkEt/O+TI9ObnJN9/3jbCGOEJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=da2DEjMTTPOqHs5JEHft5f2zNU/TKUOCNbSUhm9LjB0=;
 b=Xdq9w1W82Bn0vuY6eT/+m7Ixhu5PytVvwZhlxWBg2SLQHyxj/d3M+f+I6XTMUXj7U6TXbb/iQy0tWCcx0EkK5NKziYi26jqNG/hWnJPqxc52tbxyMvEZHgL+iLb97Ne1e+v4NZvE8GNSsfBw82n61ZI8sVOl33bdhZAemF44NjM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
	Henry Wang <xin.wang2@amd.com>, Jason Andryuk <jason.andryuk@amd.com>
Subject: [PATCH v4 8/9] tools: Introduce the "xl dt-overlay {attach,detach}" commands
Date: Thu, 23 May 2024 15:40:39 +0800
Message-ID: <20240523074040.1611264-9-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000066EC:EE_|BL1PR12MB5804:EE_
X-MS-Office365-Filtering-Correlation-Id: 543ba5a1-7b6f-43ed-6a07-08dc7afbb558
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?thVannc2U8Dx/QzE1TkJBopXvcqv5OJO1x6Qv+15CEuSHxn6lj+eFTZmWNhz?=
 =?us-ascii?Q?kI3HTlMG/cd2WkdjXcht9nqn2OuqxzADHOpbEP2ajzRD9F2iyeVAvir42gb3?=
 =?us-ascii?Q?AIuo7mxkIBIRUOy4sXFqiecwP6OKOmGA18bbPgIJPseM2h/+r5KCNTpTCrwF?=
 =?us-ascii?Q?R0UYPrGVHd5EvPe21LPzv7gegeIgqDDf1PUDzGk+sqzEgb91KVzifAqG962m?=
 =?us-ascii?Q?H/R1fZnCaY1IC1fhxyU2wilKCjc7lGG3IPKgv7tNXgUb9iMcRkHDFCKYSD9M?=
 =?us-ascii?Q?RasR7Z7A8A5AVvIIB64YeaTwxX/RXgImQ//RiV4yzA7gPOFBywzl3y7CMRcw?=
 =?us-ascii?Q?QXWMEnWhI/LY4PacYosUfohypuvWYmOG1dW8OBPuBOsJz1i1KANjeMcfgp01?=
 =?us-ascii?Q?wdt0tCSQFfBpARNRbHghaN8DCpum8/bA3H1QsWlK3KnVz9GZDZJ8Bx8Vh/6Q?=
 =?us-ascii?Q?OGdtjJLXvyRwTCT6QVFYD+rIHn2PjPAkqewG1i4E19ybdX/jkUGTe2PlVXjW?=
 =?us-ascii?Q?7v73FajVGMOlgUf0d//LUoY/XCuYuGBezRKIJfMjq1gBSUUA4OS4M3av8cyY?=
 =?us-ascii?Q?zoqH8QU8CPNj7WHmitm+oibjs98hor4MrtVMtJEoVGL2i65eniSZDFlfbKem?=
 =?us-ascii?Q?dGuVce9BT+O9Ff8O5rovFCictvdzHNPToQlQ0snKJet8SHZNNIdaDNmn+0dR?=
 =?us-ascii?Q?YcytvDIp0pBlxabx003sb/5y7LzvTkutaH7BYcOMDRy+7ZoJKYPb8mOoc+hK?=
 =?us-ascii?Q?aEJC9gdUQr0X+fufjswffBO084hTxI8x1q/XykxT4ole9P5CxoS1sQCnQ8+R?=
 =?us-ascii?Q?MeFq4/UxErk/maNgJhUGxmP7Tw/RQWabOh7mRr3U7Y7FCvEADDv2ntUGPrKG?=
 =?us-ascii?Q?iFanZejcICUQPO7ma1KMy16F+YY92cZpx4ixLBUsTfveebLDgL9OmGxdqdP2?=
 =?us-ascii?Q?tYaQn3AuICSuZKyWTli1b3uQwvy+9Bpu5XM9CQODlOTz4XA/cEdz5uEO/c8N?=
 =?us-ascii?Q?+prTFj7MieelZIXNx36DU7l6F440XeVVOjrn4PhnvwNxj9iWsiWCfvI+3Z5Y?=
 =?us-ascii?Q?oHK/y+WK3x3AdArXQEydsUurLql9HWJq2gmCAPM9mlvMhV5ZMADX1M/aZaAy?=
 =?us-ascii?Q?2wbJnrg28UHI/WdVEL0Ib48QqrSSLKNClpCCuoIYXzBD9/dqmYm7y8aBXXJT?=
 =?us-ascii?Q?xuaYY+IH9m7ipSjCVpjji/G30Del/ThGG8HUOxj82i+5JE8TMivTO1vCR9mr?=
 =?us-ascii?Q?k5sOjfCJWqftfp4bWoQQpowEcYmciPhjXzvc8zi/u8mvfyiDB6xHcY0sncY7?=
 =?us-ascii?Q?LQU8nFNJ1g0a2NWaioe1Ej952p+sb/XW3c0syXKtMC9GOV9EYHCPXrg2wWKE?=
 =?us-ascii?Q?8k1J6rmIX3z/i3cmPMAF1GeSw/pn?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 07:41:07.6055
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 543ba5a1-7b6f-43ed-6a07-08dc7afbb558
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000066EC.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5804

With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach/detach devices from the provided DT overlay to domains.
Support this by introducing a new set of "xl dt-overlay" commands and
related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
rework the command option parsing logic.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v4:
- Add Jason's Reviewed-by tag.
v3:
- Introduce new API libxl_dt_overlay_domain() and co., instead of
  reusing existing API libxl_dt_overlay().
- Add in-code comments for the LIBXL_DT_OVERLAY_* macros.
- Use find_domain() to avoid getting domain_id from strtol().
v2:
- New patch.
---
 tools/include/libxl.h               | 10 +++++++
 tools/include/xenctrl.h             |  3 +++
 tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++
 tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++
 tools/xl/xl_cmdtable.c              |  4 +--
 tools/xl/xl_vmcontrol.c             | 42 ++++++++++++++++++++---------
 6 files changed, 104 insertions(+), 14 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..6cc6d6bf6a 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -2549,8 +2549,18 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 #if defined(__arm__) || defined(__aarch64__)
+/* Values should keep consistent with the op from XEN_SYSCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_ADD                   1
+#define LIBXL_DT_OVERLAY_REMOVE                2
 int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
                      uint32_t overlay_size, uint8_t overlay_op);
+
+/* Values should keep consistent with the op from XEN_DOMCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_DOMAIN_ATTACH         1
+#define LIBXL_DT_OVERLAY_DOMAIN_DETACH         2
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op);
 #endif
 
 /*
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..9ceca0cffc 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id);
 #endif
 
 /* Compat shims */
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
index c2224c4d15..ea1da522d1 100644
--- a/tools/libs/ctrl/xc_dt_overlay.c
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -48,3 +48,34 @@ err:
 
     return err;
 }
+
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id)
+{
+    int err;
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_dt_overlay,
+        .domain = domain_id,
+        .u.dt_overlay = {
+            .overlay_op = overlay_op,
+            .overlay_fdt_size = overlay_fdt_size,
+        }
+    };
+
+    DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+        goto err;
+
+    set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, overlay_fdt);
+
+    return err;
+}
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
index a6c709a6dc..00503b76bd 100644
--- a/tools/libs/light/libxl_dt_overlay.c
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -69,3 +69,31 @@ out:
     return rc;
 }
 
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op)
+{
+    int rc;
+    int r;
+    GC_INIT(ctx);
+
+    if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+        LOG(ERROR, "Overlay DTB check failed");
+        rc = ERROR_FAIL;
+        goto out;
+    } else {
+        LOG(DEBUG, "Overlay DTB check passed");
+        rc = 0;
+    }
+
+    r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, overlay_op,
+                             domain_id);
+    if (r) {
+        LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__);
+        rc = ERROR_FAIL;
+    }
+
+out:
+    GC_FREE;
+    return rc;
+}
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 1f3c6b5897..37770b20e3 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = {
 #ifdef LIBXL_HAVE_DT_OVERLAY
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
-      "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>",
+      "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain",
+      "<operation=add|remove> <.dtbo> OR <operation=attach|detach> <.dtbo> <Domain>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 02575d5d36..ccfeec39f1 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1268,32 +1268,46 @@ int main_create(int argc, char **argv)
 #ifdef LIBXL_HAVE_DT_OVERLAY
 int main_dt_overlay(int argc, char **argv)
 {
-    const char *overlay_ops = NULL;
     const char *overlay_config_file = NULL;
     void *overlay_dtb = NULL;
     int rc;
     uint8_t op;
     int overlay_dtb_size = 0;
-    const int overlay_add_op = 1;
-    const int overlay_remove_op = 2;
+    uint32_t domain_id = 0;
+    bool domain_op = false;
 
     if (argc < 2) {
         help("dt-overlay");
         return EXIT_FAILURE;
     }
 
-    overlay_ops = argv[1];
-    overlay_config_file = argv[2];
-
-    if (strcmp(overlay_ops, "add") == 0)
-        op = overlay_add_op;
-    else if (strcmp(overlay_ops, "remove") == 0)
-        op = overlay_remove_op;
-    else {
+    if (strcmp(argv[optind], "add") == 0)
+        op = LIBXL_DT_OVERLAY_ADD;
+    else if (strcmp(argv[optind], "remove") == 0)
+        op = LIBXL_DT_OVERLAY_REMOVE;
+    else if (strcmp(argv[optind], "attach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_ATTACH;
+        domain_op = true;
+    } else if (strcmp(argv[optind], "detach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_DETACH;
+        domain_op = true;
+    } else {
         fprintf(stderr, "Invalid dt overlay operation\n");
         return EXIT_FAILURE;
     }
 
+    overlay_config_file = argv[optind+1];
+
+    if (domain_op) {
+        if (argc <= optind + 2) {
+            fprintf(stderr, "Missing domain ID\n");
+            help("dt-overlay");
+            return EXIT_FAILURE;
+        } else {
+            domain_id = find_domain(argv[optind+2]);
+        }
+    }
+
     if (overlay_config_file) {
         rc = libxl_read_file_contents(ctx, overlay_config_file,
                                       &overlay_dtb, &overlay_dtb_size);
@@ -1309,7 +1323,11 @@ int main_dt_overlay(int argc, char **argv)
         return EXIT_FAILURE;
     }
 
-    rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    if (!domain_op)
+        rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    else
+        rc = libxl_dt_overlay_domain(ctx, domain_id, overlay_dtb,
+                                     overlay_dtb_size, op);
 
     free(overlay_dtb);
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 07:47:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:47:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728125.1132959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Aa-000752-DV; Thu, 23 May 2024 07:47:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728125.1132959; Thu, 23 May 2024 07:47:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Aa-00074v-Al; Thu, 23 May 2024 07:47:24 +0000
Received: by outflank-mailman (input) for mailman id 728125;
 Thu, 23 May 2024 07:47:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5tjK=M2=linaro.org=manos.pitsidianakis@srs-se1.protection.inumbo.net>)
 id 1sA3AY-00074k-Bb
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:47:22 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aed39276-18d8-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:47:19 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-354de3c5c61so1288525f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:47:20 -0700 (PDT)
Received: from meli-email.org (adsl-49.37.6.163.tellas.gr. [37.6.163.49])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354cc7738dasm7480210f8f.18.2024.05.23.00.47.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 00:47:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aed39276-18d8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716450439; x=1717055239; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:in-reply-to
         :references:user-agent:subject:cc:to:from:date:from:to:cc:subject
         :date:message-id:reply-to;
        bh=aH6mnjfZkzeYOjZvZ/7yNuU7flVChupqdi/iX01ZoMA=;
        b=agN5QjwT6WFIwt0/QBoUxidyQBB3s5Dmgv+jlimGBIFPr5U0/cZDmSAh8SH+tx/ilK
         /L2l2PFBaKj+/fiMLViHpuB5ihXGmS+75DZrNge60oLAfS4b48EB3PIxfn6zc4WUeJAH
         UUhcBzEVOBAPOyjqn0AEfzgjuFbo6+MtZB0qqMcj95hJf0pwXSeRv3pRNbyLBL3s3CP1
         tAviuufWNCZ3egDKj4utrvGuZ8hmRERztgiJGh4tK4SReRiJNmLHAh8eykADRdDNV1VC
         i5urReAlPVLyFSDPwpcH5c5/BOhC0wi9VTx7c6OecVlW++nLa00PQjXx/hmACOqbjGeL
         iErA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716450439; x=1717055239;
        h=content-transfer-encoding:mime-version:message-id:in-reply-to
         :references:user-agent:subject:cc:to:from:date:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aH6mnjfZkzeYOjZvZ/7yNuU7flVChupqdi/iX01ZoMA=;
        b=vRHLFurXDop5P4suE0UifVWVDj91lNH/YsLW34yo/yZ/lC/HFs1Cmzg+SORrz7U4hD
         m6xojh8EfcIi8/pABr+2DMh1u4N/mQZsQ2tcyFRunWdUA+Z8BjKjmdbB6PBijTBYO0Wy
         mPl2dGVuuwtcLFGULCNDhHBVusi9YMGptqhrTDmPtMA6lFM18McvmZGQ2ztdMdXmCugb
         b06f0XbLvqWJEKRl7STnLRTZ14k39ex4veQaxXwqNHKs9j7uWYmc+KU7KHIC4bUzDB8u
         CqKpof1jqntO5jYDFUXyP1HK3w9lM6+fHUcAKQwp623ffAH9mhOaISgc3nbull4amZfJ
         Yx3Q==
X-Gm-Message-State: AOJu0Yx4bKwUd6OvkkmvZeA/5+loR28BKrGKo2C5sNBzJwHZcFP1vnfs
	bZqsnxEKLUAyATG4sLEBlzThssB7FO9uDq0HawlS/HSLrmiX0+OF436Ms5oUu7y+9nMGsoeYvW7
	c
X-Google-Smtp-Source: AGHT+IGQeMd5yBkAmawll57ZNdrxkOB8lTV5IsVrDnByQEuUMkNFKEANvw8CndcTxVK9s5YYxaoABQ==
X-Received: by 2002:a5d:4ecd:0:b0:34c:f320:247 with SMTP id ffacd0b85a97d-354d8d83640mr3198074f8f.49.1716450438824;
        Thu, 23 May 2024 00:47:18 -0700 (PDT)
Date: Thu, 23 May 2024 10:35:04 +0300
From: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
To: xen-devel@lists.xenproject.org, "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com, sstabellini@kernel.org, jgross@suse.com, "Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Bcc: 
Subject: Re: [PATCH v6 7/8] xen: mapcache: Add support for grant mappings
User-Agent: meli 0.8.5
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com> <20240516154804.1114245-8-edgar.iglesias@gmail.com>
In-Reply-To: <20240516154804.1114245-8-edgar.iglesias@gmail.com>
Message-ID: <dxhms.qrh0wkni3yz@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=utf-8; format=flowed

On Thu, 16 May 2024 18:48, "Edgar E. Iglesias" <edgar.iglesias@gmail.com> wrote:
>From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>
>Add a second mapcache for grant mappings. The mapcache for
>grants needs to work with XC_PAGE_SIZE granularity since
>we can't map larger ranges than what has been granted to us.
>
>Like with foreign mappings (xen_memory), machines using grants
>are expected to initialize the xen_grants MR and map it
>into their address-map accordingly.
>
>Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>---
> hw/xen/xen-hvm-common.c         |  12 ++-
> hw/xen/xen-mapcache.c           | 163 ++++++++++++++++++++++++++------
> include/hw/xen/xen-hvm-common.h |   3 +
> include/sysemu/xen.h            |   7 ++
> 4 files changed, 152 insertions(+), 33 deletions(-)
>
>diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
>index a0a0252da0..b8ace1c368 100644
>--- a/hw/xen/xen-hvm-common.c
>+++ b/hw/xen/xen-hvm-common.c
>@@ -10,12 +10,18 @@
> #include "hw/boards.h"
> #include "hw/xen/arch_hvm.h"
> 
>-MemoryRegion xen_memory;
>+MemoryRegion xen_memory, xen_grants;
> 
>-/* Check for xen memory.  */
>+/* Check for any kind of xen memory, foreign mappings or grants.  */
> bool xen_mr_is_memory(MemoryRegion *mr)
> {
>-    return mr == &xen_memory;
>+    return mr == &xen_memory || mr == &xen_grants;
>+}
>+
>+/* Check specifically for grants.  */
>+bool xen_mr_is_grants(MemoryRegion *mr)
>+{
>+    return mr == &xen_grants;
> }
> 
> void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
>diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
>index a07c47b0b1..1cbc2aeaa9 100644
>--- a/hw/xen/xen-mapcache.c
>+++ b/hw/xen/xen-mapcache.c
>@@ -14,6 +14,7 @@
> 
> #include <sys/resource.h>
> 
>+#include "hw/xen/xen-hvm-common.h"
> #include "hw/xen/xen_native.h"
> #include "qemu/bitmap.h"
> 
>@@ -21,6 +22,8 @@
> #include "sysemu/xen-mapcache.h"
> #include "trace.h"
> 
>+#include <xenevtchn.h>
>+#include <xengnttab.h>
> 
> #if HOST_LONG_BITS == 32
> #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
>@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
>     unsigned long *valid_mapping;
>     uint32_t lock;
> #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
>+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)

Might we get more entry kinds in the future? (for example foreign maps). 
Maybe this could be an enum.

>     uint8_t flags;
>     hwaddr size;
>     struct MapCacheEntry *next;
>@@ -71,6 +75,8 @@ typedef struct MapCache {
> } MapCache;
> 
> static MapCache *mapcache;
>+static MapCache *mapcache_grants;
>+static xengnttab_handle *xen_region_gnttabdev;
> 
> static inline void mapcache_lock(MapCache *mc)
> {
>@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>     unsigned long max_mcache_size;
>     unsigned int bucket_shift;
> 
>+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
>+    if (xen_region_gnttabdev == NULL) {
>+        error_report("mapcache: Failed to open gnttab device");
>+        exit(EXIT_FAILURE);
>+    }
>+
>     if (HOST_LONG_BITS == 32) {
>         bucket_shift = 16;
>     } else {
>@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>     mapcache = xen_map_cache_init_single(f, opaque,
>                                          bucket_shift,
>                                          max_mcache_size);
>+
>+    /*
>+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
>+     * map anything beyond the number of pages granted to us.
>+     */
>+    mapcache_grants = xen_map_cache_init_single(f, opaque,
>+                                                XC_PAGE_SHIFT,
>+                                                max_mcache_size);
>+
>     setrlimit(RLIMIT_AS, &rlimit_as);
> }
> 
>@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
>                              hwaddr size,
>                              hwaddr address_index,
>                              bool dummy,
>+                             bool grant,
>+                             bool is_write,
>                              ram_addr_t ram_offset)
> {
>     uint8_t *vaddr_base;
>-    xen_pfn_t *pfns;
>+    uint32_t *refs = NULL;
>+    xen_pfn_t *pfns = NULL;
>     int *err;

You should use g_autofree to perform automatic cleanup on function exit 
instead of manually freeing, since the allocations should only live 
within the function call.

>     unsigned int i;
>     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
> 
>     trace_xen_remap_bucket(address_index);
> 
>-    pfns = g_new0(xen_pfn_t, nb_pfn);
>+    if (grant) {
>+        refs = g_new0(uint32_t, nb_pfn);
>+    } else {
>+        pfns = g_new0(xen_pfn_t, nb_pfn);
>+    }
>     err = g_new0(int, nb_pfn);
> 
>     if (entry->vaddr_base != NULL) {
>@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
>     g_free(entry->valid_mapping);
>     entry->valid_mapping = NULL;
> 
>-    for (i = 0; i < nb_pfn; i++) {
>-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
>+    if (grant) {
>+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
>+
>+        for (i = 0; i < nb_pfn; i++) {
>+            refs[i] = grant_base + i;
>+        }
>+    } else {
>+        for (i = 0; i < nb_pfn; i++) {
>+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
>+        }
>     }
> 
>-    /*
>-     * If the caller has requested the mapping at a specific address use
>-     * MAP_FIXED to make sure it's honored.
>-     */
>+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
>+
>     if (!dummy) {
>-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
>-                                           PROT_READ | PROT_WRITE,
>-                                           vaddr ? MAP_FIXED : 0,
>-                                           nb_pfn, pfns, err);

Since err is not NULL here, the function might return a valid pointer 
but individual frames might have failed.


>+        if (grant) {
>+            int prot = PROT_READ;
>+
>+            if (is_write) {
>+                prot |= PROT_WRITE;
>+            }
>+
>+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
>+            assert(vaddr == NULL);
>+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
>+                                                         nb_pfn,
>+                                                         xen_domid, refs,
>+                                                         prot);
>+        } else {
>+            /*
>+             * If the caller has requested the mapping at a specific address use
>+             * MAP_FIXED to make sure it's honored.
>+             *
>+             * We don't yet support upgrading mappings from RO to RW, to handle
>+             * models using ordinary address_space_rw(), foreign mappings ignore
>+             * is_write and are always mapped RW.
>+             */
>+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
>+                                               PROT_READ | PROT_WRITE,
>+                                               vaddr ? MAP_FIXED : 0,
>+                                               nb_pfn, pfns, err);
>+        }
>         if (vaddr_base == NULL) {
>-            perror("xenforeignmemory_map2");
>+            perror(grant ? "xengnttab_map_domain_grant_refs"
>+                           : "xenforeignmemory_map2");
>             exit(-1);
>         }
>     } else {
>@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,
>         }
>     }
> 
>+    g_free(refs);
>     g_free(pfns);
>     g_free(err);
> }
>@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,
> static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>                                        hwaddr phys_addr, hwaddr size,
>                                        ram_addr_t ram_offset,
>-                                       uint8_t lock, bool dma, bool is_write)
>+                                       uint8_t lock, bool dma,
>+                                       bool grant, bool is_write)
> {
>     MapCacheEntry *entry, *pentry = NULL,
>                   *free_entry = NULL, *free_pentry = NULL;
>@@ -340,7 +400,7 @@ tryagain:
>         entry = g_new0(MapCacheEntry, 1);
>         pentry->next = entry;
>         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
>-                         ram_offset);
>+                         grant, is_write, ram_offset);
>     } else if (!entry->lock) {
>         if (!entry->vaddr_base || entry->paddr_index != address_index ||
>                 entry->size != cache_size ||
>@@ -348,7 +408,7 @@ tryagain:
>                     test_bit_size >> XC_PAGE_SHIFT,
>                     entry->valid_mapping)) {
>             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
>-                             ram_offset);
>+                             grant, is_write, ram_offset);
>         }
>     }
> 
>@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
>                        uint8_t lock, bool dma,
>                        bool is_write)
> {
>+    bool grant = xen_mr_is_grants(mr);
>+    MapCache *mc = grant ? mapcache_grants : mapcache;
>     uint8_t *p;
> 
>-    mapcache_lock(mapcache);
>-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
>-                               lock, dma, is_write);
>-    mapcache_unlock(mapcache);
>+    if (grant) {
>+        /*
>+         * Grants are only supported via address_space_map(). Anything
>+         * else is considered a user/guest error.
>+         *
>+         * QEMU generally doesn't expect these mappings to ever fail, so
>+         * if this happens we report an error message and abort().
>+         */
>+        if (!lock) {

Nested if conditions that can be flattened, i.e. this could be 

if (grant && !lock)

>+            error_report("Trying access a grant reference without mapping it.");

s/Trying access a grant/Tried to access a grant/

>+            abort();
>+        }
>+    }
>+
>+    mapcache_lock(mc);
>+    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
>+                               lock, dma, grant, is_write);
>+    mapcache_unlock(mc);
>     return p;
> }
> 
>@@ -449,7 +525,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
> 
> ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> {
>-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
>+    ram_addr_t addr;
>+
>+    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
>+    if (addr == RAM_ADDR_INVALID) {
>+        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
>+    }
>+
>+    return addr;
> }
> 
> static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>@@ -460,6 +543,7 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>     hwaddr paddr_index;
>     hwaddr size;
>     int found = 0;
>+    int rc;
> 
>     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
>         if (reventry->vaddr_req == buffer) {
>@@ -502,7 +586,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>     }
> 
>     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
>-    if (munmap(entry->vaddr_base, entry->size) != 0) {
>+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
>+        rc = xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
>+                             entry->size >> mc->bucket_shift);
>+    } else {
>+        rc = munmap(entry->vaddr_base, entry->size);
>+    }
>+
>+    if (rc) {
>         perror("unmap fails");
>         exit(-1);
>     }
>@@ -521,14 +612,24 @@ typedef struct XenMapCacheData {
>     uint8_t *buffer;
> } XenMapCacheData;
> 
>+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
>+{
>+    mapcache_lock(mc);
>+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
>+    mapcache_unlock(mc);
>+}
>+
>+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
>+{
>+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
>+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
>+}
>+
> static void xen_invalidate_map_cache_entry_bh(void *opaque)
> {
>     XenMapCacheData *data = opaque;
> 
>-    mapcache_lock(mapcache);
>-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
>-    mapcache_unlock(mapcache);
>-
>+    xen_invalidate_map_cache_entry_all(data->buffer);
>     aio_co_wake(data->co);
> }
> 
>@@ -543,9 +644,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
>                                 xen_invalidate_map_cache_entry_bh, &data);
>         qemu_coroutine_yield();
>     } else {
>-        mapcache_lock(mapcache);
>-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
>-        mapcache_unlock(mapcache);
>+        xen_invalidate_map_cache_entry_all(buffer);
>     }
> }
> 
>@@ -597,6 +696,7 @@ void xen_invalidate_map_cache(void)
>     bdrv_drain_all();
> 
>     xen_invalidate_map_cache_single(mapcache);
>+    xen_invalidate_map_cache_single(mapcache_grants);
> }
> 
> static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>@@ -632,13 +732,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
>         return NULL;
>     }
> 
>+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) == 0);
>+
>     address_index  = new_phys_addr >> mc->bucket_shift;
>     address_offset = new_phys_addr & (mc->bucket_size - 1);
> 
>     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> 
>     xen_remap_bucket(mc, entry, entry->vaddr_base,
>-                     cache_size, address_index, false, old_phys_addr);
>+                     cache_size, address_index, false,
>+                     false, false, old_phys_addr);
>     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
>                 test_bit_size >> XC_PAGE_SHIFT,
>                 entry->valid_mapping)) {
>diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
>index 65a51aac2e..3d796235dc 100644
>--- a/include/hw/xen/xen-hvm-common.h
>+++ b/include/hw/xen/xen-hvm-common.h
>@@ -16,6 +16,7 @@
> #include <xen/hvm/ioreq.h>
> 
> extern MemoryRegion xen_memory;
>+extern MemoryRegion xen_grants;
> extern MemoryListener xen_io_listener;
> extern DeviceListener xen_device_listener;
> 
>@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
>     do { } while (0)
> #endif
> 
>+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
>+
> static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
> {
>     return shared_page->vcpu_ioreq[i].vp_eport;
>diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
>index dc72f83bcb..19dccf4d71 100644
>--- a/include/sysemu/xen.h
>+++ b/include/sysemu/xen.h
>@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>                    struct MemoryRegion *mr, Error **errp);
> 
> bool xen_mr_is_memory(MemoryRegion *mr);
>+bool xen_mr_is_grants(MemoryRegion *mr);
> 
> #else /* !CONFIG_XEN_IS_POSSIBLE */
> 
>@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
>     return false;
> }
> 
>+static inline bool xen_mr_is_grants(MemoryRegion *mr)
>+{
>+    g_assert_not_reached();
>+    return false;
>+}
>+
> #endif /* CONFIG_XEN_IS_POSSIBLE */
> 
> #endif
>-- 
>2.40.1
>
>


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:48:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:48:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728132.1132970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3BN-0007dB-PZ; Thu, 23 May 2024 07:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728132.1132970; Thu, 23 May 2024 07:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3BN-0007d4-MT; Thu, 23 May 2024 07:48:13 +0000
Received: by outflank-mailman (input) for mailman id 728132;
 Thu, 23 May 2024 07:48:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA3BM-00074k-Sm
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:48:12 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd517669-18d8-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 09:48:11 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-420298ff5b1so68226185e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:48:11 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502baad037sm35879240f8f.71.2024.05.23.00.48.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 00:48:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd517669-18d8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716450490; x=1717055290; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=q9My7+QtGWydpbmgsayo6RoJGxa1O7HSU/jPdnD4aTY=;
        b=rGJfVkisUexLlLu6KqnUJNflpVW8TJogvjtgSMtJ+H+yoYeje7R9OryBjWQ98v5z8f
         JacU53fm5KDR5L9q50APUserea5ib6oQbwn2EjawTTIIRs3m4GSC/JK/GqOxPCTjwDS6
         8NE+cP6O4GnilW4jT33rUfTU2r0LByyP6NXE8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716450490; x=1717055290;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q9My7+QtGWydpbmgsayo6RoJGxa1O7HSU/jPdnD4aTY=;
        b=Vd9xcoQqQQ50equIbCGkS/BIFOcFMaQxgc1fZkTNxuu/8K0NEpBqd/ggh8Xr+IVewr
         FppwEj/NkYVTflQu4+RGMLU9BekjfO6M0zFXOFUt1IqlmkKBriezXbpNnq7nP9TCEcu4
         hU7V8Tf8+qgE7d1V4uFkDuU2A4y6yFuhVDyUEMtxBouCdziqWNxuAetU53a/PQ4F3/ly
         NQQ18D1jvreRMUNZ7PZWwwkES3uyr0FeC5Hn0fdnK+VEHtaUzbD1CStzgIh4jjUpJui3
         DdBnro7Xnc/ud592onyEqqUcNmqOdyn/IaIX7HrDhHHQY1ZNYrM8FTFur6KcIsnlk25t
         5Xig==
X-Gm-Message-State: AOJu0Yyf3G7BLDSWfMYRrHcGgDsbKrPYUzDod0X2lSrffVN08CA3b4Uw
	dhsI8sANkEDp9VUbBFt2C5j/azsE+746P2zM75u1E3UDkEc/9FGoe01MWO1SWOXESmspD13Mygy
	K
X-Google-Smtp-Source: AGHT+IHtWneH+zu/faamtrwyw2Wq4EhN9vOTIYrP0EAaHWSvdOMxmNnnceRBUL+bonHSPZ8YPo67CQ==
X-Received: by 2002:a5d:5447:0:b0:354:d098:8d67 with SMTP id ffacd0b85a97d-354d8da2372mr3411276f8f.57.1716450490485;
        Thu, 23 May 2024 00:48:10 -0700 (PDT)
Date: Thu, 23 May 2024 09:48:09 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v16 4/5] xen/arm: translate virtual PCI bus topology for
 guests
Message-ID: <Zk70udmiriruMt0r@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-5-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240522225927.77398-5-stewart.hildebrand@amd.com>

On Wed, May 22, 2024 at 06:59:23PM -0400, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> There are three  originators for the PCI configuration space access:
> 1. The domain that owns physical host bridge: MMIO handlers are
> there so we can update vPCI register handlers with the values
> written by the hardware domain, e.g. physical view of the registers
> vs guest's view on the configuration space.
> 2. Guest access to the passed through PCI devices: we need to properly
> map virtual bus topology to the physical one, e.g. pass the configuration
> space access to the corresponding physical devices.
> 3. Emulated host PCI bridge access. It doesn't exist in the physical
> topology, e.g. it can't be mapped to some physical host bridge.
> So, all access to the host bridge itself needs to be trapped and
> emulated.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

One unrelated question below.

> ---
> In v15:
> - base on top of ("arm/vpci: honor access size when returning an error")
> In v11:
> - Fixed format issues
> - Added ASSERT_UNREACHABLE() to the dummy implementation of
> vpci_translate_virtual_device()
> - Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
> the logic in the function
> Since v9:
> - Commend about required lock replaced with ASSERT()
> - Style fixes
> - call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
> Since v8:
> - locks moved out of vpci_translate_virtual_device()
> Since v6:
> - add pcidevs locking to vpci_translate_virtual_device
> - update wrt to the new locking scheme
> Since v5:
> - add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
>   case to simplify ifdefery
> - add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
> - reset output register on failed virtual SBDF translation
> Since v4:
> - indentation fixes
> - constify struct domain
> - updated commit message
> - updates to the new locking scheme (pdev->vpci_lock)
> Since v3:
> - revisit locking
> - move code to vpci.c
> Since v2:
>  - pass struct domain instead of struct vcpu
>  - constify arguments where possible
>  - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
> New in v2
> ---
>  xen/arch/arm/vpci.c     | 45 ++++++++++++++++++++++++++++++++---------
>  xen/drivers/vpci/vpci.c | 24 ++++++++++++++++++++++
>  xen/include/xen/vpci.h  | 12 +++++++++++
>  3 files changed, 71 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index b63a356bb4a8..516933bebfb3 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -7,33 +7,53 @@
>  
>  #include <asm/mmio.h>
>  
> -static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
> -                                     paddr_t gpa)
> +static bool vpci_sbdf_from_gpa(struct domain *d,
> +                               const struct pci_host_bridge *bridge,
> +                               paddr_t gpa, pci_sbdf_t *sbdf)
>  {
> -    pci_sbdf_t sbdf;
> +    bool translated = true;
> +
> +    ASSERT(sbdf);
>  
>      if ( bridge )
>      {
> -        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
> -        sbdf.seg = bridge->segment;
> -        sbdf.bus += bridge->cfg->busn_start;
> +        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
> +        sbdf->seg = bridge->segment;
> +        sbdf->bus += bridge->cfg->busn_start;
>      }
>      else
> -        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
> +    {
> +        /*
> +         * For the passed through devices we need to map their virtual SBDF
> +         * to the physical PCI device being passed through.
> +         */
> +        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
> +        read_lock(&d->pci_lock);
> +        translated = vpci_translate_virtual_device(d, sbdf);
> +        read_unlock(&d->pci_lock);

I would consider moving the read_{,un}lock() calls inside
vpci_translate_virtual_device(), if that's the only caller of
vpci_translate_virtual_device().  Maybe further patches add other
instances that call from an already locked context.

> +    }
>  
> -    return sbdf;
> +    return translated;
>  }
>  
>  static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>                            register_t *r, void *p)
>  {
>      struct pci_host_bridge *bridge = p;
> -    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> +    pci_sbdf_t sbdf;
>      const unsigned int access_size = (1U << info->dabt.size) * 8;
>      const register_t invalid = GENMASK_ULL(access_size - 1, 0);

Do you know why the invalid value is truncated to the access size.
Won't it be simpler to just set the whole variable to 1s? (~0)

TBH, you could just set:

*r = ~(register_t)0;

At the top of the function and get rid of the local invalid variable
plus having to set r on all error paths.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:56:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:56:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728160.1132979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Iu-0001lF-GS; Thu, 23 May 2024 07:56:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728160.1132979; Thu, 23 May 2024 07:56:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Iu-0001l8-Dy; Thu, 23 May 2024 07:56:00 +0000
Received: by outflank-mailman (input) for mailman id 728160;
 Thu, 23 May 2024 07:55:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA3It-0001l2-SH
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:55:59 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e40aef12-18d9-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:55:58 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-354de3c5d00so737834f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:55:58 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3502b89573csm35903138f8f.29.2024.05.23.00.55.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 00:55:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e40aef12-18d9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716450958; x=1717055758; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=utFsS4D+cK7bzRiCAbwpZ2XGU8VFrSx5GfBbC/QBMxs=;
        b=bIHt4B0+8z9GH/t3FY+G9NLTvV5hjr3bvKeZ5131fcd0UJ0+k/2dXH7YHqPvbYNIWb
         rjMkv3DZGE4nRwrxa7TE47j3J3ird0d9mB3WibOyqoCBjEqUm2xwe9bMgDlnLNjdk4iN
         +YGK9PcHaDqxy2SUEoEfORBuGXFFenHSPrh7g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716450958; x=1717055758;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=utFsS4D+cK7bzRiCAbwpZ2XGU8VFrSx5GfBbC/QBMxs=;
        b=bP2dq+uEKutKBJDHsg4pnCAj7t9+ORdnaWfSUQrFsHlPe3ewQj3q+fxUR9p/StXw+x
         Mt6XG/cagHC8Lvo6SdyLkHP21X9krmRuURrJXLhoLfQDP2PTYz1AdwHvAbJatsbxsVwa
         4otwtx1eU1RbteN7Mh6l4a+lP0BzRuEUqHb1k2e3+stH0EceY90kmxwTHq9x9gp58V2N
         YfSlxtPBKTLdh2tyNWV+hbrDK6APKJwdbCOOgXal+DrqFZc8yAQ+3Gm58OkUeRrFOpin
         QzWkgHS6fcdQNDIbYL//ghbMbgHrm4dVrXhSkJaD/98bNGwFR8faahWEDO2TXaVzHOBD
         /8VA==
X-Gm-Message-State: AOJu0YwuII5BpRhN205oRz2zsCKqtXfK6QyzLefBcg0uiimcxON7kVrz
	IS+hd1YWLQJR6o9LAckF/eudtCJ2zL2nFFA8M2WMiCv8k6WkhTbwyfTLwOAqReY=
X-Google-Smtp-Source: AGHT+IE4WSsAWvuTC4CXl2m8B4VSvv2BRdgFxuFm3fBD5BHb0pQ/XLqZGSgtvW+t+mWdURmLJ6YTMQ==
X-Received: by 2002:adf:fd4b:0:b0:34c:fd92:3359 with SMTP id ffacd0b85a97d-354f7521b58mr1291595f8f.21.1716450957943;
        Thu, 23 May 2024 00:55:57 -0700 (PDT)
Date: Thu, 23 May 2024 09:55:56 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH v16 1/5] arm/vpci: honor access size when returning an
 error
Message-ID: <Zk72jPtd9iXhChbc@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-2-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240522225927.77398-2-stewart.hildebrand@amd.com>

On Wed, May 22, 2024 at 06:59:20PM -0400, Stewart Hildebrand wrote:
> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> 
> Guest can try to read config space using different access sizes: 8,
> 16, 32, 64 bits. We need to take this into account when we are
> returning an error back to MMIO handler, otherwise it is possible to
> provide more data than requested: i.e. guest issues LDRB instruction
> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
> register.

Shouldn't this be taken care of in the trap handler subsystem, rather
than forcing each handler to ensure the returned data matches the
access size?

IOW, something like:

diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index 96c740d5636c..b7e12df85f87 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -37,6 +37,7 @@ static enum io_state handle_read(const struct mmio_handler *handler,
         return IO_ABORT;

     r = sign_extend(dabt, r);
+    r = r & GENMASK_ULL((1U << dabt.size) * 8 - 1, 0);

     set_user_reg(regs, dabt.reg, r);

(not even build tested)

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 07:59:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 07:59:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728167.1132990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Lx-0002Kz-Vv; Thu, 23 May 2024 07:59:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728167.1132990; Thu, 23 May 2024 07:59:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3Lx-0002Ks-RX; Thu, 23 May 2024 07:59:09 +0000
Received: by outflank-mailman (input) for mailman id 728167;
 Thu, 23 May 2024 07:59:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA3Lw-0002Km-G5
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 07:59:08 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 54ae1ae2-18da-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 09:59:07 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-354f69934feso335156f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 00:59:07 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354cc7738dasm7502550f8f.18.2024.05.23.00.59.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 00:59:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54ae1ae2-18da-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716451147; x=1717055947; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=I9VojLnm2ZfaWI+uLbpGHvHlkZwcZw8nsgXzGNTOqRY=;
        b=Rv6temIX4SiBdmedl+z2xj8qK0OS45mORKKITOuEn+L0LimhsrzeB/Kuw+zxoCgBMN
         zgls6B85BXrKDAT3hYsQUe1dLX3uaGAJVeh4MqwimkLPwfdnDwYPLuW6Y0nFCKK2YSIE
         dXAN1WNsgvaZbe2Yb5/2DawAKCKhtu3x+mWnw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716451147; x=1717055947;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=I9VojLnm2ZfaWI+uLbpGHvHlkZwcZw8nsgXzGNTOqRY=;
        b=Ui1tVtKGE+zg6I3+accrafys64SgHWD2xKlPdUKWE9QBAtVfKl4HijvP6DCVzhUfZX
         uu0dmmLL33HFloOXc+SyXFIW9iIqgxUPAcQFjLTkBI9Z9WoO8N31x5mCWPDZflE8u1W0
         0BviuaBznVOQFkdSWkOw1ZSqfGFJ2h57VN3yR8kMRBzTgUDaPSxnhJ3K1sc2sku+72ho
         vHg3vEUlxKnvPaE1H7kylf+pp/E7Ipu0TsUOEIlW+50orQtkPmHe1gQhzBCnrsQowkan
         694CANzxGfw7Gjz0cWbWxlutwMvknucJtI/l6VKCGHhiq7DREaJohE9I+IHwcJeOPTXc
         qSCQ==
X-Gm-Message-State: AOJu0YxlyVvDb0niAyzfZ/A1nGTiqwM5lqgpXynjqcIdey71oBXxPD+4
	w2XiOSyEXdKR9pu3EoJoMVkjJ0hO3vp5Q+k/8XrzPL+BfcEgSVv7d3EDH+Tik94=
X-Google-Smtp-Source: AGHT+IHRqUPt9IBG8R7IU6KhIfJGNBPA14vbOwqsq+WDUwwF95zJO+zxekmdc0NiXGYScEdmeQeJmg==
X-Received: by 2002:adf:fd85:0:b0:351:d2e3:68e9 with SMTP id ffacd0b85a97d-354f74ffd53mr1237304f8f.1.1716451146892;
        Thu, 23 May 2024 00:59:06 -0700 (PDT)
Date: Thu, 23 May 2024 09:59:05 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH v16 5/5] xen/arm: account IO handlers for emulated PCI
 MSI-X
Message-ID: <Zk73ScyJvdIrD_FN@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-6-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240522225927.77398-6-stewart.hildebrand@amd.com>

On Wed, May 22, 2024 at 06:59:24PM -0400, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> At the moment, we always allocate an extra 16 slots for IO handlers
> (see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated
> MSI-X registers we need to explicitly tell that we have additional IO
> handlers, so those are accounted.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Acked-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> ---
> This depends on a constant defined in ("vpci: add initial support for
> virtual PCI bus topology"), so cannot be committed without the
> dependency.
> 
> Since v5:
> - optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is
>   defined unconditionally
> New in v5
> ---
>  xen/arch/arm/vpci.c | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 516933bebfb3..4779bbfa9be3 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -132,6 +132,8 @@ static int vpci_get_num_handlers_cb(struct domain *d,
>  
>  unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
>  {
> +    unsigned int count;
> +
>      if ( !has_vpci(d) )
>          return 0;
>  
> @@ -152,7 +154,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
>       * For guests each host bridge requires one region to cover the
>       * configuration space. At the moment, we only expose a single host bridge.
>       */
> -    return 1;
> +    count = 1;
> +
> +    /*
> +     * There's a single MSI-X MMIO handler that deals with both PBA
> +     * and MSI-X tables per each PCI device being passed through.
> +     * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV.
> +     */
> +    if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) )
> +        count += VPCI_MAX_VIRT_DEV;

I think this was already raised in a previous version, at some point
you will need to consider making this a linker list or similar.  The
array is also not very helpful, as you still need to iterate over the
slots in order to find which handler should dispatch the access.

(Not that I oppose to this patch, but the handlers array cannot be
expanded indefinitely).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 08:04:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:04:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728178.1133000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3RE-0005Gv-N4; Thu, 23 May 2024 08:04:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728178.1133000; Thu, 23 May 2024 08:04:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3RE-0005Go-Jz; Thu, 23 May 2024 08:04:36 +0000
Received: by outflank-mailman (input) for mailman id 728178;
 Thu, 23 May 2024 08:04:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TgV8=M2=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sA3RD-0005Gi-9Q
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:04:35 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 16a6a3d4-18db-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 10:04:32 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5206a5854adso6603405e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:04:33 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f35ad3c7sm5248350e87.35.2024.05.23.01.04.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 01:04:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16a6a3d4-18db-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716451472; x=1717056272; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=X2UdYcGeBEiDNLatqEDT2A8B9sTUunerNT7zZ2p/WiE=;
        b=hT8YeGrtqVbl92xB8pP/AxZcy7wqygDsxisPcAtIUPlI/YtFCB6uyT4AyCztKcN+it
         nOHfkPEkhfMu71Mc/m4nLdtUieTbh6xAAZ8gsRhyNXZ9P/Hxick4wBDKhclQL0q3JFei
         G22Pi6xY4qROhErczoB/k7H2MhpJhBPOoihBvZ6ZlUBzSnnisyWybQEztpckRu+CPkUq
         zDSCXhLP5B8KXF58gEDilOhc6NLk2R7JQvTz1qVy7ZR8XMSH9692bu8CXqMC5pvGmxQD
         xegM3XmOn/wABMgJ4m9hp/BV08goYEwnNIAiDgINrz0QsgxTMloNhFllRm/pO5Fl1Bik
         x3BA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716451472; x=1717056272;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=X2UdYcGeBEiDNLatqEDT2A8B9sTUunerNT7zZ2p/WiE=;
        b=mSYWKbQsy+Ap3Qz3au7PqCBymaM5j6a1AsP95mz7aWFZqqg5IrwM4TsMUH1DqNUS/Y
         sbbjBWFd/fh7SWzyK2p2HdDCiN2gECtkBgWnZrh0fjVuckwrf4q4bvWLT6l6XGXJThke
         kjELIhj487kdldr7CEoOcYk1NjfD50rp07RsY3fmj68HJbjC0C4F/151eO1+ycAWvA6f
         /yBhgVc/9ntU62gletlmsBnKqNk2BkQvXhiTZ6xgPRfjkepI1QvRjql6ccmyEGyBBIva
         K9MlSJORQb4vLDZWoA38omADIdBgwjQXrZpwzkycYkZCgRd5wLOeJGBR9LZMh7GQc1kC
         gvWA==
X-Gm-Message-State: AOJu0Ywy5UZr3mcZF07mM7GKEluwRIsGV7/iaeLZGn0JJ6gMhF+lGWjO
	F8AeTDEwBEifv2DrVjUpRg6PV4lSD0b0jyDN9jAaOMJ5DFglY9vk
X-Google-Smtp-Source: AGHT+IGy2lqeeGLvLdZyq1Nzxq1SnkMMcHXq3cAIbUUtiTRHx+7HniiYP9D+sNg59A2J/RFdD198Kw==
X-Received: by 2002:a05:6512:3ee:b0:518:9cbf:cd6b with SMTP id 2adb3069b0e04-526bd69494fmr2500440e87.13.1716451472368;
        Thu, 23 May 2024 01:04:32 -0700 (PDT)
Message-ID: <a9fdb75d26839c31a8b0ceda9c940b2e13a89bab.camel@gmail.com>
Subject: Re: [for-4.19] Re: [XEN PATCH v3] arm/mem_access: add conditional
 build of mem_access.c
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, Tamas K Lengyel <tamas@tklengyel.com>, 
 Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, Stefano
 Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>,  Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Alexandru Isaila
 <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>
Date: Thu, 23 May 2024 10:04:31 +0200
In-Reply-To: <fb426cc8-fda9-4e21-983d-37649499e0e4@xen.org>
References: 
	<b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
	 <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
	 <fb426cc8-fda9-4e21-983d-37649499e0e4@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hi Julien,

On Wed, 2024-05-22 at 21:50 +0100, Julien Grall wrote:
> Hi,
>=20
> Adding Oleksii as the release manager.
>=20
> On 22/05/2024 19:27, Tamas K Lengyel wrote:
> > On Fri, May 10, 2024 at 8:32=E2=80=AFAM Alessandro Zucchelli
> > <alessandro.zucchelli@bugseng.com> wrote:
> > >=20
> > > In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
> > > changes are done:
> > > revert preprocessor conditional changes to xen/mem_access.h which
> > > had it build unconditionally, add conditional build for
> > > xen/mem_access.c
> > > as well and provide stubs in asm/mem_access.h for the users of
> > > this
> > > header.
> > >=20
> > > Signed-off-by: Alessandro Zucchelli
> > > <alessandro.zucchelli@bugseng.com>
> >=20
> > Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
>=20
> Oleksii, would you be happy if this patch is committed for 4.19?
Sure:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

>=20
> BTW, do you want to be release-ack every bug until the hard code
> freeze?=20
> Or would you be fine to levea the decision to the maintainers?
I would prefer to leave the decision to the maintainers.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 23 08:05:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:05:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728182.1133010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3SX-0005ti-W6; Thu, 23 May 2024 08:05:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728182.1133010; Thu, 23 May 2024 08:05:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3SX-0005tb-TT; Thu, 23 May 2024 08:05:57 +0000
Received: by outflank-mailman (input) for mailman id 728182;
 Thu, 23 May 2024 08:05:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TgV8=M2=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sA3SX-0005tV-8L
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:05:57 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4844ebe6-18db-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 10:05:56 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2e78fe9fc2bso31366091fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:05:56 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f35ba564sm5245334e87.99.2024.05.23.01.05.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 01:05:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4844ebe6-18db-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716451556; x=1717056356; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=cPE1TsUCa10n5DcZIyd/0GWGnxDF2jkNsyqFqDPCUcA=;
        b=LifVNs7KR5qHrg60TdRcyLM+qsWr/M+j2yv9Vk6ZFl0PDBLuS2nynpvzkX043s13F7
         tsxSsRZ8RHwaxlA6VpeJ+Z1fLG2wGf4EJlwxyT8tgQ1gbDnianh79zJ90b0dtHsKIKGo
         XB1cM37DGVZQGWGynX/2khY4FcvnJMHq3uF8UVbyHTlj+5DCm5/s3cwwrqsZ5N7JJI2Q
         yxtcWqpNmXn+ebwytwf3iAShJ2EouK0wH2HJMed5VkLSVnBSDVtoVbmYfr0CkTyRcEd3
         /ZqHhHVy0bSieXdbfN9n4+PztP/ayYW7QqBakHaKJd7QC0oh/2bA8fFj2T4YwWNgm2Wy
         XzOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716451556; x=1717056356;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cPE1TsUCa10n5DcZIyd/0GWGnxDF2jkNsyqFqDPCUcA=;
        b=hBfVuPY1wdRXp0ZJwwkAhSz41iO8TlIIUrK1qa4gSb/ce2nSCmGTBQGyb2EO+8oTJM
         hmgeKxf6rEPldX2sQMPVJY1ZNDmFTjO24+B9IVOrDB0GRJ0kFKF0q8nPw6tcARODePaJ
         FrJz/+UTOnou7zKYSqOOThzSJznnA56fhKggtPg951DRzACtjrId9j4JObod7HS1zbnW
         nAK/pAHznuPDMRYhUrRm1tdjkTIlcdAn5MF5NgtlL+mWkTL9/0wcm5lEywGQ2Xd1QJm+
         Dk4spvQ2vLySncEuRie9Ik2EojNbdknTg73UrQDnj2lAEOtkV+94G+k3Iq/cL02Zxzv+
         znlA==
X-Forwarded-Encrypted: i=1; AJvYcCWTQteuBSSCNs9hJMlZ4AZ9f3n1JIlJdbcNp4FUe87eR98c/GjFZSE6+EAwB2lGZEXqZno7hQZJ8KCj5/EAP4kog8aGZ8wvFU9vibr9qDE=
X-Gm-Message-State: AOJu0YxgDLoTGfZVqyd4DQO6oG6LDMv2XBr/hfZFJVqXDunOmgSbOBPT
	tEau7xVOUwH2PwklBeZzUJnZ7JTmn5eh1vAVjMnSAmHlBLGz+024rvBoFS+k
X-Google-Smtp-Source: AGHT+IFMkUL8eZ1Wenh2hDb/BV1olkuVdACAvLiLVtFiE43JJkKfCc9A6N4V8+DX5tUQANNWbNZN/g==
X-Received: by 2002:ac2:4c10:0:b0:518:95b6:176f with SMTP id 2adb3069b0e04-526c068ea7cmr2425012e87.50.1716451555494;
        Thu, 23 May 2024 01:05:55 -0700 (PDT)
Message-ID: <58c63d645b27dbacdb6264b30ff78a74dc0acaa0.camel@gmail.com>
Subject: Re: [PATCH] x86/shadow: don't leave trace record field uninitialized
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	 <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Roger Pau
 =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>, George Dunlap
	 <george.dunlap@cloud.com>
Date: Thu, 23 May 2024 10:05:54 +0200
In-Reply-To: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
References: <5133a01d-346d-4779-b319-a156ef944669@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

T24gV2VkLCAyMDI0LTA1LTIyIGF0IDEyOjE3ICswMjAwLCBKYW4gQmV1bGljaCB3cm90ZToKPiBU
aGUgZW11bGF0aW9uX2NvdW50IGZpZWxkIGlzIHNldCBvbmx5IGNvbmRpdGlvbmFsbHkgcmlnaHQg
bm93Lgo+IENvbnZlcnQKPiBhbGwgZmllbGQgc2V0dGluZyB0byBhbiBpbml0aWFsaXplciwgdGh1
cyBndWFyYW50ZWVpbmcgdGhhdCBmaWVsZCB0bwo+IGJlCj4gc2V0IHRvIDAgKGRlZmF1bHQgaW5p
dGlhbGl6ZWQpIHdoZW4gR1VFU1RfUEFHSU5HX0xFVkVMUyAhPSAzLgo+IAo+IFdoaWxlIHRoZXJl
IGFsc28gZHJvcCB0aGUgImV2ZW50IiBsb2NhbCB2YXJpYWJsZSwgdGh1cyBlbGltaW5hdGluZyBh
bgo+IGluc3RhbmNlIG9mIHRoZSBiZWluZyBwaGFzZWQgb3V0IHUzMiB0eXBlLgo+IAo+IENvdmVy
aXR5IElEOiAxNTk4NDMwCj4gRml4ZXM6IDlhODZhYzFhYTNkMiAoInhlbnRyYWNlIDUvNzogQWRk
aXRpb25hbCB0cmFjaW5nIGZvciB0aGUgc2hhZG93Cj4gY29kZSIpCj4gU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZWxlYXNlLWFja2VkLWJ5OiBPbGVrc2lp
IEt1cm9jaGtvIDxvbGVrc2lpLmt1cm9jaGtvLmNvbT4KCn4gT2xla3NpaQo+IAo+IC0tLSBhL3hl
bi9hcmNoL3g4Ni9tbS9zaGFkb3cvbXVsdGkuYwo+ICsrKyBiL3hlbi9hcmNoL3g4Ni9tbS9zaGFk
b3cvbXVsdGkuYwo+IEBAIC0yMDkzLDIwICsyMDkzLDE4IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCB0
cmFjZV9zaGFkb3dfZW11bGF0ZSgKPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZ3Vlc3RfbDFl
X3QgZ2wxZSwgd3JpdGVfdmFsOwo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBndWVzdF92YV90
IHZhOwo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1aW50MzJfdCBmbGFnczoyOSwgZW11bGF0
aW9uX2NvdW50OjM7Cj4gLcKgwqDCoMKgwqDCoMKgIH0gZDsKPiAtwqDCoMKgwqDCoMKgwqAgdTMy
IGV2ZW50Owo+IC0KPiAtwqDCoMKgwqDCoMKgwqAgZXZlbnQgPSBUUkNfU0hBRE9XX0VNVUxBVEUg
fCAoKEdVRVNUX1BBR0lOR19MRVZFTFMtMik8PDgpOwo+IC0KPiAtwqDCoMKgwqDCoMKgwqAgZC5n
bDFlID0gZ2wxZTsKPiAtwqDCoMKgwqDCoMKgwqAgZC53cml0ZV92YWwubDEgPSB0aGlzX2NwdSh0
cmFjZV9lbXVsYXRlX3dyaXRlX3ZhbCk7Cj4gLcKgwqDCoMKgwqDCoMKgIGQudmEgPSB2YTsKPiAr
wqDCoMKgwqDCoMKgwqAgfSBkID0gewo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC5nbDFlID0g
Z2wxZSwKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAud3JpdGVfdmFsLmwxID0gdGhpc19jcHUo
dHJhY2VfZW11bGF0ZV93cml0ZV92YWwpLAo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC52YSA9
IHZhLAo+IMKgI2lmIEdVRVNUX1BBR0lOR19MRVZFTFMgPT0gMwo+IC3CoMKgwqDCoMKgwqDCoCBk
LmVtdWxhdGlvbl9jb3VudCA9IHRoaXNfY3B1KHRyYWNlX2V4dHJhX2VtdWxhdGlvbl9jb3VudCk7
Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLmVtdWxhdGlvbl9jb3VudCA9Cj4gdGhpc19jcHUo
dHJhY2VfZXh0cmFfZW11bGF0aW9uX2NvdW50KSwKPiDCoCNlbmRpZgo+IC3CoMKgwqDCoMKgwqDC
oCBkLmZsYWdzID0gdGhpc19jcHUodHJhY2Vfc2hhZG93X3BhdGhfZmxhZ3MpOwo+ICvCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIC5mbGFncyA9IHRoaXNfY3B1KHRyYWNlX3NoYWRvd19wYXRoX2ZsYWdz
KSwKPiArwqDCoMKgwqDCoMKgwqAgfTsKPiDCoAo+IC3CoMKgwqDCoMKgwqDCoCB0cmFjZShldmVu
dCwgc2l6ZW9mKGQpLCAmZCk7Cj4gK8KgwqDCoMKgwqDCoMKgIHRyYWNlKFRSQ19TSEFET1dfRU1V
TEFURSB8ICgoR1VFU1RfUEFHSU5HX0xFVkVMUyAtIDIpIDw8IDgpLAo+ICvCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCBzaXplb2YoZCksICZkKTsKPiDCoMKgwqDCoCB9Cj4gwqB9Cj4gwqAjZW5k
aWYgLyogQ09ORklHX0hWTSAqLwoK



From xen-devel-bounces@lists.xenproject.org Thu May 23 08:21:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:21:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728193.1133019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3hu-0001Zn-Cb; Thu, 23 May 2024 08:21:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728193.1133019; Thu, 23 May 2024 08:21:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3hu-0001Zg-9Z; Thu, 23 May 2024 08:21:50 +0000
Received: by outflank-mailman (input) for mailman id 728193;
 Thu, 23 May 2024 08:21:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA3ht-0001Za-K9
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:21:49 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f040bc2-18dd-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 10:21:47 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so307453366b.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:21:47 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c822fsm1896097166b.138.2024.05.23.01.21.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:21:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f040bc2-18dd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716452506; x=1717057306; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Vi031+qGJOuHKgdSFgi7xqaLmxkWTvLlJ5rgn7hheEw=;
        b=P/Cn+rax1C5jqmKBnGmINfjKNuFO5eweGuLNBNpvWDIFkYZUXcS3FeVKhgSQ1LKvJL
         IFeDA8aQBswtoihZ7byQLKxGfvpnQAEcsNWnJAUBSWmVqLanyzmLyZ3gBge3sHJ9WwjZ
         6eQVSwkm4HjU1kRH+rJseAvRZjFL1etU8UYyCmNjw9TadHIpoXBYMVQ/fn+5/RyUpg2/
         AxLYU4nBAo1fM/BLRZ4SKwbOU0Tv7wc4frLHtp1LIJVktbE0860jq3wQ9drBhuoJmEOs
         r3M0df3UfmlRXco4jEdb/D3rm8aqIeSRyQSaQrR8Cud3DOAWnUClohL+11Hd2+ILIsg7
         gdWw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716452506; x=1717057306;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Vi031+qGJOuHKgdSFgi7xqaLmxkWTvLlJ5rgn7hheEw=;
        b=GJPoTD8c64AWAeOhJfEJcMkUQhFzv8MfAClFah9InB5r0gZWuNEQRkgG5rbW+mCHb8
         FM6XL/YIAIt6BWQj/9kYHKV/kr/pcyC/cCuKrI6sfCndZRKMr5bVIONfXaNjlKpytEFz
         /jLCKTSMAocSWImpAzjuvHweqY14DzsbtBUh7YQPbNJUQmJAczEImIF/68IRaZ9e/52h
         OajiEurxi2XPH//In7aYQOVjmiDjqKFpu1gwOAx2MO6+dbPBtgRiIr3Y9UL87YHdBldX
         9XOPbOkzMT6aVI6S0zVb3TMhVXZ/SwILPppD9UCnHnYJfH4e9DVydoFpNYiFsflDw4am
         DXzA==
X-Forwarded-Encrypted: i=1; AJvYcCX69BditXOe/Som0FsMiP7a+ySKtY2GGWgRKfQ1U2r3HZp15PHtUttFOWhuAEEM5to0vsPCBC90z/m3QjgNY1EwTKnDMeigzOVmfWByyqo=
X-Gm-Message-State: AOJu0YyjEL+KMs84UAyy3CmfLNPO6Bk5XleJiBdb8tbP8t2nl6vZMBhU
	w/dQifsVR7fUf7tKiPO4iu41zIe9zSWGPxBZlfynQZyxPJlqjvy6PNNy7JDL1BI=
X-Google-Smtp-Source: AGHT+IE6SgDfMfz2P1eoa/c5xe/G9khVO0tHSFiswaPSnQoW34giQqKJ6pGN2Zr9USFyLE/xDeJvlw==
X-Received: by 2002:a17:906:3bc3:b0:a5a:3908:f4ad with SMTP id a640c23a62f3a-a623e8e75f1mr129920766b.10.1716452506585;
        Thu, 23 May 2024 01:21:46 -0700 (PDT)
Message-ID: <3937c37d-8872-463d-8881-8794c0966580@suse.com>
Date: Thu, 23 May 2024 10:21:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/4] tools: Import standalone sd_notify()
 implementation from systemd
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-3-andrew.cooper3@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240516185804.3309725-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 16.05.24 20:58, Andrew Cooper wrote:
> ... in order to avoid linking against the whole of libsystemd.
> 
> Only minimal changes to the upstream copy, to function as a drop-in
> replacement for sd_notify() and as a header-only library.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

With s/cleanup(sd_closep)/cleanup(xen_sd_closep)/

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin Török <edwin.torok@cloud.com>
> 
> v2:
>   * New
> ---
>   tools/include/xen-sd-notify.h | 98 +++++++++++++++++++++++++++++++++++
>   1 file changed, 98 insertions(+)
>   create mode 100644 tools/include/xen-sd-notify.h
> 
> diff --git a/tools/include/xen-sd-notify.h b/tools/include/xen-sd-notify.h
> new file mode 100644
> index 000000000000..eda9d8b22d9e
> --- /dev/null
> +++ b/tools/include/xen-sd-notify.h
> @@ -0,0 +1,98 @@
> +/* SPDX-License-Identifier: MIT-0 */
> +
> +/*
> + * Implement the systemd notify protocol without external dependencies.
> + * Supports both readiness notification on startup and on reloading,
> + * according to the protocol defined at:
> + * https://www.freedesktop.org/software/systemd/man/latest/sd_notify.html
> + * This protocol is guaranteed to be stable as per:
> + * https://systemd.io/PORTABILITY_AND_STABILITY/
> + *
> + * Differences from the upstream copy:
> + * - Rename/rework as a drop-in replacement for systemd/sd-daemon.h
> + * - Only take the subset Xen cares about
> + * - Respect -Wdeclaration-after-statement
> + */
> +
> +#ifndef XEN_SD_NOTIFY
> +#define XEN_SD_NOTIFY
> +
> +#include <errno.h>
> +#include <stddef.h>
> +#include <stdlib.h>
> +#include <sys/socket.h>
> +#include <sys/un.h>
> +#include <unistd.h>
> +
> +static inline void xen_sd_closep(int *fd) {
> +  if (!fd || *fd < 0)
> +    return;
> +
> +  close(*fd);
> +  *fd = -1;
> +}
> +
> +static inline int xen_sd_notify(const char *message) {
> +  union sockaddr_union {
> +    struct sockaddr sa;
> +    struct sockaddr_un sun;
> +  } socket_addr = {
> +    .sun.sun_family = AF_UNIX,
> +  };
> +  size_t path_length, message_length;
> +  ssize_t written;
> +  const char *socket_path;
> +  int __attribute__((cleanup(sd_closep))) fd = -1;
> +
> +  /* Verify the argument first */
> +  if (!message)
> +    return -EINVAL;
> +
> +  message_length = strlen(message);
> +  if (message_length == 0)
> +    return -EINVAL;
> +
> +  /* If the variable is not set, the protocol is a noop */
> +  socket_path = getenv("NOTIFY_SOCKET");
> +  if (!socket_path)
> +    return 0; /* Not set? Nothing to do */
> +
> +  /* Only AF_UNIX is supported, with path or abstract sockets */
> +  if (socket_path[0] != '/' && socket_path[0] != '@')
> +    return -EAFNOSUPPORT;
> +
> +  path_length = strlen(socket_path);
> +  /* Ensure there is room for NUL byte */
> +  if (path_length >= sizeof(socket_addr.sun.sun_path))
> +    return -E2BIG;
> +
> +  memcpy(socket_addr.sun.sun_path, socket_path, path_length);
> +
> +  /* Support for abstract socket */
> +  if (socket_addr.sun.sun_path[0] == '@')
> +    socket_addr.sun.sun_path[0] = 0;
> +
> +  fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0);
> +  if (fd < 0)
> +    return -errno;
> +
> +  if (connect(fd, &socket_addr.sa, offsetof(struct sockaddr_un, sun_path) + path_length) != 0)
> +    return -errno;
> +
> +  written = write(fd, message, message_length);
> +  if (written != (ssize_t) message_length)
> +    return written < 0 ? -errno : -EPROTO;
> +
> +  return 1; /* Notified! */
> +}
> +
> +static inline int sd_notify(int unset_environment, const char *message) {
> +    int r = xen_sd_notify(message);
> +
> +    if (unset_environment)
> +        unsetenv("NOTIFY_SOCKET");
> +
> +    return r;
> +}
> +
> +#endif /* XEN_SD_NOTIFY */



From xen-devel-bounces@lists.xenproject.org Thu May 23 08:22:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:22:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728202.1133047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3im-0002Sk-TM; Thu, 23 May 2024 08:22:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728202.1133047; Thu, 23 May 2024 08:22:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3im-0002Sd-Oo; Thu, 23 May 2024 08:22:44 +0000
Received: by outflank-mailman (input) for mailman id 728202;
 Thu, 23 May 2024 08:22:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA3il-00026a-88
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:22:43 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a019e07e-18dd-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 10:22:42 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a5cb0e6b7so1120880366b.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:22:42 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a616b07164asm564603666b.116.2024.05.23.01.22.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:22:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a019e07e-18dd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716452562; x=1717057362; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rnnvZryhw0yRCTtGjZKhvo1G7tBBUoE48gETUVXle/4=;
        b=LOFZUm/DyoZPEs5XZLukmrTdcYFnZSha3ir4i8PKep1VPlSgmXaIbKDzp0E5QP5H/R
         grXAY9XkBkVc4KWC4wBn8E/eQVsvB+1S4Jt7OAwC7oVbsIrm99oJWEicUmF4ZqcQET0g
         APkLipN27ZxALFfla6iTfVoNP9I1l/GTa7DDRRCQA9IvoCeziNi4pykcLB7juTCyYhf8
         DuCdU4i/8KNjCp7DdH88Q7raNBeG1u7zxDSETrEGKjpQ62qxiWnwIQoIu6TO1yIsFTwo
         FodIKMfU1vMY2ZZS/+FVTWXQ3yBHZKxoN9+zq6BdG15wlE8RfCyqG3NLqRPCZ+FM/Cwl
         YH6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716452562; x=1717057362;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rnnvZryhw0yRCTtGjZKhvo1G7tBBUoE48gETUVXle/4=;
        b=Qo6q1mPbth4CD+sjbZbcqBchYCOt7dZYISTfioyF7B7ybq2mhqEH7TPC1hwWnHrDot
         OnoGTzsCqXIbuaX5oEGwIb2xb7YZL5O0YWBDBd/Jwr9QxH1OHO0QaMl4njTg15PM2hJQ
         0E5xmbbfAoVH8WuvkzP3viTL0AhEGAcqVAm4FrQy/uhEVs+10GZ+QQDhu8dhUXTqUtnQ
         uEa3Yk1b+Ik782UjLuDcnlZjdISfu3Odmio73JiJpvdyXLH/Oi1nzw7pCFVKS/X6aIC+
         KqlN8zT8xO5l2MmhSBvAWWz4JasH+ByIFtK5XNaVXF+GZfex74CGI7QIYkwHlEuPN0/H
         kdbA==
X-Forwarded-Encrypted: i=1; AJvYcCXAVZEHRdDhhpBYDm8nZSZc6W/IW3Pswvy1S4ueavmMu/Ic11PWM6jugv4jFHOCFeX2jnZKlkx+OnxHTyee+jVrd/ihcfvp1XkLbMx6hzs=
X-Gm-Message-State: AOJu0YwzYdbZy79r31lVKgaRJueBLS5yUPACW6EcbQtXWhMlPT/cdbby
	FVDl59N7ZKwzJCX4+sGQr2xYhlHuGwo53sPXCf1IbjKHnpMEJ0jRNxhHKUMPmDI=
X-Google-Smtp-Source: AGHT+IEjRK8mZUWOCvE4tRb8yz/3DAIX24rSVGFcr9neimXd+6PkVGLlmWjCpqwihbyFsVjMwHsPbg==
X-Received: by 2002:a17:906:6d85:b0:a5a:84f9:df73 with SMTP id a640c23a62f3a-a62280a12e6mr266281566b.38.1716452562064;
        Thu, 23 May 2024 01:22:42 -0700 (PDT)
Message-ID: <b2c66956-97a8-44da-b2fd-c50f84cbc170@suse.com>
Date: Thu, 23 May 2024 10:22:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/4] tools/{c,o}xenstored: Don't link against
 libsystemd
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-4-andrew.cooper3@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240516185804.3309725-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 16.05.24 20:58, Andrew Cooper wrote:
> Use the local freestanding wrapper instead.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Juergen Gross <jgross@suse.com> # tools/xenstored


Juergen

> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin Török <edwin.torok@cloud.com>
> 
> v2:
>   * Redo almost from scratch, using the freestanding wrapper instead.
> ---
>   tools/ocaml/xenstored/Makefile        | 2 --
>   tools/ocaml/xenstored/systemd_stubs.c | 2 +-
>   tools/xenstored/Makefile              | 5 -----
>   tools/xenstored/posix.c               | 4 ++--
>   4 files changed, 3 insertions(+), 10 deletions(-)
> 
> diff --git a/tools/ocaml/xenstored/Makefile b/tools/ocaml/xenstored/Makefile
> index e8aaecf2e630..a8b8bb64698e 100644
> --- a/tools/ocaml/xenstored/Makefile
> +++ b/tools/ocaml/xenstored/Makefile
> @@ -4,8 +4,6 @@ include $(OCAML_TOPLEVEL)/common.make
>   
>   # Include configure output (config.h)
>   CFLAGS += -include $(XEN_ROOT)/tools/config.h
> -CFLAGS-$(CONFIG_SYSTEMD)  += $(SYSTEMD_CFLAGS)
> -LDFLAGS-$(CONFIG_SYSTEMD) += $(SYSTEMD_LIBS)
>   
>   CFLAGS  += $(CFLAGS-y)
>   CFLAGS  += $(APPEND_CFLAGS)
> diff --git a/tools/ocaml/xenstored/systemd_stubs.c b/tools/ocaml/xenstored/systemd_stubs.c
> index f4c875075abe..7dbbdd35bf30 100644
> --- a/tools/ocaml/xenstored/systemd_stubs.c
> +++ b/tools/ocaml/xenstored/systemd_stubs.c
> @@ -25,7 +25,7 @@
>   
>   #if defined(HAVE_SYSTEMD)
>   
> -#include <systemd/sd-daemon.h>
> +#include <xen-sd-notify.h>
>   
>   CAMLprim value ocaml_sd_notify_ready(value ignore)
>   {
> diff --git a/tools/xenstored/Makefile b/tools/xenstored/Makefile
> index e0897ed1ba30..09adfe1d5064 100644
> --- a/tools/xenstored/Makefile
> +++ b/tools/xenstored/Makefile
> @@ -9,11 +9,6 @@ xenstored: LDLIBS += $(LDLIBS_libxenctrl)
>   xenstored: LDLIBS += -lrt
>   xenstored: LDLIBS += $(SOCKET_LIBS)
>   
> -ifeq ($(CONFIG_SYSTEMD),y)
> -$(XENSTORED_OBJS-y): CFLAGS += $(SYSTEMD_CFLAGS)
> -xenstored: LDLIBS += $(SYSTEMD_LIBS)
> -endif
> -
>   TARGETS := xenstored
>   
>   .PHONY: all
> diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
> index d88c82d972d7..6037d739d013 100644
> --- a/tools/xenstored/posix.c
> +++ b/tools/xenstored/posix.c
> @@ -27,7 +27,7 @@
>   #include <sys/socket.h>
>   #include <sys/un.h>
>   #if defined(HAVE_SYSTEMD)
> -#include <systemd/sd-daemon.h>
> +#include <xen-sd-notify.h>
>   #endif
>   #include <xen-tools/xenstore-common.h>
>   
> @@ -393,7 +393,7 @@ void late_init(bool live_update)
>   #if defined(HAVE_SYSTEMD)
>   	if (!live_update) {
>   		sd_notify(1, "READY=1");
> -		fprintf(stderr, SD_NOTICE "xenstored is ready\n");
> +		fprintf(stderr, "xenstored is ready\n");
>   	}
>   #endif
>   }



From xen-devel-bounces@lists.xenproject.org Thu May 23 08:22:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:22:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728205.1133059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3ip-0002jf-2v; Thu, 23 May 2024 08:22:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728205.1133059; Thu, 23 May 2024 08:22:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3ip-0002jY-0K; Thu, 23 May 2024 08:22:47 +0000
Received: by outflank-mailman (input) for mailman id 728205;
 Thu, 23 May 2024 08:22:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA3in-00026a-Dp
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:22:45 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a18a0ad6-18dd-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 10:22:44 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-52388d9ca98so11008973e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:22:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c6b1sm1901920166b.57.2024.05.23.01.22.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:22:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a18a0ad6-18dd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716452564; x=1717057364; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=U53RxWXEYKSWLg5Ju6vVotqdHlvFxcIu9IrmCSx409s=;
        b=Uj0JKcVQIVQklOb4uUns5QNfU18qPng+3vVx0PCqaM7E0xEUaW580VHhfUJexixNDZ
         B4kj2h1KhdBk4x2KL7qkQSDIBqlFp/G+XauZtG7Dvgp1beHUKgjMhuJWzhKE+Yrdde8B
         fZiR4lGRAo5QCA8blS+mCwCv7XWYAfxRVA5hhuxhEU2Mm9wa5ZCxTLhNY2LnVMW6thAY
         hEtevASNAl7g1A8EZtQJF2o0Zh5JZUNPnN8yKT8qF47G/2VCjuR05LDzj8SKu9GRqAh6
         nHfuq9jrnegunQgL77Asc4eGqQZN7qyEpyuzrMArqwK+dxXtWfVvE3Ap94ec0O7wNsLv
         8cxQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716452564; x=1717057364;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=U53RxWXEYKSWLg5Ju6vVotqdHlvFxcIu9IrmCSx409s=;
        b=IbGDRIzShOHr2833h9Z3H3GVLTV1IjaYZZCyfJMmyw+dPxMdHokjUnvsziC8kYMKT9
         j7BPZVQ8AGPbukeTKwiybII5FfXRsGtO5wsTn3W+ahnN4Ks74Oz9H4VnlIibGrn7jkDu
         ztmDqS25Cj3JlzmNEOSa0bLsVSuD/x1dwEnuRPeRkHbdH1sqtfhSKwpK0tIrlwlyQZCw
         Vv02x8QoqtS/7pYo8vC06SUvJ7/BKwNu/8PymxAQ7UjzXmieKMk3abRbPc57HTkErtI8
         dw1MaA85pgRRgFJ1Tm98rEtB/33h9bLb/gODBPfOT1vLOsuh32W2bFu8hPrltb/tlWTy
         Ov9g==
X-Forwarded-Encrypted: i=1; AJvYcCUXVF6zCOIAnCYGc6LuMDXeh8SP3yQZ9BTrIM4xNcTpCnrN5/7v2EQNa9FJzzl53dHJFFGfEVJ6ngxoex0l4Tvz6Pi0rPUGHnaOFjSaPSw=
X-Gm-Message-State: AOJu0Yzy97z8/eFz42GSJuLPZmd+eIeWYkxU0evyFMYGP+V+qH54UpHG
	YMLn643/j51slSUcqbO1/5PoWWdTUexlc60R10vhYualyootTsA9DGjJD6RWrQ==
X-Google-Smtp-Source: AGHT+IGMpH5PGljwmz0DduOePvcuPXGZUqLFJuLOTwV0U52MbDtJSnywO+KPg9qBBMJlxzH70aJ1ww==
X-Received: by 2002:a19:f510:0:b0:51d:3675:6a06 with SMTP id 2adb3069b0e04-526c12153f0mr3100052e87.66.1716452564313;
        Thu, 23 May 2024 01:22:44 -0700 (PDT)
Message-ID: <3011751b-aa59-4266-b52d-df661bb9e4ee@suse.com>
Date: Thu, 23 May 2024 10:22:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 2/2] drivers/char: Use sub-page ro API to make just
 xhci dbc cap RO
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
 <83dfaaf5cbd056ffbe381cbd39f0230348257642.1716392340.git-series.marmarek@invisiblethingslab.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <83dfaaf5cbd056ffbe381cbd39f0230348257642.1716392340.git-series.marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.05.2024 17:39, Marek Marczykowski-Górecki wrote:
> Not the whole page, which may contain other registers too. The XHCI
> specification describes DbC as designed to be controlled by a different
> driver, but does not mandate placing registers on a separate page. In fact
> on Tiger Lake and newer (at least), this page do contain other registers
> that Linux tries to use. And with share=yes, a domU would use them too.
> Without this patch, PV dom0 would fail to initialize the controller,
> while HVM would be killed on EPT violation.
> 
> With `share=yes`, this patch gives domU more access to the emulator
> (although a HVM with any emulated device already has plenty of it). This
> configuration is already documented as unsafe with untrusted guests and
> not security supported.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu May 23 08:28:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:28:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728232.1133094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3nt-00045R-Vo; Thu, 23 May 2024 08:28:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728232.1133094; Thu, 23 May 2024 08:28:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3nt-00045K-Sk; Thu, 23 May 2024 08:28:01 +0000
Received: by outflank-mailman (input) for mailman id 728232;
 Thu, 23 May 2024 08:28:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA3ns-00045E-Q3
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:28:00 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5d2af421-18de-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 10:27:59 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a623cf23927so123728266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:27:59 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8967a711sm1227007466b.2.2024.05.23.01.27.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:27:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d2af421-18de-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716452879; x=1717057679; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jhpvbBfwpJWGF5iUcQkjO7ndJQwWC1rVlWpqUpT0O4s=;
        b=SU99fmFqruFm/5CSzURqPh2QCt+P+HAABoHSbdEyiMgmvOfAOpDEQcJ+h841SUUTU3
         3kfyCdg5CS/IdyNpzp62tzbY5/O4yLgE1gsoDAbc1BAGxbtkX3WrbxPKbq9pM4+3j46Z
         Ew1ExOYP3d37+Y2ddY5rfhFtAKRiCP4X5/HC/jDEmPSVcAekviBlfZvXgWoGAWcjPBvJ
         auV1OwPkuvKmDQ5N1YW28KBwBA6NlM2AGtEl959Y7XYCQkkVGs+iCmtCIeHSy9ef+ozh
         Xd+3L7HfMQB1A/eYZt4Ku4X9EA58IfvxSqEmOBHfXSr3uz5S8s7bkvHwfz4h1fvrCN+y
         KP8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716452879; x=1717057679;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jhpvbBfwpJWGF5iUcQkjO7ndJQwWC1rVlWpqUpT0O4s=;
        b=KhDTWAbVPaUOleVuruGIisZFYA1c5o+y3/HPN5YG4Z8PJRk9l6DKlOkSAMAB2GbpyX
         Qxt0KoKijzQXlgwz8BVzrq5ERdNLBDrsLxhH8qtUBfn2ZegC2F4LEEPsvobtkVjE6TEN
         UP9MyhqOINyM/WVImfLmlDJr0tMrHGJRBZYuHMSS+LDgUd27VzJnRKNs6u8LM+tIMNGu
         g51HlfGdaS45N4Y1aOcKAH0q4Kb/Yj1jKFv73Aa7td0D18bPv7N21IoASeRhDR8l2H4R
         rz1ow77NNYrRwH3dTm3BFmbC8Dzip1k6t9R4PRe4L+2Hx1vL5RZ+wcfJoTfcAEl9NaHY
         28Fg==
X-Forwarded-Encrypted: i=1; AJvYcCUcgdBXcHztM+7pqEreJpMmnya2mGG2R4QGxZtbG/U1OWrg51OsGcm/OCizO6GoplwdqZnw6fD26Ar/2Gw+3C55asbVFfSJzrGWenuMtT8=
X-Gm-Message-State: AOJu0YxxnZHJqkcFQZCun2G/yuf5Jmh3YFQI0D+KYCfD/WPd9oY1qRxh
	cALe3SBrqybgqcjoOrlD6Q5GxtbI3IRXewp13vPqLe6nLb/3xwk8ektq2CS+dT0=
X-Google-Smtp-Source: AGHT+IEuBhjg/MG9JEuabnR/N1PzCOaBGU5xNemthbK5N8Gkevd6Z9RRqTn5FvpAjVmlMfFBN+WCnQ==
X-Received: by 2002:a17:906:4a14:b0:a59:bf27:5f2e with SMTP id a640c23a62f3a-a623e8f7e02mr121836166b.20.1716452879245;
        Thu, 23 May 2024 01:27:59 -0700 (PDT)
Message-ID: <19c7a56c-5640-438d-aece-6a62f748e893@suse.com>
Date: Thu, 23 May 2024 10:27:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] tools: Drop libsystemd as a dependency
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-5-andrew.cooper3@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240516185804.3309725-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 16.05.24 20:58, Andrew Cooper wrote:
> There are no more users, and we want to disuade people from introducing new
> users just for sd_notify() and friends.  Drop the dependency.
> 
> We still want the overall --with{,out}-systemd to gate the generation of the
> service/unit/mount/etc files.
> 
> Rerun autogen.sh, and mark the dependency as removed in the build containers.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Christian Lindig <christian.lindig@citrix.com>
> CC: Edwin Török <edwin.torok@cloud.com>
> 
> v2:
>   * Only strip out the library check.
> ---
>   automation/build/archlinux/current.dockerfile |   1 +
>   .../build/suse/opensuse-leap.dockerfile       |   1 +
>   .../build/suse/opensuse-tumbleweed.dockerfile |   1 +
>   automation/build/ubuntu/focal.dockerfile      |   1 +
>   config/Tools.mk.in                            |   2 -
>   m4/systemd.m4                                 |   9 -
>   tools/configure                               | 256 ------------------
>   7 files changed, 4 insertions(+), 267 deletions(-)
> 
> diff --git a/automation/build/archlinux/current.dockerfile b/automation/build/archlinux/current.dockerfile
> index 3e37ab5c40c1..d29f1358c2bd 100644
> --- a/automation/build/archlinux/current.dockerfile
> +++ b/automation/build/archlinux/current.dockerfile
> @@ -37,6 +37,7 @@ RUN pacman -S --refresh --sysupgrade --noconfirm --noprogressbar --needed \
>           sdl2 \
>           spice \
>           spice-protocol \
> +        # systemd for Xen < 4.19

Does this work as intended? A comment between the parameters and no "\" at the
end of the line?


Juergen


From xen-devel-bounces@lists.xenproject.org Thu May 23 08:37:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:37:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728243.1133104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3wU-0006eB-SZ; Thu, 23 May 2024 08:36:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728243.1133104; Thu, 23 May 2024 08:36:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA3wU-0006e4-Pv; Thu, 23 May 2024 08:36:54 +0000
Received: by outflank-mailman (input) for mailman id 728243;
 Thu, 23 May 2024 08:36:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA3wT-0006dr-Q1
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:36:53 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9a6630b1-18df-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 10:36:52 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51ffff16400so11911462e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:36:52 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01631sm1904138966b.153.2024.05.23.01.36.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:36:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a6630b1-18df-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716453411; x=1717058211; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=7l/2d6fclAu6FofaUz5jRdfurxhewbvq7pz++xoGK1E=;
        b=ur+aE2ORX+zmydK4u1Za4NhW8AfcV5fNWFX52LUZ7eRpkfFNF0j0GVzg1o4+xU1mzX
         tnGdpmkdvZz1pQtYp+loqWNnkhKPmBxzM+Zq1U8Bd0MmkgC+TAsDXeu3Jn3rJAfmSvRg
         85OhQkl4UAyr8tUEfeO7efjQ6lQP4M1bk7qzY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716453411; x=1717058211;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7l/2d6fclAu6FofaUz5jRdfurxhewbvq7pz++xoGK1E=;
        b=fYSu2ALbMUWj6PD+iYR1sTSuIVpLqBFuAQNoMzbib+zqFcP3sb85FFJ89gtzw0jBi8
         l1sNZRTwMpzfSlWxcna7MmEj29gTmSBJuErO0bPO7W14l1ZiECYbFVcyFdoA3DsO94E7
         EcSFF4O7otkYA3ker66Q0Tb2FSzHSgdD08g24GSgVFSbjCxwgnIA08+vEmWCnjwOkE8D
         C2xppXsc5VU2UL0fFC8KukcO8EqYDZj6x5mVfbJyuj15cULyiDLotybGZ8McsQVw8j6B
         xtMnzsMEWmEjr9/iug/BWAnsHXaPB0FcfGo35xx7d1sdxkvnhB6TnlTnGzwd5u5swsXL
         3UJw==
X-Forwarded-Encrypted: i=1; AJvYcCXPToQvsE4yrNMfZk3OtXcp97MOLM1sqLHMAVM3Jrg/FX64oK3/UkE9fKUjoVviP5Wj8WzWtDS1Spr0y0QXY5E24d828AgZOtbKqTSvUSE=
X-Gm-Message-State: AOJu0Yx+3gdgdNwazWEcoT5n53oszLp5wCpKj99pp8y0fNcBO7wRVHKn
	G6ItWqVZZuYpGvKeDBVZew0jLLTdMCieJr/kqWUB7QyEhOAgTUUaCYh4cWmZ6s4=
X-Google-Smtp-Source: AGHT+IGAJIV4q9G5CAkOexovAiYUVnf8I01Qc7hEc+So2iEDHqRNAIVrJ3/M8MzMtpIqqov5YOMfmg==
X-Received: by 2002:a05:6512:48d7:b0:51f:2a80:a982 with SMTP id 2adb3069b0e04-526c0b5dc01mr3551603e87.47.1716453411455;
        Thu, 23 May 2024 01:36:51 -0700 (PDT)
Message-ID: <0708b3dc-442d-4ae2-bdf2-a27b48a8c5c2@citrix.com>
Date: Thu, 23 May 2024 09:36:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] tools: Drop libsystemd as a dependency
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-5-andrew.cooper3@citrix.com>
 <19c7a56c-5640-438d-aece-6a62f748e893@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <19c7a56c-5640-438d-aece-6a62f748e893@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 9:27 am, Jürgen Groß wrote:
> On 16.05.24 20:58, Andrew Cooper wrote:
>> diff --git a/automation/build/archlinux/current.dockerfile
>> b/automation/build/archlinux/current.dockerfile
>> index 3e37ab5c40c1..d29f1358c2bd 100644
>> --- a/automation/build/archlinux/current.dockerfile
>> +++ b/automation/build/archlinux/current.dockerfile
>> @@ -37,6 +37,7 @@ RUN pacman -S --refresh --sysupgrade --noconfirm
>> --noprogressbar --needed \
>>           sdl2 \
>>           spice \
>>           spice-protocol \
>> +        # systemd for Xen < 4.19
>
> Does this work as intended? A comment between the parameters and no
> "\" at the
> end of the line?

Sadly, yes.

Comments are stripped out on a line-granuar basis, prior to Docker
interpreting the remainder.

This is the approved way to do comments in dockerfiles, and we already
have other examples of this in our dockerfiles.

See e.g. a0e29b316363d9 for what I'll be doing with these comments in
~3y time.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 23 08:44:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 08:44:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728249.1133114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA43t-0000sj-KJ; Thu, 23 May 2024 08:44:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728249.1133114; Thu, 23 May 2024 08:44:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA43t-0000sc-HX; Thu, 23 May 2024 08:44:33 +0000
Received: by outflank-mailman (input) for mailman id 728249;
 Thu, 23 May 2024 08:44:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA43s-0000sW-Hi
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 08:44:32 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ab948f5b-18e0-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 10:44:30 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso3832284a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 01:44:30 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733becfcc1sm19031506a12.45.2024.05.23.01.44.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 01:44:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab948f5b-18e0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716453870; x=1717058670; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DKuv3MYckGRWLgNn8D6Uv/o4O6IE46yX2fIp5S7vyAg=;
        b=ZhFm1s9J72KcjBCnPM3iQLHbm7GP8YLKHH51rtsI3WHO11Ecj/y4lK/+nAe9tm0cbs
         MUR6ce1y1gKjw0f3ZxX8+NzPxJEeDVXm84hrjwAw1Fl/uxZxfhAAznTrAHSSg5FktGPc
         3IDKAlTtETzv2UXnOTo/2/wxS8D2P8HlEwKP8YtR/cLVIVaIwuROO4xyhAr93R5sKyyO
         3E2wIOz/FReMHXM5lMvWPA6GMZT6KEkOXJRSw2lalvfgUOuvOok5qSt9YBQwSCF8Ji7G
         liKJ96gmX67KswcOcnYIlrMM+RhmX4W+OJJyNy26mhpaZBOIcW8bvLMBnYTrUjXwQp+M
         X+8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716453870; x=1717058670;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DKuv3MYckGRWLgNn8D6Uv/o4O6IE46yX2fIp5S7vyAg=;
        b=KBuyCHJfsR6fSzY4/dAoM+BgZhJ8qjB9NG6SpP+Ycne1e5H//SrY2AKVaeG6zMDQeH
         Iw/p87pxpIIcr6BvS3WJT2Rx8kMyFbpb23/1mM31GOE3KaI+/UbTF/nwgKtTm0HHUeAO
         S6QV/OItnzyR7HwciWm4b4+8RyXznixtCZ5mOImuXFfHFhpKEJVP8WTWFIXOOLgMqcgB
         48QljOyphSfugx2AS6fIvxxxCYp5mzP87McuBlJ3PaaRPDYRcKsd/cs1/UlyBoXdxL3d
         M5+2jfMxHadMCQ/26UYg3wef8XcmxWe19Qo+JZuchXZ1G7dvHQzI8EWdOh6xGyGG3gjx
         yAWA==
X-Forwarded-Encrypted: i=1; AJvYcCX9qScqErmhiH/WvTMQfoqJUioNyoG3ftIzXb0x15q6FJt/GpQfehfXkHAzkwEGNzKcSnKF/CYOhY8YMMG0l0GrsjMtNPk3dUIJQZwDDlE=
X-Gm-Message-State: AOJu0YyCadXopwtNtniGIocxMVu+jb5ryIsc6ZB6QenDFc7Mrtj4tvkT
	n1+g3zFsLDAGIVrLRoKCZMgvn3SPROYDYGK5QzYO+HawE0ja2uGxzJ251zNBN5U=
X-Google-Smtp-Source: AGHT+IGgkhcVIznYB8BZOUkgANZXwhHcDuhG4Uaff+ftDVniaQQSakKYWn76aH5g6iDM/QMd5HQQ3g==
X-Received: by 2002:a50:c2d2:0:b0:572:a123:5ed0 with SMTP id 4fb4d7f45d1cf-57832a8913emr3160490a12.21.1716453869778;
        Thu, 23 May 2024 01:44:29 -0700 (PDT)
Message-ID: <0dabccae-6570-4099-88a7-b599b5a7ed38@suse.com>
Date: Thu, 23 May 2024 10:44:28 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 4/4] tools: Drop libsystemd as a dependency
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 =?UTF-8?B?RWR3aW4gVMO2csO2aw==?= <edwin.torok@cloud.com>
References: <20240516185804.3309725-1-andrew.cooper3@citrix.com>
 <20240516185804.3309725-5-andrew.cooper3@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240516185804.3309725-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 16.05.24 20:58, Andrew Cooper wrote:
> There are no more users, and we want to disuade people from introducing new
> users just for sd_notify() and friends.  Drop the dependency.
> 
> We still want the overall --with{,out}-systemd to gate the generation of the
> service/unit/mount/etc files.
> 
> Rerun autogen.sh, and mark the dependency as removed in the build containers.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen



From xen-devel-bounces@lists.xenproject.org Thu May 23 09:20:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:20:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728262.1133125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4cA-0006uS-1G; Thu, 23 May 2024 09:19:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728262.1133125; Thu, 23 May 2024 09:19:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4c9-0006uL-TQ; Thu, 23 May 2024 09:19:57 +0000
Received: by outflank-mailman (input) for mailman id 728262;
 Thu, 23 May 2024 09:19:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6NZt=M2=bounce.vates.tech=bounce-md_30504962.664f0a39.v1-2fb4daa5dcaf486a8f71b549e7ac6e39@srs-se1.protection.inumbo.net>)
 id 1sA4c8-0006uF-N1
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 09:19:56 +0000
Received: from mail187-24.suw11.mandrillapp.com
 (mail187-24.suw11.mandrillapp.com [198.2.187.24])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9dc144ee-18e5-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 11:19:55 +0200 (CEST)
Received: from pmta09.mandrill.prod.suw01.rsglab.com (localhost [127.0.0.1])
 by mail187-24.suw11.mandrillapp.com (Mailchimp) with ESMTP id
 4VlN1555QWzJKF3q8
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:19:53 +0000 (GMT)
Received: from [37.26.189.201] by mandrillapp.com id
 2fb4daa5dcaf486a8f71b549e7ac6e39; Thu, 23 May 2024 09:19:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9dc144ee-18e5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com;
	s=mte1; t=1716455993; x=1716716493;
	bh=rb2g7/BafJk9f48N+Ig/DxIGBOHotYs1+I4pEEvyMaA=;
	h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version:
	 Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From;
	b=t3b3P7X85yze9/uayCXuewejxeoJ1jLwI2WHHbPHeik0tzKvgO4NrqGPLIYX/6fU8
	 kpz4lkN1F9zWxTYbayycAswMrSO12tqg9ynFxGXdQBxhpDhhJVw56w3qTdeOlhyYzN
	 Vp5k4mfmNIt5hJPNPs3AXKcw+Xl+UKqt2M49f5UYDe+ckHWUFBKOC+Q5n0MOCa9qV6
	 vWwdHvwG9X1bArNKuRJt9LWqlt7ulmmQkB5DNIWOreAKUfXVdMi9KDRPThSRj0CegG
	 CdX8WqGxPumzUZ61yzG6lzvYxrr7RRTI5BLnp6agNfTr1LZ2Wf0CupvauPczkmrNoj
	 F8lrPr9XV/cgg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1;
	t=1716455993; x=1716716493; i=teddy.astie@vates.tech;
	bh=rb2g7/BafJk9f48N+Ig/DxIGBOHotYs1+I4pEEvyMaA=;
	h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version:
	 Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From;
	b=aRCcRUksJ6hk/3evHqjybmx9cuR1lQTNwoN2qExfb9elL0WMgSWDdtsYRZNy7WfIu
	 4TTaExTdOFSchWKa3fZW25riCFRMb7sV1X3moYWNbx6JgKOJXFkOfr0JTKfRFaN0f8
	 xOyHtiXyashi+nK4UEW0h7ITEfQ+Zkm9zJUQA0IGqCxcViAI5koyPTbzVg304fxRIw
	 8puxTH0Zu9OaeNfbF3GOu1XsfPJnFuZvy5bUWh0zqsUUafCWhAwjSRJKeewxkJVlFu
	 UuzN6WvKh8ycB/kYsM3o9JDx7ui3XFOkyJ1Fjl1GQfCb0ESyGVpwHVxCdglnr3YER6
	 yjLxUVTdSwU/A==
From: Teddy Astie <teddy.astie@vates.tech>
Subject: =?utf-8?Q?[XEN=20PATCH]=20x86/iommu:=20Conditionally=20compile=20platform-specific=20union=20entries?=
X-Mailer: git-send-email 2.45.1
X-Bm-Disclaimer: Yes
X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2
X-Bm-Transport-Timestamp: 1716455992462
To: xen-devel@lists.xenproject.org
Cc: Teddy Astie <teddy.astie@vates.tech>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, =?utf-8?Q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Message-Id: <2744240f42dadcb1edc69d0f921d5909fe0f63a2.1716455673.git.teddy.astie@vates.tech>
X-Native-Encoded: 1
X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.2fb4daa5dcaf486a8f71b549e7ac6e39?=
X-Mandrill-User: md_30504962
Feedback-ID: 30504962:30504962.20240523:md
Date: Thu, 23 May 2024 09:19:53 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

If some platform driver isn't compiled in, remove its related union
entries as they are not used.

Signed-off-by Teddy Astie <teddy.astie@vates.tech>
---
 xen/arch/x86/include/asm/iommu.h | 4 ++++
 xen/arch/x86/include/asm/pci.h   | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/xen/arch/x86/include/asm/iommu.h b/xen/arch/x86/include/asm/iommu.h
index 8dc464fbd3..99180940c4 100644
--- a/xen/arch/x86/include/asm/iommu.h
+++ b/xen/arch/x86/include/asm/iommu.h
@@ -42,17 +42,21 @@ struct arch_iommu
     struct list_head identity_maps;
 
     union {
+        #ifdef CONFIG_INTEL_IOMMU
         /* Intel VT-d */
         struct {
             uint64_t pgd_maddr; /* io page directory machine address */
             unsigned int agaw; /* adjusted guest address width, 0 is level 2 30-bit */
             unsigned long *iommu_bitmap; /* bitmap of iommu(s) that the domain uses */
         } vtd;
+        #endif
+        #ifdef CONFIG_AMD_IOMMU
         /* AMD IOMMU */
         struct {
             unsigned int paging_mode;
             struct page_info *root_table;
         } amd;
+        #endif
     };
 };
 
diff --git a/xen/arch/x86/include/asm/pci.h b/xen/arch/x86/include/asm/pci.h
index fd5480d67d..842710f0dc 100644
--- a/xen/arch/x86/include/asm/pci.h
+++ b/xen/arch/x86/include/asm/pci.h
@@ -22,12 +22,16 @@ struct arch_pci_dev {
      */
     union {
         /* Subset of struct arch_iommu's fields, to be used in dom_io. */
+        #ifdef CONFIG_INTEL_IOMMU
         struct {
             uint64_t pgd_maddr;
         } vtd;
+        #endif
+        #ifdef CONFIG_AMD_IOMMU
         struct {
             struct page_info *root_table;
         } amd;
+        #endif
     };
     domid_t pseudo_domid;
     mfn_t leaf_mfn;
-- 
2.45.1



Teddy Astie | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech


From xen-devel-bounces@lists.xenproject.org Thu May 23 09:35:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:35:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728270.1133133 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4qy-0001rX-6g; Thu, 23 May 2024 09:35:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728270.1133133; Thu, 23 May 2024 09:35:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4qy-0001rQ-49; Thu, 23 May 2024 09:35:16 +0000
Received: by outflank-mailman (input) for mailman id 728270;
 Thu, 23 May 2024 09:35:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA4qx-0001rG-7n; Thu, 23 May 2024 09:35:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA4qx-0002VG-40; Thu, 23 May 2024 09:35:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA4qw-0003Ee-Li; Thu, 23 May 2024 09:35:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA4qw-0006zv-L4; Thu, 23 May 2024 09:35:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=J5y+U5Tc2u+nldb2psEhde9gdrIn4+vf+pqqzmdw62c=; b=ssoe/Hnk90+y9CWD03tCa/QrXL
	/pbs5y0kNYWXsgcGr8LmPUDX0lcnY6WZkOPm32W4qhwvaW549xxLysEfrpKOaMgVJkOX/O3Wsxy1n
	/mTJDi4uycqIpW5hBilTfkp+o7IQx0VhBbw55qUXCEHiW31VXqPWFK9PCeVdJTSQ5B10=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186078-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186078: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-qcow2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 09:35:14 +0000

flight 186078 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186078/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-qcow2     8 xen-boot         fail in 186066 pass in 186078
 test-armhf-armhf-xl-multivcpu  8 xen-boot                  fail pass in 186066

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 186066 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 186066 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186066
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186066
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186066
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186066
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186066
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186066
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186078  2024-05-22 12:53:24 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu May 23 09:41:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:41:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728282.1133164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xI-0004JZ-Ie; Thu, 23 May 2024 09:41:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728282.1133164; Thu, 23 May 2024 09:41:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xI-0004JQ-Eq; Thu, 23 May 2024 09:41:48 +0000
Received: by outflank-mailman (input) for mailman id 728282;
 Thu, 23 May 2024 09:41:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t6d0=M2=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sA4xH-00043S-3I
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 09:41:47 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9d7a555-18e8-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 11:41:43 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-34d7d04808bso1550423f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 02:41:43 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net (0545937c.skybroadband.com.
 [5.69.147.124]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354c7df311esm9533383f8f.3.2024.05.23.02.41.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 02:41:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9d7a555-18e8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716457303; x=1717062103; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=d4DEPVfsDZMMBgcybIhN0dPitEHEoXs+kEgdv7X07o8=;
        b=PLmfQk065Nm7BW7ZPPoD/qejUW34303F1zR1jCnXWxZwVdIIk5MGi0wZzkJejUb027
         AyAmjZV/wYQ1Q5b8unyNv83wzSdtqsAUCewS5B5XBQtkWXRXL3Llnll99WToZkRD56uG
         AkuC+9OlcmiLR7U4M5xrGNiccNoz1Ysfu0vgM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716457303; x=1717062103;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=d4DEPVfsDZMMBgcybIhN0dPitEHEoXs+kEgdv7X07o8=;
        b=VrXS6wejdV7DmGeL0BNu7FQxe5lYsYH8MUzPeib8/F0yM/M+STnoKSVPRbtlIRjazE
         xf7k4EkBSs/SpZIG0nfXZVBPw3xt2N8tAcLh7lFdS47yMC2uaAPPRA+zm96ExtoSxCoJ
         1HiT8kiPEegvgbdfeItKt/OAgBk4udy4wTDJdzZK/r8K6qMlDPVbNClGO1Oiw8WvcdIx
         LInutPIumtIx7nRiNRk4nOPAafMdVbMYFBe4pv7giZB+vfKEbjQZKYJH6RF2586jGu60
         8+VPBD1O3YYdHs53mWB18yUC5GReg1pvfUzTtfL81YUee8KHwZBROOFp5rSl9rgWbxSp
         8eqQ==
X-Gm-Message-State: AOJu0YxciTjkklnlsm0N9wXrKpHnQ9Hr9vPwVfa1oXp/v6PgpXDnBhar
	VrnkCDiIcHDCsHGWSptobf7I7hjqIT54ZEiUy/2pm8T+Q9FrbmAOngd+RZTmb+uXlF9Fq6quM3R
	M
X-Google-Smtp-Source: AGHT+IEJytYQW8m3Pu7Ukg+JaaaH8S60s+n5e0NtgoSsx2Rit7ASpqlL97DlKIg4YVIt5cqTMF5k7Q==
X-Received: by 2002:adf:9bdc:0:b0:354:e729:c100 with SMTP id ffacd0b85a97d-354e729c251mr3530141f8f.3.1716457302632;
        Thu, 23 May 2024 02:41:42 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 1/2] tools/xg: Streamline cpu policy serialise/deserialise calls
Date: Thu, 23 May 2024 10:41:29 +0100
Message-Id: <5c6ee74b60bad4eb9cc8e17dbfcf7158d38bd32c.1716457040.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716457040.git.alejandro.vallejo@cloud.com>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The idea is to use xc_cpu_policy_t as a single object containing both the
serialised and deserialised forms of the policy. Note that we need lengths
for the arrays, as the serialised policies may be shorter than the array
capacities.

* Add the serialised lengths to the struct so we can distinguish
  between length and capacity of the serialisation buffers.
* Remove explicit buffer+lengths in serialise/deserialise calls
  and use the internal buffer inside xc_cpu_policy_t instead.
* Refactor everything to use the new serialisation functions.
* Remove redundant serialization calls and avoid allocating dynamic
  memory aside from the policy objects in xen-cpuid. Also minor cleanup
  in the policy print call sites.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Better context scoping in xg_sr_common_x86.
    * Can't be const because write_record() takes non-const.
  * Adjusted line length of xen-cpuid's print_policy.
  * Adjusted error messages in xen-cpuid's print_policy.
  * Reverted removal of overscoped loop indices.
---
 tools/include/xenguest.h            |  8 ++-
 tools/libs/guest/xg_cpuid_x86.c     | 98 ++++++++++++++++++++---------
 tools/libs/guest/xg_private.h       |  2 +
 tools/libs/guest/xg_sr_common_x86.c | 56 ++++++-----------
 tools/misc/xen-cpuid.c              | 41 ++++--------
 5 files changed, 106 insertions(+), 99 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index e01f494b772a..563811cd8dde 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -799,14 +799,16 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy);
 
 /* Manipulate a policy via architectural representations. */
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
 int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                                const xen_cpuid_leaf_t *leaves,
                                uint32_t nr);
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
                               const xen_msr_entry_t *msrs, uint32_t nr);
+int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
+int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs, uint32_t *nr);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..4f4b86b59470 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
-                              unsigned int nr_leaves, unsigned int nr_entries)
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
     int rc;
 
     rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    nr_leaves, &err_leaf, &err_subleaf);
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
     if ( rc )
     {
         if ( err_leaf != -1 )
@@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
     }
 
     rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  nr_entries, &err_msr);
+                                  policy->nr_msrs, &err_msr);
     if ( rc )
     {
         if ( err_msr != -1 )
@@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
-    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
     int rc;
 
-    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
-                                 policy->msrs, &nr_msrs);
+    rc = xc_cpu_policy_serialise(xch, policy);
     if ( rc )
         return rc;
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
-                                  nr_msrs, policy->msrs,
+    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
+                                  policy->nr_msrs, policy->msrs,
                                   &err_leaf, &err_subleaf, &err_msr);
     if ( rc )
     {
@@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
     return rc;
 }
 
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
 {
+    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
+    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
     int rc;
 
-    if ( leaves )
+    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
+    if ( rc )
     {
-        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
-        if ( rc )
-        {
-            ERROR("Failed to serialize CPUID policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize CPUID policy");
+        errno = -rc;
+        return -1;
     }
 
-    if ( msrs )
+    p->nr_leaves = nr_leaves;
+
+    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &nr_msrs);
+    if ( rc )
     {
-        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
-        if ( rc )
-        {
-            ERROR("Failed to serialize MSR policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize MSR policy");
+        errno = -rc;
+        return -1;
     }
 
+    p->nr_msrs = nr_msrs;
+
     errno = 0;
     return 0;
 }
@@ -1012,6 +1012,42 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
     return rc;
 }
 
+int xc_cpu_policy_get_leaves(xc_interface *xch,
+                             const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves,
+                             uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch CPUID leaves from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *leaves = policy->leaves;
+    *nr = policy->nr_leaves;
+
+    return 0;
+}
+
+int xc_cpu_policy_get_msrs(xc_interface *xch,
+                           const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs,
+                           uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch MSRs from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *msrs = policy->msrs;
+    *nr = policy->nr_msrs;
+
+    return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest)
 {
diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
index d73947094f2e..a65dae818f3d 100644
--- a/tools/libs/guest/xg_private.h
+++ b/tools/libs/guest/xg_private.h
@@ -177,6 +177,8 @@ struct xc_cpu_policy {
     struct cpu_policy policy;
     xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
     xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
+    uint32_t nr_leaves;
+    uint32_t nr_msrs;
 };
 #endif /* x86 */
 
diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
index 563b4f016877..a0d67c3211c6 100644
--- a/tools/libs/guest/xg_sr_common_x86.c
+++ b/tools/libs/guest/xg_sr_common_x86.c
@@ -1,4 +1,5 @@
 #include "xg_sr_common_x86.h"
+#include "xg_sr_stream_format.h"
 
 int write_x86_tsc_info(struct xc_sr_context *ctx)
 {
@@ -45,54 +46,39 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
-    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
-    uint32_t nr_leaves = 0, nr_msrs = 0;
-    xc_cpu_policy_t *policy = NULL;
+    xc_cpu_policy_t *policy = xc_cpu_policy_init();
     int rc;
 
-    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
-    {
-        PERROR("Unable to get CPU Policy size");
-        return -1;
-    }
-
-    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
-    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
-    policy = xc_cpu_policy_init();
-    if ( !cpuid.data || !msrs.data || !policy )
-    {
-        ERROR("Cannot allocate memory for CPU Policy");
-        rc = -1;
-        goto out;
-    }
-
-    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
+    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
     {
         PERROR("Unable to get d%d CPU Policy", ctx->domid);
         rc = -1;
         goto out;
     }
-    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
-                                 msrs.data, &nr_msrs) )
-    {
-        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
-        rc = -1;
-        goto out;
-    }
 
-    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
-    if ( cpuid.length )
+
+    if ( policy->nr_leaves )
     {
-        rc = write_record(ctx, &cpuid);
+        struct xc_sr_record record = {
+            .type = REC_TYPE_X86_CPUID_POLICY,
+            .data = policy->leaves,
+            .length = policy->nr_leaves * sizeof(*policy->leaves),
+        };
+
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
 
-    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
-    if ( msrs.length )
+    if ( policy->nr_msrs )
     {
-        rc = write_record(ctx, &msrs);
+        struct xc_sr_record record = {
+            .type = REC_TYPE_X86_MSR_POLICY,
+            .data = policy->msrs,
+            .length = policy->nr_msrs * sizeof(*policy->msrs),
+        };
+
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
@@ -100,8 +86,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     rc = 0;
 
  out:
-    free(cpuid.data);
-    free(msrs.data);
     xc_cpu_policy_destroy(policy);
 
     return rc;
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 4c4593528dfe..488f43378406 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -156,12 +156,18 @@ static void dump_info(xc_interface *xch, bool detail)
 
     free(fs);
 }
-
-static void print_policy(const char *name,
-                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
-                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
+static void print_policy(xc_interface *xch, const char *name,
+                         const xc_cpu_policy_t *policy)
 {
     unsigned int l;
+    const xen_cpuid_leaf_t *leaves;
+    const xen_msr_entry_t *msrs;
+    uint32_t nr_leaves, nr_msrs;
+
+    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) )
+        err(1, "xc_cpu_policy_get_leaves()");
+    if ( xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
+        err(1, "xc_cpu_policy_get_msrs()");
 
     printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
     printf(" CPUID:\n");
@@ -287,8 +293,6 @@ int main(int argc, char **argv)
             [ XEN_SYSCTL_cpu_policy_pv_default ]   = "PV Default",
             [ XEN_SYSCTL_cpu_policy_hvm_default ]  = "HVM Default",
         };
-        xen_cpuid_leaf_t *leaves;
-        xen_msr_entry_t *msrs;
         uint32_t i, max_leaves, max_msrs;
 
         xc_interface *xch = xc_interface_open(0, 0, 0);
@@ -305,36 +309,21 @@ int main(int argc, char **argv)
             printf("Xen reports there are maximum %u leaves and %u MSRs\n",
                    max_leaves, max_msrs);
 
-        leaves = calloc(max_leaves, sizeof(xen_cpuid_leaf_t));
-        if ( !leaves )
-            err(1, "calloc(max_leaves)");
-        msrs = calloc(max_msrs, sizeof(xen_msr_entry_t));
-        if ( !msrs )
-            err(1, "calloc(max_msrs)");
-
         if ( domid != -1 )
         {
             char name[20];
-            uint32_t nr_leaves = max_leaves;
-            uint32_t nr_msrs = max_msrs;
 
             if ( xc_cpu_policy_get_domain(xch, domid, policy) )
                 err(1, "xc_cpu_policy_get_domain(, %d, )", domid);
-            if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                         msrs, &nr_msrs) )
-                err(1, "xc_cpu_policy_serialise");
 
             snprintf(name, sizeof(name), "Domain %d", domid);
-            print_policy(name, leaves, nr_leaves, msrs, nr_msrs);
+            print_policy(xch, name, policy);
         }
         else
         {
             /* Get system policies */
             for ( i = 0; i < ARRAY_SIZE(sys_policies); ++i )
             {
-                uint32_t nr_leaves = max_leaves;
-                uint32_t nr_msrs = max_msrs;
-
                 if ( xc_cpu_policy_get_system(xch, i, policy) )
                 {
                     if ( errno == EOPNOTSUPP )
@@ -346,18 +335,12 @@ int main(int argc, char **argv)
 
                     err(1, "xc_cpu_policy_get_system(, %s, )", sys_policies[i]);
                 }
-                if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                             msrs, &nr_msrs) )
-                    err(1, "xc_cpu_policy_serialise");
 
-                print_policy(sys_policies[i], leaves, nr_leaves,
-                             msrs, nr_msrs);
+                print_policy(xch, sys_policies[i], policy);
             }
         }
 
         xc_cpu_policy_destroy(policy);
-        free(leaves);
-        free(msrs);
         xc_interface_close(xch);
     }
     else if ( mode == MODE_INFO || mode == MODE_DETAIL )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 09:41:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:41:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728280.1133143 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xD-0003pE-VP; Thu, 23 May 2024 09:41:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728280.1133143; Thu, 23 May 2024 09:41:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xD-0003p7-Sk; Thu, 23 May 2024 09:41:43 +0000
Received: by outflank-mailman (input) for mailman id 728280;
 Thu, 23 May 2024 09:41:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t6d0=M2=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sA4xD-0003m7-6G
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 09:41:43 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a967d2d5-18e8-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 11:41:42 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-354faf5f1b4so359595f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 02:41:42 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net (0545937c.skybroadband.com.
 [5.69.147.124]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354c7df311esm9533383f8f.3.2024.05.23.02.41.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 02:41:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a967d2d5-18e8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716457302; x=1717062102; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=5MpvoXLxrGjWD7P+YQJyPs2tsu37hmHqu/VshlrxEQE=;
        b=bn58ZUMTSS7bOUmaVgn26bU6QlUJGnZ2plx6tYKAvv7GTZCldzvJtGYvtHfCILsqad
         RuDHDRewvsCa2cVJ53eLr7YUGtvXJr+iDn29ZtOwpDHlj0etsbLLjBGd+azIm3rPDJfI
         vLQDIutdMvvNC0gyco5yBYU9GLvpZu0nRxcaY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716457302; x=1717062102;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=5MpvoXLxrGjWD7P+YQJyPs2tsu37hmHqu/VshlrxEQE=;
        b=dyCWLY8HOWRe94JjexgIATEvLYUhvgHH/WQdbMLc/RkrYEm8PwWGZNhLGBaTgikDcS
         qQCNQfuDIWkziAd0OHY81uYCgip+qCnk02iG4GfbdWOg2oszsTM1iy2MjiC9FaxTqYGE
         2iyEFLg+cpVLEd2br4wK2DxdAH9sVEblU2l0344cBFfW3izeNnGgSCNx5ZR04rBLtipY
         De1XDbQ8cGiolrfF+tZ33pelnpbA9Lk53xLljqekBwu0phxqyFG/Nq/TUvhmnNtpVr8J
         l+SelTzVJBGg5AJsCWAG7K2EQ75aeRn4Ha9m/P0yWs0SD8k9R+dT6wfH5LfPBZGlAhLI
         14mg==
X-Gm-Message-State: AOJu0Yx6awJNQ3sD1TtuXze5S+4XgqDyi7nnVTUibFlNEREFNue8FeSi
	h9nYH7h5HVQXPZDwfOOeTqvzM1/IW7GHSv7ChDcH89JaFY2LqS641TcPM3no4UvXKPgS3BMga06
	t
X-Google-Smtp-Source: AGHT+IHWPgQ2tR8dl7xB4iECBSEd9EzERVPTZ4iCl7yTHZ0YMnhfSXuJ5ZYgsZjypjCFOC10ULwLZg==
X-Received: by 2002:a5d:66c6:0:b0:354:f304:cd36 with SMTP id ffacd0b85a97d-354f304ce78mr2752759f8f.70.1716457301676;
        Thu, 23 May 2024 02:41:41 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 0/2] Clean the policy manipulation path in domain creation
Date: Thu, 23 May 2024 10:41:28 +0100
Message-Id: <cover.1716457040.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

v2 -> v3:
  * Style adjustments
  * Revert of loop index scope refactors

v1 -> v2:
  * Removed xc_cpu_policy from xenguest.h (dropped v1/patch1)
  * Added accessors for xc_cpu_policy so the serialised form can be extracted.
  * Modified xen-cpuid to use accessors.

==== Original cover letter ====

In the context of creating a domain, we currently issue a lot of hypercalls
redundantly while populating its CPU policy; likely a side effect of
organic growth more than anything else.

However, the worst part is not the overhead (this is a glacially cold
path), but the insane amounts of boilerplate that make it really hard to
pick apart what's going on. One major contributor to this situation is the
fact that what's effectively "setup" and "teardown" phases in policy
manipulation are not factored out from the functions that perform said
manipulations, leading to the same getters and setter being invoked many
times, when once each would do.

Another big contributor is the code being unaware of when a policy is
serialised and when it's not.

This patch attempts to alleviate this situation, yielding over 200 LoC
reduction.

Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
         from clients of libxc/libxg.
Patch 2: Changes the (de)serialization wrappers in xenguest so they always
         serialise to/from the internal buffers of xc_cpu_policy_t. The
         struct is suitably expanded to hold extra information required.
Patch 3: Performs the refactor of the policy manipulation code so that it
         follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.


Subject: [PATCH v3 0/2] *** SUBJECT HERE ***

*** BLURB HERE ***

Alejandro Vallejo (2):
  tools/xg: Streamline cpu policy serialise/deserialise calls
  tools/xg: Clean up xend-style overrides for CPU policies

 tools/include/xenguest.h            |   8 +-
 tools/libs/guest/xg_cpuid_x86.c     | 530 ++++++++++------------------
 tools/libs/guest/xg_private.h       |   2 +
 tools/libs/guest/xg_sr_common_x86.c |  56 ++-
 tools/misc/xen-cpuid.c              |  41 +--
 5 files changed, 234 insertions(+), 403 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 09:41:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:41:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728281.1133154 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xH-000446-6a; Thu, 23 May 2024 09:41:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728281.1133154; Thu, 23 May 2024 09:41:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA4xH-00043x-2n; Thu, 23 May 2024 09:41:47 +0000
Received: by outflank-mailman (input) for mailman id 728281;
 Thu, 23 May 2024 09:41:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t6d0=M2=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sA4xG-0003m7-2V
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 09:41:46 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aabd5082-18e8-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 11:41:45 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-354faf5f1b4so359656f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 02:41:45 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net (0545937c.skybroadband.com.
 [5.69.147.124]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-354c7df311esm9533383f8f.3.2024.05.23.02.41.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 02:41:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aabd5082-18e8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716457304; x=1717062104; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VW5aCUQWseMMyScAyihOHY3dKoHCa4WtptcETHjOh8s=;
        b=lVQYS5nBLJg5QkTrB+hhnNF8RGjVvsqbg7IW5BU5gJBaIrdYcgrNWV2rkcMHifY/Hh
         jYIRG14bXQYpZrkQZd8+IofKCPXIkfsy/Qlx6TZR0iQwEQoBbj+s/5ZZ2hwDW2tnv6sb
         9V5g0SaoTxlHwXt4DW4ImclTV9YX8zLUN6chU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716457304; x=1717062104;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VW5aCUQWseMMyScAyihOHY3dKoHCa4WtptcETHjOh8s=;
        b=YCHiL6YWg5U5kT2FweVlOMUQ2TCCoFrzCSsuQBnIpyRoOUrEvP/9J+rudbHr31LG4f
         Edy2Rys9UlEmI6eyZPwyer11c0Koxv+qor73uS1sbg8HHIe8UUyMo7ZdNBreW7YQCLKX
         WJvVR6tTjEvks7mzuHQqFKrJLe5WQ/bj4+DhxXcZsIZHPXr08ymgFmzKVG354ZcN2P5U
         Bpi6DBGLHTMpmV8h2iLINy3gYYtgIp0b8qxo5J+9y4BitkvaO0iasHFSBR7Sjp8GKEzy
         WkzcbCb7iTw/IOv3Vkjs05xKOXfXWgpp8XQI4Zo3l9XlEApX5OJwlOl+gJDSeXI1wi6h
         wd6w==
X-Gm-Message-State: AOJu0YwHOms45DxSX0qXmIn15RDLaN86GvGLIuv74XPAh8CGNDHECDkr
	NwJicLhhwFLQAIq5htM1V8FxCLbv9UDjplxF9GTI+T8XxvrvA3/TMCDHi3pzbXiJl/QmXuuK+L0
	2
X-Google-Smtp-Source: AGHT+IFVKRENdmnSPQd28lJ1btVIYr2DvXBn3OkGpSrHPpByysuNEOy/pNkGL6dWdhkcIj8WXoLemA==
X-Received: by 2002:a05:6000:d87:b0:354:fa57:a0c9 with SMTP id ffacd0b85a97d-354fa57a18emr1023764f8f.33.1716457303887;
        Thu, 23 May 2024 02:41:43 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3 2/2] tools/xg: Clean up xend-style overrides for CPU policies
Date: Thu, 23 May 2024 10:41:30 +0100
Message-Id: <ae180fd606cb9a6aa0108080fe8cf7a6496a4ce3.1716457040.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716457040.git.alejandro.vallejo@cloud.com>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Factor out policy getters/setters from both (CPUID and MSR) policy override
functions. Additionally, use host policy rather than featureset when
preparing the cur policy, saving one hypercall and several lines of
boilerplate.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Restored overscoped loop indices
  * Split long line in conditional
---
 tools/libs/guest/xg_cpuid_x86.c | 438 ++++++++++----------------------
 1 file changed, 131 insertions(+), 307 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4f4b86b59470..1e631fd46d2f 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -36,6 +36,34 @@ enum {
 #define bitmaskof(idx)      (1u << ((idx) & 31))
 #define featureword_of(idx) ((idx) >> 5)
 
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
+{
+    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    int rc;
+
+    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
+    if ( rc )
+    {
+        if ( err_leaf != -1 )
+            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
+                  err_leaf, err_subleaf, -rc, strerror(-rc));
+        return rc;
+    }
+
+    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
+                                  policy->nr_msrs, &err_msr);
+    if ( rc )
+    {
+        if ( err_msr != -1 )
+            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
+                  err_msr, -rc, strerror(-rc));
+        return rc;
+    }
+
+    return 0;
+}
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
 {
     struct xen_sysctl sysctl = {};
@@ -260,102 +288,37 @@ static int compare_leaves(const void *l, const void *r)
     return 0;
 }
 
-static xen_cpuid_leaf_t *find_leaf(
-    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-    const struct xc_xend_cpuid *xend)
+static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
+                                   const struct xc_xend_cpuid *xend)
 {
     const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
 
-    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
+    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),
+                   sizeof(*p->leaves), compare_leaves);
 }
 
-static int xc_cpuid_xend_policy(
-    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
+static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
+                                const struct xc_xend_cpuid *xend,
+                                xc_cpu_policy_t *host,
+                                xc_cpu_policy_t *def,
+                                xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto fail;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -ENOMEM;
-    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        goto fail;
-    }
-
-    /* Get the domain's current policy. */
-    nr_msrs = 0;
-    nr_cur = nr_leaves;
-    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto fail;
-    }
+    if ( !xend )
+        return 0;
 
-    /* Get the domain type's default policy. */
-    nr_msrs = 0;
-    nr_def = nr_leaves;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_def, def, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto fail;
-    }
+    if ( !host || !def || !cur )
+        return -EINVAL;
 
-    /* Get the host policy. */
-    nr_msrs = 0;
-    nr_host = nr_leaves;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_host, host, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -EINVAL;
     for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
     {
-        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
-        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
-        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
+        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
+        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
+        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
 
         if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
         {
             ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
-            goto fail;
+            return -EINVAL;
         }
 
         for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
@@ -384,7 +347,7 @@ static int xc_cpuid_xend_policy(
                 {
                     ERROR("Bad character '%c' in policy[%d] string '%s'",
                           xend->policy[i][j], i, xend->policy[i]);
-                    goto fail;
+                    return -EINVAL;
                 }
 
                 clear_bit(31 - j, cur_reg);
@@ -394,25 +357,7 @@ static int xc_cpuid_xend_policy(
         }
     }
 
-    /* Feed the transformed currrent policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Success! */
-
- fail:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 static int compare_msr(const void *l, const void *r)
@@ -426,104 +371,38 @@ static int compare_msr(const void *l, const void *r)
     return lhs->idx < rhs->idx ? -1 : 1;
 }
 
-static xen_msr_entry_t *find_msr(
-    xen_msr_entry_t *msrs, unsigned int nr_msrs,
-    uint32_t index)
+static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
+                                 uint32_t index)
 {
     const xen_msr_entry_t key = { .idx = index };
 
-    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
+    return bsearch(&key, p->msrs, ARRAY_SIZE(p->msrs),
+                   sizeof(*p->msrs), compare_msr);
 }
 
-
 static int xc_msr_policy(xc_interface *xch, domid_t domid,
-                         const struct xc_msr *msr)
+                         const struct xc_msr *msr,
+                         xc_cpu_policy_t *host,
+                         xc_cpu_policy_t *def,
+                         xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto out;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto out;
-    }
-
-    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        rc = -ENOMEM;
-        goto out;
-    }
-
-    /* Get the domain's current policy. */
-    nr_leaves = 0;
-    nr_cur = nr_msrs;
-    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_leaves = 0;
-    nr_def = nr_msrs;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, NULL, &nr_def, def);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto out;
-    }
+    if ( !msr )
+        return 0;
 
-    /* Get the host policy. */
-    nr_leaves = 0;
-    nr_host = nr_msrs;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_leaves, NULL, &nr_host, host);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto out;
-    }
+    if ( !host || !def || !cur )
+        return -EINVAL;
 
     for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
     {
-        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
-        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
-        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
         unsigned int i;
+        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
+        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
+        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
 
         if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
         {
             ERROR("Missing MSR %#x", msr->index);
-            rc = -ENOENT;
-            goto out;
+            return -ENOENT;
         }
 
         for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
@@ -542,8 +421,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
             {
                 ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
                       msr->index, msr->policy[i], msr->policy);
-                rc = -EINVAL;
-                goto out;
+                return -EINVAL;
             }
 
             if ( val )
@@ -553,25 +431,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
         }
     }
 
-    /* Feed the transformed policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Success! */
-
- out:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
@@ -583,14 +443,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     int rc;
     bool hvm;
     xc_domaininfo_t di;
-    struct xc_cpu_policy *p = xc_cpu_policy_init();
-    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-    uint32_t len = ARRAY_SIZE(host_featureset);
+    unsigned int i;
 
-    if ( !p )
-        return -ENOMEM;
+    struct xc_cpu_policy *host = xc_cpu_policy_init();
+    struct xc_cpu_policy *def = xc_cpu_policy_init();
+    struct xc_cpu_policy *cur = xc_cpu_policy_init();
+
+    if ( !host || !def || !cur )
+    {
+        rc = -ENOMEM;
+        goto out;
+    }
 
     if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
     {
@@ -600,21 +463,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-    /* Get the host policy. */
-    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-                               &len, host_featureset);
-    /* Tolerate "buffer too small", as we've got the bits we need. */
-    if ( rc && errno != ENOBUFS )
+    /* Get the raw host policy */
+    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+    if ( rc )
     {
-        PERROR("Failed to obtain host featureset");
+        PERROR("Failed to obtain host policy");
         rc = -errno;
         goto out;
     }
 
     /* Get the domain's default policy. */
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, p->leaves, &nr_msrs, NULL);
+    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                                  def);
     if ( rc )
     {
         PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
@@ -622,14 +483,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
-                                    &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-              err_leaf, err_subleaf, -rc, strerror(-rc));
-        goto out;
-    }
+    /* Copy the deserialised default policy to modify it */
+    memcpy(cur, def, sizeof(*cur));
 
     if ( restore )
     {
@@ -647,18 +502,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * - Re-enable features which have become (possibly) off by default.
          */
 
-        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+        cur->policy.basic.rdrand = host->policy.basic.rdrand;
+        cur->policy.feat.hle = host->policy.feat.hle;
+        cur->policy.feat.rtm = host->policy.feat.rtm;
 
         if ( hvm )
         {
-            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+            cur->policy.feat.mpx = host->policy.feat.mpx;
         }
 
-        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
-        p->policy.feat.max_subleaf = 0;
-        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
+        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
+        cur->policy.feat.max_subleaf = 0;
+        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
     }
 
     if ( featureset )
@@ -702,17 +557,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
             }
         }
 
-        x86_cpu_featureset_to_policy(feat, &p->policy);
+        x86_cpu_featureset_to_policy(feat, &cur->policy);
     }
     else
     {
-        p->policy.extd.itsc = itsc;
+        cur->policy.extd.itsc = itsc;
 
         if ( hvm )
         {
-            p->policy.basic.pae = pae;
-            p->policy.basic.vmx = nested_virt;
-            p->policy.extd.svm = nested_virt;
+            cur->policy.basic.pae = pae;
+            cur->policy.basic.vmx = nested_virt;
+            cur->policy.extd.svm = nested_virt;
         }
     }
 
@@ -722,8 +577,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * On hardware without CPUID Faulting, PV guests see real topology.
          * As a consequence, they also need to see the host htt/cmp fields.
          */
-        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
-        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+        cur->policy.basic.htt       = host->policy.basic.htt;
+        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
     }
     else
     {
@@ -731,28 +586,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * Topology for HVM guests is entirely controlled by Xen.  For now, we
          * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
          */
-        p->policy.basic.htt = true;
-        p->policy.extd.cmp_legacy = false;
+        cur->policy.basic.htt = true;
+        cur->policy.extd.cmp_legacy = false;
 
         /*
          * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
          * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
          * overflow.
          */
-        if ( !p->policy.basic.lppp )
-            p->policy.basic.lppp = 2;
-        else if ( !(p->policy.basic.lppp & 0x80) )
-            p->policy.basic.lppp *= 2;
+        if ( !cur->policy.basic.lppp )
+            cur->policy.basic.lppp = 2;
+        else if ( !(cur->policy.basic.lppp & 0x80) )
+            cur->policy.basic.lppp *= 2;
 
-        switch ( p->policy.x86_vendor )
+        switch ( cur->policy.x86_vendor )
         {
         case X86_VENDOR_INTEL:
-            for ( i = 0; (p->policy.cache.subleaf[i].type &&
-                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
+            for ( i = 0; (cur->policy.cache.subleaf[i].type &&
+                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )
             {
-                p->policy.cache.subleaf[i].cores_per_package =
-                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
-                p->policy.cache.subleaf[i].threads_per_cache = 0;
+                cur->policy.cache.subleaf[i].cores_per_package =
+                    (cur->policy.cache.subleaf[i].cores_per_package << 1) | 1;
+                cur->policy.cache.subleaf[i].threads_per_cache = 0;
             }
             break;
 
@@ -772,49 +627,46 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
              * apic_id_size values greater than 7.  Limit the value to
              * 7 for now.
              */
-            if ( p->policy.extd.nc < 0x7f )
+            if ( cur->policy.extd.nc < 0x7f )
             {
-                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
-                    p->policy.extd.apic_id_size++;
+                if ( cur->policy.extd.apic_id_size != 0 &&
+                     cur->policy.extd.apic_id_size < 0x7 )
+                    cur->policy.extd.apic_id_size++;
 
-                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
+                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
             }
             break;
         }
     }
 
-    nr_leaves = ARRAY_SIZE(p->leaves);
-    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
-    if ( rc )
+    if ( xend || msr )
     {
-        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-        goto out;
+        /* The overrides are over the serialised form of the policy */
+        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
+            goto out;
+
+        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
+            goto out;
+        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
+            goto out;
+
+        if ( (rc = deserialize_policy(xch, cur)) )
+            goto out;
     }
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, p->leaves, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
+    rc = xc_cpu_policy_set_domain(xch, domid, cur);
     if ( rc )
     {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
         rc = -errno;
         goto out;
     }
 
-    if ( xend && (rc = xc_cpuid_xend_policy(xch, domid, xend)) )
-        goto out;
-
-    if ( msr )
-    {
-        rc = xc_msr_policy(xch, domid, msr);
-        if ( rc )
-            goto out;
-    }
-
     rc = 0;
 
 out:
-    xc_cpu_policy_destroy(p);
+    xc_cpu_policy_destroy(def);
+    xc_cpu_policy_destroy(host);
+    xc_cpu_policy_destroy(cur);
 
     return rc;
 }
@@ -834,34 +686,6 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
-{
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    int rc;
-
-    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    policy->nr_leaves, &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        if ( err_leaf != -1 )
-            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-                  err_leaf, err_subleaf, -rc, strerror(-rc));
-        return rc;
-    }
-
-    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  policy->nr_msrs, &err_msr);
-    if ( rc )
-    {
-        if ( err_msr != -1 )
-            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
-                  err_msr, -rc, strerror(-rc));
-        return rc;
-    }
-
-    return 0;
-}
-
 int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t *policy)
 {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 09:53:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 09:53:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728303.1133175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA57y-00072t-IL; Thu, 23 May 2024 09:52:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728303.1133175; Thu, 23 May 2024 09:52:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA57y-00072m-Dv; Thu, 23 May 2024 09:52:50 +0000
Received: by outflank-mailman (input) for mailman id 728303;
 Thu, 23 May 2024 09:52:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA57y-00072g-3H
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 09:52:50 +0000
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com
 [2607:f8b0:4864:20::732])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 36255a95-18ea-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 11:52:48 +0200 (CEST)
Received: by mail-qk1-x732.google.com with SMTP id
 af79cd13be357-792b8d98a56so474487685a.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 02:52:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf27fa4asm1471326385a.43.2024.05.23.02.52.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 02:52:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36255a95-18ea-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716457967; x=1717062767; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=tyTeWN/+8z/4F9vK2R+GVpI8XwMfkIXVPIglqrEEYuw=;
        b=i/m3T01BvmJYLOng/xtzzVdiuHT7+zbX7RxRRILHc4WoCkDVivbDhgIwC+2ouuL2+2
         cKBmTXSfUarVn0YqqQNuKrNCZFJQGLkdIpOwJh2hyo6VFhq2TweXSm7UYNEQZXMTV8Y4
         blWq4Y9fnCx83ftORwuInng48iz0JI0HIrU94=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716457967; x=1717062767;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tyTeWN/+8z/4F9vK2R+GVpI8XwMfkIXVPIglqrEEYuw=;
        b=ghvpZULmmYsB9EPALjTkyJsgH9OfOjplTcgI1DTfSup+7r27NL9Jj2dZiftfcI/IUx
         +JspbrA3iGy9HvnjY+/N2KkowJl9Eewu2mb4gAqipodb+foJivh0zydaDUH478TadIoS
         DyElus/iHA5/+8fx2IS9/pnaMOgXv5YurtqXJwhqoL/AVaucsFRZAathaNLYvEwtp9MF
         F+WEXISCEMfa00wec5d7NLxL+NKsFYeCDmdzq2NwvjwG++PdoRf5KQpltvDuG6qNTxAs
         1JpunMPMwD2vhqm610+zyqPYJqi13NhavJuquZnNRzKDdZNYVHr+SReIWtnC0Yl5awD3
         dSbw==
X-Gm-Message-State: AOJu0YxxXKvrxqFnE8UMtjwJNE24iRFbCScFLg5CucpTPDA1vsnQjT4t
	JFVHcdTrolhETLE3iU743Udn6OtEBmQggyLtfVK44P007ehDU5p7cjnp0ZTccdWb7HpkJceZKkx
	w
X-Google-Smtp-Source: AGHT+IENn22mYam9JRDfdyj3gtDSvwEUDhOJpucb9n0pWEveQPtT/JUC5Xa2AoBqOuSNQM+Tpz39KQ==
X-Received: by 2002:a05:620a:2888:b0:792:949d:bb0 with SMTP id af79cd13be357-794994b29b4mr683161685a.47.1716457967424;
        Thu, 23 May 2024 02:52:47 -0700 (PDT)
Date: Thu, 23 May 2024 11:52:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Teddy Astie <teddy.astie@vates.tech>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH] x86/iommu: Conditionally compile platform-specific
 union entries
Message-ID: <Zk8R7YAhD9UHtVW6@macbook>
References: <2744240f42dadcb1edc69d0f921d5909fe0f63a2.1716455673.git.teddy.astie@vates.tech>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <2744240f42dadcb1edc69d0f921d5909fe0f63a2.1716455673.git.teddy.astie@vates.tech>

On Thu, May 23, 2024 at 09:19:53AM +0000, Teddy Astie wrote:
> If some platform driver isn't compiled in, remove its related union
> entries as they are not used.
> 
> Signed-off-by Teddy Astie <teddy.astie@vates.tech>
> ---
>  xen/arch/x86/include/asm/iommu.h | 4 ++++
>  xen/arch/x86/include/asm/pci.h   | 4 ++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/xen/arch/x86/include/asm/iommu.h b/xen/arch/x86/include/asm/iommu.h
> index 8dc464fbd3..99180940c4 100644
> --- a/xen/arch/x86/include/asm/iommu.h
> +++ b/xen/arch/x86/include/asm/iommu.h
> @@ -42,17 +42,21 @@ struct arch_iommu
>      struct list_head identity_maps;
>  
>      union {
> +        #ifdef CONFIG_INTEL_IOMMU
>          /* Intel VT-d */
>          struct {
>              uint64_t pgd_maddr; /* io page directory machine address */
>              unsigned int agaw; /* adjusted guest address width, 0 is level 2 30-bit */
>              unsigned long *iommu_bitmap; /* bitmap of iommu(s) that the domain uses */
>          } vtd;
> +        #endif
> +        #ifdef CONFIG_AMD_IOMMU
>          /* AMD IOMMU */
>          struct {
>              unsigned int paging_mode;
>              struct page_info *root_table;
>          } amd;
> +        #endif
>      };
>  };
>  
> diff --git a/xen/arch/x86/include/asm/pci.h b/xen/arch/x86/include/asm/pci.h
> index fd5480d67d..842710f0dc 100644
> --- a/xen/arch/x86/include/asm/pci.h
> +++ b/xen/arch/x86/include/asm/pci.h
> @@ -22,12 +22,16 @@ struct arch_pci_dev {
>       */
>      union {
>          /* Subset of struct arch_iommu's fields, to be used in dom_io. */
> +        #ifdef CONFIG_INTEL_IOMMU
>          struct {
>              uint64_t pgd_maddr;
>          } vtd;
> +        #endif
> +        #ifdef CONFIG_AMD_IOMMU
>          struct {
>              struct page_info *root_table;
>          } amd;
> +        #endif
>      };

The #ifdef and #endif processor directives shouldn't be indented.

Would you mind adding /* CONFIG_{AMD,INTEL}_IOMMU */ comments in the
#endif directives?

I wonder if we could move the definitions of those structures to the
vendor specific headers, but that's more convoluted, and would require
including the iommu headers in pci.h

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 10:22:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 10:22:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728314.1133184 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5a3-0003a5-Nj; Thu, 23 May 2024 10:21:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728314.1133184; Thu, 23 May 2024 10:21:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5a3-0003Zy-LA; Thu, 23 May 2024 10:21:51 +0000
Received: by outflank-mailman (input) for mailman id 728314;
 Thu, 23 May 2024 10:21:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA5a2-0003Zg-9G
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 10:21:50 +0000
Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com
 [2607:f8b0:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 436b6238-18ee-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 12:21:49 +0200 (CEST)
Received: by mail-oi1-x236.google.com with SMTP id
 5614622812f47-3c9b94951d5so2997122b6e.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 03:21:49 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-79484010bcdsm392777085a.133.2024.05.23.03.21.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 03:21:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 436b6238-18ee-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716459708; x=1717064508; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=VxFEt76Z8/U7ETN/vJNgM1sjQaSyKSjkXHjl9SQVCDo=;
        b=HN9s2Y5ud5E9epKQKz0AikoG9Sczw477IrzG5Ld+hDK1Mf4oEF4EGuOGiO68QK8paq
         Dhgvzzwsh9LnTWHYdoKaOqfC1+W7P14mCvM88VR5GOSkQXKNhitWLbgEGHMNjqw2jYLi
         ECvjD7C9L5rdsJVr6cCl81dZ5VS8FX9r7A+BA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716459708; x=1717064508;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VxFEt76Z8/U7ETN/vJNgM1sjQaSyKSjkXHjl9SQVCDo=;
        b=T5iIF5cgsQnNsrVp6s9GolSzBjwNSKYCVsmA/plnTWUcpuHy7Wf+qeOkX1C/YgQv7h
         13rLaOfotqXX7h7OXmDA8NWuJgiRooI5fPBYLmAvjxRu7Ui63KN65rBHrzC2iKsM8ZiI
         /HEamOf4mTPH+0aUC71eH69ehla+bnfLUDbaok5QlnmCM6YP53Uwwrj59isx+fKPS9SC
         eu0ahnBEsxX2lnVa/HDMxbD4IiUI1Tr+TMwuFhyUvQHlxeWSTodtEyFtS241jpAX34V5
         NO3vArJMDaf8QsxE0w4CJ5G7r1hRhr6PFE6qCn1MRSnPPrIciPc6Mr7IkHKWv4D1XzzV
         2M0g==
X-Gm-Message-State: AOJu0Ywpwf4zx4N69uiBTzVfyrufzDpFp2OzrPNhlswfIL650Tgu7sEy
	cO7g8UU/1S4fi9bc91lNZLRo2IfzQdn0ZqCAT39y7oi2TGPQeOs4B18+W6Oz/6Q=
X-Google-Smtp-Source: AGHT+IHrp2K1n78aysbceZHypu7Rwm4s7kvfWgzqy55Uj5/riRKkFkig0ub6m3wSYWDCQiKQFxo/Uw==
X-Received: by 2002:aca:915:0:b0:3c9:6bd5:44aa with SMTP id 5614622812f47-3cdb67a680emr4187187b6e.34.1716459707711;
        Thu, 23 May 2024 03:21:47 -0700 (PDT)
Date: Thu, 23 May 2024 12:21:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v3 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
Message-ID: <Zk8YucyA1ZwGU4Y-@macbook>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
 <5c6ee74b60bad4eb9cc8e17dbfcf7158d38bd32c.1716457040.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5c6ee74b60bad4eb9cc8e17dbfcf7158d38bd32c.1716457040.git.alejandro.vallejo@cloud.com>

On Thu, May 23, 2024 at 10:41:29AM +0100, Alejandro Vallejo wrote:
> The idea is to use xc_cpu_policy_t as a single object containing both the
> serialised and deserialised forms of the policy. Note that we need lengths
> for the arrays, as the serialised policies may be shorter than the array
> capacities.
> 
> * Add the serialised lengths to the struct so we can distinguish
>   between length and capacity of the serialisation buffers.
> * Remove explicit buffer+lengths in serialise/deserialise calls
>   and use the internal buffer inside xc_cpu_policy_t instead.
> * Refactor everything to use the new serialisation functions.
> * Remove redundant serialization calls and avoid allocating dynamic
>   memory aside from the policy objects in xen-cpuid. Also minor cleanup
>   in the policy print call sites.
> 
> No functional change intended.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Just two comments.

> ---
> v3:
>   * Better context scoping in xg_sr_common_x86.
>     * Can't be const because write_record() takes non-const.
>   * Adjusted line length of xen-cpuid's print_policy.
>   * Adjusted error messages in xen-cpuid's print_policy.
>   * Reverted removal of overscoped loop indices.
> ---
>  tools/include/xenguest.h            |  8 ++-
>  tools/libs/guest/xg_cpuid_x86.c     | 98 ++++++++++++++++++++---------
>  tools/libs/guest/xg_private.h       |  2 +
>  tools/libs/guest/xg_sr_common_x86.c | 56 ++++++-----------
>  tools/misc/xen-cpuid.c              | 41 ++++--------
>  5 files changed, 106 insertions(+), 99 deletions(-)
> 
> diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
> index e01f494b772a..563811cd8dde 100644
> --- a/tools/include/xenguest.h
> +++ b/tools/include/xenguest.h
> @@ -799,14 +799,16 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy);
>  
>  /* Manipulate a policy via architectural representations. */
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
>  int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
>                                 const xen_cpuid_leaf_t *leaves,
>                                 uint32_t nr);
>  int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>                                const xen_msr_entry_t *msrs, uint32_t nr);
> +int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
> +int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                           const xen_msr_entry_t **msrs, uint32_t *nr);

Maybe it would be helpful to have a comment clarifying that the return
of xc_cpu_policy_get_{leaves,msrs}() is a reference to the content of
the policy, not a copy of it (and hence is tied to the lifetime of
policy, and doesn't require explicit freeing).

>  
>  /* Compatibility calculations. */
>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4453178100ad..4f4b86b59470 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
>      }
>  }
>  
> -static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
> -                              unsigned int nr_leaves, unsigned int nr_entries)
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>      int rc;
>  
>      rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> -                                    nr_leaves, &err_leaf, &err_subleaf);
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
>      if ( rc )
>      {
>          if ( err_leaf != -1 )
> @@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
>      }
>  
>      rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> -                                  nr_entries, &err_msr);
> +                                  policy->nr_msrs, &err_msr);
>      if ( rc )
>      {
>          if ( err_msr != -1 )
> @@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);
>      if ( rc )
>      {
>          errno = -rc;
> @@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);
>      if ( rc )
>      {
>          errno = -rc;
> @@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
> -    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
>      int rc;
>  
> -    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
> -                                 policy->msrs, &nr_msrs);
> +    rc = xc_cpu_policy_serialise(xch, policy);
>      if ( rc )
>          return rc;
>  
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
> -                                  nr_msrs, policy->msrs,
> +    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
> +                                  policy->nr_msrs, policy->msrs,
>                                    &err_leaf, &err_subleaf, &err_msr);
>      if ( rc )
>      {
> @@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>      return rc;
>  }
>  
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
>  {
> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
> +    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
>      int rc;
>  
> -    if ( leaves )
> +    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
> +    if ( rc )
>      {
> -        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
> -        if ( rc )
> -        {
> -            ERROR("Failed to serialize CPUID policy");
> -            errno = -rc;
> -            return -1;
> -        }
> +        ERROR("Failed to serialize CPUID policy");
> +        errno = -rc;
> +        return -1;
>      }
>  
> -    if ( msrs )
> +    p->nr_leaves = nr_leaves;

Nit: FWIW, I think you could avoid having to introduce local
nr_{leaves,msrs} variables and just use p->nr_{leaves,msrs}?  By
setting them to ARRAY_SIZE() at the top of the function and then
letting x86_{cpuid,msr}_copy_to_buffer() adjust as necessary.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 10:24:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 10:24:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728319.1133194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5cK-0004OA-7O; Thu, 23 May 2024 10:24:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728319.1133194; Thu, 23 May 2024 10:24:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5cK-0004O3-4X; Thu, 23 May 2024 10:24:12 +0000
Received: by outflank-mailman (input) for mailman id 728319;
 Thu, 23 May 2024 10:24:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VxBl=M2=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sA5cI-0004Mr-11
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 10:24:10 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 95fdc122-18ee-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 12:24:07 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56e1bbdb362so9254674a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 03:24:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95fdc122-18ee-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716459846; x=1717064646; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=s+M5wJYiisgpYj+SRPO7ZKLMPAFG0VHDNde4qfV+T6s=;
        b=XdrQ0hie5k4yCcBvsr5ovGunmJHVIBL5SJMLN21KP/hV+/bU93xub9ab+IEY19TWwc
         VFLfce1AeytgfPTMAuYFJS5ec27O4cLVzt8if31gUZew3qiXRHTEATRy+QJCpwsTgeDm
         6m53PAIM9cE4EhZ0R/aIT8n4Z2DwrnkdAClSR0IRaSCrfOTFZ8XwGLBLaZGaQfiaFB68
         Ko3j88LsjIdWXT01uUqdZvfDEEq8FLyFMBSjwwnGLD99Y6b0tGcTmPZwAv+8MkvB+YSS
         6BCVDZiPKPeZinbSPYh1Pg+iwkrgGhW+OP7BbpGjLJjlH1L9xoKnUbvryK4gO3YM0P3B
         I75g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716459846; x=1717064646;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=s+M5wJYiisgpYj+SRPO7ZKLMPAFG0VHDNde4qfV+T6s=;
        b=H41B7N1VkGvGQ+LkFszn03nQasSR+JcNdzWBTo3p43qc9o7WU1ZvIDljVhrcpBDDWx
         pzE+vCSI0xVzr8RMGwQtSk+4ASdEtClF/WR0OzPYtfJ+3viFrPbChkO2Is8eRCk4ZTXp
         OysChobzUIXm49aWQQ04eN5bOshU9dS5MYQrgpMOH3MZq0wv3n+K2Byc8EVY5uOtLQls
         yDt4mMBCFRfaDZbSgFjpLbpnJAN+O07SPfvTuULAmtWZ7hkZbNWsYA3YvruaZsvTGzYA
         WqMlEu9HLezaUcEDQ+RiTZNfhbe4t9545D8/g5vtmErgAEu9MDGxzzWuSvqJdPvxafnf
         J+FA==
X-Gm-Message-State: AOJu0YzAxMUp7+gj2qT/f8rR8gR18k+aRjEP6YByLZpp8T7iCbuh7oiA
	5OWfgEHOD4hy6OrAn1/NGH+KdpBz6TV0IMDqPOE5KGueuiewQaPykym6rLqdBlYjXK3eXPY1Jhh
	zZPCPF6CQrnUGHsRPF3pcyf/HhLg=
X-Google-Smtp-Source: AGHT+IHmpOosZkR+uOLMg9CjcV7QyQceBN6nuV+YQZ1MXXxYWRmkYKTA91fZxdK1OtxtBZBBfZl67r0KfJaxekIgzIA=
X-Received: by 2002:a50:d513:0:b0:574:ebf4:f787 with SMTP id
 4fb4d7f45d1cf-57832a3b5eemr4024151a12.19.1716459846266; Thu, 23 May 2024
 03:24:06 -0700 (PDT)
MIME-Version: 1.0
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com>
 <20240516154804.1114245-8-edgar.iglesias@gmail.com> <dxhms.qrh0wkni3yz@linaro.org>
In-Reply-To: <dxhms.qrh0wkni3yz@linaro.org>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Thu, 23 May 2024 12:23:54 +0200
Message-ID: <CAJy5ezqcnoQ=M4H3RkQ69HAie4jNMEQB42f4NSRZUEWU_5=Gkw@mail.gmail.com>
Subject: Re: [PATCH v6 7/8] xen: mapcache: Add support for grant mappings
To: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Cc: xen-devel@lists.xenproject.org, qemu-devel@nongnu.org, 
	sstabellini@kernel.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, Anthony PERARD <anthony@xenproject.org>, 
	Paul Durrant <paul@xen.org>
Content-Type: multipart/alternative; boundary="00000000000068759206191c7389"

--00000000000068759206191c7389
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, May 23, 2024 at 9:47=E2=80=AFAM Manos Pitsidianakis <
manos.pitsidianakis@linaro.org> wrote:

> On Thu, 16 May 2024 18:48, "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
> wrote:
> >From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> >Add a second mapcache for grant mappings. The mapcache for
> >grants needs to work with XC_PAGE_SIZE granularity since
> >we can't map larger ranges than what has been granted to us.
> >
> >Like with foreign mappings (xen_memory), machines using grants
> >are expected to initialize the xen_grants MR and map it
> >into their address-map accordingly.
> >
> >Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> >Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> >---
> > hw/xen/xen-hvm-common.c         |  12 ++-
> > hw/xen/xen-mapcache.c           | 163 ++++++++++++++++++++++++++------
> > include/hw/xen/xen-hvm-common.h |   3 +
> > include/sysemu/xen.h            |   7 ++
> > 4 files changed, 152 insertions(+), 33 deletions(-)
> >
> >diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> >index a0a0252da0..b8ace1c368 100644
> >--- a/hw/xen/xen-hvm-common.c
> >+++ b/hw/xen/xen-hvm-common.c
> >@@ -10,12 +10,18 @@
> > #include "hw/boards.h"
> > #include "hw/xen/arch_hvm.h"
> >
> >-MemoryRegion xen_memory;
> >+MemoryRegion xen_memory, xen_grants;
> >
> >-/* Check for xen memory.  */
> >+/* Check for any kind of xen memory, foreign mappings or grants.  */
> > bool xen_mr_is_memory(MemoryRegion *mr)
> > {
> >-    return mr =3D=3D &xen_memory;
> >+    return mr =3D=3D &xen_memory || mr =3D=3D &xen_grants;
> >+}
> >+
> >+/* Check specifically for grants.  */
> >+bool xen_mr_is_grants(MemoryRegion *mr)
> >+{
> >+    return mr =3D=3D &xen_grants;
> > }
> >
> > void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion
> *mr,
> >diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> >index a07c47b0b1..1cbc2aeaa9 100644
> >--- a/hw/xen/xen-mapcache.c
> >+++ b/hw/xen/xen-mapcache.c
> >@@ -14,6 +14,7 @@
> >
> > #include <sys/resource.h>
> >
> >+#include "hw/xen/xen-hvm-common.h"
> > #include "hw/xen/xen_native.h"
> > #include "qemu/bitmap.h"
> >
> >@@ -21,6 +22,8 @@
> > #include "sysemu/xen-mapcache.h"
> > #include "trace.h"
> >
> >+#include <xenevtchn.h>
> >+#include <xengnttab.h>
> >
> > #if HOST_LONG_BITS =3D=3D 32
> > #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
> >@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
> >     unsigned long *valid_mapping;
> >     uint32_t lock;
> > #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> >+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
>
> Might we get more entry kinds in the future? (for example foreign maps).
> Maybe this could be an enum.
>
>
Perhaps. Foreign mappings are already supported, this flag separates
ordinary foreign mappings from grant foreign mappings.
IMO, since this is not an external interface it's probably better to change
it once we have a concrete use-case at hand.



> >     uint8_t flags;
> >     hwaddr size;
> >     struct MapCacheEntry *next;
> >@@ -71,6 +75,8 @@ typedef struct MapCache {
> > } MapCache;
> >
> > static MapCache *mapcache;
> >+static MapCache *mapcache_grants;
> >+static xengnttab_handle *xen_region_gnttabdev;
> >
> > static inline void mapcache_lock(MapCache *mc)
> > {
> >@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f,
> void *opaque)
> >     unsigned long max_mcache_size;
> >     unsigned int bucket_shift;
> >
> >+    xen_region_gnttabdev =3D xengnttab_open(NULL, 0);
> >+    if (xen_region_gnttabdev =3D=3D NULL) {
> >+        error_report("mapcache: Failed to open gnttab device");
> >+        exit(EXIT_FAILURE);
> >+    }
> >+
> >     if (HOST_LONG_BITS =3D=3D 32) {
> >         bucket_shift =3D 16;
> >     } else {
> >@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f,
> void *opaque)
> >     mapcache =3D xen_map_cache_init_single(f, opaque,
> >                                          bucket_shift,
> >                                          max_mcache_size);
> >+
> >+    /*
> >+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
> >+     * map anything beyond the number of pages granted to us.
> >+     */
> >+    mapcache_grants =3D xen_map_cache_init_single(f, opaque,
> >+                                                XC_PAGE_SHIFT,
> >+                                                max_mcache_size);
> >+
> >     setrlimit(RLIMIT_AS, &rlimit_as);
> > }
> >
> >@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
> >                              hwaddr size,
> >                              hwaddr address_index,
> >                              bool dummy,
> >+                             bool grant,
> >+                             bool is_write,
> >                              ram_addr_t ram_offset)
> > {
> >     uint8_t *vaddr_base;
> >-    xen_pfn_t *pfns;
> >+    uint32_t *refs =3D NULL;
> >+    xen_pfn_t *pfns =3D NULL;
> >     int *err;
>
> You should use g_autofree to perform automatic cleanup on function exit
> instead of manually freeing, since the allocations should only live
> within the function call.
>
>
Sounds good, I'll do that in the next version.



> >     unsigned int i;
> >     hwaddr nb_pfn =3D size >> XC_PAGE_SHIFT;
> >
> >     trace_xen_remap_bucket(address_index);
> >
> >-    pfns =3D g_new0(xen_pfn_t, nb_pfn);
> >+    if (grant) {
> >+        refs =3D g_new0(uint32_t, nb_pfn);
> >+    } else {
> >+        pfns =3D g_new0(xen_pfn_t, nb_pfn);
> >+    }
> >     err =3D g_new0(int, nb_pfn);
> >
> >     if (entry->vaddr_base !=3D NULL) {
> >@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
> >     g_free(entry->valid_mapping);
> >     entry->valid_mapping =3D NULL;
> >
> >-    for (i =3D 0; i < nb_pfn; i++) {
> >-        pfns[i] =3D (address_index << (mc->bucket_shift - XC_PAGE_SHIFT=
))
> + i;
> >+    if (grant) {
> >+        hwaddr grant_base =3D address_index - (ram_offset >>
> XC_PAGE_SHIFT);
> >+
> >+        for (i =3D 0; i < nb_pfn; i++) {
> >+            refs[i] =3D grant_base + i;
> >+        }
> >+    } else {
> >+        for (i =3D 0; i < nb_pfn; i++) {
> >+            pfns[i] =3D (address_index << (mc->bucket_shift -
> XC_PAGE_SHIFT)) + i;
> >+        }
> >     }
> >
> >-    /*
> >-     * If the caller has requested the mapping at a specific address us=
e
> >-     * MAP_FIXED to make sure it's honored.
> >-     */
> >+    entry->flags &=3D ~XEN_MAPCACHE_ENTRY_GRANT;
> >+
> >     if (!dummy) {
> >-        vaddr_base =3D xenforeignmemory_map2(xen_fmem, xen_domid, vaddr=
,
> >-                                           PROT_READ | PROT_WRITE,
> >-                                           vaddr ? MAP_FIXED : 0,
> >-                                           nb_pfn, pfns, err);
>
> Since err is not NULL here, the function might return a valid pointer
> but individual frames might have failed.
>
>
Yes but AFAICT, the case when some pages fail foreign mapping is handled
further down the function (see the valid_mappings bitmap).
Note that this series isn't really changing this existing behaviour for
foreign mappings. In any case, If we spot a bug in existing code, I'm happy
to fix it.



>
> >+        if (grant) {
> >+            int prot =3D PROT_READ;
> >+
> >+            if (is_write) {
> >+                prot |=3D PROT_WRITE;
> >+            }
> >+
> >+            entry->flags |=3D XEN_MAPCACHE_ENTRY_GRANT;
> >+            assert(vaddr =3D=3D NULL);
> >+            vaddr_base =3D
> xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
> >+                                                         nb_pfn,
> >+                                                         xen_domid, ref=
s,
> >+                                                         prot);
> >+        } else {
> >+            /*
> >+             * If the caller has requested the mapping at a specific
> address use
> >+             * MAP_FIXED to make sure it's honored.
> >+             *
> >+             * We don't yet support upgrading mappings from RO to RW, t=
o
> handle
> >+             * models using ordinary address_space_rw(), foreign
> mappings ignore
> >+             * is_write and are always mapped RW.
> >+             */
> >+            vaddr_base =3D xenforeignmemory_map2(xen_fmem, xen_domid,
> vaddr,
> >+                                               PROT_READ | PROT_WRITE,
> >+                                               vaddr ? MAP_FIXED : 0,
> >+                                               nb_pfn, pfns, err);
> >+        }
> >         if (vaddr_base =3D=3D NULL) {
> >-            perror("xenforeignmemory_map2");
> >+            perror(grant ? "xengnttab_map_domain_grant_refs"
> >+                           : "xenforeignmemory_map2");
> >             exit(-1);
> >         }
> >     } else {
> >@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,
> >         }
> >     }
> >
> >+    g_free(refs);
> >     g_free(pfns);
> >     g_free(err);
> > }
> >@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,
> > static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> >                                        hwaddr phys_addr, hwaddr size,
> >                                        ram_addr_t ram_offset,
> >-                                       uint8_t lock, bool dma, bool
> is_write)
> >+                                       uint8_t lock, bool dma,
> >+                                       bool grant, bool is_write)
> > {
> >     MapCacheEntry *entry, *pentry =3D NULL,
> >                   *free_entry =3D NULL, *free_pentry =3D NULL;
> >@@ -340,7 +400,7 @@ tryagain:
> >         entry =3D g_new0(MapCacheEntry, 1);
> >         pentry->next =3D entry;
> >         xen_remap_bucket(mc, entry, NULL, cache_size, address_index,
> dummy,
> >-                         ram_offset);
> >+                         grant, is_write, ram_offset);
> >     } else if (!entry->lock) {
> >         if (!entry->vaddr_base || entry->paddr_index !=3D address_index=
 ||
> >                 entry->size !=3D cache_size ||
> >@@ -348,7 +408,7 @@ tryagain:
> >                     test_bit_size >> XC_PAGE_SHIFT,
> >                     entry->valid_mapping)) {
> >             xen_remap_bucket(mc, entry, NULL, cache_size, address_index=
,
> dummy,
> >-                             ram_offset);
> >+                             grant, is_write, ram_offset);
> >         }
> >     }
> >
> >@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
> >                        uint8_t lock, bool dma,
> >                        bool is_write)
> > {
> >+    bool grant =3D xen_mr_is_grants(mr);
> >+    MapCache *mc =3D grant ? mapcache_grants : mapcache;
> >     uint8_t *p;
> >
> >-    mapcache_lock(mapcache);
> >-    p =3D xen_map_cache_unlocked(mapcache, phys_addr, size,
> ram_addr_offset,
> >-                               lock, dma, is_write);
> >-    mapcache_unlock(mapcache);
> >+    if (grant) {
> >+        /*
> >+         * Grants are only supported via address_space_map(). Anything
> >+         * else is considered a user/guest error.
> >+         *
> >+         * QEMU generally doesn't expect these mappings to ever fail, s=
o
> >+         * if this happens we report an error message and abort().
> >+         */
> >+        if (!lock) {
>
> Nested if conditions that can be flattened, i.e. this could be
>
> if (grant && !lock)
>


Sounds good, will flatten this in the next version.


>
> >+            error_report("Trying access a grant reference without
> mapping it.");
>
> s/Trying access a grant/Tried to access a grant/
>
>
Will fix it, thanks!

Best regards,
Edgar



> >+            abort();
> >+        }
> >+    }
> >+
> >+    mapcache_lock(mc);
> >+    p =3D xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
> >+                               lock, dma, grant, is_write);
> >+    mapcache_unlock(mc);
> >     return p;
> > }
> >
> >@@ -449,7 +525,14 @@ static ram_addr_t
> xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
> >
> > ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> > {
> >-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
> >+    ram_addr_t addr;
> >+
> >+    addr =3D xen_ram_addr_from_mapcache_single(mapcache, ptr);
> >+    if (addr =3D=3D RAM_ADDR_INVALID) {
> >+        addr =3D xen_ram_addr_from_mapcache_single(mapcache_grants, ptr=
);
> >+    }
> >+
> >+    return addr;
> > }
> >
> > static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> >@@ -460,6 +543,7 @@ static void
> xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> >     hwaddr paddr_index;
> >     hwaddr size;
> >     int found =3D 0;
> >+    int rc;
> >
> >     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
> >         if (reventry->vaddr_req =3D=3D buffer) {
> >@@ -502,7 +586,14 @@ static void
> xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
> >     }
> >
> >     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size=
);
> >-    if (munmap(entry->vaddr_base, entry->size) !=3D 0) {
> >+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
> >+        rc =3D xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
> >+                             entry->size >> mc->bucket_shift);
> >+    } else {
> >+        rc =3D munmap(entry->vaddr_base, entry->size);
> >+    }
> >+
> >+    if (rc) {
> >         perror("unmap fails");
> >         exit(-1);
> >     }
> >@@ -521,14 +612,24 @@ typedef struct XenMapCacheData {
> >     uint8_t *buffer;
> > } XenMapCacheData;
> >
> >+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t
> *buffer)
> >+{
> >+    mapcache_lock(mc);
> >+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
> >+    mapcache_unlock(mc);
> >+}
> >+
> >+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
> >+{
> >+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
> >+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
> >+}
> >+
> > static void xen_invalidate_map_cache_entry_bh(void *opaque)
> > {
> >     XenMapCacheData *data =3D opaque;
> >
> >-    mapcache_lock(mapcache);
> >-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
> >-    mapcache_unlock(mapcache);
> >-
> >+    xen_invalidate_map_cache_entry_all(data->buffer);
> >     aio_co_wake(data->co);
> > }
> >
> >@@ -543,9 +644,7 @@ void coroutine_mixed_fn
> xen_invalidate_map_cache_entry(uint8_t *buffer)
> >                                 xen_invalidate_map_cache_entry_bh,
> &data);
> >         qemu_coroutine_yield();
> >     } else {
> >-        mapcache_lock(mapcache);
> >-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
> >-        mapcache_unlock(mapcache);
> >+        xen_invalidate_map_cache_entry_all(buffer);
> >     }
> > }
> >
> >@@ -597,6 +696,7 @@ void xen_invalidate_map_cache(void)
> >     bdrv_drain_all();
> >
> >     xen_invalidate_map_cache_single(mapcache);
> >+    xen_invalidate_map_cache_single(mapcache_grants);
> > }
> >
> > static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> >@@ -632,13 +732,16 @@ static uint8_t
> *xen_replace_cache_entry_unlocked(MapCache *mc,
> >         return NULL;
> >     }
> >
> >+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) =3D=3D 0);
> >+
> >     address_index  =3D new_phys_addr >> mc->bucket_shift;
> >     address_offset =3D new_phys_addr & (mc->bucket_size - 1);
> >
> >     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> >
> >     xen_remap_bucket(mc, entry, entry->vaddr_base,
> >-                     cache_size, address_index, false, old_phys_addr);
> >+                     cache_size, address_index, false,
> >+                     false, false, old_phys_addr);
> >     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> >                 test_bit_size >> XC_PAGE_SHIFT,
> >                 entry->valid_mapping)) {
> >diff --git a/include/hw/xen/xen-hvm-common.h
> b/include/hw/xen/xen-hvm-common.h
> >index 65a51aac2e..3d796235dc 100644
> >--- a/include/hw/xen/xen-hvm-common.h
> >+++ b/include/hw/xen/xen-hvm-common.h
> >@@ -16,6 +16,7 @@
> > #include <xen/hvm/ioreq.h>
> >
> > extern MemoryRegion xen_memory;
> >+extern MemoryRegion xen_grants;
> > extern MemoryListener xen_io_listener;
> > extern DeviceListener xen_device_listener;
> >
> >@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
> >     do { } while (0)
> > #endif
> >
> >+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
> >+
> > static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int
> i)
> > {
> >     return shared_page->vcpu_ioreq[i].vp_eport;
> >diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> >index dc72f83bcb..19dccf4d71 100644
> >--- a/include/sysemu/xen.h
> >+++ b/include/sysemu/xen.h
> >@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t siz=
e,
> >                    struct MemoryRegion *mr, Error **errp);
> >
> > bool xen_mr_is_memory(MemoryRegion *mr);
> >+bool xen_mr_is_grants(MemoryRegion *mr);
> >
> > #else /* !CONFIG_XEN_IS_POSSIBLE */
> >
> >@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
> >     return false;
> > }
> >
> >+static inline bool xen_mr_is_grants(MemoryRegion *mr)
> >+{
> >+    g_assert_not_reached();
> >+    return false;
> >+}
> >+
> > #endif /* CONFIG_XEN_IS_POSSIBLE */
> >
> > #endif
> >--
> >2.40.1
> >
> >
>

--00000000000068759206191c7389
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Thu, May 23, 2024 at 9:47=E2=80=AFAM M=
anos Pitsidianakis &lt;<a href=3D"mailto:manos.pitsidianakis@linaro.org">ma=
nos.pitsidianakis@linaro.org</a>&gt; wrote:<br></div><div class=3D"gmail_qu=
ote"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left:1px solid rgb(204,204,204);padding-left:1ex">On Thu, 16 May 2024 =
18:48, &quot;Edgar E. Iglesias&quot; &lt;<a href=3D"mailto:edgar.iglesias@g=
mail.com" target=3D"_blank">edgar.iglesias@gmail.com</a>&gt; wrote:<br>
&gt;From: &quot;Edgar E. Iglesias&quot; &lt;<a href=3D"mailto:edgar.iglesia=
s@amd.com" target=3D"_blank">edgar.iglesias@amd.com</a>&gt;<br>
&gt;<br>
&gt;Add a second mapcache for grant mappings. The mapcache for<br>
&gt;grants needs to work with XC_PAGE_SIZE granularity since<br>
&gt;we can&#39;t map larger ranges than what has been granted to us.<br>
&gt;<br>
&gt;Like with foreign mappings (xen_memory), machines using grants<br>
&gt;are expected to initialize the xen_grants MR and map it<br>
&gt;into their address-map accordingly.<br>
&gt;<br>
&gt;Signed-off-by: Edgar E. Iglesias &lt;<a href=3D"mailto:edgar.iglesias@a=
md.com" target=3D"_blank">edgar.iglesias@amd.com</a>&gt;<br>
&gt;Reviewed-by: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kerne=
l.org" target=3D"_blank">sstabellini@kernel.org</a>&gt;<br>
&gt;---<br>
&gt; hw/xen/xen-hvm-common.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 12 ++=
-<br>
&gt; hw/xen/xen-mapcache.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 163 ++=
++++++++++++++++++++++++------<br>
&gt; include/hw/xen/xen-hvm-common.h |=C2=A0 =C2=A03 +<br>
&gt; include/sysemu/xen.h=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =
=C2=A07 ++<br>
&gt; 4 files changed, 152 insertions(+), 33 deletions(-)<br>
&gt;<br>
&gt;diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c<br>
&gt;index a0a0252da0..b8ace1c368 100644<br>
&gt;--- a/hw/xen/xen-hvm-common.c<br>
&gt;+++ b/hw/xen/xen-hvm-common.c<br>
&gt;@@ -10,12 +10,18 @@<br>
&gt; #include &quot;hw/boards.h&quot;<br>
&gt; #include &quot;hw/xen/arch_hvm.h&quot;<br>
&gt; <br>
&gt;-MemoryRegion xen_memory;<br>
&gt;+MemoryRegion xen_memory, xen_grants;<br>
&gt; <br>
&gt;-/* Check for xen memory.=C2=A0 */<br>
&gt;+/* Check for any kind of xen memory, foreign mappings or grants.=C2=A0=
 */<br>
&gt; bool xen_mr_is_memory(MemoryRegion *mr)<br>
&gt; {<br>
&gt;-=C2=A0 =C2=A0 return mr =3D=3D &amp;xen_memory;<br>
&gt;+=C2=A0 =C2=A0 return mr =3D=3D &amp;xen_memory || mr =3D=3D &amp;xen_g=
rants;<br>
&gt;+}<br>
&gt;+<br>
&gt;+/* Check specifically for grants.=C2=A0 */<br>
&gt;+bool xen_mr_is_grants(MemoryRegion *mr)<br>
&gt;+{<br>
&gt;+=C2=A0 =C2=A0 return mr =3D=3D &amp;xen_grants;<br>
&gt; }<br>
&gt; <br>
&gt; void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion =
*mr,<br>
&gt;diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c<br>
&gt;index a07c47b0b1..1cbc2aeaa9 100644<br>
&gt;--- a/hw/xen/xen-mapcache.c<br>
&gt;+++ b/hw/xen/xen-mapcache.c<br>
&gt;@@ -14,6 +14,7 @@<br>
&gt; <br>
&gt; #include &lt;sys/resource.h&gt;<br>
&gt; <br>
&gt;+#include &quot;hw/xen/xen-hvm-common.h&quot;<br>
&gt; #include &quot;hw/xen/xen_native.h&quot;<br>
&gt; #include &quot;qemu/bitmap.h&quot;<br>
&gt; <br>
&gt;@@ -21,6 +22,8 @@<br>
&gt; #include &quot;sysemu/xen-mapcache.h&quot;<br>
&gt; #include &quot;trace.h&quot;<br>
&gt; <br>
&gt;+#include &lt;xenevtchn.h&gt;<br>
&gt;+#include &lt;xengnttab.h&gt;<br>
&gt; <br>
&gt; #if HOST_LONG_BITS =3D=3D 32<br>
&gt; #=C2=A0 define MCACHE_MAX_SIZE=C2=A0 =C2=A0 =C2=A0(1UL&lt;&lt;31) /* 2=
GB Cap */<br>
&gt;@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {<br>
&gt;=C2=A0 =C2=A0 =C2=A0unsigned long *valid_mapping;<br>
&gt;=C2=A0 =C2=A0 =C2=A0uint32_t lock;<br>
&gt; #define XEN_MAPCACHE_ENTRY_DUMMY (1 &lt;&lt; 0)<br>
&gt;+#define XEN_MAPCACHE_ENTRY_GRANT (1 &lt;&lt; 1)<br>
<br>
Might we get more entry kinds in the future? (for example foreign maps). <b=
r>
Maybe this could be an enum.<br>
<br></blockquote><div><br></div><div>Perhaps. Foreign mappings are already =
supported, this flag separates ordinary foreign mappings from grant foreign=
 mappings.</div><div>IMO, since this is not an external interface it&#39;s =
probably better to change it once we have a concrete use-case at hand.</div=
><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D=
"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-le=
ft:1ex">
&gt;=C2=A0 =C2=A0 =C2=A0uint8_t flags;<br>
&gt;=C2=A0 =C2=A0 =C2=A0hwaddr size;<br>
&gt;=C2=A0 =C2=A0 =C2=A0struct MapCacheEntry *next;<br>
&gt;@@ -71,6 +75,8 @@ typedef struct MapCache {<br>
&gt; } MapCache;<br>
&gt; <br>
&gt; static MapCache *mapcache;<br>
&gt;+static MapCache *mapcache_grants;<br>
&gt;+static xengnttab_handle *xen_region_gnttabdev;<br>
&gt; <br>
&gt; static inline void mapcache_lock(MapCache *mc)<br>
&gt; {<br>
&gt;@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, =
void *opaque)<br>
&gt;=C2=A0 =C2=A0 =C2=A0unsigned long max_mcache_size;<br>
&gt;=C2=A0 =C2=A0 =C2=A0unsigned int bucket_shift;<br>
&gt; <br>
&gt;+=C2=A0 =C2=A0 xen_region_gnttabdev =3D xengnttab_open(NULL, 0);<br>
&gt;+=C2=A0 =C2=A0 if (xen_region_gnttabdev =3D=3D NULL) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report(&quot;mapcache: Failed to ope=
n gnttab device&quot;);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 exit(EXIT_FAILURE);<br>
&gt;+=C2=A0 =C2=A0 }<br>
&gt;+<br>
&gt;=C2=A0 =C2=A0 =C2=A0if (HOST_LONG_BITS =3D=3D 32) {<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bucket_shift =3D 16;<br>
&gt;=C2=A0 =C2=A0 =C2=A0} else {<br>
&gt;@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, =
void *opaque)<br>
&gt;=C2=A0 =C2=A0 =C2=A0mapcache =3D xen_map_cache_init_single(f, opaque,<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 bucket_shift,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 max_mcache_size);<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 /*<br>
&gt;+=C2=A0 =C2=A0 =C2=A0* Grant mappings must use XC_PAGE_SIZE granularity=
 since we can&#39;t<br>
&gt;+=C2=A0 =C2=A0 =C2=A0* map anything beyond the number of pages granted =
to us.<br>
&gt;+=C2=A0 =C2=A0 =C2=A0*/<br>
&gt;+=C2=A0 =C2=A0 mapcache_grants =3D xen_map_cache_init_single(f, opaque,=
<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 XC_PAGE_SHIFT,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 max_mcache_size);<br>
&gt;+<br>
&gt;=C2=A0 =C2=A0 =C2=A0setrlimit(RLIMIT_AS, &amp;rlimit_as);<br>
&gt; }<br>
&gt; <br>
&gt;@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 hwaddr size,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 hwaddr address_index,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bool dummy,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bool grant,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bool is_write,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ram_addr_t ram_offset)<br>
&gt; {<br>
&gt;=C2=A0 =C2=A0 =C2=A0uint8_t *vaddr_base;<br>
&gt;-=C2=A0 =C2=A0 xen_pfn_t *pfns;<br>
&gt;+=C2=A0 =C2=A0 uint32_t *refs =3D NULL;<br>
&gt;+=C2=A0 =C2=A0 xen_pfn_t *pfns =3D NULL;<br>
&gt;=C2=A0 =C2=A0 =C2=A0int *err;<br>
<br>
You should use g_autofree to perform automatic cleanup on function exit <br=
>
instead of manually freeing, since the allocations should only live <br>
within the function call.<br>
<br></blockquote><div><br></div><div>Sounds good, I&#39;ll do that in the n=
ext version.</div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20=
4,204);padding-left:1ex">
&gt;=C2=A0 =C2=A0 =C2=A0unsigned int i;<br>
&gt;=C2=A0 =C2=A0 =C2=A0hwaddr nb_pfn =3D size &gt;&gt; XC_PAGE_SHIFT;<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0trace_xen_remap_bucket(address_index);<br>
&gt; <br>
&gt;-=C2=A0 =C2=A0 pfns =3D g_new0(xen_pfn_t, nb_pfn);<br>
&gt;+=C2=A0 =C2=A0 if (grant) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 refs =3D g_new0(uint32_t, nb_pfn);<br>
&gt;+=C2=A0 =C2=A0 } else {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 pfns =3D g_new0(xen_pfn_t, nb_pfn);<br>
&gt;+=C2=A0 =C2=A0 }<br>
&gt;=C2=A0 =C2=A0 =C2=A0err =3D g_new0(int, nb_pfn);<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0if (entry-&gt;vaddr_base !=3D NULL) {<br>
&gt;@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0g_free(entry-&gt;valid_mapping);<br>
&gt;=C2=A0 =C2=A0 =C2=A0entry-&gt;valid_mapping =3D NULL;<br>
&gt; <br>
&gt;-=C2=A0 =C2=A0 for (i =3D 0; i &lt; nb_pfn; i++) {<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 pfns[i] =3D (address_index &lt;&lt; (mc-&g=
t;bucket_shift - XC_PAGE_SHIFT)) + i;<br>
&gt;+=C2=A0 =C2=A0 if (grant) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 hwaddr grant_base =3D address_index - (ram=
_offset &gt;&gt; XC_PAGE_SHIFT);<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 for (i =3D 0; i &lt; nb_pfn; i++) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 refs[i] =3D grant_base + i;<=
br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
&gt;+=C2=A0 =C2=A0 } else {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 for (i =3D 0; i &lt; nb_pfn; i++) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pfns[i] =3D (address_index &=
lt;&lt; (mc-&gt;bucket_shift - XC_PAGE_SHIFT)) + i;<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; <br>
&gt;-=C2=A0 =C2=A0 /*<br>
&gt;-=C2=A0 =C2=A0 =C2=A0* If the caller has requested the mapping at a spe=
cific address use<br>
&gt;-=C2=A0 =C2=A0 =C2=A0* MAP_FIXED to make sure it&#39;s honored.<br>
&gt;-=C2=A0 =C2=A0 =C2=A0*/<br>
&gt;+=C2=A0 =C2=A0 entry-&gt;flags &amp;=3D ~XEN_MAPCACHE_ENTRY_GRANT;<br>
&gt;+<br>
&gt;=C2=A0 =C2=A0 =C2=A0if (!dummy) {<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 vaddr_base =3D xenforeignmemory_map2(xen_f=
mem, xen_domid, vaddr,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0PROT_READ | PROT_WRITE,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0vaddr ? MAP_FIXED : 0,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0nb_pfn, pfns, err);<br>
<br>
Since err is not NULL here, the function might return a valid pointer <br>
but individual frames might have failed.<br>
<br></blockquote><div><br></div><div>Yes but AFAICT, the case when some pag=
es fail foreign mapping is handled further=C2=A0down the function (see the =
valid_mappings bitmap).</div><div>Note that this series isn&#39;t really ch=
anging this existing behaviour for foreign mappings. In any case, If we spo=
t a bug in existing code, I&#39;m happy to fix it.</div><div><br></div><div=
>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (grant) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int prot =3D PROT_READ;<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (is_write) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 prot |=3D PROT=
_WRITE;<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry-&gt;flags |=3D XEN_MAP=
CACHE_ENTRY_GRANT;<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 assert(vaddr =3D=3D NULL);<b=
r>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vaddr_base =3D xengnttab_map=
_domain_grant_refs(xen_region_gnttabdev,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nb_pfn,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0xen_domid, refs,=
<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0prot);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 } else {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /*<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* If the caller has re=
quested the mapping at a specific address use<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* MAP_FIXED to make su=
re it&#39;s honored.<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* We don&#39;t yet sup=
port upgrading mappings from RO to RW, to handle<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* models using ordinar=
y address_space_rw(), foreign mappings ignore<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* is_write and are alw=
ays mapped RW.<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 vaddr_base =3D xenforeignmem=
ory_map2(xen_fmem, xen_domid, vaddr,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0PROT_READ | PROT_WRITE,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0vaddr ? MAP_FIXED : 0,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0nb_pfn, pfns, err);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vaddr_base =3D=3D NULL) {<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 perror(&quot;xenforeignmemor=
y_map2&quot;);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 perror(grant ? &quot;xengntt=
ab_map_domain_grant_refs&quot;<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0: &quot;xenforeignmemory_map2&quot;);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0exit(-1);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0 =C2=A0} else {<br>
&gt;@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; <br>
&gt;+=C2=A0 =C2=A0 g_free(refs);<br>
&gt;=C2=A0 =C2=A0 =C2=A0g_free(pfns);<br>
&gt;=C2=A0 =C2=A0 =C2=A0g_free(err);<br>
&gt; }<br>
&gt;@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,<br>
&gt; static uint8_t *xen_map_cache_unlocked(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 hwadd=
r phys_addr, hwaddr size,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ram_a=
ddr_t ram_offset,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint8_=
t lock, bool dma, bool is_write)<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint8_=
t lock, bool dma,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bool g=
rant, bool is_write)<br>
&gt; {<br>
&gt;=C2=A0 =C2=A0 =C2=A0MapCacheEntry *entry, *pentry =3D NULL,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*f=
ree_entry =3D NULL, *free_pentry =3D NULL;<br>
&gt;@@ -340,7 +400,7 @@ tryagain:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0entry =3D g_new0(MapCacheEntry, 1);<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pentry-&gt;next =3D entry;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0xen_remap_bucket(mc, entry, NULL, cac=
he_size, address_index, dummy,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0ram_offset);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0grant, is_write, ram_offset);<br>
&gt;=C2=A0 =C2=A0 =C2=A0} else if (!entry-&gt;lock) {<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!entry-&gt;vaddr_base || entry-&g=
t;paddr_index !=3D address_index ||<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0entry-&gt=
;size !=3D cache_size ||<br>
&gt;@@ -348,7 +408,7 @@ tryagain:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0test_bit_size &gt;&gt; XC_PAGE_SHIFT,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0entry-&gt;valid_mapping)) {<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0xen_remap_bucket(mc, en=
try, NULL, cache_size, address_index, dummy,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ram_offset);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0grant, is_write, ram_offset);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; <br>
&gt;@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 uint8_t lock, bool dma,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 bool is_write)<br>
&gt; {<br>
&gt;+=C2=A0 =C2=A0 bool grant =3D xen_mr_is_grants(mr);<br>
&gt;+=C2=A0 =C2=A0 MapCache *mc =3D grant ? mapcache_grants : mapcache;<br>
&gt;=C2=A0 =C2=A0 =C2=A0uint8_t *p;<br>
&gt; <br>
&gt;-=C2=A0 =C2=A0 mapcache_lock(mapcache);<br>
&gt;-=C2=A0 =C2=A0 p =3D xen_map_cache_unlocked(mapcache, phys_addr, size, =
ram_addr_offset,<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lock, dma, is_write);<br>
&gt;-=C2=A0 =C2=A0 mapcache_unlock(mapcache);<br>
&gt;+=C2=A0 =C2=A0 if (grant) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 /*<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* Grants are only supported via addr=
ess_space_map(). Anything<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* else is considered a user/guest er=
ror.<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* QEMU generally doesn&#39;t expect =
these mappings to ever fail, so<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* if this happens we report an error=
 message and abort().<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!lock) {<br>
<br>
Nested if conditions that can be flattened, i.e. this could be <br>
<br>
if (grant &amp;&amp; !lock)<br></blockquote><div><br></div><div><br></div><=
div>Sounds good, will flatten this in the next version.</div><div>=C2=A0<br=
></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report(&quot;Trying ac=
cess a grant reference without mapping it.&quot;);<br>
<br>
s/Trying access a grant/Tried to access a grant/<br>
<br></blockquote><div><br></div><div>Will fix=C2=A0it, thanks!</div><div><b=
r></div><div>Best regards,</div><div>Edgar</div><div><br></div><div>=C2=A0<=
/div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left:1px solid rgb(204,204,204);padding-left:1ex">
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
&gt;+=C2=A0 =C2=A0 }<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 mapcache_lock(mc);<br>
&gt;+=C2=A0 =C2=A0 p =3D xen_map_cache_unlocked(mc, phys_addr, size, ram_ad=
dr_offset,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lock, dma, grant, is_write);<br>
&gt;+=C2=A0 =C2=A0 mapcache_unlock(mc);<br>
&gt;=C2=A0 =C2=A0 =C2=A0return p;<br>
&gt; }<br>
&gt; <br>
&gt;@@ -449,7 +525,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_singl=
e(MapCache *mc, void *ptr)<br>
&gt; <br>
&gt; ram_addr_t xen_ram_addr_from_mapcache(void *ptr)<br>
&gt; {<br>
&gt;-=C2=A0 =C2=A0 return xen_ram_addr_from_mapcache_single(mapcache, ptr);=
<br>
&gt;+=C2=A0 =C2=A0 ram_addr_t addr;<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 addr =3D xen_ram_addr_from_mapcache_single(mapcache, ptr=
);<br>
&gt;+=C2=A0 =C2=A0 if (addr =3D=3D RAM_ADDR_INVALID) {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 addr =3D xen_ram_addr_from_mapcache_single=
(mapcache_grants, ptr);<br>
&gt;+=C2=A0 =C2=A0 }<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 return addr;<br>
&gt; }<br>
&gt; <br>
&gt; static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,<br>
&gt;@@ -460,6 +543,7 @@ static void xen_invalidate_map_cache_entry_unlocked=
(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0hwaddr paddr_index;<br>
&gt;=C2=A0 =C2=A0 =C2=A0hwaddr size;<br>
&gt;=C2=A0 =C2=A0 =C2=A0int found =3D 0;<br>
&gt;+=C2=A0 =C2=A0 int rc;<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0QTAILQ_FOREACH(reventry, &amp;mc-&gt;locked_entries=
, next) {<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (reventry-&gt;vaddr_req =3D=3D buf=
fer) {<br>
&gt;@@ -502,7 +586,14 @@ static void xen_invalidate_map_cache_entry_unlocke=
d(MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0ram_block_notify_remove(entry-&gt;vaddr_base, entry=
-&gt;size, entry-&gt;size);<br>
&gt;-=C2=A0 =C2=A0 if (munmap(entry-&gt;vaddr_base, entry-&gt;size) !=3D 0)=
 {<br>
&gt;+=C2=A0 =C2=A0 if (entry-&gt;flags &amp; XEN_MAPCACHE_ENTRY_GRANT) {<br=
>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 rc =3D xengnttab_unmap(xen_region_gnttabde=
v, entry-&gt;vaddr_base,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0entry-&gt;size &gt;&gt; mc-&gt;bucket_shi=
ft);<br>
&gt;+=C2=A0 =C2=A0 } else {<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 rc =3D munmap(entry-&gt;vaddr_base, entry-=
&gt;size);<br>
&gt;+=C2=A0 =C2=A0 }<br>
&gt;+<br>
&gt;+=C2=A0 =C2=A0 if (rc) {<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0perror(&quot;unmap fails&quot;);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0exit(-1);<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt;@@ -521,14 +612,24 @@ typedef struct XenMapCacheData {<br>
&gt;=C2=A0 =C2=A0 =C2=A0uint8_t *buffer;<br>
&gt; } XenMapCacheData;<br>
&gt; <br>
&gt;+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_=
t *buffer)<br>
&gt;+{<br>
&gt;+=C2=A0 =C2=A0 mapcache_lock(mc);<br>
&gt;+=C2=A0 =C2=A0 xen_invalidate_map_cache_entry_unlocked(mc, buffer);<br>
&gt;+=C2=A0 =C2=A0 mapcache_unlock(mc);<br>
&gt;+}<br>
&gt;+<br>
&gt;+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)<br>
&gt;+{<br>
&gt;+=C2=A0 =C2=A0 xen_invalidate_map_cache_entry_single(mapcache, buffer);=
<br>
&gt;+=C2=A0 =C2=A0 xen_invalidate_map_cache_entry_single(mapcache_grants, b=
uffer);<br>
&gt;+}<br>
&gt;+<br>
&gt; static void xen_invalidate_map_cache_entry_bh(void *opaque)<br>
&gt; {<br>
&gt;=C2=A0 =C2=A0 =C2=A0XenMapCacheData *data =3D opaque;<br>
&gt; <br>
&gt;-=C2=A0 =C2=A0 mapcache_lock(mapcache);<br>
&gt;-=C2=A0 =C2=A0 xen_invalidate_map_cache_entry_unlocked(mapcache, data-&=
gt;buffer);<br>
&gt;-=C2=A0 =C2=A0 mapcache_unlock(mapcache);<br>
&gt;-<br>
&gt;+=C2=A0 =C2=A0 xen_invalidate_map_cache_entry_all(data-&gt;buffer);<br>
&gt;=C2=A0 =C2=A0 =C2=A0aio_co_wake(data-&gt;co);<br>
&gt; }<br>
&gt; <br>
&gt;@@ -543,9 +644,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_en=
try(uint8_t *buffer)<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0xen_invalidate_map_cache_en=
try_bh, &amp;data);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0qemu_coroutine_yield();<br>
&gt;=C2=A0 =C2=A0 =C2=A0} else {<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 mapcache_lock(mapcache);<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 xen_invalidate_map_cache_entry_unlocked(ma=
pcache, buffer);<br>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 mapcache_unlock(mapcache);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 xen_invalidate_map_cache_entry_all(buffer)=
;<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; }<br>
&gt; <br>
&gt;@@ -597,6 +696,7 @@ void xen_invalidate_map_cache(void)<br>
&gt;=C2=A0 =C2=A0 =C2=A0bdrv_drain_all();<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0xen_invalidate_map_cache_single(mapcache);<br>
&gt;+=C2=A0 =C2=A0 xen_invalidate_map_cache_single(mapcache_grants);<br>
&gt; }<br>
&gt; <br>
&gt; static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,<br>
&gt;@@ -632,13 +732,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(=
MapCache *mc,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NULL;<br>
&gt;=C2=A0 =C2=A0 =C2=A0}<br>
&gt; <br>
&gt;+=C2=A0 =C2=A0 assert((entry-&gt;flags &amp; XEN_MAPCACHE_ENTRY_GRANT) =
=3D=3D 0);<br>
&gt;+<br>
&gt;=C2=A0 =C2=A0 =C2=A0address_index=C2=A0 =3D new_phys_addr &gt;&gt; mc-&=
gt;bucket_shift;<br>
&gt;=C2=A0 =C2=A0 =C2=A0address_offset =3D new_phys_addr &amp; (mc-&gt;buck=
et_size - 1);<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0trace_xen_replace_cache_entry_dummy(old_phys_addr, =
new_phys_addr);<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0xen_remap_bucket(mc, entry, entry-&gt;vaddr_base,<b=
r>
&gt;-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0cache_size, address_index, false, old_phys_addr);<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0cache_size, address_index, false,<br>
&gt;+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0false, false, old_phys_addr);<br>
&gt;=C2=A0 =C2=A0 =C2=A0if (!test_bits(address_offset &gt;&gt; XC_PAGE_SHIF=
T,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0test_bit_=
size &gt;&gt; XC_PAGE_SHIFT,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0entry-&gt=
;valid_mapping)) {<br>
&gt;diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-c=
ommon.h<br>
&gt;index 65a51aac2e..3d796235dc 100644<br>
&gt;--- a/include/hw/xen/xen-hvm-common.h<br>
&gt;+++ b/include/hw/xen/xen-hvm-common.h<br>
&gt;@@ -16,6 +16,7 @@<br>
&gt; #include &lt;xen/hvm/ioreq.h&gt;<br>
&gt; <br>
&gt; extern MemoryRegion xen_memory;<br>
&gt;+extern MemoryRegion xen_grants;<br>
&gt; extern MemoryListener xen_io_listener;<br>
&gt; extern DeviceListener xen_device_listener;<br>
&gt; <br>
&gt;@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;<br>
&gt;=C2=A0 =C2=A0 =C2=A0do { } while (0)<br>
&gt; #endif<br>
&gt; <br>
&gt;+#define XEN_GRANT_ADDR_OFF (1ULL &lt;&lt; 63)<br>
&gt;+<br>
&gt; static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, in=
t i)<br>
&gt; {<br>
&gt;=C2=A0 =C2=A0 =C2=A0return shared_page-&gt;vcpu_ioreq[i].vp_eport;<br>
&gt;diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h<br>
&gt;index dc72f83bcb..19dccf4d71 100644<br>
&gt;--- a/include/sysemu/xen.h<br>
&gt;+++ b/include/sysemu/xen.h<br>
&gt;@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t si=
ze,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 s=
truct MemoryRegion *mr, Error **errp);<br>
&gt; <br>
&gt; bool xen_mr_is_memory(MemoryRegion *mr);<br>
&gt;+bool xen_mr_is_grants(MemoryRegion *mr);<br>
&gt; <br>
&gt; #else /* !CONFIG_XEN_IS_POSSIBLE */<br>
&gt; <br>
&gt;@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr=
)<br>
&gt;=C2=A0 =C2=A0 =C2=A0return false;<br>
&gt; }<br>
&gt; <br>
&gt;+static inline bool xen_mr_is_grants(MemoryRegion *mr)<br>
&gt;+{<br>
&gt;+=C2=A0 =C2=A0 g_assert_not_reached();<br>
&gt;+=C2=A0 =C2=A0 return false;<br>
&gt;+}<br>
&gt;+<br>
&gt; #endif /* CONFIG_XEN_IS_POSSIBLE */<br>
&gt; <br>
&gt; #endif<br>
&gt;-- <br>
&gt;2.40.1<br>
&gt;<br>
&gt;<br>
</blockquote></div></div>

--00000000000068759206191c7389--


From xen-devel-bounces@lists.xenproject.org Thu May 23 10:45:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 10:45:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728327.1133204 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5wc-0007nA-T0; Thu, 23 May 2024 10:45:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728327.1133204; Thu, 23 May 2024 10:45:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5wc-0007n3-Pj; Thu, 23 May 2024 10:45:10 +0000
Received: by outflank-mailman (input) for mailman id 728327;
 Thu, 23 May 2024 10:45:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Oh0r=M2=epam.com=prvs=18734d070d=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1sA5wa-0007mx-TM
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 10:45:08 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 83647f53-18f1-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 12:45:05 +0200 (CEST)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44N4j2vf024358;
 Thu, 23 May 2024 10:44:55 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3y9y23992h-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 May 2024 10:44:55 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by PR3PR03MB6586.eurprd03.prod.outlook.com (2603:10a6:102:77::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Thu, 23 May
 2024 10:44:50 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Thu, 23 May 2024
 10:44:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83647f53-18f1-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CiZUOwIoGOYrn6LrOXYgSf14BnG3YEdMFGVqWZmJdPDxQNYsf+H4t1LGq45cwcn73zxh+A7sAcCiqbEmj2NIs5iB+P6nvVsuucn6SRVeE6MKkb0x9TIRPCxPdhJFrKb6vXNnKzxO9TBT1SL7EypqtOonRI19Ys0n05a7FObPcIiJcp/BnFZqBCAVb69ys4gs/Co8vQKQ5J09Kbrmny4IQ3SvxmdUyUq9AFjngLgbeZe/ZgAQVtk17qxM1JKZdOV6fStVdRbhdfljjW46qOM78gD9rCIqr9X65lhu1A5THlDgepZirY+Wn4obg4iMKbr4QAqO1TaPZVFd2mZVksRW5w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kpx9T7B2+zG/EYxXkneyTPeQ0oLcp7fIPBnmPbxjrsc=;
 b=GnwhHQDfi8EgF1D+B/DFR4VbUlbjEk9TI8R6xDsv+0EhvZnTnj7IJB9s4cn6Na/+OUQLvVlkyiI8p7hkW0w3c0Lyqezk1iD8NyuO28LK5Yjzp/S17Hp95Zo3B+umr34Wo+e6La6aXON9WL2Mjjevu/Lz7GUBMiqwUffdDY5rLtv2hJQafibmT1XIoUfzBnBgqO+DEV97KWXxqkykVxDPnbVkyJLluc11XqDxbeFVPwJk8x3q2H2mGxaaFgyEF5AJho4XVnzhCT74D2jUeivGHC07NVCS6OD783R3079JjtbKOEJ9rGOc/rWIBvk0FSv4h7KHMlrdJzAIssxK4YZFtw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kpx9T7B2+zG/EYxXkneyTPeQ0oLcp7fIPBnmPbxjrsc=;
 b=RJ5LI1s7Hi/FmP7nzx1l8suIYWgBroOTKkVJhYQpnGuzBcQAPr7sn7+O4T/bImfj5mOeRtDT3r3v8nWkz6sfxucBBMuw60HsI65h3BpmWYMB787VsBcNe4uVpRB7zKHCRIGg/6Ayb8zuAL2dkHsUOr6lc/TewF+3HJC1pow4LCQFZrBgtn1iK65/4ebf3JDt7cgYmBYC4aWxJh6x6kHckZg5ixDxbYv6pt1Jr/tszwdvByjCjCeXh6u+urCS9Mkm/51/KFE7XR6Wa8GG34rpLB4KMfveWqJPTyPP5T4d2n0l2Stq38Crit8fSGlHe8n57wpF0U+n/7CvMjNCrU3bnw==
Message-ID: <71559cd3-c558-4f91-9c6c-227f502645d7@epam.com>
Date: Thu, 23 May 2024 13:44:48 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with
 CONFIG_ALTP2M option
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        George Dunlap <george.dunlap@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
        Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
 <29604726-8768-4b33-a8ef-eba6ad112d44@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <29604726-8768-4b33-a8ef-eba6ad112d44@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0021.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::21) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|PR3PR03MB6586:EE_
X-MS-Office365-Filtering-Correlation-Id: 0243c2d3-d0f9-4e5d-f055-08dc7b155f63
X-LD-Processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|366007|1800799015;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?V1NFZndEVE0wQVEwOVdFOEdHUE12czBUeHk0VU8xRkpBR0pZT0ZsalJyUmd1?=
 =?utf-8?B?N0hZMjRhR3RUQU5TNkVLWHEvdmRYamRYL1QrRFFQRVd5UzhIbUhDV0hmT2o3?=
 =?utf-8?B?elJRQkErK2FsU29HSk9ZaXFSWGtxTzVNQnlHQWJrQkVLaE03M3QzK1dvcGd1?=
 =?utf-8?B?WnBvZllRSkorWDhiaVJjZG96WUErRjdzd1oyb3YwYUp1VHlBSEtIYmlBY3RX?=
 =?utf-8?B?cTV5TmNSZGwyelN4OWZNbS9sZHE4czZrVTU4RDgzVVhkSnVoYkRQZjh5SkpY?=
 =?utf-8?B?aFM5VmUyZjFqV2RZM3FaREJmL0pLWGc1WE81TjJLbCtWNmpDWWxoUWxXclUw?=
 =?utf-8?B?aXFZL0xXdEJpcExad1ZJcnFvb0tHdFhCalpLUjg4dWhTVkRoc2lpWEVRdmJq?=
 =?utf-8?B?SzN3aTA4bjBDUGpwRGhTN0Uwb3RJd1Uzby9HU2RkSHBpMXdyTW5pTTNPZmpt?=
 =?utf-8?B?Y3puNkFiTmZJVmQ3cW5XMzk3bVdrSDgyVVJBa1hXdTVXN2pRNXFDR3pyVkpV?=
 =?utf-8?B?eXNtczJqbkNuaVVpeE9pVVE1RnVVa3dTNWtvQzkvblhCTUQ0M1o2dzNlQWZF?=
 =?utf-8?B?QStwaE1iT1FiWWNRd0dPYUdvOFhzcnZaSDBDOWQrM2lOZVdxV0JWQi9lckUx?=
 =?utf-8?B?K0VKRHJ4SURxMDJnNmdJaG9VMEEydzRJSG9wODgzSlRnSUlTVWZZWFJDVnNC?=
 =?utf-8?B?UXh4dmg1Q3lPYXFYTWFDTUdZTjFONnQrbUc1L3EzZDl4d3BXVHgyVmh2aHgx?=
 =?utf-8?B?Mkpad2dEaW94VE9IUlJvbEx5TUlMMzVhQmQ5dlNQWEFrcmRwMWUvKzYyNzFv?=
 =?utf-8?B?STMwbGp0K3IzNTR4bTdMTUZZakJ2bG1wcCs3SEFTV3lJZ1ZpNmc4SzBwTlRn?=
 =?utf-8?B?dm1aSnBpZXVZY05YcG1KUGszbVdCZ1NZM2l3UHl3WUZNUDBzZnErTW1IRFVw?=
 =?utf-8?B?cTY2WU11K21NNHZGQ1JTb29BSnJrN3d2aU03OUlGSElKeHlSc25GalhlWHhj?=
 =?utf-8?B?cm9veVlTV0pKQURpcGxGc2hORkNpaDYvVllRNHU2NTQ5dlg2R0VvUmloeExs?=
 =?utf-8?B?MUNYWjJqNEJTbmZDa3RueWNobUZnb2piZjNqMjFGVVdvaVFKUE04ZWxiK2Rz?=
 =?utf-8?B?UjlOL2FvL09CYlVZeGVUTGx3NEZVYU5jbFhDanJvdnNYcWduaEpSWlFxeXJ6?=
 =?utf-8?B?QWg3RHB2MkdNcmk1bGszOXpGb0p2RUg1YjhHUnVOQVN4OHJpNXQycEZtVko2?=
 =?utf-8?B?WURmaS96WG1kWEFEc1djZm5EbWw3djRIcmFyQ1EwNDNhbmM0RHRjbmRGUlBE?=
 =?utf-8?B?UTM3ZGtJWEhxWGV2NlV4bVhDWkVOS0NxeDdsRnZOVjJ2a3lrdWdKanNTVVRU?=
 =?utf-8?B?bGhuMXdiNTBJMGJUNm5HeHlKdkVqcHlYNW5hbHJ2QXk0NnFoUFJVSUppU256?=
 =?utf-8?B?VVNpcDFlTVBVTVQxaTA0UVpIblA2ZlFGeE1hY3p0ck9wNXI1MmxqYzJOR1ZP?=
 =?utf-8?B?WmxJRnc2WDFoUG40OFJ4Z2l4VVM1ejJXWERnMTJDaGxsRmFaYUtXOHNsZUlT?=
 =?utf-8?B?V3pSOGxndUYybDM0MktGSWhKNmNqcHE1OGlHTDkzQXJrMTIvR1dHSGhudXN6?=
 =?utf-8?B?MjY5dmpjQ1pBdVdncWlGeWdUOXRvSEhGSC9FbWJmQUNGN0pWWWplc29FTnhK?=
 =?utf-8?B?R3dLL0VIeXJzTHRQNVZRdXBFOVZKeVpkNldSRGhwVUMwRys0OTNqb2hRPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?QWpQWmgxRUhMbWl1NHl3RmxpRDMySDgvUll1Z0l1dURsVUZQcWwrUitScHp2?=
 =?utf-8?B?SUtPekVhOXh2QzRqQ0lNTnNnK2M0N2ZVR3R3d0t1bTdTT3F3NFpwRWFHenpK?=
 =?utf-8?B?dTJ2QXZ6S0cxYkZHcXBrRkZ5aUpFemtHcHdyR1lxVFZsSGxtZzNmbGdiQnhV?=
 =?utf-8?B?ejhBVDg2cGhiUVEwQWpGN3F0S2ZEa2JjaCt5dlp1MjRMc2JEL0c4TGFzQi9k?=
 =?utf-8?B?TGpseVQ3b1d1NHNhbEdzNTMrTFV0TXBCSzVWQTdkTkZvV01XK3hST2IzNVNr?=
 =?utf-8?B?M3QvQ0N2Mk12eVF1S0Z4Z0RqcXFpVy9mOHBlR1AzV1N3VTZRWHYyTE81L2Zs?=
 =?utf-8?B?K2tickwxbUZJakVEUTVRNzBTUEVYZEl0OVJxZTZqTXlWU0doSmYyOU10ME1z?=
 =?utf-8?B?YzJmMjUvUjZSbFlDUkFVVGpKaEZhaGc0QXhiRGtwVzFkSnZwb1VrY1hscWp2?=
 =?utf-8?B?dVFXSGlxT2d4UFhTOTJ3Yk5RaXNaZHdtV1hpVFk3dW5yck1uRWVpU0RWYzBG?=
 =?utf-8?B?RkxYS0pGSEJuZ1Eza1hKaDRQK3VEbnBYTFZFV0JUbG5GaU9mTitrTEJnTG5F?=
 =?utf-8?B?M2tXWCtobXpNMFJDU2JLY2YrYW81WGQrbWk3c2psb3JkYnp5cmQ4OWJ0eXZp?=
 =?utf-8?B?L3hVRjJ1aHBFYTJaTEhSeEFBWnpUb2lCYVd5NHNyZ1ZNeUZCMnF1OHZTMGM5?=
 =?utf-8?B?Q1Y4a3hOYytPTkRRbG5PbU4zMHl6eURKeWdWSXdEYllzek4wWld6UFNLR1JG?=
 =?utf-8?B?eHF1R0s3Y2g1d00vLysvKzNsRWRyc1E5dGhETHRpSEhCb1Z2bHA1bTc2L1pv?=
 =?utf-8?B?VUxLQjJpdkI1U3NvaGhmS3laZkVDdFBvL1ZvWkF5MzhYdGNMZnQxVm5oOVBS?=
 =?utf-8?B?eFg4Y2ZFcjFrV3pjMkgzd29jdTdYUUR3K2FOTCtRRCtaWm5qM3pEd0QwVzNt?=
 =?utf-8?B?SUE1dGZHenE4Sit2T0t3M2p0cElkMS92RHIvU3lQajlSOEFyKy91UXFkU21I?=
 =?utf-8?B?UVZiTXQ1UlczRDVKMGVVZ25PaXVBOURLMzVWdkUvMzhOM2l2aGQ4YUZSQTlR?=
 =?utf-8?B?VVpCL0VWb2FJUmtLeXRPNS9NTXQwNkRKUm9pZTRtTTRURHFId3IrcnBhUUYy?=
 =?utf-8?B?RFFSS1prNTJkb0p6d0tYbCt4T1NLaGV2TXAySy9BRExuS0twSWNuMzBIK05p?=
 =?utf-8?B?L2hXWWR6Vm5welNKZWxwNDFXei81RDNQTEZ4QWJPL2hWWGdvZWxlb2dDbytM?=
 =?utf-8?B?am1QYzVWWnFKUmJlc3ByMnRUV21mVzR5djNxU1NsMkFJS0UrcFhVSFdwQmVp?=
 =?utf-8?B?bVpxd011OExNUzBabnNUNzFieHlDdUJnUllubnVLWHhqb05yVENtQTVqTFNF?=
 =?utf-8?B?L3pHR1pSMFNGOGQ5SUEyOUlLOGZMNHcrell5aDlUaXNJOUNhWWVmT3ptNHBF?=
 =?utf-8?B?MnZuVUlFT2JDVW1NeWRkVWpEdmQ1ZU5OcnVwMzhjb2NucEVhMWpxSEhRcEZl?=
 =?utf-8?B?aFFPRjNtUERxYzlpRDdWRVA2T0VZcUIvRGRWWjljcXJyMTBNS2t3TkxIL0Fn?=
 =?utf-8?B?dmVranhWenQveWxlNVl5WTJlek1EZFdTQzd4bFB0bkd6ZGJCN3MrUWs4azcw?=
 =?utf-8?B?MjdZdmRSN01jREdOVHMxVllSOVBOa1VEbm9FN0R4a3BreHRlVVhNNkJPdk50?=
 =?utf-8?B?V3ZIWmlDYXBPYXlGRlpWR0hvbXJPc2psZVVlcGZOVHlzYVFVcUJaTzlsTzYz?=
 =?utf-8?B?cEZiVHZKV0dYYmN1MWRCOG9pVTZDbENBcWs3MmV0Yzh0ZTVuUkZPaWR4VEx5?=
 =?utf-8?B?MGIyTS80NlRJQmtmZVVWZ0YvZnQrMVJUQWlRbFJhUFhsRnRIOG1iRTZmelY4?=
 =?utf-8?B?Q201MVFRNml0S25KTVUwdWlMWkhmVCsxZ0t3WHJwWkk0U3VVWGtxanpHNXd4?=
 =?utf-8?B?NzRXZW5kRmwvZWhoY2Z2MWVmS0V6aEIzRnN5MnpaK2FHS0t4MnBLaGhMRUxk?=
 =?utf-8?B?QTdSeEZPYkJqdEQvMjc5UlhDcHpSTHRESDd5dlZrT0g2SDdQMzBPUTV6OEY5?=
 =?utf-8?B?bGxMNlhGRndTaFlmektROHVpOE5xNzVxOEhpTlVEQmkzcVdPUDROdEtLdVYz?=
 =?utf-8?B?WmdYRUsrSzd6Q2Y2bEJFeG9GaDRFRXNnZ1BHemp6cndyb0gwWk5CSHZMbXh5?=
 =?utf-8?B?enc9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0243c2d3-d0f9-4e5d-f055-08dc7b155f63
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 10:44:50.6032
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bOIKGDpzc+VzjAaklzy5i2zE4YCyfFkL0e5/fhgwEmJ0vn77O5nvvulJ6h7Y1hf9bMGlKpTVb8Jzft+rkcv38A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR03MB6586
X-Proofpoint-ORIG-GUID: ZXXXPLTu2_ow3eO4jBR6T21HEsD97b6F
X-Proofpoint-GUID: ZXXXPLTu2_ow3eO4jBR6T21HEsD97b6F
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-23_05,2024-05-23_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 bulkscore=0 spamscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0
 impostorscore=0 malwarescore=0 phishscore=0 suspectscore=0 mlxlogscore=999
 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405010000 definitions=main-2405230072

16.05.24 14:01, Jan Beulich:
> On 15.05.2024 11:10, Sergiy Kibrik wrote:
>> @@ -38,7 +38,10 @@ static inline bool altp2m_active(const struct domain *d)
>>   }
>>   
>>   /* Only declaration is needed. DCE will optimise it out when linking. */
>> +void altp2m_vcpu_initialise(struct vcpu *v);
>> +void altp2m_vcpu_destroy(struct vcpu *v);
>>   uint16_t altp2m_vcpu_idx(const struct vcpu *v);
>> +int altp2m_vcpu_enable_ve(struct vcpu *v, gfn_t gfn);
>>   void altp2m_vcpu_disable_ve(struct vcpu *v);
> 
> These additions look unrelated, as long as the description says nothing in
> this regard.

agree, I'll update description on why these declarations are added

> 
>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>> @@ -670,7 +670,7 @@ static inline bool hvm_hap_supported(void)
>>   /* returns true if hardware supports alternate p2m's */
>>   static inline bool hvm_altp2m_supported(void)
>>   {
>> -    return hvm_funcs.caps.altp2m;
>> +    return IS_ENABLED(CONFIG_ALTP2M) && hvm_funcs.caps.altp2m;
> 
> Which in turn raises the question whether the altp2m struct field shouldn't
> become conditional upon CONFIG_ALTP2M too (or rather: instead, as the change
> here then would need to be done differently). Yet maybe that would entail
> further changes elsewhere, so may well better be left for later.
> 

  but hvm_funcs.caps.altp2m is only a capability bit -- is it worth to 
become conditional?

>> --- a/xen/arch/x86/mm/Makefile
>> +++ b/xen/arch/x86/mm/Makefile
>> @@ -1,7 +1,7 @@
>>   obj-y += shadow/
>>   obj-$(CONFIG_HVM) += hap/
>>   
>> -obj-$(CONFIG_HVM) += altp2m.o
>> +obj-$(CONFIG_ALTP2M) += altp2m.o
> 
> This change I think wants to move to patch 5.
> 

If this moves to patch 5 then HVM=y && ALTP2M=n configuration 
combination will break the build in between patch 5 and 6, so I've 
decided to put it together with fixes of these build failures in patch 6.
Maybe I can merge patch 5 & 6 together then ?

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Thu May 23 10:48:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 10:48:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728332.1133215 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5zH-0008M7-9z; Thu, 23 May 2024 10:47:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728332.1133215; Thu, 23 May 2024 10:47:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA5zH-0008M0-5w; Thu, 23 May 2024 10:47:55 +0000
Received: by outflank-mailman (input) for mailman id 728332;
 Thu, 23 May 2024 10:47:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA5zG-0008Lu-BS
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 10:47:54 +0000
Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com
 [2607:f8b0:4864:20::f2e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e798a6b4-18f1-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 12:47:53 +0200 (CEST)
Received: by mail-qv1-xf2e.google.com with SMTP id
 6a1803df08f44-6a919a4bb83so22973846d6.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 03:47:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1cd271sm141335426d6.101.2024.05.23.03.47.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 03:47:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e798a6b4-18f1-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716461272; x=1717066072; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=shb6PaAVO27bplQev1XIlZhGAu2ssNDgQXzls8G3qoQ=;
        b=tye05u/X0kyG3pszIXxgaxFbZFxsQ5gzRSZh1B2Hko8+uzcGJBDV/Qwv7900vgG5+T
         DrVtNNG/E638jQF2ZaeEHWlJhmygMl+KTM8xdmm3Wm8bmnhX5vgzV0NAd3macBx7UBLD
         gYWsiTbw5qH7yUhgxsbU4jn0eY8lQ5Nj3Kli8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716461272; x=1717066072;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=shb6PaAVO27bplQev1XIlZhGAu2ssNDgQXzls8G3qoQ=;
        b=jBiR5sA/5UkfdRLk6Z5NBgxS/ORN1BN3jySigJm3iHdAOh5CQkFNpIdAfS0ztXKcDF
         o1i/wwE8ETjpOhIlKExrLZW4aANcVJuZRYQmuJUoCj1lCXvW73y3A0UOXtZymXeoujO3
         98M0132lTDfKfvwF0OI0lGoGJbzw6E00ofzbH6dfVVgkVsyZEMg/mHm062pHjRwbTuUS
         ixwJpbv9LNpGvPA5VAkrkht1wzytGluAJkFYGJ1QfMblpuD/JP0BnD+5/BUIcoe5EQQY
         uB4LZkUmCOdFv2rl0Wuh05/wdwIDjvtEcJnd9rPDxqIsbxIK119mQxSjaoFCIyk5ZvYj
         wB2w==
X-Gm-Message-State: AOJu0YxzQySdBm53Rm7QQspwTJDzMso4rt24TW4GIPWANq3vWz3uLc11
	gRHQTsH2/jlCiYmfdfDH/XVlV6HWk/MQvOBTTsUw3mloVEnbcqd5d8Hlo5e8jqc=
X-Google-Smtp-Source: AGHT+IFWX7jzjcIJOm79SekjElAK7c6SOlxUoFlweKpFZuMzyUEqx6xsz8fLa9FsUWl+MvBOCuGeLQ==
X-Received: by 2002:a05:6214:459d:b0:6a0:a7a9:4988 with SMTP id 6a1803df08f44-6ab7f33865fmr51063316d6.13.1716461271570;
        Thu, 23 May 2024 03:47:51 -0700 (PDT)
Date: Thu, 23 May 2024 12:47:49 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 2/2] tools/xg: Clean up xend-style overrides for CPU
 policies
Message-ID: <Zk8e1RabTi0YXCgW@macbook>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
 <ae180fd606cb9a6aa0108080fe8cf7a6496a4ce3.1716457040.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ae180fd606cb9a6aa0108080fe8cf7a6496a4ce3.1716457040.git.alejandro.vallejo@cloud.com>

On Thu, May 23, 2024 at 10:41:30AM +0100, Alejandro Vallejo wrote:
> Factor out policy getters/setters from both (CPUID and MSR) policy override
> functions. Additionally, use host policy rather than featureset when
> preparing the cur policy, saving one hypercall and several lines of
> boilerplate.
> 
> No functional change intended.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v3:
>   * Restored overscoped loop indices
>   * Split long line in conditional
> ---
>  tools/libs/guest/xg_cpuid_x86.c | 438 ++++++++++----------------------
>  1 file changed, 131 insertions(+), 307 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4f4b86b59470..1e631fd46d2f 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -36,6 +36,34 @@ enum {
>  #define bitmaskof(idx)      (1u << ((idx) & 31))
>  #define featureword_of(idx) ((idx) >> 5)
>  
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
> +{
> +    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> +    int rc;
> +
> +    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
> +    if ( rc )
> +    {
> +        if ( err_leaf != -1 )
> +            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> +                  err_leaf, err_subleaf, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> +                                  policy->nr_msrs, &err_msr);
> +    if ( rc )
> +    {
> +        if ( err_msr != -1 )
> +            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
> +                  err_msr, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
>  int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
>  {
>      struct xen_sysctl sysctl = {};
> @@ -260,102 +288,37 @@ static int compare_leaves(const void *l, const void *r)
>      return 0;
>  }
>  
> -static xen_cpuid_leaf_t *find_leaf(
> -    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
> -    const struct xc_xend_cpuid *xend)
> +static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
> +                                   const struct xc_xend_cpuid *xend)
>  {
>      const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
>  
> -    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
> +    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),

Don't you need to use p->nr_leaves here, as otherwise we could check
against possibly uninitialized leaves (or leaves with stale data)?

> +                   sizeof(*p->leaves), compare_leaves);
>  }
>  
> -static int xc_cpuid_xend_policy(
> -    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
> +static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
> +                                const struct xc_xend_cpuid *xend,
> +                                xc_cpu_policy_t *host,
> +                                xc_cpu_policy_t *def,
> +                                xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    rc = -ENOMEM;
> -    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        goto fail;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_msrs = 0;
> -    nr_cur = nr_leaves;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> +    if ( !xend )
> +        return 0;
>  
> -    /* Get the domain type's default policy. */
> -    nr_msrs = 0;
> -    nr_def = nr_leaves;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_def, def, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto fail;
> -    }
> +    if ( !host || !def || !cur )
> +        return -EINVAL;
>  
> -    /* Get the host policy. */
> -    nr_msrs = 0;
> -    nr_host = nr_leaves;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_host, host, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    rc = -EINVAL;
>      for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
>      {
> -        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
> -        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
> -        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
> +        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
> +        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
> +        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
>  
>          if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
>          {
>              ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
> -            goto fail;
> +            return -EINVAL;
>          }
>  
>          for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
> @@ -384,7 +347,7 @@ static int xc_cpuid_xend_policy(
>                  {
>                      ERROR("Bad character '%c' in policy[%d] string '%s'",
>                            xend->policy[i][j], i, xend->policy[i]);
> -                    goto fail;
> +                    return -EINVAL;
>                  }
>  
>                  clear_bit(31 - j, cur_reg);
> @@ -394,25 +357,7 @@ static int xc_cpuid_xend_policy(
>          }
>      }
>  
> -    /* Feed the transformed currrent policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    /* Success! */
> -
> - fail:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  static int compare_msr(const void *l, const void *r)
> @@ -426,104 +371,38 @@ static int compare_msr(const void *l, const void *r)
>      return lhs->idx < rhs->idx ? -1 : 1;
>  }
>  
> -static xen_msr_entry_t *find_msr(
> -    xen_msr_entry_t *msrs, unsigned int nr_msrs,
> -    uint32_t index)
> +static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
> +                                 uint32_t index)
>  {
>      const xen_msr_entry_t key = { .idx = index };
>  
> -    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
> +    return bsearch(&key, p->msrs, ARRAY_SIZE(p->msrs),

Same comment here regarding the usage of ARRAY_SIZE instead of
p->nr_msrs.

> +                   sizeof(*p->msrs), compare_msr);
>  }
>  
> -
>  static int xc_msr_policy(xc_interface *xch, domid_t domid,
> -                         const struct xc_msr *msr)
> +                         const struct xc_msr *msr,
> +                         xc_cpu_policy_t *host,
> +                         xc_cpu_policy_t *def,
> +                         xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        rc = -ENOMEM;
> -        goto out;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_leaves = 0;
> -    nr_cur = nr_msrs;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Get the domain type's default policy. */
> -    nr_leaves = 0;
> -    nr_def = nr_msrs;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, NULL, &nr_def, def);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto out;
> -    }
> +    if ( !msr )
> +        return 0;
>  
> -    /* Get the host policy. */
> -    nr_leaves = 0;
> -    nr_host = nr_msrs;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_leaves, NULL, &nr_host, host);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto out;
> -    }
> +    if ( !host || !def || !cur )
> +        return -EINVAL;
>  
>      for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
>      {
> -        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
> -        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
> -        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
>          unsigned int i;
> +        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
> +        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
> +        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
>  
>          if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
>          {
>              ERROR("Missing MSR %#x", msr->index);
> -            rc = -ENOENT;
> -            goto out;
> +            return -ENOENT;
>          }
>  
>          for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
> @@ -542,8 +421,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>              {
>                  ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
>                        msr->index, msr->policy[i], msr->policy);
> -                rc = -EINVAL;
> -                goto out;
> +                return -EINVAL;
>              }
>  
>              if ( val )
> @@ -553,25 +431,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>          }
>      }
>  
> -    /* Feed the transformed policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Success! */
> -
> - out:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
> @@ -583,14 +443,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      int rc;
>      bool hvm;
>      xc_domaininfo_t di;
> -    struct xc_cpu_policy *p = xc_cpu_policy_init();
> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
> -    uint32_t len = ARRAY_SIZE(host_featureset);
> +    unsigned int i;
>  
> -    if ( !p )
> -        return -ENOMEM;
> +    struct xc_cpu_policy *host = xc_cpu_policy_init();
> +    struct xc_cpu_policy *def = xc_cpu_policy_init();
> +    struct xc_cpu_policy *cur = xc_cpu_policy_init();
> +
> +    if ( !host || !def || !cur )
> +    {
> +        rc = -ENOMEM;
> +        goto out;
> +    }
>  
>      if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
>      {
> @@ -600,21 +463,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      }
>      hvm = di.flags & XEN_DOMINF_hvm_guest;
>  
> -    /* Get the host policy. */
> -    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
> -                               &len, host_featureset);
> -    /* Tolerate "buffer too small", as we've got the bits we need. */
> -    if ( rc && errno != ENOBUFS )
> +    /* Get the raw host policy */
> +    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
> +    if ( rc )
>      {
> -        PERROR("Failed to obtain host featureset");
> +        PERROR("Failed to obtain host policy");
>          rc = -errno;
>          goto out;
>      }
>  
>      /* Get the domain's default policy. */
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, p->leaves, &nr_msrs, NULL);
> +    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> +                                           : XEN_SYSCTL_cpu_policy_pv_default,
> +                                  def);
>      if ( rc )
>      {
>          PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
> @@ -622,14 +483,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>          goto out;
>      }
>  
> -    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
> -                                    &err_leaf, &err_subleaf);
> -    if ( rc )
> -    {
> -        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> -              err_leaf, err_subleaf, -rc, strerror(-rc));
> -        goto out;
> -    }
> +    /* Copy the deserialised default policy to modify it */
> +    memcpy(cur, def, sizeof(*cur));
>  
>      if ( restore )
>      {
> @@ -647,18 +502,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * - Re-enable features which have become (possibly) off by default.
>           */
>  
> -        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
> -        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
> -        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
> +        cur->policy.basic.rdrand = host->policy.basic.rdrand;
> +        cur->policy.feat.hle = host->policy.feat.hle;
> +        cur->policy.feat.rtm = host->policy.feat.rtm;
>  
>          if ( hvm )
>          {
> -            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
> +            cur->policy.feat.mpx = host->policy.feat.mpx;
>          }
>  
> -        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
> -        p->policy.feat.max_subleaf = 0;
> -        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
> +        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
> +        cur->policy.feat.max_subleaf = 0;
> +        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
>      }
>  
>      if ( featureset )
> @@ -702,17 +557,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>              }
>          }
>  
> -        x86_cpu_featureset_to_policy(feat, &p->policy);
> +        x86_cpu_featureset_to_policy(feat, &cur->policy);
>      }
>      else
>      {
> -        p->policy.extd.itsc = itsc;
> +        cur->policy.extd.itsc = itsc;
>  
>          if ( hvm )
>          {
> -            p->policy.basic.pae = pae;
> -            p->policy.basic.vmx = nested_virt;
> -            p->policy.extd.svm = nested_virt;
> +            cur->policy.basic.pae = pae;
> +            cur->policy.basic.vmx = nested_virt;
> +            cur->policy.extd.svm = nested_virt;
>          }
>      }
>  
> @@ -722,8 +577,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * On hardware without CPUID Faulting, PV guests see real topology.
>           * As a consequence, they also need to see the host htt/cmp fields.
>           */
> -        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
> -        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
> +        cur->policy.basic.htt       = host->policy.basic.htt;
> +        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
>      }
>      else
>      {
> @@ -731,28 +586,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>           * Topology for HVM guests is entirely controlled by Xen.  For now, we
>           * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
>           */
> -        p->policy.basic.htt = true;
> -        p->policy.extd.cmp_legacy = false;
> +        cur->policy.basic.htt = true;
> +        cur->policy.extd.cmp_legacy = false;
>  
>          /*
>           * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
>           * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
>           * overflow.
>           */
> -        if ( !p->policy.basic.lppp )
> -            p->policy.basic.lppp = 2;
> -        else if ( !(p->policy.basic.lppp & 0x80) )
> -            p->policy.basic.lppp *= 2;
> +        if ( !cur->policy.basic.lppp )
> +            cur->policy.basic.lppp = 2;
> +        else if ( !(cur->policy.basic.lppp & 0x80) )
> +            cur->policy.basic.lppp *= 2;
>  
> -        switch ( p->policy.x86_vendor )
> +        switch ( cur->policy.x86_vendor )
>          {
>          case X86_VENDOR_INTEL:
> -            for ( i = 0; (p->policy.cache.subleaf[i].type &&
> -                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
> +            for ( i = 0; (cur->policy.cache.subleaf[i].type &&
> +                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )

Nit: indentation is weird here.  I would use:

            for ( i = 0; cur->policy.cache.subleaf[i].type &&
                         i < ARRAY_SIZE(cur->policy.cache.raw); ++i )

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 11:01:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:01:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728342.1133224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6C7-00036P-Hm; Thu, 23 May 2024 11:01:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728342.1133224; Thu, 23 May 2024 11:01:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6C7-00036I-FC; Thu, 23 May 2024 11:01:11 +0000
Received: by outflank-mailman (input) for mailman id 728342;
 Thu, 23 May 2024 11:01:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jOX+=M2=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1sA6C6-00036C-ON
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:01:10 +0000
Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com
 [2607:f8b0:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c24037ef-18f3-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 13:01:09 +0200 (CEST)
Received: by mail-pg1-x52d.google.com with SMTP id
 41be03b00d2f7-657a07878easo2008340a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:01:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c24037ef-18f3-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716462068; x=1717066868; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=91U11pUevJarFFJNYMAR2AcdGvDfJkQNRP9NpdObBVA=;
        b=F5vR/AUognyK2RXfKWuN4eyFi0Sw+mC/Z+mOkPMpT9iUs2yFTHusUVuyQzeHD0LX8l
         djsQK6pqOkw5OCAJZaDh5FKuqVxWb3jR2mq6tbAmnxhLRdc/dzxrTFCZez1D9aDoqpv1
         32Yr5YWiW9Fn9ZTKxi+0sXg47ivGb1bK1dYCc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716462068; x=1717066868;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=91U11pUevJarFFJNYMAR2AcdGvDfJkQNRP9NpdObBVA=;
        b=CEqBS4Az6Ira1rbY4vU47o+8e1xF/J+ViG6/TnYK+DKQC8Yii0/6N7KRaTEhzfVcQH
         ks+U3p1zzXEem2WPqkCl3wZ1E25KUxzTa+LmDNtGDKbmaa1Ln4ZiIS8MW3uJC6rkgCTw
         nXWnfN/HCjqTp1NDrKLrgtf9vz1igKt4N3eGWjY/bDcHcqVd/FuqjWqHrZhAKuKmMGc8
         ZCI/zN+bps9JTTkE0X3ewzWidJQl5KMRwGVwgOp/8aQCQ1FIhOn5XfDzUbc0p0tbecM6
         MrIshulqQOY5tS2KBTOWrSQKV1S8tsNfHyuUGc120htWF+64gU3fxCRLu6xLTi9E9mT4
         rqQA==
X-Forwarded-Encrypted: i=1; AJvYcCUHd+E0eLB3zuQ31PJqK5I6Kt32dn01DUWW9Dflra3BmE3IH0nkfUblYNOTuMu/SFxQcUYp8YG2h837il8MWBYBS8h+x8BOLpGAdIzEOOY=
X-Gm-Message-State: AOJu0YwUqvrLanD46Fd9He5g2YItJBel3leDlPyoW3kRMwI1spJh2xLh
	C6XVMEg13FfMeGkQulJhFhYXCRlrhc5R0XuH3tXSMz9fmkGgiUJUZAjhcK/YvdB2O0yXBSt7qD7
	aJd0LCAjhLGwB+EfqqSRvnLEfB5N/lE1sB5kw
X-Google-Smtp-Source: AGHT+IGvuyeyuzWjiUToY5J16n8njwnJeOg1DbjbIwk+JKoOUqhMoTuOFFBMO1x/lFU8L5/TrbpAdDsGw6w6Iw9iSQY=
X-Received: by 2002:a17:90b:1003:b0:2b3:79ad:856d with SMTP id
 98e67ed59e1d1-2bd9f5d6291mr5469546a91.48.1716462067785; Thu, 23 May 2024
 04:01:07 -0700 (PDT)
MIME-Version: 1.0
References: <20240404140833.1557953-1-ross.lagerwall@citrix.com>
 <90f07634-1600-4654-8e36-7ac9e2f457e8@perard> <CAG7k0EqJvVZJUNaf31Jd1ZfwvOon8saNbtHofon=VvB6gpj9qw@mail.gmail.com>
In-Reply-To: <CAG7k0EqJvVZJUNaf31Jd1ZfwvOon8saNbtHofon=VvB6gpj9qw@mail.gmail.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Thu, 23 May 2024 12:00:56 +0100
Message-ID: <CAG7k0Eo29eoZc+vgH+uDYe=YKiHM0a3jm6hoW5j+Wyc5kEh8tg@mail.gmail.com>
Subject: Re: [PATCH] xen-hvm: Avoid livelock while handling buffered ioreqs
To: Anthony PERARD <anthony.perard@cloud.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org, 
	qemu-devel@nongnu.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Apr 9, 2024 at 3:19=E2=80=AFPM Ross Lagerwall <ross.lagerwall@citri=
x.com> wrote:
>
> On Tue, Apr 9, 2024 at 11:20=E2=80=AFAM Anthony PERARD <anthony.perard@cl=
oud.com> wrote:
> >
> > On Thu, Apr 04, 2024 at 03:08:33PM +0100, Ross Lagerwall wrote:
> > > diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> > > index 1627da739822..1116b3978938 100644
> > > --- a/hw/xen/xen-hvm-common.c
> > > +++ b/hw/xen/xen-hvm-common.c
> > > @@ -521,22 +521,30 @@ static bool handle_buffered_iopage(XenIOState *=
state)
> > [...]
> > >
> > >  static void handle_buffered_io(void *opaque)
> > >  {
> > > +    unsigned int handled;
> > >      XenIOState *state =3D opaque;
> > >
> > > -    if (handle_buffered_iopage(state)) {
> > > +    handled =3D handle_buffered_iopage(state);
> > > +    if (handled >=3D IOREQ_BUFFER_SLOT_NUM) {
> > > +        /* We handled a full page of ioreqs. Schedule a timer to con=
tinue
> > > +         * processing while giving other stuff a chance to run.
> > > +         */
> >
> > ./scripts/checkpatch.pl report a style issue here:
> >     WARNING: Block comments use a leading /* on a separate line
> >
> > I can try to remember to fix that on commit.
>
> I copied the comment style from a few lines above but I guess it was
> wrong.
>
> >
> > >          timer_mod(state->buffered_io_timer,
> > > -                BUFFER_IO_MAX_DELAY + qemu_clock_get_ms(QEMU_CLOCK_R=
EALTIME));
> > > -    } else {
> > > +                qemu_clock_get_ms(QEMU_CLOCK_REALTIME));
> > > +    } else if (handled =3D=3D 0) {
> >
> > Just curious, why did you check for `handled =3D=3D 0` here instead of
> > `handled !=3D 0`? That would have avoided to invert the last 2 cases, a=
nd
> > the patch would just have introduce a new case without changing the
> > order of the existing ones. But not that important I guess.
> >
>
> In general I try to use conditionals with the least amount of negation
> since I think it is easier to read. I can change it if you would prefer?

It looks like this hasn't been committed anywhere. Were you expecting
an updated version with the style issue fixed or has it fallen through
the cracks?

Thanks,
Ross


From xen-devel-bounces@lists.xenproject.org Thu May 23 11:06:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:06:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728348.1133233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6H9-0003gi-3T; Thu, 23 May 2024 11:06:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728348.1133233; Thu, 23 May 2024 11:06:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6H9-0003gb-0w; Thu, 23 May 2024 11:06:23 +0000
Received: by outflank-mailman (input) for mailman id 728348;
 Thu, 23 May 2024 11:06:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA6H8-0003gR-6Y; Thu, 23 May 2024 11:06:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA6H8-00046Y-3n; Thu, 23 May 2024 11:06:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA6H7-0007Gm-Sc; Thu, 23 May 2024 11:06:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA6H7-00023E-S0; Thu, 23 May 2024 11:06:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zwPpKFWaGmQXYDMeRj5Wg64ZA8pmCf6LL49YOpX/YwU=; b=RDCpEv5Plkr1/Mx5gceLEXyruk
	uPy18AoH6OVQ/7S2fOc/HnoZYtQb5hBTjLLEZqWNbdwj6dkT5yyrvq/7sq3VJyz381e4MoWiDZSbM
	mNWqLDI+1ehz398eDvJr13HrXApgAirOcDzHeAP2nAA/U6+49aX8mMk60rAIyjw8/Uts=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186104-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186104: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d6a7fd83039af36c28bd0ae2174f12c3888ce993
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 11:06:21 +0000

flight 186104 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186104/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186064

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d6a7fd83039af36c28bd0ae2174f12c3888ce993
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186064  2024-05-21 15:04:02 Z    1 days
Testing same since   186104  2024-05-23 09:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Bobby Eshleman <bobbyeshleman@gmail.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 387 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 11:14:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:14:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728360.1133247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6PD-0005WK-Tb; Thu, 23 May 2024 11:14:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728360.1133247; Thu, 23 May 2024 11:14:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6PD-0005WD-Q1; Thu, 23 May 2024 11:14:43 +0000
Received: by outflank-mailman (input) for mailman id 728360;
 Thu, 23 May 2024 11:14:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA6PC-0005W7-Pm
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:14:42 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a6c82484-18f5-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 13:14:41 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so3425843a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:14:41 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57842fec9edsm1219709a12.43.2024.05.23.04.14.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 04:14:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6c82484-18f5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716462881; x=1717067681; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KLJpqS0EFUKkRXVB1cT+nXd/u/klamMtJaCc9IVI1lI=;
        b=YCMqi0A/iWx41JrgOhlr73Oy0cgZwZpJHryjnNhbq9EcS7Keroww0pr3voQXSnmcUU
         yQ3F301xFTGzk0cDEWzK4VamaGz3T5O6nSl1EsuwdpQeyhX+eDFZuywbyxtJlEWRWdw/
         IgP3blbeXY7tNqac96WNFJ/BhiGRsTgDKf+lNCdqrrh0fFG/mPFKFxoZzLrmSTbj92sJ
         Pe0ovZRIMXUiYMj4DKuWx09kwf9rvXz9ucW5yHXSe7g+o0+9edF8RKONY5kTfAFlhHqw
         62v63Kp+N0+6rElHpkgZ7UMZs7ZDLzD0yCpd4OdZJR0yk7t2CSAwStVpkpcAweN642b3
         IDCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716462881; x=1717067681;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KLJpqS0EFUKkRXVB1cT+nXd/u/klamMtJaCc9IVI1lI=;
        b=JYZmqCGiwVfkjbWpbax5zr/0u2tmNvcrSzjy5f0un5RTc0ijGC3gI+OsGQzxkxUEcJ
         VD23ymdBlYFR8vsAFvC2f3OLDr+2+NftfN2ZUcpRTpozT+qEBhjaV3X+Mj0X0B3LT545
         Y2T31z09YSHYXhfkvyukv+Ez16SwViCCpzbaLU1MateuTV5G3vkQyRQQwNo3LQB1d1nO
         D/qgD6xjIgxEpCjqHBdPSHh4NKhCYZychRPWsCOlsWl+g2dOkSmx28IWRbQ1dfo803Wk
         HtblzVvAYa27AnSklKEqGx9iXzlmXFuuwu6JheUWIPsbTjmER1yqH2d1Nog1Ag2410OP
         wIpw==
X-Gm-Message-State: AOJu0YyB3Ry0n6QMDxPel0gWZJ0w1x23l73Sj+vCiJqOmcXj1WkdOxiw
	jbAqFL5q0HwRfOKJTSJVEx+OynahTFmWEb1RdUPcZwMVXNh5wicXfh3ohVUjmx8=
X-Google-Smtp-Source: AGHT+IHqN2XZTEwnKPApkGhAohBfim6j8tShT33YK8omVdaG/SeSliXh4y0qU68y1sT3JRzGAZohjQ==
X-Received: by 2002:a50:9e2e:0:b0:574:ebf4:f786 with SMTP id 4fb4d7f45d1cf-57843ef8e9dmr1459296a12.16.1716462881184;
        Thu, 23 May 2024 04:14:41 -0700 (PDT)
Message-ID: <9cddfb9d-0627-4f00-b027-4cb1bbdfad4f@suse.com>
Date: Thu, 23 May 2024 13:14:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/5] x86/pvh: Set phys_base when calling xen_prepare_pvh()
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-4-jason.andryuk@amd.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240410194850.39994-4-jason.andryuk@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 10.04.24 21:48, Jason Andryuk wrote:
> phys_base needs to be set for __pa() to work in xen_pvh_init() when
> finding the hypercall page.  Set it before calling into
> xen_prepare_pvh(), which calls xen_pvh_init().  Clear it afterward to
> avoid __startup_64() adding to it and creating an incorrect value.
> 
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
> ---
> Instead of setting and clearing phys_base, a dedicated variable could be
> used just for the hypercall page.  Having phys_base set properly may
> avoid further issues if the use of phys_base or __pa() grows.
> ---
>   arch/x86/platform/pvh/head.S | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S
> index bb1e582e32b1..c08d08d8cc92 100644
> --- a/arch/x86/platform/pvh/head.S
> +++ b/arch/x86/platform/pvh/head.S
> @@ -125,7 +125,17 @@ SYM_CODE_START_LOCAL(pvh_start_xen)
>   	xor %edx, %edx
>   	wrmsr
>   
> +	/* Calculate load offset from LOAD_PHYSICAL_ADDR and store in
> +	 * phys_base.  __pa() needs phys_base set to calculate the
> +	 * hypercall page in xen_pvh_init(). */

Please use the correct style for multi-line comments:

	/*
	 * comment lines
	 * comment lines
	 */

> +	movq %rbp, %rbx
> +	subq $LOAD_PHYSICAL_ADDR, %rbx
> +	movq %rbx, phys_base(%rip)
>   	call xen_prepare_pvh
> +	/* Clear phys_base.  __startup_64 will *add* to its value,
> +	 * so reset to 0. */

Comment style again.

> +	xor  %rbx, %rbx
> +	movq %rbx, phys_base(%rip)
>   
>   	/* startup_64 expects boot_params in %rsi. */
>   	lea rva(pvh_bootparams)(%ebp), %rsi

With above fixed:

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen


From xen-devel-bounces@lists.xenproject.org Thu May 23 11:16:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:16:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728364.1133257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6R8-000636-8Z; Thu, 23 May 2024 11:16:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728364.1133257; Thu, 23 May 2024 11:16:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6R8-00062z-4s; Thu, 23 May 2024 11:16:42 +0000
Received: by outflank-mailman (input) for mailman id 728364;
 Thu, 23 May 2024 11:16:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6R6-00062t-J0
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:40 +0000
Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com
 [2607:f8b0:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ecb122b8-18f5-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 13:16:39 +0200 (CEST)
Received: by mail-ot1-x335.google.com with SMTP id
 46e09a7af769-6f12ff2da3fso2684885a34.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:39 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecb122b8-18f5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716462997; x=1717067797; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=io6NKi/HuBeU1oPKPl4oPHpp6HTYVxQANok6ys1P3tY=;
        b=ur0Go7nO/n1f/t8y2Dhmfq13i+hftaJm4rU2c25J7jWVQAvOlue96zHB8MbkaSGqoG
         XVwDR0ch+zEwmBKPOA2MGNBWkT88HEB5HZw42VONWqqfvMmsnbssy5pqbDAAIWpwneHc
         EzwHwV25alwY6LqQ1x57WqfyvOZy6HuDo+C4g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716462997; x=1717067797;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=io6NKi/HuBeU1oPKPl4oPHpp6HTYVxQANok6ys1P3tY=;
        b=P528VAFPs/snnrs1xWzfaa2eCSfruTe02Bl3uBMXrnOToN0DgRdhUpLAhuOEN0JIuq
         4JmcCQClkEciuEFMZS584XiAt2A9MjXk8vssKLLcT39pL28zwjvJu2XgcKSpdZ0ZOPGW
         HHLOxaLIdkbS2Emx03pZNrT7JssFozLaklPJcFtkOt4OYHr32MfMh/myK8lNAl8QE2Sv
         a03QqrbikOkT/CcRUKPgDmpnV67jlYXdHmr45VXEZT4yqkLrBwCEcRG5omL9fv5T/vAm
         kORo1EbCh3uU0kVDBfnQ4/rzCZe2QwhzWo/+vJoKU7Tfdk40zQzs6z5B5ypbt+N1g2uu
         P4+A==
X-Gm-Message-State: AOJu0YxcpY72mSgJaBE1CaRPOhgdKWS67g2DdXlx96cq9gGfAE5pswkN
	9MaoDaPxGJBzZ/vuuTGVHn4QUqR51kiJgwTNUgP11tsB4VWtD0qTgndgyNRqw81dbnYOce4t543
	ll/E=
X-Google-Smtp-Source: AGHT+IFEpJLgLx9LJqjO15FS64H8Szi7raczWajXLknNr9EAY6IlUsgNwKl/yarlIrlfSZV2UnwJqQ==
X-Received: by 2002:a05:6830:11ce:b0:6f0:e381:77b1 with SMTP id 46e09a7af769-6f668e68a28mr5008281a34.27.1716462996976;
        Thu, 23 May 2024 04:16:36 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 v3 0/7] x86/xstate: Fixes to size calculations
Date: Thu, 23 May 2024 12:16:20 +0100
Message-Id: <20240523111627.28896-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This has grown somewhat from v2, but is better for it IMO.

The headline change is patch 2 performing all the cross-checking at boot time.
This turned into needing prepare the Raw CPU policy earlier on boot (to avoid
further-adding to scheme we're already looking to retire).

The end result has been tested across the entire XenServer hardware lab.  This
found several false assupmtion about how the dynamic sizes behave.

Patches 1 and 6 the main bugfixes from this series.  There's still lots more
work to do in order to get AMX and/or CET working, but this is at least a 4-yo
series finally off my plate.

Andrew Cooper (7):
  x86/xstate: Fix initialisation of XSS cache
  x86/xstate: Cross-check dynamic XSTATE sizes at boot
  x86/boot: Collect the Raw CPU Policy earlier on boot
  x86/xstate: Rework xstate_ctxt_size() as xstate_uncompressed_size()
  x86/cpu-policy: Simplify recalculate_xstate()
  x86/cpuid: Fix handling of XSAVE dynamic leaves
  x86/defns: Clean up X86_{XCR0,XSS}_* constants

 xen/arch/x86/cpu-policy.c                   |  56 ++--
 xen/arch/x86/cpuid.c                        |  24 +-
 xen/arch/x86/domctl.c                       |   2 +-
 xen/arch/x86/hvm/hvm.c                      |   2 +-
 xen/arch/x86/i387.c                         |   2 +-
 xen/arch/x86/include/asm/x86-defns.h        |  55 ++--
 xen/arch/x86/include/asm/xstate.h           |   8 +-
 xen/arch/x86/setup.c                        |   4 +-
 xen/arch/x86/xstate.c                       | 286 +++++++++++++++++---
 xen/include/public/arch-x86/cpufeatureset.h |   3 +
 xen/include/xen/lib/x86/cpu-policy.h        |   2 +-
 11 files changed, 322 insertions(+), 122 deletions(-)

-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:16:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:16:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728365.1133268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RC-0006IY-G8; Thu, 23 May 2024 11:16:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728365.1133268; Thu, 23 May 2024 11:16:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RC-0006IR-BP; Thu, 23 May 2024 11:16:46 +0000
Received: by outflank-mailman (input) for mailman id 728365;
 Thu, 23 May 2024 11:16:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RB-0006Hv-MS
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:45 +0000
Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com
 [2607:f8b0:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef20935b-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:43 +0200 (CEST)
Received: by mail-oi1-x230.google.com with SMTP id
 5614622812f47-3c9b94951cfso2985814b6e.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:43 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef20935b-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463001; x=1717067801; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qvBv5IQ5vD8zkhmMyJHwoIHHhy8iEs5huUmFgDlUZVI=;
        b=rSrKe+EKldLoGvjAnhCSGhhdGbsSAqV1ug0KxVy3zGMMuN5+/t5jNCVUioLyatdXR6
         OYwQ5rzBPaishsNbA5V/2EMhCMk3DBHSgIX1dxd/fBfp62zqa5ATzKf5ZDDckpFxVnx3
         /a2BKa0q/gleXrpKx0v/cWdfdwSdKbcs71Xf4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463001; x=1717067801;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qvBv5IQ5vD8zkhmMyJHwoIHHhy8iEs5huUmFgDlUZVI=;
        b=a8zFy3E1F55SBjROkQ96m5fBrqM/M5cR/W5xx/Iy8NbvVXwvi2VtZWiYvgWlEzzhNq
         sFbOKdUa+f2cXw9oJTktX/N9RMebVy3BoSUq2iBvlxtiKLz7IrvhV8IEGXfoDJhPPLBJ
         isT40byH4rFaurRV8NewnCdHplgy/T8/mtFs8n8GhNJac7IUj8lBmj2yw6asCLsiDh0A
         Ua0tU3QcHJPFVefyic9k4FZfThxuIhByEVzBh/inc5DPp6nfcUTlwGd2exOC5fT1LJms
         vGPv8UXIxbs+aRdsTN6L6AN5tsrQu88fwr//CTLa/xEG9x/pU7e3B4cRyioDSy1DKZMt
         WPgg==
X-Gm-Message-State: AOJu0YzaRfjrq8fgYFBeP9UBcbE7AAus279Mgq7qMdRR6HEaxhGrPz3W
	XhgFtqZ6lBV6fkFyqfd2NYrB/Zv+hQSC+mTP2d77/Lr9XRfldQELZxmD8V/kjieJ2tgD/wTmfJx
	Ik0w=
X-Google-Smtp-Source: AGHT+IFI+Z81iHkbwpld0dC+maE7dJsscshL4XYvKQRclSQAfFiiFIChxlLBh7SVyOT3zLXjBG48/A==
X-Received: by 2002:a05:6870:818b:b0:23b:339d:6fbf with SMTP id 586e51a60fabf-24c68ada91bmr5712972fac.5.1716462999867;
        Thu, 23 May 2024 04:16:39 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 1/7] x86/xstate: Fix initialisation of XSS cache
Date: Thu, 23 May 2024 12:16:21 +0100
Message-Id: <20240523111627.28896-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The clobbering of this_cpu(xcr0) and this_cpu(xss) to architecturally invalid
values is to force the subsequent set_xcr0() and set_msr_xss() to reload the
hardware register.

While XCR0 is reloaded in xstate_init(), MSR_XSS isn't.  This causes
get_msr_xss() to return the invalid value, and logic of the form:

  old = get_msr_xss();
  set_msr_xss(new);
  ...
  set_msr_xss(old);

to try and restore the architecturally invalid value.

The architecturally invalid value must be purged from the cache, meaning the
hardware register must be written at least once.  This in turn highlights that
the invalid value must only be used in the case that the hardware register is
available.

Fixes: f7f4a523927f ("x86/xstate: reset cached register values on resume")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * Split out of later patch
---
 xen/arch/x86/xstate.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c
index 99cedb4f5e24..75788147966a 100644
--- a/xen/arch/x86/xstate.c
+++ b/xen/arch/x86/xstate.c
@@ -641,13 +641,6 @@ void xstate_init(struct cpuinfo_x86 *c)
         return;
     }
 
-    /*
-     * Zap the cached values to make set_xcr0() and set_msr_xss() really
-     * write it.
-     */
-    this_cpu(xcr0) = 0;
-    this_cpu(xss) = ~0;
-
     cpuid_count(XSTATE_CPUID, 0, &eax, &ebx, &ecx, &edx);
     feature_mask = (((u64)edx << 32) | eax) & XCNTXT_MASK;
     BUG_ON(!valid_xcr0(feature_mask));
@@ -657,8 +650,19 @@ void xstate_init(struct cpuinfo_x86 *c)
      * Set CR4_OSXSAVE and run "cpuid" to get xsave_cntxt_size.
      */
     set_in_cr4(X86_CR4_OSXSAVE);
+
+    /*
+     * Zap the cached values to make set_xcr0() and set_msr_xss() really write
+     * the hardware register.
+     */
+    this_cpu(xcr0) = 0;
     if ( !set_xcr0(feature_mask) )
         BUG();
+    if ( cpu_has_xsaves )
+    {
+        this_cpu(xss) = ~0;
+        set_msr_xss(0);
+    }
 
     if ( bsp )
     {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:16:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:16:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728366.1133276 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RD-0006Xd-LJ; Thu, 23 May 2024 11:16:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728366.1133276; Thu, 23 May 2024 11:16:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RD-0006XT-Ih; Thu, 23 May 2024 11:16:47 +0000
Received: by outflank-mailman (input) for mailman id 728366;
 Thu, 23 May 2024 11:16:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RC-0006Hv-Np
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:46 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id efcc8158-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:44 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-43fa6d92b4eso5467811cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:44 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efcc8158-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463002; x=1717067802; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9lVBemlknG/SLrNwwy57y6cT2xHvDd19zUouoa8Th7Q=;
        b=iqXVxyiQOw5wrq2w+PQ4Yk93emKEY8leXPK8mDy01Erwpcg0Xpgp2mdZND+sULPKI3
         060NcvmB8YLDAx+vqxNJ3+L74tl8U+b44GXyA9iSDWYt159iAO+zj2NqxzGdZlFdGDPu
         PCcPaBTQ1sXinPZh4mA76NT8tBaP1wFuhJu+4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463002; x=1717067802;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9lVBemlknG/SLrNwwy57y6cT2xHvDd19zUouoa8Th7Q=;
        b=h6bCZlmgdAU8VixVMRbdQxMPwStMBygcHHR7PcVU49tnaOyXLmnDT5u5ZTCfaL84nQ
         f8CWlDZHA9QBQc7LFhZqPfCpMgc+1ey2PP+ag1VbfxObbziPkTwxBqtiTTG6RPKTm5AG
         ZyfE6+p19K9JzEA12c36hqTj8OjBD5WbqLNnyTRJTxvA3qaklZR94ZRNEaaLI6CD2U5i
         /d2ekiQ2KOCrnD25Y66rqAAf4BR8I2R9vTa+w7UUPhF7nX6BtZn64QX82ICkb8m8RYPx
         SZYUms6goQwb4eUyKsInjsNyvLpn657UeVRqHc0SY/i4tddXYqvdFyhLcdiY862T8Ca7
         17+g==
X-Gm-Message-State: AOJu0YwZ1C1JulkUTcwLWHx2JsIBsU45JFsUJoZ2DK2HnafVT2Xxirki
	5WRo76IeoTAKUCh2gZq1MAiZDVLqbZc2kiTfXHVv4rUqHD8b82I4f74mAZdYQizXalrCbjTg52J
	B35E=
X-Google-Smtp-Source: AGHT+IH8NEooV914qM081ZP9jeUslFGVNkKno/knvhMmAz3WHl1hGmGhcv0lu/itA9GoeLdPVcJ3Qw==
X-Received: by 2002:a05:622a:164e:b0:43b:173e:ce1b with SMTP id d75a77b69052e-43fa74df0a0mr31155171cf.32.1716463001887;
        Thu, 23 May 2024 04:16:41 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 2/7] x86/xstate: Cross-check dynamic XSTATE sizes at boot
Date: Thu, 23 May 2024 12:16:22 +0100
Message-Id: <20240523111627.28896-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Right now, xstate_ctxt_size() performs a cross-check of size with CPUID in for
every call.  This is expensive, being used for domain create/migrate, as well
as to service certain guest CPUID instructions.

Instead, arrange to check the sizes once at boot.  See the code comments for
details.  Right now, it just checks hardware against the algorithm
expectations.  Later patches will add further cross-checking.

Introduce the missing X86_XCR0_* and X86_XSS_* constants, and a couple of
missing CPUID bits.  This is to maximise coverage in the sanity check, even if
we don't expect to use/virtualise some of these features any time soon.  Leave
HDC and HWP alone for now.  We don't have CPUID bits from them stored nicely.

Only perform the cross-checks in debug builds.  It's only developers or new
hardware liable to trip these checks, and Xen at least tracks "maximum value
ever seen in xcr0" for the lifetime of the VM, which we don't want to be
tickling in the general case.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * New

On Sapphire Rapids with the whole series inc diagnostics, we get this pattern:

  (XEN) *** check_new_xstate(, 0x00000003)
  (XEN) *** check_new_xstate(, 0x00000004)
  (XEN) *** check_new_xstate(, 0x000000e0)
  (XEN) *** check_new_xstate(, 0x00000200)
  (XEN) *** check_new_xstate(, 0x00060000)
  (XEN) *** check_new_xstate(, 0x00000100)
  (XEN) *** check_new_xstate(, 0x00000400)
  (XEN) *** check_new_xstate(, 0x00000800)
  (XEN) *** check_new_xstate(, 0x00001000)
  (XEN) *** check_new_xstate(, 0x00004000)
  (XEN) *** check_new_xstate(, 0x00008000)

and on Genoa, this pattern:

  (XEN) *** check_new_xstate(, 0x00000003)
  (XEN) *** check_new_xstate(, 0x00000004)
  (XEN) *** check_new_xstate(, 0x000000e0)
  (XEN) *** check_new_xstate(, 0x00000200)
  (XEN) *** check_new_xstate(, 0x00000800)
  (XEN) *** check_new_xstate(, 0x00001000)
---
 xen/arch/x86/include/asm/x86-defns.h        |  25 +++-
 xen/arch/x86/xstate.c                       | 150 ++++++++++++++++++++
 xen/include/public/arch-x86/cpufeatureset.h |   3 +
 3 files changed, 177 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h
index 48d7a3b7af45..d7602ab225c4 100644
--- a/xen/arch/x86/include/asm/x86-defns.h
+++ b/xen/arch/x86/include/asm/x86-defns.h
@@ -77,7 +77,7 @@
 #define X86_CR4_PKS        0x01000000 /* Protection Key Supervisor */
 
 /*
- * XSTATE component flags in XCR0
+ * XSTATE component flags in XCR0 | MSR_XSS
  */
 #define X86_XCR0_FP_POS           0
 #define X86_XCR0_FP               (1ULL << X86_XCR0_FP_POS)
@@ -95,11 +95,34 @@
 #define X86_XCR0_ZMM              (1ULL << X86_XCR0_ZMM_POS)
 #define X86_XCR0_HI_ZMM_POS       7
 #define X86_XCR0_HI_ZMM           (1ULL << X86_XCR0_HI_ZMM_POS)
+#define X86_XSS_PROC_TRACE        (_AC(1, ULL) <<  8)
 #define X86_XCR0_PKRU_POS         9
 #define X86_XCR0_PKRU             (1ULL << X86_XCR0_PKRU_POS)
+#define X86_XSS_PASID             (_AC(1, ULL) << 10)
+#define X86_XSS_CET_U             (_AC(1, ULL) << 11)
+#define X86_XSS_CET_S             (_AC(1, ULL) << 12)
+#define X86_XSS_HDC               (_AC(1, ULL) << 13)
+#define X86_XSS_UINTR             (_AC(1, ULL) << 14)
+#define X86_XSS_LBR               (_AC(1, ULL) << 15)
+#define X86_XSS_HWP               (_AC(1, ULL) << 16)
+#define X86_XCR0_TILE_CFG         (_AC(1, ULL) << 17)
+#define X86_XCR0_TILE_DATA        (_AC(1, ULL) << 18)
 #define X86_XCR0_LWP_POS          62
 #define X86_XCR0_LWP              (1ULL << X86_XCR0_LWP_POS)
 
+#define X86_XCR0_STATES                                                 \
+    (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS |     \
+     X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM |                 \
+     X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG |              \
+     X86_XCR0_TILE_DATA |                                               \
+     X86_XCR0_LWP)
+
+#define X86_XSS_STATES                                                  \
+    (X86_XSS_PROC_TRACE | X86_XSS_PASID | X86_XSS_CET_U |               \
+     X86_XSS_CET_S | X86_XSS_HDC | X86_XSS_UINTR | X86_XSS_LBR |        \
+     X86_XSS_HWP |                                                      \
+     0)
+
 /*
  * Debug status flags in DR6.
  *
diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c
index 75788147966a..33a5a89719ef 100644
--- a/xen/arch/x86/xstate.c
+++ b/xen/arch/x86/xstate.c
@@ -604,9 +604,156 @@ static bool valid_xcr0(uint64_t xcr0)
     if ( !(xcr0 & X86_XCR0_BNDREGS) != !(xcr0 & X86_XCR0_BNDCSR) )
         return false;
 
+    /* TILE_CFG and TILE_DATA must be the same. */
+    if ( !(xcr0 & X86_XCR0_TILE_CFG) != !(xcr0 & X86_XCR0_TILE_DATA) )
+        return false;
+
     return true;
 }
 
+struct xcheck_state {
+    uint64_t states;
+    uint32_t uncomp_size;
+    uint32_t comp_size;
+};
+
+static void __init check_new_xstate(struct xcheck_state *s, uint64_t new)
+{
+    uint32_t hw_size;
+
+    BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES);
+
+    BUG_ON(s->states & new); /* States only increase. */
+    BUG_ON(!valid_xcr0(s->states | new)); /* Xen thinks it's a good value. */
+    BUG_ON(new & ~(X86_XCR0_STATES | X86_XSS_STATES)); /* Known state. */
+    BUG_ON((new & X86_XCR0_STATES) &&
+           (new & X86_XSS_STATES)); /* User or supervisor, not both. */
+
+    s->states |= new;
+    if ( new & X86_XCR0_STATES )
+    {
+        if ( !set_xcr0(s->states & X86_XCR0_STATES) )
+            BUG();
+    }
+    else
+        set_msr_xss(s->states & X86_XSS_STATES);
+
+    /*
+     * Check the uncompressed size.  Some XSTATEs are out-of-order and fill in
+     * prior holes in the state area, so we check that the size doesn't
+     * decrease.
+     */
+    hw_size = cpuid_count_ebx(0xd, 0);
+
+    if ( hw_size < s->uncomp_size )
+        panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, uncompressed hw size %#x < prev size %#x\n",
+              s->states, &new, hw_size, s->uncomp_size);
+
+    s->uncomp_size = hw_size;
+
+    /*
+     * Check the compressed size, if available.  All components strictly
+     * appear in index order.  In principle there are no holes, but some
+     * components have their base address 64-byte aligned for efficiency
+     * reasons (e.g. AMX-TILE) and there are other components small enough to
+     * fit in the gap (e.g. PKRU) without increasing the overall length.
+     */
+    hw_size = cpuid_count_ebx(0xd, 1);
+
+    if ( cpu_has_xsavec )
+    {
+        if ( hw_size < s->comp_size )
+            panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, compressed hw size %#x < prev size %#x\n",
+                  s->states, &new, hw_size, s->comp_size);
+
+        s->comp_size = hw_size;
+    }
+    else
+        BUG_ON(hw_size); /* Compressed size reported, but no XSAVEC ? */
+}
+
+/*
+ * The {un,}compressed XSTATE sizes are reported by dynamic CPUID value, based
+ * on the current %XCR0 and MSR_XSS values.  The exact layout is also feature
+ * and vendor specific.  Cross-check Xen's understanding against real hardware
+ * on boot.
+ *
+ * Testing every combination is prohibitive, so we use a partial approach.
+ * Starting with nothing active, we add new XSTATEs and check that the CPUID
+ * dynamic values never decreases.
+ */
+static void __init noinline xstate_check_sizes(void)
+{
+    uint64_t old_xcr0 = get_xcr0();
+    uint64_t old_xss = get_msr_xss();
+    struct xcheck_state s = {};
+
+    /*
+     * User XSTATEs, increasing by index.
+     *
+     * Chronologically, Intel and AMD had identical layouts for AVX (YMM).
+     * AMD introduced LWP in Fam15h, following immediately on from YMM.  Intel
+     * left an LWP-shaped hole when adding MPX (BND{CSR,REGS}) in Skylake.
+     * AMD removed LWP in Fam17h, putting PKRU in the same space, breaking
+     * layout compatibility with Intel and having a knock-on effect on all
+     * subsequent states.
+     */
+    check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP);
+
+    if ( cpu_has_avx )
+        check_new_xstate(&s, X86_XCR0_YMM);
+
+    if ( cpu_has_mpx )
+        check_new_xstate(&s, X86_XCR0_BNDCSR | X86_XCR0_BNDREGS);
+
+    if ( cpu_has_avx512f )
+        check_new_xstate(&s, X86_XCR0_HI_ZMM | X86_XCR0_ZMM | X86_XCR0_OPMASK);
+
+    if ( cpu_has_pku )
+        check_new_xstate(&s, X86_XCR0_PKRU);
+
+    if ( boot_cpu_has(X86_FEATURE_AMX_TILE) )
+        check_new_xstate(&s, X86_XCR0_TILE_DATA | X86_XCR0_TILE_CFG);
+
+    if ( boot_cpu_has(X86_FEATURE_LWP) )
+        check_new_xstate(&s, X86_XCR0_LWP);
+
+    /*
+     * Supervisor XSTATEs, increasing by index.
+     *
+     * Intel Broadwell in particular had Processor Trace but no XSAVES.  There
+     * doesn't appear to have been a new enumeration when X86_XSS_PROC_TRACE
+     * was introduced in Skylake.
+     */
+    if ( cpu_has_xsaves )
+    {
+        if ( cpu_has_proc_trace )
+            check_new_xstate(&s, X86_XSS_PROC_TRACE);
+
+        if ( boot_cpu_has(X86_FEATURE_ENQCMD) )
+            check_new_xstate(&s, X86_XSS_PASID);
+
+        if ( boot_cpu_has(X86_FEATURE_CET_SS) ||
+             boot_cpu_has(X86_FEATURE_CET_IBT) )
+        {
+            check_new_xstate(&s, X86_XSS_CET_U);
+            check_new_xstate(&s, X86_XSS_CET_S);
+        }
+
+        if ( boot_cpu_has(X86_FEATURE_UINTR) )
+            check_new_xstate(&s, X86_XSS_UINTR);
+
+        if ( boot_cpu_has(X86_FEATURE_ARCH_LBR) )
+            check_new_xstate(&s, X86_XSS_LBR);
+    }
+
+    /* Restore old state now the test is done. */
+    if ( !set_xcr0(old_xcr0) )
+        BUG();
+    if ( cpu_has_xsaves )
+        set_msr_xss(old_xss);
+}
+
 /* Collect the information of processor's extended state */
 void xstate_init(struct cpuinfo_x86 *c)
 {
@@ -683,6 +830,9 @@ void xstate_init(struct cpuinfo_x86 *c)
 
     if ( setup_xstate_features(bsp) && bsp )
         BUG();
+
+    if ( IS_ENABLED(CONFIG_DEBUG) && bsp )
+        xstate_check_sizes();
 }
 
 int validate_xstate(const struct domain *d, uint64_t xcr0, uint64_t xcr0_accum,
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 6627453e3985..d9eba5e9a714 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -266,6 +266,7 @@ XEN_CPUFEATURE(IBPB_RET,      8*32+30) /*A  IBPB clears RSB/RAS too. */
 XEN_CPUFEATURE(AVX512_4VNNIW, 9*32+ 2) /*A  AVX512 Neural Network Instructions */
 XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A  AVX512 Multiply Accumulation Single Precision */
 XEN_CPUFEATURE(FSRM,          9*32+ 4) /*A  Fast Short REP MOVS */
+XEN_CPUFEATURE(UINTR,         9*32+ 5) /*   User-mode Interrupts */
 XEN_CPUFEATURE(AVX512_VP2INTERSECT, 9*32+8) /*a  VP2INTERSECT{D,Q} insns */
 XEN_CPUFEATURE(SRBDS_CTRL,    9*32+ 9) /*   MSR_MCU_OPT_CTRL and RNGDS_MITG_DIS. */
 XEN_CPUFEATURE(MD_CLEAR,      9*32+10) /*!A| VERW clears microarchitectural buffers */
@@ -274,8 +275,10 @@ XEN_CPUFEATURE(TSX_FORCE_ABORT, 9*32+13) /* MSR_TSX_FORCE_ABORT.RTM_ABORT */
 XEN_CPUFEATURE(SERIALIZE,     9*32+14) /*A  SERIALIZE insn */
 XEN_CPUFEATURE(HYBRID,        9*32+15) /*   Heterogeneous platform */
 XEN_CPUFEATURE(TSXLDTRK,      9*32+16) /*a  TSX load tracking suspend/resume insns */
+XEN_CPUFEATURE(ARCH_LBR,      9*32+19) /*   Architectural Last Branch Record */
 XEN_CPUFEATURE(CET_IBT,       9*32+20) /*   CET - Indirect Branch Tracking */
 XEN_CPUFEATURE(AVX512_FP16,   9*32+23) /*A  AVX512 FP16 instructions */
+XEN_CPUFEATURE(AMX_TILE,      9*32+24) /*   AMX Tile architecture */
 XEN_CPUFEATURE(IBRSB,         9*32+26) /*A  IBRS and IBPB support (used by Intel) */
 XEN_CPUFEATURE(STIBP,         9*32+27) /*A  STIBP */
 XEN_CPUFEATURE(L1D_FLUSH,     9*32+28) /*S  MSR_FLUSH_CMD and L1D flush. */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:17:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:17:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728367.1133286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RI-0006rX-2v; Thu, 23 May 2024 11:16:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728367.1133286; Thu, 23 May 2024 11:16:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RH-0006rE-Vg; Thu, 23 May 2024 11:16:51 +0000
Received: by outflank-mailman (input) for mailman id 728367;
 Thu, 23 May 2024 11:16:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RG-0006Hv-CY
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:50 +0000
Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com
 [2607:f8b0:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f22261f1-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:48 +0200 (CEST)
Received: by mail-oi1-x233.google.com with SMTP id
 5614622812f47-3cabac56b38so1546647b6e.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:48 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f22261f1-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463006; x=1717067806; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3brSZl6/vV2dpGWkfgvqrJmcT56Fa2D0F6BOtDAqDOI=;
        b=LcahzV4fVqrQv+UWToDs4J/GQMy9S0YbbRRlhQpGlgUV/HhnnTGOMlN1Whrjju8QTW
         IgkiTIFaQpJBaA+1eFackREzUbuC162gDsO2ImJL0GEBGOlDVB/4/ok6oE5mrgJ5FOwh
         aSwq0Ed6u7iS/EFZ6r5dWe2BxDdhoBn6VNLzI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463006; x=1717067806;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3brSZl6/vV2dpGWkfgvqrJmcT56Fa2D0F6BOtDAqDOI=;
        b=qYl9f9ft+clGR9+yRS9Sme1azyS731LsXSsTKkP9iTU9bXyq6YO+X5vqveVUH1Za/6
         CqoyrScm6NLdo7sdTvAvkE2d6u36E42EhoX0qq92SC/NC5enAvqcKp6uujnZqFhei/lq
         gq6LWyauE2ZH3kkYyY6iDEOZPyBmwzrQmItu2OYkp4K5RCekfog/oF1bPwBA/b+nQZNa
         iIRefhgLS2ZxoMvacQuSBNymg95WD2mxJJ+ONIFuwo5bWgXy7mPruEXQVKgh9+waC9kV
         LGqQ2kV7xkWgOjc5xzMQxQTiymiMK1jR4HSxmExcnAHDeRFU5ocZc4F68twgVC3F+wVS
         NC2A==
X-Gm-Message-State: AOJu0YxBnyQAmXYJ9z+dk3ozENG/tbBsHZYtNM9rNm8X8CHjz0xQvWuQ
	/5BdvNdiUC205LFaCWxKxUuWhc0BuEU4Cbqobq9PmUXqMbWyDqj36mQtYV+/8s98ASX0uwbI3Vs
	+QSo=
X-Google-Smtp-Source: AGHT+IFpvJcWqnaHvo7hcFKKjX515sNn6S5bC6QmZNxfnQRU35FyPFQYvA+CWP5bK1L4FB1RMRQaEg==
X-Received: by 2002:a54:4715:0:b0:3c9:6e96:88fb with SMTP id 5614622812f47-3cdb0c91a7bmr4613116b6e.4.1716463004868;
        Thu, 23 May 2024 04:16:44 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 3/7] x86/boot: Collect the Raw CPU Policy earlier on boot
Date: Thu, 23 May 2024 12:16:23 +0100
Message-Id: <20240523111627.28896-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is a tangle, but it's a small step in the right direction.

xstate_init() is shortly going to want data from the Raw policy.
calculate_raw_cpu_policy() is sufficiently separate from the other policies to
be safe to do.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

This is necessary for the forthcoming xstate_{un,}compressed_size() to perform
boot-time sanity checks on state components which aren't fully enabled yet.  I
decided that doing this was better than extending the xstate_{offsets,sizes}[]
logic that we're intending to retire in due course.

v3:
 * New.
---
 xen/arch/x86/cpu-policy.c | 1 -
 xen/arch/x86/setup.c      | 4 +++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index b96f4ee55cc4..5b66f002df05 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -845,7 +845,6 @@ static void __init calculate_hvm_def_policy(void)
 
 void __init init_guest_cpu_policies(void)
 {
-    calculate_raw_cpu_policy();
     calculate_host_policy();
 
     if ( IS_ENABLED(CONFIG_PV) )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b50c9c84af6d..8850e5637a98 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1888,7 +1888,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
 
     tsx_init(); /* Needs microcode.  May change HLE/RTM feature bits. */
 
-    identify_cpu(&boot_cpu_data);
+    calculate_raw_cpu_policy(); /* Needs microcode.  No other dependenices. */
+
+    identify_cpu(&boot_cpu_data); /* Needs microcode and raw policy. */
 
     set_in_cr4(X86_CR4_OSFXSR | X86_CR4_OSXMMEXCPT);
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:17:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728368.1133297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RJ-00077g-D0; Thu, 23 May 2024 11:16:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728368.1133297; Thu, 23 May 2024 11:16:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RJ-00077W-8w; Thu, 23 May 2024 11:16:53 +0000
Received: by outflank-mailman (input) for mailman id 728368;
 Thu, 23 May 2024 11:16:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RH-0006Hv-GC
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:51 +0000
Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com
 [2607:f8b0:4864:20::f2d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2c77944-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:49 +0200 (CEST)
Received: by mail-qv1-xf2d.google.com with SMTP id
 6a1803df08f44-6a919a4bb83so23126766d6.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:49 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2c77944-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463007; x=1717067807; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Wwexe8CnptsNPO0v+knt+q1PXvcS9l92Rfmc3cBUT9k=;
        b=OUjOBJscARRhYbbgmXwvrGmTOrxewXq/TjEwLdeXNVFOFKzFDl5YAcumil08u5LVvR
         VT6JW/AZxBZyyJL6BJKosUCZM7mjrjXEU5hresS1GMATp0ffEjuw8y3K4EKLVt+3qQbG
         fw8U8RFiNpI4q8sqmoV27vuQl4jE6h3846hqU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463007; x=1717067807;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Wwexe8CnptsNPO0v+knt+q1PXvcS9l92Rfmc3cBUT9k=;
        b=wTCaM5+MXCXcgG937V1ioB0TYrErFJ+ZPQWXbz0QyRFZNPaUwNtH/+ZBesnV5DvAFB
         bHgWlBAfCzq8eCUPeZgBqW+LYDOepJ/jktsUZjz23NxOpIC0pyPF+qOFhvpLwKkyzpPQ
         PeN3Zyk5/GhVO4j2LrfziYHS/v7ez92mj25FDy9N7cyAItsqNhv/p9KoH5Ez6Y78XVCl
         Cx0RADve1wuFJwIVK8B+fT4EhIbSFZNT8W8K5r+SfKtunjijkeHqtdrnOu4AYs9nW4+U
         WZDmRyV5TrB4P6rllnhNNpIuHB1V3kaA1aa12Hb+mgvjMdpcEGnvoHr8FybpNFFLff1k
         /p2A==
X-Gm-Message-State: AOJu0Yw1xqyTpXXNO1dBPYsMd1jG+lIsX0H2gL7zQlSaVSaOVqogVffL
	OYaU5yqD/G50BDkk/XnRUdzSBFhLRmeq+xT+P5oi4lKZP3XCZHXcPXk3mxXSSEODiHBZ2V4Qp7n
	WZfE=
X-Google-Smtp-Source: AGHT+IH0uVBwwjZDQEYD5tp1lGk2N/50w+jo3zrngapn0+tG/W/IPEGsiq58L5vBJimDo5vkz/U2BA==
X-Received: by 2002:a05:6214:3c9e:b0:6aa:600e:23c with SMTP id 6a1803df08f44-6ab808f29e9mr55730046d6.43.1716463006986;
        Thu, 23 May 2024 04:16:46 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 4/7] x86/xstate: Rework xstate_ctxt_size() as xstate_uncompressed_size()
Date: Thu, 23 May 2024 12:16:24 +0100
Message-Id: <20240523111627.28896-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We're soon going to need a compressed helper of the same form.

The size of the uncompressed image depends on the single element with the
largest offset + size.  Sadly this isn't always the element with the largest
index.

Name the per-xstate-component cpu_policy struture, for legibility of the logic
in xstate_uncompressed_size().  Cross-check with hardware during boot, and
remove hw_uncompressed_size().  This means that the migration paths don't need
to mess with XCR0 just to sanity check the buffer size.

The users of hw_uncompressed_size() in xstate_init() can (and indeed need) to
be replaced with CPUID instructions.  They run with feature_mask in XCR0, and
prior to setup_xstate_features() on the BSP.

No practical change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v2:
 * Scan all features.  LWP/APX_F are out-of-order.
v3:
 * Rebase over boot time check.
 * Use the raw CPU policy.
---
 xen/arch/x86/domctl.c                |  2 +-
 xen/arch/x86/hvm/hvm.c               |  2 +-
 xen/arch/x86/include/asm/xstate.h    |  2 +-
 xen/arch/x86/xstate.c                | 78 +++++++++++++++++-----------
 xen/include/xen/lib/x86/cpu-policy.h |  2 +-
 5 files changed, 51 insertions(+), 35 deletions(-)

diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 9a72d57333e9..c2f2016ed45a 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -833,7 +833,7 @@ long arch_do_domctl(
         uint32_t offset = 0;
 
 #define PV_XSAVE_HDR_SIZE (2 * sizeof(uint64_t))
-#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_ctxt_size(xcr0))
+#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_uncompressed_size(xcr0))
 
         ret = -ESRCH;
         if ( (evc->vcpu >= d->max_vcpus) ||
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 2c66fe0f7a16..b84f4d2387d1 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1190,7 +1190,7 @@ HVM_REGISTER_SAVE_RESTORE(CPU, hvm_save_cpu_ctxt, NULL, hvm_load_cpu_ctxt, 1,
 
 #define HVM_CPU_XSAVE_SIZE(xcr0) (offsetof(struct hvm_hw_cpu_xsave, \
                                            save_area) + \
-                                  xstate_ctxt_size(xcr0))
+                                  xstate_uncompressed_size(xcr0))
 
 static int cf_check hvm_save_cpu_xsave_states(
     struct vcpu *v, hvm_domain_context_t *h)
diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h
index c08c267884f0..f5115199d4f9 100644
--- a/xen/arch/x86/include/asm/xstate.h
+++ b/xen/arch/x86/include/asm/xstate.h
@@ -107,7 +107,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size);
 void xstate_free_save_area(struct vcpu *v);
 int xstate_alloc_save_area(struct vcpu *v);
 void xstate_init(struct cpuinfo_x86 *c);
-unsigned int xstate_ctxt_size(u64 xcr0);
+unsigned int xstate_uncompressed_size(uint64_t xcr0);
 
 static inline uint64_t xgetbv(unsigned int index)
 {
diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c
index 33a5a89719ef..1b3153600d9c 100644
--- a/xen/arch/x86/xstate.c
+++ b/xen/arch/x86/xstate.c
@@ -8,6 +8,8 @@
 #include <xen/param.h>
 #include <xen/percpu.h>
 #include <xen/sched.h>
+
+#include <asm/cpu-policy.h>
 #include <asm/current.h>
 #include <asm/processor.h>
 #include <asm/i387.h>
@@ -183,7 +185,7 @@ void expand_xsave_states(const struct vcpu *v, void *dest, unsigned int size)
     /* Check there is state to serialise (i.e. at least an XSAVE_HDR) */
     BUG_ON(!v->arch.xcr0_accum);
     /* Check there is the correct room to decompress into. */
-    BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum));
+    BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum));
 
     if ( !(xstate->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED) )
     {
@@ -245,7 +247,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size)
     u64 xstate_bv, valid;
 
     BUG_ON(!v->arch.xcr0_accum);
-    BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum));
+    BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum));
     ASSERT(!xsave_area_compressed(src));
 
     xstate_bv = ((const struct xsave_struct *)src)->xsave_hdr.xstate_bv;
@@ -553,32 +555,6 @@ void xstate_free_save_area(struct vcpu *v)
     v->arch.xsave_area = NULL;
 }
 
-static unsigned int hw_uncompressed_size(uint64_t xcr0)
-{
-    u64 act_xcr0 = get_xcr0();
-    unsigned int size;
-    bool ok = set_xcr0(xcr0);
-
-    ASSERT(ok);
-    size = cpuid_count_ebx(XSTATE_CPUID, 0);
-    ok = set_xcr0(act_xcr0);
-    ASSERT(ok);
-
-    return size;
-}
-
-/* Fastpath for common xstate size requests, avoiding reloads of xcr0. */
-unsigned int xstate_ctxt_size(u64 xcr0)
-{
-    if ( xcr0 == xfeature_mask )
-        return xsave_cntxt_size;
-
-    if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */
-        return 0;
-
-    return hw_uncompressed_size(xcr0);
-}
-
 static bool valid_xcr0(uint64_t xcr0)
 {
     /* FP must be unconditionally set. */
@@ -611,6 +587,40 @@ static bool valid_xcr0(uint64_t xcr0)
     return true;
 }
 
+unsigned int xstate_uncompressed_size(uint64_t xcr0)
+{
+    unsigned int size = XSTATE_AREA_MIN_SIZE, i;
+
+    ASSERT((xcr0 & ~X86_XCR0_STATES) == 0);
+
+    if ( xcr0 == xfeature_mask )
+        return xsave_cntxt_size;
+
+    if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */
+        return 0;
+
+    if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) )
+        return size;
+
+    /*
+     * For the non-legacy states, search all activate states and find the
+     * maximum offset+size.  Some states (e.g. LWP, APX_F) are out-of-order
+     * with respect their index.
+     */
+    xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP);
+    for_each_set_bit ( i, &xcr0, 63 )
+    {
+        const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i];
+        unsigned int s = c->offset + c->size;
+
+        ASSERT(c->offset && c->size);
+
+        size = max(size, s);
+    }
+
+    return size;
+}
+
 struct xcheck_state {
     uint64_t states;
     uint32_t uncomp_size;
@@ -619,7 +629,7 @@ struct xcheck_state {
 
 static void __init check_new_xstate(struct xcheck_state *s, uint64_t new)
 {
-    uint32_t hw_size;
+    uint32_t hw_size, xen_size;
 
     BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES);
 
@@ -651,6 +661,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new)
 
     s->uncomp_size = hw_size;
 
+    xen_size = xstate_uncompressed_size(s->states & X86_XCR0_STATES);
+
+    if ( xen_size != hw_size )
+        panic("XSTATE 0x%016"PRIx64", uncompressed hw size %#x != xen size %#x\n",
+              s->states, hw_size, xen_size);
+
     /*
      * Check the compressed size, if available.  All components strictly
      * appear in index order.  In principle there are no holes, but some
@@ -818,14 +834,14 @@ void xstate_init(struct cpuinfo_x86 *c)
          * xsave_cntxt_size is the max size required by enabled features.
          * We know FP/SSE and YMM about eax, and nothing about edx at present.
          */
-        xsave_cntxt_size = hw_uncompressed_size(feature_mask);
+        xsave_cntxt_size = cpuid_count_ebx(0xd, 0);
         printk("xstate: size: %#x and states: %#"PRIx64"\n",
                xsave_cntxt_size, xfeature_mask);
     }
     else
     {
         BUG_ON(xfeature_mask != feature_mask);
-        BUG_ON(xsave_cntxt_size != hw_uncompressed_size(feature_mask));
+        BUG_ON(xsave_cntxt_size != cpuid_count_ebx(0xd, 0));
     }
 
     if ( setup_xstate_features(bsp) && bsp )
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index d5e447e9dc06..d26012c6da78 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -248,7 +248,7 @@ struct cpu_policy
         };
 
         /* Per-component common state.  Valid for i >= 2. */
-        struct {
+        struct xstate_component {
             uint32_t size, offset;
             bool xss:1, align:1;
             uint32_t _res_d;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:17:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:17:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728369.1133306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RL-0007QQ-JS; Thu, 23 May 2024 11:16:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728369.1133306; Thu, 23 May 2024 11:16:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RL-0007QF-Gm; Thu, 23 May 2024 11:16:55 +0000
Received: by outflank-mailman (input) for mailman id 728369;
 Thu, 23 May 2024 11:16:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RK-0006Hv-Hz
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:54 +0000
Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com
 [2607:f8b0:4864:20::f2d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f4b33630-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:52 +0200 (CEST)
Received: by mail-qv1-xf2d.google.com with SMTP id
 6a1803df08f44-69b4454e2f1so8701066d6.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:52 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4b33630-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463011; x=1717067811; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=LRpbv1/wBcaMA0cTk1bCZyxrSAb4NeFKdZW8+eY/F+o=;
        b=pWUOmSIQAhLXWjJFvUvygq1VCgRvXfRv6ZyMBWU5AU3e8GPy5rSpSKYjo1caH+jl19
         NqtgitM443GhHCMLS0X/1PQEoXSzyvQQ6LZTIQt6rLnxNQ1Hqhz+tkuqZDBfxBjsrJcx
         AcTX1sDuQ81tyxB8IUMNytv9fnoQQaqXkaMFQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463011; x=1717067811;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=LRpbv1/wBcaMA0cTk1bCZyxrSAb4NeFKdZW8+eY/F+o=;
        b=ESQDGgORVouPWnmPlHMS66g1kRN656Fpn7e/Sg9UTgM8QgTWqsXDxUZdgkQTeeiLan
         TMTjRK5ZKq0zCMuIR3UsmrI2wi75+XO812F+ePYChqOA1hLLPSr1vGHuyWGHbwmt1dfT
         05JBujkJFq+ovJxwSj7+P09MQYDLIz7rc3T+Gid9RgaOa97clCi+P1ou7NSpQOs9+Slg
         FrHzGugwTPxqw9FafMhaXevoyYIhYHAv9maJcQ0p8aCPihEmUDJAOCnRZes6S5Kuy3UC
         zNmRpq1UjyNUcp1CEv0HShsFHxSJZElnQ7Gfrc3ZtZKTf1M08Db3XhTUiPOvLIyJMuYm
         1UOQ==
X-Gm-Message-State: AOJu0Yywg8xYhwr6DtPj7grBJRudEhn9v+QxUTMqPIAwE4uPEPSWUfnR
	tLbSeKOwAJddHhoJNOtGGJ1QKZOI0Q6ChnuVE/g7ATMkymnjLQT3vn65NbrYe5xkXOKQHbcIvUG
	yA0s=
X-Google-Smtp-Source: AGHT+IEYLtrmNJ4sPHn6mziSkxTZt6d8QJUEhwLHRsESbQZLZsTWtZt7iKeF6iJXaGGTuiC6cwAxrg==
X-Received: by 2002:a05:6214:5c49:b0:6ab:6f15:515e with SMTP id 6a1803df08f44-6ab7f3398f7mr48834776d6.13.1716463010789;
        Thu, 23 May 2024 04:16:50 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 5/7] x86/cpu-policy: Simplify recalculate_xstate()
Date: Thu, 23 May 2024 12:16:25 +0100
Message-Id: <20240523111627.28896-6-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Make use of xstate_uncompressed_size() helper rather than maintaining the
running calculation while accumulating feature components.

The rest of the CPUID data can come direct from the raw cpu policy.  All
per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}*
instructions.

Use for_each_set_bit() rather than opencoding a slightly awkward version of
it.  Mask the attributes in ecx down based on the visible features.  This
isn't actually necessary for any components or attributes defined at the time
of writing (up to AMX), but is added out of an abundance of caution.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v2:
 * Tie ALIGN64 to xsavec rather than xsaves.
v3:
 * Tweak commit message.
---
 xen/arch/x86/cpu-policy.c         | 55 +++++++++++--------------------
 xen/arch/x86/include/asm/xstate.h |  1 +
 2 files changed, 21 insertions(+), 35 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 5b66f002df05..304dc20cfab8 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -193,8 +193,7 @@ static void sanitise_featureset(uint32_t *fs)
 static void recalculate_xstate(struct cpu_policy *p)
 {
     uint64_t xstates = XSTATE_FP_SSE;
-    uint32_t xstate_size = XSTATE_AREA_MIN_SIZE;
-    unsigned int i, Da1 = p->xstate.Da1;
+    unsigned int i, ecx_mask = 0, Da1 = p->xstate.Da1;
 
     /*
      * The Da1 leaf is the only piece of information preserved in the common
@@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p)
         return;
 
     if ( p->basic.avx )
-    {
         xstates |= X86_XCR0_YMM;
-        xstate_size = max(xstate_size,
-                          xstate_offsets[X86_XCR0_YMM_POS] +
-                          xstate_sizes[X86_XCR0_YMM_POS]);
-    }
 
     if ( p->feat.mpx )
-    {
         xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR;
-        xstate_size = max(xstate_size,
-                          xstate_offsets[X86_XCR0_BNDCSR_POS] +
-                          xstate_sizes[X86_XCR0_BNDCSR_POS]);
-    }
 
     if ( p->feat.avx512f )
-    {
         xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM;
-        xstate_size = max(xstate_size,
-                          xstate_offsets[X86_XCR0_HI_ZMM_POS] +
-                          xstate_sizes[X86_XCR0_HI_ZMM_POS]);
-    }
 
     if ( p->feat.pku )
-    {
         xstates |= X86_XCR0_PKRU;
-        xstate_size = max(xstate_size,
-                          xstate_offsets[X86_XCR0_PKRU_POS] +
-                          xstate_sizes[X86_XCR0_PKRU_POS]);
-    }
 
-    p->xstate.max_size  =  xstate_size;
+    /* Subleaf 0 */
+    p->xstate.max_size =
+        xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY);
     p->xstate.xcr0_low  =  xstates & ~XSTATE_XSAVES_ONLY;
     p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32;
 
+    /* Subleaf 1 */
     p->xstate.Da1 = Da1;
+    if ( p->xstate.xsavec )
+        ecx_mask |= XSTATE_ALIGN64;
+
     if ( p->xstate.xsaves )
     {
+        ecx_mask |= XSTATE_XSS;
         p->xstate.xss_low   =  xstates & XSTATE_XSAVES_ONLY;
         p->xstate.xss_high  = (xstates & XSTATE_XSAVES_ONLY) >> 32;
     }
-    else
-        xstates &= ~XSTATE_XSAVES_ONLY;
 
-    for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i )
+    /* Subleafs 2+ */
+    xstates &= ~XSTATE_FP_SSE;
+    BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63);
+    for_each_set_bit ( i, &xstates, 63 )
     {
-        uint64_t curr_xstate = 1UL << i;
-
-        if ( !(xstates & curr_xstate) )
-            continue;
-
-        p->xstate.comp[i].size   = xstate_sizes[i];
-        p->xstate.comp[i].offset = xstate_offsets[i];
-        p->xstate.comp[i].xss    = curr_xstate & XSTATE_XSAVES_ONLY;
-        p->xstate.comp[i].align  = curr_xstate & xstate_align;
+        /*
+         * Pass through size (eax) and offset (ebx) directly.  Visbility of
+         * attributes in ecx limited by visible features in Da1.
+         */
+        p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a;
+        p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b;
+        p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask;
     }
 }
 
diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h
index f5115199d4f9..bfb66dd766b6 100644
--- a/xen/arch/x86/include/asm/xstate.h
+++ b/xen/arch/x86/include/asm/xstate.h
@@ -40,6 +40,7 @@ extern uint32_t mxcsr_mask;
 #define XSTATE_XSAVES_ONLY         0
 #define XSTATE_COMPACTION_ENABLED  (1ULL << 63)
 
+#define XSTATE_XSS     (1U << 0)
 #define XSTATE_ALIGN64 (1U << 1)
 
 extern u64 xfeature_mask;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:17:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:17:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728370.1133317 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RN-0007lJ-VM; Thu, 23 May 2024 11:16:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728370.1133317; Thu, 23 May 2024 11:16:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RN-0007kz-QD; Thu, 23 May 2024 11:16:57 +0000
Received: by outflank-mailman (input) for mailman id 728370;
 Thu, 23 May 2024 11:16:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RN-0006Hv-0K
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:57 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f62566fc-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:55 +0200 (CEST)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-792c3741c71so77887785a.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:55 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f62566fc-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463013; x=1717067813; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=us4603FaoecAqWNVKzEdaqnc883Hrl4rvqqHc2XtrfI=;
        b=bUd6IOtRE2wCKKwEStO+5AvFKg4I6nqhz8gwBIi5iW6t+OnMuUkNl0BWwllM45DlWW
         GyyglXfBfcSCdHJIJg3Zm/TgvZM8hy6IAUziNPEiF+4/sIpuOVIjp57rdkEf1Jg67Uf3
         T94PxcTZHr3j5zzoWw8EIpvhCe0WATJvdlIxY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463013; x=1717067813;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=us4603FaoecAqWNVKzEdaqnc883Hrl4rvqqHc2XtrfI=;
        b=eeB1tvtlrnKRsuuZ8hHYc3naJy75IIYXjc+cPUoTxD63fqqfpQ07d62QCcOPlAbP5H
         jqkQShHhvwjBufbG+vd4wjisOjBdws+ICSbrfeGsUqDQpu94/fT8mfevErU8AUBh2i1u
         hKLgYr1ppBoewaQx/lWG9Wl5aHLVRx+I98NEczdM2u9w5R/qk2+RBBv52Jb99qZojJIw
         pumt6ziU1FBSWmR1lhoCShqg0p3FnjZtv44U5UUb9XhILj9OXTbJwS83bbaevIKKWUEi
         pwbKyh49b9N6V0KflC3zWD3MqZ+/6xk55bd+NfSF9JQjlanCy1/xZ2Xac5QAJvLOYBTv
         UvKw==
X-Gm-Message-State: AOJu0YwV43xrH0WsspzvjMptag/SZ05cZ1fYKbQaGrdm8A6OB8WhAiXm
	QRzJI4I/Q2xEUvxpIabZMtcDVEzV9zUioiCBwvXoNc4gSG341y8sNSpHG81+gxWga9l0ACWDsAS
	Q+1o=
X-Google-Smtp-Source: AGHT+IFMd6G/LG3bqX6YqX/t3Cg+uvutJKi9sTJLns27rXhQaiIj1RFtnwEcd6kpIryKMzcdy+XPRw==
X-Received: by 2002:a05:6214:2b8e:b0:6ab:932d:c24f with SMTP id 6a1803df08f44-6ab932dc50bmr12661656d6.52.1716463013119;
        Thu, 23 May 2024 04:16:53 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 6/7] x86/cpuid: Fix handling of XSAVE dynamic leaves
Date: Thu, 23 May 2024 12:16:26 +0100
Message-Id: <20240523111627.28896-7-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

First, if XSAVE is available in hardware but not visible to the guest, the
dynamic leaves shouldn't be filled in.

Second, the comment concerning XSS state is wrong.  VT-x doesn't manage
host/guest state automatically, but there is provision for "host only" bits to
be set, so the implications are still accurate.

Introduce xstate_compressed_size() to mirror the uncompressed one.  Cross
check it at boot.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v3:
 * Adjust commit message about !XSAVE guests
 * Rebase over boot time cross check
 * Use raw policy
---
 xen/arch/x86/cpuid.c              | 24 ++++++++--------------
 xen/arch/x86/include/asm/xstate.h |  1 +
 xen/arch/x86/xstate.c             | 34 +++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+), 16 deletions(-)

diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 7a38e032146a..a822e80c7ea7 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -330,23 +330,15 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
     case XSTATE_CPUID:
         switch ( subleaf )
         {
-        case 1:
-            if ( !p->xstate.xsavec && !p->xstate.xsaves )
-                break;
-
-            /*
-             * TODO: Figure out what to do for XSS state.  VT-x manages host
-             * vs guest MSR_XSS automatically, so as soon as we start
-             * supporting any XSS states, the wrong XSS will be in context.
-             */
-            BUILD_BUG_ON(XSTATE_XSAVES_ONLY != 0);
-            fallthrough;
         case 0:
-            /*
-             * Read CPUID[0xD,0/1].EBX from hardware.  They vary with enabled
-             * XSTATE, and appropriate XCR0|XSS are in context.
-             */
-            res->b = cpuid_count_ebx(leaf, subleaf);
+            if ( p->basic.xsave )
+                res->b = xstate_uncompressed_size(v->arch.xcr0);
+            break;
+
+        case 1:
+            if ( p->xstate.xsavec )
+                res->b = xstate_compressed_size(v->arch.xcr0 |
+                                                v->arch.msrs->xss.raw);
             break;
         }
         break;
diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h
index bfb66dd766b6..da1d89d2f416 100644
--- a/xen/arch/x86/include/asm/xstate.h
+++ b/xen/arch/x86/include/asm/xstate.h
@@ -109,6 +109,7 @@ void xstate_free_save_area(struct vcpu *v);
 int xstate_alloc_save_area(struct vcpu *v);
 void xstate_init(struct cpuinfo_x86 *c);
 unsigned int xstate_uncompressed_size(uint64_t xcr0);
+unsigned int xstate_compressed_size(uint64_t xstates);
 
 static inline uint64_t xgetbv(unsigned int index)
 {
diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c
index 1b3153600d9c..7b7f2dcaf651 100644
--- a/xen/arch/x86/xstate.c
+++ b/xen/arch/x86/xstate.c
@@ -621,6 +621,34 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0)
     return size;
 }
 
+unsigned int xstate_compressed_size(uint64_t xstates)
+{
+    unsigned int i, size = XSTATE_AREA_MIN_SIZE;
+
+    if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */
+        return 0;
+
+    if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) )
+        return size;
+
+    /*
+     * For the compressed size, every component matters.  Some componenets are
+     * rounded up to 64 first.
+     */
+    xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP);
+    for_each_set_bit ( i, &xstates, 63 )
+    {
+        const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i];
+
+        if ( c->align )
+            size = ROUNDUP(size, 64);
+
+        size += c->size;
+    }
+
+    return size;
+}
+
 struct xcheck_state {
     uint64_t states;
     uint32_t uncomp_size;
@@ -683,6 +711,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new)
                   s->states, &new, hw_size, s->comp_size);
 
         s->comp_size = hw_size;
+
+        xen_size = xstate_compressed_size(s->states);
+
+        if ( xen_size != hw_size )
+            panic("XSTATE 0x%016"PRIx64", compressed hw size %#x != xen size %#x\n",
+                  s->states, hw_size, xen_size);
     }
     else
         BUG_ON(hw_size); /* Compressed size reported, but no XSAVEC ? */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:17:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:17:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728373.1133327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RS-0008Hr-9p; Thu, 23 May 2024 11:17:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728373.1133327; Thu, 23 May 2024 11:17:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6RS-0008HF-5H; Thu, 23 May 2024 11:17:02 +0000
Received: by outflank-mailman (input) for mailman id 728373;
 Thu, 23 May 2024 11:17:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA6RQ-0006Hv-Dp
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:17:00 +0000
Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com
 [2607:f8b0:4864:20::f2f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f81b646d-18f5-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:16:58 +0200 (CEST)
Received: by mail-qv1-xf2f.google.com with SMTP id
 6a1803df08f44-6ab86d61491so7106876d6.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:16:58 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:16:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f81b646d-18f5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716463016; x=1717067816; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BRj9LcaEwZJZDPNXhHku6vQUZRLAf7w0mcjGEgwMX0k=;
        b=lmkq/B2qFVg/LhYiMqdxSiya/WB+LaHF4618OZcsAnio7t3FIjMOdkKiNwdaDskgIJ
         9q+RwRugYlbf2JRHMGmnbRw8IFsMPNTtwE8+BhODLzc+3GT5YjhcrUADCJw5oLwmKg9o
         IaRMhwGlUswKcW+p125vtZhGD1NVEOUO0qWZ4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716463016; x=1717067816;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BRj9LcaEwZJZDPNXhHku6vQUZRLAf7w0mcjGEgwMX0k=;
        b=DkeF4O+En75NZBKqMwsNswsWOLPvMhGzrzDGUWFB0I+jFx95GgqBT9sfyrCCNzfUXr
         +7iBkpFuB107kUr0PRc+CSWgrNfxEe9QU50JifvNDRBrEbcli0K74VRr8SilSo/9mszg
         Mvbt2UW/NRfifeiARd8HX79/aVlIlhyvx6FI3OrIDlGzjfllCE7ETNBWzHh9Pb5TD9mB
         C5aEvSAtVXDW0j1YkAZwfEJEN2QEHGFmBiBmxY+GcdSSXbn3sNuR0kGxEUGT7DmpmAVc
         uV086C9wbcThY3LnGdkqKP/39XB4Uf874ftRw1nw3HxlE/8sXZanDxvLxhTA4b9ivoF1
         1HBw==
X-Gm-Message-State: AOJu0YypQRlPXANNP7eW84r7JIZmNCgNUSEtNaAiFdQSRCoiLgnJflsV
	eB6nspqFm9g448n1Luk/zkMb/k+eFqzu2Aj6WQ5pfW0SM1QXbalW4p+VrZKtG35u8Ul8SY47W71
	qSRA=
X-Google-Smtp-Source: AGHT+IE0eWG609TbIYWGyd/qWXqqj6lCVcrCWA/lJ6+u/z7kOKwE4sZ5dOb3iisX8FOF4omHXz+8Qg==
X-Received: by 2002:a05:6214:5987:b0:6ab:5b6d:f267 with SMTP id 6a1803df08f44-6ab80907622mr53889236d6.62.1716463015904;
        Thu, 23 May 2024 04:16:55 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 7/7] x86/defns: Clean up X86_{XCR0,XSS}_* constants
Date: Thu, 23 May 2024 12:16:27 +0100
Message-Id: <20240523111627.28896-8-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

With the exception of one case in read_bndcfgu() which can use ilog2(),
the *_POS defines are unused.

X86_XCR0_X87 is the name used by both the SDM and APM, rather than
X86_XCR0_FP.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v3:
 * New
---
 xen/arch/x86/i387.c                  |  2 +-
 xen/arch/x86/include/asm/x86-defns.h | 32 ++++++++++------------------
 xen/arch/x86/include/asm/xstate.h    |  4 ++--
 xen/arch/x86/xstate.c                | 18 ++++++++--------
 4 files changed, 23 insertions(+), 33 deletions(-)

diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c
index 7a4297cc921e..fcdee10a6e69 100644
--- a/xen/arch/x86/i387.c
+++ b/xen/arch/x86/i387.c
@@ -369,7 +369,7 @@ void vcpu_setup_fpu(struct vcpu *v, struct xsave_struct *xsave_area,
         {
             v->arch.xsave_area->xsave_hdr.xstate_bv &= ~XSTATE_FP_SSE;
             if ( fcw_default != FCW_DEFAULT )
-                v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_FP;
+                v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_X87;
         }
     }
 
diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h
index d7602ab225c4..3bcdbaccd3aa 100644
--- a/xen/arch/x86/include/asm/x86-defns.h
+++ b/xen/arch/x86/include/asm/x86-defns.h
@@ -79,25 +79,16 @@
 /*
  * XSTATE component flags in XCR0 | MSR_XSS
  */
-#define X86_XCR0_FP_POS           0
-#define X86_XCR0_FP               (1ULL << X86_XCR0_FP_POS)
-#define X86_XCR0_SSE_POS          1
-#define X86_XCR0_SSE              (1ULL << X86_XCR0_SSE_POS)
-#define X86_XCR0_YMM_POS          2
-#define X86_XCR0_YMM              (1ULL << X86_XCR0_YMM_POS)
-#define X86_XCR0_BNDREGS_POS      3
-#define X86_XCR0_BNDREGS          (1ULL << X86_XCR0_BNDREGS_POS)
-#define X86_XCR0_BNDCSR_POS       4
-#define X86_XCR0_BNDCSR           (1ULL << X86_XCR0_BNDCSR_POS)
-#define X86_XCR0_OPMASK_POS       5
-#define X86_XCR0_OPMASK           (1ULL << X86_XCR0_OPMASK_POS)
-#define X86_XCR0_ZMM_POS          6
-#define X86_XCR0_ZMM              (1ULL << X86_XCR0_ZMM_POS)
-#define X86_XCR0_HI_ZMM_POS       7
-#define X86_XCR0_HI_ZMM           (1ULL << X86_XCR0_HI_ZMM_POS)
+#define X86_XCR0_X87              (_AC(1, ULL) <<  0)
+#define X86_XCR0_SSE              (_AC(1, ULL) <<  1)
+#define X86_XCR0_YMM              (_AC(1, ULL) <<  2)
+#define X86_XCR0_BNDREGS          (_AC(1, ULL) <<  3)
+#define X86_XCR0_BNDCSR           (_AC(1, ULL) <<  4)
+#define X86_XCR0_OPMASK           (_AC(1, ULL) <<  5)
+#define X86_XCR0_ZMM              (_AC(1, ULL) <<  6)
+#define X86_XCR0_HI_ZMM           (_AC(1, ULL) <<  7)
 #define X86_XSS_PROC_TRACE        (_AC(1, ULL) <<  8)
-#define X86_XCR0_PKRU_POS         9
-#define X86_XCR0_PKRU             (1ULL << X86_XCR0_PKRU_POS)
+#define X86_XCR0_PKRU             (_AC(1, ULL) <<  9)
 #define X86_XSS_PASID             (_AC(1, ULL) << 10)
 #define X86_XSS_CET_U             (_AC(1, ULL) << 11)
 #define X86_XSS_CET_S             (_AC(1, ULL) << 12)
@@ -107,11 +98,10 @@
 #define X86_XSS_HWP               (_AC(1, ULL) << 16)
 #define X86_XCR0_TILE_CFG         (_AC(1, ULL) << 17)
 #define X86_XCR0_TILE_DATA        (_AC(1, ULL) << 18)
-#define X86_XCR0_LWP_POS          62
-#define X86_XCR0_LWP              (1ULL << X86_XCR0_LWP_POS)
+#define X86_XCR0_LWP              (_AC(1, ULL) << 62)
 
 #define X86_XCR0_STATES                                                 \
-    (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS |     \
+    (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS |    \
      X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM |                 \
      X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG |              \
      X86_XCR0_TILE_DATA |                                               \
diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h
index da1d89d2f416..f4a8e5f814a0 100644
--- a/xen/arch/x86/include/asm/xstate.h
+++ b/xen/arch/x86/include/asm/xstate.h
@@ -29,8 +29,8 @@ extern uint32_t mxcsr_mask;
 #define XSAVE_HDR_OFFSET          FXSAVE_SIZE
 #define XSTATE_AREA_MIN_SIZE      (FXSAVE_SIZE + XSAVE_HDR_SIZE)
 
-#define XSTATE_FP_SSE  (X86_XCR0_FP | X86_XCR0_SSE)
-#define XCNTXT_MASK    (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | \
+#define XSTATE_FP_SSE  (X86_XCR0_X87 | X86_XCR0_SSE)
+#define XCNTXT_MASK    (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | \
                         X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM | \
                         XSTATE_NONLAZY)
 
diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c
index 7b7f2dcaf651..0ed2541665b3 100644
--- a/xen/arch/x86/xstate.c
+++ b/xen/arch/x86/xstate.c
@@ -313,7 +313,7 @@ void xsave(struct vcpu *v, uint64_t mask)
                            "=m" (*ptr), \
                            "a" (lmask), "d" (hmask), "D" (ptr))
 
-    if ( fip_width == 8 || !(mask & X86_XCR0_FP) )
+    if ( fip_width == 8 || !(mask & X86_XCR0_X87) )
     {
         XSAVE("0x48,");
     }
@@ -366,7 +366,7 @@ void xsave(struct vcpu *v, uint64_t mask)
             fip_width = 8;
     }
 #undef XSAVE
-    if ( mask & X86_XCR0_FP )
+    if ( mask & X86_XCR0_X87 )
         ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET] = fip_width;
 }
 
@@ -558,7 +558,7 @@ void xstate_free_save_area(struct vcpu *v)
 static bool valid_xcr0(uint64_t xcr0)
 {
     /* FP must be unconditionally set. */
-    if ( !(xcr0 & X86_XCR0_FP) )
+    if ( !(xcr0 & X86_XCR0_X87) )
         return false;
 
     /* YMM depends on SSE. */
@@ -599,7 +599,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0)
     if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */
         return 0;
 
-    if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) )
+    if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_X87) )
         return size;
 
     /*
@@ -607,7 +607,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0)
      * maximum offset+size.  Some states (e.g. LWP, APX_F) are out-of-order
      * with respect their index.
      */
-    xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP);
+    xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_X87);
     for_each_set_bit ( i, &xcr0, 63 )
     {
         const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i];
@@ -628,14 +628,14 @@ unsigned int xstate_compressed_size(uint64_t xstates)
     if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */
         return 0;
 
-    if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) )
+    if ( xstates <= (X86_XCR0_SSE | X86_XCR0_X87) )
         return size;
 
     /*
      * For the compressed size, every component matters.  Some componenets are
      * rounded up to 64 first.
      */
-    xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP);
+    xstates &= ~(X86_XCR0_SSE | X86_XCR0_X87);
     for_each_set_bit ( i, &xstates, 63 )
     {
         const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i];
@@ -748,7 +748,7 @@ static void __init noinline xstate_check_sizes(void)
      * layout compatibility with Intel and having a knock-on effect on all
      * subsequent states.
      */
-    check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP);
+    check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_X87);
 
     if ( cpu_has_avx )
         check_new_xstate(&s, X86_XCR0_YMM);
@@ -1000,7 +1000,7 @@ uint64_t read_bndcfgu(void)
               : "=m" (*xstate)
               : "a" (X86_XCR0_BNDCSR), "d" (0), "D" (xstate) );
 
-        bndcsr = (void *)xstate + xstate_offsets[X86_XCR0_BNDCSR_POS];
+        bndcsr = (void *)xstate + xstate_offsets[ilog2(X86_XCR0_BNDCSR)];
     }
 
     if ( cr0 & X86_CR0_TS )
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 23 11:36:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:36:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728426.1133337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6kJ-0005km-Rk; Thu, 23 May 2024 11:36:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728426.1133337; Thu, 23 May 2024 11:36:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6kJ-0005kf-Oa; Thu, 23 May 2024 11:36:31 +0000
Received: by outflank-mailman (input) for mailman id 728426;
 Thu, 23 May 2024 11:36:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA6kI-0005kZ-Dk
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:36:30 +0000
Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com
 [2607:f8b0:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b1355f1f-18f8-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:36:28 +0200 (CEST)
Received: by mail-ot1-x333.google.com with SMTP id
 46e09a7af769-6f89b16d081so113046a34.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 04:36:28 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43f993b9653sm25978971cf.70.2024.05.23.04.36.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 04:36:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1355f1f-18f8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716464187; x=1717068987; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=hgI25QredcKhzXFGbYw4fFrt/5Gr4ZzTQrzrpz3tysk=;
        b=O13f/i/UuMRI0nq/Qu6Pd9AHxbnXboAohwaH6Gatjfi/xYUGtJPSbWrONL95mpSqP9
         ztbtENe7qafq/6VRwZpMF/08TO/Hn3jK7vTocpbRuggFNkn4LD/gO4A7e8DDzxkVQo58
         H0LWMPUvJ2WL531djsQ7l6FwAdhMS2YedrQiI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716464187; x=1717068987;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hgI25QredcKhzXFGbYw4fFrt/5Gr4ZzTQrzrpz3tysk=;
        b=oZFrshlX47/lWrOWoz0FDZjR4z75Yo0c6NRQDAZdC/MVqXraaSgMYr858/OAJHrKq7
         8oKT4bJhstGnb4NJ73cP4LCnMNCsEzQ+nWFTp4VbfBKE1AT5AwBINFbax1V2eS0IVNXj
         nx+uWVauZLmJ0l4UdZHC2n1oWbF6tQuttIcPGHbioEl3p+y3n92bNQYHF4n9FM/k48sm
         YnGUkiRB11tHgoS1WO8ZK4CpVE87ZhawOuWbadQcbTZzogPsTY9UudRu8OByRO5bk8ku
         hFOTMTjV09bxZdYmQ6pxT8HK8OgndfdxvgSoMAvjcxND0NLq7jcfPRCitRqBr02zI7C/
         y1ng==
X-Gm-Message-State: AOJu0YxagmBok1UDQv5BuNtKdw/WI7QRv/GETfqag0QS2G2Hq3YfeNAX
	86etVisHSIdZeBw9mn9s8VgbQQ3YfRnU0gSxi1CgpoOfsX2IjxzdaoX6McgQFi8=
X-Google-Smtp-Source: AGHT+IF8yYSGP5VaRxRahCLUK4tN1Th63DqmW6zelgbfZ1DBPfTWH1oeyjVkyY/GV9Mkj3JSMb0qCg==
X-Received: by 2002:a05:6830:142:b0:6ee:ca2:523 with SMTP id 46e09a7af769-6f669afc0ddmr4945949a34.26.1716464186998;
        Thu, 23 May 2024 04:36:26 -0700 (PDT)
Date: Thu, 23 May 2024 13:36:24 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Cc: xen-devel@lists.xenproject.org, Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
Message-ID: <Zk8qOIIYeOtnNe_0@macbook>
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240517133352.94347-3-roger.pau@citrix.com>

On Fri, May 17, 2024 at 03:33:51PM +0200, Roger Pau Monne wrote:
> Enabling it using an HVM param is fragile, and complicates the logic when
> deciding whether options that interact with altp2m can also be enabled.
> 
> Leave the HVM param value for consumption by the guest, but prevent it from
> being set.  Enabling is now done using and additional altp2m specific field in
> xen_domctl_createdomain.
> 
> Note that albeit only currently implemented in x86, altp2m could be implemented
> in other architectures, hence why the field is added to xen_domctl_createdomain
> instead of xen_arch_domainconfig.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v2:
>  - Introduce a new altp2m field in xen_domctl_createdomain.
> 
> Changes since v1:
>  - New in this version.
> ---
>  tools/libs/light/libxl_create.c     | 23 ++++++++++++++++++++++-
>  tools/libs/light/libxl_x86.c        | 26 ++++++++++++--------------
>  tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
>  xen/arch/arm/domain.c               |  6 ++++++

Could I get an Ack from one of the Arm maintainers for the trivial Arm
change?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 11:40:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 11:40:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728430.1133347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6o4-0007SZ-Bn; Thu, 23 May 2024 11:40:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728430.1133347; Thu, 23 May 2024 11:40:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA6o4-0007SS-8k; Thu, 23 May 2024 11:40:24 +0000
Received: by outflank-mailman (input) for mailman id 728430;
 Thu, 23 May 2024 11:40:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA6o3-0007SK-7m
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:40:23 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3c3412aa-18f9-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 13:40:21 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4BA3D22619;
 Thu, 23 May 2024 11:40:20 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B8B813A6C;
 Thu, 23 May 2024 11:40:19 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Joo5DiMrT2baNAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 23 May 2024 11:40:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c3412aa-18f9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716464420; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=fC+G59PsZamCV7EK4H6W6aCmEnvaA6ymHM5xbmMGACA=;
	b=A7hPOCEEuoecXkHX//ude5oI+5emYbame/qJlIqkho7FMXAzJ36/Ub6f07cmQD1L9LVDM2
	Knvq7NoOL8B//Q9nfJnJxyfmdIBtSy0arKoKNefjjEO8P84nJpIPqyRJ+eEed6a1eOqlEu
	dz7VlbNzuToNCtRZJ+/YVJg1UXX/Kh4=
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=A7hPOCEE
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716464420; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=fC+G59PsZamCV7EK4H6W6aCmEnvaA6ymHM5xbmMGACA=;
	b=A7hPOCEEuoecXkHX//ude5oI+5emYbame/qJlIqkho7FMXAzJ36/Ub6f07cmQD1L9LVDM2
	Knvq7NoOL8B//Q9nfJnJxyfmdIBtSy0arKoKNefjjEO8P84nJpIPqyRJ+eEed6a1eOqlEu
	dz7VlbNzuToNCtRZJ+/YVJg1UXX/Kh4=
Message-ID: <c2115e19-9a7e-4c1e-8b39-8e966fa8e193@suse.com>
Date: Thu, 23 May 2024 13:40:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/5] x86/kernel: Move page table macros to new header
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-5-jason.andryuk@amd.com>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240410194850.39994-5-jason.andryuk@amd.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------Zc6wuuxsAmW8Wo6yLRrsVcIm"
X-Spam-Flag: NO
X-Spam-Score: -2.46
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: 4BA3D22619
X-Spam-Level: 
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-2.46 / 50.00];
	SIGNED_PGP(-2.00)[];
	MIME_BASE64_TEXT_BOGUS(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	BAYES_HAM(-0.06)[61.06%];
	XM_UA_NO_VERSION(0.01)[];
	MX_GOOD(-0.01)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	ARC_NA(0.00)[];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RCPT_COUNT_TWELVE(0.00)[13];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	FROM_HAS_DN(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	TO_DN_SOME(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	HAS_ATTACHMENT(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	DKIM_TRACE(0.00)[suse.com:+];
	DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------Zc6wuuxsAmW8Wo6yLRrsVcIm
Content-Type: multipart/mixed; boundary="------------UJSso9HpOaweGeecjyXT1zgr";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Message-ID: <c2115e19-9a7e-4c1e-8b39-8e966fa8e193@suse.com>
Subject: Re: [PATCH 4/5] x86/kernel: Move page table macros to new header
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-5-jason.andryuk@amd.com>
In-Reply-To: <20240410194850.39994-5-jason.andryuk@amd.com>

--------------UJSso9HpOaweGeecjyXT1zgr
Content-Type: multipart/mixed; boundary="------------hIN106LTwnGKxweQq76pnwKe"

--------------hIN106LTwnGKxweQq76pnwKe
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTAuMDQuMjQgMjE6NDgsIEphc29uIEFuZHJ5dWsgd3JvdGU6DQo+IFRoZSBQVkggZW50
cnkgcG9pbnQgd2lsbCBuZWVkIGFuIGFkZGl0aW9uYWwgc2V0IG9mIHByZWJ1aWxkIHBhZ2Ug
dGFibGVzLg0KPiBNb3ZlIHRoZSBtYWNyb3MgYW5kIGRlZmluZXMgdG8gYSBuZXcgaGVhZGVy
IHNvIHRoZXkgY2FuIGJlIHJlLXVzZWQuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBKYXNvbiBB
bmRyeXVrIDxqYXNvbi5hbmRyeXVrQGFtZC5jb20+DQoNCldpdGggdGhlIG9uZSBuaXQgYmVs
b3cgYWRkcmVzc2VkOg0KDQpSZXZpZXdlZC1ieTogSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuY29tPg0KDQouLi4NCg0KPiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYva2VybmVsL3BndGFi
bGVfNjRfaGVscGVycy5oIGIvYXJjaC94ODYva2VybmVsL3BndGFibGVfNjRfaGVscGVycy5o
DQo+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+IGluZGV4IDAwMDAwMDAwMDAwMC4uMGFlODdk
NzY4Y2UyDQo+IC0tLSAvZGV2L251bGwNCj4gKysrIGIvYXJjaC94ODYva2VybmVsL3BndGFi
bGVfNjRfaGVscGVycy5oDQo+IEBAIC0wLDAgKzEsMjggQEANCj4gKy8qIFNQRFgtTGljZW5z
ZS1JZGVudGlmaWVyOiBHUEwtMi4wICovDQo+ICsjaWZuZGVmIF9fUEdUQUJMRVNfNjRfSF9f
DQo+ICsjZGVmaW5lIF9fUEdUQUJMRVNfNjRfSF9fDQo+ICsNCj4gKyNpZmRlZiBfX0FTU0VN
QkxZX18NCj4gKw0KPiArI2RlZmluZSBsNF9pbmRleCh4KQkoKCh4KSA+PiAzOSkgJiA1MTEp
DQo+ICsjZGVmaW5lIHB1ZF9pbmRleCh4KQkoKCh4KSA+PiBQVURfU0hJRlQpICYgKFBUUlNf
UEVSX1BVRC0xKSkNCg0KUGxlYXNlIGZpeCB0aGUgbWlub3Igc3R5bGUgaXNzdWUgaW4gdGhp
cyBsaW5lIGJ5IHMvLS8gLSAvDQoNCg0KSnVlcmdlbg0K
--------------hIN106LTwnGKxweQq76pnwKe
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------hIN106LTwnGKxweQq76pnwKe--

--------------UJSso9HpOaweGeecjyXT1zgr--

--------------Zc6wuuxsAmW8Wo6yLRrsVcIm
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmZPKyIFAwAAAAAACgkQsN6d1ii/Ey/H
JQf+L+3dWxouej2I8WI70EO8E9P452YXvmlPFaNIg/b6l7CgSadl9n8Bs9iBFFr6zPxaYx6YUc7N
eRwu97S45v+AdlK4QOV81mSiYzXFQJBCrm8x/3BReLdcYpY3pAv5XEUs4w3o9xLImhKIMDTa9Wrq
dQZ4kjB/qp3q57x5TRopwrZkMDYro8TKFMZoxRK4sQVQC74DqRLPUkTsktDI0jPfvNKi/3ZfhBp+
Q/RihLMHDvLE2wV92Lc7hvQwyfdRRW0LP1z6FPkET4LQVZhyCkKd7jYQJuqzo3j3/6vAJET7BgmD
6iRGLouPe+6osxfljazdJeLd6Ah6jrA8ohiRVggCRw==
=ZYxv
-----END PGP SIGNATURE-----

--------------Zc6wuuxsAmW8Wo6yLRrsVcIm--


From xen-devel-bounces@lists.xenproject.org Thu May 23 12:05:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 12:05:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728441.1133360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7CF-0003ic-EA; Thu, 23 May 2024 12:05:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728441.1133360; Thu, 23 May 2024 12:05:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7CF-0003iV-BI; Thu, 23 May 2024 12:05:23 +0000
Received: by outflank-mailman (input) for mailman id 728441;
 Thu, 23 May 2024 12:05:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VTls=M2=bounce.vates.tech=bounce-md_30504962.664f30fd.v1-65d8e32aceea4e8482b1da48d10c8cc5@srs-se1.protection.inumbo.net>)
 id 1sA7CD-0003iP-Ux
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 12:05:22 +0000
Received: from mail136-17.atl41.mandrillapp.com
 (mail136-17.atl41.mandrillapp.com [198.2.136.17])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b967d79f-18fc-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 14:05:20 +0200 (CEST)
Received: from pmta11.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1])
 by mail136-17.atl41.mandrillapp.com (Mailchimp) with ESMTP id
 4VlRgx6Gn9zPm0Qtd
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 12:05:17 +0000 (GMT)
Received: from [37.26.189.201] by mandrillapp.com id
 65d8e32aceea4e8482b1da48d10c8cc5; Thu, 23 May 2024 12:05:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b967d79f-18fc-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com;
	s=mte1; t=1716465917; x=1716726417;
	bh=JNvFeMqGoI55P8uBt4zqoVdFN/j6TBeZElOI472f5r0=;
	h=From:Subject:Message-Id:To:Cc:References:In-Reply-To:Feedback-ID:
	 Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date:
	 Subject:From;
	b=yrne4+ruQu503UAP0VGUMslSmP6DelWU31ukUwnH4BJHSxV5ZLE8/eatY728uskkb
	 Epf2reRUCjAAhNvpwSwBBWs5DTfyC5QrpLuGWhHtw/FhK8MjpmwFvjphhdPAiS7ivS
	 HkSvkSUJXVq4hlT1OPIszMF/cgYEDa3ky7sbX+jLAkfjk26QSzyF5XmfXZGwc0L2C/
	 icsxa3Z9nJKvOGPXhAiRrYGs7x8g59BQLzTLBYuhGRAbeYXweow6AiDAI74xBvApUV
	 UZdXATL90zsddgPd6I5Fesguk9/NVIwtzmzQUaMnuSTuZ6x7ooBEP1CISiwGGcJbvj
	 PtZelil88A7Qw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1;
	t=1716465917; x=1716726417; i=teddy.astie@vates.tech;
	bh=JNvFeMqGoI55P8uBt4zqoVdFN/j6TBeZElOI472f5r0=;
	h=From:Subject:Message-Id:To:Cc:References:In-Reply-To:Feedback-ID:
	 Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date:
	 Subject:From;
	b=d6+Lcviq8b2imoOqBti+Ac5C4B2ZzW2e/SrQ6Yn2rDwxNtueZA9a5uK/aBQXQQULr
	 NTJ97ONO1QylvQ2PISfBxpsIeWM0aC1guxEhnjxG7j8Dosd/clhsICFHwCD9w2H7TK
	 USrzs0MP0DnwUBuQU/x0Vc59lOIbZT2h43Bk05ltCRlDfRBekOiWc4SLYiteFL3sqj
	 nkTPBN8mrT0pjk99fb6TUpZGYZ6VO1LmCPu4/1xUzFvvTBz0y7C2R7ax98SHgfpf+a
	 OWJzLrfMfpVlUWWa6ZKdunk2cXwvz7EiuOFT6TogASFRaEM1Zi3Sn4xIQkItu4nUMQ
	 8FptYcsUi8vXg==
From: Teddy Astie <teddy.astie@vates.tech>
Subject: =?utf-8?Q?Re:=20[XEN=20PATCH]=20x86/iommu:=20Conditionally=20compile=20platform-specific=20union=20entries?=
X-Bm-Disclaimer: Yes
X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2
X-Bm-Transport-Timestamp: 1716465916649
Message-Id: <a934c9c3-7de1-4863-aec6-7bdbfbfaff5d@vates.tech>
To: =?utf-8?Q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <2744240f42dadcb1edc69d0f921d5909fe0f63a2.1716455673.git.teddy.astie@vates.tech> <Zk8R7YAhD9UHtVW6@macbook>
In-Reply-To: <Zk8R7YAhD9UHtVW6@macbook>
X-Native-Encoded: 1
X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.65d8e32aceea4e8482b1da48d10c8cc5?=
X-Mandrill-User: md_30504962
Feedback-ID: 30504962:30504962.20240523:md
Date: Thu, 23 May 2024 12:05:17 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Le 23/05/2024 =C3=A0 11:52, Roger Pau Monn=C3=A9 a =C3=A9crit=C2=A0:
> The #ifdef and #endif processor directives shouldn't be indented.
> 
> Would you mind adding /* CONFIG_{AMD,INTEL}_IOMMU */ comments in the
> #endif directives?
>

Sure, will change it for v2.

> I wonder if we could move the definitions of those structures to the
> vendor specific headers, but that's more convoluted, and would require
> including the iommu headers in pci.h

Do you mean moving the vtd/amd union entries to separate structures (e.g 
vtd_arch_iommu) and put them into another file (I don't see any 
vendor-specific headers for this, perhaps create ones ?).

> 
> Thanks, Roger.

Teddy


Teddy Astie | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech



From xen-devel-bounces@lists.xenproject.org Thu May 23 12:11:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 12:11:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728445.1133370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7Hr-0005Ll-0i; Thu, 23 May 2024 12:11:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728445.1133370; Thu, 23 May 2024 12:11:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7Hq-0005Le-U7; Thu, 23 May 2024 12:11:10 +0000
Received: by outflank-mailman (input) for mailman id 728445;
 Thu, 23 May 2024 12:11:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cGfz=M2=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sA7Ho-0005LY-WC
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 12:11:09 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87ec44d9-18fd-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 14:11:06 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 44BFA2267B;
 Thu, 23 May 2024 12:11:05 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D20D013A6C;
 Thu, 23 May 2024 12:11:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id ghWaMVgyT2YufAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 23 May 2024 12:11:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87ec44d9-18fd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716466265; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=yNEl4PpgQKZO5rROFcDvA2Q2IWXapfP6Qa1GqWmmKEg=;
	b=ehR3kYm+INskrct6vxM6/UCoBe9y7KheDC4wNjAJ1v/zZx9LDXSPIQsm7xVTS2wKuxTj0x
	o2xQPutdP6khpOYYu1QXxG3T2K1F5LM4Spm0WjBDd30k1AoR8NDTWSXUfvMFx4ljTjEIBu
	YQLM02wU27QpYbxrZt4DEPnJXmf/z4Q=
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=ehR3kYm+
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716466265; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=yNEl4PpgQKZO5rROFcDvA2Q2IWXapfP6Qa1GqWmmKEg=;
	b=ehR3kYm+INskrct6vxM6/UCoBe9y7KheDC4wNjAJ1v/zZx9LDXSPIQsm7xVTS2wKuxTj0x
	o2xQPutdP6khpOYYu1QXxG3T2K1F5LM4Spm0WjBDd30k1AoR8NDTWSXUfvMFx4ljTjEIBu
	YQLM02wU27QpYbxrZt4DEPnJXmf/z4Q=
Message-ID: <3cb5532e-d9af-4045-99f3-9c8944672073@suse.com>
Date: Thu, 23 May 2024 14:11:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/5] x86/pvh: Add 64bit relocation page tables
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-6-jason.andryuk@amd.com>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240410194850.39994-6-jason.andryuk@amd.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------KkuuWwn3SK0lxQQ4zaeHf4x3"
X-Spam-Level: 
X-Spamd-Result: default: False [-5.40 / 50.00];
	BAYES_HAM(-3.00)[100.00%];
	SIGNED_PGP(-2.00)[];
	MIME_BASE64_TEXT_BOGUS(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	XM_UA_NO_VERSION(0.01)[];
	MX_GOOD(-0.01)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	RCPT_COUNT_TWELVE(0.00)[13];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	ARC_NA(0.00)[];
	FROM_HAS_DN(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	TO_DN_SOME(0.00)[];
	DNSWL_BLOCKED(0.00)[2a07:de40:b281:106:10:150:64:167:received,2a07:de40:b281:104:10:150:64:97:from];
	SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received];
	DKIM_TRACE(0.00)[suse.com:+];
	HAS_ATTACHMENT(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[amd.com:email,suse.com:dkim,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: 44BFA2267B
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Flag: NO
X-Spam-Score: -5.40

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------KkuuWwn3SK0lxQQ4zaeHf4x3
Content-Type: multipart/mixed; boundary="------------5eXTs0qB51N2dWyATTYEzKyl";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jason Andryuk <jason.andryuk@amd.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Message-ID: <3cb5532e-d9af-4045-99f3-9c8944672073@suse.com>
Subject: Re: [PATCH 5/5] x86/pvh: Add 64bit relocation page tables
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-6-jason.andryuk@amd.com>
In-Reply-To: <20240410194850.39994-6-jason.andryuk@amd.com>

--------------5eXTs0qB51N2dWyATTYEzKyl
Content-Type: multipart/mixed; boundary="------------914VC0jSQ884gY3Ozs1Ax8Ki"

--------------914VC0jSQ884gY3Ozs1Ax8Ki
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTAuMDQuMjQgMjE6NDgsIEphc29uIEFuZHJ5dWsgd3JvdGU6DQo+IFRoZSBQVkggZW50
cnkgcG9pbnQgaXMgMzJiaXQuICBGb3IgYSA2NGJpdCBrZXJuZWwsIHRoZSBlbnRyeSBwb2lu
dCBtdXN0DQo+IHN3aXRjaCB0byA2NGJpdCBtb2RlLCB3aGljaCByZXF1aXJlcyBhIHNldCBv
ZiBwYWdlIHRhYmxlcy4gIEluIHRoZSBwYXN0LA0KPiBQVkggdXNlZCBpbml0X3RvcF9wZ3Qu
DQo+IA0KPiBUaGlzIHdvcmtzIGZpbmUgd2hlbiB0aGUga2VybmVsIGlzIGxvYWRlZCBhdCBM
T0FEX1BIWVNJQ0FMX0FERFIsIGFzIHRoZQ0KPiBwYWdlIHRhYmxlcyBhcmUgcHJlYnVpbHQg
Zm9yIHRoaXMgYWRkcmVzcy4gIElmIHRoZSBrZXJuZWwgaXMgbG9hZGVkIGF0IGENCj4gZGlm
ZmVyZW50IGFkZHJlc3MsIHRoZXkgbmVlZCB0byBiZSBhZGp1c3RlZC4NCj4gDQo+IF9fc3Rh
cnR1cF82NCgpIGFkanVzdHMgdGhlIHByZWJ1aWx0IHBhZ2UgdGFibGVzIGZvciB0aGUgcGh5
c2ljYWwgbG9hZA0KPiBhZGRyZXNzLCBidXQgaXQgaXMgNjRiaXQgY29kZS4gIFRoZSAzMmJp
dCBQVkggZW50cnkgY29kZSBjYW4ndCBjYWxsIGl0DQo+IHRvIGFkanVzdCB0aGUgcGFnZSB0
YWJsZXMsIHNvIGl0IGNhbid0IHJlYWRpbHkgYmUgcmUtdXNlZC4NCj4gDQo+IDY0Yml0IFBW
SCBlbnRyeSBuZWVkcyBwYWdlIHRhYmxlcyBzZXQgdXAgZm9yIGlkZW50aXR5IG1hcCwgdGhl
IGtlcm5lbA0KPiBoaWdoIG1hcCBhbmQgdGhlIGRpcmVjdCBtYXAuICBwdmhfc3RhcnRfeGVu
KCkgZW50ZXJzIGlkZW50aXR5IG1hcHBlZC4NCj4gSW5zaWRlIHhlbl9wcmVwYXJlX3B2aCgp
LCBpdCBqdW1wcyB0aHJvdWdoIGEgcHZfb3BzIGZ1bmN0aW9uIHBvaW50ZXINCj4gaW50byB0
aGUgaGlnaG1hcC4gIFRoZSBkaXJlY3QgbWFwIGlzIHVzZWQgZm9yIF9fdmEoKSBvbiB0aGUg
aW5pdHJhbWZzDQo+IGFuZCBvdGhlciBndWVzdCBwaHlzaWNhbCBhZGRyZXNzZXMuDQo+IA0K
PiBBZGQgYSBkZWRpY2F0ZWQgc2V0IG9mIHByZWJ1aWxkIHBhZ2UgdGFibGVzIGZvciBQVkgg
ZW50cnkuICBUaGV5IGFyZQ0KPiBhZGp1c3RlZCBpbiBhc3NlbWJseSBiZWZvcmUgbG9hZGlu
Zy4NCj4gDQo+IEFkZCBYRU5fRUxGTk9URV9QSFlTMzJfUkVMT0MgdG8gaW5kaWNhdGUgc3Vw
cG9ydCBmb3IgcmVsb2NhdGlvbg0KPiBhbG9uZyB3aXRoIHRoZSBrZXJuZWwncyBsb2FkaW5n
IGNvbnN0cmFpbnRzLiAgVGhlIG1heGltdW0gbG9hZCBhZGRyZXNzLA0KPiBLRVJORUxfSU1B
R0VfU0laRSAtIDEsIGlzIGRldGVybWluZWQgYnkgYSBzaW5nbGUgcHZoX2xldmVsMl9pZGVu
dF9wZ3QNCj4gcGFnZS4gIEl0IGNvdWxkIGJlIGxhcmdlciB3aXRoIG1vcmUgcGFnZXMuDQo+
IA0KPiBTaWduZWQtb2ZmLWJ5OiBKYXNvbiBBbmRyeXVrIDxqYXNvbi5hbmRyeXVrQGFtZC5j
b20+DQo+IC0tLQ0KPiBJbnN0ZWFkIG9mIGFkZGluZyA1IHBhZ2VzIG9mIHByZWJ1aWx0IHBh
Z2UgdGFibGVzLCB0aGV5IGNvdWxkIGJlDQo+IGNvbnRydWN0ZWQgZHluYW1pY2FsbHkgaW4g
dGhlIC5ic3MgYXJlYS4gIFRoZXkgYXJlIHRoZW4gb25seSB1c2VkIGZvcg0KPiBQVkggZW50
cnkgYW5kIHVudGlsIHRyYW5zaXRpb25pbmcgdG8gaW5pdF90b3BfcGd0LiAgVGhlIC5ic3Mg
aXMgbGF0ZXINCj4gY2xlYXJlZC4gIEl0J3Mgc2FmZXIgdG8gYWRkIHRoZSBkZWRpY2F0ZWQg
cGFnZXMsIHNvIHRoYXQgaXMgZG9uZSBoZXJlLg0KPiAtLS0NCj4gICBhcmNoL3g4Ni9wbGF0
Zm9ybS9wdmgvaGVhZC5TIHwgMTA1ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KystDQo+ICAgMSBmaWxlIGNoYW5nZWQsIDEwNCBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9u
KC0pDQo+IA0KPiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYvcGxhdGZvcm0vcHZoL2hlYWQuUyBi
L2FyY2gveDg2L3BsYXRmb3JtL3B2aC9oZWFkLlMNCj4gaW5kZXggYzA4ZDA4ZDhjYzkyLi40
YWYzY2ZiY2YyZjggMTAwNjQ0DQo+IC0tLSBhL2FyY2gveDg2L3BsYXRmb3JtL3B2aC9oZWFk
LlMNCj4gKysrIGIvYXJjaC94ODYvcGxhdGZvcm0vcHZoL2hlYWQuUw0KPiBAQCAtMjEsNiAr
MjEsOCBAQA0KPiAgICNpbmNsdWRlIDxhc20vbm9zcGVjLWJyYW5jaC5oPg0KPiAgICNpbmNs
dWRlIDx4ZW4vaW50ZXJmYWNlL2VsZm5vdGUuaD4NCj4gICANCj4gKyNpbmNsdWRlICIuLi9r
ZXJuZWwvcGd0YWJsZV82NF9oZWxwZXJzLmgiDQo+ICsNCj4gICAJX19IRUFEDQo+ICAgDQo+
ICAgLyoNCj4gQEAgLTEwMiw4ICsxMDQsNDcgQEAgU1lNX0NPREVfU1RBUlRfTE9DQUwocHZo
X3N0YXJ0X3hlbikNCj4gICAJYnRzbCAkX0VGRVJfTE1FLCAlZWF4DQo+ICAgCXdybXNyDQo+
ICAgDQo+ICsJbW92ICVlYnAsICVlYngNCj4gKwlzdWJsICRMT0FEX1BIWVNJQ0FMX0FERFIs
ICVlYnggLyogb2Zmc2V0ICovDQo+ICsJanogLkxwYWdldGFibGVfZG9uZQ0KPiArDQo+ICsJ
LyogRml4dXAgcGFnZS10YWJsZXMgZm9yIHJlbG9jYXRpb24uICovDQo+ICsJbGVhbCBydmEo
cHZoX2luaXRfdG9wX3BndCkoJWVicCksICVlZGkNCj4gKwltb3ZsICQ1MTIsICVlY3gNCg0K
UGxlYXNlIHVzZSBQVFJTX1BFUl9QR0QgaW5zdGVhZCBvZiB0aGUgbGl0ZXJhbCA1MTIuIFNp
bWlsYXIgaXNzdWUgYmVsb3cuDQoNCj4gKzI6DQo+ICsJdGVzdGwgJF9QQUdFX1BSRVNFTlQs
IDB4MDAoJWVkaSkNCj4gKwlqeiAxZg0KPiArCWFkZGwgJWVieCwgMHgwMCglZWRpKQ0KPiAr
MToNCj4gKwlhZGRsICQ4LCAlZWRpDQo+ICsJZGVjbCAlZWN4DQo+ICsJam56IDJiDQo+ICsN
Cj4gKwkvKiBMMyBpZGVudCBoYXMgYSBzaW5nbGUgZW50cnkuICovDQo+ICsJbGVhbCBydmEo
cHZoX2xldmVsM19pZGVudF9wZ3QpKCVlYnApLCAlZWRpDQo+ICsJYWRkbCAlZWJ4LCAweDAw
KCVlZGkpDQo+ICsNCj4gKwlsZWFsIHJ2YShwdmhfbGV2ZWwzX2tlcm5lbF9wZ3QpKCVlYnAp
LCAlZWRpDQo+ICsJYWRkbCAlZWJ4LCAoNDA5NiAtIDE2KSglZWRpKQ0KPiArCWFkZGwgJWVi
eCwgKDQwOTYgLSA4KSglZWRpKQ0KDQpQQUdFX1NJWkUgaW5zdGVhZCBvZiA0MDk2LCBwbGVh
c2UuDQoNCj4gKw0KPiArCS8qIHB2aF9sZXZlbDJfaWRlbnRfcGd0IGlzIGZpbmUgLSBsYXJn
ZSBwYWdlcyAqLw0KPiArDQo+ICsJLyogcHZoX2xldmVsMl9rZXJuZWxfcGd0IG5lZWRzIGFk
anVzdG1lbnQgLSBsYXJnZSBwYWdlcyAqLw0KPiArCWxlYWwgcnZhKHB2aF9sZXZlbDJfa2Vy
bmVsX3BndCkoJWVicCksICVlZGkNCj4gKwltb3ZsICQ1MTIsICVlY3gNCj4gKzI6DQo+ICsJ
dGVzdGwgJF9QQUdFX1BSRVNFTlQsIDB4MDAoJWVkaSkNCj4gKwlqeiAxZg0KPiArCWFkZGwg
JWVieCwgMHgwMCglZWRpKQ0KPiArMToNCj4gKwlhZGRsICQ4LCAlZWRpDQo+ICsJZGVjbCAl
ZWN4DQo+ICsJam56IDJiDQo+ICsNCj4gKy5McGFnZXRhYmxlX2RvbmU6DQo+ICAgCS8qIEVu
YWJsZSBwcmUtY29uc3RydWN0ZWQgcGFnZSB0YWJsZXMuICovDQo+IC0JbGVhbCBydmEoaW5p
dF90b3BfcGd0KSglZWJwKSwgJWVheA0KPiArCWxlYWwgcnZhKHB2aF9pbml0X3RvcF9wZ3Qp
KCVlYnApLCAlZWF4DQo+ICAgCW1vdiAlZWF4LCAlY3IzDQo+ICAgCW1vdiAkKFg4Nl9DUjBf
UEcgfCBYODZfQ1IwX1BFKSwgJWVheA0KPiAgIAltb3YgJWVheCwgJWNyMA0KPiBAQCAtMTk3
LDUgKzIzOCw2NyBAQCBTWU1fREFUQV9TVEFSVF9MT0NBTChlYXJseV9zdGFjaykNCj4gICAJ
LmZpbGwgQk9PVF9TVEFDS19TSVpFLCAxLCAwDQo+ICAgU1lNX0RBVEFfRU5EX0xBQkVMKGVh
cmx5X3N0YWNrLCBTWU1fTF9MT0NBTCwgZWFybHlfc3RhY2tfZW5kKQ0KPiAgIA0KPiArI2lm
ZGVmIENPTkZJR19YODZfNjQNCj4gKy8qDQo+ICsgKiBYZW4gUFZIIG5lZWRzIGEgc2V0IG9m
IGlkZW50aXR5IG1hcHBlZCBhbmQga2VybmVsIGhpZ2ggbWFwcGluZw0KPiArICogcGFnZSB0
YWJsZXMuICBwdmhfc3RhcnRfeGVuIHN0YXJ0cyBydW5uaW5nIG9uIHRoZSBpZGVudGl0eSBt
YXBwZWQNCj4gKyAqIHBhZ2UgdGFibGVzLCBidXQgeGVuX3ByZXBhcmVfcHZoIGNhbGxzIGlu
dG8gdGhlIGhpZ2ggbWFwcGluZy4NCj4gKyAqIFRoZXNlIHBhZ2UgdGFibGVzIG5lZWQgdG8g
YmUgcmVsb2NhdGFibGUgYW5kIGFyZSBvbmx5IHVzZWQgdW50aWwNCj4gKyAqIHN0YXJ0dXBf
NjQgdHJhbnNpdGlvbnMgdG8gaW5pdF90b3BfcGd0Lg0KPiArICovDQo+ICtTWU1fREFUQV9T
VEFSVF9QQUdFX0FMSUdORUQocHZoX2luaXRfdG9wX3BndCkNCj4gKwkucXVhZCAgIHB2aF9s
ZXZlbDNfaWRlbnRfcGd0IC0gX19TVEFSVF9LRVJORUxfbWFwICsgX0tFUk5QR19UQUJMRV9O
T0VOQw0KPiArCS5vcmcgICAgcHZoX2luaXRfdG9wX3BndCArIEw0X1BBR0VfT0ZGU0VUKjgs
IDANCg0KUGxlYXNlIGFkZCBhIHNwYWNlIGJlZm9yZSBhbmQgYWZ0ZXIgdGhlICcqJy4NCg0K
PiArCS5xdWFkICAgcHZoX2xldmVsM19pZGVudF9wZ3QgLSBfX1NUQVJUX0tFUk5FTF9tYXAg
KyBfS0VSTlBHX1RBQkxFX05PRU5DDQo+ICsJLm9yZyAgICBwdmhfaW5pdF90b3BfcGd0ICsg
TDRfU1RBUlRfS0VSTkVMKjgsIDANCj4gKwkvKiAoMl40OC0oMioxMDI0KjEwMjQqMTAyNCkp
LygyXjM5KSA9IDUxMSAqLw0KPiArCS5xdWFkICAgcHZoX2xldmVsM19rZXJuZWxfcGd0IC0g
X19TVEFSVF9LRVJORUxfbWFwICsgX1BBR0VfVEFCTEVfTk9FTkMNCj4gK1NZTV9EQVRBX0VO
RChwdmhfaW5pdF90b3BfcGd0KQ0KPiArDQo+ICtTWU1fREFUQV9TVEFSVF9QQUdFX0FMSUdO
RUQocHZoX2xldmVsM19pZGVudF9wZ3QpDQo+ICsJLnF1YWQJcHZoX2xldmVsMl9pZGVudF9w
Z3QgLSBfX1NUQVJUX0tFUk5FTF9tYXAgKyBfS0VSTlBHX1RBQkxFX05PRU5DDQo+ICsJLmZp
bGwJNTExLCA4LCAwDQo+ICtTWU1fREFUQV9FTkQocHZoX2xldmVsM19pZGVudF9wZ3QpDQo+
ICtTWU1fREFUQV9TVEFSVF9QQUdFX0FMSUdORUQocHZoX2xldmVsMl9pZGVudF9wZ3QpDQo+
ICsJLyoNCj4gKwkgKiBTaW5jZSBJIGVhc2lseSBjYW4sIG1hcCB0aGUgZmlyc3QgMUcuDQo+
ICsJICogRG9uJ3Qgc2V0IE5YIGJlY2F1c2UgY29kZSBydW5zIGZyb20gdGhlc2UgcGFnZXMu
DQo+ICsJICoNCj4gKwkgKiBOb3RlOiBUaGlzIHNldHMgX1BBR0VfR0xPQkFMIGRlc3BpdGUg
d2hldGhlcg0KPiArCSAqIHRoZSBDUFUgc3VwcG9ydHMgaXQgb3IgaXQgaXMgZW5hYmxlZC4g
IEJ1dCwNCj4gKwkgKiB0aGUgQ1BVIHNob3VsZCBpZ25vcmUgdGhlIGJpdC4NCj4gKwkgKi8N
Cj4gKwlQTURTKDAsIF9fUEFHRV9LRVJORUxfSURFTlRfTEFSR0VfRVhFQywgUFRSU19QRVJf
UE1EKQ0KPiArU1lNX0RBVEFfRU5EKHB2aF9sZXZlbDJfaWRlbnRfcGd0KQ0KPiArU1lNX0RB
VEFfU1RBUlRfUEFHRV9BTElHTkVEKHB2aF9sZXZlbDNfa2VybmVsX3BndCkNCj4gKwkuZmls
bAlMM19TVEFSVF9LRVJORUwsOCwwDQoNClNwYWNlcyBhZnRlciB0aGUgY29tbWFzLg0KDQo+
ICsJLyogKDJeNDgtKDIqMTAyNCoxMDI0KjEwMjQpLSgoMl4zOSkqNTExKSkvKDJeMzApID0g
NTEwICovDQo+ICsJLnF1YWQJcHZoX2xldmVsMl9rZXJuZWxfcGd0IC0gX19TVEFSVF9LRVJO
RUxfbWFwICsgX0tFUk5QR19UQUJMRV9OT0VOQw0KPiArCS5xdWFkCTAgLyogbm8gZml4bWFw
ICovDQo+ICtTWU1fREFUQV9FTkQocHZoX2xldmVsM19rZXJuZWxfcGd0KQ0KPiArDQo+ICtT
WU1fREFUQV9TVEFSVF9QQUdFX0FMSUdORUQocHZoX2xldmVsMl9rZXJuZWxfcGd0KQ0KPiAr
CS8qDQo+ICsJICogS2VybmVsIGhpZ2ggbWFwcGluZy4NCj4gKwkgKg0KPiArCSAqIFRoZSBr
ZXJuZWwgY29kZStkYXRhK2JzcyBtdXN0IGJlIGxvY2F0ZWQgYmVsb3cgS0VSTkVMX0lNQUdF
X1NJWkUgaW4NCj4gKwkgKiB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2UsIHdoaWNoIGlzIDEgR2lC
IGlmIFJBTkRPTUlaRV9CQVNFIGlzIGVuYWJsZWQsDQo+ICsJICogNTEyIE1pQiBvdGhlcndp
c2UuDQo+ICsJICoNCj4gKwkgKiAoTk9URTogYWZ0ZXIgdGhhdCBzdGFydHMgdGhlIG1vZHVs
ZSBhcmVhLCBzZWUgTU9EVUxFU19WQUREUi4pDQo+ICsJICoNCj4gKwkgKiBUaGlzIHRhYmxl
IGlzIGV2ZW50dWFsbHkgdXNlZCBieSB0aGUga2VybmVsIGR1cmluZyBub3JtYWwgcnVudGlt
ZS4NCj4gKwkgKiBDYXJlIG11c3QgYmUgdGFrZW4gdG8gY2xlYXIgb3V0IHVuZGVzaXJlZCBi
aXRzIGxhdGVyLCBsaWtlIF9QQUdFX1JXDQo+ICsJICogb3IgX1BBR0VfR0xPQkFMIGluIHNv
bWUgY2FzZXMuDQo+ICsJICovDQo+ICsJUE1EUygwLCBfX1BBR0VfS0VSTkVMX0xBUkdFX0VY
RUMsIEtFUk5FTF9JTUFHRV9TSVpFL1BNRF9TSVpFKQ0KDQpTcGFjZXMgYXJvdW5kICcvJy4N
Cg0KPiArU1lNX0RBVEFfRU5EKHB2aF9sZXZlbDJfa2VybmVsX3BndCkNCj4gKw0KPiArCUVM
Rk5PVEUoWGVuLCBYRU5fRUxGTk9URV9QSFlTMzJfUkVMT0MsDQo+ICsJCSAgICAgLmxvbmcg
Q09ORklHX1BIWVNJQ0FMX0FMSUdOOw0KPiArCQkgICAgIC5sb25nIExPQURfUEhZU0lDQUxf
QUREUjsNCj4gKwkJICAgICAubG9uZyBLRVJORUxfSU1BR0VfU0laRSAtIDEpDQo+ICsjZW5k
aWYNCj4gKw0KPiAgIAlFTEZOT1RFKFhlbiwgWEVOX0VMRk5PVEVfUEhZUzMyX0VOVFJZLA0K
PiAgIAkgICAgICAgICAgICAgX0FTTV9QVFIgKHB2aF9zdGFydF94ZW4gLSBfX1NUQVJUX0tF
Uk5FTF9tYXApKQ0KDQoNCkp1ZXJnZW4NCg==
--------------914VC0jSQ884gY3Ozs1Ax8Ki
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------914VC0jSQ884gY3Ozs1Ax8Ki--

--------------5eXTs0qB51N2dWyATTYEzKyl--

--------------KkuuWwn3SK0lxQQ4zaeHf4x3
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmZPMlgFAwAAAAAACgkQsN6d1ii/Ey8b
PAf+IDezaEERt6XB02Yo+3fUEa5yR0T/swqwWR+NkR5RaVD/saRfK4uOdvgYaBl4CkORZIvtN1pj
pNPcYFoH8tZ4COqXNM0NZY6oqVvuuJyaI8C7B4WlcU0P742nGtJ67Dr6tIQugUpwFkbz0xxD3KVs
VdYATMWBiqXAfGEqWUcflDvuSaY6D4V0LltJSTLU2kUF0sf69E+A5wNrKVXEwl4Vp4ZgZHfDwMLf
eqEJSawP4Z2ryR7qOBuO/dLSlqOS2Kn55emx8ifFb+CzN412yM/JL1wzNUijEJyaOlCa6lbUJgEt
PgXMiXqISVk+Qn6aKzpj+iQajMT/mJSFnISFcnODIg==
=KXWs
-----END PGP SIGNATURE-----

--------------KkuuWwn3SK0lxQQ4zaeHf4x3--


From xen-devel-bounces@lists.xenproject.org Thu May 23 12:38:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 12:38:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728452.1133380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7ia-0001RB-34; Thu, 23 May 2024 12:38:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728452.1133380; Thu, 23 May 2024 12:38:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA7iZ-0001R4-Vm; Thu, 23 May 2024 12:38:47 +0000
Received: by outflank-mailman (input) for mailman id 728452;
 Thu, 23 May 2024 12:38:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sA7iY-0001Qy-M6
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 12:38:46 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA7iX-0005Zy-1e; Thu, 23 May 2024 12:38:45 +0000
Received: from [15.248.2.28] (helo=[10.24.67.32])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA7iW-0004Kz-NO; Thu, 23 May 2024 12:38:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=dDnu+CLFVajFDCtc2+ScvRfwxDR3nWuOYJVtpUlniaY=; b=rr5y2vc7SdG8S15afeyAubUciK
	/6TpbzzubZhqc7QgZx3IHH2rwUczqdU/q5Labur7806hO0pmxiS5b0tlfg8sUBAacNuxeuiIr2+w/
	NxZ6lzPK+qd1oSU7jwEQxDtj34SLB7pETEzv57NMGDuOUFVe4N0sfUa32Nqs/TZJ5wDs=;
Message-ID: <94484f5f-fdcd-4c6a-ad34-5dbe7cc4eda0@xen.org>
Date: Thu, 23 May 2024 13:38:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [for-4.19] Re: [XEN PATCH v3] arm/mem_access: add conditional
 build of mem_access.c
Content-Language: en-GB
To: "Oleksii K." <oleksii.kurochko@gmail.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>
References: <b3f03c4f5a78b86b01750f10bb0cebcdb2fd35cc.1715265720.git.alessandro.zucchelli@bugseng.com>
 <CABfawh=7jZmtseCm9yaZUcKkK7i+yYiudVDvTFeF6bax_1+BjA@mail.gmail.com>
 <fb426cc8-fda9-4e21-983d-37649499e0e4@xen.org>
 <a9fdb75d26839c31a8b0ceda9c940b2e13a89bab.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a9fdb75d26839c31a8b0ceda9c940b2e13a89bab.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 23/05/2024 09:04, Oleksii K. wrote:
> On Wed, 2024-05-22 at 21:50 +0100, Julien Grall wrote:
>> Hi,
>>
>> Adding Oleksii as the release manager.
>>
>> On 22/05/2024 19:27, Tamas K Lengyel wrote:
>>> On Fri, May 10, 2024 at 8:32 AM Alessandro Zucchelli
>>> <alessandro.zucchelli@bugseng.com> wrote:
>>>>
>>>> In order to comply to MISRA C:2012 Rule 8.4 for ARM the following
>>>> changes are done:
>>>> revert preprocessor conditional changes to xen/mem_access.h which
>>>> had it build unconditionally, add conditional build for
>>>> xen/mem_access.c
>>>> as well and provide stubs in asm/mem_access.h for the users of
>>>> this
>>>> header.
>>>>
>>>> Signed-off-by: Alessandro Zucchelli
>>>> <alessandro.zucchelli@bugseng.com>
>>>
>>> Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
>>
>> Oleksii, would you be happy if this patch is committed for 4.19?
> Sure:
>   Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Thanks. It is now committed.

> 
>>
>> BTW, do you want to be release-ack every bug until the hard code
>> freeze?
>> Or would you be fine to levea the decision to the maintainers?
> I would prefer to leave the decision to the maintainers.

Ok. I will keep it in mind for the bug fixes until the hard code.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 13:00:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 13:00:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728464.1133394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA83K-0006Ca-Ru; Thu, 23 May 2024 13:00:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728464.1133394; Thu, 23 May 2024 13:00:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA83K-0006CT-PG; Thu, 23 May 2024 13:00:14 +0000
Received: by outflank-mailman (input) for mailman id 728464;
 Thu, 23 May 2024 13:00:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sA83J-0006BW-T2
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 13:00:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA83I-0005yG-SB; Thu, 23 May 2024 13:00:12 +0000
Received: from [15.248.2.28] (helo=[10.24.67.32])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA83I-0005hV-Iy; Thu, 23 May 2024 13:00:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=FnTUlkQqTxBxJc0M0YOmtLiam2iKBFMGtmRiJTAL9Ls=; b=REy2P/ulyVRf8+8ybC0rRMv/cw
	LPVlGO9eu0aT+oCbsAjLg6H0NKPGobSkYHRPFE2Q5dJFJMfL+G+02K2mFq0iobIOdp21wQsumO6Nb
	xNe1VsfsIghnL7Y+k+dZubcO94LsOvKj78/5omi9uXp+xQSD70xcNOGormY1X12k2kEY=;
Message-ID: <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
Date: Thu, 23 May 2024 14:00:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 17/05/2024 14:54, Oleksii Kurochko wrote:
> diff --git a/xen/arch/arm/arm64/livepatch.c b/xen/arch/arm/arm64/livepatch.c
> index df2cebedde..4bc8ed9be5 100644
> --- a/xen/arch/arm/arm64/livepatch.c
> +++ b/xen/arch/arm/arm64/livepatch.c
> @@ -10,7 +10,6 @@
>   #include <xen/mm.h>
>   #include <xen/vmap.h>
>   
> -#include <asm/bitops.h>

It is a bit unclear how this change is related to the patch. Can you 
explain in the commit message?

>   #include <asm/byteorder.h>
>   #include <asm/insn.h>
>   #include <asm/livepatch.h>
> diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
> index 5104334e48..8e16335e76 100644
> --- a/xen/arch/arm/include/asm/bitops.h
> +++ b/xen/arch/arm/include/asm/bitops.h
> @@ -22,9 +22,6 @@
>   #define __set_bit(n,p)            set_bit(n,p)
>   #define __clear_bit(n,p)          clear_bit(n,p)
>   
> -#define BITOP_BITS_PER_WORD     32
> -#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
> -#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
>   #define BITS_PER_BYTE           8

OOI, any reason BITS_PER_BYTE has not been moved as well? I don't expect 
the value to change across arch.

[...]

> diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
> index f14ad0d33a..6eeeff0117 100644
> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned long x)
>    * scope
>    */
>   
> +#define BITOP_BITS_PER_WORD 32
> +typedef uint32_t bitop_uint_t;
> +
> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
> +
> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> +
> +extern void __bitop_bad_size(void);
> +
> +#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
> +
>   /* --------------------- Please tidy above here --------------------- */
>   
>   #include <asm/bitops.h>
>   
> +/**
> + * generic__test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */

Sorry for only mentioning this on v10. I think this comment should be 
duplicated (or moved to) on top of test_bit() because this is what 
everyone will use. This will avoid the developper to follow the function 
calls and only notice the x86 version which says "This function is 
atomic and may not be reordered." and would be wrong for all the other arch.

> +static always_inline bool
> +generic__test_and_set_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old | mask;
> +    return (old & mask);
> +}
> +
> +/**
> + * generic__test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */

Same applies here and ...

> +static always_inline bool
> +generic__test_and_clear_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old & ~mask;
> +    return (old & mask);
> +}
> +
> +/* WARNING: non atomic and it can be reordered! */

... here.

> +static always_inline bool
> +generic__test_and_change_bit(int nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old ^ mask;
> +    return (old & mask);
> +}
> +/**
> + * generic_test_bit - Determine whether a bit is set
> + * @nr: bit number to test
> + * @addr: Address to start counting from
> + */
> +static always_inline bool generic_test_bit(int nr, const volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    const volatile bitop_uint_t *p =
> +                        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +
> +    return (*p & mask);
> +}
> +
> +static always_inline bool
> +__test_and_set_bit(int nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_set_bit
> +#define arch__test_and_set_bit generic__test_and_set_bit
> +#endif
> +
> +    return arch__test_and_set_bit(nr, addr);
> +}

NIT: It is a bit too late to change this one. But I have to admit, I 
don't understand the purpose of the static inline when you could have 
simply call...

> +#define __test_and_set_bit(nr, addr) ({             \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> +    __test_and_set_bit(nr, addr);                   \

... __arch__test_and_set_bit here.


The only two reasons I am not providing an ack is the:
  * Explanation for the removal of asm/bitops.h in livepatch.c
  * The placement of the comments

There are not too important for me.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 13:07:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 13:07:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728477.1133404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8Ab-00077P-KT; Thu, 23 May 2024 13:07:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728477.1133404; Thu, 23 May 2024 13:07:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8Ab-00077I-GE; Thu, 23 May 2024 13:07:45 +0000
Received: by outflank-mailman (input) for mailman id 728477;
 Thu, 23 May 2024 13:07:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Oh0r=M2=epam.com=prvs=18734d070d=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1sA8Aa-000772-LN
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 13:07:44 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fd8825a-1905-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 15:07:42 +0200 (CEST)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44NB9eXr024421;
 Thu, 23 May 2024 13:07:31 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3y9y239xwq-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 May 2024 13:07:31 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by PA4PR03MB7181.eurprd03.prod.outlook.com (2603:10a6:102:103::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Thu, 23 May
 2024 13:07:28 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Thu, 23 May 2024
 13:07:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fd8825a-1905-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RAwhCB8I8mXmKSQgzrvkW03a9MhgJRlRLNA2gng4nD1AkI8+zqehMnvVsxQ+YfVqFGIDgjpUeXrQxECtMXnha0za6/1yaH9XmfAAL25hDMvApVHenHCc6ACP4aGCh6pizb1PqEKLa0O2/i9marDXBJjN0aouwcy2utK6nJkKIGqEkik8vQTJxIwGmZboaSl1XR6gYZEBwWd4nLNoOMExOXpN1jJ0X6+Lrq7nj0b40hjrK5B5Kfr9BlSVgFRR5NBCs9Jp0gZbc21tPx/Sjm3UHqxsqtqZxhnbD7vKBW42YR65uEm97L+tDREeeNOiPkt2wft1pTm8DBGFkYjNigPl+w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=otKw66nEx7G9CUalIzd5MALBbDsQBfCNiIO9pmHTUm4=;
 b=EzwK3KUsYLBrDshqlnUcGqRIkQu8muHRBVtTrQ0AmjcZg8f8kWuvETqldlhe86NvRiEH2Oc8COYM5qdkqVrXr4ivIaygvNDGr71L5DPNJRovXgf33S/tTXEVu1sJiv+IPH6HE06z7DxMRaGexYp70V4i446/08/p4b2sSDw1fNtf5fHmJSGn1ggfRweiRIFZ7kRXoZdiAMb6bg773IVhJIO3ORWqdzDdduDWoLKN4X5Ez7DsLRjp3aQ+vyWD7NfrMWuba+j53/diLny/0e1qUollEOx//sSSVViu5H7Jem+LmsSPrbHHQdoIsbE2oszVcQULSCHbMMyX/91akGAk9g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=otKw66nEx7G9CUalIzd5MALBbDsQBfCNiIO9pmHTUm4=;
 b=sqDf6BCzkf5JuvxwO6sSJHd/6/v1MP3+Z0dDv2NaT23nL8Ywxj+ry5Jr3Xans6mQVC7bqWI5lOjQpo8AjMDI3wGZxU4PPWy6rjshzaXxJh/fpAmz60ez+Nv6jHY32/e8RjBIxdJXsRjzdj70/q7kMXeOUjiCc0N9QZUoWlduIG1dqD5TIVw/KvSspIxGTnAM5PoNTuYsBqWCr64azEyteIwcRsSPnW8rCW5LiAG7u6W3PkiaGMXs/k83j88Arcb7EGp96Wn2ZcPyv89ufRpDmUVI1EdXAbILb7kBKm/DTTD5IFsp5KqZNBG88xRtIIyuzEGML28w0ytTC1YHMsYOTg==
Message-ID: <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
Date: Thu, 23 May 2024 16:07:26 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
        xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
 <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA2P291CA0020.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1e::9) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|PA4PR03MB7181:EE_
X-MS-Office365-Filtering-Correlation-Id: 91e06059-3532-4e63-0b43-08dc7b294c23
X-LD-Processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?bEV2MDQxQmZPd3A2cWFMNkJmVVpDT2tXNW0wZFM4OVZWaERmbUJmUytyL0c5?=
 =?utf-8?B?NHRTRXRsS3ZJbDlTVXpCOG9LTi9VcDBPdHY5dFUzM09OQWoyN3p0Q1JKaS9Z?=
 =?utf-8?B?TzlqbjVLWkNoeTJoVkpwV0pqblY2dTZxT3gzTjN1NjcrYWtsMDdTVHFmQmFH?=
 =?utf-8?B?ZGVNcEJ3VU0zL1pWb1VrNFAyZW0yTjNGY3dUZHVQa3JBZ1J5QUM1WGlkZlIw?=
 =?utf-8?B?UWpmdUNnWEtwNHNydTF3ZGRuUmNGakFjUlV4dW93UGtDVXN2dWR6QmdiTjA4?=
 =?utf-8?B?Uk1aYWNOcDdOK29sQWhTZ0dyWnZXdVNnU3RZM3VrNzFjdTB0TDYwYTlWaTR4?=
 =?utf-8?B?SFNKd1hqZksyaGZnRmVtNWcxcnNGbWNIbk0yK0d5RDUwMklFQVlSK2JXeGc0?=
 =?utf-8?B?OWlQaUZFeWp6OG5XTzd4ck1TWGhpTTE2cno2SFlWbmhWcytoOHpWQmhycEpa?=
 =?utf-8?B?RUJqNUNPQ0hzSkFBNE00UVNtVlZaVWttTUZoWXcwcTFJaWd0bXJNMjJWZW81?=
 =?utf-8?B?TFo0Z0tEZ1R1aXhsNHl6a3Z5c25DWldwbU53VkpINm4zK2ZEWGxhK3F2b3cx?=
 =?utf-8?B?U1grWHI1Vis2aU55ZVpDQUlqNVREWkxkY2dFZG8yR1FZYlRhUVZtWTMvZnZV?=
 =?utf-8?B?NjBjL0Q5OTZFZ3lKaEU3NU11bXJVb2l5QWNGcmp5ejBDaHZEWEFzKzFSTFVI?=
 =?utf-8?B?MklLdks4WndIZnRaVG0rNjhwd3VTY2d3dVJLdERNeWlQdGIzVFVHeTVXTnY4?=
 =?utf-8?B?d1BYZm1qRGIzUk52SDNKaUtxbGxZVDBPQ3pkSnpVMTg2Uzl0Q0VpMTdNcDhG?=
 =?utf-8?B?eDR4Vkg2ZGYwT3AzTHVKa1RWaG5WL0xHeGRxS0NTUlQrOVZ5Znc2UUtyWFNx?=
 =?utf-8?B?b2t2dm9IN1NQUG83a0lFd3FiWTVmUFNKMlVyYUFWZnJiUjVhUVVyQ1dPcWFn?=
 =?utf-8?B?Vk5rNjF5NFp0TGY3VHZscEltSEhkcXdwQ1lScWtNT1RRRUYxNHhJdXFGd1Iy?=
 =?utf-8?B?aUU1YThXMzFndzY2Z0xGN1IvRmVSbTAxL3FhWWVISng2ZEUrWU1vNEplUFRp?=
 =?utf-8?B?TzAxY1JBWmpKUjZWUHp3NHdWRytZd3lxbEQ1YlJHUEM1cFhFbFlTelcwOEkv?=
 =?utf-8?B?K29MZEJlMzBLeWN6ZVV1S0Z1NXd3K3ZGVHJzSHJVVXVRbTJqV29IdmVwYnpo?=
 =?utf-8?B?dmNsZzlmSitBSHM1OU1ta2NlWEoxL25Mb0lNN1BRZm9nd3d4ck5GejNQMS8r?=
 =?utf-8?B?UnR4dXJ3dng5SDRWamtHKzNnSTJ5ZUdmdk1PR3QvemRQKzFPUTdlTytEK3M4?=
 =?utf-8?B?SWxzbk9oc0FkM09wR1NzcjJpdkx5c09jRUNIcGJpajBpSVR1TVRkUGh2N2Mr?=
 =?utf-8?B?L2lGU2o1MUpOR2U1elYyVzkzM0JiLzJxTEEvVmRjM1VHNlpaNzJWNzRQVEoy?=
 =?utf-8?B?ODl3MXQ5OVVzRjhXSjV0bFNBNjVzK2VlSjV1VENUVzVxQnVKVE1BNnFxOEpt?=
 =?utf-8?B?alhoWUZHenVtYng1cUdKRGRUNEpyWk4rRXVJbUoxMlJDVGg5QmFSUGNidVlY?=
 =?utf-8?B?cVI0M1pYUzVuOUlmY0dXcXpCanlWbUtTWmxCcFhEdStLSDU2cExwOEljQms3?=
 =?utf-8?Q?TAkSuFNEKlA/hXrHa4Ut8GvU/Zo5pfOJwA9duzsrcyBM=3D?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UVc4WnU0dWlqWDhjUHJWWVUxTnRvQTNWdGhJUWM3eWVSdVpTSE9TRHZMU3Ns?=
 =?utf-8?B?aDFuUi9KcW1GdnREQkQ3VHBnNVI1TFFrNi9iS0dmQWM1cVFwdkVDbjM1V2lW?=
 =?utf-8?B?UFV0MmxyNGw4QlUrUTZSS1Ftb0wvY1BhNS9ydUczOUZ6T1hDUGtjZUxSRmts?=
 =?utf-8?B?aHh0ZkhsTmM2N0l6WDcyYkRxL01la3hHdnpRUkNOYWhNQmxoOUZVdVVlS2FW?=
 =?utf-8?B?L0RWbHdUTTM5amFlT3l5T0h0RU9sYUVaVTBoTTBRYXI0UDNqSlg2NW1ST1VE?=
 =?utf-8?B?ckg0M3YzWXE2cXJaYXUvWXp5MFE4a2xwc2hvTXBTS2o2VXM0OFZFS1dpVGQ1?=
 =?utf-8?B?cFNQYjZkdG1KaE91NE0zZy9Uc0xtU01iNWVITytvMG9VMkpGMGYySVR2MTlB?=
 =?utf-8?B?WXh6TVZ4bmxOa3lSK2dSNjdEWEw1bHc4NEhOTDRqcU1xcEFBdG9ibERuU1Rl?=
 =?utf-8?B?RHY2ZmRWMkJndEE5N0ZxcmJnRlp4a0RORWpaQ3NkMC9MUk4zK0EybitnQnFy?=
 =?utf-8?B?Tmo3QnhmcXlqT293OVBTd2F6VnIwTGprL3BDZVNOdEdZUWFKYjY2dDNPUDRo?=
 =?utf-8?B?ZWlzaWMrRXBoVlRTNXhNNUlZM0sxMktTVTdicVZ4WjdTWUpnVE1jcDVKUkhN?=
 =?utf-8?B?eFF3VlZOZVBnQzlzcC8xWEdlOFBYaVJRTi9nNTExM0hsRnRHbnlrZ3pCaTZq?=
 =?utf-8?B?bTZDa04yNEVtdk5YMEhNU1hZOWFEeHVjL3AwcnlzVzdjTTdlS1VEUC9TR3dN?=
 =?utf-8?B?d1RKZnpRdDM0c3BWZFp3cTluUVBDcTFGSEJmTU9RK3RYOXcrVUZGQzd0bEtF?=
 =?utf-8?B?VnIxamdKdmRGUVQyTHIzaElocXpLRTNoTmljaUI3bUQyMGI2cTlqMng2ZFIy?=
 =?utf-8?B?bEY1TmEwcXozN2ZOS09WTTJteU1BL0pqZHEycWlqOXNwZitYVEdvVk5wS0d5?=
 =?utf-8?B?UDVpbUU5b0phcEYwbkhERE5PQi84UlVDSkJUbDZtTENMckp4MFgrZnZMNG9h?=
 =?utf-8?B?WVIrcjVKQjh3VXVzemU5a29aeS9ac3J4bGFROUt1WVhORWswTndEWmlJZE9z?=
 =?utf-8?B?VzF5WUs5amp0cjY1U3pYd2l3dnJmQkM5WEZUc3gzZDdsKzZMZklISS83Nk9N?=
 =?utf-8?B?MkUwVWxGR25vTEU1UEtaYW9SUzZJRlFVV1RUV1pMU055WlMxY0luL3pEekRr?=
 =?utf-8?B?ZGFRVVQwMkFzMmFGUSsyc01VMC94aUUxcGhod0lNTk02TUR0RnJERlQ0MHBn?=
 =?utf-8?B?cHJDbWZSc3ZFaEtvWGNnbXEyWWlnSUZUMFVQTytXbzFQNTNLT1VSUmVhMU03?=
 =?utf-8?B?TUhmTis1czV0MkJqTUU5OXhjRUJwckpjK3BDdTlnc1ZqYUMrUnAxWEZJdjRr?=
 =?utf-8?B?ellIcVMvNWlISnAxS0d6RE0rVzlYWndHTzh1NU9kQzBabmxORGkwTkxMRllL?=
 =?utf-8?B?d29CZmVMZkY0N2h1eHdEZDZtMlZ2MnNnVnJ6UVlINHE0OWZ0REU3MDhHTk9m?=
 =?utf-8?B?T0tWeUY0cGxHZHBpbE9XUDZMaVhFelVmZGt6YXorWWFZZUx1bWp5Vmk4VVhQ?=
 =?utf-8?B?a29QOUxJcGNnV1VaRTg1b2Fac3NXajB0TkoyZ2VHMmhxMVZWRkVYTG40b0dT?=
 =?utf-8?B?R0hpeE5COHpiM2UxSW15aVc2NXdGRjFlS242RitscWg1aWtPQlg0TWprQ1hT?=
 =?utf-8?B?Z3I5VjVSSjRIMUMzb0pVZmdodlFFRTRlWTVVQXRWMTNKWlBNdDI3ckJseG8y?=
 =?utf-8?B?Ty9vQ3Z5MHl1TXN2ZjFmUWNicCtWb2Z3T0lRb2ovRXpMbXk5SFU2c3lzQUFB?=
 =?utf-8?B?UWIxeVBlTEVkZHEyV0ljbXVmWlVtZlpWWFhpWWQyL2NiYzl5VkRCKy9McFhp?=
 =?utf-8?B?YkxGU3JJaWpYTHFaV084Rmt3cGErSURxVEM2dXNzY1FCOXdqUHlrUlF1SnVB?=
 =?utf-8?B?eEVjbkdna2NiNGNRNFNDZzNaWTg3L3I4aVBqVDlEQWlwenVwN25DbFRrWTF4?=
 =?utf-8?B?ZExDalFQSTVsWTFzdGY1YkV5Znk2eUNjWCtiam1KS2NkN3JZR28vMjd0TFVs?=
 =?utf-8?B?a3drZ0Q1MXhLaUF5T01EamhUZW5aek9nWEpvY0lSOU1qV2RnS011QUxJQVQ2?=
 =?utf-8?Q?aX83l9UNSg1dClR6JtoAso4/u?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 91e06059-3532-4e63-0b43-08dc7b294c23
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 13:07:28.2451
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: D7cAtba5OWZHj+K/sapVlkykDE7wF3gz64S0rOYiFcCxIrgbbIQ1t5iDm6+1xN5LUJ4rWpDjiMn5rP/Jq/113A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR03MB7181
X-Proofpoint-ORIG-GUID: nTFZY3rbtKpQtvBJyyedS5loxGuQJ78I
X-Proofpoint-GUID: nTFZY3rbtKpQtvBJyyedS5loxGuQJ78I
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-23_07,2024-05-23_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 bulkscore=0 spamscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0
 impostorscore=0 malwarescore=0 phishscore=0 suspectscore=0 mlxlogscore=999
 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405010000 definitions=main-2405230090

16.05.24 14:12, Jan Beulich:
> On 15.05.2024 11:12, Sergiy Kibrik wrote:
>> --- a/xen/arch/x86/include/asm/cpufeature.h
>> +++ b/xen/arch/x86/include/asm/cpufeature.h
>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>   #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>>   #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>>   #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
>> +                                  boot_cpu_has(X86_FEATURE_VMX))
>>   #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>>   #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>>   #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>   
>>   /* CPUID level 0x80000001.ecx */
>>   #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
>> +                                  boot_cpu_has(X86_FEATURE_SVM))
>>   #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>>   #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>>   #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
> 
> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
> and as a result one-off indentation on the wrapped lines) I'm not really
> certain we can do this. The description goes into detail why we would want
> this, but it doesn't cover at all why it is safe for all present (and
> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
> just to derive further knowledge from that, without them being directly
> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
> for example. While it looks to be okay there, it may give you an idea of
> what I mean.
> 
> Things might become better separated if instead for such checks we used
> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
> that's still pretty far out, I'm afraid.
> 

I've followed a suggestion you made for patch in previous series:

https://lore.kernel.org/xen-devel/8fbd604e-5e5d-410c-880f-2ad257bbe08a@suse.com/

yet if this approach can potentially be unsafe (I'm not completely sure 
it's safe), should we instead fallback to the way it was done in v1 
series? I.e. guard calls to vmx/svm-specific calls where needed, like in 
these 3 patches:

1) 
https://lore.kernel.org/xen-devel/20240416063328.3469386-1-Sergiy_Kibrik@epam.com/

2) 
https://lore.kernel.org/xen-devel/20240416063740.3469592-1-Sergiy_Kibrik@epam.com/

3) 
https://lore.kernel.org/xen-devel/20240416063947.3469718-1-Sergiy_Kibrik@epam.com/


   -Sergiy


From xen-devel-bounces@lists.xenproject.org Thu May 23 13:21:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 13:21:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728487.1133414 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8Nw-0001nb-KA; Thu, 23 May 2024 13:21:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728487.1133414; Thu, 23 May 2024 13:21:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8Nw-0001nU-HY; Thu, 23 May 2024 13:21:32 +0000
Received: by outflank-mailman (input) for mailman id 728487;
 Thu, 23 May 2024 13:21:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sA8Nu-0001nO-Vl
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 13:21:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA8No-0006JT-Vs; Thu, 23 May 2024 13:21:24 +0000
Received: from [15.248.2.28] (helo=[10.24.67.32])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA8No-0006iV-K4; Thu, 23 May 2024 13:21:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Gx66I4BrAdQd+feteMIxgc9xH71WxYCBsj3qSxHZUhs=; b=FRicO7bSvLrCPgUinRzfn2OG6J
	CZCaBbTOUiUY0CQVb4QNJz1I6rJaPZjlMVnX0/d/DO0/EItqyLhIdIUrALIiJZ3ILUMFWdmf97oUs
	EJvTtKmf3yWqddxRNiYRm4ELYYXpAP9PoDKPv4zVRQwFlPQA063qfIYZj2PdxsNE1/BA=;
Message-ID: <e9ad99ed-bdff-4acf-9f08-8819b4cbdb7c@xen.org>
Date: Thu, 23 May 2024 14:21:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 03/14] xen/bitops: implement fls{l}() in common logic
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, "Oleksii K."
 <oleksii.kurochko@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <43df611d1c7b7ac16b181fb819f5d886daa31bad.1715952103.git.oleksii.kurochko@gmail.com>
 <d26a4033-8cd1-4a9c-8549-49a36b7d52fb@suse.com>
 <9e59da99eb87a7d3fcd8cc6a3c6d6ef2753736af.camel@gmail.com>
 <df910cf3-b784-48a6-a0af-32527302fb9f@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <df910cf3-b784-48a6-a0af-32527302fb9f@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 22/05/2024 09:15, Jan Beulich wrote:
> On 22.05.2024 09:37, Oleksii K. wrote:
>> On Tue, 2024-05-21 at 13:18 +0200, Jan Beulich wrote:
>>> On 17.05.2024 15:54, Oleksii Kurochko wrote:
>>>> To avoid the compilation error below, it is needed to update to
>>>> places
>>>> in common/page_alloc.c where flsl() is used as now flsl() returns
>>>> unsigned int:
>>>>
>>>> ./include/xen/kernel.h:18:21: error: comparison of distinct pointer
>>>> types lacks a cast [-Werror]
>>>>         18 |         (void) (&_x == &_y);            \
>>>>            |                     ^~
>>>>      common/page_alloc.c:1843:34: note: in expansion of macro 'min'
>>>>       1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e
>>>> - s) - 1);
>>>>
>>>> generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is
>>>> 0,
>>>> the result in undefined.
>>>>
>>>> The prototype of the per-architecture fls{l}() functions was
>>>> changed to
>>>> return 'unsigned int' to align with the generic implementation of
>>>> these
>>>> functions and avoid introducing signed/unsigned mismatches.
>>>>
>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>> ---
>>>>   The patch is almost independent from Andrew's patch series
>>>>   (
>>>> https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
>>>> )
>>>>   except test_fls() function which IMO can be merged as a separate
>>>> patch after Andrew's patch
>>>>   will be fully ready.
>>>
>>> If there wasn't this dependency (I don't think it's "almost
>>> independent"),
>>> I'd be offering R-b with again one nit below.
>>
>> Aren't all changes, except those in xen/common/bitops.c, independent? I
>> could move these changes in xen/common/bitops.c to a separate commit. I
>> think it is safe to commit them ( an introduction of common logic for
>> fls{l}() and tests ) separately since the CI tests have passed.
> 
> Technically they might be, but contextually there are further conflicts.
> Just try "patch --dry-run" on top of a plain staging tree. You really
> need to settle, perhaps consulting Andrew, whether you want to go on top
> of his change, or ahead of it. I'm not willing to approve a patch that's
> presented one way but then is (kind of) expected to go in the other way.

I agree with what Jan wrote. I don't have any strong opinion on which 
order they should be merged. But, if your series is intended to be 
merged before Andrew's one then please rebase to vanilla staging.

I looked at the rest of the patch and it LGTM.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 13:45:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 13:45:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728497.1133424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8l2-0005AZ-BX; Thu, 23 May 2024 13:45:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728497.1133424; Thu, 23 May 2024 13:45:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8l2-0005AQ-7O; Thu, 23 May 2024 13:45:24 +0000
Received: by outflank-mailman (input) for mailman id 728497;
 Thu, 23 May 2024 13:45:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA8l0-0005AG-J6; Thu, 23 May 2024 13:45:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA8l0-0006hC-Fp; Thu, 23 May 2024 13:45:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA8l0-0004SI-5i; Thu, 23 May 2024 13:45:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA8l0-00037C-5L; Thu, 23 May 2024 13:45:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SoNK2zl9yTsWLQxbtY7DPR0vOU93so9RNmlRSeHIDTg=; b=EIub1nF2G3hRtSxJUwUnRneNSp
	3xQR5y+0N8wlkUZJGMj3AXm5k5Uu/Ip4NqCw4hilfEa4/ljH/RcTDXsmPU9VPVfMMr9jDowsPC+x1
	vTf83/uR0VplCfBGwbSJdeLV7QLpB1RY2z9FsRqb8/OAIftiGAw8MCQD2iymOuDVYtik=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186107-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186107: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d6a7fd83039af36c28bd0ae2174f12c3888ce993
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 13:45:22 +0000

flight 186107 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186107/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186064

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d6a7fd83039af36c28bd0ae2174f12c3888ce993
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186064  2024-05-21 15:04:02 Z    1 days
Testing same since   186104  2024-05-23 09:00:22 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Bobby Eshleman <bobbyeshleman@gmail.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 387 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 13:59:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 13:59:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728512.1133433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8yy-00077a-K9; Thu, 23 May 2024 13:59:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728512.1133433; Thu, 23 May 2024 13:59:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA8yy-00077T-HZ; Thu, 23 May 2024 13:59:48 +0000
Received: by outflank-mailman (input) for mailman id 728512;
 Thu, 23 May 2024 13:59:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hrMT=M2=linutronix.de=tglx@srs-se1.protection.inumbo.net>)
 id 1sA8yy-00077N-4F
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 13:59:48 +0000
Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b6025bf8-190c-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 15:59:46 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6025bf8-190c-11ef-b4bb-af5377834399
From: Thomas Gleixner <tglx@linutronix.de>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de;
	s=2020; t=1716472784;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=+N81KTAmxqigDRwmaMjHxrB3yOtm6eDGK4EsQAR05Bo=;
	b=3qmiiYvMYsQdwmhYZykoeLa6Ht9fNsda+B+xGXs1GtINIlgZdCjyTJxslL4JHmYMirFkmA
	SvFhg2Yar0leAwdeE7tbU0Bky3TyxDTKYATe8y+t/OnZ+9L32Gw7F/oWxSsWmrOwynHI0S
	pHlqIVsfymFZHsvS3PhYBeT1ZxzJa+Zp7+atlGMS3+LmymRRIBFD/HSrI44rxk/LcoBMht
	fEfMlF3G/a5qS1R5y7cncVNz7xWrTNHaZGvb/MiwRsMwXknOqBIoomgsChudRKmYvrhwx4
	3Egt3PuNvKTnlIjEc+PfKb+drs2SzYCsWB3J21eH1pJiCu9I/zMWJyTizxUfNQ==
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de;
	s=2020e; t=1716472784;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=+N81KTAmxqigDRwmaMjHxrB3yOtm6eDGK4EsQAR05Bo=;
	b=qzAcmqidEnAzvYGxC+ikZM7oqaU+gz4H5u+F3oRQcbZSXUCYE9t+BXk0btN4JIBr2WNikD
	7+9Q3vd5MlBPdbCQ==
To: Jason Andryuk <jason.andryuk@amd.com>, Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Ingo
 Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, Dave Hansen
 <dave.hansen@linux.intel.com>, x86@kernel.org, "H. Peter Anvin"
 <hpa@zytor.com>, Stefano Stabellini <sstabellini@kernel.org>, Oleksandr
 Tyshchenko <oleksandr_tyshchenko@epam.com>, Paolo Bonzini
 <pbonzini@redhat.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, Jason
 Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH 4/5] x86/kernel: Move page table macros to new header
In-Reply-To: <20240410194850.39994-5-jason.andryuk@amd.com>
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-5-jason.andryuk@amd.com>
Date: Thu, 23 May 2024 15:59:43 +0200
Message-ID: <87pltcfx2o.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain

On Wed, Apr 10 2024 at 15:48, Jason Andryuk wrote:
> ---
>  arch/x86/kernel/head_64.S            | 22 ++--------------------
>  arch/x86/kernel/pgtable_64_helpers.h | 28 ++++++++++++++++++++++++++++

That's the wrong place as you want to include it from arch/x86/platform.

arch/x86/include/asm/....

Thanks,

        tglx


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:07:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:07:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728521.1133443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA96V-0000P3-CC; Thu, 23 May 2024 14:07:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728521.1133443; Thu, 23 May 2024 14:07:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA96V-0000Ow-9O; Thu, 23 May 2024 14:07:35 +0000
Received: by outflank-mailman (input) for mailman id 728521;
 Thu, 23 May 2024 14:07:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Txa0=M2=alien8.de=bp@srs-se1.protection.inumbo.net>)
 id 1sA96U-0000Og-5Z
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:07:34 +0000
Received: from mail.alien8.de (mail.alien8.de [2a01:4f9:3051:3f93::2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cba74921-190d-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:07:32 +0200 (CEST)
Received: from localhost (localhost.localdomain [127.0.0.1])
 by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTP id 6B13F40E01E8; 
 Thu, 23 May 2024 14:07:30 +0000 (UTC)
Received: from mail.alien8.de ([127.0.0.1])
 by localhost (mail.alien8.de [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id ReU00yeVSuf2; Thu, 23 May 2024 14:07:27 +0000 (UTC)
Received: from zn.tnic (p5de8ee85.dip0.t-ipconnect.de [93.232.238.133])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature ECDSA (P-256) server-digest
 SHA256) (No client certificate requested)
 by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 03FCC40E0177;
 Thu, 23 May 2024 14:07:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cba74921-190d-11ef-b4bb-af5377834399
X-Virus-Scanned: Debian amavisd-new at mail.alien8.de
Authentication-Results: mail.alien8.de (amavisd-new); dkim=pass (4096-bit key)
	header.d=alien8.de
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=alien8;
	t=1716473247; bh=qXfYRFhu2JIkgeWZDMQpdFw4OPOQczwivSNHPDfIb/Y=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=YSNGNko7IGS7Iwb/IMz2WbftiPUUZsi34JmWbuw9cF/vczn/vm8XBrBYzd6pGPT2M
	 YImW1p8FG3vlC1Xf8Z0cAABBywLWdX/JZ6ifMWLyrZrqXdipCloCgH4/n7/Z+zJqx4
	 UJaErATL0eF+Bb6HgUvhb+eu4VQRo2HlYojKzDft1zg/G7qyF96rsQPPl4R4+6EmfH
	 uBS1RN18X2g3c+JjqpriuHArvNtLTQn74JpORYMQLt3tKeOBdT5n/RLRLzKE3olxs3
	 aJb+Kvk8kYDR81MPiy2BeAYDNhvrEKT1av9ldoDuH52sEWXph7eF6YL3LGkKyFveS2
	 Ixa4To0Xlvhl2pl3zYtXQtasJ9WRkrgsP5Z6PFp4wK55hSDZN61O/GOGIpIn4Bs2yr
	 JNhWhu3mWTgxs8UDrqDpr+8PbI3wgwdRv2MeyjqQrm/rzWiTtJBuV5MtK3WtJa1A42
	 OhOok4/xSvpXMkf7BtAcnC8cO/rc+7FoxnISTGFW/yNTGFs8ZNC+58U8dlKVwLc8Sz
	 vREojVz6bstwAcDS2ethJ3b3l88VaIiiaUEL379iIhGqpWLz/Zwj5ZtDRWwcVGb8NO
	 xwnUOieenhIbgNYvzp5mnmKKAshjRISXS9kqdHTYT3gO9NY8qxzyes9RIWg55k5ykZ
	 5DYZCdF6z0lxJDYgeeYl8iZw=
Date: Thu, 23 May 2024 16:07:09 +0200
From: Borislav Petkov <bp@alien8.de>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Andryuk <jason.andryuk@amd.com>, Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Ingo Molnar <mingo@redhat.com>,
	Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
	"H. Peter Anvin" <hpa@zytor.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Paolo Bonzini <pbonzini@redhat.com>, xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 4/5] x86/kernel: Move page table macros to new header
Message-ID: <20240523140709.GCZk9NjWBeJQ4Yqa9P@fat_crate.local>
References: <20240410194850.39994-1-jason.andryuk@amd.com>
 <20240410194850.39994-5-jason.andryuk@amd.com>
 <87pltcfx2o.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <87pltcfx2o.ffs@tglx>

On Thu, May 23, 2024 at 03:59:43PM +0200, Thomas Gleixner wrote:
> On Wed, Apr 10 2024 at 15:48, Jason Andryuk wrote:
> > ---
> >  arch/x86/kernel/head_64.S            | 22 ++--------------------
> >  arch/x86/kernel/pgtable_64_helpers.h | 28 ++++++++++++++++++++++++++++
> 
> That's the wrong place as you want to include it from arch/x86/platform.
> 
> arch/x86/include/asm/....

... and there already is a header waiting:

arch/x86/include/asm/pgtable_64.h

so no need for a new one.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:12:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:12:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728531.1133469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Ao-0002I5-2i; Thu, 23 May 2024 14:12:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728531.1133469; Thu, 23 May 2024 14:12:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Ao-0002Hy-00; Thu, 23 May 2024 14:12:02 +0000
Received: by outflank-mailman (input) for mailman id 728531;
 Thu, 23 May 2024 14:12:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TgV8=M2=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sA9Am-0002Hd-8X
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:12:00 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6acb6901-190e-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:11:58 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59b178b75bso841729366b.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:11:58 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17894d4bsm1931498566b.78.2024.05.23.07.11.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:11:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6acb6901-190e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716473518; x=1717078318; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=yVXNRusbxlss+tNEO8P//ChEiCeJpTC41lB9HVxMqNs=;
        b=ffD8iK3gle8oDGCEotOYBkTTvpj+Lkwv3tyU5AHyj5kjbBDkwaS21jAX+UPRJ/yF+j
         D168mPrMH8anPM0YfT3bSu3SZc4hVFJvNF/haelOYGrkpeAs4dwHVpCbf5fhzB9KRm1R
         W3WU6y6ZPSKogIuU3yT8z2dazCsUN/pW66zGClXy9hd/tWRi+vMS8KDzCpc5NN5kFsIQ
         SORi6dm+ecpIcC8BGmmRiMEK3z/nCkEwAiiedkjHNwZUGbIUSWNP+/queMl9HYRYImM2
         LKj2y4sCVN3C2Qo/nB+eV8H6eDyssuUxxAeb+bZGqyPewbNH/cEWorXRYvFyG1yXKoxh
         mMqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716473518; x=1717078318;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yVXNRusbxlss+tNEO8P//ChEiCeJpTC41lB9HVxMqNs=;
        b=F4XYu2ZJ26LNUbHatKdNmNUwE/pW/oX9VyuEtsAgH3VR/HJk+gwy4DDPLIH69zpV7P
         n5OgLf/11Lr82GU70bBuXt/saJu9PGmy3HmTb1crPx38cO1wos+mKKJVJVCi/qjpiM9D
         iZV6lw89aXElOrElntFPTYiJ1+Pk/iyMEZ0SyWefjmyY4hd+wVgHZeiUzGGD2upO1UTR
         U6Jg0ogZ/sJpFemdZB7JtS3bzJiV1FFdIR9HEeSjmE8LL93O4xxzRQiJuj1Pjw7Cv1kC
         Oqbf+RVZuhPX0KenigZLdiZREDdRPkqqDvLCOLHWoSL/tlv2Y/0iUTYcHNTyFdpTDqvS
         /g6w==
X-Forwarded-Encrypted: i=1; AJvYcCVHJ2T9f4z57nLBPSQhLY22HeGQinO+Jdp3qaw2pTgU3oykQBTLOkoOittLT9+nM2CkLVo268HJg4/dM4sAm5Qt7bXI6wK5TUXeq8ZM+rM=
X-Gm-Message-State: AOJu0YzsZf3fJxMPW0QJat2UwdKNgv1eWYdY1y1BVMy9QqMh/E8Js4wL
	eH2sQBmrqMQgR0dIeyx3r71IXnt99rcr+HZvInIEOBxm6QQI1eAu
X-Google-Smtp-Source: AGHT+IHit7qXKUDSGMBFMnNQSSQ5qOP+KRrpkTdFUVGpDJI7SOCmjXil9bJCSmXmhCn8EG2Rw2NioQ==
X-Received: by 2002:a17:906:ce36:b0:a58:e3d9:e2d6 with SMTP id a640c23a62f3a-a6228125456mr360154166b.56.1716473517657;
        Thu, 23 May 2024 07:11:57 -0700 (PDT)
Message-ID: <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
Date: Thu, 23 May 2024 16:11:55 +0200
In-Reply-To: <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
	 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
> Hi Oleksii,
Hi Julien,

>=20
> On 17/05/2024 14:54, Oleksii Kurochko wrote:
> > diff --git a/xen/arch/arm/arm64/livepatch.c
> > b/xen/arch/arm/arm64/livepatch.c
> > index df2cebedde..4bc8ed9be5 100644
> > --- a/xen/arch/arm/arm64/livepatch.c
> > +++ b/xen/arch/arm/arm64/livepatch.c
> > @@ -10,7 +10,6 @@
> > =C2=A0 #include <xen/mm.h>
> > =C2=A0 #include <xen/vmap.h>
> > =C2=A0=20
> > -#include <asm/bitops.h>
>=20
> It is a bit unclear how this change is related to the patch. Can you=20
> explain in the commit message?
Probably it doesn't need anymore. I will double check and if this
change is not needed, I will just drop it in the next patch version.

>=20
> > =C2=A0 #include <asm/byteorder.h>
> > =C2=A0 #include <asm/insn.h>
> > =C2=A0 #include <asm/livepatch.h>
> > diff --git a/xen/arch/arm/include/asm/bitops.h
> > b/xen/arch/arm/include/asm/bitops.h
> > index 5104334e48..8e16335e76 100644
> > --- a/xen/arch/arm/include/asm/bitops.h
> > +++ b/xen/arch/arm/include/asm/bitops.h
> > @@ -22,9 +22,6 @@
> > =C2=A0 #define __set_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 set_bit(n,p)
> > =C2=A0 #define __clear_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 clear_bit(n,p)
> > =C2=A0=20
> > -#define BITOP_BITS_PER_WORD=C2=A0=C2=A0=C2=A0=C2=A0 32
> > -#define BITOP_MASK(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 (1UL << ((nr) %
> > BITOP_BITS_PER_WORD))
> > -#define BITOP_WORD(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > =C2=A0 #define BITS_PER_BYTE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 8
>=20
> OOI, any reason BITS_PER_BYTE has not been moved as well? I don't
> expect=20
> the value to change across arch.
I can move it to generic one header too in the next patch version.

>=20
> [...]
>=20
> > diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
> > index f14ad0d33a..6eeeff0117 100644
> > --- a/xen/include/xen/bitops.h
> > +++ b/xen/include/xen/bitops.h
> > @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned long
> > x)
> > =C2=A0=C2=A0 * scope
> > =C2=A0=C2=A0 */
> > =C2=A0=20
> > +#define BITOP_BITS_PER_WORD 32
> > +typedef uint32_t bitop_uint_t;
> > +
> > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > BITOP_BITS_PER_WORD))
> > +
> > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > +
> > +extern void __bitop_bad_size(void);
> > +
> > +#define bitop_bad_size(addr) (sizeof(*(addr)) <
> > sizeof(bitop_uint_t))
> > +
> > =C2=A0 /* --------------------- Please tidy above here ----------------=
-
> > ---- */
> > =C2=A0=20
> > =C2=A0 #include <asm/bitops.h>
> > =C2=A0=20
> > +/**
> > + * generic__test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
>=20
> Sorry for only mentioning this on v10. I think this comment should be
> duplicated (or moved to) on top of test_bit() because this is what=20
> everyone will use. This will avoid the developper to follow the
> function=20
> calls and only notice the x86 version which says "This function is=20
> atomic and may not be reordered." and would be wrong for all the
> other arch.
It makes sense to add this comment on top of test_bit(), but I am
curious if it is needed to mention that for x86 arch_test_bit() "is
atomic and may not be reordered":

 * This operation is non-atomic and can be reordered. ( Exception: for
* x86 arch_test_bit() is atomic and may not be reordered )
 * If two examples of this operation race, one can appear to succeed
 * but actually fail.  You must protect multiple accesses with a lock.
 */

>=20
> > +static always_inline bool
> > +generic__test_and_set_bit(int nr, volatile void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +
> > +/**
> > + * generic__test_and_clear_bit - Clear a bit and return its old
> > value
> > + * @nr: Bit to clear
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
>=20
> Same applies here and ...
>=20
> > +static always_inline bool
> > +generic__test_and_clear_bit(int nr, volatile void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +
> > +/* WARNING: non atomic and it can be reordered! */
>=20
> ... here.
>=20
> > +static always_inline bool
> > +generic__test_and_change_bit(int nr, volatile void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_uint_t=
 *)addr +
> > BITOP_WORD(nr);
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > +
> > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > +}
> > +/**
> > + * generic_test_bit - Determine whether a bit is set
> > + * @nr: bit number to test
> > + * @addr: Address to start counting from
> > + */
> > +static always_inline bool generic_test_bit(int nr, const volatile
> > void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 const volatile bitop_uint_t *p =3D
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (cons=
t volatile bitop_uint_t *)addr +
> > BITOP_WORD(nr);
> > +
> > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > +}
> > +
> > +static always_inline bool
> > +__test_and_set_bit(int nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_set_bit
> > +#define arch__test_and_set_bit generic__test_and_set_bit
> > +#endif
> > +
> > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
> > +}
>=20
> NIT: It is a bit too late to change this one. But I have to admit, I=20
> don't understand the purpose of the static inline when you could have
> simply call...
>=20
> > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
>=20
> ... __arch__test_and_set_bit here.
I just wanted to be in sync with other non-atomic test*_bit()
operations and Andrew's patch series ( xen/bitops: Reduce the mess,
starting with ffs() ).

>=20
>=20
> The only two reasons I am not providing an ack is the:
> =C2=A0 * Explanation for the removal of asm/bitops.h in livepatch.c
> =C2=A0 * The placement of the comments
Thanks for review. I'll update the patch and sent new version of the
patch series.

~ Oleksii

>=20
> There are not too important for me.
>=20
> Cheers,
>=20



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:22:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:22:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728541.1133484 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Kg-0004BG-1M; Thu, 23 May 2024 14:22:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728541.1133484; Thu, 23 May 2024 14:22:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Kf-0004B9-Uq; Thu, 23 May 2024 14:22:13 +0000
Received: by outflank-mailman (input) for mailman id 728541;
 Thu, 23 May 2024 14:22:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TXgv=M2=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1sA9Kd-0004At-W8
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:22:12 +0000
Received: from wfhigh6-smtp.messagingengine.com
 (wfhigh6-smtp.messagingengine.com [64.147.123.157])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d622ffa1-190f-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:22:09 +0200 (CEST)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfhigh.west.internal (Postfix) with ESMTP id 3727518000B8;
 Thu, 23 May 2024 10:22:06 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Thu, 23 May 2024 10:22:06 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 23 May 2024 10:22:04 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d622ffa1-190f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:content-type:content-type:date:date
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1716474125;
	 x=1716560525; bh=wch4MQuvuoO/P1pJrQteF14ohcPjgykADHYQkfDfatc=; b=
	BjcgLiWmlR+xaH9GdQ+d4rEO0sgf1GtMiJTZCRcF2zSw5Kb+0LoFJUg6976wp0cm
	t78HTUzGk4MTKKLTRn67hNwkhj+5o20lLxhTFlwU84aJM65PwxGPhUnu7xWeud+x
	QB+lI9j2OPhzDy0Umf04mbO1KPrz7VwsU4wXCT5+irBQzllMu9m5rD609MYWwoYp
	exZXfrNmJq1qElVb/RdjYSaBJuFp8CUMF0Ey4UiXiKiTsFToWrbv8XKC1kvRH6TZ
	vtL++6GaH35hBfw2Dg6UT5514poW1hfDYl9x7x1WCVfpv35OrZW1QsJPJSygI90U
	Bi+DhF4Cfbupyorsh0dgbA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1716474125; x=1716560525; bh=wch4MQuvuoO/P1pJrQteF14ohcPj
	gykADHYQkfDfatc=; b=nXStMfr5tSxpoaGYxfs9eR5ys1YlaSRcGOiZMCyL7b6c
	5nxMWORJPS78UC+XIB4Q+N0srgF0xsC8l6ML6i+cs+W8r3wwF/NfTBNHWb35bgu3
	tKWXUi0+akkgOiqRW8djPIfpi+FMPwvmrQfEL99AW/mVfs1qN4mWaEWec6Z1F9rE
	tLB0SHzQeVTA2iGmSblsIk48DVh8sCDK3v4C5Nd4M/DdYeSAQ4QDkH2CAJFbTKqu
	qTyX19n0kLKGKWpjt7jxCqYIWDsTv9v4A97EAgy72AxwWyfFCgO5tO3595nhZ1Qz
	rwwqQQtoZsfmiyGNN9COytqPKGnxYFgOwPdQd6uVvQ==
X-ME-Sender: <xms:DVFPZo6JxlmxbjnlJCMmWAfkTl3J2M98Yrzu6PmuCIacdm5MP8xHug>
    <xme:DVFPZp5B_NOC9mvzlB3F5fRAbub8RE-kolDJp-ntVB7OLJLImRSvap9oB917eNjRh
    Ii2AYuxX_JYgQ>
X-ME-Received: <xmr:DVFPZnc8ZF6PD1bRfxiorjRvjlbby8YU-7OUp9tBnry9tnM1nr1puBTcpT5avmQ7xFnmUlVBSph1GkUSOCi9yW8Pq9WSxK5etQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeiiedgjeehucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeehvedv
    ueevheekhefhvefggffgvedugeetuefgleeivdehgfeuieeugfetteeiffenucffohhmrg
    hinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm
    pehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslh
    grsgdrtghomh
X-ME-Proxy: <xmx:DVFPZtIUmHNW2D4JaLvhXFGe1S2wjA3kWzq3x1vifkP7hPMtOIqy0w>
    <xmx:DVFPZsJNUWRXYr3juwd1PRUj32Gc9fAIIApk4My0DTFIfeg-bEUlaA>
    <xmx:DVFPZuwDOS5Hb-RMH60J-idNbHecnjLj-pLoWJllAbkcqBezT2KUHQ>
    <xmx:DVFPZgJF-p9Mt_C724P9KhikDW0T6AR9GU5UY_UAdSTtXdRlNU4cNQ>
    <xmx:DVFPZjWRm-UWZrne_iD3FxfmNc5UQ9uO2ZotSvVGIq7qC0YaVNRvWQRN>
Feedback-ID: i1568416f:Fastmail
Date: Thu, 23 May 2024 16:22:01 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Subject: Re: [PATCH v4 0/2] Add API for making parts of a MMIO page R/O and
 use it in XHCI console
Message-ID: <Zk9RChKHBa4mQAVj@mail-itl>
References: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="NVX2/ut/877rVt3E"
Content-Disposition: inline
In-Reply-To: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>


--NVX2/ut/877rVt3E
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Thu, 23 May 2024 16:22:01 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Subject: Re: [PATCH v4 0/2] Add API for making parts of a MMIO page R/O and
 use it in XHCI console

On Wed, May 22, 2024 at 05:39:02PM +0200, Marek Marczykowski-G=C3=B3recki w=
rote:
> On older systems, XHCI xcap had a layout that no other (interesting) regi=
sters
> were placed on the same page as the debug capability, so Linux was fine w=
ith
> making the whole page R/O. But at least on Tiger Lake and Alder Lake, Lin=
ux
> needs to write to some other registers on the same page too.
>=20
> Add a generic API for making just parts of an MMIO page R/O and use it to=
 fix
> USB3 console with share=3Dyes or share=3Dhwdom options. More details in c=
ommit
> messages.
>=20
> Marek Marczykowski-G=C3=B3recki (2):
>   x86/mm: add API for marking only part of a MMIO page read only
>   drivers/char: Use sub-page ro API to make just xhci dbc cap RO

Does any other x86 maintainer feel comfortable ack-ing this series? Jan
already reviewed 2/2 here (but not 1/2 in this version), but also said
he is not comfortable with letting this in without a second maintainer
approval: https://lore.kernel.org/xen-devel/7655e401-b927-4250-ae63-05361a5=
ee71d@suse.com/

>=20
>  xen/arch/x86/hvm/emulate.c      |   2 +-
>  xen/arch/x86/hvm/hvm.c          |   4 +-
>  xen/arch/x86/include/asm/mm.h   |  25 +++-
>  xen/arch/x86/mm.c               | 273 +++++++++++++++++++++++++++++++++-
>  xen/arch/x86/pv/ro-page-fault.c |   6 +-
>  xen/drivers/char/xhci-dbc.c     |  36 ++--
>  6 files changed, 327 insertions(+), 19 deletions(-)
>=20
> base-commit: b0082b908391b29b7c4dd5e6c389ebd6481926f8
> --=20
> git-series 0.9.1

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--NVX2/ut/877rVt3E
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZPUQoACgkQ24/THMrX
1yyBAgf+OY4AI4FifIusRtmZbKOTGwbDJwPe3rQRQvQIoc2uzQwcYWNiirQdxXR3
8F+ZEL1/IWAC5jJk5kK5guXBFUXhLy0+ni6tfO/jlRg6P468eoIWrHvQ0vtyFcUF
QWjk5MXs2UmPnSY+6FO90sZCU5CXv9E5T5xIXzm2IQGWyQtA5GHBBaGsDdPnsekN
ccrwinyhyVvGSruXHnRpIyCM6Ghmw4Zyq0LpRRH9Mpxw8QiY8OU1mEuHi4txkHTZ
O5RdlHc+nDg40xpY0S9GNzbm9mh9I/blzV+twHkOb+lTCzXq3UfqTH29kKdK6VVO
N427hpmTrAYKGUPuqAZLtmo+c4M5Ig==
=N8As
-----END PGP SIGNATURE-----

--NVX2/ut/877rVt3E--


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:24:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:24:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728546.1133499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ND-0004oS-NE; Thu, 23 May 2024 14:24:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728546.1133499; Thu, 23 May 2024 14:24:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ND-0004nP-Ia; Thu, 23 May 2024 14:24:51 +0000
Received: by outflank-mailman (input) for mailman id 728546;
 Thu, 23 May 2024 14:24:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqR6=M2=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1sA9NB-0004l4-Lv
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:24:49 +0000
Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com
 [2607:f8b0:4864:20::729])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3594abba-1910-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:24:48 +0200 (CEST)
Received: by mail-qk1-x729.google.com with SMTP id
 af79cd13be357-794977cbcc3so110760485a.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:24:48 -0700 (PDT)
Received: from georged-x-u.xenrt.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf34060csm1484523485a.134.2024.05.23.07.24.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:24:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3594abba-1910-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716474287; x=1717079087; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=X8Ew7kXbX6WgodqVo4gVB/w3zPAvA0jSuuYBh0aRvRU=;
        b=Urc6KGvqgtJk+ULitDI60Z/ycC6fg9TYwsZWpuEysvbIZ8UPOH6DAldMdi8IaPJjSw
         rC5hI4Uz9xWfI75p/Y+pCkDIeGc+i5/Y9VSA0Lfuc9WESps9X46NXZsSEAf9QtjorNFh
         cagPyfJ4I6JAMP3LfRd8ndJrYcVWpA3r+YeZA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716474287; x=1717079087;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=X8Ew7kXbX6WgodqVo4gVB/w3zPAvA0jSuuYBh0aRvRU=;
        b=KeD9EgXTrxVBICmetsPkNKw7MOtRdoUrMkNBhZ0D98598oxBq1uE6P/rgi9eHcQgRZ
         iA2DqOHdpbxMz28pFW3giCu+jrbDe/97EIjrkVbf62pfiKY6lVRHouwiWbt8UZn1DmD4
         xifQF+7TQ2Svp9c6Glr2063aOhUpNrQlN92xXu9s3a4E1mJi2JJF6TDTTwIMHLr4miRR
         TyTlNSkPqVFuNUDoff09lBmxcTycp0+bPcQzwmlwZqF3i94q5IhEIhWsNHlGC8mPqjh5
         9p5siVfjrL7po2A8bJpV1y8qVsGpDwbUbgEZahn8KFVgN9e4xKY/rsfUFBwvEpgz7YWO
         xr0w==
X-Gm-Message-State: AOJu0Yxo+a6VrafWvFRllOBADYl7HdIaF9dzV+jggcje8kwEmWynQ2+K
	OXTt4pjJtyoHTmuTMyFixmdm6hgQkAgBpGJUnq53/9nxTc1RdOzy5lworeER7KAGxxkpzFx7cS1
	IcYY=
X-Google-Smtp-Source: AGHT+IG5VmIs5hbF9sW/q/KrZDjCyvAZCTD8j7mB5ZNItW8pz1J6+O7F4zgT+uRykhLHaoVN5DmJtA==
X-Received: by 2002:ae9:e916:0:b0:792:5b7a:ad58 with SMTP id af79cd13be357-7949941a69fmr549719485a.8.1716474287318;
        Thu, 23 May 2024 07:24:47 -0700 (PDT)
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony Perard <anthony.perard@cloud.com>,
	Olaf Hering <olaf@aepfle.de>
Subject: [PATCH v2 2/2] tools/xenalyze: Ignore HVM_EMUL events harder
Date: Thu, 23 May 2024 15:10:20 +0100
Message-Id: <20240523141020.2256177-2-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240523141020.2256177-1-george.dunlap@cloud.com>
References: <20240523141020.2256177-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To unify certain common sanity checks, checks are done very early in
processing based only on the top-level type.

Unfortunately, when TRC_HVM_EMUL was introduced, it broke some of the
assumptions about how the top-level types worked.  Namely, traces of
this type will show up outside of HVM contexts: in idle domains and in
PV domains.

Make an explicit exception for TRC_HVM_EMUL types in a number of places:

 - Pass the record info pointer to toplevel_assert_check, so that it
   can exclude TRC_HVM_EMUL records from idle and vcpu data_mode
   checks

 - Don't attempt to set the vcpu data_type in hvm_process for
   TRC_HVM_EMUL records.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Anthony Perard <anthony.perard@cloud.com>
CC: Olaf Hering <olaf@aepfle.de>
---
 tools/xentrace/xenalyze.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 9c4463b0e8..d95e52695f 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -21,6 +21,7 @@
 #define _XOPEN_SOURCE 600
 #include <stdio.h>
 #include <stdlib.h>
+#include <stdbool.h>
 #include <argp.h>
 #include <inttypes.h>
 #include <limits.h>
@@ -5305,8 +5306,11 @@ void hvm_process(struct pcpu_info *p)
 
     assert(p->current);
 
-    if(vcpu_set_data_type(p->current, VCPU_DATA_HVM))
-        return;
+    /* HVM_EMUL types show up in all contexts */
+    if(ri->evt.sub != 0x4) {
+        if(vcpu_set_data_type(p->current, VCPU_DATA_HVM))
+            return;
+    }
 
     switch ( ri->evt.sub ) {
     case 2: /* HVM_HANDLER */
@@ -9447,9 +9451,10 @@ static struct tl_assert_mask tl_assert_checks[TOPLEVEL_MAX] = {
 /* There are a lot of common assumptions for the various processing
  * routines.  Check them all in one place, doing something else if
  * they don't pass. */
-int toplevel_assert_check(int toplevel, struct pcpu_info *p)
+int toplevel_assert_check(int toplevel, struct record_info *ri, struct pcpu_info *p)
 {
     struct tl_assert_mask mask;
+    bool is_hvm_emul = (toplevel == TOPLEVEL_HVM) && (ri->evt.sub == 0x4);
 
     mask = tl_assert_checks[toplevel];
 
@@ -9459,7 +9464,7 @@ int toplevel_assert_check(int toplevel, struct pcpu_info *p)
         goto fail;
     }
 
-    if( mask.not_idle_domain )
+    if( mask.not_idle_domain && !is_hvm_emul)
     {
         /* Can't do this check w/o first doing above check */
         assert(mask.p_current);
@@ -9478,7 +9483,8 @@ int toplevel_assert_check(int toplevel, struct pcpu_info *p)
         v = p->current;
 
         if ( ! (v->data_type == VCPU_DATA_NONE
-                || v->data_type == mask.vcpu_data_mode) )
+                || v->data_type == mask.vcpu_data_mode
+                || is_hvm_emul) )
         {
             /* This may happen for track_dirty_vram, which causes a SHADOW_WRMAP_BF trace f/ dom0 */
             fprintf(warn, "WARNING: Unexpected vcpu data type for d%dv%d on proc %d! Expected %d got %d. Not processing\n",
@@ -9525,7 +9531,7 @@ void process_record(struct pcpu_info *p) {
         return;
 
     /* Unify toplevel assertions */
-    if ( toplevel_assert_check(toplevel, p) )
+    if ( toplevel_assert_check(toplevel, ri, p) )
     {
         switch(toplevel) {
         case TRC_GEN_MAIN:
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:24:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:24:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728545.1133494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ND-0004lN-Eg; Thu, 23 May 2024 14:24:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728545.1133494; Thu, 23 May 2024 14:24:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ND-0004lG-B8; Thu, 23 May 2024 14:24:51 +0000
Received: by outflank-mailman (input) for mailman id 728545;
 Thu, 23 May 2024 14:24:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqR6=M2=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1sA9NB-0004l3-BD
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:24:49 +0000
Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com
 [2607:f8b0:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 34883ece-1910-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:24:47 +0200 (CEST)
Received: by mail-oi1-x230.google.com with SMTP id
 5614622812f47-3d04c170118so384694b6e.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:24:47 -0700 (PDT)
Received: from georged-x-u.xenrt.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-792bf34060csm1484523485a.134.2024.05.23.07.24.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:24:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34883ece-1910-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716474285; x=1717079085; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=L/vVJIERTUJT3/5WM0EN643bbOtUFlyj/VZsiJqZsfo=;
        b=bV+2hTp7AVNgTSYYFGvMhuBJZRw5uAJ195Ln/tXzoW+h0K2L5SC6FSWhT/VKQvUNpY
         a0R3A7NMWW7+/Y9YrAhwg+UOlYoBZB/YVo478U9kWRFpJEaIKlKGyQDD8YHOuXyPAdP1
         LxrefDwU+LSdTtDpfCxDIadES8RK74UkFXEjc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716474285; x=1717079085;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=L/vVJIERTUJT3/5WM0EN643bbOtUFlyj/VZsiJqZsfo=;
        b=IBzpQMmCtj2XN9muJUM9msiFnJpXwKGj9qmmQ+ENN7fyYY3o+8qD9uKMGVi8xbTKFb
         uFhlFFXPuuvtOXO1WguuR74PKGjoL4t2fxLaJgKEY19lxtImmsp28SMWQwVnFyUk4Yv6
         gQdZ27CMAciSiPnWfFhNsT2xmqhxjXtmgB1QjLNd0yThckWT7j4McVKcRUFzasFlWpr2
         rS7eGB7IdWxLxZrdDOtSdPVGv26pUau4aLpuinpbDpqcu7w573zCXqfCWsYZrbpEVPOU
         QJn2s+8Vs19wLeLpbJU4iAgXWPNlPgL9sjZp54mhZ6fYS6gOuHS3CKU0etwgn5zhuFZz
         FNfg==
X-Gm-Message-State: AOJu0Yw0mMHiCXDt9JCGvJW3KrtnNyGg9vKSL35Hg5+rEfkXaJqyhX0r
	OF2GTLcI+DUauzalbzZrLvz2ffxWh9mzwQnWzh6H7+Wdq3jjkX+z/XQdKljf/bAaQS6gyWQhUr5
	KHNI=
X-Google-Smtp-Source: AGHT+IFNXYx/jlAMsCVqeI+2FvpqxM5ElEjllnxRzeZ3tonYcegJHwHwmFWr8aWBxq7U5EaDM5L58A==
X-Received: by 2002:a05:6808:6d4:b0:3c9:6967:bbed with SMTP id 5614622812f47-3cdb7414eebmr5048236b6e.28.1716474285588;
        Thu, 23 May 2024 07:24:45 -0700 (PDT)
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Roger Pau Monne <roger.pau@cloud.com>,
	Anthony Perard <anthony.perard@cloud.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Olaf Hering <olaf@aepfle.de>
Subject: [PATCH v2 1/2] x86/hvm/trace: Use a different trace type for AMD processors
Date: Thu, 23 May 2024 15:10:19 +0100
Message-Id: <20240523141020.2256177-1-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

A long-standing usability sub-optimality with xenalyze is the
necessity to specify `--svm-mode` when analyzing AMD processors.  This
fundamentally comes about because the same trace event ID is used for
both VMX and SVM, but the contents of the trace must be interpreted
differently.

Instead, allocate separate trace events for VMX and SVM vmexits in
Xen; this will allow all readers to properly interpret the meaning of
the vmexit reason.

In xenalyze, first remove the redundant call to init_hvm_data();
there's no way to get to hvm_vmexit_process() without it being already
initialized by the set_vcpu_type call in hvm_process().

Replace this with set_hvm_exit_reson_data(), and move setting of
hvm->exit_reason_* into that function.

Modify hvm_process and hvm_vmexit_process to handle all four potential
values appropriately.

If SVM entries are encountered, set opt.svm_mode so that other
SVM-specific functionality is triggered.

Remove the `--svm-mode` command-line option, since it's now redundant.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
v2:
- Rebase to tip of staging
- Rebase over xentrace_format removal
- Fix typo in commit message
- Remove --svm-mode command-line flag

CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Roger Pau Monne <roger.pau@cloud.com>
CC: Anthony Perard <anthony.perard@cloud.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Olaf Hering <olaf@aepfle.de>
---
 tools/xentrace/xenalyze.c  | 37 +++++++++++++++++++------------------
 xen/arch/x86/hvm/svm/svm.c |  4 ++--
 xen/arch/x86/hvm/vmx/vmx.c |  4 ++--
 xen/include/public/trace.h |  6 ++++--
 4 files changed, 27 insertions(+), 24 deletions(-)

diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index ce6a85d50b..9c4463b0e8 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -1437,14 +1437,6 @@ void init_hvm_data(struct hvm_data *h, struct vcpu_data *v) {
 
     h->init = 1;
 
-    if(opt.svm_mode) {
-        h->exit_reason_max = HVM_SVM_EXIT_REASON_MAX;
-        h->exit_reason_name = hvm_svm_exit_reason_name;
-    } else {
-        h->exit_reason_max = HVM_VMX_EXIT_REASON_MAX;
-        h->exit_reason_name = hvm_vmx_exit_reason_name;
-    }
-
     if(opt.histogram_interrupt_eip) {
         int count = ((1ULL<<ADDR_SPACE_BITS)/opt.histogram_interrupt_increment);
         size_t size = count * sizeof(int);
@@ -1462,6 +1454,18 @@ void init_hvm_data(struct hvm_data *h, struct vcpu_data *v) {
         h->summary.guest_interrupt[i].count=0;
 }
 
+void set_hvm_exit_reason_data(struct hvm_data *h, unsigned event) {
+    if (((event & ~TRC_64_FLAG) == TRC_HVM_SVM_EXIT) ||
+        opt.svm_mode) {
+        opt.svm_mode = 1;
+        h->exit_reason_max = HVM_SVM_EXIT_REASON_MAX;
+        h->exit_reason_name = hvm_svm_exit_reason_name;
+    } else {
+        h->exit_reason_max = HVM_VMX_EXIT_REASON_MAX;
+        h->exit_reason_name = hvm_vmx_exit_reason_name;
+    }
+}
+
 /* PV data */
 enum {
     PV_HYPERCALL=1,
@@ -5088,13 +5092,13 @@ void hvm_vmexit_process(struct record_info *ri, struct hvm_data *h,
 
     r = (typeof(r))ri->d;
 
-    if(!h->init)
-        init_hvm_data(h, v);
+    if(!h->exit_reason_name)
+        set_hvm_exit_reason_data(h, ri->event);
 
     h->vmexit_valid=1;
     bzero(&h->inflight, sizeof(h->inflight));
 
-    if(ri->event == TRC_HVM_VMEXIT64) {
+    if(ri->event & TRC_64_FLAG) {
         if(v->guest_paging_levels != 4)
         {
             if ( verbosity >= 6 )
@@ -5316,8 +5320,10 @@ void hvm_process(struct pcpu_info *p)
         break;
     default:
         switch(ri->event) {
-        case TRC_HVM_VMEXIT:
-        case TRC_HVM_VMEXIT64:
+        case TRC_HVM_VMX_EXIT:
+        case TRC_HVM_VMX_EXIT64:
+        case TRC_HVM_SVM_EXIT:
+        case TRC_HVM_SVM_EXIT64:
             UPDATE_VOLUME(p, hvm[HVM_VOL_VMEXIT], ri->size);
             hvm_vmexit_process(ri, h, v);
             break;
@@ -10884,11 +10890,6 @@ const struct argp_option cmd_opts[] =  {
       .arg = "HZ",
       .doc = "Cpu speed of the tracing host, used to convert tsc into seconds.", },
 
-    { .name = "svm-mode",
-      .key = OPT_SVM_MODE,
-      .group = OPT_GROUP_HARDWARE,
-      .doc = "Assume AMD SVM-style vmexit error codes.  (Default is Intel VMX.)", },
-
     { .name = "progress",
       .key = OPT_PROGRESS,
       .doc = "Progress dialog.  Requires the zenity (GTK+) executable.", },
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index db530d55f2..988250dbc1 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2571,10 +2571,10 @@ void asmlinkage svm_vmexit_handler(void)
     exit_reason = vmcb->exitcode;
 
     if ( hvm_long_mode_active(v) )
-        TRACE_TIME(TRC_HVM_VMEXIT64 | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
+        TRACE_TIME(TRC_HVM_SVM_EXIT64 | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
                    exit_reason, regs->rip, regs->rip >> 32);
     else
-        TRACE_TIME(TRC_HVM_VMEXIT | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
+        TRACE_TIME(TRC_HVM_SVM_EXIT | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
                    exit_reason, regs->eip);
 
     if ( vcpu_guestmode )
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 8ba996546f..f16faa6a61 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -4105,9 +4105,9 @@ void asmlinkage vmx_vmexit_handler(struct cpu_user_regs *regs)
     __vmread(VM_EXIT_REASON, &exit_reason);
 
     if ( hvm_long_mode_active(v) )
-        TRACE_TIME(TRC_HVM_VMEXIT64, exit_reason, regs->rip, regs->rip >> 32);
+        TRACE_TIME(TRC_HVM_VMX_EXIT64, exit_reason, regs->rip, regs->rip >> 32);
     else
-        TRACE_TIME(TRC_HVM_VMEXIT, exit_reason, regs->eip);
+        TRACE_TIME(TRC_HVM_VMX_EXIT, exit_reason, regs->eip);
 
     perfc_incra(vmexits, (uint16_t)exit_reason);
 
diff --git a/xen/include/public/trace.h b/xen/include/public/trace.h
index 3c9f9c3c18..141efa0ea7 100644
--- a/xen/include/public/trace.h
+++ b/xen/include/public/trace.h
@@ -178,8 +178,10 @@
 /* trace events per subclass */
 #define TRC_HVM_NESTEDFLAG      (0x400)
 #define TRC_HVM_VMENTRY         (TRC_HVM_ENTRYEXIT + 0x01)
-#define TRC_HVM_VMEXIT          (TRC_HVM_ENTRYEXIT + 0x02)
-#define TRC_HVM_VMEXIT64        (TRC_HVM_ENTRYEXIT + TRC_64_FLAG + 0x02)
+#define TRC_HVM_VMX_EXIT        (TRC_HVM_ENTRYEXIT + 0x02)
+#define TRC_HVM_VMX_EXIT64      (TRC_HVM_ENTRYEXIT + TRC_64_FLAG + 0x02)
+#define TRC_HVM_SVM_EXIT        (TRC_HVM_ENTRYEXIT + 0x03)
+#define TRC_HVM_SVM_EXIT64      (TRC_HVM_ENTRYEXIT + TRC_64_FLAG + 0x03)
 #define TRC_HVM_PF_XEN          (TRC_HVM_HANDLER + 0x01)
 #define TRC_HVM_PF_XEN64        (TRC_HVM_HANDLER + TRC_64_FLAG + 0x01)
 #define TRC_HVM_PF_INJECT       (TRC_HVM_HANDLER + 0x02)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:29:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728558.1133513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9RF-0005vw-B3; Thu, 23 May 2024 14:29:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728558.1133513; Thu, 23 May 2024 14:29:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9RF-0005vp-8b; Thu, 23 May 2024 14:29:01 +0000
Received: by outflank-mailman (input) for mailman id 728558;
 Thu, 23 May 2024 14:28:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9RD-0005vj-SA
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:28:59 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ca317660-1910-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:28:57 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a621cb07d8fso284837666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:28:57 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7dd4sm1942816966b.139.2024.05.23.07.28.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:28:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca317660-1910-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716474537; x=1717079337; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qGYX5cZNEZCr/r7aY+MvBTpZfJhz3uIklvKnW9itnNY=;
        b=cZifHapv/BM8LNM7XHBKnQ7Iasffrbli2WyYVM6VDLzsFeWeHC4lZrRKl2aAryPG39
         pB+jQQ7WcDMQkztRup2Mp0T/hcmVJ5mPMMMg0YJ9x8L3w7HMu+wBY92DUjMN8+vyW/Nd
         wsAFmduh5FsSO19SRt5Bc5S4op7fu7xT6LhBLt9Ssfi5WPhNEjdInxLWs+MZqvE1laye
         sNHDonytIlbc1qaqDzU1v/sVzS02CnTufngBWdtyCw7SVDQLioD+7YxpErQLmYNQWwYC
         wuehlrLH0xxWoB4mKkB6P5Hja6jqD1s5dozDpjhplZaSSuNOsOiCTTgnb2pk5yJaWysF
         0PeA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716474537; x=1717079337;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qGYX5cZNEZCr/r7aY+MvBTpZfJhz3uIklvKnW9itnNY=;
        b=Y6jV6aR/RgGDvY6UzZ3w/vH2yjgRX80YjkMnUktJIiYMgMAwPm+wt3/RWN5c4oggB6
         9+95hQWzbmKKGgPWbuUv3p7kJgQiWqG11es0hgUy46GoIUh2YWDLs8Cirxi9zYxSlnel
         w5op3W+DvBPmfr0YXIShtXt+Ka2nl3caFNFOAqb+4/o3cBeJscQzuZplZ2TZv3JKHO/s
         Uetzf8m1t/BwsyH3aXDGJqWPfHccbbT96o0j3nga8UM2PDY+7Yw34HLHExjRpcNQqx9x
         +7vZxKO3D34djrQXqPmRfCGu8ET0VRy+AY2cCS9Iuw9zKh0+Jusieq+cH3oqEpCLHUDq
         hzZQ==
X-Gm-Message-State: AOJu0YwefB3s8X2MocNMx27krWDB4/ym7YKwlt9ms6DHXpY+b5bDuHvs
	OJgJ3jUiQqhsFvher7vmB+G/H3aTs4z0Aenwn5dW4ckNJRzTd2xVx6QMRxw+RQ==
X-Google-Smtp-Source: AGHT+IHZQ+W+5LFH6NClVB3/Y0RPvpfK5PnafyJS3lSxrcTlsReYUw5Il68RRT/wzjPiJppqqDVM3w==
X-Received: by 2002:a17:906:3b87:b0:a59:c28a:7eb6 with SMTP id a640c23a62f3a-a622805efc9mr370076866b.24.1716474536884;
        Thu, 23 May 2024 07:28:56 -0700 (PDT)
Message-ID: <5e2eeb3b-7ab7-429d-b113-134d526374d8@suse.com>
Date: Thu, 23 May 2024 16:28:55 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 0/2] Add API for making parts of a MMIO page R/O and
 use it in XHCI console
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
References: <cover.68462f37276d69ab6e268be94d049f866a321f73.1716392340.git-series.marmarek@invisiblethingslab.com>
 <Zk9RChKHBa4mQAVj@mail-itl>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk9RChKHBa4mQAVj@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.05.2024 16:22, Marek Marczykowski-Górecki wrote:
> On Wed, May 22, 2024 at 05:39:02PM +0200, Marek Marczykowski-Górecki wrote:
>> On older systems, XHCI xcap had a layout that no other (interesting) registers
>> were placed on the same page as the debug capability, so Linux was fine with
>> making the whole page R/O. But at least on Tiger Lake and Alder Lake, Linux
>> needs to write to some other registers on the same page too.
>>
>> Add a generic API for making just parts of an MMIO page R/O and use it to fix
>> USB3 console with share=yes or share=hwdom options. More details in commit
>> messages.
>>
>> Marek Marczykowski-Górecki (2):
>>   x86/mm: add API for marking only part of a MMIO page read only
>>   drivers/char: Use sub-page ro API to make just xhci dbc cap RO
> 
> Does any other x86 maintainer feel comfortable ack-ing this series? Jan
> already reviewed 2/2 here (but not 1/2 in this version),

Which, btw, isn't to mean I'm not going to look at it. But 2/2 was the
lower hanging fruit ...

Jan

> but also said
> he is not comfortable with letting this in without a second maintainer
> approval: https://lore.kernel.org/xen-devel/7655e401-b927-4250-ae63-05361a5ee71d@suse.com/



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:32:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:32:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728567.1133524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9UL-0007cQ-PE; Thu, 23 May 2024 14:32:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728567.1133524; Thu, 23 May 2024 14:32:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9UL-0007cJ-MJ; Thu, 23 May 2024 14:32:13 +0000
Received: by outflank-mailman (input) for mailman id 728567;
 Thu, 23 May 2024 14:32:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA9UJ-0007av-OO
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:32:11 +0000
Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com
 [2607:f8b0:4864:20::834])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3cbdf6be-1911-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:32:10 +0200 (CEST)
Received: by mail-qt1-x834.google.com with SMTP id
 d75a77b69052e-43fa6d92b4eso7109651cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:32:10 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ab8c7faf00sm10495786d6.62.2024.05.23.07.32.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:32:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cbdf6be-1911-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716474729; x=1717079529; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=BeJ6ug4ojY5nqhiZ/esNITNBOQraTn4chWVUPnRXkFg=;
        b=fTzQWyY6chvY12SQeLLLLT1d6+Rbi5to0F9wecs2fUmeDTR/fyrZOD1C2O+eR7TU5c
         YgsVSsyU7a9WfOfAB4XJt5VsRuKUZvDkpGAjEAiuwrFYTrea/EIMes5NI6rxn8aIqJR6
         orKmcN2QYrtgY/Q4tKqeJzBV0Qj+qdZ9++7IY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716474729; x=1717079529;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BeJ6ug4ojY5nqhiZ/esNITNBOQraTn4chWVUPnRXkFg=;
        b=ViJnIxL/fQrko5mmZST1d/0kBoC4MzBRwttz3iOKki/nx0PxUcbfvQvKFz8JA9yTZi
         G4Youw6etRv2fhSnlMEMbKgHboJ9CaeDhCPQ4UEO7Q251FrDL7CCV3ZLnYi96Hifo0rA
         heZbalfxOArZlq+VavGhsdOjNR9BWEH+OyKJtWkP2tzYHSBZkWOd+bPYGf3aycWwK3dR
         5AcYy91QhF4EWAkTntb+pDmNbkQh/lu+0dMWAmKrCnjogYAeAREeGUxDpSy09G2xNyhs
         qjBPxvCuoosqHoRPQmFxJoYqRLOVDRwwdj78OZqKkHbZozqg78I7D3gfuhYHkmhYm7sH
         VU6A==
X-Gm-Message-State: AOJu0YzTia7UH3XzDWubADfMiV3i4QiSbs/51FGjSVSalvYwPf77EVkL
	sbsAyERjk03N+tWQwrw0HeB1koF0G2WGV/RdLzsIIJlmTb2e3OPKty7OVfy5P58=
X-Google-Smtp-Source: AGHT+IFn+85uiEkgIEBgCzqCK7wWA6y/WRfGtkj5izqKSQl2/Ukj7jAhgKxd9BH+w+fI77QnRPUmiQ==
X-Received: by 2002:a05:6214:e46:b0:6a1:55de:7a23 with SMTP id 6a1803df08f44-6ab8f328394mr44128656d6.6.1716474728984;
        Thu, 23 May 2024 07:32:08 -0700 (PDT)
Date: Thu, 23 May 2024 16:32:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 1/8] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
Message-ID: <Zk9TZhe-WSNrnlPD@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <4095f31a88589ced2b620e8ebbb84cdc2fae8914.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4095f31a88589ced2b620e8ebbb84cdc2fae8914.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:20PM +0100, Alejandro Vallejo wrote:
> This allows the initial x2APIC ID to be sent on the migration stream. The
> hardcoded mapping x2apic_id=2*vcpu_id is maintained for the time being.
> Given the vlapic data is zero-extended on restore, fix up migrations from
> hosts without the field by setting it to the old convention if zero.
> 
> x2APIC IDs are calculated from the CPU policy where the guest topology is
> defined. For the time being, the function simply returns the old
> relationship, but will eventually return results consistent with the
> topology.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * Removed usage of SET_xAPIC_ID().
>   * Restored previous logic when exposing leaf 0xb, and gate it for HVM only.
>   * Rewrote comment in lapic_load_fixup, including the implicit assumption.
>   * Moved vlapic_cpu_policy_changed() into hvm_cpuid_policy_changed())
>   * const-ified policy in vlapic_cpu_policy_changed()
> ---
>  xen/arch/x86/cpuid.c                   | 15 ++++---------
>  xen/arch/x86/hvm/vlapic.c              | 30 ++++++++++++++++++++++++--
>  xen/arch/x86/include/asm/hvm/hvm.h     |  1 +
>  xen/arch/x86/include/asm/hvm/vlapic.h  |  2 ++
>  xen/include/public/arch-x86/hvm/save.h |  2 ++
>  xen/include/xen/lib/x86/cpu-policy.h   |  9 ++++++++
>  xen/lib/x86/policy.c                   | 11 ++++++++++
>  7 files changed, 57 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
> index 7a38e032146a..242c21ec5bb6 100644
> --- a/xen/arch/x86/cpuid.c
> +++ b/xen/arch/x86/cpuid.c
> @@ -139,10 +139,9 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
>          const struct cpu_user_regs *regs;
>  
>      case 0x1:
> -        /* TODO: Rework topology logic. */
>          res->b &= 0x00ffffffu;
>          if ( is_hvm_domain(d) )
> -            res->b |= (v->vcpu_id * 2) << 24;
> +            res->b |= vlapic_x2apic_id(vcpu_vlapic(v)) << 24;
>  
>          /* TODO: Rework vPMU control in terms of toolstack choices. */
>          if ( vpmu_available(v) &&
> @@ -311,19 +310,13 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
>          break;
>  
>      case 0xb:
> -        /*
> -         * In principle, this leaf is Intel-only.  In practice, it is tightly
> -         * coupled with x2apic, and we offer an x2apic-capable APIC emulation
> -         * to guests on AMD hardware as well.
> -         *
> -         * TODO: Rework topology logic.
> -         */
> -        if ( p->basic.x2apic )
> +        /* Don't expose topology information to PV guests */

Not sure whether we want to keep part of the comment about exposing
x2APIC to guests even when x2APIC is not present in the host.  I think
this code has changed and the comment is kind of stale now.

> +        if ( is_hvm_domain(d) && p->basic.x2apic )
>          {
>              *(uint8_t *)&res->c = subleaf;
>  
>              /* Fix the x2APIC identifier. */
> -            res->d = v->vcpu_id * 2;
> +            res->d = vlapic_x2apic_id(vcpu_vlapic(v));
>          }
>          break;
>  
> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> index 05072a21bf38..61a96474006b 100644
> --- a/xen/arch/x86/hvm/vlapic.c
> +++ b/xen/arch/x86/hvm/vlapic.c
> @@ -1069,7 +1069,7 @@ static uint32_t x2apic_ldr_from_id(uint32_t id)
>  static void set_x2apic_id(struct vlapic *vlapic)
>  {
>      const struct vcpu *v = vlapic_vcpu(vlapic);
> -    uint32_t apic_id = v->vcpu_id * 2;
> +    uint32_t apic_id = vlapic->hw.x2apic_id;
>      uint32_t apic_ldr = x2apic_ldr_from_id(apic_id);
>  
>      /*
> @@ -1083,6 +1083,22 @@ static void set_x2apic_id(struct vlapic *vlapic)
>      vlapic_set_reg(vlapic, APIC_LDR, apic_ldr);
>  }
>  
> +void vlapic_cpu_policy_changed(struct vcpu *v)
> +{
> +    struct vlapic *vlapic = vcpu_vlapic(v);
> +    const struct cpu_policy *cp = v->domain->arch.cpu_policy;
> +
> +    /*
> +     * Don't override the initial x2APIC ID if we have migrated it or
> +     * if the domain doesn't have vLAPIC at all.
> +     */
> +    if ( !has_vlapic(v->domain) || vlapic->loaded.hw )
> +        return;
> +
> +    vlapic->hw.x2apic_id = x86_x2apic_id_from_vcpu_id(cp, v->vcpu_id);
> +    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));

Nit: in case we decide to start APICs in x2APIC mode, might be good to
take this into account here and use vlapic_x2apic_mode(vlapic) to
select whether SET_xAPIC_ID() needs to be used or not:

    vlapic_set_reg(vlapic, APIC_ID,
        vlapic_x2apic_mode(vlapic) ? vlapic->hw.x2apic_id
	                           : SET_xAPIC_ID(vlapic->hw.x2apic_id));

Or similar.

> +}
> +
>  int guest_wrmsr_apic_base(struct vcpu *v, uint64_t val)
>  {
>      const struct cpu_policy *cp = v->domain->arch.cpu_policy;
> @@ -1449,7 +1465,7 @@ void vlapic_reset(struct vlapic *vlapic)
>      if ( v->vcpu_id == 0 )
>          vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
>  
> -    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
> +    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
>      vlapic_do_init(vlapic);
>  }
>  
> @@ -1514,6 +1530,16 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>      const struct vcpu *v = vlapic_vcpu(vlapic);
>      uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id);
>  
> +    /*
> +     * Loading record without hw.x2apic_id in the save stream, calculate using
> +     * the traditional "vcpu_id * 2" relation. There's an implicit assumption
> +     * that vCPU0 always has x2APIC0, which is true for the old relation, and
> +     * still holds under the new x2APIC generation algorithm. While that case
> +     * goes through the conditional it's benign because it still maps to zero.
> +     */
> +    if ( !vlapic->hw.x2apic_id )
> +        vlapic->hw.x2apic_id = v->vcpu_id * 2;
> +
>      /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */
>      if ( !vlapic_x2apic_mode(vlapic) ||
>           (vlapic->loaded.ldr == good_ldr) )
> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
> index 0c9e6f15645d..e1f0585d75a9 100644
> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -448,6 +448,7 @@ static inline void hvm_update_guest_efer(struct vcpu *v)
>  static inline void hvm_cpuid_policy_changed(struct vcpu *v)
>  {
>      alternative_vcall(hvm_funcs.cpuid_policy_changed, v);
> +    vlapic_cpu_policy_changed(v);

Note sure whether this call would better be placed in
cpu_policy_updated() inside the is_hvm_vcpu() conditional branch.

hvm_cpuid_policy_changed() &c are just wrappers around the hvm_funcs
hooks, pulling vlapic functions in there is likely to complicate the
header dependencies in the long term.

>  }
>  
>  static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
> diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
> index 88ef94524339..e8d41313abd3 100644
> --- a/xen/arch/x86/include/asm/hvm/vlapic.h
> +++ b/xen/arch/x86/include/asm/hvm/vlapic.h
> @@ -44,6 +44,7 @@
>  #define vlapic_xapic_mode(vlapic)                               \
>      (!vlapic_hw_disabled(vlapic) && \
>       !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD))
> +#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id)
>  
>  /*
>   * Generic APIC bitmap vector update & search routines.
> @@ -107,6 +108,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack);
>  
>  int  vlapic_init(struct vcpu *v);
>  void vlapic_destroy(struct vcpu *v);
> +void vlapic_cpu_policy_changed(struct vcpu *v);
>  
>  void vlapic_reset(struct vlapic *vlapic);
>  
> diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
> index 7ecacadde165..1c2ec669ffc9 100644
> --- a/xen/include/public/arch-x86/hvm/save.h
> +++ b/xen/include/public/arch-x86/hvm/save.h
> @@ -394,6 +394,8 @@ struct hvm_hw_lapic {
>      uint32_t             disabled; /* VLAPIC_xx_DISABLED */
>      uint32_t             timer_divisor;
>      uint64_t             tdt_msr;
> +    uint32_t             x2apic_id;
> +    uint32_t             rsvd_zero;

I think Jan requested for this field to be checked to be 0 for
incoming migrations, yet I don't see such check added.

>  };
>  
>  DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic);
> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
> index d5e447e9dc06..392320b9adbe 100644
> --- a/xen/include/xen/lib/x86/cpu-policy.h
> +++ b/xen/include/xen/lib/x86/cpu-policy.h
> @@ -542,6 +542,15 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>                                      const struct cpu_policy *guest,
>                                      struct cpu_policy_errors *err);
>  
> +/**
> + * Calculates the x2APIC ID of a vCPU given a CPU policy
> + *
> + * @param p          CPU policy of the domain.
> + * @param id         vCPU ID of the vCPU.
> + * @returns x2APIC ID of the vCPU.
> + */
> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
> +
>  #endif /* !XEN_LIB_X86_POLICIES_H */
>  
>  /*
> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
> index f033d22785be..4cef658feeb8 100644
> --- a/xen/lib/x86/policy.c
> +++ b/xen/lib/x86/policy.c
> @@ -2,6 +2,17 @@
>  
>  #include <xen/lib/x86/cpu-policy.h>
>  
> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
> +{
> +    /*
> +     * TODO: Derive x2APIC ID from the topology information inside `p`
> +     *       rather than from vCPU ID. This bodge is a temporary measure
> +     *       until all infra is in place to retrieve or derive the initial
> +     *       x2APIC ID from migrated domains.
> +     */
> +    return vcpu_id * 2;

I don't think this builds?

Note the parameter is plain 'id' not 'vcpu_id'.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:33:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:33:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728571.1133534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Vp-0008Eq-3w; Thu, 23 May 2024 14:33:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728571.1133534; Thu, 23 May 2024 14:33:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Vp-0008Eh-0q; Thu, 23 May 2024 14:33:45 +0000
Received: by outflank-mailman (input) for mailman id 728571;
 Thu, 23 May 2024 14:33:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sA9Vn-0008EY-R4
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:33:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA9Vn-0007Xr-7E; Thu, 23 May 2024 14:33:43 +0000
Received: from [15.248.2.28] (helo=[10.24.67.32])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sA9Vm-0002YX-Uc; Thu, 23 May 2024 14:33:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=fJXGwtxvcP/vQZnTamcjtT2VD0gb9ridDnimaDlHR44=; b=ix/rtpIuZu7EEnDnQOUlpOvn9W
	gH6i18q4iTcsb84FhUNiBnSoSiz7pWBi+Jm/ezAUEMaPg/c/g41Se8/0V8/4gKlKw2ppQcd0uSYXA
	Denmwus2mJbO1LfsO1DDe89tHNAj8LWmGT6FkDh8bqryeGlvyWLUkj1gA1qK0No69ifo=;
Message-ID: <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
Date: Thu, 23 May 2024 15:33:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 23/05/2024 15:11, Oleksii K. wrote:
> On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
>> Hi Oleksii,
> Hi Julien,
> 
>>
>> On 17/05/2024 14:54, Oleksii Kurochko wrote:
>>> diff --git a/xen/arch/arm/arm64/livepatch.c
>>> b/xen/arch/arm/arm64/livepatch.c
>>> index df2cebedde..4bc8ed9be5 100644
>>> --- a/xen/arch/arm/arm64/livepatch.c
>>> +++ b/xen/arch/arm/arm64/livepatch.c
>>> @@ -10,7 +10,6 @@
>>>    #include <xen/mm.h>
>>>    #include <xen/vmap.h>
>>>    
>>> -#include <asm/bitops.h>
>>
>> It is a bit unclear how this change is related to the patch. Can you
>> explain in the commit message?
> Probably it doesn't need anymore. I will double check and if this
> change is not needed, I will just drop it in the next patch version.
> 
>>
>>>    #include <asm/byteorder.h>
>>>    #include <asm/insn.h>
>>>    #include <asm/livepatch.h>
>>> diff --git a/xen/arch/arm/include/asm/bitops.h
>>> b/xen/arch/arm/include/asm/bitops.h
>>> index 5104334e48..8e16335e76 100644
>>> --- a/xen/arch/arm/include/asm/bitops.h
>>> +++ b/xen/arch/arm/include/asm/bitops.h
>>> @@ -22,9 +22,6 @@
>>>    #define __set_bit(n,p)            set_bit(n,p)
>>>    #define __clear_bit(n,p)          clear_bit(n,p)
>>>    
>>> -#define BITOP_BITS_PER_WORD     32
>>> -#define BITOP_MASK(nr)          (1UL << ((nr) %
>>> BITOP_BITS_PER_WORD))
>>> -#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
>>>    #define BITS_PER_BYTE           8
>>
>> OOI, any reason BITS_PER_BYTE has not been moved as well? I don't
>> expect
>> the value to change across arch.
> I can move it to generic one header too in the next patch version.
> 
>>
>> [...]
>>
>>> diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
>>> index f14ad0d33a..6eeeff0117 100644
>>> --- a/xen/include/xen/bitops.h
>>> +++ b/xen/include/xen/bitops.h
>>> @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned long
>>> x)
>>>     * scope
>>>     */
>>>    
>>> +#define BITOP_BITS_PER_WORD 32
>>> +typedef uint32_t bitop_uint_t;
>>> +
>>> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) %
>>> BITOP_BITS_PER_WORD))
>>> +
>>> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
>>> +
>>> +extern void __bitop_bad_size(void);
>>> +
>>> +#define bitop_bad_size(addr) (sizeof(*(addr)) <
>>> sizeof(bitop_uint_t))
>>> +
>>>    /* --------------------- Please tidy above here -----------------
>>> ---- */
>>>    
>>>    #include <asm/bitops.h>
>>>    
>>> +/**
>>> + * generic__test_and_set_bit - Set a bit and return its old value
>>> + * @nr: Bit to set
>>> + * @addr: Address to count from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>
>> Sorry for only mentioning this on v10. I think this comment should be
>> duplicated (or moved to) on top of test_bit() because this is what
>> everyone will use. This will avoid the developper to follow the
>> function
>> calls and only notice the x86 version which says "This function is
>> atomic and may not be reordered." and would be wrong for all the
>> other arch.
> It makes sense to add this comment on top of test_bit(), but I am
> curious if it is needed to mention that for x86 arch_test_bit() "is
> atomic and may not be reordered":

I would say no because any developper modifying common code can't 
relying it.

> 
>   * This operation is non-atomic and can be reordered. ( Exception: for
> * x86 arch_test_bit() is atomic and may not be reordered )
>   * If two examples of this operation race, one can appear to succeed
>   * but actually fail.  You must protect multiple accesses with a lock.
>   */
> 
>>
>>> +static always_inline bool
>>> +generic__test_and_set_bit(int nr, volatile void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old | mask;
>>> +    return (old & mask);
>>> +}
>>> +
>>> +/**
>>> + * generic__test_and_clear_bit - Clear a bit and return its old
>>> value
>>> + * @nr: Bit to clear
>>> + * @addr: Address to count from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>
>> Same applies here and ...
>>
>>> +static always_inline bool
>>> +generic__test_and_clear_bit(int nr, volatile void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old & ~mask;
>>> +    return (old & mask);
>>> +}
>>> +
>>> +/* WARNING: non atomic and it can be reordered! */
>>
>> ... here.
>>
>>> +static always_inline bool
>>> +generic__test_and_change_bit(int nr, volatile void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +    bitop_uint_t old = *p;
>>> +
>>> +    *p = old ^ mask;
>>> +    return (old & mask);
>>> +}
>>> +/**
>>> + * generic_test_bit - Determine whether a bit is set
>>> + * @nr: bit number to test
>>> + * @addr: Address to start counting from
>>> + */
>>> +static always_inline bool generic_test_bit(int nr, const volatile
>>> void *addr)
>>> +{
>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>> +    const volatile bitop_uint_t *p =
>>> +                        (const volatile bitop_uint_t *)addr +
>>> BITOP_WORD(nr);
>>> +
>>> +    return (*p & mask);
>>> +}
>>> +
>>> +static always_inline bool
>>> +__test_and_set_bit(int nr, volatile void *addr)
>>> +{
>>> +#ifndef arch__test_and_set_bit
>>> +#define arch__test_and_set_bit generic__test_and_set_bit
>>> +#endif
>>> +
>>> +    return arch__test_and_set_bit(nr, addr);
>>> +}
>>
>> NIT: It is a bit too late to change this one. But I have to admit, I
>> don't understand the purpose of the static inline when you could have
>> simply call...
>>
>>> +#define __test_and_set_bit(nr, addr) ({             \
>>> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
>>> +    __test_and_set_bit(nr, addr);                   \
>>
>> ... __arch__test_and_set_bit here.
> I just wanted to be in sync with other non-atomic test*_bit()
> operations and Andrew's patch series ( xen/bitops: Reduce the mess,
> starting with ffs() ).

I looked at Andrew series and I can't seem to find an example where he 
uses both the macro + static inline. He seems only use a static inline.
Do you have any pointer?

But by any chance are you referring to the pattern on x86? If so, I 
would really like to understand what's the benefits of introducing a a 
one-liner static inline which is "shadowed" by a macro...

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:34:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:34:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728576.1133544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Wm-0000To-G6; Thu, 23 May 2024 14:34:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728576.1133544; Thu, 23 May 2024 14:34:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9Wm-0000Th-Cb; Thu, 23 May 2024 14:34:44 +0000
Received: by outflank-mailman (input) for mailman id 728576;
 Thu, 23 May 2024 14:34:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9Wl-0000TZ-6S
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:34:43 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 977c0c58-1911-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:34:41 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a5a5cce2ce6so1032633666b.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:34:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b179casm1936481766b.203.2024.05.23.07.34.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:34:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 977c0c58-1911-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716474881; x=1717079681; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tNwXYm0mm5d+jelZ0oLji3E88NIrdy2izCUlWbmVy+0=;
        b=AI2i7tg8bilrhne5BnKKNBgxCmVcc+ACl96CX9KR3jakASIn5fjS6+a+jVPsp4vunh
         /Q/+0T2s+rEZPJ36IU18zvhpXHLWsa6TJZEc9s32hyNVVuy/CMriWdhYkt9MLr7rzR0U
         lkzVH7Czho9dxpDR5/7Lb79PQFo/LDwZFoTyE3Ft+sy5WZQP3OAOecz7f8jFwknPM5Tv
         SbMp3JR80Tv1kFSN3lg72g3SaD56L5vmL95d+GljzMjtLWj1fD9msXR7ZM1rpNc3cCXU
         E2SeNO6nlt0DtPwLh851twGCr+oVRhuNUxDc5rt0QQlAJ0nr1EQ6oIxxpGmfApHANRPo
         ko9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716474881; x=1717079681;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tNwXYm0mm5d+jelZ0oLji3E88NIrdy2izCUlWbmVy+0=;
        b=mz1U/XfmtGhhEAmq6Qjz9xJ4kEz3tgUM0sbZ2ZnKMZgMZEWwvQBgVZMfV/FuB38oaO
         M0DE+gv6qluoHVRpz1vYGNLSyk0nzeuloFXUWnWpVs5apygbGZ907D+RJEAzEW28kIrm
         9Z8nFCVMl+/jLwXJFZc4N/t3j3Iw563T2tZguUAm+cWD2WqGr1lF5Ig91PZgDBGLd4aH
         Sc1F1VNT4l3s+gl9rMCgghacbaYzhxtF81T9dkcQvZPSmM54XwJEEoFUvfQ80az5NXZs
         FUi6z6Ei6aVAHSWqFQqXCZWXrGdbrKMsTRQGgeCcsekPfvgk7OdkHbhBcrV/5s6jnFz1
         AFpQ==
X-Gm-Message-State: AOJu0Yw/dwPQ0+iVu80FZ3K1DNhyWAP7Gk/VDtg+Nrt2RbrJz6rBIshk
	UU7kdLrPR+symXi0ZTe7SbGqEigZYQVo6A9RTGbfk/JVIEl1SUlSLg3EPk6WsQ==
X-Google-Smtp-Source: AGHT+IGQg9juDUZsDQ/8k7BlyDhY+HqBJ/499kP5CMEUnrtk1MpboZRyXi1s/kFOHLPrR3TeN4hLoQ==
X-Received: by 2002:a17:906:3404:b0:a59:c28a:7ec2 with SMTP id a640c23a62f3a-a62280970ebmr313682966b.41.1716474881628;
        Thu, 23 May 2024 07:34:41 -0700 (PDT)
Message-ID: <189c73d8-9a5b-4983-affe-0c39878f9304@suse.com>
Date: Thu, 23 May 2024 16:34:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [xen-unstable-smoke test] 186107: regressions - FAIL
Content-Language: en-US
To: osstest service owner <osstest-admin@xenproject.org>
References: <osstest-186107-mainreport@xen.org>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <osstest-186107-mainreport@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 15:45, osstest service owner wrote:
> flight 186107 xen-unstable-smoke real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/186107/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  build-armhf                   6 xen-build                fail REGR. vs. 186064

Found ninja-1.11.1 at /usr/bin/ninja

ERROR: Clock skew detected. File /usr/bin/bash has a time stamp 1682259478.4465s in the future.

A full log can be found at /home/osstest/build.186107.build-armhf/xen/tools/qemu-xen-build/meson-logs/meson-log.txt

ERROR: meson setup failed

make: Entering directory '/home/osstest/build.186107.build-armhf/xen/tools/qemu-xen-build'
config-host.mak is out-of-date, running configure
  GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
bash: line 4: ./config.status: No such file or directory
make: *** No rule to make target 'config-host.mak', needed by 'Makefile.prereqs'.  Stop.
make: *** Waiting for unfinished jobs....
make: Leaving directory '/home/osstest/build.186107.build-armhf/xen/tools/qemu-xen-build'
make[2]: *** [Makefile:212: subdir-all-qemu-xen-dir] Error 2
make[2]: Leaving directory '/home/osstest/build.186107.build-armhf/xen/tools'
make[1]: *** [/home/osstest/build.186107.build-armhf/xen/tools/../tools/Rules.mk:199: subdirs-all] Error 2
make[1]: Leaving directory '/home/osstest/build.186107.build-armhf/xen/tools'
make: *** [Makefile:63: build-tools] Error 2

Suggest to me that there's some issue with the build host.

Jan



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:37:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:37:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728586.1133554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ZU-0001Dx-Tq; Thu, 23 May 2024 14:37:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728586.1133554; Thu, 23 May 2024 14:37:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9ZU-0001Dq-Q7; Thu, 23 May 2024 14:37:32 +0000
Received: by outflank-mailman (input) for mailman id 728586;
 Thu, 23 May 2024 14:37:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA9ZT-0001Dk-PE
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:37:31 +0000
Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com
 [2607:f8b0:4864:20::f36])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb2afffa-1911-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:37:29 +0200 (CEST)
Received: by mail-qv1-xf36.google.com with SMTP id
 6a1803df08f44-69b782287f9so11219686d6.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:37:29 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f185787sm144487306d6.45.2024.05.23.07.37.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:37:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb2afffa-1911-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716475048; x=1717079848; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=vR9yvXuCCsbyLrcpWVWbYCim0WWirbw4WHo2dHfc/sY=;
        b=kU+f+1H/qqc/tC/NDylnbmwz1WgGns2MkfZdR7/oELvAoSdf5d40mQ4L2r0dpjUe4g
         DAhHOvMj0yljpLJLkqPrI4LdOD/h8oYIYZk5+R73P/4HndxESLY2DWx46fbzmfhEQLOG
         k0z/LKL4m9ycfYtdeQmeqLTar17xCqNqLFMpA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475048; x=1717079848;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vR9yvXuCCsbyLrcpWVWbYCim0WWirbw4WHo2dHfc/sY=;
        b=aJGbDs8PA7QL16p6ofhtettvFpuGdx3ykUV45cH1RK0DPRCsGWkk/R3bAAEglO0lD9
         AClTWcVl9F7ShPxr7a+nQWRlN/p+fc6YYR/zK850xiXGnHV4bqhxj12jvl+a5fH7OYDc
         d+Me6hnNg7WB1ndlzTKUUEO1D8G0fcnSap6ZrgZ8+Kb440jU02dJXVzORxdyfiD9i9uB
         LaCxlxOfC8ZvmMlN2lF/FbFkJhhjhu/wFEQfh8XCdAtC2j03U9+V7C8Ii3NzfvfzdIHw
         TsVsPEulJFEbQ7zbS4LtGqoIG1ghos804zDNigUal32NOFzV9H0DRc5xga+vLQL6/F5l
         FVXg==
X-Gm-Message-State: AOJu0Yx4UCUSvvwXvHx0xJf1h7KQSYAKRjfBmXx4TTqtvXr6gvHhMJhr
	ysmcJE9tgHKCe4nIiBBOU9ze/iEr3IAoJ4kEMfVP98fXU9T4SM9SP4rpHHhdfwA=
X-Google-Smtp-Source: AGHT+IHUF0bpiHJdoF4nm36eJG60w2siviK6P4ncJvgfAEoDgXUISEGgb1vkS+oNXbDed9VTKl/+6Q==
X-Received: by 2002:a05:6214:5b0a:b0:6aa:7373:942c with SMTP id 6a1803df08f44-6ab7f363e3bmr67762676d6.25.1716475048462;
        Thu, 23 May 2024 07:37:28 -0700 (PDT)
Date: Thu, 23 May 2024 16:37:26 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
Message-ID: <Zk9Upg9y03IYZEXd@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:21PM +0100, Alejandro Vallejo wrote:
> Otherwise it's not possible to call functions described in hvm/vlapic.h from the
> inline functions of hvm/hvm.h.
> 
> This is because a static inline in vlapic.h depends on hvm.h, and pulls it
> transitively through vpt.h. The ultimate cause is having hvm.h included in any
> of the "v*.h" headers, so break the cycle moving the guilty inline into hvm.h.
> 
> No functional change.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

One cosmetic comment below.

> ---
> v2:
>   * New patch. Prereq to moving vlapic_cpu_policy_changed() onto hvm.h
> ---
>  xen/arch/x86/hvm/irq.c                | 6 +++---
>  xen/arch/x86/hvm/vlapic.c             | 4 ++--
>  xen/arch/x86/include/asm/hvm/hvm.h    | 6 ++++++
>  xen/arch/x86/include/asm/hvm/vlapic.h | 6 ------
>  xen/arch/x86/include/asm/hvm/vpt.h    | 1 -
>  5 files changed, 11 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
> index 4a9fe82cbd8d..4f5479b12c98 100644
> --- a/xen/arch/x86/hvm/irq.c
> +++ b/xen/arch/x86/hvm/irq.c
> @@ -512,13 +512,13 @@ struct hvm_intack hvm_vcpu_has_pending_irq(struct vcpu *v)
>      int vector;
>  
>      /*
> -     * Always call vlapic_sync_pir_to_irr so that PIR is synced into IRR when
> -     * using posted interrupts. Note this is also done by
> +     * Always call hvm_vlapic_sync_pir_to_irr so that PIR is synced into IRR
> +     * when using posted interrupts. Note this is also done by
>       * vlapic_has_pending_irq but depending on which interrupts are pending
>       * hvm_vcpu_has_pending_irq will return early without calling
>       * vlapic_has_pending_irq.
>       */
> -    vlapic_sync_pir_to_irr(v);
> +    hvm_vlapic_sync_pir_to_irr(v);
>  
>      if ( unlikely(v->arch.nmi_pending) )
>          return hvm_intack_nmi;
> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> index 61a96474006b..8a244100009c 100644
> --- a/xen/arch/x86/hvm/vlapic.c
> +++ b/xen/arch/x86/hvm/vlapic.c
> @@ -98,7 +98,7 @@ static void vlapic_clear_irr(int vector, struct vlapic *vlapic)
>  
>  static int vlapic_find_highest_irr(struct vlapic *vlapic)
>  {
> -    vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic));
> +    hvm_vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic));
>  
>      return vlapic_find_highest_vector(&vlapic->regs->data[APIC_IRR]);
>  }
> @@ -1516,7 +1516,7 @@ static int cf_check lapic_save_regs(struct vcpu *v, hvm_domain_context_t *h)
>      if ( !has_vlapic(v->domain) )
>          return 0;
>  
> -    vlapic_sync_pir_to_irr(v);
> +    hvm_vlapic_sync_pir_to_irr(v);
>  
>      return hvm_save_entry(LAPIC_REGS, v->vcpu_id, h, vcpu_vlapic(v)->regs);
>  }
> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
> index e1f0585d75a9..84911f3ebcb4 100644
> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
>          alternative_vcall(hvm_funcs.update_vlapic_mode, v);
>  }
>  
> +static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
> +{
> +    if ( hvm_funcs.sync_pir_to_irr )
> +        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);

Nit: for consistency the wrappers are usually named hvm_<hook_name>,
so in this case it would be hvm_sync_pir_to_irr(), or the hvm_funcs
field should be renamed to vlapic_sync_pir_to_irr.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:40:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:40:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728591.1133564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9c4-0002ft-9m; Thu, 23 May 2024 14:40:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728591.1133564; Thu, 23 May 2024 14:40:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9c4-0002fm-6x; Thu, 23 May 2024 14:40:12 +0000
Received: by outflank-mailman (input) for mailman id 728591;
 Thu, 23 May 2024 14:40:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9c3-0002ff-23
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:40:11 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a5d610d-1912-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:40:08 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51f72a29f13so8471526e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:40:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a8b634cdcsm1273944166b.93.2024.05.23.07.40.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:40:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a5d610d-1912-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716475208; x=1717080008; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Xqmg/5bHXpvAGEEaqsoUjDPL7Nq0u+YnyGx8vXG1hqE=;
        b=EOr79bkXTWBKk2Io6oo51ZETSbuWYihiN+3IHX1oH2ef8qRd3NfBoa+OEqNoKTIHTR
         H8l/F+0SGw/59kw7O3bEoCtu19z2AYI+mbyL2hACiZW+8wgHp/FiR+lvdOuxMDTVjBUH
         NdTG+8cUJ9E/v3luAb+3yHR+JskK4eh+E35b9QQ1jZWzuxPmSWQN9upiSwvL1Cy2ALb6
         jE6GliwooDQq8jy/2W4gi+XVfoMrJairbGjk/OQTVTQK5NxpZewUin7xQuh7ncRVr7HG
         CMeVEMnHYUQpbGrmocfPH3/cScr7F2ANEINct0yzOLRo8g45l+UR/CxSFhCusEeWwVov
         cy7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475208; x=1717080008;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Xqmg/5bHXpvAGEEaqsoUjDPL7Nq0u+YnyGx8vXG1hqE=;
        b=TRV4lQU6h8xR7t4osCmxHTtVcbPvIYKRkThdeZoZJ36LDBIDlhBZBAQCOT6d5kA6/g
         y7ECmMeWEcyzMOrUNBUrVLHgeGMK6DO8o9gCmnt635gKHCNSEhHS9inzvtLGXpnvp3gC
         8EP5Flw+6wqMnhmr+xXVFnAbFyDizY9Vr4bluL74vDgo5lQyCk971C35aAxPQtgPh0+r
         xeM1ilUwUOiNrhNWqpOzopK8UPnt3HJonLJS0XMJVzqeqp3EZuCCgP+Gvpah9+O6SQ1H
         o7a9XPT1msiCxyFrcNegiYBuGPAKA24pWgsu6B3u/wgDhRN3p/Q9BtFRVcIZg7wZQshs
         7/TQ==
X-Gm-Message-State: AOJu0Yy6NTbk53VGAOjDGXnKvEczJSgyex1gvxnudUZL6zMZBVL2Ngwc
	QKCEFBCoCgvp0J9rZasLfqBLvAo+3cxh02fNmlk6LXcg1uMOM5UjeMs8EG44Ew==
X-Google-Smtp-Source: AGHT+IEItCjRFcPmFHxm1roTCY7vzDOBfQ5d/etwRM880v92g4LJoIuF3mg/asujHHrqrPEZAbYBJw==
X-Received: by 2002:ac2:4844:0:b0:51f:b781:72a7 with SMTP id 2adb3069b0e04-526bf451ca3mr3348299e87.60.1716475208349;
        Thu, 23 May 2024 07:40:08 -0700 (PDT)
Message-ID: <8dd92e22-2a7e-414b-98b2-c10ae449f825@suse.com>
Date: Thu, 23 May 2024 16:40:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Alejandro Vallejo <alejandro.vallejo@cloud.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9Upg9y03IYZEXd@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zk9Upg9y03IYZEXd@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.05.2024 16:37, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:21PM +0100, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>> @@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
>>          alternative_vcall(hvm_funcs.update_vlapic_mode, v);
>>  }
>>  
>> +static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
>> +{
>> +    if ( hvm_funcs.sync_pir_to_irr )
>> +        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
> 
> Nit: for consistency the wrappers are usually named hvm_<hook_name>,
> so in this case it would be hvm_sync_pir_to_irr(), or the hvm_funcs
> field should be renamed to vlapic_sync_pir_to_irr.

Funny you should mention that: See my earlier comment as well as what
was committed.

Jan



From xen-devel-bounces@lists.xenproject.org Thu May 23 14:40:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:40:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728594.1133573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9cZ-00037y-IL; Thu, 23 May 2024 14:40:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728594.1133573; Thu, 23 May 2024 14:40:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9cZ-00037r-F6; Thu, 23 May 2024 14:40:43 +0000
Received: by outflank-mailman (input) for mailman id 728594;
 Thu, 23 May 2024 14:40:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA9cX-00036K-Im; Thu, 23 May 2024 14:40:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA9cX-0007gS-ES; Thu, 23 May 2024 14:40:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sA9cX-0005nt-39; Thu, 23 May 2024 14:40:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sA9cX-0001bD-2Z; Thu, 23 May 2024 14:40:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OJZv5Cc998hOWbs9eelM2YxFHBPFFpI849nuwzqPbiM=; b=mDI4h2uuNLOslR7+HHvYPaslNe
	qB4B8037iEJnf3dJon6LCiqna1OOZr0i16xmEx69VR2ulqCFP6L2jT5uWEpdzRQ/m7MbYMQbo+1kV
	0Y6JlB9ZcziXklc6cyXEig/l3pSarTuP1SuC12lA7/hqrNSIgCv7oVd5mBuB1mEWGO1Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186087-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186087: regressions - FAIL
X-Osstest-Failures:
    xen-4.17-testing:build-amd64:xen-build:fail:regression
    xen-4.17-testing:build-amd64-xsm:xen-build:fail:regression
    xen-4.17-testing:build-i386-xsm:xen-build:fail:regression
    xen-4.17-testing:build-i386:xen-build:fail:regression
    xen-4.17-testing:build-amd64-prev:xen-build:fail:regression
    xen-4.17-testing:build-i386-prev:xen-build:fail:regression
    xen-4.17-testing:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:build-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-4.17-testing:build-i386-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-livepatch:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-5:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-3:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-4:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 14:40:41 +0000

flight 186087 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186087/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 185864
 build-amd64-xsm               6 xen-build                fail REGR. vs. 185864
 build-i386-xsm                6 xen-build                fail REGR. vs. 185864
 build-i386                    6 xen-build                fail REGR. vs. 185864
 build-amd64-prev              6 xen-build                fail REGR. vs. 185864
 build-i386-prev               6 xen-build                fail REGR. vs. 185864

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-livepatch    1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-5        1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-vhd       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-1        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-2        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-3        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-4        1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   24 days
Failing since        186063  2024-05-21 10:06:36 Z    2 days    3 attempts
Testing same since   186069  2024-05-22 01:58:18 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64-xtf                                              pass    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           blocked 
 build-amd64-prev                                             fail    
 build-i386-prev                                              fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       blocked 
 test-xtf-amd64-amd64-2                                       blocked 
 test-xtf-amd64-amd64-3                                       blocked 
 test-xtf-amd64-amd64-4                                       blocked 
 test-xtf-amd64-amd64-5                                       blocked 
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   blocked 
 test-amd64-amd64-migrupgrade                                 blocked 
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 blocked 
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      blocked 
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      blocked 
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 438 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:43:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:43:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728602.1133584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9fB-00048K-3W; Thu, 23 May 2024 14:43:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728602.1133584; Thu, 23 May 2024 14:43:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9fB-00048D-0X; Thu, 23 May 2024 14:43:25 +0000
Received: by outflank-mailman (input) for mailman id 728602;
 Thu, 23 May 2024 14:43:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9fA-000487-0d
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:43:24 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cdf2cf68-1912-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:43:22 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a59c448b44aso1150947566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:43:23 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a62492a007dsm53930066b.152.2024.05.23.07.43.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:43:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf2cf68-1912-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716475402; x=1717080202; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=AS/d7sOh47fXukWezPKmzml0EbcN94a3FiVo29oxpP4=;
        b=HkhYxhxLDN9GVDuNEm4b6Tse9AaNYVBlQaEp5dOb8nFQxTxxoZyl6fZZUT0kC3QadA
         2AnKCgJPbflDrTkwBBE8/dyI5rlaeRMxx7d9rd9FpGHTRtg7Im2U/Ty42vfflUzhOX87
         jepCKNvyWPycQGLyArOsECVGGm0y0HLrYThYaQA2jaH/8TVJXIZEa3MFOoyep8FIyQey
         YpgXih7cO3FbcR0XQGgzqL7LodhqeHCwE+W7b2AJ4eFmBN9cuFVqpiAuHjMVy3sziq5F
         E0FJEYZE59hFLV3C19rxC7x0kVVSFKeXR+Gue2Ag77pfhbaneVGJ9mzvL1uFceICycOy
         zatQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475402; x=1717080202;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=AS/d7sOh47fXukWezPKmzml0EbcN94a3FiVo29oxpP4=;
        b=aSPgcE/S5HZRHH3iC2V5x8+RZbYR/AMLcZaLynpmnplWNeFzfg+GKTCB/8cNGUxlD6
         +8MUQDRr2PnslRTDiIVoJ14AGa+TvjVgunJx7a5WHJ30T/PJiIiU8WclZoEaSzJoxYJl
         hpickAIZuZ7sFkl8T+TVaDwLqFw20DcdPw7TMahY1szvdbeZk9QJUdbLNFQWiWDy+GXz
         3FACHiTWYvhdZHfIJ2FJlHilDC6Br96hlpyx1FgROpkfYTgLwfUGdwPrzdkyPlJ455+A
         vSKNBDWJj6eq4VWAVOYV82WVbYFxi1Rv+dWBOs+D+z6dmFkZlzmLC9lXx0TJrO29qmr2
         gKrQ==
X-Forwarded-Encrypted: i=1; AJvYcCUjit43TDOfG3Qo0XZZQ1wHDkkwxCRbnEzgB9gNdNLPZV2Hxq7wHMF2hVv1XfRDz3bE7CI7T5sLUiApYp0NcLhA/MyECenDotW0CYrRdas=
X-Gm-Message-State: AOJu0YyJCPBQdhjncwXLFJ/V8byKaGUMKhQ8osMYxujIFufKz3GtEm0h
	WqqEz0fC5yZNES51pCufOjRFZGkmxZqo3YbBkfGsKILRiosrGj3U1YCJd6l44A==
X-Google-Smtp-Source: AGHT+IEXG1YiRrFMeEHM2AuCVxLrS56fha12iMkgtMJ5sL9AKytR4o4dzm0txV/ErspQJAtNQCV/fA==
X-Received: by 2002:a17:906:ccd2:b0:a59:a7b7:2b8f with SMTP id a640c23a62f3a-a62280557d1mr358022466b.9.1716475402558;
        Thu, 23 May 2024 07:43:22 -0700 (PDT)
Message-ID: <4424d81a-71d6-4c9b-91b4-19b6f3ca4252@suse.com>
Date: Thu, 23 May 2024 16:43:21 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 06/15] x86/p2m: guard altp2m code with
 CONFIG_ALTP2M option
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <7a6980b1c67dedb306985f73afb23db359771e8f.1715761386.git.Sergiy_Kibrik@epam.com>
 <29604726-8768-4b33-a8ef-eba6ad112d44@suse.com>
 <71559cd3-c558-4f91-9c6c-227f502645d7@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <71559cd3-c558-4f91-9c6c-227f502645d7@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 12:44, Sergiy Kibrik wrote:
> 16.05.24 14:01, Jan Beulich:
>> On 15.05.2024 11:10, Sergiy Kibrik wrote:
>>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>>> @@ -670,7 +670,7 @@ static inline bool hvm_hap_supported(void)
>>>   /* returns true if hardware supports alternate p2m's */
>>>   static inline bool hvm_altp2m_supported(void)
>>>   {
>>> -    return hvm_funcs.caps.altp2m;
>>> +    return IS_ENABLED(CONFIG_ALTP2M) && hvm_funcs.caps.altp2m;
>>
>> Which in turn raises the question whether the altp2m struct field shouldn't
>> become conditional upon CONFIG_ALTP2M too (or rather: instead, as the change
>> here then would need to be done differently). Yet maybe that would entail
>> further changes elsewhere, so may well better be left for later.
> 
>   but hvm_funcs.caps.altp2m is only a capability bit -- is it worth to 
> become conditional?

Well, the comment was more based on the overall principle than the actual
space savings that might result. Plus as said - likely that would not work
anyway without further changes elsewhere. So perhaps okay to leave as you
have it.

>>> --- a/xen/arch/x86/mm/Makefile
>>> +++ b/xen/arch/x86/mm/Makefile
>>> @@ -1,7 +1,7 @@
>>>   obj-y += shadow/
>>>   obj-$(CONFIG_HVM) += hap/
>>>   
>>> -obj-$(CONFIG_HVM) += altp2m.o
>>> +obj-$(CONFIG_ALTP2M) += altp2m.o
>>
>> This change I think wants to move to patch 5.
>>
> 
> If this moves to patch 5 then HVM=y && ALTP2M=n configuration 
> combination will break the build in between patch 5 and 6, so I've 
> decided to put it together with fixes of these build failures in patch 6.

Hmm, yes, I think I see what you mean.

> Maybe I can merge patch 5 & 6 together then ?

Perhaps more consistent that way, yes.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:50:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:50:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728610.1133594 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9mE-0006O2-RH; Thu, 23 May 2024 14:50:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728610.1133594; Thu, 23 May 2024 14:50:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9mE-0006Nv-Nf; Thu, 23 May 2024 14:50:42 +0000
Received: by outflank-mailman (input) for mailman id 728610;
 Thu, 23 May 2024 14:50:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9mE-0006Np-5k
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:50:42 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d30ca65a-1913-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:50:40 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a5a7d28555bso1198749466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:50:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a179c7d2bsm1926534366b.120.2024.05.23.07.50.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:50:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d30ca65a-1913-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716475840; x=1717080640; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eLOdg9fpKsuZESivE/qScPwjJzL8tE6jJFJ9lWoyKK0=;
        b=UhJ+7KDguvTroUbpoZ9NT1Nl+4ov2q1/yCxjsnQEDn1CsSDYw9lN4Nur06UFTC078+
         /W39PsucO0MLjv10BSzJaIGVPtErRC3joRlD8IDJp6pmfCp79J+R/V3wNWeSfEAFUprJ
         2PO+EH+SPY/cDkUD6b9lwuFJhdBex5rov3cOlPcjBFViGjv8SBKY7hbILMJNNvLJkKn7
         mzOPf8is5LKkk4LTsYOJtZtW5fpNiyyA0laPJTBlib577DolCG7yfcMOvIbws2xTfxoV
         kbreDkAgQQhjxKYBe/VU25bKtBhN9k4a2j25VA7gBAHzBQ28TbFrfDx2RcOcxe7P6tBX
         RA/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475840; x=1717080640;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eLOdg9fpKsuZESivE/qScPwjJzL8tE6jJFJ9lWoyKK0=;
        b=g5E1u11U7qzAbknFso6Ao3IK4Ha3ui7TE1QWYFdzmZarGpwYQpGeEZU8f18pgIVhtu
         PRm88wqV8beACyhXgS59xoGusHj2nkLkt0zgB7mWAyR9ON7d83hF0XRFshVlVUK9feG+
         o3+bnJlqi6+PCByPbtLFj67vaHsDVM9uRFcUwXSJ2NZVhPaZSqmsDOGX0Sn9dugLK+Md
         c9eMDQ6ZhS1Yc5KbVnb7nFwmpTRx5CNsQr7fPcLBqg/Axw4Fcp0oE0ei8OFrFHNnYW87
         s99jrc3MApiTmuW+/gWamsMFBfpqHRRp2N8ZbEoLG/frJ7ZfiOCLHfysjvOyfrr9BlrI
         4Eig==
X-Forwarded-Encrypted: i=1; AJvYcCXiSXRcxN9xLdDAtQXn5j5qNsbZJrKQjvtiGnSLwjiO99w2VGXc0daFSC1ZeH4VR4Ge5zDdrg+FD6HeVV6XyNr1vYOGNygndc4fUMchAJE=
X-Gm-Message-State: AOJu0YwhmEFgIvlQEK74r7GSsp+8mr1hgOAPO/aID+boXCB8kxU7fCUg
	sNn/t7Vc8P0dJkYX6SOhD6aiRym453Vy5U+uF35T5vyHCorzx5NRhF5MvIeQoA==
X-Google-Smtp-Source: AGHT+IEahX/ato0feV7ZZ8OUr0aAe04RFXcdz7e5I3Fb3wxHdnvGWQ7QIoBQgnrY+SoCPUX7RN1taw==
X-Received: by 2002:a17:906:7ad8:b0:a59:a18e:3fd9 with SMTP id a640c23a62f3a-a622807b0e2mr344377566b.5.1716475840607;
        Thu, 23 May 2024 07:50:40 -0700 (PDT)
Message-ID: <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
Date: Thu, 23 May 2024 16:50:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
 <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
 <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 15:07, Sergiy Kibrik wrote:
> 16.05.24 14:12, Jan Beulich:
>> On 15.05.2024 11:12, Sergiy Kibrik wrote:
>>> --- a/xen/arch/x86/include/asm/cpufeature.h
>>> +++ b/xen/arch/x86/include/asm/cpufeature.h
>>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>   #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>>>   #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>>>   #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
>>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
>>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
>>> +                                  boot_cpu_has(X86_FEATURE_VMX))
>>>   #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>>>   #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>>>   #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
>>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>   
>>>   /* CPUID level 0x80000001.ecx */
>>>   #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
>>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
>>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
>>> +                                  boot_cpu_has(X86_FEATURE_SVM))
>>>   #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>>>   #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>>>   #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
>>
>> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
>> and as a result one-off indentation on the wrapped lines) I'm not really
>> certain we can do this. The description goes into detail why we would want
>> this, but it doesn't cover at all why it is safe for all present (and
>> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
>> just to derive further knowledge from that, without them being directly
>> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
>> for example. While it looks to be okay there, it may give you an idea of
>> what I mean.
>>
>> Things might become better separated if instead for such checks we used
>> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
>> that's still pretty far out, I'm afraid.
> 
> I've followed a suggestion you made for patch in previous series:
> 
> https://lore.kernel.org/xen-devel/8fbd604e-5e5d-410c-880f-2ad257bbe08a@suse.com/

See the "If not, ..." that I had put there. Doing the change just mechanically
isn't enough, you also need to make clear (in the description) that you
verified it's safe to have this way.

> yet if this approach can potentially be unsafe (I'm not completely sure 
> it's safe), should we instead fallback to the way it was done in v1 
> series? I.e. guard calls to vmx/svm-specific calls where needed, like in 
> these 3 patches:
> 
> 1) 
> https://lore.kernel.org/xen-devel/20240416063328.3469386-1-Sergiy_Kibrik@epam.com/
> 
> 2) 
> https://lore.kernel.org/xen-devel/20240416063740.3469592-1-Sergiy_Kibrik@epam.com/
> 
> 3) 
> https://lore.kernel.org/xen-devel/20240416063947.3469718-1-Sergiy_Kibrik@epam.com/

I don't like this sprinkling around of IS_ENABLED() very much. Maybe we want
to have two new helpers (say using_svm() and using_vmx()), to be used in place
of most but possibly not all cpu_has_{svm,vmx}? Doing such a transformation
would then kind of implicitly answer the safety question above, as at every
use site you'd need to judge whether the replacement is correct. If it's
correct everywhere, the construct(s) as proposed in this version could then be
considered to be used in this very shape (instead of introducing the two new
helpers). But of course the transition could also be done gradually then,
touching only those uses that previously you touched in 1), 2), and 3).

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:50:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:50:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728611.1133604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9mS-0006im-23; Thu, 23 May 2024 14:50:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728611.1133604; Thu, 23 May 2024 14:50:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9mR-0006id-VQ; Thu, 23 May 2024 14:50:55 +0000
Received: by outflank-mailman (input) for mailman id 728611;
 Thu, 23 May 2024 14:50:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA9mR-0006Np-54
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:50:55 +0000
Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com
 [2607:f8b0:4864:20::f2e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dac110e2-1913-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:50:54 +0200 (CEST)
Received: by mail-qv1-xf2e.google.com with SMTP id
 6a1803df08f44-6969388c36fso26644216d6.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:50:54 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6a15f1ff09fsm143585096d6.133.2024.05.23.07.50.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:50:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dac110e2-1913-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716475853; x=1717080653; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=g1lWNC7dseGX/ioIFVD4AG6Y8UpRFscXqkg6G8s0OXY=;
        b=Qs4b7WygM3eG2teiXtyZUvIRWW1s/AP96Y/maPklU6BgMLjMqA1gWbm11RVm83gS9J
         XRtSq86FBL0bDStpGh1xzwiERKPTnYRWxdUEi3vkcP5lWb5PPTSQT+hzIX36U2GoGhvf
         ePhdOBetryp6DerRJvXekBkrW5+Tq9tI5lNcI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475853; x=1717080653;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=g1lWNC7dseGX/ioIFVD4AG6Y8UpRFscXqkg6G8s0OXY=;
        b=LP6+0Cvxbk7Z+Tsm83mryAUeTzM4PpIDtm3w167VquRAdPtFLIXWjZCSdMoiE8MZOq
         I64VtIEzk5MYHnvA7Ik1hCZ3ck+46wVik3RFYOOAHlwYf0CnLuh0nNEXLha6D3ei6PeT
         78s4aRWpEG6IrDEVB/ENG2DD3qsA3ym9/6e3IRgxrE1MBLsxdOW+etRbcJSSoW18avoD
         ZdMP8w8chZzuljhCID1i5i44c7Kkz3dwCd262dq4X1bgSR3Tda0qb1ipxwaSWH6pi3E2
         4N7n9RAr9kaq+U06o7UNTPkPPbbdHsXiWpwV7xSY9zVLNaH1MHMhuH772tIhyqnYVoJi
         bDig==
X-Gm-Message-State: AOJu0Yw6u/olOtSCK22s2t7PJ1cZILDDPQL3fG/1Ren9DWnntJgeRWc5
	ytGeV099kph+B/4ZcIZ1ud2xrmIz4piIy+hIEJAXv4CzsTNsbsyPviCo5qRzJhw=
X-Google-Smtp-Source: AGHT+IHCcK8430PrNR+vfhzud2TGAqFcXFlSdYkNkxhYVmnB1lOp2TO89gfGD7Zr356cR1phIXyJbw==
X-Received: by 2002:a05:6214:398e:b0:6ab:7be5:c849 with SMTP id 6a1803df08f44-6ab7f331304mr49407946d6.4.1716475853344;
        Thu, 23 May 2024 07:50:53 -0700 (PDT)
Date: Thu, 23 May 2024 16:50:51 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration
 checks to the check hook
Message-ID: <Zk9Xy7W30WWZlb6-@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:22PM +0100, Alejandro Vallejo wrote:
> While at it, add a check for the reserved field in the hidden save area.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * New patch. Addresses the missing check for rsvd_zero in v1.

Oh, it would be better if this was done at the time when rsvd_zero is
introduced.  I think this should be moved ahead of the series, so that
the patch that introduces rsvd_zero can add the check in
lapic_check_hidden().

> ---
>  xen/arch/x86/hvm/vlapic.c | 41 ++++++++++++++++++++++++++++-----------
>  1 file changed, 30 insertions(+), 11 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> index 8a244100009c..2f06bff1b2cc 100644
> --- a/xen/arch/x86/hvm/vlapic.c
> +++ b/xen/arch/x86/hvm/vlapic.c
> @@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
>  }
>  
> -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> +static int cf_check lapic_check_hidden(const struct domain *d,
> +                                       hvm_domain_context_t *h)
>  {
>      unsigned int vcpuid = hvm_load_instance(h);
> -    struct vcpu *v;
> -    struct vlapic *s;
> +    struct hvm_hw_lapic s;
>  
>      if ( !has_vlapic(d) )
>          return -ENODEV;
>  
>      /* Which vlapic to load? */
> -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
> +    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )
>      {
>          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
>                  d->domain_id, vcpuid);
>          return -EINVAL;
>      }
> -    s = vcpu_vlapic(v);
>  
> -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )

Can't you use hvm_get_entry() to perform the sanity checks:

const struct hvm_hw_lapic *s = hvm_get_entry(LAPIC, h);

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:53:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:53:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728618.1133614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9oU-00082T-E4; Thu, 23 May 2024 14:53:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728618.1133614; Thu, 23 May 2024 14:53:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9oU-00082M-Am; Thu, 23 May 2024 14:53:02 +0000
Received: by outflank-mailman (input) for mailman id 728618;
 Thu, 23 May 2024 14:53:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA9oT-00082E-4n
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:53:01 +0000
Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com
 [2607:f8b0:4864:20::f2f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2571a117-1914-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 16:52:59 +0200 (CEST)
Received: by mail-qv1-xf2f.google.com with SMTP id
 6a1803df08f44-69cb4a046dfso26974716d6.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:52:59 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ab836aa398sm16282746d6.72.2024.05.23.07.52.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 07:52:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2571a117-1914-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716475979; x=1717080779; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=W/ZWUdIBibQlFSbb0n2SW9+SVP7VDa6YLoknaE59ejo=;
        b=jAHFywa0e1otz0xPkXz8E+xbJzYcZ+ylQIsYalqGVxL9S0SylpkTjyjI0Als6EfVmy
         pTsWf94GA9Vw7oiwSY6Nfdlkr4oQN2LZ4dEommjWpRSISCtsIVz9s5zODPz7vn9ELlpB
         aCQVDiTm4at/bN3ujPg8lb+nMvyyUZZbxMdKc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716475979; x=1717080779;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=W/ZWUdIBibQlFSbb0n2SW9+SVP7VDa6YLoknaE59ejo=;
        b=XOdIzBU7NktJ1Tvai21iSZLnfjKjybfRvRKhHJ01sOOI/TJ/a0IGRAOsIS0+/G3aaA
         NkWDIVBu9E9f9I9oPo+mpCLrBtpbn6Q/7/YmXapusb3sDzWOXqLs8P1ifdQkSSlfnL3g
         JBrxFByYIkG74XZ+pWlyW4sR7N15XPwnLyRLzMhTYeUzZSAYgtQo3y9yT267OAz3MEFG
         QixOwWRy06JBH33nnNZEbHrpXoexFrlR1Iy86prNSBLw51W6E9QSL3N1Ztp0cA2i/8tQ
         0IQtP5W2s2nKPA7oxBm2N44tdERQylDjyFJanLla37K+Z6KQ6I90fVYLTrJv2bRjz99x
         7tCA==
X-Gm-Message-State: AOJu0YxZl2EW3ClU0wJEBSv+749g/D56QTsXAyJzqYEEzFbGouogAhCZ
	0Ax1HNq6g7yIAnmhir5Y6LW+5FAmE293+TC3LQYo31MNX8LkRKbJsV50LfNUysQ=
X-Google-Smtp-Source: AGHT+IFnGYLkuY50Jpkq1CiDPeTDaFNCSYlZ/BfPkjj7LcxEVt4WjIj4qRoZm9UMhT7HIYsNqAzogQ==
X-Received: by 2002:a05:6214:5890:b0:6ab:7b2c:a7df with SMTP id 6a1803df08f44-6ab7f36d8eemr55687086d6.38.1716475978620;
        Thu, 23 May 2024 07:52:58 -0700 (PDT)
Date: Thu, 23 May 2024 16:52:56 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Alejandro Vallejo <alejandro.vallejo@cloud.com>
Subject: Re: [PATCH v2 2/8] xen/x86: Simplify header dependencies in x86/hvm
Message-ID: <Zk9YSBEGFTYFc3BC@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <00ce7005d1d6db5c1ffc2d5023d34d4bd34ff841.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9Upg9y03IYZEXd@macbook>
 <8dd92e22-2a7e-414b-98b2-c10ae449f825@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <8dd92e22-2a7e-414b-98b2-c10ae449f825@suse.com>

On Thu, May 23, 2024 at 04:40:06PM +0200, Jan Beulich wrote:
> On 23.05.2024 16:37, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:21PM +0100, Alejandro Vallejo wrote:
> >> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> >> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> >> @@ -798,6 +798,12 @@ static inline void hvm_update_vlapic_mode(struct vcpu *v)
> >>          alternative_vcall(hvm_funcs.update_vlapic_mode, v);
> >>  }
> >>  
> >> +static inline void hvm_vlapic_sync_pir_to_irr(struct vcpu *v)
> >> +{
> >> +    if ( hvm_funcs.sync_pir_to_irr )
> >> +        alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
> > 
> > Nit: for consistency the wrappers are usually named hvm_<hook_name>,
> > so in this case it would be hvm_sync_pir_to_irr(), or the hvm_funcs
> > field should be renamed to vlapic_sync_pir_to_irr.
> 
> Funny you should mention that: See my earlier comment as well as what
> was committed.

Oh, sorry, didn't realize you already replied, adjusted and committed.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 14:53:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 14:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728622.1133626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9pC-00007H-MX; Thu, 23 May 2024 14:53:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728622.1133626; Thu, 23 May 2024 14:53:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9pC-00007A-K4; Thu, 23 May 2024 14:53:46 +0000
Received: by outflank-mailman (input) for mailman id 728622;
 Thu, 23 May 2024 14:53:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sA9pB-00006r-9f
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 14:53:45 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3fae73fd-1914-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 16:53:43 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-56e1bbdb362so9613849a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 07:53:43 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b01a2esm1943060566b.185.2024.05.23.07.53.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 07:53:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3fae73fd-1914-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716476022; x=1717080822; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cpsNufTSxl5uFSh8vwPZqZgoWvtPcUnSgHFo5kmo+6M=;
        b=bIOK8SegaknRCRTlw+bkA/lyaNvqQBMS1cNeAbS9/1rwBWmBN99UOaTbV+On6J5trs
         CPZHJ/CGoXMXGAFYP0UMO1/xM0+SJQdzCBj9PhPNAPMjWjY3QxN72WW+6CNGyxC/G8zn
         flzxd8AOY77vZIXHx7hEYyFo/R9UuETeHscsSv3F7evmCkJ/mLnJQNqASHGldJM8CIzJ
         aAJEpeLF75jIxepaywiA7OQNwdqkvxbXgKwpr9q2cOiumSB4nmkMcginqRixFFXFLquk
         RjVL53yR+3aE4on1I/Ta26UGNNitY07X1u2tK6auG41okvBnXo1LtO8p5ekv74RuJXw0
         +jcQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716476022; x=1717080822;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cpsNufTSxl5uFSh8vwPZqZgoWvtPcUnSgHFo5kmo+6M=;
        b=QjEna6ZODNrghVxA3f1SIfxTOKrSkHYCzQzcuNtHs6DeCv+RTqVFbRHw43QO5VqE7U
         w4d1aH/zjJeb03FnNWaCpXihWUOvs/MkOFQwmn19Cfk17wq+W56mC6+l6g9UF1Zj07hk
         yv4rgxQIgAJwR4KWgvYHmOAJyeYc09ToJ/cNTUjWLLNEeqMRhQiaerUQ6CYnK6rKRiJS
         mlsSD6szVPjWJwjAaW8Jn/yRjK4y3Sj7FRuTDpd77709YQAaF8FEqPIq6wP/leOIZC+t
         xR3Lv+f0hoR566or7TGVlZcR6meKr14Q2IfJmX4jSKFwRmWJ23l9WCRRL4bdfLFeYSMK
         2mww==
X-Gm-Message-State: AOJu0YyI8dm+L5BDxY+Hk85M4TfqrdUVpwAFonU7R0N0+S15JQ0qh1nq
	X4ySyVaZY02DVtSXsM0gPs2EctYN1J0MqtGXAzxhOw8mn0zjWKbv3Pj6Ar3+EtGxBsz+3IoZF3s
	=
X-Google-Smtp-Source: AGHT+IH9iXYtnBw1AZH2BewLNNWfoNxFcWseBv+hERq2zEhcfoA6mdP29P96Wslv6pLobJoT64gm0w==
X-Received: by 2002:a17:906:6b93:b0:a53:ed1b:f3ba with SMTP id a640c23a62f3a-a62280d68admr471088566b.28.1716476022573;
        Thu, 23 May 2024 07:53:42 -0700 (PDT)
Message-ID: <6ad6074e-26ce-43e0-918f-63f28723397b@suse.com>
Date: Thu, 23 May 2024 16:53:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [xen-4.17-testing test] 186087: regressions - FAIL
Content-Language: en-US
To: osstest service owner <osstest-admin@xenproject.org>
References: <osstest-186087-mainreport@xen.org>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <osstest-186087-mainreport@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 16:40, osstest service owner wrote:
> flight 186087 xen-4.17-testing real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/186087/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  build-amd64                   6 xen-build                fail REGR. vs. 185864
>  build-amd64-xsm               6 xen-build                fail REGR. vs. 185864
>  build-i386-xsm                6 xen-build                fail REGR. vs. 185864
>  build-i386                    6 xen-build                fail REGR. vs. 185864
>  build-amd64-prev              6 xen-build                fail REGR. vs. 185864
>  build-i386-prev               6 xen-build                fail REGR. vs. 185864

These look to be recurring, yet at the same time these look to be infrastructure
issues. This not happening for the first time I'm not sure we can simply wait
and hope for the problem to clear itself.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:04:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:04:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728632.1133637 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9zE-00023y-Os; Thu, 23 May 2024 15:04:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728632.1133637; Thu, 23 May 2024 15:04:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9zE-00023r-M5; Thu, 23 May 2024 15:04:08 +0000
Received: by outflank-mailman (input) for mailman id 728632;
 Thu, 23 May 2024 15:04:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sA9zD-00023l-Am
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:04:07 +0000
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [2607:f8b0:4864:20::72e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b2cc2fce-1915-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 17:04:06 +0200 (CEST)
Received: by mail-qk1-x72e.google.com with SMTP id
 af79cd13be357-792ce7a1fa8so532993185a.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:04:06 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-7930f124416sm694158485a.119.2024.05.23.08.04.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 08:04:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2cc2fce-1915-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716476645; x=1717081445; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=x6bypk45n2WDAwnPN2WsudmjX9+lKvI0xbg3Obu88C4=;
        b=hyeEF/KkepvU+gYPXJMQGAjaYOa0PMH+aiKuG90Lo0x8dr3M6jrigW78fCMwDZReng
         QhoH3zyz6AmFZzdAdAtEQZ4gd18qx8IwlmsAS0EqmZad1NUotoQ0blRIyffI54OSQvAq
         T3bUGlRccOuYSvI7MF17SFQPE6FpVpJvJGEL8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716476645; x=1717081445;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=x6bypk45n2WDAwnPN2WsudmjX9+lKvI0xbg3Obu88C4=;
        b=Iit4XyFj05mU6JCYv1MgGaUnjBFaJJvTCO8qsWKvADwcn1adQOcG1mklo+FzK/5CTZ
         nMmGxPkIY09xyt2dFdEttE0Bn8fekclyCRhik8hBAJmRr0FRk0/Mx31do7nCKkzjuPTI
         sFqvdnIBqlGqJSgVnARnlV0u8R5g3gwFmUzOUO1tezDfupTQNP74PqGWhMg58Pyb+dju
         xdYUR2vUCzRjfm0XqoY46vaQEcNMCo8/0kWq+JhEz9JWDXWT3ny19hOQvbz1juFTjX7n
         4dp3BILBSFrElaQpL7Ady34a8ZMDz/cZRF4GUktqjPbO+gYrk23756suYdSaY+8dII/1
         mSbQ==
X-Forwarded-Encrypted: i=1; AJvYcCWLbV9QtNVTO3OLOZf4WN8ENOaBEnlF9Zi+MKeCBhGczgF8QGS8Yry0/YTKykyQJmaq8cLR3RDMDGm0KlxDiQMdU/Q2EgfCCX5r/LJ6H78=
X-Gm-Message-State: AOJu0YyBv4ceHXO+2BJyAQR/lw4McgXU/SvS5tRKPbGt9ewXvvHevgRU
	PdRg0BRrnUmsBbMAGmXmmYOm/nG7f5DyK7w1LHR/wp6OQli7K/2aTpwbH17eveA=
X-Google-Smtp-Source: AGHT+IGnctaEmrmxDPdmkJhj3h+m4MzjeFD846b/qWjjB+1rTTcO2V9Kroy8bTZMzWW77gqTZoSzeA==
X-Received: by 2002:a05:620a:118b:b0:792:7d2f:156f with SMTP id af79cd13be357-794994e55f4mr491068685a.77.1716476644982;
        Thu, 23 May 2024 08:04:04 -0700 (PDT)
Message-ID: <1613ceeb-916f-43d1-970e-913877f46fac@citrix.com>
Date: Thu, 23 May 2024 16:04:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/hvm/trace: Use a different trace type for AMD
 processors
To: George Dunlap <george.dunlap@cloud.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@cloud.com>,
 Anthony Perard <anthony.perard@cloud.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Olaf Hering <olaf@aepfle.de>
References: <20240523141020.2256177-1-george.dunlap@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240523141020.2256177-1-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23/05/2024 3:10 pm, George Dunlap wrote:
> A long-standing usability sub-optimality with xenalyze is the
> necessity to specify `--svm-mode` when analyzing AMD processors.  This
> fundamentally comes about because the same trace event ID is used for
> both VMX and SVM, but the contents of the trace must be interpreted
> differently.
>
> Instead, allocate separate trace events for VMX and SVM vmexits in
> Xen; this will allow all readers to properly interpret the meaning of
> the vmexit reason.
>
> In xenalyze, first remove the redundant call to init_hvm_data();
> there's no way to get to hvm_vmexit_process() without it being already
> initialized by the set_vcpu_type call in hvm_process().
>
> Replace this with set_hvm_exit_reson_data(), and move setting of
> hvm->exit_reason_* into that function.
>
> Modify hvm_process and hvm_vmexit_process to handle all four potential
> values appropriately.
>
> If SVM entries are encountered, set opt.svm_mode so that other
> SVM-specific functionality is triggered.
>
> Remove the `--svm-mode` command-line option, since it's now redundant.
>
> Signed-off-by: George Dunlap <george.dunlap@cloud.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:04:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:04:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728637.1133646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9zn-0002WE-0A; Thu, 23 May 2024 15:04:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728637.1133646; Thu, 23 May 2024 15:04:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sA9zm-0002W7-Tt; Thu, 23 May 2024 15:04:42 +0000
Received: by outflank-mailman (input) for mailman id 728637;
 Thu, 23 May 2024 15:04:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sA9zl-0002Vx-KT
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:04:41 +0000
Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com
 [2607:f8b0:4864:20::830])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c6bb7b32-1915-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 17:04:39 +0200 (CEST)
Received: by mail-qt1-x830.google.com with SMTP id
 d75a77b69052e-43de92e228aso38280301cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:04:39 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43dfd2dbd68sm174303961cf.12.2024.05.23.08.04.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 08:04:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6bb7b32-1915-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716476678; x=1717081478; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=4RC/7hDBTJxRMcKjmn3dodba7GsHJMDnhBM2/N+I+0g=;
        b=Dmewu2NtL5hlwT96zhjNNXruAmRjB7ESqxjNiffNyPR/vpJ7oSeH0/AtNSP/nY32oc
         /IQO5uqT+rNbC7LbIXUOMft3Arm51Zqq6MRsOi+k3g5jTkTI5LqEpnq0zaucEYiLUR/t
         26LOPdOO00WkCckWJgbtQ1wAqpsFxFk8arCRI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716476678; x=1717081478;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4RC/7hDBTJxRMcKjmn3dodba7GsHJMDnhBM2/N+I+0g=;
        b=LAhCUWDh5XMQAZ24FVwZAOFPGbPCaSVYCIGNOE3C4KfPcFMYQMY1VxoCmi9ePYOG+i
         BdgcWRJydDQBTqaWNtNfT2iONnv5RLtClfOFa5jHhrausgLFg96g4Cz8+hZG2lZiNij+
         S/rEOa0Ga5janlN/wZrc2u3TbGytyZJL1ROf7WgKmtu91P7KpZ+bWaEA13GZkm3K7qMt
         YWD/M8tLLJ81WFwA44auA55+LdvqN42ZK2k0aH1ik5WleVrAg0pIs0KSH27qo22Gk74O
         n2ySLDrTTLjOOn9im2Lx1ITpLKi3p+7KGEV2/B5i1xDXIfCBt77b6tvPDcPiQ8OqAXev
         XD0w==
X-Gm-Message-State: AOJu0YxUogImfG4h6PrWLpX6u8MN7U+f6/Po2cvW9Ybl0/duyhlPXVlk
	5zY0kgaob1VJn0TibVAJuJ/oYAXQyeaHdTtx3SFqaEtZk9Z9jTVAQEgcJwY5yj8=
X-Google-Smtp-Source: AGHT+IGSPFAw39UwAbeQZYJZzHIAevWVDJcWio1ULwXv/mQgrM16tXd3SFQo2Nr+yzNuUbefo79Yhg==
X-Received: by 2002:a05:622a:1114:b0:43d:fa8f:53c1 with SMTP id d75a77b69052e-43f9e194e45mr49003441cf.64.1716476678465;
        Thu, 23 May 2024 08:04:38 -0700 (PDT)
Date: Thu, 23 May 2024 17:04:36 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>,
	Alejandro Vallejo <alejandro.vallejo@cloud.com>
Subject: Re: [PATCH 4.5/8] tools/hvmloader: Further simplify SMP setup
Message-ID: <Zk9bBFmVkkWcLz3E@macbook>
References: <bd23a05ea25b2f431bb0655ca6402073f9cf49b8.1715102098.git.alejandro.vallejo@cloud.com>
 <20240509175057.1921538-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240509175057.1921538-1-andrew.cooper3@citrix.com>

On Thu, May 09, 2024 at 06:50:57PM +0100, Andrew Cooper wrote:
> Now that we're using hypercalls to start APs, we can replace the 'ap_cpuid'
> global with a regular function parameter.  This requires telling the compiler
> that we'd like the parameter in a register rather than on the stack.
> 
> While adjusting, rename to cpu_setup().  It's always been used on the BSP,
> making the name ap_start() specifically misleading.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:11:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:11:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728645.1133659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAA5o-0004dW-Ks; Thu, 23 May 2024 15:10:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728645.1133659; Thu, 23 May 2024 15:10:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAA5o-0004dP-IO; Thu, 23 May 2024 15:10:56 +0000
Received: by outflank-mailman (input) for mailman id 728645;
 Thu, 23 May 2024 15:10:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAA5n-0004dJ-6J
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:10:55 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7e9464ad-1916-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 17:09:47 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a59cf8140d0so935804166b.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:09:47 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a1787c63bsm1949324766b.51.2024.05.23.08.09.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 08:09:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e9464ad-1916-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716476987; x=1717081787; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=buMAg0LS8akaLYVn1mf1luZlef5FqtF+8iT745eiTPQ=;
        b=M4Zjy8lA++k1LH1tBehoWiNrSnil4lkDgJKQU5rzB6HtSx3zjRt8ev/RejGfXRzNHd
         U25PHBDigg7CWp44YPTJvjaDvWbUPJ5VOv/yGnjxC0+xatoTNWeY7cO5OXBGYrWfJXz/
         E9TqiGjz20V04z/W/oKVwdnrHlyTsWOH7TXEN4IIdmw1fz369RLeMduJ6gvAJUybS6Vk
         ywnpmHleg0NGxlPMklRNFJ81grAsJ2okJirihyNP0KcvgT3fvgPx1WkyzNDlE/lGV3jV
         fWS3r/L3spfbInoWnU66zHJYNhJMC0arFX9luTJBlAEwVR94fi8TszSB35XKvbAzUm95
         dnBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716476987; x=1717081787;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=buMAg0LS8akaLYVn1mf1luZlef5FqtF+8iT745eiTPQ=;
        b=sOUYi4XJHDLM4sOVpPHrU8NNadprsJ9rjcq3aXu0+mDVhGK02nWs1QsaqqDcBfIRSL
         HPBqQlQcu5AUuVv7Qn23+CkoYCNGv5yCVm/KmbqW50PU+r4pSqpRlKM93LjcbEQtArwM
         qtbd257fM27AF0HFpuaUQ6C7q2rqPpLyV9qSyX8H/pmyzwE5doxwvscOjUVZ3ejW10bv
         4yggBIxylV3bJv0MtU963Y8p3+IUvfCpVdxmKR7BRy1Es3UbKJG8CekiElHTGrmGYebH
         fgeUuP1T1G5JnpBo+N5t3r2o1BDjShpzInHA0w0k83LgpuWVx29YM7asAjVLqB/51ggF
         mmbw==
X-Forwarded-Encrypted: i=1; AJvYcCWcjLVC3wTIEvOihKzX4cc4yNZ19fR9KbpcdSj96RVcCYY8UPXD0ws183rQln+bTrV2E8CVKicdPA6fxgILR1AzlSnXHfK5pllaGf05gkY=
X-Gm-Message-State: AOJu0Yzbtfcpx5daRCWIDVUPxiEzDJCPynBz60Za6aPP7khisJdR0vyl
	WEP97I6hs+mO3fWxvvwFZIGyqJUFCNdOXj+GYr4AMDCqLvhS8MccskIoQl3bsA==
X-Google-Smtp-Source: AGHT+IFJHdPcenIMGjRAmc4re1PrzhAy95N3NTcdJzLheVYPZq5oaJ6ZqnQn+4w1/sliI+3BAX498Q==
X-Received: by 2002:a17:907:26cb:b0:a59:f30e:fecd with SMTP id a640c23a62f3a-a622807a844mr454892266b.2.1716476986989;
        Thu, 23 May 2024 08:09:46 -0700 (PDT)
Message-ID: <1def14d9-f98e-432e-b5bb-db6cd64676a2@suse.com>
Date: Thu, 23 May 2024 17:09:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/7] x86/xstate: Fix initialisation of XSS cache
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-2-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> The clobbering of this_cpu(xcr0) and this_cpu(xss) to architecturally invalid
> values is to force the subsequent set_xcr0() and set_msr_xss() to reload the
> hardware register.
> 
> While XCR0 is reloaded in xstate_init(), MSR_XSS isn't.  This causes
> get_msr_xss() to return the invalid value, and logic of the form:
> 
>   old = get_msr_xss();
>   set_msr_xss(new);
>   ...
>   set_msr_xss(old);
> 
> to try and restore the architecturally invalid value.
> 
> The architecturally invalid value must be purged from the cache, meaning the
> hardware register must be written at least once.  This in turn highlights that
> the invalid value must only be used in the case that the hardware register is
> available.
> 
> Fixes: f7f4a523927f ("x86/xstate: reset cached register values on resume")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

However, I view it as pretty unfair that now I will need to re-base
https://lists.xen.org/archives/html/xen-devel/2021-04/msg01336.html
over ...

> --- a/xen/arch/x86/xstate.c
> +++ b/xen/arch/x86/xstate.c
> @@ -641,13 +641,6 @@ void xstate_init(struct cpuinfo_x86 *c)
>          return;
>      }
>  
> -    /*
> -     * Zap the cached values to make set_xcr0() and set_msr_xss() really
> -     * write it.
> -     */
> -    this_cpu(xcr0) = 0;
> -    this_cpu(xss) = ~0;
> -
>      cpuid_count(XSTATE_CPUID, 0, &eax, &ebx, &ecx, &edx);
>      feature_mask = (((u64)edx << 32) | eax) & XCNTXT_MASK;
>      BUG_ON(!valid_xcr0(feature_mask));
> @@ -657,8 +650,19 @@ void xstate_init(struct cpuinfo_x86 *c)
>       * Set CR4_OSXSAVE and run "cpuid" to get xsave_cntxt_size.
>       */
>      set_in_cr4(X86_CR4_OSXSAVE);
> +
> +    /*
> +     * Zap the cached values to make set_xcr0() and set_msr_xss() really write
> +     * the hardware register.
> +     */
> +    this_cpu(xcr0) = 0;
>      if ( !set_xcr0(feature_mask) )
>          BUG();
> +    if ( cpu_has_xsaves )
> +    {
> +        this_cpu(xss) = ~0;
> +        set_msr_xss(0);
> +    }

... this change, kind of breaking again your nice arrangement. Seeing for how
long that change has been pending, it _really_ should have gone in ahead of
this one, with you then sorting how you'd like things to be arranged in the
combined result, rather than me re-posting and then either again not getting
any feedback for years, or you disliking what I've done. Oh well ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:34:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:34:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728656.1133676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAASc-0001Wz-CA; Thu, 23 May 2024 15:34:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728656.1133676; Thu, 23 May 2024 15:34:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAASc-0001Ws-8r; Thu, 23 May 2024 15:34:30 +0000
Received: by outflank-mailman (input) for mailman id 728656;
 Thu, 23 May 2024 15:34:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAASb-0001Wm-C8
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:34:29 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f03e2f1f-1919-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 17:34:26 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-57847c08e0cso1178183a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:34:26 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-574bcad0362sm16692090a12.20.2024.05.23.08.34.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 08:34:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f03e2f1f-1919-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716478466; x=1717083266; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/4SiQWsf5o2I0gyNSUlQoMspiZtx0wo3qkAr4NjkpFI=;
        b=NviB6wcg8YALxo3y/9o+vDl579TP3ub3DWQino4wDJq1AR7meYIsXqr2C1giW4jAcH
         xTXBwCnq6xdEQ/oqOECRdsLAWTsUdeywr9+XdPCEVqe9JYXBZIyIyuWuFHKtE/I9RvAY
         tgs5ELivwfXb6VHDU+F5ijrvtxJpsAXQq77nyVAFHdCPQf/4mRRpMq7XTAPtxcrf2EWz
         zJEmOPX+c22G8iR5zZK5qwQxVcVWpUg//shw9+lQxaLTaPgogT2hCKnnylaagneW3QrX
         F5hknuFpyzbsqhn5F7uto7sjX6mFY3iNdnbO6eB7ddpshHcPcDpWZEJEUuh9OPK/IWJt
         B3Vw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716478466; x=1717083266;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/4SiQWsf5o2I0gyNSUlQoMspiZtx0wo3qkAr4NjkpFI=;
        b=RMb4TbwaMw2CUH8lniHISY3jwtf2GeEQby8kHQSR2TNjtdV4RmNNbMUeIVSujPG487
         b+xHakEqacuzy7rMhs/xd48rs0ygx4UWO+CVQt0PhVqZtkQh4q/6QdkQ+1V6uk7TEKeN
         0MfiC/uIqEnFYifqITpAw7ZUkq4o/6DfTjtCQlsQ33GU8qH2E5urLp+clClgLPXsLjgU
         p4pcBBtxnMTL2fOglntFhdUsE1D3f4RmJFKrjoVa8aUJsQd88OAqzACtiwYsrYEDz8S3
         pX64Mi50diA1qXCWUyKHZBJjalHJdOPuhruOIv1siB6EwIjducO3QfKlWy0GNOaf+XJ5
         VDfA==
X-Forwarded-Encrypted: i=1; AJvYcCWai/lo8lawxuswVj20VVS6aTXrHr666o62W3QydokV0k0WOhMo6jekZZQ4OiycZWsB6WT7JowbY5JnNxVms0trKxm8Anzkr73WmYOgS+A=
X-Gm-Message-State: AOJu0YxqiFYXZqyDXK8yD8YtLiFaJfgtu+QMUPIYQwSu8QDqg1NsKpL2
	2sxVQPRrU9h52LhyikjgSRh9YkaLQt/h4nh+vsG9ss1vga2VMOXMIIDYViIHtg==
X-Google-Smtp-Source: AGHT+IGxbZZPcwy3Tz5tFFp/od9gLBSq2doW8l5XOPgeuKxy2gt+IWHX0DJsNAy9zqg+WqguO6uw8Q==
X-Received: by 2002:a50:d655:0:b0:572:7d77:179d with SMTP id 4fb4d7f45d1cf-57843c7a592mr1998648a12.5.1716478466214;
        Thu, 23 May 2024 08:34:26 -0700 (PDT)
Message-ID: <22e0473a-aca8-4645-a3f4-21a9d9e0ebd3@suse.com>
Date: Thu, 23 May 2024 17:34:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] x86/xstate: Cross-check dynamic XSTATE sizes at boot
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> Right now, xstate_ctxt_size() performs a cross-check of size with CPUID in for
> every call.  This is expensive, being used for domain create/migrate, as well
> as to service certain guest CPUID instructions.
> 
> Instead, arrange to check the sizes once at boot.  See the code comments for
> details.  Right now, it just checks hardware against the algorithm
> expectations.  Later patches will add further cross-checking.
> 
> Introduce the missing X86_XCR0_* and X86_XSS_* constants, and a couple of
> missing CPUID bits.  This is to maximise coverage in the sanity check, even if
> we don't expect to use/virtualise some of these features any time soon.  Leave
> HDC and HWP alone for now.  We don't have CPUID bits from them stored nicely.

Since you say "the missing", ...

> --- a/xen/arch/x86/include/asm/x86-defns.h
> +++ b/xen/arch/x86/include/asm/x86-defns.h
> @@ -77,7 +77,7 @@
>  #define X86_CR4_PKS        0x01000000 /* Protection Key Supervisor */
>  
>  /*
> - * XSTATE component flags in XCR0
> + * XSTATE component flags in XCR0 | MSR_XSS
>   */
>  #define X86_XCR0_FP_POS           0
>  #define X86_XCR0_FP               (1ULL << X86_XCR0_FP_POS)
> @@ -95,11 +95,34 @@
>  #define X86_XCR0_ZMM              (1ULL << X86_XCR0_ZMM_POS)
>  #define X86_XCR0_HI_ZMM_POS       7
>  #define X86_XCR0_HI_ZMM           (1ULL << X86_XCR0_HI_ZMM_POS)
> +#define X86_XSS_PROC_TRACE        (_AC(1, ULL) <<  8)
>  #define X86_XCR0_PKRU_POS         9
>  #define X86_XCR0_PKRU             (1ULL << X86_XCR0_PKRU_POS)
> +#define X86_XSS_PASID             (_AC(1, ULL) << 10)
> +#define X86_XSS_CET_U             (_AC(1, ULL) << 11)
> +#define X86_XSS_CET_S             (_AC(1, ULL) << 12)
> +#define X86_XSS_HDC               (_AC(1, ULL) << 13)
> +#define X86_XSS_UINTR             (_AC(1, ULL) << 14)
> +#define X86_XSS_LBR               (_AC(1, ULL) << 15)
> +#define X86_XSS_HWP               (_AC(1, ULL) << 16)
> +#define X86_XCR0_TILE_CFG         (_AC(1, ULL) << 17)
> +#define X86_XCR0_TILE_DATA        (_AC(1, ULL) << 18)

... I'm wondering if you deliberately left out APX (bit 19).

Since you're re-doing some of what I have long had in patches already,
I'd also like to ask whether the last underscores each in the two AMX
names really are useful in your opinion. While rebasing isn't going
to be difficult either way, it would be yet simpler with
X86_XCR0_TILECFG and X86_XCR0_TILEDATA, as I've had it in my patches
for over 3 years.

> --- a/xen/arch/x86/xstate.c
> +++ b/xen/arch/x86/xstate.c
> @@ -604,9 +604,156 @@ static bool valid_xcr0(uint64_t xcr0)
>      if ( !(xcr0 & X86_XCR0_BNDREGS) != !(xcr0 & X86_XCR0_BNDCSR) )
>          return false;
>  
> +    /* TILE_CFG and TILE_DATA must be the same. */
> +    if ( !(xcr0 & X86_XCR0_TILE_CFG) != !(xcr0 & X86_XCR0_TILE_DATA) )
> +        return false;
> +
>      return true;
>  }
>  
> +struct xcheck_state {
> +    uint64_t states;
> +    uint32_t uncomp_size;
> +    uint32_t comp_size;
> +};
> +
> +static void __init check_new_xstate(struct xcheck_state *s, uint64_t new)
> +{
> +    uint32_t hw_size;
> +
> +    BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES);
> +
> +    BUG_ON(s->states & new); /* States only increase. */
> +    BUG_ON(!valid_xcr0(s->states | new)); /* Xen thinks it's a good value. */
> +    BUG_ON(new & ~(X86_XCR0_STATES | X86_XSS_STATES)); /* Known state. */
> +    BUG_ON((new & X86_XCR0_STATES) &&
> +           (new & X86_XSS_STATES)); /* User or supervisor, not both. */
> +
> +    s->states |= new;
> +    if ( new & X86_XCR0_STATES )
> +    {
> +        if ( !set_xcr0(s->states & X86_XCR0_STATES) )
> +            BUG();
> +    }
> +    else
> +        set_msr_xss(s->states & X86_XSS_STATES);
> +
> +    /*
> +     * Check the uncompressed size.  Some XSTATEs are out-of-order and fill in
> +     * prior holes in the state area, so we check that the size doesn't
> +     * decrease.
> +     */
> +    hw_size = cpuid_count_ebx(0xd, 0);
> +
> +    if ( hw_size < s->uncomp_size )
> +        panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, uncompressed hw size %#x < prev size %#x\n",
> +              s->states, &new, hw_size, s->uncomp_size);
> +
> +    s->uncomp_size = hw_size;
> +
> +    /*
> +     * Check the compressed size, if available.  All components strictly
> +     * appear in index order.  In principle there are no holes, but some
> +     * components have their base address 64-byte aligned for efficiency
> +     * reasons (e.g. AMX-TILE) and there are other components small enough to
> +     * fit in the gap (e.g. PKRU) without increasing the overall length.
> +     */
> +    hw_size = cpuid_count_ebx(0xd, 1);
> +
> +    if ( cpu_has_xsavec )
> +    {
> +        if ( hw_size < s->comp_size )
> +            panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, compressed hw size %#x < prev size %#x\n",
> +                  s->states, &new, hw_size, s->comp_size);
> +
> +        s->comp_size = hw_size;
> +    }
> +    else
> +        BUG_ON(hw_size); /* Compressed size reported, but no XSAVEC ? */

I'm not quite happy with this being fatal to booting. Maybe just WARN_ON()?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:35:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:35:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728661.1133686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAATR-000236-Px; Thu, 23 May 2024 15:35:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728661.1133686; Thu, 23 May 2024 15:35:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAATR-00022z-Lg; Thu, 23 May 2024 15:35:21 +0000
Received: by outflank-mailman (input) for mailman id 728661;
 Thu, 23 May 2024 15:35:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j16U=M2=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1sAATQ-0001Wm-BD
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:35:20 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f292336-191a-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 17:35:18 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51fc01b6fe7so6890218e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:35:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f292336-191a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716478518; x=1717083318; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=WK2due6RzDqlyZIoN7S9vAmqA/HTLxu5ojLW3xzMpwM=;
        b=jNK9EnX5FS8prht7GnGQfikn6fe5ueePdIZFCKe+4oabuQQkl8Jz19+kREcjdv2t+4
         9nJ0g4QN1d1Zlsy3YkfNiUFd2407nvmv0TeF1MX3bpdyl5MWYQciD5gBrj5JvvK/q8tj
         LTGiX0TjUVhksS8A3KSlTB7hUF+s5Pnkk2e6I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716478518; x=1717083318;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WK2due6RzDqlyZIoN7S9vAmqA/HTLxu5ojLW3xzMpwM=;
        b=M4QYipgWvQUVyFPNqlhi3/sl4LougTpzdTkx8wrXjeEeDJsbLc832ifhAbrcdmisNm
         gk637lI7UtDEDWNzL4dIwbfsPKOmUBauOJi9cBKhl7fg5Eq6ASLS4PeDRM5x6cQQzzhF
         HlvQvn57eAZSpYmvvARNnWVAMcsYVSS9mi3MvZTi8KMzWrkduA28vGlJKpYaosEznUNL
         7DQZ3IyHvpWU3nS91Rc8DixirbNjBtXwBPhcD7oqoetxFGSxLZdr1QGnFuG73wR/NC/G
         lkAK1K1SXId5ETtuFyBCs8KgOzICY8nn/4DmRdIm93Ku9fyLkjjDU896XRXt+PR7ulTG
         5L/g==
X-Gm-Message-State: AOJu0Yw2acBSUjsTs47T/Z14rmPEQ3UwP9BXiYT5zQwDqbvZikvjS0QI
	MAfccPv/zMHhI7ucUxeGIYs1oWZArPC/DMX7vJwr81E8lobwR850hG4uAcmOVz9bLOmRuS8KUCo
	kzQoii5OIHLrhTAbWzmaEM8m+2mb0Tccu3gc6Pq1zgMIjGaw7IadVsQ==
X-Google-Smtp-Source: AGHT+IHiN3P2uyCiShckpnb7rRSbwrfifxWG+oM1Inw8lhMdx+Y4YRXfW6MkPl0glrNwfVTuAfQq4ReYqnaNi1cjX/Y=
X-Received: by 2002:a05:6512:2113:b0:526:92d7:52d4 with SMTP id
 2adb3069b0e04-526bf451c82mr3152501e87.61.1716478517606; Thu, 23 May 2024
 08:35:17 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Thu, 23 May 2024 16:34:41 +0100
Message-ID: <CAO-mL=wAfgraP41vxH7rYnJp0p7O6BTsiNqLg+iVXYrivAL=CQ@mail.gmail.com>
Subject: [ANNOUNCE} Postpone June Community call
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="0000000000004ea237061920cc00"

--0000000000004ea237061920cc00
Content-Type: text/plain; charset="UTF-8"

Hi all,

The next community call is on Thursday 6th June 2024, which clashes with
Xen Summit in Lisbon.

I propose we move the call a week later to *Thursday 13th June 2024, 4-5pm
(UK time). *

Many thanks,
Kelly Choi

Community Manager
Xen Project

--0000000000004ea237061920cc00
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,<div><br></div><div>The next community call is on T=
hursday 6th June 2024, which clashes with Xen Summit in Lisbon.=C2=A0</div>=
<div><br></div><div>I propose we move the call a week later to <b>Thursday =
13th June 2024, 4-5pm (UK time).=C2=A0</b></div><div><br></div><div><div><d=
iv dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"=
><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></di=
v><div><div style=3D"color:rgb(136,136,136)">Community Manager</div><div st=
yle=3D"color:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></div></div=
></div></div></div>

--0000000000004ea237061920cc00--


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:45:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:45:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728706.1133724 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAAcd-0006FP-Bq; Thu, 23 May 2024 15:44:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728706.1133724; Thu, 23 May 2024 15:44:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAAcd-0006FI-9C; Thu, 23 May 2024 15:44:51 +0000
Received: by outflank-mailman (input) for mailman id 728706;
 Thu, 23 May 2024 15:44:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAAcb-0006F7-OA
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:44:49 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6257fe40-191b-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 17:44:48 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-56e47843cc7so11032856a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:44:48 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a615e5c1a3asm630681466b.71.2024.05.23.08.44.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 08:44:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6257fe40-191b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716479087; x=1717083887; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LuimNe0L2wEtC6DnemaR8lXlDcRoIQwC7UQakkWNGXc=;
        b=WeZHSVhbqTI1w3FrgtM5yicqy8CmB/KLXZMdBpu4k62+f8/RBJPpVwcn7yJ0Uzz7+8
         1dMvxed+bRm3v7CPhwsSwPnfyvPHGSEn9oCNLFY/4/xtdJPS4JLq18+P0fPgrPg9oDjm
         S+G8YQ9gr+TQQX+OiA1HTkbKvcAsXeiKQjaCsI1B6pV1/5q8DQgsfiW2R4v+ktwD9fap
         HD8XT2ZANDBP3wQvGuCJ99qNJcbMPkTDqtq8g0JwYk6ymg5K1y3VKJ+LS3qNcaaGmtZz
         h5btc45hbERljRydZ3FexqE5MARiRaSdt8Aa2PtUUsc+bwvHY3Td8sQi6o12gUaTk9kf
         Mzsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716479087; x=1717083887;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LuimNe0L2wEtC6DnemaR8lXlDcRoIQwC7UQakkWNGXc=;
        b=m1DJzwrE8Q2H35t3SleZ7+7j/VgUAnchQO3gTw2gyzlmukoElBVvRb/SjqhS8XCiUH
         C4BvY45/sYAG1Y0oIxYuwM6xEdZxCEJd56ejfCpstRpQ4kyxed5yTZYCKmchRn3ePWCa
         8gSyEg1RZEuHtgYMNS4WFyTBoj0XOVusXGEY2N//I/QhEAcHakRlD6MfyabNTHzPPjHv
         LvarIxcEQnsreGa+pimPQHxdsvFX5ptnRkaGYdxNop1NcpPWzby+fOzYIYsxLJJhXI5o
         sE+9tGrP3V3Z5YO4Q+zrijpSRfCP6yhhjqPCpwwSnf1letdkY3Pb1uo+wUniRY6j7qqJ
         YpSA==
X-Forwarded-Encrypted: i=1; AJvYcCV/Or2XGr4r8uLZD1jXF8HFdCLU/RyVhJbDl0I9uG+IYHqfMXzaw/sYQzHUVwlnDLwrgLAbayPjmVLqax/zfnyyeKkGfBfT4yzQpM2ytFg=
X-Gm-Message-State: AOJu0YzapMqmtWNwp0auAPSw8mYr8xLOjBc63Kgkl4wz0QiFuV6oY3PA
	CtF5174y6Qh+KNhyu88HwuJMqaRkCEuw0gVtlFJS5oYuePtcojbmwC7tCkcjyQ==
X-Google-Smtp-Source: AGHT+IFB2SBLDetBouZHJ46cqlSVTXJDHrktJzbVBFe1TjHnRvKmlUzY34Jns4B/klBk+YuBDYCWjg==
X-Received: by 2002:a17:906:b6d3:b0:a59:aae5:5840 with SMTP id a640c23a62f3a-a622818fd7cmr344324766b.75.1716479087434;
        Thu, 23 May 2024 08:44:47 -0700 (PDT)
Message-ID: <8245f0ce-2964-4ecb-a31d-3e182a6d3e0b@suse.com>
Date: Thu, 23 May 2024 17:44:45 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] x86/boot: Collect the Raw CPU Policy earlier on boot
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-4-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> This is a tangle, but it's a small step in the right direction.
> 
> xstate_init() is shortly going to want data from the Raw policy.
> calculate_raw_cpu_policy() is sufficiently separate from the other policies to
> be safe to do.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Would you mind taking a look at
https://lists.xen.org/archives/html/xen-devel/2021-04/msg01335.html
to make clear (to me at least) in how far we can perhaps find common grounds
on what wants doing when? (Of course the local version I have has been
constantly re-based, so some of the function names would have changed from
what's visible there.)

> --- a/xen/arch/x86/cpu-policy.c
> +++ b/xen/arch/x86/cpu-policy.c
> @@ -845,7 +845,6 @@ static void __init calculate_hvm_def_policy(void)
>  
>  void __init init_guest_cpu_policies(void)
>  {
> -    calculate_raw_cpu_policy();
>      calculate_host_policy();
>  
>      if ( IS_ENABLED(CONFIG_PV) )
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1888,7 +1888,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
>  
>      tsx_init(); /* Needs microcode.  May change HLE/RTM feature bits. */
>  
> -    identify_cpu(&boot_cpu_data);
> +    calculate_raw_cpu_policy(); /* Needs microcode.  No other dependenices. */
> +
> +    identify_cpu(&boot_cpu_data); /* Needs microcode and raw policy. */

You don't introduce any dependency on raw policy here, and there cannot possibly
have been such a dependency before (unless there was a bug somewhere). Therefore
I consider this latter comment misleading at this point.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 15:46:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 15:46:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728715.1133740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAAe3-00072S-NE; Thu, 23 May 2024 15:46:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728715.1133740; Thu, 23 May 2024 15:46:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAAe3-00072L-KX; Thu, 23 May 2024 15:46:19 +0000
Received: by outflank-mailman (input) for mailman id 728715;
 Thu, 23 May 2024 15:46:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAAe2-00072D-W2
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 15:46:18 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9758d6d4-191b-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 17:46:16 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51ffff16400so12682511e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 08:46:16 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a615e5c1a3asm630681466b.71.2024.05.23.08.46.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 08:46:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9758d6d4-191b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716479176; x=1717083976; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wSex4tW73EOoQHbkuY5EKgDSGJZy0tnN7GO8Ad/mpCo=;
        b=J6r6NBL4t1pHWSTAw0HBcTpncF6bbO6FQNJzJ5fAe3//C0lb1Rah2uqo2JAxpqAhpk
         logFKN4zq2Abc99ykyxuAISkxSA7lNkzvDRI3Uzf+eL6NKiojmzeV/h9zGSTEUkQuK5Q
         B05TEaI/PvJhTgt/PAJJt+AutoYDht3XslO+zYoOyT3cOYBbZHDBwYG3nQwDe8WsLQZ6
         2aMNiPaMyLWgZT3ijnTFIG4WOD62LL+qOLlwHPCYjV0zwU5pyDeJcd7lG1BUC44AVDAP
         MpVN6MO+b3QvePPnm3mVhBzAhtpDF0ggChLPNbMxg4fLD0g3bxvACNgsVlULbnjus7jA
         /cDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716479176; x=1717083976;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wSex4tW73EOoQHbkuY5EKgDSGJZy0tnN7GO8Ad/mpCo=;
        b=BrlSiBaBrdbKOFoezla5daTY7kdo/dAEtLQiQSdnHraHyIFuA+Ykxb5JtieXeyt8nS
         AR74ohFQlqHG/tXUQq+8E5Y4x+U+6tMoqLFDvZieREsVFnZ0w/JmObir/5NNtNex/lIS
         LLrAMd0iO9z3w2OmHIaLYyRoWSEcajS0A0jYiZd43L57KwGRwCDgSiEGedHibsLZP11u
         JmKPtoVwbpBYBUMbZaOhfPz/d2pQ8YkvRY9zdWm1u9IAfW1wxu3YdUgGCDL/yQyhCY0m
         qen3YEzB86J3Pdb9uJMWIUPq1/9o5eUtrE/FXo0UNwXrlOEi+F1xEnAsPZv3ZkLlKPlN
         VLdg==
X-Forwarded-Encrypted: i=1; AJvYcCVLxiJ++h92cJwGC3XEApF9XkGg0rq0uFaYmfHFW2OjGMM3siAkWOanyAhce2VsrLoy5h1BpyRkSoKsnlKQ7wcJMefl0jx/q51STd4SyFs=
X-Gm-Message-State: AOJu0Yy4/JhXVwMWe1XqBhCWj+46aVcq8nWNu8U7dPziZbcRtdoiMbDT
	6ZQCBsbBQ1MzRkiJg5LQsK3bXlVR6OGRR8Tt81DwbY5cgCWPx8TuTMH4qqYhdA==
X-Google-Smtp-Source: AGHT+IHfvuApfkEzAjad5SVRK28ZVnnHSaMw6C1K+wkiIH+IaoG/f4l4fGxbbgAH8hxbCk8oi5phuQ==
X-Received: by 2002:ac2:44ca:0:b0:51d:2af8:2e7b with SMTP id 2adb3069b0e04-526c1213fd4mr4400797e87.57.1716479176135;
        Thu, 23 May 2024 08:46:16 -0700 (PDT)
Message-ID: <d67f13f3-ac3e-4f2c-8195-39c87c753547@suse.com>
Date: Thu, 23 May 2024 17:46:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 7/7] x86/defns: Clean up X86_{XCR0,XSS}_* constants
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-8-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-8-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> With the exception of one case in read_bndcfgu() which can use ilog2(),
> the *_POS defines are unused.
> 
> X86_XCR0_X87 is the name used by both the SDM and APM, rather than
> X86_XCR0_FP.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu May 23 16:09:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:09:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728726.1133756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB0I-0003xF-JZ; Thu, 23 May 2024 16:09:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728726.1133756; Thu, 23 May 2024 16:09:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB0I-0003x8-Gl; Thu, 23 May 2024 16:09:18 +0000
Received: by outflank-mailman (input) for mailman id 728726;
 Thu, 23 May 2024 16:09:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAB0H-0003x2-JD
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 16:09:17 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd8eb929-191e-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 18:09:16 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59ab4f60a6so1029600466b.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:09:16 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5cde6f8c89sm1104579766b.70.2024.05.23.09.09.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 09:09:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd8eb929-191e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716480555; x=1717085355; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pxSEiwn+dUJxSbRC+Me5htjo8uhksvt5o8XNMC4y8zg=;
        b=bEqP0NItQPQI7DrPy8VJJZ+bpKpBqAgqtOu9nGmMpctWMHFcothqB5kq/qJBdIJB47
         4yuo6uRdYiZlWtDRAz/UZgF1DJc6Ik1VXwyuST1Q17j2PtsFc3AyQIj7d5lQBUnvjmlF
         0iC+2r6/6ix9S6DxwglTEdP4H/ecgmNS3F04EvkJMIdA9ukKfbuMRH57zYTOxFuD8xTR
         xXJ0C+8SOCXnCoLdjqwwm3SqSLc+MwVT4hXOpEuZYVOxPOaxiFeJLHv/rdDhKbucLQAd
         e6nklIFMmIGutbZspCyF+zB8kpdUwKp7mEmulcq9U0MGtwto5RFYmNF7PZASnrWuvv3j
         WwvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716480555; x=1717085355;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pxSEiwn+dUJxSbRC+Me5htjo8uhksvt5o8XNMC4y8zg=;
        b=dNxJAvcokdGxhF3K0z2mcIjw7juWKl7GLyvYpM+0l5BCdAYD9oMaVy+wVCVg6Pozcs
         CIR+EiwCqjphmP/hWGIhI7OkMdbpQKB4X7LWW6KrKkEGtf0qbO+O0dYAsZDYzs28cXFX
         AYt6yQhtkaHQy5o/sML5+kdM8HBshAlvWZxQzeVmTiMgY+Vt8yUHDiXWqEJSTJwkov11
         0opP8AD5YBdNDrwlLiZK1tD8Rha7TcipDugzsyhu4Gfca84/3ReogJ1RqnrFVTHMBB31
         4FwS8YS1bL+KEgCookQW6KGFvRa11TJE2F3dfLAg4gH+BNx7bEO38lfYiFMgu0ZCIwXy
         WQZQ==
X-Forwarded-Encrypted: i=1; AJvYcCUXBKFSQciCrXUHMZ7N9THKlyfgGLe2ov50vH6+Z81EMVQ4wpd9pgspVBgBr3ISaqkIFJ+UnDOeolSzdggYiDQTh4azosDyELB332XmoiU=
X-Gm-Message-State: AOJu0YwFX7fEYbRc6VtBVo3nKsaX6mBWcWx6iw032qDhSGLsuRkYlYj4
	mFMZ3D26xHGO0Wu4EwRndfKdxRiigNTj6QdfcU7MCmeNvpXuHkM62JfJ+61wmw==
X-Google-Smtp-Source: AGHT+IH5QYdFAFKYtS8rnBO7d0q/aDRAStpjo92pUN+YQYwlrsck/y9mNADOTpuxwNdaLHUHrlyD0A==
X-Received: by 2002:a17:906:298a:b0:a62:47d7:247d with SMTP id a640c23a62f3a-a6247d72788mr138494766b.56.1716480555465;
        Thu, 23 May 2024 09:09:15 -0700 (PDT)
Message-ID: <b0d92d89-5ca7-4870-8118-139a47057a88@suse.com>
Date: Thu, 23 May 2024 18:09:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/7] x86/xstate: Rework xstate_ctxt_size() as
 xstate_uncompressed_size()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> @@ -611,6 +587,40 @@ static bool valid_xcr0(uint64_t xcr0)
>      return true;
>  }
>  
> +unsigned int xstate_uncompressed_size(uint64_t xcr0)
> +{
> +    unsigned int size = XSTATE_AREA_MIN_SIZE, i;
> +
> +    ASSERT((xcr0 & ~X86_XCR0_STATES) == 0);

I'm puzzled by the combination of this assertion and ...

> +    if ( xcr0 == xfeature_mask )
> +        return xsave_cntxt_size;

... this conditional return. Yes, right now we don't support/use any XSS
components, but without any comment the assertion looks overly restrictive
to me.

> @@ -818,14 +834,14 @@ void xstate_init(struct cpuinfo_x86 *c)
>           * xsave_cntxt_size is the max size required by enabled features.
>           * We know FP/SSE and YMM about eax, and nothing about edx at present.
>           */
> -        xsave_cntxt_size = hw_uncompressed_size(feature_mask);
> +        xsave_cntxt_size = cpuid_count_ebx(0xd, 0);
>          printk("xstate: size: %#x and states: %#"PRIx64"\n",
>                 xsave_cntxt_size, xfeature_mask);
>      }
>      else
>      {
>          BUG_ON(xfeature_mask != feature_mask);
> -        BUG_ON(xsave_cntxt_size != hw_uncompressed_size(feature_mask));
> +        BUG_ON(xsave_cntxt_size != cpuid_count_ebx(0xd, 0));
>      }

Hmm, this may make re-basing of said earlier patch touching this code yet
more interesting. Or maybe it actually simplifies things, will need to see
... The overall comment remains though: Patches pending for so long should
really take priority over creating yet more new ones. But what do I do - I
can't enforce this, unless I was now going to block your work the same way.
Which I don't mean to do.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 16:13:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:13:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728731.1133765 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB4i-0005q6-4i; Thu, 23 May 2024 16:13:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728731.1133765; Thu, 23 May 2024 16:13:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB4i-0005pz-1l; Thu, 23 May 2024 16:13:52 +0000
Received: by outflank-mailman (input) for mailman id 728731;
 Thu, 23 May 2024 16:13:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAB4g-0005pt-DP
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 16:13:50 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fa70c37-191f-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 18:13:48 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e716e302bdso59321091fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:13:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100f598d7sm28915355e9.23.2024.05.23.09.13.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 09:13:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fa70c37-191f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716480827; x=1717085627; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=fNMl+c2aUThh6P7gC9FUhiHaLSy9TBrVGvb+cYGQxWc=;
        b=Usl45pg81WsqOnKEHVWXQU2mvc4qKTedpvqMJeAh3lAmxemxgjmUN55cGga7eXGOkI
         LZc60JWN2oEKAd/QGTy4ViV4qT0lb2AFkcgPBzRjRAdyTTJmnb3bjxp/3xdydqT4IX6v
         5XTstFMUWWnwHHZD6WxU27Eh1/hTUzERPv9dg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716480827; x=1717085627;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=fNMl+c2aUThh6P7gC9FUhiHaLSy9TBrVGvb+cYGQxWc=;
        b=TwjenH9elul3qKzp46vTIJ+tvm/f2fUf644xA40pY6bW577U3g6ZAzAMXH00Oxs/bs
         WzDc+2KkrMpmy150Nxe4VCymi1YcIDXVimg8U8rGnvf2pj36fS1XYdjRUnrVDkNgnH7T
         corLj1EVplwdr20l0T/SxRv2c6hxSEu6rRQikEmc1rzb6yAzjF3rPGbBA+XILB7P5ZuF
         r0iasVpuy6FDUDlosBd4WMJJRu/UPOgiUCRIAbB6gGdWCS9XS4GUDRpG3ilXz+3zlphi
         zLzTM2P3eRPVCBurgURH0W0iMUxGR52yA8PtNSoUGyajeFZ8liD/eDbOxo46a/rEmub3
         X/iA==
X-Gm-Message-State: AOJu0Yw2hSb8yUvpHnAVHyM7h3pyc+rEKA+FxtpmEob1f2KcTtJfsVh/
	QddUZI+CTE/Itx8nDZRyhgkoDWK0weM9TlvNEZ+q2qrGm2Z7l6dJcMM5Ywyz4So=
X-Google-Smtp-Source: AGHT+IFWO8EnZg4CmyGpDarYXZFCzXACW6e7USkMCprj1r6E9eXenEKjdKat3z1+BnmxOdEerGHfpA==
X-Received: by 2002:a2e:8e8c:0:b0:2e1:aa94:cf48 with SMTP id 38308e7fff4ca-2e9495a87famr33424921fa.20.1716480827345;
        Thu, 23 May 2024 09:13:47 -0700 (PDT)
Date: Thu, 23 May 2024 18:13:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
Message-ID: <Zk9rOqooa3PJCyw1@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
> Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
> populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
> from hvmloader.
> 
> While at this also remove ap_callin, as writing the APIC ID may serve the same
> purpose.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * New patch. Replaces adding cpu policy to hvmloader in v1.
> ---
>  tools/firmware/hvmloader/config.h    |  6 ++++-
>  tools/firmware/hvmloader/hvmloader.c |  4 +--
>  tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
>  tools/firmware/hvmloader/util.h      |  5 ++++
>  xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
>  5 files changed, 47 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
> index c82adf6dc508..edf6fa9c908c 100644
> --- a/tools/firmware/hvmloader/config.h
> +++ b/tools/firmware/hvmloader/config.h
> @@ -4,6 +4,8 @@
>  #include <stdint.h>
>  #include <stdbool.h>
>  
> +#include <xen/hvm/hvm_info_table.h>
> +
>  enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
>  extern enum virtual_vga virtual_vga;
>  
> @@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
>  
>  #define IOAPIC_ID           0x01
>  
> +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> +
>  #define LAPIC_BASE_ADDRESS  0xfee00000
> -#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
> +#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
>  
>  #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
>  #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
> diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
> index c58841e5b556..1eba92229925 100644
> --- a/tools/firmware/hvmloader/hvmloader.c
> +++ b/tools/firmware/hvmloader/hvmloader.c
> @@ -342,11 +342,11 @@ int main(void)
>  
>      printf("CPU speed is %u MHz\n", get_cpu_mhz());
>  
> +    smp_initialise();
> +
>      apic_setup();
>      pci_setup();
>  
> -    smp_initialise();
> -
>      perform_tests();
>  
>      if ( bios->bios_info_setup )
> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
> index a668f15d7e1f..4d75f239c2f5 100644
> --- a/tools/firmware/hvmloader/smp.c
> +++ b/tools/firmware/hvmloader/smp.c
> @@ -29,7 +29,34 @@
>  
>  #include <xen/vcpu.h>
>  
> -static int ap_callin, ap_cpuid;
> +static int ap_cpuid;
> +
> +/**
> + * Lookup table of x2APIC IDs.
> + *
> + * Each entry is populated its respective CPU as they come online. This is required
> + * for generating the MADT with minimal assumptions about ID relationships.
> + */
> +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> +
> +static uint32_t read_apic_id(void)
> +{
> +    uint32_t apic_id;
> +
> +    cpuid(1, NULL, &apic_id, NULL, NULL);
> +    apic_id >>= 24;
> +
> +    /*
> +     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
> +     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
> +     * but only if the x2APIC feature is present. If there are that many CPUs
> +     * it's guaranteed to be there so we can avoid checking for it specifically
> +     */

Maybe I'm missing something, but given the current code won't Xen just
return the low 8 bits from the x2APIC ID?  I don't see any code in
guest_cpuid() that adjusts the IDs to be 255 when > 255.

> +    if ( apic_id == 255 )
> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);

Won't the correct logic be to check if x2APIC is set in CPUID, and
then fetch the APIC ID from leaf 0xb, otherwise fallback to fetching
the APID ID from leaf 1?

> +
> +    return apic_id;
> +}
>  
>  static void ap_start(void)
>  {
> @@ -37,12 +64,12 @@ static void ap_start(void)
>      cacheattr_init();
>      printf("done.\n");
>  
> +    wmb();
> +    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();

A comment would be helpful here, that CPU_TO_X2APICID[ap_cpuid] is
used as synchronization that the AP has started.

You probably want to assert that read_apic_id() doesn't return 0,
otherwise we are skewed.

> +
>      if ( !ap_cpuid )
>          return;
>  
> -    wmb();
> -    ap_callin = 1;
> -
>      while ( 1 )
>          asm volatile ( "hlt" );
>  }
> @@ -86,10 +113,11 @@ static void boot_cpu(unsigned int cpu)
>          BUG();
>  
>      /*
> -     * Wait for the secondary processor to complete initialisation.
> +     * Wait for the secondary processor to complete initialisation,
> +     * which is signaled by its x2APIC ID being writted to the LUT.
>       * Do not touch shared resources meanwhile.
>       */
> -    while ( !ap_callin )
> +    while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) )
>          cpu_relax();

As a further improvement, we could launch all APs in pararell, and use
a for loop to wait until all positions of the CPU_TO_X2APICID array
are set.

>  
>      /* Take the secondary processor offline. */
> diff --git a/tools/firmware/hvmloader/util.h b/tools/firmware/hvmloader/util.h
> index 14078bde1e30..51e9003bc615 100644
> --- a/tools/firmware/hvmloader/util.h
> +++ b/tools/firmware/hvmloader/util.h
> @@ -23,6 +23,11 @@ enum {
>  #define __STR(...) #__VA_ARGS__
>  #define STR(...) __STR(__VA_ARGS__)
>  
> +#define __ACCESS_ONCE(x) ({                             \
> +            (void)(typeof(x))0; /* Scalar typecheck. */ \
> +            (volatile typeof(x) *)&(x); })
> +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))

Might be better for this to be placed in common-macros.h?

> +
>  /* GDT selector values. */
>  #define SEL_CODE16          0x0008
>  #define SEL_DATA16          0x0010
> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
> index 84911f3ebcb4..6c005f0b0b38 100644
> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -16,6 +16,7 @@
>  #include <asm/current.h>
>  #include <asm/x86_emulate.h>
>  #include <asm/hvm/asid.h>
> +#include <asm/hvm/vlapic.h>

Is this a stray change?  Otherwise I don't see why this need to be
part of this patch when the rest of the changes are exclusively to
hvmloader.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 16:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728738.1133776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB7k-0006h1-Mg; Thu, 23 May 2024 16:17:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728738.1133776; Thu, 23 May 2024 16:17:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB7k-0006gu-Im; Thu, 23 May 2024 16:17:00 +0000
Received: by outflank-mailman (input) for mailman id 728738;
 Thu, 23 May 2024 16:16:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OfYh=M2=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAB7i-0006dc-IC
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 16:16:58 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0e517f8-191f-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 18:16:58 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-52389c1308dso8013337e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:16:58 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a178a9f0fsm1952966966b.69.2024.05.23.09.16.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 09:16:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0e517f8-191f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716481017; x=1717085817; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sr7oOaeYh+MMHlLzzIuDgYAR3uYP8TI+UYhioPNjceU=;
        b=gpm5kyGpGWrj64R+ASNYOcpNojg9XTleOGqpDqR3KPPPRmJ3B8XeeuoY2Qx6PBmOpb
         XWRG+8PG8T1lfs9s2SAiGMdxUNXDP7Gl7WQwLMizAZxiMW2afhkWyMxAFG6Jk9Fejjqu
         JoO1HaURZHnR0uvYHQ9Y7F+qekAxutIbKX66TJluXB5Jhv3epSe/Si5dkKfSVbdS5MLs
         HTt7QmFgBwlsnxdvsvDd5dfqwPFlvq+3Apr10VSr2WQAeraApFqHrMIyAEa9qfCuUH1L
         p6k5s3BN8LUFIDv+c0F12l+bSgSHvhx6d5vlomxWd0Y8jZAldipfa8iByuAg8mQrS5wO
         hDSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716481017; x=1717085817;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sr7oOaeYh+MMHlLzzIuDgYAR3uYP8TI+UYhioPNjceU=;
        b=XtCdWLEv/WDF9fvnFGy2XHzFig2XU4copK7Gqcngm0SF9WgqLnvcrEXaDlRbMX7S9e
         Mf4oZNbhIOWS4wfTWfZU+8r29Vh9mR+Hp+GnWNJkOaer67v+v9WyyBR4P+kOoKXYmV9M
         WNSSVeqXaFU9NIDXI3CUVAm5B17BxH0HVJGlCzZp/tsaVOvo9wCLUCT7k3xv47wrgVG9
         EHAhXOLPVQ+isFyoMAPsxcIbUU0SUCFapsDlLMipi6d/wc/qRa1zR4vw32vPH2dT1Fra
         uUCgsYbf1zVm5/wrB3h9YV1Ec/ifGWl6R/hELbUnt8vkEXjiyjTDVAyfP0QalZasjwem
         uvsg==
X-Forwarded-Encrypted: i=1; AJvYcCUq7E26NVzAoQDrNamyRQFjbhuYAW8IRLipnncHT9BvUKLCk6o4sMISLNQ94MvAG0QbpWzuXlgAPha0AaAF6I+HmW4UX742vJDvw62MorE=
X-Gm-Message-State: AOJu0YxW64brvhnpf1JJcmtf7OrhnJYUDC3jfQJobnKwRWMful8s6UYp
	EE5cJB9E0O01ds3vym62EOcB+WoQCziERyF7EHwn5QCV2cb7s7Y0nGX5VvpsR9R4gBqS88Z/e9I
	=
X-Google-Smtp-Source: AGHT+IEhMtptW6zv3AOsQhx2YiwWpvX3psi4zc+9IikG0k6N0qlQRtLl6h6mumkSM93/RqpjXLxTeg==
X-Received: by 2002:a05:6512:304d:b0:51d:3fe4:c3cc with SMTP id 2adb3069b0e04-526baa662ddmr4252546e87.0.1716481017595;
        Thu, 23 May 2024 09:16:57 -0700 (PDT)
Message-ID: <9e6803c6-3d83-4e5c-a7bd-b8b844eec66d@suse.com>
Date: Thu, 23 May 2024 18:16:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/7] x86/cpuid: Fix handling of XSAVE dynamic leaves
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240523111627.28896-1-andrew.cooper3@citrix.com>
 <20240523111627.28896-7-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240523111627.28896-7-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.05.2024 13:16, Andrew Cooper wrote:
> First, if XSAVE is available in hardware but not visible to the guest, the
> dynamic leaves shouldn't be filled in.
> 
> Second, the comment concerning XSS state is wrong.  VT-x doesn't manage
> host/guest state automatically, but there is provision for "host only" bits to
> be set, so the implications are still accurate.
> 
> Introduce xstate_compressed_size() to mirror the uncompressed one.  Cross
> check it at boot.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
Irrespective ...

> v3:
>  * Adjust commit message about !XSAVE guests
>  * Rebase over boot time cross check
>  * Use raw policy

... it should probably have occurred to me earlier on to ask: Why raw policy?
Isn't the host one the more appropriate one to use for any kind of internal
decisions?

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 23 16:17:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:17:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728741.1133786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB8M-0007Gh-UO; Thu, 23 May 2024 16:17:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728741.1133786; Thu, 23 May 2024 16:17:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAB8M-0007Ga-RT; Thu, 23 May 2024 16:17:38 +0000
Received: by outflank-mailman (input) for mailman id 728741;
 Thu, 23 May 2024 16:17:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAB8L-0007GK-SG; Thu, 23 May 2024 16:17:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAB8L-0001PK-P6; Thu, 23 May 2024 16:17:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAB8L-00088e-Hj; Thu, 23 May 2024 16:17:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAB8L-0007XJ-HJ; Thu, 23 May 2024 16:17:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+PGLZ1v+R/YJswu1PZStCxY8HenNqwOr4bBvDGp7UOY=; b=SctHZbdf35kkxmAsnkc73txuoD
	qNP3llPDpoS+8NUWQ6Bum74RCJthJiyY6E33AxypIll4LpBD4ScUv9KK8eeh0NqVoXaFFNeXFPVyp
	1KS2zdSS8zOGbkO15q9dJu/XtcaVKty2D3M3m1H8et0LanRCAZZC+SulhEyiG3YKr1mc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186108-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186108: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9e58da32cc844b3fb7612fc35ece3a96f8cbf744
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 16:17:37 +0000

flight 186108 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186108/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186064

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9e58da32cc844b3fb7612fc35ece3a96f8cbf744
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186064  2024-05-21 15:04:02 Z    2 days
Failing since        186104  2024-05-23 09:00:22 Z    0 days    3 attempts
Testing same since   186108  2024-05-23 14:00:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
  Bobby Eshleman <bobbyeshleman@gmail.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Tamas K Lengyel <tamas@tklengyel.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 406 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 16:40:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:40:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728757.1133795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sABUJ-0004lH-ND; Thu, 23 May 2024 16:40:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728757.1133795; Thu, 23 May 2024 16:40:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sABUJ-0004lA-Kd; Thu, 23 May 2024 16:40:19 +0000
Received: by outflank-mailman (input) for mailman id 728757;
 Thu, 23 May 2024 16:40:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TgV8=M2=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sABUI-0004l4-Ms
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 16:40:18 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 22bd5327-1923-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 18:40:17 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56e48d0a632so16271677a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:40:17 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5733c34fed1sm20017906a12.96.2024.05.23.09.40.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 09:40:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22bd5327-1923-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716482416; x=1717087216; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=ByR5oC3ZqlfSCZX3QYIiuX/qOz3B9BfZ8Aja4wUjKq4=;
        b=iOTMbRLFSiX/7nmoVrjIYl4gjDvZcOk8LrGBu4eHzilV94Myfn7HeCexBMl3DbyknS
         +lLEp3SOIU2gM4KCtw2rrdYRqGDqoC3LaMQPsnyMvBeOs06c0JiwipWnLDHKNKTMIQRx
         74ORVtsxmXGDfybNIdjr/zltflB+zaPknAowm7P+92D26SStpbCIwOzccPsKHfCI7nsN
         baky0cNQHTin3bNy1tZakwSIisaTfwnShTagfHgMO/S6oDq4YKvKKbOvIHb/IiMR93Os
         z3OSnbSn7rH1EH/LJWzLtvHQPEnP1i6VLVvpGx7iMLZp/poRMouhXaxQ5So+fOLaRxyX
         QSMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716482416; x=1717087216;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ByR5oC3ZqlfSCZX3QYIiuX/qOz3B9BfZ8Aja4wUjKq4=;
        b=eitkM8jg8c3OaMLChExYa38BdbHMeAdXAZC7CuhR3EbhAsjfCv/KMnJHyYXV7JfKov
         ddooNVtG6yewx03GgEIPgUBTr3QL2tbOYXiEyYZYmE+oFTd3pjEMd5ae3wl5BjZTc0mC
         0ysEpvFGzOiF63DAgmKN9V9zLCa0dxekgd7v+uoT2ivHAcwf48Am/gaKXJvhsoLqEZkb
         lGT5v4VNdyie/8qXspILkBUHSjRN4ID8V/pyPCEeDOD3303yFppIahgo6e9l+p67RfbV
         BosolURtDod1IRwUVGiWoj+hYFGx7Manac8MpLBrXsLTWyGAAA6iMKFGlhQMM3eFX1HU
         punQ==
X-Forwarded-Encrypted: i=1; AJvYcCVhMWZiEWMNsVHF4/sDjopCyrBisFynKSV2USdBsFQP5FS0nrY13MAbi1+OoQ/9/YX2aGdoWz9TAHj+qA21qXXuzFasdv3zuD4H/1lA6XA=
X-Gm-Message-State: AOJu0YwiUxTVzLneP7EZbFbOEYiBh5kA6MVP5qJDg7OwM2Wu9S4qgXOf
	UMMeSbgdx9KapRVa3QHgtQRdx81dzKmVzhlV1mfGeZh8S4CxgqyS
X-Google-Smtp-Source: AGHT+IEq1SRzfMGIJnUc0Iz+cPnRHCHij+cC8lOh6QoOo8xfEsGjbbvVPocx8HTWoNpDcHPbLbU9uQ==
X-Received: by 2002:a50:d513:0:b0:574:ebf4:f787 with SMTP id 4fb4d7f45d1cf-57832a3b5eemr4728108a12.19.1716482416219;
        Thu, 23 May 2024 09:40:16 -0700 (PDT)
Message-ID: <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, Jan
 Beulich <jbeulich@suse.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
Date: Thu, 23 May 2024 18:40:14 +0200
In-Reply-To: <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
	 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
	 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
	 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
>=20
>=20
> On 23/05/2024 15:11, Oleksii K. wrote:
> > On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
> > > Hi Oleksii,
> > Hi Julien,
> >=20
> > >=20
> > > On 17/05/2024 14:54, Oleksii Kurochko wrote:
> > > > diff --git a/xen/arch/arm/arm64/livepatch.c
> > > > b/xen/arch/arm/arm64/livepatch.c
> > > > index df2cebedde..4bc8ed9be5 100644
> > > > --- a/xen/arch/arm/arm64/livepatch.c
> > > > +++ b/xen/arch/arm/arm64/livepatch.c
> > > > @@ -10,7 +10,6 @@
> > > > =C2=A0=C2=A0 #include <xen/mm.h>
> > > > =C2=A0=C2=A0 #include <xen/vmap.h>
> > > > =C2=A0=C2=A0=20
> > > > -#include <asm/bitops.h>
> > >=20
> > > It is a bit unclear how this change is related to the patch. Can
> > > you
> > > explain in the commit message?
> > Probably it doesn't need anymore. I will double check and if this
> > change is not needed, I will just drop it in the next patch
> > version.
> >=20
> > >=20
> > > > =C2=A0=C2=A0 #include <asm/byteorder.h>
> > > > =C2=A0=C2=A0 #include <asm/insn.h>
> > > > =C2=A0=C2=A0 #include <asm/livepatch.h>
> > > > diff --git a/xen/arch/arm/include/asm/bitops.h
> > > > b/xen/arch/arm/include/asm/bitops.h
> > > > index 5104334e48..8e16335e76 100644
> > > > --- a/xen/arch/arm/include/asm/bitops.h
> > > > +++ b/xen/arch/arm/include/asm/bitops.h
> > > > @@ -22,9 +22,6 @@
> > > > =C2=A0=C2=A0 #define __set_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 set_bit(n,p)
> > > > =C2=A0=C2=A0 #define __clear_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 clear_bit(n,p)
> > > > =C2=A0=C2=A0=20
> > > > -#define BITOP_BITS_PER_WORD=C2=A0=C2=A0=C2=A0=C2=A0 32
> > > > -#define BITOP_MASK(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 (1UL << ((nr) %
> > > > BITOP_BITS_PER_WORD))
> > > > -#define BITOP_WORD(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > > > =C2=A0=C2=A0 #define BITS_PER_BYTE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 8
> > >=20
> > > OOI, any reason BITS_PER_BYTE has not been moved as well? I don't
> > > expect
> > > the value to change across arch.
> > I can move it to generic one header too in the next patch version.
> >=20
> > >=20
> > > [...]
> > >=20
> > > > diff --git a/xen/include/xen/bitops.h
> > > > b/xen/include/xen/bitops.h
> > > > index f14ad0d33a..6eeeff0117 100644
> > > > --- a/xen/include/xen/bitops.h
> > > > +++ b/xen/include/xen/bitops.h
> > > > @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned
> > > > long
> > > > x)
> > > > =C2=A0=C2=A0=C2=A0 * scope
> > > > =C2=A0=C2=A0=C2=A0 */
> > > > =C2=A0=C2=A0=20
> > > > +#define BITOP_BITS_PER_WORD 32
> > > > +typedef uint32_t bitop_uint_t;
> > > > +
> > > > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > > > BITOP_BITS_PER_WORD))
> > > > +
> > > > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > > > +
> > > > +extern void __bitop_bad_size(void);
> > > > +
> > > > +#define bitop_bad_size(addr) (sizeof(*(addr)) <
> > > > sizeof(bitop_uint_t))
> > > > +
> > > > =C2=A0=C2=A0 /* --------------------- Please tidy above here ------=
------
> > > > -----
> > > > ---- */
> > > > =C2=A0=C2=A0=20
> > > > =C2=A0=C2=A0 #include <asm/bitops.h>
> > > > =C2=A0=C2=A0=20
> > > > +/**
> > > > + * generic__test_and_set_bit - Set a bit and return its old
> > > > value
> > > > + * @nr: Bit to set
> > > > + * @addr: Address to count from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > >=20
> > > Sorry for only mentioning this on v10. I think this comment
> > > should be
> > > duplicated (or moved to) on top of test_bit() because this is
> > > what
> > > everyone will use. This will avoid the developper to follow the
> > > function
> > > calls and only notice the x86 version which says "This function
> > > is
> > > atomic and may not be reordered." and would be wrong for all the
> > > other arch.
> > It makes sense to add this comment on top of test_bit(), but I am
> > curious if it is needed to mention that for x86 arch_test_bit() "is
> > atomic and may not be reordered":
>=20
> I would say no because any developper modifying common code can't=20
> relying it.
>=20
> >=20
> > =C2=A0 * This operation is non-atomic and can be reordered. ( Exception=
:
> > for
> > * x86 arch_test_bit() is atomic and may not be reordered )
> > =C2=A0 * If two examples of this operation race, one can appear to
> > succeed
> > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesses wi=
th a
> > lock.
> > =C2=A0 */
> >=20
> > >=20
> > > > +static always_inline bool
> > > > +generic__test_and_set_bit(int nr, volatile void *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +
> > > > +/**
> > > > + * generic__test_and_clear_bit - Clear a bit and return its
> > > > old
> > > > value
> > > > + * @nr: Bit to clear
> > > > + * @addr: Address to count from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > >=20
> > > Same applies here and ...
> > >=20
> > > > +static always_inline bool
> > > > +generic__test_and_clear_bit(int nr, volatile void *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +
> > > > +/* WARNING: non atomic and it can be reordered! */
> > >=20
> > > ... here.
> > >=20
> > > > +static always_inline bool
> > > > +generic__test_and_change_bit(int nr, volatile void *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bitop_ui=
nt_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > +}
> > > > +/**
> > > > + * generic_test_bit - Determine whether a bit is set
> > > > + * @nr: bit number to test
> > > > + * @addr: Address to start counting from
> > > > + */
> > > > +static always_inline bool generic_test_bit(int nr, const
> > > > volatile
> > > > void *addr)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > +=C2=A0=C2=A0=C2=A0 const volatile bitop_uint_t *p =3D
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (c=
onst volatile bitop_uint_t *)addr +
> > > > BITOP_WORD(nr);
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > > > +}
> > > > +
> > > > +static always_inline bool
> > > > +__test_and_set_bit(int nr, volatile void *addr)
> > > > +{
> > > > +#ifndef arch__test_and_set_bit
> > > > +#define arch__test_and_set_bit generic__test_and_set_bit
> > > > +#endif
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
> > > > +}
> > >=20
> > > NIT: It is a bit too late to change this one. But I have to
> > > admit, I
> > > don't understand the purpose of the static inline when you could
> > > have
> > > simply call...
> > >=20
> > > > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=
 \
> > > > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 \
> > >=20
> > > ... __arch__test_and_set_bit here.
> > I just wanted to be in sync with other non-atomic test*_bit()
> > operations and Andrew's patch series ( xen/bitops: Reduce the mess,
> > starting with ffs() ).
>=20
> I looked at Andrew series and I can't seem to find an example where
> he=20
> uses both the macro + static inline. He seems only use a static
> inline.
> Do you have any pointer?
>=20
> But by any chance are you referring to the pattern on x86? If so, I=20
> would really like to understand what's the benefits of introducing a
> a=20
> one-liner static inline which is "shadowed" by a macro...
Yes, I was referring to the x86 pattern.

I tried to find the reason in the commit but couldn't:
https://lists.xenproject.org/archives/html/xen-changelog/2008-03/msg00097.h=
tml

For some reason, I also couldn't find the mailing list thread for this.

Perhaps Jan could help here, based on the commit message he might have
a suggestion.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 23 16:50:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 16:50:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728764.1133805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sABeQ-00077T-KR; Thu, 23 May 2024 16:50:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728764.1133805; Thu, 23 May 2024 16:50:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sABeQ-00077M-Ht; Thu, 23 May 2024 16:50:46 +0000
Received: by outflank-mailman (input) for mailman id 728764;
 Thu, 23 May 2024 16:50:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bqPc=M2=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sABeO-00077G-Tz
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 16:50:45 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97622fe6-1924-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 18:50:42 +0200 (CEST)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-4202cea9941so55185525e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 09:50:42 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-355038eb8cbsm503505f8f.33.2024.05.23.09.50.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 09:50:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97622fe6-1924-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716483041; x=1717087841; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Une7sB2nD33xp74w7FaBn0c64cKbphTpkHiHzkUm+EQ=;
        b=X1BZRMzGMn5WhqZAD8l19c1W5ZYY7SX66OX2rQ2Bl4YLbrGxJROSvzJLQG6kFPHPhy
         fUOyFeiYsFc6YOCMjqRUDAHpCAFZBTonLuSrhqF6L+ADwKvNjnHXvE0fbOxclm1INSoD
         raXmregwnldcXm5bAlHoIwsGzN6w9edsMXpAM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716483041; x=1717087841;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Une7sB2nD33xp74w7FaBn0c64cKbphTpkHiHzkUm+EQ=;
        b=G1nEp25LlxmJZpXpP+sru78mnMiwK/3zbkkxFXu+y7yqO53j+RRiuUTC4oNE+9sIMS
         vxrU/ir2U/rVxxjs1dw2h6i3o9O2tW+KLhZChxOCfGEW3FSfZQb61XcR9dMfcWhp4gBq
         40r95bYCIBwMRhFqVKg16kk13RI1lpuDWpMVjxIbuw2azJquZdCaUqIEpBgRlErxVHUx
         FC19qsTotuhtATDXRi23kYgNAoeEgkDgF7eEZ7mExfA8Z/FBSaCQ/Cz7yizxt9s1vaEK
         WNOimWEewsvzJLvOHUNRF4l/u7d8snDari/cUSmqBz3crZxFwBVRUnSypa3cQD7JRIx5
         Jmug==
X-Gm-Message-State: AOJu0Yxipyo+mKDX6yZvITdqz5aBTBH8EysBOjuv3FZm0WT0RYbjSiyg
	fNvFFcuxajwSI5sxWRJjtHZNv4W6obWLtmC/UyQ28vqb1gi9aTS2Q3HnNPy+ODI=
X-Google-Smtp-Source: AGHT+IHt95GprgDPx6yopwP2lS1+9+nnhOK4Ujf9Smw7wTQGWLbIihnbbbGJUNUW1TVnwhj6eJXbAQ==
X-Received: by 2002:a7b:cb42:0:b0:41a:2044:1b3e with SMTP id 5b1f17b1804b1-420fd34d850mr40774885e9.32.1716483041434;
        Thu, 23 May 2024 09:50:41 -0700 (PDT)
Date: Thu, 23 May 2024 18:50:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 6/8] xen/lib: Add topology generator for x86
Message-ID: <Zk9z4HJmqXpbivRu@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <1ffad529d7fed10381df67215c747fc2d69f805e.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <1ffad529d7fed10381df67215c747fc2d69f805e.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:25PM +0100, Alejandro Vallejo wrote:
> Add a helper to populate topology leaves in the cpu policy from
> threads/core and cores/package counts.
> 
> No functional change, as it's not connected to anything yet.

There is a functional change in test-cpu-policy.c.

Maybe the commit message needs to be updated to reflect the added
testing to test-cpu-policy.c using the newly introduced helper to
generate topologies?

> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * New patch. Extracted from v1/patch6
> ---
>  tools/tests/cpu-policy/test-cpu-policy.c | 128 +++++++++++++++++++++++
>  xen/include/xen/lib/x86/cpu-policy.h     |  16 +++
>  xen/lib/x86/policy.c                     |  86 +++++++++++++++
>  3 files changed, 230 insertions(+)
> 
> diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
> index 301df2c00285..0ba8c418b1b3 100644
> --- a/tools/tests/cpu-policy/test-cpu-policy.c
> +++ b/tools/tests/cpu-policy/test-cpu-policy.c
> @@ -650,6 +650,132 @@ static void test_is_compatible_failure(void)
>      }
>  }
>  
> +static void test_topo_from_parts(void)
> +{
> +    static const struct test {
> +        unsigned int threads_per_core;
> +        unsigned int cores_per_pkg;
> +        struct cpu_policy policy;
> +    } tests[] = {
> +        {
> +            .threads_per_core = 3, .cores_per_pkg = 1,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_AMD,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
> +                    [1] = { .nr_logical = 1, .level = 1, .type = 2, .id_shift = 2, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 1, .cores_per_pkg = 3,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_AMD,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 7, .cores_per_pkg = 5,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_AMD,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
> +                    [1] = { .nr_logical = 5, .level = 1, .type = 2, .id_shift = 6, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 2, .cores_per_pkg = 128,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_AMD,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
> +                    [1] = { .nr_logical = 128, .level = 1, .type = 2, .id_shift = 8, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 3, .cores_per_pkg = 1,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_INTEL,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 1, .cores_per_pkg = 3,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_INTEL,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 7, .cores_per_pkg = 5,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_INTEL,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
> +                    [1] = { .nr_logical = 35, .level = 1, .type = 2, .id_shift = 6, },
> +                },
> +            },
> +        },
> +        {
> +            .threads_per_core = 2, .cores_per_pkg = 128,
> +            .policy = {
> +                .x86_vendor = X86_VENDOR_INTEL,
> +                .topo.subleaf = {
> +                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
> +                    [1] = { .nr_logical = 256, .level = 1, .type = 2, .id_shift = 8, },

You don't need the array index in the initialization:

                .topo.subleaf = {
                    { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
                    { .nr_logical = 256, .level = 1, .type = 2,
                      .id_shift = 8, },
                }

And lines should be limited to 80 columns if possible.

> +                },
> +            },
> +        },
> +    };
> +
> +    printf("Testing topology synthesis from parts:\n");
> +
> +    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
> +    {
> +        const struct test *t = &tests[i];
> +        struct cpu_policy actual = { .x86_vendor = t->policy.x86_vendor };
> +        int rc = x86_topo_from_parts(&actual, t->threads_per_core, t->cores_per_pkg);
> +
> +        if ( rc || memcmp(&actual.topo, &t->policy.topo, sizeof(actual.topo)) )
> +        {
> +#define TOPO(n) topo.subleaf[(n)]
> +            fail("FAIL[%d] - '%s %u t/c, %u c/p'\n",
> +                 rc,
> +                 x86_cpuid_vendor_to_str(t->policy.x86_vendor),
> +                 t->threads_per_core, t->cores_per_pkg);
> +            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
> +                   "             expected_lvl=%u actual_lvl=%u\n"
> +                   "             expected_type=%u actual_type=%u\n"
> +                   "             expected_shift=%u actual_shift=%u\n",
> +                   0, t->policy.TOPO(0).nr_logical, actual.TOPO(0).nr_logical,
> +                      t->policy.TOPO(0).level,      actual.TOPO(0).level,
> +                      t->policy.TOPO(0).type,       actual.TOPO(0).type,
> +                      t->policy.TOPO(0).id_shift,   actual.TOPO(0).id_shift);
> +
> +            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
> +                   "             expected_lvl=%u actual_lvl=%u\n"
> +                   "             expected_type=%u actual_type=%u\n"
> +                   "             expected_shift=%u actual_shift=%u\n",
> +                   1, t->policy.TOPO(1).nr_logical, actual.TOPO(1).nr_logical,
> +                      t->policy.TOPO(1).level,      actual.TOPO(1).level,
> +                      t->policy.TOPO(1).type,       actual.TOPO(1).type,
> +                      t->policy.TOPO(1).id_shift,   actual.TOPO(1).id_shift);
> +#undef TOPO

Seeing the usage of the macro, maybe you could even do something like:

TOPO(n, f)  t->policy.topo.subleaf[(n)].f, actual.topo.subleaf[(n)].f

This will limit a bit the repetition of the "t->policy..., actual..."
tuple.

> +        }
> +    }
> +}
> +
>  int main(int argc, char **argv)
>  {
>      printf("CPU Policy unit tests\n");
> @@ -667,6 +793,8 @@ int main(int argc, char **argv)
>      test_is_compatible_success();
>      test_is_compatible_failure();
>  
> +    test_topo_from_parts();
> +
>      if ( nr_failures )
>          printf("Done: %u failures\n", nr_failures);
>      else
> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
> index 392320b9adbe..f5df18e9f77c 100644
> --- a/xen/include/xen/lib/x86/cpu-policy.h
> +++ b/xen/include/xen/lib/x86/cpu-policy.h
> @@ -551,6 +551,22 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>   */
>  uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
>  
> +/**
> + * Synthesise topology information in `p` given high-level constraints
> + *
> + * Topology is given in various fields accross several leaves, some of
> + * which are vendor-specific. This function uses the policy itself to
> + * derive such leaves from threads/core and cores/package.
> + *
> + * @param p                   CPU policy of the domain.
> + * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
> + * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
> + * @return                    0 on success; -errno on failure
> + */
> +int x86_topo_from_parts(struct cpu_policy *p,
> +                        unsigned int threads_per_core,
> +                        unsigned int cores_per_pkg);
> +
>  #endif /* !XEN_LIB_X86_POLICIES_H */
>  
>  /*
> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
> index 4cef658feeb8..d033ee5398dd 100644
> --- a/xen/lib/x86/policy.c
> +++ b/xen/lib/x86/policy.c
> @@ -13,6 +13,92 @@ uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>      return vcpu_id * 2;
>  }
>  
> +static unsigned int order(unsigned int n)
> +{
> +    return 8 * sizeof(n) - __builtin_clz(n);

Do we need to assert that n is not 0, otherwise the return of
__builtin_clz() is undefined.

I think the usage below doesn't pass 0 to __builtin_clz() in any case,
but better add the check IMO.

Is __builtin_clz() also available in all versions of GCC and CLANG
that we support?  I have no idea when this was introduced.

> +}
> +
> +int x86_topo_from_parts(struct cpu_policy *p,
> +                        unsigned int threads_per_core,
> +                        unsigned int cores_per_pkg)
> +{
> +    unsigned int threads_per_pkg = threads_per_core * cores_per_pkg;
> +    unsigned int apic_id_size;
> +
> +    if ( !p || !threads_per_core || !cores_per_pkg )
> +        return -EINVAL;
> +
> +    p->basic.max_leaf = MAX(0xb, p->basic.max_leaf);
> +
> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
> +
> +    /* thread level */
> +    p->topo.subleaf[0].nr_logical = threads_per_core;
> +    p->topo.subleaf[0].id_shift = 0;
> +    p->topo.subleaf[0].level = 0;
> +    p->topo.subleaf[0].type = 1;
> +    if ( threads_per_core > 1 )
> +        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
> +
> +    /* core level */
> +    p->topo.subleaf[1].nr_logical = cores_per_pkg;
> +    if ( p->x86_vendor == X86_VENDOR_INTEL )
> +        p->topo.subleaf[1].nr_logical = threads_per_pkg;
> +    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
> +    p->topo.subleaf[1].level = 1;
> +    p->topo.subleaf[1].type = 2;
> +    if ( cores_per_pkg > 1 )
> +        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
> +
> +    apic_id_size = p->topo.subleaf[1].id_shift;
> +
> +    /*
> +     * Contrary to what the name might seem to imply. HTT is an enabler for
> +     * SMP and there's no harm in setting it even with a single vCPU.
> +     */
> +    p->basic.htt = true;
> +    p->basic.lppp = MIN(0xff, p->basic.lppp);
> +
> +    switch ( p->x86_vendor )
> +    {
> +        case X86_VENDOR_INTEL: {
> +            struct cpuid_cache_leaf *sl = p->cache.subleaf;

Newline please.

> +            for ( size_t i = 0; sl->type &&
> +                                i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
> +            {
> +                sl->cores_per_package = cores_per_pkg - 1;
> +                sl->threads_per_cache = threads_per_core - 1;
> +                if ( sl->type == 3 /* unified cache */ )
> +                    sl->threads_per_cache = threads_per_pkg - 1;
> +            }
> +            break;
> +        }

Newline here also.

> +        case X86_VENDOR_AMD:
> +        case X86_VENDOR_HYGON:
> +            /* Expose p->basic.lppp */
> +            p->extd.cmp_legacy = true;
> +
> +            /* Clip NC to the maximum value it can hold */
> +            p->extd.nc = 0xff;
> +            if ( threads_per_pkg <= 0xff )
> +                p->extd.nc = threads_per_pkg - 1;
> +
> +            /* TODO: Expose leaf e1E */
> +            p->extd.topoext = false;
> +
> +            /*
> +             * Clip APIC ID to 8 bits, as that's what high core-count machines do

Overly long line?  And missing full stop.

> +             *
> +             * That what AMD EPYC 9654 does with >256 CPUs
                  ^ That's

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 23 17:36:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 17:36:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728781.1133826 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sACMi-0006dc-AG; Thu, 23 May 2024 17:36:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728781.1133826; Thu, 23 May 2024 17:36:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sACMi-0006dV-75; Thu, 23 May 2024 17:36:32 +0000
Received: by outflank-mailman (input) for mailman id 728781;
 Thu, 23 May 2024 17:36:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VxBl=M2=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sACMh-0006ST-Ib
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 17:36:31 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fcf53a47-192a-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 19:36:29 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51ffff16400so12882838e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 10:36:29 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-521f38d8c38sm5409244e87.211.2024.05.23.10.36.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 10:36:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcf53a47-192a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716485788; x=1717090588; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=d5HekHGljObQIi5oqspdisbrUbBneLRK1/u/IomN9bI=;
        b=Gh2Bdur/vMgZR0CySPWdhpj350U6Q1ghAIRsSNnip+whDFGxwqe+mxd1KDzjNs9WQ1
         Vrlnz/Nza03uRXTZRq9pfypsbSg6eyNsrdcfq4px9Zx28jZeoFmg3z0P/3MasF5V4R5O
         pVcE9oWa70V8IOuit5xr/ZAVX4uVoSdW/JxKej74w9DbXUE5Mk2hFrIFC+dgOW9le4cY
         J9x6beFvc6RXmRLRyBfskJpIjDynegKONujdWwghL+n8bSOtiD8oEgzP1UUR4ngnu1zy
         4yl01/c0TXxIn28RB/T+bp0A+jnAMC569V1EGgDlJ7gwJF5nj91OhEfxx+a90+x2AuUt
         x3fg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716485788; x=1717090588;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=d5HekHGljObQIi5oqspdisbrUbBneLRK1/u/IomN9bI=;
        b=rhKCYwjqWYFUQIdzCs+oNJPbsA+yEwydSh957Y1X3Bw+xG6V3KtRzByNiKL8FPoaTr
         YEXIeZsZrTl8bBRyFaEhJFzJRfETwm0i8qJKpBpkTAjzBNUxPsHTAmimQ4+0Nks2rpal
         4IBgd/i9481w+MeRT1H/24tohCW1bFpasTrr8xpuNZ8BYHPOboyfLJU2wVlRn4kduwIR
         0rC5gdLKHjyVQQMut0qfuf0rRnV7WBdwVGq/YItB9K3Gk/4o7QRxuZyPWfwmAErAGNId
         gddRIbbWOQ2u/O89vM1sKwIxoIX53kfEFBo/tcW0F+Ygg3fjT9dFeQSdf1MijeQgRI2f
         wyJw==
X-Gm-Message-State: AOJu0YyMDq6AZsxUY6T8GmbVa9Hiv55APFCGIIie/2Mg909TJRDm5DUT
	WcXM1D/y8y4JwBjPgO7l+U0L+a98THA3mT1srnWrV+HYc+eVWB4xv3g1PyPA
X-Google-Smtp-Source: AGHT+IEUq0+9wFqxKhc/lmLmCmUAHY/bXm30RVFWAx+aYve5caIrf4gFhTCtMm+zHduPKC9v5clgvQ==
X-Received: by 2002:ac2:554b:0:b0:51d:9e17:29f0 with SMTP id 2adb3069b0e04-526bf07e6b3mr4368830e87.24.1716485788095;
        Thu, 23 May 2024 10:36:28 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	edgar.iglesias@amd.com
Subject: [PATCH v1 1/1] xen/arm: smmuv3: Mark more init-only functions with __init
Date: Wed, 22 May 2024 15:28:29 +0200
Message-Id: <20240522132829.1278625-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240522132829.1278625-1-edgar.iglesias@gmail.com>
References: <20240522132829.1278625-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Move more functions that are only called at init to
the .init.text section.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 6904962467..cee5724022 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -1545,7 +1545,7 @@ static int arm_smmu_dt_xlate(struct device *dev,
 }
 
 /* Probing and initialisation functions */
-static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
+static int __init arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
 				   struct arm_smmu_queue *q,
 				   void __iomem *page,
 				   unsigned long prod_off,
@@ -1588,7 +1588,7 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
 	return 0;
 }
 
-static int arm_smmu_init_queues(struct arm_smmu_device *smmu)
+static int __init arm_smmu_init_queues(struct arm_smmu_device *smmu)
 {
 	int ret;
 
@@ -1724,7 +1724,7 @@ static int arm_smmu_init_strtab(struct arm_smmu_device *smmu)
 	return 0;
 }
 
-static int arm_smmu_init_structures(struct arm_smmu_device *smmu)
+static int __init arm_smmu_init_structures(struct arm_smmu_device *smmu)
 {
 	int ret;
 
@@ -1746,7 +1746,8 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_device *smmu, u32 val,
 }
 
 /* GBPA is "special" */
-static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32 clr)
+static int __init arm_smmu_update_gbpa(struct arm_smmu_device *smmu,
+                                       u32 set, u32 clr)
 {
 	int ret;
 	u32 reg, __iomem *gbpa = smmu->base + ARM_SMMU_GBPA;
@@ -1842,7 +1843,7 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu)
 static inline void arm_smmu_setup_msis(struct arm_smmu_device *smmu) { }
 #endif /* CONFIG_MSI */
 
-static void arm_smmu_free_irqs(struct arm_smmu_device *smmu)
+static void __init arm_smmu_free_irqs(struct arm_smmu_device *smmu)
 {
 	int irq;
 
@@ -1926,7 +1927,7 @@ err_free_evtq_irq:
 	return ret;
 }
 
-static int arm_smmu_setup_irqs(struct arm_smmu_device *smmu)
+static int __init arm_smmu_setup_irqs(struct arm_smmu_device *smmu)
 {
 	int ret, irq;
 	u32 irqen_flags = IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN;
@@ -1988,7 +1989,7 @@ static int arm_smmu_device_disable(struct arm_smmu_device *smmu)
 	return ret;
 }
 
-static int arm_smmu_device_reset(struct arm_smmu_device *smmu)
+static int __init arm_smmu_device_reset(struct arm_smmu_device *smmu)
 {
 	int ret;
 	u32 reg, enables;
@@ -2405,7 +2406,7 @@ static void arm_smmu_free_structures(struct arm_smmu_device *smmu)
 		xfree(smmu->strtab_cfg.l1_desc);
 }
 
-static int arm_smmu_device_probe(struct platform_device *pdev)
+static int __init arm_smmu_device_probe(struct platform_device *pdev)
 {
 	int irq, ret;
 	paddr_t ioaddr, iosize;
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 17:36:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 17:36:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728780.1133815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sACMh-0006PM-2Z; Thu, 23 May 2024 17:36:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728780.1133815; Thu, 23 May 2024 17:36:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sACMh-0006PF-02; Thu, 23 May 2024 17:36:31 +0000
Received: by outflank-mailman (input) for mailman id 728780;
 Thu, 23 May 2024 17:36:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VxBl=M2=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sACMf-0006P9-Gb
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 17:36:29 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc64ac94-192a-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 19:36:28 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e95125e257so11512871fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 10:36:28 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e4d0bbcc28sm42381361fa.11.2024.05.23.10.36.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 May 2024 10:36:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc64ac94-192a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716485787; x=1717090587; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=2nhVxn4m914yqzzG3ZTplvnUhE7IWg6qsPrEng0IE1s=;
        b=QevTmdB9syaU6km507CPmOshg7Zv6VhWw7Lg+SYURVsgUb0QO2/hD4r9ekopOII8nS
         Pz1grsYGodl0dsiWB081GRvsIqNBoJcDRYWOGyGSNsuIKfm+sv/BNMbsbNRlX0DqIeR3
         vgVnXmBGtRrCARQG5Xeaz9+machJ3pKTzWf6Xdhq08cLmCrSa55K5hye/O7aGRiw1zEv
         It8+EWCx/gDQb5D4Nk4dF2uHl8yBj0U//pROk/jo/H+5t7vR7e9xm2rcw3SMH4kGgS6o
         mcbWFe3rRJqU0gD9kSUD+q5CgEU3U/Bbr4Blglycxz9IZITCglJJjzXoHnA4IsNjWzk/
         VexQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716485787; x=1717090587;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=2nhVxn4m914yqzzG3ZTplvnUhE7IWg6qsPrEng0IE1s=;
        b=FM36IHPKpIelMhnsB3qvirE5UhM8SCPutXMbnixmXuCAZfWKmU+nopakRrASOseKiw
         UfQanOQHunI4U+/CSVIhA0rBbYJLJELZ9VXS6VQZ6cqF4WtRTWHgAKhHnR64lMotWBsb
         AbkVU0bsomuSyF8xoM4v4b3wmdUHh2E4O4/DF4khkYEUhHeUsltAD8slTtZUPP28L9ts
         hOBDyzRZRcMvCjt8XQ/+OBkvmNbFc0kYKNRh9AilvKTw32LEOuVr6zkMs1NQZPtWmJ9Z
         csgNVu58ki220Lqevy5GZ6RF3biyTloNqz1eRWSDGpluLBwuyNAWgXejcaEPNa8SP0xJ
         WRTg==
X-Gm-Message-State: AOJu0Yw0pU0PZXsBcRPfpUoFRYKhz4S8w13SnaGMZSI4HjV7jHQzvjTO
	xWhM70XT/JQ0mcwelb6f+r2OytLZ4U862YKLAqNxAhirryAisjn/6R6aUpqV
X-Google-Smtp-Source: AGHT+IEjPh2S+gCXd+sQX9LTo20O76jh2A2wHSH6oaO2/vd+qem/f2LWcrbpGqr+CFZ5BQWRJN9QJA==
X-Received: by 2002:a2e:9611:0:b0:2dd:44b:d63d with SMTP id 38308e7fff4ca-2e951b82c05mr10292831fa.12.1716485787156;
        Thu, 23 May 2024 10:36:27 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	julien@xen.org,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	michal.orzel@amd.com,
	Volodymyr_Babchuk@epam.com,
	edgar.iglesias@amd.com
Subject: [PATCH v1 0/1] xen/arm: smmuv3: Mark more init-only functions with __init
Date: Wed, 22 May 2024 15:28:28 +0200
Message-Id: <20240522132829.1278625-1-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

I was scanning for code that we could potentially move from the
.text section into .init.text and found a few candidates.

I'm not sure if this makes sense, perhaps we don't want to mark
these functions for other reasons but my scripts found this chain
of SMMUv3 init functions as only reachable by .inittext code.
Perhaps it's a little late in the release cycle to consider this...

Best regards,
Edgar


Edgar E. Iglesias (1):
  xen/arm: smmuv3: Mark more init-only functions with __init

 xen/drivers/passthrough/arm/smmu-v3.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Thu May 23 18:29:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 18:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728795.1133836 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADBG-0006LV-2J; Thu, 23 May 2024 18:28:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728795.1133836; Thu, 23 May 2024 18:28:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADBF-0006LO-VT; Thu, 23 May 2024 18:28:45 +0000
Received: by outflank-mailman (input) for mailman id 728795;
 Thu, 23 May 2024 18:28:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GulG=M2=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sADBE-0006LG-EW
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 18:28:44 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47b9feb0-1932-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 20:28:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C949E62E44;
 Thu, 23 May 2024 18:28:40 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F412C3277B;
 Thu, 23 May 2024 18:28:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47b9feb0-1932-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716488920;
	bh=qTP9TCNRUITqhF7/35KayJJ23v85C/p79Jdz5qz3R5Y=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hl/jkFDNjFbixWdXRPwz2a2PUAyqeHYiWtFxwc91Tuzm+nkD9fuN9rgMOIVMuaeIh
	 RZdMf5AW2b6AUEr50iW7n+wwayDe93kHjQkgz4xfKntuViK72jV2Cd3deTVZ59al3g
	 7v3pU3XHF3rAgWOQBkPN+hwPFL3bA73Lq6s2hroHTlVJNKJvZ8l7JgQNE8AOs8fdhM
	 D27TkD74g6Z26gS0vNR5xuyCaqjBK132+VSiG3h2eAh7f6pDIh1q0/mhMCt+M74m0p
	 ZUZgLR0wJ6/VfHCl8vBFfAD80Q0DSNGmDYYvbyXemCIeKLhJtoEY7NWDp1QIeqRhmZ
	 LEOF6AJ582q+A==
Date: Thu, 23 May 2024 11:28:36 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org, 
    Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Christian Lindig <christian.lindig@citrix.com>, 
    David Scott <dave@recoil.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
In-Reply-To: <Zk8qOIIYeOtnNe_0@macbook>
Message-ID: <alpine.DEB.2.22.394.2405231128300.2557291@ubuntu-linux-20-04-desktop>
References: <20240517133352.94347-1-roger.pau@citrix.com> <20240517133352.94347-3-roger.pau@citrix.com> <Zk8qOIIYeOtnNe_0@macbook>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-476071889-1716488920=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-476071889-1716488920=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 23 May 2024, Roger Pau Monné wrote:
> On Fri, May 17, 2024 at 03:33:51PM +0200, Roger Pau Monne wrote:
> > Enabling it using an HVM param is fragile, and complicates the logic when
> > deciding whether options that interact with altp2m can also be enabled.
> > 
> > Leave the HVM param value for consumption by the guest, but prevent it from
> > being set.  Enabling is now done using and additional altp2m specific field in
> > xen_domctl_createdomain.
> > 
> > Note that albeit only currently implemented in x86, altp2m could be implemented
> > in other architectures, hence why the field is added to xen_domctl_createdomain
> > instead of xen_arch_domainconfig.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> > Changes since v2:
> >  - Introduce a new altp2m field in xen_domctl_createdomain.
> > 
> > Changes since v1:
> >  - New in this version.
> > ---
> >  tools/libs/light/libxl_create.c     | 23 ++++++++++++++++++++++-
> >  tools/libs/light/libxl_x86.c        | 26 ++++++++++++--------------
> >  tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
> >  xen/arch/arm/domain.c               |  6 ++++++
> 
> Could I get an Ack from one of the Arm maintainers for the trivial Arm
> change?

Acked-by: Stefano Stabellini <sstabellini@kernel.org>
--8323329-476071889-1716488920=:2557291--


From xen-devel-bounces@lists.xenproject.org Thu May 23 18:30:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 18:30:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728800.1133846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADCq-00086u-DJ; Thu, 23 May 2024 18:30:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728800.1133846; Thu, 23 May 2024 18:30:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADCq-00086n-AT; Thu, 23 May 2024 18:30:24 +0000
Received: by outflank-mailman (input) for mailman id 728800;
 Thu, 23 May 2024 18:30:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GulG=M2=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sADCp-00086e-51
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 18:30:23 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 824c4d86-1932-11ef-b4bb-af5377834399;
 Thu, 23 May 2024 20:30:20 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5628F62E8A;
 Thu, 23 May 2024 18:30:19 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A444CC2BD10;
 Thu, 23 May 2024 18:30:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 824c4d86-1932-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716489019;
	bh=p7mhyN9fs29kwWMHIpMXBirzQoL/bUd/WKeU53shb4E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gU7MeHrdSslK3iJGZ/ZQb59244nQQgre2s1Gpa/dNeIVPOaGd75MV7S6KHHzL/BY4
	 mGjixWgAU7IGPyXvaKuUI2/7NbuVLXT8S79SsMzXTxznYkWFhr9CLB+5dtQo2JcczR
	 vvGdvHGb1H7/eFthqU7pWuK6RZDYJ0AzBUrmas4zToUStPDrhnim+E45QZ9STfQcLx
	 wCPWmkrJtxXgaICm6rIvJ+zVMNtCM4PKwk20rf+SYGx1hFvvu28CPugJWxxvGsw8b0
	 t6jYOmMrGOq/1EDFUqQU3I2NrQiY+IbywSiDZ+OM45r1Vn4NDOOeldOEUEeHHhtA5c
	 kVXf9r9K2T3lA==
Date: Thu, 23 May 2024 11:30:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>, 
    xen-devel@lists.xenproject.org, qemu-devel@nongnu.org, 
    sstabellini@kernel.org, jgross@suse.com, 
    "Edgar E. Iglesias" <edgar.iglesias@amd.com>, 
    Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v6 7/8] xen: mapcache: Add support for grant mappings
In-Reply-To: <CAJy5ezqcnoQ=M4H3RkQ69HAie4jNMEQB42f4NSRZUEWU_5=Gkw@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2405231129590.2557291@ubuntu-linux-20-04-desktop>
References: <20240516154804.1114245-1-edgar.iglesias@gmail.com> <20240516154804.1114245-8-edgar.iglesias@gmail.com> <dxhms.qrh0wkni3yz@linaro.org> <CAJy5ezqcnoQ=M4H3RkQ69HAie4jNMEQB42f4NSRZUEWU_5=Gkw@mail.gmail.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1340495454-1716489019=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1340495454-1716489019=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 23 May 2024, Edgar E. Iglesias wrote:
> On Thu, May 23, 2024 at 9:47 AM Manos Pitsidianakis <manos.pitsidianakis@linaro.org> wrote:
>       On Thu, 16 May 2024 18:48, "Edgar E. Iglesias" <edgar.iglesias@gmail.com> wrote:
>       >From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
>       >
>       >Add a second mapcache for grant mappings. The mapcache for
>       >grants needs to work with XC_PAGE_SIZE granularity since
>       >we can't map larger ranges than what has been granted to us.
>       >
>       >Like with foreign mappings (xen_memory), machines using grants
>       >are expected to initialize the xen_grants MR and map it
>       >into their address-map accordingly.
>       >
>       >Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
>       >Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>       >---
>       > hw/xen/xen-hvm-common.c         |  12 ++-
>       > hw/xen/xen-mapcache.c           | 163 ++++++++++++++++++++++++++------
>       > include/hw/xen/xen-hvm-common.h |   3 +
>       > include/sysemu/xen.h            |   7 ++
>       > 4 files changed, 152 insertions(+), 33 deletions(-)
>       >
>       >diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
>       >index a0a0252da0..b8ace1c368 100644
>       >--- a/hw/xen/xen-hvm-common.c
>       >+++ b/hw/xen/xen-hvm-common.c
>       >@@ -10,12 +10,18 @@
>       > #include "hw/boards.h"
>       > #include "hw/xen/arch_hvm.h"
>       >
>       >-MemoryRegion xen_memory;
>       >+MemoryRegion xen_memory, xen_grants;
>       >
>       >-/* Check for xen memory.  */
>       >+/* Check for any kind of xen memory, foreign mappings or grants.  */
>       > bool xen_mr_is_memory(MemoryRegion *mr)
>       > {
>       >-    return mr == &xen_memory;
>       >+    return mr == &xen_memory || mr == &xen_grants;
>       >+}
>       >+
>       >+/* Check specifically for grants.  */
>       >+bool xen_mr_is_grants(MemoryRegion *mr)
>       >+{
>       >+    return mr == &xen_grants;
>       > }
>       >
>       > void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
>       >diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
>       >index a07c47b0b1..1cbc2aeaa9 100644
>       >--- a/hw/xen/xen-mapcache.c
>       >+++ b/hw/xen/xen-mapcache.c
>       >@@ -14,6 +14,7 @@
>       >
>       > #include <sys/resource.h>
>       >
>       >+#include "hw/xen/xen-hvm-common.h"
>       > #include "hw/xen/xen_native.h"
>       > #include "qemu/bitmap.h"
>       >
>       >@@ -21,6 +22,8 @@
>       > #include "sysemu/xen-mapcache.h"
>       > #include "trace.h"
>       >
>       >+#include <xenevtchn.h>
>       >+#include <xengnttab.h>
>       >
>       > #if HOST_LONG_BITS == 32
>       > #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
>       >@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
>       >     unsigned long *valid_mapping;
>       >     uint32_t lock;
>       > #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
>       >+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
> 
>       Might we get more entry kinds in the future? (for example foreign maps).
>       Maybe this could be an enum.
> 
> 
> Perhaps. Foreign mappings are already supported, this flag separates ordinary foreign mappings from grant foreign mappings.
> IMO, since this is not an external interface it's probably better to change it once we have a concrete use-case at hand.
> 
>  
>       >     uint8_t flags;
>       >     hwaddr size;
>       >     struct MapCacheEntry *next;
>       >@@ -71,6 +75,8 @@ typedef struct MapCache {
>       > } MapCache;
>       >
>       > static MapCache *mapcache;
>       >+static MapCache *mapcache_grants;
>       >+static xengnttab_handle *xen_region_gnttabdev;
>       >
>       > static inline void mapcache_lock(MapCache *mc)
>       > {
>       >@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>       >     unsigned long max_mcache_size;
>       >     unsigned int bucket_shift;
>       >
>       >+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
>       >+    if (xen_region_gnttabdev == NULL) {
>       >+        error_report("mapcache: Failed to open gnttab device");
>       >+        exit(EXIT_FAILURE);
>       >+    }
>       >+
>       >     if (HOST_LONG_BITS == 32) {
>       >         bucket_shift = 16;
>       >     } else {
>       >@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
>       >     mapcache = xen_map_cache_init_single(f, opaque,
>       >                                          bucket_shift,
>       >                                          max_mcache_size);
>       >+
>       >+    /*
>       >+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
>       >+     * map anything beyond the number of pages granted to us.
>       >+     */
>       >+    mapcache_grants = xen_map_cache_init_single(f, opaque,
>       >+                                                XC_PAGE_SHIFT,
>       >+                                                max_mcache_size);
>       >+
>       >     setrlimit(RLIMIT_AS, &rlimit_as);
>       > }
>       >
>       >@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
>       >                              hwaddr size,
>       >                              hwaddr address_index,
>       >                              bool dummy,
>       >+                             bool grant,
>       >+                             bool is_write,
>       >                              ram_addr_t ram_offset)
>       > {
>       >     uint8_t *vaddr_base;
>       >-    xen_pfn_t *pfns;
>       >+    uint32_t *refs = NULL;
>       >+    xen_pfn_t *pfns = NULL;
>       >     int *err;
> 
>       You should use g_autofree to perform automatic cleanup on function exit
>       instead of manually freeing, since the allocations should only live
>       within the function call.
> 
> 
> Sounds good, I'll do that in the next version.
> 
>  
>       >     unsigned int i;
>       >     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
>       >
>       >     trace_xen_remap_bucket(address_index);
>       >
>       >-    pfns = g_new0(xen_pfn_t, nb_pfn);
>       >+    if (grant) {
>       >+        refs = g_new0(uint32_t, nb_pfn);
>       >+    } else {
>       >+        pfns = g_new0(xen_pfn_t, nb_pfn);
>       >+    }
>       >     err = g_new0(int, nb_pfn);
>       >
>       >     if (entry->vaddr_base != NULL) {
>       >@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
>       >     g_free(entry->valid_mapping);
>       >     entry->valid_mapping = NULL;
>       >
>       >-    for (i = 0; i < nb_pfn; i++) {
>       >-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
>       >+    if (grant) {
>       >+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
>       >+
>       >+        for (i = 0; i < nb_pfn; i++) {
>       >+            refs[i] = grant_base + i;
>       >+        }
>       >+    } else {
>       >+        for (i = 0; i < nb_pfn; i++) {
>       >+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
>       >+        }
>       >     }
>       >
>       >-    /*
>       >-     * If the caller has requested the mapping at a specific address use
>       >-     * MAP_FIXED to make sure it's honored.
>       >-     */
>       >+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
>       >+
>       >     if (!dummy) {
>       >-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
>       >-                                           PROT_READ | PROT_WRITE,
>       >-                                           vaddr ? MAP_FIXED : 0,
>       >-                                           nb_pfn, pfns, err);
> 
>       Since err is not NULL here, the function might return a valid pointer
>       but individual frames might have failed.
> 
> 
> Yes but AFAICT, the case when some pages fail foreign mapping is handled further down the function (see the valid_mappings bitmap).
> Note that this series isn't really changing this existing behaviour for foreign mappings. In any case, If we spot a bug in existing code,
> I'm happy to fix it.
> 
>  
> 
>       >+        if (grant) {
>       >+            int prot = PROT_READ;
>       >+
>       >+            if (is_write) {
>       >+                prot |= PROT_WRITE;
>       >+            }
>       >+
>       >+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
>       >+            assert(vaddr == NULL);
>       >+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
>       >+                                                         nb_pfn,
>       >+                                                         xen_domid, refs,
>       >+                                                         prot);
>       >+        } else {
>       >+            /*
>       >+             * If the caller has requested the mapping at a specific address use
>       >+             * MAP_FIXED to make sure it's honored.
>       >+             *
>       >+             * We don't yet support upgrading mappings from RO to RW, to handle
>       >+             * models using ordinary address_space_rw(), foreign mappings ignore
>       >+             * is_write and are always mapped RW.
>       >+             */
>       >+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
>       >+                                               PROT_READ | PROT_WRITE,
>       >+                                               vaddr ? MAP_FIXED : 0,
>       >+                                               nb_pfn, pfns, err);
>       >+        }
>       >         if (vaddr_base == NULL) {
>       >-            perror("xenforeignmemory_map2");
>       >+            perror(grant ? "xengnttab_map_domain_grant_refs"
>       >+                           : "xenforeignmemory_map2");
>       >             exit(-1);
>       >         }
>       >     } else {
>       >@@ -261,6 +319,7 @@ static void xen_remap_bucket(MapCache *mc,
>       >         }
>       >     }
>       >
>       >+    g_free(refs);
>       >     g_free(pfns);
>       >     g_free(err);
>       > }
>       >@@ -268,7 +327,8 @@ static void xen_remap_bucket(MapCache *mc,
>       > static uint8_t *xen_map_cache_unlocked(MapCache *mc,
>       >                                        hwaddr phys_addr, hwaddr size,
>       >                                        ram_addr_t ram_offset,
>       >-                                       uint8_t lock, bool dma, bool is_write)
>       >+                                       uint8_t lock, bool dma,
>       >+                                       bool grant, bool is_write)
>       > {
>       >     MapCacheEntry *entry, *pentry = NULL,
>       >                   *free_entry = NULL, *free_pentry = NULL;
>       >@@ -340,7 +400,7 @@ tryagain:
>       >         entry = g_new0(MapCacheEntry, 1);
>       >         pentry->next = entry;
>       >         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
>       >-                         ram_offset);
>       >+                         grant, is_write, ram_offset);
>       >     } else if (!entry->lock) {
>       >         if (!entry->vaddr_base || entry->paddr_index != address_index ||
>       >                 entry->size != cache_size ||
>       >@@ -348,7 +408,7 @@ tryagain:
>       >                     test_bit_size >> XC_PAGE_SHIFT,
>       >                     entry->valid_mapping)) {
>       >             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
>       >-                             ram_offset);
>       >+                             grant, is_write, ram_offset);
>       >         }
>       >     }
>       >
>       >@@ -399,12 +459,28 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
>       >                        uint8_t lock, bool dma,
>       >                        bool is_write)
>       > {
>       >+    bool grant = xen_mr_is_grants(mr);
>       >+    MapCache *mc = grant ? mapcache_grants : mapcache;
>       >     uint8_t *p;
>       >
>       >-    mapcache_lock(mapcache);
>       >-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
>       >-                               lock, dma, is_write);
>       >-    mapcache_unlock(mapcache);
>       >+    if (grant) {
>       >+        /*
>       >+         * Grants are only supported via address_space_map(). Anything
>       >+         * else is considered a user/guest error.
>       >+         *
>       >+         * QEMU generally doesn't expect these mappings to ever fail, so
>       >+         * if this happens we report an error message and abort().
>       >+         */
>       >+        if (!lock) {
> 
>       Nested if conditions that can be flattened, i.e. this could be
> 
>       if (grant && !lock)
> 
> 
> 
> Sounds good, will flatten this in the next version.
>  
> 
>       >+            error_report("Trying access a grant reference without mapping it.");
> 
>       s/Trying access a grant/Tried to access a grant/
> 
> 
> Will fix it, thanks!

Please retain my ack
--8323329-1340495454-1716489019=:2557291--


From xen-devel-bounces@lists.xenproject.org Thu May 23 18:59:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 18:59:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728808.1133855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADeY-00035C-N4; Thu, 23 May 2024 18:59:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728808.1133855; Thu, 23 May 2024 18:59:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADeY-000355-KJ; Thu, 23 May 2024 18:59:02 +0000
Received: by outflank-mailman (input) for mailman id 728808;
 Thu, 23 May 2024 18:59:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSi9=M2=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sADeX-00034z-K5
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 18:59:01 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 836192ff-1936-11ef-90a1-e314d9c70b13;
 Thu, 23 May 2024 20:58:59 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-351b683f2d8so1998884f8f.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 11:58:59 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a5a17b17733sm1972053366b.195.2024.05.23.11.58.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 11:58:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 836192ff-1936-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716490739; x=1717095539; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=+0WPKKi8leSuP+9MJGa6rPyPqrwEy/yzHtx7N8uEAMU=;
        b=QclwoxJcL2qIDHq7A9CpNfdOgJFHbVpC1sPqmvou6XK4qFgy/Z77xCjLBWalaxe4zI
         0FqwNIRB2bJcr0nGxgEVZEUEOVsdMnuPDbDKd4374MjpbSQIwq36VNAdAmZVqi2rMpqL
         ZspzoNepPJY0GZlj3W1Le+RBWvJ/OP0Hs4Oi0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716490739; x=1717095539;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+0WPKKi8leSuP+9MJGa6rPyPqrwEy/yzHtx7N8uEAMU=;
        b=sLtNlg03gkKHJJ3wn8gdQXf5gEVqi9KXsz2PVaKxpX0lHiqm+bOWttRnBwBFbw6Wmn
         Ak8SB/g8OI9lDkJSYdx4moSSq0q0G3rYJbrxlaWXfyzuwIKDZ9Y/6WkftFbRU8vAb2Yq
         LXH8KkVxlW8g9+VdIWG0BVMYvjTyGwyDo/mpKy098C/vWATcgBu/qEZEm2MhoUNPr1sw
         hp0/L+F85HMS6At9Ic1jFqPeuzgpHZ7ZXr5jjgCtaNCi0z+K6C+3MsWYGhkyXq3QMC1E
         +gm7Dqt+4TWeBTC4uc/yik/y7rOsrEaojjh7O1ZDY3konsf/rTRjNF5CPYSopEex3K6Q
         xTyg==
X-Forwarded-Encrypted: i=1; AJvYcCVi1m8KEJKh/+2DtdpDUOcTOqNbzBTcRbr3FsN4CckYti8Sdt5WENfkT6d/qJLsE8QwbmcFBIFuYJgTsMdRNSssfCXXy64MAtCaOgirN50=
X-Gm-Message-State: AOJu0YyPgbjVj1gWf7MmASeSUJebG1Dh+kYkHrjrjQlwoqS538rZp7yF
	ZIiG88XoyhtD2Rleadh1VzQTSCcApRvG13QGRrCEy7fYrqQsO68TwXGpbQowcTg=
X-Google-Smtp-Source: AGHT+IFkw03B9KSl6QVrxPN9WW1pInnSELIjhi22SpzZs6YXbTg6sT/8dTfH223+MYiKI2jQ2ByfWg==
X-Received: by 2002:a05:6000:25a:b0:354:f5f4:579 with SMTP id ffacd0b85a97d-354f5f406aemr2302618f8f.0.1716490739064;
        Thu, 23 May 2024 11:58:59 -0700 (PDT)
Message-ID: <630e0117-f401-4d55-a1de-2682d8d9b406@citrix.com>
Date: Thu, 23 May 2024 19:58:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration
 checks to the check hook
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/05/2024 1:39 pm, Alejandro Vallejo wrote:
> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> index 8a244100009c..2f06bff1b2cc 100644
> --- a/xen/arch/x86/hvm/vlapic.c
> +++ b/xen/arch/x86/hvm/vlapic.c
> @@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
>  }
>  
> -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> +static int cf_check lapic_check_hidden(const struct domain *d,
> +                                       hvm_domain_context_t *h)
>  {
>      unsigned int vcpuid = hvm_load_instance(h);
> -    struct vcpu *v;
> -    struct vlapic *s;
> +    struct hvm_hw_lapic s;
>  
>      if ( !has_vlapic(d) )
>          return -ENODEV;
>  
>      /* Which vlapic to load? */
> -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
> +    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )

As you're editing this anyway, swap for

    if ( !domain_vcpu(d, vcpuid) )

please.

>      {
>          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
>                  d->domain_id, vcpuid);
>          return -EINVAL;
>      }
> -    s = vcpu_vlapic(v);
>  
> -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )
> +        return -ENODATA;
> +
> +    /* EN=0 with EXTD=1 is illegal */
> +    if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) ==
> +         APIC_BASE_EXTD )
> +        return -EINVAL;

This is very insufficient auditing for the incoming value, but it turns
out that there's no nice logic for this at all.

As it's just a less obfuscated form of the logic from
lapic_load_hidden(), it's probably fine to stay as it is for now.

The major changes since this logic was written originally are that the
CPU policy correct (so we can reject EXTD on VMs which can't see
x2apic), and that we now prohibit VMs moving the xAPIC MMIO window away
from its default location (as this would require per-vCPU P2Ms in order
to virtualise properly.)

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 23 19:09:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 19:09:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728816.1133866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADoM-00051e-L4; Thu, 23 May 2024 19:09:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728816.1133866; Thu, 23 May 2024 19:09:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sADoM-00051X-IM; Thu, 23 May 2024 19:09:10 +0000
Received: by outflank-mailman (input) for mailman id 728816;
 Thu, 23 May 2024 19:09:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sADoL-00051N-HS; Thu, 23 May 2024 19:09:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sADoL-0004HB-FY; Thu, 23 May 2024 19:09:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sADoL-0007MK-2x; Thu, 23 May 2024 19:09:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sADoL-0004as-2S; Thu, 23 May 2024 19:09:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Z6Bo/sZsWoyQzWCnu+E1+wIJtYb4PHGcwzVPnGlLTvE=; b=7FO1lyzIPwMfXNzf/lqCIUFA2i
	Om7aSxITYpUNQdRPVYKJh95w6ed9/FPI2WEOYIDZjzDkh1dgBDGmsKX/OH2w4bK7czWROpJRLh6D5
	ZaqbK9lq03EXz97diSB0r/RjSZf4Qf1aU5baRbf5w4AXclKVOGM47+g6qZylTfs+ZwSY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186099-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186099: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-xsm:xen-build:fail:regression
    libvirt:build-amd64:xen-build:fail:regression
    libvirt:build-i386-xsm:xen-build:fail:regression
    libvirt:build-i386:xen-build:fail:regression
    libvirt:build-armhf:xen-build:fail:regression
    libvirt:build-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:build-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:build-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=66b052263d6ff046c60f4fce263e07c0d9bdd059
X-Osstest-Versions-That:
    libvirt=7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 19:09:09 +0000

flight 186099 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186099/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 186070
 build-amd64                   6 xen-build                fail REGR. vs. 186070
 build-i386-xsm                6 xen-build                fail REGR. vs. 186070
 build-i386                    6 xen-build                fail REGR. vs. 186070
 build-armhf                   6 xen-build                fail REGR. vs. 186070

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass

version targeted for testing:
 libvirt              66b052263d6ff046c60f4fce263e07c0d9bdd059
baseline version:
 libvirt              7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129

Last test of basis   186070  2024-05-22 04:20:52 Z    1 days
Testing same since   186099  2024-05-23 04:18:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Privoznik <mprivozn@redhat.com>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-amd64-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 blocked 
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 66b052263d6ff046c60f4fce263e07c0d9bdd059
Author: Michal Privoznik <mprivozn@redhat.com>
Date:   Wed May 22 09:31:50 2024 +0200

    src: Fix return types of .stateInitialize callbacks
    
    The virStateDriver struct has .stateInitialize callback which is
    declared to return virDrvStateInitResult enum. But some drivers
    return a plain int in their implementation which is UB.
    
    Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
    Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


From xen-devel-bounces@lists.xenproject.org Thu May 23 20:49:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 20:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728843.1133898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFMj-0000XO-Ov; Thu, 23 May 2024 20:48:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728843.1133898; Thu, 23 May 2024 20:48:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFMj-0000XH-Kf; Thu, 23 May 2024 20:48:45 +0000
Received: by outflank-mailman (input) for mailman id 728843;
 Thu, 23 May 2024 20:48:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAFMi-0000X5-E2; Thu, 23 May 2024 20:48:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAFMi-0005w0-AL; Thu, 23 May 2024 20:48:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAFMi-0001IH-2g; Thu, 23 May 2024 20:48:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAFMi-0001Et-25; Thu, 23 May 2024 20:48:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5K72Ej/Rh/byTPpKrHbt0dKsHNBEW4XEUR4xxSxfNuU=; b=vDt4RcwZQzvwMs9Qjt6AVcTJeK
	33CalAAt+Jam7OJEq8nN+SW+pcVVK8EMmhGwXZqUYWhL61cS+l/3N7isFo8xkrpE2Uo+aihbN2ieV
	S/bNY3lqbA2/zoMq3yXNt1fpwEAuOrF5zGtR0utPa4wHUcBSVbiKzrN3FwEqyosCrrMc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186117-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186117: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2a40b106e92aaa7ce808c8608dd6473edc67f608
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 20:48:44 +0000

flight 186117 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186117/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2a40b106e92aaa7ce808c8608dd6473edc67f608
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186064  2024-05-21 15:04:02 Z    2 days
Failing since        186104  2024-05-23 09:00:22 Z    0 days    4 attempts
Testing same since   186117  2024-05-23 17:02:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bobby Eshleman <bobbyeshleman@gmail.com>
  Christian Lindig <christian.lindig@cloud.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Olaf Hering <olaf@aepfle.de>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Tamas K Lengyel <tamas@tklengyel.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ced21fbb28..2a40b106e9  2a40b106e92aaa7ce808c8608dd6473edc67f608 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 23 21:24:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 21:24:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728864.1133913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFuo-0005CI-BG; Thu, 23 May 2024 21:23:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728864.1133913; Thu, 23 May 2024 21:23:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFuo-0005CB-8E; Thu, 23 May 2024 21:23:58 +0000
Received: by outflank-mailman (input) for mailman id 728864;
 Thu, 23 May 2024 21:23:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAFun-0005C5-8k
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 21:23:57 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAFun-0006VZ-0W; Thu, 23 May 2024 21:23:57 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAFum-0004JX-Q2; Thu, 23 May 2024 21:23:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=N6kjwu2fTaT6KkSOu1pbvEUSSozPo48Ld5dSPHla/Tw=; b=wKs6pIxCJsQXouhcBlVuzZlwWL
	7JjYKSFpmUXxLRel54Em2PK2Jty+rkQMXD2lEDGnac40QVAaLbrK124yHSX0fi8DFx791pF9N39Bx
	fbt4Et7+JK8wzSiwGFMkIBIAc3XC3N8nYjG9yyDnGvoaUhPO7O1psN3l2nvk7BhsRs2E=;
Message-ID: <55b10b42-44ae-473e-8b85-95e9f8e03413@xen.org>
Date: Thu, 23 May 2024 22:23:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 2/9] xen/arm, doc: Add a DT property to specify IOMMU
 for Dom0less domUs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-3-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-3-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> There are some use cases in which the dom0less domUs need to have
> the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
> example, the dynamic dtbo feature allows the domain to be assigned
> a device that is behind the IOMMU at runtime. For these use cases,
> we need to have a way to specify the domain will need the IOMMU
> mapping at domain construction time.
> 
> Introduce a "passthrough" DT property for Dom0less DomUs following
> the same entry as the xl.cfg. Currently only provide two options,
> i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
> construction time based on the property.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 21:27:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 21:27:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728874.1133928 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFy8-0005ns-RQ; Thu, 23 May 2024 21:27:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728874.1133928; Thu, 23 May 2024 21:27:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAFy8-0005nl-P0; Thu, 23 May 2024 21:27:24 +0000
Received: by outflank-mailman (input) for mailman id 728874;
 Thu, 23 May 2024 21:27:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAFy8-0005nf-7b
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 21:27:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAFy7-0006Zg-Tq; Thu, 23 May 2024 21:27:23 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAFy7-0004QV-PD; Thu, 23 May 2024 21:27:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=rCmAIsR+oj6kcNc/BpUQyQ7KNqfoJU5JzPpseoOvmN0=; b=mqTfmnhFOQemzV37MAwikd4Zkd
	ULMtfKVzYA7eD6Dv1LlLd+8NwCf+SRcdGGCCwjCIfceSZhpMRdof8tA/dNBBzrUvQc9hazUqqiW1G
	UnSPuZ1ed2Mz1HfBPd01abzhCP6aHz7Qll++wYw/zVPZEa+kl9ygRvbjqiu9SgX7oQec=;
Message-ID: <94652928-866e-4492-bcd0-62a20ec795ce@xen.org>
Date: Thu, 23 May 2024 22:27:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 4/9] xen/arm/gic: Allow adding interrupt to running VMs
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-5-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-5-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> Currently, adding physical interrupts are only allowed at
> the domain creation time. For use cases such as dynamic device
> tree overlay addition, the adding of physical IRQ to
> running domains should be allowed.
> 
> Drop the above-mentioned domain creation check. Since this
> will introduce interrupt state unsync issues for cases when the
> interrupt is active or pending in the guest, therefore for these
> cases we simply reject the operation. Do it for both new and old
> vGIC implementations.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

With one remark below:

Reviewed-by: Julien Grall <jgrall@amazon.com>

> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index b9463a5f27..048e12c562 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
>   
>       if ( connect )                      /* assign a mapped IRQ */
>       {
> -        /* The VIRQ should not be already enabled by the guest */
> -        if ( !irq->hw && !irq->enabled )
> +        /*
> +         * The VIRQ should not be already enabled by the guest nor
> +         * active/pending in the guest

Typo: Missing full stop.

It can be fixed on commit.

> +         */
> +        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
>           {
>               irq->hw = true;
>               irq->hwintid = desc->irq;

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 21:29:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 21:29:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728879.1133939 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAG04-0006OH-9e; Thu, 23 May 2024 21:29:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728879.1133939; Thu, 23 May 2024 21:29:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAG04-0006OA-6t; Thu, 23 May 2024 21:29:24 +0000
Received: by outflank-mailman (input) for mailman id 728879;
 Thu, 23 May 2024 21:29:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAG02-0006Nx-HV
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 21:29:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAG02-0006bP-5M; Thu, 23 May 2024 21:29:22 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAG01-0004SE-Vp; Thu, 23 May 2024 21:29:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=KAxC7cfXsEysTexnM0Q5OFJjNNM5aUTWf2jS9ID6sIA=; b=zdbToxkgSSuBNUVdLiTp55OyVf
	dTmJ62t+QhjC8v3vkE2OyalvGkLPSjcNCesoNLXuBnYK2n6rQE4mM9TDwDimC/oc7G7LfeS7Ks243
	urMLOZsnJksaGpctD3hi36jc7omaekMYqDWHvrqfgijGskLCafzbuZ4TEUPsXxE/w2eY=;
Message-ID: <d76cb59b-012d-4d3a-99e6-69e151f81951@xen.org>
Date: Thu, 23 May 2024 22:29:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 0/9] Remaining patches for dynamic node programming
 using overlay dtbo
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-1-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 23/05/2024 08:40, Henry Wang wrote:
> Hi all,

Hi Henry,

> This is the remaining series for the full functional "dynamic node
> programming using overlay dtbo" feature. The first part [1] has
> already been merged.
> 
> Quoting from the original series, the first part has already made
> Xen aware of new device tree node which means updating the dt_host
> with overlay node information, and in this series, the goal is to
> map IRQ and IOMMU during runtime, where we will do the actual IOMMU
> and IRQ mapping and unmapping to a running domain. Also, documentation
> of the "dynamic node programming using overlay dtbo" feature is added.
> 
> During the discussion in v3, I was recommended to split the overlay
> devices attach/detach to/from running domains to separated patches [3].
> But I decided to only expose the xl user interfaces together to the
> users after device attach/detach is fully functional, so I didn't
> split the toolstack patch (#8).

So I was asking to split so we can get some of the work merged for 4.19. 
Can you clarify, whether the intention is to merge only patches #1-5?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 21:55:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 21:55:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728888.1133955 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAGP6-00022t-9X; Thu, 23 May 2024 21:55:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728888.1133955; Thu, 23 May 2024 21:55:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAGP6-00022m-6P; Thu, 23 May 2024 21:55:16 +0000
Received: by outflank-mailman (input) for mailman id 728888;
 Thu, 23 May 2024 21:55:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAGP4-00022g-Pb
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 21:55:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAGP4-00071t-C4; Thu, 23 May 2024 21:55:14 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAGP4-00062r-1i; Thu, 23 May 2024 21:55:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=LrvouJBRSenBDORUTSo8DkVLhcr/S1bCKVT07dT2whk=; b=yWpyGSOKBHWTkZ2q3xaGkPMAWl
	iRX8+Pxm/0EZNdb2y+KHV9B6g4x9nk46vOS87MigZBujOX2MVcQI+PvVsulDBaYckvJ6ssBeBBm9u
	d65aG7pne+rELyETajAihKsFVWM6sPnkk+IbdqtYiB0TsuX5zP8OsmBA2/1ZICd596Bo=;
Message-ID: <311c645f-d0cf-45ef-abb3-c7d8a58f1130@xen.org>
Date: Thu, 23 May 2024 22:55:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 5/9] xen/arm: Add XEN_DOMCTL_dt_overlay and device
 attachment to domains
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-6-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-6-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> In order to support the dynamic dtbo device assignment to a running
> VM, the add/remove of the DT overlay and the attach/detach of the
> device from the DT overlay should happen separately. Therefore,
> repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
> overlay to Xen device tree

I think it would be worth mentioning in the commit message why changing 
the sysctl behavior is fine. The feature is experimental and therefore 
breaking compatibility is ok.

> , instead of assigning the device to the
> hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with
> operations XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment
> to the domain.
> 
> The hypervisor firstly checks the DT overlay passed from the toolstack
> is valid. Then the device nodes are retrieved from the overlay tracker
> based on the DT overlay. The attach of the device is implemented by
> mapping the IRQ and IOMMU resources.

So, the expectation is the user will always want to attach all the 
devices in the overlay to a single domain. Is that correct?

> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> ---
> v4:
> - Split the original patch, only do the device attachment.
> v3:
> - Style fixes for arch-selection #ifdefs.
> - Do not include public/domctl.h, only add a forward declaration of
>    struct xen_domctl_dt_overlay.
> - Extract the overlay track entry finding logic to a function, drop
>    the unused variables.
> - Use op code 1&2 for XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH}.
> v2:
> - New patch.
> ---
>   xen/arch/arm/domctl.c        |   3 +
>   xen/common/dt-overlay.c      | 199 ++++++++++++++++++++++++++---------
>   xen/include/public/domctl.h  |  14 +++
>   xen/include/public/sysctl.h  |  11 +-
>   xen/include/xen/dt-overlay.h |   7 ++
>   5 files changed, 176 insertions(+), 58 deletions(-)
> 
> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> index ad56efb0f5..12a12ee781 100644
> --- a/xen/arch/arm/domctl.c
> +++ b/xen/arch/arm/domctl.c
> @@ -5,6 +5,7 @@
>    * Copyright (c) 2012, Citrix Systems
>    */
>   
> +#include <xen/dt-overlay.h>
>   #include <xen/errno.h>
>   #include <xen/guest_access.h>
>   #include <xen/hypercall.h>
> @@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
>   
>           return rc;
>       }
> +    case XEN_DOMCTL_dt_overlay:
> +        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
>       default:
>           return subarch_do_domctl(domctl, d, u_domctl);
>       }
> diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
> index 9cece79067..1087f9b502 100644
> --- a/xen/common/dt-overlay.c
> +++ b/xen/common/dt-overlay.c
> @@ -356,6 +356,42 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
>       return 0;
>   }
>   
> +/* This function should be called with the overlay_lock taken */
> +static struct overlay_track *
> +find_track_entry_from_tracker(const void *overlay_fdt,
> +                              uint32_t overlay_fdt_size)
> +{
> +    struct overlay_track *entry, *temp;
> +    bool found_entry = false;
> +
> +    ASSERT(spin_is_locked(&overlay_lock));
> +
> +    /*
> +     * First check if dtbo is correct i.e. it should one of the dtbo which was
> +     * used when dynamically adding the node.
> +     * Limitation: Cases with same node names but different property are not
> +     * supported currently. We are relying on user to provide the same dtbo
> +     * as it was used when adding the nodes.
> +     */
> +    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> +    {
> +        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
> +        {
> +            found_entry = true;
> +            break;
> +        }
> +    }
> +
> +    if ( !found_entry )
> +    {
> +        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
> +               " Operation is supported only for prior added dtbo.\n");
> +        return NULL;
> +    }
> +
> +    return entry;
> +}
> +
>   /* Check if node itself can be removed and remove node from IOMMU. */
>   static int remove_node_resources(struct dt_device_node *device_node)
>   {
> @@ -485,8 +521,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
>                                           uint32_t overlay_fdt_size)
>   {
>       int rc;
> -    struct overlay_track *entry, *temp, *track;
> -    bool found_entry = false;
> +    struct overlay_track *entry;
>   
>       rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
>       if ( rc )
> @@ -494,29 +529,10 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
>   
>       spin_lock(&overlay_lock);
>   
> -    /*
> -     * First check if dtbo is correct i.e. it should one of the dtbo which was
> -     * used when dynamically adding the node.
> -     * Limitation: Cases with same node names but different property are not
> -     * supported currently. We are relying on user to provide the same dtbo
> -     * as it was used when adding the nodes.
> -     */
> -    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> -    {
> -        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
> -        {
> -            track = entry;
> -            found_entry = true;
> -            break;
> -        }
> -    }
> -
> -    if ( !found_entry )
> +    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
> +    if ( entry == NULL )
>       {
>           rc = -EINVAL;
> -
> -        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
> -               " Removing nodes is supported only for prior added dtbo.\n");
>           goto out;
>   
>       }
> @@ -620,15 +636,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
>               return -EFAULT;
>           }
>   
> -        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
> -                           tr->iomem_ranges,
> -                           tr->irq_ranges);
>           write_unlock(&dt_host_lock);
> -        if ( rc )
> -        {
> -            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> -            return rc;
> -        }
>   
>           /* Keep overlay_node address in tracker. */
>           tr->nodes_address[j] = (unsigned long)overlay_node;
> @@ -638,9 +646,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
>   }
>   /*
>    * Adds device tree nodes under target node.
> - * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
> - * is done to avoid the removal of device_tree generation, iomem regions mapping
> - * to hardware domain done by handle_node().
> + * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
>    */
>   static long handle_add_overlay_nodes(void *overlay_fdt,
>                                        uint32_t overlay_fdt_size)
> @@ -774,20 +780,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
>           goto err;
>       }
>   
> -    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
> -    if (tr->irq_ranges == NULL)
> -    {
> -        printk(XENLOG_ERR "Creating IRQ rangeset failed");
> -        goto err;
> -    }
> -
> -    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
> -    if (tr->iomem_ranges == NULL)
> -    {
> -        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
> -        goto err;
> -    }
> -
>       rc = add_nodes(tr, nodes_full_path);
>       if ( rc )
>       {
> @@ -843,14 +835,83 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
>       xfree(tr->nodes_address);
>       xfree(tr->fdt);
>   
> -    rangeset_destroy(tr->irq_ranges);
> -    rangeset_destroy(tr->iomem_ranges);
> -
>       xfree(tr);
>   
>       return rc;
>   }
>   
> +static long handle_attach_overlay_nodes(struct domain *d,
> +                                        const void *overlay_fdt,
> +                                        uint32_t overlay_fdt_size)
> +{
> +    int rc;
> +    unsigned int j;
> +    struct overlay_track *entry;
> +
> +    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> +    if ( rc )
> +        return rc;
> +
> +    spin_lock(&overlay_lock);
> +
> +    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
> +    if ( entry == NULL )
> +    {
> +        rc = -EINVAL;
> +        goto out;
> +    }
> +
> +    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
> +    if (entry->irq_ranges == NULL)
> +    {
> +        rc = -ENOMEM;
> +        printk(XENLOG_ERR "Creating IRQ rangeset failed");
> +        goto out;
> +    }
> +
> +    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
> +    if (entry->iomem_ranges == NULL)
> +    {
> +        rc = -ENOMEM;
> +        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
> +        goto out;
> +    }
> +
> +    for ( j = 0; j < entry->num_nodes; j++ )
> +    {
> +        struct dt_device_node *overlay_node;
> +
> +        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
> +        if ( overlay_node == NULL )
> +        {
> +            rc = -EINVAL;
> +            goto out;
> +        }
> +
> +        write_lock(&dt_host_lock);
> +        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
> +                           entry->iomem_ranges, entry->irq_ranges);
> +        write_unlock(&dt_host_lock);
> +        if ( rc )
> +        {
> +            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> +            goto out;
> +        }
> +    }
> +
> +    spin_unlock(&overlay_lock);
> +
> +    return 0;
> +
> + out:
> +    spin_unlock(&overlay_lock);
> +
> +    rangeset_destroy(entry->irq_ranges);
> +    rangeset_destroy(entry->iomem_ranges);
> +
> +    return rc;
> +}
> +
>   long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
>   {
>       long ret;
> @@ -890,6 +951,42 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
>       return ret;
>   }
>   
> +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
> +{
> +    long ret;
> +    void *overlay_fdt;
> +
> +    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
> +        return -EOPNOTSUPP;
> +
> +    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )

Let's avoid to hardocode KB(500) a second time (it alredy exists in the 
sysctl path).

But I would actually consider to allow up to 512KB because...

> +        return -EINVAL;
> +
> +    if ( op->pad[0] || op->pad[1] || op->pad[2] )
> +        return -EINVAL;
> +
> +    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);

... xmalloc_bytes() will be more efficient. If the size is not power of 
2 pages, it will free the leftover. Anyway, this could be a follow-up patch.

> +
> +    if ( overlay_fdt == NULL )
> +        return -ENOMEM;
> +
> +    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
> +    if ( ret )
> +    {
> +        gprintk(XENLOG_ERR, "copy from guest failed\n");
> +        xfree(overlay_fdt);
> +
> +        return -EFAULT;
> +    }
> +
> +    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
> +        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);

I think you would return 0 if the operation is not supported. But I 
think we need to return -EOPNOTSUPP.

> +
> +    xfree(overlay_fdt);
> +
> +    return ret;
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index a33f9ec32b..ac3c2a7c4c 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h

If I am not mistkane, this is the first change in the domctl header for 
4.19. So you want to bump XEN_DOMCTL_INTERFACE_VERSION.

> @@ -1190,6 +1190,16 @@ struct xen_domctl_vmtrace_op {
>   typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
>   DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
>   
> +#if defined(__arm__) || defined(__aarch64__)
> +struct xen_domctl_dt_overlay {
> +    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
> +    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
> +#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1

OOI, why not starting from 0?

> +    uint8_t overlay_op;                     /* IN: Attach. */
> +    uint8_t pad[3];                         /* IN: Must be zero. */
> +};
> +#endif
> +
>   struct xen_domctl {
>       uint32_t cmd;
>   #define XEN_DOMCTL_createdomain                   1
> @@ -1277,6 +1287,7 @@ struct xen_domctl {
>   #define XEN_DOMCTL_vmtrace_op                    84
>   #define XEN_DOMCTL_get_paging_mempool_size       85
>   #define XEN_DOMCTL_set_paging_mempool_size       86
> +#define XEN_DOMCTL_dt_overlay                    87
>   #define XEN_DOMCTL_gdbsx_guestmemio            1000
>   #define XEN_DOMCTL_gdbsx_pausevcpu             1001
>   #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
> @@ -1339,6 +1350,9 @@ struct xen_domctl {
>           struct xen_domctl_vuart_op          vuart_op;
>           struct xen_domctl_vmtrace_op        vmtrace_op;
>           struct xen_domctl_paging_mempool    paging_mempool;
> +#if defined(__arm__) || defined(__aarch64__)
> +        struct xen_domctl_dt_overlay        dt_overlay;
> +#endif
>           uint8_t                             pad[128];
>       } u;
>   };
> diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
> index febaa4b16a..3a6e7d48f0 100644
> --- a/xen/include/public/sysctl.h
> +++ b/xen/include/public/sysctl.h
> @@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
>   DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
>   #endif
>   
> -#if defined(__arm__) || defined (__aarch64__)
> +#if defined(__arm__) || defined(__aarch64__)

This seems like unrelated change. Maybe mention in the commit message 
that you are fixing the coding style?

>   /*
>    * XEN_SYSCTL_dt_overlay
> - * Performs addition/removal of device tree nodes under parent node using dtbo.
> - * This does in three steps:
> - *  - Adds/Removes the nodes from dt_host.
> - *  - Adds/Removes IRQ permission for the nodes.
> - *  - Adds/Removes MMIO accesses.
> + * Performs addition/removal of device tree nodes under parent node using dtbo
> + * from dt_host.
>    */
>   struct xen_sysctl_dt_overlay {
>       XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
> @@ -1265,7 +1262,7 @@ struct xen_sysctl {
>           struct xen_sysctl_cpu_policy        cpu_policy;
>   #endif
>   
> -#if defined(__arm__) || defined (__aarch64__)
> +#if defined(__arm__) || defined(__aarch64__)
>           struct xen_sysctl_dt_overlay        dt_overlay;
>   #endif
>           uint8_t                             pad[128];
> diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
> index c0567741ee..7f0f3741b5 100644
> --- a/xen/include/xen/dt-overlay.h
> +++ b/xen/include/xen/dt-overlay.h
> @@ -39,15 +39,22 @@ struct overlay_track {
>   };
>   
>   struct xen_sysctl_dt_overlay;
> +struct xen_domctl_dt_overlay;
>   
>   #ifdef CONFIG_OVERLAY_DTB
>   long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
> +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
>   #else
>   #include <xen/errno.h>
>   static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
>   {
>       return -EOPNOTSUPP;
>   }

NIT: Newline here please.

> +static inline long dt_overlay_domctl(struct domain *d,
> +                                     struct xen_domctl_dt_overlay *op)
> +{
> +    return -EOPNOTSUPP;
> +}
>   #endif
>   
>   #endif /* __XEN_DT_OVERLAY_H__ */

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 22:42:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 22:42:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728902.1133976 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAH8e-0008UJ-Mw; Thu, 23 May 2024 22:42:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728902.1133976; Thu, 23 May 2024 22:42:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAH8e-0008UC-Jw; Thu, 23 May 2024 22:42:20 +0000
Received: by outflank-mailman (input) for mailman id 728902;
 Thu, 23 May 2024 22:42:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAH8c-0008U6-HV
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 22:42:18 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAH8c-0007n0-DT; Thu, 23 May 2024 22:42:18 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAH8c-0008RA-4n; Thu, 23 May 2024 22:42:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=MKSSdCDliWElWe1MT0ed4Gi7k4ta2XLjuxWA8fnaqIM=; b=qjpfxFELQDNAXeCjNizUrQ9Xb7
	y5LOtFuQfPogR5EuZXbwojbcp3SC4EanmI+egvyhOhNCbV7KC3E0q+7DjUqJAmWq0AH+X2mJ0c07A
	Vk7s8JzieClcrFFB1UEdX8DanxGHCv4yrQpVWmUAtNINf1whPWI+N9vMGjaBhzz1Pfsk=;
Message-ID: <a9033982-9c7b-4b16-b2c0-e47a3f73e452@xen.org>
Date: Thu, 23 May 2024 23:42:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 7/9] xen/arm: Support device detachment from domains
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-8-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-8-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> Similarly as the device attachment from DT overlay to domain, this
> commit implements the device detachment from domain. The DOMCTL
> XEN_DOMCTL_dt_overlay op is extended to have the operation
> XEN_DOMCTL_DT_OVERLAY_DETACH. The detachment of the device is
> implemented by unmapping the IRQ and IOMMU resources. Note that with
> these changes, the device de-registration from the IOMMU driver should
> only happen at the time when the DT overlay is removed from the Xen
> device tree.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> ---
> v4:
> - Split the original patch, only do device detachment from domain.
> ---
>   xen/common/dt-overlay.c     | 243 ++++++++++++++++++++++++++++--------
>   xen/include/public/domctl.h |   3 +-
>   2 files changed, 194 insertions(+), 52 deletions(-)
> 
> diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
> index 1087f9b502..693b6e4777 100644
> --- a/xen/common/dt-overlay.c
> +++ b/xen/common/dt-overlay.c
> @@ -392,24 +392,100 @@ find_track_entry_from_tracker(const void *overlay_fdt,
>       return entry;
>   }
>   
> +static int remove_irq(unsigned long s, unsigned long e, void *data)
> +{
> +    struct domain *d = data;
> +    int rc = 0;
> +
> +    /*
> +     * IRQ should always have access unless there are duplication of
> +     * of irqs in device tree. There are few cases of xen device tree
> +     * where there are duplicate interrupts for the same node.
> +     */
> +    if (!irq_access_permitted(d, s))

Because of this check, it means that ...

> +        return 0;
> +    /*
> +     * TODO: We don't handle shared IRQs for now. So, it is assumed that
> +     * the IRQs was not shared with another domain.
> +     */
> +    rc = irq_deny_access(d, s);
> +    if ( rc )
> +    {
> +        printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s);
> +        return rc;
> +    }
> +
> +    rc = release_guest_irq(d, s);

... release_guest_irq() fails on the next retry it will pass. I don't 
think this is what we want.

Instead, we probably want to re-order the call.

> +    if ( rc )
> +    {
> +        printk(XENLOG_ERR "unable to release irq %ld\n", s);
> +        return rc;
> +    }
> +
> +    return rc;
> +}
> +
> +static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d)
> +{
> +    return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d);
> +}
> +
> +static int remove_iomem(unsigned long s, unsigned long e, void *data)
> +{
> +    struct domain *d = data;
> +    int rc = 0;
> +    p2m_type_t t;
> +    mfn_t mfn;
> +
> +    mfn = p2m_lookup(d, _gfn(s), &t);

What are you trying to addres with this check? For instance, the fact 
that the first MFN is mapped, doesn't guarantee the rest is.

> +    if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL )

I don't understand why we are checking for 0 here. In theory, it is 
valid MFN. Also, the second part wants to be INVALID_MFN.

> +        return -EINVAL;
> +
> +    rc = iomem_deny_access(d, s, e);

iomem_deny_access() works on MFN but here you pass an MFN. Are you 
assuming the GFN == MFN? How would that work for domains that are not 
direct mapped?

> +    if ( rc )
> +    {
> +        printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n",
> +               d, s, e);
> +        return rc;
> +    }
> +
> +    rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s));
> +    if ( rc )
> +        return rc;
> +
> +    return rc;
> +}
> +
> +static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d)
> +{
> +    return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d);
> +}
> +
>   /* Check if node itself can be removed and remove node from IOMMU. */
> -static int remove_node_resources(struct dt_device_node *device_node)
> +static int remove_node_resources(struct dt_device_node *device_node,
> +                                 struct domain *d)
>   {
>       int rc = 0;
>       unsigned int len;
>       domid_t domid;
>   
> -    domid = dt_device_used_by(device_node);
> +    if ( !d )

I looked at the code, I am a bit unsure how "d" can be NULL. Do you have 
any pointer?

> +    {
> +        domid = dt_device_used_by(device_node);
>   
> -    dt_dprintk("Checking if node %s is used by any domain\n",
> -               device_node->full_name);
> +        dt_dprintk("Checking if node %s is used by any domain\n",
> +                   device_node->full_name);
>   
> -    /* Remove the node if only it's assigned to hardware domain or domain io. */
> -    if ( domid != hardware_domain->domain_id && domid != DOMID_IO )
> -    {
> -        printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n",
> -               device_node->full_name, domid);
> -        return -EINVAL;
> +        /*
> +         * We also check if device is assigned to DOMID_IO as when a domain
> +         * is destroyed device is assigned to DOMID_IO.
> +         */
> +        if ( domid != DOMID_IO )
> +        {
> +            printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n",
> +                   device_node->full_name, DOMID_IO, domid);
> +            return -EINVAL;
> +        }
>       }
>   
>       /* Check if iommu property exists. */
> @@ -417,9 +493,12 @@ static int remove_node_resources(struct dt_device_node *device_node)
>       {
>           if ( dt_device_is_protected(device_node) )
>           {
> -            rc = iommu_remove_dt_device(device_node);
> -            if ( rc < 0 )
> -                return rc;
> +            if ( !list_empty(&device_node->domain_list) )
> +            {
> +                rc = iommu_deassign_dt_device(d, device_node);
> +                if ( rc < 0 )
> +                    return rc;
> +            }
>           }
>       }
>   
> @@ -428,7 +507,8 @@ static int remove_node_resources(struct dt_device_node *device_node)
>   
>   /* Remove all descendants from IOMMU. */
>   static int
> -remove_descendant_nodes_resources(const struct dt_device_node *device_node)
> +remove_descendant_nodes_resources(const struct dt_device_node *device_node,
> +                                  struct domain *d)
>   {
>       int rc = 0;
>       struct dt_device_node *child_node;
> @@ -438,12 +518,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node)
>       {
>           if ( child_node->child )
>           {
> -            rc = remove_descendant_nodes_resources(child_node);
> +            rc = remove_descendant_nodes_resources(child_node, d);
>               if ( rc )
>                   return rc;
>           }
>   
> -        rc = remove_node_resources(child_node);
> +        rc = remove_node_resources(child_node, d);
>           if ( rc )
>               return rc;
>       }
> @@ -456,8 +536,7 @@ static int remove_nodes(const struct overlay_track *tracker)
>   {
>       int rc = 0;
>       struct dt_device_node *overlay_node;
> -    unsigned int j;
> -    struct domain *d = hardware_domain;
> +    unsigned int j, len;
>   
>       for ( j = 0; j < tracker->num_nodes; j++ )
>       {
> @@ -467,18 +546,15 @@ static int remove_nodes(const struct overlay_track *tracker)
>   
>           write_lock(&dt_host_lock);
>   
> -        rc = remove_descendant_nodes_resources(overlay_node);
> -        if ( rc )
> +        /* Check if iommu property exists. */
> +        if ( dt_get_property(overlay_node, "iommus", &len) )

Why do we need to check for the property in the "iommus" rather than ...

>           {
> -            write_unlock(&dt_host_lock);
> -            return rc;
> -        }
> -
> -        rc = remove_node_resources(overlay_node);
> -        if ( rc )
> -        {
> -            write_unlock(&dt_host_lock);
> -            return rc;
> +            if ( dt_device_is_protected(overlay_node) )

... relying on dt_device_is_protected()?

> +            {
> +                rc = iommu_remove_dt_device(overlay_node);
> +                if ( rc < 0 )
> +                    return rc;
> +            }
>           }
>   
>           dt_dprintk("Removing node: %s\n", overlay_node->full_name);
> @@ -493,22 +569,6 @@ static int remove_nodes(const struct overlay_track *tracker)
>           write_unlock(&dt_host_lock);
>       }
>   
> -    /* Remove IRQ access. */
> -    if ( tracker->irq_ranges )
> -    {
> -        rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d);
> -        if ( rc )
> -            return rc;
> -    }
> -
> -   /* Remove mmio access. */
> -    if ( tracker->iomem_ranges )
> -    {
> -        rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d);
> -        if ( rc )
> -            return rc;
> -    }
> -
>       return rc;
>   }
>   
> @@ -534,7 +594,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
>       {
>           rc = -EINVAL;
>           goto out;
> -

Valid but unrelated change?

>       }
>   
>       rc = remove_nodes(entry);
> @@ -552,9 +611,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
>   
>       xfree(entry->nodes_address);
>   
> -    rangeset_destroy(entry->irq_ranges);
> -    rangeset_destroy(entry->iomem_ranges);
> -
>       xfree(entry);
>   
>    out:
> @@ -840,6 +896,88 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
>       return rc;
>   }
>   
> +static long handle_detach_overlay_nodes(struct domain *d,
> +                                        const void *overlay_fdt,
> +                                        uint32_t overlay_fdt_size)
> +{
> +    int rc;
> +    unsigned int j;
> +    struct overlay_track *entry;
> +
> +    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> +    if ( rc )
> +        return rc;
> +
> +    spin_lock(&overlay_lock);
> +
> +    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
> +    if ( entry == NULL )
> +    {
> +        rc = -EINVAL;
> +        goto out;
> +    }
> +
> +    for ( j = 0; j < entry->num_nodes; j++ )
> +    {
> +        struct dt_device_node *overlay_node;
> +
> +        overlay_node = (struct dt_device_node *)entry->nodes_address[j];

Can you remind me why nodes_address is storing unsigned long rather 
struct dt_device_node?

> +        if ( overlay_node == NULL )
> +        {
> +            rc = -EINVAL;
> +            goto out;
> +        }
> +
> +        write_lock(&dt_host_lock);
> +        rc = remove_descendant_nodes_resources(overlay_node, d);
> +        if ( rc )
> +        {
> +            write_unlock(&dt_host_lock);
> +            goto out;
> +        }
> +
> +        rc = remove_node_resources(overlay_node, d);
> +        if ( rc )
> +        {
> +            write_unlock(&dt_host_lock);
> +            goto out;
> +        }
> +        write_unlock(&dt_host_lock);
> +
> +        rc = remove_all_irqs(entry->irq_ranges, d);

I am really confused. This is called in the for loop but entry doesn't 
seem to change. So why do we need to call it for every iteration? Also...


> +        if ( rc )
> +            goto out;
> +
> +        rc = remove_all_iomems(entry->iomem_ranges, d);
> +        if ( rc )
> +            goto out;
> +    }
> +
> +    /* Remove IRQ access. */
> +    if ( entry->irq_ranges )
> +    {
> +        rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d);

... remove_all_irqs() will revert the permission. So why do we need to 
do it again?

> +        if ( rc )
> +            goto out;
> +    }
> +
> +    /* Remove mmio access. */
> +    if ( entry->iomem_ranges )
> +    {
> +        rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d);

Same question here.

> +        if ( rc )
> +            goto out;
> +    }
> +
> +    rangeset_destroy(entry->irq_ranges);
> +    rangeset_destroy(entry->iomem_ranges);
> +
> + out:
> +    spin_unlock(&overlay_lock);
> +
> +    return rc;
> +}
> +
>   static long handle_attach_overlay_nodes(struct domain *d,
>                                           const void *overlay_fdt,
>                                           uint32_t overlay_fdt_size)
> @@ -956,7 +1094,8 @@ long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
>       long ret;
>       void *overlay_fdt;
>   
> -    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
> +    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH &&
> +         op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH )
>           return -EOPNOTSUPP;
>   
>       if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
> @@ -979,7 +1118,9 @@ long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
>           return -EFAULT;
>       }
>   
> -    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
> +    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH )
> +        ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
> +    else
>           ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
>   
>       xfree(overlay_fdt);
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index ac3c2a7c4c..e91f743fa4 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -1195,7 +1195,8 @@ struct xen_domctl_dt_overlay {
>       XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
>       uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
>   #define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
> -    uint8_t overlay_op;                     /* IN: Attach. */
> +#define XEN_DOMCTL_DT_OVERLAY_DETACH                2
> +    uint8_t overlay_op;                     /* IN: Attach/Detach. */

Changing the comment for every operation we add is not really scalable 
:). If you want an explanation, can it be generic?

>       uint8_t pad[3];                         /* IN: Must be zero. */
>   };
>   #endif

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 22:48:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 22:48:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728907.1133987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHEC-0000hl-BE; Thu, 23 May 2024 22:48:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728907.1133987; Thu, 23 May 2024 22:48:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHEC-0000he-8N; Thu, 23 May 2024 22:48:04 +0000
Received: by outflank-mailman (input) for mailman id 728907;
 Thu, 23 May 2024 22:48:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAHEA-0000hY-Uu
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 22:48:02 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHEA-0007so-Li; Thu, 23 May 2024 22:48:02 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHEA-0000KJ-GL; Thu, 23 May 2024 22:48:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=GbNAxB2Q9Ar/rzjRdCeKi29r70K4caa/qZut9mqrfH0=; b=6wTEN93yl53ro01RSmQ71iOjvO
	JScXd25bGkZLqJj28vULfGrehFMVruPgn8odqwsqODqFAJt3PhHjIZSbLpADEqlhJpAt5rXbEDx15
	0QvSHBXDFoHb6S7UZ8dlEQpg6XilFiRXah2Xg2Y7KzLotilr3BWZsjrYweLh9fmzVpB0=;
Message-ID: <a437fe4c-bb21-456f-acf3-53ae8916f4f6@xen.org>
Date: Thu, 23 May 2024 23:48:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 9/9] docs: Add device tree overlay documentation
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-10-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-10-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v4:
> - No change.
> v3:
> - No change.
> v2:
> - Update the content based on the changes in this version.
> ---
>   docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++
>   1 file changed, 99 insertions(+)
>   create mode 100644 docs/misc/arm/overlay.txt
> 
> diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
> new file mode 100644
> index 0000000000..811a6de369
> --- /dev/null
> +++ b/docs/misc/arm/overlay.txt
> @@ -0,0 +1,99 @@
> +# Device Tree Overlays support in Xen
> +
> +Xen now supports dynamic device assignment to running domains,

This reads as we "support" the feature. I would prefer if we write "Xen 
expirementally supports..." or similar.

> +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and
> +attaching/detaching them to/from a running domain with given $domid.
> +
> +Dynamic node assignment works in two steps:
> +
> +## Add/Remove device tree overlay to/from Xen device tree
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
> +
> +## Attach/Detach device from the DT overlay to/from domain
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor attach/detach the device to/from the user-provided $domid by
> +   mapping/unmapping node resources in the DT overlay.
> +
> +# Examples
> +
> +Here are a few examples on how to use it.
> +
> +## Dom0 device add
> +
> +For assigning a device tree overlay to Dom0, user should firstly properly
> +prepare the DT overlay. More information about device tree overlays can be
> +found in [1]. Then, in Dom0, enter the following:
> +
> +    (dom0) xl dt-overlay add overlay.dtbo
> +
> +This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
> +
> +To assign the newly added device from the dtbo to Dom0:
> +
> +    (dom0) xl dt-overlay attach overlay.dtbo 0
> +
> +Next, if the user wants to add the same device tree overlay to dom0
> +Linux, execute the following:
> +
> +    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> +    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> +
> +Finally if needed, the relevant Linux kernel drive can be loaded using:
> +
> +    (dom0) modprobe module_name.ko
> +
> +## Dom0 device remove
> +
> +For removing the device from Dom0, first detach the device from Dom0:
> +
> +    (dom0) xl dt-overlay detach overlay.dtbo 0
> +
> +NOTE: The user is expected to unload any Linux kernel modules which
> +might be accessing the devices in overlay.dtbo before detach the device.
> +Detaching devices without unloading the modules might result in a crash.
> +
> +Then remove the overlay from Xen device tree:
> +
> +    (dom0) xl dt-overlay remove overlay.dtbo
> +
> +## DomU device add/remove
> +
> +All the nodes in dtbo will be assigned to a domain; the user will need
> +to prepare the dtb for the domU. For example, the `interrupt-parent` property
> +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`.
> +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
> +
> +User will need to create the DomU with below properties properly configured
> +in the xl config file:
> +- `iomem`

I don't quite understand how the user can specify the MMIO region if the 
device is attached after the domain is created.

> +- `passthrough` (if IOMMU is needed)
> +
> +User will also need to modprobe the relevant drivers.
> +
> +Example for domU device add:
> +
> +    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
> +    (dom0) xl dt-overlay attach overlay.dtbo $domid

Can how clarify how the MMIO will be mapped? Is it direct mapped? If so, 
couldn't this result to clash with other part of the address space (e.g. 
RAM?).

> +    (dom0) xl console $domid                         # To access $domid console
> +
> +Next, if the user needs to modify/prepare the overlay.dtbo suitable for
> +the domU:
> +
> +    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> +    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> +
> +Finally, if needed, the relevant Linux kernel drive can be probed:
> +
> +    (domU) modprobe module_name.ko
> +
> +Example for domU overlay remove:
> +
> +    (dom0) xl dt-overlay detach overlay.dtbo $domid
> +    (dom0) xl dt-overlay remove overlay.dtbo

I assume we have safety check in place to ensure we can't remove the 
device if it is already attached. Is that correct?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 22:54:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 22:54:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728914.1133998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHK7-0002OM-1N; Thu, 23 May 2024 22:54:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728914.1133998; Thu, 23 May 2024 22:54:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHK6-0002OF-T9; Thu, 23 May 2024 22:54:10 +0000
Received: by outflank-mailman (input) for mailman id 728914;
 Thu, 23 May 2024 22:54:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAHK5-0002O5-Hm; Thu, 23 May 2024 22:54:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAHK5-0007yU-Cm; Thu, 23 May 2024 22:54:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAHK5-0004EA-3K; Thu, 23 May 2024 22:54:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAHK5-0002TZ-2m; Thu, 23 May 2024 22:54:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dPhNq4YCb2BAGi5ETzT2er8fmg3nMW4FPpSFumkHeQ0=; b=2bQvCURWm6H+jPe0c1a1NA4OO9
	9rEUI5DnDlzkEbRbzajQnB6Sl1PBWQvUScts/NzDgLihZ0zYGSGG864xfGoYLrdkdgAmQfg2enaZI
	HPMON06vxXYsyrUtymPGMfvEY69kQPN5YjBluOPoS+gieOqJ5D/4OzkRhfL6T+9UR+/U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186103-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186103: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-amd64-xsm:xen-build:fail:regression
    linux-linus:build-amd64:xen-build:fail:regression
    linux-linus:build-i386:xen-build:fail:regression
    linux-linus:build-i386-xsm:xen-build:fail:regression
    linux-linus:build-armhf:xen-build:fail:regression
    linux-linus:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:build-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:build-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:build-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-bios:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-uefi:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=c760b3725e52403dc1b28644fb09c47a83cacea6
X-Osstest-Versions-That:
    linux=8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 May 2024 22:54:09 +0000

flight 186103 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186103/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 186052
 build-amd64                   6 xen-build                fail REGR. vs. 186052
 build-i386                    6 xen-build                fail REGR. vs. 186052
 build-i386-xsm                6 xen-build                fail REGR. vs. 186052
 build-armhf                   6 xen-build                fail REGR. vs. 186052

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-vhd       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine      1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-bios  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                c760b3725e52403dc1b28644fb09c47a83cacea6
baseline version:
 linux                8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6

Last test of basis   186052  2024-05-21 01:42:42 Z    2 days
Failing since        186065  2024-05-21 16:10:24 Z    2 days    3 attempts
Testing same since   186103  2024-05-23 05:27:56 Z    0 days    1 attempts

------------------------------------------------------------
435 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                blocked 
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-amd64-examine                                     blocked 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 blocked 
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      blocked 
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      blocked 
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 36039 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu May 23 23:06:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 23:06:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728923.1134007 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHVQ-0004Bk-18; Thu, 23 May 2024 23:05:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728923.1134007; Thu, 23 May 2024 23:05:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHVP-0004Bd-UG; Thu, 23 May 2024 23:05:51 +0000
Received: by outflank-mailman (input) for mailman id 728923;
 Thu, 23 May 2024 23:05:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAHVO-0004BX-TI
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 23:05:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHVO-0008Cf-JI; Thu, 23 May 2024 23:05:50 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHVO-0001KZ-DX; Thu, 23 May 2024 23:05:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=5z+uVk4AZgOoyi41Yz1XiORy3HPDEZSb2w8edttFTOU=; b=rp0F+pvXnP1qMrQoAREObl9EiE
	3kcse7649JaO0vXf/QTIO6OrVSgBOGP/HJHC7A0ZiY2l6KyBjjPmvGLhQFC4SEXZQ6+bqWPacw0FW
	SNgmdbR0aTzfFlePVkiDC0gGZ8QMbpBR90q8tSTb7ASfnIwsf0AQSmU1uJfCOR07yDgA=;
Message-ID: <02e22734-2e0c-4a0a-8c5a-42beee0cc396@xen.org>
Date: Fri, 24 May 2024 00:05:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 3/9] tools/arm: Introduce the "nr_spis" xl config entry
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>,
 Jason Andryuk <jason.andryuk@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-4-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-4-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> Currently, the number of SPIs allocated to the domain is only
> configurable for Dom0less DomUs. Xen domains are supposed to be
> platform agnostics and therefore the numbers of SPIs for libxl
> guests should not be based on the hardware.
> 
> Introduce a new xl config entry for Arm to provide a method for
> user to decide the number of SPIs. This would help to avoid
> bumping the `config->arch.nr_spis` in libxl everytime there is a
> new platform with increased SPI numbers.
> 
> Update the doc and the golang bindings accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> ---
> v4:
> - Add Jason's Reviewed-by tag.
> v3:
> - Reword documentation to avoid ambiguity.
> v2:
> - New patch to replace the original patch in v1:
>    "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
> ---
>   docs/man/xl.cfg.5.pod.in             | 14 ++++++++++++++
>   tools/golang/xenlight/helpers.gen.go |  2 ++
>   tools/golang/xenlight/types.gen.go   |  1 +
>   tools/libs/light/libxl_arm.c         |  4 ++--
>   tools/libs/light/libxl_types.idl     |  1 +
>   tools/xl/xl_parse.c                  |  3 +++
>   6 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 8f2b375ce9..416d582844 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3072,6 +3072,20 @@ raised.
>   
>   =back
>   
> +=over 4
> +
> +=item B<nr_spis="NR_SPIS">
> +
> +An optional 32-bit integer parameter specifying the number of SPIs (Shared

We can't support that much SPIs :). The limit would be 991 SPIs.

> +Peripheral Interrupts) to allocate for the domain. If the value specified by
> +the `nr_spis` parameter is smaller than the number of SPIs calculated by the
> +toolstack based on the devices allocated for the domain, or the `nr_spis`
> +parameter is not specified, the value calculated by the toolstack will be used
> +for the domain. Otherwise, the value specified by the `nr_spis` parameter will
> +be used.

I think it would be worth mentioning that the number of SPIs should 
match the highest interrupt ID that will be assigned to the domain 
(rather than the number of SPIs planned to be assigned).

> +
> +=back
> +
>   =head3 x86
>   
>   =over 4
> diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
> index b9cb5b33c7..fe5110474d 100644
> --- a/tools/golang/xenlight/helpers.gen.go
> +++ b/tools/golang/xenlight/helpers.gen.go
> @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
>   x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
>   x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
>   x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
> +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
>   if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>   }
> @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
>   xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
>   xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
>   xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
> +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
>   if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>   }
> diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
> index 5b293755d7..c9e45b306f 100644
> --- a/tools/golang/xenlight/types.gen.go
> +++ b/tools/golang/xenlight/types.gen.go
> @@ -597,6 +597,7 @@ ArchArm struct {
>   GicVersion GicVersion
>   Vuart VuartType
>   SveVl SveType
> +NrSpis uint32
>   }
>   ArchX86 struct {
>   MsrRelaxed Defbool
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 1cb89fa584..a4029e3ac8 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>   
>       LOG(DEBUG, "Configure the domain");
>   
> -    config->arch.nr_spis = nr_spis;
> -    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
> +    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);

I am not entirely sure about using max(). To me if the user specifies a 
lower limit, then we should throw an error because this is likely an 
indication that the SPIs they will want to assign will clash with the 
emulated ones.

So it would be better to warn at domain creation rather than waiting 
until the IRQs are assigned.

I would like Anthony's opinion on this one. Given he is away this month, 
I guess we could get this patch merged (with other comments addressed) 
and have a follow-up if wanted before 4.19.

> +    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
>   
>       switch (d_config->b_info.arch_arm.gic_version) {
>       case LIBXL_GIC_VERSION_DEFAULT:
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 79e9c656cc..4e65e6fda5 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>       ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                  ("vuart", libxl_vuart_type),
>                                  ("sve_vl", libxl_sve_type),
> +                               ("nr_spis", uint32),

 From my understandig, any change in the .idl requires a corresponding 
LIBXL_HAVE_... in include/libxl.h. This is in order to allow external 
toolstack (such as libvirt) to be able to select at build time between 
multiple version of libxl.

>                                 ])),
>       ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                                 ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index c504ab3711..e3a4800f6e 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -2935,6 +2935,9 @@ skip_usbdev:
>           }
>       }
>   
> +    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
> +        b_info->arch_arm.nr_spis = l;
> +
>       parse_vkb_list(config, d_config);
>   
>       d_config->virtios = NULL;

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 23:11:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 23:11:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728930.1134018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHah-0005yi-Ou; Thu, 23 May 2024 23:11:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728930.1134018; Thu, 23 May 2024 23:11:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHah-0005yb-KK; Thu, 23 May 2024 23:11:19 +0000
Received: by outflank-mailman (input) for mailman id 728930;
 Thu, 23 May 2024 23:11:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAHag-0005yV-9m
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 23:11:18 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHaf-0008I4-W3; Thu, 23 May 2024 23:11:17 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAHaf-0001oZ-Qd; Thu, 23 May 2024 23:11:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EwDu3x4EZ7cn1iaD41BLePUbCeqetI5jMQx2kpFydyo=; b=5+q/n0sqtgiM06k6YSoKLLTjyh
	l2/VwMwKZjrU2RY94YW7bsR9kN7HX484zTP0CbX19S7HiEgKotNIpXMVFVRZIudM9tI4ZsYJ+Z24I
	ITmxAzGMlXjl5qQIDKybBCdAsEIItXwAggvOspygw7X43NHylS43KFnN431gws8sX2ZE=;
Message-ID: <fc2ff110-0af7-439b-a220-2cb21c63c54e@xen.org>
Date: Fri, 24 May 2024 00:11:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 8/9] tools: Introduce the "xl dt-overlay
 {attach,detach}" commands
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jason Andryuk <jason.andryuk@amd.com>
References: <20240523074040.1611264-1-xin.wang2@amd.com>
 <20240523074040.1611264-9-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240523074040.1611264-9-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 23/05/2024 08:40, Henry Wang wrote:
> With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> attach/detach devices from the provided DT overlay to domains.
> Support this by introducing a new set of "xl dt-overlay" commands and
> related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
> rework the command option parsing logic.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> ---
> v4:
> - Add Jason's Reviewed-by tag.
> v3:
> - Introduce new API libxl_dt_overlay_domain() and co., instead of
>    reusing existing API libxl_dt_overlay().
> - Add in-code comments for the LIBXL_DT_OVERLAY_* macros.
> - Use find_domain() to avoid getting domain_id from strtol().
> v2:
> - New patch.
> ---
>   tools/include/libxl.h               | 10 +++++++
>   tools/include/xenctrl.h             |  3 +++
>   tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++
>   tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++
>   tools/xl/xl_cmdtable.c              |  4 +--
>   tools/xl/xl_vmcontrol.c             | 42 ++++++++++++++++++++---------
>   6 files changed, 104 insertions(+), 14 deletions(-)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index 62cb07dea6..6cc6d6bf6a 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h

I think you also need to introduce LIBXL_HAVE_...

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 23 23:37:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 May 2024 23:37:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728944.1134026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHzo-0000kR-IB; Thu, 23 May 2024 23:37:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728944.1134026; Thu, 23 May 2024 23:37:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAHzo-0000kK-Fj; Thu, 23 May 2024 23:37:16 +0000
Received: by outflank-mailman (input) for mailman id 728944;
 Thu, 23 May 2024 23:37:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GulG=M2=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAHzm-0000jy-U8
 for xen-devel@lists.xenproject.org; Thu, 23 May 2024 23:37:15 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a021ce2-195d-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 01:37:03 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 45217CE16DE;
 Thu, 23 May 2024 23:36:59 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 85C54C2BD10;
 Thu, 23 May 2024 23:36:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a021ce2-195d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716507418;
	bh=kHkeO9Fj3Fyo5IpsnSa8jWlvVs+nvEzXybD8jzMh25c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WllI793XLeK614a/otG82rIFNKD466NFp+PU16NevhA3gP0a2KNK0OR+nrzNswmcR
	 TzyT9Z6nwaDdCzp8dRx7gxocyRkz6HmobY8dTDD3r96Vei64lf0ZC63xoN9TeNJBrv
	 QOaLn6PKyYPfrnrr9Fgvxwvjw5uQ+KZNiIgwtUCcFaqrtEHOyU0qrOKL4x1EFsctXp
	 EuRxH4211sDzhBasK5dVLF5CGHpNV2L1goHYxiYeoPcjSqy5sLX7rqbXol2oogVos1
	 6gQadxcWEBv1ixAjkEYP5kdP9BqBy/FX+b0R4XbSfRxDE0xwsFCKvA+7eBIfRL658Z
	 /HD7+ZRllvIXA==
Date: Thu, 23 May 2024 16:36:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Sergiy Kibrik <sergiy_kibrik@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Xenia Ragiadakou <xenia.ragiadakou@amd.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
In-Reply-To: <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405231634290.2557291@ubuntu-linux-20-04-desktop>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com> <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com> <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com> <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
 <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 May 2024, Jan Beulich wrote:
> On 23.05.2024 15:07, Sergiy Kibrik wrote:
> > 16.05.24 14:12, Jan Beulich:
> >> On 15.05.2024 11:12, Sergiy Kibrik wrote:
> >>> --- a/xen/arch/x86/include/asm/cpufeature.h
> >>> +++ b/xen/arch/x86/include/asm/cpufeature.h
> >>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
> >>>   #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
> >>>   #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
> >>>   #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
> >>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
> >>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
> >>> +                                  boot_cpu_has(X86_FEATURE_VMX))
> >>>   #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
> >>>   #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
> >>>   #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
> >>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
> >>>   
> >>>   /* CPUID level 0x80000001.ecx */
> >>>   #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
> >>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
> >>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
> >>> +                                  boot_cpu_has(X86_FEATURE_SVM))
> >>>   #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
> >>>   #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
> >>>   #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
> >>
> >> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
> >> and as a result one-off indentation on the wrapped lines) I'm not really
> >> certain we can do this. The description goes into detail why we would want
> >> this, but it doesn't cover at all why it is safe for all present (and
> >> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
> >> just to derive further knowledge from that, without them being directly
> >> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
> >> for example. While it looks to be okay there, it may give you an idea of
> >> what I mean.
> >>
> >> Things might become better separated if instead for such checks we used
> >> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
> >> that's still pretty far out, I'm afraid.
> > 
> > I've followed a suggestion you made for patch in previous series:
> > 
> > https://lore.kernel.org/xen-devel/8fbd604e-5e5d-410c-880f-2ad257bbe08a@suse.com/
> 
> See the "If not, ..." that I had put there. Doing the change just mechanically
> isn't enough, you also need to make clear (in the description) that you
> verified it's safe to have this way.

What does it mean to "verified it's safe to have this way"? "Safe" in
what way?


> > yet if this approach can potentially be unsafe (I'm not completely sure 
> > it's safe), should we instead fallback to the way it was done in v1 
> > series? I.e. guard calls to vmx/svm-specific calls where needed, like in 
> > these 3 patches:
> > 
> > 1) 
> > https://lore.kernel.org/xen-devel/20240416063328.3469386-1-Sergiy_Kibrik@epam.com/
> > 
> > 2) 
> > https://lore.kernel.org/xen-devel/20240416063740.3469592-1-Sergiy_Kibrik@epam.com/
> > 
> > 3) 
> > https://lore.kernel.org/xen-devel/20240416063947.3469718-1-Sergiy_Kibrik@epam.com/
> 
> I don't like this sprinkling around of IS_ENABLED() very much. Maybe we want
> to have two new helpers (say using_svm() and using_vmx()), to be used in place
> of most but possibly not all cpu_has_{svm,vmx}? Doing such a transformation
> would then kind of implicitly answer the safety question above, as at every
> use site you'd need to judge whether the replacement is correct. If it's
> correct everywhere, the construct(s) as proposed in this version could then be
> considered to be used in this very shape (instead of introducing the two new
> helpers). But of course the transition could also be done gradually then,
> touching only those uses that previously you touched in 1), 2), and 3).



From xen-devel-bounces@lists.xenproject.org Fri May 24 00:09:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 00:09:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728952.1134037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAIUx-0005km-Sv; Fri, 24 May 2024 00:09:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728952.1134037; Fri, 24 May 2024 00:09:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAIUx-0005kf-Pj; Fri, 24 May 2024 00:09:27 +0000
Received: by outflank-mailman (input) for mailman id 728952;
 Fri, 24 May 2024 00:09:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAIUw-0005kV-DH; Fri, 24 May 2024 00:09:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAIUw-0001O4-9C; Fri, 24 May 2024 00:09:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAIUv-000640-Vi; Fri, 24 May 2024 00:09:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAIUv-0004dQ-VJ; Fri, 24 May 2024 00:09:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dIwaN6cwckeTA/H7GxoghjwQWPL0Zf/zZPwZ/CwR4Ek=; b=vYHlfPLGdIWSRpgv24MlRuERL4
	Z0DH7JlJqX2OAgFoRFVjK8bzB0R9cgncl5vmEfv+zbLpdvC9Vo2FuSamH2cclcGLyQ6g+XWKUgv2B
	gtMlDuiF8A8IwV3oaBLloKrXgcVIyMBFaKdZcmqD1J09PJ32BNevYnev63wgqC4rjoRc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186109-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186109: regressions - FAIL
X-Osstest-Failures:
    xen-4.17-testing:build-arm64-pvops:kernel-build:fail:regression
    xen-4.17-testing:build-amd64:xen-build:fail:regression
    xen-4.17-testing:build-amd64-xsm:xen-build:fail:regression
    xen-4.17-testing:build-i386:xen-build:fail:regression
    xen-4.17-testing:build-amd64-prev:xen-build:fail:regression
    xen-4.17-testing:build-i386-xsm:xen-build:fail:regression
    xen-4.17-testing:build-i386-prev:xen-build:fail:regression
    xen-4.17-testing:test-armhf-armhf-xl-credit1:host-ping-check-xen:fail:heisenbug
    xen-4.17-testing:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-5:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:build-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:build-i386-libvirt:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-4:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-3:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-xtf-amd64-amd64-2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-livepatch:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 00:09:25 +0000

flight 186109 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186109/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvops             6 kernel-build             fail REGR. vs. 185864
 build-amd64                   6 xen-build      fail in 186087 REGR. vs. 185864
 build-amd64-xsm               6 xen-build      fail in 186087 REGR. vs. 185864
 build-i386                    6 xen-build      fail in 186087 REGR. vs. 185864
 build-amd64-prev              6 xen-build      fail in 186087 REGR. vs. 185864
 build-i386-xsm                6 xen-build      fail in 186087 REGR. vs. 185864
 build-i386-prev               6 xen-build      fail in 186087 REGR. vs. 185864

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1  10 host-ping-check-xen        fail pass in 186087
 test-armhf-armhf-xl           8 xen-boot                   fail pass in 186087

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)    blocked in 186087 n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked in 186087 n/a
 test-xtf-amd64-amd64-5        1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)   blocked in 186087 n/a
 build-amd64-libvirt           1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 186087 n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)          blocked in 186087 n/a
 test-amd64-amd64-libvirt      1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)   blocked in 186087 n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)        blocked in 186087 n/a
 build-i386-libvirt            1 build-check(1)           blocked in 186087 n/a
 test-xtf-amd64-amd64-1        1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)      blocked in 186087 n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)      blocked in 186087 n/a
 test-amd64-amd64-xl-raw       1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)          blocked in 186087 n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)          blocked in 186087 n/a
 test-xtf-amd64-amd64-4        1 build-check(1)           blocked in 186087 n/a
 test-xtf-amd64-amd64-3        1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)   blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)   blocked in 186087 n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl           1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-pygrub       1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)         blocked in 186087 n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-pair         1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)          blocked in 186087 n/a
 test-xtf-amd64-amd64-2        1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)   blocked in 186087 n/a
 test-amd64-amd64-livepatch    1 build-check(1)           blocked in 186087 n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 186087 n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)        blocked in 186087 n/a
 test-amd64-amd64-xl-vhd       1 build-check(1)           blocked in 186087 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm     15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl-xsm 16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-xl-credit1 15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl-credit1 16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-xl         15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl     16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-xl-credit2 15 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl-credit2 16 saverestore-support-check fail in 186087 never pass
 test-armhf-armhf-xl         15 migrate-support-check fail in 186087 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 186087 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186087 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check fail in 186087 never pass
 test-arm64-arm64-xl-vhd     14 migrate-support-check fail in 186087 never pass
 test-arm64-arm64-xl-vhd 15 saverestore-support-check fail in 186087 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185864
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   24 days
Failing since        186063  2024-05-21 10:06:36 Z    2 days    4 attempts
Testing same since   186069  2024-05-22 01:58:18 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            fail    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 438 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri May 24 00:55:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 00:55:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728966.1134050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAJDS-0003KP-ER; Fri, 24 May 2024 00:55:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728966.1134050; Fri, 24 May 2024 00:55:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAJDS-0003KI-AS; Fri, 24 May 2024 00:55:26 +0000
Received: by outflank-mailman (input) for mailman id 728966;
 Fri, 24 May 2024 00:55:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAJDR-0003KC-4r
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 00:55:25 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4cf91056-1968-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 02:55:23 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 4CFAD62F8E;
 Fri, 24 May 2024 00:55:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4FC41C32782;
 Fri, 24 May 2024 00:55:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cf91056-1968-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716512122;
	bh=6RxAEnmgcpovAf+YRLeuGIhS6hRBpuv9Njh3jcdd17E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=c7tjwfIcB2+K2mwOUXy08JWFlEijIEXfi5phWckLpgZvpuG8n/58SzD5b0vIsUFHx
	 2ZMyot2EUK6YytgcY/FlpKd4PFFLj1X3tCEVsWPdCbeq55iAhuFpHVMLv/+p1s/t5o
	 CO1sFT0hsw/LTn5xPQu9aUPuPHlG5s7vr2xgAjCGDYSArGvTNNET8jLInFVbaXn+Is
	 XbQkf2j3Q+Hlgozzb84ao7WLmcgFWeKWI2JyfC1peHFxid+agFSpqrFcU7ratH1WpV
	 1STzY/vcNJSIdqtlm5lp294Cm9wFWMiHpP6ooJP9UOk4tUBQpxXRjzv8euoToezDMa
	 m5l9brE0qgnFw==
Date: Thu, 23 May 2024 17:55:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, Anthony PERARD <anthony@xenproject.org>, 
    Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v4 1/9] tools/xl: Correct the help information and exit
 code of the dt-overlay command
In-Reply-To: <20240523074040.1611264-2-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405231755120.2557291@ubuntu-linux-20-04-desktop>
References: <20240523074040.1611264-1-xin.wang2@amd.com> <20240523074040.1611264-2-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 May 2024, Henry Wang wrote:
> Fix the name mismatch in the xl dt-overlay command, the
> command name should be "dt-overlay" instead of "dt_overlay".
> Add the missing "," in the cmdtable.
> 
> Fix the exit code of the dt-overlay command, use EXIT_FAILURE
> instead of ERROR_FAIL.
> 
> Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
> Suggested-by: Anthony PERARD <anthony@xenproject.org>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> v4:
> - No change.
> v3:
> - Add Jason's Reviewed-by tag.
> v2:
> - New patch
> ---
>  tools/xl/xl_cmdtable.c  | 2 +-
>  tools/xl/xl_vmcontrol.c | 6 +++---
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index 62bdb2aeaa..1f3c6b5897 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
>      { "dt-overlay",
>        &main_dt_overlay, 0, 1,
>        "Add/Remove a device tree overlay",
> -      "add/remove <.dtbo>"
> +      "add/remove <.dtbo>",
>        "-h print this help\n"
>      },
>  #endif
> diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
> index 98f6bd2e76..02575d5d36 100644
> --- a/tools/xl/xl_vmcontrol.c
> +++ b/tools/xl/xl_vmcontrol.c
> @@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
>      const int overlay_remove_op = 2;
>  
>      if (argc < 2) {
> -        help("dt_overlay");
> +        help("dt-overlay");
>          return EXIT_FAILURE;
>      }
>  
> @@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
>              fprintf(stderr, "failed to read the overlay device tree file %s\n",
>                      overlay_config_file);
>              free(overlay_dtb);
> -            return ERROR_FAIL;
> +            return EXIT_FAILURE;
>          }
>      } else {
>          fprintf(stderr, "overlay dtbo file not provided\n");
> -        return ERROR_FAIL;
> +        return EXIT_FAILURE;
>      }
>  
>      rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
> -- 
> 2.34.1
> 
> 


From xen-devel-bounces@lists.xenproject.org Fri May 24 02:17:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:17:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728979.1134066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKUI-0003Mn-Rx; Fri, 24 May 2024 02:16:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728979.1134066; Fri, 24 May 2024 02:16:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKUI-0003Mg-PJ; Fri, 24 May 2024 02:16:54 +0000
Received: by outflank-mailman (input) for mailman id 728979;
 Fri, 24 May 2024 02:16:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKUH-0003Ma-Ru
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:16:53 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac31f690-1973-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:16:48 +0200 (CEST)
Received: from BY3PR05CA0052.namprd05.prod.outlook.com (2603:10b6:a03:39b::27)
 by LV3PR12MB9095.namprd12.prod.outlook.com (2603:10b6:408:1a6::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Fri, 24 May
 2024 02:16:41 +0000
Received: from CO1PEPF000075F4.namprd03.prod.outlook.com
 (2603:10b6:a03:39b:cafe::e4) by BY3PR05CA0052.outlook.office365.com
 (2603:10b6:a03:39b::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.9 via Frontend
 Transport; Fri, 24 May 2024 02:16:41 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075F4.mail.protection.outlook.com (10.167.249.43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:16:39 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:16:38 -0500
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:16:38 -0500
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Thu, 23 May 2024 21:16:37 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac31f690-1973-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VGMT2Yk7yr05+5JxfPAA31OIMzY4dLl1ebSMMVQTjjNzkO2mb5CCWaPCLnPBgFnuPCZ9hI/Mz49t7mSnhx7yAX7d8KS92GRIyZf3KQxiL2bxJUCCuaYsDwlcGYNAIM8K8HjJoONfe5E9VOk9El1DA3cgrSNTmVGM3kgw/96sF7Z1OCnv5t6ntAaFSdabfaVhOj0ymkQHMn8NaEddy5smYX80a9+HbF/PU6Lfwyfbr3Xq+nrQcp6vK6u3nc92xm01/AoawPnuOUASY/tPpqlszFOZDB0SbW3PXFDVoGZC/WxZV1FRuAUrc6A3zqTBbMI/5iBsu5/mBQXHU5pfd19hFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ppfuxHZIBQNKASSxfSPTIvxjawH5qY0rEI4KTAaepq4=;
 b=UdmTR9dQxz1JHsL8/Hk/IDRzfQtOFf9GaCLPIDBtt6VVL4r5FGTRfN9QTYGXuORYYEmt1ZNDUOwt8yTQxz/GelmDn4wXbump6X4siQIKOH9ou9gNCw36SVqMmqDviXrPtwICg8fAURkKzLaifJyOt/BliT9XHDEmFSV6PqZniI1/UvGZSx1ezdQme31Jbi3+KHjBQUYKb8fyJD3/npk0Yegv1w/qzj0MouBeI3xC0gRJAyo95ia3+R1QzIt8BhT1jAFaUB+D/J6KfuNLUJ+aVa9My2vFJhEChcs8q5qwOMljOxnI4c9ZoYl8Bl1nWC2LOgaqKHbzV9QZThKtJ+6acw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ppfuxHZIBQNKASSxfSPTIvxjawH5qY0rEI4KTAaepq4=;
 b=qPaPuXFJyikb5iff+kfTpvJHy02QEMG3vgUNjP1SnRHj/QbW5pC6xYqbvFdcLZQQz67j/RoICznQbNSAp339yhYuIT8ml3KnYHX6T7PDdouHlTeT30PurQZK8YYlqYEEJ/i8AlM+eGbO8UbLlKWrxeNqEfADVX2ZnRq8+UfMDeQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Date: Thu, 23 May 2024 19:16:37 -0700
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH v5 0/7] Remaining patches for dynamic node programming using
 overlay dtbo
Message-ID: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075F4:EE_|LV3PR12MB9095:EE_
X-MS-Office365-Filtering-Correlation-Id: a74b48ed-88f9-48f2-e73f-08dc7b978c22
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|7416005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?MKO0EjWV6yNc48/xgPNK/fKyOXBuE5iukPttnxCyZGbaMDrT5EFBtl/Qa3DG?=
 =?us-ascii?Q?IrbvpVCY8DvT54g+ZU+9QQtVbYDnPfjNg0I+pd32H0AjTYgCM+OCBtCTuwcJ?=
 =?us-ascii?Q?3Ys20U64Plc+o43DVKWquuoeKNgSDO0OFL/7bjoHdti3cjUM6PvqXQdufnUP?=
 =?us-ascii?Q?9zKK+tlWe2JmYsygqcRwuodZvEu7OMQ/Bote/x0qwLjAxY4fbTfBSI4fdG95?=
 =?us-ascii?Q?XgfObadcMlASw0LeXrR8Afm92gWUkdoLTacSfRIYYRwAqD+6bANC5hSdhmuQ?=
 =?us-ascii?Q?X4cVdc7iqcNBOZMPGsThNCLbYccX8QihXbeqzvzr6IvCjeozAQ1ssgqaZBjg?=
 =?us-ascii?Q?Qi2oBAFHHljAQ6L6NF685HgQX7VuBSTCXV2T2Vq4eLrFgNIn7mwPXB9FZ08z?=
 =?us-ascii?Q?K5gR4CTJKyxH/7vFmZNDmJ1tq6EcablFSrgxOu7s3VSv3TtKx/4vbiL9m3jI?=
 =?us-ascii?Q?LC3dymy3/VXqJCHO3yU+UaI2v4pW7zGy7p3+3C+DNY9TSSyBUtdPvAskeuLa?=
 =?us-ascii?Q?wJYGRRZhffXy+qUw4gu59D+ytlqhIyqbLRIpcw0Rng6FasiDy4/G1vQ59R+g?=
 =?us-ascii?Q?6W4c1iL12Xwo13ABmYWlvHiJ1cF8ExJTk9KewJimmaqpWNRMGWicOhLXlmkQ?=
 =?us-ascii?Q?qGwXJS3tIigxVCSU/N+gcGsE705ITF4X8IyGteXOn8vcqZyY1jok8vmUDot/?=
 =?us-ascii?Q?43wtLT8dREqwQxeNRl7I+F/8tbrCk+mXIPOAWC8QZRSNo76wId5run/ESYyQ?=
 =?us-ascii?Q?fP/0NEb02dftblAthchaas/b+9OkhEgg2JIV5N5dlzLxzIbSuDjez/sdWd0W?=
 =?us-ascii?Q?V61EmEhZg8+QrUWFJtOmZQ5cGA+Cztq72Ieb38sIFhnWOg/+MVgC2c30LMft?=
 =?us-ascii?Q?m94eYWSmFPQ8F8PDXURIXV70BmmF2pr+xTdMPs3xjTP13rGYSV5rev0cxI3M?=
 =?us-ascii?Q?u0m5Ls9F9U2I7yjrvenCXlYILL+jOB0XVXSgjNuQctous+p3BmfvMwUZzXsw?=
 =?us-ascii?Q?qjN+QOlLZDk2cK/g+bCrAdpISEL4Z2YUGLpP/l7dHb8XbLYN3OvdowEARDcl?=
 =?us-ascii?Q?pAXZiwsgRXCPgGKChBzipjyds11SKicJzcrPpOzIZ2lWwWySx0lGXYNb8W8l?=
 =?us-ascii?Q?0OQUEM95/2Gmj72uBE99/vDtVQMfSfySATVijTkoRt4k7LlJ7QeAMJlhiA8U?=
 =?us-ascii?Q?rORCNestOkjuC02T8Mg8tP+MhYeD2oq8HhBmOII2o8XK/K1fJi10QXb5YpUa?=
 =?us-ascii?Q?ERwSPo2osgy/8qsT+WplzEXqladmp+n2AjzDYlXVndfg+sMxDj6QrJ4AFhv1?=
 =?us-ascii?Q?ii8IGZdM6Ll4St8kTC0jUl16?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230031)(82310400017)(376005)(7416005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:16:39.9268
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a74b48ed-88f9-48f2-e73f-08dc7b978c22
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075F4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9095

Hi all,

This is the remaining series for the full functional "dynamic node
programming using overlay dtbo" feature. The first part [1] has
already been merged.

Quoting from the original series, the first part has already made
Xen aware of new device tree node which means updating the dt_host
with overlay node information, and in this series, the goal is to
map IRQ and IOMMU during runtime, where we will do the actual IOMMU
and IRQ mapping and unmapping to a running domain. Also, documentation
of the "dynamic node programming using overlay dtbo" feature is added.

During the discussion in v3, I was recommended to split the overlay
devices attach/detach to/from running domains to separated patches [3].
But I decided to only expose the xl user interfaces together to the
users after device attach/detach is fully functional, so I didn't
split the toolstack patch (#8).

Patch 1 is a fix of the existing code which is noticed during my local
tests, details please see the commit message.

Gitlab CI for this series can be found in [2].

[1] https://lore.kernel.org/xen-devel/20230906011631.30310-1-vikram.garhwal@amd.com/
[2] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1301720278
[3] https://lore.kernel.org/xen-devel/e743d3d2-5884-4e55-8627-85985ba339f9@amd.com/


Changes in v5:
- address Julien's comments
- remove patches and mentions of the "detach" operation
- add a check for xen,reg and return error if present

- Stefano



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728983.1134075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW2-0003tx-5x; Fri, 24 May 2024 02:18:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728983.1134075; Fri, 24 May 2024 02:18:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW2-0003tq-3G; Fri, 24 May 2024 02:18:42 +0000
Received: by outflank-mailman (input) for mailman id 728983;
 Fri, 24 May 2024 02:18:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW0-0003tc-Gm
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:40 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ee6c294a-1973-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:18:38 +0200 (CEST)
Received: from MW4PR04CA0066.namprd04.prod.outlook.com (2603:10b6:303:6b::11)
 by CH3PR12MB7617.namprd12.prod.outlook.com (2603:10b6:610:140::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 02:18:33 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::f8) by MW4PR04CA0066.outlook.office365.com
 (2603:10b6:303:6b::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 02:18:32 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:32 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:29 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:28 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee6c294a-1973-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QfnTtZrjHcOZuTG1RV24o7aioszq+2i1wi9SL+VzVENbB1OeNZ0AVReOMnsIPL7G+l6OfoesjLxad6etSxq7qDsXuBPKYeL37r/s7b3WlnRAWgsUc9ZYMsNRV9ugD+vJL1YHwp8B594Lzxs58kumF2vDBpKJLOKAFaN1/w5y7kf2qTShZE7hf61jeU26aAw7xixiowl4mVovg3yoZbt2jazISTBv+Uu63VupuXgc7KdpmE5B38AcbSFIWAGRklt+H6KV1iKK9Vka4zXWgTYxdq/CzlQ9rGhx3z3lsKH7Ov1902M5cj2WhS3mbXSCY2/JsSCM4S9Vqs6n1AqnGzxbgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DJzU+Lo23+pblv8U5ELG1eHDNPAMdYWdjjMuN9k4W2I=;
 b=f9skBc+HMDg/vp2x+inOpaYSEwC5VF/9qu6pJBpF3dqnX9lLjmrSh761TqNmy7Plq8EuqiKnewY3lekPO/AqRnTarALX+2iwObvFLuT0pQcOBHDvlHhl2RfnIJU4FEnT8DnRKmdCOU9/341mVsPkkNNP5Vx3D9iMPCUYABOTqxgN8l6ha+UrKVqyPA5JVd0UG2EsHNN4Fhtw7ljOT4BL76P4/JNXOFE43jAh0wBSfNO2bttEHX9Bq257dJor6T8NIUwc/eaReYW9h88ZFpnA3nOrSSQze2jjOsqYNxDtqWCv5Z840kP/DLUEWIEpm4DKSkt0M1Kkz9Q+i1sYdXx7Eg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DJzU+Lo23+pblv8U5ELG1eHDNPAMdYWdjjMuN9k4W2I=;
 b=OW9r2Ob3j4jAh6VUZ6GJ4nhym9xZtlIgNbhUhys/8sFSnl7nffFfL0dfXKWHYaNkz8csVIkjuxPxHC8GSOqqEQpPjpVNvvHMEoYWhX1PaYln64q29HhZJGUi0Z8EJnK76nwmzVuKFms9Jiz0W2gqXoDF8QeXFWceKscPNAG1ghA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v5 1/7] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Thu, 23 May 2024 19:18:08 -0700
Message-ID: <20240524021814.2666257-1-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|CH3PR12MB7617:EE_
X-MS-Office365-Filtering-Correlation-Id: bc86422b-143b-4581-7f6c-08dc7b97ceed
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?uaq+5J7VJsp1R+HKQxmoUH+RO6C2TKR545pzmUQ5G22Xks5xPqG5QpmK7hDp?=
 =?us-ascii?Q?DnE+47ejbwree+MPqGh2q7aFJpuyI6CAdwYxZiNqgZk6aw3CRW/IZKRa+Wy5?=
 =?us-ascii?Q?zgTdwnvd07jTcV/CQJuNjnbtQScsfrk6L89m15vxJ3YO2NAMFOvoBeHrcBWF?=
 =?us-ascii?Q?BmXmSqYEI4EIDx7PDQfpJ3bySZbhyd0nWpgiyD+ZXvRx3ucyLyzJDy7H11gd?=
 =?us-ascii?Q?VUyLiLb5zmwwcnQyo4dSUH7nJ3OnU1b8iblErzSLg70bJbneMyJmLOFF0HCx?=
 =?us-ascii?Q?LudUVN6P+z3w2cTYSBK3MzKoqdivGohF+UhhDeM3qL0LBUODHQ8ruWT68nz1?=
 =?us-ascii?Q?n0klytLuHcM7NvWzbTklciVyQpA6XVAotPCzgMFLHOrCgIQU+kewjz11t/8W?=
 =?us-ascii?Q?FpcgNKzElpvXGaUv9iQ9wWXHkMsQoJHuuGQKG/cG+03H48KNfdWRsE7h2kNN?=
 =?us-ascii?Q?BtGV9noCK9DV8T64UplMKb3/skKxl9tWBB0amY9Nne+/kFnkhb2wJY2Bn4XD?=
 =?us-ascii?Q?joiyHDt8CElj59v0/61Ri83FTzVD6bVO0/rYTbDMGH2LaroFfji6SRv0TRpY?=
 =?us-ascii?Q?/ShNzPnQPO6iIAyDCA4EccW3USmyePLTPPPGz4t7tLzIZZBExMXHysG8uL/3?=
 =?us-ascii?Q?qgPgqb/tdSkJBrO06k6ZWJQfpBi/UjiZ+RgWbk2j6L5+daPWsx208ZpEX9lb?=
 =?us-ascii?Q?MH7grKIDq9rQ0q/ag2ZYLVSFSlGCaInHfePx5GzSvfltWKCmCJfCmO1dcNLt?=
 =?us-ascii?Q?CgAkryECoTpGYzszNNNJup4bqBKbIMABLHqvZPSKJIIO8cWKiUjDj1jIhCg+?=
 =?us-ascii?Q?Y3c1swBDADy1xU5SAUKPn57nSIfT/YEaxHH2NlrhndEI3hrLEeqosJFGMTEX?=
 =?us-ascii?Q?VP3maKkX+kYuff0Lpe/mWj2hECW//sVQ7DRwaqvxAc1CO97VhD1z93uKO+lZ?=
 =?us-ascii?Q?nSDzHgdkePhPGmPnMkpNYnPToLtydCp+jWY4HkHT8RuoJAq2IKnet8+yKC7F?=
 =?us-ascii?Q?G7cDYuXrpZ6nEVU3J26Z02jyRZ9ZCNP9Z5RxtX29RMMyOb1cx3BDKW3/mCG9?=
 =?us-ascii?Q?+JRINt/HJr87Apmu7wKiUkVxxxSezDyJ6XwoXbbVBVfXn4CCSbzZXy24oZ+U?=
 =?us-ascii?Q?4rjuQfJ+UrJN9XkH5EHNa+phNgl/1lAUygTQZjOA/czT0tAz6t6ewHmr6TQ2?=
 =?us-ascii?Q?n4einKpOsSvh9bvQyshJFSUbzx8Yh4iP/2o78DEFEicQsHosuTlHzAplgaMT?=
 =?us-ascii?Q?7diX0j+KlDrA1VApnOPru+JxR2oINSGrEsx02GtM9GkXhUsxF9CieQQJDFNe?=
 =?us-ascii?Q?f9gvaXXqPXjKnRcKdalYijsLNQ43bDnPdEdLccK4BOk6L/br1yybMs6OOuwO?=
 =?us-ascii?Q?EPIF7l0=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:32.0310
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: bc86422b-143b-4581-7f6c-08dc7b97ceed
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7617

From: Henry Wang <xin.wang2@amd.com>

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".
Add the missing "," in the cmdtable.

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony@xenproject.org>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 tools/xl/xl_cmdtable.c  | 2 +-
 tools/xl/xl_vmcontrol.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1f3c6b5897 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
       "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>"
+      "add/remove <.dtbo>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728984.1134080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW2-0003wS-E0; Fri, 24 May 2024 02:18:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728984.1134080; Fri, 24 May 2024 02:18:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW2-0003vW-9M; Fri, 24 May 2024 02:18:42 +0000
Received: by outflank-mailman (input) for mailman id 728984;
 Fri, 24 May 2024 02:18:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW0-0003tc-Nt
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:40 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ee563b6e-1973-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:18:39 +0200 (CEST)
Received: from MW4PR04CA0088.namprd04.prod.outlook.com (2603:10b6:303:6b::33)
 by DS0PR12MB6629.namprd12.prod.outlook.com (2603:10b6:8:d3::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 02:18:35 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::57) by MW4PR04CA0088.outlook.office365.com
 (2603:10b6:303:6b::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 02:18:33 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:33 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:30 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:29 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee563b6e-1973-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j3WIzEpIHCPlXtoXUXhyn/HedDkRCrCvBT/QC58XOqfNHcZ03lmY6cm+XorMsd9+SVcQ6Ar4IBTc5D1nUIETuu88KwH7rqpxbspJXNqqHEBn9n7hrOLu170rfMy5dgsbZrUXPYF3VuSofJsgPKpjhsFr6API/kr8E3uXxyX8ai2BYhZinzGYFMETz4jHYkbbr07EbqBi1+mFfDZ/8akekNhCN2tOYE5nuS3plGJ1cg22t5XZbnO5agCHeBRKKa/VtAyH08jAQ2Fuy0Qm9GUkKGqPBax3wM33K3ReECjYTBV8rlizs5RWEdVxsvFGvZw+fD63WfDf4xshF0XUgHVO/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xRJXSt/i9q6c1C8Y7abmDgKw7TBf2B9wiQ/rWE+b2Go=;
 b=CG63nlEYG1l25gWL8g6nobKFBpKDm4Sd8VuOGcj/t2NKQvTRrkcAy5E4Whj+1FxnfblkZWhoyStb/3Xvx8HQKTihkwJLdvhttNr53VbANBZoY5c81bFdXCXdGEiMaktQ7Zbesyk8wg8RvJOe8cNApdCLqNIn/xpcQBVzyGsBsinF+qDPuyszjMH2x17i1whbtV44F7/FYC70Gb4rbKObBWBH2NNSZVovEneBGbDCyAHbG8nlIONaVAVK4m49nP31cyJXiqr5XE2IY4TiKeabIy+zYaMSvGySaCTX91udGXpa6PzUkRTJBSteUMgf3e+hNpxyIrIu1PXymPb2odJukA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xRJXSt/i9q6c1C8Y7abmDgKw7TBf2B9wiQ/rWE+b2Go=;
 b=zjhRv5QNzmd6ikViC6Ww2u9xJ9EPI9RXq1n8ziDkhHxZLw5bz99FBgkkRJ4kXPTz+33n3Z+4hlBxl1fifi9E7rptnZUqtuvUpvi0J6EL4PE5i8ougj+X0PdIOu/QOJ6wMY3ND4SzM6uRU5bBzDo4fW/7ihfaF1v0Pt2QKQb1MkA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Julien Grall
	<jgrall@amazon.com>
Subject: [PATCH v5 2/7] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
Date: Thu, 23 May 2024 19:18:09 -0700
Message-ID: <20240524021814.2666257-2-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|DS0PR12MB6629:EE_
X-MS-Office365-Filtering-Correlation-Id: d30934c1-8e23-4436-4e62-08dc7b97cfbf
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?ZhWZs+o9k/4kk5rtRKSY62PMffnLmD5TNjCHCUppI8IrLmLkm0MQSao8twv3?=
 =?us-ascii?Q?v/M8mREIxzJKLtxT+fd/R9xS/vDnzL5rxyPlYTpbEten4/YspKOHqhUPoNiZ?=
 =?us-ascii?Q?ZYwB+AJIjgA02ixesTg+HSlgSX/XdrHqYfJf+3pImFS5j6FRu8QYPWJFdIHp?=
 =?us-ascii?Q?PDwQn2iXrD+FaU0bGVzkomDuXhPdgEHjtuwY2EPDhC03MY74frnXwAY/J6pz?=
 =?us-ascii?Q?WlrjgRqG16NDJWMDn2F0x3T4m3cOTVABJ/go4/Spdbi2gxD6bx0TY+R926oR?=
 =?us-ascii?Q?2IrwbsV6JDO56Ggg2xRG3GJZ3joFAbcVx4YJNeYU76or6tLQiL5CbndFCKFL?=
 =?us-ascii?Q?tqdDwtd/P+Gdxvb91SvNzuMb5Aqy82ccai38e8BwBzNW5x80UtvmLDWq+s3Q?=
 =?us-ascii?Q?+8+7b7wNr3unv3tBxDXWzYLqhp+XPk+5mtIM3fh/fy1bn92UeN6d0VZEvPM/?=
 =?us-ascii?Q?Y2GE3ZTzqAVyptCBe7OdPLOVbKp5EzPR1GcQ4ErOK5vHoAMl1SmKmor7A2KQ?=
 =?us-ascii?Q?ZLQfqr/nva9Q3Y8fQCtyrbvQ9UlZUPFjfN6hV3FDPuFwd2B3RNxgrbIZajJa?=
 =?us-ascii?Q?O5tZaJVIq1f3ZcbpBCc28olOOCY4+LSxGiBuuOi1RckddiVk961rR4lCLn1i?=
 =?us-ascii?Q?xOtLTeQnrYuXVrpE3Ebu/isr0QQibxK0+Pdq3FRJHIe0tW9UufUEY/6Q2FEI?=
 =?us-ascii?Q?X1Qvp9aRinAJUBm+CLO516Fv5jhJvgfAxIF9KtFOadu6z4XUfYJyiuhehQxf?=
 =?us-ascii?Q?ZRsnLmTSmjZStQYRbU44H9SIoNzrY8wt31RohdG/hm4kA+bp649lPo3Je6Ul?=
 =?us-ascii?Q?/IUOGeIaCdboCdT6UHtv96EtDdY3WZiOZ6VrsprnAkYvrkhBcdOapZdnl8U+?=
 =?us-ascii?Q?otR0KGob++MrgNOn8XDmwI3etKXNeVDJVy73b8GpjVwig3NbPXLY3hlgRgOU?=
 =?us-ascii?Q?iXZC+eevJ4AbsKKA22xq9kM4P8KHC1rlaHng4QZcfMR03a0dIsYiMQQfJIK8?=
 =?us-ascii?Q?cjBVWEmXU7Y3McZ7iLKtmnHcZg1YpZs3P6ckme0/F39+MvCUfL5PMF7lWH6z?=
 =?us-ascii?Q?wVTof09EeBCRnw7oqFvY5GZyyLv1QE8ZiPFxhMQ1ftkoBS7IBdFbxwsFr1fv?=
 =?us-ascii?Q?/wUkdNNsupJv2moe40FCWG+7XHob7LI3JdWxdj2solkxk2th/rdaQCxufZw4?=
 =?us-ascii?Q?qPhVMSWYjnxkEQcFVfQ1nX2I+iMXt4wlOBAo8mgqPEvtYdblPOd2WxQntlL/?=
 =?us-ascii?Q?a/2NvEoEl49qZGiZnR1a21SZnk5Snc0fFE0e99rhf9vsRT0rETwjKex0Mox0?=
 =?us-ascii?Q?AvzgcU0eYtdYtjqC+RiLo3QI0IzErkIoGwAWa220Q3ZJDSg1IoHMQWIEiZRW?=
 =?us-ascii?Q?u5Q7CZC78OW5yojAaPlJPNBnWPqH?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:33.4060
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d30934c1-8e23-4436-4e62-08dc7b97cfbf
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6629

From: Henry Wang <xin.wang2@amd.com>

There are some use cases in which the dom0less domUs need to have
the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
example, the dynamic dtbo feature allows the domain to be assigned
a device that is behind the IOMMU at runtime. For these use cases,
we need to have a way to specify the domain will need the IOMMU
mapping at domain construction time.

Introduce a "passthrough" DT property for Dom0less DomUs following
the same entry as the xl.cfg. Currently only provide two options,
i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
construction time based on the property.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 docs/misc/arm/device-tree/booting.txt | 16 ++++++++++++++++
 xen/arch/arm/dom0less-build.c         | 11 +++++++++--
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..f1fd069c87 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -260,6 +260,22 @@ with the following properties:
     value specified by Xen command line parameter gnttab_max_maptrack_frames
     (or its default value if unspecified, i.e. 1024) is used.
 
+- passthrough
+
+    A string property specifying whether IOMMU mappings are enabled for the
+    domain and hence whether it will be enabled for passthrough hardware.
+    Possible property values are:
+
+    - "enabled"
+    IOMMU mappings are enabled for the domain. Note that this option is the
+    default if the user provides the device partial passthrough device tree
+    for the domain.
+
+    - "disabled"
+    IOMMU mappings are disabled for the domain and so hardware may not be
+    passed through. This option is the default if this property is missing
+    and the user does not provide the device partial device tree for the domain.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..5830a7051d 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -848,6 +848,8 @@ static int __init construct_domU(struct domain *d,
 void __init create_domUs(void)
 {
     struct dt_device_node *node;
+    const char *dom0less_iommu;
+    bool iommu = false;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
 
@@ -895,8 +897,13 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
-             iommu_enabled )
+        if ( !dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
+             !strcmp(dom0less_iommu, "enabled") )
+            iommu = true;
+
+        if ( iommu_enabled &&
+             (iommu || dt_find_compatible_node(node, NULL,
+                                               "multiboot,device-tree")) )
             d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728985.1134095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW4-0004Oh-Qz; Fri, 24 May 2024 02:18:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728985.1134095; Fri, 24 May 2024 02:18:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW4-0004OY-O3; Fri, 24 May 2024 02:18:44 +0000
Received: by outflank-mailman (input) for mailman id 728985;
 Fri, 24 May 2024 02:18:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW3-0003tc-Aw
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:43 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f02977bf-1973-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:18:42 +0200 (CEST)
Received: from MW4PR04CA0078.namprd04.prod.outlook.com (2603:10b6:303:6b::23)
 by PH7PR12MB9202.namprd12.prod.outlook.com (2603:10b6:510:2ef::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Fri, 24 May
 2024 02:18:37 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::5e) by MW4PR04CA0078.outlook.office365.com
 (2603:10b6:303:6b::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 02:18:36 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:36 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:32 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:31 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f02977bf-1973-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=K+oOt94oKv7T9anVoojoQ2aQupFhYcyms6MN4ZJUxkgGKyV7SOeL1eoG1czNxiDCT3Wv41uE/WssM3QHtG1HYsJmBvxeXnRav7esw0ITS8cExVKmPsGyNGTAABMOEGv9JsOczd17BLY24gqdG8zlype73oh12aWAHfIX/mf79DrtEWoy37H+Gk+QFw2Gdqx9f9O7V2uYEtXw7chSTLSNcjcopqKJ2bvHUApbpFOpYFTCGm/WYJkorgHAiuYrTSNF+i9OcCyoPZ2PV29wAcfzjAY6tQogjuyDvT4+b6phGoyuPJPgOkr8C/A1q5bZeZ9vDFgEg1w0b2Vvj1PRuoOiTQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dUxLvykGPrn0TdFclj7NWwLQlt3XRena1NmCNYfl8WY=;
 b=kOHAkf8zFzgffGuktIgxoefAhdk7IO3sZwPyYyrQEZtfy6CphD985vHT2HIVG1Rve4b4cRcKdxQMJvbPDGDbiP7ZeGOUxItXDkulg1tjRDuVF8YMVBWczd55daSr5mzzWqtvr7P3ubKdyPIkHowD7B2HUx8esZSNRdgYvOCgcBr3eibBgnXPzYI53ow5wknpvp/rGAZnYJSNYs9G3bEpEq6qRVh+Dm5Iy+gG9hHrQWYhanWatm19yP6eKEpaSaiAYmuw4DN4F1V+C8oHyqpEbq/UbKDNArXv+0J0biTVT5Fjykh2ZnIoltmuNxmk6JGjsrCvvn4H5i7ed0vyc5BtQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dUxLvykGPrn0TdFclj7NWwLQlt3XRena1NmCNYfl8WY=;
 b=xNoUp4SBKWr0BOsbJwkmTaOnbTNp6wYpQjmn3AdCAvG08lon9xk40uOTwGl8f4DjnMzH4k/6jdLl10EiRXcIoGvscVRT4BJKLEVC82K4mD+oPk4Z91zjqsUSXH0y3u7pmZi7YSRl6oNs/oljEIkqxxrkAoIc0E/2Z2u6Ge8W3JM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH v5 4/7] xen/arm/gic: Allow adding interrupt to running VMs
Date: Thu, 23 May 2024 19:18:11 -0700
Message-ID: <20240524021814.2666257-4-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|PH7PR12MB9202:EE_
X-MS-Office365-Filtering-Correlation-Id: 6ea84ffd-f719-4948-71b9-08dc7b97d1c9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?5Kfs9OGs6EUji5agDbShs9dUhfuVFkFUkPASr4/FE9kDWmBylwHxf2AnAG2Q?=
 =?us-ascii?Q?EHxqTwSng8cZMvmoqbpdhMdSgX4httlMo9Kh8P0xU5Hhh1kM1LudrjUyjKrj?=
 =?us-ascii?Q?CZyT72h8XUyLPFbQQeDGK5D6iQAuXd70FOtthS+r815pbM+e19tRslRO0b9A?=
 =?us-ascii?Q?SThM2jNAbjPN0diH4p4JWInRcC2uEWTupT87amcXdDeCF6qefNBoDtJCQpuK?=
 =?us-ascii?Q?6M8H7ms4/Ib/CGtKYBd9CwdHNrMTIzDIq9hun0z4KMSi5NHce2ry61KxRGVV?=
 =?us-ascii?Q?JRWHwNgQmLasYHI1mgm2Rb5rUhz10dfzWFeGZ3iFWpvdJtaO+WxYzIPX89NU?=
 =?us-ascii?Q?BMGIaZfxfSQRsSl9OD8PCYJZIlKBHrhvViYwJWbFjWZNeFDQHgbkt8O8UbIN?=
 =?us-ascii?Q?wEOxx/HK7r2Pl/s50965EgK8OMoq6k4DnDGCLNsmoLv/NrDAIi81qVdHtZBe?=
 =?us-ascii?Q?Fc86+jzSygvLXxSqzzI2nPdm63vxgrHmCO/ZZH55LD84CWPDo1BK0MfaeQga?=
 =?us-ascii?Q?s7QurGFQpSE+ca9k43licMmDctNuOPTXgEN+NVWOtLPthvVh389vucFdGHla?=
 =?us-ascii?Q?6XtIMSLhUf9fquyJYIhBr1IhnI9Ufv0u+ftV1T4SXLcy8DwSw1qSQuazzPbl?=
 =?us-ascii?Q?KLBUDE/I8SDYHG/qMk+YXLFV7PP2/nySjI55hhnMdGfx97JQAQ+CftYsy3bA?=
 =?us-ascii?Q?Leb96KISCwmue/xSswO0D9t8XDZYcddOFOqYrHxR5FhfLgWir8IaMoPHL78f?=
 =?us-ascii?Q?NysTlhpIpNed1yBf+F49z86vJjm3xqCXAAWIssdu9chmEJccGwfSzGKkK+Z1?=
 =?us-ascii?Q?x+Y7Gpm5JFAgDZdFStcKZvpfhwfDRFa6S19LRGYhX5BB+efqQsnljHzgyavD?=
 =?us-ascii?Q?BmqHNoQVCb2cFFKc3/XNvozVaeT+poQwmur8S/vnPCsrX2wPNTuluUd/irx6?=
 =?us-ascii?Q?Xzc1xb4+JZ5qBZZPykLN7SIG1oXBFTZalE2eNelPkxQxMtYPsdl/7UbT1k8i?=
 =?us-ascii?Q?51dVd2RGamVfWXIBKicmbQAHrzGqLj8pHYTKeZ6sqckQ6Mj9i4wj2ivmcgHV?=
 =?us-ascii?Q?hOMY5t4YKvwdNuTY/0EyLH1tZidxZ9fHH6cxVhmW+p0ujDk7DVfKAgtZhrst?=
 =?us-ascii?Q?LCp/9St9xoIRnCcp6JhRsyYpoR71scN1qVj5DJMKB7/2KOmiKu1iYdRiMXqq?=
 =?us-ascii?Q?xifCS2geNMDxRAA9Od7C+ljLWb+pAp6sxk5orf0A1/Z3N2pFRcMOIwSL7pAM?=
 =?us-ascii?Q?YxO3pXjEnQKHudUsO/qpfwxH5G8BO6M+oRXYFOoFzycDMIM3CjBVJpgd0LBo?=
 =?us-ascii?Q?wGOtMMTSZaPrsZIUS+6V1ZfWaza8vFKCSkqoOcWUg3TfYjLxJwhXL3fr3WJU?=
 =?us-ascii?Q?ZqCO7vAGaPXMMwDcZSg+gfDim3Xv?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:36.8123
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ea84ffd-f719-4948-71b9-08dc7b97d1c9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9202

From: Henry Wang <xin.wang2@amd.com>

Currently, adding physical interrupts are only allowed at
the domain creation time. For use cases such as dynamic device
tree overlay addition, the adding of physical IRQ to
running domains should be allowed.

Drop the above-mentioned domain creation check. Since this
will introduce interrupt state unsync issues for cases when the
interrupt is active or pending in the guest, therefore for these
cases we simply reject the operation. Do it for both new and old
vGIC implementations.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 xen/arch/arm/gic-vgic.c  | 9 +++++++--
 xen/arch/arm/gic.c       | 8 --------
 xen/arch/arm/vgic/vgic.c | 7 +++++--
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 56490dbc43..b99e287224 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -442,9 +442,14 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
 
     if ( connect )
     {
-        /* The VIRQ should not be already enabled by the guest */
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
         if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             p->desc = desc;
         else
             ret = -EBUSY;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86de..b3467a76ae 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
     ASSERT(virq < vgic_num_irqs(d));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-    if ( d->creation_finished )
-        return -EBUSY;
-
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
     if ( ret )
         return ret;
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b9463a5f27..6cabd0496d 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
 
     if ( connect )                      /* assign a mapped IRQ */
     {
-        /* The VIRQ should not be already enabled by the guest */
-        if ( !irq->hw && !irq->enabled )
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
+        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
         {
             irq->hw = true;
             irq->hwintid = desc->irq;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728986.1134103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW5-0004Rs-7U; Fri, 24 May 2024 02:18:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728986.1134103; Fri, 24 May 2024 02:18:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW5-0004Qj-02; Fri, 24 May 2024 02:18:45 +0000
Received: by outflank-mailman (input) for mailman id 728986;
 Fri, 24 May 2024 02:18:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW3-0004I4-I9
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:43 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef6d3d0c-1973-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:18:41 +0200 (CEST)
Received: from MW4PR04CA0087.namprd04.prod.outlook.com (2603:10b6:303:6b::32)
 by CH3PR12MB9341.namprd12.prod.outlook.com (2603:10b6:610:1cd::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 02:18:36 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::58) by MW4PR04CA0087.outlook.office365.com
 (2603:10b6:303:6b::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Fri, 24 May 2024 02:18:36 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:36 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:31 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:30 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef6d3d0c-1973-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jZ1cVyFSk45uNtO4JrLahA3+fbml3QYQGvxMfv7PlKKRMWX9MGtHcvRMPr7jnd4qchyQpGkg2y9vYpWBF/qwMZxYNptbD3dbsjtqpcTZdaDdEBN3tQXhyPKzSmEooGARLivQ+y4hKBJhbvA4oYp9AmUqaEtDVvhMUrakTgXDLArP3sji41u6X3l67JofKRzyot5QmWPXH5XjluWfeT88rzRcJ1Mcvm65yKoH1O5NPLoUd2NgZY1YkwraduWypRhPzAh7lSorZY39Wm/n20HYE2gmLvaGqBuWuvrbOArFtMy56FtxvQitrLekh20M1dddzPFz9VkoqCwCBkDVPFwH7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=0mNtauRAgY1rxPk1todAfJ8ZkfStKgBmhTiRwKw4AV4=;
 b=BWZrqlIqrIxiKqV6e+GcVw2sR19dpRDtWWzNGbK/4h09inHRlIf0lWYL4z2/yt2qnvXbCNp+tW2pP/hdeEDq/EPVYBwQ3QXdaYtsS+0gJQA3/+9T7ZgxSzUk5grnJIwThfwA+wdTAyguTsjq9XYS24Kl9zhzhyCo+iooMmmmZy0OeH8u2yxW5IbZmYyn19+ijU6/gS+/9ZMBrFriKYDCm4SVxEDjvOL2ttkSbWimuqbcjc1jlk2mdoqrsZSBx7e4Ssxw9RQ1cB9UJelfa0RXxvnmeRwbXs66PTQGuXY6uH7PRz50eWgZGemfw9NkMOzr7Z08fTcjVPI8SRsvJWM7cw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0mNtauRAgY1rxPk1todAfJ8ZkfStKgBmhTiRwKw4AV4=;
 b=WjLVyQOMxWRcT3Z7Uoh7cQgmq6ZMIfqxqguqpuJ+QivQuyx2+P/zI7B/Am596KkRSES/Vyt5f0WvWXc8mBIXdFNSe77iCAcxsOtejttJ0hsl3La/6anEcokRfYOfJNb/nXalpBR7hZN8OH4dKcZylYrVS6pmSyrfrnTFsae2wuw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v5 3/7] tools/arm: Introduce the "nr_spis" xl config entry
Date: Thu, 23 May 2024 19:18:10 -0700
Message-ID: <20240524021814.2666257-3-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|CH3PR12MB9341:EE_
X-MS-Office365-Filtering-Correlation-Id: 0533cb20-300c-4f3c-9cf6-08dc7b97d173
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?xvzB2S8Mdd/V1lOkTCHoHsYjbP8BJ82IUoJz4SG44qqujlM/JeEmeg/o4hWK?=
 =?us-ascii?Q?jOUEOXhgOOOt98smd6i/4mfeYcM7TWc+D5dG3t65waDdDu3icSO/y3UfnSVy?=
 =?us-ascii?Q?1CVPp6Bo0ntXNfregaD8ZmjkvD1mY3bT5oi8ODXDQscq1UPST2AuOEjqanyI?=
 =?us-ascii?Q?SqhL8Q5fC7pKJSsZO4q04UEwOxB2Cmv/eS9BrQksOjKFzSM0hxBkyaisU2eE?=
 =?us-ascii?Q?4t2pIEuXT/YRXERnaWznhEsGh5C8I3kvFO2E/zBi8OGOlCfNQSOr8egoYnTf?=
 =?us-ascii?Q?DYuzGxz+u7l+HwkzOYDaQYoyIu5eTv7FS2ztk6EXGVXMc4LcnWC4MUrTMCpx?=
 =?us-ascii?Q?uDDjo4Fo0lmXiprdWzOCOy7qQQDDtDMNQjyiNp1inkvv1FIOVrydzgu7ZrcT?=
 =?us-ascii?Q?fR85vlT02Cfj6X/VKjDSOfgywu0Qk/+qltZhVGe7G+Nz7wAclhFBX7GN0o5z?=
 =?us-ascii?Q?Q9zAw2khbYv5iqzz/sFEITNPFIs78rgLctOxM1MW59Tw7DkkF3mWMu37jjI2?=
 =?us-ascii?Q?+9uaBoREG4xB8sND7RKhxgSwQAJOr6cu0Qqn/LLablqVNdi6QN57crfeA7fh?=
 =?us-ascii?Q?JsPLdNLS1K6yiym8GXSr/xEUVBoxkYgH9Ye8hdTXTWMNHuZEXTVFG4PwMjTQ?=
 =?us-ascii?Q?+nsP9Xo4cTM5BwsjyZFxbxNrCqZkrbNLHjDc5P7i1KpW+8r5ftcsyphDuPIb?=
 =?us-ascii?Q?w35C6OxktiV6Qd0CqG39INMB49QoAYgUW1DUqylXbYt+nrIBk80XyMzz2WPI?=
 =?us-ascii?Q?AdwhBmf6tUgLJeLr7MMu086V4CJJplYl4lPji7dojF+dQU9jjZsXGOnxPzOr?=
 =?us-ascii?Q?+K92pAezFiDT9l4JGpjnqJMmr94wwgBGlVpX0sDeMfkgAQWYoqM7CzTCKUva?=
 =?us-ascii?Q?6As87XSQVqnveykz2hItApdOCYZ00e54NvQoNe8B8A0NVmhHRpvd1go1EVWR?=
 =?us-ascii?Q?PYmBravb0PFbmAEQD8APkAhnWR5/Eb+Qv5YtMLJM69PcGhOsQ0yNUPesrD/E?=
 =?us-ascii?Q?hQusKKbHN4/0Vb2zGzDGS+Dxx/PDujFxifLrrbiJ7J4t8RueB7Er9Sh6L0RC?=
 =?us-ascii?Q?StGMhrHQDI4LyUqmuZCKk3Q0IIt5L2EFkFECSWt5XY3pp19W6QAIDyj8N1sK?=
 =?us-ascii?Q?bdOcvM58KN57KL2OWN6xqF+1WjV7nzpUgAB2v591BLCFRORrIC6q4zyIqLYe?=
 =?us-ascii?Q?Jib9lZ8VBrpWUMNFdNRiAnT9uhTYdSS03TvFVn/IquK726zV+t+CgYIfgKLS?=
 =?us-ascii?Q?iJC3qkE4tifNkJUS0ZmQDGP9Rd0QVbdELO2uUOkEZo3/O82oFBV+ZlR/VqF7?=
 =?us-ascii?Q?6Bp/D6tUY63IcCIxo0oO/GjEpKmm0Oxsn9kNWYLPr7iYl3Y+Cu2RnMJeWQEx?=
 =?us-ascii?Q?D4+l13/EOahBPH0nDw/+lLeMnWYH?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:36.2498
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0533cb20-300c-4f3c-9cf6-08dc7b97d173
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9341

From: Henry Wang <xin.wang2@amd.com>

Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.

Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.

Update the doc and the golang bindings accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
 docs/man/xl.cfg.5.pod.in             | 16 ++++++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/include/libxl.h                |  7 +++++++
 tools/libs/light/libxl_arm.c         |  4 ++--
 tools/libs/light/libxl_types.idl     |  1 +
 tools/xl/xl_parse.c                  |  3 +++
 7 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..ac3f88fd57 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,22 @@ raised.
 
 =back
 
+=over 4
+
+=item B<nr_spis="NR_SPIS">
+
+An optional integer parameter specifying the number of SPIs (Shared
+Peripheral Interrupts) to allocate for the domain. Max is 991 SPIs. If
+the value specified by the `nr_spis` parameter is smaller than the
+number of SPIs calculated by the toolstack based on the devices
+allocated for the domain, or the `nr_spis` parameter is not specified,
+the value calculated by the toolstack will be used for the domain.
+Otherwise, the value specified by the `nr_spis` parameter will be used.
+The number of SPIs should match the highest interrupt ID that will be
+assigned to the domain.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index b9cb5b33c7..fe5110474d 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
+x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
+xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 5b293755d7..c9e45b306f 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -597,6 +597,7 @@ ArchArm struct {
 GicVersion GicVersion
 Vuart VuartType
 SveVl SveType
+NrSpis uint32
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..3b5c18b48b 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -636,6 +636,13 @@
  */
 #define LIBXL_HAVE_XEN_9PFS 1
 
+/*
+ * LIBXL_HAVE_NR_SPIS indicates the presence of the nr_spis field in
+ * libxl_domain_build_info that specifies the number of SPIs interrupts
+ * for the guest.
+ */
+#define LIBXL_HAVE_NR_SPIS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1cb89fa584..a4029e3ac8 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 
     LOG(DEBUG, "Configure the domain");
 
-    config->arch.nr_spis = nr_spis;
-    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
+    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
+    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
 
     switch (d_config->b_info.arch_arm.gic_version) {
     case LIBXL_GIC_VERSION_DEFAULT:
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..4e65e6fda5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
                                ("sve_vl", libxl_sve_type),
+                               ("nr_spis", uint32),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..e3a4800f6e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2935,6 +2935,9 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
+        b_info->arch_arm.nr_spis = l;
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728987.1134108 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW5-0004ZQ-Ix; Fri, 24 May 2024 02:18:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728987.1134108; Fri, 24 May 2024 02:18:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW5-0004Wz-CA; Fri, 24 May 2024 02:18:45 +0000
Received: by outflank-mailman (input) for mailman id 728987;
 Fri, 24 May 2024 02:18:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW4-0004I4-8L
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:44 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef9c251e-1973-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:18:41 +0200 (CEST)
Received: from MW4PR04CA0086.namprd04.prod.outlook.com (2603:10b6:303:6b::31)
 by CH3PR12MB8234.namprd12.prod.outlook.com (2603:10b6:610:125::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 02:18:38 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::b4) by MW4PR04CA0086.outlook.office365.com
 (2603:10b6:303:6b::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20 via Frontend
 Transport; Fri, 24 May 2024 02:18:37 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:37 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:33 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:32 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef9c251e-1973-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SiHs1HzmOyszYoEjU5xOTYRtzcqcrgOvQv3lyrAS2kF2ddF+EBMyhi4SGPld5fX9NBqynMypFD+FoqqnerlmE6pY2Urr2sYXe++KBH5A7dyscbS+8QEQHonzZcvnWOQ40tjLRE+q+MR8fUxllHvMWxEwiBnltuQLxia746T5LTNhaqAknGmVgXTr3YmsoZ/XIU3//CunJ+MGAXJs2cuKhpVCwjn2g1cA5VWKXLX3bJDpqqQKkFjKBUyp5YJHN3Z1AWcvUuwCoLyaDSxdziodumkJ2TlijLezdKIdx4B0WXT+DMI57ZD53sdnTMrkyhqI833Dz5L0iR9mJ1yH3oqOww==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6RAlekm6oSAtsD1z/+65lehwcriofxvGBLLV+0kmPzI=;
 b=Q1YqmJMlKiPu4Tr2tfKYtCac+6uWrH1bKBGMK6nCNKbNSiDq4wKVXp+scGj8DAmdGqykntLmtWH6yOx7rYytGUDT8jhQJR3+hM3SgjLTLBrqzXNA71I+VMWAvTB62N5zm+r/5v6oDAjAQErcq8uEZUuvic7e/23oEyh7fQie/4F+fjf/hh7Yr3x1KVqccZG1+xxKVal2VC3ilhyF1M4nH7464bZuVHfitR9aY1MyofoX1ZagX+Up+ZVC3zzTjEPzXsQt55tID7mtZWewW+BJ7Ga6UeclYYIwHdK5yPlaIeEB/89LFm4ra3anF93BCD9BVxnKaEwybYzQjt+O3ZlZqA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6RAlekm6oSAtsD1z/+65lehwcriofxvGBLLV+0kmPzI=;
 b=AdOkzR2v7KN7omCEqZMBSj8Z0o66l7aJaYZBThUqIRVe8oZWy1L1Q4UT6zDhnSZppCyZf9lmh22d+tvqmmGKERkrPk5RD25eMgm6A/E9B32Qx1sRyGUBxMjx7NSn0cdO0pdtU/LIhLfdb/I9/sh4Ie/dy041kp2gEYKIdnWMRLI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Stefano Stabellini <stefano.stabellini@amd.com>
Subject: [PATCH v5 5/7] xen/arm: Add XEN_DOMCTL_dt_overlay and device attachment to domains
Date: Thu, 23 May 2024 19:18:12 -0700
Message-ID: <20240524021814.2666257-5-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|CH3PR12MB8234:EE_
X-MS-Office365-Filtering-Correlation-Id: 243daead-779b-4673-ba46-08dc7b97d253
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|36860700004|376005|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?jf2NIh+QHSzWhLOFWQJGnQ9Z+seGPhGnO1JTOS03Z3WEPNElMpQYSDmxVudX?=
 =?us-ascii?Q?1c3rumi27SXYS81SPkzfuR0V7L0MuLtVmEdV8SRtTMqnzeChFQqbweHgAh9r?=
 =?us-ascii?Q?8+Xsg3TsG7FUAvwppaRP8ZulIkPpKjrJ/KC1eF2aC0zqiURfmYMFtqLX09SC?=
 =?us-ascii?Q?vJ54+XKwBaDGOuXVNdy3y8QFo5/eFInZl1Xc0IniX3etGCkU+s20oueH6UsS?=
 =?us-ascii?Q?+f2TUN2kDExJzF+Oz4sIn4HJa1DK2yL/aCJK7atDcdUaKcSL97jx47gCKWBA?=
 =?us-ascii?Q?hSL38997nUGG2IzhKvnvnW0Datej0sYbLmCDNby/O6wRYnIv51S14fK4rGGF?=
 =?us-ascii?Q?QOkL0PUXDcn72zgpybkQ6bI5L4aB1Tev7vbr1sktbcYTNsBlEV8OcochhDve?=
 =?us-ascii?Q?Av3/iLq1u2IBrGj4rRaelEkVOlKowOxbJi+gQ05ev7CwF0d6XIMlNElBEOrM?=
 =?us-ascii?Q?aRtkIq8POGhGwymlL7S9zDFmR4B+VuF+mp71Y41+rjlmJhFh50U1WFG/XerI?=
 =?us-ascii?Q?bafgSa6IhUG+unWS53iWCDUbyjRT8Ztalh6Y/hYpMbTl22F+RiaT9aecj35T?=
 =?us-ascii?Q?BDhlzTEFbr/tuOADL55K1qbQC1zcyCFFVbjQRoFXhNIahnXFJVJ+cjWBRzrH?=
 =?us-ascii?Q?P81QBATf1Dzeem+0FMaCm/2tgXgQCL5LMY7bojFa2wnQd7GHaBYFlT/yn7wm?=
 =?us-ascii?Q?r70Cb/ZbIXLiVvAxVRF0QHxsKh1UrQobR0RyFsykouArtPFXP0ZfQujS5Li7?=
 =?us-ascii?Q?CVgUKJal87hmbSVxHGhUFFc65vewmgafibHWNitQgZKLe9rms2MUP0NwB7Aq?=
 =?us-ascii?Q?F96HeNhnpW/L9UyuLodTTIdCb89N2YMeI2QGIegupVdIuxW0CNL8J0T5ONJp?=
 =?us-ascii?Q?5PdkaNib/qUT2XqdzZOHysCFu4n2iYHDF2nmGgrszIDlDFT9fglsbxu/yb6d?=
 =?us-ascii?Q?ZQsJveGA/95mQlHZBahohToDk3pKLOlbGuPYBEtLxSREvJtWpwctz52Wq2G0?=
 =?us-ascii?Q?yAAJOljCS5xuwoTJMR7EfnBuYeLL9wgLNIC9I5mxmZvNbkCUjDI0HM40T77b?=
 =?us-ascii?Q?06eJjiobD9P/h3o8i/aJmw0HNsfFRcmiQoUJPOksHHkT+G3iVR2AC0mewfAs?=
 =?us-ascii?Q?1H2QXKcs4V8NEvIUQczmJ6fSEjNhQaF5gJ+XZ5S4Rwh8Y/FCL0ySHXhWWI39?=
 =?us-ascii?Q?/Lb5ze7q9ppN9bMRxlgIZ4zJyFkyvFq641ljmfQ/vkeY6QdErqt5HaunC77U?=
 =?us-ascii?Q?acD4KcsLcBWMGWO2rGmGplOoBX5RJ+juXB+n/ZNXUlDbCgMmYadbjvmhBNZm?=
 =?us-ascii?Q?af/DnyGgClAIhc6eqyc9RGPLK6VafrCU+jRbohoa2VkGrBDrtjt23ItkGzbs?=
 =?us-ascii?Q?9EHA7qs=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(376005)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:37.7342
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 243daead-779b-4673-ba46-08dc7b97d253
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8234

From: Henry Wang <xin.wang2@amd.com>

In order to support the dynamic dtbo device assignment to a running
VM, the add/remove of the DT overlay and the attach/detach of the
device from the DT overlay should happen separately. Therefore,
repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
overlay to Xen device tree, instead of assigning the device to the
hardware domain at the same time. It is OK to change the sysctl behavior
as this feature is experimental so changing sysctl behavior and breaking
compatibility is OK.

Add the XEN_DOMCTL_dt_overlay with operations
XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment to the domain.

The hypervisor firstly checks the DT overlay passed from the toolstack
is valid. Then the device nodes are retrieved from the overlay tracker
based on the DT overlay. The attach of the device is implemented by
mapping the IRQ and IOMMU resources. All devices in the overlay are
assigned to a single domain.

Also take the opportunity to make one coding style fix in sysctl.h.

xen,reg is to be used to handle non-1:1 mappings but it is currently
unsupported.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 xen/arch/arm/domctl.c        |   3 +
 xen/common/dt-overlay.c      | 207 ++++++++++++++++++++++++++---------
 xen/include/public/domctl.h  |  16 ++-
 xen/include/public/sysctl.h  |  11 +-
 xen/include/xen/dt-overlay.h |   8 ++
 5 files changed, 186 insertions(+), 59 deletions(-)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad56efb0f5..12a12ee781 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -5,6 +5,7 @@
  * Copyright (c) 2012, Citrix Systems
  */
 
+#include <xen/dt-overlay.h>
 #include <xen/errno.h>
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
@@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+    case XEN_DOMCTL_dt_overlay:
+        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 9cece79067..c2b03865a7 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -356,6 +356,42 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
     return 0;
 }
 
+/* This function should be called with the overlay_lock taken */
+static struct overlay_track *
+find_track_entry_from_tracker(const void *overlay_fdt,
+                              uint32_t overlay_fdt_size)
+{
+    struct overlay_track *entry, *temp;
+    bool found_entry = false;
+
+    ASSERT(spin_is_locked(&overlay_lock));
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Operation is supported only for prior added dtbo.\n");
+        return NULL;
+    }
+
+    return entry;
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
 static int remove_node_resources(struct dt_device_node *device_node)
 {
@@ -485,8 +521,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
                                         uint32_t overlay_fdt_size)
 {
     int rc;
-    struct overlay_track *entry, *temp, *track;
-    bool found_entry = false;
+    struct overlay_track *entry;
 
     rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
     if ( rc )
@@ -494,29 +529,10 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     spin_lock(&overlay_lock);
 
-    /*
-     * First check if dtbo is correct i.e. it should one of the dtbo which was
-     * used when dynamically adding the node.
-     * Limitation: Cases with same node names but different property are not
-     * supported currently. We are relying on user to provide the same dtbo
-     * as it was used when adding the nodes.
-     */
-    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
-    {
-        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
-        {
-            track = entry;
-            found_entry = true;
-            break;
-        }
-    }
-
-    if ( !found_entry )
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
     {
         rc = -EINVAL;
-
-        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
-               " Removing nodes is supported only for prior added dtbo.\n");
         goto out;
 
     }
@@ -620,15 +636,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return -EFAULT;
         }
 
-        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
-                           tr->iomem_ranges,
-                           tr->irq_ranges);
         write_unlock(&dt_host_lock);
-        if ( rc )
-        {
-            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-            return rc;
-        }
 
         /* Keep overlay_node address in tracker. */
         tr->nodes_address[j] = (unsigned long)overlay_node;
@@ -638,9 +646,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
 }
 /*
  * Adds device tree nodes under target node.
- * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
- * is done to avoid the removal of device_tree generation, iomem regions mapping
- * to hardware domain done by handle_node().
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
  */
 static long handle_add_overlay_nodes(void *overlay_fdt,
                                      uint32_t overlay_fdt_size)
@@ -774,20 +780,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
         goto err;
     }
 
-    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
-    if (tr->irq_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IRQ rangeset failed");
-        goto err;
-    }
-
-    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
-    if (tr->iomem_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
-        goto err;
-    }
-
     rc = add_nodes(tr, nodes_full_path);
     if ( rc )
     {
@@ -843,14 +835,89 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     xfree(tr->nodes_address);
     xfree(tr->fdt);
 
-    rangeset_destroy(tr->irq_ranges);
-    rangeset_destroy(tr->iomem_ranges);
-
     xfree(tr);
 
     return rc;
 }
 
+static long handle_attach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j, len;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
+    if (entry->irq_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IRQ rangeset failed");
+        goto out;
+    }
+
+    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
+    if (entry->iomem_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        if ( dt_get_property(overlay_node, "xen,reg", &len) )
+        {
+            printk(XENLOG_ERR "xen,reg not supported yet in overlay\n");
+            rc = -EOPNOTSUPP;
+            goto out;
+        }
+        write_lock(&dt_host_lock);
+        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
+                           entry->iomem_ranges, entry->irq_ranges);
+        write_unlock(&dt_host_lock);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+            goto out;
+        }
+    }
+
+    spin_unlock(&overlay_lock);
+
+    return 0;
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+    return rc;
+}
+
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     long ret;
@@ -890,6 +957,44 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
     return ret;
 }
 
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
+{
+    long ret;
+    void *overlay_fdt;
+
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        return -EOPNOTSUPP;
+
+    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
+        return -EINVAL;
+
+    if ( op->pad[0] || op->pad[1] || op->pad[2] )
+        return -EINVAL;
+
+    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+    if ( overlay_fdt == NULL )
+        return -ENOMEM;
+
+    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+    if ( ret )
+    {
+        gprintk(XENLOG_ERR, "copy from guest failed\n");
+        xfree(overlay_fdt);
+
+        return -EFAULT;
+    }
+
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+    else
+        ret = -EOPNOTSUPP;
+
+    xfree(overlay_fdt);
+
+    return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..6cd39ea46d 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -21,7 +21,7 @@
 #include "hvm/save.h"
 #include "memory.h"
 
-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
 
 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -1190,6 +1190,16 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+#if defined(__arm__) || defined(__aarch64__)
+struct xen_domctl_dt_overlay {
+    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
+    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
+#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
+    uint8_t overlay_op;                     /* IN: Attach. */
+    uint8_t pad[3];                         /* IN: Must be zero. */
+};
+#endif
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1287,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_dt_overlay                    87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1350,9 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+#if defined(__arm__) || defined(__aarch64__)
+        struct xen_domctl_dt_overlay        dt_overlay;
+#endif
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index febaa4b16a..3a6e7d48f0 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
 DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
 /*
  * XEN_SYSCTL_dt_overlay
- * Performs addition/removal of device tree nodes under parent node using dtbo.
- * This does in three steps:
- *  - Adds/Removes the nodes from dt_host.
- *  - Adds/Removes IRQ permission for the nodes.
- *  - Adds/Removes MMIO accesses.
+ * Performs addition/removal of device tree nodes under parent node using dtbo
+ * from dt_host.
  */
 struct xen_sysctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
@@ -1265,7 +1262,7 @@ struct xen_sysctl {
         struct xen_sysctl_cpu_policy        cpu_policy;
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
         struct xen_sysctl_dt_overlay        dt_overlay;
 #endif
         uint8_t                             pad[128];
diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
index c0567741ee..b2cedf3f2a 100644
--- a/xen/include/xen/dt-overlay.h
+++ b/xen/include/xen/dt-overlay.h
@@ -39,15 +39,23 @@ struct overlay_track {
 };
 
 struct xen_sysctl_dt_overlay;
+struct xen_domctl_dt_overlay;
 
 #ifdef CONFIG_OVERLAY_DTB
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
 #else
 #include <xen/errno.h>
 static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     return -EOPNOTSUPP;
 }
+
+static inline long dt_overlay_domctl(struct domain *d,
+                                     struct xen_domctl_dt_overlay *op)
+{
+    return -EOPNOTSUPP;
+}
 #endif
 
 #endif /* __XEN_DT_OVERLAY_H__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728988.1134126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW8-0005D2-50; Fri, 24 May 2024 02:18:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728988.1134126; Fri, 24 May 2024 02:18:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW7-0005Cn-W3; Fri, 24 May 2024 02:18:48 +0000
Received: by outflank-mailman (input) for mailman id 728988;
 Fri, 24 May 2024 02:18:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW6-0003tc-NW
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:46 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2405::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f23483f6-1973-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:18:45 +0200 (CEST)
Received: from MW4PR04CA0064.namprd04.prod.outlook.com (2603:10b6:303:6b::9)
 by DS0PR12MB6416.namprd12.prod.outlook.com (2603:10b6:8:cb::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7611.22; Fri, 24 May 2024 02:18:39 +0000
Received: from CO1PEPF000075EF.namprd03.prod.outlook.com
 (2603:10b6:303:6b:cafe::46) by MW4PR04CA0064.outlook.office365.com
 (2603:10b6:303:6b::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 02:18:38 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000075EF.mail.protection.outlook.com (10.167.249.38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 02:18:38 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:34 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:33 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f23483f6-1973-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PLhI1eeIczr16i9d20o6T+a6XaXi7/1BvgoIwx9TlY7RjjQ63/qVEKaZ4c6RPO7NttTdlrygJM+MzoSxi5e5L+w0uwUXm379IvVVfrbck6PbxG9pqTHfUn2ayR7uhWT94BMtdTvFvs83AHHa6YJiPIj2Pxf2OyZNmUZYRNShTvXy26/IogGrMuoyuZFQ2fE8W13lgzM7A9bYiCCfjBY7IOxYUnWg7doXQ5mYKs78yYwsUcCc5KPmZfAcwRc7rQYGTjqnugNkk0cX76+gBlk7rzvabLsjB8vX0eOJgkkGPGZrk6U0tHI+GUIRnjRTVAvDcV7cNi6G1AAfdFD6bd09/A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NSERoOa3ZYR0kdpGTWAKdhvJ11sFwvda7ns+AIDXeNI=;
 b=KFj9jB2Qs2LWNHaInMWOjQQLSCZVJJjfbm4qjSOzI7n/isvjAeVe0RzfDZ7nzkoV3POy0A+KCE9CVFRlROUeH5aS4D4uuknHY45DFl/sncd14Nc7W5uJFQvjvMpCiXREltn1PBt0tjrI+ygY2hviqQZO7yVKB/61fHWKo6/kIxqnANPci1sN/NYp7xFy3auszkfj+IKZdxxrmYc2HTcs/AXIYiDWtXxJNyGDA8HjErYfeMy5wdhpqX09VVGvLm+UfOYJ0pRXH1Zj1O1HI7eCLrAoz/k15VLtQ+epN9rPXck0pSdnWLB58K9/JKqjMU1KABXDOMqPiLsuRmfkG/FeSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NSERoOa3ZYR0kdpGTWAKdhvJ11sFwvda7ns+AIDXeNI=;
 b=PxV+7V2MZooKFL0Z6fO1O0fZHCxBywvWQslCIL6Bvca9qw01BcFQrnkuio2tOd5InENv6ITPh3jvpMG6EeFk30gLs5TMzT6KwHsX5A13JxSjzCfGyqllOoJShU+jhbz58F6gv/3lcmMF4LwnYEpRX9kFYZedwBm5nWXFOMtPaNo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v5 6/7] tools: Introduce the "xl dt-overlay attach" command
Date: Thu, 23 May 2024 19:18:13 -0700
Message-ID: <20240524021814.2666257-6-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000075EF:EE_|DS0PR12MB6416:EE_
X-MS-Office365-Filtering-Correlation-Id: 1539460b-d240-48f7-9f12-08dc7b97d2fa
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?A4L37uh5lyCQVNYh7YmEKmcKs+6lWcOekLbFMJmHJGZLCPY8VQoy1DpeXhYS?=
 =?us-ascii?Q?VfcCFGwC9f2FJNhJBnPyqiR5t3r8kFjT+dj5ZWiWx94pHPcSt+WCcpGL3oQe?=
 =?us-ascii?Q?IM9FG9O8+kFfyQoW5QRljeieWDJyHEoRVWLFJ9xtSJ1jw/V8N0sq08II9ISt?=
 =?us-ascii?Q?Kxzppd2vVXTq+Qx/wabjFb1kotBKwT8osrG9DgaO8Z97L9IidVj5OZqTqoE9?=
 =?us-ascii?Q?PLOodMTVAYwil2D+u2M8M0K0HyZVoFHXgKBLfB0ykilwJjTY+UDL92UTZmwH?=
 =?us-ascii?Q?pp0B2zt738Tb4cY+8CM1xGpw0blWyUc5uxku8ZOHJe1OWK3bLPZqurivCga0?=
 =?us-ascii?Q?+iKB9/b2zyygw3rAu78K8CDTTay9hvbfj/MHhtqqS3dJCGDW05eSFrmXuiMf?=
 =?us-ascii?Q?DQMgPsBqo5Hfzati6I2yFFSAYPH4HFswIfZWutWda6T7PNBLDQ3zWg1e6wU3?=
 =?us-ascii?Q?CMedWUjpbdTRnUoVckynMjnBPOWI+vhFtVjvTIt09F8ZLFlCOCaTzCSNamfB?=
 =?us-ascii?Q?kAqu/H4ACooVZ6kIAz3Ng2w1DalwJNmRKuuRzN4AP/ueLVs2Exv+qFN8X5Oz?=
 =?us-ascii?Q?4clyBphgKC/Nkn/90iwjhSt687HXlQ9deliZxAQa2q105eYgW1SPoNg8uvER?=
 =?us-ascii?Q?5CI2Wbw94lK3wI41n7cqgj2vaVfuTpxV5jUl+a9MZpIvc5QkXHs5UHqii8OV?=
 =?us-ascii?Q?L3Em1rbBUbaIzdxgc8nWyin7wC3R61033js9VUZG/Gl+gK1UsV2P4IpTihsi?=
 =?us-ascii?Q?dS26AoOFo5SvovKzqZTnEthTWmbEWI1N4EG1RIg8rENHt9qSTfqh3wy8uNbG?=
 =?us-ascii?Q?xDjngG955lpz5gt2gA1h5ZGLD/xxeAvt9Eo6YEFbETnE87n+2aJdQ86jAEUM?=
 =?us-ascii?Q?95FZ/ZdXKEdp/VzAqMmrYXi2hnq5expsP79qxmOF36IWIQpele5QcWBJyOOp?=
 =?us-ascii?Q?dquOFdpv7zmGM7ecjw4RSD+nsEXyjcz3WXFFgJh4ra/q1MXlXUfe/OiKBg5F?=
 =?us-ascii?Q?6zRdShIVcRdjjui2gNqodxUAc+4rNjry8PTuhCvAa7rKTh8vA5HERZl2AWwg?=
 =?us-ascii?Q?gDpHEXRkHdGk4BswE2kO54OSbUB+UzAtL++Pqe8Uldzb2NPcPRwe1TFbgTpw?=
 =?us-ascii?Q?YaN2+Sy4y0M4nvl5+5lsBGGbxKfQj1xR8Msj5WTQhsWrXVH/LjLy/fWs3gJF?=
 =?us-ascii?Q?Gf/rx2KXfP3mYaMD+wSEOy6Jsp6WVJH6ahp3c29OPdua8RxporAPst0lm2Yl?=
 =?us-ascii?Q?XYP1wxM+UfgZWrnUIoaKi9BeQu3bAQtYT7s6qRooiuo6de7Hez1e7GW91fw4?=
 =?us-ascii?Q?YyFRIR8jVWdPAyYxuib2Q3ZmSq9dEsiDEVG+aOlxkqTE+Ot0yx1PLWv48qRO?=
 =?us-ascii?Q?1RdMF/CaobzpIIUbbPiw3jLFyBTK?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:38.8123
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1539460b-d240-48f7-9f12-08dc7b97d2fa
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000075EF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6416

From: Henry Wang <xin.wang2@amd.com>

With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach (in the future also detach) devices from the provided DT overlay
to domains. Support this by introducing a new "xl dt-overlay" command
and related documentation, i.e. "xl dt-overlay attach. Slightly rework
the command option parsing logic.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 tools/include/libxl.h               | 15 +++++++++++
 tools/include/xenctrl.h             |  3 +++
 tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++++
 tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++++
 tools/xl/xl_cmdtable.c              |  4 +--
 tools/xl/xl_vmcontrol.c             | 39 ++++++++++++++++++++---------
 6 files changed, 106 insertions(+), 14 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 3b5c18b48b..f2e19ec592 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -643,6 +643,12 @@
  */
 #define LIBXL_HAVE_NR_SPIS 1
 
+/*
+ * LIBXL_HAVE_OVERLAY_DOMAIN indicates the presence of
+ * libxl_dt_overlay_domain.
+ */
+#define LIBXL_HAVE_OVERLAY_DOMAIN 1
+
 /*
  * libxl memory management
  *
@@ -2556,8 +2562,17 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 #if defined(__arm__) || defined(__aarch64__)
+/* Values should keep consistent with the op from XEN_SYSCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_ADD                   1
+#define LIBXL_DT_OVERLAY_REMOVE                2
 int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
                      uint32_t overlay_size, uint8_t overlay_op);
+
+/* Values should keep consistent with the op from XEN_DOMCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_DOMAIN_ATTACH         1
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op);
 #endif
 
 /*
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..9ceca0cffc 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id);
 #endif
 
 /* Compat shims */
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
index c2224c4d15..ea1da522d1 100644
--- a/tools/libs/ctrl/xc_dt_overlay.c
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -48,3 +48,34 @@ err:
 
     return err;
 }
+
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id)
+{
+    int err;
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_dt_overlay,
+        .domain = domain_id,
+        .u.dt_overlay = {
+            .overlay_op = overlay_op,
+            .overlay_fdt_size = overlay_fdt_size,
+        }
+    };
+
+    DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+        goto err;
+
+    set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, overlay_fdt);
+
+    return err;
+}
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
index a6c709a6dc..00503b76bd 100644
--- a/tools/libs/light/libxl_dt_overlay.c
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -69,3 +69,31 @@ out:
     return rc;
 }
 
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op)
+{
+    int rc;
+    int r;
+    GC_INIT(ctx);
+
+    if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+        LOG(ERROR, "Overlay DTB check failed");
+        rc = ERROR_FAIL;
+        goto out;
+    } else {
+        LOG(DEBUG, "Overlay DTB check passed");
+        rc = 0;
+    }
+
+    r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, overlay_op,
+                             domain_id);
+    if (r) {
+        LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__);
+        rc = ERROR_FAIL;
+    }
+
+out:
+    GC_FREE;
+    return rc;
+}
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 1f3c6b5897..42751228c1 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = {
 #ifdef LIBXL_HAVE_DT_OVERLAY
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
-      "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>",
+      "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain",
+      "<operation=add|remove> <.dtbo> OR <operation=attach> <.dtbo> <Domain>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 02575d5d36..fa1a4420e3 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1268,32 +1268,43 @@ int main_create(int argc, char **argv)
 #ifdef LIBXL_HAVE_DT_OVERLAY
 int main_dt_overlay(int argc, char **argv)
 {
-    const char *overlay_ops = NULL;
     const char *overlay_config_file = NULL;
     void *overlay_dtb = NULL;
     int rc;
     uint8_t op;
     int overlay_dtb_size = 0;
-    const int overlay_add_op = 1;
-    const int overlay_remove_op = 2;
+    uint32_t domain_id = 0;
+    bool domain_op = false;
 
     if (argc < 2) {
         help("dt-overlay");
         return EXIT_FAILURE;
     }
 
-    overlay_ops = argv[1];
-    overlay_config_file = argv[2];
-
-    if (strcmp(overlay_ops, "add") == 0)
-        op = overlay_add_op;
-    else if (strcmp(overlay_ops, "remove") == 0)
-        op = overlay_remove_op;
-    else {
+    if (strcmp(argv[optind], "add") == 0)
+        op = LIBXL_DT_OVERLAY_ADD;
+    else if (strcmp(argv[optind], "remove") == 0)
+        op = LIBXL_DT_OVERLAY_REMOVE;
+    else if (strcmp(argv[optind], "attach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_ATTACH;
+        domain_op = true;
+    } else {
         fprintf(stderr, "Invalid dt overlay operation\n");
         return EXIT_FAILURE;
     }
 
+    overlay_config_file = argv[optind+1];
+
+    if (domain_op) {
+        if (argc <= optind + 2) {
+            fprintf(stderr, "Missing domain ID\n");
+            help("dt-overlay");
+            return EXIT_FAILURE;
+        } else {
+            domain_id = find_domain(argv[optind+2]);
+        }
+    }
+
     if (overlay_config_file) {
         rc = libxl_read_file_contents(ctx, overlay_config_file,
                                       &overlay_dtb, &overlay_dtb_size);
@@ -1309,7 +1320,11 @@ int main_dt_overlay(int argc, char **argv)
         return EXIT_FAILURE;
     }
 
-    rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    if (!domain_op)
+        rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    else
+        rc = libxl_dt_overlay_domain(ctx, domain_id, overlay_dtb,
+                                     overlay_dtb_size, op);
 
     free(overlay_dtb);
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:18:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:18:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.728989.1134129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW8-0005GH-F9; Fri, 24 May 2024 02:18:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 728989.1134129; Fri, 24 May 2024 02:18:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKW8-0005Ew-9O; Fri, 24 May 2024 02:18:48 +0000
Received: by outflank-mailman (input) for mailman id 728989;
 Fri, 24 May 2024 02:18:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAKW7-0004I4-DO
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:18:47 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f19a7382-1973-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:18:45 +0200 (CEST)
Received: from BN1PR13CA0028.namprd13.prod.outlook.com (2603:10b6:408:e2::33)
 by SA3PR12MB7808.namprd12.prod.outlook.com (2603:10b6:806:31b::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 02:18:40 +0000
Received: from BN3PEPF0000B374.namprd21.prod.outlook.com
 (2603:10b6:408:e2:cafe::71) by BN1PR13CA0028.outlook.office365.com
 (2603:10b6:408:e2::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.5 via Frontend
 Transport; Fri, 24 May 2024 02:18:40 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN3PEPF0000B374.mail.protection.outlook.com (10.167.243.171) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7633.0 via Frontend Transport; Fri, 24 May 2024 02:18:38 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 23 May
 2024 21:18:35 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 23 May 2024 21:18:34 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f19a7382-1973-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Fs3ERBiqLH43cXK4d92rJTJk5XKoqn/N9iAZyMIGSBTo5kAMwcidPAxPDhubgzub6ENMkj+OhIYrwIi0ReCD258Dp2yEagpUf8fn1iEXO84D01rJimDzpNZ5/3b0tx7K05rd+TiaO2izJq1ovDCDZ5vDb1pPM5TCnWhkTWftQp7gT/O3IPrdm9f86mkd31ZXF2aCJZVCLFKfvIPLXtWQbAA861nOnCagd9EG1gs2hhC1SvjM9zW7XCr+VROOTjcVrMsNyzBxxbqasfIuhy+FoyvOT3HPYTLpH/VQ5nxICfUyruFvcpWdFNyxJyPNglhwtOC/8P7t6M58ZqPzRn7opQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=spfuWcKUhy4IpKgxdQbUBhNNhHU158grwSfyfQ+eUYo=;
 b=HYKmb8aI6KtNvlloVTSrsMxCRN8LKkj0H3Gjb3IhMghKn/0ldjNg9siSvVAUbyDcBZQgTjXiVRwBym+u0X7EzkXUlJ8MrYsgwhBlIgJPlHs7YnnN9ZasgyoTX2DQxcuNteUzKSj5avlVla+C1YgVEtajPqftWoTkqqfhVJA00lh20l9p5tMgU5B9VtrvTeBydgcBoEA+S91O4DLhk2FReLf3ZYYU7iJeXvf8NoTB7GPQIJ4NEcc83zOVHgOHfQ+T5f/K3P8nhLMpvOzJ2vSF9FZ7xOC+9J5NeiaO8ZmEzZbfdB9hZdl7r6yoTKDNMrVT8qHKolD973086XctB1v0NA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=spfuWcKUhy4IpKgxdQbUBhNNhHU158grwSfyfQ+eUYo=;
 b=qgUa3GAcc0X79CF2OsHLFWP/KK341C23/WpjuZEUvrTlsGHnln3bYg5KILYth5xNqjQ1bSuZgdLU9nhxR43hXbJBFiIj3eK5nQuwBQmuL3Inxf2+JI2jKGbDvpg4UZf+ZfDxS8c2lA+IsuslAMk/N+rQ5AKzqdD3Y+pF8PU86JY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Vikram Garhwal <fnu.vikram@xilinx.com>,
	"Stefano Stabellini" <stefano.stabellini@xilinx.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v5 7/7] docs: Add device tree overlay documentation
Date: Thu, 23 May 2024 19:18:14 -0700
Message-ID: <20240524021814.2666257-7-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B374:EE_|SA3PR12MB7808:EE_
X-MS-Office365-Filtering-Correlation-Id: e4961a22-ac01-47d0-80d5-08dc7b97d2f3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|82310400017|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?WLmn1WgWEulHuBgsdlmgC06rLAkeRjI7lKTDnoUxl9HAFHN6vGddY044lrNl?=
 =?us-ascii?Q?uLTx+mWFd+cxpC2lMCdMqcPqX8XmAOx+QPgxxokOHrQPtAGO5UWVj/Ftmhin?=
 =?us-ascii?Q?XXFoiUZXPkojMntMhhGWWuQRpAMInUIksgXx2o35I9b/2CJEiE1BindYw3m2?=
 =?us-ascii?Q?QN4YUSfjO90iqzbjNqBLMcmu1F77Mdi7ftUZ47nMR7tVs31EoETftWHXfAKg?=
 =?us-ascii?Q?XfIsNGjsrgNzmilFbAJ5cr7UX7OHWBtyDmjtjZtYZRq8bkoSyN6nXjdekVAK?=
 =?us-ascii?Q?9Xp9Z5u6XXhj5rWDF6Oe5W1u3ZqmjUQo5Td7An5Zta/k++Xh7Yp8t5po8ZG+?=
 =?us-ascii?Q?TzJLJdtcXTvrmjZwvBDI5JbanPxdEkDoPmTbusKAejzWSJwVue4aK9vD7n20?=
 =?us-ascii?Q?8/F6Thu3Ingk7lr3tCo0eJXIOY1PJrIctaCt9wSvt97uKicW4c+8pobrbitE?=
 =?us-ascii?Q?GIDH5LI29ZiedKIrwkfFp4yy4Mh6r1DNQfL09Ejsq6MliuEy33iCWrevANoh?=
 =?us-ascii?Q?APLojt9fz4NNo2uMFvU6Gyxk5Xz2q3tdwDawlqlcD6+h5dFU2q1UaC3SPzWx?=
 =?us-ascii?Q?9xmeHiGV9q8SwTzrdCBO4bmtZ6KSbkweDXNQdd+ujTp2PSI7703HDiQqtWMI?=
 =?us-ascii?Q?UQnPybyZVtNbmU5cNJrSzJLPAPwA0XsfB/LimTDpIUT3q9Z6P86+ctVPez/6?=
 =?us-ascii?Q?Z79GrF9rPMgqRrzN+GErd2/pGDNb8KPEh166gRZ3g4GiHMcJfsHYXP/PKsph?=
 =?us-ascii?Q?HRcEtAtJ5iQnsP3fyHBd2UGvzyfi+DuHSy1ofHc1dJnjRH4NpMReCtv0qVVY?=
 =?us-ascii?Q?XP6GOq6l5apBfPM19R8Rr5yUdz7+lKvf76t75hKWVk/Drt6iOaFtbyowigQc?=
 =?us-ascii?Q?2vgrgq6LGSYTaCQQxsAdIz2bL9kHwq0eEnq4x/Ev6OAUOhfA3+6KtlAXQxVl?=
 =?us-ascii?Q?LoQcSmDHRLfVvCvr8ST2Po9fyGsVue+QmGanaYulI6e2tEzEGKQKJLP0z9md?=
 =?us-ascii?Q?GL7iVTtHfzzLjbmx33rp73aotZ/VCL6rPi1D94RrF/lkV0971IlqWDluIrG5?=
 =?us-ascii?Q?DegjslMF/Iy/kI/odgdngVL8sQEqdK1MPepPA90+z8qMvGjBV1A1k08iy/Xr?=
 =?us-ascii?Q?Ah+KhGzkyYRCsfGp8oZgM+HG8Y0RMVwDpAccnr/wcAcnJ11xnMSdeojJG4hD?=
 =?us-ascii?Q?FfXEEIGHEGEKK8JLb5o0VzBS8pkTJW26v3v3h8PswvJsuC8S6QTY11mVAWrL?=
 =?us-ascii?Q?1EGmoLEp7AS1CEeVl6e/wU9Zwh6OahGOXC88L5aRv9NwMi6wxsGl03jT6JxK?=
 =?us-ascii?Q?LEZY/ZxGkBn4ESo4sUIhUWUH?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(82310400017)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 02:18:38.8750
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e4961a22-ac01-47d0-80d5-08dc7b97d2f3
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B374.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7808

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
 docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)
 create mode 100644 docs/misc/arm/overlay.txt

diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
new file mode 100644
index 0000000000..0a2dee951a
--- /dev/null
+++ b/docs/misc/arm/overlay.txt
@@ -0,0 +1,82 @@
+# Device Tree Overlays support in Xen
+
+Xen experimentally supports dynamic device assignment to running
+domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device
+tree, and attaching them to a running domain with given $domid.
+
+Dynamic node assignment works in two steps:
+
+## Add/Remove device tree overlay to/from Xen device tree
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
+
+## Attach device from the DT overlay to domain
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor attach the device to the user-provided $domid by
+   mapping node resources in the DT overlay.
+
+# Examples
+
+Here are a few examples on how to use it.
+
+## Dom0 device add
+
+For assigning a device tree overlay to Dom0, user should firstly properly
+prepare the DT overlay. More information about device tree overlays can be
+found in [1]. Then, in Dom0, enter the following:
+
+    (dom0) xl dt-overlay add overlay.dtbo
+
+This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
+
+To assign the newly added device from the dtbo to Dom0:
+
+    (dom0) xl dt-overlay attach overlay.dtbo 0
+
+Next, if the user wants to add the same device tree overlay to dom0
+Linux, execute the following:
+
+    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally if needed, the relevant Linux kernel drive can be loaded using:
+
+    (dom0) modprobe module_name.ko
+
+## DomU device add/remove
+
+All the nodes in dtbo will be assigned to a domain; the user will need
+to prepare the dtb for the domU. For example, the `interrupt-parent`
+property of the DomU overlay should be changed to the Xen hardcoded
+value `0xfde8`, and the xen,reg property should be added to specify the
+address mappings. If xen,reg is not present, it is assumed 1:1 mapping.
+Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
+
+For new domains to be created, the user will need to create the DomU
+with below properties properly configured in the xl config file:
+- `iomem`
+- `passthrough` (if IOMMU is needed)
+
+User will also need to modprobe the relevant drivers. For already
+running domains, the user can use the xl dt-overlay attach command,
+example:
+
+    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
+    (dom0) xl dt-overlay attach overlay.dtbo $domid
+    (dom0) xl console $domid                         # To access $domid console
+
+Next, if the user needs to modify/prepare the overlay.dtbo suitable for
+the domU:
+
+    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally, if needed, the relevant Linux kernel drive can be probed:
+
+    (domU) modprobe module_name.ko
+
+[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 02:29:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:29:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729025.1134145 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKgT-0000xb-C7; Fri, 24 May 2024 02:29:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729025.1134145; Fri, 24 May 2024 02:29:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKgT-0000xU-97; Fri, 24 May 2024 02:29:29 +0000
Received: by outflank-mailman (input) for mailman id 729025;
 Fri, 24 May 2024 02:29:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAKWv-0004I4-Nw
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:19:37 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 102fbd00-1974-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:19:35 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id BABEE62F9E;
 Fri, 24 May 2024 02:19:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A481BC2BD10;
 Fri, 24 May 2024 02:19:33 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 102fbd00-1974-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716517174;
	bh=UmGN7YKj+VvapXY7aISIsBXEob+KUsS793EdeP09GtI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=sSpRa2gdRPXttpB+r/Suk18d4Ffvc40hZUhJBIDjtVhmQK4QBLChm0EyV6w844JBn
	 LtYH/JLrSHiA+2tHorqt5qmi0WjjDv0jQIBSvey1RXxlVtlKvWfI5I19K0hWI+yWx7
	 30IUZ4L8Eje47IxMBICJQuE/GiFGn2FxS8zVaogEKDGY4Zyu4fbTBQVxhuPxSynx5E
	 mjpcmNz2TDXbhsf0sxeFwh5bu8Q3rCcJlp+HTvjaDvVqbG9MQC7IE9BN4mm8bsGGCm
	 jnhe3RgvsGROeDcHRqS5y/tw5Bv9XiMYjl9vLHCXesEr7yxfIpMzqQ2fJT5zL1K/vf
	 Z9nJY8uKbHQ5A==
Date: Thu, 23 May 2024 19:19:32 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, 
    Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v4 8/9] tools: Introduce the "xl dt-overlay {attach,detach}"
 commands
In-Reply-To: <fc2ff110-0af7-439b-a220-2cb21c63c54e@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405231822540.2557291@ubuntu-linux-20-04-desktop>
References: <20240523074040.1611264-1-xin.wang2@amd.com> <20240523074040.1611264-9-xin.wang2@amd.com> <fc2ff110-0af7-439b-a220-2cb21c63c54e@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> On 23/05/2024 08:40, Henry Wang wrote:
> > With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> > attach/detach devices from the provided DT overlay to domains.
> > Support this by introducing a new set of "xl dt-overlay" commands and
> > related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly
> > rework the command option parsing logic.
> > 
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> > ---
> > v4:
> > - Add Jason's Reviewed-by tag.
> > v3:
> > - Introduce new API libxl_dt_overlay_domain() and co., instead of
> >    reusing existing API libxl_dt_overlay().
> > - Add in-code comments for the LIBXL_DT_OVERLAY_* macros.
> > - Use find_domain() to avoid getting domain_id from strtol().
> > v2:
> > - New patch.
> > ---
> >   tools/include/libxl.h               | 10 +++++++
> >   tools/include/xenctrl.h             |  3 +++
> >   tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++
> >   tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++
> >   tools/xl/xl_cmdtable.c              |  4 +--
> >   tools/xl/xl_vmcontrol.c             | 42 ++++++++++++++++++++---------
> >   6 files changed, 104 insertions(+), 14 deletions(-)
> > 
> > diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> > index 62cb07dea6..6cc6d6bf6a 100644
> > --- a/tools/include/libxl.h
> > +++ b/tools/include/libxl.h
> 
> I think you also need to introduce LIBXL_HAVE_...

Added

I have removed the LIBXL_DT_OVERLAY_DOMAIN_DETACH and the relate
mentions. I kept Jasons' ack.


From xen-devel-bounces@lists.xenproject.org Fri May 24 02:29:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:29:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729026.1134156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKgd-0001MG-PT; Fri, 24 May 2024 02:29:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729026.1134156; Fri, 24 May 2024 02:29:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKgd-0001M7-MS; Fri, 24 May 2024 02:29:39 +0000
Received: by outflank-mailman (input) for mailman id 729026;
 Fri, 24 May 2024 02:29:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAKX0-0004I4-Vq
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:19:42 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 134734f7-1974-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 04:19:41 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B6B9562FBA;
 Fri, 24 May 2024 02:19:39 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 760C6C2BD10;
 Fri, 24 May 2024 02:19:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 134734f7-1974-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716517179;
	bh=fJq2+WWu20TIMSRKeUvrIZ8uc1GT9bi5wl+3P9HWW68=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=qbJgtNjSQTOo3YAFdn+5GuA/tamyUEE2tI3r0TpiRUpyj9pyNUMIZ8uBhnNmvNObF
	 keFmwHLlLPliizXSj7DLpG11WeKHHqW01LV4NXDuV8lpL7Gh5NxmzNmd26/BSxm6VH
	 Gkcm+YX7dJdkhcV1dRVCche4AIJcOvdXs+GBpuws0J3xthN80aC401a9bLCUq+s1Zu
	 AvTA5ZzFtuw+dkeYSWpnzFVIuD27Ygq47fqDOuL/TAR1ML7GsOoqJgoUB0fIQ9NvGM
	 X51+LblZOypdDg1fDIX8HfJD0/OUPwcTBMyhmMaNZYWqa2d4PCUSbncPGjjUSReCAe
	 nkEBZAixbxaIg==
Date: Thu, 23 May 2024 19:19:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v4 9/9] docs: Add device tree overlay documentation
In-Reply-To: <a437fe4c-bb21-456f-acf3-53ae8916f4f6@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405231835090.2557291@ubuntu-linux-20-04-desktop>
References: <20240523074040.1611264-1-xin.wang2@amd.com> <20240523074040.1611264-10-xin.wang2@amd.com> <a437fe4c-bb21-456f-acf3-53ae8916f4f6@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> On 23/05/2024 08:40, Henry Wang wrote:
> > From: Vikram Garhwal <fnu.vikram@xilinx.com>
> > 
> > Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > ---
> > v4:
> > - No change.
> > v3:
> > - No change.
> > v2:
> > - Update the content based on the changes in this version.
> > ---
> >   docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 99 insertions(+)
> >   create mode 100644 docs/misc/arm/overlay.txt
> > 
> > diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
> > new file mode 100644
> > index 0000000000..811a6de369
> > --- /dev/null
> > +++ b/docs/misc/arm/overlay.txt
> > @@ -0,0 +1,99 @@
> > +# Device Tree Overlays support in Xen
> > +
> > +Xen now supports dynamic device assignment to running domains,
> 
> This reads as we "support" the feature. I would prefer if we write "Xen
> expirementally supports..." or similar.

Done


> > +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and
> > +attaching/detaching them to/from a running domain with given $domid.
> > +
> > +Dynamic node assignment works in two steps:
> > +
> > +## Add/Remove device tree overlay to/from Xen device tree
> > +
> > +1. Xen tools check the dtbo given and parse all other user provided
> > arguments
> > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> > +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
> > +
> > +## Attach/Detach device from the DT overlay to/from domain
> > +
> > +1. Xen tools check the dtbo given and parse all other user provided
> > arguments
> > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> > +3. Xen hypervisor attach/detach the device to/from the user-provided $domid
> > by
> > +   mapping/unmapping node resources in the DT overlay.
> > +
> > +# Examples
> > +
> > +Here are a few examples on how to use it.
> > +
> > +## Dom0 device add
> > +
> > +For assigning a device tree overlay to Dom0, user should firstly properly
> > +prepare the DT overlay. More information about device tree overlays can be
> > +found in [1]. Then, in Dom0, enter the following:
> > +
> > +    (dom0) xl dt-overlay add overlay.dtbo
> > +
> > +This will allocate the devices mentioned in overlay.dtbo to Xen device
> > tree.
> > +
> > +To assign the newly added device from the dtbo to Dom0:
> > +
> > +    (dom0) xl dt-overlay attach overlay.dtbo 0
> > +
> > +Next, if the user wants to add the same device tree overlay to dom0
> > +Linux, execute the following:
> > +
> > +    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> > +    (dom0) cat overlay.dtbo >
> > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> > +
> > +Finally if needed, the relevant Linux kernel drive can be loaded using:
> > +
> > +    (dom0) modprobe module_name.ko
> > +
> > +## Dom0 device remove
> > +
> > +For removing the device from Dom0, first detach the device from Dom0:
> > +
> > +    (dom0) xl dt-overlay detach overlay.dtbo 0
> > +
> > +NOTE: The user is expected to unload any Linux kernel modules which
> > +might be accessing the devices in overlay.dtbo before detach the device.
> > +Detaching devices without unloading the modules might result in a crash.
> > +
> > +Then remove the overlay from Xen device tree:
> > +
> > +    (dom0) xl dt-overlay remove overlay.dtbo
> > +
> > +## DomU device add/remove
> > +
> > +All the nodes in dtbo will be assigned to a domain; the user will need
> > +to prepare the dtb for the domU. For example, the `interrupt-parent`
> > property
> > +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`.
> > +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
> > +
> > +User will need to create the DomU with below properties properly configured
> > +in the xl config file:
> > +- `iomem`
> 
> I don't quite understand how the user can specify the MMIO region if the
> device is attached after the domain is created.

I think this was meant for a domain about to be created (not already
running). I clarified.


> 
> > +- `passthrough` (if IOMMU is needed)
> > +
> > +User will also need to modprobe the relevant drivers.
> > +
> > +Example for domU device add:
> > +
> > +    (dom0) xl dt-overlay add overlay.dtbo            # If not executed
> > before
> > +    (dom0) xl dt-overlay attach overlay.dtbo $domid
> 
> Can how clarify how the MMIO will be mapped? Is it direct mapped? If so,
> couldn't this result to clash with other part of the address space (e.g.
> RAM?).

Yes, it is reusing the same code as dom0, which makes the code nice but
it doesn't support non-1:1 mappings. I think those should be done via
the xen,reg property. My suggestion would be this:

- if xen,reg is present, use it
- if xen,reg is not present, fall back to 1:1 mapping based on reg

For the next version of the series, I'd just document the current
limitation of the implementation. I added this to patch #4:

diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 0f8b25ccb4..c2b03865a7 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -845,7 +845,7 @@ static long handle_attach_overlay_nodes(struct domain *d,
                                         uint32_t overlay_fdt_size)
 {
     int rc;
-    unsigned int j;
+    unsigned int j, len;
     struct overlay_track *entry;
 
     rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
@@ -888,6 +888,12 @@ static long handle_attach_overlay_nodes(struct domain *d,
             goto out;
         }
 
+        if ( dt_get_property(overlay_node, "xen,reg", &len) )
+        {
+            printk(XENLOG_ERR "xen,reg not supported yet in overlay\n");
+            rc = -EOPNOTSUPP;
+            goto out;
+        }
         write_lock(&dt_host_lock);
         rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
                            entry->iomem_ranges, entry->irq_ranges);



> > +    (dom0) xl console $domid                         # To access $domid
> > console
> > +
> > +Next, if the user needs to modify/prepare the overlay.dtbo suitable for
> > +the domU:
> > +
> > +    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> > +    (domU) cat overlay_domu.dtbo >
> > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> > +
> > +Finally, if needed, the relevant Linux kernel drive can be probed:
> > +
> > +    (domU) modprobe module_name.ko
> > +
> > +Example for domU overlay remove:
> > +
> > +    (dom0) xl dt-overlay detach overlay.dtbo $domid
> > +    (dom0) xl dt-overlay remove overlay.dtbo
> 
> I assume we have safety check in place to ensure we can't remove the device if
> it is already attached. Is that correct?

I'll remove this part of the doc.


From xen-devel-bounces@lists.xenproject.org Fri May 24 02:30:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:30:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729033.1134166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKh4-000369-2G; Fri, 24 May 2024 02:30:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729033.1134166; Fri, 24 May 2024 02:30:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKh3-000362-UK; Fri, 24 May 2024 02:30:05 +0000
Received: by outflank-mailman (input) for mailman id 729033;
 Fri, 24 May 2024 02:30:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAKWq-0003tc-9f
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:19:32 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0d4c5ebd-1974-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:19:31 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D4C8D62FA9;
 Fri, 24 May 2024 02:19:29 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14004C2BD10;
 Fri, 24 May 2024 02:19:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d4c5ebd-1974-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716517169;
	bh=OeUveWZsipaAlOiI4w9FYgFuXr/DUI3Ezg3sFoJuz/s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UoZOQuSwP4sUVPbgW/GAkT7i6yrCDdgbSOR5WTlKmWmiFjIIDYk4hGznKAgfQ9Vrt
	 bjnCwzWqvda65XoZWUc2vWPXhUt/CmsxDLBvpoiy5EhdYO63x75BpeaupyXp/KueuP
	 ygS6KQgzYChfBnQK6Hl1saEu4IOyJnuB2bjEkQ2tMm7sChf3UcXF9W2gW+Js31WKhX
	 jurkDWd1Ureg+8IKOXL2CuJUPsJQ+EIdTWEBZYUtBk9jbvfQSsj8esSa8hSSTQ/Mvx
	 emr3noGgP845RCKFkcIfGujqRkCewyZeeKD+yBPr/KokTDz9lVNtllprTxehuM0P7K
	 WuCpklM3TOPzg==
Date: Thu, 23 May 2024 19:19:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v4 5/9] xen/arm: Add XEN_DOMCTL_dt_overlay and device
 attachment to domains
In-Reply-To: <311c645f-d0cf-45ef-abb3-c7d8a58f1130@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405231805500.2557291@ubuntu-linux-20-04-desktop>
References: <20240523074040.1611264-1-xin.wang2@amd.com> <20240523074040.1611264-6-xin.wang2@amd.com> <311c645f-d0cf-45ef-abb3-c7d8a58f1130@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> On 23/05/2024 08:40, Henry Wang wrote:
> > In order to support the dynamic dtbo device assignment to a running
> > VM, the add/remove of the DT overlay and the attach/detach of the
> > device from the DT overlay should happen separately. Therefore,
> > repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
> > overlay to Xen device tree
> 
> I think it would be worth mentioning in the commit message why changing the
> sysctl behavior is fine. The feature is experimental and therefore breaking
> compatibility is ok.

Added


> > , instead of assigning the device to the
> > hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with
> > operations XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment
> > to the domain.
> > 
> > The hypervisor firstly checks the DT overlay passed from the toolstack
> > is valid. Then the device nodes are retrieved from the overlay tracker
> > based on the DT overlay. The attach of the device is implemented by
> > mapping the IRQ and IOMMU resources.
> 
> So, the expectation is the user will always want to attach all the devices in
> the overlay to a single domain. Is that correct?

Yes, also added to the commit message

> > 
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> > ---
> > v4:
> > - Split the original patch, only do the device attachment.
> > v3:
> > - Style fixes for arch-selection #ifdefs.
> > - Do not include public/domctl.h, only add a forward declaration of
> >    struct xen_domctl_dt_overlay.
> > - Extract the overlay track entry finding logic to a function, drop
> >    the unused variables.
> > - Use op code 1&2 for XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH}.
> > v2:
> > - New patch.
> > ---
> >   xen/arch/arm/domctl.c        |   3 +
> >   xen/common/dt-overlay.c      | 199 ++++++++++++++++++++++++++---------
> >   xen/include/public/domctl.h  |  14 +++
> >   xen/include/public/sysctl.h  |  11 +-
> >   xen/include/xen/dt-overlay.h |   7 ++
> >   5 files changed, 176 insertions(+), 58 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> > index ad56efb0f5..12a12ee781 100644
> > --- a/xen/arch/arm/domctl.c
> > +++ b/xen/arch/arm/domctl.c
> > @@ -5,6 +5,7 @@
> >    * Copyright (c) 2012, Citrix Systems
> >    */
> >   +#include <xen/dt-overlay.h>
> >   #include <xen/errno.h>
> >   #include <xen/guest_access.h>
> >   #include <xen/hypercall.h>
> > @@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct
> > domain *d,
> >             return rc;
> >       }
> > +    case XEN_DOMCTL_dt_overlay:
> > +        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
> >       default:
> >           return subarch_do_domctl(domctl, d, u_domctl);
> >       }
> > diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
> > index 9cece79067..1087f9b502 100644
> > --- a/xen/common/dt-overlay.c
> > +++ b/xen/common/dt-overlay.c
> > @@ -356,6 +356,42 @@ static int overlay_get_nodes_info(const void *fdto,
> > char **nodes_full_path)
> >       return 0;
> >   }
> >   +/* This function should be called with the overlay_lock taken */
> > +static struct overlay_track *
> > +find_track_entry_from_tracker(const void *overlay_fdt,
> > +                              uint32_t overlay_fdt_size)
> > +{
> > +    struct overlay_track *entry, *temp;
> > +    bool found_entry = false;
> > +
> > +    ASSERT(spin_is_locked(&overlay_lock));
> > +
> > +    /*
> > +     * First check if dtbo is correct i.e. it should one of the dtbo which
> > was
> > +     * used when dynamically adding the node.
> > +     * Limitation: Cases with same node names but different property are
> > not
> > +     * supported currently. We are relying on user to provide the same dtbo
> > +     * as it was used when adding the nodes.
> > +     */
> > +    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> > +    {
> > +        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0
> > )
> > +        {
> > +            found_entry = true;
> > +            break;
> > +        }
> > +    }
> > +
> > +    if ( !found_entry )
> > +    {
> > +        printk(XENLOG_ERR "Cannot find any matching tracker with input
> > dtbo."
> > +               " Operation is supported only for prior added dtbo.\n");
> > +        return NULL;
> > +    }
> > +
> > +    return entry;
> > +}
> > +
> >   /* Check if node itself can be removed and remove node from IOMMU. */
> >   static int remove_node_resources(struct dt_device_node *device_node)
> >   {
> > @@ -485,8 +521,7 @@ static long handle_remove_overlay_nodes(const void
> > *overlay_fdt,
> >                                           uint32_t overlay_fdt_size)
> >   {
> >       int rc;
> > -    struct overlay_track *entry, *temp, *track;
> > -    bool found_entry = false;
> > +    struct overlay_track *entry;
> >         rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> >       if ( rc )
> > @@ -494,29 +529,10 @@ static long handle_remove_overlay_nodes(const void
> > *overlay_fdt,
> >         spin_lock(&overlay_lock);
> >   -    /*
> > -     * First check if dtbo is correct i.e. it should one of the dtbo which
> > was
> > -     * used when dynamically adding the node.
> > -     * Limitation: Cases with same node names but different property are
> > not
> > -     * supported currently. We are relying on user to provide the same dtbo
> > -     * as it was used when adding the nodes.
> > -     */
> > -    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> > -    {
> > -        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0
> > )
> > -        {
> > -            track = entry;
> > -            found_entry = true;
> > -            break;
> > -        }
> > -    }
> > -
> > -    if ( !found_entry )
> > +    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
> > +    if ( entry == NULL )
> >       {
> >           rc = -EINVAL;
> > -
> > -        printk(XENLOG_ERR "Cannot find any matching tracker with input
> > dtbo."
> > -               " Removing nodes is supported only for prior added
> > dtbo.\n");
> >           goto out;
> >         }
> > @@ -620,15 +636,7 @@ static long add_nodes(struct overlay_track *tr, char
> > **nodes_full_path)
> >               return -EFAULT;
> >           }
> >   -        rc = handle_device(hardware_domain, overlay_node,
> > p2m_mmio_direct_c,
> > -                           tr->iomem_ranges,
> > -                           tr->irq_ranges);
> >           write_unlock(&dt_host_lock);
> > -        if ( rc )
> > -        {
> > -            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> > -            return rc;
> > -        }
> >             /* Keep overlay_node address in tracker. */
> >           tr->nodes_address[j] = (unsigned long)overlay_node;
> > @@ -638,9 +646,7 @@ static long add_nodes(struct overlay_track *tr, char
> > **nodes_full_path)
> >   }
> >   /*
> >    * Adds device tree nodes under target node.
> > - * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
> > This
> > - * is done to avoid the removal of device_tree generation, iomem regions
> > mapping
> > - * to hardware domain done by handle_node().
> > + * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
> >    */
> >   static long handle_add_overlay_nodes(void *overlay_fdt,
> >                                        uint32_t overlay_fdt_size)
> > @@ -774,20 +780,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
> >           goto err;
> >       }
> >   -    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays:
> > Interrupts", 0);
> > -    if (tr->irq_ranges == NULL)
> > -    {
> > -        printk(XENLOG_ERR "Creating IRQ rangeset failed");
> > -        goto err;
> > -    }
> > -
> > -    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory",
> > 0);
> > -    if (tr->iomem_ranges == NULL)
> > -    {
> > -        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
> > -        goto err;
> > -    }
> > -
> >       rc = add_nodes(tr, nodes_full_path);
> >       if ( rc )
> >       {
> > @@ -843,14 +835,83 @@ static long handle_add_overlay_nodes(void
> > *overlay_fdt,
> >       xfree(tr->nodes_address);
> >       xfree(tr->fdt);
> >   -    rangeset_destroy(tr->irq_ranges);
> > -    rangeset_destroy(tr->iomem_ranges);
> > -
> >       xfree(tr);
> >         return rc;
> >   }
> >   +static long handle_attach_overlay_nodes(struct domain *d,
> > +                                        const void *overlay_fdt,
> > +                                        uint32_t overlay_fdt_size)
> > +{
> > +    int rc;
> > +    unsigned int j;
> > +    struct overlay_track *entry;
> > +
> > +    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> > +    if ( rc )
> > +        return rc;
> > +
> > +    spin_lock(&overlay_lock);
> > +
> > +    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
> > +    if ( entry == NULL )
> > +    {
> > +        rc = -EINVAL;
> > +        goto out;
> > +    }
> > +
> > +    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
> > +    if (entry->irq_ranges == NULL)
> > +    {
> > +        rc = -ENOMEM;
> > +        printk(XENLOG_ERR "Creating IRQ rangeset failed");
> > +        goto out;
> > +    }
> > +
> > +    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
> > +    if (entry->iomem_ranges == NULL)
> > +    {
> > +        rc = -ENOMEM;
> > +        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
> > +        goto out;
> > +    }
> > +
> > +    for ( j = 0; j < entry->num_nodes; j++ )
> > +    {
> > +        struct dt_device_node *overlay_node;
> > +
> > +        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
> > +        if ( overlay_node == NULL )
> > +        {
> > +            rc = -EINVAL;
> > +            goto out;
> > +        }
> > +
> > +        write_lock(&dt_host_lock);
> > +        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
> > +                           entry->iomem_ranges, entry->irq_ranges);
> > +        write_unlock(&dt_host_lock);
> > +        if ( rc )
> > +        {
> > +            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> > +            goto out;
> > +        }
> > +    }
> > +
> > +    spin_unlock(&overlay_lock);
> > +
> > +    return 0;
> > +
> > + out:
> > +    spin_unlock(&overlay_lock);
> > +
> > +    rangeset_destroy(entry->irq_ranges);
> > +    rangeset_destroy(entry->iomem_ranges);
> > +
> > +    return rc;
> > +}
> > +
> >   long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
> >   {
> >       long ret;
> > @@ -890,6 +951,42 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay
> > *op)
> >       return ret;
> >   }
> >   +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay
> > *op)
> > +{
> > +    long ret;
> > +    void *overlay_fdt;
> > +
> > +    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
> > +        return -EOPNOTSUPP;
> > +
> > +    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
> 
> Let's avoid to hardocode KB(500) a second time (it alredy exists in the sysctl
> path).
> 
> But I would actually consider to allow up to 512KB because...

I left it as is because I wasn't sure what you wanted as a change. Did
you mean that you would like the check to be removed, or did you mean
that you would like an #define instead of writing KB(500)?

It is fine either way for me, or to keep it as is, please make the
desired change on commit.


> > +        return -EINVAL;
> > +
> > +    if ( op->pad[0] || op->pad[1] || op->pad[2] )
> > +        return -EINVAL;
> > +
> > +    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
> 
> ... xmalloc_bytes() will be more efficient. If the size is not power of 2
> pages, it will free the leftover. Anyway, this could be a follow-up patch.

Leaving this as is for now


> > +
> > +    if ( overlay_fdt == NULL )
> > +        return -ENOMEM;
> > +
> > +    ret = copy_from_guest(overlay_fdt, op->overlay_fdt,
> > op->overlay_fdt_size);
> > +    if ( ret )
> > +    {
> > +        gprintk(XENLOG_ERR, "copy from guest failed\n");
> > +        xfree(overlay_fdt);
> > +
> > +        return -EFAULT;
> > +    }
> > +
> > +    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
> > +        ret = handle_attach_overlay_nodes(d, overlay_fdt,
> > op->overlay_fdt_size);
> 
> I think you would return 0 if the operation is not supported. But I think we
> need to return -EOPNOTSUPP.

I made the change


> > +
> > +    xfree(overlay_fdt);
> > +
> > +    return ret;
> > +}
> > +
> >   /*
> >    * Local variables:
> >    * mode: C
> > diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> > index a33f9ec32b..ac3c2a7c4c 100644
> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> 
> If I am not mistkane, this is the first change in the domctl header for 4.19.
> So you want to bump XEN_DOMCTL_INTERFACE_VERSION.

Done


> > @@ -1190,6 +1190,16 @@ struct xen_domctl_vmtrace_op {
> >   typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
> >   DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
> >   +#if defined(__arm__) || defined(__aarch64__)
> > +struct xen_domctl_dt_overlay {
> > +    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
> > +    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
> > +#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
> 
> OOI, why not starting from 0?

I think it was to avoid the risk of confusing zero as uninitialized with
a real operation.


> > +    uint8_t overlay_op;                     /* IN: Attach. */
> > +    uint8_t pad[3];                         /* IN: Must be zero. */
> > +};
> > +#endif
> > +
> >   struct xen_domctl {
> >       uint32_t cmd;
> >   #define XEN_DOMCTL_createdomain                   1
> > @@ -1277,6 +1287,7 @@ struct xen_domctl {
> >   #define XEN_DOMCTL_vmtrace_op                    84
> >   #define XEN_DOMCTL_get_paging_mempool_size       85
> >   #define XEN_DOMCTL_set_paging_mempool_size       86
> > +#define XEN_DOMCTL_dt_overlay                    87
> >   #define XEN_DOMCTL_gdbsx_guestmemio            1000
> >   #define XEN_DOMCTL_gdbsx_pausevcpu             1001
> >   #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
> > @@ -1339,6 +1350,9 @@ struct xen_domctl {
> >           struct xen_domctl_vuart_op          vuart_op;
> >           struct xen_domctl_vmtrace_op        vmtrace_op;
> >           struct xen_domctl_paging_mempool    paging_mempool;
> > +#if defined(__arm__) || defined(__aarch64__)
> > +        struct xen_domctl_dt_overlay        dt_overlay;
> > +#endif
> >           uint8_t                             pad[128];
> >       } u;
> >   };
> > diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
> > index febaa4b16a..3a6e7d48f0 100644
> > --- a/xen/include/public/sysctl.h
> > +++ b/xen/include/public/sysctl.h
> > @@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy
> > xen_sysctl_cpu_policy_t;
> >   DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
> >   #endif
> >   -#if defined(__arm__) || defined (__aarch64__)
> > +#if defined(__arm__) || defined(__aarch64__)
> 
> This seems like unrelated change. Maybe mention in the commit message that you
> are fixing the coding style?

Done


> >   /*
> >    * XEN_SYSCTL_dt_overlay
> > - * Performs addition/removal of device tree nodes under parent node using
> > dtbo.
> > - * This does in three steps:
> > - *  - Adds/Removes the nodes from dt_host.
> > - *  - Adds/Removes IRQ permission for the nodes.
> > - *  - Adds/Removes MMIO accesses.
> > + * Performs addition/removal of device tree nodes under parent node using
> > dtbo
> > + * from dt_host.
> >    */
> >   struct xen_sysctl_dt_overlay {
> >       XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
> > @@ -1265,7 +1262,7 @@ struct xen_sysctl {
> >           struct xen_sysctl_cpu_policy        cpu_policy;
> >   #endif
> >   -#if defined(__arm__) || defined (__aarch64__)
> > +#if defined(__arm__) || defined(__aarch64__)
> >           struct xen_sysctl_dt_overlay        dt_overlay;
> >   #endif
> >           uint8_t                             pad[128];
> > diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
> > index c0567741ee..7f0f3741b5 100644
> > --- a/xen/include/xen/dt-overlay.h
> > +++ b/xen/include/xen/dt-overlay.h
> > @@ -39,15 +39,22 @@ struct overlay_track {
> >   };
> >     struct xen_sysctl_dt_overlay;
> > +struct xen_domctl_dt_overlay;
> >     #ifdef CONFIG_OVERLAY_DTB
> >   long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
> > +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
> >   #else
> >   #include <xen/errno.h>
> >   static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
> >   {
> >       return -EOPNOTSUPP;
> >   }
> 
> NIT: Newline here please.

Done

> 
> > +static inline long dt_overlay_domctl(struct domain *d,
> > +                                     struct xen_domctl_dt_overlay *op)
> > +{
> > +    return -EOPNOTSUPP;
> > +}
> >   #endif
> >     #endif /* __XEN_DT_OVERLAY_H__ */
> 
> Cheers,
> 
> -- 
> Julien Grall
> 


From xen-devel-bounces@lists.xenproject.org Fri May 24 02:30:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 02:30:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729035.1134175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKh9-0003VL-83; Fri, 24 May 2024 02:30:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729035.1134175; Fri, 24 May 2024 02:30:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAKh9-0003VA-5E; Fri, 24 May 2024 02:30:11 +0000
Received: by outflank-mailman (input) for mailman id 729035;
 Fri, 24 May 2024 02:30:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAKWk-0003tc-KI
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 02:19:26 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 09d8cb2f-1974-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 04:19:25 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id CBA4B62F9E;
 Fri, 24 May 2024 02:19:23 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7BE25C2BD10;
 Fri, 24 May 2024 02:19:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09d8cb2f-1974-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716517163;
	bh=ww9wwzaIabqdPH5v4U3tm6QEcQmHmaxF/Yt+pirjSPU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ssZlT1lj9Zv7nDQYLzT8SnolJr8VbY+FEWccmnq6v74gq9vvHv134yqtntovF2U3+
	 PvijpMk3St2FmkTML6kGjk7v7Chznc/7RAaeaFmvAb63gp6iEd2bB8+MaEucvkD0dd
	 r8ZuYcYAU9gAuwiRAW0Hw39Aj+roNtXvciAJakODwdG+TDFR8H8eMpA97EOgVzpMhT
	 Y2+m0xZgkgDdUpVzK6MoZASXLSvBAddo6zmpUpm2ev+ZSOHwD6sQ7ESH4a2kNsvaWE
	 WRj+lnzqpBzXXhxA7cHLr5dHB2L1RzgojSDyGeYa7JRD7UHYIKAY6PokXJ983EnbCz
	 orWVek/To9fMw==
Date: Thu, 23 May 2024 19:19:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Anthony PERARD <anthony@xenproject.org>, 
    George Dunlap <george.dunlap@citrix.com>, 
    Nick Rosbrook <rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>, 
    Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v4 3/9] tools/arm: Introduce the "nr_spis" xl config
 entry
In-Reply-To: <02e22734-2e0c-4a0a-8c5a-42beee0cc396@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405231802010.2557291@ubuntu-linux-20-04-desktop>
References: <20240523074040.1611264-1-xin.wang2@amd.com> <20240523074040.1611264-4-xin.wang2@amd.com> <02e22734-2e0c-4a0a-8c5a-42beee0cc396@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> On 23/05/2024 08:40, Henry Wang wrote:
> > Currently, the number of SPIs allocated to the domain is only
> > configurable for Dom0less DomUs. Xen domains are supposed to be
> > platform agnostics and therefore the numbers of SPIs for libxl
> > guests should not be based on the hardware.
> > 
> > Introduce a new xl config entry for Arm to provide a method for
> > user to decide the number of SPIs. This would help to avoid
> > bumping the `config->arch.nr_spis` in libxl everytime there is a
> > new platform with increased SPI numbers.
> > 
> > Update the doc and the golang bindings accordingly.
> > 
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> > ---
> > v4:
> > - Add Jason's Reviewed-by tag.
> > v3:
> > - Reword documentation to avoid ambiguity.
> > v2:
> > - New patch to replace the original patch in v1:
> >    "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160"
> > ---
> >   docs/man/xl.cfg.5.pod.in             | 14 ++++++++++++++
> >   tools/golang/xenlight/helpers.gen.go |  2 ++
> >   tools/golang/xenlight/types.gen.go   |  1 +
> >   tools/libs/light/libxl_arm.c         |  4 ++--
> >   tools/libs/light/libxl_types.idl     |  1 +
> >   tools/xl/xl_parse.c                  |  3 +++
> >   6 files changed, 23 insertions(+), 2 deletions(-)
> > 
> > diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> > index 8f2b375ce9..416d582844 100644
> > --- a/docs/man/xl.cfg.5.pod.in
> > +++ b/docs/man/xl.cfg.5.pod.in
> > @@ -3072,6 +3072,20 @@ raised.
> >     =back
> >   +=over 4
> > +
> > +=item B<nr_spis="NR_SPIS">
> > +
> > +An optional 32-bit integer parameter specifying the number of SPIs (Shared
> 
> We can't support that much SPIs :). The limit would be 991 SPIs.

I change it


> > +Peripheral Interrupts) to allocate for the domain. If the value specified
> > by
> > +the `nr_spis` parameter is smaller than the number of SPIs calculated by
> > the
> > +toolstack based on the devices allocated for the domain, or the `nr_spis`
> > +parameter is not specified, the value calculated by the toolstack will be
> > used
> > +for the domain. Otherwise, the value specified by the `nr_spis` parameter
> > will
> > +be used.
> 
> I think it would be worth mentioning that the number of SPIs should match the
> highest interrupt ID that will be assigned to the domain (rather than the
> number of SPIs planned to be assigned).

I added it


> > +
> > +=back
> > +
> >   =head3 x86
> >     =over 4
> > diff --git a/tools/golang/xenlight/helpers.gen.go
> > b/tools/golang/xenlight/helpers.gen.go
> > index b9cb5b33c7..fe5110474d 100644
> > --- a/tools/golang/xenlight/helpers.gen.go
> > +++ b/tools/golang/xenlight/helpers.gen.go
> > @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
> >   x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
> >   x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
> >   x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
> > +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
> >   if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil
> > {
> >   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
> >   }
> > @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
> >   xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
> >   xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
> >   xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
> > +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
> >   if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
> >   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
> >   }
> > diff --git a/tools/golang/xenlight/types.gen.go
> > b/tools/golang/xenlight/types.gen.go
> > index 5b293755d7..c9e45b306f 100644
> > --- a/tools/golang/xenlight/types.gen.go
> > +++ b/tools/golang/xenlight/types.gen.go
> > @@ -597,6 +597,7 @@ ArchArm struct {
> >   GicVersion GicVersion
> >   Vuart VuartType
> >   SveVl SveType
> > +NrSpis uint32
> >   }
> >   ArchX86 struct {
> >   MsrRelaxed Defbool
> > diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> > index 1cb89fa584..a4029e3ac8 100644
> > --- a/tools/libs/light/libxl_arm.c
> > +++ b/tools/libs/light/libxl_arm.c
> > @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> >         LOG(DEBUG, "Configure the domain");
> >   -    config->arch.nr_spis = nr_spis;
> > -    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
> > +    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
> 
> I am not entirely sure about using max(). To me if the user specifies a lower
> limit, then we should throw an error because this is likely an indication that
> the SPIs they will want to assign will clash with the emulated ones.
> 
> So it would be better to warn at domain creation rather than waiting until the
> IRQs are assigned.
> 
> I would like Anthony's opinion on this one. Given he is away this month, I
> guess we could get this patch merged (with other comments addressed) and have
> a follow-up if wanted before 4.19.

I left it as is for now


> > +    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
> >         switch (d_config->b_info.arch_arm.gic_version) {
> >       case LIBXL_GIC_VERSION_DEFAULT:
> > diff --git a/tools/libs/light/libxl_types.idl
> > b/tools/libs/light/libxl_types.idl
> > index 79e9c656cc..4e65e6fda5 100644
> > --- a/tools/libs/light/libxl_types.idl
> > +++ b/tools/libs/light/libxl_types.idl
> > @@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
> >       ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
> >                                  ("vuart", libxl_vuart_type),
> >                                  ("sve_vl", libxl_sve_type),
> > +                               ("nr_spis", uint32),
> 
> From my understandig, any change in the .idl requires a corresponding
> LIBXL_HAVE_... in include/libxl.h. This is in order to allow external
> toolstack (such as libvirt) to be able to select at build time between
> multiple version of libxl.

I added it


> >                                 ])),
> >       ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
> >                                 ])),
> > diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> > index c504ab3711..e3a4800f6e 100644
> > --- a/tools/xl/xl_parse.c
> > +++ b/tools/xl/xl_parse.c
> > @@ -2935,6 +2935,9 @@ skip_usbdev:
> >           }
> >       }
> >   +    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
> > +        b_info->arch_arm.nr_spis = l;
> > +
> >       parse_vkb_list(config, d_config);
> >         d_config->virtios = NULL;
> 
> Cheers,
> 
> -- 
> Julien Grall
> 


From xen-devel-bounces@lists.xenproject.org Fri May 24 04:10:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 04:10:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729061.1134189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAMG3-0002Si-3j; Fri, 24 May 2024 04:10:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729061.1134189; Fri, 24 May 2024 04:10:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAMG2-0002Sb-WA; Fri, 24 May 2024 04:10:18 +0000
Received: by outflank-mailman (input) for mailman id 729061;
 Fri, 24 May 2024 04:10:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAMG1-0002SR-Iq; Fri, 24 May 2024 04:10:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAMG1-0004wM-FA; Fri, 24 May 2024 04:10:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAMG1-0001Q4-0C; Fri, 24 May 2024 04:10:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAMG0-00076S-Vu; Fri, 24 May 2024 04:10:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mVrL0x6mgS5IFnWSHxKh1OEdl1xZCr5tDX3f2sjDNng=; b=E8jyqRb/OzaLsLvSu5LLluryPz
	Zw6+crucjKiXP+ZKvNZmGbPZyFAvDW0wOEd75+V4zmvKEivOgvkYSep4GIRM+hJgxLodqxQg1gwiI
	uDbeh7Q8OlkP0cWPuI15QsTVFmJAMe31BLniYpBIEDq0Bt41QrAo/3jXZ/DHcWIbDGqI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186105-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186105: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-armhf:xen-build:fail:regression
    xen-unstable:build-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 04:10:16 +0000

flight 186105 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186105/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186078

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186078
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186078
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186078
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186078
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186078
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186105  2024-05-23 09:38:07 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Fri May 24 06:48:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 06:48:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729087.1134211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAOjD-00024Z-8C; Fri, 24 May 2024 06:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729087.1134211; Fri, 24 May 2024 06:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAOjD-00024S-4u; Fri, 24 May 2024 06:48:35 +0000
Received: by outflank-mailman (input) for mailman id 729087;
 Fri, 24 May 2024 06:48:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=s7SU=M3=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAOjB-00024M-QK
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 06:48:33 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a17edb0f-1999-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 08:48:30 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a6269ad7db2so42998366b.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 23:48:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93a807sm78653366b.85.2024.05.23.23.48.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 23:48:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a17edb0f-1999-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716533310; x=1717138110; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=f2GVa449HW6Mxmf8Yz112W0MFbdzqzb+nt302Z0RJYw=;
        b=YZLjRhG6IHsGqWUaE7FZnSUWP9+Ur3GsfLrB0lecJ4E8wUfr5KmbBONH+J9W6teEDT
         ToFQtEnMUx6EyVby3I8V3/Gw+33hQMLQFvbqdtgAaipfJpFKXCS0VQaKODYu/4TmZSMg
         5p2W2Y3nQx9dt36o8522k01tGZ83BG5z8z3IM2LHHvGj8ObNqpSk05DbSmV7HSfitb8Y
         xIjcoxy1BBGCopaQXwNwpAWpTs0Za2b0L7L3y7SyQu+fzXOEKH2E45ARYjo67iYNmpEs
         rGketC1EtE8Esl9SqUZQ67K0Jgci6HvTvS6ZNrWshl/zY+9clo50wlqVG6p+QykcnffR
         qSHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716533310; x=1717138110;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=f2GVa449HW6Mxmf8Yz112W0MFbdzqzb+nt302Z0RJYw=;
        b=Au+ASZFu7t7iE58Dl+jXhzrYWM/5l1ztfjpvPbLgojDZtn4TbptDCxYj9SOqNNfvFF
         TjJOyIhg/lGllGm2qvq48EOZoymCfbZWzZL9KclQzmlLg/hjCeB1Jwzz16uvWeZGJOra
         ofRjdnV0SwrmrB+q1nBqNgV7UjCMWDQVk/BNDoWHuLQRITezvyjgDhCM4PJp+J+WDVgf
         7f1kT1zHl62p7vZxh1HrUXOuh8hoUNDV6LBo1U9P5r4/SaGD0TTUFcUDFL5bqgjMquxj
         TedJQ//RkPN8vHD8FllMCvXlfi9G5IyT1uIBRjSxlN7SaCckRYYVhNfG6ss++Eno4eE3
         7A3A==
X-Forwarded-Encrypted: i=1; AJvYcCUnfaUEYIurEC6wpTw/KSBABOTDpTDKIXFB/vU/zGXSxwpjfCuSwyz/Kk20T9jAdSB8qI6jk+zynKi0Yoc0B6gArg3GIyOEPq+siWekbQ0=
X-Gm-Message-State: AOJu0YwDWn4YDS5wtUV0D6ttUoCy54FuX5nZfjM1gH/+pKE32OAW0CrK
	wbqHxm8ihLhwIjxPKxkKzouhWP+Ot8eU8oMRpn3gc63S4t7p0wZiDeMBWo4foA==
X-Google-Smtp-Source: AGHT+IHFXujYj49jtg7pzKAGb5tN4EFkGz94J4b+XO00RUGKFCDqzbwM/MXSavMKNUu/CbDYQk8Qag==
X-Received: by 2002:a17:906:2509:b0:a5c:db98:c2a0 with SMTP id a640c23a62f3a-a6264f0e4e4mr83686866b.58.1716533309609;
        Thu, 23 May 2024 23:48:29 -0700 (PDT)
Message-ID: <ed0fec75-2b8c-4003-94ca-edbbb06e185f@suse.com>
Date: Fri, 24 May 2024 08:48:27 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org, Julien Grall <julien@xen.org>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
 <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.05.2024 18:40, Oleksii K. wrote:
> On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
>> On 23/05/2024 15:11, Oleksii K. wrote:
>>> On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
>>>> On 17/05/2024 14:54, Oleksii Kurochko wrote:
>>>>> diff --git a/xen/arch/arm/arm64/livepatch.c
>>>>> b/xen/arch/arm/arm64/livepatch.c
>>>>> index df2cebedde..4bc8ed9be5 100644
>>>>> --- a/xen/arch/arm/arm64/livepatch.c
>>>>> +++ b/xen/arch/arm/arm64/livepatch.c
>>>>> @@ -10,7 +10,6 @@
>>>>>    #include <xen/mm.h>
>>>>>    #include <xen/vmap.h>
>>>>>    
>>>>> -#include <asm/bitops.h>
>>>>
>>>> It is a bit unclear how this change is related to the patch. Can
>>>> you
>>>> explain in the commit message?
>>> Probably it doesn't need anymore. I will double check and if this
>>> change is not needed, I will just drop it in the next patch
>>> version.
>>>
>>>>
>>>>>    #include <asm/byteorder.h>
>>>>>    #include <asm/insn.h>
>>>>>    #include <asm/livepatch.h>
>>>>> diff --git a/xen/arch/arm/include/asm/bitops.h
>>>>> b/xen/arch/arm/include/asm/bitops.h
>>>>> index 5104334e48..8e16335e76 100644
>>>>> --- a/xen/arch/arm/include/asm/bitops.h
>>>>> +++ b/xen/arch/arm/include/asm/bitops.h
>>>>> @@ -22,9 +22,6 @@
>>>>>    #define __set_bit(n,p)            set_bit(n,p)
>>>>>    #define __clear_bit(n,p)          clear_bit(n,p)
>>>>>    
>>>>> -#define BITOP_BITS_PER_WORD     32
>>>>> -#define BITOP_MASK(nr)          (1UL << ((nr) %
>>>>> BITOP_BITS_PER_WORD))
>>>>> -#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
>>>>>    #define BITS_PER_BYTE           8
>>>>
>>>> OOI, any reason BITS_PER_BYTE has not been moved as well? I don't
>>>> expect
>>>> the value to change across arch.
>>> I can move it to generic one header too in the next patch version.
>>>
>>>>
>>>> [...]
>>>>
>>>>> diff --git a/xen/include/xen/bitops.h
>>>>> b/xen/include/xen/bitops.h
>>>>> index f14ad0d33a..6eeeff0117 100644
>>>>> --- a/xen/include/xen/bitops.h
>>>>> +++ b/xen/include/xen/bitops.h
>>>>> @@ -65,10 +65,141 @@ static inline int generic_flsl(unsigned
>>>>> long
>>>>> x)
>>>>>     * scope
>>>>>     */
>>>>>    
>>>>> +#define BITOP_BITS_PER_WORD 32
>>>>> +typedef uint32_t bitop_uint_t;
>>>>> +
>>>>> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) %
>>>>> BITOP_BITS_PER_WORD))
>>>>> +
>>>>> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
>>>>> +
>>>>> +extern void __bitop_bad_size(void);
>>>>> +
>>>>> +#define bitop_bad_size(addr) (sizeof(*(addr)) <
>>>>> sizeof(bitop_uint_t))
>>>>> +
>>>>>    /* --------------------- Please tidy above here ------------
>>>>> -----
>>>>> ---- */
>>>>>    
>>>>>    #include <asm/bitops.h>
>>>>>    
>>>>> +/**
>>>>> + * generic__test_and_set_bit - Set a bit and return its old
>>>>> value
>>>>> + * @nr: Bit to set
>>>>> + * @addr: Address to count from
>>>>> + *
>>>>> + * This operation is non-atomic and can be reordered.
>>>>> + * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>> + */
>>>>
>>>> Sorry for only mentioning this on v10. I think this comment
>>>> should be
>>>> duplicated (or moved to) on top of test_bit() because this is
>>>> what
>>>> everyone will use. This will avoid the developper to follow the
>>>> function
>>>> calls and only notice the x86 version which says "This function
>>>> is
>>>> atomic and may not be reordered." and would be wrong for all the
>>>> other arch.
>>> It makes sense to add this comment on top of test_bit(), but I am
>>> curious if it is needed to mention that for x86 arch_test_bit() "is
>>> atomic and may not be reordered":
>>
>> I would say no because any developper modifying common code can't 
>> relying it.
>>
>>>
>>>   * This operation is non-atomic and can be reordered. ( Exception:
>>> for
>>> * x86 arch_test_bit() is atomic and may not be reordered )
>>>   * If two examples of this operation race, one can appear to
>>> succeed
>>>   * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>>   */
>>>
>>>>
>>>>> +static always_inline bool
>>>>> +generic__test_and_set_bit(int nr, volatile void *addr)
>>>>> +{
>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>>>> BITOP_WORD(nr);
>>>>> +    bitop_uint_t old = *p;
>>>>> +
>>>>> +    *p = old | mask;
>>>>> +    return (old & mask);
>>>>> +}
>>>>> +
>>>>> +/**
>>>>> + * generic__test_and_clear_bit - Clear a bit and return its
>>>>> old
>>>>> value
>>>>> + * @nr: Bit to clear
>>>>> + * @addr: Address to count from
>>>>> + *
>>>>> + * This operation is non-atomic and can be reordered.
>>>>> + * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>> + */
>>>>
>>>> Same applies here and ...
>>>>
>>>>> +static always_inline bool
>>>>> +generic__test_and_clear_bit(int nr, volatile void *addr)
>>>>> +{
>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>>>> BITOP_WORD(nr);
>>>>> +    bitop_uint_t old = *p;
>>>>> +
>>>>> +    *p = old & ~mask;
>>>>> +    return (old & mask);
>>>>> +}
>>>>> +
>>>>> +/* WARNING: non atomic and it can be reordered! */
>>>>
>>>> ... here.
>>>>
>>>>> +static always_inline bool
>>>>> +generic__test_and_change_bit(int nr, volatile void *addr)
>>>>> +{
>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
>>>>> BITOP_WORD(nr);
>>>>> +    bitop_uint_t old = *p;
>>>>> +
>>>>> +    *p = old ^ mask;
>>>>> +    return (old & mask);
>>>>> +}
>>>>> +/**
>>>>> + * generic_test_bit - Determine whether a bit is set
>>>>> + * @nr: bit number to test
>>>>> + * @addr: Address to start counting from
>>>>> + */
>>>>> +static always_inline bool generic_test_bit(int nr, const
>>>>> volatile
>>>>> void *addr)
>>>>> +{
>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>> +    const volatile bitop_uint_t *p =
>>>>> +                        (const volatile bitop_uint_t *)addr +
>>>>> BITOP_WORD(nr);
>>>>> +
>>>>> +    return (*p & mask);
>>>>> +}
>>>>> +
>>>>> +static always_inline bool
>>>>> +__test_and_set_bit(int nr, volatile void *addr)
>>>>> +{
>>>>> +#ifndef arch__test_and_set_bit
>>>>> +#define arch__test_and_set_bit generic__test_and_set_bit
>>>>> +#endif
>>>>> +
>>>>> +    return arch__test_and_set_bit(nr, addr);
>>>>> +}
>>>>
>>>> NIT: It is a bit too late to change this one. But I have to
>>>> admit, I
>>>> don't understand the purpose of the static inline when you could
>>>> have
>>>> simply call...
>>>>
>>>>> +#define __test_and_set_bit(nr, addr) ({             \
>>>>> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
>>>>> +    __test_and_set_bit(nr, addr);                   \
>>>>
>>>> ... __arch__test_and_set_bit here.
>>> I just wanted to be in sync with other non-atomic test*_bit()
>>> operations and Andrew's patch series ( xen/bitops: Reduce the mess,
>>> starting with ffs() ).
>>
>> I looked at Andrew series and I can't seem to find an example where
>> he 
>> uses both the macro + static inline. He seems only use a static
>> inline.
>> Do you have any pointer?
>>
>> But by any chance are you referring to the pattern on x86? If so, I 
>> would really like to understand what's the benefits of introducing a
>> a 
>> one-liner static inline which is "shadowed" by a macro...
> Yes, I was referring to the x86 pattern.
> 
> I tried to find the reason in the commit but couldn't:
> https://lists.xenproject.org/archives/html/xen-changelog/2008-03/msg00097.html
> 
> For some reason, I also couldn't find the mailing list thread for this.
> 
> Perhaps Jan could help here, based on the commit message he might have
> a suggestion.

There's a lot of apparently unrelated context left, so I'm trying to guess
on what the actual question is, from the old change you're pointing at: With

#define __test_and_set_bit(nr, addr) ({             \
   if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
   __test_and_set_bit(nr, addr);                   \

why do we have that wrapping macro? If that's indeed the question, then may
I suggest you consider what would happen to the sizeof() in bitop_bad_size()
if the invocation was moved to the inline function, taking pointer-to-void?

However, I can't relate that old change to the question I think Julien
raised (invoking __test_and_set_bit() vs arch__test_and_set_bit()), so
perhaps I'm missing something.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 24 06:49:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 06:49:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729092.1134220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAOkO-0002gP-KM; Fri, 24 May 2024 06:49:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729092.1134220; Fri, 24 May 2024 06:49:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAOkO-0002gI-Hh; Fri, 24 May 2024 06:49:48 +0000
Received: by outflank-mailman (input) for mailman id 729092;
 Fri, 24 May 2024 06:49:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=s7SU=M3=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAOkN-0002g4-9H
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 06:49:47 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ce304178-1999-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 08:49:45 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a5a4bc9578cso1205498666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 23 May 2024 23:49:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93a807sm78653366b.85.2024.05.23.23.49.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 May 2024 23:49:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce304178-1999-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716533385; x=1717138185; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FnPAIiIXWxMiuZAd3kSHvNQulM6dhBK4pPIRRxXJNMM=;
        b=TLQ5h6KSUJYuZ0KAIh9gdMKbb3hKpCYapR6kolUjfVj3OiMxIcOPmbCuL0YhlIb3l5
         wCHY83TIMNkwrVhG6qFxcLHQ/xj4nSep17S5OWMJoaCX71Yz8REiUXcz2f0jlVs0/zBA
         NWmjnhlHqOLF4rGCWKhFLfEQ9fJmfONVmg3aK1joETpNndNc0oZCLjd9hFwCxUOMsqmF
         6cPiLIBP0Slssgyicq9rhYfan/x/IzIVk8jjRmMGv4vTKIrXkFlyEg/OxyrjA6U5NgA5
         Fc0GkbQZKzgFuuJG60ghkwYn2RAf4FffRJydCxBqPMdjPLeW3Lgzqx3STR7xS/yaaF1S
         1w+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716533385; x=1717138185;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FnPAIiIXWxMiuZAd3kSHvNQulM6dhBK4pPIRRxXJNMM=;
        b=BsoSlCuGWCtvPqC2dcbvzLctAJu103CdY2SbtsCzZ1jkqKmW8pMTR3y6MhQiVpZg/b
         mo8FyLh1eUhcWAYFnlB2HO5UkJzbiIZ6f3UsoJtUdRwtocR4b2NE5YAM01SGDGsG0y31
         WGo5wN/sMb35o+JTTm3aWpzGMSO7UbdmVOflXToOKK2D7Y4Hau/VQbM2p+8J+l7+1TiA
         1hIDiNXMX/zMqYhl86GZdeU+1C1xF0jJ8P7VimZifTqCc+eeZ0w6mbVVWErKMzoTq4mX
         G3R9bRxgv6r6vW7JUkzwvIaYjlPE9T8FGTB5d8Eh0Wm+Pek3YdDEzm69NLMuOyRlBhtk
         /SgA==
X-Forwarded-Encrypted: i=1; AJvYcCUSTpWA9KtQ6RyJVtrHm6LYp5/GEoW2BoFDqY86preyN3Qm+iY2pDFbj96G7cLhxQE2MmKD689QKo8VTfgh5HVf5y71cLgWpPkSyPJZVr0=
X-Gm-Message-State: AOJu0YyAxlwtvDa8A/qgeXVxSdw3BukGJ1SGD0CP2TEHij/+Tzd7VxwO
	HGHmBzP/I7+XJTHANO9DuMQm7ujRubHa0cuBrrVewGoslb68H7ducDqzUEtHEw==
X-Google-Smtp-Source: AGHT+IHEgyKuceJTC55l+lQaiejqdPEnqPBG7f3Ac6quwNXjgIp9yfMmsaZ/bdVoaeeVRrjHcfthiQ==
X-Received: by 2002:a17:906:2ada:b0:a59:c941:e5e5 with SMTP id a640c23a62f3a-a62646d5a8amr124102066b.45.1716533384790;
        Thu, 23 May 2024 23:49:44 -0700 (PDT)
Message-ID: <7caffe42-3fa2-406d-94f0-fce21b44d52b@suse.com>
Date: Fri, 24 May 2024 08:49:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Sergiy Kibrik <sergiy_kibrik@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
 <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
 <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
 <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
 <alpine.DEB.2.22.394.2405231634290.2557291@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2405231634290.2557291@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 01:36, Stefano Stabellini wrote:
> On Thu, 23 May 2024, Jan Beulich wrote:
>> On 23.05.2024 15:07, Sergiy Kibrik wrote:
>>> 16.05.24 14:12, Jan Beulich:
>>>> On 15.05.2024 11:12, Sergiy Kibrik wrote:
>>>>> --- a/xen/arch/x86/include/asm/cpufeature.h
>>>>> +++ b/xen/arch/x86/include/asm/cpufeature.h
>>>>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>>   #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>>>>>   #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>>>>>   #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
>>>>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
>>>>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
>>>>> +                                  boot_cpu_has(X86_FEATURE_VMX))
>>>>>   #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>>>>>   #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>>>>>   #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
>>>>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>>   
>>>>>   /* CPUID level 0x80000001.ecx */
>>>>>   #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
>>>>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
>>>>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
>>>>> +                                  boot_cpu_has(X86_FEATURE_SVM))
>>>>>   #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>>>>>   #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>>>>>   #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
>>>>
>>>> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
>>>> and as a result one-off indentation on the wrapped lines) I'm not really
>>>> certain we can do this. The description goes into detail why we would want
>>>> this, but it doesn't cover at all why it is safe for all present (and
>>>> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
>>>> just to derive further knowledge from that, without them being directly
>>>> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
>>>> for example. While it looks to be okay there, it may give you an idea of
>>>> what I mean.
>>>>
>>>> Things might become better separated if instead for such checks we used
>>>> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
>>>> that's still pretty far out, I'm afraid.
>>>
>>> I've followed a suggestion you made for patch in previous series:
>>>
>>> https://lore.kernel.org/xen-devel/8fbd604e-5e5d-410c-880f-2ad257bbe08a@suse.com/
>>
>> See the "If not, ..." that I had put there. Doing the change just mechanically
>> isn't enough, you also need to make clear (in the description) that you
>> verified it's safe to have this way.
> 
> What does it mean to "verified it's safe to have this way"? "Safe" in
> what way?

"Safe" as in "not breaking existing logic", anywhere.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 24 07:21:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 07:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729104.1134231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPEs-0007SZ-0f; Fri, 24 May 2024 07:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729104.1134231; Fri, 24 May 2024 07:21:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPEr-0007SS-St; Fri, 24 May 2024 07:21:17 +0000
Received: by outflank-mailman (input) for mailman id 729104;
 Fri, 24 May 2024 07:21:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAPEr-0007S6-5i
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 07:21:17 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 34852698-199e-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 09:21:15 +0200 (CEST)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-4200ee78f34so27640125e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 00:21:15 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100fbb140sm45215505e9.44.2024.05.24.00.21.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 00:21:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34852698-199e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716535274; x=1717140074; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Ot/MgWABH9HqREAPl6rw3Run923e5V+E6Z3ZVH5fA+c=;
        b=H3ulg6kl4et+qSlsK7xcEiJsbAa8TmqdDZOKi4/kBX8mlmcGOxuT7N0c3EIoGCVhS4
         A4gye9VGe1LyJwo923/8+GkQwFA/H9ewJI0iYNtHNA6NpjBVKgJs+p3uRNCA0eLW+j/p
         XNta9cU71pDdTwZvo9eTAbemhjz/U6Dv+O4io=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716535274; x=1717140074;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Ot/MgWABH9HqREAPl6rw3Run923e5V+E6Z3ZVH5fA+c=;
        b=bwD8da2Y2cJ+t1Gyp1qSmbZ6+al1KUHebJl6yR7rzYUFtUY8DAPlAqMmE4+pnb8UC6
         4JfPlAmKdUhg19ohrqwLIhVCVQuWUBqP9m4a4oBzZ9/mtND5E+v5aA+TGHnHorCRpxuJ
         v+z4qsHyesNGJe9v+RtAnxz7YkAFq1x5BTEK3Zn17F/AdfYrFbDh7gdacQoggy/8QPUr
         7GX4YX/Z1all1Bb09AhVQJ3Kmffksn/6nQ2q7rA8A6q46LS099H38SOp4hKpfuVO+7/m
         6OfJvnKQ8X5WK+zTHwAI9WKmZqmA8IITT0+1bWMdbP+dyWNCdvH2DVGaXfmV8ET4qigv
         nNzw==
X-Gm-Message-State: AOJu0Yyjsd/xPRWbbHW86Sdads36yGVck6vf0KKWhy3qtnO+FTXJCjFG
	v4xX1AAvj1QpZw9Zq/dzIjZyooR53028plM8IT06j9kpMsVfw8LnxoFi9SGheV/0VwNuGB1KsGc
	f
X-Google-Smtp-Source: AGHT+IExFByH1foFIEoIqFB6OUUiH2/8wABd6YVSG6aqJhNwT64F+8HLY41ZqppFzIzMa8QPsy7Nnw==
X-Received: by 2002:a05:600c:6a93:b0:41b:f359:2b53 with SMTP id 5b1f17b1804b1-42108a14eaemr11195605e9.37.1716535274219;
        Fri, 24 May 2024 00:21:14 -0700 (PDT)
Date: Fri, 24 May 2024 09:21:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
Message-ID: <ZlA_6Abtw-u4a84J@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
> Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
> populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
> from hvmloader.
> 
> While at this also remove ap_callin, as writing the APIC ID may serve the same
> purpose.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * New patch. Replaces adding cpu policy to hvmloader in v1.
> ---
>  tools/firmware/hvmloader/config.h    |  6 ++++-
>  tools/firmware/hvmloader/hvmloader.c |  4 +--
>  tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
>  tools/firmware/hvmloader/util.h      |  5 ++++
>  xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
>  5 files changed, 47 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
> index c82adf6dc508..edf6fa9c908c 100644
> --- a/tools/firmware/hvmloader/config.h
> +++ b/tools/firmware/hvmloader/config.h
> @@ -4,6 +4,8 @@
>  #include <stdint.h>
>  #include <stdbool.h>
>  
> +#include <xen/hvm/hvm_info_table.h>
> +
>  enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
>  extern enum virtual_vga virtual_vga;
>  
> @@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
>  
>  #define IOAPIC_ID           0x01
>  
> +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> +
>  #define LAPIC_BASE_ADDRESS  0xfee00000
> -#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
> +#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
>  
>  #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
>  #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
> diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
> index c58841e5b556..1eba92229925 100644
> --- a/tools/firmware/hvmloader/hvmloader.c
> +++ b/tools/firmware/hvmloader/hvmloader.c
> @@ -342,11 +342,11 @@ int main(void)
>  
>      printf("CPU speed is %u MHz\n", get_cpu_mhz());
>  
> +    smp_initialise();
> +
>      apic_setup();
>      pci_setup();
>  
> -    smp_initialise();
> -
>      perform_tests();
>  
>      if ( bios->bios_info_setup )
> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
> index a668f15d7e1f..4d75f239c2f5 100644
> --- a/tools/firmware/hvmloader/smp.c
> +++ b/tools/firmware/hvmloader/smp.c
> @@ -29,7 +29,34 @@
>  
>  #include <xen/vcpu.h>
>  
> -static int ap_callin, ap_cpuid;
> +static int ap_cpuid;
> +
> +/**
> + * Lookup table of x2APIC IDs.
> + *
> + * Each entry is populated its respective CPU as they come online. This is required
> + * for generating the MADT with minimal assumptions about ID relationships.
> + */
> +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> +
> +static uint32_t read_apic_id(void)
> +{
> +    uint32_t apic_id;
> +
> +    cpuid(1, NULL, &apic_id, NULL, NULL);
> +    apic_id >>= 24;
> +
> +    /*
> +     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
> +     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
> +     * but only if the x2APIC feature is present. If there are that many CPUs
> +     * it's guaranteed to be there so we can avoid checking for it specifically
> +     */
> +    if ( apic_id == 255 )
> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
> +
> +    return apic_id;
> +}
>  
>  static void ap_start(void)
>  {
> @@ -37,12 +64,12 @@ static void ap_start(void)
>      cacheattr_init();
>      printf("done.\n");
>  
> +    wmb();
> +    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();

Further thinking about this: do we really need the wmb(), given the
usage of ACCESS_ONCE()?

wmb() is a compiler barrier, and the usage of volatile in
ACCESS_ONCE() should already prevent any compiler re-ordering.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 07:22:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 07:22:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729110.1134242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPGI-0007zu-9f; Fri, 24 May 2024 07:22:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729110.1134242; Fri, 24 May 2024 07:22:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPGI-0007zn-5h; Fri, 24 May 2024 07:22:46 +0000
Received: by outflank-mailman (input) for mailman id 729110;
 Fri, 24 May 2024 07:22:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAPGH-0007zQ-7R
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 07:22:45 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 695a506b-199e-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 09:22:43 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-420180b5838so44040855e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 00:22:43 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a1c93c6sm889533f8f.85.2024.05.24.00.22.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 00:22:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 695a506b-199e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716535363; x=1717140163; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=hDYO1vx3pxdXiZRN27quhtULXuy3BunbmQgjoaLa634=;
        b=DyMLKLuhNS8pYKU7Kw60N0Qr909BtRIcouDTTuEcOoZOC1849kfkMo3RDMIPdynDE3
         WoSnNV26Z9lfSiOVMo898kmiP9FMoG2IGgh8AqNOjoTJVfKIwxVk5OTZQt7qQgrFzPO3
         G5fz/ZXrgS/nw7jWkuPU33gq/KjHFiJu3gl9E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716535363; x=1717140163;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hDYO1vx3pxdXiZRN27quhtULXuy3BunbmQgjoaLa634=;
        b=hy0PK5bguYckueD4ZD0NgnNfdG0hxkhTx9V89B310HKXEiVSS7JyRfoeHnlFzVLdXk
         j6lFxUntMIOIf8Ae5ZFeK+ymFB6EZmCsIXFlSQStnkchfMQaqd81PZWFUkLEoVoD4/Y9
         VkSGgbrhtyWlZ2e63Me4swtVVv+VMatt8MXpvT+CD/A5Om6dNR1Z6vtwZrpSuLWyRJ9U
         aF9HEjFPDfhcLwTNhFWv4jWidmMzpQqggmpzG8rjvBLRUyin9AQSHmYOCpAJKTsaVAlQ
         UD0kZcK6gHbFhW8lh3iNQUeAKzAHLFN32TNaOzcPue9/UJWgHBvZjUBFTcoYaxRcxplQ
         gWRw==
X-Forwarded-Encrypted: i=1; AJvYcCXODwXxwO+xkpu5kO2nFXdyYUD9g2iG6YdRXxEioyLoA3aXTSUkyewJ8y/d3/HFe1VL1a7f3yY/v+yWnVLyyRVfbrFoN+UM+zsJWPxvT30=
X-Gm-Message-State: AOJu0YwPF6yVeO4tAhNfcAejqLEOELvAaiWcFxpYyn4Ui+MDRe8tMqxs
	HuVL4TlW5jP8FC/sxS3enfpGkPjk+Kn0SFKxgWViSY0KMeBZ4ExKNlghwaNInPI=
X-Google-Smtp-Source: AGHT+IEZxBS18z8UiBenD2B5gQoP1aU6T1KWFMp4np+gywznQz84I0TpVTRLuHvpklGdp0wgDA/ODQ==
X-Received: by 2002:a5d:6342:0:b0:354:c329:90d1 with SMTP id ffacd0b85a97d-3552fdefa90mr1185753f8f.63.1716535363001;
        Fri, 24 May 2024 00:22:43 -0700 (PDT)
Date: Fri, 24 May 2024 09:22:41 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration
 checks to the check hook
Message-ID: <ZlBAQbgE7FvobVsu@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
 <630e0117-f401-4d55-a1de-2682d8d9b406@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <630e0117-f401-4d55-a1de-2682d8d9b406@citrix.com>

On Thu, May 23, 2024 at 07:58:57PM +0100, Andrew Cooper wrote:
> On 08/05/2024 1:39 pm, Alejandro Vallejo wrote:
> > diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> > index 8a244100009c..2f06bff1b2cc 100644
> > --- a/xen/arch/x86/hvm/vlapic.c
> > +++ b/xen/arch/x86/hvm/vlapic.c
> > @@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
> >                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
> >  }
> >  
> > -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> > +static int cf_check lapic_check_hidden(const struct domain *d,
> > +                                       hvm_domain_context_t *h)
> >  {
> >      unsigned int vcpuid = hvm_load_instance(h);
> > -    struct vcpu *v;
> > -    struct vlapic *s;
> > +    struct hvm_hw_lapic s;
> >  
> >      if ( !has_vlapic(d) )
> >          return -ENODEV;
> >  
> >      /* Which vlapic to load? */
> > -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
> > +    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )
> 
> As you're editing this anyway, swap for
> 
>     if ( !domain_vcpu(d, vcpuid) )
> 
> please.
> 
> >      {
> >          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
> >                  d->domain_id, vcpuid);
> >          return -EINVAL;
> >      }
> > -    s = vcpu_vlapic(v);
> >  
> > -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> > +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )
> > +        return -ENODATA;
> > +
> > +    /* EN=0 with EXTD=1 is illegal */
> > +    if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) ==
> > +         APIC_BASE_EXTD )
> > +        return -EINVAL;
> 
> This is very insufficient auditing for the incoming value, but it turns
> out that there's no nice logic for this at all.
> 
> As it's just a less obfuscated form of the logic from
> lapic_load_hidden(), it's probably fine to stay as it is for now.
> 
> The major changes since this logic was written originally are that the
> CPU policy correct (so we can reject EXTD on VMs which can't see
> x2apic), and that we now prohibit VMs moving the xAPIC MMIO window away
> from its default location (as this would require per-vCPU P2Ms in order
> to virtualise properly.)

Since this is just migration of the existing checks I think keeping
them as-is is best.  Adding new checks should be done in a followup
patch.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 07:25:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 07:25:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729116.1134251 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPIa-000071-Kz; Fri, 24 May 2024 07:25:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729116.1134251; Fri, 24 May 2024 07:25:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPIa-00006u-Hd; Fri, 24 May 2024 07:25:08 +0000
Received: by outflank-mailman (input) for mailman id 729116;
 Fri, 24 May 2024 07:25:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sAPIZ-00006o-AF
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 07:25:07 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id be5a7a0f-199e-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 09:25:06 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a621cb07d8fso372331266b.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 00:25:06 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda48e6sm83299966b.203.2024.05.24.00.25.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 00:25:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be5a7a0f-199e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716535506; x=1717140306; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=BCcRSzFZVY8kb90HA4XASBL0rwstT/aWeoSawvquxcg=;
        b=nd/3/EaBmE0B8SoWBJ4evHNR1ZwEJo+VZ1fCR74H2/fowUnnDca2L+rJL9pPg6G4us
         JnddakAfUoCPbqKLvzppxrgVnoYKnI74r46O18Zfa9yy7f7vzEZZttCI0jnEtF3aIBnD
         x9nsZ+uWjMmEPFLprk6GnNTcDXTuaXvSS+rguLQcx4SiDGFKoKvUkTVh3QN3hpFygGeN
         AdokYWVT/ZLRgJemONRtFBOoEBfax++bxwruVpGNmHXLZOchErBi03eigubUiDYDc9kY
         GFExXZsPPlRt9cYVF6qx62Iy3xd0TkN+fWGtWyKcy5V9Lz3qGL2h0/6vCtgYHMSJrqqw
         3vCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716535506; x=1717140306;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BCcRSzFZVY8kb90HA4XASBL0rwstT/aWeoSawvquxcg=;
        b=NA8lM9FistrNvI0gg8LJelfTJvlGlJFMezQicmiZNl876DaqzPoNMmwv46mYrQVBAN
         l+30N59/TcR8vYV+duqini0UsjZhW+Pqk3wMVk3LDkgIFSNtRYPyDe1ZRxqcZ+lBqy2w
         +LfqKP4J0GCrAtoxk4IYL89ejRgk4OWLO9fmsnsRaFpHrIaP5Cg6/BCOv2mAy98zSIKp
         acVijquGNIh+ySK83yEVrq85Vg8HA65VVL6Oz/M/MVRbF8WOAUIDTeyp10GGzsw0MqMo
         LjamrndqEj60VIy9kWtCoHyqdtFMBQ3UG3HtjvK0eWxRhhcHH9qGkr4976oHY0mIx6Zy
         XW8Q==
X-Forwarded-Encrypted: i=1; AJvYcCW/9Sep1TyaZjyrbl9Dp2RTChzOUoXgYf8j/Ia8rxdvuXG1hnJzT4Va4cFj6XBU1/All47ZwwBKbw+RIlPCFWIyGu4Qfx0M3mAG9rYwkhA=
X-Gm-Message-State: AOJu0Yxhx4KN7Jk6pMLkl1a0JZhmxnOKCJgWgu5ckvgtY+1EUmt5XYgd
	GRPjZbt1IeqIczUJ/XBqEbKSwziel3OQHAIlYhWL5++CpRADxZbS
X-Google-Smtp-Source: AGHT+IGEuLDZavc2AIrUowkQrU9JJVHgyGwRxyWxMjmJHQO5cPlBWFvUqg+GyeEdqhQRXSxjuzXzHQ==
X-Received: by 2002:a17:906:91d1:b0:a5a:8a63:9fee with SMTP id a640c23a62f3a-a626407b93amr102098166b.3.1716535505024;
        Fri, 24 May 2024 00:25:05 -0700 (PDT)
Message-ID: <c0c8f4665b3cdc80394577d59a6676fa3f41d414.camel@gmail.com>
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>,  xen-devel@lists.xenproject.org
Date: Fri, 24 May 2024 09:25:03 +0200
In-Reply-To: <ed0fec75-2b8c-4003-94ca-edbbb06e185f@suse.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
	 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
	 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
	 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
	 <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
	 <ed0fec75-2b8c-4003-94ca-edbbb06e185f@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Fri, 2024-05-24 at 08:48 +0200, Jan Beulich wrote:
> On 23.05.2024 18:40, Oleksii K. wrote:
> > On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
> > > On 23/05/2024 15:11, Oleksii K. wrote:
> > > > On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
> > > > > On 17/05/2024 14:54, Oleksii Kurochko wrote:
> > > > > > diff --git a/xen/arch/arm/arm64/livepatch.c
> > > > > > b/xen/arch/arm/arm64/livepatch.c
> > > > > > index df2cebedde..4bc8ed9be5 100644
> > > > > > --- a/xen/arch/arm/arm64/livepatch.c
> > > > > > +++ b/xen/arch/arm/arm64/livepatch.c
> > > > > > @@ -10,7 +10,6 @@
> > > > > > =C2=A0=C2=A0 #include <xen/mm.h>
> > > > > > =C2=A0=C2=A0 #include <xen/vmap.h>
> > > > > > =C2=A0=C2=A0=20
> > > > > > -#include <asm/bitops.h>
> > > > >=20
> > > > > It is a bit unclear how this change is related to the patch.
> > > > > Can
> > > > > you
> > > > > explain in the commit message?
> > > > Probably it doesn't need anymore. I will double check and if
> > > > this
> > > > change is not needed, I will just drop it in the next patch
> > > > version.
> > > >=20
> > > > >=20
> > > > > > =C2=A0=C2=A0 #include <asm/byteorder.h>
> > > > > > =C2=A0=C2=A0 #include <asm/insn.h>
> > > > > > =C2=A0=C2=A0 #include <asm/livepatch.h>
> > > > > > diff --git a/xen/arch/arm/include/asm/bitops.h
> > > > > > b/xen/arch/arm/include/asm/bitops.h
> > > > > > index 5104334e48..8e16335e76 100644
> > > > > > --- a/xen/arch/arm/include/asm/bitops.h
> > > > > > +++ b/xen/arch/arm/include/asm/bitops.h
> > > > > > @@ -22,9 +22,6 @@
> > > > > > =C2=A0=C2=A0 #define __set_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 set_bit(n,p)
> > > > > > =C2=A0=C2=A0 #define __clear_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 clear_bit(n,p)
> > > > > > =C2=A0=C2=A0=20
> > > > > > -#define BITOP_BITS_PER_WORD=C2=A0=C2=A0=C2=A0=C2=A0 32
> > > > > > -#define BITOP_MASK(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 (1UL << ((nr) %
> > > > > > BITOP_BITS_PER_WORD))
> > > > > > -#define BITOP_WORD(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 ((nr) /
> > > > > > BITOP_BITS_PER_WORD)
> > > > > > =C2=A0=C2=A0 #define BITS_PER_BYTE=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 8
> > > > >=20
> > > > > OOI, any reason BITS_PER_BYTE has not been moved as well? I
> > > > > don't
> > > > > expect
> > > > > the value to change across arch.
> > > > I can move it to generic one header too in the next patch
> > > > version.
> > > >=20
> > > > >=20
> > > > > [...]
> > > > >=20
> > > > > > diff --git a/xen/include/xen/bitops.h
> > > > > > b/xen/include/xen/bitops.h
> > > > > > index f14ad0d33a..6eeeff0117 100644
> > > > > > --- a/xen/include/xen/bitops.h
> > > > > > +++ b/xen/include/xen/bitops.h
> > > > > > @@ -65,10 +65,141 @@ static inline int
> > > > > > generic_flsl(unsigned
> > > > > > long
> > > > > > x)
> > > > > > =C2=A0=C2=A0=C2=A0 * scope
> > > > > > =C2=A0=C2=A0=C2=A0 */
> > > > > > =C2=A0=C2=A0=20
> > > > > > +#define BITOP_BITS_PER_WORD 32
> > > > > > +typedef uint32_t bitop_uint_t;
> > > > > > +
> > > > > > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > > > > > BITOP_BITS_PER_WORD))
> > > > > > +
> > > > > > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > > > > > +
> > > > > > +extern void __bitop_bad_size(void);
> > > > > > +
> > > > > > +#define bitop_bad_size(addr) (sizeof(*(addr)) <
> > > > > > sizeof(bitop_uint_t))
> > > > > > +
> > > > > > =C2=A0=C2=A0 /* --------------------- Please tidy above here --=
------
> > > > > > ----
> > > > > > -----
> > > > > > ---- */
> > > > > > =C2=A0=C2=A0=20
> > > > > > =C2=A0=C2=A0 #include <asm/bitops.h>
> > > > > > =C2=A0=C2=A0=20
> > > > > > +/**
> > > > > > + * generic__test_and_set_bit - Set a bit and return its
> > > > > > old
> > > > > > value
> > > > > > + * @nr: Bit to set
> > > > > > + * @addr: Address to count from
> > > > > > + *
> > > > > > + * This operation is non-atomic and can be reordered.
> > > > > > + * If two examples of this operation race, one can appear
> > > > > > to
> > > > > > succeed
> > > > > > + * but actually fail.=C2=A0 You must protect multiple accesses
> > > > > > with
> > > > > > a
> > > > > > lock.
> > > > > > + */
> > > > >=20
> > > > > Sorry for only mentioning this on v10. I think this comment
> > > > > should be
> > > > > duplicated (or moved to) on top of test_bit() because this is
> > > > > what
> > > > > everyone will use. This will avoid the developper to follow
> > > > > the
> > > > > function
> > > > > calls and only notice the x86 version which says "This
> > > > > function
> > > > > is
> > > > > atomic and may not be reordered." and would be wrong for all
> > > > > the
> > > > > other arch.
> > > > It makes sense to add this comment on top of test_bit(), but I
> > > > am
> > > > curious if it is needed to mention that for x86 arch_test_bit()
> > > > "is
> > > > atomic and may not be reordered":
> > >=20
> > > I would say no because any developper modifying common code can't
> > > relying it.
> > >=20
> > > >=20
> > > > =C2=A0 * This operation is non-atomic and can be reordered. (
> > > > Exception:
> > > > for
> > > > * x86 arch_test_bit() is atomic and may not be reordered )
> > > > =C2=A0 * If two examples of this operation race, one can appear to
> > > > succeed
> > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesse=
s with
> > > > a
> > > > lock.
> > > > =C2=A0 */
> > > >=20
> > > > >=20
> > > > > > +static always_inline bool
> > > > > > +generic__test_and_set_bit(int nr, volatile void *addr)
> > > > > > +{
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bito=
p_uint_t
> > > > > > *)addr +
> > > > > > BITOP_WORD(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > +
> > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > +}
> > > > > > +
> > > > > > +/**
> > > > > > + * generic__test_and_clear_bit - Clear a bit and return
> > > > > > its
> > > > > > old
> > > > > > value
> > > > > > + * @nr: Bit to clear
> > > > > > + * @addr: Address to count from
> > > > > > + *
> > > > > > + * This operation is non-atomic and can be reordered.
> > > > > > + * If two examples of this operation race, one can appear
> > > > > > to
> > > > > > succeed
> > > > > > + * but actually fail.=C2=A0 You must protect multiple accesses
> > > > > > with
> > > > > > a
> > > > > > lock.
> > > > > > + */
> > > > >=20
> > > > > Same applies here and ...
> > > > >=20
> > > > > > +static always_inline bool
> > > > > > +generic__test_and_clear_bit(int nr, volatile void *addr)
> > > > > > +{
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bito=
p_uint_t
> > > > > > *)addr +
> > > > > > BITOP_WORD(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > +
> > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > +}
> > > > > > +
> > > > > > +/* WARNING: non atomic and it can be reordered! */
> > > > >=20
> > > > > ... here.
> > > > >=20
> > > > > > +static always_inline bool
> > > > > > +generic__test_and_change_bit(int nr, volatile void *addr)
> > > > > > +{
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile bito=
p_uint_t
> > > > > > *)addr +
> > > > > > BITOP_WORD(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > +
> > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > +}
> > > > > > +/**
> > > > > > + * generic_test_bit - Determine whether a bit is set
> > > > > > + * @nr: bit number to test
> > > > > > + * @addr: Address to start counting from
> > > > > > + */
> > > > > > +static always_inline bool generic_test_bit(int nr, const
> > > > > > volatile
> > > > > > void *addr)
> > > > > > +{
> > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > +=C2=A0=C2=A0=C2=A0 const volatile bitop_uint_t *p =3D
> > > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 (const volatile bitop_uint_t
> > > > > > *)addr +
> > > > > > BITOP_WORD(nr);
> > > > > > +
> > > > > > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > > > > > +}
> > > > > > +
> > > > > > +static always_inline bool
> > > > > > +__test_and_set_bit(int nr, volatile void *addr)
> > > > > > +{
> > > > > > +#ifndef arch__test_and_set_bit
> > > > > > +#define arch__test_and_set_bit generic__test_and_set_bit
> > > > > > +#endif
> > > > > > +
> > > > > > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
> > > > > > +}
> > > > >=20
> > > > > NIT: It is a bit too late to change this one. But I have to
> > > > > admit, I
> > > > > don't understand the purpose of the static inline when you
> > > > > could
> > > > > have
> > > > > simply call...
> > > > >=20
> > > > > > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_siz=
e(); \
> > > > > > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > > > >=20
> > > > > ... __arch__test_and_set_bit here.
> > > > I just wanted to be in sync with other non-atomic test*_bit()
> > > > operations and Andrew's patch series ( xen/bitops: Reduce the
> > > > mess,
> > > > starting with ffs() ).
> > >=20
> > > I looked at Andrew series and I can't seem to find an example
> > > where
> > > he=20
> > > uses both the macro + static inline. He seems only use a static
> > > inline.
> > > Do you have any pointer?
> > >=20
> > > But by any chance are you referring to the pattern on x86? If so,
> > > I=20
> > > would really like to understand what's the benefits of
> > > introducing a
> > > a=20
> > > one-liner static inline which is "shadowed" by a macro...
> > Yes, I was referring to the x86 pattern.
> >=20
> > I tried to find the reason in the commit but couldn't:
> > https://lists.xenproject.org/archives/html/xen-changelog/2008-03/msg000=
97.html
> >=20
> > For some reason, I also couldn't find the mailing list thread for
> > this.
> >=20
> > Perhaps Jan could help here, based on the commit message he might
> > have
> > a suggestion.
>=20
> There's a lot of apparently unrelated context left, so I'm trying to
> guess
> on what the actual question is, from the old change you're pointing
> at: With
>=20
> #define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> =C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> =C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 \
>=20
> why do we have that wrapping macro? If that's indeed the question,
> then may
> I suggest you consider what would happen to the sizeof() in
> bitop_bad_size()
> if the invocation was moved to the inline function, taking pointer-
> to-void?
>=20
> However, I can't relate that old change to the question I think
> Julien
> raised (invoking __test_and_set_bit() vs arch__test_and_set_bit()),
> so
> perhaps I'm missing something.
If I understand the question correctly then the question is why do we
need static inline function. Can't we just do the following:

#ifndef arch_test_bit
#define arch_test_bit generic_test_bit
#endif

#define test_bit(nr, addr) ({                           \
    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
    arch_test_bit(nr, addr);                            \
})

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 24 07:35:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 07:35:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729127.1134260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPSq-000201-Mv; Fri, 24 May 2024 07:35:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729127.1134260; Fri, 24 May 2024 07:35:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPSq-0001zu-JW; Fri, 24 May 2024 07:35:44 +0000
Received: by outflank-mailman (input) for mailman id 729127;
 Fri, 24 May 2024 07:35:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=s7SU=M3=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sAPSo-0001zl-K8
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 07:35:42 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 38e337f9-19a0-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 09:35:41 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a6265d3ba8fso45908966b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 00:35:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc4fedesm83960166b.118.2024.05.24.00.35.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 00:35:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38e337f9-19a0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716536141; x=1717140941; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Myy0lmU1C4B0UctkiR5xddYSBODCil9IrI+xzRHLXQo=;
        b=TR746UKHMkJRRjmSNqDhwAYc0j3WrGH2ZTc1hox0QMsXxu8e/orZG1OKQ6+KnSgW8i
         d3mF7fU0y/azQqrJqY3Is13qYtGOd2pSPl2tmjyqfG8VHEJYZEcnhrU9vvgsZyja33TW
         qZ3qL7f17Lv6gAxdY6HrRQqNYZs1lpyQ46fPp2vS1k6LlSHyOfxhHDphfqt3qXiMH05+
         UT+KqcvNj21W23us9lqF8KqjzInlZvcfA+BeZi8ttVE3khzq4NDMycLsBry8HrrCexx7
         GISf+mfFl+LM5WuxC9DVRGf93XPwHfWP99Q3gkGhTDcDb/uS3uGAgkEMLIDKsDDvreIP
         avpw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716536141; x=1717140941;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Myy0lmU1C4B0UctkiR5xddYSBODCil9IrI+xzRHLXQo=;
        b=TktckaLcU/FamVhR51RaVBeymzRR+nuW9TEYXFEspm/MIWe7NjgeuY8GjjE3nEVsTw
         aOI9tJLxGgfxXsx0/3jCJcafr46eeUJgxHzWldSyce0JLk6Q7luYUkC10OXL+uyBPkj6
         76UdngQoGIpYtmEppi1RnbO5JlBlGtUIVHT0hCoTlISMklLnyakx/1oKv1ON3ixRdOTq
         YlI83ze8oF5x2wc/YjbZ8e70UJZcP+sR+OdKdGe0mTw0qQWzXwCAl8OSX2Sqv6RnJmnp
         QOGWU/oDvd5aKq6S0TpjHTaUfUyYoYbOs0R0p4sJmS3o3c0hLDCBEIswHK680i5IiD8c
         shcQ==
X-Forwarded-Encrypted: i=1; AJvYcCXTxn50vSBdB9AOtRhcP5cjglIX9kz0MZ+8LjdPNeD/keYSjwaxE1MvkQw2si9eEW3DD3FStCMNp/n3tYT+6gFQJ709fa27r2QwaP0dMZc=
X-Gm-Message-State: AOJu0YzkCDveyXy1gV85jRv6t3K7vKFePlvWAtuamo949JGYgWkBadw7
	NrrvVhk2ST1PPu2zGY8NGDUH2Z3fKuTqQc28r2+Rmtn5lFBX6aWdTwjXUAVpIA==
X-Google-Smtp-Source: AGHT+IGCQyiu8FBfqm49scxa2cbGbQjPVO9ATHbAX9royLmuuHC+39SHirZWHeaZjHPfr2PkHuracA==
X-Received: by 2002:a17:906:4f97:b0:a5a:6367:717e with SMTP id a640c23a62f3a-a62641dd8a2mr95042966b.17.1716536140585;
        Fri, 24 May 2024 00:35:40 -0700 (PDT)
Message-ID: <2d87e13f-7731-49ed-9628-b3d2b23ebdd4@suse.com>
Date: Fri, 24 May 2024 09:35:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org, Julien Grall <julien@xen.org>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
 <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
 <ed0fec75-2b8c-4003-94ca-edbbb06e185f@suse.com>
 <c0c8f4665b3cdc80394577d59a6676fa3f41d414.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c0c8f4665b3cdc80394577d59a6676fa3f41d414.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 24.05.2024 09:25, Oleksii K. wrote:
> On Fri, 2024-05-24 at 08:48 +0200, Jan Beulich wrote:
>> On 23.05.2024 18:40, Oleksii K. wrote:
>>> On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
>>>> On 23/05/2024 15:11, Oleksii K. wrote:
>>>>> On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
>>>>>> On 17/05/2024 14:54, Oleksii Kurochko wrote:
>>>>>>> diff --git a/xen/arch/arm/arm64/livepatch.c
>>>>>>> b/xen/arch/arm/arm64/livepatch.c
>>>>>>> index df2cebedde..4bc8ed9be5 100644
>>>>>>> --- a/xen/arch/arm/arm64/livepatch.c
>>>>>>> +++ b/xen/arch/arm/arm64/livepatch.c
>>>>>>> @@ -10,7 +10,6 @@
>>>>>>>    #include <xen/mm.h>
>>>>>>>    #include <xen/vmap.h>
>>>>>>>    
>>>>>>> -#include <asm/bitops.h>
>>>>>>
>>>>>> It is a bit unclear how this change is related to the patch.
>>>>>> Can
>>>>>> you
>>>>>> explain in the commit message?
>>>>> Probably it doesn't need anymore. I will double check and if
>>>>> this
>>>>> change is not needed, I will just drop it in the next patch
>>>>> version.
>>>>>
>>>>>>
>>>>>>>    #include <asm/byteorder.h>
>>>>>>>    #include <asm/insn.h>
>>>>>>>    #include <asm/livepatch.h>
>>>>>>> diff --git a/xen/arch/arm/include/asm/bitops.h
>>>>>>> b/xen/arch/arm/include/asm/bitops.h
>>>>>>> index 5104334e48..8e16335e76 100644
>>>>>>> --- a/xen/arch/arm/include/asm/bitops.h
>>>>>>> +++ b/xen/arch/arm/include/asm/bitops.h
>>>>>>> @@ -22,9 +22,6 @@
>>>>>>>    #define __set_bit(n,p)            set_bit(n,p)
>>>>>>>    #define __clear_bit(n,p)          clear_bit(n,p)
>>>>>>>    
>>>>>>> -#define BITOP_BITS_PER_WORD     32
>>>>>>> -#define BITOP_MASK(nr)          (1UL << ((nr) %
>>>>>>> BITOP_BITS_PER_WORD))
>>>>>>> -#define BITOP_WORD(nr)          ((nr) /
>>>>>>> BITOP_BITS_PER_WORD)
>>>>>>>    #define BITS_PER_BYTE           8
>>>>>>
>>>>>> OOI, any reason BITS_PER_BYTE has not been moved as well? I
>>>>>> don't
>>>>>> expect
>>>>>> the value to change across arch.
>>>>> I can move it to generic one header too in the next patch
>>>>> version.
>>>>>
>>>>>>
>>>>>> [...]
>>>>>>
>>>>>>> diff --git a/xen/include/xen/bitops.h
>>>>>>> b/xen/include/xen/bitops.h
>>>>>>> index f14ad0d33a..6eeeff0117 100644
>>>>>>> --- a/xen/include/xen/bitops.h
>>>>>>> +++ b/xen/include/xen/bitops.h
>>>>>>> @@ -65,10 +65,141 @@ static inline int
>>>>>>> generic_flsl(unsigned
>>>>>>> long
>>>>>>> x)
>>>>>>>     * scope
>>>>>>>     */
>>>>>>>    
>>>>>>> +#define BITOP_BITS_PER_WORD 32
>>>>>>> +typedef uint32_t bitop_uint_t;
>>>>>>> +
>>>>>>> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) %
>>>>>>> BITOP_BITS_PER_WORD))
>>>>>>> +
>>>>>>> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
>>>>>>> +
>>>>>>> +extern void __bitop_bad_size(void);
>>>>>>> +
>>>>>>> +#define bitop_bad_size(addr) (sizeof(*(addr)) <
>>>>>>> sizeof(bitop_uint_t))
>>>>>>> +
>>>>>>>    /* --------------------- Please tidy above here --------
>>>>>>> ----
>>>>>>> -----
>>>>>>> ---- */
>>>>>>>    
>>>>>>>    #include <asm/bitops.h>
>>>>>>>    
>>>>>>> +/**
>>>>>>> + * generic__test_and_set_bit - Set a bit and return its
>>>>>>> old
>>>>>>> value
>>>>>>> + * @nr: Bit to set
>>>>>>> + * @addr: Address to count from
>>>>>>> + *
>>>>>>> + * This operation is non-atomic and can be reordered.
>>>>>>> + * If two examples of this operation race, one can appear
>>>>>>> to
>>>>>>> succeed
>>>>>>> + * but actually fail.  You must protect multiple accesses
>>>>>>> with
>>>>>>> a
>>>>>>> lock.
>>>>>>> + */
>>>>>>
>>>>>> Sorry for only mentioning this on v10. I think this comment
>>>>>> should be
>>>>>> duplicated (or moved to) on top of test_bit() because this is
>>>>>> what
>>>>>> everyone will use. This will avoid the developper to follow
>>>>>> the
>>>>>> function
>>>>>> calls and only notice the x86 version which says "This
>>>>>> function
>>>>>> is
>>>>>> atomic and may not be reordered." and would be wrong for all
>>>>>> the
>>>>>> other arch.
>>>>> It makes sense to add this comment on top of test_bit(), but I
>>>>> am
>>>>> curious if it is needed to mention that for x86 arch_test_bit()
>>>>> "is
>>>>> atomic and may not be reordered":
>>>>
>>>> I would say no because any developper modifying common code can't
>>>> relying it.
>>>>
>>>>>
>>>>>   * This operation is non-atomic and can be reordered. (
>>>>> Exception:
>>>>> for
>>>>> * x86 arch_test_bit() is atomic and may not be reordered )
>>>>>   * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>>   * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>>   */
>>>>>
>>>>>>
>>>>>>> +static always_inline bool
>>>>>>> +generic__test_and_set_bit(int nr, volatile void *addr)
>>>>>>> +{
>>>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t
>>>>>>> *)addr +
>>>>>>> BITOP_WORD(nr);
>>>>>>> +    bitop_uint_t old = *p;
>>>>>>> +
>>>>>>> +    *p = old | mask;
>>>>>>> +    return (old & mask);
>>>>>>> +}
>>>>>>> +
>>>>>>> +/**
>>>>>>> + * generic__test_and_clear_bit - Clear a bit and return
>>>>>>> its
>>>>>>> old
>>>>>>> value
>>>>>>> + * @nr: Bit to clear
>>>>>>> + * @addr: Address to count from
>>>>>>> + *
>>>>>>> + * This operation is non-atomic and can be reordered.
>>>>>>> + * If two examples of this operation race, one can appear
>>>>>>> to
>>>>>>> succeed
>>>>>>> + * but actually fail.  You must protect multiple accesses
>>>>>>> with
>>>>>>> a
>>>>>>> lock.
>>>>>>> + */
>>>>>>
>>>>>> Same applies here and ...
>>>>>>
>>>>>>> +static always_inline bool
>>>>>>> +generic__test_and_clear_bit(int nr, volatile void *addr)
>>>>>>> +{
>>>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t
>>>>>>> *)addr +
>>>>>>> BITOP_WORD(nr);
>>>>>>> +    bitop_uint_t old = *p;
>>>>>>> +
>>>>>>> +    *p = old & ~mask;
>>>>>>> +    return (old & mask);
>>>>>>> +}
>>>>>>> +
>>>>>>> +/* WARNING: non atomic and it can be reordered! */
>>>>>>
>>>>>> ... here.
>>>>>>
>>>>>>> +static always_inline bool
>>>>>>> +generic__test_and_change_bit(int nr, volatile void *addr)
>>>>>>> +{
>>>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>>>> +    volatile bitop_uint_t *p = (volatile bitop_uint_t
>>>>>>> *)addr +
>>>>>>> BITOP_WORD(nr);
>>>>>>> +    bitop_uint_t old = *p;
>>>>>>> +
>>>>>>> +    *p = old ^ mask;
>>>>>>> +    return (old & mask);
>>>>>>> +}
>>>>>>> +/**
>>>>>>> + * generic_test_bit - Determine whether a bit is set
>>>>>>> + * @nr: bit number to test
>>>>>>> + * @addr: Address to start counting from
>>>>>>> + */
>>>>>>> +static always_inline bool generic_test_bit(int nr, const
>>>>>>> volatile
>>>>>>> void *addr)
>>>>>>> +{
>>>>>>> +    bitop_uint_t mask = BITOP_MASK(nr);
>>>>>>> +    const volatile bitop_uint_t *p =
>>>>>>> +                        (const volatile bitop_uint_t
>>>>>>> *)addr +
>>>>>>> BITOP_WORD(nr);
>>>>>>> +
>>>>>>> +    return (*p & mask);
>>>>>>> +}
>>>>>>> +
>>>>>>> +static always_inline bool
>>>>>>> +__test_and_set_bit(int nr, volatile void *addr)
>>>>>>> +{
>>>>>>> +#ifndef arch__test_and_set_bit
>>>>>>> +#define arch__test_and_set_bit generic__test_and_set_bit
>>>>>>> +#endif
>>>>>>> +
>>>>>>> +    return arch__test_and_set_bit(nr, addr);
>>>>>>> +}
>>>>>>
>>>>>> NIT: It is a bit too late to change this one. But I have to
>>>>>> admit, I
>>>>>> don't understand the purpose of the static inline when you
>>>>>> could
>>>>>> have
>>>>>> simply call...
>>>>>>
>>>>>>> +#define __test_and_set_bit(nr, addr) ({             \
>>>>>>> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
>>>>>>> +    __test_and_set_bit(nr, addr);                   \
>>>>>>
>>>>>> ... __arch__test_and_set_bit here.
>>>>> I just wanted to be in sync with other non-atomic test*_bit()
>>>>> operations and Andrew's patch series ( xen/bitops: Reduce the
>>>>> mess,
>>>>> starting with ffs() ).
>>>>
>>>> I looked at Andrew series and I can't seem to find an example
>>>> where
>>>> he 
>>>> uses both the macro + static inline. He seems only use a static
>>>> inline.
>>>> Do you have any pointer?
>>>>
>>>> But by any chance are you referring to the pattern on x86? If so,
>>>> I 
>>>> would really like to understand what's the benefits of
>>>> introducing a
>>>> a 
>>>> one-liner static inline which is "shadowed" by a macro...
>>> Yes, I was referring to the x86 pattern.
>>>
>>> I tried to find the reason in the commit but couldn't:
>>> https://lists.xenproject.org/archives/html/xen-changelog/2008-03/msg00097.html
>>>
>>> For some reason, I also couldn't find the mailing list thread for
>>> this.
>>>
>>> Perhaps Jan could help here, based on the commit message he might
>>> have
>>> a suggestion.
>>
>> There's a lot of apparently unrelated context left, so I'm trying to
>> guess
>> on what the actual question is, from the old change you're pointing
>> at: With
>>
>> #define __test_and_set_bit(nr, addr) ({             \
>>    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
>>    __test_and_set_bit(nr, addr);                   \
>>
>> why do we have that wrapping macro? If that's indeed the question,
>> then may
>> I suggest you consider what would happen to the sizeof() in
>> bitop_bad_size()
>> if the invocation was moved to the inline function, taking pointer-
>> to-void?
>>
>> However, I can't relate that old change to the question I think
>> Julien
>> raised (invoking __test_and_set_bit() vs arch__test_and_set_bit()),
>> so
>> perhaps I'm missing something.
> If I understand the question correctly then the question is why do we
> need static inline function. Can't we just do the following:
> 
> #ifndef arch_test_bit
> #define arch_test_bit generic_test_bit
> #endif
> 
> #define test_bit(nr, addr) ({                           \
>     if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
>     arch_test_bit(nr, addr);                            \
> })

I think we can. But then how did that old commit come into play?

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 24 08:06:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 08:06:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729159.1134270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPwS-0006zq-6T; Fri, 24 May 2024 08:06:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729159.1134270; Fri, 24 May 2024 08:06:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAPwS-0006zj-3p; Fri, 24 May 2024 08:06:20 +0000
Received: by outflank-mailman (input) for mailman id 729159;
 Fri, 24 May 2024 08:06:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sAPwR-0006zd-Ew
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 08:06:19 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7fb3fdc9-19a4-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 10:06:18 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-529646df247so474447e87.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 01:06:17 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52970c35d35sm122440e87.201.2024.05.24.01.06.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 01:06:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fb3fdc9-19a4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716537977; x=1717142777; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=gc46/0Bs/DeTMnCrfIbH5Ji2KJ7kjqNYrVz/R4sgfDk=;
        b=lnoKQaamHcdI1YVEa4CvFGPTQ+MnR/wCbutND2kDs0UGDtYHqY9LPdLHI2ZRTNjaX8
         rINZkN8Rt+1k8/mymmT3UrAHO4s2D3W6BLk/aDFEeUCJHnD0LDVsXduy07Q+9TDGG3P5
         j8ir9jy7E6UQOZ6k8sELdZO7Ugt971L37NMPT/9LzYoLTA4y5YJ6hKXqg/YbdQpoSekW
         /af9jhEq4zKusnl6hAee0PObwm2g4oDC8CP+cEXOOBDlD4hbh2bfIP/p+0Ypaw9uRuae
         QDSMQ03JDbIs8ExBOvnywQ5mcVnGTtmLcvb6I0Q4UjWHr42Dpq0O6ksbKHRQtD59LiYb
         mPEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716537977; x=1717142777;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gc46/0Bs/DeTMnCrfIbH5Ji2KJ7kjqNYrVz/R4sgfDk=;
        b=nzzCvJ/P5LTN0BlStkYazHfBGFG3d0m2GiXqhHmzt8acxgV99f4W7+FG1s3V393RvH
         ipLs8DgN+FLdw7InRYYj5HYqR6zLc2Xgod9FUJ+Yv9GiZsQ5h5NEZ8nscaRAG+ChGO+E
         9WsmVI3PHEZgal8sN+WJEd0YRS60WZsJCo++9Ke9dBLEY6jeHST6UeB75T0IimacHH7N
         IER3pYtGDHiQeILypDgPL+OdlPIdB9Mo1wjbXF0bGA0Lt9eIYAhmiH9Ii6SLB+1QYuhw
         n/U4StWzzg7QZt6XlCKamI4SRS0toGXUuBVYcUZkHnFwVM4q7OVS3skF9mWM8t/XWUmi
         IDvA==
X-Forwarded-Encrypted: i=1; AJvYcCWbU7xLmmP7W357dS1+4y2oHJQA3E0CHQ/2s8JQY0JSQVkdrqAHYAr2MQpnnXBeOBeKSUeE0wnRGJIQo5KtGGRRzkQCQ7deDBHsnThxB/k=
X-Gm-Message-State: AOJu0YxDk9IzcnBtjI5FtpcsZGyeyvdp0Hmh6WBFKCUrodMUkxkFVrzp
	vacBKrI9TC1FGv7j6ft99nx8iX4HVYF+bwpFZ4y0HqfsP20GdTOx
X-Google-Smtp-Source: AGHT+IFIK6JglEIYWeON8ju/EUU7VQpPhAb7zmgKHL94c7SqnB5JQT5yOsRhGXmuLB4XL7vWNaRxMg==
X-Received: by 2002:a05:6512:239f:b0:516:c763:b4f5 with SMTP id 2adb3069b0e04-52964109a9cmr1146567e87.3.1716537977101;
        Fri, 24 May 2024 01:06:17 -0700 (PDT)
Message-ID: <120edca82851167832573f3b4876040dedb43b1c.camel@gmail.com>
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>,  xen-devel@lists.xenproject.org
Date: Fri, 24 May 2024 10:06:15 +0200
In-Reply-To: <2d87e13f-7731-49ed-9628-b3d2b23ebdd4@suse.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
	 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
	 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
	 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
	 <ea2c01d19160431ed9f6e4e7b6c122ab27aa676b.camel@gmail.com>
	 <ed0fec75-2b8c-4003-94ca-edbbb06e185f@suse.com>
	 <c0c8f4665b3cdc80394577d59a6676fa3f41d414.camel@gmail.com>
	 <2d87e13f-7731-49ed-9628-b3d2b23ebdd4@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Fri, 2024-05-24 at 09:35 +0200, Jan Beulich wrote:
> On 24.05.2024 09:25, Oleksii K. wrote:
> > On Fri, 2024-05-24 at 08:48 +0200, Jan Beulich wrote:
> > > On 23.05.2024 18:40, Oleksii K. wrote:
> > > > On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
> > > > > On 23/05/2024 15:11, Oleksii K. wrote:
> > > > > > On Thu, 2024-05-23 at 14:00 +0100, Julien Grall wrote:
> > > > > > > On 17/05/2024 14:54, Oleksii Kurochko wrote:
> > > > > > > > diff --git a/xen/arch/arm/arm64/livepatch.c
> > > > > > > > b/xen/arch/arm/arm64/livepatch.c
> > > > > > > > index df2cebedde..4bc8ed9be5 100644
> > > > > > > > --- a/xen/arch/arm/arm64/livepatch.c
> > > > > > > > +++ b/xen/arch/arm/arm64/livepatch.c
> > > > > > > > @@ -10,7 +10,6 @@
> > > > > > > > =C2=A0=C2=A0 #include <xen/mm.h>
> > > > > > > > =C2=A0=C2=A0 #include <xen/vmap.h>
> > > > > > > > =C2=A0=C2=A0=20
> > > > > > > > -#include <asm/bitops.h>
> > > > > > >=20
> > > > > > > It is a bit unclear how this change is related to the
> > > > > > > patch.
> > > > > > > Can
> > > > > > > you
> > > > > > > explain in the commit message?
> > > > > > Probably it doesn't need anymore. I will double check and
> > > > > > if
> > > > > > this
> > > > > > change is not needed, I will just drop it in the next patch
> > > > > > version.
> > > > > >=20
> > > > > > >=20
> > > > > > > > =C2=A0=C2=A0 #include <asm/byteorder.h>
> > > > > > > > =C2=A0=C2=A0 #include <asm/insn.h>
> > > > > > > > =C2=A0=C2=A0 #include <asm/livepatch.h>
> > > > > > > > diff --git a/xen/arch/arm/include/asm/bitops.h
> > > > > > > > b/xen/arch/arm/include/asm/bitops.h
> > > > > > > > index 5104334e48..8e16335e76 100644
> > > > > > > > --- a/xen/arch/arm/include/asm/bitops.h
> > > > > > > > +++ b/xen/arch/arm/include/asm/bitops.h
> > > > > > > > @@ -22,9 +22,6 @@
> > > > > > > > =C2=A0=C2=A0 #define __set_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 set_bit(n,p)
> > > > > > > > =C2=A0=C2=A0 #define __clear_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 clear_bit(n,p)
> > > > > > > > =C2=A0=C2=A0=20
> > > > > > > > -#define BITOP_BITS_PER_WORD=C2=A0=C2=A0=C2=A0=C2=A0 32
> > > > > > > > -#define BITOP_MASK(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 (1UL << ((nr) %
> > > > > > > > BITOP_BITS_PER_WORD))
> > > > > > > > -#define BITOP_WORD(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 ((nr) /
> > > > > > > > BITOP_BITS_PER_WORD)
> > > > > > > > =C2=A0=C2=A0 #define BITS_PER_BYTE=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 8
> > > > > > >=20
> > > > > > > OOI, any reason BITS_PER_BYTE has not been moved as well?
> > > > > > > I
> > > > > > > don't
> > > > > > > expect
> > > > > > > the value to change across arch.
> > > > > > I can move it to generic one header too in the next patch
> > > > > > version.
> > > > > >=20
> > > > > > >=20
> > > > > > > [...]
> > > > > > >=20
> > > > > > > > diff --git a/xen/include/xen/bitops.h
> > > > > > > > b/xen/include/xen/bitops.h
> > > > > > > > index f14ad0d33a..6eeeff0117 100644
> > > > > > > > --- a/xen/include/xen/bitops.h
> > > > > > > > +++ b/xen/include/xen/bitops.h
> > > > > > > > @@ -65,10 +65,141 @@ static inline int
> > > > > > > > generic_flsl(unsigned
> > > > > > > > long
> > > > > > > > x)
> > > > > > > > =C2=A0=C2=A0=C2=A0 * scope
> > > > > > > > =C2=A0=C2=A0=C2=A0 */
> > > > > > > > =C2=A0=C2=A0=20
> > > > > > > > +#define BITOP_BITS_PER_WORD 32
> > > > > > > > +typedef uint32_t bitop_uint_t;
> > > > > > > > +
> > > > > > > > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > > > > > > > BITOP_BITS_PER_WORD))
> > > > > > > > +
> > > > > > > > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > > > > > > > +
> > > > > > > > +extern void __bitop_bad_size(void);
> > > > > > > > +
> > > > > > > > +#define bitop_bad_size(addr) (sizeof(*(addr)) <
> > > > > > > > sizeof(bitop_uint_t))
> > > > > > > > +
> > > > > > > > =C2=A0=C2=A0 /* --------------------- Please tidy above her=
e ----
> > > > > > > > ----
> > > > > > > > ----
> > > > > > > > -----
> > > > > > > > ---- */
> > > > > > > > =C2=A0=C2=A0=20
> > > > > > > > =C2=A0=C2=A0 #include <asm/bitops.h>
> > > > > > > > =C2=A0=C2=A0=20
> > > > > > > > +/**
> > > > > > > > + * generic__test_and_set_bit - Set a bit and return
> > > > > > > > its
> > > > > > > > old
> > > > > > > > value
> > > > > > > > + * @nr: Bit to set
> > > > > > > > + * @addr: Address to count from
> > > > > > > > + *
> > > > > > > > + * This operation is non-atomic and can be reordered.
> > > > > > > > + * If two examples of this operation race, one can
> > > > > > > > appear
> > > > > > > > to
> > > > > > > > succeed
> > > > > > > > + * but actually fail.=C2=A0 You must protect multiple
> > > > > > > > accesses
> > > > > > > > with
> > > > > > > > a
> > > > > > > > lock.
> > > > > > > > + */
> > > > > > >=20
> > > > > > > Sorry for only mentioning this on v10. I think this
> > > > > > > comment
> > > > > > > should be
> > > > > > > duplicated (or moved to) on top of test_bit() because
> > > > > > > this is
> > > > > > > what
> > > > > > > everyone will use. This will avoid the developper to
> > > > > > > follow
> > > > > > > the
> > > > > > > function
> > > > > > > calls and only notice the x86 version which says "This
> > > > > > > function
> > > > > > > is
> > > > > > > atomic and may not be reordered." and would be wrong for
> > > > > > > all
> > > > > > > the
> > > > > > > other arch.
> > > > > > It makes sense to add this comment on top of test_bit(),
> > > > > > but I
> > > > > > am
> > > > > > curious if it is needed to mention that for x86
> > > > > > arch_test_bit()
> > > > > > "is
> > > > > > atomic and may not be reordered":
> > > > >=20
> > > > > I would say no because any developper modifying common code
> > > > > can't
> > > > > relying it.
> > > > >=20
> > > > > >=20
> > > > > > =C2=A0 * This operation is non-atomic and can be reordered. (
> > > > > > Exception:
> > > > > > for
> > > > > > * x86 arch_test_bit() is atomic and may not be reordered )
> > > > > > =C2=A0 * If two examples of this operation race, one can appear
> > > > > > to
> > > > > > succeed
> > > > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple acc=
esses
> > > > > > with
> > > > > > a
> > > > > > lock.
> > > > > > =C2=A0 */
> > > > > >=20
> > > > > > >=20
> > > > > > > > +static always_inline bool
> > > > > > > > +generic__test_and_set_bit(int nr, volatile void *addr)
> > > > > > > > +{
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile =
bitop_uint_t
> > > > > > > > *)addr +
> > > > > > > > BITOP_WORD(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > > > +
> > > > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old | mask;
> > > > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +/**
> > > > > > > > + * generic__test_and_clear_bit - Clear a bit and
> > > > > > > > return
> > > > > > > > its
> > > > > > > > old
> > > > > > > > value
> > > > > > > > + * @nr: Bit to clear
> > > > > > > > + * @addr: Address to count from
> > > > > > > > + *
> > > > > > > > + * This operation is non-atomic and can be reordered.
> > > > > > > > + * If two examples of this operation race, one can
> > > > > > > > appear
> > > > > > > > to
> > > > > > > > succeed
> > > > > > > > + * but actually fail.=C2=A0 You must protect multiple
> > > > > > > > accesses
> > > > > > > > with
> > > > > > > > a
> > > > > > > > lock.
> > > > > > > > + */
> > > > > > >=20
> > > > > > > Same applies here and ...
> > > > > > >=20
> > > > > > > > +static always_inline bool
> > > > > > > > +generic__test_and_clear_bit(int nr, volatile void
> > > > > > > > *addr)
> > > > > > > > +{
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile =
bitop_uint_t
> > > > > > > > *)addr +
> > > > > > > > BITOP_WORD(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > > > +
> > > > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old & ~mask;
> > > > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +/* WARNING: non atomic and it can be reordered! */
> > > > > > >=20
> > > > > > > ... here.
> > > > > > >=20
> > > > > > > > +static always_inline bool
> > > > > > > > +generic__test_and_change_bit(int nr, volatile void
> > > > > > > > *addr)
> > > > > > > > +{
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 volatile bitop_uint_t *p =3D (volatile =
bitop_uint_t
> > > > > > > > *)addr +
> > > > > > > > BITOP_WORD(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t old =3D *p;
> > > > > > > > +
> > > > > > > > +=C2=A0=C2=A0=C2=A0 *p =3D old ^ mask;
> > > > > > > > +=C2=A0=C2=A0=C2=A0 return (old & mask);
> > > > > > > > +}
> > > > > > > > +/**
> > > > > > > > + * generic_test_bit - Determine whether a bit is set
> > > > > > > > + * @nr: bit number to test
> > > > > > > > + * @addr: Address to start counting from
> > > > > > > > + */
> > > > > > > > +static always_inline bool generic_test_bit(int nr,
> > > > > > > > const
> > > > > > > > volatile
> > > > > > > > void *addr)
> > > > > > > > +{
> > > > > > > > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > > > > > > > +=C2=A0=C2=A0=C2=A0 const volatile bitop_uint_t *p =3D
> > > > > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 (const volatile bitop_uint_t
> > > > > > > > *)addr +
> > > > > > > > BITOP_WORD(nr);
> > > > > > > > +
> > > > > > > > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static always_inline bool
> > > > > > > > +__test_and_set_bit(int nr, volatile void *addr)
> > > > > > > > +{
> > > > > > > > +#ifndef arch__test_and_set_bit
> > > > > > > > +#define arch__test_and_set_bit
> > > > > > > > generic__test_and_set_bit
> > > > > > > > +#endif
> > > > > > > > +
> > > > > > > > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr)=
;
> > > > > > > > +}
> > > > > > >=20
> > > > > > > NIT: It is a bit too late to change this one. But I have
> > > > > > > to
> > > > > > > admit, I
> > > > > > > don't understand the purpose of the static inline when
> > > > > > > you
> > > > > > > could
> > > > > > > have
> > > > > > > simply call...
> > > > > > >=20
> > > > > > > > +#define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > > > +=C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad=
_size(); \
> > > > > > > > +=C2=A0=C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > > > >=20
> > > > > > > ... __arch__test_and_set_bit here.
> > > > > > I just wanted to be in sync with other non-atomic
> > > > > > test*_bit()
> > > > > > operations and Andrew's patch series ( xen/bitops: Reduce
> > > > > > the
> > > > > > mess,
> > > > > > starting with ffs() ).
> > > > >=20
> > > > > I looked at Andrew series and I can't seem to find an example
> > > > > where
> > > > > he=20
> > > > > uses both the macro + static inline. He seems only use a
> > > > > static
> > > > > inline.
> > > > > Do you have any pointer?
> > > > >=20
> > > > > But by any chance are you referring to the pattern on x86? If
> > > > > so,
> > > > > I=20
> > > > > would really like to understand what's the benefits of
> > > > > introducing a
> > > > > a=20
> > > > > one-liner static inline which is "shadowed" by a macro...
> > > > Yes, I was referring to the x86 pattern.
> > > >=20
> > > > I tried to find the reason in the commit but couldn't:
> > > > https://lists.xenproject.org/archives/html/xen-changelog/2008-03/ms=
g00097.html
> > > >=20
> > > > For some reason, I also couldn't find the mailing list thread
> > > > for
> > > > this.
> > > >=20
> > > > Perhaps Jan could help here, based on the commit message he
> > > > might
> > > > have
> > > > a suggestion.
> > >=20
> > > There's a lot of apparently unrelated context left, so I'm trying
> > > to
> > > guess
> > > on what the actual question is, from the old change you're
> > > pointing
> > > at: With
> > >=20
> > > #define __test_and_set_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > =C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> > > =C2=A0=C2=A0 __test_and_set_bit(nr, addr);=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 \
> > >=20
> > > why do we have that wrapping macro? If that's indeed the
> > > question,
> > > then may
> > > I suggest you consider what would happen to the sizeof() in
> > > bitop_bad_size()
> > > if the invocation was moved to the inline function, taking
> > > pointer-
> > > to-void?
> > >=20
> > > However, I can't relate that old change to the question I think
> > > Julien
> > > raised (invoking __test_and_set_bit() vs
> > > arch__test_and_set_bit()),
> > > so
> > > perhaps I'm missing something.
> > If I understand the question correctly then the question is why do
> > we
> > need static inline function. Can't we just do the following:
> >=20
> > #ifndef arch_test_bit
> > #define arch_test_bit generic_test_bit
> > #endif
> >=20
> > #define test_bit(nr, addr) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > =C2=A0=C2=A0=C2=A0 if ( bitop_bad_size(addr) ) __bitop_bad_size();=C2=
=A0=C2=A0=C2=A0=C2=A0 \
> > =C2=A0=C2=A0=C2=A0 arch_test_bit(nr, addr);=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > })
>=20
> I think we can. But then how did that old commit come into play?
In the old commit, this approach was introduced where we have a 'static
inline function' called within a macro, but I only now realized that it
was done solely because of the check "if (bitop_bad_size(addr))
__bitop_bad_size();".

As far as I understand, Julien doesn't insist on dropping the 'static
inline function' and using arch_test_bit() directly in the test_bit()
macro, so I prefer to use the option implemented in this patch.
However, if anyone has objections to this approach and thinks it would
be better to drop the 'static inline functions' and call arch_*()
directly in a macro, I am okay with reworking it today and sending a
new patch version where this changes will be taking into account.

Anyway, I will send a new version of this patch today as Julien asked
me to add comments on top of the functions generic_test_bit() and
generic__test_and_change_bit(). Additionally, "#define BITS_PER_BYTE 8"
will be moved to xen/bitops.h as it seems to be always the same for
every architecture.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri May 24 08:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 08:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729177.1134281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQSZ-0003C8-QE; Fri, 24 May 2024 08:39:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729177.1134281; Fri, 24 May 2024 08:39:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQSZ-0003C1-Nh; Fri, 24 May 2024 08:39:31 +0000
Received: by outflank-mailman (input) for mailman id 729177;
 Fri, 24 May 2024 08:39:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAQSY-0003Ac-ME
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 08:39:30 +0000
Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com
 [2607:f8b0:4864:20::f2c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2231b63e-19a9-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 10:39:29 +0200 (CEST)
Received: by mail-qv1-xf2c.google.com with SMTP id
 6a1803df08f44-6ab20178a78so24057536d6.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 01:39:29 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac162fe1b7sm5182616d6.104.2024.05.24.01.39.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 01:39:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2231b63e-19a9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716539968; x=1717144768; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=eWZCUoWZhFd+uL07Tq+rkzT187dValeJDIWvpb88kRM=;
        b=aesUN46O7uCeIv/Hybieer1YRBSaWJdOXmMzOXJKQoIJZcFrPxFEHn65njbjwVZ6KH
         xRIj8IyZDcUTCfdwz5aB1+36Qny/jCcL25acAMbZv4xsro5uEAPRwOGNQ2WtxeY6pwWy
         lDgA2VwnOl2GWR2UbLLNZVLMVtHybkz8aQUuc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716539968; x=1717144768;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eWZCUoWZhFd+uL07Tq+rkzT187dValeJDIWvpb88kRM=;
        b=pkuSc+Y4+1aUS5j/2H0sUVU9sgBPXQz5Kqm6bapOrVdDQv/EbyGD1+sWQdGUTatl+T
         lO7YysF7iKe3mcYI4j1OICfMnlnTtExavm/53iriRC5tTii2Jgkkq12WyudUK/jv7nJu
         MS0dHCBiZ4MGyV74coeEaKGmz6VtRt2Odl3WLf2xLVq9uz8rt1kVYX+NVEXH4yNByySz
         uWtdrMhQIl1DGMehRtcHWEBNWbWl0igBrufTjsZkFWE7+6HifGMmKYUKQjgZfoPUXcma
         tIc/kJPrHakl0Bk5GJAepbqwAuXS3qZsIsc/ZizllOZObYI27yAHlmpfjFnDqhYGY4eh
         CjTg==
X-Gm-Message-State: AOJu0YzZfvyY6lIojvu9STZeq2VViESBRDCf5Dn1cI09o2zMNNTB7Jnh
	NGQ2t0uxIBIxrpQPGk/9It3jRITJDQTDZP51QxrUZe8ItX8FitOhnUGNEJll35A=
X-Google-Smtp-Source: AGHT+IHgu684rlCc2LndIkxcDq8n8IjIqsHj9vkhT6WKe61kuPiVfLcRsQqh3sR/yrdjWcA3MB2AwQ==
X-Received: by 2002:a0c:f106:0:b0:6ad:5d19:d9d5 with SMTP id 6a1803df08f44-6ad5d19dd5bmr4453706d6.38.1716539968231;
        Fri, 24 May 2024 01:39:28 -0700 (PDT)
Date: Fri, 24 May 2024 10:39:25 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 7/8] xen/x86: Derive topologically correct x2APIC IDs
 from the policy
Message-ID: <ZlBSPYCpEAosNVzo@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <87a2a4589e330472b7260ff6ab513744596a4488.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <87a2a4589e330472b7260ff6ab513744596a4488.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:26PM +0100, Alejandro Vallejo wrote:
> Implements the helper for mapping vcpu_id to x2apic_id given a valid
> topology in a policy. The algo is written with the intention of extending
> it to leaves 0x1f and e26 in the future.

Using 0x1f and e26 is kind of confusing.  I would word as "0x1f and
extended leaf 0x26" to avoid confusion.

> 
> Toolstack doesn't set leaf 0xb and the HVM default policy has it cleared,
> so the leaf is not implemented. In that case, the new helper just returns
> the legacy mapping.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * const-ify the test definitions
>   * Cosmetic changes (newline + parameter name in prototype)
> ---
>  tools/tests/cpu-policy/test-cpu-policy.c | 63 ++++++++++++++++++++
>  xen/include/xen/lib/x86/cpu-policy.h     |  2 +
>  xen/lib/x86/policy.c                     | 73 ++++++++++++++++++++++--
>  3 files changed, 133 insertions(+), 5 deletions(-)
> 
> diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
> index 0ba8c418b1b3..82a6aeb23317 100644
> --- a/tools/tests/cpu-policy/test-cpu-policy.c
> +++ b/tools/tests/cpu-policy/test-cpu-policy.c
> @@ -776,6 +776,68 @@ static void test_topo_from_parts(void)
>      }
>  }
>  
> +static void test_x2apic_id_from_vcpu_id_success(void)
> +{
> +    static const struct test {
> +        unsigned int vcpu_id;
> +        unsigned int threads_per_core;
> +        unsigned int cores_per_pkg;
> +        uint32_t x2apic_id;
> +        uint8_t x86_vendor;
> +    } tests[] = {
> +        {
> +            .vcpu_id = 3, .threads_per_core = 3, .cores_per_pkg = 8,
> +            .x2apic_id = 1 << 2,
> +        },
> +        {
> +            .vcpu_id = 6, .threads_per_core = 3, .cores_per_pkg = 8,
> +            .x2apic_id = 2 << 2,
> +        },
> +        {
> +            .vcpu_id = 24, .threads_per_core = 3, .cores_per_pkg = 8,
> +            .x2apic_id = 1 << 5,
> +        },
> +        {
> +            .vcpu_id = 35, .threads_per_core = 3, .cores_per_pkg = 8,
> +            .x2apic_id = (35 % 3) | (((35 / 3) % 8)  << 2) | ((35 / 24) << 5),
> +        },
> +        {
> +            .vcpu_id = 96, .threads_per_core = 7, .cores_per_pkg = 3,
> +            .x2apic_id = (96 % 7) | (((96 / 7) % 3)  << 3) | ((96 / 21) << 5),
                                                      ^ extra space (same above)

> +        },
> +    };
> +
> +    const uint8_t vendors[] = {
> +        X86_VENDOR_INTEL,
> +        X86_VENDOR_AMD,
> +        X86_VENDOR_CENTAUR,
> +        X86_VENDOR_SHANGHAI,
> +        X86_VENDOR_HYGON,
> +    };
> +
> +    printf("Testing x2apic id from vcpu id success:\n");
> +
> +    /* Perform the test run on every vendor we know about */
> +    for ( size_t i = 0; i < ARRAY_SIZE(vendors); ++i )
> +    {
> +        struct cpu_policy policy = { .x86_vendor = vendors[i] };

Newline.

> +        for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
> +        {
> +            const struct test *t = &tests[i];
> +            uint32_t x2apic_id;
> +            int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);

Overly long line.

Won't it be better to define `policy` in this scope, so that for each
test you start with a clean policy, rather than having leftover data
from the previous test?

Also you could initialize x2apic_id at definition:

const struct test *t = &tests[j];
struct cpu_policy policy = { .x86_vendor = vendors[i] };
int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);
uint32_t x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);

> +
> +            x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);
> +            if ( rc || x2apic_id != t->x2apic_id )
> +                fail("FAIL[%d] - '%s cpu%u %u t/c %u c/p'. bad x2apic_id: expected=%u actual=%u\n",
> +                     rc,
> +                     x86_cpuid_vendor_to_str(policy.x86_vendor),
> +                     t->vcpu_id, t->threads_per_core, t->cores_per_pkg,
> +                     t->x2apic_id, x2apic_id);
> +        }
> +    }
> +}
> +
>  int main(int argc, char **argv)
>  {
>      printf("CPU Policy unit tests\n");
> @@ -794,6 +856,7 @@ int main(int argc, char **argv)
>      test_is_compatible_failure();
>  
>      test_topo_from_parts();
> +    test_x2apic_id_from_vcpu_id_success();
>  
>      if ( nr_failures )
>          printf("Done: %u failures\n", nr_failures);
> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
> index f5df18e9f77c..2cbc2726a861 100644
> --- a/xen/include/xen/lib/x86/cpu-policy.h
> +++ b/xen/include/xen/lib/x86/cpu-policy.h
> @@ -545,6 +545,8 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>  /**
>   * Calculates the x2APIC ID of a vCPU given a CPU policy
>   *
> + * If the policy lacks leaf 0xb falls back to legacy mapping of apic_id=cpu*2
> + *
>   * @param p          CPU policy of the domain.
>   * @param id         vCPU ID of the vCPU.
>   * @returns x2APIC ID of the vCPU.
> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
> index d033ee5398dd..e498e32f8fd7 100644
> --- a/xen/lib/x86/policy.c
> +++ b/xen/lib/x86/policy.c
> @@ -2,15 +2,78 @@
>  
>  #include <xen/lib/x86/cpu-policy.h>
>  
> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
> +{
> +    /*
> +     * `nr_logical` reported by Intel is the number of THREADS contained in
> +     * the next topological scope. For example, assuming a system with 2
> +     * threads/core and 3 cores/module in a fully symmetric topology,
> +     * `nr_logical` at the core level will report 6. Because it's reporting
> +     * the number of threads in a module.
> +     *
> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
> +     * level (cores/complex, etc) so we can return it as-is.
> +     */
> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
> +        return p->topo.subleaf[lvl].nr_logical;
> +
> +    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;

Line length here and in the function declaration.

> +}
> +
>  uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>  {
> +    uint32_t shift = 0, x2apic_id = 0;
> +
> +    /* In the absence of topology leaves, fallback to traditional mapping */
> +    if ( !p->topo.subleaf[0].type )
> +        return id * 2;
> +
>      /*
> -     * TODO: Derive x2APIC ID from the topology information inside `p`
> -     *       rather than from vCPU ID. This bodge is a temporary measure
> -     *       until all infra is in place to retrieve or derive the initial
> -     *       x2APIC ID from migrated domains.

I'm a bit confused with this, the policy is domain wide, so we will
always need to pass the vCPU ID into x86_x2apic_id_from_vcpu_id()?
IOW: the x2APIC ID will always be derived from the vCPU ID.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 08:58:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 08:58:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729187.1134291 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQl1-000602-BG; Fri, 24 May 2024 08:58:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729187.1134291; Fri, 24 May 2024 08:58:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQl1-0005zv-8c; Fri, 24 May 2024 08:58:35 +0000
Received: by outflank-mailman (input) for mailman id 729187;
 Fri, 24 May 2024 08:58:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAQkz-0005zp-Tt
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 08:58:33 +0000
Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com
 [2607:f8b0:4864:20::f2c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cbb95f1b-19ab-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 10:58:32 +0200 (CEST)
Received: by mail-qv1-xf2c.google.com with SMTP id
 6a1803df08f44-6aa93d19a25so24001236d6.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 01:58:32 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac070f3597sm5496936d6.58.2024.05.24.01.58.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 01:58:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbb95f1b-19ab-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716541112; x=1717145912; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=bGddc/d20WfES02t8eky1QTcJaXCspHyZojH3X0VL2E=;
        b=i9LTyMzEf1Ygnw4udQVgWWTLXQIRvuin8n1K0h8hJ3NXa2ZaWCrxhJ5zJ3jMBA+fgv
         QJQeh4O4/mVgee9OFWNtpi+c8+zZhxhKZJn0SWQ68ooW+Y8CiOl39FHjleKMqrC9hFCC
         Ao0h6gUhz1i+YWzi7PwODc8NHbmKOPK+jOQU0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716541112; x=1717145912;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bGddc/d20WfES02t8eky1QTcJaXCspHyZojH3X0VL2E=;
        b=EhBhUdPc2ifJ3V68vYOebS1oJHQ7oWLcZrXsnYzc92cXLI+2uJAwBrg4tDfSD4kr+a
         Ja3PH6r+JysINJKiNYtXnjQvRx22+SG9ncUlRV3TtoAbtsn3R5bJrl3o5Gqaqhf3jJ94
         SQpX5MG1ex4478mjk89hxpj2XbKu2PYraJHm3VbJf4s395ykb/OTvjpUaJED25GZkycT
         X6ktnqMw73cobgzHFiuY2524b8jARS49pNeHJ7kNXQieZhwic+75o/FuHZeTRAuVSi44
         NWsE6gzzFuUABaH75nAtbGQUfLSFAhfE52Lj/xL9uBFTM2jismeGrumPnPtp80kDBiGF
         HeWA==
X-Gm-Message-State: AOJu0YxcruQx77Pnsmb2evLFYw8vvBOBNFJWWgqpXuKBeudfEyiU7e98
	pGhJzYed2ndYHqsI7+2fcQ2vBFbcdwi23Ze+R2CCruI0znHwLzi3suQ2lll+RB8=
X-Google-Smtp-Source: AGHT+IHHgdYxQLDhYbhAGcyi8pYRv0UsQ+cs0MwDGoqFrJgfHuHis0KG8K5BnuwxByAlX3jOGw7jyQ==
X-Received: by 2002:ad4:4d07:0:b0:6ad:60d8:183c with SMTP id 6a1803df08f44-6ad60d81deemr1019106d6.20.1716541111720;
        Fri, 24 May 2024 01:58:31 -0700 (PDT)
Date: Fri, 24 May 2024 10:58:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 8/8] xen/x86: Synthesise domain topologies
Message-ID: <ZlBWtXkpkqzh1EWC@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>

On Wed, May 08, 2024 at 01:39:27PM +0100, Alejandro Vallejo wrote:
> Expose sensible topologies in leaf 0xb. At the moment it synthesises non-HT
> systems, in line with the previous code intent.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> ---
> v2:
>   * Zap the topology leaves of (pv/hvm)_(def/max)_policy rather than the host policy
> ---
>  tools/libs/guest/xg_cpuid_x86.c | 62 +++++----------------------------
>  xen/arch/x86/cpu-policy.c       |  9 +++--
>  2 files changed, 15 insertions(+), 56 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4453178100ad..8170769dbe43 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -584,7 +584,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      bool hvm;
>      xc_domaininfo_t di;
>      struct xc_cpu_policy *p = xc_cpu_policy_init();
> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>      uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
>      uint32_t len = ARRAY_SIZE(host_featureset);
> @@ -727,59 +727,15 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      }
>      else
>      {
> -        /*
> -         * Topology for HVM guests is entirely controlled by Xen.  For now, we
> -         * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
> -         */
> -        p->policy.basic.htt = true;
> -        p->policy.extd.cmp_legacy = false;
> -
> -        /*
> -         * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
> -         * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
> -         * overflow.
> -         */
> -        if ( !p->policy.basic.lppp )
> -            p->policy.basic.lppp = 2;
> -        else if ( !(p->policy.basic.lppp & 0x80) )
> -            p->policy.basic.lppp *= 2;
> -
> -        switch ( p->policy.x86_vendor )
> +        /* TODO: Expose the ability to choose a custom topology for HVM/PVH */
> +        unsigned int threads_per_core = 1;
> +        unsigned int cores_per_pkg = di.max_vcpu_id + 1;

Newline.

> +        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);

I assume this generates the same topology as the current code, or will
the population of the leaves be different in some way?

> +        if ( rc )
>          {
> -        case X86_VENDOR_INTEL:
> -            for ( i = 0; (p->policy.cache.subleaf[i].type &&
> -                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
> -            {
> -                p->policy.cache.subleaf[i].cores_per_package =
> -                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
> -                p->policy.cache.subleaf[i].threads_per_cache = 0;
> -            }
> -            break;
> -
> -        case X86_VENDOR_AMD:
> -        case X86_VENDOR_HYGON:
> -            /*
> -             * Leaf 0x80000008 ECX[15:12] is ApicIdCoreSize.
> -             * Leaf 0x80000008 ECX[7:0] is NumberOfCores (minus one).
> -             * Update to reflect vLAPIC_ID = vCPU_ID * 2.  But avoid
> -             * - overflow,
> -             * - going out of sync with leaf 1 EBX[23:16],
> -             * - incrementing ApicIdCoreSize when it's zero (which changes the
> -             *   meaning of bits 7:0).
> -             *
> -             * UPDATE: I addition to avoiding overflow, some
> -             * proprietary operating systems have trouble with
> -             * apic_id_size values greater than 7.  Limit the value to
> -             * 7 for now.
> -             */
> -            if ( p->policy.extd.nc < 0x7f )
> -            {
> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
> -                    p->policy.extd.apic_id_size++;
> -
> -                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
> -            }
> -            break;
> +            ERROR("Failed to generate topology: t/c=%u c/p=%u",
> +                  threads_per_core, cores_per_pkg);

Could you also print the error code?

> +            goto out;
>          }
>      }
>  
> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
> index 4b6d96276399..0ad871732ba0 100644
> --- a/xen/arch/x86/cpu-policy.c
> +++ b/xen/arch/x86/cpu-policy.c
> @@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
>  
>      p->basic.raw[0x8] = EMPTY_LEAF;
>  
> -    /* TODO: Rework topology logic. */
> -    memset(p->topo.raw, 0, sizeof(p->topo.raw));
> -
>      p->basic.raw[0xc] = EMPTY_LEAF;
>  
>      p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
> @@ -621,6 +618,9 @@ static void __init calculate_pv_max_policy(void)
>      recalculate_xstate(p);
>  
>      p->extd.raw[0xa] = EMPTY_LEAF; /* No SVM for PV guests. */
> +
> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>  }
>  
>  static void __init calculate_pv_def_policy(void)
> @@ -773,6 +773,9 @@ static void __init calculate_hvm_max_policy(void)
>  
>      /* It's always possible to emulate CPUID faulting for HVM guests */
>      p->platform_info.cpuid_faulting = true;
> +
> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */

Line length.

/* Wipe host topology.  Populated by toolstack. */

Would be OK IMO.

Thanks, Roger.
> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));

Note that currently the host policy also gets the topology leaves
cleared, is it intended to not clear them anymore after this change?

(as you only clear the leaves for the guest {max,def} policies)

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 08:59:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 08:59:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729190.1134300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQlW-0006Qo-Il; Fri, 24 May 2024 08:59:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729190.1134300; Fri, 24 May 2024 08:59:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAQlW-0006Qh-FP; Fri, 24 May 2024 08:59:06 +0000
Received: by outflank-mailman (input) for mailman id 729190;
 Fri, 24 May 2024 08:59:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sAQlU-0006Q7-Jk
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 08:59:04 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dcb9bcad-19ab-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 10:59:00 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e52181c228so84190861fa.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 01:59:00 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bcc4a5esm1150761fa.17.2024.05.24.01.58.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 01:58:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dcb9bcad-19ab-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716541140; x=1717145940; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=TE++0mqN830mI0zg2Twwha2LqgPNu+D/hgMhSYXEPEc=;
        b=CreWCAzCYO8zcDD2+mpRWp/n/4Ta9wLNugC2Z4zE5UGSyycAehneIP0tS8ioExDx68
         wL7jIW/CwiIShL6sW/LePjikVslRenop5bv6zmhcLyQKS/wHGKMpombUaPDFjNrv4zJ1
         pRMsRnVqw4470EpXbQJntajrLucK5L3So/uyBesCtnrPWPkUM9MrOxKJtHNq+Xsnm6eO
         Y1rc57V2bbRidAmxgclydIaR0Dh0Rp4Igh8RQu2Mn5sJflNduWbrYwcAKsktSCNEezEK
         0yISjOi4aqyZai3m6SX3RPMgYsGvIRweN9AEOkHmld6fbtUk2YoZ9xQHzCKuKW7+gDH0
         uypw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716541140; x=1717145940;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TE++0mqN830mI0zg2Twwha2LqgPNu+D/hgMhSYXEPEc=;
        b=QPWInHJBCZmYFcmYN/u1rbUfmtnLLK5vUX/t2mu37SWFlY1zlylKC2F90dJqFSpTDs
         swUMI6FMHrluHGQ98JP54xVxQNg/7ZYoEGQ/4wmoLaSNVQkPr3Cqb8bb/Ovz/cNlmClW
         4MBd/OZdTHKEsMlezwpjW4V+O4FuplwUvvfDf8l0qL5/4WROhBTdaNww1No3VC15CAeC
         1uBE27UE4PLfzUemLIukcpvlEtcUMzCEW4LjGA0etx/+r6I2HpXYGYZf6RDCRYWTmI2h
         3n2yMg1lwN+7ieoys3Oc5lKsiJeFJ17qspG6FQRcI+aWFLjkLx5SAOHZxgfT1xfZ1jvC
         xAUQ==
X-Forwarded-Encrypted: i=1; AJvYcCWme8tL0EvyCie27ASo7iOYllPetp3drsubTa0FgoVLHPzWaNEGHDtn62SQVusMJCjpU1rSHn8dBUePJGuNiLuPvmC9wIbFcJ4JfwvUOpU=
X-Gm-Message-State: AOJu0YzPhkggtL0pZLcT0xgo1xF/BUpVZDegaJRcTa/kqvFpMI2WeYhq
	qv+4poOMAEHk3rYVVJDLsgXovZHSgAAQkEA6DWSdhI2Dyaz+LZ6x
X-Google-Smtp-Source: AGHT+IHkni//8e/T8wQnYgjD9Z35amuMBlrv2KQoD1BxDfFiT9oRX8c00EbbRVVGAVqOAmK1U5JPpg==
X-Received: by 2002:a2e:bc08:0:b0:2e9:570e:1cf with SMTP id 38308e7fff4ca-2e95b3089d9mr10591321fa.52.1716541139842;
        Fri, 24 May 2024 01:58:59 -0700 (PDT)
Message-ID: <82b8828112ffb05170472310d7d510ceb4edc555.camel@gmail.com>
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Ross
 Lagerwall <ross.lagerwall@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>,  Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
Date: Fri, 24 May 2024 10:58:58 +0200
In-Reply-To: <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
	 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
	 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
	 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
	 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
> > > > =C2=A0 =C2=A0 #include <asm/bitops.h>
> > > > =C2=A0=C2=A0=C2=A0=20
> > > > +/**
> > > > + * generic__test_and_set_bit - Set a bit and return its old
> > > > value
> > > > + * @nr: Bit to set
> > > > + * @addr: Address to count from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > >=20
> > > Sorry for only mentioning this on v10. I think this comment
> > > should be
> > > duplicated (or moved to) on top of test_bit() because this is
> > > what
> > > everyone will use. This will avoid the developper to follow the
> > > function
> > > calls and only notice the x86 version which says "This function
> > > is
> > > atomic and may not be reordered." and would be wrong for all the
> > > other arch.
> > It makes sense to add this comment on top of test_bit(), but I am
> > curious if it is needed to mention that for x86 arch_test_bit() "is
> > atomic and may not be reordered":
>=20
> I would say no because any developper modifying common code can't=20
> relying it.
But won't then be confusion that if not generic implementation of
test_bit() is chosen then test_bit() can be " atomic and cannot be
reordered " ( as it is in case of x86 )?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Fri May 24 09:17:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:17:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729200.1134311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAR3F-0001DP-0I; Fri, 24 May 2024 09:17:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729200.1134311; Fri, 24 May 2024 09:17:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAR3E-0001DI-Tx; Fri, 24 May 2024 09:17:24 +0000
Received: by outflank-mailman (input) for mailman id 729200;
 Fri, 24 May 2024 09:17:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAR3E-0001DC-7y
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:17:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAR3D-00066I-Rb; Fri, 24 May 2024 09:17:23 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAR3D-0002qI-GT; Fri, 24 May 2024 09:17:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=9AZj1h9UOhZHEe/Py/Og1fOziSe/VD3m91jq1FB5heI=; b=cqfAA9bP8Y7fLI3W7qUv0jN1eX
	leVAPMYMBRRMnUgYAZFE6lqBPcXdbn/NI5fvc9aVROOEz5uh/zfY2pwTP99uUDBcX5RV3VSNQP3AM
	tJRAoggKdFxN9w5Te2IGcJED2IpxIvkE5buh7zTpsAByeEmSmG1PcUB2rcGqFoR65L7E=;
Message-ID: <555a80ba-3981-4b3e-877d-77ff172ac186@xen.org>
Date: Fri, 24 May 2024 10:17:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 02/14] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <cover.1715952103.git.oleksii.kurochko@gmail.com>
 <219df9d840a183fc55de02aff011c0972a68587c.1715952103.git.oleksii.kurochko@gmail.com>
 <1554d534-7998-4ad4-9218-579b2568b744@xen.org>
 <9f02d973c2520d36ee33d01b871378de16ede4d0.camel@gmail.com>
 <a535a3fc-5b90-454f-8d3f-202de631bc49@xen.org>
 <82b8828112ffb05170472310d7d510ceb4edc555.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <82b8828112ffb05170472310d7d510ceb4edc555.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 24/05/2024 09:58, Oleksii K. wrote:
> On Thu, 2024-05-23 at 15:33 +0100, Julien Grall wrote:
>>>>>      #include <asm/bitops.h>
>>>>>      
>>>>> +/**
>>>>> + * generic__test_and_set_bit - Set a bit and return its old
>>>>> value
>>>>> + * @nr: Bit to set
>>>>> + * @addr: Address to count from
>>>>> + *
>>>>> + * This operation is non-atomic and can be reordered.
>>>>> + * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>> + */
>>>>
>>>> Sorry for only mentioning this on v10. I think this comment
>>>> should be
>>>> duplicated (or moved to) on top of test_bit() because this is
>>>> what
>>>> everyone will use. This will avoid the developper to follow the
>>>> function
>>>> calls and only notice the x86 version which says "This function
>>>> is
>>>> atomic and may not be reordered." and would be wrong for all the
>>>> other arch.
>>> It makes sense to add this comment on top of test_bit(), but I am
>>> curious if it is needed to mention that for x86 arch_test_bit() "is
>>> atomic and may not be reordered":
>>
>> I would say no because any developper modifying common code can't
>> relying it.
> But won't then be confusion that if not generic implementation of
> test_bit() is chosen then test_bit() can be " atomic and cannot be
> reordered " ( as it is in case of x86 )?

I don't understand what confusion could arise. A developper cannot rely 
on the x86 behavior in common code. They have to write code that works 
for every arch.

The first thing a developer will do is to check test_bit(). The comment 
on top will say they can't relying on ordering. And that what most of 
the developper needs to rely on.

If someone wants to write more optimized code for x86, they are free to 
look at the implementation. But I don't think this should be detailed on 
top of the common wrapper (the x86 implementation would still be 
compatible with the comment).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:27:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:27:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729210.1134321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARD7-00035b-Va; Fri, 24 May 2024 09:27:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729210.1134321; Fri, 24 May 2024 09:27:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARD7-00035U-Sx; Fri, 24 May 2024 09:27:37 +0000
Received: by outflank-mailman (input) for mailman id 729210;
 Fri, 24 May 2024 09:27:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sARD6-00035O-Vk
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:27:36 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARD6-0006HI-Az; Fri, 24 May 2024 09:27:36 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARD6-0003Ge-3N; Fri, 24 May 2024 09:27:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=WxQ8/sE4b6bNUwlJcvudr7BBemUV0Y/a9eCa0Swo9fU=; b=PDLaKjbo6WYrv0nCiW105NPZEk
	D7qTIJ7W0Nxly7owPD8J3au4aJRHBOAVNQrkqxprzsZGm7Ega7/6zSmyB4h/H2RiXT0xN6zQcvIu6
	o32jip4G/iMFEP6POLFpQsIpHOepf2XKaSivtTEEOpYGnax2sSfLfIjryqml/+tfeYgY=;
Message-ID: <56bf92b7-9bcc-4093-b51c-b8ec0b43209c@xen.org>
Date: Fri, 24 May 2024 10:27:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 3/7] tools/arm: Introduce the "nr_spis" xl config entry
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Henry Wang <xin.wang2@amd.com>, Jason Andryuk <jason.andryuk@amd.com>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
 <20240524021814.2666257-3-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524021814.2666257-3-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 24/05/2024 03:18, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> Currently, the number of SPIs allocated to the domain is only
> configurable for Dom0less DomUs. Xen domains are supposed to be
> platform agnostics and therefore the numbers of SPIs for libxl
> guests should not be based on the hardware.
> 
> Introduce a new xl config entry for Arm to provide a method for
> user to decide the number of SPIs. This would help to avoid
> bumping the `config->arch.nr_spis` in libxl everytime there is a
> new platform with increased SPI numbers.
> 
> Update the doc and the golang bindings accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> ---
>   docs/man/xl.cfg.5.pod.in             | 16 ++++++++++++++++
>   tools/golang/xenlight/helpers.gen.go |  2 ++
>   tools/golang/xenlight/types.gen.go   |  1 +
>   tools/include/libxl.h                |  7 +++++++
>   tools/libs/light/libxl_arm.c         |  4 ++--
>   tools/libs/light/libxl_types.idl     |  1 +
>   tools/xl/xl_parse.c                  |  3 +++
>   7 files changed, 32 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 8f2b375ce9..ac3f88fd57 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3072,6 +3072,22 @@ raised.
>   
>   =back
>   
> +=over 4
> +
> +=item B<nr_spis="NR_SPIS">
> +
> +An optional integer parameter specifying the number of SPIs (Shared
> +Peripheral Interrupts) to allocate for the domain. Max is 991 SPIs. If
> +the value specified by the `nr_spis` parameter is smaller than the
> +number of SPIs calculated by the toolstack based on the devices
> +allocated for the domain, or the `nr_spis` parameter is not specified,
> +the value calculated by the toolstack will be used for the domain.
> +Otherwise, the value specified by the `nr_spis` parameter will be used.
> +The number of SPIs should match the highest interrupt ID that will be
> +assigned to the domain.
> +
> +=back
> +
>   =head3 x86
>   
>   =over 4
> diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
> index b9cb5b33c7..fe5110474d 100644
> --- a/tools/golang/xenlight/helpers.gen.go
> +++ b/tools/golang/xenlight/helpers.gen.go
> @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
>   x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
>   x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
>   x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
> +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
>   if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>   }
> @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
>   xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
>   xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
>   xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
> +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
>   if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>   }
> diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
> index 5b293755d7..c9e45b306f 100644
> --- a/tools/golang/xenlight/types.gen.go
> +++ b/tools/golang/xenlight/types.gen.go
> @@ -597,6 +597,7 @@ ArchArm struct {
>   GicVersion GicVersion
>   Vuart VuartType
>   SveVl SveType
> +NrSpis uint32
>   }
>   ArchX86 struct {
>   MsrRelaxed Defbool
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index 62cb07dea6..3b5c18b48b 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -636,6 +636,13 @@
>    */
>   #define LIBXL_HAVE_XEN_9PFS 1
>   
> +/*
> + * LIBXL_HAVE_NR_SPIS indicates the presence of the nr_spis field in
> + * libxl_domain_build_info that specifies the number of SPIs interrupts
> + * for the guest.
> + */
> +#define LIBXL_HAVE_NR_SPIS 1
> +

Looking at the other arch.arm field, I think this wants to be:

/*
  * libxl_domain_build_info has the arch_arm.nr_spis field
  */
#define LIBXL_HAVE_BUILDINFO_ARCH_NR_SPIS 1

This would also clarify that the field is Arm specific.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:34:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:34:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729215.1134331 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARJw-0004tJ-Ki; Fri, 24 May 2024 09:34:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729215.1134331; Fri, 24 May 2024 09:34:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARJw-0004tC-Hz; Fri, 24 May 2024 09:34:40 +0000
Received: by outflank-mailman (input) for mailman id 729215;
 Fri, 24 May 2024 09:34:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sARJv-0004t6-IG
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:34:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARJv-0006No-1j; Fri, 24 May 2024 09:34:39 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARJu-0003ju-QJ; Fri, 24 May 2024 09:34:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zAgA/zrtVk/TEkpUAt+31TjtPcftSSa8FMCT76BStYE=; b=fskNRylt+MXgai1HpYkrIQAbsH
	T81a4vzK/GjgwqLeGjYkanh294wI+EkqmoCD8QFbN3E0ETDbr7TYj2uK4//8RxO4EKAnH5JfidHZS
	E2zvZ1efBitCqvU18OVbKg2h3jfXG++bptBBW7xgU1QhBlGEcGyKl7zOQf0WYdiKvdag=;
Message-ID: <760dba0c-8731-48a3-81d8-cd8fa2dada77@xen.org>
Date: Fri, 24 May 2024 10:34:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 5/7] xen/arm: Add XEN_DOMCTL_dt_overlay and device
 attachment to domains
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Henry Wang <xin.wang2@amd.com>, Vikram Garhwal <fnu.vikram@xilinx.com>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
 <20240524021814.2666257-5-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524021814.2666257-5-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 03:18, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> In order to support the dynamic dtbo device assignment to a running
> VM, the add/remove of the DT overlay and the attach/detach of the
> device from the DT overlay should happen separately. Therefore,
> repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
> overlay to Xen device tree, instead of assigning the device to the
> hardware domain at the same time. It is OK to change the sysctl behavior
> as this feature is experimental so changing sysctl behavior and breaking
> compatibility is OK.
> 
> Add the XEN_DOMCTL_dt_overlay with operations
> XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment to the domain.
> 
> The hypervisor firstly checks the DT overlay passed from the toolstack
> is valid. Then the device nodes are retrieved from the overlay tracker
> based on the DT overlay. The attach of the device is implemented by
> mapping the IRQ and IOMMU resources. All devices in the overlay are
> assigned to a single domain.
> 
> Also take the opportunity to make one coding style fix in sysctl.h.
> 
> xen,reg is to be used to handle non-1:1 mappings but it is currently
> unsupported.

This means that we would still try to use 1:1 mappings for non-directmap 
domain. Given that the overlay is a blob, I am a bit concerned that the 
user may not notice any clash and it would be difficult to debug.

Therefore, I would like xen,reg to be mandatory when using non 
directmapped domain. For now, the best approach would be to prevent 
device assignment if !is_domain_direct_mapped().

> +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
> +{
> +    long ret;
> +    void *overlay_fdt;
> +
> +    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
> +        return -EOPNOTSUPP;
> +
> +    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )

Please add #define DT_OVERLAY_MAX_SIZE KB(500) and use it here and the 
other place.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:36:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:36:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729220.1134341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARLj-0005RU-Us; Fri, 24 May 2024 09:36:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729220.1134341; Fri, 24 May 2024 09:36:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARLj-0005RN-S3; Fri, 24 May 2024 09:36:31 +0000
Received: by outflank-mailman (input) for mailman id 729220;
 Fri, 24 May 2024 09:36:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sARLi-0005RD-NZ; Fri, 24 May 2024 09:36:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sARLi-0006Qm-Lo; Fri, 24 May 2024 09:36:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sARLi-0005ox-A5; Fri, 24 May 2024 09:36:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sARLi-0007p2-9S; Fri, 24 May 2024 09:36:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ukFZc+RUt8tXgWbrobQQJHa0wv9i1Ib8GIEaMxTat2c=; b=p5etMaqavZ7kMEMI8r8hL3fhfG
	Yc1MQjHvV2fsaT5oKEvTsjYAUkZBuiHbcWXyoQrY4Ky33EUNIt4RRgH4IPATQEHQPWoyfSftgorbd
	yw/PLsetqvNUt9qEcvrRQpQbf2F93GpyXM4LXiVNFcZtdSCOIVGbtOMFpF3E3VnflVVY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186127-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186127: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6d69b6c12fce479fde7bc06f686212451688a102
X-Osstest-Versions-That:
    linux=8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 09:36:30 +0000

flight 186127 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186127/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 186052
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186052
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186052
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186052
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6d69b6c12fce479fde7bc06f686212451688a102
baseline version:
 linux                8f6a15f095a63a83b096d9b29aaff4f0fbe6f6e6

Last test of basis   186052  2024-05-21 01:42:42 Z    3 days
Failing since        186065  2024-05-21 16:10:24 Z    2 days    4 attempts
Testing same since   186127  2024-05-23 23:10:22 Z    0 days    1 attempts

------------------------------------------------------------
524 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   8f6a15f095a6..6d69b6c12fce  6d69b6c12fce479fde7bc06f686212451688a102 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:37:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:37:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729226.1134351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARMd-0005zW-6y; Fri, 24 May 2024 09:37:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729226.1134351; Fri, 24 May 2024 09:37:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARMd-0005zP-4S; Fri, 24 May 2024 09:37:27 +0000
Received: by outflank-mailman (input) for mailman id 729226;
 Fri, 24 May 2024 09:37:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sARMb-0005zC-Jz
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:37:25 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARMa-0006Rh-VL; Fri, 24 May 2024 09:37:24 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARMa-0003t8-Oq; Fri, 24 May 2024 09:37:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=M/ap9Jl/BXKwelNgETJCV0tL+eXZ4a2GRGwQsFB6O8o=; b=BUdzL8IVN0nr0XocRyTfCRJ0uR
	0SD3XSPKk9JEyfAdvvLxHQ/Bf5Kuh/AW+NRVZvGExOgvL5rHB6seubfwinE8XHb2n7S5fziqpsTCO
	Tdg+JhsXH567GhikVz9LXO+guj2311oeaVs/WyV9YJo9UDuAc9bCwtM1twxhYbmlZReU=;
Message-ID: <59607ca9-3d78-4f8e-ad02-22df89cd9161@xen.org>
Date: Fri, 24 May 2024 10:37:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 6/7] tools: Introduce the "xl dt-overlay attach"
 command
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Henry Wang <xin.wang2@amd.com>, Jason Andryuk <jason.andryuk@amd.com>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
 <20240524021814.2666257-6-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524021814.2666257-6-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 03:18, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> attach (in the future also detach) devices from the provided DT overlay
> to domains. Support this by introducing a new "xl dt-overlay" command
> and related documentation, i.e. "xl dt-overlay attach. Slightly rework
> the command option parsing logic.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>   tools/include/libxl.h               | 15 +++++++++++
>   tools/include/xenctrl.h             |  3 +++
>   tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++++
>   tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++++
>   tools/xl/xl_cmdtable.c              |  4 +--
>   tools/xl/xl_vmcontrol.c             | 39 ++++++++++++++++++++---------
>   6 files changed, 106 insertions(+), 14 deletions(-)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index 3b5c18b48b..f2e19ec592 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -643,6 +643,12 @@
>    */
>   #define LIBXL_HAVE_NR_SPIS 1
>   
> +/*
> + * LIBXL_HAVE_OVERLAY_DOMAIN indicates the presence of
> + * libxl_dt_overlay_domain.
> + */
> +#define LIBXL_HAVE_OVERLAY_DOMAIN 1
I think this wants to gain DT_ just before OVERLAY. So from the name it 
is clearer we are talking about the Device-Tree overlay and not 
filesystem (or anything else where overlays are involved).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:55:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729236.1134360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAReN-00014b-OA; Fri, 24 May 2024 09:55:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729236.1134360; Fri, 24 May 2024 09:55:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAReN-00014U-Ld; Fri, 24 May 2024 09:55:47 +0000
Received: by outflank-mailman (input) for mailman id 729236;
 Fri, 24 May 2024 09:55:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fH/U=M3=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1sAReM-00014I-IG
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:55:46 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c960a540-19b3-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 11:55:44 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-574bf7ab218so13345231a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 02:55:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c960a540-19b3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716544543; x=1717149343; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=L+g0FtDwU683l5qSvCAdrglSzPW3nPeatyB/ievqbjc=;
        b=RsTfL+sQcGh415aYJe6bHeC69eTdgMpPsZxL33IPmZ3lgJcbNbftGhPUqa+ldkLr9l
         BHo1FoxmNVBarHfP95iQ1pg0IEy8QiRg7nXYtcitb66LEARAdVr5XRVx42W6Jr3RtWxq
         nBuvPt3MOCXS2vZOmv+shuMoWVtczPh8toVho=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716544543; x=1717149343;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=L+g0FtDwU683l5qSvCAdrglSzPW3nPeatyB/ievqbjc=;
        b=QKBDXuLm9vkAgYVq6Jxek193/vZxD2U9lR6ARHXEsclSkZAMg5pfivauUfmybRM8mt
         AZJBOjKBBkIaLZDMaxo5D7SDdJEvJUWOfDdxscTdrVWYI79dAyj1jRncJGgjWUElsF9Y
         lwSwpG2RJWx9zwc6lxB8DD/Z8jLLTfCzyVytM8FwfvdIcyxdm3GWAfZAT1ix1Srt3Xti
         PGsroG4EBeKmDDBbkpV4LJgToRqVfa68LNWdGOwNdfkRR9SyvbBdS9lsQ0aHngYa5S+w
         ymj94TLj9DIQOc0nFb6GDu1l6bXWvq8zQB4RGwmpIzOoy/Qp5J2be/zw73adZL7EuKGV
         ZNxA==
X-Gm-Message-State: AOJu0YyzxqpbNNEHl70keww8dS6b38ikNA0FOS6iaGw5/d/NxcvBdgiD
	48C/dyjtgpk3l525HD7s8N4KFZRjdUeYnbZBtHeneDjv5MeayiMO3lL09t6FTEFkFwlpvXojAXb
	VC4QW6+9wB47MiACjrhTOMX0sO7kR2WxNr0CE4sK39Df/O8au5JQY0w==
X-Google-Smtp-Source: AGHT+IHlaRv6cTerqP7QEtIiYnEzFc4oSzkqG3Rk+qU9WCQTroJIZ/XqUtGHgXFaEESQOJpO59rKchgHBPKuMNDKkek=
X-Received: by 2002:a50:aa88:0:b0:578:341f:86b8 with SMTP id
 4fb4d7f45d1cf-57851a295f5mr982421a12.38.1716544543275; Fri, 24 May 2024
 02:55:43 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Fri, 24 May 2024 10:55:07 +0100
Message-ID: <CAO-mL=ytbLHE6QYfB9tTYepqqV_pcFbhgwXnQBbUH6=dhVVDkQ@mail.gmail.com>
Subject: [REMINDER] Xen Summit 2024 - Verification code for design sessions
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000be72de0619302be4"

--000000000000be72de0619302be4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Xen Community,

Day 2 design sessions will be locked end of today, so get voting!
<https://design-sessions.xenproject.org/>
Day 3 design sessions are still open, and will be finalized 3 June 2024
after 5pm Lisbon Time.

*The verification code is 'LFXEN24'. *

The final schedule will be allocated and arranged by the highest-voted
sessions.

Virtual links for the community to join Xen Summit design sessions will be
shared next week.  For in-person tickets, click here
<https://events.linuxfoundation.org/xen-project-summit/register/>.

------------------------------


We look forward to holding the Design Sessions at the upcoming Xen Project
Summit. The design sessions will be on Wednesday, 5 June, and Thursday, 6
June 2024.

We encourage everyone to submit a Design Session, the verification code is:
=E2=80=9C*LFXEN24*=E2=80=9D.

*SUBMIT A DESIGN SESSION* <https://design-sessions.xenproject.org/>



The process involves the following steps:

   - Anyone interested can propose
   <https://design-sessions.xenproject.org/discussion/new> a topic.
   - All participants review the list of sessions
   <https://design-sessions.xenproject.org/list/discussion>, indicating
   their interest in attending each one.
   - The session scheduler optimizes the schedule
   <https://design-sessions.xenproject.org/schedule> to accommodate as many
   preferences as possible.

Participants can also propose long-form talks by adding [TALK] to the
session title.

For suggested topics, sample Design Session submissions, and more tips
check out the Xen Design Session page
<https://events.linuxfoundation.org/xen-project-summit/program/design-sessi=
ons/>
for
more information.


Best Regards,
Xen Project Events Team


Many thanks,
Kelly Choi

Community Manager
Xen Project

--000000000000be72de0619302be4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div dir=3D"ltr" class=3D"gmail_signature"><div dir=
=3D"ltr"><div>Hello Xen Community,</div><div><br></div><div>Day 2 design se=
ssions will be locked end of today, <a href=3D"https://design-sessions.xenp=
roject.org/" style=3D"">so get voting!</a></div><div>Day 3 design sessions =
are still open, and=C2=A0will be finalized 3 June 2024 after 5pm Lisbon Tim=
e.</div><div><br></div><div><b>The verification code is &#39;LFXEN24&#39;.=
=C2=A0</b><br></div><div><b><br></b></div><div>The final schedule will be a=
llocated and arranged by the highest-voted=C2=A0<span class=3D"gmail-il">se=
ssions</span>.=C2=A0</div><div><br></div><div>Virtual links for the communi=
ty to join Xen Summit=C2=A0<span class=3D"gmail-il">design</span>=C2=A0<spa=
n class=3D"gmail-il">sessions</span>=C2=A0will be shared next week.=C2=A0 F=
or in-person tickets,=C2=A0<a href=3D"https://events.linuxfoundation.org/xe=
n-project-summit/register/" target=3D"_blank">click here</a>.=C2=A0</div><d=
iv><p class=3D"MsoNormal"><u></u></p><table border=3D"0" cellspacing=3D"0" =
cellpadding=3D"0" width=3D"100%" style=3D"width:949.612px"><tbody><tr><td s=
tyle=3D"padding:0.75pt"><div class=3D"MsoNormal" align=3D"center" style=3D"=
text-align:center"><hr size=3D"1" width=3D"100%" noshade align=3D"center" s=
tyle=3D"color:rgb(160,160,160)"></div><p class=3D"MsoNormal"></p></td></tr>=
</tbody></table></div><div><br></div></div></div></div><div class=3D"gmail_=
quote"><div><div lang=3D"EN-US" id=3D"m_-5551937392530054680m_6486684977004=
734091m_-2685003709366885060body"><div><div id=3D"m_-5551937392530054680m_6=
486684977004734091m_-2685003709366885060mail-editor-reference-message-conta=
iner"><div><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D=
"100%" style=3D"width:1051.99px;float:left"><tbody><tr><td style=3D"backgro=
und:rgb(239,240,242);padding:0in"><div align=3D"center"><table border=3D"0"=
 cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99p=
x;float:left"><tbody><tr><td style=3D"padding:0in"><div align=3D"center"><t=
able border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=
=3D"width:1051.99px;background:white"><tbody><tr><td style=3D"padding:0in">=
<div align=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"=
0" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td valign=3D"top" s=
tyle=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0=
" align=3D"left" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td st=
yle=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0"=
 width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td style=3D"padding:0=
in"><div align=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=
=3D"0" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td style=3D"pad=
ding:0in"><p class=3D"MsoNormal" align=3D"center" style=3D"text-align:cente=
r"><img border=3D"0" width=3D"699" id=3D"m_-5551937392530054680m_6486684977=
004734091m_-2685003709366885060_x0000_i1028" src=3D"https://custom.cvent.co=
m/64779B2AC74D4D2BBFF4459DE93A5253/pix/d770244914e047019777029a32e20c53.png=
?d=3D699" class=3D"gmail-CToWUd gmail-a6T" tabindex=3D"0" style=3D"cursor: =
pointer; outline: 0px; width: 7.2812in;"></p></td></tr></tbody></table></di=
v><p class=3D"MsoNormal" align=3D"center" style=3D"text-align:center"></p><=
div align=3D"center"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0=
" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td style=3D"padding:=
0in"></td></tr></tbody></table></div><p class=3D"MsoNormal" align=3D"center=
" style=3D"text-align:center"><span style=3D"font-size:10pt;font-family:&qu=
ot;Times New Roman&quot;,serif"><u></u><u></u></span></p></td></tr></tbody>=
</table></td></tr></tbody></table></td></tr></tbody></table></div></td></tr=
></tbody></table></div></td></tr><tr><td style=3D"padding:0in;word-break:br=
eak-word"><div align=3D"center"><table border=3D"0" cellspacing=3D"0" cellp=
adding=3D"0" width=3D"100%" style=3D"width:1051.99px;background:white;float=
:left"><tbody><tr><td style=3D"padding:0in;float:left"><div align=3D"center=
"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" st=
yle=3D"width:1051.99px;float:left"><tbody><tr><td valign=3D"top" style=3D"p=
adding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=
=3D"left" width=3D"100%" style=3D"width:1051.99px;word-break:break-word"><t=
body><tr><td style=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" ce=
llpadding=3D"0" width=3D"100%" style=3D"width:1051.99px"><tbody><tr><td sty=
le=3D"padding:11.25pt 11.25pt 0in"><table border=3D"0" cellspacing=3D"0" ce=
llpadding=3D"0" width=3D"100%" style=3D"width:1021.99px"><tbody><tr><td sty=
le=3D"padding:0in"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" =
width=3D"100%" style=3D"width:1021.99px"><tbody><tr><td style=3D"padding:3.=
75pt 0in"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=
=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:black">We look forw=
ard to holding the=C2=A0<span class=3D"gmail-il">Design</span>=C2=A0<span c=
lass=3D"gmail-il">Sessions</span>=C2=A0at the upcoming Xen Project Summit. =
The=C2=A0<span class=3D"gmail-il">design</span>=C2=A0<span class=3D"gmail-i=
l">sessions</span>=C2=A0will be on Wednesday, 5 June, and Thursday, 6 June =
2024.<br><br>We encourage everyone to submit a=C2=A0<span class=3D"gmail-il=
">Design</span>=C2=A0<span class=3D"gmail-il">Session</span>, the=C2=A0</sp=
an><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(3=
3,33,33)">verification code</span><span style=3D"font-size:10.5pt;font-fami=
ly:Arial,sans-serif;color:black">=C2=A0is: =E2=80=9C</span><b><span style=
=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,33,33)">LFXE=
N24</span></b><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;=
color:black">=E2=80=9D.<u></u><u></u></span></p></td></tr></tbody></table><=
/td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></tabl=
e></td></tr><tr><td valign=3D"top" style=3D"padding:0in"><table border=3D"0=
" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"100%" style=
=3D"width:1051.99px;float:left"><tbody><tr><td style=3D"padding:0in"><table=
 border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"w=
idth:1051.99px;border-radius:15px"><tbody><tr><td style=3D"padding:11.25pt"=
><div align=3D"center"><table border=3D"1" cellspacing=3D"0" cellpadding=3D=
"0" style=3D"background:rgb(114,190,68);border:4.5pt solid rgb(114,190,68)"=
><tbody><tr><td style=3D"border:none;padding:3.75pt 15pt"><p class=3D"MsoNo=
rmal" align=3D"center" style=3D"text-align:center"><span style=3D"color:bla=
ck"><a href=3D"https://design-sessions.xenproject.org/" target=3D"_blank"><=
b><span style=3D"font-size:13pt;font-family:Arial,sans-serif;color:white;te=
xt-decoration-line:none">SUBMIT A=C2=A0<span class=3D"gmail-il">DESIGN</spa=
n>=C2=A0<span class=3D"gmail-il">SESSION</span></span></b></a></span></p></=
td></tr></tbody></table></div><p class=3D"MsoNormal" align=3D"center" style=
=3D"text-align:center"><span style=3D"font-size:1pt">=C2=A0</span></p></td>=
</tr></tbody></table></td></tr></tbody></table></td></tr><tr><td valign=3D"=
top" style=3D"padding:0in;word-break:break-word"><table border=3D"0" cellsp=
acing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"100%" style=3D"width:=
1051.99px;float:left"><tbody><tr><td style=3D"padding:0in;float:left"><tabl=
e border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"=
width:872.825px;word-break:break-word"><tbody><tr><td style=3D"padding:0in =
11.25pt 11.25pt"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" wi=
dth=3D"100%" style=3D"width:842.825px"><tbody><tr><td style=3D"padding:0in"=
><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" sty=
le=3D"width:842.825px"><tbody><tr><td style=3D"padding:3.75pt 0in"><p class=
=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-size:10.5pt=
;font-family:Arial,sans-serif;color:rgb(33,37,41)">The process involves the=
 following steps:</span><span style=3D"font-size:10.5pt;font-family:Arial,s=
ans-serif;color:black"><u></u><u></u></span></p></td></tr><tr><td style=3D"=
padding:3.75pt 0in;word-break:break-word"><ul type=3D"disc"><li class=3D"Ms=
oNormal" style=3D"margin-left:15px;color:black;line-height:13.5pt"><span st=
yle=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,37,41)">A=
nyone interested can=C2=A0</span><a href=3D"https://design-sessions.xenproj=
ect.org/discussion/new" target=3D"_blank"><span style=3D"font-size:10.5pt;f=
ont-family:Arial,sans-serif;color:rgb(13,110,253)">propose</span></a><span =
style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,37,41)"=
>=C2=A0a topic.</span><span style=3D"font-size:10.5pt;font-family:Arial,san=
s-serif"><u></u><u></u></span></li><li class=3D"MsoNormal" style=3D"margin-=
left:15px;color:black;line-height:13.5pt"><span style=3D"font-size:10.5pt;f=
ont-family:Arial,sans-serif;color:rgb(33,37,41)">All participants review th=
e=C2=A0</span><a href=3D"https://design-sessions.xenproject.org/list/discus=
sion" target=3D"_blank"><span style=3D"font-size:10.5pt;font-family:Arial,s=
ans-serif;color:rgb(13,110,253)">list of=C2=A0<span class=3D"gmail-il">sess=
ions</span></span></a><span style=3D"font-size:10.5pt;font-family:Arial,san=
s-serif;color:rgb(33,37,41)">, indicating their interest in attending each =
one.</span><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif"><u=
></u><u></u></span></li><li class=3D"MsoNormal" style=3D"margin-left:15px;c=
olor:black;line-height:13.5pt"><span style=3D"font-size:10.5pt;font-family:=
Arial,sans-serif;color:rgb(33,37,41)">The=C2=A0<span class=3D"gmail-il">ses=
sion</span>=C2=A0scheduler optimizes the=C2=A0</span><a href=3D"https://des=
ign-sessions.xenproject.org/schedule" target=3D"_blank"><span style=3D"font=
-size:10.5pt;font-family:Arial,sans-serif;color:rgb(13,110,253)">schedule</=
span></a><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color=
:rgb(33,37,41)">=C2=A0to accommodate as many preferences as possible.</span=
><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif"><u></u><u></=
u></span></li></ul></td></tr><tr><td style=3D"padding:3.75pt 0in;word-break=
:break-word"><p class=3D"MsoNormal" style=3D"line-height:13.5pt"><span styl=
e=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,37,41)">Par=
ticipants can also propose long-form talks by adding [TALK] to the=C2=A0<sp=
an class=3D"gmail-il">session</span>=C2=A0title.</span><span style=3D"font-=
size:10.5pt;font-family:Arial,sans-serif;color:black"><u></u><u></u></span>=
</p></td></tr><tr><td style=3D"padding:3.75pt 0in;word-break:break-word"><p=
 class=3D"MsoNormal" style=3D"line-height:13.5pt"><span style=3D"font-size:=
10.5pt;font-family:Arial,sans-serif;color:black">For suggested topics, samp=
le=C2=A0<span class=3D"gmail-il">Design</span>=C2=A0<span class=3D"gmail-il=
">Session</span>=C2=A0submissions, and more tips check out the=C2=A0</span>=
<a href=3D"https://events.linuxfoundation.org/xen-project-summit/program/de=
sign-sessions/" target=3D"_blank"><span style=3D"font-size:10.5pt;font-fami=
ly:Arial,sans-serif">Xen=C2=A0<span class=3D"gmail-il">Design</span>=C2=A0<=
span class=3D"gmail-il">Session</span>=C2=A0page</span></a><span style=3D"f=
ont-size:10.5pt;font-family:Arial,sans-serif;color:black">=C2=A0for more in=
formation.<u></u><u></u></span></p></td></tr><tr><td style=3D"padding:3.75p=
t 0in;word-break:break-word"><p class=3D"MsoNormal" style=3D"line-height:13=
.5pt"><span style=3D"font-size:10.5pt;font-family:Arial,sans-serif;color:bl=
ack"><br>Best Regards,<br>Xen Project Events Team<u></u><u></u></span></p><=
/td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></tabl=
e></td></tr></tbody></table></td></tr></tbody></table></div></td></tr></tbo=
dy></table></div></td></tr><tr><td style=3D"padding:0in"><div align=3D"cent=
er"><table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" =
style=3D"width:1051.99px;background:rgb(98,171,48);float:left"><tbody><tr><=
td style=3D"padding:0in"><div align=3D"center"><table border=3D"0" cellspac=
ing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px;float:l=
eft"><tbody><tr><td valign=3D"top" style=3D"padding:0in"><table border=3D"0=
" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" width=3D"100%" style=
=3D"width:1051.99px"><tbody><tr><td style=3D"padding:0in"><table border=3D"=
0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.9=
9px"><tbody><tr><td style=3D"padding:11.25pt"></td></tr></tbody></table></t=
d></tr></tbody></table></td></tr></tbody></table></div></td></tr></tbody></=
table></div></td></tr></tbody></table></div></td></tr></tbody></table><p cl=
ass=3D"MsoNormal"><u></u>=C2=A0<u></u></p><table border=3D"0" cellspacing=
=3D"0" cellpadding=3D"0" width=3D"100%" style=3D"width:1051.99px"><tbody><t=
r><td style=3D"padding:0.75pt"></td></tr></tbody></table></div></div></div>=
</div></div></div><div><div dir=3D"ltr" class=3D"gmail_signature" data-smar=
tmail=3D"gmail_signature"><div dir=3D"ltr"><div>Many thanks,</div><div>Kell=
y Choi</div><div><br></div><div><div style=3D"color:rgb(136,136,136)">Commu=
nity Manager</div><div style=3D"color:rgb(136,136,136)">Xen Project=C2=A0<b=
r></div></div></div></div></div></div>

--000000000000be72de0619302be4--


From xen-devel-bounces@lists.xenproject.org Fri May 24 09:56:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 09:56:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729244.1134387 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARen-0001zc-Ej; Fri, 24 May 2024 09:56:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729244.1134387; Fri, 24 May 2024 09:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sARen-0001zV-Br; Fri, 24 May 2024 09:56:13 +0000
Received: by outflank-mailman (input) for mailman id 729244;
 Fri, 24 May 2024 09:56:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sARem-0001wi-BK
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 09:56:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARel-0006lq-Hl; Fri, 24 May 2024 09:56:11 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sARel-0005Nv-A1; Fri, 24 May 2024 09:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=EEDfjkRt6blO2trxyslNclOy3h3ZXVBkdsotf+FX3lI=; b=ZX2EtO9odUk1nDPpIKW0BWyc56
	9Rg93BrHoNv4kph7p61Q7Y2hA4KMDIkuP+cm/+QXnvPHO8iaSyNu28F9wxj//RDSrUsGUjEA34dSe
	ZrfwVK6muhKssUFvQR7F8r8wwmejBHzTWEGdwVRXNcb45VAOGII6fTsIrPD8Rt0tREx0=;
Message-ID: <a7a0456d-9eac-4db4-8782-ae94c7e338b1@xen.org>
Date: Fri, 24 May 2024 10:56:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 7/7] docs: Add device tree overlay documentation
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Vikram Garhwal <fnu.vikram@xilinx.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>,
 Henry Wang <xin.wang2@amd.com>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop>
 <20240524021814.2666257-7-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524021814.2666257-7-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 03:18, Stefano Stabellini wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>   docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++
>   1 file changed, 82 insertions(+)
>   create mode 100644 docs/misc/arm/overlay.txt
> 
> diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
> new file mode 100644
> index 0000000000..0a2dee951a
> --- /dev/null
> +++ b/docs/misc/arm/overlay.txt
> @@ -0,0 +1,82 @@
> +# Device Tree Overlays support in Xen
> +
> +Xen experimentally supports dynamic device assignment to running
> +domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device
> +tree, and attaching them to a running domain with given $domid.
> +
> +Dynamic node assignment works in two steps:
> +
> +## Add/Remove device tree overlay to/from Xen device tree
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
> +
> +## Attach device from the DT overlay to domain
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor attach the device to the user-provided $domid by
> +   mapping node resources in the DT overlay.
> +
> +# Examples
> +
> +Here are a few examples on how to use it.
> +
> +## Dom0 device add
> +
> +For assigning a device tree overlay to Dom0, user should firstly properly
> +prepare the DT overlay. More information about device tree overlays can be
> +found in [1]. Then, in Dom0, enter the following:
> +
> +    (dom0) xl dt-overlay add overlay.dtbo
> +
> +This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
> +
> +To assign the newly added device from the dtbo to Dom0:
> +
> +    (dom0) xl dt-overlay attach overlay.dtbo 0
> +
> +Next, if the user wants to add the same device tree overlay to dom0
> +Linux, execute the following:
> +
> +    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> +    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> +
> +Finally if needed, the relevant Linux kernel drive can be loaded using:
> +
> +    (dom0) modprobe module_name.ko
> +
> +## DomU device add/remove
> +
> +All the nodes in dtbo will be assigned to a domain; the user will need
> +to prepare the dtb for the domU.

s/dtb/dtbo/? But I am little bit confused with the wording. I think you 
may want to add *different dtbo* so it clarifies from the start (this 
only becomes obvious at the end of the section) that the user is not 
meant to use the same for all the commands.

  For example, the `interrupt-parent`
> +property of the DomU overlay should be changed to the Xen hardcoded
> +value `0xfde8`, and the xen,reg property should be added to specify the
> +address mappings. 
> If xen,reg is not present, it is assumed 1:1 mapping.

Repeating an earlier comment here. I think xen,reg should be mandatory 
for non-direct mapped domain.

Also, can you clarify what is the expect property layout for xen,reg?

> +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
> +
> +For new domains to be created, the user will need to create the DomU
> +with below properties properly configured in the xl config file:
> +- `iomem`

I looked at your reply in v4 and I am afraid I still don't understand 
why we are mentioning 'iomem'. If we want to use the commands below, 
then the domain needs to be created in advance. So you can't yet know 
'iomem'.

You could avoid "xl dt-overlay attach" but then you need the user to 
specify both "irqs" and "iomem". From a user point of view, it would be 
easier to add a new propery in the configuration file listing the 
overlays. Something like:

dt_overlays = [ "overlay.dtbo", ... ]

Anyway, that somewhat separate. For now, I think we want to drop 'iomem' 
from the list and reword this paragraph to say that the 'passthrough' 
property needs to be set if you plan to use DT overlay and devices 
requiring the IOMMU.

> +- `passthrough` (if IOMMU is needed)

This property is required at the start because we don't support enabling 
the IOMMU lazily.

> +
> +User will also need to modprobe the relevant drivers. For already
> +running domains, the user can use the xl dt-overlay attach command,
> +example:
> +
> +    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
> +    (dom0) xl dt-overlay attach overlay.dtbo $domid
> +    (dom0) xl console $domid                         # To access $domid console
> +
> +Next, if the user needs to modify/prepare the overlay.dtbo suitable for
> +the domU:
> +
> +    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> +    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> +
> +Finally, if needed, the relevant Linux kernel drive can be probed:
> +
> +    (domU) modprobe module_name.ko
> +
> +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 10:32:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729293.1134396 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASDV-0001J2-0F; Fri, 24 May 2024 10:32:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729293.1134396; Fri, 24 May 2024 10:32:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASDU-0001Iv-Tf; Fri, 24 May 2024 10:32:04 +0000
Received: by outflank-mailman (input) for mailman id 729293;
 Fri, 24 May 2024 10:32:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sASDT-0001HK-HJ
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:32:03 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id db168bd8-19b8-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 12:32:01 +0200 (CEST)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-355080e6ff9so497269f8f.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:32:01 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a08aba8sm1269932f8f.42.2024.05.24.03.31.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 03:32:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db168bd8-19b8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716546721; x=1717151521; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/j1AH2CW1D72ZHr3W8o3rzzMsGfMwZCkPJJE+Ii7QK8=;
        b=AlTZbcRy8yjW4YWNh41T4CVLrToPhFXsqosbyE4W3cfu0Cg8VXVbqdfCXIPRUK9ZHH
         2UTkHUtTATUcbfcM97Y6xIwrL9Emph6DCCd4NNR1iqbrgqOyswWW/fDt8wa4jWAbzt6P
         RhnwyoeViOb0sRSuq1DU4LYb4qW/hcX++WVG4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716546721; x=1717151521;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/j1AH2CW1D72ZHr3W8o3rzzMsGfMwZCkPJJE+Ii7QK8=;
        b=uDM8Sgu8rlbgAFpRPOgQYa07cSlm9UEv7IPS3DbtnFvoTt7I8ZTCc5QKnXgB3zp/+k
         RGHBc10Q2wyZR149Mg8u0swrwOu6N4q+AWsizG30Bs9ydFyYA4t9P3ieVZXkDg7pq+tz
         bgXpTCrnnxBa3I33089bOuRIrzject8YqpVEfdjdc9aSH6p13bKphl2xJO4coep6lLnc
         e4NALfZUS3GEPDEdmr1EGHZxeL+15sLUTCxuz77gEx00Iu7h0FSkW6VpJub7jJ7BNpDk
         mDNKMW1RtuSTMiSOiPKvUE5MfCzxMPbN1K2FDmqCNHpL2My4MxPlWKu0z0tAqfBubX2k
         ZsPA==
X-Gm-Message-State: AOJu0Yy1oLBRJugl49CDNFY5u4KEiIB4R1kf5iiLBKA3hwa+s48nKu5W
	jS3w7/qbc3saQtw2VLSc5GGBjfST1BEUVXLFZhSmIaMLwNJHkDrOCoNPvDfHfEc=
X-Google-Smtp-Source: AGHT+IFAdmBE26nT1NqSf+apueplaOS+4Wh9gEgmrQZMeYVdmDMTDYFFSGA4t30yjSDOwCshjNFq3A==
X-Received: by 2002:adf:e402:0:b0:34a:2852:f119 with SMTP id ffacd0b85a97d-3552fda1cffmr1100385f8f.36.1716546720766;
        Fri, 24 May 2024 03:32:00 -0700 (PDT)
Message-ID: <cb48ebac-3406-47e3-9323-ad75f487265a@cloud.com>
Date: Fri, 24 May 2024 11:31:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/2] tools/xg: Clean up xend-style overrides for CPU
 policies
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
 <ae180fd606cb9a6aa0108080fe8cf7a6496a4ce3.1716457040.git.alejandro.vallejo@cloud.com>
 <Zk8e1RabTi0YXCgW@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zk8e1RabTi0YXCgW@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 11:47, Roger Pau Monné wrote:
>> -static xen_cpuid_leaf_t *find_leaf(
>> -    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
>> -    const struct xc_xend_cpuid *xend)
>> +static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
>> +                                   const struct xc_xend_cpuid *xend)
>>  {
>>      const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
>>  
>> -    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
>> +    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),
> 
> Don't you need to use p->nr_leaves here, as otherwise we could check
> against possibly uninitialized leaves (or leaves with stale data)?

Indeed. Good catch (same on the MSR side).

>> -        switch ( p->policy.x86_vendor )
>> +        switch ( cur->policy.x86_vendor )
>>          {
>>          case X86_VENDOR_INTEL:
>> -            for ( i = 0; (p->policy.cache.subleaf[i].type &&
>> -                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
>> +            for ( i = 0; (cur->policy.cache.subleaf[i].type &&
>> +                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )
> 
> Nit: indentation is weird here.  I would use:
> 
>             for ( i = 0; cur->policy.cache.subleaf[i].type &&
>                          i < ARRAY_SIZE(cur->policy.cache.raw); ++i )
> 
> Thanks, Roger.

Sure. Leftover from removing the size_t in v2.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 10:32:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:32:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729298.1134406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASEJ-0001op-B1; Fri, 24 May 2024 10:32:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729298.1134406; Fri, 24 May 2024 10:32:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASEJ-0001oi-8N; Fri, 24 May 2024 10:32:55 +0000
Received: by outflank-mailman (input) for mailman id 729298;
 Fri, 24 May 2024 10:32:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sASEI-0001HK-39
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:32:54 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f97dd3e6-19b8-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 12:32:52 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-354e0cc9c5cso2554692f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:32:52 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a0931c9sm1269766f8f.65.2024.05.24.03.32.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 03:32:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f97dd3e6-19b8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716546772; x=1717151572; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4sC4hyVDtv8Dy6ruPjJsuJvbEvg/9VKuCLBxdu6Om/s=;
        b=IPAwYGatLJ3hCN9VKOetL2DgXzVby+d37CI9pa3oyy1+3mYPn5zshS5P+vKYDH8x83
         xalCPaCfl1gKX22zpzEAMbPnYpouaXTdUKxvOn9/s6u7d1dWbxep6INU3eVrhHUsIhC1
         GHtW+hxxg1eV0gCX+gnfgiM3uEPixKWsYOVvg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716546772; x=1717151572;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4sC4hyVDtv8Dy6ruPjJsuJvbEvg/9VKuCLBxdu6Om/s=;
        b=DKXV/bdDNAFVewKY7L4dVaB/qu8DRKe4obarwoP/MXeXyr0wSYpjBC9amHy0fZESMb
         DJWl9SvoadcdZECNL6lIGdNamPfUHsBZHcYL9EroICk4cBh6tiRzLCd2C5vyTcIwFDX+
         5OS04DCY+lVntM7SP2hTvN263gfOJtWuYUBrqp+jpFA9qn/M+Sluw7ySZKTZSA9c48Pw
         GcNVQF7mrP6lAf3kWHO6oh20v9WjUTWwRFIk/FQw21CxSmBCtJ/XM9NML7meFMUh9ESV
         ODqmBmFsO+m81cmEtSDKCbL0Iq9y0e7lK0WB/FuAXC/UNk4YS19CypoRjeklNThXvDem
         HkXw==
X-Gm-Message-State: AOJu0Yytu7i4yC7ZKxy8QWR4rYB1JDG4PM8MxyR/hJ8sB3xEQY3oX5Ow
	g3/5sN5HaeDdGHo26jM7EKIx18AqPvlrZL8WhWoQD183/I6F6//ZXJyyPNeS7uQ=
X-Google-Smtp-Source: AGHT+IF8Lbrj01EXtzdBE+xuUilIfxojctlcogSMfV5y6PYVLJqta20UIWoEe39z6tZvVBiybN2AAw==
X-Received: by 2002:adf:ce07:0:b0:354:f286:4f0f with SMTP id ffacd0b85a97d-35526c3a2e7mr1284687f8f.30.1716546771785;
        Fri, 24 May 2024 03:32:51 -0700 (PDT)
Message-ID: <de256e4b-fdc4-48a8-9173-38092ab90716@cloud.com>
Date: Fri, 24 May 2024 11:32:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
 <5c6ee74b60bad4eb9cc8e17dbfcf7158d38bd32c.1716457040.git.alejandro.vallejo@cloud.com>
 <Zk8YucyA1ZwGU4Y-@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zk8YucyA1ZwGU4Y-@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 11:21, Roger Pau Monné wrote:
> On Thu, May 23, 2024 at 10:41:29AM +0100, Alejandro Vallejo wrote:
>> The idea is to use xc_cpu_policy_t as a single object containing both the
>> serialised and deserialised forms of the policy. Note that we need lengths
>> for the arrays, as the serialised policies may be shorter than the array
>> capacities.
>>
>> * Add the serialised lengths to the struct so we can distinguish
>>   between length and capacity of the serialisation buffers.
>> * Remove explicit buffer+lengths in serialise/deserialise calls
>>   and use the internal buffer inside xc_cpu_policy_t instead.
>> * Refactor everything to use the new serialisation functions.
>> * Remove redundant serialization calls and avoid allocating dynamic
>>   memory aside from the policy objects in xen-cpuid. Also minor cleanup
>>   in the policy print call sites.
>>
>> No functional change intended.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> 
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Just two comments.
> 
>> ---
>> v3:
>>   * Better context scoping in xg_sr_common_x86.
>>     * Can't be const because write_record() takes non-const.
>>   * Adjusted line length of xen-cpuid's print_policy.
>>   * Adjusted error messages in xen-cpuid's print_policy.
>>   * Reverted removal of overscoped loop indices.
>> ---
>>  tools/include/xenguest.h            |  8 ++-
>>  tools/libs/guest/xg_cpuid_x86.c     | 98 ++++++++++++++++++++---------
>>  tools/libs/guest/xg_private.h       |  2 +
>>  tools/libs/guest/xg_sr_common_x86.c | 56 ++++++-----------
>>  tools/misc/xen-cpuid.c              | 41 ++++--------
>>  5 files changed, 106 insertions(+), 99 deletions(-)
>>
>> diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
>> index e01f494b772a..563811cd8dde 100644
>> --- a/tools/include/xenguest.h
>> +++ b/tools/include/xenguest.h
>> @@ -799,14 +799,16 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>>                               xc_cpu_policy_t *policy);
>>  
>>  /* Manipulate a policy via architectural representations. */
>> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
>> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
>> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
>> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
>>  int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
>>                                 const xen_cpuid_leaf_t *leaves,
>>                                 uint32_t nr);
>>  int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>>                                const xen_msr_entry_t *msrs, uint32_t nr);
>> +int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
>> +                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
>> +int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
>> +                           const xen_msr_entry_t **msrs, uint32_t *nr);
> 
> Maybe it would be helpful to have a comment clarifying that the return
> of xc_cpu_policy_get_{leaves,msrs}() is a reference to the content of
> the policy, not a copy of it (and hence is tied to the lifetime of
> policy, and doesn't require explicit freeing).

Sure.

> 
>>  
>>  /* Compatibility calculations. */
>>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
>> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
>> index 4453178100ad..4f4b86b59470 100644
>> --- a/tools/libs/guest/xg_cpuid_x86.c
>> +++ b/tools/libs/guest/xg_cpuid_x86.c
>> @@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
>>      }
>>  }
>>  
>> -static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
>> -                              unsigned int nr_leaves, unsigned int nr_entries)
>> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
>>  {
>>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>>      int rc;
>>  
>>      rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
>> -                                    nr_leaves, &err_leaf, &err_subleaf);
>> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
>>      if ( rc )
>>      {
>>          if ( err_leaf != -1 )
>> @@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
>>      }
>>  
>>      rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
>> -                                  nr_entries, &err_msr);
>> +                                  policy->nr_msrs, &err_msr);
>>      if ( rc )
>>      {
>>          if ( err_msr != -1 )
>> @@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
>>          return rc;
>>      }
>>  
>> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
>> +    policy->nr_leaves = nr_leaves;
>> +    policy->nr_msrs = nr_msrs;
>> +
>> +    rc = deserialize_policy(xch, policy);
>>      if ( rc )
>>      {
>>          errno = -rc;
>> @@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
>>          return rc;
>>      }
>>  
>> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
>> +    policy->nr_leaves = nr_leaves;
>> +    policy->nr_msrs = nr_msrs;
>> +
>> +    rc = deserialize_policy(xch, policy);
>>      if ( rc )
>>      {
>>          errno = -rc;
>> @@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>>                               xc_cpu_policy_t *policy)
>>  {
>>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>> -    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
>> -    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
>>      int rc;
>>  
>> -    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
>> -                                 policy->msrs, &nr_msrs);
>> +    rc = xc_cpu_policy_serialise(xch, policy);
>>      if ( rc )
>>          return rc;
>>  
>> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
>> -                                  nr_msrs, policy->msrs,
>> +    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
>> +                                  policy->nr_msrs, policy->msrs,
>>                                    &err_leaf, &err_subleaf, &err_msr);
>>      if ( rc )
>>      {
>> @@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>>      return rc;
>>  }
>>  
>> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
>> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
>> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
>> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
>>  {
>> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
>> +    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
>>      int rc;
>>  
>> -    if ( leaves )
>> +    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
>> +    if ( rc )
>>      {
>> -        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
>> -        if ( rc )
>> -        {
>> -            ERROR("Failed to serialize CPUID policy");
>> -            errno = -rc;
>> -            return -1;
>> -        }
>> +        ERROR("Failed to serialize CPUID policy");
>> +        errno = -rc;
>> +        return -1;
>>      }
>>  
>> -    if ( msrs )
>> +    p->nr_leaves = nr_leaves;
> 
> Nit: FWIW, I think you could avoid having to introduce local
> nr_{leaves,msrs} variables and just use p->nr_{leaves,msrs}?  By
> setting them to ARRAY_SIZE() at the top of the function and then
> letting x86_{cpuid,msr}_copy_to_buffer() adjust as necessary.
> 
> Thanks, Roger.

The intent was to avoid mutating the policy object in the error cases
during deserialise. Then I adjusted the serialise case to have symmetry.

It's true the preservation is not meaningful in the serialise case
because at that point the serialised form is already corrupted.

I don't mind either way. Seeing how I'm sending one final version with
the comments of patch2 I'll just adjust as you proposed.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 10:52:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:52:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729311.1134427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWn-0005vc-WE; Fri, 24 May 2024 10:52:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729311.1134427; Fri, 24 May 2024 10:52:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWn-0005vS-T2; Fri, 24 May 2024 10:52:01 +0000
Received: by outflank-mailman (input) for mailman id 729311;
 Fri, 24 May 2024 10:52:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lVY=M3=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sASWn-0005h8-5G
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:52:01 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a55713ff-19bb-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 12:51:59 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e95abc7259so6768751fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:51:59 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bcf4c32sm1421121fa.66.2024.05.24.03.51.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 03:51:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a55713ff-19bb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716547919; x=1717152719; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=Tg3FNMBboZx20BiYsPxa1SM6q5Vk4G46zzScjpLP00Uqa5Xw8JFsocs6SpaboyuE8K
         x5LSBxJ05zW32ttuLuYZ8X8rO7kth3g4BqjS2QUhjz9/k1emvqCUDtWRTmzKtwoJSKsn
         p9yHOGLAze1OWuehrp+z8LQJ2/5YR+fqiani+hbfQMPkhd7P9dsmmW9BW8lYav9O9ECr
         T/jYD/76s30Em5nSLAF3zgEWjzzSEQHlasRCnLRdEEFhknsCkwBLzgTn7MOT0yPTPJxn
         ePozly39GugDYgSfHGLOoX0ZCUKxuJQoZMi8DRrilElyzwry8euM4q0Mhg+rHkePZdNy
         AORQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716547919; x=1717152719;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=fN9ym5fCjHRLOSH9dQk7lE5m/6/OkwME3hH9o79RHsGTCD0MbzZRzBW8JfAmauO3Ui
         j0A0GXpaz3x7mDxPUsbym67ihEJc4SUE52H379yUSff78KjF09LkS1+WlCYR1H6s0LAz
         ODddyE+ry5YnRgB3wiFFKc9+jJZ2lM7lw60RCh+gMCjbjbRiWCy+KxEPwS7waF5G4z1f
         0ubBa67dg9s22Zi/Uz1+8Mr3lyJaY6Dk1lua1wsYWn5KNTuL/mF7eJSQV+G+FBugsStp
         NLhDrHLukVPUn1cKqcTr5cCG5hqgW4B57wXj61gv+8TnyLSU7b6NmX0XXOGPUok/dpse
         rdiA==
X-Forwarded-Encrypted: i=1; AJvYcCXodxwlhHdV7HpjqiJZgVb/kPpGSuGf/EkVpLIXajx/OndWBhco47CMTt9+sT0+ybp9GV0zsSX02jORvv3BXHkhpKVby22TPGuZKkBB0Ss=
X-Gm-Message-State: AOJu0Yz17MvWknYm2bSFxrQptGIGGIpvWElIvj51+1+RpsKgWZ3qUwQi
	R88ABImeL12PDAu5WQQF2eBRT+Ke4mg5829I+1pzSTzf/kbFTz+z
X-Google-Smtp-Source: AGHT+IG7/58L7fP8XfkA17hx/P2HA+GooUSMBu7K0MzZTbJxr4odywyDF44eC1SlJ9lDqF4xGtjtyw==
X-Received: by 2002:a2e:8417:0:b0:2e5:751d:4b0f with SMTP id 38308e7fff4ca-2e95b2822ebmr10863631fa.34.1716547918939;
        Fri, 24 May 2024 03:51:58 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v7 2/8] xen: mapcache: Unmap first entries in buckets
Date: Fri, 24 May 2024 12:51:46 +0200
Message-Id: <20240524105152.1301842-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
         return;
     }
     entry->lock--;
-    if (entry->lock > 0 || pentry == NULL) {
+    if (entry->lock > 0) {
         return;
     }
 
-    pentry->next = entry->next;
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
     if (munmap(entry->vaddr_base, entry->size) != 0) {
         perror("unmap fails");
         exit(-1);
     }
+
     g_free(entry->valid_mapping);
-    g_free(entry);
+    if (pentry) {
+        pentry->next = entry->next;
+        g_free(entry);
+    } else {
+        memset(entry, 0, sizeof *entry);
+    }
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 10:52:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:52:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729312.1134437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWp-0006A7-6v; Fri, 24 May 2024 10:52:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729312.1134437; Fri, 24 May 2024 10:52:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWp-0006A0-42; Fri, 24 May 2024 10:52:03 +0000
Received: by outflank-mailman (input) for mailman id 729312;
 Fri, 24 May 2024 10:52:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lVY=M3=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sASWo-0005va-7b
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:52:02 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a6794cf7-19bb-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 12:52:01 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2e95a7545bdso8103881fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:52:01 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95be00a9esm1417431fa.103.2024.05.24.03.51.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 03:51:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6794cf7-19bb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716547921; x=1717152721; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YP28Iks2aCd2MOvUAbQc4yYuz/leCz2VtuKFu0fMfns=;
        b=Wl4j6IP6/aHZ0ifQHdheTEn2PWngmtcVEywDgaBYPaEqm+g8RSgs2o0BC0A2bWNOnZ
         tlysKLtQA7gFr+scuWAYicl5i1AVFnUDiW7hS+0EJVHSB8PqqRTsyfRP++BvVQPYwZ67
         33VVXtFJX81rnzXeVZ4xoPch0ZedFMw+9ngfqi9UyITAq792g1k/+YTjXRdp6wEwKEBi
         xsh9PnesERgcpz/FK/kwm18ZZH3VPQbCI1xp/RkdFlfHcApaGJowGAd5r5kNy9NyE/Vy
         4wDTYOzEXOzOJPCxIDxYQymHTpqMHyHO42RvY3fGzFqL+eafUBIs13EMZPKGTwgLFrvd
         Pz4g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716547921; x=1717152721;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YP28Iks2aCd2MOvUAbQc4yYuz/leCz2VtuKFu0fMfns=;
        b=fZXfhN4GP1MxoipY23lKgTSI5BENqQDpCA/MYD5jIG3UsOtlZ910HninnRIf9egpOs
         qrVuk1q1Vtp0oKuBzqLJoAxM+mTtj1nFfhCePNbNnqdb2f4SVTgCt5bIldXwOL0McMQ0
         yGePB0Yn1GFAenSBVfRjx/x/Ae8BMpi18Mf9gCDmL8vCzyWmCLrxMzavl0+awx512Gj/
         XK5rFvNpClBIfzrlZyIvaWof5oGObdx+CMfT7gLN321gocGOdDu3WttzK4upJT9Z23H9
         UXN+hh3m27VIefosxPIp2UEKOc69WRo/6VXr4B6KP5z+7tbRGt8/tzwB4sYkNpWptcJK
         l4Nw==
X-Forwarded-Encrypted: i=1; AJvYcCWuFZpVxQ1iBZjP1In0AB9zOsAWF/dyhla6K0Td2QkzakQq0zYVBHGrwhZdy0MJKj6+NV/+4IU+rseL1Vcru3U4zpXkuUTJUHM3/HQ4S4U=
X-Gm-Message-State: AOJu0YyiPTWZA5p7adDfqva5KCdWX44rzf67ewSOqgKHYn2Y0TgLpIwD
	6NoiIgBRgHSIulmpOp6d81IpLnHep8HLBUt80JDE5UlGvbpqg0gf
X-Google-Smtp-Source: AGHT+IHxf48OGn9vlQXF6Pl1G+8i2NXaRVtBjeCMc7C8+mcWr77Zr/it24OxvWZYA9xQzmTZ7G5dGA==
X-Received: by 2002:a2e:6802:0:b0:2e5:67a8:2e6b with SMTP id 38308e7fff4ca-2e95b04115amr13608211fa.8.1716547920962;
        Fri, 24 May 2024 03:52:00 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v7 3/8] xen: Add xen_mr_is_memory()
Date: Fri, 24 May 2024 12:51:47 +0200
Message-Id: <20240524105152.1301842-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
---
 hw/xen/xen-hvm-common.c | 10 ++++++++--
 include/sysemu/xen.h    |  8 ++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    return mr == &xen_memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
                    Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
         return;
     }
 
-    if (mr == &xen_memory) {
+    if (xen_mr_is_memory(mr)) {
         return;
     }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
     XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-    if (section->mr == &xen_memory) {
+    if (xen_mr_is_memory(section->mr)) {
         return;
     } else {
         if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
     g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 10:52:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:52:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729310.1134418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWm-0005hL-QW; Fri, 24 May 2024 10:52:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729310.1134418; Fri, 24 May 2024 10:52:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWm-0005hE-M4; Fri, 24 May 2024 10:52:00 +0000
Received: by outflank-mailman (input) for mailman id 729310;
 Fri, 24 May 2024 10:51:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lVY=M3=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sASWl-0005h8-KC
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:51:59 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a3e90c24-19bb-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 12:51:57 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e6f51f9de4so100058001fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:51:57 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bcd7c20sm1491561fa.33.2024.05.24.03.51.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 03:51:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3e90c24-19bb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716547916; x=1717152716; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=dQHWug0UlN40N6YIbgnyJtYw6nyjlpIX2Srd7N+YUqHdNXQdbrH7S12OwLtQJJLe2T
         XmQRLNgf0IpHPvUVlP3XAGjLm37pfjyAWSsgPAOGLIHl11Z/1juo7i8jXh0PqpkOkww/
         BmhacJkobhIK2mZtIFTqSdvbwlanNwX8Pe052Q5Hl1DUV0wpDndRUANd1w+ZZbT2nbFf
         Oy/asv5LBx+brESTw9j+z2fLLADmqOgZ/ad9qx5n6v0NqrbVg5AOWNSmFj+nEY7D2MBe
         xsHHRXq3Jss4iITLE2UoQBr6gTEwAv5EdR55kG9/WJhLPM9weeJRR+YGTjRqdZN7IDSL
         4Vow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716547916; x=1717152716;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=GwyqCGjIFGy4F5T13OtaTp+rmp2/H+weqHpamZN4tGomll4vmMSJ/JSm6Cp6DNkKkB
         U8fObKoyeERkJ7h3TzQuSW2Nj+c/M2x/Fw0TeptRG/ckxgLUkwpRrTi0/nqZXt+RRZpX
         EjjyGD5uw62PD6zLuV3RtxgmXoUnxH6zMGCgwzKQApG9vqD/a5pNEygBpqxnyR4yyv5z
         SmB91gUb6HyQe4H6Q9ORpAtDPZJtpCqjhpvMIQh23K5xykMSsad8iJYhCWZxlYOUi1Dw
         5HPl1p+X5OdsYxKePkbw4KJKcYq4CQaqh79IfCkqqauTQNVV/dxModYtytcaQ9ScA8wm
         XvGQ==
X-Forwarded-Encrypted: i=1; AJvYcCWZMFzThER9vTRX30lgcVaX087c5gfSxKhOJziaG6Dpz3P1UqF9n5nl19/kDIiwzA61zn0NpsMMqdkczMVxHMQ5V2WAWdB5HVkucw2k+wY=
X-Gm-Message-State: AOJu0YxRUxLK1qYML5HbqelcFxTwC9JMLBvuuuStNNuYoUVD0UYwVQPQ
	7k9d0yt93j51g1T/zW5QfP11YhwL+phBERq1D/lf1FSXnIjf/LunHUNTlH+R
X-Google-Smtp-Source: AGHT+IEGg/j9kr5jKj/tv+syV4xN0QPHZQ3VcNTdBVPyyQs2WD0/L2C8QLd0gT/jlHUTsM+RTiWqvg==
X-Received: by 2002:a2e:b784:0:b0:2e0:6313:fe3a with SMTP id 38308e7fff4ca-2e95b256875mr16230641fa.35.1716547916526;
        Fri, 24 May 2024 03:51:56 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v7 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
Date: Fri, 24 May 2024 12:51:45 +0200
Message-Id: <20240524105152.1301842-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 54 ++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE     (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
     /* For most cases (>99.9%), the page address is the same. */
     MapCacheEntry *last_entry;
     unsigned long max_mcache_size;
-    unsigned int mcache_bucket_shift;
+    unsigned int bucket_shift;
+    unsigned long bucket_size;
 
     phys_offset_to_gaddr_t phys_offset_to_gaddr;
     QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
                                            void *opaque,
+                                           unsigned int bucket_shift,
                                            unsigned long max_size)
 {
     unsigned long size;
     MapCache *mc;
 
+    assert(bucket_shift >= XC_PAGE_SHIFT);
+
     mc = g_new0(MapCache, 1);
 
     mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
     QTAILQ_INIT(&mc->locked_entries);
 
+    mc->bucket_shift = bucket_shift;
+    mc->bucket_size = 1UL << bucket_shift;
     mc->max_mcache_size = max_size;
 
     mc->nr_buckets =
         (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
-         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+          (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+         (bucket_shift - XC_PAGE_SHIFT));
 
     size = mc->nr_buckets * sizeof(MapCacheEntry);
     size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
 {
     struct rlimit rlimit_as;
     unsigned long max_mcache_size;
+    unsigned int bucket_shift;
+
+    if (HOST_LONG_BITS == 32) {
+        bucket_shift = 16;
+    } else {
+        bucket_shift = 20;
+    }
 
     if (geteuid() == 0) {
         rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
         }
     }
 
-    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+    mapcache = xen_map_cache_init_single(f, opaque,
+                                         bucket_shift,
+                                         max_mcache_size);
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
     entry->valid_mapping = NULL;
 
     for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
     }
 
     /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
     bool dummy = false;
 
 tryagain:
-    address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = phys_addr >> mc->bucket_shift;
+    address_offset = phys_addr & (mc->bucket_size - 1);
 
     trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
         return mc->last_entry->vaddr_base + address_offset;
     }
 
-    /* size is always a multiple of MCACHE_BUCKET_SIZE */
+    /* size is always a multiple of mc->bucket_size */
     if (size) {
         cache_size = size + address_offset;
-        if (cache_size % MCACHE_BUCKET_SIZE) {
-            cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+        if (cache_size % mc->bucket_size) {
+            cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
         }
     } else {
-        cache_size = MCACHE_BUCKET_SIZE;
+        cache_size = mc->bucket_size;
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
         trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
         raddr = RAM_ADDR_INVALID;
     } else {
-        raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+        raddr = (reventry->paddr_index << mc->bucket_shift) +
              ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
     }
     mapcache_unlock(mc);
@@ -585,8 +597,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     hwaddr address_index, address_offset;
     hwaddr test_bit_size, cache_size = size;
 
-    address_index  = old_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = old_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = old_phys_addr >> mc->bucket_shift;
+    address_offset = old_phys_addr & (mc->bucket_size - 1);
 
     assert(size);
     /* test_bit_size is always a multiple of XC_PAGE_SIZE */
@@ -595,8 +607,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         test_bit_size += XC_PAGE_SIZE - (test_bit_size % XC_PAGE_SIZE);
     }
     cache_size = size + address_offset;
-    if (cache_size % MCACHE_BUCKET_SIZE) {
-        cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+    if (cache_size % mc->bucket_size) {
+        cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -609,8 +621,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
-    address_index  = new_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = new_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = new_phys_addr >> mc->bucket_shift;
+    address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 10:52:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:52:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729313.1134446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWu-0006Vj-I9; Fri, 24 May 2024 10:52:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729313.1134446; Fri, 24 May 2024 10:52:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWu-0006Vc-FY; Fri, 24 May 2024 10:52:08 +0000
Received: by outflank-mailman (input) for mailman id 729313;
 Fri, 24 May 2024 10:52:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lVY=M3=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sASWs-0005va-Vm
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:52:06 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a962440f-19bb-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 12:52:06 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-52327368e59so9717754e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:52:06 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296e887b1asm152787e87.32.2024.05.24.03.52.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 03:52:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a962440f-19bb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716547926; x=1717152726; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GFr6EaV5waf3qgpK3N8zYV0AbUxb9xkgU9bBgeST51Y=;
        b=P3AaTpJ52DVX+7ywX6EwtgmL0B3zaO91fBBJ4+IxAspAorXYzTu/HYfHkuvIxQz9HY
         d6jXlbphUk9Wckchxh1dzRWnS3f0P4jwVq1aVIGRBpf5x+hvaef/my/RIvZ4uCdJ5c55
         9e8hWqDZdNGCXydERQWz79woop0dEo+XN1tHHha8OsddYmG4FDLdcjF23UGCq6g0jNzI
         gVFeRpsJWDiawCSsoo28OQ3krSYhfVPkG9RTXq8iWywNF6fYgyO5KNfeI0+cXWyTmrqE
         zIdAqbMtDB+sCI0/FrtGz3xh0si5O+F2A21PthRFeHKIzZhELKFCxQ7OV0MbFcjECpes
         aBzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716547926; x=1717152726;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GFr6EaV5waf3qgpK3N8zYV0AbUxb9xkgU9bBgeST51Y=;
        b=Q349NR6434sQhL3nL7bnQ5QTXZFhr5M5sHNa45y7SOAtERErNruegkd5oDYEdT1wnT
         UTYlbv5xXxunYvD9lkWqsG1fwIhRYN/HFYe+/k34S46QAwfY1WNP1fxuXg+Cyu/DnYKL
         fkuxt2DfQ1IqREjGwsJv6MEoa0cV3jh85wDHUvy83Dxo9M4Cxl9x18BvUVKjuf7LTazm
         q6Fqu8a4w3Eme3z8+RGOvFXdgMOQrlaBnsr7vxPycUwGzjD6vuVwvpZL9Xux2/DBER2k
         WbiSdJUyWqDgzxFg4wVW4Wl6VfEeutzaMbh/zit/0VsU8qkMIIs6g3s2jZwvt6ED4NOr
         X1sA==
X-Forwarded-Encrypted: i=1; AJvYcCV1VJhjJlESgs6YhEIDWtLRMYtmxU4W2Dp3GB/jMjPW2nEypQngCINFbAkYIjfuwXyXVtdrqzlaTiq360XZ9knSu0v1gJ9MA3YQD8kw+C0=
X-Gm-Message-State: AOJu0YwR8BzOCC5tLJhMlVFG+SUBM5z7U6radr7gdK108Iu8UwuL6ef5
	Ugc0F2xjMhUeF/vXsf6gtabHr+6IAA5+xb28Zo+7A++UWjzVaevy
X-Google-Smtp-Source: AGHT+IFH/GXyxK3UUwsYBMOqKtKS+ki8gwKfJtjwB628taZMcmokUZth9SWriOw8oaOj3H6sRigelA==
X-Received: by 2002:a19:9148:0:b0:523:bbf5:4b36 with SMTP id 2adb3069b0e04-529646df136mr834902e87.20.1716547925723;
        Fri, 24 May 2024 03:52:05 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v7 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()
Date: Fri, 24 May 2024 12:51:50 +0200
Message-Id: <20240524105152.1301842-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c         | 16 +++++++++++-----
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c              |  9 +++++----
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
                              void *vaddr,
                              hwaddr size,
                              hwaddr address_index,
-                             bool dummy)
+                             bool dummy,
+                             ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
     xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
+                                       ram_addr_t ram_offset,
                                        uint8_t lock, bool dma, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
     if (!entry) {
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
-        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                         ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
                 !test_bits(address_offset >> XC_PAGE_SHIFT,
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
-            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                             ram_offset);
         }
     }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
                        hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
     uint8_t *p;
 
     mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+                               lock, dma, is_write);
     mapcache_unlock(mapcache);
     return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false);
+                     cache_size, address_index, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
                         void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
                                      hwaddr phys_addr,
                                      hwaddr size,
+                                     ram_addr_t ram_addr_offset,
                                      uint8_t lock,
                                      bool dma,
                                      bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
          */
         if (xen_mr_is_memory(block->mr)) {
             return xen_map_cache(block->mr, block->offset + addr,
-                                 len, lock, lock,
-                                 is_write);
+                                 len, block->offset,
+                                 lock, lock, is_write);
         }
 
         block->host = xen_map_cache(block->mr, block->offset,
-                                    block->max_length, 1,
-                                    lock, is_write);
+                                    block->max_length,
+                                    block->offset,
+                                    1, lock, is_write);
     }
 
     return ramblock_ptr(block, addr);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 10:52:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:52:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729314.1134457 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWx-0006p6-QA; Fri, 24 May 2024 10:52:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729314.1134457; Fri, 24 May 2024 10:52:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASWx-0006ow-Mk; Fri, 24 May 2024 10:52:11 +0000
Received: by outflank-mailman (input) for mailman id 729314;
 Fri, 24 May 2024 10:52:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lVY=M3=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sASWv-0005h8-Oz
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:52:09 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aa3e8c9b-19bb-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 12:52:07 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-5241b49c0daso7005963e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:52:07 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296e887a61sm154331e87.62.2024.05.24.03.52.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 03:52:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa3e8c9b-19bb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716547927; x=1717152727; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3di7O7tdApxOVr1CI1JAQSoCPPyk5/k7Mjm3E5eqE1c=;
        b=jN3z9wtM4t82GsLPogJgNzy7eWWyvkz+/hIKpThBnhfJtnWPx9JNuZOis7W6c+tOOi
         tvFXigdcx9p/moZvnwgtOPsX4HF8BHKTW9FK8myrMsxaAcwoXvXpr7J6lhb1b0ALoB/s
         3ZheCfrRTywd2PWUlwa4TZxChUmUP/+xIBCJfRXnXkVpLaz5oGkKZiZiX43jLve9lCmK
         E2M/UJvp27ks8Xc/9yz6BXem8YEp3BQjM6YdTjc3rlNrXARb+r0M8aiNeKi2QcIzC36D
         JH45V+3mMi2ku/fFUOjFVQ1GbVjUFCMdFTtq9IaTDK5WvZq7KkeOvOnSKMoymUslsoxv
         WNJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716547927; x=1717152727;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3di7O7tdApxOVr1CI1JAQSoCPPyk5/k7Mjm3E5eqE1c=;
        b=f3rN9SLLAO0r14lm005c15EmBxEIdTg6kiMIa0ytlWms/6H9UpF0MnlFon08xCozJa
         uip+3nsxiIA8XNI5nourw5eptIC9Wts0gzKLsOZgZtqRpM4uPeLFQ69saLqtGmwLONO5
         TJI2loSwJMaHRMSnLKUVmyRuPfIbJ12MGAw5Kfl7LFUxuj1cGT1NSkf2Wiap+iVik4cC
         xdM044Pk6JJ+yFd/fhF1Lizj9hEQK+3E2kiQJbDFh1MIqK31Ie5bweJ+3PLheQP6Ufux
         qyYA8wtmgxwugP1+H4Ja/0c2+5HtgcVsCQrnHzfWK8o4dl2Dw299MZxpqoLtEyQW0aHK
         s2wA==
X-Forwarded-Encrypted: i=1; AJvYcCXpeDemqxzpDRBIfZX3Tk/ksnVl9jI7NZb82AVAnsTMXKDo6UIk0UnqTiGCDHSQ7Zt/rFSQHS2ZqxELVnkuFxRrRdgYsIcIUpZ8RXz0VOg=
X-Gm-Message-State: AOJu0YwaYC/hzoargSP4kAyyNTq4Gnc2l1dgFY4Cwingrq4tXYrQfd/V
	5h9uMF3z34OwJ5KgOtKuCX4/o8YZl23M10Hl61ljYvqZ4CKswWJ1
X-Google-Smtp-Source: AGHT+IE5CtpteyyNazgBiJLIGSkabCL02X/CfF2Skfk8qN39LOCmJrRorp9B6cUUA2WoRJaRtcGV5Q==
X-Received: by 2002:ac2:5932:0:b0:521:b1ca:9c99 with SMTP id 2adb3069b0e04-529645e2556mr1097227e87.5.1716547927180;
        Fri, 24 May 2024 03:52:07 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Manos Pitsidianakis <manos.pitsidianakis@linaro.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v7 7/8] xen: mapcache: Add support for grant mappings
Date: Fri, 24 May 2024 12:51:51 +0200
Message-Id: <20240524105152.1301842-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

CC: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-hvm-common.c         |  12 ++-
 hw/xen/xen-mapcache.c           | 165 +++++++++++++++++++++++++-------
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h            |   7 ++
 4 files changed, 150 insertions(+), 37 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-    return mr == &xen_memory;
+    return mr == &xen_memory || mr == &xen_grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    return mr == &xen_grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..5f23b0adbe 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include <sys/resource.h>
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include <xenevtchn.h>
+#include <xengnttab.h>
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
     unsigned long *valid_mapping;
     uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
     uint8_t flags;
     hwaddr size;
     struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     unsigned long max_mcache_size;
     unsigned int bucket_shift;
 
+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
+    if (xen_region_gnttabdev == NULL) {
+        error_report("mapcache: Failed to open gnttab device");
+        exit(EXIT_FAILURE);
+    }
+
     if (HOST_LONG_BITS == 32) {
         bucket_shift = 16;
     } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     mapcache = xen_map_cache_init_single(f, opaque,
                                          bucket_shift,
                                          max_mcache_size);
+
+    /*
+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+     * map anything beyond the number of pages granted to us.
+     */
+    mapcache_grants = xen_map_cache_init_single(f, opaque,
+                                                XC_PAGE_SHIFT,
+                                                max_mcache_size);
+
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
                              hwaddr size,
                              hwaddr address_index,
                              bool dummy,
+                             bool grant,
+                             bool is_write,
                              ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
-    xen_pfn_t *pfns;
-    int *err;
+    g_autofree uint32_t *refs = NULL;
+    g_autofree xen_pfn_t *pfns = NULL;
+    g_autofree int *err;
     unsigned int i;
     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
     trace_xen_remap_bucket(address_index);
 
-    pfns = g_new0(xen_pfn_t, nb_pfn);
+    if (grant) {
+        refs = g_new0(uint32_t, nb_pfn);
+    } else {
+        pfns = g_new0(xen_pfn_t, nb_pfn);
+    }
     err = g_new0(int, nb_pfn);
 
     if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
     g_free(entry->valid_mapping);
     entry->valid_mapping = NULL;
 
-    for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+    if (grant) {
+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+        for (i = 0; i < nb_pfn; i++) {
+            refs[i] = grant_base + i;
+        }
+    } else {
+        for (i = 0; i < nb_pfn; i++) {
+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+        }
     }
 
-    /*
-     * If the caller has requested the mapping at a specific address use
-     * MAP_FIXED to make sure it's honored.
-     */
+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
+
     if (!dummy) {
-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
-                                           PROT_READ | PROT_WRITE,
-                                           vaddr ? MAP_FIXED : 0,
-                                           nb_pfn, pfns, err);
+        if (grant) {
+            int prot = PROT_READ;
+
+            if (is_write) {
+                prot |= PROT_WRITE;
+            }
+
+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
+            assert(vaddr == NULL);
+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
+                                                         nb_pfn,
+                                                         xen_domid, refs,
+                                                         prot);
+        } else {
+            /*
+             * If the caller has requested the mapping at a specific address use
+             * MAP_FIXED to make sure it's honored.
+             *
+             * We don't yet support upgrading mappings from RO to RW, to handle
+             * models using ordinary address_space_rw(), foreign mappings ignore
+             * is_write and are always mapped RW.
+             */
+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
+                                               PROT_READ | PROT_WRITE,
+                                               vaddr ? MAP_FIXED : 0,
+                                               nb_pfn, pfns, err);
+        }
         if (vaddr_base == NULL) {
-            perror("xenforeignmemory_map2");
+            perror(grant ? "xengnttab_map_domain_grant_refs"
+                           : "xenforeignmemory_map2");
             exit(-1);
         }
     } else {
@@ -260,15 +318,13 @@ static void xen_remap_bucket(MapCache *mc,
             bitmap_set(entry->valid_mapping, i, 1);
         }
     }
-
-    g_free(pfns);
-    g_free(err);
 }
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
                                        ram_addr_t ram_offset,
-                                       uint8_t lock, bool dma, bool is_write)
+                                       uint8_t lock, bool dma,
+                                       bool grant, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
                   *free_entry = NULL, *free_pentry = NULL;
@@ -340,7 +396,7 @@ tryagain:
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                         ram_offset);
+                         grant, is_write, ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
@@ -348,7 +404,7 @@ tryagain:
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                             ram_offset);
+                             grant, is_write, ram_offset);
         }
     }
 
@@ -399,12 +455,26 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
+    bool grant = xen_mr_is_grants(mr);
+    MapCache *mc = grant ? mapcache_grants : mapcache;
     uint8_t *p;
 
-    mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
-                               lock, dma, is_write);
-    mapcache_unlock(mapcache);
+    if (grant && !lock) {
+        /*
+         * Grants are only supported via address_space_map(). Anything
+         * else is considered a user/guest error.
+         *
+         * QEMU generally doesn't expect these mappings to ever fail, so
+         * if this happens we report an error message and abort().
+         */
+        error_report("Tried to access a grant reference without mapping it.");
+        abort();
+    }
+
+    mapcache_lock(mc);
+    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
+                               lock, dma, grant, is_write);
+    mapcache_unlock(mc);
     return p;
 }
 
@@ -449,7 +519,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 {
-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    ram_addr_t addr;
+
+    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    if (addr == RAM_ADDR_INVALID) {
+        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
+    }
+
+    return addr;
 }
 
 static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
@@ -460,6 +537,7 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     hwaddr paddr_index;
     hwaddr size;
     int found = 0;
+    int rc;
 
     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
         if (reventry->vaddr_req == buffer) {
@@ -502,7 +580,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     }
 
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
-    if (munmap(entry->vaddr_base, entry->size) != 0) {
+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
+        rc = xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
+                             entry->size >> mc->bucket_shift);
+    } else {
+        rc = munmap(entry->vaddr_base, entry->size);
+    }
+
+    if (rc) {
         perror("unmap fails");
         exit(-1);
     }
@@ -521,14 +606,24 @@ typedef struct XenMapCacheData {
     uint8_t *buffer;
 } XenMapCacheData;
 
+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
+{
+    mapcache_lock(mc);
+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
+    mapcache_unlock(mc);
+}
+
+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
+{
+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
+}
+
 static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
     XenMapCacheData *data = opaque;
 
-    mapcache_lock(mapcache);
-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
-    mapcache_unlock(mapcache);
-
+    xen_invalidate_map_cache_entry_all(data->buffer);
     aio_co_wake(data->co);
 }
 
@@ -543,9 +638,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
                                 xen_invalidate_map_cache_entry_bh, &data);
         qemu_coroutine_yield();
     } else {
-        mapcache_lock(mapcache);
-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
-        mapcache_unlock(mapcache);
+        xen_invalidate_map_cache_entry_all(buffer);
     }
 }
 
@@ -597,6 +690,7 @@ void xen_invalidate_map_cache(void)
     bdrv_drain_all();
 
     xen_invalidate_map_cache_single(mapcache);
+    xen_invalidate_map_cache_single(mapcache_grants);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
@@ -632,13 +726,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) == 0);
+
     address_index  = new_phys_addr >> mc->bucket_shift;
     address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false, old_phys_addr);
+                     cache_size, address_index, false,
+                     false, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 65a51aac2e..3d796235dc 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -16,6 +16,7 @@
 #include <xen/hvm/ioreq.h>
 
 extern MemoryRegion xen_memory;
+extern MemoryRegion xen_grants;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
 
@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
     do { } while (0)
 #endif
 
+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
+
 static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
 {
     return shared_page->vcpu_ioreq[i].vp_eport;
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index dc72f83bcb..19dccf4d71 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -35,6 +35,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
                    struct MemoryRegion *mr, Error **errp);
 
 bool xen_mr_is_memory(MemoryRegion *mr);
+bool xen_mr_is_grants(MemoryRegion *mr);
 
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
@@ -55,6 +56,12 @@ static inline bool xen_mr_is_memory(MemoryRegion *mr)
     return false;
 }
 
+static inline bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    g_assert_not_reached();
+    return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 10:58:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 10:58:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729337.1134467 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASdL-0008TY-Gi; Fri, 24 May 2024 10:58:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729337.1134467; Fri, 24 May 2024 10:58:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASdL-0008TR-Dj; Fri, 24 May 2024 10:58:47 +0000
Received: by outflank-mailman (input) for mailman id 729337;
 Fri, 24 May 2024 10:58:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sASdK-0008TJ-RB
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 10:58:46 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 975ee6b1-19bc-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 12:58:45 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e95a883101so8655681fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 03:58:45 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421088f9d3csm17891275e9.0.2024.05.24.03.58.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 03:58:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 975ee6b1-19bc-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716548325; x=1717153125; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wq0y+5gnfEroqCGPQ7L/DPisiCcV/vib+/kZuqGJw4w=;
        b=Vf4FZdSnI8tBDYlp4K4XHYiM6voXfXwt8b/zaPcMkqAiwos02QjUmw0h2vqbryIe8y
         lbq72aCzr4KYcVwI9L7NJjuF7Bbq1q2USZkL4/CKYj9/Yt3p/zO/dQT5Q5N9D0k//NeK
         0xgp8RdRnaM4c3NFD9dTCU3EffZFRNIA6tpQA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548325; x=1717153125;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wq0y+5gnfEroqCGPQ7L/DPisiCcV/vib+/kZuqGJw4w=;
        b=L2fZ4jMVMHuHtXmTR51S+jVBMG7Dn3zkIw33BFP9cGltM57sWkBuWZkKa0Zru43Zcn
         BPpMzjQfs25uWZnnFEd7N9N6/BJBh/J+vFuSkgBBvfTfLpjVVG3tMCsPrkFPp9Ng0HUy
         GQR6y76zljGcOg40KvIdo330OpDmRSrXOZJNk+fe+UGKi3qMK8/YD1leOX0qjy2i0v2D
         oWqpJ0EbM/2vMIr/75UNc/4jTa0izg/WT6+hdPHqsMdMGMt32PmFid5oo2TGqbVV+WDX
         EmFgJ74E5dcNbXWDTUoqwZ5HzyD6JhDtPdG/d9AOliRgglBD1ON8TjZnVOkV5/lCwZwc
         oiFA==
X-Gm-Message-State: AOJu0YzFXZwcv5ozLRIFZTH+BP0r2FDzw0nPVgUBImqx7rZ2wP72vhqA
	BqvgLwj0hGBknjaoIFGGXRMCtMOvlUHqDDaD5ZpNABWu/m2M2CRW92Jr/nfbzgg=
X-Google-Smtp-Source: AGHT+IEQixrthMolI9Bj5IhdBnaNVYfD2O91JfHMl4b9Rvkfiml7hhOOYy6HZPzKis9exdlzVSm93g==
X-Received: by 2002:a2e:7c14:0:b0:2e0:ffea:4298 with SMTP id 38308e7fff4ca-2e95b261b93mr12646101fa.34.1716548325034;
        Fri, 24 May 2024 03:58:45 -0700 (PDT)
Message-ID: <dbd87d6b-c19a-48cc-8838-32b958da980d@cloud.com>
Date: Fri, 24 May 2024 11:58:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/8] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <4095f31a88589ced2b620e8ebbb84cdc2fae8914.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9TZhe-WSNrnlPD@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zk9TZhe-WSNrnlPD@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 15:32, Roger Pau Monné wrote:
>>      case 0xb:
>> -        /*
>> -         * In principle, this leaf is Intel-only.  In practice, it is tightly
>> -         * coupled with x2apic, and we offer an x2apic-capable APIC emulation
>> -         * to guests on AMD hardware as well.
>> -         *
>> -         * TODO: Rework topology logic.
>> -         */
>> -        if ( p->basic.x2apic )
>> +        /* Don't expose topology information to PV guests */
> 
> Not sure whether we want to keep part of the comment about exposing
> x2APIC to guests even when x2APIC is not present in the host.  I think
> this code has changed and the comment is kind of stale now.

The comment is definitely stale. Nowadays x2APIC is fully supported by
AMD, as is leaf 0xb. The fact we emulate the x2APIC seems hardly
relevant in a CPUID leaf about topology. I could keep a note showing...

    /* Exposed alongside x2apic, as it's tightly coupled with it */

... although that's directly implied by the conditional.

>> +void vlapic_cpu_policy_changed(struct vcpu *v)
>> +{
>> +    struct vlapic *vlapic = vcpu_vlapic(v);
>> +    const struct cpu_policy *cp = v->domain->arch.cpu_policy;
>> +
>> +    /*
>> +     * Don't override the initial x2APIC ID if we have migrated it or
>> +     * if the domain doesn't have vLAPIC at all.
>> +     */
>> +    if ( !has_vlapic(v->domain) || vlapic->loaded.hw )
>> +        return;
>> +
>> +    vlapic->hw.x2apic_id = x86_x2apic_id_from_vcpu_id(cp, v->vcpu_id);
>> +    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
> 
> Nit: in case we decide to start APICs in x2APIC mode, might be good to
> take this into account here and use vlapic_x2apic_mode(vlapic) to
> select whether SET_xAPIC_ID() needs to be used or not:>
>     vlapic_set_reg(vlapic, APIC_ID,
>         vlapic_x2apic_mode(vlapic) ? vlapic->hw.x2apic_id
> 	                           : SET_xAPIC_ID(vlapic->hw.x2apic_id));
> 
> Or similar.
> 

I like it. Sure.

>> +}
>> +
>>  int guest_wrmsr_apic_base(struct vcpu *v, uint64_t val)
>>  {
>>      const struct cpu_policy *cp = v->domain->arch.cpu_policy;
>> @@ -1449,7 +1465,7 @@ void vlapic_reset(struct vlapic *vlapic)
>>      if ( v->vcpu_id == 0 )
>>          vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
>>  
>> -    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
>> +    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
>>      vlapic_do_init(vlapic);
>>  }
>>  
>> @@ -1514,6 +1530,16 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>>      const struct vcpu *v = vlapic_vcpu(vlapic);
>>      uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id);
>>  
>> +    /*
>> +     * Loading record without hw.x2apic_id in the save stream, calculate using
>> +     * the traditional "vcpu_id * 2" relation. There's an implicit assumption
>> +     * that vCPU0 always has x2APIC0, which is true for the old relation, and
>> +     * still holds under the new x2APIC generation algorithm. While that case
>> +     * goes through the conditional it's benign because it still maps to zero.
>> +     */
>> +    if ( !vlapic->hw.x2apic_id )
>> +        vlapic->hw.x2apic_id = v->vcpu_id * 2;
>> +
>>      /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */
>>      if ( !vlapic_x2apic_mode(vlapic) ||
>>           (vlapic->loaded.ldr == good_ldr) )
>> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
>> index 0c9e6f15645d..e1f0585d75a9 100644
>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>> @@ -448,6 +448,7 @@ static inline void hvm_update_guest_efer(struct vcpu *v)
>>  static inline void hvm_cpuid_policy_changed(struct vcpu *v)
>>  {
>>      alternative_vcall(hvm_funcs.cpuid_policy_changed, v);
>> +    vlapic_cpu_policy_changed(v);
> 
> Note sure whether this call would better be placed in
> cpu_policy_updated() inside the is_hvm_vcpu() conditional branch.
> 
> hvm_cpuid_policy_changed() &c are just wrappers around the hvm_funcs
> hooks, pulling vlapic functions in there is likely to complicate the
> header dependencies in the long term.
> 

That's how it was in v1 and I moved it in v2 answering one of Jan's
feedback points.

I don't mind either way.

>>  }
>>  
>>  static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
>> diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
>> index 88ef94524339..e8d41313abd3 100644
>> --- a/xen/arch/x86/include/asm/hvm/vlapic.h
>> +++ b/xen/arch/x86/include/asm/hvm/vlapic.h
>> @@ -44,6 +44,7 @@
>>  #define vlapic_xapic_mode(vlapic)                               \
>>      (!vlapic_hw_disabled(vlapic) && \
>>       !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD))
>> +#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id)
>>  
>>  /*
>>   * Generic APIC bitmap vector update & search routines.
>> @@ -107,6 +108,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack);
>>  
>>  int  vlapic_init(struct vcpu *v);
>>  void vlapic_destroy(struct vcpu *v);
>> +void vlapic_cpu_policy_changed(struct vcpu *v);
>>  
>>  void vlapic_reset(struct vlapic *vlapic);
>>  
>> diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
>> index 7ecacadde165..1c2ec669ffc9 100644
>> --- a/xen/include/public/arch-x86/hvm/save.h
>> +++ b/xen/include/public/arch-x86/hvm/save.h
>> @@ -394,6 +394,8 @@ struct hvm_hw_lapic {
>>      uint32_t             disabled; /* VLAPIC_xx_DISABLED */
>>      uint32_t             timer_divisor;
>>      uint64_t             tdt_msr;
>> +    uint32_t             x2apic_id;
>> +    uint32_t             rsvd_zero;
> 
> I think Jan requested for this field to be checked to be 0 for
> incoming migrations, yet I don't see such check added.
> 

It's on the next patch, when the checks are moved into the check hook.
Doing it all as part of the same patch seemed dirty. I guess I can
invert them.

>>  };
>>  
>>  DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic);
>> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
>> index d5e447e9dc06..392320b9adbe 100644
>> --- a/xen/include/xen/lib/x86/cpu-policy.h
>> +++ b/xen/include/xen/lib/x86/cpu-policy.h
>> @@ -542,6 +542,15 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>>                                      const struct cpu_policy *guest,
>>                                      struct cpu_policy_errors *err);
>>  
>> +/**
>> + * Calculates the x2APIC ID of a vCPU given a CPU policy
>> + *
>> + * @param p          CPU policy of the domain.
>> + * @param id         vCPU ID of the vCPU.
>> + * @returns x2APIC ID of the vCPU.
>> + */
>> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
>> +
>>  #endif /* !XEN_LIB_X86_POLICIES_H */
>>  
>>  /*
>> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
>> index f033d22785be..4cef658feeb8 100644
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -2,6 +2,17 @@
>>  
>>  #include <xen/lib/x86/cpu-policy.h>
>>  
>> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>> +{
>> +    /*
>> +     * TODO: Derive x2APIC ID from the topology information inside `p`
>> +     *       rather than from vCPU ID. This bodge is a temporary measure
>> +     *       until all infra is in place to retrieve or derive the initial
>> +     *       x2APIC ID from migrated domains.
>> +     */
>> +    return vcpu_id * 2;
> 
> I don't think this builds?
> 
> Note the parameter is plain 'id' not 'vcpu_id'.
> 
> Thanks, Roger.

Bah. mid-patches after v1 don't get much attention. Ack.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 11:02:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:02:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729348.1134477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAShI-0002WL-3N; Fri, 24 May 2024 11:02:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729348.1134477; Fri, 24 May 2024 11:02:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAShI-0002WE-0g; Fri, 24 May 2024 11:02:52 +0000
Received: by outflank-mailman (input) for mailman id 729348;
 Fri, 24 May 2024 11:02:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAShH-0002Ur-5N
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:02:51 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 290c866f-19bd-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:02:50 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-354df3ee1a9so2653047f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:02:50 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a1c9363sm1330653f8f.72.2024.05.24.04.02.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:02:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 290c866f-19bd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716548569; x=1717153369; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=u3BswUzSzXy/a4rA08jYVQviuF2QhYJsjbScxmPbKlg=;
        b=hqsfEL6zg2B5guMfexKGvrh5JhGpQCQ8uE4BK4CWbmUcel+wPT4Lqbxw1Bj3xSBYSk
         VfO1iY4guEOqp7Gq95tl4HlG4dmGR+K41elasgXd861IxBrzlUWK7wREIsm1AxX/WGoJ
         xG4ZPkfWnGbs0rouSxqN6UbtDEfdS1bzw1SMY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548569; x=1717153369;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=u3BswUzSzXy/a4rA08jYVQviuF2QhYJsjbScxmPbKlg=;
        b=Mjez/naF1qurq25GU37rVXq2MWXoy322sEGqM5R3ckl5x1cnPe2MbkV2Vp0x6n8NRZ
         9EiWDBxWnC83/2WPjpLUv9gKJys6HTDeeMVfGt0pCvUdZSkLdAfAq9u3NVL+4GYwrLIA
         2pON/suBTMvfkwzuSwopIoo2aybzrkgxC9H53gAbCQtiR9Nbs0qAWrzyJlxuBMTJKZP8
         cgtkrrRYwNzquKgM9SbmhKIgHYuITMp01xE27fjqwoBTConmura1TnwuSv9WXJwHa4l3
         mcEVnAjm2hUo02sgCBFxAommA5pYdMkGtGwp7n9Tcf5UX0UWJYdNvd2lOMGVWoyiqbuv
         IFQA==
X-Gm-Message-State: AOJu0YyOic9H3s+s8/hlA3VRV4uyN/p6pUTBrPM97V9kiF+S9eRbk8My
	weo56/Eptq1Qm3lbwYG1vtcibTksbx8dey46flrwCdxLuEKgCJwjYrxSgl0LeZo=
X-Google-Smtp-Source: AGHT+IEqtcAzHinqqC9eCH7ICNYDHiCNIjwJsWk1eRdq9UgyJ+cK/+bC7KnMe74uxUSS+ccCBYHyog==
X-Received: by 2002:adf:a18d:0:b0:355:586:2245 with SMTP id ffacd0b85a97d-35526c2889bmr1197043f8f.23.1716548569431;
        Fri, 24 May 2024 04:02:49 -0700 (PDT)
Date: Fri, 24 May 2024 13:02:48 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v3 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
Message-ID: <ZlBz2HSAjvNWJ41B@macbook>
References: <cover.1716457040.git.alejandro.vallejo@cloud.com>
 <5c6ee74b60bad4eb9cc8e17dbfcf7158d38bd32c.1716457040.git.alejandro.vallejo@cloud.com>
 <Zk8YucyA1ZwGU4Y-@macbook>
 <de256e4b-fdc4-48a8-9173-38092ab90716@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <de256e4b-fdc4-48a8-9173-38092ab90716@cloud.com>

On Fri, May 24, 2024 at 11:32:50AM +0100, Alejandro Vallejo wrote:
> On 23/05/2024 11:21, Roger Pau Monné wrote:
> > On Thu, May 23, 2024 at 10:41:29AM +0100, Alejandro Vallejo wrote:
> >> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
> >> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> >> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
> >> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
> >>  {
> >> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
> >> +    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
> >>      int rc;
> >>  
> >> -    if ( leaves )
> >> +    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
> >> +    if ( rc )
> >>      {
> >> -        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
> >> -        if ( rc )
> >> -        {
> >> -            ERROR("Failed to serialize CPUID policy");
> >> -            errno = -rc;
> >> -            return -1;
> >> -        }
> >> +        ERROR("Failed to serialize CPUID policy");
> >> +        errno = -rc;
> >> +        return -1;
> >>      }
> >>  
> >> -    if ( msrs )
> >> +    p->nr_leaves = nr_leaves;
> > 
> > Nit: FWIW, I think you could avoid having to introduce local
> > nr_{leaves,msrs} variables and just use p->nr_{leaves,msrs}?  By
> > setting them to ARRAY_SIZE() at the top of the function and then
> > letting x86_{cpuid,msr}_copy_to_buffer() adjust as necessary.
> > 
> > Thanks, Roger.
> 
> The intent was to avoid mutating the policy object in the error cases
> during deserialise. Then I adjusted the serialise case to have symmetry.

It's currently unavoidable for the policy to be likely mutated even in
case of error, as x86_{cpuid,msr}_copy_to_buffer() are two separate
operations, and hence the first succeeding but the second failing will
already result in the policy being mutated on error.

> It's true the preservation is not meaningful in the serialise case
> because at that point the serialised form is already corrupted.
> 
> I don't mind either way. Seeing how I'm sending one final version with
> the comments of patch2 I'll just adjust as you proposed.

I'm fine either way (hence why prefix it with "nit:") albeit I have a
preference for not introducing the local variables if they are not
needed.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729358.1134511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003il-Dz; Fri, 24 May 2024 11:09:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729358.1134511; Fri, 24 May 2024 11:09:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003iW-9Q; Fri, 24 May 2024 11:09:05 +0000
Received: by outflank-mailman (input) for mailman id 729358;
 Fri, 24 May 2024 11:09:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnH-0003D0-Ns
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:03 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07469569-19be-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:09:02 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a6269ad71b6so82172366b.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:03 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07469569-19be-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548942; x=1717153742; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7d2EqxK4VlcyZgrHG2d9Y7mDD/1pyOMs5CW5hdSjjiU=;
        b=glSE4kwn5qEQxFFXIbJUWaxkiOqug39gdIUW8yMlimFqG3+LV/oQpGWW8hiYLWDyML
         6WR/nGySJF9CL3DxNQ8/ZItewsk0kXchapcovvLn/DW8dU8g/AcHM08YIkaIuDIS+0ki
         MHHHnuPJhpImhlak8fSvs3eZfV4cTLIhneclhLiJ6ATsLRwdl9B7yRjzbWc6vPRa616J
         XoZjLsZFF0o5HAsyhOtKWeqagTQba1Kx2JpmEgS01v7f6VD3kLA1/ZimP9P2ccMuzXRr
         +QqAteghF0QwramrAzEmftS47GeP8arRFp71hu3gk6ZN2NLeiqzNsUr+ax/0ddoeaFq1
         mlzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548942; x=1717153742;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7d2EqxK4VlcyZgrHG2d9Y7mDD/1pyOMs5CW5hdSjjiU=;
        b=QxKWkHbyqon0e/46TRhm6mkp3BAQeMlpGURe3KbhO5V4WJmxWuEcspsblJo3gevmMe
         zNjtPXttimcjQqeAKUchLsPh0cE1KSCNFopy51msM0eIdbezIoypXMQtTI717dP5q7Xz
         suXotDRFIEzqhFv77w1VFLgk+PX+aGnbFLDiBWvOpiOKZgo8m+aZ1ZTYge11bU6I1mLP
         B80QBHAt3bDUivLxiVuQhx+w6wnP7hTPmg57oJukm7mMaWFZ94iciTYyy9tTRGQDEFRK
         xjr1lqKGbPRGqQH98wHkG3enhDgHOygQxvjLMI3rwcCrpjPi+lE/wtUvCoZW0njqJRjL
         ZTPA==
X-Gm-Message-State: AOJu0YzABATIiHnYNFPFvl7UFUUwOGQLRVw0LiMfrSTZJm9NI3NFpAF4
	5z5k2eE4F6EKkpPdHFBDU6OcKadM4OSSA3kQk1TmEvqR0Do/JecVYoVDri2p
X-Google-Smtp-Source: AGHT+IFYmUG+cdFWEbzCaV2+43OdHFBaS5Oay+qaniUbOIU4fOv63gV5vPi3OV9HxRioNeHsSaDeCw==
X-Received: by 2002:a17:906:19cf:b0:a59:b342:3ed5 with SMTP id a640c23a62f3a-a62641b5482mr128417666b.15.1716548942063;
        Fri, 24 May 2024 04:09:02 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 4/9] xen/riscv: introduce bitops.h
Date: Fri, 24 May 2024 13:08:50 +0200
Message-ID: <154d59893ac84668f73690fbf94898b2ce0710a4.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V11:
 - Nothing changed. Only rebase was done.
---
Changes in V10:
 - update the error message BITS_PER_LONG -> BITOP_BITS_PER_WORD
---
Changes in V9:
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects
   to work with 32-bit quantities.
 - s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
 - define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of
   "introduce generic non-atomic test_*bit()".
 - drop duplicated __set_bit() and __clear_bit().
 - drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
 - update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
 - drop 1 padding blank in test_and_op_bit_ord().
 - update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
   change return type to bool.
 - change addr argument type of test_and_change_bit(): unsigned long * -> void *.
 - move test_and_change_bit() closer to other test_and-s function.
 - Code style fixes: tabs -> space.
 - s/#undef __op_bit/#undef op_bit.
 - update the commit message: delete information about generic-non-atomic.h changes as now
   it is a separate patch.
---
Changes in V7:
 - Update the commit message.
 - Drop "__" for __op_bit and __op_bit_ord as they are atomic.
 - add comment above __set_bit and __clear_bit about why they are defined as atomic.
 - align bitops_uint_t with __AMO().
 - make changes after  generic non-atomic test_*bit() were changed.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
 - rebase clean ups were done: drop unused asm-generic includes
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..7f7af3fda1
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline bool test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline bool test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline bool test_and_change_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729359.1134517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003nF-MW; Fri, 24 May 2024 11:09:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729359.1134517; Fri, 24 May 2024 11:09:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003li-Gw; Fri, 24 May 2024 11:09:05 +0000
Received: by outflank-mailman (input) for mailman id 729359;
 Fri, 24 May 2024 11:09:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnH-0003Cz-RW
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:03 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06b269ad-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:09:02 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a62769075adso51717966b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:02 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.08.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06b269ad-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548941; x=1717153741; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yd/DGVLnKJ3GtywGuSmeUy0gbzAiv6eqVrDkcFb+xAQ=;
        b=LmQszo4JSC5P2geKR+tI6RLKfJcUH3JqyXSvybyZh1EF5p3ayn0krV9eSpZ78A3rw+
         lPQtx9TyhhtY9h2VVv7zEOMGz2cXkBKqfBbPsTGoHUj3DriyyEjZwfFVyYWrok+6bRMz
         ryRsfw2LJ9qcE07vipfZaIkJMcoMBJJOXjweDS5oUciUqLwDF1fEY0C5oOxprBqf1mp2
         GQSxpllKaRxEU3DusEP02oW/yb8y6PdctUAZO05w0K1XKhaPtd8jAK+9UEw+xKbRIrId
         c7mMCzXwiCywzzwd5S5BjVFpqLKZJOmCyJwnee138Qk3D/B05B62tzAv5hLr/VUhjHCV
         lYKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548941; x=1717153741;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=yd/DGVLnKJ3GtywGuSmeUy0gbzAiv6eqVrDkcFb+xAQ=;
        b=DldyTDxGlYw5qYAbaaYaOqWyWNAsJuTL+5a/zVDOYp/MeTPXB3inIcyAA1ekHE+RwF
         xU0WSQWLviXfKusgv6Kh/BIgFzJGaj97TlUbFUK2+WHgr1wTKk49PySEZTtiyNZliEAU
         lltPaVJ64yEXeyA2RqPi6IsCyd+d60P8Aij4/Gq2es3W+awkSv/7V0uwqMQ2XzZ9FXtM
         G5GhBM3/F0fkZ/DCPrdoakA6Yh3+YQZu2UqDpTMtVPRXLSE+trP7aaboPF6eGcbYkDk4
         b4AK7e86hvrGSFvIdU8Ba2J5KmkGTD1JODC0Zk5T4t4kzM031WZ/Kee8I5n5JviSJmkK
         4xWw==
X-Gm-Message-State: AOJu0YxXanfNmUc6gKQWmPhdFcPFOMDzDzUpbPFTgIXVLYm7cF6tTU70
	kNy7eDY+vnOAE2ZmHt1Jy8P58w9Y9/5bVu+WhljhdATV+Ye70m7Fzy+ngnVH
X-Google-Smtp-Source: AGHT+IHhqAZDjBHBhgoru8Yb8g2qsIBr02+0uTRek59eZRvOMQ4xA+qVHiPyAFEp4ebYGzazT1zCCg==
X-Received: by 2002:a17:906:4f01:b0:a59:b784:ced4 with SMTP id a640c23a62f3a-a6265118800mr123826166b.75.1716548940991;
        Fri, 24 May 2024 04:09:00 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v11 3/9] xen/bitops: implement fls{l}() in common logic
Date: Fri, 24 May 2024 13:08:49 +0200
Message-ID: <e48f86210422eff81dc2dbcfe50f6a9fcdaac6ac.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To avoid the compilation error below, it is needed to update to places
in common/page_alloc.c where flsl() is used as now flsl() returns unsigned int:

./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
       18 |         (void) (&_x == &_y);            \
          |                     ^~
    common/page_alloc.c:1843:34: note: in expansion of macro 'min'
     1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);

generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0,
the result in undefined.

The prototype of the per-architecture fls{l}() functions was changed to
return 'unsigned int' to align with the generic implementation of these
functions and avoid introducing signed/unsigned mismatches.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 The current one patch can be merged w/o waiting of Andrew's patch series.
 
 Andrew C. could you please consider the following patch to be as a part of your bit operation
 patch series if your patch series will be in staging before mine:
   https://gitlab.com/xen-project/people/olkur/xen/-/commit/24a346c7aa4f51ba34eacb7bfee2808e431daf00
 
 Thanks in advance.
---
Changes in V11:
 - drop unnessary case and fix code style for x86's arch_flsl()
 - rebase on top of staging, so it can be merged to staging if necessary apporves will be given
 - move changes related to fls related function to separate patch on top of Andrew's patch series
   as the C file with tests is introduced in his patch series. The patch is mentioned in footer.
---
Changes in V10:
 - update return type of arch_flsl() across arcitectures to 'unsigned int' to be aligned
   with return type of generic flsl() in xen/bitops.h.
 - switch inline to always_inline for arch_flsl() across architectures to be in sync
   with other similar changes.
 - define arch_flsl as arch_fls not just only fls.
 - update the commit message ( add information that per-arch fls{l)() protypes were
   changed ).
---
Changes in V9:
 - update return type of fls and flsl() to unsigned int to be aligned with other
   bit ops.
 - update places where return value of fls() and flsl() is compared with int.
 - update the commit message.
---
Changes in V8:
 - do proper rebase: back definition of fls{l} to the current patch.
 - drop the changes which removed ffz() in PPC. it should be done not
   in this patch.
 - add a message after Signed-off.
---
Changes in V7:
 - Code style fixes
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm32/bitops.h |  2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  6 ++----
 xen/arch/arm/include/asm/bitops.h       |  9 +++------
 xen/arch/ppc/include/asm/bitops.h       |  2 --
 xen/arch/x86/include/asm/bitops.h       | 12 +++++++-----
 xen/common/page_alloc.c                 |  4 ++--
 xen/include/xen/bitops.h                | 24 ++++++++++++++++++++++++
 7 files changed, 39 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/bitops.h b/xen/arch/arm/include/asm/arm32/bitops.h
index d0309d47c1..9ee96f568b 100644
--- a/xen/arch/arm/include/asm/arm32/bitops.h
+++ b/xen/arch/arm/include/asm/arm32/bitops.h
@@ -1,7 +1,7 @@
 #ifndef _ARM_ARM32_BITOPS_H
 #define _ARM_ARM32_BITOPS_H
 
-#define flsl fls
+#define arch_flsl arch_fls
 
 /*
  * Little endian assembly bitops.  nr = 0 -> byte 0 bit 0.
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 906d84e5f2..d942077392 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -1,17 +1,15 @@
 #ifndef _ARM_ARM64_BITOPS_H
 #define _ARM_ARM64_BITOPS_H
 
-static inline int flsl(unsigned long x)
+static always_inline unsigned int arch_flsl(unsigned long x)
 {
         uint64_t ret;
 
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
         asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
 
         return BITS_PER_LONG - ret;
 }
+#define arch_flsl arch_flsl
 
 /* Based on linux/include/asm-generic/bitops/find.h */
 
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index af38fbffdd..52fd33a797 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -76,17 +76,14 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p,
  * the clz instruction for much better code efficiency.
  */
 
-static inline int fls(unsigned int x)
+static always_inline unsigned int arch_fls(unsigned int x)
 {
-        int ret;
-
-        if (__builtin_constant_p(x))
-               return generic_fls(x);
+        unsigned int ret;
 
         asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
         return 32 - ret;
 }
-
+#define arch_fls arch_fls
 
 #define ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
 #define ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 71bbb5f16a..82f9d92c7a 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -117,8 +117,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-#define flsl(x) generic_flsl(x)
-#define fls(x) generic_fls(x)
 #define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
 #define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
 
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index c4e7a06155..e488b39728 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -434,20 +434,21 @@ static inline int ffs(unsigned int x)
  *
  * This is defined the same way as ffs.
  */
-static inline int flsl(unsigned long x)
+static always_inline unsigned int arch_flsl(unsigned long x)
 {
-    long r;
+    unsigned long r;
 
     asm ( "bsr %1,%0\n\t"
           "jnz 1f\n\t"
           "mov $-1,%0\n"
           "1:" : "=r" (r) : "rm" (x));
-    return (int)r+1;
+    return r + 1;
 }
+#define arch_flsl arch_flsl
 
-static inline int fls(unsigned int x)
+static always_inline unsigned int arch_fls(unsigned int x)
 {
-    int r;
+    unsigned int r;
 
     asm ( "bsr %1,%0\n\t"
           "jnz 1f\n\t"
@@ -455,6 +456,7 @@ static inline int fls(unsigned int x)
           "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_fls arch_fls
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 7c1bdfc046..d4d05d02b7 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1842,7 +1842,7 @@ static void _init_heap_pages(const struct page_info *pg,
          * Note that the value of ffsl() and flsl() starts from 1 so we need
          * to decrement it by 1.
          */
-        unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
+        unsigned int inc_order = min(MAX_ORDER + 0U, flsl(e - s) - 1);
 
         if ( s )
             inc_order = min(inc_order, ffsl(s) - 1U);
@@ -2266,7 +2266,7 @@ void __init xenheap_max_mfn(unsigned long mfn)
     ASSERT(!first_node_initialised);
     ASSERT(!xenheap_bits);
     BUILD_BUG_ON((PADDR_BITS - PAGE_SHIFT) >= BITS_PER_LONG);
-    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS);
+    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS + 0U);
     printk(XENLOG_INFO "Xen heap: %u bits\n", xenheap_bits);
 }
 
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 9f89232ba6..c98eb827e3 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -290,6 +290,30 @@ static always_inline bool test_bit(int nr, const volatile void *addr)
     test_bit(nr, addr);                                 \
 })
 
+static always_inline __pure unsigned int fls(unsigned int x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_fls(x);
+
+#ifndef arch_fls
+#define arch_fls generic_fls
+#endif
+
+    return arch_fls(x);
+}
+
+static always_inline __pure unsigned int flsl(unsigned long x)
+{
+    if ( __builtin_constant_p(x) )
+        return generic_flsl(x);
+
+#ifndef arch_flsl
+#define arch_flsl generic_flsl
+#endif
+
+    return arch_flsl(x);
+}
+
 #ifndef find_next_bit
 /**
  * find_next_bit - find the next set bit in a memory region
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729357.1134507 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003gx-5l; Fri, 24 May 2024 11:09:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729357.1134507; Fri, 24 May 2024 11:09:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003gq-2c; Fri, 24 May 2024 11:09:05 +0000
Received: by outflank-mailman (input) for mailman id 729357;
 Fri, 24 May 2024 11:09:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnH-0003Cz-8A
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:03 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 060dc183-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:09:00 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a62769075adso51716266b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:00 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.08.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:08:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 060dc183-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548940; x=1717153740; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+VAAhE05amsEXGMuoLHu119Hm7q5hqdBYgzStrwEHgM=;
        b=LkIuXS6VBuwouDn3Z+NJ8C2lM4gnQtJpQmcDf5tEr+N8bKfn3yILOH2cueNm0n3aXU
         En6ZIhTeoOYP7xi3KB0v2/v0Pape3R7J+5WrN+vFjeLsMd9ECF8uOge5xHgjIYq2O8fo
         kFNHdN6liySXBTlU3dBud7oVC3vwZwoHZbVflR7+KKCREQZlPSeAUmUUDxEnunXC8ni5
         BnHB3yoKg96zg4iWWqOy0DEv29gF5aH0e2ZYfQRRGu2B+0PrLF4qU81tDBEaHsZRq4Jt
         ufvCaXRa0gzpUEac01riNtASPxAclfus/UN/D0jaPvM05n3SmZCKpwLQ0eKQZtSVxzB2
         At7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548940; x=1717153740;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+VAAhE05amsEXGMuoLHu119Hm7q5hqdBYgzStrwEHgM=;
        b=qDu2/aRMkxnXSS+M/T1/FpQgvbQM0vss0Csquu3X49csH3fJNxziuX7BYQ/o3dVNUn
         Ulq+UCjKi1dnBYGvr/oC44Z3deJq0+5tUd8uPZPmpTFM3H9lzvtn/Ie4Qq8DSnSaimSk
         I22EsoMGEhNVrqCG1Fpus48vXaaqN7YNo3Pxc9G3tO6+EbbRvV0n4hnhDvue46yZTJMZ
         5LbhAOrdPz6L+2tuzk5Pe5JgV/u4uELzVmW/+2oIJjdRUn7nXvJB/ZeoxyrnlRce+krP
         drW/o5imnWdpQ/53dPdi3AR7awPu6lwZ4v9kukcB6NQmlvm4TB3/DNuca6PGyJAIYcx5
         WVzA==
X-Gm-Message-State: AOJu0YzGChUoevLEX/6dVSF9Xzg+vT/8JwYj0KLtrvIJUA5yMnddscQ6
	A8xIn7GAV9Ia1wFotG3IGWkZMrWSStW/MsN4SwvCVogmdox9VmQLMytFhYz8
X-Google-Smtp-Source: AGHT+IH8Xhcb1dv8lZUHDI5J6yAYaAcBV72erzycFafIGTAk/CxjjZTaBtujwXSoOE/FwfWwC9cqVA==
X-Received: by 2002:a17:907:3f29:b0:a5a:6122:d64b with SMTP id a640c23a62f3a-a62641a33e7mr165738766b.5.1716548939777;
        Fri, 24 May 2024 04:08:59 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
Date: Fri, 24 May 2024 13:08:48 +0200
Message-ID: <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

BITS_PER_BYTE was moved to xen/bitops.h as BITS_PER_BYTE is the same
across architectures.

The following approach was chosen for generic*() and arch*() bit
operation functions:
If the bit operation function that is going to be generic starts
with the prefix "__", then the corresponding generic/arch function
will also contain the "__" prefix. For example:
 * test_bit() will be defined using arch_test_bit() and
   generic_test_bit().
 * __test_and_set_bit() will be defined using
   arch__test_and_set_bit() and generic__test_and_set_bit().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Reviewed-by: Jan Beulich jbeulich@suse.com? Jan gave his R-by for the previous
 version of the patch, but some changes were done, so I wasn't sure if I could
 use the R-by in this patch version.


 The current patch can be merged without waiting for Andrew's patch series as
 it was rebased on top of staging.
---
Changes in V11:
 - fix identation in generic_test_bit() function.
 - move definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h
 - drop the changes in arm64/livepatch.c.
 - update the the comments on top of functions: generic__test_and_set_bit(),
   generic__test_and_clear_bit(),  generic__test_and_change_bit(),
   generic_test_bit().
 - Update footer after Signed-off section.
 - Rebase the patch on top of staging branch, so it can be merged when necessary
   approves will be given.
 - Update the commit message.
---
Changes in V10:
 - update the commit message. ( re-order paragraphs and add explanation usage of prefix "__" in bit
   operation function names )
 - add  parentheses around the whole expression of bitop_bad_size() macros.
 - move macros bitop_bad_size() above asm/bitops.h as it is not arch-specific anymore and there is
   no need for overriding it.
 - drop macros check_bitop_size() and use "if ( bitop_bad_size(addr) ) __bitop_bad_size();" implictly
   where it is needed.
 - in <xen/bitops.h> use 'int' as a first parameter for __test_and_*(), generic__test_and_*() to be
   consistent with how the mentioned functions were declared in the original per-arch functions.
 - add 'const' to p variable in generic_test_bit().
 - move definition of BITOP_BITS_PER_WORD and bitop_uint_t to xen/bitops.h as we don't allow for arch
   overrides these definitions anymore.
---
Changes in V9:
  - move up xen/bitops.h in ppc/asm/page.h.
  - update defintion of arch_check_bitop_size.
    And drop correspondent macros from x86/asm/bitops.h
  - drop parentheses in generic__test_and_set_bit() for definition of
    local variable p.
  - fix indentation inside #ifndef BITOP_TYPE...#endif
  - update the commit message.
---
 Changes in V8:
  - drop __pure for function which uses volatile.
  - drop unnessary () in generic__test_and_change_bit() for addr casting.
  - update prototype of generic_test_bit() and test_bit(): now it returns bool
    instead of int.
  - update generic_test_bit() to use BITOP_MASK().
  - Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
  - add a footer with explanation of dependency on an uncommitted patch after Signed-off.
  - abstract bitop_size().
  - move BITOP_TYPE define to <xen/types.h>.
---
 Changes in V7:
  - move everything to xen/bitops.h to follow the same approach for all generic
    bit ops.
  - put together BITOP_BITS_PER_WORD and bitops_uint_t.
  - make BITOP_MASK more generic.
  - drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
    enough.
  - drop "_" for generic__{test_and_set_bit,...}().
  - drop " != 0" for functions which return bool.
  - add volatile during the cast for generic__{...}().
  - update the commit message.
  - update arch related code to follow the proposed generic approach.
---
 Changes in V6:
  - Nothing changed ( only rebase )
---
 Changes in V5:
   - new patch
---
 xen/arch/arm/include/asm/bitops.h |  69 -----------
 xen/arch/ppc/include/asm/bitops.h |  54 ---------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  31 ++---
 xen/include/xen/bitops.h          | 185 ++++++++++++++++++++++++++++++
 6 files changed, 196 insertions(+), 147 deletions(-)

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index ab030b6cb0..af38fbffdd 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -22,11 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
-#define BITS_PER_BYTE           8
-
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
@@ -76,70 +71,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 /*
  * On ARMv5 and above those functions can be implemented around
  * the clz instruction for much better code efficiency.
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index bea655796d..71bbb5f16a 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -15,11 +15,6 @@
 #define __set_bit(n, p)         set_bit(n, p)
 #define __clear_bit(n, p)       clear_bit(n, p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
-#define BITS_PER_BYTE           8
-
 /* PPC bit number conversion */
 #define PPC_BITLSHIFT(be)    (BITS_PER_LONG - 1 - (be))
 #define PPC_BIT(bit)         (1UL << PPC_BITLSHIFT(bit))
@@ -69,17 +64,6 @@ static inline void clear_bit(int nr, volatile void *addr)
     clear_bits(BITOP_MASK(nr), (volatile unsigned int *)addr + BITOP_WORD(nr));
 }
 
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-    const volatile unsigned int *p = addr;
-    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
-}
-
 static inline unsigned int test_and_clear_bits(
     unsigned int mask,
     volatile unsigned int *p)
@@ -133,44 +117,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old | mask;
-    return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old & ~mask;
-    return (old & mask) != 0;
-}
-
 #define flsl(x) generic_flsl(x)
 #define fls(x) generic_fls(x)
 #define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
diff --git a/xen/arch/ppc/include/asm/page.h b/xen/arch/ppc/include/asm/page.h
index 890e285051..6d4cd2611c 100644
--- a/xen/arch/ppc/include/asm/page.h
+++ b/xen/arch/ppc/include/asm/page.h
@@ -2,9 +2,9 @@
 #ifndef _ASM_PPC_PAGE_H
 #define _ASM_PPC_PAGE_H
 
+#include <xen/bitops.h>
 #include <xen/types.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
diff --git a/xen/arch/ppc/mm-radix.c b/xen/arch/ppc/mm-radix.c
index ab5a10695c..9055730997 100644
--- a/xen/arch/ppc/mm-radix.c
+++ b/xen/arch/ppc/mm-radix.c
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 5a71afbc89..c4e7a06155 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -19,9 +19,6 @@
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
-extern void __bitop_bad_size(void);
-#define bitop_bad_size(addr) (sizeof(*(addr)) < 4)
-
 /**
  * set_bit - Atomically set a bit in memory
  * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index e3c5a4ccf3..9f89232ba6 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -103,8 +103,193 @@ static inline int generic_flsl(unsigned long x)
  * Include this here because some architectures need generic_ffs/fls in
  * scope
  */
+
+#define BITOP_BITS_PER_WORD 32
+typedef uint32_t bitop_uint_t;
+
+#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
+
+#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
+
+#define BITS_PER_BYTE 8
+
+extern void __bitop_bad_size(void);
+
+#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
+
+/* --------------------- Please tidy above here --------------------- */
+
 #include <asm/bitops.h>
 
+/**
+ * generic__test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_set_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask);
+}
+
+/**
+ * generic__test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_clear_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask);
+}
+
+/**
+ * generic__test_and_change_bit - Change a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic__test_and_change_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask);
+}
+
+/**
+ * generic_test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool generic_test_bit(int nr, const volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    const volatile bitop_uint_t *p =
+        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+
+    return (*p & mask);
+}
+
+/**
+ * __test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+__test_and_set_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_set_bit
+#define arch__test_and_set_bit generic__test_and_set_bit
+#endif
+
+    return arch__test_and_set_bit(nr, addr);
+}
+#define __test_and_set_bit(nr, addr) ({             \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_set_bit(nr, addr);                   \
+})
+
+/**
+ * __test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+__test_and_clear_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_clear_bit
+#define arch__test_and_clear_bit generic__test_and_clear_bit
+#endif
+
+    return arch__test_and_clear_bit(nr, addr);
+}
+#define __test_and_clear_bit(nr, addr) ({           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_clear_bit(nr, addr);                 \
+})
+
+/**
+ * __test_and_change_bit - Change a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+__test_and_change_bit(int nr, volatile void *addr)
+{
+#ifndef arch__test_and_change_bit
+#define arch__test_and_change_bit generic__test_and_change_bit
+#endif
+
+    return arch__test_and_change_bit(nr, addr);
+}
+#define __test_and_change_bit(nr, addr) ({              \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    __test_and_change_bit(nr, addr);                    \
+})
+
+/**
+ * test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool test_bit(int nr, const volatile void *addr)
+{
+#ifndef arch_test_bit
+#define arch_test_bit generic_test_bit
+#endif
+
+    return arch_test_bit(nr, addr);
+}
+#define test_bit(nr, addr) ({                           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    test_bit(nr, addr);                                 \
+})
+
 #ifndef find_next_bit
 /**
  * find_next_bit - find the next set bit in a memory region
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729356.1134496 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnG-0003RR-U7; Fri, 24 May 2024 11:09:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729356.1134496; Fri, 24 May 2024 11:09:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnG-0003RK-RV; Fri, 24 May 2024 11:09:02 +0000
Received: by outflank-mailman (input) for mailman id 729356;
 Fri, 24 May 2024 11:09:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnE-0003D0-Sc
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:00 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 05423ceb-19be-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:08:59 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a6265d3c488so95339466b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:08:59 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.08.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:08:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05423ceb-19be-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548939; x=1717153739; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VObOJh3vdjUSCat91cLLs4dKVo8jpzpt3RbhRwfCXX4=;
        b=BT3hrZH+0zQcsy8aojUBE1iXiTjgohradSxzFr0lWS1lZZMWajmn6cKfM87ISrd3dJ
         JXNF6VPwBYzhaOwluaDWtx65Yd08KRwANoqgL8UClRqmrJzIobLIG1glN3jTZtfRQCJF
         cv3bxEQJqwZXvEZYTVpmgLRMnO85pTSIshZ2OE9HQQjhUqpbwAKBjLFWcyWYTf5NKqce
         LJPdYwa9FW4LdR2vxe9b9LkCetmMTSPjXITFTAElX64LH9RaBF9UlmtwY+XdYCuDSODb
         PhaguEL3wybPi2Lw1BFRakJMT3HgxDOeXaJQu5IkXCCJNqCYcu6fVxgXSeJnsNCPVDeV
         du+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548939; x=1717153739;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VObOJh3vdjUSCat91cLLs4dKVo8jpzpt3RbhRwfCXX4=;
        b=oSJH6jYMwnMfjs/mUFdu5dQie0nF5xnSJua3T3hn0WARxH2l/GnFyEvJ6X7X+mKHUn
         lzYgsEl96+Kk6qhy6z1ud7ECXCacISDwaTK2i8tWOEiocn4KUKNCjjIbGuGOv4FKCNRz
         7gGngY8XSmeHXdjpbqE0KuO1RrICpW8PfTvmV41b6iQXap5aQplIb16DByBa26J3yqs4
         txML7UBEK/MKZoXkF0Z2n+U+5tLA0eJuq0lWzHrbjU8RmR32cfHG3VyjOAVSOPXlzd9h
         nB8H+0PSmj5RG1WsqM5hyhQQRk3OeaArRyGrAb9rh8Ed6affHtD+EToj/3bjuY+rGswl
         xQlg==
X-Gm-Message-State: AOJu0Yw6ipFMikCMmnNSbMTAVp3BX2TA8eLO4kQ/S6A5vcHY/H+0O3uS
	vddq+r9XpykaToHu4VnDf526039azoJXeCzhxF1os6f97RUlfrktKCXtfvjg
X-Google-Smtp-Source: AGHT+IEnfRajkGsXOcdDFf+N0NCog0aw4WP70ktRaE8iDhxiyttE5UG/CVauN9Avk8uaqYIdE5AXJA==
X-Received: by 2002:a17:906:2dd7:b0:a59:ccbb:595b with SMTP id a640c23a62f3a-a62642e937amr189197966b.25.1716548938644;
        Fri, 24 May 2024 04:08:58 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v11 1/9] xen/riscv: disable unnecessary configs
Date: Fri, 24 May 2024 13:08:47 +0200
Message-ID: <77cfff76ddcad4e41efce46c1e68d504e376bd32.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Only configs which lead to compilation issues were disabled.

Remove lines related to disablement of configs which aren't affected
compilation:
 -# CONFIG_SCHED_CREDIT is not set
 -# CONFIG_SCHED_RTDS is not set
 -# CONFIG_SCHED_NULL is not set
 -# CONFIG_SCHED_ARINC653 is not set
 -# CONFIG_TRACEBUFFER is not set
 -# CONFIG_HYPFS is not set
 -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set

To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
check_get_page_from_gfn() from xen/p2m-common.h.

Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was
included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
aren't declared started to occur.

CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.

CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
acpi_disabled, so it is disabled for now.

PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
also disabled for now, as RISC-V hasn't introduced this headers yet.

LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V10-V11:
 - Nothing changed. Only rebase.
---
Changes in V9:
 - update the commit message: add info about LIVEPATCH and PERF_COUNTERS.
---
Changes in V8:
 - disabled CPU_BOOT_TIME_CPUPOOLS as it requires an introduction of cpu_physical_id() and acpi_disabled.
 - leave XSM disabled, add explanation in the commit message.
 - drop HYPFS as the patch was provided to resolve compilation issue when this condif is enabled for RISC-V.
 - include asm/p2m.h to asm/domain.h, and xen/errno.h to asm/p2m.h to drop ARGO config from
   tiny64_defconfing and build.yaml.
 - update the commit message.
---
Changes in V7:
 - Disable only configs which cause compilation issues.
 - Update the commit message.
---
Changes in V6:
 - Nothing changed. Only rebase.
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         |  4 ++++
 xen/arch/riscv/configs/tiny64_defconfig | 12 +++++-------
 xen/arch/riscv/include/asm/domain.h     |  2 ++
 xen/arch/riscv/include/asm/p2m.h        |  2 ++
 4 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 49d6265ad5..ff5c9055d1 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -494,10 +494,14 @@ alpine-3.18-gcc-debug-arm64-earlyprintk:
 .riscv-fixed-randconfig:
   variables: &riscv-fixed-randconfig
     EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_BOOT_TIME_CPUPOOLS=n
       CONFIG_COVERAGE=n
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..fc7a04872f 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -1,12 +1,10 @@
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
+# CONFIG_BOOT_TIME_CPUPOOLS is not set
 # CONFIG_GRANT_TABLE is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_XSM is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 027bfa8a93..16a9dd57aa 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -5,6 +5,8 @@
 #include <xen/xmalloc.h>
 #include <public/hvm/params.h>
 
+#include <asm/p2m.h>
+
 struct hvm_domain
 {
     uint64_t              params[HVM_NR_PARAMS];
diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
index 387f372b5d..26860c0ae7 100644
--- a/xen/arch/riscv/include/asm/p2m.h
+++ b/xen/arch/riscv/include/asm/p2m.h
@@ -2,6 +2,8 @@
 #ifndef __ASM_RISCV_P2M_H__
 #define __ASM_RISCV_P2M_H__
 
+#include <xen/errno.h>
+
 #include <asm/page-bits.h>
 
 #define paddr_bits PADDR_BITS
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729360.1134524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnK-0003wU-69; Fri, 24 May 2024 11:09:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729360.1134524; Fri, 24 May 2024 11:09:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnJ-0003vl-UV; Fri, 24 May 2024 11:09:05 +0000
Received: by outflank-mailman (input) for mailman id 729360;
 Fri, 24 May 2024 11:09:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnI-0003D0-Nx
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:04 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07e29255-19be-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:09:04 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a62614b9ae1so102337066b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:04 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07e29255-19be-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548943; x=1717153743; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GUAxAxl17Un/j2tMvIXlYjzldu2txD+7ziiQaQEA83I=;
        b=hYJP88TGNRM52W25def6gIE/lefzt0O6evaSDhD9VMT2hsWtzKrUtJUn4ttvFEyBx1
         pc5U98BYIeDlUMvgMAQSDm6OiPURDbP3ed9YhKjH4yl5pLaiVPzuvTk2EZosNGBbZvf/
         1X5tEkiEZV5UlD8bnGJiloE9lrKV+cqkHytOTdhlzvio9HTBSn8pGlNpkM2T7aqjV6B8
         uXBAOL/ddK933N9o+QEy9fwYSKcJ0SOmuUQzznbyZwivhBk1asUVVqLXvJUVVpBTZSbR
         +wpn4wV0u8Tj4Ifj9KFMqYsVgExXqpYXkhvYDma48JZDzWtiiVHbuUzhKO2AktVi2WGA
         MhXA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548943; x=1717153743;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GUAxAxl17Un/j2tMvIXlYjzldu2txD+7ziiQaQEA83I=;
        b=GdlTraNMOIL9OxYHHs/hLLVNnTBVTqxh84pNhW+BUES9t9KEhVyhXNpb8cz/zg1tO0
         xphWwjtfeJRXJ+TttcCxnRpr+2UF3Mw8DGjKbH+inq/ulTdWDj1GMlIXj0wrhIXKv1ax
         MHcCh0N2Zxj7PKf/CJoEM0ZFLOIrXXUZZs+BjO+FUXHPmFy75LTaoUYayZ3pdc72Rth5
         WIkk+diRpJMyv65cDsUKBEFtLv2DzmM6skcepscXmZoQb/w/jA0CeJd2okhcTqbL3VLq
         yBQ6oNTd4suW/IqjFe0ATfiZuerbzGXQwiILjgaHSGTtf+4YSotT8NO7ZtJvzPUvlhs+
         wKcQ==
X-Gm-Message-State: AOJu0YxacdgaFa8KAMFfIJurnOy+xD8w+3zCGkPVEq7Hn1gm+8av7NS4
	q7u6LJkhg0Oak71foC2sPWWEd23ZWlwqVb8aWZX5FxOcssUo+MnCsmNMnM1A
X-Google-Smtp-Source: AGHT+IHE/a5m9fzX2m1slVE057V0x6OasQt0uN7p/KQTRS+O+SESh703FTrh7+fom4J4NszhaHKe5g==
X-Received: by 2002:a17:906:b21a:b0:a61:9877:a26b with SMTP id a640c23a62f3a-a62647cdcf1mr113951166b.43.1716548943237;
        Fri, 24 May 2024 04:09:03 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 5/9] xen/riscv: add definition of __read_mostly
Date: Fri, 24 May 2024 13:08:51 +0200
Message-ID: <09d5f6bdd3bbd8fe1f97317009ee6d0d0fc7dd39.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

Considering that there is still no still final decision regarding patch [2] my suggestion
is to merge RISC-V specific patch and just drop the changes in patch [2].

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V9-V11:
 - Only rebase was done.
---
Change in V8:
 - update the footer after Signed-off.
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729355.1134487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnF-0003DI-OW; Fri, 24 May 2024 11:09:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729355.1134487; Fri, 24 May 2024 11:09:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnF-0003DB-Kb; Fri, 24 May 2024 11:09:01 +0000
Received: by outflank-mailman (input) for mailman id 729355;
 Fri, 24 May 2024 11:09:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnE-0003Cz-IQ
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:00 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0473d943-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:08:58 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-578517c7ae9so891483a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:08:58 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.08.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:08:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0473d943-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548937; x=1717153737; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=fJnzblV3ATfhEvm8cpMubqr58bn3DL3t5seeZzPNX1U=;
        b=QsFFGkJWk1BN/99HxBh2Xh0fBoTOV7BcMath7avcRT0AzTHpUSml4LFf+GlqdCnRoj
         9/9B89dFoMs0Ng0lKmSNOh29FSmG0lK5/qPXoEZ/jf3D3Up0kVMlp454+6hnsLMOXERs
         JjQUZb93gGV0cMlpA4ABsaPnWOZ9fFdxlY/TQsbr8AbBx3zmpWDveN9nydiHeuDyyj3m
         92p8RkLUEaG6CmYFr77QSQ9FLmNP1Q7yatZrNlVKaPGuou30d2bE9Sa7J7wciVcFRcrr
         hCOCcvPEWiEEMvQhEp63B+DDlhlX/HVgaup5JKKgFaNHQ82uOIGT3kMjFwIwf5Ur9ktK
         9y8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548937; x=1717153737;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fJnzblV3ATfhEvm8cpMubqr58bn3DL3t5seeZzPNX1U=;
        b=BIGttqGtX4y/OkCFwKU6PNh+kaOq/NP/EMdO/82f+6MLvHE+5ZV0/jDCb2VVhWtg/L
         VeTbFsPl5+MfPSAm7prLZUserSwW+PbtHVYMh2t1aOxxCpozYYh7oqO0LXiwWE/ajPvG
         WaFuvIeyl3JahajkWOzQqmQ44zF/h+iWys0YMxm3HHklR38iqAo7fK6eg2a+aTmY6sH6
         HzN2zM6/Gvd18oU4YNsWheCwX5EB28mppq9oDjTDBrr9w1TTtXlJoWa79wVS082nVY1e
         0ibXfPxwfQ1RbIbD/SXBUoZXIEyTY8qAMIzgn/rrAxrydXHwH6vi5xCsPjUPDB2mjVnJ
         xViQ==
X-Gm-Message-State: AOJu0YwNeC8JhjCSmLAjRX6ACRzm3o+f3LSqtXy/TaTXc2mvoL0npodU
	0+Bt5xOPPQQ5LkCTT5iPs5aLnGVTmyxnos/Y4sR9jEIu2fNGhbuAYOL2tPbj
X-Google-Smtp-Source: AGHT+IGQajXeuKHlMohaunJCnJ8WItq0bGBPNWrIA+V3IdrWxBKN5BeU3lg2l2s8Gh+mKxqfr0RkYw==
X-Received: by 2002:a17:906:f59f:b0:a59:bd14:1f51 with SMTP id a640c23a62f3a-a62645d654amr155463766b.41.1716548937002;
        Fri, 24 May 2024 04:08:57 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v11 0/9] Enable build of full Xen for RISC-V
Date: Fri, 24 May 2024 13:08:46 +0200
Message-ID: <cover.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.
As an option, it can be considered to merge arch-specific patch and then just
rebase [2] and drop arch-specific changes.

[1] https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

---
Changes in V11:
  - Patch was merged to staging:
    - [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
	  - [PATCH v10 06/14] xen/riscv: introduce atomic.h
	  - [PATCH v10 07/14] xen/riscv: introduce monitor.h
	  - [PATCH v10 09/14] xen/riscv: add required things to current.h
	  - [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V10:
  - Patch was merged to staging:
    - [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V9:
 - Patch was merged to staging:
    - [PATCH v8 07/17] xen/riscv: introduce io.h
  	- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V8:
 - Patch was merged to staging:
    - [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
    - [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
 - Update the commit message:
     - drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
     - Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---
Oleksii Kurochko (9):
  xen/riscv: disable unnecessary configs
  xen: introduce generic non-atomic test_*bit()
  xen/bitops: implement fls{l}() in common logic
  xen/riscv: introduce bitops.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                  |   4 +
 automation/gitlab-ci/build.yaml         |   4 +
 xen/arch/arm/include/asm/arm32/bitops.h |   2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |   6 +-
 xen/arch/arm/include/asm/bitops.h       |  78 +----
 xen/arch/ppc/include/asm/bitops.h       |  56 ----
 xen/arch/ppc/include/asm/page.h         |   2 +-
 xen/arch/ppc/mm-radix.c                 |   2 +-
 xen/arch/riscv/Makefile                 |  17 +-
 xen/arch/riscv/arch.mk                  |   4 -
 xen/arch/riscv/configs/tiny64_defconfig |  12 +-
 xen/arch/riscv/early_printk.c           | 168 ----------
 xen/arch/riscv/include/asm/bitops.h     | 137 ++++++++
 xen/arch/riscv/include/asm/cache.h      |   2 +
 xen/arch/riscv/include/asm/domain.h     |   2 +
 xen/arch/riscv/include/asm/mm.h         | 240 ++++++++++++++
 xen/arch/riscv/include/asm/p2m.h        |   2 +
 xen/arch/riscv/mm.c                     |  52 ++-
 xen/arch/riscv/setup.c                  |  10 +-
 xen/arch/riscv/stubs.c                  | 415 ++++++++++++++++++++++++
 xen/arch/riscv/traps.c                  |  25 ++
 xen/arch/x86/include/asm/bitops.h       |  43 +--
 xen/common/page_alloc.c                 |   4 +-
 xen/include/xen/bitops.h                | 209 ++++++++++++
 24 files changed, 1147 insertions(+), 349 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/stubs.c

-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729361.1134547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnM-0004id-FF; Fri, 24 May 2024 11:09:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729361.1134547; Fri, 24 May 2024 11:09:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnM-0004iT-Bq; Fri, 24 May 2024 11:09:08 +0000
Received: by outflank-mailman (input) for mailman id 729361;
 Fri, 24 May 2024 11:09:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnL-0003Cz-Bc
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:07 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 08c59efb-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:09:05 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so346715766b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:05 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08c59efb-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548944; x=1717153744; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iqJ1wT8hnhMs1hMxcLWkaeFwzBbEHjS02ZfwhfKipNg=;
        b=ioHp2lSbiIZZIKrgyYTGn7UBgOZx2/2Y9smawuaGbrnOkoNd98xs5HGSnNj8t9W2pI
         37MAV5HSCu1TB0bZT0MbiRXxgvs4atzc8T764CxCilONElazFRTzZJeq6sYoLzf7reQK
         rzp82Pi8wa72gdigqn0NdZ2ws6pDBiHnkcm/XEFGPiZxkSTKI05/cRaEDjWzux2RLEL3
         K8WR6aWBSSbh5ISgZ3b3c/AmJ1N7smDPRbbuRyZDsZuojn82tKN4J9DoepAcWz+XZXF1
         ZBcxFiNeXolVQbiS8ASjoBHckSwOJmennaFTBvSCY8mTbyzYhA5vO/MFLqJ9pn+026PU
         RiOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548944; x=1717153744;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iqJ1wT8hnhMs1hMxcLWkaeFwzBbEHjS02ZfwhfKipNg=;
        b=LPIIc/hVIkF9f9Vjdvu2pB098lnTps+U8GDzFGXr/FHlZ+Vetn+pV99UO1HtLxHMMH
         8WvLZHMu5lzVH0fiz7Qq0oVbezygYhWL/E84fEA6oq7bAfahzZDNpmajEnamIuRvnJYs
         DHiXAKNCpEFIyc2gEjZhAagVK+HAp/Ph8wB4FfhGvy8BUBmJTDii5Ux4/0IrXHdh9WHc
         s4Fr/lxYBdW5CMECDt6gwj+GYi92LAw18oS0/OTJ2JB57CbZusl9hB580ou1nNopiHiQ
         3En3VWVonsN5ZdxP1Ia0zkjDQrWS1qR7CVD4BchffNQpEBKttpiLsz9n0NZEdbXCB8qM
         UNAQ==
X-Gm-Message-State: AOJu0Yz7qx8GmULwdWb2jhgKkNh31hW7Ow4rYmm8d11LMccphI0tYDnd
	NDP9grsCtWsKnflaP+kWaLtt2Lty/9FABO42KP6kyg47CJ6GUdHbqOF4smyt
X-Google-Smtp-Source: AGHT+IG0VXTttRuxlnlxcJlGUr29VJYw1XA7TbNsldzb3AMHVdhNRXxQyQCA3jE9fmND9BmfE1pu4A==
X-Received: by 2002:a17:906:a009:b0:a5a:3ea6:689e with SMTP id a640c23a62f3a-a626536efeemr136080866b.71.1716548944270;
        Fri, 24 May 2024 04:09:04 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 6/9] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Fri, 24 May 2024 13:08:52 +0200
Message-ID: <55506cc9a19615a6f25b4b989c739a894b308968.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V8-V11:
 - Nothing changed only rebase.
---
Changes in V7:
 - update argument type of maddr_to_virt() function: unsigned long -> paddr_t
 - rename argument of PFN_ORDER(): pfn -> pg.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V6:
 - drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
 - parenthesize va in definition of vmap_to_mfn().
 - Code style fixes.
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 242 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..cc4a07a71c 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,246 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729362.1134556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnN-000523-VA; Fri, 24 May 2024 11:09:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729362.1134556; Fri, 24 May 2024 11:09:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnN-00051d-R2; Fri, 24 May 2024 11:09:09 +0000
Received: by outflank-mailman (input) for mailman id 729362;
 Fri, 24 May 2024 11:09:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnM-0003D0-55
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:08 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 09f980d5-19be-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:09:07 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5751bcb3139so10252483a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:07 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09f980d5-19be-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548947; x=1717153747; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XB3BQ9Ks71hVlpF8icLLnQrVXglnlCkVcQJiFBF5cYk=;
        b=kCmgAl7x5+CvmMKUDN4s9Q5mzDfAcb187cuBbQdXMUWF/A1rMWO1lH+YhZ2cBJW6CR
         DXsigv8yGPDBjOTbqoQVNrMDREYO73FFLTR6FhHreDA5jxwFWJWizbjvsIsVXmTtQk/I
         jkfsICERj2SdZ36n/YAYTPf2wLbnjg22F9buRVGmwhkyYuC8F2+y/HWxyUHU6XdhOSk2
         zeNCs+q4jodZC543tDdtOIkP68SRs8NPmnhWBMVdKiOy34hBhbooGB8Xq6+/ey1DL3UL
         1F3wh9wKTJqtsFe/YvqbYtcRUy0c4REdv++ssRZxxJFW+ASKGgS8JTnpGoz/LKdfwM82
         NR1g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548947; x=1717153747;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XB3BQ9Ks71hVlpF8icLLnQrVXglnlCkVcQJiFBF5cYk=;
        b=uWWYepng1XWP7ZkRlq5irqvxGjw62quLY7PJNR9nql1By0eMlZOuWCgjEPTI+C7ZFS
         Jiiyc3CgAJGDEuMQlFCBsnTHCRdG4yygg1m6THcSCH3VnXcwICmJD3mUzRj1nFrbQdoW
         JqCW+VvWh0so51+nMUYZaN1oGMgo/TZ6YEV17Tk+WwlkuiONEMjc0+2QxGMVRakmvPn9
         7fz/zLjEMz8X2dGgEaL/hSuHZ5c3LAecnprWEVvkF3r2nx8IquFf/r8NWANzlsD3BP1P
         akCe79Dza7oJnmFyAYf0gpm/jCGxoLpVjoQt6JABh/hF03oWW5aGJW1Xu9lIotxe782H
         aRDw==
X-Gm-Message-State: AOJu0YxHu/dwaOBhIBvE9dqSC+RbuWyBh3wjn2asdCydlnE3NGGZeijQ
	guTRukwcWxmlWKeShxrIJ/zwpe51nfmhaeanE3ZXAxn0WAn4XYIREgIXZ+bX
X-Google-Smtp-Source: AGHT+IE/+7IppVaU2G3CaM5P+fJkei1eqO2U3AsyLE9MeoJBZJMccDaLoNzpcShLv8KPR9U78aVM0g==
X-Received: by 2002:a17:907:60a:b0:a5c:db98:c29e with SMTP id a640c23a62f3a-a62641b4507mr143859366b.18.1716548946975;
        Fri, 24 May 2024 04:09:06 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 9/9] xen/README: add compiler and binutils versions for RISC-V64
Date: Fri, 24 May 2024 13:08:55 +0200
Message-ID: <b2964e31488c735344eb33e64e60d0eb1f791a59.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
--
Changes in V5-V11:
 - Nothing changed. Only rebase.
---
 Changes in V6:
  - update the message in README.
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README b/README
index c8a108449e..30da5ff9c0 100644
--- a/README
+++ b/README
@@ -48,6 +48,10 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+          Older GCC and GNU Binutils would work, but this is not a guarantee.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729363.1134562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnO-00054W-CK; Fri, 24 May 2024 11:09:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729363.1134562; Fri, 24 May 2024 11:09:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnO-00053h-2Q; Fri, 24 May 2024 11:09:10 +0000
Received: by outflank-mailman (input) for mailman id 729363;
 Fri, 24 May 2024 11:09:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnM-0003Cz-Bu
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:08 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0919f488-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:09:06 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso128759731fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:06 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0919f488-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548945; x=1717153745; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8soMyFnI8t1ld70bWlw2X55VwVVkeYWhI/zk046fYOA=;
        b=HEx2jywNHGwkZybkNLgWDx0mC/+qt5l3sGLf5NuXHUO2IZ3pvcEPgRdojYbVvmgJCs
         W9FbAAPhiN58yur60+rodF1S2FBgRbalhhVofLW4xH6B0eFHS9Vd9P9O6uUFrYkRdsBl
         Cks38qS1vTJKEZlmdK595VFkdfm4Mc8HwcjRsibATWSieDcmNUz0ppCdqeGDKFHmGfgi
         dAD44s83iY6SRVbah91TAey0vl0QWc2h+RpetKzUUR2/e3Ff4/pd4umohcASkGP3pjw+
         jVPhUst6XIa6PPQwamm8rUV+ya/Ca/HgIb5Vt7Oul6Pi4yI4ySCOUr/Z3i/ly/clqlXD
         8Tfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548945; x=1717153745;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8soMyFnI8t1ld70bWlw2X55VwVVkeYWhI/zk046fYOA=;
        b=RKJikxWZunn7LmUCDZdlokyHaTFzH6JYdLmhPUlaakiB74+RpooJJmCODOiFNBF02B
         9wP210vDUDXyvI1OCLEiWfiZEQHdW6xj8gQvGS91cpp2WfAsFADAFZWyuvW48K7hXXSG
         J0cPDG0Hr5ZGWH1XzrRGAPSacLFVJZf4aVGcdsHy8aP3HdwLvXGSQxcbm049xR2ICIed
         5NfPj83EcWVggRulK4vM/ik+oTEvxY8HVWP2PHW1XmDvOtpq7YgzNgDbW48c799ZvhuH
         IP3VTqYSiDy9P5yi1PqWUyKWkdOXAIIZHegntPiy71DMOsmZbNYsigM4Eo/Jv15IFfWK
         kbXg==
X-Gm-Message-State: AOJu0Yz5m+G8ZV811FkSj1fShJBeKVdjNf7BfylMPsjzLYinjd9Sl0Cx
	+t1zxocMYjfVGjF8Cce1GETATnGCAeeixX9O14uQzk1JYEldnngSDQy7D6/E
X-Google-Smtp-Source: AGHT+IFMmS7tWE3sAjBB+4Kh33RHI5xwLFwXue+VgMCUrIRS+RXeg5+XNW1aif+08Wj0lxUzWN07Qg==
X-Received: by 2002:a2e:b2c7:0:b0:2e8:5d2:5e9b with SMTP id 38308e7fff4ca-2e95b096b9cmr17169121fa.13.1716548945146;
        Fri, 24 May 2024 04:09:05 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 7/9] xen/riscv: add minimal amount of stubs to build full Xen
Date: Fri, 24 May 2024 13:08:53 +0200
Message-ID: <94d628999fd4cf32d7ca82a5f6f20297d06d3cef.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7-V11:
 - Only rebase was done.
---
Changes in V6:
 - update the commit in stubs.c around /* ... common/irq.c ... */
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 439 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 523 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..8285bcffef
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, but common/irq.c isn't
+ * built for now. These changes will be removed there when common/irq.c is
+ * ready.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:09:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:09:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729364.1134571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnP-0005FR-3a; Fri, 24 May 2024 11:09:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729364.1134571; Fri, 24 May 2024 11:09:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASnO-0005CV-N3; Fri, 24 May 2024 11:09:10 +0000
Received: by outflank-mailman (input) for mailman id 729364;
 Fri, 24 May 2024 11:09:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vqrK=M3=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sASnN-0003Cz-Bt
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:09:09 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09aac8da-19be-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:09:06 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-529614b8c29so1080110e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:09:06 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817c81sm116051366b.28.2024.05.24.04.09.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:09:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09aac8da-19be-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716548946; x=1717153746; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=b94wI6kNLTmJ8tMNKvMfG5/MPujiYvHPiuL4CtLKoSc=;
        b=TGApL4xbpj1bi+C+A/R1tnjLrosORRiReivdaxuhQOpKsnNvCNJ+0seR7+K0OZpTlo
         sUo+XqjON0yg0Xr5KPiOj5+KCToKwH4dvXM5AS4D/Eml9gfVI67C1ignHcYCfTfW8V4d
         sKgxw8BEyaQFuBYwBQpjfynBvC7MiK5rrFdHSzFJRCt3EpxSJXh688seRzHWcrwBJ5iU
         vCa2xMa87+iHMeNVjZVBq7v3YFM0HL8Pu2pILBHqTBbKISXCt50nfiRmYyz/umwAFU6e
         a6MtJ9GSnkVeF4r31tMNiuPakaoQnXKgVQGN8gTbXiFIT9PZXy7GXX+VrgNN6q1D9TYm
         Hc3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716548946; x=1717153746;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=b94wI6kNLTmJ8tMNKvMfG5/MPujiYvHPiuL4CtLKoSc=;
        b=R5bDn17ycChOAUX2Nq4Qbjw2NAsVDpwpSy1nsdC9x839okL7JPI8SnGdwKTKtc+wlG
         p4zoZjC9CxoxUn9Ty7bpF5yDuBPtij2O5qlg/bI6UT+rn02XfYJe57ZMxTp9hfnF+5A5
         6dWRc4bkRPAnCOTvINZK6GPhmCwLixdMDX8aJBbdz6MAc07+PsEGdSy49sOiiS8ubA7H
         JfzbT1B37Wkm96dYrEwoFtoGsSscdsGMZyaDEu87yyXoix/OfFRCYQPPMw4nZuuwdg3q
         NbCKtLEFD3t79+F3zNDvU2j4pVBILfe8Dtok0kQsQu1V7FdqXtinTabRXndwuEdp2ofz
         aSBQ==
X-Gm-Message-State: AOJu0YzATpinc9dqHr10zXU+MqqBlK8oQc9+fBM1rR1sluWGb1VA4RVk
	nF7TR39MxXvsHbaZFAwGRpzgw89eiBArDZfQNFoObPCiVImfU4PdRhH58tLS
X-Google-Smtp-Source: AGHT+IGTH7Y8u4ycvlLBGeDfRcjwpph4JM2NgKLV/prJHx2KwU7K2TBeDUfL+BiJjbWFST7ABLYYbA==
X-Received: by 2002:ac2:4c49:0:b0:51d:8842:f0b8 with SMTP id 2adb3069b0e04-5296529053bmr2240082e87.13.1716548946203;
        Fri, 24 May 2024 04:09:06 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v11 8/9] xen/riscv: enable full Xen build
Date: Fri, 24 May 2024 13:08:54 +0200
Message-ID: <0a684923efb23fb6094aeab680022b6b14196eef.1716547693.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1716547693.git.oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 At least this patch cann't be merged w/o Andrew's patch series is merged as ffs related
 functions are used from that patch series:
  https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
---
Changes in V5-V11:
 - Nothing changed. Only rebase.
 - Add the footer after Signed-off section.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---

 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  24 -----
 4 files changed, 15 insertions(+), 197 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8c071aff65..17827c302c 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 8285bcffef..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, but common/irq.c isn't
- * built for now. These changes will be removed there when common/irq.c is
- * ready.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 11:15:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:15:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729406.1134586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAStR-0002rZ-UZ; Fri, 24 May 2024 11:15:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729406.1134586; Fri, 24 May 2024 11:15:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAStR-0002rS-Rr; Fri, 24 May 2024 11:15:25 +0000
Received: by outflank-mailman (input) for mailman id 729406;
 Fri, 24 May 2024 11:15:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAStQ-0002rJ-Bi
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:15:24 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e9e550e1-19be-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 13:15:23 +0200 (CEST)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-4202ca70270so36382055e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:15:23 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4210897c66esm17816325e9.26.2024.05.24.04.15.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 04:15:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9e550e1-19be-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716549323; x=1717154123; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Yt0ufe3+7IhTAuIyCHiiU+nbTX+TPtsjdalzDA21Zmw=;
        b=puhLmWxt7lsaFd98Knt5vTczwRjw1R6wH518L8ssBR9B7dBhysZFRQQ/ltiiQVNwJZ
         otjvxPi8Vp7DDOWivujQcIu2liy7yvTZb34fILwmEC9N5Y+oTNIo9qbX6zSYR2zHV/hV
         02KwvREXd6NRf80jfU44AXSjcr3YWU6EjMOEA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716549323; x=1717154123;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Yt0ufe3+7IhTAuIyCHiiU+nbTX+TPtsjdalzDA21Zmw=;
        b=XwFiC2O9ndsFC9J3f1bBDYkl8YFLJLElB2jXV6PEE+CDkTHD6Y6VFCotM9UimJL+93
         Fxq8ZYMB6Pa5fTCz/q/9k/qWWa3VxsOpwfHLYDNIFYr2ah17EHJf+xkCDnvcuLAQhIlX
         Jyn/bcDQQmjNkTwhMNv26JtCf5u0qMWMHGclIUyPIHq12rTzfEEoAxydnK5Z9NGSeGyQ
         MoHuEIa9UKmLO1kHx8tdI1945ULA4u1Zsy1rlXQGuMInxioqoSrb6CqJHuJcXCv8iQhV
         rXfauC6JAS1oTV1oIP3bdC4AgH7w/ETluEGXUqydMJp7WJQGwVvF8s1KHCQHGeOqvi4P
         LkMw==
X-Gm-Message-State: AOJu0Yx2Scey38l9NwYnIphiTbf4+ahoehfMf7594pVCUPIDKDnCDuqe
	HAY5/xOHcQECW/fy1xYyvphqKpxyhpbZZF4D4owy4t83X4Qemj3jtLcouXoTuVM=
X-Google-Smtp-Source: AGHT+IGzBm2+zUjErnX1Wm++atYMIRi6N1NeXAdIXKNmYaztjiAvuQLNEYbrl0qpRiHv9WrAuf/RWw==
X-Received: by 2002:a05:600c:3509:b0:41f:e7e6:7f27 with SMTP id 5b1f17b1804b1-42108a99f23mr18518045e9.26.1716549322693;
        Fri, 24 May 2024 04:15:22 -0700 (PDT)
Date: Fri, 24 May 2024 13:15:21 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 1/8] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
Message-ID: <ZlB2yZNkZ07_wVAC@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <4095f31a88589ced2b620e8ebbb84cdc2fae8914.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9TZhe-WSNrnlPD@macbook>
 <dbd87d6b-c19a-48cc-8838-32b958da980d@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <dbd87d6b-c19a-48cc-8838-32b958da980d@cloud.com>

On Fri, May 24, 2024 at 11:58:44AM +0100, Alejandro Vallejo wrote:
> On 23/05/2024 15:32, Roger Pau Monné wrote:
> >>      case 0xb:
> >> -        /*
> >> -         * In principle, this leaf is Intel-only.  In practice, it is tightly
> >> -         * coupled with x2apic, and we offer an x2apic-capable APIC emulation
> >> -         * to guests on AMD hardware as well.
> >> -         *
> >> -         * TODO: Rework topology logic.
> >> -         */
> >> -        if ( p->basic.x2apic )
> >> +        /* Don't expose topology information to PV guests */
> > 
> > Not sure whether we want to keep part of the comment about exposing
> > x2APIC to guests even when x2APIC is not present in the host.  I think
> > this code has changed and the comment is kind of stale now.
> 
> The comment is definitely stale. Nowadays x2APIC is fully supported by
> AMD, as is leaf 0xb. The fact we emulate the x2APIC seems hardly
> relevant in a CPUID leaf about topology. I could keep a note showing...
> 
>     /* Exposed alongside x2apic, as it's tightly coupled with it */
> 
> ... although that's directly implied by the conditional.

Yeah, I haven't gone through the history of this file, but I bet at
some point before the introduction of CPUID policies we leaked (part
of) the host CPUID contents in here.

It's also no longer true that the leaf is Intel only.

I fine either adding your newly proposed comment, or leaving it as-is.

> >> +}
> >> +
> >>  int guest_wrmsr_apic_base(struct vcpu *v, uint64_t val)
> >>  {
> >>      const struct cpu_policy *cp = v->domain->arch.cpu_policy;
Le> >> @@ -1449,7 +1465,7 @@ void vlapic_reset(struct vlapic *vlapic)
> >>      if ( v->vcpu_id == 0 )
> >>          vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
> >>  
> >> -    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
> >> +    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
> >>      vlapic_do_init(vlapic);
> >>  }
> >>  
> >> @@ -1514,6 +1530,16 @@ static void lapic_load_fixup(struct vlapic *vlapic)
> >>      const struct vcpu *v = vlapic_vcpu(vlapic);
> >>      uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id);
> >>  
> >> +    /*
> >> +     * Loading record without hw.x2apic_id in the save stream, calculate using
> >> +     * the traditional "vcpu_id * 2" relation. There's an implicit assumption
> >> +     * that vCPU0 always has x2APIC0, which is true for the old relation, and
> >> +     * still holds under the new x2APIC generation algorithm. While that case
> >> +     * goes through the conditional it's benign because it still maps to zero.
> >> +     */
> >> +    if ( !vlapic->hw.x2apic_id )
> >> +        vlapic->hw.x2apic_id = v->vcpu_id * 2;
> >> +
> >>      /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */
> >>      if ( !vlapic_x2apic_mode(vlapic) ||
> >>           (vlapic->loaded.ldr == good_ldr) )
> >> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
> >> index 0c9e6f15645d..e1f0585d75a9 100644
> >> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> >> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> >> @@ -448,6 +448,7 @@ static inline void hvm_update_guest_efer(struct vcpu *v)
> >>  static inline void hvm_cpuid_policy_changed(struct vcpu *v)
> >>  {
> >>      alternative_vcall(hvm_funcs.cpuid_policy_changed, v);
> >> +    vlapic_cpu_policy_changed(v);
> > 
> > Note sure whether this call would better be placed in
> > cpu_policy_updated() inside the is_hvm_vcpu() conditional branch.
> > 
> > hvm_cpuid_policy_changed() &c are just wrappers around the hvm_funcs
> > hooks, pulling vlapic functions in there is likely to complicate the
> > header dependencies in the long term.
> > 
> 
> That's how it was in v1 and I moved it in v2 answering one of Jan's
> feedback points.
> 
> I don't mind either way.

Oh (goes and reads Jan's reply to v1) I see.  Let's leave it as-is
then.

> 
> >>  }
> >>  
> >>  static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
> >> diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
> >> index 88ef94524339..e8d41313abd3 100644
> >> --- a/xen/arch/x86/include/asm/hvm/vlapic.h
> >> +++ b/xen/arch/x86/include/asm/hvm/vlapic.h
> >> @@ -44,6 +44,7 @@
> >>  #define vlapic_xapic_mode(vlapic)                               \
> >>      (!vlapic_hw_disabled(vlapic) && \
> >>       !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD))
> >> +#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id)
> >>  
> >>  /*
> >>   * Generic APIC bitmap vector update & search routines.
> >> @@ -107,6 +108,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack);
> >>  
> >>  int  vlapic_init(struct vcpu *v);
> >>  void vlapic_destroy(struct vcpu *v);
> >> +void vlapic_cpu_policy_changed(struct vcpu *v);
> >>  
> >>  void vlapic_reset(struct vlapic *vlapic);
> >>  
> >> diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
> >> index 7ecacadde165..1c2ec669ffc9 100644
> >> --- a/xen/include/public/arch-x86/hvm/save.h
> >> +++ b/xen/include/public/arch-x86/hvm/save.h
> >> @@ -394,6 +394,8 @@ struct hvm_hw_lapic {
> >>      uint32_t             disabled; /* VLAPIC_xx_DISABLED */
> >>      uint32_t             timer_divisor;
> >>      uint64_t             tdt_msr;
> >> +    uint32_t             x2apic_id;
> >> +    uint32_t             rsvd_zero;
> > 
> > I think Jan requested for this field to be checked to be 0 for
> > incoming migrations, yet I don't see such check added.
> > 
> 
> It's on the next patch, when the checks are moved into the check hook.
> Doing it all as part of the same patch seemed dirty. I guess I can
> invert them.

Yeah, sorry, realized later.  I think the introduction of the lapic
record checking wants to be the first patch in the series, so that
when you introduce the field here you can check it's zero.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 11:16:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 11:16:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729409.1134597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASu9-0003KO-7O; Fri, 24 May 2024 11:16:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729409.1134597; Fri, 24 May 2024 11:16:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sASu9-0003KH-4W; Fri, 24 May 2024 11:16:09 +0000
Received: by outflank-mailman (input) for mailman id 729409;
 Fri, 24 May 2024 11:16:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sASu8-0003K3-0X
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 11:16:08 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 017ba667-19bf-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 13:16:02 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a6265d3ba8fso69688666b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 04:16:02 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8e2b2sm116641866b.171.2024.05.24.04.16.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 04:16:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 017ba667-19bf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716549362; x=1717154162; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LErNZqDdT1LDKm+B+1gqdgsfgkPjrgXE96aziTJ654M=;
        b=O0fV8+BeehFz3tkqFSf2Oy3Az4qwv1I8ue45oh9WX3D5jA71sdf71w756IwQM6ijfH
         NTvZHHOorD7xHyn2bLo6vY08BkpcdJpj8bKNnmDLOEdK/XWzDYC+bpqTS7C1qUZrWzXo
         DnRRtm72T045nqAJ7r6S9gw319fFRXvTN3m5A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716549362; x=1717154162;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LErNZqDdT1LDKm+B+1gqdgsfgkPjrgXE96aziTJ654M=;
        b=eSch5IAoIrNEnjPVTSmZxlT36ttnXRinj0kH6XcSzPTzretH1wp5ZrRP2fTTDgMoah
         FQJYmPV9YaibGWqTWWMRmaGQwsW6JAq/FS8WXv+ZrCc8Zuecp3AwTW0PX54h+jZPEKWM
         eC2+JTrVo8rYmb/sOy85KLy6/e96rCHb0VI3BS2nk15uFQvQ3h0iv9a2z0aM6puQUMsj
         e2H3o7dkNVLGcF0K8OgGjOF10bae80CZh13gtEnTsclYD7o0MtSk2BrDXNM5Q+Xyi7Qf
         NlOTmqkzHSNgn4SmrPzI9KczThwdeYu1kWv45xO4H9dUzKEHioBPWTPugliID6KNkl3r
         zpLQ==
X-Gm-Message-State: AOJu0YwotMVCfO4M05eGEjgLiiBsGHG+/1MkZ3CDZvWWONYcd3vQJVxo
	6tGfoWs25Kz1Aa6aYv2Yt6quD3/BNBJ+wh77+qRKr6gsqCoT0+4StHmvyFrti6E=
X-Google-Smtp-Source: AGHT+IFvAbBYwIfdM6sb67Rlktjl3lRkBTwQx2ML+6+wxE1Y5nXsbhxs98rjKHoeHWHh5vdPjqDIzw==
X-Received: by 2002:a17:907:3a13:b0:a59:cb29:3fb5 with SMTP id a640c23a62f3a-a62641de8a2mr113291766b.20.1716549362182;
        Fri, 24 May 2024 04:16:02 -0700 (PDT)
Message-ID: <a1f7a182-4cf2-4fd1-ba8f-5b12a34b1af0@cloud.com>
Date: Fri, 24 May 2024 12:16:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration
 checks to the check hook
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9Xy7W30WWZlb6-@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zk9Xy7W30WWZlb6-@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 15:50, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:22PM +0100, Alejandro Vallejo wrote:
>> While at it, add a check for the reserved field in the hidden save area.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * New patch. Addresses the missing check for rsvd_zero in v1.
> 
> Oh, it would be better if this was done at the time when rsvd_zero is
> introduced.  I think this should be moved ahead of the series, so that
> the patch that introduces rsvd_zero can add the check in
> lapic_check_hidden().

I'll give that a whirl.

> 
>> ---
>>  xen/arch/x86/hvm/vlapic.c | 41 ++++++++++++++++++++++++++++-----------
>>  1 file changed, 30 insertions(+), 11 deletions(-)
>>
>> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
>> index 8a244100009c..2f06bff1b2cc 100644
>> --- a/xen/arch/x86/hvm/vlapic.c
>> +++ b/xen/arch/x86/hvm/vlapic.c
>> @@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>>                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
>>  }
>>  
>> -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
>> +static int cf_check lapic_check_hidden(const struct domain *d,
>> +                                       hvm_domain_context_t *h)
>>  {
>>      unsigned int vcpuid = hvm_load_instance(h);
>> -    struct vcpu *v;
>> -    struct vlapic *s;
>> +    struct hvm_hw_lapic s;
>>  
>>      if ( !has_vlapic(d) )
>>          return -ENODEV;
>>  
>>      /* Which vlapic to load? */
>> -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
>> +    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )
>>      {
>>          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
>>                  d->domain_id, vcpuid);
>>          return -EINVAL;
>>      }
>> -    s = vcpu_vlapic(v);
>>  
>> -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
>> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )
> 
> Can't you use hvm_get_entry() to perform the sanity checks:
> 
> const struct hvm_hw_lapic *s = hvm_get_entry(LAPIC, h);
> 
> Thanks, Roger.

I don't think I can. Because the last field (rsvd_zero) might or might
not be there, so it needs to be zero-extended. Unless I misunderstood
what hvm_get_entry() is meant to do. It seems to check for exact sizes.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 12:31:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:31:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729463.1134617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAU59-0005JJ-4D; Fri, 24 May 2024 12:31:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729463.1134617; Fri, 24 May 2024 12:31:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAU59-0005JA-0G; Fri, 24 May 2024 12:31:35 +0000
Received: by outflank-mailman (input) for mailman id 729463;
 Fri, 24 May 2024 12:31:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gspi=M3=bounce.vates.tech=bounce-md_30504962.665088a1.v1-da0278549b014913b2e166d18d4b292c@srs-se1.protection.inumbo.net>)
 id 1sAU57-00051V-Ae
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:31:33 +0000
Received: from mail136-17.atl41.mandrillapp.com
 (mail136-17.atl41.mandrillapp.com [198.2.136.17])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8c00e4f2-19c9-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:31:30 +0200 (CEST)
Received: from pmta11.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1])
 by mail136-17.atl41.mandrillapp.com (Mailchimp) with ESMTP id
 4Vm4Cj5TjBzPm0PQQ
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 12:31:29 +0000 (GMT)
Received: from [37.26.189.201] by mandrillapp.com id
 da0278549b014913b2e166d18d4b292c; Fri, 24 May 2024 12:31:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c00e4f2-19c9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com;
	s=mte1; t=1716553889; x=1716814389;
	bh=vQSwGscX7oGUBHgB5gR11ubLD7SNlB1T+5N3wTUTxgQ=;
	h=From:Subject:To:Cc:Message-Id:In-Reply-To:References:Feedback-ID:
	 Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date:
	 Subject:From;
	b=AWWwB9ddXa9t7TcsC1gN4YLINavWjjn5iVZnWJb+XtrdDlUSOFxkL9PL5GyvwEkPl
	 R2sIV3ExJ53hAMgEh45/VW50LHla6MCXg4clKVNSFtQhBo0YRIHeyDyhp+6dBIWVVO
	 W2qJex6ac831sps+yH4bDzEoMS9XL0n7N9YTg35OcM63CGTzset8bLCTWyU0p1SVIa
	 sS8lFSDPSxioFmy8KLnV28rV+aM6wtoBvXKjNFm7epLDug8LAcGVD9VC6zndz+NfAA
	 RxqO9LHrnKTxtc0zmkFGx2q/oy1kQJKO9Q91On6NylJ5mO4j9QCnMIS2boqM/K6Ofx
	 Qt/9jNj83GUGg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1;
	t=1716553889; x=1716814389; i=vaishali.thakkar@vates.tech;
	bh=vQSwGscX7oGUBHgB5gR11ubLD7SNlB1T+5N3wTUTxgQ=;
	h=From:Subject:To:Cc:Message-Id:In-Reply-To:References:Feedback-ID:
	 Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date:
	 Subject:From;
	b=teUPaiNHVWeCSKcs1oZc2IoX4tLO0+6BSrIGRBtrRe+zEby2l/NwPFNvhDq/n8liA
	 42Oj2e4Gv0N3ktdJiEDfx2ac2037FphrMFkMYdLM8skS93qlUfRiiorkCz+EU48nEQ
	 wEkzNtLv1UxMz7QUzyi456ZFx1rPQ6UAEXs9yJnnd8BWKi/KrEr0DGwwhnXS+TtNQq
	 XhJR6RFrIgDXaMbD0fYEojk3JT8/wFTN6LNZBUNuYcjeQsFGgE0wFgtnRD5BY0QD+5
	 sr+8tQX+zxVPwagEq67LyyvDtQz86rPPf/mtQNQOWr+W2ySs47t4FOgdshyVVfnjJS
	 Tr7SSGNXHiphA==
From: Vaishali Thakkar <vaishali.thakkar@vates.tech>
Subject: =?utf-8?Q?[RFC=20for-4.20=20v1=201/1]=20x86/hvm:=20Introduce=20Xen-wide=20ASID=20Allocator?=
X-Mailer: git-send-email 2.45.0
X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2
X-Bm-Transport-Timestamp: 1716553888282
To: xen-devel@lists.xenproject.org
Cc: jbeulich@suse.com, andrew.cooper3@citrix.com, roger.pau@citrix.com, george.dunlap@citrix.com, Vaishali Thakkar <vaishali.thakkar@vates.tech>
Message-Id: <f15042aa7953d986b6dbd4dc1512024ba6362420.1716551380.git.vaishali.thakkar@vates.tech>
In-Reply-To: <cover.1716551380.git.vaishali.thakkar@vates.tech>
References: <cover.1716551380.git.vaishali.thakkar@vates.tech>
X-Native-Encoded: 1
X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.da0278549b014913b2e166d18d4b292c?=
X-Mandrill-User: md_30504962
Feedback-ID: 30504962:30504962.20240524:md
Date: Fri, 24 May 2024 12:31:29 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Currently ASID generation and management is done per-PCPU. This
scheme is incompatible with SEV technologies as SEV VMs need to
have a fixed ASID associated with all vcpus of the VM throughout
it's lifetime.

This commit introduces a Xen-wide allocator which initializes
the asids at the start of xen and allows to have a fixed asids
throughout the lifecycle of all domains. Having a fixed asid
for non-SEV domains also presents us with the opportunity to
further take use of AMD instructions like TLBSYNC and INVLPGB
for broadcasting the TLB invalidations.

Signed-off-by: Vaishali Thakkar <vaishali.thakkar@vates.tech>
---
 xen/arch/x86/flushtlb.c                |  1 -
 xen/arch/x86/hvm/asid.c                | 61 ++++++++++++++------------
 xen/arch/x86/hvm/emulate.c             |  3 --
 xen/arch/x86/hvm/hvm.c                 |  2 +-
 xen/arch/x86/hvm/nestedhvm.c           |  4 +-
 xen/arch/x86/hvm/svm/asid.c            | 28 +++++++-----
 xen/arch/x86/hvm/svm/nestedsvm.c       |  2 +-
 xen/arch/x86/hvm/svm/svm.c             | 33 ++++++--------
 xen/arch/x86/hvm/svm/svm.h             |  1 -
 xen/arch/x86/hvm/vmx/vmcs.c            |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c             | 15 +++----
 xen/arch/x86/hvm/vmx/vvmx.c            |  6 +--
 xen/arch/x86/include/asm/hvm/asid.h    | 19 ++++----
 xen/arch/x86/include/asm/hvm/domain.h  |  1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  2 +-
 xen/arch/x86/include/asm/hvm/svm/svm.h |  1 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  6 +--
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  3 +-
 xen/arch/x86/mm/hap/hap.c              |  4 +-
 xen/arch/x86/mm/p2m.c                  |  6 +--
 xen/arch/x86/mm/paging.c               |  2 +-
 xen/arch/x86/setup.c                   |  7 +++
 22 files changed, 108 insertions(+), 101 deletions(-)

diff --git a/xen/arch/x86/flushtlb.c b/xen/arch/x86/flushtlb.c
index 18748b2bc8..69d72944d7 100644
--- a/xen/arch/x86/flushtlb.c
+++ b/xen/arch/x86/flushtlb.c
@@ -124,7 +124,6 @@ void switch_cr3_cr4(unsigned long cr3, unsigned long cr4)
 
     if ( tlb_clk_enabled )
         t = pre_flush();
-    hvm_flush_guest_tlbs();
 
     old_cr4 = read_cr4();
     ASSERT(!(old_cr4 & X86_CR4_PCIDE) || !(old_cr4 & X86_CR4_PGE));
diff --git a/xen/arch/x86/hvm/asid.c b/xen/arch/x86/hvm/asid.c
index 8d27b7dba1..f343bfcdb9 100644
--- a/xen/arch/x86/hvm/asid.c
+++ b/xen/arch/x86/hvm/asid.c
@@ -27,8 +27,8 @@ boolean_param("asid", opt_asid_enabled);
  * the TLB.
  *
  * Sketch of the Implementation:
- *
- * ASIDs are a CPU-local resource.  As preemption of ASIDs is not possible,
+ * TODO(vaishali): Update this comment
+ * ASIDs are Xen-wide resource.  As preemption of ASIDs is not possible,
  * ASIDs are assigned in a round-robin scheme.  To minimize the overhead of
  * ASID invalidation, at the time of a TLB flush,  ASIDs are tagged with a
  * 64-bit generation.  Only on a generation overflow the code needs to
@@ -38,20 +38,21 @@ boolean_param("asid", opt_asid_enabled);
  * ASID useage to retain correctness.
  */
 
-/* Per-CPU ASID management. */
+/* Xen-wide ASID management */
 struct hvm_asid_data {
-   uint64_t core_asid_generation;
+   uint64_t asid_generation;
    uint32_t next_asid;
    uint32_t max_asid;
+   uint32_t min_asid;
    bool disabled;
 };
 
-static DEFINE_PER_CPU(struct hvm_asid_data, hvm_asid_data);
+static struct hvm_asid_data asid_data;
 
 void hvm_asid_init(int nasids)
 {
     static int8_t g_disabled = -1;
-    struct hvm_asid_data *data = &this_cpu(hvm_asid_data);
+    struct hvm_asid_data *data = &asid_data;
 
     data->max_asid = nasids - 1;
     data->disabled = !opt_asid_enabled || (nasids <= 1);
@@ -64,67 +65,73 @@ void hvm_asid_init(int nasids)
     }
 
     /* Zero indicates 'invalid generation', so we start the count at one. */
-    data->core_asid_generation = 1;
+    data->asid_generation = 1;
 
     /* Zero indicates 'ASIDs disabled', so we start the count at one. */
     data->next_asid = 1;
 }
 
-void hvm_asid_flush_vcpu_asid(struct hvm_vcpu_asid *asid)
+void hvm_asid_flush_domain_asid(struct hvm_domain_asid *asid)
 {
     write_atomic(&asid->generation, 0);
 }
 
-void hvm_asid_flush_vcpu(struct vcpu *v)
+void hvm_asid_flush_domain(struct domain *d)
 {
-    hvm_asid_flush_vcpu_asid(&v->arch.hvm.n1asid);
-    hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
+    hvm_asid_flush_domain_asid(&d->arch.hvm.n1asid);
+    //hvm_asid_flush_domain_asid(&vcpu_nestedhvm(v).nv_n2asid);
 }
 
-void hvm_asid_flush_core(void)
+void hvm_asid_flush_all(void)
 {
-    struct hvm_asid_data *data = &this_cpu(hvm_asid_data);
+    struct hvm_asid_data *data = &asid_data;
 
-    if ( data->disabled )
+    if ( data->disabled)
         return;
 
-    if ( likely(++data->core_asid_generation != 0) )
+    if ( likely(++data->asid_generation != 0) )
         return;
 
     /*
-     * ASID generations are 64 bit.  Overflow of generations never happens.
-     * For safety, we simply disable ASIDs, so correctness is established; it
-     * only runs a bit slower.
-     */
+    * ASID generations are 64 bit.  Overflow of generations never happens.
+    * For safety, we simply disable ASIDs, so correctness is established; it
+    * only runs a bit slower.
+    */
     printk("HVM: ASID generation overrun. Disabling ASIDs.\n");
     data->disabled = 1;
 }
 
-bool hvm_asid_handle_vmenter(struct hvm_vcpu_asid *asid)
+/* This function is called only when first vmenter happens after creating a new domain */
+bool hvm_asid_handle_vmenter(struct hvm_domain_asid *asid)
 {
-    struct hvm_asid_data *data = &this_cpu(hvm_asid_data);
+    struct hvm_asid_data *data = &asid_data;
 
     /* On erratum #170 systems we must flush the TLB. 
      * Generation overruns are taken here, too. */
     if ( data->disabled )
         goto disabled;
 
-    /* Test if VCPU has valid ASID. */
-    if ( read_atomic(&asid->generation) == data->core_asid_generation )
+    /* Test if domain has valid ASID. */
+    if ( read_atomic(&asid->generation) == data->asid_generation )
         return 0;
 
     /* If there are no free ASIDs, need to go to a new generation */
     if ( unlikely(data->next_asid > data->max_asid) )
     {
-        hvm_asid_flush_core();
+        // TODO(vaishali): Add a check to pick the asid from the reclaimable asids if any
+        hvm_asid_flush_all();
         data->next_asid = 1;
         if ( data->disabled )
             goto disabled;
     }
 
-    /* Now guaranteed to be a free ASID. */
-    asid->asid = data->next_asid++;
-    write_atomic(&asid->generation, data->core_asid_generation);
+    /* Now guaranteed to be a free ASID. Only assign a new asid if the ASID is 1 */
+    if (asid->asid == 0)
+    {
+        asid->asid = data->next_asid++;
+    }
+
+    write_atomic(&asid->generation, data->asid_generation);
 
     /*
      * When we assign ASID 1, flush all TLB entries as we are starting a new
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 02e378365b..9b8bdd311e 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -2519,10 +2519,7 @@ static int cf_check hvmemul_tlb_op(
 
     case x86emul_invpcid:
         if ( x86emul_invpcid_type(aux) != X86_INVPCID_INDIV_ADDR )
-        {
-            hvm_asid_flush_vcpu(current);
             break;
-        }
         aux = x86emul_invpcid_pcid(aux);
         /* fall through */
     case x86emul_invlpga:
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 2c66fe0f7a..65e63b736b 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1569,7 +1569,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
     int rc;
     struct domain *d = v->domain;
 
-    hvm_asid_flush_vcpu(v);
+    //hvm_asid_flush_vcpu(v);
 
     spin_lock_init(&v->arch.hvm.tm_lock);
     INIT_LIST_HEAD(&v->arch.hvm.tm_list);
diff --git a/xen/arch/x86/hvm/nestedhvm.c b/xen/arch/x86/hvm/nestedhvm.c
index 451c4da6d4..49e54d0a8c 100644
--- a/xen/arch/x86/hvm/nestedhvm.c
+++ b/xen/arch/x86/hvm/nestedhvm.c
@@ -41,7 +41,7 @@ nestedhvm_vcpu_reset(struct vcpu *v)
     nv->stale_np2m = false;
     nv->np2m_generation = 0;
 
-    hvm_asid_flush_vcpu_asid(&nv->nv_n2asid);
+    //hvm_asid_flush_vcpu_asid(&nv->nv_n2asid);
 
     alternative_vcall(hvm_funcs.nhvm_vcpu_reset, v);
 
@@ -86,7 +86,7 @@ static void cf_check nestedhvm_flushtlb_ipi(void *info)
      * This is cheaper than flush_tlb_local() and has
      * the same desired effect.
      */
-    hvm_asid_flush_core();
+    // hvm_asid_flush_core();
     vcpu_nestedhvm(v).nv_p2m = NULL;
     vcpu_nestedhvm(v).stale_np2m = true;
 }
diff --git a/xen/arch/x86/hvm/svm/asid.c b/xen/arch/x86/hvm/svm/asid.c
index 7977a8e86b..9a4fc6c328 100644
--- a/xen/arch/x86/hvm/svm/asid.c
+++ b/xen/arch/x86/hvm/svm/asid.c
@@ -7,31 +7,35 @@
 #include <asm/amd.h>
 #include <asm/hvm/nestedhvm.h>
 #include <asm/hvm/svm/svm.h>
-
+#include <asm/processor.h>
 #include "svm.h"
+#include "xen/cpumask.h"
 
-void svm_asid_init(const struct cpuinfo_x86 *c)
+void svm_asid_init(void)
 {
+    unsigned int cpu = smp_processor_id();
+    const struct cpuinfo_x86 *c;
     int nasids = 0;
 
-    /* Check for erratum #170, and leave ASIDs disabled if it's present. */
-    if ( !cpu_has_amd_erratum(c, AMD_ERRATUM_170) )
-        nasids = cpuid_ebx(0x8000000aU);
-
+    for_each_online_cpu( cpu ) {
+        c = &cpu_data[cpu];
+        /* Check for erratum #170, and leave ASIDs disabled if it's present. */
+        if ( !cpu_has_amd_erratum(c, AMD_ERRATUM_170) )
+            nasids += cpuid_ebx(0x8000000aU);
+    }
     hvm_asid_init(nasids);
 }
 
 /*
- * Called directly before VMRUN.  Checks if the VCPU needs a new ASID,
+ * Called directly before first VMRUN.  Checks if the domain needs an ASID,
  * assigns it, and if required, issues required TLB flushes.
  */
 void svm_asid_handle_vmrun(void)
 {
-    struct vcpu *curr = current;
-    struct vmcb_struct *vmcb = curr->arch.hvm.svm.vmcb;
-    struct hvm_vcpu_asid *p_asid =
-        nestedhvm_vcpu_in_guestmode(curr)
-        ? &vcpu_nestedhvm(curr).nv_n2asid : &curr->arch.hvm.n1asid;
+    struct vcpu *v = current;
+    struct domain *d = current->domain;
+    struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
+    struct hvm_domain_asid *p_asid = &d->arch.hvm.n1asid;
     bool need_flush = hvm_asid_handle_vmenter(p_asid);
 
     /* ASID 0 indicates that ASIDs are disabled. */
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index 35a2cbfd7d..08da1f7e84 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -699,7 +699,7 @@ nsvm_vcpu_vmentry(struct vcpu *v, struct cpu_user_regs *regs,
     if ( svm->ns_asid != vmcb_get_asid(ns_vmcb))
     {
         nv->nv_flushp2m = 1;
-        hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
+//        hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
         svm->ns_asid = vmcb_get_asid(ns_vmcb);
     }
 
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index db530d55f2..61f1ad3085 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -179,17 +179,15 @@ static void cf_check svm_update_guest_cr(
         break;
     case 3:
         vmcb_set_cr3(vmcb, v->arch.hvm.hw_cr[3]);
-        if ( !nestedhvm_enabled(v->domain) )
+        /*if ( !nestedhvm_enabled(v->domain) )
         {
             if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
                 hvm_asid_flush_vcpu(v);
-        }
-        else if ( nestedhvm_vmswitch_in_progress(v) )
+        }*/
+        if ( nestedhvm_vmswitch_in_progress(v) )
             ; /* CR3 switches during VMRUN/VMEXIT do not flush the TLB. */
-        else if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
-            hvm_asid_flush_vcpu_asid(
-                nestedhvm_vcpu_in_guestmode(v)
-                ? &vcpu_nestedhvm(v).nv_n2asid : &v->arch.hvm.n1asid);
+        /*else if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
+            hvm_asid_flush_vcpu_asid(&v->arch.hvm.n1asid);*/
         break;
     case 4:
         value = HVM_CR4_HOST_MASK;
@@ -988,8 +986,8 @@ static void noreturn cf_check svm_do_resume(void)
         v->arch.hvm.svm.launch_core = smp_processor_id();
         hvm_migrate_timers(v);
         hvm_migrate_pirqs(v);
-        /* Migrating to another ASID domain.  Request a new ASID. */
-        hvm_asid_flush_vcpu(v);
+        /* Migrating to another ASID domain. Request a new ASID. */
+        hvm_asid_flush_domain(v->domain);
     }
 
     if ( !vcpu_guestmode && !vlapic_hw_disabled(vlapic) )
@@ -1570,9 +1568,6 @@ static int _svm_cpu_up(bool bsp)
     /* check for erratum 383 */
     svm_init_erratum_383(c);
 
-    /* Initialize core's ASID handling. */
-    svm_asid_init(c);
-
     /* Initialize OSVW bits to be used by guests */
     svm_host_osvw_init();
 
@@ -2331,13 +2326,11 @@ static void svm_vmexit_do_invalidate_cache(struct cpu_user_regs *regs,
     __update_guest_eip(regs, inst_len);
 }
 
+//TODO(vaishali): Handle for nested virtualization as well
 static void svm_invlpga_intercept(
-    struct vcpu *v, unsigned long linear, uint32_t asid)
+    struct domain *d, unsigned long linear, uint32_t asid)
 {
-    svm_invlpga(linear,
-                (asid == 0)
-                ? v->arch.hvm.n1asid.asid
-                : vcpu_nestedhvm(v).nv_n2asid.asid);
+    svm_invlpga(linear, d->arch.hvm.n1asid.asid);
 }
 
 static void svm_invlpg_intercept(unsigned long linear)
@@ -2358,8 +2351,9 @@ static bool cf_check is_invlpg(
 
 static void cf_check svm_invlpg(struct vcpu *v, unsigned long linear)
 {
+    v = current;
     /* Safe fallback. Take a new ASID. */
-    hvm_asid_flush_vcpu(v);
+    hvm_asid_flush_domain(v->domain);
 }
 
 static bool cf_check svm_get_pending_event(
@@ -2533,6 +2527,7 @@ void asmlinkage svm_vmexit_handler(void)
     struct cpu_user_regs *regs = guest_cpu_user_regs();
     uint64_t exit_reason;
     struct vcpu *v = current;
+    struct domain *d = current->domain;
     struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
     int insn_len, rc;
     vintr_t intr;
@@ -2927,7 +2922,7 @@ void asmlinkage svm_vmexit_handler(void)
         }
         if ( (insn_len = svm_get_insn_len(v, INSTR_INVLPGA)) == 0 )
             break;
-        svm_invlpga_intercept(v, regs->rax, regs->ecx);
+        svm_invlpga_intercept(d, regs->rax, regs->ecx);
         __update_guest_eip(regs, insn_len);
         break;
 
diff --git a/xen/arch/x86/hvm/svm/svm.h b/xen/arch/x86/hvm/svm/svm.h
index 8dbf37ff49..be6e5aa3e5 100644
--- a/xen/arch/x86/hvm/svm/svm.h
+++ b/xen/arch/x86/hvm/svm/svm.h
@@ -15,7 +15,6 @@ struct cpu_user_regs;
 struct cpuinfo_x86;
 struct vcpu;
 
-void svm_asid_init(const struct cpuinfo_x86 *c);
 void svm_asid_handle_vmrun(void);
 
 unsigned long *svm_msrbit(unsigned long *msr_bitmap, uint32_t msr);
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 9b6dc51f36..3f6f2f205a 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -1956,7 +1956,7 @@ void cf_check vmx_do_resume(void)
          */
         v->arch.hvm.vmx.hostenv_migrated = 1;
 
-        hvm_asid_flush_vcpu(v);
+        hvm_asid_flush_domain(v->domain);
     }
 
     debug_state = v->domain->debugger_attached
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 8ba996546f..5f25ed8b10 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1467,7 +1467,8 @@ static void cf_check vmx_handle_cd(struct vcpu *v, unsigned long value)
             vmx_set_msr_intercept(v, MSR_IA32_CR_PAT, VMX_MSR_RW);
 
             wbinvd();               /* flush possibly polluted cache */
-            hvm_asid_flush_vcpu(v); /* invalidate memory type cached in TLB */
+            // TODO(vaishali): Is it still correct to flush here?
+            hvm_asid_flush_domain(v->domain); /* invalidate memory type cached in TLB */
             v->arch.hvm.cache_mode = NO_FILL_CACHE_MODE;
         }
         else
@@ -1476,7 +1477,7 @@ static void cf_check vmx_handle_cd(struct vcpu *v, unsigned long value)
             vmx_set_guest_pat(v, *pat);
             if ( !is_iommu_enabled(v->domain) || iommu_snoop )
                 vmx_clear_msr_intercept(v, MSR_IA32_CR_PAT, VMX_MSR_RW);
-            hvm_asid_flush_vcpu(v); /* no need to flush cache */
+            /*hvm_asid_flush_vcpu(v);  no need to flush cache */
         }
     }
 }
@@ -1836,8 +1837,6 @@ static void cf_check vmx_update_guest_cr(
 
         __vmwrite(GUEST_CR3, v->arch.hvm.hw_cr[3]);
 
-        if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
-            hvm_asid_flush_vcpu(v);
         break;
 
     default:
@@ -4820,7 +4819,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
     struct vcpu *curr = current;
     struct domain *currd = curr->domain;
     u32 new_asid, old_asid;
-    struct hvm_vcpu_asid *p_asid;
+    struct hvm_domain_asid *p_asid;
     bool need_flush;
 
     ASSERT(hvmemul_cache_disabled(curr));
@@ -4834,10 +4833,10 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
 
     if ( !cpu_has_vmx_vpid )
         goto out;
-    if ( nestedhvm_vcpu_in_guestmode(curr) )
+/*    if ( nestedhvm_vcpu_in_guestmode(curr) )
         p_asid = &vcpu_nestedhvm(curr).nv_n2asid;
-    else
-        p_asid = &curr->arch.hvm.n1asid;
+    else*/
+        p_asid = &currd->arch.hvm.n1asid;
 
     old_asid = p_asid->asid;
     need_flush = hvm_asid_handle_vmenter(p_asid);
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 39290c9861..168173a26b 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1254,7 +1254,7 @@ static void virtual_vmentry(struct cpu_user_regs *regs)
 
         if ( nvmx->guest_vpid != new_vpid )
         {
-            hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
+//            hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
             nvmx->guest_vpid = new_vpid;
         }
     }
@@ -2055,8 +2055,8 @@ static int nvmx_handle_invvpid(struct cpu_user_regs *regs)
     case INVVPID_INDIVIDUAL_ADDR:
     case INVVPID_SINGLE_CONTEXT:
     case INVVPID_ALL_CONTEXT:
-        hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(current).nv_n2asid);
-        break;
+  //      hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(current).nv_n2asid);
+      break;
     default:
         vmfail(regs, VMX_INSN_INVEPT_INVVPID_INVALID_OP);
         return X86EMUL_OKAY;
diff --git a/xen/arch/x86/include/asm/hvm/asid.h b/xen/arch/x86/include/asm/hvm/asid.h
index 17c58353d1..65c702f809 100644
--- a/xen/arch/x86/include/asm/hvm/asid.h
+++ b/xen/arch/x86/include/asm/hvm/asid.h
@@ -8,25 +8,24 @@
 #ifndef __ASM_X86_HVM_ASID_H__
 #define __ASM_X86_HVM_ASID_H__
 
+struct hvm_domain;
+struct hvm_domain_asid;
 
-struct vcpu;
-struct hvm_vcpu_asid;
-
-/* Initialise ASID management for the current physical CPU. */
+/* Initialise ASID management distributed across all CPUs. */
 void hvm_asid_init(int nasids);
 
 /* Invalidate a particular ASID allocation: forces re-allocation. */
-void hvm_asid_flush_vcpu_asid(struct hvm_vcpu_asid *asid);
+void hvm_asid_flush_domain_asid(struct hvm_domain_asid *asid);
 
-/* Invalidate all ASID allocations for specified VCPU: forces re-allocation. */
-void hvm_asid_flush_vcpu(struct vcpu *v);
+/* Invalidate all ASID allocations for specified domain */
+void hvm_asid_flush_domain(struct domain *d);
 
-/* Flush all ASIDs on this processor core. */
-void hvm_asid_flush_core(void);
+/* Flush all ASIDs on all processor cores */
+void hvm_asid_flush_all(void);
 
 /* Called before entry to guest context. Checks ASID allocation, returns a
  * boolean indicating whether all ASIDs must be flushed. */
-bool hvm_asid_handle_vmenter(struct hvm_vcpu_asid *asid);
+bool hvm_asid_handle_vmenter(struct hvm_domain_asid *asid);
 
 #endif /* __ASM_X86_HVM_ASID_H__ */
 
diff --git a/xen/arch/x86/include/asm/hvm/domain.h b/xen/arch/x86/include/asm/hvm/domain.h
index dd9d837e84..2d02e227fc 100644
--- a/xen/arch/x86/include/asm/hvm/domain.h
+++ b/xen/arch/x86/include/asm/hvm/domain.h
@@ -141,6 +141,7 @@ struct hvm_domain {
     } write_map;
 
     struct hvm_pi_ops pi_ops;
+    struct hvm_domain_asid n1asid;
 
     union {
         struct vmx_domain vmx;
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index 1c01e22c8e..1ab816c48a 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -464,7 +464,7 @@ static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
 static inline void hvm_flush_guest_tlbs(void)
 {
     if ( hvm_enabled )
-        hvm_asid_flush_core();
+        hvm_asid_flush_all();
 }
 
 static inline unsigned int
diff --git a/xen/arch/x86/include/asm/hvm/svm/svm.h b/xen/arch/x86/include/asm/hvm/svm/svm.h
index 4eeeb25da9..19b79c490f 100644
--- a/xen/arch/x86/include/asm/hvm/svm/svm.h
+++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
@@ -21,6 +21,7 @@ bool svm_load_segs(unsigned int ldt_ents, unsigned long ldt_base,
                    unsigned long fs_base, unsigned long gs_base,
                    unsigned long gs_shadow);
 
+extern void svm_asid_init(void);
 extern u32 svm_feature_flags;
 
 #define SVM_FEATURE_NPT            0 /* Nested page table support */
diff --git a/xen/arch/x86/include/asm/hvm/vcpu.h b/xen/arch/x86/include/asm/hvm/vcpu.h
index 64c7a6fede..b185522611 100644
--- a/xen/arch/x86/include/asm/hvm/vcpu.h
+++ b/xen/arch/x86/include/asm/hvm/vcpu.h
@@ -17,7 +17,7 @@
 #include <asm/mtrr.h>
 #include <public/hvm/ioreq.h>
 
-struct hvm_vcpu_asid {
+struct hvm_domain_asid {
     uint64_t generation;
     uint32_t asid;
 };
@@ -90,7 +90,7 @@ struct nestedvcpu {
     bool stale_np2m; /* True when p2m_base in VMCx02 is no longer valid */
     uint64_t np2m_generation;
 
-    struct hvm_vcpu_asid nv_n2asid;
+    // struct hvm_vcpu_asid nv_n2asid;
 
     bool nv_vmentry_pending;
     bool nv_vmexit_pending;
@@ -152,8 +152,6 @@ struct hvm_vcpu {
     /* (MFN) hypervisor page table */
     pagetable_t         monitor_table;
 
-    struct hvm_vcpu_asid n1asid;
-
     u64                 msr_tsc_adjust;
 
     union {
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmx.h b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
index 1489dd05c2..c8a9ca707d 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
@@ -525,6 +525,7 @@ void ept_sync_domain(struct p2m_domain *p2m);
 
 static inline void vpid_sync_vcpu_gva(struct vcpu *v, unsigned long gva)
 {
+    struct domain *d = current->domain;
     int type = INVVPID_INDIVIDUAL_ADDR;
 
     /*
@@ -544,7 +545,7 @@ static inline void vpid_sync_vcpu_gva(struct vcpu *v, unsigned long gva)
         type = INVVPID_ALL_CONTEXT;
 
 execute_invvpid:
-    __invvpid(type, v->arch.hvm.n1asid.asid, (u64)gva);
+    __invvpid(type, d->arch.hvm.n1asid.asid, (u64)gva);
 }
 
 static inline void vpid_sync_all(void)
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index d2011fde24..53098c7f8c 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -739,13 +739,13 @@ static bool cf_check flush_tlb(const unsigned long *vcpu_bitmap)
         if ( !flush_vcpu(v, vcpu_bitmap) )
             continue;
 
-        hvm_asid_flush_vcpu(v);
-
         cpu = read_atomic(&v->dirty_cpu);
         if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )
             __cpumask_set_cpu(cpu, mask);
     }
 
+    hvm_asid_flush_domain(d);
+
     /*
      * Trigger a vmexit on all pCPUs with dirty vCPU state in order to force an
      * ASID/VPID change and hence accomplish a guest TLB flush. Note that vCPUs
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index db5d9b6c2a..358e7f677e 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1393,7 +1393,7 @@ p2m_flush(struct vcpu *v, struct p2m_domain *p2m)
     ASSERT(v->domain == p2m->domain);
     vcpu_nestedhvm(v).nv_p2m = NULL;
     p2m_flush_table(p2m);
-    hvm_asid_flush_vcpu(v);
+    hvm_asid_flush_domain(v->domain);
 }
 
 void
@@ -1452,7 +1452,7 @@ static void assign_np2m(struct vcpu *v, struct p2m_domain *p2m)
 
 static void nvcpu_flush(struct vcpu *v)
 {
-    hvm_asid_flush_vcpu(v);
+//    hvm_asid_flush_domain(&v->domain->arch.hvm);
     vcpu_nestedhvm(v).stale_np2m = true;
 }
 
@@ -1572,7 +1572,7 @@ void np2m_schedule(int dir)
             if ( !np2m_valid )
             {
                 /* This vCPU's np2m was flushed while it was not runnable */
-                hvm_asid_flush_core();
+                hvm_asid_flush_all();
                 vcpu_nestedhvm(curr).nv_p2m = NULL;
             }
             else
diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c
index bca320fffa..5e09b84322 100644
--- a/xen/arch/x86/mm/paging.c
+++ b/xen/arch/x86/mm/paging.c
@@ -964,7 +964,7 @@ void paging_update_nestedmode(struct vcpu *v)
     else
         /* TODO: shadow-on-shadow */
         v->arch.paging.nestedmode = NULL;
-    hvm_asid_flush_vcpu(v);
+    hvm_asid_flush_domain(v->domain);
 }
 
 int __init paging_set_allocation(struct domain *d, unsigned int pages,
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b50c9c84af..78f12e7f9e 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -59,6 +59,7 @@
 #include <asm/microcode.h>
 #include <asm/prot-key.h>
 #include <asm/pv/domain.h>
+#include <asm/hvm/svm/svm.h>
 
 /* opt_nosmp: If true, secondary processors are ignored. */
 static bool __initdata opt_nosmp;
@@ -2013,6 +2014,12 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         printk(XENLOG_INFO "Parked %u CPUs\n", num_parked);
     smp_cpus_done();
 
+    /* Initialize xen-wide ASID handling */
+    #ifdef CONFIG_HVM
+    if ( cpu_has_svm )
+        svm_asid_init();
+    #endif
+
     do_initcalls();
 
     if ( opt_watchdog ) 
-- 
2.45.0


From xen-devel-bounces@lists.xenproject.org Fri May 24 12:31:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:31:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729462.1134607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAU52-00052z-SQ; Fri, 24 May 2024 12:31:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729462.1134607; Fri, 24 May 2024 12:31:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAU52-00052s-PQ; Fri, 24 May 2024 12:31:28 +0000
Received: by outflank-mailman (input) for mailman id 729462;
 Fri, 24 May 2024 12:31:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/Vf=M3=bounce.vates.tech=bounce-md_30504962.6650889a.v1-b9480d857674453791d98acf668f11dd@srs-se1.protection.inumbo.net>)
 id 1sAU51-00051V-6z
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:31:27 +0000
Received: from mail136-17.atl41.mandrillapp.com
 (mail136-17.atl41.mandrillapp.com [198.2.136.17])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8867d8cf-19c9-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:31:24 +0200 (CEST)
Received: from pmta11.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1])
 by mail136-17.atl41.mandrillapp.com (Mailchimp) with ESMTP id
 4Vm4CZ55cDzPm0PRK
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 12:31:22 +0000 (GMT)
Received: from [37.26.189.201] by mandrillapp.com id
 b9480d857674453791d98acf668f11dd; Fri, 24 May 2024 12:31:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8867d8cf-19c9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com;
	s=mte1; t=1716553882; x=1716814382;
	bh=tlJyPYKFhaSXXv4BnrdLWt4xxrI76gRAyevGwxWPheE=;
	h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version:
	 Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From;
	b=JF4TV6zbXv9jcw8mX6ePjox7oAR+1V+12cSJD+KrFlg8wzNlvMwFpH6oBQ48Sg8RG
	 2OVYFnhFaDGtOWpqLQ/be4d43YzcusyP3gpaZxtVI/uAQlYxaeoYp8MmouY6PODs5F
	 h8ukRZRxSSaM/P1mapiYW1FouGOf7hkefNa7nqkTO1Z0QYS7kijMf3n7qsevTsuwRJ
	 mqvnQNvUReVy4ktDr9yPPzG6Yvxa0Ug3bEGo20TJW7bC7tRlBJHVHQmbo4KtuD6DLY
	 RtE3KGFuyw/f4iGM1KW9r6XAb/FgXpGRFXAwh/60QtNHXsbfdWE0T4OOlji725YzaJ
	 hMtIEPquMkPWQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1;
	t=1716553882; x=1716814382; i=vaishali.thakkar@vates.tech;
	bh=tlJyPYKFhaSXXv4BnrdLWt4xxrI76gRAyevGwxWPheE=;
	h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version:
	 Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From;
	b=b4Qlf5nZ+7sZ7vuIQWbHEjgN0ZmWdslRHeXkrd+ofgSJnRXkBr60i2WrmTigpMA7r
	 v7b5S3QjakYog3TWZXzxNzwcQ7xzIgA371qmPbt5xBEXBjaG+VoARSM1hAqdS3xfcC
	 vOg8g1LKKp9jFNW+uRzeC2jMboDD694G226SCgtSDRgQV6TIOHZPYIgxeLSi5sGtPB
	 9jE7QhtSw87tNqJvs4rg5kL0oOcJPI4BVLhffUnhd0WphPJzdaZ+vmgXoP+mQSFVU+
	 go2kbVhhFCfAyQJ4CG4IQAG2tbnfptz0ciUFSywJcBjBtFxY8yy1ISNKroAE2kA7bH
	 VzsDJ7X1crAUA==
From: Vaishali Thakkar <vaishali.thakkar@vates.tech>
Subject: =?utf-8?Q?[RFC=20for-4.20=20v1=200/1]=20x86/hvm:=20Introduce=20Xen-wide=20ASID=20allocator?=
X-Mailer: git-send-email 2.45.0
X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2
X-Bm-Transport-Timestamp: 1716553881342
To: xen-devel@lists.xenproject.org
Cc: jbeulich@suse.com, andrew.cooper3@citrix.com, roger.pau@citrix.com, george.dunlap@citrix.com, Vaishali Thakkar <vaishali.thakkar@vates.tech>
Message-Id: <cover.1716551380.git.vaishali.thakkar@vates.tech>
X-Native-Encoded: 1
X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.b9480d857674453791d98acf668f11dd?=
X-Mandrill-User: md_30504962
Feedback-ID: 30504962:30504962.20240524:md
Date: Fri, 24 May 2024 12:31:22 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Motivation:
-----------
This is part of the effort to enable AMD SEV technologies in Xen. For
AMD SEV support, we need a fixed ASID associated with all vcpus of the
same domain throughout the domain's lifetime. This is because for SEV/
SEV-{ES,SNP} VM, the ASID is the index which is associated with the
encryption key.

Currently, ASID generation and management is done per-PCPU in Xen. And
at the time of each VMENTER, the ASID associated with vcpus of the
domain is changed. This implementation is incompatible with SEV
technologies for the above mentioned reasons. In a discussion with
Andrew Cooper, it came up that it=E2=80=99ll be nice to have fixed ASIDs no=
t
only for SEV VMs but also for all VMs. Because it opens up the
opportunity to use instructions like TLBSYNC and INVLPGB (Section
5.5.3 in AMD Architecture manual[0]) for broadcasting the TLB
Invalidations.

Why is this RFC?
----------------
This is only tested on AMD SVM at the moment. There are a few points
that I would like to discuss and get a feedback on from the community
before further development and testing. I=E2=80=99ve also submitted a desig=
n
session for this RFC to discuss further at the Xen Summit.

Points of discussion:
---------------------
1. I=E2=80=99m not sure how this should be handled for the nestedhvm. To st=
art
with, at the moment all the values seem to be handled via struct
nestedvcpu. Do we want to keep it that way or do we want to have
something like nestedhvm_domain to associate certain values like asid?
I=E2=80=99ve not handled this as part of this RFC as I would like to know t=
he
opinions and plans of those working on nested virtualization.

2. I=E2=80=99m doing initialization of xen-wide asids at the moment in setu=
p.c
but is that the right place to do it? I=E2=80=99m asking this because I=E2=
=80=99ve
been seeing a weird bug with the code in this RFC. Dom0 is able to
have a fixed asid through the lifecycle of it. But if I start a domU
with 2/4 vcpus via xl, sometimes it only brings up the one vcpu and
shows =E2=80=98tsc: Unable to calibrate against PIT=E2=80=99 while booting =
the kernel.

Notes:
-----
1. Currently the RFC doesn=E2=80=99t demonstrate the use of TLBSYNC and INV=
LPGB.
It can further be added if required. I'm not sure if it should be part
of the same patch series or not.

2. This is a basic RFC to start the discussion on the above points but
I further plan to add a logic to reclaim the asids that are no longer
in use and add a check to pick the asid from such stack before doing
hvm_asid_flush_all.

[0] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/pr=
ogrammer-references/24593.pdf



Vaishali Thakkar (1):
  x86/hvm: Introduce Xen-wide ASID Allocator

 xen/arch/x86/flushtlb.c                |  1 -
 xen/arch/x86/hvm/asid.c                | 61 ++++++++++++++------------
 xen/arch/x86/hvm/emulate.c             |  3 --
 xen/arch/x86/hvm/hvm.c                 |  2 +-
 xen/arch/x86/hvm/nestedhvm.c           |  4 +-
 xen/arch/x86/hvm/svm/asid.c            | 28 +++++++-----
 xen/arch/x86/hvm/svm/nestedsvm.c       |  2 +-
 xen/arch/x86/hvm/svm/svm.c             | 33 ++++++--------
 xen/arch/x86/hvm/svm/svm.h             |  1 -
 xen/arch/x86/hvm/vmx/vmcs.c            |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c             | 15 +++----
 xen/arch/x86/hvm/vmx/vvmx.c            |  6 +--
 xen/arch/x86/include/asm/hvm/asid.h    | 19 ++++----
 xen/arch/x86/include/asm/hvm/domain.h  |  1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  2 +-
 xen/arch/x86/include/asm/hvm/svm/svm.h |  1 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  6 +--
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  3 +-
 xen/arch/x86/mm/hap/hap.c              |  4 +-
 xen/arch/x86/mm/p2m.c                  |  6 +--
 xen/arch/x86/mm/paging.c               |  2 +-
 xen/arch/x86/setup.c                   |  7 +++
 22 files changed, 108 insertions(+), 101 deletions(-)

-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729479.1134634 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-00087F-Dt; Fri, 24 May 2024 12:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729479.1134634; Fri, 24 May 2024 12:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-000867-78; Fri, 24 May 2024 12:41:13 +0000
Received: by outflank-mailman (input) for mailman id 729479;
 Fri, 24 May 2024 12:41:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUES-00083d-38
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:12 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id e52d0a9d-19ca-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:41:09 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DFCE71688;
 Fri, 24 May 2024 05:41:32 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B9D423F641;
 Fri, 24 May 2024 05:41:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e52d0a9d-19ca-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v4 3/7] xen/p2m: put reference for level 2 superpage
Date: Fri, 24 May 2024 13:40:51 +0100
Message-Id: <20240524124055.3871399-4-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

We are doing foreign memory mapping for static shared memory, and
there is a great possibility that it could be super mapped.
But today, p2m_put_l3_page could not handle superpages.

This commits implements a new function p2m_put_l2_superpage to handle
2MB superpages, specifically for helping put extra references for
foreign superpages.

Modify relinquish_p2m_mapping as well to take into account preemption
when type is foreign memory and order is above 9 (2MB).

Currently 1GB superpages are not handled because Xen is not preemptible
and therefore some work is needed to handle such superpages, for which
at some point Xen might end up freeing memory and therefore for such a
big mapping it could end up in a very long operation.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v4 changes:
 - optimised the path to call put_page() on the foreign mapping as
   Julien suggested. Add a comment in p2m_put_l2_superpage to state
   that any changes needs to take into account some change in the
   relinquish code. (Julien)
v3 changes:
 - Add reasoning why we don't support now 1GB superpage, remove level_order
   variable from p2m_put_l2_superpage, update TODO comment inside
   p2m_free_entry, use XEN_PT_LEVEL_ORDER(2) instead of value 9 inside
   relinquish_p2m_mapping. (Michal)
v2:
 - Do not handle 1GB super page as there might be some issue where
   a lot of calls to put_page(...) might be issued which could lead
   to free memory that is a long operation.
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
---
 xen/arch/arm/mmu/p2m.c | 82 +++++++++++++++++++++++++++++++-----------
 1 file changed, 62 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 41fcca011cf4..986c5a03c54b 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -753,34 +753,66 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
     return rc;
 }
 
-/*
- * Put any references on the single 4K page referenced by pte.
- * TODO: Handle superpages, for now we only take special references for leaf
- * pages (specifically foreign ones, which can't be super mapped today).
- */
-static void p2m_put_l3_page(const lpae_t pte)
+static void p2m_put_foreign_page(struct page_info *pg)
 {
-    mfn_t mfn = lpae_get_mfn(pte);
-
-    ASSERT(p2m_is_valid(pte));
-
     /*
-     * TODO: Handle other p2m types
-     *
      * It's safe to do the put_page here because page_alloc will
      * flush the TLBs if the page is reallocated before the end of
      * this loop.
      */
-    if ( p2m_is_foreign(pte.p2m.type) )
+    put_page(pg);
+}
+
+/* Put any references on the single 4K page referenced by mfn. */
+static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
+{
+    /* TODO: Handle other p2m types */
+    if ( p2m_is_foreign(type) )
     {
         ASSERT(mfn_valid(mfn));
-        put_page(mfn_to_page(mfn));
+        p2m_put_foreign_page(mfn_to_page(mfn));
     }
     /* Detect the xenheap page and mark the stored GFN as invalid. */
-    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
+    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
         page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
 }
 
+/* Put any references on the superpage referenced by mfn. */
+static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
+{
+    struct page_info *pg;
+    unsigned int i;
+
+    /*
+     * TODO: Handle other p2m types, but be aware that any changes to handle
+     * different types should require an update on the relinquish code to handle
+     * preemption.
+     */
+    if ( !p2m_is_foreign(type) )
+        return;
+
+    ASSERT(mfn_valid(mfn));
+
+    pg = mfn_to_page(mfn);
+
+    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++, pg++ )
+        p2m_put_foreign_page(pg);
+}
+
+/* Put any references on the page referenced by pte. */
+static void p2m_put_page(const lpae_t pte, unsigned int level)
+{
+    mfn_t mfn = lpae_get_mfn(pte);
+
+    ASSERT(p2m_is_valid(pte));
+
+    /* We have a second level 2M superpage */
+    if ( p2m_is_superpage(pte, level) && (level == 2) )
+        return p2m_put_l2_superpage(mfn, pte.p2m.type);
+    else if ( level == 3 )
+        return p2m_put_l3_page(mfn, pte.p2m.type);
+}
+
 /* Free lpae sub-tree behind an entry */
 static void p2m_free_entry(struct p2m_domain *p2m,
                            lpae_t entry, unsigned int level)
@@ -809,9 +841,16 @@ static void p2m_free_entry(struct p2m_domain *p2m,
 #endif
 
         p2m->stats.mappings[level]--;
-        /* Nothing to do if the entry is a super-page. */
-        if ( level == 3 )
-            p2m_put_l3_page(entry);
+        /*
+         * TODO: Currently we don't handle 1GB super-page, Xen is not
+         * preemptible and therefore some work is needed to handle such
+         * superpages, for which at some point Xen might end up freeing memory
+         * and therefore for such a big mapping it could end up in a very long
+         * operation.
+         */
+        if ( level >= 2 )
+            p2m_put_page(entry, level);
+
         return;
     }
 
@@ -1558,9 +1597,12 @@ int relinquish_p2m_mapping(struct domain *d)
 
         count++;
         /*
-         * Arbitrarily preempt every 512 iterations.
+         * Arbitrarily preempt every 512 iterations or when type is foreign
+         * mapping and the order is above 9 (2MB).
          */
-        if ( !(count % 512) && hypercall_preempt_check() )
+        if ( (!(count % 512) ||
+              (p2m_is_foreign(t) && (order > XEN_PT_LEVEL_ORDER(2)))) &&
+             hypercall_preempt_check() )
         {
             rc = -ERESTART;
             break;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729482.1134661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEV-0000OP-8Q; Fri, 24 May 2024 12:41:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729482.1134661; Fri, 24 May 2024 12:41:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEV-0000N3-3c; Fri, 24 May 2024 12:41:15 +0000
Received: by outflank-mailman (input) for mailman id 729482;
 Fri, 24 May 2024 12:41:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUET-00082M-C8
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:13 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id e471831c-19ca-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 14:41:08 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9716A1684;
 Fri, 24 May 2024 05:41:31 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A56513F641;
 Fri, 24 May 2024 05:41:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e471831c-19ca-11ef-90a1-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 2/7] xen/arm: Wrap shared memory mapping code in one function
Date: Fri, 24 May 2024 13:40:50 +0100
Message-Id: <20240524124055.3871399-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Wrap the code and logic that is calling assign_shared_memory
and map_regions_p2mt into a new function 'handle_shared_mem_bank',
it will become useful later when the code will allow the user to
don't pass the host physical address.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - check return value of dt_property_read_string, add R-by Michal
v2 changes:
 - add blank line, move owner_dom_io computation inside
   handle_shared_mem_bank in order to reduce args count, remove
   not needed BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 86 +++++++++++++++++++++++--------------
 1 file changed, 53 insertions(+), 33 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 0a1c327e90ea..c15a65130659 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -180,6 +180,53 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
     return 0;
 }
 
+static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
+                                         const char *role_str,
+                                         const struct membank *shm_bank)
+{
+    bool owner_dom_io = true;
+    paddr_t pbase, psize;
+    int ret;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+
+    /*
+     * "role" property is optional and if it is defined explicitly,
+     * then the owner domain is not the default "dom_io" domain.
+     */
+    if ( role_str != NULL )
+        owner_dom_io = false;
+
+    /*
+     * DOMID_IO is a fake domain and is not described in the Device-Tree.
+     * Therefore when the owner of the shared region is DOMID_IO, we will
+     * only find the borrowers.
+     */
+    if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
+         (!owner_dom_io && strcmp(role_str, "owner") == 0) )
+    {
+        /*
+         * We found the first borrower of the region, the owner was not
+         * specified, so they should be assigned to dom_io.
+         */
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        if ( ret )
+            return ret;
+    }
+
+    if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
+    {
+        /* Set up P2M foreign mapping for borrower domain. */
+        ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
+                               _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
+        if ( ret )
+            return ret;
+    }
+
+    return 0;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -196,7 +243,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         unsigned int i;
         const char *role_str;
         const char *shm_id;
-        bool owner_dom_io = true;
 
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
@@ -237,39 +283,13 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                 return -EINVAL;
             }
 
-        /*
-         * "role" property is optional and if it is defined explicitly,
-         * then the owner domain is not the default "dom_io" domain.
-         */
-        if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
-            owner_dom_io = false;
+        /* "role" property is optional */
+        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
+            role_str = NULL;
 
-        /*
-         * DOMID_IO is a fake domain and is not described in the Device-Tree.
-         * Therefore when the owner of the shared region is DOMID_IO, we will
-         * only find the borrowers.
-         */
-        if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) ||
-             (!owner_dom_io && strcmp(role_str, "owner") == 0) )
-        {
-            /*
-             * We found the first borrower of the region, the owner was not
-             * specified, so they should be assigned to dom_io.
-             */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
-                                       boot_shm_bank);
-            if ( ret )
-                return ret;
-        }
-
-        if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) )
-        {
-            /* Set up P2M foreign mapping for borrower domain. */
-            ret = map_regions_p2mt(d, _gfn(PFN_UP(gbase)), PFN_DOWN(psize),
-                                   _mfn(PFN_UP(pbase)), p2m_map_foreign_rw);
-            if ( ret )
-                return ret;
-        }
+        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
+        if ( ret )
+            return ret;
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729481.1134656 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEU-0000JL-Vp; Fri, 24 May 2024 12:41:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729481.1134656; Fri, 24 May 2024 12:41:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEU-0000I7-Oh; Fri, 24 May 2024 12:41:14 +0000
Received: by outflank-mailman (input) for mailman id 729481;
 Fri, 24 May 2024 12:41:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUES-00083d-PL
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:12 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id e5e42a2e-19ca-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:41:10 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 197AD168F;
 Fri, 24 May 2024 05:41:34 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0E04B3F641;
 Fri, 24 May 2024 05:41:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5e42a2e-19ca-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 4/7] xen/arm: Parse xen,shared-mem when host phys address is not provided
Date: Fri, 24 May 2024 13:40:52 +0100
Message-Id: <20240524124055.3871399-5-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Handle the parsing of the 'xen,shared-mem' property when the host physical
address is not provided, this commit is introducing the logic to parse it,
but the functionality is still not implemented and will be part of future
commits.

Rework the logic inside process_shm_node to check the shm_id before doing
the other checks, because it ease the logic itself, add more comment on
the logic.
Now when the host physical address is not provided, the value
INVALID_PADDR is chosen to signal this condition and it is stored as
start of the bank, due to that change also early_print_info_shmem and
init_sharedmem_pages are changed, to not handle banks with start equal
to INVALID_PADDR.

Another change is done inside meminfo_overlap_check, to skip banks that
are starting with the start address INVALID_PADDR, that function is used
to check banks from reserved memory, shared memory and ACPI and since
the comment above the function states that wrapping around is not handled,
it's unlikely for these bank to have the start address as INVALID_PADDR.
Same change is done inside consider_modules, find_unallocated_memory and
dt_unreserved_regions functions, in order to skip banks that starts with
INVALID_PADDR from any computation.
The changes above holds because of this consideration.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v4 changes:
 - Fix wrong condition for paddr_assigned (Michal)
v3 changes:
 - fix typo in commit msg, add R-by Michal
v2 changes:
 - fix comments, add parenthesis to some conditions, remove unneeded
   variables, remove else branch, increment counter in the for loop,
   skip INVALID_PADDR start banks from also consider_modules,
   find_unallocated_memory and dt_unreserved_regions. (Michal)
---
 xen/arch/arm/arm32/mmu/mm.c |  11 +++-
 xen/arch/arm/domain_build.c |   5 ++
 xen/arch/arm/setup.c        |  14 +++-
 xen/arch/arm/static-shmem.c | 125 +++++++++++++++++++++++++-----------
 4 files changed, 111 insertions(+), 44 deletions(-)

diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c
index be480c31ea05..30a7aa1e8e51 100644
--- a/xen/arch/arm/arm32/mmu/mm.c
+++ b/xen/arch/arm/arm32/mmu/mm.c
@@ -101,8 +101,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
     nr += reserved_mem->nr_banks;
     for ( ; i - nr < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - nr].start;
-        paddr_t r_e = r_s + shmem->bank[i - nr].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - nr].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - nr].size;
 
         if ( s < r_e && r_s < e )
         {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 968c497efc78..02e741685102 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -927,6 +927,11 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
         for ( j = 0; j < mem_banks[i]->nr_banks; j++ )
         {
             start = mem_banks[i]->bank[j].start;
+
+            /* Shared memory banks can contain INVALID_PADDR as start */
+            if ( INVALID_PADDR == start )
+                continue;
+
             end = mem_banks[i]->bank[j].start + mem_banks[i]->bank[j].size;
             res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
                                         PFN_DOWN(end - 1));
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index c4e5c19b11d6..0c2fdaceaf21 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -240,8 +240,15 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
     offset = reserved_mem->nr_banks;
     for ( ; i - offset < shmem->nr_banks; i++ )
     {
-        paddr_t r_s = shmem->bank[i - offset].start;
-        paddr_t r_e = r_s + shmem->bank[i - offset].size;
+        paddr_t r_s, r_e;
+
+        r_s = shmem->bank[i - offset].start;
+
+        /* Shared memory banks can contain INVALID_PADDR as start */
+        if ( INVALID_PADDR == r_s )
+            continue;
+
+        r_e = r_s + shmem->bank[i - offset].size;
 
         if ( s < r_e && r_s < e )
         {
@@ -272,7 +279,8 @@ static bool __init meminfo_overlap_check(const struct membanks *mem,
         bank_start = mem->bank[i].start;
         bank_end = bank_start + mem->bank[i].size;
 
-        if ( region_end <= bank_start || region_start >= bank_end )
+        if ( INVALID_PADDR == bank_start || region_end <= bank_start ||
+             region_start >= bank_end )
             continue;
         else
         {
diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index c15a65130659..bc093b9da9ea 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -264,6 +264,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
+        if ( INVALID_PADDR == pbase )
+        {
+            printk("%pd: host physical address must be chosen by users at the moment", d);
+            return -EINVAL;
+        }
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -377,7 +383,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
 {
     const struct fdt_property *prop, *prop_id, *prop_role;
     const __be32 *cell;
-    paddr_t paddr, gaddr, size, end;
+    paddr_t paddr = INVALID_PADDR;
+    paddr_t gaddr, size, end;
     struct membanks *mem = bootinfo_get_shmem();
     struct shmem_membank_extra *shmem_extra = bootinfo_get_shmem_extra();
     unsigned int i;
@@ -432,24 +439,37 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     if ( !prop )
         return -ENOENT;
 
+    cell = (const __be32 *)prop->data;
     if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) )
     {
-        if ( len == dt_cells_to_size(size_cells + address_cells) )
-            printk("fdt: host physical address must be chosen by users at the moment.\n");
-
-        printk("fdt: invalid `xen,shared-mem` property.\n");
-        return -EINVAL;
+        if ( len == dt_cells_to_size(address_cells + size_cells) )
+            device_tree_get_reg(&cell, address_cells, size_cells, &gaddr,
+                                &size);
+        else
+        {
+            printk("fdt: invalid `xen,shared-mem` property.\n");
+            return -EINVAL;
+        }
     }
+    else
+    {
+        device_tree_get_reg(&cell, address_cells, address_cells, &paddr,
+                            &gaddr);
+        size = dt_next_cell(size_cells, &cell);
 
-    cell = (const __be32 *)prop->data;
-    device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
-    size = dt_next_cell(size_cells, &cell);
+        if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+        {
+            printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+                paddr);
+            return -EINVAL;
+        }
 
-    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
-    {
-        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
-               paddr);
-        return -EINVAL;
+        end = paddr + size;
+        if ( end <= paddr )
+        {
+            printk("fdt: static shared memory region %s overflow\n", shm_id);
+            return -EINVAL;
+        }
     }
 
     if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
@@ -471,39 +491,64 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
         return -EINVAL;
     }
 
-    end = paddr + size;
-    if ( end <= paddr )
-    {
-        printk("fdt: static shared memory region %s overflow\n", shm_id);
-        return -EINVAL;
-    }
-
     for ( i = 0; i < mem->nr_banks; i++ )
     {
         /*
          * Meet the following check:
-         * 1) The shm ID matches and the region exactly match
-         * 2) The shm ID doesn't match and the region doesn't overlap
-         * with an existing one
+         * - when host address is provided:
+         *   1) The shm ID matches and the region exactly match
+         *   2) The shm ID doesn't match and the region doesn't overlap
+         *      with an existing one
+         * - when host address is not provided:
+         *   1) The shm ID matches and the region size exactly match
          */
-        if ( paddr == mem->bank[i].start && size == mem->bank[i].size )
+        bool paddr_assigned = (INVALID_PADDR != paddr);
+
+        if ( strncmp(shm_id, shmem_extra[i].shm_id, MAX_SHM_ID_LENGTH) == 0 )
         {
-            if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                         MAX_SHM_ID_LENGTH) == 0  )
+            /*
+             * Regions have same shm_id (cases):
+             * 1) physical host address is supplied:
+             *    - OK:   paddr is equal and size is equal (same region)
+             *    - Fail: paddr doesn't match or size doesn't match (there
+             *            cannot exists two shmem regions with same shm_id)
+             * 2) physical host address is NOT supplied:
+             *    - OK:   size is equal (same region)
+             *    - Fail: size is not equal (same shm_id must identify only one
+             *            region, there can't be two different regions with same
+             *            shm_id)
+             */
+            bool start_match = paddr_assigned ? (paddr == mem->bank[i].start) :
+                                                true;
+
+            if ( start_match && (size == mem->bank[i].size) )
                 break;
             else
             {
-                printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n",
+                printk("fdt: different shared memory region could not share the same shm ID %s\n",
                        shm_id);
                 return -EINVAL;
             }
         }
-        else if ( strncmp(shm_id, shmem_extra[i].shm_id,
-                          MAX_SHM_ID_LENGTH) != 0 )
+
+        /*
+         * Regions have different shm_id (cases):
+         * 1) physical host address is supplied:
+         *    - OK:   paddr different, or size different (case where paddr
+         *            is equal but psize is different are wrong, but they
+         *            are handled later when checking for overlapping)
+         *    - Fail: paddr equal and size equal (the same region can't be
+         *            identified with different shm_id)
+         * 2) physical host address is NOT supplied:
+         *    - OK:   Both have different shm_id so even with same size they
+         *            can exists
+         */
+        if ( !paddr_assigned || (paddr != mem->bank[i].start) ||
+             (size != mem->bank[i].size) )
             continue;
         else
         {
-            printk("fdt: different shared memory region could not share the same shm ID %s\n",
+            printk("fdt: xen,shm-id %s does not match for all the nodes using the same region\n",
                    shm_id);
             return -EINVAL;
         }
@@ -513,7 +558,8 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     {
         if (i < mem->max_banks)
         {
-            if ( check_reserved_regions_overlap(paddr, size) )
+            if ( (paddr != INVALID_PADDR) &&
+                 check_reserved_regions_overlap(paddr, size) )
                 return -EINVAL;
 
             /* Static shared memory shall be reserved from any other use. */
@@ -583,13 +629,13 @@ void __init early_print_info_shmem(void)
 {
     const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
+    unsigned int printed = 0;
 
-    for ( bank = 0; bank < shmem->nr_banks; bank++ )
-    {
-        printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", bank,
-               shmem->bank[bank].start,
-               shmem->bank[bank].start + shmem->bank[bank].size - 1);
-    }
+    for ( bank = 0; bank < shmem->nr_banks; bank++, printed++ )
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            printk(" SHMEM[%u]: %"PRIpaddr" - %"PRIpaddr"\n", printed,
+                shmem->bank[bank].start,
+                shmem->bank[bank].start + shmem->bank[bank].size - 1);
 }
 
 void __init init_sharedmem_pages(void)
@@ -598,7 +644,8 @@ void __init init_sharedmem_pages(void)
     unsigned int bank;
 
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
-        init_staticmem_bank(&shmem->bank[bank]);
+        if ( shmem->bank[bank].start != INVALID_PADDR )
+            init_staticmem_bank(&shmem->bank[bank]);
 }
 
 int __init remove_shm_from_rangeset(const struct kernel_info *kinfo,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729478.1134626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-000841-3b; Fri, 24 May 2024 12:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729478.1134626; Fri, 24 May 2024 12:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-00083u-12; Fri, 24 May 2024 12:41:13 +0000
Received: by outflank-mailman (input) for mailman id 729478;
 Fri, 24 May 2024 12:41:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUER-00082M-NC
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:11 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id e3343d1c-19ca-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 14:41:06 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6F192339;
 Fri, 24 May 2024 05:41:29 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7D1253F641;
 Fri, 24 May 2024 05:41:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3343d1c-19ca-11ef-90a1-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 0/7] Static shared memory followup v2 - pt2
Date: Fri, 24 May 2024 13:40:48 +0100
Message-Id: <20240524124055.3871399-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This serie is a partial rework of this other serie:
https://patchwork.kernel.org/project/xen-devel/cover/20231206090623.1932275-1-Penny.Zheng@arm.com/

The original serie is addressing an issue of the static shared memory feature
that impacts the memory footprint of other component when the feature is
enabled, another issue impacts the device tree generation for the guests when
the feature is enabled and used and the last one is a missing feature that is
the option to have a static shared memory region that is not from the host
address space.

This serie is handling some comment on the original serie and it is splitting
the rework in two part, this first part is addressing the memory footprint issue
and the device tree generation and currently is fully merged
(https://patchwork.kernel.org/project/xen-devel/cover/20240418073652.3622828-1-luca.fancellu@arm.com/),
this serie is addressing the static shared memory allocation from the Xen heap.

Luca Fancellu (5):
  xen/arm: Lookup bootinfo shm bank during the mapping
  xen/arm: Wrap shared memory mapping code in one function
  xen/arm: Parse xen,shared-mem when host phys address is not provided
  xen/arm: Rework heap page allocation outside allocate_bank_memory
  xen/arm: Implement the logic for static shared memory from Xen heap

Penny Zheng (2):
  xen/p2m: put reference for level 2 superpage
  xen/docs: Describe static shared memory when host address is not
    provided

 docs/misc/arm/device-tree/booting.txt   |  52 ++-
 xen/arch/arm/arm32/mmu/mm.c             |  11 +-
 xen/arch/arm/dom0less-build.c           |   4 +-
 xen/arch/arm/domain_build.c             |  84 +++--
 xen/arch/arm/include/asm/domain_build.h |   9 +-
 xen/arch/arm/mmu/p2m.c                  |  82 +++--
 xen/arch/arm/setup.c                    |  14 +-
 xen/arch/arm/static-shmem.c             | 432 +++++++++++++++++-------
 8 files changed, 502 insertions(+), 186 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729483.1134667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEV-0000VE-NZ; Fri, 24 May 2024 12:41:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729483.1134667; Fri, 24 May 2024 12:41:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEV-0000St-IZ; Fri, 24 May 2024 12:41:15 +0000
Received: by outflank-mailman (input) for mailman id 729483;
 Fri, 24 May 2024 12:41:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUET-00083d-Qj
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:13 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id e6a020a3-19ca-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:41:11 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D4A41691;
 Fri, 24 May 2024 05:41:35 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3BAF53F641;
 Fri, 24 May 2024 05:41:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6a020a3-19ca-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 5/7] xen/arm: Rework heap page allocation outside allocate_bank_memory
Date: Fri, 24 May 2024 13:40:53 +0100
Message-Id: <20240524124055.3871399-6-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The function allocate_bank_memory allocates pages from the heap and
maps them to the guest using guest_physmap_add_page.

As a preparation work to support static shared memory bank when the
host physical address is not provided, Xen needs to allocate memory
from the heap, so rework allocate_bank_memory moving out the page
allocation in a new function called allocate_domheap_memory.

The function allocate_domheap_memory takes a callback function and
a pointer to some extra information passed to the callback and this
function will be called for every region, until a defined size is
reached.

In order to keep allocate_bank_memory functionality, the callback
passed to allocate_domheap_memory is a wrapper for
guest_physmap_add_page.

Let allocate_domheap_memory be externally visible, in order to use
it in the future from the static shared memory module.

Take the opportunity to change the signature of allocate_bank_memory
and remove the 'struct domain' parameter, which can be retrieved from
'struct kernel_info'.

No functional changes is intended.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - Add R-by Michal
v2:
 - Reduced scope of pg var in allocate_domheap_memory, removed not
   necessary BUG_ON(), changed callback to return bool and fix
   comment. (Michal)
---
 xen/arch/arm/dom0less-build.c           |  4 +-
 xen/arch/arm/domain_build.c             | 79 +++++++++++++++++--------
 xen/arch/arm/include/asm/domain_build.h |  9 ++-
 3 files changed, 62 insertions(+), 30 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242f4..20ddf6f8f250 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -60,12 +60,12 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
 
     mem->nr_banks = 0;
     bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
                                bank_size) )
         goto fail;
 
     bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem);
-    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
+    if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
                                bank_size) )
         goto fail;
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 02e741685102..669970c86fd5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -417,30 +417,15 @@ static void __init allocate_memory_11(struct domain *d,
 }
 
 #ifdef CONFIG_DOM0LESS_BOOT
-bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                                 gfn_t sgfn, paddr_t tot_size)
+bool __init allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                                    alloc_domheap_mem_cb cb, void *extra)
 {
-    struct membanks *mem = kernel_info_get_mem(kinfo);
-    int res;
-    struct page_info *pg;
-    struct membank *bank;
-    unsigned int max_order = ~0;
-
-    /*
-     * allocate_bank_memory can be called with a tot_size of zero for
-     * the second memory bank. It is not an error and we can safely
-     * avoid creating a zero-size memory bank.
-     */
-    if ( tot_size == 0 )
-        return true;
-
-    bank = &mem->bank[mem->nr_banks];
-    bank->start = gfn_to_gaddr(sgfn);
-    bank->size = tot_size;
+    unsigned int max_order = UINT_MAX;
 
     while ( tot_size > 0 )
     {
         unsigned int order = get_allocation_size(tot_size);
+        struct page_info *pg;
 
         order = min(max_order, order);
 
@@ -463,17 +448,61 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
             continue;
         }
 
-        res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
-        if ( res )
-        {
-            dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        if ( !cb(d, pg, order, extra) )
             return false;
-        }
 
-        sgfn = gfn_add(sgfn, 1UL << order);
         tot_size -= (1ULL << (PAGE_SHIFT + order));
     }
 
+    return true;
+}
+
+static bool __init guest_map_pages(struct domain *d, struct page_info *pg,
+                                   unsigned int order, void *extra)
+{
+    gfn_t *sgfn = (gfn_t *)extra;
+    int res;
+
+    BUG_ON(!sgfn);
+    res = guest_physmap_add_page(d, *sgfn, page_to_mfn(pg), order);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        return false;
+    }
+
+    *sgfn = gfn_add(*sgfn, 1UL << order);
+
+    return true;
+}
+
+bool __init allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                                 paddr_t tot_size)
+{
+    struct membanks *mem = kernel_info_get_mem(kinfo);
+    struct domain *d = kinfo->d;
+    struct membank *bank;
+
+    /*
+     * allocate_bank_memory can be called with a tot_size of zero for
+     * the second memory bank. It is not an error and we can safely
+     * avoid creating a zero-size memory bank.
+     */
+    if ( tot_size == 0 )
+        return true;
+
+    bank = &mem->bank[mem->nr_banks];
+    bank->start = gfn_to_gaddr(sgfn);
+    bank->size = tot_size;
+
+    /*
+     * Allocate pages from the heap until tot_size is zero and map them to the
+     * guest using guest_map_pages, passing the starting gfn as extra parameter
+     * for the map operation.
+     */
+    if ( !allocate_domheap_memory(d, tot_size, guest_map_pages, &sgfn) )
+        return false;
+
     mem->nr_banks++;
     kinfo->unassigned_mem -= bank->size;
 
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 45936212ca21..e712afbc7fbf 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -5,9 +5,12 @@
 #include <asm/kernel.h>
 
 typedef __be32 gic_interrupt_t[3];
-
-bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
-                          gfn_t sgfn, paddr_t tot_size);
+typedef bool (*alloc_domheap_mem_cb)(struct domain *d, struct page_info *pg,
+                                     unsigned int order, void *extra);
+bool allocate_domheap_memory(struct domain *d, paddr_t tot_size,
+                             alloc_domheap_mem_cb cb, void *extra);
+bool allocate_bank_memory(struct kernel_info *kinfo, gfn_t sgfn,
+                          paddr_t tot_size);
 int construct_domain(struct domain *d, struct kernel_info *kinfo);
 int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit);
 int make_chosen_node(const struct kernel_info *kinfo);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729480.1134638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-0008Cw-MM; Fri, 24 May 2024 12:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729480.1134638; Fri, 24 May 2024 12:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUET-0008Ax-Fo; Fri, 24 May 2024 12:41:13 +0000
Received: by outflank-mailman (input) for mailman id 729480;
 Fri, 24 May 2024 12:41:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUES-00082M-Bo
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:12 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id e3cc5583-19ca-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 14:41:07 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 83158DA7;
 Fri, 24 May 2024 05:41:30 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 917403F641;
 Fri, 24 May 2024 05:41:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3cc5583-19ca-11ef-90a1-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 1/7] xen/arm: Lookup bootinfo shm bank during the mapping
Date: Fri, 24 May 2024 13:40:49 +0100
Message-Id: <20240524124055.3871399-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The current static shared memory code is using bootinfo banks when it
needs to find the number of borrowers, so every time assign_shared_memory
is called, the bank is searched in the bootinfo.shmem structure.

There is nothing wrong with it, however the bank can be used also to
retrieve the start address and size and also to pass less argument to
assign_shared_memory. When retrieving the information from the bootinfo
bank, it's also possible to move the checks on alignment to
process_shm_node in the early stages.

So create a new function find_shm_bank_by_id() which takes a
'struct shared_meminfo' structure and the shared memory ID, to look for a
bank with a matching ID, take the physical host address and size from the
bank, pass the bank to assign_shared_memory() removing the now unnecessary
arguments and finally remove the acquire_nr_borrower_domain() function
since now the information can be extracted from the passed bank.
Move the "xen,shm-id" parsing early in process_shm to bail out quickly in
case of errors (unlikely), as said above, move the checks on alignment
to process_shm_node.

Drawback of this change is that now the bootinfo are used also when the
bank doesn't need to be allocated, however it will be convenient later
to use it as an argument for assign_shared_memory when dealing with
the use case where the Host physical address is not supplied by the user.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v3 changes:
 - switch strncmp with strcmp in find_shm_bank_by_id, fix commit msg typo,
   add R-by Michal.
v2 changes:
 - fix typo commit msg, renamed find_shm() to find_shm_bank_by_id(),
   swap region size check different from zero and size alignment, remove
   not necessary BUGON(). (Michal)
---
 xen/arch/arm/static-shmem.c | 100 +++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 47 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 78881dd1d3f7..0a1c327e90ea 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -19,29 +19,21 @@ static void __init __maybe_unused build_assertions(void)
                  offsetof(struct shared_meminfo, bank)));
 }
 
-static int __init acquire_nr_borrower_domain(struct domain *d,
-                                             paddr_t pbase, paddr_t psize,
-                                             unsigned long *nr_borrowers)
+static const struct membank __init *
+find_shm_bank_by_id(const struct membanks *shmem, const char *shm_id)
 {
-    const struct membanks *shmem = bootinfo_get_shmem();
     unsigned int bank;
 
-    /* Iterate reserved memory to find requested shm bank. */
     for ( bank = 0 ; bank < shmem->nr_banks; bank++ )
     {
-        paddr_t bank_start = shmem->bank[bank].start;
-        paddr_t bank_size = shmem->bank[bank].size;
-
-        if ( (pbase == bank_start) && (psize == bank_size) )
+        if ( strcmp(shm_id, shmem->bank[bank].shmem_extra->shm_id) == 0 )
             break;
     }
 
     if ( bank == shmem->nr_banks )
-        return -ENOENT;
+        return NULL;
 
-    *nr_borrowers = shmem->bank[bank].shmem_extra->nr_shm_borrowers;
-
-    return 0;
+    return &shmem->bank[bank];
 }
 
 /*
@@ -103,14 +95,18 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     return smfn;
 }
 
-static int __init assign_shared_memory(struct domain *d,
-                                       paddr_t pbase, paddr_t psize,
-                                       paddr_t gbase)
+static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       const struct membank *shm_bank)
 {
     mfn_t smfn;
     int ret = 0;
     unsigned long nr_pages, nr_borrowers, i;
     struct page_info *page;
+    paddr_t pbase, psize;
+
+    pbase = shm_bank->start;
+    psize = shm_bank->size;
+    nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
     printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
            d, pbase, pbase + psize);
@@ -135,14 +131,6 @@ static int __init assign_shared_memory(struct domain *d,
         }
     }
 
-    /*
-     * Get the right amount of references per page, which is the number of
-     * borrower domains.
-     */
-    ret = acquire_nr_borrower_domain(d, pbase, psize, &nr_borrowers);
-    if ( ret )
-        return ret;
-
     /*
      * Instead of letting borrower domain get a page ref, we add as many
      * additional reference as the number of borrowers when the owner
@@ -199,6 +187,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
 
     dt_for_each_child_node(node, shm_node)
     {
+        const struct membank *boot_shm_bank;
         const struct dt_property *prop;
         const __be32 *cells;
         uint32_t addr_cells, size_cells;
@@ -212,6 +201,23 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") )
             continue;
 
+        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
+        {
+            printk("%pd: invalid \"xen,shm-id\" property", d);
+            return -EINVAL;
+        }
+        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
+
+        boot_shm_bank = find_shm_bank_by_id(bootinfo_get_shmem(), shm_id);
+        if ( !boot_shm_bank )
+        {
+            printk("%pd: static shared memory bank not found: '%s'", d, shm_id);
+            return -ENOENT;
+        }
+
+        pbase = boot_shm_bank->start;
+        psize = boot_shm_bank->size;
+
         /*
          * xen,shared-mem = <pbase, gbase, size>;
          * TODO: pbase is optional.
@@ -221,20 +227,7 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
-        psize = dt_read_paddr(cells, size_cells);
-        if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
-        {
-            printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
-                   d, pbase, gbase);
-            return -EINVAL;
-        }
-        if ( !IS_ALIGNED(psize, PAGE_SIZE) )
-        {
-            printk("%pd: size 0x%"PRIpaddr" is not suitably aligned\n",
-                   d, psize);
-            return -EINVAL;
-        }
+        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
         for ( i = 0; i < PFN_DOWN(psize); i++ )
             if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
@@ -251,13 +244,6 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         if ( dt_property_read_string(shm_node, "role", &role_str) == 0 )
             owner_dom_io = false;
 
-        if ( dt_property_read_string(shm_node, "xen,shm-id", &shm_id) )
-        {
-            printk("%pd: invalid \"xen,shm-id\" property", d);
-            return -EINVAL;
-        }
-        BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH));
-
         /*
          * DOMID_IO is a fake domain and is not described in the Device-Tree.
          * Therefore when the owner of the shared region is DOMID_IO, we will
@@ -270,8 +256,8 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
              * We found the first borrower of the region, the owner was not
              * specified, so they should be assigned to dom_io.
              */
-            ret = assign_shared_memory(owner_dom_io ? dom_io : d,
-                                       pbase, psize, gbase);
+            ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                       boot_shm_bank);
             if ( ret )
                 return ret;
         }
@@ -439,12 +425,32 @@ int __init process_shm_node(const void *fdt, int node, uint32_t address_cells,
     device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr);
     size = dt_next_cell(size_cells, &cell);
 
+    if ( !IS_ALIGNED(paddr, PAGE_SIZE) )
+    {
+        printk("fdt: physical address 0x%"PRIpaddr" is not suitably aligned.\n",
+               paddr);
+        return -EINVAL;
+    }
+
+    if ( !IS_ALIGNED(gaddr, PAGE_SIZE) )
+    {
+        printk("fdt: guest address 0x%"PRIpaddr" is not suitably aligned.\n",
+               gaddr);
+        return -EINVAL;
+    }
+
     if ( !size )
     {
         printk("fdt: the size for static shared memory region can not be zero\n");
         return -EINVAL;
     }
 
+    if ( !IS_ALIGNED(size, PAGE_SIZE) )
+    {
+        printk("fdt: size 0x%"PRIpaddr" is not suitably aligned\n", size);
+        return -EINVAL;
+    }
+
     end = paddr + size;
     if ( end <= paddr )
     {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729485.1134687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEX-000135-7T; Fri, 24 May 2024 12:41:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729485.1134687; Fri, 24 May 2024 12:41:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEW-00010E-UO; Fri, 24 May 2024 12:41:16 +0000
Received: by outflank-mailman (input) for mailman id 729485;
 Fri, 24 May 2024 12:41:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUEU-00083d-Pf
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:14 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id e72be675-19ca-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:41:12 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 417081692;
 Fri, 24 May 2024 05:41:36 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4FA0C3F641;
 Fri, 24 May 2024 05:41:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e72be675-19ca-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 6/7] xen/arm: Implement the logic for static shared memory from Xen heap
Date: Fri, 24 May 2024 13:40:54 +0100
Message-Id: <20240524124055.3871399-7-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This commit implements the logic to have the static shared memory banks
from the Xen heap instead of having the host physical address passed from
the user.

When the host physical address is not supplied, the physical memory is
taken from the Xen heap using allocate_domheap_memory, the allocation
needs to occur at the first handled DT node and the allocated banks
need to be saved somewhere.

Introduce the 'shm_heap_banks' for that reason, a struct that will hold
the banks allocated from the heap, its field bank[].shmem_extra will be
used to point to the bootinfo shared memory banks .shmem_extra space, so
that there is not further allocation of memory and every bank in
shm_heap_banks can be safely identified by the shm_id to reconstruct its
traceability and if it was allocated or not.

A search into 'shm_heap_banks' will reveal if the banks were allocated
or not, in case the host address is not passed, and the callback given
to allocate_domheap_memory will store the banks in the structure and
map them to the current domain, to do that, some changes to
acquire_shared_memory_bank are made to let it differentiate if the bank
is from the heap and if it is, then assign_pages is called for every
bank.

When the bank is already allocated, for every bank allocated with the
corresponding shm_id, handle_shared_mem_bank is called and the mapping
are done.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v4 changes:
 - Add R-by Michal
v3 changes:
 - reworded commit msg section, swap role_str and gbase in
   alloc_heap_pages_cb_extra to avoid padding hole in arm32, remove
   not needed printk, modify printk to print KB instead of KB, swap
   strncmp for strcmp, reduced memory footprint for shm_heap_banks.
   (Michal)
v2 changes:
 - add static inline get_shmem_heap_banks(), given the changes to the
   struct membanks interface. Rebase changes due to removal of
   owner_dom_io arg from handle_shared_mem_bank.
   Change save_map_heap_pages return type given the changes to the
   allocate_domheap_memory callback type.
---
 xen/arch/arm/static-shmem.c | 187 ++++++++++++++++++++++++++++++------
 1 file changed, 155 insertions(+), 32 deletions(-)

diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index bc093b9da9ea..dbb017c7d76e 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -9,6 +9,25 @@
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
 
+typedef struct {
+    struct domain *d;
+    const char *role_str;
+    paddr_t gbase;
+    struct shmem_membank_extra *bank_extra_info;
+} alloc_heap_pages_cb_extra;
+
+static struct {
+    struct membanks_hdr common;
+    struct membank bank[NR_SHMEM_BANKS];
+} shm_heap_banks __initdata = {
+    .common.max_banks = NR_SHMEM_BANKS
+};
+
+static inline struct membanks *get_shmem_heap_banks(void)
+{
+    return container_of(&shm_heap_banks.common, struct membanks, common);
+}
+
 static void __init __maybe_unused build_assertions(void)
 {
     /*
@@ -63,7 +82,8 @@ static bool __init is_shm_allocated_to_domio(paddr_t pbase)
 }
 
 static mfn_t __init acquire_shared_memory_bank(struct domain *d,
-                                               paddr_t pbase, paddr_t psize)
+                                               paddr_t pbase, paddr_t psize,
+                                               bool bank_from_heap)
 {
     mfn_t smfn;
     unsigned long nr_pfns;
@@ -83,19 +103,31 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d,
     d->max_pages += nr_pfns;
 
     smfn = maddr_to_mfn(pbase);
-    res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+    if ( bank_from_heap )
+        /*
+         * When host address is not provided, static shared memory is
+         * allocated from heap and shall be assigned to owner domain.
+         */
+        res = assign_pages(maddr_to_page(pbase), nr_pfns, d, 0);
+    else
+        res = acquire_domstatic_pages(d, smfn, nr_pfns, 0);
+
     if ( res )
     {
-        printk(XENLOG_ERR
-               "%pd: failed to acquire static memory: %d.\n", d, res);
-        d->max_pages -= nr_pfns;
-        return INVALID_MFN;
+        printk(XENLOG_ERR "%pd: failed to %s static memory: %d.\n", d,
+               bank_from_heap ? "assign" : "acquire", res);
+        goto fail;
     }
 
     return smfn;
+
+ fail:
+    d->max_pages -= nr_pfns;
+    return INVALID_MFN;
 }
 
 static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
+                                       bool bank_from_heap,
                                        const struct membank *shm_bank)
 {
     mfn_t smfn;
@@ -108,10 +140,7 @@ static int __init assign_shared_memory(struct domain *d, paddr_t gbase,
     psize = shm_bank->size;
     nr_borrowers = shm_bank->shmem_extra->nr_shm_borrowers;
 
-    printk("%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n",
-           d, pbase, pbase + psize);
-
-    smfn = acquire_shared_memory_bank(d, pbase, psize);
+    smfn = acquire_shared_memory_bank(d, pbase, psize, bank_from_heap);
     if ( mfn_eq(smfn, INVALID_MFN) )
         return -EINVAL;
 
@@ -182,6 +211,7 @@ append_shm_bank_to_domain(struct kernel_info *kinfo, paddr_t start,
 
 static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
                                          const char *role_str,
+                                         bool bank_from_heap,
                                          const struct membank *shm_bank)
 {
     bool owner_dom_io = true;
@@ -210,7 +240,8 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
          * We found the first borrower of the region, the owner was not
          * specified, so they should be assigned to dom_io.
          */
-        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase, shm_bank);
+        ret = assign_shared_memory(owner_dom_io ? dom_io : d, gbase,
+                                   bank_from_heap, shm_bank);
         if ( ret )
             return ret;
     }
@@ -227,6 +258,39 @@ static int __init handle_shared_mem_bank(struct domain *d, paddr_t gbase,
     return 0;
 }
 
+static bool __init save_map_heap_pages(struct domain *d, struct page_info *pg,
+                                       unsigned int order, void *extra)
+{
+    alloc_heap_pages_cb_extra *b_extra = (alloc_heap_pages_cb_extra *)extra;
+    int idx = shm_heap_banks.common.nr_banks;
+    int ret = -ENOSPC;
+
+    BUG_ON(!b_extra);
+
+    if ( idx < shm_heap_banks.common.max_banks )
+    {
+        shm_heap_banks.bank[idx].start = page_to_maddr(pg);
+        shm_heap_banks.bank[idx].size = (1ULL << (PAGE_SHIFT + order));
+        shm_heap_banks.bank[idx].shmem_extra = b_extra->bank_extra_info;
+        shm_heap_banks.common.nr_banks++;
+
+        ret = handle_shared_mem_bank(b_extra->d, b_extra->gbase,
+                                     b_extra->role_str, true,
+                                     &shm_heap_banks.bank[idx]);
+        if ( !ret )
+        {
+            /* Increment guest physical address for next mapping */
+            b_extra->gbase += shm_heap_banks.bank[idx].size;
+            return true;
+        }
+    }
+
+    printk("Failed to allocate static shared memory from Xen heap: (%d)\n",
+           ret);
+
+    return false;
+}
+
 int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                        const struct dt_device_node *node)
 {
@@ -264,38 +328,97 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         pbase = boot_shm_bank->start;
         psize = boot_shm_bank->size;
 
-        if ( INVALID_PADDR == pbase )
-        {
-            printk("%pd: host physical address must be chosen by users at the moment", d);
-            return -EINVAL;
-        }
+        /* "role" property is optional */
+        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
+            role_str = NULL;
 
         /*
-         * xen,shared-mem = <pbase, gbase, size>;
-         * TODO: pbase is optional.
+         * xen,shared-mem = <[pbase,] gbase, size>;
+         * pbase is optional.
          */
         addr_cells = dt_n_addr_cells(shm_node);
         size_cells = dt_n_size_cells(shm_node);
         prop = dt_find_property(shm_node, "xen,shared-mem", NULL);
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
-        gbase = dt_read_paddr(cells + addr_cells, addr_cells);
 
-        for ( i = 0; i < PFN_DOWN(psize); i++ )
-            if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
-            {
-                printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
-                       d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
-                return -EINVAL;
-            }
+        if ( pbase != INVALID_PADDR )
+        {
+            /* guest phys address is after host phys address */
+            gbase = dt_read_paddr(cells + addr_cells, addr_cells);
+
+            for ( i = 0; i < PFN_DOWN(psize); i++ )
+                if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
+                {
+                    printk("%pd: invalid physical address 0x%"PRI_mfn"\n",
+                        d, mfn_x(mfn_add(maddr_to_mfn(pbase), i)));
+                    return -EINVAL;
+                }
+
+            /* The host physical address is supplied by the user */
+            ret = handle_shared_mem_bank(d, gbase, role_str, false,
+                                         boot_shm_bank);
+            if ( ret )
+                return ret;
+        }
+        else
+        {
+            /*
+             * The host physical address is not supplied by the user, so it
+             * means that the banks needs to be allocated from the Xen heap,
+             * look into the already allocated banks from the heap.
+             */
+            const struct membank *alloc_bank =
+                find_shm_bank_by_id(get_shmem_heap_banks(), shm_id);
 
-        /* "role" property is optional */
-        if ( dt_property_read_string(shm_node, "role", &role_str) != 0 )
-            role_str = NULL;
+            /* guest phys address is right at the beginning */
+            gbase = dt_read_paddr(cells, addr_cells);
 
-        ret = handle_shared_mem_bank(d, gbase, role_str, boot_shm_bank);
-        if ( ret )
-            return ret;
+            if ( !alloc_bank )
+            {
+                alloc_heap_pages_cb_extra cb_arg = { d, role_str, gbase,
+                    boot_shm_bank->shmem_extra };
+
+                /* shm_id identified bank is not yet allocated */
+                if ( !allocate_domheap_memory(NULL, psize, save_map_heap_pages,
+                                              &cb_arg) )
+                {
+                    printk(XENLOG_ERR
+                           "Failed to allocate (%"PRIpaddr"KB) pages as static shared memory from heap\n",
+                           psize >> 10);
+                    return -EINVAL;
+                }
+            }
+            else
+            {
+                /* shm_id identified bank is already allocated */
+                const struct membank *end_bank =
+                        &shm_heap_banks.bank[shm_heap_banks.common.nr_banks];
+                paddr_t gbase_bank = gbase;
+
+                /*
+                 * Static shared memory banks that are taken from the Xen heap
+                 * are allocated sequentially in shm_heap_banks, so starting
+                 * from the first bank found identified by shm_id, the code can
+                 * just advance by one bank at the time until it reaches the end
+                 * of the array or it finds another bank NOT identified by
+                 * shm_id
+                 */
+                for ( ; alloc_bank < end_bank; alloc_bank++ )
+                {
+                    if ( strcmp(shm_id, alloc_bank->shmem_extra->shm_id) != 0 )
+                        break;
+
+                    ret = handle_shared_mem_bank(d, gbase_bank, role_str, true,
+                                                 alloc_bank);
+                    if ( ret )
+                        return ret;
+
+                    /* Increment guest physical address for next mapping */
+                    gbase_bank += alloc_bank->size;
+                }
+            }
+        }
 
         /*
          * Record static shared memory region info for later setting
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:41:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:41:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729486.1134692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEX-00015Q-Ks; Fri, 24 May 2024 12:41:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729486.1134692; Fri, 24 May 2024 12:41:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUEX-00014Q-8P; Fri, 24 May 2024 12:41:17 +0000
Received: by outflank-mailman (input) for mailman id 729486;
 Fri, 24 May 2024 12:41:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BPhY=M3=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sAUEV-00082M-CV
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:41:15 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTP
 id e7f8bfe5-19ca-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 14:41:14 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8970B169C;
 Fri, 24 May 2024 05:41:37 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 63B653F641;
 Fri, 24 May 2024 05:41:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7f8bfe5-19ca-11ef-90a1-e314d9c70b13
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v4 7/7] xen/docs: Describe static shared memory when host address is not provided
Date: Fri, 24 May 2024 13:40:55 +0100
Message-Id: <20240524124055.3871399-8-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240524124055.3871399-1-luca.fancellu@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

This commit describe the new scenario where host address is not provided
in "xen,shared-mem" property and a new example is added to the page to
explain in details.

Take the occasion to fix some typos in the page.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
v2:
 - Add Michal R-by
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-10-Penny.Zheng@arm.com/
   with some changes in the commit message.
---
 docs/misc/arm/device-tree/booting.txt | 52 ++++++++++++++++++++-------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2f6..ac4bad6fe5e0 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -590,7 +590,7 @@ communication.
     An array takes a physical address, which is the base address of the
     shared memory region in host physical address space, a size, and a guest
     physical address, as the target address of the mapping.
-    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
+    e.g. xen,shared-mem = < [host physical address] [guest address] [size] >;
 
     It shall also meet the following criteria:
     1) If the SHM ID matches with an existing region, the address range of the
@@ -601,8 +601,8 @@ communication.
     The number of cells for the host address (and size) is the same as the
     guest pseudo-physical address and they are inherited from the parent node.
 
-    Host physical address is optional, when missing Xen decides the location
-    (currently unimplemented).
+    Host physical address is optional, when missing Xen decides the location.
+    e.g. xen,shared-mem = < [guest address] [size] >;
 
 - role (Optional)
 
@@ -629,7 +629,7 @@ chosen {
         role = "owner";
         xen,shm-id = "my-shared-mem-0";
         xen,shared-mem = <0x10000000 0x10000000 0x10000000>;
-    }
+    };
 
     domU1 {
         compatible = "xen,domain";
@@ -640,25 +640,36 @@ chosen {
         vpl011;
 
         /*
-         * shared memory region identified as 0x0(xen,shm-id = <0x0>)
-         * is shared between Dom0 and DomU1.
+         * shared memory region "my-shared-mem-0" is shared
+         * between Dom0 and DomU1.
          */
         domU1-shared-mem@10000000 {
             compatible = "xen,domain-shared-memory-v1";
             role = "borrower";
             xen,shm-id = "my-shared-mem-0";
             xen,shared-mem = <0x10000000 0x50000000 0x10000000>;
-        }
+        };
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between DomU1 and DomU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * DomU1 and DomU2.
          */
         domU1-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x60000000 0x20000000>;
-        }
+        };
+
+        /*
+         * shared memory region "my-shared-mem-2" is shared between
+         * DomU1 and DomU2.
+         */
+        domU1-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "owner";
+            xen,shared-mem = <0x80000000 0x20000000>;
+        };
 
         ......
 
@@ -672,14 +683,21 @@ chosen {
         cpus = <1>;
 
         /*
-         * shared memory region identified as 0x1(xen,shm-id = <0x1>)
-         * is shared between domU1 and domU2.
+         * shared memory region "my-shared-mem-1" is shared between
+         * domU1 and domU2.
          */
         domU2-shared-mem@50000000 {
             compatible = "xen,domain-shared-memory-v1";
             xen,shm-id = "my-shared-mem-1";
             xen,shared-mem = <0x50000000 0x70000000 0x20000000>;
-        }
+        };
+
+        domU2-shared-mem-2 {
+            compatible = "xen,domain-shared-memory-v1";
+            xen,shm-id = "my-shared-mem-2";
+            role = "borrower";
+            xen,shared-mem = <0x90000000 0x20000000>;
+        };
 
         ......
     };
@@ -699,3 +717,11 @@ shared between DomU1 and DomU2. It will get mapped at 0x60000000 in DomU1 guest
 physical address space, and at 0x70000000 in DomU2 guest physical address space.
 DomU1 and DomU2 are both the borrower domain, the owner domain is the default
 owner domain DOMID_IO.
+
+For the static shared memory region "my-shared-mem-2", since host physical
+address is not provided by user, Xen will automatically allocate 512MB
+from heap as static shared memory to be shared between DomU1 and DomU2.
+The automatically allocated static shared memory will get mapped at
+0x80000000 in DomU1 guest physical address space, and at 0x90000000 in DomU2
+guest physical address space. DomU1 is explicitly defined as the owner domain,
+and DomU2 is the borrower domain.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 12:53:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:53:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729522.1134707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUQ1-0005m6-Sn; Fri, 24 May 2024 12:53:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729522.1134707; Fri, 24 May 2024 12:53:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUQ1-0005lz-Oj; Fri, 24 May 2024 12:53:09 +0000
Received: by outflank-mailman (input) for mailman id 729522;
 Fri, 24 May 2024 12:53:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LySr=M3=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sAUQ0-0005lt-F3
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:53:08 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 90b266ac-19cc-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 14:53:06 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2e6f33150bcso90738441fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 05:53:06 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a08ab81sm1542635f8f.39.2024.05.24.05.53.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 05:53:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90b266ac-19cc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716555186; x=1717159986; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Fv9tvMzcEEK8t6/deedDI7o04tn2bqSoB87nl6HbIC4=;
        b=bXCFHitsPnvm+imLzr4/yh5J9MG7duWNI6KPe8CNshThGJeMi/gMVJZoyrYoWjz9Zp
         9t10ptVTy9JoilK2Bushs8YufJR0xA1HrvraZZxh8RNOEM/IlVN3VdCMUi3tq7Skgm0q
         57Sb+LRzads2gNUsmj4rPsojVqLaoGKK8UCtQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716555186; x=1717159986;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Fv9tvMzcEEK8t6/deedDI7o04tn2bqSoB87nl6HbIC4=;
        b=LddsE+K5auuyjVCiGBj8+kqeQwF4IbNCLVSZnM7+21+4BytLpg8YMWl5i38mNVzltq
         szN1LpKcOulTRImTlpwA0JhQD4tJ/nhN6hFVwmJDbwL5Ue71JzEM9GhiRSp8PGbpsaLw
         V3RaDBQXicVr3G6EnT05uhVCMVmKvF3muNIq2PdxV7+Sa9Zg8LOIF+OmmUvbahW93I4E
         Kp/vhWTcGXz4uSkEfq8qYUtWQRmf2QTarsPCUm5GN70+BpMO9cVyAsJEQVLNfXp+MDju
         qx0Tp72rVte9XLKhvwCQjAV1C+TgKOLFSTNQxj0wO+22bWdgil2uINDg4x4JaJfuc/RG
         wJDA==
X-Gm-Message-State: AOJu0YzT6raHBEyTXa+00LkJcLfAQJEQlXcO1d07aGTxE6oOwCZ5W80t
	Q9DIOYny/zq47td0qvo/1pS6BaPriyn2k4Wj9KctLyLQiFXBaYVO3wFP9eAyIwc=
X-Google-Smtp-Source: AGHT+IFw6JSGevYtv9reLfexATI1GsqRB+8/kqwEt5fP6ePiGiw1LNLd+1bLKYbQB/wOAjMnsTP3+Q==
X-Received: by 2002:a2e:84c6:0:b0:2d6:dba1:6d37 with SMTP id 38308e7fff4ca-2e95b073d7cmr14585991fa.11.1716555185531;
        Fri, 24 May 2024 05:53:05 -0700 (PDT)
Date: Fri, 24 May 2024 14:53:04 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 3/8] x86/vlapic: Move lapic_load_hidden migration
 checks to the check hook
Message-ID: <ZlCNsKmXKCFC4AZz@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <499e029a7d2fce4fb9118b1e508313f369b37c79.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9Xy7W30WWZlb6-@macbook>
 <a1f7a182-4cf2-4fd1-ba8f-5b12a34b1af0@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a1f7a182-4cf2-4fd1-ba8f-5b12a34b1af0@cloud.com>

On Fri, May 24, 2024 at 12:16:00PM +0100, Alejandro Vallejo wrote:
> On 23/05/2024 15:50, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:22PM +0100, Alejandro Vallejo wrote:
> >> While at it, add a check for the reserved field in the hidden save area.
> >>
> >> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> >> ---
> >> v2:
> >>   * New patch. Addresses the missing check for rsvd_zero in v1.
> > 
> > Oh, it would be better if this was done at the time when rsvd_zero is
> > introduced.  I think this should be moved ahead of the series, so that
> > the patch that introduces rsvd_zero can add the check in
> > lapic_check_hidden().
> 
> I'll give that a whirl.
> 
> > 
> >> ---
> >>  xen/arch/x86/hvm/vlapic.c | 41 ++++++++++++++++++++++++++++-----------
> >>  1 file changed, 30 insertions(+), 11 deletions(-)
> >>
> >> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> >> index 8a244100009c..2f06bff1b2cc 100644
> >> --- a/xen/arch/x86/hvm/vlapic.c
> >> +++ b/xen/arch/x86/hvm/vlapic.c
> >> @@ -1573,35 +1573,54 @@ static void lapic_load_fixup(struct vlapic *vlapic)
> >>                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
> >>  }
> >>  
> >> -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> >> +static int cf_check lapic_check_hidden(const struct domain *d,
> >> +                                       hvm_domain_context_t *h)
> >>  {
> >>      unsigned int vcpuid = hvm_load_instance(h);
> >> -    struct vcpu *v;
> >> -    struct vlapic *s;
> >> +    struct hvm_hw_lapic s;
> >>  
> >>      if ( !has_vlapic(d) )
> >>          return -ENODEV;
> >>  
> >>      /* Which vlapic to load? */
> >> -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
> >> +    if ( vcpuid >= d->max_vcpus || d->vcpu[vcpuid] == NULL )
> >>      {
> >>          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
> >>                  d->domain_id, vcpuid);
> >>          return -EINVAL;
> >>      }
> >> -    s = vcpu_vlapic(v);
> >>  
> >> -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> >> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) )
> > 
> > Can't you use hvm_get_entry() to perform the sanity checks:
> > 
> > const struct hvm_hw_lapic *s = hvm_get_entry(LAPIC, h);
> > 
> > Thanks, Roger.
> 
> I don't think I can. Because the last field (rsvd_zero) might or might
> not be there, so it needs to be zero-extended. Unless I misunderstood
> what hvm_get_entry() is meant to do. It seems to check for exact sizes.

Oh, indeed, hvm_get_entry() uses strict checking and will refuse to
return the entry if sizes don't match.  There seems to be no way to
avoid the copy if we want to do this in a sane way.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 24 12:56:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 12:56:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729528.1134717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUT3-0006K0-99; Fri, 24 May 2024 12:56:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729528.1134717; Fri, 24 May 2024 12:56:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUT3-0006Jt-5R; Fri, 24 May 2024 12:56:17 +0000
Received: by outflank-mailman (input) for mailman id 729528;
 Fri, 24 May 2024 12:56:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAUT1-0006Jn-Jc
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 12:56:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUT1-0001cc-8h; Fri, 24 May 2024 12:56:15 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUT1-00023j-1R; Fri, 24 May 2024 12:56:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=KSo6E1/ypMC6exwEzyNt1iyZhi/Mxf+YhL6fenkFbKw=; b=fN0IRUm0zeYwB8LPnynmOuoZka
	bMWL8jaIEymffoLA0b5s2kLJ+VbVfR+mPf75zPKqpLK/n8rZRYlUhRKtFdD6EtMVtQseQKoptFAol
	8Gy5jZfKpzVycOmB5sgfv9MuD6b/b+MEFNsw6Bj6MpfFhHPnwf6Z77qouZ2VErX4xLTs=;
Message-ID: <30d78526-da89-4228-9cf4-e9792ed2468d@xen.org>
Date: Fri, 24 May 2024 13:56:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 3/7] xen/p2m: put reference for level 2 superpage
Content-Language: en-GB
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
 <20240524124055.3871399-4-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524124055.3871399-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Luca,

On 24/05/2024 13:40, Luca Fancellu wrote:
> From: Penny Zheng <Penny.Zheng@arm.com>
> 
> We are doing foreign memory mapping for static shared memory, and
> there is a great possibility that it could be super mapped.
> But today, p2m_put_l3_page could not handle superpages.
> 
> This commits implements a new function p2m_put_l2_superpage to handle
> 2MB superpages, specifically for helping put extra references for
> foreign superpages.
> 
> Modify relinquish_p2m_mapping as well to take into account preemption
> when type is foreign memory and order is above 9 (2MB).
> 
> Currently 1GB superpages are not handled because Xen is not preemptible
> and therefore some work is needed to handle such superpages, for which
> at some point Xen might end up freeing memory and therefore for such a
> big mapping it could end up in a very long operation.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> v4 changes:
>   - optimised the path to call put_page() on the foreign mapping as
>     Julien suggested. Add a comment in p2m_put_l2_superpage to state
>     that any changes needs to take into account some change in the
>     relinquish code. (Julien)
> v3 changes:
>   - Add reasoning why we don't support now 1GB superpage, remove level_order
>     variable from p2m_put_l2_superpage, update TODO comment inside
>     p2m_free_entry, use XEN_PT_LEVEL_ORDER(2) instead of value 9 inside
>     relinquish_p2m_mapping. (Michal)
> v2:
>   - Do not handle 1GB super page as there might be some issue where
>     a lot of calls to put_page(...) might be issued which could lead
>     to free memory that is a long operation.
> v1:
>   - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
> ---
>   xen/arch/arm/mmu/p2m.c | 82 +++++++++++++++++++++++++++++++-----------
>   1 file changed, 62 insertions(+), 20 deletions(-)
> 
> diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
> index 41fcca011cf4..986c5a03c54b 100644
> --- a/xen/arch/arm/mmu/p2m.c
> +++ b/xen/arch/arm/mmu/p2m.c
> @@ -753,34 +753,66 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
>       return rc;
>   }
>   
> -/*
> - * Put any references on the single 4K page referenced by pte.
> - * TODO: Handle superpages, for now we only take special references for leaf
> - * pages (specifically foreign ones, which can't be super mapped today).
> - */
> -static void p2m_put_l3_page(const lpae_t pte)
> +static void p2m_put_foreign_page(struct page_info *pg)
>   {
> -    mfn_t mfn = lpae_get_mfn(pte);
> -
> -    ASSERT(p2m_is_valid(pte));
> -
>       /*
> -     * TODO: Handle other p2m types
> -     *
>        * It's safe to do the put_page here because page_alloc will
>        * flush the TLBs if the page is reallocated before the end of
>        * this loop.
>        */
> -    if ( p2m_is_foreign(pte.p2m.type) )
> +    put_page(pg);
> +}
> +
> +/* Put any references on the single 4K page referenced by mfn. */
> +static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
> +{
> +    /* TODO: Handle other p2m types */
> +    if ( p2m_is_foreign(type) )
>       {
>           ASSERT(mfn_valid(mfn));
> -        put_page(mfn_to_page(mfn));
> +        p2m_put_foreign_page(mfn_to_page(mfn));
>       }
>       /* Detect the xenheap page and mark the stored GFN as invalid. */
> -    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
> +    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
>           page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
>   }
>   
> +/* Put any references on the superpage referenced by mfn. */
> +static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
> +{
> +    struct page_info *pg;
> +    unsigned int i;
> +
> +    /*
> +     * TODO: Handle other p2m types, but be aware that any changes to handle
> +     * different types should require an update on the relinquish code to handle
> +     * preemption.
> +     */
> +    if ( !p2m_is_foreign(type) )
> +        return;
> +
> +    ASSERT(mfn_valid(mfn));
> +
> +    pg = mfn_to_page(mfn);
> +
> +    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++, pg++ )
> +        p2m_put_foreign_page(pg);
> +}
> +
> +/* Put any references on the page referenced by pte. */
> +static void p2m_put_page(const lpae_t pte, unsigned int level)
> +{
> +    mfn_t mfn = lpae_get_mfn(pte);
> +
> +    ASSERT(p2m_is_valid(pte));
> +
> +    /* We have a second level 2M superpage */
> +    if ( p2m_is_superpage(pte, level) && (level == 2) )

AFAICT, p2m_put_page() can only be called if the pte points to a 
superpage or page:

     if ( p2m_is_superpage(entry, level) || (level == 3) )
     {
        ...
        p2m_put_page()

     }

So do we actually need to check p2m_is_superpage()?

> +        return p2m_put_l2_superpage(mfn, pte.p2m.type);
> +    else if ( level == 3 )
> +        return p2m_put_l3_page(mfn, pte.p2m.type);
> +}
> +
>   /* Free lpae sub-tree behind an entry */
>   static void p2m_free_entry(struct p2m_domain *p2m,
>                              lpae_t entry, unsigned int level)
> @@ -809,9 +841,16 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>   #endif
>   
>           p2m->stats.mappings[level]--;
> -        /* Nothing to do if the entry is a super-page. */
> -        if ( level == 3 )
> -            p2m_put_l3_page(entry);
> +        /*
> +         * TODO: Currently we don't handle 1GB super-page, Xen is not
> +         * preemptible and therefore some work is needed to handle such
> +         * superpages, for which at some point Xen might end up freeing memory
> +         * and therefore for such a big mapping it could end up in a very long
> +         * operation.
> +         */
> +        if ( level >= 2 )

The code in p2m_put_page() can properly handle level 1. So I would 
rather prefer if we remove this check (again in order to reduce the 
amount of work in the P2M code).

This TODO could be moved there.

> +            p2m_put_page(entry, level);
> +
>           return;
>       }
>   
> @@ -1558,9 +1597,12 @@ int relinquish_p2m_mapping(struct domain *d)
>   
>           count++;
>           /*
> -         * Arbitrarily preempt every 512 iterations.
> +         * Arbitrarily preempt every 512 iterations or when type is foreign
> +         * mapping and the order is above 9 (2MB).

Let's avoid mentioning 2MB. This is only valid when using 4KB pages.

This is also talking about a specific when the code below is not 4KB 
specific. I would rework anything after the 'or' to:

'we have a level-2 foreign mapping'

The rest of the LGTM to me but I don't feel confortable to do all those 
changes on commit. So this will want a respin.

For simplicity, you can only resend this patch.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:01:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:01:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729532.1134726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUXX-0008DC-MY; Fri, 24 May 2024 13:00:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729532.1134726; Fri, 24 May 2024 13:00:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUXX-0008D5-Jm; Fri, 24 May 2024 13:00:55 +0000
Received: by outflank-mailman (input) for mailman id 729532;
 Fri, 24 May 2024 13:00:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAUXV-0008Cv-SH; Fri, 24 May 2024 13:00:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAUXV-0001jg-Q6; Fri, 24 May 2024 13:00:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAUXV-0002R3-Cr; Fri, 24 May 2024 13:00:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAUXV-0000l8-CM; Fri, 24 May 2024 13:00:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ytsFVcH0Dn+0zVHJ1XPOwEh6Llrd8oUZG3KtWjWbk5g=; b=Zv9oZBM7kkYa8awuYwCrZKANNq
	hpKr6ycYkt4+B3YUEEoJTYxTDgm7Qh417pBWyeRoTWhD/QZrGUyJ5c5OU0O11y/lR9lv48PR+zRl8
	GicPkxwXYYsaGdCwyx3Ld4uL5lhrRBLv/bowsYrGewVQA3DPSSdkDHBfeX8zUfXXQjNU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186129-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186129: regressions - FAIL
X-Osstest-Failures:
    xen-4.17-testing:build-arm64-pvops:kernel-build:fail:regression
    xen-4.17-testing:test-armhf-armhf-xl-credit1:host-ping-check-xen:fail:heisenbug
    xen-4.17-testing:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-4.17-testing:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 13:00:53 +0000

flight 186129 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186129/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvops             6 kernel-build   fail in 186109 REGR. vs. 185864

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit1 10 host-ping-check-xen fail in 186109 pass in 186129
 test-armhf-armhf-xl           8 xen-boot         fail in 186109 pass in 186129
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186109

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-xl           1 build-check(1)           blocked in 186109 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)           blocked in 186109 n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185864
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   25 days
Failing since        186063  2024-05-21 10:06:36 Z    3 days    5 attempts
Testing same since   186069  2024-05-22 01:58:18 Z    2 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 438 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:17:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:17:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729546.1134737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUnL-0002Tk-58; Fri, 24 May 2024 13:17:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729546.1134737; Fri, 24 May 2024 13:17:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUnL-0002Td-2A; Fri, 24 May 2024 13:17:15 +0000
Received: by outflank-mailman (input) for mailman id 729546;
 Fri, 24 May 2024 13:17:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAUnK-0002TX-E8
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:17:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUnK-00020p-0T; Fri, 24 May 2024 13:17:14 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUnJ-0003Hk-O7; Fri, 24 May 2024 13:17:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=PZG83prhR7sl336lFyk5R5T1yEEiU5iJ5MdzhJGFSsM=; b=IqOb9/vDFzmHmhY6ZqoNlcXxgq
	PjGBlW78AojAo880NJamZdrfbRMSD9Ayl4bsXxJkhsj4cmK+fjuBJ/37UHpgZAVURdY803SspkGrl
	BJIcgvZLGWssNqsSaaTiT1PqGJa1IuUxSAXSv7j4xzgc7JcBlPYdxG3ljA5xUHa1/y8s=;
Message-ID: <2842184e-98ab-4c4c-922d-05972ef88a21@xen.org>
Date: Fri, 24 May 2024 14:17:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v15 4/5] xen/arm: translate virtual PCI bus topology for
 guests
Content-Language: en-GB
To: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240517170619.45088-1-stewart.hildebrand@amd.com>
 <20240517170619.45088-5-stewart.hildebrand@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240517170619.45088-5-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stewart,

On 17/05/2024 18:06, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> There are three  originators for the PCI configuration space access:
> 1. The domain that owns physical host bridge: MMIO handlers are
> there so we can update vPCI register handlers with the values
> written by the hardware domain, e.g. physical view of the registers
> vs guest's view on the configuration space.
> 2. Guest access to the passed through PCI devices: we need to properly
> map virtual bus topology to the physical one, e.g. pass the configuration
> space access to the corresponding physical devices.
> 3. Emulated host PCI bridge access. It doesn't exist in the physical
> topology, e.g. it can't be mapped to some physical host bridge.
> So, all access to the host bridge itself needs to be trapped and
> emulated.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

With one remark below, for Arm:

Acked-by: Julien Grall <jgrall@amazon.com>

[...]

> diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
> index 23722634d50b..98b294f09688 100644
> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -81,6 +81,30 @@ static int add_virtual_device(struct pci_dev *pdev)
>       return 0;
>   }
>   
> +/*
> + * Find the physical device which is mapped to the virtual device
> + * and translate virtual SBDF to the physical one.
> + */
> +bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
> +{
> +    const struct pci_dev *pdev;
> +
> +    ASSERT(!is_hardware_domain(d));
> +    ASSERT(rw_is_locked(&d->pci_lock));
> +
> +    for_each_pdev ( d, pdev )

(This doesn't need to be addressed now)


I see that we have other places with for_each_pdev() in the vCPI. So are 
we expecting the list to be smallish?

If not, then we may want to consider reworking the datastructure or put 
a limit on the number of PCI devices assigned.


> +    {
> +        if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) )
> +        {
> +            /* Replace guest SBDF with the physical one. */
> +            *sbdf = pdev->sbdf;
> +            return true;
> +        }
> +    }
> +
> +    return false;
> +}
> +
>   #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
>   
>   void vpci_deassign_device(struct pci_dev *pdev)
> diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
> index 980aded26fc1..7e5a0f0c50c1 100644
> --- a/xen/include/xen/vpci.h
> +++ b/xen/include/xen/vpci.h
> @@ -303,6 +303,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v)
>   }
>   #endif
>   
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
> +#else
> +static inline bool vpci_translate_virtual_device(const struct domain *d,
> +                                                 pci_sbdf_t *sbdf)
> +{
> +    ASSERT_UNREACHABLE();
> +
> +    return false;
> +}
> +#endif
> +
>   #endif
>   
>   /*

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:21:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:21:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729551.1134746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUrA-00047u-J8; Fri, 24 May 2024 13:21:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729551.1134746; Fri, 24 May 2024 13:21:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUrA-00047n-Gc; Fri, 24 May 2024 13:21:12 +0000
Received: by outflank-mailman (input) for mailman id 729551;
 Fri, 24 May 2024 13:21:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAUr9-00047h-AY
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:21:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUr8-00025R-Vv; Fri, 24 May 2024 13:21:10 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUr8-0003P6-R7; Fri, 24 May 2024 13:21:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YtU9PpJWiO5LENmtkdVEs1npibNUMkg2mhdyNGLvFD8=; b=s0MMSNPjJ9ru2V4tgKlDWC3x5G
	L9KfZveGQWagPmhzc/IwspJCRpYjbtUzl/+qP2H1t5PRroCgwEhOtHnU5ViS9i1R1Su9L2tjQwDjF
	eYdxygPml/iZgKqbxGjKeeIW0ybFBx6T3ZVPJ5AoO9AvUDAOQBT53NkkjNoFtNjNgIJY=;
Message-ID: <4afe33f2-72e6-4755-98ce-d7f9da374e90@xen.org>
Date: Fri, 24 May 2024 14:21:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v16 4/5] xen/arm: translate virtual PCI bus topology for
 guests
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-5-stewart.hildebrand@amd.com>
 <Zk70udmiriruMt0r@macbook>
From: Julien Grall <julien@xen.org>
In-Reply-To: <Zk70udmiriruMt0r@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

Sorry I didn't notice there was a v16 and posted comments on the v15. 
The only one is about the size of the list we iterate.

On 23/05/2024 08:48, Roger Pau Monné wrote:
> On Wed, May 22, 2024 at 06:59:23PM -0400, Stewart Hildebrand wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>
>> There are three  originators for the PCI configuration space access:
>> 1. The domain that owns physical host bridge: MMIO handlers are
>> there so we can update vPCI register handlers with the values
>> written by the hardware domain, e.g. physical view of the registers
>> vs guest's view on the configuration space.
>> 2. Guest access to the passed through PCI devices: we need to properly
>> map virtual bus topology to the physical one, e.g. pass the configuration
>> space access to the corresponding physical devices.
>> 3. Emulated host PCI bridge access. It doesn't exist in the physical
>> topology, e.g. it can't be mapped to some physical host bridge.
>> So, all access to the host bridge itself needs to be trapped and
>> emulated.
>>
>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> 
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

For Arm:

Acked-by: Julien Grall <jgrall@amazon.com>

> 
> One unrelated question below.
> 
>> ---
>> In v15:
>> - base on top of ("arm/vpci: honor access size when returning an error")
>> In v11:
>> - Fixed format issues
>> - Added ASSERT_UNREACHABLE() to the dummy implementation of
>> vpci_translate_virtual_device()
>> - Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
>> the logic in the function
>> Since v9:
>> - Commend about required lock replaced with ASSERT()
>> - Style fixes
>> - call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
>> Since v8:
>> - locks moved out of vpci_translate_virtual_device()
>> Since v6:
>> - add pcidevs locking to vpci_translate_virtual_device
>> - update wrt to the new locking scheme
>> Since v5:
>> - add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
>>    case to simplify ifdefery
>> - add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
>> - reset output register on failed virtual SBDF translation
>> Since v4:
>> - indentation fixes
>> - constify struct domain
>> - updated commit message
>> - updates to the new locking scheme (pdev->vpci_lock)
>> Since v3:
>> - revisit locking
>> - move code to vpci.c
>> Since v2:
>>   - pass struct domain instead of struct vcpu
>>   - constify arguments where possible
>>   - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
>> New in v2
>> ---
>>   xen/arch/arm/vpci.c     | 45 ++++++++++++++++++++++++++++++++---------
>>   xen/drivers/vpci/vpci.c | 24 ++++++++++++++++++++++
>>   xen/include/xen/vpci.h  | 12 +++++++++++
>>   3 files changed, 71 insertions(+), 10 deletions(-)
>>
>> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
>> index b63a356bb4a8..516933bebfb3 100644
>> --- a/xen/arch/arm/vpci.c
>> +++ b/xen/arch/arm/vpci.c
>> @@ -7,33 +7,53 @@
>>   
>>   #include <asm/mmio.h>
>>   
>> -static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
>> -                                     paddr_t gpa)
>> +static bool vpci_sbdf_from_gpa(struct domain *d,
>> +                               const struct pci_host_bridge *bridge,
>> +                               paddr_t gpa, pci_sbdf_t *sbdf)
>>   {
>> -    pci_sbdf_t sbdf;
>> +    bool translated = true;
>> +
>> +    ASSERT(sbdf);
>>   
>>       if ( bridge )
>>       {
>> -        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
>> -        sbdf.seg = bridge->segment;
>> -        sbdf.bus += bridge->cfg->busn_start;
>> +        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
>> +        sbdf->seg = bridge->segment;
>> +        sbdf->bus += bridge->cfg->busn_start;
>>       }
>>       else
>> -        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
>> +    {
>> +        /*
>> +         * For the passed through devices we need to map their virtual SBDF
>> +         * to the physical PCI device being passed through.
>> +         */
>> +        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
>> +        read_lock(&d->pci_lock);
>> +        translated = vpci_translate_virtual_device(d, sbdf);
>> +        read_unlock(&d->pci_lock);
> 
> I would consider moving the read_{,un}lock() calls inside
> vpci_translate_virtual_device(), if that's the only caller of
> vpci_translate_virtual_device().  Maybe further patches add other
> instances that call from an already locked context.
> 
>> +    }
>>   
>> -    return sbdf;
>> +    return translated;
>>   }
>>   
>>   static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>>                             register_t *r, void *p)
>>   {
>>       struct pci_host_bridge *bridge = p;
>> -    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
>> +    pci_sbdf_t sbdf;
>>       const unsigned int access_size = (1U << info->dabt.size) * 8;
>>       const register_t invalid = GENMASK_ULL(access_size - 1, 0);
> 
> Do you know why the invalid value is truncated to the access size.

Because no other callers are doing the truncation and therefore the 
guest would read 1s even for 8-byte unsigned access.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:26:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:26:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729563.1134757 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUwG-0004x2-4e; Fri, 24 May 2024 13:26:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729563.1134757; Fri, 24 May 2024 13:26:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUwG-0004wv-1u; Fri, 24 May 2024 13:26:28 +0000
Received: by outflank-mailman (input) for mailman id 729563;
 Fri, 24 May 2024 13:26:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAUwE-0004wp-Q4
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:26:26 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUwE-0002Ax-FS; Fri, 24 May 2024 13:26:26 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAUwE-0003ns-8B; Fri, 24 May 2024 13:26:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=jac7bR+7bZ91CBiafCoUuw2taOp9HF8ytcW5DOkX+yY=; b=ckH/Xw9AD01MtB0OqeppC4ttLp
	ve4moZr9puFECrb18h0qujPo3CO71yQjuaZBuRClUBIGtM0eCjK/Y4oaCbHoxXKd6g8pvZ/nQtbAr
	Uf/My2rOL5Zum2XKQRYAOlJ0CxnCA5Andxm2O3Frd1bnct4SQKLUTubo0nsiuAavmbYI=;
Message-ID: <69f9c0b7-b3f3-49bb-9f9a-42c6ef6d0edc@xen.org>
Date: Fri, 24 May 2024 14:26:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] xen/arm: Set correct per-cpu cpu_core_mask
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, Henry Wang <xin.wang2@amd.com>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>
References: <20240321035706.165253-1-xin.wang2@amd.com>
 <9deeb66d-1a69-442e-a860-08006067ff44@amd.com>
 <e724dd97-03fc-4136-9a53-94bbddce160f@amd.com>
 <e81356e7-fcf2-4fdb-8585-e544ac4a15df@amd.com>
 <e9e188ef-6de5-4cc7-8edd-dd7d004670de@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <e9e188ef-6de5-4cc7-8edd-dd7d004670de@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 21/05/2024 08:57, Michal Orzel wrote:
> 
> 
> On 21/05/2024 09:51, Henry Wang wrote:
>> Hi Michal,
>>
>> On 5/21/2024 3:47 PM, Michal Orzel wrote:
>>> Hi Henry.
>>>
>>> On 3/21/2024 11:57 AM, Henry Wang wrote:
>>>>> In the common sysctl command XEN_SYSCTL_physinfo, the value of
>>>>> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
>>>>> Currently on Arm this is a fixed value 1 (can be checked via xl info),
>>>>> which is not correct. This is because during the Arm CPU online
>>>>> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
>>>>> cpu_core_mask for itself.
>>>>>
>>>>> cores_per_socket refers to the number of cores that belong to the same
>>>>> socket (NUMA node). Currently Xen on Arm does not support physical
>>>>> CPU hotplug and NUMA, also we assume there is no multithread. Therefore
>>>>> cores_per_socket means all possible CPUs detected from the device
>>>>> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
>>>>> accordingly. Modify the in-code comment which seems to be outdated. Add
>>>>> a warning to users if Xen is running on processors with multithread
>>>>> support.
>>>>>
>>>>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>>>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>>> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
>>
>> Thanks.
>>
>>>>>     /* ID of the PCPU we're running on */
>>>>>     DEFINE_PER_CPU(unsigned int, cpu_id);
>>>>> -/* XXX these seem awfully x86ish... */
>>>>> +/*
>>>>> + * Although multithread is part of the Arm spec, there are not many
>>>>> + * processors support multithread and current Xen on Arm assumes there
>>> NIT: s/support/supporting
>>
>> Sorry, it should have been spotted locally before sending. Anyway, I
>> will correct this in v4 with your Reviewed-by tag taken. Thanks for
>> pointing this out.
> I don't think there is a need to resend a patch just for fixing this typo. It can be done on commit.

Fixed and committed.

Cheers,

> 
> ~Michal
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:29:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:29:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729567.1134767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUyh-0005Wj-GE; Fri, 24 May 2024 13:28:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729567.1134767; Fri, 24 May 2024 13:28:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAUyh-0005Wc-DY; Fri, 24 May 2024 13:28:59 +0000
Received: by outflank-mailman (input) for mailman id 729567;
 Fri, 24 May 2024 13:28:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nTfd=M3=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sAUyf-0005WT-LY
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:28:57 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 91b57b92-19d1-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 15:28:55 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so557611066b.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 06:28:55 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9376edsm132712666b.59.2024.05.24.06.28.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 06:28:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91b57b92-19d1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716557335; x=1717162135; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3NcilalvADXfBySrQYa0n8hNSTrv08kwAHmEPErC5Dg=;
        b=JG7xY9EVOweJPQMsRSF5nw3g6/AhN+KrCvRvbVhfm3cuvytCnnT6naxyeIwEgIvr0q
         g0Twucanp5F5LWI3k/c8lVdfOnYdf87ZGXBIiwDMmbCWPPNUpICQ7d7PNj9LYK+SbSka
         W+eiZClqa/mO6E0OISuVhSZicpDZh5ts+yiNxJl0y2EXidolraRLUtZI7G4MrKj90g9g
         MuWDkVUdzfA4IIgOta2jMwa+tZTP53bLf8zmnImtueLpAA92WXGG1Y3rl4rtJdv/FqEV
         vSQi02hQ8JbD5ZNwFlN5rJJsAAJcd4LMsZ8YHdLLQp55ULAUUZU4rz1ExZvquQlSNOKD
         QeqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716557335; x=1717162135;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3NcilalvADXfBySrQYa0n8hNSTrv08kwAHmEPErC5Dg=;
        b=mlV5/W1WYnwbzsIodQSkFBLV7W54jFVjSACjJ9wOsQeVD6Hm7TyEFo+hiv790G/tTo
         AyjOuVTCumhslNwsk1cWo6eeee7LOPhLiuiaoBJtggm4iD7Mv5eq7ew8eUx5CSEEirfo
         lgogerul5ug3njIKRKB5MRczeslxe7lkx6AG9VhglGiRR4YKIJAExHASBHPSjhzoNR21
         svo+hK/qluqIG2B0L1c8Dc5QOyh6LOjAhVmY4EcEN/NNkGxv6KPtrnS1bTNirY6tfbw7
         DKHDQMhdRA007NYiZVa9Km8r96/U8+VZOQFPgq2n8RXJAyNYZ88Cc0Ve0V8aGYSB+IjG
         VxIQ==
X-Forwarded-Encrypted: i=1; AJvYcCX9GZYwH/c0KiBcpJpLUsHzBXoKB0lZZUjkXdXPsgkrEPdL5FVsrR/YXUKuOYvraMBxQ2ySxt5rNZKi4oMLHZEv1Hy4NjNJG2hV8NjqqiQ=
X-Gm-Message-State: AOJu0Yz3DPrDovNea1MK4bkcPqe6/rVxroYD3wXzQImN6nZudWvDmTj5
	kLdXisQ1bYCwnlQGrnUZy4eYlL9jLY7RBBsaBJyfK7CebRi0Z9mLP3+lxs1LI5g=
X-Google-Smtp-Source: AGHT+IFtDuHED1hdSpxXhxPqDbPnJHfqtWqQV3lWCA5XYYdtu1Yb0HgbFEJ/SAbPwr9HtzulYJKkgQ==
X-Received: by 2002:a17:906:275a:b0:a5a:3e00:6317 with SMTP id a640c23a62f3a-a623ea0380bmr413697766b.31.1716557335063;
        Fri, 24 May 2024 06:28:55 -0700 (PDT)
Message-ID: <6cc5222a-1dfb-434d-8d84-e1f02be38511@suse.com>
Date: Fri, 24 May 2024 15:28:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 2/3] xen: enable altp2m at create domain
 domctl
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <20240517133352.94347-3-roger.pau@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240517133352.94347-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 17.05.24 15:33, Roger Pau Monne wrote:
> Enabling it using an HVM param is fragile, and complicates the logic when
> deciding whether options that interact with altp2m can also be enabled.
> 
> Leave the HVM param value for consumption by the guest, but prevent it from
> being set.  Enabling is now done using and additional altp2m specific field in
> xen_domctl_createdomain.
> 
> Note that albeit only currently implemented in x86, altp2m could be implemented
> in other architectures, hence why the field is added to xen_domctl_createdomain
> instead of xen_arch_domainconfig.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Juergen Gross <jgross@suse.com> # tools/libs/


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:37:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:37:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729576.1134781 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAV78-0007bp-By; Fri, 24 May 2024 13:37:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729576.1134781; Fri, 24 May 2024 13:37:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAV78-0007bi-8q; Fri, 24 May 2024 13:37:42 +0000
Received: by outflank-mailman (input) for mailman id 729576;
 Fri, 24 May 2024 13:37:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nTfd=M3=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sAV77-0007bc-17
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:37:41 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c95effc4-19d2-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 15:37:38 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 18B6E33B2B;
 Fri, 24 May 2024 13:37:38 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A616313A3D;
 Fri, 24 May 2024 13:37:36 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id wr10GiCYUGbAMAAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 24 May 2024 13:37:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c95effc4-19d2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716557858; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=JKHA+2p18tBMJqJgM8JceeW8T9TO/bDEUGloQ6VEz/o=;
	b=qy9drsJUa5/BPf+w/P64RE3ruc3tnU95RQyGeW1R9ZlSldD9XjEEq2sjg2K/d1+7QpWChl
	KokgJFTHZr5Ml2tJ8s1R4b/RwfJDtqakOR0hejgPG4URnIl1F7EToQCOvT8CCr/lmD6kNf
	t5+nawkWosh5PgIj4fCVhH88h7vs/xg=
Authentication-Results: smtp-out1.suse.de;
	none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716557858; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=JKHA+2p18tBMJqJgM8JceeW8T9TO/bDEUGloQ6VEz/o=;
	b=qy9drsJUa5/BPf+w/P64RE3ruc3tnU95RQyGeW1R9ZlSldD9XjEEq2sjg2K/d1+7QpWChl
	KokgJFTHZr5Ml2tJ8s1R4b/RwfJDtqakOR0hejgPG4URnIl1F7EToQCOvT8CCr/lmD6kNf
	t5+nawkWosh5PgIj4fCVhH88h7vs/xg=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	sstabellini@kernel.org
Subject: [GIT PULL] xen: branch for v6.10-rc1
Date: Fri, 24 May 2024 15:37:33 +0200
Message-Id: <20240524133733.27825-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Result: default: False [-2.15 / 50.00];
	BAYES_HAM(-2.35)[96.98%];
	MID_CONTAINS_FROM(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_MISSING_CHARSET(0.50)[];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.10)[text/plain];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	MIME_TRACE(0.00)[0:+];
	ARC_NA(0.00)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo];
	FROM_EQ_ENVFROM(0.00)[];
	FROM_HAS_DN(0.00)[];
	RCPT_COUNT_THREE(0.00)[4];
	RCVD_COUNT_TWO(0.00)[2];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	TO_DN_NONE(0.00)[];
	RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -2.15
X-Spam-Flag: NO

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.10a-rc1-tag

xen: branch for v6.10-rc1

It contains the following patches:

- a small cleanup in the drivers/xen/xenbus Makefile
- a fix of the Xen xenstore driver to improve connecting to a late started
  Xenstore
- an enhancement for better support of ballooning in PVH guests
- a cleanup using try_cmpxchg() instead of open coding it

Thanks.

Juergen

 arch/x86/xen/enlighten.c          | 33 +++++++++++++++++++++++++++++++++
 arch/x86/xen/p2m.c                | 11 +++++------
 drivers/xen/xenbus/Makefile       | 14 ++++++--------
 drivers/xen/xenbus/xenbus_probe.c | 36 +++++++++++++++++++++++-------------
 4 files changed, 67 insertions(+), 27 deletions(-)

Andy Shevchenko (1):
      xen/xenbus: Use *-y instead of *-objs in Makefile

Henry Wang (1):
      drivers/xen: Improve the late XenStore init protocol

Roger Pau Monne (1):
      xen/x86: add extra pages to unpopulated-alloc if available

Uros Bizjak (1):
      locking/x86/xen: Use try_cmpxchg() in xen_alloc_p2m_entry()


From xen-devel-bounces@lists.xenproject.org Fri May 24 13:59:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 13:59:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729581.1134790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAVRl-0002e9-0P; Fri, 24 May 2024 13:59:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729581.1134790; Fri, 24 May 2024 13:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAVRk-0002e2-Tx; Fri, 24 May 2024 13:59:00 +0000
Received: by outflank-mailman (input) for mailman id 729581;
 Fri, 24 May 2024 13:58:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAVRj-0002dw-Ha
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 13:58:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAVRi-0002jF-Uz; Fri, 24 May 2024 13:58:58 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAVRi-0005bX-NH; Fri, 24 May 2024 13:58:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YfsqmkJ1Z95qqct4pIoLj7B7xXCx9BrsGe475nzDNW4=; b=bQX33TxQ3dqqhYSfksyjGNE/bC
	2ZSOBWOTTnhPPaaHgBBBzm5PUkA5aZaOXbin8LRs3fdp7W4sismZyXQrGI/OKtarGSaf7GsMMwb+9
	8pmofRZBwUzEcrtrcW1EyQ+STcVmK8srs+kxejbZvoAvCMITUOZVZiRoCe6vy/sOsDQo=;
Message-ID: <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
Date: Fri, 24 May 2024 14:58:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Juergen Gross <jgross@suse.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240517032156.1490515-3-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

+ Juergen as the Xenstore maintainers. I'd like his opinion on the 
approach. The documentation of the new logic is in:

https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/

FWIW I am happy in principle with the logic (this is what we discussed 
on the call last week). Some comments below.

On 17/05/2024 04:21, Henry Wang wrote:
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> The "magic page" is a terminology used in the toolstack as reserved
> pages for the VM to have access to virtual platform capabilities.
> Currently the magic pages for Dom0less DomUs are populated by the
> init-dom0less app through populate_physmap(), and populate_physmap()
> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> cannot be true for the magic pages that are allocated later from the
> init-dom0less application executed in Dom0. For domain using statically
> allocated memory but not 1:1 direct-mapped, similar error "failed to
> retrieve a reserved page" can be seen as the reserved memory list is
> empty at that time.
> 
> Since for init-dom0less, the magic page region is only for XenStore.
> To solve above issue, this commit allocates the XenStore page for
> Dom0less DomUs at the domain construction time. The PFN will be
> noted and communicated to the init-dom0less application executed
> from Dom0. To keep the XenStore late init protocol, set the connection
> status to XENSTORE_RECONNECT.

So this commit is allocating the page, but it will not be used by 
init-dom0less until the next patch. But Linux could use it. So would 
this break bisection? If so, then I think patch #3 needs to be folded in 
this patch.

> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
> v3:
> - Only allocate XenStore page. (Julien)
> - Set HVM_PARAM_STORE_PFN and the XenStore connection status directly
>    from hypervisor. (Stefano)
> v2:
> - Reword the commit msg to explain what is "magic page" and use generic
>    terminology "hypervisor reserved pages" in commit msg. (Daniel)
> - Also move the offset definition of magic pages. (Michal)
> - Extract the magic page allocation logic to a function. (Michal)
> ---
>   xen/arch/arm/dom0less-build.c | 44 ++++++++++++++++++++++++++++++++++-
>   1 file changed, 43 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..95c4fd1a2d 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -1,5 +1,6 @@
>   /* SPDX-License-Identifier: GPL-2.0-only */
>   #include <xen/device_tree.h>
> +#include <xen/domain_page.h>
>   #include <xen/err.h>
>   #include <xen/event.h>
>   #include <xen/grant_table.h>
> @@ -10,6 +11,8 @@
>   #include <xen/sizes.h>
>   #include <xen/vmap.h>
>   
> +#include <public/io/xs_wire.h>
> +
>   #include <asm/arm64/sve.h>
>   #include <asm/dom0less-build.h>
>   #include <asm/domain_build.h>
> @@ -739,6 +742,42 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
>       return 0;
>   }
>   
> +#define XENSTORE_PFN_OFFSET 1
> +static int __init alloc_xenstore_page(struct domain *d)
> +{
> +    struct page_info *xenstore_pg;
> +    struct xenstore_domain_interface *interface;
> +    mfn_t mfn;
> +    gfn_t gfn;
> +    int rc;
> +
> +    d->max_pages += 1;

Sorry I should have spotted it earlier. But you want to check 
d->max_pages is not overflowing. You can look at 
acquire_shared_memory_bank() for how to do it.

Also, maybe we want an helper to do it so it is not open-coded in 
multiple places.

> +    xenstore_pg = alloc_domheap_page(d, 0);

I think we may want to restrict where the page is allocated. For 
instance, 32-bit domain using short page-tables will not be able to 
address all the physical memory.

I would consider to try to allocate the page below 32-bit (using 
MEMF-bits(32). And then fallback to above 32-bit only 64-bit for domain.

Also, just to note that in theory alloc_domheap_page() could return MFN 
0. In practice we have excluded MFN 0 because it breaks the page 
allocator so far.

But I would still prefer if we add a check on the MFN. This will make 
easier to spot any issue if we ever give MFN 0 to the allocator.

A possible implementation would be to call alloc_domphea_page() a second 
time and then free the first one (e.g. MFN 0).

> +    if ( xenstore_pg == NULL )
> +        return -ENOMEM;
> +
> +    mfn = page_to_mfn(xenstore_pg);
> +    if ( !is_domain_direct_mapped(d) )
> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE +
> +                           (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
> +    else
> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
> +
> +    rc = guest_physmap_add_page(d, gfn, mfn, 0); > +    if ( rc )
> +    {
> +        free_domheap_page(xenstore_pg);
> +        return rc;
> +    }
> +
> +    d->arch.hvm.params[HVM_PARAM_STORE_PFN] = gfn_x(gfn);
> +    interface = (struct xenstore_domain_interface *)map_domain_page(mfn);

map_domain_page() will return a void *. So why do you need to an 
explicit case?

> +    interface->connection = XENSTORE_RECONNECT;
> +    unmap_domain_page(interface);
> +
> +    return 0;
> +}
> +
>   static int __init construct_domU(struct domain *d,
>                                    const struct dt_device_node *node)
>   {
> @@ -839,7 +878,10 @@ static int __init construct_domU(struct domain *d,
>           rc = alloc_xenstore_evtchn(d);
>           if ( rc < 0 )
>               return rc;
> -        d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
> +
> +        rc = alloc_xenstore_page(d);
> +        if ( rc < 0 )
> +            return rc;
>       }
>   
>       return rc;

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 14:30:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 14:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729595.1134801 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAVwS-0000Jf-Bx; Fri, 24 May 2024 14:30:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729595.1134801; Fri, 24 May 2024 14:30:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAVwS-0000JY-9J; Fri, 24 May 2024 14:30:44 +0000
Received: by outflank-mailman (input) for mailman id 729595;
 Fri, 24 May 2024 14:30:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nTfd=M3=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sAVwR-0000JK-7i
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 14:30:43 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 328e77d1-19da-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 16:30:41 +0200 (CEST)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2e576057c56so88332621fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 07:30:41 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578524ba2ecsm1814048a12.89.2024.05.24.07.30.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 07:30:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 328e77d1-19da-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716561041; x=1717165841; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UskfpYP3fNcFRRBO5RaGcacGbH/+MAmEZ1kZgYhopzY=;
        b=KJ/FsZq42hctno7kMg+H6Ft6hgZD6Mqktqal+BwdZGxoO0En4snQGKSCf3wXFHjWdU
         tQgdM34UMdg6Ci1dYxRsQ/gT+cVlkB+Ap3MVjqoyX6Pmvc9zM5AVEyN6cd+FEHv4G3og
         xkrys5G4lvxFQ6trCnoujDFYB5zwtoTzgd5IolW4RHw4PRuOD6w8/NIsqOlS8Ebg5eRe
         FkHB43L0SpTruUOCZsuYUQBsQJnS6dCEI2/zaQZH/v+35BwtxqEU0FUf/+D1NVSw77Cj
         0tfpTdG8Jvgq8zRrCSY9XqtaQgNWrvpsK940l/Y2FTxHzaEwFlbrLSQ78fq/0grmyIzG
         L4sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716561041; x=1717165841;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UskfpYP3fNcFRRBO5RaGcacGbH/+MAmEZ1kZgYhopzY=;
        b=iuo+EdWFmlQpkcijVGnbPinCXstLZsETjKbe+lkoYwo8OR4GouONATDY/HfWIM7jyS
         AEkFE3Pg5FhXTHQLmafsgJciQQj151mK36rVDoxQP8oh3/ZxQOw3xoVNH4mtvzUK4XBz
         AMD0YG4KNXgYSYcdH6tUvvxqv6EGFCkm7VPedu3XAo+IaxRYuhAY63tbG3ChbBk7e/Sq
         quYJG8AF6O5/RLTtsYuozST5erUCNEopJ/XJnIpQS81PhfBEpZOCclItF7xDZPjJpN89
         mi9SQxWshJymyKD7vur3zdXfW16m8+GESinMPU/2jbTkEqm5Bf4ZlH/mFT4qAJ/DN6ef
         iInQ==
X-Forwarded-Encrypted: i=1; AJvYcCUkoDIzkb3kpyPUZMBPy8l2gz6r+aG5GIKXqiemZn0QcWq6YSlZltgGWrCvW4ZPJcB96x6oosY7Z8NPAKcPlmrHWZiedkO+viEIrNqX5JY=
X-Gm-Message-State: AOJu0YwkUp/kQR5vUimS2pk1eMIcQscCq634YJe6TcqQDNeYg5V8uJxj
	a4h4yVA/AKCvZLY8rhk6U86sTuMX8eDqOdABtyHOBrImeKPaNzCbfAEACBtnFBo=
X-Google-Smtp-Source: AGHT+IEyuE87zelNLVGr34Afmmm9yCAyTVX1WQhxWWNJzDh3NIctepE6D+gV6VDQ3vrU61R+nV0mCQ==
X-Received: by 2002:a19:9143:0:b0:518:dbcc:4f8b with SMTP id 2adb3069b0e04-529644ec1demr1292112e87.12.1716561040782;
        Fri, 24 May 2024 07:30:40 -0700 (PDT)
Message-ID: <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
Date: Fri, 24 May 2024 16:30:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
To: Julien Grall <julien@xen.org>, Henry Wang <xin.wang2@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 24.05.24 15:58, Julien Grall wrote:
> Hi Henry,
> 
> + Juergen as the Xenstore maintainers. I'd like his opinion on the approach. The 
> documentation of the new logic is in:
> 
> https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
> 
> FWIW I am happy in principle with the logic (this is what we discussed on the 
> call last week). Some comments below.

I'm not against this logic, but I'm wondering why it needs to be so
complicated.

Can't the domU itself allocate the Xenstore page from its RAM pages,
write the PFN into the Xenstore grant tab entry, and then make it
public via setting HVM_PARAM_STORE_PFN?

The init-dom0less application could then check HVM_PARAM_STORE_PFN
being set and call XS_introduce_domain.

Note that at least C-xenstored does not need the PFN of the Xenstore
page, as it is just using GNTTAB_RESERVED_XENSTORE for mapping the
page.


Juergen


From xen-devel-bounces@lists.xenproject.org Fri May 24 14:41:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 14:41:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729607.1134811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAW6J-0002G5-9e; Fri, 24 May 2024 14:40:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729607.1134811; Fri, 24 May 2024 14:40:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAW6J-0002Fy-6U; Fri, 24 May 2024 14:40:55 +0000
Received: by outflank-mailman (input) for mailman id 729607;
 Fri, 24 May 2024 14:40:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nTfd=M3=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sAW6H-0002Fn-IO
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 14:40:53 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e30d040-19db-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 16:40:51 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BFB4A33B83;
 Fri, 24 May 2024 14:40:50 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 98C3113A6B;
 Fri, 24 May 2024 14:40:48 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /CMLGPCmUGbEWQAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 24 May 2024 14:40:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e30d040-19db-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716561650; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=riUbNz8090F/aCxwRYIWhw+gpOC1R1iOokj7sIQEEUg=;
	b=CfJCGk3JIUQnKJqV9f+nnbD0aAI8XmKw9gkpmWhECrUHi4ECrAYLFunUXs/UhokS2y4XX9
	oxM0+LoS4fILWwjueS5/Tlfmz9GwhadnRc5bB0U1e9ex8HQKQGzSosHBwZNj7gQA5+iPD+
	V0GfNH4dZxM5j0Rl/fo2sS05u7i6nag=
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=CfJCGk3J
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716561650; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=riUbNz8090F/aCxwRYIWhw+gpOC1R1iOokj7sIQEEUg=;
	b=CfJCGk3JIUQnKJqV9f+nnbD0aAI8XmKw9gkpmWhECrUHi4ECrAYLFunUXs/UhokS2y4XX9
	oxM0+LoS4fILWwjueS5/Tlfmz9GwhadnRc5bB0U1e9ex8HQKQGzSosHBwZNj7gQA5+iPD+
	V0GfNH4dZxM5j0Rl/fo2sS05u7i6nag=
Message-ID: <85991a0d-ba43-4701-bc1d-c63b52a05c93@suse.com>
Date: Fri, 24 May 2024 16:40:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, Henry Wang <xin.wang2@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
 <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
Content-Language: en-US
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------0JS4HoVF0CdaNsppIN81Phlu"
X-Spam-Level: 
X-Spamd-Result: default: False [-8.40 / 50.00];
	BAYES_HAM(-3.00)[99.98%];
	SIGNED_PGP(-2.00)[];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	NEURAL_HAM_SHORT(-0.20)[-1.000];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	MX_GOOD(-0.01)[];
	XM_UA_NO_VERSION(0.01)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	ARC_NA(0.00)[];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	FROM_HAS_DN(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	TO_DN_SOME(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCPT_COUNT_SEVEN(0.00)[9];
	HAS_ATTACHMENT(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	DKIM_TRACE(0.00)[suse.com:+];
	DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim]
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: BFB4A33B83
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Flag: NO
X-Spam-Score: -8.40

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------0JS4HoVF0CdaNsppIN81Phlu
Content-Type: multipart/mixed; boundary="------------g4fXSKxlImaQsBgaeHNovRnH";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, Henry Wang <xin.wang2@amd.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
Message-ID: <85991a0d-ba43-4701-bc1d-c63b52a05c93@suse.com>
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
 <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
In-Reply-To: <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>

--------------g4fXSKxlImaQsBgaeHNovRnH
Content-Type: multipart/mixed; boundary="------------ayNag0yu1eLYaC2E0XRDMQ0x"

--------------ayNag0yu1eLYaC2E0XRDMQ0x
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjQuMDUuMjQgMTY6MzAsIErDvHJnZW4gR3Jvw58gd3JvdGU6DQo+IE9uIDI0LjA1LjI0
IDE1OjU4LCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+PiBIaSBIZW5yeSwNCj4+DQo+PiArIEp1
ZXJnZW4gYXMgdGhlIFhlbnN0b3JlIG1haW50YWluZXJzLiBJJ2QgbGlrZSBoaXMgb3Bpbmlv
biBvbiB0aGUgYXBwcm9hY2guIA0KPj4gVGhlIGRvY3VtZW50YXRpb24gb2YgdGhlIG5ldyBs
b2dpYyBpcyBpbjoNCj4+DQo+PiBodHRwczovL2xvcmUua2VybmVsLm9yZy94ZW4tZGV2ZWwv
MjAyNDA1MTcwMzIxNTYuMTQ5MDUxNS01LXhpbi53YW5nMkBhbWQuY29tLw0KPj4NCj4+IEZX
SVcgSSBhbSBoYXBweSBpbiBwcmluY2lwbGUgd2l0aCB0aGUgbG9naWMgKHRoaXMgaXMgd2hh
dCB3ZSBkaXNjdXNzZWQgb24gdGhlIA0KPj4gY2FsbCBsYXN0IHdlZWspLiBTb21lIGNvbW1l
bnRzIGJlbG93Lg0KPiANCj4gSSdtIG5vdCBhZ2FpbnN0IHRoaXMgbG9naWMsIGJ1dCBJJ20g
d29uZGVyaW5nIHdoeSBpdCBuZWVkcyB0byBiZSBzbw0KPiBjb21wbGljYXRlZC4NCj4gDQo+
IENhbid0IHRoZSBkb21VIGl0c2VsZiBhbGxvY2F0ZSB0aGUgWGVuc3RvcmUgcGFnZSBmcm9t
IGl0cyBSQU0gcGFnZXMsDQo+IHdyaXRlIHRoZSBQRk4gaW50byB0aGUgWGVuc3RvcmUgZ3Jh
bnQgdGFiIGVudHJ5LCBhbmQgdGhlbiBtYWtlIGl0DQo+IHB1YmxpYyB2aWEgc2V0dGluZyBI
Vk1fUEFSQU1fU1RPUkVfUEZOPw0KPiANCj4gVGhlIGluaXQtZG9tMGxlc3MgYXBwbGljYXRp
b24gY291bGQgdGhlbiBjaGVjayBIVk1fUEFSQU1fU1RPUkVfUEZODQo+IGJlaW5nIHNldCBh
bmQgY2FsbCBYU19pbnRyb2R1Y2VfZG9tYWluLg0KPiANCj4gTm90ZSB0aGF0IGF0IGxlYXN0
IEMteGVuc3RvcmVkIGRvZXMgbm90IG5lZWQgdGhlIFBGTiBvZiB0aGUgWGVuc3RvcmUNCj4g
cGFnZSwgYXMgaXQgaXMganVzdCB1c2luZyBHTlRUQUJfUkVTRVJWRURfWEVOU1RPUkUgZm9y
IG1hcHBpbmcgdGhlDQo+IHBhZ2UuDQoNCkhtbSwgc2VlbXMgYXMgaWYgTy14ZW5zdG9yZWQg
aXMgdXNpbmcgdGhlIG1hcF9mb3JlaWduX3BhZ2UgbWVjaGFuaXNtIHdpdGgNCnRoZSBQRk4g
dG8gbWFwIHRoZSBYZW5zdG9yZSByaW5nIHBhZ2UuIE1heWJlIHRoaXMgc2hvdWxkIGJlIGNo
YW5nZWQgdG8gdXNlDQp0aGUgZ3JhbnQgZW50cnksIHRvbz8NCg0KDQpKdWVyZ2VuDQoNCg==

--------------ayNag0yu1eLYaC2E0XRDMQ0x
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------ayNag0yu1eLYaC2E0XRDMQ0x--

--------------g4fXSKxlImaQsBgaeHNovRnH--

--------------0JS4HoVF0CdaNsppIN81Phlu
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmZQpu4FAwAAAAAACgkQsN6d1ii/Ey9j
+QgAnFXDi3Iz9s9w15kGC/dSJWQEAaYvil0h0OnM2eiHUdhtuM3etZOEtOypo7SJqMOAGmo05OOc
uDaAiVZfZfzwT2SE3H2hjlQWGbN6ktduUwdhquF5CQISKpNkFGpgkVjj+g1ONuQzSLHz60Axi++c
FBZRQvEmK/6h7WpmpphVs2EuYUrcNFsqyIwwus68M/4TW+zhUWHf1koPjaBzvD93ZVQYAvmm/bHk
yzVs9WIxmfq/BrihKuo4FP59Fnr67sJr+ibpka+YHdtMH5PLy6pmsRciN+2A67MkvAL+5HX6Jylg
QGZ3OvNGmp5JUfGn1XZIfvGFU1cYTebdzo1iRLaT1A==
=efVD
-----END PGP SIGNATURE-----

--------------0JS4HoVF0CdaNsppIN81Phlu--


From xen-devel-bounces@lists.xenproject.org Fri May 24 15:15:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 15:15:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729621.1134829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAWdt-0006rJ-UB; Fri, 24 May 2024 15:15:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729621.1134829; Fri, 24 May 2024 15:15:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAWdt-0006rC-Qz; Fri, 24 May 2024 15:15:37 +0000
Received: by outflank-mailman (input) for mailman id 729621;
 Fri, 24 May 2024 15:15:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sAWds-0006r6-WA
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 15:15:37 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7882f9e8-19e0-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 17:15:35 +0200 (CEST)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-351da5838fcso7063945f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 08:15:35 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a090366sm1794956f8f.56.2024.05.24.08.15.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 08:15:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7882f9e8-19e0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716563735; x=1717168535; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Pzha0SXnIFFw19paCm4++P584G1jtIlrmAuQIZQA6dE=;
        b=limjjwZ1pDHvLoffNejEbuTU6wiIhLlSd+SOX+Quft2zDJ1w4C4JBVgwUDygdYK6wj
         xskaPi87Imu7mUTwUzBxC6CEYMICMpQjFfbepbnt95/q2vSM1yULkrCtXBxe7k+ybT/W
         IP6FXkPi2G5mV1cWylH1etRkYw8T97UV84mj8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716563735; x=1717168535;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Pzha0SXnIFFw19paCm4++P584G1jtIlrmAuQIZQA6dE=;
        b=uvfesDQROwqMoffwLjniWHqafilVmGR5YT+fFi5MD9tzDQkBumg4o4nyD8i8uKt6Lh
         LdlIsWHBEiX8C5Q4DAZYj+MPmy5au+yLOGEDueTExVw5r/Q45z+uyEVyVlBmkUM8Hqmi
         2ZQcx3KgbOksMOWoN9I9K3XHBsCWaBVy4B1nE3AlOmiwX8BOCIJtBSK/RsbF2StPoi/l
         SFFmNv5u/tGzUEr6BXmVp2SQHV5J/1I2/74Gl2MBWj6bAMgveazJ9HesHhJJmSAUXOq7
         1ehfod1RIzMJALwEjKqhO9xUiGSviPtAGDPNZlUkJZqOYuz5fgqEFTLVBxG+f2upFY87
         y75g==
X-Gm-Message-State: AOJu0YycwJZXaF2puH09HQlu3gr84KwnbBW1qDKN8ohpA91Q/sqvsSnN
	tD6/rkeLB0OEs9pSICRqSFochas2AUejNtOUNPcnz++THgsLnsdEZp9vep37O/c=
X-Google-Smtp-Source: AGHT+IHlHiW2Hh/ZZWF3LWZwfHs3V9smVwqZIdKzr8tYVGynHLEvWLfT0e4s9D02jD5e8EqOefi+HQ==
X-Received: by 2002:a5d:4a0b:0:b0:34c:9a04:466f with SMTP id ffacd0b85a97d-35527055733mr1868554f8f.50.1716563735135;
        Fri, 24 May 2024 08:15:35 -0700 (PDT)
Message-ID: <8cac1707-855b-478a-b88d-7fd619f19352@cloud.com>
Date: Fri, 24 May 2024 16:15:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlA_6Abtw-u4a84J@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZlA_6Abtw-u4a84J@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 24/05/2024 08:21, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
>> Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
>> populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
>> from hvmloader.
>>
>> While at this also remove ap_callin, as writing the APIC ID may serve the same
>> purpose.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * New patch. Replaces adding cpu policy to hvmloader in v1.
>> ---
>>  tools/firmware/hvmloader/config.h    |  6 ++++-
>>  tools/firmware/hvmloader/hvmloader.c |  4 +--
>>  tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
>>  tools/firmware/hvmloader/util.h      |  5 ++++
>>  xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
>>  5 files changed, 47 insertions(+), 9 deletions(-)
>>
>> diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
>> index c82adf6dc508..edf6fa9c908c 100644
>> --- a/tools/firmware/hvmloader/config.h
>> +++ b/tools/firmware/hvmloader/config.h
>> @@ -4,6 +4,8 @@
>>  #include <stdint.h>
>>  #include <stdbool.h>
>>  
>> +#include <xen/hvm/hvm_info_table.h>
>> +
>>  enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
>>  extern enum virtual_vga virtual_vga;
>>  
>> @@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
>>  
>>  #define IOAPIC_ID           0x01
>>  
>> +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
>> +
>>  #define LAPIC_BASE_ADDRESS  0xfee00000
>> -#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
>> +#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
>>  
>>  #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
>>  #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
>> diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
>> index c58841e5b556..1eba92229925 100644
>> --- a/tools/firmware/hvmloader/hvmloader.c
>> +++ b/tools/firmware/hvmloader/hvmloader.c
>> @@ -342,11 +342,11 @@ int main(void)
>>  
>>      printf("CPU speed is %u MHz\n", get_cpu_mhz());
>>  
>> +    smp_initialise();
>> +
>>      apic_setup();
>>      pci_setup();
>>  
>> -    smp_initialise();
>> -
>>      perform_tests();
>>  
>>      if ( bios->bios_info_setup )
>> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
>> index a668f15d7e1f..4d75f239c2f5 100644
>> --- a/tools/firmware/hvmloader/smp.c
>> +++ b/tools/firmware/hvmloader/smp.c
>> @@ -29,7 +29,34 @@
>>  
>>  #include <xen/vcpu.h>
>>  
>> -static int ap_callin, ap_cpuid;
>> +static int ap_cpuid;
>> +
>> +/**
>> + * Lookup table of x2APIC IDs.
>> + *
>> + * Each entry is populated its respective CPU as they come online. This is required
>> + * for generating the MADT with minimal assumptions about ID relationships.
>> + */
>> +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
>> +
>> +static uint32_t read_apic_id(void)
>> +{
>> +    uint32_t apic_id;
>> +
>> +    cpuid(1, NULL, &apic_id, NULL, NULL);
>> +    apic_id >>= 24;
>> +
>> +    /*
>> +     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
>> +     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
>> +     * but only if the x2APIC feature is present. If there are that many CPUs
>> +     * it's guaranteed to be there so we can avoid checking for it specifically
>> +     */
>> +    if ( apic_id == 255 )
>> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
>> +
>> +    return apic_id;
>> +}
>>  
>>  static void ap_start(void)
>>  {
>> @@ -37,12 +64,12 @@ static void ap_start(void)
>>      cacheattr_init();
>>      printf("done.\n");
>>  
>> +    wmb();
>> +    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();
> 
> Further thinking about this: do we really need the wmb(), given the
> usage of ACCESS_ONCE()?
> 
> wmb() is a compiler barrier, and the usage of volatile in
> ACCESS_ONCE() should already prevent any compiler re-ordering.
> 
> Thanks, Roger.

volatile reads/writes cannot be reordered with other volatile
reads/writes, but volatile reads/writes can be reordered with
non-volatile reads/writes, AFAIR.

My intent here was to prevent the compiler omitting the write (though in
practice it didn't). I think the barrier is still required to prevent
reordering according to the spec.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 15:16:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 15:16:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729626.1134838 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAWeK-0007F5-4P; Fri, 24 May 2024 15:16:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729626.1134838; Fri, 24 May 2024 15:16:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAWeK-0007Ey-1o; Fri, 24 May 2024 15:16:04 +0000
Received: by outflank-mailman (input) for mailman id 729626;
 Fri, 24 May 2024 15:16:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sAWeJ-0006r6-FZ
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 15:16:03 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8897f6c5-19e0-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 17:16:02 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-421087b6f3fso9691095e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 08:16:02 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421089667fbsm23226005e9.9.2024.05.24.08.16.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 08:16:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8897f6c5-19e0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716563762; x=1717168562; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9JrTo/FWXvXLGJX0A+cyjzMM3Yze1fosVh7VTTyCRZk=;
        b=ikmQ7uxo75JExPHvDmtLuGsedR7A+KaosMosLcVHfrnSfzfpp38IOD7XhYANhGNn+I
         sKD/ujKdpernJ/P9YGbfuH5DZJc4LG6vw+KZ98J5c6+cOCSTifpMacvRbPrVKujvO0U3
         C2rk4VTQdu03YUSL6GxrQtAVjYMKSWj4c8HjA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716563762; x=1717168562;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9JrTo/FWXvXLGJX0A+cyjzMM3Yze1fosVh7VTTyCRZk=;
        b=peqSCWFM2Rre+crqFaDAm1bYJLeVdBu9wIIs+JLOJ+879RCuqGMmf8c/s/9dTLysCo
         BYuCmTnJCjNa0HJX3qkM/ffSCVWsLWHESRvzQxYIZMdLPcHBfhSFsfoO7UVl+AG5lOW7
         J6jvbHJ3XVXffsG8zFeBEY69Ko2HXsGhHyUt3z/QbAjK6gDqxIflc0scqUgpgiWepjVv
         cre8YFSsCwXy86PyeRzkbsJ0qtDAQqMKmHIvkypBJbv2RCAyChbSs8Lh+6UlusAnXnB4
         uSqyk/9bkGh28fNKpiRUte23aUxNtlkESYtxLJYzwEfBrC/umnxYVo5wDaI6MfxAXT95
         +Zxg==
X-Gm-Message-State: AOJu0YwdFjr/dqpSvYca2eg9Ey9Ot/AI8QPPFL9H634nWgJamlGOp5OK
	tA2+kBZy6SKtA7DfIFs2jbnySKxwtjJZ6jFXLbXPD+k/1aUAcecMXpkiJdkUaPE=
X-Google-Smtp-Source: AGHT+IElGLDWgvok5fpHRZ3Aj+ujcGFBt08ISngygkdwl+RuHW3A2M4ezk/7P4Ux6GC0kRpFBsTo3Q==
X-Received: by 2002:a7b:cbd0:0:b0:420:1592:da3f with SMTP id 5b1f17b1804b1-421089d9db8mr22247835e9.11.1716563762200;
        Fri, 24 May 2024 08:16:02 -0700 (PDT)
Message-ID: <d6ebb2f0-fb15-4ca4-9833-cd635511b2d3@cloud.com>
Date: Fri, 24 May 2024 16:16:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9rOqooa3PJCyw1@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <Zk9rOqooa3PJCyw1@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 17:13, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
>> Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
>> populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
>> from hvmloader.
>>
>> While at this also remove ap_callin, as writing the APIC ID may serve the same
>> purpose.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * New patch. Replaces adding cpu policy to hvmloader in v1.
>> ---
>>  tools/firmware/hvmloader/config.h    |  6 ++++-
>>  tools/firmware/hvmloader/hvmloader.c |  4 +--
>>  tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
>>  tools/firmware/hvmloader/util.h      |  5 ++++
>>  xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
>>  5 files changed, 47 insertions(+), 9 deletions(-)
>>
>> diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
>> index c82adf6dc508..edf6fa9c908c 100644
>> --- a/tools/firmware/hvmloader/config.h
>> +++ b/tools/firmware/hvmloader/config.h
>> @@ -4,6 +4,8 @@
>>  #include <stdint.h>
>>  #include <stdbool.h>
>>  
>> +#include <xen/hvm/hvm_info_table.h>
>> +
>>  enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
>>  extern enum virtual_vga virtual_vga;
>>  
>> @@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
>>  
>>  #define IOAPIC_ID           0x01
>>  
>> +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
>> +
>>  #define LAPIC_BASE_ADDRESS  0xfee00000
>> -#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
>> +#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
>>  
>>  #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
>>  #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
>> diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
>> index c58841e5b556..1eba92229925 100644
>> --- a/tools/firmware/hvmloader/hvmloader.c
>> +++ b/tools/firmware/hvmloader/hvmloader.c
>> @@ -342,11 +342,11 @@ int main(void)
>>  
>>      printf("CPU speed is %u MHz\n", get_cpu_mhz());
>>  
>> +    smp_initialise();
>> +
>>      apic_setup();
>>      pci_setup();
>>  
>> -    smp_initialise();
>> -
>>      perform_tests();
>>  
>>      if ( bios->bios_info_setup )
>> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
>> index a668f15d7e1f..4d75f239c2f5 100644
>> --- a/tools/firmware/hvmloader/smp.c
>> +++ b/tools/firmware/hvmloader/smp.c
>> @@ -29,7 +29,34 @@
>>  
>>  #include <xen/vcpu.h>
>>  
>> -static int ap_callin, ap_cpuid;
>> +static int ap_cpuid;
>> +
>> +/**
>> + * Lookup table of x2APIC IDs.
>> + *
>> + * Each entry is populated its respective CPU as they come online. This is required
>> + * for generating the MADT with minimal assumptions about ID relationships.
>> + */
>> +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
>> +
>> +static uint32_t read_apic_id(void)
>> +{
>> +    uint32_t apic_id;
>> +
>> +    cpuid(1, NULL, &apic_id, NULL, NULL);
>> +    apic_id >>= 24;
>> +
>> +    /*
>> +     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
>> +     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
>> +     * but only if the x2APIC feature is present. If there are that many CPUs
>> +     * it's guaranteed to be there so we can avoid checking for it specifically
>> +     */
> 
> Maybe I'm missing something, but given the current code won't Xen just
> return the low 8 bits from the x2APIC ID?  I don't see any code in
> guest_cpuid() that adjusts the IDs to be 255 when > 255.>
>> +    if ( apic_id == 255 )
>> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
> 
> Won't the correct logic be to check if x2APIC is set in CPUID, and
> then fetch the APIC ID from leaf 0xb, otherwise fallback to fetching
> the APID ID from leaf 1?

I was pretty sure that was the behaviour of real HW, but clearly I was
wrong. Just checked on a beefy machine and that's indeed the low 8 bits,
just as Xen emulates. Got confused with the core count, that does clip
to 255.

Will adjust by explicitly checking for the x2apic_id bit.

> 
>> +
>> +    return apic_id;
>> +}
>>  
>>  static void ap_start(void)
>>  {
>> @@ -37,12 +64,12 @@ static void ap_start(void)
>>      cacheattr_init();
>>      printf("done.\n");
>>  
>> +    wmb();
>> +    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();
> 
> A comment would be helpful here, that CPU_TO_X2APICID[ap_cpuid] is
> used as synchronization that the AP has started.
> 
> You probably want to assert that read_apic_id() doesn't return 0,
> otherwise we are skewed.

Not a bad idea. Sure

> 
>> +
>>      if ( !ap_cpuid )
>>          return;
>>  
>> -    wmb();
>> -    ap_callin = 1;
>> -
>>      while ( 1 )
>>          asm volatile ( "hlt" );
>>  }
>> @@ -86,10 +113,11 @@ static void boot_cpu(unsigned int cpu)
>>          BUG();
>>  
>>      /*
>> -     * Wait for the secondary processor to complete initialisation.
>> +     * Wait for the secondary processor to complete initialisation,
>> +     * which is signaled by its x2APIC ID being writted to the LUT.
>>       * Do not touch shared resources meanwhile.
>>       */
>> -    while ( !ap_callin )
>> +    while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) )
>>          cpu_relax();
> 
> As a further improvement, we could launch all APs in pararell, and use
> a for loop to wait until all positions of the CPU_TO_X2APICID array
> are set.

I thought about it, but then we'd need locking for the prints as well,
or refactor things so only the BSP prints on success.

The time taken is truly negligible, so I reckon it's better left for
another patch.

> 
>>  
>>      /* Take the secondary processor offline. */
>> diff --git a/tools/firmware/hvmloader/util.h b/tools/firmware/hvmloader/util.h
>> index 14078bde1e30..51e9003bc615 100644
>> --- a/tools/firmware/hvmloader/util.h
>> +++ b/tools/firmware/hvmloader/util.h
>> @@ -23,6 +23,11 @@ enum {
>>  #define __STR(...) #__VA_ARGS__
>>  #define STR(...) __STR(__VA_ARGS__)
>>  
>> +#define __ACCESS_ONCE(x) ({                             \
>> +            (void)(typeof(x))0; /* Scalar typecheck. */ \
>> +            (volatile typeof(x) *)&(x); })
>> +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
> 
> Might be better for this to be placed in common-macros.h?

Sure.

> 
>> +
>>  /* GDT selector values. */
>>  #define SEL_CODE16          0x0008
>>  #define SEL_DATA16          0x0010
>> diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
>> index 84911f3ebcb4..6c005f0b0b38 100644
>> --- a/xen/arch/x86/include/asm/hvm/hvm.h
>> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
>> @@ -16,6 +16,7 @@
>>  #include <asm/current.h>
>>  #include <asm/x86_emulate.h>
>>  #include <asm/hvm/asid.h>
>> +#include <asm/hvm/vlapic.h>
> 
> Is this a stray change?  Otherwise I don't see why this need to be
> part of this patch when the rest of the changes are exclusively to
> hvmloader.
> 
> Thanks, Roger.

Should've been part of the vlapic_policy_update change. That line got
lost in the v1->v2 conversion. I just moved to where it logically
belongs now.

Cheers,
Alejandro




From xen-devel-bounces@lists.xenproject.org Fri May 24 16:00:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 16:00:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729641.1134849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAXL0-00067a-Dd; Fri, 24 May 2024 16:00:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729641.1134849; Fri, 24 May 2024 16:00:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAXL0-00067T-Au; Fri, 24 May 2024 16:00:10 +0000
Received: by outflank-mailman (input) for mailman id 729641;
 Fri, 24 May 2024 16:00:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAXKy-00067I-U3; Fri, 24 May 2024 16:00:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAXKy-0005Wu-RL; Fri, 24 May 2024 16:00:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAXKy-0001kz-Ga; Fri, 24 May 2024 16:00:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAXKy-0005Ym-Fz; Fri, 24 May 2024 16:00:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6nbpPd891jysUl7mV3rZhV6UGpUb6cWxlH7BVZjsZDw=; b=DYDWITbNVdtOWoHvzryHa4/i34
	vWX/8UjzdKIkyd/Pl1X0roWn66CcOjP9rMOO1faXC7p0F91BkZjaAiGPi0caeqqO+5V/7IlcAAPpB
	khv6vsflQf+WkvirFONNLxp9ya9gEwP+bN12EnApR6bCl0z2nHLq0QMfGY5iGoSWszi8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186136-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186136: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=701190abb3d5873cf77349b3d90bd11ded74871a
X-Osstest-Versions-That:
    xen=2a40b106e92aaa7ce808c8608dd6473edc67f608
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 16:00:08 +0000

flight 186136 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186136/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186117

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  701190abb3d5873cf77349b3d90bd11ded74871a
baseline version:
 xen                  2a40b106e92aaa7ce808c8608dd6473edc67f608

Last test of basis   186117  2024-05-23 17:02:09 Z    0 days
Testing same since   186136  2024-05-24 14:02:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Henry Wang <Henry.Wang@arm.com>
  Henry Wang <xin.wang2@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 701190abb3d5873cf77349b3d90bd11ded74871a
Author: Henry Wang <xin.wang2@amd.com>
Date:   Thu Mar 21 11:57:06 2024 +0800

    xen/arm: Set correct per-cpu cpu_core_mask
    
    In the common sysctl command XEN_SYSCTL_physinfo, the value of
    cores_per_socket is calculated based on the cpu_core_mask of CPU0.
    Currently on Arm this is a fixed value 1 (can be checked via xl info),
    which is not correct. This is because during the Arm CPU online
    process at boot time, setup_cpu_sibling_map() only sets the per-cpu
    cpu_core_mask for itself.
    
    cores_per_socket refers to the number of cores that belong to the same
    socket (NUMA node). Currently Xen on Arm does not support physical
    CPU hotplug and NUMA, also we assume there is no multithread. Therefore
    cores_per_socket means all possible CPUs detected from the device
    tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
    accordingly. Modify the in-code comment which seems to be outdated. Add
    a warning to users if Xen is running on processors with multithread
    support.
    
    Signed-off-by: Henry Wang <Henry.Wang@arm.com>
    Signed-off-by: Henry Wang <xin.wang2@amd.com>
    Reviewed-by: Michal Orzel <michal.orzel@amd.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Fri May 24 17:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 17:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729659.1134861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAYKI-00055W-Pj; Fri, 24 May 2024 17:03:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729659.1134861; Fri, 24 May 2024 17:03:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAYKI-00055P-Mt; Fri, 24 May 2024 17:03:30 +0000
Received: by outflank-mailman (input) for mailman id 729659;
 Fri, 24 May 2024 17:03:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sAYKG-00055B-PM
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 17:03:28 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 891eb371-19ef-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 19:03:27 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-42108856c33so5780545e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 10:03:26 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100fadcabsm57605785e9.35.2024.05.24.10.03.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 10:03:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 891eb371-19ef-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716570205; x=1717175005; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=l4N1HbDa+as1czqlaMAoz2GDemeBb+xIjSoFaK5NyIA=;
        b=RRpaUyzexFR53fMMjOgTd23f71kQSIOSREGkkIg2wCBzKqrAJHRxHE9hmsHXVYmjkF
         6oDLUEx4DtEdsMw1w1scZvtEhRDGpVoiEASYTLhNWbBZg4y1lYeL3kJa/NThvdAeTUfq
         F0hXyjIkNgh6qkFPGcegBdABoHdDIxuPq97P0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716570205; x=1717175005;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=l4N1HbDa+as1czqlaMAoz2GDemeBb+xIjSoFaK5NyIA=;
        b=PhMfe3J9VQ1lmpQWUirTToClBVpMSu0wLEu+UoY29GWRPtqpX+FS49T8CG91GmFHr/
         SOuiGT1Pt881kwjvuMEPUn2k+GAWfLrAlT+FoAWweXUZjV1GuFXsmNB++/NqazP+bmAs
         TDK8zjI7YBzj62wq8/HDqWQTPqhWrVtU4RZmmFZ8W0KcSKRTBBfgvGXkdQ2Autv/axdC
         NtX5HuiVlTzj45uzmgkyJuRU+bJgwEfKnhp3WDy28zYux13+o2HGIlLaOU4EFHt8UHJg
         Tep7VXHLPJTMISbbuH8TtrxDHo2MoNV3ss3mu45vdfKd1B6BRKfPEe0B7cZjNS5I7wfT
         1KrA==
X-Gm-Message-State: AOJu0Yyuj+8R3zL31OMmSbD0k9PFFNpDZiybXOEBm8qYriCUEA0dzKA9
	GKnxxKvbrWS6MTvrs4bA0P8QqBt/YEbm/WInjBrvNadWpNIDuzNKEw43hnqIycE=
X-Google-Smtp-Source: AGHT+IFU3/6IHcS/ifs31twOKN3VRepoofgHFSr95ASnRD2vKHmf/GUteE8F6DywKYa9znfEcW6Jtg==
X-Received: by 2002:a05:600c:3b1c:b0:418:2981:c70f with SMTP id 5b1f17b1804b1-421015fdc48mr50714235e9.19.1716570205533;
        Fri, 24 May 2024 10:03:25 -0700 (PDT)
Message-ID: <1e16ffa9-3f6c-43ed-b36c-55fe132c8145@cloud.com>
Date: Fri, 24 May 2024 18:03:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 7/8] xen/x86: Derive topologically correct x2APIC IDs
 from the policy
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <87a2a4589e330472b7260ff6ab513744596a4488.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlBSPYCpEAosNVzo@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZlBSPYCpEAosNVzo@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 24/05/2024 09:39, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:26PM +0100, Alejandro Vallejo wrote:
>> Implements the helper for mapping vcpu_id to x2apic_id given a valid
>> topology in a policy. The algo is written with the intention of extending
>> it to leaves 0x1f and e26 in the future.
> 
> Using 0x1f and e26 is kind of confusing.  I would word as "0x1f and
> extended leaf 0x26" to avoid confusion.
> 
>>
>> Toolstack doesn't set leaf 0xb and the HVM default policy has it cleared,
>> so the leaf is not implemented. In that case, the new helper just returns
>> the legacy mapping.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * const-ify the test definitions
>>   * Cosmetic changes (newline + parameter name in prototype)
>> ---
>>  tools/tests/cpu-policy/test-cpu-policy.c | 63 ++++++++++++++++++++
>>  xen/include/xen/lib/x86/cpu-policy.h     |  2 +
>>  xen/lib/x86/policy.c                     | 73 ++++++++++++++++++++++--
>>  3 files changed, 133 insertions(+), 5 deletions(-)
>>
>> diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
>> index 0ba8c418b1b3..82a6aeb23317 100644
>> --- a/tools/tests/cpu-policy/test-cpu-policy.c
>> +++ b/tools/tests/cpu-policy/test-cpu-policy.c
>> @@ -776,6 +776,68 @@ static void test_topo_from_parts(void)
>>      }
>>  }
>>  
>> +static void test_x2apic_id_from_vcpu_id_success(void)
>> +{
>> +    static const struct test {
>> +        unsigned int vcpu_id;
>> +        unsigned int threads_per_core;
>> +        unsigned int cores_per_pkg;
>> +        uint32_t x2apic_id;
>> +        uint8_t x86_vendor;
>> +    } tests[] = {
>> +        {
>> +            .vcpu_id = 3, .threads_per_core = 3, .cores_per_pkg = 8,
>> +            .x2apic_id = 1 << 2,
>> +        },
>> +        {
>> +            .vcpu_id = 6, .threads_per_core = 3, .cores_per_pkg = 8,
>> +            .x2apic_id = 2 << 2,
>> +        },
>> +        {
>> +            .vcpu_id = 24, .threads_per_core = 3, .cores_per_pkg = 8,
>> +            .x2apic_id = 1 << 5,
>> +        },
>> +        {
>> +            .vcpu_id = 35, .threads_per_core = 3, .cores_per_pkg = 8,
>> +            .x2apic_id = (35 % 3) | (((35 / 3) % 8)  << 2) | ((35 / 24) << 5),
>> +        },
>> +        {
>> +            .vcpu_id = 96, .threads_per_core = 7, .cores_per_pkg = 3,
>> +            .x2apic_id = (96 % 7) | (((96 / 7) % 3)  << 3) | ((96 / 21) << 5),
>                                                       ^ extra space (same above)
> 
>> +        },
>> +    };
>> +
>> +    const uint8_t vendors[] = {
>> +        X86_VENDOR_INTEL,
>> +        X86_VENDOR_AMD,
>> +        X86_VENDOR_CENTAUR,
>> +        X86_VENDOR_SHANGHAI,
>> +        X86_VENDOR_HYGON,
>> +    };
>> +
>> +    printf("Testing x2apic id from vcpu id success:\n");
>> +
>> +    /* Perform the test run on every vendor we know about */
>> +    for ( size_t i = 0; i < ARRAY_SIZE(vendors); ++i )
>> +    {
>> +        struct cpu_policy policy = { .x86_vendor = vendors[i] };
> 
> Newline.

Ack

> 
>> +        for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
>> +        {
>> +            const struct test *t = &tests[i];
>> +            uint32_t x2apic_id;
>> +            int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);
> 
> Overly long line.
> 
> Won't it be better to define `policy` in this scope, so that for each
> test you start with a clean policy, rather than having leftover data
> from the previous test?

The leftover data is overridden during setup by x86_topo_from_parts(),
but I can see the appeal. Sure.

> 
> Also you could initialize x2apic_id at definition:
> 
> const struct test *t = &tests[j];
> struct cpu_policy policy = { .x86_vendor = vendors[i] };
> int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);
> uint32_t x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);

Seeing this snippet I just realized there's a bug. The second loop
should use j rather than i. Ugh.

As for the initialization, I want to prevent feeding garbage into
x86_x2apic_id_from_vcpu_id(). For which there's an "if ( !rc )" missing
to gate the call.

I'll sort both of those.

> 
>> +
>> +            x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);
>> +            if ( rc || x2apic_id != t->x2apic_id )
>> +                fail("FAIL[%d] - '%s cpu%u %u t/c %u c/p'. bad x2apic_id: expected=%u actual=%u\n",
>> +                     rc,
>> +                     x86_cpuid_vendor_to_str(policy.x86_vendor),
>> +                     t->vcpu_id, t->threads_per_core, t->cores_per_pkg,
>> +                     t->x2apic_id, x2apic_id);
>> +        }
>> +    }
>> +}
>> +
>>  int main(int argc, char **argv)
>>  {
>>      printf("CPU Policy unit tests\n");
>> @@ -794,6 +856,7 @@ int main(int argc, char **argv)
>>      test_is_compatible_failure();
>>  
>>      test_topo_from_parts();
>> +    test_x2apic_id_from_vcpu_id_success();
>>  
>>      if ( nr_failures )
>>          printf("Done: %u failures\n", nr_failures);
>> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
>> index f5df18e9f77c..2cbc2726a861 100644
>> --- a/xen/include/xen/lib/x86/cpu-policy.h
>> +++ b/xen/include/xen/lib/x86/cpu-policy.h
>> @@ -545,6 +545,8 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>>  /**
>>   * Calculates the x2APIC ID of a vCPU given a CPU policy
>>   *
>> + * If the policy lacks leaf 0xb falls back to legacy mapping of apic_id=cpu*2
>> + *
>>   * @param p          CPU policy of the domain.
>>   * @param id         vCPU ID of the vCPU.
>>   * @returns x2APIC ID of the vCPU.
>> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
>> index d033ee5398dd..e498e32f8fd7 100644
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -2,15 +2,78 @@
>>  
>>  #include <xen/lib/x86/cpu-policy.h>
>>  
>> +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
>> +{
>> +    /*
>> +     * `nr_logical` reported by Intel is the number of THREADS contained in
>> +     * the next topological scope. For example, assuming a system with 2
>> +     * threads/core and 3 cores/module in a fully symmetric topology,
>> +     * `nr_logical` at the core level will report 6. Because it's reporting
>> +     * the number of threads in a module.
>> +     *
>> +     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
>> +     * level (cores/complex, etc) so we can return it as-is.
>> +     */
>> +    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
>> +        return p->topo.subleaf[lvl].nr_logical;
>> +
>> +    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;
> 
> Line length here and in the function declaration.
> 

Ack

>> +}
>> +
>>  uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>>  {
>> +    uint32_t shift = 0, x2apic_id = 0;
>> +
>> +    /* In the absence of topology leaves, fallback to traditional mapping */
>> +    if ( !p->topo.subleaf[0].type )
>> +        return id * 2;
>> +
>>      /*
>> -     * TODO: Derive x2APIC ID from the topology information inside `p`
>> -     *       rather than from vCPU ID. This bodge is a temporary measure
>> -     *       until all infra is in place to retrieve or derive the initial
>> -     *       x2APIC ID from migrated domains.
> 
> I'm a bit confused with this, the policy is domain wide, so we will
> always need to pass the vCPU ID into x86_x2apic_id_from_vcpu_id()?
> IOW: the x2APIC ID will always be derived from the vCPU ID.
> 
> Thanks, Roger.

The x2APIC ID is derived (after the series) from the vCPU ID _and_ the
topology information. The vCPU alone will work out in all cases because
it'll be cached in the vlapic hvm structure.

I guess the comment could be rewritten as "... rather than from the vCPU
ID alone..."

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 17:16:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 17:16:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729670.1134873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAYWT-0006zK-SN; Fri, 24 May 2024 17:16:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729670.1134873; Fri, 24 May 2024 17:16:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAYWT-0006zD-OC; Fri, 24 May 2024 17:16:05 +0000
Received: by outflank-mailman (input) for mailman id 729670;
 Fri, 24 May 2024 17:16:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=di0w=M3=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sAYWS-0006z4-6t
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 17:16:04 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4c194d21-19f1-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 19:16:02 +0200 (CEST)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-35507dfe221so705819f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 10:16:02 -0700 (PDT)
Received: from [192.168.0.16] (0545937c.skybroadband.com. [5.69.147.124])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a1c9546sm2018171f8f.82.2024.05.24.10.16.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 24 May 2024 10:16:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c194d21-19f1-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716570962; x=1717175762; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=brtySv79pi2/FIgEkDor3mhBZT1VVY0tx1m7DGmBBms=;
        b=jkLW+oMAQ/2UF4ZmxvfMeb5jRpXamoZnhqUIME7wsvzP03kxZdaN5LAleX/28Y/im3
         +oU0e7dsM1T7oiSMSGPopdVjQFsb0P6yNvPtLcxryXJi/jhdSuSW7K9jgcyi6QZVfubf
         ifUlPIhp34Padwg+CRxnPVeLaxYYyhFX8qU/o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716570962; x=1717175762;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=brtySv79pi2/FIgEkDor3mhBZT1VVY0tx1m7DGmBBms=;
        b=PzRd+DwQdt70eSs4Tp/nAbrFU/rGK458RPGp4HrrkT9tTknBV7oQAxsYnVC7eBPr/X
         Er7QvjbD3KwnHh9zT2y+h264HLpFj+6jNxIcQwF1IXGoM5TJO883F7st5itWdB48imh0
         7LAQ0rBVVxzoPsYU9cHb0I8i8I8cgHX7UxouBQ2NtT9Zt7lP2aOcc5WKNgyKCJXLG7H9
         plqXaGMEORnqMOja/Ak9KQOigic5DZIuIFAe88PWaVkrB5TbXvPEgPJSRALU82s8xWBV
         KF6X24/ms4WJwv/R2N6QukbmdN3H5dOON2yP7BZeKlJ8nl4Nmbvc3q0v08ODbGxZPd10
         SDRg==
X-Gm-Message-State: AOJu0YwmFYL3gRWowAQJR9UMQbxTgbQtaBSNYA8LYQ8uyQymcTngXy9l
	ryXQj+EGm6tlrxQB2cC5SGKMkTeu//Hytz8t6KA0dJ0oQEio6drsQ3a8VMiGvgw=
X-Google-Smtp-Source: AGHT+IHW0IoZ1nORonOTuJonlGtmcvQfv0/kRvnBxKiSO/oRrLTkoFuUVGgYbXuvKEKvBiYPjnWfsA==
X-Received: by 2002:adf:e809:0:b0:34d:9161:2253 with SMTP id ffacd0b85a97d-355245dec55mr2687495f8f.12.1716570962107;
        Fri, 24 May 2024 10:16:02 -0700 (PDT)
Message-ID: <6e1c52a3-98d2-42e9-96b7-91010806cdf5@cloud.com>
Date: Fri, 24 May 2024 18:16:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 8/8] xen/x86: Synthesise domain topologies
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlBWtXkpkqzh1EWC@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZlBWtXkpkqzh1EWC@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 24/05/2024 09:58, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:27PM +0100, Alejandro Vallejo wrote:
>> Expose sensible topologies in leaf 0xb. At the moment it synthesises non-HT
>> systems, in line with the previous code intent.
>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * Zap the topology leaves of (pv/hvm)_(def/max)_policy rather than the host policy
>> ---
>>  tools/libs/guest/xg_cpuid_x86.c | 62 +++++----------------------------
>>  xen/arch/x86/cpu-policy.c       |  9 +++--
>>  2 files changed, 15 insertions(+), 56 deletions(-)
>>
>> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
>> index 4453178100ad..8170769dbe43 100644
>> --- a/tools/libs/guest/xg_cpuid_x86.c
>> +++ b/tools/libs/guest/xg_cpuid_x86.c
>> @@ -584,7 +584,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>      bool hvm;
>>      xc_domaininfo_t di;
>>      struct xc_cpu_policy *p = xc_cpu_policy_init();
>> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
>> +    unsigned int nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
>>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>>      uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
>>      uint32_t len = ARRAY_SIZE(host_featureset);
>> @@ -727,59 +727,15 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>>      }
>>      else
>>      {
>> -        /*
>> -         * Topology for HVM guests is entirely controlled by Xen.  For now, we
>> -         * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
>> -         */
>> -        p->policy.basic.htt = true;
>> -        p->policy.extd.cmp_legacy = false;
>> -
>> -        /*
>> -         * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
>> -         * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
>> -         * overflow.
>> -         */
>> -        if ( !p->policy.basic.lppp )
>> -            p->policy.basic.lppp = 2;
>> -        else if ( !(p->policy.basic.lppp & 0x80) )
>> -            p->policy.basic.lppp *= 2;
>> -
>> -        switch ( p->policy.x86_vendor )
>> +        /* TODO: Expose the ability to choose a custom topology for HVM/PVH */
>> +        unsigned int threads_per_core = 1;
>> +        unsigned int cores_per_pkg = di.max_vcpu_id + 1;
> 
> Newline.

ack

> 
>> +        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);
> 
> I assume this generates the same topology as the current code, or will
> the population of the leaves be different in some way?
> 

The current code does not populate 0xb. This generates a topology
consistent with the existing INTENDED topology. The actual APIC IDs will
be different though (because there's no skipping of odd values).

All the dance in patch 1 was to make this migrate-safe. The x2apic ID is
stored in the lapic hidden regs so differences with previous behaviour
don't matter.

IOW, The differences are:
  * 0xb is exposed, whereas previously it wasn't
  * APIC IDs are compacted such that new_apicid=old_apicid/2
  * There's also a cleanup of the murkier paths to put the right core
    counts in the right leaves (whereas previously it was bonkers)

>> +        if ( rc )
>>          {
>> -        case X86_VENDOR_INTEL:
>> -            for ( i = 0; (p->policy.cache.subleaf[i].type &&
>> -                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
>> -            {
>> -                p->policy.cache.subleaf[i].cores_per_package =
>> -                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
>> -                p->policy.cache.subleaf[i].threads_per_cache = 0;
>> -            }
>> -            break;
>> -
>> -        case X86_VENDOR_AMD:
>> -        case X86_VENDOR_HYGON:
>> -            /*
>> -             * Leaf 0x80000008 ECX[15:12] is ApicIdCoreSize.
>> -             * Leaf 0x80000008 ECX[7:0] is NumberOfCores (minus one).
>> -             * Update to reflect vLAPIC_ID = vCPU_ID * 2.  But avoid
>> -             * - overflow,
>> -             * - going out of sync with leaf 1 EBX[23:16],
>> -             * - incrementing ApicIdCoreSize when it's zero (which changes the
>> -             *   meaning of bits 7:0).
>> -             *
>> -             * UPDATE: I addition to avoiding overflow, some
>> -             * proprietary operating systems have trouble with
>> -             * apic_id_size values greater than 7.  Limit the value to
>> -             * 7 for now.
>> -             */
>> -            if ( p->policy.extd.nc < 0x7f )
>> -            {
>> -                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
>> -                    p->policy.extd.apic_id_size++;
>> -
>> -                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
>> -            }
>> -            break;
>> +            ERROR("Failed to generate topology: t/c=%u c/p=%u",
>> +                  threads_per_core, cores_per_pkg);
> 
> Could you also print the error code?

Sure

> 
>> +            goto out;
>>          }
>>      }
>>  
>> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
>> index 4b6d96276399..0ad871732ba0 100644
>> --- a/xen/arch/x86/cpu-policy.c
>> +++ b/xen/arch/x86/cpu-policy.c
>> @@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
>>  
>>      p->basic.raw[0x8] = EMPTY_LEAF;
>>  
>> -    /* TODO: Rework topology logic. */
>> -    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>> -
>>      p->basic.raw[0xc] = EMPTY_LEAF;
>>  
>>      p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
>> @@ -621,6 +618,9 @@ static void __init calculate_pv_max_policy(void)
>>      recalculate_xstate(p);
>>  
>>      p->extd.raw[0xa] = EMPTY_LEAF; /* No SVM for PV guests. */
>> +
>> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>>  }
>>  
>>  static void __init calculate_pv_def_policy(void)
>> @@ -773,6 +773,9 @@ static void __init calculate_hvm_max_policy(void)
>>  
>>      /* It's always possible to emulate CPUID faulting for HVM guests */
>>      p->platform_info.cpuid_faulting = true;
>> +
>> +    /* Wipe host topology. Toolstack is expected to synthesise a sensible one */
> 
> Line length.
> 
> /* Wipe host topology.  Populated by toolstack. */
> 
> Would be OK IMO.

Ack

> 
> Thanks, Roger.
>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
> 
> Note that currently the host policy also gets the topology leaves
> cleared, is it intended to not clear them anymore after this change?
> 
> (as you only clear the leaves for the guest {max,def} policies)
> 
> Thanks, Roger.

It was like that originally in v1, I changed in v2 as part of feedback
from Jan.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Fri May 24 17:54:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 17:54:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729681.1134882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAZ7U-0004AL-Gv; Fri, 24 May 2024 17:54:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729681.1134882; Fri, 24 May 2024 17:54:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAZ7U-0004AE-Cs; Fri, 24 May 2024 17:54:20 +0000
Received: by outflank-mailman (input) for mailman id 729681;
 Fri, 24 May 2024 17:54:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wxWv=M3=kernel.org=pr-tracker-bot@srs-se1.protection.inumbo.net>)
 id 1sAZ7T-0004A6-1Q
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 17:54:19 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a39f1e49-19f6-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 19:54:18 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 95D8B63220;
 Fri, 24 May 2024 17:54:16 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPS id 45C7FC2BBFC;
 Fri, 24 May 2024 17:54:16 +0000 (UTC)
Received: from aws-us-west-2-korg-oddjob-1.ci.codeaurora.org
 (localhost.localdomain [127.0.0.1])
 by aws-us-west-2-korg-oddjob-1.ci.codeaurora.org (Postfix) with ESMTP id
 2F5ADC4332E; Fri, 24 May 2024 17:54:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a39f1e49-19f6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716573256;
	bh=71Tp+L1wf0lXLrMfxl6H7PNMVHNtLSbbAAf5z1JLfnw=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=dk0atlaW8wjp1W03BUxoBSCE1tel44OPPxr5G4zSrzVKQb+pmZUIzLKjt35P1O/yE
	 a1HBOh4uaiT3prEI5hgEn3Biy3RA+M3VcRdMBqkBmAMOKIvGObgW7hLCNjL3tdHDXK
	 yGauezXmwwLOJogIpOSPr+1R2QdxT0xDOf4bkbi51S/Y5nnTxBoGsN0x7M0iypLwad
	 9LUdAMnlya4ulT835zXdZutTJKsBqtIsrAV7FYiO/V98lIEWYPL0LOVmeYdCd4sGXV
	 I7MQ5BBcE+jd/TmHKKu6BgrZdCmhoojd+f9AYylxD6O7BeJVylKrGy9Boj1BcEqqQ5
	 axpqpq+vhOATA==
Subject: Re: [GIT PULL] xen: branch for v6.10-rc1
From: pr-tracker-bot@kernel.org
In-Reply-To: <20240524133733.27825-1-jgross@suse.com>
References: <20240524133733.27825-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20240524133733.27825-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.10a-rc1-tag
X-PR-Tracked-Commit-Id: a3607581cd49c17128a486a526a36a97bafcb2bb
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: 9351f138d1dcbe504cd829abe590ba7f3387f09c
Message-Id: <171657325618.4634.8934129604090299962.pr-tracker-bot@kernel.org>
Date: Fri, 24 May 2024 17:54:16 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, sstabellini@kernel.org

The pull request you sent on Fri, 24 May 2024 15:37:33 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.10a-rc1-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/9351f138d1dcbe504cd829abe590ba7f3387f09c

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Fri May 24 18:15:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 18:15:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729689.1134892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAZRb-0007YQ-6Y; Fri, 24 May 2024 18:15:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729689.1134892; Fri, 24 May 2024 18:15:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAZRb-0007YJ-3j; Fri, 24 May 2024 18:15:07 +0000
Received: by outflank-mailman (input) for mailman id 729689;
 Fri, 24 May 2024 18:15:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAZRa-0007Y8-45; Fri, 24 May 2024 18:15:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAZRa-0003Mg-00; Fri, 24 May 2024 18:15:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAZRZ-0005up-N5; Fri, 24 May 2024 18:15:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAZRZ-0000Cf-Ma; Fri, 24 May 2024 18:15:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=P8yXWmKNXSfPcnct+KvFK0UUHOvH6P0v1QsrCNvKbr8=; b=Vg+P7T6bNNfeqlWG4XWtqusIxj
	X/Zzf8zpEhCLa5/1JTDQRc6qDApj8y93gYJisi6klaJOMwkC2I00QvF1w2h6+FgXO1gTsodHzq4Ib
	9zpHzEyG1FtGqXuR117q9V2nGZkZNYeytL1lumMHHJQ8fxHPOBOib2/MMCkEyyh/domk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186137-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186137: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3e722403cd16388a0e4044e705a2b34c841d76ca
X-Osstest-Versions-That:
    ovmf=7142e648416ff5d3eac6c6d607874805f5de0ca8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 18:15:05 +0000

flight 186137 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186137/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3e722403cd16388a0e4044e705a2b34c841d76ca
baseline version:
 ovmf                 7142e648416ff5d3eac6c6d607874805f5de0ca8

Last test of basis   186054  2024-05-21 02:43:06 Z    3 days
Testing same since   186137  2024-05-24 16:12:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ard Biesheuvel <ardb@kernel.org>
  Doug Flick <dougflick@microsoft.com>
  Doug Flick [MSFT] <doug.edk2@gmail.com>
  Flickdm <douglas.flick@microsoft.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Jiewe Yao <Jiewen.yao@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   7142e64841..3e722403cd  3e722403cd16388a0e4044e705a2b34c841d76ca -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 24 19:51:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 19:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729712.1134902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAawk-0003sq-UY; Fri, 24 May 2024 19:51:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729712.1134902; Fri, 24 May 2024 19:51:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAawk-0003sj-Rx; Fri, 24 May 2024 19:51:22 +0000
Received: by outflank-mailman (input) for mailman id 729712;
 Fri, 24 May 2024 19:51:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAawk-0003sX-0c; Fri, 24 May 2024 19:51:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAawj-0000eh-UD; Fri, 24 May 2024 19:51:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAawj-0001KT-KF; Fri, 24 May 2024 19:51:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAawj-00075p-Jn; Fri, 24 May 2024 19:51:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wy/ELFo1Mt8L5gz/86Mux333Ta7k4b4ouSA1cm4LpWw=; b=fQ8S4VWU5gHwpg/GRDxdlXqxJm
	jDgQpZyE/euJ+o+2uYH2YFm3OvHLChzOg1d1/pLq/kYNLTEGC4B44lyklUQn4Zc9fdVmweSdkJqNZ
	YffwFWQSDhIeaBUaNlIixiaxpaeyTobDeKeIxgsDhsd2bGAyc83oZZe8iRPYuN+4StKU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186139-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186139: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
X-Osstest-Versions-That:
    xen=2a40b106e92aaa7ce808c8608dd6473edc67f608
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 19:51:21 +0000

flight 186139 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186139/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
baseline version:
 xen                  2a40b106e92aaa7ce808c8608dd6473edc67f608

Last test of basis   186117  2024-05-23 17:02:09 Z    1 days
Failing since        186136  2024-05-24 14:02:11 Z    0 days    2 attempts
Testing same since   186139  2024-05-24 17:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Henry Wang <Henry.Wang@arm.com>
  Henry Wang <xin.wang2@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2a40b106e9..2172a01c4c  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 24 19:58:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 19:58:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729736.1134979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb3d-0005RR-Hw; Fri, 24 May 2024 19:58:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729736.1134979; Fri, 24 May 2024 19:58:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb3d-0005RK-Es; Fri, 24 May 2024 19:58:29 +0000
Received: by outflank-mailman (input) for mailman id 729736;
 Fri, 24 May 2024 19:58:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAb3c-0005R8-P9; Fri, 24 May 2024 19:58:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAb3c-00015H-NZ; Fri, 24 May 2024 19:58:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAb3c-0001US-10; Fri, 24 May 2024 19:58:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAb3c-0003gP-0T; Fri, 24 May 2024 19:58:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iSqx2LVthd58w2Pwv+fjyNqCF4KMFN8kZ5zVt+e7978=; b=tvTfgppi1ZUmWgi2aYqBfsUmK1
	ci4eq0MJ/GabZ1fTFMkP0rhprrFmoxQyhaYmzkdEbw/K2fcfV2VH7IiNlTj6zX4ndv3NkxssqEdBG
	C7acuXr/bdSupWZvZtwYKv/vFtY0i2S3iUMz7h8OWyf2TSUc4NuFmSuiSeH2dcn/8Ub8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186132-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186132: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2a40b106e92aaa7ce808c8608dd6473edc67f608
X-Osstest-Versions-That:
    xen=ced21fbb2842ac4655048bdee56232974ff9ff9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 19:58:28 +0000

flight 186132 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186132/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186078
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186105
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186105
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186105
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186105
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186105
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2a40b106e92aaa7ce808c8608dd6473edc67f608
baseline version:
 xen                  ced21fbb2842ac4655048bdee56232974ff9ff9c

Last test of basis   186105  2024-05-23 09:38:07 Z    1 days
Testing same since   186132  2024-05-24 04:13:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alejandro Vallejo <alejandro.vallejo@cloud.com>
  Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bobby Eshleman <bobbyeshleman@gmail.com>
  Christian Lindig <christian.lindig@cloud.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Olaf Hering <olaf@aepfle.de>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Tamas K Lengyel <tamas@tklengyel.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ced21fbb28..2a40b106e9  2a40b106e92aaa7ce808c8608dd6473edc67f608 -> master


From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729747.1135020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8n-0007nn-3L; Fri, 24 May 2024 20:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729747.1135020; Fri, 24 May 2024 20:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8m-0007mt-UA; Fri, 24 May 2024 20:03:48 +0000
Received: by outflank-mailman (input) for mailman id 729747;
 Fri, 24 May 2024 20:03:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8l-000773-Ln
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:47 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ba408386-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:46 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a6265d3ccbdso119940266b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:46 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba408386-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581025; x=1717185825; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TaUcCJcK4BqMwPMepfH6t6Qsv4SxK/nVUJrRoe6o34w=;
        b=mcHmkHjUoslQCFVUUDVK61dolAGHbnYX1FUg2gGl/K3AJY6Nw6Kggp2e0+AsUSYZRs
         oSfwJz6/aP0o3Wmqmvpfpc2EE669FIYhXACVEgM641/p8Q460Pn38PKfKYska7TungVt
         gYmSp9V66Kdn2T2JUjQzlX9p2x/ESKoY1FO7c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581025; x=1717185825;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=TaUcCJcK4BqMwPMepfH6t6Qsv4SxK/nVUJrRoe6o34w=;
        b=OJ9gibelfdSO8X4kgtWJoEF0hU9FKEz/oclUvDmNaidzEQ78iwpAoIGMLzveRBST+2
         EEHFnrdGBdwHQ6XZ1SrkUX+8dAADAG4AueRkBn50jIAYpNnNkPjdlsmHx+rZP2kV7l1a
         rg5doEm/6KKUDYXkbDA4uHLCf/xvN/fqEU726MZxRYTwpmnIjQ+WU5yQ3NkKlrhfT9sg
         po4fGjPpeJMX3TqHRxwsy0XehJvbyfeF4f+A5sZ6Rk0JM7J6cACCjZmZPBRXJIhD4/Ar
         2Es3AK9cHxz/TC4UoquMkUUiCnSD+fAbLAXkOTOMep0XLBgmSwZ0IWPlKCmusVH1zfkb
         vw0A==
X-Gm-Message-State: AOJu0Yzne5SbKxM73+nUpfrd7YFGFiOhci0D0cNYa028m1NIo7RpHyA0
	KXnpbWIT2UEx8r+cfpRwkG5C7LSOJVxW8ShhDKB/zNy1CTTwSxbDlRZi5VeK+XOGW3OYYgPN6+r
	M3g4=
X-Google-Smtp-Source: AGHT+IEBtxw2KlfDn/D6rOsVW7w/BH+oTZGKvt35ZLp5TbEGg+n73+jDyDgG5CyxaxQOKjliu8EmvQ==
X-Received: by 2002:a17:906:1d4c:b0:a5c:e2ea:ba59 with SMTP id a640c23a62f3a-a62642dab49mr169622666b.29.1716581025156;
        Fri, 24 May 2024 13:03:45 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 02/13] xen/bitops: Cleanup ahead of rearrangements
Date: Fri, 24 May 2024 21:03:27 +0100
Message-Id: <20240524200338.1232391-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

 * Rename __attribute_pure__ to just __pure before it gains users.
 * Introduce __constructor which is going to be used in lib/, and is
   unconditionally cf_check.
 * Identify the areas of xen/bitops.h which are a mess.
 * Introduce xen/boot-check.h as helpers for compile and boot time testing.
   This provides a statement of the ABI, and a confirmation that arch-specific
   implementations behave as expected.

Sadly Clang 7 and older isn't happy with the compile time checks.  Skip them,
and just rely on the runtime checks.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Break macros out into a header as they're going to be used elsewhere too
 * Use panic() rather than BUG_ON() to be more helpful when something fails
 * Brackets in HIDE()
 * Alignment adjustments
 * Skip COMPILE_CHECK() for Clang < 8
---
 xen/include/xen/bitops.h     | 13 ++++++--
 xen/include/xen/boot-check.h | 60 ++++++++++++++++++++++++++++++++++++
 xen/include/xen/compiler.h   |  3 +-
 3 files changed, 72 insertions(+), 4 deletions(-)
 create mode 100644 xen/include/xen/boot-check.h

diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index e3c5a4ccf321..9b40f20381a2 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -1,5 +1,7 @@
-#ifndef _LINUX_BITOPS_H
-#define _LINUX_BITOPS_H
+#ifndef XEN_BITOPS_H
+#define XEN_BITOPS_H
+
+#include <xen/compiler.h>
 #include <xen/types.h>
 
 /*
@@ -103,8 +105,13 @@ static inline int generic_flsl(unsigned long x)
  * Include this here because some architectures need generic_ffs/fls in
  * scope
  */
+
+/* --------------------- Please tidy above here --------------------- */
+
 #include <asm/bitops.h>
 
+/* --------------------- Please tidy below here --------------------- */
+
 #ifndef find_next_bit
 /**
  * find_next_bit - find the next set bit in a memory region
@@ -294,4 +301,4 @@ static inline __u32 ror32(__u32 word, unsigned int shift)
 
 #define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
 
-#endif
+#endif /* XEN_BITOPS_H */
diff --git a/xen/include/xen/boot-check.h b/xen/include/xen/boot-check.h
new file mode 100644
index 000000000000..250f9a40d3b0
--- /dev/null
+++ b/xen/include/xen/boot-check.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/*
+ * Helpers for boot-time checks of basic logic, including confirming that
+ * examples which should be calculated by the compiler are.
+ */
+#ifndef XEN_BOOT_CHECK_H
+#define XEN_BOOT_CHECK_H
+
+#include <xen/lib.h>
+
+/* Hide a value from the optimiser. */
+#define HIDE(x)                                                         \
+    ({ typeof(x) _x = (x); asm volatile ( "" : "+r" (_x) ); _x; })
+
+/*
+ * Check that fn(val) can be calcuated by the compiler, and that it gives the
+ * expected answer.
+ *
+ * Clang < 8 can't fold constants through static inlines, causing this to
+ * fail.  Simply skip it for incredibly old compilers.
+ */
+#if !CONFIG_CC_IS_CLANG || CONFIG_CLANG_VERSION >= 80000
+#define COMPILE_CHECK(fn, val, res)                                     \
+    do {                                                                \
+        typeof(fn(val)) real = fn(val);                                 \
+                                                                        \
+        if ( !__builtin_constant_p(real) )                              \
+            asm ( ".error \"'" STR(fn(val)) "' not compile-time constant\"" ); \
+        else if ( real != res )                                         \
+            asm ( ".error \"Compile time check '" STR(fn(val) == res) "' failed\"" ); \
+    } while ( 0 )
+#else
+#define COMPILE_CHECK(fn, val, res)
+#endif
+
+/*
+ * Check that Xen's runtime logic for fn(val) gives the expected answer.  This
+ * requires using HIDE() to prevent the optimiser from collapsing the logic
+ * into a constant.
+ */
+#define RUNTIME_CHECK(fn, val, res)                     \
+    do {                                                \
+        typeof(fn(val)) real = fn(HIDE(val));           \
+                                                        \
+        if ( real != res )                              \
+            panic("%s: %s(%s) expected %u, got %u\n",   \
+                  __func__, #fn, #val, real, res);      \
+    } while ( 0 )
+
+/*
+ * Perform compiletime and runtime checks for fn(val) == res.
+ */
+#define CHECK(fn, val, res)                     \
+    do {                                        \
+        COMPILE_CHECK(fn, val, res);            \
+        RUNTIME_CHECK(fn, val, res);            \
+    } while ( 0 )
+
+#endif /* XEN_BOOT_CHECK_H */
diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
index 179ff23e62c5..444bf80142c7 100644
--- a/xen/include/xen/compiler.h
+++ b/xen/include/xen/compiler.h
@@ -86,7 +86,8 @@
 #define inline inline __init
 #endif
 
-#define __attribute_pure__  __attribute__((__pure__))
+#define __constructor       __attribute__((__constructor__)) cf_check
+#define __pure              __attribute__((__pure__))
 #define __attribute_const__ __attribute__((__const__))
 #define __transparent__     __attribute__((__transparent_union__))
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729745.1134993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-0007AI-D9; Fri, 24 May 2024 20:03:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729745.1134993; Fri, 24 May 2024 20:03:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-00079e-99; Fri, 24 May 2024 20:03:47 +0000
Received: by outflank-mailman (input) for mailman id 729745;
 Fri, 24 May 2024 20:03:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8k-000773-1Y
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:46 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b8f00521-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:43 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-57851e8b891so1960666a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:43 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8f00521-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581023; x=1717185823; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=4DI2nCTbDNK8GGdRlkLTCOYJkS9ZJd3XmZjCbJg+i8E=;
        b=CTURHOIdWxwiOiOg7oKUlc8hcvygFq6JFi0vfUj1DHPGbK4bCK4H50FJgT7gi8EcAT
         fbmI1Hgjw1B16SJIplFVG34aykFRlEMnDfR4ssdMLXscMc1WKign4W7suEiyuqXa7IqI
         tBqxhCSFjhqvrAeJZTxKtKzukOCylLCU175iw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581023; x=1717185823;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=4DI2nCTbDNK8GGdRlkLTCOYJkS9ZJd3XmZjCbJg+i8E=;
        b=UIwxZcYkIysnjT2sBfLTd5tS0Vsa4pnBixMOkIdAfEgf2cRLWBijr6Lig6yWx1lfKL
         7uXy8d/+QZz/G6QjFEWbo57r2oovKxbuk7XXeV63appU5/iobDnBDnGxFRO1Zpfm0GVG
         PEWxQwywpXZtSehHcykSzDMCHkYIJk/9Q2jXBGemT0u98juyik1UvLlaDViEx8nNAnDJ
         ECkZfx8DBoOhTRfhDUhqOZfrfUpj29vCtVI5DSew5odvientpduZIjPohp32FzR2fJJn
         D4NLaecJKfcuvVvKy3rAHqSb61EXYBmFwK/M0Jfs8IieZNB+hSCYCNYYxaedxA9hj09n
         Ys+w==
X-Gm-Message-State: AOJu0YyTU1h4iysD3RrCg3Q8MWn/zXsHs5fVKkihIweqEnSNdv6NzTVm
	s99mXBVFDnTB5D7YMELtMKp0IwctwhytDrxQjR1m8cw6meN/Dz4h/3+IOJ1BuxNmZZFIrI/GW2I
	pNzE=
X-Google-Smtp-Source: AGHT+IHtx2CakoNT3GAJbyRmjB+SV1sRndmi8tJ1g5NZaHxGMRzrt42n3ko48q6I8MnNs1MxySMElg==
X-Received: by 2002:a17:906:cccf:b0:a5d:239:1a59 with SMTP id a640c23a62f3a-a623e6d544fmr502847466b.3.1716581022890;
        Fri, 24 May 2024 13:03:42 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 for-4.19 00/13] xen/bitops: Untangle ffs()/fls() infrastructure
Date: Fri, 24 May 2024 21:03:25 +0100
Message-Id: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

bitops.h is a mess.  It has grown organtically over many years, and forces
unreasonable repsonsibilities out into the per-arch stubs.

Start cleaning it up with ffs() and friends.  Across the board, this adds:

 * Functioning bitops without arch-specific asm
 * An option for arches to provide more optimal code generation
 * Compile-time constant folding
 * Testing at both compile time and during init that the basic operations
   behave according to spec.

and the only reason this series isn't a net reduction in code alone is the
because of the new unit testing.

This form is superior in many ways, including getting RISC-V support for free.

v2:
 * Many changes.  See patches for details
 * Include the fls() side of the infrastructure too.

Testing:
  https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1304664544
  https://cirrus-ci.com/github/andyhhp/xen/

Series-wide net bloat-o-meter:

  x86:   up/down: 51/-247 (-196)
  ARM64: up/down: 40/-400 (-360)

and PPC64 reproduced in full, just to demonstrate how absurd it was to have
generic_f?s() as static inlines...

  add/remove: 1/0 grow/shrink: 1/11 up/down: 228/-4832 (-4604)
  Function                                     old     new   delta
  init_constructors                              -     220    +220
  start_xen                                     92     100      +8
  alloc_heap_pages                            1980    1744    -236
  xenheap_max_mfn                              360     120    -240
  free_heap_pages                              784     536    -248
  find_next_zero_bit                           564     276    -288
  find_next_bit                                548     260    -288
  find_first_zero_bit                          444     148    -296
  find_first_bit                               444     132    -312
  xmem_pool_free                              1776    1440    -336
  __do_softirq                                 604     252    -352
  init_heap_pages                             2328    1416    -912
  xmem_pool_alloc                             2920    1596   -1324


Andrew Cooper (12):
  ppc/boot: Run constructors on boot
  xen/bitops: Cleanup ahead of rearrangements
  ARM/bitops: Change find_first_set_bit() to be a define
  xen/page_alloc: Coerce min(flsl(), foo) expressions to being unsigned
  xen/bitops: Implement generic_f?sl() in lib/
  xen/bitops: Implement ffs() in common logic
  x86/bitops: Improve arch_ffs() in the general case
  xen/bitops: Implement ffsl() in common logic
  xen/bitops: Replace find_first_set_bit() with ffsl() - 1
  xen/bitops: Delete find_first_set_bit()
  xen/bitops: Clean up ffs64()/fls64() definitions
  xen/bitops: Rearrange the top of xen/bitops.h

Oleksii Kurochko (1):
  xen/bitops: Implement fls()/flsl() in common logic

 xen/arch/arm/include/asm/arm32/bitops.h      |   2 -
 xen/arch/arm/include/asm/arm64/bitops.h      |  12 --
 xen/arch/arm/include/asm/bitops.h            |  35 +---
 xen/arch/ppc/include/asm/bitops.h            |  17 +-
 xen/arch/ppc/setup.c                         |   2 +
 xen/arch/x86/guest/xen/xen.c                 |   4 +-
 xen/arch/x86/hvm/dom0_build.c                |   2 +-
 xen/arch/x86/hvm/hpet.c                      |   8 +-
 xen/arch/x86/include/asm/bitops.h            | 114 +++++++------
 xen/arch/x86/include/asm/pt-contig-markers.h |   2 +-
 xen/arch/x86/mm.c                            |   2 +-
 xen/arch/x86/mm/p2m-pod.c                    |   4 +-
 xen/common/Makefile                          |   1 +
 xen/common/bitops.c                          |  89 +++++++++++
 xen/common/page_alloc.c                      |   6 +-
 xen/common/softirq.c                         |   2 +-
 xen/drivers/passthrough/amd/iommu_map.c      |   2 +-
 xen/drivers/passthrough/iommu.c              |   4 +-
 xen/drivers/passthrough/x86/iommu.c          |   4 +-
 xen/include/xen/bitops.h                     | 159 ++++++++-----------
 xen/include/xen/boot-check.h                 |  60 +++++++
 xen/include/xen/compiler.h                   |   3 +-
 xen/lib/Makefile                             |   2 +
 xen/lib/generic-ffsl.c                       |  65 ++++++++
 xen/lib/generic-flsl.c                       |  68 ++++++++
 25 files changed, 444 insertions(+), 225 deletions(-)
 create mode 100644 xen/common/bitops.c
 create mode 100644 xen/include/xen/boot-check.h
 create mode 100644 xen/lib/generic-ffsl.c
 create mode 100644 xen/lib/generic-flsl.c

-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729748.1135025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8n-0007rh-CP; Fri, 24 May 2024 20:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729748.1135025; Fri, 24 May 2024 20:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8n-0007r0-7c; Fri, 24 May 2024 20:03:49 +0000
Received: by outflank-mailman (input) for mailman id 729748;
 Fri, 24 May 2024 20:03:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8m-00076x-7T
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:48 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bb530fd6-1a08-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 22:03:47 +0200 (CEST)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2e576057c2bso137040041fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:47 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb530fd6-1a08-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581027; x=1717185827; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HcvFli469CD4OUU+qKqIt2aUdVZOLqd4ksEwvUfVXhg=;
        b=tMjwoj/WoBzXXrRXI2vSSvfnBCJUWWz2JveF3LyAps+Pya3YK+JxJBS6kXaBfBZeN/
         xwpVYmd255e+fgjFbdMkTNpVmNeZgfSt/QJpJ54YSxj2G/6bqgv+/2ue7gFY8Pr9slWf
         VANQO/Rex4kK+QKsF3EQfMDB3lPTApNOdUqIk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581027; x=1717185827;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=HcvFli469CD4OUU+qKqIt2aUdVZOLqd4ksEwvUfVXhg=;
        b=n9LqQtZqtqt8J12rNKWvpCLDqcRJQ2rPm7CgOU2xMFA8gOtJaMGpCeLarlwIfRErtj
         ApLX5wTbLxO7XYFn72P6/KpzRP+Ju5u1C0RMtQ9j92gU9kvu4p6K66XBCPUniE9CJDUE
         6m5XldV8uUCPygnY+2ar/fWFnwRvSv6+KgaIf2xYPQ/Sw1BZ31uWBMG9KsemRFcedlDr
         YHzt9eGP+TL9RvoaSS2HBUQhZ2qwRSmRERvVyA1RvxQla8F6yObamWnyx/r5qaJ2SL+Z
         UwyCUoMVKPpsmsphynTEzcE6BI97oaqy1Y6wZcWqzPaxfOZEp8NIXX33UE8W1GJgmb31
         OA9w==
X-Gm-Message-State: AOJu0YxuS15NSPVwp/73eGplKVLxIaLWqw46gsusG1yYHnKFZyEEfFoV
	vkVoJe8lBkMOHbUHv9sjzNrNxN0n9kC/83ZLM01AF0+RyC7hdZXMgm0nbqNp0EI0iAdd96xudeJ
	4LNs=
X-Google-Smtp-Source: AGHT+IFW77E6sOaTPxeY1Anh235eDcr+kQYnA1qSCVR/tfE2QVye//bW0h/5/1gXeVXq88hbfi6MTA==
X-Received: by 2002:a2e:8910:0:b0:2e2:64d:6849 with SMTP id 38308e7fff4ca-2e95b27b09bmr24937071fa.50.1716581026945;
        Fri, 24 May 2024 13:03:46 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 04/13] xen/page_alloc: Coerce min(flsl(), foo) expressions to being unsigned
Date: Fri, 24 May 2024 21:03:29 +0100
Message-Id: <20240524200338.1232391-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is in order to maintain bisectability through the subsequent changes,
where flsl() changes sign-ness non-atomically by architecture.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New
---
 xen/common/page_alloc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 7c1bdfc046bf..8d3342e95236 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1842,7 +1842,7 @@ static void _init_heap_pages(const struct page_info *pg,
          * Note that the value of ffsl() and flsl() starts from 1 so we need
          * to decrement it by 1.
          */
-        unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
+        unsigned int inc_order = min(MAX_ORDER + 0U, flsl(e - s) - 1U);
 
         if ( s )
             inc_order = min(inc_order, ffsl(s) - 1U);
@@ -2266,7 +2266,7 @@ void __init xenheap_max_mfn(unsigned long mfn)
     ASSERT(!first_node_initialised);
     ASSERT(!xenheap_bits);
     BUILD_BUG_ON((PADDR_BITS - PAGE_SHIFT) >= BITS_PER_LONG);
-    xenheap_bits = min(flsl(mfn + 1) - 1 + PAGE_SHIFT, PADDR_BITS);
+    xenheap_bits = min(flsl(mfn + 1) - 1U + PAGE_SHIFT, PADDR_BITS + 0U);
     printk(XENLOG_INFO "Xen heap: %u bits\n", xenheap_bits);
 }
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729744.1134991 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-00077K-6I; Fri, 24 May 2024 20:03:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729744.1134991; Fri, 24 May 2024 20:03:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-00077D-1y; Fri, 24 May 2024 20:03:47 +0000
Received: by outflank-mailman (input) for mailman id 729744;
 Fri, 24 May 2024 20:03:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8j-00076x-Ka
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:45 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b96d3871-1a08-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 22:03:44 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f462so1170233a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:44 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b96d3871-1a08-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581024; x=1717185824; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0cmvQAA1/fqugFr137dsLZT1PnHQ0umZh9dz+pm+nU8=;
        b=VPdJVyqgwasHqrUATPyBDmILeCRS/BnJotEdwvxpS8wB7ARW0hFTX2+ZwVAsTDFHhB
         ytswLBFA8+4GpToqoV7r8hzNQpjyWTTuza3yPdRzX/szY1UNQGS2SiEQnILFPK/QRJgZ
         +rF0NfkIYOGoeccIzU13c8zjB9/5CPak4BOts=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581024; x=1717185824;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0cmvQAA1/fqugFr137dsLZT1PnHQ0umZh9dz+pm+nU8=;
        b=S4za0inlRGdPqdmNHN3fgK9depTk1vshPnH0MomCsJEHfmjQaTBGYuz6KiszduLQ64
         pAwyI80Jhepcp4Ty6AYuBInmfTQV0GHUGZ9BYxclci8swoojpou2dlf0XZ8XXxjHVTOQ
         Eq0AqOWOpdg+dzjPfO9XTzSlMFjwnRnM1hoOxrW8vpYWp2cavYe+vKfucKQaZ/auMIt/
         sJHxT0Bwp0QFAgMtxuq5zjNNq5zDzmjJmCmQlnysKS4wzP0K8OX1lAUd6vztt0sY3rl1
         05nrzzq2+91J0i/dwViPmWaV2/58jZcDyww1O81Ef6tnwMg9xdicISKV+7YhVclOpqpe
         IxQw==
X-Gm-Message-State: AOJu0YwO84DycTcod+1811+cGFZLJdAXCdAloztrAam9xjurdpmjd6Uc
	1pRux1QwHlrFVfNFu/MB4FHNIZ+1iuZdfuhDYF8iNjhY7evfMXE9rBmaHTLHS8dfUbw4ueXCMeu
	ln7Y=
X-Google-Smtp-Source: AGHT+IFBtXPfaDlofqCV/C7m0DlQIHtiBrnlQHz5V05sHUxMp24nS1J5nbmw96zUOc2AFoBrVjV2mw==
X-Received: by 2002:a17:906:3683:b0:a59:a431:f951 with SMTP id a640c23a62f3a-a6265148b8amr226929166b.48.1716581023916;
        Fri, 24 May 2024 13:03:43 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 01/13] ppc/boot: Run constructors on boot
Date: Fri, 24 May 2024 21:03:26 +0100
Message-Id: <20240524200338.1232391-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

PPC collects constructors, but doesn't run them yet.  Do so.

They'll shortly be used to confirm correct behaviour of the bitops primitives.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

CI: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/6931084695

v2:
 * New

RISC-V collects them too, but can't call init_constructors() until lib/ctors.c
is included in the build.

Constructors is the only way to get these tests working on PPC/RISC-V as
neither suvivie boot with initcalls() active.  Then again, initcalls() are
just a not-invented-here constructor, and we'd probably do well to move them
over..
---
 xen/arch/ppc/setup.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/ppc/setup.c b/xen/arch/ppc/setup.c
index 101bdd8bb648..7fe06aa4bfb0 100644
--- a/xen/arch/ppc/setup.c
+++ b/xen/arch/ppc/setup.c
@@ -39,6 +39,8 @@ void __init noreturn start_xen(unsigned long r3, unsigned long r4,
 
     setup_initial_pagetables();
 
+    init_constructors();
+
     early_printk("Hello, ppc64le!\n");
 
     for ( ; ; )
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729746.1135003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-0007Jn-T5; Fri, 24 May 2024 20:03:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729746.1135003; Fri, 24 May 2024 20:03:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8l-0007Hm-Mc; Fri, 24 May 2024 20:03:47 +0000
Received: by outflank-mailman (input) for mailman id 729746;
 Fri, 24 May 2024 20:03:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8l-00076x-5Z
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:47 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bab09028-1a08-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 22:03:46 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a626ac4d299so130911666b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:46 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bab09028-1a08-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581026; x=1717185826; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rnH7+DLTFV7y8fHOYV2l/9gV4Q+4MdvhQykzDmFzA1Q=;
        b=BXxkz8TrhETVdIlnQW5HMSmfVEsfaRN3l//4D2li3A3D1stDBnxa98ECMXVDsjVB2d
         Rq0LXvRoUpXz3VM6u0IZ5wK73LPSX8aTfC9ORpdaSvfWzdrspAgD+lfLzlv8qigzsEBv
         7BBImx+RFrAaqgOy8yOBcKW15JtnGs5DTCz6o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581026; x=1717185826;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rnH7+DLTFV7y8fHOYV2l/9gV4Q+4MdvhQykzDmFzA1Q=;
        b=vi21uvdPX0YLhvrV93YXtIBciV8s/2f8ufX5pn5MVEHP2KSdYTw8DDAVTqHv7T58wb
         CGrNDwGDfrGqowxB89JLWR+mDPZbakE1HlM6wG7hRQaGB4AIlWxYYrRGEyG/KAA2tCra
         hxpRFTzxN3Maw+ZMXsuBOyz+SZ0FEZesCk74S/9ch5+2Gn+JrM59f7w2FPvv3HMUT1zY
         m1odqGMofvzKX1pWpn3Wz4vO6ZmKJiXmdfEFVCOmVXsk38alTl52eoBbovWRL3FJZEtP
         ywokkiuXr2pThthSqlKrrcG86a3vlDJCvPHzxXGDRx1zQuGhpgpg8cCjdcon5286JAmC
         +JPw==
X-Gm-Message-State: AOJu0Yz1ngR92wJJDAzVPbXFQVSotggbni2Ma5Cp6O7YUkVCdsVxTKFm
	2+i/u5lvRTXux9tYeX1lXdkrWqzHGr2FglXRdZcy10zENCFfndckxtqd2Ll2IJks68Em9mxVPsm
	U9Ks=
X-Google-Smtp-Source: AGHT+IFQfNCCCpPoRMWhGk3Fjgu6D6H4jwZJNkukRDDsfp149c37Pbw35L5hdzTbWaMHsYivMRl++A==
X-Received: by 2002:a17:906:f9c9:b0:a5a:f16:32b1 with SMTP id a640c23a62f3a-a62641cfb1dmr238210266b.31.1716581026128;
        Fri, 24 May 2024 13:03:46 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 03/13] ARM/bitops: Change find_first_set_bit() to be a define
Date: Fri, 24 May 2024 21:03:28 +0100
Message-Id: <20240524200338.1232391-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is in order to maintain bisectability through the subsequent changes, as
the order of definitions is altered.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New
---
 xen/arch/arm/include/asm/bitops.h | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index ab030b6cb032..199252201291 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -167,10 +167,7 @@ static inline int fls(unsigned int x)
  * Returns the bit-number of the first set bit (first bit being 0).
  * The input must *not* be zero.
  */
-static inline unsigned int find_first_set_bit(unsigned long word)
-{
-        return ffsl(word) - 1;
-}
+#define find_first_set_bit(w) (ffsl(w) - 1)
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729749.1135040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8p-0008Ma-R9; Fri, 24 May 2024 20:03:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729749.1135040; Fri, 24 May 2024 20:03:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8p-0008MR-N6; Fri, 24 May 2024 20:03:51 +0000
Received: by outflank-mailman (input) for mailman id 729749;
 Fri, 24 May 2024 20:03:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8n-00076x-DI
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:49 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bbbae797-1a08-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 22:03:48 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a626ac4d299so130913866b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:48 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbbae797-1a08-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581028; x=1717185828; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bu+ePM1Qu+LYpA9RNu9XXOypr94Ji2YBQcexG1JyIro=;
        b=TbjXv1tHzR34Fx5UGQJaxQolIWj9YEHixaD1SyQBMNID3ug8TGtF8/0E/7LHD6Lrud
         eCtUXM02q/AnGODv4nbSGDcCNC1P04MjVLsVp3IdS3u2e//2sIaDoYkvzBI6rbS1AdP5
         A6hVosMtYeKceyhHvAzG1apaLakpjd5SXMdSQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581028; x=1717185828;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bu+ePM1Qu+LYpA9RNu9XXOypr94Ji2YBQcexG1JyIro=;
        b=ietkOnjL9OekdGIH8rzK4gBJDxvdcFMBSLT8qo/1/2G+78ncJy1p4wOSgQK1tTI1fU
         TOC7LL+zsVSxmUd8G58L/6Bbpf0HFRxnQyUzaKzpIWVfRe6W7uXg2pRMcsSr8tRO0fVE
         XYfAvIsQfgh187fTtaYTC7FdZAn69xsEWqlvdjtvugC7DuSkjgG35gQtROPIl0CKl8ww
         g41EHQazrotoxzhqE9+SoLjO1obBMMvyog9gX0wTYEgGITxipka+DUfpnXTd3/2ij8j3
         Wv3OA0QDM0urIaNSeSqTXep3n7KGwKrHroyEYUqesPbv0lRuGjwRNWiGakaYoK/5WkGj
         8agw==
X-Gm-Message-State: AOJu0YySZuK8L+BmQyPM4u7mh9vRXrg83WGYTFxZgt5mhHRiqVwDrQCa
	y/0PqSRJDlZsxbarxhbnBabuNkn+n7MWPRQy52NMuhciSXPRb8HXeCYwIP4Pr+XnwZlkdIcg1CE
	HVWg=
X-Google-Smtp-Source: AGHT+IGl+jcja4m/MU00cdn/ZAVvrCiijYwwRz1no2Yal4kdantqZIbqF2IKRw0UMtOqBz/1vjY/fg==
X-Received: by 2002:a17:906:e907:b0:a59:bd78:34ca with SMTP id a640c23a62f3a-a62643e2443mr220690166b.36.1716581027828;
        Fri, 24 May 2024 13:03:47 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 05/13] xen/bitops: Implement generic_f?sl() in lib/
Date: Fri, 24 May 2024 21:03:30 +0100
Message-Id: <20240524200338.1232391-6-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

generic_f?s() being static inline is the cause of lots of the complexity
between the common and arch-specific bitops.h

They appear to be static inline for constant-folding reasons (ARM uses them
for this), but there are better ways to achieve the same effect.

It is presumptuous that an unrolled binary search is the right algorithm to
use on all microarchitectures.  Indeed, it's not for the eventual users, but
that can be addressed at a later point.

It is also nonsense to implement the int form as the base primitive and
construct the long form from 2x int in 64-bit builds, when it's just one extra
step to operate at the native register width.

Therefore, implement generic_f?sl() in lib/.  They're not actually needed in
x86/ARM/PPC by the end of the cleanup (i.e. the functions will be dropped by
the linker), and they're only expected be needed by RISC-V on hardware which
lacks the Zbb extension.

Implement generic_fls() in terms of generic_flsl() for now, but this will be
cleaned up in due course.

Provide basic runtime testing using __constructor inside the lib/ file.  This
is important, as it means testing runs if and only if generic_f?sl() are used
elsewhere in Xen.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New

I suspect we want to swap CONFIG_DEBUG for CONFIG_BOOT_UNIT_TESTS in due
course.  These ought to be able to be used in a release build too.
---
 xen/arch/arm/include/asm/bitops.h |  2 +-
 xen/arch/ppc/include/asm/bitops.h |  2 +-
 xen/include/xen/bitops.h          | 89 ++-----------------------------
 xen/lib/Makefile                  |  2 +
 xen/lib/generic-ffsl.c            | 65 ++++++++++++++++++++++
 xen/lib/generic-flsl.c            | 68 +++++++++++++++++++++++
 6 files changed, 142 insertions(+), 86 deletions(-)
 create mode 100644 xen/lib/generic-ffsl.c
 create mode 100644 xen/lib/generic-flsl.c

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 199252201291..ec1cf7b9b323 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -150,7 +150,7 @@ static inline int fls(unsigned int x)
         int ret;
 
         if (__builtin_constant_p(x))
-               return generic_fls(x);
+               return generic_flsl(x);
 
         asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
         return 32 - ret;
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index bea655796d64..ab692d01717b 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -172,7 +172,7 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
 }
 
 #define flsl(x) generic_flsl(x)
-#define fls(x) generic_fls(x)
+#define fls(x) generic_flsl(x)
 #define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
 #define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
 
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 9b40f20381a2..cd405df96180 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -15,91 +15,12 @@
     (((~0ULL) << (l)) & (~0ULL >> (BITS_PER_LLONG - 1 - (h))))
 
 /*
- * ffs: find first bit set. This is defined the same way as
- * the libc and compiler builtin ffs routines, therefore
- * differs in spirit from the above ffz (man ffs).
- */
-
-static inline int generic_ffs(unsigned int x)
-{
-    int r = 1;
-
-    if (!x)
-        return 0;
-    if (!(x & 0xffff)) {
-        x >>= 16;
-        r += 16;
-    }
-    if (!(x & 0xff)) {
-        x >>= 8;
-        r += 8;
-    }
-    if (!(x & 0xf)) {
-        x >>= 4;
-        r += 4;
-    }
-    if (!(x & 3)) {
-        x >>= 2;
-        r += 2;
-    }
-    if (!(x & 1)) {
-        x >>= 1;
-        r += 1;
-    }
-    return r;
-}
-
-/*
- * fls: find last bit set.
+ * Find First/Last Set bit.
+ *
+ * Bits are labelled from 1.  Returns 0 if given 0.
  */
-
-static inline int generic_fls(unsigned int x)
-{
-    int r = 32;
-
-    if (!x)
-        return 0;
-    if (!(x & 0xffff0000u)) {
-        x <<= 16;
-        r -= 16;
-    }
-    if (!(x & 0xff000000u)) {
-        x <<= 8;
-        r -= 8;
-    }
-    if (!(x & 0xf0000000u)) {
-        x <<= 4;
-        r -= 4;
-    }
-    if (!(x & 0xc0000000u)) {
-        x <<= 2;
-        r -= 2;
-    }
-    if (!(x & 0x80000000u)) {
-        x <<= 1;
-        r -= 1;
-    }
-    return r;
-}
-
-#if BITS_PER_LONG == 64
-
-static inline int generic_ffsl(unsigned long x)
-{
-    return !x || (u32)x ? generic_ffs(x) : generic_ffs(x >> 32) + 32;
-}
-
-static inline int generic_flsl(unsigned long x)
-{
-    u32 h = x >> 32;
-
-    return h ? generic_fls(h) + 32 : generic_fls(x);
-}
-
-#else
-# define generic_ffsl generic_ffs
-# define generic_flsl generic_fls
-#endif
+unsigned int __pure generic_ffsl(unsigned long x);
+unsigned int __pure generic_flsl(unsigned long x);
 
 /*
  * Include this here because some architectures need generic_ffs/fls in
diff --git a/xen/lib/Makefile b/xen/lib/Makefile
index e63798e1d452..a48541596470 100644
--- a/xen/lib/Makefile
+++ b/xen/lib/Makefile
@@ -4,6 +4,8 @@ lib-y += bsearch.o
 lib-y += ctors.o
 lib-y += ctype.o
 lib-y += find-next-bit.o
+lib-y += generic-ffsl.o
+lib-y += generic-flsl.o
 lib-y += list-sort.o
 lib-y += memchr.o
 lib-y += memchr_inv.o
diff --git a/xen/lib/generic-ffsl.c b/xen/lib/generic-ffsl.c
new file mode 100644
index 000000000000..804cbd752efe
--- /dev/null
+++ b/xen/lib/generic-ffsl.c
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <xen/bitops.h>
+#include <xen/boot-check.h>
+#include <xen/init.h>
+
+unsigned int generic_ffsl(unsigned long x)
+{
+    unsigned int r = 1;
+
+    if ( !x )
+        return 0;
+
+#if BITS_PER_LONG > 32
+    if ( !(x & 0xffffffffU) )
+    {
+        x >>= 32;
+        r += 32;
+    }
+#endif
+    if ( !(x & 0xffff) )
+    {
+        x >>= 16;
+        r += 16;
+    }
+    if ( !(x & 0xff) )
+    {
+        x >>= 8;
+        r += 8;
+    }
+    if ( !(x & 0xf) )
+    {
+        x >>= 4;
+        r += 4;
+    }
+    if ( !(x & 3) )
+    {
+        x >>= 2;
+        r += 2;
+    }
+    if ( !(x & 1) )
+    {
+        x >>= 1;
+        r += 1;
+    }
+
+    return r;
+}
+
+#ifdef CONFIG_DEBUG
+static void __init __constructor test_generic_ffsl(void)
+{
+    RUNTIME_CHECK(generic_ffsl, 0, 0);
+    RUNTIME_CHECK(generic_ffsl, 1, 1);
+    RUNTIME_CHECK(generic_ffsl, 3, 1);
+    RUNTIME_CHECK(generic_ffsl, 7, 1);
+    RUNTIME_CHECK(generic_ffsl, 6, 2);
+
+    RUNTIME_CHECK(generic_ffsl, 1UL << (BITS_PER_LONG - 1), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    RUNTIME_CHECK(generic_ffsl, 1UL << 32, 33);
+    RUNTIME_CHECK(generic_ffsl, 1UL << 63, 64);
+#endif
+}
+#endif /* CONFIG_DEBUG */
diff --git a/xen/lib/generic-flsl.c b/xen/lib/generic-flsl.c
new file mode 100644
index 000000000000..e4543aeaf100
--- /dev/null
+++ b/xen/lib/generic-flsl.c
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <xen/bitops.h>
+#include <xen/boot-check.h>
+#include <xen/init.h>
+
+/* Mask of type UL with the upper x bits set. */
+#define UPPER_MASK(x) (~0UL << (BITS_PER_LONG - (x)))
+
+unsigned int generic_flsl(unsigned long x)
+{
+    unsigned int r = BITS_PER_LONG;
+
+    if ( !x )
+        return 0;
+
+#if BITS_PER_LONG > 32
+    if ( !(x & UPPER_MASK(32)) )
+    {
+        x <<= 32;
+        r -= 32;
+    }
+#endif
+    if ( !(x & UPPER_MASK(16)) )
+    {
+        x <<= 16;
+        r -= 16;
+    }
+    if ( !(x & UPPER_MASK(8)) )
+    {
+        x <<= 8;
+        r -= 8;
+    }
+    if ( !(x & UPPER_MASK(4)) )
+    {
+        x <<= 4;
+        r -= 4;
+    }
+    if ( !(x & UPPER_MASK(2)) )
+    {
+        x <<= 2;
+        r -= 2;
+    }
+    if ( !(x & UPPER_MASK(1)) )
+    {
+        x <<= 1;
+        r -= 1;
+    }
+
+    return r;
+}
+
+#ifdef CONFIG_DEBUG
+static void __init __constructor test_generic_flsl(void)
+{
+    RUNTIME_CHECK(generic_flsl, 0, 0);
+    RUNTIME_CHECK(generic_flsl, 1, 1);
+    RUNTIME_CHECK(generic_flsl, 3, 2);
+    RUNTIME_CHECK(generic_flsl, 7, 3);
+    RUNTIME_CHECK(generic_flsl, 6, 3);
+
+    RUNTIME_CHECK(generic_flsl, 1 | (1UL << (BITS_PER_LONG - 1)), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    RUNTIME_CHECK(generic_flsl, 1 | (1UL << 32), 33);
+    RUNTIME_CHECK(generic_flsl, 1 | (1UL << 63), 64);
+#endif
+}
+#endif /* CONFIG_DEBUG */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729750.1135046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8q-0008Qz-BB; Fri, 24 May 2024 20:03:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729750.1135046; Fri, 24 May 2024 20:03:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8q-0008Q7-0z; Fri, 24 May 2024 20:03:52 +0000
Received: by outflank-mailman (input) for mailman id 729750;
 Fri, 24 May 2024 20:03:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8p-00076x-4k
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:51 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bd08fe0b-1a08-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 22:03:50 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f465so1148359a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:50 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd08fe0b-1a08-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581030; x=1717185830; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=s4dvCcdXKh6qx9C0SJSN8op2mWkgo8zuNPTidlZTBXU=;
        b=reUBOJmf1e5428vJskaJS1GtdgRJ7z02uxzxYV1OCXvzkopMKeVvnOrkE2L8n8tYc9
         oETtHEk8gD2P40lb1QmZ2WagdYLSyeUdMAt5IRPPV+vh25DWgk4drijRtLL3akvCrV3u
         9ZIOXjH0Gxbw/IZ1wLuRB2WD627VaEKDw3GCU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581030; x=1717185830;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=s4dvCcdXKh6qx9C0SJSN8op2mWkgo8zuNPTidlZTBXU=;
        b=wl7Etr7+vgdJBonoBoFVHQ9z2M7DpAdlBR+LwdVNw/oUYiu0nwC9EYnDNdQbdcfSNZ
         daFn7zlZCWW/QdDzLxyqsae/WF5ztLadihc8/DBIpI+6XX7BrWxFM8CIBgJQK1o2oIWC
         wBEBPuT59WjR3uOylwwXhzv8BZBY2lRFRMmmW+vm2YylJmhlB8OZ6NgpdgZaD/H5hv7K
         Ixvenn+VkAP9vWZ/9VcaqegWWOTKQvc8Gp98Ro/I3FbKjJ2l+NvMLQYnTBP3LbC59d4v
         3c1+uE6Px/mJ+M4fokGXuSxfhv5K6MWafvaJ+ePEa+q/QPdGaSmMeu9Ixjqt5ga2KYT4
         bxFw==
X-Gm-Message-State: AOJu0YxWRaUfSwAckQh2KVkGucuzwqprVsa5tM2MZ0ii0pDtpRFyJohT
	fBKL5k4bNBGIw28S+2/Rjmv1SBrUTyGJ4kVvmFq1If+FTqYw40zuqiFv3VOtKCg7FKQKnsS3pAX
	5F+k=
X-Google-Smtp-Source: AGHT+IGpbR8qQpWJhSYJH93pbLDUKTmP1Mf1+5HUrKKu6ErvFgLI74jMTr42QIaJ6HBcUl8sMDuvhw==
X-Received: by 2002:a17:906:c0e:b0:a62:49ba:8242 with SMTP id a640c23a62f3a-a62651118admr194060366b.77.1716581029839;
        Fri, 24 May 2024 13:03:49 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general case
Date: Fri, 24 May 2024 21:03:32 +0100
Message-Id: <20240524200338.1232391-8-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The asm in arch_ffs() is safe but inefficient.

CMOV would be an improvement over a conditional branch, but for 64bit CPUs
both Intel and AMD have provided enough details about the behaviour for a zero
input.  It is safe to pre-load the destination register with -1 and drop the
conditional logic.

However, it is common to find ffs() in a context where the optimiser knows
that x in nonzero even if it the value isn't known precisely, and in that case
it's safe to drop the preload of -1 too.

There are only a handful of uses of ffs() in the x86 build, and all of them
improve as a result of this:

  add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-31 (-31)
  Function                                     old     new   delta
  mask_write                                   114     107      -7
  xmem_pool_alloc                             1063    1039     -24

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New.
 * Use __builtin_constant_p(x > 0) to optimise better.
---
 xen/arch/x86/include/asm/bitops.h | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 122767fc0d10..1d7aea6065ef 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
 
 static always_inline unsigned int arch_ffs(unsigned int x)
 {
-    int r;
+    unsigned int r;
+
+    if ( __builtin_constant_p(x > 0) && x > 0 )
+    {
+        /* Safe, when the compiler knows that x is nonzero. */
+        asm ( "bsf %[val], %[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x) );
+    }
+    else
+    {
+        /*
+         * The AMD manual states that BSF won't modify the destination
+         * register if x=0.  The Intel manual states that the result is
+         * undefined, but the architects have said that the register is
+         * written back with it's old value (zero extended as normal).
+         */
+        asm ( "bsf %[val], %[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x), "[res]" (-1) );
+    }
 
-    asm ( "bsf %1,%0\n\t"
-          "jnz 1f\n\t"
-          "mov $-1,%0\n"
-          "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
 #define arch_ffs arch_ffs
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729751.1135059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8r-0000Ot-HW; Fri, 24 May 2024 20:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729751.1135059; Fri, 24 May 2024 20:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8r-0000Nd-Ce; Fri, 24 May 2024 20:03:53 +0000
Received: by outflank-mailman (input) for mailman id 729751;
 Fri, 24 May 2024 20:03:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8q-000773-2L
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:52 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bcd8def2-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:50 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-578517c7ae9so1585194a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:50 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcd8def2-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581029; x=1717185829; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kIl6ox/sphrOz6UFI6xM1xqU962sO2kBKKT4aLgRbJg=;
        b=hZfnLyQsdBdRA5tWILs1WM3bIsDtlcwqINDLfI4JpCMKKtsBWOAFSiOlMgaRUZSDQH
         FzrqPEa8yMfRKCm1Nbjc96gv11s4VpsjugV0pM3l6QC8wLYo/aUL1VzHxN/qmpm2BZjB
         anmgeeG0KCXFRZReYOZ+PAdHzExgfNo4lQzI4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581029; x=1717185829;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kIl6ox/sphrOz6UFI6xM1xqU962sO2kBKKT4aLgRbJg=;
        b=n62X7eyHznajFpCF94oATfA9z43bUfDvB2VV1IfFo3sbnQNaGNco1KqYVuWHfHrWpJ
         9/nHvBKjZ61NHD29ZHCuGFdODQCgE6hrmnsMIkGK5zBtEk26jhbFpntgmnYDEXOIGELP
         izZglrg3acJxtNVtjhEWQ5BnYwcOA71cQeGAbC+KrjB/ertbJuEexfFohvnWbat+ot2p
         ih/hVL0BH5jdPvnkjrXpC4wvF6jNXnKd9+ShKXdSNQoOFIs+mtOUlMiA3RCw0lqTJtRW
         674VyyIdG5M7LQXyuLXS1xlbkQcSMMjwaSDeDOdU9QKoTjmIykixMmHnXo+hwQ5ssfXL
         C5Pg==
X-Gm-Message-State: AOJu0Yz/fOwOt5blhYpg9wQjoEsODu8o6AdC9wS8ybYoqHXfK2VphgAq
	U/YknOlKy69ymLRiC2qUdxZojAKyP6ONyW0q1fffwm30fTzjv0eO2DOH2POdSa2aq41Z3Js6Wp2
	X3VA=
X-Google-Smtp-Source: AGHT+IEUVNV4EpZfmOgPpyYnZ5/OwkBVDwrdSNszXAXbvrntU8FqgakDCrtsvjr4fA673YXJNwZzFw==
X-Received: by 2002:a17:907:7844:b0:a5e:612e:fd58 with SMTP id a640c23a62f3a-a6264f126demr219304566b.51.1716581029010;
        Fri, 24 May 2024 13:03:49 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
Date: Fri, 24 May 2024 21:03:31 +0100
Message-Id: <20240524200338.1232391-7-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Perform constant-folding unconditionally, rather than having it implemented
inconsistency between architectures.

Confirm the expected behaviour with compile time and boot time tests.

For non-constant inputs, use arch_ffs() if provided but fall back to
generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin that
works in all configurations.

For x86, rename ffs() to arch_ffs() and adjust the prototype.

For PPC, __builtin_ctz() is 1/3 of the size of size of the transform to
generic_fls().  Drop the definition entirely.  ARM too benefits in the general
case by using __builtin_ctz(), but less dramatically because it using
optimised asm().

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Fall back to generic, not builtin.
 * Extend the testing with multi-bit values.
 * Use always_inline for x86
 * Defer x86 optimisation to a later change
---
 xen/arch/arm/include/asm/bitops.h |  2 +-
 xen/arch/ppc/include/asm/bitops.h |  2 +-
 xen/arch/x86/include/asm/bitops.h |  3 ++-
 xen/common/Makefile               |  1 +
 xen/common/bitops.c               | 19 +++++++++++++++++++
 xen/include/xen/bitops.h          | 17 +++++++++++++++++
 6 files changed, 41 insertions(+), 3 deletions(-)
 create mode 100644 xen/common/bitops.c

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index ec1cf7b9b323..a88ec2612e16 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -157,7 +157,7 @@ static inline int fls(unsigned int x)
 }
 
 
-#define ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
+#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
 
 /**
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index ab692d01717b..5c36a6cc0ce3 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -173,7 +173,7 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
 
 #define flsl(x) generic_flsl(x)
 #define fls(x) generic_flsl(x)
-#define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
+#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
 
 /**
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 5a71afbc89d5..122767fc0d10 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -430,7 +430,7 @@ static inline int ffsl(unsigned long x)
     return (int)r+1;
 }
 
-static inline int ffs(unsigned int x)
+static always_inline unsigned int arch_ffs(unsigned int x)
 {
     int r;
 
@@ -440,6 +440,7 @@ static inline int ffs(unsigned int x)
           "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_ffs arch_ffs
 
 /**
  * fls - find last bit set
diff --git a/xen/common/Makefile b/xen/common/Makefile
index d512cad5243f..21a4fb4c7166 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -1,5 +1,6 @@
 obj-$(CONFIG_ARGO) += argo.o
 obj-y += bitmap.o
+obj-bin-$(CONFIG_DEBUG) += bitops.init.o
 obj-$(CONFIG_GENERIC_BUG_FRAME) += bug.o
 obj-$(CONFIG_HYPFS_CONFIG) += config_data.o
 obj-$(CONFIG_CORE_PARKING) += core_parking.o
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
new file mode 100644
index 000000000000..8c161b8ea7fa
--- /dev/null
+++ b/xen/common/bitops.c
@@ -0,0 +1,19 @@
+#include <xen/bitops.h>
+#include <xen/boot-check.h>
+#include <xen/init.h>
+
+static void __init test_ffs(void)
+{
+    /* unsigned int ffs(unsigned int) */
+    CHECK(ffs, 0, 0);
+    CHECK(ffs, 1, 1);
+    CHECK(ffs, 3, 1);
+    CHECK(ffs, 7, 1);
+    CHECK(ffs, 6, 2);
+    CHECK(ffs, 0x80000000U, 32);
+}
+
+static void __init __constructor test_bitops(void)
+{
+    test_ffs();
+}
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index cd405df96180..f7e90a2893a5 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -31,6 +31,23 @@ unsigned int __pure generic_flsl(unsigned long x);
 
 #include <asm/bitops.h>
 
+/*
+ * Find First/Last Set bit (all forms).
+ *
+ * Bits are labelled from 1.  Returns 0 if given 0.
+ */
+static always_inline __pure unsigned int ffs(unsigned int x)
+{
+    if ( __builtin_constant_p(x) )
+        return __builtin_ffs(x);
+
+#ifdef arch_ffs
+    return arch_ffs(x);
+#else
+    return generic_ffsl(x);
+#endif
+}
+
 /* --------------------- Please tidy below here --------------------- */
 
 #ifndef find_next_bit
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729752.1135070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8t-0000mB-61; Fri, 24 May 2024 20:03:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729752.1135070; Fri, 24 May 2024 20:03:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8t-0000lM-0c; Fri, 24 May 2024 20:03:55 +0000
Received: by outflank-mailman (input) for mailman id 729752;
 Fri, 24 May 2024 20:03:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8r-000773-GY
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:53 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bdc52165-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:51 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-57863a8f4b2so697576a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:51 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdc52165-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581031; x=1717185831; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nXTYEDY4WICxpm22K4ZAGRtYdV5nmr0pV6oeY/cb0HI=;
        b=svZVCs81BDol0Nqi7hrpXxJqPkVkkSFCmLJJCc240DBF13QxqFCT7f2N51das48CRM
         BbWvagSsw/7+U+vxw8jB0vGF3Q2B7mVPq6kbx3V/zkV0X7El6ej/lOWYhnA2S7RsZVfF
         gPSWHE+9TWy2IOXRQQGSAFGdSDUNTxwBhI0dU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581031; x=1717185831;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nXTYEDY4WICxpm22K4ZAGRtYdV5nmr0pV6oeY/cb0HI=;
        b=dndMvJkb8jP+dmoEXIe3+if+gEbmR1+gIzYkdz8NYkuEZdMGCbkVOQcry4lwX35ZMh
         0niO8G/Nw/0Q4O8vM70uOtyLDa0g/8nuygLKRLEA0KP0cY+ivhZuNUkiR0Wfhd0BYGDX
         GB21mfzzAw5swWjfqkiNJU+6ItGVFhuWRySlUtY97ohlNHlN37G94iSodczeFWuWLvlc
         lYkZJnAuj0CAPJH856yWskG0MLWKn7M8VrxkvKeId9j1RQFW/FwDU82+M62JjvnagXDv
         DNl7Ihc0HK4sL3lXl7HJZAkCXXHDgUdqzuf7TJjEEsS+FdnD979yoLTScY3bMQwZsTjb
         yDIA==
X-Gm-Message-State: AOJu0YyCA6Kw/734pZruKaeKrRnDn3WLtQDOzYZ0qWt9twzvRVd4Uk+8
	QRytzUCmLRLGPfNebGt8WC94gqfg8DwA8/dcc0oNw7Ajf1PFZfoDPiiDsEyV/Onup3eozdJ9LRm
	wC4k=
X-Google-Smtp-Source: AGHT+IFb3M2wCqEZnAg0P7Oi7/XtKFSPQytPacHnXNOWFHuZBirGiXxcH2/y/aqny8HR74UaaGekqQ==
X-Received: by 2002:a17:906:3ed1:b0:a59:c23d:85d8 with SMTP id a640c23a62f3a-a6264f15e2cmr222340566b.51.1716581031018;
        Fri, 24 May 2024 13:03:51 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 08/13] xen/bitops: Implement ffsl() in common logic
Date: Fri, 24 May 2024 21:03:33 +0100
Message-Id: <20240524200338.1232391-9-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Just like ffs() in the previous changes.  Express the upper bound of the
testing in terms of BITS_PER_LONG as it varies between architectures.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Swap to #if BITS_PER_LONG > 32 to avoid a compile error on arm32
 * Changes to mirror ffs() v2.
---
 xen/arch/arm/include/asm/bitops.h |  2 +-
 xen/arch/ppc/include/asm/bitops.h |  2 +-
 xen/arch/x86/include/asm/bitops.h | 35 ++++++++++++++++---------------
 xen/common/bitops.c               | 13 ++++++++++++
 xen/include/xen/bitops.h          | 12 +++++++++++
 5 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index a88ec2612e16..ba39802c9de3 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -158,7 +158,7 @@ static inline int fls(unsigned int x)
 
 
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
-#define ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
+#define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 
 /**
  * find_first_set_bit - find the first set bit in @word
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 5c36a6cc0ce3..ce0f6436f727 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -174,7 +174,7 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
 #define flsl(x) generic_flsl(x)
 #define fls(x) generic_flsl(x)
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
-#define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
+#define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 1d7aea6065ef..51d3c0f40473 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -413,23 +413,6 @@ static inline unsigned int find_first_set_bit(unsigned long word)
     return (unsigned int)word;
 }
 
-/**
- * ffs - find first bit set
- * @x: the word to search
- *
- * This is defined the same way as the libc and compiler builtin ffs routines.
- */
-static inline int ffsl(unsigned long x)
-{
-    long r;
-
-    asm ( "bsf %1,%0\n\t"
-          "jnz 1f\n\t"
-          "mov $-1,%0\n"
-          "1:" : "=r" (r) : "rm" (x));
-    return (int)r+1;
-}
-
 static always_inline unsigned int arch_ffs(unsigned int x)
 {
     unsigned int r;
@@ -458,6 +441,24 @@ static always_inline unsigned int arch_ffs(unsigned int x)
 }
 #define arch_ffs arch_ffs
 
+static always_inline unsigned int arch_ffsl(unsigned long x)
+{
+    unsigned int r;
+
+    /* See arch_ffs() for safety discussions. */
+    if ( __builtin_constant_p(x > 0) && x > 0 )
+        asm ( "bsf %[val], %q[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x) );
+    else
+        asm ( "bsf %[val], %q[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x), "[res]" (-1) );
+
+    return r + 1;
+}
+#define arch_ffsl arch_ffsl
+
 /**
  * fls - find last bit set
  * @x: the word to search
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index 8c161b8ea7fa..b3813f818198 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -11,6 +11,19 @@ static void __init test_ffs(void)
     CHECK(ffs, 7, 1);
     CHECK(ffs, 6, 2);
     CHECK(ffs, 0x80000000U, 32);
+
+    /* unsigned int ffsl(unsigned long) */
+    CHECK(ffsl, 0, 0);
+    CHECK(ffsl, 1, 1);
+    CHECK(ffsl, 3, 1);
+    CHECK(ffsl, 7, 1);
+    CHECK(ffsl, 6, 2);
+
+    CHECK(ffsl, 1UL << (BITS_PER_LONG - 1), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    CHECK(ffsl, 1UL << 32, 33);
+    CHECK(ffsl, 1UL << 63, 64);
+#endif
 }
 
 static void __init __constructor test_bitops(void)
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index f7e90a2893a5..88cf27a88bcf 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -48,6 +48,18 @@ static always_inline __pure unsigned int ffs(unsigned int x)
 #endif
 }
 
+static always_inline __pure unsigned int ffsl(unsigned long x)
+{
+    if ( __builtin_constant_p(x) )
+        return __builtin_ffsl(x);
+
+#ifdef arch_ffs
+    return arch_ffsl(x);
+#else
+    return generic_ffsl(x);
+#endif
+}
+
 /* --------------------- Please tidy below here --------------------- */
 
 #ifndef find_next_bit
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729753.1135079 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8u-00012E-Go; Fri, 24 May 2024 20:03:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729753.1135079; Fri, 24 May 2024 20:03:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8u-00011V-Ay; Fri, 24 May 2024 20:03:56 +0000
Received: by outflank-mailman (input) for mailman id 729753;
 Fri, 24 May 2024 20:03:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8t-000773-3P
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:55 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id be7c9349-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:53 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so1184828466b.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:53 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be7c9349-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581032; x=1717185832; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=h4H8slafbFD5MZ8pWJsbqpEr4l2E5HCz8G8Qyr9VoPE=;
        b=Z/CNqc6CpHqjwoOREq64r09/7yOcRxQblHLWOt3tTFmMwrrAovbMMUuBr++ejXxYid
         b1dzew5fMHnG9wWnUp2aRVjKUCkb4ZodzX9hyDvYTsu9U0Ua5dxJdNVJUkuTXmKm/R28
         NQdRB7Pz1GHhLIX6A1t0V1XGyEYqHa2nELA4U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581032; x=1717185832;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=h4H8slafbFD5MZ8pWJsbqpEr4l2E5HCz8G8Qyr9VoPE=;
        b=bkZ3C3IGJqRn91vfuYFBKZ9ukstM+WR/Lfd6aZ+53s4xTOWEJ52S63SLOyz6A5K1o3
         s/YfFK0croI5/ceyLU/dy28e1HE2+Cl1EiYUpc5Xa8uHDFn3GSXS1ZYVmsDI8JAaBM8/
         fL6OnG8WFWzLz3lqvUZyWTRDL/5uG5AHPXNdGG0s1NEwIhIbYI+wMJcCs8RQ6WFTl+50
         BxqTey3TbeUZHdFUCzEwGgG3vwtWe4F6mDUMukYcCX1REKm2yY9+dH8T44qtuOyp8HV6
         KW3QwLniDimRsD1BRnuxj8hpMdD98coPxwKbwLXfzqHSqUSjVmv8FPdxdBNgbxpKcAHT
         QQ5w==
X-Gm-Message-State: AOJu0Yzi30Zlvj5NO8+YUs2AG3ydCzjF0t+qQsM1TRIyE++HAK+AZAOp
	CtnlhDU/2bG0W3pEbNB7okkr1cx1qMrBQisbLO7BgFgQA/hi1JQI6Lsevyi8v9Mgv4EaPgoauaF
	O+xs=
X-Google-Smtp-Source: AGHT+IEFnNtUkkX+5j+8K47eW4QSiYZz+ttxOCsa+8nNZwxQCIEI2xbVmGmD+ZqUVhiUQomAbDPrRg==
X-Received: by 2002:a17:906:c214:b0:a5a:8b8c:6203 with SMTP id a640c23a62f3a-a62646d7f32mr215106166b.45.1716581032090;
        Fri, 24 May 2024 13:03:52 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 09/13] xen/bitops: Replace find_first_set_bit() with ffsl() - 1
Date: Fri, 24 May 2024 21:03:34 +0100
Message-Id: <20240524200338.1232391-10-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

find_first_set_bit() is a Xen-ism which has undefined behaviour with a 0
input.  The latter is well defined with an input of 0, and is a found outside
of Xen too.

_init_heap_pages() is the one special case here, comparing the LSB of two
different addresses.  The -1 cancels off both sides of the expression.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Reorder from later in the series to keep ARM bisectable

In an x86 build, we get the following delta:

  add/remove: 0/0 grow/shrink: 2/4 up/down: 39/-52 (-13)
  Function                                     old     new   delta
  hpet_write                                  2183    2206     +23
  init_heap_pages                             1222    1238     +16
  dom0_construct_pvh                          3959    3958      -1
  mapping_order                                139     126     -13
  guest_physmap_mark_populate_on_demand       1301    1285     -16
  vcpumask_to_pcpumask                         525     503     -22

so the optimiser improvements for ffsl() really do speak for themselves.

I'm surprised by the increase in hpet_write(), but looking at the code, it
very clearly wants the same treatment as:

  commit 188fa82305e72b725473db9146e20cc9abf7bff3
  Author: Andrew Cooper <andrew.cooper3@citrix.com>
  Date:   Fri Mar 15 11:31:33 2024

      xen/vpci: Improve code generation in mask_write()

which I'm confident will end up as a net improvement.
---
 xen/arch/x86/guest/xen/xen.c                 | 4 ++--
 xen/arch/x86/hvm/dom0_build.c                | 2 +-
 xen/arch/x86/hvm/hpet.c                      | 8 ++++----
 xen/arch/x86/include/asm/pt-contig-markers.h | 2 +-
 xen/arch/x86/mm.c                            | 2 +-
 xen/arch/x86/mm/p2m-pod.c                    | 4 ++--
 xen/common/page_alloc.c                      | 2 +-
 xen/common/softirq.c                         | 2 +-
 xen/drivers/passthrough/amd/iommu_map.c      | 2 +-
 xen/drivers/passthrough/iommu.c              | 4 ++--
 xen/drivers/passthrough/x86/iommu.c          | 4 ++--
 11 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c
index d9768cc9527d..7484b3f73ad3 100644
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -168,14 +168,14 @@ static void cf_check xen_evtchn_upcall(void)
 
     while ( pending )
     {
-        unsigned int l1 = find_first_set_bit(pending);
+        unsigned int l1 = ffsl(pending) - 1;
         unsigned long evtchn = xchg(&XEN_shared_info->evtchn_pending[l1], 0);
 
         __clear_bit(l1, &pending);
         evtchn &= ~XEN_shared_info->evtchn_mask[l1];
         while ( evtchn )
         {
-            unsigned int port = find_first_set_bit(evtchn);
+            unsigned int port = ffsl(evtchn) - 1;
 
             __clear_bit(port, &evtchn);
             port += l1 * BITS_PER_LONG;
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index b0cb96c3bc76..68c08bbe94f7 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -139,7 +139,7 @@ static int __init pvh_populate_memory_range(struct domain *d,
         order = get_order_from_pages(end - start + 1);
         order = min(order ? order - 1 : 0, max_order);
         /* The order allocated and populated must be aligned to the address. */
-        order = min(order, start ? find_first_set_bit(start) : MAX_ORDER);
+        order = min(order, start ? ffsl(start) - 1 : MAX_ORDER);
         page = alloc_domheap_pages(d, order, dom0_memflags | MEMF_no_scrub);
         if ( page == NULL )
         {
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 12b00b770257..37e765e97df9 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -335,7 +335,7 @@ static void timer_sanitize_int_route(HPETState *h, unsigned int tn)
      * enabled pick the first irq.
      */
     timer_config(h, tn) |=
-        MASK_INSR(find_first_set_bit(timer_int_route_cap(h, tn)),
+        MASK_INSR(ffsl(timer_int_route_cap(h, tn)) - 1,
                   HPET_TN_ROUTE);
 }
 
@@ -409,7 +409,7 @@ static int cf_check hpet_write(
         {
             bool active;
 
-            i = find_first_set_bit(new_val);
+            i = ffsl(new_val) - 1;
             if ( i >= HPET_TIMER_NUM )
                 break;
             __clear_bit(i, &new_val);
@@ -535,14 +535,14 @@ static int cf_check hpet_write(
     /* stop/start timers whos state was changed by this write. */
     while (stop_timers)
     {
-        i = find_first_set_bit(stop_timers);
+        i = ffsl(stop_timers) - 1;
         __clear_bit(i, &stop_timers);
         hpet_stop_timer(h, i, guest_time);
     }
 
     while (start_timers)
     {
-        i = find_first_set_bit(start_timers);
+        i = ffsl(start_timers) - 1;
         __clear_bit(i, &start_timers);
         hpet_set_timer(h, i, guest_time);
     }
diff --git a/xen/arch/x86/include/asm/pt-contig-markers.h b/xen/arch/x86/include/asm/pt-contig-markers.h
index b3c1fe803534..e8c8157d605f 100644
--- a/xen/arch/x86/include/asm/pt-contig-markers.h
+++ b/xen/arch/x86/include/asm/pt-contig-markers.h
@@ -60,7 +60,7 @@ static bool pt_update_contig_markers(uint64_t *pt, unsigned int idx,
     /* Step 1: Reduce markers in lower numbered entries. */
     while ( i )
     {
-        b = find_first_set_bit(i);
+        b = ffsl(i) - 1;
         i &= ~(1U << b);
         if ( GET_MARKER(pt[i]) <= b )
             break;
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc7315..2a84bdae670b 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -3424,7 +3424,7 @@ static int vcpumask_to_pcpumask(
         {
             unsigned int cpu;
 
-            vcpu_id = find_first_set_bit(vmask);
+            vcpu_id = ffsl(vmask) - 1;
             vmask &= ~(1UL << vcpu_id);
             vcpu_id += vcpu_bias;
             if ( (vcpu_id >= d->max_vcpus) )
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 1c727c78c833..bd84fe9e27ee 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -684,7 +684,7 @@ unsigned long
 p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, unsigned int order)
 {
     unsigned long left = 1UL << order, ret = 0;
-    unsigned int chunk_order = find_first_set_bit(gfn_x(gfn) | left);
+    unsigned int chunk_order = ffsl(gfn_x(gfn) | left) - 1;
 
     do {
         ret += decrease_reservation(d, gfn, chunk_order);
@@ -1393,7 +1393,7 @@ guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,
                                       unsigned int order)
 {
     unsigned long left = 1UL << order;
-    unsigned int chunk_order = find_first_set_bit(gfn | left);
+    unsigned int chunk_order = ffsl(gfn | left) - 1;
     int rc;
 
     if ( !paging_mode_translate(d) )
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 8d3342e95236..054b7edb3989 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1819,7 +1819,7 @@ static void _init_heap_pages(const struct page_info *pg,
     if ( unlikely(!avail[nid]) )
     {
         bool use_tail = IS_ALIGNED(s, 1UL << MAX_ORDER) &&
-                        (find_first_set_bit(e) <= find_first_set_bit(s));
+                        (ffsl(e) <= ffsl(s));
         unsigned long n;
 
         n = init_node_heap(nid, s, nr_pages, &use_tail);
diff --git a/xen/common/softirq.c b/xen/common/softirq.c
index 321d26902d37..bee4a82009c3 100644
--- a/xen/common/softirq.c
+++ b/xen/common/softirq.c
@@ -48,7 +48,7 @@ static void __do_softirq(unsigned long ignore_mask)
              || cpu_is_offline(cpu) )
             break;
 
-        i = find_first_set_bit(pending);
+        i = ffsl(pending) - 1;
         clear_bit(i, &softirq_pending(cpu));
         (*softirq_handlers[i])();
     }
diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c
index e0f4fe736a8d..f1061bfc798c 100644
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -137,7 +137,7 @@ static void set_iommu_ptes_present(unsigned long pt_mfn,
         ASSERT(!pde->u);
 
         if ( pde > table )
-            ASSERT(pde->ign0 == find_first_set_bit(pde - table));
+            ASSERT(pde->ign0 == ffsl(pde - table) - 1);
         else
             ASSERT(pde->ign0 == CONTIG_LEVEL_SHIFT);
 
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index ba18136c461c..50bfd62553ae 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -305,7 +305,7 @@ static unsigned int mapping_order(const struct domain_iommu *hd,
 {
     unsigned long res = dfn_x(dfn) | mfn_x(mfn);
     unsigned long sizes = hd->platform_ops->page_sizes;
-    unsigned int bit = find_first_set_bit(sizes), order = 0;
+    unsigned int bit = ffsl(sizes) - 1, order = 0;
 
     ASSERT(bit == PAGE_SHIFT);
 
@@ -313,7 +313,7 @@ static unsigned int mapping_order(const struct domain_iommu *hd,
     {
         unsigned long mask;
 
-        bit = find_first_set_bit(sizes);
+        bit = ffsl(sizes) - 1;
         mask = (1UL << bit) - 1;
         if ( nr <= mask || (res & mask) )
             break;
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 304a2f5480c7..cc0062b02712 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -641,7 +641,7 @@ struct page_info *iommu_alloc_pgtable(struct domain_iommu *hd,
     if ( contig_mask )
     {
         /* See pt-contig-markers.h for a description of the marker scheme. */
-        unsigned int i, shift = find_first_set_bit(contig_mask);
+        unsigned int i, shift = ffsl(contig_mask) - 1;
 
         ASSERT((CONTIG_LEVEL_SHIFT & (contig_mask >> shift)) == CONTIG_LEVEL_SHIFT);
 
@@ -652,7 +652,7 @@ struct page_info *iommu_alloc_pgtable(struct domain_iommu *hd,
 
         for ( i = 4; i < PAGE_SIZE / sizeof(*p); i += 4 )
         {
-            p[i + 0] = (find_first_set_bit(i) + 0ULL) << shift;
+            p[i + 0] = (ffsl(i) - 1ULL) << shift;
             p[i + 1] = 0;
             p[i + 2] = 1ULL << shift;
             p[i + 3] = 0;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729754.1135084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8v-00019e-DS; Fri, 24 May 2024 20:03:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729754.1135084; Fri, 24 May 2024 20:03:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8u-000199-Um; Fri, 24 May 2024 20:03:56 +0000
Received: by outflank-mailman (input) for mailman id 729754;
 Fri, 24 May 2024 20:03:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8t-000773-H2
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:55 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bef910bf-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:53 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a626ac4d299so130919566b.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:53 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bef910bf-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581033; x=1717185833; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=maIVB65jYfI2PBFJy7D4AeslonYmTipzI0EBcOkzwDs=;
        b=sSaqMABUIPyLRyf6r687lQ1thueAYWaRJs9DlJbq1+yOmHENekFQ5FZxIjxyrcfHNx
         KALPRB72LhM1GdiCsX6lpwPPn34KK8kfSogSWTy0sVhA5Ah1KC036viIFktdANOwxTvZ
         rZ6kbdfncTnc3a4aG8YxUuLnB4r0gCDQR4NLQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581033; x=1717185833;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=maIVB65jYfI2PBFJy7D4AeslonYmTipzI0EBcOkzwDs=;
        b=JdUk45ey20tgqN1zew/P3VN/7TswOr+gKjxheoc5R10b/XI1Qyn05qF3vs8kSf1/7y
         jqweRteMzvxiaKoyoMJWfgK7neDZnOTzezNFKzIhw+KdqOZoLRpESnjjz0FXr94+40Vh
         R2RUptBq2dQfa/7rTgIDagcHJfV24ei/bgwU9Ui+32m0DoCxW1PwoSRMD9B0oKsnoCka
         wEz+b3c3CvdgSmijIghoOLqPUUFvG6Wtt6BAKUaECU6aUzs2uitbwo9OxK6VIuzDJdYO
         1eR73iuqXU3KNE5iA1DYqyVhUAGuYbb5YCxQ3W0ErrsWTBzA8upIVq9VlyTJUA/EvweM
         J0Jw==
X-Gm-Message-State: AOJu0Yx0CtKp+Ohfix2xZqWEaJDjAWk2ruhfkPAp33QdSBuZ3vianH1L
	ASKNj955FW7FgF/RIy6SnIYSCOP4XGxlj+uHKOXpmvQFwOnSvaRqSw0eMhf0CxG83/ghGMvDOlu
	v+yU=
X-Google-Smtp-Source: AGHT+IHQSVGsB2hFnOlOMuOQReZaremOY3IsH9dioCeOBypWLpouROaY+1nCR6o1hiTm3xLeTealKA==
X-Received: by 2002:a17:906:c44c:b0:a5a:3a6c:8b56 with SMTP id a640c23a62f3a-a6264179f89mr213603866b.11.1716581033309;
        Fri, 24 May 2024 13:03:53 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 10/13] xen/bitops: Delete find_first_set_bit()
Date: Fri, 24 May 2024 21:03:35 +0100
Message-Id: <20240524200338.1232391-11-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

No more users.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Reorder from later in the series to keep ARM bisectable
---
 xen/arch/arm/include/asm/bitops.h |  9 ---------
 xen/arch/ppc/include/asm/bitops.h |  9 ---------
 xen/arch/x86/include/asm/bitops.h | 12 ------------
 3 files changed, 30 deletions(-)

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index ba39802c9de3..d30ba44598e3 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -160,15 +160,6 @@ static inline int fls(unsigned int x)
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 
-/**
- * find_first_set_bit - find the first set bit in @word
- * @word: the word to search
- *
- * Returns the bit-number of the first set bit (first bit being 0).
- * The input must *not* be zero.
- */
-#define find_first_set_bit(w) (ffsl(w) - 1)
-
 /**
  * hweightN - returns the hamming weight of a N-bit word
  * @x: the word to weigh
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index ce0f6436f727..761361291e6f 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -187,13 +187,4 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
 #define hweight16(x) __builtin_popcount((uint16_t)(x))
 #define hweight8(x)  __builtin_popcount((uint8_t)(x))
 
-/**
- * find_first_set_bit - find the first set bit in @word
- * @word: the word to search
- *
- * Returns the bit-number of the first set bit (first bit being 0).
- * The input must *not* be zero.
- */
-#define find_first_set_bit(x) (ffsl(x) - 1)
-
 #endif /* _ASM_PPC_BITOPS_H */
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 51d3c0f40473..830e488f33a0 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -401,18 +401,6 @@ static always_inline unsigned int __scanbit(unsigned long val, unsigned int max)
     r__;                                                                    \
 })
 
-/**
- * find_first_set_bit - find the first set bit in @word
- * @word: the word to search
- * 
- * Returns the bit-number of the first set bit. The input must *not* be zero.
- */
-static inline unsigned int find_first_set_bit(unsigned long word)
-{
-    asm ( "rep; bsf %1,%0" : "=r" (word) : "rm" (word) );
-    return (unsigned int)word;
-}
-
 static always_inline unsigned int arch_ffs(unsigned int x)
 {
     unsigned int r;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729755.1135094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8x-0001ZF-21; Fri, 24 May 2024 20:03:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729755.1135094; Fri, 24 May 2024 20:03:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8w-0001XB-Qw; Fri, 24 May 2024 20:03:58 +0000
Received: by outflank-mailman (input) for mailman id 729755;
 Fri, 24 May 2024 20:03:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8u-000773-TT
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:56 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf9fe58b-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:55 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f464so1184464a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:55 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf9fe58b-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581034; x=1717185834; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BArFVTq1bFwBqGRKgiR15fKQGJoXMQ04ywKksSWcrsw=;
        b=nce6VDKq+A7CD02EOSHb4AOw7H264K5pwaW25ZnM3aasO4mUoSAUz8Xg99jXVkyUxp
         WdkxEvouGhx14pxxNTTC8t04OUh1F9nXBwY8fJCNs7o2bKnqcxYs6sr2/HBQtR5pX+YN
         ZJNCMx6+UW+5+I0MAbZhtfSg3fgtSUGd8oApc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581034; x=1717185834;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=BArFVTq1bFwBqGRKgiR15fKQGJoXMQ04ywKksSWcrsw=;
        b=UvIwFnqnUxtHAvnJomBst7ofJMhQJkwzq4OVPaHkqVXi0NHczF7JGLoeWvIVOKEXBF
         3PP1+JzwVDkmuI/VZAKbnmX8OCcLkhy7TYkV4NMjjhOQyIqNgdIv7oXlzDTD/05vfSjD
         y3fAgiz9usW++eb0Yw8GdjFXP4gB/PxQPfvleqQMFIJiftoBDu6tnMSTDj+DBM8IvmkO
         cCk5X5WPLq0UOTedsCtj++g4lxpVOHgXJo6FEQaEJ9VnpE1IHUmyG5UxJhyvPjSwEJTl
         u1Qp0ZBj9HP6uhITwRCvba08Lxw2ms5KprDFcflhSE5XhRqaG+Shny8uNm92CzqHF+sJ
         BwoQ==
X-Gm-Message-State: AOJu0YwtlVGiqB/Nei71T2EuusE7moY+qYMnJ/n7ZjlR5C6VP69w2BOU
	J9rsT/t39zGJVVe4x/RuPjPHEgdS+WdV2fGju8o7zCx4n4JJGtO01BAzNNY/3ZqSoD9/9X24V8r
	/0j4=
X-Google-Smtp-Source: AGHT+IFS0sRKjwwnSdc3AQTMI7Dh0SyIHhDvuOE50/zkVG/FaN3PWKT6+IRAPUcfHZtNjlNVcg8Ydg==
X-Received: by 2002:a17:907:6d07:b0:a58:c80e:edd9 with SMTP id a640c23a62f3a-a626525da3bmr218463666b.77.1716581034101;
        Fri, 24 May 2024 13:03:54 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 11/13] xen/bitops: Implement fls()/flsl() in common logic
Date: Fri, 24 May 2024 21:03:36 +0100
Message-Id: <20240524200338.1232391-12-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Oleksii Kurochko <oleksii.kurochko@gmail.com>

This is most easily done together because of how arm32 is currently
structured, but it does just mirror the existing ffs()/ffsl() work.

Introduce compile and boot time testing.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New, incorperated from Oleksii's RISC-V series and adjusted.

for x86:

  add/remove: 0/0 grow/shrink: 3/17 up/down: 28/-153 (-125)
  Function                                     old     new   delta
  pci_enable_msi                              1033    1049     +16
  vlapic_lowest_prio                           330     338      +8
  kexec_early_calculations                      53      57      +4
  pci_restore_msi_state                       1159    1157      -2
  arch_hwdom_irqs                               61      59      -2
  control_read                                 132     129      -3
  pci_enable_msi.cold                          121     117      -4
  arch_get_dma_bitsize                         173     169      -4
  xmem_pool_alloc                             1039    1032      -7
  xenheap_max_mfn                               49      42      -7
  mba_sanitize_thrtl                            83      76      -7
  xstate_init                                  807     799      -8
  offline_page                                 965     957      -8
  apicid_to_socket                             160     152      -8
  vlapic_find_highest_vector                    61      48     -13
  xmem_pool_free                               983     967     -16
  iommu_alloc                                  935     919     -16
  free_heap_pages                             1512    1496     -16
  detect_ht                                    318     302     -16
  alloc_heap_pages                            1569    1553     -16

showing that the optimiser can now do a better job in most cases.
---
 xen/arch/arm/include/asm/arm32/bitops.h |  2 --
 xen/arch/arm/include/asm/arm64/bitops.h | 12 -------
 xen/arch/arm/include/asm/bitops.h       | 19 ++--------
 xen/arch/ppc/include/asm/bitops.h       |  4 +--
 xen/arch/x86/include/asm/bitops.h       | 46 +++++++++++++++----------
 xen/common/bitops.c                     | 25 ++++++++++++++
 xen/include/xen/bitops.h                | 24 +++++++++++++
 7 files changed, 80 insertions(+), 52 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/bitops.h b/xen/arch/arm/include/asm/arm32/bitops.h
index d0309d47c188..0d7bb12d5c19 100644
--- a/xen/arch/arm/include/asm/arm32/bitops.h
+++ b/xen/arch/arm/include/asm/arm32/bitops.h
@@ -1,8 +1,6 @@
 #ifndef _ARM_ARM32_BITOPS_H
 #define _ARM_ARM32_BITOPS_H
 
-#define flsl fls
-
 /*
  * Little endian assembly bitops.  nr = 0 -> byte 0 bit 0.
  */
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 906d84e5f295..a6135838dcfa 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -1,18 +1,6 @@
 #ifndef _ARM_ARM64_BITOPS_H
 #define _ARM_ARM64_BITOPS_H
 
-static inline int flsl(unsigned long x)
-{
-        uint64_t ret;
-
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
-        asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
-
-        return BITS_PER_LONG - ret;
-}
-
 /* Based on linux/include/asm-generic/bitops/find.h */
 
 #ifndef CONFIG_GENERIC_FIND_FIRST_BIT
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index d30ba44598e3..8f4bdc09d128 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -140,25 +140,10 @@ static inline int test_bit(int nr, const volatile void *addr)
         return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
 }
 
-/*
- * On ARMv5 and above those functions can be implemented around
- * the clz instruction for much better code efficiency.
- */
-
-static inline int fls(unsigned int x)
-{
-        int ret;
-
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
-        asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
-        return 32 - ret;
-}
-
-
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
+#define arch_fls(x)  ((x) ? 32 - __builtin_clz(x) : 0)
+#define arch_flsl(x) ((x) ? BITS_PER_LONG - __builtin_clzl(x) : 0)
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 761361291e6f..8119b5ace877 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -171,10 +171,10 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
     return (old & mask) != 0;
 }
 
-#define flsl(x) generic_flsl(x)
-#define fls(x) generic_flsl(x)
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
+#define arch_fls(x)  ((x) ? 32 - __builtin_clz(x) : 0)
+#define arch_flsl(x) ((x) ? BITS_PER_LONG - __builtin_clzl(x) : 0)
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 830e488f33a0..fc9fe73ad5ba 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -447,33 +447,41 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
 }
 #define arch_ffsl arch_ffsl
 
-/**
- * fls - find last bit set
- * @x: the word to search
- *
- * This is defined the same way as ffs.
- */
-static inline int flsl(unsigned long x)
+static always_inline unsigned int arch_fls(unsigned int x)
 {
-    long r;
+    unsigned int r;
+
+    /* See arch_ffs() for safety discussions. */
+    if ( __builtin_constant_p(x > 0) && x > 0 )
+        asm ( "bsr %[val], %[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x) );
+    else
+        asm ( "bsr %[val], %[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x), "[res]" (-1) );
 
-    asm ( "bsr %1,%0\n\t"
-          "jnz 1f\n\t"
-          "mov $-1,%0\n"
-          "1:" : "=r" (r) : "rm" (x));
-    return (int)r+1;
+    return r + 1;
 }
+#define arch_fls arch_fls
 
-static inline int fls(unsigned int x)
+static always_inline unsigned int arch_flsl(unsigned long x)
 {
-    int r;
+    unsigned int r;
+
+    /* See arch_ffs() for safety discussions. */
+    if ( __builtin_constant_p(x > 0) && x > 0 )
+        asm ( "bsr %[val], %q[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x) );
+    else
+        asm ( "bsr %[val], %q[res]"
+              : [res] "=r" (r)
+              : [val] "rm" (x), "[res]" (-1) );
 
-    asm ( "bsr %1,%0\n\t"
-          "jnz 1f\n\t"
-          "mov $-1,%0\n"
-          "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_flsl arch_flsl
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index b3813f818198..b4845d9e84d1 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -26,7 +26,32 @@ static void __init test_ffs(void)
 #endif
 }
 
+static void __init test_fls(void)
+{
+    /* unsigned int fls(unsigned int) */
+    CHECK(fls, 0, 0);
+    CHECK(fls, 1, 1);
+    CHECK(fls, 3, 2);
+    CHECK(fls, 7, 3);
+    CHECK(fls, 6, 3);
+    CHECK(fls, 0x80000000U, 32);
+
+    /* unsigned int flsl(unsigned long) */
+    CHECK(flsl, 0, 0);
+    CHECK(flsl, 1, 1);
+    CHECK(flsl, 3, 2);
+    CHECK(flsl, 7, 3);
+    CHECK(flsl, 6, 3);
+
+    CHECK(flsl, 1 | (1UL << (BITS_PER_LONG - 1)), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    CHECK(flsl, 1 | (1UL << 32), 33);
+    CHECK(flsl, 1 | (1UL << 63), 64);
+#endif
+}
+
 static void __init __constructor test_bitops(void)
 {
     test_ffs();
+    test_fls();
 }
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 88cf27a88bcf..e7df6377372d 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -60,6 +60,30 @@ static always_inline __pure unsigned int ffsl(unsigned long x)
 #endif
 }
 
+static always_inline __pure unsigned int fls(unsigned int x)
+{
+    if ( __builtin_constant_p(x) )
+        return x ? 32 - __builtin_clz(x) : 0;
+
+#ifdef arch_fls
+    return arch_fls(x);
+#else
+    return generic_flsl(x);
+#endif
+}
+
+static always_inline __pure unsigned int flsl(unsigned long x)
+{
+    if ( __builtin_constant_p(x) )
+        return x ? BITS_PER_LONG - __builtin_clzl(x) : 0;
+
+#ifdef arch_fls
+    return arch_flsl(x);
+#else
+    return generic_flsl(x);
+#endif
+}
+
 /* --------------------- Please tidy below here --------------------- */
 
 #ifndef find_next_bit
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:04:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:04:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729756.1135101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8x-0001f0-RS; Fri, 24 May 2024 20:03:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729756.1135101; Fri, 24 May 2024 20:03:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8x-0001e3-AW; Fri, 24 May 2024 20:03:59 +0000
Received: by outflank-mailman (input) for mailman id 729756;
 Fri, 24 May 2024 20:03:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8v-000773-Eb
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:57 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c01d67a2-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:55 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso649410766b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:55 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c01d67a2-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581035; x=1717185835; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0OUrbEItmp/7smeXVfEuek5P5UjrbBJNpLNlS999tow=;
        b=u+aF8XMjIi9Fnwe7rnYtGFtvSyasc/4Qvlnxsuz4Ubw+8Y4TfXwt7RvTNBKwerXNB+
         zVlnRRtoGYJdivfzyYAg/kzQFenKpxMMOv7VL+gdZt5bNPiUZqf2+5YS+xgt2B4hAtjQ
         Q72uhwQ7EJxf1QX1BfUhhrZMf9UdnIAXsAfyE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581035; x=1717185835;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0OUrbEItmp/7smeXVfEuek5P5UjrbBJNpLNlS999tow=;
        b=c9sRBk9pReIdHVEmlzB3wAluzBEeWXksyATgeV0zxOFN2CjranK1Z6cHQ1yqxEnFt2
         equzeV+cmwB8REz0HOWA0rPFUMlA2NyFSYuN/39JzLw0mFKm+MsIBur47tamaz5lBvIU
         rUwOrlBX/i+wI7mg5lDQFtW0V0MpJpEhRharIsRKQ7e4N/kgnARiZqS9GS0itTUosSU2
         w9OYP3B9p+6WVe+iXUuZrZ7TEMn/VtC578GGghKmO7W/LObHedwt7Pzo52EnOmLPwrht
         62SpIs+NkZTEJP4Zik3YNhemaxt05cLPhjJbrLm/YvF39cauSLTZc2zWj2sKadL4XrNg
         8qvQ==
X-Gm-Message-State: AOJu0YyXLGoJ9to+O3CBg5HLoh6A5HvBbKylEsTspt9skiYHqmY5z2Du
	dr2GxnIQW0HBFsz9ZDWY9jxR7j8x6x/qTqMLWXFEYtjA2dPyu/k6PYzUbOSwbix4zVvsdzP7rj+
	4HA8=
X-Google-Smtp-Source: AGHT+IFDepIWI/VDxApQZRZYemyfm2VE+hQJWb/SbSRzW6+QTax//5EAno8CL93TfAhRjnW5+HYXQQ==
X-Received: by 2002:a17:906:a04e:b0:a62:44e1:3f56 with SMTP id a640c23a62f3a-a6244e14068mr444618466b.37.1716581035180;
        Fri, 24 May 2024 13:03:55 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 12/13] xen/bitops: Clean up ffs64()/fls64() definitions
Date: Fri, 24 May 2024 21:03:37 +0100
Message-Id: <20240524200338.1232391-13-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implement ffs64() and fls64() as plain static inlines, dropping the ifdefary
and intermediate generic_f?s64() forms.

Add tests for all interesting bit positions at 32bit boundaries.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * Use ULL rather than a uint64_t cast.
 * Extend to fls64() too.
---
 xen/common/bitops.c      | 32 ++++++++++++++++++++++++++++++
 xen/include/xen/bitops.h | 42 +++++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 22 deletions(-)

diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index b4845d9e84d1..5482e5a1218d 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -24,6 +24,22 @@ static void __init test_ffs(void)
     CHECK(ffsl, 1UL << 32, 33);
     CHECK(ffsl, 1UL << 63, 64);
 #endif
+
+    /*
+     * unsigned int ffs64(uint64_t)
+     *
+     * 32-bit builds of Xen have to split this into two adjacent operations,
+     * so test all interesting bit positions across the divide.
+     */
+    CHECK(ffs64, 0, 0);
+    CHECK(ffs64, 1, 1);
+    CHECK(ffs64, 3, 1);
+    CHECK(ffs64, 7, 1);
+    CHECK(ffs64, 6, 2);
+
+    CHECK(ffs64, 0x8000000080000000ULL, 32);
+    CHECK(ffs64, 0x8000000100000000ULL, 33);
+    CHECK(ffs64, 0x8000000000000000ULL, 64);
 }
 
 static void __init test_fls(void)
@@ -48,6 +64,22 @@ static void __init test_fls(void)
     CHECK(flsl, 1 | (1UL << 32), 33);
     CHECK(flsl, 1 | (1UL << 63), 64);
 #endif
+
+    /*
+     * unsigned int ffl64(uint64_t)
+     *
+     * 32-bit builds of Xen have to split this into two adjacent operations,
+     * so test all interesting bit positions across the divide.
+     */
+    CHECK(fls64, 0, 0);
+    CHECK(fls64, 1, 1);
+    CHECK(fls64, 3, 2);
+    CHECK(fls64, 7, 3);
+    CHECK(fls64, 6, 3);
+
+    CHECK(fls64, 0x0000000080000001ULL, 32);
+    CHECK(fls64, 0x0000000100000001ULL, 33);
+    CHECK(fls64, 0x8000000000000001ULL, 64);
 }
 
 static void __init __constructor test_bitops(void)
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index e7df6377372d..c5518d2c8552 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -60,6 +60,14 @@ static always_inline __pure unsigned int ffsl(unsigned long x)
 #endif
 }
 
+static always_inline __pure unsigned int ffs64(uint64_t x)
+{
+    if ( BITS_PER_LONG == 64 )
+        return ffsl(x);
+    else
+        return !x || (uint32_t)x ? ffs(x) : ffs(x >> 32) + 32;
+}
+
 static always_inline __pure unsigned int fls(unsigned int x)
 {
     if ( __builtin_constant_p(x) )
@@ -84,6 +92,18 @@ static always_inline __pure unsigned int flsl(unsigned long x)
 #endif
 }
 
+static always_inline __pure unsigned int fls64(uint64_t x)
+{
+    if ( BITS_PER_LONG == 64 )
+        return flsl(x);
+    else
+    {
+        uint32_t h = x >> 32;
+
+        return h ? fls(h) + 32 : fls(x);
+    }
+}
+
 /* --------------------- Please tidy below here --------------------- */
 
 #ifndef find_next_bit
@@ -134,28 +154,6 @@ extern unsigned long find_first_zero_bit(const unsigned long *addr,
                                          unsigned long size);
 #endif
 
-#if BITS_PER_LONG == 64
-# define fls64 flsl
-# define ffs64 ffsl
-#else
-# ifndef ffs64
-static inline int generic_ffs64(__u64 x)
-{
-    return !x || (__u32)x ? ffs(x) : ffs(x >> 32) + 32;
-}
-#  define ffs64 generic_ffs64
-# endif
-# ifndef fls64
-static inline int generic_fls64(__u64 x)
-{
-    __u32 h = x >> 32;
-
-    return h ? fls(h) + 32 : fls(x);
-}
-#  define fls64 generic_fls64
-# endif
-#endif
-
 static inline int get_bitmask_order(unsigned int count)
 {
     int order;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 20:04:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 20:04:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729757.1135107 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8y-0001q4-Tw; Fri, 24 May 2024 20:04:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729757.1135107; Fri, 24 May 2024 20:04:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAb8y-0001nD-6u; Fri, 24 May 2024 20:04:00 +0000
Received: by outflank-mailman (input) for mailman id 729757;
 Fri, 24 May 2024 20:03:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PGXh=M3=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sAb8w-000773-8d
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 20:03:58 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c09c0368-1a08-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 22:03:56 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a6265d3ba8fso122388366b.0
 for <xen-devel@lists.xenproject.org>; Fri, 24 May 2024 13:03:56 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c9377d8sm173553066b.55.2024.05.24.13.03.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 May 2024 13:03:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c09c0368-1a08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716581036; x=1717185836; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GlC1OJvyxmbXQCbGIy6Vwf8C5QyN1N/4JG+CgGnlpRU=;
        b=W4Vn8jeMcf3e3Cf/cA2EbswwJAAdH3KK18eI/yxPtwDuKoqzUGRjtN3B0aseVZk9rK
         d1wPt2Z+onl0uBjpQB304H5v59sdyYfF0at5/rWFQP1U+rVMPPu0d5D/uRjhSthuwrZg
         M68n9HH+In3Qcx3hW6+KkVaycHp0jIQvGN+pk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716581036; x=1717185836;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GlC1OJvyxmbXQCbGIy6Vwf8C5QyN1N/4JG+CgGnlpRU=;
        b=fyPa+BHgkBftErxLTBSL8tlwlmE4HcMJhKUFWKI1CJmZfcxzdWRVevv/ozlhbTM/a1
         ePxJxG5CkYFWc+vMbhx+XXC33cqJlOxB1TJa3SvQvBuZ6SJX3CMyTh/adHDvJ9Q9Dmi1
         Nkw8+Aadyg+i98tn+JqGwwQxxOerVfZ3jbAflMJMM0n5wbe7DzsTW4kw3vjKttPjWbUA
         ktCf+We0OSvrRYg9TiDF1BxrXmR/B1JEJ9pU7+CEkT1Ec2kqd1UMf9WzxkiKIQNyxWOo
         /K+KM+91nGhHCMbN10VthXq9qE5qCMlYqEXARS2GcH+o+JRvSJfBhL0cF+/RT4iJDHfz
         NDLQ==
X-Gm-Message-State: AOJu0Yx5PvClYnxw99THLKLfQOJEB7vR/sByZ86HSnUXcsrvjfHq87SP
	aRQTSz0+pLiwfBRYpNHpISsPZy6p6A55k77beqfU7ga2v6bO3OWevvdTN3pgn7te7cyrPNic78A
	aVWE=
X-Google-Smtp-Source: AGHT+IF8O0rWqo0YlTUhztmqD2tpEsu9ss1HcoDAnauTBrFqCne9vXBOwm/Ri806Jb93zik3jeTNJw==
X-Received: by 2002:a17:906:4ac1:b0:a5a:8bc4:f503 with SMTP id a640c23a62f3a-a62641dfe3fmr206635666b.25.1716581036021;
        Fri, 24 May 2024 13:03:56 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 13/13] xen/bitops: Rearrange the top of xen/bitops.h
Date: Fri, 24 May 2024 21:03:38 +0100
Message-Id: <20240524200338.1232391-14-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The #include <asm/bitops.h> can move to the top of the file now now that
generic_f?s() have been untangled.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2:
 * New
---
 xen/include/xen/bitops.h | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)

diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index c5518d2c8552..6a5e28730a25 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -4,6 +4,8 @@
 #include <xen/compiler.h>
 #include <xen/types.h>
 
+#include <asm/bitops.h>
+
 /*
  * Create a contiguous bitmask starting at bit position @l and ending at
  * position @h. For example GENMASK(30, 21) gives us 0x7fe00000ul.
@@ -15,27 +17,13 @@
     (((~0ULL) << (l)) & (~0ULL >> (BITS_PER_LLONG - 1 - (h))))
 
 /*
- * Find First/Last Set bit.
+ * Find First/Last Set bit (all forms).
  *
  * Bits are labelled from 1.  Returns 0 if given 0.
  */
 unsigned int __pure generic_ffsl(unsigned long x);
 unsigned int __pure generic_flsl(unsigned long x);
 
-/*
- * Include this here because some architectures need generic_ffs/fls in
- * scope
- */
-
-/* --------------------- Please tidy above here --------------------- */
-
-#include <asm/bitops.h>
-
-/*
- * Find First/Last Set bit (all forms).
- *
- * Bits are labelled from 1.  Returns 0 if given 0.
- */
 static always_inline __pure unsigned int ffs(unsigned int x)
 {
     if ( __builtin_constant_p(x) )
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri May 24 21:44:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 21:44:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729847.1135130 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAchj-0000Mv-Km; Fri, 24 May 2024 21:43:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729847.1135130; Fri, 24 May 2024 21:43:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAchj-0000Mo-Gi; Fri, 24 May 2024 21:43:59 +0000
Received: by outflank-mailman (input) for mailman id 729847;
 Fri, 24 May 2024 21:43:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAchh-0000Mf-JG
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 21:43:57 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b72d0254-1a16-11ef-90a1-e314d9c70b13;
 Fri, 24 May 2024 23:43:55 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id A18E1CE19F1;
 Fri, 24 May 2024 21:43:49 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F521C2BBFC;
 Fri, 24 May 2024 21:43:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b72d0254-1a16-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716587028;
	bh=/tvhAKH8diu82OuYBC/gDMZewjW2e2ICgNJ5qfZa3Mw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=g4ZcjC+HwVU1fnitzyPpLoiNO1GjC3zxfpsFup8YXndXquKa98n65zVIpRq5vpoBY
	 Lf9wNjFgvBBbXuLXEMNtuvLHiuftnIgC3zEPMLYG+y6+vYQVY+WyEzn2rJYe4Zt3o4
	 YZVzCkeatC+ls1TdYLr+itEfQPt6Uxb0hceJLkgrzB7f2w+FE/AFW+GzNPN44ihrLX
	 Lcz56Sn+BLnV8/kpil8114EB8aghIjUHiry4gJX7+F2xShbM61OXlFZGF5Af2uUwMn
	 8EJ0DbehGR4J6WBcqhQ3iQ1g/UlMB8aVHUaTFASWujqryiw4BMjHTyXf3eF9GbLQGO
	 K5sUpmxTsXXyQ==
Date: Fri, 24 May 2024 14:43:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <stefano.stabellini@amd.com>, 
    xen-devel@lists.xenproject.org, anthony@xenproject.org, 
    sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
    Volodymyr_Babchuk@epam.com, Henry Wang <xin.wang2@amd.com>, 
    Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v5 3/7] tools/arm: Introduce the "nr_spis" xl config
 entry
In-Reply-To: <56bf92b7-9bcc-4093-b51c-b8ec0b43209c@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241443410.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop> <20240524021814.2666257-3-stefano.stabellini@amd.com> <56bf92b7-9bcc-4093-b51c-b8ec0b43209c@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 May 2024, Julien Grall wrote:
> Hi,
> 
> On 24/05/2024 03:18, Stefano Stabellini wrote:
> > From: Henry Wang <xin.wang2@amd.com>
> > 
> > Currently, the number of SPIs allocated to the domain is only
> > configurable for Dom0less DomUs. Xen domains are supposed to be
> > platform agnostics and therefore the numbers of SPIs for libxl
> > guests should not be based on the hardware.
> > 
> > Introduce a new xl config entry for Arm to provide a method for
> > user to decide the number of SPIs. This would help to avoid
> > bumping the `config->arch.nr_spis` in libxl everytime there is a
> > new platform with increased SPI numbers.
> > 
> > Update the doc and the golang bindings accordingly.
> > 
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> > ---
> >   docs/man/xl.cfg.5.pod.in             | 16 ++++++++++++++++
> >   tools/golang/xenlight/helpers.gen.go |  2 ++
> >   tools/golang/xenlight/types.gen.go   |  1 +
> >   tools/include/libxl.h                |  7 +++++++
> >   tools/libs/light/libxl_arm.c         |  4 ++--
> >   tools/libs/light/libxl_types.idl     |  1 +
> >   tools/xl/xl_parse.c                  |  3 +++
> >   7 files changed, 32 insertions(+), 2 deletions(-)
> > 
> > diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> > index 8f2b375ce9..ac3f88fd57 100644
> > --- a/docs/man/xl.cfg.5.pod.in
> > +++ b/docs/man/xl.cfg.5.pod.in
> > @@ -3072,6 +3072,22 @@ raised.
> >     =back
> >   +=over 4
> > +
> > +=item B<nr_spis="NR_SPIS">
> > +
> > +An optional integer parameter specifying the number of SPIs (Shared
> > +Peripheral Interrupts) to allocate for the domain. Max is 991 SPIs. If
> > +the value specified by the `nr_spis` parameter is smaller than the
> > +number of SPIs calculated by the toolstack based on the devices
> > +allocated for the domain, or the `nr_spis` parameter is not specified,
> > +the value calculated by the toolstack will be used for the domain.
> > +Otherwise, the value specified by the `nr_spis` parameter will be used.
> > +The number of SPIs should match the highest interrupt ID that will be
> > +assigned to the domain.
> > +
> > +=back
> > +
> >   =head3 x86
> >     =over 4
> > diff --git a/tools/golang/xenlight/helpers.gen.go
> > b/tools/golang/xenlight/helpers.gen.go
> > index b9cb5b33c7..fe5110474d 100644
> > --- a/tools/golang/xenlight/helpers.gen.go
> > +++ b/tools/golang/xenlight/helpers.gen.go
> > @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
> >   x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
> >   x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
> >   x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
> > +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
> >   if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil
> > {
> >   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
> >   }
> > @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
> >   xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
> >   xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
> >   xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
> > +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
> >   if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
> >   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
> >   }
> > diff --git a/tools/golang/xenlight/types.gen.go
> > b/tools/golang/xenlight/types.gen.go
> > index 5b293755d7..c9e45b306f 100644
> > --- a/tools/golang/xenlight/types.gen.go
> > +++ b/tools/golang/xenlight/types.gen.go
> > @@ -597,6 +597,7 @@ ArchArm struct {
> >   GicVersion GicVersion
> >   Vuart VuartType
> >   SveVl SveType
> > +NrSpis uint32
> >   }
> >   ArchX86 struct {
> >   MsrRelaxed Defbool
> > diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> > index 62cb07dea6..3b5c18b48b 100644
> > --- a/tools/include/libxl.h
> > +++ b/tools/include/libxl.h
> > @@ -636,6 +636,13 @@
> >    */
> >   #define LIBXL_HAVE_XEN_9PFS 1
> >   +/*
> > + * LIBXL_HAVE_NR_SPIS indicates the presence of the nr_spis field in
> > + * libxl_domain_build_info that specifies the number of SPIs interrupts
> > + * for the guest.
> > + */
> > +#define LIBXL_HAVE_NR_SPIS 1
> > +
> 
> Looking at the other arch.arm field, I think this wants to be:
> 
> /*
>  * libxl_domain_build_info has the arch_arm.nr_spis field
>  */
> #define LIBXL_HAVE_BUILDINFO_ARCH_NR_SPIS 1
> 
> This would also clarify that the field is Arm specific.

I made the change


From xen-devel-bounces@lists.xenproject.org Fri May 24 21:48:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 21:48:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729854.1135139 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcmV-00011G-7Q; Fri, 24 May 2024 21:48:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729854.1135139; Fri, 24 May 2024 21:48:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcmV-000119-4g; Fri, 24 May 2024 21:48:55 +0000
Received: by outflank-mailman (input) for mailman id 729854;
 Fri, 24 May 2024 21:48:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAcmU-00010z-17; Fri, 24 May 2024 21:48:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAcmT-0007Rm-Va; Fri, 24 May 2024 21:48:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAcmT-0005q4-Jl; Fri, 24 May 2024 21:48:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAcmT-0006u3-JE; Fri, 24 May 2024 21:48:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=P6kcGPanWtHCctdb2uaTRzuVhytsz8ZfTxlWYu+/SZc=; b=Og/4l3wW63EbGpyb4DSN8aaZtd
	yO6lG80GCM0I+WGjfX85Ko6P2oHyjiHcOrDxZleNpIRHPKbzqVKinihE+LFt0fWy4lU+GLwS2kbG8
	n6f/zdCc9m6WRmi/vuCcALMglhzmlc/mLxMyp3GCIiOHXjhrFzpWuqI7fkTaQwy+TFyQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186133-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186133: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=3b3efef58dc4bf6c07a73862c280e30f2023054d
X-Osstest-Versions-That:
    libvirt=7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 May 2024 21:48:53 +0000

flight 186133 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186133/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186070
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              3b3efef58dc4bf6c07a73862c280e30f2023054d
baseline version:
 libvirt              7dda4a03ac77bbe14b12b7b8f3a509a0e09f3129

Last test of basis   186070  2024-05-22 04:20:52 Z    2 days
Failing since        186099  2024-05-23 04:18:41 Z    1 days    2 attempts
Testing same since   186133  2024-05-24 04:18:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>
  Laine Stump <laine@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   7dda4a03ac..3b3efef58d  3b3efef58dc4bf6c07a73862c280e30f2023054d -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 24 21:51:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 21:51:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729861.1135150 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcpF-0002e4-MK; Fri, 24 May 2024 21:51:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729861.1135150; Fri, 24 May 2024 21:51:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcpF-0002dx-II; Fri, 24 May 2024 21:51:45 +0000
Received: by outflank-mailman (input) for mailman id 729861;
 Fri, 24 May 2024 21:51:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAcpE-0002dr-Eg
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 21:51:44 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cde6008a-1a17-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 23:51:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id EA5F661E05;
 Fri, 24 May 2024 21:51:40 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33601C2BBFC;
 Fri, 24 May 2024 21:51:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cde6008a-1a17-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716587500;
	bh=zMZN/QOtwKl3M1pGVQQ0yPsaz6DyxlYrcCWIu8Yd1a8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rNy6m76RtxfVtHf9XAsYwm0Y/QC11GAKL6DSL6V34oK41XRu7VT+MEVCkeVF8z9nK
	 YxLBE2xXY+Uq+yN688NVdEkPP1Q/TZRsweRHscR198D/6nn/lOskTpUXyHkRUOorKz
	 UJOeAQVg59DmS5XEyTbKUjo4nFzqEkHhGUFPIrGEQnWyvd5Gi/9HDw1MThiPp4BaF/
	 KkrCyhqplJ7k84gQ3GZXIKFtv91pIvNiDZTyeTlphOsuQu4uFJZT95CJSJOU48Nkr2
	 iB08ccmNOvozzOHe4rURdM4J20rE5EL2sa0ggnewByR8p6Vi6cGCOk5ylnlyIZ7xl+
	 4DkwToH5GMsGA==
Date: Fri, 24 May 2024 14:51:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <stefano.stabellini@amd.com>, 
    xen-devel@lists.xenproject.org, anthony@xenproject.org, 
    sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
    Volodymyr_Babchuk@epam.com, Henry Wang <xin.wang2@amd.com>, 
    Vikram Garhwal <fnu.vikram@xilinx.com>
Subject: Re: [PATCH v5 5/7] xen/arm: Add XEN_DOMCTL_dt_overlay and device
 attachment to domains
In-Reply-To: <760dba0c-8731-48a3-81d8-cd8fa2dada77@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241444170.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop> <20240524021814.2666257-5-stefano.stabellini@amd.com> <760dba0c-8731-48a3-81d8-cd8fa2dada77@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 24/05/2024 03:18, Stefano Stabellini wrote:
> > From: Henry Wang <xin.wang2@amd.com>
> > 
> > In order to support the dynamic dtbo device assignment to a running
> > VM, the add/remove of the DT overlay and the attach/detach of the
> > device from the DT overlay should happen separately. Therefore,
> > repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
> > overlay to Xen device tree, instead of assigning the device to the
> > hardware domain at the same time. It is OK to change the sysctl behavior
> > as this feature is experimental so changing sysctl behavior and breaking
> > compatibility is OK.
> > 
> > Add the XEN_DOMCTL_dt_overlay with operations
> > XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment to the domain.
> > 
> > The hypervisor firstly checks the DT overlay passed from the toolstack
> > is valid. Then the device nodes are retrieved from the overlay tracker
> > based on the DT overlay. The attach of the device is implemented by
> > mapping the IRQ and IOMMU resources. All devices in the overlay are
> > assigned to a single domain.
> > 
> > Also take the opportunity to make one coding style fix in sysctl.h.
> > 
> > xen,reg is to be used to handle non-1:1 mappings but it is currently
> > unsupported.
> 
> This means that we would still try to use 1:1 mappings for non-directmap
> domain. Given that the overlay is a blob, I am a bit concerned that the user
> may not notice any clash and it would be difficult to debug.
> 
> Therefore, I would like xen,reg to be mandatory when using non directmapped
> domain. For now, the best approach would be to prevent device assignment if
> !is_domain_direct_mapped().

That's fine, I'll make the change


> > +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
> > +{
> > +    long ret;
> > +    void *overlay_fdt;
> > +
> > +    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
> > +        return -EOPNOTSUPP;
> > +
> > +    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
> 
> Please add #define DT_OVERLAY_MAX_SIZE KB(500) and use it here and the other
> place.

OK


From xen-devel-bounces@lists.xenproject.org Fri May 24 21:52:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 21:52:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729865.1135159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcq7-0003BT-TU; Fri, 24 May 2024 21:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729865.1135159; Fri, 24 May 2024 21:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAcq7-0003BM-Qy; Fri, 24 May 2024 21:52:39 +0000
Received: by outflank-mailman (input) for mailman id 729865;
 Fri, 24 May 2024 21:52:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAcq6-0002dr-FQ
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 21:52:38 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ed779019-1a17-11ef-b4bb-af5377834399;
 Fri, 24 May 2024 23:52:36 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 1B1B3CE17F2;
 Fri, 24 May 2024 21:52:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 028EFC2BBFC;
 Fri, 24 May 2024 21:52:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed779019-1a17-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716587552;
	bh=lqg/FdrrZfv0OltkAYJZheRT6+j3ucts1UtyGs8yhb4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VMDchUYTgJe4oHNMuUk6+D59xfuadjX3ZAB4oN+Idh1x1Ya93HSJftS6sdrVKoPSC
	 Z5RI6gZ2Kph6Yw5pErEh1Hye9Ypvv2j85GeS1HEAhwYzyEwvyqhRkIin6SbB2lFFLf
	 rT7wq5jtYKFMgbMOvnr+hmlT/6BcBdde6fZKV8uUqyqgu42QthIuFe7stb6YWW2oWZ
	 OipfV/ObC38ml6aq6eQjiUHmk3QoPO6wrlwL3Ul/jYbGeqw/pQAImrAK5XM2WhOa1j
	 aV3uHFVMFkY8315tZHdygc83H6v3uEuzOD3zuVHd+geBtBzholFxFdqPQRrEoXsPQ4
	 are12XJ6YjLzg==
Date: Fri, 24 May 2024 14:52:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <stefano.stabellini@amd.com>, 
    xen-devel@lists.xenproject.org, anthony@xenproject.org, 
    sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
    Volodymyr_Babchuk@epam.com, Henry Wang <xin.wang2@amd.com>, 
    Jason Andryuk <jason.andryuk@amd.com>
Subject: Re: [PATCH v5 6/7] tools: Introduce the "xl dt-overlay attach"
 command
In-Reply-To: <59607ca9-3d78-4f8e-ad02-22df89cd9161@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241452240.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop> <20240524021814.2666257-6-stefano.stabellini@amd.com> <59607ca9-3d78-4f8e-ad02-22df89cd9161@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 24/05/2024 03:18, Stefano Stabellini wrote:
> > From: Henry Wang <xin.wang2@amd.com>
> > 
> > With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
> > attach (in the future also detach) devices from the provided DT overlay
> > to domains. Support this by introducing a new "xl dt-overlay" command
> > and related documentation, i.e. "xl dt-overlay attach. Slightly rework
> > the command option parsing logic.
> > 
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> > ---
> >   tools/include/libxl.h               | 15 +++++++++++
> >   tools/include/xenctrl.h             |  3 +++
> >   tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++++
> >   tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++++
> >   tools/xl/xl_cmdtable.c              |  4 +--
> >   tools/xl/xl_vmcontrol.c             | 39 ++++++++++++++++++++---------
> >   6 files changed, 106 insertions(+), 14 deletions(-)
> > 
> > diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> > index 3b5c18b48b..f2e19ec592 100644
> > --- a/tools/include/libxl.h
> > +++ b/tools/include/libxl.h
> > @@ -643,6 +643,12 @@
> >    */
> >   #define LIBXL_HAVE_NR_SPIS 1
> >   +/*
> > + * LIBXL_HAVE_OVERLAY_DOMAIN indicates the presence of
> > + * libxl_dt_overlay_domain.
> > + */
> > +#define LIBXL_HAVE_OVERLAY_DOMAIN 1
> I think this wants to gain DT_ just before OVERLAY. So from the name it is
> clearer we are talking about the Device-Tree overlay and not filesystem (or
> anything else where overlays are involved).

Done


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:14:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:14:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729878.1135170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdBF-0006RN-J5; Fri, 24 May 2024 22:14:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729878.1135170; Fri, 24 May 2024 22:14:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdBF-0006RG-GE; Fri, 24 May 2024 22:14:29 +0000
Received: by outflank-mailman (input) for mailman id 729878;
 Fri, 24 May 2024 22:14:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdBE-0006RA-7V
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:14:28 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fb6a087f-1a1a-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:14:27 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B72AE61508;
 Fri, 24 May 2024 22:14:25 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3F5FC2BBFC;
 Fri, 24 May 2024 22:14:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb6a087f-1a1a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716588865;
	bh=/X5DEcP1Js14mnr8xs0vj1UKxVeCX4tpbdd7BN1HbkI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MpC5iXLFuoBhf073m3kUij8QupCSJs6Xrm+r933buBF9zVhGwpnXxKNTyg5mv+C7O
	 3seluiyUPvKCiZ7/ARagM9darcgyFPOdhx0STQ2C92kKZoEFAdiTPqsE8ul728e1oz
	 VceYqiX7PfzyALnGsbDdOrWGqmVX0OFXvK8fuQY0I7fgp7KibCMgelVu/POZh8ju/E
	 TC9bx3uR6WtkjI/bRL3EBefmwSwNgHKhQaBvxdiZ38o8gljSHMy+M+0pGCAjSE9QDi
	 mYVVFaNW7j65ON7pprA/puvHeEJvjdFJZFxp1cI1VQyxsdMgDkcLlgRgrK0IuzR59T
	 H0hh8/iXtMjiQ==
Date: Fri, 24 May 2024 15:14:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <stefano.stabellini@amd.com>, 
    xen-devel@lists.xenproject.org, anthony@xenproject.org, 
    sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
    Volodymyr_Babchuk@epam.com, Vikram Garhwal <fnu.vikram@xilinx.com>, 
    Stefano Stabellini <stefano.stabellini@xilinx.com>, 
    Henry Wang <xin.wang2@amd.com>
Subject: Re: [PATCH v5 7/7] docs: Add device tree overlay documentation
In-Reply-To: <a7a0456d-9eac-4db4-8782-ae94c7e338b1@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241456230.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405231914360.2557291@ubuntu-linux-20-04-desktop> <20240524021814.2666257-7-stefano.stabellini@amd.com> <a7a0456d-9eac-4db4-8782-ae94c7e338b1@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 24/05/2024 03:18, Stefano Stabellini wrote:
> > From: Vikram Garhwal <fnu.vikram@xilinx.com>
> > 
> > Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > ---
> >   docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 82 insertions(+)
> >   create mode 100644 docs/misc/arm/overlay.txt
> > 
> > diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
> > new file mode 100644
> > index 0000000000..0a2dee951a
> > --- /dev/null
> > +++ b/docs/misc/arm/overlay.txt
> > @@ -0,0 +1,82 @@
> > +# Device Tree Overlays support in Xen
> > +
> > +Xen experimentally supports dynamic device assignment to running
> > +domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device
> > +tree, and attaching them to a running domain with given $domid.
> > +
> > +Dynamic node assignment works in two steps:
> > +
> > +## Add/Remove device tree overlay to/from Xen device tree
> > +
> > +1. Xen tools check the dtbo given and parse all other user provided
> > arguments
> > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> > +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
> > +
> > +## Attach device from the DT overlay to domain
> > +
> > +1. Xen tools check the dtbo given and parse all other user provided
> > arguments
> > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> > +3. Xen hypervisor attach the device to the user-provided $domid by
> > +   mapping node resources in the DT overlay.
> > +
> > +# Examples
> > +
> > +Here are a few examples on how to use it.
> > +
> > +## Dom0 device add
> > +
> > +For assigning a device tree overlay to Dom0, user should firstly properly
> > +prepare the DT overlay. More information about device tree overlays can be
> > +found in [1]. Then, in Dom0, enter the following:
> > +
> > +    (dom0) xl dt-overlay add overlay.dtbo
> > +
> > +This will allocate the devices mentioned in overlay.dtbo to Xen device
> > tree.
> > +
> > +To assign the newly added device from the dtbo to Dom0:
> > +
> > +    (dom0) xl dt-overlay attach overlay.dtbo 0
> > +
> > +Next, if the user wants to add the same device tree overlay to dom0
> > +Linux, execute the following:
> > +
> > +    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> > +    (dom0) cat overlay.dtbo >
> > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> > +
> > +Finally if needed, the relevant Linux kernel drive can be loaded using:
> > +
> > +    (dom0) modprobe module_name.ko
> > +
> > +## DomU device add/remove
> > +
> > +All the nodes in dtbo will be assigned to a domain; the user will need
> > +to prepare the dtb for the domU.
> 
> s/dtb/dtbo/? 

Yes, done


> But I am little bit confused with the wording. I think you may
> want to add *different dtbo* so it clarifies from the start (this only becomes
> obvious at the end of the section) that the user is not meant to use the same
> for all the commands.

Yes it was confusing, I tried to clarify it. This is a doc so we could
improve it further after the freeze.


>  For example, the `interrupt-parent`
> > +property of the DomU overlay should be changed to the Xen hardcoded
> > +value `0xfde8`, and the xen,reg property should be added to specify the
> > +address mappings. If xen,reg is not present, it is assumed 1:1 mapping.
> 
> Repeating an earlier comment here. I think xen,reg should be mandatory for
> non-direct mapped domain.

That's OK


> Also, can you clarify what is the expect property layout for xen,reg?

I'll add a pointer


> > +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`.
> > +
> > +For new domains to be created, the user will need to create the DomU
> > +with below properties properly configured in the xl config file:
> > +- `iomem`
> 
> I looked at your reply in v4 and I am afraid I still don't understand why we
> are mentioning 'iomem'. If we want to use the commands below, then the domain
> needs to be created in advance. So you can't yet know 'iomem'.
> 
> You could avoid "xl dt-overlay attach" but then you need the user to specify
> both "irqs" and "iomem". From a user point of view, it would be easier to add
> a new propery in the configuration file listing the overlays. Something like:
> 
> dt_overlays = [ "overlay.dtbo", ... ]
> 
> Anyway, that somewhat separate. For now, I think we want to drop 'iomem' from
> the list and reword this paragraph to say that the 'passthrough' property
> needs to be set if you plan to use DT overlay and devices requiring the IOMMU.

OK, I did that


> 
> > +- `passthrough` (if IOMMU is needed)
> 
> This property is required at the start because we don't support enabling the
> IOMMU lazily.

Yeah. I think it is much clearer now.


> > +
> > +User will also need to modprobe the relevant drivers. For already
> > +running domains, the user can use the xl dt-overlay attach command,
> > +example:
> > +
> > +    (dom0) xl dt-overlay add overlay.dtbo            # If not executed
> > before
> > +    (dom0) xl dt-overlay attach overlay.dtbo $domid
> > +    (dom0) xl console $domid                         # To access $domid
> > console
> > +
> > +Next, if the user needs to modify/prepare the overlay.dtbo suitable for
> > +the domU:
> > +
> > +    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> > +    (domU) cat overlay_domu.dtbo >
> > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> > +
> > +Finally, if needed, the relevant Linux kernel drive can be probed:
> > +
> > +    (domU) modprobe module_name.ko
> > +
> > +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:16:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:16:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729882.1135180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDC-0006xr-Tc; Fri, 24 May 2024 22:16:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729882.1135180; Fri, 24 May 2024 22:16:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDC-0006xk-Qc; Fri, 24 May 2024 22:16:30 +0000
Received: by outflank-mailman (input) for mailman id 729882;
 Fri, 24 May 2024 22:16:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDB-0006xe-Gd
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:16:29 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on2060c.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::60c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 42c2a4fe-1a1b-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:16:27 +0200 (CEST)
Received: from PH0PR07CA0007.namprd07.prod.outlook.com (2603:10b6:510:5::12)
 by SJ1PR12MB6121.namprd12.prod.outlook.com (2603:10b6:a03:45c::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 22:16:22 +0000
Received: from CY4PEPF0000EE30.namprd05.prod.outlook.com
 (2603:10b6:510:5:cafe::a4) by PH0PR07CA0007.outlook.office365.com
 (2603:10b6:510:5::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend
 Transport; Fri, 24 May 2024 22:16:21 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000EE30.mail.protection.outlook.com (10.167.242.36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:21 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:20 -0500
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Fri, 24 May 2024 17:16:19 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42c2a4fe-1a1b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GS0mQMh+LkWGDzPh4Va4ANT0ULRmwtgEZ1kD6CRGs87btuTwfeN4ZcI+6zAcn13plyPBhnG5r2kLRYBUe3TVZ2mwASV/sKxCqyps1MBmXe4lhHteE2h8tPXV2iPJJs9dlQz2HkA8NDbVjWH48fzL1VKR9yvVGnUJv+LtUw4RwZScs5136jifTLo05G3G2kanosiOdwL4yc1lUTDkO5SuVjkvQgf5mfQfFrAvC9fGrt8q7HgZPd1kVFHrIvt/RyqJd4gZiOnIQbXGe6mNqH3K/CTYawpyKKrnlsYELxSKCOsRxiyIJt99W5QDmuW0kSeEb8GKxTKSj0ODibxcq/Jxvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ptcLF8WT724EvyJqTvtI6fTRdxA3T83ZeikxU5Q7GAs=;
 b=ZoNWWw0kMjC1AdwywpA9fX73CQHax4wSadNJZV4a0mU2/6Vd65fiUDO27tgt16CFWWHqKGlIFwKGB6pZNqz1fcfC5jDeLLvOi+SKiE/T1aTCchJTi5nZ0G7eXKErfFxfmfkAXbcyRQFk91hfAssdDJP8EaN3MyF9NgdJSx2tNGJGKQd6Ce1PBpD7MU9wQ+Z3DRfyuTGYRnhDSvGQf5tciSAh22g0IGKPlLh7q446X1sLStD1GM9yFVJVFNEeYjB+NFmXY3tiMima8pz5dlv1bAAI8B6DZHFq5LfiyTlZ6NIXuFdapmRH+fzYvJAZNm03G++Y0q29Mc61JwmMusdnUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ptcLF8WT724EvyJqTvtI6fTRdxA3T83ZeikxU5Q7GAs=;
 b=zo5TY71//DFDd2mzo6g0oiBv+5L5tsz83GDrIV31e1XHLTmXwfuuP8MuQ9aQq7HcbWOdCU6zHmPAlsuyAFUewfLuNL/kV4PqOOQF+RbymMHsZfglUpLQK+68f3HfSO2UhNtOVbqgAtESctiGmvl3pqDcVd7tC2LB72qQEpiRswU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Date: Fri, 24 May 2024 15:16:19 -0700
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony@xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, "Juergen
 Gross" <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH v6 0/7] Remaining patches for dynamic node programming using
 overlay dtbo
Message-ID: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE30:EE_|SJ1PR12MB6121:EE_
X-MS-Office365-Filtering-Correlation-Id: 5378656e-9bbc-45e9-7a55-08dc7c3f2429
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017|7416005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?NzzVz8xnSRgo//uVWWo5hLjhNenm1STuX/s2ZCQPDn2dIcS6o2zGDeMwaRpV?=
 =?us-ascii?Q?IFJHgKWnRSDsiCTggpn8p5oUGn5NojstsSrQfJIPKNMt1omxAG49O/hfHKsU?=
 =?us-ascii?Q?HmNo6NTsd3Y5BkOrRyUZz6GI8idYfSpDwbil4dI3e8lW8xvhzKwemQ8RzI4s?=
 =?us-ascii?Q?2x5OFPdpSriqNDvvCM/WhfL5p/i4Tt16cP8vrY7uRXHZWm+pEZoESFhdxx0H?=
 =?us-ascii?Q?H4hBop+qvAkkDrpxCetQ5Gk5N7bNstlqW/pCGTaKP991ipj0qN1cGbSHAfrL?=
 =?us-ascii?Q?kqQ72XN4+60U+zB5QdKEbB5+IdkEIyLsKG8FtMv5RwtxH3xQNPbqFUfFz9m+?=
 =?us-ascii?Q?HFn4W9yiEJYpFqqtoVenevtlrvRzvbGvJspmbaY4XBJ3J1rqDoMMQYalqeix?=
 =?us-ascii?Q?Nn36rBFokc7mWzES/Yki3oVfcJ5BNS4dmZAbzDAezdV5TWjZ/7z4jkxyfGy9?=
 =?us-ascii?Q?9SMc44l2k7yATrijaHuimJzNZh4Q9q+SorBpAmWklYf/rZ18zvR3G+GLqxaf?=
 =?us-ascii?Q?q1lED5MMIMHcRpgXmIGHios3YqMi1iZHAczHoAjjo7d4RTo95krSxHqdA/DK?=
 =?us-ascii?Q?wA4VAZwxrjfwTNMXMMRx143HTa/VtTmmx+M/Hmhlzgnc2q75Q+UCQ6QfcD5w?=
 =?us-ascii?Q?/jORAazbWxRyMj478ZhquM1PKbqzvpkgWWt+7JqhrhQmniiwTs6aNTruCdWI?=
 =?us-ascii?Q?+3FmxmMgFKUD0/DIpAGDEz2fl+JMM9JTRdBOVTdZATQHqQAwQE2g9LI/PU9I?=
 =?us-ascii?Q?ouGFxetbUr2o9PskNbqqfoLj5Y5wWWbVL9FYhWTNNpC4QlgRrea23LLOHKQ5?=
 =?us-ascii?Q?8arNFbzAKLGC08vGNqWQ/gvZA38bDO44Z7lau19OttUSZ+mK/mmkj6jyu624?=
 =?us-ascii?Q?KVGmDS/phdpp7MDgiFflhRddgoJiOGBYzGXtH57rwxaMxrfxm5xk2tqptN+n?=
 =?us-ascii?Q?peFaRKtlkrR5HpauQFSKrOMM17Tp6rahAn6qh2/QycknudVRJtJBvbcyK9rq?=
 =?us-ascii?Q?UvsRvP1YeJR0mbYHlzPhpPrA0Ar8J1sJ+Cqv26SxR/feudVXccJUin2wuivk?=
 =?us-ascii?Q?1XpGEgrKjdqopBQRat7tlMkXkAXXykNfxuqlGqYrdowy4v1dcp9B7OkydjQp?=
 =?us-ascii?Q?NARCaigC5Z/7oRJUS74+AHN9UAcJGd5HPefFIkTDTUXZBRMaUIfDzDVoL/q7?=
 =?us-ascii?Q?LhQl/btlR8VNgbjPb1o79fWzHR54bAZtgY9LA6NaVUhjQhxy/6fVnyhE8Nby?=
 =?us-ascii?Q?f7VhVzxtvAVgn/m3cNXVcWQ9pPId6k/Yd4Gj/vf/uupt+UoDBIKsm/BKGLF2?=
 =?us-ascii?Q?QXZaH7Cv/jp4L5wCbdUru1+v5qwPdDDk2Uu/uOe036VqQw=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017)(7416005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:21.0187
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5378656e-9bbc-45e9-7a55-08dc7c3f2429
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE30.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6121

Hi all,

This is the remaining series for the full functional "dynamic node
programming using overlay dtbo" feature. The first part [1] has
already been merged.

Quoting from the original series, the first part has already made
Xen aware of new device tree node which means updating the dt_host
with overlay node information, and in this series, the goal is to
map IRQ and IOMMU during runtime, where we will do the actual IOMMU
and IRQ mapping and unmapping to a running domain. Also, documentation
of the "dynamic node programming using overlay dtbo" feature is added.

During the discussion in v3, I was recommended to split the overlay
devices attach/detach to/from running domains to separated patches [3].
But I decided to only expose the xl user interfaces together to the
users after device attach/detach is fully functional, so I didn't
split the toolstack patch (#8).

Patch 1 is a fix of the existing code which is noticed during my local
tests, details please see the commit message.

Gitlab CI for this series can be found in [2].

[1] https://lore.kernel.org/xen-devel/20230906011631.30310-1-vikram.garhwal@amd.com/
[2] https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1301720278
[3] https://lore.kernel.org/xen-devel/e743d3d2-5884-4e55-8627-85985ba339f9@amd.com/


Changes in v6:
- address Julien's comments
- use a !is_domain_direct_mapped check

- Stefano


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:16:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:16:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729886.1135190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDe-0007U9-8f; Fri, 24 May 2024 22:16:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729886.1135190; Fri, 24 May 2024 22:16:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDe-0007TX-4w; Fri, 24 May 2024 22:16:58 +0000
Received: by outflank-mailman (input) for mailman id 729886;
 Fri, 24 May 2024 22:16:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDc-0007Iy-Fr
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:16:56 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 53c72a6f-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:16:55 +0200 (CEST)
Received: from DM5PR08CA0054.namprd08.prod.outlook.com (2603:10b6:4:60::43) by
 PH8PR12MB7205.namprd12.prod.outlook.com (2603:10b6:510:227::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.26; Fri, 24 May
 2024 22:16:52 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::19) by DM5PR08CA0054.outlook.office365.com
 (2603:10b6:4:60::43) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.26 via Frontend
 Transport; Fri, 24 May 2024 22:16:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:51 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:50 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:49 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53c72a6f-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CTzZuTGnG5+05W5Uf1a//3vm5/5e2izw06KDCV27HWIBe8C0mueHPbJEf61VdJYE2h/EQDbk5cHJmxBuWViFhsbLc1TRVxjqlhVQ3O+JscEXBzvsjcMHxBe8zUteq/eVg3zEVfn7yO8+h25hUX6mX9+Fl5aLT0yusC88UQbnbEvhDdWTtjwLB1O8Eepmr1btqpKQsv4N8Gtx2zMJpuwaoSUWdzDUd+ywW87OXCarIwUghPUBynQukRbNrRaNnyOorhxwyOAnJsy2ByQrBA08uIlWUAE/JOqwp7GALbmOcuBYrWEDPMyHEyI08WJInsb55N2qeN8AJpEYXawVaD2PLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NlizJg2iClGMdik+Fxv6BQH90USOSUhLLxid+7iVuz0=;
 b=Ej/sc3hrLkuTD/ZOdBItZcOTZgCvZkgb14atfgI2HWOGx2Hlr9dTAJBAQoHiimYxLdpzkKy7cHEMpFf6EkT7slTVllV/crg4av+54S19jgWryPTrzhb0rj4fo8i3C3mRIbTDPnHjIuim6NDdjhpg0wGTe+slmOLob0t920A5JL8eAT1JGzx36RcY2JbD2P7OUft3I6SrrtGN3UtAa7nhqg+XKIbNEGokDox64xcT7Z83lu55pMPnoW21yHPOHIN02vCdEkrIHVXRr66QbrQ7kMfJOWWtEllMMrozwjh1WTYC44C1QgbyN7Wjbd0EKhosghoQIbQe0NoYtpnzWpBtPw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NlizJg2iClGMdik+Fxv6BQH90USOSUhLLxid+7iVuz0=;
 b=2dQAv7rvWO8m5WNGBKOKSy9yTmkfMUtvBuJYnKqkD8u4hGBbRVxAkt9Ww7puLpDeAHHdXohRQfLe541nZPmSGoQqjkCk++kuG5b2ratM2PZY8VdygRyOqI6DOBq2MRCfHR7IP+hgcfyx4tiUVjpkFCo9/AHXbnK7RcF+XxX85UA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v6 3/7] tools/arm: Introduce the "nr_spis" xl config entry
Date: Fri, 24 May 2024 15:16:43 -0700
Message-ID: <20240524221647.2860711-3-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|PH8PR12MB7205:EE_
X-MS-Office365-Filtering-Correlation-Id: e9ca015d-c878-4a30-07cd-08dc7c3f3643
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?OMGWerVL3CvW7SrQ9+jFQR7rc6NW++MjeBSJIczUJ3cIrT1WuiLL/mLyUJj/?=
 =?us-ascii?Q?Wpu1awTo/LxQNe41fHOQxNhj8fbJmAhLTWtqv8CacRwVFJhFUM1cdlkLEhgI?=
 =?us-ascii?Q?B1hYz5Gl8/sfKAyv6/rcdSM01BhHLhklG6uh/1k7Aq+AVO8KQ9Ewkv4eLsel?=
 =?us-ascii?Q?M9Fu8df3AxaOlkFutEz5n7tGWgVv5uzZM+xWJ+yXSf3enZX/FwCZi7hztWge?=
 =?us-ascii?Q?+HibR6azoD0BviPVMw8j5/WZZPaBrMm8SeeF33AFsj5RrGWIadiGlt3HBAER?=
 =?us-ascii?Q?FYjbCoNZLdE8hz/JL77erBLq012qpjY3wNlgpGgoMo0s0y/FyAqF5igvUSan?=
 =?us-ascii?Q?1Y3JknGpqBUhxtAyQ8cLSj2W1qXVDIoHSHXAWYa7pjF7ADBuLfZ3HNgCu/xp?=
 =?us-ascii?Q?aevzch5SB+nuBaQh222JSdzARflDwd+QEfdq49wZhxV8TrJStP1WZ7NbMG+S?=
 =?us-ascii?Q?o/wVnjibjfSL9m7HEc82zo/bvxOX8v3WIaOhHpif6eidgJeDTHRhP00Akku8?=
 =?us-ascii?Q?dTZrqBMH1D3Sn86qmiQxKMFXExSAoSaoKOgsSExsRQ0Ih+CNNr35JzskaCNu?=
 =?us-ascii?Q?xG1GOLudtHQAgAKjE5NC9TmnNwjiPNos+xUbAMCR4kjvlo0oDgQjmi0G8wo/?=
 =?us-ascii?Q?evWBeSMbgj7tcZj0h/+av2cAWxtA3Gvf4XCEwi8VC72o7l2Zf/7obynimB+G?=
 =?us-ascii?Q?Rl4YDu1Khg7BVpHWFd5rfX6jSfM/niN6KCf0i79Ms5rqVBRk5YTSiPf8JS8K?=
 =?us-ascii?Q?9+Ub0pTympVGJ/qmB64UM91Knb/wMESmDpAf3MfORrGm+D0GRq1xwC7X8nvA?=
 =?us-ascii?Q?Kmqff43AfePIptfJe6IDawd0BdrKqPneZPaa5JIboHRDqda1C3bl0xIJHvCU?=
 =?us-ascii?Q?+yeX+MTd/0T+CMzXURCmmunvRQjsK1p5N65cOJ/G3FvIwG1AE5ai+8HP+SOm?=
 =?us-ascii?Q?SPwuCnf0SKcCmScrUefZrAlYesZZ4jAKbk6hMeikPlYf51OPD3ZG3W2bGIiS?=
 =?us-ascii?Q?B2cTd4F97ScwGiREx+inlG+zG2Ze0SfEcTt+JSTvMzOvCsa1EzZbVtNtx29L?=
 =?us-ascii?Q?4As2Itcs75kcrmf5gN1OApJNbrfCUWlK/BEZzwEffrtraJUNyWFpDLI6OLdO?=
 =?us-ascii?Q?NYY/NUTAiQ/PJhymBRyLTFmx83oA2gw1AMwTBJgU1eXB613qFTzjIg+FklPm?=
 =?us-ascii?Q?D/n2064WZZEdmKhq6Ax4AK3rh8C+YqlhwhLNiGUkb9PJEKgoI1CeIx4W1qdG?=
 =?us-ascii?Q?wRGLGLFHivsM1bAEz82uUTaGVZVhWeuUuo1fOzifnjqMggU9bwdOOAWTnpsW?=
 =?us-ascii?Q?jREUPY54aHGRnKxEQQTXKXYyqAfBv7voQxpGDACVD43w/t/EbSsmcl/aSgLF?=
 =?us-ascii?Q?6eXMS5A+wKukOrM0l8ltG2APEy1+?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:51.3820
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9ca015d-c878-4a30-07cd-08dc7c3f3643
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7205

From: Henry Wang <xin.wang2@amd.com>

Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.

Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.

Update the doc and the golang bindings accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
 docs/man/xl.cfg.5.pod.in             | 16 ++++++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/include/libxl.h                |  5 +++++
 tools/libs/light/libxl_arm.c         |  4 ++--
 tools/libs/light/libxl_types.idl     |  1 +
 tools/xl/xl_parse.c                  |  3 +++
 7 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 8f2b375ce9..ac3f88fd57 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,6 +3072,22 @@ raised.
 
 =back
 
+=over 4
+
+=item B<nr_spis="NR_SPIS">
+
+An optional integer parameter specifying the number of SPIs (Shared
+Peripheral Interrupts) to allocate for the domain. Max is 991 SPIs. If
+the value specified by the `nr_spis` parameter is smaller than the
+number of SPIs calculated by the toolstack based on the devices
+allocated for the domain, or the `nr_spis` parameter is not specified,
+the value calculated by the toolstack will be used for the domain.
+Otherwise, the value specified by the `nr_spis` parameter will be used.
+The number of SPIs should match the highest interrupt ID that will be
+assigned to the domain.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index b9cb5b33c7..fe5110474d 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl)
+x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl)
+xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 5b293755d7..c9e45b306f 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -597,6 +597,7 @@ ArchArm struct {
 GicVersion GicVersion
 Vuart VuartType
 SveVl SveType
+NrSpis uint32
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 62cb07dea6..a3d05c840b 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -308,6 +308,11 @@
  */
 #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SVE_VL 1
 
+/*
+ * libxl_domain_build_info has the arch_arm.nr_spis field
+ */
+#define LIBXL_HAVE_BUILDINFO_ARCH_NR_SPIS 1
+
 /*
  * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
  * 'soft reset' for domains and there is 'soft_reset' shutdown reason
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1cb89fa584..a4029e3ac8 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 
     LOG(DEBUG, "Configure the domain");
 
-    config->arch.nr_spis = nr_spis;
-    LOG(DEBUG, " - Allocate %u SPIs", nr_spis);
+    config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis);
+    LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis);
 
     switch (d_config->b_info.arch_arm.gic_version) {
     case LIBXL_GIC_VERSION_DEFAULT:
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 79e9c656cc..4e65e6fda5 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
                                ("sve_vl", libxl_sve_type),
+                               ("nr_spis", uint32),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index c504ab3711..e3a4800f6e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2935,6 +2935,9 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_long (config, "nr_spis", &l, 0))
+        b_info->arch_arm.nr_spis = l;
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:16:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:16:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729887.1135196 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDe-0007X1-L6; Fri, 24 May 2024 22:16:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729887.1135196; Fri, 24 May 2024 22:16:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDe-0007Vi-Bg; Fri, 24 May 2024 22:16:58 +0000
Received: by outflank-mailman (input) for mailman id 729887;
 Fri, 24 May 2024 22:16:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDc-0006xe-Gp
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:16:56 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 534f98c6-1a1b-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:16:54 +0200 (CEST)
Received: from DM5PR08CA0041.namprd08.prod.outlook.com (2603:10b6:4:60::30) by
 SN7PR12MB7811.namprd12.prod.outlook.com (2603:10b6:806:34f::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 22:16:50 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::a9) by DM5PR08CA0041.outlook.office365.com
 (2603:10b6:4:60::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 22:16:49 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:48 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:48 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:47 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 534f98c6-1a1b-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ns9JCnCAXGoAjQfpP9wTbqYGU7n90JKF+cuIZPfDk/R14fZpmhkxc7Lfyc1c+Zj0gU4kMbg3DVQ5uNluhrqy0QC1uIfO7sukvVSTgSWaHizcFf7YfcE+CYWevdrPUscDIeVLCSc2GR2ZlmMUnZesH1aZPi1jx59TDIn7r2saF2jpC2bU1OCJROwplSWO+nx/WLm/Cf84NWGd7LcfmYZ5OdoK8Af2w23DDIuygvc+flp+aDvQWa1/plWMuvY6kG3bIfx2KiGeHg96iedhe1gl//+3Cliw8cNuod7jGDkQ8LOqyvUo1judm1qLbl9/6wLyM3jSpbGaRJ5dYc5kNByhOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DJzU+Lo23+pblv8U5ELG1eHDNPAMdYWdjjMuN9k4W2I=;
 b=bp5c6uxrCNfQZZipQ4TC9ZG/hqYGc0hO4mq7bqpZqgrL5JcW8seJhm2L2aaHGPkGeVwlsGUQfKz04vTgF9xuQ3C2h+gEkw2AArIELGI6kwrSB/vKevmRYIDM107anhmob+PWDx/ocBBcs90GxXkAGzwM11qgdKmiuv03jF+S60APTK+uOQVSp7723ytAif/rTlqdUTfkw4wEbrvQ2/pYvGnUseGpIaWYrpoviKuST1h7j0qZCEF3vbwu9GY8OCWiQM3Q+AN9bIiYUexKF+FLUn95HcoWX0qm2gtbpcalv7xkS23h7+Nwm8DcgV/3FSQsF7WD/c5WbsVcE61lTHer5w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DJzU+Lo23+pblv8U5ELG1eHDNPAMdYWdjjMuN9k4W2I=;
 b=GiNFzdCzMXUm73p1jFdMZbwrVPzY6M1lm2agrxUnPzjtm3vbUi6PwInsDTbhyk1SCsoGEIUfoqMS4J/xL9LV5yHhchshune1moFCe6xaWW/fn+jFhdbKyO+Fl9d4bz13cVLOkOqkinNBSB0KlduiVl3BeABiKhhmcY2GnKzxrDk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v6 1/7] tools/xl: Correct the help information and exit code of the dt-overlay command
Date: Fri, 24 May 2024 15:16:41 -0700
Message-ID: <20240524221647.2860711-1-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|SN7PR12MB7811:EE_
X-MS-Office365-Filtering-Correlation-Id: 3b1469f8-0f67-42e1-a46d-08dc7c3f34b9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?We6+HSNrH5PCtFIOkHpGgt47i80SS+XBiQvm/anNS7mrXE0NZIsDs+wIJdcw?=
 =?us-ascii?Q?5+ZLaSs55myOP+JkzbHmu4KL7rCMkWewFqo2Fyk6rAX12ecPnGpHjZ/26Yfg?=
 =?us-ascii?Q?C8Ote7p//1M8iWpYtW4jYlety5h20Ru+oP0S1MsKOMHH4dfI1XM0EStbdbfF?=
 =?us-ascii?Q?Tlv4/Kmfs1jr9eW53iOVjW8MJAVifHma0rVtl4FI9RHlavMaiDhXTL8nCd8e?=
 =?us-ascii?Q?mYOqnQxRKPTTitxMvSU8yTUfgdAR5Z6zFr2ZfZg94CEnZxkaHR79Vq+1UmS3?=
 =?us-ascii?Q?E5BCXVuri1RnWJT5hyRCI/2SsZY1GWQy3DJgAbUrx0suLVNnFCC4HhEV+jUi?=
 =?us-ascii?Q?zHNsC/pVU+5DYGMA6FV8UVzpBIOF1Vsfu5ul1YX8lz55ZbjHhoWJra+5ybG0?=
 =?us-ascii?Q?mEwtGdZgdLGPkjvO6U9KQEUEEkrIW4KPqqqEVepjpfDobc6nVAtH3+bxw5T5?=
 =?us-ascii?Q?Evx/1K3WZ1cb44GBS1ZDJcq5KZ9DsgvCEscSbzoAkuhlEIhjjIO2Ufvj1mJe?=
 =?us-ascii?Q?gnqKwBunPzRMYJ9mr0Qx2ZYOLOcYEDau061FdvbJWEG41Bggq/bCmqQtl2It?=
 =?us-ascii?Q?BWiD2K5a2SarXocGPtG8wUx/eiTL1vEwBRts84MPG1kKyhJR6KibCLVaSHpU?=
 =?us-ascii?Q?u/nO1dZu8LDwNDM00k/RGTve8SKJF/CcvRaYF1Nt+FwfFT23jx6h6vpQtNRn?=
 =?us-ascii?Q?ZRkQD4/VTalHvLRBpOmiKYW9uKEiP9qea4KdR4ylkBFa2MVbPzuf0Yt9oQSN?=
 =?us-ascii?Q?GNSecdAAEXFM7bljdq5IxyP6piPIvKnUkA7UZWfA/x9qym+lD0f0DuZA+oxn?=
 =?us-ascii?Q?twDW7w6KNka/W3X3Ig7z4A3hdYvOkwDTHR+G6pLLXc1H/44okZl2sE07jJ7k?=
 =?us-ascii?Q?6ObFOYJ+Ytdy5JDnEkLeG8gmS59h+bTE6wcMcyV2DTLc+Fl1jRZfe4gLoa3o?=
 =?us-ascii?Q?1PzAdTuyq89J/fjWWPwvJ+8swkunyrp/Q5SwGcXK+uHg+g65uybFXdMlc0xy?=
 =?us-ascii?Q?mj60imeCLGUAefkehRGk9M/njRhBmu/sDZDsKbFI6sSK4k9W7iDI/lVoz65h?=
 =?us-ascii?Q?0x0B1PDRhyBr3+q/VcHMomXjoMDFKUtY4UFAsv7VPuOKzgrnlTjDkY7KUtGQ?=
 =?us-ascii?Q?5l2J+vKSDZ6QbK/gqiwwtcj00sTSaMdCRantg1xOaI5ZGDgvP1h6jbOXk4i5?=
 =?us-ascii?Q?+/WjC/opFE1G6nVyzcMrziUIpXFUfRp+wfU1dgK8KchFQFVzchwIwVPAkf45?=
 =?us-ascii?Q?2zdyH9tBOoeMqxz5TZJEyVczFvIZvFNuVG+/Mw2IljLEdEKxbxbzQAIA310X?=
 =?us-ascii?Q?VZH3bu3AJUbYs59v3Qugofqlo5nmq20g3nofn0XUIL3pBZifQsZ7hosrb1Wy?=
 =?us-ascii?Q?Z7rcljk=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:48.8195
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3b1469f8-0f67-42e1-a46d-08dc7c3f34b9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7811

From: Henry Wang <xin.wang2@amd.com>

Fix the name mismatch in the xl dt-overlay command, the
command name should be "dt-overlay" instead of "dt_overlay".
Add the missing "," in the cmdtable.

Fix the exit code of the dt-overlay command, use EXIT_FAILURE
instead of ERROR_FAIL.

Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support")
Suggested-by: Anthony PERARD <anthony@xenproject.org>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 tools/xl/xl_cmdtable.c  | 2 +-
 tools/xl/xl_vmcontrol.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1f3c6b5897 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = {
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
       "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>"
+      "add/remove <.dtbo>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..02575d5d36 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv)
     const int overlay_remove_op = 2;
 
     if (argc < 2) {
-        help("dt_overlay");
+        help("dt-overlay");
         return EXIT_FAILURE;
     }
 
@@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv)
             fprintf(stderr, "failed to read the overlay device tree file %s\n",
                     overlay_config_file);
             free(overlay_dtb);
-            return ERROR_FAIL;
+            return EXIT_FAILURE;
         }
     } else {
         fprintf(stderr, "overlay dtbo file not provided\n");
-        return ERROR_FAIL;
+        return EXIT_FAILURE;
     }
 
     rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:16:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:16:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729888.1135209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDf-0007z0-Oq; Fri, 24 May 2024 22:16:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729888.1135209; Fri, 24 May 2024 22:16:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDf-0007yQ-LT; Fri, 24 May 2024 22:16:59 +0000
Received: by outflank-mailman (input) for mailman id 729888;
 Fri, 24 May 2024 22:16:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDe-0007Iy-Ni
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:16:58 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5588d913-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:16:58 +0200 (CEST)
Received: from DM5PR08CA0025.namprd08.prod.outlook.com (2603:10b6:4:60::14) by
 IA1PR12MB6329.namprd12.prod.outlook.com (2603:10b6:208:3e5::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Fri, 24 May
 2024 22:16:52 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::fb) by DM5PR08CA0025.outlook.office365.com
 (2603:10b6:4:60::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.26 via Frontend
 Transport; Fri, 24 May 2024 22:16:52 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:52 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:50 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:50 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5588d913-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HzIcS/jnelNSjX5wmxlhAnjjAVdohjceaUs9ETjTs5Y+YhOn5igMhInyZZ/EJ1Z5OoyDnhq2SMISzSjRBTq/+cUPxEH3lZbK5XEjt/JwccDSRf9rmelHCz4ggiuWYMtc2nVFk1njrZXcsdZI5MSihSFhnP14iqlYo9BJRlaGYluzA9VivtjA+/3Asn+yb9ZT8nu0N3ZZGo5Hov8q+TD581ijD8qVrZfMXPKZvn3Tp2Lw0pWzgBtNvQJFyOnSddHp6X/XfQJLKRFgdSAGpHzgeCb0UBuIhYUz5ESy+YIXxPAkfx50U7P/thQgPDB1uGfdGIzBODRKXUEoZUkS7V633g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dUxLvykGPrn0TdFclj7NWwLQlt3XRena1NmCNYfl8WY=;
 b=kaBZAr3pU4zrWLveeJm1a1tA0bhHgywJpIeS2nRlAMsHHPnTJeKbx+u3YG2Pqyzn+HE5DXvXqltQn5BiYZSWcspoUKcW/eoS1Rz9lruKIWqN6pqS19KZPQOOfHaYjyE/3LH4lVsTubjnrv1dQXT7y+EraAfPBMhJhPGeSM4Btp1YjMZU4q80fv4SSJC8GoWBHHv7IUhUTFabCTVk/06YJO0+CPf92nxT9/m/gse05/uOd9X0S1Vez58Ejez0OYdCuO91mmCaHo3AakoNWO+IvjGJS5D/dNfQClZnNP1/f2LR1fJSJBMNnts345dJBeNcf8Xnmz0vJcFh9/d/K0QhMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dUxLvykGPrn0TdFclj7NWwLQlt3XRena1NmCNYfl8WY=;
 b=44iYRey/bQX/UqeeEG1oty/E0pXIS2Fa5yCEm5zFSJpI1TQ3ebtVDkmsiqfC6Lc7UokVtlM7RN6rsBvknALGs40MWSpr3MX7bbXf5Vk59Cxdt3q2PYsyZeym4Zg3vJUjxwIqkVhi5uz+oM3P2rWH9t94E3sxwFly/JsKwx3a/sw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH v6 4/7] xen/arm/gic: Allow adding interrupt to running VMs
Date: Fri, 24 May 2024 15:16:44 -0700
Message-ID: <20240524221647.2860711-4-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|IA1PR12MB6329:EE_
X-MS-Office365-Filtering-Correlation-Id: e3ffe385-56df-4e60-27c3-08dc7c3f36e5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?x7bV7ZqjgUF7VWv5tEb0t7r9/1QAF3128hqXshja0lxs7s/Tn52FXT3oFHsS?=
 =?us-ascii?Q?ZT2zhsIwWX9WmVCIbeyGtfYr8jPZjh1JEYvQdtmOoATSEVTGHSBJMLJ2UoVG?=
 =?us-ascii?Q?j6dUkA4J0WItM5O/5BecsfapkAmTqG9XKMGWl/VOXpSRhSUTcP3FOtyHlxKF?=
 =?us-ascii?Q?dpJBx2XkgtsfaDeptG0jvlTVzj7kfIPkaFQJKX/TUy27EvTt0143HRTQExXm?=
 =?us-ascii?Q?BhRH/YDC4bD1Cw/Zs0MRzkxOwVPOzQUmprIMGNWY8+O/76vMs6x03QS/X5if?=
 =?us-ascii?Q?tlL27RkOWIALa9YBK3rFGVxyXZsgLEbndrDAntOMgxAWxfpiGtXHygvUybX6?=
 =?us-ascii?Q?EjchwqpbJVmQ+t62P5McG3mK+lFyL6awNcJJSTaYfewSJSFd3Z1jQwOSyRdc?=
 =?us-ascii?Q?cQvnaOuXa3snhaMz2R7c0K2OsU/loLa2i3BkiYYa00oTrrA/AamDNVtvk4xV?=
 =?us-ascii?Q?tA7x1TC+kuBQVRboW/EdhySZxNBZCeZ/d/QmGSpbKfuKtzY/SetOSN0fOWLU?=
 =?us-ascii?Q?qRLe9jvHdH02rYu17zOatsvltddMKd7TpUX7vwp1H+fzWF29NRtylfYKI2P5?=
 =?us-ascii?Q?A8z5zNFVnE1KC1YPtVCDuYAV0IpxPeeJmUlCQJGpUtL6JoVwVSqul8DnBo9w?=
 =?us-ascii?Q?XXoDUeCq2KHVGDGfajg59FAGsK2PPUB+yoUVbnUu8jzx6YbICMJJrOl+flCM?=
 =?us-ascii?Q?nUwXkUeRQLgiS+9aubFdVsFAJZcSVsA/EaZLt2TPQdxEPG3JI6Lu55MG8+AS?=
 =?us-ascii?Q?y/I3ST/A9pbVxZem8uNBcgNRJQcYCgz+qP4UcqAsExXn5rw/Dvdy2XlD3B93?=
 =?us-ascii?Q?+fJ2yu7bOaT8JHNpNGbYseCEAigDJekk9IDr3d+UT1AZnSSUjy5FMTuy2jMF?=
 =?us-ascii?Q?8HZbr71te46vq01pgwrvo/6WrYLkQOjaNbuEqSniqbYdbqdF45cxq1JRqNG+?=
 =?us-ascii?Q?r5jbCzBWQo6lmNogYCUk4LIQFJkazvrl5T/Qzc1GOsCz3wpj4rxN5ZDWzMsI?=
 =?us-ascii?Q?5aPUQlrM5ZvqkBsvoDNQRR791iDpT7C/T5+Lq0t12xZx+wPgFjQegNFI5wlk?=
 =?us-ascii?Q?3a7laNehWr2PjvNShs/LIG8VQNzFlRrB/jZ1CB2vCZxZy1rWHtsvwqTMGa5u?=
 =?us-ascii?Q?eK+POV4b4FBwTdtAiZQQm6jnuEDu+BrK4ipwsZYFpyQAn1TsWTAJWB+1y7GK?=
 =?us-ascii?Q?fZwqxTtc+MqneLz/2aNx4tdUuHp9MtexrJV/Ocrv0V4YbwPD5vans9K4JpQw?=
 =?us-ascii?Q?wjvMBa3vq3gaEESRQDESTPL/eGk4hpcZ6F7qXcXGwg11p3aP/E04qgUNGb6J?=
 =?us-ascii?Q?zrRnnZk9EEW/qVxAOdiMjmIbeMzAEO1oWAnL4u1hUkjYlo+DRZlhuSt336uG?=
 =?us-ascii?Q?QBeB9YOLN2h+QucIJMfIDN+Yl49R?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:52.4601
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e3ffe385-56df-4e60-27c3-08dc7c3f36e5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6329

From: Henry Wang <xin.wang2@amd.com>

Currently, adding physical interrupts are only allowed at
the domain creation time. For use cases such as dynamic device
tree overlay addition, the adding of physical IRQ to
running domains should be allowed.

Drop the above-mentioned domain creation check. Since this
will introduce interrupt state unsync issues for cases when the
interrupt is active or pending in the guest, therefore for these
cases we simply reject the operation. Do it for both new and old
vGIC implementations.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 xen/arch/arm/gic-vgic.c  | 9 +++++++--
 xen/arch/arm/gic.c       | 8 --------
 xen/arch/arm/vgic/vgic.c | 7 +++++--
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c
index 56490dbc43..b99e287224 100644
--- a/xen/arch/arm/gic-vgic.c
+++ b/xen/arch/arm/gic-vgic.c
@@ -442,9 +442,14 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq,
 
     if ( connect )
     {
-        /* The VIRQ should not be already enabled by the guest */
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
         if ( !p->desc &&
-             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
+             !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) &&
+             !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) )
             p->desc = desc;
         else
             ret = -EBUSY;
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 44c40e86de..b3467a76ae 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
     ASSERT(virq < vgic_num_irqs(d));
     ASSERT(!is_lpi(virq));
 
-    /*
-     * When routing an IRQ to guest, the virtual state is not synced
-     * back to the physical IRQ. To prevent get unsync, restrict the
-     * routing to when the Domain is been created.
-     */
-    if ( d->creation_finished )
-        return -EBUSY;
-
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
     if ( ret )
         return ret;
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index b9463a5f27..6cabd0496d 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu,
 
     if ( connect )                      /* assign a mapped IRQ */
     {
-        /* The VIRQ should not be already enabled by the guest */
-        if ( !irq->hw && !irq->enabled )
+        /*
+         * The VIRQ should not be already enabled by the guest nor
+         * active/pending in the guest.
+         */
+        if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch )
         {
             irq->hw = true;
             irq->hwintid = desc->irq;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:17:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:17:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729889.1135220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDi-0008Gq-5g; Fri, 24 May 2024 22:17:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729889.1135220; Fri, 24 May 2024 22:17:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDi-0008Gh-2D; Fri, 24 May 2024 22:17:02 +0000
Received: by outflank-mailman (input) for mailman id 729889;
 Fri, 24 May 2024 22:17:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDg-0007Iy-S2
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:17:00 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on2060d.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::60d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 55fd81fe-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:16:59 +0200 (CEST)
Received: from DM5PR08CA0054.namprd08.prod.outlook.com (2603:10b6:4:60::43) by
 DM4PR12MB8497.namprd12.prod.outlook.com (2603:10b6:8:180::15) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7611.22; Fri, 24 May 2024 22:16:53 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::38) by DM5PR08CA0054.outlook.office365.com
 (2603:10b6:4:60::43) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.26 via Frontend
 Transport; Fri, 24 May 2024 22:16:53 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:53 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:51 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:51 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55fd81fe-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QeXeaYcAhNT879ugktFaQEjg1g4Ny4UwtfkPJYFyGHHUPBiIoVj/1eGGjO4K9msD5Ugabtoku1MuSVCWyuAJnpP4BdUgZ3KkcN7opchHjnxMfDpEYEXga7XaZqckEDOvLSyWunA3+bWtSouL0psh5VXLX477W+Rfku61Ik7T7ip3SwcOv+8i70QXAV9C1YmKWqRmuiBVDABmSDy0BmgN5U4koNOfibBJ4FDMOvY5qWd5hC7Y2rBQIHONOLJaQvlPF7+qwsp+/+Bbl830bzFNfu2cQpeCEHXhklQzlnlkCxaM3nVGHZrJP06G8zfY26c760ry+mT/p6N+hjFXi8RyDA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=iRI1HqDlL5ZdxdN/WlhV5sNzuCLmoN3HyEJY9Ux8c04=;
 b=lDe0INtBJnnPmVzEUEdO//GSBkgnnZlKn/fMvZzbAqqQ/jA77GeWGMf310HhcgB02GpF+1oK/VtiaFNqQjMEccYPa2/b9ljEa8hcaBKh1A8N/4qy5Kw3IE82P4GEkcRplOha8SlgWwuqSaT185IUYLOXTIVTPT4LjgsspWfxtiVgNpNVbQXBqNtXmYtPbBsGkCjJo/BCN1I/1LUN8cCgx+AgV0OPSWPkhBaPe3h4ahQT0zidzlLy2MueiXfupGhfKGJpRBalM5eTWDn2dsTynVBkCLS8VptHBwg8PXmrdTQrLEgL2UJgNVF3fe74QYLQCVEu/13apVunV7yUNWDQGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iRI1HqDlL5ZdxdN/WlhV5sNzuCLmoN3HyEJY9Ux8c04=;
 b=Yio6nUeEFewAbh3RSiJLaD1gd/5ZV3efbRgmIDUtM1NtzbpzwIW6BVtx6EDlKmWa644AG9G+E3iGJe1qD+9+ENtnMhYG3mse316E6QvSlqDrM3BbnYFcsynxyS9TGUswiQdKM486EkZemrWjDezDCl0p15D4sTF4zO5XYzsQvKY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Stefano Stabellini <stefano.stabellini@amd.com>
Subject: [PATCH v6 5/7] xen/arm: Add XEN_DOMCTL_dt_overlay and device attachment to domains
Date: Fri, 24 May 2024 15:16:45 -0700
Message-ID: <20240524221647.2860711-5-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|DM4PR12MB8497:EE_
X-MS-Office365-Filtering-Correlation-Id: 1afe2021-d29c-4367-48ca-08dc7c3f3768
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?0zS8WKN9gYe0mffDjmraAbod5ztY2uhf1w7dACwXABVzCSN+hAdwAfH0lO49?=
 =?us-ascii?Q?oafYStPymArANWC76Ti9PkC+xxSWfzLBi+2PnLzU7KPXgv0v5vTYo4pUL6RO?=
 =?us-ascii?Q?/r2QttNSQa4wWd2uTc9rs5ufEEsEx5Ttut/ZKs1w707tUtlj4A9LrKO65xqZ?=
 =?us-ascii?Q?pZYGuMwAL+EQ4fMGdb2xBBaKKeuwpYvYLengJd175roUoJevBykCbfI6TX1s?=
 =?us-ascii?Q?Ig2MPrk7lfRguGfLHEf/ijU/4MdPl4jHcAPG5gcf9HTnImdAyrHQbiHyLgsE?=
 =?us-ascii?Q?rgbkJ4xNgTN8EhYLhVqHGSn/RUGYfxR9BoZFfdwrl9pGtcKesAjpIHvEiN9B?=
 =?us-ascii?Q?8DHzsNEi+dll2UDJ7TOOZUvP0+sGtdC9bKDripMYSMag7OIjm3DSc1Ap7KUA?=
 =?us-ascii?Q?r6KuMgqcOS1RklRsbZxG9HtehIPhZDQUdpVrvJatvF1eCOTpdfysu1d4MP4T?=
 =?us-ascii?Q?b1lnk392S9GIVqIoLnmW31kqlNwIP8bK4XM3jRH31wLdwYwCgwHRZpWFpn/G?=
 =?us-ascii?Q?AkF4aBdMXs/QU7RUUEHnnCCNwA/ebl1EB2u4X0jI7PfYuTbWSki0/+kXv/l5?=
 =?us-ascii?Q?Ka1+PKazuapvLg8dMjDA+PtA/IgTpmzggro3KDHbjsJixbENgqzAzFXtn7fe?=
 =?us-ascii?Q?RSHfSLN39W62GgKONDDPApFKFPpGIb3BtzPkeIBHSHBgyj3g6ouYEK2S5Qja?=
 =?us-ascii?Q?9iD+ueIeR8hr/fZRwhG8EIZKJ6tcNDU5i49/KhV+CKPFhOp51DZHJ7PArzW3?=
 =?us-ascii?Q?B9lcner3rE9fuu4h7HDXFsfmPVgYhdTdwP4IGfMj6+F7mPzz0C7R6EdwYvcV?=
 =?us-ascii?Q?Xn7Fj0hEXFQoIYRpo0xUPLixH8Iiajgi5KxAtwzJldjZM9r6UFuCml5Mopgi?=
 =?us-ascii?Q?JiVlG0Yog0LBHnco5MqNhrUZ59iqKZ/j60gkRR8Rrf7O5S8OYSF2V36t3X3i?=
 =?us-ascii?Q?O76+ykXJQbXJKZaZHi1kAUut91rzv5xiEjoLh8NQeCA3r03RaNjO8AMzC1hD?=
 =?us-ascii?Q?lUNTZJ/WT+I2vE3w6Wf0gBrDqnc3JLGGWW5gsIal1T+D6dGS1H6XR8Fjsan4?=
 =?us-ascii?Q?HmMY00zRNjNMvcFw8w7P4CKAf3m9nw1VowL47YxCbEbxL7oq/U4rVeO1yQlf?=
 =?us-ascii?Q?tnsZYL6HxrKUe3c2TicAKNC8EqZXy8AtuhtjARxsyxI2++fP2YnJLee45bdi?=
 =?us-ascii?Q?vp20hcCZ7BUib4D1d+ZkWNMFm6IgV4N/i83ytL+ItaV+QHarLGwe4mn/rP/b?=
 =?us-ascii?Q?8wvlvfN5jIfukUJuhFGdVD6e7bBqFo5Cxn8q042xzEZXi68gIvEhyycUe6U+?=
 =?us-ascii?Q?ciBP/RamkTKvj7S8CCSo9r8z3W84fNsfkUMmXIhWMi79OjPXtwkicdZCyLYd?=
 =?us-ascii?Q?+Gg2xf4=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:53.3039
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1afe2021-d29c-4367-48ca-08dc7c3f3768
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8497

From: Henry Wang <xin.wang2@amd.com>

In order to support the dynamic dtbo device assignment to a running
VM, the add/remove of the DT overlay and the attach/detach of the
device from the DT overlay should happen separately. Therefore,
repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
overlay to Xen device tree, instead of assigning the device to the
hardware domain at the same time. It is OK to change the sysctl behavior
as this feature is experimental so changing sysctl behavior and breaking
compatibility is OK.

Add the XEN_DOMCTL_dt_overlay with operations
XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment to the domain.

The hypervisor firstly checks the DT overlay passed from the toolstack
is valid. Then the device nodes are retrieved from the overlay tracker
based on the DT overlay. The attach of the device is implemented by
mapping the IRQ and IOMMU resources. All devices in the overlay are
assigned to a single domain.

Also take the opportunity to make one coding style fix in sysctl.h.
Introduce DT_OVERLAY_MAX_SIZE and use it to avoid repetitions of
KB(500).

xen,reg is to be used to handle non-1:1 mappings but it is currently
unsupported. For now return errors for not-1:1 mapped domains.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 xen/arch/arm/domctl.c        |   3 +
 xen/common/dt-overlay.c      | 211 ++++++++++++++++++++++++++---------
 xen/include/public/domctl.h  |  16 ++-
 xen/include/public/sysctl.h  |  11 +-
 xen/include/xen/dt-overlay.h |   8 ++
 5 files changed, 189 insertions(+), 60 deletions(-)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad56efb0f5..12a12ee781 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -5,6 +5,7 @@
  * Copyright (c) 2012, Citrix Systems
  */
 
+#include <xen/dt-overlay.h>
 #include <xen/errno.h>
 #include <xen/guest_access.h>
 #include <xen/hypercall.h>
@@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+    case XEN_DOMCTL_dt_overlay:
+        return dt_overlay_domctl(d, &domctl->u.dt_overlay);
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c
index 9cece79067..d53b4706cd 100644
--- a/xen/common/dt-overlay.c
+++ b/xen/common/dt-overlay.c
@@ -15,6 +15,8 @@
 #include <xen/libfdt/libfdt.h>
 #include <xen/xmalloc.h>
 
+#define DT_OVERLAY_MAX_SIZE KB(500)
+
 static LIST_HEAD(overlay_tracker);
 static DEFINE_SPINLOCK(overlay_lock);
 
@@ -356,6 +358,42 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path)
     return 0;
 }
 
+/* This function should be called with the overlay_lock taken */
+static struct overlay_track *
+find_track_entry_from_tracker(const void *overlay_fdt,
+                              uint32_t overlay_fdt_size)
+{
+    struct overlay_track *entry, *temp;
+    bool found_entry = false;
+
+    ASSERT(spin_is_locked(&overlay_lock));
+
+    /*
+     * First check if dtbo is correct i.e. it should one of the dtbo which was
+     * used when dynamically adding the node.
+     * Limitation: Cases with same node names but different property are not
+     * supported currently. We are relying on user to provide the same dtbo
+     * as it was used when adding the nodes.
+     */
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+        {
+            found_entry = true;
+            break;
+        }
+    }
+
+    if ( !found_entry )
+    {
+        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+               " Operation is supported only for prior added dtbo.\n");
+        return NULL;
+    }
+
+    return entry;
+}
+
 /* Check if node itself can be removed and remove node from IOMMU. */
 static int remove_node_resources(struct dt_device_node *device_node)
 {
@@ -485,8 +523,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
                                         uint32_t overlay_fdt_size)
 {
     int rc;
-    struct overlay_track *entry, *temp, *track;
-    bool found_entry = false;
+    struct overlay_track *entry;
 
     rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
     if ( rc )
@@ -494,29 +531,10 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt,
 
     spin_lock(&overlay_lock);
 
-    /*
-     * First check if dtbo is correct i.e. it should one of the dtbo which was
-     * used when dynamically adding the node.
-     * Limitation: Cases with same node names but different property are not
-     * supported currently. We are relying on user to provide the same dtbo
-     * as it was used when adding the nodes.
-     */
-    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
-    {
-        if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
-        {
-            track = entry;
-            found_entry = true;
-            break;
-        }
-    }
-
-    if ( !found_entry )
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
     {
         rc = -EINVAL;
-
-        printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
-               " Removing nodes is supported only for prior added dtbo.\n");
         goto out;
 
     }
@@ -620,15 +638,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
             return -EFAULT;
         }
 
-        rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c,
-                           tr->iomem_ranges,
-                           tr->irq_ranges);
         write_unlock(&dt_host_lock);
-        if ( rc )
-        {
-            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
-            return rc;
-        }
 
         /* Keep overlay_node address in tracker. */
         tr->nodes_address[j] = (unsigned long)overlay_node;
@@ -638,9 +648,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path)
 }
 /*
  * Adds device tree nodes under target node.
- * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
- * is done to avoid the removal of device_tree generation, iomem regions mapping
- * to hardware domain done by handle_node().
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened.
  */
 static long handle_add_overlay_nodes(void *overlay_fdt,
                                      uint32_t overlay_fdt_size)
@@ -774,20 +782,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
         goto err;
     }
 
-    tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0);
-    if (tr->irq_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IRQ rangeset failed");
-        goto err;
-    }
-
-    tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0);
-    if (tr->iomem_ranges == NULL)
-    {
-        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
-        goto err;
-    }
-
     rc = add_nodes(tr, nodes_full_path);
     if ( rc )
     {
@@ -843,14 +837,83 @@ static long handle_add_overlay_nodes(void *overlay_fdt,
     xfree(tr->nodes_address);
     xfree(tr->fdt);
 
-    rangeset_destroy(tr->irq_ranges);
-    rangeset_destroy(tr->iomem_ranges);
-
     xfree(tr);
 
     return rc;
 }
 
+static long handle_attach_overlay_nodes(struct domain *d,
+                                        const void *overlay_fdt,
+                                        uint32_t overlay_fdt_size)
+{
+    int rc;
+    unsigned int j;
+    struct overlay_track *entry;
+
+    rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+    if ( rc )
+        return rc;
+
+    spin_lock(&overlay_lock);
+
+    entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size);
+    if ( entry == NULL )
+    {
+        rc = -EINVAL;
+        goto out;
+    }
+
+    entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0);
+    if (entry->irq_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IRQ rangeset failed");
+        goto out;
+    }
+
+    entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0);
+    if (entry->iomem_ranges == NULL)
+    {
+        rc = -ENOMEM;
+        printk(XENLOG_ERR "Creating IOMMU rangeset failed");
+        goto out;
+    }
+
+    for ( j = 0; j < entry->num_nodes; j++ )
+    {
+        struct dt_device_node *overlay_node;
+
+        overlay_node = (struct dt_device_node *)entry->nodes_address[j];
+        if ( overlay_node == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+
+        write_lock(&dt_host_lock);
+        rc = handle_device(d, overlay_node, p2m_mmio_direct_c,
+                           entry->iomem_ranges, entry->irq_ranges);
+        write_unlock(&dt_host_lock);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+            goto out;
+        }
+    }
+
+    spin_unlock(&overlay_lock);
+
+    return 0;
+
+ out:
+    spin_unlock(&overlay_lock);
+
+    rangeset_destroy(entry->irq_ranges);
+    rangeset_destroy(entry->iomem_ranges);
+
+    return rc;
+}
+
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     long ret;
@@ -860,7 +923,8 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
          op->overlay_op != XEN_SYSCTL_DT_OVERLAY_REMOVE )
         return -EOPNOTSUPP;
 
-    if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) )
+    if ( op->overlay_fdt_size == 0 ||
+         op->overlay_fdt_size > DT_OVERLAY_MAX_SIZE )
         return -EINVAL;
 
     if ( op->pad[0] || op->pad[1] || op->pad[2] )
@@ -890,6 +954,49 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
     return ret;
 }
 
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op)
+{
+    long ret;
+    void *overlay_fdt;
+
+    if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        return -EOPNOTSUPP;
+
+    if ( op->overlay_fdt_size == 0 ||
+         op->overlay_fdt_size > DT_OVERLAY_MAX_SIZE )
+        return -EINVAL;
+
+    if ( op->pad[0] || op->pad[1] || op->pad[2] )
+        return -EINVAL;
+
+    /* TODO: add support for non-1:1 domains using xen,reg */
+    if ( !is_domain_direct_mapped(d) )
+        return -EOPNOTSUPP;
+
+    overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+    if ( overlay_fdt == NULL )
+        return -ENOMEM;
+
+    ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+    if ( ret )
+    {
+        gprintk(XENLOG_ERR, "copy from guest failed\n");
+        xfree(overlay_fdt);
+
+        return -EFAULT;
+    }
+
+    if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_ATTACH )
+        ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size);
+    else
+        ret = -EOPNOTSUPP;
+
+    xfree(overlay_fdt);
+
+    return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a33f9ec32b..6cd39ea46d 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -21,7 +21,7 @@
 #include "hvm/save.h"
 #include "memory.h"
 
-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
 
 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -1190,6 +1190,16 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+#if defined(__arm__) || defined(__aarch64__)
+struct xen_domctl_dt_overlay {
+    XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
+    uint32_t overlay_fdt_size;              /* IN: Overlay dtb size. */
+#define XEN_DOMCTL_DT_OVERLAY_ATTACH                1
+    uint8_t overlay_op;                     /* IN: Attach. */
+    uint8_t pad[3];                         /* IN: Must be zero. */
+};
+#endif
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1277,6 +1287,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vmtrace_op                    84
 #define XEN_DOMCTL_get_paging_mempool_size       85
 #define XEN_DOMCTL_set_paging_mempool_size       86
+#define XEN_DOMCTL_dt_overlay                    87
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1339,6 +1350,9 @@ struct xen_domctl {
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
         struct xen_domctl_paging_mempool    paging_mempool;
+#if defined(__arm__) || defined(__aarch64__)
+        struct xen_domctl_dt_overlay        dt_overlay;
+#endif
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index febaa4b16a..3a6e7d48f0 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
 DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
 /*
  * XEN_SYSCTL_dt_overlay
- * Performs addition/removal of device tree nodes under parent node using dtbo.
- * This does in three steps:
- *  - Adds/Removes the nodes from dt_host.
- *  - Adds/Removes IRQ permission for the nodes.
- *  - Adds/Removes MMIO accesses.
+ * Performs addition/removal of device tree nodes under parent node using dtbo
+ * from dt_host.
  */
 struct xen_sysctl_dt_overlay {
     XEN_GUEST_HANDLE_64(const_void) overlay_fdt;  /* IN: overlay fdt. */
@@ -1265,7 +1262,7 @@ struct xen_sysctl {
         struct xen_sysctl_cpu_policy        cpu_policy;
 #endif
 
-#if defined(__arm__) || defined (__aarch64__)
+#if defined(__arm__) || defined(__aarch64__)
         struct xen_sysctl_dt_overlay        dt_overlay;
 #endif
         uint8_t                             pad[128];
diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h
index c0567741ee..b2cedf3f2a 100644
--- a/xen/include/xen/dt-overlay.h
+++ b/xen/include/xen/dt-overlay.h
@@ -39,15 +39,23 @@ struct overlay_track {
 };
 
 struct xen_sysctl_dt_overlay;
+struct xen_domctl_dt_overlay;
 
 #ifdef CONFIG_OVERLAY_DTB
 long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op);
+long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op);
 #else
 #include <xen/errno.h>
 static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op)
 {
     return -EOPNOTSUPP;
 }
+
+static inline long dt_overlay_domctl(struct domain *d,
+                                     struct xen_domctl_dt_overlay *op)
+{
+    return -EOPNOTSUPP;
+}
 #endif
 
 #endif /* __XEN_DT_OVERLAY_H__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:17:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729890.1135230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDj-00006b-JR; Fri, 24 May 2024 22:17:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729890.1135230; Fri, 24 May 2024 22:17:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDj-00006Q-Bq; Fri, 24 May 2024 22:17:03 +0000
Received: by outflank-mailman (input) for mailman id 729890;
 Fri, 24 May 2024 22:17:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDh-0007Iy-SJ
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:17:01 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on2060c.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::60c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5761ee32-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:17:00 +0200 (CEST)
Received: from DM5PR08CA0042.namprd08.prod.outlook.com (2603:10b6:4:60::31) by
 PH0PR12MB7888.namprd12.prod.outlook.com (2603:10b6:510:28b::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 22:16:51 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::a8) by DM5PR08CA0042.outlook.office365.com
 (2603:10b6:4:60::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 22:16:50 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:50 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:49 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:48 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5761ee32-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HglzBUSu0Q3uLWrJxDq4dyGWmR3pEAjys9pbN6rERGCSdO8hH/HEyQ1/N0Ni3LIeYLlkglg981luM1UuEDf+b6fIlgkRvFNTcD9MA47iYzppFucr2Qlx5kC/e819XcBXvW9rPmFkxwjKHAsurg21PEGevJfH+dGhFa0wfBL/Ew8A944cI5DRopBnbc13FIh4Na3Qsnhv2jXynWcKDezjNDR6U34EWAvfAf4Mvx9lbouLHuhxuOtY7M3XLi9EDjZsPE5rVLv1DV2Wf/7EQOnuuUGOsI4WqYoqvKb6sYjlu0bCCzUOYUferjo/z2KujU2Y5tS0bYaOFR83xoOs9uXktg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xRJXSt/i9q6c1C8Y7abmDgKw7TBf2B9wiQ/rWE+b2Go=;
 b=D/JdGi+CtOOBbF8Anrxl7pSdmvnLsoE+IhUc9SJ9OXzBF+hO3lw6Haf6oVnzRzLCa4nfnNngS7Hu43T0lxLQqHfSL0URPwLQ1OybuI+KVYc/ZPGEMHbuWMVUv46M4uaYMiULFyoNk3wctF9yv3bl8+rAorB1wlBzF0T0wUs3nJ2jx7dT+hD45+5Tm/3OZDPU/cNHOMHrBpgk6/dpj/KYR9YZ6DOqCSbuASM0uPDIy5Fm0BU7i6YrAjLfDG7hGqG+6k5YzbR/TeKkb1oi3nEIR4se4/Ua63Hb4OTQV++50eS/ExcjjBb3dBLe/bBf0z/86Gol1+8Ujfl1uyOsYEg2aw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xRJXSt/i9q6c1C8Y7abmDgKw7TBf2B9wiQ/rWE+b2Go=;
 b=kyEbJu5MxFV3zc8+DDSkzY2Gt4n/wMqXF4TCPw+k6xPNtV9FdJAKdUFOjmNdC191AgLFovxVICh/8BWZtLVLXSkYy/F4qu2IjMBIwZhnED4PFn3C3QZmlrd+rrqRbjY5/yqErTrY0VMJKJAEsq5j4rgombQNFg6+rQ46Y6TG9nA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Julien Grall
	<jgrall@amazon.com>
Subject: [PATCH v6 2/7] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs
Date: Fri, 24 May 2024 15:16:42 -0700
Message-ID: <20240524221647.2860711-2-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|PH0PR12MB7888:EE_
X-MS-Office365-Filtering-Correlation-Id: c2982c3e-5f43-45cf-85fd-08dc7c3f359e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?VnEuSgH1Nm9hpjmMVJ2Ssfx+PXLQKIGbFygjMcZWfXmTX6CZwojlSidlxN8e?=
 =?us-ascii?Q?EKmu7MIaZFZd1EgPWGpaJItUmEz/oTpI+vCib6LJLfpVkhucRFFLdRM22cT2?=
 =?us-ascii?Q?lvFhaNWgMvKr/d0LinGaY2RmTOJB+XzlJDdBe1FJ99mo3/xbAbjLq6tFKJNh?=
 =?us-ascii?Q?54AkGtA0c1AgDZoToClT9VtYrl949DVZxt8YfZjKio8QDHtcPXGiKs2kXBDi?=
 =?us-ascii?Q?qbsM83ju+J522cZpG8kmJuz18/4XmSKeRulnlY4XIto4lsfTeKNuDF1NWU74?=
 =?us-ascii?Q?LgDq2FXA+j88/uXig5QGpZRuVRBM6q6KXoFf/BfXxj4KxdfAS6QwYF0mW1XX?=
 =?us-ascii?Q?fAi9vyf2pPs03ZyjMearB/kuRZeh1qo64Bc8qWyg6d/Wen7ohzO8AyAA1egE?=
 =?us-ascii?Q?EBoukhdxuvPztyLuVpb4urhPSTmkTmiSpzPd8JOVKXinojs3Bo8dwicFs61b?=
 =?us-ascii?Q?sWFpzsFmtPp6GjXrkBHiDiRDrrfTxBP3/MqVGa0i5CkIyN+vT9A3usprz96z?=
 =?us-ascii?Q?xKrPpS921mOyeWomn8VuegPyipqvGUZPp8VvyF+XDjhCd4cG1vPqUOfWU7Bz?=
 =?us-ascii?Q?2tUL97e8ubOSS/yss7syRmtrf5utsVO51Vp63RzZdS7taEcqg61QmH0G7/4c?=
 =?us-ascii?Q?ZR5u0dlZZmm6IkyeLsi/OayOY0kgxHbKPCgBL2dcAIinJlk56zctwgPcD6YT?=
 =?us-ascii?Q?T27bS82XfdiHk1AZ5+Po+f00bfpEfNbroqxjgKfdzJpNeZd4CT4E59KPHwzN?=
 =?us-ascii?Q?TialEmqMXu+ZUpP3pX1yaKtoGftqY8LP0rfZDWv5V/Nx6kwbMR3C/laqFaSc?=
 =?us-ascii?Q?ll0162PheXCZjSjOKkTXoWvlHnjD97HTcORnQZ0LtDPyk66g1J+UXC8/dE7b?=
 =?us-ascii?Q?Zj2U4xu+KdsaGViGw3pA7PuXW3V2M9hpOHP+0wORWfmvQ290oXJFkf/MvIrW?=
 =?us-ascii?Q?3dGQfbxlcAtD4iwgswkLMdsv04ZAHaArU1d6a7zCG7aO2gIX7oEiJIMmnsyj?=
 =?us-ascii?Q?hAJYdvjPyVEviZmVqxkolrKo4U8jkXar/oXXMq/DY4/GyE/U3LtaOLn8QQ+J?=
 =?us-ascii?Q?dq/OfG4Zx+5KoUBEC+/Ik/YKe11c8pwi2vgpmtwukpVTPw9Rbftu63Zxz0zq?=
 =?us-ascii?Q?DwTKIsmyIlpqvdOJ8np03B2bv+SzNPyCBU0q9mxkq/GbgDYhhrBXV+PEEmqy?=
 =?us-ascii?Q?UwS1pXk+FppK9hLLu+1MbMOP1QBaa6Eh/nbcforwtQbGwsEQxiLAEJ4xHDHy?=
 =?us-ascii?Q?wGsyKWSbTZf+gmN9WDMqavy7dcK3CEfqcvGHw0dwZ2UxlUrhloj+Yld9h2xh?=
 =?us-ascii?Q?wXhSOUjdMEH0hslYOF7fu3aJnM/49ZcCOhKf3TVtjvFeJVEHam5dzp0K62Dc?=
 =?us-ascii?Q?fE/pDOwDhJiFBIt08duxCU2T2NTZ?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:50.3039
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c2982c3e-5f43-45cf-85fd-08dc7c3f359e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7888

From: Henry Wang <xin.wang2@amd.com>

There are some use cases in which the dom0less domUs need to have
the XEN_DOMCTL_CDF_iommu set at the domain construction time. For
example, the dynamic dtbo feature allows the domain to be assigned
a device that is behind the IOMMU at runtime. For these use cases,
we need to have a way to specify the domain will need the IOMMU
mapping at domain construction time.

Introduce a "passthrough" DT property for Dom0less DomUs following
the same entry as the xl.cfg. Currently only provide two options,
i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain
construction time based on the property.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 docs/misc/arm/device-tree/booting.txt | 16 ++++++++++++++++
 xen/arch/arm/dom0less-build.c         | 11 +++++++++--
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..f1fd069c87 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -260,6 +260,22 @@ with the following properties:
     value specified by Xen command line parameter gnttab_max_maptrack_frames
     (or its default value if unspecified, i.e. 1024) is used.
 
+- passthrough
+
+    A string property specifying whether IOMMU mappings are enabled for the
+    domain and hence whether it will be enabled for passthrough hardware.
+    Possible property values are:
+
+    - "enabled"
+    IOMMU mappings are enabled for the domain. Note that this option is the
+    default if the user provides the device partial passthrough device tree
+    for the domain.
+
+    - "disabled"
+    IOMMU mappings are disabled for the domain and so hardware may not be
+    passed through. This option is the default if this property is missing
+    and the user does not provide the device partial device tree for the domain.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..5830a7051d 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -848,6 +848,8 @@ static int __init construct_domU(struct domain *d,
 void __init create_domUs(void)
 {
     struct dt_device_node *node;
+    const char *dom0less_iommu;
+    bool iommu = false;
     const struct dt_device_node *cpupool_node,
                                 *chosen = dt_find_node_by_path("/chosen");
 
@@ -895,8 +897,13 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") &&
-             iommu_enabled )
+        if ( !dt_property_read_string(node, "passthrough", &dom0less_iommu) &&
+             !strcmp(dom0less_iommu, "enabled") )
+            iommu = true;
+
+        if ( iommu_enabled &&
+             (iommu || dt_find_compatible_node(node, NULL,
+                                               "multiboot,device-tree")) )
             d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729891.1135235 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDk-0000Br-2q; Fri, 24 May 2024 22:17:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729891.1135235; Fri, 24 May 2024 22:17:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDj-0000AD-RT; Fri, 24 May 2024 22:17:03 +0000
Received: by outflank-mailman (input) for mailman id 729891;
 Fri, 24 May 2024 22:17:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDi-0007Iy-SM
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:17:02 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 56dc309d-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:17:00 +0200 (CEST)
Received: from DM5PR08CA0034.namprd08.prod.outlook.com (2603:10b6:4:60::23) by
 CY5PR12MB6179.namprd12.prod.outlook.com (2603:10b6:930:24::22) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7611.26; Fri, 24 May 2024 22:16:55 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::75) by DM5PR08CA0034.outlook.office365.com
 (2603:10b6:4:60::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.25 via Frontend
 Transport; Fri, 24 May 2024 22:16:54 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:54 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:53 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:53 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56dc309d-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g1WOooZrS4FH+fyJ4SSehSaqtVvNVnbUijGatl/DGrn68l53YfnP8aBy6pCVotcgP0MzVC5O5aCfGywTSoEGYgvSGDDyM2WFbIpsj9s4+ib1Rd1agRBHvfvoWzcu6tOAgYH58Dp8Z+KTDoKA4cYZ8xXOTP4v2gRhiKuCeYoDQCf9RCDf8OC6xzuJHRoTMLwbeFL9JuL2DWXoFrWccFuCAaJKrU+obhl88HaGrK7oXDqarXx5ysV4SIq3wUgJ2bsDh935baZGdhgDYnZZy7xEtAYhAQZlJz6PgckpIb7lwwXaBtiiBYnslEfv04elBYo9rhM+H794tfh+DVAP53PbOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=B3jJMlOrEINeOqECOvhj7YrjOfi6yjCyyFlom/2lhcc=;
 b=AHLzFVxfdmPKSLY1YtAaJiQufbIrNc+5WpangG7AxgdIT1xQptvcjPMsifLBQzvZk46sJHhsUZltDuNJ3g/PxePAd+KfbP3H7tBlO91bOLIAtdXjIksoxZ0jX8kEU9jP5fH9Y8KqgTjodQ27a2/sFcojoYtoJXq1Vr/cFfj2UmNwsTe9ggceC3CvDmm+1ShCmqW4LKoByT4CHhntNcFFqr85367OSRg/FGObT3zsBq1NmjQ+urdPL/5hS6AGd9kmeohveRRwhW/DiwTa28szsngWFZ46MMHwT0ssiIMNiCs2ue/AyMQT0zjcXIG4yNgb9tqzjBJK2Nvd2cRo5Jm+/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=B3jJMlOrEINeOqECOvhj7YrjOfi6yjCyyFlom/2lhcc=;
 b=FDgOXMKihEKrR3R36cgnYUEr4THgczch2FnwFnc1Zi3ZExA01ySutsrcWZB01iybLOrd2MkfqlYi6cdxRj6ivpmRJPEVz/XoVCvS1sYyMlFPa8eyhX06LlOXGcNzfnMKDOQ+EVnYr5pVw1oF71zLAHZ/IhB2k8GdCnLIT8yghNs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Vikram Garhwal <fnu.vikram@xilinx.com>,
	"Stefano Stabellini" <stefano.stabellini@xilinx.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v6 7/7] docs: Add device tree overlay documentation
Date: Fri, 24 May 2024 15:16:47 -0700
Message-ID: <20240524221647.2860711-7-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|CY5PR12MB6179:EE_
X-MS-Office365-Filtering-Correlation-Id: cc71ca9e-005e-4f8d-eef1-08dc7c3f3851
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?deCGLzX/V1y8E1Tqx8Fh3fF2X4SFWVIXBAZZTwvkHrIvwzSx9AcJYK+dzWOX?=
 =?us-ascii?Q?+QriQOY6UE4ri6HE0WpZWblf64Skvx23+4xd0KlJHy63+jfTGXoRBz0L94CM?=
 =?us-ascii?Q?vU/pXAX5kHCjV0QEl8+b5BKr8iJLY1+Wk924OcjWwo6Q4OnCyf6e8eeljC7A?=
 =?us-ascii?Q?43WlawRJQwIzzEBQxSFU0/WohKSfq1dk5og8URauTXAeiCkGBsqDAXavhWcs?=
 =?us-ascii?Q?iB5lRm62iQ7CB7ZEMwS3x4PIrVGsL0EcaE0ts4vwZENa7BYIA2D7zQpQwEaL?=
 =?us-ascii?Q?mQ3DjfrujikEg/ajUmzBNbdAoi+kWc2ndWKFl0EQAiB8DVo6RIhYC8GH7qXt?=
 =?us-ascii?Q?Fw4n/iuyAX+3TacF1tiA0s9Ro6xqns3PmDqWBXZXz0wyb8gn1oWmwIjnD/CI?=
 =?us-ascii?Q?bnOgzZ/p3M/eNgm5hvWvn/0CI198tBUMAmwMrA/qYTGtFWWLebWagMq+jreL?=
 =?us-ascii?Q?m+6xIQKdgwDePDicDePYGXm+L4Aio8qagKHInO9S8IpQ1P/grvLT5BtaOgo4?=
 =?us-ascii?Q?07H9nKOPiqC3PhLFG/OvxT9ggf2hUT5fbIg7QdYnU4hnkf1Tig0Hi1gI6D1Z?=
 =?us-ascii?Q?f1WzSfpsS1qGXreJvhXQjbwKPKd1nFCxJCugqozB9fiGtFlaxtluS+VxQCDa?=
 =?us-ascii?Q?f6zKFyygE1Ajtt5nhhX0WYAFBTyj1XSONKqstknPmpIMHAeBimJH+FY9ZPQ9?=
 =?us-ascii?Q?7MELPM9j9EPB391ByY91WelFad13kl9UopNCfLqxxLnjGHy+W02gaaUyKVAf?=
 =?us-ascii?Q?OhVcfYjlLaIgEhoQtPPZs3coeW8vOsaSrdeoVOm7nVrd49afc/9z86h6BJ4E?=
 =?us-ascii?Q?tCjYTNp2N6i45czoGbFc7CudCpNhyb9TZbgKKuFuH1t1mAWh0UV/6Oa9I0Z0?=
 =?us-ascii?Q?4PgqgplJmOHYOBQDQrNMHKVaNoH1XVqd8wDXo6QUbjohM8d5KnWEEvMRaDNC?=
 =?us-ascii?Q?UhT4FvuesX5j44SFxHIk3I6diVlpOFkbY4C3UYlkMyf5MACHphLtd6/ZhodE?=
 =?us-ascii?Q?6d9t1F8n2eRZbFcPhgecu6qZ5VPeNnVuhxCEfNNgkhtgYmT1adIYYaOmGmKr?=
 =?us-ascii?Q?HvFuheVTbHYFwuP1htWGVS4IasF2F6XChpS5rhIfRVi4mXpD9tHAiCp2i8Ny?=
 =?us-ascii?Q?41ynCKbmhCxv1WuVMtgcN6WWRZB81PmlVkA3pILVy/no/QjAQMZokTAvpuxv?=
 =?us-ascii?Q?FG0qCsVbPb1sjj14KYFtG0BIWzbUz9hPn2BsePcm7ieitz81SpoWXcjiX0ns?=
 =?us-ascii?Q?+/yw7UaWyZn6eiAd9HufssvZURQ8vG51dNVWfYivAxq/juPb8noNfTfC76NV?=
 =?us-ascii?Q?PCbcHvffcem74FQ/46HeJidN?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:54.8195
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: cc71ca9e-005e-4f8d-eef1-08dc7c3f3851
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6179

From: Vikram Garhwal <fnu.vikram@xilinx.com>

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
 docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)
 create mode 100644 docs/misc/arm/overlay.txt

diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
new file mode 100644
index 0000000000..ef3ef792f7
--- /dev/null
+++ b/docs/misc/arm/overlay.txt
@@ -0,0 +1,82 @@
+# Device Tree Overlays support in Xen
+
+Xen experimentally supports dynamic device assignment to running
+domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device
+tree, and attaching them to a running domain with given $domid.
+
+Dynamic node assignment works in two steps:
+
+## Add/Remove device tree overlay to/from Xen device tree
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
+
+## Attach device from the DT overlay to domain
+
+1. Xen tools check the dtbo given and parse all other user provided arguments
+2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
+3. Xen hypervisor attach the device to the user-provided $domid by
+   mapping node resources in the DT overlay.
+
+# Examples
+
+Here are a few examples on how to use it.
+
+## Dom0 device add
+
+For assigning a device tree overlay to Dom0, user should firstly properly
+prepare the DT overlay. More information about device tree overlays can be
+found in [1]. Then, in Dom0, enter the following:
+
+    (dom0) xl dt-overlay add overlay.dtbo
+
+This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
+
+To assign the newly added device from the dtbo to Dom0:
+
+    (dom0) xl dt-overlay attach overlay.dtbo 0
+
+Next, if the user wants to add the same device tree overlay to dom0
+Linux, execute the following:
+
+    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally if needed, the relevant Linux kernel drive can be loaded using:
+
+    (dom0) modprobe module_name.ko
+
+## DomU device add/remove
+
+All the nodes in dtbo will be assigned to one domain. The user will need
+to prepare a different dtbo for the domU. For example, the
+`interrupt-parent` property of the DomU overlay should be changed to the
+Xen hardcoded value `0xfde8` and the xen,reg property should be added to
+specify the address mappings. If the domain is not 1:1 mapped, xen,reg
+must be present. See the xen,reg format description in
+docs/misc/arm/passthrough.txt. Below assumes the properly written DomU
+dtbo is `overlay_domu.dtbo`.
+
+You need to set the `passthrough` property in the xl config file if you
+plan to use DT overlay and devices requiring an IOMMU.
+
+User will also need to modprobe the relevant drivers. For already
+running domains, the user can use the xl dt-overlay attach command,
+example:
+
+    (dom0) xl dt-overlay add overlay.dtbo            # If not executed before
+    (dom0) xl dt-overlay attach overlay_domu.dtbo $domid
+    (dom0) xl console $domid                         # To access $domid console
+
+Next, if the user needs to modify/prepare the overlay.dtbo suitable for
+the domU:
+
+    (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
+    (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
+
+Finally, if needed, the relevant Linux kernel drive can be probed:
+
+    (domU) modprobe module_name.ko
+
+[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:17:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:17:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729892.1135246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDl-0000Z3-Bo; Fri, 24 May 2024 22:17:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729892.1135246; Fri, 24 May 2024 22:17:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdDl-0000Y2-7c; Fri, 24 May 2024 22:17:05 +0000
Received: by outflank-mailman (input) for mailman id 729892;
 Fri, 24 May 2024 22:17:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdDj-0007Iy-SR
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:17:03 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 56bec488-1a1b-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:17:00 +0200 (CEST)
Received: from DM5PR08CA0053.namprd08.prod.outlook.com (2603:10b6:4:60::42) by
 PH7PR12MB7186.namprd12.prod.outlook.com (2603:10b6:510:202::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Fri, 24 May
 2024 22:16:54 +0000
Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com
 (2603:10b6:4:60:cafe::eb) by DM5PR08CA0053.outlook.office365.com
 (2603:10b6:4:60::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22 via Frontend
 Transport; Fri, 24 May 2024 22:16:53 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:16:53 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:16:52 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:16:52 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56bec488-1a1b-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n1WIa5hyBbCCmlFsWsiq13AwcNfDf/prH6dSxgm6oIo3UjuP/GqZTPKrsRepQOPZlyuzfn5lhTrHU2E/kYnz12viCULvm1LYNwq4P5tSZriOz47NU6p0EgNlGTEqIOtfyo08zWXQ8yg0NtIESkV65YY7K4unrZiAzaqrZUBj2w/MCZqha3W5MoQ5QnpEJjcDYJ8YwdtoWahmNunPJqXAe5Wg2N7KXY+61l64dYoDTWwLwduInYTCbFw/h9GlvuiKBJIMCUH02mRLY+ImMMolcrFsImwG15zbIFWWT97Xd8P0aRf69Quepyeq5l4Qa+zCzc2sFzAX6gUwyhhHfb2ArQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8840+SwwfsZRigVHTRSue5zpJzKftzBoXAWMQbq7J+M=;
 b=Qrk8E90Rgt6OOmUIFY0qT0O8Z83C04195iMCs2ce652csFpzX5gnnbQc1gQQ5GABIEC3LM9Tt59zURQ7XNUCbv9UFML6sC0ZzcOBuMag/TwfTXb0sYsxny66B5ER5+bqjoOD1nETJ11vqrUOS+MiwiAV/6pbADZAYQwh+UBqDu9bSQ6sYdR/FL1MVYCHUgyQY6IaGiscYziPOMGxiZtYLTVUPwsbbu+kgjguffDNj0qTp4KHYQJqZDy4b6UKjAFm0EaQdb+T/qTdwUE4dy+OtEtUHdH84ciaOtcpSJk1awiuoWreYWqNj6V2ZtGSIUxlf09MV/29PZpMLkOlSxSMUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8840+SwwfsZRigVHTRSue5zpJzKftzBoXAWMQbq7J+M=;
 b=Fq9Fg5caWy1L9wZTOdDnjbfjD7lAO/71QqmWN29AMXvkT6W2gt2puCYt+6zEkevBLQezaHP3geXk9Ep94o4lztrZ+lyZFXq88zUkREbCTZ5A9o/e5E0uBpZ+5K+fHXKWWLb1GUaL5Pm4g/mUxJAxiljI4qAGi1vXXuv3vFj6iEo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <anthony@xenproject.org>, <sstabellini@kernel.org>, <julien@xen.org>,
	<bertrand.marquis@arm.com>, <michal.orzel@amd.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, "Stefano
 Stabellini" <stefano.stabellini@amd.com>, Jason Andryuk
	<jason.andryuk@amd.com>
Subject: [PATCH v6 6/7] tools: Introduce the "xl dt-overlay attach" command
Date: Fri, 24 May 2024 15:16:46 -0700
Message-ID: <20240524221647.2860711-6-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|PH7PR12MB7186:EE_
X-MS-Office365-Filtering-Correlation-Id: 31ee4b0c-592c-4e4e-93e8-08dc7c3f37b4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|82310400017|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?U4fvEoe62dzmheReT3IX5YSBS6THqI7f9eCy8/PXK3G2ySg6+fuUd3nDyGK8?=
 =?us-ascii?Q?tG91lPI7TuDG4N3+enj7jOGF/PS8vKIk/NUhumPlOkCfNmA9xGttOoWv+8go?=
 =?us-ascii?Q?iSO4Pgn6ZE8pCskl1obbxJHSgHK8TUejGKJfyk2UUZb7GBMhUYjNnnso1ngs?=
 =?us-ascii?Q?i6rNYkCs67hMA2ZJOsktatvcMO195MonofhLr3vngg32r8HhlolMVXULhwUz?=
 =?us-ascii?Q?SS7bnr4/VL4LDnbFV0E56OsaMFMFyKzk+APf/s5Pe85yty4SgcTYe5KcPiBr?=
 =?us-ascii?Q?HBS3FGNVAKhuh4Q3sN87MaPQtqyKnb3ouvRwoTxozA/ddP+R+1NTEw9Yewt1?=
 =?us-ascii?Q?woYmszs0kDIYaHc8c21RC8nisZEFBbHFfUxrlItAiqCtVT9T6J109+WVAiWF?=
 =?us-ascii?Q?r9k6GVRqdSDayzPcdV1eAAJuISNDHVK3XBqCB3VoCY6nzG7FgRdyV4Yk2aZK?=
 =?us-ascii?Q?u7vX6nYUAhxmtBEoYBhZknadZ26P5lWyaLPXfG3JQGkwp3IUzSbykpXjEbSJ?=
 =?us-ascii?Q?3x/wPyy/7aIOmBGbpAFpoIG9NasLMp8rXUd+teYwZKFrgbeglyyajaSOyvQS?=
 =?us-ascii?Q?OEk0dgF9pwT9Ye6sVI0nd6WVAZxH20aqSoSME5zXQfmrB1sDf1FoOsawCaG5?=
 =?us-ascii?Q?bczEW55Z9mV9fcZAQAVL8FS74EFaeR/22UFNcFIrWvSoHiDLhjcK+7WEB+EC?=
 =?us-ascii?Q?PgGN7AVw/diyymphRb/DsVyvyrZrm/nh80RfzZI1wVB0tRnwUFVyll72LsHg?=
 =?us-ascii?Q?wVLcWcDLZ7FbMUVjfJxYC0XmtyGcVF214Mu2abQXWYOl8u4aWRrcjbEbzobf?=
 =?us-ascii?Q?OD1xH8gQFgLBaZ4e+IpoOeqW9hNeLCqp2Pt59kiLVRrQv2g61n024xxwqDV7?=
 =?us-ascii?Q?Dg4j50cp3TKDqe1klIkKn+Q67JKRrGDOEcZ3ly78yFsYmsyHFbZJt5x2zcBP?=
 =?us-ascii?Q?WKcP/vVLZHG+f/UFQiWkJ9Eu/KzBPaKCBlKZOJX2n1fNjXExzlBdiCJtMw/n?=
 =?us-ascii?Q?PruPBfolRqYCAu686/c5qYthZTGD3t/XrZVVemULFPRzdDowogcz62GPi7ne?=
 =?us-ascii?Q?s5TKw4tgqBeenvioCzTgvyySJQozI6OSp4HTjxBIjS/kf4PeoLruOB9CiTBl?=
 =?us-ascii?Q?eUwvwmmBShi0vdBhbWjfq4iRzqL49qxVc50u0PUp91loQ2NSJUiUrQFIC6og?=
 =?us-ascii?Q?oAojShaBmycYaPTa2u4abnd98Dti94R09cb7ehUG1NStysdnZRmUJycW/xG/?=
 =?us-ascii?Q?F+ydTDXqnak4eGY0h7svTjZKF/S8HW2obWdgv+VjVQb90hnvhD6kQYqDL9kh?=
 =?us-ascii?Q?jEa6wmKNjI6IY3rgrKBrYicxpxK73OoErtb6O7x/h0O6q+fBawH7SeQduZJ0?=
 =?us-ascii?Q?LTd9aTLasnDpEi80051JEwHcKWjx?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(82310400017)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:16:53.8195
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 31ee4b0c-592c-4e4e-93e8-08dc7c3f37b4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7186

From: Henry Wang <xin.wang2@amd.com>

With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach (in the future also detach) devices from the provided DT overlay
to domains. Support this by introducing a new "xl dt-overlay" command
and related documentation, i.e. "xl dt-overlay attach. Slightly rework
the command option parsing logic.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 tools/include/libxl.h               | 15 +++++++++++
 tools/include/xenctrl.h             |  3 +++
 tools/libs/ctrl/xc_dt_overlay.c     | 31 +++++++++++++++++++++++
 tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++++
 tools/xl/xl_cmdtable.c              |  4 +--
 tools/xl/xl_vmcontrol.c             | 39 ++++++++++++++++++++---------
 6 files changed, 106 insertions(+), 14 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index a3d05c840b..f5c7167742 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -641,6 +641,12 @@
  */
 #define LIBXL_HAVE_XEN_9PFS 1
 
+/*
+ * LIBXL_HAVE_DT_OVERLAY_DOMAIN indicates the presence of
+ * libxl_dt_overlay_domain.
+ */
+#define LIBXL_HAVE_DT_OVERLAY_DOMAIN 1
+
 /*
  * libxl memory management
  *
@@ -2554,8 +2560,17 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 #if defined(__arm__) || defined(__aarch64__)
+/* Values should keep consistent with the op from XEN_SYSCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_ADD                   1
+#define LIBXL_DT_OVERLAY_REMOVE                2
 int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
                      uint32_t overlay_size, uint8_t overlay_op);
+
+/* Values should keep consistent with the op from XEN_DOMCTL_dt_overlay */
+#define LIBXL_DT_OVERLAY_DOMAIN_ATTACH         1
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op);
 #endif
 
 /*
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 4996855944..9ceca0cffc 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id);
 #endif
 
 /* Compat shims */
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
index c2224c4d15..ea1da522d1 100644
--- a/tools/libs/ctrl/xc_dt_overlay.c
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -48,3 +48,34 @@ err:
 
     return err;
 }
+
+int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt,
+                         uint32_t overlay_fdt_size, uint8_t overlay_op,
+                         uint32_t domain_id)
+{
+    int err;
+    struct xen_domctl domctl = {
+        .cmd = XEN_DOMCTL_dt_overlay,
+        .domain = domain_id,
+        .u.dt_overlay = {
+            .overlay_op = overlay_op,
+            .overlay_fdt_size = overlay_fdt_size,
+        }
+    };
+
+    DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+        goto err;
+
+    set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, overlay_fdt);
+
+    return err;
+}
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
index a6c709a6dc..00503b76bd 100644
--- a/tools/libs/light/libxl_dt_overlay.c
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -69,3 +69,31 @@ out:
     return rc;
 }
 
+int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id,
+                            void *overlay_dt, uint32_t overlay_dt_size,
+                            uint8_t overlay_op)
+{
+    int rc;
+    int r;
+    GC_INIT(ctx);
+
+    if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+        LOG(ERROR, "Overlay DTB check failed");
+        rc = ERROR_FAIL;
+        goto out;
+    } else {
+        LOG(DEBUG, "Overlay DTB check passed");
+        rc = 0;
+    }
+
+    r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, overlay_op,
+                             domain_id);
+    if (r) {
+        LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__);
+        rc = ERROR_FAIL;
+    }
+
+out:
+    GC_FREE;
+    return rc;
+}
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 1f3c6b5897..42751228c1 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = {
 #ifdef LIBXL_HAVE_DT_OVERLAY
     { "dt-overlay",
       &main_dt_overlay, 0, 1,
-      "Add/Remove a device tree overlay",
-      "add/remove <.dtbo>",
+      "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain",
+      "<operation=add|remove> <.dtbo> OR <operation=attach> <.dtbo> <Domain>",
       "-h print this help\n"
     },
 #endif
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 02575d5d36..fa1a4420e3 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1268,32 +1268,43 @@ int main_create(int argc, char **argv)
 #ifdef LIBXL_HAVE_DT_OVERLAY
 int main_dt_overlay(int argc, char **argv)
 {
-    const char *overlay_ops = NULL;
     const char *overlay_config_file = NULL;
     void *overlay_dtb = NULL;
     int rc;
     uint8_t op;
     int overlay_dtb_size = 0;
-    const int overlay_add_op = 1;
-    const int overlay_remove_op = 2;
+    uint32_t domain_id = 0;
+    bool domain_op = false;
 
     if (argc < 2) {
         help("dt-overlay");
         return EXIT_FAILURE;
     }
 
-    overlay_ops = argv[1];
-    overlay_config_file = argv[2];
-
-    if (strcmp(overlay_ops, "add") == 0)
-        op = overlay_add_op;
-    else if (strcmp(overlay_ops, "remove") == 0)
-        op = overlay_remove_op;
-    else {
+    if (strcmp(argv[optind], "add") == 0)
+        op = LIBXL_DT_OVERLAY_ADD;
+    else if (strcmp(argv[optind], "remove") == 0)
+        op = LIBXL_DT_OVERLAY_REMOVE;
+    else if (strcmp(argv[optind], "attach") == 0) {
+        op = LIBXL_DT_OVERLAY_DOMAIN_ATTACH;
+        domain_op = true;
+    } else {
         fprintf(stderr, "Invalid dt overlay operation\n");
         return EXIT_FAILURE;
     }
 
+    overlay_config_file = argv[optind+1];
+
+    if (domain_op) {
+        if (argc <= optind + 2) {
+            fprintf(stderr, "Missing domain ID\n");
+            help("dt-overlay");
+            return EXIT_FAILURE;
+        } else {
+            domain_id = find_domain(argv[optind+2]);
+        }
+    }
+
     if (overlay_config_file) {
         rc = libxl_read_file_contents(ctx, overlay_config_file,
                                       &overlay_dtb, &overlay_dtb_size);
@@ -1309,7 +1320,11 @@ int main_dt_overlay(int argc, char **argv)
         return EXIT_FAILURE;
     }
 
-    rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    if (!domain_op)
+        rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+    else
+        rc = libxl_dt_overlay_domain(ctx, domain_id, overlay_dtb,
+                                     overlay_dtb_size, op);
 
     free(overlay_dtb);
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:36:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:36:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729935.1135260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdWm-0006YX-0D; Fri, 24 May 2024 22:36:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729935.1135260; Fri, 24 May 2024 22:36:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdWl-0006YQ-Ti; Fri, 24 May 2024 22:36:43 +0000
Received: by outflank-mailman (input) for mailman id 729935;
 Fri, 24 May 2024 22:36:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdWj-0006YK-T4
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:36:41 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15854dde-1a1e-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:36:39 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 015AF63213;
 Fri, 24 May 2024 22:36:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 824EDC2BBFC;
 Fri, 24 May 2024 22:36:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15854dde-1a1e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716590197;
	bh=FD3LbHotEvN7GcWuRojZMB3q2dpVE2HQASc0TM7igCc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=BV6oKu/O4bShNOV1esRebtOo7jfGqlVP4lwLxSTdZfSYy3X/gfsezGeJjYqBVkMuB
	 FOJHEJ6q8jyIHrvEOJ/5ho6rxOWrNKC5BVNbOgN7ziUE2LbvDa2KIxd9EF2GxdJFBa
	 5ulYVwG4wpqYwhMfZUJ1sgGEXjP7nMYUlhU4qTN2UcJeAkI0yRg0DpsjtiNvsu0uMC
	 PJxPdUeJ9ibkm2vlg0S3JKS6mnC5ihV8OSVClVis0VT67heCqfkwNSXWMHsu8tps3M
	 lQXAnalrgcZWh6/mwO9Jc9U7XUQQ4pgwIO3+e2mD8dDzKsz5z7i/sQ47OX69t6HOF3
	 a9PsoLQFA+AKA==
Date: Fri, 24 May 2024 15:36:35 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org, 
    Luca Fancellu <Luca.Fancellu@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 1/4] xen/arm/static-shmem: Static-shmem should be
 direct-mapped for direct-mapped domains
In-Reply-To: <e53fbcba-f7dd-48a5-9481-a69bfb501cee@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405241535570.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-2-xin.wang2@amd.com> <ee20798f-a459-492a-a7a1-308472878fe1@amd.com> <86c91b57-68b4-4271-87ec-b80525ed3373@amd.com> <f0342d7a-1275-4a28-9688-326f07334a71@amd.com>
 <c0b59ccd-b3b0-4495-bbb4-2855fa2d7d16@amd.com> <93336435-309d-4aa5-9977-9ce0d4725811@amd.com> <e53fbcba-f7dd-48a5-9481-a69bfb501cee@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 21 May 2024, Henry Wang wrote:
> Hi Michal,
> 
> On 5/21/2024 12:09 AM, Michal Orzel wrote:
> > > > > Thanks. I will take the tag if you are ok with above diff (for the
> > > > > case
> > > > > if this series goes in later than Luca's).
> > > > I would move this check to process_shm() right after "gbase =
> > > > dt_read_paddr" setting.
> > > > This would be the most natural placement for such a check.
> > > That sounds good. Thanks! IIUC we only need to add the check for the
> > > pbase != INVALID_PADDR case correct?
> > Yes, but at the same time I wonder whether we should also return error if a
> > user omits pbase
> > for direct mapped domain.
> 
> I think this makes sense. So I will add also a check for the case if users
> omit pbase in the device tree for the direct mapped domain.

I fixed the NIT and added the ack, but as Luca's series hasn't been
committed yet, I have not made this change. I'll leave it to Julien when
he commits both series.


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:49:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:49:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729944.1135271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdis-0000VQ-4y; Fri, 24 May 2024 22:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729944.1135271; Fri, 24 May 2024 22:49:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdis-0000VJ-0G; Fri, 24 May 2024 22:49:14 +0000
Received: by outflank-mailman (input) for mailman id 729944;
 Fri, 24 May 2024 22:49:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdiq-0000VD-VK
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:49:12 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5e86990-1a1f-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:49:11 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 4E40A61255;
 Fri, 24 May 2024 22:49:10 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 462EFC2BBFC;
 Fri, 24 May 2024 22:49:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5e86990-1a1f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716590950;
	bh=0rp9M/83s6/hHN25ro65L52vQUSP8Gvb0ejiv3SUi3s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DgiBPkJSJiBgYey3oQHiOGSRrmbUVl0OBVAUOP8hZIxNmMbBghYevOFqWg2HKlgVx
	 56zFoZZEVYXlx1fnvuMVNiw/JDBMlMGH2bXvYkvre3Dik/+khK6UcrfJqlWFPOpgKX
	 05PylNUzm78Hs7JOcYSqblmvjwV7N7KHqSEs2NHe7M1E++Vi6XBs/LtXRkGk72gYhm
	 x4BhNaLDmEH+OsW1Nav+B77B9VAaz43mCShb6JNPSsK0h8SXgW5R0MYmSEnd2c1LTB
	 kNOC5EbdfxDl3lk/i9gUoM+U9y5Z2RDDZxOYYQGVMeZSqS1DHtVRrx/GKAMKPc9GoK
	 Ob2hG+I7Bq95A==
Date: Fri, 24 May 2024 15:49:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Alec Kwapis <alec.kwapis@medtronic.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
In-Reply-To: <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241536510.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-3-xin.wang2@amd.com> <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Henry,
> 
> + Juergen as the Xenstore maintainers. I'd like his opinion on the approach.
> The documentation of the new logic is in:
> 
> https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
> 
> FWIW I am happy in principle with the logic (this is what we discussed on the
> call last week). Some comments below.
> 
> On 17/05/2024 04:21, Henry Wang wrote:
> > There are use cases (for example using the PV driver) in Dom0less
> > setup that require Dom0less DomUs start immediately with Dom0, but
> > initialize XenStore later after Dom0's successful boot and call to
> > the init-dom0less application.
> > 
> > An error message can seen from the init-dom0less application on
> > 1:1 direct-mapped domains:
> > ```
> > Allocating magic pages
> > memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> > Error on alloc magic pages
> > ```
> > 
> > The "magic page" is a terminology used in the toolstack as reserved
> > pages for the VM to have access to virtual platform capabilities.
> > Currently the magic pages for Dom0less DomUs are populated by the
> > init-dom0less app through populate_physmap(), and populate_physmap()
> > automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> > cannot be true for the magic pages that are allocated later from the
> > init-dom0less application executed in Dom0. For domain using statically
> > allocated memory but not 1:1 direct-mapped, similar error "failed to
> > retrieve a reserved page" can be seen as the reserved memory list is
> > empty at that time.
> > 
> > Since for init-dom0less, the magic page region is only for XenStore.
> > To solve above issue, this commit allocates the XenStore page for
> > Dom0less DomUs at the domain construction time. The PFN will be
> > noted and communicated to the init-dom0less application executed
> > from Dom0. To keep the XenStore late init protocol, set the connection
> > status to XENSTORE_RECONNECT.
> 
> So this commit is allocating the page, but it will not be used by
> init-dom0less until the next patch. But Linux could use it. So would this
> break bisection? If so, then I think patch #3 needs to be folded in this
> patch.

I think that's fine, I'll leave that with you on commit. I'll resend as
is addressing the other comments.


> > 
> > Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> > Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > ---
> > v3:
> > - Only allocate XenStore page. (Julien)
> > - Set HVM_PARAM_STORE_PFN and the XenStore connection status directly
> >    from hypervisor. (Stefano)
> > v2:
> > - Reword the commit msg to explain what is "magic page" and use generic
> >    terminology "hypervisor reserved pages" in commit msg. (Daniel)
> > - Also move the offset definition of magic pages. (Michal)
> > - Extract the magic page allocation logic to a function. (Michal)
> > ---
> >   xen/arch/arm/dom0less-build.c | 44 ++++++++++++++++++++++++++++++++++-
> >   1 file changed, 43 insertions(+), 1 deletion(-)
> > 
> > diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> > index 74f053c242..95c4fd1a2d 100644
> > --- a/xen/arch/arm/dom0less-build.c
> > +++ b/xen/arch/arm/dom0less-build.c
> > @@ -1,5 +1,6 @@
> >   /* SPDX-License-Identifier: GPL-2.0-only */
> >   #include <xen/device_tree.h>
> > +#include <xen/domain_page.h>
> >   #include <xen/err.h>
> >   #include <xen/event.h>
> >   #include <xen/grant_table.h>
> > @@ -10,6 +11,8 @@
> >   #include <xen/sizes.h>
> >   #include <xen/vmap.h>
> >   +#include <public/io/xs_wire.h>
> > +
> >   #include <asm/arm64/sve.h>
> >   #include <asm/dom0less-build.h>
> >   #include <asm/domain_build.h>
> > @@ -739,6 +742,42 @@ static int __init alloc_xenstore_evtchn(struct domain
> > *d)
> >       return 0;
> >   }
> >   +#define XENSTORE_PFN_OFFSET 1
> > +static int __init alloc_xenstore_page(struct domain *d)
> > +{
> > +    struct page_info *xenstore_pg;
> > +    struct xenstore_domain_interface *interface;
> > +    mfn_t mfn;
> > +    gfn_t gfn;
> > +    int rc;
> > +
> > +    d->max_pages += 1;
> 
> Sorry I should have spotted it earlier. But you want to check d->max_pages is
> not overflowing. You can look at acquire_shared_memory_bank() for how to do
> it.
> 
> Also, maybe we want an helper to do it so it is not open-coded in multiple
> places.

Makes sense. I open-coded it because I wasn't sure if you preferred a
static inline or a macro and where to add the implementation.


> > +    xenstore_pg = alloc_domheap_page(d, 0);
> 
> I think we may want to restrict where the page is allocated. For instance,
> 32-bit domain using short page-tables will not be able to address all the
> physical memory.
> 
> I would consider to try to allocate the page below 32-bit (using
> MEMF-bits(32). And then fallback to above 32-bit only 64-bit for domain.

done


> Also, just to note that in theory alloc_domheap_page() could return MFN 0. In
> practice we have excluded MFN 0 because it breaks the page allocator so far.
> 
> But I would still prefer if we add a check on the MFN. This will make easier
> to spot any issue if we ever give MFN 0 to the allocator.

Good idea, but for simplicity I added a check on MFN and if the result
is zero return an error. At least we have explicit check.


> A possible implementation would be to call alloc_domphea_page() a second time
> and then free the first one (e.g. MFN 0).
>
> > +    if ( xenstore_pg == NULL )
> > +        return -ENOMEM;
> > +
> > +    mfn = page_to_mfn(xenstore_pg);
> > +    if ( !is_domain_direct_mapped(d) )
> > +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE +
> > +                           (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
> > +    else
> > +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
> > +
> > +    rc = guest_physmap_add_page(d, gfn, mfn, 0); > +    if ( rc )
> > +    {
> > +        free_domheap_page(xenstore_pg);
> > +        return rc;
> > +    }
> > +
> > +    d->arch.hvm.params[HVM_PARAM_STORE_PFN] = gfn_x(gfn);
> > +    interface = (struct xenstore_domain_interface *)map_domain_page(mfn);
> 
> map_domain_page() will return a void *. So why do you need to an explicit
> case?

Changed


> > +    interface->connection = XENSTORE_RECONNECT;
> > +    unmap_domain_page(interface);
> > +
> > +    return 0;
> > +}
> > +
> >   static int __init construct_domU(struct domain *d,
> >                                    const struct dt_device_node *node)
> >   {
> > @@ -839,7 +878,10 @@ static int __init construct_domU(struct domain *d,
> >           rc = alloc_xenstore_evtchn(d);
> >           if ( rc < 0 )
> >               return rc;
> > -        d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
> > +
> > +        rc = alloc_xenstore_page(d);
> > +        if ( rc < 0 )
> > +            return rc;
> >       }
> >         return rc;
> 
> Cheers,
> 
> -- 
> Julien Grall
> 


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:50:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:50:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729949.1135280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdkR-0002ZG-D4; Fri, 24 May 2024 22:50:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729949.1135280; Fri, 24 May 2024 22:50:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdkR-0002Z9-AR; Fri, 24 May 2024 22:50:51 +0000
Received: by outflank-mailman (input) for mailman id 729949;
 Fri, 24 May 2024 22:50:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdkP-0002Z1-QC
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:50:49 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0eccace8-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:50:47 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D25516150B;
 Fri, 24 May 2024 22:50:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C2C9C2BBFC;
 Fri, 24 May 2024 22:50:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0eccace8-1a20-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716591045;
	bh=9wwxkrL7EisBY3XiHMlCsZUncfQqWk8OHDsvBxUz7GI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dS1dEX4qOyrK6c9vNO+Gs4Y0myjI1duyCJ//Rygx4ptGei1hfBopiWya0CIm1Bc+z
	 gZhmGITBgsNcsa4OJ5vNIrKYX+Hvfmd4Lbr+1ybAl0jxVEffifTFpDnyXPI0RaaHAi
	 kRCWj8cS/dKE3pdYWES5zUlKCIBEllw9uPQpaC+ZphwKdbsSNJ2AWztEATTGU8qkYR
	 dor7XuCmsecNAMIhJcBmdFuskf4R3d7iPQHQnowmL1RmtmsX268nl7h5cWpG6EPEqw
	 m+Ig5op7+mC5XdmYRlyEOtkSvMGeNPcSVMv67ta0QbZAR5/pQMcc70YDuPywyZC15J
	 GzGAw8bF/J9pg==
Date: Fri, 24 May 2024 15:50:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Henry Wang <xin.wang2@amd.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH v3 4/4] docs/features/dom0less: Update the late XenStore
 init protocol
In-Reply-To: <20240517032156.1490515-5-xin.wang2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405241550370.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-5-xin.wang2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 May 2024, Henry Wang wrote:
> With the new allocation strategy of Dom0less DomUs XenStore page,
> update the doc of the late XenStore init protocol accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> v3:
> - Wording change.
> v2:
> - New patch.
> ---
>  docs/features/dom0less.pandoc | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/docs/features/dom0less.pandoc b/docs/features/dom0less.pandoc
> index 725afa0558..8b178edee0 100644
> --- a/docs/features/dom0less.pandoc
> +++ b/docs/features/dom0less.pandoc
> @@ -110,9 +110,10 @@ hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.
>  The implementation works as follows:
>  - Xen allocates the xenstore event channel for each dom0less domU that
>    has the "xen,enhanced" property, and sets HVM_PARAM_STORE_EVTCHN
> -- Xen does *not* allocate the xenstore page and sets HVM_PARAM_STORE_PFN
> -  to ~0ULL (invalid)
> -- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid
> +- Xen allocates the xenstore page and sets HVM_PARAM_STORE_PFN as well
> +  as the connection status to XENSTORE_RECONNECT.
> +- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to
> +  ~0ULL (invalid) or the connection status is *not* XENSTORE_CONNECTED.
>      - Old kernels will continue without xenstore support (Note: some old
>        buggy kernels might crash because they don't check the validity of
>        HVM_PARAM_STORE_PFN before using it! Disable "xen,enhanced" in
> @@ -121,13 +122,14 @@ The implementation works as follows:
>        channel (HVM_PARAM_STORE_EVTCHN) before continuing with the
>        initialization
>  - Once dom0 is booted, init-dom0less is executed:
> -    - it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN
> +    - it gets the xenstore shared page from HVM_PARAM_STORE_PFN
>      - it calls xs_introduce_domain
>  - Xenstored notices the new domain, initializes interfaces as usual, and
>    sends an event channel notification to the domain using the xenstore
>    event channel (HVM_PARAM_STORE_EVTCHN)
>  - The Linux domU kernel receives the event channel notification, checks
> -  HVM_PARAM_STORE_PFN again and continue with the initialization
> +  HVM_PARAM_STORE_PFN and the connection status again and continue with
> +  the initialization
>  
>  
>  Limitations
> -- 
> 2.34.1
> 
> 


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:51:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:51:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729953.1135290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdl6-00034d-LF; Fri, 24 May 2024 22:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729953.1135290; Fri, 24 May 2024 22:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdl6-00034W-Il; Fri, 24 May 2024 22:51:32 +0000
Received: by outflank-mailman (input) for mailman id 729953;
 Fri, 24 May 2024 22:51:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdl5-00031k-4F
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:51:31 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 28c4ce51-1a20-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 00:51:30 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5AFC861255;
 Fri, 24 May 2024 22:51:29 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D348C2BBFC;
 Fri, 24 May 2024 22:51:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28c4ce51-1a20-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716591089;
	bh=XAwoBH/hq9BnOLuTHwhew6AJ+pucCPFCnbPge3mmXEA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=CL2boOtXt92MYSilm58eCafWmfnZqQIRmdlYqhIb85rNa6ufsX/V7YmefsUGlBfQ3
	 etEuczF4mGwABDWsqKNfIL2iMw1RaYeF82Jt/rqCBV7E8sYad9mkbtuKuihBxXt0ce
	 kFzoWO4ZmvHwRXW6bOFrhOGaORXW6RYHh5TzLd8blNRe/snv6Bs9uaaiq9Na2vIf6L
	 ikElAxEiIMKx/noCC7Z/CiE724BGwLjnre59NtKkoYmdUscaU05vCOGXHyqZskQ4oM
	 IjbFUAfTwmJAZAvYrRSN2cUDQBAVbB6rZ2tsDCyK/qNjIb6u+jZP0sh9QdX+igTJrl
	 uDnoCCM/dvQqw==
Date: Fri, 24 May 2024 15:51:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jason Andryuk <jason.andryuk@amd.com>
cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Anthony PERARD <anthony@xenproject.org>, 
    Alec Kwapis <alec.kwapis@medtronic.com>
Subject: Re: [PATCH v3 3/4] tools/init-dom0less: Avoid hardcoding
 GUEST_MAGIC_BASE
In-Reply-To: <46e1d5f3-20e5-47ea-a1cd-10d120211f0d@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405241551220.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-4-xin.wang2@amd.com> <46e1d5f3-20e5-47ea-a1cd-10d120211f0d@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 20 May 2024, Jason Andryuk wrote:
> On 2024-05-16 23:21, Henry Wang wrote:
> > Currently the GUEST_MAGIC_BASE in the init-dom0less application is
> > hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
> > DomUs.
> > 
> > Since the guest magic region allocation from init-dom0less is for
> > XenStore, and the XenStore page is now allocated from the hypervisor,
> > instead of hardcoding the guest magic pages region, use
> > xc_hvm_param_get() to get the XenStore page PFN. Rename alloc_xs_page()
> > to get_xs_page() to reflect the changes.
> > 
> > With this change, some existing code is not needed anymore, including:
> > (1) The definition of the XenStore page offset.
> > (2) Call to xc_domain_setmaxmem() and xc_clear_domain_page() as we
> >      don't need to set the max mem and clear the page anymore.
> > (3) Foreign mapping of the XenStore page, setting of XenStore interface
> >      status and HVM_PARAM_STORE_PFN from init-dom0less, as they are set
> >      by the hypervisor.
> > 
> > Take the opportunity to do some coding style improvements when possible.
> > 
> > Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> 
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Fri May 24 22:55:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:55:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729961.1135300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdoq-0003hd-4x; Fri, 24 May 2024 22:55:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729961.1135300; Fri, 24 May 2024 22:55:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdoq-0003hW-29; Fri, 24 May 2024 22:55:24 +0000
Received: by outflank-mailman (input) for mailman id 729961;
 Fri, 24 May 2024 22:55:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdoo-0003hQ-91
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:55:22 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b12c02b0-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:55:19 +0200 (CEST)
Received: from MN2PR10CA0017.namprd10.prod.outlook.com (2603:10b6:208:120::30)
 by DS0PR12MB8271.namprd12.prod.outlook.com (2603:10b6:8:fb::6) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7611.19; Fri, 24 May 2024 22:55:16 +0000
Received: from MN1PEPF0000ECD6.namprd02.prod.outlook.com
 (2603:10b6:208:120:cafe::76) by MN2PR10CA0017.outlook.office365.com
 (2603:10b6:208:120::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19 via Frontend
 Transport; Fri, 24 May 2024 22:55:16 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000ECD6.mail.protection.outlook.com (10.167.242.135) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:55:15 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:15 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:15 -0500
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Fri, 24 May 2024 17:55:14 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b12c02b0-1a20-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P86ml2ImLLdk0tR6fwVig6HAwv6f1oKe4wo8swBuavm1i8csO3VXvyeV881K++Xq8A6jePiAwrevgE8xTpb0aAAhxUki8hBPZee+S0mC387RSMuGT64JZGHq5sLoQzqmX+AWl558TsUP0MUdyhk7cjINDMRqXJAOp1D8K9XNkBX/Ww85+jkuX+pGU6WvCZ0mP7FJboaDs3gBe5/8RwVG7tw9Gfrkc9IuaoUFYc+19aWp41Re3x9Znx2YdXEWXJZYyW1hrDy7MFM66s+jv9EoiVoYux6BTLEyG1rMO9FK8QnSS3jBoHqZh9IrbR1DE+UPWxZTh8hpQtwxMhKdhbV4rQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zd6BFqoJqToWgUD+RZyQpDUm3wIFhBhW1r1MhJgBycU=;
 b=HCtN/bF8na234H4nRW/nRWfKOqQ5OcMBtDCUCqZ5J2UaFzHdQC7vQyI80/VdGc1UukNqEGuBRksbRtbAVUId6G54hM9PeDQ4rYNa4JW2XhLla12zzjOI+SY/qbnW8l490Xs/uO3kK6nfRuMQCFBIb0jk3iHkiU3geb79GO1W89kG4dmZ0p9HA6zJ5bScqXV1rXs3HpynM6/cFPMrRlPNgqFmH1JBuo7unOk6ti5qmxMyie127IKk1jR+A3zvwryDIM1fOxsd/OueFHeO+ScAN+x4QRuHD8J9T4wkW2XAbcOL3+AqNiboXuCds9iBtpofK4ghKbmpqHfvvrPyrPV3Bw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zd6BFqoJqToWgUD+RZyQpDUm3wIFhBhW1r1MhJgBycU=;
 b=VSsEzGCuLGL5QYFaKjUbZ3QABhzl8S3f8cqLOGZa7b+7qVL+9qURueBlgvVU1fLYZpT0Exp3VglABrRSyn7hERcHrHZn76ifdSde2BbvjwkYAXJP1Rco0ZcJyX3HWtuFVx347YOyu5OmfsQqLxikuJGMGOh7KDDv/Q7Wk+MKcfA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Date: Fri, 24 May 2024 15:55:14 -0700
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Anthony PERARD
	<anthony@xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v4 0/4] Guest XenStore page allocation for 11 Dom0less
 domUs
Message-ID: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD6:EE_|DS0PR12MB8271:EE_
X-MS-Office365-Filtering-Correlation-Id: 88e9a090-8f59-498d-f46e-08dc7c4493c5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?2eIF8jllDoHxAC+ykwwriGf8r73p+yCM5gVA1usKfJHwRUZ89XYCF622QmLL?=
 =?us-ascii?Q?726aSCcRCTAIuxmcPZ77ZeM1Gq1saWWAnZcFYd9jSt6/JvKhF8gZs9YKDPH9?=
 =?us-ascii?Q?4S8P4iZ6W+kuQTOAf+nAUIB/7jDbgh/L+vSsx5dMx0n9Aw1f6ASP8RqbN2uB?=
 =?us-ascii?Q?HPX2agCDaWKETKlWoZNQ0AHW+qbF7FGRSp2bexx5bJc04nDc6Cu/qPgS3LgI?=
 =?us-ascii?Q?ThV+ycnpepWq2wLkEA5vAZWbhD//aDNBssCGbSOVHamsNs8cDVYT/PkInWyC?=
 =?us-ascii?Q?qvF7IScvM4QtKIUx/5P9QpweK53DV201YCk+uYcM4IiGLPqC8fSGRSfJ/b7J?=
 =?us-ascii?Q?iLiZtxaYO0SVOwx/jrZfT/gcjDwkx+YgyQJ/qftATddJAadxAd4DDp+lw9BL?=
 =?us-ascii?Q?3u0uREoMqA4X2l897sYCCF1VMQKm4maHKOcV3zpN+MTLnmvsja+r4POWXcKf?=
 =?us-ascii?Q?V6DjYkKYNGKJnffNsICORSqIkGCiDbx4DkJe0TzhHFEpxRTgJnaGwybXeB45?=
 =?us-ascii?Q?u/DjnB6HLrcDi2MabXU9YN15It5ZgrSVTOprBJIrrj7YtbaGuimfU4tSkFdV?=
 =?us-ascii?Q?+tfbpSp1n6xYCdIPG+pZgmC+h/vEK1b5LeQoswWzhf/O2guc/5pw/+zqZmjm?=
 =?us-ascii?Q?sQ2gnaiz+nJK22W4GyOfTYu3+jEZpeoov0so3d5Tn0BjjA/o+3foNKasXvxq?=
 =?us-ascii?Q?uIkZQ2RQ7uWVX+vE5KuahltRM3MR8yFspcsZGT/dEhRGaBUVHW19PXSfG9Pt?=
 =?us-ascii?Q?lAlvInYqq6kSxquF3DCzOl9SsxdijPJSUP588Ek66wQAEN88efmqEd6O/QU8?=
 =?us-ascii?Q?A2LWoT5VxIXwBVDQFbUdLXoOM9URqyHTBpyYvINmGjUDYOhOLuBvQschkEWc?=
 =?us-ascii?Q?lc3ZeQjs4oKtWhbrf7/tPxc1e+7kW68Fa4nT5EuwZxaZvB90P4xovE+i2CuY?=
 =?us-ascii?Q?ezZop7Fn5TrF1AVbyLUqgDY+fuoZpRwlHP3Xq6/c+D/cXx5osegfnxcpRd6M?=
 =?us-ascii?Q?TdoSrXwQPis0wD0pNQY1R5ghsFnyWH5DwzuajTPVhW6GYCn41Lzwxaid3/Jf?=
 =?us-ascii?Q?Jbr/9EAm84gqkVuMIJhjUoLpktkI0VhBNTj6D0rShblqajtAg8YunqQJXs7Y?=
 =?us-ascii?Q?OvOg6HRaQwhgbFR+7+czZtX79E8WjXxNgXFiQNjL7wzGu1AeXFB8DsIWCArT?=
 =?us-ascii?Q?lwjgwzML6JK3EC2UdYmrujlQSv12DTwQoEIyLOKHp8jTGmZqvwH7ugjOi2y2?=
 =?us-ascii?Q?VBQ/QRTQbRpH22zdfQG5f2VRYlP9O0ueA99iVnSPsBPRAlS3tNQLvKFVPszz?=
 =?us-ascii?Q?HiRU0FcCEcFiFrCYKb6uNqsyFc4p9nl3xl+bxSXkmuH4Xw=3D=3D?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:55:15.8299
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 88e9a090-8f59-498d-f46e-08dc7c4493c5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD6.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8271

Hi all,

This series is trying to fix the reported guest magic region alloc
issue for 11 Dom0less domUs, an error message can seen from the
init-dom0less application on 1:1 direct-mapped Dom0less DomUs:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

This is because populate_physmap() automatically assumes gfn == mfn
for direct mapped domains. This cannot be true for the magic pages
that are allocated later for 1:1 Dom0less DomUs from the init-dom0less
helper application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list
is empty at that time.

In [1] I've tried to fix this issue by the domctl approach, and
discussions in [2] and [3] indicates that a domctl is not really
necessary, as we can simplify the issue to "allocate the Dom0less
guest magic regions at the Dom0less DomU build time and pass the
region base PFN to init-dom0less application". The later on
discussion [4] reached an agreement that we only need to allocate
one single page for XenStore, and set the HVM_PARAM_STORE_PFN from
hypervisor with some Linux XenStore late init protocol improvements.
Therefore, this series tries to fix the issue based on all discussions.
The first patch puts a restriction that static shared memory on
direct-mapped DomUs should also be direct mapped, as otherwise it will
clash [5]. Patch 2 allocates the XenStore page from Xen and set the
initial connection status to RECONNECTED. Patch 3 is the update of the
init-dom0less application with all of the changes. Patch 4 is the doc
change to reflect the changes introduced by this series.

**NOTE**: This series should work with the Linux change [6].

[1] https://lore.kernel.org/xen-devel/20240409045357.236802-1-xin.wang2@amd.com/
[2] https://lore.kernel.org/xen-devel/c7857223-eab8-409a-b618-6ec70f6165aa@apertussolutions.com/
[3] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2404251508470.3940@ubuntu-linux-20-04-desktop/
[4] https://lore.kernel.org/xen-devel/d33ea00d-890d-45cc-9583-64c953abd70f@xen.org/
[5] https://lore.kernel.org/xen-devel/686ba256-f8bf-47e7-872f-d277bf7df0aa@xen.org/
[6] https://lore.kernel.org/xen-devel/20240517011516.1451087-1-xin.wang2@amd.com/

Henry Wang (4):
  xen/arm/static-shmem: Static-shmem should be direct-mapped for
    direct-mapped domains
  xen/arm: Alloc XenStore page for Dom0less DomUs from hypervisor
  tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
  docs/features/dom0less: Update the late XenStore init protocol



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:55:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:55:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729962.1135310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdox-0003yz-Gg; Fri, 24 May 2024 22:55:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729962.1135310; Fri, 24 May 2024 22:55:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdox-0003ys-Dh; Fri, 24 May 2024 22:55:31 +0000
Received: by outflank-mailman (input) for mailman id 729962;
 Fri, 24 May 2024 22:55:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdow-0003hQ-L7
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:55:30 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b661fd4d-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:55:28 +0200 (CEST)
Received: from BLAPR03CA0088.namprd03.prod.outlook.com (2603:10b6:208:329::33)
 by SA3PR12MB9177.namprd12.prod.outlook.com (2603:10b6:806:39d::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 22:55:24 +0000
Received: from MN1PEPF0000ECD5.namprd02.prod.outlook.com
 (2603:10b6:208:329:cafe::14) by BLAPR03CA0088.outlook.office365.com
 (2603:10b6:208:329::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.27 via Frontend
 Transport; Fri, 24 May 2024 22:55:24 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000ECD5.mail.protection.outlook.com (10.167.242.133) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:55:24 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:23 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:23 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:55:22 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b661fd4d-1a20-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KHKRq0WLq+gvjaSBU2cBm+tOtKgFYAloEbDoytH7ShGaisA+nWjqSHfCeR54E9z51o+0otJioyy62fZW4rXxvH/lpNs5lImuQzjkJbETUf0fIr63Nfw5nEqPNXQmEbPgyHjPQksjnVQpFFyEHXukn1PeBXJqzbMPjPcXVgMuxrz1jGWM5pFO1o7X+BI6xLVHu/PjXwdJu/o+9ZtDFEp49cvUNjx8vvVpJcU/1UtP8I4K8AtrpBFkNhl4fva+hg4j2ApAL6KYEwMvYfb1HHNkp/AvjXv6fsB9YkwGuarm1azj2CQKaGmfLoSaG4zFsXGT6+zQLq0ix+qKrC4PISoPXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lzV5nEyCRZZ9wvgQ5/ihrFl2ogjKMReVCJclNQHfPF0=;
 b=GhCUGIJsy+h6Eb2FQ9I//g+AqcFCWpvi7j0oR5TtapXwYD85Et/JA2s15vhOtA3+UzpdzqIScMYZMIojxo2PhzhtJXEnWUZlyJGT43KNwmFyVgPNHLxTFc+/rgCwuTJ+ZnWhn/+eaztYCKb0zTgQP+ZRj4blkXK6clOvgzjsL0yjRLnSzOTLy6FVMxeXp8VK9EBJaD71sAXHODeLah9ScqJYNycLz2gtEgJ85tzrCPSDUiA8kq81JH2HzjNFJUPstMIO8uENgU6IN7QmCfZTfGF1cPFkrra3D+a/NpkNi82Y8r8EJlsAgRRlS2THFI5RRWES+K0UvcOz41621+znig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lzV5nEyCRZZ9wvgQ5/ihrFl2ogjKMReVCJclNQHfPF0=;
 b=Lr3XcTWlRaOLU2k/CYK7CGyJJuzf+av0wIXQUNYkNc0Y47kDqDnMeap1EfWmPqrzyEMff8fR5kgPFXvkwsaOrkPwa9xhsFls14jfA7l49un8LixuCTTXkwqVupkwgiGw9UHbwwpSZra5BwdtwNi92XFBnK/MPN/zKNXG1dSS1+k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, <Volodymyr_Babchuk@epam.com>, Henry Wang
	<xin.wang2@amd.com>, Stefano Stabellini <stefano.stabellini@amd.com>
Subject: [PATCH v4 1/4] xen/arm/static-shmem: Static-shmem should be direct-mapped for direct-mapped domains
Date: Fri, 24 May 2024 15:55:19 -0700
Message-ID: <20240524225522.2878481-1-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD5:EE_|SA3PR12MB9177:EE_
X-MS-Office365-Filtering-Correlation-Id: 2be1aaed-b1e3-49ab-7fd0-08dc7c4498bb
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|1800799015|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?CKxoSez6zEGXESf2xzEkSlkhtrjlI6zvRjKJYtL8eGwP0v6qqkZKAGzs6Ys5?=
 =?us-ascii?Q?2zcXcYWc5qresS26+ofH7GV4O9vn/upHrWsS+/Xd9JF/fVC7ziLH1rxJnfHb?=
 =?us-ascii?Q?4d2IZNNFZkYJWDRWn+l2zUWr1VYS1owaKF9SyQ+DSmXKDil6VVEi8glSxByF?=
 =?us-ascii?Q?sxy03qaif6k0OiYEcFsWw4rbI4X46IdaxPD18oQVou8UNSw6g9kk9UV4vWLL?=
 =?us-ascii?Q?rNFRvR4GUmvn5LCGXMFSLI+un0Y5stMgTrgznUUzZdtMxtPnBgN03GHsRwhN?=
 =?us-ascii?Q?Q1cu/w8zK+xa82hVILrqVLh8xYoR4j5Til655Cr45JqhM2wtqFkcMMPTqn0Y?=
 =?us-ascii?Q?tzHxeJAc3RYpkn9Wrh6MEO1F8I1MxX9gG/F43BKTOFVJGDABF9wLm9LTJ3vA?=
 =?us-ascii?Q?+FOEP5n5AmhdsLyWmaHIjbTid/o1URydnIH4lcpjk/OX5T3yPpXg2vGnf0kA?=
 =?us-ascii?Q?Psf1Hy3p6+wmNQaC/KXjy9Ye/mPwNGAsvmybVvVCcuwHulxQqUyrFDwMqHnS?=
 =?us-ascii?Q?ZMtVT3iiJsW3U/Y1rEGONxjU5tBl6BCbqCSfhIqlB/DO89wUP+wr8ZuAxqty?=
 =?us-ascii?Q?TQ/Nd4bp/Q+D6h+hn3b1ipHAPDhZpDlvuemZZTplHl+LQVmLeMkcBJq3hsTZ?=
 =?us-ascii?Q?5vDgmNbldgAijnhx1lfPYaOXmy0tQZxwImskua2FvwLF28sS5BCzIqGAxNlH?=
 =?us-ascii?Q?YZG8te5lf36YMnKBXD1FYx/3IrMHIFvUu06woQD8sxYXLXlzlPrriGu+CG1o?=
 =?us-ascii?Q?pY5W6ZmUYKRFmaRVJF460Z0KQ+0HmfaxO2V56uBB7La9kYQFl5BmPReLWMfh?=
 =?us-ascii?Q?NGmQHelKF8NatsFEILBsubdc4Zv8MMYhyL00xGRRKe+jtO8uGYRttolp/fgH?=
 =?us-ascii?Q?tmj/VmwS+gvIEson5Vl5gCpiKZCGuV8sGi2erPVWtqhOBGvciqPoHxbwFwCW?=
 =?us-ascii?Q?W4fegxvZUzcpM43FQ3cpkUl6BZ65eY/+1SgeUC+Ne+XBKTikD2I8sKmhJuiV?=
 =?us-ascii?Q?vKUMNb6f1gg6yzL31qktbT7YBZNQuPkyUEKrBT2YU2W2SwXjzSBPicl+WrOM?=
 =?us-ascii?Q?XxfHORJWceML6j5G/daih+HEVUA73DO6v9Eh31qLFbiAfz++Wv8Zf6dZmaag?=
 =?us-ascii?Q?LBpZE6ef95uG7VoDeXpfZdeaILE1+eJ0OEJ96UhQDP2Iq+QyUWrCoXg71o99?=
 =?us-ascii?Q?VExruOf/gNJPNNyiTWrpkuuYtDnh3xO461SilVw/6TJ85vng/KDba4CFdR4i?=
 =?us-ascii?Q?HM/2KRPlC1JcgtiIIkZN5x85UgRAHAGXOYjweV82J5oJL4Q4N1GJvSf/YhHc?=
 =?us-ascii?Q?rYydJArzOyCXEX1ydqjpXh+MLAUVRatdYnLQaky5nm4KBdJzjqhP+Mt8OQZf?=
 =?us-ascii?Q?byLolmwvURxK3f9Y4EDI8vSGecqw?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:55:24.1344
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2be1aaed-b1e3-49ab-7fd0-08dc7c4498bb
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD5.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB9177

From: Henry Wang <xin.wang2@amd.com>

Currently, users are allowed to map static shared memory in a
non-direct-mapped way for direct-mapped domains. This can lead to
clashing of guest memory spaces. Also, the current extended region
finding logic only removes the host physical addresses of the
static shared memory areas for direct-mapped domains, which may be
inconsistent with the guest memory map if users map the static
shared memory in a non-direct-mapped way. This will lead to incorrect
extended region calculation results.

To make things easier, add restriction that static shared memory
should also be direct-mapped for direct-mapped domains. Check the
host physical address to be matched with guest physical address when
parsing the device tree. Document this restriction in the doc.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
---
 docs/misc/arm/device-tree/booting.txt | 3 +++
 xen/arch/arm/static-shmem.c           | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index bbd955e9c2..c994e48391 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -591,6 +591,9 @@ communication.
     shared memory region in host physical address space, a size, and a guest
     physical address, as the target address of the mapping.
     e.g. xen,shared-mem = < [host physical address] [guest address] [size] >
+    Note that if a domain is direct-mapped, i.e. the Dom0 and the Dom0less
+    DomUs with `direct-map` device tree property, the static shared memory
+    should also be direct-mapped (host physical address == guest address).
 
     It shall also meet the following criteria:
     1) If the SHM ID matches with an existing region, the address range of the
diff --git a/xen/arch/arm/static-shmem.c b/xen/arch/arm/static-shmem.c
index 78881dd1d3..5bf1916e5e 100644
--- a/xen/arch/arm/static-shmem.c
+++ b/xen/arch/arm/static-shmem.c
@@ -235,6 +235,12 @@ int __init process_shm(struct domain *d, struct kernel_info *kinfo,
                    d, psize);
             return -EINVAL;
         }
+        if ( is_domain_direct_mapped(d) && (pbase != gbase) )
+        {
+            printk("%pd: physical address 0x%"PRIpaddr" and guest address 0x%"PRIpaddr" are not direct-mapped.\n",
+                   d, pbase, gbase);
+            return -EINVAL;
+        }
 
         for ( i = 0; i < PFN_DOWN(psize); i++ )
             if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:55:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:55:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729963.1135321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdoz-0004FQ-RY; Fri, 24 May 2024 22:55:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729963.1135321; Fri, 24 May 2024 22:55:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdoz-0004FG-MY; Fri, 24 May 2024 22:55:33 +0000
Received: by outflank-mailman (input) for mailman id 729963;
 Fri, 24 May 2024 22:55:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdoy-0003hQ-B1
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:55:32 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b7c3f78f-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:55:30 +0200 (CEST)
Received: from BL0PR05CA0026.namprd05.prod.outlook.com (2603:10b6:208:91::36)
 by MN0PR12MB6031.namprd12.prod.outlook.com (2603:10b6:208:3cd::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 24 May
 2024 22:55:26 +0000
Received: from MN1PEPF0000ECDA.namprd02.prod.outlook.com
 (2603:10b6:208:91:cafe::5a) by BL0PR05CA0026.outlook.office365.com
 (2603:10b6:208:91::36) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.9 via Frontend
 Transport; Fri, 24 May 2024 22:55:26 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000ECDA.mail.protection.outlook.com (10.167.242.134) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:55:26 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:25 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:25 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:55:24 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7c3f78f-1a20-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PRdSxu+XN/ZW2GQ5HrLeUE2ojTfJtJP5H833elqm8+dCzt4nobooZ9enOSnLfx1sBhcK0z7NysVwM4nBWhx3KVhwTZBXSX83xRve4KoBun0wHFzxBcKB35N60jLpD4OfIrkVpomjqF0hvXt8loDlCGavuRXNXGVIKmeDMidkm4KQ8syCaKuFlfyMSnoU0SA/4BhgWLvV6IFUmnQVN9QQdPDv5M3yhnOpresV/ZovNonTqb+H9ns3igJnXUnVq62Qf0Tg9dNv0S2bthGfiU75PpARVeoIr+ojQWtQPXX3wUvefE/8totGVUOAoxywb10wwDoBdQ4vNn+pY+pWX7H5kw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FumphntemJECKnhgVYvneYgiginHtvLBa1WGte3c0qA=;
 b=afGXK/5nfdz4FNrBl1zmsXjAqV0DNrafnleTKdh0nwANYSGWdpz6PAJ4HTo6D+m3CaGOCsF+rwIURAG8408/b5AbIT/+jGhu5MlMITDs8wOFqRdHcylWD0X7zhMVp0w7XiLYzybUoI9d5l7yChu77iZOmTFCHszaDlZvMLCJBEmIdn9911Lz+jic96WIyQxBC5hwqof7nMv1uj+q/3FW8gc1tPNJyUNzOCaGXiAf4KrhwbC3LzKNqMn5BI9P2zM6M1vNew8B4ZMDh0eDigbIbNDBXUFb1D3ejaad/k8VwhRcoGMDF5tDs1BFGM1L2EXWVr9NSe/CvvHlj6QWiY9aCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FumphntemJECKnhgVYvneYgiginHtvLBa1WGte3c0qA=;
 b=fGBXjkIVv5++lnx8mqLMZJXu5BKMcnsT1Zk7j10NfC1qSTJpPkQOS65aHnIEjBOAzuJGpfRo0dQ1mD81C8BFg7AITkZeQ3BjcIicChOuJEN1px5RbOi/VRuKuTRheRN4oYbSzLfD2U30dWrm76wswsGVGNqvYsy0wcReCokSUIQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, <Volodymyr_Babchuk@epam.com>, Henry Wang
	<xin.wang2@amd.com>, Alec Kwapis <alec.kwapis@medtronic.com>, Jason Andryuk
	<jason.andryuk@amd.com>, <anthony@xenproject.org>
Subject: [PATCH v4 3/4] tools/init-dom0less: Avoid hardcoding GUEST_MAGIC_BASE
Date: Fri, 24 May 2024 15:55:21 -0700
Message-ID: <20240524225522.2878481-3-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECDA:EE_|MN0PR12MB6031:EE_
X-MS-Office365-Filtering-Correlation-Id: 309ceecc-faa3-4e74-3bbf-08dc7c4499e1
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?KIxqkTzY8QQ85inv/yQ3OyXd+w3NONL2ZDLaH64xcnGIDqYEbhLNMVoLWc2Y?=
 =?us-ascii?Q?SwXHbgtNyLcicxUs5myfcVddrbXi533X5Mci20STU0Aqu/HcR8rFraGjiXnA?=
 =?us-ascii?Q?aa+dpqQYDuEqDpK4p1+sGe8CxJ8M7/Glt9Oa1jAXMGl+YUzKdGoiIZW/xn7g?=
 =?us-ascii?Q?mo1Xg8fcNj9OUJ5pxLqp0na2TrmtVjCnQpEVyV2VXAfQsBQozup7Qz1t4CPI?=
 =?us-ascii?Q?76QfEXhqUpBS6WLCBC5CQc4Vl7mAiHly55wibBY+ozJdxnO6YtHc/LJBM3R7?=
 =?us-ascii?Q?wosG6eN+RNcWOXGLWlP5IhZhPD1O4pXzkEgVAT7YnUGj+zd0NAZs4ubenZxo?=
 =?us-ascii?Q?rnqr3RFHr88dR8mmXOx/Zpom1xkTQAA3CH+jBQ1KHjvvH+H9olsLOeTs9Ol/?=
 =?us-ascii?Q?pBkIErNyCb6QOnsuCfcR9K64SogHHYJ3qhakRWseWHBJlCfvbsy+ScF1Ct5O?=
 =?us-ascii?Q?lifg5IisDeZ4QFhwmCrzs7RWjPIp7ZVB7q4Lsd4pf9YuYJ+S9V+pEh6jS7MI?=
 =?us-ascii?Q?o1sLzscU4jCt1jRFvbq904eHSB8Ogdq+KQI7yeSn0NugeoDmE/QuM90EP4de?=
 =?us-ascii?Q?NsoF3gN9Nob0WlqBOg47QOhrRng8ve+LHX/F63aDiWGwW8631/BeRacO2iAR?=
 =?us-ascii?Q?XQRScUGQsF58pM+k5q/Bw/FZ0IcvWAZqZTk1WlglXN+izXbGoLKnfpWFG1jq?=
 =?us-ascii?Q?A7oxxB2t4EOuOXaH2wSaDFn3LnKQ2LmXCNakMkTiMVm7bsgNcevsmlW1Y4+r?=
 =?us-ascii?Q?znm3xnXYmWj2VNpzuK3PMelvj8OZLgwk6MkSexLZimPYGJg3geoJ/CE/bR9c?=
 =?us-ascii?Q?6it/uiiPZLBaSXWBeGtg+rqZfFGMuauxkRL/uR/qQFsoizNrV4P00cfI8WSr?=
 =?us-ascii?Q?mAj0mP9Cozdj5w4hEjVwcE842i5GYVtakAcgeRNFRh4uKGvWyoVtywRS/hvt?=
 =?us-ascii?Q?fL3DpSP2b6ItJ8PeGlVl6FjcHZFL96p1zV6KQI+XKwtqWwrchaOipYU1sKEY?=
 =?us-ascii?Q?2xbykewpuOtAviGC1zECBwbCBFRQU9A3m7KfqtmOCoXiUyISxUecV4xjDeTm?=
 =?us-ascii?Q?ohxQu0P6RF/Ou5j6K33SZbcl9YtA5qMamCsJ6/fRy7SBMDxGzzPC957LSMVL?=
 =?us-ascii?Q?shh7nv27eJJ6a3Km8TUVozIZzLoAH9xHTN1Zbd7YMiyZrb8ewn4oJlj4Z9Po?=
 =?us-ascii?Q?eNTwVYFi6l7EXfhCwNZFthmlbHoMHh/xSvsNUqY6p6l+UGcdOSYm1uVenBdv?=
 =?us-ascii?Q?yZVayBquQ3O8lKbPRcaC8ZdLsoRNjXMQOoC95k6r1lyl/TaxJXqa7W8jjqfG?=
 =?us-ascii?Q?CEEZQpi/LoJImHsrNVrxRNtBTS5QHnKoN4WVPY0d2/vaayrJVkdu5bTBItIK?=
 =?us-ascii?Q?VLy6Vodm64w+Zy2HvsioqFewCo0R?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:55:26.0594
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 309ceecc-faa3-4e74-3bbf-08dc7c4499e1
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECDA.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6031

From: Henry Wang <xin.wang2@amd.com>

Currently the GUEST_MAGIC_BASE in the init-dom0less application is
hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
DomUs.

Since the guest magic region allocation from init-dom0less is for
XenStore, and the XenStore page is now allocated from the hypervisor,
instead of hardcoding the guest magic pages region, use
xc_hvm_param_get() to get the XenStore page PFN. Rename alloc_xs_page()
to get_xs_page() to reflect the changes.

With this change, some existing code is not needed anymore, including:
(1) The definition of the XenStore page offset.
(2) Call to xc_domain_setmaxmem() and xc_clear_domain_page() as we
    don't need to set the max mem and clear the page anymore.
(3) Foreign mapping of the XenStore page, setting of XenStore interface
    status and HVM_PARAM_STORE_PFN from init-dom0less, as they are set
    by the hypervisor.

Take the opportunity to do some coding style improvements when possible.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
CC: anthony@xenproject.org
---
 tools/helpers/init-dom0less.c | 58 +++++++++--------------------------
 1 file changed, 14 insertions(+), 44 deletions(-)

diff --git a/tools/helpers/init-dom0less.c b/tools/helpers/init-dom0less.c
index fee93459c4..2b51965fa7 100644
--- a/tools/helpers/init-dom0less.c
+++ b/tools/helpers/init-dom0less.c
@@ -16,30 +16,18 @@
 
 #include "init-dom-json.h"
 
-#define XENSTORE_PFN_OFFSET 1
 #define STR_MAX_LENGTH 128
 
-static int alloc_xs_page(struct xc_interface_core *xch,
-                         libxl_dominfo *info,
-                         uint64_t *xenstore_pfn)
+static int get_xs_page(struct xc_interface_core *xch, libxl_dominfo *info,
+                       uint64_t *xenstore_pfn)
 {
     int rc;
-    const xen_pfn_t base = GUEST_MAGIC_BASE >> XC_PAGE_SHIFT;
-    xen_pfn_t p2m = (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET;
 
-    rc = xc_domain_setmaxmem(xch, info->domid,
-                             info->max_memkb + (XC_PAGE_SIZE/1024));
-    if (rc < 0)
-        return rc;
-
-    rc = xc_domain_populate_physmap_exact(xch, info->domid, 1, 0, 0, &p2m);
-    if (rc < 0)
-        return rc;
-
-    *xenstore_pfn = base + XENSTORE_PFN_OFFSET;
-    rc = xc_clear_domain_page(xch, info->domid, *xenstore_pfn);
-    if (rc < 0)
-        return rc;
+    rc = xc_hvm_param_get(xch, info->domid, HVM_PARAM_STORE_PFN, xenstore_pfn);
+    if (rc < 0) {
+        printf("Failed to get HVM_PARAM_STORE_PFN\n");
+        return 1;
+    }
 
     return 0;
 }
@@ -100,6 +88,7 @@ static bool do_xs_write_vm(struct xs_handle *xsh, xs_transaction_t t,
  */
 static int create_xenstore(struct xs_handle *xsh,
                            libxl_dominfo *info, libxl_uuid uuid,
+                           uint64_t xenstore_pfn,
                            evtchn_port_t xenstore_port)
 {
     domid_t domid;
@@ -145,8 +134,7 @@ static int create_xenstore(struct xs_handle *xsh,
     rc = snprintf(target_memkb_str, STR_MAX_LENGTH, "%"PRIu64, info->current_memkb);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
-    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%lld",
-                  (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET);
+    rc = snprintf(ring_ref_str, STR_MAX_LENGTH, "%"PRIu64, xenstore_pfn);
     if (rc < 0 || rc >= STR_MAX_LENGTH)
         return rc;
     rc = snprintf(xenstore_port_str, STR_MAX_LENGTH, "%u", xenstore_port);
@@ -230,7 +218,6 @@ static int init_domain(struct xs_handle *xsh,
     libxl_uuid uuid;
     uint64_t xenstore_evtchn, xenstore_pfn;
     int rc;
-    struct xenstore_domain_interface *intf;
 
     printf("Init dom0less domain: %u\n", info->domid);
 
@@ -245,20 +232,11 @@ static int init_domain(struct xs_handle *xsh,
     if (!xenstore_evtchn)
         return 0;
 
-    /* Alloc xenstore page */
-    if (alloc_xs_page(xch, info, &xenstore_pfn) != 0) {
-        printf("Error on alloc magic pages\n");
-        return 1;
-    }
-
-    intf = xenforeignmemory_map(xfh, info->domid, PROT_READ | PROT_WRITE, 1,
-                                &xenstore_pfn, NULL);
-    if (!intf) {
-        printf("Error mapping xenstore page\n");
+    /* Get xenstore page */
+    if (get_xs_page(xch, info, &xenstore_pfn) != 0) {
+        printf("Error on getting xenstore page\n");
         return 1;
     }
-    intf->connection = XENSTORE_RECONNECT;
-    xenforeignmemory_unmap(xfh, intf, 1);
 
     rc = xc_dom_gnttab_seed(xch, info->domid, true,
                             (xen_pfn_t)-1, xenstore_pfn, 0, 0);
@@ -272,19 +250,11 @@ static int init_domain(struct xs_handle *xsh,
     if (rc)
         err(1, "gen_stub_json_config");
 
-    /* Now everything is ready: set HVM_PARAM_STORE_PFN */
-    rc = xc_hvm_param_set(xch, info->domid, HVM_PARAM_STORE_PFN,
-                          xenstore_pfn);
-    if (rc < 0)
-        return rc;
-
-    rc = create_xenstore(xsh, info, uuid, xenstore_evtchn);
+    rc = create_xenstore(xsh, info, uuid, xenstore_pfn, xenstore_evtchn);
     if (rc)
         err(1, "writing to xenstore");
 
-    rc = xs_introduce_domain(xsh, info->domid,
-            (GUEST_MAGIC_BASE >> XC_PAGE_SHIFT) + XENSTORE_PFN_OFFSET,
-            xenstore_evtchn);
+    rc = xs_introduce_domain(xsh, info->domid, xenstore_pfn, xenstore_evtchn);
     if (!rc)
         err(1, "xs_introduce_domain");
     return 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:55:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:55:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729965.1135330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdp3-0004Xj-37; Fri, 24 May 2024 22:55:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729965.1135330; Fri, 24 May 2024 22:55:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdp2-0004XS-W3; Fri, 24 May 2024 22:55:36 +0000
Received: by outflank-mailman (input) for mailman id 729965;
 Fri, 24 May 2024 22:55:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdp1-0003hQ-Pe
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:55:35 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20612.outbound.protection.outlook.com
 [2a01:111:f400:7e88::612])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9b2213b-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:55:34 +0200 (CEST)
Received: from DS0PR17CA0010.namprd17.prod.outlook.com (2603:10b6:8:191::26)
 by CYYPR12MB8921.namprd12.prod.outlook.com (2603:10b6:930:c7::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Fri, 24 May
 2024 22:55:31 +0000
Received: from DS3PEPF0000C37B.namprd04.prod.outlook.com
 (2603:10b6:8:191:cafe::2a) by DS0PR17CA0010.outlook.office365.com
 (2603:10b6:8:191::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Fri, 24 May 2024 22:55:31 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF0000C37B.mail.protection.outlook.com (10.167.23.5) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:55:31 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:26 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:26 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:55:25 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9b2213b-1a20-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dnrCBpNrmfHy7+Kd8KYhHEIbY0uedHDgUD5UuN6PTWYJGbhtzdX+pat6YhwZd8YhtS06+qszkr+qsqqKIUDqPNCHYmb3uReTcLly5PkDnH1p8tKGZeQ769VKEDUI7OW3bCuXkDYTE9LpfI6hqdNV6UuuWF1YdBg8Ydoh6tCFpyMZkebFMyNdW3qkJmHFfVzHoPSJcstujIDLcfsdbqJ6COqzth/yZzzeFfhEo5Xe7frSZySoa80F/eA2BL5rMVqemVsBSycna6Yb+882JTfPrpuCAvQCaMmHrP7FGyDcjy2Vjso5lv40CkhYjqnRK3dRqlLGNcsxWCof/mKppkfQUg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=bm9n7d0lv39csuk5HxxqaBGOy78rYQuqOosnK5AZnWo=;
 b=jDfES2qfzbdw9HZD2zYMxLw1k9YsFzQX4MTi0v6q2BGnyF3CXEj7u3lzhZsFCn1byxjzs2M1QF58ur8uEDOOmGhNcMwpQ2qIj2Z/4jNKdKS8amclzW4zoN8IQfsYXTM7Wp7+PGcGHXFSnlOmo3SrZ39/7/2a89r63+kBojiKRNFjYWivCLQOqKG5RMPld6BK9dhEOPNxNPLTbG9YUbXiAlDT1VB448cIgLX7VQW8vnnVWdpq6mlZb/3MqSdAX6yqnRhXW28/6ztOP0O3JUVnbXSEIeqvu8iagFZHXJhUOjoi5uvOzamy1XELIjCWwoI3rlWufWF3JDE6mGsWoMAjgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bm9n7d0lv39csuk5HxxqaBGOy78rYQuqOosnK5AZnWo=;
 b=IwQEMAv+Oe/Lx2FiUMjPDh28/vGS7kgI+uRC+EcE93yD+IXaOpvb7JGIRs/x9tf2FZ9c2QNa/FPAFtcpxwLruKZRVAvufSPIAnD3RFRFMh5MJ1FHhG6kqbAqsGgTxEdRqTt4Fx87RrfE33/osYpCg/8QpykRycCDZBGPALbudZY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, <Volodymyr_Babchuk@epam.com>, Henry Wang
	<xin.wang2@amd.com>
Subject: [PATCH v4 4/4] docs/features/dom0less: Update the late XenStore init protocol
Date: Fri, 24 May 2024 15:55:22 -0700
Message-ID: <20240524225522.2878481-4-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB05.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37B:EE_|CYYPR12MB8921:EE_
X-MS-Office365-Filtering-Correlation-Id: 536ff34d-cb6c-4bc7-555f-08dc7c449cea
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?3ZTtasn9ADcl2gwGhIWw2r5zzUZzb2+JF2TMnHN/LUZ702OrTzcXGgC3LbqV?=
 =?us-ascii?Q?Lz2s2SG8RzBwXzZnDheg/6dP5ASdKx+yFOawIYRzjNXEMxSO4++6AUhPj9sK?=
 =?us-ascii?Q?9UH6lpSAfbuApknpQ1vpNC3R9pCQNa7wljFHsOKj8SaAP9F7si3CsW7yv5Lx?=
 =?us-ascii?Q?xiYekh+92jHeNOANh2iXVM1+E7dECzj2d15F6TAGuaa+VnH6rbpax5qU2Wo0?=
 =?us-ascii?Q?oW92xCC3mEnXLXHW5Sgt1IaEfgnK9PZakScP0fEOZYJn4dyKbICaRRU7kGWT?=
 =?us-ascii?Q?L9KkJl+0JTSmCwyt53mu25iAcdEtx1bxFq0vZwBzus6/gQ4mi2PXANOd1Fnt?=
 =?us-ascii?Q?qkoVuWDejpMTdHDcZ/8jEnpkhUTdOX1ojzZIaR6Wxqykom+SjnbZW4l0i031?=
 =?us-ascii?Q?JFZLtgE9SHHMQrow9KADZ0rmtyhjT2RCxLy5dUqsANYAHG7LBRAYVXhpst3x?=
 =?us-ascii?Q?bJJxeo3BBNTG86cAZTPYZ7yUwMQkXVY29Ebpd7Lf7ohGG/aKEwC/j7JBL3rD?=
 =?us-ascii?Q?S88uggv1/xdHToUPdQkQ5aoVI4AVMZy49emXOypRYCN+xGLlXKuWJBWVf6+c?=
 =?us-ascii?Q?bJVDFZYj93IycEm2mD2d8giq7Kb1A24Zgk1DWLmoONh/MT/lSvq1E9iW/9qg?=
 =?us-ascii?Q?9LQqzEQzyTXVFXlc5fHk8snClDJXmuEws1YNvoY7KHU0+ZUyvcLx/tFr9C2n?=
 =?us-ascii?Q?7VzwCdh+LXvFttB0opPW1U7mh7ZBiAI3LqWh8pM6cAyM3uFOPubp0pFPus4W?=
 =?us-ascii?Q?Dlrk8KSdUDy54s3EizePCRMY10yNLOJlklp1LQxkkYERDBuagctqcgobiI+g?=
 =?us-ascii?Q?2/DwuSRRNc/KWSkGD+gU5cpIf/vsSHBtvMlAxCkVon6XBH/R9XpjHIgfBk/G?=
 =?us-ascii?Q?15Qpds+H/20VFFXnEK9K8vQCEkGPaYI1nJNEmYwRFFznZeQ8z5NK4A5xnioF?=
 =?us-ascii?Q?Z3rNruJKrbWglZe4ij/PJBhIX/qLOVQ3xbt44h7A+sqHP050xqXQphFjB8nz?=
 =?us-ascii?Q?rfvbzHfNdIh0xQ2nns3Ojhe7RXG47J+ZH+tNJCNr0lXhrqAWpkq6S6JbySEg?=
 =?us-ascii?Q?GZuY9YiKGivY95QJ7dP5AR7MgryuMFzQJq0QqI3sU7fyUm5w7qgnm2IG7U4e?=
 =?us-ascii?Q?PESaZubfER1A07ByZIooYD39ww4s0ONrBUMxTokZERxOf5DBcLcb4PbEN5BC?=
 =?us-ascii?Q?NCV9rCNnu4A6hbJQ2Guk9uXlbG7TLhhTk/70X0l2X8WFOiCRHVjQo6dw97wS?=
 =?us-ascii?Q?bdrALJNd1Nb1YLIE5V7ZcUrv43CYWEdtCWMw0CXUAxrJAi1fR0zSsK9kJiTy?=
 =?us-ascii?Q?25R2K6mYbVZd6NV8dw64ux4IKTpn8HQd9E4ipF1M7niILpF9RdE+7ZFUDac5?=
 =?us-ascii?Q?zVYSsxFVx5/eON+mvSZXh9DSGRE9?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:55:31.1053
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 536ff34d-cb6c-4bc7-555f-08dc7c449cea
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF0000C37B.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8921

From: Henry Wang <xin.wang2@amd.com>

With the new allocation strategy of Dom0less DomUs XenStore page,
update the doc of the late XenStore init protocol accordingly.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
 docs/features/dom0less.pandoc | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/docs/features/dom0less.pandoc b/docs/features/dom0less.pandoc
index 725afa0558..8b178edee0 100644
--- a/docs/features/dom0less.pandoc
+++ b/docs/features/dom0less.pandoc
@@ -110,9 +110,10 @@ hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.
 The implementation works as follows:
 - Xen allocates the xenstore event channel for each dom0less domU that
   has the "xen,enhanced" property, and sets HVM_PARAM_STORE_EVTCHN
-- Xen does *not* allocate the xenstore page and sets HVM_PARAM_STORE_PFN
-  to ~0ULL (invalid)
-- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid
+- Xen allocates the xenstore page and sets HVM_PARAM_STORE_PFN as well
+  as the connection status to XENSTORE_RECONNECT.
+- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to
+  ~0ULL (invalid) or the connection status is *not* XENSTORE_CONNECTED.
     - Old kernels will continue without xenstore support (Note: some old
       buggy kernels might crash because they don't check the validity of
       HVM_PARAM_STORE_PFN before using it! Disable "xen,enhanced" in
@@ -121,13 +122,14 @@ The implementation works as follows:
       channel (HVM_PARAM_STORE_EVTCHN) before continuing with the
       initialization
 - Once dom0 is booted, init-dom0less is executed:
-    - it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN
+    - it gets the xenstore shared page from HVM_PARAM_STORE_PFN
     - it calls xs_introduce_domain
 - Xenstored notices the new domain, initializes interfaces as usual, and
   sends an event channel notification to the domain using the xenstore
   event channel (HVM_PARAM_STORE_EVTCHN)
 - The Linux domU kernel receives the event channel notification, checks
-  HVM_PARAM_STORE_PFN again and continue with the initialization
+  HVM_PARAM_STORE_PFN and the connection status again and continue with
+  the initialization
 
 
 Limitations
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:55:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:55:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729966.1135340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdp4-0004ot-IV; Fri, 24 May 2024 22:55:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729966.1135340; Fri, 24 May 2024 22:55:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdp4-0004og-Ec; Fri, 24 May 2024 22:55:38 +0000
Received: by outflank-mailman (input) for mailman id 729966;
 Fri, 24 May 2024 22:55:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4rgR=M3=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1sAdp2-0003hQ-Po
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:55:36 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9ee1500-1a20-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 00:55:34 +0200 (CEST)
Received: from DS0PR17CA0007.namprd17.prod.outlook.com (2603:10b6:8:191::14)
 by DS0PR12MB6534.namprd12.prod.outlook.com (2603:10b6:8:c1::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.20; Fri, 24 May
 2024 22:55:30 +0000
Received: from DS3PEPF0000C37B.namprd04.prod.outlook.com
 (2603:10b6:8:191:cafe::fa) by DS0PR17CA0007.outlook.office365.com
 (2603:10b6:8:191::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend
 Transport; Fri, 24 May 2024 22:55:25 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF0000C37B.mail.protection.outlook.com (10.167.23.5) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7611.14 via Frontend Transport; Fri, 24 May 2024 22:55:25 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:24 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 24 May
 2024 17:55:24 -0500
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 24 May 2024 17:55:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9ee1500-1a20-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H0ZwdWwFHgIU2CPMt51i20GT77LOt4RDg0m3ViVoe+VbLR2+nwy2hqJMwpFO2IuUd6Ca/FGhpyHnEbjp1ogHiQPr7vB7SP8LWqc+AEkoMXGuJJorunMj/W7SP+9ekbG0L2wefMc3Rm/7qxWosBX/3fQ/F+sasNMBA6i9vNMc3EBLaKHRc4/uIN+9Gg5D+HNiSrt3TsoL8alcUa+yFJVOMGxBpUbRpK3WguMej4o/RJxNyG7A5fdAHmVYAB9dya0Zy9/OzECyRiqluOb3QtYtkbhPW6tA0CX1ch5rF7UkH3FEfZOE47NMfSu5CTtm8J4yrlL7hOtMDHGYp+L3NMhKfg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kGJKl+YgAt+elTNKQCErwYXKgmqb2xKN/qNfttPWbAw=;
 b=Gt6IeM3OQ9OlNsZBJvR/YroCFBRM8YlaVUHqhTkdhLh9kI0G3RBF47rZlNUg7prCjaKDS0f27DJzOd8V72B/B2Gi+E79GY2NcGCzhwZ15s5+zrySnGA10L/NzsRLdtUX382X2hMeynof0XJUxhBLuxCbhRDiZE60JjxQA8m3O6+t1/M5u1Mihns8SFSsCN9IRWZnV47XE1xrPvorMUboTSvkfFNeaNmxO8g+vVtJwdmWpR2l4RX3l0+hQ8PmKzoQtTusOgrHks9ZE2HtLFYqoHrJQHiwgtK6IExDffab3tjVtiYbTcOz0f6SE7cfWvgA2Zc9SAGl9MXUXIFe/4UFhQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kGJKl+YgAt+elTNKQCErwYXKgmqb2xKN/qNfttPWbAw=;
 b=Kss64V5TJPeoTcxFJKrTeGpbOWUFId2KMI6M+sZsx5WXqRgqyQUu/GkXmFjVtywU99TzzsXOqWuPubE5lJlB0CuwpIhcvF7ImB5j05dr/34yTwK75zUOalGvL3Y4bbk8svTsNHNuhxJSPtsBgV1vWW2bLlEgTYVkLuXTIEOm6XM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, <Volodymyr_Babchuk@epam.com>, Henry Wang
	<xin.wang2@amd.com>, Alec Kwapis <alec.kwapis@medtronic.com>, "Daniel P .
 Smith" <dpsmith@apertussolutions.com>, Stefano Stabellini
	<stefano.stabellini@amd.com>
Subject: [PATCH v4 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs from hypervisor
Date: Fri, 24 May 2024 15:55:20 -0700
Message-ID: <20240524225522.2878481-2-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37B:EE_|DS0PR12MB6534:EE_
X-MS-Office365-Filtering-Correlation-Id: 7b482132-61e9-491c-4234-08dc7c44996c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|376005|1800799015|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?b9wWaVUXtHuKsJimiwJE+gdixTUTt56rWyUww7V5ZOSjdNSw832v/fKtmPVp?=
 =?us-ascii?Q?saMupljRZ4Ou4H6ufSNAv7VOQEYsRSGM4VSwJC8gY5MWWf4ggYh3q4GTg9sK?=
 =?us-ascii?Q?zDZGWLrZ+wTB5BF5aURYCi38Gizq1drsgHNjzRKqaIDCSXWrSLFCPDaIWOt9?=
 =?us-ascii?Q?z0KfPI6XidMECWoYSyqHSKG1wkkJ5qNPrMiJEdld6eSLWLqfe0muqXHuzf2O?=
 =?us-ascii?Q?7/2Zx9B4SQqWstFaNernwQ4hil2zn69EbU7y3hs/1r4Led1BHaCKAfl4slMN?=
 =?us-ascii?Q?6nKucpsshlZWNS+Jg1zvjTy5aNGkFHu1vpX1fNFwG8ItGJrQxIG43DzfmdbY?=
 =?us-ascii?Q?jNZu2wzzb2BDrrpmC+4P6rZ+RMDKYd9FTDGdwLHpqWOjujlz8QzAQtGHseaN?=
 =?us-ascii?Q?mQOdqgFfMDPOQ9PljvyF8BywXBOS7F0Ep2yiHfXmKvsyuiNq+D+BiGRrVjPa?=
 =?us-ascii?Q?l5TpPq3iXgEKZC1E+UNpd/QM+dIH3TDm9uXsgFlpA38ifyWE7kCdDS2jvaG3?=
 =?us-ascii?Q?TUifqczF7gOZdStPQlEyqVAO7610RRsS90EVwVB/jN+bfTi2FEokN6sTo8eD?=
 =?us-ascii?Q?o1WHskQSSuFsBK9EZiYKft2mdDdQvS+Lo5Iz8p2DN8J7062Wz/FKZi0Gko0Z?=
 =?us-ascii?Q?uGyqM2j1ghFCNuMtKwakH9FHBSbYVR1R9E5vlsKB4CZUN0xRmbAQHXIvfPBj?=
 =?us-ascii?Q?hV811+pc/Hnn+3oJXeqNTe0L25sPVuRIx5jV6/pnzZwLzFKc1MlPpyQV4tkC?=
 =?us-ascii?Q?8NGIlqFwkTu1yfoCjlTtAq1vY6gWWAAziNOiDaCuEUmphKajq/SjXLeFBYHm?=
 =?us-ascii?Q?06qZJfmqqKNEUj0KsdgKVq+TbbEmV9pF9mheXMsSvhn2HS4c6fplev6DMKAX?=
 =?us-ascii?Q?k2OsPo/CtvThgIp01OknEy4SicbpoGCZbJfj2QR4HMBPF8vvvzJEPlaSd/+V?=
 =?us-ascii?Q?3NjEOCNi4JOYL9ClVobtb6e4XfyxWHTOCtWpLzUaWdQj85lo5IvEUODc6/mS?=
 =?us-ascii?Q?VCgwyVsLpGz6os6x87M875A89ylPhOquj5CYeGHDnnRvAZMrqPRij4u09hiM?=
 =?us-ascii?Q?uF4It7aNBYNbrxxvha47DLZ/loF4qLNh2X3trWeN6KdvhZDfdE+Za3d/rdmA?=
 =?us-ascii?Q?oXWWyHqHYkx0Yph5tuJBblmtEzWF65PVxBVVn6uYVlZoKKGGedX6uIYknuqO?=
 =?us-ascii?Q?RrPkhwb8NJl86a1mgCk1fy5/FsZAb0I0sz7tBxpvSr3sP7oh9th7anzmlhtx?=
 =?us-ascii?Q?pOoppul0kdT/pdHSE75JhHt3+klpFcNtDoSf9JbialTPdnGooU+DDQtKgQgo?=
 =?us-ascii?Q?+AjB0/KOljPRli71sBwuIYOJ3brYMmnZg/ep6B9OeQMcXHT+VEQtJY7/hjw0?=
 =?us-ascii?Q?t5gfU+ko4gb8UD/iabrRIiG173er?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(376005)(1800799015)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 22:55:25.2459
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b482132-61e9-491c-4234-08dc7c44996c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF0000C37B.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6534

From: Henry Wang <xin.wang2@amd.com>

There are use cases (for example using the PV driver) in Dom0less
setup that require Dom0less DomUs start immediately with Dom0, but
initialize XenStore later after Dom0's successful boot and call to
the init-dom0less application.

An error message can seen from the init-dom0less application on
1:1 direct-mapped domains:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

The "magic page" is a terminology used in the toolstack as reserved
pages for the VM to have access to virtual platform capabilities.
Currently the magic pages for Dom0less DomUs are populated by the
init-dom0less app through populate_physmap(), and populate_physmap()
automatically assumes gfn == mfn for 1:1 direct mapped domains. This
cannot be true for the magic pages that are allocated later from the
init-dom0less application executed in Dom0. For domain using statically
allocated memory but not 1:1 direct-mapped, similar error "failed to
retrieve a reserved page" can be seen as the reserved memory list is
empty at that time.

Since for init-dom0less, the magic page region is only for XenStore.
To solve above issue, this commit allocates the XenStore page for
Dom0less DomUs at the domain construction time. The PFN will be
noted and communicated to the init-dom0less application executed
from Dom0. To keep the XenStore late init protocol, set the connection
status to XENSTORE_RECONNECT.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 xen/arch/arm/dom0less-build.c | 55 ++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 74f053c242..2963ecc0b3 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 #include <xen/device_tree.h>
+#include <xen/domain_page.h>
 #include <xen/err.h>
 #include <xen/event.h>
 #include <xen/grant_table.h>
@@ -10,6 +11,8 @@
 #include <xen/sizes.h>
 #include <xen/vmap.h>
 
+#include <public/io/xs_wire.h>
+
 #include <asm/arm64/sve.h>
 #include <asm/dom0less-build.h>
 #include <asm/domain_build.h>
@@ -739,6 +742,53 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
     return 0;
 }
 
+#define XENSTORE_PFN_OFFSET 1
+static int __init alloc_xenstore_page(struct domain *d)
+{
+    struct page_info *xenstore_pg;
+    struct xenstore_domain_interface *interface;
+    mfn_t mfn;
+    gfn_t gfn;
+    int rc;
+
+    if ( (UINT_MAX - d->max_pages) < 1 )
+    {
+        printk(XENLOG_ERR "%pd: Over-allocation for d->max_pages by 1 page.\n",
+               d);
+        return -EINVAL;
+    }
+    d->max_pages += 1;
+    xenstore_pg = alloc_domheap_page(d, MEMF_bits(32));
+    if ( xenstore_pg == NULL && is_64bit_domain(d) )
+        xenstore_pg = alloc_domheap_page(d, 0);
+    if ( xenstore_pg == NULL )
+        return -ENOMEM;
+
+    mfn = page_to_mfn(xenstore_pg);
+    if ( !mfn_x(mfn) )
+        return -ENOMEM;
+
+    if ( !is_domain_direct_mapped(d) )
+        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE +
+                           (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
+    else
+        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
+
+    rc = guest_physmap_add_page(d, gfn, mfn, 0);
+    if ( rc )
+    {
+        free_domheap_page(xenstore_pg);
+        return rc;
+    }
+
+    d->arch.hvm.params[HVM_PARAM_STORE_PFN] = gfn_x(gfn);
+    interface = map_domain_page(mfn);
+    interface->connection = XENSTORE_RECONNECT;
+    unmap_domain_page(interface);
+
+    return 0;
+}
+
 static int __init construct_domU(struct domain *d,
                                  const struct dt_device_node *node)
 {
@@ -839,7 +889,10 @@ static int __init construct_domU(struct domain *d,
         rc = alloc_xenstore_evtchn(d);
         if ( rc < 0 )
             return rc;
-        d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
+
+        rc = alloc_xenstore_page(d);
+        if ( rc < 0 )
+            return rc;
     }
 
     return rc;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 24 22:58:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 22:58:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.729994.1135349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdrb-0006VX-Ux; Fri, 24 May 2024 22:58:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 729994.1135349; Fri, 24 May 2024 22:58:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdrb-0006VQ-SM; Fri, 24 May 2024 22:58:15 +0000
Received: by outflank-mailman (input) for mailman id 729994;
 Fri, 24 May 2024 22:58:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAdrb-0006VK-1b
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 22:58:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAdra-0003Hl-Ik; Fri, 24 May 2024 22:58:14 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAdra-0003Ng-CS; Fri, 24 May 2024 22:58:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=XeDCiYZVFF+cmokofTmJDlj2/4rsSpqWzJHWL5giGz4=; b=Ts2+E48IksBFbOmg7f8l7dtV6E
	ZG74J4R3JqdaVe8jHX64C96dktL9T9C8Wx8Dz/do802KOfv/3DoDFwnO+z60rUe3IwYflqO5PUgcl
	paGqmxPRSsVS/NQVZrlZzFEM+veKt8o0it595CAxJoYj3nh9NbrU7xuRAbaVVi7yx9RU=;
Message-ID: <0a153f00-04bf-4c98-ae98-c904ee52af13@xen.org>
Date: Fri, 24 May 2024 23:58:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Juergen Gross <jgross@suse.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
 <alpine.DEB.2.22.394.2405241536510.2557291@ubuntu-linux-20-04-desktop>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405241536510.2557291@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 23:49, Stefano Stabellini wrote:
> On Fri, 24 May 2024, Julien Grall wrote:
>> Hi Henry,
>>
>> + Juergen as the Xenstore maintainers. I'd like his opinion on the approach.
>> The documentation of the new logic is in:
>>
>> https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
>>
>> FWIW I am happy in principle with the logic (this is what we discussed on the
>> call last week). Some comments below.
>>
>> On 17/05/2024 04:21, Henry Wang wrote:
>>> There are use cases (for example using the PV driver) in Dom0less
>>> setup that require Dom0less DomUs start immediately with Dom0, but
>>> initialize XenStore later after Dom0's successful boot and call to
>>> the init-dom0less application.
>>>
>>> An error message can seen from the init-dom0less application on
>>> 1:1 direct-mapped domains:
>>> ```
>>> Allocating magic pages
>>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>>> Error on alloc magic pages
>>> ```
>>>
>>> The "magic page" is a terminology used in the toolstack as reserved
>>> pages for the VM to have access to virtual platform capabilities.
>>> Currently the magic pages for Dom0less DomUs are populated by the
>>> init-dom0less app through populate_physmap(), and populate_physmap()
>>> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
>>> cannot be true for the magic pages that are allocated later from the
>>> init-dom0less application executed in Dom0. For domain using statically
>>> allocated memory but not 1:1 direct-mapped, similar error "failed to
>>> retrieve a reserved page" can be seen as the reserved memory list is
>>> empty at that time.
>>>
>>> Since for init-dom0less, the magic page region is only for XenStore.
>>> To solve above issue, this commit allocates the XenStore page for
>>> Dom0less DomUs at the domain construction time. The PFN will be
>>> noted and communicated to the init-dom0less application executed
>>> from Dom0. To keep the XenStore late init protocol, set the connection
>>> status to XENSTORE_RECONNECT.
>>
>> So this commit is allocating the page, but it will not be used by
>> init-dom0less until the next patch. But Linux could use it. So would this
>> break bisection? If so, then I think patch #3 needs to be folded in this
>> patch.
> 
> I think that's fine, 

I am not sure what you mean. Are you saying it is ok to break bisection?

> I'll leave that with you on commit.

I am sorry but I don't think the folding should be done on commit. It 
should happen before hand because the commit message will also need to 
be updated.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:02:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:02:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730001.1135360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdvU-0000a7-Dq; Fri, 24 May 2024 23:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730001.1135360; Fri, 24 May 2024 23:02:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdvU-0000a0-Ah; Fri, 24 May 2024 23:02:16 +0000
Received: by outflank-mailman (input) for mailman id 730001;
 Fri, 24 May 2024 23:02:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAdvS-0000Zu-Uh
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:02:14 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a747bfc9-1a21-11ef-b4bb-af5377834399;
 Sat, 25 May 2024 01:02:12 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 1AAB2632A9;
 Fri, 24 May 2024 23:02:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 005FDC2BBFC;
 Fri, 24 May 2024 23:02:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a747bfc9-1a21-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716591730;
	bh=ry/BE4GGdY0GMaJHtALiuLIzm7exxh0X0tdD8b8xN5E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JkPxrEmpEh0ILdCde0vkTJZuoyrurKUZCXKCAPmsZBLayYPXhfCNjLZ+XODgxrBMD
	 Bf/eib3fBcj6O20JELw/x6JXxtVIC3l4XfHzpMty3lvAOb+BreHxTtsVoPXVbvSm6G
	 gxo+5truNZVmbrQxpjVc9J1rAb3PrGAk+NKjZZ8ct5oCNPBr/QtTPox7oFSd/nbcLl
	 D8IzLXErKxOvTsdbAHUHanPsuCHNMQ6rK7K9pm89rT3AaEugiEubYsutYgKSCuO1Qe
	 UgchuvaLLJmBF+VxqBxLp57TDtg6zRtBqYT0HZWzI5NUg/18EMV1jKQVnoGrIhL6wX
	 5eA55bmgA7zCQ==
Date: Fri, 24 May 2024 16:02:08 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Alec Kwapis <alec.kwapis@medtronic.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
In-Reply-To: <0a153f00-04bf-4c98-ae98-c904ee52af13@xen.org>
Message-ID: <alpine.DEB.2.22.394.2405241559480.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-3-xin.wang2@amd.com> <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org> <alpine.DEB.2.22.394.2405241536510.2557291@ubuntu-linux-20-04-desktop>
 <0a153f00-04bf-4c98-ae98-c904ee52af13@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Julien Grall wrote:
> Hi Stefano,
> 
> On 24/05/2024 23:49, Stefano Stabellini wrote:
> > On Fri, 24 May 2024, Julien Grall wrote:
> > > Hi Henry,
> > > 
> > > + Juergen as the Xenstore maintainers. I'd like his opinion on the
> > > approach.
> > > The documentation of the new logic is in:
> > > 
> > > https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
> > > 
> > > FWIW I am happy in principle with the logic (this is what we discussed on
> > > the
> > > call last week). Some comments below.
> > > 
> > > On 17/05/2024 04:21, Henry Wang wrote:
> > > > There are use cases (for example using the PV driver) in Dom0less
> > > > setup that require Dom0less DomUs start immediately with Dom0, but
> > > > initialize XenStore later after Dom0's successful boot and call to
> > > > the init-dom0less application.
> > > > 
> > > > An error message can seen from the init-dom0less application on
> > > > 1:1 direct-mapped domains:
> > > > ```
> > > > Allocating magic pages
> > > > memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> > > > Error on alloc magic pages
> > > > ```
> > > > 
> > > > The "magic page" is a terminology used in the toolstack as reserved
> > > > pages for the VM to have access to virtual platform capabilities.
> > > > Currently the magic pages for Dom0less DomUs are populated by the
> > > > init-dom0less app through populate_physmap(), and populate_physmap()
> > > > automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> > > > cannot be true for the magic pages that are allocated later from the
> > > > init-dom0less application executed in Dom0. For domain using statically
> > > > allocated memory but not 1:1 direct-mapped, similar error "failed to
> > > > retrieve a reserved page" can be seen as the reserved memory list is
> > > > empty at that time.
> > > > 
> > > > Since for init-dom0less, the magic page region is only for XenStore.
> > > > To solve above issue, this commit allocates the XenStore page for
> > > > Dom0less DomUs at the domain construction time. The PFN will be
> > > > noted and communicated to the init-dom0less application executed
> > > > from Dom0. To keep the XenStore late init protocol, set the connection
> > > > status to XENSTORE_RECONNECT.
> > > 
> > > So this commit is allocating the page, but it will not be used by
> > > init-dom0less until the next patch. But Linux could use it. So would this
> > > break bisection? If so, then I think patch #3 needs to be folded in this
> > > patch.
> > 
> > I think that's fine, 
> 
> I am not sure what you mean. Are you saying it is ok to break bisection?

No, I meant to say that it is fine to merge on commit.


> > I'll leave that with you on commit.
> 
> I am sorry but I don't think the folding should be done on commit. It should
> happen before hand because the commit message will also need to be updated.

Understood. I'll send one more version with the patches merged (ideally
with an ack?)


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:06:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:06:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730007.1135370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdzj-0001UQ-UR; Fri, 24 May 2024 23:06:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730007.1135370; Fri, 24 May 2024 23:06:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAdzj-0001UJ-RP; Fri, 24 May 2024 23:06:39 +0000
Received: by outflank-mailman (input) for mailman id 730007;
 Fri, 24 May 2024 23:06:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAdzi-0001UD-A2
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:06:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAdzh-0003po-PT; Fri, 24 May 2024 23:06:37 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAdzh-0003ys-JQ; Fri, 24 May 2024 23:06:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=GMaAu7+p3av2dZWMaIa3mSBAbBCabMdwFqvx2RFKTzc=; b=uYKRMJKUiLNMqJUaXt8o94mYv+
	FtiPken7mVIzeUcRDTDx5tMWvhO3nYjNu4FzJKNbjtxxyVUk0rC22JlSkBsT1+9xoCAHbj9DvrSFM
	oHze7jjE7t7qGphvLDXQEukIqN/wFEU6XKFWMQpF7J2CFoW4OZhGfA5weBtaD6xAdNvU=;
Message-ID: <697aadfd-a8c1-4f1b-8806-6a5acbf343ba@xen.org>
Date: Sat, 25 May 2024 00:06:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com,
 Volodymyr_Babchuk@epam.com, Henry Wang <xin.wang2@amd.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Oleksii <oleksii.kurochko@gmail.com>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
 <20240524225522.2878481-2-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524225522.2878481-2-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

You have sent a new version. But you didn't reply to Juergen's comment.

While he is not the maintainer of the Arm code, he is the Xenstore 
maintainer. Even if I agree with this approach I don't feel this is 
right to even ack this patch without pending questions addressed.

In this case, we are changing yet another time the sequence for 
initializing Xenstore dom0less domain. I would rather not want to have 
to change a 4th time...

I don't think it is a big problem if this is not merged for the code 
freeze as this is technically a bug fix.

Cheers,

On 24/05/2024 23:55, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> The "magic page" is a terminology used in the toolstack as reserved
> pages for the VM to have access to virtual platform capabilities.
> Currently the magic pages for Dom0less DomUs are populated by the
> init-dom0less app through populate_physmap(), and populate_physmap()
> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> cannot be true for the magic pages that are allocated later from the
> init-dom0less application executed in Dom0. For domain using statically
> allocated memory but not 1:1 direct-mapped, similar error "failed to
> retrieve a reserved page" can be seen as the reserved memory list is
> empty at that time.
> 
> Since for init-dom0less, the magic page region is only for XenStore.
> To solve above issue, this commit allocates the XenStore page for
> Dom0less DomUs at the domain construction time. The PFN will be
> noted and communicated to the init-dom0less application executed
> from Dom0. To keep the XenStore late init protocol, set the connection
> status to XENSTORE_RECONNECT.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
>   xen/arch/arm/dom0less-build.c | 55 ++++++++++++++++++++++++++++++++++-
>   1 file changed, 54 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..2963ecc0b3 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -1,5 +1,6 @@
>   /* SPDX-License-Identifier: GPL-2.0-only */
>   #include <xen/device_tree.h>
> +#include <xen/domain_page.h>
>   #include <xen/err.h>
>   #include <xen/event.h>
>   #include <xen/grant_table.h>
> @@ -10,6 +11,8 @@
>   #include <xen/sizes.h>
>   #include <xen/vmap.h>
>   
> +#include <public/io/xs_wire.h>
> +
>   #include <asm/arm64/sve.h>
>   #include <asm/dom0less-build.h>
>   #include <asm/domain_build.h>
> @@ -739,6 +742,53 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
>       return 0;
>   }
>   
> +#define XENSTORE_PFN_OFFSET 1
> +static int __init alloc_xenstore_page(struct domain *d)
> +{
> +    struct page_info *xenstore_pg;
> +    struct xenstore_domain_interface *interface;
> +    mfn_t mfn;
> +    gfn_t gfn;
> +    int rc;
> +
> +    if ( (UINT_MAX - d->max_pages) < 1 )
> +    {
> +        printk(XENLOG_ERR "%pd: Over-allocation for d->max_pages by 1 page.\n",
> +               d);
> +        return -EINVAL;
> +    }
> +    d->max_pages += 1;
> +    xenstore_pg = alloc_domheap_page(d, MEMF_bits(32));
> +    if ( xenstore_pg == NULL && is_64bit_domain(d) )
> +        xenstore_pg = alloc_domheap_page(d, 0);
> +    if ( xenstore_pg == NULL )
> +        return -ENOMEM;
> +
> +    mfn = page_to_mfn(xenstore_pg);
> +    if ( !mfn_x(mfn) )
> +        return -ENOMEM;
> +
> +    if ( !is_domain_direct_mapped(d) )
> +        gfn = gaddr_to_gfn(GUEST_MAGIC_BASE +
> +                           (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
> +    else
> +        gfn = gaddr_to_gfn(mfn_to_maddr(mfn));
> +
> +    rc = guest_physmap_add_page(d, gfn, mfn, 0);
> +    if ( rc )
> +    {
> +        free_domheap_page(xenstore_pg);
> +        return rc;
> +    }
> +
> +    d->arch.hvm.params[HVM_PARAM_STORE_PFN] = gfn_x(gfn);
> +    interface = map_domain_page(mfn);
> +    interface->connection = XENSTORE_RECONNECT;
> +    unmap_domain_page(interface);
> +
> +    return 0;
> +}
> +
>   static int __init construct_domU(struct domain *d,
>                                    const struct dt_device_node *node)
>   {
> @@ -839,7 +889,10 @@ static int __init construct_domU(struct domain *d,
>           rc = alloc_xenstore_evtchn(d);
>           if ( rc < 0 )
>               return rc;
> -        d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL;
> +
> +        rc = alloc_xenstore_page(d);
> +        if ( rc < 0 )
> +            return rc;
>       }
>   
>       return rc;

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:09:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:09:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730014.1135380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe2L-00026w-EE; Fri, 24 May 2024 23:09:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730014.1135380; Fri, 24 May 2024 23:09:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe2L-00026p-B0; Fri, 24 May 2024 23:09:21 +0000
Received: by outflank-mailman (input) for mailman id 730014;
 Fri, 24 May 2024 23:09:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAe2J-00026j-HM
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:09:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe2J-0003yb-26; Fri, 24 May 2024 23:09:19 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe2I-000479-Rb; Fri, 24 May 2024 23:09:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=DCxSyPk4lJlnfqkWiqJVMqmq4VpfZklhKqTV9JyJuZY=; b=gbf8w3lW5uZUPj1UUzejxE9uzf
	rAimasYKpRQSqLg8dloTbbY9JnqgZM+q3j/I1i5j054lYHs4Q1r8Eg6Mnoiv0GJ2EC/4bdxxavxMn
	Qv7W0kYXKUx11s/YlWloqXa7Ud2PRf5JxvDPbmgIAzPUBWR7wFqqKwjXbJUzEvvNSkHc=;
Message-ID: <f754eff3-54f1-46cb-8c31-ed5ab564c7a2@xen.org>
Date: Sat, 25 May 2024 00:09:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>,
 Juergen Gross <jgross@suse.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
 <alpine.DEB.2.22.394.2405241536510.2557291@ubuntu-linux-20-04-desktop>
 <0a153f00-04bf-4c98-ae98-c904ee52af13@xen.org>
 <alpine.DEB.2.22.394.2405241559480.2557291@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2405241559480.2557291@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 25/05/2024 00:02, Stefano Stabellini wrote:
> On Fri, 24 May 2024, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 24/05/2024 23:49, Stefano Stabellini wrote:
>>> On Fri, 24 May 2024, Julien Grall wrote:
>>>> Hi Henry,
>>>>
>>>> + Juergen as the Xenstore maintainers. I'd like his opinion on the
>>>> approach.
>>>> The documentation of the new logic is in:
>>>>
>>>> https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
>>>>
>>>> FWIW I am happy in principle with the logic (this is what we discussed on
>>>> the
>>>> call last week). Some comments below.
>>>>
>>>> On 17/05/2024 04:21, Henry Wang wrote:
>>>>> There are use cases (for example using the PV driver) in Dom0less
>>>>> setup that require Dom0less DomUs start immediately with Dom0, but
>>>>> initialize XenStore later after Dom0's successful boot and call to
>>>>> the init-dom0less application.
>>>>>
>>>>> An error message can seen from the init-dom0less application on
>>>>> 1:1 direct-mapped domains:
>>>>> ```
>>>>> Allocating magic pages
>>>>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>>>>> Error on alloc magic pages
>>>>> ```
>>>>>
>>>>> The "magic page" is a terminology used in the toolstack as reserved
>>>>> pages for the VM to have access to virtual platform capabilities.
>>>>> Currently the magic pages for Dom0less DomUs are populated by the
>>>>> init-dom0less app through populate_physmap(), and populate_physmap()
>>>>> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
>>>>> cannot be true for the magic pages that are allocated later from the
>>>>> init-dom0less application executed in Dom0. For domain using statically
>>>>> allocated memory but not 1:1 direct-mapped, similar error "failed to
>>>>> retrieve a reserved page" can be seen as the reserved memory list is
>>>>> empty at that time.
>>>>>
>>>>> Since for init-dom0less, the magic page region is only for XenStore.
>>>>> To solve above issue, this commit allocates the XenStore page for
>>>>> Dom0less DomUs at the domain construction time. The PFN will be
>>>>> noted and communicated to the init-dom0less application executed
>>>>> from Dom0. To keep the XenStore late init protocol, set the connection
>>>>> status to XENSTORE_RECONNECT.
>>>>
>>>> So this commit is allocating the page, but it will not be used by
>>>> init-dom0less until the next patch. But Linux could use it. So would this
>>>> break bisection? If so, then I think patch #3 needs to be folded in this
>>>> patch.
>>>
>>> I think that's fine,
>>
>> I am not sure what you mean. Are you saying it is ok to break bisection?
> 
> No, I meant to say that it is fine to merge on commit.
> 
> 
>>> I'll leave that with you on commit.
>>
>> I am sorry but I don't think the folding should be done on commit. It should
>> happen before hand because the commit message will also need to be updated.
> 
> Understood. I'll send one more version with the patches merged (ideally
> with an ack?)
Sorry I don't feel it is right for me to ack this patch with the pending 
questions from Juergen.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:11:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:11:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730018.1135389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe3z-0003zD-NT; Fri, 24 May 2024 23:11:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730018.1135389; Fri, 24 May 2024 23:11:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe3z-0003z6-Kd; Fri, 24 May 2024 23:11:03 +0000
Received: by outflank-mailman (input) for mailman id 730018;
 Fri, 24 May 2024 23:11:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAe3y-0003xT-4a
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:11:02 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe3x-00045E-Oe; Fri, 24 May 2024 23:11:01 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe3x-0004Dj-Js; Fri, 24 May 2024 23:11:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YEE0nsQxqil5WQHvWkqkH3PfCuFWkuWW0g1Tlj6XwYo=; b=XLaO3LJj/qMaxf314cxkO2gwuM
	LmqGx7xxc80Jr8fFw0w7esmclWlfDaWvQGSkrDJV6TO6dfHLQxnTf5hTxG+hGPju8LkqwhLPf4X5t
	1QBPerFIk9ln1K3SmX2+D5ecuAnXpheZrDTU+0S8GWzgTthKZuuoBHxPRFDSNEZ+0Vtc=;
Message-ID: <30438a6b-a4d9-408e-b7cb-2d64b63ba5d4@xen.org>
Date: Sat, 25 May 2024 00:10:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 5/7] xen/arm: Add XEN_DOMCTL_dt_overlay and device
 attachment to domains
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Henry Wang <xin.wang2@amd.com>, Vikram Garhwal <fnu.vikram@xilinx.com>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
 <20240524221647.2860711-5-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524221647.2860711-5-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 23:16, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> In order to support the dynamic dtbo device assignment to a running
> VM, the add/remove of the DT overlay and the attach/detach of the
> device from the DT overlay should happen separately. Therefore,
> repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT
> overlay to Xen device tree, instead of assigning the device to the
> hardware domain at the same time. It is OK to change the sysctl behavior
> as this feature is experimental so changing sysctl behavior and breaking
> compatibility is OK.
> 
> Add the XEN_DOMCTL_dt_overlay with operations
> XEN_DOMCTL_DT_OVERLAY_ATTACH to do the device assignment to the domain.
> 
> The hypervisor firstly checks the DT overlay passed from the toolstack
> is valid. Then the device nodes are retrieved from the overlay tracker
> based on the DT overlay. The attach of the device is implemented by
> mapping the IRQ and IOMMU resources. All devices in the overlay are
> assigned to a single domain.
> 
> Also take the opportunity to make one coding style fix in sysctl.h.
> Introduce DT_OVERLAY_MAX_SIZE and use it to avoid repetitions of
> KB(500).
> 
> xen,reg is to be used to handle non-1:1 mappings but it is currently
> unsupported. For now return errors for not-1:1 mapped domains.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:13:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:13:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730031.1135400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe5x-0005AO-2I; Fri, 24 May 2024 23:13:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730031.1135400; Fri, 24 May 2024 23:13:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAe5w-0005AH-Vs; Fri, 24 May 2024 23:13:04 +0000
Received: by outflank-mailman (input) for mailman id 730031;
 Fri, 24 May 2024 23:13:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sAe5w-00058v-10
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:13:04 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe5v-0004Fy-KG; Fri, 24 May 2024 23:13:03 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sAe5v-0004Vg-FB; Fri, 24 May 2024 23:13:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=RL4ZX7klA5U0PTSL6uIu8UyuYsdsyCmM2AhccTjf8pM=; b=iqnRXM4ynJoYV+BoE75/fDllwU
	OMv7ohPnDZ45KU6nbaKaq+zXov8TWX0vzYbeYb5nM/PAc2ZCHiEiULtvQGTCIcOFd5NHhA79+L+uv
	O0W7fctweS8KO5zt4MhZvuf40w70X4ardH0c/RjhluWEWr+K9Sqz4XHaxlmswe6QTsIw=;
Message-ID: <0bd12ab7-98e3-4ef6-8305-5c78e95cab76@xen.org>
Date: Sat, 25 May 2024 00:13:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 7/7] docs: Add device tree overlay documentation
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: anthony@xenproject.org, sstabellini@kernel.org, bertrand.marquis@arm.com,
 michal.orzel@amd.com, Volodymyr_Babchuk@epam.com,
 Vikram Garhwal <fnu.vikram@xilinx.com>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>,
 Henry Wang <xin.wang2@amd.com>
References: <alpine.DEB.2.22.394.2405241515210.2557291@ubuntu-linux-20-04-desktop>
 <20240524221647.2860711-7-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240524221647.2860711-7-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 24/05/2024 23:16, Stefano Stabellini wrote:
> From: Vikram Garhwal <fnu.vikram@xilinx.com>
> 
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>   docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++
>   1 file changed, 82 insertions(+)
>   create mode 100644 docs/misc/arm/overlay.txt
> 
> diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt
> new file mode 100644
> index 0000000000..ef3ef792f7
> --- /dev/null
> +++ b/docs/misc/arm/overlay.txt
> @@ -0,0 +1,82 @@
> +# Device Tree Overlays support in Xen
> +
> +Xen experimentally supports dynamic device assignment to running
> +domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device
> +tree, and attaching them to a running domain with given $domid.
> +
> +Dynamic node assignment works in two steps:
> +
> +## Add/Remove device tree overlay to/from Xen device tree
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree.
> +
> +## Attach device from the DT overlay to domain
> +
> +1. Xen tools check the dtbo given and parse all other user provided arguments
> +2. Xen tools pass the dtbo to Xen hypervisor via hypercall.
> +3. Xen hypervisor attach the device to the user-provided $domid by
> +   mapping node resources in the DT overlay.
> +
> +# Examples
> +
> +Here are a few examples on how to use it.
> +
> +## Dom0 device add
> +
> +For assigning a device tree overlay to Dom0, user should firstly properly
> +prepare the DT overlay. More information about device tree overlays can be
> +found in [1]. Then, in Dom0, enter the following:
> +
> +    (dom0) xl dt-overlay add overlay.dtbo
> +
> +This will allocate the devices mentioned in overlay.dtbo to Xen device tree.
> +
> +To assign the newly added device from the dtbo to Dom0:
> +
> +    (dom0) xl dt-overlay attach overlay.dtbo 0
> +
> +Next, if the user wants to add the same device tree overlay to dom0
> +Linux, execute the following:
> +
> +    (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay
> +    (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo
> +
> +Finally if needed, the relevant Linux kernel drive can be loaded using:
> +
> +    (dom0) modprobe module_name.ko
> +
> +## DomU device add/remove
> +
> +All the nodes in dtbo will be assigned to one domain. The user will need
> +to prepare a different dtbo for the domU. For example, the
> +`interrupt-parent` property of the DomU overlay should be changed to the
> +Xen hardcoded value `0xfde8` and the xen,reg property should be added to
> +specify the address mappings. If the domain is not 1:1 mapped, xen,reg
> +must be present. See the xen,reg format description in
> +docs/misc/arm/passthrough.txt. Below assumes the properly written DomU
> +dtbo is `overlay_domu.dtbo`.
> +
> +You need to set the `passthrough` property in the xl config file if you

s/You need/The user needs/ to match the rest of the documentation.

With that addresed:

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 24 23:19:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 May 2024 23:19:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730039.1135410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAeCU-0006Hz-OB; Fri, 24 May 2024 23:19:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730039.1135410; Fri, 24 May 2024 23:19:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAeCU-0006Hs-L0; Fri, 24 May 2024 23:19:50 +0000
Received: by outflank-mailman (input) for mailman id 730039;
 Fri, 24 May 2024 23:19:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K6bJ=M3=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sAeCT-0006Hm-T0
 for xen-devel@lists.xenproject.org; Fri, 24 May 2024 23:19:49 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1cae37b9-1a24-11ef-90a1-e314d9c70b13;
 Sat, 25 May 2024 01:19:48 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B5232632A9;
 Fri, 24 May 2024 23:19:46 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A089BC2BBFC;
 Fri, 24 May 2024 23:19:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cae37b9-1a24-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716592786;
	bh=Y8RlXmRX4MU9EfvBRdC6XMdY2PMPSUSBAEST/7avcQs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hMqGlYsPPCBOeqK6QvmrA2nq5vdENKDlK9opW7YtOBKLcNyRN4sj81JHygv7KuG98
	 t8A6FzpxfOTYTL3AezGMBehsGF0vJJoIVwtqmE7YJFrqBeUPKyNqGMWZV+k5vlnwwf
	 JVmz3u0m4gu0wJdhQb3uSYXe8HT/lv++prYKIJikqtC+xljbsTGMSMCLJD7VzrsV3Z
	 XpQDXg3gmi/DYqmBU26KWPCxgEVE/6a5R7mRsJkWcJRj2sFZI7mdZCXsUz8ugjg5ng
	 Oos5KsBz/mSENwbpPswdE05vwaV6E+duUJwvDBSLpwj0j1TMv/ROjJhqXQuqKoo59E
	 mlHccUCl5Dd8Q==
Date: Fri, 24 May 2024 16:19:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?J=C3=BCrgen_Gro=C3=9F?= <jgross@suse.com>
cc: Julien Grall <julien@xen.org>, Henry Wang <xin.wang2@amd.com>, 
    xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Alec Kwapis <alec.kwapis@medtronic.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
In-Reply-To: <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405241614370.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-3-xin.wang2@amd.com> <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org> <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-2138729461-1716592786=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2138729461-1716592786=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 May 2024, Jürgen Groß wrote:
> On 24.05.24 15:58, Julien Grall wrote:
> > Hi Henry,
> > 
> > + Juergen as the Xenstore maintainers. I'd like his opinion on the approach.
> > The documentation of the new logic is in:
> > 
> > https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
> > 
> > FWIW I am happy in principle with the logic (this is what we discussed on
> > the call last week). Some comments below.
> 
> I'm not against this logic, but I'm wondering why it needs to be so
> complicated.

Actually the reason I like it is that in my view, this is the simplest
approach. You allocate a domain, you also allocate the xenstore page
together with it. Initially the xenstore connection has an
"uninitialized" state, as it should be. That's it. At some point, when
xenstored is ready, the state changes to CONNECTED.


> Can't the domU itself allocate the Xenstore page from its RAM pages,
> write the PFN into the Xenstore grant tab entry, and then make it
> public via setting HVM_PARAM_STORE_PFN?

This is not simpler in my view


> The init-dom0less application could then check HVM_PARAM_STORE_PFN
> being set and call XS_introduce_domain.
> 
> Note that at least C-xenstored does not need the PFN of the Xenstore
> page, as it is just using GNTTAB_RESERVED_XENSTORE for mapping the
> page.
--8323329-2138729461-1716592786=:2557291--


From xen-devel-bounces@lists.xenproject.org Sat May 25 03:37:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 03:37:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730087.1135447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAiD8-0005n2-7U; Sat, 25 May 2024 03:36:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730087.1135447; Sat, 25 May 2024 03:36:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAiD8-0005mv-4n; Sat, 25 May 2024 03:36:46 +0000
Received: by outflank-mailman (input) for mailman id 730087;
 Sat, 25 May 2024 03:36:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiD7-0005ml-91; Sat, 25 May 2024 03:36:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiD6-0002Jt-WD; Sat, 25 May 2024 03:36:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiD6-0001Hy-KJ; Sat, 25 May 2024 03:36:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiD6-0008Gd-Jp; Sat, 25 May 2024 03:36:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6/asTN7iO/hEtDQqSuAJogzCyPbVgXJBDbupXdZacKg=; b=B0/Le13rePsMUSMI8JYYHnJg3U
	O1XzdRKgAq8f20TB+THkzVuZDx/7Ot2wLWbvcFIcNA0GWZcQ6qEAgAfgf41I/ewOtqHjmr3MtOKaL
	s6foaszy9+Pi+VB6Twot2gcWhlIjzkvqcSFG7GmGownfIjdMngcVGPFrQ8paZ/uubV6Q=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186142-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186142: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
X-Osstest-Versions-That:
    xen=2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 03:36:44 +0000

flight 186142 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186142/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0
baseline version:
 xen                  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5

Last test of basis   186139  2024-05-24 17:00:22 Z    0 days
Testing same since   186142  2024-05-25 00:02:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Henry Wang <xin.wang2@amd.com>
  Julien Grall <jgrall@amazon.com>
  Stefano Stabellini <stefano.stabellini@amd.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Vikram Garhwal <fnu.vikram@xilinx.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2172a01c4c..ac572152e5  ac572152e578a8853de0534384c1539ec21f11e0 -> smoke


From xen-devel-bounces@lists.xenproject.org Sat May 25 03:40:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 03:40:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730093.1135459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAiGn-0007EW-Pe; Sat, 25 May 2024 03:40:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730093.1135459; Sat, 25 May 2024 03:40:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAiGn-0007EP-KE; Sat, 25 May 2024 03:40:33 +0000
Received: by outflank-mailman (input) for mailman id 730093;
 Sat, 25 May 2024 03:40:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiGm-0007ED-Al; Sat, 25 May 2024 03:40:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiGm-0002VS-8E; Sat, 25 May 2024 03:40:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiGl-0001OD-W0; Sat, 25 May 2024 03:40:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAiGl-0005Xv-VW; Sat, 25 May 2024 03:40:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=V9WMuEhtIsSBkZha32Wr7wRscDsTg1FsRu1tXufcMyI=; b=NcQY8KPlxl6mgOfNhcw0Q2ezi5
	SyRtaDqN7Ozx823Krmi48+/+ydphVddVuxxxcfgBZ3uncgBf8hIfM9TWtj6yMLytZZwS1yvo79m9A
	aOMobw4cgSZviNxgpnuD2Xm9i6pD1dFa6qK98+4SvMzqQktUqFFoWCBh3wKVCHxCq6oI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186135-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 186135: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-4.17-testing:test-amd64-coresched-amd64-xl:guest-start/debian.repeat:fail:heisenbug
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
X-Osstest-Versions-That:
    xen=effcf70f020ff12d34c80e2abde0ecb00ce92bda
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 03:40:31 +0000

flight 186135 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186135/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail in 186129 pass in 186135
 test-amd64-coresched-amd64-xl 22 guest-start/debian.repeat fail pass in 186129

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 185864
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 185864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 185864
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44
baseline version:
 xen                  effcf70f020ff12d34c80e2abde0ecb00ce92bda

Last test of basis   185864  2024-04-29 08:08:55 Z   25 days
Failing since        186063  2024-05-21 10:06:36 Z    3 days    6 attempts
Testing same since   186069  2024-05-22 01:58:18 Z    3 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Juergen Gross <jgross@suse.com>
  Leigh Brown <leigh@solinno.co.uk>
  Roger Pau Monné <roger.pau@citrix.com>
  Ross Lagerwall <ross.lagerwall@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   effcf70f02..3c7c9225ff  3c7c9225ffa5605bf0603f9dd1666f3f786e2c44 -> stable-4.17


From xen-devel-bounces@lists.xenproject.org Sat May 25 09:25:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 09:25:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730173.1135539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAndv-0002Sz-0j; Sat, 25 May 2024 09:24:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730173.1135539; Sat, 25 May 2024 09:24:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAndu-0002Ss-UI; Sat, 25 May 2024 09:24:46 +0000
Received: by outflank-mailman (input) for mailman id 730173;
 Sat, 25 May 2024 09:24:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAndt-0002Si-N1; Sat, 25 May 2024 09:24:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAndt-0005LJ-JF; Sat, 25 May 2024 09:24:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAndt-000167-5S; Sat, 25 May 2024 09:24:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAndt-0004Xb-50; Sat, 25 May 2024 09:24:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=vwuirJ7rYO6Jcx5dADWfPf2KGwdWJuyFFGRs98GRRWI=; b=qZ31koNB81kyEyxMBsxXMRDNBH
	Mln912DnGgXxuT9hIjnDa3KopOYdpJ5v4SdZc3JJwsyWdSPQtTo1vIrUk7hkug7+JcehYYg53lqOn
	HUfTw3U/r2cHqapUGoELgqt8JJCEoZnGTvnWQ/BmdKL6CmB6kk+vOfY4D7TkC+mptXDM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186140-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186140: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=02c438bbfffeabf8c958108f9cf88cdb1a11a323
X-Osstest-Versions-That:
    linux=6d69b6c12fce479fde7bc06f686212451688a102
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 09:24:45 +0000

flight 186140 linux-linus real [real]
flight 186143 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186140/
http://logs.test-lab.xenproject.org/osstest/logs/186143/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-raw       8 xen-boot                 fail REGR. vs. 186127

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 186143-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 186143-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 186143 like 186127
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 186143 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186143 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186143 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186127
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                02c438bbfffeabf8c958108f9cf88cdb1a11a323
baseline version:
 linux                6d69b6c12fce479fde7bc06f686212451688a102

Last test of basis   186127  2024-05-23 23:10:22 Z    1 days
Testing same since   186140  2024-05-24 17:10:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhinav Saxena <xandfury@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Andy Chi <andy.chi@canonical.com>
  Arnd Bergmann <arnd@arndb.de>
  Boris Burkov <boris@bur.io>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Starke <daniel.starke@siemens.com>
  David Sterba <dsterba@suse.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Erick Archer <erick.archer@outlook.com>
  Fenglin Wu <quic_fenglinw@quicinc.com>
  Filipe Manana <fdmanana@suse.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Gao Xiang <xiang@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Hongzhen Luo <hongzhen@linux.alibaba.com>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  j51569436@gmail.com
  Jack Yu <jack.yu@realtek.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jian-Hong Pan <jhp@endlessos.org>
  Joel Selvaraj <joelselvaraj.oss@gmail.com>
  Karel Balej <balejk@matfyz.cz>
  Kees Cook <keescook@chromium.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Kirill Artemev <artewar6767@gmail.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lu Yao <yaolu@kylinos.cn>
  Manuel Barrio Linares <mbarriolinares@gmail.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Brown <broonie@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
  Pin-yen Lin <treapking@chromium.org>
  Qu Wenruo <wqu@suse.com>
  Ricardo Ribalda <ribalda@chromium.org>
  Rob Herring (Arm) <robh@kernel.org>
  Shenghao Ding <shenghao-ding@ti.com>
  Stephen Boyd <swboyd@chromium.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Takashi Iwai <tiwai@suse.de>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Xu Yang <xu.yang_2@nxp.com>
  Youling Tang <tangyouling@kylinos.cn>
  Zhang Yi <zhangyi@everest-semi.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2000 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat May 25 11:36:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 11:36:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730215.1135563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAphb-000817-7G; Sat, 25 May 2024 11:36:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730215.1135563; Sat, 25 May 2024 11:36:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAphb-000810-2Q; Sat, 25 May 2024 11:36:43 +0000
Received: by outflank-mailman (input) for mailman id 730215;
 Sat, 25 May 2024 11:36:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAphZ-00080q-9X; Sat, 25 May 2024 11:36:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAphZ-0007kr-7k; Sat, 25 May 2024 11:36:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAphY-0006Pd-RR; Sat, 25 May 2024 11:36:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAphY-0002Ib-Qw; Sat, 25 May 2024 11:36:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rQ50Emrv5SvSFAHKhNe57USF1vuoOseD3eJEQ2XNG5o=; b=Dhog2FxHeRkXqFH1cG1R52FomD
	CgSZo1esTb84JXG2Om/6rwl9AL0Y6vjK4QdOeD/xeYLuf2IBDlfpJg01TTtX/ki8+Dt1cO1P3mQ+r
	oup1wOHFQ63YWN9sEupU78da1nL0Hc+6aKfxkFSYOXaOvWP3NHF2esWcOh79dnPF2P3g=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186141-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186141: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
X-Osstest-Versions-That:
    xen=2a40b106e92aaa7ce808c8608dd6473edc67f608
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 11:36:40 +0000

flight 186141 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186141/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186132
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186132
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186132
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186132
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186132
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186132
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
baseline version:
 xen                  2a40b106e92aaa7ce808c8608dd6473edc67f608

Last test of basis   186132  2024-05-24 04:13:21 Z    1 days
Testing same since   186141  2024-05-24 20:10:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Henry Wang <Henry.Wang@arm.com>
  Henry Wang <xin.wang2@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2a40b106e9..2172a01c4c  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5 -> master


From xen-devel-bounces@lists.xenproject.org Sat May 25 18:01:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 18:01:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730291.1135572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAvhq-0006JB-L1; Sat, 25 May 2024 18:01:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730291.1135572; Sat, 25 May 2024 18:01:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAvhq-0006J3-GN; Sat, 25 May 2024 18:01:22 +0000
Received: by outflank-mailman (input) for mailman id 730291;
 Sat, 25 May 2024 18:01:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAvhp-0006It-N0; Sat, 25 May 2024 18:01:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAvhp-0006WQ-F2; Sat, 25 May 2024 18:01:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAvhp-0007lX-3d; Sat, 25 May 2024 18:01:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAvhp-0000kf-3C; Sat, 25 May 2024 18:01:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1mUq35lxqh2LmLfKZzxggxYc0sJg2FeOiNsmWSUOIVk=; b=LNem0L07WUYkEJJjkLNMp6dCjo
	MmR2mFMcHAHPmC7fc4rSBNAkIdJrXG7FFjNTNTEY+llme/mop5Cl6l+iOw88HLWbmzlVeZU1C3WUu
	fujp+sB/EbcN7KKJ6hBKMIgz02tC9eMsosE+EIcRiWj4dcxW84SuTXwaoeqWsQcnErUg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186144-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186144: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-xl-qcow2:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
X-Osstest-Versions-That:
    linux=6d69b6c12fce479fde7bc06f686212451688a102
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 18:01:21 +0000

flight 186144 linux-linus real [real]
flight 186147 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186144/
http://logs.test-lab.xenproject.org/osstest/logs/186147/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   8 xen-boot                 fail REGR. vs. 186127

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-pvhv2-intel 20 guest-localmigrate/x10 fail pass in 186147-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 186127

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-qcow2     8 xen-boot                     fail  like 186127
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186127
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186127
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
baseline version:
 linux                6d69b6c12fce479fde7bc06f686212451688a102

Last test of basis   186127  2024-05-23 23:10:22 Z    1 days
Failing since        186140  2024-05-24 17:10:45 Z    1 days    2 attempts
Testing same since   186144  2024-05-25 09:27:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhinav Saxena <xandfury@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alex Deucher <alexander.deucher@amd.com>
  Alexandre Ghiti <alexghiti@rivosinc.com>
  Amer Al Shanawany <amer.shanawany@gmail.com>
  Anatoliy Klymenko <anatoliy.klymenko@amd.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Chi <andy.chi@canonical.com>
  Andy Chiu <andy.chiu@sifive.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Antonino Maniscalco <antonino.maniscalco@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Björn Töpel <bjorn@rivosinc.com>
  Boris Brezillon <boris.brezillon@collabora.com>
  Boris Burkov <boris@bur.io>
  Charlie Jenkins <charlie@rivosinc.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Starke <daniel.starke@siemens.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  David Sterba <dsterba@suse.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Emil Renner Berthing <emil.renner.berthing@canonical.com>
  Eric Smith <eric.smith@collabora.com>
  Erick Archer <erick.archer@outlook.com>
  Fenglin Wu <quic_fenglinw@quicinc.com>
  Filipe Manana <fdmanana@suse.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Gao Xiang <xiang@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Henry Wang <xin.wang2@amd.com>
  Hongzhen Luo <hongzhen@linux.alibaba.com>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  j51569436@gmail.com
  Jack Yu <jack.yu@realtek.com>
  Jani Nikula <jani.nikula@intel.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jeff Xu <jeffxu@chromium.org>
  Jian-Hong Pan <jhp@endlessos.org>
  Joel Selvaraj <joelselvaraj.oss@gmail.com>
  Juergen Gross <jgross@suse.com>
  Karel Balej <balejk@matfyz.cz>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Kirill Artemev <artewar6767@gmail.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lang Yu <Lang.Yu@amd.com>
  Li Ma <li.ma@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Liu Ying <victor.liu@nxp.com>
  Liviu Dudau <liviu.dudau@arm.com>
  Lu Yao <yaolu@kylinos.cn>
  Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
  Manuel Barrio Linares <mbarriolinares@gmail.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Brown <broonie@kernel.org>
  Matthew Bystrin <dev.mbstr@gmail.com>
  Michal Orzel <michal.orzel@amd.com>
  Mohamed Ahmed <mohamedahmedegypt2001@gmail.com>
  Nam Cao <namcao@linutronix.de>
  Nathan Chancellor <nathan@kernel.org>
  Palmer Dabbelt <palmer@rivosinc.com>
  Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
  Pin-yen Lin <treapking@chromium.org>
  Puranjay Mohan <puranjay@kernel.org>
  Qingfang Deng <qingfang.deng@siflower.com.cn>
  Qu Wenruo <wqu@suse.com>
  Ricardo Ribalda <ribalda@chromium.org>
  Rob Herring (Arm) <robh@kernel.org>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Samuel Holland <samuel.holland@sifive.com>
  Shenghao Ding <shenghao-ding@ti.com>
  Stephen Boyd <swboyd@chromium.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Sui Jingfeng <suijingfeng@bosc.ac.cn>
  Takashi Iwai <tiwai@suse.de>
  Thomas Zimmermann <tzimmermann@suse.de>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Uros Bizjak <ubizjak@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wayne Lin <Wayne.Lin@amd.com>
  Xiao Wang <xiao.w.wang@intel.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Yang <xu.yang_2@nxp.com>
  Youling Tang <tangyouling@kylinos.cn>
  Zhang Yi <zhangyi@everest-semi.com>
  Zhao Ke <ke.zhao@shingroup.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3823 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat May 25 21:33:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 May 2024 21:33:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730337.1135581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAz0O-0002hU-50; Sat, 25 May 2024 21:32:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730337.1135581; Sat, 25 May 2024 21:32:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sAz0O-0002hN-2E; Sat, 25 May 2024 21:32:44 +0000
Received: by outflank-mailman (input) for mailman id 730337;
 Sat, 25 May 2024 21:32:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAz0N-0002hD-AN; Sat, 25 May 2024 21:32:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAz0N-0001Yj-5y; Sat, 25 May 2024 21:32:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sAz0M-0007UR-QR; Sat, 25 May 2024 21:32:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sAz0M-0004I0-Pz; Sat, 25 May 2024 21:32:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Uv0JKnjVgO/XAyLEI+5ZNj2nMUzd+PAg36r6+atFjpc=; b=1EGsLmVDaR/g8q24Pk05HWMm7h
	O6rS7SGAmnojyX8u3ap9sUrAym+zLLNTZAhlvy/MUw6cSxN/k2gbHM84Nazz2ShdxWy1Gcnkf0JVA
	jTzfLtUWxrmSCwE0qUdJXafvmbSNfAP+A7ohWJGM+dedGU7wTvmn58WKFwoGT4InAFH8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186145-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186145: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-raw:debian-di-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
X-Osstest-Versions-That:
    xen=2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 May 2024 21:32:42 +0000

flight 186145 xen-unstable real [real]
flight 186149 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186145/
http://logs.test-lab.xenproject.org/osstest/logs/186149/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 186149-retest
 test-armhf-armhf-xl-raw      12 debian-di-install   fail pass in 186149-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 186149 like 186141
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 186149 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 186149 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 186149 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186141
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186141
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186141
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186141
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186141
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0
baseline version:
 xen                  2172a01c4cecbaa1d79bad200bfe3b996a3e4ba5

Last test of basis   186141  2024-05-24 20:10:36 Z    1 days
Testing same since   186145  2024-05-25 11:41:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Henry Wang <xin.wang2@amd.com>
  Julien Grall <jgrall@amazon.com>
  Stefano Stabellini <stefano.stabellini@amd.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Vikram Garhwal <fnu.vikram@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2172a01c4c..ac572152e5  ac572152e578a8853de0534384c1539ec21f11e0 -> master


From xen-devel-bounces@lists.xenproject.org Sun May 26 01:15:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 01:15:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730388.1135620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sB2Tc-0008Fq-14; Sun, 26 May 2024 01:15:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730388.1135620; Sun, 26 May 2024 01:15:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sB2Tb-0008Fj-UB; Sun, 26 May 2024 01:15:07 +0000
Received: by outflank-mailman (input) for mailman id 730388;
 Sun, 26 May 2024 01:15:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB2Ta-0008FZ-Kv; Sun, 26 May 2024 01:15:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB2Ta-0004xS-Gk; Sun, 26 May 2024 01:15:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB2Ta-0005iQ-2P; Sun, 26 May 2024 01:15:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sB2Ta-0007RD-1x; Sun, 26 May 2024 01:15:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=X1SPTlD5YlMqKFS98RH+9IFWMqAlS7hMdBsDO/s6FKc=; b=fE711h+shgnkrEjnXzeuwv/3zP
	WmuOADc9zh2JdHb3TiwrKXvMZlRlMLT8jHRkJG0sE/KvRVXyYbLQgeXj46qMQfLwjjMnJLt+mtVuS
	V0+waHftCLRhic0VgtCa/htS1dCiE6tSYhBoMQLY5BSlZLbkRHEc09HvW23qPq8LYyt4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186146-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 186146: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    linux-6.1:build-armhf:<job status>:broken:regression
    linux-6.1:build-armhf:host-build-prep:fail:regression
    linux-6.1:build-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=88690811da69826fdb59d908a6e5e9d0c63b581a
X-Osstest-Versions-That:
    linux=4078fa637fcd80c8487680ec2e4ef7c58308e9aa
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 May 2024 01:15:06 +0000

flight 186146 linux-6.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186146/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                     <job status>                 broken
 build-armhf                   5 host-build-prep          fail REGR. vs. 186028

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186028
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                88690811da69826fdb59d908a6e5e9d0c63b581a
baseline version:
 linux                4078fa637fcd80c8487680ec2e4ef7c58308e9aa

Last test of basis   186028  2024-05-17 10:16:47 Z    8 days
Testing same since   186146  2024-05-25 14:44:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aidan MacDonald <aidanmacdonald.0x0@gmail.com>
  Ajay Kaher <ajay.kaher@broadcom.com>
  Akira Yokosawa <akiyks@gmail.com>
  Alex Deucher <alexander.deucher@amd.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Carlos Llamas <cmllamas@google.com>
  Chandan Babu R <chandanbabu@kernel.org>
  Chuck Lever <chuck.lever@oracle.com>
  Daniel Thompson <daniel.thompson@linaro.org>
  Darrick J. Wong <djwong@kernel.org>
  Dave Chinner <dchinner@redhat.com>
  David S. Miller <davem@davemloft.net>
  Eric Sandeen <sandeen@redhat.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guo Xuenan <guoxuenan@huawei.com>
  Heikki Krogerus <heikki.krogerus@linux.intel.com>
  Hironori Shiina <shiina.hironori@fujitsu.com>
  Hironori Shiina <shiina.hironori@gmail.com>
  Jacob Keller <jacob.e.keller@intel.com>
  Jakub Kicinski <kuba@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org>
  Javier Carrasco <javier.carrasco@wolfvision.net>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Corbet <corbet@lwn.net>
  Jose Fernandez <josef@netflix.com>
  Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
  Kelsey Steele <kelseysteele@linux.microsoft.com>
  Leah Rumancik <leah.rumancik@gmail.com>
  Lee Jones <lee@kernel.org>
  Linus Walleij <linus.walleij@linaro.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Long Li <leo.lilong@huawei.com>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Mathieu Poirier <mathieu.poirier@linaro.org>
  Mengqi Zhang <mengqi.zhang@mediatek.com>
  NeilBrown <neilb@suse.de>
  Oleksandr Tymoshenko <ovt@google.com>
  Pavel Machek (CIP) <pavel@denx.de>
  Prashanth K <quic_prashk@quicinc.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
  Ron Economos <re@w6rz.net>
  Ronald Wahl <ronald.wahl@raritan.com>
  Salvatore Bonaccorso <carnil@debian.org>
  SeongJae Park <sj@kernel.org>
  Sergey Shtylyov <s.shtylyov@omp.ru>
  Shuah Khan <skhan@linuxfoundation.org>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Will Deacon <will@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  broken  
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-armhf broken

Not pushing.

(No revision log; it would be 1940 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun May 26 05:10:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 05:10:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730425.1135630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sB69L-0008OJ-Jp; Sun, 26 May 2024 05:10:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730425.1135630; Sun, 26 May 2024 05:10:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sB69L-0008OC-Gf; Sun, 26 May 2024 05:10:27 +0000
Received: by outflank-mailman (input) for mailman id 730425;
 Sun, 26 May 2024 05:10:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB69J-0008O2-VM; Sun, 26 May 2024 05:10:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB69J-0001Ki-Ph; Sun, 26 May 2024 05:10:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sB69J-0007Qj-J9; Sun, 26 May 2024 05:10:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sB697-0000UF-LF; Sun, 26 May 2024 05:10:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zc84FE41caoTN8lGl9LHwxKKH/Fxhon4mAg4B11D2aU=; b=iLLKL7MPOkxK/9ICOyDvBVzvIa
	rLfkPyp6Ie8C2s2qlyyxZp02Ehg0QkQjlRS7u4+kJ7QdB6dK55CBgCzw+ODcsjigPNinKW2XBLrC4
	EIHNXsMIatuQvTiZWhbXsdMpzgG4S8BCugYljeG6IHJgPyTJftpXcGFsYPQ3ZQfkQJaY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186148-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186148: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
X-Osstest-Versions-That:
    linux=6d69b6c12fce479fde7bc06f686212451688a102
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 May 2024 05:10:13 +0000

flight 186148 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186148/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186127
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186127
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186127
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
baseline version:
 linux                6d69b6c12fce479fde7bc06f686212451688a102

Last test of basis   186127  2024-05-23 23:10:22 Z    2 days
Failing since        186140  2024-05-24 17:10:45 Z    1 days    3 attempts
Testing same since   186144  2024-05-25 09:27:52 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhinav Saxena <xandfury@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alex Deucher <alexander.deucher@amd.com>
  Alexandre Ghiti <alexghiti@rivosinc.com>
  Amer Al Shanawany <amer.shanawany@gmail.com>
  Anatoliy Klymenko <anatoliy.klymenko@amd.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Chi <andy.chi@canonical.com>
  Andy Chiu <andy.chiu@sifive.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Antonino Maniscalco <antonino.maniscalco@collabora.com>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Björn Töpel <bjorn@rivosinc.com>
  Boris Brezillon <boris.brezillon@collabora.com>
  Boris Burkov <boris@bur.io>
  Charlie Jenkins <charlie@rivosinc.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Starke <daniel.starke@siemens.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  David Sterba <dsterba@suse.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Emil Renner Berthing <emil.renner.berthing@canonical.com>
  Eric Smith <eric.smith@collabora.com>
  Erick Archer <erick.archer@outlook.com>
  Fenglin Wu <quic_fenglinw@quicinc.com>
  Filipe Manana <fdmanana@suse.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Gao Xiang <xiang@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Henry Wang <xin.wang2@amd.com>
  Hongzhen Luo <hongzhen@linux.alibaba.com>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  j51569436@gmail.com
  Jack Yu <jack.yu@realtek.com>
  Jani Nikula <jani.nikula@intel.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jeff Xu <jeffxu@chromium.org>
  Jian-Hong Pan <jhp@endlessos.org>
  Joel Selvaraj <joelselvaraj.oss@gmail.com>
  Juergen Gross <jgross@suse.com>
  Karel Balej <balejk@matfyz.cz>
  Kees Cook <keescook@chromium.org>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Kirill Artemev <artewar6767@gmail.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lang Yu <Lang.Yu@amd.com>
  Li Ma <li.ma@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Liu Ying <victor.liu@nxp.com>
  Liviu Dudau <liviu.dudau@arm.com>
  Lu Yao <yaolu@kylinos.cn>
  Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
  Manuel Barrio Linares <mbarriolinares@gmail.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Brown <broonie@kernel.org>
  Matthew Bystrin <dev.mbstr@gmail.com>
  Michal Orzel <michal.orzel@amd.com>
  Mohamed Ahmed <mohamedahmedegypt2001@gmail.com>
  Nam Cao <namcao@linutronix.de>
  Nathan Chancellor <nathan@kernel.org>
  Palmer Dabbelt <palmer@rivosinc.com>
  Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
  Pin-yen Lin <treapking@chromium.org>
  Puranjay Mohan <puranjay@kernel.org>
  Qingfang Deng <qingfang.deng@siflower.com.cn>
  Qu Wenruo <wqu@suse.com>
  Ricardo Ribalda <ribalda@chromium.org>
  Rob Herring (Arm) <robh@kernel.org>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Samuel Holland <samuel.holland@sifive.com>
  Shenghao Ding <shenghao-ding@ti.com>
  Stephen Boyd <swboyd@chromium.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Sui Jingfeng <suijingfeng@bosc.ac.cn>
  Takashi Iwai <tiwai@suse.de>
  Thomas Zimmermann <tzimmermann@suse.de>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Uros Bizjak <ubizjak@gmail.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wayne Lin <Wayne.Lin@amd.com>
  Xiao Wang <xiao.w.wang@intel.com>
  Xingyou Chen <rockrush@rockwork.org>
  Xu Yang <xu.yang_2@nxp.com>
  Youling Tang <tangyouling@kylinos.cn>
  Zhang Yi <zhangyi@everest-semi.com>
  Zhao Ke <ke.zhao@shingroup.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6d69b6c12fce..56fb6f92854f  56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun May 26 11:14:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 11:14:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730487.1135640 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBBp4-0003jO-Sl; Sun, 26 May 2024 11:13:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730487.1135640; Sun, 26 May 2024 11:13:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBBp4-0003jG-O4; Sun, 26 May 2024 11:13:54 +0000
Received: by outflank-mailman (input) for mailman id 730487;
 Sun, 26 May 2024 11:13:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBBp3-0003j6-Ks; Sun, 26 May 2024 11:13:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBBp3-0007sL-Ha; Sun, 26 May 2024 11:13:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBBp3-0006Ag-7M; Sun, 26 May 2024 11:13:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBBp3-0000Km-6v; Sun, 26 May 2024 11:13:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rwYdhfq11Du1rknXIKdO3U6R6whrKYsap0ykqDvGo1g=; b=SO+kK/0BCEd35Oq9UKkMaeTUVZ
	2BFfMso7tM7jbIefqIhKG1xs7aTSPojvyHg18EttAY9zkpd+Qxk0lr6+INd7fdqt4O4lqcrvm/3vu
	ySYEw0Pzf61cvttns2jq5zle+omCcKFxuuLizFQAL1H83hP3uPknD83EHdLk3pVWGyPs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186150-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 186150: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=88690811da69826fdb59d908a6e5e9d0c63b581a
X-Osstest-Versions-That:
    linux=4078fa637fcd80c8487680ec2e4ef7c58308e9aa
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 May 2024 11:13:53 +0000

flight 186150 linux-6.1 real [real]
flight 186153 linux-6.1 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186150/
http://logs.test-lab.xenproject.org/osstest/logs/186153/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-examine      8 reboot              fail pass in 186153-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186028
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186028
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186028
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                88690811da69826fdb59d908a6e5e9d0c63b581a
baseline version:
 linux                4078fa637fcd80c8487680ec2e4ef7c58308e9aa

Last test of basis   186028  2024-05-17 10:16:47 Z    9 days
Testing same since   186146  2024-05-25 14:44:52 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aidan MacDonald <aidanmacdonald.0x0@gmail.com>
  Ajay Kaher <ajay.kaher@broadcom.com>
  Akira Yokosawa <akiyks@gmail.com>
  Alex Deucher <alexander.deucher@amd.com>
  AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
  Carlos Llamas <cmllamas@google.com>
  Chandan Babu R <chandanbabu@kernel.org>
  Chuck Lever <chuck.lever@oracle.com>
  Daniel Thompson <daniel.thompson@linaro.org>
  Darrick J. Wong <djwong@kernel.org>
  Dave Chinner <dchinner@redhat.com>
  David S. Miller <davem@davemloft.net>
  Eric Sandeen <sandeen@redhat.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guo Xuenan <guoxuenan@huawei.com>
  Heikki Krogerus <heikki.krogerus@linux.intel.com>
  Hironori Shiina <shiina.hironori@fujitsu.com>
  Hironori Shiina <shiina.hironori@gmail.com>
  Jacob Keller <jacob.e.keller@intel.com>
  Jakub Kicinski <kuba@kernel.org>
  Jarkko Sakkinen <jarkko@kernel.org>
  Javier Carrasco <javier.carrasco@wolfvision.net>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Corbet <corbet@lwn.net>
  Jose Fernandez <josef@netflix.com>
  Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
  Kelsey Steele <kelseysteele@linux.microsoft.com>
  Leah Rumancik <leah.rumancik@gmail.com>
  Lee Jones <lee@kernel.org>
  Linus Walleij <linus.walleij@linaro.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Long Li <leo.lilong@huawei.com>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Mathieu Poirier <mathieu.poirier@linaro.org>
  Mengqi Zhang <mengqi.zhang@mediatek.com>
  NeilBrown <neilb@suse.de>
  Oleksandr Tymoshenko <ovt@google.com>
  Pavel Machek (CIP) <pavel@denx.de>
  Prashanth K <quic_prashk@quicinc.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
  Ron Economos <re@w6rz.net>
  Ronald Wahl <ronald.wahl@raritan.com>
  Salvatore Bonaccorso <carnil@debian.org>
  SeongJae Park <sj@kernel.org>
  Sergey Shtylyov <s.shtylyov@omp.ru>
  Shuah Khan <skhan@linuxfoundation.org>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Weißschuh <linux@weissschuh.net>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Will Deacon <will@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4078fa637fcd..88690811da69  88690811da69826fdb59d908a6e5e9d0c63b581a -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Sun May 26 14:26:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 14:26:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730520.1135653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBEoT-0006cp-1Y; Sun, 26 May 2024 14:25:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730520.1135653; Sun, 26 May 2024 14:25:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBEoS-0006ci-Uw; Sun, 26 May 2024 14:25:28 +0000
Received: by outflank-mailman (input) for mailman id 730520;
 Sun, 26 May 2024 14:25:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBEoS-0006cY-Ky; Sun, 26 May 2024 14:25:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBEoS-0002bW-IK; Sun, 26 May 2024 14:25:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBEoS-00029A-4X; Sun, 26 May 2024 14:25:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBEoS-00022D-45; Sun, 26 May 2024 14:25:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3MoGKVhRjISu3jmEGuIlxgAZbi10kZgZK6B/U2OrhQk=; b=elEoEPuxGJ6HdWRrx3fxkJdubq
	mJNbP+A0eAPjyS9CBSK8gLWoczmnMVp4SQAPhgYDUP4QfBZULcXMztjwXeEK26CL9WN1yHYvS0xw6
	org/QmioBHfNeiPlej25qnad7TSRdOf4zghRLqyzyiGazxaWQcNQTkcxe94xHjxcJZ+A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186151-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186151: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-armhf:xen-build:fail:regression
    xen-unstable:build-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
X-Osstest-Versions-That:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 May 2024 14:25:28 +0000

flight 186151 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186151/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186145

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186145
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186145
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186145
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186145
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186145
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0
baseline version:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0

Last test of basis   186151  2024-05-26 01:51:58 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun May 26 19:05:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 19:05:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730542.1135665 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBJBN-0002ev-3Q; Sun, 26 May 2024 19:05:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730542.1135665; Sun, 26 May 2024 19:05:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBJBN-0002eo-0p; Sun, 26 May 2024 19:05:25 +0000
Received: by outflank-mailman (input) for mailman id 730542;
 Sun, 26 May 2024 19:05:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBJBL-0002eb-LP; Sun, 26 May 2024 19:05:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBJBL-0007hj-It; Sun, 26 May 2024 19:05:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBJBL-0008UV-7G; Sun, 26 May 2024 19:05:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBJBL-0004kr-6n; Sun, 26 May 2024 19:05:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=X957tiZ9MRaT+tXQx3Ram7PLEC4vDi50TZebOw9A0q0=; b=aQh2mpXttVHLFZsngAlBnQL5iN
	wi5rPcf3oN23X1YVULWft04LTzRZ0tzY/OP5bL8k/NJayECRJwjkw/tc6NspTCTa6Tqz2ngXG0J3V
	Z/HTQY3d/BTB9NeZP5E+56P2g52m28UspnZQi9jrXMTZHyohRBySmcqE8rUF2PHYCGSE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186152-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186152: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-libvirt-vhd:guest-start/debian.repeat:fail:regression
    linux-linus:build-armhf:xen-build:fail:regression
    linux-linus:build-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=9b62e02e63363f5678d5598ee7372064301587f7
X-Osstest-Versions-That:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 May 2024 19:05:23 +0000

flight 186152 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186152/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-vhd 19 guest-start/debian.repeat fail REGR. vs. 186148
 build-armhf                   6 xen-build                fail REGR. vs. 186148

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186148
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                9b62e02e63363f5678d5598ee7372064301587f7
baseline version:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c

Last test of basis   186148  2024-05-25 18:11:58 Z    1 days
Testing same since   186152  2024-05-26 05:15:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Allen Pais <allen.lkml@gmail.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Konovalov <andreyknvl@gmail.com>
  Arnd Bergmann <arnd@kernel.org>
  Artem Ikonnikov <artem@datacrunch.io>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Carsten Tolkmit <ctolkmit@ennit.de>
  Chengming Zhou <chengming.zhou@linux.dev>
  Christian Heusel <christian@heusel.eu>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Corey Minyard <cminyard@mvista.com>
  Corey Minyard <minyard@acm.org>
  David Hildenbrand <david@redhat.com>
  Dev Jain <dev.jain@arm.com>
  dicken.ding <dicken.ding@mediatek.com>
  Dongli Zhang <dongli.zhang@oracle.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Frank Li <Frank.Li@nxp.com>
  Guenter Roeck <linux@roeck-us.net>
  Hailong.Liu <hailong.liu@oppo.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Ilya Dryomov <idryomov@gmail.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Klara Modin <klarasmodin@gmail.com>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kunwu Chan <chentao@kylinos.cn>
  Lenko Donchev <lenko.donchev@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Mia Lin <mimi05633@gmail.com>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Hocko <mhocko@suse.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nandor Kracser <bonifaido@gmail.com>
  Nico Pache <npache@redhat.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Palmer Dabbelt <palmer@rivosinc.com>
  Randy Dunlap <rdunlap@infradead.org>
  Richard Weinberger <richard@nod.at>
  Roberto Sassu <roberto.sassu@huawei.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Satya Priya Kakitapalli <quic_skakitap@quicinc.com>
  Steve French <stfrench@microsoft.com>
  Suren Baghdasaryan <surenb@google.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tiwei Bie <tiwei.btw@antgroup.com>
  Tony Luck <tony.luck@intel.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wang Yong <wang.yong12@zte.com.cn>
  Waqar Hameed <waqar.hameed@axis.com>
  Will Deacon <will@kernel.org>
  Xiubo Li <xiubli@redhat.com>
  Yuanyuan Zhong <yzhong@purestorage.com>
  Yueh-Shun Li <shamrocklee@posteo.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 fail    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2816 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun May 26 23:55:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 May 2024 23:55:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730564.1135675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBNi2-0007Y2-2S; Sun, 26 May 2024 23:55:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730564.1135675; Sun, 26 May 2024 23:55:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBNi1-0007Xv-Vz; Sun, 26 May 2024 23:55:25 +0000
Received: by outflank-mailman (input) for mailman id 730564;
 Sun, 26 May 2024 23:55:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=QFAz=M5=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1sBNi0-0007Xp-OX
 for xen-devel@lists.xenproject.org; Sun, 26 May 2024 23:55:24 +0000
Received: from mail-oo1-xc31.google.com (mail-oo1-xc31.google.com
 [2607:f8b0:4864:20::c31])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 69800bfb-1bbb-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 01:55:22 +0200 (CEST)
Received: by mail-oo1-xc31.google.com with SMTP id
 006d021491bc7-5b5254f9c32so4005375eaf.0
 for <xen-devel@lists.xenproject.org>; Sun, 26 May 2024 16:55:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69800bfb-1bbb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716767721; x=1717372521; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1FtBCXJECgH5jG4zdwJ6hIO5ONat+ArbMrNEVsu+qeM=;
        b=BgBq+tk5iAJWriLYxPZBgk+AUYkP/qgK3gceXaYly8saUIZT2A0IHvp9OyAcD/I87Z
         KBNJ8rcBh8g7U/Gs4JWMSgpuRcfcw4verh+S/qDR3+nAz/Hl8sS49DYzWzYXKxnN0SG8
         4OM1ccHVp6N0ne7+4OQketZmjErTwRovuBUhB7Or3ktGmJOxHs0gsNrgkI8bdCiNaRaI
         /PJrkVX/LO/fGiRFygEMIYxwPtOw78xcyMdGnGkofMC9L6/kO2iOu2WE3sSHqTv+ni06
         mi94wyzwa5N7D2jWpvZjZBs9B6v68StugUAiFCGXF0qXRhsuMsS5Pjz6BR4HpWIHEVsB
         fmBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716767721; x=1717372521;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1FtBCXJECgH5jG4zdwJ6hIO5ONat+ArbMrNEVsu+qeM=;
        b=Qj2vmjuwnqZJzXwdfDTmgWAwus+KWaFTW/k4+1Tn4m7rxuKNDmqUxxk1XDgu5qBBYx
         BPF6UXnUKHcyAiLBaYrO3FF5JpxSwqv2F+JpgmhIQuS+90yigK+3fn6fzsxm/K+lhIm3
         SftQHEGpuo7jBWP1k00y0aZ6J4a+KQdjfHHzmoOEP2FvxuCiFy+Sl246JcBgRs+G3/V5
         jL4v5R8hkyTlyevpgFEWpUknuse51bwheYX8QZRdCG2K6Q6M77IdQyyXjHRSfFb5Humi
         cqJI87fXMYS//DFI1Rp+BRXIUiERjznReZQVfvS9ToDgD6yUJmrvnfQ0tCrW3NtL3Des
         1f1Q==
X-Forwarded-Encrypted: i=1; AJvYcCV/TIZJqSC53iWsEgwaCOshlteXkKHxIYtdrs9rQ6QVUrhRZu9DBC2ZVPtWa/07AcQBkT4MNjdA65aanu3EQ0IxBJKYK7ZoyqeCgss4aqE=
X-Gm-Message-State: AOJu0Yx3DryPRJBCDh5X/CU+oIE1Nvct9DhfycZOkdudyYgJKCeM9xiD
	6i9P5GyKF7dEyVhmu+HmxhbExSFyLJl4bbgaNptIrtmiKIAcpBcgrjpv9HNr/VvL5/jBehqfmds
	tCXeXGYE8V3tVHm3qLsWzN0YaQLc=
X-Google-Smtp-Source: AGHT+IHqFmBtelvkkarGAKz5L+Vf5A9nRqAWmELa9QVrAg6u0FlXGqFvMFQ1uOF3djmvv90IY39NQ8sQGvYbIqNjDhE=
X-Received: by 2002:a05:6870:218c:b0:24f:bd2e:9c3f with SMTP id
 586e51a60fabf-24fbd2fea0dmr6752334fac.11.1716767720815; Sun, 26 May 2024
 16:55:20 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1716029860.git.w1benny@gmail.com> <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
 <21196eb3-82ca-470b-8dc9-1a95b427f3b9@suse.com>
In-Reply-To: <21196eb3-82ca-470b-8dc9-1a95b427f3b9@suse.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Mon, 27 May 2024 01:55:09 +0200
Message-ID: <CAKBKdXj7xUyuy-wPCjR0a+kNwqCmGXOoO=-pMQa7=Yiyaom_mg@mail.gmail.com>
Subject: Re: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m
 views configurable
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Tamas K Lengyel <tamas@tklengyel.com>, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 21, 2024 at 12:59=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> The compared entities don't really fit together. I think we want a new
> MAX_NR_ALTP2M, which - for the time being - could simply be
>
> #define MAX_NR_ALTP2M MAX_EPTP
>
> in the header. That would then be a suitable replacement for the
> min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) that you're adjusting
> elsewhere. Which however raises the question whether in EPT-specific
> code the min() wouldn't better survive, as min(d->nr_altp2m, MAX_EPTP).
>

As you mentioned in a previous email, I've removed all the min(...,
MAX_EPTP) invocations from the code, since nr_altp2m is validated to
be no greater than that value. The only remaining places where this
value occurs are:

- In my newly introduced condition in arch_sanitise_domain_config:

if ( config->nr_altp2m > MAX_EPTP )
{
    dprintk(XENLOG_INFO, "nr_altp2m must be <=3D %lu\n", MAX_NR_ALTP2M);
    return -EINVAL;
}

- In hap_enable():

for ( i =3D 0; i < MAX_EPTP; i++ )
{
    d->arch.altp2m_eptp[i] =3D mfn_x(INVALID_MFN);
    d->arch.altp2m_visible_eptp[i] =3D mfn_x(INVALID_MFN);
}

Note that altp2m_eptp/altp2m_visible_eptp is never accessed beyond
nr_altp2m. From what you're saying, it sounds to me like I should only
replace the first mentioned occurrence with MAX_NR_ALTP2M. Correct me
if I'm wrong.

> > @@ -5228,7 +5234,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t=
 p2midx)
> >      if ( !hvm_is_singlestep_supported() )
> >          return;
> >
> > -    if ( p2midx >=3D MAX_ALTP2M )
> > +    if ( p2midx >=3D v->domain->nr_altp2m )
> >          return;
>
> You don't introduce a new local variable here. I'd like to ask that you a=
lso
> don't ...
>
> > @@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
> >      /* altp2m view 0 is treated as the hostp2m */
> >      if ( altp2m_idx )
> >      {
> > -        if ( altp2m_idx >=3D min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_E=
PTP) ||
> > -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EP=
TP)] =3D=3D
> > -             mfn_x(INVALID_MFN) )
> > +        if ( altp2m_idx >=3D d->nr_altp2m ||
> > +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_=
altp2m)]
> > +             =3D=3D mfn_x(INVALID_MFN) )
>
> Please don't break previously correct style: Binary operators (here: =3D=
=3D )
> belong onto the end of the earlier line. That'll render the line too long
> again, but you want to deal with that e.g. thus:
>
>              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx,
>                                                     d->nr_altp2m)] =3D=3D
>              mfn_x(INVALID_MFN) )
>

Roger suggested introducing the altp2m_get_p2m() function, which I
like. I think introducing altp2m_get_eptp/visible_eptp and
altp2m_set_eptp/visible_eptp would also elegantly solve the issue of
overly long lines. My question is: if I go this route, should I
strictly replace with these functions only accesses that use
array_index_nospec()? Or should I replace all array accesses? For
example:

for ( i =3D 0; i < d->nr_altp2m; i++ )
{
    struct p2m_domain *p2m;

    if ( d->arch.altp2m_eptp[i] =3D=3D mfn_x(INVALID_MFN) )
        continue;

    p2m =3D d->arch.altp2m_p2m[i];

    p2m_lock(p2m);
    p2m->ept.ad =3D value;
    p2m_unlock(p2m);
}

... should I be consistent and also replace these accesses with
altp2m_get_eptp/altp2m_get_p2m (which will internally use
array_index_nospec), or should I leave them as they are?

P.


From xen-devel-bounces@lists.xenproject.org Mon May 27 01:00:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 01:00:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730571.1135685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBOil-0004Ek-LT; Mon, 27 May 2024 01:00:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730571.1135685; Mon, 27 May 2024 01:00:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBOil-0004Dz-Is; Mon, 27 May 2024 01:00:15 +0000
Received: by outflank-mailman (input) for mailman id 730571;
 Mon, 27 May 2024 01:00:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBOij-0003e1-Oh; Mon, 27 May 2024 01:00:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBOij-0002qe-MO; Mon, 27 May 2024 01:00:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBOij-0003II-Ch; Mon, 27 May 2024 01:00:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBOij-0003hm-CJ; Mon, 27 May 2024 01:00:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qo1meo2cVFi+bh2MluYvsbPoaAAMhy+qidCZ3isOMtQ=; b=GxmGykNcKrujCRHNK486bl/Gd7
	GDHbITuxtycnnLRrqeE93Mhv7mU/4JoOZcLveofzMdieVNJO3RW30ZrRAHXW6yB7Y5vM4yrDmJXaN
	twD8QmTOzo4IuqllX2tfVCBPDsgzy0/QAwvNYmLul61SdN7s5gHLF8ffNiiNmsUtl1aY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186155-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186155: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:build-armhf:xen-build:fail:regression
    linux-linus:build-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6fbf71854e2ddea7c99397772fbbb3783bfe15b5
X-Osstest-Versions-That:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 01:00:13 +0000

flight 186155 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186155/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-credit1 22 guest-start/debian.repeat fail REGR. vs. 186148
 build-armhf                   6 xen-build                fail REGR. vs. 186148

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186148
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6fbf71854e2ddea7c99397772fbbb3783bfe15b5
baseline version:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c

Last test of basis   186148  2024-05-25 18:11:58 Z    1 days
Failing since        186152  2024-05-26 05:15:42 Z    0 days    2 attempts
Testing same since   186155  2024-05-26 19:13:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Allen Pais <allen.lkml@gmail.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Konovalov <andreyknvl@gmail.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@kernel.org>
  Artem Ikonnikov <artem@datacrunch.io>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Carsten Tolkmit <ctolkmit@ennit.de>
  Chengming Zhou <chengming.zhou@linux.dev>
  Christian Heusel <christian@heusel.eu>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Corey Minyard <cminyard@mvista.com>
  Corey Minyard <minyard@acm.org>
  David Hildenbrand <david@redhat.com>
  David Howells <dhowells@redhat.com>
  Dev Jain <dev.jain@arm.com>
  dicken.ding <dicken.ding@mediatek.com>
  Dongli Zhang <dongli.zhang@oracle.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Frank Li <Frank.Li@nxp.com>
  Guenter Roeck <linux@roeck-us.net>
  Hailong.Liu <hailong.liu@oppo.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Ilya Dryomov <idryomov@gmail.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Klara Modin <klarasmodin@gmail.com>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kunwu Chan <chentao@kylinos.cn>
  Lenko Donchev <lenko.donchev@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Mia Lin <mimi05633@gmail.com>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Hocko <mhocko@suse.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nandor Kracser <bonifaido@gmail.com>
  Nico Pache <npache@redhat.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Palmer Dabbelt <palmer@rivosinc.com>
  Randy Dunlap <rdunlap@infradead.org>
  Richard Weinberger <richard@nod.at>
  Roberto Sassu <roberto.sassu@huawei.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Satya Priya Kakitapalli <quic_skakitap@quicinc.com>
  Steve French <stfrench@microsoft.com>
  Suren Baghdasaryan <surenb@google.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tiwei Bie <tiwei.btw@antgroup.com>
  Tony Luck <tony.luck@intel.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wang Yong <wang.yong12@zte.com.cn>
  Waqar Hameed <waqar.hameed@axis.com>
  Will Deacon <will@kernel.org>
  Xiubo Li <xiubli@redhat.com>
  Yuanyuan Zhong <yzhong@purestorage.com>
  Yueh-Shun Li <shamrocklee@posteo.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2966 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 27 05:18:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 05:18:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730582.1135696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBSkF-00008K-EF; Mon, 27 May 2024 05:18:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730582.1135696; Mon, 27 May 2024 05:18:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBSkF-00008D-B8; Mon, 27 May 2024 05:18:03 +0000
Received: by outflank-mailman (input) for mailman id 730582;
 Mon, 27 May 2024 05:18:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBSkD-000083-TO; Mon, 27 May 2024 05:18:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBSkD-00072n-RY; Mon, 27 May 2024 05:18:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBSkD-0006W5-KO; Mon, 27 May 2024 05:18:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBSkD-0007SE-K1; Mon, 27 May 2024 05:18:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=oANKl1ASreOnaU4xrFDKbuYv5jAelqccWXJ18cx/S98=; b=Y4JsJ5DKjfPDyMV6/XWrNNIKIY
	3G3KjEJuJnM4bY1SiO+sBsAglHAu3lPs6E0DcuVFGkf5ZkTIPQCSVW+9Ad5fBo3O6wcla4Y4naIyG
	Eb8w65qkOtnlqRsGP9OL5K/KWDakbk9EtN6ljuK9291sWO+gmJm4lgj4ezakIcswZEes=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186158-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186158: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=88a4de450f17c6a319c3e8b2135cd7068a07d0f8
X-Osstest-Versions-That:
    ovmf=3e722403cd16388a0e4044e705a2b34c841d76ca
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 05:18:01 +0000

flight 186158 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186158/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 88a4de450f17c6a319c3e8b2135cd7068a07d0f8
baseline version:
 ovmf                 3e722403cd16388a0e4044e705a2b34c841d76ca

Last test of basis   186137  2024-05-24 16:12:58 Z    2 days
Testing same since   186158  2024-05-27 02:13:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ning Feng <ning.feng@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3e722403cd..88a4de450f  88a4de450f17c6a319c3e8b2135cd7068a07d0f8 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon May 27 06:20:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 06:20:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730590.1135706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBTi4-00070R-Tb; Mon, 27 May 2024 06:19:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730590.1135706; Mon, 27 May 2024 06:19:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBTi4-00070K-Qg; Mon, 27 May 2024 06:19:52 +0000
Received: by outflank-mailman (input) for mailman id 730590;
 Mon, 27 May 2024 06:19:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBTi4-00070D-Ao
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 06:19:52 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1bd87021-1bf1-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 08:19:44 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-5751bcb3139so12404200a12.1
 for <xen-devel@lists.xenproject.org>; Sun, 26 May 2024 23:19:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817ee7sm456767466b.33.2024.05.26.23.19.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 26 May 2024 23:19:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bd87021-1bf1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716790783; x=1717395583; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=pPEoGC6JBivgVNjVQ7HTaR0H3WFVtC4CHpf6XL3VwVo=;
        b=L2fmp96aF1y5PgAtUgtzJIjQmwwP4nTwc6UFiUyU4WlyMtl0cVcxY6/MXGKiG7SqwE
         ENCTDVa+aDUi0zCqi9GpQpUuJ73adxZc7Y+Dwh++3XKbcq58ReuUkCL6wo2dsT76bqNv
         ogEmRPL4NbP5sjcHz9DoAVAyV36O/NStcaHWMbCuuo7fLyPCQmCi408R9YupLKoarb38
         RWagl3eBPztkJLxy4G486Rrol0sL9+ImK4tMJvBtVuoWwmzulngHcDbeQdFQNOBffBHa
         +nG9Qe9VbX2+r8CjKMVXRM76FGABZBNHpgjBee/cDZJB+uImENa7kg2cPcNDjKGBRsKZ
         Shlg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716790783; x=1717395583;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=pPEoGC6JBivgVNjVQ7HTaR0H3WFVtC4CHpf6XL3VwVo=;
        b=VctHLKUQfVx1D7X3zqHPr9MwZNjnyxUCfcc1dtdPvxmulhvF1jIlQ1/sa5rJCSnQ0s
         TxvNorlosM+H/TLGzlHh5fWSjxxXmwA3P2vQoL6brsmuoEmGLAHt7hLqTg1EWlkzSiwh
         xHXnXOT2aJtbrhROJgFKWU4aklyV0ZVZNOz9fzJ8rHRZq7p4GC/3bXSjEeepr4I+8Oiw
         islGH/01avmz6xD/5L6FMDmstGsUrR5hPwrL0QrVW/940oo9Uo3fSyhbUdsKFtHm7sAS
         LQyxuMMddD26678prV1L58eR7Y/4bD7/vzsTsg4+zBxr5+KO3zsY+9imt6SjYi1unCAL
         1cFA==
X-Forwarded-Encrypted: i=1; AJvYcCUMhbLqxms+6sTAWarNMoSv0Gh7XM98HcSgSZA3mR8J4IlS4H21+sZKNWSOxmNMiFeRVTOSoZ7eZhc2mpbl9POCzIai03j+DgMRElv5Zo4=
X-Gm-Message-State: AOJu0Yx5xvjgxf+4V4VGA9P9TsZ4E7zkdMi8GZEQ01Kf0UAmBGP2rIB5
	TXT+CDYZdXZxYMv8PSPdIKWngNdb1QSkCkrkNW0c2VFRMGel64HO9fNFx3MC4Q==
X-Google-Smtp-Source: AGHT+IFOfZ5hLGKVeXcvYK9fUE8Wahf90KAa/8ajN1R9RYiivdm1+raj8QxOZdFqq2J5nio0rkvswA==
X-Received: by 2002:a17:906:d961:b0:a62:196c:b2af with SMTP id a640c23a62f3a-a6264f01e2dmr563294366b.52.1716790783528;
        Sun, 26 May 2024 23:19:43 -0700 (PDT)
Message-ID: <12335f4d-adde-42d3-aa25-b662e701b96d@suse.com>
Date: Mon, 27 May 2024 08:19:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m
 views configurable
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <cover.1716029860.git.w1benny@gmail.com>
 <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
 <21196eb3-82ca-470b-8dc9-1a95b427f3b9@suse.com>
 <CAKBKdXj7xUyuy-wPCjR0a+kNwqCmGXOoO=-pMQa7=Yiyaom_mg@mail.gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAKBKdXj7xUyuy-wPCjR0a+kNwqCmGXOoO=-pMQa7=Yiyaom_mg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27.05.2024 01:55, Petr Beneš wrote:
> On Tue, May 21, 2024 at 12:59 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> The compared entities don't really fit together. I think we want a new
>> MAX_NR_ALTP2M, which - for the time being - could simply be

Note that you've stripped too much context - "the compared entities" is
left without any meaning here, yet that's relevant to my earlier reply.

>> #define MAX_NR_ALTP2M MAX_EPTP
>>
>> in the header. That would then be a suitable replacement for the
>> min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) that you're adjusting
>> elsewhere. Which however raises the question whether in EPT-specific
>> code the min() wouldn't better survive, as min(d->nr_altp2m, MAX_EPTP).
>>
> 
> As you mentioned in a previous email, I've removed all the min(...,
> MAX_EPTP) invocations from the code, since nr_altp2m is validated to
> be no greater than that value. The only remaining places where this
> value occurs are:
> 
> - In my newly introduced condition in arch_sanitise_domain_config:
> 
> if ( config->nr_altp2m > MAX_EPTP )
> {
>     dprintk(XENLOG_INFO, "nr_altp2m must be <= %lu\n", MAX_NR_ALTP2M);
>     return -EINVAL;
> }

This is suspicious: You compare against one value but log another. This
isn't EPT-specific, so shouldn't use MAX_EPTP.

> - In hap_enable():
> 
> for ( i = 0; i < MAX_EPTP; i++ )
> {
>     d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
>     d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
> }
> 
> Note that altp2m_eptp/altp2m_visible_eptp is never accessed beyond
> nr_altp2m. From what you're saying, it sounds to me like I should only
> replace the first mentioned occurrence with MAX_NR_ALTP2M. Correct me
> if I'm wrong.

Yes. I suspect though that there may be further places that want adjusting.

>>> @@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
>>>      /* altp2m view 0 is treated as the hostp2m */
>>>      if ( altp2m_idx )
>>>      {
>>> -        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
>>> -             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
>>> -             mfn_x(INVALID_MFN) )
>>> +        if ( altp2m_idx >= d->nr_altp2m ||
>>> +             d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, d->nr_altp2m)]
>>> +             == mfn_x(INVALID_MFN) )
>>
>> Please don't break previously correct style: Binary operators (here: == )
>> belong onto the end of the earlier line. That'll render the line too long
>> again, but you want to deal with that e.g. thus:
>>
>>              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx,
>>                                                     d->nr_altp2m)] ==
>>              mfn_x(INVALID_MFN) )
>>
> 
> Roger suggested introducing the altp2m_get_p2m() function, which I
> like. I think introducing altp2m_get_eptp/visible_eptp and
> altp2m_set_eptp/visible_eptp would also elegantly solve the issue of
> overly long lines. My question is: if I go this route, should I
> strictly replace with these functions only accesses that use
> array_index_nospec()? Or should I replace all array accesses? For
> example:
> 
> for ( i = 0; i < d->nr_altp2m; i++ )
> {
>     struct p2m_domain *p2m;
> 
>     if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
>         continue;
> 
>     p2m = d->arch.altp2m_p2m[i];
> 
>     p2m_lock(p2m);
>     p2m->ept.ad = value;
>     p2m_unlock(p2m);
> }
> 
> ... should I be consistent and also replace these accesses with
> altp2m_get_eptp/altp2m_get_p2m (which will internally use
> array_index_nospec), or should I leave them as they are?

Perhaps leave them as they are, unless you can technically justify the
adjustment.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 06:26:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 06:26:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730596.1135715 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBTod-0008TL-JP; Mon, 27 May 2024 06:26:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730596.1135715; Mon, 27 May 2024 06:26:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBTod-0008TE-Gh; Mon, 27 May 2024 06:26:39 +0000
Received: by outflank-mailman (input) for mailman id 730596;
 Mon, 27 May 2024 06:26:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBToc-0008Ss-B2
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 06:26:38 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 124ba49f-1bf2-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 08:26:37 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so889236366b.1
 for <xen-devel@lists.xenproject.org>; Sun, 26 May 2024 23:26:37 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc4fec8sm460223366b.99.2024.05.26.23.26.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 26 May 2024 23:26:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 124ba49f-1bf2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716791197; x=1717395997; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=O3KlDk3at8kArWpRw4gaoY2c83aieDAcfR2I+/iMY18=;
        b=ATXk/X9gtzYaLmmOeSfjOS8cP26YUUw2sLf+AF2gzKrtvowGEwxe1tNwuxG/jzb3Ys
         tokM599T+nfxrc4BJhrOb8oNeHyQMy42p2iLC0/uZftiL7sUN7DUAHhpYPiUSIQFNxKo
         bjWxz7l8fjoQqEzyiidFDm2gl0SIccd2oekXXsiiObVUTj9ZjIMboaujDtiBmp8aQo+l
         G/039U1b8zJSGMl5Z6XNkhGhH/gWasmEFUOYEhqux+36x8buU32zRF7mbg9urLD8xvlY
         oMUHX+HLlnzOYA2tibthABLY2HuzkEwZJDRm6oHUOON5+junJABMpgZLUbmz8KJq06rD
         eK8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716791197; x=1717395997;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=O3KlDk3at8kArWpRw4gaoY2c83aieDAcfR2I+/iMY18=;
        b=HA8b1HEz3Iv1ktKsasuDoTzqNXBpPYYuBSFj3HN00cG8UTg+ucruRty4DDAm1Mt6ik
         qBgcqPnlQOtYU+3GR355LQjOjXsFRAY6UhVCc7NwzQAXDxgD6E4QfEcNg5FIOtApMjCT
         oP+h8EpAvz9JAcs5dE0sZyruRbDWKoeK1LGiI1nfi05xOoG6IyZclWASTicx8rik1gZC
         5d5BGoMWtfgi1I+LGkpV7QtIopuEAUu/7h8sHw+CuqC0e0YRJtzdZp3evkDfE9hXTHVS
         ICUSG3cj5pbNaqxnQPSukEoLRQFDlIhEwVKkzURrIFCjnGPZwoV3+QQErzRrXeEdAk2R
         rfCw==
X-Forwarded-Encrypted: i=1; AJvYcCX5bVcPUSFaA5SrbG8sjJIwi6f4uENwojmypUSR+uROXN9Cp20Vueh2g5wKkvWZ9dshxMZWe+UlzAGI/2jap3W8Nnyh6o/tVvzYDjlvgos=
X-Gm-Message-State: AOJu0YwRRS01vPUwEFLLkyhtnLyh0d/9xqT7ew8VZ05n0me9NjdaKiBX
	HYTJl/qLDQMQNDDKo++/N01GXIW7f27FFCT8ncRJIqcNEzuwbD1F1REh9LVJdw==
X-Google-Smtp-Source: AGHT+IFSNdFXrWqgM5qr6/IzgalHyqRwX2Z2L/AmaAQNHRbS9ZdfS5vGdTpGcrSuirAagqPn1+o+Ew==
X-Received: by 2002:a17:906:874c:b0:a62:2cac:b78e with SMTP id a640c23a62f3a-a62616df030mr745198766b.6.1716791197017;
        Sun, 26 May 2024 23:26:37 -0700 (PDT)
Message-ID: <d94f3c36-66f1-4db0-bc57-477e48c61240@suse.com>
Date: Mon, 27 May 2024 08:26:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 04/13] xen/page_alloc: Coerce min(flsl(), foo)
 expressions to being unsigned
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> This is in order to maintain bisectability through the subsequent changes,
> where flsl() changes sign-ness non-atomically by architecture.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 06:46:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 06:46:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730600.1135726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBU7W-0002m2-58; Mon, 27 May 2024 06:46:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730600.1135726; Mon, 27 May 2024 06:46:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBU7W-0002lv-21; Mon, 27 May 2024 06:46:10 +0000
Received: by outflank-mailman (input) for mailman id 730600;
 Mon, 27 May 2024 06:46:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBU7U-0002ll-QY; Mon, 27 May 2024 06:46:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBU7U-0001Qb-Mh; Mon, 27 May 2024 06:46:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBU7U-0001Pt-9Q; Mon, 27 May 2024 06:46:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBU7U-00067T-8t; Mon, 27 May 2024 06:46:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NYKSB9XhV6HQkhyxBisy4hY0tU1UnHZxkNHQBXwfBXo=; b=zCUrgtYmlonKuD3e+gE1hGFjV+
	oXY0Y0rgGDAar6RcbY5r5A9DDFCi+r+LLc/P8wyKPTf4oRpoY1VjL7I6n/YkJyNHhsBx8SDo0e31e
	IlzU1C4NSaLdd9kJix34S/j1fGoWfBiSIrGCLz/qj2DTWeLGzaW6jQjUjMTz3DCoPY3w=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186156-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186156: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:build-armhf:xen-build:fail:regression
    linux-linus:build-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
X-Osstest-Versions-That:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 06:46:08 +0000

flight 186156 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186156/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 186148
 build-armhf                   6 xen-build                fail REGR. vs. 186148

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186148
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
baseline version:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c

Last test of basis   186148  2024-05-25 18:11:58 Z    1 days
Failing since        186152  2024-05-26 05:15:42 Z    1 days    3 attempts
Testing same since   186156  2024-05-27 01:11:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Allen Pais <allen.lkml@gmail.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Konovalov <andreyknvl@gmail.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@kernel.org>
  Artem Ikonnikov <artem@datacrunch.io>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Carsten Tolkmit <ctolkmit@ennit.de>
  Chengming Zhou <chengming.zhou@linux.dev>
  Christian Heusel <christian@heusel.eu>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Corey Minyard <cminyard@mvista.com>
  Corey Minyard <minyard@acm.org>
  David Hildenbrand <david@redhat.com>
  David Howells <dhowells@redhat.com>
  Dev Jain <dev.jain@arm.com>
  dicken.ding <dicken.ding@mediatek.com>
  Dongli Zhang <dongli.zhang@oracle.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Frank Li <Frank.Li@nxp.com>
  Guenter Roeck <linux@roeck-us.net>
  Hailong.Liu <hailong.liu@oppo.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Ilya Dryomov <idryomov@gmail.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Klara Modin <klarasmodin@gmail.com>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kunwu Chan <chentao@kylinos.cn>
  Lenko Donchev <lenko.donchev@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Mia Lin <mimi05633@gmail.com>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Hocko <mhocko@suse.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nandor Kracser <bonifaido@gmail.com>
  Nico Pache <npache@redhat.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Palmer Dabbelt <palmer@rivosinc.com>
  Randy Dunlap <rdunlap@infradead.org>
  Richard Weinberger <richard@nod.at>
  Roberto Sassu <roberto.sassu@huawei.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Satya Priya Kakitapalli <quic_skakitap@quicinc.com>
  Steve French <stfrench@microsoft.com>
  Suren Baghdasaryan <surenb@google.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tiwei Bie <tiwei.btw@antgroup.com>
  Tony Luck <tony.luck@intel.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wang Yong <wang.yong12@zte.com.cn>
  Waqar Hameed <waqar.hameed@axis.com>
  Will Deacon <will@kernel.org>
  Xiubo Li <xiubli@redhat.com>
  Yuanyuan Zhong <yzhong@purestorage.com>
  Yueh-Shun Li <shamrocklee@posteo.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2993 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon May 27 07:45:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 07:45:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730610.1135736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBV2y-0001CJ-J3; Mon, 27 May 2024 07:45:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730610.1135736; Mon, 27 May 2024 07:45:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBV2y-0001CC-GE; Mon, 27 May 2024 07:45:32 +0000
Received: by outflank-mailman (input) for mailman id 730610;
 Mon, 27 May 2024 07:45:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0flO=M6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBV2x-0001C6-Rh
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 07:45:31 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 174f1edf-1bfd-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 09:45:30 +0200 (CEST)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2e719bab882so104065261fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 00:45:30 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100fb9042sm132704485e9.39.2024.05.27.00.45.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 00:45:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 174f1edf-1bfd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716795930; x=1717400730; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=OCwXm4CkTF9x2cDNqDZPQHJI1jrYKUj/dD8FNi/knVI=;
        b=i4YqivSLcC3XoNSu+X6gMjwKPRA7yy6kDc6UwWmiZJNXCsA0l/vq/fq3gPZgEucOWO
         37mvldd8tq6HudGvX7uqmeM2WQi6fM9LZOFA6XhIzOTbpvLN/8033eELSUSRMGsBNQ4A
         fygyEyy2+AJNKmkjq9m/9ib4e8LlDxyi5hnaI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716795930; x=1717400730;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OCwXm4CkTF9x2cDNqDZPQHJI1jrYKUj/dD8FNi/knVI=;
        b=AMzsPf89/siNnZDK3dTGjP3G+gguEBijtKdo1rgQLXr3ApPGlK115qi2pleLNEvh0s
         pwmQyRY8s5UsJm0lhFm27ONjLYfrMIE9A8V6DlN4tp0J4JcpqU5d2nwUrlleckQOyUVv
         th1/pRpVDJYj2gAuDleLBxYzoKoqAQrpeY3VWKfPTRuWiPC67eai8z3VwiLMj20dhcya
         aZ78FAt2ehSnBhiTGgQs4Q1m5QDNCc0wfBXVjQ0uJ47Hk8HFvvvuzUatv+AqRc26m9dV
         pZPDWmd5+gRWposyqAmjsDvVx2+FU7TCnik6osPiWyCI/4yg7c+Q3OIbDuOcyVLloEn9
         M0IQ==
X-Forwarded-Encrypted: i=1; AJvYcCXgkJ1nDHJdwXaC55CDjds3J2y7F1PfFTnkc7LQna8M9vKQc1jMk+USGFr9oDuvoyafsqQSoxL3ZuOP87hK5Sz6qZxiYd98Qh79K68gDEc=
X-Gm-Message-State: AOJu0YxGtURFiBpr+x0DzneedXcKEAuCKln9FGUaHq726TY4Dx0JpaEn
	z3FEF3+OMcpOEy4SWnFmy5oJUfFw3vozYUfzC67P5D8NAW67mOGUSgz5IfePCBY=
X-Google-Smtp-Source: AGHT+IHNfO1uozzSIFlA74/2waqzTYuaRNA2g91sAU1dtroDz76kpwdIn0iCG1zTtiVxR77FxmZCFA==
X-Received: by 2002:a2e:2c16:0:b0:2e5:8720:50d2 with SMTP id 38308e7fff4ca-2e95af3493dmr53328111fa.0.1716795929812;
        Mon, 27 May 2024 00:45:29 -0700 (PDT)
Date: Mon, 27 May 2024 09:45:28 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v16 4/5] xen/arm: translate virtual PCI bus topology for
 guests
Message-ID: <ZlQ6GNO6vv8lTHTM@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-5-stewart.hildebrand@amd.com>
 <Zk70udmiriruMt0r@macbook>
 <4afe33f2-72e6-4755-98ce-d7f9da374e90@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4afe33f2-72e6-4755-98ce-d7f9da374e90@xen.org>

On Fri, May 24, 2024 at 02:21:09PM +0100, Julien Grall wrote:
> Hi,
> 
> Sorry I didn't notice there was a v16 and posted comments on the v15. The
> only one is about the size of the list we iterate.
> 
> On 23/05/2024 08:48, Roger Pau Monné wrote:
> > On Wed, May 22, 2024 at 06:59:23PM -0400, Stewart Hildebrand wrote:
> > > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > +    }
> > > -    return sbdf;
> > > +    return translated;
> > >   }
> > >   static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> > >                             register_t *r, void *p)
> > >   {
> > >       struct pci_host_bridge *bridge = p;
> > > -    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> > > +    pci_sbdf_t sbdf;
> > >       const unsigned int access_size = (1U << info->dabt.size) * 8;
> > >       const register_t invalid = GENMASK_ULL(access_size - 1, 0);
> > 
> > Do you know why the invalid value is truncated to the access size.
> 
> Because no other callers are doing the truncation and therefore the guest
> would read 1s even for 8-byte unsigned access.

I think forcing all handlers to do the truncation is a lot of
duplication, and more risky than just doing it in the dispatcher
itself (handle_read()), see my reply to 1/5.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:06:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:06:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730619.1135745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVMz-0004R3-9b; Mon, 27 May 2024 08:06:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730619.1135745; Mon, 27 May 2024 08:06:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVMz-0004Qw-6k; Mon, 27 May 2024 08:06:13 +0000
Received: by outflank-mailman (input) for mailman id 730619;
 Mon, 27 May 2024 08:06:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0flO=M6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBVMx-0004Qq-VO
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:06:11 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa00ddb7-1bff-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 10:06:09 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-420180b58c5so44361875e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:06:09 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421089ae976sm99890605e9.38.2024.05.27.01.06.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 01:06:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa00ddb7-1bff-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716797169; x=1717401969; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=UCNZOPVcSv6vgS5x24ELfr5/tjnkICUuAG3SIv/QHNM=;
        b=jKPTSqY1z5tltO2mMhTbgWP9yK5rJCNCRkJIIqQgmjqziTCy1CWrXvHrdPjcsb99RJ
         kyqZ3fRCIK6aACXP0lZ1dBYZiqKvScKnLTlUbNraKfTz3uowe5a57F+ELSqWAK8jdmzh
         01K79P2sqsYryD0PdbsCVJUMbNpEStAHQvtyE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716797169; x=1717401969;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UCNZOPVcSv6vgS5x24ELfr5/tjnkICUuAG3SIv/QHNM=;
        b=C8BFC7EMVtUqnD6eZ64/NesYYHJv+uCNFwzC7xXpNjnpj1APPsN5gV1UXFPfZFi7H6
         g6i9gOFqN3h8lU69TpumkqpMIOw5yZVmiL4mGDb1+8AYDpua14bJmr3L4jpYGNf4fuLX
         bJ67kXeUhCIOTdD7xsXCISRQFE5Ab9ah43qtDi0+LfRKX0s9iU5XPwbSPWxYOyGmIEZ4
         ViLL4/bq1JMMCrAhxZuvototW9Q3QWEaOW1oS2tS7kui9bJVhrMGZxktAgWM7DqET6bV
         VeSbQPT29CSuGmcz9CQPRTDNcB6Kzml1cZ3oItq5VOY92rnqxh1Ria4GPodIjk6yRNe0
         wHgg==
X-Gm-Message-State: AOJu0YzZeUTetQMcipeodcE4nRyup+YDcImo+C1MwUP9/OPOLp49SY35
	elJeA29Cqa6xRgW22uljri46/j9QHCzn0fOLCWfCfBYWth5Dz2gfCs44rJV3vBY=
X-Google-Smtp-Source: AGHT+IHSJSvQdnZzenFvFruvxK3Dm5P//mmkDbY65D90oaTnglYrgIB45e5seH+GnEjPJPf3jixtNw==
X-Received: by 2002:a05:600c:4592:b0:41a:bdaf:8c78 with SMTP id 5b1f17b1804b1-421089b2201mr73713255e9.8.1716797169111;
        Mon, 27 May 2024 01:06:09 -0700 (PDT)
Date: Mon, 27 May 2024 10:06:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 7/8] xen/x86: Derive topologically correct x2APIC IDs
 from the policy
Message-ID: <ZlQ-7w0Kmzlmvfb1@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <87a2a4589e330472b7260ff6ab513744596a4488.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlBSPYCpEAosNVzo@macbook>
 <1e16ffa9-3f6c-43ed-b36c-55fe132c8145@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1e16ffa9-3f6c-43ed-b36c-55fe132c8145@cloud.com>

On Fri, May 24, 2024 at 06:03:22PM +0100, Alejandro Vallejo wrote:
> On 24/05/2024 09:39, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:26PM +0100, Alejandro Vallejo wrote:
> > 
> > Also you could initialize x2apic_id at definition:
> > 
> > const struct test *t = &tests[j];
> > struct cpu_policy policy = { .x86_vendor = vendors[i] };
> > int rc = x86_topo_from_parts(&policy, t->threads_per_core, t->cores_per_pkg);
> > uint32_t x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);
> 
> Seeing this snippet I just realized there's a bug. The second loop
> should use j rather than i. Ugh.

Well, you shadow the outer variable with the inner one, which makes it
still fine.  Yet I don't like that shadowing much.  I was going to
comment, but for the requested change you need to not shadow the outer
loop variable (in the example chunk I've used 'j' to signal the outer
loop index).

> >> +}
> >> +
> >>  uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
> >>  {
> >> +    uint32_t shift = 0, x2apic_id = 0;
> >> +
> >> +    /* In the absence of topology leaves, fallback to traditional mapping */
> >> +    if ( !p->topo.subleaf[0].type )
> >> +        return id * 2;
> >> +
> >>      /*
> >> -     * TODO: Derive x2APIC ID from the topology information inside `p`
> >> -     *       rather than from vCPU ID. This bodge is a temporary measure
> >> -     *       until all infra is in place to retrieve or derive the initial
> >> -     *       x2APIC ID from migrated domains.
> > 
> > I'm a bit confused with this, the policy is domain wide, so we will
> > always need to pass the vCPU ID into x86_x2apic_id_from_vcpu_id()?
> > IOW: the x2APIC ID will always be derived from the vCPU ID.
> > 
> > Thanks, Roger.
> 
> The x2APIC ID is derived (after the series) from the vCPU ID _and_ the
> topology information. The vCPU alone will work out in all cases because
> it'll be cached in the vlapic hvm structure.
> 
> I guess the comment could be rewritten as "... rather than from the vCPU
> ID alone..."

Yup, that's clearer :).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:20:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:20:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730626.1135756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVbA-0007F0-Ga; Mon, 27 May 2024 08:20:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730626.1135756; Mon, 27 May 2024 08:20:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVbA-0007Et-DG; Mon, 27 May 2024 08:20:52 +0000
Received: by outflank-mailman (input) for mailman id 730626;
 Mon, 27 May 2024 08:20:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0flO=M6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBVb8-0007En-P2
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:20:50 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05b8eebb-1c02-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 10:20:48 +0200 (CEST)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-356c4e926a3so1710423f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:20:48 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3587724c6f4sm3336591f8f.28.2024.05.27.01.20.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 01:20:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05b8eebb-1c02-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716798048; x=1717402848; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+DBJ1woGsE1g35pnBuR+vdgr+wti5M2VoyQK+WzYEAQ=;
        b=DxUfvlHJrhZ6jMjPtZm570xy5/lKXn2gwV/NjBgdX7J2cCecF6kF8AiUXAYyMFjVy8
         PWIfZia69c0Ul3ZzBY955o2I0dLFTXqDyKtOqZVmyMuMwgpzo/KrtDWcYpPkRTKG6ZPs
         BJxxVQDJdpX060pEBODrGnvJ6VPlsUCNfSj5U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716798048; x=1717402848;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+DBJ1woGsE1g35pnBuR+vdgr+wti5M2VoyQK+WzYEAQ=;
        b=VD5LsTeltS0NZ+4VnpXO9P5u3P56kYekSVZUr3GLoKH9ZtkUORLV8h+PMNt9aN07aM
         XAGtvKXXKOdG0xiqh+HEvKKvsfjM7tFIcBuwl5v6x+n5y/WgEj3HXZGQoAY2Q9oHAMYL
         XKGS0DyD8FAmzxRr4DEM2HT71J7XonJabBTTzu646h6trnLAYoUYnsp7EohZX889HZgH
         u3ifrBGv7QxwWVld1t2Sqfn7/tk9Z+NltafXtSCjiyK0SC4ENTa3xEXgnMBURwGfV/+T
         Xt8hJG6VyWfuG7Kaqmf9KNrhJWDeE7ATksu61lM7vQg92oC71/DBPaotXLuIYp36+3Um
         jQeg==
X-Gm-Message-State: AOJu0YxjwbFSLO/snqR9YmSq2enybtfClO31DQtpUzhxgy8aLFBrceqQ
	VesrBIlrB6OYTxJu1Ia3Z7QjctBljPCsx/KzxduKZTxWwCwfMrhvPWFtlqTGenE=
X-Google-Smtp-Source: AGHT+IGhVN18oedPG+wfaoRL0TJAbh8n8SUEfimQDMiXQcjH4MwupcAZouUmwhwVx9WLYsqvO/YNbA==
X-Received: by 2002:a05:6000:a8f:b0:354:e775:19ff with SMTP id ffacd0b85a97d-35527057550mr6750167f8f.38.1716798047736;
        Mon, 27 May 2024 01:20:47 -0700 (PDT)
Date: Mon, 27 May 2024 10:20:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 8/8] xen/x86: Synthesise domain topologies
Message-ID: <ZlRCXmjhsM5o4tVw@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlBWtXkpkqzh1EWC@macbook>
 <6e1c52a3-98d2-42e9-96b7-91010806cdf5@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <6e1c52a3-98d2-42e9-96b7-91010806cdf5@cloud.com>

On Fri, May 24, 2024 at 06:16:01PM +0100, Alejandro Vallejo wrote:
> On 24/05/2024 09:58, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:27PM +0100, Alejandro Vallejo wrote:
> > 
> >> +        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);
> > 
> > I assume this generates the same topology as the current code, or will
> > the population of the leaves be different in some way?
> > 
> 
> The current code does not populate 0xb. This generates a topology
> consistent with the existing INTENDED topology. The actual APIC IDs will
> be different though (because there's no skipping of odd values).
> 
> All the dance in patch 1 was to make this migrate-safe. The x2apic ID is
> stored in the lapic hidden regs so differences with previous behaviour
> don't matter.

What about systems without CPU policy in the migration stream, will
those also get restored as expected?

I think you likely need to check whether 'restore' is set and keep the
old logic in that case?

As otherwise migrated systems without a CPU policy will get the new
topology information instead of the old one?

> IOW, The differences are:
>   * 0xb is exposed, whereas previously it wasn't
>   * APIC IDs are compacted such that new_apicid=old_apicid/2
>   * There's also a cleanup of the murkier paths to put the right core
>     counts in the right leaves (whereas previously it was bonkers)

This needs to be in the commit message IMO.

> > 
> > Note that currently the host policy also gets the topology leaves
> > cleared, is it intended to not clear them anymore after this change?
> > 
> > (as you only clear the leaves for the guest {max,def} policies)
> > 
> > Thanks, Roger.
> 
> It was like that originally in v1, I changed in v2 as part of feedback
> from Jan.

I think that's fine, but this divergence from current behavior of
cleaning the topology for the host policy needs to be mentioned in
the commit message.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:24:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:24:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730632.1135766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVeX-0007op-U7; Mon, 27 May 2024 08:24:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730632.1135766; Mon, 27 May 2024 08:24:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVeX-0007oi-RU; Mon, 27 May 2024 08:24:21 +0000
Received: by outflank-mailman (input) for mailman id 730632;
 Mon, 27 May 2024 08:24:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBVeW-0007oc-9M
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:24:20 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82aa0e91-1c02-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 10:24:18 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-57864327f6eso3282063a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:24:18 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57860a2eab3sm4401359a12.28.2024.05.27.01.24.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 01:24:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82aa0e91-1c02-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716798257; x=1717403057; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BaTlOnCaTA/1XHNDwnVyAuCDZhHsZEyVzojdDKal27s=;
        b=XxdvfbM28j5EisTjHpqxQfQfSflHXClW7ikd9KKZ0XuXsc4c4I6iLoVGlDAR7zFOiO
         r9bWslAXRosyWiHkp1Avj+YERK5ZOmmzjEvKU0nQ1GJjs02hzJQTZrTg2tK1eeWebCN/
         FrtPXKeb704iKWXbRK7lzPh+7hwCMrpjEGvnqiMhmkFeSdu2VmIpidssSpVfPhf5/yaY
         2NRHkzZjeNVRfdXhW8R4JpODAe4zoMVPM5AbIDAPm1GSpNZz0XOBrQmqkulEn0Dqy9zh
         nPCVrnXALtFm86Q1kGzlA9ZAYjiojcLU+cWa+MebKcDG6Vq+UCLpamI5fsxXX/T3Dfyc
         iNpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716798257; x=1717403057;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BaTlOnCaTA/1XHNDwnVyAuCDZhHsZEyVzojdDKal27s=;
        b=KumRrXNYV6O09uz2YxqKROy0l/achY3NLCM19uXRT5lZGFtMqjSLogqoPZPZpRxh0O
         AqCL6chS3GykoBn/6eUwEb+H/sF/zUE1EI7LHA4wD5EM0FjnOrh+QqawzYy4YOsUbc+Y
         dsmKu0I0LzcwXThGRXzMuYpbACaUiDP2pInYCjKHA8r38FxSb3MYr6VnWqrB0NS4eGtm
         NU+SPlsaEDtNI50/yS2flWW4VooMCXWsEBUxvCu3Pbv9Q2RrBwHJ2KY7Wct+KLsv/8p5
         2yTmV1lpVYrQQwYH8lYkhJOrApAC9xz1uXDPP0BnmxE48OT2Z25FZkNdpA8bO8SXvHpn
         d+hQ==
X-Forwarded-Encrypted: i=1; AJvYcCU0oUpcyA4FtL+IqJGjC2H7C+6pYhdiTry6Nt9s7RLwsk/zri7C9kR26kLAKZWCgLQ+gwkVYtCst85WHRP2J5mliUy0dMB+7wksyPqUENM=
X-Gm-Message-State: AOJu0Yw7d4SkP/V08m6vnEjaTj4EppXTWHLIwHwRpjQeLLpl744K08NI
	BiEc371i3SlLmUVU/VVZyt7Yvf0Sc5CQ1XhN8CYoPFqC+nQ6QTwX9dOdBT+LZA==
X-Google-Smtp-Source: AGHT+IGgnmxEyutQ0DzAbAPiRUjk+mU6UIbJa0hcLR8dIaVnTVUrpe34xgvjHGLwUjoZB/2FFfcHyg==
X-Received: by 2002:a50:8d4c:0:b0:572:3fac:e085 with SMTP id 4fb4d7f45d1cf-57843f1589fmr10741572a12.13.1716798257439;
        Mon, 27 May 2024 01:24:17 -0700 (PDT)
Message-ID: <c70f769b-57b9-4f71-9c31-8052b773d28a@suse.com>
Date: Mon, 27 May 2024 10:24:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 02/13] xen/bitops: Cleanup ahead of rearrangements
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
>  * Rename __attribute_pure__ to just __pure before it gains users.
>  * Introduce __constructor which is going to be used in lib/, and is
>    unconditionally cf_check.
>  * Identify the areas of xen/bitops.h which are a mess.
>  * Introduce xen/boot-check.h as helpers for compile and boot time testing.
>    This provides a statement of the ABI, and a confirmation that arch-specific
>    implementations behave as expected.
> 
> Sadly Clang 7 and older isn't happy with the compile time checks.  Skip them,
> and just rely on the runtime checks.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Further remarks, though:

> ---
>  xen/include/xen/bitops.h     | 13 ++++++--
>  xen/include/xen/boot-check.h | 60 ++++++++++++++++++++++++++++++++++++
>  xen/include/xen/compiler.h   |  3 +-
>  3 files changed, 72 insertions(+), 4 deletions(-)
>  create mode 100644 xen/include/xen/boot-check.h

The bulk of the changes isn't about bitops; it's just that you're intending
to first use it for testing there. The subject prefix therefore is somewhat
misleading.

> --- /dev/null
> +++ b/xen/include/xen/boot-check.h
> @@ -0,0 +1,60 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +/*
> + * Helpers for boot-time checks of basic logic, including confirming that
> + * examples which should be calculated by the compiler are.
> + */
> +#ifndef XEN_BOOT_CHECK_H
> +#define XEN_BOOT_CHECK_H
> +
> +#include <xen/lib.h>
> +
> +/* Hide a value from the optimiser. */
> +#define HIDE(x)                                                         \
> +    ({ typeof(x) _x = (x); asm volatile ( "" : "+r" (_x) ); _x; })

In principle this is a macro that could be of use elsewhere. That's also
reflected in its entirely generic name. It therefore feels mis-placed in
this header. Otoh though the use of "+r" is more restricting than truly
necessary: While I'm not sure if "+g" would work, i.e. if that wouldn't
cause issues with literals, pretty surely "+rm" ought to work, removing
the strict requirement for the compiler to put a certain value in a
register.

Assuming you may have reservations against "+g" / "+rm" (and hence the
construct wants keeping here), maybe rename to e.g. BOOT_CHECK_HIDE()?
Alternatively, if generalized, moving to xen/macros.h would seem
appropriate to me.

Finally, plainly as a remark with no request for any change (but
possibly a minor argument against moving to xen/macros.h), this construct
won't, afaict, work if x is of array(-of-const) type. A more specialized
variant may need introducing, should any such use ever appear.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:44:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:44:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730638.1135775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVxg-0002St-Fz; Mon, 27 May 2024 08:44:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730638.1135775; Mon, 27 May 2024 08:44:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBVxg-0002Sm-DR; Mon, 27 May 2024 08:44:08 +0000
Received: by outflank-mailman (input) for mailman id 730638;
 Mon, 27 May 2024 08:44:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBVxf-0002Sg-Tt
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:44:07 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 466a72a3-1c05-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 10:44:05 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52965199234so3262682e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:44:05 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578524b5898sm5531726a12.82.2024.05.27.01.44.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 01:44:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 466a72a3-1c05-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716799445; x=1717404245; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/fhmiO0NEDD4yqRoUXsKlv3IeX5bLl/f6Z453pOqPyE=;
        b=E6SUAvhm35qFpRd0VzVjCtYPhhNPf3tCTPreo8Zbp0VyBzPh7YqRESlLWdcsXUQL2t
         SN0/UK+U/qvxdDrw3shWwEqSuaGi2Z59DjGNsvSZZ64ZHPXmApe5H/hd8JMAXN1MSB9s
         M94AoXcWbKoAmG34RdDi42KUned7QYSoYDOu2ukjVshQAM5kEuR2A0MooQNpXNUVIHWq
         IEUoY8KBwMWGa1tQBsgmolOAvIcYtmowwDPsDZSw2T5GYgksIkN0p64vtr3ad3QgqzXs
         dGvGYxMTm5knY+ixD//d9z+twoIp26aE7tRbrpeRguo1EvshSsRX8S1rTwHtXIo4S9mL
         fR0w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716799445; x=1717404245;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/fhmiO0NEDD4yqRoUXsKlv3IeX5bLl/f6Z453pOqPyE=;
        b=wfHVoOBCJcYHF2H94SYz+Pk7SRysU3PQ+ElxA75H+ZlkSE+ipgcG7VVJ3okX7wdtWU
         iqJo7HuW6yH6PovMEzXVztxIwpOSz64nG3wRGGRep/P93xuQbu8OXh8v4etATi7Js3FU
         2ZCswa/+CPs+LxyDfqRdXVZ509P+GAvaXCgUdnhI4VQ6MRTzQy9VzdNWobfiNPtX8l98
         RTTXFY758AJOWaOYsWzswZ9bLJ9BoH0VNvXa/hSn9zo5QOyG5z8hLAp1WEzeb2h9qcBg
         mj/z6/LokuBT3pv5+vau0asCwh2A5lbs/xnSroYZTWjetUabHkuf38Ore4ZoAOrAtD4L
         okjw==
X-Forwarded-Encrypted: i=1; AJvYcCXnsjN/7pb6E/3CdIZ1RBsEDs7gsjcr20MDJt/jtsAJaZJydM6sa+cOGT+hIS/2JNt5VjdxhIMGlQ3DyK0PgiXhen5vWLZCqq36TZlOOnA=
X-Gm-Message-State: AOJu0YwKohQMcvvT2zI4Vypw7UboDY9b/rTIyAeZJePb0RiFadr/u0y0
	19SsNzlh9JEkiaW9F+l06xkkEdOde7LwtF1WVTfa2idERUQ84DsVLLl8DNPhlw==
X-Google-Smtp-Source: AGHT+IH+nnLt1sRnNaTWLZ1/9FmTTIN1KqF58OD6bQp7YgaCJuthf9zdpVTjWqJGXRNFSjOCdt/lzQ==
X-Received: by 2002:a05:6512:21e:b0:51a:d5d3:ee52 with SMTP id 2adb3069b0e04-529649c5eb2mr5492525e87.17.1716799444791;
        Mon, 27 May 2024 01:44:04 -0700 (PDT)
Message-ID: <7305a8b9-5f43-47e4-987b-c5ca76bcc312@suse.com>
Date: Mon, 27 May 2024 10:44:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/13] xen/bitops: Implement generic_f?sl() in lib/
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-6-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-6-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> generic_f?s() being static inline is the cause of lots of the complexity
> between the common and arch-specific bitops.h
> 
> They appear to be static inline for constant-folding reasons (ARM uses them
> for this), but there are better ways to achieve the same effect.
> 
> It is presumptuous that an unrolled binary search is the right algorithm to
> use on all microarchitectures.  Indeed, it's not for the eventual users, but
> that can be addressed at a later point.
> 
> It is also nonsense to implement the int form as the base primitive and
> construct the long form from 2x int in 64-bit builds, when it's just one extra
> step to operate at the native register width.
> 
> Therefore, implement generic_f?sl() in lib/.  They're not actually needed in
> x86/ARM/PPC by the end of the cleanup (i.e. the functions will be dropped by
> the linker), and they're only expected be needed by RISC-V on hardware which
> lacks the Zbb extension.
> 
> Implement generic_fls() in terms of generic_flsl() for now, but this will be
> cleaned up in due course.
> 
> Provide basic runtime testing using __constructor inside the lib/ file.  This
> is important, as it means testing runs if and only if generic_f?sl() are used
> elsewhere in Xen.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with a suggestion and a question.

> I suspect we want to swap CONFIG_DEBUG for CONFIG_BOOT_UNIT_TESTS in due
> course.  These ought to be able to be used in a release build too.

+1

> --- /dev/null
> +++ b/xen/lib/generic-ffsl.c
> @@ -0,0 +1,65 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#include <xen/bitops.h>
> +#include <xen/boot-check.h>
> +#include <xen/init.h>
> +
> +unsigned int generic_ffsl(unsigned long x)
> +{
> +    unsigned int r = 1;
> +
> +    if ( !x )
> +        return 0;
> +
> +#if BITS_PER_LONG > 32

To be future-proof, perhaps ahead of this

#if BITS_PER_LONG > 64
# error "..."
#endif

or a functionally similar BUILD_BUG_ON()?

> --- /dev/null
> +++ b/xen/lib/generic-flsl.c
> @@ -0,0 +1,68 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#include <xen/bitops.h>
> +#include <xen/boot-check.h>
> +#include <xen/init.h>
> +
> +/* Mask of type UL with the upper x bits set. */
> +#define UPPER_MASK(x) (~0UL << (BITS_PER_LONG - (x)))
> +
> +unsigned int generic_flsl(unsigned long x)
> +{
> +    unsigned int r = BITS_PER_LONG;
> +
> +    if ( !x )
> +        return 0;
> +
> +#if BITS_PER_LONG > 32
> +    if ( !(x & UPPER_MASK(32)) )
> +    {
> +        x <<= 32;
> +        r -= 32;
> +    }
> +#endif
> +    if ( !(x & UPPER_MASK(16)) )
> +    {
> +        x <<= 16;
> +        r -= 16;
> +    }
> +    if ( !(x & UPPER_MASK(8)) )
> +    {
> +        x <<= 8;
> +        r -= 8;
> +    }
> +    if ( !(x & UPPER_MASK(4)) )
> +    {
> +        x <<= 4;
> +        r -= 4;
> +    }
> +    if ( !(x & UPPER_MASK(2)) )
> +    {
> +        x <<= 2;
> +        r -= 2;
> +    }
> +    if ( !(x & UPPER_MASK(1)) )
> +    {
> +        x <<= 1;
> +        r -= 1;
> +    }
> +
> +    return r;
> +}

While, as you say, the expectation is for this code to not commonly come
into actual use, I still find the algorithm a little inefficient in terms
of the constants used, specifically considering how they would need
instantiating in resulting assembly. It may be that Arm's fancy constant-
move insns can actually efficiently synthesize them, but I think on most
other architectures it would be more efficient (and presumably no less
efficient on Arm) to shift the "remaining" value right, thus allowing for
successively smaller (and hence easier to instantiate) constants to be
used.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:52:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:52:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730644.1135785 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBW5R-0004Lu-CI; Mon, 27 May 2024 08:52:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730644.1135785; Mon, 27 May 2024 08:52:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBW5R-0004Ln-9f; Mon, 27 May 2024 08:52:09 +0000
Received: by outflank-mailman (input) for mailman id 730644;
 Mon, 27 May 2024 08:52:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0flO=M6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBW5Q-0004Lh-0K
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:52:08 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6542de2f-1c06-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 10:52:06 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-355080e6ff9so2228813f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:52:06 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a1c95absm8358761f8f.87.2024.05.27.01.52.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 01:52:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6542de2f-1c06-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716799926; x=1717404726; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=5Nj/Vs328r6a6QlUFucJaFjZBMVd7L2aGPqvHoZfrn4=;
        b=YSpEG4AnAwwKiHqAozzmBoLkaLhb02JENQ6b4d4upaIyiF2ZKu+ay6h1n2UHV96Lhp
         P6Ade8BvLmV6YnGtaLqvAU/Bb/AJ9B8L2exmezwRLAu3LSYd81sBHbRWQlHiikW975AO
         aeLf3PRrtruN0P2UZvrxq3I3lrcMAR1a5MKJ0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716799926; x=1717404726;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5Nj/Vs328r6a6QlUFucJaFjZBMVd7L2aGPqvHoZfrn4=;
        b=BzapTNCY81d4x9cwHpHH1OdvHFSr4YVSt5RvwOnII+CU2brdCujdSe6QYaW7dMGEoV
         j+oJ1EBFG4vMXaTiJq8yXuB6GD4NlvRSgh+KQBPHqrpsQA8+JuYVi/uroh5qLYKerL+D
         PY+AA/y5iUPqOxgarYoRipgdDQDeTLf6tolmVzvUnNViv74qtxonJpxw+ok1Ab6m7Skt
         w5vMf60jdGNxAK6mHZTlrVheH80pRssHHf6IMV+ZkAYUc+Z7NwEqCsbxvyOpbtmBDjVf
         Hd9uN2rIJD6nkPO3372Pid+LzNoGLiqSabsZ8HxbCq0a4tUyeirvV5bSFHglAegqZaws
         NdSg==
X-Gm-Message-State: AOJu0YwH5yqHNY8KftpGrp+wYt1KF0PkT22vdUsVRWu2/1Qk40SIosea
	D3e5UTvaOe5LwRZ816ozAOAUFeCUQMs9WHhza6cxVCUcaJF98pwZAbpy/R6ZkDY=
X-Google-Smtp-Source: AGHT+IGL8jcBCEHtxR6QSk/ZQ0mcFuCa85folu6oCF9qiGNxcjtSXlU61yzDYkQOuULLmcF/ElfyDA==
X-Received: by 2002:a5d:42c8:0:b0:346:e9c4:65b with SMTP id ffacd0b85a97d-3552f9cfc66mr5133300f8f.27.1716799926147;
        Mon, 27 May 2024 01:52:06 -0700 (PDT)
Date: Mon, 27 May 2024 10:52:05 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
Message-ID: <ZlRJtN7h32uOHEOi@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlA_6Abtw-u4a84J@macbook>
 <8cac1707-855b-478a-b88d-7fd619f19352@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <8cac1707-855b-478a-b88d-7fd619f19352@cloud.com>

On Fri, May 24, 2024 at 04:15:34PM +0100, Alejandro Vallejo wrote:
> On 24/05/2024 08:21, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
> >> Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
> >> populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
> >> from hvmloader.
> >>
> >> While at this also remove ap_callin, as writing the APIC ID may serve the same
> >> purpose.
> >>
> >> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
> >> ---
> >> v2:
> >>   * New patch. Replaces adding cpu policy to hvmloader in v1.
> >> ---
> >>  tools/firmware/hvmloader/config.h    |  6 ++++-
> >>  tools/firmware/hvmloader/hvmloader.c |  4 +--
> >>  tools/firmware/hvmloader/smp.c       | 40 +++++++++++++++++++++++-----
> >>  tools/firmware/hvmloader/util.h      |  5 ++++
> >>  xen/arch/x86/include/asm/hvm/hvm.h   |  1 +
> >>  5 files changed, 47 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
> >> index c82adf6dc508..edf6fa9c908c 100644
> >> --- a/tools/firmware/hvmloader/config.h
> >> +++ b/tools/firmware/hvmloader/config.h
> >> @@ -4,6 +4,8 @@
> >>  #include <stdint.h>
> >>  #include <stdbool.h>
> >>  
> >> +#include <xen/hvm/hvm_info_table.h>
> >> +
> >>  enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
> >>  extern enum virtual_vga virtual_vga;
> >>  
> >> @@ -49,8 +51,10 @@ extern uint8_t ioapic_version;
> >>  
> >>  #define IOAPIC_ID           0x01
> >>  
> >> +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> >> +
> >>  #define LAPIC_BASE_ADDRESS  0xfee00000
> >> -#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
> >> +#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
> >>  
> >>  #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
> >>  #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
> >> diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
> >> index c58841e5b556..1eba92229925 100644
> >> --- a/tools/firmware/hvmloader/hvmloader.c
> >> +++ b/tools/firmware/hvmloader/hvmloader.c
> >> @@ -342,11 +342,11 @@ int main(void)
> >>  
> >>      printf("CPU speed is %u MHz\n", get_cpu_mhz());
> >>  
> >> +    smp_initialise();
> >> +
> >>      apic_setup();
> >>      pci_setup();
> >>  
> >> -    smp_initialise();
> >> -
> >>      perform_tests();
> >>  
> >>      if ( bios->bios_info_setup )
> >> diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
> >> index a668f15d7e1f..4d75f239c2f5 100644
> >> --- a/tools/firmware/hvmloader/smp.c
> >> +++ b/tools/firmware/hvmloader/smp.c
> >> @@ -29,7 +29,34 @@
> >>  
> >>  #include <xen/vcpu.h>
> >>  
> >> -static int ap_callin, ap_cpuid;
> >> +static int ap_cpuid;
> >> +
> >> +/**
> >> + * Lookup table of x2APIC IDs.
> >> + *
> >> + * Each entry is populated its respective CPU as they come online. This is required
> >> + * for generating the MADT with minimal assumptions about ID relationships.
> >> + */
> >> +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
> >> +
> >> +static uint32_t read_apic_id(void)
> >> +{
> >> +    uint32_t apic_id;
> >> +
> >> +    cpuid(1, NULL, &apic_id, NULL, NULL);
> >> +    apic_id >>= 24;
> >> +
> >> +    /*
> >> +     * APIC IDs over 255 are represented by 255 in leaf 1 and are meant to be
> >> +     * read from topology leaves instead. Xen exposes x2APIC IDs in leaf 0xb,
> >> +     * but only if the x2APIC feature is present. If there are that many CPUs
> >> +     * it's guaranteed to be there so we can avoid checking for it specifically
> >> +     */
> >> +    if ( apic_id == 255 )
> >> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
> >> +
> >> +    return apic_id;
> >> +}
> >>  
> >>  static void ap_start(void)
> >>  {
> >> @@ -37,12 +64,12 @@ static void ap_start(void)
> >>      cacheattr_init();
> >>      printf("done.\n");
> >>  
> >> +    wmb();
> >> +    ACCESS_ONCE(CPU_TO_X2APICID[ap_cpuid]) = read_apic_id();
> > 
> > Further thinking about this: do we really need the wmb(), given the
> > usage of ACCESS_ONCE()?
> > 
> > wmb() is a compiler barrier, and the usage of volatile in
> > ACCESS_ONCE() should already prevent any compiler re-ordering.
> > 
> > Thanks, Roger.
> 
> volatile reads/writes cannot be reordered with other volatile
> reads/writes, but volatile reads/writes can be reordered with
> non-volatile reads/writes, AFAIR.

Right, I've read the C99 spec and it does indeed only guarantee the
state of volatile objects to be as expected at sequence points.  I
think however this specific code would still be fine since the
function calls are considered to have side-effects, and those must all
be completed before the volatile access.

> My intent here was to prevent the compiler omitting the write (though in
> practice it didn't). I think the barrier is still required to prevent
> reordering according to the spec.

Yes, my understating is that you added the ACCESS_ONCE() to possibly
prevent the compiler from re-ordering the CPU_TO_X2APICID[ap_cpuid]) =
read_apic_id() after the 'hlt' loop?

AFAICT the expressions in the `for` statement are considered sequence
points, and the calling `read_apic_id()` could have side-effects, so
the compiler won't be able to elide or move the setting of
CPU_TO_X2APICID[] due to all side-effects of previous evaluations must
be complete at sequence points.

I'm fine with leaving the wmb() + ACCESS_ONCE().

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 27 08:55:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 08:55:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730648.1135795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBW95-0004ti-Sr; Mon, 27 May 2024 08:55:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730648.1135795; Mon, 27 May 2024 08:55:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBW95-0004tb-Px; Mon, 27 May 2024 08:55:55 +0000
Received: by outflank-mailman (input) for mailman id 730648;
 Mon, 27 May 2024 08:55:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0flO=M6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBW94-0004tV-Gg
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 08:55:54 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eba2da90-1c06-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 10:55:52 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4202c1d19d5so77046395e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 01:55:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100ee954bsm134581655e9.4.2024.05.27.01.55.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 01:55:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eba2da90-1c06-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716800151; x=1717404951; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Q09bIT5UaOFKGQ3FhV6lga2ObNJfZ+ZqQjmXP2nUV/w=;
        b=sP683EVaUXalBTlDGQ7aLiPQBLVSjzUAwHaKO6HVuMu2YCBF0pT2daMEYRhRbO1748
         YC83CYggGlTTJs/Ni8PFT7F5OkopvlR0UsKu3JgapQGSc4Kgv87lKXpSxaIpomq3iGZS
         VaIVfS1v1FNPtZwAIOHYlvBiywUd3NTBGrBJU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716800151; x=1717404951;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Q09bIT5UaOFKGQ3FhV6lga2ObNJfZ+ZqQjmXP2nUV/w=;
        b=U/OSgwUtpTVfYhBhQ1BAADyrL7tUakzDiwpPNC4iMzTbbOEKLuSp9nWBHNEJlhr+vK
         0rS3m2hXFR1XnBaEumuu75KjP5ohjzoui4/xujyUjXNfaf/G7eQLHfr/15aMLs19BrEK
         P4M4Y6IY4j/EcpwS6S2BWTEoJbXHpEEMYyqHMhaSksncAF039/Y6nRYfI5PtPvbhN93C
         Dcj3FhP9CdtsGrhScYr9roBY98Ick5um6mBwvFLygkQkaxYgdcIgr+ivxlabFw2+VumH
         t9AbVYXUMhmLAquShwFsPEZIt86eY2BoeSpQvuXStn6E5v2xQP1xSd0Ow6pviuDsaoeI
         5b6w==
X-Gm-Message-State: AOJu0YwynPTPFKCk/rBG10/EDo6xR+FICkQGHsVU+bSn66wn4tADA0H1
	3z+r08J2qb+dNKjzvk24FWE2Q/aPdjluK966hOJGj9m+G9bv02xjb/5gFrxgPEM=
X-Google-Smtp-Source: AGHT+IFK1cxfMvoevF0DZE6dn3cMZG1fA9lMLNH9Ow2yiZGbXjRmxpjrKtRaocvaIknAt36HjiKsUg==
X-Received: by 2002:a05:600c:4450:b0:420:2cbe:7f00 with SMTP id 5b1f17b1804b1-421089f98fdmr61159985e9.39.1716800151545;
        Mon, 27 May 2024 01:55:51 -0700 (PDT)
Date: Mon, 27 May 2024 10:55:50 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
Message-ID: <ZlRKljHLB_qBu4rP@macbook>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9rOqooa3PJCyw1@macbook>
 <d6ebb2f0-fb15-4ca4-9833-cd635511b2d3@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d6ebb2f0-fb15-4ca4-9833-cd635511b2d3@cloud.com>

On Fri, May 24, 2024 at 04:16:01PM +0100, Alejandro Vallejo wrote:
> On 23/05/2024 17:13, Roger Pau Monné wrote:
> > On Wed, May 08, 2024 at 01:39:24PM +0100, Alejandro Vallejo wrote:
> >> @@ -86,10 +113,11 @@ static void boot_cpu(unsigned int cpu)
> >>          BUG();
> >>  
> >>      /*
> >> -     * Wait for the secondary processor to complete initialisation.
> >> +     * Wait for the secondary processor to complete initialisation,
> >> +     * which is signaled by its x2APIC ID being writted to the LUT.
> >>       * Do not touch shared resources meanwhile.
> >>       */
> >> -    while ( !ap_callin )
> >> +    while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) )
> >>          cpu_relax();
> > 
> > As a further improvement, we could launch all APs in pararell, and use
> > a for loop to wait until all positions of the CPU_TO_X2APICID array
> > are set.
> 
> I thought about it, but then we'd need locking for the prints as well,
> or refactor things so only the BSP prints on success.

Hm, I see, yes, we would likely need to refactor the printing a bit so
each AP only prints one line, and then add locking around the calls in
`cpu_setup()`.

> The time taken is truly negligible, so I reckon it's better left for
> another patch.

Oh, indeed, sorry if I made it look it should be part of this patch,
that wasn't my intention.  Just something that might be worth looking
into doing in the future.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon May 27 10:27:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 10:27:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730656.1135806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXZg-0008Fa-AT; Mon, 27 May 2024 10:27:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730656.1135806; Mon, 27 May 2024 10:27:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXZg-0008FT-6z; Mon, 27 May 2024 10:27:28 +0000
Received: by outflank-mailman (input) for mailman id 730656;
 Mon, 27 May 2024 10:27:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=inlS=M6=epam.com=prvs=1877a19e80=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1sBXZe-0008FN-Fz
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 10:27:26 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b445abe9-1c13-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 12:27:24 +0200 (CEST)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44R8qwD9018387;
 Mon, 27 May 2024 10:27:13 GMT
Received: from eur01-db5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2050.outbound.protection.outlook.com [104.47.2.50])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ycq290fas-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 May 2024 10:27:12 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by AM0PR03MB6321.eurprd03.prod.outlook.com (2603:10a6:20b:157::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.16; Mon, 27 May
 2024 10:27:08 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Mon, 27 May 2024
 10:27:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b445abe9-1c13-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Tmz42Rh7jCnqz9RANnpXHWQJxJ7Cej+bGJ4DEsCE+J0eJTafCnTb3fdciIsfxTVXcON+JCp3RQJFroKJmMJYbatYtN7SLbI0dVsDF4hsHH6Raqdu4RMecZKRoK7Cx7zaWJOWUZ2fiVLB+y5FNzrQbnPCSeftM/h2oAzuNe91L8Wot4DY9rD5jGmWIctFA0vVpYRO4itU14srbyM7OK6hZ5VvIq2tma2kATpux3qJUfgNDbuQzJB9WQ4f5xlsvqiGIbYcDfrPTRoBwp0x+DwQDtBOdiSVF7bmGLwQzBLFuXYgqEDLKyqw4SHmPnScpc6gU92Ys4xkGl5Vfp2kaynrNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5ivWqJLAjJTPVyi8hEc+mEIZ0uZN56341fQFAnFC6go=;
 b=CoFpUmMWTOUqxo5FSkLy7fE23K4Esx554BhhtTh3ZYf1mBUUWWVmL/ETqMQfztr9CcqjFC2HjiCB5UlcZlP7o5YOX+RhqllwEv4t/f6aF9kg3Uaj9uaDrPp+Wa2iXn+axJ4bJmFfHudxYB2CXWSWDe3Qwj+seXxZr1m3c03EXFrHRcyMW2cHP43ot2WJz2Pr92AtyueqfEKrpcsTXJcq9L+vSKKd6wXPeFZ1mKjU29J+bYiBZCj3F74vuf3nIkqO1UtHx6F9UwQkVusfllDZt3nggZRjaPOy3EVEsEa8Wiyn5wQhQBj8IqPs8YMoUd7WgPYFKHxu+hQZomH2vIWDGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5ivWqJLAjJTPVyi8hEc+mEIZ0uZN56341fQFAnFC6go=;
 b=f41Npl6tuxHua3XyLp8timQUUM8giO+bj3iUtPAryXFHqlx4t+rzKoaqA6Fhh3fkMXqej5hDT5MMgmGAapL8PbqiOfl3FZwuYbEZ8Ng7QgJ9qv2UJHGi1RYxJ8QQD6JL8lViE8vNT5wCXYpACHWHHIEqmt4wjXjag7PVa5aCsAMwbpHv4/UNw44waMaB51bE4FEIoNoQkK5xek0uELuQjMn2dNAyb5k0Y/6Yu+vVQ43jqRADlkoBd4vBACvd+t0/Rp9hYaF/lArVKz6xLFhUpRblssXKhr0AbK9GFmIuCCpzdYZV5ALjS+9CatDhp5P0lGbeRaJpvbxPd25talcK7Q==
Message-ID: <0b2292da-a68f-4608-93a1-bbf54611e3e1@epam.com>
Date: Mon, 27 May 2024 13:27:05 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
        xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
 <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
 <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
 <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0015.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::13) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|AM0PR03MB6321:EE_
X-MS-Office365-Filtering-Correlation-Id: 5bbf770c-a652-4607-ba87-08dc7e379002
X-LD-Processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|376005|1800799015;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?ajFvN2JKVE1QcHVPc2htTEhzZkhKYS9xZDZoNlJPYURuNVJOZk5sWnYvdnYv?=
 =?utf-8?B?RDdtU1FmNUZSRk85Q255LzYwS1Z2M2cxbTQyQ2NzRmVTRTZyZ3M3b2M3Y3lu?=
 =?utf-8?B?K2hXTkR2ejBkVmRZUFNHZ1ZhODVaTGQvY1h6cXpBVEMrUmQ0MGcvMXkzUlJq?=
 =?utf-8?B?emM1SDNDZTdKcUo5U0ROallDQmYzQThTcCtHRHMxbkhuZ3NVNGwxMFk5cTZp?=
 =?utf-8?B?YnVqcndkZ2lwa25IbU1vSzQ4enRXcGVNM1Vla2t6UmxmS1Q4K0xCTlpVaVg5?=
 =?utf-8?B?czltTno0S0ZMa0RNQk5Zc2FpNWRGekJXVUlaNnp5TDZWUytNdkNFYmt5TS9Z?=
 =?utf-8?B?UVdJUktkUGs1eUl4SzcwTnFoK3VQbFAzVk52SWw4a2dzMHVDRVFEd0tQQUxL?=
 =?utf-8?B?amR5Z2FncWxmTzQrV0hIaUxDaHNrWDQ0dTlOcStqclhCNk9nYUFqaTVIOVhl?=
 =?utf-8?B?UXhFcXFIT1c2cFhXNWpnU3pGT05zaWNaYnVHejJxRnB4cmhhVUVOby9xUytl?=
 =?utf-8?B?QUpGSTZiNjd3UDU3bmdmcUFmYlJ3WDl5ZURhKzNIVjd4M2tUTEdLUUhOQW4v?=
 =?utf-8?B?cGpEb0xtOVc2SkdQQzlNYXBGOTJxNFFSUDgvQk81VnpuSXM3U1l6VzEyZGJk?=
 =?utf-8?B?Q2IrR01YYmplMklMWVpoWFdlNExvVUh0aklZQW9nZ25wWkVKSk5OUk1UOE5M?=
 =?utf-8?B?cVNPTGU0WUZ6MGxXNXZ3L2dic1lKSzVvWkdoMjVNV0tHelZmY1dFOEtIS3J0?=
 =?utf-8?B?QVhMM1JVa251Sis1cWZ1cWpHbDRIM003dWRRaFVEaDNoVG0wUnpTZ3RVZnhi?=
 =?utf-8?B?K0VFbGtoalBhZE9PNVgzcThGTjV6eWZHZDhwbDhtZnJSeWhZRG5YVEZqQU5O?=
 =?utf-8?B?NTZLbThMSmhubllacHM4RlQvZEExekRyZEZpTTM3UXRXRUY4clRxVkhiS1Jr?=
 =?utf-8?B?b0k5SFlxc1JrZE44cFQ0UmIzN2NVU1ZFWFlFTEc2UjVEa3F2SXRCcHlrdHlx?=
 =?utf-8?B?T2FPS2d3S25QRHlRZDFhMmpGd3oyekNrTU5LN0lNUFFKYmxOL2tZcWtaNHlh?=
 =?utf-8?B?Q0ozSGJHQVM0Vlk5MUtsVnBZK05FdHM1TkRDNDdHNXZNVktzTFRYKzR6cXhW?=
 =?utf-8?B?eUZIQ2p2ZC9qTk81UU9aaXBNK2kvNzJ2RXZUZVZaRk5qdU55azlrdlNMOTk2?=
 =?utf-8?B?aHZVOUVjOGg5bnhBV3A3Yjh3YXBGajVFSXV5RXJPUndMZ2dIdlY4eHJlZVln?=
 =?utf-8?B?dzdEcFBvWWx5ajh4QzM0bGxuRGRpbi9RYTFxMUlrUkFLK25hNlVRcks1a3ZP?=
 =?utf-8?B?MGxqWjlMcHFEOXlNMnJHWUtHd0psNkhUYVc1SkhDVEdjSUpNc3d2YzViVy9Y?=
 =?utf-8?B?cE5EMUZoUkZKQU1IbjJtN0ZXVEc4RytTRmFZQ1NBNWNOVTQ2elFrSk8wcUFD?=
 =?utf-8?B?S3ZTTTZwSkJQT1hWK0ZEalBVWGVtdmMzU0RRTnd1RGNBZ3JBS0RjcFdKdEhS?=
 =?utf-8?B?WldKQS9iM25IdTBGcHVCa2plRjQzQzRqcm5rWFd3MVAxVlcvRXpOWTRUUjhU?=
 =?utf-8?B?djU1N01oWVRIeFczSEJsMkoydENvTDNNNkJUM3c4c3FOT2lIdmFnTWM3RDhB?=
 =?utf-8?B?NWRhRmtFc3lyaU16anpuSEpURi8xOVBJSVlyb21LTkdrMEs3OFZuNE1ILzBv?=
 =?utf-8?B?c2FFRFVsSWt6b2E3UE5WNjg0TENUOUVYVHNwbGozNlhwUlptRTIrbE5BPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?a1pZcmE3M0JBdTBTNFQyeXpBbzE0b3kwenZkMDY3ZkRIcUlVNTVubHlzTWV3?=
 =?utf-8?B?TnBYSjM3M3VxSGJSbDd1ZjZ3WmdLWkluWi8zVU40V3JPcnRFVThZSDRxVTJz?=
 =?utf-8?B?V3QzYlIvNC9DVEFXRUpMbjdQMlpOSTYvUm5aTEdheVFybmhJYlJlYTZ0SzZp?=
 =?utf-8?B?eXcwVmhPS0hUdUlsU2E2eEJDemUrSzNMYzlyY3YzMFdCeGFGaTJUNDZyZ2JX?=
 =?utf-8?B?UWZyeXB2MEhHWUtTbVYxeTdvMlZNL1orY0JnRXZ6QyswU05qRkpsZGRaNDZU?=
 =?utf-8?B?MG52Q1hxVm5tL3dRTHNUNmZsaU42YUE2dXp6Mmc5cnhOWXZOK1YreU1zSFJ2?=
 =?utf-8?B?TmtwOU1RN1ZCcFhjem5WNisxNEhsRDFZZjYxWXZDTnp5MlQxTjYrczB4YW54?=
 =?utf-8?B?d3dJUTRSbklrMVVsMWIybHl6ZjU0b09PNUdESmI4eVRrNVgxQzdRKzhvVmZ3?=
 =?utf-8?B?YTBBZU00NmxaWTFwK2ZHK1FHT2NhY09uUVh5YWpaR01qZmh1dEIrMkhTOFFJ?=
 =?utf-8?B?L3g0SzNoYk5OdHdMMXBKUHhqQ1RCQ3ZJQ3IxdmZXMU1HVUxRZnZ5STZOSlds?=
 =?utf-8?B?bVhFR0M1Yks1RlhzMjZ5NHVPaEVMd255NktVc1BiSGYrbzRHbm5Rc09PQzBY?=
 =?utf-8?B?OEJiRWVva3JUblVwd1prTjlhZTh2Tzg3RkRMZnovQzg2Y3JEZk92L2FtdzNs?=
 =?utf-8?B?OVk2RGxVZUVOZStmODlLeUkxaThhYWdLR2hwaWJkT2wydUsxcEdPeGphTkVr?=
 =?utf-8?B?NTU0eU1GOFE4OHhYaXNzMmdaTnV4VW42U3J4UVNlSlZEQlNhb1ZWVmJnMkc2?=
 =?utf-8?B?dDhxM2Z3WG9Jb250QUJkZkpSREhOQjdSNHlaQmtyY0Q3TUEzVzluOUI0WkRk?=
 =?utf-8?B?cEpNNENZeVI5WklTTzB1cmpwMnZQK1ltdWQvQytpTmYya04yNEZZcEJ1V2Z6?=
 =?utf-8?B?N3VYOXNGbkFmUGVqVE8reDJwQlVQMGMvS2lrWFA5K3FoaHNXTDN4cVZaQTRJ?=
 =?utf-8?B?SGtGQ3hjVzJQMG9seUVYR1ZBeTF5cXVLMHk2eDVNWXY4emhKM0JwN0s4azlk?=
 =?utf-8?B?SmY5MTJNazBZc2h0UGRKUkh6OVdIVU5JTHFBby9ybHRkSGVjM1VyK1BNV0tq?=
 =?utf-8?B?a1ppOC9qcVFHRVY0ZUhpQ2hoVi9VUDloaHNvcXBKZ2szQkl6cnZTeGxBNGVj?=
 =?utf-8?B?RUZ2SXU3aFBHdXZWZkRRU0pRUC8zV2p2ZXluUHhTY0ZncThnK08yNEdNdXcz?=
 =?utf-8?B?RXBtSHVvOWlyQTJuVlYxL3BsdmI4VDZjWi9ZKzE4TDJaZUtkaWdPZUVkVnFv?=
 =?utf-8?B?RFlvek1qY29lelJiYWNRbWxwWFk1N3ZDNXlValh4M09QRFg1Q0xZTDBUL1g2?=
 =?utf-8?B?b3Q1bUZ1WVlLOWd2RGZUVEVqajNUdloxNTBZSEVrd2dHdzdYZnhRVUwzWVQ2?=
 =?utf-8?B?Z2g2OXJwWkhPSmpicytIZEFCTGthUHhtZUFyYlRFN1RVbm45ODN2UXpRUDJr?=
 =?utf-8?B?SktZSU5UR2ZEQkxuRG5aTU9OUHRCQWtseStuM3JTY3N5elV0eXZGMk44b1JN?=
 =?utf-8?B?NWNCY3BwcXVWUmpFZGFnclRmWWZvaDYyYWZZL0h1YTRpcXBOWmsrMFpHMFRF?=
 =?utf-8?B?Q1EyUmRkUy9jSDRwRXVjcmFzTlo1ZUUwT1hWYmhLZ1U3aENZS1JraHdYNGpx?=
 =?utf-8?B?VEJaRHluWGhjTStHQUhaRERBRS9VZGFHaFc1cDJXSUNSRThFMmNvcFZHL3N3?=
 =?utf-8?B?TS8vejRITkgrTjYzYU5Pb1pjQWJCckoxSll5M2V1R2dxdmVoZlliQUtNWjhu?=
 =?utf-8?B?K1lKYUZKNTNIampaQ05Pa3QrOGsyaTBSZStKMWUrWGZicmtpNGRjb3FyQ1gr?=
 =?utf-8?B?Z29iK0gzRHdhZzdxQWRRdnhRRktXQ1lBK3lVNWhucFBSdnBLeFpzcUhQdHox?=
 =?utf-8?B?OGc5RXIzdEFYSkNZRG1PNmRSV2ZvK1h0cEpvRXdQcFArY1JubGlHZW5BZEhq?=
 =?utf-8?B?UVVsOEI0U3htUEVwc01lOE1TNnpQVjZ0VjI1L2I0c1FucUEyQStVc3VBd0ZC?=
 =?utf-8?B?cjNLcEgvOUk1U0VJYnAvdDVnekNmTUxhdGI4SFdFUWczT1BtMnNxdG5PaHUx?=
 =?utf-8?B?a2pRMHJtQThPT0RRQmp1WFNUVDFSK2dmTkhBOXliTUdQQ2xGcGZaU1lHRDhq?=
 =?utf-8?B?cVE9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5bbf770c-a652-4607-ba87-08dc7e379002
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2024 10:27:08.5236
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Olc6Pqz3HWBlNhPdKOm1I80gB7g/2cSHpyB7X3SQ1Du1dWgp1oj1bqjYnqWHz6FQSQymscC0B/qsKZFmwmWnXw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6321
X-Proofpoint-GUID: 4jSKZTqr4h-jQFwJXOgSaYmTpaXPMjv1
X-Proofpoint-ORIG-GUID: 4jSKZTqr4h-jQFwJXOgSaYmTpaXPMjv1
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-26_09,2024-05-24_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 clxscore=1015 impostorscore=0 mlxscore=0 mlxlogscore=999
 priorityscore=1501 phishscore=0 malwarescore=0 bulkscore=0 adultscore=0
 suspectscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405170001 definitions=main-2405270085

23.05.24 17:50, Jan Beulich:
> On 23.05.2024 15:07, Sergiy Kibrik wrote:
>> 16.05.24 14:12, Jan Beulich:
>>> On 15.05.2024 11:12, Sergiy Kibrik wrote:
>>>> --- a/xen/arch/x86/include/asm/cpufeature.h
>>>> +++ b/xen/arch/x86/include/asm/cpufeature.h
>>>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>    #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>>>>    #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>>>>    #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
>>>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
>>>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
>>>> +                                  boot_cpu_has(X86_FEATURE_VMX))
>>>>    #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>>>>    #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>>>>    #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
>>>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>    
>>>>    /* CPUID level 0x80000001.ecx */
>>>>    #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
>>>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
>>>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
>>>> +                                  boot_cpu_has(X86_FEATURE_SVM))
>>>>    #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>>>>    #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>>>>    #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
>>>
>>> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
>>> and as a result one-off indentation on the wrapped lines) I'm not really
>>> certain we can do this. The description goes into detail why we would want
>>> this, but it doesn't cover at all why it is safe for all present (and
>>> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
>>> just to derive further knowledge from that, without them being directly
>>> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
>>> for example. While it looks to be okay there, it may give you an idea of
>>> what I mean.
>>>
>>> Things might become better separated if instead for such checks we used
>>> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
>>> that's still pretty far out, I'm afraid.
>>
>> I've followed a suggestion you made for patch in previous series:
>>
[..]
> 
> See the "If not, ..." that I had put there. Doing the change just mechanically
> isn't enough, you also need to make clear (in the description) that you
> verified it's safe to have this way.
> 
>> yet if this approach can potentially be unsafe (I'm not completely sure
>> it's safe), should we instead fallback to the way it was done in v1
>> series? I.e. guard calls to vmx/svm-specific calls where needed, like in
>> these 3 patches:
>>
[..]
> 
> I don't like this sprinkling around of IS_ENABLED() very much. Maybe we want
> to have two new helpers (say using_svm() and using_vmx()), to be used in place
> of most but possibly not all cpu_has_{svm,vmx}? Doing such a transformation
> would then kind of implicitly answer the safety question above, as at every
> use site you'd need to judge whether the replacement is correct. If it's
> correct everywhere, the construct(s) as proposed in this version could then be
> considered to be used in this very shape (instead of introducing the two new
> helpers). But of course the transition could also be done gradually then,
> touching only those uses that previously you touched in 1), 2), and 3).
> 

now I might be seeing your concerns, if I understood correctly, 
situation is the following.

  As an example of cpu_has_vmx macro, it can be used to prove either of 
following two statements: 1) VMX features can be used or 2) CPU provides 
VMX features.
Currently they're the same for Xen, yet after this patch series they're 
not, as the situation possible when non-vmx build would be able to get 
executed on vmx-enabled machine. E.g. the case of PV guest, or (if that 
makes any sense) at least hypervisor's code is still able to run until 
HVM guest has to be created. Changes in this patch makes 
indistinguishable for a user whether VMX support is absent in code or in 
hardware -- hence we may need two separate macros for these.

Still the question remains whether a separate macro to check if CPU 
provides VMX/SVM is really needed at all at this point.

I've counted only 16 uses of cpu_has_vmx in the code, not that much to 
check every one of them, so I did that.
Most of uses are obviously checks before using vmx features, so logic 
not broken.
As for the others, the surrounding context presumes that HVM domain 
required there or had already been created. But non-vmx build can't 
create HVM VMX domain anyway, so the logic not broken either.

As for cpu_has_svm only 8 uses I've counted, all but one also don't seem 
to break logic as described above. One check of cpu_has_svm in 
init_speculation_mitigations(), where default speculation control flag 
gets set, not uses SVM features directly. Yet from the comment I can 
assume that it's also related to running HVM domain and usage of VMX 
features at later time.

With all above, at the moment there doesn't seem to be uses of 
cpu_has_{svm,vmx} macros without subsequent usage of svm/vmx features, 
so this patch should be quite safe.

Please let me know whether the above reasoning makes sense.

  -Sergiy


From xen-devel-bounces@lists.xenproject.org Mon May 27 10:43:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 10:43:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730663.1135815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXod-0002dG-Lt; Mon, 27 May 2024 10:42:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730663.1135815; Mon, 27 May 2024 10:42:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXod-0002d9-JK; Mon, 27 May 2024 10:42:55 +0000
Received: by outflank-mailman (input) for mailman id 730663;
 Mon, 27 May 2024 10:42:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBXoc-0002cz-Ev; Mon, 27 May 2024 10:42:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBXoc-0001AR-Ao; Mon, 27 May 2024 10:42:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBXoc-0002QS-1J; Mon, 27 May 2024 10:42:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBXoc-00035H-0v; Mon, 27 May 2024 10:42:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=D330lUr5rYXVMHKFMrT7Zfh0E+pmjzeJAo8k/8Jr+vk=; b=zwbcP0kOld96hJls1ulNnrKGIZ
	g94Hk98aCc4o0uwAE+sYHq9NgFFCF+gYxV5iNuopKOe4LKEyYKn9WDoZ+AAbaLduVkNUa9ByDyEaU
	YF6oeo1s+vZ1N/HZYaLHhodQpnj47M8zFzBPy8O49icf6vBwhUEmn8edA6LdwmBrvEm8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186157-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186157: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-armhf:xen-build:fail:regression
    xen-unstable:build-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
X-Osstest-Versions-That:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 10:42:54 +0000

flight 186157 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186157/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 186145

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186151
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186151
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186151
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186151
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186151
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0
baseline version:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0

Last test of basis   186157  2024-05-27 01:53:44 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  fail    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    blocked 
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      blocked 
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon May 27 10:49:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 10:49:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730670.1135825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXur-0003ER-Af; Mon, 27 May 2024 10:49:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730670.1135825; Mon, 27 May 2024 10:49:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBXur-0003EK-87; Mon, 27 May 2024 10:49:21 +0000
Received: by outflank-mailman (input) for mailman id 730670;
 Mon, 27 May 2024 10:49:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBXuq-0003EE-6x
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 10:49:20 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c4d662cd-1c16-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 12:49:19 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a61b70394c0so903218666b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 03:49:19 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c818362sm479015566b.34.2024.05.27.03.49.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 03:49:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c4d662cd-1c16-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716806958; x=1717411758; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8dvO+/8haWXIPjweNYb9W8ujvzQUrct9skr+FbumigQ=;
        b=FAtvIyG8OijPeGZR2vHm70j91oPDwAzHjUwMrh+DCnThIaAcBn2z1/LTG0ekFMnofX
         qe8DFk9YlPuQFTo4IqHkON3l6ENbqIChL3yZ4yRx0zdkJjlLUKOJAQS5lFxWuy1+GvKJ
         3A7dRCvenmQgqeFXLp251CnRzAXNDLiK2EZD8T8xyEYntowpdmk6b/zgj2jZoHM6n7AZ
         3h1AdqOIN47acPfmZvz7E+Lmv2Oxyn0UHSrknBfrsKCS9kQnxKIlbj4dQ5CCjI3OXOvu
         ZMlciwGw8aHWmfMWx1umgKFldrp49vjftC4WiWQO53Cd2iA9xhg+OeITtEey3cuLi3kw
         pV0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716806958; x=1717411758;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8dvO+/8haWXIPjweNYb9W8ujvzQUrct9skr+FbumigQ=;
        b=J4FcEvEg3ZwA43aHxaczfRHee3hKLllCqO1k/7mg+cudfQL0dOnCbgKxHPYOFLnRWL
         f6R3VBmcTGZoOPlbSlF0KEBhEKE5wevu/PPBzhHJFsrBXDbCtyCwPOTV8l9XNAIYEgb5
         xulY+y70DHmPLDexLvKPXvWAzUb2E/BhdHWHyTvbS6JRpKyoQ0as2MFIquSnTjDXJjVw
         A2iGwXKgNsc3S2P/r6FxNUav5k4MbFOilqgkp6/W517my9z3wTHGH6vf4ZsXu1XtgGcW
         tNIlL+br2x4gsGrDel+SAN55g3VNF3Jkh4j1yfNkViEm+m5UcH2Kp6yxjMzn8i0LIAGm
         za1w==
X-Forwarded-Encrypted: i=1; AJvYcCUMG3Nc4WFJTP3sc1myx+ypxclkfQJUW9Ar7Pj9l+3iyiTupLKaY6bMC877vCmeHsQ5qvWMf7pgCE9BYmN5o5GQ/njFqZqwqLQh0PRkiWY=
X-Gm-Message-State: AOJu0Yz58NCUXVANHftC7L4xhkFygwlwqnZDY99o2ZRachJbc0TdDH4g
	u4Ku9MDfhrIrAbUhPJ0MU2h5lSl4T5pMjKRKPuNxMoQjm1Mwk1mn4erYC0omqcehAh88W7J8CjI
	=
X-Google-Smtp-Source: AGHT+IFNPnrVTZsmXBSmjLfqrz6gNHNFefuMq9DDR/uKlpOFez43acCnEwchlG3eOb3DkfQP04jX0w==
X-Received: by 2002:a17:906:2890:b0:a62:2ef9:13d with SMTP id a640c23a62f3a-a626308f5cbmr602170466b.0.1716806958281;
        Mon, 27 May 2024 03:49:18 -0700 (PDT)
Message-ID: <a695f51e-aaaf-4132-bbf5-d7d108cdc37a@suse.com>
Date: Mon, 27 May 2024 12:49:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 07/15] x86: guard cpu_has_{svm/vmx} macros with
 CONFIG_{SVM/VMX}
Content-Language: en-US
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <09f1336974c8fd2f788fe8e1d3ca5fee91da5a81.1715761386.git.Sergiy_Kibrik@epam.com>
 <a881c6a6-2c36-4e5c-8336-21cd0e14b873@suse.com>
 <4b02bf84-0d81-467e-8577-f33dbfb71e8f@epam.com>
 <201c4e7f-c493-4156-a855-b2207107e6c4@suse.com>
 <0b2292da-a68f-4608-93a1-bbf54611e3e1@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0b2292da-a68f-4608-93a1-bbf54611e3e1@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 12:27, Sergiy Kibrik wrote:
> 23.05.24 17:50, Jan Beulich:
>> On 23.05.2024 15:07, Sergiy Kibrik wrote:
>>> 16.05.24 14:12, Jan Beulich:
>>>> On 15.05.2024 11:12, Sergiy Kibrik wrote:
>>>>> --- a/xen/arch/x86/include/asm/cpufeature.h
>>>>> +++ b/xen/arch/x86/include/asm/cpufeature.h
>>>>> @@ -81,7 +81,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>>    #define cpu_has_sse3            boot_cpu_has(X86_FEATURE_SSE3)
>>>>>    #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
>>>>>    #define cpu_has_monitor         boot_cpu_has(X86_FEATURE_MONITOR)
>>>>> -#define cpu_has_vmx             boot_cpu_has(X86_FEATURE_VMX)
>>>>> +#define cpu_has_vmx             ( IS_ENABLED(CONFIG_VMX) && \
>>>>> +                                  boot_cpu_has(X86_FEATURE_VMX))
>>>>>    #define cpu_has_eist            boot_cpu_has(X86_FEATURE_EIST)
>>>>>    #define cpu_has_ssse3           boot_cpu_has(X86_FEATURE_SSSE3)
>>>>>    #define cpu_has_fma             boot_cpu_has(X86_FEATURE_FMA)
>>>>> @@ -109,7 +110,8 @@ static inline bool boot_cpu_has(unsigned int feat)
>>>>>    
>>>>>    /* CPUID level 0x80000001.ecx */
>>>>>    #define cpu_has_cmp_legacy      boot_cpu_has(X86_FEATURE_CMP_LEGACY)
>>>>> -#define cpu_has_svm             boot_cpu_has(X86_FEATURE_SVM)
>>>>> +#define cpu_has_svm             ( IS_ENABLED(CONFIG_SVM) && \
>>>>> +                                  boot_cpu_has(X86_FEATURE_SVM))
>>>>>    #define cpu_has_sse4a           boot_cpu_has(X86_FEATURE_SSE4A)
>>>>>    #define cpu_has_xop             boot_cpu_has(X86_FEATURE_XOP)
>>>>>    #define cpu_has_skinit          boot_cpu_has(X86_FEATURE_SKINIT)
>>>>
>>>> Hmm, leaving aside the style issue (stray blanks after opening parentheses,
>>>> and as a result one-off indentation on the wrapped lines) I'm not really
>>>> certain we can do this. The description goes into detail why we would want
>>>> this, but it doesn't cover at all why it is safe for all present (and
>>>> ideally also future) uses. I wouldn't be surprised if we had VMX/SVM checks
>>>> just to derive further knowledge from that, without them being directly
>>>> related to the use of VMX/SVM. Take a look at calculate_hvm_max_policy(),
>>>> for example. While it looks to be okay there, it may give you an idea of
>>>> what I mean.
>>>>
>>>> Things might become better separated if instead for such checks we used
>>>> host and raw CPU policies instead of cpuinfo_x86.x86_capability[]. But
>>>> that's still pretty far out, I'm afraid.
>>>
>>> I've followed a suggestion you made for patch in previous series:
>>>
> [..]
>>
>> See the "If not, ..." that I had put there. Doing the change just mechanically
>> isn't enough, you also need to make clear (in the description) that you
>> verified it's safe to have this way.
>>
>>> yet if this approach can potentially be unsafe (I'm not completely sure
>>> it's safe), should we instead fallback to the way it was done in v1
>>> series? I.e. guard calls to vmx/svm-specific calls where needed, like in
>>> these 3 patches:
>>>
> [..]
>>
>> I don't like this sprinkling around of IS_ENABLED() very much. Maybe we want
>> to have two new helpers (say using_svm() and using_vmx()), to be used in place
>> of most but possibly not all cpu_has_{svm,vmx}? Doing such a transformation
>> would then kind of implicitly answer the safety question above, as at every
>> use site you'd need to judge whether the replacement is correct. If it's
>> correct everywhere, the construct(s) as proposed in this version could then be
>> considered to be used in this very shape (instead of introducing the two new
>> helpers). But of course the transition could also be done gradually then,
>> touching only those uses that previously you touched in 1), 2), and 3).
>>
> 
> now I might be seeing your concerns, if I understood correctly, 
> situation is the following.
> 
>   As an example of cpu_has_vmx macro, it can be used to prove either of 
> following two statements: 1) VMX features can be used or 2) CPU provides 
> VMX features.
> Currently they're the same for Xen, yet after this patch series they're 
> not, as the situation possible when non-vmx build would be able to get 
> executed on vmx-enabled machine. E.g. the case of PV guest, or (if that 
> makes any sense) at least hypervisor's code is still able to run until 
> HVM guest has to be created. Changes in this patch makes 
> indistinguishable for a user whether VMX support is absent in code or in 
> hardware -- hence we may need two separate macros for these.
> 
> Still the question remains whether a separate macro to check if CPU 
> provides VMX/SVM is really needed at all at this point.
> 
> I've counted only 16 uses of cpu_has_vmx in the code, not that much to 
> check every one of them, so I did that.
> Most of uses are obviously checks before using vmx features, so logic 
> not broken.
> As for the others, the surrounding context presumes that HVM domain 
> required there or had already been created. But non-vmx build can't 
> create HVM VMX domain anyway, so the logic not broken either.
> 
> As for cpu_has_svm only 8 uses I've counted, all but one also don't seem 
> to break logic as described above. One check of cpu_has_svm in 
> init_speculation_mitigations(), where default speculation control flag 
> gets set, not uses SVM features directly. Yet from the comment I can 
> assume that it's also related to running HVM domain and usage of VMX 
> features at later time.

Note how to comment (necessarily) mixes SVM and VT-x. The !cpu_have_svm
really looks to mean cpu_has_vmx there (i.e. absence of SVM on a HVM-only
path implying VT-x). This would be broken if cpu_has_svm had
IS_ENABLED() added to it. You'll want to consult the commit introducing
the construct as well as perhaps Andrew directly as to whether switching
to cpu_has_vmx here would be appropriate. I'm pretty sure there was a
reason why this was written using the negative (!SVM) form.

> With all above, at the moment there doesn't seem to be uses of 
> cpu_has_{svm,vmx} macros without subsequent usage of svm/vmx features, 
> so this patch should be quite safe.
> 
> Please let me know whether the above reasoning makes sense.

Fundamentally it does, with said one exception. Yet even in the absence
of that exception the question would then remain whether we really want
to close the road to one of the two possible uses of these constructs.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 10:58:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 10:58:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730682.1135841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBY3a-0005C8-6r; Mon, 27 May 2024 10:58:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730682.1135841; Mon, 27 May 2024 10:58:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBY3a-0005C0-3q; Mon, 27 May 2024 10:58:22 +0000
Received: by outflank-mailman (input) for mailman id 730682;
 Mon, 27 May 2024 10:58:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=9k1s=M6=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sBY3Y-0005Bm-Qh
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 10:58:20 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06609585-1c18-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 12:58:18 +0200 (CEST)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B5BA821B1B;
 Mon, 27 May 2024 10:58:17 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 83C3713A6B;
 Mon, 27 May 2024 10:58:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /1KJHklnVGbEAwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 27 May 2024 10:58:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06609585-1c18-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716807497; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=OUaamV24W6oWhF4J0lRbAl4NhZatFacXTrb9FeqRlLc=;
	b=m0/klhO6ex5yNmvPBdXsTDgQTvyqY4Mz0Ml1rWgfBnUEtwaGsJjwkYlaIgBVa6Bxzn0+Lq
	rKjsCV7xjZLaZkXdy/QuJ591/9IE82kMmVm1+6lJgx3lyINQtO9xOqRYwUsdi4aIwiztXV
	FKH4hmcHg6lc7UMel3+3UiehvVNiw+w=
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="m0/klhO6"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1716807497; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=OUaamV24W6oWhF4J0lRbAl4NhZatFacXTrb9FeqRlLc=;
	b=m0/klhO6ex5yNmvPBdXsTDgQTvyqY4Mz0Ml1rWgfBnUEtwaGsJjwkYlaIgBVa6Bxzn0+Lq
	rKjsCV7xjZLaZkXdy/QuJ591/9IE82kMmVm1+6lJgx3lyINQtO9xOqRYwUsdi4aIwiztXV
	FKH4hmcHg6lc7UMel3+3UiehvVNiw+w=
Message-ID: <9fba239d-2e22-425b-b45f-86ecf016c2fd@suse.com>
Date: Mon, 27 May 2024 12:58:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Juergen Gross <jgross@suse.com>
Subject: Re: CVE-2021-47377: kernel: xen/balloon: use a kernel thread instead
 a workqueue
To: gregkh@linuxfoundation.org
Cc: cve@kernel.org, linux-cve-announce@vger.kernel.org,
 linux-kernel@vger.kernel.org,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Xen.org security team" <security@xen.org>
References: <2024041747-CVE-2024-26908-4690@gregkh>
Content-Language: en-US
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <2024041747-CVE-2024-26908-4690@gregkh>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------dVaJww6PxA11a6ZZ7Kz9oRix"
X-Spam-Level: 
X-Spamd-Result: default: False [-7.20 / 50.00];
	BAYES_HAM(-2.80)[99.15%];
	SIGNED_PGP(-2.00)[];
	DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	MIME_BASE64_TEXT_BOGUS(1.00)[];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	NEURAL_HAM_SHORT(-0.20)[-0.992];
	MIME_UNKNOWN(0.10)[application/pgp-keys];
	MIME_BASE64_TEXT(0.10)[];
	MX_GOOD(-0.01)[];
	XM_UA_NO_VERSION(0.01)[];
	MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	TO_DN_SOME(0.00)[];
	DKIM_SIGNED(0.00)[suse.com:s=susede1];
	TO_DN_EQ_ADDR_SOME(0.00)[];
	ARC_NA(0.00)[];
	FUZZY_BLOCKED(0.00)[rspamd.com];
	RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	FROM_HAS_DN(0.00)[];
	RCPT_COUNT_FIVE(0.00)[6];
	DNSWL_BLOCKED(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received];
	DKIM_TRACE(0.00)[suse.com:+];
	HAS_ATTACHMENT(0.00)[];
	DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim]
X-Rspamd-Action: no action
X-Rspamd-Queue-Id: B5BA821B1B
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Flag: NO
X-Spam-Score: -7.20

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------dVaJww6PxA11a6ZZ7Kz9oRix
Content-Type: multipart/mixed; boundary="------------G3ihLASy2vtu3dMCGzkOqHQy";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: gregkh@linuxfoundation.org
Cc: cve@kernel.org, linux-cve-announce@vger.kernel.org,
 linux-kernel@vger.kernel.org,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Xen.org security team" <security@xen.org>
Message-ID: <9fba239d-2e22-425b-b45f-86ecf016c2fd@suse.com>
Subject: Re: CVE-2021-47377: kernel: xen/balloon: use a kernel thread instead
 a workqueue
References: <2024041747-CVE-2024-26908-4690@gregkh>
In-Reply-To: <2024041747-CVE-2024-26908-4690@gregkh>
Autocrypt-Gossip: addr=security@xen.org; keydata=
 xsBNBE+hNqgBCADYua5OFR0/Jeu0rByk+Obk6+SewIeGej1FAcjo+Cvpcr1dfnLBAhmmhbfM
 b++qr6SG6Ek+cUQogYAFvZcEcusbRPy4MIzJkqoPSyOUhCxZoxWNWUfhDdt0TWA3Hs1vYmFO
 e+2jvlL3h7yAsGMYO8jo6ow8ceBEOmf8Q5BLq2OPkNpGcaHEhbSv0VZ3mdHM30ynY6GubIws
 c68LZ5hTORTSjKaj2WVCe4OorBMZte5Im+6MOEUbCjynqPJSU9KNFhIhUuyXp1vn0gZ2N5QS
 pkghpzBJLzeBNEI6ecV3Q0p+/pq8EvEAuUSNLUEbIZ/NSLqyTVMc9HZxnPu59im8wB9rABEB
 AAHNK1hlbi5vcmcgKGluY29taW5nIGVtYWlsKSA8c2VjdXJpdHlAeGVuLm9yZz7CwHgEEwEC
 ACIFAk+hNqgCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEHQ6P8qC06lk1y0H/2Pj
 jQyPDZVS4zIVnR4xQOQ1KphPCdSTPlhj+VVrjZZNXWGCUKvJShL84XIONH62fIgQE/6CTWXJ
 tx6i4u1oAtFH4+8HayFjg609lxx9frJ4tJkJitw5TT6VEGAambchIG5QaP9hepgyrVXjQ0X2
 ot0jgpwL6G3sx0L1gewiMALXtGT6oTqLjXius/nv69yRe26wxU1GX80oWWH/5p585xt54C1X
 nhDEVzp0S9UW7VAAVDCWuSefSrihh3jZi4QE1fnGRwO0RfeLh1sXeuMn9uFIz0CmaCbAp5Pe
 UyNb6wgG60h4JLCDyhJntoHfq8pQLEJ8G9nvjDfw8BLvkBKYNvbOwE0ET6E2qAEIALqWNlGF
 d3uIj+DXZ40/i7fsoPb+HaYaG6Y+7+ZWxMxUeQDTLBnTYiAa+EGVutc4v52BXH8RZc9I/NH9
 lBT2/AwaEVSomxLicbixXUGoFC9kMp/VP1xwWJ+gm+ZEnQzY+2AFJGMvqEsGocQA7yLw121J
 UOrorny3CqpHykPUF3fqp4n/GL47VTaKxlsoV8o2JgZZ62NJlkBtnbA4ODzhWr6cA21smWFg
 sfFJ+EkXb1NEeYLs8CWtTn2EiQXlZTQ8OgBPahfvLZ+AJ4sM/Raoi2c3UIQrlCsg9BoojKMk
 Li8XUrywr8HEJYjhBYObCgbmaeIEfmrw5XJqOKlMg40XY+MAEQEAAcLAXwQYAQIACQUCT6E2
 qAIbDAAKCRB0Oj/KgtOpZDhJB/0XtxrlVuRttpjK1PEYK/A/9h47VH9p0UvVYCH+ZS2a+sTg
 sapx0zp4uni8wtytkvGw/EM06D4ZoaWAUcjXILNKGdi62q/z+WAfdEY/WrONxAbr2Dtv/LT0
 0/2nifYU9O1vGYS1Kx/B3D8fU0w+2Sjv+hYjbGDWn619etC8dNEIxczH6V/cVOZf0D2KhoBf
 MCHUoKeuAfaIKDMxOZjb7sajfUW70cxFFWYqH96Py01oxDroOKzy0x62iVdsYFGB3FvcD9tD
 WsxVWwGHA8DKEfKMuNPiuapzdxdrNm5AQilSUlfD65KK9d3kQdoOUPdPWoIQnz8GnHMPDe99
 7SuwxWGb

--------------G3ihLASy2vtu3dMCGzkOqHQy
Content-Type: multipart/mixed; boundary="------------emd4kqbqlUBoHXDuj7s7yzqQ"

--------------emd4kqbqlUBoHXDuj7s7yzqQ
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

SGksDQoNCkknZCBsaWtlIHRvIGRpc3B1dGUgQ1ZFLTIwMjEtNDczNzc6IHRoZSBpc3N1ZSBm
aXhlZCBieSB1cHN0cmVhbSBjb21taXQNCjg0ODBlZDljMmJiZDU2ZmM4NjUyNDk5OGU1ZjJl
M2UyMmY1MDM4ZjYgY2FuIGluIG5vIHdheSBiZSB0cmlnZ2VyZWQgYnkNCmFuIHVucHJpdmls
ZWdlZCB1c2VyIG9yIGJ5IGEgcmVtb3RlIGF0dGFjayBvZiB0aGUgc3lzdGVtLCBhcyBpdCBy
ZXF1aXJlcw0KaW5pdGlhdGlvbiBvZiBtZW1vcnkgYmFsbG9vbmluZyBvZiB0aGUgcnVubmlu
ZyBzeXN0ZW0uIFRoaXMgY2FuIGJlIGRvbmUNCm9ubHkgYnkgZWl0aGVyIGEgaG9zdCBhZG1p
biBvciBieSBhbiBhZG1pbiBvZiB0aGUgZ3Vlc3Qgd2hpY2ggbWlnaHQNCnN1ZmZlciB0aGUg
ZGV0ZWN0aW9uIG9mIHRoZSBoYW5naW5nIHdvcmtxdWV1ZS4NCg0KUGxlYXNlIHJldm9rZSB0
aGlzIENWRS4NCg0KDQpKdWVyZ2VuDQo=
--------------emd4kqbqlUBoHXDuj7s7yzqQ
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----


--------------emd4kqbqlUBoHXDuj7s7yzqQ--

--------------G3ihLASy2vtu3dMCGzkOqHQy--

--------------dVaJww6PxA11a6ZZ7Kz9oRix
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmZUZ0kFAwAAAAAACgkQsN6d1ii/Ey9F
GQf9FgVPyzTFKIgeWrwPDxN9H9ZslHo23VxxgV3zePI48TpLefea4JqyovD7dc/BBWK5FW/Fre2N
s5pRkzPERlvTXx8HqvIEdIAAOYx8j1MZPRUMPwZ6YOb+4o6LtE0XxRXHnTlnNOBbV/L9W3R9Nu13
ZnQVqOfy+GXrwoD/K2Ja/gh8jRtZUEriAKH3x7oJIDuyD4mpaLK36tqLq1b/cKca4tYfY+wzoX1i
/Lf2+M51VWLLL0qJl1pk2wUnXej7BnpGit5f4I5qDiiAY6Kuh1CknXadw7r4XWYuWNcm2eVOlJ5e
h2qRzZ+V42KALNAs9bXpebMhKVDQnEfgrRAxU+03tQ==
=FxZe
-----END PGP SIGNATURE-----

--------------dVaJww6PxA11a6ZZ7Kz9oRix--


From xen-devel-bounces@lists.xenproject.org Mon May 27 11:35:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 11:35:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730694.1135851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBYd5-0002HW-37; Mon, 27 May 2024 11:35:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730694.1135851; Mon, 27 May 2024 11:35:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBYd4-0002HP-WE; Mon, 27 May 2024 11:35:03 +0000
Received: by outflank-mailman (input) for mailman id 730694;
 Mon, 27 May 2024 11:35:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBYd3-0002HF-Mx; Mon, 27 May 2024 11:35:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBYd3-0002Po-LK; Mon, 27 May 2024 11:35:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBYd3-0003sl-9y; Mon, 27 May 2024 11:35:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBYd3-0003Rd-9M; Mon, 27 May 2024 11:35:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6e8NIsWcB0oDnMEEAJUcw5Za6nRD2KdlyX/rOwZeCWA=; b=xW2J31IxHjeognIaH0CG9OI0a0
	G2cmwwsQbErtOPHh4uDU6NYNlVl+0oHfU4g6tojxStvo8J8hWouwSGT+BbRWMZwKAtvVEJA/poMP+
	U6zhIiHSnaUKWp5qAIjuVjtYv6q0acOEE2QNdHIzVR68hPnD/vXWYubd/0niuVntkeLM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186160-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186160: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=08281572aab5b1f7e05bf26de4148af19eddc8b7
X-Osstest-Versions-That:
    ovmf=88a4de450f17c6a319c3e8b2135cd7068a07d0f8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 11:35:01 +0000

flight 186160 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186160/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 08281572aab5b1f7e05bf26de4148af19eddc8b7
baseline version:
 ovmf                 88a4de450f17c6a319c3e8b2135cd7068a07d0f8

Last test of basis   186158  2024-05-27 02:13:38 Z    0 days
Testing same since   186160  2024-05-27 09:43:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jiewen Yao <Jiewen.yao@intel.com>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   88a4de450f..08281572aa  08281572aab5b1f7e05bf26de4148af19eddc8b7 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon May 27 12:33:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:33:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730710.1135861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZXW-0000hF-Ik; Mon, 27 May 2024 12:33:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730710.1135861; Mon, 27 May 2024 12:33:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZXW-0000h8-Fr; Mon, 27 May 2024 12:33:22 +0000
Received: by outflank-mailman (input) for mailman id 730710;
 Mon, 27 May 2024 12:33:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBZXV-0000h2-2e
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:33:21 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4c8af4c5-1c25-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 14:33:19 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-57863e2846aso2297066a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:33:19 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c8176e1sm482254366b.8.2024.05.27.05.33.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 05:33:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c8af4c5-1c25-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716813199; x=1717417999; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pyh2Q8fs8dzasn3zI/WdkZ9hhYD4APQT0yrKwptzCVo=;
        b=Pv92bRCUc7YG/Qnl2wipwYIMxA6P4Iykf1devqibdhHGvXDoTttEGORwve0WATddJ9
         7n6JrFVW0x18KFR79fa98lypsOmieBbksA96lB1JkwEo+z0CKVr/Z6n8fYfsi6WIHuBc
         7WzvUOvLGR5NvcqlWF6HPzEymU8K/zf+0E5BQvmeDUPSFl6wzTP+kjdQrQpHqXG6Wq8h
         73RnWAk5sgf+7or82zdXHQeuv+0XvP/DXRvoE5AptUa8E0Hgz2m3BzQn35Y+ZcdCoDi0
         xkL34CmIvH8uX1zrysj0o0PlrG4vp29+VgNloga94JkT0f1NQQk+e5IC9mQJQuwWwIkz
         qYkw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716813199; x=1717417999;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pyh2Q8fs8dzasn3zI/WdkZ9hhYD4APQT0yrKwptzCVo=;
        b=BX3CPidKouKggzCKicTpIWE6oMsY73QeTTyAvVwy0qkFPzzUn6DG3LADyDHgoMKumd
         wmrXxWJ5ia7LfiqpL7a46RRImpyWlhBHeqo4W5Xh5bsqCxNcMUFYRH1P3iDh6gr3fO0y
         HNLqYbGwut5sWDuJK5aCBNWe3XI0APhYITGlzDnSGXKMIUq7+2uRvrrpCvvCsnDoWSb9
         AtHYMjV3WxrzZYdI2os0xlAYiMZLN2P0+O7xRsKkbbSlw+0yIJtA+1zgjcnJp+1i5uJf
         AqOIFLRSc+pFL8IsbBV/gMdZefQCZ+REMixqGUPd3VON7d9xn5D4TCq/WzJdBmhAXDhK
         BAMg==
X-Forwarded-Encrypted: i=1; AJvYcCXtaGD4mEDzlZCbKkQ9kk3N/WqO02xTNU8gdZIw2mYj1t7X4ivhwjxcasde3qfUGXbFTWt0hO4bkmACVoeWu2MiLr0z7/adtV0aGeNGXm8=
X-Gm-Message-State: AOJu0YwfHlh6G5ogaYhZ6z6CUiKUN8m0Eu9OqM+QK4s/Qfnw+eITHfDw
	zoilvXIpotjbRYxrnXDOftq7kaNee89oaO0wFxL70d3478PB/arVfDTd82phdg==
X-Google-Smtp-Source: AGHT+IHx4QXo/coC8rMzpNCwsnazGNaMQgCvYDG/UlkwNDbXD9Cr9KkmGq+Ze5GhV27GKipmzOX02w==
X-Received: by 2002:a17:906:478c:b0:a5a:7a1:5d9c with SMTP id a640c23a62f3a-a6265146984mr647600566b.62.1716813198980;
        Mon, 27 May 2024 05:33:18 -0700 (PDT)
Message-ID: <1f885616-3c14-4a80-bc83-8cf712a02d8f@suse.com>
Date: Mon, 27 May 2024 14:33:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-7-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-7-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> Perform constant-folding unconditionally, rather than having it implemented
> inconsistency between architectures.
> 
> Confirm the expected behaviour with compile time and boot time tests.
> 
> For non-constant inputs, use arch_ffs() if provided but fall back to
> generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin that
> works in all configurations.
> 
> For x86, rename ffs() to arch_ffs() and adjust the prototype.
> 
> For PPC, __builtin_ctz() is 1/3 of the size of size of the transform to
> generic_fls().  Drop the definition entirely.  ARM too benefits in the general
> case by using __builtin_ctz(), but less dramatically because it using
> optimised asm().
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 12:40:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:40:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730713.1135870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZe8-0002FS-6y; Mon, 27 May 2024 12:40:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730713.1135870; Mon, 27 May 2024 12:40:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZe8-0002FL-4H; Mon, 27 May 2024 12:40:12 +0000
Received: by outflank-mailman (input) for mailman id 730713;
 Mon, 27 May 2024 12:40:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBZe7-0002FF-Bc
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:40:11 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4161ac84-1c26-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 14:40:10 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-579cd80450fso1670558a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:40:10 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93a828sm489279866b.83.2024.05.27.05.40.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 05:40:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4161ac84-1c26-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716813610; x=1717418410; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FxKlfdChlSYZ4PmUC9Mn34y5/B81hF6zVELLg0A85aM=;
        b=X0vw6FRzhwmIT1rShSUhmBwBOzz0AVeoD/7KI3O4OxWj2BxeWLVvMHo+p4msTWWaZ4
         eInqVgNl7SedqoYdrNAXivm9kx2QaIY+tmOG4WLGisUVYdx79//EXvLdReBzsGYn8C4f
         ta8Su5YklH+9cG7d2uoaSy32uHYM83YFgeXFzmDhV8ft+/CRNDyTVDpnAyiZDPZS8FK+
         ik25nlSv+r0h60gISowQULqVKKUrMyXuv1O/pEiBBKkxvQFAWOQqYgf7A6Ap0zyoNLp0
         8410HBoH189k2OE9f10mDEEQrSGbtSvwPuYnUz1if++bh1etqNWokyehXY0zdS7K6Jcc
         2HuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716813610; x=1717418410;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FxKlfdChlSYZ4PmUC9Mn34y5/B81hF6zVELLg0A85aM=;
        b=WqcesRxxRIw0gk5I8aMmy0ecRp/FE+Zmp/cLyaEG1cSScdYQH+AC9f/aI5wDCHDL7/
         wmWDNDwZgSgWUG9cCXzICV1kr6Ar+KbEjnKySlyDqRknqYvJIyk51BMAiQ2IAflmz5Nv
         1aMPg49cQgnvBxjcxWz1pQ8A5dLI/R1pVX5+8AvO1OfPztCy1NOSa9xcxm7npw7Ek6Gr
         rAiSnDc0jCY/KMiwKpxJ4EdfhR7tqdglTQygdUav73myp60+hqi/Ldy7Z/8ArdBXsTwO
         dHDqOgxfiz+/BbvEvzp21wKjTqtjSCD6GBQYsHIkkPwymbL5lN+BkOx28L6Ere7zD+wc
         A8aA==
X-Forwarded-Encrypted: i=1; AJvYcCU/6Dbt9qag7od8DVOn0lKhaxsP+dWLU+i/wQev8p0ghn/OwOSVGeN3pz5RK1VCapsWMzU0dTIrvlJ0n1jpcKT9ffqzdyRcrUsNAas6+tA=
X-Gm-Message-State: AOJu0YyBDabja5dFYmxSEsTa88dXsTX1O4/A7UGE/Z9vAt1PggGYlT6h
	UphEpX+0aKTDjuRgDKmGzBQoq1KqWYsGTsZgaBvGNZ9mpDB8Yiu4H6gvf+nTDg==
X-Google-Smtp-Source: AGHT+IFhIqN6IW7XC1qNeMbmSeC7SRlcUtgruu2r+2H9TNLDI9NbP3F+iYHpTigGaJQRX/rLzxV6hA==
X-Received: by 2002:a17:907:7da2:b0:a62:ef39:19c2 with SMTP id a640c23a62f3a-a62ef391e87mr303033766b.4.1716813609825;
        Mon, 27 May 2024 05:40:09 -0700 (PDT)
Message-ID: <ebae5dff-009b-495f-8b07-169956199208@suse.com>
Date: Mon, 27 May 2024 14:40:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general
 case
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-8-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-8-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> The asm in arch_ffs() is safe but inefficient.
> 
> CMOV would be an improvement over a conditional branch, but for 64bit CPUs
> both Intel and AMD have provided enough details about the behaviour for a zero
> input.  It is safe to pre-load the destination register with -1 and drop the
> conditional logic.
> 
> However, it is common to find ffs() in a context where the optimiser knows
> that x in nonzero even if it the value isn't known precisely, and in that case
> it's safe to drop the preload of -1 too.
> 
> There are only a handful of uses of ffs() in the x86 build, and all of them
> improve as a result of this:
> 
>   add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-31 (-31)
>   Function                                     old     new   delta
>   mask_write                                   114     107      -7
>   xmem_pool_alloc                             1063    1039     -24
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one suggestion:

> --- a/xen/arch/x86/include/asm/bitops.h
> +++ b/xen/arch/x86/include/asm/bitops.h
> @@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
>  
>  static always_inline unsigned int arch_ffs(unsigned int x)
>  {
> -    int r;
> +    unsigned int r;
> +
> +    if ( __builtin_constant_p(x > 0) && x > 0 )

__builtin_constant_p(x) surely will do. In fact even the other "> 0" could
in principle be left out here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 12:43:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:43:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730720.1135881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZhE-0002p5-LF; Mon, 27 May 2024 12:43:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730720.1135881; Mon, 27 May 2024 12:43:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZhE-0002oy-IQ; Mon, 27 May 2024 12:43:24 +0000
Received: by outflank-mailman (input) for mailman id 730720;
 Mon, 27 May 2024 12:43:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBZhD-0002or-5a
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:43:23 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b3770559-1c26-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 14:43:21 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so9597057a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:43:21 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-5785240d1f9sm5763350a12.51.2024.05.27.05.43.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 05:43:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3770559-1c26-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716813801; x=1717418601; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+QaOBFvZ06TO5nsytNdlDNIECp34V16fy6mGEJPwn38=;
        b=V8IQcxEy50BIucgl76try8A05uhe720ixeF9u6L4c2dq9Of+BUCgb60Cpb8hQAONy7
         N8wflAqXfoJrzXon2TwPDLuwsiulGy8vOvmPshlEgo9Ct7tXLW4C38IwJqRekreslrLQ
         XZuBWTTmmPuLtZ9FuKHeM2w+HTxSLqRhIP1Kdiu1LObYiKxcW23IMjJQQ/tc4K8ucTUc
         AnXyki5iLtIAbh1SNGwQDaNmLlbsuTgCm8kXxvCIOiyLMA1Na4KN5L9ttZi6vDrROcXU
         +OfMaAX+xJYptb6KDDluYFBJUzpb7MsLRKFH2sS+2Efu0phwkuhbevnNGt3TbA86giOW
         ynXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716813801; x=1717418601;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+QaOBFvZ06TO5nsytNdlDNIECp34V16fy6mGEJPwn38=;
        b=dVqeyQbgFJZbBgXCmW3lSXkjujcLqkhDJ1/urrn+UC4Ho7biTLoHDtAHiKx7ygrFNf
         sfCOS7fEQDJ8UoHbq93Q7CxJj26dDZFIoHO4AD4pIZvgkABnW0J1NcmpHtwAX48LP5ho
         BSv1ymrd8I7VRXdlfR5kEznZZZQnMD1Z5xZz2M2XrHZsrFiDKlIGanAY0rGXOiLav8R+
         VOxeBeVU0bXRPpFxJRiID0vUSo4W6/nGPCY+MuByBXaY5JIVEXhaG8Jbxp993bgpzS64
         7XCfaghdds14YMJ6YqMjYXKTzAoGwDGCUKxoYhyi2dKwxH/4xHyBMxeQpc98xjByr8GK
         aeGA==
X-Forwarded-Encrypted: i=1; AJvYcCX/edS+AuRxXiVM6HIm4we1S8TEf5m9dapRd2bpxlbvgqNakI4c1vSyWqH+HDIpQnEQEBEYRc8WbDTw7XsQgwVSv8CIieiLpNd0TVygA9U=
X-Gm-Message-State: AOJu0YxIoj8vKAPQiVBdvrrCPUN9NamuNh52u0CIRe9DgrQ5MfYv0sis
	ITYd5+v8qG/j33zr+LBVxMFlhvbak4FZSvNrpoRGx6f8LTa5s6Ng8BIOoFf9HA==
X-Google-Smtp-Source: AGHT+IHKgLSBgAuP+vB89eWE0GtmsyHtfLICeKn5NCPyIJIR9ZZHIA/WvKJnRmV0rLhr4INrzKX2bw==
X-Received: by 2002:a50:9f63:0:b0:578:69be:6e7a with SMTP id 4fb4d7f45d1cf-57869be803amr4999912a12.1.1716813801331;
        Mon, 27 May 2024 05:43:21 -0700 (PDT)
Message-ID: <db5f4449-e333-426f-bbe8-e5b77daba16d@suse.com>
Date: Mon, 27 May 2024 14:43:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 08/13] xen/bitops: Implement ffsl() in common logic
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-9-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-9-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> Just like ffs() in the previous changes.  Express the upper bound of the
> testing in terms of BITS_PER_LONG as it varies between architectures.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> @@ -458,6 +441,24 @@ static always_inline unsigned int arch_ffs(unsigned int x)
>  }
>  #define arch_ffs arch_ffs
>  
> +static always_inline unsigned int arch_ffsl(unsigned long x)
> +{
> +    unsigned int r;
> +
> +    /* See arch_ffs() for safety discussions. */
> +    if ( __builtin_constant_p(x > 0) && x > 0 )

See remark on arch_ffs() for possible slight reduction of code.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 12:58:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:58:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730729.1135890 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZvI-0004XT-V2; Mon, 27 May 2024 12:57:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730729.1135890; Mon, 27 May 2024 12:57:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZvI-0004XM-SW; Mon, 27 May 2024 12:57:56 +0000
Received: by outflank-mailman (input) for mailman id 730729;
 Mon, 27 May 2024 12:57:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBZvH-0004XG-As
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:57:55 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bade3e22-1c28-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 14:57:53 +0200 (CEST)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2e6f51f9de4so136655851fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:57:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a62c501e85dsm247769766b.117.2024.05.27.05.57.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 05:57:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bade3e22-1c28-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716814672; x=1717419472; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9Fvwildf8cnAtHd0zQYSd/l8qEHNqNAOg6zUFk8KfZg=;
        b=D4+NJP7TaSaMEigajgCxugL/tpqK4O8LKtA1DBcmx0FsWOTX3ivIYMFQfJRCevl320
         jhPhKjTET4wBH9WY6JvDU27MARSjURNKl8sVWPlAphS6TZ3wZmh9z47J1liMw6JzNR1/
         Bg52ojrmnW38PDssREDHZQ2Re/MDG/CT7m40y/8f1mxhV+nY/FyX0q/WFR8ptct3YtYA
         X4vX3q7YAk3SvawBF3ix947OWEkZAzd/14SNSo2y0oGrLfrQYzcdu+Pd85zfDFlNjnBt
         x6TO88kXv90SaQ86151LnjM3lyMfqBh/ukEt7NGuPrvtdEcFAogBFBLYNJKP30d/Q6ko
         GdCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716814672; x=1717419472;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9Fvwildf8cnAtHd0zQYSd/l8qEHNqNAOg6zUFk8KfZg=;
        b=wBYkbIljtW6cW2+f0oL3Idsyfw8CiNAxUPEsuKr80MwXjhAGqt/Zh1MRGzJXo2kaBh
         yL6TJtnGTon+tHIhTz1RAs+9IB8xDbMp2SKs2I9bojfi1SzLAWsQb98yoEffjOV9olmK
         rRe1IJWn9hI19MKwvRJzt94I+R3hRvIpd+JXR7viivxTcT5Ei6iySbcVTr+2bwQZVBho
         XFRsBHkWZXan0GbDumPFs4Y/Y+bD0yYaKELJSv0+zOe8buo7Ut3IDGiyNyNSx4FvGwyH
         3dMoX3s72+ZItBG32mygPbQ/5dAjvbdZJLf4nzNcPqQEqpwdMUgFMUy4lhJ9N6E84aGO
         iIxA==
X-Forwarded-Encrypted: i=1; AJvYcCX+XuduFdaqi2JhKFRl4U9kkwhQ30xTDJFcjaG4pDShpdloDiFMEQSVjcyTY+GzbEBvg3NrLUednlh6OX7SC+/Y52zgc8Fn/k1uSztgzbQ=
X-Gm-Message-State: AOJu0Yyn/w0BNfUhjWKvCChYPUrkjmKXSQIfvfswUTPLDXTSRCtxPosV
	RnwQk8wVkcrld8zEXFfH1X29cHWX8Ijl1PXF6Il+aTdCwQkECllsQcZf2qzMxQ==
X-Google-Smtp-Source: AGHT+IE2flPVjnRYafEEF37l7wv3+ENaWRXBlbtOqLd3oW1iSkrvs7+8BE732cGwR4Hz21mhIpwOug==
X-Received: by 2002:a2e:a782:0:b0:2e7:2c84:4f65 with SMTP id 38308e7fff4ca-2e95af0ec44mr92309331fa.0.1716814672480;
        Mon, 27 May 2024 05:57:52 -0700 (PDT)
Message-ID: <fb5e42db-38fc-4426-b111-ccd454237119@suse.com>
Date: Mon, 27 May 2024 14:57:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 09/13] xen/bitops: Replace find_first_set_bit() with
 ffsl() - 1
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-10-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-10-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> --- a/xen/arch/x86/hvm/hpet.c
> +++ b/xen/arch/x86/hvm/hpet.c
> @@ -335,7 +335,7 @@ static void timer_sanitize_int_route(HPETState *h, unsigned int tn)
>       * enabled pick the first irq.
>       */
>      timer_config(h, tn) |=
> -        MASK_INSR(find_first_set_bit(timer_int_route_cap(h, tn)),
> +        MASK_INSR(ffsl(timer_int_route_cap(h, tn)) - 1,
>                    HPET_TN_ROUTE);
>  }

This can be just ffs().

> @@ -409,7 +409,7 @@ static int cf_check hpet_write(
>          {
>              bool active;
>  
> -            i = find_first_set_bit(new_val);
> +            i = ffsl(new_val) - 1;
>              if ( i >= HPET_TIMER_NUM )
>                  break;

This in principle can be, too, but would require a little further care.

> @@ -535,14 +535,14 @@ static int cf_check hpet_write(
>      /* stop/start timers whos state was changed by this write. */
>      while (stop_timers)
>      {
> -        i = find_first_set_bit(stop_timers);
> +        i = ffsl(stop_timers) - 1;
>          __clear_bit(i, &stop_timers);
>          hpet_stop_timer(h, i, guest_time);
>      }
>  
>      while (start_timers)
>      {
> -        i = find_first_set_bit(start_timers);
> +        i = ffsl(start_timers) - 1;
>          __clear_bit(i, &start_timers);
>          hpet_set_timer(h, i, guest_time);
>      }

Same here; in fact {start,stop}_timers are needlessly unsigned long in
the first place.

> --- a/xen/arch/x86/include/asm/pt-contig-markers.h
> +++ b/xen/arch/x86/include/asm/pt-contig-markers.h
> @@ -60,7 +60,7 @@ static bool pt_update_contig_markers(uint64_t *pt, unsigned int idx,
>      /* Step 1: Reduce markers in lower numbered entries. */
>      while ( i )
>      {
> -        b = find_first_set_bit(i);
> +        b = ffsl(i) - 1;
>          i &= ~(1U << b);

Considering i's type and the immediately following expression, this again
can easily be just ffs().

> --- a/xen/drivers/passthrough/amd/iommu_map.c
> +++ b/xen/drivers/passthrough/amd/iommu_map.c
> @@ -137,7 +137,7 @@ static void set_iommu_ptes_present(unsigned long pt_mfn,
>          ASSERT(!pde->u);
>  
>          if ( pde > table )
> -            ASSERT(pde->ign0 == find_first_set_bit(pde - table));
> +            ASSERT(pde->ign0 == ffsl(pde - table) - 1);

pde pointing into the page starting at table, this can be ffs(), too.

Preferably with at least the easy adjustments done:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 12:58:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730734.1135900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZvw-00055E-9j; Mon, 27 May 2024 12:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730734.1135900; Mon, 27 May 2024 12:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZvw-000557-7A; Mon, 27 May 2024 12:58:36 +0000
Received: by outflank-mailman (input) for mailman id 730734;
 Mon, 27 May 2024 12:58:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBZvv-00054x-2P
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:58:35 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d333dd8e-1c28-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 14:58:34 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a62972c88a9so236161766b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:58:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a62c501e85dsm247769766b.117.2024.05.27.05.58.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 05:58:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d333dd8e-1c28-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716814713; x=1717419513; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+Yme6rd5RZYashNZq5SnUrewWqWptJp8i6Jp+GbeKLw=;
        b=DR8Wku8IflbSgrHzpw4Y9BAXcKrPHjZ/B3g6/xNKmRI/tZRgkocxTrPJE4Ee2HGrYN
         ZiVAiZALy/UojRPEahf2JHvpDvTCLW9QeM3ADKvZSlXeRzB5Ct6ufV7QMaUnWHD0CRjG
         Fxd9pMb9wWYark0AHeYUBswug5aGZDtJ2qyrPPU5tb0rAgBIZScW2JUdYu2kMk6qL4P1
         v4RHv/hmTy8lhrsUE3uYvn/kCegwHXqIXZYWjJ2QkJomHZl9r0h2OxFjxXxmCF+94ZiC
         Lkew5tD9o3IFUbY1IN+LWBZQJgVp7cPFkQJGlaqNAMixgBYbfkBu5AZbSYFHijfcMyWj
         gF5A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716814713; x=1717419513;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+Yme6rd5RZYashNZq5SnUrewWqWptJp8i6Jp+GbeKLw=;
        b=meXIGTQIIyomkdishavYbLYeZO0va6GnFLa85W+WRoE3nkBpkFoD6IL1mvHEFYqaOY
         wbUbcoJHUxa0uuYoqwnI+bs/yhr3U9pJ0FaEHFBadxX88oUWu1sk0ANXJBb7GiwIzEUL
         tI1m4cIQtM1D2xKb9xFtiM/cWCS0AwrupPQk9dolINK7Dhns8VXELyLEb+iX2H67MH6h
         bkaulh+pq0MqS3ELp9BjcN+db7vYdErEGH39XeL6rk91ahvqzolysAeQ0b6D9fYL7ZQ7
         Ec39NZPZdf7bRHVlmFYc/6aa0ODE39D02nUbyTOBnxXjtZI0lqWDL0RZux51ozmqL537
         ZgvA==
X-Forwarded-Encrypted: i=1; AJvYcCVevuFYFWEtGmZdwlLXZvoahTZ28kzzB6ohALsvzXcv0QlqnrLEKNsKI492pJkT4tKRPcX54paVat2NAlVBALt89JnUD+4lqcx4v5AmfqI=
X-Gm-Message-State: AOJu0YwpPkoA/jq6VR7QUo9Y6chTz69ArBz7AR9ns1eU5x3+wjv2yHJS
	vlaPIs6UI0XW4hA8jI1mrCMhj73NgSHwJH10kWqjl3oiZifeVuWnh0vSWpr0Ag==
X-Google-Smtp-Source: AGHT+IGbz1zNzGOOQzvC3vXB7vasindAwIi0MBbktLeKFgA2UWNqPL4IucJUAE+oaqG8uq7eWWsqrg==
X-Received: by 2002:a17:906:c49:b0:a5c:dd34:2312 with SMTP id a640c23a62f3a-a6264f01525mr590867766b.62.1716814713475;
        Mon, 27 May 2024 05:58:33 -0700 (PDT)
Message-ID: <a9b7ddb7-a485-4d0f-b2cc-7207093514c1@suse.com>
Date: Mon, 27 May 2024 14:58:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 10/13] xen/bitops: Delete find_first_set_bit()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-11-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-11-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> No more users.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 12:59:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:59:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730725.1135910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZwY-0005bY-In; Mon, 27 May 2024 12:59:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730725.1135910; Mon, 27 May 2024 12:59:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZwY-0005bR-GI; Mon, 27 May 2024 12:59:14 +0000
Received: by outflank-mailman (input) for mailman id 730725;
 Mon, 27 May 2024 12:55:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WtIC=M6=gmail.com=yskelg@srs-se1.protection.inumbo.net>)
 id 1sBZsz-0004Tb-KA
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:55:33 +0000
Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com
 [2607:f8b0:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 667be261-1c28-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 14:55:32 +0200 (CEST)
Received: by mail-pl1-x630.google.com with SMTP id
 d9443c01a7336-1f44b4404dfso21862055ad.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 05:55:32 -0700 (PDT)
Received: from paran-QEMU-Virtual-Machine.. ([118.32.98.101])
 by smtp.gmail.com with ESMTPSA id
 d9443c01a7336-1f44c757845sm60382005ad.5.2024.05.27.05.55.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 05:55:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 667be261-1c28-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716814531; x=1717419331; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=pMe2OjnAw9DOpVmn/tQbHcQU5IfzyQdTOAHTtCD7gi8=;
        b=FvS2KSZd7lE/ejf3q/peyRA7xt55tW0gw5w+/3UNtK89gxHuSZNpXFJkLKXpXaK9fi
         ktL9zEVoSbPLHDmJ5BXwPKIMBnYy5iyaxrAPBmxjYHwndXvRE9YTKdxT6YFN4OhnErxT
         /YF58gf766seTiVIcHcX6CjQ86hJ/wy1+Oh311YQpPh6Ux2x6DOzZErKMJKdSJLE8ll/
         a5LQ1aopzkXQVoWWvU0r4NWZ1jhoXwHOJHKIRvUPCZVgnihQMpPXQhCPH5Yvj404sNzh
         vh6S8KG31/sjldYsYLRAqQP6fco35EsmLnJK2HPZjGwT0S/H+7mLyJS1EHodpNFUF6Ze
         6Paw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716814531; x=1717419331;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pMe2OjnAw9DOpVmn/tQbHcQU5IfzyQdTOAHTtCD7gi8=;
        b=qfyIdZHWIHwwNWZi/PC/+3GSBBiLBj2yapXaGjbIhtinUghHCeRDxH46RutvvhLs9X
         ltP6ayIJv5v4G+Civ/7KAnn1Uza/02ik4u2ZRU8TYgxt5WhD7F9eYHxqnGKMTYMvN9wd
         7HXnD9Eh6Cv+DpijMEp91HbMh8dCqdPNT1DOr5LyEX9PDpgFI40n6G+IIsxokxRJqHL8
         2dLId82mBH8Rg7uxmLKii+jXo6YgB33mV/iank/KOBvsmQ70lJoVYIVpdDyxLch8I2/f
         bYO92sElmrg1CjFA6EKGTRbZuY/RfJM3l1SmKSLoOVQlKTvHkeOxkiU1UfgBZkeeYlrz
         M5YQ==
X-Forwarded-Encrypted: i=1; AJvYcCUjJTPbPXKbmBfN2fgR96nQgGBGjt/R8yrxKwkwu+pul6at4iSs416vqxaQEjgnbnQiKVAQV0+Lr1Ani+ozS+t20OL68sr96E7N1SJ8Q98=
X-Gm-Message-State: AOJu0YwekmPCPnQd+jpcgqMX9BToHLXvEKHfmD/anv2Wq237joqbeev6
	LzmcVvdudphT9lbZZfmhPDEdHu+DtfyXniPY4aXf5wmciXvbkpMmcztQMzuXMaI=
X-Google-Smtp-Source: AGHT+IEvCWcm6q+tC3X0xxKR+wWh5xFppmrVISfccNDFu9TXeDUUje3ns00KN5KoFpXURDLsUc3FBA==
X-Received: by 2002:a17:902:d50c:b0:1f4:b313:3e9a with SMTP id d9443c01a7336-1f4b3134380mr10896515ad.65.1716814530796;
        Mon, 27 May 2024 05:55:30 -0700 (PDT)
From: yskelg@gmail.com
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Anthony PERARD <anthony@xenproject.org>
Cc: Austin Kim <austindh.kim@gmail.com>,
	shjy180909@gmail.com,
	xen-devel@lists.xenproject.org,
	Yunseong Kim <yskelg@gmail.com>
Subject: [PATCH] loadpolicy: Verifies memory allocation during policy loading
Date: Mon, 27 May 2024 21:54:39 +0900
Message-Id: <20240527125438.66349-1-yskelg@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Yunseong Kim <yskelg@gmail.com>

memory allocation failure handling in the loadpolicy module.

Signed-off-by: Yunseong Kim <yskelg@gmail.com>
---
 tools/flask/utils/loadpolicy.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/flask/utils/loadpolicy.c b/tools/flask/utils/loadpolicy.c
index 76710a256c..7f6bab4dcd 100644
--- a/tools/flask/utils/loadpolicy.c
+++ b/tools/flask/utils/loadpolicy.c
@@ -58,6 +58,11 @@ int main (int argCnt, const char *args[])
     }
 
     polMemCp = malloc(info.st_size);
+    if (!polMemCp) {
+        fprintf(stderr, "Error occurred allocating %ld bytes\n", info.st_size);
+        ret = -ENOMEM;
+        goto cleanup;
+    }
 
 #ifdef USE_MMAP
     polMem = mmap(NULL, info.st_size, PROT_READ, MAP_SHARED, polFd, 0);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 27 12:59:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 12:59:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730727.1135917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZwY-0005es-Sz; Mon, 27 May 2024 12:59:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730727.1135917; Mon, 27 May 2024 12:59:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBZwY-0005dw-MT; Mon, 27 May 2024 12:59:14 +0000
Received: by outflank-mailman (input) for mailman id 730727;
 Mon, 27 May 2024 12:56:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Kte=M6=linux.intel.com=ilpo.jarvinen@srs-se1.protection.inumbo.net>)
 id 1sBZtj-0004UW-Ph
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 12:56:20 +0000
Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 806f077d-1c28-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 14:56:16 +0200 (CEST)
Received: from orviesa010.jf.intel.com ([10.64.159.150])
 by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 27 May 2024 05:56:14 -0700
Received: from unknown (HELO localhost) ([10.245.247.139])
 by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 27 May 2024 05:56:10 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 806f077d-1c28-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1716814576; x=1748350576;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=/GjJOKz80/OPeh1IZUhwdO/jTNvJXvgYw28YHmRy+ms=;
  b=IIMT0WaDdedSP4Lxd8EpFBDtky/oZjEwfIdoNjA+eaVum20oD3cRGT4G
   3p3SZp7QYOUjKNT0mS1/pZzVyuKyG4fPhiPhT1xMx9vqJEbLHCt+rifvf
   Y+/mNMvGv0lNHl3lVcVeP0QoLBjyCSYdblQRTctXFR4R7Z/TGKeaAlfRc
   ipOvmUHasbO/x/55Eh3GlHUPDLmgBwtBjPKl1f4bL74xVTITQfUwk2UpX
   +PGRpXfnkp1pSJfin9p2d+5RwufdcVaAm1TwoMVW20525RGfsGAQsm+MH
   K196RIpakrMrBeNTmHMnbx5IAgmoeSXWCrwo3ZrlpG4yChkr8vcyaxtUh
   w==;
X-CSE-ConnectionGUID: +uXa3yUFT2+riveQYP0R5w==
X-CSE-MsgGUID: G164Qc3CQ+qH5BtCEOAweg==
X-IronPort-AV: E=McAfee;i="6600,9927,11084"; a="23734146"
X-IronPort-AV: E=Sophos;i="6.08,192,1712646000"; 
   d="scan'208";a="23734146"
X-CSE-ConnectionGUID: YFWi0+SrQV+rzQS/udVCIg==
X-CSE-MsgGUID: ZBHpgWfbTRyNJw3j2NIY2A==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="6.08,192,1712646000"; 
   d="scan'208";a="34643688"
From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
To: Juergen Gross <jgross@suse.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	x86@kernel.org,
	"H. Peter Anvin" <hpa@zytor.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ian Campbell <ian.campbell@citrix.com>,
	xen-devel@lists.xenproject.org,
	linux-pci@vger.kernel.org,
	linux-kernel@vger.kernel.org
Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
	stable@vger.kernel.org
Subject: [PATCH 3/4] x86/pci/xen: Fix PCIBIOS_* return code handling
Date: Mon, 27 May 2024 15:55:37 +0300
Message-Id: <20240527125538.13620-3-ilpo.jarvinen@linux.intel.com>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240527125538.13620-1-ilpo.jarvinen@linux.intel.com>
References: <20240527125538.13620-1-ilpo.jarvinen@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

xen_pcifront_enable_irq() uses pci_read_config_byte() that returns
PCIBIOS_* codes. The error handling, however, assumes the codes are
normal errnos because it checks for < 0.

xen_pcifront_enable_irq() also returns the PCIBIOS_* code back to the
caller but the function is used as the (*pcibios_enable_irq) function
which should return normal errnos.

Convert the error check to plain non-zero check which works for
PCIBIOS_* return codes and convert the PCIBIOS_* return code using
pcibios_err_to_errno() into normal errno before returning it.

Fixes: 3f2a230caf21 ("xen: handled remapped IRQs when enabling a pcifront PCI device.")
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Cc: stable@vger.kernel.org
---
 arch/x86/pci/xen.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 652cd53e77f6..0f2fe524f60d 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -38,10 +38,10 @@ static int xen_pcifront_enable_irq(struct pci_dev *dev)
 	u8 gsi;
 
 	rc = pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &gsi);
-	if (rc < 0) {
+	if (rc) {
 		dev_warn(&dev->dev, "Xen PCI: failed to read interrupt line: %d\n",
 			 rc);
-		return rc;
+		return pcibios_err_to_errno(rc);
 	}
 	/* In PV DomU the Xen PCI backend puts the PIRQ in the interrupt line.*/
 	pirq = gsi;
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Mon May 27 13:27:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:27:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730750.1135930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaNg-0002ae-UI; Mon, 27 May 2024 13:27:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730750.1135930; Mon, 27 May 2024 13:27:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaNg-0002aX-RV; Mon, 27 May 2024 13:27:16 +0000
Received: by outflank-mailman (input) for mailman id 730750;
 Mon, 27 May 2024 13:27:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBaNf-0002aR-6i
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:27:15 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d4374d88-1c2c-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:27:13 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-529661f2552so3066232e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:27:13 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578649b3001sm4032343a12.77.2024.05.27.06.27.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 06:27:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4374d88-1c2c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716816433; x=1717421233; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rKSNs4523+4teDwcbjKLKgWAgOut0WjV9YrmmUEFspI=;
        b=O/CFUgmt8LgfRo28yhsK9/rLcoUPsI4bIlpwoiD9/SV7XN0TeDyIV3KC9Pwoi8X182
         kbj+y4Ftt0KWr4cQc82crCv0FNO6UqszbSALzI2J+fPKe0xECbNfcNDSGCarMaiKcz8E
         uoF2vJtbqAJvU8IJQbzOzERBSbQ8U5ghX/ojKChgtRlJ7jh9rEt85Yr1gar6RqKMo6/U
         mfJ1osxkrqhVLKbS1eLcvKG08LWlnTgQeI7FEiZZ0+epn6Qx4DLaPyIQf+Eral4FaQpT
         xuODPrRsJA3Ukcuqw+3AXvv/pOUlYi3j0GAA+NaCGmNQSyx2+uNW5khoIPSFLwi27jYB
         S24A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716816433; x=1717421233;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rKSNs4523+4teDwcbjKLKgWAgOut0WjV9YrmmUEFspI=;
        b=YMZ683VJTrgumcddAll1VDr6Mz1gtrDxh15R6RswGxEVagUUVe2EMZuXABvh+Fmp+Q
         PjM+YB0smeeO9c2JxT9ntU2Xd+qEZzo7oq0UVxb+5W8xscrv3c1VJ3ccJsB2pVN+e6HA
         1CdEr4jlbt62pEguKjZAHNQ1bkdMKR1aKabMdsaawzRD9NRSCbvl5ELpe0W8jr79pn8b
         90Xzd27mzzWEOmbe04lcl17FQ56hOXJeyX9Ra62nkvOGDxX6FD0xFLesIi47uZSJ97kE
         Arof3n90KVahyHpCErAvGpXJsqZv9/8LXfS2lT2KIcM9qsF04tEyQqUKh9e3DbAV0YjE
         aFpw==
X-Forwarded-Encrypted: i=1; AJvYcCXLGpY3jHesAYwM6q9mxuK0Lq8R4FHAxsQEG+JCCpWsLFxTbASrItcFLDFckZB0q+szE2LgL/w+rrbtH1SoaEOTB3LZKRG0czMX/sa+wRg=
X-Gm-Message-State: AOJu0YzEyupy0/2/BON0wK4P9qhckOsIfA4rZAlst3K73oGjac60tdaK
	cd8h0FJp1xD6ceZc/cmzQ2GvPhoK/J6A6QaJ3uw+vdPCLx4VuiQk6EH+ifhLEQ==
X-Google-Smtp-Source: AGHT+IFKRzD2OdT2IGhCCThP3gDFk3CxPFQhNOcCbfeLY3vK7Uw0Y1y6iSRYPWqUqGhwHC7kGcHSlg==
X-Received: by 2002:a05:6512:3684:b0:51e:f8ae:db35 with SMTP id 2adb3069b0e04-529668ca029mr5061217e87.43.1716816433122;
        Mon, 27 May 2024 06:27:13 -0700 (PDT)
Message-ID: <1660a2a7-cea4-4e6f-9286-0c134c34b6fb@suse.com>
Date: Mon, 27 May 2024 15:27:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general
 case
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-8-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-8-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> --- a/xen/arch/x86/include/asm/bitops.h
> +++ b/xen/arch/x86/include/asm/bitops.h
> @@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
>  
>  static always_inline unsigned int arch_ffs(unsigned int x)
>  {
> -    int r;
> +    unsigned int r;
> +
> +    if ( __builtin_constant_p(x > 0) && x > 0 )
> +    {
> +        /* Safe, when the compiler knows that x is nonzero. */
> +        asm ( "bsf %[val], %[res]"
> +              : [res] "=r" (r)
> +              : [val] "rm" (x) );
> +    }

In patch 11 relevant things are all in a single patch, making it easier
to spot that this is dead code: The sole caller already has a
__builtin_constant_p(), hence I don't see how the one here could ever
return true. With that the respective part of the description is then
questionable, too, I'm afraid: Where did you observe any actual effect
from this? Or if you did - what am I missing?

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 13:37:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730756.1135940 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaXP-0004dW-R8; Mon, 27 May 2024 13:37:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730756.1135940; Mon, 27 May 2024 13:37:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaXP-0004dP-OP; Mon, 27 May 2024 13:37:19 +0000
Received: by outflank-mailman (input) for mailman id 730756;
 Mon, 27 May 2024 13:37:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBaXO-0004dJ-MS
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:37:18 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3b3648e3-1c2e-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 15:37:16 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-5766cd9ca1bso10323227a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:37:16 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a631e324816sm27876466b.15.2024.05.27.06.37.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 06:37:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b3648e3-1c2e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716817035; x=1717421835; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=7Gny4Ab2e3yPKopA6ax7hto72c8SMIy0PsmVtYocx2c=;
        b=GwBcSnwjJUnA8Tfng8K4f/qbHHEZgrO6VyT3Yg2+SKhUnxG4lz4c89LOj5N4RBLKA8
         gVx0/iD5HZnNMfKQ5v52B6yUf8mO6Qo6QrcKzKobPx/dcKQGOwFVs2VINE6+zAUl8wzO
         nsbrSZLfgdpsG2bf0Xdb/qWAX52njIrExlhM4mbXkY6/qBukub01Ch6UYEodCU6rg28N
         OPt6J3cxn6HOk7SVNrQwD2dtk6hYGqJSO2hqKRdJ+siy/gxLY2vmyb3Kvh210vb8p5oh
         XqUVFs33umsAnr0bt65e77oqgJyksL1GhRwJa9jB7cK7GgYJW+YvJBU1RiZebzxhMOab
         +Txw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716817035; x=1717421835;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=7Gny4Ab2e3yPKopA6ax7hto72c8SMIy0PsmVtYocx2c=;
        b=p5r1kM6PoC/i+G50B1dT6ihL6H1m3vU+MPTAX3/jR/ANgxJqPycue6MVUDJ93qPF5W
         frE7pdw8/y13FCCB5ZR7tfF4jQfDnEaqahVbREDbriKMgEWwAkpQdM1zccRICIUDyNNt
         7Ana45Q1/Y2l9vUTSmYmrIssJnHg1aRGryg9E+JFmW/oiM7uPcdRX/3ODS/Kq1Qwdgpa
         v5oU2a9b4N9TFN6rASWdiTvznpUa27TwxD96q6qGvkJ38DiNUvvl0Ds/VOyy84vWEWcU
         nFJT4KPOPASpw/43+l0pxtEQmMewCnIgvc2PpUzcF9+kAnmNuIf3HNlOQDyI+qx3ZETv
         zsCw==
X-Forwarded-Encrypted: i=1; AJvYcCVrk+psHkJgkxTiDn7fjWR3jo+DEcggyawwU+6IQFNpT8OrNLsBPYbLueb2K+CZnCTdKWBHj79XFUZcNr0xA11NaUg/NPEmTjVtAj1dvOM=
X-Gm-Message-State: AOJu0YwuPzB43JFTvRI2Dx05iyXRf6bi5EshyA4Y/5gHu/vho8kLSJVg
	mse/Rn6J5iL81a9suMloavj1krfdaIF+yQCpTjF5ZGfdyCvl0HYPJUEoIRT8WQ==
X-Google-Smtp-Source: AGHT+IFcrn1pbcmLn/tN42dFrOZymzeN0hd1z0TZ69y5uIyI+tDGAf5wz1oDXqbS43JMGdKABTq96Q==
X-Received: by 2002:a17:906:f88b:b0:a5c:d67a:1e1c with SMTP id a640c23a62f3a-a62641b8ba2mr700764566b.20.1716817035487;
        Mon, 27 May 2024 06:37:15 -0700 (PDT)
Message-ID: <57a47c76-c484-4309-8a87-a51f79dd48b6@suse.com>
Date: Mon, 27 May 2024 15:37:14 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general
 case
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-8-andrew.cooper3@citrix.com>
 <1660a2a7-cea4-4e6f-9286-0c134c34b6fb@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1660a2a7-cea4-4e6f-9286-0c134c34b6fb@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 15:27, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
>> --- a/xen/arch/x86/include/asm/bitops.h
>> +++ b/xen/arch/x86/include/asm/bitops.h
>> @@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
>>  
>>  static always_inline unsigned int arch_ffs(unsigned int x)
>>  {
>> -    int r;
>> +    unsigned int r;
>> +
>> +    if ( __builtin_constant_p(x > 0) && x > 0 )
>> +    {
>> +        /* Safe, when the compiler knows that x is nonzero. */
>> +        asm ( "bsf %[val], %[res]"
>> +              : [res] "=r" (r)
>> +              : [val] "rm" (x) );
>> +    }
> 
> In patch 11 relevant things are all in a single patch, making it easier
> to spot that this is dead code: The sole caller already has a
> __builtin_constant_p(), hence I don't see how the one here could ever
> return true. With that the respective part of the description is then
> questionable, too, I'm afraid: Where did you observe any actual effect
> from this? Or if you did - what am I missing?

Hmm, thinking about it: I suppose that's why you have
__builtin_constant_p(x > 0), not __builtin_constant_p(x). I have to admit
I'm (positively) surprised that the former may return true when the latter
doesn't. Nevertheless I'm inclined to think this deserves a brief comment.

As an aside, to better match the comment inside the if()'s body, how about

    if ( __builtin_constant_p(!!x) && x )

? That also may make a little more clear that this isn't just a style
choice, but actually needed for the intended purpose.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 13:38:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:38:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730762.1135951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaYs-0005DS-9L; Mon, 27 May 2024 13:38:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730762.1135951; Mon, 27 May 2024 13:38:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaYs-0005DL-5A; Mon, 27 May 2024 13:38:50 +0000
Received: by outflank-mailman (input) for mailman id 730762;
 Mon, 27 May 2024 13:38:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBaYr-0005DF-Jz
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:38:49 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 725cd2ec-1c2e-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:38:48 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so1364184466b.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:38:48 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a631e324816sm27876466b.15.2024.05.27.06.38.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 06:38:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 725cd2ec-1c2e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716817128; x=1717421928; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YeNLZwyQvcwTIaVv1Yo57KS7N37BKKunue1q8dV1ijs=;
        b=WhEDnCBH4j4ahmz62FUJCLQeeoqljO81z21xOK3P7j404D7gTG0NrHWXLpXdH5JNDn
         GGF0luxVP6WBlD5xlplnw7qmU+9xh/IDKbD+VS2hPfvUOvoBCcmmX/JkCLIHZbWSJVxs
         BkPIA3ENYPuoCSLWWMgvpQDTulZVveTPkf+1Wa9FVnw4w9YPRXQScFyxkBkv4Nhd9Qon
         N4GYbYLFMj+AJh7VfXOyCFPE+i0esXiet08WTKZWzAELslubLceDechlHoHJQVMkZkHr
         IJXEMezo9kHlmd307XsKio3w1dFGTxSoB8mpQNIPKcJwtSEMiT/QqxagcfCjegBy+PXi
         YT2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716817128; x=1717421928;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YeNLZwyQvcwTIaVv1Yo57KS7N37BKKunue1q8dV1ijs=;
        b=KkArJBZPJhmxjDsO3oqWkcxkoE+OYzvKk0eCKOXKswc1owgIaafnfEVCHB7q4UfS2n
         E+oGnTIUmVKElpwobsHsaH6tgHG9tV7N1c7ly4DzBdjJni0GDR8NEIjRK+Cpvm2nOULQ
         mthuMbUM2ZMT6izANpnzmjE8nUsaVUWMLjdmqNbno3sU3M7pilr64eFxqrjNXOu0jAwC
         dShaKsxd5z1OMNeWicw7NTAwO0lDuVdyQm2QOnrKGxU8pTlE3Cui+BYIUEC8ri/r70dr
         802Ve54LVBI2hjucgvLCfXBi9W36HW0oVE6OwREzIlbVqoRVzCqiQl7fgm/Uhvm5ScD6
         rYEQ==
X-Forwarded-Encrypted: i=1; AJvYcCWKvR19cWbrCJy4QRyHy7z5XQBfX8uBOcycn58fRtsFIEKRvfqAYd4a/f6cTR4xAbVjIOiyiKU4I5GRZDSH+ZZcNyRHFuOpd896uEvHDtk=
X-Gm-Message-State: AOJu0YzKqv9JSN4P3j4H/jHYd9W5N0v0rbPeZYLnPsff4ie/0JYX9CPv
	OXTDIxSOyIG8ySCadfA0cL3ESNOKd0eEeNPu5AfEmy/V9DH6K+KzJPiUozMvVA==
X-Google-Smtp-Source: AGHT+IEwuuLl3W6kAbIY9pznGBgrGWXnNs0bEvw3yM4tN88TVO8dSE2qpL5jwgkOaNOrw0/++pgLkQ==
X-Received: by 2002:a17:906:390d:b0:a62:2ba5:7bc2 with SMTP id a640c23a62f3a-a62635aee1cmr545760066b.0.1716817127931;
        Mon, 27 May 2024 06:38:47 -0700 (PDT)
Message-ID: <930e108d-3637-4835-b4c9-e92da7f6d654@suse.com>
Date: Mon, 27 May 2024 15:38:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 11/13] xen/bitops: Implement fls()/flsl() in common
 logic
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-12-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-12-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> This is most easily done together because of how arm32 is currently
> structured, but it does just mirror the existing ffs()/ffsl() work.
> 
> Introduce compile and boot time testing.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with small adjustments possibly to be done on the earlier similar patches
also done here.

Jan



From xen-devel-bounces@lists.xenproject.org Mon May 27 13:45:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:45:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730770.1135961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaea-0007M3-Qn; Mon, 27 May 2024 13:44:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730770.1135961; Mon, 27 May 2024 13:44:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBaea-0007Lw-N7; Mon, 27 May 2024 13:44:44 +0000
Received: by outflank-mailman (input) for mailman id 730770;
 Mon, 27 May 2024 13:44:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBaeZ-0007Lq-3A
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:44:43 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 44ff45ce-1c2f-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:44:42 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-5789733769dso2911176a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:44:42 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93842csm492827166b.74.2024.05.27.06.44.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 06:44:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44ff45ce-1c2f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716817481; x=1717422281; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=t31DnLWpFYc078If3GDECs6MGoQMeqPsm+twD5oOddM=;
        b=Y2bds7srz58f+Nr7GZZsE49m0Lewo0iSPkHX+WZUk/ttC3c9dHj3HVHUT+puD+qcJ3
         PKJ2GHOvfISxzskLjVvg6oLTiub1vdLAavoCo3DmfcfA7IBYe2CYdtSrFzXFg/Y5s/ON
         iisB6lVmS6ApC8CGn3n7i2hEaU0E/V/VvDLvs+2A968bO000AQwIJwoiLIlT6l9bPFoI
         PjdeUF6OueC6yqlW2YSMi7eVHqlJNLjsWsput3bR1qli7PnVWHpxtPuJULWnKmyP4MQi
         DkHIgdMXCSnyJha9iNC1rfzRbRAqRYoBqpNaJzc2eMAUvAVMvvoCuG5RKE+7NfG+vyLj
         wvcw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716817481; x=1717422281;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=t31DnLWpFYc078If3GDECs6MGoQMeqPsm+twD5oOddM=;
        b=pmaRPhillqMOyt0696Tu72E5h+JF2qcWzQ6XRXctLo4e3T/wt5ZkT+kkFm0QkUhB9J
         WDGsC4wxoVSJasKJ6kNG4BvY4Eo4aRKoQcff7/KjmdaPxVA1QdduMCvHgdCjvbGYMrS1
         Fmc5W2rpKCx2QQ6tH/M2moN0BZ5L5UsxshLe6npoXSJfUBDgkeZ74b3PIAPnPuxEmLqN
         Mv+waYaNMwY0TzUFNO0gE/QeDMxLPm7o4iyX9eVYXlnhSpwUHluLd7+OM1JwE07AeG/h
         kQxxji3vQY8BmcM1h9B6sELJ9GEkPh0YTWNmRvbWTiMHSBq0W9fSWIM0gqkbCTGBkH6m
         8Ivg==
X-Forwarded-Encrypted: i=1; AJvYcCUg/F8bX3/ar1oIWpUf4+ntUJVR6TFa2ANdEi3TbSwvrKtZ5tlRHazfb/k6aF+El9We0Jyp3N1vImCkKJuBFxyE+3oG1vpLtiJdOenZzjU=
X-Gm-Message-State: AOJu0Yx5Ak8/kr0EiQ7e6EsjUWUUYpQkbGziVAZ7LEGtrWU1IP0C7CR+
	24m9nciU1kyr3MtQI3qJypbfmA6i4nJAFkmd0emLklgG/vuk1LG462m2M7hdbw==
X-Google-Smtp-Source: AGHT+IEmK6S9V6yeBsj6KLS4hc44ghGJYb3wkQS15wa+HOWJfAMdJfqrkkkjx4uzv/3pLJAcold6tQ==
X-Received: by 2002:a17:906:a3cc:b0:a59:b17c:c9d2 with SMTP id a640c23a62f3a-a623e8d8909mr981938266b.12.1716817481244;
        Mon, 27 May 2024 06:44:41 -0700 (PDT)
Message-ID: <1cf28a31-976f-45dc-8dfb-ceecdc60cac7@suse.com>
Date: Mon, 27 May 2024 15:44:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 12/13] xen/bitops: Clean up ffs64()/fls64() definitions
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-13-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-13-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> Implement ffs64() and fls64() as plain static inlines, dropping the ifdefary
> and intermediate generic_f?s64() forms.
> 
> Add tests for all interesting bit positions at 32bit boundaries.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with two remarks:

> --- a/xen/common/bitops.c
> +++ b/xen/common/bitops.c
> @@ -24,6 +24,22 @@ static void __init test_ffs(void)
>      CHECK(ffsl, 1UL << 32, 33);
>      CHECK(ffsl, 1UL << 63, 64);
>  #endif
> +
> +    /*
> +     * unsigned int ffs64(uint64_t)
> +     *
> +     * 32-bit builds of Xen have to split this into two adjacent operations,
> +     * so test all interesting bit positions across the divide.
> +     */
> +    CHECK(ffs64, 0, 0);
> +    CHECK(ffs64, 1, 1);
> +    CHECK(ffs64, 3, 1);
> +    CHECK(ffs64, 7, 1);
> +    CHECK(ffs64, 6, 2);
> +
> +    CHECK(ffs64, 0x8000000080000000ULL, 32);
> +    CHECK(ffs64, 0x8000000100000000ULL, 33);
> +    CHECK(ffs64, 0x8000000000000000ULL, 64);

With the intermediate blank line, the respective part of the comment doesn't
look to be related to these 3 lines. Could I talk you into moving that part
down?

> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -60,6 +60,14 @@ static always_inline __pure unsigned int ffsl(unsigned long x)
>  #endif
>  }
>  
> +static always_inline __pure unsigned int ffs64(uint64_t x)
> +{
> +    if ( BITS_PER_LONG == 64 )

In principle >= 64 would be okay here, and hence I'd prefer if we used that
less strict form. Yet I'm not going to insist.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 13:50:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:50:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730777.1135970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBakM-00015L-Cq; Mon, 27 May 2024 13:50:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730777.1135970; Mon, 27 May 2024 13:50:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBakM-00015E-9w; Mon, 27 May 2024 13:50:42 +0000
Received: by outflank-mailman (input) for mailman id 730777;
 Mon, 27 May 2024 13:50:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBakL-000158-D1
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:50:41 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1a9d48cd-1c30-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:50:40 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-5785f861868so3480151a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:50:40 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579da710ef2sm838401a12.44.2024.05.27.06.50.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 06:50:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a9d48cd-1c30-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716817840; x=1717422640; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VtX0VWMZK1Ppi59ELfJ3+RZ4NX9NlUvrgou+sSJ+07Q=;
        b=JlRNsCF7II8x0Cgx9VJ4to18bQT3QHoID70nHtYCupLw7sYfQIj0pUjcvvZGis/O9b
         gFQajuPTznSuQokMQImuph4GsEg0/2HdNrcgSf1Zjz49ANFjv5ROb0d/WWwg9CSiNT72
         ffp9R6mc9xyuD8HjA9qOFmP244nE7zSkfzCnU6igdYzbYG27JQOZOMJxj5fLqSFGfEra
         32/zcRC++BF9efEWR/MEuRMiii+LOWhZ6FTKOutycuiX8i9D3faj+KbPEG/Ka43MNhjA
         Se2zekfXcygBpTfXLUOTHWQ/btNYZl7FLxjwQshs3L1E6ATKpigUQ3oJ3mmG0zalzEwn
         zncw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716817840; x=1717422640;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VtX0VWMZK1Ppi59ELfJ3+RZ4NX9NlUvrgou+sSJ+07Q=;
        b=e0adjQ39kT/Y5F5dAnEPSJmOguWcB2evxY6AVnlHp2/6CpBVckDatC6uJ2pdFHYDeJ
         e3Na8lhNvG3gInP8y5R7WZ/N7LYeaiGwBlSkJVyXi+CWJLi0VAu/QrQ6TBN5XnpUrqnv
         NcBU+KQlfkAZrho4ks90JxemISDNRwDJl5RjD+b57z6W6BYrlYACqI6NuGXpe4uWWYxz
         HZHNxuUl8aF4zEGbjkkCDvrMw6GHFFW94eZRrnNSmc6SQoXU2I8gWSwpFwW8KOAGvorB
         ytQCclWmPfMOpzPbLxBpQCD/hjHZCk0C/3vcy4NFo+iJ2gncjw+1OVCTi5kgQJRTLzZw
         PvqQ==
X-Forwarded-Encrypted: i=1; AJvYcCVLUzEPIMHY5XrgGOquawZ5zOemuo12udLVg93Oti2v7jIQanL7w5/ZR6McR86HCnFeDLrlexc/CezPb2ATpYdu7eZd79/k6EllE7ql7Ig=
X-Gm-Message-State: AOJu0Yx+Ys2iQZ8aB3rGxP5a36V6GJRsGMvx3wVP+KW0DQZYYbs43RD2
	K+1+jkBNsp6ZYV2w5e9aPNe5BogPkB7kj3e4Tav2dcEOqGjLaLgkF9BiIJuIrg==
X-Google-Smtp-Source: AGHT+IH/I9bNBnD2Hfc0Imhbdsgpo03bBo+OOKaleZk4sIOyiZSZZYJexDApaWK5ZuQWyAUSJQbnXQ==
X-Received: by 2002:a50:a41d:0:b0:578:e1fd:1978 with SMTP id 4fb4d7f45d1cf-578e1fd1c52mr4091833a12.39.1716817839891;
        Mon, 27 May 2024 06:50:39 -0700 (PDT)
Message-ID: <d3d3f01c-5bee-4684-883f-a25a3bb734f6@suse.com>
Date: Mon, 27 May 2024 15:50:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 13/13] xen/bitops: Rearrange the top of xen/bitops.h
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-14-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240524200338.1232391-14-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 22:03, Andrew Cooper wrote:
> The #include <asm/bitops.h> can move to the top of the file now now that
> generic_f?s() have been untangled.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 13:51:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:51:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730781.1135981 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBal6-0001ah-LF; Mon, 27 May 2024 13:51:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730781.1135981; Mon, 27 May 2024 13:51:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBal6-0001aa-IT; Mon, 27 May 2024 13:51:28 +0000
Received: by outflank-mailman (input) for mailman id 730781;
 Mon, 27 May 2024 13:51:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBal4-000158-Sk
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:51:26 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 35fe4ad1-1c30-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:51:26 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f45dso3239668a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:51:26 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579ced17435sm2033996a12.53.2024.05.27.06.51.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 06:51:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35fe4ad1-1c30-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716817886; x=1717422686; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=goesb8cXz2px5dcrm7tFtR9L+lofzK1L1CKjQDQyGdI=;
        b=QD8Nijtr1UYwMBNTe/utRi93AK4YW+4DdR92gBLcJ4Fqtcq64VBiEhxQ4Hd4QWnj0j
         KiQUWTnETzfy9ClDnHx9YL0h2Q0C9/nt+KOPPRjMtsSM3PDlWjRYLzwEobhOnSId9rpS
         6LVh/TGalpoccWLeXc5yoanBVoH7eYmYdx20elE9uB773SQOxdbH4x6Fxp07JAudr4sS
         mN/YrI3RN6mAv1FH5IkgOOGkQ5/vXL0N9zwJcFhyTPETSsVEWfmYPTDdInHtU4a6BqDm
         9NpKQ+70FhAKaO3stLLmd0lFPcx01PEUKpDTs+TXsu3V5A5joYh1z1kpVQy5uY3rnGHG
         0kdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716817886; x=1717422686;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=goesb8cXz2px5dcrm7tFtR9L+lofzK1L1CKjQDQyGdI=;
        b=Ys/G7WQoKJlQOxqxpBJW1qwlDgCfddYKba+oEBbnk1PfMNfBHQ+oweTXA83REJnlDg
         5jCTg3JZB9Aey200rCG2UWVTABn2lkGE7ALg9+af09WRYe8JeRDr1vn5BMeiH7rbrSKN
         ngUoQKE8Yd2jiMarus8GAWOOhlJSkN/wwk0+XlN/VbKlPXvD+iYVNjT8lMZvQ3xRjwQu
         ySZIhyx+y+I9iNWeFazcILpBrEhMF0ef3nP2R8FYcnA7VmWB98JNoUCidP8B+KbhKDRb
         BXuPKhpfokm2xD/colrp9+lDBGF9V/KFG1xMLgeK6BPY2p6EYyHMDH/kNkWBaLLrkyBz
         cCeA==
X-Forwarded-Encrypted: i=1; AJvYcCX4NQaRxGz1DbSsgQLIqMxS+GTfhpX6QPKxNx2oDstttP5Za6spv80YegnoCubijCqukvgNXKzYHS+JiCrUDXOE+XzVBOr6L1e7MpibQ+k=
X-Gm-Message-State: AOJu0YwyJi6dm1lj934wv6C7vyknowq7L2E3a4pwDXxQ71Nobv6LD/JZ
	WNgXNt+grdTHu+mInrNrlNZu1TLZ+GYrWQDqnfKl5sNiNz7BJHbF
X-Google-Smtp-Source: AGHT+IGLFqvAwRhFNeA9giLHQ+PVc4GkANuaLEKUT6l//dQ7BFtjPTVECXHxo56g5xr0tgxpt5O3uQ==
X-Received: by 2002:a50:aa84:0:b0:56d:c928:ad76 with SMTP id 4fb4d7f45d1cf-57851a8daebmr5999057a12.26.1716817885455;
        Mon, 27 May 2024 06:51:25 -0700 (PDT)
Message-ID: <494232556bdafdd96a6795638bb1376a434cb2b8.camel@gmail.com>
Subject: Re: [PATCH v2 for-4.19 00/13] xen/bitops: Untangle ffs()/fls()
 infrastructure
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>, 
 Michal Orzel <michal.orzel@amd.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, "consulting @ bugseng . com"
 <consulting@bugseng.com>, Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>, Nicola Vetrini
 <nicola.vetrini@bugseng.com>
Date: Mon, 27 May 2024 15:51:24 +0200
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

SSB0aGluayB3ZSBjYW4gY29uc2lkZXIgdG8gaGF2ZSB0aGlzIHBhdGNoIHNlcmllcyBpbiBYZW4g
NC4xOSByZWxlYXNlOgogUmVsZWFzZS1hY2tlZC1ieTogT2xla3NpaSBLdXJvY2hrbyA8b2xla3Np
aS5rdXJvY2hrb0BnbWFpbC5jb20+IAoKfiBPbGVrc2lpCk9uIEZyaSwgMjAyNC0wNS0yNCBhdCAy
MTowMyArMDEwMCwgQW5kcmV3IENvb3BlciB3cm90ZToKPiBiaXRvcHMuaCBpcyBhIG1lc3MuwqAg
SXQgaGFzIGdyb3duIG9yZ2FudGljYWxseSBvdmVyIG1hbnkgeWVhcnMsIGFuZAo+IGZvcmNlcwo+
IHVucmVhc29uYWJsZSByZXBzb25zaWJpbGl0aWVzIG91dCBpbnRvIHRoZSBwZXItYXJjaCBzdHVi
cy4KPiAKPiBTdGFydCBjbGVhbmluZyBpdCB1cCB3aXRoIGZmcygpIGFuZCBmcmllbmRzLsKgIEFj
cm9zcyB0aGUgYm9hcmQsIHRoaXMKPiBhZGRzOgo+IAo+IMKgKiBGdW5jdGlvbmluZyBiaXRvcHMg
d2l0aG91dCBhcmNoLXNwZWNpZmljIGFzbQo+IMKgKiBBbiBvcHRpb24gZm9yIGFyY2hlcyB0byBw
cm92aWRlIG1vcmUgb3B0aW1hbCBjb2RlIGdlbmVyYXRpb24KPiDCoCogQ29tcGlsZS10aW1lIGNv
bnN0YW50IGZvbGRpbmcKPiDCoCogVGVzdGluZyBhdCBib3RoIGNvbXBpbGUgdGltZSBhbmQgZHVy
aW5nIGluaXQgdGhhdCB0aGUgYmFzaWMKPiBvcGVyYXRpb25zCj4gwqDCoCBiZWhhdmUgYWNjb3Jk
aW5nIHRvIHNwZWMuCj4gCj4gYW5kIHRoZSBvbmx5IHJlYXNvbiB0aGlzIHNlcmllcyBpc24ndCBh
IG5ldCByZWR1Y3Rpb24gaW4gY29kZSBhbG9uZQo+IGlzIHRoZQo+IGJlY2F1c2Ugb2YgdGhlIG5l
dyB1bml0IHRlc3RpbmcuCj4gCj4gVGhpcyBmb3JtIGlzIHN1cGVyaW9yIGluIG1hbnkgd2F5cywg
aW5jbHVkaW5nIGdldHRpbmcgUklTQy1WIHN1cHBvcnQKPiBmb3IgZnJlZS4KPiAKPiB2MjoKPiDC
oCogTWFueSBjaGFuZ2VzLsKgIFNlZSBwYXRjaGVzIGZvciBkZXRhaWxzCj4gwqAqIEluY2x1ZGUg
dGhlIGZscygpIHNpZGUgb2YgdGhlIGluZnJhc3RydWN0dXJlIHRvby4KPiAKPiBUZXN0aW5nOgo+
IMKgCj4gaHR0cHM6Ly9naXRsYWIuY29tL3hlbi1wcm9qZWN0L3Blb3BsZS9hbmR5aGhwL3hlbi8t
L3BpcGVsaW5lcy8xMzA0NjY0NTQ0Cj4gwqAgaHR0cHM6Ly9jaXJydXMtY2kuY29tL2dpdGh1Yi9h
bmR5aGhwL3hlbi8KPiAKPiBTZXJpZXMtd2lkZSBuZXQgYmxvYXQtby1tZXRlcjoKPiAKPiDCoCB4
ODY6wqDCoCB1cC9kb3duOiA1MS8tMjQ3ICgtMTk2KQo+IMKgIEFSTTY0OiB1cC9kb3duOiA0MC8t
NDAwICgtMzYwKQo+IAo+IGFuZCBQUEM2NCByZXByb2R1Y2VkIGluIGZ1bGwsIGp1c3QgdG8gZGVt
b25zdHJhdGUgaG93IGFic3VyZCBpdCB3YXMKPiB0byBoYXZlCj4gZ2VuZXJpY19mP3MoKSBhcyBz
dGF0aWMgaW5saW5lcy4uLgo+IAo+IMKgIGFkZC9yZW1vdmU6IDEvMCBncm93L3NocmluazogMS8x
MSB1cC9kb3duOiAyMjgvLTQ4MzIgKC00NjA0KQo+IMKgIEZ1bmN0aW9uwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IG9sZMKgwqDCoMKgIG5ld8KgwqAgZGVsdGEKPiDCoCBpbml0X2NvbnN0cnVjdG9yc8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLcKgwqDC
oMKgIDIyMMKgwqDCoCArMjIwCj4gwqAgc3RhcnRfeGVuwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDkywqDCoMKg
wqAgMTAwwqDCoMKgwqDCoCArOAo+IMKgIGFsbG9jX2hlYXBfcGFnZXPCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMTk4MMKgwqDCoCAxNzQ0wqDC
oMKgIC0yMzYKPiDCoCB4ZW5oZWFwX21heF9tZm7CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDM2MMKgwqDCoMKgIDEyMMKgwqDCoCAtMjQw
Cj4gwqAgZnJlZV9oZWFwX3BhZ2VzwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA3ODTCoMKgwqDCoCA1MzbCoMKgwqAgLTI0OAo+IMKgIGZp
bmRfbmV4dF96ZXJvX2JpdMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgNTY0wqDCoMKgwqAgMjc2wqDCoMKgIC0yODgKPiDCoCBmaW5kX25leHRfYml0
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgNTQ4wqDCoMKgwqAgMjYwwqDCoMKgIC0yODgKPiDCoCBmaW5kX2ZpcnN0X3plcm9fYml0
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNDQ0wqDC
oMKgwqAgMTQ4wqDCoMKgIC0yOTYKPiDCoCBmaW5kX2ZpcnN0X2JpdMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA0NDTCoMKgwqDCoCAx
MzLCoMKgwqAgLTMxMgo+IMKgIHhtZW1fcG9vbF9mcmVlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxNzc2wqDCoMKgIDE0NDDCoMKgwqAg
LTMzNgo+IMKgIF9fZG9fc29mdGlyccKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNjA0wqDCoMKgwqAgMjUywqDCoMKgIC0zNTIK
PiDCoCBpbml0X2hlYXBfcGFnZXPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCAyMzI4wqDCoMKgIDE0MTbCoMKgwqAgLTkxMgo+IMKgIHhtZW1f
cG9vbF9hbGxvY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIDI5MjDCoMKgwqAgMTU5NsKgwqAgLTEzMjQKPiAKPiAKPiBBbmRyZXcgQ29vcGVy
ICgxMik6Cj4gwqAgcHBjL2Jvb3Q6IFJ1biBjb25zdHJ1Y3RvcnMgb24gYm9vdAo+IMKgIHhlbi9i
aXRvcHM6IENsZWFudXAgYWhlYWQgb2YgcmVhcnJhbmdlbWVudHMKPiDCoCBBUk0vYml0b3BzOiBD
aGFuZ2UgZmluZF9maXJzdF9zZXRfYml0KCkgdG8gYmUgYSBkZWZpbmUKPiDCoCB4ZW4vcGFnZV9h
bGxvYzogQ29lcmNlIG1pbihmbHNsKCksIGZvbykgZXhwcmVzc2lvbnMgdG8gYmVpbmcKPiB1bnNp
Z25lZAo+IMKgIHhlbi9iaXRvcHM6IEltcGxlbWVudCBnZW5lcmljX2Y/c2woKSBpbiBsaWIvCj4g
wqAgeGVuL2JpdG9wczogSW1wbGVtZW50IGZmcygpIGluIGNvbW1vbiBsb2dpYwo+IMKgIHg4Ni9i
aXRvcHM6IEltcHJvdmUgYXJjaF9mZnMoKSBpbiB0aGUgZ2VuZXJhbCBjYXNlCj4gwqAgeGVuL2Jp
dG9wczogSW1wbGVtZW50IGZmc2woKSBpbiBjb21tb24gbG9naWMKPiDCoCB4ZW4vYml0b3BzOiBS
ZXBsYWNlIGZpbmRfZmlyc3Rfc2V0X2JpdCgpIHdpdGggZmZzbCgpIC0gMQo+IMKgIHhlbi9iaXRv
cHM6IERlbGV0ZSBmaW5kX2ZpcnN0X3NldF9iaXQoKQo+IMKgIHhlbi9iaXRvcHM6IENsZWFuIHVw
IGZmczY0KCkvZmxzNjQoKSBkZWZpbml0aW9ucwo+IMKgIHhlbi9iaXRvcHM6IFJlYXJyYW5nZSB0
aGUgdG9wIG9mIHhlbi9iaXRvcHMuaAo+IAo+IE9sZWtzaWkgS3Vyb2Noa28gKDEpOgo+IMKgIHhl
bi9iaXRvcHM6IEltcGxlbWVudCBmbHMoKS9mbHNsKCkgaW4gY29tbW9uIGxvZ2ljCj4gCj4gwqB4
ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtMzIvYml0b3BzLmjCoMKgwqDCoMKgIHzCoMKgIDIg
LQo+IMKgeGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L2JpdG9wcy5owqDCoMKgwqDCoCB8
wqAgMTIgLS0KPiDCoHhlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9iaXRvcHMuaMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgfMKgIDM1ICstLS0KPiDCoHhlbi9hcmNoL3BwYy9pbmNsdWRlL2FzbS9iaXRv
cHMuaMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgIDE3ICstCj4gwqB4ZW4vYXJjaC9wcGMvc2V0
dXAuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDC
oCAyICsKPiDCoHhlbi9hcmNoL3g4Ni9ndWVzdC94ZW4veGVuLmPCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCB8wqDCoCA0ICstCj4gwqB4ZW4vYXJjaC94ODYvaHZtL2RvbTBfYnVpbGQu
Y8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCAyICstCj4gwqB4ZW4vYXJjaC94
ODYvaHZtL2hwZXQuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8
wqDCoCA4ICstCj4gwqB4ZW4vYXJjaC94ODYvaW5jbHVkZS9hc20vYml0b3BzLmjCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgIHwgMTE0ICsrKysrKystLS0tLS0KPiDCoHhlbi9hcmNoL3g4Ni9pbmNsdWRl
L2FzbS9wdC1jb250aWctbWFya2Vycy5oIHzCoMKgIDIgKy0KPiDCoHhlbi9hcmNoL3g4Ni9tbS5j
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHzC
oMKgIDIgKy0KPiDCoHhlbi9hcmNoL3g4Ni9tbS9wMm0tcG9kLmPCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCA0ICstCj4gwqB4ZW4vY29tbW9uL01ha2VmaWxlwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgwqAgMSAr
Cj4gwqB4ZW4vY29tbW9uL2JpdG9wcy5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgfMKgIDg5ICsrKysrKysrKysrCj4gwqB4ZW4vY29tbW9uL3BhZ2Vf
YWxsb2MuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCA2
ICstCj4gwqB4ZW4vY29tbW9uL3NvZnRpcnEuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCAyICstCj4gwqB4ZW4vZHJpdmVycy9wYXNzdGhyb3Vn
aC9hbWQvaW9tbXVfbWFwLmPCoMKgwqDCoMKgIHzCoMKgIDIgKy0KPiDCoHhlbi9kcml2ZXJzL3Bh
c3N0aHJvdWdoL2lvbW11LmPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCA0ICstCj4g
wqB4ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC94ODYvaW9tbXUuY8KgwqDCoMKgwqDCoMKgwqDCoCB8
wqDCoCA0ICstCj4gwqB4ZW4vaW5jbHVkZS94ZW4vYml0b3BzLmjCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgMTU5ICsrKysrKysrLS0tLS0tLS0tCj4gLS0KPiDCoHhl
bi9pbmNsdWRlL3hlbi9ib290LWNoZWNrLmjCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCB8wqAgNjAgKysrKysrKwo+IMKgeGVuL2luY2x1ZGUveGVuL2NvbXBpbGVyLmjCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgwqAgMyArLQo+IMKgeGVuL2xpYi9NYWtlZmls
ZcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IHzCoMKgIDIgKwo+IMKgeGVuL2xpYi9nZW5lcmljLWZmc2wuY8KgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHzCoCA2NSArKysrKysrKwo+IMKgeGVuL2xpYi9nZW5l
cmljLWZsc2wuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHzC
oCA2OCArKysrKysrKwo+IMKgMjUgZmlsZXMgY2hhbmdlZCwgNDQ0IGluc2VydGlvbnMoKyksIDIy
NSBkZWxldGlvbnMoLSkKPiDCoGNyZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vY29tbW9uL2JpdG9wcy5j
Cj4gwqBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2luY2x1ZGUveGVuL2Jvb3QtY2hlY2suaAo+IMKg
Y3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9saWIvZ2VuZXJpYy1mZnNsLmMKPiDCoGNyZWF0ZSBtb2Rl
IDEwMDY0NCB4ZW4vbGliL2dlbmVyaWMtZmxzbC5jCj4gCgo=



From xen-devel-bounces@lists.xenproject.org Mon May 27 13:58:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 13:58:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730789.1135990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBas8-0002JF-Eo; Mon, 27 May 2024 13:58:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730789.1135990; Mon, 27 May 2024 13:58:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBas8-0002J8-CE; Mon, 27 May 2024 13:58:44 +0000
Received: by outflank-mailman (input) for mailman id 730789;
 Mon, 27 May 2024 13:58:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBas8-0002J2-3S
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 13:58:44 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3a4b2887-1c31-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 15:58:43 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a61b70394c0so929036366b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 06:58:43 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cd83a1csm490519766b.182.2024.05.27.06.58.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 06:58:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a4b2887-1c31-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716818322; x=1717423122; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=HjNPVFcP/yRzwM+RW4KiStmJdpjIWyn2MZUIplWjrN0=;
        b=NkWpAyF81ZZwVMsh9hAUtk/mVsR9euRf+cfZpTIsjZxTirZG5OFmWOjNLl9/Ex7nlm
         AGEN5WWYCN4BfPYF8KzLEgqK2zOwT8qo2RNjPtaU4OjI3vvpw4keB/rBMSdTd7U4+/Pe
         ep/zXvFgUgF3l6eCiqvVKf5XfLY76AbEvnRzEiJDVD0nrz0pQTl0AOCUHfUJl0xxOHbS
         tSTH4cpkLyH05Fgg1nhjniv2DhxOlLScOVXVhFESi2sqU1bPsIyiasMBfINWM8zODscM
         iytzQ+d8YvdI7O/Q25zoPVNpUh/n5VZSxCHlIPWPElLodh0piGCMiFJRxjTZDChMms+x
         rsEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716818322; x=1717423122;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HjNPVFcP/yRzwM+RW4KiStmJdpjIWyn2MZUIplWjrN0=;
        b=cCdzWUE0u1zYknoO5aSapx/PqYh6gr/wyVbrE9Cj50+T6gFx1JQrMHcYhUIPrwY1G6
         2jVQYvCfA1gnS+xHNlG3wH80/tYSwcXApFzrU4/EujnFFxuZvlSpNuQxgdZJzPRQN/uU
         kAd/vOSuJ3h/fXeoc00rqZT2ecQyA6qnPOB7c9ziMgRfL7M5hpwl1NvewW7i/tZpUn5O
         CtYe1hOAzgBn5QUa8OO71m/YwArM/D8b2eJq4paWXVaLeEkv3Mv6BXYOvJNkNYP8BCFB
         dcRXmkm+bMS6T+GWzK2QrCVV4SmIX5VLhBjGVe3EiYtk6//sPjOL5J75yZn5W4OQX0+h
         fYxQ==
X-Gm-Message-State: AOJu0YwlOJyebwZr4cKZ6ulBZQCWJj4EK1YQDHnF07JyOD2nmwracHID
	DQ2RZdj4LbSMUo4EBg6L/KwkSaZ6GR9mHcpwfGGy/a5vfyJ9MRHxCf9ZPcYi
X-Google-Smtp-Source: AGHT+IH8djn/ZBi6vk/x6rhEleERvQbHR2nTiwFYN6Km/iqZi40BYad/dDanDVcSFL2Z15zUmx7uIw==
X-Received: by 2002:a17:906:174f:b0:a59:cb67:ceb9 with SMTP id a640c23a62f3a-a62643eb960mr552513266b.35.1716818321975;
        Mon, 27 May 2024 06:58:41 -0700 (PDT)
Message-ID: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
Subject: Code freeze date for Xen 4.19 is 31.05.2024
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Mon, 27 May 2024 15:58:41 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hi all,

I would like to remind you that the code freeze date for Xen 4.19 is
May 31, 2024.

I'm okay with bug fixes being committed without my release ack (just CC
me), except in cases where a one of maintainers gives a strong NACK.

Have a nice week!

Best regards,
Oleksii


From xen-devel-bounces@lists.xenproject.org Mon May 27 14:29:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:29:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730796.1136001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbLQ-00081s-Cx; Mon, 27 May 2024 14:29:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730796.1136001; Mon, 27 May 2024 14:29:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbLQ-00081l-9o; Mon, 27 May 2024 14:29:00 +0000
Received: by outflank-mailman (input) for mailman id 730796;
 Mon, 27 May 2024 14:28:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBbLO-00081f-UA
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:28:58 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 734ae4b3-1c35-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 16:28:56 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-579d02657abso1539364a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 07:28:56 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579c9424a83sm2466466a12.80.2024.05.27.07.28.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 07:28:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 734ae4b3-1c35-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716820136; x=1717424936; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=B5SsIbiB/UPq8UseSJXM17X13uGn/7XaU7r+r8SpVdI=;
        b=a930fbmihn8zqQGTcDWV53ap1EXavl3d7kRf0lJnhUTACAGBwkoabAZ2kK3qZqH7AE
         uzMJgr2ZAVDbrzTKpOca5WcPDb6tCKi/eBwEHuFv7vVBWnnFHMSFjNW5HRNS0HW/ZoWB
         +LIQOk8XWvleXREz1jKbYVQh5fZFAVu9lu61EPiOGgwx/z3rzrWh0Y2U7Vl1+YnsmLmj
         Cc2f1QCQdO/qjPxBBJ1YH9LvGd4boh/6ud48L3ClynchZuTW7DuxB0XHSsSQ7D12OneS
         y1ZQpucAL1pBJuSj8UufOyGmKsbU1Q+6/uYTsqVzQ4W/kib4W4vcw9+qbgSt7+2YBwux
         RskQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716820136; x=1717424936;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=B5SsIbiB/UPq8UseSJXM17X13uGn/7XaU7r+r8SpVdI=;
        b=a1LKWBUEHaRjsZi0bZ94cox/yhdWNlgS04lWQT8zz+oEbbLJ1GgCUwHR4pCrwr5t+K
         wkxt31MUzJ6iHUhzJPzJeDVOyBFImH1jqpKZmESE9bP89zFGqNgkNYtKhpipfldKiKS7
         xyP1aI4Zici2m9YZTbnF8Iszm9IuDizdbcfSXa/QK8QyFnukZAXqkns7wgb8sV0hhwpg
         9kHRzp+vN9SiW8UqkTxheLFLGIxVwDH8u5+hQzL57axFzZUwybm5oqGq/ioBxeI6eJz6
         yU740TAVfc7lRNNIETjcGBA4eJBgc8toR6Hm8+jV7lR6SAQTaOg9BuFwfomVhsP7Hebr
         xLbA==
X-Forwarded-Encrypted: i=1; AJvYcCU3B4iwIRmoNj8fkQHYa0MNuFiyiKXvGiXOQ2Kx7sHgUTeMlFVWQIW8E1Rbt2Y8mqngB8oevwbbYW3sT53GqhyC36VxWHrOvoFQ/vx5/QM=
X-Gm-Message-State: AOJu0Yw1Vm33bZJRsHyejzpS6hVDz4YsQ/ABaWT7014nP3LtNTZzQ9dF
	wOQsJ0WDJBVfNrWtddIz0VIBkv0EV0P4hyB4x8Pb444jsNmDfJTi
X-Google-Smtp-Source: AGHT+IFCw2rKjS48GZrEfhVjk2bb8DLuVjUYBBAWWZlaZ5MUTGp2boy2Lq13OcWFYHesXUuNRP/ITQ==
X-Received: by 2002:a50:9fae:0:b0:578:60a6:7c60 with SMTP id 4fb4d7f45d1cf-57860a67f55mr8244202a12.12.1716820135882;
        Mon, 27 May 2024 07:28:55 -0700 (PDT)
Message-ID: <b9c8e762af9ca04d9194fdaa0379f2fe9096af29.camel@gmail.com>
Subject: Re: [PATCH v4 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, Stefano Stabellini
	 <stefano.stabellini@amd.com>, xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, bertrand.marquis@arm.com, michal.orzel@amd.com, 
 Volodymyr_Babchuk@epam.com, Henry Wang <xin.wang2@amd.com>, Alec Kwapis
 <alec.kwapis@medtronic.com>, "Daniel P . Smith"
 <dpsmith@apertussolutions.com>
Date: Mon, 27 May 2024 16:28:54 +0200
In-Reply-To: <697aadfd-a8c1-4f1b-8806-6a5acbf343ba@xen.org>
References: 
	<alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
	 <20240524225522.2878481-2-stefano.stabellini@amd.com>
	 <697aadfd-a8c1-4f1b-8806-6a5acbf343ba@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Sat, 2024-05-25 at 00:06 +0100, Julien Grall wrote:
> Hi Stefano,
>=20
> You have sent a new version. But you didn't reply to Juergen's
> comment.
>=20
> While he is not the maintainer of the Arm code, he is the Xenstore=20
> maintainer. Even if I agree with this approach I don't feel this is=20
> right to even ack this patch without pending questions addressed.
>=20
> In this case, we are changing yet another time the sequence for=20
> initializing Xenstore dom0less domain. I would rather not want to
> have=20
> to change a 4th time...
>=20
> I don't think it is a big problem if this is not merged for the code=20
> freeze as this is technically a bug fix.
Agree, this is not a problem as it is still looks to me as a bug fix.

~ Oleksii

>=20
> Cheers,
>=20
> On 24/05/2024 23:55, Stefano Stabellini wrote:
> > From: Henry Wang <xin.wang2@amd.com>
> >=20
> > There are use cases (for example using the PV driver) in Dom0less
> > setup that require Dom0less DomUs start immediately with Dom0, but
> > initialize XenStore later after Dom0's successful boot and call to
> > the init-dom0less application.
> >=20
> > An error message can seen from the init-dom0less application on
> > 1:1 direct-mapped domains:
> > ```
> > Allocating magic pages
> > memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> > Error on alloc magic pages
> > ```
> >=20
> > The "magic page" is a terminology used in the toolstack as reserved
> > pages for the VM to have access to virtual platform capabilities.
> > Currently the magic pages for Dom0less DomUs are populated by the
> > init-dom0less app through populate_physmap(), and
> > populate_physmap()
> > automatically assumes gfn =3D=3D mfn for 1:1 direct mapped domains.
> > This
> > cannot be true for the magic pages that are allocated later from
> > the
> > init-dom0less application executed in Dom0. For domain using
> > statically
> > allocated memory but not 1:1 direct-mapped, similar error "failed
> > to
> > retrieve a reserved page" can be seen as the reserved memory list
> > is
> > empty at that time.
> >=20
> > Since for init-dom0less, the magic page region is only for
> > XenStore.
> > To solve above issue, this commit allocates the XenStore page for
> > Dom0less DomUs at the domain construction time. The PFN will be
> > noted and communicated to the init-dom0less application executed
> > from Dom0. To keep the XenStore late init protocol, set the
> > connection
> > status to XENSTORE_RECONNECT.
> >=20
> > Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> > Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> > Signed-off-by: Henry Wang <xin.wang2@amd.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > ---
> > =C2=A0 xen/arch/arm/dom0less-build.c | 55
> > ++++++++++++++++++++++++++++++++++-
> > =C2=A0 1 file changed, 54 insertions(+), 1 deletion(-)
> >=20
> > diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-
> > build.c
> > index 74f053c242..2963ecc0b3 100644
> > --- a/xen/arch/arm/dom0less-build.c
> > +++ b/xen/arch/arm/dom0less-build.c
> > @@ -1,5 +1,6 @@
> > =C2=A0 /* SPDX-License-Identifier: GPL-2.0-only */
> > =C2=A0 #include <xen/device_tree.h>
> > +#include <xen/domain_page.h>
> > =C2=A0 #include <xen/err.h>
> > =C2=A0 #include <xen/event.h>
> > =C2=A0 #include <xen/grant_table.h>
> > @@ -10,6 +11,8 @@
> > =C2=A0 #include <xen/sizes.h>
> > =C2=A0 #include <xen/vmap.h>
> > =C2=A0=20
> > +#include <public/io/xs_wire.h>
> > +
> > =C2=A0 #include <asm/arm64/sve.h>
> > =C2=A0 #include <asm/dom0less-build.h>
> > =C2=A0 #include <asm/domain_build.h>
> > @@ -739,6 +742,53 @@ static int __init alloc_xenstore_evtchn(struct
> > domain *d)
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
> > =C2=A0 }
> > =C2=A0=20
> > +#define XENSTORE_PFN_OFFSET 1
> > +static int __init alloc_xenstore_page(struct domain *d)
> > +{
> > +=C2=A0=C2=A0=C2=A0 struct page_info *xenstore_pg;
> > +=C2=A0=C2=A0=C2=A0 struct xenstore_domain_interface *interface;
> > +=C2=A0=C2=A0=C2=A0 mfn_t mfn;
> > +=C2=A0=C2=A0=C2=A0 gfn_t gfn;
> > +=C2=A0=C2=A0=C2=A0 int rc;
> > +
> > +=C2=A0=C2=A0=C2=A0 if ( (UINT_MAX - d->max_pages) < 1 )
> > +=C2=A0=C2=A0=C2=A0 {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 printk(XENLOG_ERR "%pd: Ove=
r-allocation for d->max_pages
> > by 1 page.\n",
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 d);
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -EINVAL;
> > +=C2=A0=C2=A0=C2=A0 }
> > +=C2=A0=C2=A0=C2=A0 d->max_pages +=3D 1;
> > +=C2=A0=C2=A0=C2=A0 xenstore_pg =3D alloc_domheap_page(d, MEMF_bits(32)=
);
> > +=C2=A0=C2=A0=C2=A0 if ( xenstore_pg =3D=3D NULL && is_64bit_domain(d) =
)
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xenstore_pg =3D alloc_domhe=
ap_page(d, 0);
> > +=C2=A0=C2=A0=C2=A0 if ( xenstore_pg =3D=3D NULL )
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -ENOMEM;
> > +
> > +=C2=A0=C2=A0=C2=A0 mfn =3D page_to_mfn(xenstore_pg);
> > +=C2=A0=C2=A0=C2=A0 if ( !mfn_x(mfn) )
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -ENOMEM;
> > +
> > +=C2=A0=C2=A0=C2=A0 if ( !is_domain_direct_mapped(d) )
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gfn =3D gaddr_to_gfn(GUEST_=
MAGIC_BASE +
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 (XENSTORE_PFN_OFFSET << PAGE_SHIFT));
> > +=C2=A0=C2=A0=C2=A0 else
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gfn =3D gaddr_to_gfn(mfn_to=
_maddr(mfn));
> > +
> > +=C2=A0=C2=A0=C2=A0 rc =3D guest_physmap_add_page(d, gfn, mfn, 0);
> > +=C2=A0=C2=A0=C2=A0 if ( rc )
> > +=C2=A0=C2=A0=C2=A0 {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 free_domheap_page(xenstore_=
pg);
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return rc;
> > +=C2=A0=C2=A0=C2=A0 }
> > +
> > +=C2=A0=C2=A0=C2=A0 d->arch.hvm.params[HVM_PARAM_STORE_PFN] =3D gfn_x(g=
fn);
> > +=C2=A0=C2=A0=C2=A0 interface =3D map_domain_page(mfn);
> > +=C2=A0=C2=A0=C2=A0 interface->connection =3D XENSTORE_RECONNECT;
> > +=C2=A0=C2=A0=C2=A0 unmap_domain_page(interface);
> > +
> > +=C2=A0=C2=A0=C2=A0 return 0;
> > +}
> > +
> > =C2=A0 static int __init construct_domU(struct domain *d,
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const struct d=
t_device_node
> > *node)
> > =C2=A0 {
> > @@ -839,7 +889,10 @@ static int __init construct_domU(struct domain
> > *d,
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rc =3D alloc_xen=
store_evtchn(d);
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( rc < 0 )
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 return rc;
> > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d->arch.hvm.params[HVM_PARA=
M_STORE_PFN] =3D ~0ULL;
> > +
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rc =3D alloc_xenstore_page(=
d);
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( rc < 0 )
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret=
urn rc;
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }
> > =C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return rc;
>=20



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:48:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:48:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730801.1136011 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbe1-00039g-Ml; Mon, 27 May 2024 14:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730801.1136011; Mon, 27 May 2024 14:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbe1-00039Z-K5; Mon, 27 May 2024 14:48:13 +0000
Received: by outflank-mailman (input) for mailman id 730801;
 Mon, 27 May 2024 14:48:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=9k1s=M6=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sBbe0-00039T-8V
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:48:12 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 21d56d6c-1c38-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 16:48:08 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso948315566b.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 07:48:08 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93b54esm501293266b.89.2024.05.27.07.48.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 07:48:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21d56d6c-1c38-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716821288; x=1717426088; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=G/akndZUowcan2n2TtFR3JQ/wHLAG3/zGHe8EITN8Q0=;
        b=ZwtGp4RFggH0dwZkAKvoVLaNIVkkcwM3rCFQqxm285OyafQpBnwTxXG79+K/yz7rOC
         nO2GUNVkxMG57ZbNOvy/yIp1FKG6ECabNgs3dzAIbzBH4ezRfm2Ai+x1YnCgb5tWotkd
         oUydQn6SH2YkJi8+6yfwC6smd5mDiP5DFtdfTomCnDEtQM111ejx9cBq9VZHVt2YYxuf
         JX9GgQ4VF97qr/gUY690tOmtuaQd/wYdgIlkK6RvJx4B0OWgMXV2p178yQwxm2FePs/0
         /Pec0LA4bi9HzOxl3/1mcr4AffharrdhqNQN0G59544AHUTVDHwuCQehWJwwh/TzghE0
         49Zg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716821288; x=1717426088;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=G/akndZUowcan2n2TtFR3JQ/wHLAG3/zGHe8EITN8Q0=;
        b=P6aYxmzB3pniurxYmxPQjwK/e3CKduY7kjlkGeDY+q6zPArNFtGWaMXibnYFiDyZvk
         +2YSMoWQlazwHTYJC3xRYS2+lh4PX1c7B5uEa6lxIZRZWJgtXnG0Ln+3lKluLDm4J+8R
         OfPBPFdsZcV59GBJAAG0ecNx5sY1lGK+9pjYjUwzpOYWFBO8+kX3kHqJNH81+b/0G1wz
         MRGZZS0Notgd9Ec4MK6cmHM5oxV/+mtUaGMffM159hcliY6BLiPk1sZySfI8C3dMYJRp
         EbWPxVRS5aYU/YlH/aaFaAn7ZeBs0+a17C/RZUjnx60NJ0911Aj/1aAuHMtxy44dvEIU
         f6Tg==
X-Forwarded-Encrypted: i=1; AJvYcCVMtzJ2U0im8E/PPLey+XHdUI4G2JPvn1PH22MADb15pmcVTgeojzy/v1AYGb3RMtYyek3/QH9CI5zauTUBu4QYmpCCSRf0l/BSryzgbo8=
X-Gm-Message-State: AOJu0YwGr6w//6Pix/YqfSt6689vEfy4TlNhTnKr1vIbY8Ak+R1Cvqan
	J68I1a/UvftUAqnn8Q6PVC1V+H+1VFg5oWUz0Vw3rxuOnrJMQMsLyx37Fq4XOiI=
X-Google-Smtp-Source: AGHT+IGUGz+k/k1nUKygyac88+AdCLKn3pLf2n7A+4lnDgtaO/UkXgOwKt+pgC6cahMXyr427gbZKQ==
X-Received: by 2002:a17:906:69d0:b0:a59:c62c:344d with SMTP id a640c23a62f3a-a623e8d879emr976668066b.9.1716821287868;
        Mon, 27 May 2024 07:48:07 -0700 (PDT)
Message-ID: <480ccee9-a50a-4b30-92c4-3f39e4202bca@suse.com>
Date: Mon, 27 May 2024 16:48:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, Henry Wang <xin.wang2@amd.com>,
 xen-devel@lists.xenproject.org, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240517032156.1490515-1-xin.wang2@amd.com>
 <20240517032156.1490515-3-xin.wang2@amd.com>
 <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org>
 <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com>
 <alpine.DEB.2.22.394.2405241614370.2557291@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <alpine.DEB.2.22.394.2405241614370.2557291@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 25.05.24 01:19, Stefano Stabellini wrote:
> On Fri, 24 May 2024, Jürgen Groß wrote:
>> On 24.05.24 15:58, Julien Grall wrote:
>>> Hi Henry,
>>>
>>> + Juergen as the Xenstore maintainers. I'd like his opinion on the approach.
>>> The documentation of the new logic is in:
>>>
>>> https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
>>>
>>> FWIW I am happy in principle with the logic (this is what we discussed on
>>> the call last week). Some comments below.
>>
>> I'm not against this logic, but I'm wondering why it needs to be so
>> complicated.
> 
> Actually the reason I like it is that in my view, this is the simplest
> approach. You allocate a domain, you also allocate the xenstore page
> together with it. Initially the xenstore connection has an
> "uninitialized" state, as it should be. That's it. At some point, when
> xenstored is ready, the state changes to CONNECTED.
> 
> 
>> Can't the domU itself allocate the Xenstore page from its RAM pages,
>> write the PFN into the Xenstore grant tab entry, and then make it
>> public via setting HVM_PARAM_STORE_PFN?
> 
> This is not simpler in my view

Okay, fine with me. I had the impression that violating the 1:1 mapping
of the domain would add complexity, but if you are fine with that I don't
mind your approach.


Juergen



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730808.1136046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj7-0005d3-2U; Mon, 27 May 2024 14:53:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730808.1136046; Mon, 27 May 2024 14:53:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj6-0005cb-Rp; Mon, 27 May 2024 14:53:28 +0000
Received: by outflank-mailman (input) for mailman id 730808;
 Mon, 27 May 2024 14:53:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uY6H=M6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sBbj5-00056W-BA
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:53:27 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df545d30-1c38-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 16:53:26 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.194])
 by support.bugseng.com (Postfix) with ESMTPSA id AE69A4EE0744;
 Mon, 27 May 2024 16:53:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df545d30-1c38-11ef-90a1-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 4/4] x86/traps: address violation of MISRA C Rule 8.4
Date: Mon, 27 May 2024 16:53:19 +0200
Message-Id: <9d33b2c2aeb59ca67136ed1b65c06e6ef4fd1ed8.1716814609.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716814609.git.nicola.vetrini@bugseng.com>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 8.4 states: "A compatible declaration shall be visible when
an object or function with external linkage is defined".

The function do_general_protection is either used is asm code
or only within this unit, so there is no risk of this getting
out of sync with its definition, but the function must remain
extern.

Therefore, this function is deviated using a comment-based deviation.
No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/arch/x86/traps.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 468a03608102..9906e874d593 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1257,6 +1257,7 @@ void asmlinkage do_int3(struct cpu_user_regs *regs)
     pv_inject_hw_exception(X86_EXC_BP, X86_EVENT_NO_EC);
 }
 
+/* SAF-1-safe */
 void do_general_protection(struct cpu_user_regs *regs)
 {
 #ifdef CONFIG_PV
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730806.1136026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj5-00059F-Ig; Mon, 27 May 2024 14:53:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730806.1136026; Mon, 27 May 2024 14:53:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj5-00058P-CQ; Mon, 27 May 2024 14:53:27 +0000
Received: by outflank-mailman (input) for mailman id 730806;
 Mon, 27 May 2024 14:53:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uY6H=M6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sBbj4-00056W-LY
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:53:26 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df0196c3-1c38-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 16:53:25 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 2B1544EE0745;
 Mon, 27 May 2024 16:53:25 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df0196c3-1c38-11ef-90a1-e314d9c70b13
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 3/4] x86: address violations of MISRA C Rule 8.4
Date: Mon, 27 May 2024 16:53:18 +0200
Message-Id: <cf926194a541d11e02670516a8d337de27836dce.1716814609.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716814609.git.nicola.vetrini@bugseng.com>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 8.4 states: "A compatible declaration shall be visible when
an object or function with external linkage is defined."

These variables are only referenced from asm modules, so they
need to be extern and there is negligible risk of them being
used improperly without noticing.

As a result, they can be exempted using a comment-based deviation.
No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
Adding the asmlinkage macro to variables is not appropriate, as
this pseudo-attribute may expand, for instance, to a different calling
convention in the future (e.g. stdcall)
---
 xen/arch/x86/desc.c | 1 +
 xen/arch/x86/mm.c   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/desc.c b/xen/arch/x86/desc.c
index 39080ca67211..9f639281540a 100644
--- a/xen/arch/x86/desc.c
+++ b/xen/arch/x86/desc.c
@@ -91,6 +91,7 @@ seg_desc_t boot_compat_gdt[PAGE_SIZE / sizeof(seg_desc_t)] =
  * References boot_cpu_gdt_table for a short period, until the CPUs switch
  * onto their per-CPU GDTs.
  */
+/* SAF-1-safe */
 const struct desc_ptr boot_gdtr = {
     .limit = LAST_RESERVED_GDT_BYTE,
     .base = (unsigned long)(boot_gdt - FIRST_RESERVED_GDT_ENTRY),
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d968bbbc7315..17987eb5199e 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -144,7 +144,7 @@
 l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE)
     l1_fixmap[L1_PAGETABLE_ENTRIES];
 l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE)
-    l1_fixmap_x[L1_PAGETABLE_ENTRIES];
+    l1_fixmap_x[L1_PAGETABLE_ENTRIES]; /* SAF-1-safe */
 
 bool __read_mostly machine_to_phys_mapping_valid;
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730807.1136041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj6-0005Zh-O1; Mon, 27 May 2024 14:53:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730807.1136041; Mon, 27 May 2024 14:53:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj6-0005Za-KI; Mon, 27 May 2024 14:53:28 +0000
Received: by outflank-mailman (input) for mailman id 730807;
 Mon, 27 May 2024 14:53:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uY6H=M6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sBbj5-00056V-8W
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:53:27 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id de628393-1c38-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 16:53:24 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 066114EE0742;
 Mon, 27 May 2024 16:53:23 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de628393-1c38-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Subject: [XEN PATCH 1/4] docs/misra: exclude gdbsx from MISRA compliance
Date: Mon, 27 May 2024 16:53:16 +0200
Message-Id: <83d89e6fa141bb91d7716cf33f086edf9a6d7ae1.1716814609.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716814609.git.nicola.vetrini@bugseng.com>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These files are used when debugging Xen, and are not meant to comply
with MISRA rules at the moment.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 docs/misra/exclude-list.json | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
index cd6976542793..2567e8467c78 100644
--- a/docs/misra/exclude-list.json
+++ b/docs/misra/exclude-list.json
@@ -77,6 +77,14 @@
             "rel_path": "arch/x86/dmi_scan.c",
             "comment": "Imported from Linux, ignore for now"
         },
+        {
+            "rel_path": "arch/x86/gdbsx.c",
+            "comment": "Used for debugging Xen, ignore for now"
+        },
+        {
+            "rel_path": "arch/x86/include/asm/gdbsx.h",
+            "comment": "Used for debugging Xen, ignore for now"
+        },
         {
             "rel_path": "arch/x86/mpparse.c",
             "comment": "Imported from Linux, ignore for now"
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730809.1136061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj8-00063Y-BX; Mon, 27 May 2024 14:53:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730809.1136061; Mon, 27 May 2024 14:53:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj8-00062i-7I; Mon, 27 May 2024 14:53:30 +0000
Received: by outflank-mailman (input) for mailman id 730809;
 Mon, 27 May 2024 14:53:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uY6H=M6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sBbj6-00056V-8h
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:53:28 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id deaf5cdf-1c38-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 16:53:25 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.194])
 by support.bugseng.com (Postfix) with ESMTPSA id A1BF94EE0743;
 Mon, 27 May 2024 16:53:24 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: deaf5cdf-1c38-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH 2/4] automation/eclair_analysis: avoid an ECLAIR warning about escaping
Date: Mon, 27 May 2024 16:53:17 +0200
Message-Id: <f489cc3d668830eab2f2ce724164d65dc623baa7.1716814609.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716814609.git.nicola.vetrini@bugseng.com>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The parentheses in this regular expression should be doubly
escaped because they are undergo escaping twice.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index b9b377c56b25..cf62a874d928 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -405,8 +405,8 @@ explicit comment indicating the fallthrough intention is present."
 #
 
 -doc_begin="printf()-like functions are allowed to use the variadic features provided by stdarg.h."
--config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\(.*\)$)))"}
--config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\(.*\)$)))"}
+-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\\(.*\\)$)))"}
+-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\\(.*\\)$)))"}
 -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(panic)&&kind(function))))"}
 -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(elf_call_log_callback)&&kind(function))))"}
 -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(vprintk_common)&&kind(function))))"}
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon May 27 14:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 14:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730805.1136021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj5-00056o-9S; Mon, 27 May 2024 14:53:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730805.1136021; Mon, 27 May 2024 14:53:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBbj5-00056h-6G; Mon, 27 May 2024 14:53:27 +0000
Received: by outflank-mailman (input) for mailman id 730805;
 Mon, 27 May 2024 14:53:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uY6H=M6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sBbj4-00056V-Jq
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 14:53:26 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id de1a8f6f-1c38-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 16:53:24 +0200 (CEST)
Received: from nico.bugseng.com (unknown [46.228.253.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 7E3D44EE0738;
 Mon, 27 May 2024 16:53:22 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de1a8f6f-1c38-11ef-b4bb-af5377834399
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [XEN PATCH 0/4] various ECLAIR and MISRA improvements
Date: Mon, 27 May 2024 16:53:15 +0200
Message-Id: <cover.1716814609.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi all,

this series contains various miscellaneous changes to the ECLAIR and deviations
for MISRA rules

Nicola Vetrini (4):
  docs/misra: exclude gdbsx from MISRA compliance
  automation/eclair_analysis: avoid an ECLAIR warning about escaping
  x86: address violations of MISRA C Rule 8.4
  x86/traps: address violation of MISRA C Rule 8.4

 automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++--
 docs/misra/exclude-list.json                     | 8 ++++++++
 xen/arch/x86/desc.c                              | 1 +
 xen/arch/x86/mm.c                                | 2 +-
 xen/arch/x86/traps.c                             | 1 +
 5 files changed, 13 insertions(+), 3 deletions(-)

-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Mon May 27 15:12:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:12:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730835.1136072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc15-00027H-TS; Mon, 27 May 2024 15:12:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730835.1136072; Mon, 27 May 2024 15:12:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc15-00027A-Ol; Mon, 27 May 2024 15:12:03 +0000
Received: by outflank-mailman (input) for mailman id 730835;
 Mon, 27 May 2024 15:12:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBc15-00026J-6Y
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:12:03 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 783bf1c7-1c3b-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 17:12:01 +0200 (CEST)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2e95a74d51fso52518141fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:12:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579c2026406sm3047825a12.37.2024.05.27.08.12.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:12:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 783bf1c7-1c3b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716822721; x=1717427521; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LYVU3Lt0wdYOJAEy0vlDaUBkPVEZk/L21I2tclE2jO8=;
        b=flG5S/HjscMcrgSuVhGZqauL3feNaTcxgAtlHMfCDEyJ6C1ERJ6ZSyYna45fIM7j8r
         6CDG+6q7bpiKjDZ2uZJfq0MxyWZaxR8cgzu6xYrbwBdvqDVpdSpDfetbz0j+BFq0L93U
         lbQy9wR9PPZepXlJGI4letSU0NBjTAfySo/YjLa8LrDmZcHLhAogQPs7XxL2Zz4Z0zFL
         iM0PGxTzqsiEtiBnNxGxAc0ry+Y8zUU6HjpF+iHOOeHrK/cI8kuAOyz+nwTbC8jcZVaL
         n9MPuMXVA9eBk/36wSYzWMIi1C33H0XxoEcEIUuguGjlx4DlP2llutTjn4mZb7d14s5B
         haqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716822721; x=1717427521;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LYVU3Lt0wdYOJAEy0vlDaUBkPVEZk/L21I2tclE2jO8=;
        b=pDgJrJBvS4bK5fl+SuFpk7C4g26JC4JA3VVHlXLatVJMnseeQpbwEbrhhswtk96gU2
         ordMBAvD9nHtx85KHNSPwsjy8rYMpuvi/+wZjYDbd+t7/kN0PLRKodQaPKzMFsxjsSb8
         OIC99GFj4L7XcKp3Dz15XhXr1T/yJvvtcuwEYNcEKUnq55WWjALyCPv4thiBs6ffviQ1
         6HtZw7cmaAPzVEsBkqazZHQWTvt/n0QdfbuGcB+zKd2gtkkgx4Fz7vNsVKQoOz8lUJu6
         hhQFDAaRCMxPKa5A0LrZV0zFOEKKOnTzeJWp9/fdmPVkTY2S3K21blzM3aN7VVHgIltg
         Dbpg==
X-Forwarded-Encrypted: i=1; AJvYcCXC9a0j4FsF7gxbbLoMN73U/hjjN07hkl9ajDvATSzuzDuWQvUgx6/w8q5dBoFBuAwTPueKVuh80zxwL7uyX1fH7qrqCCNaq1jhS4Es7Tw=
X-Gm-Message-State: AOJu0YxB5Kzr7tYscBKfHNZpsOYpOStyZ1nswH3BR/kW1gNo3rP71dv9
	E7ZhDfjVxszqspN1kmtdSQiPCGSv9cJqUE6QOSbYAXlyKsGXEBn0TW434w4oSA==
X-Google-Smtp-Source: AGHT+IEpMcRLYjqAJagDX3kfhhlSZrUOaZImRArzaE/Yq/y8UWXivhWf48ZgZA75BI2JVseVYBNY8g==
X-Received: by 2002:a2e:9e04:0:b0:2d8:5fe6:820d with SMTP id 38308e7fff4ca-2e95b0411c8mr76249161fa.11.1716822721194;
        Mon, 27 May 2024 08:12:01 -0700 (PDT)
Message-ID: <a1e65a79-d8a4-4b8a-ad4d-a18544389e5a@suse.com>
Date: Mon, 27 May 2024 17:12:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Code freeze date for Xen 4.19 is 31.05.2024
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
References: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 15:58, Oleksii K. wrote:
> I would like to remind you that the code freeze date for Xen 4.19 is
> May 31, 2024.

I may be confused: With feature freeze having been last Friday aiui, isn't
this a little too short a period? I was actually expecting a longer-than-
normal one to account for the Xen Summit week.

Jan

> I'm okay with bug fixes being committed without my release ack (just CC
> me), except in cases where a one of maintainers gives a strong NACK.
> 
> Have a nice week!
> 
> Best regards,
> Oleksii



From xen-devel-bounces@lists.xenproject.org Mon May 27 15:12:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:12:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730838.1136080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc1k-0002aH-3y; Mon, 27 May 2024 15:12:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730838.1136080; Mon, 27 May 2024 15:12:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc1k-0002aA-14; Mon, 27 May 2024 15:12:44 +0000
Received: by outflank-mailman (input) for mailman id 730838;
 Mon, 27 May 2024 15:12:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBc1i-00026J-KM
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:12:42 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90263c40-1c3b-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 17:12:41 +0200 (CEST)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2e724bc46c4so90426741fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:12:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579c2026406sm3047825a12.37.2024.05.27.08.12.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:12:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90263c40-1c3b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716822761; x=1717427561; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P7Jh/8RdqOL74XKYE/SJmV4ggngj0/VZwa7qcHlFTbw=;
        b=gRldI3ufelrr55TEE4cxVerhxnlAFa0dOrX9HaDNM3ZNFcIYBlQou8ymC+oQQphMSH
         Tco3gje52AfwSWn2NXAhyEPr69Td1I6dXnzta0i20tRhfhyJfv4KBKggVw/AznKN03og
         Oi3i8m75AExPjZY3fLBToCA6ACqra/1BZCy6zALhLCq3ozQbtCQXH59jR3O+66McTL/I
         Dc2VheYtmLkFHEqmCy1yenuis/ylPpesOV/Z5z1dMn/VVWh+BLBTzUqlHCYt5fJauny3
         rmtD+y1u4qz7icQKSRNkHKmfsFv2D8139xsqQkkd4V79xnKeRkedEFd53BWtwFlywDD/
         Znww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716822761; x=1717427561;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P7Jh/8RdqOL74XKYE/SJmV4ggngj0/VZwa7qcHlFTbw=;
        b=rwH9W+6d7oIkjVPYkLxKe1KijDSrpQwihTdLTB39m3X+yiVKRZSGzPfyOEZ+hUJAmx
         Qzg7wcmDuw/hKb8lA6+Uepw2iJT1vZsQmsiYcwVh9+CxN85i0I+Do24SzRHsLnOODaKl
         IVneGhK5Kyfq2KGd7qXyeeRKBaDeuTsEh3EaBJFL+bCb/fK64teNbh7SIsf3m6FP+Z4C
         BkMec9wwfmxy9Adw1zhpWNVU9hH9l9pA+RuznAZ6mJ9Ypk+WeTaJ7Z0dN2HNETdLleZe
         uajo/wKYOcM4kp/FgAI3hQJfSFJyAp4GTetcjIPPDcFjPGBGxQK50qFukmtYpCkBBNVt
         +x+Q==
X-Forwarded-Encrypted: i=1; AJvYcCVc3DS3nyBikrDS0BlAfa6LaiRveM480nQabMabofjmmGJIfXqw5josVs1BB15p5UjgpaKpy8kkiaAmVuYOh2T3zLW0mvUbwitthavlVcM=
X-Gm-Message-State: AOJu0YzbVMiXz/lmAw2FM099aZ4mcw6qv3BaoPRPPjtcQzbPzFm4JWNp
	1L3UTm28l9Iya+P5KSMrlduqIoKTGQLaEHJjwQL6go1VGy+5LWRGnpi6WUZYng==
X-Google-Smtp-Source: AGHT+IGc+OVBxYrQlplVJwO4XOrS+Wk+NMzdMI/oNmtoW1i03lV/kEhbp83lB93lobjbNu2VZ39jbg==
X-Received: by 2002:a2e:a265:0:b0:2e6:f4c1:31e5 with SMTP id 38308e7fff4ca-2e95b209930mr61950551fa.23.1716822761469;
        Mon, 27 May 2024 08:12:41 -0700 (PDT)
Message-ID: <b2aa5bc0-8566-40f2-88b9-523b9b8fef3f@suse.com>
Date: Mon, 27 May 2024 17:12:40 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/4] docs/misra: exclude gdbsx from MISRA compliance
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
 <83d89e6fa141bb91d7716cf33f086edf9a6d7ae1.1716814609.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <83d89e6fa141bb91d7716cf33f086edf9a6d7ae1.1716814609.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 16:53, Nicola Vetrini wrote:
> These files are used when debugging Xen, and are not meant to comply
> with MISRA rules at the moment.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 15:13:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:13:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730841.1136092 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc2W-00038T-Dg; Mon, 27 May 2024 15:13:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730841.1136092; Mon, 27 May 2024 15:13:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBc2W-00038M-9A; Mon, 27 May 2024 15:13:32 +0000
Received: by outflank-mailman (input) for mailman id 730841;
 Mon, 27 May 2024 15:13:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBc2U-00037V-Pr
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:13:30 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aceacbfe-1c3b-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 17:13:30 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f45eso2560281a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:13:30 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579c2026406sm3047825a12.37.2024.05.27.08.13.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:13:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aceacbfe-1c3b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716822810; x=1717427610; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VvbTbEsKmn4mnDBuGKxl6geDipRxQV4dq73yiizKjYY=;
        b=bWzRVmhfOIi0HOsmOI+0bN2sSHfZGq3uilo1VldEtqYZRp35+gO9Y9JaMu2Uz7D4s9
         pYG85VPs+ws9NFhW52DRF+oxEeHDrRhbnJhAw9RJyZWZ9AIBNMSRctZup83TWtrfglTk
         Qs1321y5wxijNDOdBzO0VUovAdr1z2tb81YITwuFSeLl2YkGg+8dpuyeBCPzyUkNhJfl
         AJAbLNmimpLziLbwjXn3F5U+JyW7wtjiR9VvrAdlQJCJM9kFz9Ct146rSXsCgsdL03ru
         ezWhq3/QvKux1/fLpumyqGMyizPbawnR70QFu+SQFdo+SstRPfXsy0jaoCzcrihkDnz8
         9CpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716822810; x=1717427610;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VvbTbEsKmn4mnDBuGKxl6geDipRxQV4dq73yiizKjYY=;
        b=gyAClc0k3AD+Uye1qFwwXObDkPnkb5MtXr0I3Hgs6rjlcMuImOOzA/mgb2DSvV8XHU
         sfG5EKF/8+70RIELj5THcuUXEJXn/PVXwwN+zP7NaN3V3tLZtYJCtXvOvAYCqz8GtEDX
         aRK8Be9CyapMirPn5B5S469rEjVbj+zjqmc3YoxlrWmfbDacZCAQu8YYl2AUI9FEbQnh
         CaKZk0eWGhTcw4U84LV3W64h0CopMVvLe1sVwSuvHvGbQon8ZZjSwpie7YtzuhkE5sdC
         vTtko1Y9rkqhDt5YK/UaBvaIh8yP4o79Am7pK5Xogh0meCZZecn/ZjQo3oZrVsx+6inf
         cPmg==
X-Forwarded-Encrypted: i=1; AJvYcCVmBbWNEWScqRBaIbTum5k7Ag5gCpALVZ79Wh3Gyild5VX3zkNaRFtJ5C6Mr5DfoskLKd6TNoomD/FEzV7x4H11VQPXub57yIvjhFgzL40=
X-Gm-Message-State: AOJu0YwT5Zrt8DGiqGmTw+u4P5XlHVr5OgyBw1k7PDRVDWgQwmn2TpG5
	6/4tMXJ+ONecUqLGn+qzdmnGyf7o+HUJZWzQt/5IOoHuxZ5uCOfVj1FyzOJlhw==
X-Google-Smtp-Source: AGHT+IE/uJGMOHzeNk5N+KyK1QM5Qndle606c2a6aXzqOCAXMvaPXORbHOa/tEReLnSkVKQvOgx6UA==
X-Received: by 2002:a50:d655:0:b0:578:6410:1d50 with SMTP id 4fb4d7f45d1cf-57864101db2mr7029360a12.34.1716822809714;
        Mon, 27 May 2024 08:13:29 -0700 (PDT)
Message-ID: <86e1e74b-f1e3-4e16-b87e-a9c512dcc655@suse.com>
Date: Mon, 27 May 2024 17:13:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/4] automation/eclair_analysis: avoid an ECLAIR
 warning about escaping
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
 <f489cc3d668830eab2f2ce724164d65dc623baa7.1716814609.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f489cc3d668830eab2f2ce724164d65dc623baa7.1716814609.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 16:53, Nicola Vetrini wrote:
> The parentheses in this regular expression should be doubly
> escaped because they are undergo escaping twice.

Do you maybe mean "undergo expansion twice"?

Jan

> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
>  automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> index b9b377c56b25..cf62a874d928 100644
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -405,8 +405,8 @@ explicit comment indicating the fallthrough intention is present."
>  #
>  
>  -doc_begin="printf()-like functions are allowed to use the variadic features provided by stdarg.h."
> --config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\(.*\)$)))"}
> --config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\(.*\)$)))"}
> +-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\\(.*\\)$)))"}
> +-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\\(.*\\)$)))"}
>  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(panic)&&kind(function))))"}
>  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(elf_call_log_callback)&&kind(function))))"}
>  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(vprintk_common)&&kind(function))))"}



From xen-devel-bounces@lists.xenproject.org Mon May 27 15:41:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:41:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730854.1136100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcTt-0007qP-GU; Mon, 27 May 2024 15:41:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730854.1136100; Mon, 27 May 2024 15:41:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcTt-0007qI-Dn; Mon, 27 May 2024 15:41:49 +0000
Received: by outflank-mailman (input) for mailman id 730854;
 Mon, 27 May 2024 15:41:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBcTs-0007qC-2O
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:41:48 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9f61ece2-1c3f-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 17:41:45 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-5786988ae9bso2821516a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:41:45 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda1a97sm504051866b.188.2024.05.27.08.41.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:41:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f61ece2-1c3f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716824505; x=1717429305; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7DJhBKEUSk4v7qdZPx9Ged9Zc+8cyZuGmrSSTuulOfQ=;
        b=AnhDEnZpC+vyZ6iOcHzUrmKyAhdReZMMnISF1t7Pp1GwpDxt+W5nh9w5XWje1DtXmg
         xovk6I8Uf5nUpmWGNZqPUlK++vpv05DzR62+AxLqzh0yX9wDK0lnzxSKKRbSviKyTyHJ
         K6I+MCyBdn7iMQ8QgmpmqdiYZc6H7M/Jslrj0E7kbxVjOn05+kAXR4hWnLWoYFjyH3z1
         m4KEMQ9bzJMTCcAgR6fB+3WFPEeSIE5+OCQseKvpiQkCN5ayp9mEVKh+Rsk4engXPf94
         AW2BgKWfDBtQge2N1G3w0yAAqLA521cZJ9RhYCNUsoTbvFVzaw1j5vXVLBQNDfjaoESq
         ukfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716824505; x=1717429305;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7DJhBKEUSk4v7qdZPx9Ged9Zc+8cyZuGmrSSTuulOfQ=;
        b=cTLYQgW8haImOThxzQep8jurmr/arn3OOQC1sibWaIbIQEXuOMmajrgs9P8acIZmqW
         PjM719lqf5u8VPz7jRoeLLdCzJ/XNdIFfYih7e6pisg+curM4zjGxGU47jOmzv3YdThZ
         RRLdnPCOp7CsIIZEsGqJCs7XKDR6bW3R0F7mQ8Gn/PGSqjlcgfMvUqSNFf29oww2H5m2
         NMCrxeISVw/JGo5adIzR+IbXat3fydBKBBWoS8DYdCuITgWzIYiVe67BtQOt4K/+j+T3
         CdoXSjE62DgLnki4wAB3QU0+dvXDIbO02ZEJP16a/HDrPe8D66Spv8ntPDpx9pAFnY1k
         VsWQ==
X-Forwarded-Encrypted: i=1; AJvYcCWpothdlCri2LQ+i2PBLRpbDlcxiV6twww9JxVcl/nUXwcWMQPLK2wkzIZS+P1ejfm2tkT+VShT7vA34WELGhuiDMwn4BMTx+kH/KmaaIk=
X-Gm-Message-State: AOJu0YyKko1Py5o9R92HC3234Dm9V5HY710RQlcUFCBrbp3H9O2mYAvB
	SviEwEGZsoWZ5zg/UgBcWkSHRY1wMEuySNIEF9+lueHf1wSNIvo/2UEsrz60GA==
X-Google-Smtp-Source: AGHT+IExvN7gqoCunX6bfygeh3BZNTKhOuZOHrs5O/re1wjYAwv6DFKqXekcGMauBQUw7BNG18wZhg==
X-Received: by 2002:a17:907:350f:b0:a5d:107b:60be with SMTP id a640c23a62f3a-a62650116c9mr532775866b.69.1716824504051;
        Mon, 27 May 2024 08:41:44 -0700 (PDT)
Message-ID: <2724a1fe-d0d4-4d27-a39e-4286f95c6236@suse.com>
Date: Mon, 27 May 2024 17:41:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/3] x86/intel: move vmce_has_lmce() routine to
 header
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
 <db547833fa3887e2e36c8418d3f8a142bebb847d.1716366581.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <db547833fa3887e2e36c8418d3f8a142bebb847d.1716366581.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 10:40, Sergiy Kibrik wrote:
> --- a/xen/arch/x86/msr.c
> +++ b/xen/arch/x86/msr.c
> @@ -24,6 +24,8 @@
>  
>  #include <public/hvm/params.h>
>  
> +#include "cpu/mcheck/mce.h"

Considering that I asked about this before, I'm a little irritated
that this is is entirely unadorned. Such an unusual #include wants
explaining some, you'll find similar comments elsewhere:

#include "cpu/mcheck/mce.h" /* for vmce_has_lmce() */

With that, which could also be adjusted while committing,
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 15:43:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:43:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730858.1136111 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcVX-0008RY-VD; Mon, 27 May 2024 15:43:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730858.1136111; Mon, 27 May 2024 15:43:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcVX-0008RR-S0; Mon, 27 May 2024 15:43:31 +0000
Received: by outflank-mailman (input) for mailman id 730858;
 Mon, 27 May 2024 15:43:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBcVW-0008RH-Ra; Mon, 27 May 2024 15:43:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBcVW-0006aU-O9; Mon, 27 May 2024 15:43:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBcVW-0001KV-AO; Mon, 27 May 2024 15:43:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBcVW-0007Td-9t; Mon, 27 May 2024 15:43:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2wopBiKCeS5Qwr2v35GjjCpTxPBRk1QlqXsSMYJQv6c=; b=aEatt7bXxQ1IA26mUzavFhnWoF
	2v2XYxACCC7k9dEn07b+AXxwgj89pdZYqIwUXcZlmDzRujpUvDcNeSMKXyImkS3TyG9Fh2eG7CoTk
	xK+nR7IYjqu7Gps/zb9zfF+qqWTpGgeRDGthfTsVZ4wrm8PF7TzzRCAxVHWZATHxFfmA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186159-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186159: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
X-Osstest-Versions-That:
    linux=56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 15:43:30 +0000

flight 186159 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186159/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186144
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186148
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186148
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186148
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 linux                1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
baseline version:
 linux                56fb6f92854f29dcb6c3dc3ba92eeda1b615e88c

Last test of basis   186148  2024-05-25 18:11:58 Z    1 days
Failing since        186152  2024-05-26 05:15:42 Z    1 days    4 attempts
Testing same since   186156  2024-05-27 01:11:41 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Allen Pais <allen.lkml@gmail.com>
  Allen Pais <apais@linux.microsoft.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Konovalov <andreyknvl@gmail.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@kernel.org>
  Artem Ikonnikov <artem@datacrunch.io>
  Billy Tsai <billy_tsai@aspeedtech.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Carsten Tolkmit <ctolkmit@ennit.de>
  Chengming Zhou <chengming.zhou@linux.dev>
  Christian Heusel <christian@heusel.eu>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Corey Minyard <cminyard@mvista.com>
  Corey Minyard <minyard@acm.org>
  David Hildenbrand <david@redhat.com>
  David Howells <dhowells@redhat.com>
  Dev Jain <dev.jain@arm.com>
  dicken.ding <dicken.ding@mediatek.com>
  Dongli Zhang <dongli.zhang@oracle.com>
  Duoming Zhou <duoming@zju.edu.cn>
  Frank Li <Frank.Li@nxp.com>
  Guenter Roeck <linux@roeck-us.net>
  Hailong.Liu <hailong.liu@oppo.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Ilya Dryomov <idryomov@gmail.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Josh Poimboeuf <jpoimboe@kernel.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Klara Modin <klarasmodin@gmail.com>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Krzysztof Kozlowski <krzk@kernel.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kunwu Chan <chentao@kylinos.cn>
  Lenko Donchev <lenko.donchev@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Mia Lin <mimi05633@gmail.com>
  Miaohe Lin <linmiaohe@huawei.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Hocko <mhocko@suse.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nandor Kracser <bonifaido@gmail.com>
  Nico Pache <npache@redhat.com>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Palmer Dabbelt <palmer@rivosinc.com>
  Randy Dunlap <rdunlap@infradead.org>
  Richard Weinberger <richard@nod.at>
  Roberto Sassu <roberto.sassu@huawei.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Satya Priya Kakitapalli <quic_skakitap@quicinc.com>
  Steve French <stfrench@microsoft.com>
  Suren Baghdasaryan <surenb@google.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tiwei Bie <tiwei.btw@antgroup.com>
  Tony Luck <tony.luck@intel.com>
  Tzung-Bi Shih <tzungbi@kernel.org>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wang Yong <wang.yong12@zte.com.cn>
  Waqar Hameed <waqar.hameed@axis.com>
  Will Deacon <will@kernel.org>
  Xiubo Li <xiubli@redhat.com>
  Yuanyuan Zhong <yzhong@purestorage.com>
  Yueh-Shun Li <shamrocklee@posteo.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   56fb6f92854f..1613e604df0c  1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon May 27 15:45:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:45:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730864.1136120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcXi-0000Yy-BL; Mon, 27 May 2024 15:45:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730864.1136120; Mon, 27 May 2024 15:45:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcXi-0000Yr-8E; Mon, 27 May 2024 15:45:46 +0000
Received: by outflank-mailman (input) for mailman id 730864;
 Mon, 27 May 2024 15:45:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBcXh-0000Yl-Ad
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:45:45 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2d9b7df5-1c40-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 17:45:44 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a63036f2daaso103007266b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:45:44 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8adf3sm503577766b.152.2024.05.27.08.45.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:45:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d9b7df5-1c40-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716824743; x=1717429543; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z4r0Jsgg7TsD8pi/X5L7DjWiTiuCLXHQWMb03Np8clM=;
        b=TDAZMCXKC/Q820AkHqKTzq6lxT/j5/QIzD5umxcHgy00qcIHo4zbH2sp5nusvv8E11
         j4z8qPROrWx6K+POxwRXNtcSLXWllKH4drBN258Zw8YCbEY6AR3185HoaMbYu4dK8qRp
         8LoYhOHNSkjakLAcBVykYMmDT9qCy8JHyyfIF3Y0FNWcgi4j6cMqJaGGoRfwGZE1cQuq
         /EeECX0lg99jeq3YMWRLYLYekPIToKkfSTOOWgWwJn/wL/5BbZ6WqiQhOdICGaVJGkWv
         0Xx8z4VLnMXPNfDBi6PoeTDhvt8TuVfeF/+vSh1qnsxhUGJQnCJFDoTfWyd+nXa6CbVg
         1ORQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716824743; x=1717429543;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z4r0Jsgg7TsD8pi/X5L7DjWiTiuCLXHQWMb03Np8clM=;
        b=ZHxyosMxUVectm+LjwXeMVHd2GYcy1GcjiHdZITBBqVJppNCvLRP9YZZCDbG6w7nOA
         rKNLHeYM4AEHKogRJoXMQUefMhdgOcWJw6ywPax+JVpK96sJNCE/zX9ll88r8sMCaPcv
         4vUnRgn6FMCsLuCCTF3GaSJsnTNF/KGqlewk50/ruvKpCnJbyWDLLxg4eCo1CjF4PrHl
         8oE1qowd3D2nqJHYGFEUsDJ4hI+L9PnFQNxptDX6kSHqmX+2QOU0HobNdNpY4gdQy7mF
         D+abGkkxGegOP6TLm2DNSVVgyW6BSp1Y+d89JrfvMy2Cuw7AaVMzFGn9QQ5+sUPulvCg
         qNlg==
X-Forwarded-Encrypted: i=1; AJvYcCUotNtZUb5HVntlGJ+emIdbVM/y8jc92X4ocn4uVr9rTd/k/wo5WmpoFqO54PCFJiR5gI3f14iOKWjEjFwGK221/9DfkZZ2GUfkMnTudI0=
X-Gm-Message-State: AOJu0YxK9sVmU1Lf7vKaDLR13zevEpFNfagoJF5GxpVnq6gjeXnf8ULH
	7qJnzLRYwWYcwJiCdhIZ7UoKrny+Fxa/j2MHSNqwpK08+qIwa/wRpg9NehUD4jyq8PAIKBGJ7/Y
	=
X-Google-Smtp-Source: AGHT+IFeB0Eh+2k61aR5lWGzoTlJxC+celrvHqWmZD5+Aqd6pEncsLarR/4kgxfF2ycUlPEOwLwlvw==
X-Received: by 2002:a17:906:f5a1:b0:a62:f6f6:badb with SMTP id a640c23a62f3a-a62f6f6bb87mr218359166b.54.1716824743715;
        Mon, 27 May 2024 08:45:43 -0700 (PDT)
Message-ID: <193913eb-e7e8-4670-a236-8942a718686a@suse.com>
Date: Mon, 27 May 2024 17:45:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 2/3] x86/MCE: add default switch case in
 init_nonfatal_mce_checker()
Content-Language: en-US
To: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
 <9ec888c37a261c86a52f360146cbbb3ad437ad30.1716366581.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9ec888c37a261c86a52f360146cbbb3ad437ad30.1716366581.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.05.2024 10:42, Sergiy Kibrik wrote:
> The default switch case block is wanted here, to handle situation
> e.g. of unexpected c->x86_vendor value -- then no mcheck init is done, but
> misleading message still gets logged anyway.
> 
> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon May 27 15:47:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:47:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730870.1136131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcZE-00018G-Kg; Mon, 27 May 2024 15:47:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730870.1136131; Mon, 27 May 2024 15:47:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcZE-000189-I3; Mon, 27 May 2024 15:47:20 +0000
Received: by outflank-mailman (input) for mailman id 730870;
 Mon, 27 May 2024 15:47:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBcZD-000183-6u
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:47:19 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6591179a-1c40-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 17:47:18 +0200 (CEST)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-57863e2846aso2499341a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:47:18 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8adf3sm503577766b.152.2024.05.27.08.47.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:47:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6591179a-1c40-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716824837; x=1717429637; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ATlsmN+bmj/HiE0VTmIBWCuXSVFuzG7kUKN7+vs1uyA=;
        b=AREJred5v1C1MIVsQ+9N6XbraPg4lhwhf/zl6vxoyc6B/QNpLnkazU59xJ9nj2tTMi
         8yGow02Y2aPATL7bm9tna9C1bDmbet9QjmCAHI0IJVHG/obcevxYfYkbE8LCXCgdtq7h
         4OLmnXObVMMYYjW7PUsLCxY7u/MO1p7IniD4yHzFNz1bpkyTf01ZJsbyESC6jcCoH6J7
         vFcLFCVRtVzv+6rWjJk2+c2P69CPswGzw01rW3La1qNFuDiNH0WW07YJpUZpSrw59qsd
         P3fDJIT8nx4ZkigBEHZ5/WHi8zISBY2NRdnVN2PL7eVFjxi4m4rPcNRjlxizJYkcXfCM
         Q0Iw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716824837; x=1717429637;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ATlsmN+bmj/HiE0VTmIBWCuXSVFuzG7kUKN7+vs1uyA=;
        b=FGPYvuJxiFNXFTBQ4NLlG+LpoH4svKbJuGpve2Y8sHwo5Ukvgvziy7oD0Im2K65B4s
         BL720KJ6Li7nMJlEAt5kdoonpE+8qtArkEyUdwhjbojiL2Sk5GfQ5W4nqoA8o+iuag3j
         pyVpsjwqxcDGWr8+funYEdvkk+qabsROLmq8siZ5AHVmMf7R3jY2M3T60LNaxRdwQZbP
         lXAFOjTcsxX9J6zH8RPem5f7PzQUp9mo5wJgkRAGo8qjWTULTwgCrZZzK8J91TGnJg0A
         aypF8XOyvfDPQ727FiFPwooulCSoFfx74Mbc82j33nBHEtz6PoFgyCNUEfSjo5P6CwSo
         GQPQ==
X-Forwarded-Encrypted: i=1; AJvYcCUK49i4dzKjgUvEq5sOHpf8S0FxucrhpZuaIOccCKlc7ajvgVQAPhZCSiLgh2qKQ+X4JH69joa1BXdkIly+o6olvx1uQAoF43FYu8ShoIk=
X-Gm-Message-State: AOJu0YwagdRX1c2sUTxJHoJzogrTTIHXxOKk23JqvfCclMpqPMrYzCfJ
	v+4iAAyArd9EIDvjldeWeO134wsU78iMBqA1Af7C3cOnBM2aVYdHW6JM2U7cvQ==
X-Google-Smtp-Source: AGHT+IHfdG2Mi/nqGAejZUWAIpzGuWpq09htjd7hE3z91IIk7E9ZquVmK1AkDaOEztQxI3sK7paiYQ==
X-Received: by 2002:a17:907:868c:b0:a62:dfa1:c9ea with SMTP id a640c23a62f3a-a62dfa1caa7mr348286866b.3.1716824837585;
        Mon, 27 May 2024 08:47:17 -0700 (PDT)
Message-ID: <9b5890de-ce5b-424c-80f9-3be83e9b94f0@suse.com>
Date: Mon, 27 May 2024 17:47:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 0/3] x86: make Intel/AMD vPMU & MCE support
 configurable
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>, xen-devel@lists.xenproject.org
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Oleksii,

On 22.05.2024 10:37, Sergiy Kibrik wrote:
> Three remaining patches to separate support of Intel & AMD CPUs in Xen build.
> Most of related patches from previous series had already been merged.
> Specific changes since v3 are provided per-patch.
> 
> v3 series here:
> https://lore.kernel.org/xen-devel/cover.1715673586.git.Sergiy_Kibrik@epam.com/
> 
>   -Sergiy
> 
> Sergiy Kibrik (3):
>   x86/intel: move vmce_has_lmce() routine to header
>   x86/MCE: add default switch case in init_nonfatal_mce_checker()
>   x86/MCE: optional build of AMD/Intel MCE code

As I'm apparently confused as to the state 4.19 is in, may I please ask
whether this series is still okay to go in, or whether it should be
postponed until after branching.

Thanks, Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 15:52:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:52:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730877.1136141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcdz-00039w-8l; Mon, 27 May 2024 15:52:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730877.1136141; Mon, 27 May 2024 15:52:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcdz-00039p-5S; Mon, 27 May 2024 15:52:15 +0000
Received: by outflank-mailman (input) for mailman id 730877;
 Mon, 27 May 2024 15:52:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBcdx-00039T-N3
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:52:13 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 11baa1ab-1c41-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 17:52:07 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a61b70394c0so943928166b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:52:07 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8a21esm502620066b.150.2024.05.27.08.52.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 08:52:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11baa1ab-1c41-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716825126; x=1717429926; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=P6oRdxwr3jCk/MnK0Bry251F3aONczSBnKKHYZ+r0ds=;
        b=bMFJAPKtdyElMT/pfbxGLE+kVCPHmRkq5uY/gLUcDFnmq8+fcIyL2DG3aaCq8pmTqO
         bHtcz5/W5c1VJ/+fzSV+rYUR92tkPFQoavdNXy1XPAd33ByXmKRAnWvGVfeEhoM8pUaL
         /7OnKphuUhtxfmDCKLfEwByBE9lDg9tbMWDdOLiLTKiBzTT/fuD2WgkAgUj1jQSD+REl
         INuD8wNefvxzamj5EdSNaWaIf5cZbZcmLFX0m+1egvjQS6wYFFtxAn+BjxBtT4QtrAPo
         GZzdgatPqyefWEYTBWSbk2dNIqk0KwqDKnBhDTGg7RNfp2fFjwyqUtPmpTmWACcsjA4y
         EWaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716825126; x=1717429926;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P6oRdxwr3jCk/MnK0Bry251F3aONczSBnKKHYZ+r0ds=;
        b=Yecl0pN48a/C+9hHuOHyvs2Oc4GQ0cyYRWE+6PI2amQz1KddbK4yzBi9wN9ra5czV+
         Iw+94mV0dD+v07UkZ46B1qq5dxBvcdp3KhqEERjbxzEGf6z6M+3sQeM5IQklxDlpNEuT
         MEZSsLxv62CT0U8TV90NoMtzEsNXhZWx1WG1adLEOTjUq8NP3bfGTWv0KDQiFQGcdGdk
         3OzPI6FD1IrivIidlQm7Y19xWryuLdyDw/7ZFoIWWYG8gi1WKXK8oA8NU5XO3q7eZgeH
         u8J8mzCc+7nWiWe3xZF3MkmqJhrBOcxdq6RYoGluQUr4sGkyMauE8J/tJ9SPAbe/XtSB
         2dKA==
X-Forwarded-Encrypted: i=1; AJvYcCUbxIiBjWrq0SZwS/5ME9yI1xFDljtOIjlvIoE25P3ogiy5ET10OLdN7BY9h6PA7vf6+ic0tx1OHeJRcLZZtH5eopGxndgUN4vAKcW1lTw=
X-Gm-Message-State: AOJu0Yz3C31OjO9wy7V0d7hWZzFazM6RRc9RyrhdxkxHFkSB4e//ka1k
	63PV44Aq7B4D5TuM7GSHKxIMywBQdj9r+3tqV4S6CKg/coUUUayB
X-Google-Smtp-Source: AGHT+IFlKHuq1rqNjEyRsTXAg9iGa84Z09U95oGtalNOqiYyQI/FRKfGYdIJOY3vi4/0v/jlkQkuyw==
X-Received: by 2002:a17:906:28ce:b0:a52:6fcb:564a with SMTP id a640c23a62f3a-a62641a8d45mr594457466b.9.1716825126061;
        Mon, 27 May 2024 08:52:06 -0700 (PDT)
Message-ID: <48f5e40e1f13ef96e8448b59f76878b522876030.camel@gmail.com>
Subject: Re: Code freeze date for Xen 4.19 is 31.05.2024
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
Date: Mon, 27 May 2024 17:52:05 +0200
In-Reply-To: <a1e65a79-d8a4-4b8a-ad4d-a18544389e5a@suse.com>
References: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
	 <a1e65a79-d8a4-4b8a-ad4d-a18544389e5a@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-27 at 17:12 +0200, Jan Beulich wrote:
> On 27.05.2024 15:58, Oleksii K. wrote:
> > I would like to remind you that the code freeze date for Xen 4.19
> > is
> > May 31, 2024.
>=20
> I may be confused: With feature freeze having been last Friday aiui,
> isn't
> this a little too short a period? I was actually expecting a longer-
> than-
> normal one to account for the Xen Summit week.
It makes sense to move the last day to June 14. Consequently, we would
need to adjust the schedule as follows:

Hard Code Freeze: from June 21 to June 28
Final commits: from July 5 to July 12
Release: July 17

And this schedule also looks good to me.

However, another option is to combine the Code Freeze and Hard Code
Freeze periods since both phases are about accepting only bug fixes,
differing only in the severity of the fixes.

Thoughts?

~ Oleksii
>=20
> Jan
>=20
> > I'm okay with bug fixes being committed without my release ack
> > (just CC
> > me), except in cases where a one of maintainers gives a strong
> > NACK.
> >=20
> > Have a nice week!
> >=20
> > Best regards,
> > Oleksii
>=20



From xen-devel-bounces@lists.xenproject.org Mon May 27 15:55:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 15:55:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730882.1136151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBch8-0003i5-NA; Mon, 27 May 2024 15:55:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730882.1136151; Mon, 27 May 2024 15:55:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBch8-0003hy-JV; Mon, 27 May 2024 15:55:30 +0000
Received: by outflank-mailman (input) for mailman id 730882;
 Mon, 27 May 2024 15:55:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=O60l=M6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBch7-0003hl-EO
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 15:55:29 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8943609b-1c41-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 17:55:27 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e724bc466fso101325001fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 08:55:27 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc51a02sm502241066b.106.2024.05.27.08.55.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 08:55:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8943609b-1c41-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716825327; x=1717430127; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OWcz2N3u29S4FPgRjETbFz7nerJVp+5gmcPdSqg682A=;
        b=em0BY+xXT1ry3WLsz/1J8WWRxcQsL+BgJc2Hwidxcfb9XVCmVxpgjWE6BIbbDT1gvl
         KW05khlKETC33qnfvVvcrpDVcfeNyS+zUBCYmAwoWusxr0Fck7U9ToQ/PM5J02Bf7Rvs
         CUjVEUq7DQtK4auIVdIWVxmqgaUK+vvGHtHS96okFPYbY4Rd+pVYr6HyAn5n6A2V7FAN
         ZTC+JUZU0AsIB1O/sT1lAFAkkJVS7HGyAiXUAuX8Z9s/RQu/x0V0M3GY1pyZx6mjpbm8
         PPw0Hwrn8hNlooW/0i292q3MQ/DvOG4NHtHh9TcTeKNv9hIvA+XElNUtJC/E5TIKG7nh
         fg0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716825327; x=1717430127;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OWcz2N3u29S4FPgRjETbFz7nerJVp+5gmcPdSqg682A=;
        b=AwLt1nYpIWg64Of1JMzDb+TwCV8+BdZzy+iR1KuHgNnc5q1gbF6/qsIicErM6Ee6UR
         eh6CA7Zra6S85qWOn8eJX6tORjVPXwYqCWzmZ9IRfYTf0XxOETigx4OEmWGc+atLIFTx
         Jr6E0lgJmSpsnwXzxayKqlRPQpCGSfuHxcd7Pt3JzmzDIVgW04SUoeXB/S4UZQ6z12hb
         v8A4Mk3IL6udt0ihApljsDn05vudXhfUtjXFGt35yP32hBbA+4BRKdbVpwuFvlC2rrsJ
         l+t+OMB+D62Sk/ykN/yhvUgHJ7/L1jwD6MDQK/MzfRif9ZAyJtzqk7DRb6L3wddEnmo3
         lHjA==
X-Forwarded-Encrypted: i=1; AJvYcCXUKLMsSofiD/unSUMTE/wxvX9YG61aQmpOzwcwu7szKMrF0i3GpZEeNv/zHtZJZODMEeS241gcUbNXxZ+tRoQDVklxw5G7u0zc1Z4G9xM=
X-Gm-Message-State: AOJu0Yy8/j2Ayj+yhfjAGIRhrmKHhwlex3+06/+XwUJe+pq/rtEzkebL
	WxKaRvmISs3qvqKM5cq/RrSWRkiLA/FbfSTmYL+O98smGxRABdx0XzMq692Ieg==
X-Google-Smtp-Source: AGHT+IFltXxPGB+H12n6mhEniu14tVABxmwB1itbOGn1BiOCF91gddOdX5nTMRnQMVMprglFUCD1yw==
X-Received: by 2002:a19:8c46:0:b0:516:cf23:588 with SMTP id 2adb3069b0e04-5296556fca3mr7248597e87.27.1716825326876;
        Mon, 27 May 2024 08:55:26 -0700 (PDT)
Message-ID: <2d0c1ebf-fecd-43a5-b616-9869e98ece5e@suse.com>
Date: Mon, 27 May 2024 17:55:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Code freeze date for Xen 4.19 is 31.05.2024
Content-Language: en-US
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
References: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
 <a1e65a79-d8a4-4b8a-ad4d-a18544389e5a@suse.com>
 <48f5e40e1f13ef96e8448b59f76878b522876030.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <48f5e40e1f13ef96e8448b59f76878b522876030.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 17:52, Oleksii K. wrote:
> On Mon, 2024-05-27 at 17:12 +0200, Jan Beulich wrote:
>> On 27.05.2024 15:58, Oleksii K. wrote:
>>> I would like to remind you that the code freeze date for Xen 4.19
>>> is
>>> May 31, 2024.
>>
>> I may be confused: With feature freeze having been last Friday aiui,
>> isn't
>> this a little too short a period? I was actually expecting a longer-
>> than-
>> normal one to account for the Xen Summit week.
> It makes sense to move the last day to June 14. Consequently, we would
> need to adjust the schedule as follows:
> 
> Hard Code Freeze: from June 21 to June 28
> Final commits: from July 5 to July 12
> Release: July 17
> 
> And this schedule also looks good to me.
> 
> However, another option is to combine the Code Freeze and Hard Code
> Freeze periods since both phases are about accepting only bug fixes,
> differing only in the severity of the fixes.

I'm okay with whichever way you want it. All I'm seeking is clarity.

Jan


From xen-devel-bounces@lists.xenproject.org Mon May 27 16:02:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 16:02:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730890.1136161 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcna-0006Fh-Bp; Mon, 27 May 2024 16:02:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730890.1136161; Mon, 27 May 2024 16:02:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcna-0006Fa-8B; Mon, 27 May 2024 16:02:10 +0000
Received: by outflank-mailman (input) for mailman id 730890;
 Mon, 27 May 2024 16:02:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBcnY-0006En-B1
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 16:02:08 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7715486e-1c42-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 18:02:06 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-52388d9ca98so17109068e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 09:02:06 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c818152sm514494166b.13.2024.05.27.09.02.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 09:02:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7715486e-1c42-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716825726; x=1717430526; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=PPYCjJtr/byV3Z9fbDfTBCC80oHiW158IudInU7r1Tk=;
        b=Ba1DuGPZ+l7Egk0YlWw9M4NWUDDjEgxMmiSpbqaa4ec36BC9QBVpIyKDB0opqseBrT
         5XSoKjWJ10f2GSR58xZ08sqHa0P5iFDlD7f7ZZmJCEJ2OoLavbFPLazAb9S8U9eIKHP5
         1wzXvhLvC+pKCdUaCdR6aZYnl9i5ud3QvDgk5WFqyQoYvqrmZ7Bo0Lw19+A7E134M8Mw
         EZQe3H1sfesTDuMtH0iiSAj70qo5c8CQmYDJwkzDozg4ZIfLr5yOwYZrTZq3jFRzrW6a
         2rx+ajxihsgyeeJ402JCaMV8JEsaALAHSV04BPcnSaPHvQ2zne/uI8r7QOTRf2AW8YE/
         sNow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716825726; x=1717430526;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PPYCjJtr/byV3Z9fbDfTBCC80oHiW158IudInU7r1Tk=;
        b=We2uJN7g5pVnt+dc4xYK+sUNW5RIJyDXAhGXsYixecYP6lA1SG+CdD2T8wMZuFi53Y
         CUmBfWakKOjQ40mU/ltkeqgn5diXZHP3vnu76+c6uaoPW3NH2HZgn7IgkknL+o54QMYX
         /WEtSSUnTIr7WT5aPwKsYS9zohcye0D3bP/cuLxV3X9GsroY3Nf709lU36Ua9v7BFkYc
         cYOCo+7nNU0vPk1x02KkWosXEbSfMOmLTZSMsQUF/wXyUZk5fZXmqxxwK5m3aX+R7i0l
         ZkYjoNedZeGv2EiYyPFH4oHlDq4sCGYxUwEI7nuBBlaEXIBKkGXzuTTk+6XVfLc0quns
         3TuQ==
X-Forwarded-Encrypted: i=1; AJvYcCURuUZkHvHMeZty+f4fOPsGkg7JM7uXqBys9hwGhVNbu4mO91KsVvi3bgQBiu1yo8q+bxHtNa3u72BPG1zYqTBsdNeeJww5i+0r9CXbIaM=
X-Gm-Message-State: AOJu0YyZNqkKzR0HroZOWcFmFrl1IDiw7qDxt1NCSkdnWtpWIdop4qfy
	Seqc3tCfJxCcswJ/ahR3P/06P3rStA5QZfOKXZTrKel8t6WDWMF0
X-Google-Smtp-Source: AGHT+IFifkSSZTwwlly6vlVp45YaE9JfHSduk5IroTNfnlIzcYno42kxI5urO1niXl5aXsBzWsG0QA==
X-Received: by 2002:a05:6512:549:b0:51d:5f0a:8839 with SMTP id 2adb3069b0e04-529652906bdmr7430452e87.5.1716825725385;
        Mon, 27 May 2024 09:02:05 -0700 (PDT)
Message-ID: <fa2f6393431cbd06a19e2555308c42238da7b8e4.camel@gmail.com>
Subject: Re: [PATCH v2 3/3] CHANGELOG: Mention libxl blktap/tapback support
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jason Andryuk <jandryuk@gmail.com>, xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jason.andryuk@amd.com>, Community Manager
	 <community.manager@xenproject.org>
Date: Mon, 27 May 2024 18:02:04 +0200
In-Reply-To: <20240407204953.60442-4-jandryuk@gmail.com>
References: <20240407204953.60442-1-jandryuk@gmail.com>
	 <20240407204953.60442-4-jandryuk@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Sun, 2024-04-07 at 16:49 -0400, Jason Andryuk wrote:
> From: Jason Andryuk <jason.andryuk@amd.com>
>=20
> Add entry for backendtype=3Dtap support in libxl.=C2=A0 blktap needs some
> changes to work with libxl, which haven't been merged.=C2=A0 They are
> available from this PR:
> https://github.com/xapi-project/blktap/pull/394
>=20
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
> ---
> =C2=A0CHANGELOG.md | 1 +
> =C2=A01 file changed, 1 insertion(+)
>=20
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 8041cfb7d2..036328433d 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -21,6 +21,7 @@ The format is based on [Keep a
> Changelog](https://keepachangelog.com/en/1.0.0/)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for IPIs and Physical addressing mode for =
external interrupts.
> =C2=A0 - Add a new 9pfs backend running as a daemon in dom0. First user i=
s
> =C2=A0=C2=A0=C2=A0 Xenstore-stubdom now being able to support full Xensto=
re trace
> capability.
> + - libxl support for backendtype=3Dtap with tapback.
> =C2=A0
> =C2=A0### Removed
> =C2=A0- caml-stubdom.=C2=A0 It hasn't built since 2014, was pinned to Oca=
ml
> 4.02, and has

Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon May 27 16:10:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 16:10:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730894.1136170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcvC-0007BN-1k; Mon, 27 May 2024 16:10:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730894.1136170; Mon, 27 May 2024 16:10:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcvB-0007Aq-Un; Mon, 27 May 2024 16:10:01 +0000
Received: by outflank-mailman (input) for mailman id 730894;
 Mon, 27 May 2024 16:10:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ClYa=M6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBcvA-0007AY-Ua
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 16:10:00 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 90ca1256-1c43-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 18:09:59 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a6265d3c488so439963666b.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 09:09:59 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c938502sm504376666b.73.2024.05.27.09.09.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 May 2024 09:09:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90ca1256-1c43-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716826198; x=1717430998; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=6jrcUzHGOlskCQ/YF23KTYoqp2ziYcD5a/4kXi7Mv04=;
        b=Lqk62Tt5E7XdTsg+k5YDfvRlKBQVMK7IyXwlH1FSCBfFVyOvAmKXUvVMy0KJOX0Prk
         a7hxK85ROl43yCkUI9bdlZdBTYWjofyNalMJUo2mM7mwlN/6sZFo8Y1ORlh/lB86mzgk
         WumUwJe+Yk+BiaR7HkLmYR2X13nBHSe6XXYQMhC8YZBujjZHhBLANRlrFGOZ4Wf4HOHK
         IqYtJWxOhf332jOKy8JvkBFsKW160flVVQlQwdPU+WZW5GYD5/oVql0HOR5QGL0BY86G
         eZEuwP/I81F6SFBHvrUX+tDY9+yppGTvBn28i71iRXtHt/bBDGlo6ij0cGy9Ztj6JoRK
         67ZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716826198; x=1717430998;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6jrcUzHGOlskCQ/YF23KTYoqp2ziYcD5a/4kXi7Mv04=;
        b=DHXofkf2qJCGjt90MyUz7KcAtLZacGUQWYB0Ft5HU7sPV2eyYGPzTCkZ3x4Bj9kXWO
         VsCiS1kdAFbaiUspxBP2y7u8SbOCEZBKN1AC3gcBWICZKvcKjvG/zEC6w0x2CCpdasbT
         73u5olox0TyMVNa7CkDNs47Ec4aGq1CBiYEevBoVl20npENtsd7okpp8K0+0ex8Bo6qL
         EKJrrPQoAsYmtpY8QF5YPaKzNObpEqnZC14+GkaUIdgEHc4/9Nj5F+p57FMXUWCfou6m
         VSSavzETxFIpPWDfQQX9h2mR6DkQUrn7zZu9eukXvVCwVDXk3KMigDaD3xxUgqRmcuAW
         qCyw==
X-Forwarded-Encrypted: i=1; AJvYcCW9AEr7Yp4i5J8gMY0ZegKMXVKuoV4RFwFXaUBXQVgByWoI+WnUxLFCh3gR5ylOFbau3aKf9PmepdFLl0WM29S13rw5IBCTknB8JdXChUM=
X-Gm-Message-State: AOJu0Yw12IicYbm9k12VYBcOwulG1iDplAD1b1d8RRWhBKBl8kjNCrGg
	ul0G/KHZrno3ARJ5wciJFyZTSv2VV7N4ujznMCmN4w+aOK1ZKD7m
X-Google-Smtp-Source: AGHT+IGmlP4JkXDvaatweWV00vtHK/G6WptJlpmYG4sygTD4j55h9c7OmeDFzWXrjaurrlTQVfGh1g==
X-Received: by 2002:a17:906:2ada:b0:a59:c941:e5e5 with SMTP id a640c23a62f3a-a62646d5a8amr899802366b.45.1716826198067;
        Mon, 27 May 2024 09:09:58 -0700 (PDT)
Message-ID: <4ef493874d6a3dc5a6403d5c5668e7cef276ce33.camel@gmail.com>
Subject: Re: Code freeze date for Xen 4.19 is 31.05.2024
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>, xen-devel@lists.xenproject.org
Date: Mon, 27 May 2024 18:09:57 +0200
In-Reply-To: <2d0c1ebf-fecd-43a5-b616-9869e98ece5e@suse.com>
References: <6a4323b46dd3dbb170c48fa24e78a23fa030b2a2.camel@gmail.com>
	 <a1e65a79-d8a4-4b8a-ad4d-a18544389e5a@suse.com>
	 <48f5e40e1f13ef96e8448b59f76878b522876030.camel@gmail.com>
	 <2d0c1ebf-fecd-43a5-b616-9869e98ece5e@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-27 at 17:55 +0200, Jan Beulich wrote:
> On 27.05.2024 17:52, Oleksii K. wrote:
> > On Mon, 2024-05-27 at 17:12 +0200, Jan Beulich wrote:
> > > On 27.05.2024 15:58, Oleksii K. wrote:
> > > > I would like to remind you that the code freeze date for Xen
> > > > 4.19
> > > > is
> > > > May 31, 2024.
> > >=20
> > > I may be confused: With feature freeze having been last Friday
> > > aiui,
> > > isn't
> > > this a little too short a period? I was actually expecting a
> > > longer-
> > > than-
> > > normal one to account for the Xen Summit week.
> > It makes sense to move the last day to June 14. Consequently, we
> > would
> > need to adjust the schedule as follows:
> >=20
> > Hard Code Freeze: from June 21 to June 28
> > Final commits: from July 5 to July 12
> > Release: July 17
> >=20
> > And this schedule also looks good to me.
> >=20
> > However, another option is to combine the Code Freeze and Hard Code
> > Freeze periods since both phases are about accepting only bug
> > fixes,
> > differing only in the severity of the fixes.
>=20
> I'm okay with whichever way you want it. All I'm seeking is clarity.
I realized that we have the Xen Summit coming up, which means we'll
lose almost a week from the review process perspective.

I would like to hear the other maintainers' thoughts on updating our
release schedule to have the release on July 17.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon May 27 16:13:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 16:13:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730901.1136180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcyR-0000cb-FS; Mon, 27 May 2024 16:13:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730901.1136180; Mon, 27 May 2024 16:13:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBcyR-0000cU-Ci; Mon, 27 May 2024 16:13:23 +0000
Received: by outflank-mailman (input) for mailman id 730901;
 Mon, 27 May 2024 16:13:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VIbX=M6=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1sBcyP-0000b9-Tu
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 16:13:22 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08e42e38-1c44-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 18:13:20 +0200 (CEST)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-357f1c0b86fso1373853f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 09:13:20 -0700 (PDT)
Received: from [192.168.69.100] ([176.176.152.134])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a0908e4sm9460093f8f.63.2024.05.27.09.13.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 09:13:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08e42e38-1c44-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716826400; x=1717431200; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mfmM0j3HF+vLiXdV40d+v8fDYCHro8ZxhgNASHgkLhU=;
        b=e8O3e1n8bu90OulnfG2zmDriGUuacLyWe9Hx3Em7qWyhyESzr7vem3CHm6fU5up8Y5
         5FizYRkAQBgj/KR/dQHWqslKkREH/Ls2UbNshRdkZ0LDD3CN2HvIpIAbaR6CM7r+Q0iM
         sQntRtLzz7zaHzHNVsBXQ7h8fqnv15KXVIsOUPSgb42jqiCgQRxv1UIt0PY1LIvvlXAo
         mE/Plnal+qvB7CoWOviZAxij0wZQqa04sHvVJXaAlVpAsnYXXZZG1Gv8g56UHlRE1r39
         K86gP9zTFKhUpDQrQCT1AVgrAnYLEmTncBFH2b9kAf8H6Q+5iwOiNm4urAtaDemWHEDU
         1uCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716826400; x=1717431200;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=mfmM0j3HF+vLiXdV40d+v8fDYCHro8ZxhgNASHgkLhU=;
        b=MHFSJhV6HIOIWitPTGA8INqPaARozl4f4op6nqLTXAjWytXQf7hEORzL3+NBfjuPYK
         vFSbmfrlE9s0WTqFPHvN0Ferqo7nPWCDjm+wUF8yQLPLWjCpwKwxH9FVGFC6+PwPVAHV
         MyLWju2gJZd77gwAJgPvaGD8IP/LWmBEdBX2Spd+AnH4GNjlRGpZSIrg+vNIVXpyCWpM
         LAEaQbYL6YuACWi3LC0MGS/wpTXXJ9oGJt9ySG7HoRDhLwyINZWK718LP2PNC5kxoJ4Q
         McPKEZky1cWAE9ILUvEDu3fj5ssU3ZkL8VOsYkroCTqfyeAQxcDeK/fBKigpKmNZRvDB
         zUXQ==
X-Forwarded-Encrypted: i=1; AJvYcCXpmBA5zlQB5xJoMB7EJ07UAKxZyWr/8QxhixeHlMHzUi4Ve9kIa4t27jpj5FWU0bVfwOksRHJgJy5Gvu7VL+QUj4adSeagx3+QjC8+1JY=
X-Gm-Message-State: AOJu0YyuZaHeEt8OLBw/HtRcU9P8Oipa3o9gowHoQOM0Y+qQ/AyrCI3D
	J+f6cGUVdKg1rDMMMM32xgJM90VD+Noj9LOJh47+ijp7rhlVCqr+yVm8qtkNJ8Y=
X-Google-Smtp-Source: AGHT+IFPUBbJ4AV9nNMW5an0kxmRJqDQYgV9r0unWjNl5K2YvpMuoRlLjuLz+k0jRnurcN/gTOehvQ==
X-Received: by 2002:adf:dd89:0:b0:354:e021:51ce with SMTP id ffacd0b85a97d-35526c27950mr6660105f8f.12.1716826399989;
        Mon, 27 May 2024 09:13:19 -0700 (PDT)
Message-ID: <f4918050-11f4-4610-bcf3-2cca4ac0f3e4@linaro.org>
Date: Mon, 27 May 2024 18:13:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 6/8] xen: mapcache: Pass the ram_addr offset to
 xen_map_cache()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 David Hildenbrand <david@redhat.com>, Anthony PERARD
 <anthony@xenproject.org>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>,
 xen-devel@lists.xenproject.org
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
 <20240524105152.1301842-7-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240524105152.1301842-7-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 24/5/24 12:51, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Pass the ram_addr offset to xen_map_cache.
> This is in preparation for adding grant mappings that need
> to compute the address within the RAMBlock.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>   hw/xen/xen-mapcache.c         | 16 +++++++++++-----
>   include/sysemu/xen-mapcache.h |  2 ++
>   system/physmem.c              |  9 +++++----
>   3 files changed, 18 insertions(+), 9 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 27 16:25:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 16:25:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730907.1136191 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBdAH-00032I-GH; Mon, 27 May 2024 16:25:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730907.1136191; Mon, 27 May 2024 16:25:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBdAH-00032B-DA; Mon, 27 May 2024 16:25:37 +0000
Received: by outflank-mailman (input) for mailman id 730907;
 Mon, 27 May 2024 16:25:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VIbX=M6=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1sBdAG-000325-Lx
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 16:25:36 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id be5fd511-1c45-11ef-b4bb-af5377834399;
 Mon, 27 May 2024 18:25:34 +0200 (CEST)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-420298ff5b1so111546515e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 09:25:34 -0700 (PDT)
Received: from [192.168.69.100] ([176.176.152.134])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a08a8c9sm9425250f8f.40.2024.05.27.09.25.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 09:25:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be5fd511-1c45-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716827134; x=1717431934; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pHCRB1EKjD0Nhx9Lp4qHLpHLpjYqMbPoKLCmaUHOhBk=;
        b=FZBwif6X1r4eUhmRPQcUb+hT966bGafJBklkOnFdpWyp6RkicmNU1iCm69AAMTrR4E
         d1D9Qu68n6qmZUJwLE8TtPFPuCedkheM0Pbv5Jq9fXLkBsl22w4uTjrzHy8ZobjDXRfh
         Idkb8UW1xaiMfydeDIpxy/TtsAPTtv1+UIyeN8Ea43f/00l6pr7kDPSq9MujCNs0xjUK
         76S+fYJ29jG0CbKdX9dRtUjTMXRTPEQIoLql6/c1BPy9ct0kgDoNuRrMlY25NF8+9+8u
         l891w+rSf/lHfaJWYwOD2jL+j5bSD0ah1VcM3OiT+gH/OSUPQ3UtdlEV5V5Q2V5ahX2K
         CDdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716827134; x=1717431934;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pHCRB1EKjD0Nhx9Lp4qHLpHLpjYqMbPoKLCmaUHOhBk=;
        b=eGlkFDw0wyVq0oVzFTn4Iq8G7lhZbBFXpcfdT4nT9AhXUSKsm5xqZ2iFjOtIsV3DGS
         pc3JlQjOpKTwHmCM5suwSYCXztfJGRHDiZ1YZguOqSxSyFNwmDw2m4OG5RFWjWvEgYQ7
         DZyhZdLw1mR77Fgf+x8LmZn2vAoo9cpamMNSqO9kSSfTM5C/1rPj7DpnyCE6WKYyc3Sd
         gq4tSMwQo+9jpImhsfLG4pZmuGD2MFRIj2JVnRKH3cczr+e8SXVTBolgMAfOs0Dw792M
         jWGhcJvxnJvLFdc675sUKiz0oNemv8ETZ0by4o0YclU1TVcbpBRz7mLpSayy9DgnDMmh
         VZ0A==
X-Forwarded-Encrypted: i=1; AJvYcCVAPxi6HzKVnBMe5aj5Uf3H0lkC4VR+r0YTSkGUzwYEOBRbByjs3p9WnWD4sXuMb1s/ufgPU+eTVKqm4NOG4OLpuGtbDv+l6RvlE98MPKM=
X-Gm-Message-State: AOJu0YxzYekXDT5rJyEi3KgscrwwqP3tzc34bALsMICHOWuYBpdBQ1gD
	e89hFm60x6RTjROlG8+FB2ABQKd2gtqq7s8GkpZLOK96/ei3gORVMixfU8Ctt4M=
X-Google-Smtp-Source: AGHT+IH6WMXwWBG4+mCGuCUb/g8oSTSVIPe9BmZQhyjU+xKoK4iDxbkQ67gIKHpGtUg5mEm1R3VGJA==
X-Received: by 2002:a05:600c:138e:b0:41b:fa34:9e48 with SMTP id 5b1f17b1804b1-42108a99ea9mr89258365e9.30.1716827133954;
        Mon, 27 May 2024 09:25:33 -0700 (PDT)
Message-ID: <4025f82f-242c-4c3e-952e-8ecf7a66838f@linaro.org>
Date: Mon, 27 May 2024 18:25:31 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 3/8] xen: Add xen_mr_is_memory()
To: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, qemu-devel@nongnu.org
Cc: sstabellini@kernel.org, jgross@suse.com,
 "Edgar E. Iglesias" <edgar.iglesias@amd.com>,
 David Hildenbrand <david@redhat.com>, Anthony PERARD
 <anthony@xenproject.org>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
 <20240524105152.1301842-4-edgar.iglesias@gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240524105152.1301842-4-edgar.iglesias@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Edgar,

On 24/5/24 12:51, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> 
> Add xen_mr_is_memory() to abstract away tests for the
> xen_memory MR.
> 
> No functional changes.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> Acked-by: David Hildenbrand <david@redhat.com>
> ---
>   hw/xen/xen-hvm-common.c | 10 ++++++++--
>   include/sysemu/xen.h    |  8 ++++++++
>   2 files changed, 16 insertions(+), 2 deletions(-)

To consolidate we could add:

   static MemoryRegion xen_memory;

   MemoryRegion *xen_mr_memory_init(uint64_t block_len)
   {
      assert(!xen_memory.size);
      memory_region_init_ram(&xen_memory, NULL, "xen.ram", block_len, 
&error_fatal);
      return &xen_memory;
   }

and remove the extern declaration.

> diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> index 754ec2e6cb..dc72f83bcb 100644
> --- a/include/sysemu/xen.h
> +++ b/include/sysemu/xen.h
> @@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length);
>   void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>                      struct MemoryRegion *mr, Error **errp);
>   
> +bool xen_mr_is_memory(MemoryRegion *mr);
> +
>   #else /* !CONFIG_XEN_IS_POSSIBLE */
>   
>   #define xen_enabled() 0
> @@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
>       g_assert_not_reached();
>   }
>   
> +static inline bool xen_mr_is_memory(MemoryRegion *mr)
> +{
> +    g_assert_not_reached();
> +    return false;

No need for the stub, just always declare xen_mr_is_memory() ...
> +}
> +
>   #endif /* CONFIG_XEN_IS_POSSIBLE */

... here.

>   #endif

Removing the stub:
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>



From xen-devel-bounces@lists.xenproject.org Mon May 27 18:12:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 18:12:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730915.1136201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBepe-0002Pt-TN; Mon, 27 May 2024 18:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730915.1136201; Mon, 27 May 2024 18:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBepe-0002Pm-QM; Mon, 27 May 2024 18:12:26 +0000
Received: by outflank-mailman (input) for mailman id 730915;
 Mon, 27 May 2024 18:12:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MISB=M6=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sBepd-0002Pg-Pw
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 18:12:25 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aa393001-1c54-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 20:12:24 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5A20761B86;
 Mon, 27 May 2024 18:12:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B1D0C2BBFC;
 Mon, 27 May 2024 18:12:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa393001-1c54-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716833542;
	bh=fBmuH4cXhZojeW2O6Rp+qJH/rY/Fl3e4iWLDvnMo+Lw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=P51+ghXmSHFgkDI6AiSxJOaorD3l79ZtVBHyW/fLO8yc9d/A8n07E8DKeK4zu1/bf
	 1QdQStHjMPigr6hNsZVcvHkY6DeVAwJ/ZshWx1/WykXEwEcLCkKBrwgSrzUqfy34BC
	 CykfTG+ufFNeG4v6+24avoQyOwuuZFGfaIAYFQnDtVMod4DmaBmRPl315HYhLUCM09
	 hsz0sa4S3T00LScV5zvJknEGAEULHxzPgc4WvAQ9vgbqzfV+uD4yMTU/l6seVF8Bak
	 b+4F37TLQGRlVzwj6YYWstlLEtO3l9oP5YLxuyytO2456FC4CMZ3H/C3kTPHofoS19
	 cj13eqhR6iWLA==
Date: Mon, 27 May 2024 11:12:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?J=C3=BCrgen_Gro=C3=9F?= <jgross@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Alec Kwapis <alec.kwapis@medtronic.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH v3 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
In-Reply-To: <480ccee9-a50a-4b30-92c4-3f39e4202bca@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405271111220.2557291@ubuntu-linux-20-04-desktop>
References: <20240517032156.1490515-1-xin.wang2@amd.com> <20240517032156.1490515-3-xin.wang2@amd.com> <cf0a096a-0fe8-4d91-a064-910bd6c43951@xen.org> <d3aca96b-2b6b-4bb8-99bf-7c0ede4c94fe@suse.com> <alpine.DEB.2.22.394.2405241614370.2557291@ubuntu-linux-20-04-desktop>
 <480ccee9-a50a-4b30-92c4-3f39e4202bca@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-99592716-1716833542=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-99592716-1716833542=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 27 May 2024, Jürgen Groß wrote:
> On 25.05.24 01:19, Stefano Stabellini wrote:
> > On Fri, 24 May 2024, Jürgen Groß wrote:
> > > On 24.05.24 15:58, Julien Grall wrote:
> > > > Hi Henry,
> > > > 
> > > > + Juergen as the Xenstore maintainers. I'd like his opinion on the
> > > > approach.
> > > > The documentation of the new logic is in:
> > > > 
> > > > https://lore.kernel.org/xen-devel/20240517032156.1490515-5-xin.wang2@amd.com/
> > > > 
> > > > FWIW I am happy in principle with the logic (this is what we discussed
> > > > on
> > > > the call last week). Some comments below.
> > > 
> > > I'm not against this logic, but I'm wondering why it needs to be so
> > > complicated.
> > 
> > Actually the reason I like it is that in my view, this is the simplest
> > approach. You allocate a domain, you also allocate the xenstore page
> > together with it. Initially the xenstore connection has an
> > "uninitialized" state, as it should be. That's it. At some point, when
> > xenstored is ready, the state changes to CONNECTED.
> > 
> > 
> > > Can't the domU itself allocate the Xenstore page from its RAM pages,
> > > write the PFN into the Xenstore grant tab entry, and then make it
> > > public via setting HVM_PARAM_STORE_PFN?
> > 
> > This is not simpler in my view
> 
> Okay, fine with me. I had the impression that violating the 1:1 mapping
> of the domain would add complexity, but if you are fine with that I don't
> mind your approach.

Yes, that's fine. Thanks!
--8323329-99592716-1716833542=:2557291--


From xen-devel-bounces@lists.xenproject.org Mon May 27 18:22:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 18:22:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730920.1136212 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBez2-00047t-Nt; Mon, 27 May 2024 18:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730920.1136212; Mon, 27 May 2024 18:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBez2-00047m-JQ; Mon, 27 May 2024 18:22:08 +0000
Received: by outflank-mailman (input) for mailman id 730920;
 Mon, 27 May 2024 18:22:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MISB=M6=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sBez0-00047g-Ko
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 18:22:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 04996966-1c56-11ef-90a1-e314d9c70b13;
 Mon, 27 May 2024 20:22:05 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C7A1061B96;
 Mon, 27 May 2024 18:22:03 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4D0FC2BBFC;
 Mon, 27 May 2024 18:22:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04996966-1c56-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716834123;
	bh=MfpcuSDLTv7mzqhKIv7ioKH7DL90/sDNckRrjGnI/eY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NitukqZ3qBdcfsJRI/ZNlwnxHHOGEEY7audBq+BwX5WzAP0k6vogL2tYdtelgo+rV
	 W/j/DR0LH4JXz+4KSCDovhNqs3xYEBbfdxuY/CDF2Bq4Gmovsnme10IMD7JyI1q9Aj
	 lAPSGPcVMTNPkGhAXrVqLF+2N0bcgFPwh6rQyn/J+ZC0Q9pKXtFZ1z8IQqhm7v0RGb
	 aXZ9+GmXkbLEaZWVlO+2rgzclshiZ3qsJve7WaZZ+Nu3RfUmVc40WpG6d5ur2akl8M
	 f5uO+mq5540vm32jwlwb0rlezojrWvpW36nDQa23W6RbJoLqxC4Fa/ZvVX8PRqUzrg
	 9k79Ns9K7cncg==
Date: Mon, 27 May 2024 11:22:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Michal Orzel <michal.orzel@amd.com>, roberto.bagnara@bugseng.com, 
    consulting@bugseng.com, xen-devel@lists.xenproject.org
Subject: Re: [PATCH] docs/misra: add D4.12
In-Reply-To: <alpine.DEB.2.22.394.2405151513530.2544314@ubuntu-linux-20-04-desktop>
Message-ID: <alpine.DEB.2.22.394.2405271120470.2557291@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2405141611170.2544314@ubuntu-linux-20-04-desktop> <bd1072c5-1533-46b7-b6c8-fab1c0f80ab3@suse.com> <alpine.DEB.2.22.394.2405151513530.2544314@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 May 2024, Stefano Stabellini wrote:
> On Wed, 15 May 2024, Jan Beulich wrote:
> > On 15.05.2024 01:15, Stefano Stabellini wrote:
> > > Add D4.12 with the same explanation as the rules of the R21 series.
> > > D4.12 refers to the standard library memory allocation functions and
> > > similar third party libraries with memory allocation functions. It
> > > doesn't refer to the in-tree implementation we have in Xen which is
> > > subject to MISRA C rules and MISRA C scanning.
> > > 
> > > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > > 
> > > diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> > > index 80e5e972ad..bc8506add4 100644
> > > --- a/docs/misra/rules.rst
> > > +++ b/docs/misra/rules.rst
> > > @@ -76,6 +76,11 @@ maintainers if you want to suggest a change.
> > >         considered libraries from MISRA C point of view as they are
> > >         imported in source form)
> > >  
> > > +   * - `Dir 4.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/D_04_12.c>`_
> > > +     - Required
> > > +     - Dynamic memory allocation shall not be used
> > > +     - Xen doesn't provide, use, or link against a Standard Library [#xen-stdlib]_
> > 
> > I'm having trouble connecting this remark with the directive. We do have
> > dynamic memory allocation routines, and we use them. It doesn't really
> > matter that they don't come from an external library, does it?
> 
> Similarly to the 21.x rules series, it makes a difference if they are
> external libraries or code within the project. The rule points out that
> the standard library memory allocation functions can lead to undefined
> behavior. On the other hand, our own implementation under xen.git is
> subject to MISRA C scanning and all the other MISRA C rules.
> 
> The example in the link above, shows a use-after-free error that in our
> case it should be caught by other MISRA C rules scanning.

Just to close the loop on this -- I spoke with Roberto about D4.12, and
we decided that it is best to leave out this directive for now.


From xen-devel-bounces@lists.xenproject.org Mon May 27 21:15:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 21:15:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730936.1136221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBhgM-00061N-MT; Mon, 27 May 2024 21:15:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730936.1136221; Mon, 27 May 2024 21:15:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBhgM-00061G-J4; Mon, 27 May 2024 21:15:02 +0000
Received: by outflank-mailman (input) for mailman id 730936;
 Mon, 27 May 2024 21:15:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sBhgL-00061A-OR
 for xen-devel@lists.xenproject.org; Mon, 27 May 2024 21:15:01 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBhgL-0004pk-EX; Mon, 27 May 2024 21:15:01 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.244])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBhgL-0003MN-7i; Mon, 27 May 2024 21:15:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Cb7GtgMHZEoaRf7aO+7KJNTqXiRe/k2oFZCQtLhQiB0=; b=k/SdWyGqU2DumVjimhwwmrOOw0
	EO65OAqrD4Clws6tbT6Nyg0yPPfakzIWwkmkfcV2hBBBlgDAk0mXjbW+gKTSXmf2dX/+WPjjkM2V3
	WnvD5JkfitH6lq9oOX76lMIF2M1hDxwmp1t+uuCAz8kHU/QCSMupfD8nX7a1D6QlJrr4=;
Message-ID: <3b3fd3a2-6b5c-41a2-a0b3-219a362f645f@xen.org>
Date: Mon, 27 May 2024 22:14:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v16 1/5] arm/vpci: honor access size when returning an
 error
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-2-stewart.hildebrand@amd.com>
 <Zk72jPtd9iXhChbc@macbook>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <Zk72jPtd9iXhChbc@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Roger,

On 23/05/2024 08:55, Roger Pau Monné wrote:
> On Wed, May 22, 2024 at 06:59:20PM -0400, Stewart Hildebrand wrote:
>> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
>>
>> Guest can try to read config space using different access sizes: 8,
>> 16, 32, 64 bits. We need to take this into account when we are
>> returning an error back to MMIO handler, otherwise it is possible to
>> provide more data than requested: i.e. guest issues LDRB instruction
>> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
>> register.
> 
> Shouldn't this be taken care of in the trap handler subsystem, rather
> than forcing each handler to ensure the returned data matches the
> access size?

I understand how this can be useful when we return all 1s.

However, in most of the current cases, we already need to deal with the 
masking because the data is extracted from a wider field (for instance, 
see the vGIC emulation). For those handlers, I would argue it would be 
concerning/ a bug if the handler return bits above the access size. 
Although, this would only impact the guest itself.

So overall, this seems to be a matter of taste and I don't quite (yet) 
see the benefits to do it in io.c. Regardless that...

> 
> IOW, something like:
> 
> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> index 96c740d5636c..b7e12df85f87 100644
> --- a/xen/arch/arm/io.c
> +++ b/xen/arch/arm/io.c
> @@ -37,6 +37,7 @@ static enum io_state handle_read(const struct mmio_handler *handler,
>           return IO_ABORT;
> 
>       r = sign_extend(dabt, r);
> +    r = r & GENMASK_ULL((1U << dabt.size) * 8 - 1, 0);

... in some case we need to sign extend up to the width of the register 
(even if the access is 8-byte). So we would need to do the masking 
*before* calling sign_extend().

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon May 27 23:46:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 May 2024 23:46:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730958.1136231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBk2p-00059Q-5X; Mon, 27 May 2024 23:46:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730958.1136231; Mon, 27 May 2024 23:46:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBk2p-00059J-20; Mon, 27 May 2024 23:46:23 +0000
Received: by outflank-mailman (input) for mailman id 730958;
 Mon, 27 May 2024 23:46:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBk2n-000599-P8; Mon, 27 May 2024 23:46:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBk2n-0007GQ-GG; Mon, 27 May 2024 23:46:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBk2n-0007jo-6k; Mon, 27 May 2024 23:46:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBk2n-0003Am-6A; Mon, 27 May 2024 23:46:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3wlLor5cGeADnmIWS1xMRnVafOkCtG363GuRCYuI6jo=; b=ceKB+AjDjHUGqEF/Ek95/22a/K
	8yzE2+HY00TNTbTIofBlzpITKTHHD8v5DI+V+RTyFOs8JatARL3lkbeIekVRssMEmD3RS0sYi9Wxg
	ogkl/YrJwJvJ95JBJ5j6qoivq3wcatJ7h8kx6LNeIsCusTcwShPmf4FUZA/b4hEBQZR8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186161-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186161: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=2bfcfd584ff5ccc8bb7acde19b42570414bf880b
X-Osstest-Versions-That:
    linux=1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 May 2024 23:46:21 +0000

flight 186161 linux-linus real [real]
flight 186162 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186161/
http://logs.test-lab.xenproject.org/osstest/logs/186162/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-examine      8 reboot              fail pass in 186162-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186159
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186159
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186159
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186159
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186159
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186159
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186159
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 linux                2bfcfd584ff5ccc8bb7acde19b42570414bf880b
baseline version:
 linux                1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0

Last test of basis   186159  2024-05-27 06:48:45 Z    0 days
Testing same since   186161  2024-05-27 16:11:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Ford <aford173@gmail.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Christian Brauner <brauner@kernel.org>
  David Howells <dhowells@redhat.com>
  Fedor Pchelkin <pchelkin@ispras.ru>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Dionne <marc.dionne@auristor.com>
  Shengjiu Wang <shengjiu.wang@nxp.com>
  syzbot+d7c7a495a5e466c031b6@syzkaller.appspotmail.com
  Ulf Hansson <ulf.hansson@linaro.org>
  Xu Yang <xu.yang_2@nxp.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   1613e604df0c..2bfcfd584ff5  2bfcfd584ff5ccc8bb7acde19b42570414bf880b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue May 28 01:56:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 01:56:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730966.1136240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBm46-0001Gu-O6; Tue, 28 May 2024 01:55:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730966.1136240; Tue, 28 May 2024 01:55:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBm46-0001Gn-LC; Tue, 28 May 2024 01:55:50 +0000
Received: by outflank-mailman (input) for mailman id 730966;
 Tue, 28 May 2024 01:55:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M10Z=M7=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sBm44-0001Gh-UI
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 01:55:49 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 65f1e29c-1c95-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 03:55:46 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 09A7561D45;
 Tue, 28 May 2024 01:55:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A5AAC2BBFC;
 Tue, 28 May 2024 01:55:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65f1e29c-1c95-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716861344;
	bh=kyys+qhbR8Blneups9GbfVsfnA+bWp+ksbo9rw9AxUI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=b0GnTcGD7lgfAbhwESprAmy8HEpNRLBcq7qYbv0q3Rih5EA1+R52qTHnK6bqoQdX0
	 3rrX4GC6RPxHya+qi8U94qZA4VGjsGe8z36Io8upvNFbQ8mY1iEEQ2hYtCSpQd5Uco
	 V/gLo0+qw/7M7DODwxz8KfI+TS+eaKaOcGehHhJ8TDpJCqfbL5uh5IkS4FAULuMCzy
	 cXjoEzNzUyhXuxvlLTvYe9Cjoy+8EpzY8J+/ReyLncJNEB/rDrd7G9mW4sslPwJiu6
	 TOY5wQzg9e7UqtqBvIb67vSk0DRBnvmqw2JnxK6YR3r3LhAjE3mbkpFqkYtoGio8a7
	 m6hipquzkLHcg==
Date: Mon, 27 May 2024 18:55:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 2/4] automation/eclair_analysis: avoid an ECLAIR
 warning about escaping
In-Reply-To: <86e1e74b-f1e3-4e16-b87e-a9c512dcc655@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405271855070.2557291@ubuntu-linux-20-04-desktop>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com> <f489cc3d668830eab2f2ce724164d65dc623baa7.1716814609.git.nicola.vetrini@bugseng.com> <86e1e74b-f1e3-4e16-b87e-a9c512dcc655@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 May 2024, Jan Beulich wrote:
> On 27.05.2024 16:53, Nicola Vetrini wrote:
> > The parentheses in this regular expression should be doubly
> > escaped because they are undergo escaping twice.
> 
> Do you maybe mean "undergo expansion twice"?

Ahah yes. I fixed it on commit:

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

 
> > Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> > ---
> >  automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> > index b9b377c56b25..cf62a874d928 100644
> > --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> > +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> > @@ -405,8 +405,8 @@ explicit comment indicating the fallthrough intention is present."
> >  #
> >  
> >  -doc_begin="printf()-like functions are allowed to use the variadic features provided by stdarg.h."
> > --config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\(.*\)$)))"}
> > --config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\(.*\)$)))"}
> > +-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printk\\(.*\\)$)))"}
> > +-config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(^.*printf\\(.*\\)$)))"}
> >  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(panic)&&kind(function))))"}
> >  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(elf_call_log_callback)&&kind(function))))"}
> >  -config=MC3R1.R17.1,reports+={deliberate,"any_area(^.*va_list.*$&&context(ancestor_or_self(name(vprintk_common)&&kind(function))))"}
> 


From xen-devel-bounces@lists.xenproject.org Tue May 28 05:42:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 05:42:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730987.1136267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBpbU-0000yh-Ip; Tue, 28 May 2024 05:42:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730987.1136267; Tue, 28 May 2024 05:42:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBpbU-0000ya-FD; Tue, 28 May 2024 05:42:32 +0000
Received: by outflank-mailman (input) for mailman id 730987;
 Tue, 28 May 2024 05:42:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBpbT-0000yQ-LQ; Tue, 28 May 2024 05:42:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBpbT-0005Kg-Jd; Tue, 28 May 2024 05:42:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBpbT-0002cm-AT; Tue, 28 May 2024 05:42:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBpbT-0006b5-9p; Tue, 28 May 2024 05:42:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=M2JgA0DYsF47x/Q84lBZbfJBA2YpSW5e6yRIh3Puktk=; b=3l3T8hgDUhTEhK94TWYHTUHTjZ
	DgKrxYeRrN8LLAWekOlrzTpuAMXN7zyHsuxkMn/ne6OdKJ2l3+H4LLBWbeuMSHiz9wrP/JZ6mBF5l
	2hoIMdc9WOoIJqFwGCa8OKuAM45hR3mFuEV4s4MUbiOGEYKMWOoOplZswrV8MGgwfYJY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186164-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186164: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=96af090e33130b0bf0953f3ccab8e7a163392318
X-Osstest-Versions-That:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 05:42:31 +0000

flight 186164 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186164/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  96af090e33130b0bf0953f3ccab8e7a163392318
baseline version:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0

Last test of basis   186142  2024-05-25 00:02:11 Z    3 days
Testing same since   186164  2024-05-28 02:00:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ac572152e5..96af090e33  96af090e33130b0bf0953f3ccab8e7a163392318 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 28 05:46:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 05:46:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.730976.1136277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBpfb-0001XU-K1; Tue, 28 May 2024 05:46:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 730976.1136277; Tue, 28 May 2024 05:46:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBpfb-0001XN-7E; Tue, 28 May 2024 05:46:47 +0000
Received: by outflank-mailman (input) for mailman id 730976;
 Tue, 28 May 2024 03:28:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8s2a=M7=gmail.com=yskelg@srs-se1.protection.inumbo.net>)
 id 1sBnVr-0003La-4d
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 03:28:37 +0000
Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com
 [2607:f8b0:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5a363683-1ca2-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 05:28:30 +0200 (CEST)
Received: by mail-pl1-x631.google.com with SMTP id
 d9443c01a7336-1f4a52d2688so3213765ad.3
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 20:28:30 -0700 (PDT)
Received: from [11.12.35.19] ([211.210.76.6]) by smtp.gmail.com with ESMTPSA id
 d9443c01a7336-1f44c9ccdb0sm70989045ad.273.2024.05.27.20.28.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 20:28:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a363683-1ca2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716866909; x=1717471709; darn=lists.xenproject.org;
        h=subject:from:cc:to:content-language:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=pgBAbs2bznNfrnwJJbR/R940dofXEe8oJPIVFYJE37M=;
        b=M8WBg+c9hl46eexReq3BKPDs/C3DWamK3WGsoaOOuSxAbzfzr/pqlrujtN9VTXdtRE
         6bZYhbNn5dxvudjBA8VUKZWBrnSoicSJyRz/lFbNbXnzNn5EXtOneUDGV2XUMX3a56S3
         f8IQ46x8zHDzod8cUqXZoiXTUaxCrzNqEY9ueFYKleHZThkHbtTxaQsy2t5azihdcruA
         COcNkto3GINbGP6xeg0sq2HbEaFCyLDXoNIITRREKdFSnhdyl94+UMgyJPTGHwbI7ELT
         3sPWsUeVk1w4mxOZQKHn9pEtqoWxRRW1u6bZiGgAjwUFbnQlX5dY2YDfX4g5/NSHHNKz
         E7ZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716866909; x=1717471709;
        h=subject:from:cc:to:content-language:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pgBAbs2bznNfrnwJJbR/R940dofXEe8oJPIVFYJE37M=;
        b=WiRN7mcGAjSAZwysKq9HGAmdnzLq7Y1inODUdxyZsg95FBmzNhuSvXg6Noadnr7acH
         LQbLwu0D4EMFZQ1+eIP2ziNZaUirmEnf02t24v55ijARfnRpP3DdslDex0ixDdk8PNEt
         DRYJENzfK9qTnZsbYbzIED5EqFYmrELa/f28Ake45405EtaPm8a6AEPY/IRt9ta6pS6P
         iBa2H5T7ODCag1ZkCsxu3XyD6JaJ9PzYcDjpIbGVI0O7ao2tiMzYca4K/z1nMHmS791R
         5xeAvbfDk9z/LK8CUjUPGShW0J6ym8kbz1VwGw97cvTn5uJqqpoT47irOs7/z/anZD2t
         Jgmw==
X-Forwarded-Encrypted: i=1; AJvYcCVZi/ZB9SM1f3drwdIPGGzwpYOsz9fO1EUC0cLDod553Y/Y9oC24IZPNTMYsoIrZNfFGU6Z6+m9QmuzutfnOpnnEmO2NHuiISYYi3dxjbI=
X-Gm-Message-State: AOJu0YyorbiUjTT1kldmxeFv25MY7N2khGXYuUqKIyzV2HHTgOxVAaiw
	SxcHVCFCpvZipX5u8pGPK9aKwXSDEgvI4yRahg9CiOtROp72NefRrxd0HPSsyOM=
X-Google-Smtp-Source: AGHT+IFdIF6no4DooNFbNVEfLN6tMURTe5oW1+F4CDOShys5gizGHvzhapvjNNLTyNXd3Y+NmkV0Mg==
X-Received: by 2002:a17:902:ec8c:b0:1f4:985a:ba83 with SMTP id d9443c01a7336-1f4985ac477mr55356835ad.48.1716866907923;
        Mon, 27 May 2024 20:28:27 -0700 (PDT)
Content-Type: multipart/mixed; boundary="------------p60WVPVwD5u0Q02s6TthpwEk"
Message-ID: <264e7962-cdfa-4bc9-b457-9eecf2ba8062@gmail.com>
Date: Tue, 28 May 2024 12:28:09 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: jbeulich@suse.com
Cc: andrew.cooper3@citrix.com, consulting@bugseng.com,
 roberto.bagnara@bugseng.com, roger.pau@citrix.com, sstabellini@kernel.org,
 xen-devel@lists.xenproject.org
From: Yunseong Kim <yskelg@gmail.com>
Subject: Re: Questions when put in the compiler -wExtra option

This is a multi-part message in MIME format.
--------------p60WVPVwD5u0Q02s6TthpwEk
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi, I am Yunseong Kim.

I was looking for an appropriate topic for -wExtra compilation options
and came across this threads.

The below make build a lot of warnings(attachement file are build log
with warnings) compared to the Linux kernel,
is it something that would benefit the Xen hypervisor to apply removing
warning for the code?

The additional no options were added to ensure that Xen compiles
successfully when the wError compiler flag is added.

They are all printed as warnings.

$ make allyesconfig

$ CFLAGS="-g -O0 -Wextra -Wno-error=unused-parameter
-Wno-error=sign-compare -Wno-error=empty-body
-Wno-error=old-style-declaration -Wno-error=type-limits" time make -j4
2>&1 | tee xen-build.log


I have a question about the prioritization of fixes for the warning items.

If some of the warnings the Xen hypervisor detects aren't critical for
its operation, I wouldn't want to submit a patch that might introduce
unnecessary complexity.

Would it be helpful to clarify which fixes are considered some of priority?


My compile environment is Arm64 Ubuntu 22.04

$ uname -a
Linux paran-QEMU-Virtual-Machine 6.5.0-35-generic #35~22.04.1-Ubuntu SMP
PREEMPT_DYNAMIC Tue May  7 11:19:33 UTC 2 aarch64 aarch64 aarch64 GNU/Linux

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/12/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
12.3.0-1ubuntu1~22.04'
--with-bugurl=file:///usr/share/doc/gcc-12/README.Bugs
--enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr
--with-gcc-major-version-only --program-suffix=-12
--program-prefix=aarch64-linux-gnu- --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --libdir=/usr/lib
--enable-nls --enable-clocale=gnu --enable-libstdcxx-debug
--enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new
--enable-gnu-unique-object --disable-libquadmath
--disable-libquadmath-support --enable-plugin --enable-default-pie
--with-system-zlib --enable-libphobos-checking=release
--with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch
--enable-fix-cortex-a53-843419 --disable-werror
--enable-checking=release --build=aarch64-linux-gnu
--host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.3.0 (Ubuntu 12.3.0-1ubuntu1~22.04)


Warm Regards,

Yunseong Kim
--------------p60WVPVwD5u0Q02s6TthpwEk
Content-Type: text/plain; charset=UTF-8; name="xen-build.log"
Content-Disposition: attachment; filename="xen-build.log"
Content-Transfer-Encoding: base64

ICBXUkFQICAgIGFyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9hbHRwMm0uaAogIFdS
QVAgICAgYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL2RldmljZS5oCiAgV1JBUCAg
ICBhcmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vaGFyZGlycS5oCiAgV1JBUCAgICBh
cmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vaW9jYXAuaAogIFdSQVAgICAgYXJjaC9h
cm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL3BhZ2luZy5oCiAgV1JBUCAgICBhcmNoL2FybS9p
bmNsdWRlL2dlbmVyYXRlZC9hc20vcGVyY3B1LmgKICBXUkFQICAgIGFyY2gvYXJtL2luY2x1
ZGUvZ2VuZXJhdGVkL2FzbS9yYW5kb20uaAogIFdSQVAgICAgYXJjaC9hcm0vaW5jbHVkZS9n
ZW5lcmF0ZWQvYXNtL3NvZnRpcnEuaAogIFdSQVAgICAgYXJjaC9hcm0vaW5jbHVkZS9nZW5l
cmF0ZWQvYXNtL3ZtX2V2ZW50LmgKICBIT1NUQ0MgIHRvb2xzL3N5bWJvbHMKICBCQU5ORVIg
IC5iYW5uZXIKICBVUEQgICAgIGluY2x1ZGUveGVuL2NvbXBpbGUuaAogWGVuIDQuMTktdW5z
dGFibGUKICBDUFAgICAgIGluY2x1ZGUvaHlwZXJjYWxsLWRlZnMuaQogIENISyAgICAgaW5j
bHVkZS9oZWFkZXJzLmNoawogIENISyAgICAgaW5jbHVkZS9oZWFkZXJzOTkuY2hrCiAgQ0hL
ICAgICBpbmNsdWRlL2hlYWRlcnMrKy5jaGsKICBHRU4gICAgIGluY2x1ZGUveGVuL2h5cGVy
Y2FsbC1kZWZzLmgKbWFrZVsxXTogJ2luY2x1ZGUn7J2EKOulvCkg7JyE7ZW0IO2VoCDsnbzs
nbQg7JeG7Iq164uI64ukLgpnY2MgLU1NRCAtTVAgLU1GIC4vLmFzbS1vZmZzZXRzLnMuZCAt
ZyAtTzAgLVdleHRyYSAtV25vLWVycm9yPXVudXNlZC1wYXJhbWV0ZXIgLVduby1lcnJvcj1z
aWduLWNvbXBhcmUgLVduby1lcnJvcj1lbXB0eS1ib2R5IC1Xbm8tZXJyb3I9b2xkLXN0eWxl
LWRlY2xhcmF0aW9uIC1Xbm8tZXJyb3I9dHlwZS1saW1pdHMgLURCVUlMRF9JRCAtZm5vLXN0
cmljdC1hbGlhc2luZyAtc3RkPWdudTk5IC1XYWxsIC1Xc3RyaWN0LXByb3RvdHlwZXMgLVdu
by11bnVzZWQtYnV0LXNldC12YXJpYWJsZSAtV25vLXVudXNlZC1sb2NhbC10eXBlZGVmcyAt
TzEgLWZuby1vbWl0LWZyYW1lLXBvaW50ZXIgLW5vc3RkaW5jIC1mbm8tYnVpbHRpbiAtZm5v
LWNvbW1vbiAtV2Vycm9yIC1XcmVkdW5kYW50LWRlY2xzIC1Xd3JpdGUtc3RyaW5ncyAtV25v
LXBvaW50ZXItYXJpdGggLVdkZWNsYXJhdGlvbi1hZnRlci1zdGF0ZW1lbnQgLVd1bmluaXRp
YWxpemVkIC1XdmxhIC1XaW5pdC1zZWxmIC1waXBlIC1EX19YRU5fXyAtaW5jbHVkZSAuL2lu
Y2x1ZGUveGVuL2NvbmZpZy5oIC1mZnVuY3Rpb24tc2VjdGlvbnMgLWZkYXRhLXNlY3Rpb25z
IC1nIC1tY3B1PWdlbmVyaWMgLW1nZW5lcmFsLXJlZ3Mtb25seSAtbW5vLW91dGxpbmUtYXRv
bWljcyAtSS4vaW5jbHVkZSAtSS4vYXJjaC9hcm0vaW5jbHVkZSAtSS4vYXJjaC9hcm0vaW5j
bHVkZS9nZW5lcmF0ZWQgLWZuby1waWUgLWZuby1zdGFjay1wcm90ZWN0b3IgLWZuby1leGNl
cHRpb25zIC1mbm8tYXN5bmNocm9ub3VzLXVud2luZC10YWJsZXMgLVduZXN0ZWQtZXh0ZXJu
cyAtUyAtZzAgLW8gYXNtLW9mZnNldHMucy5uZXcgLU1RIGFzbS1vZmZzZXRzLnMgYXJjaC9h
cm0vYXJtNjQvYXNtLW9mZnNldHMuYwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9hcm02
NC9hc20tb2Zmc2V0cy5jOjk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGlu
dCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDox
ODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAg
ICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoy
NDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAg
ICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVl
X2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IElu
IGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3Qg
bWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19p
bl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAg
ICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdl
X3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUv
eGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxv
Y2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQg
cmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9h
Y2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNx
dWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
NzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3Vy
Y2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQg
cDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHji
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9t
YWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3Qg
cSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2Nr
KHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2ln
bl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwg
c3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0
cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVh
ZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lf
cmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBk
YXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRh
dGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6
Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9j
ZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKA
mToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdh
eXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgppZiAhIGNtcCAtcyBhc20tb2Zmc2V0cy5zLm5l
dyBhc20tb2Zmc2V0cy5zOyB0aGVuIG12IC1mIGFzbS1vZmZzZXRzLnMubmV3IGFzbS1vZmZz
ZXRzLnM7IGVsc2Ugcm0gLWYgYXNtLW9mZnNldHMucy5uZXc7IGZpCiAgQ0MgICAgICB4c20v
eHNtX2NvcmUubwogIENDICAgICAgbGliL2JzZWFyY2gubwogIENDICAgICAgY29tbW9uL3N5
bWJvbHMtZHVtbXkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20geHNtL3hzbV9jb3JlLmM6MTY6
Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt
4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9m
IOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKA
mHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWdu
ZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBp
bnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5h
dGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Rp
bWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxs
Lmg6OSwKICAgICAgICAgICAgICAgICBmcm9tIHhzbS94c21fY29yZS5jOjE4OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3Np
Ymxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVk
IGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGxpYi9jdG9ycy5vCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9oeXBlcmNhbGwuaDoxMToKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBj
b21tb24vYXJnby5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94c20veHNtLmg6MTksCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2FsbC5oOjIyOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
CiAgQ0MgICAgICBsaWIvY3R5cGUubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3Rv
X25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRp
YyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2FyZ28uaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1v
bi9hcmdvLmM6MTg6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBm
dW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVf
bnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzcg
fCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5o
OjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDi
gJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICpt
YXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJl
ZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEy
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
ZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5
c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9j
ay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9f
SFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
dW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
NCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1f
ZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWlu
dDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAg
ICBsaWIvZmluZC1uZXh0LWJpdC5vCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNo
X3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcnds
b2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN0
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8
ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBw
Y2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25f
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90
IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
ZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQz
Ml90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlu
bGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1
aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVp
bnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sg
dnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0
X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0
YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rp
b24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hl
ZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0Njox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
b21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNv
bnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBp
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNo
biAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3
MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQo
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVt
b3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzkz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQw
NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3Bo
eXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFn
ZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZh
dWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2No
ZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9t
YXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2Rv
bWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAq
ZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5t
YXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1h
cF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZv
aWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGly
cSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6
CiAgQ0MgICAgICBkcml2ZXJzL2NoYXIvY29uc29sZS5vCi4vaW5jbHVkZS94c20veHNtLmg6
NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWlu
dDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Bj
aV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50
MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3Np
Z25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAq
ZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVm
YXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3Vy
Y2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTgg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0
X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2
Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVn
X2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRl
ZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1
cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYs
IGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVw
X21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vf
b2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
aHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1o
dm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIz
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19p
bl9kaXJlY3RtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tZW1fYWNjZXNz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3Rt
YXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4K
Li9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3Jt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90
IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1f
b3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9w
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWlu
X3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9s
aXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IElu
IGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxi
Zmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXAp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3Vu
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoz
MTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3Vy
Y2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
NzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9ldmVudC5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9jaGFyL2NvbnNv
bGUuYzoxNjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9z
aWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29s
IHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5o
OjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDox
OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhE
RUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUo
RU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+
fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAg
ICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAq
aCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g
4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9U
WVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+
fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAg
Y3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChu
eHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBi
b29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192
YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3Rh
dGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0
cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBz
dHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3Rh
dGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVh
ZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
d3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRh
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNz
X3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEu
aDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6
MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZCht
Zm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMy
Yml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8
IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5y
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSBjb21tb24vYXJnby5jOjIwOgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6IEluIGZ1
bmN0aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNsdWRlL3hlbi9k
b21haW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2b2lkIHVubWFw
X2RvbWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjE3LAogICAgICAg
ICAgICAgICAgIGZyb20gY29tbW9uL2FyZ28uYzoyMjoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vYXJnby5jOjI2OgouL2luY2x1ZGUv
eGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNs
dWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBj
aGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkg
e2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBv
dGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUg
PyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAg
ICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9h
Y2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNz
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL2FyZ28uYzoyOToKLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFp
bmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIz
NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
c2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWlu
ICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1
MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21f
ZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZh
dWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91
bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwg
ZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCi4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5faW50ZXJkb21haW7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjcz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyODU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3Rh
dHVz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
OTc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50
MzJfdCBmbGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dy
YW50X2NvcHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0
X2hhcmR3YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzY2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9nZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6
NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRp
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2
YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVu
dF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3BhZ2XigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDA2OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21fcGh5c21h
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9wYWdlc+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY29u
c29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMzOjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzODo0
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2RlZmF1bHRf
dCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zY2hlZG9w
X3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDQgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21hcF9k
b21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWlu
X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRh
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91bm1hcF9k
b21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2Rv
bWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAq
ZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBz
dHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2ly
cV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODkg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1
aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90
IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgogIENDICAgICAgbGliL2xpc3Qtc29ydC5v
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Bl
cm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGlu
ZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3Bs
dWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1
aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29y
ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9k
ZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU2OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90
IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jl
c291cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9k
ZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0
OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhz
bV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2
MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVk
IGxvbmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4
c21faHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1
aW50NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWlu
Zm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
bWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3Vy
Y2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0
aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5
OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAq
Z21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9hcmdvLmM6IElu
IGZ1bmN0aW9uIOKAmHJpbmdidWZfaW5zZXJ04oCZOgpjb21tb24vYXJnby5jOjgxMzo1ODog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhsb25nIHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICA4MTMgfCAgICAgICAgICAgICBzaXplb2Yoc3RydWN0IHhlbl9h
cmdvX3JpbmdfbWVzc2FnZV9oZWFkZXIpKSA+PSBzcCApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KY29tbW9u
L2FyZ28uYzo4ODM6MjI6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDi
gJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDg4MyB8ICAgICAgICAgaWYgKCBpb3ZfbGVu
ID4gc3AgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2ln
bl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9y
ZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRp
YyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBi
b29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rp
b24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEz
MTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9k
b21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBp
bnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gZHJpdmVycy9jaGFy
L2NvbnNvbGUuYzoxODoKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDigJhu
b19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdhcm5p
bmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9u
ZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9
IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVd
CiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9
KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBkcml2ZXJzL2NoYXIvY29u
c29sZS5jOjE5OgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOiBBdCB0b3AgbGV2ZWw6Ci4vaW5j
bHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5pbmc6IOKAmGlubGluZeKAmSBpcyBub3Qg
YXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlvbl0K
ICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhjaV9kYmNfdWFydF9pbml0KHZvaWQpIHt9
CiAgICAgIHwgXn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBkcml2ZXJzL2NoYXIvY29u
c29sZS5jOjI1OgouL2luY2x1ZGUveGVuL3ZpZGVvLmg6IEluIGZ1bmN0aW9uIOKAmHZpZGVv
X3B1dHPigJk6Ci4vaW5jbHVkZS94ZW4vdmlkZW8uaDoyMDo0Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzdHLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdmlkZW9fcHV0cyhjb25zdCBjaGFyICpzdHIsIHNpemVfdCBucikg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hlbi92aWRlby5oOjIwOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZpZGVvX3B1dHMoY29uc3QgY2hhciAqc3RyLCBzaXplX3QgbnIpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRyaXZlcnMvY2hhci9jb25zb2xlLmM6MjY6
Ci4vaW5jbHVkZS94ZW4va2V4ZWMuaDogSW4gZnVuY3Rpb24g4oCYc2V0X2tleGVjX2NyYXNo
X2FyZWFfc2l6ZeKAmToKLi9pbmNsdWRlL3hlbi9rZXhlYy5oOjkzOjUwOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHN5c3RlbV9yYW3igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgOTMgfCBzdGF0aWMgaW5saW5lIHZvaWQgc2V0X2tleGVjX2NyYXNoX2FyZWFfc2l6ZSh1
NjQgc3lzdGVtX3JhbSkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+Xn5+fn5+fn5+fgpjb21tb24vYXJnby5jOiBJbiBmdW5j
dGlvbiDigJhzZW5kduKAmToKY29tbW9uL2FyZ28uYzoyMDc4OjI2OiB3YXJuaW5nOiBvcGVy
YW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGludOKAmSB0byDi
gJh1bnNpZ25lZCBpbnTigJkgZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5k
IFstV3NpZ24tY29tcGFyZV0KIDIwNzggfCAgICAgcmV0dXJuICggcmV0IDwgMCApID8gcmV0
IDogbGVuOwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICBefn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hlbi9jb25maWcuaDoxNywKICAgICAgICAgICAg
ICAgICBmcm9tIDxjb21tYW5kLWxpbmU+OgpsaWIvbGlzdC1zb3J0LmM6IEluIGZ1bmN0aW9u
IOKAmGxpc3Rfc29ydOKAmToKbGliL2xpc3Qtc29ydC5jOjE0Mjo0Mjogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAxNDIgfCAgICAgICAgICAgICAgICAgICAgICAgICBpZiAodW5saWtlbHkobGV2ID49
IEFSUkFZX1NJWkUocGFydCktMSkpIHsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY29tcGlsZXIuaDoyMTo0Mzog
bm90ZTogaW4gZGVmaW5pdGlvbiBvZiBtYWNybyDigJh1bmxpa2VseeKAmQogICAyMSB8ICNk
ZWZpbmUgdW5saWtlbHkoeCkgICBfX2J1aWx0aW5fZXhwZWN0KCEhKHgpLDApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAg
ICAgICBmcm9tIGRyaXZlcnMvY2hhci9jb25zb2xlLmM6MzA6Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRkb21h
aW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
MzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0NiB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFp
biAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWN0bCh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGwoeHNt
X2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjYxOjQ5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFkY29uc29sZSh4c21fZGVm
YXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNzM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MTox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50
X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwOToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMxNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3Ry
YW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyMToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2NvcHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlf
c2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRf
aGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3
MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjM3Njo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc2V0X3BvZF90
YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2Xi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzgxOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4MSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2UoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3Rh
dF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTM6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkz
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
YWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDA2OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQw
NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21fcGh5c21hcOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxODoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0
cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9wYWdlc+KAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY29uc29sZV9pb+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzODo0Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRkb3du
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDQgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21hcF9kb21haW5fcGly
cSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDY1
OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91bm1hcF9kb21haW5fcGly
cSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9pcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3Nzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
dW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgz
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVu
X2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNz
aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFs
bG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUw
NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1aW50MTZfdCBzdGFydCwKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25z
dCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6
NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNp
KHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1
Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNt
X2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJj
ZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291
cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVs
dF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3Vy
Y2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt
X2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVt
X2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2
NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBk
ZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDcz
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMg
fCBzdGF0aWMgaW5saW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKAmToKICBDQyAgICAgIHhzbS94c21fcG9saWN5
Lm8KLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRyaXZl
cnMvY2hhci9jb25zb2xlLmM6MzM6Ci4vaW5jbHVkZS94ZW4vcHZfY29uc29sZS5oOiBJbiBm
dW5jdGlvbiDigJhwdl9jb25zb2xlX3NldF9yeF9oYW5kbGVy4oCZOgouL2luY2x1ZGUveGVu
L3B2X2NvbnNvbGUuaDoxODo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhmbuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxOCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBwdl9j
b25zb2xlX3NldF9yeF9oYW5kbGVyKHNlcmlhbF9yeF9mbiBmbikgeyB9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vcHZfY29uc29sZS5oOiBJbiBmdW5jdGlvbiDigJhwdl9jb25z
b2xlX3B1dHPigJk6Ci4vaW5jbHVkZS94ZW4vcHZfY29uc29sZS5oOjIwOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGJ1ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBwdl9jb25zb2xlX3B1dHMoY29uc3QgY2hhciAqYnVm
LCBzaXplX3QgbnIpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vcHZfY29uc29sZS5oOjIwOjYw
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDIwIHwgc3RhdGljIGlubGluZSB2b2lkIHB2X2NvbnNvbGVfcHV0cyhjb25zdCBj
aGFyICpidWYsIHNpemVfdCBucikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5efgpkcml2ZXJzL2NoYXIv
Y29uc29sZS5jOiBJbiBmdW5jdGlvbiDigJhkb190b2dnbGVfZ3Vlc3TigJk6CmRyaXZlcnMv
Y2hhci9jb25zb2xlLmM6MjgyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVu
dXNlZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MiB8IHN0YXRpYyB2b2lkIGNmX2No
ZWNrIGRvX3RvZ2dsZV9ndWVzdCh1bnNpZ25lZCBjaGFyIGtleSwgYm9vbCB1bnVzZWQpCmRy
aXZlcnMvY2hhci9jb25zb2xlLmM6IEluIGZ1bmN0aW9uIOKAmGRvX2luY190aHJlc2jigJk6
CmRyaXZlcnMvY2hhci9jb25zb2xlLmM6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHVudXNlZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyB2
b2lkIGNmX2NoZWNrIGRvX2luY190aHJlc2godW5zaWduZWQgY2hhciBrZXksIGJvb2wgdW51
c2VkKQpkcml2ZXJzL2NoYXIvY29uc29sZS5jOiBJbiBmdW5jdGlvbiDigJhkb19kZWNfdGhy
ZXNo4oCZOgpkcml2ZXJzL2NoYXIvY29uc29sZS5jOjMxNTo2MDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh1bnVzZWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCBz
dGF0aWMgdm9pZCBjZl9jaGVjayBkb19kZWNfdGhyZXNoKHVuc2lnbmVkIGNoYXIga2V5LCBi
b29sIHVudXNlZCkKZHJpdmVycy9jaGFyL2NvbnNvbGUuYzogSW4gZnVuY3Rpb24g4oCYc3dp
dGNoX3NlcmlhbF9pbnB1dOKAmToKZHJpdmVycy9jaGFyL2NvbnNvbGUuYzo1MDA6MjQ6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDigJhpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDUwMCB8ICAgICAgICAgaWYgKCBuZXh0X3J4KysgPj0gbWF4X2NvbnNvbGVf
cnggKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXn4KZHJpdmVycy9jaGFyL2Nv
bnNvbGUuYzogSW4gZnVuY3Rpb24g4oCYbm90aWZ5X2RvbTBfY29uX3JpbmfigJk6CmRyaXZl
cnMvY2hhci9jb25zb2xlLmM6NjA1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHVudXNlZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNSB8IHN0YXRpYyB2b2lkIGNm
X2NoZWNrIG5vdGlmeV9kb20wX2Nvbl9yaW5nKHZvaWQgKnVudXNlZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn5+fn4KZHJp
dmVycy9jaGFyL2NvbnNvbGUuYzogSW4gZnVuY3Rpb24g4oCYX19wcmludGtfcmF0ZWxpbWl0
4oCZOgpkcml2ZXJzL2NoYXIvY29uc29sZS5jOjEyMjA6MTU6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
bG9uZyB1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAx
MjIwIHwgICAgIGlmICggdG9rcyA+IChyYXRlbGltaXRfYnVyc3QgKiByYXRlbGltaXRfbXMp
KQogICAgICB8ICAgICAgICAgICAgICAgXgpkcml2ZXJzL2NoYXIvY29uc29sZS5jOjEyMjI6
MTU6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYbG9uZyB1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAxMjIyIHwgICAgIGlmICggdG9rcyA+PSByYXRlbGltaXRf
bXMgKQogICAgICB8ICAgICAgICAgICAgICAgXn4KZHJpdmVycy9jaGFyL2NvbnNvbGUuYzog
SW4gZnVuY3Rpb24g4oCYc3VzcGVuZF9zdGVhbF9mbuKAmToKZHJpdmVycy9jaGFyL2NvbnNv
bGUuYzoxMzEzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHN0cuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTMxMyB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIHN1c3BlbmRf
c3RlYWxfZm4oY29uc3QgY2hhciAqc3RyLCBzaXplX3QgbnIpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+CmRyaXZl
cnMvY2hhci9jb25zb2xlLmM6MTMxMzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTMxMyB8IHN0YXRpYyB2b2lkIGNmX2No
ZWNrIHN1c3BlbmRfc3RlYWxfZm4oY29uc3QgY2hhciAqc3RyLCBzaXplX3QgbnIpIHsgfQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hzbS94c20u
aDoxOSwKICAgICAgICAgICAgICAgICBmcm9tIHhzbS94c21fcG9saWN5LmM6MjE6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
c3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCY
X19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2su
aDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAg
ICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBe
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8
ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
ZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTi
gJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5s
aW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgogIENDICAgICAgbGliL21lbWNoci5vCi4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZv
ICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAg
ICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9s
b2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1f
YWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hl
Y2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2Fj
Y2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1
aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3
NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJj
ZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21h
aW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBx
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBi
b29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2so
c3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWdu
X2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBz
dGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFk
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9y
ZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNi
ZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBj
aV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRh
dGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0
YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nl
c3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5j
bHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZ
OgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5
c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyNDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRkb21haW5pbmZv4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9t
Y3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMzU6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21k
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0
bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0NiB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTE6NDQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWN0bCh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjU2OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBk
ZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFkY29uc29sZSh4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNo
bl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNzM6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjcz
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2
dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyODU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRj
aG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyOTc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFu
dF9tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyLCB1aW50MzJfdCBmbGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2dyYW50X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dyYW50X2NvcHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMyNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6
NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9t
YWluKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJn
ZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9nZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3NldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNzYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzgxOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9t
ZW1vcnlfZXhjaGFuZ2UoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5
X2FkanVzdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzg3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlv
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNt
YXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21m
bl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxODoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4g
KnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9wYWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2Vz
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2Nv
bnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tl
eGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzODo0Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21hcF9kb21haW5fcGlycSh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5t
YXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDY1OjU1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2
NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91bm1hcF9kb21haW5fcGlycSh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3NzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzcg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5f
ZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2ly
ceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5k
X3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFsbG93KQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDE6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRf
dCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGNpX2Nv
bmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwNzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90
IG1hY2hpbmVfYmRmLCB1aW50MTZfdCBzdGFydCwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9k
dGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBh
dGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9w
bHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBk
ZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29y
ZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1
aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dz
aeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50
IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlz
Yyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU4NDo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZs
aW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBm
c19vcCh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1
bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRf
dm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2
Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3Jt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90
IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1f
b3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9w
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWlu
X3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGxp
Yi9tZW1jaHJfaW52Lm8KICBDQyAgICAgIGxpYi9tZW1jbXAubwogIENDICAgICAgeHNtL2R1
bW15Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0
ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hzbS9kdW1teS5oOjIxLAogICAg
ICAgICAgICAgICAgIGZyb20geHNtL2R1bW15LmM6MTQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgogIENDICAgICAgbGliL21lbWNweS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5j
dGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2
OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgbGliL21lbW1vdmUubwpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251
bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25p
ZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIz
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3Zj
cHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGxpYi9tZW1zZXQubwpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBm
dW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMy
Yml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2VjdXJpdHlf
ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjEyNToyMDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI1IHwg
ICAgIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2dldGRvbWFpbmluZm8g
KmluZm8pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOjEyNTo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpbmZv4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTI1IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4
ZW5fZG9tY3RsX2dldGRvbWFpbmluZm8gKmluZm8pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9p
bmNsdWRlL3hzbS9kdW1teS5oOjEzMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzc2lkcmVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMxIHwgICAgIFhTTV9ERUZB
VUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1
bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MTQ1OjQzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGNtZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE0NSB8ICAg
ICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRl
L3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKA
mToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjE1MTo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhjbWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNTEgfCBzdGF0aWMgWFNN
X0lOTElORSBpbnQgY2ZfY2hlY2sgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoWFNNX0RFRkFV
TFRfQVJHIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+Ci4vaW5j
bHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4v
aW5jbHVkZS94c20vZHVtbXkuaDoxNTg6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE1OCB8ICAgICBYU01fREVGQVVMVF9B
Ukcgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOjE1
ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTU4IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCBz
dHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxODI6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY21k4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTgyIHwgc3RhdGljIFhTTV9JTkxJTkUgaW50IGNmX2NoZWNrIHhzbV9zeXNjdGwo
WFNNX0RFRkFVTFRfQVJHIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRl
L3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5j
bHVkZS94c20vZHVtbXkuaDoxODg6NzM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Y2xlYXLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODggfCBzdGF0aWMgWFNNX0lOTElO
RSBpbnQgY2ZfY2hlY2sgeHNtX3JlYWRjb25zb2xlKFhTTV9ERUZBVUxUX0FSRyB1aW50MzJf
dCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hbGxvY19zZWN1cml0eV9kb21haW7igJk6
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxOTQ6NzM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBYU01fSU5M
SU5FIGludCBjZl9jaGVjayB4c21fYWxsb2Nfc2VjdXJpdHlfZG9tYWluKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL2R1bW15
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9mcmVlX3NlY3VyaXR5X2RvbWFpbuKAmToKLi9pbmNs
dWRlL3hzbS9kdW1teS5oOjE5OTo3Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk5IHwgc3RhdGljIFhTTV9JTkxJTkUgdm9p
ZCBjZl9jaGVjayB4c21fZnJlZV9zZWN1cml0eV9kb21haW4oc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjIwNTo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhmbGFnc+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDIwNSB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFncykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjI4MTo0Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhvcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MSB8
ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToK
Li9pbmNsdWRlL3hzbS9kdW1teS5oOjMwMjoyMzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwYWdl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAyIHwgICAgIHN0cnVjdCBw
YWdlX2luZm8gKnBhZ2UpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTi
gJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDozMTU6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY2hu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIFhTTV9E
RUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlk
X3QgaWQyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDozMTU6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzE1IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0
Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7i
gJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDozMjI6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY2hhbjHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjIgfCAgICAgWFNN
X0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwg
c3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjMyMzoyMDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjaGFuMuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMyMyB8ICAgICBzdHJ1Y3QgZXZ0Y2huICpjaGFuMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZXZ0Y2huX2Nsb3NlX3Bvc3TigJk6Ci4vaW5jbHVkZS94c20vZHVt
bXkuaDozMjk6NzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzI5IHwgc3RhdGljIFhTTV9JTkxJTkUgdm9pZCBjZl9jaGVj
ayB4c21fZXZ0Y2huX2Nsb3NlX3Bvc3Qoc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjMzNTo1NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjaG7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMzUgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBl
dnRjaG4gKmNobikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MzQy
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNobuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM0MiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2FsbG9jX3NlY3VyaXR5X2V2dGNobnPigJk6Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDozNTY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hu
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzU2IHwgICAgIHN0cnVjdCBldnRjaG4gY2hu
W10sIHVuc2lnbmVkIGludCBucikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6MzU2OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzU2IHwgICAgIHN0cnVjdCBldnRj
aG4gY2huW10sIHVuc2lnbmVkIGludCBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2ZyZWVfc2VjdXJpdHlfZXZ0Y2huc+KAmToKLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOjM2MjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjaG7igJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjIgfCAgICAgc3RydWN0IGV2dGNobiBjaG5bXSwgdW5zaWdu
ZWQgaW50IG5yKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDozNjI6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjIgfCAgICAgc3RydWN0IGV2dGNobiBjaG5bXSwg
dW5zaWduZWQgaW50IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
c2hvd19zZWN1cml0eV9ldnRjaG7igJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDozNjg6MjA6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2OCB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBzdHJ1Y3QgZXZ0Y2huICpj
aG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjM2ODo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjaG7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjggfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0
IGV2dGNobiAqY2huKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NDI3OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGR0cGF0aOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQyNyB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgY29u
c3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL2R1
bW15Lmg6NDM0OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGR0cGF0aOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzNCB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0
IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+fn5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6NDU1OjMwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hY2hpbmVfYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU1IHwgICAgIFhT
TV9ERUZBVUxUX0FSRyB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDo0NjI6MzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFjaGlu
ZV9iZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjIgfCAgICAgWFNNX0RFRkFVTFRf
QVJHIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6
NDY5OjMwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hY2hpbmVfYmRm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY5IHwgICAgIFhTTV9ERUZBVUxUX0FSRyB1aW50MzJf
dCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jl
c291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjQ3NTo3NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnc2nigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzUgfCBzdGF0aWMgWFNNX0lOTElORSBpbnQgY2ZfY2hlY2sgeHNtX3Jlc291cmNlX3Nl
dHVwX2dzaShYU01fREVGQVVMVF9BUkcgaW50IGdzaSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9w
YWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo0ODc6NzQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYY21k4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg3
IHwgc3RhdGljIFhTTV9JTkxJTkUgaW50IGNmX2NoZWNrIHhzbV9wYWdlX29mZmxpbmUoWFNN
X0RFRkFVTFRfQVJHIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvX3hzbV9v
cOKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjQ5OTo3Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhvcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5OSB8IHN0YXRpYyBY
U01fSU5MSU5FIGxvbmcgY2ZfY2hlY2sgeHNtX2RvX3hzbV9vcChYRU5fR1VFU1RfSEFORExF
X1BBUkFNKHZvaWQpIG9wKQouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zaG93X2lycV9zaWTigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1MTE6NTU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTExIHwgc3RhdGljIFhTTV9JTkxJTkUgY2hhciAqY2ZfY2hlY2sgeHNtX3Nob3dfaXJx
X3NpZChpbnQgaXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjUyNDo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpcnHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1MjQgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4g
KmQsIGludCBpcnEsIGNvbnN0IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1
MjQ6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDUyNCB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGlycSwgY29uc3Qgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4v
aW5jbHVkZS94c20vZHVtbXkuaDo1Mzg6NzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYYmluZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzOCB8ICAgICBYU01fREVGQVVM
VF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEg
KmJpbmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveHNtL2R1bW15
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6NTQ1OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGJpbmTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDUgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjU1Mjo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpcnHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1NTIgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4g
KmQsIGludCBpcnEsIGNvbnN0IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1
NTI6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU1MiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGlycSwgY29uc3Qgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1NTk6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OSB8ICAgICBYU01fREVG
QVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6NTU5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGFsbG934oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU5IHwgICAgIFhTTV9ERUZB
VUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1NjY6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU2NiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qg
cywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjU2Njo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTY2IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpk
LCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
XgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NTY2OjcxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGFsbG934oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY2IHwgICAgIFhTTV9E
RUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1
aW50OF90IGFsbG93KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5efn5+fgouL2luY2x1ZGUv
eHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2lu
Y2x1ZGUveHNtL2R1bW15Lmg6NTczOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzMgfCAgICAgWFNNX0RFRkFVTFRfQVJH
IHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxs
b3cpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDo1NzM6NjA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3MyB8ICAgICBYU01f
REVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwg
dWludDhfdCBhbGxvdykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOjU3
Mzo3MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhhbGxvd+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU3MyB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6
NTgwOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hY2hpbmVfYmRm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTgwIHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3Qg
ZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
fn5+fn5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NTgwOjcwOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHN0YXJ04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTgwIHwgICAg
IFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2Jk
ZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn4KLi9pbmNs
dWRlL3hzbS9kdW1teS5oOjU4MToxNDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhl
bmTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODEgfCAgICAgdWludDE2X3QgZW5kLCB1
aW50OF90IGFjY2VzcykKICAgICAgfCAgICAgfn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
ZHVtbXkuaDo1ODE6Mjc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYWNjZXNz4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTgxIHwgICAgIHVpbnQxNl90IGVuZCwgdWludDhf
dCBhY2Nlc3MpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5efn5+fn4KLi9p
bmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6NjA5OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG9w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA5IHwgICAgIFhTTV9ERUZBVUxU
X0FSRyBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2lu
Y2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s
4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NjQzOjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1vZGXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NDMgfCAgICAgWFNN
X0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fgouL2lu
Y2x1ZGUveHNtL2R1bW15Lmg6NjQzOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG9w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjQzIHwgICAgIFhTTV9ERUZBVUxUX0FS
RyBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efgouL2luY2x1
ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9p
bmNsdWRlL3hzbS9kdW1teS5oOjY3Mzo3Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhvcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY3MyB8IHN0YXRpYyBYU01fSU5MSU5F
IGludCBjZl9jaGVjayB4c21fcGxhdGZvcm1fb3AoWFNNX0RFRkFVTFRfQVJHIHVpbnQzMl90
IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94c20vZHVtbXku
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2FyZ29fZW5hYmxl4oCZOgouL2luY2x1ZGUveHNtL2R1
bW15Lmg6NzkzOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA3OTMgfCBzdGF0aWMgWFNNX0lOTElORSBpbnQgY2ZfY2hlY2sg
eHNtX2FyZ29fZW5hYmxlKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Fy
Z29fcmVnaXN0ZXJfc2luZ2xlX3NvdXJjZeKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjc5
OToyNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNzk5IHwgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVj
dCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNs
dWRlL3hzbS9kdW1teS5oOjc5OTo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh0
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzk5IHwgICAgIGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQsIGNvbnN0IHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXJnb19yZWdpc3Rlcl9hbnlfc291cmNl4oCZOgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6ODA1OjI2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA4MDUgfCAgICAgY29uc3Qgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgouL2luY2x1
ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hcmdvX3NlbmTigJk6Ci4vaW5j
bHVkZS94c20vZHVtbXkuaDo4MTE6MjY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDgxMSB8ICAgICBjb25zdCBzdHJ1Y3QgZG9t
YWluICpkLCBjb25zdCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDo4MTE6NTA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYdOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDgxMSB8ICAg
ICBjb25zdCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBzdHJ1Y3QgZG9tYWluICp0KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CiAgQ0MgICAgICBsaWIvbXVsZGl2NjQubwogIENDICAgICAgbGliL3BhcnNlLXNpemUubwog
IENDICAgICAgbGliL3JidHJlZS5vCiAgQ0MgICAgICBkcml2ZXJzL2NoYXIvbnMxNjU1MC5v
CiAgQ0MgICAgICBsaWIvc29ydC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL2lycS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL2NoYXIvbnMxNjU1
MC5jOjIyOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MzYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL2lycS5oOjU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGlu
dCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
eGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5efn5+CiAgQ0MgICAgICBsaWIvc3RyY2FzZWNtcC5vCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSBkcml2ZXJzL2NoYXIvbnMxNjU1MC5jOjIzOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6
IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJh
bS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25l
ZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25n
IHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5k
IFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQo
ZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hh
ci9uczE2NTUwLmM6MjQ6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJht
Zm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwg
c3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
ZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1f
bWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3Ap
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMg
aW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFn
ZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5
cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5l
IHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21l
bW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYw
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xp
Y3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21w
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2
aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5s
aW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMg
aW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWdu
ZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwg
c3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6
NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVk
IGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBf
X211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKA
mGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWlu
KGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gZHJpdmVycy9jaGFyL25zMTY1NTAuYzoyNjoKLi9pbmNsdWRlL3hl
bi9zZXJpYWwuaDogQXQgdG9wIGxldmVsOgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2Njox
OiB3YXJuaW5nOiDigJhpbmxpbmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJh
dGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5s
aW5lIHhoY2lfZGJjX3VhcnRfaW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgogIENDICAg
ICAgbGliL3N0cmNoci5vCmRyaXZlcnMvY2hhci9uczE2NTUwLmM6IEluIGZ1bmN0aW9uIOKA
mG5zMTY1NTBfaW50ZXJydXB04oCZOgpkcml2ZXJzL2NoYXIvbnMxNjU1MC5jOjE3ODo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxNzggfCBzdGF0aWMgdm9pZCBjZl9jaGVjayBuczE2NTUwX2ludGVycnVwdChpbnQg
aXJxLCB2b2lkICpkZXZfaWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fl5+fgpkcml2ZXJzL2NoYXIvbnMxNjU1MC5jOiBJbiBmdW5jdGlv
biDigJhwY2lfc2VyaWFsX2Vhcmx5X2luaXTigJk6CmRyaXZlcnMvY2hhci9uczE2NTUwLmM6
MjcyOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVhcnTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzIgfCBzdGF0aWMgdm9pZCBwY2lfc2VyaWFsX2Vhcmx5X2luaXQo
c3RydWN0IG5zMTY1NTAgKnVhcnQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+CmRyaXZlcnMvY2hhci9uczE2NTUwLmM6
IEluIGZ1bmN0aW9uIOKAmG5zMTY1NTBfaW5pdF9pcnHigJk6CmRyaXZlcnMvY2hhci9uczE2
NTUwLmM6Mzg1OjY2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBvcnTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzODUgfCBzdGF0aWMgdm9pZCBfX2luaXQgY2ZfY2hlY2sg
bnMxNjU1MF9pbml0X2lycShzdHJ1Y3Qgc2VyaWFsX3BvcnQgKnBvcnQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+CmRyaXZlcnMvY2hhci9uczE2NTUwLmM6IEluIGZ1bmN0aW9uIOKAmG5z
MTY1NTBfZW5kYm9vdOKAmToKZHJpdmVycy9jaGFyL25zMTY1NTAuYzo1ODY6NjU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcG9ydOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU4NiB8IHN0YXRpYyB2b2lkIF9faW5pdCBjZl9jaGVjayBuczE2NTUwX2VuZGJvb3Qoc3Ry
dWN0IHNlcmlhbF9wb3J0ICpwb3J0KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmRyaXZlcnMv
Y2hhci9uczE2NTUwLmM6IEluIGZ1bmN0aW9uIOKAmG5zMTY1NTBfdWFydF9kdF9pbml04oCZ
Ogpkcml2ZXJzL2NoYXIvbnMxNjU1MC5jOjE3NTg6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTc1OCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHZvaWQgKmRhdGEpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gZHJpdmVycy9jaGFyL25zMTY1NTAuYzoxODM4
OgouL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRfY3N0YXRl
X2xpbWl04oCZOgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTI6NTU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTky
IHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdGF0ZV9saW1pdCh1bnNpZ25lZCBp
bnQgbmV3X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgouL2luY2x1ZGUv
eGVuL2FjcGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl04oCZ
OgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTQ6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3
X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgpkcml2ZXJzL2NoYXIv
bnMxNjU1MC5jOiBJbiBmdW5jdGlvbiDigJhuczE2NTUwX2FjcGlfdWFydF9pbml04oCZOgpk
cml2ZXJzL2NoYXIvbnMxNjU1MC5jOjE4NDA6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTg0MCB8IHN0YXRpYyBpbnQg
X19pbml0IG5zMTY1NTBfYWNwaV91YXJ0X2luaXQoY29uc3Qgdm9pZCAqZGF0YSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fl5+fn4KICBDQyAgICAgIGxpYi9zdHJjbXAubwogIENDICAgICAgbGliL3N0cmxjYXQubwps
aWIvc3RybGNhdC5jOiBJbiBmdW5jdGlvbiDigJhzdHJsY2F04oCZOgpsaWIvc3RybGNhdC5j
OjIzOjI3OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGxvbmcgaW504oCZIGFuZCDigJhzaXplX3TigJkg
e2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogICAyMyB8
ICAgICAgICAgd2hpbGUgKChwIC0gZGVzdCkgPCBzaXplKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXgogIENDICAgICAgbGliL3N0cmxjcHkubwogIENDICAgICAgbGli
L3N0cmxlbi5vCiAgQ0MgICAgICBsaWIvc3RybmNhc2VjbXAubwogIENDICAgICAgeHNtL3Np
bG8ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveHNtL2R1bW15Lmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSB4c20vc2lsby5jOjIxOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3Rh
dGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
ICBDQyAgICAgIGxpYi9zdHJuY21wLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
cHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0
aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xB
UkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6
Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
cndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19s
YXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIz
MCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAg
IF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAq
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CiAgQ0MgICAgICBkcml2ZXJzL2NoYXIvY2FkZW5j
ZS11YXJ0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9p
bmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5j
bHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVp
ZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVj
dG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0
X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hf
ZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMy
X3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
dW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgbGliL3N0cm5sZW4ubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19l
bXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNv
dXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZj
cHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1
bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
dm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5h
dGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Rp
bWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxMywK
ICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9jYWRlbmNlLXVhcnQuYzoyMToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1
bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
cHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlv
biDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9T
QVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09N
UEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBh
cmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3Bv
bGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
OSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1
Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDogQXQg
dG9wIGxldmVsOgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2NjoxOiB3YXJuaW5nOiDigJhp
bmxpbmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5
bGUtZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5saW5lIHhoY2lfZGJjX3Vh
cnRfaW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
Nzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2Rldmlj
ZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNp
X2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGlu
ZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlu
bGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVk
IGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19t
dXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJx
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9jYWRlbmNlLXVhcnQu
YzoyMzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9pcnEuaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9v
bCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3NlY3VyaXR5X2RvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxMjU6MjA6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyNSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9nZXRk
b21haW5pbmZvICppbmZvKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS94c20vZHVtbXkuaDoxMjU6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaW5m
b+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyNSB8ICAgICBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgeGVuX2RvbWN0bF9nZXRkb21haW5pbmZvICppbmZvKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVh
dGXigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxMzE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc3NpZHJlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMSB8ICAg
ICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9t
Y3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjE0NTo0Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjbWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxNDUgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hl
ZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxNTE6NzY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY21k4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTUxIHwg
c3RhdGljIFhTTV9JTkxJTkUgaW50IGNmX2NoZWNrIHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w
KFhTTV9ERUZBVUxUX0FSRyBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFy
Z2V04oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MTU4OjM2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNTggfCAgICAgWFNN
X0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20v
ZHVtbXkuaDoxNTg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE1OCB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRv
bWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNtL2R1bW15
Lmg6MTgyOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNtZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE4MiB8IHN0YXRpYyBYU01fSU5MSU5FIGludCBjZl9jaGVjayB4
c21fc3lzY3RsKFhTTV9ERUZBVUxUX0FSRyBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl
4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MTg4OjczOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNsZWFy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg4IHwgc3RhdGlj
IFhTTV9JTkxJTkUgaW50IGNmX2NoZWNrIHhzbV9yZWFkY29uc29sZShYU01fREVGQVVMVF9B
UkcgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn4KLi9p
bmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWxsb2Nfc2VjdXJpdHlf
ZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MTk0OjczOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0
aWMgWFNNX0lOTElORSBpbnQgY2ZfY2hlY2sgeHNtX2FsbG9jX3NlY3VyaXR5X2RvbWFpbihz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRl
L3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZnJlZV9zZWN1cml0eV9kb21haW7i
gJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoxOTk6NzM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5OSB8IHN0YXRpYyBYU01f
SU5MSU5FIHZvaWQgY2ZfY2hlY2sgeHNtX2ZyZWVfc2VjdXJpdHlfZG9tYWluKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL2R1
bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDoyMDU6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZmxhZ3Pi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMDUgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoyODE6NDM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3DigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODEgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIGludCBv
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5e
fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGlu
X3BhZ2XigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDozMDI6MjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGFnZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMiB8ICAg
ICBzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNo
bl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MzE1OjU0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGNobuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8
ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAq
Y2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MzE1
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkMuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxNSB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX2lu
dGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MzIyOjU1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGNoYW4x4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIy
IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNo
biAqY2hhbjEsIHN0cnVjdCBkb21haW4gKmQyLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDozMjM6MjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hhbjLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjMgfCAgICAgc3RydWN0IGV2dGNobiAqY2hh
bjIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94c20vZHVt
bXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9jbG9zZV9wb3N04oCZOgouL2luY2x1
ZGUveHNtL2R1bW15Lmg6MzI5OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNo
buKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyOSB8IHN0YXRpYyBYU01fSU5MSU5FIHZv
aWQgY2ZfY2hlY2sgeHNtX2V2dGNobl9jbG9zZV9wb3N0KHN0cnVjdCBldnRjaG4gKmNobikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDoz
MzU6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzM1IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1teS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS9k
dW1teS5oOjM0Mjo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjaG7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNDIgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hbGxvY19zZWN1cml0eV9ldnRjaG5z4oCZ
OgouL2luY2x1ZGUveHNtL2R1bW15Lmg6MzU2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGNobuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1NiB8ICAgICBzdHJ1Y3Qg
ZXZ0Y2huIGNobltdLCB1bnNpZ25lZCBpbnQgbnIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOjM1NjozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1NiB8ICAgICBz
dHJ1Y3QgZXZ0Y2huIGNobltdLCB1bnNpZ25lZCBpbnQgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveHNtL2R1bW15Lmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9mcmVlX3NlY3VyaXR5X2V2dGNobnPigJk6Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDozNjI6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hu
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYyIHwgICAgIHN0cnVjdCBldnRjaG4gY2hu
W10sIHVuc2lnbmVkIGludCBucikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6MzYyOjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYyIHwgICAgIHN0cnVjdCBldnRj
aG4gY2huW10sIHVuc2lnbmVkIGludCBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3Nob3dfc2VjdXJpdHlfZXZ0Y2hu4oCZOgouL2luY2x1ZGUveHNtL2R1bW15
Lmg6MzY4OjIwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjggfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0
IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94
c20vZHVtbXkuaDozNjg6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY2hu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY4IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIGNv
bnN0IHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjQyNzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkdHBhdGjigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MjcgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21h
aW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn5+fgouL2luY2x1ZGUveHNtL2R1
bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNs
dWRlL3hzbS9kdW1teS5oOjQzNDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
dHBhdGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzQgfCAgICAgWFNNX0RFRkFVTFRf
QVJHIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVn
X3BjaeKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjQ1NTozMDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYWNoaW5lX2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NSB8ICAgICBYU01fREVGQVVMVF9BUkcgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6NDYyOjMwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hY2hpbmVfYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYyIHwgICAgIFhT
TV9ERUZBVUxUX0FSRyB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hz
bS9kdW1teS5oOjQ2OTozMDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYWNoaW5l
X2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2OSB8ICAgICBYU01fREVGQVVMVF9B
UkcgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+fn5+fn5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo0
NzU6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ3Np4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc1IHwgc3RhdGljIFhTTV9JTkxJTkUgaW50IGNmX2NoZWNrIHhzbV9y
ZXNvdXJjZV9zZXR1cF9nc2koWFNNX0RFRkFVTFRfQVJHIGludCBnc2kpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NDg3Ojc0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNtZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ4NyB8IHN0YXRpYyBYU01fSU5MSU5FIGludCBjZl9jaGVjayB4c21fcGFnZV9v
ZmZsaW5lKFhTTV9ERUZBVUxUX0FSRyB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9kb194c21fb3DigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo0OTk6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTkg
fCBzdGF0aWMgWFNNX0lOTElORSBsb25nIGNmX2NoZWNrIHhzbV9kb194c21fb3AoWEVOX0dV
RVNUX0hBTkRMRV9QQVJBTSh2b2lkKSBvcCkKLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fc2hvd19pcnFfc2lk4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6
NTExOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDUxMSB8IHN0YXRpYyBYU01fSU5MSU5FIGNoYXIgKmNmX2NoZWNrIHhz
bV9zaG93X2lycV9zaWQoaW50IGlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZy
b20gZHJpdmVycy9jaGFyL2NhZGVuY2UtdWFydC5jOjI3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6NTI0OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDUyNCB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0
IGRvbWFpbiAqZCwgaW50IGlycSwgY29uc3Qgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRlL3hzbS9k
dW1teS5oOjUyNDo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTI0IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3Qg
ZG9tYWluICpkLCBpbnQgaXJxLCBjb25zdCB2b2lkICpkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2ly
ceKAmToKLi9pbmNsdWRlL3hzbS9kdW1teS5oOjUzODo3MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhiaW5k4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM4IHwgICAgIFhT
TV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5k
X3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5j
bHVkZS94c20vZHVtbXkuaDo1NDU6NzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
YmluZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NSB8ICAgICBYU01fREVGQVVMVF9B
Ukcgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJp
bmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6NTUyOjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0
IGRvbWFpbiAqZCwgaW50IGlycSwgY29uc3Qgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9pbmNsdWRlL3hzbS9k
dW1teS5oOjU1Mjo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUyIHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3Qg
ZG9tYWluICpkLCBpbnQgaXJxLCBjb25zdCB2b2lkICpkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlz
c2lvbuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
bmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4
IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDo1NTk6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGly
ceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OSB8ICAgICBYU01fREVGQVVMVF9BUkcg
c3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fgouL2luY2x1ZGUv
eHNtL2R1bW15Lmg6NTU5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGFsbG93
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU5IHwgICAgIFhTTV9ERUZBVUxUX0FSRyBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fl5+
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNo
X2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgou
L2luY2x1ZGUveHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNz
aW9u4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NTY2OjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjYgfCAgICAgWFNN
X0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUs
IHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDo1NjY6NjA6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2
NiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qgcywg
dWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl4KLi9pbmNsdWRlL3hz
bS9kdW1teS5oOjU2Njo3MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhhbGxvd+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2NiB8ICAgICBYU01fREVGQVVMVF9BUkcgc3Ry
dWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS9kdW1t
eS5oOjU3Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTczIHwgICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWlu
ICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+XgouL2luY2x1
ZGUveHNtL2R1bW15Lmg6NTczOjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzMgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0
cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20vZHVtbXkuaDo1NzM6NzE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYYWxsb3figJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
NzMgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMs
IHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fl5+
fn5+Ci4vaW5jbHVkZS94c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25m
aWdfcGVybWlzc2lvbuKAmToKZHJpdmVycy9jaGFyL2NhZGVuY2UtdWFydC5jOiBJbiBmdW5j
dGlvbiDigJhjdWFydF9pbnRlcnJ1cHTigJk6CmRyaXZlcnMvY2hhci9jYWRlbmNlLXVhcnQu
Yzo0MzozMzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpcnHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNDMgfCBzdGF0aWMgdm9pZCBjdWFydF9pbnRlcnJ1cHQoaW50IGly
cSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fl5+fgouL2luY2x1ZGUveHNtL2R1bW15Lmg6NTgwOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hY2hpbmVfYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTgwIHwg
ICAgIFhTTV9ERUZBVUxUX0FSRyBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5l
X2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fgpkcml2ZXJzL2NoYXIvY2FkZW5j
ZS11YXJ0LmM6IEluIGZ1bmN0aW9uIOKAmGN1YXJ0X2lyceKAmToKZHJpdmVycy9jaGFyL2Nh
ZGVuY2UtdWFydC5jOjEyMzo0NDogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFu
Z2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5zaWduZWQgaW504oCZIGR1
ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAg
MTIzIHwgICAgIHJldHVybiAoICh1YXJ0LT5pcnEgPiAwKSA/IHVhcnQtPmlycSA6IC0xICk7
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo1ODA6NzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc3RhcnTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODAgfCAgICAgWFNNX0RF
RkFVTFRfQVJHIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1aW50
MTZfdCBzdGFydCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6NTgxOjE0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGVuZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4MSB8ICAgICB1aW50MTZfdCBlbmQsIHVpbnQ4X3Qg
YWNjZXNzKQogICAgICB8ICAgICB+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS9kdW1teS5o
OjU4MToyNzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhhY2Nlc3PigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1ODEgfCAgICAgdWludDE2X3QgZW5kLCB1aW50OF90IGFjY2Vz
cykKICAgICAgfCAgICAgICAgICAgICAgICAgICB+fn5+fn5+fl5+fn5+fgouL2luY2x1ZGUv
eHNtL2R1bW15Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDo2MDk6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Di
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDkgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4v
aW5jbHVkZS94c20vZHVtbXkuaDo2NDM6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbW9kZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY0MyB8ICAgICBYU01fREVGQVVM
VF9BUkcgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn5+Ci4vaW5jbHVkZS94
c20vZHVtbXkuaDo2NDM6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3DigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NDMgfCAgICAgWFNNX0RFRkFVTFRfQVJHIHN0cnVj
dCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+Ci4vaW5jbHVkZS94c20v
ZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1ZGUv
eHNtL2R1bW15Lmg6NjczOjczOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjczIHwgc3RhdGljIFhTTV9JTkxJTkUgaW50IGNm
X2NoZWNrIHhzbV9wbGF0Zm9ybV9vcChYU01fREVGQVVMVF9BUkcgdWludDMyX3Qgb3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fYXJnb19lbmFibGXigJk6Ci4vaW5jbHVkZS94c20vZHVtbXkuaDo3
OTM6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDc5MyB8IHN0YXRpYyBYU01fSU5MSU5FIGludCBjZl9jaGVjayB4c21fYXJn
b19lbmFibGUoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXJnb19yZWdp
c3Rlcl9zaW5nbGVfc291cmNl4oCZOgouL2luY2x1ZGUveHNtL2R1bW15Lmg6Nzk5OjI2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA3OTkgfCAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0IGRvbWFp
biAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNt
L2R1bW15Lmg6Nzk5OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA3OTkgfCAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqZCwg
Y29uc3Qgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL2R1bW15Lmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9hcmdvX3JlZ2lzdGVyX2FueV9zb3VyY2XigJk6Ci4vaW5jbHVk
ZS94c20vZHVtbXkuaDo4MDU6MjY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDgwNSB8ICAgICBjb25zdCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20v
ZHVtbXkuaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FyZ29fc2VuZOKAmToKLi9pbmNsdWRlL3hz
bS9kdW1teS5oOjgxMToyNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgODExIHwgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKmQs
IGNvbnN0IHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KLi9pbmNsdWRlL3hzbS9kdW1teS5oOjgxMTo1MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgODExIHwgICAgIGNvbnN0
IHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KICBDQyAg
ICAgIGxpYi9zdHJwYnJrLm8KICBDQyAgICAgIGxpYi9zdHJyY2hyLm8KICBDQyAgICAgIGRy
aXZlcnMvY2hhci9wbDAxMS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NlcmlhbC5oOjEzLAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9jaGFyL3Bs
MDExLmM6MjE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5j
dGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJf
c2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3Nlcmlh
bC5oOiBBdCB0b3AgbGV2ZWw6Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5p
bmc6IOKAmGlubGluZeKAmSBpcyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFst
V29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhj
aV9kYmNfdWFydF9pbml0KHZvaWQpIHt9CiAgICAgIHwgXn5+fn5+CiAgQ0MgICAgICBsaWIv
c3Ryc2VwLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9wbDAxMS5jOjIzOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAg
ICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAg
aWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUg
fCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2lycS5o
OjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
bG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwg
c3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9s
b2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL2NoYXIvcGww
MTEuYzoyNzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAo
dW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwy
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRl
cuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBh
Z2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpkcml2ZXJz
L2NoYXIvcGwwMTEuYzogSW4gZnVuY3Rpb24g4oCYcGwwMTFfaW50ZXJydXB04oCZOgpkcml2
ZXJzL2NoYXIvcGwwMTEuYzo4NTozMzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhp
cnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgODUgfCBzdGF0aWMgdm9pZCBwbDAxMV9p
bnRlcnJ1cHQoaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fl5+fgpkcml2ZXJzL2NoYXIvcGwwMTEuYzogSW4gZnVuY3Rpb24g
4oCYcGwwMTFfaXJx4oCZOgpkcml2ZXJzL2NoYXIvcGwwMTEuYzoxOTg6NDM6IHdhcm5pbmc6
IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYaW504oCZ
IHRvIOKAmHVuc2lnbmVkIGludOKAmSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9w
ZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5OCB8ICAgICByZXR1cm4gKCh1YXJ0LT5pcnEg
PiAwKSA/IHVhcnQtPmlycSA6IC0xKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBefgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3NldHVwLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWNwaS5oOjI1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU4LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL2FjcGkuaDo1NywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMv
Y2hhci9wbDAxMS5jOjMzODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJz
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGxpYi9zdHJzcG4ubwouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVz
b3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRw
Mm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwg
dW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KICBNS0ZMQVNLIHhzbS9mbGFzay9pbmNsdWRlL2ZsYXNrLmgKLi9pbmNsdWRlL3hlbi9h
Y3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNs
dWRlL3hlbi9hY3BpLmg6MTkyOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5l
d19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUg
dm9pZCBhY3BpX3NldF9jc3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyBy
ZXR1cm47IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1
bmN0aW9uIOKAmGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9h
Y3BpLmg6MTk0OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3Bp
X3NldF9jc3Vic3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47
IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KZHJpdmVycy9jaGFyL3BsMDExLmM6IEluIGZ1bmN0
aW9uIOKAmHBsMDExX2FjcGlfdWFydF9pbml04oCZOgpkcml2ZXJzL2NoYXIvcGwwMTEuYzoz
NDA6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM0MCB8IHN0YXRpYyBpbnQgX19pbml0IHBsMDExX2FjcGlfdWFydF9p
bml0KGNvbnN0IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+fgogIE1LQUNDRVNTIFZFQ1RPUiB4c20v
Zmxhc2svaW5jbHVkZS9hdl9wZXJtX3RvX3N0cmluZy5oCiAgQ0MgICAgICB4c20vZmxhc2sv
YXZjLm8KICBDQyAgICAgIGxpYi9zdHJzdHIubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
eHNtL2ZsYXNrL2F2Yy5jOjIzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBp
bnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDo3LAogICAgICAgICAgICAgICAgIGZyb20geHNtL2ZsYXNr
L2F2Yy5jOjI2OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19sYXN04oCZOgogIENDICAgICAgbGliL3N0cnRvbC5vCi4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5y
X2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNp
Z25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5y
X2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxv
Y19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hl
bi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90
b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5o
OjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKA
mGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcy
OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90
aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNz
Lmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlv
biDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vz
c19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
c3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJv
b2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2Fs
dHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9h
bHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dm
buKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAg
ICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwg
cDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIENDICAgICAgbGliL3N0cnRvbGwubwouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQg
dnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xgp4c20vZmxhc2svYXZjLmM6IEluIGZ1bmN0aW9uIOKAmGF2Y19wcmludGvigJk6
CnhzbS9mbGFzay9hdmMuYzoxMjY6MTI6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICAx
MjYgfCAgICAgaWYgKCBpIDwgYnVmLT5mcmVlICkKICAgICAgfCAgICAgICAgICAgIF4KeHNt
L2ZsYXNrL2F2Yy5jOiBJbiBmdW5jdGlvbiDigJhhdmNfZHVtcF9hduKAmToKeHNtL2ZsYXNr
L2F2Yy5jOjE2NzoxNTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAxNjcgfCAgICAgd2hpbGUgKCBpIDwg
c2l6ZW9mKGF2KSAqIDggKQogICAgICB8ICAgICAgICAgICAgICAgXgp4c20vZmxhc2svYXZj
LmM6MTcxOjMwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYbG9uZyB1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDE3MSB8ICAgICAgICAgICAgIGZvciAoIGky
ID0gMDsgaTIgPCBBUlJBWV9TSVpFKGF2X3Blcm1fdG9fc3RyaW5nKTsgaTIrKyApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCnhzbS9mbGFzay9hdmMuYzoxNzQ6
NTE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBh
bmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAxNzQgfCAgICAgICAgICAgICAgICAg
ICAgICAoYXZfcGVybV90b19zdHJpbmdbaTJdLnZhbHVlID09IHBlcm0pICkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+Cnhz
bS9mbGFzay9hdmMuYzoxNzc6MjE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhs
b25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTc3IHwgICAgICAgICAg
ICAgaWYgKCBpMiA8IEFSUkFZX1NJWkUoYXZfcGVybV90b19zdHJpbmcpICkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgIF4KeHNtL2ZsYXNrL2F2Yy5jOiBJbiBmdW5jdGlvbiDigJhh
dmNfYWxsb2Nfbm9kZeKAmToKeHNtL2ZsYXNrL2F2Yy5jOjM1ODo0Nzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MzU4IHwgICAgIGlmICggYXRvbWljX3JlYWQoJmF2Y19jYWNoZS5hY3RpdmVfbm9kZXMpID4g
YXZjX2NhY2hlX3RocmVzaG9sZCApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIF4KeHNtL2ZsYXNrL2F2Yy5jOiBJbiBmdW5jdGlvbiDi
gJhhdmNfbGF0ZXN0X25vdGlmX3VwZGF0ZeKAmToKeHNtL2ZsYXNrL2F2Yy5jOjQzNToyMDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVk
IGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDQzNSB8ICAgICAgICAgaWYgKCBzZXFubyA8
IGF2Y19jYWNoZS5sYXRlc3Rfbm90aWYgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBe
CnhzbS9mbGFzay9hdmMuYzo0NDQ6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICA0
NDQgfCAgICAgICAgIGlmICggc2Vxbm8gPiBhdmNfY2FjaGUubGF0ZXN0X25vdGlmICkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgXgogIENDICAgICAgbGliL3N0cnRvdWwubwogIEND
ICAgICAgbGliL3N0cnRvdWxsLm8KICBDQyAgICAgIGxpYi9idWlsdF9pbi5vCiAgQVIgICAg
ICBsaWIvbGliLmEKICBDQyAgICAgIGRyaXZlcnMvY2hhci9tZXNvbi11YXJ0Lm8KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIGRyaXZlcnMvY2hhci9tZXNvbi11YXJ0LmM6MjI6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJu
YXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90
aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoz
NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3Vu
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSBk
cml2ZXJzL2NoYXIvbWVzb24tdWFydC5jOjIzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBkcml2ZXJzL2NoYXIvbWVzb24t
dWFydC5jOjI0OgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOiBBdCB0b3AgbGV2ZWw6Ci4vaW5j
bHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5pbmc6IOKAmGlubGluZeKAmSBpcyBub3Qg
YXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlvbl0K
ICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhjaV9kYmNfdWFydF9pbml0KHZvaWQpIHt9
CiAgICAgIHwgXn5+fn5+CmRyaXZlcnMvY2hhci9tZXNvbi11YXJ0LmM6IEluIGZ1bmN0aW9u
IOKAmG1lc29uX3VhcnRfaW50ZXJydXB04oCZOgpkcml2ZXJzL2NoYXIvbWVzb24tdWFydC5j
OjY0OjM4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA2NCB8IHN0YXRpYyB2b2lkIG1lc29uX3VhcnRfaW50ZXJydXB0KGlu
dCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fl5+fgogIExEUyAgICAgYXJjaC9hcm0veGVuLmxkcwogIENDICAgICAgZHJp
dmVycy9jaGFyL212ZWJ1LXVhcnQubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9pcnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9jaGFyL212ZWJ1
LXVhcnQuYzoyMjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAg
ICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAg
ICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxs
b2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tf
dmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192
YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjM2LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9pcnEuaDo1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5s
b2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5o
OjIzMywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9tdmVidS11YXJ0LmM6
MjM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNf
aW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6
NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1
Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwg
c3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVp
bnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Zmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3Rp
bWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3Rp
bWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIGRyaXZlcnMvY2hhci9tdmVidS11YXJ0LmM6MjQ6Ci4vaW5jbHVkZS94ZW4v
c2VyaWFsLmg6IEF0IHRvcCBsZXZlbDoKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxNjY6MTog
d2FybmluZzog4oCYaW5saW5l4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRp
b24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDE2NiB8IHN0YXRpYyB2b2lkIGlubGlu
ZSB4aGNpX2RiY191YXJ0X2luaXQodm9pZCkge30KICAgICAgfCBefn5+fn4KZHJpdmVycy9j
aGFyL212ZWJ1LXVhcnQuYzogSW4gZnVuY3Rpb24g4oCYbXZlYnUzNzAwX3VhcnRfaW50ZXJy
dXB04oCZOgpkcml2ZXJzL2NoYXIvbXZlYnUtdWFydC5jOjcwOjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA3MCB8IHN0
YXRpYyB2b2lkIG12ZWJ1MzcwMF91YXJ0X2ludGVycnVwdChpbnQgaXJxLCB2b2lkICpkYXRh
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+
CiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9saWIvbWVtY3B5Lm8KICBDQyAgICAgIHhzbS9m
bGFzay9ob29rcy5vCiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9saWIvbWVtY21wLm8KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIHhzbS9mbGFzay9ob29rcy5jOjEyOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9j
cHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9j
cHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKICBDQyAgICAgIGNvbW1v
bi9iaXRtYXAubwouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDozNiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vaXJxLmg6NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CiAgQ0MgICAgICBhcmNoL2Fy
bS9hcm02NC9saWIvbWVtbW92ZS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
cHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlv
biDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9T
QVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09N
UEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBkcml2ZXJzL2NoYXIvb21hcC11
YXJ0Lm8KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vYXJtNjQvbGli
L21lbXNldC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vYml0bWFwLmM6MTI6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAg
ICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBe
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8
ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9i
aXRtYXAuYzoxMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9j
cHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9u
cl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NlcmlhbC5oOjEzLAogICAgICAgICAgICAgICAgIGZy
b20gZHJpdmVycy9jaGFyL29tYXAtdWFydC5jOjE0OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3Rh
dGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAg
ICAgICBmcm9tIHhzbS9mbGFzay9ob29rcy5jOjE0OgouL2luY2x1ZGUveGVuL251bWEuaDog
SW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5
OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5f
dCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5o
OjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5z
X2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEw
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1
NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2
b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3Rh
dGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWlu
dDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVD
TEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVO
RCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9zZXJp
YWwuaDogQXQgdG9wIGxldmVsOgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2NjoxOiB3YXJu
aW5nOiDigJhpbmxpbmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBb
LVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5saW5lIHho
Y2lfZGJjX3VhcnRfaW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19l
bXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDox
Mjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1m
bl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVf
cmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1
Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1
ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9p
cnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9jaGFyL29tYXAtdWFydC5j
OjE2OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19s
YXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIz
MCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAg
IF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAq
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL2lycS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwg
dW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUx
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3
IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g
4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxi
Lmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAq
bWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+
fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRy
ZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3Qg
KmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9y
ZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIz
IHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2
ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25n
IGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBz
dGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBk
YXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5
X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CiAgQ0MgICAg
ICBhcmNoL2FybS9hcm02NC9saWIvbWVtY2hyLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIGRy
aXZlcnMvY2hhci9vbWFwLXVhcnQuYzoyMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hl
Y2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9u
ZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzgg
fCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1l
c3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1l
c3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9s
aWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KZHJpdmVycy9jaGFy
L29tYXAtdWFydC5jOiBJbiBmdW5jdGlvbiDigJhvbWFwX3VhcnRfaW50ZXJydXB04oCZOgpk
cml2ZXJzL2NoYXIvb21hcC11YXJ0LmM6NjM6Mzc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDYzIHwgc3RhdGljIHZvaWQg
b21hcF91YXJ0X2ludGVycnVwdChpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+fgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wg
aXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25f
ZGV2aWNl4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi8uL2luY2x1ZGUveGVuL2NvbmZp
Zy5oOjE3LAogICAgICAgICAgICAgICAgIGZyb20gPGNvbW1hbmQtbGluZT46CmRyaXZlcnMv
Y2hhci9vbWFwLXVhcnQuYzogSW4gZnVuY3Rpb24g4oCYb21hcF91YXJ0X3R4X3JlYWR54oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxp
bmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2Rl
diAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+CmRyaXZlcnMvY2hhci9vbWFwLXVhcnQuYzoy
NjE6MTc6IHdhcm5pbmc6IGNvbXBhcmlzb24gaXMgYWx3YXlzIHRydWUgZHVlIHRvIGxpbWl0
ZWQgcmFuZ2Ugb2YgZGF0YSB0eXBlIFstV3R5cGUtbGltaXRzXQogIDI2MSB8ICAgICBBU1NF
UlQoIGNudCA+PSAwICYmIGNudCA8PSB1YXJ0LT5maWZvX3NpemUgKTsKICAgICAgfCAgICAg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBp
biBkZWZpbml0aW9uIG9mIG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1
bmxpa2VseSh4KSAgIF9fYnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmRyaXZlcnMvY2hhci9vbWFw
LXVhcnQuYzoyNjE6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKA
mQogIDI2MSB8ICAgICBBU1NFUlQoIGNudCA+PSAwICYmIGNudCA8PSB1YXJ0LT5maWZvX3Np
emUgKTsKICAgICAgfCAgICAgXn5+fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0
YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+CmRyaXZlcnMvY2hhci9vbWFwLXVhcnQuYzogSW4gZnVuY3Rpb24g4oCYb21hcF91
YXJ0X2lyceKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
cHJvY2Vzc19wZW5kaW5n4oCZOgpkcml2ZXJzL2NoYXIvb21hcC11YXJ0LmM6Mjg4OjQzOiB3
YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKA
mGludOKAmSB0byDigJh1bnNpZ25lZCBpbnTigJkgZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBv
dGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAyODggfCAgICAgcmV0dXJuICgodWFy
dC0+aXJxID4gMCkgPyB1YXJ0LT5pcnEgOiAtMSk7CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAw
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJv
Y2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIHhzbS9mbGFzay9ob29rcy5jOjE3OgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0
ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3Di
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQg
Y21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBk
b21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3Rs
KHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2
MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0
X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2
dGNobl9zZW5k4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwg
c3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20veHNt
Lmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0
YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4p
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWlu
dDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
cmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5p
dF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9z
ZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4
Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3Rf
cmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEy
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWln
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lv
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9w
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9t
YWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFp
bl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0
X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9i
aW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEg
KmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0
X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVt
X21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWlu
dDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVy
bWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5l
X2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1
Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVp
bnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291
cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2Rl
ZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
bWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVz
b3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2Rl
ZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6
NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNt
X2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYw
NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVp
bnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5m
b+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50
IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9t
ZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZG1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3hlbl92ZXJzaW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
ICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L2xpYi9jbGVhcl9wYWdlLm8KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSB4c20vZmxhc2sv
aG9va3MuYzoyMjoKLi9pbmNsdWRlL3hlbi94ZW5vcHJvZi5oOiBJbiBmdW5jdGlvbiDigJhh
Y3F1aXJlX3BtdV9vd25lcnNoaXDigJk6Ci4vaW5jbHVkZS94ZW4veGVub3Byb2YuaDozOTo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwbXVfb3duZXJzaGlw4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDM5IHwgc3RhdGljIGlubGluZSBpbnQgYWNxdWlyZV9wbXVf
b3duZXJzaGlwKGludCBwbXVfb3duZXJzaGlwKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVu
L3hlbm9wcm9mLmg6IEluIGZ1bmN0aW9uIOKAmHJlbGVhc2VfcG11X293bmVyc2hpcOKAmToK
Li9pbmNsdWRlL3hlbi94ZW5vcHJvZi5oOjQ0OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBtdV9vd25lcnNoaXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmVsZWFzZV9wbXVfb3duZXJzaGlwKGludCBwbXVfb3duZXJz
aGlwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fl5+fn5+fn5+fn5+fn4KeHNtL2ZsYXNrL2hvb2tzLmM6IEluIGZ1bmN0aW9uIOKAmGdl
dF9pcnFfc2lk4oCZOgp4c20vZmxhc2svaG9va3MuYzoxMTY6MTQ6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDEx
NiB8ICAgICBpZiAoIGlycSA+PSBucl9pcnFzIHx8IGlycSA8IDAgKQogICAgICB8ICAgICAg
ICAgICAgICBefgp4c20vZmxhc2svaG9va3MuYzogSW4gZnVuY3Rpb24g4oCYZmxhc2tfc2hv
d19zZWN1cml0eV9ldnRjaG7igJk6CnhzbS9mbGFzay9ob29rcy5jOjM4NDoyMDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg0
IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBldnRjaG4gKmNobikKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+Xgp4c20vZmxhc2svaG9va3MuYzogSW4gZnVuY3Rp
b24g4oCYZmxhc2tfbWVtb3J5X3Bpbl9wYWdl4oCZOgp4c20vZmxhc2svaG9va3MuYzo0Nzk6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3OSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIsIHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgp4c20vZmxh
c2svaG9va3MuYzogSW4gZnVuY3Rpb24g4oCYZmxhc2tfbWFwX2RvbWFpbl9tc2nigJk6Cnhz
bS9mbGFzay9ob29rcy5jOjk1MjoyMDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgOTUyIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IGludCBpcnEsIGNvbnN0IHZvaWQgKmRhdGEsIHVpbnQzMl90ICpzaWQsCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+fl4KeHNtL2ZsYXNrL2hvb2tzLmM6OTUyOjI3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDk1MiB8
ICAgICBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCBjb25zdCB2b2lkICpkYXRhLCB1aW50
MzJfdCAqc2lkLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+CnhzbS9m
bGFzay9ob29rcy5jOjk1Mjo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgOTUyIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IGludCBpcnEsIGNvbnN0IHZvaWQgKmRhdGEsIHVpbnQzMl90ICpzaWQsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn4KeHNtL2ZsYXNr
L2hvb2tzLmM6OTUyOjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDk1MiB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
aXJxLCBjb25zdCB2b2lkICpkYXRhLCB1aW50MzJfdCAqc2lkLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+Xn5+
CnhzbS9mbGFzay9ob29rcy5jOjk1MzoyODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDk1MyB8ICAgICBzdHJ1Y3QgYXZjX2F1
ZGl0X2RhdGEgKmFkKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Cnhz
bS9mbGFzay9ob29rcy5jOiBJbiBmdW5jdGlvbiDigJhmbGFza191bm1hcF9kb21haW5fbXNp
4oCZOgp4c20vZmxhc2svaG9va3MuYzoxMDE5OjIwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEwMTkgfCAgICAgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGlycSwgY29uc3Qgdm9pZCAqZGF0YSwgdWludDMyX3QgKnNpZCwKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+Xgp4c20vZmxhc2svaG9va3MuYzoxMDE5OjI3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogMTAxOSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCBjb25zdCB2b2lkICpk
YXRhLCB1aW50MzJfdCAqc2lkLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+
Xn5+CnhzbS9mbGFzay9ob29rcy5jOjEwMTk6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTAxOSB8ICAgICBzdHJ1Y3Qg
ZG9tYWluICpkLCBpbnQgaXJxLCBjb25zdCB2b2lkICpkYXRhLCB1aW50MzJfdCAqc2lkLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efn5+
CnhzbS9mbGFzay9ob29rcy5jOjEwMTk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2lk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMDE5IHwgICAgIHN0cnVjdCBkb21h
aW4gKmQsIGludCBpcnEsIGNvbnN0IHZvaWQgKmRhdGEsIHVpbnQzMl90ICpzaWQsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5efn4KeHNtL2ZsYXNrL2hvb2tzLmM6MTAyMDoyODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTAyMCB8ICAgICBz
dHJ1Y3QgYXZjX2F1ZGl0X2RhdGEgKmFkKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+CnhzbS9mbGFzay9ob29rcy5jOiBJbiBmdW5jdGlvbiDigJhmbGFza191bmJp
bmRfcHRfaXJx4oCZOgp4c20vZmxhc2svaG9va3MuYzoxMDgyOjU0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGJpbmTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEwODIgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJp
bmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+CnhzbS9mbGFzay9ob29rcy5jOiBJbiBmdW5jdGlvbiDigJhmbGFz
a19pcnFfcGVybWlzc2lvbuKAmToKeHNtL2ZsYXNrL2hvb2tzLmM6MTA4ODoyNzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MDg4IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFjY2VzcykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+fn4KICBDQyAgICAgIGFyY2gv
YXJtL2FybTY0L2xpYi9iaXRvcHMubwp4c20vZmxhc2svaG9va3MuYzogSW4gZnVuY3Rpb24g
4oCYZmxhc2tfdm1fZXZlbnRfY29udHJvbOKAmToKeHNtL2ZsYXNrL2hvb2tzLmM6MTMzNjo2
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtb2Rl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAxMzM2IHwgc3RhdGljIGludCBjZl9jaGVjayBmbGFza192bV9ldmVudF9jb250
cm9sKHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5efn5+CnhzbS9mbGFzay9ob29rcy5jOjEzMzY6NzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYb3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEzMzYgfCBzdGF0aWMg
aW50IGNmX2NoZWNrIGZsYXNrX3ZtX2V2ZW50X2NvbnRyb2woc3RydWN0IGRvbWFpbiAqZCwg
aW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efgogIEND
ICAgICAgYXJjaC9hcm0vYXJtNjQvbGliL3N0cmNoci5vCiAgQ0MgICAgICBkcml2ZXJzL2No
YXIvc2NpZi11YXJ0Lm8KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L2xpYi9zdHJjbXAubwpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2
ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5
c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUu
aDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxMywKICAg
ICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9zY2lmLXVhcnQuYzoyMzoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6IEF0IHRvcCBsZXZl
bDoKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxNjY6MTogd2FybmluZzog4oCYaW5saW5l4oCZ
IGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xh
cmF0aW9uXQogIDE2NiB8IHN0YXRpYyB2b2lkIGlubGluZSB4aGNpX2RiY191YXJ0X2luaXQo
dm9pZCkge30KICAgICAgfCBefn5+fn4KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L2xpYi9z
dHJsZW4ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9pcnEuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9jaGFyL3NjaWYtdWFydC5jOjI1OgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAg
ICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAg
ICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2ly
cS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIw
IHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICps
b2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVh
ZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL2NoYXIv
c2NpZi11YXJ0LmM6MjY6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJl
duKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2Vf
aW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xp
c3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODoz
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVz
aF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4K
ZHJpdmVycy9jaGFyL3NjaWYtdWFydC5jOiBJbiBmdW5jdGlvbiDigJhzY2lmX3VhcnRfaW50
ZXJydXB04oCZOgpkcml2ZXJzL2NoYXIvc2NpZi11YXJ0LmM6MTA1OjM3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwNSB8
IHN0YXRpYyB2b2lkIHNjaWZfdWFydF9pbnRlcnJ1cHQoaW50IGlycSwgdm9pZCAqZGF0YSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KICBDQyAg
ICAgIGFyY2gvYXJtL2FybTY0L2xpYi9zdHJuY21wLm8KZHJpdmVycy9jaGFyL3NjaWYtdWFy
dC5jOiBJbiBmdW5jdGlvbiDigJhzY2lmX3VhcnRfaXJx4oCZOgpkcml2ZXJzL2NoYXIvc2Np
Zi11YXJ0LmM6MjUwOjQzOiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMg
c2lnbmVkbmVzcyBmcm9tIOKAmGludOKAmSB0byDigJh1bnNpZ25lZCBpbnTigJkgZHVlIHRv
IHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAyNTAg
fCAgICAgcmV0dXJuICgodWFydC0+aXJxID4gMCkgPyB1YXJ0LT5pcnEgOiAtMSk7CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KICBDQyAg
ICAgIGFyY2gvYXJtL2FybTY0L2xpYi9zdHJubGVuLm8KICBDQyAgICAgIGFyY2gvYXJtL2Fy
bTY0L2xpYi9zdHJyY2hyLm8KICBCSU5GSUxFIGNvbW1vbi9jb25maWdfZGF0YS5TCmd6aXAg
LW4gLWMgLmNvbmZpZyA+Y29tbW9uL2NvbmZpZy5negogIExEICAgICAgYXJjaC9hcm0vYXJt
NjQvbGliL2J1aWx0X2luLm8KICBDQyAgICAgIGNvbW1vbi9jcHUubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gY29tbW9uL2NwdS5jOjE6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIz
MDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7
CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMp
CiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNr
X3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBm
cmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vY3B1Lmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIGNvbW1vbi9jcHUuYzoyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIGNv
bW1vbi9jcHUuYzozOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZu
X3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0
YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CiAgQ0MgICAgICBhcmNoL2Fy
bS9hcm02NC9tbXUvaGVhZC5vCi4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3Rf
ZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9m
aWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91
bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICps
b2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21l
bV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IElu
IGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9t
ZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZv
aWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdl
X2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlw
ZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUg
dm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXAp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBj
aV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBp
bmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25l
ZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2ln
bmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9f
bXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCY
aXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4o
Y29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2dsb2JhbOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBpbnQgdmly
cSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vYXJtNjQvbW11L21tLm8KY29tbW9u
L2NwdS5jOiBJbiBmdW5jdGlvbiDigJhfdGFrZV9jcHVfZG93buKAmToKY29tbW9uL2NwdS5j
OjkyOjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVudXNlZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA5MiB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIF90YWtlX2NwdV9k
b3duKHZvaWQgKnVudXNlZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn5+fn4KY29tbW9uL2NwdS5jOiBJbiBmdW5jdGlvbiDigJhkaXNh
YmxlX25vbmJvb3RfY3B1c+KAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6Mzc1OjIwOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAzNzUgfCAgICAgICAgICAgICAgKGNwdSkgPCBucl9jcHVfaWRzOyAgICAg
ICAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6NDU5OjM2OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9y
X2VhY2hfY3B14oCZCiAgNDU5IHwgI2RlZmluZSBmb3JfZWFjaF9vbmxpbmVfY3B1KGNwdSkg
ICBmb3JfZWFjaF9jcHUoY3B1LCAmY3B1X29ubGluZV9tYXApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn4KY29tbW9uL2NwdS5jOjE5
OTo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfb25saW5lX2Nw
deKAmQogIDE5OSB8ICAgICBmb3JfZWFjaF9vbmxpbmVfY3B1ICggY3B1ICkKICAgICAgfCAg
ICAgXn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vY3B1LmM6IEluIGZ1bmN0aW9uIOKAmGVu
YWJsZV9ub25ib290X2NwdXPigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM3NToyMDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMzc1IHwgICAgICAgICAgICAgIChjcHUpIDwgbnJfY3B1X2lkczsgICAg
ICAgICAgICAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjQ2MDozNjogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZv
cl9lYWNoX2NwdeKAmQogIDQ2MCB8ICNkZWZpbmUgZm9yX2VhY2hfcHJlc2VudF9jcHUoY3B1
KSAgZm9yX2VhY2hfY3B1KGNwdSwgJmNwdV9wcmVzZW50X21hcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+fgpjb21tb24vY3B1LmM6
MjI0OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9wcmVzZW50
X2NwdeKAmQogIDIyNCB8ICAgICBmb3JfZWFjaF9wcmVzZW50X2NwdSAoIGNwdSApCiAgICAg
IHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM3
NToyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgMzc1IHwgICAgICAgICAgICAgIChjcHUpIDwgbnJfY3B1X2lk
czsgICAgICAgICAgICAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICBeCmNvbW1v
bi9jcHUuYzoyMzk6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNo
X2NwdeKAmQogIDIzOSB8ICAgICBmb3JfZWFjaF9jcHUgKCBjcHUsICZmcm96ZW5fY3B1cyAp
CiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vYXJt
NjQvbW11L21tLmM6NDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAg
ICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAg
ICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
YWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFz
a192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9u
IOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10
cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3NldHVwLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2FybTY0L21tdS9t
bS5jOjc6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIw
IHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICps
b2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVh
ZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8
IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVu
dF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMx
IHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0
IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZf
dCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBn
Zm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92
bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGRy
aXZlcnMvY2hhci9pbXgtbHB1YXJ0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20va2VybmVsLmg6MTAsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N0YXRpYy1tZW1vcnkuaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gYXJjaC9hcm0vYXJtNjQvbW11L21tLmM6ODoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGlu
ZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9t
ZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2
MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdt
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9pcnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVy
cy9jaGFyL2lteC1scHVhcnQuYzoyMjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMw
OjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsK
ICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykK
ICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZy
ZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDox
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjM2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9pcnEuaDo1OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwg
c3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklY
X0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194
KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhE
RUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUo
RU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+
fn5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvY2hhci9p
bXgtbHB1YXJ0LmM6MjM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJl
duKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2Vf
aW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xp
c3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODoz
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVz
aF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRyaXZlcnMvY2hhci9pbXgtbHB1YXJ0LmM6MjQ6Ci4v
aW5jbHVkZS94ZW4vc2VyaWFsLmg6IEF0IHRvcCBsZXZlbDoKLi9pbmNsdWRlL3hlbi9zZXJp
YWwuaDoxNjY6MTogd2FybmluZzog4oCYaW5saW5l4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcg
b2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDE2NiB8IHN0YXRp
YyB2b2lkIGlubGluZSB4aGNpX2RiY191YXJ0X2luaXQodm9pZCkge30KICAgICAgfCBefn5+
fn4KZHJpdmVycy9jaGFyL2lteC1scHVhcnQuYzogSW4gZnVuY3Rpb24g4oCYaW14X2xwdWFy
dF9pbnRlcnJ1cHTigJk6CmRyaXZlcnMvY2hhci9pbXgtbHB1YXJ0LmM6Mzk6Mzg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDM5IHwgc3RhdGljIHZvaWQgaW14X2xwdWFydF9pbnRlcnJ1cHQoaW50IGlycSwgdm9pZCAq
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+
CmRyaXZlcnMvY2hhci9pbXgtbHB1YXJ0LmM6IEluIGZ1bmN0aW9uIOKAmGlteF9scHVhcnRf
aXJx4oCZOgpkcml2ZXJzL2NoYXIvaW14LWxwdWFydC5jOjE0MTo0Mzogd2FybmluZzogb3Bl
cmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g
4oCYdWludDMyX3TigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25l
ZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTQxIHwgICAgIHJl
dHVybiAoKHVhcnQtPmlycSA+IDApID8gdWFydC0+aXJxIDogLTEpOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CiAgQ0MgICAgICB4c20v
Zmxhc2svZmxhc2tfb3AubwogIENDICAgICAgY29tbW9uL2RlYnVndHJhY2UubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3Mu
aDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGlt
ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSB4c20vZmxhc2svZmxhc2tfb3AuYzoxMjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9j
cHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRp
YyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2NwdS5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSBjb21tb24vZGVidWd0cmFjZS5jOjk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3Pi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNw
dSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6
MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVf
RklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZ
UEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9p
bmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5j
bHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVp
ZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVj
dG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0
X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hf
ZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMy
X3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
dW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
ZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1f
bWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3Ap
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMg
aW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFn
ZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5
cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NzAsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vZGVidWd0cmFjZS5jOjEzOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAg
ICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAg
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNs
ZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAg
fCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJl
ZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7i
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1
X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEw
Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3Rf
ZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9m
aWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9kZWJ1Z3RyYWNlLmM6MTQ6Ci4vaW5jbHVkZS94
ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1
ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNo
YW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7
YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90
aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/
ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gY29tbW9uL2RlYnVndHJhY2UuYzoxNjoKLi9pbmNsdWRlL3hlbi9zZXJp
YWwuaDogQXQgdG9wIGxldmVsOgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2NjoxOiB3YXJu
aW5nOiDigJhpbmxpbmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBb
LVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5saW5lIHho
Y2lfZGJjX3VhcnRfaW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3Rh
dGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5l
IGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KY29tbW9uL2RlYnVndHJhY2UuYzogSW4gZnVuY3Rp
b24g4oCYZGVidWd0cmFjZV9rZXnigJk6CmNvbW1vbi9kZWJ1Z3RyYWNlLmM6MjM2OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGtleeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDIzNiB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIGRlYnVndHJhY2Vfa2V5KHVuc2lnbmVk
IGNoYXIga2V5KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9kZWJ1Z3RyYWNlLmM6IEluIGZ1bmN0aW9uIOKA
mGRlYnVndHJhY2VfY3B1X2NhbGxiYWNr4oCZOgpjb21tb24vZGVidWd0cmFjZS5jOjI2Nzoy
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZmLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNjcgfCAgICAgc3RydWN0IG5vdGlmaWVyX2Jsb2NrICpuZmIsIHVuc2lnbmVk
IGxvbmcgYWN0aW9uLCB2b2lkICpoY3B1KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgpjb21tb24vZGVidWd0cmFjZS5jOiBBdCB0b3AgbGV2ZWw6CmNvbW1vbi9k
ZWJ1Z3RyYWNlLmM6MzA1OjE6IHdhcm5pbmc6IOKAmHN0YXRpY+KAmSBpcyBub3QgYXQgYmVn
aW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAzMDUg
fCBfX2luaXRjYWxsKGRlYnVndHJhY2VfaW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+fgouL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKA
mToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdh
eXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9ldmVudC5oOjE3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZXZlbnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFy
Y2hfdmlycV9pc19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIHhzbS9mbGFzay9mbGFza19vcC5jOjEzOgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0
ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3Di
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQg
Y21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBk
b21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3Rs
KHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2
MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Nl
bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
OTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
cmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFu
dF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1
ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9p
bml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9w
b2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hh
bmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5
X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVf
aW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVm
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0
ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWlu
X3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWlu
X3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5f
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVy
bWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhf
dCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxv
dykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
Y29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVn
X3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91
bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3Jl
KHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVz
b3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9w
Y2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9y
ZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2Rl
ZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9w
YXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYx
MiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RtX29w4oCZOgogIExEICAgICAgYXJjaC9hcm0vYXJtNjQvbW11L2J1
aWx0X2luLm8KLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNp
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
b3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21h
aW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3
NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20geHNtL2ZsYXNrL2ZsYXNrX29wLmM6MTY6Ci4vaW5jbHVkZS94ZW4vcGFy
YW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1ZGUveGVu
L3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMg
c2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7YWthIOKA
mGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9w
ZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/ICh7IEFT
U0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIENDICAgICAgYXJjaC9h
cm0vYXJtNjQvY2FjaGUubwp4c20vZmxhc2svZmxhc2tfb3AuYzogSW4gZnVuY3Rpb24g4oCY
Zmxhc2tfc2VjdXJpdHlfcmVzb2x2ZV9ib29s4oCZOgp4c20vZmxhc2svZmxhc2tfb3AuYzoy
NDc6MjM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdWludDMyX3TigJkge2FrYSDigJh1bnNpZ25lZCBp
bnTigJl9IGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0NyB8ICAgICBpZiAo
IGFyZy0+Ym9vbF9pZCAhPSAtMSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIF5+
CnhzbS9mbGFzay9mbGFza19vcC5jOiBJbiBmdW5jdGlvbiDigJhmbGFza19zZWN1cml0eV9z
ZXRfYm9vbOKAmToKeHNtL2ZsYXNrL2ZsYXNrX29wLmM6Mjg1OjI3OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmHVpbnQzMl904oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBhbmQg4oCYaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyODUgfCAgICAgICAgIGlmICggYXJnLT5ib29sX2lkID49
IG51bSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICBefgp4c20vZmxhc2sv
Zmxhc2tfb3AuYzozMDU6MzQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdWludDMyX3TigJkge2FrYSDi
gJh1bnNpZ25lZCBpbnTigJl9IGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDMw
NSB8ICAgICAgICAgaWYgKCAhcnYgJiYgYXJnLT5ib29sX2lkID49IGJvb2xfbnVtICkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgogIENDICAgICAgYXJj
aC9hcm0vYXJtNjQvY3B1ZmVhdHVyZS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAg
IGZyb20gYXJjaC9hcm0vYXJtNjQvY3B1ZmVhdHVyZS5jOjc1OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAx
IHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL2FybTY0L2NwdWZlYXR1cmUu
Yzo3NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9jcHVmZWF0dXJlLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdWlkX2ZlYXR1cmVfZXh0cmFjdF9maWVsZF93aWR0aOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9jcHVmZWF0dXJlLmg6ODA6MTc6IHdhcm5pbmc6IG9w
ZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYaW504oCZIHRv
IOKAmHVuc2lnbmVkIGludOKAmSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJh
bmQgWy1Xc2lnbi1jb21wYXJlXQogICA4MCB8ICAgICAgICAgICAgICAgICBjcHVpZF9mZWF0
dXJlX2V4dHJhY3Rfc2lnbmVkX2ZpZWxkX3dpZHRoKGZlYXR1cmVzLCBmaWVsZCwgd2lkdGgp
IDoKICAgICAgfCAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIENDICAgICAgZHJpdmVy
cy9jaGFyL2FybS11YXJ0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9hc20tZ2VuZXJpYy9kZXZpY2UuaDo0MiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL2RldmljZS5oOjEs
CiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL2NoYXIvYXJtLXVhcnQuYzoyMDoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9u
cl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAx
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNp
Z25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5o
OjY6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
cHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo3OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKA
mF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRyaXZlcnMvY2hh
ci9hcm0tdWFydC5jOjI0OgouL2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKA
mG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2Fy
bmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhs
b25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKA
mX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFy
ZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7
IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRyaXZlcnMvY2hhci9h
cm0tdWFydC5jOjI1OgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOiBBdCB0b3AgbGV2ZWw6Ci4v
aW5jbHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5pbmc6IOKAmGlubGluZeKAmSBpcyBu
b3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlv
bl0KICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhjaV9kYmNfdWFydF9pbml0KHZvaWQp
IHt9CiAgICAgIHwgXn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zZXR1cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FjcGkuaDoyNSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3Bs
YXRmb3JtL2FjbGludXguaDo1OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9h
Y3BpL3BsYXRmb3JtL2FjZW52Lmg6MTQyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL2FjcGkvYWNwaS5oOjU2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9hY3BpLmg6NTcsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL2NoYXIvYXJtLXVh
cnQuYzoyNzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAo
dW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwy
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRl
cuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBh
Z2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2Fj
Y2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJv
bV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEg
fCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3Qg
KnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1
bmN0aW9uIOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3Bp
Lmg6MTkyOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3Nl
dF9jc3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFj
cGlfc2V0X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTk0OjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Vic3Rh
dGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+fn5+fn4KICBDQyAgICAgIGNvbW1vbi9kZXZpY2VfdHJlZS5vCiAgQ0MgICAgICBk
cml2ZXJzL2NoYXIvc2VyaWFsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1v
bi9kZXZpY2VfdHJlZS5jOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBp
bnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAg
ICBmcm9tIGRyaXZlcnMvY2hhci9zZXJpYWwuYzoxMToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0
YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJf
Y3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJf
Y3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9j
X2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGlu
ZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUv
eGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hl
bi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZu
X3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWlu
Lmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVu
c2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29u
c3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFn
ZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEx
IHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ss
IHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdl
X3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdl
X3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUv
eGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nl
c3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNj
ZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
YWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
YWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZf
dCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJt
X2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21f
Z2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwg
ICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0
LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwg
c3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Bl
cmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAg
ICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAg
IF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3lj
bGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQw
IHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192
YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3Rh
dGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZy
ZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vYXJtNjQvYnBpLm8KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVu
c2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9i
bG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9t
ZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGlu
dCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNz
X3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3Rh
dGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3
IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwo
c3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKA
mGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWlu
KGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwg
c3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVp
bnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Zmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3Rp
bWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3Rp
bWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIGRyaXZlcnMvY2hhci9zZXJpYWwuYzoxMjoKLi9pbmNsdWRlL3hlbi9wYXJh
bS5oOiBJbiBmdW5jdGlvbiDigJhub19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4v
cGFyYW0uaDoxOTM6MTk6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBz
aWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCY
bG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3Bl
cmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8gKHsgQVNT
RVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAg
ICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSBkcml2ZXJzL2NoYXIvc2VyaWFsLmM6MTM6Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6
IEF0IHRvcCBsZXZlbDoKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxNjY6MTogd2FybmluZzog
4oCYaW5saW5l4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xk
LXN0eWxlLWRlY2xhcmF0aW9uXQogIDE2NiB8IHN0YXRpYyB2b2lkIGlubGluZSB4aGNpX2Ri
Y191YXJ0X2luaXQodm9pZCkge30KICAgICAgfCBefn5+fn4KZHJpdmVycy9jaGFyL3Nlcmlh
bC5jOiBJbiBmdW5jdGlvbiDigJhzZXJpYWxfaW5pdF9wcmVpcnHigJk6CmRyaXZlcnMvY2hh
ci9zZXJpYWwuYzo0NDU6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhsb25n
IHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNDQ1IHwgICAgIGZvciAoIGkg
PSAwOyBpIDwgQVJSQVlfU0laRShjb20pOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICBeCmRyaXZlcnMvY2hhci9zZXJpYWwuYzogSW4gZnVuY3Rpb24g4oCYc2VyaWFsX2lu
aXRfcG9zdGlyceKAmToKZHJpdmVycy9jaGFyL3NlcmlhbC5jOjQ2MjoyMDogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICA0NjIgfCAgICAgZm9yICggaSA9IDA7IGkgPCBBUlJBWV9TSVpFKGNvbSk7IGkr
KyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL2RldmljZV90cmVlLmM6
IEluIGZ1bmN0aW9uIOKAmGR0X2J1c19kZWZhdWx0X2dldF9mbGFnc+KAmToKY29tbW9uL2Rl
dmljZV90cmVlLmM6NjIwOjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGFkZHLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjAgfCBzdGF0aWMgdW5zaWduZWQgaW50IGR0
X2J1c19kZWZhdWx0X2dldF9mbGFncyhjb25zdCBfX2JlMzIgKmFkZHIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn5+CmRyaXZlcnMvY2hhci9zZXJpYWwuYzogSW4gZnVuY3Rpb24g4oCYc2VyaWFsX2Vu
ZGJvb3TigJk6CmNvbW1vbi9kZXZpY2VfdHJlZS5jOiBJbiBmdW5jdGlvbiDigJhkdF9idXNf
cGNpX2NvdW50X2NlbGxz4oCZOgpjb21tb24vZGV2aWNlX3RyZWUuYzo2NTM6NjU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2
NTMgfCBzdGF0aWMgdm9pZCBkdF9idXNfcGNpX2NvdW50X2NlbGxzKGNvbnN0IHN0cnVjdCBk
dF9kZXZpY2Vfbm9kZSAqbnAsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+CmNvbW1vbi9kZXZpY2Vf
dHJlZS5jOiBJbiBmdW5jdGlvbiDigJhkdF9tYXRjaF9idXPigJk6CmNvbW1vbi9kZXZpY2Vf
dHJlZS5jOjc0NDoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA3NDQgfCAgICAgZm9yICggaSA9IDA7
IGkgPCBBUlJBWV9TSVpFKGR0X2J1c3Nlcyk7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgIF4KZHJpdmVycy9jaGFyL3NlcmlhbC5jOjQ3NToyMDogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICA0NzUgfCAgICAgZm9yICggaSA9IDA7IGkgPCBBUlJBWV9TSVpFKGNvbSk7IGkrKyApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL2RldmljZV90cmVlLmM6IEluIGZ1
bmN0aW9uIOKAmGR0X2dldF9hZGRyZXNz4oCZOgpkcml2ZXJzL2NoYXIvc2VyaWFsLmM6IElu
IGZ1bmN0aW9uIOKAmHNlcmlhbF9pcnHigJk6CmNvbW1vbi9kZXZpY2VfdHJlZS5jOjc4MToy
NDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IGFu
ZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDc4MSB8ICAgICBmb3IgKCBpID0gMDsg
cHNpemUgPj0gb25lc2l6ZTsgcHNpemUgLT0gb25lc2l6ZSwgcHJvcCArPSBvbmVzaXplLCBp
KysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXn4KZHJpdmVycy9jaGFyL3Nl
cmlhbC5jOjQ4MjoyOTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA0ODIgfCAgICAgaWYgKCAoaWR4ID49
IDApICYmIChpZHggPCBBUlJBWV9TSVpFKGNvbSkpICYmCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIF4KY29tbW9uL2RldmljZV90cmVlLmM6NzgzOjE2OiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICA3ODMgfCAgICAgICAgIGlmICggaSA9PSBpbmRleCApCiAgICAgIHwgICAgICAgICAg
ICAgICAgXn4KZHJpdmVycy9jaGFyL3NlcmlhbC5jOiBJbiBmdW5jdGlvbiDigJhzZXJpYWxf
dnVhcnRfaW5mb+KAmToKY29tbW9uL2RldmljZV90cmVlLmM6IEluIGZ1bmN0aW9uIOKAmGR0
X3RyYW5zbGF0ZV9vbmXigJk6CmRyaXZlcnMvY2hhci9zZXJpYWwuYzo0OTE6Mjk6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgNDkxIHwgICAgIGlmICggKGlkeCA+PSAwKSAmJiAoaWR4IDwgQVJSQVlf
U0laRShjb20pKSAmJgogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNv
bW1vbi9kZXZpY2VfdHJlZS5jOjgyNToxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBp
bnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgODI1IHwgICAgIGZvciAo
IDsgcmxlbiA+PSByb25lOyBybGVuIC09IHJvbmUsIHJhbmdlcyArPSByb25lICkKICAgICAg
fCAgICAgICAgICAgICAgICAgIF5+CmRyaXZlcnMvY2hhci9zZXJpYWwuYzogSW4gZnVuY3Rp
b24g4oCYc2VyaWFsX3N1c3BlbmTigJk6CmRyaXZlcnMvY2hhci9zZXJpYWwuYzo1MDE6MjA6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgNTAxIHwgICAgIGZvciAoIGkgPSAwOyBpIDwgQVJSQVlfU0la
RShjb20pOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBeCmRyaXZlcnMvY2hh
ci9zZXJpYWwuYzogSW4gZnVuY3Rpb24g4oCYc2VyaWFsX3Jlc3VtZeKAmToKZHJpdmVycy9j
aGFyL3NlcmlhbC5jOjUwOToyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxv
bmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA1MDkgfCAgICAgZm9yICgg
aSA9IDA7IGkgPCBBUlJBWV9TSVpFKGNvbSk7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgIF4KY29tbW9uL2RldmljZV90cmVlLmM6IEluIGZ1bmN0aW9uIOKAmGR0X2Zvcl9l
YWNoX3Jhbmdl4oCZOgpjb21tb24vZGV2aWNlX3RyZWUuYzoxMDYwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0K
IDEwNjAgfCAgICAgZm9yICggOyBybGVuID49IHJvbmU7IHJsZW4gLT0gcm9uZSwgcmFuZ2Vz
ICs9IHJvbmUgKQogICAgICB8ICAgICAgICAgICAgICAgICAgXn4KY29tbW9uL2RldmljZV90
cmVlLmM6IEluIGZ1bmN0aW9uIOKAmGR0X2Zvcl9lYWNoX2lycV9tYXDigJk6CmNvbW1vbi9k
ZXZpY2VfdHJlZS5jOjEzMjY6MjQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KIDEzMjYg
fCAgICAgICAgIGZvciAoIGkgPSAwOyBpIDwgcGludHNpemU7IGkrKyApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICBeCiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9kb21jdGwu
bwpjb21tb24vZGV2aWNlX3RyZWUuYzogSW4gZnVuY3Rpb24g4oCYZHRfaXJxX21hcF9yYXfi
gJk6CmNvbW1vbi9kZXZpY2VfdHJlZS5jOjE0NTk6Mjg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29t
cGFyZV0KIDE0NTkgfCAgICAgICAgICAgICBmb3IgKCBpID0gMDsgaSA8IGludHNpemU7IGkr
KyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vZGV2aWNl
X3RyZWUuYzoxNDk2OjI4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZ
IHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAxNDk2IHwgICAg
ICAgICAgICAgZm9yICggaSA9IDA7IGkgPCBhZGRyc2l6ZSAmJiBtYXRjaDsgKytpICkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9kZXZpY2VfdHJlZS5j
OjE1MDE6MjM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDi
gJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KIDE1MDEgfCAgICAgICAgICAg
ICBmb3IgKCA7IGkgPCAoYWRkcnNpemUgKyBpbnRzaXplKSAmJiBtYXRjaDsgKytpICkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vZGV2aWNlX3RyZWUuYzogSW4g
ZnVuY3Rpb24g4oCYX19kdF9wYXJzZV9waGFuZGxlX3dpdGhfYXJnc+KAmToKY29tbW9uL2Rl
dmljZV90cmVlLmM6MTc4NTozMjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUz
MuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTc4NSB8
ICAgICAgICAgICAgICAgICBmb3IgKCBpID0gMDsgaSA8IGNvdW50OyBpKysgKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL2FybTY0L2RvbWN0bC5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlu
bGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9F
TkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAg
IHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgs
IHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVf
aWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVf
aWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZv
aWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CiAgQ0Mg
ICAgICB4c20vZmxhc2svc3MvZWJpdG1hcC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJht
Zm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwg
c3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9y
YWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hl
Y2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWdu
ZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNy
ZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6
NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIHhzbS9mbGFzay9zcy9lYml0bWFwLmM6MTY6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1
X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEw
Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2
aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZw
Y2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90
IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19t
dXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3go
dm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0
X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0
YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9t
IGFyY2gvYXJtL2FybTY0L2RvbWN0bC5jOjExOgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
MjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0ZG9tYWluaW5mb+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjM1OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50
IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9z
eXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFy
Z2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDYgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjUxOjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0
X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBk
ZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjczOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
ZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJl
ZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJl
X2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2Rf
dGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6
NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21l
bW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2
YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19w
aHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFw
X2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9t
YWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9w
YWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
Y21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9m
aWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
eGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9w
dF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxf
YmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWlu
dDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Bj
aV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50
MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3Np
Z25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAq
ZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVm
YXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3Vy
Y2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTgg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0
X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2
Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVn
X2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRl
ZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1
cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYs
IGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVw
X21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vf
b2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
aHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1o
dm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIz
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3Pi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0
Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50
MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0
aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVy
c2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rv
bWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CnhzbS9mbGFz
ay9zcy9lYml0bWFwLmM6IEluIGZ1bmN0aW9uIOKAmGViaXRtYXBfY29udGFpbnPigJk6Cnhz
bS9mbGFzay9zcy9lYml0bWFwLmM6OTM6MjQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgIDkzIHwgICAg
ICAgICBmb3IgKCBpID0gMDsgaSA8IEVCSVRNQVBfVU5JVF9OVU1TOyBpKysgKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS9hcm02NC9kb21jdGwuYzogSW4g
ZnVuY3Rpb24g4oCYc3ViYXJjaF9kb19kb21jdGzigJk6CmFyY2gvYXJtL2FybTY0L2RvbWN0
bC5jOjU1OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVfZG9tY3Rs4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU1IHwgICAgICAgICAgICAgICAgICAgICAgICBYRU5f
R1VFU1RfSEFORExFX1BBUkFNKHhlbl9kb21jdGxfdCkgdV9kb21jdGwpCnhzbS9mbGFzay9z
cy9lYml0bWFwLmM6IEluIGZ1bmN0aW9uIOKAmGViaXRtYXBfcmVhZOKAmToKeHNtL2ZsYXNr
L3NzL2ViaXRtYXAuYzoyNjE6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICAyNjEg
fCAgICAgZm9yICggaSA9IDA7IGkgPCBjb3VudDsgaSsrICkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgXgogIENDICAgICAgYXJjaC9hcm0vYXJtNjQvZG9tYWluLm8KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAg
ICAgIGZyb20gYXJjaC9hcm0vYXJtNjQvZG9tYWluLmM6MToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8
IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNM
QVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5E
LCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nt
cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1
IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQg
PT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29s
IGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGlj
IGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1
ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3Qg
bWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9t
YWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5y
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFw
KHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAq
cGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVs
MihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQx
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0
ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
YWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBw
YWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9h
Y2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1f
YWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IElu
IGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFy
Y2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDoz
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50
MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQg
cDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zy
b21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYx
IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90
ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3
IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5
X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShz
dHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVz
dF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNf
cHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29u
c3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICB4
c20vZmxhc2svc3MvaGFzaHRhYi5vCiAgQ0MgICAgICB4c20vZmxhc2svc3Mvc3ltdGFiLm8K
eHNtL2ZsYXNrL3NzL3N5bXRhYi5jOiBJbiBmdW5jdGlvbiDigJhzeW1jbXDigJk6CnhzbS9m
bGFzay9zcy9zeW10YWIuYzozMDoyMTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMwIHwgICAgIHN0cnVjdCBoYXNodGFiICpo
LCBjb25zdCB2b2lkICprZXkxLCBjb25zdCB2b2lkICprZXkyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fn5+XgogIExEICAgICAgZHJpdmVycy9jaGFyL2J1aWx0X2luLm8KICBDQyAg
ICAgIHhzbS9mbGFzay9zcy9zaWR0YWIubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20geHNt
L2ZsYXNrL3NzL3NpZHRhYi5jOjEyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KICBDQyAg
ICAgIGFyY2gvYXJtL2FybTY0L2VudHJ5Lm8KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L2hl
YWQubwogIENDICAgICAgZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaW9tbXUubwogIENDICAg
ICAgYXJjaC9hcm0vYXJtNjQvaW5zbi5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jdXJyZW50Lmg6OCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9hc20tZ2VuZXJpYy9wZXJjcHUuaDo4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vcGVyY3B1Lmg6
MSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyY3B1Lmg6MzAsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RldmljZV90cmVlLmg6MjIsCiAgICAgICAg
ICAgICAgICAgZnJvbSBkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21tdS5jOjE4OgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25y
X3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2ln
bmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6
NjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFz
dOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAg
fCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAg
ICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
Y3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBe
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFz
a192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwg
c3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFz
a192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3
bG9jay5oOjc6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCY
X19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9pcnEuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wY2kuaDox
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaW9tbXUuaDoyNSwKICAg
ICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lvbW11LmM6MTk6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vcGNpLmg6MTQ6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5f
dG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3Rh
dGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CmRyaXZlcnMvcGFzc3Ro
cm91Z2gvYXJtL2lvbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFyY2hfaW9tbXVfY2hlY2tfYXV0
b3RyYW5zbGF0ZWRfaHdkb23igJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lvbW11LmM6
MTIwOjcyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMjAgfCB2b2lkIF9faHdkb21faW5pdCBhcmNoX2lvbW11X2NoZWNrX2F1
dG90cmFuc2xhdGVkX2h3ZG9tKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lvbW11LmM6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfaW9tbXVfZG9tYWluX2Rlc3Ryb3nigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gv
YXJtL2lvbW11LmM6MTMxOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzEgfCB2b2lkIGFyY2hfaW9tbXVfZG9tYWluX2Rl
c3Ryb3koc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21t
dS5jOiBJbiBmdW5jdGlvbiDigJhhcmNoX2lvbW11X2h3ZG9tX2luaXTigJk6CmRyaXZlcnMv
cGFzc3Rocm91Z2gvYXJtL2lvbW11LmM6MTM1OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzUgfCB2b2lkIF9faHdkb21f
aW5pdCBhcmNoX2lvbW11X2h3ZG9tX2luaXQoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21tdS5jOiBJbiBmdW5jdGlvbiDigJhhcmNo
X2lvbW11X3VzZV9wZXJtaXR0ZWTigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lvbW11
LmM6MTUwOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxNTAgfCBib29sIGFyY2hfaW9tbXVfdXNlX3Blcm1pdHRlZChjb25z
dCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L2xp
dmVwYXRjaC5vCiAgQ0MgICAgICB4c20vZmxhc2svc3MvYXZ0YWIubwogIENDICAgICAgZHJp
dmVycy9wYXNzdGhyb3VnaC9hcm0vaW9tbXVfaGVscGVycy5vCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSB4c20vZmxhc2svc3Mvc2lkdGFiLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJv
bSB4c20vZmxhc2svc3MvcG9saWN5ZGIuaDozMSwKICAgICAgICAgICAgICAgICBmcm9tIHhz
bS9mbGFzay9zcy9hdnRhYi5jOjI5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9saXZlcGF0Y2guaDoxNywKICAgICAgICAgICAgICAg
ICBmcm9tIGFyY2gvYXJtL2FybTY0L2xpdmVwYXRjaC5jOjk6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKA
mToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3Rh
dGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWlu
dDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUg
aW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUp
IFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VO
ROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FW
RV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0
IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxp
bmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNp
emUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgp4c20v
Zmxhc2svc3MvYXZ0YWIuYzogSW4gZnVuY3Rpb24g4oCYYXZ0YWJfaW5zZXJ0X25vZGXigJk6
CnhzbS9mbGFzay9zcy9hdnRhYi5jOjM4OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGN1cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzOCB8ICAgICBzdHJ1Y3QgYXZ0
YWJfbm9kZSAqIHByZXYsIHN0cnVjdCBhdnRhYl9ub2RlICogY3VyLCBzdHJ1Y3QgYXZ0YWJf
a2V5ICprZXksCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2FybTY0L2xpdmVw
YXRjaC5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVu
Y3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25y
X3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KeHNtL2ZsYXNrL3NzL2F2dGFiLmM6
IEluIGZ1bmN0aW9uIOKAmGF2dGFiX2Rlc3Ryb3nigJk6CnhzbS9mbGFzay9zcy9hdnRhYi5j
OjI0MDoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKA
mHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBmb3IgKCBp
ID0gMDsgaSA8IGgtPm5zbG90OyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBe
CnhzbS9mbGFzay9zcy9hdnRhYi5jOiBJbiBmdW5jdGlvbiDigJhhdnRhYl9hbGxvY+KAmToK
eHNtL2ZsYXNrL3NzL2F2dGFiLmM6MjkwOjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVd
CiAgMjkwIHwgICAgIGZvciAoIGkgPSAwOyBpIDwgbnNsb3Q7IGkrKyApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgIF4KeHNtL2ZsYXNrL3NzL2F2dGFiLmM6IEluIGZ1bmN0aW9uIOKA
mGF2dGFiX2hhc2hfZXZhbOKAmToKeHNtL2ZsYXNrL3NzL2F2dGFiLmM6MzA5OjIwOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW50
4oCZfSBbLVdzaWduLWNvbXBhcmVdCiAgMzA5IHwgICAgIGZvciAoIGkgPSAwOyBpIDwgaC0+
bnNsb3Q7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KeHNtL2ZsYXNrL3Nz
L2F2dGFiLmM6IEluIGZ1bmN0aW9uIOKAmGF2dGFiX3JlYWRfaXRlbeKAmToKeHNtL2ZsYXNr
L3NzL2F2dGFiLmM6NDE2OjI0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYbG9u
ZyB1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDQxNiB8ICAgICAgICAgZm9y
ICggaSA9IDA7IGkgPCBzaXplb2Yoc3BlY19vcmRlcikvc2l6ZW9mKHUxNik7IGkrKyApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICBeCnhzbS9mbGFzay9zcy9hdnRhYi5jOjQ1
ODoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICA0NTggfCAgICAgZm9yICggaSA9IDA7IGkgPCBBUlJB
WV9TSVpFKHNwZWNfb3JkZXIpOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBe
CnhzbS9mbGFzay9zcy9hdnRhYi5jOiBJbiBmdW5jdGlvbiDigJhhdnRhYl9pbnNlcnRm4oCZ
Ogp4c20vZmxhc2svc3MvYXZ0YWIuYzo0ODY6NzI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4NiB8ICAgICBzdHJ1Y3QgYXZ0
YWIgKmEsIHN0cnVjdCBhdnRhYl9rZXkgKmssIHN0cnVjdCBhdnRhYl9kYXR1bSAqZCwgdm9p
ZCAqcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcGNpLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL2lvbW11Lmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL3Bh
c3N0aHJvdWdoL2FybS9pb21tdV9oZWxwZXJzLmM6MjY6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2Nw
dV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2Nw
dV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19j
cHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFw
KHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2lycS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BjaS5oOjEzOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVu
Y3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBw
YWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwg
ICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgphcmNoL2FybS9h
cm02NC9saXZlcGF0Y2guYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9saXZlcGF0Y2hfc3ltYm9s
X2RlbnnigJk6CmFyY2gvYXJtL2FybTY0L2xpdmVwYXRjaC5jOjgzOjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGVsZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA4MyB8
IGJvb2wgYXJjaF9saXZlcGF0Y2hfc3ltYm9sX2RlbnkoY29uc3Qgc3RydWN0IGxpdmVwYXRj
aF9lbGYgKmVsZiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vYXJtNjQvbGl2ZXBhdGNo
LmM6ODQ6NjU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc3lt4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDg0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgbGl2ZXBhdGNoX2VsZl9zeW0gKnN5bSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+CmFyY2gvYXJtL2FybTY0L2xpdmVwYXRjaC5jOiBJbiBmdW5jdGlvbiDigJhhcmNoX2xp
dmVwYXRjaF9wZXJmb3JtX3JlbOKAmToKYXJjaC9hcm0vYXJtNjQvbGl2ZXBhdGNoLmM6MjM0
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGVsZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDIzNCB8IGludCBhcmNoX2xpdmVwYXRjaF9wZXJmb3JtX3JlbChzdHJ1Y3Qg
bGl2ZXBhdGNoX2VsZiAqZWxmLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL2FybTY0L2xpdmVwYXRj
aC5jOjIzNTo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhiYXNl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBsaXZlcGF0Y2hfZWxmX3NlYyAqYmFzZSwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+CmFyY2gvYXJtL2FybTY0L2xpdmVwYXRjaC5jOjIzNjo2NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWxh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM2IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBsaXZlcGF0Y2hf
ZWxmX3NlYyAqcmVsYSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3BjaS5oOjE0OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4g
ZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBt
Zm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAg
ICAgZnJvbSBkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21tdV9oZWxwZXJzLmM6Mjg6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hl
Y2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWdu
ZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1
X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAq
dikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9u
IOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0
ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0
X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUg
aW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25f
ZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0
YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nl
c3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0
cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29s
IGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fl4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaW9tbXVfaGVscGVycy5jOiBJ
biBmdW5jdGlvbiDigJhhcm1faW9tbXVfbWFwX3BhZ2XigJk6CmRyaXZlcnMvcGFzc3Rocm91
Z2gvYXJtL2lvbW11X2hlbHBlcnMuYzozNTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhmbHVzaF9mbGFnc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCAqZmx1c2hfZmxh
Z3MpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+fn5+fn5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21tdV9oZWxw
ZXJzLmM6IEluIGZ1bmN0aW9uIOKAmGFybV9pb21tdV91bm1hcF9wYWdl4oCZOgpkcml2ZXJz
L3Bhc3N0aHJvdWdoL2FybS9pb21tdV9oZWxwZXJzLmM6Njc6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZmx1c2hfZmxhZ3PigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NjcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCAqZmx1c2hfZmxhZ3MpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn5+fn5+fn5+CiAgQ0MgICAgICBkcml2ZXJzL3Bh
c3N0aHJvdWdoL2FybS9pb21tdV9md3NwZWMubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9wY2kuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vaW9tbXUuaDoyNSwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvcGFzc3Ro
cm91Z2gvYXJtL2lvbW11X2Z3c3BlYy5jOjI2OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGNpLmg6MTM6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5s
b2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgogIENDICAgICAgY29tbW9uL2RtLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcGNpLmg6MTQ6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBm
dW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1m
bikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3Vlc3RfYWNjZXNz
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZ3Vlc3RfYWNjZXNz
Lmg6MTAsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vZG0uYzoxODoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3go
dm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nt
cC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAg
ICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAg
IGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlu
bGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1
IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21h
aW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUg
bm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgogIENDICAgICAgZHJpdmVycy9wYXNzdGhy
b3VnaC9hcm0vc21tdS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4u
aDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9h
bHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9n
Zm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQs
IHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFy
Y2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzo0
NDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFz
dOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAg
fCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAg
ICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
Y3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBe
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFz
a192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwg
c3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFz
a192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0
aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
MzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRf
ZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2
ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5
c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUu
aDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjM2LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9pcnEuaDo1OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxp
bmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGlu
dCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJj
YWxsLmg6MjIsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vZG0uYzoxOToKLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dldGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDIyOSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1
bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRl
ZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBz
dHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9t
Y3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X3N5c2N0bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDoyNjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25z
b2xlKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2
dGNobl9zZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4g
KmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2hu
X3Jlc2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozMDM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZ3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZ3JhbnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMyMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
cmFudF9xdWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21faW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6
NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9zZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9leGNoYW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbW9yeV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5Mzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlz
bWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9j
b25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBz
dGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4
OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVk
b3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX21hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFp
bl9pcnHigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2
IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZv
aWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NB
VkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+
fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQg
KmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Vu
bWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5t
YXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcx
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2
b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4
MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ4OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBp
cnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVp
bnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxs
b3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0
LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWdu
X2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0
cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVz
b3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRm
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNl
X3BsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3Vy
Y2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1Njkg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3
NDo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBf
Z3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29m
ZmxpbmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQz
Ml90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9oeXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2
bSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9v
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJf
dCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51
bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4
c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3
Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQz
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9t
YWluX3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2so
cmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVy
cy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjQ3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFp
bi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEs
CiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6
NTA6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZ
OgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGlu
ZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIz
IHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2
ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25n
IGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIENDICAgICAg
Y29tbW9uL2RvbWFpbi5vCiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9zbWMubwouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQg
dnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Y29tbW9uL2RvbWFpbi5jOjEyOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9u
IOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTog
d2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDi
gJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGlu
dOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29t
cGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAt
IHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlv
biDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMx
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2Rv
bWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KZHJp
dmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOiBJbiBmdW5jdGlvbiDigJhwY2lfZm9yX2Vh
Y2hfZG1hX2FsaWFz4oCZOgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6MTkxOjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxOTEgfCBzdGF0aWMgaW5saW5lIGludCBwY2lfZm9yX2VhY2hfZG1hX2FsaWFz
KHN0cnVjdCBwY2lfZGV2ICpwZGV2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KZHJpdmVycy9wYXNzdGhy
b3VnaC9hcm0vc21tdS5jOjE5Mjo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhm
buKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgaW50ICgqZm4pIChzdHJ1Y3QgcGNpX2RldiAqcGRldiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgMTkzIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHUxNiBhbGlhcywgdm9pZCAqZGF0
YSksCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KZHJpdmVycy9wYXNzdGhyb3VnaC9h
cm0vc21tdS5jOjE5NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk0IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB2b2lkICpkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdo
L2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFybV9zbW11X2R0X2FkZF9kZXZpY2VfbGVn
YWN54oCZOgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6ODU0OjIzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICA4NTQgfCAgICAgICAgIGZvciAoaSA9IDA7IGkgPCBmd3NwZWMtPm51bV9pZHM7ICsr
aSkgewogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICBeCmRyaXZlcnMvcGFzc3Rocm91
Z2gvYXJtL3NtbXUuYzogSW4gZnVuY3Rpb24g4oCYcmVnaXN0ZXJfc21tdV9tYXN0ZXLigJk6
CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzo4OTQ6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgODk0IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZGV2aWNlICpkZXYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn4K
ZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOiBJbiBmdW5jdGlvbiDigJhhcm1fc21t
dV9kdF9yZW1vdmVfZGV2aWNlX2dlbmVyaWPigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJt
L3NtbXUuYzo5MjA6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGV2Zm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA5MjAgfCBzdGF0aWMgaW50IGFybV9zbW11X2R0X3Jl
bW92ZV9kZXZpY2VfZ2VuZXJpYyh1OCBkZXZmbiwgc3RydWN0IGRldmljZSAqZGV2KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fl5+
fn5+CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzogSW4gZnVuY3Rpb24g4oCYYXJt
X3NtbXVfZHRfYWRkX2RldmljZV9nZW5lcmlj4oCZOgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2Fy
bS9zbW11LmM6OTM2OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRldmZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgOTM2IHwgc3RhdGljIGludCBhcm1fc21tdV9kdF9h
ZGRfZGV2aWNlX2dlbmVyaWModTggZGV2Zm4sIHN0cnVjdCBkZXZpY2UgKmRldikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5efn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2
ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5
c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUu
aDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAg
ICAgICAgICAgICAgZnJvbSBjb21tb24vZG9tYWluLmM6MTM6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFy
bV9zbW11X2NvbnRleHRfZmF1bHTigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUu
YzoxMDQyOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogMTA0MiB8IHN0YXRpYyBpcnFyZXR1cm5fdCBhcm1fc21tdV9jb250
ZXh0X2ZhdWx0KGludCBpcnEsIHZvaWQgKmRldikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+CmRyaXZlcnMvcGFzc3Rocm91Z2gv
YXJtL3NtbXUuYzogSW4gZnVuY3Rpb24g4oCYYXJtX3NtbXVfZ2xvYmFsX2ZhdWx04oCZOgpk
cml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6MTA3NTo0Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEwNzUgfCBzdGF0
aWMgaXJxcmV0dXJuX3QgYXJtX3NtbXVfZ2xvYmFsX2ZhdWx0KGludCBpcnEsIHZvaWQgKmRl
dikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hlbi9jb25maWcuaDox
NywKICAgICAgICAgICAgICAgICBmcm9tIDxjb21tYW5kLWxpbmU+Ogpkcml2ZXJzL3Bhc3N0
aHJvdWdoL2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFybV9zbW11X2luaXRfZG9tYWlu
X2NvbnRleHTigJk6Ci4vaW5jbHVkZS94ZW4vZXJyLmg6MTc6Mzg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgIDE3IHwgI2RlZmluZSBJU19FUlJfVkFMVUUoeCkgdW5saWtlbHkoKHgpID49ICh1bnNp
Z25lZCBsb25nKS1NQVhfRVJSTk8pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY29tcGlsZXIuaDoyMTo0Mzogbm90ZTog
aW4gZGVmaW5pdGlvbiBvZiBtYWNybyDigJh1bmxpa2VseeKAmQogICAyMSB8ICNkZWZpbmUg
dW5saWtlbHkoeCkgICBfX2J1aWx0aW5fZXhwZWN0KCEhKHgpLDApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpkcml2ZXJzL3Bhc3N0aHJv
dWdoL2FybS9zbW11LmM6MTM3OToxMzogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mElTX0VSUl9WQUxVReKAmQogMTM3OSB8ICAgICAgICAgaWYgKElTX0VSUl9WQUxVRShyZXQp
KQogICAgICB8ICAgICAgICAgICAgIF5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2Vy
ci5oOjE3OjM4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYbG9uZyB1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogICAxNyB8ICNkZWZpbmUgSVNfRVJSX1ZBTFVF
KHgpIHVubGlrZWx5KCh4KSA+PSAodW5zaWduZWQgbG9uZyktTUFYX0VSUk5PKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVu
L2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g4oCYdW5s
aWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGluX2V4cGVj
dCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIF4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjEzOTc6MTM6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhJU19FUlJfVkFMVUXigJkKIDEzOTcgfCAgICAg
ICAgIGlmIChJU19FUlJfVkFMVUUocmV0KSkgewogICAgICB8ICAgICAgICAgICAgIF5+fn5+
fn5+fn5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKA
mGFybV9zbW11X2ZpbmRfc21l4oCZOgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6
MTU2NzoyMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKA
mHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTU2NyB8ICAgICAgICAgZm9y
IChpID0gMDsgaSA8IHNtbXUtPm51bV9tYXBwaW5nX2dyb3VwczsgKytpKSB7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgIF4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5j
OiBJbiBmdW5jdGlvbiDigJhhcm1fc21tdV9tYXN0ZXJfYWxsb2Nfc21lc+KAmToKZHJpdmVy
cy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjYzMDo0OTogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNjMwIHwgICAg
ICAgICBmb3IgKGkgPSAwOyBpZHggPSAoY2ZnKS0+c21lbmR4W2ldLCAoaSkgPCAobnVtKTsg
KysoaSkpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgXgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6MTYyMTo5OiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfY2ZnX3NtZeKAmQogMTYyMSB8
ICAgICAgICAgZm9yX2VhY2hfY2ZnX3NtZShjZmcsIGksIGlkeCwgZndzcGVjLT5udW1faWRz
KSB7CiAgICAgIHwgICAgICAgICBefn5+fn5+fn5+fn5+fn5+CmRyaXZlcnMvcGFzc3Rocm91
Z2gvYXJtL3NtbXUuYzo2MzA6NDk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDYzMCB8ICAgICAgICAgZm9yIChp
ID0gMDsgaWR4ID0gKGNmZyktPnNtZW5keFtpXSwgKGkpIDwgKG51bSk7ICsrKGkpKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4K
ZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjE2NDI6OTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX2NmZ19zbWXigJkKIDE2NDIgfCAgICAgICAgIGZv
cl9lYWNoX2NmZ19zbWUoY2ZnLCBpLCBpZHgsIGZ3c3BlYy0+bnVtX2lkcykgewogICAgICB8
ICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11
LmM6IEluIGZ1bmN0aW9uIOKAmGFybV9zbW11X2RvbWFpbl9hZGRfbWFzdGVy4oCZOgpkcml2
ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6NjMwOjQ5OiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA2MzAgfCAg
ICAgICAgIGZvciAoaSA9IDA7IGlkeCA9IChjZmcpLT5zbWVuZHhbaV0sIChpKSA8IChudW0p
OyArKyhpKSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBeCmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzoxNjY4Ojk6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9jZmdfc21l4oCZCiAxNjY4
IHwgICAgICAgICBmb3JfZWFjaF9jZmdfc21lKGNmZywgaSwgaWR4LCBmd3NwZWMtPm51bV9p
ZHMpIHsKICAgICAgfCAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn4KZHJpdmVycy9wYXNzdGhy
b3VnaC9hcm0vc21tdS5jOiBJbiBmdW5jdGlvbiDigJhhcm1fc21tdV9hdHRhY2hfZGV24oCZ
OgouL2luY2x1ZGUveGVuL2Vyci5oOjE3OjM4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogICAxNyB8ICNk
ZWZpbmUgSVNfRVJSX1ZBTFVFKHgpIHVubGlrZWx5KCh4KSA+PSAodW5zaWduZWQgbG9uZykt
TUFYX0VSUk5PKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRp
b24gb2YgbWFjcm8g4oCYdW5saWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgp
ICAgX19idWlsdGluX2V4cGVjdCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIF4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21t
dS5jOjE3MjE6MjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhJU19FUlJfVkFM
VUXigJkKIDE3MjEgfCAgICAgICAgICAgICAgICAgaWYgKElTX0VSUl9WQUxVRShyZXQpKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+CmRyaXZlcnMvcGFzc3Ro
cm91Z2gvYXJtL3NtbXUuYzogSW4gZnVuY3Rpb24g4oCYX19hcm1fc21tdV9nZXRfcGNpX3Np
ZOKAmToKZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjIwNjY6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMjA2
NiB8IHN0YXRpYyBpbnQgX19hcm1fc21tdV9nZXRfcGNpX3NpZChzdHJ1Y3QgcGNpX2RldiAq
cGRldiwgdTE2IGFsaWFzLCB2b2lkICpkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgpkcml2ZXJzL3Bhc3N0aHJv
dWdoL2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFybV9zbW11X2RldmljZV9yZXNldOKA
mToKZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjIyMTM6MjM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFst
V3NpZ24tY29tcGFyZV0KIDIyMTMgfCAgICAgICAgIGZvciAoaSA9IDA7IGkgPCBzbW11LT5u
dW1fbWFwcGluZ19ncm91cHM7ICsraSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
Xgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6MjIxNzoyMzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1X
c2lnbi1jb21wYXJlXQogMjIxNyB8ICAgICAgICAgZm9yIChpID0gMDsgaSA8IHNtbXUtPm51
bV9jb250ZXh0X2JhbmtzOyArK2kpIHsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
Xgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6IEluIGZ1bmN0aW9uIOKAmGFybV9z
bW11X2RldmljZV9jZmdfcHJvYmXigJk6CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3
bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFz
dOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAg
fCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAg
ICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
Y3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBe
fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6MjM2NToyMzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KIDIzNjUgfCAgICAgICAgIGZvciAoaSA9IDA7IGkgPCBzaXplOyBpKyspCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNr
X3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBm
cmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KZHJpdmVycy9w
YXNzdGhyb3VnaC9hcm0vc21tdS5jOiBJbiBmdW5jdGlvbiDigJhhcm1fc21tdV9kZXZpY2Vf
ZHRfcHJvYmXigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzoyNDk4OjMwOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQg
aW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAyNDk4IHwgICAgICAgICAgICAgICAgIGlmIChu
dW1faXJxcyA+IHNtbXUtPm51bV9nbG9iYWxfaXJxcykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIF4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjI1NTk6
MjM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNp
Z25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KIDI1NTkgfCAgICAgICAgIGZvciAoaSA9
IDA7IGkgPCBzbW11LT5udW1fZ2xvYmFsX2lycXM7ICsraSkgewogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICBeCmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzogSW4gZnVu
Y3Rpb24g4oCYYXJtX3NtbXVfaW90bGJfZmx1c2jigJk6CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlv
biDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CmRyaXZl
cnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzoyNzA5OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRmbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMjcwOSB8IHN0YXRpYyBp
bnQgX19tdXN0X2NoZWNrIGFybV9zbW11X2lvdGxiX2ZsdXNoKHN0cnVjdCBkb21haW4gKmQs
IGRmbl90IGRmbiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KZHJpdmVycy9wYXNzdGhy
b3VnaC9hcm0vc21tdS5jOjI3MTA6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGFnZV9jb3VudOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMjcxMCB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgcGFnZV9j
b3VudCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn5+fn5+fn4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0v
c21tdS5jOiBJbiBmdW5jdGlvbiDigJhhcm1fc21tdV9hc3NpZ25fZGV24oCZOgpkcml2ZXJz
L3Bhc3N0aHJvdWdoL2FybS9zbW11LmM6Mjc2OTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZXZmbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMjc2OSB8IHN0YXRpYyBp
bnQgYXJtX3NtbXVfYXNzaWduX2RldihzdHJ1Y3QgZG9tYWluICpkLCB1OCBkZXZmbiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+Xn5+fn4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vc21tdS5jOjI3NzA6NTY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZmxhZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
Mjc3MCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZGV2aWNlICpk
ZXYsIHUzMiBmbGFnKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJ
biBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVk
IGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9k
aXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9w
cmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmRyaXZlcnMv
cGFzc3Rocm91Z2gvYXJtL3NtbXUuYzogSW4gZnVuY3Rpb24g4oCYYXJtX3NtbXVfZHRfaW5p
dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3Rf
ZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVu
Y3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1h
c2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn5+fn5+fn5+fn5+CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL3NtbXUuYzoyOTQ2OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KIDI5NDYgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0
IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5efn5+CiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9zbXBib290Lm8K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1Ogou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEz
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJz
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9w
YWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1f
dHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICB4c20vZmxhc2sv
c3MvcG9saWN5ZGIubwouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Js
b2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fz
c2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4
IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2
ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNl
KHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lf
cmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZw
Y2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJf
dCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90
IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoz
MDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9w
cm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFp
buKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBh
bHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9ldmVudC5oOjE3LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2Rv
bWFpbi5jOjE2OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZl
bnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9p
c19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL3ZtX2V2ZW50Lmg6MSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdm1fZXZlbnQuaDoyOCwKICAg
ICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9kb21haW4uYzoxNzoKLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X2luaXRfZG9tYWlu
4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoxNDo1NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDE0IHwg
c3RhdGljIGlubGluZSBpbnQgdm1fZXZlbnRfaW5pdF9kb21haW4oc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5j
dGlvbiDigJh2bV9ldmVudF90b2dnbGVfc2luZ2xlc3RlcOKAmToKLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL3ZtX2V2ZW50Lmg6MjU6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2
bV9ldmVudF90b2dnbGVfc2luZ2xlc3RlcChzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQu
aDoyNTo3ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDI1IHwgc3RhdGljIGlubGluZSB2b2lkIHZtX2V2ZW50X3RvZ2dsZV9z
aW5nbGVzdGVwKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQu
aDoyNjo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjYgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9u
IOKAmHZtX2V2ZW50X3JlZ2lzdGVyX3dyaXRlX3Jlc3VtZeKAmToKLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL3ZtX2V2ZW50Lmg6MzI6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMiB8IHZvaWQgdm1fZXZlbnRfcmVnaXN0
ZXJfd3JpdGVfcmVzdW1lKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNwb25zZV90ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MzI6NzQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDMyIHwgdm9pZCB2bV9ldmVudF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUoc3RydWN0IHZjcHUg
KnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJh2bV9ldmVudF9lbXVsYXRlX2NoZWNr4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1f
ZXZlbnQuaDozODo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDM4IHwgdm9pZCB2bV9ldmVudF9lbXVsYXRlX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2Vu
ZXJpYy92bV9ldmVudC5oOjM4OjY2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJz
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzOCB8IHZvaWQgdm1fZXZlbnRfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJ
biBmdW5jdGlvbiDigJh2bV9ldmVudF9zeW5jX2V2ZW504oCZOgouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvdm1fZXZlbnQuaDo0NDozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQ0IHwgdm9pZCB2bV9ldmVudF9zeW5jX2V2
ZW50KHN0cnVjdCB2Y3B1ICp2LCBib29sIHZhbHVlKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZl
bnQuaDo0NDo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YWx1ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA0NCB8IHZvaWQgdm1fZXZlbnRfc3luY19ldmVudChzdHJ1
Y3QgdmNwdSAqdiwgYm9vbCB2YWx1ZSkKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50
Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3Jlc2V0X3ZtdHJhY2XigJk6Ci4vaW5jbHVk
ZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjUwOjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCB2b2lkIHZtX2V2ZW50
X3Jlc2V0X3ZtdHJhY2Uoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21t
b24vZG9tYWluLmM6MjI6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rp
b24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs4oCZOgouL2luY2x1ZGUveGVuL2RvbWFp
bl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9t
YWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZhKSB7fTsKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVn
cy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90
aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6
NTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2Vu
di5oOjE0MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL2FjcGkuaDo1
NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vYWNwaS5oOjU3LAogICAg
ICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vYXJtNjQvc21wYm9vdC5jOjE6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2li
bGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQg
aW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9oeXBl
cmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9kb21haW4uYzoyNToK
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2dldGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDIyOSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxf
c2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVs
dF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fZG9tY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21k
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3N5c2N0bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyNjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
YWRjb25zb2xlKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlk
X3QgaWQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2V2dGNobl9zZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBl
dnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozMDM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMwOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZ3JhbnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMyMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21faW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozNzE6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFy
Z2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9zZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbW9yeV9leGNoYW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzgxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM5MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5f
cGFnZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIs
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0MDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJv
bV9waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWlt
X3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21f
ZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21h
cF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZv
aWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
dW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJx
LCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50
IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUs
IHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3Qg
YWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0
YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNz
aWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIg
KmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVf
YmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2Jk
ZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToKSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3Vy
Y2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZ
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhE
RUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUo
RU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+
fn5+fn4KLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU3NDo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vf
c2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21p
c2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9w
YWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVm
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHZtX3BhcmFtX2Fs
dHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAy
bWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1
aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9n
ZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9j
b250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3Ap
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYz
NTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9y
bV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lv
bih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4
c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9ncmFudF90YWJsZS5oOjMxLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2RvbWFp
bi5jOjMyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0
aW9uIOKAmGdudHRhYl9tYXJrX2RpcnR54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dy
YW50X3RhYmxlLmg6MjA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBnbnR0YWJf
bWFya19kaXJ0eShzdHJ1Y3QgZG9tYWluICpkLCBtZm5fdCBtZm4pCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjIwOjYyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBnbnR0YWJfbWFya19kaXJ0eShzdHJ1Y3QgZG9tYWluICpkLCBtZm5f
dCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50
X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9ob3N0X21hcHBpbmdfZ2V0X3BhZ2Vf
dHlwZeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjI3OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJv4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDI3IHwgc3RhdGljIGlubGluZSBib29sIGdudHRhYl9ob3N0X21hcHBpbmdfZ2V0X3Bh
Z2VfdHlwZShib29sIHJvLAouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6
Mjg6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKmxkLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5o
OjI5Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDI5IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICpyZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUu
aDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX3JlbGVhc2VfaG9zdF9tYXBwaW5nc+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjM0OjcwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzQgfCBzdGF0
aWMgaW5saW5lIGJvb2wgZ250dGFiX3JlbGVhc2VfaG9zdF9tYXBwaW5ncyhjb25zdCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIGNvbW1vbi9kb21haW4uYzozMzoKLi9pbmNsdWRlL3hlbi94ZW5vcHJvZi5oOiBJ
biBmdW5jdGlvbiDigJhhY3F1aXJlX3BtdV9vd25lcnNoaXDigJk6Ci4vaW5jbHVkZS94ZW4v
eGVub3Byb2YuaDozOTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwbXVfb3du
ZXJzaGlw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM5IHwgc3RhdGljIGlubGluZSBp
bnQgYWNxdWlyZV9wbXVfb3duZXJzaGlwKGludCBwbXVfb3duZXJzaGlwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fn5+fn5+fn5+
fgouL2luY2x1ZGUveGVuL3hlbm9wcm9mLmg6IEluIGZ1bmN0aW9uIOKAmHJlbGVhc2VfcG11
X293bmVyc2hpcOKAmToKLi9pbmNsdWRlL3hlbi94ZW5vcHJvZi5oOjQ0OjQ2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBtdV9vd25lcnNoaXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNDQgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmVsZWFzZV9wbXVfb3duZXJzaGlw
KGludCBwbXVfb3duZXJzaGlwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fl5+fn5+fn5+fn5+fn4KY29tbW9uL2RvbWFpbi5jOiBJbiBm
dW5jdGlvbiDigJhsYXRlX2h3ZG9tX2luaXTigJk6CmNvbW1vbi9kb21haW4uYzozMDY6NDM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwNiB8IHN0YXRpYyBpbnQgbGF0ZV9od2RvbV9pbml0KHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi8uL2luY2x1ZGUveGVuL2NvbmZpZy5oOjE3LAogICAg
ICAgICAgICAgICAgIGZyb20gPGNvbW1hbmQtbGluZT46CmNvbW1vbi9kb21haW4uYzogSW4g
ZnVuY3Rpb24g4oCYX2RvbWFpbl9kZXN0cm954oCZOgpjb21tb24vZG9tYWluLmM6NDk0OjM2
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICA0OTQgfCAgICAgQlVHX09OKGF0b21pY19yZWFkKCZkLT5yZWZjbnQp
ICE9IERPTUFJTl9ERVNUUk9ZRUQpOwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBp
biBkZWZpbml0aW9uIG9mIG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1
bmxpa2VseSh4KSAgIF9fYnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9kb21haW4uYzo0
OTQ6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmEJVR19PTuKAmQogIDQ5NCB8
ICAgICBCVUdfT04oYXRvbWljX3JlYWQoJmQtPnJlZmNudCkgIT0gRE9NQUlOX0RFU1RST1lF
RCk7CiAgICAgIHwgICAgIF5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJm
Yy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzAsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWNwaS5oOjI1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51
eC5oOjU4OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWdu
ZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBw
YWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFn
ZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxi
Zmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6
NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1
bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+
fn5+CmNvbW1vbi9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYY29tbW9uX3ZjcHVfb3DigJk6
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+CmNvbW1vbi9kb21haW4uYzoxODY0OjI5OiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE4NjQgfCAgICAg
ICAgICAgICBpZiAoIHYtPnZjcHVfaWQgIT0gdmNwdWlkICYmICF0ZXN0X2JpdChfVlBGX2Rv
d24sICZ2LT5wYXVzZV9mbGFncykgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBefgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL2RvbWFpbi5jOjE5MzQ6MzM6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYdWludDY0X3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gYW5k
IOKAmGxvbmcgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE5MzQgfCAgICAgICAgIGlmICgg
c2V0LnRpbWVvdXRfYWJzX25zIDwgTk9XKCkgKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
ZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1f
bWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3Ap
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMg
aW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFn
ZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5
cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20geHNtL2ZsYXNrL3NzL3NpZHRhYi5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20geHNt
L2ZsYXNrL3NzL3BvbGljeWRiLmg6MzEsCiAgICAgICAgICAgICAgICAgZnJvbSB4c20vZmxh
c2svc3MvcG9saWN5ZGIuYzozNjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUg
aW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3RhdGVfbGltaXTigJk6Ci4v
aW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTIgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQp
IHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJ
biBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94
ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGlt
aXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQg
YWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0
dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBm
dW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CnhzbS9mbGFzay9zcy9wb2xpY3lkYi5j
OiBJbiBmdW5jdGlvbiDigJhwb2xpY3lkYl9sb29rdXBfY29tcGF04oCZOgp4c20vZmxhc2sv
c3MvcG9saWN5ZGIuYzoxNTc6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhs
b25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTU3IHwgICAgIGZvciAo
IGkgPSAwOyBpIDwgc2l6ZW9mKHBvbGljeWRiX2NvbXBhdCkvc2l6ZW9mKCppbmZvKTsgaSsr
ICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3MvcG9saWN5ZGIu
YzogSW4gZnVuY3Rpb24g4oCYY29tbW9uX2luZGV44oCZOgp4c20vZmxhc2svc3MvcG9saWN5
ZGIuYzoyNjM6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYa2V54oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjYzIHwgc3RhdGljIGludCBjZl9jaGVjayBjb21tb25faW5k
ZXgodm9pZCAqa2V5LCB2b2lkICpkYXR1bSwgdm9pZCAqZGF0YXApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CnhzbS9mbGFzay9zcy9wb2xp
Y3lkYi5jOjI2Mzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXR1beKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MyB8IHN0YXRpYyBpbnQgY2ZfY2hlY2sgY29tbW9u
X2luZGV4KHZvaWQgKmtleSwgdm9pZCAqZGF0dW0sIHZvaWQgKmRhdGFwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fn4K
eHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6MjYzOjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRhdGFw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjYzIHwgc3RhdGljIGlu
dCBjZl9jaGVjayBjb21tb25faW5kZXgodm9pZCAqa2V5LCB2b2lkICpkYXR1bSwgdm9pZCAq
ZGF0YXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fn4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6
IEluIGZ1bmN0aW9uIOKAmHBlcm1fZGVzdHJveeKAmToKeHNtL2ZsYXNrL3NzL3BvbGljeWRi
LmM6NTEyOjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1MTIgfCBzdGF0aWMgaW50IGNmX2NoZWNrIHBlcm1fZGVzdHJveSh2
b2lkICprZXksIHZvaWQgKmRhdHVtLCB2b2lkICpwKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNt
L2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKAmGNvbW1vbl9kZXN0cm954oCZ
Ogp4c20vZmxhc2svc3MvcG9saWN5ZGIuYzo1MTk6NjY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUxOSB8IHN0YXRpYyBpbnQg
Y2ZfY2hlY2sgY29tbW9uX2Rlc3Ryb3kodm9pZCAqa2V5LCB2b2lkICpkYXR1bSwgdm9pZCAq
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1
bmN0aW9uIOKAmGNsYXNzX2Rlc3Ryb3nigJk6CnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOjUz
MTo2NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTMxIHwgc3RhdGljIGludCBjZl9jaGVjayBjbGFzc19kZXN0cm95KHZvaWQg
KmtleSwgdm9pZCAqZGF0dW0sIHZvaWQgKnApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNtL2Zs
YXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKAmHJvbGVfZGVzdHJveeKAmToKeHNt
L2ZsYXNrL3NzL3BvbGljeWRiLmM6NTc4OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzggfCBzdGF0aWMgaW50IGNmX2No
ZWNrIHJvbGVfZGVzdHJveSh2b2lkICprZXksIHZvaWQgKmRhdHVtLCB2b2lkICpwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fl4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKA
mHR5cGVfZGVzdHJveeKAmToKeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6NTkwOjY0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
OTAgfCBzdGF0aWMgaW50IGNmX2NoZWNrIHR5cGVfZGVzdHJveSh2b2lkICprZXksIHZvaWQg
KmRhdHVtLCB2b2lkICpwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNtL2ZsYXNrL3NzL3BvbGlj
eWRiLmM6IEluIGZ1bmN0aW9uIOKAmHVzZXJfZGVzdHJveeKAmToKeHNtL2ZsYXNrL3NzL3Bv
bGljeWRiLmM6NTk3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1OTcgfCBzdGF0aWMgaW50IGNmX2NoZWNrIHVzZXJfZGVz
dHJveSh2b2lkICprZXksIHZvaWQgKmRhdHVtLCB2b2lkICpwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKAmHNlbnNfZGVzdHJv
eeKAmToKeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6NjExOjY0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTEgfCBzdGF0aWMg
aW50IGNmX2NoZWNrIHNlbnNfZGVzdHJveSh2b2lkICprZXksIHZvaWQgKmRhdHVtLCB2b2lk
ICpwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1
bmN0aW9uIOKAmGNhdF9kZXN0cm954oCZOgp4c20vZmxhc2svc3MvcG9saWN5ZGIuYzo2MjM6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYyMyB8IHN0YXRpYyBpbnQgY2ZfY2hlY2sgY2F0X2Rlc3Ryb3kodm9pZCAqa2V5
LCB2b2lkICpkYXR1bSwgdm9pZCAqcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KeHNtL2ZsYXNrL3Nz
L3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKAmHBvbGljeWRiX2Rlc3Ryb3nigJk6CnhzbS9m
bGFzay9zcy9wb2xpY3lkYi5jOjcyMzoyNDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQog
IDcyMyB8ICAgICAgICAgZm9yICggaSA9IDA7IGkgPCBwLT5wX3R5cGVzLm5wcmltOyBpKysg
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3MvcG9saWN5
ZGIuYzogSW4gZnVuY3Rpb24g4oCYcGVybV9yZWFk4oCZOgp4c20vZmxhc2svc3MvcG9saWN5
ZGIuYzo5NTI6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDk1MiB8IHN0YXRpYyBpbnQgcGVybV9yZWFkKHN0cnVjdCBwb2xp
Y3lkYiAqcCwgc3RydWN0IGhhc2h0YWIgKmgsIHZvaWQgKmZwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xgp4c20vZmxhc2svc3MvcG9saWN5ZGIu
YzogSW4gZnVuY3Rpb24g4oCYY29tbW9uX3JlYWTigJk6CnhzbS9mbGFzay9zcy9wb2xpY3lk
Yi5jOjEwMzQ6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2Fr
YSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KIDEwMzQgfCAgICAgZm9y
ICggaSA9IDA7IGkgPCBuZWw7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4K
eHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6IEluIGZ1bmN0aW9uIOKAmHJlYWRfY29uc19oZWxw
ZXLigJk6CnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOjEwODM6MjQ6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3Np
Z24tY29tcGFyZV0KIDEwODMgfCAgICAgICAgIGZvciAoIGogPSAwOyBqIDwgbmV4cHI7IGor
KyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICBeCnhzbS9mbGFzay9zcy9wb2xp
Y3lkYi5jOiBJbiBmdW5jdGlvbiDigJhjbGFzc19yZWFk4oCZOgp4c20vZmxhc2svc3MvcG9s
aWN5ZGIuYzoxMjA1OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZ
IHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAxMjA1IHwgICAg
IGZvciAoIGkgPSAwOyBpIDwgbmVsOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICBeCnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOiBJbiBmdW5jdGlvbiDigJhzZW5zX3JlYWTi
gJk6CnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOjE0ODU6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTQ4NSB8IHN0YXRpYyBp
bnQgY2ZfY2hlY2sgc2Vuc19yZWFkKHN0cnVjdCBwb2xpY3lkYiAqcCwgc3RydWN0IGhhc2h0
YWIgKmgsIHZvaWQgKmZwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+Xgp4c20vZmxhc2svc3MvcG9saWN5ZGIuYzogSW4gZnVuY3Rp
b24g4oCYY2F0X3JlYWTigJk6CnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOjE1NDA6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
MTU0MCB8IHN0YXRpYyBpbnQgY2ZfY2hlY2sgY2F0X3JlYWQoc3RydWN0IHBvbGljeWRiICpw
LCBzdHJ1Y3QgaGFzaHRhYiAqaCwgdm9pZCAqZnApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl4KeHNtL2ZsYXNrL3NzL3BvbGljeWRi
LmM6IEluIGZ1bmN0aW9uIOKAmHBvbGljeWRiX3JlYWTigJk6CnhzbS9mbGFzay9zcy9wb2xp
Y3lkYi5jOjE4Njg6MzA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDi
gJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogMTg2OCB8ICAgICBpZiAoIGxlMzJfdG9fY3B1
KGJ1ZlsyXSkgIT0gaW5mby0+c3ltX251bSB8fAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgXn4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6MTg2OTozMDogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAxODY5IHwgICAgICAgICAgbGUzMl90b19jcHUoYnVmWzNdKSAhPSBpbmZvLT5vY29u
X251bSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgp4c20vZmxh
c2svc3MvcG9saWN5ZGIuYzoxODg1OjI0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAx
ODg1IHwgICAgICAgICBmb3IgKCBqID0gMDsgaiA8IG5lbDsgaisrICkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgIF4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6MTkxMToyMDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVk
IGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTkxMSB8ICAgICBmb3IgKCBpID0gMDsgaSA8
IG5lbDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3Mv
cG9saWN5ZGIuYzoxOTQ3OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy
4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAxOTQ3IHwg
ICAgIGZvciAoIGkgPSAwOyBpIDwgbmVsOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICBeCnhzbS9mbGFzay9zcy9wb2xpY3lkYi5jOjIwMDM6MjQ6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3Np
Z24tY29tcGFyZV0KIDIwMDMgfCAgICAgICAgIGZvciAoIGogPSAwOyBqIDwgbmVsOyBqKysg
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3MvcG9saWN5
ZGIuYzoyMTc1OjI0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZIHth
a2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAyMTc1IHwgICAgICAg
ICBmb3IgKCBpID0gMDsgaSA8IG5lbDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgIF4KeHNtL2ZsYXNrL3NzL3BvbGljeWRiLmM6MjIyMzoyMDogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1X
c2lnbi1jb21wYXJlXQogMjIyMyB8ICAgICBmb3IgKCBpID0gMDsgaSA8IHAtPnBfdHlwZXMu
bnByaW07IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KICBDQyAgICAgIGFy
Y2gvYXJtL2FybTY0L3N2ZS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvc3ZlLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9h
cm02NC9zdmUuYzoxMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAg
fCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDox
Mzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAg
fCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVu
Y3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4p
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVu
c2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8
IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAg
ICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2
b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGlt
ZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2Nr
KHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2so
c3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1f
YWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJl
X3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29s
IHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKA
mGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoy
OTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3Ry
dWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJ
biBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0
cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoy
NToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0
aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0
IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFk
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBj
aV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93
cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3Nf
cGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19p
bmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2FybS9hcm02NC9zdmUtYXNt
Lm8KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L3RyYXBzLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20g
YXJjaC9hcm0vYXJtNjQvdHJhcHMuYzoyMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1
X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1
X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2
b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251
bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25p
ZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIz
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3Zj
cHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBj
aV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODgg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVp
bnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2
cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRf
ZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3Rh
dGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2FybS9h
cm02NC92ZnAubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9hcm02NC92ZnAuYzoxOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1
X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDox
MDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVu
c2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9u
IOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcnds
b2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN0
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8
ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9f
bmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGlj
IGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9u
IOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10
cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1
X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9l
dmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50
MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9u
ZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwg
c3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVw
ZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVz
dF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9y
eV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRl
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZw
Y2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikg
eyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVu
ZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2
Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJ
biBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2
XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KICBDQyAgICAgIGFyY2gvYXJtL2FybTY0L3ZzeXNyZWcubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSBhcmNoL2FybS9hcm02NC92c3lzcmVnLmM6MTk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhf
Q09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3go
dm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1m
bl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFp
bi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1
bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBh
Z2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIo
c3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQog
ICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNr
LCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFn
ZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFn
ZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNj
ZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2
X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAy
bV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9t
X2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8
ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAq
dCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQg
YXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0
cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVh
c3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5l
IHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
d3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3Rf
Y2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2
XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEz
MSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gYXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5jOjIxOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1aWRfZmVhdHVy
ZV9leHRyYWN0X2ZpZWxkX3dpZHRo4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L2NwdWZlYXR1cmUuaDo4MDoxNzogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFu
Z2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5zaWduZWQgaW504oCZIGR1
ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAg
IDgwIHwgICAgICAgICAgICAgICAgIGNwdWlkX2ZlYXR1cmVfZXh0cmFjdF9zaWduZWRfZmll
bGRfd2lkdGgoZmVhdHVyZXMsIGZpZWxkLCB3aWR0aCkgOgogICAgICB8ICAgICAgICAgICAg
ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzogSW4gZnVuY3Rpb24g
4oCYdnJlZ19lbXVsYXRlX1NDVExSX0VMMeKAmToKYXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5j
OjM2OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ3PigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMzYgfCBzdGF0aWMgYm9vbCB2cmVnX2VtdWxhdGVfIyNyZWcoc3Ry
dWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsICAgICAgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9h
cm0vYXJtNjQvdnN5c3JlZy5jOjUxOjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhUVk1fUkVH4oCZCiAgIDUxIHwgVFZNX1JFRyhTQ1RMUl9FTDEpCiAgICAgIHwgXn5+fn5+
fgphcmNoL2FybS9hcm02NC92c3lzcmVnLmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdfZW11bGF0
ZV9UVEJSMF9FTDHigJk6CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzozNjo1NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDM2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9y
ZWdzICpyZWdzLCAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNy
ZWcuYzo1MjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQog
ICA1MiB8IFRWTV9SRUcoVFRCUjBfRUwxKQogICAgICB8IF5+fn5+fn4KYXJjaC9hcm0vYXJt
NjQvdnN5c3JlZy5jOiBJbiBmdW5jdGlvbiDigJh2cmVnX2VtdWxhdGVfVFRCUjFfRUwx4oCZ
OgphcmNoL2FybS9hcm02NC92c3lzcmVnLmM6MzY6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVnc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNiB8IHN0YXRpYyBi
b29sIHZyZWdfZW11bGF0ZV8jI3JlZyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywgICAg
ICAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9hcm02NC92c3lzcmVnLmM6NTM6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUfigJkKICAgNTMgfCBUVk1fUkVH
KFRUQlIxX0VMMSkKICAgICAgfCBefn5+fn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzog
SW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX1RDUl9FTDHigJk6CmFyY2gvYXJtL2FybTY0
L3ZzeXNyZWcuYzozNjo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRl
XyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCAgICAgICAgICBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzo1NDoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA1NCB8IFRWTV9SRUcoVENSX0VMMSkKICAgICAg
fCBefn5+fn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJl
Z19lbXVsYXRlX0VTUl9FTDHigJk6CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzozNjo1NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDM2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVjdCBjcHVf
dXNlcl9yZWdzICpyZWdzLCAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2FybTY0
L3ZzeXNyZWcuYzo1NToxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JF
R+KAmQogICA1NSB8IFRWTV9SRUcoRVNSX0VMMSkKICAgICAgfCBefn5+fn5+CmFyY2gvYXJt
L2FybTY0L3ZzeXNyZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX0ZBUl9FTDHi
gJk6CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzozNjo1NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM2IHwgc3RhdGlj
IGJvb2wgdnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCAg
ICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzo1NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA1NiB8IFRWTV9S
RUcoRkFSX0VMMSkKICAgICAgfCBefn5+fn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzog
SW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX0FGU1IwX0VMMeKAmToKYXJjaC9hcm0vYXJt
NjQvdnN5c3JlZy5jOjM2OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ3Pi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzYgfCBzdGF0aWMgYm9vbCB2cmVnX2VtdWxh
dGVfIyNyZWcoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsICAgICAgICAgIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KYXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5jOjU3OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhUVk1fUkVH4oCZCiAgIDU3IHwgVFZNX1JFRyhBRlNSMF9FTDEpCiAg
ICAgIHwgXn5+fn5+fgphcmNoL2FybS9hcm02NC92c3lzcmVnLmM6IEluIGZ1bmN0aW9uIOKA
mHZyZWdfZW11bGF0ZV9BRlNSMV9FTDHigJk6CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzoz
Njo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDM2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVj
dCBjcHVfdXNlcl9yZWdzICpyZWdzLCAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJt
L2FybTY0L3ZzeXNyZWcuYzo1ODoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
VFZNX1JFR+KAmQogICA1OCB8IFRWTV9SRUcoQUZTUjFfRUwxKQogICAgICB8IF5+fn5+fn4K
YXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5jOiBJbiBmdW5jdGlvbiDigJh2cmVnX2VtdWxhdGVf
TUFJUl9FTDHigJk6CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzozNjo1NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM2
IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdz
ICpyZWdzLCAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcu
Yzo1OToxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA1
OSB8IFRWTV9SRUcoTUFJUl9FTDEpCiAgICAgIHwgXn5+fn5+fgphcmNoL2FybS9hcm02NC92
c3lzcmVnLmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdfZW11bGF0ZV9BTUFJUl9FTDHigJk6CmFy
Y2gvYXJtL2FybTY0L3ZzeXNyZWcuYzozNjo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM2IHwgc3RhdGljIGJvb2wg
dnJlZ19lbXVsYXRlXyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCAgICAgICAg
ICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzo2MDoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA2MCB8IFRWTV9SRUcoQU1B
SVJfRUwxKQogICAgICB8IF5+fn5+fn4KYXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5jOiBJbiBm
dW5jdGlvbiDigJh2cmVnX2VtdWxhdGVfQ09OVEVYVElEUl9FTDHigJk6CmFyY2gvYXJtL2Fy
bTY0L3ZzeXNyZWcuYzozNjo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVs
YXRlXyMjcmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCAgICAgICAgICBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+CmFyY2gvYXJtL2FybTY0L3ZzeXNyZWcuYzo2MToxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA2MSB8IFRWTV9SRUcoQ09OVEVYVElEUl9F
TDEpCiAgICAgIHwgXn5+fn5+fgogIExEICAgICAgYXJjaC9hcm0vYXJtNjQvYnVpbHRfaW4u
bwogIENDICAgICAgY29tbW9uL2R0LW92ZXJsYXkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbl9idWlsZC5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSBjb21tb24vZHQtb3ZlcmxheS5jOjExOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9j
cHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9j
cHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2Nf
Y3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5l
IHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CiAg
Q0MgICAgICBhcmNoL2FybS9tbXUvcDJtLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1m
bl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBz
dGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3Rf
ZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVu
Y3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1h
c2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBkcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pcG1t
dS12bXNhLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJl
ZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEy
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDi
gJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMg
fCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZl
bnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAz
MSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2
X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
dm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5p
dHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2NwdS5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSBhcmNoL2FybS9tbXUvcDJtLmM6MjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNw
dV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUg
KnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlv
biDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVz
dF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5l
IGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWdu
X2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYg
KnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluX3Bh
Z2UuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL21tdS9wMm0uYzozOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAg
ICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAg
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNs
ZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAg
fCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Bj
aS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9pb21tdS5oOjI1
LAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaXBtbXUt
dm1zYS5jOjQyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVu
Y3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25y
X3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tf
dmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhm
cmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tf
dmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5k
aW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5j
bHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGlu
ZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKA
mF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVu
Y3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1m
buKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDox
MDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3Rf
ZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9m
aWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9kb21haW5f
cGFnZV9nbG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA3MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9iYWwoY29u
c3Qgdm9pZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
aW9yZXEuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL21tdS9wMm0uYzo0
OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUg
bm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2Nr
KHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2so
c3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1f
YWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJl
X3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29s
IHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2lycS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BjaS5oOjEzOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAg
ICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYg
KG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2lycS5oOjU6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hf
dmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwg
c3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3Qg
cGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwg
dWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1
aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNr
IHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcGNpLmg6MTQ6Ci4v
aW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2lu
Y2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2Rl
aWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVu
Y3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJp
dF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9tbXUvcDJtLmM6MTA6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBp
bnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvcGFzc3Rocm91Z2gv
YXJtL2lwbW11LXZtc2EuYzo0NjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vbGliLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjg1OgphcmNoL2FybS9tbXUvcDJtLmM6IEluIGZ1bmN0
aW9uIOKAmHAybV9nZXRfZW50cnnigJk6CmFyY2gvYXJtL21tdS9wMm0uYzo0NjI6Mjk6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYbG9uZyB1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGxvbmcgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICA0NjIgfCAgICAgQlVJTERfQlVHX09OKFRISVJEX01BU0sg
IT0gUEFHRV9NQVNLKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4K
Li9pbmNsdWRlL3hlbi9tYWNyb3MuaDo0Njo0ODogbm90ZTogaW4gZGVmaW5pdGlvbiBvZiBt
YWNybyDigJhCVUlMRF9CVUdfT07igJkKICAgNDYgfCAjZGVmaW5lIEJVSUxEX0JVR19PTihj
b25kKSAoeyBfU3RhdGljX2Fzc2VydCghKGNvbmQpLCAiISgiICNjb25kICIpIik7IH0pCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBe
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZl
cnMvcGFzc3Rocm91Z2gvYXJtL2lwbW11LXZtc2EuYzo0NzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2ln
bmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9l
dmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmFy
Y2gvYXJtL21tdS9wMm0uYzogSW4gZnVuY3Rpb24g4oCYc2V0dXBfdmlydF9wYWdpbmdfb25l
4oCZOgphcmNoL2FybS9tbXUvcDJtLmM6MTY0MDo0MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxNjQwIHwgc3RhdGljIHZv
aWQgc2V0dXBfdmlydF9wYWdpbmdfb25lKHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgphcmNo
L2FybS9tbXUvcDJtLmM6IEluIGZ1bmN0aW9uIOKAmGNwdV92aXJ0X3BhZ2luZ19jYWxsYmFj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1v
cnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KYXJjaC9h
cm0vbW11L3AybS5jOjE3OTA6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmZi
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxNzkwIHwgc3RhdGljIGludCBjcHVfdmlydF9w
YWdpbmdfY2FsbGJhY2soc3RydWN0IG5vdGlmaWVyX2Jsb2NrICpuZmIsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KYXJjaC9hcm0vbW11L3AybS5jOjE3OTI6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTc5MiB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHZvaWQgKmhjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgphcmNoL2FybS9tbXUv
cDJtLmM6IEF0IHRvcCBsZXZlbDoKYXJjaC9hcm0vbW11L3AybS5jOjE4MjY6MTogd2Fybmlu
Zzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1X
b2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogMTgyNiB8IF9faW5pdGNhbGwoY3B1X3ZpcnRfcGFn
aW5nX2luaXQpOwogICAgICB8IF5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVz
dF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNf
cHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29u
c3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCmRyaXZlcnMvcGFz
c3Rocm91Z2gvYXJtL2lwbW11LXZtc2EuYzogSW4gZnVuY3Rpb24g4oCYaXBtbXVfZG9tYWlu
X2FsbG9jYXRlX2NvbnRleHTigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lwbW11LXZt
c2EuYzo1MDk6MTQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDUwOSB8ICAgICBpZiAoIHJldCAhPSBtbXUtPm51
bV9jdHggKQogICAgICB8ICAgICAgICAgICAgICBefgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2Fy
bS9pcG1tdS12bXNhLmM6IEluIGZ1bmN0aW9uIOKAmGlwbW11X2lyceKAmToKZHJpdmVycy9w
YXNzdGhyb3VnaC9hcm0vaXBtbXUtdm1zYS5jOjY0NToyNzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NDUgfCBzdGF0aWMg
dm9pZCBpcG1tdV9pcnEoaW50IGlycSwgdm9pZCAqZGV2KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+Xn5+CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lwbW11LXZtc2Eu
YzogSW4gZnVuY3Rpb24g4oCYaXBtbXVfaW90bGJfZmx1c2jigJk6CmRyaXZlcnMvcGFzc3Ro
cm91Z2gvYXJtL2lwbW11LXZtc2EuYzoxMDAzOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRmbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTAwMyB8IHN0YXRpYyBpbnQg
X19tdXN0X2NoZWNrIGlwbW11X2lvdGxiX2ZsdXNoKHN0cnVjdCBkb21haW4gKmQsIGRmbl90
IGRmbiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0v
aXBtbXUtdm1zYS5jOjEwMDQ6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFn
ZV9jb3VudOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTAwNCB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgcGFnZV9jb3VudCwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fn5+fn5+fn4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaXBtbXUtdm1z
YS5jOiBJbiBmdW5jdGlvbiDigJhpcG1tdV9hc3NpZ25fZGV2aWNl4oCZOgpkcml2ZXJzL3Bh
c3N0aHJvdWdoL2FybS9pcG1tdS12bXNhLmM6MTExODo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZXZmbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTExOCB8IHN0YXRp
YyBpbnQgaXBtbXVfYXNzaWduX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLCB1OCBkZXZmbiwg
c3RydWN0IGRldmljZSAqZGV2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5efn5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2Fy
bS9pcG1tdS12bXNhLmM6MTExOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhm
bGFn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTE5IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVpbnQzMl90IGZsYWcpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaXBt
bXUtdm1zYS5jOiBJbiBmdW5jdGlvbiDigJhpcG1tdV9hZGRfZGV2aWNl4oCZOgpkcml2ZXJz
L3Bhc3N0aHJvdWdoL2FybS9pcG1tdS12bXNhLmM6MTI4MzozMjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZXZmbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTI4MyB8IHN0
YXRpYyBpbnQgaXBtbXVfYWRkX2RldmljZSh1OCBkZXZmbiwgc3RydWN0IGRldmljZSAqZGV2
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5efn5+fgpkcml2ZXJz
L3Bhc3N0aHJvdWdoL2FybS9pcG1tdS12bXNhLmM6IEluIGZ1bmN0aW9uIOKAmGlwbW11X2lu
aXTigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lwbW11LXZtc2EuYzoxMzY3OjcxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KIDEzNjcgfCBzdGF0aWMgX19pbml0IGludCBpcG1tdV9pbml0KHN0cnVjdCBkdF9kZXZp
Y2Vfbm9kZSAqbm9kZSwgY29uc3Qgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+Xn5+fgogIENDICAgICAgeHNtL2ZsYXNrL3NzL3NlcnZpY2VzLm8KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIHhzbS9mbGFzay9zcy9zZXJ2aWNlcy5jOjQyOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBj
cHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3Rs
Lmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5o
OjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQz
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NB
VkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAg
ICAgICAgICAgICAgICBmcm9tIHhzbS9mbGFzay9zcy9zZXJ2aWNlcy5jOjQzOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAg
ICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAg
aWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUg
fCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+CnhzbS9mbGFzay9zcy9zZXJ2aWNlcy5jOiBJbiBmdW5jdGlvbiDigJh2
YWxpZGF0ZV9jbGFzc2Vz4oCZOgp4c20vZmxhc2svc3Mvc2VydmljZXMuYzoxMTgxOjIwOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQg
aW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAxMTgxIHwgICAgIGZvciAoIGkgPSAxOyBpIDwg
a2RlZnMtPmN0c19sZW47IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KeHNt
L2ZsYXNrL3NzL3NlcnZpY2VzLmM6MTE4NjoxNjogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJl
XQogMTE4NiB8ICAgICAgICAgaWYgKCBpID4gcC0+cF9jbGFzc2VzLm5wcmltICkKICAgICAg
fCAgICAgICAgICAgICAgICBeCnhzbS9mbGFzay9zcy9zZXJ2aWNlcy5jOjEyMDI6MjA6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBp
bnTigJl9IFstV3NpZ24tY29tcGFyZV0KIDEyMDIgfCAgICAgZm9yICggaSA9IDA7IGkgPCBr
ZGVmcy0+YXZfcHRzX2xlbjsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgp4
c20vZmxhc2svc3Mvc2VydmljZXMuYzogSW4gZnVuY3Rpb24g4oCYY29udmVydF9jb250ZXh0
4oCZOgp4c20vZmxhc2svc3Mvc2VydmljZXMuYzoxMjc5OjQxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGtleeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTI3OSB8IHN0YXRp
YyBpbnQgY2ZfY2hlY2sgY29udmVydF9jb250ZXh0KHUzMiBrZXksIHN0cnVjdCBjb250ZXh0
ICpjLCB2b2lkICpwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5efn4KeHNtL2ZsYXNrL3NzL3NlcnZpY2VzLmM6IEluIGZ1bmN0aW9uIOKAmHNl
Y3VyaXR5X2ZpbmRfYm9vbOKAmToKeHNtL2ZsYXNrL3NzL3NlcnZpY2VzLmM6MTc2NzoyMDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVk
IGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTc2NyB8ICAgICBmb3IgKCBpID0gMDsgaSA8
IHBvbGljeWRiLnBfYm9vbHMubnByaW07IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgIF4KICBDQyAgICAgIGNvbW1vbi9ldmVudF8ybC5vCkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZXZlbnQuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9l
dmVudF9jaGFubmVsLmg6MywKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9ldmVudF8y
bC5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rp
b24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3Np
YmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IElu
IGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAg
ICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxv
Y19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDi
gJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5
IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEw
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQg
bG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVk
IGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9s
aXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IElu
IGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxi
Zmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXAp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3Vu
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoz
MTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3Vy
Y2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
NzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5z
aWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3Zt
X2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92
Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNw
dSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0
aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVw
ZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1
ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxp
bmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2Rl
diAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6
NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRl
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3By
b2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5n
KHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBi
b29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZl
bnQuaDoxNzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50
Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNf
Z2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CiAgTEQgICAgICBkcml2ZXJz
L3Bhc3N0aHJvdWdoL2FybS9idWlsdF9pbi5vCiAgQ0MgICAgICBkcml2ZXJzL3Bhc3N0aHJv
dWdoL2lvbW11Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gZHJpdmVycy9wYXNzdGhyb3VnaC9pb21t
dS5jOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rp
b24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3Np
YmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGNvbW1vbi9ldmVudF9j
aGFubmVsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3Rs
Lmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5o
OjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQz
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NB
VkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAg
ICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYg
KG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxMiwK
ICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9ldmVudF9jaGFubmVsLmg6MywKICAgICAg
ICAgICAgICAgICBmcm9tIGNvbW1vbi9ldmVudF9jaGFubmVsLmM6MTc6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGlu
Z3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50
IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4u
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgou
L2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9p
bmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9k
ZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVu
Y3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25l
ZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
cHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdl
X2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJm
bHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29s
IHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3Qg
KnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4
NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdl
dF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBw
Mm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAg
ICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAg
ICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
YWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFz
a192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5
OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAq
Z21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5j
dGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBp
bnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vz
c19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3Ry
dWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDog
SW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19w
dl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTcs
CiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJzL3Bhc3N0aHJvdWdoL2lvbW11LmM6MTk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92
aXJxX2lzX2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1
bnNpZ25lZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwg
cGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1
ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCY
dGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIo
Y3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUu
aDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gZHJpdmVy
cy9wYXNzdGhyb3VnaC9pb21tdS5jOjIwOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1
bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5
MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3Mg
ZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2ln
bmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3Np
Z24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBz
KTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8
IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVu
dF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMx
IHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
ZHJpdmVycy9wYXNzdGhyb3VnaC9pb21tdS5jOjIzOgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0ZG9tYWluaW5m
b+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjM1OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
aW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRf
dGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDYgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjUxOjQ0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1
Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsKHhzbV9kZWZh
dWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRf
dCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjczOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1
Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1h
cHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
MTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2Zl
cuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3
YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9w
b2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
NzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVz
ZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90
b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
bWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
ZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFp
bV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBp
bnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9w
cm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1
Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0
X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ4OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFsbG93KQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1
aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDE6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4
NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYs
IHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQz
Ml90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91
bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0
X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2
OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hp
bmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dm
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50
IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlz
Yyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVu
c2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODQg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0X3QgZGVm
LCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVsdF90IGRl
ZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJh
beKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRf
dm51bWFpbmZv4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94c20veHNtLmg6NjIz
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVs
dF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90
IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
Ojc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291
cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2lvbW11LmM6IEluIGZ1bmN0aW9uIOKAmGlvbW11
X2R1bXBfcGFnZV90YWJsZXPigJk6CmRyaXZlcnMvcGFzc3Rocm91Z2gvaW9tbXUuYzoyNDI6
NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYa2V54oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjQyIHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgaW9tbXVfZHVtcF9wYWdlX3Rh
Ymxlcyh1bnNpZ25lZCBjaGFyIGtleSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9s
aWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fz
c2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGlu
ZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAq
dikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVu
Y3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJp
dF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTc6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lz
X2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25l
ZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9ldmVu
dF9jaGFubmVsLmM6Mjc6Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9kb21haW5fY3JlYXRl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
MjQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90
IHNzaWRyZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjIyOTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0
ZG9tYWluaW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVk
dWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NDA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjQwIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVs
ZXJfb3AoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2RvbWN0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2RvbWN0bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBj
bWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNv
bGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9yZWFkY29uc29sZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
Y2xlYXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5fdW5ib3VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2
dGNobiAqY2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRv
bWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hh
bjEsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyODU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3Rh
dHVz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
OTc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50
MzJfdCBmbGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dy
YW50X2NvcHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0
X2hhcmR3YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzY2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9nZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Nl
dF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzgxOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM4MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhj
aGFuZ2UoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9y
ZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozOTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9tZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM5OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVtb3ZlX2Zyb21fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9jbGFpbV9wYWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0Mjg6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8o
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9wcm9maWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3Ap
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQzODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fa2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ0NDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21h
aW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX21hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
NjA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWlu
X3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV91bm1hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDcxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRf
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2Jp
bmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAq
YmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQ4OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
aW9tZW1fcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDk1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRf
dCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1f
bWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50
NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGNpX2NvbmZpZ19wZXJt
aXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwNzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDcgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVf
YmRmLCB1aW50MTZfdCBzdGFydCwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
NDA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVn
X3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1
aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWlu
dDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3Vy
Y2VfdW5wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2
MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZSh4c21fZGVm
YXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1Njk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBt
YWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVm
YXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1
MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9k
ZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTg5OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21f
ZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2h2bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBs
b25nIG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWlu
dDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fdm1fZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjI5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
bW9kZSwgaW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21l
bV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4
c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNl
X21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL2V2ZW50X2NoYW5uZWwu
YzogSW4gZnVuY3Rpb24g4oCYZGVmYXVsdF94ZW5fbm90aWZpY2F0aW9uX2Zu4oCZOgpjb21t
b24vZXZlbnRfY2hhbm5lbC5jOjg3OjM0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBvcnTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgODcgfCAgICAgc3RydWN0IHZjcHUg
KnYsIHVuc2lnbmVkIGludCBwb3J0KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KY29tbW9uL2V2ZW50X2NoYW5uZWwuYzogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9ldnRjaG5fYnVja2V04oCZOgpjb21tb24vZXZlbnRfY2hhbm5lbC5jOjE0NTo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTQ1IHwgc3RhdGljIHZvaWQgZnJlZV9ldnRjaG5fYnVja2V0KHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCBldnRjaG4gKmJ1Y2tldCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgpjb21tb24vZXZlbnRfY2hhbm5lbC5jOiBJ
biBmdW5jdGlvbiDigJhnZXRfZnJlZV9wb3J04oCZOgpjb21tb24vZXZlbnRfY2hhbm5lbC5j
OjI0NjoyNjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjQ2IHwgICAgIGZvciAoIHBvcnQgPSAwOyBwb3J0IDw9
IGQtPm1heF9ldnRjaG5fcG9ydDsgcG9ydCsrICkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgXn4KY29tbW9uL2V2ZW50X2NoYW5uZWwuYzogSW4gZnVuY3Rpb24g4oCYZXZ0
Y2huX2dldF9wb3J04oCZOgpjb21tb24vZXZlbnRfY2hhbm5lbC5jOjMwMDoxMjogd2Fybmlu
Zzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTi
gJkgdG8g4oCYZXZ0Y2huX3BvcnRfdOKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gZHVl
IHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAz
MDAgfCAgICAgcmV0dXJuIHJjID86IHBvcnQ7CiAgICAgIHwgICAgICAgICAgICBefgpjb21t
b24vZXZlbnRfY2hhbm5lbC5jOiBJbiBmdW5jdGlvbiDigJhldnRjaG5fYmluZF92aXJx4oCZ
Ogpjb21tb24vZXZlbnRfY2hhbm5lbC5jOjQ3MTozMDogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA0NzEg
fCAgICAgaWYgKCAodmlycSA8IDApIHx8ICh2aXJxID49IEFSUkFZX1NJWkUodi0+dmlycV90
b19ldnRjaG4pKSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgpj
b21tb24vZXZlbnRfY2hhbm5lbC5jOiBJbiBmdW5jdGlvbiDigJhldnRjaG5fYmluZF9waXJx
4oCZOgpjb21tb24vZXZlbnRfY2hhbm5lbC5jOjU5Nzo0OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhiaW5k4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTk3IHwgc3RhdGlj
IGludCBldnRjaG5fYmluZF9waXJxKGV2dGNobl9iaW5kX3BpcnFfdCAqYmluZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuLy4vaW5jbHVkZS94ZW4vY29uZmlnLmg6MTcsCiAg
ICAgICAgICAgICAgICAgZnJvbSA8Y29tbWFuZC1saW5lPjoKY29tbW9uL2V2ZW50X2NoYW5u
ZWwuYzogSW4gZnVuY3Rpb24g4oCYZXZ0Y2huX2Nsb3Nl4oCZOgpjb21tb24vZXZlbnRfY2hh
bm5lbC5jOjcyOTo2Mjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhldnRjaG5fcG9ydF904oCZIHtha2Eg
4oCYdW5zaWduZWQgaW504oCZfSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA3
MjkgfCAgICAgICAgIEFTU0VSVChyZWFkX2F0b21pYygmdi0+dmlycV90b19ldnRjaG5bY2hu
MS0+dS52aXJxXSkgPT0gcG9ydDEpOwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVu
L2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g4oCYdW5s
aWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGluX2V4cGVj
dCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIF4KY29tbW9uL2V2ZW50X2NoYW5uZWwuYzo3Mjk6OTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKAmQogIDcyOSB8ICAgICAgICAgQVNTRVJUKHJlYWRf
YXRvbWljKCZ2LT52aXJxX3RvX2V2dGNobltjaG4xLT51LnZpcnFdKSA9PSBwb3J0MSk7CiAg
ICAgIHwgICAgICAgICBefn5+fn4KY29tbW9uL2V2ZW50X2NoYW5uZWwuYzogSW4gZnVuY3Rp
b24g4oCYZXZ0Y2huX2NoZWNrX3BvbGxlcnPigJk6CmNvbW1vbi9ldmVudF9jaGFubmVsLmM6
MTUxODo1Njogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAxNTE4IHwgICAgICAgICBpZiAoICgodi0+cG9sbF9ldnRj
aG4gPD0gMCkgfHwgKHYtPnBvbGxfZXZ0Y2huID09IHBvcnQpKSAmJgogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgpj
b21tb24vZXZlbnRfY2hhbm5lbC5jOiBBdCB0b3AgbGV2ZWw6CmNvbW1vbi9ldmVudF9jaGFu
bmVsLmM6MTcxNDoxOiB3YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmlu
ZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAxNzE0IHwgX19p
bml0Y2FsbChkdW1wX2V2dGNobl9pbmZvX2tleV9pbml0KTsKICAgICAgfCBefn5+fn5+fn5+
CiAgQ0MgICAgICBhcmNoL2FybS9tbXUvcHQubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW5fcGFnZS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0v
bW11L3B0LmM6ODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9j
cHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9u
cl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAg
ICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAg
ICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxs
b2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tf
dmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192
YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVu
c2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1
Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8
IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKA
mHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVy
KGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6IEluIGZ1bmN0aW9uIOKAmHVu
bWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5o
OjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZh4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2b2lkIHVubWFwX2RvbWFpbl9wYWdl
X2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWNwaS5oOjI1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU4LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2FjcGkuaDo1NywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZml4bWFwLmg6NywKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL21tdS9wdC5jOjE1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwg
dm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVj
dCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVl
cnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlu
bGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhh
Y3BpX3NldF9jc3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxOTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xp
bWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+
fn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vi
c3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVu
c2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+
CiAgQ0MgICAgICBkcml2ZXJzL3Bhc3N0aHJvdWdoL2RldmljZV90cmVlLm8KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJl
bnQuaDo4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3Bl
cmNwdS5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2Vu
ZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
cndsb2NrLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNl
X3RyZWUuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvcGFzc3Rocm91Z2gv
ZGV2aWNlX3RyZWUuYzoxODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50
IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3J3bG9jay5oOjY6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIz
MDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7
CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMp
CiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNr
X3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBm
cmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo3OgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBp
bmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90
IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZ3Vlc3RfYWNjZXNzLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZ3Vlc3RfYWNjZXNzLmg6MTAsCiAgICAgICAgICAgICAg
ICAgZnJvbSBkcml2ZXJzL3Bhc3N0aHJvdWdoL2RldmljZV90cmVlLmM6MTk6Ci4vaW5jbHVk
ZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUv
eGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBt
Zm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21h
aW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAo
dW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwy
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRl
cuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBh
Z2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2Fj
Y2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJv
bV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEg
fCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3Qg
KnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lk
IGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlf
cG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6
NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2Rl
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGlu
ZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0
X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19w
dl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEx
MzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25z
dCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIGRyaXZlcnMvcGFzc3Rocm91Z2gvZGV2aWNlX3RyZWUuYzoyMzoKLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dldGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDIyOSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1
bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRl
ZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBz
dHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9t
Y3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X3N5c2N0bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDoyNjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25z
b2xlKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2
dGNobl9zZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4g
KmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2hu
X3Jlc2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozMDM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZ3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZ3JhbnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMyMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
cmFudF9xdWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21faW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6
NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9zZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9leGNoYW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbW9yeV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5Mzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlz
bWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9j
b25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBz
dGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4
OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVk
b3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFw
X2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21h
aW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRh
dGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFw
X2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBf
ZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lk
ICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4Mzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
aXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEs
IHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4
X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0
ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0
aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3Vy
Y2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVz
b3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3Bs
dWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vf
c2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np
KHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxp
bmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90
IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9o
eXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2bSh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9vcOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJfdCBv
cCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFp
bmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6
NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVm
YXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWlu
X3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgpkcml2ZXJzL3Bhc3N0aHJvdWdoL2RldmljZV90cmVlLmM6IEluIGZ1bmN0
aW9uIOKAmGlvbW11X2RvX2R0X2RvbWN0bOKAmToKZHJpdmVycy9wYXNzdGhyb3VnaC9kZXZp
Y2VfdHJlZS5jOjI2MTo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh1X2RvbWN0
bOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgWEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh4ZW5fZG9tY3RsX3QpIHVfZG9tY3RsKQogIEND
ICAgICAgYXJjaC9hcm0vbW11L3NldHVwLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0v
bW11L3NldHVwLmM6MTQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJ
biBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhj
cHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBj
cHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5y
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3
IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwo
c3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlv
biDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2Zp
bHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgt
dHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlv
biDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1
X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
NCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1f
ZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWlu
dDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZpeG1hcC5oOjcsCiAgICAg
ICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9tbXUvc2V0dXAuYzoxNToKLi9pbmNsdWRlL3hl
bi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9p
bmNsdWRlL3hlbi9hY3BpLmg6MTkyOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5ld19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBhY3BpX3NldF9jc3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkg
eyByZXR1cm47IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IElu
IGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hl
bi9hY3BpLmg6MTk0OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1p
dOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBh
Y3BpX3NldF9jc3Vic3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1
cm47IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KYXJjaC9hcm0vbW11L3NldHVwLmM6IEluIGZ1
bmN0aW9uIOKAmHNldHVwX3BhZ2V0YWJsZXPigJk6CmFyY2gvYXJtL21tdS9zZXR1cC5jOjI1
MjoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyNTIgfCAgICAgZm9yICggaSA9IDA7IGkgPCBYRU5f
TlJfRU5UUklFUygzKTsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgphcmNo
L2FybS9tbXUvc2V0dXAuYzoyNzI6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjcyIHwgICAgIGZv
ciAoIGkgPSAwOyBpIDwgWEVOX05SX0VOVFJJRVMoMik7IGkrKyApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgIF4KICBMRCAgICAgIGRyaXZlcnMvcGFzc3Rocm91Z2gvYnVpbHRfaW4u
bwogIENDICAgICAgYXJjaC9hcm0vbW11L3NtcGJvb3QubwpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gYXJj
aC9hcm0vbW11L3NtcGJvb3QuYzo4OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KICBDQyAg
ICAgIHhzbS9mbGFzay9zcy9jb25kaXRpb25hbC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20geHNt
L2ZsYXNrL3NzL2NvbmRpdGlvbmFsLmM6MTY6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMg
aW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGly
ZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVu
c2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9F
TkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAg
IHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgs
IHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29u
c3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcy
IHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8g
KnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAg
ICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjEx
OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdl
Lmg6IEluIGZ1bmN0aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNs
dWRlL3hlbi9kb21haW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHZh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2
b2lkIHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVl
Lmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nl
dHVwLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL21tdS9zbXBib290LmM6
MTA6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
bG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwg
c3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9s
b2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJv
b2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9z
dCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwg
c3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZj
cHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBp
ZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
Z2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYx
OjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4s
IHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9l
dmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vbW11L3Nt
cGJvb3QuYzogSW4gZnVuY3Rpb24g4oCYcHJlcGFyZV9zZWNvbmRhcnlfbW3igJk6CmFyY2gv
YXJtL21tdS9zbXBib290LmM6OTc6MzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Y3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDk3IHwgaW50IHByZXBhcmVfc2Vjb25k
YXJ5X21tKGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5e
fn4KeHNtL2ZsYXNrL3NzL2NvbmRpdGlvbmFsLmM6IEluIGZ1bmN0aW9uIOKAmGNvbmRfZGVz
dHJveV9ib29s4oCZOgp4c20vZmxhc2svc3MvY29uZGl0aW9uYWwuYzoxOTM6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5
MyB8IGludCBjZl9jaGVjayBjb25kX2Rlc3Ryb3lfYm9vbCh2b2lkICprZXksIHZvaWQgKmRh
dHVtLCB2b2lkICpwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCnhzbS9mbGFzay9zcy9jb25kaXRpb25h
bC5jOiBJbiBmdW5jdGlvbiDigJhjb25kX3JlYWRfYm9vbOKAmToKeHNtL2ZsYXNrL3NzL2Nv
bmRpdGlvbmFsLmM6MjI0OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHDigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCBpbnQgY2ZfY2hlY2sgY29uZF9yZWFkX2Jv
b2woc3RydWN0IHBvbGljeWRiICpwLCBzdHJ1Y3QgaGFzaHRhYiAqaCwgdm9pZCAqZnApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xgp4
c20vZmxhc2svc3MvY29uZGl0aW9uYWwuYzogSW4gZnVuY3Rpb24g4oCYY29uZF9pbnNlcnRm
4oCZOgp4c20vZmxhc2svc3MvY29uZGl0aW9uYWwuYzoyNzM6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYYeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICBz
dHJ1Y3QgYXZ0YWIgKmEsIHN0cnVjdCBhdnRhYl9rZXkgKmssIHN0cnVjdCBhdnRhYl9kYXR1
bSAqZCwgdm9pZCAqcHRyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl4KeHNtL2ZsYXNr
L3NzL2NvbmRpdGlvbmFsLmM6IEluIGZ1bmN0aW9uIOKAmGNvbmRfcmVhZF9hdl9saXN04oCZ
Ogp4c20vZmxhc2svc3MvY29uZGl0aW9uYWwuYzozODk6MjA6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24t
Y29tcGFyZV0KICAzODkgfCAgICAgZm9yICggaSA9IDA7IGkgPCBsZW47IGkrKyApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgIF4KICBMRCAgICAgIGFyY2gvYXJtL21tdS9idWlsdF9p
bi5vCiAgQ0MgICAgICBkcml2ZXJzL2FjcGkvdGFibGVzL3RidXRpbHMubwogIENDICAgICAg
YXJjaC9hcm0vYWNwaS9saWIubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU0LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2
ZXJzL2FjcGkvdGFibGVzL3RidXRpbHMuYzo0NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9j
cHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRp
YyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0u
aDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2FjcGkuaDo1NywKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL2FjcGkvbGliLmM6MjQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5l
IGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZv
aWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdl
X2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlw
ZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJ
biBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FW
RV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0
Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAg
IHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6
MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAg
IHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KZHJpdmVycy9hY3BpL3Rh
Ymxlcy90YnV0aWxzLmM6IEluIGZ1bmN0aW9uIOKAmGFjcGlfdGJfY2hlY2tfeHNkdOKAmToK
ZHJpdmVycy9hY3BpL3RhYmxlcy90YnV0aWxzLmM6OTU6MjM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24t
Y29tcGFyZV0KICAgOTUgfCAgICAgICAgIGZvciAoaSA9IDA7IGkgPCB0YWJsZV9jb3VudDsg
aSsrKSB7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIF4KZHJpdmVycy9hY3BpL3Rh
Ymxlcy90YnV0aWxzLmM6MTA1OjE1OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAgMTA1
IHwgICAgICAgICBpZiAoaSA8IHRhYmxlX2NvdW50KQogICAgICB8ICAgICAgICAgICAgICAg
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNf
aW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3Rt
YXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAg
ICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFn
ZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFk
X2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21l
bV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1f
YWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291
cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lk
IHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR4
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRv
bWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90
IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlf
c2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTkyOjU1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3RhdGVfbGltaXQo
dW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+fn5+fn4K
Li9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdWJzdGF0
ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTk0OjU4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5
NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Vic3RhdGVfbGltaXQodW5zaWdu
ZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KYXJj
aC9hcm0vYWNwaS9saWIuYzogSW4gZnVuY3Rpb24g4oCYYWNwaV9nZXRfdGFibGVfb2Zmc2V0
4oCZOgphcmNoL2FybS9hY3BpL2xpYi5jOjEyMzoyMDogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmEVGSV9NRU1fUkVT4oCZIFstV3NpZ24tY29tcGFyZV0KICAxMjMgfCAgICAg
Zm9yICggaSA9IDA7IGkgPCBpbmRleDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgXgogIENDICAgICAgYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQubwpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDox
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAg
IGZyb20gYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzoxMzoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEw
MSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+CiAgQ0MgICAgICB4c20vZmxhc2svc3MvbWxzLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxp
bmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNp
emUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAg
ICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAg
ICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhh
bGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFz
a192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNr
X3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSB4c20vZmxhc2svc3Mvc2lkdGFiLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSB4
c20vZmxhc2svc3MvbWxzLmM6MjE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5l
IGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEw
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQg
bG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVk
IGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFn
ZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+fn5+fn5+fn5+fgogIENDICAgICAgZHJpdmVycy9hY3BpL3RhYmxlcy90
YmZhZHQubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAg
ICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzoxNDoKLi9p
bmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5j
bHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVp
ZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIw
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
dW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19l
bXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSB4c20vZmxhc2svc3MvY29udGV4dC5oOjIxLAogICAgICAgICAgICAg
ICAgIGZyb20geHNtL2ZsYXNrL3NzL3NpZHRhYi5oOjEzOgp4c20vZmxhc2svc3MvbWxzLmM6
IEluIGZ1bmN0aW9uIOKAmG1sc19jb21wdXRlX2NvbnRleHRfbGVu4oCZOgp4c20vZmxhc2sv
c3MvZWJpdG1hcC5oOjExOToxNTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUz
MuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDExOSB8
ICAgICAgICAgICBiaXQgPCBlYml0bWFwX2xlbmd0aChlKTsgICAgICAgICAgICAgICAgICAg
XAogICAgICB8ICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3MvbWxzLmM6NTA6OTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGViaXRtYXBfZm9yX2VhY2hfcG9zaXRpdmVf
Yml04oCZCiAgIDUwIHwgICAgICAgICBlYml0bWFwX2Zvcl9lYWNoX3Bvc2l0aXZlX2JpdChl
LCBub2RlLCBpKQogICAgICB8ICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn4KeHNtL2ZsYXNrL3NzL21scy5jOiBJbiBmdW5jdGlvbiDigJhtbHNfc2lkX3RvX2NvbnRl
eHTigJk6CnhzbS9mbGFzay9zcy9lYml0bWFwLmg6MTE5OjE1OiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWdu
LWNvbXBhcmVdCiAgMTE5IHwgICAgICAgICAgIGJpdCA8IGViaXRtYXBfbGVuZ3RoKGUpOyAg
ICAgICAgICAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICBeCnhzbS9mbGFzay9z
cy9tbHMuYzoxMTU6OTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGViaXRtYXBf
Zm9yX2VhY2hfcG9zaXRpdmVfYml04oCZCiAgMTE1IHwgICAgICAgICBlYml0bWFwX2Zvcl9l
YWNoX3Bvc2l0aXZlX2JpdChlLCBub2RlLCBpKQogICAgICB8ICAgICAgICAgXn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn4KeHNtL2ZsYXNrL3NzL21scy5jOiBJbiBmdW5jdGlvbiDi
gJhtbHNfbGV2ZWxfaXN2YWxpZOKAmToKeHNtL2ZsYXNrL3NzL2ViaXRtYXAuaDoxMTk6MTU6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25l
ZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICAxMTkgfCAgICAgICAgICAgYml0IDwgZWJp
dG1hcF9sZW5ndGgoZSk7ICAgICAgICAgICAgICAgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgIF4KeHNtL2ZsYXNrL3NzL21scy5jOjE4Mzo1OiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYZWJpdG1hcF9mb3JfZWFjaF9wb3NpdGl2ZV9iaXTigJkKICAxODMgfCAgICAg
ZWJpdG1hcF9mb3JfZWFjaF9wb3NpdGl2ZV9iaXQoJmwtPmNhdCwgbm9kZSwgaSkKICAgICAg
fCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgp4c20vZmxhc2svc3MvbWxzLmM6MTg1OjE2OiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdTMy4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBh
cmVdCiAgMTg1IHwgICAgICAgICBpZiAoIGkgPiBwLT5wX2NhdHMubnByaW0gKQogICAgICB8
ICAgICAgICAgICAgICAgIF4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2Fs
dHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0
IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpk
LCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fl4KeHNtL2ZsYXNrL3NzL21scy5jOiBJbiBmdW5jdGlvbiDi
gJhtbHNfY29udGV4dF90b19zaWTigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6CnhzbS9mbGFz
ay9zcy9tbHMuYzozMzc6NTA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1MzLi
gJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICAzMzcgfCAg
ICAgICAgICAgICAgICAgICAgIGZvciAoIGkgPSBjYXRkYXR1bS0+dmFsdWU7IGkgPCBybmdk
YXR1bS0+dmFsdWU7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIF4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0
MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgp4c20vZmxhc2svc3Mv
bWxzLmM6MjUwOjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHPigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNTAgfCAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBj
b250ZXh0ICpjb250ZXh0LCBzdHJ1Y3Qgc2lkdGFiICpzKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4K
eHNtL2ZsYXNrL3NzL21scy5jOiBJbiBmdW5jdGlvbiDigJhtbHNfY29udmVydF9jb250ZXh0
4oCZOgp4c20vZmxhc2svc3MvZWJpdG1hcC5oOjExOToxNTogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1j
b21wYXJlXQogIDExOSB8ICAgICAgICAgICBiaXQgPCBlYml0bWFwX2xlbmd0aChlKTsgICAg
ICAgICAgICAgICAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgXgp4c20vZmxhc2svc3Mv
bWxzLmM6NDI2Ojk6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhlYml0bWFwX2Zv
cl9lYWNoX3Bvc2l0aXZlX2JpdOKAmQogIDQyNiB8ICAgICAgICAgZWJpdG1hcF9mb3JfZWFj
aF9wb3NpdGl2ZV9iaXQoJmMtPnJhbmdlLmxldmVsW2xdLmNhdCwgbm9kZSwgaSkKICAgICAg
fCAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9s
aWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fz
c2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGlu
ZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAq
dikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVu
Y3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJp
dF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xp
bnV4Lmg6NTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9y
bS9hY2Vudi5oOjE0MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL2Fj
cGkuaDo1NiwKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvYWNwaS90YWJsZXMvdGJm
YWR0LmM6NDU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5j
dGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJf
c2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzoxNToKLi9pbmNsdWRlL3hlbi9hY3Bp
Lmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRl
L3hlbi9hY3BpLmg6MTkyOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19s
aW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCBhY3BpX3NldF9jc3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1
cm47IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0
aW9uIOKAmGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3Bp
Lmg6MTk0OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3Nl
dF9jc3Vic3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vZXZlbnQuaDoxNywKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2FjcGkvZG9t
YWluX2J1aWxkLmM6MTY6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVu
Y3Rpb24g4oCYYXJjaF92aXJxX2lzX2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9ldmVudC5oOjUwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92
aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6NzAsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3NldHVwLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYWNwaS5oOjI1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU4OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVf
aWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVf
aWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZv
aWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CmFyY2gv
YXJtL2FjcGkvZG9tYWluX2J1aWxkLmM6IEluIGZ1bmN0aW9uIOKAmGFjcGlfcm91dGVfc3Bp
c+KAmToKYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzo3MjozMTogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
IDcyIHwgICAgIGZvciggaSA9IE5SX0xPQ0FMX0lSUVM7IGkgPCB2Z2ljX251bV9pcnFzKGQp
OyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KYXJjaC9h
cm0vYWNwaS9kb21haW5fYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYYWNwaV9tYXBfb3RoZXJf
dGFibGVz4oCZOgphcmNoL2FybS9hY3BpL2RvbWFpbl9idWlsZC5jOjE5NDoxOTogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHUzMuKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKA
mX0gWy1Xc2lnbi1jb21wYXJlXQogIDE5NCB8ICAgICBmb3IoIGkgPSAwOyBpIDwgYWNwaV9n
Ymxfcm9vdF90YWJsZV9saXN0LmNvdW50OyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgIF4KYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYYWNw
aV94c2R0X21vZGlmeV9lbnRyeeKAmToKYXJjaC9hcm0vYWNwaS9kb21haW5fYnVpbGQuYzoy
NTQ6MTk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjU0IHwgICAgIGZvciggaSA9IDA7IGkgPCBlbnRy
eV9jb3VudDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICBeCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
ZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xp
c3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4g
ZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJm
bHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNw
dW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlv
biDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVh
ZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4g
ZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5s
b2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVs
YXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMx
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJj
ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3
NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUg
KnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNp
Z25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1f
ZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgpk
cml2ZXJzL2FjcGkvdGFibGVzL3RiZmFkdC5jOiBBdCB0b3AgbGV2ZWw6CmRyaXZlcnMvYWNw
aS90YWJsZXMvdGJmYWR0LmM6NTI6MTogd2FybmluZzog4oCYaW5saW5l4oCZIGlzIG5vdCBh
dCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQog
ICA1MiB8IHN0YXRpYyB2b2lkIGlubGluZQogICAgICB8IF5+fn5+fgpkcml2ZXJzL2FjcGkv
dGFibGVzL3RiZmFkdC5jOjEyODoxOiB3YXJuaW5nOiDigJhpbmxpbmXigJkgaXMgbm90IGF0
IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAg
MTI4IHwgc3RhdGljIHZvaWQgaW5saW5lCiAgICAgIHwgXn5+fn5+CiAgQ0MgICAgICBkcml2
ZXJzL2FjcGkvdGFibGVzL3RiaW5zdGFsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjZW52Lmg6MTQyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvYWNwaS5oOjU2LAogICAgICAgICAgICAgICAgIGZy
b20gZHJpdmVycy9hY3BpL3RhYmxlcy90Ymluc3RhbC5jOjQ1OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAx
IHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6
MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVf
RklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZ
UEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwLAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zZXR1cC5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FjcGkuaDoyNSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1ODoKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAg
ICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAg
IGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlu
bGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1
IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5o
OjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgogIENDICAgICAgYXJjaC9hcm0vYWNwaS9ib290Lm8KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291
cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJt
X2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVu
c2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92
bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xp
bnV4Lmg6NTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9y
bS9hY2Vudi5oOjE0MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL2Fj
cGkuaDo1NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vYWNwaS5oOjU3
LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vYWNwaS9ib290LmM6Mjg6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
c3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCY
X19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzAsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWNwaS5oOjI1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5o
OjU4OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19s
YXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIz
MCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAg
IF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAq
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3By
ZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9s
aXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6
Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1
c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRf
cGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJt
X3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIENDICAgICAgZHJpdmVycy9h
Y3BpL3RhYmxlcy90YnhmYWNlLm8KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9u
IOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTky
OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Rh
dGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0
X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTk0OjU4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Vic3RhdGVfbGlt
aXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL2FjcGkvYm9vdC5jOjMyOgou
L2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKA
mToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDi
gJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhz
aXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVk
bmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50
IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwog
ICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4K
YXJjaC9hcm0vYWNwaS9ib290LmM6IEluIGZ1bmN0aW9uIOKAmGFjcGlfbWFwX2dpY19jcHVf
aW50ZXJmYWNl4oCZOgphcmNoL2FybS9hY3BpL2Jvb3QuYzo5NjoyMDogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
IDk2IHwgICAgIGZvciAoIGkgPSAxOyBpIDwgZW5hYmxlZF9jcHVzOyBpKysgKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJtL2FjcGkvYm9vdC5jOiBJbiBmdW5jdGlv
biDigJhhY3BpX3NtcF9pbml0X2NwdXPigJk6CmFyY2gvYXJtL2FjcGkvYm9vdC5jOjE2Njoy
MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMTY2IHwgICAgIGZvciAoIGkgPSAwOyBpIDwgZW5hYmxlZF9jcHVz
OyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJtL2FjcGkvYm9v
dC5jOiBJbiBmdW5jdGlvbiDigJhkdF9zY2FuX2RlcHRoMV9ub2Rlc+KAmToKYXJjaC9hcm0v
YWNwaS9ib290LmM6MjEzOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGZkdOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIxMyB8IHN0YXRpYyBpbnQgX19pbml0IGR0X3Nj
YW5fZGVwdGgxX25vZGVzKGNvbnN0IHZvaWQgKmZkdCwgaW50IG5vZGUsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn5+CmFy
Y2gvYXJtL2FjcGkvYm9vdC5jOjIxMzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhub2Rl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjEzIHwgc3RhdGljIGludCBfX2lu
aXQgZHRfc2Nhbl9kZXB0aDFfbm9kZXMoY29uc3Qgdm9pZCAqZmR0LCBpbnQgbm9kZSwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5efn5+CmFyY2gvYXJtL2FjcGkvYm9vdC5jOjIxNTo0NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhhZGRyZXNzX2NlbGxz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjE1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdTMy
IGFkZHJlc3NfY2VsbHMsIHUzMiBzaXplX2NlbGxzLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn5+fn5+fn5+fn5+CmFyY2gvYXJtL2Fj
cGkvYm9vdC5jOjIxNTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXplX2Nl
bGxz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjE1IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdTMyIGFkZHJlc3NfY2VsbHMsIHUzMiBzaXplX2NlbGxz
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+Xn5+fn5+fn5+fgphcmNoL2FybS9hY3BpL2Jvb3QuYzoyMTY6
NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDIxNiB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2Fj
cGkvcGxhdGZvcm0vYWNsaW51eC5oOjU0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJvbSBkcml2ZXJz
L2FjcGkvdGFibGVzL3RieGZhY2UuYzo0NjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5z
X2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEw
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9p
bmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAg
ICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJm
bHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9t
ZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNv
dXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlk
eOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBk
b21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlf
dCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCiAgSU5JVF9PICBhcmNoL2FybS9hY3BpL2RvbWFpbl9idWlsZC5pbml0
Lm8KICBDQyAgICAgIGRyaXZlcnMvYWNwaS90YWJsZXMvdGJ4ZnJvb3QubwogIElOSVRfTyAg
YXJjaC9hcm0vYWNwaS9ib290LmluaXQubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJv
bSBkcml2ZXJzL2FjcGkvdGFibGVzL3RieGZyb290LmM6NDU6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoy
MSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9G
SVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVD
TEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVO
RCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+
fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAg
IHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgs
IHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzAsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWNwaS5oOjI1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU4OgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAg
ICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAg
aWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUg
fCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+CiAgTEQgICAgICB4c20vZmxhc2svc3MvYnVpbHRfaW4ubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0
bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFn
ZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVu
Y3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEy
MDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1
X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFk
X3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KICBMRCAgICAgIGFy
Y2gvYXJtL2FjcGkvYnVpbHRfaW4ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9v
bCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0
ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhn
ZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6
Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwg
cDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIElOSVRfTyAgZHJpdmVy
cy9hY3BpL3RhYmxlcy90YmZhZHQuaW5pdC5vCiAgSU5JVF9PICBkcml2ZXJzL2FjcGkvdGFi
bGVzL3RiaW5zdGFsLmluaXQubwogIENDICAgICAgY29tbW9uL2V2ZW50X2ZpZm8ubwogIExE
ICAgICAgeHNtL2ZsYXNrL2J1aWx0X2luLm8KICBJTklUX08gIGRyaXZlcnMvYWNwaS90YWJs
ZXMvdGJ4ZmFjZS5pbml0Lm8KICBDQyAgICAgIGFyY2gvYXJtL3BsYXRmb3Jtcy9zZWF0dGxl
Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJu
YXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90
aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjEyLAogICAgICAg
ICAgICAgICAgIGZyb20gY29tbW9uL2V2ZW50X2NoYW5uZWwuaDozLAogICAgICAgICAgICAg
ICAgIGZyb20gY29tbW9uL2V2ZW50X2ZpZm8uYzoxMDoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0
YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1
NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2
b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3Rh
dGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWlu
dDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVD
TEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVO
RCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVy
bmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
dGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9j
ay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcGxhdGZvcm0u
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vcGxhdGZvcm1zL3NlYXR0bGUu
YzoyMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJs
aW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVu
Y3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIElOSVRfTyAgZHJpdmVycy9hY3BpL3Rh
Ymxlcy90Ynhmcm9vdC5pbml0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19u
aWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMg
aW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAg
ICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAg
ICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhh
bGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFz
a192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNr
X3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVu
c2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1
Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8
IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKA
mHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVy
KGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+
fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVl
Lmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIw
IHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICps
b2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVh
ZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KICBMRCAgICAgIGRyaXZlcnMvYWNwaS90YWJs
ZXMvYnVpbHRfaW4ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1f
YWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVt
X2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk
4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlu
bGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25n
IGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlz
dF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBm
dW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Zmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1
bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
cmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2ln
bmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9l
dmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNw
dV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUg
KnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlv
biDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVz
dF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5l
IGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWdu
X2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYg
KnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRh
dGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9j
ZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwg
c3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhz
dHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5
X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Z21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5
ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBi
b29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZl
bnQuaDoxNzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50
Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNf
Z2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25f
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVh
ZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMg
aW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0
aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWdu
ZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL2V2ZW50
X2ZpZm8uYzoxODoKLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDi
gJh1bm1hcF9kb21haW5fcGFnZV9nbG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3Bh
Z2UuaDo3MTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA3MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5f
cGFnZV9nbG9iYWwoY29uc3Qgdm9pZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lu
bGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fl4KY29tbW9uL2V2ZW50X2ZpZm8uYzogSW4gZnVuY3Rpb24g
4oCYZXZ0Y2huX2ZpZm9fc2V0X3ByaW9yaXR54oCZOgpjb21tb24vZXZlbnRfZmlmby5jOjM4
MDoyMDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzgwIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmV2
dGNobiwgdW5zaWduZWQgaW50IHByaW9yaXR5KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fn5eCmNvbW1vbi9ldmVudF9maWZvLmM6IEluIGZ1bmN0aW9uIOKAmGluaXRfcXVldWXigJk6
CmNvbW1vbi9ldmVudF9maWZvLmM6NDYwOjM3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCBzdGF0aWMgdm9pZCBpbml0
X3F1ZXVlKHN0cnVjdCB2Y3B1ICp2LCBzdHJ1Y3QgZXZ0Y2huX2ZpZm9fcXVldWUgKnEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgogIExEICAgICAg
eHNtL2J1aWx0X2luLm8KICBDQyAgICAgIGFyY2gvYXJtL3BsYXRmb3Jtcy9zdW54aS5vCiAg
Q0MgICAgICBkcml2ZXJzL2FjcGkvdXRpbGl0aWVzL3V0Z2xvYmFsLm8KICBDQyAgICAgIGRy
aXZlcnMvYWNwaS91dGlsaXRpZXMvdXRtaXNjLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjZW52Lmg6MTQyLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvYWNwaS5oOjU2LAogICAgICAgICAgICAgICAg
IGZyb20gZHJpdmVycy9hY3BpL3V0aWxpdGllcy91dGdsb2JhbC5jOjQ5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxp
bmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGlu
dCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3BsYXRmb3Jtcy9zdW54aS5j
OjE5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g
4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxp
bmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1
bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IElu
IGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjcwOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3
MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hY3Bp
Lmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9h
Y2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAg
ICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAg
ICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFs
bG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwg
cGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1
ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCY
dGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIo
Y3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUu
aDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBw
YWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFn
ZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxi
Zmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6
NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1
bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Bs
YXRmb3JtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3BsYXRmb3Jtcy9z
dW54aS5jOjIxOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3Rv
X25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRp
YyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwg
dm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2Vudi5oOjE0MiwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL2FjcGkuaDo1NiwKICAgICAg
ICAgICAgICAgICBmcm9tIGRyaXZlcnMvYWNwaS91dGlsaXRpZXMvdXRtaXNjLmM6NDU6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVf
bnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEw
MTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5z
aWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25l
ZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFU
X0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAg
ICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAq
aCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1f
U0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3Ry
dWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBp
bmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90
IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVf
SFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAs
IHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3Zj
cHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1
ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBk
YXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vl
c3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGlu
ZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2ln
bl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwg
c3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2
ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUo
cGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
YXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJv
Y2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmco
c3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgogIENDICAgICAgYXJjaC9h
cm0vcGxhdGZvcm1zL3RodW5kZXJ4Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVy
ZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zZXR1cC5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FjcGkuaDoy
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGlu
dXguaDo1ODoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAg
ICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAg
ICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2Nf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2
NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IElu
IGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZf
MzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5o
OjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0
IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVy
eV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wbGF0Zm9ybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9wbGF0Zm9y
bXMvdGh1bmRlcnguYzoyMToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50
IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CmRyaXZlcnMvYWNwaS91
dGlsaXRpZXMvdXRtaXNjLmM6IEluIGZ1bmN0aW9uIOKAmGFjcGlfdXRfaW5mb+KAmToKZHJp
dmVycy9hY3BpL3V0aWxpdGllcy91dG1pc2MuYzoxNjQ6MjY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbW9kdWxlX25hbWXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNjQg
fCBhY3BpX3V0X2luZm8oY29uc3QgY2hhciAqbW9kdWxlX25hbWUsIHUzMiBsaW5lX251bWJl
ciwgY29uc3QgY2hhciAqZm9ybWF0LCAuLi4pCiAgICAgIHwgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fl5+fn5+fn5+fn5+CmRyaXZlcnMvYWNwaS91dGlsaXRpZXMvdXRtaXNjLmM6MTY0
OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxpbmVfbnVtYmVy4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTY0IHwgYWNwaV91dF9pbmZvKGNvbnN0IGNoYXIgKm1vZHVs
ZV9uYW1lLCB1MzIgbGluZV9udW1iZXIsIGNvbnN0IGNoYXIgKmZvcm1hdCwgLi4uKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+fn5+fn5+
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNM
QVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5E
LCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nt
cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1
IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQg
PT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29s
IGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGlj
IGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1
ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3Qg
bWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl5+fgogIElOSVRfTyAgZHJpdmVycy9hY3BpL3V0aWxpdGllcy91
dG1pc2MuaW5pdC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDox
NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
bmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4
IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGlt
ZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2
b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGlt
ZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlu
bGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5o
OjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g
4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3Nf
ZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29s
IHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1
aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRw
Mm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7i
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAg
c3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAy
bV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KICBMRCAgICAgIGRyaXZlcnMvYWNwaS91dGlsaXRpZXMv
YnVpbHRfaW4ubwogIENDICAgICAgZHJpdmVycy9hY3BpL29zbC5vCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9s
aWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGFy
Y2gvYXJtL3BsYXRmb3Jtcy94Z2VuZS1zdG9ybS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3
ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNl
KHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
ZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5l
IHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5s
aW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwg
c3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211
c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlz
X3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
MTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNv
bnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L2lvLmg6
MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvLmg6
NywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvYWNwaS9vc2wuYzoyNjoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcGxhdGZvcm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vcGxhdGZv
cm1zL3hnZW5lLXN0b3JtLmM6MjE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5l
IGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2Vudi5oOjE0MiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL2FjcGkuaDo1NiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vYWNwaS5oOjU3LAogICAgICAgICAgICAgICAgIGZy
b20gZHJpdmVycy9hY3BpL29zbC5jOjMxOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUg
aW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUp
IFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4
NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQp
OwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9T
QVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxp
bmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNp
emUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9tbS5oOjcwLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9zZXR1cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FjcGkuaDoyNSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3Bp
L3BsYXRmb3JtL2FjbGludXguaDo1ODoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMw
OjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsK
ICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykK
ICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZy
ZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2xhc3TigJk6CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAg
ICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQw
OjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAg
ICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUy
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9j
cHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlz
dF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBm
dW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Zmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1
bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
cmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2ln
bmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9l
dmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251
bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9f
bmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDox
NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
bmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4
IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3RhdGVfbGltaXTigJk6
Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTIgfCBzdGF0aWMg
aW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGlt
aXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5o
OiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3RhdGVfbGltaXTigJk6Ci4vaW5jbHVk
ZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdf
bGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsg
cmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBk
cml2ZXJzL2FjcGkvb3NsLmM6MzI6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlv
biDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gZHJpdmVycy9hY3BpL29zbC5jOjM3OgouL2luY2x1ZGUveGVuL2RvbWFp
bl9wYWdlLmg6IEluIGZ1bmN0aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToK
Li9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHZh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlu
bGluZSB2b2lkIHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5efgpkcml2ZXJzL2FjcGkvb3NsLmM6IEluIGZ1bmN0aW9uIOKAmGFjcGlf
b3Nfd3JpdGVfcG9ydOKAmToKZHJpdmVycy9hY3BpL29zbC5jOjE0NDo1ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh2YWx1ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE0
NCB8IGFjcGlfc3RhdHVzIGFjcGlfb3Nfd3JpdGVfcG9ydChhY3BpX2lvX2FkZHJlc3MgcG9y
dCwgdTMyIHZhbHVlLCB1MzIgd2lkdGgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3Mu
aDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJz
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNx
dWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0
cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBp
ZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2Fs
dHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAg
IHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBw
Mm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3Rh
dGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNo
X3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0
IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2ln
bl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2Jk
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWlu
dDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0
ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUs
IHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwg
dWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVj
ayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJi
aXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwg
c3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2Fy
bS9wbGF0Zm9ybXMvYnJjbS1yYXNwYmVycnktcGkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9h
cm0vcGxhdGZvcm1zL2JyY20tcmFzcGJlcnJ5LXBpLmM6MjE6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgogIENDICAgICAgYXJjaC9hcm0vcGxhdGZvcm1zL2lteDhxbS5vCiAgQ0MgICAgICBk
cml2ZXJzL2FjcGkvdGFibGVzLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9u
IOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVy
ZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAg
ICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAg
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNs
ZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAg
fCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJl
ZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7i
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9wbGF0Zm9ybXMv
aW14OHFtLmM6MTQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBm
dW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVf
bnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBkcml2ZXJzL2FjcGkvdGFibGVz
LmM6Mjc6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmln
X3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVy
YW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZ
IHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8g
dW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8
ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJs
ZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wbGF0Zm9ybS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9wbGF0Zm9ybXMvYnJjbS1yYXNwYmVy
cnktcGkuYzoyNDoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90
b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0
aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlu
bGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0
aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50
MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FW
RV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0
IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gZHJpdmVycy9hY3BpL3RhYmxlcy5jOjI4OgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAg
ICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYg
KG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nl
c3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNj
ZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo2OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3Rh
dGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1
aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3
NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJj
ZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21h
aW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBx
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBi
b29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0
bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwu
aDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDozNiwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIGRyaXZlcnMvYWNwaS90YWJsZXMuYzozMToKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E
4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1f
U0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3Ry
dWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lk
czsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lk
cykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lk
IGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxv
Y2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9t
YWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDox
MToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6
Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5l
IG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVz
dF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDox
NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NldHVwLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWNwaS5o
OjI1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNs
aW51eC5oOjU4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvcGxhdGZv
cm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9h
Y3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2FjcGkuaDo1
NywKICAgICAgICAgICAgICAgICBmcm9tIGRyaXZlcnMvYWNwaS90YWJsZXMuYzozMzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9k
b21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0
aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8g
KnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2Rl
bDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
MSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQy
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAq
cGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVs
MihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQx
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0
ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
YWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBw
YWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90
aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3Mu
aDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nl
c3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g
4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAy
bV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDoz
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50
MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQg
cDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zy
b21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYx
IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90
ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3
IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhn
ZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6
Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwg
cDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL2Fj
cGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRfY3N0YXRlX2xpbWl04oCZOgouL2luY2x1
ZGUveGVuL2FjcGkuaDoxOTI6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3
X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTkyIHwgc3RhdGljIGlubGluZSB2
b2lkIGFjcGlfc2V0X2NzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJl
dHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVu
Y3Rpb24g4oCYYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl04oCZOgouL2luY2x1ZGUveGVuL2Fj
cGkuaDoxOTQ6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk0IHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlf
c2V0X2NzdWJzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJldHVybjsg
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fn5+fn5+fgpkcml2ZXJzL2FjcGkvdGFibGVzLmM6IEluIGZ1bmN0
aW9uIOKAmGFjcGlfZ2V0X2VudHJ54oCZOgpkcml2ZXJzL2FjcGkvdGFibGVzLmM6MjcyOjM1
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAyNzIgfCAgICAgICAgICAgICAgICAgICAgICAgICBpZiAoY291bnQg
PT0gZW50cnlfaW5kZXgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF5+CmRyaXZlcnMvYWNwaS90YWJsZXMuYzogSW4gZnVuY3Rpb24g4oCYYWNwaV9wYXJz
ZV9lbnRyaWVz4oCZOgpkcml2ZXJzL2FjcGkvdGFibGVzLmM6MzQyOjQ3OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAzNDIgfCAgICAgICAgICAgICAgICAgICAgICYmICghbWF4X2VudHJpZXMgfHwgY291bnQg
PCBtYXhfZW50cmllcykpIHsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgXgpkcml2ZXJzL2FjcGkvdGFibGVzLmM6MzUzOjM0OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAzNTMgfCAgICAgICAgIGlmIChtYXhfZW50cmllcyAmJiBjb3VudCA+IG1heF9l
bnRyaWVzKSB7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3Zj
cHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1
ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBk
YXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vl
c3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
CiAgQ0MgICAgICBhcmNoL2FybS9wbGF0Zm9ybXMveGlsaW54LXp5bnFtcC5vCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBj
aV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlu
bGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7
IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcg
fCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5k
aW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZj
cHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IElu
IGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZf
MzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3BsYXRmb3Jt
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3BsYXRmb3Jtcy94aWxpbngt
enlucW1wLmM6MjA6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBm
dW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVf
bnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIElOSVRfTyAgZHJpdmVy
cy9hY3BpL3RhYmxlcy5pbml0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2Nr
Lmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3Ogou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAg
ICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAg
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNs
ZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAg
fCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJl
ZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn5+CiAgTEQgICAgICBkcml2ZXJzL2FjcGkvYnVpbHRfaW4ubwpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVu
L251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9u
dW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3Rv
X25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl5+fgogIExEICAgICAgZHJpdmVycy9idWlsdF9pbi5vCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9y
YWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hl
Y2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWdu
ZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1
X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAq
dikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9u
IOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0
ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0
X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUg
aW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25f
ZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0
YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nl
c3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0
cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29s
IGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fl4KICBDQyAgICAgIGFyY2gvYXJtL3BsYXRmb3Jtcy94aWxpbngtenlucW1w
LWVlbWkubwogIENDICAgICAgY29tbW9uL2dyYW50X3RhYmxlLm8KICBDQyAgICAgIGNvbW1v
bi9ndWVzdGNvcHkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3JlZ3MuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9t
IGFyY2gvYXJtL3BsYXRmb3Jtcy94aWxpbngtenlucW1wLWVlbWkuYzoxOToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAog
ICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vcGxhdGZvcm1zL3hpbGlueC16eW5xbXAt
ZWVtaS5jOjIwOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKA
mF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9p
b2NhcC5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2dyYW50X3RhYmxlLmM6
Mjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGlu
Z3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
Ojc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
cHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0
aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xB
UkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6
Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1m
bl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBz
dGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2Rl
bChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0
aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hf
ZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRp
eC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0
aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEz
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhy
Y3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2
bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQo
bWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVj
dCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4g
ZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMx
MCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1
Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlu
bGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFz
c2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgz
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3Jp
dGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUo
cGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
cHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRp
bmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19l
bXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNv
dXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZj
cHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1
bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
dm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5h
dGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Rp
bWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1ZXN0X2FjY2Vz
cy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1ZXN0X2FjY2Vz
cy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2d1ZXN0Y29weS5jOjI6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVf
bnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEw
MTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5z
aWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9u
IOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6
Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9t
YWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNo
L2FybS9wbGF0Zm9ybXMveGlsaW54LXp5bnFtcC1lZW1pLmM6IEluIGZ1bmN0aW9uIOKAmGRv
bWFpbl9oYXNfbm9kZV9hY2Nlc3PigJk6CmFyY2gvYXJtL3BsYXRmb3Jtcy94aWxpbngtenlu
cW1wLWVlbWkuYzo0Mjo3MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhub2RlaWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDIgfCBzdGF0aWMgaW5saW5lIGJvb2wgZG9t
YWluX2hhc19ub2RlX2FjY2VzcyhzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBub2RlaWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+CmFyY2gvYXJtL3BsYXRmb3Jtcy94aWxp
bngtenlucW1wLWVlbWkuYzogSW4gZnVuY3Rpb24g4oCYZG9tYWluX2hhc19yZXNldF9hY2Nl
c3PigJk6CmFyY2gvYXJtL3BsYXRmb3Jtcy94aWxpbngtenlucW1wLWVlbWkuYzo0Nzo3MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3TigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNDcgfCBzdGF0aWMgaW5saW5lIGJvb2wgZG9tYWluX2hhc19yZXNldF9hY2Nlc3Mo
c3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgcnN0KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF92Y3B1X2Jsb2Nr4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nw
aW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g
4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FW
RV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5e
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xB
UkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhz
dHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
c3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25f
ZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0
YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQo
cGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3Jl
YWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2Jk
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2
cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNp
X3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0
YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRh
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vz
c19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6
Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlz
X2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9n
cmFudF90YWJsZS5jOjMxOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKA
mG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2Fy
bmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhs
b25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKA
mX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFy
ZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7
IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
ZXZlbnQuaDoxNywKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9ncmFudF90YWJsZS5j
OjMyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9i
YWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAg
ICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxv
Y19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmg6MzEsCiAgICAgICAgICAgICAgICAgZnJvbSBj
b21tb24vZ3JhbnRfdGFibGUuYzozNDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90
YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfbWFya19kaXJ0eeKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjIwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0IGRvbWFpbiAqZCwgbWZuX3QgbWZu
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyMDo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0
IGRvbWFpbiAqZCwgbWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfaG9zdF9t
YXBwaW5nX2dldF9wYWdlX3R5cGXigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRf
dGFibGUuaDoyNzo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyb+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBnbnR0YWJfaG9z
dF9tYXBwaW5nX2dldF9wYWdlX3R5cGUoYm9vbCBybywKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9ncmFudF90YWJsZS5oOjI4Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI4IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICps
ZCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZ3JhbnRfdGFibGUuaDoyOTo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyOSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAq
cmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9yZWxlYXNlX2hvc3Rf
bWFwcGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDozNDo3
MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDM0IHwgc3RhdGljIGlubGluZSBib29sIGdudHRhYl9yZWxlYXNlX2hvc3RfbWFw
cGluZ3MoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2FsbC5oOjIyLAog
ICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2dyYW50X3RhYmxlLmM6MzY6Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
MjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVkdWxl
cl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRfdCBkZWYs
IGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3Ry
dWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWN0
bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9z
eXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFkY29uc29s
ZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4v
aW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2lu
Y2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2Rl
aWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBz
dHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFn
cykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3Nl
dHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2NvcHni
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
MzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3YXJl
X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRf
cG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2RfdGFy
Z2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzgxOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM4MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2UoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZhdGlv
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozOTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1v
cnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVt
b3ZlX2Zyb21fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDEyIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0
IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9jbGFpbV9wYWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIz
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDI4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxl
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQzODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4
ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9zY2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEs
IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
aXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJp
bmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
aW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNz
aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90
IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4
X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90
IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rl
YXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNo
YXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIg
fCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAq
cGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAg
ICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hzbS94c20uaDo1
NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdf
cGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3Vu
cGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9k
ZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vf
dW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJj
ZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRf
dCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNl
X3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9h
bHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2
bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50
MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRf
dm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250
cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hz
bS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRm
b3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJz
aW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0
ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNl
X21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFn
ZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5
cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Y29tbW9uL2dyYW50X3RhYmxlLmM6Mzc6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDog
SW4gZnVuY3Rpb24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs4oCZOgouL2luY2x1ZGUv
eGVuL2RvbWFpbl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
dmHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZhKSB7fTsKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fl5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6
NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0
IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEw
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj
dCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KY29tbW9uL2dyYW50X3RhYmxlLmM6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9tYXBf
Z3JhbnRfcmVm4oCZOgpjb21tb24vZ3JhbnRfdGFibGUuYzoxMzY3OjIwOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
IDEzNjcgfCAgICAgZm9yICggaSA9IDA7IGkgPCBjb3VudDsgaSsrICkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgXgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9h
c3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2Rl
diAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2Rldmlj
ZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNp
X3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2
cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMy
X3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJf
dCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lf
cHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
ICBMRCAgICAgIGFyY2gvYXJtL3BsYXRmb3Jtcy9idWlsdF9pbi5vCi4vaW5jbHVkZS94ZW4v
c2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1
ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUg
Ym9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5eCmNvbW1vbi9ncmFudF90YWJsZS5jOiBJbiBmdW5jdGlvbiDigJhn
cmFudF90YWJsZV9pbml04oCZOgpjb21tb24vZ3JhbnRfdGFibGUuYzoxOTkyOjI3OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KIDE5OTIgfCAgICAgaWYgKCBtYXhfZ3JhbnRfZnJhbWVzIDwgSU5JVElBTF9OUl9H
UkFOVF9GUkFNRVMgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21t
b24vZ3JhbnRfdGFibGUuYzogSW4gZnVuY3Rpb24g4oCYZ250dGFiX3RyYW5zZmVy4oCZOgpj
b21tb24vZ3JhbnRfdGFibGUuYzoyMjczOjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDIyNzMgfCAgICAgZm9y
ICggaSA9IDA7IGkgPCBjb3VudDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
XgogIENDICAgICAgY29tbW9uL2d6aXAvZ3VuemlwLm8KY29tbW9uL2dyYW50X3RhYmxlLmM6
IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9nZXRfc3RhdHVzX2ZyYW1lc+KAmToKY29tbW9uL2dy
YW50X3RhYmxlLmM6MzM2NjoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVp
bnQzMl904oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAz
MzY2IHwgICAgIGZvciAoIGkgPSAwOyBpIDwgb3AubnJfZnJhbWVzOyBpKysgKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICBeCiAgQ0MgICAgICBjb21tb24vaHlwZnMubwpjb21tb24v
Z3JhbnRfdGFibGUuYzogSW4gZnVuY3Rpb24g4oCYZ250dGFiX3N3YXBfZ3JhbnRfcmVm4oCZ
Ogpjb21tb24vZ3JhbnRfdGFibGUuYzozNTA4OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDM1MDggfCAgICAg
Zm9yICggaSA9IDA7IGkgPCBjb3VudDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgXgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAog
ICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2d6aXAvZ3VuemlwLmM6NToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CmNvbW1vbi9ncmFudF90YWJsZS5jOiBJbiBmdW5jdGlvbiDigJhncmFu
dF90YWJsZV9kZXN0cm954oCZOgpjb21tb24vZ3JhbnRfdGFibGUuYzo0MDEzOjIwOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KIDQwMTMgfCAgICAgZm9yICggaSA9IDA7IGkgPCBucl9ncmFudF9mcmFtZXModCk7
IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAgICAgICAg
ICAgICBmcm9tIGNvbW1vbi9oeXBmcy5jOjk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMg
aW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpjb21t
b24vZ3JhbnRfdGFibGUuYzo0MDIwOjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDQwMjAgfCAgICAgZm9yICgg
aSA9IDA7IGkgPCBucl9hY3RpdmVfZ3JhbnRfZnJhbWVzKHQpOyBpKysgKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICBeCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bp
bmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDi
gJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBv
ZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hW
TV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+
fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFU
XyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9ncmFudF90YWJsZS5jOjQwMjQ6MjA6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogNDAyNCB8ICAgICBmb3IgKCBpID0gMDsgaSA8IG5yX3N0YXR1c19mcmFtZXMo
dCk7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBp
bmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90
IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL2dyYW50
X3RhYmxlLmM6IEF0IHRvcCBsZXZlbDoKY29tbW9uL2dyYW50X3RhYmxlLmM6NDM2OToxOiB3
YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlv
biBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiA0MzY5IHwgX19pbml0Y2FsbChnbnR0YWJf
dXNhZ2VfaW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAg
ICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9j
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFy
KGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5o
OjcwOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19s
YXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIz
MCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAg
IF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAq
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5f
dG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3Rh
dGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3Rf
ZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2Rv
bWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7i
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlv
biDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2Zp
bHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL2d6aXAvZ3VuemlwLmM6
NzI6CmNvbW1vbi9nemlwL2luZmxhdGUuYzogSW4gZnVuY3Rpb24g4oCYaHVmdF9idWlsZOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KY29tbW9uL2d6aXAvaW5mbGF0ZS5jOjQxNTo0OTogd2FybmluZzogb3BlcmFuZCBv
ZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5z
aWduZWQgaW504oCZIGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdz
aWduLWNvbXBhcmVdCiAgNDE1IHwgICAgICAgICAgICAgICAgIHogPSAoeiA9IGcgLSB3KSA+
ICh1bnNpZ25lZClsID8gbCA6IHo7ICAvKiB1cHBlciBsaW1pdCBvbiB0YWJsZSBzaXplICov
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgXgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0
IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVy
eV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KY29tbW9uL2d6aXAvaW5mbGF0ZS5jOiBJbiBmdW5jdGlvbiDigJhp
bmZsYXRlX2R5bmFtaWPigJk6CmNvbW1vbi9nemlwL2luZmxhdGUuYzo5NDA6MjE6IHdhcm5p
bmc6IHN1Z2dlc3QgYnJhY2VzIGFyb3VuZCBlbXB0eSBib2R5IGluIGFuIOKAmGlm4oCZIHN0
YXRlbWVudCBbLVdlbXB0eS1ib2R5XQogIDk0MCB8ICAgICAgICBERUJHKCJkeW43ICIpOwog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vZ3ppcC9pbmZsYXRlLmM6OTQ3
OjE3OiB3YXJuaW5nOiBzdWdnZXN0IGJyYWNlcyBhcm91bmQgZW1wdHkgYm9keSBpbiBhbiDi
gJhpZuKAmSBzdGF0ZW1lbnQgWy1XZW1wdHktYm9keV0KICA5NDcgfCAgICAgICAgREVCRygi
PiIpOwogICAgICB8ICAgICAgICAgICAgICAgICBeCmNvbW1vbi9nemlwL2luZmxhdGUuYzog
SW4gZnVuY3Rpb24g4oCYbWFrZWNyY+KAmToKY29tbW9uL2d6aXAvaW5mbGF0ZS5jOjEwNjM6
MTk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAxMDYzIHwgICAgIGZvciAoaSA9IDA7IGkgPCBzaXplb2Yo
cCkvc2l6ZW9mKGludCk7IGkrKykKICAgICAgfCAgICAgICAgICAgICAgICAgICBeCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9i
bG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9t
ZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGlu
dCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNz
X3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3Rh
dGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQu
aDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2Fs
bC5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2h5cGZzLmM6MTA6Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyMjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVk
dWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRfdCBk
ZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2Rv
bWN0bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFkY29u
c29sZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg1IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2hu
ICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNo
bl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFncykKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwOToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3NldHVw4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyMToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2NvcHnigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMzM6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0
IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzcx
OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfcG9kX3RhcmdldCh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2RfdGFyZ2V04oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1v
cnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzgxOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4MSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2UoeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3BhZ2Xi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDA2
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21fcGh5
c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9wYWdl
c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Y29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMzOjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQz
ODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2RlZmF1
bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zY2hl
ZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDQg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21h
cF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9t
YWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpk
YXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91bm1h
cF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFw
X2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9p
ZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJx
LCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50
OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjUwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1aW50MTZfdCBzdGFydCwK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9k
dGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBh
dGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291
cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jl
c291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU1MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9w
bHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNl
X3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6
NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dz
aSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZs
aW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
aHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVsdF90IGRlZikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0o
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3Di
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qg
b3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3ZudW1h
aW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJvbOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1
NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4
OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2Rl
ZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWFp
bl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9oeXBmcy5jOjEzOgou
L2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKA
mToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDi
gJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhz
aXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVk
bmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50
IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwog
ICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4K
Y29tbW9uL2h5cGZzLmM6IEluIGZ1bmN0aW9uIOKAmGh5cGZzX25vZGVfZXhpdOKAmToKY29t
bW9uL2h5cGZzLmM6MTIyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGVudHJ5
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIyIHwgdm9pZCBjZl9jaGVjayBoeXBmc19u
b2RlX2V4aXQoY29uc3Qgc3RydWN0IGh5cGZzX2VudHJ5ICplbnRyeSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn5+CmNvbW1vbi9oeXBmcy5jOiBJbiBmdW5jdGlvbiDigJhoeXBmc19sZWFmX2ZpbmRlbnRy
eeKAmToKY29tbW9uL2h5cGZzLmM6Mjk0OjM1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRpcuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8ICAgICBjb25zdCBzdHJ1
Y3QgaHlwZnNfZW50cnlfZGlyICpkaXIsIGNvbnN0IGNoYXIgKm5hbWUsIHVuc2lnbmVkIGlu
dCBuYW1lX2xlbikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+CmNvbW1vbi9oeXBmcy5jOjI5NDo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhuYW1l4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgICAgIGNvbnN0IHN0cnVj
dCBoeXBmc19lbnRyeV9kaXIgKmRpciwgY29uc3QgY2hhciAqbmFtZSwgdW5zaWduZWQgaW50
IG5hbWVfbGVuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL2h5cGZzLmM6Mjk0OjcxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5hbWVfbGVu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk0IHwgICAgIGNvbnN0IHN0cnVjdCBoeXBmc19lbnRyeV9kaXIgKmRpciwgY29uc3QgY2hh
ciAqbmFtZSwgdW5zaWduZWQgaW50IG5hbWVfbGVuKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+fn5+fgpjb21tb24vaHlwZnMuYzogSW4gZnVuY3Rpb24g4oCYaHlwZnNfZHluZGly
X2VudGVy4oCZOgpjb21tb24vaHlwZnMuYzo0MDQ6MzE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZW50cnnigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDQgfCAgICAgY29u
c3Qgc3RydWN0IGh5cGZzX2VudHJ5ICplbnRyeSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpjb21tb24vaHlwZnMuYzogSW4gZnVuY3Rpb24g4oCYaHlw
ZnNfZHluZGlyX2ZpbmRlbnRyeeKAmToKY29tbW9uL2h5cGZzLmM6NDE1OjM1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRpcuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQx
NSB8ICAgICBjb25zdCBzdHJ1Y3QgaHlwZnNfZW50cnlfZGlyICpkaXIsIGNvbnN0IGNoYXIg
Km5hbWUsIHVuc2lnbmVkIGludCBuYW1lX2xlbikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9oeXBmcy5jOiBJbiBmdW5jdGlvbiDigJho
eXBmc19yZWFkX2R5bmRpcuKAmToKY29tbW9uL2h5cGZzLmM6NDI2OjMxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGVudHJ54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI2
IHwgICAgIGNvbnN0IHN0cnVjdCBoeXBmc19lbnRyeSAqZW50cnksIFhFTl9HVUVTVF9IQU5E
TEVfUEFSQU0odm9pZCkgdWFkZHIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KY29tbW9uL2h5cGZzLmM6IEluIGZ1bmN0aW9uIOKAmGh5cGZzX3dyaXRl
X2RlbnnigJk6CmNvbW1vbi9oeXBmcy5jOjY2MTozMDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsZWFm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjYxIHwgICAgIHN0cnVj
dCBoeXBmc19lbnRyeV9sZWFmICpsZWFmLCBYRU5fR1VFU1RfSEFORExFX1BBUkFNKGNvbnN0
X3ZvaWQpIHVhZGRyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgpjb21tb24vaHlwZnMuYzo2NjE6NzE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
dWFkZHLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NjEgfCAgICAgc3RydWN0IGh5cGZz
X2VudHJ5X2xlYWYgKmxlYWYsIFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0oY29uc3Rfdm9pZCkg
dWFkZHIsCmNvbW1vbi9oeXBmcy5jOjY2MjoxODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh1bGVu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjYyIHwgICAgIHVuc2lnbmVk
IGludCB1bGVuKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJj
aC9hcm0vdGVlL2ZmYS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6NjksCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbl9w
YWdlLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS90ZWUvZmZhLmM6NTA6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhj
cHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3Mo
dW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9w
ZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3Ti
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwg
ICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAg
ICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5
Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0
MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4K
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tf
dmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhm
cmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tf
dmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVu
Y3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1m
buKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDox
MDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3Rf
ZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9m
aWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9kb21haW5f
cGFnZV9nbG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA3MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9iYWwoY29u
c3Qgdm9pZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3RlZS9m
ZmEuYzo1NToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19u
aWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMg
aW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3Jl
YWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
bWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRl
X2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21l
bV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
YWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVz
b3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZv
aWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhp
ZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hl
Y2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3Qg
ZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5
X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Js
b2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fz
c2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4
IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2
ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNl
KHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lf
cmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZw
Y2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJf
dCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90
IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoz
MDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9w
cm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFp
buKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBh
bHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJj
aC9hcm0vdGVlL2ZmYS5jOjYwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZXZlbnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFy
Y2hfdmlycV9pc19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9h
cm0vdGVlL2ZmYS5jOiBJbiBmdW5jdGlvbiDigJhmZmFfZ2V0X3ZlcnNpb27igJk6CmFyY2gv
YXJtL3RlZS9mZmEuYzoxMDE6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYbG9uZyB1bnNpZ25lZCBp
bnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTAxIHwgICAgIGlmICgg
cmVzcC5hMCA9PSBGRkFfUkVUX05PVF9TVVBQT1JURUQgKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgXn4KYXJjaC9hcm0vdGVlL2ZmYS5jOiBJbiBmdW5jdGlvbiDigJhmZmFfdGVhcmRv
d25fdGltZXJfY2FsbGJhY2vigJk6CmFyY2gvYXJtL3RlZS9mZmEuYzo0MTA6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYXJn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDEwIHwgc3RhdGljIHZvaWQgZmZhX3RlYXJkb3duX3RpbWVyX2NhbGxiYWNrKHZvaWQgKmFy
ZykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+CmFyY2gvYXJtL3RlZS9mZmEuYzogSW4gZnVuY3Rpb24g4oCYZmZhX3JlbGlucXVp
c2hfcmVzb3VyY2Vz4oCZOgphcmNoL2FybS90ZWUvZmZhLmM6NDQyOjUyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDIgfCBz
dGF0aWMgaW50IGZmYV9yZWxpbnF1aXNoX3Jlc291cmNlcyhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KICBJTklUX08gIGNvbW1vbi9nemlwL2d1bnppcC5pbml0Lm8KICBDQyAgICAgIGFy
Y2gvYXJtL3RlZS9mZmFfc2htLm8KICBMRCAgICAgIGNvbW1vbi9nemlwL2J1aWx0X2luLm8K
ICBDQyAgICAgIGNvbW1vbi9pb3JlcS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vaW9yZXEu
YzoxOToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTi
gJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5s
aW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gv
YXJtL3RlZS9mZmFfc2htLmM6OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUg
aW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZ
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0
aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50
MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FW
RV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0
IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5o
OjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1l
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWVyLmg6MTEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBj
cHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNp
Z25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUg
PCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9
PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMg
aW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNM
QVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5E
LCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21t
Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh0
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAg
ICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3Rh
bXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2Fy
bS90ZWUvZmZhX3ByaXZhdGUuaDoxNSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJt
L3RlZS9mZmFfc2htLmM6MTY6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDi
gJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5
IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9h
bHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9n
Zm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQs
IHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVy
ZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN0
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8
ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNo
X3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJl
duKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2Vf
aW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bDLigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGlj
IGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9k
ZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBw
Y2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAg
ICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dy
aXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRl
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5k
aW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3Rh
bXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2Rv
bWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRp
YyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZt
X2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVp
bnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBs
b25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
cDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSBjb21tb24vaW9yZXEuYzoyMDoKLi9pbmNsdWRlL3hlbi9kb21h
aW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9kb21haW5fcGFnZV9nbG9iYWzigJk6
Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA3MSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9iYWwoY29uc3Qgdm9pZCAqdmEpIHt9
OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoyNSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9pb3JlcS5jOjIxOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2
aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3Nf
cGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVj
dCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDox
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFs
KHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBj
b21tb24vaW9yZXEuYzozMToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOiBJbiBm
dW5jdGlvbiDigJhoYW5kbGVfcGlv4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvcmVx
Lmg6NDE6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcG9ydOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA0MSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBoYW5kbGVfcGlvKHVp
bnQxNl90IHBvcnQsIHVuc2lnbmVkIGludCBzaXplLCBpbnQgZGlyKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9pb3JlcS5oOjQxOjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDEgfCBzdGF0aWMgaW5saW5lIGJvb2wg
aGFuZGxlX3Bpbyh1aW50MTZfdCBwb3J0LCB1bnNpZ25lZCBpbnQgc2l6ZSwgaW50IGRpcikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvcmVxLmg6NDE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGly4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDQxIHwgc3RhdGljIGlubGluZSBib29sIGhhbmRsZV9waW8odWludDE2X3QgcG9y
dCwgdW5zaWduZWQgaW50IHNpemUsIGludCBkaXIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOiBJbiBmdW5jdGlvbiDigJhtc2l4
X3dyaXRlX2NvbXBsZXRpb27igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vaW9yZXEuaDo1
MTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDUxIHwgc3RhdGljIGlubGluZSB2b2lkIG1zaXhfd3JpdGVfY29tcGxldGlv
bihzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24vaXJxLm8KY29t
bW9uL2lvcmVxLmM6IEluIGZ1bmN0aW9uIOKAmGlvcmVxX3NlbmRfYnVmZmVyZWTigJk6CmNv
bW1vbi9pb3JlcS5jOjEyMTA6NTk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdWludDMyX3TigJkge2Fr
YSDigJh1bnNpZ25lZCBpbnTigJl9IGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
MTIxMCB8ICAgICBpZiAoIChwZy0+cHRycy53cml0ZV9wb2ludGVyIC0gcGctPnB0cnMucmVh
ZF9wb2ludGVyKSA+PQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBefgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9pcnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2lycS5j
OjE6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDozNiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vaXJxLmg6NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlu
bGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KY29tbW9uL2lycS5jOiBJbiBmdW5jdGlvbiDigJhub19hY3Rpb27igJk6CmNvbW1v
bi9pcnEuYzozMjoyOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcGzigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMzIgfCB2b2lkIGNmX2NoZWNrIG5vX2FjdGlvbihpbnQg
Y3BsLCB2b2lkICpkZXZfaWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fl5+fgpjb21tb24vaXJxLmM6MzI6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGV2X2lk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMyIHwgdm9pZCBjZl9jaGVjayBu
b19hY3Rpb24oaW50IGNwbCwgdm9pZCAqZGV2X2lkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fn5+fgpjb21tb24vaXJxLmM6IEluIGZ1bmN0
aW9uIOKAmGlycV9hY3Rvcl9ub25l4oCZOgpjb21tb24vaXJxLmM6MzY6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVzY+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAz
NiB8IHZvaWQgY2ZfY2hlY2sgaXJxX2FjdG9yX25vbmUoc3RydWN0IGlycV9kZXNjICpkZXNj
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+CmNvbW1vbi9pcnEuYzogSW4gZnVuY3Rpb24g4oCYaXJxX3N0YXJ0dXBfbm9uZeKA
mToKY29tbW9uL2lycS5jOjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
c2PigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDAgfCB1bnNpZ25lZCBpbnQgY2ZfY2hl
Y2sgaXJxX3N0YXJ0dXBfbm9uZShzdHJ1Y3QgaXJxX2Rlc2MgKmRlc2MpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CiAgQ0MgICAgICBjb21tb24va2VybmVsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNv
bW1vbi9rZXJuZWwuYzoxMDoKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDi
gJhub19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdh
cm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCY
bG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTi
gJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBh
cmVdCiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBz
OyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9rZXJuZWwuYzox
MjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5n
cyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5j
dGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2
OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZu
X3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0
YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9k
ZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFk
aXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2so
cmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
dm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVk
IGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVu
dF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9i
bG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9t
ZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGlu
dCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNz
X3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3Rh
dGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQu
aDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2Fs
bC5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2tlcm5lbC5jOjE1OgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hl
ZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0Njox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
b21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNv
bnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBp
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNo
biAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3
MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQo
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVt
b3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzkz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQw
NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3Bo
eXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFn
ZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZh
dWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2No
ZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9t
YXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2Rv
bWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAq
ZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5t
YXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1h
cF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZv
aWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGly
cSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWlu
dDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxv
dykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQs
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25f
ZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRw
YXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNv
dXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9y
ZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vf
cGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJj
ZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0
OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9n
c2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2Zm
bGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMy
X3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBz
dGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90
IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVt
YWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2
NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjcz
ODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDM6
NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhzbV9k
ZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21h
aW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+CmNvbW1vbi9rZXJuZWwuYzogSW4gZnVuY3Rpb24g4oCYcGFyc2VfcGFy
YW1z4oCZOgpjb21tb24va2VybmVsLmM6ODk6MjY6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYbG9uZyBp
bnTigJkgYW5kIOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAg
ODkgfCAgICAgICAgICAgICBpZiAoIChxLW9wdCkgPCAoc2l6ZW9mKG9wdCktMSkgKSAvKiBh
dm9pZCBvdmVyZmxvdyAqLwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNv
bW1vbi9rZXJuZWwuYzoxMjQ6NDI6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYc2l6ZV904oCZIHtha2Eg
4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGFuZCDigJhsb25nIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMTI0IHwgICAgICAgICAgICAgICAgICAgICAgc3RybGVuKHBhcmFtLT5uYW1l
KSA9PSBxICsgMSAtIG9wdCAmJgogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXn4KY29tbW9uL2tlcm5lbC5jOiBJbiBmdW5jdGlvbiDigJhwYXJz
ZV9ib29s4oCZOgpjb21tb24va2VybmVsLmM6MjM0OjIyOiB3YXJuaW5nOiBvcGVyYW5kIG9m
IOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKA
mHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWdu
ZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDIzNCB8ICAgICBz
aXplX3QgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4o
cyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpjb21tb24va2VybmVsLmM6IEluIGZ1bmN0aW9uIOKAmHBhcnNlX2Jvb2xlYW7i
gJk6CmNvbW1vbi9rZXJuZWwuYzoyOTE6MTY6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzri
gJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6ZV90
4oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mg
b2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMjkxIHwgICAgIHNsZW4gPSBl
ID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAg
ICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9rZXJu
ZWwuYzogSW4gZnVuY3Rpb24g4oCYcGFyc2Vfc2lnbmVkX2ludGVnZXLigJk6CmNvbW1vbi9r
ZXJuZWwuYzozMjk6MTY6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBz
aWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCY
bG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3Bl
cmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMzI5IHwgICAgIHNsZW4gPSBlID8gKHsgQVNTRVJU
KGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAgICAg
ICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9rZXJuZWwuYzogQXQgdG9w
IGxldmVsOgpjb21tb24va2VybmVsLmM6NDgxOjE6IHdhcm5pbmc6IOKAmHN0YXRpY+KAmSBp
cyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJh
dGlvbl0KICA0ODEgfCBfX2luaXRjYWxsKGJ1aWxkaW5mb19pbml0KTsKICAgICAgfCBefn5+
fn5+fn5+CmNvbW1vbi9rZXJuZWwuYzo1MDI6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlz
IG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0
aW9uXQogIDUwMiB8IF9faW5pdGNhbGwocGFyYW1faW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+
fgogIENDICAgICAgYXJjaC9hcm0vdGVlL2ZmYV9wYXJ0aW5mby5vCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21jY2MuaDoxOSwKICAg
ICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3RlZS9mZmFfcGFydGluZm8uYzoxMDoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9u
cl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAx
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNp
Z25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bp
bmxvY2suaDo0MywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2
OSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3RlZS9mZmFfcHJpdmF0ZS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vdGVlL2ZmYV9wYXJ0aW5mby5jOjEz
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NB
VkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBj
cHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54
dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0
aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21t
Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh0
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAg
ICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3Rh
bXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2Fy
bS90ZWUvZmZhX3ByaXZhdGUuaDoxNToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0
aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5s
b2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1f
YWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVt
X2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
cDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6
MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWlu
dDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lk
IHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9m
cm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2
MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVf
dCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0
eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQx
NyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZv
aWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9y
eV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3ko
c3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3
ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNl
KHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
ZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5l
IHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5s
aW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwg
c3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211
c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlz
X3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
MTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNv
bnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgogIENDICAgICAg
Y29tbW9uL2tleWhhbmRsZXIubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3JlZ3MuaDoxMSwKICAgICAgICAgICAgICAg
ICBmcm9tIGNvbW1vbi9rZXloYW5kbGVyLmM6NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9j
cHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRp
YyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24va2V5aGFuZGxlci5jOjg6Ci4vaW5jbHVkZS94
ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1
ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNo
YW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7
YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90
aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/
ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZXZlbnQuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9r
ZXloYW5kbGVyLmM6MTA6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
Ojc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90
b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0
aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4
LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhy
Y3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMw
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZt
X2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVp
bnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBs
b25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
cDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQg
dnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjE3
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
dmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1MDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9iYWwo
dW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNv
bW1vbi9rZXloYW5kbGVyLmM6MTI6Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6IEF0IHRvcCBs
ZXZlbDoKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDoxNjY6MTogd2FybmluZzog4oCYaW5saW5l
4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRl
Y2xhcmF0aW9uXQogIDE2NiB8IHN0YXRpYyB2b2lkIGlubGluZSB4aGNpX2RiY191YXJ0X2lu
aXQodm9pZCkge30KICAgICAgfCBefn5+fn4KY29tbW9uL2tleWhhbmRsZXIuYzogSW4gZnVu
Y3Rpb24g4oCYa2V5cHJlc3NfYWN0aW9u4oCZOgpjb21tb24va2V5aGFuZGxlci5jOjc0OjQ0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVudXNlZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA3NCB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIGtleXByZXNzX2FjdGlvbih2
b2lkICp1bnVzZWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl5+fn5+fgpjb21tb24va2V5aGFuZGxlci5jOiBJbiBmdW5jdGlvbiDigJhy
ZWJvb3RfbWFjaGluZeKAmToKY29tbW9uL2tleWhhbmRsZXIuYzoyNDg6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYdW51c2Vk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjQ4IHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgcmVib290X21hY2hpbmUodW5zaWduZWQgY2hh
ciBrZXksIGJvb2wgdW51c2VkKQpjb21tb24va2V5aGFuZGxlci5jOiBJbiBmdW5jdGlvbiDi
gJhkdW1wX2RvbWFpbnPigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwg
KHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL2tleWhhbmRsZXIuYzozMjI6
MTM6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0
X3ZjcHXigJkKICAzMjIgfCAgICAgICAgICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUg
KCB1bml0LCB2ICkKICAgICAgfCAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn4KY29tbW9uL2tleWhhbmRsZXIuYzogSW4gZnVuY3Rpb24g4oCYcmVhZF9jbG9ja3Nfc2xh
dmXigJk6CmNvbW1vbi9rZXloYW5kbGVyLmM6Mzc1OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHVudXNlZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NSB8IHN0YXRp
YyB2b2lkIGNmX2NoZWNrIHJlYWRfY2xvY2tzX3NsYXZlKHZvaWQgKnVudXNlZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn5+fn4K
Y29tbW9uL2tleWhhbmRsZXIuYzogSW4gZnVuY3Rpb24g4oCYcmVhZF9jbG9ja3PigJk6CmNv
bW1vbi9rZXloYW5kbGVyLmM6Mzg3OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGtleeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8IHN0YXRpYyB2b2lkIGNmX2No
ZWNrIHJlYWRfY2xvY2tzKHVuc2lnbmVkIGNoYXIga2V5KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9rZXloYW5k
bGVyLmM6IEluIGZ1bmN0aW9uIOKAmHJ1bl9hbGxfbm9uaXJxX2tleWhhbmRsZXJz4oCZOgpj
b21tb24va2V5aGFuZGxlci5jOjQ2MDoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA0NjAgfCAgICAg
Zm9yICggayA9IDA7IGsgPCBBUlJBWV9TSVpFKGtleV90YWJsZSk7IGsrKyApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL2tleWhhbmRsZXIuYzo0NTI6NTU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdW51c2Vk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDUyIHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgcnVuX2FsbF9ub25pcnFfa2V5aGFuZGxl
cnModm9pZCAqdW51c2VkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fn5+fgpjb21tb24va2V5aGFuZGxlci5jOiBJ
biBmdW5jdGlvbiDigJhkb190b2dnbGVfYWx0X2tleeKAmToKY29tbW9uL2tleWhhbmRsZXIu
Yzo1MDE6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdW51c2Vk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgZG9fdG9nZ2xl
X2FsdF9rZXkodW5zaWduZWQgY2hhciBrZXksIGJvb2wgdW51c2VkKQogIENDICAgICAgYXJj
aC9hcm0vdGVlL2ZmYV9yeHR4Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9t
YWluX3BhZ2UuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3RlZS9mZmFf
cnh0eC5jOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5j
dGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJf
c2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAg
ICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9j
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFy
KGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNf
aW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVu
Y3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBw
YWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4
OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0
bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNr
LCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+
fn5+fn4KLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1h
cF9kb21haW5fcGFnZV9nbG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3
MTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA3MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9n
bG9iYWwoY29uc3Qgdm9pZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gv
YXJtL3RlZS9mZmFfcHJpdmF0ZS5oOjE1LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9h
cm0vdGVlL2ZmYV9yeHR4LmM6MTU6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlv
biDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2Fj
Y2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJv
bV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEg
fCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3Qg
KnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lk
IGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlf
cG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6
NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2Rl
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGlu
ZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0
X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19w
dl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEx
MzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25z
dCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGNv
bW1vbi9saXZlcGF0Y2gubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9jcHUuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2xpdmVwYXRjaC5jOjY6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhj
cHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3Mo
dW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1ZXN0
X2FjY2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1ZXN0
X2FjY2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2xpdmVwYXRjaC5j
Ojk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90
b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0
aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4
LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhy
Y3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMw
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZt
X2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVp
bnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBs
b25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
cDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQg
dnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL2xpdmVwYXRjaC5jOjMwOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmly
cV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1MDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9iYWwodW5z
aWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL2xpdmVwYXRjaC5jOiBJbiBmdW5j
dGlvbiDigJhzZWN1cmVfcGF5bG9hZOKAmToKY29tbW9uL2xpdmVwYXRjaC5jOjQwMjo3NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhlbGbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0MDIgfCBzdGF0aWMgaW50IHNlY3VyZV9wYXlsb2FkKHN0cnVjdCBwYXlsb2FkICpw
YXlsb2FkLCBzdHJ1Y3QgbGl2ZXBhdGNoX2VsZiAqZWxmKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KY29tbW9uL2xpdmVwYXRjaC5jOiBJbiBmdW5jdGlvbiDigJhsaXZlcGF0
Y2hfdmVyaWZ5X2V4cGVjdGF0aW9uX2Zu4oCZOgpjb21tb24vbGl2ZXBhdGNoLmM6NTc4OjE5
OiB3YXJuaW5nOiBjb21wYXJpc29uIGlzIGFsd2F5cyBmYWxzZSBkdWUgdG8gbGltaXRlZCBy
YW5nZSBvZiBkYXRhIHR5cGUgWy1XdHlwZS1saW1pdHNdCiAgNTc4IHwgICAgIGlmICggZXhw
LT5sZW4gPiBzaXplb2YoZXhwLT5kYXRhKSkKICAgICAgfCAgICAgICAgICAgICAgICAgICBe
CmNvbW1vbi9saXZlcGF0Y2guYzogSW4gZnVuY3Rpb24g4oCYbGl2ZXBhdGNoX2NoZWNrX2V4
cGVjdGF0aW9uc+KAmToKY29tbW9uL2xpdmVwYXRjaC5jOjYwNToyMDogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
NjA1IHwgICAgIGZvciAoIGkgPSAwOyBpIDwgcGF5bG9hZC0+bmZ1bmNzOyBpKysgKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9saXZlcGF0Y2guYzogSW4gZnVuY3Rp
b24g4oCYbGl2ZXBhdGNoX2xpc3TigJk6CmNvbW1vbi9saXZlcGF0Y2guYzoxMzE2OjEyOiB3
YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKA
mGludOKAmSB0byDigJh1bnNpZ25lZCBpbnTigJkgZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBv
dGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KIDEzMTYgfCAgICAgcmV0dXJuIHJjID8g
OiBpZHg7CiAgICAgIHwgICAgICAgICAgICBefgpjb21tb24vbGl2ZXBhdGNoLmM6IEluIGZ1
bmN0aW9uIOKAmHdhc19hY3Rpb25fY29uc2lzdGVudOKAmToKY29tbW9uL2xpdmVwYXRjaC5j
OjE0NzE6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogMTQ3MSB8ICAgICBmb3IgKCBpID0gMDsgaSA8IGRhdGEt
Pm5mdW5jczsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vbGl2
ZXBhdGNoLmM6IEluIGZ1bmN0aW9uIOKAmHRhc2tsZXRfZm7igJk6CmNvbW1vbi9saXZlcGF0
Y2guYzoxNjg4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVudXNlZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogMTY4OCB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIHRhc2ts
ZXRfZm4odm9pZCAqdW51c2VkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+fn5+CmNvbW1vbi9saXZlcGF0Y2guYzogSW4gZnVuY3Rpb24g4oCY
bGl2ZXBhdGNoX3NwaW7igJk6CmNvbW1vbi9saXZlcGF0Y2guYzoxNjk4OjM0OiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KIDE2OTggfCAgICAgd2hpbGUgKCBhdG9taWNfcmVhZChjb3VudGVyKSAhPSBjcHVzICYm
IE5PVygpIDwgdGltZW91dCApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgXn4KY29tbW9uL2xpdmVwYXRjaC5jOjE3MDI6MzE6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogMTcwMiB8
ICAgICBpZiAoIGF0b21pY19yZWFkKGNvdW50ZXIpICE9IGNwdXMgKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIF5+CiAgQ0MgICAgICBhcmNoL2FybS90ZWUvdGVl
Lm8KY29tbW9uL2xpdmVwYXRjaC5jOiBJbiBmdW5jdGlvbiDigJhjcHVfY2FsbGJhY2vigJk6
CmNvbW1vbi9saXZlcGF0Y2guYzoyMjMxOjI4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5mYuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMjIzMSB8ICAgICBzdHJ1Y3Qgbm90
aWZpZXJfYmxvY2sgKm5mYiwgdW5zaWduZWQgbG9uZyBhY3Rpb24sIHZvaWQgKmhjcHUpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9saXZlcGF0Y2gu
YzogQXQgdG9wIGxldmVsOgpjb21tb24vbGl2ZXBhdGNoLmM6MjI2MzoxOiB3YXJuaW5nOiDi
gJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQt
c3R5bGUtZGVjbGFyYXRpb25dCiAyMjYzIHwgX19pbml0Y2FsbChsaXZlcGF0Y2hfaW5pdCk7
CiAgICAgIHwgXn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3JlZ3MuaDoxMSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGVlL3RlZS5oOjIwLAogICAgICAgICAg
ICAgICAgIGZyb20gYXJjaC9hcm0vdGVlL3RlZS5jOjIzOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwg
c3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KYXJjaC9hcm0vdGVlL3RlZS5jOiBBdCB0b3AgbGV2ZWw6CmFyY2gvYXJtL3RlZS90ZWUu
Yzo5NzoxOiB3YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBk
ZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgIDk3IHwgX19pbml0Y2Fs
bCh0ZWVfaW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+fgogIENDICAgICAgYXJjaC9hcm0vdGVl
L29wdGVlLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL3RlZS9vcHRlZS5jOjI3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1
X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1
X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2
b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjc6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKA
mToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3Rh
dGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWlu
dDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUg
aW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUp
IFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDoxMiwKICAgICAgICAgICAgICAg
ICBmcm9tIGFyY2gvYXJtL3RlZS9vcHRlZS5jOjI4OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3Rf
ZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVu
Y3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1h
c2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rp
b24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs4oCZOgouL2luY2x1ZGUveGVuL2RvbWFp
bl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9t
YWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZhKSB7fTsKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1ZXN0X2FjY2Vzcy5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1ZXN0X2FjY2Vzcy5oOjEwLAogICAgICAg
ICAgICAgICAgIGZyb20gYXJjaC9hcm0vdGVlL29wdGVlLmM6MzA6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251
bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9f
bmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4
LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9t
YWluLmg6MTU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1
X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9l
dmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50
MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9u
ZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwg
c3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVw
ZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVz
dF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9y
eV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRl
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZw
Y2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikg
eyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVu
ZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2
Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJ
biBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2
XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL3RlZS9vcHRlZS5jOjM0Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmly
cV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1MDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9iYWwodW5z
aWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KICBDQyAgICAgIGNvbW1vbi9saXZlcGF0Y2hf
ZWxmLm8KYXJjaC9hcm0vdGVlL29wdGVlLmM6IEluIGZ1bmN0aW9uIOKAmG9wdGVlX2RvbWFp
bl90ZWFyZG93buKAmToKYXJjaC9hcm0vdGVlL29wdGVlLmM6MjcxOjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzEgfCBz
dGF0aWMgaW50IG9wdGVlX2RvbWFpbl90ZWFyZG93bihzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4K
YXJjaC9hcm0vdGVlL29wdGVlLmM6IEluIGZ1bmN0aW9uIOKAmGFsbG9jYXRlX3N0ZF9jYWxs
4oCZOgphcmNoL2FybS90ZWUvb3B0ZWUuYzozMDQ6MTY6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDMwNCB8ICAg
ICBpZiAoIGNvdW50ID09IG1heF9vcHRlZV90aHJlYWRzICkKICAgICAgfCAgICAgICAgICAg
ICAgICBefgphcmNoL2FybS90ZWUvb3B0ZWUuYzogSW4gZnVuY3Rpb24g4oCYdHJhbnNsYXRl
X25vbmNvbnRpZ+KAmToKYXJjaC9hcm0vdGVlL29wdGVlLmM6NzA4OjU1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGNhbGzigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3MDgg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IG9wdGVlX3N0ZF9jYWxs
ICpjYWxsLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vdGVlL29wdGVlLmM6IEluIGZ1bmN0aW9u
IOKAmGNvcHlfc3RkX3JlcXVlc3RfYmFja+KAmToKYXJjaC9hcm0vdGVlL29wdGVlLmM6OTcw
OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGN0eOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDk3MCB8IHN0YXRpYyB2b2lkIGNvcHlfc3RkX3JlcXVlc3RfYmFjayhzdHJ1
Y3Qgb3B0ZWVfZG9tYWluICpjdHgsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS90ZWUvb3B0ZWUu
YzogSW4gZnVuY3Rpb24g4oCYaXNzdWVfcnBjX2NtZF9mcmVl4oCZOgphcmNoL2FybS90ZWUv
b3B0ZWUuYzoxMzEwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGN0eOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogMTMxMCB8IHN0YXRpYyBib29sIGlzc3VlX3JwY19jbWRf
ZnJlZShzdHJ1Y3Qgb3B0ZWVfZG9tYWluICpjdHgsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS90ZWUv
b3B0ZWUuYzogSW4gZnVuY3Rpb24g4oCYaGFuZGxlX3hlbl9ycGNfcmV0dXJu4oCZOgphcmNo
L2FybS90ZWUvb3B0ZWUuYzoxMzYzOjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGN0eOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTM2MyB8IHN0YXRpYyB2b2lkIGhhbmRs
ZV94ZW5fcnBjX3JldHVybihzdHJ1Y3Qgb3B0ZWVfZG9tYWluICpjdHgsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgphcmNoL2FybS90ZWUvb3B0ZWUuYzoxMzY0OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ3PigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEzNjQgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbGl2ZXBhdGNo
Lmg6MTcsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vbGl2ZXBhdGNoX2VsZi5jOjk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FW
RV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFU
XyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgogIENDICAgICAgY29tbW9uL21lbV9hY2Nlc3MubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSBjb21tb24vbWVtX2FjY2Vzcy5jOjIzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9j
cHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9j
cHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2Nf
Y3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5l
IHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251
bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9f
bmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDox
NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
bmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4
IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGlt
ZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2
b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGlt
ZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlu
bGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5o
OjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g
4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3Nf
ZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29s
IHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1
aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRw
Mm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7i
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAg
c3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAy
bV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hf
dmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwg
c3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3Qg
cGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwg
dWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1
aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNr
IHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJp
dF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBz
dGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAgICAgICAgICAgICAgZnJv
bSBjb21tb24vbWVtX2FjY2Vzcy5jOjI1OgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjQ6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0ZG9tYWluaW5mb+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rv
bWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjM1OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNt
ZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNj
dGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V0
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDYgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjUxOjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3
MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBl
dnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkx
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0
Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2
OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2Rv
bWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFy
Z2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
bWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRp
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlz
bWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dt
Zm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWlu
ICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdl
cyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9j
b25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21k
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9r
ZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Vu
bWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVu
X2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9p
cnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmlu
ZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0
X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9j
b25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJf
dCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25f
ZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRw
YXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVz
b3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0
NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVs
dF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vf
cGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBz
dGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3Qg
ZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2Nv
cmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9n
c2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGlu
dCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21p
c2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2Zm
bGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlw
ZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
dW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0
X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9y
bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZG1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lv
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBv
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFp
bl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS92bV9ldmVudC5o
OjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3ZtX2V2ZW50Lmg6Mjgs
CiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vbWVtX2FjY2Vzcy5jOjI2OgouL2luY2x1
ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfaW5p
dF9kb21haW7igJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjE0OjU1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTQgfCBzdGF0aWMgaW5saW5lIGludCB2bV9ldmVudF9pbml0X2RvbWFpbihzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVw4oCZOgouL2luY2x1
ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoyNTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI1IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVwKHN0cnVjdCBkb21haW4gKmQsIHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92
bV9ldmVudC5oOjI1Ojc4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgdm1fZXZlbnRf
dG9nZ2xlX3NpbmdsZXN0ZXAoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHZjcHUgKnYsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92
bV9ldmVudC5oOjI2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNiB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB2bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4g
ZnVuY3Rpb24g4oCYdm1fZXZlbnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1l4oCZOgouL2luY2x1
ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDozMjo1MDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMyIHwgdm9pZCB2bV9ldmVu
dF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUoc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3Bv
bnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDozMjo3
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMzIgfCB2b2lkIHZtX2V2ZW50X3JlZ2lzdGVyX3dyaXRlX3Jlc3VtZShzdHJ1
Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1
bmN0aW9uIOKAmHZtX2V2ZW50X2VtdWxhdGVfY2hlY2vigJk6Ci4vaW5jbHVkZS9hc20tZ2Vu
ZXJpYy92bV9ldmVudC5oOjM4OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzggfCB2b2lkIHZtX2V2ZW50X2VtdWxhdGVf
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRl
L2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6Mzg6NjY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM4IHwgdm9pZCB2bV9ldmVu
dF9lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNwb25zZV90ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2
ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3N5bmNfZXZlbnTigJk6Ci4vaW5jbHVk
ZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjQ0OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDQgfCB2b2lkIHZtX2V2ZW50
X3N5bmNfZXZlbnQoc3RydWN0IHZjcHUgKnYsIGJvb2wgdmFsdWUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJp
Yy92bV9ldmVudC5oOjQ0OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZhbHVl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQ0IHwgdm9pZCB2bV9ldmVudF9zeW5jX2V2
ZW50KHN0cnVjdCB2Y3B1ICp2LCBib29sIHZhbHVlKQouL2luY2x1ZGUvYXNtLWdlbmVyaWMv
dm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfcmVzZXRfdm10cmFjZeKAmToK
Li9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6NTA6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHZvaWQg
dm1fZXZlbnRfcmVzZXRfdm10cmFjZShzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGNvbW1vbi9t
ZW1vcnkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
bHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bp
bmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL21lbW9yeS5jOjEwOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxp
bmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGlu
dCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9t
Y3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0
bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZN
X1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1
NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2
b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21w
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAg
ICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYg
KG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMz
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25l
ZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9u
ZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzgg
fCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1l
c3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1l
c3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4v
ZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rpb24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs
4oCZOgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZh
KSB7fTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTIs
CiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vbWVtb3J5LmM6MTI6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3Jh
ZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
ZG9tYWluLmg6MTU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0
aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEz
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhy
Y3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2
bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Js
b2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3
IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKA
mHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9n
dWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21l
bW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2
aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3Nf
cGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVj
dCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDox
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFs
KHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmg6MzEsCiAgICAgICAgICAgICAgICAgZnJvbSBj
b21tb24vbWVtb3J5LmM6MTM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUu
aDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX21hcmtfZGlydHnigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlubGluZSB2
b2lkIGdudHRhYl9tYXJrX2RpcnR5KHN0cnVjdCBkb21haW4gKmQsIG1mbl90IG1mbikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MjA6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIwIHwgc3RhdGljIGlubGluZSB2b2lkIGdudHRhYl9tYXJrX2RpcnR5KHN0cnVjdCBkb21h
aW4gKmQsIG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZ3JhbnRfdGFibGUuaDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX2hvc3RfbWFwcGlu
Z19nZXRfcGFnZV90eXBl4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxl
Lmg6Mjc6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcm/igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjcgfCBzdGF0aWMgaW5saW5lIGJvb2wgZ250dGFiX2hvc3RfbWFw
cGluZ19nZXRfcGFnZV90eXBlKGJvb2wgcm8sCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3Jh
bnRfdGFibGUuaDoyODo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyOCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqbGQsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dy
YW50X3RhYmxlLmg6Mjk6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjkgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKnJkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9n
cmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBp
bmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MzQ6NzA6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzNCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBpbmdz
KGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAg
ICAgICAgICAgICBmcm9tIGNvbW1vbi9tZW1vcnkuYzoxNToKLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFp
bmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIz
NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
c2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWlu
ICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1
MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21f
ZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZh
dWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3Mzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRf
dW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJh
bnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
Mjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9z
aXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9o
YXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
ZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcx
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3Rh
cmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0
X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9h
ZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3Ry
dWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
Y2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5l
IGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQz
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1
NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJx
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6
NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJx
KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91
bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5f
ZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Np
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxs
b3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUw
MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0
IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2ko
eHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVn
X3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3Jlc291cmNlX3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTU4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21f
ZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNl
X3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3Vy
Y2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0
X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJj
ZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjYwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1f
YWx0cDJtaHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBz
dGF0aWMgaW5saW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1f
YWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1
NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRl
ZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
eGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21h
cCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL21lbW9yeS5jOjIzOgouL2luY2x1ZGUveGVu
L3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRl
L3hlbi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFu
Z2VzIHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2Fr
YSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhl
ciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAo
eyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KICBMRCAgICAgIGFy
Y2gvYXJtL3RlZS9idWlsdF9pbi5vCiAgQ0MgICAgICBhcmNoL2FybS9hbHRlcm5hdGl2ZS5v
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAg
ICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9hbHRlcm5hdGl2ZS5jOjEyOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxp
bmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGlu
dCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9t
Y3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0
bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZN
X1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1
NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2
b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21w
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAg
ICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYg
KG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMz
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25l
ZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9u
ZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzgg
fCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1l
c3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1l
c3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CmFyY2gvYXJtL2FsdGVy
bmF0aXZlLmM6IEluIGZ1bmN0aW9uIOKAmHBhdGNoX2FsdGVybmF0aXZl4oCZOgphcmNoL2Fy
bS9hbHRlcm5hdGl2ZS5jOjk0OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVuc2lnbmVkIGludOKA
mSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAgOTQgfCAgICAgZm9yICggaSA9
IDA7IGkgPCBucl9pbnN0OyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICBeCiAg
Q0MgICAgICBhcmNoL2FybS9ib290ZmR0Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUu
aDoxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAg
ICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2Jvb3RmZHQuYzoxMjoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEw
MSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nt
cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjY6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAg
ICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBp
ZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9t
Y3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0
bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo3Ogou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVf
RklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZ
UEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94c20veHNtLmg6MTksCiAgICAgICAgICAg
ICAgICAgZnJvbSBhcmNoL2FybS9ib290ZmR0LmM6MTY6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDox
Mjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1m
bl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21t
Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0
IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVy
eV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVu
Y3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9i
bG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGlj
IGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9h
c3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2Rl
diAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2Rldmlj
ZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNp
X3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2
cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMy
X3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJf
dCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lf
cHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21h
aW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMg
YWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFpbmluZm/i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIzNToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
c3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3Rh
cmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICplKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1MTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTY6
NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21fZGVmYXVs
dF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2
NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAz
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdW5tYXBy
ZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJhbnNmZXLi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGNvbW1vbi9tdWx0aWNhbGwubwou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9k
b21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3Bv
ZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3Rhcmdl
dOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5
X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92
ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQx
MiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Y2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVj
KHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGly
cSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAq
YmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1p
c3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0
X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWlu
dDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2
X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qg
Y2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFj
aGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1
MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hp
bmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19j
b3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJj
ZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBf
bWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRl
ZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBk
ZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFy
YW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1f
YWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0
cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUs
IHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50
X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBv
cCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRm
b3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJz
aW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL2Jvb3RmZHQuYzogSW4gZnVuY3Rpb24g
4oCYcHJvY2Vzc19tZW1vcnlfbm9kZeKAmToKYXJjaC9hcm0vYm9vdGZkdC5jOjI0NDo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuYW1l4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjQ0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBjaGFyICpuYW1lLCBpbnQgZGVwdGgsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2Jvb3RmZHQuYzoy
NDQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVwdGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNDQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGNvbnN0IGNoYXIgKm5hbWUsIGludCBkZXB0aCwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn5+fgph
cmNoL2FybS9ib290ZmR0LmM6IEluIGZ1bmN0aW9uIOKAmHByb2Nlc3NfcmVzZXJ2ZWRfbWVt
b3J54oCZOgphcmNoL2FybS9ib290ZmR0LmM6MjY5OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG5hbWXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjkgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBjaGFyICpuYW1lLCBp
bnQgZGVwdGgsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9ib290ZmR0LmM6MjY5OjY1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlcHRo4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjY5IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29u
c3QgY2hhciAqbmFtZSwgaW50IGRlcHRoLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn5+fgphcmNo
L2FybS9ib290ZmR0LmM6MjcwOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGFk
ZHJlc3NfY2VsbHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1MzIgYWRkcmVzc19jZWxscywgdTMy
IHNpemVfY2VsbHMpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fl5+fn5+fn5+fn5+fn4KYXJjaC9hcm0vYm9vdGZkdC5jOjI3MDo2Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXplX2NlbGxz4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjcwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdTMyIGFkZHJlc3NfY2VsbHMsIHUzMiBzaXplX2NlbGxzKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+Xn5+fn5+fn5+fgphcmNoL2FybS9ib290ZmR0LmM6IEluIGZ1bmN0aW9uIOKAmHBy
b2Nlc3NfY2hvc2VuX25vZGXigJk6CmFyY2gvYXJtL2Jvb3RmZHQuYzozNTg6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmFtZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM1OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3QgY2hh
ciAqbmFtZSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vYm9vdGZkdC5jOiBJbiBmdW5jdGlvbiDigJhw
cm9jZXNzX2RvbWFpbl9ub2Rl4oCZOgphcmNoL2FybS9ib290ZmR0LmM6NDIyOjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5hbWXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MjIgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IGNo
YXIgKm5hbWUsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2Jvb3RmZHQuYzogSW4gZnVuY3Rpb24g4oCY
ZWFybHlfc2Nhbl9ub2Rl4oCZOgphcmNoL2FybS9ib290ZmR0LmM6NDQyOjQxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NDIgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdm9pZCAqZGF0YSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgphcmNo
L2FybS9ib290ZmR0LmM6IEluIGZ1bmN0aW9uIOKAmGVhcmx5X3ByaW50X2luZm/igJk6CmFy
Y2gvYXJtL2Jvb3RmZHQuYzo0ODU6MjE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW50
4oCZIGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDQ4NSB8ICAgICBmb3IgKCBp
ID0gMCA7IGkgPCBtb2RzLT5ucl9tb2RzOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgXgphcmNoL2FybS9ib290ZmR0LmM6IEluIGZ1bmN0aW9uIOKAmHN3YXBfbWVtb3J5
X25vZGXigJk6CmFyY2gvYXJtL2Jvb3RmZHQuYzo1MjI6NjQ6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUyMiB8IHN0YXRp
YyB2b2lkIF9faW5pdCBzd2FwX21lbW9yeV9ub2RlKHZvaWQgKl9hLCB2b2lkICpfYiwgc2l6
ZV90IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+Xn5+fgphcmNoL2FybS9ib290ZmR0LmM6IElu
IGZ1bmN0aW9uIOKAmGJvb3RfZmR0X2luZm/igJk6CmFyY2gvYXJtL2Jvb3RmZHQuYzo1NTM6
MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDigJhpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDU1MyB8ICAgICBmb3IgKCBpID0gMDsgaSA8IG5yX3JzdmQ7IGkr
KyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1v
bi9tdWx0aWNhbGwuYzo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDog
SW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Y3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQg
Y3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBp
bmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90
IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8
ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEz
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8
ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVt
YXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFp
bi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEs
CiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vbXVsdGljYWxsLmM6ODoKLi9pbmNsdWRl
L3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1m
bl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
cmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDi
gJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9s
b2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVu
Y3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2Nr
KHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5p
dHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYs
IHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25l
ZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVf
YmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g
4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRl
X2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3Rf
bWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBp
bnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vz
c19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3Ry
dWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wg
aXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5o
OjE3LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL211bHRpY2FsbC5jOjk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lz
X2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25l
ZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9tdWx0
aWNhbGwuYzoxMjoKLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rv
bWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3Np
ZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21h
aW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVy
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9v
cCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9t
Y3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVh
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNo
bl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2hu
ICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4
NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5Nzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90
IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRf
Y29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFy
ZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYg
fCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3Bv
ZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
ODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5n
ZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2Vy
dmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0
IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQy
ODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3By
b2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9w
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGly
ceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9w
dF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5k
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21l
bV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMs
IHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBw
aW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90
IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Np
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYs
IHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQz
Ml90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91
bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0
X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2
OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hp
bmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNl
X3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0
X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUwOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1
bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODk6
NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9kZWZh
dWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
aHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
b3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHZt
X3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
aHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRf
dCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92
bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2
MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2Rl
LCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2Fj
Y2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94
ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpjb21tb24vbXVsdGljYWxsLmM6IEluIGZ1
bmN0aW9uIOKAmGRvX211bHRpY2FsbOKAmToKY29tbW9uL211bHRpY2FsbC5jOjkxOjQ4OiB3
YXJuaW5nOiBzdWdnZXN0IGJyYWNlcyBhcm91bmQgZW1wdHkgYm9keSBpbiBhbiDigJhpZuKA
mSBzdGF0ZW1lbnQgWy1XZW1wdHktYm9keV0KICAgOTEgfCAgICAgICAgICAgICAgICAgLyog
bm90aGluZywgYmVzdCBlZmZvcnQgb25seSAqLzsKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KICBDQyAgICAgIGFyY2gvYXJtL2Nw
dWVycmF0YS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2NwdS5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9jcHVlcnJhdGEuYzoyOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25y
X3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2ln
bmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGNvbW1vbi9ub3RpZmllci5vCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E
4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1f
U0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3Ry
dWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1v
bi9ub3RpZmllci5jOiBJbiBmdW5jdGlvbiDigJhub3RpZmllcl9jaGFpbl91bnJlZ2lzdGVy
4oCZOgpjb21tb24vbm90aWZpZXIuYzo0NzoyNzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhuaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA0NyB8ICAgICBzdHJ1Y3Qgbm90
aWZpZXJfaGVhZCAqbmgsIHN0cnVjdCBub3RpZmllcl9ibG9jayAqbikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2Fy
bS9jcHVlcnJhdGEuYzozOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8
ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEz
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8
ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVt
YXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gYXJj
aC9hcm0vY3B1ZXJyYXRhLmM6NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBh
Z2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6
NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRs
YmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjEx
OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNw
dW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEx
IHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ss
IHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9hcm0vY3B1ZXJyYXRhLmM6NjoKLi9p
bmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDigJhub19jb25maWdfcGFyYW3igJk6
Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCY
PzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6
ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5l
c3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwgICAgIGludCBs
ZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAg
ICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmFy
Y2gvYXJtL2NwdWVycmF0YS5jOiBJbiBmdW5jdGlvbiDigJhpbnN0YWxsX2JwX2hhcmRlbmlu
Z192ZWPigJk6CmFyY2gvYXJtL2NwdWVycmF0YS5jOjEyMjoxNTogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJh1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTIy
IHwgICAgIGlmICggc2xvdCA9PSAtMSApCiAgICAgIHwgICAgICAgICAgICAgICBefgphcmNo
L2FybS9jcHVlcnJhdGEuYzogSW4gZnVuY3Rpb24g4oCYYXJtX2VuYWJsZV93YTJfaGFuZGxp
bmfigJk6CmFyY2gvYXJtL2NwdWVycmF0YS5jOjM3Mzo3Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhhbHTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzMgfCB2b2lkIGFz
bWxpbmthZ2UgX19pbml0IGFybV9lbmFibGVfd2EyX2hhbmRsaW5nKGNvbnN0IHN0cnVjdCBh
bHRfaW5zdHIgKmFsdCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS9j
cHVlcnJhdGEuYzozNzQ6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3JpZ3B0
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3QgdWludDMyX3QgKm9yaWdwdHIsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+Xn5+fn5+fgphcmNoL2FybS9jcHVlcnJhdGEuYzogSW4gZnVuY3Rp
b24g4oCYY3B1X2VycmF0YV9jYWxsYmFja+KAmToKYXJjaC9hcm0vY3B1ZXJyYXRhLmM6NzEx
OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5mYuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDcxMSB8IHN0YXRpYyBpbnQgY3B1X2VycmF0YV9jYWxsYmFjayhzdHJ1Y3Qg
bm90aWZpZXJfYmxvY2sgKm5mYiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vY3B1ZXJyYXRhLmM6
NzEzOjM4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhjcHXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA3MTMgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdm9p
ZCAqaGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
Xn5+fgphcmNoL2FybS9jcHVlcnJhdGEuYzogQXQgdG9wIGxldmVsOgphcmNoL2FybS9jcHVl
cnJhdGEuYzo3NTY6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5p
bmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDc1NiB8IF9f
aW5pdGNhbGwoY3B1X2VycmF0YV9ub3RpZmllcl9pbml0KTsKICAgICAgfCBefn5+fn5+fn5+
CiAgQ0MgICAgICBjb21tb24vcGFnZV9hbGxvYy5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL2RvbWFpbl9wYWdlLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24v
cGFnZV9hbGxvYy5jOjEyMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50
IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAg
fCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDox
Mzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAg
fCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFy
Y2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcg
bnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2Rl
bChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0
aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hf
ZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6IEluIGZ1bmN0aW9u
IOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNsdWRlL3hlbi9kb21haW5f
cGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZh4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2b2lkIHVubWFwX2RvbWFp
bl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gY29t
bW9uL3BhZ2VfYWxsb2MuYzoxMjQ6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlv
biDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2Fj
Y2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJv
bV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEg
fCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3Qg
KnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lk
IGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlf
cG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6
NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2Rl
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGlu
ZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0
X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19w
dl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEx
MzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25z
dCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxNzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vcGFnZV9hbGxvYy5jOjEz
MjoKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDigJhub19jb25maWdfcGFy
YW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdhcm5pbmc6IG9wZXJhbmQg
b2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g
4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNp
Z25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwgICAg
IGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihz
KTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+CmNvbW1vbi9wYWdlX2FsbG9jLmM6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2hlYXBfcGFn
ZXPigJk6CmNvbW1vbi9wYWdlX2FsbG9jLmM6OTY3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGxv
bmcgdW5zaWduZWQgaW504oCZIGFuZCDigJhsb25nIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgOTY3IHwgICAgIGlmICggKG91dHN0YW5kaW5nX2NsYWltcyArIHJlcXVlc3QgPiB0b3Rh
bF9hdmFpbF9wYWdlcykgJiYKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBeCmNvbW1vbi9wYWdlX2FsbG9jLmM6MTAyMjoyMDogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJh1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAx
MDIyIHwgICAgIGZvciAoIGkgPSAwOyBpIDwgKDEgPDwgb3JkZXIpOyBpKysgKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9wYWdlX2FsbG9jLmM6IEluIGZ1bmN0aW9u
IOKAmHJlc2VydmVfb2ZmbGluZWRfcGFnZeKAmToKY29tbW9uL3BhZ2VfYWxsb2MuYzoxMTI2
OjI3OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFst
V3NpZ24tY29tcGFyZV0KIDExMjYgfCAgICAgICAgIHdoaWxlICggY3VyX29yZGVyIDwgaGVh
ZF9vcmRlciApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9w
YWdlX2FsbG9jLmM6IEluIGZ1bmN0aW9uIOKAmGZyZWVfaGVhcF9wYWdlc+KAmToKY29tbW9u
L3BhZ2VfYWxsb2MuYzoxNDcyOjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVuc2lnbmVkIGludOKA
mSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE0NzIgfCAgICAgZm9yICggaSA9
IDA7IGkgPCAoMSA8PCBvcmRlcik7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
IF4KY29tbW9uL3BhZ2VfYWxsb2MuYzogSW4gZnVuY3Rpb24g4oCYYXZhaWxfaGVhcF9wYWdl
c+KAmToKY29tbW9uL3BhZ2VfYWxsb2MuYzoxOTM5OjI0OiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVu
c2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE5MzkgfCAg
ICAgICAgICAgICBpZiAoIChub2RlID09IC0xKSB8fCAobm9kZSA9PSBpKSApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICBefgpjb21tb24vcGFnZV9hbGxvYy5jOiBJbiBmdW5j
dGlvbiDigJhzbXBfc2NydWJfaGVhcF9wYWdlc+KAmToKY29tbW9uL3BhZ2VfYWxsb2MuYzoy
MDE0OjU0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KIDIwMTQgfCAgICAgICAgIGlmICggci0+cmVtICYmIChjcHVt
YXNrX3dlaWdodCgmci0+Y3B1cykgLSAxID09IGNwdV9pZHgpICkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CmNvbW1v
bi9wYWdlX2FsbG9jLmM6IEluIGZ1bmN0aW9uIOKAmGZpbmRfbm9uX3NtdOKAmToKY29tbW9u
L3BhZ2VfYWxsb2MuYzoyMDMyOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5v
ZGXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDIwMzIgfCBzdGF0aWMgaW50IF9faW5pdCBm
aW5kX25vbl9zbXQodW5zaWduZWQgaW50IG5vZGUsIGNwdW1hc2tfdCAqZGVzdCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KICBD
QyAgICAgIGFyY2gvYXJtL2NwdWZlYXR1cmUubwpjb21tb24vcGFnZV9hbGxvYy5jOiBJbiBm
dW5jdGlvbiDigJhmcmVlX2RvbWhlYXBfcGFnZXPigJk6CmNvbW1vbi9wYWdlX2FsbG9jLmM6
MjUwNzoyNDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAyNTA3IHwgICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgx
IDw8IG9yZGVyKTsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgIF4KY29t
bW9uL3BhZ2VfYWxsb2MuYzoyNTI0OjI4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVuc2lnbmVkIGlu
dOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDI1MjQgfCAgICAgICAgICAg
ICBmb3IgKCBpID0gMDsgaSA8ICgxIDw8IG9yZGVyKTsgaSsrICkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9wYWdlX2FsbG9jLmM6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VhbGxvY19pbmZv4oCZOgpjb21tb24vcGFnZV9hbGxvYy5jOjI2MDM6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYa2V54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAyNjAzIHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgcGFnZWFsbG9jX2luZm8odW5zaWduZWQg
Y2hhciBrZXkpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL3BhZ2VfYWxsb2MuYzogQXQgdG9wIGxldmVsOgpj
b21tb24vcGFnZV9hbGxvYy5jOjI2MzU6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5v
dCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9u
XQogMjYzNSB8IF9faW5pdGNhbGwocGFnZWFsbG9jX2tleWhhbmRsZXJfaW5pdCk7CiAgICAg
IHwgXn5+fn5+fn5+fgpjb21tb24vcGFnZV9hbGxvYy5jOjI2ODM6MTogd2FybmluZzog4oCY
c3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0
eWxlLWRlY2xhcmF0aW9uXQogMjY4MyB8IF9faW5pdGNhbGwocmVnaXN0ZXJfaGVhcF90cmln
Z2VyKTsKICAgICAgfCBefn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9jcHVmZWF0
dXJlLmM6MTE6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAg
ICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9j
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFy
KGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo2OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3Np
Ymxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVk
IGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
ZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5
c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9j
ay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L3N2
ZS5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vY3B1ZmVhdHVyZS5jOjEz
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NB
VkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWlu
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4v
aW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5v
ZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3By
ZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9s
aXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6
Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1
c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRf
cGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJt
X3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5
OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAq
Z21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25f
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVh
ZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMg
aW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0
aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWdu
ZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9u
IOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6
Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9t
YWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNo
L2FybS9jcHVmZWF0dXJlLmM6IEluIGZ1bmN0aW9uIOKAmGhhc19zYl9pbnN0cnVjdGlvbuKA
mToKYXJjaC9hcm0vY3B1ZmVhdHVyZS5jOjIxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGVudHJ54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIxIHwgc3RhdGljIGJv
b2wgaGFzX3NiX2luc3RydWN0aW9uKGNvbnN0IHN0cnVjdCBhcm1fY3B1X2NhcGFiaWxpdGll
cyAqZW50cnkpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KYXJjaC9hcm0vY3B1ZmVhdHVy
ZS5jOiBBdCB0b3AgbGV2ZWw6CmFyY2gvYXJtL2NwdWZlYXR1cmUuYzoyNDE6MTogd2Fybmlu
Zzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1X
b2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDI0MSB8IF9faW5pdGNhbGwoY3JlYXRlX2RvbWFp
bl9jcHVpbmZvKTsKICAgICAgfCBefn5+fn5+fn5+CiAgQ0MgICAgICBhcmNoL2FybS9kZWNv
ZGUubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1ZXN0X2Fj
Y2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1ZXN0X2Fj
Y2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZGVjb2RlLmM6MTE6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhj
cHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3Mo
dW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
Ojc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KICBDQyAgICAgIGNvbW1vbi9wZHgubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEu
aDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6
MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZCht
Zm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
bS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwg
dm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVj
dCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVl
cnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlu
bGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVf
YmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6
NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
YXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjgz
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBj
aV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQz
Ml90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMy
X3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNp
X3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAg
ICAgICAgICAgICAgZnJvbSBjb21tb24vcGR4LmM6MTk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2Rv
bWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRp
YyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIENDICAgICAgY29tbW9uL3Bl
cmZjLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21w
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19p
bl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAg
ICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdl
X3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vcGVy
ZmMuYzozOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jdXJyZW50Lmg6OCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NjoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MywKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9wZXJmYy5jOjc6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAg
ICAgICAgICAgICBmcm9tIGNvbW1vbi9wZXJmYy5jOjg6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlz
dF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
MuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBm
dW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Zmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1
bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fn5+fn5+fn5+fn4KICBDQyAgICAgIGNvbW1vbi9wcmVlbXB0Lm8KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3Vlc3RfYWNjZXNzLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZ3Vlc3RfYWNjZXNzLmg6MTAsCiAgICAgICAgICAg
ICAgICAgZnJvbSBjb21tb24vcGVyZmMuYzo5OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4g
ZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBt
Zm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1Ogou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEz
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJz
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9w
YWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1f
dHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3Rf
bWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6
NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3Bv
bGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpn
bXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGFyY2gvYXJt
L2RldmljZS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jdXJyZW50Lmg6OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS9hc20tZ2VuZXJpYy9wZXJjcHUuaDo4LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vcGVyY3B1Lmg6MSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyY3B1Lmg6MzAsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3ByZWVtcHQuaDoxMywKICAgICAgICAgICAgICAgICBmcm9t
IGNvbW1vbi9wcmVlbXB0LmM6MjI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5l
IGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vaXJxLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9wcmVlbXB0
LmM6MjM6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAg
ICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAg
ICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2Nw
dW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1
MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g
4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJi
aXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KY29tbW9uL3BlcmZjLmM6IEluIGZ1bmN0aW9uIOKAmHBlcmZjX3ByaW50YWxs4oCZOgpj
b21tb24vcGVyZmMuYzozMTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhrZXni
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCB2b2lkIGNmX2NoZWNrIHBlcmZjX3By
aW50YWxsKHVuc2lnbmVkIGNoYXIga2V5KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zcGlubG9jay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOjM2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9p
cnEuaDo1OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9f
SFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NB
VkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+
fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBv
ZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hW
TV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+
fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9hc20tZ2VuZXJpYy9kZXZpY2UuaDo0MiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL2RldmljZS5oOjEs
CiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9kZXZpY2UuYzoxMToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0
aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEz
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhy
Y3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcndsb2NrLmg6NjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAg
fCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDox
Mzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAg
fCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3J3bG9jay5oOjc6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gYXJj
aC9hcm0vZGV2aWNlLmM6MTI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25l
ZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
cHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdl
X2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJm
bHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29s
IHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3Qg
KnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4
NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdl
dF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBw
Mm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL2lvY2FwLmg6MTAsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9kZXZp
Y2UuYzoxNToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19u
aWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMg
aW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUg
dm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXAp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBj
aV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBp
bmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25l
ZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2ln
bmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9f
bXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCY
aXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4o
Y29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAg
ICBjb21tb24vcmFuZG9tLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAg
ICAgICAgICBmcm9tIGNvbW1vbi9yYW5kb20uYzozOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3Rh
dGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
Y29tbW9uL3JhbmRvbS5jOiBBdCB0b3AgbGV2ZWw6CmNvbW1vbi9yYW5kb20uYzozOToxOiB3
YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlv
biBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgIDM5IHwgX19pbml0Y2FsbChpbml0X2Jv
b3RfcmFuZG9tKTsKICAgICAgfCBefn5+fn5+fn5+CiAgQ0MgICAgICBjb21tb24vcmFuZ2Vz
ZXQubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vcmFuZ2VzZXQuYzoxMDoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3go
dm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nt
cC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAg
ICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAg
IGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlu
bGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1
IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21h
aW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUg
bm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25l
ZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
cHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdl
X2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJm
bHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9j
a190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29s
IHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3Qg
KnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4
NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdl
dF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBw
Mm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5
X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Z21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5
ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2ln
bl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9y
ZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRp
YyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBi
b29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rp
b24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEz
MTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9k
b21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vcmFuZ2VzZXQuYzoxMzoKLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dl
dGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50
IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
ZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0
bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9z
ZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V0
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9x
dWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
aW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRf
cG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNo
YW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xl
X2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRl
ZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1
dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFp
bl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFp
bl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWlu
X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRh
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVj
dCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Bl
cm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4
X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxs
b3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjUwMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1
Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vf
dW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6
NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29y
ZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBf
cGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9k
ZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9y
ZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19v
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1f
cGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2
MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIz
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVs
dF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90
IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
Ojc0ODo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291
cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgogIENDICAgICAgYXJjaC9hcm0vZG0ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2d1ZXN0X2FjY2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2d1ZXN0X2FjY2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZy
b20gYXJjaC9hcm0vZG0uYzo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBp
bnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6
MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwog
ICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQog
ICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192
YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJl
ZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJ
biBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6
NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90
IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6
MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNf
aW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3Rt
YXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAg
ICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFn
ZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFk
X2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21l
bV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1f
YWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291
cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lk
IHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR4
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRv
bWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90
IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9j
ayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlu
bGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgogIENDICAgICAgY29tbW9uL3JhZGl4LXRyZWUubwpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBm
dW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMy
Yml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxsLmg6MjIs
CiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9kbS5jOjg6Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRk
b21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBj
bWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0
NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRv
bWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWN0bCh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGwo
eHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjYx
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFkY29uc29sZSh4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90IGlkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2Vu
ZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5
MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9yZXNldOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzAzOjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFncykKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dy
YW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwOToxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9t
YWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3NldHVw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50
X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyMToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2NvcHnigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfcXVl
cnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMzM6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2lu
aXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzcxOjUyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfcG9kX3RhcmdldCh4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc2V0X3Bv
ZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFu
Z2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzgxOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4MSB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2UoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlf
c3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTM6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0
IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDA2OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21fcGh5c21hcOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxODox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9wYWdlc+KAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY29uc29sZV9p
b+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQz
MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzODo0Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRk
b3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NDQgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21hcF9kb21haW5f
cGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX2lyceKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91bm1hcF9kb21haW5f
cGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9p
cnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3
NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODM6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
eGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJt
aXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODkgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCB1aW50OF90
IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fcGVybWlzc2lvbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjUwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1aW50MTZfdCBzdGFydCwKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBj
b25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdf
cGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3Vu
cGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTU4OjU2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUo
eHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNv
dXJjZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3Bj
aeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jl
c291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVm
YXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVz
b3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHlwZnNfb3Di
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3Bh
cmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEy
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3ZudW1haW5mbyh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJvbOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
bWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRf
dCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3
NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBk
ZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3
NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJj
ZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJu
YXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90
aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoz
NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0
LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3JhZGl4LXRyZWUuYzoyMjoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6Mzc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgpjb21tb24vcmFkaXgtdHJlZS5jOiBJbiBmdW5jdGlvbiDigJhyY3Vfbm9kZV9hbGxvY+KA
mToKY29tbW9uL3JhZGl4LXRyZWUuYzo1NTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhhcmfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTUgfCBzdGF0aWMgc3RydWN0
IHJhZGl4X3RyZWVfbm9kZSAqY2ZfY2hlY2sgcmN1X25vZGVfYWxsb2Modm9pZCAqYXJnKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn4KY29tbW9uL3JhZGl4LXRyZWUuYzogSW4gZnVuY3Rpb24g4oCY
cmN1X25vZGVfZnJlZeKAmToKY29tbW9uL3JhZGl4LXRyZWUuYzo2ODo3Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhhcmfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNjgg
fCBzdGF0aWMgdm9pZCBjZl9jaGVjayByY3Vfbm9kZV9mcmVlKHN0cnVjdCByYWRpeF90cmVl
X25vZGUgKm5vZGUsIHZvaWQgKmFyZykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpj
b21tb24vcmFkaXgtdHJlZS5jOiBJbiBmdW5jdGlvbiDigJhyYWRpeF90cmVlX25vZGVfZGVz
dHJveeKAmToKY29tbW9uL3JhZGl4LXRyZWUuYzo2ODQ6MjM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
Njg0IHwgICAgICAgICBmb3IgKGkgPSAwOyBpIDwgUkFESVhfVFJFRV9NQVBfU0laRTsgaSsr
KSB7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3JhZGl4LXRyZWUu
YzogQXQgdG9wIGxldmVsOgpjb21tb24vcmFkaXgtdHJlZS5jOjc1NzoxOiB3YXJuaW5nOiDi
gJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQt
c3R5bGUtZGVjbGFyYXRpb25dCiAgNzU3IHwgcHJlc21wX2luaXRjYWxsKHJhZGl4X3RyZWVf
aW5pdF9tYXhpbmRleCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBhcmNo
L2FybS9kb20wbGVzcy1idWlsZC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jdXJyZW50Lmg6OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9hc20tZ2VuZXJpYy9wZXJjcHUuaDo4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vcGVyY3B1Lmg6MSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyY3B1Lmg6MzAsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RldmljZV90cmVlLmg6MjIsCiAgICAgICAgICAg
ICAgICAgZnJvbSBhcmNoL2FybS9kb20wbGVzcy1idWlsZC5jOjI6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3Pi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNw
dSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2OgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAg
ICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAg
aWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUg
fCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2Rv
bWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNj
dGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo0MywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NzoK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhE
RUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUo
RU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+
fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAg
ICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAq
aCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g
4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9U
WVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+
fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTIsCiAgICAgICAg
ICAgICAgICAgZnJvbSBhcmNoL2FybS9kb20wbGVzcy1idWlsZC5jOjQ6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4u
aDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9h
bHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9n
Zm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQs
IHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFy
Y2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8z
MmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEg
fCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxNzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxl
Lmg6MzEsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9kb20wbGVzcy1idWlsZC5j
OjU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDogSW4gZnVuY3Rpb24g
4oCYZ250dGFiX21hcmtfZGlydHnigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRf
dGFibGUuaDoyMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlubGluZSB2b2lkIGdudHRhYl9tYXJr
X2RpcnR5KHN0cnVjdCBkb21haW4gKmQsIG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MjA6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlu
bGluZSB2b2lkIGdudHRhYl9tYXJrX2RpcnR5KHN0cnVjdCBkb21haW4gKmQsIG1mbl90IG1m
bikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFi
bGUuaDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX2hvc3RfbWFwcGluZ19nZXRfcGFnZV90eXBl
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6Mjc6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcm/igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjcgfCBzdGF0aWMgaW5saW5lIGJvb2wgZ250dGFiX2hvc3RfbWFwcGluZ19nZXRfcGFnZV90
eXBlKGJvb2wgcm8sCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyODo3
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyOCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqbGQsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6Mjk6
NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjkgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKnJkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOiBJ
biBmdW5jdGlvbiDigJhnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MzQ6NzA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBpbmdzKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gYXJjaC9hcm0vZG9tMGxlc3MtYnVpbGQuYzo5OgouL2luY2x1ZGUveGVuL3NlcmlhbC5o
OiBBdCB0b3AgbGV2ZWw6Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5pbmc6
IOKAmGlubGluZeKAmSBpcyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29s
ZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhjaV9k
YmNfdWFydF9pbml0KHZvaWQpIHt9CiAgICAgIHwgXn5+fn5+CmFyY2gvYXJtL2RvbTBsZXNz
LWJ1aWxkLmM6IEluIGZ1bmN0aW9uIOKAmGlzX2RvbTBsZXNzX21vZGXigJk6CmFyY2gvYXJt
L2RvbTBsZXNzLWJ1aWxkLmM6Mjg6MjE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW50
4oCZIGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogICAyOCB8ICAgICBmb3IgKCBp
ID0gMCA7IGkgPCBtb2RzLT5ucl9tb2RzIDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgIF4KYXJjaC9hcm0vZG9tMGxlc3MtYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYZG9t
YWluX2hhbmRsZV9kdGJfYm9vdG1vZHVsZeKAmToKYXJjaC9hcm0vZG9tMGxlc3MtYnVpbGQu
Yzo1MTk6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDUxOSB8IHN0YXRpYyBpbnQgX19pbml0IGRvbWFpbl9oYW5kbGVfZHRi
X2Jvb3Rtb2R1bGUoc3RydWN0IGRvbWFpbiAqZCwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9h
cm0vZG9tMGxlc3MtYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYY3JlYXRlX2RvbVVz4oCZOgph
cmNoL2FybS9kb20wbGVzcy1idWlsZC5jOjg4MTozNTogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgODgxIHwgICAg
ICAgICBpZiAoIChtYXhfaW5pdF9kb21pZCArIDEpID49IERPTUlEX0ZJUlNUX1JFU0VSVkVE
ICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoxODoKLi9pbmNs
dWRlL3hlbi9rZXJuZWwuaDo0MzoyMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1aW50MzJfdOKAmSB7
YWthIOKAmHVuc2lnbmVkIGludOKAmX0gYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgIDQzIHwgI2RlZmluZSBNQVgoeCx5KSAoKHgpID4gKHkpID8gKHgpIDogKHkpKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJtL2RvbTBsZXNzLWJ1aWxkLmM6
OTM4OjM4OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYTUFY4oCZCiAgOTM4IHwg
ICAgICAgICAgICAgICAgIGRfY2ZnLmFyY2gubnJfc3BpcyA9IE1BWChkX2NmZy5hcmNoLm5y
X3NwaXMsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+
fgouL2luY2x1ZGUveGVuL2tlcm5lbC5oOjQzOjM3OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKA
mD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGludOKAmSB0byDigJh1aW50MzJf
dOKAmSB7YWthIOKAmHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBv
dGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAgNDMgfCAjZGVmaW5lIE1BWCh4LHkp
ICgoeCkgPiAoeSkgPyAoeCkgOiAoeSkpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXn5+CmFyY2gvYXJtL2RvbTBsZXNzLWJ1aWxkLmM6OTM4OjM4OiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYTUFY4oCZCiAgOTM4IHwgICAgICAgICAg
ICAgICAgIGRfY2ZnLmFyY2gubnJfc3BpcyA9IE1BWChkX2NmZy5hcmNoLm5yX3NwaXMsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fgogIENDICAg
ICAgY29tbW9uL3JjdXBkYXRlLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9yY3Vw
ZGF0ZS5jOjM3OgouL2luY2x1ZGUveGVuL3BhcmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2Nv
bmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzog
b3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGlu
dOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVl
IHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAx
OTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDog
c3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9yY3VwZGF0ZS5j
OjM4OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g
4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxp
bmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1
bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9yY3Vw
ZGF0ZS5jOjM5OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAg
ICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxv
Y19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBj
b21tb24vcmN1cGRhdGUuYzo0MDoKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIw
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
dW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gY29t
bW9uL3JjdXBkYXRlLmM6NDE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDi
gJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5
IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEw
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQg
bG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVk
IGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9s
aXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IElu
IGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxi
Zmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXAp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRf
cGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJt
X3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5
OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAq
Z21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25f
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVh
ZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMg
aW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0
aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWdu
ZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgogIENDICAgICAgYXJjaC9hcm0vZG9tYWluLm8KLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6
Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlz
X2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KY29tbW9uL3JjdXBkYXRlLmM6IEluIGZ1bmN0aW9u
IOKAmHJjdV9iYXJyaWVyX2NhbGxiYWNr4oCZOgpjb21tb24vcmN1cGRhdGUuYzoxNzA6NjA6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE3MCB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIHJjdV9iYXJyaWVyX2NhbGxiYWNr
KHN0cnVjdCByY3VfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL3JjdXBk
YXRlLmM6IEluIGZ1bmN0aW9uIOKAmHJjdV9pZGxlX3RpbWVyX2hhbmRsZXLigJk6CmNvbW1v
bi9yY3VwZGF0ZS5jOjU3ODo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc4IHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sg
cmN1X2lkbGVfdGltZXJfaGFuZGxlcih2b2lkKiBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgpjb21tb24vcmN1
cGRhdGUuYzogSW4gZnVuY3Rpb24g4oCYcmN1X2NoZWNrX2NhbGxiYWNrc+KAmToKY29tbW9u
L3JjdXBkYXRlLmM6NTkwOjMwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU5MCB8IHZvaWQgcmN1X2NoZWNrX2NhbGxiYWNr
cyhpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+CmNv
bW1vbi9yY3VwZGF0ZS5jOiBJbiBmdW5jdGlvbiDigJhjcHVfY2FsbGJhY2vigJk6CmNvbW1v
bi9yY3VwZGF0ZS5jOjY0NToyODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZmLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NDUgfCAgICAgc3RydWN0IG5vdGlmaWVyX2Js
b2NrICpuZmIsIHVuc2lnbmVkIGxvbmcgYWN0aW9uLCB2b2lkICpoY3B1KQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYXNtLWdlbmVyaWMvcGVyY3B1Lmg6OCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL3BlcmNw
dS5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmNwdS5oOjMw
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ncmFudF90YWJsZS5oOjI3LAogICAg
ICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZG9tYWluLmM6NDoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEw
MSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nt
cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjY6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAg
ICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBp
ZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9t
Y3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0
bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo3Ogou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVf
RklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERF
Q0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShF
TkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+
fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZ
UEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUu
aDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ncmFudF90YWJsZS5o
OjMxOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKA
mToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxp
bmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNf
aW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVu
Y3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBw
YWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4
OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0
bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNr
LCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+
fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5o
OjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFk
X2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhy
Y3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBi
b29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRf
c3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2
MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2Zu
LCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1f
ZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQx
Nzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21t
b24vcndsb2NrLm8KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9j
ayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlu
bGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8
IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3Jl
YWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNp
X3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2
cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3Qg
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBk
YXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAw
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJv
Y2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNy
ZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FzbS1nZW5l
cmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9yd2xvY2su
YzoxOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g
4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxp
bmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBm
dW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMy
Yml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDi
gJhnbnR0YWJfbWFya19kaXJ0eeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90
YWJsZS5oOjIwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX21hcmtf
ZGlydHkoc3RydWN0IGRvbWFpbiAqZCwgbWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyMDo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0IGRvbWFpbiAqZCwgbWZuX3QgbWZu
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJs
ZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfaG9zdF9tYXBwaW5nX2dldF9wYWdlX3R5cGXi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyNzo1OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyb+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
NyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBnbnR0YWJfaG9zdF9tYXBwaW5nX2dldF9wYWdlX3R5
cGUoYm9vbCBybywKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjI4Ojc1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICpsZCwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyOTo3
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyOSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqcmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6IElu
IGZ1bmN0aW9uIOKAmGdudHRhYl9yZWxlYXNlX2hvc3RfbWFwcGluZ3PigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDozNDo3MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM0IHwgc3RhdGljIGlu
bGluZSBib29sIGdudHRhYl9yZWxlYXNlX2hvc3RfbWFwcGluZ3MoY29uc3Qgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3J3bG9jay5oOjY6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAg
IHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6
MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAg
IHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAgICAgICAgICAgICAg
ZnJvbSBhcmNoL2FybS9kb21haW4uYzo2OgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjQ6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0ZG9tYWluaW5mb+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rv
bWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjM1OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNt
ZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNj
dGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V0
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDYgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjUxOjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3
MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBl
dnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkx
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0
Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2
OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2Rv
bWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFy
Z2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
bWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRp
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlz
bWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dt
Zm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWlu
ICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdl
cyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9j
b25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21k
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9r
ZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Vu
bWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVu
X2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9p
cnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmlu
ZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0
X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9j
b25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJf
dCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25f
ZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRw
YXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVz
b3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0
NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVs
dF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vf
cGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBz
dGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3Qg
ZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2Nv
cmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9n
c2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGlu
dCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21p
c2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2Zm
bGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlw
ZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
dW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0
X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9y
bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Rt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZG1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lv
buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBv
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFp
bl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSBhcmNoL2FybS9kb21haW4uYzoyMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9l
dmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9kb21haW4uYzozMzoKYXJjaC9h
cm0vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJhkb21haW5fdnBjaV9pbml04oCZOgphcmNoL2Fy
bS92cGNpLmg6MjI6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyMiB8IHN0YXRpYyBpbmxpbmUgaW50IGRvbWFpbl92cGNp
X2luaXQoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJhkb21haW5fdnBjaV9nZXRfbnVtX21taW9faGFuZGxlcnPigJk6CmFyY2gvYXJt
L3ZwY2kuaDoyNzo3Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDI3IHwgc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgZG9t
YWluX3ZwY2lfZ2V0X251bV9tbWlvX2hhbmRsZXJzKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vZG9tYWluLmM6IEluIGZ1bmN0
aW9uIOKAmGNvbnRpbnVlX3J1bm5pbmfigJk6CmFyY2gvYXJtL2RvbWFpbi5jOjMzNjozNjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYW1l4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzM2IHwgdm9pZCBjb250aW51ZV9ydW5uaW5nKHN0cnVjdCB2Y3B1ICpzYW1lKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2Fy
bS9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYc3luY192Y3B1X2V4ZWNzdGF0ZeKAmToKYXJj
aC9hcm0vZG9tYWluLmM6MzQ2OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNDYgfCB2b2lkIHN5bmNfdmNwdV9leGVjc3Rh
dGUoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2RvbWFpbi5jOiBJbiBmdW5jdGlvbiDigJhkdW1wX3Bh
Z2VmcmFtZV9pbmZv4oCZOgphcmNoL2FybS9kb21haW4uYzo0OTY6NDE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NiB8IHZv
aWQgZHVtcF9wYWdlZnJhbWVfaW5mbyhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2RvbWFpbi5j
OiBJbiBmdW5jdGlvbiDigJhhbGxvY192Y3B1X3N0cnVjdOKAmToKYXJjaC9hcm0vZG9tYWlu
LmM6NTExOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1MTEgfCBzdHJ1Y3QgdmNwdSAqYWxsb2NfdmNwdV9zdHJ1Y3QoY29u
c3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9kb21haW4uYzogSW4gZnVu
Y3Rpb24g4oCYYXJjaF9kb21haW5fY3JlYXRl4oCZOgphcmNoL2FybS9kb21haW4uYzo2OTY6
Mzc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZmxhZ3PigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA2OTYgfCAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBm
bGFncykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
fgphcmNoL2FybS9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9kb21haW5fc2h1dGRv
d27igJk6CmFyY2gvYXJtL2RvbWFpbi5jOjg1NTo0Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgODU1IHwgdm9pZCBhcmNoX2Rv
bWFpbl9zaHV0ZG93bihzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9kb21haW4uYzogSW4gZnVu
Y3Rpb24g4oCYYXJjaF9kb21haW5fcGF1c2XigJk6CmFyY2gvYXJtL2RvbWFpbi5jOjg1OToz
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgODU5IHwgdm9pZCBhcmNoX2RvbWFpbl9wYXVzZShzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgphcmNoL2Fy
bS9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9kb21haW5fdW5wYXVzZeKAmToKYXJj
aC9hcm0vZG9tYWluLmM6ODYzOjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA4NjMgfCB2b2lkIGFyY2hfZG9tYWluX3VucGF1
c2Uoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYYXJj
aF9kb21haW5fc29mdF9yZXNldOKAmToKYXJjaC9hcm0vZG9tYWluLmM6ODY3OjQzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA4
NjcgfCBpbnQgYXJjaF9kb21haW5fc29mdF9yZXNldChzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9h
cm0vZG9tYWluLmM6IEluIGZ1bmN0aW9uIOKAmGFyY2hfaW5pdGlhbGlzZV92Y3B14oCZOgph
cmNoL2FybS9kb21haW4uYzo5NzM6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDk3MyB8IGludCBhcmNoX2luaXRpYWxpc2Vf
dmNwdShzdHJ1Y3QgdmNwdSAqdiwgWEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh2b2lkKSBhcmcp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCmFyY2gv
YXJtL2RvbWFpbi5jOjk3Mzo3MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhhcmfi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA5NzMgfCBpbnQgYXJjaF9pbml0aWFsaXNlX3Zj
cHUoc3RydWN0IHZjcHUgKnYsIFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0odm9pZCkgYXJnKQph
cmNoL2FybS9kb21haW4uYzogSW4gZnVuY3Rpb24g4oCYdmNwdV9ibG9ja191bmxlc3NfZXZl
bnRfcGVuZGluZ+KAmToKYXJjaC9hcm0vZG9tYWluLmM6MTE4NDo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTg0IHwgdm9p
ZCB2Y3B1X2Jsb2NrX3VubGVzc19ldmVudF9wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9pcnEuaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gY29tbW9uL3J3bG9jay5jOjI6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Y29tbW9uL3J3bG9jay5jOiBJbiBmdW5jdGlvbiDigJhxdWV1ZV93cml0ZV9sb2NrX3Nsb3dw
YXRo4oCZOgpjb21tb24vcndsb2NrLmM6OTQ6NTA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1MzLigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFy
ZV0KICAgOTQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNudHMgfCBfUVdfV0FJ
VElORykgPT0gY250cykgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBefgogIENDICAgICAgY29tbW9uL3NodXRkb3duLm8KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9zaHV0ZG93bi5jOjM6Ci4vaW5jbHVkZS94ZW4v
cGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1ZGUv
eGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5n
ZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7YWth
IOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVy
IG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/ICh7
IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSBjb21tb24vc2h1dGRvd24uYzo0OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlu
bGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KICBDQyAg
ICAgIGNvbW1vbi9zb2Z0aXJxLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9u
IOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcnds
b2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN0
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8
ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9f
bmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGlj
IGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAgICAgICAgZnJvbSBj
b21tb24vc29mdGlycS5jOjEzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBp
bnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
ZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xp
c3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4g
ZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJm
bHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNw
dW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlv
biDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVh
ZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4g
ZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5s
b2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVs
YXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMx
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJj
ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3
NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUg
KnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNp
Z25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1f
ZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9p
ZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9p
ZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVt
YXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVj
dG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9j
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1v
cnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwg
cGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBh
Z2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAg
ICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxi
Lmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1l
c3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2Zp
bHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAg
ICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zb2Z0aXJxLmM6MTU6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251
bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9f
bmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4
LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9t
YWluLmg6MTU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1
X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJ
biBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2
XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5o
OjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g
4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3Nf
ZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29s
IHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL3NodXRkb3duLmM6MTA6
Ci4vaW5jbHVkZS94ZW4va2V4ZWMuaDogSW4gZnVuY3Rpb24g4oCYc2V0X2tleGVjX2NyYXNo
X2FyZWFfc2l6ZeKAmToKLi9pbmNsdWRlL3hlbi9rZXhlYy5oOjkzOjUwOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHN5c3RlbV9yYW3igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgOTMgfCBzdGF0aWMgaW5saW5lIHZvaWQgc2V0X2tleGVjX2NyYXNoX2FyZWFfc2l6ZSh1
NjQgc3lzdGVtX3JhbSkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+Xn5+fn5+fn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0
IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZf
dCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBn
Zm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92
bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3Rh
dGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0
ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9t
ZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9w
b2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBm
dW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMy
Yml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KICBDQyAgICAgIGNvbW1vbi9zbXAubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL2hhcmRpcnEuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2dlbmVyYXRlZC9hc20vaGFyZGlycS5oOjEsCiAgICAgICAgICAgICAgICAgZnJv
bSBjb21tb24vc21wLmM6MTk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAg
IHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6
MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAg
IHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo2
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCY
Y3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdz
KHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
cHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3NtcC5jOjIx
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NB
VkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQ
RShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+
fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KICBDQyAgICAgIGNvbW1vbi9zcGlubG9jay5vCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2NwdS5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSBjb21tb24vc3BpbmxvY2suYzoxOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KICBDQyAg
ICAgIGNvbW1vbi9zdG9wX21hY2hpbmUubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9pcnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3NwaW5sb2Nr
LmM6MzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9pcnEuaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9t
IGNvbW1vbi9zdG9wX21hY2hpbmUuYzoyMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSBjb21tb24vc3BpbmxvY2suYzo1OgouL2luY2x1ZGUveGVuL3Bh
cmFtLmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hl
bi9wYXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2Vz
IHNpZ25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDi
gJhsb25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBv
cGVyYW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBB
U1NFUlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZ3Vlc3RfYWNjZXNzLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZ3Vlc3RfYWNjZXNzLmg6MTAsCiAgICAgICAgICAgICAgICAg
ZnJvbSBjb21tb24vc3BpbmxvY2suYzo5OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVu
Y3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4p
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFw
KHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZv
ICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAg
ICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3Mu
aDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJz
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFs
bG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2
X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
dm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5p
dHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJ
biBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6
NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90
IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6
NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0
IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEw
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj
dCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0u
aDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZu
c19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDox
MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVj
dG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxp
bmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2Rl
diAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6
NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRl
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3By
b2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5n
KHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKA
mGlzX3B2XzMyYml0X2RvbWFpbuKAmToKSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
MzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRf
ZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KY29tbW9uL3NwaW5sb2NrLmM6IEluIGZ1bmN0aW9uIOKAmGNwdV9sb2Nr
ZGVidWdfY2FsbGJhY2vigJk6CmNvbW1vbi9zcGlubG9jay5jOjM0OjY3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5mYuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNCB8
IHN0YXRpYyBpbnQgY2ZfY2hlY2sgY3B1X2xvY2tkZWJ1Z19jYWxsYmFjayhzdHJ1Y3Qgbm90
aWZpZXJfYmxvY2sgKm5mYiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL3NwaW5s
b2NrLmM6IEF0IHRvcCBsZXZlbDoKY29tbW9uL3NwaW5sb2NrLmM6Nzk6MTogd2FybmluZzog
4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xk
LXN0eWxlLWRlY2xhcmF0aW9uXQogICA3OSB8IHByZXNtcF9pbml0Y2FsbChsb2NrZGVidWdf
aW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
NCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1f
ZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgpjb21tb24vc3BpbmxvY2suYzozMTI6MTogd2FybmluZzog4oCYaW5saW5l4oCZIGlz
IG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0
aW9uXQogIDMxMiB8IHN0YXRpYyB2b2lkIGFsd2F5c19pbmxpbmUgc3Bpbl9sb2NrX2NvbW1v
bihzcGlubG9ja190aWNrZXRzX3QgKnQsCiAgICAgIHwgXn5+fn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KY29t
bW9uL3NwaW5sb2NrLmM6MzYyOjE6IHdhcm5pbmc6IOKAmGlubGluZeKAmSBpcyBub3QgYXQg
YmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAz
NjIgfCBzdGF0aWMgdm9pZCBhbHdheXNfaW5saW5lIHNwaW5fdW5sb2NrX2NvbW1vbihzcGlu
bG9ja190aWNrZXRzX3QgKnQsCiAgICAgIHwgXn5+fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgpjb21tb24v
c3BpbmxvY2suYzozOTE6MTogd2FybmluZzog4oCYaW5saW5l4oCZIGlzIG5vdCBhdCBiZWdp
bm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDM5MSB8
IHN0YXRpYyBib29sIGFsd2F5c19pbmxpbmUgc3Bpbl9pc19sb2NrZWRfY29tbW9uKGNvbnN0
IHNwaW5sb2NrX3RpY2tldHNfdCAqdCkKICAgICAgfCBefn5+fn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1
bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl4KY29tbW9uL3NwaW5sb2NrLmM6NDA1OjE6IHdhcm5pbmc6IOKA
mGlubGluZeKAmSBpcyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFstV29sZC1z
dHlsZS1kZWNsYXJhdGlvbl0KICA0MDUgfCBzdGF0aWMgYm9vbCBhbHdheXNfaW5saW5lIHNw
aW5fdHJ5bG9ja19jb21tb24oc3BpbmxvY2tfdGlja2V0c190ICp0LAogICAgICB8IF5+fn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1f
ZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQx
Nzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmNvbW1vbi9zcGlubG9j
ay5jOjQ0MToxOiB3YXJuaW5nOiDigJhpbmxpbmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBv
ZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgNDQxIHwgc3RhdGlj
IHZvaWQgYWx3YXlzX2lubGluZSBzcGluX2JhcnJpZXJfY29tbW9uKHNwaW5sb2NrX3RpY2tl
dHNfdCAqdCwKICAgICAgfCBefn5+fn4KY29tbW9uL3NwaW5sb2NrLmM6IEluIGZ1bmN0aW9u
IOKAmHNwaW5sb2NrX3Byb2ZpbGVfcHJpbnRfZWxlbeKAmToKY29tbW9uL3NwaW5sb2NrLmM6
NjM2OjM4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhcuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYzNiB8ICAgICBpbnQzMl90IHR5cGUsIGludDMyX3QgaWR4LCB2b2lk
ICpwYXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+
fgpjb21tb24vc3BpbmxvY2suYzogSW4gZnVuY3Rpb24g4oCYc3BpbmxvY2tfcHJvZmlsZV9w
cmludGFsbOKAmToKY29tbW9uL3NwaW5sb2NrLmM6NjY1OjU1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGtleeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY2NSB8IHZvaWQg
Y2ZfY2hlY2sgc3BpbmxvY2tfcHJvZmlsZV9wcmludGFsbCh1bnNpZ25lZCBjaGFyIGtleSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KY29tbW9uL3NwaW5sb2NrLmM6IEluIGZ1bmN0aW9uIOKAmHNwaW5sb2Nr
X3Byb2ZpbGVfcmVzZXRfZWxlbeKAmToKY29tbW9uL3NwaW5sb2NrLmM6Njc3OjEzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHR5cGXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA2NzcgfCAgICAgaW50MzJfdCB0eXBlLCBpbnQzMl90IGlkeCwgdm9pZCAqcGFyKQogICAg
ICB8ICAgICB+fn5+fn5+fl5+fn4KY29tbW9uL3NwaW5sb2NrLmM6Njc3OjI3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY3
NyB8ICAgICBpbnQzMl90IHR5cGUsIGludDMyX3QgaWR4LCB2b2lkICpwYXIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5efn4KY29tbW9uL3NwaW5sb2NrLmM6Njc3OjM4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhcuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDY3NyB8ICAgICBpbnQzMl90IHR5cGUsIGludDMyX3QgaWR4LCB2b2lkICpwYXIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpjb21t
b24vc3BpbmxvY2suYzogQXQgdG9wIGxldmVsOgpjb21tb24vc3BpbmxvY2suYzo4MDQ6MTog
d2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRp
b24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDgwNCB8IF9faW5pdGNhbGwobG9ja19w
cm9mX2luaXQpOwogICAgICB8IF5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2
b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1v
cnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5
KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
Nzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2Rldmlj
ZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNp
X2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGlu
ZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlu
bGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVk
IGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19t
dXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhp
c19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
IDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihj
b25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KY29tbW9uL3N0
b3BfbWFjaGluZS5jOiBJbiBmdW5jdGlvbiDigJhzdG9wbWFjaGluZV93YWl0X3N0YXRl4oCZ
Ogpjb21tb24vc3RvcF9tYWNoaW5lLmM6NjY6NDk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogICA2NiB8ICAgICB3
aGlsZSAoIGF0b21pY19yZWFkKCZzdG9wbWFjaGluZV9kYXRhLmRvbmUpICE9IHN0b3BtYWNo
aW5lX2RhdGEubnJfY3B1cyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXn4KY29tbW9uL3N0b3BfbWFjaGluZS5jOiBJbiBmdW5j
dGlvbiDigJhjcHVfY2FsbGJhY2vigJk6CmNvbW1vbi9zdG9wX21hY2hpbmUuYzoxODY6Mjg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmZi4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg2IHwgICAgIHN0cnVjdCBub3RpZmllcl9ibG9jayAqbmZiLCB1bnNpZ25lZCBs
b25nIGFjdGlvbiwgdm9pZCAqaGNwdSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KY29tbW9uL3N0b3BfbWFjaGluZS5jOiBBdCB0b3AgbGV2ZWw6CmNvbW1vbi9z
dG9wX21hY2hpbmUuYzoyMTI6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBi
ZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDIx
MiB8IF9faW5pdGNhbGwoY3B1X3N0b3BtYWNoaW5lX2luaXQpOwogICAgICB8IF5+fn5+fn5+
fn4KICBDQyAgICAgIGFyY2gvYXJtL2RvbWFpbl9idWlsZC5vCiAgQ0MgICAgICBjb21tb24v
c3ltYm9scy5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NjksCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9kb21haW5fYnVpbGQuYzo1Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1
X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDox
MDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVu
c2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9u
IOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1v
bi9zeW1ib2xzLmM6MTg6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJ
biBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhj
cHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBj
cHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5y
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3
IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwo
c3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlv
biDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2Zp
bHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL2RvbWFpbl9idWls
ZC5jOjY6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmln
X3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVy
YW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZ
IHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8g
dW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8
ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJs
ZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9hcm0vZG9tYWluX2J1aWxkLmM6
NzoKLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9k
b21haW5fcGFnZV9nbG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA3MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9i
YWwoY29uc3Qgdm9pZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJt
L2RvbWFpbl9idWlsZC5jOjg6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDi
gJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5
IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAg
ICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zeW1ib2xzLmM6MjE6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoz
MTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGlu
ZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFz
a192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwg
c3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1
aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRw
Mm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7i
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAg
c3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAy
bV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGNvbW1vbi90YXNrbGV0Lm8KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQo
bWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1m
buKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVj
dCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4g
ZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMx
MCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1
Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDog
SW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVy
KGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIw
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
dW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19l
bXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
Mzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19p
bmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
ODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFn
ZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5
cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
YXJjaC9hcm0vZG9tYWluX2J1aWxkLmM6MTg6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBm
dW5jdGlvbiDigJhhY3BpX3NldF9jc3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNw
aS5oOjE5Mjo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9z
ZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhh
Y3BpX3NldF9jc3Vic3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0
YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9kb21haW5fYnVp
bGQuYzoyNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wY2kuaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9wY2lfaW5pdF9wZGV24oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3BjaS5oOjEz
NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTM1IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfcGNpX2luaXRfcGRl
dihzdHJ1Y3QgcGNpX2RldiAqcGRldikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcGNpLmg6IEluIGZ1bmN0aW9uIOKAmHBjaV9nZXRfaG9zdF9icmlkZ2Vfc2Vn
bWVudOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wY2kuaDoxMzc6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbm9kZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEz
NyB8IHN0YXRpYyBpbmxpbmUgaW50IHBjaV9nZXRfaG9zdF9icmlkZ2Vfc2VnbWVudChjb25z
dCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKm5vZGUsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3BjaS5oOjEzODo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzZWdtZW504oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTM4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVpbnQxNl90ICpzZWdtZW50KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+Xn5+fn5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMg
aW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9y
eV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTcsCiAgICAgICAgICAgICAgICAgZnJv
bSBhcmNoL2FybS9kb21haW5fYnVpbGQuYzozNDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9l
dmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9y
eV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlu
bGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFz
c2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgz
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZ3JhbnRfdGFibGUuaDozMSwKICAgICAgICAgICAg
ICAgICBmcm9tIGFyY2gvYXJtL2RvbWFpbl9idWlsZC5jOjM3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9tYXJrX2RpcnR5
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MjA6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBnbnR0YWJfbWFya19kaXJ0eShzdHJ1Y3QgZG9tYWlu
ICpkLCBtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90
YWJsZS5oOjIwOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBnbnR0YWJfbWFy
a19kaXJ0eShzdHJ1Y3QgZG9tYWluICpkLCBtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKA
mGdudHRhYl9ob3N0X21hcHBpbmdfZ2V0X3BhZ2VfdHlwZeKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9ncmFudF90YWJsZS5oOjI3OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJv4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI3IHwgc3RhdGljIGlubGluZSBi
b29sIGdudHRhYl9ob3N0X21hcHBpbmdfZ2V0X3BhZ2VfdHlwZShib29sIHJvLAouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6Mjg6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjggfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBkb21haW4gKmxkLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjI5Ojc1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI5IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgZG9tYWluICpyZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDogSW4gZnVuY3Rpb24g4oCYZ250dGFi
X3JlbGVhc2VfaG9zdF9tYXBwaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFu
dF90YWJsZS5oOjM0OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgZ250dGFiX3Jl
bGVhc2VfaG9zdF9tYXBwaW5ncyhjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwg
c3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIGFyY2gvYXJtL2RvbWFpbl9idWlsZC5jOjM5OgouL2luY2x1ZGUveGVuL3Nlcmlh
bC5oOiBBdCB0b3AgbGV2ZWw6Ci4vaW5jbHVkZS94ZW4vc2VyaWFsLmg6MTY2OjE6IHdhcm5p
bmc6IOKAmGlubGluZeKAmSBpcyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFst
V29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAxNjYgfCBzdGF0aWMgdm9pZCBpbmxpbmUgeGhj
aV9kYmNfdWFydF9pbml0KHZvaWQpIHt9CiAgICAgIHwgXn5+fn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19t
dXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vZG9tYWluX2J1aWxkLmM6IEluIGZ1bmN0aW9uIOKA
mHdyaXRlX3Byb3BlcnRpZXPigJk6CmFyY2gvYXJtL2RvbWFpbl9idWlsZC5jOjUzMzo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTMzIHwgc3RhdGljIGludCBfX2luaXQgd3JpdGVfcHJvcGVydGllcyhzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3Qga2VybmVsX2luZm8gKmtpbmZvLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9kb21h
aW5fYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYZG9tYWluX2ZkdF9iZWdpbl9ub2Rl4oCZOgph
cmNoL2FybS9kb21haW5fYnVpbGQuYzo3NDc6MTQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJhsb25nIHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNzQ3IHwg
ICAgIGlmICggcmV0ID49IHNpemVvZihidWYpICkKICAgICAgfCAgICAgICAgICAgICAgXn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hlbi9jb25maWcuaDoxNywKICAg
ICAgICAgICAgICAgICBmcm9tIDxjb21tYW5kLWxpbmU+OgphcmNoL2FybS9kb21haW5fYnVp
bGQuYzogSW4gZnVuY3Rpb24g4oCYbWFrZV9tZW1vcnlfbm9kZeKAmToKYXJjaC9hcm0vZG9t
YWluX2J1aWxkLmM6ODAxOjI1OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYbG9u
ZyB1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDgwMSB8ICAgICAgICAgQlVH
X09OKG5yX2NlbGxzID49IEFSUkFZX1NJWkUocmVnKSk7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBp
biBkZWZpbml0aW9uIG9mIG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1
bmxpa2VseSh4KSAgIF9fYnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJtL2RvbWFpbl9i
dWlsZC5jOjgwMTo5OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYQlVHX09O4oCZ
CiAgODAxIHwgICAgICAgICBCVUdfT04obnJfY2VsbHMgPj0gQVJSQVlfU0laRShyZWcpKTsK
ICAgICAgfCAgICAgICAgIF5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0
aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
MzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRf
ZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgpj
b21tb24vc3ltYm9scy5jOiBJbiBmdW5jdGlvbiDigJhnZXRfc3ltYm9sX29mZnNldOKAmToK
Y29tbW9uL3N5bWJvbHMuYzo5NToxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mGxvbmcgdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAgOTUgfCAgICAgZm9y
KGkgPSAwOyBpIDwgKHBvcyYweEZGKTsgaSsrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
XgphcmNoL2FybS9kb21haW5fYnVpbGQuYzogSW4gZnVuY3Rpb24g4oCYZmluZF9nbnR0YWJf
cmVnaW9u4oCZOgphcmNoL2FybS9kb21haW5fYnVpbGQuYzoxOTY4OjU0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDE5NjggfCBz
dGF0aWMgdm9pZCBfX2luaXQgZmluZF9nbnR0YWJfcmVnaW9uKHN0cnVjdCBkb21haW4gKmQs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi90YXNrbGV0LmM6MTc6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
c3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCY
X19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2su
aDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAg
ICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBe
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8
ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
ZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTi
gJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5s
aW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwg
cGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1
ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCY
dGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIo
Y3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUu
aDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCY
cmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwg
Ym9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50
X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoz
NjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdm
biwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3Zt
X2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0
MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0
aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRl
X2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21l
bW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3Bv
bGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9h
c3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBz
dGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2ln
bmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcg
fCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUg
KnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1
bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJi
aXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi8uL2luY2x1ZGUveGVuL2NvbmZpZy5oOjE3LAog
ICAgICAgICAgICAgICAgIGZyb20gPGNvbW1hbmQtbGluZT46CmNvbW1vbi90YXNrbGV0LmM6
IEluIGZ1bmN0aW9uIOKAmGRvX3Rhc2tsZXRfd29ya+KAmToKY29tbW9uL3Rhc2tsZXQuYzo4
ODo2MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgIDg4IHwgICAgIEJVR19PTih0LT5pc19kZWFkIHx8IHQtPmlz
X3J1bm5pbmcgfHwgKHQtPnNjaGVkdWxlZF9vbiAhPSBjcHUpKTsKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY29tcGlsZXIuaDoyMTo0Mzogbm90ZTogaW4gZGVmaW5pdGlvbiBv
ZiBtYWNybyDigJh1bmxpa2VseeKAmQogICAyMSB8ICNkZWZpbmUgdW5saWtlbHkoeCkgICBf
X2J1aWx0aW5fZXhwZWN0KCEhKHgpLDApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vdGFza2xldC5jOjg4OjU6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhCVUdfT07igJkKICAgODggfCAgICAgQlVHX09OKHQt
PmlzX2RlYWQgfHwgdC0+aXNfcnVubmluZyB8fCAodC0+c2NoZWR1bGVkX29uICE9IGNwdSkp
OwogICAgICB8ICAgICBefn5+fn4KY29tbW9uL3Rhc2tsZXQuYzogSW4gZnVuY3Rpb24g4oCY
bWlncmF0ZV90YXNrbGV0c19mcm9tX2NwdeKAmToKY29tbW9uL3Rhc2tsZXQuYzoxOTM6MzI6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDE5MyB8ICAgICAgICAgQlVHX09OKHQtPnNjaGVkdWxlZF9vbiAhPSBj
cHUpOwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g
4oCYdW5saWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGlu
X2V4cGVjdCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIF4KY29tbW9uL3Rhc2tsZXQuYzoxOTM6OTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmEJVR19PTuKAmQogIDE5MyB8ICAgICAgICAgQlVHX09OKHQtPnNj
aGVkdWxlZF9vbiAhPSBjcHUpOwogICAgICB8ICAgICAgICAgXn5+fn5+CmNvbW1vbi90YXNr
bGV0LmM6IEluIGZ1bmN0aW9uIOKAmGNwdV9jYWxsYmFja+KAmToKY29tbW9uL3Rhc2tsZXQu
YzoyMTk6Mjg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmZi4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjE5IHwgICAgIHN0cnVjdCBub3RpZmllcl9ibG9jayAqbmZiLCB1
bnNpZ25lZCBsb25nIGFjdGlvbiwgdm9pZCAqaGNwdSkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGNvbW1vbi90aW1lLm8KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAg
IGZyb20gY29tbW9uL3RpbWUuYzoxODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxp
bmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E
4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1f
U0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3Ry
dWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lk
czsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lk
cykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lk
IGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEu
aDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6
MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZCht
Zm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
bS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwg
dm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVj
dCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVl
cnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlu
bGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVf
YmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6
NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
YXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjgz
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBj
aV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQz
Ml90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMy
X3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNp
X3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9t
YWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGlj
IGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24vdGltZXIu
bwogIENDICAgICAgY29tbW9uL3RyYWNlLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3Rp
bWVyLmM6MTE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5j
dGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJf
c2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NB
VkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQog
ICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9l
bmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L2lvLmg6MjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIGNvbW1vbi90cmFjZS5jOjE5OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwg
c3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vdHJhY2UuYzoyMToKLi9pbmNsdWRlL3hl
bi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDigJhub19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVk
ZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hh
bmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHth
a2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3Ro
ZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8g
KHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAg
ICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJ
biBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6
NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90
IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAg
ICAgICAgIGZyb20gY29tbW9uL3RyYWNlLmM6MjI6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9k
ZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFk
aXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2so
cmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
dm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGNvbW1v
bi92ZXJzaW9uLm8KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNx
dWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0
cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBp
ZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2Fs
dHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAg
IHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBw
Mm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3Rh
dGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9j
ay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAg
ICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAg
IF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3lj
bGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQw
IHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192
YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3Rh
dGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZy
ZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25p
ZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBp
bmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0
X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5
OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAq
Z21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBj
aV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBp
bmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25l
ZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2ln
bmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9f
bXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAq
cGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVs
MihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQx
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0
ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
YWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBw
YWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxv
Y2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNf
cHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29u
c3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi90aW1lci5jOjEzOgouL2luY2x1ZGUveGVuL3BhcmFt
Lmg6IEluIGZ1bmN0aW9uIOKAmG5vX2NvbmZpZ19wYXJhbeKAmToKLi9pbmNsdWRlL3hlbi9w
YXJhbS5oOjE5MzoxOTogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFuZ2VzIHNp
Z25lZG5lc3MgZnJvbSDigJhsb25nIGludOKAmSB0byDigJhzaXplX3TigJkge2FrYSDigJhs
b25nIHVuc2lnbmVkIGludOKAmX0gZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVy
YW5kIFstV3NpZ24tY29tcGFyZV0KICAxOTMgfCAgICAgaW50IGxlbiA9IGUgPyAoeyBBU1NF
UlQoZSA+PSBzKTsgZSAtIHM7IH0pIDogc3RybGVuKHMpOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWlu
dDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmNvbW1vbi90
aW1lci5jOiBJbiBmdW5jdGlvbiDigJhkdW1wX3RpbWVyceKAmToKY29tbW9uL3RpbWVyLmM6
NTU2OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGtleeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU1NiB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIGR1bXBfdGltZXJxKHVu
c2lnbmVkIGNoYXIga2V5KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi90aW1lci5jOiBJbiBmdW5jdGlvbiDigJhj
cHVfY2FsbGJhY2vigJk6CmNvbW1vbi90aW1lci5jOjY0ODoyODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhuZmLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NDggfCAgICAg
c3RydWN0IG5vdGlmaWVyX2Jsb2NrICpuZmIsIHVuc2lnbmVkIGxvbmcgYWN0aW9uLCB2b2lk
ICpoY3B1KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxv
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcg
fCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
dXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVt
b3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhw
ZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQg
dnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjE3
LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3RyYWNlLmM6MjY6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2dsb2Jh
bOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBpbnQg
dmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgpjb21tb24vdHJhY2UuYzogSW4gZnVuY3Rpb24g4oCYY3B1X2Nh
bGxiYWNr4oCZOgpjb21tb24vdHJhY2UuYzo3ODoyODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhuZmLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNzggfCAgICAgc3RydWN0
IG5vdGlmaWVyX2Jsb2NrICpuZmIsIHVuc2lnbmVkIGxvbmcgYWN0aW9uLCB2b2lkICpoY3B1
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpjb21tb24vdHJhY2Uu
YzogSW4gZnVuY3Rpb24g4oCYYWxsb2NfdHJhY2VfYnVmc+KAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6Mzc1OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNzUgfCAgICAgICAgICAgICAgKGNwdSkg
PCBucl9jcHVfaWRzOyAgICAgICAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6NDU5OjM2OiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfY3B14oCZCiAgNDU5IHwgI2RlZmluZSBmb3JfZWFj
aF9vbmxpbmVfY3B1KGNwdSkgICBmb3JfZWFjaF9jcHUoY3B1LCAmY3B1X29ubGluZV9tYXAp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+
fn4KY29tbW9uL3RyYWNlLmM6MjAxOjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhmb3JfZWFjaF9vbmxpbmVfY3B14oCZCiAgMjAxIHwgICAgIGZvcl9lYWNoX29ubGluZV9j
cHUoY3B1KQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi90cmFjZS5j
OjIwNjoyNDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjA2IHwgICAgICAgICBmb3IgKCBpID0gMDsgaSA8IHBh
Z2VzOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNzU6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM3NSB8ICAgICAgICAgICAgICAo
Y3B1KSA8IG5yX2NwdV9pZHM7ICAgICAgICAgICAgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDo0NTk6MzY6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9jcHXigJkKICA0NTkgfCAjZGVmaW5lIGZv
cl9lYWNoX29ubGluZV9jcHUoY3B1KSAgIGZvcl9lYWNoX2NwdShjcHUsICZjcHVfb25saW5l
X21hcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+
fn5+fn5+fgpjb21tb24vdHJhY2UuYzoyMjM6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmGZvcl9lYWNoX29ubGluZV9jcHXigJkKICAyMjMgfCAgICAgZm9yX2VhY2hfb25s
aW5lX2NwdShjcHUpCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3Ry
YWNlLmM6MjM5OjI0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzkgfCAgICAgICAgIGZvciAoIGkgPSAwOyBp
IDwgcGFnZXM7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1v
bi90cmFjZS5jOjI0NToxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQ1IHwgICAgIGZvcihpID0gMDsgaSA8
IHRfaW5mb19wYWdlczsgaSsrKQogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNzU6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM3NSB8ICAgICAgICAgICAg
ICAoY3B1KSA8IG5yX2NwdV9pZHM7ICAgICAgICAgICAgICAgIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDo0NTk6MzY6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9jcHXigJkKICA0NTkgfCAjZGVmaW5l
IGZvcl9lYWNoX29ubGluZV9jcHUoY3B1KSAgIGZvcl9lYWNoX2NwdShjcHUsICZjcHVfb25s
aW5lX21hcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+
fn5+fn5+fn5+fgpjb21tb24vdHJhY2UuYzoyNjA6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmGZvcl9lYWNoX29ubGluZV9jcHXigJkKICAyNjAgfCAgICAgZm9yX2VhY2hf
b25saW5lX2NwdShjcHUpCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9u
L3RyYWNlLmM6MjY1OjI0OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNjUgfCAgICAgICAgIGZvciAoIGkgPSAw
OyBpIDwgcGFnZXM7IGkrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICBeCmNv
bW1vbi90cmFjZS5jOiBJbiBmdW5jdGlvbiDigJh0Yl9jb250cm9s4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNzU6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM3NSB8ICAgICAgICAgICAgICAo
Y3B1KSA8IG5yX2NwdV9pZHM7ICAgICAgICAgICAgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDo0NTk6MzY6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9jcHXigJkKICA0NTkgfCAjZGVmaW5lIGZv
cl9lYWNoX29ubGluZV9jcHUoY3B1KSAgIGZvcl9lYWNoX2NwdShjcHUsICZjcHVfb25saW5l
X21hcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+
fn5+fn5+fgpjb21tb24vdHJhY2UuYzo0MTc6OTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmGZvcl9lYWNoX29ubGluZV9jcHXigJkKICA0MTcgfCAgICAgICAgIGZvcl9lYWNo
X29ubGluZV9jcHUoaSkKICAgICAgfCAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hlbi9jb25maWcuaDoxNywKICAgICAg
ICAgICAgICAgICBmcm9tIDxjb21tYW5kLWxpbmU+Ogpjb21tb24vdHJhY2UuYzogSW4gZnVu
Y3Rpb24g4oCYY2FsY191bmNvbnN1bWVkX2J5dGVz4oCZOgpjb21tb24vdHJhY2UuYzo0NzM6
MTQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYczMy4oCZIHtha2Eg4oCYaW504oCZfSBhbmQg4oCYdTMy
4oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAgNDczIHwg
ICAgIEFTU0VSVCh4IDw9IGRhdGFfc2l6ZSk7CiAgICAgIHwgICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vY29tcGlsZXIuaDoyMTo0Mzogbm90ZTogaW4gZGVmaW5pdGlvbiBvZiBt
YWNybyDigJh1bmxpa2VseeKAmQogICAyMSB8ICNkZWZpbmUgdW5saWtlbHkoeCkgICBfX2J1
aWx0aW5fZXhwZWN0KCEhKHgpLDApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgXgpjb21tb24vdHJhY2UuYzo0NzM6NTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKAmQogIDQ3MyB8ICAgICBBU1NFUlQoeCA8PSBk
YXRhX3NpemUpOwogICAgICB8ICAgICBefn5+fn4KY29tbW9uL3RyYWNlLmM6IEluIGZ1bmN0
aW9uIOKAmGNhbGNfYnl0ZXNfdG9fd3JhcOKAmToKY29tbW9uL3RyYWNlLmM6NDkyOjE0OiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmHMzMuKAmSB7YWthIOKAmGludOKAmX0gYW5kIOKAmHUzMuKAmSB7
YWthIOKAmHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDQ5MiB8ICAgICBB
U1NFUlQoeCA8PSBkYXRhX3NpemUpOwogICAgICB8ICAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g
4oCYdW5saWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGlu
X2V4cGVjdCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIF4KY29tbW9uL3RyYWNlLmM6NDkyOjU6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhBU1NFUlTigJkKICA0OTIgfCAgICAgQVNTRVJUKHggPD0gZGF0YV9z
aXplKTsKICAgICAgfCAgICAgXn5+fn5+CmNvbW1vbi90cmFjZS5jOiBJbiBmdW5jdGlvbiDi
gJh0cmFjZV9ub3RpZnlfZG9tMOKAmToKY29tbW9uL3RyYWNlLmM6NjYxOjQ2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHVudXNlZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDY2MSB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIHRyYWNlX25vdGlmeV9kb20wKHZvaWQgKnVu
dXNlZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5efn5+fn4KICBDQyAgICAgIGNvbW1vbi92aXJ0dWFsX3JlZ2lvbi5vCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5o
OjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1l
Lmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAgICAg
ICAgZnJvbSBjb21tb24vdmlydHVhbF9yZWdpb24uYzo3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwg
c3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklY
X0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194
KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhE
RUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUo
RU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+
fn5+fn4KICBDQyAgICAgIGFyY2gvYXJtL2RvbWN0bC5vCkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAg
ICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAg
ICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2Nf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2
NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBh
Z2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6
NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRs
YmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjEx
OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNw
dW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEx
IHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ss
IHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY29tbW9uL3ZpcnR1YWxfcmVnaW9uLmM6ODoK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
YXNtLWdlbmVyaWMvcGVyY3B1Lmg6OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL3BlcmNwdS5oOjEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3BlcmNwdS5oOjMwLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9yd2xvY2suaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9kZXZpY2VfdHJlZS5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9kdC1vdmVybGF5Lmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2Fy
bS9kb21jdGwuYzo4OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4g
ZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B1
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1
X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFk
X2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KY29t
bW9uL3ZpcnR1YWxfcmVnaW9uLmM6IEF0IHRvcCBsZXZlbDoKY29tbW9uL3ZpcnR1YWxfcmVn
aW9uLmM6MTAxOjE6IHdhcm5pbmc6IOKAmGlubGluZeKAmSBpcyBub3QgYXQgYmVnaW5uaW5n
IG9mIGRlY2xhcmF0aW9uIFstV29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAxMDEgfCBzdGF0
aWMgdm9pZCBpbmxpbmUgcmVtb3ZlX3ZpcnR1YWxfcmVnaW9uKHN0cnVjdCB2aXJ0dWFsX3Jl
Z2lvbiAqcikKICAgICAgfCBefn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcnds
b2NrLmg6NjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAg
ICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAg
ICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2Nf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2
NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3J3bG9jay5oOjc6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJt
L2RvbWN0bC5jOjEwOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZu
X3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0
YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9k
ZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFk
aXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2so
cmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
dm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVk
IGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVu
dF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9i
bG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9t
ZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGlu
dCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNz
X3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3Rh
dGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQu
aDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2Fs
bC5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZG9tY3RsLmM6MTE6Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fY3JlYXRl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRyZWYpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyOTo1MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyMjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWluaW5mbyh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3RsX3Nj
aGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQwIHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3AoeHNtX2RlZmF1bHRf
dCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQ2
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2RvbWN0bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGzigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZWFk
Y29uc29sZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fdW5ib3VuZOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2huLCBkb21pZF90
IGlkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg1
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0
Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2
dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBmbGFncykKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwOTox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3NldHVw4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMyMToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2NvcHnigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMzM6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfcG9kX3Rhcmdl
dCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2RfdGFyZ2V04oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9t
ZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzgxOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4
MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2UoeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZhdGlvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
OTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfcGluX3Bh
Z2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5OSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLAog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVtb3ZlX2Zyb21f
cGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDEyIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jbGFpbV9w
YWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDIzIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6NDg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDMz
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9maWxlKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQzODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fa2V4ZWMoeHNtX2Rl
ZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9z
Y2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0NDoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGlyceKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X21hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBf
ZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lk
ICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV91
bm1hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Vu
bWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcxOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwg
dm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpiaW5kKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4OToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBw
aXJxLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fcGVybWlzc2lvbuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1
aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGluZ+KAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBlLCB1aW50OF90IGFs
bG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjUwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1aW50MTZfdCBzdGFy
dCwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2ln
bl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpk
dHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2Jk
ZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJj
ZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291
cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVw
X2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9v
ZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50
MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVsdF90IGRlZikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1o
dm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1f
b3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgbW9kZSwgdWludDMy
X3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3Zu
dW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJv
bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjY1NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGxhdGZvcm1fb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0
Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2Rv
bWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vZG9tY3RsLmM6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
Z2V0X2RvbWFpbl9pbmZv4oCZOgphcmNoL2FybS9kb21jdGwuYzoyMDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwg
dm9pZCBhcmNoX2dldF9kb21haW5faW5mbyhjb25zdCBzdHJ1Y3QgZG9tYWluICpkLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgog
IENDICAgICAgY29tbW9uL3ZtX2V2ZW50Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3Zt
X2V2ZW50LmM6MjM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBm
dW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVf
bnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0
aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBw
YWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFn
ZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAg
ICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhy
Y3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3Jl
YWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vz
c19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3Mu
aDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoy
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29u
c3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2Fj
Y2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2Fs
dHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0
IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpk
LCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3
OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVj
dCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4g
ZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMx
MCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1
Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZp
Y2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGlj
IGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9k
ZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBw
Y2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
d3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3Jp
dGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3Bl
bmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUv
eGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5s
aW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9ldmVudC5oOjE3LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3ZtX2V2ZW50LmM6
MjQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF92aXJxX2lzX2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUw
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2Jh
bCh1bnNpZ25lZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2dlbmVyYXRlZC9hc20vdm1fZXZlbnQuaDoxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi92bV9ldmVudC5oOjI4LAogICAgICAgICAg
ICAgICAgIGZyb20gY29tbW9uL3ZtX2V2ZW50LmM6MjY6Ci4vaW5jbHVkZS9hc20tZ2VuZXJp
Yy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF9pbml0X2RvbWFpbuKAmToK
Li9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MTQ6NTU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxNCB8IHN0YXRp
YyBpbmxpbmUgaW50IHZtX2V2ZW50X2luaXRfZG9tYWluKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g
4oCYdm1fZXZlbnRfdG9nZ2xlX3NpbmdsZXN0ZXDigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJp
Yy92bV9ldmVudC5oOjI1OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgdm1fZXZl
bnRfdG9nZ2xlX3NpbmdsZXN0ZXAoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MjU6
Nzg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyNSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2bV9ldmVudF90b2dnbGVfc2luZ2xl
c3RlcChzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MjY6
Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDI2IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2
bV9ldmVudF9yZWdpc3Rlcl93cml0ZV9yZXN1bWXigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJp
Yy92bV9ldmVudC5oOjMyOjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzIgfCB2b2lkIHZtX2V2ZW50X3JlZ2lzdGVyX3dy
aXRlX3Jlc3VtZShzdHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjMyOjc0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMiB8
IHZvaWQgdm1fZXZlbnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1lKHN0cnVjdCB2Y3B1ICp2LCB2
bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1f
ZXZlbnRfZW11bGF0ZV9jaGVja+KAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50
Lmg6Mzg6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzOCB8IHZvaWQgdm1fZXZlbnRfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMv
dm1fZXZlbnQuaDozODo2Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzggfCB2b2lkIHZtX2V2ZW50X2VtdWxhdGVfY2hl
Y2soc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVu
Y3Rpb24g4oCYdm1fZXZlbnRfc3luY19ldmVudOKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmlj
L3ZtX2V2ZW50Lmg6NDQ6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA0NCB8IHZvaWQgdm1fZXZlbnRfc3luY19ldmVudChz
dHJ1Y3QgdmNwdSAqdiwgYm9vbCB2YWx1ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
NDQ6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmFsdWXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNDQgfCB2b2lkIHZtX2V2ZW50X3N5bmNfZXZlbnQoc3RydWN0IHZj
cHUgKnYsIGJvb2wgdmFsdWUpCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJ
biBmdW5jdGlvbiDigJh2bV9ldmVudF9yZXNldF92bXRyYWNl4oCZOgouL2luY2x1ZGUvYXNt
LWdlbmVyaWMvdm1fZXZlbnQuaDo1MDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgdm9pZCB2bV9ldmVudF9yZXNl
dF92bXRyYWNlKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tb25pdG9yLmg6MzAsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21t
b24vdm1fZXZlbnQuYzoyOToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2FsbG93X3VzZXJzcGFjZeKAmToKLi9pbmNsdWRl
L2FzbS1nZW5lcmljL21vbml0b3IuaDoyMjo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIyIHwgdm9pZCBhcmNoX21vbml0
b3JfYWxsb3dfdXNlcnNwYWNlKHN0cnVjdCBkb21haW4gKmQsIGJvb2wgYWxsb3dfdXNlcnNw
YWNlKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6MjI6NTg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYWxsb3dfdXNlcnNwYWNl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDIyIHwgdm9pZCBhcmNoX21vbml0b3JfYWxsb3dfdXNlcnNwYWNlKHN0
cnVjdCBkb21haW4gKmQsIGJvb2wgYWxsb3dfdXNlcnNwYWNlKQouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvbW9uaXRvci5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21vbml0b3JfZG9tY3RsX29w
4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOjI3OjQzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjcgfCBp
bnQgYXJjaF9tb25pdG9yX2RvbWN0bF9vcChzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVu
X2RvbWN0bF9tb25pdG9yX29wICptb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5o
OjI3Ojc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1vcOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAyNyB8IGludCBhcmNoX21vbml0b3JfZG9tY3RsX29wKHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX21vbml0b3Jfb3AgKm1vcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3Iu
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2luaXRfZG9tYWlu4oCZOgouL2luY2x1
ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOjQzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDMgfCBpbnQgYXJjaF9tb25p
dG9yX2luaXRfZG9tYWluKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJp
Yy9tb25pdG9yLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbW9uaXRvcl9jbGVhbnVwX2RvbWFp
buKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDo1MDo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwg
dm9pZCBhcmNoX21vbml0b3JfY2xlYW51cF9kb21haW4oc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW9uaXRvci5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X21vbml0b3JfZ2V0X2NhcGFiaWxpdGllc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
b25pdG9yLmg6MzU6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAzNSB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgYXJjaF9t
b25pdG9yX2dldF9jYXBhYmlsaXRpZXMoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi92bV9ldmVudC5jOjM2
OgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0
ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0
bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0
MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZh
dWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjI0NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBj
bWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9t
aWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNs
dWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
MDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21h
aW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90
YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bp
bl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9m
cm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xh
aW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhz
bV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
bWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2
MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwg
dm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBp
cnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmlu
ZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBp
bnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Np
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3Qg
ZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhf
dCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qg
c3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVh
c3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0
MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hh
ciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGlu
ZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1Mzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVf
YmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVz
b3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9y
ZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9z
ZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlz
Y+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Bh
Z2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwg
dWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0
cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJt
aHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVp
bnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dl
dF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2Nv
bnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3Jt
X29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9u
KHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi92bV9ldmVudC5jOiBJbiBmdW5jdGlvbiDigJh2
bV9ldmVudF93YWtlX3F1ZXVlZOKAmToKY29tbW9uL3ZtX2V2ZW50LmM6MTYxOjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
NjEgfCBzdGF0aWMgdm9pZCB2bV9ldmVudF93YWtlX3F1ZXVlZChzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3Qgdm1fZXZlbnRfZG9tYWluICp2ZWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgpjb21tb24vdm1fZXZlbnQuYzog
SW4gZnVuY3Rpb24g4oCYbW9uaXRvcl9ub3RpZmljYXRpb27igJk6CmNvbW1vbi92bV9ldmVu
dC5jOjU0OTo3Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwb3J04oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNTQ5IHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgbW9uaXRvcl9u
b3RpZmljYXRpb24oc3RydWN0IHZjcHUgKnYsIHVuc2lnbmVkIGludCBwb3J0KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vZWZpL2VmaS1kb20w
Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJu
YXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90
aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vY29tbW9uL2VmaS9lZmkuaDo5LAogICAgICAgICAgICAg
ICAgIGZyb20gYXJjaC9hcm0vZWZpL2VmaS1kb20wLmM6MjQ6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgogIENDICAgICAgY29tbW9uL3ZtYXAubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4g
ZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGji
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4
NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQp
OwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAg
ICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAg
ICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2Nw
dW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1
MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25l
ZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
cHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBh
Z2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdl
X2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJm
bHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL2VmaS9lZmktZG9tMC5jOjI1OgouL2luY2x1ZGUveGVu
L251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9u
dW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3Rv
X25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRp
eC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2Rv
bWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlv
biDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1
X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24v
dm1hcC5jOjU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5j
dGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1
ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJf
c2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZt
X2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVp
bnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBs
b25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
cDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VO
ROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FW
RV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0
IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxp
bmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNp
emUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVf
YmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g
4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRl
X2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3Rf
bWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lk
czsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lk
cykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lk
IGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRl
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZw
Y2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikg
eyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVu
ZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2
Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNf
aW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4g
ZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8z
MmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KY29tbW9uL3ZtYXAuYzogSW4g
ZnVuY3Rpb24g4oCYdnphbGxvY+KAmToKY29tbW9uL3ZtYXAuYzozMzI6MjA6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJhzaXplX3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVk
IGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDMzMiB8ICAgICBmb3IgKCBpID0gMDsgaSA8
IHNpemU7IGkgKz0gUEFHRV9TSVpFICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgog
IENDICAgICAgYXJjaC9hcm0vZWZpL2Jvb3QubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9jb21tb24v
ZWZpL2VmaS5oOjksCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9lZmkvYm9vdC5j
OjE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGlu
Z3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVu
Y3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJl
duKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2Vf
aW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xp
c3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODoz
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVz
aF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL2VmaS9ib290LmM6OToKLi9pbmNsdWRl
L3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9kb21haW5fcGFnZV9n
bG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA3MSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9iYWwoY29uc3Qgdm9p
ZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJt
L2VmaS9ib290LmM6MTU6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCY
bm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJu
aW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxv
bmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZ
fSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJl
XQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsg
fSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fgphcmNoL2FybS9lZmkvYm9vdC5jOiBJbiBmdW5jdGlvbiDigJhl
ZmlfcmVsb2NhdGVfZXNydOKAmToKYXJjaC9hcm0vZWZpL2Jvb3QuYzo2MzE6NTY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYU3lzdGVtVGFibGXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA2MzEgfCBzdGF0aWMgdm9pZCBfX2luaXQgZWZpX3JlbG9jYXRlX2VzcnQoRUZJ
X1NZU1RFTV9UQUJMRSAqU3lzdGVtVGFibGUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+fn5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZWZpL2VmaS1ib290Lmg6OCwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL2VmaS9ib290LmM6NzAxOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOiBJbiBmdW5jdGlv
biDigJhsb29rdXBfZmR0X2NvbmZpZ190YWJsZeKAmToKYXJjaC9hcm0vZWZpL2VmaS1ib290
Lmg6MTQ5OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYVUlOVE7igJkge2Fr
YSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDE0OSB8ICAg
ICBmb3IgKCBpID0gMDsgaSA8IHN5c190YWJsZS0+TnVtYmVyT2ZUYWJsZUVudHJpZXM7IGkr
KyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIF4KYXJjaC9hcm0vZWZpL2VmaS1ib290
Lmg6IEluIGZ1bmN0aW9uIOKAmGVmaV9wcm9jZXNzX21lbW9yeV9tYXBfYm9vdGluZm/igJk6
CmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOjE5MDoyODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmFVJTlRO4oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IFstV3Np
Z24tY29tcGFyZV0KICAxOTAgfCAgICAgZm9yICggSW5kZXggPSAwOyBJbmRleCA8IChtbWFw
X3NpemUgLyBkZXNjX3NpemUpOyBJbmRleCsrICkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBeCmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOiBJbiBmdW5jdGlvbiDigJhl
ZmlfYXJjaF9yZWxvY2F0ZV9pbWFnZeKAmToKYXJjaC9hcm0vZWZpL2VmaS1ib290Lmg6Mzcw
OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlbHRh4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzcwIHwgc3RhdGljIHZvaWQgX19pbml0IGVmaV9hcmNoX3JlbG9jYXRl
X2ltYWdlKHVuc2lnbmVkIGxvbmcgZGVsdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fn4KYXJjaC9hcm0v
ZWZpL2VmaS1ib290Lmg6IEluIGZ1bmN0aW9uIOKAmGVmaV9hcmNoX2NmZ19maWxlX2xhdGXi
gJk6CmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOjQyODo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhpbWFnZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRp
YyB2b2lkIF9faW5pdCBlZmlfYXJjaF9jZmdfZmlsZV9sYXRlKGNvbnN0IEVGSV9MT0FERURf
SU1BR0UgKmltYWdlLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CmFyY2gvYXJtL2VmaS9l
ZmktYm9vdC5oOjQyOTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkaXJfaGFu
ZGxl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI5IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgRUZJX0ZJTEVfSEFORExFIGRpcl9oYW5kbGUsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fl5+fn5+fn5+fn4KYXJjaC9hcm0vZWZpL2VmaS1ib290Lmg6NDMwOjU1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNlY3Rpb27igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MzAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBjb25zdCBjaGFyICpzZWN0aW9uKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fn5+fn4KYXJjaC9hcm0vZWZpL2Vm
aS1ib290Lmg6IEluIGZ1bmN0aW9uIOKAmGVmaV9hcmNoX2VkaWTigJk6CmFyY2gvYXJtL2Vm
aS9lZmktYm9vdC5oOjQ0OTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnb3Bf
aGFuZGxl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ5IHwgc3RhdGljIHZvaWQgX19p
bml0IGVmaV9hcmNoX2VkaWQoRUZJX0hBTkRMRSBnb3BfaGFuZGxlKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fn5+fn5+fgphcmNo
L2FybS9lZmkvZWZpLWJvb3QuaDogSW4gZnVuY3Rpb24g4oCYZWZpX2FyY2hfaGFuZGxlX21v
ZHVsZeKAmToKYXJjaC9hcm0vZWZpL2VmaS1ib290Lmg6NTIxOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5hbWXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MjEgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBDSEFSMTYg
Km5hbWUsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOiBJbiBmdW5j
dGlvbiDigJhlZmlfYXJjaF9jb25zb2xlX2luaXTigJk6CmFyY2gvYXJtL2VmaS9lZmktYm9v
dC5oOjk4MDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjb2xz4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgOTgwIHwgc3RhdGljIHZvaWQgX19pbml0IGVmaV9hcmNoX2Nv
bnNvbGVfaW5pdChVSU5UTiBjb2xzLCBVSU5UTiByb3dzKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgphcmNoL2FybS9lZmkv
ZWZpLWJvb3QuaDo5ODA6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcm93c+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDk4MCB8IHN0YXRpYyB2b2lkIF9faW5pdCBlZmlf
YXJjaF9jb25zb2xlX2luaXQoVUlOVE4gY29scywgVUlOVE4gcm93cykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl5+fn4KYXJjaC9hcm0vZWZpL2VmaS1ib290Lmg6IEluIGZ1bmN0aW9uIOKAmGVmaV9hcmNo
X3ZpZGVvX2luaXTigJk6CmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOjk4NDo3MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhnb3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA5
ODQgfCBzdGF0aWMgdm9pZCBfX2luaXQgZWZpX2FyY2hfdmlkZW9faW5pdChFRklfR1JBUEhJ
Q1NfT1VUUFVUX1BST1RPQ09MICpnb3AsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmFy
Y2gvYXJtL2VmaS9lZmktYm9vdC5oOjk4NTo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhpbmZvX3NpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA5ODUgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBVSU5UTiBpbmZvX3NpemUsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fn5+
fn5+CmFyY2gvYXJtL2VmaS9lZmktYm9vdC5oOjk4Njo3ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtb2RlX2luZm/igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA5ODYgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBFRklfR1JBUEhJQ1NfT1VU
UFVUX01PREVfSU5GT1JNQVRJT04gKm1vZGVfaW5mbykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+fn5+fgphcmNoL2FybS9lZmkvYm9vdC5jOiBJbiBmdW5jdGlvbiDi
gJhlZmlfY29uc29sZV9zZXRfbW9kZeKAmToKYXJjaC9hcm0vZWZpL2Jvb3QuYzo4NzU6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDigJhJTlQzMuKAmSB7YWth
IOKAmGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDg3NSB8ICAgICAgICAgICBpIDwgU3Rk
T3V0LT5Nb2RlLT5NYXhNb2RlOyArK2kgKQogICAgICB8ICAgICAgICAgICAgIF4KYXJjaC9h
cm0vZWZpL2Jvb3QuYzo4ODQ6MTU6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZ
IGFuZCDigJhJTlQzMuKAmSB7YWthIOKAmGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogIDg4
NCB8ICAgICBpZiAoIGJlc3QgIT0gU3RkT3V0LT5Nb2RlLT5Nb2RlICkKICAgICAgfCAgICAg
ICAgICAgICAgIF5+CiAgQ0MgICAgICBjb21tb24vdnNwcmludGYubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSBjb21tb24vdnNwcmludGYuYzoyMjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1
X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1
X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2
b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251
bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25p
ZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIz
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3Zj
cHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGNvbW1vbi93YWl0Lm8KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNp
X2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsg
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRp
bmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlu
bGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNw
dSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4g
ZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8z
MmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5eCmNvbW1vbi92c3ByaW50Zi5jOiBJbiBmdW5jdGlvbiDigJhwb2ludGVy4oCZOgpjb21t
b24vdnNwcmludGYuYzozNzk6MjI6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZ
IGFuZCDigJhpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM3OSB8ICAgICAgICAgICAgIGlm
ICggKytpID09IGZpZWxkX3dpZHRoICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICBe
fgpjb21tb24vdnNwcmludGYuYzogSW4gZnVuY3Rpb24g4oCYdnNjbnByaW50ZuKAmToKY29t
bW9uL3ZzcHJpbnRmLmM6NzA1OjExOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
c2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFy
ZV0KICA3MDUgfCAgICAgaWYgKGkgPj0gc2l6ZSkKICAgICAgfCAgICAgICAgICAgXn4KY29t
bW9uL3ZzcHJpbnRmLmM6IEluIGZ1bmN0aW9uIOKAmHNjbnByaW50ZuKAmToKY29tbW9uL3Zz
cHJpbnRmLmM6NzU3OjExOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYc2l6ZV90
4oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IFstV3NpZ24tY29tcGFyZV0KICA3
NTcgfCAgICAgaWYgKGkgPj0gc2l6ZSkKICAgICAgfCAgICAgICAgICAgXn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAg
ICAgIGZyb20gY29tbW9uL3dhaXQuYzoyMjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBp
bmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1
X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1
X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2
b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251
bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25p
ZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIz
MywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3Zj
cHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBj
aV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODgg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVp
bnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2
cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRf
ZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3Rh
dGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24veG1h
bGxvY190bHNmLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi94bWFsbG9jX3Rsc2YuYzoyNjoKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAg
ICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4K
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwg
ICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGlj
IGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOjM2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9p
cnEuaDo1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rp
b24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3Np
YmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IElu
IGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3Jl
YWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAg
ICAgICAgICBmcm9tIGNvbW1vbi94bWFsbG9jX3Rsc2YuYzoyNzoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0
bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWdu
ZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFn
ZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgou
L2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3Rh
bXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+fn5+fn5+fn5+fgogIENDICAgICAgY29tbW9uL2RvbWN0bC5vCkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVn
cy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90
aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAg
ICAgICAgZnJvbSBjb21tb24vZG9tY3RsLmM6MTI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0
aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2Nw
dV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2Nw
dV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19j
cHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVu
c2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8
IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAg
ICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2
b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGlt
ZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2
Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9kb21jdGwuYzox
MzoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6
Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5l
IG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJ
biBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3Mu
aDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nl
c3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJt
X2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWlu
ICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0
cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBz
dHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3Rh
dGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVh
ZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
d3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRh
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNz
X3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1
ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToK
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNf
aW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9ldmVudC5oOjE3LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2RvbWN0bC5j
OjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9i
YWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vZ3JhbnRfdGFibGUuaDozMSwKICAgICAgICAgICAgICAgICBmcm9t
IGNvbW1vbi9kb21jdGwuYzoxNjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJs
ZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfbWFya19kaXJ0eeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjIwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0IGRvbWFpbiAqZCwgbWZuX3QgbWZuKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyMDo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0IGRv
bWFpbiAqZCwgbWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9ncmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfaG9zdF9tYXBw
aW5nX2dldF9wYWdlX3R5cGXigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFi
bGUuaDoyNzo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyb+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAyNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBnbnR0YWJfaG9zdF9t
YXBwaW5nX2dldF9wYWdlX3R5cGUoYm9vbCBybywKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9n
cmFudF90YWJsZS5oOjI4Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICpsZCwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Z3JhbnRfdGFibGUuaDoyOTo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyOSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqcmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9yZWxlYXNlX2hvc3RfbWFw
cGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDozNDo3MDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDM0IHwgc3RhdGljIGlubGluZSBib29sIGdudHRhYl9yZWxlYXNlX2hvc3RfbWFwcGlu
Z3MoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vZG9tY3RsLmM6MTc6Ci4vaW5jbHVkZS94ZW4v
ZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rpb24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs
4oCZOgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZh
KSB7fTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L2h5cGVyY2FsbC5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL2RvbWN0bC5j
OjI1OgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2Ny
ZWF0ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
Mjk6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8o
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5
c2N0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI0MCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9k
ZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjI0NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9kb21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91
bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwg
ZG9taWRfdCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3Ry
dWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9p
bmNsdWRlL3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ldnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3Mp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozMDk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoz
MjE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9k
b21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3Bv
ZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3Rhcmdl
dOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5
X3Bpbl9wYWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMiwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQwNjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92
ZV9mcm9tX3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQx
MiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Y2xhaW1fcGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQyOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQzMzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVj
KHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fc2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ2MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGly
cSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAq
YmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1p
c3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0
X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWlu
dDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2
X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZGVhc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU0MCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qg
Y2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFj
aGluZV9iZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1
MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hp
bmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19j
b3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDU2OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJj
ZV9zZXR1cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBf
bWlzY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3BhZ2Vfb2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRl
ZiwgdWludDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1ODkgfCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBk
ZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFy
YW3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NjEyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1f
YWx0cDJtaHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0
cDJtaHZtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUs
IHVpbnQzMl90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2dldF92bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50
X2NvbnRyb2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBv
cCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjM1OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRm
b3JtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo3NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJz
aW9uKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9kb21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS92bV9ldmVudC5oOjEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3ZtX2V2ZW50Lmg6MjgsCiAgICAgICAgICAgICAgICAgZnJv
bSBjb21tb24vZG9tY3RsLmM6MjY6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5o
OiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF9pbml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL2Fz
bS1nZW5lcmljL3ZtX2V2ZW50Lmg6MTQ6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxNCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZtX2V2ZW50X2luaXRfZG9tYWluKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2lu
Y2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRf
dG9nZ2xlX3NpbmdsZXN0ZXDigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5o
OjI1OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgdm1fZXZlbnRfdG9nZ2xlX3Np
bmdsZXN0ZXAoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MjU6Nzg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB2bV9ldmVudF90b2dnbGVfc2luZ2xlc3RlcChzdHJ1Y3Qg
ZG9tYWluICpkLCBzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MjY6Njg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI2
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHZtX2V2
ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF9yZWdp
c3Rlcl93cml0ZV9yZXN1bWXigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5o
OjMyOjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzIgfCB2b2lkIHZtX2V2ZW50X3JlZ2lzdGVyX3dyaXRlX3Jlc3VtZShz
dHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjMyOjc0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMiB8IHZvaWQgdm1fZXZl
bnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1lKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNw
b25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
YXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfZW11bGF0
ZV9jaGVja+KAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6Mzg6NDI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzOCB8IHZvaWQgdm1fZXZlbnRfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdm1f
ZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoz
ODo2Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzggfCB2b2lkIHZtX2V2ZW50X2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1f
ZXZlbnRfc3luY19ldmVudOKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
NDQ6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA0NCB8IHZvaWQgdm1fZXZlbnRfc3luY19ldmVudChzdHJ1Y3QgdmNwdSAq
diwgYm9vbCB2YWx1ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6NDQ6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmFsdWXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNDQgfCB2b2lkIHZtX2V2ZW50X3N5bmNfZXZlbnQoc3RydWN0IHZjcHUgKnYsIGJvb2wg
dmFsdWUpCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJh2bV9ldmVudF9yZXNldF92bXRyYWNl4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1f
ZXZlbnQuaDo1MDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgdm9pZCB2bV9ldmVudF9yZXNldF92bXRyYWNlKHN0
cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgogIENDICAgICAgY29tbW9uL21vbml0b3IubwpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21t
b24vbW9uaXRvci5jOjIyOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDog
SW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
Y3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQg
Y3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBp
bmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90
IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKA
mQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2
ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAg
ICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQw
OjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAg
ICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUy
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9j
cHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBm
dW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1m
bikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5f
ZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAo
dW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29u
c3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcy
IHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8g
KnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAg
ICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjEx
OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJj
dV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21l
bV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVp
cmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAy
bV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFp
biAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJv
b2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhz
dHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
c3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgogIENDICAgICAgYXJjaC9hcm0vZWZpL3BlLm8KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2ln
bl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9y
ZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRp
YyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBi
b29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rp
b24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEz
MTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9k
b21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBp
bnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2dlbmVyYXRlZC9hc20vdm1fZXZlbnQuaDoxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi92bV9ldmVudC5oOjI4LAogICAgICAgICAgICAgICAgIGZyb20g
Y29tbW9uL21vbml0b3IuYzoyNToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X2luaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUvYXNt
LWdlbmVyaWMvdm1fZXZlbnQuaDoxNDo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDE0IHwgc3RhdGljIGlubGluZSBpbnQg
dm1fZXZlbnRfaW5pdF9kb21haW4oc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5j
bHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF90
b2dnbGVfc2luZ2xlc3RlcOKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
MjU6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2bV9ldmVudF90b2dnbGVfc2lu
Z2xlc3RlcChzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoyNTo3ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI1IHwg
c3RhdGljIGlubGluZSB2b2lkIHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVwKHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoyNjo2ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjYg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdm1fZXZl
bnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3JlZ2lz
dGVyX3dyaXRlX3Jlc3VtZeKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
MzI6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAzMiB8IHZvaWQgdm1fZXZlbnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1lKHN0
cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRl
L2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MzI6NzQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMyIHwgdm9pZCB2bV9ldmVu
dF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUoc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3Bv
bnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9h
c20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF9lbXVsYXRl
X2NoZWNr4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDozODo0Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDM4IHwgdm9pZCB2bV9ldmVudF9lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB2bV9l
dmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjM4
OjY2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAzOCB8IHZvaWQgdm1fZXZlbnRfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9l
dmVudF9zeW5jX2V2ZW504oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDo0
NDozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDQ0IHwgdm9pZCB2bV9ldmVudF9zeW5jX2V2ZW50KHN0cnVjdCB2Y3B1ICp2
LCBib29sIHZhbHVlKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDo0NDo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh2YWx1ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA0NCB8IHZvaWQgdm1fZXZlbnRfc3luY19ldmVudChzdHJ1Y3QgdmNwdSAqdiwgYm9vbCB2
YWx1ZSkKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKA
mHZtX2V2ZW50X3Jlc2V0X3ZtdHJhY2XigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9l
dmVudC5oOjUwOjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTAgfCB2b2lkIHZtX2V2ZW50X3Jlc2V0X3ZtdHJhY2Uoc3Ry
dWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vbW9uaXRvci5jOjI2Ogou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9z
Y2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0
X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0
NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9kb21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVh
ZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRf
dCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2
dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0
cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM3MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJn
ZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
bWVtb3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
ODEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MzkzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9w
YWdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQwNjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9t
X3BoeXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1f
cGFnZXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2NvbnNvbGVfaW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQy
OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQz
Mzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9k
ZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
c2NoZWRvcF9zaHV0ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDQ0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0
IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV9tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFw
X2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9p
ZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
dW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91
bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEs
IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
cGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwg
dWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBh
bGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3Rh
cnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3Np
Z25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAq
ZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9y
ZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9i
ZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRm
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3Vy
Y2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1
cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vf
b2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWlu
dDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkg
fCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEy
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJt
aHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQz
Ml90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92
bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRy
b2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29w
KHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3
NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
b21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvZ2VuZXJhdGVkL2FzbS9hbHRwMm0uaDoxLAogICAgICAgICAgICAgICAgIGZyb20gY29t
bW9uL21vbml0b3IuYzoyNzoKLi9pbmNsdWRlL2FzbS1nZW5lcmljL2FsdHAybS5oOiBJbiBm
dW5jdGlvbiDigJhhbHRwMm1fYWN0aXZl4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvYWx0
cDJtLmg6MTE6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbHRwMm1fYWN0aXZl
KGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVy
aWMvYWx0cDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFsdHAybV92Y3B1X2lkeOKAmToKLi9pbmNs
dWRlL2FzbS1nZW5lcmljL2FsdHAybS5oOjE4OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTggfCBzdGF0aWMgaW5saW5l
IHVuc2lnbmVkIGludCBhbHRwMm1fdmNwdV9pZHgoY29uc3Qgc3RydWN0IHZjcHUgKnYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21vbml0b3IuaDozMCwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9tb25p
dG9yLmM6Mjg6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfbW9uaXRvcl9hbGxvd191c2Vyc3BhY2XigJk6Ci4vaW5jbHVkZS9hc20tZ2Vu
ZXJpYy9tb25pdG9yLmg6MjI6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMiB8IHZvaWQgYXJjaF9tb25pdG9yX2FsbG93
X3VzZXJzcGFjZShzdHJ1Y3QgZG9tYWluICpkLCBib29sIGFsbG93X3VzZXJzcGFjZSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOjIyOjU4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGFsbG93X3VzZXJzcGFjZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyMiB8IHZvaWQgYXJjaF9tb25pdG9yX2FsbG93X3VzZXJzcGFjZShzdHJ1Y3QgZG9t
YWluICpkLCBib29sIGFsbG93X3VzZXJzcGFjZSkKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21v
bml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2RvbWN0bF9vcOKAmToKLi9p
bmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDoyNzo0Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI3IHwgaW50IGFyY2hf
bW9uaXRvcl9kb21jdGxfb3Aoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxf
bW9uaXRvcl9vcCAqbW9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDoyNzo3Njog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtb3DigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjcgfCBpbnQgYXJjaF9tb25pdG9yX2RvbWN0bF9vcChzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgeGVuX2RvbWN0bF9tb25pdG9yX29wICptb3ApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfbW9uaXRvcl9pbml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL21vbml0b3IuaDo0Mzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQzIHwgaW50IGFyY2hfbW9uaXRvcl9pbml0
X2RvbWFpbihzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRv
ci5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21vbml0b3JfY2xlYW51cF9kb21haW7igJk6Ci4v
aW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6NTA6NDk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHZvaWQgYXJj
aF9tb25pdG9yX2NsZWFudXBfZG9tYWluKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21vbml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9y
X2dldF9jYXBhYmlsaXRpZXPigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW9uaXRvci5o
OjM1OjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMzUgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IGFyY2hfbW9uaXRvcl9n
ZXRfY2FwYWJpbGl0aWVzKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVy
bmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
dGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9j
ay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2NvbW1vbi9lZmkvZWZpLmg6OSwKICAgICAgICAgICAg
ICAgICBmcm9tIGFyY2gvYXJtL2VmaS9wZS5jOjI0OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3Rh
dGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NP
TVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2
IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZv
aWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVf
SFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAs
IHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0
aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50
MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNM
QVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5E
LCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+
fn4KICBDQyAgICAgIGNvbW1vbi9zeXNjdGwubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xh
c3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMw
IHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAg
ICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAg
Xn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICpt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zeXNjdGwu
YzoxMToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJs
aW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBm
dW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hl
YWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVj
dCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwg
cGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1
ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCY
dGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIo
Y3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Nt
cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwg
bnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0g
bnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFs
bG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlu
bGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNf
aW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3Rt
YXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAg
ICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFn
ZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZs
dXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3N5c2N0
bC5jOjEyOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25p
ZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBp
bmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9t
ZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNv
dXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlk
eOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBk
b21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlf
dCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxv
Y2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNz
aWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzgg
fCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYg
KnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9y
ZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBj
aV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
ZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90
IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3Qg
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMw
MDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3By
b2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
aW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu
4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFs
d2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL2V2ZW50Lmg6MTcsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vc3lz
Y3RsLmM6MTQ6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF92aXJxX2lzX2dsb2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVu
dC5oOjUwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lz
X2dsb2JhbCh1bnNpZ25lZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9ncmFudF90YWJsZS5oOjMxLAogICAgICAgICAgICAgICAg
IGZyb20gY29tbW9uL3N5c2N0bC5jOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50
X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9tYXJrX2RpcnR54oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MjA6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBnbnR0YWJfbWFya19kaXJ0eShzdHJ1Y3QgZG9tYWluICpkLCBtZm5fdCBt
Zm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjIwOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBnbnR0YWJfbWFya19kaXJ0eShzdHJ1
Y3QgZG9tYWluICpkLCBtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9ob3N0
X21hcHBpbmdfZ2V0X3BhZ2VfdHlwZeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFu
dF90YWJsZS5oOjI3OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJv4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI3IHwgc3RhdGljIGlubGluZSBib29sIGdudHRhYl9o
b3N0X21hcHBpbmdfZ2V0X3BhZ2VfdHlwZShib29sIHJvLAouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2dyYW50X3RhYmxlLmg6Mjg6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjggfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBkb21haW4g
KmxkLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9ncmFudF90YWJsZS5oOjI5Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI5IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZG9tYWlu
ICpyZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZ3JhbnRfdGFibGUuaDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX3JlbGVhc2VfaG9z
dF9tYXBwaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjM0
OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgZ250dGFiX3JlbGVhc2VfaG9zdF9t
YXBwaW5ncyhjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9zeXNjdGwuYzoxNjoKLi9pbmNsdWRl
L3hlbi9kb21haW5fcGFnZS5oOiBJbiBmdW5jdGlvbiDigJh1bm1hcF9kb21haW5fcGFnZV9n
bG9iYWzigJk6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDo3MTo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh2YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA3MSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1bm1hcF9kb21haW5fcGFnZV9nbG9iYWwoY29uc3Qgdm9p
ZCAqdmEpIHt9OwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+Xn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vc3lz
Y3RsLmM6MjM6Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21h
aW5fY3JlYXRl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRy
ZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjIyOTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWlu
aW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9v
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjQwIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0
bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2RvbWN0bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9z
eXNjdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9yZWFkY29uc29sZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5f
dW5ib3VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAq
Y2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBm
bGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50
X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2Nv
cHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3
YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9n
ZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2Rf
dGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzgx
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM4MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2Uo
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZh
dGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozOTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9t
ZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVtb3ZlX2Zyb21fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9jbGFpbV9wYWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9m
aWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQzODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
a2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0
NDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGly
ceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX21hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBp
bnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV91bm1hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3VubWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcx
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRf
aXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ4OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1f
cGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1
aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGlu
Z+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBl
LCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1
aW50MTZfdCBzdGFydCwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBj
b25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
bWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5w
bHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5l
X2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9z
ZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9w
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9w
YXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYx
OCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qg
bW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1f
ZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwg
aW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nl
c3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
bV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVu
X3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1v
bi9zeXNjdGwuYzozMDoKLi9pbmNsdWRlL3hlbi9jb3ZlcmFnZS5oOiBJbiBmdW5jdGlvbiDi
gJhzeXNjdGxfY292X29w4oCZOgouL2luY2x1ZGUveGVuL2NvdmVyYWdlLmg6ODozOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh1bnVzZWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgIDggfCBzdGF0aWMgaW5saW5lIGludCBzeXNjdGxfY292X29wKHZvaWQgKnVudXNl
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fn5+
fgogIENDICAgICAgYXJjaC9hcm0vZWZpL2VibWFsbG9jLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
Y29tbW9uL2VmaS9lZmkuaDo5LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZWZp
L2VibWFsbG9jLmM6MToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAg
ICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAg
ICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
YWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1h
c2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFz
a192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9u
IOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
fn5+fn5+fn5+fgogIENDICAgICAgYXJjaC9hcm0vZWZpL3J1bnRpbWUubwpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDox
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAg
IGZyb20gLi9jb21tb24vZWZpL2VmaS5oOjksCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNo
L2FybS9lZmkvcnVudGltZS5jOjE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5l
IGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKA
mToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3Rh
dGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWlu
dDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUg
aW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUp
IFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgogIENDICAg
ICAgY29tbW9uL3Vic2FuL3Vic2FuLm8KICBDQyAgICAgIGNvbW1vbi9zY2hlZC9ib290LWNw
dXBvb2wubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5z
X2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEw
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9p
bmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAg
ICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJm
bHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIGNvbW1vbi91YnNhbi91YnNhbi5jOjEzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNy
ZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6
NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXgu
aDo1NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2Fj
ZW52Lmg6MTQyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvYWNwaS5o
OjU2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9hY3BpLmg6NTcsCiAg
ICAgICAgICAgICAgICAgZnJvbSBjb21tb24vc2NoZWQvYm9vdC1jcHVwb29sLmM6MTA6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVf
bnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEw
MTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5z
aWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9n
dWVzdF9hY2Nlc3MuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9n
dWVzdF9hY2Nlc3MuaDoxMCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2VmaS9y
dW50aW1lLmM6NToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90
b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0
aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6Mzcs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVs
YXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJt
X21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVf
cmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1
Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1
ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
c3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCY
X19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1f
U0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+
fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8j
I194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9u
IG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVf
SFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nw
aW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g
4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FW
RV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5e
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBB
VF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xB
UkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJj
aF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xp
Y3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0
IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3Np
Z25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNi
ZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQz
Ml90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3Jp
dGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlu
bGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUs
IHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hl
Y2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xgpjb21tb24vdWJzYW4vdWJzYW4uYzogSW4gZnVuY3Rpb24g4oCYX191YnNhbl9o
YW5kbGVfZGl2cmVtX292ZXJmbG934oCZOgpjb21tb24vdWJzYW4vdWJzYW4uYzoyNTY6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbGhz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjU2IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBs
b25nIGxocywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hY3BpLmg6MjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAg
ICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBe
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8
ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDi
gJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFp
bihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVj
dG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBh
Z2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAg
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lk
IHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0
YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Rh
dGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
OTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVk
IGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVk
ZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3RhdGVfbGltaXTi
gJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBu
ZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSBj
b21tb24vc2NoZWQvYm9vdC1jcHVwb29sLmM6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJ
biBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6
NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90
IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
Xn5+CiAgSU5JVF9PICBhcmNoL2FybS9lZmkvZWZpLWRvbTAuaW5pdC5vCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMg
aW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1
ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9y
eV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGlj
eSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3Np
Z25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
cmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0
aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBz
dGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2ln
bmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0
aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
MzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRf
ZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgog
IElOSVRfTyAgYXJjaC9hcm0vZWZpL2Jvb3QuaW5pdC5vCiAgQ0MgICAgICBjb21tb24vbGli
ZmR0L2ZkdC5vCiAgSU5JVF9PICBhcmNoL2FybS9lZmkvcGUuaW5pdC5vCiAgTEQgICAgICBh
cmNoL2FybS9lZmkvYnVpbHRfaW4ubwogIENDICAgICAgYXJjaC9hcm0vZ2ljLm8KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdz
Lmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Rp
bWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6Nzgs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAgICAgICAgICAg
ICAgICBmcm9tIGFyY2gvYXJtL2dpYy5jOjEzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBv
ZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTi
gJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVf
aWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVf
aWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1
bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZv
aWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1
ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1
ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBw
YWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFn
ZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAg
ICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL2lycS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9naWMuYzox
NDoKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9s
b2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1
X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZ2ljLmM6MTU6Ci4v
aW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2lu
Y2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2Rl
aWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9v
bCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0
ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24vc2NoZWQvY3B1
cG9vbC5vCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVf
cmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1
Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1
ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1
X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEw
Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2
aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZw
Y2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90
IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQz
Ml90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBj
aV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2Rv
bWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRp
YyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9jcHUuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9u
L3NjaGVkL2NwdXBvb2wuYzoxNDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUg
aW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZ
OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0
aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50
MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2
OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZ
UEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZt
X3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNp
emXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBp
bnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FW
RV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0
IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSBhcmNoL2FybS9naWMuYzoyMDoKLi9pbmNsdWRlL3hlbi9hY3BpLmg6
IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hl
bi9hY3BpLmg6MTkyOjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1p
dOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9pZCBh
Y3BpX3NldF9jc3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47
IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9u
IOKAmGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6
MTk0OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9j
c3Vic3RhdGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hl
bi9jb25maWcuaDoxNywKICAgICAgICAgICAgICAgICBmcm9tIDxjb21tYW5kLWxpbmU+Ogph
cmNoL2FybS9naWMuYzogSW4gZnVuY3Rpb24g4oCYZ2ljX3JvdXRlX2lycV90b194ZW7igJk6
CmFyY2gvYXJtL2dpYy5jOjExNDoyMjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTE0IHwgICAgIEFTU0VSVChk
ZXNjLT5pcnEgPCBnaWNfbnVtYmVyX2xpbmVzKCkpOy8qIENhbid0IHJvdXRlIGludGVycnVw
dHMgdGhhdCBkb24ndCBleGlzdCAqLwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgIF4K
Li9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBpbiBkZWZpbml0aW9uIG9m
IG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1bmxpa2VseSh4KSAgIF9f
YnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJtL2dpYy5jOjExNDo1OiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYQVNTRVJU4oCZCiAgMTE0IHwgICAgIEFTU0VSVChkZXNj
LT5pcnEgPCBnaWNfbnVtYmVyX2xpbmVzKCkpOy8qIENhbid0IHJvdXRlIGludGVycnVwdHMg
dGhhdCBkb24ndCBleGlzdCAqLwogICAgICB8ICAgICBefn5+fn4KYXJjaC9hcm0vZ2ljLmM6
IEluIGZ1bmN0aW9uIOKAmG1haW50ZW5hbmNlX2ludGVycnVwdOKAmToKYXJjaC9hcm0vZ2lj
LmM6MzkyOjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM5MiB8IHN0YXRpYyB2b2lkIG1haW50ZW5hbmNlX2ludGVycnVw
dChpbnQgaXJxLCB2b2lkICpkZXZfaWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5efn4KYXJjaC9hcm0vZ2ljLmM6MzkyOjUwOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRldl9pZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5
MiB8IHN0YXRpYyB2b2lkIG1haW50ZW5hbmNlX2ludGVycnVwdChpbnQgaXJxLCB2b2lkICpk
ZXZfaWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl5+fn5+fgphcmNoL2FybS9naWMuYzogSW4gZnVuY3Rpb24g4oCYY3B1X2dp
Y19jYWxsYmFja+KAmToKYXJjaC9hcm0vZ2ljLmM6NDUzOjUyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG5mYuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MyB8IHN0YXRp
YyBpbnQgY3B1X2dpY19jYWxsYmFjayhzdHJ1Y3Qgbm90aWZpZXJfYmxvY2sgKm5mYiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KYXJjaC9hcm0vZ2ljLmM6NDU1OjM1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTUgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdm9pZCAqaGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+Xn5+fgphcmNoL2FybS9naWMuYzogQXQgdG9wIGxldmVsOgphcmNo
L2FybS9naWMuYzo0ODA6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdp
bm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDQ4MCB8
IF9faW5pdGNhbGwoY3B1X2dpY19ub3RpZmllcl9pbml0KTsKICAgICAgfCBefn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBjb21tb24vc2NoZWQvY3B1cG9vbC5jOjE1OgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAg
ICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAg
ICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2Rv
bWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
MTEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1ZXN0
X2FjY2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1ZXN0
X2FjY2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3NjaGVkL2NwdXBv
b2wuYzoxNjoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19u
aWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMg
aW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwg
dW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUx
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3
IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g
4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxi
Lmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAq
bWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+
fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRy
ZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3Qg
KmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9y
ZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIz
IHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2
ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25n
IGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIExEICAgICAg
Y29tbW9uL3Vic2FuL2J1aWx0X2luLm8KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFy
Y2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KICBDQyAgICAgIGNvbW1vbi9wbWFwLm8KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9zY2hlZC9jcHVwb29sLmM6
MjI6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3Bh
cmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5k
IG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRv
IOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5z
aWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAg
ICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4o
cyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhpc191bml0X29u
bGluZeKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRf
aW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjkxOjU6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkK
ICAgOTEgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAg
IHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5o
OiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5hYmxl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6MTA4OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhm
b3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxMDggfCAgICAgZm9yX2VhY2hfc2NoZWRf
dW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5h
YmxlX3N0YXRl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+
bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTM1OjU6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3Zj
cHXigJkKICAxMzUgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiAp
CiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJp
dmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRfcmVz4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAg
IFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNv
bW1vbi9zY2hlZC9wcml2YXRlLmg6MTU0OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxNTQgfCAgICAgZm9yX2VhY2hf
c2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hl
ZF9zZXRfcGF1c2VfZmxhZ3PigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lk
IDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUu
aDoxNzY6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVk
X3VuaXRfdmNwdeKAmQogIDE3NiB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1
bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2NsZWFyX3BhdXNlX2ZsYWdz
4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9s
aXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTg2OjU6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAx
ODYgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwg
ICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJ
biBmdW5jdGlvbiDigJhzY2hlZF91bml0X3BhdXNlX25vc3luY+KAmToKLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAg
ICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpj
b21tb24vc2NoZWQvcHJpdmF0ZS5oOjUzMDo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgNTMwIHwgICAgIGZvcl9lYWNo
X3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYc2No
ZWRfdW5pdF91bnBhdXNl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8
ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
NTM4OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91
bml0X3ZjcHXigJkKICA1MzggfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5p
dCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2No
ZWQvY3B1cG9vbC5jOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9ncmFuX2dldOKAmToKY29tbW9u
L3NjaGVkL2NwdXBvb2wuYzoxMDM6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c3Ry4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAzIHwgc3RhdGljIGludCBzY2hlZF9n
cmFuX2dldChjb25zdCBjaGFyICpzdHIsIGVudW0gc2NoZWRfZ3JhbiAqbW9kZSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+fgpjb21tb24vc2No
ZWQvY3B1cG9vbC5jOjEwMzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtb2Rl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAzIHwgc3RhdGljIGludCBzY2hlZF9ncmFu
X2dldChjb25zdCBjaGFyICpzdHIsIGVudW0gc2NoZWRfZ3JhbiAqbW9kZSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5efn5+CmNvbW1vbi9zY2hlZC9jcHVwb29sLmM6IEluIGZ1bmN0aW9uIOKAmGNwdXBv
b2xfYXNzaWduX2NwdV9sb2NrZWTigJk6CmNvbW1vbi9zY2hlZC9jcHVwb29sLmM6NDQ5OjMw
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICA0NDkgfCAgICAgaWYgKCAoY3B1cG9vbF9tb3ZpbmdfY3B1ID09IGNw
dSkgJiYgKGMgIT0gY3B1cG9vbF9jcHVfbW92aW5nKSApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBefgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjQ1ODoyODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgNDU4IHwgICAgIGlmIChjcHVwb29sX21vdmluZ19jcHUgPT0gY3B1KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CmNvbW1vbi9zY2hlZC9jcHVwb29s
LmM6IEluIGZ1bmN0aW9uIOKAmGNwdXBvb2xfdW5hc3NpZ25fY3B1X3N0YXJ04oCZOgpjb21t
b24vc2NoZWQvY3B1cG9vbC5jOjUyODoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBp
bnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNTI4IHwgICAgICAgICAg
JiYgKGNwdSAhPSBjcHVwb29sX21vdmluZ19jcHUpICkKICAgICAgfCAgICAgICAgICAgICAg
ICAgIF5+CmNvbW1vbi9zY2hlZC9jcHVwb29sLmM6NTM3OjE1OiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA1Mzcg
fCAgICAgICAgICAoY3B1ICE9IGNwdXBvb2xfbW92aW5nX2NwdSkgKQogICAgICB8ICAgICAg
ICAgICAgICAgXn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rpb24g4oCYY3B1
cG9vbF91bmFzc2lnbl9jcHXigJk6CmNvbW1vbi9zY2hlZC9jcHVwb29sLmM6NjI2OjE5OiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICA2MjYgfCAgICAgaWYgKCB3b3JrX2NwdSA9PSBtYXN0ZXJfY3B1ICkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICBefgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjYyOToy
Mzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgNjI5IHwgICAgICAgICBpZiAoIHdvcmtfY3B1ID09IG1hc3Rlcl9j
cHUgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICBefgpjb21tb24vc2NoZWQvY3B1
cG9vbC5jOiBJbiBmdW5jdGlvbiDigJhjcHVwb29sX2NwdV9hZGTigJk6CmNvbW1vbi9zY2hl
ZC9jcHVwb29sLmM6NzExOjMxOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA3MTEgfCAgICAgICAgICBjcHVtYXNr
X3dlaWdodChjcHVzKSA9PSBjcHVwb29sX2dldF9ncmFudWxhcml0eShjcHVwb29sMCkgKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CmNvbW1vbi9zY2hlZC9j
cHVwb29sLmM6IEluIGZ1bmN0aW9uIOKAmGR1bXBfcnVuceKAmToKY29tbW9uL3NjaGVkL2Nw
dXBvb2wuYzo5ODc6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYa2V54oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgOTg3IHwgdm9pZCBjZl9jaGVjayBkdW1wX3J1bnEodW5z
aWduZWQgY2hhciBrZXkpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rpb24g4oCYY3B1
X2NhbGxiYWNr4oCZOgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjEwMTc6Mjg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmZi4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMDE3
IHwgICAgIHN0cnVjdCBub3RpZmllcl9ibG9jayAqbmZiLCB1bnNpZ25lZCBsb25nIGFjdGlv
biwgdm9pZCAqaGNwdSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
Y29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rpb24g4oCYY3B1cG9vbF9kaXJfcmVh
ZOKAmToKY29tbW9uL3NjaGVkL2NwdXBvb2wuYzoxMDkxOjMxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGVudHJ54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMDkxIHwgICAg
IGNvbnN0IHN0cnVjdCBoeXBmc19lbnRyeSAqZW50cnksIFhFTl9HVUVTVF9IQU5ETEVfUEFS
QU0odm9pZCkgdWFkZHIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rpb24g4oCYY3B1cG9vbF9k
aXJfZXhpdOKAmToKY29tbW9uL3NjaGVkL2NwdXBvb2wuYzoxMTQxOjY1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGVudHJ54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTQx
IHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgY3B1cG9vbF9kaXJfZXhpdChjb25zdCBzdHJ1Y3Qg
aHlwZnNfZW50cnkgKmVudHJ5KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpjb21tb24vc2No
ZWQvY3B1cG9vbC5jOiBJbiBmdW5jdGlvbiDigJhjcHVwb29sX2Rpcl9maW5kZW50cnnigJk6
CmNvbW1vbi9zY2hlZC9jcHVwb29sLmM6MTE0OTozNTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkaXLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExNDkgfCAgICAgY29uc3Qg
c3RydWN0IGh5cGZzX2VudHJ5X2RpciAqZGlyLCBjb25zdCBjaGFyICpuYW1lLCB1bnNpZ25l
ZCBpbnQgbmFtZV9sZW4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOiBJbiBmdW5jdGlvbiDigJhjcHVwb29s
X2dyYW5fcmVhZOKAmToKY29tbW9uL3NjaGVkL2NwdXBvb2wuYzoxMTY4OjMxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGVudHJ54oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MTY4IHwgICAgIGNvbnN0IHN0cnVjdCBoeXBmc19lbnRyeSAqZW50cnksIFhFTl9HVUVTVF9I
QU5ETEVfUEFSQU0odm9pZCkgdWFkZHIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rpb24g4oCY
aHlwZnNfZ3Jhbl9nZXRzaXpl4oCZOgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjExODY6NzU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZW50cnnigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KIDExODYgfCBzdGF0aWMgdW5zaWduZWQgaW50IGNmX2NoZWNrIGh5cGZzX2dyYW5f
Z2V0c2l6ZShjb25zdCBzdHJ1Y3QgaHlwZnNfZW50cnkgKmVudHJ5KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzogSW4gZnVuY3Rp
b24g4oCYY3B1cG9vbF9ncmFuX3dyaXRl4oCZOgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjEy
MDI6MzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbGVhZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogMTIwMiB8ICAgICBzdHJ1Y3QgaHlwZnNfZW50cnlfbGVhZiAqbGVhZiwg
WEVOX0dVRVNUX0hBTkRMRV9QQVJBTShjb25zdF92b2lkKSB1YWRkciwKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL2NwdXBvb2wuYzog
QXQgdG9wIGxldmVsOgpjb21tb24vc2NoZWQvY3B1cG9vbC5jOjEzMjk6MTogd2FybmluZzog
4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xk
LXN0eWxlLWRlY2xhcmF0aW9uXQogMTMyOSB8IF9faW5pdGNhbGwoY3B1cG9vbF9pbml0KTsK
ICAgICAgfCBefn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L2lycS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21tb24vcG1hcC5jOjM6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAg
ICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAg
ICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2Nw
dW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2
NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDozNiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaXJx
Lmg6NToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJs
aW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBm
dW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFk
X2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhs
b2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lk
IHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3BtYXAuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gY29tbW9uL3BtYXAuYzo2OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rp
b24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNo
dGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9m
aWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tf
dCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9h
Y2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9w
bGF0Zm9ybS9hY2xpbnV4Lmg6NTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
YWNwaS9wbGF0Zm9ybS9hY2Vudi5oOjE0MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS9hY3BpL2FjcGkuaDo1NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vYWNwaS5oOjU3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9maXhtYXAuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wbWFwLmg6NjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJz
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9w
YWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1f
dHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24vbGli
ZmR0L2ZkdF9yby5vCi4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3Bp
X3NldF9jc3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxOTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0
KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+
Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3Rh
dGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
OTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2ln
bmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+CiAg
Q0MgICAgICBjb21tb24vd2FybmluZy5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nv
ZnRpcnEuaDoxNiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi93YXJuaW5nLmM6NDoK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAg
ICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAg
IF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3lj
bGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQw
IHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192
YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3Rh
dGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZy
ZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjY6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgogIENDICAgICAgYXJjaC9hcm0vZ2ljLXYyLm8KICBDQyAgICAgIGFyY2gvYXJtL2d1
ZXN0Y29weS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
cGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NjksCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9naWMtdjIuYzoxMzoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAg
ICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAg
IGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlu
bGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1
IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBt
YXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9t
bS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW5fcGFn
ZS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vZ3Vlc3Rjb3B5LmM6MjoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1
bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVj
dG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nw
aW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g
4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194
KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9m
aWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJf
dCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2lycS5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSBhcmNoL2FybS9naWMtdjIuYzoxNToKLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2Nw
dV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2Nw
dV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
aW9jYXAuaDoxMCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2dpYy12Mi5jOjE2
OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUg
bm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2
b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9h
Y2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21l
bV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVp
cmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0
OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAy
bV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFp
biAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJv
b2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGlj
IGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9n
dWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1v
cnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xp
Y3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L2RvbWFpbl9wYWdlLmg6IEluIGZ1bmN0aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2Jh
bOKAmToKLi9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHZh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3Rh
dGljIGlubGluZSB2b2lkIHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2
YSkge307CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNp
X2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2Rl
dmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2
cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2d1ZXN0X2FjY2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL2d1ZXN0X2FjY2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9h
cm0vZ3Vlc3Rjb3B5LmM6MzoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKA
mG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkg
fCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFk
X2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUu
aDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxv
Y2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZj
cHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nh
bml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0
IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2ln
bmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9l
dmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
aW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu
4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFs
d2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5
X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShz
dHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFy
Y2gvYXJtL2dpYy12Mi5jOjI0OgouL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVuY3Rpb24g
4oCYYWNwaV9zZXRfY3N0YXRlX2xpbWl04oCZOgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTI6
NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTkyIHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdGF0
ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fn5+fn5+fgouL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRf
Y3N1YnN0YXRlX2xpbWl04oCZOgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTQ6NTg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTk0IHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1p
dCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1Ogou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBp
bmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVh
c3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNp
X2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dy
aXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRl
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5k
aW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vZ2lj
LXYyLmM6IEluIGZ1bmN0aW9uIOKAmGdpY3YyX2Rpc3RfaW5pdOKAmToKYXJjaC9hcm0vZ2lj
LXYyLmM6MzU3OjIxOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgZm9yICggaSA9IDMyOyBpIDwg
bnJfbGluZXM7IGkgKz0gMTYgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgXgphcmNo
L2FybS9naWMtdjIuYzozNjE6MjE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM2MSB8ICAgICBmb3IgKCBpID0g
MzI7IGkgPCBucl9saW5lczsgaSArPSA0ICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
IF4KYXJjaC9hcm0vZ2ljLXYyLmM6MzY1OjIxOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNjUgfCAgICAgZm9y
ICggaSA9IDMyOyBpIDwgbnJfbGluZXM7IGkgKz0gNCApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICBeCmFyY2gvYXJtL2dpYy12Mi5jOjM2OToyMTogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzY5IHwg
ICAgIGZvciAoIGkgPSAzMjsgaSA8IG5yX2xpbmVzOyBpICs9IDMyICkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgIF4KYXJjaC9hcm0vZ2ljLXYyLmM6IEluIGZ1bmN0aW9uIOKAmGdp
Y3YyX3JlYWRfYXBy4oCZOgphcmNoL2FybS9naWMtdjIuYzo1ODc6NDA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYYXByX3JlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4
NyB8IHN0YXRpYyB1bnNpZ25lZCBpbnQgZ2ljdjJfcmVhZF9hcHIoaW50IGFwcl9yZWcpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+fn5+fgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFp
buKAmToKYXJjaC9hcm0vZ2ljLXYyLmM6IEluIGZ1bmN0aW9uIOKAmGdpY3YyX2lycV9hY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3
YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vZ2ljLXYyLmM6NjM2OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlc2PigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MzYgfCBzdGF0aWMgdm9pZCBnaWN2Ml9pcnFfYWNrKHN0cnVjdCBpcnFfZGVzYyAq
ZGVzYykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgphcmNoL2FybS9naWMtdjIuYzogSW4gZnVuY3Rpb24g4oCYZ2ljdjJfaXNfYWxp
YXNlZOKAmToKYXJjaC9hcm0vZ2ljLXYyLmM6OTA0OjM4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGNiYXNl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgOTA0IHwgc3RhdGlj
IGJvb2wgZ2ljdjJfaXNfYWxpYXNlZChwYWRkcl90IGNiYXNlLCBwYWRkcl90IGNzaXplKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5efn5+fgphcmNo
L2FybS9naWMtdjIuYzogSW4gZnVuY3Rpb24g4oCYZ2ljdjJfZ2V0X2h3ZG9tX2V4dHJhX21h
ZHRfc2l6ZeKAmToKYXJjaC9hcm0vZ2ljLXYyLmM6MTEwMjo3NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTAyIHwgc3RhdGlj
IHVuc2lnbmVkIGxvbmcgZ2ljdjJfZ2V0X2h3ZG9tX2V4dHJhX21hZHRfc2l6ZShjb25zdCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNoL2Fy
bS9naWMtdjIuYzogSW4gZnVuY3Rpb24g4oCYZ2ljdjJfZG9fTFBJ4oCZOgphcmNoL2FybS9n
aWMtdjIuYzoxMzA4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxwaeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogMTMwOCB8IHN0YXRpYyB2b2lkIGdpY3YyX2RvX0xQSSh1
bnNpZ25lZCBpbnQgbHBpKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL2dpYy12Mi5jOiBBdCB0b3AgbGV2ZWw6CmFyY2gvYXJt
L2dpYy12Mi5jOjEzMTQ6MTogd2FybmluZzog4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdp
bm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogMTMxNCB8
IGNvbnN0IHN0YXRpYyBzdHJ1Y3QgZ2ljX2h3X29wZXJhdGlvbnMgZ2ljdjJfb3BzID0gewog
ICAgICB8IF5+fn5+CmFyY2gvYXJtL2dpYy12Mi5jOiBJbiBmdW5jdGlvbiDigJhnaWN2Ml9k
dF9wcmVpbml04oCZOgphcmNoL2FybS9naWMtdjIuYzoxMzUyOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEzNTIgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHZvaWQgKmRhdGEpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5e
fn5+CmFyY2gvYXJtL2dpYy12Mi5jOiBJbiBmdW5jdGlvbiDigJhnaWN2Ml9hY3BpX3ByZWlu
aXTigJk6CmFyY2gvYXJtL2dpYy12Mi5jOjEzNzU6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTM3NSB8IHN0YXRpYyBp
bnQgX19pbml0IGdpY3YyX2FjcGlfcHJlaW5pdChjb25zdCB2b2lkICpkYXRhKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efn5+
CiAgQ0MgICAgICBhcmNoL2FybS9ndWVzdF9hdG9taWNzLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vY3B1Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL2d1ZXN0X2F0b21pY3MuYzo1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlu
bGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9F
TkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAg
IHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgs
IHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ndWVzdF9hdG9taWNzLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIGFyY2gvYXJtL2d1ZXN0X2F0b21pY3MuYzo3OgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4u
aDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9h
bHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9n
Zm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQs
IHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFy
Y2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8z
MmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEg
fCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vZ3Vlc3Rf
YXRvbWljcy5jOiBJbiBmdW5jdGlvbiDigJhjcHVfZ3Vlc3Rfc2FmZV9hdG9taWNfY2FsbGJh
Y2vigJk6CmFyY2gvYXJtL2d1ZXN0X2F0b21pY3MuYzo0OTo2Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhuZmLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDkgfCBzdGF0
aWMgaW50IGNwdV9ndWVzdF9zYWZlX2F0b21pY19jYWxsYmFjayhzdHJ1Y3Qgbm90aWZpZXJf
YmxvY2sgKm5mYiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS9ndWVzdF9hdG9t
aWNzLmM6NTE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA1MSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHZvaWQgKmhjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+fgphcmNoL2FybS9ndWVzdF9hdG9taWNz
LmM6IEF0IHRvcCBsZXZlbDoKYXJjaC9hcm0vZ3Vlc3RfYXRvbWljcy5jOjcxOjE6IHdhcm5p
bmc6IOKAmHN0YXRpY+KAmSBpcyBub3QgYXQgYmVnaW5uaW5nIG9mIGRlY2xhcmF0aW9uIFst
V29sZC1zdHlsZS1kZWNsYXJhdGlvbl0KICAgNzEgfCBwcmVzbXBfaW5pdGNhbGwoZ3Vlc3Rf
c2FmZV9hdG9taWNfaW5pdCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBj
b21tb24vbGliZmR0L2ZkdF93aXAubwogIENDICAgICAgYXJjaC9hcm0vZ3Vlc3Rfd2Fsay5v
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6MTIsCiAgICAg
ICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9ndWVzdF93YWxrLmM6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5n
c+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQg
Y3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0
bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwu
aDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0
MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9T
QVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6
NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZ
UEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FW
RV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+
fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAg
Y3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChu
eHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBi
b29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192
YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3Rh
dGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+fgogIENDICAgICAgY29tbW9uL2xpYmZkdC9mZHRfc3cubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6IEluIGZ1bmN0
aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNsdWRlL3hlbi9kb21h
aW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZh4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2b2lkIHVubWFwX2Rv
bWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAg
ICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2d1ZXN0X3dhbGsuYzo4OgouL2luY2x1ZGUveGVu
L251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9u
dW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3Rv
X25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRp
eC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2Rv
bWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlv
biDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1
X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
NCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1f
ZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0
cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWlu
dDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9j
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1v
cnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBk
ZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2
cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rldmlj
ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYp
IHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3Bl
bmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGlj
IGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3Qg
dmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDog
SW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19w
dl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5eCiAgQ0MgICAgICBjb21tb24vbGliZmR0L2ZkdF9ydy5vCiAgQ0MgICAgICBhcmNo
L2FybS9odm0ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2d1
ZXN0X2FjY2Vzcy5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2d1
ZXN0X2FjY2Vzcy5oOjEwLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vaHZtLmM6
MTE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVy
ZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGlu
Z3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVu
Y3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNr
X2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAg
ICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAg
ICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2Nw
dW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1
MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1m
bl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBz
dGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25n
IG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3Rf
ZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVu
Y3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Zs
dXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1h
c2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3Jh
ZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDi
gJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9s
b2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVu
Y3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2Nr
KHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5p
dHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYs
IHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25l
ZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVf
YmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g
4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRl
X2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3Rf
bWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBp
bnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vz
c19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3Ry
dWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgogIENDICAgICAgY29tbW9uL3Nj
aGVkL2FyaW5jNjUzLm8KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhp
c19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
IDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihj
b25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAgICAgICAg
ICAgICAgZnJvbSBhcmNoL2FybS9odm0uYzoxMjoKLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFpbmluZm/i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIzNToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGlu
dCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
c3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3Rh
cmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICplKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1MTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTY6
NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21fZGVmYXVs
dF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2
NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAz
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdW5tYXBy
ZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJhbnNmZXLi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9oYXJkd2Fy
ZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9k
X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2
OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3RhcmdldCh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9t
ZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4Nzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2Vy
dmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9f
cGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21h
cF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQx
OCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRv
bWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1f
cGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4
c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50
IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJv
ZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV91bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRf
cHRfaXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3Rs
X2JpbmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ5NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVp
bnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9w
Y2lfY29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWlu
dDMyX3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNz
aWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIg
KmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X3Jlc291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2Rl
ZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291
cmNlX3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
NjM6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1
Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0
dXBfZ3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVm
LCBpbnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTc5OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1
cF9taXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdl
X29mZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X2h5cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYw
NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJt
aHZt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMg
aW5saW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dldF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYy
MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxh
dGZvcm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWlu
dDMyX3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9kbV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3Zl
cnNpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMy
X3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9k
b21haW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA3NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3Mu
aDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGlt
ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAg
ICAgICAgZnJvbSBjb21tb24vc2NoZWQvYXJpbmM2NTMuYzoyODoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEw
MSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+CiAgQ0MgICAgICBjb21tb24vbGliZmR0L2ZkdF9zdHJlcnJvci5vCkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E
4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBz
dGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1
aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGlu
ZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6
ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1f
U0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3Ry
dWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2Yg
ZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZ
IFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lk
czsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25z
IG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGlu
dOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lk
cykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lk
IGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEu
aDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6
MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZCht
Zm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
bS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9y
ZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwg
dm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9j
aGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVj
dCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVl
cnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlu
bGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVf
YmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6
NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
YXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjgz
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZp
Y2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBj
aV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQz
Ml90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMy
X3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNp
X3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9t
YWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGlj
IGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3VuaXRfb25saW5l4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6OTE6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZv
cl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogICA5MSB8ICAgICBmb3JfZWFjaF9zY2hlZF91
bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHVuaXRfcnVubmFi
bGXigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2lu
X2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDoxMDg6NTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQog
IDEwOCB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAg
fCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHVuaXRfcnVubmFibGVfc3RhdGXigJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29t
bW9uL3NjaGVkL3ByaXZhdGUuaDoxMzU6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogIDEzNSB8ICAgICBmb3JfZWFjaF9z
Y2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVk
X3NldF9yZXPigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5u
ZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDoxNTQ6NTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNw
deKAmQogIDE1NCB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkK
ICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX3NldF9wYXVzZV9mbGFnc+KAmToKLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9p
ZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjE3Njo1OiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgMTc2IHwgICAgIGZv
cl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g
4oCYc2NoZWRfY2xlYXJfcGF1c2VfZmxhZ3PigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoz
NTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYp
LT52Y3B1X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVk
L3ByaXZhdGUuaDoxODY6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9l
YWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogIDE4NiB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0
X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
CiAgQ0MgICAgICBhcmNoL2FybS9pby5vCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHNjaGVkX3VuaXRfcGF1c2Vfbm9zeW5j4oCZOgouL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1v
bi9zY2hlZC9wcml2YXRlLmg6NTMwOjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICA1MzAgfCAgICAgZm9yX2VhY2hfc2No
ZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF91
bml0X3VucGF1c2XigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUp
LT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDo1Mzg6
NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRf
dmNwdeKAmQogIDUzOCB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2
ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9h
cmluYzY1My5jOiBJbiBmdW5jdGlvbiDigJhmaW5kX3VuaXTigJk6CmNvbW1vbi9zY2hlZC9h
cmluYzY1My5jOjE5MzoyNjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwgICAgICAgICAgICAgICYmICh1
bml0X2lkID09IGF1bml0LT51bml0LT51bml0X2lkKSApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgIF5+CmNvbW1vbi9zY2hlZC9hcmluYzY1My5jOiBJbiBmdW5jdGlvbiDi
gJhhcmluYzY1M19zY2hlZF9zZXTigJk6CmNvbW1vbi9zY2hlZC9hcmluYzY1My5jOjI2MToy
NDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhzX3RpbWVfdOKAmSB7YWthIOKAmGxvbmcgaW504oCZfSBh
bmQg4oCYdWludDY0X3TigJkge2FrYSDigJhsb25nIHVuc2lnbmVkIGludOKAmX0gWy1Xc2ln
bi1jb21wYXJlXQogIDI2MSB8ICAgICBpZiAoIHRvdGFsX3J1bnRpbWUgPiBzY2hlZHVsZS0+
bWFqb3JfZnJhbWUgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24v
c2NoZWQvYXJpbmM2NTMuYzogSW4gZnVuY3Rpb24g4oCYYTY1M3NjaGVkX2FsbG9jX3VkYXRh
4oCZOgpjb21tb24vc2NoZWQvYXJpbmM2NTMuYzozODY6Mjk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODYgfCAgICAgICAg
ICAgICAgICAgICAgICAgdm9pZCAqZGQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+CmNvbW1vbi9zY2hlZC9hcmluYzY1My5jOiBJbiBmdW5jdGlvbiDigJhhNjUz
c2NoZWRfdW5pdF9zbGVlcOKAmToKY29tbW9uL3NjaGVkL2FyaW5jNjUzLmM6NDczOjQ2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MyB8IGE2NTNzY2hlZF91bml0X3NsZWVwKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIg
Km9wcywgc3RydWN0IHNjaGVkX3VuaXQgKnVuaXQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9hcmluYzY1
My5jOiBJbiBmdW5jdGlvbiDigJhhNjUzc2NoZWRfdW5pdF93YWtl4oCZOgpjb21tb24vc2No
ZWQvYXJpbmM2NTMuYzo0OTM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Bz
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDkzIHwgYTY1M3NjaGVkX3VuaXRfd2FrZShj
b25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHN0cnVjdCBzY2hlZF91bml0ICp1bml0KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
CmNvbW1vbi9zY2hlZC9hcmluYzY1My5jOiBJbiBmdW5jdGlvbiDigJhhNjUzc2NoZWRfcGlj
a19yZXNvdXJjZeKAmToKY29tbW9uL3NjaGVkL2FyaW5jNjUzLmM6NjA4OjQ5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYw
OCB8IGE2NTNzY2hlZF9waWNrX3Jlc291cmNlKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9w
cywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KY29tbW9uL3NjaGVkL2FyaW5jNjUzLmM6IEluIGZ1bmN0aW9uIOKAmGE2NTNf
c3dpdGNoX3NjaGVk4oCZOgpjb21tb24vc2NoZWQvYXJpbmM2NTMuYzo2Mzg6Mzc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3X29wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYzOCB8IGE2NTNfc3dpdGNoX3NjaGVkKHN0cnVjdCBzY2hlZHVsZXIgKm5ld19vcHMs
IHVuc2lnbmVkIGludCBjcHUsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2lvcmVx
Lmg6MjIsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9pby5jOjEwOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3Np
Ymxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVk
IGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMv
ZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5
c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9j
ay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9f
SFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklY
X0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NB
VkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+
fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBv
ZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hW
TV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+
fn5+fn5+fn5+fn5+fn5+fn4KICBDQyAgICAgIGNvbW1vbi9saWJmZHQvZmR0X2VtcHR5X3Ry
ZWUubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1m
bl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
bW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFp
bi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1
bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBh
Z2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIo
c3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQog
ICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNr
LCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFn
ZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFn
ZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5l
IHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNj
ZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2
X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAy
bV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9t
X2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8
ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAq
dCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQg
YXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9w
b2xpY3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MDkgfCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9t
YWluLmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0
cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVh
c3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5l
IHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
d3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3Rf
Y2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2
XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEz
MSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gYXJjaC9hcm0vaW8uYzoxNzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9pb3Jl
cS5oOiBJbiBmdW5jdGlvbiDigJhoYW5kbGVfcGlv4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2lvcmVxLmg6NDE6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcG9ydOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA0MSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBoYW5k
bGVfcGlvKHVpbnQxNl90IHBvcnQsIHVuc2lnbmVkIGludCBzaXplLCBpbnQgZGlyKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOjQxOjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDEgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgaGFuZGxlX3Bpbyh1aW50MTZfdCBwb3J0LCB1bnNpZ25lZCBpbnQgc2l6ZSwg
aW50IGRpcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvcmVx
Lmg6NDE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGly4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDQxIHwgc3RhdGljIGlubGluZSBib29sIGhhbmRsZV9waW8odWlu
dDE2X3QgcG9ydCwgdW5zaWduZWQgaW50IHNpemUsIGludCBkaXIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOiBJbiBmdW5jdGlv
biDigJhtc2l4X3dyaXRlX2NvbXBsZXRpb27igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
aW9yZXEuaDo1MTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDUxIHwgc3RhdGljIGlubGluZSB2b2lkIG1zaXhfd3JpdGVf
Y29tcGxldGlvbihzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2lvLmM6IElu
IGZ1bmN0aW9uIOKAmHN3YXBfbW1pb19oYW5kbGVy4oCZOgphcmNoL2FybS9pby5jOjc0OjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNzQgfCBzdGF0aWMgdm9pZCBzd2FwX21taW9faGFuZGxlcih2b2lkICpfYSwg
dm9pZCAqX2IsIHNpemVfdCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fl5+fn4KICBDQyAgICAgIGNvbW1v
bi9saWJmZHQvZmR0X2FkZHJlc3Nlcy5vCiAgQ0MgICAgICBjb21tb24vbGliZmR0L2ZkdF9v
dmVybGF5Lm8KICBDQyAgICAgIGFyY2gvYXJtL2lvcmVxLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL2lvcmVxLmM6ODoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKA
mG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkg
fCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6
MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZXIuaDoxMSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3Pi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNw
dSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwu
aDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6
MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FW
RV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFU
XyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzAsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjc6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAg
IGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAo
bnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tf
dmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJj
aF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5f
ZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25l
eHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1
bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8
ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDoz
NywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJt
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0
IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAy
bV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0
cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1f
cXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBjb21tb24vbGliZmR0L2ZkdF9jaGVjay5v
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
dmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZj
cHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5j
dGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBn
dWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaW9yZXEuaDoyMiwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL2lvcmVxLmM6OToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2ln
bl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9y
ZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRp
YyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVu
c2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBi
b29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2FybS9pcnEubwouL2luY2x1
ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToK
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNf
aW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9hcm0v
aW9yZXEuYzoxMjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOiBJbiBmdW5jdGlv
biDigJhoYW5kbGVfcGlv4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvcmVxLmg6NDE6
NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcG9ydOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA0MSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBoYW5kbGVfcGlvKHVpbnQxNl90
IHBvcnQsIHVuc2lnbmVkIGludCBzaXplLCBpbnQgZGlyKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9pb3JlcS5oOjQxOjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDEgfCBzdGF0aWMgaW5saW5lIGJvb2wgaGFuZGxl
X3Bpbyh1aW50MTZfdCBwb3J0LCB1bnNpZ25lZCBpbnQgc2l6ZSwgaW50IGRpcikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2lvcmVxLmg6NDE6Njk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGly4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDQxIHwgc3RhdGljIGlubGluZSBib29sIGhhbmRsZV9waW8odWludDE2X3QgcG9ydCwgdW5z
aWduZWQgaW50IHNpemUsIGludCBkaXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9pb3JlcS5oOiBJbiBmdW5jdGlvbiDigJhtc2l4X3dyaXRl
X2NvbXBsZXRpb27igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vaW9yZXEuaDo1MTo1NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDUxIHwgc3RhdGljIGlubGluZSB2b2lkIG1zaXhfd3JpdGVfY29tcGxldGlvbihzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2lvcmVxLmM6IEluIGZ1bmN0aW9uIOKA
mGFyY2hfdmNwdV9pb3JlcV9jb21wbGV0aW9u4oCZOgphcmNoL2FybS9pb3JlcS5jOjEzODo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjb21wbGV0aW9u4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTM4IHwgYm9vbCBhcmNoX3ZjcHVfaW9yZXFfY29tcGxldGlvbihl
bnVtIHZpb19jb21wbGV0aW9uIGNvbXBsZXRpb24pCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+fn5+fn4KYXJjaC9h
cm0vaW9yZXEuYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9pb3JlcV9zZXJ2ZXJfbWFwX3BhZ2Vz
4oCZOgphcmNoL2FybS9pb3JlcS5jOjE1Mjo1NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTUyIHwgaW50IGFyY2hfaW9yZXFf
c2VydmVyX21hcF9wYWdlcyhzdHJ1Y3QgaW9yZXFfc2VydmVyICpzKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNo
L2FybS9pb3JlcS5jOiBJbiBmdW5jdGlvbiDigJhhcmNoX2lvcmVxX3NlcnZlcl91bm1hcF9w
YWdlc+KAmToKYXJjaC9hcm0vaW9yZXEuYzoxNTc6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE1NyB8IHZvaWQgYXJjaF9p
b3JlcV9zZXJ2ZXJfdW5tYXBfcGFnZXMoc3RydWN0IGlvcmVxX3NlcnZlciAqcykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KYXJjaC9hcm0vaW9yZXEuYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9pb3JlcV9zZXJ2
ZXJfZW5hYmxl4oCZOgphcmNoL2FybS9pb3JlcS5jOjE2MTo1Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTYxIHwgdm9pZCBh
cmNoX2lvcmVxX3NlcnZlcl9lbmFibGUoc3RydWN0IGlvcmVxX3NlcnZlciAqcykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CmFyY2gvYXJtL2lvcmVxLmM6IEluIGZ1bmN0aW9uIOKAmGFyY2hfaW9yZXFfc2VydmVyX2Rp
c2FibGXigJk6CmFyY2gvYXJtL2lvcmVxLmM6MTY1OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNjUgfCB2b2lkIGFyY2hf
aW9yZXFfc2VydmVyX2Rpc2FibGUoc3RydWN0IGlvcmVxX3NlcnZlciAqcykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgph
cmNoL2FybS9pb3JlcS5jOiBJbiBmdW5jdGlvbiDigJhhcmNoX2lvcmVxX3NlcnZlcl9kZXN0
cm954oCZOgphcmNoL2FybS9pb3JlcS5jOjE2OTo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTY5IHwgdm9pZCBhcmNoX2lv
cmVxX3NlcnZlcl9kZXN0cm95KHN0cnVjdCBpb3JlcV9zZXJ2ZXIgKnMpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KYXJj
aC9hcm0vaW9yZXEuYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9pb3JlcV9zZXJ2ZXJfbWFwX21l
bV90eXBl4oCZOgphcmNoL2FybS9pb3JlcS5jOjE3Mzo1MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTczIHwgaW50IGFyY2hf
aW9yZXFfc2VydmVyX21hcF9tZW1fdHlwZShzdHJ1Y3QgZG9tYWluICpkLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgphcmNo
L2FybS9pb3JlcS5jOjE3NDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBzdHJ1Y3QgaW9yZXFfc2VydmVyICpzLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNoL2Fy
bS9pb3JlcS5jOjE3NTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhmbGFnc+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fgphcmNoL2FybS9pb3JlcS5jOiBJbiBm
dW5jdGlvbiDigJhhcmNoX2lvcmVxX3NlcnZlcl9tYXBfbWVtX3R5cGVfY29tcGxldGVk4oCZ
OgphcmNoL2FybS9pb3JlcS5jOjE4MDo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTgwIHwgdm9pZCBhcmNoX2lvcmVxX3Nl
cnZlcl9tYXBfbWVtX3R5cGVfY29tcGxldGVkKHN0cnVjdCBkb21haW4gKmQsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fl4KYXJjaC9hcm0vaW9yZXEuYzoxODE6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4MSB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgaW9yZXFfc2VydmVy
ICpzLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL2lvcmVxLmM6MTgyOjU2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGZsYWdz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTgyIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fgphcmNoL2FybS9pb3JlcS5jOiBJ
biBmdW5jdGlvbiDigJhhcmNoX2lvcmVxX3NlcnZlcl9kZXN0cm95X2FsbOKAmToKYXJjaC9h
cm0vaW9yZXEuYzoxODY6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NiB8IGJvb2wgYXJjaF9pb3JlcV9zZXJ2ZXJfZGVz
dHJveV9hbGwoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vaW9yZXEuYzogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9pb3JlcV9zZXJ2ZXJfZ2V0X3R5cGVfYWRkcuKAmToKYXJjaC9h
cm0vaW9yZXEuYzoxOTE6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5MSB8IGJvb2wgYXJjaF9pb3JlcV9zZXJ2ZXJfZ2V0
X3R5cGVfYWRkcihjb25zdCBzdHJ1Y3QgZG9tYWluICpkLAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCmFyY2gv
YXJtL2lvcmVxLmM6IEluIGZ1bmN0aW9uIOKAmGFyY2hfaW9yZXFfZG9tYWluX2luaXTigJk6
CmFyY2gvYXJtL2lvcmVxLmM6MjA2OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMDYgfCB2b2lkIGFyY2hfaW9yZXFfZG9t
YWluX2luaXQoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9jcHUuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vaXJx
LmM6MTE6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlv
biDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVh
dHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2li
bGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4g
ZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGji
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUu
aDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4
NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQp
OwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9pcnEuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0v
aXJxLmM6MTQ6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVt
YXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5p
bmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2ln
bmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21w
YXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAg
ICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9j
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFy
KGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vaXJxLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIw
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDog
SW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
dW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgogIExEICAgICAgY29t
bW9uL2xpYmZkdC9saWJmZHQtdGVtcC5vCiAgQ0MgICAgICBjb21tb24vc2NoZWQvY3JlZGl0
Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL2lycS5jOjE3OgouL2luY2x1ZGUveGVuL251bWEuaDog
SW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5
OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5f
dCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5o
OjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5z
X2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEw
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0
bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6
Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9p
bmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAg
ICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJm
bHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2
bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Js
b2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3
IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKA
mHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9n
dWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21l
bW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2
aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3Nf
cGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVj
dCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KYXJjaC9hcm0vaXJxLmM6IEluIGZ1bmN0aW9uIOKAmGNwdV9jYWxsYmFja+KA
mToKYXJjaC9hcm0vaXJxLmM6MTI2OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5mYuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyNiB8IHN0YXRpYyBpbnQgY3B1X2Nh
bGxiYWNrKHN0cnVjdCBub3RpZmllcl9ibG9jayAqbmZiLCB1bnNpZ25lZCBsb25nIGFjdGlv
biwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgphcmNoL2FybS9pcnEuYzogSW4gZnVuY3Rpb24g4oCYYWxsb2NfcGlycV9zdHJ1
Y3TigJk6CmFyY2gvYXJtL2lycS5jOjYwMjo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjAyIHwgc3RydWN0IHBpcnEgKmFs
bG9jX3BpcnFfc3RydWN0KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vaXJxLmM6IElu
IGZ1bmN0aW9uIOKAmHBpcnFfZ3Vlc3RfYmluZOKAmToKYXJjaC9hcm0vaXJxLmM6NjEyOjM0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA2MTIgfCBpbnQgcGlycV9ndWVzdF9iaW5kKHN0cnVjdCB2Y3B1ICp2LCBzdHJ1Y3Qg
cGlycSAqcGlycSwgaW50IHdpbGxfc2hhcmUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9pcnEuYzo2MTI6NTA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYxMiB8IGlu
dCBwaXJxX2d1ZXN0X2JpbmQoc3RydWN0IHZjcHUgKnYsIHN0cnVjdCBwaXJxICpwaXJxLCBp
bnQgd2lsbF9zaGFyZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9pcnEuYzo2MTI6NjA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYd2lsbF9zaGFyZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDYxMiB8IGludCBwaXJxX2d1ZXN0X2JpbmQoc3RydWN0IHZjcHUgKnYsIHN0cnVjdCBw
aXJxICpwaXJxLCBpbnQgd2lsbF9zaGFyZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+fn5+fn5+fn4KYXJj
aC9hcm0vaXJxLmM6IEluIGZ1bmN0aW9uIOKAmHBpcnFfZ3Vlc3RfdW5iaW5k4oCZOgogIE9C
SkNPUFkgY29tbW9uL2xpYmZkdC9saWJmZHQubwphcmNoL2FybS9pcnEuYzo2MTc6Mzk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDYxNyB8IHZvaWQgcGlycV9ndWVzdF91bmJpbmQoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IHBpcnEgKnBpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCmFyY2gvYXJtL2lycS5jOjYxNzo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjE3IHwgdm9pZCBwaXJxX2d1
ZXN0X3VuYmluZChzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgcGlycSAqcGlycSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+CmFyY2gvYXJtL2lycS5jOiBJbiBmdW5jdGlvbiDigJhwaXJxX3NldF9hZmZpbml0
eeKAmToKYXJjaC9hcm0vaXJxLmM6NjIyOjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjIgfCB2b2lkIHBpcnFfc2V0X2Fm
ZmluaXR5KHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCBjb25zdCBjcHVtYXNrX3QgKm1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmFy
Y2gvYXJtL2lycS5jOjYyMjo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwaXJx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIyIHwgdm9pZCBwaXJxX3NldF9hZmZpbml0
eShzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgY29uc3QgY3B1bWFza190ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fl5+
fn4KYXJjaC9hcm0vaXJxLmM6NjIyOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjIgfCB2b2lkIHBpcnFfc2V0X2Fm
ZmluaXR5KHN0cnVjdCBkb21haW4gKmQsIGludCBwaXJxLCBjb25zdCBjcHVtYXNrX3QgKm1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CiAgTEQgICAgICBjb21tb24vbGliZmR0
L2J1aWx0X2luLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGNvbW1vbi9zY2hlZC9jcmVkaXQu
YzoxMzoKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDigJhub19jb25maWdf
cGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdhcm5pbmc6IG9wZXJh
bmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYbG9uZyBpbnTigJkg
dG8g4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTigJl9IGR1ZSB0byB1
bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAgMTkzIHwg
ICAgIGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBzOyB9KSA6IHN0cmxl
bihzKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fs
dGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
NiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zY2hlZC9jcmVkaXQuYzoxNDoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9u
cl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAx
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNp
Z25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVi
bGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1Ymxp
Yy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bp
bmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDi
gJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4K
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBv
ZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hW
TV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+
fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFU
XyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBjb21tb24vY29uZmlnX2RhdGEubwog
IENDICAgICAgYXJjaC9hcm0va2VybmVsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
cndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19s
YXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIz
MCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAg
IF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAq
bWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g
4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5f
dG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3Rh
dGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3Rh
dGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBt
Zm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFy
Y2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcg
bnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2Rl
bChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0
aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hf
ZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+fn5+fn5+fn5+fgogIElOSVRfTyAgY29tbW9uL3BtYXAuaW5pdC5vCkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNz
Lmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlv
biDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vz
c19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
c3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJv
b2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2Fs
dHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9h
bHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dm
buKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAg
ICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwg
cDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgogIElOSVRfTyAgY29tbW9uL3dhcm5pbmcuaW5pdC5v
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
dmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZj
cHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5j
dGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBn
dWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5s
aW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNz
aWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3
OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0
ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9w
cm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAw
IHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGlu
ZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4v
c2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1
ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUg
Ym9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2FybS9saXZlcGF0Y2gubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjE3LAogICAgICAgICAgICAg
ICAgIGZyb20gY29tbW9uL3NjaGVkL2NyZWRpdC5jOjE3OgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZXZlbnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh2aXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGlj
IGlubGluZSBib29sIGFyY2hfdmlycV9pc19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYaXNfdW5p
dF9vbmxpbmXigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5u
ZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDo5MTo1OiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1
4oCZCiAgIDkxIHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQog
ICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZh
dGUuaDogSW4gZnVuY3Rpb24g4oCYdW5pdF9ydW5uYWJsZeKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21t
b24vc2NoZWQvcHJpdmF0ZS5oOjEwODo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g
4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgMTA4IHwgICAgIGZvcl9lYWNoX3Nj
aGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYdW5pdF9y
dW5uYWJsZV9zdGF0ZeKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAo
dSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjEz
NTo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5p
dF92Y3B14oCZCiAgMTM1IHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQs
IHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVk
L3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYc2NoZWRfc2V0X3Jlc+KAmToKLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7
ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Xgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjE1NDo1OiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgMTU0IHwgICAgIGZvcl9l
YWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCY
c2NoZWRfc2V0X3BhdXNlX2ZsYWdz4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQw
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNw
dV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2
YXRlLmg6MTc2OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9z
Y2hlZF91bml0X3ZjcHXigJkKICAxNzYgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1
ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21t
b24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9jbGVhcl9wYXVzZV9m
bGFnc+KAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFy
aXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDi
gJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRf
aW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjE4Njo1OiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZ
CiAgMTg2IHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAg
ICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3ByaXZhdGUu
aDogSW4gZnVuY3Rpb24g4oCYc2NoZWRfdW5pdF9wYXVzZV9ub3N5bmPigJk6Ci4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQp
OyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDo1MzA6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogIDUzMCB8ICAgICBmb3Jf
ZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHNjaGVkX3VuaXRfdW5wYXVzZeKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVf
aWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0
ZS5oOjUzODo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2No
ZWRfdW5pdF92Y3B14oCZCiAgNTM4IHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAo
IHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9saXZlcGF0Y2guaDoxNywKICAgICAgICAgICAg
ICAgICBmcm9tIGFyY2gvYXJtL2xpdmVwYXRjaC5jOjk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi8uL2luY2x1ZGUveGVuL2NvbmZpZy5oOjE3LAogICAgICAgICAgICAg
ICAgIGZyb20gPGNvbW1hbmQtbGluZT46CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVu
Y3Rpb24g4oCYX19ydW5xX3RpY2tsZeKAmToKY29tbW9uL3NjaGVkL2NyZWRpdC5jOjM4OTo2
MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMzg5IHwgICAgICAgICBBU1NFUlQoY3B1bWFza19jeWNsZShjcHUs
IHVuaXQtPmNwdV9oYXJkX2FmZmluaXR5KSA9PSBjcHUpOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBpbiBkZWZpbml0aW9uIG9mIG1h
Y3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1bmxpa2VseSh4KSAgIF9fYnVp
bHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jcmVkaXQuYzozODk6OTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKAmQogIDM4OSB8ICAgICAgICAgQVNT
RVJUKGNwdW1hc2tfY3ljbGUoY3B1LCB1bml0LT5jcHVfaGFyZF9hZmZpbml0eSkgPT0gY3B1
KTsKICAgICAgfCAgICAgICAgIF5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9kb21haW5fcGFnZS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0va2Vy
bmVsLmM6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9z
aWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzog
SW4gZnVuY3Rpb24g4oCYY3NjaGVkX2RlaW5pdF9wZGF0YeKAmToKY29tbW9uL3NjaGVkL2Ny
ZWRpdC5jOjU1NzoyMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTigJkgYW5kIOKA
mGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNTU3IHwgICAgIGlmICggKHBydi0+bWFzdGVy
ID09IGNwdSkgJiYgKHBydi0+bmNwdXMgPiAwKSApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgIF5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzo1NjI6MzQ6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYdWludDMyX3TigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IGFuZCDigJhpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDU2MiB8ICAgICBpZiAoIHBydi0+YmFsYW5jZV9iaWFzW25v
ZGVdID09IGNwdSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Xn4KY29tbW9uL3NjaGVkL2NyZWRpdC5jOiBJbiBmdW5jdGlvbiDigJhjc2NoZWRfYWxsb2Nf
cGRhdGHigJk6CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzo1NzY6NDQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc2IHwgY3Nj
aGVkX2FsbG9jX3BkYXRhKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzo1NzY6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc2IHwgY3NjaGVkX2FsbG9j
X3BkYXRhKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgaW50IGNwdSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+
CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYX2NzY2hlZF9jcHVfcGlj
a+KAmToKY29tbW9uL3NjaGVkL2NyZWRpdC5jOjc2OToxOTogd2FybmluZzogb3BlcmFuZCBv
ZiDigJg/OuKAmSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5z
aWduZWQgaW504oCZIGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdz
aWduLWNvbXBhcmVdCiAgNzY5IHwgICAgICAgICAgICAgICAgIDogY3B1bWFza19jeWNsZShz
Y2hlZF91bml0X21hc3Rlcih1bml0KSwgY3B1cyk7CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9u
L3NjaGVkL2NyZWRpdC5jOjc4OTozODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTi
gJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNzg5IHwgICAgICAgICBpZiAo
IHNjaGVkX3VuaXRfbWFzdGVyKHVuaXQpID09IGNwdSAmJiBpc19ydW5xX2lkbGUoY3B1KSAp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNv
bW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVkX2FsbG9jX3VkYXRh
4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0LmM6MTAwODoyOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhvcHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEwMDggfCAgICAgY29u
c3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBzdHJ1Y3Qgc2NoZWRfdW5pdCAqdW5pdCwgdm9p
ZCAqZGQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpjb21tb24v
c2NoZWQvY3JlZGl0LmM6IEluIGZ1bmN0aW9uIOKAmGNzY2hlZF9mcmVlX3VkYXRh4oCZOgpj
b21tb24vc2NoZWQvY3JlZGl0LmM6MTA1NDo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhvcHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEwNTQgfCBjc2NoZWRfZnJlZV91
ZGF0YShjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHZvaWQgKnByaXYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9z
Y2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVkX3VuaXRfc2xlZXDigJk6CmNv
bW1vbi9zY2hlZC9jcmVkaXQuYzoxMDkxOjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTA5MSB8IGNzY2hlZF91bml0X3Ns
ZWVwKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgc3RydWN0IHNjaGVkX3VuaXQgKnVu
aXQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVkX3VuaXRf
d2FrZeKAmToKY29tbW9uL3NjaGVkL2NyZWRpdC5jOjExMTY6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTE2IHwgY3Nj
aGVkX3VuaXRfd2FrZShjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHN0cnVjdCBzY2hl
ZF91bml0ICp1bml0KQogICAgICB8ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3Nj
aGVkX3VuaXRfeWllbGTigJk6CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzoxMTc3OjQzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
MTE3NyB8IGNzY2hlZF91bml0X3lpZWxkKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywg
c3RydWN0IHNjaGVkX3VuaXQgKnVuaXQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVu
Y3Rpb24g4oCYY3NjaGVkX2FmZl9jbnRs4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0LmM6MTIz
MTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhvcHPigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KIDEyMzEgfCBjc2NoZWRfYWZmX2NudGwoY29uc3Qgc3RydWN0IHNjaGVkdWxl
ciAqb3BzLCBzdHJ1Y3Qgc2NoZWRfdW5pdCAqdW5pdCwKICAgICAgfCAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzox
MjMyOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNvZnTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDEyMzIgfCAgICAgICAgICAgICAgICAgY29uc3QgY3B1bWFza190ICpo
YXJkLCBjb25zdCBjcHVtYXNrX3QgKnNvZnQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CmNvbW1vbi9zY2hl
ZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVkX2FsbG9jX2RvbWRhdGHigJk6CmNv
bW1vbi9zY2hlZC9jcmVkaXQuYzoxMzAyOjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTMwMiB8IGNzY2hlZF9hbGxvY19k
b21kYXRhKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgc3RydWN0IGRvbWFpbiAqZG9t
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fgpjb21tb24vc2NoZWQvY3JlZGl0LmM6IEluIGZ1bmN0aW9uIOKAmGNzY2hlZF9mcmVl
X2RvbWRhdGHigJk6CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzoxMzIwOjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTMyMCB8
IGNzY2hlZF9mcmVlX2RvbWRhdGEoY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCB2b2lk
ICpkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVk
X3J1bnFfc29ydOKAmToKY29tbW9uL3NjaGVkL2NyZWRpdC5jOjEzNDM6MjE6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1aW50MzJfdOKAmSB7YWthIOKAmHVuc2lnbmVkIGlu
dOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTM0MyB8ICAgICBpZiAoIHNvcnRfZXBvY2ggPT0g
c3BjLT5ydW5xX3NvcnRfbGFzdCApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICBefgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJmYy5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NzA6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9p
ZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9p
ZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KY29tbW9uL3NjaGVkL2NyZWRpdC5jOiBJbiBm
dW5jdGlvbiDigJhjc2NoZWRfYWNjdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVt
YXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgpjb21tb24v
c2NoZWQvY3JlZGl0LmM6MTUyMjoyOTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVpbnQzMl904oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVd
CiAxNTIyIHwgICAgICAgICAgICAgICAgICAgICAgY3JlZGl0IDwgLWNyZWRpdF9jYXAgJiYK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL2Ny
ZWRpdC5jOjE1MzA6Mjk6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogMTUzMCB8ICAgICAgICAgICAgICAgICBpZiAo
IGNyZWRpdCA8IC1wcnYtPmNyZWRpdHNfcGVyX3RzbGljZSApCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdC5jOjE1NTU6Mjk6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogMTU1NSB8ICAgICAgICAgICAgICAgICBpZiAoIGNyZWRpdCA+IHBydi0+Y3Jl
ZGl0c19wZXJfdHNsaWNlICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5h
dGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Rp
bWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2su
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAg
ICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vbGl2ZXBhdGNoLmM6MTA6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGlu
Z3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50
IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgpjb21tb24vc2NoZWQvY3JlZGl0LmM6IEluIGZ1bmN0aW9uIOKAmGNzY2hl
ZF9sb2FkX2JhbGFuY2XigJk6CmNvbW1vbi9zY2hlZC9jcmVkaXQuYzoxNzQ5OjI4OiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KIDE3NDkgfCAgICAgICAgICAgICBpZiAoIGZpcnN0X2NwdSA+PSBucl9jcHVfaWRz
ICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9t
bS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9t
bS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0
X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1
ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1
bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9m
bHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVt
YXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6IEluIGZ1bmN0
aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToKLi9pbmNsdWRlL3hlbi9kb21h
aW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZh4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlubGluZSB2b2lkIHVubWFwX2Rv
bWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5efgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ndWVzdF9hY2Nlc3MuaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ndWVzdF9hY2Nlc3MuaDoxMCwKICAgICAg
ICAgICAgICAgICBmcm9tIGFyY2gvYXJtL2tlcm5lbC5jOjk6Ci4vaW5jbHVkZS94ZW4vbnVt
YS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEu
aDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlk
KG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUv
eGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxv
Y2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQg
cmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAg
IGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1f
YWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVt
X2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJ
biBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNw
dW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4
NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdl
dF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBw
Mm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZl
bnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBu
cikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCY
cGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1
NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZ
OgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVs
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBh
Z2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFn
ZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNw
dV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3Rh
dGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMx
MDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KYXJjaC9hcm0vbGl2ZXBhdGNoLmM6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfbGl2ZXBhdGNoX3N5bWJvbF9va+KAmToKYXJjaC9hcm0vbGl2ZXBhdGNoLmM6MTEzOjU5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGVsZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDExMyB8IGJvb2wgYXJjaF9saXZlcGF0Y2hfc3ltYm9sX29rKGNvbnN0IHN0cnVj
dCBsaXZlcGF0Y2hfZWxmICplbGYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Jl
YWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGlj
IGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3Rh
dGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlv
biDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMx
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2Rv
bWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KICBD
QyAgICAgIGFyY2gvYXJtL21lbV9hY2Nlc3MubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjY5LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9tZW1fYWNjZXNzLmg6MjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9t
ZW1fYWNjZXNzLmM6ODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkK
ICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVf
ZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19I
Vk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBhcmNo
L2FybS9tbS5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MDoKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1
IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQg
PT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29s
IGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGlj
IGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5o
OjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6Mzc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRl
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5p
dHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NjksCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJv
bSBhcmNoL2FybS9tbS5jOjExOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUu
aDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBp
bnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGFy
Y2gvYXJtL21lbV9hY2Nlc3MuYzoxMDoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0
aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5s
b2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9k
b21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lz
Y3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19I
Vk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FW
RV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+
fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194
KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5j
dGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJj
aF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBw
Mm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJv
bV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEg
fCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3Qg
KnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcg
fCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
cGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcw
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN0
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8
ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAg
ICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19j
eWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
NDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNr
X3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
ZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNo
X3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGlj
eeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBw
Y2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25f
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90
IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
ZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQz
Ml90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4
dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVu
Y3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1
Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBw
YWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwg
ICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dy
aXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRl
KHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmls
dGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICpt
YXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
aXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRh
dGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9j
ZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwg
c3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhz
dHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vZG9t
YWluX3BhZ2UuaDogSW4gZnVuY3Rpb24g4oCYdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFs4oCZ
OgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6NzE6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYdmHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNzEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFsKGNvbnN0IHZvaWQgKnZhKSB7
fTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fl5+CiAgQ0MgICAgICBhcmNoL2FybS9tb25pdG9yLm8KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDo3LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9ncmFudF90YWJsZS5oOjMxLAogICAgICAgICAgICAg
ICAgIGZyb20gYXJjaC9hcm0vbW0uYzoxMjoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1
bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZu
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
dW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAg
fCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKA
mGlzX3B2XzMyYml0X2RvbWFpbuKAmToKSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJv
b2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9z
dCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwg
c3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEz
MTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9k
b21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS92
bV9ldmVudC5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3ZtX2V2
ZW50Lmg6MjgsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9tZW1fYWNjZXNzLmM6
MTE6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2
bV9ldmVudF9pbml0X2RvbWFpbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjE0OjU1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTQgfCBz
dGF0aWMgaW5saW5lIGludCB2bV9ldmVudF9pbml0X2RvbWFpbihzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0
aW9uIOKAmHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVw4oCZOgouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvdm1fZXZlbnQuaDoyNTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI1IHwgc3RhdGljIGlubGluZSB2b2lkIHZt
X2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVwKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5o
OjI1Ojc4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgdm1fZXZlbnRfdG9nZ2xlX3Np
bmdsZXN0ZXAoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2ln
bmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoyNjo2ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMjYgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfcmVnaXN0ZXJfd3Jp
dGVfcmVzdW1l4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDozMjo1MDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDMyIHwgdm9pZCB2bV9ldmVudF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUoc3RydWN0IHZj
cHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvdm1fZXZlbnQuaDozMjo3NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
c3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzIgfCB2b2lkIHZtX2V2ZW50X3JlZ2lz
dGVyX3dyaXRlX3Jlc3VtZShzdHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAq
cnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5l
cmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X2VtdWxhdGVfY2hlY2vi
gJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjM4OjQyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzggfCB2
b2lkIHZtX2V2ZW50X2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jl
c3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6Mzg6NjY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDM4IHwgdm9pZCB2bV9ldmVudF9lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB2
bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3N5
bmNfZXZlbnTigJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjQ0OjM5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNDQgfCB2b2lkIHZtX2V2ZW50X3N5bmNfZXZlbnQoc3RydWN0IHZjcHUgKnYsIGJvb2wg
dmFsdWUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjQ0OjQ3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHZhbHVl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQ0IHwg
dm9pZCB2bV9ldmVudF9zeW5jX2V2ZW50KHN0cnVjdCB2Y3B1ICp2LCBib29sIHZhbHVlKQou
L2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZl
bnRfcmVzZXRfdm10cmFjZeKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
NTA6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1MCB8IHZvaWQgdm1fZXZlbnRfcmVzZXRfdm10cmFjZShzdHJ1Y3QgdmNw
dSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL21lbV9hY2Nlc3MuYzoxMzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFf
aXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2ln
bmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL21lbV9hY2Nlc3MuYzogSW4gZnVu
Y3Rpb24g4oCYcDJtX3NldF9tZW1fYWNjZXNz4oCZOgphcmNoL2FybS9tZW1fYWNjZXNzLmM6
MzQ1OjM4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGFsdHAybV9pZHjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzNDUgfCAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgYWx0cDJtX2lkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fn5+fn5+fgphcmNoL2FybS9tZW1fYWNjZXNzLmM6IEluIGZ1bmN0
aW9uIOKAmHAybV9zZXRfbWVtX2FjY2Vzc19tdWx0aeKAmToKYXJjaC9hcm0vbWVtX2FjY2Vz
cy5jOjQyMzo0Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDIzIHwgbG9uZyBwMm1fc2V0X21lbV9hY2Nlc3NfbXVsdGkoc3Ry
dWN0IGRvbWFpbiAqZCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL21lbV9hY2Nlc3MuYzo0MjQ6Njg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGZuX2xpc3TigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBYRU5fR1VFU1Rf
SEFORExFKGNvbnN0X3VpbnQ2NCkgcGZuX2xpc3QsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn5+fn5+CmFyY2gvYXJtL21lbV9hY2Nlc3MuYzo0MjU6Njc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYYWNjZXNzX2xpc3TigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBYRU5fR1VFU1RfSEFORExF
KGNvbnN0X3VpbnQ4KSBhY2Nlc3NfbGlzdCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fn5+
fn5+fgphcmNoL2FybS9tZW1fYWNjZXNzLmM6NDI2OjQwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI2IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgbnIsIHVpbnQzMl90IHN0YXJ0LCB1aW50
MzJfdCBtYXNrLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+CmFyY2gvYXJtL21lbV9hY2Nlc3MuYzo0MjY6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc3RhcnTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjYgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50MzJfdCBuciwgdWludDMyX3Qgc3RhcnQs
IHVpbnQzMl90IG1hc2ssCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+CmFyY2gvYXJtL21lbV9hY2Nlc3MuYzo0MjY6
Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQyNiB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90
IG5yLCB1aW50MzJfdCBzdGFydCwgdWludDMyX3QgbWFzaywKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn4KYXJjaC9hcm0vbWVtX2FjY2Vzcy5jOjQyNzo0NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhhbHRwMm1faWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDI3
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGFsdHAybV9p
ZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9j
ayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlu
bGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8
IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3Jl
YWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNp
X3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2
cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3Qg
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBk
YXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAw
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJv
Y2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNy
ZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6
NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi92bV9ldmVudC5oOjI2LAogICAgICAgICAg
ICAgICAgIGZyb20gYXJjaC9hcm0vbW9uaXRvci5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwg
c3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9k
b21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0
aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9ncmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0YWJfbWFya19kaXJ0eeKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjIwOjUzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjAgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX21hcmtfZGlydHkoc3RydWN0IGRvbWFpbiAqZCwg
bWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUu
aDoyMDo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX21hcmtfZGly
dHkoc3RydWN0IGRvbWFpbiAqZCwgbWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOiBJbiBmdW5jdGlvbiDigJhnbnR0
YWJfaG9zdF9tYXBwaW5nX2dldF9wYWdlX3R5cGXigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZ3JhbnRfdGFibGUuaDoyNzo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
b+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBn
bnR0YWJfaG9zdF9tYXBwaW5nX2dldF9wYWdlX3R5cGUoYm9vbCBybywKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOjI4Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGxk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI4IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpsZCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyOTo3NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyOSB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IGRvbWFpbiAqcmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6IEluIGZ1bmN0aW9uIOKAmGdudHRhYl9yZWxl
YXNlX2hvc3RfbWFwcGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFi
bGUuaDozNDo3MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDM0IHwgc3RhdGljIGlubGluZSBib29sIGdudHRhYl9yZWxlYXNl
X2hvc3RfbWFwcGluZ3MoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9tbS5jOjE2OgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fZ2V0ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjI5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hl
ZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0Njox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBkb21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
b21jdGwoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fc3lzY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNv
bnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBp
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
ZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNo
biAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRj
aG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMwMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9ncmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzA5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9ncmFudF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzIxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2dyYW50X3F1ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMzMyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9pbml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3
MTo1Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQo
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3NldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVt
b3J5X2V4Y2hhbmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEg
fCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWVtb3J5X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzkz
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQw
NjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3Bo
eXNtYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDog
SW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2Zp
bGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9r
ZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZF
X1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+
fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDYwIHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpk
YXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
dW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91
bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEs
IHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQg
cGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwg
dWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBh
bGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3Rh
cnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTM0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3Np
Z25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAq
ZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9y
ZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9i
ZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRm
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3Vy
Y2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fcmVzb3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV9zZXR1cF9wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTc0OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1
cF9nc2koeHNtX2RlZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vf
b2ZmbGluZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3Rh
dGljIGlubGluZSBpbnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWlu
dDMyX3QgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2h5cGZzX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkg
fCBzdGF0aWMgaW5saW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2h2bV9wYXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEy
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDYxMiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJt
aHZtKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQz
Ml90IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MjM6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92
bnVtYWluZm8oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRy
b2zigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDYzNSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo2NTU6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29w
KHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjczODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3
NDM6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
b21haW5fcmVzb3VyY2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL21tLmM6IEluIGZ1bmN0aW9uIOKAmHN0ZWFsX3Bh
Z2XigJk6CmFyY2gvYXJtL21tLmM6ODY6MjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA4NiB8ICAgICBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCB1bnNpZ25lZCBpbnQgbWVtZmxhZ3MpCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vbW0uYzo4Njo0MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDg2IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHVu
c2lnbmVkIGludCBtZW1mbGFncykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9tbS5jOjg2OjYwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1lbWZsYWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDg2
IHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHVuc2ln
bmVkIGludCBtZW1mbGFncykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+CmFyY2gvYXJtL21tLmM6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfaXNfcmFtX3R5cGXigJk6CmFyY2gvYXJtL21tLmM6OTE6
MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDkxIHwgaW50IHBhZ2VfaXNfcmFtX3R5cGUodW5zaWduZWQgbG9uZyBtZm4s
IHVuc2lnbmVkIGxvbmcgbWVtX3R5cGUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vbW0uYzo5MTo1NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtZW1fdHlwZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA5MSB8
IGludCBwYWdlX2lzX3JhbV90eXBlKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG1lbV90eXBlKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fn5+fn5+CmFyY2gvYXJtL21tLmM6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfbWVtb3J5X29w4oCZOgphcmNoL2FybS9tbS5jOjI3Mzo1ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhhcmfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzMg
fCBsb25nIGFyY2hfbWVtb3J5X29wKGludCBvcCwgWEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh2
b2lkKSBhcmcpCmFyY2gvYXJtL21tLmM6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX3R5cGXi
gJk6CmFyY2gvYXJtL21tLmM6MzcyOjM3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBhZ2XigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzIgfCBpbnQgZ2V0X3BhZ2VfdHlw
ZShzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCB1bnNpZ25lZCBsb25nIHR5cGUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9tbS5j
OjM3Mjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh0eXBl4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzcyIHwgaW50IGdldF9wYWdlX3R5cGUoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgdW5zaWduZWQgbG9uZyB0eXBlKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS9t
bS5jOiBJbiBmdW5jdGlvbiDigJhwdXRfcGFnZV90eXBl4oCZOgphcmNoL2FybS9tbS5jOjM3
NzozODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdl4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzc3IHwgdm9pZCBwdXRfcGFnZV90eXBlKHN0cnVjdCBwYWdlX2luZm8g
KnBhZ2UpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6
NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNz
aW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25l
ZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBu
cl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBu
cl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxs
b2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4u
aDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KA
mToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGlj
IGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vz
cy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rp
b24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBi
b29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9h
bHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90
IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1f
YWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9n
Zm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAg
ICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQs
IHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFy
Y2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVj
dCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNz
aWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1
aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dy
aXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2No
ZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8z
MmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEg
fCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNtL3ZtX2V2ZW50Lmg6MSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdm1fZXZlbnQuaDoyODoKLi9p
bmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50
X2luaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoxNDo1
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDE0IHwgc3RhdGljIGlubGluZSBpbnQgdm1fZXZlbnRfaW5pdF9kb21haW4oc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVu
dC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF90b2dnbGVfc2luZ2xlc3RlcOKAmToKLi9p
bmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MjU6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2bV9ldmVudF90b2dnbGVfc2luZ2xlc3RlcChzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVy
aWMvdm1fZXZlbnQuaDoyNTo3ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI1IHwgc3RhdGljIGlubGluZSB2b2lkIHZtX2V2
ZW50X3RvZ2dsZV9zaW5nbGVzdGVwKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVy
aWMvdm1fZXZlbnQuaDoyNjo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3Di
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjYgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3JlZ2lzdGVyX3dyaXRlX3Jlc3VtZeKAmToKLi9p
bmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6MzI6NTA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMiB8IHZvaWQgdm1f
ZXZlbnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1lKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9y
ZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6
MzI6NzQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDMyIHwgdm9pZCB2bV9ldmVudF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUo
c3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOiBJ
biBmdW5jdGlvbiDigJh2bV9ldmVudF9lbXVsYXRlX2NoZWNr4oCZOgouL2luY2x1ZGUvYXNt
LWdlbmVyaWMvdm1fZXZlbnQuaDozODo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM4IHwgdm9pZCB2bV9ldmVudF9lbXVs
YXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9yZXNwb25zZV90ICpyc3ApCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5j
bHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjM4OjY2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzOCB8IHZvaWQgdm1f
ZXZlbnRfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2Vf
dCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92
bV9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJh2bV9ldmVudF9zeW5jX2V2ZW504oCZOgouL2lu
Y2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDo0NDozOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQ0IHwgdm9pZCB2bV9l
dmVudF9zeW5jX2V2ZW50KHN0cnVjdCB2Y3B1ICp2LCBib29sIHZhbHVlKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdl
bmVyaWMvdm1fZXZlbnQuaDo0NDo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
YWx1ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA0NCB8IHZvaWQgdm1fZXZlbnRfc3lu
Y19ldmVudChzdHJ1Y3QgdmNwdSAqdiwgYm9vbCB2YWx1ZSkKLi9pbmNsdWRlL2FzbS1nZW5l
cmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3Jlc2V0X3ZtdHJhY2Xi
gJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjUwOjQyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCB2
b2lkIHZtX2V2ZW50X3Jlc2V0X3ZtdHJhY2Uoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21vbml0b3IuaDozMCwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL21vbml0b3IuYzoxMjoKLi9pbmNsdWRlL2FzbS1nZW5l
cmljL21vbml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2FsbG93X3VzZXJz
cGFjZeKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDoyMjo1MDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIy
IHwgdm9pZCBhcmNoX21vbml0b3JfYWxsb3dfdXNlcnNwYWNlKHN0cnVjdCBkb21haW4gKmQs
IGJvb2wgYWxsb3dfdXNlcnNwYWNlKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy9tb25p
dG9yLmg6MjI6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYWxsb3dfdXNlcnNw
YWNl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIyIHwgdm9pZCBhcmNoX21vbml0b3Jf
YWxsb3dfdXNlcnNwYWNlKHN0cnVjdCBkb21haW4gKmQsIGJvb2wgYWxsb3dfdXNlcnNwYWNl
KQouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X21vbml0b3JfZG9tY3RsX29w4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5o
OjI3OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjcgfCBpbnQgYXJjaF9tb25pdG9yX2RvbWN0bF9vcChzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9tb25pdG9yX29wICptb3ApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNt
LWdlbmVyaWMvbW9uaXRvci5oOjI3Ojc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1vcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNyB8IGludCBhcmNoX21vbml0b3Jf
ZG9tY3RsX29wKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX21vbml0b3Jf
b3AgKm1vcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2Fz
bS1nZW5lcmljL21vbml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2luaXRf
ZG9tYWlu4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOjQzOjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NDMgfCBpbnQgYXJjaF9tb25pdG9yX2luaXRfZG9tYWluKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
aW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbW9u
aXRvcl9jbGVhbnVwX2RvbWFpbuKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3Iu
aDo1MDo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDUwIHwgdm9pZCBhcmNoX21vbml0b3JfY2xlYW51cF9kb21haW4oc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW9uaXRvci5oOiBJ
biBmdW5jdGlvbiDigJhhcmNoX21vbml0b3JfZ2V0X2NhcGFiaWxpdGllc+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9tb25pdG9yLmg6MzU6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNSB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgYXJjaF9tb25pdG9yX2dldF9jYXBhYmlsaXRpZXMoc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGFyY2gvYXJtL3AybS5v
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0
aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGlt
ZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5o
OjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vaW9jYXAuaDoxMCwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL3AybS5jOjI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0
aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5y
X2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNp
Z25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5y
X2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxv
Y19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hl
bi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90
b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5o
OjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKA
mGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5z
X2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcy
OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90
aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZ
OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMg
aW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNz
Lmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlv
biDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vz
c19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
c3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJv
b2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2Fs
dHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9h
bHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dm
buKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAg
ICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwg
cDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJj
aF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xp
Y3nigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0
IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6MzEwOjc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3Np
Z25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNi
ZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQz
Ml90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3Jp
dGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlu
bGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVn
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
LCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUs
IHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hl
Y2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMy
Yml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8
IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gYXJjaC9hcm0vcDJtLmM6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2V2ZW50Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
dmlyceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX3ZpcnFfaXNfZ2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CmFy
Y2gvYXJtL3AybS5jOiBJbiBmdW5jdGlvbiDigJhtZW1vcnlfdHlwZV9jaGFuZ2Vk4oCZOgph
cmNoL2FybS9wMm0uYzozNzo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM3IHwgdm9pZCBtZW1vcnlfdHlwZV9jaGFuZ2Vk
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vcDJtLmM6IEluIGZ1bmN0aW9uIOKAmGd1ZXN0X3Bo
eXNtYXBfbWFya19wb3B1bGF0ZV9vbl9kZW1hbmTigJk6CmFyY2gvYXJtL3AybS5jOjg2OjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgODYgfCBpbnQgZ3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZChz
dHJ1Y3QgZG9tYWluICpkLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vcDJtLmM6ODc6NTc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ2Zu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDg3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgbG9uZyBnZm4sCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vcDJtLmM6ODg6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3JkZXLigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgb3JkZXIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+CmFyY2gvYXJtL3AybS5jOiBJbiBm
dW5jdGlvbiDigJhwMm1fcG9kX2RlY3JlYXNlX3Jlc2VydmF0aW9u4oCZOgphcmNoL2FybS9w
Mm0uYzo5Mzo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDkzIHwgdW5zaWduZWQgbG9uZyBwMm1fcG9kX2RlY3JlYXNlX3Jl
c2VydmF0aW9uKHN0cnVjdCBkb21haW4gKmQsIGdmbl90IGdmbiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xgph
cmNoL2FybS9wMm0uYzo5Mzo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnZm7i
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgOTMgfCB1bnNpZ25lZCBsb25nIHAybV9wb2Rf
ZGVjcmVhc2VfcmVzZXJ2YXRpb24oc3RydWN0IGRvbWFpbiAqZCwgZ2ZuX3QgZ2ZuLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5efn4KYXJjaC9hcm0vcDJtLmM6OTQ6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYb3JkZXLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgOTQg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IG9yZGVyKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn4KYXJjaC9hcm0vcDJtLmM6IEluIGZ1bmN0aW9u
IOKAmHAybV9hbGxvY192bWlk4oCZOgphcmNoL2FybS9wMm0uYzoyODM6MTM6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDI4MyB8ICAgICBpZiAoIG5yID09IE1BWF9WTUlEICkKICAgICAgfCAgICAgICAgICAg
ICBefgogIENDICAgICAgYXJjaC9hcm0vcGVyY3B1Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9w
ZXJjcHUuaDoxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUu
aDozMCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3BlcmNwdS5jOjI6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgogIENDICAgICAgYXJjaC9hcm0vcGxhdGZvcm0ubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL2NwdS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSBh
cmNoL2FybS9wZXJjcHUuYzozOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1
bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wbGF0Zm9ybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBh
cmNoL2FybS9wbGF0Zm9ybS5jOjExOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjcwLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vcGVy
Y3B1LmM6NToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAg
ICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAg
ICAgICBefgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwu
aDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6
MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FW
RV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFU
XyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxs
b2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tf
dmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjM2NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192
YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVu
c2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
cm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZu
c19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1
Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8
IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6
NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKA
mHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVy
KGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1h
c2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+
fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL3BlcmNwdS5jOjY6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlu
bGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3Vu
bG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwg
c3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxv
Y2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vcGVyY3B1LmM6IEluIGZ1bmN0aW9uIOKAmGluaXRf
cGVyY3B1X2FyZWHigJk6CmFyY2gvYXJtL3BlcmNwdS5jOjIyOjMyOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGxvbmcgdW5zaWduZWQgaW504oCZIGFuZCDigJhsb25nIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgIDIyIHwgICAgIGlmICggX19wZXJfY3B1X29mZnNldFtjcHVdICE9IElOVkFM
SURfUEVSQ1BVX0FSRUEgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBefgphcmNoL2FybS9wZXJjcHUuYzogSW4gZnVuY3Rpb24g4oCYY3B1X3BlcmNwdV9jYWxs
YmFja+KAmToKYXJjaC9hcm0vcGVyY3B1LmM6NTQ6Mjg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbmZi4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU0IHwgICAgIHN0cnVj
dCBub3RpZmllcl9ibG9jayAqbmZiLCB1bnNpZ25lZCBsb25nIGFjdGlvbiwgdm9pZCAqaGNw
dSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vcGVy
Y3B1LmM6IEF0IHRvcCBsZXZlbDoKYXJjaC9hcm0vcGVyY3B1LmM6ODU6MTogd2FybmluZzog
4oCYc3RhdGlj4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcgb2YgZGVjbGFyYXRpb24gWy1Xb2xk
LXN0eWxlLWRlY2xhcmF0aW9uXQogICA4NSB8IHByZXNtcF9pbml0Y2FsbChwZXJjcHVfcHJl
c21wX2luaXQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rp
b24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVj
dG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBh
Z2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAg
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lk
IHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0
YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6
NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0
IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEw
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj
dCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2Rldmlj
ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMg
aW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Rl
YXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBj
aV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBj
aV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93
cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0
ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVu
ZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94
ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxp
bmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5eCiAgQ0MgICAgICBhcmNoL2FybS9wbGF0Zm9ybV9oeXBlcmNh
bGwubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRl
cm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYs
CiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9wbGF0Zm9ybV9oeXBlcmNhbGwuYzox
MToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5n
cyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5j
dGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2
OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vcGh5c2Rldi5v
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CiAgQ0MgICAgICBhcmNoL2FybS9wcm9jZXNzb3IubwpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1h
cOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVk
IGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25l
ZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVj
dCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2Vf
bGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJt
L3BoeXNkZXYuYzoxMToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IElu
IGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNw
dV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZv
ICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAg
ICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9s
b2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3Vw
ZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1
YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
RU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAg
ICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpo
LCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNs
dWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9t
ZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9h
Y3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNv
dXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlk
eOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBk
b21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlf
dCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGlu
ZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cHJvY2luZm8uaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3Byb2Nlc3Nv
ci5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rp
b24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3Np
YmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFz
a19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
IDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAg
ICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1
OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBm
dW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDi
gJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5
IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9j
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1
cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vl
c3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1v
cnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3Rt
YXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25l
ZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWdu
ZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDox
ODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAg
ICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoy
NDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAg
ICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBz
dGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0
X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9s
aXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3Ry
dWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IElu
IGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxi
Zmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXAp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3Vu
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoz
MTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3Vy
Y2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
NzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1f
Y2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1
ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5z
aWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3Zt
X2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5l
IGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWdu
X2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYg
KnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1h
Lmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQo
bWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhy
ZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQg
dnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90
IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3Qg
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMw
MDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3By
b2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGly
ZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5z
aWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVu
c2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNs
dWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwg
cGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBh
Z2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAg
ICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxi
Lmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1l
c3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2Zp
bHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1
bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJi
aXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hf
dmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwg
c3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9v
bCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0
ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBz
dGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNw
dSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
ODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhn
ZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6
Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwg
cDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2
ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Jl
YWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGlj
IGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3Rh
dGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25l
ZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlv
biDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMx
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2Rv
bWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAg
ICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9wbGF0Zm9ybV9oeXBlcmNhbGwuYzoxMzoKLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2dldGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDIyOSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2No
ZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90
IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
ZG9tY3RsKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX3N5c2N0bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI2MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fcmVhZGNvbnNvbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFy
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3
IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKA
mHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9n
dWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21l
bW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRf
dCBpZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fZXZ0Y2huX3NlbmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2
dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyOTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9l
dnRjaG5fcmVzZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBm
cm9tIGFyY2gvYXJtL3BoeXNkZXYuYzoxMjoKLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI0
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFpbmluZm/igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9k
b21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIzNToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBj
bWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lz
Y3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3Rhcmdl
dOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICplKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1MTo0NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTY6NDQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21fZGVmYXVsdF90
IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjYx
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZhdWx0X3QgZGVm
LCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2Nzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0
Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3MzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2
dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dy
YW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAzIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdW5tYXByZWbi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJhbnNmZXLigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM2
Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9oYXJkd2FyZV9k
b21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9kX3Rh
cmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2OjUy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3RhcmdldCh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1v
cnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4NzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2VydmF0
aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9fcGh5
c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9n
bWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFp
biAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFn
ZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
Y29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNt
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmls
ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
a2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91
bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhl
bl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRf
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2Jp
bmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5
NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2
NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lf
Y29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMy
X3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWdu
X2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0
cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9y
ZXNvdXJjZV9wbHVnX3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9i
ZGYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZXNvdXJjZV91bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA1NTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRm
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9y
ZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBj
aV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fcmVzb3VyY2VfcGx1Z19jb3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3Vu
cGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhz
bV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9wY2nigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVu
ZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2
Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5l
X2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9z
ZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9w
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9w
YXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYx
OCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qg
bW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1f
ZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwg
aW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nl
c3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
bV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVu
X3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS9waHlz
ZGV2LmM6IEluIGZ1bmN0aW9uIOKAmGRvX2FybV9waHlzZGV2X29w4oCZOgphcmNoL2FybS9w
aHlzZGV2LmM6MTU6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYXJn4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDE1IHwgaW50IGRvX2FybV9waHlzZGV2X29wKGludCBj
bWQsIFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0odm9pZCkgYXJnKQouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRf
dHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNf
cHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAx
MTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29u
c3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94
c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAq
ZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRf
aGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM2Njo1ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0
X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNo
YW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jl
c2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQyODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21fY29uc29sZV9pbyh4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFp
bl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2
MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5f
cGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDY1IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYmluZF9wdF9p
cnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmlu
ZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRfaXJx4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRfaXJxICpi
aW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9p
b21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5NToxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90
IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pb21lbV9t
YXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDEgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IHMsIHVpbnQ2
NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lfY29uZmlnX3Blcm1p
c3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwNyB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3QgbWFjaGluZV9i
ZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWduX2R0ZGV2aWNl4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0
MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdf
cGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDcgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1bHRfdCBkZWYsIHVp
bnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50
MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3BsdWdfY29yZeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJj
ZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6NTg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTYz
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19jb3JlKHhzbV9kZWZh
dWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjU2OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1h
Y2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfZ3Np4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0aWMgaW5saW5lIGludCB4
c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgZ3NpKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291
cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5OjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3
OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9taXNjKHhzbV9kZWZh
dWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg0OjUw
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdlX29mZmxpbmUoeHNtX2Rl
ZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1
ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h5cGZzX29wKHhzbV9k
ZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYwNzoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MDcg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
aHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTg6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50
NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldF92bnVtYWluZm/i
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV92bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyOToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBt
b2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWVt
X2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZvcm1fb3DigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kbV9vcOKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGljIGlubGluZSBpbnQg
eHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNpb27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhz
bV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qgb3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21haW5fcmVzb3VyY2Vf
bWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
YXJjaC9hcm0vcGxhdGZvcm1faHlwZXJjYWxsLmM6MTg6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2dsb2JhbOKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBpbnQgdmlycSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgogIENDICAgICAgYXJjaC9hcm0vcHNjaS5vCiAgQ0MgICAgICBhcmNoL2FybS9z
ZXR1cC5vCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fs
dGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9y
bS9hY2xpbnV4Lmg6NTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9w
bGF0Zm9ybS9hY2Vudi5oOjE0MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9h
Y3BpL2FjcGkuaDo1NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vYWNw
aS5oOjU3LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vcHNjaS5jOjEyOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25y
X3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2ln
bmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKA
mF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVf
RklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+Xgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRf
IyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAgICAgICAg
ICAgICBmcm9tIGFyY2gvYXJtL3NldHVwLmM6MTI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
Y3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0
aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2OgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHBy
ZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNp
Z25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUg
PCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9
PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMg
aW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NzoKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0u
aDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGly
ZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0
aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0
IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAg
ICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGlj
IGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90
IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFt
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0
bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFt
cCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3Vf
cmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5j
bHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxh
dGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0u
aDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6MTIsCiAg
ICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zZXR1cC5jOjEzOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNp
Z25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5z
aWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVz
b3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRw
Mm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwg
dW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2RvbWFp
bl9wYWdlLmg6IEluIGZ1bmN0aW9uIOKAmHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbOKAmToK
Li9pbmNsdWRlL3hlbi9kb21haW5fcGFnZS5oOjcxOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHZh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDcxIHwgc3RhdGljIGlu
bGluZSB2b2lkIHVubWFwX2RvbWFpbl9wYWdlX2dsb2JhbChjb25zdCB2b2lkICp2YSkge307
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5efgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21h
aW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90
YWJsZS5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2dyYW50X3Rh
YmxlLmg6MzEsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zZXR1cC5jOjE0Ogou
L2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9p
bmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9k
ZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Rh
dGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
OTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVk
IGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVk
ZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3RhdGVfbGltaXTi
gJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBu
ZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0
YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3Rf
bWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlf
cG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNp
X2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsg
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRp
bmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlu
bGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNw
dSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4g
ZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8z
MmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDogSW4gZnVuY3Rpb24g
4oCYZ250dGFiX21hcmtfZGlydHnigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRf
dGFibGUuaDoyMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlubGluZSB2b2lkIGdudHRhYl9tYXJr
X2RpcnR5KHN0cnVjdCBkb21haW4gKmQsIG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MjA6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIwIHwgc3RhdGljIGlu
bGluZSB2b2lkIGdudHRhYl9tYXJrX2RpcnR5KHN0cnVjdCBkb21haW4gKmQsIG1mbl90IG1m
bikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFi
bGUuaDogSW4gZnVuY3Rpb24g4oCYZ250dGFiX2hvc3RfbWFwcGluZ19nZXRfcGFnZV90eXBl
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6Mjc6NTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcm/igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjcgfCBzdGF0aWMgaW5saW5lIGJvb2wgZ250dGFiX2hvc3RfbWFwcGluZ19nZXRfcGFnZV90
eXBlKGJvb2wgcm8sCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ3JhbnRfdGFibGUuaDoyODo3
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyOCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IGRvbWFpbiAqbGQsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6Mjk6
NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmTigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjkgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKnJkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ncmFudF90YWJsZS5oOiBJ
biBmdW5jdGlvbiDigJhnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2dyYW50X3RhYmxlLmg6MzQ6NzA6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBnbnR0YWJfcmVsZWFzZV9ob3N0X21hcHBpbmdzKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gYXJjaC9hcm0vc2V0dXAuYzoxNzoKLi9pbmNsdWRlL3hlbi9zZXJpYWwuaDogQXQgdG9w
IGxldmVsOgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2NjoxOiB3YXJuaW5nOiDigJhpbmxp
bmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUt
ZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5saW5lIHhoY2lfZGJjX3VhcnRf
aW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJj
aC9hcm0vc2V0dXAuYzoyNDoKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBmdW5jdGlvbiDi
gJhub19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDoxOTM6MTk6IHdh
cm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCY
bG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNpZ25lZCBpbnTi
gJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBh
cmVdCiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0gcyk7IGUgLSBz
OyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBhcmNoL2FybS9zaHV0ZG93bi5vCkluIGZp
bGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9zZXR1cC5jOjMzOgouL2luY2x1ZGUveGVuL2Fj
cGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRfY3N0YXRlX2xpbWl04oCZOgouL2luY2x1
ZGUveGVuL2FjcGkuaDoxOTI6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3
X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTkyIHwgc3RhdGljIGlubGluZSB2
b2lkIGFjcGlfc2V0X2NzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJl
dHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgouL2luY2x1ZGUveGVuL2FjcGkuaDogSW4gZnVu
Y3Rpb24g4oCYYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl04oCZOgouL2luY2x1ZGUveGVuL2Fj
cGkuaDoxOTQ6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk0IHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlf
c2V0X2NzdWJzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQgbmV3X2xpbWl0KSB7IHJldHVybjsg
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9oeXBlcmNhbGwuaDoyMiwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3Nl
dHVwLmM6MzU6Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21h
aW5fY3JlYXRl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIyNDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyMjQgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IHNzaWRy
ZWYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRkb21haW5pbmZv4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjIyOTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyMjkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0ZG9tYWlu
aW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3RsX3NjaGVkdWxlcl9v
cOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMzU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjM1IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fc3lzY3RsX3NjaGVkdWxlcl9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDA6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjQwIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3Ao
eHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF90YXJnZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjQ2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0NiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFpbiAqZSkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWN0
bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTE6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjUxIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX2RvbWN0bCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9z
eXNjdGzigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjU2OjQ0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI1NiB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9zeXNjdGwoeHNtX2RlZmF1bHRfdCBkZWYsIGludCBjbWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVhZGNvbnNvbGXigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjYxOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2MSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9yZWFkY29uc29sZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY2xlYXIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5f
dW5ib3VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjY3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAq
Y2huLCBkb21pZF90IGlkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9pbnRlcmRvbWFpbuKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDoyNzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjczIHwgICAgIHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGV2dGNobiAqY2hhbjEsCiAg
ICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9ldnRjaG5fc2VuZOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyODU6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjg1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fc3RhdHVz4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjI5MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTEgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl9yZXNldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyOTc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwg
c3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9tYXByZWbigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6MzAzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyLCB1aW50MzJfdCBm
bGFncykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3VubWFwcmVm4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjMwOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50
X3NldHVw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMxNToxOTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTUgfCAgICAg
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpk
MikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3RyYW5zZmVy4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMyMToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMjEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X2Nv
cHnigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzI3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyNyB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZ3JhbnRfcXVlcnlfc2l6ZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozMzM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzMzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pbml0X2hhcmR3
YXJlX2RvbWFpbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNjY6NTg6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY2IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2luaXRfaGFyZHdhcmVfZG9tYWluKHhzbV9kZWZhdWx0
X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzcxOjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9n
ZXRfcG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NldF9wb2Rf
dGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3Njo1Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNzYgfCBzdGF0
aWMgaW5saW5lIGludCB4c21fc2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9tZW1vcnlfZXhjaGFuZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzgx
OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM4MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1vcnlfZXhjaGFuZ2Uo
eHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2FkanVzdF9yZXNlcnZh
dGlvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODc6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzg3IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tZW1vcnlfc3RhdF9yZXNlcnZhdGlvbuKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDozOTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzkzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9t
ZW1vcnlfcGluX3BhZ2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzk5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM5
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYWRkX3RvX3BoeXNtYXDigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDA2OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQwNiB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
cmVtb3ZlX2Zyb21fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTI6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDEyIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tYXBfZ21mbl9mb3JlaWdu4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTggfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkb21haW4gKnQpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9jbGFpbV9wYWdlc+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MjM6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NsYWltX3BhZ2VzKHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fY29uc29sZV9pb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mjg6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDI4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2NvbnNvbGVfaW8oeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Byb2ZpbGXigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NDMzOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzMyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wcm9m
aWxlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgb3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2tleGVj4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQzODo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
a2V4ZWMoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9zY2hlZG9wX3NodXRkb3du4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ0
NDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NDQgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5fcGly
ceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NTQ6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU0IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX21hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9tYXBfZG9tYWluX2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjA6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDYwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBp
bnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX3BpcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDY1OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2NSB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV91bm1hcF9kb21haW5fcGlycSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3VubWFwX2RvbWFpbl9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDcx
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2JpbmRfcHRfaXJx4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2JpbmRfcHRf
aXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo0ODM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDgzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2lycV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ4OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0ODkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBwaXJxLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1f
cGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0OTU6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDk1IHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1
aW50NjRfdCBlLCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faW9tZW1fbWFwcGlu
Z+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTAxIHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50NjRfdCBzLCB1aW50NjRfdCBl
LCB1aW50OF90IGFsbG93KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGNpX2NvbmZpZ19wZXJtaXNzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MDcgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQzMl90IG1hY2hpbmVfYmRmLCB1
aW50MTZfdCBzdGFydCwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2Fzc2lnbl9kdGRldmljZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDA6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBj
b25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX3BjaeKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJf
dCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
bWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9wbHVnX2NvcmXigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5w
bHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2MyB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZSh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1Njk6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5l
X2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dzaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50IGdzaSkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9z
ZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzkgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlzYyh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4NDo1MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZsaW5lKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTg5OjQ2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBmc19vcCh4c21fZGVmYXVs
dF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjA3IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIG9w
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h2bV9w
YXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2
bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjE4OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYx
OCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qg
bW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfdm51bWFpbmZv4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdm1f
ZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2Mjk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjI5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgbW9kZSwg
aW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX21lbV9hY2Nl
c3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3JtX29w4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
cGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1fb3DigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9k
bV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5saW5lIGludCB4c21feGVu
X3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX3Jlc291cmNlX21hcOKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4IHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KYXJjaC9hcm0vc2V0dXAuYzogSW4gZnVuY3Rp
b24g4oCYYWRkX2Jvb3RfbW9kdWxl4oCZOgphcmNoL2FybS9zZXR1cC5jOjM4MzoyMTogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTigJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMzgzIHwgICAgIGZvciAoIGkgPSAwIDsgaSA8IG1vZHMtPm5yX21vZHMgOyBp
KysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS9zZXR1cC5jOiBJ
biBmdW5jdGlvbiDigJhib290X2NtZGxpbmVfZmluZF9ieV9raW5k4oCZOgphcmNoL2FybS9z
ZXR1cC5jOjQ1ODoyMTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVk
IGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNDU4IHwgICAgIGZvciAoIGkgPSAwIDsgaSA8
IGNtZHMtPm5yX21vZHMgOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgXgph
cmNoL2FybS9zZXR1cC5jOiBJbiBmdW5jdGlvbiDigJhib290X21vZHVsZV9maW5kX2J5X2Fk
ZHJfYW5kX2tpbmTigJk6CmFyY2gvYXJtL3NldHVwLmM6NDg5OjIwOiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA0
ODkgfCAgICAgZm9yIChpID0gMCA7IGkgPCBtb2RzLT5ucl9tb2RzIDsgaSsrICkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS9zZXR1cC5jOiBJbiBmdW5jdGlvbiDi
gJhpbml0X3BkeOKAmToKYXJjaC9hcm0vc2V0dXAuYzo2MDA6Mjc6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDYw
MCB8ICAgICBmb3IgKCBiYW5rID0gMCA7IGJhbmsgPCBtZW0tPm5yX2JhbmtzOyBiYW5rKysg
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS9zZXR1cC5j
OjYwODoyNzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgNjA4IHwgICAgIGZvciAoIGJhbmsgPSAwIDsgYmFuayA8
IG1lbS0+bnJfYmFua3M7IGJhbmsrKyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICBeCmFyY2gvYXJtL3NldHVwLmM6NjE5OjI3OiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA2MTkgfCAgICAg
Zm9yICggYmFuayA9IDAgOyBiYW5rIDwgbWVtLT5ucl9iYW5rczsgYmFuaysrICkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KYXJjaC9hcm0vc2V0dXAuYzogSW4gZnVu
Y3Rpb24g4oCYc3RhcnRfeGVu4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNzU6MjA6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDM3NSB8ICAgICAgICAgICAgICAoY3B1KSA8IG5yX2NwdV9pZHM7ICAg
ICAgICAgICAgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDo0NjA6MzY6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhm
b3JfZWFjaF9jcHXigJkKICA0NjAgfCAjZGVmaW5lIGZvcl9lYWNoX3ByZXNlbnRfY3B1KGNw
dSkgIGZvcl9lYWNoX2NwdShjcHUsICZjcHVfcHJlc2VudF9tYXApCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn4KYXJjaC9hcm0vc2V0
dXAuYzo4NDE6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3By
ZXNlbnRfY3B14oCZCiAgODQxIHwgICAgIGZvcl9lYWNoX3ByZXNlbnRfY3B1ICggaSApCiAg
ICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+CmFyY2gvYXJtL3NldHVwLmM6ODQzOjMz
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICA4NDMgfCAgICAgICAgIGlmICggKG51bV9vbmxpbmVfY3B1cygpIDwg
bnJfY3B1X2lkcykgJiYgIWNwdV9vbmxpbmUoaSkgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgXgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9jcHUuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vc2h1dGRvd24u
YzozOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g
4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxp
bmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAgIGFyY2gvYXJtL3NtcC5vCklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRl
L3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01Q
QVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8
ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lk
ICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFS
RV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwg
MCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSBhcmNoL2FybS9zaHV0ZG93bi5jOjc6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJu
YXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90
aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo2OSwKICAg
ICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3NtcC5jOjI6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEg
fCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAg
ICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wbGF0Zm9ybS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zaHV0ZG93bi5jOjg6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVu
L251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5f
dG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21h
aW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hf
bWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAo
dW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwy
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRl
cuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBh
Z2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVk
ZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9p
bmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2sp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBj
cHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54
dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0
aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3Mu
aDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9u
IOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJz
cOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0
aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g
4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNx
dWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0
cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBp
ZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2Fs
dHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAg
IHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBw
Mm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3Rh
dGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL21tLmg6MjMzOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1
bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
Zm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9u
ZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVj
dG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAg
ICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2lu
Zm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy
4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0
X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hf
ZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190
ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMy
X3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNw
dV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUg
KnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlv
biDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVz
dF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ2ljLmg6MTYzLAogICAgICAg
ICAgICAgICAgIGZyb20gYXJjaC9hcm0vc21wLmM6NjoKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3Jl
YWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgph
cmNoL2FybS9zbXAuYzogSW4gZnVuY3Rpb24g4oCYYXJjaF9mbHVzaF90bGJfbWFza+KAmToK
YXJjaC9hcm0vc21wLmM6OTo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgICA5IHwgdm9pZCBhcmNoX2ZsdXNoX3RsYl9t
YXNrKGNvbnN0IGNwdW1hc2tfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1
NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0
cnVjdCBwY2lfZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVh
c3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5l
IHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lf
d3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBp
bnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQg
cmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3Rf
Y2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2
XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEz
MSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS9zaHV0
ZG93bi5jOiBJbiBmdW5jdGlvbiDigJhoYWx0X3RoaXNfY3B14oCZOgphcmNoL2FybS9zaHV0
ZG93bi5jOjExOjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGFyZ+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyB2b2lkIG5vcmV0dXJuIGhhbHRfdGhp
c19jcHUodm9pZCAqYXJnKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+Xn5+CiAgQ0MgICAgICBhcmNoL2FybS9zbXBib290Lm8KICBDQyAgICAg
IGFyY2gvYXJtL3N0YXRpYy1ldnRjaG4ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNsaW51eC5oOjU0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FjcGkvcGxhdGZvcm0vYWNlbnYuaDoxNDIsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9hY3BpLmg6NTYsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL2FjcGkuaDo1NywKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gv
YXJtL3NtcGJvb3QuYzoxMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50
IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IGFyY2gvYXJtL3N0YXRpYy1ldnRjaG4uYzozOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcGVyZmMuaDo3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjcwLAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9zZXR1cC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FjcGkuaDoyNSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1ODoKLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1
IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hl
bi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQg
PT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGluZSBib29s
IGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGlj
IGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNrKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5o
OjEyLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMzoKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYs
IHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8
ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rp
b24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEz
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8
ICAgICAgICAgICAgIF5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhw
Mm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVt
YXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUg
fCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1h
c2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFp
bi5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6
Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgou
L2luY2x1ZGUveGVuL251bWEuaDoxMjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBu
b2RlaWRfdCBtZm5fdG9fbmlkKG1mbl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBm
dW5jdGlvbiDigJhhY3BpX3NldF9jc3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNw
aS5oOjE5Mjo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9z
ZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhh
Y3BpX3NldF9jc3Vic3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0
YXRlX2xpbWl0KHVuc2lnbmVkIGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS9zbXBib290LmM6
MTU6Ci4vaW5jbHVkZS94ZW4vZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rpb24g4oCYdW5tYXBf
ZG9tYWluX3BhZ2VfZ2xvYmFs4oCZOgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6NzE6
NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNzEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9tYWluX3BhZ2VfZ2xv
YmFsKGNvbnN0IHZvaWQgKnZhKSB7fTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSBhcmNoL2FybS9zbXBib290LmM6MTk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4g
ZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6
MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVz
cyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5z
aWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1X
c2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49
IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vc21wYm9v
dC5jOjIwOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25p
ZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBp
bmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzcg
fCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5o
OjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDi
gJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIu
aDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRl
cihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICpt
YXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+
fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6
MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFy
Y2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6
NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0
IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEw
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj
dCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5o
OjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFk
X2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhy
Y3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBi
b29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRf
c3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4
OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uo
c3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxp
bmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVz
dF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2Fs
dHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9h
bHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dm
buKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAg
ICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwg
cDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0
aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjEx
MzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRf
ZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+Xgph
cmNoL2FybS9zbXBib290LmM6IEluIGZ1bmN0aW9uIOKAmGR0X3NtcF9pbml0X2NwdXPigJk6
CmFyY2gvYXJtL3NtcGJvb3QuYzoxNjk6MjI6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYdTMy4oCZIHth
a2Eg4oCYdW5zaWduZWQgaW504oCZfSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAxNjkgfCAgICAgICAgIGlmICggcmVnX2xlbiA8IGR0X2NlbGxzX3RvX3NpemUoZHRfbl9h
ZGRyX2NlbGxzKGNwdSkpICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICBeCmFyY2gv
YXJtL3NtcGJvb3QuYzogSW4gZnVuY3Rpb24g4oCYY3B1X3VwX3NlbmRfc2dp4oCZOgphcmNo
L2FybS9zbXBib290LmM6NDcxOjMyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNw
deKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MSB8IGludCBfX2luaXQgY3B1X3VwX3Nl
bmRfc2dpKGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fl5+fgphcmNoL2FybS9zbXBib290LmM6IEluIGZ1bmN0aW9uIOKAmGNwdV9zbXBib290X2Nh
bGxiYWNr4oCZOgphcmNoL2FybS9zbXBib290LmM6NTcwOjU2OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG5mYuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU3MCB8IHN0YXRp
YyBpbnQgY3B1X3NtcGJvb3RfY2FsbGJhY2soc3RydWN0IG5vdGlmaWVyX2Jsb2NrICpuZmIs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgphcmNoL2FybS9zbXBib290LmM6IEF0IHRvcCBsZXZlbDoKYXJjaC9h
cm0vc21wYm9vdC5jOjYwODoxOiB3YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJl
Z2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgNjA4
IHwgcHJlc21wX2luaXRjYWxsKGNwdV9zbXBib290X25vdGlmaWVyX2luaXQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBm
dW5jdGlvbiDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1
X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9k
b21haW4uaDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEw
Ojc1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L3NjaGVkLmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X2Fzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjc4IHwgc3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4
Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2
aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZw
Y2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90
IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGlu
ZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2
b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQz
Ml90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDozMDA6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBj
aV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2Rv
bWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRp
YyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9ldmVudC5oOjE3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZXZlbnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh2aXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfdmlycV9pc19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4K
YXJjaC9hcm0vc3RhdGljLWV2dGNobi5jOiBJbiBmdW5jdGlvbiDigJhnZXRfZXZ0Y2huX2R0
X3Byb3BlcnR54oCZOgphcmNoL2FybS9zdGF0aWMtZXZ0Y2huLmM6MjI6MjI6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYdWludDMyX3TigJkge2FrYSDigJh1bnNpZ25lZCBpbnTigJl9IGFuZCDigJhp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogICAyMiB8ICAgICBpZiAoICFsZW4gfHwgbGVuIDwg
ZHRfY2VsbHNfdG9fc2l6ZShTVEFUSUNfRVZUQ0hOX05PREVfU0laRV9DRUxMUykgKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgIF4KICBDQyAgICAgIGFyY2gvYXJtL3N0YXRpYy1t
ZW1vcnkubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9h
bHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bp
bmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zdGF0aWMtbWVtb3J5LmM6MzoK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1
bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUv
cHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1
YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlv
biDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9T
QVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJF
X0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBe
fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09N
UEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVD
TEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAg
IHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgQ0MgICAgICBhcmNoL2FybS9zdGF0aWMtc2ht
ZW0ubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1
bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8
IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09
IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVt
YXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFz
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
bGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBp
bmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94
ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1m
bl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2N1cnJlbnQuaDo4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL2FzbS1nZW5lcmljL3BlcmNwdS5oOjgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2FzbS9wZXJjcHUuaDoxLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9wZXJjcHUuaDozMCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZGV2aWNlX3RyZWUuaDoyMiwKICAgICAgICAgICAgICAg
ICBmcm9tIGFyY2gvYXJtL3N0YXRpYy1zaG1lbS5jOjM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFw
KHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZu
LCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4v
aW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKA
mHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZv
ICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAg
ICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVz
aHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVz
aF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9s
b2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vcndsb2NrLmg6NjoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAg
ICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1
bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0
MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRp
ZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBb
LVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAg
ICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0
aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEz
MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhy
Y3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxv
Y19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoy
MSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjc6Ci4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09N
UEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYg
fCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9p
ZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9I
Vk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwg
c3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4v
aW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xB
UkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQs
IDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2Fj
cXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291
cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lk
IHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR4
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRv
bWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90
IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rp
b24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9j
ayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlu
bGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hl
ZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3Np
Z25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8
IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShz
dHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3Jl
YWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNp
X3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2
cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3Qg
ZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBk
YXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAw
OjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJv
Y2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMSwKICAgICAgICAgICAgICAg
ICBmcm9tIGFyY2gvYXJtL3N0YXRpYy1zaG1lbS5jOjY6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDox
Mjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1m
bl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21t
Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9k
b21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0
aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21t
Lmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9u
IOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmls
dGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3Qg
Km1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10
cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1
X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9l
dmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vi
gJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50
MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9u
ZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nh
bml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwg
c3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30K
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVw
ZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVz
dF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9y
eV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVu
L3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRl
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZw
Y2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikg
eyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDi
gJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVu
ZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2
Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJ
biBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2
XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fl4KYXJjaC9hcm0vc3RhdGljLXNobWVtLmM6IEluIGZ1bmN0aW9uIOKAmGFjcXVpcmVf
bnJfYm9ycm93ZXJfZG9tYWlu4oCZOgphcmNoL2FybS9zdGF0aWMtc2htZW0uYzoyMjo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDIyIHwgc3RhdGljIGludCBfX2luaXQgYWNxdWlyZV9ucl9ib3Jyb3dlcl9kb21haW4o
c3RydWN0IGRvbWFpbiAqZCwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL3N0YXRpYy1zaG1l
bS5jOiBJbiBmdW5jdGlvbiDigJhhc3NpZ25fc2hhcmVkX21lbW9yeeKAmToKYXJjaC9hcm0v
c3RhdGljLXNobWVtLmM6MTY5OjE3OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIHVuc2lnbmVk
IGV4cHJlc3Npb24gaW4g4oCYPj0gMOKAmSBpcyBhbHdheXMgdHJ1ZSBbLVd0eXBlLWxpbWl0
c10KICAxNjkgfCAgICAgd2hpbGUgKCAtLWkgPj0gMCApCiAgICAgIHwgICAgICAgICAgICAg
ICAgIF5+CiAgQ0MgICAgICBhcmNoL2FybS9zeXNjdGwubwpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYXNtLWdlbmVyaWMvcGVyY3B1Lmg6OCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQvYXNt
L3BlcmNwdS5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3BlcmNw
dS5oOjMwLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo0
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kZXZpY2VfdHJlZS5oOjIy
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kdC1vdmVybGF5Lmg6MTQs
CiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zeXNjdGwuYzoxMjoKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJs
aW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBp
bnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5oOjY6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAg
ICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBe
Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8
ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJs
aWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2su
aDo3OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZN
X1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1
Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3Jv
IOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVf
VFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+
fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1
NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2
b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KICBDQyAgICAgIGFyY2gvYXJtL3RpbWUubwpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hzbS94c20uaDoxOSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vaHlwZXJjYWxsLmg6MjIsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS9zeXNj
dGwuYzoxNDoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19u
aWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMg
aW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwg
dW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9t
Zm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IElu
IGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUx
OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8
ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3
IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4v
bW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0u
aDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g
4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxi
Lmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAq
bWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+
fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRy
ZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3Vf
cmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3Qg
KmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g
4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9y
ZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIz
IHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2
ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1
Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQx
Nl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25n
IGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2Fu
aXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDox
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVu
dC5oOjgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYXNtLWdlbmVyaWMvcGVy
Y3B1Lmg6OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5l
cmF0ZWQvYXNtL3BlcmNwdS5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3BlcmNwdS5oOjMwLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9y
d2xvY2suaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kZXZpY2Vf
dHJlZS5oOjIyLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vdGltZS5jOjEyOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1
X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDox
MDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVu
c2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hf
dmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5
4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwg
c3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2
Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3Qg
cGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1
bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2ku
aDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50
MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXpl
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBp
bmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJl
ZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwg
dWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1
aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNr
IHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21w
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NjoKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAgICAgICAg
ICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4KLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwgICAgIGlm
IChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGljIGlubGlu
ZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVfY3B1bWFz
a192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzY1IHwg
c3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJfdCBtYXNr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2
XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEz
MSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWluX2NyZWF0ZeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDoyMjQ6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI0IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkLCB1aW50MzJfdCBzc2lkcmVmKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ2V0
ZG9tYWluaW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyMjk6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjI5IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX2dldGRvbWFpbmluZm8oeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RvbWN0bF9zY2hlZHVsZXJfb3DigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjM1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDIzNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bF9zY2hlZHVsZXJfb3Di
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjQwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI0MCB8IHN0YXRpYyBpbmxp
bmUgaW50IHhzbV9zeXNjdGxfc2NoZWR1bGVyX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQg
Y21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9zZXRfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0NjoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBk
b21haW4gKmUpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21jdGzigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6MjUxOjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDI1MSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21jdGwoeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc3lzY3Rs4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjI1Njo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyNTYgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lzY3Rs
KHhzbV9kZWZhdWx0X3QgZGVmLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3JlYWRjb25zb2xl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2
MTo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyNjEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVhZGNvbnNvbGUoeHNt
X2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNsZWFyKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3VuYm91bmTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MjY3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI2NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBldnRjaG4gKmNobiwgZG9taWRfdCBpZDIpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9ldnRjaG5faW50ZXJkb21haW7igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjczOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI3MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZDEsIHN0cnVjdCBldnRjaG4gKmNoYW4xLAogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Nl
bmTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjg1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2dGNobiAqY2huKQog
ICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fZXZ0Y2huX3N0YXR1c+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
OTE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjkxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgZXZ0Y2huICpjaG4pCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ldnRjaG5fcmVzZXTi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mjk3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfbWFwcmVm4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMwMzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAzMDMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZG9tYWluICpkMiwgdWludDMyX3QgZmxhZ3MpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9n
cmFudF91bm1hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDk6MTk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5
IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRv
bWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20v
eHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9zZXR1cOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozMTU6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzE1IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFu
dF90cmFuc2ZlcuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjE6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzIxIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9jb3B54oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjMyNzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzMjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dyYW50X3F1
ZXJ5X3NpemXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzMzOjE5OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMzMyB8ICAg
ICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4g
KmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21faW5pdF9oYXJkd2FyZV9kb21haW7igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MzY2OjU4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9p
bml0X2hhcmR3YXJlX2RvbWFpbih4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2dldF9wb2RfdGFyZ2V04oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM3MTo1Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNzEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3BvZF90YXJnZXQoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDozNzY6NTI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3NldF9w
b2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X2V4Y2hh
bmdl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4MTo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzODEgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fbWVtb3J5X2V4Y2hhbmdlKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21lbW9yeV9hZGp1c3RfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6Mzg3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM4NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0
IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5
X3N0YXRfcmVzZXJ2YXRpb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzkzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM5MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVj
dCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtb3J5X3Bpbl9wYWdl4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjM5OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTkgfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMiwKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX2FkZF90b19waHlzbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQwNjoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0MDYgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3JlbW92ZV9mcm9tX3BoeXNtYXDigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6NDEyOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxMiB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fbWFwX2dtZm5fZm9yZWlnbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MTg6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
LCBzdHJ1Y3QgZG9tYWluICp0KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fY2xhaW1fcGFnZXPigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDIzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyMyB8IHN0YXRpYyBpbmxpbmUgaW50
IHhzbV9jbGFpbV9wYWdlcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NvbnNvbGVf
aW/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDI4OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQyOCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9jb25zb2xlX2lvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9wcm9maWxl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQzMzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MzMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcHJvZmlsZSh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgaW50IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9rZXhlY+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Mzg6NDM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDM4IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2tleGVjKHhzbV9kZWZhdWx0X3QgZGVm
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2NoZWRvcF9zaHV0
ZG93buKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NDQ6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDQ0IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9tYXBfZG9tYWluX3BpcnHigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NDU0OjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tYXBfZG9tYWlu
X3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9pcnHi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDYwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ2MCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGlycSwgdm9pZCAqZGF0YSkKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMgaW5saW5lIGludCB4c21fdW5tYXBfZG9tYWlu
X3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5f
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ3MToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2lyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0
Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5kX3B0X2lycSAqYmluZCkKICAgICAgfCAgICAg
fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCY
eHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVy
bWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhf
dCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxv
dykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21hcHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNt
Lmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX3BjaV9jb25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo1MDc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCB1aW50MzJfdCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9hc3NpZ25fZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTM0OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDUzNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwg
Y29uc3QgY2hhciAqZHRwYXRoKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZGVhc3NpZ25fZHRkZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTQwOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU0MCB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY2hhciAqZHRwYXRoKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fcmVzb3VyY2VfcGx1Z19wY2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NTQ3OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDU0NyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVn
X3BjaSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV91
bnBsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1MzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NTMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fcmVzb3VyY2VfcGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU1ODo1
Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NTggfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19jb3Jl
KHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDog
SW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19jb3Jl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU2Mzo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVz
b3VyY2VfdW5wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4v
aW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9zZXR1cF9w
Y2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTY5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU2OSB8ICAgICB4c21f
ZGVmYXVsdF90IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9y
ZXNvdXJjZV9zZXR1cF9nc2nigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc0OjU2OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDU3NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9nc2koeHNtX2Rl
ZmF1bHRfdCBkZWYsIGludCBnc2kpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfbWlzY+KAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1Nzk6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jl
c291cmNlX3NldHVwX21pc2MoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BhZ2Vfb2ZmbGluZeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1ODQ6NTA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg0IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX3BhZ2Vfb2ZmbGluZSh4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3QgY21kKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h5cGZzX29w
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU4OTo0Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1ODkgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21faHlwZnNfb3AoeHNtX2RlZmF1bHRfdCBkZWYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW3igJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NjA3OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDYwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBvcCkKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2h2bV9w
YXJhbV9hbHRwMm1odm3igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjEyOjU3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYx
MiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9odm1fcGFyYW1fYWx0cDJtaHZtKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fYWx0cDJtaHZtX29w4oCZOgouL2lu
Y2x1ZGUveHNtL3hzbS5oOjYxODoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
ZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTggfCAgICAgeHNtX2RlZmF1bHRfdCBk
ZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2NF90IG1vZGUsIHVpbnQzMl90IG9wKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ2V0X3ZudW1haW5mb+KAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MjM6
NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNjIzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF92bnVtYWluZm8oeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3ZtX2V2ZW50X2NvbnRyb2zigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NjI5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyOSB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgaW50IG1vZGUsIGludCBvcCkKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X21lbV9hY2Nlc3PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjM1OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYzNSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9tZW1fYWNjZXNzKHhzbV9kZWZhdWx0X3QgZGVmLCBz
dHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9u
IOKAmHhzbV9wbGF0Zm9ybV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2NTU6NDk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjU1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3BsYXRmb3JtX29wKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RtX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjczODo0Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA3MzggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG1fb3AoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV94ZW5fdmVyc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDM6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NzQzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3hlbl92ZXJzaW9uKHhzbV9kZWZhdWx0X3Qg
ZGVmLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NzQ4OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDc0OCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9kb21haW5fcmVzb3Vy
Y2VfbWFwKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+CmFyY2gvYXJtL3N5c2N0bC5jOiBJbiBmdW5jdGlvbiDigJhhcmNoX2RvX3N5c2N0bOKA
mToKYXJjaC9hcm0vc3lzY3RsLmM6Mjc6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYdV9zeXNjdGzigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjcgfCAgICAgICAgICAg
ICAgICAgICAgIFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0oeGVuX3N5c2N0bF90KSB1X3N5c2N0
bCkKSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo3OgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBB
VF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwg
ICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQg
KmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9z
YXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZN
X1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0
cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJF
X0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAw
LCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJxLmg6NSwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL3RpbWUuYzoxNDoKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3Jl
YWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
cmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1
ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZv
aWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJ
biBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL3RpbWUuYzoxNjoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29s
IGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxv
bmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KICBDQyAgICAgIGFyY2gv
YXJtL3RyYXBzLm8KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3
Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9p
bmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
ZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0
cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAg
ICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExLAogICAgICAgICAgICAgICAgIGZyb20g
YXJjaC9hcm0vdGltZS5jOjE4OgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g
4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEy
OSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nl
c3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhz
dHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9h
Y2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNz
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVf
cmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYz
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2No
ZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9h
bHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVj
dCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAq
ZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IElu
IGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wg
cDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCY
YXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5
Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1
Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2
b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
MTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3Ry
dWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fgogIENDICAgICAgYXJjaC9hcm0vdmNwcmVnLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1NCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjZW52Lmg6MTQyLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvYWNwaS5oOjU2LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9hY3BpLmg6NTcsCiAgICAgICAgICAgICAgICAg
ZnJvbSBhcmNoL2FybS90cmFwcy5jOjEwOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZl
YXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlu
bGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBk
ZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2
cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rldmlj
ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYp
IHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g
4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3Bl
bmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGlj
IGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3Qg
dmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJ
biBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FW
RV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6
MTcsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS90aW1lLmM6MjI6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBp
bnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92Y3ByZWcu
YzoxMDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJs
aW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo3
MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc2V0dXAu
aDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hY3Bp
Lmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYWNwaS9wbGF0Zm9ybS9h
Y2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAg
ICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAg
ICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFs
bG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
bW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBp
bmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1
bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21m
bnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJ
biBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYg
fCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsK
ICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FW
RV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAg
IDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2Vu
ZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSBhcmNoL2FybS90aW1lLmM6MjM6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlv
biDigJhhY3BpX3NldF9jc3RhdGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6
NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4v
aW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFn
ZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0Nzcg
fCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9s
aXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9t
bS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTI6NTU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTkyIHwgc3RhdGljIGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdGF0ZV9saW1pdCh1bnNpZ25l
ZCBpbnQgbmV3X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgouL2luY2x1
ZGUveGVuL2FjcGkuaDogSW4gZnVuY3Rpb24g4oCYYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0
4oCZOgouL2luY2x1ZGUveGVuL2FjcGkuaDoxOTQ6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYbmV3X2xpbWl04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk0IHwgc3Rh
dGljIGlubGluZSB2b2lkIGFjcGlfc2V0X2NzdWJzdGF0ZV9saW1pdCh1bnNpZ25lZCBpbnQg
bmV3X2xpbWl0KSB7IHJldHVybjsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3Rh
dGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQz
Ml90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVz
dGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVz
dGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3Jj
dXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94
ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9j
a+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBy
Y3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4v
aW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5e
fn5+CmFyY2gvYXJtL3RpbWUuYzogSW4gZnVuY3Rpb24g4oCYaHRpbWVyX2ludGVycnVwdOKA
mToKYXJjaC9hcm0vdGltZS5jOjI0NDozNDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDQgfCBzdGF0aWMgdm9pZCBodGlt
ZXJfaW50ZXJydXB0KGludCBpcnEsIHZvaWQgKmRldl9pZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5efn4KYXJjaC9hcm0vdGltZS5jOjI0NDo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZXZfaWTigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyNDQgfCBzdGF0aWMgdm9pZCBodGltZXJfaW50ZXJydXB0KGludCBpcnEsIHZvaWQg
KmRldl9pZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl5+fn5+fgphcmNoL2FybS90aW1lLmM6IEluIGZ1bmN0aW9uIOKAmHZ0aW1lcl9p
bnRlcnJ1cHTigJk6CmFyY2gvYXJtL3RpbWUuYzoyNTg6MzQ6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYaXJx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU4IHwgc3RhdGlj
IHZvaWQgdnRpbWVyX2ludGVycnVwdChpbnQgaXJxLCB2b2lkICpkZXZfaWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+Xn5+CmFyY2gvYXJtL3RpbWUuYzoy
NTg6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGV2X2lk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjU4IHwgc3RhdGljIHZvaWQgdnRpbWVyX2ludGVycnVwdChpbnQg
aXJxLCB2b2lkICpkZXZfaWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5efn5+fn4KYXJjaC9hcm0vdGltZS5jOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfdmNwdV9zeXN0ZW1fdGltZeKAmToKYXJjaC9hcm0vdGltZS5jOjM1ODo0Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzU4IHwgdm9pZCB1cGRhdGVfdmNwdV9zeXN0ZW1fdGltZShzdHJ1Y3QgdmNwdSAqdikK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCmFy
Y2gvYXJtL3RpbWUuYzogSW4gZnVuY3Rpb24g4oCYY3B1X3RpbWVfY2FsbGJhY2vigJk6CmFy
Y2gvYXJtL3RpbWUuYzozNzU6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbmZi
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzc1IHwgc3RhdGljIGludCBjcHVfdGltZV9j
YWxsYmFjayhzdHJ1Y3Qgbm90aWZpZXJfYmxvY2sgKm5mYiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJt
L3RpbWUuYzozNzc6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGNwdeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdm9pZCAqaGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl5+fn4KYXJjaC9hcm0vdGltZS5jOiBBdCB0b3AgbGV2ZWw6CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2Vt
dWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIz
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0
IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5p
dHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0v
dGltZS5jOjQwMToxOiB3YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJlZ2lubmlu
ZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgNDAxIHwgX19p
bml0Y2FsbChjcHVfdGltZV9ub3RpZmllcl9pbml0KTsKICAgICAgfCBefn5+fn5+fn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJt
X2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWlu
ICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9z
bXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVu
c2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNw
dSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94
ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDi
gJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0
ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9j
cHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNs
dWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVk
ZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7i
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90
IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9u
IOKAmGFjcGlfc2V0X2NzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTky
OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDE5MiB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Rh
dGVfbGltaXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+fn5+fn4KLi9pbmNsdWRlL3hlbi9hY3BpLmg6IEluIGZ1bmN0aW9uIOKAmGFjcGlfc2V0
X2NzdWJzdGF0ZV9saW1pdOKAmToKLi9pbmNsdWRlL3hlbi9hY3BpLmg6MTk0OjU4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5ld19saW1pdOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhY3BpX3NldF9jc3Vic3RhdGVfbGlt
aXQodW5zaWduZWQgaW50IG5ld19saW1pdCkgeyByZXR1cm47IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+
fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL3RyYXBzLmM6MTE6Ci4vaW5j
bHVkZS94ZW4vZG9tYWluX3BhZ2UuaDogSW4gZnVuY3Rpb24g4oCYdW5tYXBfZG9tYWluX3Bh
Z2VfZ2xvYmFs4oCZOgouL2luY2x1ZGUveGVuL2RvbWFpbl9wYWdlLmg6NzE6NTc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NzEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdW5tYXBfZG9tYWluX3BhZ2VfZ2xvYmFsKGNvbnN0
IHZvaWQgKnZhKSB7fTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJ
biBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVk
IGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9t
bS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9k
aXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlv
biDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpo
ZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9w
cmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3Rf
ZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2Vf
bGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4
OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZs
dXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1h
c2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwg
c3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVp
bnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+
fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6
Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9j
a+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3Rh
dGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2Nr
X3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2No
ZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94c20veHNtLmg6MTks
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2FsbC5oOjIyLAog
ICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vdHJhcHMuYzoxMzoKLi9pbmNsdWRlL3hl
bi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4v
bnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90
b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVt
X2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4g
ZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21l
bV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDog
SW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlu
bGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wg
YXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0
OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVp
bnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4K
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2Vf
ZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBl
X3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5p
dHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBh
cmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3Bv
bGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
OSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1
Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5j
bHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3Ry
dWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJ
biBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFz
c2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWlu
dDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUg
dWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93
cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGlu
dCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9j
aGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2so
c3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19w
dl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEx
MzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25z
dCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVk
ZS94c20veHNtLmg6MjI0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dl
dGRvbWFpbmluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV9kb21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjIzNToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBk
b21haW4gKmQsIGludCBjbWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21fc3lzY3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50
IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDi
gJh4c21fc2V0X3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MjQ2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
ZG9tYWluICplKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjI1MTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDoyNTY6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0
bCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoy
NjE6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjYxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhz
bV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI2NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZXZ0Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjI3MzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyNzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQxLCBzdHJ1Y3QgZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9z
ZW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
MjkxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgc3RydWN0IGV2dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V0
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2Rl
ZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAg
ICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2dyYW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6
MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzAzIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpk
MSwgc3RydWN0IGRvbWFpbiAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
Z3JhbnRfdW5tYXByZWbigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMw
OSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBk
b21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6MzE1OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3Jh
bnRfdHJhbnNmZXLigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21h
aW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDozMjc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9x
dWVyeV9zaXpl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWlu
ICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjM2Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21f
aW5pdF9oYXJkd2FyZV9kb21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9nZXRfcG9kX3RhcmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzcxIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6Mzc2OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRf
cG9kX3RhcmdldCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNo
YW5nZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9tZW1vcnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjM4NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9y
eV9zdGF0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzOTMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1
Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9p
bmNsdWRlL3hzbS94c20uaDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
ZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3Qg
ZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9hZGRfdG9fcGh5c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDA2IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAg
fCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX21hcF9nbWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgc3RydWN0IGRvbWFpbiAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGlu
dCB4c21fY2xhaW1fcGFnZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xl
X2lv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMg
aW5saW5lIGludCB4c21fY29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRv
bWFpbiAqZCwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fcHJvZmlsZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDMzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21fa2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQzOCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRl
ZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1
dGRvd27igJk6CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQy
KQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJ
biBmdW5jdGlvbiDigJh4c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQ1NDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFp
bl9waXJxKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx
4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBp
bmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVh
c3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAg
fCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZv
aWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVh
c3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VubWFwX2RvbWFpbl9w
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ2NTo1NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjUgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fdW5tYXBfZG9tYWluX3BpcnEoeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV91bm1hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjQ3MToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA0NzEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9iaW5kX3B0X2ly
ceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0Nzc6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9iaW5k
X3B0X2lycSAqYmluZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3VuYmluZF9wdF9pcnHigJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDgzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4MyB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhlbl9kb21jdGxfYmluZF9wdF9pcnEgKmJp
bmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9pcnFfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94
c20uaDo0ODk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDg5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3Qg
ZG9tYWluICpkLCBpbnQgcGlycSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lv
bWVtX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDk1OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ5
NSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3Qg
cywgdWludDY0X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2lvbWVtX21h
cHBpbmfigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTAxOjE5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDUwMSB8ICAgICB4
c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDY0X3QgcywgdWludDY0
X3QgZSwgdWludDhfdCBhbGxvdykKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNi
ZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQz
Ml90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVp
bnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNp
emUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BjaV9j
b25maWdfcGVybWlzc2lvbuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1MDc6MTk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NTA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1aW50MzJf
dCBtYWNoaW5lX2JkZiwgdWludDE2X3Qgc3RhcnQsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hc3NpZ25f
ZHRkZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNp
X3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQg
aW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50
IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
c2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBz
aXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0
X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hzbS94c20uaDo1MzQ6MTk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTM0IHwgICAg
IHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBh
dGgpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9kZWFzc2lnbl9kdGRldmljZeKAmToKLi9pbmNsdWRlL3hz
bS94c20uaDo1NDA6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQwIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkLCBjb25zdCBjaGFyICpkdHBhdGgpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNv
dXJjZV9wbHVnX3BjaeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NDc6NTU6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTQ3
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfcGNpKHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3VucGx1Z19wY2nigJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NTUzOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1MyB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgdWludDMyX3QgbWFjaGluZV9iZGYpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9yZXNvdXJjZV9w
bHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTU4OjU2OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDU1OCB8IHN0
YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9wbHVnX2NvcmUoeHNtX2RlZmF1bHRfdCBk
ZWYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fcmVzb3VyY2VfdW5wbHVnX2NvcmXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTYzOjU4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDU2MyB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV91bnBsdWdfY29y
ZSh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX3BjaeKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1Njk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTY5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCB1
aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNlX3NldHVwX2dz
aeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NzQ6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTc0IHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Jlc291cmNlX3NldHVwX2dzaSh4c21fZGVmYXVsdF90IGRlZiwgaW50
IGdzaSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV9zZXR1cF9taXNj4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3OTo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA1NzkgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfbWlz
Yyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5o
OiBJbiBmdW5jdGlvbiDigJh4c21fcGFnZV9vZmZsaW5l4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjU4NDo1MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA1ODQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcGFnZV9vZmZs
aW5lKHhzbV9kZWZhdWx0X3QgZGVmLCB1aW50MzJfdCBjbWQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHlwZnNfb3DigJk6Ci4vaW5jbHVkZS94
c20veHNtLmg6NTg5OjQ2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDU4OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9oeXBm
c19vcCh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9wYXJhbeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2MDc6MTk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNjA3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCB1
bnNpZ25lZCBsb25nIG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faHZtX3BhcmFtX2FsdHAybWh2beKA
mToKLi9pbmNsdWRlL3hzbS94c20uaDo2MTI6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjEyIHwgc3RhdGljIGlubGlu
ZSBpbnQgeHNtX2h2bV9wYXJhbV9hbHRwMm1odm0oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2h2bV9hbHRwMm1odm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6
NjE4OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDYxOCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFp
biAqZCwgdWludDY0X3QgbW9kZSwgdWludDMyX3Qgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRf
dm51bWFpbmZv4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYyMzo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MjMgfCBz
dGF0aWMgaW5saW5lIGludCB4c21fZ2V0X3ZudW1haW5mbyh4c21fZGVmYXVsdF90IGRlZiwg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fdm1fZXZlbnRfY29udHJvbOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo2
Mjk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNjI5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkLCBpbnQgbW9kZSwgaW50IG9wKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fbWVtX2FjY2Vzc+KAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo2MzU6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjM1IHwgc3RhdGljIGlubGluZSBp
bnQgeHNtX21lbV9hY2Nlc3MoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3BsYXRmb3Jt
X29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjY1NTo0OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2NTUgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcGxhdGZvcm1fb3AoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90
IG9wKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG1f
b3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzM4OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDczOCB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV9kbV9vcCh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3hlbl92ZXJzaW9u
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0Mzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3NDMgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21feGVuX3ZlcnNpb24oeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IG9w
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZG9tYWlu
X3Jlc291cmNlX21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3NDg6NTc6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzQ4
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2RvbWFpbl9yZXNvdXJjZV9tYXAoeHNtX2RlZmF1
bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5j
bHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGlu
ZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL3RyYXBz
LmM6MjE6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDogSW4gZnVuY3Rpb24g4oCYbm9fY29uZmln
X3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFtLmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVy
YW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGxvbmcgaW504oCZ
IHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBkdWUgdG8g
dW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8
ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChlID49IHMpOyBlIC0gczsgfSkgOiBzdHJs
ZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRl
X1NDVExS4oCZOgphcmNoL2FybS92Y3ByZWcuYzo2Mjo0MDogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDYyIHwgc3RhdGlj
IGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywgdHlwZSMjc3ojI190ICpy
LCBib29sIHJlYWQpICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5jOjc1OjM0OiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA3NSB8ICNkZWZpbmUgVFZNX1JFRzMy
KHJlZ25hbWUsIHhyZWcpIFRWTV9SRUcoMzIsIHZyZWdfZW11bGF0ZV8jI3JlZ25hbWUsIHhy
ZWcpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fgph
cmNoL2FybS92Y3ByZWcuYzoxMjU6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mFRWTV9SRUczMuKAmQogIDEyNSB8IFRWTV9SRUczMihTQ1RMUiwgU0NUTFJfRUwxKQogICAg
ICB8IF5+fn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19l
bXVsYXRlX1RUQlIwXzMy4oCZOgphcmNoL2FybS92Y3ByZWcuYzo2Mjo0MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDYy
IHwgc3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywgdHlwZSMj
c3ojI190ICpyLCBib29sIHJlYWQpICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5jOjc1OjM0OiBub3Rl
OiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA3NSB8ICNkZWZpbmUg
VFZNX1JFRzMyKHJlZ25hbWUsIHhyZWcpIFRWTV9SRUcoMzIsIHZyZWdfZW11bGF0ZV8jI3Jl
Z25hbWUsIHhyZWcpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Xn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzoxMzc6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmFRWTV9SRUczMuKAmQogIDEzNyB8IFRWTV9SRUczMihUVEJSMF8zMiwgVFRC
UjBfRUwxKQogICAgICB8IF5+fn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzogSW4gZnVuY3Rp
b24g4oCYdnJlZ19lbXVsYXRlX1RUQlIxXzMy4oCZOgphcmNoL2FybS92Y3ByZWcuYzo2Mjo0
MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDYyIHwgc3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAq
cmVncywgdHlwZSMjc3ojI190ICpyLCBib29sIHJlYWQpICAgIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5j
Ojc1OjM0OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA3
NSB8ICNkZWZpbmUgVFZNX1JFRzMyKHJlZ25hbWUsIHhyZWcpIFRWTV9SRUcoMzIsIHZyZWdf
ZW11bGF0ZV8jI3JlZ25hbWUsIHhyZWcpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgXn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzoxMzg6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUczMuKAmQogIDEzOCB8IFRWTV9SRUczMihU
VEJSMV8zMiwgVFRCUjFfRUwxKQogICAgICB8IF5+fn5+fn5+fgphcmNoL2FybS92Y3ByZWcu
YzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX1RUQlIw4oCZOgphcmNoL2FybS92Y3By
ZWcuYzo2Mjo0MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDYyIHwgc3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3Vz
ZXJfcmVncyAqcmVncywgdHlwZSMjc3ojI190ICpyLCBib29sIHJlYWQpICAgIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJt
L3ZjcHJlZy5jOjc2OjM0OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JF
R+KAmQogICA3NiB8ICNkZWZpbmUgVFZNX1JFRzY0KHJlZ25hbWUsIHhyZWcpIFRWTV9SRUco
NjQsIHZyZWdfZW11bGF0ZV8jI3JlZ25hbWUsIHhyZWcpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzoxNDA6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUc2NOKAmQogIDE0MCB8IFRW
TV9SRUc2NChUVEJSMCwgVFRCUjBfRUwxKQogICAgICB8IF5+fn5+fn5+fgphcmNoL2FybS92
Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX1RUQlIx4oCZOgphcmNoL2Fy
bS92Y3ByZWcuYzo2Mjo0MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDYyIHwgc3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3Qg
Y3B1X3VzZXJfcmVncyAqcmVncywgdHlwZSMjc3ojI190ICpyLCBib29sIHJlYWQpICAgIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFy
Y2gvYXJtL3ZjcHJlZy5jOjc2OjM0OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
VFZNX1JFR+KAmQogICA3NiB8ICNkZWZpbmUgVFZNX1JFRzY0KHJlZ25hbWUsIHhyZWcpIFRW
TV9SRUcoNjQsIHZyZWdfZW11bGF0ZV8jI3JlZ25hbWUsIHhyZWcpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzox
NDE6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUc2NOKAmQogIDE0
MSB8IFRWTV9SRUc2NChUVEJSMSwgVFRCUjFfRUwxKQogICAgICB8IF5+fn5+fn5+fgphcmNo
L2FybS92Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX1RDUl9FTDHigJk6
CmFyY2gvYXJtL3ZjcHJlZy5jOjg2OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJlZ3PigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgODYgfCBzdGF0aWMgYm9vbCB2cmVn
X2VtdWxhdGVfIyN4cmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCByZWdpc3Rlcl90
ICpyLCAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6MTQzOjE6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhUVk1fUkVHMzJfQ09NQklORUTigJkKICAxNDMgfCBU
Vk1fUkVHMzJfQ09NQklORUQoVFRCQ1IsIFRUQkNSMiwgVENSX0VMMSkKICAgICAgfCBefn5+
fn5+fn5+fn5+fn5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdf
ZW11bGF0ZV9EQUNS4oCZOgphcmNoL2FybS92Y3ByZWcuYzo2Mjo0MDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDYyIHwg
c3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywgdHlwZSMjc3oj
I190ICpyLCBib29sIHJlYWQpICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5jOjc1OjM0OiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFR+KAmQogICA3NSB8ICNkZWZpbmUgVFZN
X1JFRzMyKHJlZ25hbWUsIHhyZWcpIFRWTV9SRUcoMzIsIHZyZWdfZW11bGF0ZV8jI3JlZ25h
bWUsIHhyZWcpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn5+
fn5+fgphcmNoL2FybS92Y3ByZWcuYzoxNDQ6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmFRWTV9SRUczMuKAmQogIDE0NCB8IFRWTV9SRUczMihEQUNSLCBEQUNSMzJfRUwy
KQogICAgICB8IF5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9hcm0vdHJh
cHMuYzozNjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50
Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNf
Z2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5j
OiBJbiBmdW5jdGlvbiDigJh2cmVnX2VtdWxhdGVfREZTUuKAmToKYXJjaC9hcm0vdmNwcmVn
LmM6NjI6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVnc+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA2MiB8IHN0YXRpYyBib29sIGZ1bmMoc3RydWN0IGNwdV91c2Vy
X3JlZ3MgKnJlZ3MsIHR5cGUjI3N6IyNfdCAqciwgYm9vbCByZWFkKSAgICBcCiAgICAgIHwg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS92
Y3ByZWcuYzo3NTozNDogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUfi
gJkKICAgNzUgfCAjZGVmaW5lIFRWTV9SRUczMihyZWduYW1lLCB4cmVnKSBUVk1fUkVHKDMy
LCB2cmVnX2VtdWxhdGVfIyNyZWduYW1lLCB4cmVnKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIF5+fn5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6MTQ1OjE6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhUVk1fUkVHMzLigJkKICAxNDUgfCBUVk1f
UkVHMzIoREZTUiwgRVNSX0VMMSkKICAgICAgfCBefn5+fn5+fn4KYXJjaC9hcm0vdmNwcmVn
LmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdfZW11bGF0ZV9JRlNS4oCZOgphcmNoL2FybS92Y3By
ZWcuYzo2Mjo0MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDYyIHwgc3RhdGljIGJvb2wgZnVuYyhzdHJ1Y3QgY3B1X3Vz
ZXJfcmVncyAqcmVncywgdHlwZSMjc3ojI190ICpyLCBib29sIHJlYWQpICAgIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJt
L3ZjcHJlZy5jOjc1OjM0OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JF
R+KAmQogICA3NSB8ICNkZWZpbmUgVFZNX1JFRzMyKHJlZ25hbWUsIHhyZWcpIFRWTV9SRUco
MzIsIHZyZWdfZW11bGF0ZV8jI3JlZ25hbWUsIHhyZWcpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fgphcmNoL2FybS92Y3ByZWcuYzoxNDY6MTog
bm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUczMuKAmQogIDE0NiB8IFRW
TV9SRUczMihJRlNSLCBJRlNSMzJfRUwyKQogICAgICB8IF5+fn5+fn5+fgphcmNoL2FybS92
Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX0ZBUl9FTDHigJk6CmFyY2gv
YXJtL3ZjcHJlZy5jOjg2OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ3Pi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgODYgfCBzdGF0aWMgYm9vbCB2cmVnX2VtdWxh
dGVfIyN4cmVnKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCByZWdpc3Rlcl90ICpyLCAg
XAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6MTQ4OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhUVk1fUkVHMzJfQ09NQklORUTigJkKICAxNDggfCBUVk1fUkVH
MzJfQ09NQklORUQoREZBUiwgSUZBUiwgRkFSX0VMMSkKICAgICAgfCBefn5+fn5+fn5+fn5+
fn5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdfZW11bGF0ZV9B
REZTUuKAmToKYXJjaC9hcm0vdmNwcmVnLmM6NjI6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYcmVnc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA2MiB8IHN0YXRpYyBi
b29sIGZ1bmMoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHR5cGUjI3N6IyNfdCAqciwg
Ym9vbCByZWFkKSAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgphcmNoL2FybS92Y3ByZWcuYzo3NTozNDogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmFRWTV9SRUfigJkKICAgNzUgfCAjZGVmaW5lIFRWTV9SRUczMihy
ZWduYW1lLCB4cmVnKSBUVk1fUkVHKDMyLCB2cmVnX2VtdWxhdGVfIyNyZWduYW1lLCB4cmVn
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+fn4KYXJj
aC9hcm0vdmNwcmVnLmM6MTQ5OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhU
Vk1fUkVHMzLigJkKICAxNDkgfCBUVk1fUkVHMzIoQURGU1IsIEFGU1IwX0VMMSkKICAgICAg
fCBefn5+fn5+fn4KYXJjaC9hcm0vdmNwcmVnLmM6IEluIGZ1bmN0aW9uIOKAmHZyZWdfZW11
bGF0ZV9BSUZTUuKAmToKYXJjaC9hcm0vdmNwcmVnLmM6NjI6NDA6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVnc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA2MiB8IHN0
YXRpYyBib29sIGZ1bmMoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHR5cGUjI3N6IyNf
dCAqciwgYm9vbCByZWFkKSAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS92Y3ByZWcuYzo3NTozNDogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmFRWTV9SRUfigJkKICAgNzUgfCAjZGVmaW5lIFRWTV9S
RUczMihyZWduYW1lLCB4cmVnKSBUVk1fUkVHKDMyLCB2cmVnX2VtdWxhdGVfIyNyZWduYW1l
LCB4cmVnKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+
fn4KYXJjaC9hcm0vdmNwcmVnLmM6MTUwOjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNy
byDigJhUVk1fUkVHMzLigJkKICAxNTAgfCBUVk1fUkVHMzIoQUlGU1IsIEFGU1IxX0VMMSkK
ICAgICAgfCBefn5+fn5+fn4KYXJjaC9hcm0vdHJhcHMuYzogQXQgdG9wIGxldmVsOgphcmNo
L2FybS90cmFwcy5jOjE0NzoxOiB3YXJuaW5nOiDigJhzdGF0aWPigJkgaXMgbm90IGF0IGJl
Z2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUtZGVjbGFyYXRpb25dCiAgMTQ3
IHwgX19pbml0Y2FsbCh1cGRhdGVfc2Vycm9yc19jcHVfY2Fwcyk7CiAgICAgIHwgXn5+fn5+
fn5+fgphcmNoL2FybS92Y3ByZWcuYzogSW4gZnVuY3Rpb24g4oCYdnJlZ19lbXVsYXRlX01B
SVJfRUwx4oCZOgphcmNoL2FybS92Y3ByZWcuYzo4Njo1NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDg2IHwgc3RhdGlj
IGJvb2wgdnJlZ19lbXVsYXRlXyMjeHJlZyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywg
cmVnaXN0ZXJfdCAqciwgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZjcHJlZy5jOjE1Mjox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYVFZNX1JFRzMyX0NPTUJJTkVE4oCZ
CiAgMTUyIHwgVFZNX1JFRzMyX0NPTUJJTkVEKE1BSVIwLCBNQUlSMSwgTUFJUl9FTDEpCiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+CmFyY2gvYXJtL3ZjcHJlZy5jOiBJbiBmdW5jdGlv
biDigJh2cmVnX2VtdWxhdGVfQU1BSVJfRUwx4oCZOgphcmNoL2FybS92Y3ByZWcuYzo4Njo1
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWdz4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDg2IHwgc3RhdGljIGJvb2wgdnJlZ19lbXVsYXRlXyMjeHJlZyhzdHJ1Y3Qg
Y3B1X3VzZXJfcmVncyAqcmVncywgcmVnaXN0ZXJfdCAqciwgIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+CmFy
Y2gvYXJtL3ZjcHJlZy5jOjE1NDoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
VFZNX1JFRzMyX0NPTUJJTkVE4oCZCiAgMTU0IHwgVFZNX1JFRzMyX0NPTUJJTkVEKEFNQUlS
MCwgQU1BSVIxLCBBTUFJUl9FTDEpCiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+CmFyY2gv
YXJtL3ZjcHJlZy5jOiBJbiBmdW5jdGlvbiDigJh2cmVnX2VtdWxhdGVfQ09OVEVYVElEUuKA
mToKYXJjaC9hcm0vdmNwcmVnLmM6NjI6NDA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYcmVnc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA2MiB8IHN0YXRpYyBib29sIGZ1
bmMoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHR5cGUjI3N6IyNfdCAqciwgYm9vbCBy
ZWFkKSAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgphcmNoL2FybS92Y3ByZWcuYzo3NTozNDogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmFRWTV9SRUfigJkKICAgNzUgfCAjZGVmaW5lIFRWTV9SRUczMihyZWduYW1l
LCB4cmVnKSBUVk1fUkVHKDMyLCB2cmVnX2VtdWxhdGVfIyNyZWduYW1lLCB4cmVnKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+fn4KYXJjaC9hcm0v
dmNwcmVnLmM6MTU1OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhUVk1fUkVH
MzLigJkKICAxNTUgfCBUVk1fUkVHMzIoQ09OVEVYVElEUiwgQ09OVEVYVElEUl9FTDEpCiAg
ICAgIHwgXn5+fn5+fn5+CmFyY2gvYXJtL3RyYXBzLmM6IEluIGZ1bmN0aW9uIOKAmHNob3df
cmVnaXN0ZXJzXzMy4oCZOgphcmNoL2FybS90cmFwcy5jOjc2MDo1MDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzYwIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZjcHUgKnYpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fl4K
YXJjaC9hcm0vdHJhcHMuYzogSW4gZnVuY3Rpb24g4oCYc2hvd19yZWdpc3RlcnNfNjTigJk6
CmFyY2gvYXJtL3RyYXBzLmM6ODM3OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA4MzcgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS90cmFwcy5j
OiBJbiBmdW5jdGlvbiDigJhkZXBfc2NoZWRfb3BfY29tcGF04oCZOgphcmNoL2FybS90cmFw
cy5jOjEzNDA6MzA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY21k4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAxMzQwIHwgbG9uZyBkZXBfc2NoZWRfb3BfY29tcGF0KGludCBj
bWQsIHVuc2lnbmVkIGxvbmcgYXJnKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+Xn5+CmFyY2gvYXJtL3RyYXBzLmM6MTM0MDo0OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhhcmfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDEzNDAgfCBsb25nIGRl
cF9zY2hlZF9vcF9jb21wYXQoaW50IGNtZCwgdW5zaWduZWQgbG9uZyBhcmcpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+CmFy
Y2gvYXJtL3RyYXBzLmM6IEluIGZ1bmN0aW9uIOKAmGRlcF9ldmVudF9jaGFubmVsX29wX2Nv
bXBhdOKAmToKYXJjaC9hcm0vdHJhcHMuYzoxMzQ1OjcwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHVvcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTM0NSB8IGxvbmcgZGVw
X2V2ZW50X2NoYW5uZWxfb3BfY29tcGF0KFhFTl9HVUVTVF9IQU5ETEVfUEFSQU0oZXZ0Y2hu
X29wX3QpIHVvcCkKYXJjaC9hcm0vdHJhcHMuYzogSW4gZnVuY3Rpb24g4oCYZGVwX3BoeXNk
ZXZfb3BfY29tcGF04oCZOgphcmNoL2FybS90cmFwcy5jOjEzNTA6NjU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYdW9w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMzUwIHwg
bG9uZyBkZXBfcGh5c2Rldl9vcF9jb21wYXQoWEVOX0dVRVNUX0hBTkRMRV9QQVJBTShwaHlz
ZGV2X29wX3QpIHVvcCkKYXJjaC9hcm0vdHJhcHMuYzogSW4gZnVuY3Rpb24g4oCYZG9fdHJh
cF9oeXBlcmNhbGzigJk6CmFyY2gvYXJtL3RyYXBzLmM6MTQyMzo2Mzogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJh1aW50NjRfdOKAmSB7YWthIOKAmGxvbmcgdW5zaWduZWQgaW504oCZfSBhbmQg4oCY
aW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE0MjMgfCAgICAgaWYgKCAhY3Vyci0+aGNhbGxf
cHJlZW1wdGVkICYmIEhZUEVSQ0FMTF9SRVNVTFRfUkVHKHJlZ3MpICE9IC1FTk9TWVMgKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXn4KYXJjaC9hcm0vdHJhcHMuYzogSW4gZnVuY3Rpb24g4oCYaGFu
ZGxlX3dvX3dp4oCZOgphcmNoL2FybS90cmFwcy5jOjE2NTk6MjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYcmVnaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxNjU5IHwg
ICAgICAgICAgICAgICAgICAgaW50IHJlZ2lkeCwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICB+fn5+Xn5+fn5+CmFyY2gvYXJtL3RyYXBzLmM6IEluIGZ1bmN0aW9uIOKAmGdldF9oZmFy
4oCZOgphcmNoL2FybS90cmFwcy5jOjE3MTM6Mzc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaXNfZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTcxMyB8IHN0YXRpYyBp
bmxpbmUgdmFkZHJfdCBnZXRfaGZhcihib29sIGlzX2RhdGEpCiAgQ0MgICAgICBhcmNoL2Fy
bS92Z2ljL3ZnaWMubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92Z2ljL3ZnaWMuYzoy
MDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKA
mGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5n
cyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5j
dGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hW
TV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNM
QVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAg
fCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2
OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwg
REVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAg
ICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2lu
Y2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tf
bGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21w
YXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6
IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAy
MzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAg
ICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzog
Y29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRu
ZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVd
CiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAg
ICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1
bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUy
IHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3Qg
Km1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1
bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNs
dWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZu
X3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0
YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9k
ZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFk
aXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKA
mHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xv
Y2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2so
cmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUg
KnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
dm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hl
Y2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3Ay
bS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0
OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwg
dWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVk
IGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDi
gJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVu
dF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9i
bG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9t
ZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4g
ZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGlu
dCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2Rl
dmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0
aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rp
b24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2ku
aDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lf
c2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2Jk
Zl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNz
X3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3Rh
dGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1
Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQu
aDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVu
L3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBhcmNoL2FybS92Z2ljL3ZnaWMuYzoy
MToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6NTA6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNfZ2xvYmFs
KHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy5jOiBJ
biBmdW5jdGlvbiDigJh2Z2ljX2lycV9jbXDigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy5jOjIx
MjozMTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwcml24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjEyIHwgc3RhdGljIGludCB2Z2ljX2lycV9jbXAodm9pZCAqcHJpdiwg
c3RydWN0IGxpc3RfaGVhZCAqYSwgc3RydWN0IGxpc3RfaGVhZCAqYikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy5jOiBJ
biBmdW5jdGlvbiDigJh2Z2ljX3F1ZXVlX2lycV91bmxvY2vigJk6CmFyY2gvYXJtL3ZnaWMv
dmdpYy5jOjI4MTo0Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjgxIHwgdm9pZCB2Z2ljX3F1ZXVlX2lycV91bmxvY2soc3Ry
dWN0IGRvbWFpbiAqZCwgc3RydWN0IHZnaWNfaXJxICppcnEsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgphcmNoL2FybS92Z2ljL3ZnaWMu
YzogSW4gZnVuY3Rpb24g4oCYdmdpY19mbHVzaF9scl9zdGF0ZeKAmToKYXJjaC9hcm0vdmdp
Yy92Z2ljLmM6NTY2OjM4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVz
c2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA1NjYgfCAgICAgaWYgKCBjb21wdXRlX2Fw
X2xpc3RfZGVwdGgodmNwdSkgPiBnaWNfZ2V0X25yX2xycygpICkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS92Z2ljL3ZnaWMuYzo1
Nzg6MjA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDU3OCB8ICAgICAgICAgaWYgKCBjb3VudCA9PSBnaWNfZ2V0
X25yX2xycygpICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgXn4KYXJjaC9hcm0vdmdp
Yy92Z2ljLmM6IEluIGZ1bmN0aW9uIOKAmHZnaWNfYWxsb2NhdGVfdmlyceKAmToKYXJjaC9h
cm0vdmdpYy92Z2ljLmM6NzQ1OjE5OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHVuc2lnbmVkIGludOKA
mSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29tcGFyZV0KICA3NDUgfCAgICAgICAgIGlmICgg
dmlycSA+PSBlbmQgKQogICAgICB8ICAgICAgICAgICAgICAgICAgIF5+CmFyY2gvYXJtL3Zn
aWMvdmdpYy5jOiBJbiBmdW5jdGlvbiDigJh2Z2ljX2NsZWFyX3BlbmRpbmdfaXJxc+KAmToK
YXJjaC9hcm0vdmdpYy92Z2ljLmM6NzgyOjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3ODIgfCB2b2lkIHZnaWNfY2xlYXJf
cGVuZGluZ19pcnFzKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vdmdpYy92Z2ljLmM6IEluIGZ1
bmN0aW9uIOKAmHZnaWNfZW11bGF0ZeKAmToKYXJjaC9hcm0vdmdpYy92Z2ljLmM6ODUyOjQx
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ3PigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA4NTIgfCBib29sIHZnaWNfZW11bGF0ZShzdHJ1Y3QgY3B1X3VzZXJfcmVncyAq
cmVncywgdW5pb24gaHNyIGhzcikKICAgICAgfCAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS92Z2ljL3ZnaWMuYzo4NTI6NTc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaHNy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
ODUyIHwgYm9vbCB2Z2ljX2VtdWxhdGUoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHVu
aW9uIGhzciBoc3IpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5efn4KYXJjaC9hcm0vdmdpYy92Z2ljLmM6IEluIGZ1
bmN0aW9uIOKAmHZnaWNfY29ubmVjdF9od19pcnHigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy5j
Ojg5MzozNTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1aW50MzJfdOKAmSB7YWthIOKAmHVuc2lnbmVk
IGludOKAmX0gYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgODkzIHwgICAgICAg
ICBpZiAoIGRlc2MgJiYgaXJxLT5od2ludGlkICE9IGRlc2MtPmlycSApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuLy4vaW5jbHVkZS94ZW4vY29uZmlnLmg6MTcsCiAgICAgICAgICAgICAgICAgZnJvbSA8
Y29tbWFuZC1saW5lPjoKYXJjaC9hcm0vdmdpYy92Z2ljLmM6IEluIGZ1bmN0aW9uIOKAmHZn
aWNfc3luY19oYXJkd2FyZV9pcnHigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy5jOjkyNzozMzog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVpbnQzMl904oCZIHtha2Eg4oCYdW5z
aWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAgOTI3IHwgICAgIEFTU0VSVChpcnEt
Pmh3ICYmIGRlc2MtPmlycSA9PSBpcnEtPmh3aW50aWQpOwogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQz
OiBub3RlOiBpbiBkZWZpbml0aW9uIG9mIG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwg
I2RlZmluZSB1bmxpa2VseSh4KSAgIF9fYnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmFyY2gvYXJt
L3ZnaWMvdmdpYy5jOjkyNzo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYQVNT
RVJU4oCZCiAgOTI3IHwgICAgIEFTU0VSVChpcnEtPmh3ICYmIGRlc2MtPmlycSA9PSBpcnEt
Pmh3aW50aWQpOwogICAgICB8ICAgICBefn5+fn4KICBDQyAgICAgIGFyY2gvYXJtL3ZtX2V2
ZW50Lm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0
ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5s
b2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2
LAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vdm1fZXZlbnQuYzoxMDoKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9z
aWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25l
ZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGlj
L2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9z
eXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3Bpbmxv
Y2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9T
QVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+
fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMj
X3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24g
b2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9I
Vk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+
fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3J3bG9jay5o
OjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NzoKLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfbGFzdOKAmToK
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMzAgfCAgICAg
ICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAgICAgICAgICAgIF4K
Li9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1hc2tfY3ljbGXi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjQwIHwg
ICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAgICAgICBefgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2NfY3B1bWFza192YXLi
gJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzUyIHwgc3RhdGlj
IGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgKm1hc2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGZyZWVf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2NTo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIoY3B1bWFza192YXJf
dCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9k
b21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKA
mToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxp
bmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJt
NjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhucuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNf
aW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVu
Y3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVh
ZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0
IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9s
aXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBw
YWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0
X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4
OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0
bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDox
MTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihj
cHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAx
MSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNr
LCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+
fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5o
OjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFk
X2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8
IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9j
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhy
Y3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRf
bG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1
ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNj
ZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBi
b29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRf
c3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlu
bGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2
Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3Qg
aWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2
MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2Zu
LCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1f
ZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQx
Nzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVf
Z3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVt
b3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9s
aWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hl
bi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fz
c2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToK
Li9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0
YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWdu
ZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVu
Y3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGlu
ZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAq
dikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVu
Y3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJp
dF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvZ2VuZXJhdGVkL2Fz
bS92bV9ldmVudC5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Zt
X2V2ZW50Lmg6MjgsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92bV9ldmVudC5j
OjExOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCY
dm1fZXZlbnRfaW5pdF9kb21haW7igJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVu
dC5oOjE0OjU1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTQgfCBzdGF0aWMgaW5saW5lIGludCB2bV9ldmVudF9pbml0X2Rv
bWFpbihzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmlj
L3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVw
4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDoyNTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI1IHwg
c3RhdGljIGlubGluZSB2b2lkIHZtX2V2ZW50X3RvZ2dsZV9zaW5nbGVzdGVwKHN0cnVjdCBk
b21haW4gKmQsIHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9h
c20tZ2VuZXJpYy92bV9ldmVudC5oOjI1Ojc4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjUgfCBzdGF0aWMgaW5saW5lIHZv
aWQgdm1fZXZlbnRfdG9nZ2xlX3NpbmdsZXN0ZXAoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0
IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9h
c20tZ2VuZXJpYy92bV9ldmVudC5oOjI2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNiB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2bV9ldmVudF9yZXNwb25zZV90ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1f
ZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfcmVnaXN0ZXJfd3JpdGVfcmVzdW1l
4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1fZXZlbnQuaDozMjo1MDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMyIHwg
dm9pZCB2bV9ldmVudF9yZWdpc3Rlcl93cml0ZV9yZXN1bWUoc3RydWN0IHZjcHUgKnYsIHZt
X2V2ZW50X3Jlc3BvbnNlX3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvdm1f
ZXZlbnQuaDozMjo3NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMzIgfCB2b2lkIHZtX2V2ZW50X3JlZ2lzdGVyX3dyaXRl
X3Jlc3VtZShzdHJ1Y3QgdmNwdSAqdiwgdm1fZXZlbnRfcmVzcG9uc2VfdCAqcnNwKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2
ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X2VtdWxhdGVfY2hlY2vigJk6Ci4vaW5j
bHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjM4OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzggfCB2b2lkIHZtX2V2
ZW50X2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsIHZtX2V2ZW50X3Jlc3BvbnNlX3Qg
KnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6Mzg6NjY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM4IHwg
dm9pZCB2bV9ldmVudF9lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB2bV9ldmVudF9y
ZXNwb25zZV90ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL3ZtX2V2ZW50Lmg6IEluIGZ1bmN0aW9uIOKAmHZtX2V2ZW50X3N5bmNfZXZlbnTi
gJk6Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjQ0OjM5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNDQgfCB2
b2lkIHZtX2V2ZW50X3N5bmNfZXZlbnQoc3RydWN0IHZjcHUgKnYsIGJvb2wgdmFsdWUpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVk
ZS9hc20tZ2VuZXJpYy92bV9ldmVudC5oOjQ0OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHZhbHVl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDQ0IHwgdm9pZCB2bV9l
dmVudF9zeW5jX2V2ZW50KHN0cnVjdCB2Y3B1ICp2LCBib29sIHZhbHVlKQouL2luY2x1ZGUv
YXNtLWdlbmVyaWMvdm1fZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfcmVzZXRf
dm10cmFjZeKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL3ZtX2V2ZW50Lmg6NTA6NDI6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1MCB8IHZvaWQgdm1fZXZlbnRfcmVzZXRfdm10cmFjZShzdHJ1Y3QgdmNwdSAqdikKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KYXJjaC9h
cm0vdm1fZXZlbnQuYzogSW4gZnVuY3Rpb24g4oCYdm1fZXZlbnRfbW9uaXRvcl9uZXh0X2lu
dGVycnVwdOKAmToKYXJjaC9hcm0vdm1fZXZlbnQuYzozNDo1MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDM0IHwgdm9pZCB2
bV9ldmVudF9tb25pdG9yX25leHRfaW50ZXJydXB0KHN0cnVjdCB2Y3B1ICp2KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgog
IENDICAgICAgYXJjaC9hcm0vdnRpbWVyLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjbGludXguaDo1NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9hY3BpL3BsYXRmb3JtL2FjZW52Lmg6MTQyLAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL2FjcGkvYWNwaS5oOjU2LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9hY3BpLmg6NTcsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNo
L2FybS92dGltZXIuYzoxMToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6
IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50
IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMg
aW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJf
dCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6
IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXi
gJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3Nh
dmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQg
X19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4
NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9U
WVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2
bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3BlcmZjLmg6NywKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDo3MCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc2V0dXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9hY3BpLmg6MjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
YWNwaS9wbGF0Zm9ybS9hY2xpbnV4Lmg6NTg6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9p
ZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9p
ZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IElu
IGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVt
YXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1
ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
bWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9p
ZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzM6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVj
dG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVk
ZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVk
ZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBm
dW5jdGlvbiDigJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0
NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2Vf
bGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBh
Z2VfbGlzdF9kZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4g
ZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdl
LCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAg
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5o
OiBJbiBmdW5jdGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJht
YXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lk
IHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0
YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0
ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1
bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDox
MjA6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDEyMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJj
dV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVh
ZF91bmxvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNz
X2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMjMgfCBib29sIHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3Ry
dWN0IHZjcHUgKnYsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICAyNCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJzcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6MzE6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogICAzMSB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19z
YW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTc0IHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQs
IHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAy
bV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Rh
dGVfbGltaXTigJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5Mjo1NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
OTIgfCBzdGF0aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N0YXRlX2xpbWl0KHVuc2lnbmVk
IGludCBuZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+Ci4vaW5jbHVk
ZS94ZW4vYWNwaS5oOiBJbiBmdW5jdGlvbiDigJhhY3BpX3NldF9jc3Vic3RhdGVfbGltaXTi
gJk6Ci4vaW5jbHVkZS94ZW4vYWNwaS5oOjE5NDo1ODogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhuZXdfbGltaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTQgfCBzdGF0
aWMgaW5saW5lIHZvaWQgYWNwaV9zZXRfY3N1YnN0YXRlX2xpbWl0KHVuc2lnbmVkIGludCBu
ZXdfbGltaXQpIHsgcmV0dXJuOyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEsCiAgICAgICAgICAgICAgICAgZnJvbSBh
cmNoL2FybS92dGltZXIuYzoxNDoKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9u
IOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0
YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3Rf
bWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlf
cG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNp
X2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsg
fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9yZWFk4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8
IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5z
aWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4
NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwg
dW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyOTQgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI5NTo0NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRp
bmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlu
bGluZSBib29sIF9fbXVzdF9jaGVjayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNw
dSAqdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4g
ZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRfZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MTEzMTo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8z
MmJpdF9kb21haW4oY29uc3Qgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5eCiAgQ0MgICAgICBhcmNoL2FybS92Z2ljL3ZnaWMtdjIubwpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3VycmVudC5oOjgs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUvYXNtLWdlbmVyaWMvcGVyY3B1Lmg6
OCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9nZW5lcmF0ZWQv
YXNtL3BlcmNwdS5oOjEsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Bl
cmNwdS5oOjMwLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2su
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbWlv
Lmg6MjMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL25l
d192Z2ljLmg6MjAsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92Z2ljL3ZnaWMt
djIuYzoxODoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0
aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVm
ZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9z
aWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3J3bG9jay5oOjY6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNv
bXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwg
ICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwg
ICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKA
mGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVt
YXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJ
biBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1h
c2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1h
c2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzLAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9yd2xvY2suaDo3OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IElu
IGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4g
ZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8
IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwog
ICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZF
X0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAg
ODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5k
KTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL25ld192Z2ljLmg6MjI6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
YXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcg
bWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhu
cuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBh
cmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25n
IG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTE6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfcHJlduKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDU3IHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzi
gJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9k
ZWwoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X2RlbDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NzcgfCBwYWdlX2xpc3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOjQ3ODozOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBw
YWdlX2xpc3RfaGVhZCAqaGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5efn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tbS5oOjU4OToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5j
dGlvbiDigJh0bGJmbHVzaF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDoxMTo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNo
X2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vaXJx
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZ2lj
Lmg6MTYzLAogICAgICAgICAgICAgICAgIGZyb20gYXJjaC9hcm0vdmdpYy92Z2ljLXYyLmM6
MTk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRf
bG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwg
c3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2Nr
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJj
dV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9s
b2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVk
ZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMSwKICAgICAgICAgICAgICAgICBmcm9tIGFyY2gvYXJtL3ZnaWMvdmdpYy12
Mi5jOjIxOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25p
ZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBp
bmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjcsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCB2
bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDozMTo2Mzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVj
ayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBz
dGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNrKHN0cnVjdCBk
b21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaWR44oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1
aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVu
Y3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKA
mHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Js
b2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3
IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikg
e30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKA
mHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9n
dWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21l
bW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2
aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3Nf
cGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVj
dCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KYXJjaC9hcm0vdmdpYy92Z2ljLXYyLmM6IEluIGZ1bmN0aW9uIOKAmHZnaWNf
djJfcG9wdWxhdGVfbHLigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy12Mi5jOjE5MjozOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2Y3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTkyIHwgdm9pZCB2Z2ljX3YyX3BvcHVsYXRlX2xyKHN0cnVjdCB2Y3B1ICp2Y3B1LCBz
dHJ1Y3QgdmdpY19pcnEgKmlycSwgaW50IGxyKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgphcmNoL2FybS92Z2ljL3ZnaWMtdjIuYzogSW4g
ZnVuY3Rpb24g4oCYdmdpY192Ml9lbmFibGXigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy12Mi5j
OjI1MjozNDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2Y3B14oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjUyIHwgdm9pZCB2Z2ljX3YyX2VuYWJsZShzdHJ1Y3QgdmNwdSAq
dmNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CiAg
Q0MgICAgICBhcmNoL2FybS92cGwwMTEubwpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL2V2ZW50Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92cGwwMTEu
YzoxNDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9u
IOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0
dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJs
aW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUvcHVibGljL2RvbWN0bC5oOjIxLAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBm
dW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJWF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6NTY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBf
X0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fl4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9G
SVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBp
biBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2
IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7
CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3NtcC5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NtcC5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL3J3bG9jay5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3Nj
aGVkLmg6NzoKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNwdW1h
c2tfbGFzdOKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjMwOjE4OiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICAyMzAgfCAgICAgICAgICAgICAgY3B1IDwgbnJfY3B1X2lkczsKICAgICAgfCAgICAgICAg
ICAgICAgICAgIF4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGNw
dW1hc2tfY3ljbGXigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjI0MDoxMzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjQwIHwgICAgIGlmIChueHQgPT0gbnJfY3B1X2lkcykKICAgICAgfCAgICAgICAg
ICAgICBefgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYYWxsb2Nf
Y3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM1Mjo1Mzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzUyIHwgc3RhdGljIGlubGluZSBib29sIGFsbG9jX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgKm1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGZyZWVfY3B1bWFza192YXLigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjM2
NTo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzY1IHwgc3RhdGljIGlubGluZSB2b2lkIGZyZWVfY3B1bWFza192YXIo
Y3B1bWFza192YXJfdCBtYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCY
bWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8
IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CiAgQ0MgICAgICBhcmNo
L2FybS92Z2ljL3ZnaWMtbW1pby5vCi4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g
4oCYcGFnZV9saXN0X25leHTigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTE6NDU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQ1MSB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVh
ZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfcHJl
duKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDU3IHwgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWzigJk6Ci4vaW5jbHVkZS94
ZW4vbW0uaDo0NzI6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3MiB8IHBhZ2VfbGlzdF9kZWwoc3RydWN0IHBhZ2Vf
aW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2Rl
bDLigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzc6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzcgfCBwYWdlX2xp
c3RfZGVsMihzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQxLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOjQ3ODoz
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDQ3OCB8ICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAq
aGVhZDIpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5e
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjU4OToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOiBJbiBmdW5jdGlvbiDigJh0bGJmbHVz
aF9maWx0ZXLigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo0Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtYXNr4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNr
X3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZmx1c2h0bGIuaDoxMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwYWdlX3RpbWVzdGFtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50
MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1Ogou
L2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vi
gJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMCB8IHN0YXRp
YyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVh
ZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEz
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVu
L21lbV9hY2Nlc3MuaDozNywKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6NzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6
IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2vigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjIzOjQ4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjMgfCBib29sIHAy
bV9tZW1fYWNjZXNzX2VtdWxhdGVfY2hlY2soc3RydWN0IHZjcHUgKnYsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjI0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHJzcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNCB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qgdm1fZXZlbnRfc3QgKnJz
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNz
Lmg6IEluIGZ1bmN0aW9uIOKAmHAybV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVja+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MzE6NjM6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFp
biAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYYXJjaF9hY3F1aXJlX3Jlc291cmNlX2NoZWNr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL3AybS5oOjE3NDo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTc0IHwgc3RhdGljIGlubGluZSBi
b29sIGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlv
biDigJhwMm1fYWx0cDJtX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OjE4NDozNjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjQ4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGlkeOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8
IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGdldF9w
YWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjM2MTo2OTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBwMm1f
dHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+XgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDE3IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2so
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3Rf
bWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6
NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3Bv
bGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpn
bXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IElu
IGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBm
cm9tIGFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLmM6MTc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBz
dGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+
fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3Rh
dGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90
IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNp
X3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vz
c19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0
YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3Ry
dWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgogIENDICAgICAgYXJjaC9hcm0v
dnNtYy5vCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJi
aXRfZG9tYWlu4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9k
b21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lz
Y3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2Nr
Lmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19I
Vk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
OjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhf
Q09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FW
RV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+
fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194
KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9m
IG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZN
X1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+
fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMx
IHwgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qg
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL2V2ZW50Lmg6MTc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2dsb2JhbOKAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBpbnQgdmlycSkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9hcm0vdnBsMDExLmM6MjE6Ci4v
aW5jbHVkZS94ZW4vc2VyaWFsLmg6IEF0IHRvcCBsZXZlbDoKLi9pbmNsdWRlL3hlbi9zZXJp
YWwuaDoxNjY6MTogd2FybmluZzog4oCYaW5saW5l4oCZIGlzIG5vdCBhdCBiZWdpbm5pbmcg
b2YgZGVjbGFyYXRpb24gWy1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9uXQogIDE2NiB8IHN0YXRp
YyB2b2lkIGlubGluZSB4aGNpX2RiY191YXJ0X2luaXQodm9pZCkge30KICAgICAgfCBefn5+
fn4KYXJjaC9hcm0vdnBsMDExLmM6IEluIGZ1bmN0aW9uIOKAmHZwbDAxMV9tbWlvX3JlYWTi
gJk6CmFyY2gvYXJtL3ZwbDAxMS5jOjM0NjozNTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhwcml24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzQ2IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHZvaWQgKnByaXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl5+fn4KYXJjaC9hcm0vdnBsMDExLmM6IEluIGZ1bmN0aW9uIOKA
mHZwbDAxMV9tbWlvX3dyaXRl4oCZOgphcmNoL2FybS92cGwwMTEuYzo0MzU6MzY6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcHJpduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDQzNSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdm9pZCAqcHJpdikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fn4KYXJjaC9hcm0vdnBs
MDExLmM6IEluIGZ1bmN0aW9uIOKAmHZwbDAxMV9ub3RpZmljYXRpb27igJk6CmFyY2gvYXJt
L3ZwbDAxMS5jOjYwMTo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwb3J04oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNjAxIHwgc3RhdGljIHZvaWQgdnBsMDExX25vdGlm
aWNhdGlvbihzdHJ1Y3QgdmNwdSAqdiwgdW5zaWduZWQgaW50IHBvcnQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
c21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBj
cHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUv
eGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54
dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4v
Y3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNs
dWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0
aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWVycmF0YS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSBhcmNoL2FybS92c21jLmM6MTM6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhj
cHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3Mo
dW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3Rv
X25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRp
YyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDMsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbW9uaXRvci5oOjI1LAogICAgICAgICAgICAgICAgIGZyb20gYXJj
aC9hcm0vdnNtYy5jOjE1OgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6IEluIGZ1bmN0
aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJho4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZN
X1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xB
UkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6
Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9D
T01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBE
RUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAg
ICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3By
ZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9s
aXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6
Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1
c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRf
cGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJt
X3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhj
cHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWdu
ZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBh
cmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAg
ICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDi
gJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAg
ICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFs
bG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNr
X3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2su
aDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tf
dmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGlu
ZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9t
ZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2
MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9s
aWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdt
cCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rp
b24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
cGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50
IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2
aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRp
YyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRl
dikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlv
biDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRm
LCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5o
OiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
OTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0
OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRm
X3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3Nf
cGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVj
dCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5o
OiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4v
c2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlz
X3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQs
CiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0u
aDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAg
fCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBs
b25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5efgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOiBJbiBmdW5jdGlvbiDigJh2Z2ljX21t
aW9fcmVhZF9yYXrigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLmM6MjQ6NDc6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAyNCB8IHVuc2lnbmVkIGxvbmcgdmdpY19tbWlvX3JlYWRfcmF6KHN0cnVjdCB2Y3B1ICp2
Y3B1LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5efn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLmM6MjU6NDI6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYYWRkcuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhZGRyX3QgYWRkciwgdW5zaWdu
ZWQgaW50IGxlbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fl5+fn4KYXJjaC9hcm0vdmdpYy92Z2ljLW1taW8uYzoyNTo2MTogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhsZW7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IGFkZHIsIHVuc2ln
bmVkIGludCBsZW4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlv
LmM6IEluIGZ1bmN0aW9uIOKAmHZnaWNfbW1pb19yZWFkX3Jhb+KAmToKYXJjaC9hcm0vdmdp
Yy92Z2ljLW1taW8uYzozMDo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2Y3B1
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMwIHwgdW5zaWduZWQgbG9uZyB2Z2ljX21t
aW9fcmVhZF9yYW8oc3RydWN0IHZjcHUgKnZjcHUsCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vdmdpYy92Z2lj
LW1taW8uYzozMTo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhhZGRy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcGFkZHJfdCBhZGRyLCB1bnNpZ25lZCBpbnQgbGVuKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+Xn5+fgphcmNoL2FybS92Z2ljL3Zn
aWMtbW1pby5jOjMxOjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxlbuKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAzMSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHBhZGRyX3QgYWRkciwgdW5zaWduZWQgaW50IGxlbikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn4KYXJjaC9hcm0vdmdpYy92Z2ljLW1taW8uYzogSW4gZnVuY3Rpb24g4oCYdmdpY19tbWlv
X3dyaXRlX3dp4oCZOgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOjM2OjM4OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MzYgfCB2b2lkIHZnaWNfbW1pb193cml0ZV93aShzdHJ1Y3QgdmNwdSAqdmNwdSwgcGFkZHJf
dCBhZGRyLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLmM6MzY6NTI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYYWRkcuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNiB8IHZvaWQg
dmdpY19tbWlvX3dyaXRlX3dpKHN0cnVjdCB2Y3B1ICp2Y3B1LCBwYWRkcl90IGFkZHIsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+Xn5+fgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOjM3OjM4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGxlbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNyB8ICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBsZW4sIHVuc2lnbmVkIGxvbmcg
dmFsKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4K
YXJjaC9hcm0vdmdpYy92Z2ljLW1taW8uYzozNzo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJh2YWzigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzcgfCAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgbGVuLCB1bnNpZ25lZCBsb25nIHZhbCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5j
bHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVh
ZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcy
OjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2Us
IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3Ry
dWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1
aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90
aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90
aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5j
bHVkZWQgZnJvbSAuLy4vaW5jbHVkZS94ZW4vY29uZmlnLmg6MTcsCiAgICAgICAgICAgICAg
ICAgZnJvbSA8Y29tbWFuZC1saW5lPjoKYXJjaC9hcm0vdmdpYy92Z2ljLW1taW8uYzogSW4g
ZnVuY3Rpb24g4oCYdmdpY19tbWlvX3dyaXRlX3NwZW5kaW5n4oCZOgphcmNoL2FybS92Z2lj
L3ZnaWMtbW1pby5jOjIxNjo0MTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4
cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVp
bnQzMl904oCZIHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAg
MjE2IHwgICAgICAgICAgICAgQVNTRVJUKGlycS0+aHcgJiYgZGVzYy0+aXJxID09IGlycS0+
aHdpbnRpZCk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS94ZW4vY29tcGlsZXIuaDoyMTo0Mzogbm90ZTogaW4gZGVmaW5p
dGlvbiBvZiBtYWNybyDigJh1bmxpa2VseeKAmQogICAyMSB8ICNkZWZpbmUgdW5saWtlbHko
eCkgICBfX2J1aWx0aW5fZXhwZWN0KCEhKHgpLDApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5j
OjIxNjoxMzogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKAmQogIDIx
NiB8ICAgICAgICAgICAgIEFTU0VSVChpcnEtPmh3ICYmIGRlc2MtPmlycSA9PSBpcnEtPmh3
aW50aWQpOwogICAgICB8ICAgICAgICAgICAgIF5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
YXJjaC9hcm0vdmdpYy92Z2ljLW1taW8uYzogSW4gZnVuY3Rpb24g4oCYdmdpY19tbWlvX3dy
aXRlX2NwZW5kaW5n4oCZOgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOjI2OTo0MTogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBz
aWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVpbnQzMl904oCZIHtha2Eg4oCYdW5zaWdu
ZWQgaW504oCZfSBbLVdzaWduLWNvbXBhcmVdCiAgMjY5IHwgICAgICAgICAgICAgQVNTRVJU
KGlycS0+aHcgJiYgZGVzYy0+aXJxID09IGlycS0+aHdpbnRpZCk7CiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY29t
cGlsZXIuaDoyMTo0Mzogbm90ZTogaW4gZGVmaW5pdGlvbiBvZiBtYWNybyDigJh1bmxpa2Vs
eeKAmQogICAyMSB8ICNkZWZpbmUgdW5saWtlbHkoeCkgICBfX2J1aWx0aW5fZXhwZWN0KCEh
KHgpLDApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgXgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOjI2OToxMzogbm90ZTogaW4gZXhwYW5z
aW9uIG9mIG1hY3JvIOKAmEFTU0VSVOKAmQogIDI2OSB8ICAgICAgICAgICAgIEFTU0VSVChp
cnEtPmh3ICYmIGRlc2MtPmlycSA9PSBpcnEtPmh3aW50aWQpOwogICAgICB8ICAgICAgICAg
ICAgIF5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNj
ZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5j
dGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBm
dW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5l
IGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJt
X2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2
X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAy
bV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgphcmNo
L2FybS92Z2ljL3ZnaWMtbW1pby5jOiBJbiBmdW5jdGlvbiDigJh2Z2ljX21taW9fcmVhZF9j
b25maWfigJk6CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLmM6NDMzOjIwOiB3YXJuaW5nOiBj
b21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5l
c3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0K
ICA0MzMgfCAgICAgZm9yICggaSA9IDA7IGkgPCBsZW4gKiA0OyBpKysgKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICBeCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1
bmN0aW9uIOKAmGdldF9wYWdlX2Zyb21fZ2Zu4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOjM2MTo2OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzYxIHwgICAgIHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVk
IGxvbmcgZ2ZuLCBwMm1fdHlwZV90ICp0LCBwMm1fcXVlcnlfdCBxKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+XgphcmNoL2FybS92Z2ljL3ZnaWMtbW1pby5jOiBJbiBmdW5jdGlvbiDigJh2
Z2ljX21taW9fd3JpdGVfY29uZmln4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr4oCZOgphcmNoL2Fy
bS92Z2ljL3ZnaWMtbW1pby5jOjQ1NDoyMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNDU0IHwgICAgIGZvciAo
IGkgPSAwOyBpIDwgbGVuICogNDsgaSsrICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3IHwgc3RhdGlj
IGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3Zj
cHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0
YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNw
dSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoz
MTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBj
aV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODgg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVp
bnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2
cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRf
ZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3Rh
dGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21vbml0b3IuaDozMDoKLi9pbmNsdWRlL2FzbS1n
ZW5lcmljL21vbml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2FsbG93X3Vz
ZXJzcGFjZeKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0b3IuaDoyMjo1MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIyIHwgdm9pZCBhcmNoX21vbml0b3JfYWxsb3dfdXNlcnNwYWNlKHN0cnVjdCBkb21haW4g
KmQsIGJvb2wgYWxsb3dfdXNlcnNwYWNlKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vaW5jbHVkZS9hc20tZ2VuZXJpYy9t
b25pdG9yLmg6MjI6NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYYWxsb3dfdXNl
cnNwYWNl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIyIHwgdm9pZCBhcmNoX21vbml0
b3JfYWxsb3dfdXNlcnNwYWNlKHN0cnVjdCBkb21haW4gKmQsIGJvb2wgYWxsb3dfdXNlcnNw
YWNlKQouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX21vbml0b3JfZG9tY3RsX29w4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRv
ci5oOjI3OjQzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjcgfCBpbnQgYXJjaF9tb25pdG9yX2RvbWN0bF9vcChzdHJ1Y3Qg
ZG9tYWluICpkLCBzdHJ1Y3QgeGVuX2RvbWN0bF9tb25pdG9yX29wICptb3ApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUv
YXNtLWdlbmVyaWMvbW9uaXRvci5oOjI3Ojc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmG1vcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAyNyB8IGludCBhcmNoX21vbml0
b3JfZG9tY3RsX29wKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX21vbml0
b3Jfb3AgKm1vcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L2FzbS1nZW5lcmljL21vbml0b3IuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tb25pdG9yX2lu
aXRfZG9tYWlu4oCZOgouL2luY2x1ZGUvYXNtLWdlbmVyaWMvbW9uaXRvci5oOjQzOjQ1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgNDMgfCBpbnQgYXJjaF9tb25pdG9yX2luaXRfZG9tYWluKHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vaW5jbHVkZS9hc20tZ2VuZXJpYy9tb25pdG9yLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hf
bW9uaXRvcl9jbGVhbnVwX2RvbWFpbuKAmToKLi9pbmNsdWRlL2FzbS1nZW5lcmljL21vbml0
b3IuaDo1MDo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDUwIHwgdm9pZCBhcmNoX21vbml0b3JfY2xlYW51cF9kb21haW4o
c3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW9uaXRvci5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX21vbml0b3JfZ2V0X2NhcGFiaWxpdGllc+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9tb25pdG9yLmg6MzU6Njk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAzNSB8IHN0YXRpYyBp
bmxpbmUgdWludDMyX3QgYXJjaF9tb25pdG9yX2dldF9jYXBhYmlsaXRpZXMoc3RydWN0IGRv
bWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vdnNtYy5jOiBJbiBm
dW5jdGlvbiDigJhpc19wc2NpX2ZpZOKAmToKYXJjaC9hcm0vdnNtYy5jOjE5NjoxNTogd2Fy
bmluZzogY29tcGFyaXNvbiBvZiB1bnNpZ25lZCBleHByZXNzaW9uIGluIOKAmD49IDDigJkg
aXMgYWx3YXlzIHRydWUgWy1XdHlwZS1saW1pdHNdCiAgMTk2IHwgICAgIHJldHVybiBmbiA+
PSBQU0NJX0ZOVU1fTUlOX1ZBTFVFICYmIGZuIDw9IFBTQ0lfRk5VTV9NQVhfVkFMVUU7CiAg
ICAgIHwgICAgICAgICAgICAgICBefgphcmNoL2FybS92c21jLmM6IEluIGZ1bmN0aW9uIOKA
mHZzbWNjY19oYW5kbGVfY2FsbOKAmToKYXJjaC9hcm0vdnNtYy5jOjI3NDoxNzogd2Fybmlu
ZzogY29tcGFyaXNvbiBvZiB1bnNpZ25lZCBleHByZXNzaW9uIGluIOKAmD49IDDigJkgaXMg
YWx3YXlzIHRydWUgWy1XdHlwZS1saW1pdHNdCiAgMjc0IHwgICAgIGlmICggZnVuY2lkID49
IEFSTV9TTUNDQ19SRVNFUlZFRF9SQU5HRV9TVEFSVCAmJgogICAgICB8ICAgICAgICAgICAg
ICAgICBefgogIENDICAgICAgYXJjaC9hcm0vdnBzY2kubwpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBh
cmNoL2FybS92cHNjaS5jOjM6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5o
OiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGlu
dCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAg
ZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZt
L3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToK
Li9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGlj
IGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMy
X3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4Njox
OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF
4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9z
YXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJs
aWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50
IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9p
bmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDox
ODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAg
ICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoy
NDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAg
ICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0
aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoz
NTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFz
ay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVu
L2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVl
X2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IElu
IGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3Qg
bWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5e
fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDox
MiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19p
bl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1
Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1h
cCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1m
biwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0u
aDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDi
gJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0
NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21t
Lmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21t
Lmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5m
byAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAg
ICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1
c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdl
X3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6
MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1
c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBk
YXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRf
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1
cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUv
eGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxv
Y2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQg
cmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9h
Y2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1f
bWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVt
X2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9h
Y2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNx
dWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDox
NzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3Vy
Y2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQg
cDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHji
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9t
YWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3Qg
cSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlv
biDigJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWlu
Lmg6Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2Nr
KHN0cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBzdHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2ln
bl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwg
c3RhdGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCY
dnBjaV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0
cnVjdCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVh
ZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lf
cmVhZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5j
bHVkZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
YmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lk
IHZwY2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBk
YXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRh
dGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6
Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9j
ZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKA
mToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdh
eXNfaW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4g
KmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gYXJjaC9h
cm0vdnBzY2kuYzo5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50Lmg6IEluIGZ1bmN0
aW9uIOKAmGFyY2hfdmlycV9pc19nbG9iYWzigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZXZlbnQuaDo1MDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh2aXJx4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDUwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfdmly
cV9pc19nbG9iYWwodW5zaWduZWQgaW50IHZpcnEpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KYXJjaC9hcm0vdnBz
Y2kuYzogSW4gZnVuY3Rpb24g4oCYZG9fcHNjaV9jcHVfb2Zm4oCZOgphcmNoL2FybS92cHNj
aS5jOjEwMjo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwb3dlcl9zdGF0ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMiB8IHN0YXRpYyBpbnQzMl90IGRvX3BzY2lf
Y3B1X29mZih1aW50MzJfdCBwb3dlcl9zdGF0ZSkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+fn5+fn4KYXJjaC9hcm0vdnBzY2kuYzog
SW4gZnVuY3Rpb24g4oCYZG9fcHNjaV8wXzJfY3B1X3N1c3BlbmTigJk6CmFyY2gvYXJtL3Zw
c2NpLmM6MTE5OjUyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBvd2VyX3N0YXRl
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTE5IHwgc3RhdGljIHJlZ2lzdGVyX3QgZG9f
cHNjaV8wXzJfY3B1X3N1c3BlbmQodWludDMyX3QgcG93ZXJfc3RhdGUsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fn5+
fn5+fn4KYXJjaC9hcm0vdnBzY2kuYzoxMjA6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZW50cnlfcG9pbnTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByZWdpc3Rlcl90IGVudHJ5
X3BvaW50LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+Xn5+fn5+fn5+fn4KYXJjaC9hcm0vdnBzY2kuYzoxMjE6NTQ6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY29udGV4dF9pZOKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDEyMSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHJlZ2lzdGVyX3QgY29udGV4dF9pZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn5+fn5+fn4KYXJjaC9hcm0v
dnBzY2kuYzogSW4gZnVuY3Rpb24g4oCYZG9fdnBzY2lfMF8xX2NhbGzigJk6CmFyY2gvYXJt
L3Zwc2NpLmM6MjM3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGZpZOKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDIzNyB8IGJvb2wgZG9fdnBzY2lfMF8xX2NhbGwoc3Ry
dWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHVpbnQzMl90IGZpZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5e
fn4KICBDQyAgICAgIGNvbW1vbi9zY2hlZC9jcmVkaXQyLm8KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIGNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6MTU6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDog
SW4gZnVuY3Rpb24g4oCYbm9fY29uZmlnX3BhcmFt4oCZOgouL2luY2x1ZGUveGVuL3BhcmFt
Lmg6MTkzOjE5OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZIGNoYW5nZXMgc2lnbmVk
bmVzcyBmcm9tIOKAmGxvbmcgaW504oCZIHRvIOKAmHNpemVfdOKAmSB7YWthIOKAmGxvbmcg
dW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVyIG9wZXJhbmQg
Wy1Xc2lnbi1jb21wYXJlXQogIDE5MyB8ICAgICBpbnQgbGVuID0gZSA/ICh7IEFTU0VSVChl
ID49IHMpOyBlIC0gczsgfSkgOiBzdHJsZW4ocyk7CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAgICAg
ICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBjb21t
b24vc2NoZWQvY3JlZGl0Mi5jOjE2OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1
cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGljIGlubGlu
ZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGlj
L2h2bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTi
gJk6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0
YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVp
bnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6
ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVf
VFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBo
dm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUv
cHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
c2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5l
IGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXpl
KSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2
ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9T
QVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1
Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KSW4gZmls
ZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4g
ZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoy
MzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBk
aWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkg
Wy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRz
OwogICAgICB8ICAgICAgICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDog
SW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRz
KQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBm
dW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNr
Lmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFz
a192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1h
c2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQg
ZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUg
aW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjUsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MTE6Ci4vaW5jbHVkZS94ZW4vbnVtYS5o
OiBJbiBmdW5jdGlvbiDigJhtZm5fdG9fbmlk4oCZOgouL2luY2x1ZGUveGVuL251bWEuaDox
Mjk6NDE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMTI5IHwgc3RhdGljIGlubGluZSBub2RlaWRfdCBtZm5fdG9fbmlkKG1m
bl90IG1mbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21t
Lmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6MjMzLAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDo3
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDoxNToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOiBJbiBmdW5jdGlvbiDigJhhcmNoX21m
bnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJl
Y3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5e
fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwg
c3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9u
ZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9y
Y3VwZGF0ZS5oOjEyMDo1MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9y
ZWFkX2xvY2socmN1X3JlYWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNs
dWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2
b2lkIHJjdV9yZWFkX3VubG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3Ogou
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJt
X21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0
ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1f
YWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCY
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9t
ZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhhcmNo
X2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJt
Lmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJlX3Jl
c291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1fY2hl
Y2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2
b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
aWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJtX2No
ZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3RydWN0
IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9xdWVy
eV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KICBDQyAgICAgIGFyY2gvYXJtL3Z1YXJ0Lm8KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9kb21haW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3Rh
dGljIGlubGluZSB2b2lkIGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0
ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFp
bi5oOjMwOTo2MDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9t
ZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhnbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9w
b2xpY3kgKmdtcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lf
YXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX2RlYXNzaWduX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwg
c3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNp
Z25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3
IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1
bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyODg6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBzaXplKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDI5NCB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2Jk
ZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92
cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAyOTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGlu
Z+KAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5s
aW5lIGJvb2wgX19tdXN0X2NoZWNrIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1
ICp2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBm
dW5jdGlvbiDigJhpc19wdl8zMmJpdF9kb21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMy
Yml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZXZlbnQuaDoxNywKICAg
ICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6MTk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOjUwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
NTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2dsb2JhbCh1bnNpZ25lZCBp
bnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlv
biDigJhpc191bml0X29ubGluZeKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVf
aWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0
ZS5oOjkxOjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hl
ZF91bml0X3ZjcHXigJkKICAgOTEgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICgg
dW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24v
c2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5hYmxl4oCZOgouL2lu
Y2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0
X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTA4OjU6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxMDggfCAgICAg
Zm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlv
biDigJh1bml0X3J1bm5hYmxlX3N0YXRl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3
OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+
dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9w
cml2YXRlLmg6MTM1OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFj
aF9zY2hlZF91bml0X3ZjcHXigJkKICAxMzUgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92
Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpj
b21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRfcmVz4oCZ
OgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0
LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTU0OjU6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxNTQg
fCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAg
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBm
dW5jdGlvbiDigJhzY2hlZF9zZXRfcGF1c2VfZmxhZ3PigJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9u
L3NjaGVkL3ByaXZhdGUuaDoxNzY6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogIDE3NiB8ICAgICBmb3JfZWFjaF9zY2hl
ZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2Ns
ZWFyX3BhdXNlX2ZsYWdz4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8
ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
MTg2OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91
bml0X3ZjcHXigJkKICAxODYgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5p
dCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDox
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5o
OjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAg
ICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAg
ICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAg
ICAgZnJvbSBhcmNoL2FybS92dWFydC5jOjI1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Nw
dWZlYXR1cmUuaDogSW4gZnVuY3Rpb24g4oCYY3B1X25yX3NpYmxpbmdz4oCZOgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL2NwdWZlYXR1cmUuaDoxMDE6NDg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTAxIHwgc3RhdGlj
IGlubGluZSBpbnQgY3B1X25yX3NpYmxpbmdzKHVuc2lnbmVkIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn4KY29t
bW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYc2NoZWRfdW5pdF9wYXVzZV9u
b3N5bmPigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBh
cmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog
4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1
NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0
X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDo1MzA6NTogbm90
ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKA
mQogIDUzMCB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAg
ICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRl
Lmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX3VuaXRfdW5wYXVzZeKAmToKLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJl
c3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2ln
bmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAg
ICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpj
b21tb24vc2NoZWQvcHJpdmF0ZS5oOjUzODo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgNTM4IHwgICAgIGZvcl9lYWNo
X3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi9pbmNsdWRlL3hlbi9jb25m
aWcuaDoxNywKICAgICAgICAgICAgICAgICBmcm9tIDxjb21tYW5kLWxpbmU+Ogpjb21tb24v
c2NoZWQvY3JlZGl0Mi5jOiBJbiBmdW5jdGlvbiDigJhnZXRfZmFsbGJhY2tfY3B14oCZOgpj
b21tb24vc2NoZWQvY3JlZGl0Mi5jOjc5MDoyNTogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgNzkwIHwgICAgICAg
ICBpZiAoIGxpa2VseShjcHUgPCBucl9jcHVfaWRzKSApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NvbXBpbGVyLmg6MjA6NDM6IG5vdGU6IGlu
IGRlZmluaXRpb24gb2YgbWFjcm8g4oCYbGlrZWx54oCZCiAgIDIwIHwgI2RlZmluZSBsaWtl
bHkoeCkgICAgIF9fYnVpbHRpbl9leHBlY3QoISEoeCksMSkKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jcmVkaXQy
LmM6IEluIGZ1bmN0aW9uIOKAmHNhbWVfbm9kZeKAmToKY29tbW9uL3NjaGVkL2NyZWRpdDIu
Yzo4NDM6NDM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B1YeKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDg0MyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBzYW1lX25vZGUodW5z
aWduZWQgaW50IGNwdWEsIHVuc2lnbmVkIGludCBjcHViKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL2NyZWRp
dDIuYzo4NDM6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B1YuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDg0MyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBzYW1lX25vZGUo
dW5zaWduZWQgaW50IGNwdWEsIHVuc2lnbmVkIGludCBjcHViKQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKAmHNhbWVfc29ja2V0
4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOjg0ODo0NTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhjcHVh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgODQ4IHwgc3RhdGlj
IGlubGluZSBib29sIHNhbWVfc29ja2V0KHVuc2lnbmVkIGludCBjcHVhLCB1bnNpZ25lZCBp
bnQgY3B1YikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzo4NDg6NjQ6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYY3B1YuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDg0OCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBzYW1lX3NvY2tldCh1bnNpZ25lZCBpbnQgY3B1YSwgdW5z
aWduZWQgaW50IGNwdWIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgogIENDICAgICAgYXJjaC9h
cm0vdmdpYy92Z2ljLW1taW8tdjIubwpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOiBJbiBmdW5j
dGlvbiDigJhjcHVfYWRkX3RvX3J1bnF1ZXVl4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0Mi5j
Ojk4MDo1MTogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9m
IGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKA
mSBbLVdzaWduLWNvbXBhcmVdCiAgOTgwIHwgICAgICAgICAgICAgICAgIGlmICggY3B1bWFz
a193ZWlnaHQoJnJxZC0+YWN0aXZlKSA+PSBvcHRfbWF4X2NwdXNfcnVucXVldWUgKQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Xn4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX21heF93
ZWlnaHTigJk6CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6MTA2MzoyMTogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAx
MDYzIHwgICAgIGlmICggbmV3X3dlaWdodCA+IHJxZC0+bWF4X3dlaWdodCApCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6MTA2ODoyNjog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAxMDY4IHwgICAgIGVsc2UgaWYgKCBvbGRfd2VpZ2h0ID09IHJxZC0+bWF4
X3dlaWdodCApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CmNvbW1vbi9z
Y2hlZC9jcmVkaXQyLmM6MTA3NzozMDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJh1bnNpZ25lZCBpbnTi
gJkgYW5kIOKAmGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAxMDc3IHwgICAgICAgICAgICAg
aWYgKCBzdmMtPndlaWdodCA+IG1heF93ZWlnaHQgKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOiBJbiBmdW5jdGlvbiDi
gJh1cGRhdGVfcnVucV9sb2Fk4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9w
dWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVu
L3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rp
b24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9o
dm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1f
U0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNp
b24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFS
RV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NP
TVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERF
Q0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAg
ICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOjEzMDE6
NDc6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYbG9uZyBsb25nIHVuc2lnbmVkIGludOKAmSBhbmQg4oCY
c190aW1lX3TigJkge2FrYSDigJhsb25nIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTMw
MSB8ICAgICBpZiAoIHJxZC0+bG9hZF9sYXN0X3VwZGF0ZSArICgxVUxMIDw8IFcpICA8IG5v
dyApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzogSW4gZnVuY3Rpb24g4oCYdXBkYXRlX3N2
Y19sb2Fk4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOjEzNzg6NDY6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYbG9uZyBsb25nIHVuc2lnbmVkIGludOKAmSBhbmQg4oCYc190aW1lX3TigJkge2Fr
YSDigJhsb25nIGludOKAmX0gWy1Xc2lnbi1jb21wYXJlXQogMTM3OCB8ICAgICBpZiAoIHN2
Yy0+bG9hZF9sYXN0X3VwZGF0ZSArICgxVUxMIDw8IFcpIDwgbm93ICkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hl
ZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKAmGlzX3ByZWVtcHRhYmxl4oCZOgpjb21tb24v
c2NoZWQvY3JlZGl0Mi5jOjE1MDM6NDY6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdl
ciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYbG9uZyB1bnNpZ25l
ZCBpbnTigJkgYW5kIOKAmGxvbmcgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE1MDMgfCAg
ICAgcmV0dXJuIG5vdyAtIHN2Yy0+dW5pdC0+c3RhdGVfZW50cnlfdGltZSA+CiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24v
c2NoZWQvY3JlZGl0Mi5jOiBJbiBmdW5jdGlvbiDigJhydW5xX3RpY2tsZeKAmToKY29tbW9u
L3NjaGVkL2NyZWRpdDIuYzoxNjM2OjYwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE2MzYgfCAgICAgICAgIEFT
U0VSVChjcHVtYXNrX2N5Y2xlKGNwdSwgdW5pdC0+Y3B1X2hhcmRfYWZmaW5pdHkpID09IGNw
dSk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2NvbXBpbGVyLmg6MjE6NDM6IG5v
dGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g4oCYdW5saWtlbHnigJkKICAgMjEgfCAjZGVm
aW5lIHVubGlrZWx5KHgpICAgX19idWlsdGluX2V4cGVjdCghISh4KSwwKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVk
L2NyZWRpdDIuYzoxNjM2Ojk6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhBU1NF
UlTigJkKIDE2MzYgfCAgICAgICAgIEFTU0VSVChjcHVtYXNrX2N5Y2xlKGNwdSwgdW5pdC0+
Y3B1X2hhcmRfYWZmaW5pdHkpID09IGNwdSk7CiAgICAgIHwgICAgICAgICBefn5+fn4KY29t
bW9uL3NjaGVkL2NyZWRpdDIuYzoxNjY3OjE2OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE2NjcgfCAgICAgICAg
IGlmICggaSA8IG5yX2NwdV9pZHMgKQogICAgICB8ICAgICAgICAgICAgICAgIF4KY29tbW9u
L3NjaGVkL2NyZWRpdDIuYzoxNjgzOjE2OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVn
ZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg
4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE2ODMgfCAgICAgICAgIGlm
ICggaSA8IG5yX2NwdV9pZHMgKQogICAgICB8ICAgICAgICAgICAgICAgIF4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6Mzc1OjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNzUgfCAgICAgICAgICAgICAg
KGNwdSkgPCBucl9jcHVfaWRzOyAgICAgICAgICAgICAgICBcCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzoxNzI4OjU6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9jcHXigJkKIDE3MjggfCAgICAgZm9yX2Vh
Y2hfY3B1KGksICZtYXNrKQogICAgICB8ICAgICBefn5+fn5+fn5+fn4KY29tbW9uL3NjaGVk
L2NyZWRpdDIuYzoxNzMzOjE4OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDE3MzMgfCAgICAgICAgIEFTU0VSVChp
ICE9IGNwdSk7CiAgICAgIHwgICAgICAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVuL2Nv
bXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g4oCYdW5saWtl
bHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGluX2V4cGVjdCgh
ISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzoxNzMzOjk6IG5vdGU6IGluIGV4cGFuc2lv
biBvZiBtYWNybyDigJhBU1NFUlTigJkKIDE3MzMgfCAgICAgICAgIEFTU0VSVChpICE9IGNw
dSk7CiAgICAgIHwgICAgICAgICBefn5+fn4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzogSW4g
ZnVuY3Rpb24g4oCYYnVybl9jcmVkaXRz4oCZOgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOjE4
Nzg6Mzg6IHdhcm5pbmc6IHN1Z2dlc3QgYnJhY2VzIGFyb3VuZCBlbXB0eSBib2R5IGluIGFu
IOKAmGlm4oCZIHN0YXRlbWVudCBbLVdlbXB0eS1ib2R5XQogMTg3OCB8ICAgICAgICAgICAg
ICAgICAgICAgIHN2Yy0+c3RhcnRfdGltZSk7CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2Nr
Lmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3Ogou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZ
OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24g
b2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW50
4oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAg
ICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAg
XgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNs
ZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAg
fCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3Zh
cuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0
aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJl
ZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zh
cl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5efn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9u
IOKAmGNzY2hlZDJfdW5pdF95aWVsZOKAmToKY29tbW9uL3NjaGVkL2NyZWRpdDIuYzoyMjk2
OjQ0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogMjI5NiB8IGNzY2hlZDJfdW5pdF95aWVsZChjb25zdCBzdHJ1Y3Qgc2NoZWR1
bGVyICpvcHMsIHN0cnVjdCBzY2hlZF91bml0ICp1bml0KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYWx0ZXJuYXRpdmUuaDo0LAogICAgICAgICAg
ICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zeXNyZWdzLmg6MTQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3RpbWUuaDo0LAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi90aW1lLmg6NzgsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NSwKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo2LAogICAgICAgICAgICAgICAgIGZyb20g
YXJjaC9hcm0vdmdpYy92Z2ljLW1taW8tdjIuYzoxNjoKLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9jcHVmZWF0dXJlLmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0
YXRpYyBpbmxpbmUgaW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+
CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKAmGJhbGFuY2VfbG9hZOKA
mToKY29tbW9uL3NjaGVkL2NyZWRpdDIuYzoyNzgzOjMyOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHNf
dGltZV904oCZIHtha2Eg4oCYbG9uZyBpbnTigJl9IGFuZCDigJhsb25nIGxvbmcgdW5zaWdu
ZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDI3ODMgfCAgICAgICAgICAgICBpZiAoIHN0
LmxvYWRfZGVsdGEgPCAoMVVMTCA8PCAocHJ2LT5sb2FkX3ByZWNpc2lvbl9zaGlmdCArCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1
bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZu
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4K
Y29tbW9uL3NjaGVkL2NyZWRpdDIuYzoyNzg4OjMyOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmHNfdGlt
ZV904oCZIHtha2Eg4oCYbG9uZyBpbnTigJl9IGFuZCDigJhsb25nIGxvbmcgdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KIDI3ODggfCAgICAgICAgICAgICBpZiAoIHN0Lmxv
YWRfZGVsdGEgPCAoMVVMTCA8PCAocHJ2LT5sb2FkX3ByZWNpc2lvbl9zaGlmdCArCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL2NyZWRp
dDIuYzogSW4gZnVuY3Rpb24g4oCYY3NjaGVkMl9hZmZfY250bOKAmToKSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvZG9tY3RsLmg6MjEsCiAgICAgICAgICAgICAg
ICAgZnJvbSAuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oOjE4LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjQzOgouL2luY2x1ZGUvcHVibGljL2h2
bS9zYXZlLmg6IEluIGZ1bmN0aW9uIOKAmF9fSFZNX1NBVkVfRklYX0NPTVBBVF9FTkTigJk6
Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRp
YyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQz
Ml90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUvcHVi
bGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6
ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGlu
dCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBc
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5o
Ojg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hWTV9TQVZF
X1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBzdHJ1Y3Qg
aHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3Nj
aGVkL2NyZWRpdDIuYzozMTExOjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9w
c+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMzExMSB8IGNzY2hlZDJfYWZmX2NudGwoY29u
c3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBzdHJ1Y3Qgc2NoZWRfdW5pdCAqdW5pdCwKICAg
ICAgfCAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBm
aWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAg
ICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAg
ICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVj
dG1hcOKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2ln
bmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNp
Z25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+CmNvbW1vbi9z
Y2hlZC9jcmVkaXQyLmM6MzExMjo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhz
b2Z04oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAzMTEyIHwgICAgICAgICAgICAgICAgICBj
b25zdCBjcHVtYXNrX3QgKmhhcmQsIGNvbnN0IGNwdW1hc2tfdCAqc29mdCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKAmGNzY2hlZDJf
ZnJlZV91ZGF0YeKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdl
X2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwg
ICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAg
ICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgou
L2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgou
L2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBj
b25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0u
aDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5o
OjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpw
YWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwy
KHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikK
ICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Cklu
IGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRl
cuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFz
aywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBh
Z2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlu
bGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBh
Z2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgpjb21tb24v
c2NoZWQvY3JlZGl0Mi5jOjMyNDI6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
b3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAzMjQyIHwgY3NjaGVkMl9mcmVlX3VkYXRh
KGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgdm9pZCAqcHJpdikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hl
ZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKAmGNzY2hlZDJfdW5pdF9yZW1vdmXigJk6CmNv
bW1vbi9zY2hlZC9jcmVkaXQyLmM6MzI1MDo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhvcHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDMyNTAgfCBjc2NoZWQyX3VuaXRf
cmVtb3ZlKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgc3RydWN0IHNjaGVkX3VuaXQg
KnVuaXQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJl
ZS5oOjI0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDo1OgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9y
ZWFkX2xvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMjA6NTE6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEy
MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF9sb2NrKHJjdV9yZWFkX2xvY2tfdCAq
bG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDi
gJhyY3VfcmVhZF91bmxvY2vigJk6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDoxMzA6NTM6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbG9ja+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDEzMCB8IHN0YXRpYyBpbmxpbmUgdm9pZCByY3VfcmVhZF91bmxvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IElu
IGZ1bmN0aW9uIOKAmHJ1bnFfY2FuZGlkYXRl4oCZOgpJbiBmaWxlIGluY2x1ZGVkIGZyb20g
Li9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
bWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2No
ZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
IDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2
LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAg
MjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZt
X2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlf
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCmNvbW1vbi9zY2hlZC9jcmVk
aXQyLmM6MzM5ODo0ODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Np
b25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhsb25nIHVuc2lnbmVkIGludOKAmSBh
bmQg4oCYbG9uZyBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogMzM5OCB8ICAgICAgICAgIChu
b3cgLSBzY3Vyci0+dW5pdC0+c3RhdGVfZW50cnlfdGltZSkgPCBNSUNST1NFQ1MocHJ2LT5y
YXRlbGltaXRfdXMpICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzozNTA1OjQzOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmHVuc2lnbmVkIGludOKAmSBhbmQg4oCYaW504oCZIFstV3NpZ24tY29t
cGFyZV0KIDM1MDUgfCAgICAgICAgIGlmICggc2NoZWRfdW5pdF9tYXN0ZXIoc3ZjLT51bml0
KSAhPSBjcHUKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBefgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJl
X3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1f
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJt
X2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3Ry
dWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9x
dWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMg
aW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vcndsb2NrLmg6
NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDo3OgouL2lu
Y2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19sYXN04oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDIzMCB8ICAgICAg
ICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAgICAgICAgICAgICAgXgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1bWFza19jeWNsZeKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyNDAgfCAg
ICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAgICAgIF5+Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxvY19jcHVtYXNrX3ZhcuKA
mToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNTIgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192YXJfdCAqbWFzaykKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYZnJlZV9j
cHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzY1OjUxOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAz
NjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90
IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5efn5+CmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6IEluIGZ1bmN0aW9uIOKA
mGluaXRfY3B1X3J1bnF1ZXVl4oCZOgogIElOSVRfTyAgYXJjaC9hcm0vYm9vdGZkdC5pbml0
Lm8KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzozOTczOjQxOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KIDM5NzMgfCAg
ICAgQVNTRVJUKGNwdW1hc2tfd2VpZ2h0KCZycWQtPmFjdGl2ZSkgPT0gcnFkLT5ucl9jcHVz
KTsKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4K
Li9pbmNsdWRlL3hlbi9jb21waWxlci5oOjIxOjQzOiBub3RlOiBpbiBkZWZpbml0aW9uIG9m
IG1hY3JvIOKAmHVubGlrZWx54oCZCiAgIDIxIHwgI2RlZmluZSB1bmxpa2VseSh4KSAgIF9f
YnVpbHRpbl9leHBlY3QoISEoeCksMCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jcmVkaXQyLmM6Mzk3Mzo1OiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYQVNTRVJU4oCZCiAzOTczIHwgICAgIEFT
U0VSVChjcHVtYXNrX3dlaWdodCgmcnFkLT5hY3RpdmUpID09IHJxZC0+bnJfY3B1cyk7CiAg
ICAgIHwgICAgIF5+fn5+fgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOiBJbiBmdW5jdGlvbiDi
gJhjc2NoZWQyX2RlaW5pdF9wZGF0YeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lk
IGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlf
cG9saWN54oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MzA5IHwgc3RhdGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0
cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Rv
bWFpbi5oOjMxMDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9p
bmNsdWRlL3hlbi9kb21haW4uaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRl
L3hlbi9zY2hlZC5oOjExOgouL2luY2x1ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCY
bWZuX3RvX25pZOKAmToKLi9pbmNsdWRlL3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8
IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3QgbWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl5+fgpjb21tb24vc2No
ZWQvY3JlZGl0Mi5jOjQwNzU6NDE6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBl
eHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1
bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogNDA3NSB8ICAgICBBU1NFUlQoY3B1
bWFza193ZWlnaHQoJnJxZC0+YWN0aXZlKSA9PSBycWQtPm5yX2NwdXMpOwogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefgouL2luY2x1ZGUveGVu
L2NvbXBpbGVyLmg6MjE6NDM6IG5vdGU6IGluIGRlZmluaXRpb24gb2YgbWFjcm8g4oCYdW5s
aWtlbHnigJkKICAgMjEgfCAjZGVmaW5lIHVubGlrZWx5KHgpICAgX19idWlsdGluX2V4cGVj
dCghISh4KSwwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIF4KY29tbW9uL3NjaGVkL2NyZWRpdDIuYzo0MDc1OjU6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhBU1NFUlTigJkKIDQwNzUgfCAgICAgQVNTRVJUKGNwdW1hc2tf
d2VpZ2h0KCZycWQtPmFjdGl2ZSkgPT0gcnFkLT5ucl9jcHVzKTsKICAgICAgfCAgICAgXn5+
fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6MjU6Ci4v
aW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9kZXZpY2Xi
gJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3RhdGljIGlu
bGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQogICAg
ICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9kZWFz
c2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgz
IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4v
aW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1
ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5l
IGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWdu
ZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0
L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2lu
X2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgpjb21tb24vc2NoZWQvY3JlZGl0Mi5jOjQwODQ6
MzA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDigJhpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogNDA4NCB8ICAgICBlbHNlIGlmICggcnFkLT5waWNrX2JpYXMgPT0g
Y3B1ICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2
cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZw
Y2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKA
mToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVu
L21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVu
L21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rp
b24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3Qg
cGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hl
bi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAg
ICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVk
ZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3Qg
cGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGluZSB2b2lkIHRs
YmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2VfdGltZXN0YW1w
KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKSW4gZmlsZSBpbmNs
dWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vcmFkaXgtdHJlZS5oOjI0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo1OgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9uIOKAmHJjdV9yZWFkX2xvY2vigJk6Ci4vaW5jbHVk
ZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0aWMgYWx3YXlzX2lubGluZSBi
b29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fl4KLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
MTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3JlYWRfbG9ja190
ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IEluIGZ1bmN0aW9u
IOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEzMDo1
Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3VubG9jayhyY3Vf
cmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4v
aW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAgIGZyb20gLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21l
bV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVj
a+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6NDg6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAy
MyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGFyY2gvYXJtL3Z1YXJ0LmM6Mjg6Ci4vaW5jbHVk
ZS94ZW4vc2VyaWFsLmg6IEF0IHRvcCBsZXZlbDoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9t
ZW1fYWNjZXNzLmg6MjQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGNvbnN0IHN0cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g
4oCYcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL21lbV9hY2Nlc3MuaDozMTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAy
bV9tZW1fYWNjZXNzX3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2luY2x1ZGUveGVuL3NlcmlhbC5oOjE2NjoxOiB3YXJuaW5nOiDigJhpbmxp
bmXigJkgaXMgbm90IGF0IGJlZ2lubmluZyBvZiBkZWNsYXJhdGlvbiBbLVdvbGQtc3R5bGUt
ZGVjbGFyYXRpb25dCiAgMTY2IHwgc3RhdGljIHZvaWQgaW5saW5lIHhoY2lfZGJjX3VhcnRf
aW5pdCh2b2lkKSB7fQogICAgICB8IF5+fn5+fgphcmNoL2FybS92dWFydC5jOiBJbiBmdW5j
dGlvbiDigJh2dWFydF9tbWlvX3JlYWTigJk6CmFyY2gvYXJtL3Z1YXJ0LmM6MTAwOjQ5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHByaXbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMDAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICByZWdpc3Rlcl90ICpyLCB2
b2lkICpwcml2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fl5+fn4KYXJjaC9hcm0vdnVhcnQuYzogSW4gZnVuY3Rpb24g4oCYdnVh
cnRfbW1pb193cml0ZeKAmToKYXJjaC9hcm0vdnVhcnQuYzoxMTg6NDk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYcHJpduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDExOCB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICByZWdpc3Rlcl90IHIsIHZvaWQgKnByaXYp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhh
cmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
cDJtLmg6MTc0OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxNzQgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9hY3F1aXJl
X3Jlc291cmNlX2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV9hbHRwMm1f
Y2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MTg0OjM2OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYaWR44oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTg0IHwgdm9pZCBwMm1fYWx0cDJt
X2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYZ2V0X3BhZ2VfZnJvbV9nZm7igJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzNjEgfCAgICAgc3Ry
dWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90eXBlX3QgKnQsIHAybV9x
dWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2vigJk6
Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NDE3OjYxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MTcgfCBzdGF0aWMg
aW5saW5lIGJvb2wgcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDogSW4g
ZnVuY3Rpb24g4oCYYXJjaF92Y3B1X2Jsb2Nr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L2RvbWFpbi5oOjI5Nzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk3IHwgc3RhdGljIGlubGluZSB2b2lkIGFyY2hfdmNw
dV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20v
ZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN54oCZ
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMwOTo2MDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzA5IHwgc3Rh
dGljIGlubGluZSB2b2lkIHVwZGF0ZV9ndWVzdF9tZW1vcnlfcG9saWN5KHN0cnVjdCB2Y3B1
ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjMx
MDo3NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhnbXDigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzMTAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgc3RydWN0IGd1ZXN0X21lbW9yeV9wb2xpY3kgKmdtcCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9zY2hlZC5oOjI1OgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9hc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyNzg6NTQ6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgcGNp
X2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0
aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDoy
ODM6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI4MyB8IHN0YXRpYyBpbmxpbmUgdm9pZCB2cGNpX2RlYXNzaWduX2Rl
dmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikgeyB9CiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3JlYWTigJk6Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4Nzo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJfdCB2
cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODc6NjQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg3IHwgc3RhdGljIGlubGluZSB1aW50MzJf
dCB2cGNpX3JlYWQocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyODg6NDc6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4OCB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX3dyaXRl4oCZ
OgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6NDI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYc2JkZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4K
Li9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjYxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NCB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB2cGNpX3dyaXRlKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjQ0OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGF0YeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50
MzJfdCBkYXRhKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBm
dW5jdGlvbiDigJh2cGNpX3Byb2Nlc3NfcGVuZGluZ+KAmToKLi9pbmNsdWRlL3hlbi92cGNp
Lmg6MzAwOjY3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzMDAgfCBzdGF0aWMgaW5saW5lIGJvb2wgX19tdXN0X2NoZWNrIHZw
Y2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOiBJbiBmdW5jdGlvbiDigJhpc19wdl8zMmJpdF9k
b21haW7igJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMTMxOjY3OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KIDExMzEgfCBzdGF0
aWMgYWx3YXlzX2lubGluZSBib29sIGlzX3B2XzMyYml0X2RvbWFpbihjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fl4KYXJjaC9hcm0vdmdpYy92Z2ljLW1t
aW8tdjIuYzogSW4gZnVuY3Rpb24g4oCYdmdpY19tbWlvX3JlYWRfdjJfbWlzY+KAmToKYXJj
aC9hcm0vdmdpYy92Z2ljLW1taW8tdjIuYzoyNDo3Mjogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhsZW7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhZGRyX3QgYWRkciwgdW5zaWdu
ZWQgaW50IGxlbikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgphcmNoL2FybS92Z2lj
L3ZnaWMtbW1pby12Mi5jOiBJbiBmdW5jdGlvbiDigJh2Z2ljX21taW9fd3JpdGVfdjJfbWlz
Y+KAmToKYXJjaC9hcm0vdmdpYy92Z2ljLW1taW8tdjIuYzo1MTo2NDogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhsZW7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTEgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IGFkZHIsIHVuc2ln
bmVkIGludCBsZW4sCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+CmFyY2gvYXJtL3ZnaWMvdmdpYy1t
bWlvLXYyLmM6IEluIGZ1bmN0aW9uIOKAmHZnaWNfbW1pb193cml0ZV9zZ2ly4oCZOgphcmNo
L2FybS92Z2ljL3ZnaWMtbW1pby12Mi5jOjg1OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGFkZHLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgODUgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IGFkZHIsIHVuc2lnbmVkIGludCBsZW4s
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5efn5+
CmFyY2gvYXJtL3ZnaWMvdmdpYy1tbWlvLXYyLmM6ODU6NjE6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbGVu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDg1IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcGFkZHJfdCBhZGRyLCB1bnNpZ25lZCBpbnQg
bGVuLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl5+fgogIElOSVRfTyAgYXJjaC9hcm0vZG9tMGxlc3MtYnVp
bGQuaW5pdC5vCiAgSU5JVF9PICBhcmNoL2FybS9kb21haW5fYnVpbGQuaW5pdC5vCiAgSU5J
VF9PICBhcmNoL2FybS9rZXJuZWwuaW5pdC5vCiAgSU5JVF9PICBhcmNoL2FybS9zdGF0aWMt
ZXZ0Y2huLmluaXQubwogIElOSVRfTyAgYXJjaC9hcm0vc3RhdGljLW1lbW9yeS5pbml0Lm8K
ICBJTklUX08gIGFyY2gvYXJtL3N0YXRpYy1zaG1lbS5pbml0Lm8KICBDQyAgICAgIGFyY2gv
YXJtL3ZnaWMvdmdpYy1pbml0Lm8KICBDQyAgICAgIGNvbW1vbi9zY2hlZC9ydC5vCkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6
NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVn
cy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90
aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAg
ICAgICAgICBmcm9tIGFyY2gvYXJtL3ZnaWMvdmdpYy1pbml0LmM6MTk6Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJfc2libGlu
Z3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50
IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+
fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21j
dGwuaDoyMSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3Rs
Lmg6MTgsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6
NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1f
U0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2
OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09N
UEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVk
ZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g
4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9U
WVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+
fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2
IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZv
aWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2lu
Y2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1h
Y3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NB
VkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+
fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRl
L2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAu
aDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVk
ZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAg
ICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2Yg
aW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZ
IGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBp
ZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAgIHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRl
L3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgou
L2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxp
bmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1h
c2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8
IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFz
aykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWlu
Lmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToK
Li9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4v
aW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5v
ZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5j
bHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUv
eGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IElu
IGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9kaXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJv
b2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQg
bG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21t
Lmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2Rp
cmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9u
IOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTEgfCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhl
YWQpCiAgICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3By
ZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRl
L3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUv
eGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIgfCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdl
X2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9k
ZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9s
aXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAqcGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkMSwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6
Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0NzggfCAgICAgICAgICAgICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQg
KmhlYWQyKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1
c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFz
a190ICptYXNrLCB1aW50MzJfdCBwYWdlX3RpbWVzdGFtcCkge30KICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBz
dGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWlu
dDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+
CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToK
Li9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJbiBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr
4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0
aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3Jl
YWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAx
MzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tf
dCAqbG9jaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5o
OiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyMzo0ODogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJh24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBw
Mm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gv
YXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoyNDo2MTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhyc3DigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpy
c3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vz
cy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0
aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6
IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUg
Ym9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rp
b24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0u
aDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4NDo0ODogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQg
fCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRf
cGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJt
X3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9h
cmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50
X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9vbCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNr
KHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hbHRlcm5hdGl2ZS5oOjQsCiAgICAgICAgICAg
ICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3N5c3JlZ3MuaDoxNCwKICAgICAg
ICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vdGltZS5oOjQsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3RpbWUuaDo3OCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo1LAogICAgICAgICAgICAgICAg
IGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjYsCiAgICAgICAgICAgICAgICAgZnJvbSBj
b21tb24vc2NoZWQvcnQuYzoxNjoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9jcHVmZWF0dXJl
Lmg6IEluIGZ1bmN0aW9uIOKAmGNwdV9ucl9zaWJsaW5nc+KAmToKLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9jcHVmZWF0dXJlLmg6MTAxOjQ4OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEwMSB8IHN0YXRpYyBpbmxpbmUg
aW50IGNwdV9ucl9zaWJsaW5ncyh1bnNpZ25lZCBpbnQgY3B1KQogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0v
aW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfdmNwdV9ibG9ja+KA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDoyOTc6NDk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NyB8IHN0
YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYpIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDigJh1cGRh
dGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21h
aW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxpbmUgdm9pZCB1cGRhdGVfZ3Vlc3Rf
bWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzEwIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBndWVzdF9tZW1vcnlf
cG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KICBDQyAgICAg
IGNvbW1vbi9zY2hlZC9udWxsLm8KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94
ZW4vc2NoZWQuaDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZw
Y2lfYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNzggfCBzdGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBj
aV9kZXYgKnBkZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5j
dGlvbiDigJh2cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
MjgzOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1
ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2JkZuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMyX3Qg
dnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHJl
Z+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODgg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0ZeKA
mToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVp
bnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTUg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwgdWlu
dDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4g
ZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVjayB2
cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+
fn5eCi4vaW5jbHVkZS94ZW4vc2NoZWQuaDogSW4gZnVuY3Rpb24g4oCYaXNfcHZfMzJiaXRf
ZG9tYWlu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MTEzMTo2Nzogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAxMTMxIHwgc3Rh
dGljIGFsd2F5c19pbmxpbmUgYm9vbCBpc19wdl8zMmJpdF9kb21haW4oY29uc3Qgc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5eCmFyY2gvYXJtL3ZnaWMvdmdpYy1p
bml0LmM6IEluIGZ1bmN0aW9uIOKAmGRvbWFpbl92Z2ljX2ZyZWXigJk6CmFyY2gvYXJtL3Zn
aWMvdmdpYy1pbml0LmM6MjIwOjIwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAyMjAgfCAgICAgZm9yICggaSA9
IDA7IGkgPCBkaXN0LT5ucl9zcGlzOyBpKysgKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICBeCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUvcHVibGljL2RvbWN0bC5oOjIx
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9zeXNjdGwuaDoxOCwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc3BpbmxvY2suaDo0MzoKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOiBJbiBmdW5jdGlvbiDigJhfX0hWTV9TQVZFX0ZJ
WF9DT01QQVRfRU5E4oCZOgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6NTY6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
ICA1NiB8ICAgICBzdGF0aWMgaW5saW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNf
eCh2b2lkICpoLCB1aW50MzJfdCBzaXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fl4KLi9pbmNsdWRlL3B1Ymxp
Yy9odm0vc2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNM
QVJFX0hWTV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5E
LCAwLCBzdHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+
fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjY4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAg
c3RhdGljIGlubGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwg
dWludDMyX3Qgc2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS9w
dWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCY
REVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZCiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBF
KEVORCwgMCwgc3RydWN0IGh2bV9zYXZlX2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fs
dGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJjaC9hcm0vaW5jbHVk
ZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94
ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
NiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9zY2hlZC9udWxsLmM6MzE6Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJbiBmdW5jdGlvbiDigJhjcHVfbnJf
c2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOjEwMTo0
ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhjcHXigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBjcHVfbnJfc2libGluZ3ModW5zaWdu
ZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNs
dWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9z
bXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9yd2xvY2suaDo2
LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5j
bHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2xhc3TigJk6Ci4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMjMwIHwgICAgICAg
ICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAgIHwgICAgICAgICAgICAgICAgICBeCi4v
aW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZ
OgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAg
ICAgICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5j
bHVkZS9wdWJsaWMvaHZtL3NhdmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhf
Q09NUEFUX0VOROKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAgICAg
ICAgIF5+Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo1Njo1Njogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJho4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAg
IHN0YXRpYyBpbmxpbmUgaW50IF9fSFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgs
IHVpbnQzMl90IHNpemUpIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+XgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZl
Lmg6ODY6MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NB
VkVfVFlQReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVj
dCBodm1fc2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1
ZGUvcHVibGljL2h2bS9zYXZlLmg6NTY6Njg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICA1NiB8ICAgICBzdGF0aWMgaW5s
aW5lIGludCBfX0hWTV9TQVZFX0ZJWF9DT01QQVRfIyNfeCh2b2lkICpoLCB1aW50MzJfdCBz
aXplKSBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3B1YmxpYy9odm0v
c2F2ZS5oOjg2OjE6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhERUNMQVJFX0hW
TV9TQVZFX1RZUEXigJkKICAgODYgfCBERUNMQVJFX0hWTV9TQVZFX1RZUEUoRU5ELCAwLCBz
dHJ1Y3QgaHZtX3NhdmVfZW5kKTsKICAgICAgfCBefn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9p
bmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9uIOKAmGFsbG9jX2NwdW1hc2tfdmFy
4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6NTM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM1MiB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3ZhcihjcHVtYXNrX3Zhcl90ICptYXNrKQog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhmcmVl
X2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNjU6NTE6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2NwdW1hc2tfdmFyKGNwdW1hc2tfdmFy
X3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+
fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4v
ZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1bmN0aW9uIOKAmG1mbl90b19uaWTi
gJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMjkgfCBzdGF0aWMgaW5s
aW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZuKQogICAgICB8ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc21wLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc21wLmg6NCwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVk
ZS94ZW4vcndsb2NrLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4v
c2NoZWQuaDo3OgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCYY3B1
bWFza19sYXN04oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyMzA6MTg6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDIzMCB8ICAgICAgICAgICAgICBjcHUgPCBucl9jcHVfaWRzOwogICAgICB8ICAgICAg
ICAgICAgICAgICAgXgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVuY3Rpb24g4oCY
Y3B1bWFza19jeWNsZeKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MjQwOjEzOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAyNDAgfCAgICAgaWYgKG54dCA9PSBucl9jcHVfaWRzKQogICAgICB8ICAgICAg
ICAgICAgIF5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlvbiDigJhhbGxv
Y19jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6MzUyOjUzOiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNTIgfCBzdGF0aWMgaW5saW5lIGJvb2wgYWxsb2NfY3B1bWFza192YXIoY3B1bWFza192
YXJfdCAqbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDogSW4gZnVu
Y3Rpb24g4oCYZnJlZV9jcHVtYXNrX3ZhcuKAmToKLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6
MzY1OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAzNjUgfCBzdGF0aWMgaW5saW5lIHZvaWQgZnJlZV9jcHVtYXNrX3Zh
cihjcHVtYXNrX3Zhcl90IG1hc2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL21tLmg6MTIsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL21tLmg6MjMzLAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo0LAogICAgICAgICAgICAgICAgIGZyb20gLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9kb21haW4uaDo3LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9kb21haW4uaDoxNToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5o
OiBJbiBmdW5jdGlvbiDigJhhcmNoX21mbnNfaW5fZGlyZWN0bWFw4oCZOgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NTc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYbWZu4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGlu
ZSBib29sIGFyY2hfbWZuc19pbl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2ln
bmVkIGxvbmcgbnIpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9tbS5oOjEwOjc2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG5y4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgIDEwIHwgc3RhdGljIGlubGluZSBib29sIGFyY2hfbWZuc19p
bl9kaXJlY3RtYXAodW5zaWduZWQgbG9uZyBtZm4sIHVuc2lnbmVkIGxvbmcgbnIpCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5j
dGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1MTo0NTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlzdF9oZWFk
ICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlz
dF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcgfCAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5j
bHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToKLi9pbmNs
dWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChzdHJ1Y3Qg
cGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhwYWdlX2xp
c3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3NyB8IHBh
Z2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xpc3Rf
aGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVuL21tLmg6
NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2VfbGlzdF9o
ZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fl5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5
OgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRs
YmZsdXNoX2ZpbHRlcuKAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjEx
OjQ3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNw
dW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHBhZ2VfdGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDEx
IHwgc3RhdGljIGlubGluZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ss
IHVpbnQzMl90IHBhZ2VfdGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+
fn5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9kb21haW4uaDo1LAog
ICAgICAgICAgICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjExOgouL2luY2x1
ZGUveGVuL251bWEuaDogSW4gZnVuY3Rpb24g4oCYbWZuX3RvX25pZOKAmToKLi9pbmNsdWRl
L3hlbi9udW1hLmg6MTI5OjQxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyOSB8IHN0YXRpYyBpbmxpbmUgbm9kZWlkX3Qg
bWZuX3RvX25pZChtZm5fdCBtZm4pCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hl
bi9yYWRpeC10cmVlLmg6MjQsCiAgICAgICAgICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL3AybS5oOjU6Ci4vaW5jbHVkZS94ZW4vcmN1cGRhdGUuaDogSW4gZnVuY3Rp
b24g4oCYcmN1X3JlYWRfbG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOjEyMDo1
MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMTIwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX2xvY2socmN1X3Jl
YWRfbG9ja190ICpsb2NrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3JjdXBkYXRlLmg6IElu
IGZ1bmN0aW9uIOKAmHJjdV9yZWFkX3VubG9ja+KAmToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0
ZS5oOjEzMDo1Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhsb2Nr4oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMTMwIHwgc3RhdGljIGlubGluZSB2b2lkIHJjdV9yZWFkX3Vu
bG9jayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vbWVtX2FjY2Vzcy5oOjM3LAogICAgICAgICAgICAgICAg
IGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDo3OgouL2FyY2gvYXJtL2luY2x1
ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3NfZW11
bGF0ZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjM6
NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogICAyMyB8IGJvb2wgcDJtX21lbV9hY2Nlc3NfZW11bGF0ZV9jaGVjayhzdHJ1Y3Qg
dmNwdSAqdiwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tZW1fYWNjZXNzLmg6MjQ6NjE6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcnNw4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgIDI0IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCB2bV9ldmVudF9zdCAqcnNwKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDogSW4gZnVuY3Rpb24g4oCYcDJtX21lbV9hY2Nlc3Nf
c2FuaXR5X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3MuaDoz
MTo2Mzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgIDMxIHwgc3RhdGljIGlubGluZSBib29sIHAybV9tZW1fYWNjZXNzX3Nhbml0
eV9jaGVjayhzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGlu
Y2x1ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9tbS5oOjEyLAogICAgICAgICAg
ICAgICAgIGZyb20gLi9pbmNsdWRlL3hlbi9tbS5oOjIzMywKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6NywKICAgICAgICAgICAgICAg
ICBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6MTU6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vYXJtNjQvbW0uaDogSW4gZnVuY3Rpb24g4oCYYXJjaF9tZm5zX2luX2RpcmVjdG1hcOKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9tbS5oOjEwOjU3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1mbuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxv
bmcgbWZuLCB1bnNpZ25lZCBsb25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vYXJtNjQvbW0uaDoxMDo3Njogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhucuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMCB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBhcmNoX21mbnNfaW5fZGlyZWN0bWFwKHVuc2lnbmVkIGxvbmcgbWZuLCB1bnNpZ25lZCBs
b25nIG5yKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+Ci4vYXJjaC9hcm0vaW5j
bHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWlyZV9yZXNvdXJjZV9j
aGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6NjM6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE3NCB8
IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2VfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9w
Mm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToKLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCY
duKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJtX2FsdHAybV9jaGVj
ayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjE4
NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYs
IHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOiBJbiBm
dW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOiBJ
biBmdW5jdGlvbiDigJhwYWdlX2xpc3RfbmV4dOKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ1
MTo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhoZWFk4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgNDUxIHwgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBhZ2VfbGlz
dF9oZWFkICpoZWFkKQogICAgICB8ICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBh
Z2VfbGlzdF9wcmV24oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDU3OjQ1OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTcg
fCAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
Ci4vaW5jbHVkZS94ZW4vbW0uaDogSW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X2RlbOKAmToK
Li9pbmNsdWRlL3hlbi9tbS5oOjQ3Mjo2Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhoZWFk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcyIHwgcGFnZV9saXN0X2RlbChz
dHJ1Y3QgcGFnZV9pbmZvICpwYWdlLCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVsMuKAmToKLi9pbmNsdWRlL3hlbi9tbS5oOjQ3Nzo2Mzogd2FybmluZzog
dW51c2VkIHBhcmFtZXRlciDigJhoZWFkMeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8IHBhZ2VfbGlzdF9kZWwyKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdl
X2xpc3RfaGVhZCAqaGVhZDEsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+fgouL2luY2x1ZGUveGVu
L21tLmg6NDc4OjM5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQy4oCZIFst
V3VudXNlZC1wYXJhbWV0ZXJdCiAgNDc4IHwgICAgICAgICAgICAgICAgc3RydWN0IHBhZ2Vf
bGlzdF9oZWFkICpoZWFkMikKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fl5+fn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6MzYxOjY5OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAzNjEgfCAgICAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sIHAybV90
eXBlX3QgKnQsIHAybV9xdWVyeV90IHEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5eCkluIGZp
bGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL21tLmg6NTg5OgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6IEluIGZ1bmN0aW9uIOKAmHRsYmZsdXNoX2ZpbHRlcuKA
mToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9mbHVzaHRsYi5oOjExOjQ3OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG1hc2vigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTEg
fCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hfZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywg
dWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2Fz
bS9mbHVzaHRsYi5oOjExOjYyOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBhZ2Vf
dGltZXN0YW1w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDExIHwgc3RhdGljIGlubGlu
ZSB2b2lkIHRsYmZsdXNoX2ZpbHRlcihjcHVtYXNrX3QgKm1hc2ssIHVpbnQzMl90IHBhZ2Vf
dGltZXN0YW1wKSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn5+fn5+fn5+fn5+fgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL3AybS5oOiBJbiBmdW5jdGlvbiDigJhwMm1fdm1fZXZlbnRfc2FuaXR5
X2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQxNzo2MTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDE3
IHwgc3RhdGljIGlubGluZSBib29sIHAybV92bV9ldmVudF9zYW5pdHlfY2hlY2soc3RydWN0
IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJ
biBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3Mu
aDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nl
c3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJt
X2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWlu
ICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0
cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBz
dHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6IEluIGZ1bmN0aW9u
IOKAmGFyY2hfdmNwdV9ibG9ja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4u
aDoyOTc6NDk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDI5NyB8IHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2so
c3RydWN0IHZjcHUgKnYpIHt9CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5o
OiBJbiBmdW5jdGlvbiDigJh1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeeKAmToKLi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMDk6NjA6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8IHN0YXRpYyBpbmxp
bmUgdm9pZCB1cGRhdGVfZ3Vlc3RfbWVtb3J5X3BvbGljeShzdHJ1Y3QgdmNwdSAqdiwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+
fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDozMTA6NzU6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZ21w4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzEwIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHN0cnVjdCBndWVzdF9tZW1vcnlfcG9saWN5ICpnbXApCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5efn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWdu
X2RldmljZeKAmToKSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vaW5jbHVkZS94ZW4vc2NoZWQu
aDoyNToKLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfYXNzaWdu
X2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjc4OjU0OiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNzggfCBz
dGF0aWMgaW5saW5lIGludCB2cGNpX2Fzc2lnbl9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBk
ZXYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2
cGNpX2RlYXNzaWduX2RldmljZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6MjgzOjU3OiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBkZXbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAyODMgfCBzdGF0aWMgaW5saW5lIHZvaWQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IHBjaV9kZXYgKnBkZXYpIHsgfQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3Zw
Y2kuaDoyNzg6NTQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGRlduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDI3OCB8IHN0YXRpYyBpbmxpbmUgaW50IHZwY2lfYXNzaWdu
X2RldmljZShzdHJ1Y3QgcGNpX2RldiAqcGRldikKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hl
bi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfZGVhc3NpZ25fZGV2aWNl4oCZOgouL2lu
Y2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV9yZWFk4oCZOgouL2luY2x1
ZGUveGVuL3ZwY2kuaDoyODc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2Jk
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWludDMy
X3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAgICAg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fl5+fn4KLi9p
bmNsdWRlL3hlbi92cGNpLmg6Mjg3OjY0OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mHJlZ+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI4NyB8IHN0YXRpYyBpbmxpbmUgdWlu
dDMyX3QgdnBjaV9yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg4OjQ3OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNpemXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODggfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6
ZSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBjaV93cml0
ZeKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk0OjQyOiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMg
aW5saW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCBy
ZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NDo2MTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTQgfCBzdGF0aWMgaW5s
aW5lIHZvaWQgdnBjaV93cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcs
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo0NDogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
Mjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUs
IHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjk1OjU5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRhdGHigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
OTUgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSwg
dWludDMyX3QgZGF0YSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDog
SW4gZnVuY3Rpb24g4oCYdnBjaV9wcm9jZXNzX3BlbmRpbmfigJk6Ci4vaW5jbHVkZS94ZW4v
dnBjaS5oOjMwMDo2Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3Vu
dXNlZC1wYXJhbWV0ZXJdCiAgMzAwIHwgc3RhdGljIGlubGluZSBib29sIF9fbXVzdF9jaGVj
ayB2cGNpX3Byb2Nlc3NfcGVuZGluZyhzdHJ1Y3QgdmNwdSAqdikKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5eCi4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjgzIHwgc3RhdGlj
IGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9u
IOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6Mjg3OjQ1OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
ODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNiZGYs
IHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Nzo2NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChwY2lfc2JkZl90IHNi
ZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6
IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5
NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm4oCZIFstV3VudXNlZC1w
YXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3Ni
ZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTQ6
NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfd3JpdGUocGNpX3NiZGZf
dCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYc2l6ZeKA
mSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRhKQogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94
ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkYXRh4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+
fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcHJvY2Vzc19w
ZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwMCB8IHN0YXRp
YyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0
IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhpc191bml0
X29ubGluZeKAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29t
cGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNz
OiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAg
MzU3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5l
eHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjkxOjU6IG5v
dGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXi
gJkKICAgOTEgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAg
ICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0
ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5hYmxl4oCZOgouL2luY2x1ZGUveGVuL3Nj
aGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lv
bnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQg
aW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1v
bi9zY2hlZC9wcml2YXRlLmg6MTA4OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDi
gJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxMDggfCAgICAgZm9yX2VhY2hfc2No
ZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1
bm5hYmxlX3N0YXRl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5n
OiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25l
ZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFy
ZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1
KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTM1
OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0
X3ZjcHXigJkKICAxMzUgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwg
diApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQv
cHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRfcmVz4oCZOgouL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsg
ICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBe
CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTU0OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxNTQgfCAgICAgZm9yX2Vh
Y2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhz
Y2hlZF9zZXRfcGF1c2VfZmxhZ3PigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1
X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZh
dGUuaDoxNzY6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3Nj
aGVkX3VuaXRfdmNwdeKAmQogIDE3NiB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUg
KCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1v
bi9zY2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2NsZWFyX3BhdXNlX2Zs
YWdz4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9p
bl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTg2OjU6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkK
ICAxODYgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAg
IHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1ZGUveGVuL3NjaGVkLmg6
IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToKLi9pbmNsdWRlL3hlbi9z
Y2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNfaW5saW5lIGJvb2wgaXNf
cHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9ldmVudC5oOjE3
LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3NjaGVkL3J0LmM6MTk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZXZlbnQuaDogSW4gZnVuY3Rpb24g4oCYYXJjaF92aXJxX2lzX2ds
b2JhbOKAmToKY29tbW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYc2NoZWRf
dW5pdF9wYXVzZV9ub3N5bmPigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdh
cm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQg
c2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1j
b21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lk
IDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUu
aDo1MzA6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVk
X3VuaXRfdmNwdeKAmQogIDUzMCB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1
bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX3VuaXRfdW5wYXVzZeKAmToK
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBp
bnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkg
YW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+
dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjUzODo1OiBub3RlOiBpbiBleHBh
bnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgNTM4IHwg
ICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBe
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5o
OjUwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHZpcnHigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgNTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF92aXJxX2lzX2ds
b2JhbCh1bnNpZ25lZCBpbnQgdmlycSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+Xn5+fgpjb21tb24vc2NoZWQvbnVsbC5j
OiBJbiBmdW5jdGlvbiDigJhudWxsX2FsbG9jX3BkYXRh4oCZOgpjb21tb24vc2NoZWQvbnVs
bC5jOjE4MTo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhvcHPigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxODEgfCBzdGF0aWMgdm9pZCAqY2ZfY2hlY2sgbnVsbF9hbGxv
Y19wZGF0YShjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIGludCBjcHUpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9udWxsLmM6MTgxOjczOiB3YXJuaW5nOiB1bnVz
ZWQgcGFyYW1ldGVyIOKAmGNwdeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4MSB8IHN0
YXRpYyB2b2lkICpjZl9jaGVjayBudWxsX2FsbG9jX3BkYXRhKGNvbnN0IHN0cnVjdCBzY2hl
ZHVsZXIgKm9wcywgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5efn4KY29t
bW9uL3NjaGVkL251bGwuYzogSW4gZnVuY3Rpb24g4oCYbnVsbF9mcmVlX3BkYXRh4oCZOgpj
b21tb24vc2NoZWQvbnVsbC5jOjE5MzoyOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhvcHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxOTMgfCAgICAgY29uc3Qgc3RydWN0
IHNjaGVkdWxlciAqb3BzLCB2b2lkICpwY3B1LCBpbnQgY3B1KQogICAgICB8ICAgICB+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL3NjaGVkL251bGwuYzoxOTM6NTA6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYY3B14oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMTkzIHwgICAgIGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgdm9pZCAqcGNwdSwg
aW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB+fn5+Xn5+CmNvbW1vbi9zY2hlZC9udWxsLmM6IEluIGZ1bmN0aW9uIOKAmG51
bGxfYWxsb2NfdWRhdGHigJk6CmNvbW1vbi9zY2hlZC9udWxsLmM6MTk5OjI5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE5
OSB8ICAgICBjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHN0cnVjdCBzY2hlZF91bml0
ICp1bml0LCB2b2lkICpkZCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+CmNvbW1vbi9zY2hlZC9udWxsLmM6MTk5OjY1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTk5IHwgICAgIGNvbnN0IHN0
cnVjdCBzY2hlZHVsZXIgKm9wcywgc3RydWN0IHNjaGVkX3VuaXQgKnVuaXQsIHZvaWQgKmRk
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB+fn5+fn5efgpjb21tb24vc2NoZWQvbnVsbC5jOiBJbiBmdW5jdGlv
biDigJhudWxsX2ZyZWVfdWRhdGHigJk6CmNvbW1vbi9zY2hlZC9udWxsLmM6MjE1OjYyOiB3
YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmG9wc+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDIxNSB8IHN0YXRpYyB2b2lkIGNmX2NoZWNrIG51bGxfZnJlZV91ZGF0YShjb25zdCBz
dHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHZvaWQgKnByaXYpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+fgpj
b21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhpc191bml0X29ubGluZeKA
mToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlz
dC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjkxOjU6IG5vdGU6IGluIGV4
cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAgOTEg
fCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAg
IF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBm
dW5jdGlvbiDigJh1bml0X3J1bm5hYmxl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3
OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+
dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9w
cml2YXRlLmg6MTA4OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFj
aF9zY2hlZF91bml0X3ZjcHXigJkKICAxMDggfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92
Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpj
b21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5hYmxlX3N0
YXRl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJp
c29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKA
mGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcg
fCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9p
bl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTM1OjU6IG5vdGU6
IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkK
ICAxMzUgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAg
IHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5o
OiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRfcmVz4oCZOgouL2luY2x1ZGUveGVuL3NjaGVk
Lmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMg
b2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW50
4oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6MTU0OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhm
b3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxNTQgfCAgICAgZm9yX2VhY2hfc2NoZWRf
dW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRf
cGF1c2VfZmxhZ3PigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6
IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVk
bmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJl
XQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUp
LT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDoxNzY6
NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRf
dmNwdeKAmQogIDE3NiB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2
ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9w
cml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2NsZWFyX3BhdXNlX2ZsYWdz4oCZOgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51
bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTg2OjU6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxODYgfCAg
ICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvbnVsbC5jOiBJbiBmdW5jdGlv
biDigJhudWxsX3NjaGVkdWxl4oCZOgpjb21tb24vc2NoZWQvbnVsbC5jOjgxNzo2ODogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhub3figJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA4MTcgfCAgICAgY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBzdHJ1Y3Qgc2NoZWRf
dW5pdCAqcHJldiwgc190aW1lX3Qgbm93LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5efn4KY29t
bW9uL3NjaGVkL251bGwuYzogSW4gZnVuY3Rpb24g4oCYZHVtcF91bml04oCZOgpjb21tb24v
c2NoZWQvbnVsbC5jOjk1NTo2NDogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKAmSBjaGFu
Z2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5zaWduZWQgaW504oCZIGR1
ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBhcmVdCiAg
OTU1IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzY2hlZF91bml0X21hc3Rl
cihudmMtPnVuaXQpIDogLTEpOwogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+CmNvbW1vbi9zY2hlZC9u
dWxsLmM6OTUwOjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHByduKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDk1MCB8IHN0YXRpYyBpbmxpbmUgdm9pZCBkdW1wX3VuaXQo
Y29uc3Qgc3RydWN0IG51bGxfcHJpdmF0ZSAqcHJ2LAogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9z
Y2hlZC9wcml2YXRlLmg6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX3VuaXRfcGF1c2Vfbm9zeW5j
4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9s
aXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6NTMwOjU6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICA1
MzAgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwg
ICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJ
biBmdW5jdGlvbiDigJhzY2hlZF91bml0X3VucGF1c2XigJk6Ci4vaW5jbHVkZS94ZW4vc2No
ZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9u
cyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBp
bnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAog
ICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9u
L3NjaGVkL3ByaXZhdGUuaDo1Mzg6NTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKA
mGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogIDUzOCB8ICAgICBmb3JfZWFjaF9zY2hl
ZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+CmNvbW1vbi9zY2hlZC9ydC5jOiBJbiBmdW5jdGlvbiDigJhydF9kdW1wX3VuaXTi
gJk6CmNvbW1vbi9zY2hlZC9ydC5jOjMwOTozODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhvcHPigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBydF9kdW1wX3VuaXQo
Y29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBjb25zdCBzdHJ1Y3QgcnRfdW5pdCAqc3Zj
KQogICAgICB8ICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KY29t
bW9uL3NjaGVkL3J0LmM6IEluIGZ1bmN0aW9uIOKAmHJ0X3Jlc19waWNrX2xvY2tlZOKAmToK
Y29tbW9uL3NjaGVkL3J0LmM6NjUxOjE1OiB3YXJuaW5nOiBvcGVyYW5kIG9mIOKAmD864oCZ
IGNoYW5nZXMgc2lnbmVkbmVzcyBmcm9tIOKAmGludOKAmSB0byDigJh1bnNpZ25lZCBpbnTi
gJkgZHVlIHRvIHVuc2lnbmVkbmVzcyBvZiBvdGhlciBvcGVyYW5kIFstV3NpZ24tY29tcGFy
ZV0KICA2NTEgfCAgICAgICAgICAgICA6IGNwdW1hc2tfY3ljbGUoc2NoZWRfdW5pdF9tYXN0
ZXIodW5pdCksIGNwdXMpOwogICAgICB8ICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL3J0LmM6IEluIGZ1
bmN0aW9uIOKAmHJ0X3Jlc19waWNr4oCZOgpjb21tb24vc2NoZWQvcnQuYzo2NjM6Mzc6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgNjYzIHwgcnRfcmVzX3BpY2soY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBjb25z
dCBzdHJ1Y3Qgc2NoZWRfdW5pdCAqdW5pdCkKICAgICAgfCAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL3NjaGVkL3J0LmM6IEluIGZ1bmN0aW9uIOKA
mHJ0X2RlaW5pdF9wZGF0YeKAmToKY29tbW9uL3NjaGVkL3J0LmM6Nzc1OjUyOiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmHBjcHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3
NzUgfCBydF9kZWluaXRfcGRhdGEoY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCB2b2lk
ICpwY3B1LCBpbnQgY3B1KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIH5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL3J0LmM6IEluIGZ1bmN0
aW9uIOKAmHJ0X2FsbG9jX3VkYXRh4oCZOgpjb21tb24vc2NoZWQvcnQuYzo4NDc6NDA6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgODQ3IHwgcnRfYWxsb2NfdWRhdGEoY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBz
dHJ1Y3Qgc2NoZWRfdW5pdCAqdW5pdCwgdm9pZCAqZGQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9ydC5jOiBJbiBm
dW5jdGlvbiDigJhydF9mcmVlX3VkYXRh4oCZOgpjb21tb24vc2NoZWQvcnQuYzo4NzU6Mzk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgODc1IHwgcnRfZnJlZV91ZGF0YShjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMs
IHZvaWQgKnByaXYpCiAgICAgIHwgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5efn4KY29tbW9uL3NjaGVkL3J0LmM6IEluIGZ1bmN0aW9uIOKAmGJ1cm5fYnVkZ2V0
4oCZOgpjb21tb24vc2NoZWQvcnQuYzo5NTA6Mzc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgOTUwIHwgYnVybl9idWRnZXQo
Y29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCBzdHJ1Y3QgcnRfdW5pdCAqc3ZjLCBzX3Rp
bWVfdCBub3cpCiAgICAgIHwgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
Xn5+CmNvbW1vbi9zY2hlZC9ydC5jOiBJbiBmdW5jdGlvbiDigJhydW5xX3RpY2tsZeKAmToK
Y29tbW9uL3NjaGVkL3J0LmM6MTIxODoxNjogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRl
Z2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5k
IOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAxMjE4IHwgICAgIHdoaWxl
ICggY3B1IT0gbnJfY3B1X2lkcyApCiAgICAgIHwgICAgICAgICAgICAgICAgXn4KICBMRCAg
ICAgIGFyY2gvYXJtL3ZnaWMvYnVpbHRfaW4ubwogIExEICAgICAgYXJjaC9hcm0vYnVpbHRf
aW4ubwogIENDICAgICAgY29tbW9uL3NjaGVkL2NvcmUubwpJbiBmaWxlIGluY2x1ZGVkIGZy
b20gY29tbW9uL3NjaGVkL2NvcmUuYzoxOToKLi9pbmNsdWRlL3hlbi9wYXJhbS5oOiBJbiBm
dW5jdGlvbiDigJhub19jb25maWdfcGFyYW3igJk6Ci4vaW5jbHVkZS94ZW4vcGFyYW0uaDox
OTM6MTk6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzrigJkgY2hhbmdlcyBzaWduZWRuZXNz
IGZyb20g4oCYbG9uZyBpbnTigJkgdG8g4oCYc2l6ZV904oCZIHtha2Eg4oCYbG9uZyB1bnNp
Z25lZCBpbnTigJl9IGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdz
aWduLWNvbXBhcmVdCiAgMTkzIHwgICAgIGludCBsZW4gPSBlID8gKHsgQVNTRVJUKGUgPj0g
cyk7IGUgLSBzOyB9KSA6IHN0cmxlbihzKTsKICAgICAgfCAgICAgICAgICAgICAgICAgICBe
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FsdGVybmF0aXZlLmg6NCwKICAgICAgICAgICAgICAgICBm
cm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vc3lzcmVncy5oOjE0LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS90aW1lLmg6NCwKICAgICAgICAgICAg
ICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vdGltZS5oOjc4LAogICAgICAgICAgICAgICAgIGZy
b20gLi9pbmNsdWRlL3hlbi9zcGlubG9jay5oOjUsCiAgICAgICAgICAgICAgICAgZnJvbSAu
L2luY2x1ZGUveGVuL3NjaGVkLmg6NiwKICAgICAgICAgICAgICAgICBmcm9tIGNvbW1vbi9z
Y2hlZC9jb3JlLmM6MjA6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vY3B1ZmVhdHVyZS5oOiBJ
biBmdW5jdGlvbiDigJhjcHVfbnJfc2libGluZ3PigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9h
c20vY3B1ZmVhdHVyZS5oOjEwMTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhj
cHXigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDEgfCBzdGF0aWMgaW5saW5lIGludCBj
cHVfbnJfc2libGluZ3ModW5zaWduZWQgaW50IGNwdSkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fgpJbiBmaWxlIGluY2x1ZGVk
IGZyb20gLi9pbmNsdWRlL3B1YmxpYy9kb21jdGwuaDoyMSwKICAgICAgICAgICAgICAgICBm
cm9tIC4vaW5jbHVkZS9wdWJsaWMvc3lzY3RsLmg6MTgsCiAgICAgICAgICAgICAgICAgZnJv
bSAuL2luY2x1ZGUveGVuL3NwaW5sb2NrLmg6NDM6Ci4vaW5jbHVkZS9wdWJsaWMvaHZtL3Nh
dmUuaDogSW4gZnVuY3Rpb24g4oCYX19IVk1fU0FWRV9GSVhfQ09NUEFUX0VOROKAmToKLi9p
bmNsdWRlL3B1YmxpYy9odm0vc2F2ZS5oOjU2OjU2OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGjigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgNTYgfCAgICAgc3RhdGljIGlu
bGluZSBpbnQgX19IVk1fU0FWRV9GSVhfQ09NUEFUXyMjX3godm9pZCAqaCwgdWludDMyX3Qg
c2l6ZSkgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5eCi4vaW5jbHVkZS9wdWJsaWMvaHZtL3NhdmUuaDo4NjoxOiBu
b3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYREVDTEFSRV9IVk1fU0FWRV9UWVBF4oCZ
CiAgIDg2IHwgREVDTEFSRV9IVk1fU0FWRV9UWVBFKEVORCwgMCwgc3RydWN0IGh2bV9zYXZl
X2VuZCk7CiAgICAgIHwgXn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS9wdWJsaWMv
aHZtL3NhdmUuaDo1Njo2ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzaXpl4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgIDU2IHwgICAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
SFZNX1NBVkVfRklYX0NPTVBBVF8jI194KHZvaWQgKmgsIHVpbnQzMl90IHNpemUpIFwKICAg
ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgfn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUvcHVibGljL2h2bS9zYXZlLmg6ODY6
MTogbm90ZTogaW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmERFQ0xBUkVfSFZNX1NBVkVfVFlQ
ReKAmQogICA4NiB8IERFQ0xBUkVfSFZNX1NBVkVfVFlQRShFTkQsIDAsIHN0cnVjdCBodm1f
c2F2ZV9lbmQpOwogICAgICB8IF5+fn5+fn5+fn5+fn5+fn5+fn5+fgpJbiBmaWxlIGluY2x1
ZGVkIGZyb20gLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9zbXAuaDo1LAogICAgICAgICAgICAg
ICAgIGZyb20gLi9pbmNsdWRlL3hlbi9zbXAuaDo0LAogICAgICAgICAgICAgICAgIGZyb20g
Li9pbmNsdWRlL3hlbi9yd2xvY2suaDo2LAogICAgICAgICAgICAgICAgIGZyb20gLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjc6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5jdGlv
biDigJhjcHVtYXNrX2xhc3TigJk6Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOjIzMDoxODog
d2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVu
dCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWdu
LWNvbXBhcmVdCiAgMjMwIHwgICAgICAgICAgICAgIGNwdSA8IG5yX2NwdV9pZHM7CiAgICAg
IHwgICAgICAgICAgICAgICAgICBeCi4vaW5jbHVkZS94ZW4vY3B1bWFzay5oOiBJbiBmdW5j
dGlvbiDigJhjcHVtYXNrX2N5Y2xl4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDoyNDA6
MTM6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZm
ZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1X
c2lnbi1jb21wYXJlXQogIDI0MCB8ICAgICBpZiAobnh0ID09IG5yX2NwdV9pZHMpCiAgICAg
IHwgICAgICAgICAgICAgXn4KLi9pbmNsdWRlL3hlbi9jcHVtYXNrLmg6IEluIGZ1bmN0aW9u
IOKAmGFsbG9jX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2NwdW1hc2suaDozNTI6
NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDM1MiB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhbGxvY19jcHVtYXNrX3Zhcihj
cHVtYXNrX3Zhcl90ICptYXNrKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vY3B1bWFzay5o
OiBJbiBmdW5jdGlvbiDigJhmcmVlX2NwdW1hc2tfdmFy4oCZOgouL2luY2x1ZGUveGVuL2Nw
dW1hc2suaDozNjU6NTE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYbWFza+KAmSBb
LVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2NSB8IHN0YXRpYyBpbmxpbmUgdm9pZCBmcmVlX2Nw
dW1hc2tfdmFyKGNwdW1hc2tfdmFyX3QgbWFzaykKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fn4KSW4gZmlsZSBpbmNsdWRl
ZCBmcm9tIC4vaW5jbHVkZS94ZW4vZG9tYWluLmg6NSwKICAgICAgICAgICAgICAgICBmcm9t
IC4vaW5jbHVkZS94ZW4vc2NoZWQuaDoxMToKLi9pbmNsdWRlL3hlbi9udW1hLmg6IEluIGZ1
bmN0aW9uIOKAmG1mbl90b19uaWTigJk6Ci4vaW5jbHVkZS94ZW4vbnVtYS5oOjEyOTo0MTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAxMjkgfCBzdGF0aWMgaW5saW5lIG5vZGVpZF90IG1mbl90b19uaWQobWZuX3QgbWZu
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn4K
SW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbW0uaDoxMiwK
ICAgICAgICAgICAgICAgICBmcm9tIC4vaW5jbHVkZS94ZW4vbW0uaDoyMzMsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjQsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOjcsCiAgICAg
ICAgICAgICAgICAgZnJvbSAuL2luY2x1ZGUveGVuL2RvbWFpbi5oOjE1OgouL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L21tLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfbWZuc19pbl9k
aXJlY3RtYXDigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvbW0uaDoxMDo1Nzog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhtZm7igJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAgMTAgfCBzdGF0aWMgaW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1
bnNpZ25lZCBsb25nIG1mbiwgdW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2Fy
Y2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L21tLmg6MTA6NzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbnLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAgMTAgfCBzdGF0aWMg
aW5saW5lIGJvb2wgYXJjaF9tZm5zX2luX2RpcmVjdG1hcCh1bnNpZ25lZCBsb25nIG1mbiwg
dW5zaWduZWQgbG9uZyBucikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efgouL2lu
Y2x1ZGUveGVuL21tLmg6IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9uZXh04oCZOgouL2lu
Y2x1ZGUveGVuL21tLmg6NDUxOjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhl
YWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NTEgfCAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQpCiAgICAgIHwgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDog
SW4gZnVuY3Rpb24g4oCYcGFnZV9saXN0X3ByZXbigJk6Ci4vaW5jbHVkZS94ZW4vbW0uaDo0
NTc6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYaGVhZOKAmSBbLVd1bnVzZWQt
cGFyYW1ldGVyXQogIDQ1NyB8ICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi9tbS5oOiBJbiBmdW5jdGlvbiDigJhw
YWdlX2xpc3RfZGVs4oCZOgouL2luY2x1ZGUveGVuL21tLmg6NDcyOjYyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzIg
fCBwYWdlX2xpc3RfZGVsKHN0cnVjdCBwYWdlX2luZm8gKnBhZ2UsIHN0cnVjdCBwYWdlX2xp
c3RfaGVhZCAqaGVhZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL21tLmg6
IEluIGZ1bmN0aW9uIOKAmHBhZ2VfbGlzdF9kZWwy4oCZOgouL2luY2x1ZGUveGVuL21tLmg6
NDc3OjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGhlYWQx4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNDc3IHwgcGFnZV9saXN0X2RlbDIoc3RydWN0IHBhZ2VfaW5mbyAq
cGFnZSwgc3RydWN0IHBhZ2VfbGlzdF9oZWFkICpoZWFkMSwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fl5+
fn5+Ci4vaW5jbHVkZS94ZW4vbW0uaDo0Nzg6Mzk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYaGVhZDLigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NzggfCAgICAgICAgICAg
ICAgICBzdHJ1Y3QgcGFnZV9saXN0X2hlYWQgKmhlYWQyKQogICAgICB8ICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+fn4KSW4gZmlsZSBpbmNsdWRlZCBmcm9t
IC4vaW5jbHVkZS94ZW4vbW0uaDo1ODk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZmx1c2h0
bGIuaDogSW4gZnVuY3Rpb24g4oCYdGxiZmx1c2hfZmlsdGVy4oCZOgouL2FyY2gvYXJtL2lu
Y2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6NDc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYbWFza+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogICAxMSB8IHN0YXRpYyBpbmxpbmUg
dm9pZCB0bGJmbHVzaF9maWx0ZXIoY3B1bWFza190ICptYXNrLCB1aW50MzJfdCBwYWdlX3Rp
bWVzdGFtcCkge30KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+Xn5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2ZsdXNodGxiLmg6MTE6
NjI6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcGFnZV90aW1lc3RhbXDigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICAgMTEgfCBzdGF0aWMgaW5saW5lIHZvaWQgdGxiZmx1c2hf
ZmlsdGVyKGNwdW1hc2tfdCAqbWFzaywgdWludDMyX3QgcGFnZV90aW1lc3RhbXApIHt9CiAg
ICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fn5+fn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2lu
Y2x1ZGUveGVuL3JhZGl4LXRyZWUuaDoyNCwKICAgICAgICAgICAgICAgICBmcm9tIC4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6NToKLi9pbmNsdWRlL3hlbi9yY3VwZGF0ZS5oOiBJ
biBmdW5jdGlvbiDigJhyY3VfcmVhZF9sb2Nr4oCZOgouL2luY2x1ZGUveGVuL3JjdXBkYXRl
Lmg6MTIwOjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAxMjAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1X3JlYWRfbG9j
ayhyY3VfcmVhZF9sb2NrX3QgKmxvY2spCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vcmN1cGRh
dGUuaDogSW4gZnVuY3Rpb24g4oCYcmN1X3JlYWRfdW5sb2Nr4oCZOgouL2luY2x1ZGUveGVu
L3JjdXBkYXRlLmg6MTMwOjUzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGxvY2vi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMzAgfCBzdGF0aWMgaW5saW5lIHZvaWQgcmN1
X3JlYWRfdW5sb2NrKHJjdV9yZWFkX2xvY2tfdCAqbG9jaykKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+Xn5+fgpJbiBmaWxl
IGluY2x1ZGVkIGZyb20gLi9pbmNsdWRlL3hlbi9tZW1fYWNjZXNzLmg6MzcsCiAgICAgICAg
ICAgICAgICAgZnJvbSAuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL3AybS5oOjc6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVtX2Fj
Y2Vzc19lbXVsYXRlX2NoZWNr4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nl
c3MuaDoyMzo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh24oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgIDIzIHwgYm9vbCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNr
KHN0cnVjdCB2Y3B1ICp2LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL21lbV9hY2Nlc3Mu
aDoyNDo2MTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyc3DigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAgMjQgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IHZtX2V2ZW50X3N0ICpyc3ApCiAgICAgIHwgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Xn5+Ci4vYXJj
aC9hcm0vaW5jbHVkZS9hc20vbWVtX2FjY2Vzcy5oOiBJbiBmdW5jdGlvbiDigJhwMm1fbWVt
X2FjY2Vzc19zYW5pdHlfY2hlY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vbWVtX2Fj
Y2Vzcy5oOjMxOjYzOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGTigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAgMzEgfCBzdGF0aWMgaW5saW5lIGJvb2wgcDJtX21lbV9hY2Nl
c3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5eCi4v
YXJjaC9hcm0vaW5jbHVkZS9hc20vcDJtLmg6IEluIGZ1bmN0aW9uIOKAmGFyY2hfYWNxdWly
ZV9yZXNvdXJjZV9jaGVja+KAmToKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxNzQ6
NjM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDE3NCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX2FjcXVpcmVfcmVzb3VyY2Vf
Y2hlY2soc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9p
bmNsdWRlL2FzbS9wMm0uaDogSW4gZnVuY3Rpb24g4oCYcDJtX2FsdHAybV9jaGVja+KAmToK
Li9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDoxODQ6MzY6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDE4NCB8IHZvaWQgcDJt
X2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAqdiwgdWludDE2X3QgaWR4KQogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUv
YXNtL3AybS5oOjE4NDo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhpZHjigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAxODQgfCB2b2lkIHAybV9hbHRwMm1fY2hlY2soc3Ry
dWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgouL2FyY2gvYXJtL2luY2x1ZGUvYXNt
L3AybS5oOiBJbiBmdW5jdGlvbiDigJhnZXRfcGFnZV9mcm9tX2dmbuKAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDozNjE6Njk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYceKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDM2MSB8ICAgICBzdHJ1Y3QgZG9tYWlu
ICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgcDJtX3R5cGVfdCAqdCwgcDJtX3F1ZXJ5X3QgcSkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wMm0uaDog
SW4gZnVuY3Rpb24g4oCYcDJtX3ZtX2V2ZW50X3Nhbml0eV9jaGVja+KAmToKLi9hcmNoL2Fy
bS9pbmNsdWRlL2FzbS9wMm0uaDo0MTc6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxNyB8IHN0YXRpYyBpbmxpbmUgYm9v
bCBwMm1fdm1fZXZlbnRfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+
fn5+fn5+XgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2RvbWFpbi5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZjcHVfYmxvY2vigJk6Ci4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6
Mjk3OjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAyOTcgfCBzdGF0aWMgaW5saW5lIHZvaWQgYXJjaF92Y3B1X2Jsb2NrKHN0
cnVjdCB2Y3B1ICp2KSB7fQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fl4KLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9kb21haW4uaDog
SW4gZnVuY3Rpb24g4oCYdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3nigJk6Ci4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzA5OjYwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmHbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMDkgfCBzdGF0aWMgaW5saW5l
IHZvaWQgdXBkYXRlX2d1ZXN0X21lbW9yeV9wb2xpY3koc3RydWN0IHZjcHUgKnYsCiAgICAg
IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5eCi4vYXJjaC9hcm0vaW5jbHVkZS9hc20vZG9tYWluLmg6MzEwOjc1OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGdtcOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDMxMCB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBz
dHJ1Y3QgZ3Vlc3RfbWVtb3J5X3BvbGljeSAqZ21wKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+CkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2luY2x1ZGUveGVuL3NjaGVkLmg6
MjU6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOiBJbiBmdW5jdGlvbiDigJh2cGNpX2Fzc2lnbl9k
ZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI3ODo1NDogd2FybmluZzogdW51c2Vk
IHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjc4IHwgc3Rh
dGljIGlubGluZSBpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+
fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDogSW4gZnVuY3Rpb24g4oCYdnBj
aV9kZWFzc2lnbl9kZXZpY2XigJk6Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4Mzo1Nzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhwZGV24oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMjgzIHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KSB7IH0KICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNp
Lmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfcmVhZOKAmToKLi9pbmNsdWRlL3hlbi92cGNpLmg6
Mjg3OjQ1OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmHNiZGbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVhZChw
Y2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5efn5+Ci4vaW5jbHVkZS94ZW4vdnBj
aS5oOjI4Nzo2NDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhyZWfigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICAyODcgfCBzdGF0aWMgaW5saW5lIHVpbnQzMl90IHZwY2lfcmVh
ZChwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVkIGludCByZWcsCiAgICAgIHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI4ODo0Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhzaXpl4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjg4IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fl5+fn4KLi9pbmNs
dWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKAmHZwY2lfd3JpdGXigJk6Ci4vaW5jbHVk
ZS94ZW4vdnBjaS5oOjI5NDo0Mjogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhzYmRm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZw
Y2lfd3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+Xn5+fgouL2luY2x1ZGUv
eGVuL3ZwY2kuaDoyOTQ6NjE6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYcmVn4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk0IHwgc3RhdGljIGlubGluZSB2b2lkIHZwY2lf
d3JpdGUocGNpX3NiZGZfdCBzYmRmLCB1bnNpZ25lZCBpbnQgcmVnLAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveGVuL3ZwY2kuaDoyOTU6NDQ6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYc2l6ZeKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5NSB8ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBzaXplLCB1aW50MzJfdCBkYXRh
KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5e
fn5+Ci4vaW5jbHVkZS94ZW4vdnBjaS5oOjI5NTo1OTogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkYXRh4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjk1IHwgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNpemUsIHVpbnQzMl90IGRhdGEp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIH5+fn5+fn5+fl5+fn4KLi9pbmNsdWRlL3hlbi92cGNpLmg6IEluIGZ1bmN0aW9uIOKA
mHZwY2lfcHJvY2Vzc19wZW5kaW5n4oCZOgouL2luY2x1ZGUveGVuL3ZwY2kuaDozMDA6Njc6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYduKAmSBbLVd1bnVzZWQtcGFyYW1ldGVy
XQogIDMwMCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBfX211c3RfY2hlY2sgdnBjaV9wcm9jZXNz
X3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+XgouL2luY2x1
ZGUveGVuL3NjaGVkLmg6IEluIGZ1bmN0aW9uIOKAmGlzX3B2XzMyYml0X2RvbWFpbuKAmToK
Li9pbmNsdWRlL3hlbi9zY2hlZC5oOjExMzE6Njc6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYZOKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogMTEzMSB8IHN0YXRpYyBhbHdheXNf
aW5saW5lIGJvb2wgaXNfcHZfMzJiaXRfZG9tYWluKGNvbnN0IHN0cnVjdCBkb21haW4gKmQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fn5+fn5+fn5+fn5+fn5+fn5+fn5+XgpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9pbmNsdWRl
L3hlbi9ldmVudC5oOjE3LAogICAgICAgICAgICAgICAgIGZyb20gY29tbW9uL3NjaGVkL2Nv
cmUuYzoyMzoKLi9hcmNoL2FybS9pbmNsdWRlL2FzbS9ldmVudC5oOiBJbiBmdW5jdGlvbiDi
gJhhcmNoX3ZpcnFfaXNfZ2xvYmFs4oCZOgouL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2V2ZW50
Lmg6NTA6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdmlyceKAmSBbLVd1bnVz
ZWQtcGFyYW1ldGVyXQogICA1MCB8IHN0YXRpYyBpbmxpbmUgYm9vbCBhcmNoX3ZpcnFfaXNf
Z2xvYmFsKHVuc2lnbmVkIGludCB2aXJxKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5efn5+CkluIGZpbGUgaW5jbHVkZWQg
ZnJvbSAuL2luY2x1ZGUveGVuL2h5cGVyY2FsbC5oOjIyLAogICAgICAgICAgICAgICAgIGZy
b20gY29tbW9uL3NjaGVkL2NvcmUuYzozMjoKLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVu
Y3Rpb24g4oCYeHNtX2RvbWFpbl9jcmVhdGXigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjI0
OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDIyNCB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCwgdWludDMyX3Qgc3NpZHJlZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9p
bmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dldGRvbWFpbmluZm/igJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MjI5OjUxOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDIyOSB8IHN0YXRpYyBpbmxpbmUg
aW50IHhzbV9nZXRkb21haW5pbmZvKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9k
b21jdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjIzNToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyMzUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGludCBj
bWQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6
IEluIGZ1bmN0aW9uIOKAmHhzbV9zeXNjdGxfc2NoZWR1bGVyX29w4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI0MDo1Nzogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyNDAgfCBzdGF0aWMgaW5saW5lIGludCB4c21fc3lz
Y3RsX3NjaGVkdWxlcl9vcCh4c21fZGVmYXVsdF90IGRlZiwgaW50IGNtZCkKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fc2V0X3Rhcmdl
dOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNDY6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjQ2IHwgICAgIHhzbV9k
ZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgZG9tYWluICplKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZG9tY3Rs4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI1MTo0NDogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICAyNTEgfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tY3RsKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgY21kKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX3N5c2N0bOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNTY6NDQ6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMjU2IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3N5c2N0bCh4c21fZGVmYXVsdF90
IGRlZiwgaW50IGNtZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+
fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhz
bV9yZWFkY29uc29sZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDoyNjE6NDk6IHdhcm5pbmc6
IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMjYx
IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3JlYWRjb25zb2xlKHhzbV9kZWZhdWx0X3QgZGVm
LCB1aW50MzJfdCBjbGVhcikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rp
b24g4oCYeHNtX2V2dGNobl91bmJvdW5k4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI2Nzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICAyNjcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQx
LCBzdHJ1Y3QgZXZ0Y2huICpjaG4sIGRvbWlkX3QgaWQyKQogICAgICB8ICAgICB+fn5+fn5+
fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0
Y2huX2ludGVyZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjI3MzoxOTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAy
NzMgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3Qg
ZXZ0Y2huICpjaGFuMSwKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRl
L3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2V2dGNobl9zZW5k4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjI4NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAyODUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBldnRjaG4gKmNobikKICAgICAgfCAgICAgfn5+
fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNt
X2V2dGNobl9zdGF0dXPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MjkxOjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDI5
MSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGV2
dGNobiAqY2huKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZXZ0Y2huX3Jlc2V04oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjI5NzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICAyOTcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dy
YW50X21hcHJlZuKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMDM6MTk6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzAzIHwg
ICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFp
biAqZDIsIHVpbnQzMl90IGZsYWdzKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdW5tYXByZWbi
gJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzA5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1l
dGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMwOSB8ICAgICB4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAg
ICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5j
dGlvbiDigJh4c21fZ3JhbnRfc2V0dXDigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6MzE1OjE5
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDMxNSB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEs
IHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2lu
Y2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fZ3JhbnRfdHJhbnNmZXLigJk6
Ci4vaW5jbHVkZS94c20veHNtLmg6MzIxOjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDMyMSB8ICAgICB4c21fZGVmYXVs
dF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fZ3JhbnRfY29weeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozMjc6MTk6IHdh
cm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAgMzI3IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3Ry
dWN0IGRvbWFpbiAqZDIpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9ncmFudF9xdWVyeV9zaXpl4oCZOgou
L2luY2x1ZGUveHNtL3hzbS5oOjMzMzoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDi
gJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzMzMgfCAgICAgeHNtX2RlZmF1bHRf
dCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX2luaXRfaGFyZHdhcmVfZG9tYWlu4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM2
Njo1ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICAzNjYgfCBzdGF0aWMgaW5saW5lIGludCB4c21faW5pdF9oYXJkd2FyZV9k
b21haW4oeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9nZXRfcG9kX3Rh
cmdldOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDozNzE6NTI6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzcxIHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2dldF9wb2RfdGFyZ2V0KHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1
Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlv
biDigJh4c21fc2V0X3BvZF90YXJnZXTigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6Mzc2OjUy
OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1l
dGVyXQogIDM3NiB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9zZXRfcG9kX3RhcmdldCh4c21f
ZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9leGNoYW5nZeKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDozODE6NTM6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMzgxIHwgc3RhdGljIGlubGluZSBpbnQgeHNt
X21lbW9yeV9leGNoYW5nZSh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkK
ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1v
cnlfYWRqdXN0X3Jlc2VydmF0aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM4NzoxOTog
d2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRl
cl0KICAzODcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBz
dHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNs
dWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9zdGF0X3Jlc2VydmF0
aW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjM5MzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAzOTMgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikK
ICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4g
ZnVuY3Rpb24g4oCYeHNtX21lbW9yeV9waW5fcGFnZeKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDozOTk6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgMzk5IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIsCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9hZGRfdG9fcGh5
c21hcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MDY6MTk6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDA2IHwgICAgIHhz
bV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkMSwgc3RydWN0IGRvbWFpbiAqZDIp
CiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IElu
IGZ1bmN0aW9uIOKAmHhzbV9yZW1vdmVfZnJvbV9waHlzbWFw4oCZOgouL2luY2x1ZGUveHNt
L3hzbS5oOjQxMjoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1X
dW51c2VkLXBhcmFtZXRlcl0KICA0MTIgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVj
dCBkb21haW4gKmQxLCBzdHJ1Y3QgZG9tYWluICpkMikKICAgICAgfCAgICAgfn5+fn5+fn5+
fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9n
bWZuX2ZvcmVpZ27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDE4OjE5OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQxOCB8
ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGRvbWFp
biAqdCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2NsYWltX3BhZ2Vz4oCZOgouL2luY2x1ZGUveHNtL3hz
bS5oOjQyMzo0OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51
c2VkLXBhcmFtZXRlcl0KICA0MjMgfCBzdGF0aWMgaW5saW5lIGludCB4c21fY2xhaW1fcGFn
ZXMoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQpCiAgICAgIHwgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9jb25zb2xlX2lv4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjQyODo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0MjggfCBzdGF0aWMgaW5saW5lIGludCB4c21f
Y29uc29sZV9pbyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IGNt
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcHJvZmls
ZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0MzM6NDU6IHdhcm5pbmc6IHVudXNlZCBwYXJh
bWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDMzIHwgc3RhdGljIGlu
bGluZSBpbnQgeHNtX3Byb2ZpbGUoeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIGludCBvcCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+
fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21f
a2V4ZWPigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDM4OjQzOiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQzOCB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9rZXhlYyh4c21fZGVmYXVsdF90IGRlZikKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hz
bS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3NjaGVkb3Bfc2h1dGRvd27igJk6Ci4vaW5j
bHVkZS94c20veHNtLmg6NDQ0OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRl
ZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ0NCB8ICAgICB4c21fZGVmYXVsdF90IGRl
Ziwgc3RydWN0IGRvbWFpbiAqZDEsIHN0cnVjdCBkb21haW4gKmQyKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fbWFwX2RvbWFpbl9waXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ1NDo1Mzogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA0NTQgfCBzdGF0aWMgaW5saW5lIGludCB4c21fbWFwX2RvbWFpbl9waXJxKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94
c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX21hcF9kb21haW5faXJx4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjQ2MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA0NjAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGludCBpcnEsIHZvaWQgKmRhdGEpCiAgICAgIHwgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91
bm1hcF9kb21haW5fcGlyceKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo0NjU6NTU6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAg
NDY1IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3VubWFwX2RvbWFpbl9waXJxKHhzbV9kZWZh
dWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNt
L3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fdW5tYXBfZG9tYWluX2lyceKAmToKLi9pbmNs
dWRlL3hzbS94c20uaDo0NzE6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm
4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNDcxIHwgICAgIHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkLCBpbnQgaXJxLCB2b2lkICpkYXRhKQogICAgICB8ICAgICB+
fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4
c21fYmluZF9wdF9pcnHigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NDc3OjE5OiB3YXJuaW5n
OiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ3
NyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHhl
bl9kb21jdGxfYmluZF9wdF9pcnEgKmJpbmQpCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+
Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV91bmJpbmRfcHRf
aXJx4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ4MzoxOTogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA0ODMgfCAgICAgeHNt
X2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCB4ZW5fZG9tY3RsX2Jp
bmRfcHRfaXJxICpiaW5kKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1
ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21faXJxX3Blcm1pc3Npb27igJk6Ci4v
aW5jbHVkZS94c20veHNtLmg6NDg5OjE5OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKA
mGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDQ4OSB8ICAgICB4c21fZGVmYXVsdF90
IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgaW50IHBpcnEsIHVpbnQ4X3QgYWxsb3cpCiAgICAg
IHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0
aW9uIOKAmHhzbV9pb21lbV9wZXJtaXNzaW9u4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjQ5
NToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBh
cmFtZXRlcl0KICA0OTUgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4g
KmQsIHVpbnQ2NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9pb21lbV9tYXBwaW5n4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUwMToxOTogd2Fy
bmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0K
ICA1MDEgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIHVpbnQ2
NF90IHMsIHVpbnQ2NF90IGUsIHVpbnQ4X3QgYWxsb3cpCiAgICAgIHwgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wY2lf
Y29uZmlnX3Blcm1pc3Npb27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTA3OjE5OiB3YXJu
aW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQog
IDUwNyB8ICAgICB4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCwgdWludDMy
X3QgbWFjaGluZV9iZGYsIHVpbnQxNl90IHN0YXJ0LAogICAgICB8ICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fYXNzaWdu
X2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjUzNDoxOTogd2FybmluZzogdW51
c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1MzQgfCAg
ICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0
cGF0aCkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20u
aDogSW4gZnVuY3Rpb24g4oCYeHNtX2RlYXNzaWduX2R0ZGV2aWNl4oCZOgouL2luY2x1ZGUv
eHNtL3hzbS5oOjU0MDoxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkg
Wy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NDAgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0
cnVjdCBkb21haW4gKmQsIGNvbnN0IGNoYXIgKmR0cGF0aCkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jl
c291cmNlX3BsdWdfcGNp4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU0Nzo1NTogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1
NDcgfCBzdGF0aWMgaW5saW5lIGludCB4c21fcmVzb3VyY2VfcGx1Z19wY2koeHNtX2RlZmF1
bHRfdCBkZWYsIHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUv
eHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2VfdW5wbHVnX3BjaeKAmToK
Li9pbmNsdWRlL3hzbS94c20uaDo1NTM6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg
4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTUzIHwgICAgIHhzbV9kZWZhdWx0
X3QgZGVmLCB1aW50MzJfdCBtYWNoaW5lX2JkZikKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX3Jlc291cmNl
X3BsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NTg6NTY6IHdhcm5pbmc6IHVu
dXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTU4IHwg
c3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3BsdWdfY29yZSh4c21fZGVmYXVsdF90
IGRlZikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKA
mHhzbV9yZXNvdXJjZV91bnBsdWdfY29yZeKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo1NjM6
NTg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJh
bWV0ZXJdCiAgNTYzIHwgc3RhdGljIGlubGluZSBpbnQgeHNtX3Jlc291cmNlX3VucGx1Z19j
b3JlKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hz
bS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBfcGNp4oCZOgouL2luY2x1
ZGUveHNtL3hzbS5oOjU2OToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbi
gJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYs
IHVpbnQzMl90IG1hY2hpbmVfYmRmKQogICAgICB8ICAgICB+fn5+fn5+fn5+fn5+fl5+fgou
L2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcmVzb3VyY2Vfc2V0dXBf
Z3Np4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjU3NDo1Njogd2FybmluZzogdW51c2VkIHBh
cmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA1NzQgfCBzdGF0aWMg
aW5saW5lIGludCB4c21fcmVzb3VyY2Vfc2V0dXBfZ3NpKHhzbV9kZWZhdWx0X3QgZGVmLCBp
bnQgZ3NpKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfn5+fn5+fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g
4oCYeHNtX3Jlc291cmNlX3NldHVwX21pc2PigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NTc5
OjU3OiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFy
YW1ldGVyXQogIDU3OSB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9yZXNvdXJjZV9zZXR1cF9t
aXNjKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNt
Lmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9wYWdlX29mZmxpbmXigJk6Ci4vaW5jbHVkZS94c20v
eHNtLmg6NTg0OjUwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1
bnVzZWQtcGFyYW1ldGVyXQogIDU4NCB8IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9wYWdlX29m
ZmxpbmUoeHNtX2RlZmF1bHRfdCBkZWYsIHVpbnQzMl90IGNtZCkKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVk
ZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9oeXBmc19vcOKAmToKLi9pbmNsdWRl
L3hzbS94c20uaDo1ODk6NDY6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZ
IFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNTg5IHwgc3RhdGljIGlubGluZSBpbnQgeHNtX2h5
cGZzX29wKHhzbV9kZWZhdWx0X3QgZGVmKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faHZtX3BhcmFt4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYwNzox
OTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFt
ZXRlcl0KICA2MDcgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21haW4gKmQs
IHVuc2lnbmVkIGxvbmcgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5j
bHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9odm1fcGFyYW1fYWx0cDJtaHZt
4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjYxMjo1Nzogd2FybmluZzogdW51c2VkIHBhcmFt
ZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MTIgfCBzdGF0aWMgaW5s
aW5lIGludCB4c21faHZtX3BhcmFtX2FsdHAybWh2bSh4c21fZGVmYXVsdF90IGRlZiwgc3Ry
dWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBm
dW5jdGlvbiDigJh4c21faHZtX2FsdHAybWh2bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20u
aDo2MTg6MTk6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNl
ZC1wYXJhbWV0ZXJdCiAgNjE4IHwgICAgIHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9t
YWluICpkLCB1aW50NjRfdCBtb2RlLCB1aW50MzJfdCBvcCkKICAgICAgfCAgICAgfn5+fn5+
fn5+fn5+fn5efn4KLi9pbmNsdWRlL3hzbS94c20uaDogSW4gZnVuY3Rpb24g4oCYeHNtX2dl
dF92bnVtYWluZm/igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjIzOjUxOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDYyMyB8
IHN0YXRpYyBpbmxpbmUgaW50IHhzbV9nZXRfdm51bWFpbmZvKHhzbV9kZWZhdWx0X3QgZGVm
LCBzdHJ1Y3QgZG9tYWluICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1
bmN0aW9uIOKAmHhzbV92bV9ldmVudF9jb250cm9s4oCZOgouL2luY2x1ZGUveHNtL3hzbS5o
OjYyOToxOTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICA2MjkgfCAgICAgeHNtX2RlZmF1bHRfdCBkZWYsIHN0cnVjdCBkb21h
aW4gKmQsIGludCBtb2RlLCBpbnQgb3ApCiAgICAgIHwgICAgIH5+fn5+fn5+fn5+fn5+Xn5+
Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9tZW1fYWNjZXNz4oCZ
OgouL2luY2x1ZGUveHNtL3hzbS5oOjYzNTo0ODogd2FybmluZzogdW51c2VkIHBhcmFtZXRl
ciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA2MzUgfCBzdGF0aWMgaW5saW5l
IGludCB4c21fbWVtX2FjY2Vzcyh4c21fZGVmYXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAq
ZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+
fn5+fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21fcGxhdGZv
cm1fb3DigJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NjU1OjQ5OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDY1NSB8IHN0YXRp
YyBpbmxpbmUgaW50IHhzbV9wbGF0Zm9ybV9vcCh4c21fZGVmYXVsdF90IGRlZiwgdWludDMy
X3Qgb3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fn5+fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9k
bV9vcOKAmToKLi9pbmNsdWRlL3hzbS94c20uaDo3Mzg6NDM6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYZGVm4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgNzM4IHwgc3RhdGlj
IGlubGluZSBpbnQgeHNtX2RtX29wKHhzbV9kZWZhdWx0X3QgZGVmLCBzdHJ1Y3QgZG9tYWlu
ICpkKQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+
fl5+fgouL2luY2x1ZGUveHNtL3hzbS5oOiBJbiBmdW5jdGlvbiDigJh4c21feGVuX3ZlcnNp
b27igJk6Ci4vaW5jbHVkZS94c20veHNtLmg6NzQzOjQ5OiB3YXJuaW5nOiB1bnVzZWQgcGFy
YW1ldGVyIOKAmGRlZuKAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDc0MyB8IHN0YXRpYyBp
bmxpbmUgaW50IHhzbV94ZW5fdmVyc2lvbih4c21fZGVmYXVsdF90IGRlZiwgdWludDMyX3Qg
b3ApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+
fn5+fn5+Xn5+Ci4vaW5jbHVkZS94c20veHNtLmg6IEluIGZ1bmN0aW9uIOKAmHhzbV9kb21h
aW5fcmVzb3VyY2VfbWFw4oCZOgouL2luY2x1ZGUveHNtL3hzbS5oOjc0ODo1Nzogd2Fybmlu
ZzogdW51c2VkIHBhcmFtZXRlciDigJhkZWbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICA3
NDggfCBzdGF0aWMgaW5saW5lIGludCB4c21fZG9tYWluX3Jlc291cmNlX21hcCh4c21fZGVm
YXVsdF90IGRlZiwgc3RydWN0IGRvbWFpbiAqZCkKICAgICAgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fl5+fgpjb21tb24vc2No
ZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhpc191bml0X29ubGluZeKAmToKLi9pbmNs
dWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2Vy
IGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKA
mHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAg
ICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9p
ZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgXgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOjkxOjU6IG5vdGU6IGluIGV4cGFuc2lvbiBv
ZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAgOTEgfCAgICAgZm9y
X2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+
fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDi
gJh1bml0X3J1bm5hYmxl4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJu
aW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNp
Z25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29t
cGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8
ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
MTA4OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91
bml0X3ZjcHXigJkKICAxMDggfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5p
dCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2No
ZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJh1bml0X3J1bm5hYmxlX3N0YXRl4oCZOgou
L2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGlu
dGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBh
bmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51
bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTM1OjU6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxMzUgfCAg
ICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5j
dGlvbiDigJhzY2hlZF9zZXRfcmVz4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQw
OiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVy
ZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3Np
Z24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNw
dV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2
YXRlLmg6MTU0OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9z
Y2hlZF91bml0X3ZjcHXigJkKICAxNTQgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1
ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21t
b24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhzY2hlZF9zZXRfcGF1c2VfZmxh
Z3PigJk6Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6IHdhcm5pbmc6IGNvbXBhcmlz
b24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCY
aW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogIDM1NyB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1X2lkIDwgKHUpLT5uZXh0X2lu
X2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL3ByaXZhdGUuaDoxNzY6NTogbm90ZTog
aW4gZXhwYW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQog
IDE3NiB8ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAg
fCAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6
IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2NsZWFyX3BhdXNlX2ZsYWdz4oCZOgouL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsg
ICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBe
CmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6MTg2OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBt
YWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICAxODYgfCAgICAgZm9yX2Vh
Y2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvcHJpdmF0ZS5oOiBJbiBmdW5jdGlvbiDigJhz
Y2hlZF91bml0X3BhdXNlX25vc3luY+KAmToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0
MDogd2FybmluZzogY29tcGFyaXNvbiBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZl
cmVudCBzaWduZWRuZXNzOiDigJhpbnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdz
aWduLWNvbXBhcmVdCiAgMzU3IHwgICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZj
cHVfaWQgPCAodSktPm5leHRfaW5fbGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvcHJp
dmF0ZS5oOjUzMDo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hf
c2NoZWRfdW5pdF92Y3B14oCZCiAgNTMwIHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNw
dSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29t
bW9uL3NjaGVkL3ByaXZhdGUuaDogSW4gZnVuY3Rpb24g4oCYc2NoZWRfdW5pdF91bnBhdXNl
4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29u
IG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGlu
dOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9s
aXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9wcml2YXRlLmg6NTM4OjU6IG5vdGU6IGlu
IGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICA1
MzggfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwg
ICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvY29yZS5jOiBJbiBm
dW5jdGlvbiDigJhzY2hlZF9pZGxlX3Jlc19waWNr4oCZOgpjb21tb24vc2NoZWQvY29yZS5j
OjEwMjo0NTogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJhvcHPigJkgWy1XdW51c2Vk
LXBhcmFtZXRlcl0KICAxMDIgfCBzY2hlZF9pZGxlX3Jlc19waWNrKGNvbnN0IHN0cnVjdCBz
Y2hlZHVsZXIgKm9wcywgY29uc3Qgc3RydWN0IHNjaGVkX3VuaXQgKnVuaXQpCiAgICAgIHwg
ICAgICAgICAgICAgICAgICAgICB+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KY29tbW9u
L3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRfaWRsZV9hbGxvY191ZGF0YeKA
mToKY29tbW9uL3NjaGVkL2NvcmUuYzoxMDg6NDg6IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0
ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTA4IHwgc2NoZWRfaWRsZV9h
bGxvY191ZGF0YShjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHN0cnVjdCBzY2hlZF91
bml0ICp1bml0LAogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jb3JlLmM6MTA4OjcyOiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmHVuaXTigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMDgg
fCBzY2hlZF9pZGxlX2FsbG9jX3VkYXRhKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywg
c3RydWN0IHNjaGVkX3VuaXQgKnVuaXQsCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fn5+fn5+fn5+fn5efn5+
CmNvbW1vbi9zY2hlZC9jb3JlLmM6MTA5OjMwOiB3YXJuaW5nOiB1bnVzZWQgcGFyYW1ldGVy
IOKAmGRk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTA5IHwgICAgICAgICAgICAgICAg
ICAgICAgICB2b2lkICpkZCkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+
fl5+CmNvbW1vbi9zY2hlZC9jb3JlLmM6IEluIGZ1bmN0aW9uIOKAmHNjaGVkX2lkbGVfZnJl
ZV91ZGF0YeKAmToKY29tbW9uL3NjaGVkL2NvcmUuYzoxMTY6NDc6IHdhcm5pbmc6IHVudXNl
ZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTE2IHwgc2No
ZWRfaWRsZV9mcmVlX3VkYXRhKGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgdm9pZCAq
cHJpdikKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgfn5+fn5+fn5+fn5+fn5+fn5+
fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jb3JlLmM6MTE2OjU4OiB3YXJuaW5nOiB1bnVzZWQg
cGFyYW1ldGVyIOKAmHByaXbigJkgWy1XdW51c2VkLXBhcmFtZXRlcl0KICAxMTYgfCBzY2hl
ZF9pZGxlX2ZyZWVfdWRhdGEoY29uc3Qgc3RydWN0IHNjaGVkdWxlciAqb3BzLCB2b2lkICpw
cml2KQogICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIH5+fn5+fl5+fn4KY29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g
4oCYc2NoZWRfaWRsZV9zY2hlZHVsZeKAmToKY29tbW9uL3NjaGVkL2NvcmUuYzoxMjE6Mjk6
IHdhcm5pbmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYb3Bz4oCZIFstV3VudXNlZC1wYXJhbWV0
ZXJdCiAgMTIxIHwgICAgIGNvbnN0IHN0cnVjdCBzY2hlZHVsZXIgKm9wcywgc3RydWN0IHNj
aGVkX3VuaXQgKnVuaXQsIHNfdGltZV90IG5vdywKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn5+Xn5+CmNvbW1vbi9zY2hlZC9jb3JlLmM6MTIxOjY4OiB3YXJuaW5nOiB1
bnVzZWQgcGFyYW1ldGVyIOKAmG5vd+KAmSBbLVd1bnVzZWQtcGFyYW1ldGVyXQogIDEyMSB8
ICAgICBjb25zdCBzdHJ1Y3Qgc2NoZWR1bGVyICpvcHMsIHN0cnVjdCBzY2hlZF91bml0ICp1
bml0LCBzX3RpbWVfdCBub3csCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH5+fn5+fn5+fl5+fgpjb21tb24vc2No
ZWQvY29yZS5jOjEyMjoxMDogd2FybmluZzogdW51c2VkIHBhcmFtZXRlciDigJh0YXNrbGV0
X3dvcmtfc2NoZWR1bGVk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAgMTIyIHwgICAgIGJv
b2wgdGFza2xldF93b3JrX3NjaGVkdWxlZCkKY29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVu
Y3Rpb24g4oCYc2NoZWRfZnJlZV91bml04oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3
OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+
dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9j
b3JlLmM6NDE1OjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9z
Y2hlZF91bml0X3ZjcHXigJkKICA0MTUgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1
ICggdW5pdCwgdnVuaXQgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4K
Y29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRfbW92ZV9pcnFz4oCZ
OgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9m
IGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKA
mSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAg
ICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0
LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jb3JlLmM6NjUxOjU6IG5vdGU6IGluIGV4cGFu
c2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHXigJkKICA2NTEgfCAg
ICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICggdW5pdCwgdiApCiAgICAgIHwgICAgIF5+
fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvY29yZS5jOiBJbiBmdW5jdGlv
biDigJhzY2hlZF9tb3ZlX2RvbWFpbuKAmToKY29tbW9uL3NjaGVkL2NvcmUuYzo3ODI6MzU6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYdW5zaWduZWQgaW504oCZIGFuZCDigJhpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDc4MiB8ICAgICAgICAgaWYgKCB1bml0LT51bml0X2lkICsgZ3JhbiA9
PSB2LT52Y3B1X2lkICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgXn4KLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNv
biBvZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhp
bnTigJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwg
ICAgICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5f
bGlzdC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvY29yZS5jOjgwMTo5OiBub3RlOiBpbiBl
eHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAgODAx
IHwgICAgICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAg
fCAgICAgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgpjb21tb24vc2NoZWQvY29yZS5j
OiBJbiBmdW5jdGlvbiDigJhzY2hlZF91bml0X21vdmVfbG9ja2Vk4oCZOgouL2luY2x1ZGUv
eGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhw
cmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5z
aWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsg
ICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBe
CmNvbW1vbi9zY2hlZC9jb3JlLmM6MTAzOTo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFj
cm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAxMDM5IHwgICAgIGZvcl9lYWNo
X3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+
fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRf
dW5pdF9taWdyYXRlX3N0YXJ04oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3
YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50
IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFstV3NpZ24t
Y29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+dmNwdV9p
ZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9jb3JlLmM6
MTA4ODo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRf
dW5pdF92Y3B14oCZCiAxMDg4IHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVu
aXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29tbW9uL3Nj
aGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRfdW5pdF9taWdyYXRlX2ZpbmlzaOKA
mToKLi9pbmNsdWRlL3hlbi9zY2hlZC5oOjM1Nzo0MDogd2FybmluZzogY29tcGFyaXNvbiBv
ZiBpbnRlZ2VyIGV4cHJlc3Npb25zIG9mIGRpZmZlcmVudCBzaWduZWRuZXNzOiDigJhpbnTi
gJkgYW5kIOKAmHVuc2lnbmVkIGludOKAmSBbLVdzaWduLWNvbXBhcmVdCiAgMzU3IHwgICAg
ICAgICAgICAgICAgICAgICAgICAgICAodiktPnZjcHVfaWQgPCAodSktPm5leHRfaW5fbGlz
dC0+dW5pdF9pZCk7ICAgICBcCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgXgpjb21tb24vc2NoZWQvY29yZS5jOjExMTA6NTogbm90ZTogaW4gZXhw
YW5zaW9uIG9mIG1hY3JvIOKAmGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNwdeKAmQogMTExMCB8
ICAgICBmb3JfZWFjaF9zY2hlZF91bml0X3ZjcHUgKCB1bml0LCB2ICkKICAgICAgfCAgICAg
Xn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+Ci4vaW5jbHVkZS94ZW4vc2NoZWQuaDozNTc6NDA6
IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJl
bnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2ln
bi1jb21wYXJlXQogIDM1NyB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHYpLT52Y3B1
X2lkIDwgKHUpLT5uZXh0X2luX2xpc3QtPnVuaXRfaWQpOyAgICAgXAogICAgICB8ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4KY29tbW9uL3NjaGVkL2NvcmUu
YzoxMTcwOjU6IG5vdGU6IGluIGV4cGFuc2lvbiBvZiBtYWNybyDigJhmb3JfZWFjaF9zY2hl
ZF91bml0X3ZjcHXigJkKIDExNzAgfCAgICAgZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B1ICgg
dW5pdCwgdiApCiAgICAgIHwgICAgIF5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgouL2luY2x1
ZGUveGVuL3NjaGVkLmg6MzU3OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIg
ZXhwcmVzc2lvbnMgb2YgZGlmZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCY
dW5zaWduZWQgaW504oCZIFstV3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAg
ICAgICAgICAgICAgICh2KS0+dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lk
KTsgICAgIFwKICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBeCmNvbW1vbi9zY2hlZC9jb3JlLmM6MTE4Nzo1OiBub3RlOiBpbiBleHBhbnNpb24gb2Yg
bWFjcm8g4oCYZm9yX2VhY2hfc2NoZWRfdW5pdF92Y3B14oCZCiAxMTg3IHwgICAgIGZvcl9l
YWNoX3NjaGVkX3VuaXRfdmNwdSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+
fn5+fn5+fn5+fn5+fn4KY29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2No
ZWRfY2hlY2tfYWZmaW5pdHlfYnJva2Vu4oCZOgouL2luY2x1ZGUveGVuL3NjaGVkLmg6MzU3
OjQwOiB3YXJuaW5nOiBjb21wYXJpc29uIG9mIGludGVnZXIgZXhwcmVzc2lvbnMgb2YgZGlm
ZmVyZW50IHNpZ25lZG5lc3M6IOKAmGludOKAmSBhbmQg4oCYdW5zaWduZWQgaW504oCZIFst
V3NpZ24tY29tcGFyZV0KICAzNTcgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICh2KS0+
dmNwdV9pZCA8ICh1KS0+bmV4dF9pbl9saXN0LT51bml0X2lkKTsgICAgIFwKICAgICAgfCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeCmNvbW1vbi9zY2hlZC9j
b3JlLmM6MTE5NTo1OiBub3RlOiBpbiBleHBhbnNpb24gb2YgbWFjcm8g4oCYZm9yX2VhY2hf
c2NoZWRfdW5pdF92Y3B14oCZCiAxMTk1IHwgICAgIGZvcl9lYWNoX3NjaGVkX3VuaXRfdmNw
dSAoIHVuaXQsIHYgKQogICAgICB8ICAgICBefn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KY29t
bW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYZG9tYWluX3dhdGNoZG9n4oCZOgpj
b21tb24vc2NoZWQvY29yZS5jOjE1NjI6NTA6IHdhcm5pbmc6IG9wZXJhbmQgb2Yg4oCYPzri
gJkgY2hhbmdlcyBzaWduZWRuZXNzIGZyb20g4oCYaW504oCZIHRvIOKAmHVpbnQzMl904oCZ
IHtha2Eg4oCYdW5zaWduZWQgaW504oCZfSBkdWUgdG8gdW5zaWduZWRuZXNzIG9mIG90aGVy
IG9wZXJhbmQgWy1Xc2lnbi1jb21wYXJlXQogMTU2MiB8ICAgICAgICAgcmV0dXJuIGlkID09
IE5SX0RPTUFJTl9XQVRDSERPR19USU1FUlMgPyAtRU5PU1BDIDogaWQgKyAxOwogICAgICB8
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefn5+
fn5+CmNvbW1vbi9zY2hlZC9jb3JlLmM6IEluIGZ1bmN0aW9uIOKAmHZjcHVfdGVtcG9yYXJ5
X2FmZmluaXR54oCZOgpjb21tb24vc2NoZWQvY29yZS5jOjE2NDA6NjU6IHdhcm5pbmc6IGNv
bXBhcmlzb24gb2YgaW50ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVz
czog4oCYaW504oCZIGFuZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQog
MTY0MCB8ICAgICAgICAgICAgICAoc2NoZWRfY2hlY2tfYWZmaW5pdHlfYnJva2VuKHVuaXQp
ICYmIHYtPnByb2Nlc3NvciAhPSBjcHUpICkKICAgICAgfCAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4KY29tbW9u
L3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRfc3dpdGNoX3VuaXRz4oCZOgpj
b21tb24vc2NoZWQvY29yZS5jOjIyMTk6NTQ6IHdhcm5pbmc6IGNvbXBhcmlzb24gb2YgaW50
ZWdlciBleHByZXNzaW9ucyBvZiBkaWZmZXJlbnQgc2lnbmVkbmVzczog4oCYaW504oCZIGFu
ZCDigJh1bnNpZ25lZCBpbnTigJkgWy1Xc2lnbi1jb21wYXJlXQogMjIxOSB8ICAgICAgICAg
aWYgKCB2cHJldiAhPSB2bmV4dCB8fCB2cHJldi0+cnVuc3RhdGUuc3RhdGUgIT0gdm5leHQt
Pm5ld19zdGF0ZSApCiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBefgpjb21tb24vc2NoZWQvY29yZS5jOiBJbiBmdW5jdGlv
biDigJhzX3RpbWVyX2Zu4oCZOgpjb21tb24vc2NoZWQvY29yZS5jOjI3MTQ6Mzk6IHdhcm5p
bmc6IHVudXNlZCBwYXJhbWV0ZXIg4oCYdW51c2Vk4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJd
CiAyNzE0IHwgc3RhdGljIHZvaWQgY2ZfY2hlY2sgc190aW1lcl9mbih2b2lkICp1bnVzZWQp
CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+fn5+fn5efn5+fn4K
Y29tbW9uL3NjaGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYY3B1X3NjaGVkdWxlX2NhbGxi
YWNr4oCZOgpjb21tb24vc2NoZWQvY29yZS5jOjI4NTc6Mjg6IHdhcm5pbmc6IHVudXNlZCBw
YXJhbWV0ZXIg4oCYbmZi4oCZIFstV3VudXNlZC1wYXJhbWV0ZXJdCiAyODU3IHwgICAgIHN0
cnVjdCBub3RpZmllcl9ibG9jayAqbmZiLCB1bnNpZ25lZCBsb25nIGFjdGlvbiwgdm9pZCAq
aGNwdSkKICAgICAgfCAgICAgfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5efn4KY29tbW9uL3Nj
aGVkL2NvcmUuYzogSW4gZnVuY3Rpb24g4oCYc2NoZWRfZ2V0X2lkX2J5X25hbWXigJk6CmNv
bW1vbi9zY2hlZC9jb3JlLmM6Mjk4MDo0Njogd2FybmluZzogb3BlcmFuZCBvZiDigJg/OuKA
mSBjaGFuZ2VzIHNpZ25lZG5lc3MgZnJvbSDigJhpbnTigJkgdG8g4oCYdW5zaWduZWQgaW50
4oCZIGR1ZSB0byB1bnNpZ25lZG5lc3Mgb2Ygb3RoZXIgb3BlcmFuZCBbLVdzaWduLWNvbXBh
cmVdCiAyOTgwIHwgICAgIHJldHVybiBzY2hlZHVsZXIgPyBzY2hlZHVsZXItPnNjaGVkX2lk
IDogLTE7CiAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgXn4KICBJTklUX08gIGNvbW1vbi9zY2hlZC9ib290LWNwdXBvb2wuaW5pdC5vCiAg
TEQgICAgICBjb21tb24vc2NoZWQvYnVpbHRfaW4ubwogIExEICAgICAgY29tbW9uL2J1aWx0
X2luLm8KICBMRCAgICAgIHByZWxpbmsubwpsZCAgICAtRUwgIC0tZml4LWNvcnRleC1hNTMt
ODQzNDE5IC1UIGFyY2gvYXJtL3hlbi5sZHMgLU4gcHJlbGluay5vIFwKICAgIC4vY29tbW9u
L3N5bWJvbHMtZHVtbXkubyAtbyAuLy54ZW4tc3ltcy4wCm5tIC1wYSAtLWZvcm1hdD1zeXN2
IC4vLnhlbi1zeW1zLjAgXAoJfCAuL3Rvb2xzL3N5bWJvbHMgLS1hbGwtc3ltYm9scyAtLXNv
cnQtYnktbmFtZSAtLXN5c3YgLS1zb3J0IFwKCT4gLi8ueGVuLXN5bXMuMC5TCm1ha2UgLWYg
Li9SdWxlcy5tayBvYmo9LiAuLy54ZW4tc3ltcy4wLm8KICBDQyAgICAgIC54ZW4tc3ltcy4w
Lm8KbGQgICAgLUVMICAtLWZpeC1jb3J0ZXgtYTUzLTg0MzQxOSAtVCBhcmNoL2FybS94ZW4u
bGRzIC1OIHByZWxpbmsubyBcCiAgICAuLy54ZW4tc3ltcy4wLm8gLW8gLi8ueGVuLXN5bXMu
MQpubSAtcGEgLS1mb3JtYXQ9c3lzdiAuLy54ZW4tc3ltcy4xIFwKCXwgLi90b29scy9zeW1i
b2xzIC0tYWxsLXN5bWJvbHMgLS1zb3J0LWJ5LW5hbWUgLS1zeXN2IC0tc29ydCBcCgk+IC4v
Lnhlbi1zeW1zLjEuUwptYWtlIC1mIC4vUnVsZXMubWsgb2JqPS4gLi8ueGVuLXN5bXMuMS5v
CiAgQ0MgICAgICAueGVuLXN5bXMuMS5vCmxkICAgIC1FTCAgLS1maXgtY29ydGV4LWE1My04
NDM0MTkgLVQgYXJjaC9hcm0veGVuLmxkcyAtTiBwcmVsaW5rLm8gLS1idWlsZC1pZD1zaGEx
IFwKICAgIC4vLnhlbi1zeW1zLjEubyAtbyB4ZW4tc3ltcwpubSAtcGEgLS1mb3JtYXQ9c3lz
diB4ZW4tc3ltcyBcCgl8IC4vdG9vbHMvc3ltYm9scyAtLWFsbC1zeW1ib2xzIC0teGVuc3lt
cyAtLXN5c3YgLS1zb3J0IFwKCT4geGVuLXN5bXMubWFwCnJtIC1mIC4vLnhlbi1zeW1zLlsw
LTldKgpvYmpjb3B5IC1PIGJpbmFyeSAtUyB4ZW4tc3ltcyB4ZW4KbG4gLXNmIHhlbiB4ZW4u
ZWZpCjM2Ljg0dXNlciAzLjc2c3lzdGVtIDA6MTMuMTllbGFwc2VkIDMwNyVDUFUgKDBhdmd0
ZXh0KzBhdmdkYXRhIDE0NTczMm1heHJlc2lkZW50KWsKNDc3NTk0OGlucHV0cys0Nzc1Nzhv
dXRwdXRzICg1Nm1ham9yKzI3NjY1NDFtaW5vcilwYWdlZmF1bHRzIDBzd2Fwcwo=

--------------p60WVPVwD5u0Q02s6TthpwEk--


From xen-devel-bounces@lists.xenproject.org Tue May 28 06:21:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 06:21:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731001.1136287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqCp-0006Lj-5H; Tue, 28 May 2024 06:21:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731001.1136287; Tue, 28 May 2024 06:21:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqCp-0006Lc-2W; Tue, 28 May 2024 06:21:07 +0000
Received: by outflank-mailman (input) for mailman id 731001;
 Tue, 28 May 2024 06:21:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBqCn-0006LV-NM
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 06:21:05 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 746d3cac-1cba-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 08:21:01 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a61b70394c0so49730166b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 23:21:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93a7fcsm575210366b.53.2024.05.27.23.21.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 23:21:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 746d3cac-1cba-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716877261; x=1717482061; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=z7UcWDt5wNKUlCr8ZGJivw06ckTHfyRAkzLstIHpPc0=;
        b=KxyrqGdEhuQlV2WjCHBSBs6jyvE4NGZKgvq1VrwEvqhDRdQS6PQGWT7Au8MT1gJYFH
         NO4Eazc4slHh8YErLcdA09xq1jsHX4yOE7RHcrXsJQBeEKWJ6PkAoH/psJCdIe5O5C38
         F6mDXmkF+8mgVYNNFpLM5NjcyLLCoiBQyw5tFHis3AGR2UzUDRpR2guLkgpwZ9PXZATK
         8v/sLL4rvUvnuQCQTcOmk318ECYxK2wTtTJHjReRnipTs4KfvoQPEZ2KCrM6qbrcO/Gt
         ePVHg3YKzS2I3r8V9Fxz2OdUW7FWE7Vz70aBLhpXNzeYO01xEgTeWNYZxGIMblyhL/lJ
         LHHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716877261; x=1717482061;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=z7UcWDt5wNKUlCr8ZGJivw06ckTHfyRAkzLstIHpPc0=;
        b=io6NUc16VtXcD0tmKq6ylN8SQahAumEu5hNzUum5tLRbHm6fpaLjYIhAPpTVGhWNNb
         SeC0/UcPvJsJxf4+TqSNrBKylod9uNBbJ/m8F7muxeQAVTNpyFceLmvIPd4dw+96In3I
         fnzkt1KQale7ebTgCVZyZgWDfcYkRyOQe6V0WEoI8PnXrS56gFQlCNF5X+Bp73ceEKaw
         gT7iy1XgGAeJ4CJ6rD/5UBWVBgxL/Mv/yhNBrAq8H/v9ZwFnBbMU8kd0o/cl5+aX1Kwj
         opFCsXfDkUpngdRA9brbJtPOX4xu576x6pjP6OyIoTo/LfClS/plFslQLiJjoLJ72nQt
         KCWA==
X-Forwarded-Encrypted: i=1; AJvYcCUyi0NXeaQryFmCpHPbJ8E1y8GMejo7VMrdpBtHaNQf24h4RF3f1qcaY82OXu4NdZSEQ+Hr2jZ4roAVFeeUsvaB3jYMP5xSxej18Sfow8k=
X-Gm-Message-State: AOJu0Ywjd0d9FUQ1+S+GcdLsKNPBXqw34ZBsXtt1y12NZOmDRntTV/hI
	1JORht8jmrZqQGIt3/0QSUAUsd4Vofsx+KtsbS12w6NCgGmPDRbVmWyLVggPYg==
X-Google-Smtp-Source: AGHT+IGEBkCe8UxWdrk/xN55QFQ7zzMqXgyGRdAErZNswuc6RQTxPfIdyypZ63+zbWHL/0f7FaBCAw==
X-Received: by 2002:a17:906:8348:b0:a62:3488:e22f with SMTP id a640c23a62f3a-a6264f129a4mr706201266b.58.1716877260781;
        Mon, 27 May 2024 23:21:00 -0700 (PDT)
Message-ID: <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
Date: Tue, 28 May 2024 08:20:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 24.05.2024 13:08, Oleksii Kurochko wrote:
> The following generic functions were introduced:
> * test_bit
> * generic__test_and_set_bit
> * generic__test_and_clear_bit
> * generic__test_and_change_bit
> 
> These functions and macros can be useful for architectures
> that don't have corresponding arch-specific instructions.
> 
> Also, the patch introduces the following generics which are
> used by the functions mentioned above:
> * BITOP_BITS_PER_WORD
> * BITOP_MASK
> * BITOP_WORD
> * BITOP_TYPE
> 
> BITS_PER_BYTE was moved to xen/bitops.h as BITS_PER_BYTE is the same
> across architectures.
> 
> The following approach was chosen for generic*() and arch*() bit
> operation functions:
> If the bit operation function that is going to be generic starts
> with the prefix "__", then the corresponding generic/arch function
> will also contain the "__" prefix. For example:
>  * test_bit() will be defined using arch_test_bit() and
>    generic_test_bit().
>  * __test_and_set_bit() will be defined using
>    arch__test_and_set_bit() and generic__test_and_set_bit().
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  Reviewed-by: Jan Beulich jbeulich@suse.com? Jan gave his R-by for the previous
>  version of the patch, but some changes were done, so I wasn't sure if I could
>  use the R-by in this patch version.

That really depends on the nature of the changes. Seeing the pretty
long list below, I think it was appropriate to drop the R-b.

> ---
> Changes in V11:
>  - fix identation in generic_test_bit() function.
>  - move definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h

I realize you were asked to do this, but I'm not overly happy with it.
BITS_PER_BYTE is far more similar to BITS_PER_LONG than to
BITOP_BITS_PER_WORD. Plus in any event that change is entirely unrelated
here.

>  - drop the changes in arm64/livepatch.c.
>  - update the the comments on top of functions: generic__test_and_set_bit(),
>    generic__test_and_clear_bit(),  generic__test_and_change_bit(),
>    generic_test_bit().
>  - Update footer after Signed-off section.
>  - Rebase the patch on top of staging branch, so it can be merged when necessary
>    approves will be given.
>  - Update the commit message.
> ---
>  xen/arch/arm/include/asm/bitops.h |  69 -----------
>  xen/arch/ppc/include/asm/bitops.h |  54 ---------
>  xen/arch/ppc/include/asm/page.h   |   2 +-
>  xen/arch/ppc/mm-radix.c           |   2 +-
>  xen/arch/x86/include/asm/bitops.h |  31 ++---
>  xen/include/xen/bitops.h          | 185 ++++++++++++++++++++++++++++++
>  6 files changed, 196 insertions(+), 147 deletions(-)
> 
> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -103,8 +103,193 @@ static inline int generic_flsl(unsigned long x)
>   * Include this here because some architectures need generic_ffs/fls in
>   * scope
>   */
> +
> +#define BITOP_BITS_PER_WORD 32
> +typedef uint32_t bitop_uint_t;
> +
> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
> +
> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> +
> +#define BITS_PER_BYTE 8

This, if to be moved at all, very clearly doesn't belong here. As much
as it's unrelated to this change, it's also unrelated to bitops as a
whole.

> +extern void __bitop_bad_size(void);
> +
> +#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
> +
> +/* --------------------- Please tidy above here --------------------- */
> +
>  #include <asm/bitops.h>
>  
> +/**
> + * generic__test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed

Why "examples"? Do you mean "instances"? It's further unclear what
"succeed" and "fail" here mean. The function after all has two
purposes: Checking and setting the specified bit. Therefore I think
you mean "succeed in updating the bit in memory", yet then it's
still unclear what the "appear" here means: The caller has no
indication of success/failure. Therefore I think "appear to" also
wants dropping.

> + * but actually fail.  You must protect multiple accesses with a lock.

That's not "must". An often better alternative is to use the atomic
forms instead. "Multiple" is imprecise, too: "Potentially racy" or
some such would come closer.

Also did Julien(?) really ask that these comments be reproduced on
both the functions supposed to be used throughout the codebase _and_
the generic_*() ones (being merely internal helpers/fallbacks)?

> +/**
> + * generic_test_bit - Determine whether a bit is set
> + * @nr: bit number to test
> + * @addr: Address to start counting from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */

You got carried away updating comments - there's no raciness for
simple test_bit(). As is also expressed by its name not having those
double underscores that the others have.

> +static always_inline bool generic_test_bit(int nr, const volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    const volatile bitop_uint_t *p =
> +        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);
> +
> +    return (*p & mask);
> +}
> +
> +/**
> + * __test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline bool
> +__test_and_set_bit(int nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_set_bit
> +#define arch__test_and_set_bit generic__test_and_set_bit
> +#endif
> +
> +    return arch__test_and_set_bit(nr, addr);
> +}

See Julien's comments on the earlier version as well as what Andrew has
now done for ffs()/fls(), avoiding the largely pointless fallback
#define.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 28 06:28:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 06:28:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731007.1136296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqJj-0006yP-RX; Tue, 28 May 2024 06:28:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731007.1136296; Tue, 28 May 2024 06:28:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqJj-0006yI-Oc; Tue, 28 May 2024 06:28:15 +0000
Received: by outflank-mailman (input) for mailman id 731007;
 Tue, 28 May 2024 06:28:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBqJi-0006yC-8u
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 06:28:14 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 75d56642-1cbb-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 08:28:13 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so75943266b.1
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 23:28:13 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93707csm577614466b.57.2024.05.27.23.28.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 23:28:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75d56642-1cbb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716877693; x=1717482493; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0k/DAqxBkB14KmxSi3o3PzV4C0hovLEz8huYFNBItaQ=;
        b=I3lGrNX8FqKLsI2g2qAIDdUOiwHuSlXej5lb+k8gG3wCW49kntCUI8MBSLfgoJh2qR
         x75pwCy6HRuE5rqcJL1wiouc88ycefPYSVrfhPp4s3QlcFVS070qhGIMRYuuemYwFy9Y
         Az6jQ1Do8/+Vnoi/W883/pQHhpkIDn2WCJuxivPwmECF31coTT2jZRQbxLotpJOLcelz
         Yt6/XUgIa8qWQWLnhb5vhI+NjxeoNas1KbpIz+CGXUzUQoWy7GTwWsXAygJxiWJT8qo9
         p0EK577GkUrIk45X31+6bBm+uNYOLKw1C/VjSIL0YewSp6bGMm6CMmP+mHxqEmNaWDvZ
         TqTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716877693; x=1717482493;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0k/DAqxBkB14KmxSi3o3PzV4C0hovLEz8huYFNBItaQ=;
        b=bj2lAHmzG2tzNqPQXY/SGeVZ29SFOVus0j7RIOt3i/DhPRUnHIAJql+IYByAFKR4DC
         k/pUJZmC50zaLz++lZ248+/b0uMf8/EanAuwuEEAh4oSQsExyNhcpCdqSyVsN9Pf8Nia
         wd6UGkcDroZsxsgjui0QzLPa+fXXnVTBzIeTl8Ytj33MhcFmnwbtfITnqYuDfteVL9I7
         MolFn3XpfZ6/NMHeY28ISopy7cT0jWvQnCd+NWGsOEfwPvRg/yJwGO64LOt0Z0ODPe08
         5JDbucJ1aOHslsDBK+ZNhlWKtzO7dYYOdALOQ3j5q8pTvBk29yqhJKdotm6Qi8VbK2tm
         MCHA==
X-Forwarded-Encrypted: i=1; AJvYcCXBUG4hg8QGaQcJw1rxb8XmdHE90b2hQVTAvGQNpY0ePLKIOzwMiu1EmaLcjsQFLLPdHAyR9VoaSL+SpNjN9EFiUnJ85BVfOHKJpLKr+C4=
X-Gm-Message-State: AOJu0YwhuQzhS6NABMm1ycvZd7KIAGgZNHc4M2v7dUUatCWEci/iCvCx
	d3ofudLrgaooBe0MF/Ap0pb5QVsGjK0p0AwLNTjEPm0kqxGBM+F5dYjmZz/3fg==
X-Google-Smtp-Source: AGHT+IEMcbxBw+42tN6WNr05NnGa8D05ElJr5wq0UJPTXbmmqtBm2qUP+WpAG5A8zHgTzetYW2QZJw==
X-Received: by 2002:a17:907:7da2:b0:a62:ef39:19c2 with SMTP id a640c23a62f3a-a62ef391e87mr517112066b.4.1716877692836;
        Mon, 27 May 2024 23:28:12 -0700 (PDT)
Message-ID: <ffb55f45-d08e-4c60-af71-f9a56856bca9@suse.com>
Date: Tue, 28 May 2024 08:28:11 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/4] x86: address violations of MISRA C Rule 8.4
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
 <cf926194a541d11e02670516a8d337de27836dce.1716814609.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cf926194a541d11e02670516a8d337de27836dce.1716814609.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 16:53, Nicola Vetrini wrote:
> Rule 8.4 states: "A compatible declaration shall be visible when
> an object or function with external linkage is defined."
> 
> These variables are only referenced from asm modules, so they
> need to be extern and there is negligible risk of them being
> used improperly without noticing.

"asm modules" isn't quite correct, as there's one use from inline
assembly. I have to admit I have no good wording suggestion other than
explicitly covering both: "asm modules or inline assembly". Yet that
then is ambiguous, as a use in inline assembly may also mean that
symbol is actually visible to the compiler by being mentioned as on of
the operands. Better ideas?

> As a result, they can be exempted using a comment-based deviation.
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

With suitably adjusted wording:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 28 06:31:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 06:31:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731010.1136307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqNG-00009h-BR; Tue, 28 May 2024 06:31:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731010.1136307; Tue, 28 May 2024 06:31:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBqNG-00009a-7N; Tue, 28 May 2024 06:31:54 +0000
Received: by outflank-mailman (input) for mailman id 731010;
 Tue, 28 May 2024 06:31:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBqNF-00009U-06
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 06:31:53 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f80e481a-1cbb-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 08:31:51 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2e95a74d51fso4677381fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 May 2024 23:31:51 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc4fe7csm572324666b.112.2024.05.27.23.31.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 May 2024 23:31:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f80e481a-1cbb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716877911; x=1717482711; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gASJJ4wNyPdL+cvjolySShMRc6uB1G+SwsB4kMeBM/s=;
        b=OkoSlPaxSOuEbIqD6z9vSm2k9G+4SOhbFy5OHY3t9VLqjfl98BACzr9G/Ni/A17gC3
         b2mMrlq9Gl7dioftNAHsUSFVTE+PWbejz4aMolyUsZjRBTmuxteUEpPuU3qx6yIfW5Jl
         /spRsVgAZYj7mlcD8LCV8VbC053ezuc1+g3Oks8UhXTcCFhOa96e+83+s/MU5K5jFXQk
         9Hgq96ZIWEeRAABlrInRvRApn9KOr4OoCM+AtqRdzMxnYUS27Q5n3wwAYn1g0zoEUO5b
         ht12o0/rDx0URj94IyEjuHrh/YvHjKweH2BEtryPiMQHXpi8S6u7Gtl+7zwQIwBZl7Lv
         Yw8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716877911; x=1717482711;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gASJJ4wNyPdL+cvjolySShMRc6uB1G+SwsB4kMeBM/s=;
        b=iYYlN2eCb7EmUTRbxLtUkGrEetNF7kXl8wvHwcaPCBrVLMqOgv+3QtithRpCY13h2P
         KU59S999ouEE8fqlcZl+1bkJo7NjI0ifxi8l9fc1x9DuEVE4UNgBIj8dI+z0DJf0pUCW
         FTv4HU+CkZIx4Cik25PlRmqPz8KAW1OubeJw7ROSiC0yFJKAM8WjhsMy/B3uyY+bG60M
         bGjKUr8U4CMDRkP+LKsuLJJSE66D95jrraa39Dt+5e3pnNV1P1zfPCfEKjZ+44Zh8+3P
         ctCDUZWlQ2hV4cWErVlDsvQXOo3cZ59rM2T53+XfRPPmSS/7+v9Inq71jv9yzt0fLDpb
         zHLg==
X-Forwarded-Encrypted: i=1; AJvYcCVYET0rckQVj0nVIcRnz2UdTXhkctPqiPuvl1ugBuyx/H7WR4hWOyDBxXu8mI2A90qdoEB/fK4cWtn0qku7ZO0TbWtfmNXAjdQrGWtwVZ4=
X-Gm-Message-State: AOJu0YxjK8wlQ+EBpGjlkxvmqT8VBA8XiDGuEPh/PY3mWhWcKoSalo8f
	fEn6YGHOoSo8uwrnoIoGuts9YECmDmlc/pt/MbCkNjy8E/9atHE7wV11xHTw7mn8cBXmQHIHsPI
	=
X-Google-Smtp-Source: AGHT+IFkY58S4aBJS/bzhpJ/8Rsgw+8ELk1wEjYBCXs/yJud+DCp1dZ/jvao9T5VkBD3XPy5XSZ1aQ==
X-Received: by 2002:a2e:b005:0:b0:2e0:c363:f9dc with SMTP id 38308e7fff4ca-2e95b256611mr95075341fa.40.1716877911197;
        Mon, 27 May 2024 23:31:51 -0700 (PDT)
Message-ID: <7260c3e6-9e25-4493-991d-affdb593fa81@suse.com>
Date: Tue, 28 May 2024 08:31:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/4] x86/traps: address violation of MISRA C Rule 8.4
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
 <9d33b2c2aeb59ca67136ed1b65c06e6ef4fd1ed8.1716814609.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9d33b2c2aeb59ca67136ed1b65c06e6ef4fd1ed8.1716814609.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.05.2024 16:53, Nicola Vetrini wrote:
> Rule 8.4 states: "A compatible declaration shall be visible when
> an object or function with external linkage is defined".
> 
> The function do_general_protection is either used is asm code
> or only within this unit, so there is no risk of this getting
> out of sync with its definition, but the function must remain
> extern.
> 
> Therefore, this function is deviated using a comment-based deviation.
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Albeit here, too, I'm not entirely happy with the wording, but I'll let
it go as is. While "there is no risk of this getting out of sync with
its definition" is true for the C side, there's always the risk of
assembly going out of sync with C, simply because the two cannot be
properly connected by any means.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 28 07:12:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 07:12:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731018.1136325 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBr07-0005jy-4R; Tue, 28 May 2024 07:12:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731018.1136325; Tue, 28 May 2024 07:12:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBr07-0005jr-1I; Tue, 28 May 2024 07:12:03 +0000
Received: by outflank-mailman (input) for mailman id 731018;
 Tue, 28 May 2024 07:12:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wewE=M7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBr06-0005jl-9G
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 07:12:02 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 93677143-1cc1-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 09:12:00 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-794ab4480beso38246685a.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 00:12:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd0740csm359485785a.81.2024.05.28.00.11.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 00:11:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93677143-1cc1-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716880319; x=1717485119; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Bx/7DHgYYqjvSrEkle2Btan5DZ6jhOnCHtTbyFruf2Q=;
        b=G2+cu9rnevPFePIRPXXaAPCszfErYehNRi80Y4OkxrlF0nmyBI0hn+1ZBNwx/ee8Ct
         1sAXusIaYzFKuNNmoLfk0LuBQlLBPiD6mLf8Nq8Ow5ZkaSgzRTjemIf/qMEd8U6HaDKF
         LMpV/3YlvnQcwfViJbI9c+Ar8liS6417vXVtM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716880319; x=1717485119;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Bx/7DHgYYqjvSrEkle2Btan5DZ6jhOnCHtTbyFruf2Q=;
        b=ANEloFvTto8+FqoQPY0a0e6JxQ2IDRTTa43HlnWwMQvR1ExCAE+5eTh9Y4wQbHXtzR
         4YzS1hYN1xCgZ92tpB6DxdGZtXkNwLEFXjlHdT1Jh8vS7xUMHZZCgjAXjvgdvo44BCq9
         ylGsNYcOuBnpeOhO0Al6D/cx+f3qrNwqltdsCCAYoBNfq4qZnQIeuRQ4oERh6vGcIcSQ
         cDvMWcYd3kG5ZbmCW759HMTlGt0egdLQvbLRA6X3uF7DmQt7ycKzYwyw7msiNd4XrcOs
         6TFbHsKAqC2bOfVD/IhKFE4nG1EQ5zZ527mB3nug2/WaI+WWaV0poLSKxWfcVgHZmb19
         0+CQ==
X-Forwarded-Encrypted: i=1; AJvYcCXPoi69taJg07FjWyjwiKXKcMVYvRxp9wRpRlssnhMTHJsWSUb17pta+bXoZP2/HemO/oFVkBMf6xN+oIdpo4Q6q+LR8FV/wY6pb9up5wE=
X-Gm-Message-State: AOJu0YxU1RbB0tfZt+/pzWHaTo3LADVXzuzosgWSQgbdUcyCfTIrhGHY
	l3uympcc6nHtxnv8QVNL3SR2LUA0FjNUfMPf8lX7/oY2DdwbRVhElDv2FmnGbgk=
X-Google-Smtp-Source: AGHT+IHhYd/o2Rr1kyCHXRUoTk6NAXl1LIcDDR6rA5A9BPx1JfMcb3KVrMm2oXvck1NL2rPImhC3Ow==
X-Received: by 2002:a05:620a:379b:b0:792:eac7:7bf4 with SMTP id af79cd13be357-794ab07cb70mr1174026285a.26.1716880319222;
        Tue, 28 May 2024 00:11:59 -0700 (PDT)
Date: Tue, 28 May 2024 09:11:56 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	xen-devel@lists.xenproject.org,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH v16 1/5] arm/vpci: honor access size when returning an
 error
Message-ID: <ZlWDvJ9_Om_3gX3b@macbook>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-2-stewart.hildebrand@amd.com>
 <Zk72jPtd9iXhChbc@macbook>
 <3b3fd3a2-6b5c-41a2-a0b3-219a362f645f@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <3b3fd3a2-6b5c-41a2-a0b3-219a362f645f@xen.org>

On Mon, May 27, 2024 at 10:14:59PM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 23/05/2024 08:55, Roger Pau Monné wrote:
> > On Wed, May 22, 2024 at 06:59:20PM -0400, Stewart Hildebrand wrote:
> > > From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> > > 
> > > Guest can try to read config space using different access sizes: 8,
> > > 16, 32, 64 bits. We need to take this into account when we are
> > > returning an error back to MMIO handler, otherwise it is possible to
> > > provide more data than requested: i.e. guest issues LDRB instruction
> > > to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
> > > register.
> > 
> > Shouldn't this be taken care of in the trap handler subsystem, rather
> > than forcing each handler to ensure the returned data matches the
> > access size?
> 
> I understand how this can be useful when we return all 1s.
> 
> However, in most of the current cases, we already need to deal with the
> masking because the data is extracted from a wider field (for instance, see
> the vGIC emulation). For those handlers, I would argue it would be
> concerning/ a bug if the handler return bits above the access size.
> Although, this would only impact the guest itself.

Even if there was a bug in the handler, it would be mitigated by the
truncation done in io.c.

> So overall, this seems to be a matter of taste and I don't quite (yet) see
> the benefits to do it in io.c. Regardless that...

It's up to you really, it's all ARM code so I don't really have a
stake.  IMO it makes the handlers more complicated and fragile.

If nothing else I would at least add an ASSERT() in io.c to ensure
that the data returned from the handler matches the size constrains
you expect.

> > 
> > IOW, something like:
> > 
> > diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> > index 96c740d5636c..b7e12df85f87 100644
> > --- a/xen/arch/arm/io.c
> > +++ b/xen/arch/arm/io.c
> > @@ -37,6 +37,7 @@ static enum io_state handle_read(const struct mmio_handler *handler,
> >           return IO_ABORT;
> > 
> >       r = sign_extend(dabt, r);
> > +    r = r & GENMASK_ULL((1U << dabt.size) * 8 - 1, 0);
> 
> ... in some case we need to sign extend up to the width of the register
> (even if the access is 8-byte). So we would need to do the masking *before*
> calling sign_extend().

I would consider doing the truncation in sign_extend() if suitable,
even if that's doing more than what the function name implies.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 28 08:30:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 08:30:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731034.1136335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsDo-0007bM-Ua; Tue, 28 May 2024 08:30:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731034.1136335; Tue, 28 May 2024 08:30:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsDo-0007bF-Rs; Tue, 28 May 2024 08:30:16 +0000
Received: by outflank-mailman (input) for mailman id 731034;
 Tue, 28 May 2024 08:30:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YQ9m=M7=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1sBsDn-0007b9-Bx
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 08:30:15 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 (mail-am7eur03on20601.outbound.protection.outlook.com
 [2a01:111:f403:260e::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 80f85e68-1ccc-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 10:30:13 +0200 (CEST)
Received: from AS9PR06CA0488.eurprd06.prod.outlook.com (2603:10a6:20b:49b::14)
 by PA6PR08MB10594.eurprd08.prod.outlook.com (2603:10a6:102:3cd::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 28 May
 2024 08:30:09 +0000
Received: from AMS0EPF00000190.eurprd05.prod.outlook.com
 (2603:10a6:20b:49b:cafe::10) by AS9PR06CA0488.outlook.office365.com
 (2603:10a6:20b:49b::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30 via Frontend
 Transport; Tue, 28 May 2024 08:30:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF00000190.mail.protection.outlook.com (10.167.16.213) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15
 via Frontend Transport; Tue, 28 May 2024 08:30:06 +0000
Received: ("Tessian outbound fffbb209f6c2:v327");
 Tue, 28 May 2024 08:30:06 +0000
Received: from 98b7a0e92509.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D6DC9DC1-2FBA-4603-8956-3BE188D63193.1; 
 Tue, 28 May 2024 08:29:59 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 98b7a0e92509.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 May 2024 08:29:59 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by PA4PR08MB7594.eurprd08.prod.outlook.com (2603:10a6:102:270::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.29; Tue, 28 May
 2024 08:29:57 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::6204:b901:9cc6:bf2b%6]) with mapi id 15.20.7611.030; Tue, 28 May 2024
 08:29:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80f85e68-1ccc-11ef-90a1-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=dLiDz2ODCF8zLR9Ju5g/3D5keVH5g6q4VFOS27mZs3wxDGnA8civ9qyTyfnOBGcsZXEZHFD0WqqhnIO2fRszrXrkWmXdUELhlDzdDJpJvATC+rekFy6CDLWgKQnuGCAIp4M50PTiDjj7iEn160KRNz8hfugsWTX0zqIqCz8+Tu5qU1t4oz8w6xMo7lJhGW2+Eh3S1DWRiZiLGf3pqqCHzYnLQ4tmXwYepJtq2RbrwigPMkQ4NsF2AH77+uNoppKkeCiySQLUouLHr+T86pHgqjioOE1HyQG2iZecpz8opMSl9c4mtubuH4SnrabblXChj+qUF4W+Sv1VinQTZls7wQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7VOcNoNlr9GAIkTXpODPS2/BPKKMfCbMqMjV39EpSNY=;
 b=VEnG7OTTCNJgeZTVptpzZcEcbiDb0Zuxu+NPjqF+HSMvSJN0GhEj16HqPlxwDd3AnFJ9Kxi5cYse2IeP3Z288+pJ43UqdtLQmm4wakgaA+m0fw8sGR8KB6nkRz15My+vgqlwPMGesZY7qkItgfjm+RcINxusykpPn7BjPDTM4L0/6N3fzm1BWoLeHL3Fbp0lERE8DXDRwN3aZEf2DJf2UlmyQOmp+mB3r9BQpZHE+735BiqLhFCaH5f3GfCRtAQsng+gtoAbOA61YD1Qy1S1D/ls4RNhENccTXygQzxEHV+hWE+svBUJiDNzh+EaMWrP3gJT9rKQWUEloYJ9Gt46XA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7VOcNoNlr9GAIkTXpODPS2/BPKKMfCbMqMjV39EpSNY=;
 b=WlE5mcWixwfs80CgBCPZPYjRywbpo0DPaIvI8ULuWpiAEe5RYXOwB2xP05Y2fhcP4OisVHRNuvGeje+SdsyelpLN684B9pX7RxErBEcQVTJKf72lkduNXCbO214mwxYjcu1bs+0hACNQSoEOPJMx3oohc0wNntYgAHHVUAf2Nrw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 254b4696c162d274
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fFCJwbcePffJiGLnHGHoRXj1Xy5LAy58IiLWfiKkggaC3utWqv50fT2Bf+M8lJM2G4nQcqAN+gGH7Hxzx+XeTytxrxzosoX+CZwtg6wO5SbGKkqgeG62PyqB9GJ5WOCOgUzcHRjd1p1q9FWIIolwEuGhOs17qy9u5bcr1y8IGZYj6thN7jCjNpM6eurHptKnl7boFc8IE8XTXWsumhdqxhmZ+lWd6gATEvmGE7or3C9GB0OWnSC2Egja9i82sfnLwRigMGl0xpim54jhNU50BTcDsWSN1GwAauXlanlhV52pe127BMggfnj6OILlna+HmDpgQ8ruVlFrTJiBtwoqJg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7VOcNoNlr9GAIkTXpODPS2/BPKKMfCbMqMjV39EpSNY=;
 b=ablMDDMXWswRaltQCxH9973/+TrQeFpi9ygWJdrCGWfe7fJ5AyJ0kgbNMtiHRxd0VAFduLM536gOa5s/XTGeN7yvw7+ACiMrdbBfzVxdl4mTFPJ4ZHXfQcs2QMHy553nNc2sLj/EMBu3R/0VU9tcKAuorPy2PH6NFYA9MciL8MiGhhtXxID4iEj01m2C1j/4vy1ItxAm7CgWr58uJYJ8/X2pUjCcbOKfviYThUwKVKYLFo3bJMq+c7h37WFfSSTay4hvYShDEDozCxH1U/i8PWA+afuaYghKtjno962TJtVL9+fEV7vQRlA/eHufIKyxfPUELq6WwQZKHRbCD9Z9kg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7VOcNoNlr9GAIkTXpODPS2/BPKKMfCbMqMjV39EpSNY=;
 b=WlE5mcWixwfs80CgBCPZPYjRywbpo0DPaIvI8ULuWpiAEe5RYXOwB2xP05Y2fhcP4OisVHRNuvGeje+SdsyelpLN684B9pX7RxErBEcQVTJKf72lkduNXCbO214mwxYjcu1bs+0hACNQSoEOPJMx3oohc0wNntYgAHHVUAf2Nrw=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Julien Grall <julien@xen.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Penny Zheng
	<Penny.Zheng@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Bertrand
 Marquis <Bertrand.Marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v4 3/7] xen/p2m: put reference for level 2 superpage
Thread-Topic: [PATCH v4 3/7] xen/p2m: put reference for level 2 superpage
Thread-Index: AQHarde5MtnBBoqXhEOFOHlm9nE79rGmV9OAgAX+4YA=
Date: Tue, 28 May 2024 08:29:55 +0000
Message-ID: <7E96D55F-2579-4409-8250-3FA905A15CCD@arm.com>
References: <20240524124055.3871399-1-luca.fancellu@arm.com>
 <20240524124055.3871399-4-luca.fancellu@arm.com>
 <30d78526-da89-4228-9cf4-e9792ed2468d@xen.org>
In-Reply-To: <30d78526-da89-4228-9cf4-e9792ed2468d@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.600.62)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|PA4PR08MB7594:EE_|AMS0EPF00000190:EE_|PA6PR08MB10594:EE_
X-MS-Office365-Filtering-Correlation-Id: 635e9a3d-569e-4112-25f0-08dc7ef0614e
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|376005|1800799015|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?utf-8?B?enVxNE96WFRqRkZwcFdGaFJ6VXM5aUxzV3d3dkd4Rmk0bHNXMzVFM2Y1aVZK?=
 =?utf-8?B?cU05T0QweFFhdnc3TEdvTm1NRVg3RytPMDI5ZFNTZ3dUOWU4N2JFeU5rQjds?=
 =?utf-8?B?RVBoVWlDMVpHU3JUSjVkRURHTWFJb2pJdDBBYnhsRVVVMWZhZktGdzc1NWZ1?=
 =?utf-8?B?RWhNRmcxb2taYytUNW9zTHQ3aWErZnBSVzJmQTNyZUNrSURqcElYZHpnMG9i?=
 =?utf-8?B?NUlXdEhLSW5JMk00VjNQaHpkajQxQkVBZVV0SVUwNm0rSUFBcjZMcUhBc2JL?=
 =?utf-8?B?K1czd3AxQ3Z6TkFaZWlBYVRlTDZ4bjkxSCtROStuNEl0UjFrWDhPQXQ1eUpk?=
 =?utf-8?B?c29wMnZDb2lFdGNuNHFqWXBEYTBJYnR0b0YzaWIrQlFDWFlWRHR4Zzk5eVpn?=
 =?utf-8?B?N09wSWw2TTdDR01QeERGOXdxZTBxOVI3Mld1VExiamd3UGs3d2FjODg4NG40?=
 =?utf-8?B?NGluQTZNTHQ2TDRyMjBYbUFndE5YR1AyK2h6V1R4UEFIUDQwMG1FUTB2bkRF?=
 =?utf-8?B?dXNVV0N0TkhNTFY1SEVDSGlxQTRhTEQ0b0dpbCtsbmVkMHNXbmxIbE5ic2dw?=
 =?utf-8?B?T0VxY2FkTkdzaEllcGl5aFdSSXp0WXFScVBuM3pwREtkc1FDUk1nN3lXUUMz?=
 =?utf-8?B?dWtReHpwd2NkSWoybTRvN293MXhSM3pWWlNiUk9xVm1MS29OUFFUWEZTZG54?=
 =?utf-8?B?dU9rbE9qdXptaUFyeDZLdXFqYUpCdTJCY3NhRUNoMTJDQkRUbWJQUk8xbVN1?=
 =?utf-8?B?SnBGbTdCZGpiR1dQV1lScnQzUE5NZU5lRzNzNUFnVVBWUW1NRTcrUGxQRjdi?=
 =?utf-8?B?SUhoaWVSRzN1MWpvYjZyNFkrZnh2ajRPZkdqck1qUGpNSzBTaVBydXdENFlL?=
 =?utf-8?B?cTIvS1YxRWs4R1d6UmphL2ZKRFNSbzlGa0kyc0IraDdqUVZvbWEva3FPNjJw?=
 =?utf-8?B?dkhmbDZ3MkdWMUd1ZjZ4bE5qU2l3K0RscXJ2MWF6dU90YThBOGtFYndsZ2FT?=
 =?utf-8?B?MlJPK1lBVTFkcVAxU2dweFd2T2pnNENuM2xCazF2S2hUcG04T3hnRCs1RzJP?=
 =?utf-8?B?V05rYkc3Qmc3N0k0RklxUUI4SVQwL1YvcXBBYTgrS3B2OHRnSjNjZWp6UDFB?=
 =?utf-8?B?OTIvdU15ekdVVXp4VXVtVFhIV0Y0bmx1eHV0aDgyZXFIOGFLRU9XaDgveWpG?=
 =?utf-8?B?NXdTSTNHQ05uKzB0bHQrN3hrcnBXWXJPOHhLZ094bjAvaEFHRGs2UEJEeWZ1?=
 =?utf-8?B?NlJGalAvWHNFbk1uZDVnSnNVeXMvYlJOcE1UanBjTGNpR0p4ZVVVTG9UWXVv?=
 =?utf-8?B?eUduQmp2UzlpeWRUajdQUUJWMFkvcHZsV0F5NC85S2lHelVrKzN5eUpIcy9U?=
 =?utf-8?B?dG9iQis3K2lUTXBmVFF5Q2FRTG9NV2hvRzFFcmdhYlU3SGltU3pZK3F4UFpz?=
 =?utf-8?B?UTlZZWNtNEZSK21xVnhCL29MQzJoU3lKdnNMTE9sNnQzQVVzc25JUVBOMmJD?=
 =?utf-8?B?RkZxZEVVd3pPaTVBcnQ4VGlyNkRnQWxMMUwwWnY5aDdlb0dpSnF6MUVYVlV1?=
 =?utf-8?B?T2ZzTitsdkovclR2ZWZSd0pxOWdPRjJPQXVKQjUrYVVpZXp3MlZ5NmlTdGp6?=
 =?utf-8?B?Z3BacktvU2UrdE8yWDdnQmRHRGlhMmVIYU5ONjAwR3RTbjh3QXlPOVBZWWRQ?=
 =?utf-8?B?ZENEWVRBakVnYjlzZjhUS1lUcGpMdU1HVWVjdU1MUElsVktnQVg2VWlnPT0=?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <DC81D2003622CE4F9BAA0D2B7E690EC8@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7594
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF00000190.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	38272fcd-9b4e-4e1b-e7e6-08dc7ef05a9a
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?dEdLVlg3MjhBc3FaRDFPV25PT2FaTEpzQ1Vkd05ZWExjRFVtSW56b2tmOEdl?=
 =?utf-8?B?ODRwVjZud24vdXV2d0RRaUk4WXhrNCtuS09kb29EOW9HMUg4SjNGaUlBUE5w?=
 =?utf-8?B?ZG1GU3g2VnRZc0toOHBtRUFvVi9FSzJLNkU2Yy9zT2Fhb3hDTGpMaTRRNm5s?=
 =?utf-8?B?REx3eW1HL21hbGMrWnBVNmJaSFVCTzlXVkNkd2NqODBmOGVkQStNU0lhSFVo?=
 =?utf-8?B?TFFLQWJZNWROSVRxbkJCOU51UmVjVXBSLzNQNjgxZDNac0hiUjB5MGlnMmF5?=
 =?utf-8?B?c0EwdUY4b0Jwbi94ZGJ1cGUrbTBLcXpEcFhnVGlxaWlNdzYydzhuR1dpSk1D?=
 =?utf-8?B?cDBRVkRucVhIQzRTNFErQjZXOXk2akxjdnQvc0dWT011UTJlRGo2b2ZEbGhu?=
 =?utf-8?B?aEJ5ZE9LUFhQVUlLS1hiOVM2ajZkSVFzRlZUSVY0Nm1YQStZZXh0SkYvNzZV?=
 =?utf-8?B?SUhkTWxQSjVNM3c4T1BXQXl1TW4vVXJqc0RJNkRWWXZlZDVYM1pTZDVIekJP?=
 =?utf-8?B?K0JFeXh5dWlvc0w3T0k2dVNUSWcxbHpHMHB3UCtSa0F1NitNWmxML0FYUzVB?=
 =?utf-8?B?RXBIdUtzYXZTTU4xcjFHc1BpU3p6WWNYSEZaaVgxTk9JK1hTL0ZXU0ZlM2JB?=
 =?utf-8?B?KzNtejBvRUlYS1IwdFFDVzZaVlBuRG1pYVJ1V1h3cW5LOS9iSmFHVjZWY1VB?=
 =?utf-8?B?MzE0QXljNVRHTjV3ME5UUFQxNEl0WFlrT3RXTCtmZExrazJEY1ZuTzZaN3BW?=
 =?utf-8?B?N1lselNHeGh1eU5sa0d6Z3dtcGhqTDlTdVpxRmFhZVdTVTFubmh1OHFGSkFO?=
 =?utf-8?B?RHpCTlpTcytuaTZMcGZQV055cXVVWldLTzZyYXBmdHRGcFdtNjhSTUpWOHFM?=
 =?utf-8?B?WGNQYXlQVGNuTHc0QzVINEUzSmExQndobnV3K0lEU3lQOUN4QlRWN29uMFk2?=
 =?utf-8?B?alorelNhZVhiTHBvNzdTblhqUDNFd1ZTWjBUYnNXWkhrTVNWUUt5emZ2aGtF?=
 =?utf-8?B?V21MT3ZjazdxSFRYUTFaY2lSbG11Ri94QVRMZlg5QkFMbTVNSVdXUFFTUWp6?=
 =?utf-8?B?cGJJTjZrYVJkVTB0QjFlbzZKcjU0L2hvN3RTQUZaajllOG53RlF5YTlrK2Iw?=
 =?utf-8?B?K0x6SFZRU1g3TGE3dCt4UmtNWVVXWDBydGpqTGVLQzJuaWhMbGR6SjBEOSs1?=
 =?utf-8?B?R1FhSVVjN3NsQmxIMkh6R2c4S011ek5zY3RRUyt6OEpoVWV4ekwwckQ2dStV?=
 =?utf-8?B?SnJwYzN6dzRqRDczby95bmtDUlpBbmptTXN5aExJNjE0YUdpaVZyRi95UDEv?=
 =?utf-8?B?Z3RXOFJrN3BnZk9HdDZrTjRyc2tlVHBjL2h6SWRoaWFlaTFJK041N2Z3ZzhC?=
 =?utf-8?B?cy94YWNCeVI4Z3ZTYm9ORExuaVJwcEwxbUQ1cGNWVGdBdlo2RXNUT3pnMGUy?=
 =?utf-8?B?WmRKZUdKT3Q1b0pUS0Q4aElYMTNCS200elYvbnVsR2FBQWhWVlptY3BOWkwz?=
 =?utf-8?B?WjlUR1NieEdYV09KK0p1UE43WjBQdGI1eTFrTWdoQmVkTktvbjhPVkExYnpR?=
 =?utf-8?B?NWI0NFlpNVQzdzAxTElrSDQ0b2RUZ0tuQVpYcnBEUURZMmdDckQ1VCtaVlVF?=
 =?utf-8?B?WHEvWlJhMngrVjd3UGN6OVFuaFE4MFY4bGQ1NWllbHVCZWY3bFRDb1F5OUNu?=
 =?utf-8?B?ek1hd015dk9KQXRnRlRzVWZYZmNIa0tZTHBBcGgyQWVmTFZ6OGROZThmNWNT?=
 =?utf-8?Q?S4z+jdEUjAIxrpdCh7S4oTXM528vMbIDIEvjaIa?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2024 08:30:06.8592
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 635e9a3d-569e-4112-25f0-08dc7ef0614e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF00000190.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA6PR08MB10594

SGkgSnVsaWVuLA0KDQo+IE9uIDI0IE1heSAyMDI0LCBhdCAxMzo1NiwgSnVsaWVuIEdyYWxsIDxq
dWxpZW5AeGVuLm9yZz4gd3JvdGU6DQo+IA0KPiBIaSBMdWNhLA0KPiANCj4gT24gMjQvMDUvMjAy
NCAxMzo0MCwgTHVjYSBGYW5jZWxsdSB3cm90ZToNCj4+IEZyb206IFBlbm55IFpoZW5nIDxQZW5u
eS5aaGVuZ0Bhcm0uY29tPg0KPj4gV2UgYXJlIGRvaW5nIGZvcmVpZ24gbWVtb3J5IG1hcHBpbmcg
Zm9yIHN0YXRpYyBzaGFyZWQgbWVtb3J5LCBhbmQNCj4+IHRoZXJlIGlzIGEgZ3JlYXQgcG9zc2li
aWxpdHkgdGhhdCBpdCBjb3VsZCBiZSBzdXBlciBtYXBwZWQuDQo+PiBCdXQgdG9kYXksIHAybV9w
dXRfbDNfcGFnZSBjb3VsZCBub3QgaGFuZGxlIHN1cGVycGFnZXMuDQo+PiBUaGlzIGNvbW1pdHMg
aW1wbGVtZW50cyBhIG5ldyBmdW5jdGlvbiBwMm1fcHV0X2wyX3N1cGVycGFnZSB0byBoYW5kbGUN
Cj4+IDJNQiBzdXBlcnBhZ2VzLCBzcGVjaWZpY2FsbHkgZm9yIGhlbHBpbmcgcHV0IGV4dHJhIHJl
ZmVyZW5jZXMgZm9yDQo+PiBmb3JlaWduIHN1cGVycGFnZXMuDQo+PiBNb2RpZnkgcmVsaW5xdWlz
aF9wMm1fbWFwcGluZyBhcyB3ZWxsIHRvIHRha2UgaW50byBhY2NvdW50IHByZWVtcHRpb24NCj4+
IHdoZW4gdHlwZSBpcyBmb3JlaWduIG1lbW9yeSBhbmQgb3JkZXIgaXMgYWJvdmUgOSAoMk1CKS4N
Cj4+IEN1cnJlbnRseSAxR0Igc3VwZXJwYWdlcyBhcmUgbm90IGhhbmRsZWQgYmVjYXVzZSBYZW4g
aXMgbm90IHByZWVtcHRpYmxlDQo+PiBhbmQgdGhlcmVmb3JlIHNvbWUgd29yayBpcyBuZWVkZWQg
dG8gaGFuZGxlIHN1Y2ggc3VwZXJwYWdlcywgZm9yIHdoaWNoDQo+PiBhdCBzb21lIHBvaW50IFhl
biBtaWdodCBlbmQgdXAgZnJlZWluZyBtZW1vcnkgYW5kIHRoZXJlZm9yZSBmb3Igc3VjaCBhDQo+
PiBiaWcgbWFwcGluZyBpdCBjb3VsZCBlbmQgdXAgaW4gYSB2ZXJ5IGxvbmcgb3BlcmF0aW9uLg0K
Pj4gU2lnbmVkLW9mZi1ieTogUGVubnkgWmhlbmcgPHBlbm55LnpoZW5nQGFybS5jb20+DQo+PiBT
aWduZWQtb2ZmLWJ5OiBMdWNhIEZhbmNlbGx1IDxsdWNhLmZhbmNlbGx1QGFybS5jb20+DQo+PiAt
LS0NCj4+IHY0IGNoYW5nZXM6DQo+PiAgLSBvcHRpbWlzZWQgdGhlIHBhdGggdG8gY2FsbCBwdXRf
cGFnZSgpIG9uIHRoZSBmb3JlaWduIG1hcHBpbmcgYXMNCj4+ICAgIEp1bGllbiBzdWdnZXN0ZWQu
IEFkZCBhIGNvbW1lbnQgaW4gcDJtX3B1dF9sMl9zdXBlcnBhZ2UgdG8gc3RhdGUNCj4+ICAgIHRo
YXQgYW55IGNoYW5nZXMgbmVlZHMgdG8gdGFrZSBpbnRvIGFjY291bnQgc29tZSBjaGFuZ2UgaW4g
dGhlDQo+PiAgICByZWxpbnF1aXNoIGNvZGUuIChKdWxpZW4pDQo+PiB2MyBjaGFuZ2VzOg0KPj4g
IC0gQWRkIHJlYXNvbmluZyB3aHkgd2UgZG9uJ3Qgc3VwcG9ydCBub3cgMUdCIHN1cGVycGFnZSwg
cmVtb3ZlIGxldmVsX29yZGVyDQo+PiAgICB2YXJpYWJsZSBmcm9tIHAybV9wdXRfbDJfc3VwZXJw
YWdlLCB1cGRhdGUgVE9ETyBjb21tZW50IGluc2lkZQ0KPj4gICAgcDJtX2ZyZWVfZW50cnksIHVz
ZSBYRU5fUFRfTEVWRUxfT1JERVIoMikgaW5zdGVhZCBvZiB2YWx1ZSA5IGluc2lkZQ0KPj4gICAg
cmVsaW5xdWlzaF9wMm1fbWFwcGluZy4gKE1pY2hhbCkNCj4+IHYyOg0KPj4gIC0gRG8gbm90IGhh
bmRsZSAxR0Igc3VwZXIgcGFnZSBhcyB0aGVyZSBtaWdodCBiZSBzb21lIGlzc3VlIHdoZXJlDQo+
PiAgICBhIGxvdCBvZiBjYWxscyB0byBwdXRfcGFnZSguLi4pIG1pZ2h0IGJlIGlzc3VlZCB3aGlj
aCBjb3VsZCBsZWFkDQo+PiAgICB0byBmcmVlIG1lbW9yeSB0aGF0IGlzIGEgbG9uZyBvcGVyYXRp
b24uDQo+PiB2MToNCj4+ICAtIHBhdGNoIGZyb20gaHR0cHM6Ly9wYXRjaHdvcmsua2VybmVsLm9y
Zy9wcm9qZWN0L3hlbi1kZXZlbC9wYXRjaC8yMDIzMTIwNjA5MDYyMy4xOTMyMjc1LTktUGVubnku
WmhlbmdAYXJtLmNvbS8NCj4+IC0tLQ0KPj4gIHhlbi9hcmNoL2FybS9tbXUvcDJtLmMgfCA4MiAr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0tLS0NCj4+ICAxIGZpbGUgY2hh
bmdlZCwgNjIgaW5zZXJ0aW9ucygrKSwgMjAgZGVsZXRpb25zKC0pDQo+PiBkaWZmIC0tZ2l0IGEv
eGVuL2FyY2gvYXJtL21tdS9wMm0uYyBiL3hlbi9hcmNoL2FybS9tbXUvcDJtLmMNCj4+IGluZGV4
IDQxZmNjYTAxMWNmNC4uOTg2YzVhMDNjNTRiIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJt
L21tdS9wMm0uYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL21tdS9wMm0uYw0KPj4gQEAgLTc1Mywz
NCArNzUzLDY2IEBAIHN0YXRpYyBpbnQgcDJtX21lbV9hY2Nlc3NfcmFkaXhfc2V0KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmbl90IGdmbiwNCj4+ICAgICAgcmV0dXJuIHJjOw0KPj4gIH0NCj4+
ICAtLyoNCj4+IC0gKiBQdXQgYW55IHJlZmVyZW5jZXMgb24gdGhlIHNpbmdsZSA0SyBwYWdlIHJl
ZmVyZW5jZWQgYnkgcHRlLg0KPj4gLSAqIFRPRE86IEhhbmRsZSBzdXBlcnBhZ2VzLCBmb3Igbm93
IHdlIG9ubHkgdGFrZSBzcGVjaWFsIHJlZmVyZW5jZXMgZm9yIGxlYWYNCj4+IC0gKiBwYWdlcyAo
c3BlY2lmaWNhbGx5IGZvcmVpZ24gb25lcywgd2hpY2ggY2FuJ3QgYmUgc3VwZXIgbWFwcGVkIHRv
ZGF5KS4NCj4+IC0gKi8NCj4+IC1zdGF0aWMgdm9pZCBwMm1fcHV0X2wzX3BhZ2UoY29uc3QgbHBh
ZV90IHB0ZSkNCj4+ICtzdGF0aWMgdm9pZCBwMm1fcHV0X2ZvcmVpZ25fcGFnZShzdHJ1Y3QgcGFn
ZV9pbmZvICpwZykNCj4+ICB7DQo+PiAtICAgIG1mbl90IG1mbiA9IGxwYWVfZ2V0X21mbihwdGUp
Ow0KPj4gLQ0KPj4gLSAgICBBU1NFUlQocDJtX2lzX3ZhbGlkKHB0ZSkpOw0KPj4gLQ0KPj4gICAg
ICAvKg0KPj4gLSAgICAgKiBUT0RPOiBIYW5kbGUgb3RoZXIgcDJtIHR5cGVzDQo+PiAtICAgICAq
DQo+PiAgICAgICAqIEl0J3Mgc2FmZSB0byBkbyB0aGUgcHV0X3BhZ2UgaGVyZSBiZWNhdXNlIHBh
Z2VfYWxsb2Mgd2lsbA0KPj4gICAgICAgKiBmbHVzaCB0aGUgVExCcyBpZiB0aGUgcGFnZSBpcyBy
ZWFsbG9jYXRlZCBiZWZvcmUgdGhlIGVuZCBvZg0KPj4gICAgICAgKiB0aGlzIGxvb3AuDQo+PiAg
ICAgICAqLw0KPj4gLSAgICBpZiAoIHAybV9pc19mb3JlaWduKHB0ZS5wMm0udHlwZSkgKQ0KPj4g
KyAgICBwdXRfcGFnZShwZyk7DQo+PiArfQ0KPj4gKw0KPj4gKy8qIFB1dCBhbnkgcmVmZXJlbmNl
cyBvbiB0aGUgc2luZ2xlIDRLIHBhZ2UgcmVmZXJlbmNlZCBieSBtZm4uICovDQo+PiArc3RhdGlj
IHZvaWQgcDJtX3B1dF9sM19wYWdlKG1mbl90IG1mbiwgcDJtX3R5cGVfdCB0eXBlKQ0KPj4gK3sN
Cj4+ICsgICAgLyogVE9ETzogSGFuZGxlIG90aGVyIHAybSB0eXBlcyAqLw0KPj4gKyAgICBpZiAo
IHAybV9pc19mb3JlaWduKHR5cGUpICkNCj4+ICAgICAgew0KPj4gICAgICAgICAgQVNTRVJUKG1m
bl92YWxpZChtZm4pKTsNCj4+IC0gICAgICAgIHB1dF9wYWdlKG1mbl90b19wYWdlKG1mbikpOw0K
Pj4gKyAgICAgICAgcDJtX3B1dF9mb3JlaWduX3BhZ2UobWZuX3RvX3BhZ2UobWZuKSk7DQo+PiAg
ICAgIH0NCj4+ICAgICAgLyogRGV0ZWN0IHRoZSB4ZW5oZWFwIHBhZ2UgYW5kIG1hcmsgdGhlIHN0
b3JlZCBHRk4gYXMgaW52YWxpZC4gKi8NCj4+IC0gICAgZWxzZSBpZiAoIHAybV9pc19yYW0ocHRl
LnAybS50eXBlKSAmJiBpc194ZW5faGVhcF9tZm4obWZuKSApDQo+PiArICAgIGVsc2UgaWYgKCBw
Mm1faXNfcmFtKHR5cGUpICYmIGlzX3hlbl9oZWFwX21mbihtZm4pICkNCj4+ICAgICAgICAgIHBh
Z2Vfc2V0X3hlbmhlYXBfZ2ZuKG1mbl90b19wYWdlKG1mbiksIElOVkFMSURfR0ZOKTsNCj4+ICB9
DQo+PiAgKy8qIFB1dCBhbnkgcmVmZXJlbmNlcyBvbiB0aGUgc3VwZXJwYWdlIHJlZmVyZW5jZWQg
YnkgbWZuLiAqLw0KPj4gK3N0YXRpYyB2b2lkIHAybV9wdXRfbDJfc3VwZXJwYWdlKG1mbl90IG1m
biwgcDJtX3R5cGVfdCB0eXBlKQ0KPj4gK3sNCj4+ICsgICAgc3RydWN0IHBhZ2VfaW5mbyAqcGc7
DQo+PiArICAgIHVuc2lnbmVkIGludCBpOw0KPj4gKw0KPj4gKyAgICAvKg0KPj4gKyAgICAgKiBU
T0RPOiBIYW5kbGUgb3RoZXIgcDJtIHR5cGVzLCBidXQgYmUgYXdhcmUgdGhhdCBhbnkgY2hhbmdl
cyB0byBoYW5kbGUNCj4+ICsgICAgICogZGlmZmVyZW50IHR5cGVzIHNob3VsZCByZXF1aXJlIGFu
IHVwZGF0ZSBvbiB0aGUgcmVsaW5xdWlzaCBjb2RlIHRvIGhhbmRsZQ0KPj4gKyAgICAgKiBwcmVl
bXB0aW9uLg0KPj4gKyAgICAgKi8NCj4+ICsgICAgaWYgKCAhcDJtX2lzX2ZvcmVpZ24odHlwZSkg
KQ0KPj4gKyAgICAgICAgcmV0dXJuOw0KPj4gKw0KPj4gKyAgICBBU1NFUlQobWZuX3ZhbGlkKG1m
bikpOw0KPj4gKw0KPj4gKyAgICBwZyA9IG1mbl90b19wYWdlKG1mbik7DQo+PiArDQo+PiArICAg
IGZvciAoIGkgPSAwOyBpIDwgWEVOX1BUX0xQQUVfRU5UUklFUzsgaSsrLCBwZysrICkNCj4+ICsg
ICAgICAgIHAybV9wdXRfZm9yZWlnbl9wYWdlKHBnKTsNCj4+ICt9DQo+PiArDQo+PiArLyogUHV0
IGFueSByZWZlcmVuY2VzIG9uIHRoZSBwYWdlIHJlZmVyZW5jZWQgYnkgcHRlLiAqLw0KPj4gK3N0
YXRpYyB2b2lkIHAybV9wdXRfcGFnZShjb25zdCBscGFlX3QgcHRlLCB1bnNpZ25lZCBpbnQgbGV2
ZWwpDQo+PiArew0KPj4gKyAgICBtZm5fdCBtZm4gPSBscGFlX2dldF9tZm4ocHRlKTsNCj4+ICsN
Cj4+ICsgICAgQVNTRVJUKHAybV9pc192YWxpZChwdGUpKTsNCj4+ICsNCj4+ICsgICAgLyogV2Ug
aGF2ZSBhIHNlY29uZCBsZXZlbCAyTSBzdXBlcnBhZ2UgKi8NCj4+ICsgICAgaWYgKCBwMm1faXNf
c3VwZXJwYWdlKHB0ZSwgbGV2ZWwpICYmIChsZXZlbCA9PSAyKSApDQo+IA0KPiBBRkFJQ1QsIHAy
bV9wdXRfcGFnZSgpIGNhbiBvbmx5IGJlIGNhbGxlZCBpZiB0aGUgcHRlIHBvaW50cyB0byBhIHN1
cGVycGFnZSBvciBwYWdlOg0KPiANCj4gICAgaWYgKCBwMm1faXNfc3VwZXJwYWdlKGVudHJ5LCBs
ZXZlbCkgfHwgKGxldmVsID09IDMpICkNCj4gICAgew0KPiAgICAgICAuLi4NCj4gICAgICAgcDJt
X3B1dF9wYWdlKCkNCj4gDQo+ICAgIH0NCj4gDQo+IFNvIGRvIHdlIGFjdHVhbGx5IG5lZWQgdG8g
Y2hlY2sgcDJtX2lzX3N1cGVycGFnZSgpPw0KPiANCj4+ICsgICAgICAgIHJldHVybiBwMm1fcHV0
X2wyX3N1cGVycGFnZShtZm4sIHB0ZS5wMm0udHlwZSk7DQo+PiArICAgIGVsc2UgaWYgKCBsZXZl
bCA9PSAzICkNCj4+ICsgICAgICAgIHJldHVybiBwMm1fcHV0X2wzX3BhZ2UobWZuLCBwdGUucDJt
LnR5cGUpOw0KPj4gK30NCj4+ICsNCj4+ICAvKiBGcmVlIGxwYWUgc3ViLXRyZWUgYmVoaW5kIGFu
IGVudHJ5ICovDQo+PiAgc3RhdGljIHZvaWQgcDJtX2ZyZWVfZW50cnkoc3RydWN0IHAybV9kb21h
aW4gKnAybSwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBscGFlX3QgZW50cnksIHVu
c2lnbmVkIGludCBsZXZlbCkNCj4+IEBAIC04MDksOSArODQxLDE2IEBAIHN0YXRpYyB2b2lkIHAy
bV9mcmVlX2VudHJ5KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sDQo+PiAgI2VuZGlmDQo+PiAgICAg
ICAgICAgIHAybS0+c3RhdHMubWFwcGluZ3NbbGV2ZWxdLS07DQo+PiAtICAgICAgICAvKiBOb3Ro
aW5nIHRvIGRvIGlmIHRoZSBlbnRyeSBpcyBhIHN1cGVyLXBhZ2UuICovDQo+PiAtICAgICAgICBp
ZiAoIGxldmVsID09IDMgKQ0KPj4gLSAgICAgICAgICAgIHAybV9wdXRfbDNfcGFnZShlbnRyeSk7
DQo+PiArICAgICAgICAvKg0KPj4gKyAgICAgICAgICogVE9ETzogQ3VycmVudGx5IHdlIGRvbid0
IGhhbmRsZSAxR0Igc3VwZXItcGFnZSwgWGVuIGlzIG5vdA0KPj4gKyAgICAgICAgICogcHJlZW1w
dGlibGUgYW5kIHRoZXJlZm9yZSBzb21lIHdvcmsgaXMgbmVlZGVkIHRvIGhhbmRsZSBzdWNoDQo+
PiArICAgICAgICAgKiBzdXBlcnBhZ2VzLCBmb3Igd2hpY2ggYXQgc29tZSBwb2ludCBYZW4gbWln
aHQgZW5kIHVwIGZyZWVpbmcgbWVtb3J5DQo+PiArICAgICAgICAgKiBhbmQgdGhlcmVmb3JlIGZv
ciBzdWNoIGEgYmlnIG1hcHBpbmcgaXQgY291bGQgZW5kIHVwIGluIGEgdmVyeSBsb25nDQo+PiAr
ICAgICAgICAgKiBvcGVyYXRpb24uDQo+PiArICAgICAgICAgKi8NCj4+ICsgICAgICAgIGlmICgg
bGV2ZWwgPj0gMiApDQo+IA0KPiBUaGUgY29kZSBpbiBwMm1fcHV0X3BhZ2UoKSBjYW4gcHJvcGVy
bHkgaGFuZGxlIGxldmVsIDEuIFNvIEkgd291bGQgcmF0aGVyIHByZWZlciBpZiB3ZSByZW1vdmUg
dGhpcyBjaGVjayAoYWdhaW4gaW4gb3JkZXIgdG8gcmVkdWNlIHRoZSBhbW91bnQgb2Ygd29yayBp
biB0aGUgUDJNIGNvZGUpLg0KPiANCj4gVGhpcyBUT0RPIGNvdWxkIGJlIG1vdmVkIHRoZXJlLg0K
PiANCj4+ICsgICAgICAgICAgICBwMm1fcHV0X3BhZ2UoZW50cnksIGxldmVsKTsNCj4+ICsNCj4+
ICAgICAgICAgIHJldHVybjsNCj4+ICAgICAgfQ0KPj4gIEBAIC0xNTU4LDkgKzE1OTcsMTIgQEAg
aW50IHJlbGlucXVpc2hfcDJtX21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICAgICAgICAg
ICAgY291bnQrKzsNCj4+ICAgICAgICAgIC8qDQo+PiAtICAgICAgICAgKiBBcmJpdHJhcmlseSBw
cmVlbXB0IGV2ZXJ5IDUxMiBpdGVyYXRpb25zLg0KPj4gKyAgICAgICAgICogQXJiaXRyYXJpbHkg
cHJlZW1wdCBldmVyeSA1MTIgaXRlcmF0aW9ucyBvciB3aGVuIHR5cGUgaXMgZm9yZWlnbg0KPj4g
KyAgICAgICAgICogbWFwcGluZyBhbmQgdGhlIG9yZGVyIGlzIGFib3ZlIDkgKDJNQikuDQo+IA0K
PiBMZXQncyBhdm9pZCBtZW50aW9uaW5nIDJNQi4gVGhpcyBpcyBvbmx5IHZhbGlkIHdoZW4gdXNp
bmcgNEtCIHBhZ2VzLg0KPiANCj4gVGhpcyBpcyBhbHNvIHRhbGtpbmcgYWJvdXQgYSBzcGVjaWZp
YyB3aGVuIHRoZSBjb2RlIGJlbG93IGlzIG5vdCA0S0Igc3BlY2lmaWMuIEkgd291bGQgcmV3b3Jr
IGFueXRoaW5nIGFmdGVyIHRoZSAnb3InIHRvOg0KPiANCj4gJ3dlIGhhdmUgYSBsZXZlbC0yIGZv
cmVpZ24gbWFwcGluZycNCj4gDQo+IFRoZSByZXN0IG9mIHRoZSBMR1RNIHRvIG1lIGJ1dCBJIGRv
bid0IGZlZWwgY29uZm9ydGFibGUgdG8gZG8gYWxsIHRob3NlIGNoYW5nZXMgb24gY29tbWl0LiBT
byB0aGlzIHdpbGwgd2FudCBhIHJlc3Bpbi4NCj4gDQo+IEZvciBzaW1wbGljaXR5LCB5b3UgY2Fu
IG9ubHkgcmVzZW5kIHRoaXMgcGF0Y2guDQoNClRoYW5rcywgSeKAmWxsIGFkZHJlc3MgeW91ciBj
b21tZW50IGFuZCBwdXNoIHRoZSBuZXcgcGF0Y2ggYXNhcC4NCg0KQ2hlZXJzLA0KTHVjYQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue May 28 08:37:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 08:37:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731039.1136346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsKy-0008FA-Oa; Tue, 28 May 2024 08:37:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731039.1136346; Tue, 28 May 2024 08:37:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsKy-0008F3-Ju; Tue, 28 May 2024 08:37:40 +0000
Received: by outflank-mailman (input) for mailman id 731039;
 Tue, 28 May 2024 08:37:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfFg=M7=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBsKy-0008Ex-4x
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 08:37:40 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 89f56926-1ccd-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 10:37:38 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52449b7aa2bso680608e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 01:37:38 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b067sm912136e87.168.2024.05.28.01.37.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 01:37:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89f56926-1ccd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716885457; x=1717490257; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=outSS5upUI878TOs8kNbB+Im+t4Am+q/COUWrTdKTfA=;
        b=as+zuYYOclSleembN3m1+KLSKU4zNpZtdU7IG1jBgUtMJZD7EzgRcdHByCCLUbQsx2
         BgIcjLl8+Wyp2H8SUVABAZuS39P8IKao6i/LlG5mV8UZm2s6yDc3WDrtubRwR7IJLakG
         WxFWJ3AFkuhUt4dXlLysupoxw83Vg1xV/EC9H847FqWtTxNaukzp9C97/rZjPZP1zah/
         CUTZ/vKsISBQ9SrVTubJRXSJVGmTBv8yBJ93UPmN1UiSZLaDy6PflPGDFQBf5h8Kh3ae
         wf8jmhF0A9DTtKu8N0qOhVHC2HZ6QmwRow1Qk/binFH2gN4kngDPEwHrtdBZD/SyfBdc
         yteA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716885457; x=1717490257;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=outSS5upUI878TOs8kNbB+Im+t4Am+q/COUWrTdKTfA=;
        b=pQUzI2ej9/aqUuRm7rfClwpICnv7089W7zJBF91CJ8fVmPBp03hQ8tTccqNcEzPEGq
         zY22BttAaP71+jkaPgLuxbqFYVPu03teH5nzdlCV93uTuPg9yJaT+N+PxhEnB7I/ube4
         e+O1lV6Wfip3FZpYReFXsAJhzrawoRY+O0Or02MRYpcVAQgWq2OvaEG96635p2zP18OR
         6b1IXeB0OId1ObwNqFkmQZjaysHhm2OqQ7HsSzuyVYhZPtNEYzl6leMgDC1tg0Xw+iqP
         1/OR3LdWob4ajUF1M5GTWABThHIjvg1oMkKnvxghdwXbuIlAXRek1v+06WmiFVxCg9F6
         n06A==
X-Forwarded-Encrypted: i=1; AJvYcCXyw9tKBrjoL/n55pG+1LUuKD5kdTxy4D5vFz0PlI1Sgry3ojO+ryQa+UP21T9tYJ6IcEolFgf9lVLpNcNDegspBQT4Vf18CAUbte6MCXs=
X-Gm-Message-State: AOJu0YzMp2dGfqmvxrlO+yGbDvx75HYRgxN8ktt4eZkreu1WsYU85Dop
	Dt+Et8iVsaVMPnyHELEPUvjLjJuMXTdjOUkGhDyMYPVNx3GT9OGi
X-Google-Smtp-Source: AGHT+IGReejBsf7WTkkUV1AcHCYWdewmnQ1dOFtfjT8JwugYhXBwYgtsl0jpB5U8RMh8RG/1witM/g==
X-Received: by 2002:a05:6512:2208:b0:522:4062:6e79 with SMTP id 2adb3069b0e04-5296519a4f5mr6920445e87.31.1716885457067;
        Tue, 28 May 2024 01:37:37 -0700 (PDT)
Message-ID: <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Tue, 28 May 2024 10:37:35 +0200
In-Reply-To: <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
	 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
	 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-28 at 08:20 +0200, Jan Beulich wrote:
> On 24.05.2024 13:08, Oleksii Kurochko wrote:
> > The following generic functions were introduced:
> > * test_bit
> > * generic__test_and_set_bit
> > * generic__test_and_clear_bit
> > * generic__test_and_change_bit
> >=20
> > These functions and macros can be useful for architectures
> > that don't have corresponding arch-specific instructions.
> >=20
> > Also, the patch introduces the following generics which are
> > used by the functions mentioned above:
> > * BITOP_BITS_PER_WORD
> > * BITOP_MASK
> > * BITOP_WORD
> > * BITOP_TYPE
> >=20
> > BITS_PER_BYTE was moved to xen/bitops.h as BITS_PER_BYTE is the
> > same
> > across architectures.
> >=20
> > The following approach was chosen for generic*() and arch*() bit
> > operation functions:
> > If the bit operation function that is going to be generic starts
> > with the prefix "__", then the corresponding generic/arch function
> > will also contain the "__" prefix. For example:
> > =C2=A0* test_bit() will be defined using arch_test_bit() and
> > =C2=A0=C2=A0 generic_test_bit().
> > =C2=A0* __test_and_set_bit() will be defined using
> > =C2=A0=C2=A0 arch__test_and_set_bit() and generic__test_and_set_bit().
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > =C2=A0Reviewed-by: Jan Beulich jbeulich@suse.com? Jan gave his R-by for
> > the previous
> > =C2=A0version of the patch, but some changes were done, so I wasn't sur=
e
> > if I could
> > =C2=A0use the R-by in this patch version.
>=20
> That really depends on the nature of the changes. Seeing the pretty
> long list below, I think it was appropriate to drop the R-b.
>=20
> > ---
> > Changes in V11:
> > =C2=A0- fix identation in generic_test_bit() function.
> > =C2=A0- move definition of BITS_PER_BYTE from <arch>/bitops.h to
> > xen/bitops.h
>=20
> I realize you were asked to do this, but I'm not overly happy with
> it.
> BITS_PER_BYTE is far more similar to BITS_PER_LONG than to
> BITOP_BITS_PER_WORD. Plus in any event that change is entirely
> unrelated
> here.
So where then it should be introduced? x86 introduces that in config.h,
Arm in asm/bitops.h.
I am okay to revert this change and make a separate patch.

>=20
> > =C2=A0- drop the changes in arm64/livepatch.c.
> > =C2=A0- update the the comments on top of functions:
> > generic__test_and_set_bit(),
> > =C2=A0=C2=A0 generic__test_and_clear_bit(),=C2=A0 generic__test_and_cha=
nge_bit(),
> > =C2=A0=C2=A0 generic_test_bit().
> > =C2=A0- Update footer after Signed-off section.
> > =C2=A0- Rebase the patch on top of staging branch, so it can be merged
> > when necessary
> > =C2=A0=C2=A0 approves will be given.
> > =C2=A0- Update the commit message.
> > ---
> > =C2=A0xen/arch/arm/include/asm/bitops.h |=C2=A0 69 -----------
> > =C2=A0xen/arch/ppc/include/asm/bitops.h |=C2=A0 54 ---------
> > =C2=A0xen/arch/ppc/include/asm/page.h=C2=A0=C2=A0 |=C2=A0=C2=A0 2 +-
> > =C2=A0xen/arch/ppc/mm-radix.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 +-
> > =C2=A0xen/arch/x86/include/asm/bitops.h |=C2=A0 31 ++---
> > =C2=A0xen/include/xen/bitops.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 | 185
> > ++++++++++++++++++++++++++++++
> > =C2=A06 files changed, 196 insertions(+), 147 deletions(-)
> >=20
> > --- a/xen/include/xen/bitops.h
> > +++ b/xen/include/xen/bitops.h
> > @@ -103,8 +103,193 @@ static inline int generic_flsl(unsigned long
> > x)
> > =C2=A0 * Include this here because some architectures need
> > generic_ffs/fls in
> > =C2=A0 * scope
> > =C2=A0 */
> > +
> > +#define BITOP_BITS_PER_WORD 32
> > +typedef uint32_t bitop_uint_t;
> > +
> > +#define BITOP_MASK(nr)=C2=A0 ((bitop_uint_t)1 << ((nr) %
> > BITOP_BITS_PER_WORD))
> > +
> > +#define BITOP_WORD(nr)=C2=A0 ((nr) / BITOP_BITS_PER_WORD)
> > +
> > +#define BITS_PER_BYTE 8
>=20
> This, if to be moved at all, very clearly doesn't belong here. As
> much
> as it's unrelated to this change, it's also unrelated to bitops as a
> whole.
>=20
> > +extern void __bitop_bad_size(void);
> > +
> > +#define bitop_bad_size(addr) (sizeof(*(addr)) <
> > sizeof(bitop_uint_t))
> > +
> > +/* --------------------- Please tidy above here ------------------
> > --- */
> > +
> > =C2=A0#include <asm/bitops.h>
> > =C2=A0
> > +/**
> > + * generic__test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
>=20
> Why "examples"? Do you mean "instances"? It's further unclear what
> "succeed" and "fail" here mean. The function after all has two
> purposes: Checking and setting the specified bit. Therefore I think
> you mean "succeed in updating the bit in memory", yet then it's
> still unclear what the "appear" here means: The caller has no
> indication of success/failure. Therefore I think "appear to" also
> wants dropping.
>=20
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
>=20
> That's not "must". An often better alternative is to use the atomic
> forms instead. "Multiple" is imprecise, too: "Potentially racy" or
> some such would come closer.
I think we can go only with:
 " This operation is non-atomic and can be reordered."
and drop:
 " If two examples of this operation race, one can appear to ... "

>=20
> Also did Julien(?) really ask that these comments be reproduced on
> both the functions supposed to be used throughout the codebase _and_
> the generic_*() ones (being merely internal helpers/fallbacks)?
At least, I understood that in this way.

>=20
> > +/**
> > + * generic_test_bit - Determine whether a bit is set
> > + * @nr: bit number to test
> > + * @addr: Address to start counting from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
>=20
> You got carried away updating comments - there's no raciness for
> simple test_bit(). As is also expressed by its name not having those
> double underscores that the others have.
Then it is true for every function in this header. Based on the naming
the conclusion can be done if it is atomic/npn-atomic and can/can't be
reordered. So the comments aren't seem very useful.

~ Oleksii

>=20
> > +static always_inline bool generic_test_bit(int nr, const volatile
> > void *addr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 bitop_uint_t mask =3D BITOP_MASK(nr);
> > +=C2=A0=C2=A0=C2=A0 const volatile bitop_uint_t *p =3D
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (const volatile bitop_uint_=
t *)addr + BITOP_WORD(nr);
> > +
> > +=C2=A0=C2=A0=C2=A0 return (*p & mask);
> > +}
> > +
> > +/**
> > + * __test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.=C2=A0 You must protect multiple accesses with a
> > lock.
> > + */
> > +static always_inline bool
> > +__test_and_set_bit(int nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_set_bit
> > +#define arch__test_and_set_bit generic__test_and_set_bit
> > +#endif
> > +
> > +=C2=A0=C2=A0=C2=A0 return arch__test_and_set_bit(nr, addr);
> > +}
>=20
> See Julien's comments on the earlier version as well as what Andrew
> has
> now done for ffs()/fls(), avoiding the largely pointless fallback
> #define.
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Tue May 28 08:53:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 08:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731046.1136354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsaR-0002kE-2j; Tue, 28 May 2024 08:53:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731046.1136354; Tue, 28 May 2024 08:53:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsaQ-0002k7-Vz; Tue, 28 May 2024 08:53:38 +0000
Received: by outflank-mailman (input) for mailman id 731046;
 Tue, 28 May 2024 08:53:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sBsaQ-0002k1-D4
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 08:53:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBsaJ-00019p-S9; Tue, 28 May 2024 08:53:31 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBsaJ-0003Ku-Kh; Tue, 28 May 2024 08:53:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=mIApNu2+YKmSjAOSkzaCtu5hTXIYSB/3p0oKKfEAhJ4=; b=ciuCuDIhvWOUUaxJ6EPCAi7PAG
	VGcVy0amPl5tJJjqatADcpRG0JQk5qhhRL5v6pA3MHsWUTp64X7k9ahRJumMwxrW1s+FQ/kZXfduS
	Qch9Ml3lOCLPjuxVwt2ePt/EYa/nsV4ADQAteQm0RkUssOjrOisKa6p6JxDIV2Jca1eI=;
Message-ID: <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
Date: Tue, 28 May 2024 09:53:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: "Oleksii K." <oleksii.kurochko@gmail.com>, Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 28/05/2024 09:37, Oleksii K. wrote:
> On Tue, 2024-05-28 at 08:20 +0200, Jan Beulich wrote:
>> On 24.05.2024 13:08, Oleksii Kurochko wrote:
>>> The following generic functions were introduced:
>>> * test_bit
>>> * generic__test_and_set_bit
>>> * generic__test_and_clear_bit
>>> * generic__test_and_change_bit
>>>
>>> These functions and macros can be useful for architectures
>>> that don't have corresponding arch-specific instructions.
>>>
>>> Also, the patch introduces the following generics which are
>>> used by the functions mentioned above:
>>> * BITOP_BITS_PER_WORD
>>> * BITOP_MASK
>>> * BITOP_WORD
>>> * BITOP_TYPE
>>>
>>> BITS_PER_BYTE was moved to xen/bitops.h as BITS_PER_BYTE is the
>>> same
>>> across architectures.
>>>
>>> The following approach was chosen for generic*() and arch*() bit
>>> operation functions:
>>> If the bit operation function that is going to be generic starts
>>> with the prefix "__", then the corresponding generic/arch function
>>> will also contain the "__" prefix. For example:
>>>   * test_bit() will be defined using arch_test_bit() and
>>>     generic_test_bit().
>>>   * __test_and_set_bit() will be defined using
>>>     arch__test_and_set_bit() and generic__test_and_set_bit().
>>>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>>   Reviewed-by: Jan Beulich jbeulich@suse.com? Jan gave his R-by for
>>> the previous
>>>   version of the patch, but some changes were done, so I wasn't sure
>>> if I could
>>>   use the R-by in this patch version.
>>
>> That really depends on the nature of the changes. Seeing the pretty
>> long list below, I think it was appropriate to drop the R-b.
>>
>>> ---
>>> Changes in V11:
>>>   - fix identation in generic_test_bit() function.
>>>   - move definition of BITS_PER_BYTE from <arch>/bitops.h to
>>> xen/bitops.h
>>
>> I realize you were asked to do this, but I'm not overly happy with
>> it.
>> BITS_PER_BYTE is far more similar to BITS_PER_LONG than to
>> BITOP_BITS_PER_WORD. Plus in any event that change is entirely
>> unrelated
>> here.
> So where then it should be introduced? x86 introduces that in config.h,
> Arm in asm/bitops.h.

I would be fine if it is defined in config.h for Arm.

> I am okay to revert this change and make a separate patch.

[...]


>> Also did Julien(?) really ask that these comments be reproduced on
>> both the functions supposed to be used throughout the codebase _and_
>> the generic_*() ones (being merely internal helpers/fallbacks)?
> At least, I understood that in this way.

I suggested to duplicate. But I also proposed an alternative to move the 
comment:

"I think this comment should be duplicated (or moved to) on top of"

I don't have a strong preference which one is used.

> 
>>
>>> +/**
>>> + * generic_test_bit - Determine whether a bit is set
>>> + * @nr: bit number to test
>>> + * @addr: Address to start counting from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>
>> You got carried away updating comments - there's no raciness for
>> simple test_bit(). As is also expressed by its name not having those
>> double underscores that the others have.
> Then it is true for every function in this header. Based on the naming
> the conclusion can be done if it is atomic/npn-atomic and can/can't be
> reordered. 

So let me start with that my only request is to keep the existing 
comments as you move it. It looks like there were none of test_bit() before.

 > So the comments aren't seem very useful.

The comments *are* useful. We had several instances where non-Arm folks 
thought all the operations were atomic on Arm as well. And the usual 
suggestion is to add barriers in the Arm version which is a no-go.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 28 09:18:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 09:18:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731052.1136365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsyD-0005qg-QD; Tue, 28 May 2024 09:18:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731052.1136365; Tue, 28 May 2024 09:18:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBsyD-0005qZ-Mo; Tue, 28 May 2024 09:18:13 +0000
Received: by outflank-mailman (input) for mailman id 731052;
 Tue, 28 May 2024 09:18:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sBsyC-0005qT-Ud
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 09:18:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBsyC-0001a8-IJ; Tue, 28 May 2024 09:18:12 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sBsyC-0004sA-8b; Tue, 28 May 2024 09:18:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YJ93bg5WbEIzO+FmzdVwg7AAzvKyktDzrPdI/cCDzWY=; b=lfko9Eo4lImfUs1mFLkCvIJR6R
	W4HH1PeFMpUo8GymZq1k3My4uC/T1KqjXSbDZ1taqemZHq+KWRrmvDRB5cc3AvA8Bhy7XwuslQ36q
	Y3/O4D33snPKMRtqyDFL9z1JDuadcKuM8g66R9NbSJRzQrpu3MQZN7ldckGlPtil9WFQ=;
Message-ID: <bc6660ef-59f1-4514-9792-067d987e3fbc@xen.org>
Date: Tue, 28 May 2024 10:18:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v16 1/5] arm/vpci: honor access size when returning an
 error
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240522225927.77398-1-stewart.hildebrand@amd.com>
 <20240522225927.77398-2-stewart.hildebrand@amd.com>
 <Zk72jPtd9iXhChbc@macbook> <3b3fd3a2-6b5c-41a2-a0b3-219a362f645f@xen.org>
 <ZlWDvJ9_Om_3gX3b@macbook>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ZlWDvJ9_Om_3gX3b@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Roger,

On 28/05/2024 08:11, Roger Pau Monné wrote:
> On Mon, May 27, 2024 at 10:14:59PM +0100, Julien Grall wrote:
>> Hi Roger,
>>
>> On 23/05/2024 08:55, Roger Pau Monné wrote:
>>> On Wed, May 22, 2024 at 06:59:20PM -0400, Stewart Hildebrand wrote:
>>>> From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
>>>>
>>>> Guest can try to read config space using different access sizes: 8,
>>>> 16, 32, 64 bits. We need to take this into account when we are
>>>> returning an error back to MMIO handler, otherwise it is possible to
>>>> provide more data than requested: i.e. guest issues LDRB instruction
>>>> to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
>>>> register.
>>>
>>> Shouldn't this be taken care of in the trap handler subsystem, rather
>>> than forcing each handler to ensure the returned data matches the
>>> access size?
>>
>> I understand how this can be useful when we return all 1s.
>>
>> However, in most of the current cases, we already need to deal with the
>> masking because the data is extracted from a wider field (for instance, see
>> the vGIC emulation). For those handlers, I would argue it would be
>> concerning/ a bug if the handler return bits above the access size.
>> Although, this would only impact the guest itself.
> 
> Even if there was a bug in the handler, it would be mitigated by the
> truncation done in io.c.
> 
>> So overall, this seems to be a matter of taste and I don't quite (yet) see
>> the benefits to do it in io.c. Regardless that...
> 
> It's up to you really, it's all ARM code so I don't really have a
> stake.  IMO it makes the handlers more complicated and fragile.

I will let the other Arm folks commenting on it.

> 
> If nothing else I would at least add an ASSERT() in io.c to ensure
> that the data returned from the handler matches the size constrains
> you expect.
That would be a good idea.

> 
>>>
>>> IOW, something like:
>>>
>>> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
>>> index 96c740d5636c..b7e12df85f87 100644
>>> --- a/xen/arch/arm/io.c
>>> +++ b/xen/arch/arm/io.c
>>> @@ -37,6 +37,7 @@ static enum io_state handle_read(const struct mmio_handler *handler,
>>>            return IO_ABORT;
>>>
>>>        r = sign_extend(dabt, r);
>>> +    r = r & GENMASK_ULL((1U << dabt.size) * 8 - 1, 0);
>>
>> ... in some case we need to sign extend up to the width of the register
>> (even if the access is 8-byte). So we would need to do the masking *before*
>> calling sign_extend().
> 
> I would consider doing the truncation in sign_extend() if suitable,
> even if that's doing more than what the function name implies.

If we decide to do a general truncation, then I would rather prefer if 
it happens outside of sign_extend(). Or the function needs to be renamed 
(I can't find a good name so far).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue May 28 09:39:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 09:39:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731058.1136386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtIa-0000lS-Ni; Tue, 28 May 2024 09:39:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731058.1136386; Tue, 28 May 2024 09:39:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtIa-0000lL-It; Tue, 28 May 2024 09:39:16 +0000
Received: by outflank-mailman (input) for mailman id 731058;
 Tue, 28 May 2024 09:39:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FWDT=M7=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1sBtIZ-0000X2-AL
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 09:39:15 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 24f36bf4-1cd6-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 11:39:14 +0200 (CEST)
Received: from truciolo.bugseng.com (unknown [78.210.197.134])
 by support.bugseng.com (Postfix) with ESMTPSA id 53D754EE073F;
 Tue, 28 May 2024 11:39:11 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24f36bf4-1cd6-11ef-90a1-e314d9c70b13
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Subject: [XEN PATCH] automation/eclair: extend and modifiy existing deviations of MISRA C:2012 Rule 16.3
Date: Tue, 28 May 2024 11:39:03 +0200
Message-Id: <f65f0e884038773952d50e35552bb97c6ae8ea19.1716888049.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to deviate more cases where an
unintentional fallthrough cannot happen.

Add Rule 16.3 to the monitored set and tag as clean for arm.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
In previous discussions about Rule 16.3, the preference expressed was to deviate
cases where a fallthrough cannot occur (rather that refactoring the code).
---
 .../eclair_analysis/ECLAIR/deviations.ecl     | 30 ++++++++++++++-----
 .../eclair_analysis/ECLAIR/monitored.ecl      |  1 +
 automation/eclair_analysis/ECLAIR/tagging.ecl |  2 +-
 docs/misra/deviations.rst                     | 28 +++++++++++++++--
 4 files changed, 49 insertions(+), 12 deletions(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index cf62a874d9..95f07718ba 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -364,14 +364,29 @@ therefore it is deemed better to leave such files as is."
 -config=MC3R1.R16.2,reports+={deliberate, "any_area(any_loc(file(x86_emulate||x86_svm_emulate)))"}
 -doc_end
 
--doc_begin="Switch clauses ending with continue, goto, return statements are
-safe."
--config=MC3R1.R16.3,terminals+={safe, "node(continue_stmt||goto_stmt||return_stmt)"}
+-doc_begin="Statements that change the control flow (i.e., break, continue, goto, return) and calls to functions that does not return the control back are \"allowed terminal statements\"."
+-stmt_selector+={r16_3_allowed_terminal, "node(break_stmt||continue_stmt||goto_stmt||return_stmt)||call(property(noreturn))"}
+-config=MC3R1.R16.3,terminals+={safe, "r16_3_allowed_terminal"}
+-doc_end
+
+-doc_begin="An if-else statement having both branches ending with an allowed terminal statement is itself an allowed terminal statement."
+-stmt_selector+={r16_3_if, "node(if_stmt)&&(child(then,r16_3_allowed_terminal)||child(then,any_stmt(stmt,-1,r16_3_allowed_terminal)))"}
+-stmt_selector+={r16_3_else, "node(if_stmt)&&(child(else,r16_3_allowed_terminal)||child(else,any_stmt(stmt,-1,r16_3_allowed_terminal)))"}
+-stmt_selector+={r16_3_if_else, "r16_3_if&&r16_3_else"}
+-config=MC3R1.R16.3,terminals+={safe, "r16_3_if_else"}
+-doc_end
+
+-doc_begin="An if-else statement having an always true condition and the true branch ending with an allowed terminal statement is itself an allowed terminal statement."
+-stmt_selector+={r16_3_if_true, "r16_3_if&&child(cond,definitely_in(1..))"}
+-config=MC3R1.R16.3,terminals+={safe, "r16_3_if_true"}
+-doc_end
+
+-doc_begin="Switch clauses ending with a statement expression which, in turn, ends with an allowed terminal statement are safe."
+-config=MC3R1.R16.3,terminals+={safe, "node(stmt_expr)&&child(stmt,node(compound_stmt)&&any_stmt(stmt,-1,r16_3_allowed_terminal||r16_3_if_else||r16_3_if_true))"}
 -doc_end
 
--doc_begin="Switch clauses ending with a call to a function that does not give
-the control back (i.e., a function with attribute noreturn) are safe."
--config=MC3R1.R16.3,terminals+={safe, "call(property(noreturn))"}
+-doc_begin="Switch clauses ending with a do-while-false which, in turn, ends with an allowed terminal statement are safe, except for debug macro ASSERT_UNREACHABLE()."
+-config=MC3R1.R16.3,terminals+={safe, "!macro(name(ASSERT_UNREACHABLE))&&node(do_stmt)&&child(cond,definitely_in(0))&&child(body,any_stmt(stmt,-1,r16_3_allowed_terminal||r16_3_if_else||r16_3_if_true))"}
 -doc_end
 
 -doc_begin="Switch clauses ending with pseudo-keyword \"fallthrough\" are
@@ -383,8 +398,7 @@ safe."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
 -doc_end
 
--doc_begin="Switch clauses not ending with the break statement are safe if an
-explicit comment indicating the fallthrough intention is present."
+-doc_begin="Switch clauses ending with an explicit comment indicating the fallthrough intention is present are safe."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
 -doc_end
 
diff --git a/automation/eclair_analysis/ECLAIR/monitored.ecl b/automation/eclair_analysis/ECLAIR/monitored.ecl
index 4daecb0c83..45a60074f9 100644
--- a/automation/eclair_analysis/ECLAIR/monitored.ecl
+++ b/automation/eclair_analysis/ECLAIR/monitored.ecl
@@ -22,6 +22,7 @@
 -enable=MC3R1.R14.1
 -enable=MC3R1.R14.4
 -enable=MC3R1.R16.2
+-enable=MC3R1.R16.3
 -enable=MC3R1.R16.6
 -enable=MC3R1.R16.7
 -enable=MC3R1.R17.1
diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index a354ff322e..07de2e7b65 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -105,7 +105,7 @@ if(string_equal(target,"x86_64"),
 )
 
 if(string_equal(target,"arm64"),
-    service_selector({"additional_clean_guidelines","MC3R1.R14.4||MC3R1.R16.6||MC3R1.R20.12||MC3R1.R2.1||MC3R1.R5.3||MC3R1.R7.2||MC3R1.R7.3||MC3R1.R8.6||MC3R1.R9.3"})
+    service_selector({"additional_clean_guidelines","MC3R1.R14.4||MC3R1.R16.3||MC3R1.R16.6||MC3R1.R20.12||MC3R1.R2.1||MC3R1.R5.3||MC3R1.R7.2||MC3R1.R7.3||MC3R1.R8.6||MC3R1.R9.3"})
 )
 
 -reports+={clean:added,"service(clean_guidelines_common||additional_clean_guidelines)"}
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 36959aa44a..88164fb0e1 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -309,12 +309,34 @@ Deviations related to MISRA C:2012 Rules:
      - Tagged as `deliberate` for ECLAIR.
 
    * - R16.3
-     - Switch clauses ending with continue, goto, return statements are safe.
+     - Statements that change the control flow (i.e., break, continue, goto,
+       return) and calls to functions that does not return the control back are
+       \"allowed terminal statements\".
      - Tagged as `safe` for ECLAIR.
 
    * - R16.3
-     - Switch clauses ending with a call to a function that does not give
-       the control back (i.e., a function with attribute noreturn) are safe.
+     - An if-else statement having both branches ending with one of the allowed
+       terminal statemets is itself an allowed terminal statements.
+     - Tagged as `safe` for ECLAIR.
+
+   * - R16.3
+     - An if-else statement having an always true condition and the true
+       branch ending with an allowed terminal statement is itself an allowed
+       terminal statement.
+     - Tagged as `safe` for ECLAIR.
+
+   * - R16.3
+     - Switch clauses ending with a statement expression which, in turn, ends
+       with an allowed terminal statement are safe (e.g., the expansion of
+       generate_exception()).
+     - Tagged as `safe` for ECLAIR.
+
+   * - R16.3
+     - Switch clauses ending with a do-while-false which, in turn, ends with an
+       allowed terminal statement are safe (e.g., PARSE_ERR_RET()).
+       Being ASSERT_UNREACHABLE() a construct that is effective in debug builds
+       only, it is not considered as an allowed terminal statement, despite its
+       definition.
      - Tagged as `safe` for ECLAIR.
 
    * - R16.3
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 28 09:39:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 09:39:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731057.1136375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtIZ-0000XJ-FP; Tue, 28 May 2024 09:39:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731057.1136375; Tue, 28 May 2024 09:39:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtIZ-0000XC-CU; Tue, 28 May 2024 09:39:15 +0000
Received: by outflank-mailman (input) for mailman id 731057;
 Tue, 28 May 2024 09:39:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBtIY-0000X2-3i
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 09:39:14 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 219905aa-1cd6-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 11:39:08 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2e95a74d51fso7358381fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 02:39:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda46e1sm592227166b.196.2024.05.28.02.39.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 02:39:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 219905aa-1cd6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716889148; x=1717493948; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=OmmOJgb9cOlQkpl935RLGJTN/8f87RlqaEkuudmwh9Q=;
        b=MlkQCMKaPjvRQJd5nfDC/Lge42a5wCNMlYvdiUdEwuVNOd4XAQJaHdDUuRhZHkUbeS
         03KspmETjVGhf4x5IArfQrtKP6pWaebltP6N9u8Z3yKiK+DLqzy7DYaMYz7b8QRSkvXd
         omBU2cp5OC10b5Xuuabs4sTcHWQf4n3feb0GZV/Rc8PBJNmNnFJ4B+tsa80UQkOfN7V+
         IEa8xRDFqmxPp7JRHEKraMDwb7ha8PO5gIr2reiH9iooyrGqm7Ap63e0ylv6kbt4c9EF
         I2IrTCVPcs44Pd3f6zDwZwiDD2OcBSzZUd5tEJKjvSvJ+CjrbWsdGrYGcglOOadR4DTL
         JC/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716889148; x=1717493948;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OmmOJgb9cOlQkpl935RLGJTN/8f87RlqaEkuudmwh9Q=;
        b=h9gWFVzFJEnIGrw9yC15a22E4Bi42GtbrHk3jw/fN+vUd9mF0cw9zBxEKOZ7yku8DO
         g5TAVGNolOnqVPuJ//kR22rtdIprl55/VMbnUmhlt5PY0HwRLM3Ujhej/33IQwqPEWr5
         4esmYerl3irs8/jqCZ2BHlE+1zRYcTOJD5ymcHazDCmnVZTajiWsnwem5wxB4yQBQuy+
         ddtcf3HimFxXHnaPppZNhZqafP0b7jyAs18l69PrO5bdoHNK1dz2iGrqRH0c/OqySkxc
         MtmwsUP0H/3G2VFTV7GSjaXgYwxK9JM3jVlXhjgub/lBr3rgjDv1oFD2zvv6ieQBB5d1
         ydXQ==
X-Forwarded-Encrypted: i=1; AJvYcCUWgUzAHw4zxxuMxxjVUJsxfrdjvYxzeTghMUBktTZCcDGFTEgyVTW08yVQ8QxLJSlAKsOusd4+ZER5IyzCAe7ydOglKREf0uQGl1+bIu4=
X-Gm-Message-State: AOJu0Yw/8bJP6H1bX37gFhxHZwELexE9ATShzWbAgC76WrxU3rFGAzxo
	bXgzZhpwKn3u6oMoLuH2eMwU0nIJTkkpjA2nxl4GBgu6m0Z7hGRKfjHxWvfR+g==
X-Google-Smtp-Source: AGHT+IFzMCTOJ1peU8vOLHt8SMxS3EEwszQpeTkbL1Q+aFq43O4IYBnn8Sbdt/jRb/Mom/b7l4q9zw==
X-Received: by 2002:a2e:961a:0:b0:2e5:6957:187a with SMTP id 38308e7fff4ca-2e95b041638mr91060931fa.4.1716889148005;
        Tue, 28 May 2024 02:39:08 -0700 (PDT)
Message-ID: <26c35562-e9ba-41cd-8bae-69469a5f0f88@suse.com>
Date: Tue, 28 May 2024 11:39:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.05.2024 10:37, Oleksii K. wrote:
> On Tue, 2024-05-28 at 08:20 +0200, Jan Beulich wrote:
>> On 24.05.2024 13:08, Oleksii Kurochko wrote:
>>> +/**
>>> + * generic_test_bit - Determine whether a bit is set
>>> + * @nr: bit number to test
>>> + * @addr: Address to start counting from
>>> + *
>>> + * This operation is non-atomic and can be reordered.
>>> + * If two examples of this operation race, one can appear to
>>> succeed
>>> + * but actually fail.  You must protect multiple accesses with a
>>> lock.
>>> + */
>>
>> You got carried away updating comments - there's no raciness for
>> simple test_bit(). As is also expressed by its name not having those
>> double underscores that the others have.
> Then it is true for every function in this header. Based on the naming
> the conclusion can be done if it is atomic/npn-atomic and can/can't be
> reordered. So the comments aren't seem very useful.

I disagree - test_bit() is different, in not being a read-modify-write
operation.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 28 10:02:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 10:02:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731070.1136394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtfJ-0005Jq-Cp; Tue, 28 May 2024 10:02:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731070.1136394; Tue, 28 May 2024 10:02:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBtfJ-0005Jj-A8; Tue, 28 May 2024 10:02:45 +0000
Received: by outflank-mailman (input) for mailman id 731070;
 Tue, 28 May 2024 10:02:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBtfI-0005JZ-Kx; Tue, 28 May 2024 10:02:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBtfI-0002Oi-Fr; Tue, 28 May 2024 10:02:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBtfI-0001DX-42; Tue, 28 May 2024 10:02:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBtfI-0008GO-3h; Tue, 28 May 2024 10:02:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MBqRTOjl0s5IyNypCUW2KZE5LifGKGAjtkeCPjagYH4=; b=hom6wodAXaVXVybNy0xVho5yVu
	dqBfJBNZcoKWnuY/YmZ5+Gtu1M8+gmkmcRe/y86y4xtDXFYHuC6L37aPkzLKJIMNro58vZX7/r5nE
	xPFMDozDKoByil2q8GYBXAP+I9ebp4HTI62WSvSS9j3P1rR3HxCgGS1xNosEzihsPh3Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186163-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186163: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-armhf:xen-build:fail:regression
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-unstable:build-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
X-Osstest-Versions-That:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 10:02:44 +0000

flight 186163 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186163/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build      fail in 186157 REGR. vs. 186163

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail pass in 186157

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-raw       1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-qcow2     1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl           1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-libvirt      1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-examine      1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)          blocked in 186157 n/a
 build-armhf-libvirt           1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-libvirt-vhd  1 build-check(1)           blocked in 186157 n/a
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 186157
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186157
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186157
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186157
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186157
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186157
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0
baseline version:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0

Last test of basis   186163  2024-05-28 01:53:34 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue May 28 10:36:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 10:36:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731082.1136404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuBk-0000xE-UL; Tue, 28 May 2024 10:36:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731082.1136404; Tue, 28 May 2024 10:36:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuBk-0000x7-Rn; Tue, 28 May 2024 10:36:16 +0000
Received: by outflank-mailman (input) for mailman id 731082;
 Tue, 28 May 2024 10:36:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuBj-0000wx-CI; Tue, 28 May 2024 10:36:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuBj-0002u6-6c; Tue, 28 May 2024 10:36:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuBi-00021g-Vf; Tue, 28 May 2024 10:36:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuBi-00040V-VD; Tue, 28 May 2024 10:36:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1m8qx6mEe6gdNMs3zu9lnat5v18BMwKXa6w6j5Hic0w=; b=KGpZyKaZXZ6soKPx8sGVbmvvvh
	Ub7xEKoBVPh5q3JyJhWc2Jm9JaWq63SVDh3aPzRwDJTFvePsAYb08d+9nWaZzjOuWHWTb09h6RrAC
	vB8feNKvgNq00W3CRpqe8N/p4JNE6YZxDbILEbWSa1AEnkgbNo6jTbVsdHeLPC/pRhRo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186166-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186166: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:test-armhf-armhf-xl:xen-boot:fail:regression
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d93f78bfe25f695d8ffb61d110da9df293ed71b
X-Osstest-Versions-That:
    xen=96af090e33130b0bf0953f3ccab8e7a163392318
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 10:36:14 +0000

flight 186166 xen-unstable-smoke real [real]
flight 186167 xen-unstable-smoke real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186166/
http://logs.test-lab.xenproject.org/osstest/logs/186167/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl           8 xen-boot                 fail REGR. vs. 186164

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2d93f78bfe25f695d8ffb61d110da9df293ed71b
baseline version:
 xen                  96af090e33130b0bf0953f3ccab8e7a163392318

Last test of basis   186164  2024-05-28 02:00:23 Z    0 days
Testing same since   186166  2024-05-28 07:02:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 2d93f78bfe25f695d8ffb61d110da9df293ed71b
Author: Nicola Vetrini <nicola.vetrini@bugseng.com>
Date:   Tue May 28 08:52:27 2024 +0200

    x86/traps: address violation of MISRA C Rule 8.4
    
    Rule 8.4 states: "A compatible declaration shall be visible when
    an object or function with external linkage is defined".
    
    The function do_general_protection is either used is asm code
    or only within this unit, so there is no risk of this getting
    out of sync with its definition, but the function must remain
    extern.
    
    Therefore, this function is deviated using a comment-based deviation.
    No functional change.
    
    Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 8b977fe57254ff8d343e4bf50cf98fa6c2b36b8b
Author: Jason Andryuk <jason.andryuk@amd.com>
Date:   Tue May 28 08:52:15 2024 +0200

    CHANGELOG: Mention libxl blktap/tapback support
    
    Add entry for backendtype=tap support in libxl.  blktap needs some
    changes to work with libxl, which haven't been merged.  They are
    available from this PR: https://github.com/xapi-project/blktap/pull/394
    
    Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
    Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue May 28 10:58:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 10:58:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731091.1136418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuWV-0003m3-Ll; Tue, 28 May 2024 10:57:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731091.1136418; Tue, 28 May 2024 10:57:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuWV-0003lw-Hm; Tue, 28 May 2024 10:57:43 +0000
Received: by outflank-mailman (input) for mailman id 731091;
 Tue, 28 May 2024 10:57:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuWU-0003lm-Ku; Tue, 28 May 2024 10:57:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuWU-0003GB-I3; Tue, 28 May 2024 10:57:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuWU-0002Vj-6Q; Tue, 28 May 2024 10:57:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sBuWU-00037C-5g; Tue, 28 May 2024 10:57:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NtL5aIA9kL1aOBeFeBeXpkMj2tC6ehzNYFPx6ULQXns=; b=FNqUDozIWePVmlFC/4pv9ZiQV+
	Lo7gY4NblNB/Lb8nwSsKd3UmXyTEhRS6MAIUSThVps9h0WY8pGDLynHQHAtqP8Cquy17YJ+1d1iSf
	LabJZlYp3bn5TrP3XPCBnZCBy4OZh3rGeud3A3u1U0qpulQ5R6mctIZnm8gPaIB4nExU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186165-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186165: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=a4f38f6ffe6a9edc001d18890ccfc3f38e72fb94
X-Osstest-Versions-That:
    libvirt=3b3efef58dc4bf6c07a73862c280e30f2023054d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 10:57:42 +0000

flight 186165 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186165/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186133
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              a4f38f6ffe6a9edc001d18890ccfc3f38e72fb94
baseline version:
 libvirt              3b3efef58dc4bf6c07a73862c280e30f2023054d

Last test of basis   186133  2024-05-24 04:18:53 Z    4 days
Testing same since   186165  2024-05-28 04:20:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andi Chandler <andi@gowling.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Laine Stump <laine@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   3b3efef58d..a4f38f6ffe  a4f38f6ffe6a9edc001d18890ccfc3f38e72fb94 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue May 28 11:08:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 11:08:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731100.1136427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBugQ-0005O7-II; Tue, 28 May 2024 11:07:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731100.1136427; Tue, 28 May 2024 11:07:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBugQ-0005O0-Fe; Tue, 28 May 2024 11:07:58 +0000
Received: by outflank-mailman (input) for mailman id 731100;
 Tue, 28 May 2024 11:07:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBugP-0005Nu-C7
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 11:07:57 +0000
Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com
 [2607:f8b0:4864:20::b30])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8821b56b-1ce2-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 13:07:55 +0200 (CEST)
Received: by mail-yb1-xb30.google.com with SMTP id
 3f1490d57ef6-df457d734abso588112276.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 04:07:55 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abca58e6sm372955585a.18.2024.05.28.04.07.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 04:07:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8821b56b-1ce2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716894474; x=1717499274; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:to:from:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GN839hZMDpwiAhOLZEJxgz+JU1RtbVta04Wq5Uy7jNg=;
        b=YE1c9ZiPgMq7/Y9bN7aPhm2LDVZG3jZAFnUy+dOSzQCcZuo2kVe/3/YIw++yDqw9da
         3MhzFqYM9vOwuYhVbfM4JUUmZyaXNGdxDy1FWIB76arxz07yDWcPPgt7IHTfWvPHm4jt
         jPTITW60EY0RUO3db+TzT7pDkcUgSPYXj0Khg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716894474; x=1717499274;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:to:from:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GN839hZMDpwiAhOLZEJxgz+JU1RtbVta04Wq5Uy7jNg=;
        b=rLxhRabNr3M+wkcTR+QO2rUdVAerDgm9F/4KUyrOrlik8xxqXNl4+DpkzA5CMFllUQ
         VWilX4X1IxUtf33Z2Ako2H27BBO3TciKl5I3F1+we8scOF+N7IqTfIew4ysgsXFTnjnX
         5Gd1g4xwtVHOSARybW41MLY4EgjuE7scmizjdwLv6XMttaV53743WzrpJHLGJAXTbjD0
         CZ3+K8M8fnGnXUwoD/ykuoZXmRsmH6FMgwiyc0/O97xIplE4lPo7PwRdxHRuJksSpuSL
         LW1SFso3uLvJe3iT3NoTgu+5fYYEMvSL1ur7EPH2lpYrTTwftDgWMxkmEIguJYdaJqp1
         YCxg==
X-Forwarded-Encrypted: i=1; AJvYcCWyZtjKj4ZfORZ/j2EmKXbIn/aLFglnI1put+8kBtN6fJ/1v9Py0Ik72AQtAxKGpeAo+VJOBDDPY7lm+lFNdH+eOiFt+VjJNKsy+NpeO/Y=
X-Gm-Message-State: AOJu0Yz15UGMTGQmnkd1epJlSNtRiZulWJ/y7pFVC5hgLzfWIoQjW+Yp
	dBuREoiKuK8Cj3AYCrSqpyNeVlzrtb8nXDsf/2zryxR0C641MRD4RPJ3LrTuxl5neKLSWrIOwGT
	yKdw=
X-Google-Smtp-Source: AGHT+IFJLCJWmC5hDusy9Thd1NXH7E6g7dQqe371cws5FnhIU8xJCIyINLrPG715Qz+H6HLt5BxWzg==
X-Received: by 2002:a81:5209:0:b0:609:fff2:3bbc with SMTP id 00721157ae682-62a08f152c5mr112070127b3.49.1716894472527;
        Tue, 28 May 2024 04:07:52 -0700 (PDT)
Message-ID: <cda7e754-5b6a-47ff-a2b3-dc3c11261644@citrix.com>
Date: Tue, 28 May 2024 12:07:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Linux HVM fails to start with PANIC: early exception 0x00 IP
 0010:clear_page_orig+0x12/0x40 error 0
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel <xen-devel@lists.xenproject.org>
References: <ZkAmdiHI-DfUMiOV@mail-itl>
 <653ba4f5-dc1e-48b9-9833-35b47bfc5fc4@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <653ba4f5-dc1e-48b9-9833-35b47bfc5fc4@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12/05/2024 3:48 pm, Andrew Cooper wrote:
> On 12/05/2024 3:16 am, Marek Marczykowski-Górecki wrote:
>> Hi,
>>
>> I've got a report[1] that after some update Linux HVM fails to start with the
>> error as in the subject. It looks to be caused by some change between
>> Xen 4.17.3 and 4.17.4. Here the failure is on Linux 6.6.25 (both dom0
>> and domU), but the 6.1.62 that worked with older Xen before, now fails
>> too. The full error (logged via earlyprintk=xen) is:
>>
>>     [    0.009500] Using GB pages for direct mapping
>>     PANIC: early exception 0x00 IP 10:ffffffffb01c32e2 error 0 cr2 0xffffa08649801000
>>     [    0.009606] CPU: 0 PID: 0 Comm: swapper Not tainted 6.6.25-1.qubes.fc37.x86_64 #1
>>     [    0.009665] Hardware name: Xen HVM domU, BIOS 4.17.4 04/26/2024
>>     [    0.009710] RIP: 0010:clear_page_orig+0x12/0x40
>>     [    0.009766] Code: 84 00 00 00 00 00 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 b9 40 00 00 00 0f 1f 44 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 47
>>     [    0.009862] RSP: 0000:ffffffffb0e03d58 EFLAGS: 00010016 ORIG_RAX: 0000000000000000
>>     [    0.009915] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000003f
>>     [    0.009967] RDX: 0000000000009801 RSI: 0000000000000000 RDI: ffffa08649801000
>>     [    0.010015] RBP: 0000000000000001 R08: 0000000000000001 R09: 6b7f283562d74b16
>>     [    0.010063] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
>>     [    0.010112] R13: 0000000000000000 R14: ffffffffb0e22a08 R15: ffffa08640000000
>>     [    0.010161] FS:  0000000000000000(0000) GS:ffffffffb16ea000(0000) knlGS:0000000000000000
>>     [    0.010214] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>     [    0.010257] CR2: ffffa08649801000 CR3: 0000000008e80000 CR4: 00000000000000b0
>>     [    0.010310] Call Trace:
>>     [    0.010341]  <TASK>
>>     [    0.010372]  ? early_fixup_exception+0xf7/0x190
>>     [    0.010416]  ? early_idt_handler_common+0x2f/0x3a
>>     [    0.010460]  ? clear_page_orig+0x12/0x40
>>     [    0.010501]  ? alloc_low_pages+0xeb/0x150
>>     [    0.010541]  ? __kernel_physical_mapping_init+0x1d2/0x630
>>     [    0.010588]  ? init_memory_mapping+0x83/0x160
>>     [    0.010631]  ? init_mem_mapping+0x9a/0x460
>>     [    0.010669]  ? memblock_reserve+0x6d/0xf0
>>     [    0.010709]  ? setup_arch+0x796/0xf90
>>     [    0.010748]  ? start_kernel+0x63/0x420
>>     [    0.010787]  ? x86_64_start_reservations+0x18/0x30
>>     [    0.010828]  ? x86_64_start_kernel+0x96/0xa0
>>     [    0.010868]  ? secondary_startup_64_no_verify+0x18f/0x19b
>>     [    0.010918]  </TASK>
>>
>> I'm pretty sure the exception 0 is misleading here, I don't see how it
>> could be #DE.
>>
>> More logs (including full hypervisor log) are attached to the linked
>> issue.
>>
>> This is on HP 240 g7, and my educated guess is it's Intel Celeron N4020
>> CPU. I cannot reproduce the issue on different hardware.
>>
>> PVH domains seems to work.
>>
>> Any ideas what could have happened here?
> Yes.
>
> Revert the microcode back to revision 0x38 for now.

The regression is fixed in revision 0x42.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 11:22:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 11:22:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731106.1136438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuuO-00083b-L0; Tue, 28 May 2024 11:22:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731106.1136438; Tue, 28 May 2024 11:22:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBuuO-00083U-H5; Tue, 28 May 2024 11:22:24 +0000
Received: by outflank-mailman (input) for mailman id 731106;
 Tue, 28 May 2024 11:22:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wewE=M7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sBuuM-000833-Jb
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 11:22:22 +0000
Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com
 [2607:f8b0:4864:20::b32])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8bac1603-1ce4-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 13:22:20 +0200 (CEST)
Received: by mail-yb1-xb32.google.com with SMTP id
 3f1490d57ef6-df771cae64aso615675276.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 04:22:19 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abcc1009sm373155385a.33.2024.05.28.04.22.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 04:22:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bac1603-1ce4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716895338; x=1717500138; darn=lists.xenproject.org;
        h=content-disposition:mime-version:message-id:subject:cc:to:from:date
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nzM3xHL4uqemtoYcFsksOMODFqpqWyU1iMQLlMOajCI=;
        b=HsNKG+XNK7ZZmdr286jYvsIckEMAXYmrPVhIQ+FELX/izdJiYrabbuNo+kmQIM/lBQ
         SRxAvFx4UzG4YgsP2UfZaqSor1lrvmKU1C/TKnKRbsvJjjbORn9Z2SpuAtGfGvW/XttW
         pKCu+E64ssNEMb4AA2ojUama772D4fekxWj2E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716895338; x=1717500138;
        h=content-disposition:mime-version:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nzM3xHL4uqemtoYcFsksOMODFqpqWyU1iMQLlMOajCI=;
        b=E1LUNwJBrWb36P1kSDKL5Qxrpv4tIxpykRIxYL3fFGMDqnPrdRsdbHpkWbSDwHJ895
         Sf9Y7pPGRH8Wskx9MCRnP/2ex+lGyBetez/ohmAggbIxINUmlXQhPX0g8PVlJVcAVB+A
         b0sBtLCRjSNt/O3tFzXUWYvDxThPe0m1K+rImTqO/0rkm5i3VyuDLLVTct/pjy7GAg9u
         BU+Z33tiKjuoyrSg7IKSpOjLuUWp4MKhQ46GUT1gk23yUIIIbB/QvSf3KItEMZnUIHrH
         GYIYDBocq0ZztseqYM2T2wu705OFGW9uGAkgfP0lyqqPk4hlvV9HU+THy6Kf7BB1xtRN
         yFgQ==
X-Gm-Message-State: AOJu0Yx42LLXtnV0opyuLtCjOIwRXvsqOhJy/G0+q7ZawrSEDXMqmO6a
	G31kJlrtH9nKTYI/fKo6h2P+OvtH+QjFQsYxXDvJSLEUHsGDfXOFAg2jbD6obAz9zvkyOHFVbUx
	3
X-Google-Smtp-Source: AGHT+IEfiJTSseUgDEdaVsoCp7jDYWZVg87x7v86n7abjE0gNCiS40BabDAGSMWNNzvg5ZRYJYF3FA==
X-Received: by 2002:a5b:d06:0:b0:de6:3c4:c3a with SMTP id 3f1490d57ef6-df77217db00mr9716437276.10.1716895337946;
        Tue, 28 May 2024 04:22:17 -0700 (PDT)
Date: Tue, 28 May 2024 13:22:15 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>, Dario Faggioli <dfaggioli@suse.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
Message-ID: <ZlW-ZyFeAzvh3pGG@macbook>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline

Hello,

When the stop_machine_run() call in cpu_down() fails and calls the CPU
notifier CPU_DOWN_FAILED hook the following assert triggers in the
scheduling code:

Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
CPU:    0
RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
RFLAGS: 0000000000010093   CONTEXT: hypervisor
rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
cr3: 00000000574c2000   cr2: 0000000000000000
fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
 fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
Xen stack trace from rsp=ffff83203ffffd30:
   ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
   0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
   0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
   ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
   ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
   00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
   ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
   ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
   ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
   ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
   ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
   0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
   ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
   0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
   ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
Xen call trace:
   [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
   [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
   [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
   [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
   [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
   [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
   [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
   [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
   [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
   [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
   [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
   [<ffff82d040324391>] F continue_running+0x5b/0x5d


****************************************
Panic on CPU 0:
Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
****************************************

The issue seems to be that since the CPU hasn't been removed, it's
still part of prv->initialized and the assert in csched2_free_pdata()
called as part of free_cpu_rm_data() triggers.

It's easy to reproduce by substituting the stop_machine_run() call in
cpu_down() with an error.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue May 28 12:30:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 12:30:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731115.1136447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBvy4-00081f-By; Tue, 28 May 2024 12:30:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731115.1136447; Tue, 28 May 2024 12:30:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBvy4-00081Y-98; Tue, 28 May 2024 12:30:16 +0000
Received: by outflank-mailman (input) for mailman id 731115;
 Tue, 28 May 2024 12:30:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBvy3-00081R-VA
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 12:30:15 +0000
Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com
 [2607:f8b0:4864:20::734])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 073657a7-1cee-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 14:30:13 +0200 (CEST)
Received: by mail-qk1-x734.google.com with SMTP id
 af79cd13be357-794ab01e747so53484085a.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 05:30:13 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac070dc2aesm44245496d6.42.2024.05.28.05.30.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 05:30:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 073657a7-1cee-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716899411; x=1717504211; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ShwQ8ihl8BeD0n9LIaKqMxNwbnSItoXELrXM9XhMk8c=;
        b=dJnn8PHyAyikkwZfFGGe0k1Ss1lsviAEOVQ8JpnsWQ+EwHbK+ZQUyuEd+fCrD8JDPk
         u9b9VqOz1qmgYoZsKtDfngPfkrzAu+QOrswMbLR2M57Q6BNRcWVK3DW7iD0pHKCAjyoi
         u0O8n6uTlz3JcZNrWK7/RkWnApB0yZyYpFDMw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716899411; x=1717504211;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ShwQ8ihl8BeD0n9LIaKqMxNwbnSItoXELrXM9XhMk8c=;
        b=uOrtufIMboZbUgzA+MWf9yWVVlukwWJTfmCVThL0J5dpyRg/jhXDil2ZxnM7iiRqu6
         WwMMqT+5qXtPLfrAcdfm7HUGrKoNJqC2CX7z3pfr93Zt9Tto/3fnjzVQeiUUABP1XQQA
         3OhFgojDqqXbQDmMQJC8jB2Ey0Kd5rb4dqeQyYUTj8of4GChqN+gpHXySZ3zY4sMnKRj
         WXs5G9YZVCN2alEzNGQOXoyGfXS2MZ+HvVAQvOpzJG7ZVJNvby0Uxcu9jTy8ECXo5sC/
         8/c280PxxFQoeSXQFsjGjiZUDWcrTH0m+gO5Soqr+gWNm4W6UTweb+2d37K57FzGlACF
         vafQ==
X-Forwarded-Encrypted: i=1; AJvYcCUNa/baEebjmMldI2kJ3Nc6AxXMKdInpKWdoHoNnSQvyAE8wqnK51hRlf+1SeoEcaFhDK6qvDdFB0rqqxQIQPutebls7mRxy17dryAeoWw=
X-Gm-Message-State: AOJu0YyG2BBdzPOjpITYhgiU0dxZ0wF9Lons3p4ifkKUKfGn8wFsHWvA
	n6G+kUDdR0mJfw056VIUrSN7aHQwo0Dq+vkFfnpgMiZX/KqrkoyWhG40OtYjIHU=
X-Google-Smtp-Source: AGHT+IExbMiaLbkK3N1rrjO2XlfQlPFZnOp7wrZDUU930F2h2M7hqYJ8FQ/M408hFC/+RaRQUCcxYQ==
X-Received: by 2002:ad4:4eee:0:b0:6ad:8d50:5ced with SMTP id 6a1803df08f44-6ad8d505d22mr35472026d6.11.1716899411447;
        Tue, 28 May 2024 05:30:11 -0700 (PDT)
Message-ID: <b0838a62-1e6a-473a-a757-97091c84e164@citrix.com>
Date: Tue, 28 May 2024 13:30:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general
 case
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-8-andrew.cooper3@citrix.com>
 <1660a2a7-cea4-4e6f-9286-0c134c34b6fb@suse.com>
 <57a47c76-c484-4309-8a87-a51f79dd48b6@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <57a47c76-c484-4309-8a87-a51f79dd48b6@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/05/2024 2:37 pm, Jan Beulich wrote:
> On 27.05.2024 15:27, Jan Beulich wrote:
>> On 24.05.2024 22:03, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/include/asm/bitops.h
>>> +++ b/xen/arch/x86/include/asm/bitops.h
>>> @@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
>>>  
>>>  static always_inline unsigned int arch_ffs(unsigned int x)
>>>  {
>>> -    int r;
>>> +    unsigned int r;
>>> +
>>> +    if ( __builtin_constant_p(x > 0) && x > 0 )
>>> +    {
>>> +        /* Safe, when the compiler knows that x is nonzero. */
>>> +        asm ( "bsf %[val], %[res]"
>>> +              : [res] "=r" (r)
>>> +              : [val] "rm" (x) );
>>> +    }
>> In patch 11 relevant things are all in a single patch, making it easier
>> to spot that this is dead code: The sole caller already has a
>> __builtin_constant_p(), hence I don't see how the one here could ever
>> return true. With that the respective part of the description is then
>> questionable, too, I'm afraid: Where did you observe any actual effect
>> from this? Or if you did - what am I missing?
> Hmm, thinking about it: I suppose that's why you have
> __builtin_constant_p(x > 0), not __builtin_constant_p(x). I have to admit
> I'm (positively) surprised that the former may return true when the latter
> doesn't.

So was I, but this recommendation came straight from the GCC mailing
list.  And it really does work, even back in obsolete versions of GCC.

__builtin_constant_p() operates on an expression not a value, and is
documented as such.


>  Nevertheless I'm inclined to think this deserves a brief comment.

There is a comment, and it's even visible in the snippet.

> As an aside, to better match the comment inside the if()'s body, how about
>
>     if ( __builtin_constant_p(!!x) && x )
>
> ? That also may make a little more clear that this isn't just a style
> choice, but actually needed for the intended purpose.

I am not changing the logic.

Apart from anything else, your suggestion is trivially buggy.  I care
about whether the RHS collapses to a constant, and the only way of doing
that correctly is asking the compiler about the *exact* expression. 
Asking about some other expression which you hope - but do not know -
that the compiler will treat equivalently is bogus.  It would be
strictly better to only take the else clause, than to have both halves
emitted.

This is the form I've tested extensively.  It's also the clearest form
IMO.  You can experiment with alternative forms when we're not staring
down code freeze of 4.19.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 12:36:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 12:36:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731124.1136457 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBw3n-0000EM-T2; Tue, 28 May 2024 12:36:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731124.1136457; Tue, 28 May 2024 12:36:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBw3n-0000EF-QS; Tue, 28 May 2024 12:36:11 +0000
Received: by outflank-mailman (input) for mailman id 731124;
 Tue, 28 May 2024 12:36:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+/BF=M7=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sBw3m-0000E9-VA
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 12:36:11 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dbd1ccb1-1cee-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 14:36:08 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a635c83bb7eso3372766b.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 05:36:08 -0700 (PDT)
Received: from [10.80.5.21] ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579ce607407sm3488800a12.74.2024.05.28.05.36.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 05:36:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbd1ccb1-1cee-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716899768; x=1717504568; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bq2xCfVw6p5MkufH5QCKHLXDWhQgnTKEifRaEzGjd+U=;
        b=WEJuIEfPIQyGAfCG/SKPEzV2wBe1TWw3iYyiLPFaMxgDve86Hb4KCFv5WvtzTri2RC
         CkYBDSkaQrqnraR1xw1UhJLjKtzJpMqpngn3F4fn1Nlvimeb8sAzGIwaVdUr2oEv7x4U
         S6CsX+CivXhtij9jnWkYZ+yBvnvDJc+ME1wLQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716899768; x=1717504568;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bq2xCfVw6p5MkufH5QCKHLXDWhQgnTKEifRaEzGjd+U=;
        b=aqqTrZSIxQmO7YoodWXLA3CTY0XjLhf05l1cLWSdWttMR9qAxa3SrJ2C0MyPLAQN1N
         /OgvaMrVMB3JCcCA30nDab+wfqFV2aGWKVevnqVnwr5cTdlfKU+o6Tdjt5+ZYrrF6kOB
         JAh4+OrV0oWCOT3Qk/oHZ0apyZ08+5SRLpYKv6VN9dHlsCciuLyxznF60/5wAB4RIlQR
         t6k0ErXvnekrmTXe7MZMd+ZNuu08FJuf/QEidlH68lqNNoytT/rVb0ME/Kzk+NDrl8O2
         Fc5IJxalh74uxRB0b+y8D/HymXeIM0CBSYvQRtvYm3WxzzU3sjjFKcEm6yiHTZxOGtS3
         wFSg==
X-Gm-Message-State: AOJu0YyvC3a6Evug3eb22dCELgvdAAyN74kQM1uJkBDXV+bt7l1q7OdE
	IBDNa+ukE0oq6eKAZdMnFqAOHSxdAFIzDmeHaGW0ZpHBCX5nj5MXiB9jGrEuOck=
X-Google-Smtp-Source: AGHT+IFrq3xdKGuEDVBdSdV7rVEmUbzAGB9Pipyt0H/i1u8+jBVEC0GA1YGhQ+wI9RUrZOlmS3CVBA==
X-Received: by 2002:a50:c052:0:b0:578:649c:f7f0 with SMTP id 4fb4d7f45d1cf-578649cf802mr9090049a12.0.1716899768273;
        Tue, 28 May 2024 05:36:08 -0700 (PDT)
Message-ID: <97f7caeb-85e8-488b-b763-9860720f02cb@cloud.com>
Date: Tue, 28 May 2024 13:35:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/8] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <b2d4109cd30c82e0af153d36f8dce77c59f03695.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlA_6Abtw-u4a84J@macbook> <8cac1707-855b-478a-b88d-7fd619f19352@cloud.com>
 <ZlRJtN7h32uOHEOi@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZlRJtN7h32uOHEOi@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/05/2024 09:52, Roger Pau Monné wrote:
>> My intent here was to prevent the compiler omitting the write (though in
>> practice it didn't). I think the barrier is still required to prevent
>> reordering according to the spec.
> 
> Yes, my understating is that you added the ACCESS_ONCE() to possibly
> prevent the compiler from re-ordering the CPU_TO_X2APICID[ap_cpuid]) =
> read_apic_id() after the 'hlt' loop?

Yes, but not only that. Also preventing the compiler from eliding the
write altogether on the basis that it's not read after within the
function. I have suffered that particular behaviour on older versions of
GCC and it stung very much.

> 
> AFAICT the expressions in the `for` statement are considered sequence
> points, and the calling `read_apic_id()` could have side-effects, so
> the compiler won't be able to elide or move the setting of
> CPU_TO_X2APICID[] due to all side-effects of previous evaluations must
> be complete at sequence points.
> > I'm fine with leaving the wmb() + ACCESS_ONCE().
> 
> Thanks, Roger.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Tue May 28 12:56:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 12:56:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731130.1136468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwNT-0003BK-HL; Tue, 28 May 2024 12:56:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731130.1136468; Tue, 28 May 2024 12:56:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwNT-0003BD-DO; Tue, 28 May 2024 12:56:31 +0000
Received: by outflank-mailman (input) for mailman id 731130;
 Tue, 28 May 2024 12:56:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YQ9m=M7=arm.com=luca.fancellu@srs-se1.protection.inumbo.net>)
 id 1sBwNS-0003B7-LT
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 12:56:30 +0000
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTP
 id b1ece388-1cf1-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 14:56:27 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D6F56339;
 Tue, 28 May 2024 05:56:50 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.43])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9991E3F641;
 Tue, 28 May 2024 05:56:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1ece388-1cf1-11ef-b4bb-af5377834399
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Penny Zheng <Penny.Zheng@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Penny Zheng <penny.zheng@arm.com>
Subject: [PATCH v4.2] xen/p2m: put reference for level 2 superpage
Date: Tue, 28 May 2024 13:56:03 +0100
Message-Id: <20240528125603.2467640-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Penny Zheng <Penny.Zheng@arm.com>

We are doing foreign memory mapping for static shared memory, and
there is a great possibility that it could be super mapped.
But today, p2m_put_l3_page could not handle superpages.

This commits implements a new function p2m_put_l2_superpage to handle
level 2 superpages, specifically for helping put extra references for
foreign superpages.

Modify relinquish_p2m_mapping as well to take into account preemption
when we have a level-2 foreign mapping.

Currently level 1 superpages are not handled because Xen is not
preemptible and therefore some work is needed to handle such superpages,
for which at some point Xen might end up freeing memory and therefore
for such a big mapping it could end up in a very long operation.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v4.2 changes:
 - rework commit message to don't explicit say the size of the mapping
   but only say the level, remove unnecessary p2m_is_superpage() check
   in p2m_put_page, remove (level >= 2) condition to call p2m_put_page
   inside p2m_free_entry because the code in that function can already
   handle the levels, move TODO comment inside p2m_put_page, avoid
   mentioning 2MB in the comments. (Julien)
 - This patch is meant to superseed the homonymous patch in the serie:
   https://patchwork.kernel.org/project/xen-devel/cover/20240524124055.3871399-1-luca.fancellu@arm.com/
v4 changes:
 - optimised the path to call put_page() on the foreign mapping as
   Julien suggested. Add a comment in p2m_put_l2_superpage to state
   that any changes needs to take into account some change in the
   relinquish code. (Julien)
v3 changes:
 - Add reasoning why we don't support now 1GB superpage, remove level_order
   variable from p2m_put_l2_superpage, update TODO comment inside
   p2m_free_entry, use XEN_PT_LEVEL_ORDER(2) instead of value 9 inside
   relinquish_p2m_mapping. (Michal)
v2:
 - Do not handle 1GB super page as there might be some issue where
   a lot of calls to put_page(...) might be issued which could lead
   to free memory that is a long operation.
v1:
 - patch from https://patchwork.kernel.org/project/xen-devel/patch/20231206090623.1932275-9-Penny.Zheng@arm.com/
---
 xen/arch/arm/mmu/p2m.c | 81 +++++++++++++++++++++++++++++++-----------
 1 file changed, 61 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 41fcca011cf4..1725cca649b5 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -753,34 +753,72 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
     return rc;
 }
 
-/*
- * Put any references on the single 4K page referenced by pte.
- * TODO: Handle superpages, for now we only take special references for leaf
- * pages (specifically foreign ones, which can't be super mapped today).
- */
-static void p2m_put_l3_page(const lpae_t pte)
+static void p2m_put_foreign_page(struct page_info *pg)
 {
-    mfn_t mfn = lpae_get_mfn(pte);
-
-    ASSERT(p2m_is_valid(pte));
-
     /*
-     * TODO: Handle other p2m types
-     *
      * It's safe to do the put_page here because page_alloc will
      * flush the TLBs if the page is reallocated before the end of
      * this loop.
      */
-    if ( p2m_is_foreign(pte.p2m.type) )
+    put_page(pg);
+}
+
+/* Put any references on the single 4K page referenced by mfn. */
+static void p2m_put_l3_page(mfn_t mfn, p2m_type_t type)
+{
+    /* TODO: Handle other p2m types */
+    if ( p2m_is_foreign(type) )
     {
         ASSERT(mfn_valid(mfn));
-        put_page(mfn_to_page(mfn));
+        p2m_put_foreign_page(mfn_to_page(mfn));
     }
     /* Detect the xenheap page and mark the stored GFN as invalid. */
-    else if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
+    else if ( p2m_is_ram(type) && is_xen_heap_mfn(mfn) )
         page_set_xenheap_gfn(mfn_to_page(mfn), INVALID_GFN);
 }
 
+/* Put any references on the superpage referenced by mfn. */
+static void p2m_put_l2_superpage(mfn_t mfn, p2m_type_t type)
+{
+    struct page_info *pg;
+    unsigned int i;
+
+    /*
+     * TODO: Handle other p2m types, but be aware that any changes to handle
+     * different types should require an update on the relinquish code to handle
+     * preemption.
+     */
+    if ( !p2m_is_foreign(type) )
+        return;
+
+    ASSERT(mfn_valid(mfn));
+
+    pg = mfn_to_page(mfn);
+
+    for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++, pg++ )
+        p2m_put_foreign_page(pg);
+}
+
+/* Put any references on the page referenced by pte. */
+static void p2m_put_page(const lpae_t pte, unsigned int level)
+{
+    mfn_t mfn = lpae_get_mfn(pte);
+
+    ASSERT(p2m_is_valid(pte));
+
+    /*
+     * TODO: Currently we don't handle level 1 super-page, Xen is not
+     * preemptible and therefore some work is needed to handle such
+     * superpages, for which at some point Xen might end up freeing memory
+     * and therefore for such a big mapping it could end up in a very long
+     * operation.
+     */
+    if ( level == 2 )
+        return p2m_put_l2_superpage(mfn, pte.p2m.type);
+    else if ( level == 3 )
+        return p2m_put_l3_page(mfn, pte.p2m.type);
+}
+
 /* Free lpae sub-tree behind an entry */
 static void p2m_free_entry(struct p2m_domain *p2m,
                            lpae_t entry, unsigned int level)
@@ -809,9 +847,9 @@ static void p2m_free_entry(struct p2m_domain *p2m,
 #endif
 
         p2m->stats.mappings[level]--;
-        /* Nothing to do if the entry is a super-page. */
-        if ( level == 3 )
-            p2m_put_l3_page(entry);
+
+        p2m_put_page(entry, level);
+
         return;
     }
 
@@ -1558,9 +1596,12 @@ int relinquish_p2m_mapping(struct domain *d)
 
         count++;
         /*
-         * Arbitrarily preempt every 512 iterations.
+         * Arbitrarily preempt every 512 iterations or when we have a level-2
+         * foreign mapping.
          */
-        if ( !(count % 512) && hypercall_preempt_check() )
+        if ( (!(count % 512) ||
+              (p2m_is_foreign(t) && (order > XEN_PT_LEVEL_ORDER(2)))) &&
+             hypercall_preempt_check() )
         {
             rc = -ERESTART;
             break;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue May 28 12:56:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 12:56:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731132.1136478 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwNm-0003aw-Nd; Tue, 28 May 2024 12:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731132.1136478; Tue, 28 May 2024 12:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwNm-0003ap-Kn; Tue, 28 May 2024 12:56:50 +0000
Received: by outflank-mailman (input) for mailman id 731132;
 Tue, 28 May 2024 12:56:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+UYp=M7=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sBwNl-0003ZG-64
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 12:56:49 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id be5889de-1cf1-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 14:56:47 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a59cc765c29so27172466b.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 05:56:48 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8b980sm608662466b.154.2024.05.28.05.56.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 05:56:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be5889de-1cf1-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716901007; x=1717505807; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KKO5pF0AuV1bNrDCOnTfXew1j1QeUsaF7EgHEwrkyxQ=;
        b=JbOb4Yn7DzD4LqHb5z95DtSN/5ltdKSihlwLmtpyPeBTc/0oKzJIRSlnX8i/fR1Lxw
         OXl40UV2IO0efdrScYkycjXgf4PGKdAHwnwnw7SzgM3P7bpGx5S2dD1Pyb4Yjb0nwyb4
         xecdWJ33VP3XrcymiyLaAoInvRjomqEFqlECqW0/c+e/bpkMHs28n3RlXqLIDraTGxOp
         8iugLVP8tzPPTe1nAZ0ZCBdQMDpCXCqn5yot1gcARtf7VxK5yHiGEu7jM0GV/avS7rGw
         TYplCuCttSKgV4mcMr2gv1agWjNpetELm/3/k/cr2muqjvuw2xh8kJNyamGMSqcKOjVu
         vpQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716901007; x=1717505807;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KKO5pF0AuV1bNrDCOnTfXew1j1QeUsaF7EgHEwrkyxQ=;
        b=WCae1GuCSfE8dxSLHxa0+h+DwvaF4j2B5Um6QsIzfcfv1WGmBbLQPx/OFEgmGYHeK1
         E+DcJD+iE2eWFGDXkzzlPuIqwMHP1PdKPUf0euTLCqNTLwxSI6nBD3RYtU6gTU5JLE76
         wETxmRXr2BtX5SZdHHV4cyKDfYnAsZPBV+1S8gqwdxul/ETczKj3TC2naN4108Qtb3yU
         kfvTWbUk1MZsVpFNxjwpv+0UeDxnE0kxTv3iBhoSRwhRoxcEze+e3nab9FwG6+kAUtUU
         rxEaCQSt26xz206UO/n38PE58ZJM1MtUL8ZyAS2Xm8OvsY3VL/xqSH824HFTzw8jF80X
         I+lA==
X-Forwarded-Encrypted: i=1; AJvYcCXGAQqrPGhE3/B9elxCRFhB3j+35rHPb0s2pZBkNxLCOHkxzh8pI5bwLUkRCR6r6bBHB4anpw9BU9nzkQ4DCYMd6YU1PzfE6EvHNXnxvOg=
X-Gm-Message-State: AOJu0YzpSVAXtfq2a3dXiyX48UiAhcbgHsS4XAWOvREir2+/2DuurBZ5
	rU3ls48zVE/GlC27NBp/seKmHzR8DHl4IswQYneC/A+oFO2D4StrGri9LRlrNw8=
X-Google-Smtp-Source: AGHT+IHOiGiSxeg0Z6jHmUu8nh2M1BlKRDGSc7iXkeKPNevCo2TqBxYSAtMJcUWemYo2FMUZfXtVIQ==
X-Received: by 2002:a17:906:6d88:b0:a5a:6367:7186 with SMTP id a640c23a62f3a-a626536e538mr707619866b.70.1716901007554;
        Tue, 28 May 2024 05:56:47 -0700 (PDT)
Message-ID: <293200d3-5fe8-44da-a0c3-95e6f9899670@suse.com>
Date: Tue, 28 May 2024 14:56:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] x86/pci/xen: Fix PCIBIOS_* return code handling
To: =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
 Bjorn Helgaas <bhelgaas@google.com>, Thomas Gleixner <tglx@linutronix.de>,
 Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
 Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
 "H. Peter Anvin" <hpa@zytor.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ian Campbell <ian.campbell@citrix.com>, xen-devel@lists.xenproject.org,
 linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: stable@vger.kernel.org
References: <20240527125538.13620-1-ilpo.jarvinen@linux.intel.com>
 <20240527125538.13620-3-ilpo.jarvinen@linux.intel.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240527125538.13620-3-ilpo.jarvinen@linux.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 27.05.24 14:55, Ilpo Järvinen wrote:
> xen_pcifront_enable_irq() uses pci_read_config_byte() that returns
> PCIBIOS_* codes. The error handling, however, assumes the codes are
> normal errnos because it checks for < 0.
> 
> xen_pcifront_enable_irq() also returns the PCIBIOS_* code back to the
> caller but the function is used as the (*pcibios_enable_irq) function
> which should return normal errnos.
> 
> Convert the error check to plain non-zero check which works for
> PCIBIOS_* return codes and convert the PCIBIOS_* return code using
> pcibios_err_to_errno() into normal errno before returning it.
> 
> Fixes: 3f2a230caf21 ("xen: handled remapped IRQs when enabling a pcifront PCI device.")
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Cc: stable@vger.kernel.org

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen



From xen-devel-bounces@lists.xenproject.org Tue May 28 13:12:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 13:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731142.1136487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwcm-0006aG-0l; Tue, 28 May 2024 13:12:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731142.1136487; Tue, 28 May 2024 13:12:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwcl-0006a9-Tq; Tue, 28 May 2024 13:12:19 +0000
Received: by outflank-mailman (input) for mailman id 731142;
 Tue, 28 May 2024 13:12:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBwck-0006a3-Um
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 13:12:18 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e842f300-1cf3-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 15:12:17 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a6303e13ffeso96305966b.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 06:12:17 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c81771fsm611413966b.27.2024.05.28.06.12.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 06:12:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e842f300-1cf3-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716901937; x=1717506737; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=NOQgVVjwD2+/MccmulpZwJ6lvktvbBRRFLuQltr8H6E=;
        b=TyHuIVaNpUNyo4/if+USbGvd8EZuhTApff1HifVE6Z8vSGseSMhRKMCWuATaRLZ7Wk
         +IpC9UCOLMYkbOyZPh7LS2751ee3JrN8dCqpHzME4HZkXIcg+VB/f9iXEuNsyp+6NbEz
         DIBog5SU5P53bEyO62JCcuHDzMmtUr9djts2Va1hApUlDQhkBLtPhDUf7/kugEdujOlk
         Bpe8+1lZfFWsto56MuHFFfJRzY3aDnv8tuWwRJteamPMExjfvI9QmIlshUMSkHpQN6on
         g+5Tqaoa2X3R1cWWvoZUb9pzvma/ova0C+SybkenamKiDKQCo57q9dDQGYVp56dYvZ1A
         Lt/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716901937; x=1717506737;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NOQgVVjwD2+/MccmulpZwJ6lvktvbBRRFLuQltr8H6E=;
        b=sn70nTdu5id++AEjpsSPBRncZyhD9YZ21vMl1OUY/DS0XrF5To5AJEoJa/hzGY05zt
         zqItW7Hm50faLwxnkGxRg1/qm0k4S+ShW1Wqru3nopqM98X4kQkiMclnOai4xugzpLrJ
         GAGGuOkyZsAWhCI3N4ge7MmeDoPKLfdBjYN/90NBo2TYBDa+tcJBQq4IWFx2+WYJAtIM
         IteAh6yTIwiEdenZjADd2q0FXFuzmos6mXS3J8uoXyKveH/1c5nF9abI81BwAdzaSXIE
         lnKBvwbUnx4f+FRzqjMBIILGewXYSg2vMsQNPeeLkadD3lUiB563/wNQMruUd9Qv0gUd
         ZXTQ==
X-Forwarded-Encrypted: i=1; AJvYcCWxiCmV1b2LPzGLBzm9sJTEXbPyte4MdDvITS5i5DNwvcK2q1fT3qHoapasmEThhTSOcLLDVma/qRQYuA1dngHriDzuumDVdbgcxdcBE7I=
X-Gm-Message-State: AOJu0Yz3qHlxF5bmHHz85ZQ2d7tNj4EOLeg8a7nfgAqGz4fVQzVnKugY
	RdO1N8JnSU6Cy34+LBs6EAUn0+7NEBzJvbbtrVX6+M450CJgboR8+IP8Iz8nog==
X-Google-Smtp-Source: AGHT+IGmVOZOQQ073oqP2Bir1Scy4J9sq3uOmV8Y1rrYfOvotmzOIbYXQHOeiM1b1yjYrlyP2mTeZw==
X-Received: by 2002:a17:907:1391:b0:a59:bd5b:a0f3 with SMTP id a640c23a62f3a-a62646cd962mr620247566b.38.1716901936473;
        Tue, 28 May 2024 06:12:16 -0700 (PDT)
Message-ID: <df7bb467-c778-43fb-bd04-f81f6e3dfd01@suse.com>
Date: Tue, 28 May 2024 15:12:15 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/13] x86/bitops: Improve arch_ffs() in the general
 case
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-8-andrew.cooper3@citrix.com>
 <1660a2a7-cea4-4e6f-9286-0c134c34b6fb@suse.com>
 <57a47c76-c484-4309-8a87-a51f79dd48b6@suse.com>
 <b0838a62-1e6a-473a-a757-97091c84e164@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b0838a62-1e6a-473a-a757-97091c84e164@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.05.2024 14:30, Andrew Cooper wrote:
> On 27/05/2024 2:37 pm, Jan Beulich wrote:
>> On 27.05.2024 15:27, Jan Beulich wrote:
>>> On 24.05.2024 22:03, Andrew Cooper wrote:
>>>> --- a/xen/arch/x86/include/asm/bitops.h
>>>> +++ b/xen/arch/x86/include/asm/bitops.h
>>>> @@ -432,12 +432,28 @@ static inline int ffsl(unsigned long x)
>>>>  
>>>>  static always_inline unsigned int arch_ffs(unsigned int x)
>>>>  {
>>>> -    int r;
>>>> +    unsigned int r;
>>>> +
>>>> +    if ( __builtin_constant_p(x > 0) && x > 0 )
>>>> +    {
>>>> +        /* Safe, when the compiler knows that x is nonzero. */
>>>> +        asm ( "bsf %[val], %[res]"
>>>> +              : [res] "=r" (r)
>>>> +              : [val] "rm" (x) );
>>>> +    }
>>> In patch 11 relevant things are all in a single patch, making it easier
>>> to spot that this is dead code: The sole caller already has a
>>> __builtin_constant_p(), hence I don't see how the one here could ever
>>> return true. With that the respective part of the description is then
>>> questionable, too, I'm afraid: Where did you observe any actual effect
>>> from this? Or if you did - what am I missing?
>> Hmm, thinking about it: I suppose that's why you have
>> __builtin_constant_p(x > 0), not __builtin_constant_p(x). I have to admit
>> I'm (positively) surprised that the former may return true when the latter
>> doesn't.
> 
> So was I, but this recommendation came straight from the GCC mailing
> list.  And it really does work, even back in obsolete versions of GCC.
> 
> __builtin_constant_p() operates on an expression not a value, and is
> documented as such.

Of course.

>>  Nevertheless I'm inclined to think this deserves a brief comment.
> 
> There is a comment, and it's even visible in the snippet.

The comment is about the asm(); it is neither placed to clearly relate
to __builtin_constant_p(), nor is it saying anything about this specific
property of it. You said you were equally surprised; don't you think
that when both of us are surprised, a specific (even if brief) comment
is warranted?

>> As an aside, to better match the comment inside the if()'s body, how about
>>
>>     if ( __builtin_constant_p(!!x) && x )
>>
>> ? That also may make a little more clear that this isn't just a style
>> choice, but actually needed for the intended purpose.
> 
> I am not changing the logic.
> 
> Apart from anything else, your suggestion is trivially buggy.  I care
> about whether the RHS collapses to a constant, and the only way of doing
> that correctly is asking the compiler about the *exact* expression. 
> Asking about some other expression which you hope - but do not know -
> that the compiler will treat equivalently is bogus.  It would be
> strictly better to only take the else clause, than to have both halves
> emitted.
> 
> This is the form I've tested extensively.  It's also the clearest form
> IMO.  You can experiment with alternative forms when we're not staring
> down code freeze of 4.19.

"Clearest form" is almost always a matter of taste. To me, comparing
unsigned values with > or < against 0 is generally at least suspicious.
Using != is typically better (again: imo), and simply omitting the != 0
then is shorter with no difference in effect. Except in peculiar cases
like this one, where indeed it took me some time to figure why the
comparison operator may not be omitted.

All that said: I'm not going to insist on any change; the R-b previously
offered still stands. I would highly appreciate though if the (further)
comment asked for could be added.

What I definitely dislike here is you - not for the first time - turning
down remarks because a change of yours is late. This feels even more so
bad when considering that I'm typically replying to your patches with
pretty little turnaround. Whereas various of mine, pending in part for
years, do not seem to deserve any review comments at all. Unlike before,
where it was "only" improvements or feature additions, meanwhile even
bug fixes are left sit like that. If I may be blunt: This may not work
this way for much longer. At some point I will need to artificially
delay reviews, making them dependent on my own work also being allowed
to make progress. I question though whether that would be in everyone's
interest.

Jan


From xen-devel-bounces@lists.xenproject.org Tue May 28 13:20:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 13:20:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731147.1136497 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwl3-0008Gr-Rk; Tue, 28 May 2024 13:20:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731147.1136497; Tue, 28 May 2024 13:20:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwl3-0008Gk-PA; Tue, 28 May 2024 13:20:53 +0000
Received: by outflank-mailman (input) for mailman id 731147;
 Tue, 28 May 2024 13:20:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBwl2-0008Ge-A3
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 13:20:52 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 199163dc-1cf5-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 15:20:49 +0200 (CEST)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-6ab9e0a4135so29382586d6.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 06:20:49 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad7a4da6f7sm23989776d6.98.2024.05.28.06.20.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 06:20:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 199163dc-1cf5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716902449; x=1717507249; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=jt55f7uGGjZV9XnmbRPk/ICKgW7KTxcKEzUfXVckBEM=;
        b=PRnmGu8gIuOyB1cRW7nmzWZXMxUGNysuhde43TYRN3z1xJ4hyffYPpiF8IJlTODS+J
         X5BCKZKGMUeFlEWiUIu4uz9DY/6G+yfIv7TjTXCZzLLnF6lFWtw/1QbJMuT54GxW+yyO
         gsIh/EvttmSS9AkE8+4g2Rp+s0vgN8Tmz1mYE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716902449; x=1717507249;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jt55f7uGGjZV9XnmbRPk/ICKgW7KTxcKEzUfXVckBEM=;
        b=UpHpqqcokkR0Gny+g6vH52dcG2TaBXVt1ws/1wJ7IupxvUYdKaF2jC+d+Dk9O0Z4wS
         QK/hP7MWsC1VNgkgHdS+Gjz9h/G6FYaBOk4GgOdlFRJuMJLuqQl8C/vTHd/bWDngodDA
         qXnvibjnRswBjVnd2WYYu+3f3an5RYTdRuxXbMe1mEmxIybhrBW1ixW8UIenw/rx1dY1
         O8CJP7DVLrRh88Fy4KrXCgoEX7cLLtitnmcbLLbdItcgGOt9fwtwdqt6YKZ+AIBipbjp
         WAeB2jL2EZFF5jU/7pKl9hbbJxKCCH59NW6/YnVblF7iftlN9rg5kbnLsmvRXm8C/+Iq
         /7sQ==
X-Forwarded-Encrypted: i=1; AJvYcCXUGaCDIPIXiajBWXxY9lXdJSJVDcIPAWnRJuwwe9R2KOoYLYpa0i+b1Lu/cnIJKkzDh7Rd7z7JIazuR6QiyZF1vI5AnjKighSns0m8wKQ=
X-Gm-Message-State: AOJu0YyzTdHoXwDqmjBI96gCPMtctCqSCvE/EqlDtcdUxOO8kSpd/IsF
	aZTiSYKIzq11tk2kr+tkWIerSWKeosIjFRRHQXfiYgblFsadabhLMAjblNTh1d8=
X-Google-Smtp-Source: AGHT+IFbfbwa3NsL2kJPdLZZc/15FNGV9gn6h8K/YYqupjGHVJ6O8fl33Ho3aumRepddoEWQRScXdA==
X-Received: by 2002:a05:6214:45a0:b0:6a8:dcc7:9088 with SMTP id 6a1803df08f44-6ab9d06e5c9mr256805756d6.26.1716902448813;
        Tue, 28 May 2024 06:20:48 -0700 (PDT)
Message-ID: <07f14337-a646-4041-be49-1f6e5ccab32e@citrix.com>
Date: Tue, 28 May 2024 14:20:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/13] xen/bitops: Implement generic_f?sl() in lib/
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-6-andrew.cooper3@citrix.com>
 <7305a8b9-5f43-47e4-987b-c5ca76bcc312@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7305a8b9-5f43-47e4-987b-c5ca76bcc312@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/05/2024 9:44 am, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
>> generic_f?s() being static inline is the cause of lots of the complexity
>> between the common and arch-specific bitops.h
>>
>> They appear to be static inline for constant-folding reasons (ARM uses them
>> for this), but there are better ways to achieve the same effect.
>>
>> It is presumptuous that an unrolled binary search is the right algorithm to
>> use on all microarchitectures.  Indeed, it's not for the eventual users, but
>> that can be addressed at a later point.
>>
>> It is also nonsense to implement the int form as the base primitive and
>> construct the long form from 2x int in 64-bit builds, when it's just one extra
>> step to operate at the native register width.
>>
>> Therefore, implement generic_f?sl() in lib/.  They're not actually needed in
>> x86/ARM/PPC by the end of the cleanup (i.e. the functions will be dropped by
>> the linker), and they're only expected be needed by RISC-V on hardware which
>> lacks the Zbb extension.
>>
>> Implement generic_fls() in terms of generic_flsl() for now, but this will be
>> cleaned up in due course.
>>
>> Provide basic runtime testing using __constructor inside the lib/ file.  This
>> is important, as it means testing runs if and only if generic_f?sl() are used
>> elsewhere in Xen.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with a suggestion and a question.
>
>> I suspect we want to swap CONFIG_DEBUG for CONFIG_BOOT_UNIT_TESTS in due
>> course.  These ought to be able to be used in a release build too.
> +1

Actually - I might as well do this now.  Start as we mean to go on.

>
>> --- /dev/null
>> +++ b/xen/lib/generic-ffsl.c
>> @@ -0,0 +1,65 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +
>> +#include <xen/bitops.h>
>> +#include <xen/boot-check.h>
>> +#include <xen/init.h>
>> +
>> +unsigned int generic_ffsl(unsigned long x)
>> +{
>> +    unsigned int r = 1;
>> +
>> +    if ( !x )
>> +        return 0;
>> +
>> +#if BITS_PER_LONG > 32
> To be future-proof, perhaps ahead of this
>
> #if BITS_PER_LONG > 64
> # error "..."
> #endif
>
> or a functionally similar BUILD_BUG_ON()?

Good point.  I'll fold this in to both files.

>
>> --- /dev/null
>> +++ b/xen/lib/generic-flsl.c
>> @@ -0,0 +1,68 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +
>> +#include <xen/bitops.h>
>> +#include <xen/boot-check.h>
>> +#include <xen/init.h>
>> +
>> +/* Mask of type UL with the upper x bits set. */
>> +#define UPPER_MASK(x) (~0UL << (BITS_PER_LONG - (x)))
>> +
>> +unsigned int generic_flsl(unsigned long x)
>> +{
>> +    unsigned int r = BITS_PER_LONG;
>> +
>> +    if ( !x )
>> +        return 0;
>> +
>> +#if BITS_PER_LONG > 32
>> +    if ( !(x & UPPER_MASK(32)) )
>> +    {
>> +        x <<= 32;
>> +        r -= 32;
>> +    }
>> +#endif
>> +    if ( !(x & UPPER_MASK(16)) )
>> +    {
>> +        x <<= 16;
>> +        r -= 16;
>> +    }
>> +    if ( !(x & UPPER_MASK(8)) )
>> +    {
>> +        x <<= 8;
>> +        r -= 8;
>> +    }
>> +    if ( !(x & UPPER_MASK(4)) )
>> +    {
>> +        x <<= 4;
>> +        r -= 4;
>> +    }
>> +    if ( !(x & UPPER_MASK(2)) )
>> +    {
>> +        x <<= 2;
>> +        r -= 2;
>> +    }
>> +    if ( !(x & UPPER_MASK(1)) )
>> +    {
>> +        x <<= 1;
>> +        r -= 1;
>> +    }
>> +
>> +    return r;
>> +}
> While, as you say, the expectation is for this code to not commonly come
> into actual use, I still find the algorithm a little inefficient in terms
> of the constants used, specifically considering how they would need
> instantiating in resulting assembly. It may be that Arm's fancy constant-
> move insns can actually efficiently synthesize them, but I think on most
> other architectures it would be more efficient (and presumably no less
> efficient on Arm) to shift the "remaining" value right, thus allowing for
> successively smaller (and hence easier to instantiate) constants to be
> used.

ARM can only synthesise UPPER_MASK(16) and narrower masks, I think.

That said, I'm not concerned about the (in)efficiency seeing as this
doesn't get included in x86/ARM/PPC builds by the end of the series.

It's RISC-V which matters, and I'm pretty sure this is the wrong
algorithm to be using.

Incidentally, this algorithm is terrible for superscalar pipelines,
because each branch is inherently unpredictable.

Both these files want rewriting based on an analysis of the H-capable
Zbb-incapable RISC-V cores which exist.

I expect that what we actually want is the De Bruijn form which is an
O(1) algorithm, given a decent hardware multiplier.  If not, there's a
loop form which I expect would still be better than this.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 13:28:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 13:28:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731154.1136508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwsb-0000bO-FR; Tue, 28 May 2024 13:28:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731154.1136508; Tue, 28 May 2024 13:28:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBwsb-0000bH-CY; Tue, 28 May 2024 13:28:41 +0000
Received: by outflank-mailman (input) for mailman id 731154;
 Tue, 28 May 2024 13:28:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+/BF=M7=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sBwsa-0000bB-O5
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 13:28:40 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 30f67980-1cf6-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 15:28:38 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a63036f2daaso106361866b.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 06:28:38 -0700 (PDT)
Received: from [10.80.5.21] ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a63570e5d0esm24566866b.214.2024.05.28.06.28.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 06:28:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30f67980-1cf6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716902918; x=1717507718; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:subject:from:user-agent:mime-version:date:message-id:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Stzh1/6q+z2/sSILrF67K/1rlkNSVEaKd0fowUH80q4=;
        b=eRb4xAiPilIflQnBP2UJhSgmkLGZkUhCaLCvOGOFTfYU6dkwTgWpbVbvSTm9lCKs82
         cL5zEDOA8VE+ErzM4bYfujJkfCOsVyUTv9qkuAWhxpO4bEeJTkwL1DyEqS19ZAMDuWXf
         96tDepBO5W7KDiwC8yGL/toQK3d/1wZHLs+u4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716902918; x=1717507718;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:subject:from:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Stzh1/6q+z2/sSILrF67K/1rlkNSVEaKd0fowUH80q4=;
        b=ID3cQlOrpXv2BZjynBy5I9YNnklOzw4yenLZlf9L2OPdccnHxJ5HEty64gSJfJbWg2
         WnK/QPabhl+WqE+3i9v5epM5/FW7o+7TBDse3gN/0W7ZCcwvniVcRgqr/XD4S9H5/ViF
         +GC53Mfg5BbZKOW7CG94Uekl42aOS6vxUdwcw13K4PLmZvYD+X0gA5CZE6FKlJXjjh4d
         8Ucp+yAUg3S/ROdHKJ4jF0g5MZT0GBh+a0dtiR4Q2ER6rN8JJbBOv3rACEJnqxH4ODk0
         E9dPylj/vxGnbsvRF2DR0bNhaLZTFnWCxEzBX6maTK9aaNW+Nl06f9TPzwkGjAaDlP0e
         Cfcw==
X-Gm-Message-State: AOJu0YxIdZS5csQX/pe264NrHs5hbcxRjJSlxCYH8uOjgWU8T+nRefxo
	9Cq97w7UD+FTb0ZuxH/VQ+9C5weVrJIJ3lhnRIhZl+Q+vK2jn+yQFBAe87C7p+w=
X-Google-Smtp-Source: AGHT+IFvrXlE421GnEB77R2Ak42rqJUMCknFT4TuGWbtszvpa78z3EQ6vS5ZmS8dhJJB7EwY6EpTcg==
X-Received: by 2002:a17:906:4903:b0:a62:ad6:4286 with SMTP id a640c23a62f3a-a62642d9799mr910331966b.7.1716902917446;
        Tue, 28 May 2024 06:28:37 -0700 (PDT)
Message-ID: <ec59063d-fb06-428b-9e00-41f5f7aaebf1@cloud.com>
Date: Tue, 28 May 2024 14:28:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Subject: Re: [PATCH v2 6/8] xen/lib: Add topology generator for x86
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <1ffad529d7fed10381df67215c747fc2d69f805e.1715102098.git.alejandro.vallejo@cloud.com>
 <Zk9z4HJmqXpbivRu@macbook>
Content-Language: en-GB
In-Reply-To: <Zk9z4HJmqXpbivRu@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/05/2024 17:50, Roger Pau Monné wrote:
> On Wed, May 08, 2024 at 01:39:25PM +0100, Alejandro Vallejo wrote:
>> Add a helper to populate topology leaves in the cpu policy from
>> threads/core and cores/package counts.
>>
>> No functional change, as it's not connected to anything yet.
> 
> There is a functional change in test-cpu-policy.c.
> 
> Maybe the commit message needs to be updated to reflect the added
> testing to test-cpu-policy.c using the newly introduced helper to
> generate topologies?
> 

Sure to this and all formatting feedback below.

>>
>> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
>> ---
>> v2:
>>   * New patch. Extracted from v1/patch6
>> ---
>>  tools/tests/cpu-policy/test-cpu-policy.c | 128 +++++++++++++++++++++++
>>  xen/include/xen/lib/x86/cpu-policy.h     |  16 +++
>>  xen/lib/x86/policy.c                     |  86 +++++++++++++++
>>  3 files changed, 230 insertions(+)
>>
>> diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
>> index 301df2c00285..0ba8c418b1b3 100644
>> --- a/tools/tests/cpu-policy/test-cpu-policy.c
>> +++ b/tools/tests/cpu-policy/test-cpu-policy.c
>> @@ -650,6 +650,132 @@ static void test_is_compatible_failure(void)
>>      }
>>  }
>>  
>> +static void test_topo_from_parts(void)
>> +{
>> +    static const struct test {
>> +        unsigned int threads_per_core;
>> +        unsigned int cores_per_pkg;
>> +        struct cpu_policy policy;
>> +    } tests[] = {
>> +        {
>> +            .threads_per_core = 3, .cores_per_pkg = 1,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_AMD,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
>> +                    [1] = { .nr_logical = 1, .level = 1, .type = 2, .id_shift = 2, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 1, .cores_per_pkg = 3,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_AMD,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
>> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 7, .cores_per_pkg = 5,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_AMD,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
>> +                    [1] = { .nr_logical = 5, .level = 1, .type = 2, .id_shift = 6, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 2, .cores_per_pkg = 128,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_AMD,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
>> +                    [1] = { .nr_logical = 128, .level = 1, .type = 2, .id_shift = 8, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 3, .cores_per_pkg = 1,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_INTEL,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
>> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 1, .cores_per_pkg = 3,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_INTEL,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
>> +                    [1] = { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 7, .cores_per_pkg = 5,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_INTEL,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
>> +                    [1] = { .nr_logical = 35, .level = 1, .type = 2, .id_shift = 6, },
>> +                },
>> +            },
>> +        },
>> +        {
>> +            .threads_per_core = 2, .cores_per_pkg = 128,
>> +            .policy = {
>> +                .x86_vendor = X86_VENDOR_INTEL,
>> +                .topo.subleaf = {
>> +                    [0] = { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
>> +                    [1] = { .nr_logical = 256, .level = 1, .type = 2, .id_shift = 8, },
> 
> You don't need the array index in the initialization:
> 
>                 .topo.subleaf = {
>                     { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
>                     { .nr_logical = 256, .level = 1, .type = 2,
>                       .id_shift = 8, },
>                 }
> 
> And lines should be limited to 80 columns if possible.
> 
>> +                },
>> +            },
>> +        },
>> +    };
>> +
>> +    printf("Testing topology synthesis from parts:\n");
>> +
>> +    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
>> +    {
>> +        const struct test *t = &tests[i];
>> +        struct cpu_policy actual = { .x86_vendor = t->policy.x86_vendor };
>> +        int rc = x86_topo_from_parts(&actual, t->threads_per_core, t->cores_per_pkg);
>> +
>> +        if ( rc || memcmp(&actual.topo, &t->policy.topo, sizeof(actual.topo)) )
>> +        {
>> +#define TOPO(n) topo.subleaf[(n)]
>> +            fail("FAIL[%d] - '%s %u t/c, %u c/p'\n",
>> +                 rc,
>> +                 x86_cpuid_vendor_to_str(t->policy.x86_vendor),
>> +                 t->threads_per_core, t->cores_per_pkg);
>> +            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
>> +                   "             expected_lvl=%u actual_lvl=%u\n"
>> +                   "             expected_type=%u actual_type=%u\n"
>> +                   "             expected_shift=%u actual_shift=%u\n",
>> +                   0, t->policy.TOPO(0).nr_logical, actual.TOPO(0).nr_logical,
>> +                      t->policy.TOPO(0).level,      actual.TOPO(0).level,
>> +                      t->policy.TOPO(0).type,       actual.TOPO(0).type,
>> +                      t->policy.TOPO(0).id_shift,   actual.TOPO(0).id_shift);
>> +
>> +            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
>> +                   "             expected_lvl=%u actual_lvl=%u\n"
>> +                   "             expected_type=%u actual_type=%u\n"
>> +                   "             expected_shift=%u actual_shift=%u\n",
>> +                   1, t->policy.TOPO(1).nr_logical, actual.TOPO(1).nr_logical,
>> +                      t->policy.TOPO(1).level,      actual.TOPO(1).level,
>> +                      t->policy.TOPO(1).type,       actual.TOPO(1).type,
>> +                      t->policy.TOPO(1).id_shift,   actual.TOPO(1).id_shift);
>> +#undef TOPO
> 
> Seeing the usage of the macro, maybe you could even do something like:
> 
> TOPO(n, f)  t->policy.topo.subleaf[(n)].f, actual.topo.subleaf[(n)].f
> 
> This will limit a bit the repetition of the "t->policy..., actual..."
> tuple.
> 

Hm. Sure, but bear in mind it ends up looking rather cryptic.

>> +        }
>> +    }
>> +}
>> +
>>  int main(int argc, char **argv)
>>  {
>>      printf("CPU Policy unit tests\n");
>> @@ -667,6 +793,8 @@ int main(int argc, char **argv)
>>      test_is_compatible_success();
>>      test_is_compatible_failure();
>>  
>> +    test_topo_from_parts();
>> +
>>      if ( nr_failures )
>>          printf("Done: %u failures\n", nr_failures);
>>      else
>> diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
>> index 392320b9adbe..f5df18e9f77c 100644
>> --- a/xen/include/xen/lib/x86/cpu-policy.h
>> +++ b/xen/include/xen/lib/x86/cpu-policy.h
>> @@ -551,6 +551,22 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
>>   */
>>  uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
>>  
>> +/**
>> + * Synthesise topology information in `p` given high-level constraints
>> + *
>> + * Topology is given in various fields accross several leaves, some of
>> + * which are vendor-specific. This function uses the policy itself to
>> + * derive such leaves from threads/core and cores/package.
>> + *
>> + * @param p                   CPU policy of the domain.
>> + * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
>> + * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
>> + * @return                    0 on success; -errno on failure
>> + */
>> +int x86_topo_from_parts(struct cpu_policy *p,
>> +                        unsigned int threads_per_core,
>> +                        unsigned int cores_per_pkg);
>> +
>>  #endif /* !XEN_LIB_X86_POLICIES_H */
>>  
>>  /*
>> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
>> index 4cef658feeb8..d033ee5398dd 100644
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -13,6 +13,92 @@ uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>>      return vcpu_id * 2;
>>  }
>>  
>> +static unsigned int order(unsigned int n)
>> +{
>> +    return 8 * sizeof(n) - __builtin_clz(n);
> 
> Do we need to assert that n is not 0, otherwise the return of
> __builtin_clz() is undefined.
> 
> I think the usage below doesn't pass 0 to __builtin_clz() in any case,
> but better add the check IMO.

I doesn't, but asserting sanity sounds good.

> 
> Is __builtin_clz() also available in all versions of GCC and CLANG
> that we support?  I have no idea when this was introduced.
> 

Works on GCC 4.1.2 and Clang 3.5 according to godbolt.

>> +}
>> +
>> +int x86_topo_from_parts(struct cpu_policy *p,
>> +                        unsigned int threads_per_core,
>> +                        unsigned int cores_per_pkg)
>> +{
>> +    unsigned int threads_per_pkg = threads_per_core * cores_per_pkg;
>> +    unsigned int apic_id_size;
>> +
>> +    if ( !p || !threads_per_core || !cores_per_pkg )
>> +        return -EINVAL;
>> +
>> +    p->basic.max_leaf = MAX(0xb, p->basic.max_leaf);
>> +
>> +    memset(p->topo.raw, 0, sizeof(p->topo.raw));
>> +
>> +    /* thread level */
>> +    p->topo.subleaf[0].nr_logical = threads_per_core;
>> +    p->topo.subleaf[0].id_shift = 0;
>> +    p->topo.subleaf[0].level = 0;
>> +    p->topo.subleaf[0].type = 1;
>> +    if ( threads_per_core > 1 )
>> +        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
>> +
>> +    /* core level */
>> +    p->topo.subleaf[1].nr_logical = cores_per_pkg;
>> +    if ( p->x86_vendor == X86_VENDOR_INTEL )
>> +        p->topo.subleaf[1].nr_logical = threads_per_pkg;
>> +    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
>> +    p->topo.subleaf[1].level = 1;
>> +    p->topo.subleaf[1].type = 2;
>> +    if ( cores_per_pkg > 1 )
>> +        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
>> +
>> +    apic_id_size = p->topo.subleaf[1].id_shift;
>> +
>> +    /*
>> +     * Contrary to what the name might seem to imply. HTT is an enabler for
>> +     * SMP and there's no harm in setting it even with a single vCPU.
>> +     */
>> +    p->basic.htt = true;
>> +    p->basic.lppp = MIN(0xff, p->basic.lppp);
>> +
>> +    switch ( p->x86_vendor )
>> +    {
>> +        case X86_VENDOR_INTEL: {
>> +            struct cpuid_cache_leaf *sl = p->cache.subleaf;
> 
> Newline please.
> 
>> +            for ( size_t i = 0; sl->type &&
>> +                                i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
>> +            {
>> +                sl->cores_per_package = cores_per_pkg - 1;
>> +                sl->threads_per_cache = threads_per_core - 1;
>> +                if ( sl->type == 3 /* unified cache */ )
>> +                    sl->threads_per_cache = threads_per_pkg - 1;
>> +            }
>> +            break;
>> +        }
> 
> Newline here also.
> 
>> +        case X86_VENDOR_AMD:
>> +        case X86_VENDOR_HYGON:
>> +            /* Expose p->basic.lppp */
>> +            p->extd.cmp_legacy = true;
>> +
>> +            /* Clip NC to the maximum value it can hold */
>> +            p->extd.nc = 0xff;
>> +            if ( threads_per_pkg <= 0xff )
>> +                p->extd.nc = threads_per_pkg - 1;
>> +
>> +            /* TODO: Expose leaf e1E */
>> +            p->extd.topoext = false;
>> +
>> +            /*
>> +             * Clip APIC ID to 8 bits, as that's what high core-count machines do
> 
> Overly long line?  And missing full stop.

I've reduced the indentation of the case statement to align them to the
switch. The line fits afterwards.

> 
>> +             *
>> +             * That what AMD EPYC 9654 does with >256 CPUs
>                   ^ That's
> 
> Thanks, Roger.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Tue May 28 14:06:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 14:06:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731167.1136524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBxTO-0006Ly-7N; Tue, 28 May 2024 14:06:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731167.1136524; Tue, 28 May 2024 14:06:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBxTO-0006Lr-4j; Tue, 28 May 2024 14:06:42 +0000
Received: by outflank-mailman (input) for mailman id 731167;
 Tue, 28 May 2024 14:06:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+/BF=M7=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sBxTN-0006Ll-Ll
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 14:06:41 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8020c10a-1cfb-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 16:06:38 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-57864327f6eso1474494a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 07:06:38 -0700 (PDT)
Received: from [10.125.230.17] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578638009b1sm5719158a12.17.2024.05.28.07.06.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 07:06:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8020c10a-1cfb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716905198; x=1717509998; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Bhn2iLHzD8RvektTs6H45lIDox1xDUH0MHd7mOG3ujU=;
        b=D6wCgbxEQ/OaTRyNeyEAyzP0/UJYVaM9WkgcDmI0K4/gVN+d2eIQHr0I0Jlroq6GLo
         dfF3xraTxcHTZ2rZiYrpDJdokCBC+Q2IiUWsFEAJ7xl1GEXdxua6QcSl+l+8dxD4IZ+v
         mqW8Wy3F/x0/iTYNTVbBAr947NTOzmITJWI4E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716905198; x=1717509998;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Bhn2iLHzD8RvektTs6H45lIDox1xDUH0MHd7mOG3ujU=;
        b=V9cIoA8U3Y7aKnRPFcO5oaFvyVNTWRHVZtWgaJWW6ioeTDmUWDYYqoxx/YRcyytEQj
         rzhXPRD0S9OTV+R0BZy1LilAuidFVKP6pBwU4s0srY2tpA38kTq0EAwjuNK16sWeZQtI
         N5cIGE4eAUIv94ePI5J3zcjTP8Zy4Rm2QoiyR8G42inrqhCUWa+a8pyQLXRoLnkAPkoq
         30tZN07NVtpp828sOBjeE3aA0l/lwYkVKdzByPaCRymM3k/YtPDn8QvzVrgKM5/rGYKA
         hpKfuMb3mEyz9dJ4txO3xp1Acpnx/MQY/gQKh9fxh9+zkfURwXBzWO8VR3oHsrvqwXX9
         8tAA==
X-Gm-Message-State: AOJu0YyvKA35EiQCMaEhgsrIHM+dF8zbQ7EgAw7OCsMeqKtMb9bTfmMK
	jhRXhLEv0n1LBKtOFi9MCaFGMIj2Cm3+3tSmdaJzMKO3J6lp7PIhBYRlFPRTuIXbwPok37oGqAv
	Cr68=
X-Google-Smtp-Source: AGHT+IFxxosmwm5fJLRksdu5GoaFebftUhqfuKMLz9fQvz5jhSkuIx/ZWMEW47zD7NnhNranfn4A9w==
X-Received: by 2002:a50:cd01:0:b0:578:6019:265a with SMTP id 4fb4d7f45d1cf-578601927e8mr9871190a12.8.1716905197814;
        Tue, 28 May 2024 07:06:37 -0700 (PDT)
Message-ID: <5d00435f-510e-463e-ae1a-d7b3bce31151@cloud.com>
Date: Tue, 28 May 2024 15:06:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 8/8] xen/x86: Synthesise domain topologies
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715102098.git.alejandro.vallejo@cloud.com>
 <f51b54328a09c510c9320f1317c2da371ef16eb5.1715102098.git.alejandro.vallejo@cloud.com>
 <ZlBWtXkpkqzh1EWC@macbook> <6e1c52a3-98d2-42e9-96b7-91010806cdf5@cloud.com>
 <ZlRCXmjhsM5o4tVw@macbook>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <ZlRCXmjhsM5o4tVw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/05/2024 09:20, Roger Pau Monné wrote:
> On Fri, May 24, 2024 at 06:16:01PM +0100, Alejandro Vallejo wrote:
>> On 24/05/2024 09:58, Roger Pau Monné wrote:
>>> On Wed, May 08, 2024 at 01:39:27PM +0100, Alejandro Vallejo wrote:
>>>
>>>> +        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);
>>>
>>> I assume this generates the same topology as the current code, or will
>>> the population of the leaves be different in some way?
>>>
>>
>> The current code does not populate 0xb. This generates a topology
>> consistent with the existing INTENDED topology. The actual APIC IDs will
>> be different though (because there's no skipping of odd values).
>>
>> All the dance in patch 1 was to make this migrate-safe. The x2apic ID is
>> stored in the lapic hidden regs so differences with previous behaviour
>> don't matter.
> 
> What about systems without CPU policy in the migration stream, will
> those also get restored as expected?
> 
> I think you likely need to check whether 'restore' is set and keep the
> old logic in that case?
> 
> As otherwise migrated systems without a CPU policy will get the new
> topology information instead of the old one?

Bah. I hoped the x2apic ID restoration would mean I could get away with
removing all that junk, but migrations from Xen v.StoneAge do cause
mayhem. And it'd be very bizarre because the new topology leaves would
not reflect the existing x2apic IDs either.

I'll condense that blob of nonsense with the old scheme into a separate
function so we can easily deprecate it in the future.

> 
>> IOW, The differences are:
>>   * 0xb is exposed, whereas previously it wasn't
>>   * APIC IDs are compacted such that new_apicid=old_apicid/2
>>   * There's also a cleanup of the murkier paths to put the right core
>>     counts in the right leaves (whereas previously it was bonkers)
> 
> This needs to be in the commit message IMO.
> 

Sure.

>>>
>>> Note that currently the host policy also gets the topology leaves
>>> cleared, is it intended to not clear them anymore after this change?
>>>
>>> (as you only clear the leaves for the guest {max,def} policies)
>>>
>>> Thanks, Roger.
>>
>> It was like that originally in v1, I changed in v2 as part of feedback
>> from Jan.
> 
> I think that's fine, but this divergence from current behavior of
> cleaning the topology for the host policy needs to be mentioned in
> the commit message.
> 
> Thanks, Roger.

Sure.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Tue May 28 14:22:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 14:22:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731173.1136534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBxiu-0000o5-L4; Tue, 28 May 2024 14:22:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731173.1136534; Tue, 28 May 2024 14:22:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBxiu-0000ny-H0; Tue, 28 May 2024 14:22:44 +0000
Received: by outflank-mailman (input) for mailman id 731173;
 Tue, 28 May 2024 14:22:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBxit-0000ns-Po
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 14:22:43 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id be44446e-1cfd-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 16:22:41 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a635a74e0deso30209466b.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 07:22:41 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8e295sm617318766b.185.2024.05.28.07.22.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 07:22:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be44446e-1cfd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716906161; x=1717510961; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hiFfRD3NkJZafGxeBd15GFIv6yknVt/RJZXHd7avlw0=;
        b=RivQJyvB3GjeaWMiuUAdljKgxl9FTHLISnuz/yz6SDbsrVW4a5b715/lcAnC9xOU2R
         xmO6563dj7t6Y6qttYnd5J0za2dDZ7lGougpon3An1z5SdXYR+scfYjk9CXttuuriAPJ
         KvjgwXbR/8MtDz2yHm54SA7BMiop3rQsrj9Gc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716906161; x=1717510961;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=hiFfRD3NkJZafGxeBd15GFIv6yknVt/RJZXHd7avlw0=;
        b=HlxO+Z7Dkr//vnQzVZXIyTTh5Gh/KRoVem8MaaiZFPA1i6RngTCjaqVnoPdgHMEfaQ
         gn/O0TcYPsrnq3QYSy8fjMGKXuLSsyQVUCp8TLH5Ac+NBQId9K50YIHMKic+c7hEpfbN
         qWIRMPJKSFmjiHXWMkqraGmXtCj5hFXh5CaeIAu6OYB8YBlU2+OmaskL0XxixQ2Z6ts/
         eEgcO3xpcgdbJ47+7ctY5z3WBKcEs2QngJOCw5LAsbsttEEKXZOvk+OoV8EvKdmsbS4y
         hUX3XVHIkVlW+6ilLZ62+fWhLAPbrxZm36xnOK31FZ1vm47tvwDUaM4bscQl3kgbxeYL
         gg8w==
X-Gm-Message-State: AOJu0Ywju2Gc3a66JfdfKJFo0Zw7NiCB1Lx3y26EJ9jfCe30D6fcUWjy
	Q2SpxZPegiGUmihLU6qtl0IdbOambuGH1FYkohlC4d/rbVGkYpCMK96FU8c3MyYIbVs7kK2jtaM
	919Q=
X-Google-Smtp-Source: AGHT+IHvXG7uFMRmADwZ9KZBFro9Zxnay4o8Kz3oIymPjgQ/DwjM9yR/g3ZWlo0Vecjg+21SveG2YQ==
X-Received: by 2002:a17:906:3d0:b0:a63:5421:85b0 with SMTP id a640c23a62f3a-a6354218711mr80265266b.75.1716906160935;
        Tue, 28 May 2024 07:22:40 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	"consulting @ bugseng . com" <consulting@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Federico Serafini <federico.serafini@bugseng.com>,
	Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: [PATCH v2 for-4.19 0.5/13] xen: Introduce CONFIG_SELF_TESTS
Date: Tue, 28 May 2024 15:22:38 +0100
Message-Id: <20240528142238.1340228-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

... and move x86's stub_selftest() under this new option.

There is value in having these tests included in release builds too.

It will shortly be used to gate the bitops unit tests on all architectures.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
CC: consulting@bugseng.com <consulting@bugseng.com>
CC: Simone Ballarin <simone.ballarin@bugseng.com>
CC: Federico Serafini <federico.serafini@bugseng.com>
CC: Nicola Vetrini <nicola.vetrini@bugseng.com>

v2.5:
 * As suggested in "[PATCH v2 05/13] xen/bitops: Implement generic_f?sl() in
 lib/"

I've gone with SELF_TESTS rather than BOOT_TESTS, because already in bitops
we've got compile time tests (which aren't strictly boot time), and the
livepatching testing wants to be included here and is definitely not boot
time.
---
 xen/Kconfig.debug      | 6 ++++++
 xen/arch/x86/extable.c | 4 ++--
 xen/arch/x86/setup.c   | 2 +-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/xen/Kconfig.debug b/xen/Kconfig.debug
index 61b24ac552cd..07ff7eb7ba83 100644
--- a/xen/Kconfig.debug
+++ b/xen/Kconfig.debug
@@ -29,6 +29,12 @@ config FRAME_POINTER
 	  maybe slower, but it gives very useful debugging information
 	  in case of any Xen bugs.
 
+config SELF_TESTS
+	bool "Extra self-testing"
+	default DEBUG
+	help
+	  Enable extra unit and functional testing.
+
 config COVERAGE
 	bool "Code coverage support"
 	depends on !LIVEPATCH
diff --git a/xen/arch/x86/extable.c b/xen/arch/x86/extable.c
index 8415cd1fa249..705cf9eb94ca 100644
--- a/xen/arch/x86/extable.c
+++ b/xen/arch/x86/extable.c
@@ -144,7 +144,7 @@ search_exception_table(const struct cpu_user_regs *regs, unsigned long *stub_ra)
     return 0;
 }
 
-#ifdef CONFIG_DEBUG
+#ifdef CONFIG_SELF_TESTS
 #include <asm/setup.h>
 #include <asm/traps.h>
 
@@ -214,7 +214,7 @@ int __init cf_check stub_selftest(void)
     return 0;
 }
 __initcall(stub_selftest);
-#endif
+#endif /* CONFIG_SELF_TESTS */
 
 unsigned long asmlinkage search_pre_exception_table(struct cpu_user_regs *regs)
 {
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b50c9c84af6d..dd51e68dbe5b 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -742,7 +742,7 @@ static void noreturn init_done(void)
     system_state = SYS_STATE_active;
 
     /* Re-run stub recovery self-tests with CET-SS active. */
-    if ( IS_ENABLED(CONFIG_DEBUG) && cpu_has_xen_shstk )
+    if ( IS_ENABLED(CONFIG_SELF_TESTS) && cpu_has_xen_shstk )
         stub_selftest();
 
     domain_unpause_by_systemcontroller(dom0);

base-commit: 2d93f78bfe25f695d8ffb61d110da9df293ed71b
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 28 14:56:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 14:56:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731184.1136543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sByFf-0004lA-5E; Tue, 28 May 2024 14:56:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731184.1136543; Tue, 28 May 2024 14:56:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sByFf-0004l3-2O; Tue, 28 May 2024 14:56:35 +0000
Received: by outflank-mailman (input) for mailman id 731184;
 Tue, 28 May 2024 14:56:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sByFe-0004kt-2z; Tue, 28 May 2024 14:56:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sByFe-0007NN-0f; Tue, 28 May 2024 14:56:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sByFd-0003tb-MX; Tue, 28 May 2024 14:56:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sByFd-0006Yk-M1; Tue, 28 May 2024 14:56:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TZr1DP0m1AOAdm0XvLH15nHNpWrpRY/WPb+8MiSM6qA=; b=kcXBoHN77AN+/d3QyNp9N/KX56
	qI8+jhwvtmI/MeKPwfATPkF7DpQDVxW6P8cYMVoZMbT15ah+O8OX/bqQr8wmd5PR1uNY+KN2c5Ejy
	AFuxrRJWFOFbhCxvvpFbX3ERBPcV7G87sAC+5iFht5w34fM3MkOhDAhZv7ROBCgIwJfU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186170-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186170: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d93f78bfe25f695d8ffb61d110da9df293ed71b
X-Osstest-Versions-That:
    xen=96af090e33130b0bf0953f3ccab8e7a163392318
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 14:56:33 +0000

flight 186170 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186170/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2d93f78bfe25f695d8ffb61d110da9df293ed71b
baseline version:
 xen                  96af090e33130b0bf0953f3ccab8e7a163392318

Last test of basis   186164  2024-05-28 02:00:23 Z    0 days
Testing same since   186166  2024-05-28 07:02:06 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   96af090e33..2d93f78bfe  2d93f78bfe25f695d8ffb61d110da9df293ed71b -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 28 15:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731193.1136554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymB-00018c-Mr; Tue, 28 May 2024 15:30:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731193.1136554; Tue, 28 May 2024 15:30:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymB-00018V-JE; Tue, 28 May 2024 15:30:11 +0000
Received: by outflank-mailman (input) for mailman id 731193;
 Tue, 28 May 2024 15:30:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5inG=M7=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1sBymA-00018E-6O
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:30:10 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2a3f3851-1d07-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 17:30:08 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5295ae273c8so1228654e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:30:08 -0700 (PDT)
Received: from fhilly.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937444sm621673366b.61.2024.05.28.08.30.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:30:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a3f3851-1d07-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716910207; x=1717515007; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=cM8bna0Wd5m2xXZDWh7XLMYuwWxXgUnRwXpmL//Hrrk=;
        b=WtSLXsHcqBK2dnyZebTNYiGWpwSgCyuEVp55WBuWznd6gAVwZPgOVrYhMtIeEUqLmD
         /q/GRsSrkrmfEM3SwmM8Ht8/C1KaZpeHtracoOiOA9rXpcNzuq23OKoH6pCzgD+ZS80l
         3s4uqPSfy6ouMXtKlon9ZiJN1drJNIdfOJ7bI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910207; x=1717515007;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=cM8bna0Wd5m2xXZDWh7XLMYuwWxXgUnRwXpmL//Hrrk=;
        b=qNAqvwkk/m8aj11GSk2B92CEMXFhuAlf2BeJ8If8sXFxFgog+zD/n/KVkKsXWV2ZOv
         zGQdfWQ7hG4+b7L2y/B4y1hrqV2oWlZmXuI8zdMRCThJLO5kIn983P+hffbhQlcZduXi
         kLUZXm3hyCqjki1dTt3gu/frlA1Q/Nf6/nLiZ3axJpEHxNcVy2v6q6QxZysjxUKP+JFq
         hWrydKXmw6peuzQOz8aKhZ4CAWzrm9j4FEZhPnX1VyKSmSJHEtHrAG+CcvHm4RVL7GaX
         1NmkdUT3CLIZ9zs4jLvA8rm8jd+kD4g0oa0h07SJ6qOpycajeDf5nfy0DHJF97g9w4EW
         a1sA==
X-Gm-Message-State: AOJu0Yzn5PCzUkSbcAqErDkjM4G+AW+UjJr+O0afR+2G5CKJ/m8Qk7Pb
	sWhmCgdysV0kZ+h+Bu4DycAhtb/2EDy9AT63+Qpq43OkVEbB+YnnmB01TEUzXpOnXzuysAxsZbz
	63Tg=
X-Google-Smtp-Source: AGHT+IGyfGvPVKupYkhjK39ncSBP2u8Cxf4mPdXyZBNWrdHIDqu+ocbS2jLPwe9aNNYwG93WY40ECw==
X-Received: by 2002:ac2:5dd3:0:b0:521:a96:bf15 with SMTP id 2adb3069b0e04-52965198d34mr7347722e87.38.1716910207599;
        Tue, 28 May 2024 08:30:07 -0700 (PDT)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v4 0/4]  x86/xen-ucode: Introduce --force option
Date: Tue, 28 May 2024 16:29:39 +0100
Message-ID: <20240528152943.3915760-1-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Refactor and introduce --force option to xen-ucode, which skips microcode
version check when updating x86 CPU micocode. A new hypercall introduced
with flags field to facilitate the new option and allow for future flags
as needed.
This change is required to enable developers to load ucode that is the same version as the
one already loaded or downgrade for testing.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>

Fouad Hilly (4):
  x86/ucode: Introduce PF_microcode_update2 with flags parameter
  x86/ucode: refactor xen-ucode to utilize getopt
  x86/ucode: Introduce --force option to xen-ucode to force skipping
    microcode version check
  x86/ucode: Utilize ucode_force and remove opt_ucode_allow_same

 docs/misc/xen-command-line.pandoc    |  7 +---
 tools/include/xenctrl.h              |  3 +-
 tools/libs/ctrl/xc_misc.c            | 12 +++---
 tools/misc/xen-ucode.c               | 61 ++++++++++++++++++++++------
 xen/arch/x86/cpu/microcode/amd.c     |  7 ----
 xen/arch/x86/cpu/microcode/core.c    | 20 +++++----
 xen/arch/x86/cpu/microcode/intel.c   |  4 --
 xen/arch/x86/cpu/microcode/private.h |  2 -
 xen/arch/x86/include/asm/microcode.h |  3 +-
 xen/arch/x86/platform_hypercall.c    | 13 +++++-
 xen/include/public/platform.h        | 14 +++++++
 11 files changed, 97 insertions(+), 49 deletions(-)

-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731196.1136584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymD-0001pR-G0; Tue, 28 May 2024 15:30:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731196.1136584; Tue, 28 May 2024 15:30:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymD-0001pK-C2; Tue, 28 May 2024 15:30:13 +0000
Received: by outflank-mailman (input) for mailman id 731196;
 Tue, 28 May 2024 15:30:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5inG=M7=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1sBymC-00018F-6c
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:30:12 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2c014c96-1d07-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 17:30:11 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a63359aaaa6so126666066b.2
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:30:11 -0700 (PDT)
Received: from fhilly.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937444sm621673366b.61.2024.05.28.08.30.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:30:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c014c96-1d07-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716910210; x=1717515010; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=z7mcoezWV0j0H2zzn7JIYS8Z2/p6a2i2l5f1Q/llZE0=;
        b=CWQ9MkMxFzYjqVq8JEzhTJ2ctT11Uq0m1Aib1E4zLZMy6d2llX6MxmHmRyOXsxM4yF
         lY2w+uV/D2G8l7SkEej/y8Dl6HdQqozFCI2M/I9dy+CPPZ6i9lVc/RcUyZ9f73ues3rJ
         QJDnisQS1Cik7CiHF8f3X3WC43NUjndyejmsc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910210; x=1717515010;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=z7mcoezWV0j0H2zzn7JIYS8Z2/p6a2i2l5f1Q/llZE0=;
        b=N0raUnfeVDjtZtrpUEpJQ6EZv54SGUiVi+jUQqgzCEtAPdhV4ikeHea+K/4uYIWish
         7mFKkfqUzpDxcyk1/TtcMEtarX3hB8TtF2mfY187U0ULZ2Hkpg0aJ7hZA5Q2429YzMiQ
         KZdKNIY5IiYxPwGBKFOFA7gu08r2iiikPPaAdu+R7P/diJZCF7eUWe99AwmJ/QdhxA6H
         Xlo37LyLyIULJThoC93cbgThHR1hhPNSGfvZciRwtgqx3KHlGYqhKXa2yZL38VYAfHKl
         Y6mlfQ6+Z15qJ+TY2ANi/ZYJjXxVkd8+i46ua5jC+edAc21tdIqdB77BTa22IXi19URe
         yN9Q==
X-Gm-Message-State: AOJu0Yzn6/U2LcSvXk4aAir6uS5rgocu5QNViWEQJowp5WEPbwUfsfLk
	JMlCD31X5/ksYk2/aIH12u4Bf0aTlwimVWN9Z9Ml38JYqcfOjB10pXUKL+r8N81v1qvQIOrAAgI
	S2U0=
X-Google-Smtp-Source: AGHT+IG4ytefBo9qwZrlmg2vHgWyH70OIDv7RtLE8pO6Uf1I0FZqM0xsPwXkUBHRuVwjEpLdxigKzQ==
X-Received: by 2002:a17:906:a287:b0:a59:cb29:3fb3 with SMTP id a640c23a62f3a-a6265148babmr864229366b.53.1716910210477;
        Tue, 28 May 2024 08:30:10 -0700 (PDT)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v4 4/4] x86/ucode: Utilize ucode_force and remove opt_ucode_allow_same
Date: Tue, 28 May 2024 16:29:43 +0100
Message-ID: <20240528152943.3915760-5-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
In-Reply-To: <20240528152943.3915760-1-fouad.hilly@cloud.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Pass ucode_force to common micorocde checks and utilize it to allow for microcode downgrade
or reapply the same version of the microcode.
Update low level Intel and AMD to check for valid signature only. Any version checks is done
at core.c.
While adding ucode_force, opt_ucode_allow_same was removed.
Remove opt_ucode_allow_same from documentation.

Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
[4]
1- As opt_ucode_allow_same is not required anymore, it has been removed while introducing ucode_force.
2- Apply the changes for both AMD and Intel.
3- Remove the mention of opt_ucode_allow_same from documentation.
---
 docs/misc/xen-command-line.pandoc    | 7 +------
 xen/arch/x86/cpu/microcode/amd.c     | 7 -------
 xen/arch/x86/cpu/microcode/core.c    | 9 +++------
 xen/arch/x86/cpu/microcode/intel.c   | 4 ----
 xen/arch/x86/cpu/microcode/private.h | 2 --
 5 files changed, 4 insertions(+), 25 deletions(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 1dea7431fab6..a42ce1039fed 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2648,7 +2648,7 @@ performance.
    Alternatively, selecting `tsx=1` will re-enable TSX at the users own risk.
 
 ### ucode
-> `= List of [ <integer> | scan=<bool>, nmi=<bool>, allow-same=<bool> ]`
+> `= List of [ <integer> | scan=<bool>, nmi=<bool> ]`
 
     Applicability: x86
     Default: `nmi`
@@ -2680,11 +2680,6 @@ precedence over `scan`.
 stop_machine context. In NMI handler, even NMIs are blocked, which is
 considered safer. The default value is `true`.
 
-'allow-same' alters the default acceptance policy for new microcode to permit
-trying to reload the same version.  Many CPUs will actually reload microcode
-of the same version, and this allows for easy testing of the late microcode
-loading path.
-
 ### unrestricted_guest (Intel)
 > `= <boolean>`
 
diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
index f76a563c8b84..4bcc79f1ab2d 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -225,13 +225,6 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
     if ( result == MIS_UCODE )
         return -EINVAL;
 
-    /*
-     * Allow application of the same revision to pick up SMT-specific changes
-     * even if the revision of the other SMT thread is already up-to-date.
-     */
-    if ( result == OLD_UCODE )
-        return -EEXIST;
-
     if ( check_final_patch_levels(sig) )
     {
         printk(XENLOG_ERR
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 8a9e744489b9..fc8ad8cfdd76 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -100,8 +100,6 @@ static bool __initdata ucode_scan;
 /* By default, ucode loading is done in NMI handler */
 static bool ucode_in_nmi = true;
 
-bool __read_mostly opt_ucode_allow_same;
-
 /* Protected by microcode_mutex */
 static struct microcode_patch *microcode_cache;
 
@@ -128,8 +126,6 @@ static int __init cf_check parse_ucode(const char *s)
 
         if ( (val = parse_boolean("nmi", s, ss)) >= 0 )
             ucode_in_nmi = val;
-        else if ( (val = parse_boolean("allow-same", s, ss)) >= 0 )
-            opt_ucode_allow_same = val;
         else if ( !ucode_mod_forced ) /* Not forced by EFI */
         {
             if ( (val = parse_boolean("scan", s, ss)) >= 0 )
@@ -583,6 +579,7 @@ static long cf_check microcode_update_helper(void *data)
     struct ucode_buf *buffer = data;
     unsigned int cpu, updated;
     struct microcode_patch *patch;
+    bool ucode_force = buffer->flags == XENPF_UCODE_FORCE;
 
     /* cpu_online_map must not change during update */
     if ( !get_cpu_maps() )
@@ -636,12 +633,12 @@ static long cf_check microcode_update_helper(void *data)
                                   microcode_cache);
 
         if ( result != NEW_UCODE &&
-             !(opt_ucode_allow_same && result == SAME_UCODE) )
+             (!ucode_force || (result & ~SAME_UCODE)) )
         {
             spin_unlock(&microcode_mutex);
             printk(XENLOG_WARNING
                    "microcode: couldn't find any newer%s revision in the provided blob!\n",
-                   opt_ucode_allow_same ? " (or the same)" : "");
+                   ucode_force? " (or a valid)" : "");
             microcode_free_patch(patch);
             ret = -EEXIST;
 
diff --git a/xen/arch/x86/cpu/microcode/intel.c b/xen/arch/x86/cpu/microcode/intel.c
index f505aa1b7888..5e1b528ffe05 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -297,10 +297,6 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
     if ( result == MIS_UCODE )
         return -EINVAL;
 
-    if ( result == OLD_UCODE ||
-         (result == SAME_UCODE && !opt_ucode_allow_same) )
-        return -EEXIST;
-
     wbinvd();
 
     wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)patch->data);
diff --git a/xen/arch/x86/cpu/microcode/private.h b/xen/arch/x86/cpu/microcode/private.h
index da556fe5060a..aea51678a662 100644
--- a/xen/arch/x86/cpu/microcode/private.h
+++ b/xen/arch/x86/cpu/microcode/private.h
@@ -3,8 +3,6 @@
 
 #include <asm/microcode.h>
 
-extern bool opt_ucode_allow_same;
-
 enum microcode_match_result {
     OLD_UCODE, /* signature matched, but revision id is older */
     SAME_UCODE, /* signature matched, but revision id is the same */
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731197.1136589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymD-0001sk-Pm; Tue, 28 May 2024 15:30:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731197.1136589; Tue, 28 May 2024 15:30:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymD-0001s8-Jp; Tue, 28 May 2024 15:30:13 +0000
Received: by outflank-mailman (input) for mailman id 731197;
 Tue, 28 May 2024 15:30:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5inG=M7=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1sBymC-00018E-AS
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:30:12 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2b9df50d-1d07-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 17:30:10 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-354faf5f1b4so779629f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:30:10 -0700 (PDT)
Received: from fhilly.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937444sm621673366b.61.2024.05.28.08.30.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:30:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b9df50d-1d07-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716910210; x=1717515010; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=29kKGaSNhLed8Jk9dwMB1HDWbyK77p3VZnpuSymtJak=;
        b=ishgnpllRC/svCCzlkPPL7nnUgrCjrP1JeLR6CvohTRc/AnaYHsPUJ1/JnXjxJMvtb
         48stk6/tUkjyr8RwLuO3mXGIbNhkcRXf+IhENdzwrukS/Og/m6C7Ev9UmUpqaGIibYiI
         dV5aOYRRt9cb4OKd3rxdHRBwLVKDWTOrZ7BmY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910210; x=1717515010;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=29kKGaSNhLed8Jk9dwMB1HDWbyK77p3VZnpuSymtJak=;
        b=PLzoKtte6ONCmDChBq39npOQAra0eIcUuzyiHKer9A3l4MlVdZHUB/ZlAF1z1osgZF
         5oOXO34DBkc5edfxHanu+k7ZNR+2ZldOUN71ASj4l9NuEDB2Rc9zis9nfw+EWzWiVq+N
         AMcHA0yRVN9Mgstp9zxsZATwco90kIFXz8MyirNo1MwxZWLro0EvWSzoRujLAixCT55o
         KhEnW+AC3ntiNzLUyotmdNJ/HVpQ69uVVVzPVSJMcksgMIdGf4OpEQRSelexml+vKNGV
         2hxwnStGu6XJo4EkiMrOzFB6Tk4zYodcaUvO6YnfKaZ6mEETRzWoBEuR4TwJcpCdyzSP
         xkNg==
X-Gm-Message-State: AOJu0YybK6Jr4tSM22vIzh7pgksLsY9daxQUluflw7X0OOFmEBnDadXZ
	xBuB1VHvZyeYDs8qWsVr2CDA7Yyw6A73rUPNi3turjzBDQPF45n5vy8Qzy75poBYxW17wk8iG8C
	oY48=
X-Google-Smtp-Source: AGHT+IHJ5pIptnLo5qJAv1DVP0wUcqG2/5riYCtRVCTsvS+oNS+wekQ6nvMZBPq3NglWopzcliQulQ==
X-Received: by 2002:adf:e681:0:b0:34c:b80a:233c with SMTP id ffacd0b85a97d-3552f4fd1b5mr10569229f8f.15.1716910209647;
        Tue, 28 May 2024 08:30:09 -0700 (PDT)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v4 3/4] x86/ucode: Introduce --force option to xen-ucode to force skipping microcode version check
Date: Tue, 28 May 2024 16:29:42 +0100
Message-ID: <20240528152943.3915760-4-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
In-Reply-To: <20240528152943.3915760-1-fouad.hilly@cloud.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Introduce --force option to xen-ucode to force skipping microcode version check, which
allows the user to update x86 microcode even if both versions are the same or downgrade.
xc_microcode_update() refactored to accept flags and utilize xenpf_microcode_update2.

Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
[4]
1- Add --force to xen-ucode options.
2- Update xc_microcode_update() to accept and handle flags.
---
 tools/include/xenctrl.h   |  3 ++-
 tools/libs/ctrl/xc_misc.c | 12 +++++++-----
 tools/misc/xen-ucode.c    | 14 +++++++++++---
 3 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 499685594427..7fb409bc6dc4 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1171,7 +1171,8 @@ typedef uint32_t xc_node_to_node_dist_t;
 int xc_physinfo(xc_interface *xch, xc_physinfo_t *info);
 int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
                    xc_cputopo_t *cputopo);
-int xc_microcode_update(xc_interface *xch, const void *buf, size_t len);
+int xc_microcode_update(xc_interface *xch, const void *buf,
+                        size_t len, unsigned int flags);
 int xc_get_cpu_version(xc_interface *xch, struct xenpf_pcpu_version *cpu_ver);
 int xc_get_ucode_revision(xc_interface *xch,
                           struct xenpf_ucode_revision *ucode_rev);
diff --git a/tools/libs/ctrl/xc_misc.c b/tools/libs/ctrl/xc_misc.c
index 50282fd60dcc..6a60216bda03 100644
--- a/tools/libs/ctrl/xc_misc.c
+++ b/tools/libs/ctrl/xc_misc.c
@@ -203,11 +203,12 @@ int xc_physinfo(xc_interface *xch,
     return 0;
 }
 
-int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
+int xc_microcode_update(xc_interface *xch, const void *buf,
+                        size_t len, unsigned int flags)
 {
     int ret;
     struct xen_platform_op platform_op = {};
-    DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
+    DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update2, uc);
 
     uc = xc_hypercall_buffer_alloc(xch, uc, len);
     if ( uc == NULL )
@@ -215,9 +216,10 @@ int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
 
     memcpy(uc, buf, len);
 
-    platform_op.cmd = XENPF_microcode_update;
-    platform_op.u.microcode.length = len;
-    set_xen_guest_handle(platform_op.u.microcode.data, uc);
+    platform_op.cmd = XENPF_microcode_update2;
+    platform_op.u.microcode2.length = len;
+    platform_op.u.microcode2.flags = flags;
+    set_xen_guest_handle(platform_op.u.microcode2.data, uc);
 
     ret = do_platform_op(xch, &platform_op);
 
diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
index 6f9dd2a7e431..b878edf2399a 100644
--- a/tools/misc/xen-ucode.c
+++ b/tools/misc/xen-ucode.c
@@ -13,6 +13,8 @@
 #include <xenctrl.h>
 #include <getopt.h>
 
+#include <xen/platform.h>
+
 static xc_interface *xch;
 
 static const char intel_id[] = "GenuineIntel";
@@ -24,7 +26,8 @@ static void usage(FILE *stream, const char *name)
            "Usage: %s [microcode file] [options]\n"
            "options:\n"
            "  -h, --help            display this help and exit\n"
-           "  -s, --show-cpu-info   show CPU information and exit\n",
+           "  -s, --show-cpu-info   show CPU information and exit\n"
+           "  -f, --force           force to skip microcode version check\n",
            name, name);
 }
 
@@ -89,10 +92,12 @@ int main(int argc, char *argv[])
     size_t len;
     struct stat st;
     int opt;
+    uint32_t ucode_flags = 0;
 
     static const struct option options[] = {
         {"help", no_argument, NULL, 'h'},
         {"show-cpu-info", no_argument, NULL, 's'},
+        {"force", no_argument, NULL, 'f'},
         {NULL, no_argument, NULL, 0}
     };
 
@@ -104,7 +109,7 @@ int main(int argc, char *argv[])
         exit(1);
     }
 
-    while ( (opt = getopt_long(argc, argv, "hs", options, NULL)) != -1 )
+    while ( (opt = getopt_long(argc, argv, "hsf", options, NULL)) != -1 )
     {
         switch (opt)
         {
@@ -114,6 +119,9 @@ int main(int argc, char *argv[])
         case 's':
             show_curr_cpu(stdout);
             exit(EXIT_SUCCESS);
+        case 'f':
+            ucode_flags = XENPF_UCODE_FORCE;
+            break;
         default:
             goto ext_err;
         }
@@ -147,7 +155,7 @@ int main(int argc, char *argv[])
     }
 
     errno = 0;
-    ret = xc_microcode_update(xch, buf, len);
+    ret = xc_microcode_update(xch, buf, len, ucode_flags);
     if ( ret == -1 && errno == EEXIST )
         printf("Microcode already up to date\n");
     else if ( ret )
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731195.1136566 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymC-0001Fo-8i; Tue, 28 May 2024 15:30:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731195.1136566; Tue, 28 May 2024 15:30:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymC-0001FH-1d; Tue, 28 May 2024 15:30:12 +0000
Received: by outflank-mailman (input) for mailman id 731195;
 Tue, 28 May 2024 15:30:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5inG=M7=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1sBymB-00018F-6L
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:30:11 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2accabe2-1d07-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 17:30:09 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a6341cf2c99so98851466b.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:30:09 -0700 (PDT)
Received: from fhilly.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937444sm621673366b.61.2024.05.28.08.30.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:30:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2accabe2-1d07-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716910208; x=1717515008; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bJX73ZmntNKRAs0BEZRhybEqHcJwu5VuVHE4B8+pY60=;
        b=jsH+tYRG95sELT8cWmsfGHEKCkPHYDNU3TW66Xhd9e98bau26M3lUVmgGOsC5GKd5m
         iLneUP35aqUVOdJkUOawonZiMCtloe9/+7D1B/kwKdD+flGQ50Ft56fS61TkqKFajSOA
         7vSqM+qFtDmfgQMqZ3KwmBHMP5KWaoDy4cNQ4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910208; x=1717515008;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bJX73ZmntNKRAs0BEZRhybEqHcJwu5VuVHE4B8+pY60=;
        b=Eh4PuwG0b4G7cGw3bmb766wb1N6DFYSygO5MnYEVKbB/BXNIrWGIPYo2PnK6PPTMV0
         ZfILHwDUmZR8MGxlAqvYPIPoJUBIawN2xwlM56ffgGUV7tFr1SkpD38KXAl8FVrvpEXk
         5sdY3XPhuQnXU+S0IYZyJvDSjg7aFBl47QQJNmJLOgJ3uc82MCwc2BTyd+DmZhUvyaWB
         T5ZV1wb51zOfngWOSVX5Dyw48yQC+MQ78KzNkuhCWd7hLmaNWj0OGO6tDs9+D9FptV6M
         OOLcKBzlOjpHlRx6frUUoW5KDN2wAzyOYYt4gN9cMVaHBczBI8Js1IzOJVmvlUNOKDYa
         QkgA==
X-Gm-Message-State: AOJu0Yx87t1DNWHPTAp4KSCxodswAWOx/5HE0Q4V+SsmTbJ4m55P9sSx
	9yl7pH/UcdORY25+lupDdXO5nIVd2RGFjNk32ofg9QyP7g9wxE9OrV+CM6w4yWxJc7YoIYSyElX
	9U48=
X-Google-Smtp-Source: AGHT+IHzLvVpGVHVUdhsRJaFBLhxIExE5P2Hb13QIBulTEa2VIkMdvPfsb8s3kG1g7MR5q0a2ZRzUg==
X-Received: by 2002:a17:906:3741:b0:a5a:1f4f:cb4c with SMTP id a640c23a62f3a-a62641b4532mr886848066b.7.1716910208404;
        Tue, 28 May 2024 08:30:08 -0700 (PDT)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v4 1/4] x86/ucode: Introduce PF_microcode_update2 with flags parameter
Date: Tue, 28 May 2024 16:29:40 +0100
Message-ID: <20240528152943.3915760-2-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
In-Reply-To: <20240528152943.3915760-1-fouad.hilly@cloud.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Refactor microcode_update() by adding flags field.
struct xenpf_microcode_update2 added with uint32_t flags field.
Introduce XENPF_microcode_update2 hypercall with flags field.

Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
[v4]
1- Commit message and description updated.
2- Changing the order of the patches.
[v3]
1- Updated Commit message description.
2- Revereted changes to a stable ABI and introduced a new struct.
3- ucode_force_flag updated from static to a local variable.
4- microcode_update() updated to reject unsupported flags yet.

[v2]
1- Update message description to highlight interface change.
2- Removed extra empty lines.
3- removed unnecessary define.
4- Corrected long lines.
5- Removed ternary operator.
6- Introduced static ucode_update_flags, which will be used later to determine local ucode_force_flag.
---
 xen/arch/x86/cpu/microcode/core.c    | 11 ++++++++---
 xen/arch/x86/include/asm/microcode.h |  3 ++-
 xen/arch/x86/platform_hypercall.c    | 13 ++++++++++++-
 xen/include/public/platform.h        | 14 ++++++++++++++
 4 files changed, 36 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index e90055772acf..8a9e744489b9 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -40,6 +40,8 @@
 #include <asm/processor.h>
 #include <asm/setup.h>
 
+#include <public/platform.h>
+
 #include "private.h"
 
 /*
@@ -570,6 +572,7 @@ static int cf_check do_microcode_update(void *patch)
 }
 
 struct ucode_buf {
+    unsigned int flags;
     unsigned int len;
     char buffer[];
 };
@@ -708,13 +711,14 @@ static long cf_check microcode_update_helper(void *data)
     return ret;
 }
 
-int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long len)
+int microcode_update(XEN_GUEST_HANDLE(const_void) buf,
+                     unsigned long len, unsigned int flags)
 {
     int ret;
     struct ucode_buf *buffer;
 
-    if ( len != (uint32_t)len )
-        return -E2BIG;
+    if ( flags & ~XENPF_UCODE_FORCE )
+        return -EINVAL;
 
     if ( !ucode_ops.apply_microcode )
         return -EINVAL;
@@ -730,6 +734,7 @@ int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long len)
         return -EFAULT;
     }
     buffer->len = len;
+    buffer->flags = flags;
 
     /*
      * Always queue microcode_update_helper() on CPU0.  Most of the logic
diff --git a/xen/arch/x86/include/asm/microcode.h b/xen/arch/x86/include/asm/microcode.h
index 8f59b20b0289..57c08205d475 100644
--- a/xen/arch/x86/include/asm/microcode.h
+++ b/xen/arch/x86/include/asm/microcode.h
@@ -22,7 +22,8 @@ struct cpu_signature {
 DECLARE_PER_CPU(struct cpu_signature, cpu_sig);
 
 void microcode_set_module(unsigned int idx);
-int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long len);
+int microcode_update(XEN_GUEST_HANDLE(const_void) buf,
+                     unsigned long len, unsigned int flags);
 int early_microcode_init(unsigned long *module_map,
                          const struct multiboot_info *mbi);
 int microcode_init_cache(unsigned long *module_map,
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index 95467b88ab64..7e3278109300 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -311,7 +311,18 @@ ret_t do_platform_op(
 
         guest_from_compat_handle(data, op->u.microcode.data);
 
-        ret = microcode_update(data, op->u.microcode.length);
+        ret = microcode_update(data, op->u.microcode.length, 0);
+        break;
+    }
+
+    case XENPF_microcode_update2:
+    {
+        XEN_GUEST_HANDLE(const_void) data;
+
+        guest_from_compat_handle(data, op->u.microcode2.data);
+
+        ret = microcode_update(data, op->u.microcode2.length,
+                               op->u.microcode2.flags);
         break;
     }
 
diff --git a/xen/include/public/platform.h b/xen/include/public/platform.h
index 15777b541690..2725b8d1044f 100644
--- a/xen/include/public/platform.h
+++ b/xen/include/public/platform.h
@@ -624,6 +624,19 @@ struct xenpf_ucode_revision {
 typedef struct xenpf_ucode_revision xenpf_ucode_revision_t;
 DEFINE_XEN_GUEST_HANDLE(xenpf_ucode_revision_t);
 
+/* Hypercall to microcode_update with flags */
+#define XENPF_microcode_update2    66
+struct xenpf_microcode_update2 {
+    /* IN variables. */
+    uint32_t flags;                   /* Flags to be passed with ucode. */
+/* Force to skip microcode version check */
+#define XENPF_UCODE_FORCE           1
+    uint32_t length;                  /* Length of microcode data. */
+    XEN_GUEST_HANDLE(const_void) data;/* Pointer to microcode data */
+};
+typedef struct xenpf_microcode_update2 xenpf_microcode_update2_t;
+DEFINE_XEN_GUEST_HANDLE(xenpf_microcode_update2_t);
+
 /*
  * ` enum neg_errnoval
  * ` HYPERVISOR_platform_op(const struct xen_platform_op*);
@@ -656,6 +669,7 @@ struct xen_platform_op {
         xenpf_symdata_t               symdata;
         xenpf_dom0_console_t          dom0_console;
         xenpf_ucode_revision_t        ucode_revision;
+        xenpf_microcode_update2_t     microcode2;
         uint8_t                       pad[128];
     } u;
 };
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731194.1136559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymC-0001BX-0n; Tue, 28 May 2024 15:30:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731194.1136559; Tue, 28 May 2024 15:30:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBymB-0001AM-Qy; Tue, 28 May 2024 15:30:11 +0000
Received: by outflank-mailman (input) for mailman id 731194;
 Tue, 28 May 2024 15:30:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5inG=M7=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1sBymA-00018F-P2
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:30:10 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2af7124b-1d07-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 17:30:09 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a62ef52e837so116111266b.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:30:09 -0700 (PDT)
Received: from fhilly.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937444sm621673366b.61.2024.05.28.08.30.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:30:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2af7124b-1d07-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716910209; x=1717515009; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eiZixxDd1y7HmvULwDCnAUl2gq5YDTcVfKI5Yqc7A6o=;
        b=Sh7lNcJHSd6q+jXD65e0hzkl69rRH8HrJdzb2lAtbzI+AM3jMi7xoX0ZuAge387TF3
         S1c7tYqb3wCirwJNNuZnd001THXFe0dy40jCgaDen+HnqnJiIvcAW47jyyOb8rkFKuge
         mjKEdd0JF92934BaXrT7B5jkk9pXM0zBTntZc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910209; x=1717515009;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=eiZixxDd1y7HmvULwDCnAUl2gq5YDTcVfKI5Yqc7A6o=;
        b=G0L3j1cAHsoGMQAakdms81MHhTuT7k8vlvf/grpJzX6dVveUw72r+8Nu+LTPK8Mh+b
         +1tDCBPII6jfu6RzICf8HiCsTv1TnyDjStwK/uwNop9McHazdA1iSpYHBvSu4oJDzl/c
         pACPABuu+9BNSPSMgdQoIH78Zq1Dq8Vl8H1jDY8SbxWNoNcGUnLM4J/J0QuquGNDlwCQ
         9ZXJOIONcd+PTZadx5xn2m2a87yzc1lzvQF7iv+2cXRa5o+aalfLV/FP4pq56wgVO0mt
         5p4yD7RU5URxyxkS4n2Nvh1FTpaVfvCCUluhpUU+VNYaFzOv22e3YmNHV7hsvs/yiyjA
         V/ng==
X-Gm-Message-State: AOJu0YwpQRQI1+FO5LU72mqYOn63gA82/WP3xf7bZUYFXnmpLPzxFqz/
	gEpwh2YM2vPxPqziUt8hRtsbTRqOpXnI84L/JBhTPv3KRLyjG6uwyNctsmxt72VKIXgtwkYWSur
	+zP8=
X-Google-Smtp-Source: AGHT+IGrPVCUH8m9ucxJfQ8Bfo55q54C/wA4hv+O7jryDjihHoTJ4VxJksixq/14le3M65dEFfPECg==
X-Received: by 2002:a17:906:1945:b0:a59:b807:330a with SMTP id a640c23a62f3a-a62641cf941mr896251366b.32.1716910208980;
        Tue, 28 May 2024 08:30:08 -0700 (PDT)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v4 2/4] x86/ucode: refactor xen-ucode to utilize getopt
Date: Tue, 28 May 2024 16:29:41 +0100
Message-ID: <20240528152943.3915760-3-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
In-Reply-To: <20240528152943.3915760-1-fouad.hilly@cloud.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Use getopt_long() to handle command line arguments.
Introduce ext_err for common exit with errors.
xc_microcode_update() refactored to accept flags and utilize xenpf_microcode_update2
Introducing usage() to handle usage\help messages in a common block.
show_curr_cpu is printed to stdout only.

Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
[v4]
1- Merge three patches into one.
2- usage() to print messages to the correct stream.
3- Update commit message and description.
---
 tools/misc/xen-ucode.c | 51 ++++++++++++++++++++++++++++++++----------
 1 file changed, 39 insertions(+), 12 deletions(-)

diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
index 390969db3d1c..6f9dd2a7e431 100644
--- a/tools/misc/xen-ucode.c
+++ b/tools/misc/xen-ucode.c
@@ -11,12 +11,23 @@
 #include <sys/stat.h>
 #include <fcntl.h>
 #include <xenctrl.h>
+#include <getopt.h>
 
 static xc_interface *xch;
 
 static const char intel_id[] = "GenuineIntel";
 static const char   amd_id[] = "AuthenticAMD";
 
+static void usage(FILE *stream, const char *name)
+{
+    fprintf(stream, "%s: Xen microcode updating tool\n"
+           "Usage: %s [microcode file] [options]\n"
+           "options:\n"
+           "  -h, --help            display this help and exit\n"
+           "  -s, --show-cpu-info   show CPU information and exit\n",
+           name, name);
+}
+
 static void show_curr_cpu(FILE *f)
 {
     int ret;
@@ -77,6 +88,13 @@ int main(int argc, char *argv[])
     char *filename, *buf;
     size_t len;
     struct stat st;
+    int opt;
+
+    static const struct option options[] = {
+        {"help", no_argument, NULL, 'h'},
+        {"show-cpu-info", no_argument, NULL, 's'},
+        {NULL, no_argument, NULL, 0}
+    };
 
     xch = xc_interface_open(NULL, NULL, 0);
     if ( xch == NULL )
@@ -86,22 +104,25 @@ int main(int argc, char *argv[])
         exit(1);
     }
 
-    if ( argc < 2 )
+    while ( (opt = getopt_long(argc, argv, "hs", options, NULL)) != -1 )
     {
-        fprintf(stderr,
-                "xen-ucode: Xen microcode updating tool\n"
-                "Usage: %s [<microcode file> | show-cpu-info]\n", argv[0]);
-        show_curr_cpu(stderr);
-        exit(2);
+        switch (opt)
+        {
+        case 'h':
+            usage(stdout, argv[0]);
+            exit(EXIT_SUCCESS);
+        case 's':
+            show_curr_cpu(stdout);
+            exit(EXIT_SUCCESS);
+        default:
+            goto ext_err;
+        }
     }
 
-    if ( !strcmp(argv[1], "show-cpu-info") )
-    {
-        show_curr_cpu(stdout);
-        return 0;
-    }
+    if ( optind == argc )
+        goto ext_err;
 
-    filename = argv[1];
+    filename = argv[optind];
     fd = open(filename, O_RDONLY);
     if ( fd < 0 )
     {
@@ -146,4 +167,10 @@ int main(int argc, char *argv[])
     close(fd);
 
     return 0;
+
+ ext_err:
+    fprintf(stderr,
+            "%s: unable to process command line arguments\n", argv[0]);
+    usage(stderr, argv[0]);
+    exit(EXIT_FAILURE);
 }
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:32:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:32:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731216.1136603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sByo5-0003mB-3g; Tue, 28 May 2024 15:32:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731216.1136603; Tue, 28 May 2024 15:32:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sByo5-0003m4-1C; Tue, 28 May 2024 15:32:09 +0000
Received: by outflank-mailman (input) for mailman id 731216;
 Tue, 28 May 2024 15:32:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sByo2-0003hj-U6
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:32:06 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 708e402e-1d07-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 17:32:06 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso1278570a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:32:06 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579d65934ddsm2736883a12.38.2024.05.28.08.32.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:32:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 708e402e-1d07-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716910325; x=1717515125; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=0xHve6PL7KT+Q9OZMD1lypR2DI/5gusYMZC8r2FNE2M=;
        b=JgVZuKd0ok1aaNVOYUjJQwDTg1arhOQZLRCmu1NqSAytLTzzoGsc6dw9VAtass9B8a
         G36U/JSgETyOfB4OC9TKj8DIFUr5Zax0A1AkDojK+fUVKqFByC/69NI0yXaZ+q1uKUUg
         HZmjG37ZVC1ldnBns31qUnie4jxE2QnMiPtTA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910325; x=1717515125;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0xHve6PL7KT+Q9OZMD1lypR2DI/5gusYMZC8r2FNE2M=;
        b=SCOzdkDfOGWfRi8sfvS9ZcjgTqzxQqWQBI0aZ9Lm6HM2/v9T+5Hp2AeuA6uwOiIwXD
         p+IF1Ox4nHPPBY5ma3vlR8W3uHWA81XKJi0O5E4ngYuBqGm7UfZpEWEPoF9v96LeSMMl
         XjpyChNHz97lfYFJl99PYpTzYp/JNKzJWX0VnBTn/msmxaCMaT0vvkivwEEYQCqpKAmr
         9SUIz8Ti+6PVQzqJVl4WjMQxlwpAd4zOMDDpzef/2lhKkVt4Ton0gNfvH35dkV3MzZTB
         UE8tMtpXwZiTcDDNCRNkQBvvFZGd5jvt5ru/emOE0U/BfnoDY8jQ/p71NCm6koB2ztTw
         fmtg==
X-Gm-Message-State: AOJu0Yzd8jycxXGUDptCdEFVSntc+MyZcd+J9wDdn6WwS7r9Nra501Iu
	A7uJA28p9SKI9UJBMRM2dD84bgqdRoOGlUSO5wwAZFGqAfbNyq4DdBnpQpFqI4So3d/JOe1mish
	bx4U=
X-Google-Smtp-Source: AGHT+IHp1sX/p4lLI2vNr7/D5sa+edVHYvSRUcQ1KYFoLpit8HagOdhaxu0d/2yF3P3upXxQfBMMEA==
X-Received: by 2002:a50:9f8f:0:b0:579:d23d:c6d8 with SMTP id 4fb4d7f45d1cf-579d9ff4d77mr3044829a12.18.1716910325439;
        Tue, 28 May 2024 08:32:05 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] x86/svm: Rework VMCB_ACCESSORS() to use a plain type name
Date: Tue, 28 May 2024 16:32:04 +0100
Message-Id: <20240528153204.1350919-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This avoids having a function call in a typeof() expression.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/include/asm/hvm/svm/vmcb.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/include/asm/hvm/svm/vmcb.h b/xen/arch/x86/include/asm/hvm/svm/vmcb.h
index 0396d10b90ba..28f715e37692 100644
--- a/xen/arch/x86/include/asm/hvm/svm/vmcb.h
+++ b/xen/arch/x86/include/asm/hvm/svm/vmcb.h
@@ -630,7 +630,7 @@ vmcb_get_ ## name(const struct vmcb_struct *vmcb) \
 }
 
 #define VMCB_ACCESSORS(name, cleanbit) \
-        VMCB_ACCESSORS_(name, typeof(alloc_vmcb()->_ ## name), cleanbit)
+    VMCB_ACCESSORS_(name, typeof(((struct vmcb_struct){})._ ## name), cleanbit)
 
 VMCB_ACCESSORS(cr_intercepts, intercepts)
 VMCB_ACCESSORS(dr_intercepts, intercepts)

base-commit: 2d93f78bfe25f695d8ffb61d110da9df293ed71b
prerequisite-patch-id: 4f38737ad5ea15b0341664ee1c6b96d0d00b6700
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:37:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:37:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731223.1136614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBytV-0004Rc-Mu; Tue, 28 May 2024 15:37:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731223.1136614; Tue, 28 May 2024 15:37:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBytV-0004RV-Jx; Tue, 28 May 2024 15:37:45 +0000
Received: by outflank-mailman (input) for mailman id 731223;
 Tue, 28 May 2024 15:37:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tjWx=M7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sBytU-0004RP-D8
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:37:44 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 38705bb1-1d08-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 17:37:41 +0200 (CEST)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2e9819a630fso15462821fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:37:41 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda1a97sm626713366b.188.2024.05.28.08.37.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 08:37:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38705bb1-1d08-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716910661; x=1717515461; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=vSEy2ETKGHjyHgCOnRsKIiBnMDV8gzkuxwZ76brN06M=;
        b=ZYpPB9WRldWFzQNHWdRh58jYhX7fL17mnbXanGxgnQVfUhQWpC6CE4VyQdQWnj/HGL
         ryAiCvI1kESMSBc/cDtVy5R7j8bdhLgyYC7JCN2oHOIwuISWP+QKAn2P3f5NcbWQVosC
         N24Ot5oZCwZ4okrlkNETnuwj0BfNPDGFlgDNNHINYjm+4VdVQh9lzVatWGSBRCWI8xg7
         vHJZxkZ+vZI5JoOHn/WQHqbrAUNCn2Wc+QN1ZJ+QPmhe0Rhq/rf4dFLBB5lNtQ6ZaoSz
         4NBL4mp9iPjQIq1AK0K1tOwZz+G7sVy3Cmq4wnKlKP7shtkNyFbw4R7c+jngquzcL37H
         TFMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716910661; x=1717515461;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vSEy2ETKGHjyHgCOnRsKIiBnMDV8gzkuxwZ76brN06M=;
        b=ncJtU/o4uO3aqdxx1wL9KyWiBPm2y/JAjTa8KdBJ2h+tZ87nol/xTdNx4MysUOYHVi
         IEVavwA5HJykyYh+i5gaH/kGyClJaUwsV0X2lbd2zaOEJlYzoyAShlz731IXe3FaJ3ci
         6DNs4cQBufHzNz/3xvm9VRAPIzPax0ixQ45ACeFXooZMmbRmLIBdXE2EBQhGfgDdvM+0
         M4k94b3Kgq3IOVpKapOVKgJIEPyamp+M+tUwfFA8K89GSu5a7o1l3qYkjqq5ixg5y8Io
         ohKfASzLAby1Vtbqa+A/64yEI3/iZ+CD3zw4xzVwC/FR/69ffj+B14ofNkEmPmfQTuHB
         b+eg==
X-Forwarded-Encrypted: i=1; AJvYcCVsp7879JuluI4t2+A+Z99EbKUoLvf/OinZHA8hMGmMdeHTxhBM8a8c4jx9QQ0KMoOU/pvU6OakxBUKFU6Ke101qckmUkVrgYRbDtS/GKk=
X-Gm-Message-State: AOJu0YwqIXDnr7/kYxTaTpW50KE3dVjluFuqfXnDbBC4oS/jmnwhM0aa
	P5EO6BspjfOB4a5LYjSw7uPcUTNNvXfmjqDhsqOWwMTUbGfLzdQWM+i9o0h1OA==
X-Google-Smtp-Source: AGHT+IFtDUZFPFxGUj7vs/4Ncw624wqhwAu4aXC+RFb9/UtJnuxPTEvHKsz2fF2aYtlBH1yNilBbGA==
X-Received: by 2002:a2e:9b0a:0:b0:2e6:935f:b6d3 with SMTP id 38308e7fff4ca-2e95b096b74mr94554951fa.14.1716910661058;
        Tue, 28 May 2024 08:37:41 -0700 (PDT)
Message-ID: <b05a744d-cbc3-4de2-b138-5591c54d01e3@suse.com>
Date: Tue, 28 May 2024 17:37:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/svm: Rework VMCB_ACCESSORS() to use a plain type name
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240528153204.1350919-1-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240528153204.1350919-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.05.2024 17:32, Andrew Cooper wrote:
> This avoids having a function call in a typeof() expression.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Tue May 28 15:47:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 15:47:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731230.1136624 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBz2Q-0006TV-Jg; Tue, 28 May 2024 15:46:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731230.1136624; Tue, 28 May 2024 15:46:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBz2Q-0006TO-GM; Tue, 28 May 2024 15:46:58 +0000
Received: by outflank-mailman (input) for mailman id 731230;
 Tue, 28 May 2024 15:46:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GfFg=M7=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sBz2P-0006TD-BZ
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 15:46:57 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82bc74ab-1d09-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 17:46:55 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f45cso1196019a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 08:46:55 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578523d2507sm7383833a12.40.2024.05.28.08.46.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 May 2024 08:46:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82bc74ab-1d09-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716911215; x=1717516015; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=36xqcj3D3x7SY2kxbtaQtK/GYAFT3HqUVPcpqPSdoRw=;
        b=hIFznWZp4TvEiABZ2LHMoFL8XFppZgVNUj9Odv/0iVwZ2cMPFosuectSeLBlEoQHpg
         VilUoyDEb2hzlA/OE8aI8QzzGi7s1A3pF2w38uK14r974pquvqSItgPyGNq907h0JyGr
         /BfFIpfPCYTUCB2/JG3buapt+PW8SpllD71IcyyBGPiyikPQAgL2pEjNJo9CPJ6SJaA+
         jDk8RsaPh5zceTjCKfg2+1CC8i34RrL7y66oEW/h3JAeegORL/K31SiAZFM8o8qqwMH+
         5Sy1IEb0EzmckJUbX9wRCoHWTZjFwUJWbhyFxDOCFL7YB5izMq5INaWEVdmMP4O9b4MS
         emhQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716911215; x=1717516015;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=36xqcj3D3x7SY2kxbtaQtK/GYAFT3HqUVPcpqPSdoRw=;
        b=OND9dom4eJVFIGuZOGqdmTssR2IxjczNEM9lnXhfWnD7Ye66xp57AgJyxjN/v2ky/t
         pVYWJTT2jYA4l9XU1lnYZBoyktd5+0sEvJVgszFYrXl1stG+JCuGGaBFqcajhnErRHZG
         +z3dLq1DR5A3POR+crjx52NnTocKC/zIeTLBu22rfAuSR+RdyXxAhk36+lAX+wF54hOS
         4P053MN4uPRjVSobZfmyNOLvz3zPNiKE/KiKF04eKjgZr56aiIkTDlOrMloKHKZ6BPdA
         NO8lkhHyrbIc2gHG0PCfFvIpcla6qaR6XkotR9VCLR996p1CBRjrX1irmcxMbe/oM/Nu
         lr4w==
X-Forwarded-Encrypted: i=1; AJvYcCXRjoAno2u0+KZqSukFdIXP6OZf6PARzEli+10ZfUbkq8GgiEcX5eSRwZmy2dfNsvGTW8L6FIi4Aof7rVQ8PwBoGUomwhUIpnyzFum6vmQ=
X-Gm-Message-State: AOJu0Yxg3+AIwqodgqp6lJMMyC/VRv5BG0h25beQTIv2QCbgUXcVslyG
	Oc8XtGijFZ+aBZRpNEhLnhdoiuvPIrV64yp8h6hYE6GMM6plOnzI
X-Google-Smtp-Source: AGHT+IHaMaBKbolltDWHTNB/Tsp7JkK2hgDFSv2Zd2C71IoyCoOZQu9FAlZhjZOjzxVXehFiop2Igg==
X-Received: by 2002:a50:9f8f:0:b0:579:d23d:c6d8 with SMTP id 4fb4d7f45d1cf-579d9ff4d77mr3069533a12.18.1716911214996;
        Tue, 28 May 2024 08:46:54 -0700 (PDT)
Message-ID: <8de7d95c331ca5887b42743192bae2351c7c741e.camel@gmail.com>
Subject: Re: [PATCH] x86/svm: Rework VMCB_ACCESSORS() to use a plain type
 name
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Date: Tue, 28 May 2024 17:46:54 +0200
In-Reply-To: <b05a744d-cbc3-4de2-b138-5591c54d01e3@suse.com>
References: <20240528153204.1350919-1-andrew.cooper3@citrix.com>
	 <b05a744d-cbc3-4de2-b138-5591c54d01e3@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-28 at 17:37 +0200, Jan Beulich wrote:
> On 28.05.2024 17:32, Andrew Cooper wrote:
> > This avoids having a function call in a typeof() expression.
> >=20
> > No functional change.
> >=20
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
>=20
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Tue May 28 16:22:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 16:22:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731235.1136634 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBzax-0004lE-9z; Tue, 28 May 2024 16:22:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731235.1136634; Tue, 28 May 2024 16:22:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBzax-0004l7-6r; Tue, 28 May 2024 16:22:39 +0000
Received: by outflank-mailman (input) for mailman id 731235;
 Tue, 28 May 2024 16:22:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBzaw-0004l1-3Z
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 16:22:38 +0000
Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com
 [2607:f8b0:4864:20::72d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7e792d75-1d0e-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 18:22:36 +0200 (CEST)
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-794ab0ae817so68874885a.2
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 09:22:36 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abcdb0eesm393735385a.62.2024.05.28.09.22.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 09:22:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e792d75-1d0e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716913355; x=1717518155; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=sUnYhND7PQlHomPbEnz4JdcDacy+I5ez0ubP1Mezda8=;
        b=CBA+G77+6TGc/ief98T7Fp2GEE154KcZVu4hD92Dy43pq2yanH3RtOitgXzSyFIS/9
         vk4v+RRQT5udwULAbzg8G1zEW6kgTqcEr1D8l9VVbDZ8JBgGkZ8a60Jfpr9ffsvrKK6I
         opA7SgqOfymoUW2RsklEaMC6SBZOyvfPBbInQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716913355; x=1717518155;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=sUnYhND7PQlHomPbEnz4JdcDacy+I5ez0ubP1Mezda8=;
        b=uEW1C3cmwZ/n+0FtJIzJMme0BxrErUadIH8nyFP3BYtp7Dl2BqZp75JKrcDoCGXI67
         QRAhNw4Scn4n6v3G6f+aQn1OO3i0eD8PZmvW14M6T7QPzU5072iMeOv+RbC7P3hXPuAz
         /Rbrko29PwHUupZRCxfqboN6XK4Aqy6ZXf03cFNfT8rYGitIXu17OH5/uXafXssauk3I
         pf9Ni/eIRXY5Cry8kmVIomjKCfrk5TsVPfPejAfaA56dYE5QFLSNoqHA2+ueCEEs6/MP
         Y/5PZBtYA/e/OPM21T9ZqhG/V61DpOZudyYAN4W7MglRz9tTF2NcgI39PFWXTzh+0KzY
         7Mlg==
X-Forwarded-Encrypted: i=1; AJvYcCV0ycxF2Q/3sq4twao0/YYn8FyD1q+DaunwG/1m5K4kK5kzcy9ts6JmD19qe3pSgThGSNIQ53ErC3fjdmCFtMmu38vxkf+xPyfsap8Wf04=
X-Gm-Message-State: AOJu0YyyUuTFI6Z2nCKPpCayGYrxQiGwe7zyr98u+4uZRQIJ2oXgRz8i
	pvhtWAKLCMsdzNI3ey7KEMhgIbkAgyB7rzPARY+XVOjvOIHhxGwTU2UcqKutMkg=
X-Google-Smtp-Source: AGHT+IFxJH8fEv+nRbiTF535AWb9/2KsS5YtYvlvmPKXOuz/Cc01WE4bH71x12PLQN7E6azaI/IN4Q==
X-Received: by 2002:a05:620a:904:b0:792:989e:7c4e with SMTP id af79cd13be357-794ab13e8d4mr1273718185a.72.1716913355523;
        Tue, 28 May 2024 09:22:35 -0700 (PDT)
Message-ID: <7ae01b1a-2c22-4cca-aea1-b5c5fe76fa73@citrix.com>
Date: Tue, 28 May 2024 17:22:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] xen/x86: Drop useless non-Kconfig CONFIG_* variables
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Roberto Bagnara <roberto.bagnara@bugseng.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
 <20240521171546.3650664-3-andrew.cooper3@citrix.com>
 <bd0fe385-73a4-4f8f-b7b0-28acc9078f40@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <bd0fe385-73a4-4f8f-b7b0-28acc9078f40@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22/05/2024 7:03 am, Jan Beulich wrote:
> On 21.05.2024 19:15, Andrew Cooper wrote:
>> These are all either completely unused, or do nothing useful.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Not an objection, i.e. you're fine to commit as is with Stefano's R-b, yet
> still a question:
>
>> @@ -30,11 +29,8 @@
>>  /* Intel P4 currently has largest cache line (L2 line size is 128 bytes). */
>>  #define CONFIG_X86_L1_CACHE_SHIFT 7
>>  
>> -#define CONFIG_ACPI_SRAT 1
>>  #define CONFIG_ACPI_CSTATE 1
>>  
>> -#define CONFIG_WATCHDOG 1
> I wonder if this wouldn't make sense to become a proper Kconfig setting,
> thus ...
>
>> --- a/xen/include/xen/watchdog.h
>> +++ b/xen/include/xen/watchdog.h
>> @@ -9,8 +9,6 @@
>>  
>>  #include <xen/types.h>
>>  
>> -#ifdef CONFIG_WATCHDOG
>> -
>>  /* Try to set up a watchdog. */
>>  int watchdog_setup(void);
>>  
>> @@ -23,13 +21,4 @@ void watchdog_disable(void);
>>  /* Is the watchdog currently enabled. */
>>  bool watchdog_enabled(void);
>>  
>> -#else
>> -
>> -#define watchdog_setup() ((void)0)
>> -#define watchdog_enable() ((void)0)
>> -#define watchdog_disable() ((void)0)
>> -#define watchdog_enabled() ((void)0)
>> -
>> -#endif
> ... assigning purpose to these stubs.

Actually I need to keep CONFIG_WATCHDOG to not break the build on other
architectures.

watchdog_{en,dis}able() are called from common/{debugtrace,keyhandler}.c.

I'll leave it in place for now, and we can Kconfig it up later.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 16:34:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 16:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731247.1136662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBzlk-0006ye-Bi; Tue, 28 May 2024 16:33:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731247.1136662; Tue, 28 May 2024 16:33:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sBzlk-0006yX-97; Tue, 28 May 2024 16:33:48 +0000
Received: by outflank-mailman (input) for mailman id 731247;
 Tue, 28 May 2024 16:33:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sBzli-0006y6-Tx
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 16:33:46 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0d94facb-1d10-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 18:33:45 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-57822392a0dso1371162a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 09:33:45 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a632fde2922sm124737766b.195.2024.05.28.09.33.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 09:33:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d94facb-1d10-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716914025; x=1717518825; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Yo7/8kkuoSKCY5+XVcDVY2ZAzakX8OVvNd4tyd1qGf8=;
        b=YsfaLxnX2fRmHi8jpFQ0ta7x37q7NUFX+zzszbCBJeXGNResH1SLfQKidA0Y8i65Md
         ht00mg9dmZoQFqr+nxfE/V9UKKNaOCXCrQ3KSCpXjVxgUwrZGQfwJA4JsYn0WH1K+XtC
         HAdpoL6eGt4EPphT5+iTIm0ZN9Ym0OuLhdH5I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716914025; x=1717518825;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Yo7/8kkuoSKCY5+XVcDVY2ZAzakX8OVvNd4tyd1qGf8=;
        b=oGqgWcb2bYRDHL1O/PMmbaTBfmUcNxhN9FvbnyG0mUs38NVRueoK1PBMvTe6SpgUR5
         s9cLHA5x6BJYizAZvNZgIeqRachAnOmGPFxYxOP7ggWCuIi7bkSXBv8+H/ehqsupRiKO
         BlWjYesHFvFRJIjJEcAnoRmBLuX+/CPyGRKk2B0jgh3lJPEiL33mZkKHbz2sD9lhBbRM
         SnPOJ+UHyUeyX9zvIsLGiqxCi/6EKCoAxAZbot5iz6Fb28Fq0DFt6lcONcUquPlBAfbu
         3dt0laP+rTJ8pAQQKH4sU2DTPxqkzPmzdewq1SIYwHJCwwu2Xm55/sbFnR5TE8Yo6aLS
         hEZw==
X-Forwarded-Encrypted: i=1; AJvYcCWPTvL+SV2Kg/75qcceeL12itTkAAhN33qcTJH+jSlsbKMePZWGXLAsUCVFlPPyLyXrNdgGDMq1ChExcDIzwk/sY3rutvayqpHjlcJj6ls=
X-Gm-Message-State: AOJu0Yw3Z6N75PcU8YhnpoDcTIm9S5eLntb6HYIXhk4GmcbmD63Eb8Uv
	aqDsXQxkR4RdxjKGbFExO8grEhYCuQ8fymVJUJ9rWY4d+MayS+fmKYVhiE3CE+Y=
X-Google-Smtp-Source: AGHT+IFk/1DkviZmCaC14fYYKiV3QdKDTEO2jqsnw/kfB6xRpFbwh9PYSfxNhi8TBt1SRaOiAazM4A==
X-Received: by 2002:a17:906:22db:b0:a63:7e9e:6271 with SMTP id a640c23a62f3a-a637e9e63a4mr23683366b.22.1716914025070;
        Tue, 28 May 2024 09:33:45 -0700 (PDT)
Message-ID: <d14651e2-aa5f-4f85-afbd-b3437eac9e82@citrix.com>
Date: Tue, 28 May 2024 17:33:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 1/4] x86/ucode: Introduce PF_microcode_update2 with
 flags parameter
To: Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
 <20240528152943.3915760-2-fouad.hilly@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240528152943.3915760-2-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/05/2024 4:29 pm, Fouad Hilly wrote:
> Refactor microcode_update() by adding flags field.
> struct xenpf_microcode_update2 added with uint32_t flags field.
> Introduce XENPF_microcode_update2 hypercall with flags field.
>
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>

PF_ in the subject wants expanding to the full name, i.e.
XENPF_microcode_update2.  Can be fixed on commit.

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Tue May 28 16:57:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 16:57:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731255.1136673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC08v-00028d-5k; Tue, 28 May 2024 16:57:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731255.1136673; Tue, 28 May 2024 16:57:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC08v-00028W-1x; Tue, 28 May 2024 16:57:45 +0000
Received: by outflank-mailman (input) for mailman id 731255;
 Tue, 28 May 2024 16:57:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sC08t-00028Q-Nt
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 16:57:43 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64fde130-1d13-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 18:57:41 +0200 (CEST)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-6ab8e0c42d3so4820976d6.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 09:57:41 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac070c2edfsm45583876d6.23.2024.05.28.09.57.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 09:57:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64fde130-1d13-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716915460; x=1717520260; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=/eHTXTVwEYNnZ/xW+SMGR390e6jeab4vsoSzjxmcj30=;
        b=TnvehRktMzrLzd9q7DzUKwykVtAaphzyY1H0/67rk2yU4TxD8r/hpDmKSkBiUZZ8Je
         jtxQMf/ygX38ta/l43wBbE+T7VcTVeMLGpxPrEvnCHbCC8wZAi7rLZNTEiPIFnn1nEEE
         xhG9qlHwEaMXjUdpko9C1y2u/5XVLjV8HT0R8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716915460; x=1717520260;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/eHTXTVwEYNnZ/xW+SMGR390e6jeab4vsoSzjxmcj30=;
        b=W1rGtGfgDz1lbpTZQGbVg5aAWeJHwPxbDNbKXKct2nO54XHW6K50DdSAzBeTPqpVn6
         Nbp/NRJl/trN778da1CUDvxmcQhZY0bv3LlciOojSrBk4WtSy880v+qVrvo2OitFFFhh
         jMlfljDM8R0oHRxfsOREQjzoaENVpWC1z1/pcWJLdAILLhERQPtKXl/mWcaLRlIll1RU
         HH5UCDkMPYYIkmtqv9bHeC6ZhG3Prw5ZcjDXHUd5aYCpeeMW5IUczBDsx7zUIK9vHPfK
         4EK7qada80V8PkJSX5uGSfB1yxIvxOsGLJsnRmofu5YruV47uJNp/TrBMilTgw1Wb1EH
         c4lA==
X-Forwarded-Encrypted: i=1; AJvYcCVPEvalzv3ro+IbotWVQjw/0U+MlZwkz3Tm1KoyWkQRZPdfTg8tH4Col5EALJxP8Hv+3XIA3PRz2fuSuAbHUX6tENtXvb8wVRAdNxexKeg=
X-Gm-Message-State: AOJu0YzYqOhFu59yVNMKbnn9/BS1PhaPGkisoQ7HYv1bD5yluw6gv3UU
	cQCuHGBkDQ5MLC8sqCjZDRO+YIjWZghiA39mXqqSFctciaMguUhbaRdfbO9a63RFYUCeX8jTb62
	d2d8=
X-Google-Smtp-Source: AGHT+IHceOzp/InMKPF77JJNvFDPBrbiW12Y92Q9lwTChgipJzCd8FoD3DHwVPePaLXl4/TARNW0uw==
X-Received: by 2002:a05:6214:3281:b0:6aa:12cc:5adb with SMTP id 6a1803df08f44-6abcd0dee8dmr129085676d6.33.1716915460082;
        Tue, 28 May 2024 09:57:40 -0700 (PDT)
Message-ID: <95c38b73-0f37-4077-aede-8b5b6fce3fa7@citrix.com>
Date: Tue, 28 May 2024 17:57:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 2/4] x86/ucode: refactor xen-ucode to utilize getopt
To: Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
 <20240528152943.3915760-3-fouad.hilly@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240528152943.3915760-3-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/05/2024 4:29 pm, Fouad Hilly wrote:
> Use getopt_long() to handle command line arguments.
> Introduce ext_err for common exit with errors.
> xc_microcode_update() refactored to accept flags and utilize xenpf_microcode_update2

Importantly, not.  That's deferred until the next patch.

> Introducing usage() to handle usage\help messages in a common block.
> show_curr_cpu is printed to stdout only.
>
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> ---
> [v4]
> 1- Merge three patches into one.
> 2- usage() to print messages to the correct stream.
> 3- Update commit message and description.
> ---
>  tools/misc/xen-ucode.c | 51 ++++++++++++++++++++++++++++++++----------
>  1 file changed, 39 insertions(+), 12 deletions(-)
>
> diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
> index 390969db3d1c..6f9dd2a7e431 100644
> --- a/tools/misc/xen-ucode.c
> +++ b/tools/misc/xen-ucode.c
> @@ -11,12 +11,23 @@
>  #include <sys/stat.h>
>  #include <fcntl.h>
>  #include <xenctrl.h>
> +#include <getopt.h>
>  
>  static xc_interface *xch;
>  
>  static const char intel_id[] = "GenuineIntel";
>  static const char   amd_id[] = "AuthenticAMD";
>  
> +static void usage(FILE *stream, const char *name)
> +{
> +    fprintf(stream, "%s: Xen microcode updating tool\n"
> +           "Usage: %s [microcode file] [options]\n"

Options are typically expressed first when writing this.

> +           "options:\n"
> +           "  -h, --help            display this help and exit\n"
> +           "  -s, --show-cpu-info   show CPU information and exit\n",

The "and exit" is pointless to state like this.

> +           name, name);

Alignment is out by one column here.

Also, previously, usage() was always followed by show_curr_cpu(). 
That's still easy to retain by reordering the two static functions.

> +}
> +
>  static void show_curr_cpu(FILE *f)
>  {
>      int ret;
> @@ -77,6 +88,13 @@ int main(int argc, char *argv[])
>      char *filename, *buf;
>      size_t len;
>      struct stat st;
> +    int opt;
> +
> +    static const struct option options[] = {
> +        {"help", no_argument, NULL, 'h'},
> +        {"show-cpu-info", no_argument, NULL, 's'},
> +        {NULL, no_argument, NULL, 0}
> +    };

Typically we prefer static variables before automatic variables.  This
block can just be repositioned.

>  
>      xch = xc_interface_open(NULL, NULL, 0);
>      if ( xch == NULL )
> @@ -86,22 +104,25 @@ int main(int argc, char *argv[])
>          exit(1);
>      }
>  
> -    if ( argc < 2 )
> +    while ( (opt = getopt_long(argc, argv, "hs", options, NULL)) != -1 )
>      {
> -        fprintf(stderr,
> -                "xen-ucode: Xen microcode updating tool\n"
> -                "Usage: %s [<microcode file> | show-cpu-info]\n", argv[0]);
> -        show_curr_cpu(stderr);
> -        exit(2);
> +        switch (opt)
> +        {
> +        case 'h':
> +            usage(stdout, argv[0]);
> +            exit(EXIT_SUCCESS);

Blank line here, ...

> +        case 's':
> +            show_curr_cpu(stdout);
> +            exit(EXIT_SUCCESS);

... and here.

> +        default:
> +            goto ext_err;
> +        }
>      }
>  
> -    if ( !strcmp(argv[1], "show-cpu-info") )
> -    {
> -        show_curr_cpu(stdout);
> -        return 0;
> -    }
> +    if ( optind == argc )
> +        goto ext_err;

This is a change in API, because now "show-cpu-info" needs to take a --
to be interpreted.

I suspect we want:

    if ( optind == argc )
        goto ext_err;

    /* For backwards compatibility to the pre-getopt() cmdline handling */
    if ( !strcmp(argv[optind], "show-cpu-info") )
    {
        show_curr_cpu(stdout);
        return 0;
    }

to retain compatibility.  (Also, this means you won't have the code
rejected by XenRT when this gets added to Xen.)

All of this I can fix on commit, but there's quite a lot in the way of
changes.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 17:08:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 17:08:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731261.1136682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC0Jh-0004Fc-3L; Tue, 28 May 2024 17:08:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731261.1136682; Tue, 28 May 2024 17:08:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC0Jh-0004FV-0r; Tue, 28 May 2024 17:08:53 +0000
Received: by outflank-mailman (input) for mailman id 731261;
 Tue, 28 May 2024 17:08:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sC0Jg-0004FP-3w
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 17:08:52 +0000
Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com
 [2607:f8b0:4864:20::831])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3954c0c-1d14-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 19:08:50 +0200 (CEST)
Received: by mail-qt1-x831.google.com with SMTP id
 d75a77b69052e-43de92e234dso31031cf.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 10:08:50 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb16bed16sm44270981cf.17.2024.05.28.10.08.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 10:08:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3954c0c-1d14-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716916129; x=1717520929; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=0VnGD27iR18NGYFpXaECx07fWw283SIdLHOT0uxClus=;
        b=t0QdmHiDrpi8fib8zqXOqUSgojul1ZTtdX2a7TgfIKtwauTZKhbWbLXDcNTt7CyGu3
         gZlL7j7UjQy7DmXldg6CIdDpNSyzk0wkHmzAhnh9SlazN60H3zWmmfcGQhhyaOVRmu82
         1ibnL+4K20nYHUATtdaqd13V6iRjKIZcwC0oo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716916129; x=1717520929;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0VnGD27iR18NGYFpXaECx07fWw283SIdLHOT0uxClus=;
        b=VnSEV4HsuYnf13/5kOmmxwRbD0AfL41GGMLaNGZ1zemVAJDENT7MY5IlTyBRsFray3
         sMXrZWgKktm2i2U9ubC+KNj87BSiqSC7tAeJkPb9l8l4X8NyNRZPQ87smjCLwGGblv/6
         uPLW68VKX7rCiGhJ63ij9vM1K6XgB3mPyU97eZ5yGHc//0QIDmYPcssSmKpnw2cjkwHh
         9QLT8L8xZdcc1m3NFKy4rXh1ntE/xbfK9taWhECm/sslU3i0FN2u2cCIUgxCU83/E4Nh
         OFFLeXgSnF3gYUN2fWxWl3aAmXoIR928f/M9AW/aPQSaQUcQmKg+qGX/pcl3z4E/CWa7
         40zw==
X-Forwarded-Encrypted: i=1; AJvYcCVFSby+VsJ6UmBPqojy/MjfkXkwTk7QFUd8Tn+GpSURLZ71MtJoICeH1tsLqzWq9g0jfzcnVfJMYH9Fdxa1OWpXjTTOThT5iWnBMa0z4bM=
X-Gm-Message-State: AOJu0Yxv5/hjgrBTuDIKuKY72ko5wrybPxWi6afx+lvdzB+UrHKv8cb1
	YYtiNDyFg6/o60gGnnxboe1vIcBQ5Jk3b6BP7pl+1mhhOjB6hMi1xpY82KEC4Lg=
X-Google-Smtp-Source: AGHT+IFhs/t1q2nipb+JyQdfdLJnq6P3Rx+8boe6Rsfy5U1KI6yZCOz0RdeWrz+dCM4Ouw8LxFgHIA==
X-Received: by 2002:ac8:584a:0:b0:43a:d7a9:386 with SMTP id d75a77b69052e-43fafb19bdbmr256359691cf.17.1716916128882;
        Tue, 28 May 2024 10:08:48 -0700 (PDT)
Message-ID: <522f2449-030d-4b3b-9ace-5ff20ceafc2d@citrix.com>
Date: Tue, 28 May 2024 18:08:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 3/4] x86/ucode: Introduce --force option to xen-ucode
 to force skipping microcode version check
To: Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
 <20240528152943.3915760-4-fouad.hilly@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240528152943.3915760-4-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/05/2024 4:29 pm, Fouad Hilly wrote:
> Introduce --force option to xen-ucode to force skipping microcode version check, which
> allows the user to update x86 microcode even if both versions are the same or downgrade.
> xc_microcode_update() refactored to accept flags and utilize xenpf_microcode_update2.
>
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>

I think it would be better to stop the subject at "... to xen-ucode". 
The commit message itself covers what has changed.

> diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
> index 6f9dd2a7e431..b878edf2399a 100644
> --- a/tools/misc/xen-ucode.c
> +++ b/tools/misc/xen-ucode.c
> @@ -13,6 +13,8 @@
>  #include <xenctrl.h>
>  #include <getopt.h>
>  
> +#include <xen/platform.h>
> +
>  static xc_interface *xch;
>  
>  static const char intel_id[] = "GenuineIntel";
> @@ -24,7 +26,8 @@ static void usage(FILE *stream, const char *name)
>             "Usage: %s [microcode file] [options]\n"
>             "options:\n"
>             "  -h, --help            display this help and exit\n"
> -           "  -s, --show-cpu-info   show CPU information and exit\n",
> +           "  -s, --show-cpu-info   show CPU information and exit\n"
> +           "  -f, --force           force to skip microcode version check\n",

I'd phrase this as "skip certain checks; do not use unless you know
exactly what you are doing"

which makes it very clear that people get to keep all pieces if they try
this.

Otherwise (and subject to the style cleanup in the previous patch),
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Tue May 28 17:19:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 17:19:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731269.1136693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC0U8-0006Nv-50; Tue, 28 May 2024 17:19:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731269.1136693; Tue, 28 May 2024 17:19:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC0U8-0006No-2C; Tue, 28 May 2024 17:19:40 +0000
Received: by outflank-mailman (input) for mailman id 731269;
 Tue, 28 May 2024 17:19:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bNtp=M7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sC0U6-0006MV-P7
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 17:19:38 +0000
Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com
 [2607:f8b0:4864:20::729])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74ac3f19-1d16-11ef-b4bb-af5377834399;
 Tue, 28 May 2024 19:19:36 +0200 (CEST)
Received: by mail-qk1-x729.google.com with SMTP id
 af79cd13be357-79305314956so79016585a.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 10:19:36 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac07100a89sm45661156d6.68.2024.05.28.10.19.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 10:19:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74ac3f19-1d16-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716916775; x=1717521575; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=TPDO1mb7r21QsV8AV9LX1kCPFbWEzhggozge8QMrGBY=;
        b=ZcS0HHaCra+SMPc2ylkz2CCgkEPyBp2Ycf9HttMq/bUlhqNqwyPBIQX50fdomrQg4f
         vy5dbp3V5nTU7bFIXWi2hXGgRi9hZ8XVWok3u3VDhwuHhpO10q9dHthqPDQ4lX4XA7Bf
         qfseyJvtapiRLmEXTNE7vc0Ywg1euDisAw0ms=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716916775; x=1717521575;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TPDO1mb7r21QsV8AV9LX1kCPFbWEzhggozge8QMrGBY=;
        b=GRYCqXZmi5HlhEpPKjCZ1HdRo2nV3+9k+Pyavc8BbjbS8GVvfKpdaHWzW+USHX6cwc
         pAua9xvrJBhvrPpohQQ4nspbV+PCtfcmZpsD0hPqXPtt7NsyTZfx4cSUvf9+gz9wbKGo
         LQ7c/t6Immg1B1wnBFa1trtg91Z7tRWqVFXd51tcDxC+dQHVS0y0+gqQVboUJvziOlBL
         PZHif4h1o3F9yfW8kmOJtOPsEPYUL8XhduEqZy4vjf5hV+F4DTh40D8b1tOlr31+xa0S
         77ES+xLjKrV7BPgF0ji+vnOkMqJqk5hx2/HoCkY5wxOvH4lNE+UENPOScv9x+Hg+iR14
         dQ3g==
X-Forwarded-Encrypted: i=1; AJvYcCUJPLwrsbDX3UpZPlQiEaLnF4gRTByVkNk2EcIRvABVg0r9RhWkD6gNoKlflN1sg5uFt38ick3FmZ6HN5HxkdXv9ztaWGb30PPpL1HLNDU=
X-Gm-Message-State: AOJu0YwR9ZTvwCfUgXMetS/8bavI4onxMgFnNZH1PVqNNd7tUqeRg5CR
	Fit0Ykw2/WzDdfueNyCgO9nIymU7wGOgZdc5cS9LeS+aVvpk1g8/5NmUsZLYJZE=
X-Google-Smtp-Source: AGHT+IFqTiPCLeyEZbfisNnGoazSQe4xweRRZ+7QCr6O3UR25XxhV/sswJu/nwAGQEl8C4HmetC/5w==
X-Received: by 2002:a05:6214:4345:b0:6a8:f00e:cfb0 with SMTP id 6a1803df08f44-6abcd0de75fmr168451756d6.47.1716916774817;
        Tue, 28 May 2024 10:19:34 -0700 (PDT)
Message-ID: <3319c42c-3eb5-47ae-b722-952583829cce@citrix.com>
Date: Tue, 28 May 2024 18:19:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 4/4] x86/ucode: Utilize ucode_force and remove
 opt_ucode_allow_same
To: Fouad Hilly <fouad.hilly@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Jan Beulich
 <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240528152943.3915760-1-fouad.hilly@cloud.com>
 <20240528152943.3915760-5-fouad.hilly@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240528152943.3915760-5-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28/05/2024 4:29 pm, Fouad Hilly wrote:
> Pass ucode_force to common micorocde checks and utilize it to allow for microcode downgrade
> or reapply the same version of the microcode.
> Update low level Intel and AMD to check for valid signature only. Any version checks is done
> at core.c.
> While adding ucode_force, opt_ucode_allow_same was removed.
> Remove opt_ucode_allow_same from documentation.
>
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> ---
> [4]
> 1- As opt_ucode_allow_same is not required anymore, it has been removed while introducing ucode_force.
> 2- Apply the changes for both AMD and Intel.
> 3- Remove the mention of opt_ucode_allow_same from documentation.
> ---
>  docs/misc/xen-command-line.pandoc    | 7 +------
>  xen/arch/x86/cpu/microcode/amd.c     | 7 -------
>  xen/arch/x86/cpu/microcode/core.c    | 9 +++------
>  xen/arch/x86/cpu/microcode/intel.c   | 4 ----
>  xen/arch/x86/cpu/microcode/private.h | 2 --
>  5 files changed, 4 insertions(+), 25 deletions(-)
>
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 1dea7431fab6..a42ce1039fed 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -2648,7 +2648,7 @@ performance.
>     Alternatively, selecting `tsx=1` will re-enable TSX at the users own risk.
>  
>  ### ucode
> -> `= List of [ <integer> | scan=<bool>, nmi=<bool>, allow-same=<bool> ]`
> +> `= List of [ <integer> | scan=<bool>, nmi=<bool> ]`
>  
>      Applicability: x86
>      Default: `nmi`
> @@ -2680,11 +2680,6 @@ precedence over `scan`.
>  stop_machine context. In NMI handler, even NMIs are blocked, which is
>  considered safer. The default value is `true`.
>  
> -'allow-same' alters the default acceptance policy for new microcode to permit
> -trying to reload the same version.  Many CPUs will actually reload microcode
> -of the same version, and this allows for easy testing of the late microcode
> -loading path.
> -
>  ### unrestricted_guest (Intel)
>  > `= <boolean>`
>  
> diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
> index f76a563c8b84..4bcc79f1ab2d 100644
> --- a/xen/arch/x86/cpu/microcode/amd.c
> +++ b/xen/arch/x86/cpu/microcode/amd.c
> @@ -225,13 +225,6 @@ static int cf_check apply_microcode(const struct microcode_patch *patch)
>      if ( result == MIS_UCODE )
>          return -EINVAL;
>  
> -    /*
> -     * Allow application of the same revision to pick up SMT-specific changes
> -     * even if the revision of the other SMT thread is already up-to-date.
> -     */
> -    if ( result == OLD_UCODE )
> -        return -EEXIST;
> -

I'm afraid that because of the observation leading to 977d98e67c2e, I
see no option other than to plumb the force flag down into
apply_microcode().

This check cannot be deleted unconditionally, or we'll try downgrading
microcode even without the force flag being passed.

Unless we can fix the cacheing layer to not treat "I didn't load ucode
at boot" as "no idea of the symmetry of the system".

I'm unsure which of these two is going to be less ugly...

>      if ( check_final_patch_levels(sig) )
>      {
>          printk(XENLOG_ERR
> diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
> index 8a9e744489b9..fc8ad8cfdd76 100644
> --- a/xen/arch/x86/cpu/microcode/core.c
> +++ b/xen/arch/x86/cpu/microcode/core.c
> @@ -100,8 +100,6 @@ static bool __initdata ucode_scan;
>  /* By default, ucode loading is done in NMI handler */
>  static bool ucode_in_nmi = true;
>  
> -bool __read_mostly opt_ucode_allow_same;
> -
>  /* Protected by microcode_mutex */
>  static struct microcode_patch *microcode_cache;
>  
> @@ -128,8 +126,6 @@ static int __init cf_check parse_ucode(const char *s)
>  
>          if ( (val = parse_boolean("nmi", s, ss)) >= 0 )
>              ucode_in_nmi = val;
> -        else if ( (val = parse_boolean("allow-same", s, ss)) >= 0 )
> -            opt_ucode_allow_same = val;
>          else if ( !ucode_mod_forced ) /* Not forced by EFI */
>          {
>              if ( (val = parse_boolean("scan", s, ss)) >= 0 )
> @@ -583,6 +579,7 @@ static long cf_check microcode_update_helper(void *data)
>      struct ucode_buf *buffer = data;
>      unsigned int cpu, updated;
>      struct microcode_patch *patch;
> +    bool ucode_force = buffer->flags == XENPF_UCODE_FORCE;
>  
>      /* cpu_online_map must not change during update */
>      if ( !get_cpu_maps() )
> @@ -636,12 +633,12 @@ static long cf_check microcode_update_helper(void *data)
>                                    microcode_cache);
>  
>          if ( result != NEW_UCODE &&
> -             !(opt_ucode_allow_same && result == SAME_UCODE) )
> +             (!ucode_force || (result & ~SAME_UCODE)) )

What is "result & ~SAME_UCODE" trying to do?

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue May 28 18:57:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 18:57:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731284.1136719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC20O-0001Qc-6e; Tue, 28 May 2024 18:57:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731284.1136719; Tue, 28 May 2024 18:57:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC20O-0001QV-3g; Tue, 28 May 2024 18:57:04 +0000
Received: by outflank-mailman (input) for mailman id 731284;
 Tue, 28 May 2024 18:57:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC20M-0001P1-LO; Tue, 28 May 2024 18:57:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC20M-0003YU-Jf; Tue, 28 May 2024 18:57:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC20M-00018N-7a; Tue, 28 May 2024 18:57:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sC20M-00052Z-4k; Tue, 28 May 2024 18:57:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=WwBYsfJqir+eHhJkRhZyJrzqO6oiR2gv8NrQvxc+JeA=; b=B6LUM7knUpJnG5vIa8sH92uyo9
	21e5WFGeTl1F2CQaxw5ar4+4jsxBCs8C2xXL/l5oJaVTnejCQidnBqp2W0rNaUbiALqek3Jz++dea
	o3m2261IEpaQVqjqD4/BI1sMnXYuVVw0kBRz66FB03Z20KUNiJn1fjZ1iD8SBymW0jf0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186168-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186168: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=96af090e33130b0bf0953f3ccab8e7a163392318
X-Osstest-Versions-That:
    xen=ac572152e578a8853de0534384c1539ec21f11e0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 18:57:02 +0000

flight 186168 xen-unstable real [real]
flight 186173 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186168/
http://logs.test-lab.xenproject.org/osstest/logs/186173/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2   8 xen-boot            fail pass in 186173-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186173-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 186173 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 186173 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186163
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186163
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186163
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186163
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186163
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186163
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  96af090e33130b0bf0953f3ccab8e7a163392318
baseline version:
 xen                  ac572152e578a8853de0534384c1539ec21f11e0

Last test of basis   186163  2024-05-28 01:53:34 Z    0 days
Testing same since   186168  2024-05-28 10:08:43 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ac572152e5..96af090e33  96af090e33130b0bf0953f3ccab8e7a163392318 -> master


From xen-devel-bounces@lists.xenproject.org Tue May 28 19:03:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 19:03:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731292.1136729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC269-00030q-UV; Tue, 28 May 2024 19:03:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731292.1136729; Tue, 28 May 2024 19:03:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC269-00030j-Rf; Tue, 28 May 2024 19:03:01 +0000
Received: by outflank-mailman (input) for mailman id 731292;
 Tue, 28 May 2024 19:03:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KRkd=M7=linuxfoundation.org=gregkh@srs-se1.protection.inumbo.net>)
 id 1sC268-00030d-Un
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 19:03:01 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e5e74b7d-1d24-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 21:02:59 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DDB8962348;
 Tue, 28 May 2024 19:02:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45F71C32786;
 Tue, 28 May 2024 19:02:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5e74b7d-1d24-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1716922977;
	bh=GD3/dN6ZzfzJRaXladDHQA1hNTErMD4rTa0AbrBxOys=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=K19noRN/T7YC+i1E7Fkaoig7m7TchlX7Ihp54S2kNAmAvFxD77KVCgZp7oBpY/nrE
	 vH9MWNO2aQmpDABcsobPDhIHqyq3jmSTGwQ0RhfWTuQP1x93puLo6ULybvUuu4fFns
	 +Um0sNgoRC2z1kA5bcjukNYj3/wU+18y3tkozg5Q=
Date: Tue, 28 May 2024 21:03:01 +0200
From: Greg KH <gregkh@linuxfoundation.org>
To: Juergen Gross <jgross@suse.com>
Cc: cve@kernel.org, linux-cve-announce@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"Xen.org security team" <security@xen.org>
Subject: Re: CVE-2021-47377: kernel: xen/balloon: use a kernel thread instead
 a workqueue
Message-ID: <2024052826-overpass-manliness-7f2c@gregkh>
References: <2024041747-CVE-2024-26908-4690@gregkh>
 <9fba239d-2e22-425b-b45f-86ecf016c2fd@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <9fba239d-2e22-425b-b45f-86ecf016c2fd@suse.com>

On Mon, May 27, 2024 at 12:58:16PM +0200, Juergen Gross wrote:
> Hi,
> 
> I'd like to dispute CVE-2021-47377: the issue fixed by upstream commit
> 8480ed9c2bbd56fc86524998e5f2e3e22f5038f6 can in no way be triggered by
> an unprivileged user or by a remote attack of the system, as it requires
> initiation of memory ballooning of the running system. This can be done
> only by either a host admin or by an admin of the guest which might
> suffer the detection of the hanging workqueue.
> 
> Please revoke this CVE.

Ah, good catch, this came in as part of the GSD import, and I missed
that this required that type of permissions.  Now revoked, thanks for
the review!

greg k-h


From xen-devel-bounces@lists.xenproject.org Tue May 28 19:53:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 19:53:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731297.1136739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC2sS-0000RU-Jr; Tue, 28 May 2024 19:52:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731297.1136739; Tue, 28 May 2024 19:52:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC2sS-0000RN-Fk; Tue, 28 May 2024 19:52:56 +0000
Received: by outflank-mailman (input) for mailman id 731297;
 Tue, 28 May 2024 19:52:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC2sQ-0000RD-PA; Tue, 28 May 2024 19:52:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC2sQ-0004ZL-MR; Tue, 28 May 2024 19:52:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC2sQ-000433-Df; Tue, 28 May 2024 19:52:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sC2sQ-0007ID-DE; Tue, 28 May 2024 19:52:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dtwf1Y8MMzXAmV1IgHiQGRK4/U+h32cvmplmqrBneBs=; b=t+/fZ7hZ4FXnwXHIbl/iC6jEYm
	/1NaJScLHI0I8NlFEgWmNarW7IBQgBKoWC4qnWUnTj5xIEbaaT0ZZDpw96Rv3m9Dqs9vi1ysMyNAY
	1YsgvGVf4KWJl/FPcvPRUwDnHwll2sFA0FDJNJQMcK2F/xxAQkGrPIeFD68Vvo0gl/9A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186172-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186172: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d27c2835e0005200d464b57156c76455d46f74bb
X-Osstest-Versions-That:
    xen=2d93f78bfe25f695d8ffb61d110da9df293ed71b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 19:52:54 +0000

flight 186172 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186172/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d27c2835e0005200d464b57156c76455d46f74bb
baseline version:
 xen                  2d93f78bfe25f695d8ffb61d110da9df293ed71b

Last test of basis   186170  2024-05-28 11:00:23 Z    0 days
Testing same since   186172  2024-05-28 17:01:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2d93f78bfe..d27c2835e0  d27c2835e0005200d464b57156c76455d46f74bb -> smoke


From xen-devel-bounces@lists.xenproject.org Tue May 28 20:40:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 20:40:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731305.1136748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC3c1-0005Xk-Vh; Tue, 28 May 2024 20:40:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731305.1136748; Tue, 28 May 2024 20:40:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC3c1-0005Xd-Re; Tue, 28 May 2024 20:40:01 +0000
Received: by outflank-mailman (input) for mailman id 731305;
 Tue, 28 May 2024 20:40:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OYOI=M7=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sC3c0-0005QA-G0
 for xen-devel@lists.xenproject.org; Tue, 28 May 2024 20:40:00 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 730cbdbc-1d32-11ef-90a1-e314d9c70b13;
 Tue, 28 May 2024 22:39:59 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 6A2014EE073F;
 Tue, 28 May 2024 22:39:58 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 730cbdbc-1d32-11ef-90a1-e314d9c70b13
MIME-Version: 1.0
Date: Tue, 28 May 2024 22:39:58 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 3/4] x86: address violations of MISRA C Rule 8.4
In-Reply-To: <ffb55f45-d08e-4c60-af71-f9a56856bca9@suse.com>
References: <cover.1716814609.git.nicola.vetrini@bugseng.com>
 <cf926194a541d11e02670516a8d337de27836dce.1716814609.git.nicola.vetrini@bugseng.com>
 <ffb55f45-d08e-4c60-af71-f9a56856bca9@suse.com>
Message-ID: <7f39c844e1c781504ef71fecbd086009@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-05-28 08:28, Jan Beulich wrote:
> On 27.05.2024 16:53, Nicola Vetrini wrote:
>> Rule 8.4 states: "A compatible declaration shall be visible when
>> an object or function with external linkage is defined."
>> 
>> These variables are only referenced from asm modules, so they
>> need to be extern and there is negligible risk of them being
>> used improperly without noticing.
> 
> "asm modules" isn't quite correct, as there's one use from inline
> assembly. I have to admit I have no good wording suggestion other than
> explicitly covering both: "asm modules or inline assembly". Yet that
> then is ambiguous, as a use in inline assembly may also mean that
> symbol is actually visible to the compiler by being mentioned as on of
> the operands. Better ideas?
> 

Maybe generically "asm code" or just "asm"? It's not really relevant 
whether it's inline or not, as far as I understand.

>> As a result, they can be exempted using a comment-based deviation.
>> No functional change.
>> 
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> With suitably adjusted wording:
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> Jan

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Tue May 28 22:27:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 May 2024 22:27:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731319.1136759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC5Hn-0000Qs-CW; Tue, 28 May 2024 22:27:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731319.1136759; Tue, 28 May 2024 22:27:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC5Hn-0000Ql-9e; Tue, 28 May 2024 22:27:15 +0000
Received: by outflank-mailman (input) for mailman id 731319;
 Tue, 28 May 2024 22:27:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC5Hm-0000Qb-BM; Tue, 28 May 2024 22:27:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC5Hm-0007W3-AG; Tue, 28 May 2024 22:27:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC5Hm-0003HB-0R; Tue, 28 May 2024 22:27:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sC5Hl-000789-WA; Tue, 28 May 2024 22:27:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JxkF7QNCrhu90eMVHwUHx0LhH9TSPW/ZVyNfvxZHQnI=; b=V56NpweybTbT7HWzWDwoPsIlCi
	Y2+w7ZWbhoeUQRkEGHyKzejDpoLQOpTEl1QvX1UeNhdbTdc+27GI9KA35vPRNJAPzUOu+U1w111Xe
	p5+rLn19BoPw6ucXvXBnkjgqg5uF4uXtv704pmj7gwdzRePgJlzjHevPmBF6bH9sZUr4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186176-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186176: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=0e3189d406684e44608e01c93f7e2d53fa07b40a
X-Osstest-Versions-That:
    ovmf=08281572aab5b1f7e05bf26de4148af19eddc8b7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 May 2024 22:27:13 +0000

flight 186176 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186176/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 0e3189d406684e44608e01c93f7e2d53fa07b40a
baseline version:
 ovmf                 08281572aab5b1f7e05bf26de4148af19eddc8b7

Last test of basis   186160  2024-05-27 09:43:09 Z    1 days
Testing same since   186176  2024-05-28 20:43:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael D Kinney <michael.d.kinney@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   08281572aa..0e3189d406  0e3189d406684e44608e01c93f7e2d53fa07b40a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 29 02:26:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 02:26:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731346.1136768 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC91A-0007xa-9E; Wed, 29 May 2024 02:26:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731346.1136768; Wed, 29 May 2024 02:26:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC91A-0007xT-6H; Wed, 29 May 2024 02:26:20 +0000
Received: by outflank-mailman (input) for mailman id 731346;
 Wed, 29 May 2024 02:26:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC918-0007xJ-Tu; Wed, 29 May 2024 02:26:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC918-0003I3-SS; Wed, 29 May 2024 02:26:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sC918-0002rk-Eu; Wed, 29 May 2024 02:26:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sC918-0007CW-EM; Wed, 29 May 2024 02:26:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QTzI2klCqMStrjEUnT17UZET3SAGn8im0nMDnHBblK0=; b=txLEq/CkyfOZR3KGKehfN8oWNG
	BS5OQUzUg1zfac9FPhElhjkbzNbEnjeropJ/ZFaDvTYy3T5JTXc9+6EqIZUo6bpRwEBNdw7RH1QLX
	kLX3raANF9YUSnQAkXtJusLivGim4rHPz5Yro9GnIMAluSpBq3NRrSAukCV5Ub5DCQ7s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186178-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186178: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=55f8bddade205c9cbe3583d5d31d0048cdf26ed4
X-Osstest-Versions-That:
    ovmf=0e3189d406684e44608e01c93f7e2d53fa07b40a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 02:26:18 +0000

flight 186178 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186178/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 55f8bddade205c9cbe3583d5d31d0048cdf26ed4
baseline version:
 ovmf                 0e3189d406684e44608e01c93f7e2d53fa07b40a

Last test of basis   186176  2024-05-28 20:43:05 Z    0 days
Testing same since   186178  2024-05-29 00:41:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   0e3189d406..55f8bddade  55f8bddade205c9cbe3583d5d31d0048cdf26ed4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 29 02:41:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 02:41:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731354.1136779 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC9G3-0001zx-EK; Wed, 29 May 2024 02:41:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731354.1136779; Wed, 29 May 2024 02:41:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sC9G3-0001zq-BP; Wed, 29 May 2024 02:41:43 +0000
Received: by outflank-mailman (input) for mailman id 731354;
 Wed, 29 May 2024 02:41:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9Qc4=NA=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1sC9G2-0001zk-Ex
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 02:41:42 +0000
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2407::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f954a231-1d64-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 04:41:39 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by IA1PR12MB9031.namprd12.prod.outlook.com (2603:10b6:208:3f9::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Wed, 29 May
 2024 02:41:35 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7611.030; Wed, 29 May 2024
 02:41:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f954a231-1d64-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RfmOa6Y8+E+p7J7EGWgjbx0mkoU9S+7NSVX5ldBlIxe7IUe9uEAJwmQPDH4uZOmSODm13Wdc3W3jAVjYTqeGJoqPeV2ugYIA04CasHhD1xt5DgEqNpsu2+E9kChXOI6mvkSGY+8Y/hMYt0x5fINr9c7ak0trQBKlB5HFQJ1sOeEkY+XL/I96pwBe7wlCzjR9/R7LfIE9FTqVzjuMMthV4ATDfsADtUuQDPo2cIOFxBIDcQjTye49ZDcrjnr2pDqR/cUGzN04OcG0R3CfM7upImvbIMuyPjh+Tf0DvXmh4sIVIpyOgoCwglTds04RdljTNpcCmezelwtRo7NXsVqtXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=IDkjhV3rZSLjsIcf85zwMATrC5rqia9SuBxLTHhyLR0=;
 b=i8JPfn3gjp8bICdgTXNBNylPoIu65hGFkvxeMmf37Z54TJ4DA8KwC5e0CA7Oktisoo46Vd+pUMz1+R7I9Hx6HgOKQ6lItGXhU2KXacMXFm9ilZ2YuSeID+jA6UOn/GT1K+U9Vj3GKHsd+UpPfO7luEAAUObBr2FEx2C1EZ/S8YZQ+02LOYJfpB/To0JH6j1HFO/L7PL82j5KEYokZ83RlmGdceZadFyLiZE3a5JYwzc6m7I5n6i4SIlAN6fNPOFb/X061lmIjYrZItbUF00y/P848XMDdecg0sYlqm05gmw/j1NFd0yTZgqofx+SxMvir8wBS8HI0qoPgn1tt+c0Jw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IDkjhV3rZSLjsIcf85zwMATrC5rqia9SuBxLTHhyLR0=;
 b=gWEwO6bbAksZ1uHTYPeIipisL9Sy99vZ0attYYJAit+1GTQBOsP4uE3HYws/lknQPd1u2/UnoougKWthWOm0RzSdvdHvBcCpLyKJmYqwWLEf3B5FMaCp+/62wm9xyI/L2kPTJ/xv0dNE8MCtS0V7wVXw7sh550NeYKkoupM3S4M=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index:
 AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4D//4e1AIAAis+A//+FmgCAEsTugA==
Date: Wed, 29 May 2024 02:41:35 +0000
Message-ID:
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
In-Reply-To: <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7611.030)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|IA1PR12MB9031:EE_
x-ms-office365-filtering-correlation-id: be91c9b1-43db-40f8-e418-08dc7f88db8d
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|7416005|1800799015|366007|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?Sm1uaDJmdWNDMWF3VUFrTEtnWE9MOE02ZGtFMWFqZk12RkxudU1YbWdHNktv?=
 =?utf-8?B?MUZuVjBFVFNZcnYxam9lMkxqZlQ4VEVTOTMzM2k0dElIbkc1eVE0Y2JZdUhJ?=
 =?utf-8?B?QnVseUVFLzIveHBtaGJvM1dHYjdDWjJVWDBzWG5xQTRNWUFLWXV0MnRHZUd1?=
 =?utf-8?B?M2tPUUNGMWtDeUlKNkdZU3dmZTVRelNORkVPa3R4V3FmRWxzTlVndnVXVnJM?=
 =?utf-8?B?ZEJ2SGJ1Ykd4WDZWMXZFZzNiSHZmUHVxK3VDMyszUFZlcDNqb1RaVHhna0Nm?=
 =?utf-8?B?ai9KYWFwRlh3dnNsUlAxd3Rwc01uMXJTODY4d2RXWXZxT0VXMzBVMUFRRTU2?=
 =?utf-8?B?TVpPeDVaQXhxMWJGc2JXVmJuRmVqUm1hL0NXYnJ4YVNVT3FTRzZRNzR0dUVD?=
 =?utf-8?B?SnRFN016aE9Na0c4c3orTE5zVXRqUTQrTUxjYndPTEQvUFFmRUpTbTNkaTVh?=
 =?utf-8?B?ZHBLdlZhbllSZk9vMG5SenRXcFRURWV4dCtKRnZiSG9nQml5TVAwZ2J5U3Rm?=
 =?utf-8?B?SU1RMkswb2dKQlR6ZjNhMkJoV1hDZFdSa0xiVU1aYzhQRmVtZkluTlZRN09J?=
 =?utf-8?B?dytXZkJUeE1FUmEveU9uMk1kZFIvRy8rWERLVnpiUGJWdmFrRStEZVo2dXNp?=
 =?utf-8?B?c2IzOU81TUtNNCtsellFVEpqd2YvOU5jOEx0OTJFdStvbStobVd5bmZ6bERL?=
 =?utf-8?B?TUJoeUg0ZVRXbVZtUkwrc1B1RWYxcHA4cHhrUkdDZXEya1YyTGVjcEYyRU95?=
 =?utf-8?B?THk3T210bXJhV05kbFFqYkltVmdvQWJFSC80Y0wwUXUwL25lb0NmbS9PNy9q?=
 =?utf-8?B?QTlySWJFNEZRZlVoc0c3QUpiNFc4MlJwK3l3NDZydWZrMHJIODdobmpxWW1v?=
 =?utf-8?B?TzZTUHBCRWFUOW9hd0ZhU1VIM0NZcjZWbnhvbTRVUXFmTXdXSHFEUXFQQmVq?=
 =?utf-8?B?ZWNDcFJUaFBmSlRvMGZ1ZW5hMjUrNG9EL0gwcC92dWRDNUhNVS9RaWlWTXRG?=
 =?utf-8?B?UnhEeVBoVVVXT0FoNE9yb0p4MzMvRDF6ZDhaQ01OdmFtbGs0VEg0bDRjcTZw?=
 =?utf-8?B?ZVFmVXJOZlk1b1JsTEd5ZnBMZTNpUUREdWNzWVFUQ0NIa2ljK2VMTVFaalVu?=
 =?utf-8?B?LzMybW90ZTRpV3NMcDNka0hQTWpCU0paZE9qL00wRWhaaDFhNktrRHZ3amQx?=
 =?utf-8?B?QWl0dktvV0hLbFRQOHdXYWkzWmplSUgra0JmdjkwWUx4QlhMREpJY3RLR0ZO?=
 =?utf-8?B?WGZIdEJUOUJsVCswaGpRNE9RdVVab3VLUFNObWgvN3E3ekdFWGdXMzFBVEFy?=
 =?utf-8?B?OHpJRUxqa1ZDSys3a291ejk4NXBZbnBFT2ZmQjlHYzYyZ0R5Y1FXMlJZK1Fs?=
 =?utf-8?B?L1N0NzdIcXVEY0xodHhjbE1MdDM1WXk2V0R2Q2pvaTBmSE43SWhVay9QVXM1?=
 =?utf-8?B?cDI3NS9VUDNPSlJFL1hhRGxjLzdGVFVTOSt3RjI4bkpTNXJvajVTMWhLdXVh?=
 =?utf-8?B?N1FhWmxhZmIxcWRWQkFucHdQdFF3d3JxRzR3cUNaV3RWMlhiMmZHNys5d1RN?=
 =?utf-8?B?dnVzdmhsdjV2VWRPSHRJYWV0R3pEMFlEMC9Ca1FsR1pUa1hsMXhkOHNwZEk5?=
 =?utf-8?B?RkRvcHNQVDh6emRFMVAvYWdvMUlQbmtWY280aUZvcXBUU2wrQzJSZTlOMGY2?=
 =?utf-8?B?MGRNbEVDbXZzRW5rd3hQNFJud1BJMEE5NURHaWN3OWdjMnhEQkpndStKOEpQ?=
 =?utf-8?B?dXgwUVhpWnpqakxkUFo3Y3FSRzBZZVRYQ1kzR21pZWNwdEdsQjNxcUpPU0JL?=
 =?utf-8?B?OGticEdBQ2pEUDFkMVJ2QT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?WUpFOGxEeUdZd01ZZlVqYXJFMklLZ2xDWFFoRmNyWXZiVGlSTnRPZlZFSlZE?=
 =?utf-8?B?a2dSdkhkemdZZHdhWjZkNTFmbTB2aFNCR2YxUlJDc1FuQXh5OGFqTk9wTUd5?=
 =?utf-8?B?N1BPWW1nNDdiaUYwQmtqNVM4STNVYjN6ZUxrUjFvREU0UHRLWStRZGF2NVZ1?=
 =?utf-8?B?ajNiTm5KQnlENGhOUVAza2xaRnpCcFpJRUtzTkkxS2pUUGlZVEtWOC8ydUR0?=
 =?utf-8?B?N3JxZWl0T2c4UTg4QXhFNlBtcjM3MHVqYlNOdm5yS2hNODNlWFR4TkxJZUNP?=
 =?utf-8?B?S21zUTBCSWtIVVh4eXp6N0duWGtOaXBRdnpmSW94bXE1TmRCUXVrVmQrRE9k?=
 =?utf-8?B?Rmx4dll2L0h4RlBHK1ZKaFVEY3J3S2tMbVhBbjFmLzFtOVNZREsxUy9QV3o0?=
 =?utf-8?B?Sk9KTGM4NkVkemFMSVRyV2h1MGpiZUl5N3ZleW0zN1lGM3ozdnBkWlNncjI2?=
 =?utf-8?B?VnBhRnNKcjg1bzF6b1E5K3pMZFczM29EQXpPVjk2WTA2anlhR1IrL2FoN3Bl?=
 =?utf-8?B?Nm9TV0ZQQlowb0pabXdsMTAzZHRucVV6d2o3UHFBZVNFNHp4V1Rtdm9OTHpo?=
 =?utf-8?B?UTB1cEpKQXRGZ0tpclJhOUxzR0E3cTIvUERqZ29sVUpSVzJaRjk5WlVkTmxV?=
 =?utf-8?B?cVJrV1c5Q0JramM2eDlSOEtFM2tDLzJteTNjODFOZ0lKb3JsMzdPWXNDN1NW?=
 =?utf-8?B?aUFWRDR2R0l5MDdzb2RZVXZkWnZyUnVJdm8wQzM4eWZBVWRkMXZvV04yYW92?=
 =?utf-8?B?TUtaOW1DbHpqZjEyL0JON1U2YTlOS1NlQ05LTmJ3WFUrMS9XSTBoYTVCSWdr?=
 =?utf-8?B?UmwyNFhOY1J1NFpQL2JONU1lY2JmcVdGdzdlRU5YWVcvS0JlbFFIT21Hejlh?=
 =?utf-8?B?c0o4RW0rYlB4b1pPNC9UWUNuaTg1ekZ3MXFBK1FEQmRRVU1ka0MwT3p2bXNw?=
 =?utf-8?B?b0NjNVRDTDJUR0pFYjBPbncwQzNweUR5QmJhL1JhQ3VTZHp6VUNCcWgrV3VU?=
 =?utf-8?B?RjFWeHVFQVVzMThsYzhKRlBYdTNWSDhVeDZtQmg2UUtWTExxL21TU25hdHNR?=
 =?utf-8?B?RDBaNmZrWnZTTGx3dVBsdGN3MTEzWWN4RHJVK1ZjUWNRRlBnMC9Layt5R09D?=
 =?utf-8?B?YVdBOUZXc0RPZk9CK1ZwMHNwdTFxb2ZHdWlDVS9IcFZMVTMvSEV1N3M1Zk9i?=
 =?utf-8?B?ZzZQK005NWk3U0hSa1JMdTJiWGxCY3Z5MTB4dmRUajlpTEp3cUg1RnhhTDI2?=
 =?utf-8?B?VkQ4MDlpTUZjdERmdG1IaTVkcXhXd2tWTW4rNmZiVGh6S3ZMaGU4VzBtbEpQ?=
 =?utf-8?B?azU4RzZFRU5HNkpDZFRaWndKQ200elhTalNjOU9DMHlYN3VkMVlYQjhZQ1VM?=
 =?utf-8?B?MU5qY1RaYnNvL2pNR2cySTN1eTM0OENiem9USjZ6MEhNNE53UmZJcU02a3BN?=
 =?utf-8?B?SFBzMWYycVUwdy9wbWlzaFdaVDB3TiszMXhGVUFGMnJyMmRiT1ZSU1VIdG4r?=
 =?utf-8?B?ZnNzNTc1RVI5cGdnbXlMazdSRWQ1QUVPY2VhRnBxL3hDLzNJKzQ4RUtPYzcy?=
 =?utf-8?B?RGhObTRZYmFQKzhOUjU1aEE5WFdSdU9Xb2pGbVNZeU1vSHpoZjJydGd0Vjll?=
 =?utf-8?B?bnJ1bWdNQ3ZyMDg0Y2UvcnArNGlORlp2WHhPZ25QSnZVSWR5bG56VVhOR1Yw?=
 =?utf-8?B?c1RPQ2tjR0M2QWlEL2NJSXpRSUxwSWNQV3VGWUN5dkx2dFE1VE1jTHIxNzRj?=
 =?utf-8?B?R0ZoL2VqTnpnRFJBTmZJT1AzN290U0dsbktpUjdUckh2ak11QVdCdzdpdkZ0?=
 =?utf-8?B?ZGtlcUZHNVd6YUxZbXQrWkhaUGp0UjNIUjkzWExidkdrMkh4cmx4TGFFc3c4?=
 =?utf-8?B?MEx1OVNoWFhUUDRtdFlsRkpsdE9xamFQdVc4a3p3UWVFRjdOdGpHK0xGS1Nm?=
 =?utf-8?B?TXhEOWsvNnNvWTNHelVPUUlNajF4ZWNVSGZ2QUdyb2w1d1RYWEp2Z3BKMmho?=
 =?utf-8?B?MU91b3UwQ0JNTlVKSjVYZU8vVkR2eUVsNlpoNnMydVVmelVLZnpyeVE1amxE?=
 =?utf-8?B?aWVrY2hLUmt2OXZRY0dwTmFsWUxJZDFWa25RSFIyeHllbXpIZUZHSlFGMW04?=
 =?utf-8?Q?kHGQ=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <5212D35F83E7CB46A35322337E6ECC40@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: be91c9b1-43db-40f8-e418-08dc7f88db8d
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2024 02:41:35.5531
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4Rv42bhHKaRxiSnBlvJ7aQm3AlZmuUN0poY1ni73JS4Trfz1Ct0LEF5gQr4Bv6tp5He+wtNHgz0duUZ/3miJpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB9031

SGksDQpPbiAyMDI0LzUvMTcgMTk6NTAsIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBPbiAxNy4wNS4y
MDI0IDEzOjE0LCBDaGVuLCBKaXFpYW4gd3JvdGU6DQo+PiBPbiAyMDI0LzUvMTcgMTg6NTEsIEph
biBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDE3LjA1LjIwMjQgMTI6NDUsIENoZW4sIEppcWlhbiB3
cm90ZToNCj4+Pj4gT24gMjAyNC81LzE2IDIyOjAxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4+Pj4+
IE9uIDE2LjA1LjIwMjQgMTE6NTIsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4+Pj4+ICsgICAgICAg
IGlmICggZ3NpID49IG5yX2lycXNfZ3NpICkNCj4+Pj4+PiArICAgICAgICB7DQo+Pj4+Pj4gKyAg
ICAgICAgICAgIHJldCA9IC1FSU5WQUw7DQo+Pj4+Pj4gKyAgICAgICAgICAgIGJyZWFrOw0KPj4+
Pj4+ICsgICAgICAgIH0NCj4+Pj4+PiArDQo+Pj4+Pj4gKyAgICAgICAgaWYgKCAhaXJxX2FjY2Vz
c19wZXJtaXR0ZWQoY3VycmVudC0+ZG9tYWluLCBnc2kpIHx8DQo+Pj4+Pg0KPj4+Pj4gSS5lLiBh
c3N1bWluZyBJUlEgPT0gR1NJPyBJcyB0aGF0IGEgdmFsaWQgYXNzdW1wdGlvbiB3aGVuIGFueSBu
dW1iZXIgb2YNCj4+Pj4+IHNvdXJjZSBvdmVycmlkZXMgbWF5IGJlIHN1cmZhY2VkIGJ5IEFDUEk/
DQo+Pj4+IEFsbCBpcnFzIHNtYWxsZXIgdGhhbiBucl9pcnFzX2dzaSBhcmUgZ3NpLCBhcmVuJ3Qg
dGhleT8NCj4+Pg0KPj4+IFRoZXkgYXJlLCBidXQgdGhlcmUncyBub3QgbmVjZXNzYXJpbHkgYSAx
OjEgbWFwcGluZy4NCj4+IE9oLCBzbyBkbyBJIG5lZWQgdG8gYWRkIGEgbmV3IGdzaV9jYXBzIHRv
IHN0b3JlIGdyYW50ZWQgZ3NpPw0KPiANCj4gUHJvYmFibHkgbm90LiBZb3Ugb3VnaHQgdG8gYmUg
YWJsZSB0byB0cmFuc2xhdGUgYmV0d2VlbiBHU0kgYW5kIElSUSwNCj4gYW5kIHRoZW4gY29udGlu
dWUgdG8gcmVjb3JkIGluIC8gY2hlY2sgYWdhaW5zdCBJUlEgcGVybWlzc2lvbnMuDQpCdXQgSSBm
b3VuZCBpbiBmdW5jdGlvbiBpbml0X2lycV9kYXRhOg0KICAgIGZvciAoIGlycSA9IDA7IGlycSA8
IG5yX2lycXNfZ3NpOyBpcnErKyApDQogICAgew0KICAgICAgICBpbnQgcmM7DQoNCiAgICAgICAg
ZGVzYyA9IGlycV90b19kZXNjKGlycSk7DQogICAgICAgIGRlc2MtPmlycSA9IGlycTsNCg0KICAg
ICAgICByYyA9IGluaXRfb25lX2lycV9kZXNjKGRlc2MpOw0KICAgICAgICBpZiAoIHJjICkNCiAg
ICAgICAgICAgIHJldHVybiByYzsNCiAgICB9DQpEb2VzIGl0IG1lYW4gdGhhdCB3aGVuIGlycSA8
IG5yX2lycXNfZ3NpLCB0aGUgZ3NpIGFuZCBpcnEgaXMgYSAxOjEgbWFwcGluZz8NCldoYXQncyBt
b3JlLCB3aGVuIHVzaW5nIFBIWVNERVZPUF9zZXR1cF9nc2ksIGl0IGNhbGxzIG1wX3JlZ2lzdGVy
X2dzaSwNCmFuZCBpbiBtcF9yZWdpc3Rlcl9nc2ksIGl0IHVzZXMgIiBkZXNjID0gaXJxX3RvX2Rl
c2MoZ3NpKTsgIiB0byBnZXQgaXJxX2Rlc2MgZGlyZWN0bHkuDQoNCkNvbWJpbmluZyBhYm92ZSwg
Y2FuIHdlIGNvbnNpZGVyICJnc2kgPT0gaXJxIiB3aGVuIGlycSA8IG5yX2lycXNfZ3NpID8NCj4g
DQo+IEphbg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Wed May 29 04:26:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 04:26:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731362.1136788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCAsj-00056V-M6; Wed, 29 May 2024 04:25:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731362.1136788; Wed, 29 May 2024 04:25:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCAsj-00056O-JR; Wed, 29 May 2024 04:25:45 +0000
Received: by outflank-mailman (input) for mailman id 731362;
 Wed, 29 May 2024 04:25:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCAsi-00056E-35; Wed, 29 May 2024 04:25:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCAsh-0005c9-TL; Wed, 29 May 2024 04:25:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCAsh-0005pa-GW; Wed, 29 May 2024 04:25:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCAsh-0000G6-Fu; Wed, 29 May 2024 04:25:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iCj7nDWFIKxoNBninhBcoJ3GPyUZst/Sc4uwhrL+F50=; b=Qg+9ExpSGYkxZq0+IFpZ8xlEK6
	D9HEu1n3YpiXbjbmK3MfaEsutvooY1FaxjLC31WkgZCQ6lcFUAgivuGiMORHF/UpDBBbE3s7R+9Pb
	6EcZEx6P/vLf3AwbAPDI/rJOtRI8Y0wHIYg1QSmALx1HES1gxf86Bb+yjEylze0zkIzk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186174-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186174: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
X-Osstest-Versions-That:
    linux=2bfcfd584ff5ccc8bb7acde19b42570414bf880b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 04:25:43 +0000

flight 186174 linux-linus real [real]
flight 186177 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186174/
http://logs.test-lab.xenproject.org/osstest/logs/186177/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-raw       8 xen-boot            fail pass in 186177-retest
 test-armhf-armhf-xl-arndale   8 xen-boot            fail pass in 186177-retest
 test-armhf-armhf-xl-credit1   8 xen-boot            fail pass in 186177-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186177-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 186177 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 186177 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186177 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186177 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 186177 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 186177 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186161
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186161
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186161
 test-armhf-armhf-examine      8 reboot                       fail  like 186161
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186161
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186161
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186161
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186161
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
baseline version:
 linux                2bfcfd584ff5ccc8bb7acde19b42570414bf880b

Last test of basis   186161  2024-05-27 16:11:54 Z    1 days
Testing same since   186174  2024-05-28 18:10:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrii Nakryiko <andrii@kernel.org>
  Carlos López <clopez@suse.de>
  Carol Soto <csoto@nvidia.com>
  Jarkko Sakkinen <jarkko@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthew R. Ochs <mochs@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   2bfcfd584ff5..e0cce98fe279  e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed May 29 04:51:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 04:51:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731369.1136799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBHk-0000Jj-Kh; Wed, 29 May 2024 04:51:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731369.1136799; Wed, 29 May 2024 04:51:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBHk-0000Jc-HY; Wed, 29 May 2024 04:51:36 +0000
Received: by outflank-mailman (input) for mailman id 731369;
 Wed, 29 May 2024 04:51:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=qV8p=NA=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1sCBHj-0000JW-7A
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 04:51:35 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1ee7747d-1d77-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 06:51:33 +0200 (CEST)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-35b6467754cso912495f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 21:51:33 -0700 (PDT)
Received: from [192.168.69.100] ([176.187.204.141])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93b6d4sm677740766b.88.2024.05.28.21.51.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 21:51:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ee7747d-1d77-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716958292; x=1717563092; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=u1nYdfp4fuN4YG4QO33DKDAeJvAL793XESNugrAHys8=;
        b=xSqPfmXmmJ0QSd0GT72z+063yHKrJM5Q48Oa4b8R8DNjt4UoTFjz8GpLJB6pLKZm6T
         2hkskis7YDxZW5dky/tUCWVPuBzmWbd6HOhkOZHiIcPTA054odaNyay2tu1D+i3k9Xo5
         kkdKFgXZPNJ9KQoN4rDozM8dBloHp5htpFACzXQy0FXp+Kdx4ZJ+78VKXGm0j/y7iIdV
         QvI8TErlLJv5l/0ESPjPExB49lnPkcXeQhIvGVXgd003blL+2Vy03jSIULWdlCIg3+NJ
         gsZlY0ieJHvQ8XOom1F7jCqI4cge2U6XFIPJDAfcE65eBwF2xokRd+QKYnXlJnipnS16
         vQkA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716958292; x=1717563092;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=u1nYdfp4fuN4YG4QO33DKDAeJvAL793XESNugrAHys8=;
        b=OnICOj7bGIOBOLPbzHvFSM0cUonSvf6/GOXG01xJkUTCCYki+9SjUO8oBvdoZzThUx
         CHkG88QPH6c1igPbnp8E9XAUBcGYJM+DG9WgNnPs+8yWS1bJHtupelTdiwNImNHZxTER
         VcHP3kjah9IMZRzfYHByoV9ps0FfswuBQAhfKvS3veozf/Kl099RjrdBevT/6kaoDnXV
         Ppwq3q61blPdH17HQEJk7K75WSTgobw0DX4h0mFWqknyR1cElLjqHsVEO2YD1UO/iUkI
         OkxV5HmohpJPm3YAoVCVFI8/7TLn7yIYgu+rw7Aiw4MCSAgvotTXC80td3abPZTXPBqF
         zmHg==
X-Forwarded-Encrypted: i=1; AJvYcCW8oSEzPOTfidqUzcKeNVgP4PG+0+g7ZSpmvV6sH5dWAyhX7welRnnEy/cOWWv2jf5+19SXI/cz4qT6W2UIMHt9Cubu4kXhzPoNMc0ePaQ=
X-Gm-Message-State: AOJu0Yzyvk5SFLvgImbBBkawx6Wr69kevnTasKi9kQ43V+urbFr7UnV7
	WIfJZSDRz0VxTKSN/H5x4/ioJgu2aT7bzXePa+/3OaWwNY34a2osQr/77m1FotU=
X-Google-Smtp-Source: AGHT+IGT3NnjnHd9X0oouBZX33hk8pFTpyFyM+V1o+EOw+IGtUHkSyasUXrVcfyfdl3dvCH6VgD+zg==
X-Received: by 2002:adf:e648:0:b0:357:40e8:e574 with SMTP id ffacd0b85a97d-35740e8e60amr7150400f8f.37.1716958292268;
        Tue, 28 May 2024 21:51:32 -0700 (PDT)
Message-ID: <506f4ebe-50de-433b-8d06-9d2ceddde600@linaro.org>
Date: Wed, 29 May 2024 06:51:29 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 0/7] hw/xen: Simplify legacy backends handling
To: qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
Content-Language: en-US
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240510104908.76908-1-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

ping?

On 10/5/24 12:49, Philippe Mathieu-Daudé wrote:
> Respin of Paolo's Xen patches from
> https://lore.kernel.org/qemu-devel/20240509170044.190795-1-pbonzini@redhat.com/
> rebased on one of my cleanup branches making backend
> structures const. Treat xenfb as other backends.
> 
> Paolo Bonzini (2):
>    hw/xen: initialize legacy backends from xen_bus_init()
>    hw/xen: register legacy backends via xen_backend_init
> 
> Philippe Mathieu-Daudé (5):
>    hw/xen: Remove declarations left over in 'xen-legacy-backend.h'
>    hw/xen: Constify XenLegacyDevice::XenDevOps
>    hw/xen: Constify xenstore_be::XenDevOps
>    hw/xen: Make XenDevOps structures const
>    hw/xen: Register framebuffer backend via xen_backend_init()
> 
>   include/hw/xen/xen-legacy-backend.h | 15 +--------------
>   include/hw/xen/xen_pvdev.h          |  3 +--
>   hw/9pfs/xen-9p-backend.c            |  8 +++++++-
>   hw/display/xenfb.c                  | 15 +++++++++++++--
>   hw/i386/pc.c                        |  1 -
>   hw/usb/xen-usb.c                    | 14 ++++----------
>   hw/xen/xen-bus.c                    |  4 ++++
>   hw/xen/xen-hvm-common.c             |  2 --
>   hw/xen/xen-legacy-backend.c         | 24 ++++--------------------
>   hw/xenpv/xen_machine_pv.c           |  7 +------
>   10 files changed, 35 insertions(+), 58 deletions(-)
> 



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731377.1136849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVD-0003Di-1q; Wed, 29 May 2024 05:05:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731377.1136849; Wed, 29 May 2024 05:05:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVC-0003Cu-Ty; Wed, 29 May 2024 05:05:30 +0000
Received: by outflank-mailman (input) for mailman id 731377;
 Wed, 29 May 2024 05:05:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVA-0002IO-QW
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:28 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 105a5f3c-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:28 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBV3-00000002pVu-0qka; Wed, 29 May 2024 05:05:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 105a5f3c-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=1JKlKb0rmvW5PqJBHaV8Z8irK92Phe1LJwE6MWVRDfE=; b=hJnNXsXQgRIiijHRvkaC6khCDF
	FhMzB8UJBdujjVljnDILBpYyunHOLvQg97pJOqG8XS5Whld4a0FcSgNMLgv3e8z5FfNuUuyM35mH1
	rA33ugsElhDFF0pZpkQit2lJ4OqMhM5E7KKhT1x78n7aAnbtwYNeMsxWsjmJqay24Qv9y1kPNe/5F
	7ec0yGLmxMbLmdigGhh3bMS9CnfUVyYMLLol/0ucZtenyTuC0ERyCTtGXeffDCkZv/aLYTFu8u/Ul
	ryNj7V2iyodmaJw9byE7qPkk5yipU3slWKM7DX1jCQo+bbwptA6rN1NwEfIFSAbQCLytl/7ejWh+z
	iHKPUiaQ==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 04/12] sd: add a sd_disable_discard helper
Date: Wed, 29 May 2024 07:04:06 +0200
Message-ID: <20240529050507.1392041-5-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Add helper to disable discard when it is not supported and use it
instead of sd_config_discard in the I/O completion handler.  This avoids
touching more fields than required in the I/O completion handler and
prepares for converting sd to use the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 15d0035048d902..a8838381823254 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -821,6 +821,12 @@ static unsigned char sd_setup_protect_cmnd(struct scsi_cmnd *scmd,
 	return protect;
 }
 
+static void sd_disable_discard(struct scsi_disk *sdkp)
+{
+	sdkp->provisioning_mode = SD_LBP_DISABLE;
+	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
+}
+
 static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 {
 	struct request_queue *q = sdkp->disk->queue;
@@ -2245,12 +2251,12 @@ static int sd_done(struct scsi_cmnd *SCpnt)
 		case 0x24:	/* INVALID FIELD IN CDB */
 			switch (SCpnt->cmnd[0]) {
 			case UNMAP:
-				sd_config_discard(sdkp, SD_LBP_DISABLE);
+				sd_disable_discard(sdkp);
 				break;
 			case WRITE_SAME_16:
 			case WRITE_SAME:
 				if (SCpnt->cmnd[1] & 8) { /* UNMAP */
-					sd_config_discard(sdkp, SD_LBP_DISABLE);
+					sd_disable_discard(sdkp);
 				} else {
 					sdkp->device->no_write_same = 1;
 					sd_config_write_same(sdkp);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731376.1136830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVB-0002f0-Rx; Wed, 29 May 2024 05:05:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731376.1136830; Wed, 29 May 2024 05:05:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVB-0002cQ-KE; Wed, 29 May 2024 05:05:29 +0000
Received: by outflank-mailman (input) for mailman id 731376;
 Wed, 29 May 2024 05:05:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBV9-0002IO-N5
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0f7b9e1b-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:27 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBV0-00000002pUR-1qVO; Wed, 29 May 2024 05:05:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f7b9e1b-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=82BW6iQIk3FEFa4vhGtCkVzTHLP2megWywmDpB65ENg=; b=hT/SqbUWW+toXI5dMcqp4EA6Ee
	2PfJQLcR6iUvvvln1Iq2dGvlxDcvOm3MLw+bOGmfLSogQIOvsjUL5vqFC3t73TNL6yM38fzzTaHrF
	n3z3Uvx+PGLTAB560kOxs7K1nVBdUv6SHEDRKzIXrl7rcheULwalz5Awc07DH0HXLqz81X274x5m9
	4XGVCGLrrm21fNbYM3Tc6cbQkSazy+xeHzyQl85scPGKr1yLk0nmDJ6mvHN/CJr/bsmdhPJwzPQcW
	Cp1nnfhjKm1SYDktgoBXFVQycuzK0RJ/33dd5Y/upj+g1VIUurP/Ax3IWFmNEyPIcqKcAmzwwhJtu
	1DkDX8nA==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 03/12] sd: simplify the ZBC case in provisioning_mode_store
Date: Wed, 29 May 2024 07:04:05 +0200
Message-ID: <20240529050507.1392041-4-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Don't reset the discard settings to no-op over and over when a user
writes to the provisioning attribute as that is already the default
mode for ZBC devices.  In hindsight we should have made writing to
the attribute fail for ZBC devices, but the code has probably been
around for far too long to change this now.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 3dff9150ce11e2..15d0035048d902 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -461,14 +461,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 	if (!capable(CAP_SYS_ADMIN))
 		return -EACCES;
 
-	if (sd_is_zoned(sdkp)) {
-		sd_config_discard(sdkp, SD_LBP_DISABLE);
-		return count;
-	}
-
 	if (sdp->type != TYPE_DISK)
 		return -EINVAL;
 
+	/* ignore the proivisioning mode for ZBB devices */
+	if (sd_is_zoned(sdkp))
+		return count;
+
 	mode = sysfs_match_string(lbp_mode, buf);
 	if (mode < 0)
 		return -EINVAL;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731375.1136818 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVB-0002Og-7s; Wed, 29 May 2024 05:05:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731375.1136818; Wed, 29 May 2024 05:05:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVB-0002Mv-2m; Wed, 29 May 2024 05:05:29 +0000
Received: by outflank-mailman (input) for mailman id 731375;
 Wed, 29 May 2024 05:05:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBV9-0002IO-Cd
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0daed0b5-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:23 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBUs-00000002pSJ-2CVr; Wed, 29 May 2024 05:05:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0daed0b5-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:
	Content-ID:Content-Description:In-Reply-To:References;
	bh=TkLi7QJRJk+eDm0XKX9y4O+chg+2e2GHQobCf0qIp3Q=; b=j6TH1jV+bD9SRCoTjPn2/w5mPx
	Sn9ZORQd/0Vc8GCHMbe57GctL2xLaw94IPMXbiJcX95T1JOMiz6/rTaTDfBNMHzt2nFlgKG30WzoF
	dIAc0Wxprcex1TUAE9kVEamco0ms0NAnDTyHDmkwxAjOMGcnTzwA/qjowsfaaOxk+Q7ohP80n6DFh
	v0tR+RqojdE8wkTeuKbpKCEu64xWSJW+O63gy+h/ERmksEVT5Z+j3VJtipRNQyIKdFcIPJifoT7Lc
	JNKkuNwua/4XXVb82UFUzCKKgQOn1LTgSY21EVqNBDHbXJZj6E9u2XVKVoPDNoZ/0QiJ/ZgSUMdps
	IHDNZnvQ==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: convert the SCSI ULDs to the atomic queue limits API
Date: Wed, 29 May 2024 07:04:02 +0200
Message-ID: <20240529050507.1392041-1-hch@lst.de>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Hi all,

this series converts the SCSI upper level drivers to the atomic queue
limits API.

The first patch is a bug fix for ubd that later patches depend on and
might be worth picking up for 6.10.

The second patch changes the max_sectors calculation to take the optimal
I/O size into account so that sd, nbd and rbd don't have to mess with
the user max_sector value.  I'd love to see a careful review from the
nbd and rbd maintainers for this one!

The following patches clean up a few lose ends in the sd driver, and
then convert sd and sr to the atomic queue limits API.  The final
patches remove the now unused block APIs, and convert a few to be
specific to their now more narrow use case.

The patches are against Jens' block-6.10 tree.  Due to the amount of
block layer changes in here, and other that will depend on it, it
would be good if this could eventually be merged through the block
tree, or at least a shared branch between the SCSI and block trees.

Diffstat:
 arch/um/drivers/ubd_kern.c   |   10 +
 block/blk-settings.c         |  238 +------------------------------------------
 drivers/block/nbd.c          |    2 
 drivers/block/rbd.c          |    1 
 drivers/block/xen-blkfront.c |    4 
 drivers/scsi/sd.c            |  218 ++++++++++++++++++++-------------------
 drivers/scsi/sd.h            |    6 -
 drivers/scsi/sd_zbc.c        |   27 ++--
 drivers/scsi/sr.c            |   42 ++++---
 include/linux/blkdev.h       |   40 +++----
 10 files changed, 196 insertions(+), 392 deletions(-)


From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731374.1136813 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVB-0002L3-0b; Wed, 29 May 2024 05:05:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731374.1136813; Wed, 29 May 2024 05:05:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVA-0002Kd-SW; Wed, 29 May 2024 05:05:28 +0000
Received: by outflank-mailman (input) for mailman id 731374;
 Wed, 29 May 2024 05:05:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBV9-0002IP-7q
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0d8db1fb-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:24 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBUx-00000002pTc-49ZV; Wed, 29 May 2024 05:05:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d8db1fb-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=kkKXG4BPL9slh8VpntlZ2DVJFx0Ai+B2JWcJoYF05nw=; b=HnvQ4wSiS6N7e8PXLIagPtScJa
	9/CgQf5oHlA+0rUcPiwh+x8U1RVP3Swl11ecx0Ee6huxhdXFhz5IzhMmddCvwLO1YkXuZgnKHIyZv
	AOVIEy1TgjZeACrC1Wsy6aVLlvm0QgjUq0kp0HR3UzCsdxIH18itiz4xA3D/wB7hkO0VExVnzB8rs
	Ny7wFqmGC0E9tY75dJoAetU6VdBpDVUTO9E/9ZAV2FjtPP7/7inVd3uages1phxoxggirTFJYeodu
	7kl6Sd5Gbz+ioQGkhFm1LHTkWvL9+n2TaFTuftelgS7DtSTnFh2nQ4oxoLBRlTZwwpPhhY7PFNUIZ
	DgeOm/3A==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 02/12] block: take io_opt and io_min into account for max_sectors
Date: Wed, 29 May 2024 07:04:04 +0200
Message-ID: <20240529050507.1392041-3-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

The soft max_sectors limit is normally capped by the hardware limits and
an arbitrary upper limit enforced by the kernel, but can be modified by
the user.  A few drivers want to increase this limit (nbd, rbd) or
adjust it up or down based on hardware capabilities (sd).

Change blk_validate_limits to default max_sectors to the optimal I/O
size, or upgrade it to the preferred minimal I/O size if that is
larger than the kernel default if no optimal I/O size is provided based
on the logic in the SD driver.

This keeps the existing kernel default for drivers that do not provide
an io_opt or very big io_min value, but picks a much more useful
default for those who provide these hints, and allows to remove the
hacks to set the user max_sectors limit in nbd, rbd and sd.

Note that rd picks a different value for the optimal I/O size vs the
user max_sectors value, so this is a bit of a behavior change that
could use careful review from people familiar with rbd.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/blk-settings.c |  7 +++++++
 drivers/block/nbd.c  |  2 +-
 drivers/block/rbd.c  |  1 -
 drivers/scsi/sd.c    | 29 +++++------------------------
 4 files changed, 13 insertions(+), 26 deletions(-)

diff --git a/block/blk-settings.c b/block/blk-settings.c
index effeb9a639bb45..a49abdb3554834 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -153,6 +153,13 @@ static int blk_validate_limits(struct queue_limits *lim)
 		if (lim->max_user_sectors < PAGE_SIZE / SECTOR_SIZE)
 			return -EINVAL;
 		lim->max_sectors = min(max_hw_sectors, lim->max_user_sectors);
+	} else if (lim->io_opt) {
+		lim->max_sectors =
+			min(max_hw_sectors, lim->io_opt >> SECTOR_SHIFT);
+	} else if (lim->io_min &&
+		   lim->io_min > (BLK_DEF_MAX_SECTORS_CAP << SECTOR_SHIFT)) {
+		lim->max_sectors =
+			min(max_hw_sectors, lim->io_min >> SECTOR_SHIFT);
 	} else {
 		lim->max_sectors = min(max_hw_sectors, BLK_DEF_MAX_SECTORS_CAP);
 	}
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 22a79a62cc4eab..ad887d614d5b3f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1808,7 +1808,7 @@ static struct nbd_device *nbd_dev_add(int index, unsigned int refs)
 {
 	struct queue_limits lim = {
 		.max_hw_sectors		= 65536,
-		.max_user_sectors	= 256,
+		.io_opt			= 256 << SECTOR_SHIFT,
 		.max_segments		= USHRT_MAX,
 		.max_segment_size	= UINT_MAX,
 	};
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 26ff5cd2bf0abc..05096172f334a3 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -4954,7 +4954,6 @@ static int rbd_init_disk(struct rbd_device *rbd_dev)
 	    rbd_dev->layout.object_size * rbd_dev->layout.stripe_count;
 	struct queue_limits lim = {
 		.max_hw_sectors		= objset_bytes >> SECTOR_SHIFT,
-		.max_user_sectors	= objset_bytes >> SECTOR_SHIFT,
 		.io_min			= rbd_dev->opts->alloc_size,
 		.io_opt			= rbd_dev->opts->alloc_size,
 		.max_segments		= USHRT_MAX,
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index f6c822c9cbd2d3..3dff9150ce11e2 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -3593,7 +3593,7 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	struct request_queue *q = sdkp->disk->queue;
 	sector_t old_capacity = sdkp->capacity;
 	unsigned char *buffer;
-	unsigned int dev_max, rw_max;
+	unsigned int dev_max;
 
 	SCSI_LOG_HLQUEUE(3, sd_printk(KERN_INFO, sdkp,
 				      "sd_revalidate_disk\n"));
@@ -3675,34 +3675,15 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	else
 		blk_queue_io_min(sdkp->disk->queue, 0);
 
-	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
-		q->limits.io_opt = logical_to_bytes(sdp, sdkp->opt_xfer_blocks);
-		rw_max = logical_to_sectors(sdp, sdkp->opt_xfer_blocks);
-	} else {
-		q->limits.io_opt = 0;
-		rw_max = min_not_zero(logical_to_sectors(sdp, dev_max),
-				      (sector_t)BLK_DEF_MAX_SECTORS_CAP);
-	}
-
 	/*
 	 * Limit default to SCSI host optimal sector limit if set. There may be
 	 * an impact on performance for when the size of a request exceeds this
 	 * host limit.
 	 */
-	rw_max = min_not_zero(rw_max, sdp->host->opt_sectors);
-
-	/* Do not exceed controller limit */
-	rw_max = min(rw_max, queue_max_hw_sectors(q));
-
-	/*
-	 * Only update max_sectors if previously unset or if the current value
-	 * exceeds the capabilities of the hardware.
-	 */
-	if (sdkp->first_scan ||
-	    q->limits.max_sectors > q->limits.max_dev_sectors ||
-	    q->limits.max_sectors > q->limits.max_hw_sectors) {
-		q->limits.max_sectors = rw_max;
-		q->limits.max_user_sectors = rw_max;
+	q->limits.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
+	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
+		q->limits.io_opt = min_not_zero(q->limits.io_opt,
+				logical_to_bytes(sdp, sdkp->opt_xfer_blocks));
 	}
 
 	sdkp->first_scan = 0;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731378.1136852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVD-0003HC-Bc; Wed, 29 May 2024 05:05:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731378.1136852; Wed, 29 May 2024 05:05:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVD-0003Gn-6c; Wed, 29 May 2024 05:05:31 +0000
Received: by outflank-mailman (input) for mailman id 731378;
 Wed, 29 May 2024 05:05:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVC-0002IP-IA
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:30 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 10c503f8-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:29 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBV5-00000002pXT-2RwQ; Wed, 29 May 2024 05:05:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10c503f8-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=TFSwqKWMmudUxPiHaYQymKxOAPLlhLvIfGoe7lZ0Qm8=; b=rkKRbg5+hgq4FGzAhQuNxXCeww
	mYQ8LVxno936/Ke445SPr6vK77vH49ODPijMtTeGT6y17DO9VfLcNXcbJC5f4YwikvZ/giTD5aSWr
	2/AHxlrhMzrmMO9T4Siy0wWkSgBb4UwcUKe9nErCSp2M0KBoTzzxuF7KCfENr2yMCRDFL5eOvmh5m
	DeI3cPeP2Ved5/finWngzKaglnh7GYz6CIk2ozhCEb5yygEKOZrT2sr+DGXVaF5ohZ5wPEa/h0W27
	kIoeJXcGLgBclxYAHBILjIeu4JHHgDvp6JkiOgDF6dY2FxeqMeWGRaO5M9Fa1XPikTiqcIhm1oz/O
	/sPDQu3w==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 05/12] sd: add a sd_disable_write_same helper
Date: Wed, 29 May 2024 07:04:07 +0200
Message-ID: <20240529050507.1392041-6-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Add helper to disable WRITE SAME when it is not supported and use it
instead of sd_config_write_same in the I/O completion handler.  This
avoids touching more fields than required in the I/O completion handler
and  prepares for converting sd to use the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index a8838381823254..09ffe9d826aeac 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -1004,6 +1004,13 @@ static blk_status_t sd_setup_write_zeroes_cmnd(struct scsi_cmnd *cmd)
 	return sd_setup_write_same10_cmnd(cmd, false);
 }
 
+static void sd_disable_write_same(struct scsi_disk *sdkp)
+{
+	sdkp->device->no_write_same = 1;
+	sdkp->max_ws_blocks = 0;
+	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
+}
+
 static void sd_config_write_same(struct scsi_disk *sdkp)
 {
 	struct request_queue *q = sdkp->disk->queue;
@@ -2258,8 +2265,7 @@ static int sd_done(struct scsi_cmnd *SCpnt)
 				if (SCpnt->cmnd[1] & 8) { /* UNMAP */
 					sd_disable_discard(sdkp);
 				} else {
-					sdkp->device->no_write_same = 1;
-					sd_config_write_same(sdkp);
+					sd_disable_write_same(sdkp);
 					req->rq_flags |= RQF_QUIET;
 				}
 				break;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731373.1136808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVA-0002Is-Ot; Wed, 29 May 2024 05:05:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731373.1136808; Wed, 29 May 2024 05:05:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVA-0002Il-Lu; Wed, 29 May 2024 05:05:28 +0000
Received: by outflank-mailman (input) for mailman id 731373;
 Wed, 29 May 2024 05:05:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBV6-0002IO-VQ
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0cd62887-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:23 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBUv-00000002pSP-1Lzk; Wed, 29 May 2024 05:05:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cd62887-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=/OLpY6jO6BGmZUZluvnHNfe+GDP4sT7VwC1y29SpC6U=; b=OeadjJJGioCa4ge9VNZLLPIeJ8
	O9S5rUx9FafsfCUd/nVJdDPOpkJsv/E9yKCxbOOv290nnhE62p3wpyYQNPmrOmNyU3hzVOzhGPQxa
	5a4T1lTIFSW+n3MtalzxAOCYtI3DQHhzspZ8bYnEdLFBuQ+bNkNK5b871ug/c8Zf1MXixhuq1fwcL
	BxKay23FFqMmNqBL5VSxy6HUEZFvyI7zrsX4GqEOmWHd2HuIc7ZktQuuiNYT9hnqC19jbRgfOjXtD
	KR9+Ssjbvj0F4kFxGdIepfLmbgfwNS35I/CDczeE6V2t2s7vUtO9/Gkm7F3aD4SXdu4zuGlpm8kbx
	t86dAPmA==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 01/12] ubd: untagle discard vs write zeroes not support handling
Date: Wed, 29 May 2024 07:04:03 +0200
Message-ID: <20240529050507.1392041-2-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Discard and Write Zeroes are different operation and implemented
by different fallocate opcodes for ubd.  If one fails the other one
can work and vice versa.

Split the code to disable the operations in ubd_handler to only
disable the operation that actually failed.

Fixes: 50109b5a03b4 ("um: Add support for DISCARD in the UBD Driver")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 arch/um/drivers/ubd_kern.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index ef805eaa9e013d..a79a3b7c33a647 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -471,9 +471,14 @@ static void ubd_handler(void)
 		for (count = 0; count < n/sizeof(struct io_thread_req *); count++) {
 			struct io_thread_req *io_req = (*irq_req_buffer)[count];
 
-			if ((io_req->error == BLK_STS_NOTSUPP) && (req_op(io_req->req) == REQ_OP_DISCARD)) {
-				blk_queue_max_discard_sectors(io_req->req->q, 0);
-				blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
+			if (io_req->error == BLK_STS_NOTSUPP) {
+				struct request_queue *q = io_req->req->q;
+
+				if (req_op(io_req->req) == REQ_OP_DISCARD)
+					blk_queue_max_discard_sectors(q, 0);
+				if (req_op(io_req->req) == REQ_OP_WRITE_ZEROES)
+					blk_queue_max_write_zeroes_sectors(q,
+							0);
 			}
 			blk_mq_end_request(io_req->req, io_req->error);
 			kfree(io_req);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731379.1136869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVF-0003mh-Iy; Wed, 29 May 2024 05:05:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731379.1136869; Wed, 29 May 2024 05:05:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVF-0003mP-F9; Wed, 29 May 2024 05:05:33 +0000
Received: by outflank-mailman (input) for mailman id 731379;
 Wed, 29 May 2024 05:05:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVE-0002IO-8I
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:32 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 124ad58e-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:31 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBV8-00000002pZH-1Swh; Wed, 29 May 2024 05:05:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 124ad58e-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=YGmOwuIW0DehI4KXZwtqg3QuYxifQjapgz3/0XZe+RA=; b=KhFS/hxRb8UpO9uvnt4jA68rPG
	xVySn1pHD4jIRemyRbCCEdqXZl0xtjnDLGWGYQO/VqtIGw3kO7PMotHaHvYeF3y7+zrWI6JoWm7Pu
	K36zrJ5O9E7hHC8jFyqb0xPEFI8urWGQgmpxKmKvuJcI1lc4KbirfQgyMqZqdMiNOH0Qn6YwwfkIE
	fgfo+VGAtqPz4gTfBDOjANwyIRtH8WAho9J73pWTOMMk8H6xVuduHp7xwShRKISMLaacYAKT5bsRv
	hrsO1G72IRV/pFLruJ2/ijU+K3F58b+tzy0n/xqUnJBi8VNnAK04O7UkNZ4e+JaGUDKq6/ezYTnGO
	RPYgZo2w==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 06/12] sd: simplify the disable case in sd_config_discard
Date: Wed, 29 May 2024 07:04:08 +0200
Message-ID: <20240529050507.1392041-7-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Fall through to the main call to blk_queue_max_discard_sectors given that
max_blocks has been initialized to zero above instead of duplicating the
call.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 09ffe9d826aeac..437743e3a0d37d 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -844,8 +844,7 @@ static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 
 	case SD_LBP_FULL:
 	case SD_LBP_DISABLE:
-		blk_queue_max_discard_sectors(q, 0);
-		return;
+		break;
 
 	case SD_LBP_UNMAP:
 		max_blocks = min_not_zero(sdkp->max_unmap_blocks,
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731380.1136879 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVJ-0004AT-Tm; Wed, 29 May 2024 05:05:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731380.1136879; Wed, 29 May 2024 05:05:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVJ-0004AG-Po; Wed, 29 May 2024 05:05:37 +0000
Received: by outflank-mailman (input) for mailman id 731380;
 Wed, 29 May 2024 05:05:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVH-0002IO-UF
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:35 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 14852e35-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:35 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVB-00000002pbP-0kqv; Wed, 29 May 2024 05:05:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14852e35-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=zW0EkOuxIkEXAr+kUSNAXR4zW6yla1v+V3lES4DaK0U=; b=JV2Lk341hcuT/7Vl5ccbx6rEF4
	LSvvlB9GQx61zLW6Ek9rMBriZIqorh2yE6n19JvBK8pX9/bCGIKdK8ZSpyJEO6QCmV4FTj9dJx9ct
	2mmqIL366QPU6zYx5zGQSrSTnDBwzdEgAuHzVnaP8EbOoN2JW7yLhrxCvxx4q95t/vZOmQ/fs/cKy
	Hwkml4dg7O0d9si4j5qAHqQU/FbHR3FB0PuKk8zFIIWsEMD/MnhlQRi1PZcYYk0TFsmHCcsFDcNLx
	/+4Z/3TAjBBZlqT2hEMfDtWsLUrRt4XNvmIyqd0q1bw8vLAqtevVrvfYTv+SnaOzLd/GwgYDhosg1
	p2ZtR9BA==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 07/12] sd: factor out a sd_discard_mode helper
Date: Wed, 29 May 2024 07:04:09 +0200
Message-ID: <20240529050507.1392041-8-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Split the logic to pick the right discard mode into a little helper
to prepare for further changes.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 437743e3a0d37d..2d08b69154b995 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -3201,6 +3201,25 @@ static void sd_read_app_tag_own(struct scsi_disk *sdkp, unsigned char *buffer)
 	return;
 }
 
+static unsigned int sd_discard_mode(struct scsi_disk *sdkp)
+{
+	if (!sdkp->lbpvpd) {
+		/* LBP VPD page not provided */
+		if (sdkp->max_unmap_blocks)
+			return SD_LBP_UNMAP;
+		return SD_LBP_WS16;
+	}
+
+	/* LBP VPD page tells us what to use */
+	if (sdkp->lbpu && sdkp->max_unmap_blocks)
+		return SD_LBP_UNMAP;
+	if (sdkp->lbpws)
+		return SD_LBP_WS16;
+	if (sdkp->lbpws10)
+		return SD_LBP_WS10;
+	return SD_LBP_DISABLE;
+}
+
 /**
  * sd_read_block_limits - Query disk device for preferred I/O sizes.
  * @sdkp: disk to query
@@ -3239,23 +3258,7 @@ static void sd_read_block_limits(struct scsi_disk *sdkp)
 			sdkp->unmap_alignment =
 				get_unaligned_be32(&vpd->data[32]) & ~(1 << 31);
 
-		if (!sdkp->lbpvpd) { /* LBP VPD page not provided */
-
-			if (sdkp->max_unmap_blocks)
-				sd_config_discard(sdkp, SD_LBP_UNMAP);
-			else
-				sd_config_discard(sdkp, SD_LBP_WS16);
-
-		} else {	/* LBP VPD page tells us what to use */
-			if (sdkp->lbpu && sdkp->max_unmap_blocks)
-				sd_config_discard(sdkp, SD_LBP_UNMAP);
-			else if (sdkp->lbpws)
-				sd_config_discard(sdkp, SD_LBP_WS16);
-			else if (sdkp->lbpws10)
-				sd_config_discard(sdkp, SD_LBP_WS10);
-			else
-				sd_config_discard(sdkp, SD_LBP_DISABLE);
-		}
+		sd_config_discard(sdkp, sd_discard_mode(sdkp));
 	}
 
  out:
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731382.1136889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVN-0004dV-5k; Wed, 29 May 2024 05:05:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731382.1136889; Wed, 29 May 2024 05:05:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVN-0004dG-1y; Wed, 29 May 2024 05:05:41 +0000
Received: by outflank-mailman (input) for mailman id 731382;
 Wed, 29 May 2024 05:05:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVL-0002IP-Bf
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:39 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15fa2b76-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:37 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVE-00000002pch-0ecf; Wed, 29 May 2024 05:05:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15fa2b76-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=FxMZ/TuFQWi8ppJCXfgMd7wJRUIIOyQbd6+WyEUQ+N4=; b=sP8WPUo5vAWKUtYTFsHH8zz4lY
	DJJMR33fYr4eAsLHFTZ8Z9howNp0W6fwZ2fiNjcxUBC3ODEyO2i6vzUcK33pr0IzzNeqbE0OGj1ST
	5S7X+/6iNra2QHh6wsNSXPnXFeKcQpoKWDT2h0ez391hq4EcHyXFSepiXhTUC3N2xV1AZcowiOldm
	4EXlw2b1iSu+BV1NPRisPBN1MDDygrM+eCbEpyBDZ0QTDVC2nwgktvILS31bY5S++5+So+r6x+Kgv
	BwpsZkqpUIOpxNp0XUC+iXoyBN5nUfj9Zb4EGtzt7UNxwaRV/VE3oyhLo7REgxmsgSTIrs8QWAfko
	q33GTzcg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 08/12] sd: cleanup zoned queue limits initialization
Date: Wed, 29 May 2024 07:04:10 +0200
Message-ID: <20240529050507.1392041-9-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Consolidate setting zone-related queue limits in sd_zbc_read_zones
instead of splitting them between sd_zbc_revalidate_zones and
sd_zbc_read_zones, and move the early_zone_information initialization
in sd_zbc_read_zones above setting up the queue limits.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd_zbc.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/sd_zbc.c b/drivers/scsi/sd_zbc.c
index 806036e48abeda..1c24c844e8d178 100644
--- a/drivers/scsi/sd_zbc.c
+++ b/drivers/scsi/sd_zbc.c
@@ -565,12 +565,6 @@ int sd_zbc_revalidate_zones(struct scsi_disk *sdkp)
 	sdkp->zone_info.zone_blocks = zone_blocks;
 	sdkp->zone_info.nr_zones = nr_zones;
 
-	blk_queue_chunk_sectors(q,
-			logical_to_sectors(sdkp->device, zone_blocks));
-
-	/* Enable block layer zone append emulation */
-	blk_queue_max_zone_append_sectors(q, 0);
-
 	flags = memalloc_noio_save();
 	ret = blk_revalidate_disk_zones(disk);
 	memalloc_noio_restore(flags);
@@ -625,6 +619,10 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	if (ret != 0)
 		goto err;
 
+	nr_zones = round_up(sdkp->capacity, zone_blocks) >> ilog2(zone_blocks);
+	sdkp->early_zone_info.nr_zones = nr_zones;
+	sdkp->early_zone_info.zone_blocks = zone_blocks;
+
 	/* The drive satisfies the kernel restrictions: set it up */
 	blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q);
 	if (sdkp->zones_max_open == U32_MAX)
@@ -632,10 +630,10 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	else
 		disk_set_max_open_zones(disk, sdkp->zones_max_open);
 	disk_set_max_active_zones(disk, 0);
-	nr_zones = round_up(sdkp->capacity, zone_blocks) >> ilog2(zone_blocks);
-
-	sdkp->early_zone_info.nr_zones = nr_zones;
-	sdkp->early_zone_info.zone_blocks = zone_blocks;
+	blk_queue_chunk_sectors(q,
+			logical_to_sectors(sdkp->device, zone_blocks));
+	/* Enable block layer zone append emulation */
+	blk_queue_max_zone_append_sectors(q, 0);
 
 	return 0;
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731389.1136899 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVQ-0005Fl-MN; Wed, 29 May 2024 05:05:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731389.1136899; Wed, 29 May 2024 05:05:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVQ-0005F4-II; Wed, 29 May 2024 05:05:44 +0000
Received: by outflank-mailman (input) for mailman id 731389;
 Wed, 29 May 2024 05:05:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVP-0002IP-0n
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:43 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 17abc142-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:40 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVG-00000002pdu-3MWe; Wed, 29 May 2024 05:05:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17abc142-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=g8y7nq0IaspBzIVHof01sJaQZdVu+uaVxTj/xqPJYAg=; b=vGmbn0LG7XcNXsNzw/dWRxONf6
	g/0GjIDqzMKQjUxmnfwq/WFkBUpWhxR0shgcCxQv5xgxgu5RR8pj77YjAIFDYNAXiT41EPupr6keD
	BdtWbJRaw6/8YtA27IgXMe5PClk0cX2nXIDVJeHlH7jOldDFnQZHeYkKvsl3GdcRLZRgK5VozGQ1j
	tmqBCOXktUhci0x7aeBv0dyc1uAHHNmvhyEIIe/uo55/UpCGUcOuvzhawZ+2DS0d46Iob187PUtDv
	t4nlOEEtz/1BA8tWGcbVqnU04hmZTaLYic/SUs745VGldtP1Zu2wEz3/i4b0y6DsQfpJiYxa+C5To
	U9Uphqsg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 09/12] sd: convert to the atomic queue limits API
Date: Wed, 29 May 2024 07:04:11 +0200
Message-ID: <20240529050507.1392041-10-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Assign all queue limits through a local queue_limits variable and
queue_limits_commit_update so that we can't race updating them from
multiple places, and free the queue when updating them so that
in-progress I/O submissions don't see half-updated limits.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sd.c     | 126 ++++++++++++++++++++++++------------------
 drivers/scsi/sd.h     |   6 +-
 drivers/scsi/sd_zbc.c |  15 ++---
 3 files changed, 84 insertions(+), 63 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 2d08b69154b995..03e67936b27928 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -101,12 +101,13 @@ MODULE_ALIAS_SCSI_DEVICE(TYPE_ZBC);
 
 #define SD_MINORS	16
 
-static void sd_config_discard(struct scsi_disk *, unsigned int);
-static void sd_config_write_same(struct scsi_disk *);
+static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned int mode);
+static void sd_config_write_same(struct scsi_disk *sdkp,
+		struct queue_limits *lim);
 static int  sd_revalidate_disk(struct gendisk *);
 static void sd_unlock_native_capacity(struct gendisk *disk);
 static void sd_shutdown(struct device *);
-static void sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer);
 static void scsi_disk_release(struct device *cdev);
 
 static DEFINE_IDA(sd_index_ida);
@@ -456,7 +457,8 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 {
 	struct scsi_disk *sdkp = to_scsi_disk(dev);
 	struct scsi_device *sdp = sdkp->device;
-	int mode;
+	struct queue_limits lim;
+	int mode, err;
 
 	if (!capable(CAP_SYS_ADMIN))
 		return -EACCES;
@@ -472,8 +474,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 	if (mode < 0)
 		return -EINVAL;
 
-	sd_config_discard(sdkp, mode);
-
+	lim = queue_limits_start_update(sdkp->disk->queue);
+	sd_config_discard(sdkp, &lim, mode);
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
 	return count;
 }
 static DEVICE_ATTR_RW(provisioning_mode);
@@ -556,6 +563,7 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
 {
 	struct scsi_disk *sdkp = to_scsi_disk(dev);
 	struct scsi_device *sdp = sdkp->device;
+	struct queue_limits lim;
 	unsigned long max;
 	int err;
 
@@ -577,8 +585,13 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
 		sdkp->max_ws_blocks = max;
 	}
 
-	sd_config_write_same(sdkp);
-
+	lim = queue_limits_start_update(sdkp->disk->queue);
+	sd_config_write_same(sdkp, &lim);
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
 	return count;
 }
 static DEVICE_ATTR_RW(max_write_same_blocks);
@@ -827,17 +840,15 @@ static void sd_disable_discard(struct scsi_disk *sdkp)
 	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
 }
 
-static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
+static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned int mode)
 {
-	struct request_queue *q = sdkp->disk->queue;
 	unsigned int logical_block_size = sdkp->device->sector_size;
 	unsigned int max_blocks = 0;
 
-	q->limits.discard_alignment =
-		sdkp->unmap_alignment * logical_block_size;
-	q->limits.discard_granularity =
-		max(sdkp->physical_block_size,
-		    sdkp->unmap_granularity * logical_block_size);
+	lim->discard_alignment = sdkp->unmap_alignment * logical_block_size;
+	lim->discard_granularity = max(sdkp->physical_block_size,
+			sdkp->unmap_granularity * logical_block_size);
 	sdkp->provisioning_mode = mode;
 
 	switch (mode) {
@@ -875,7 +886,8 @@ static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 		break;
 	}
 
-	blk_queue_max_discard_sectors(q, max_blocks * (logical_block_size >> 9));
+	lim->max_hw_discard_sectors = max_blocks *
+		(logical_block_size >> SECTOR_SHIFT);
 }
 
 static void *sd_set_special_bvec(struct request *rq, unsigned int data_len)
@@ -1010,9 +1022,9 @@ static void sd_disable_write_same(struct scsi_disk *sdkp)
 	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
 }
 
-static void sd_config_write_same(struct scsi_disk *sdkp)
+static void sd_config_write_same(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
-	struct request_queue *q = sdkp->disk->queue;
 	unsigned int logical_block_size = sdkp->device->sector_size;
 
 	if (sdkp->device->no_write_same) {
@@ -1066,8 +1078,8 @@ static void sd_config_write_same(struct scsi_disk *sdkp)
 	}
 
 out:
-	blk_queue_max_write_zeroes_sectors(q, sdkp->max_ws_blocks *
-					 (logical_block_size >> 9));
+	lim->max_write_zeroes_sectors =
+		sdkp->max_ws_blocks * (logical_block_size >> 9);
 }
 
 static blk_status_t sd_setup_flush_cmnd(struct scsi_cmnd *cmd)
@@ -2523,7 +2535,7 @@ static void read_capacity_error(struct scsi_disk *sdkp, struct scsi_device *sdp,
 #define READ_CAPACITY_RETRIES_ON_RESET	10
 
 static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
-						unsigned char *buffer)
+		struct queue_limits *lim, unsigned char *buffer)
 {
 	unsigned char cmd[16];
 	struct scsi_sense_hdr sshdr;
@@ -2597,7 +2609,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
 
 	/* Lowest aligned logical block */
 	alignment = ((buffer[14] & 0x3f) << 8 | buffer[15]) * sector_size;
-	blk_queue_alignment_offset(sdp->request_queue, alignment);
+	lim->alignment_offset = alignment;
 	if (alignment && sdkp->first_scan)
 		sd_printk(KERN_NOTICE, sdkp,
 			  "physical block alignment offset: %u\n", alignment);
@@ -2608,7 +2620,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
 		if (buffer[14] & 0x40) /* LBPRZ */
 			sdkp->lbprz = 1;
 
-		sd_config_discard(sdkp, SD_LBP_WS16);
+		sd_config_discard(sdkp, lim, SD_LBP_WS16);
 	}
 
 	sdkp->capacity = lba + 1;
@@ -2711,13 +2723,14 @@ static int sd_try_rc16_first(struct scsi_device *sdp)
  * read disk capacity
  */
 static void
-sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
+sd_read_capacity(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned char *buffer)
 {
 	int sector_size;
 	struct scsi_device *sdp = sdkp->device;
 
 	if (sd_try_rc16_first(sdp)) {
-		sector_size = read_capacity_16(sdkp, sdp, buffer);
+		sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
 		if (sector_size == -EOVERFLOW)
 			goto got_data;
 		if (sector_size == -ENODEV)
@@ -2737,7 +2750,7 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
 			int old_sector_size = sector_size;
 			sd_printk(KERN_NOTICE, sdkp, "Very big device. "
 					"Trying to use READ CAPACITY(16).\n");
-			sector_size = read_capacity_16(sdkp, sdp, buffer);
+			sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
 			if (sector_size < 0) {
 				sd_printk(KERN_NOTICE, sdkp,
 					"Using 0xffffffff as device size\n");
@@ -2796,9 +2809,8 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
 		 */
 		sector_size = 512;
 	}
-	blk_queue_logical_block_size(sdp->request_queue, sector_size);
-	blk_queue_physical_block_size(sdp->request_queue,
-				      sdkp->physical_block_size);
+	lim->logical_block_size = sector_size;
+	lim->physical_block_size = sdkp->physical_block_size;
 	sdkp->device->sector_size = sector_size;
 
 	if (sdkp->capacity > 0xffffffff)
@@ -3220,11 +3232,11 @@ static unsigned int sd_discard_mode(struct scsi_disk *sdkp)
 	return SD_LBP_DISABLE;
 }
 
-/**
- * sd_read_block_limits - Query disk device for preferred I/O sizes.
- * @sdkp: disk to query
+/*
+ * Query disk device for preferred I/O sizes.
  */
-static void sd_read_block_limits(struct scsi_disk *sdkp)
+static void sd_read_block_limits(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
 	struct scsi_vpd *vpd;
 
@@ -3258,7 +3270,7 @@ static void sd_read_block_limits(struct scsi_disk *sdkp)
 			sdkp->unmap_alignment =
 				get_unaligned_be32(&vpd->data[32]) & ~(1 << 31);
 
-		sd_config_discard(sdkp, sd_discard_mode(sdkp));
+		sd_config_discard(sdkp, lim, sd_discard_mode(sdkp));
 	}
 
  out:
@@ -3278,10 +3290,10 @@ static void sd_read_block_limits_ext(struct scsi_disk *sdkp)
 }
 
 /**
- * sd_read_block_characteristics - Query block dev. characteristics
- * @sdkp: disk to query
+ * Query block dev. characteristics
  */
-static void sd_read_block_characteristics(struct scsi_disk *sdkp)
+static void sd_read_block_characteristics(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
 	struct request_queue *q = sdkp->disk->queue;
 	struct scsi_vpd *vpd;
@@ -3307,29 +3319,26 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 
 #ifdef CONFIG_BLK_DEV_ZONED /* sd_probe rejects ZBD devices early otherwise */
 	if (sdkp->device->type == TYPE_ZBC) {
-		/*
-		 * Host-managed.
-		 */
-		disk_set_zoned(sdkp->disk);
+		lim->zoned = true;
 
 		/*
 		 * Per ZBC and ZAC specifications, writes in sequential write
 		 * required zones of host-managed devices must be aligned to
 		 * the device physical block size.
 		 */
-		blk_queue_zone_write_granularity(q, sdkp->physical_block_size);
+		lim->zone_write_granularity = sdkp->physical_block_size;
 	} else {
 		/*
 		 * Host-aware devices are treated as conventional.
 		 */
-		WARN_ON_ONCE(blk_queue_is_zoned(q));
+		lim->zoned = false;
 	}
 #endif /* CONFIG_BLK_DEV_ZONED */
 
 	if (!sdkp->first_scan)
 		return;
 
-	if (blk_queue_is_zoned(q))
+	if (lim->zoned)
 		sd_printk(KERN_NOTICE, sdkp, "Host-managed zoned block device\n");
 	else if (sdkp->zoned == 1)
 		sd_printk(KERN_NOTICE, sdkp, "Host-aware SMR disk used as regular disk\n");
@@ -3605,8 +3614,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	struct scsi_device *sdp = sdkp->device;
 	struct request_queue *q = sdkp->disk->queue;
 	sector_t old_capacity = sdkp->capacity;
+	struct queue_limits lim;
 	unsigned char *buffer;
 	unsigned int dev_max;
+	int err;
 
 	SCSI_LOG_HLQUEUE(3, sd_printk(KERN_INFO, sdkp,
 				      "sd_revalidate_disk\n"));
@@ -3627,12 +3638,14 @@ static int sd_revalidate_disk(struct gendisk *disk)
 
 	sd_spinup_disk(sdkp);
 
+	lim = queue_limits_start_update(sdkp->disk->queue);
+
 	/*
 	 * Without media there is no reason to ask; moreover, some devices
 	 * react badly if we do.
 	 */
 	if (sdkp->media_present) {
-		sd_read_capacity(sdkp, buffer);
+		sd_read_capacity(sdkp, &lim, buffer);
 		/*
 		 * Some USB/UAS devices return generic values for mode pages
 		 * until the media has been accessed. Trigger a READ operation
@@ -3651,10 +3664,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
 
 		if (scsi_device_supports_vpd(sdp)) {
 			sd_read_block_provisioning(sdkp);
-			sd_read_block_limits(sdkp);
+			sd_read_block_limits(sdkp, &lim);
 			sd_read_block_limits_ext(sdkp);
-			sd_read_block_characteristics(sdkp);
-			sd_zbc_read_zones(sdkp, buffer);
+			sd_read_block_characteristics(sdkp, &lim);
+			sd_zbc_read_zones(sdkp, &lim, buffer);
 			sd_read_cpr(sdkp);
 		}
 
@@ -3683,28 +3696,33 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	q->limits.max_dev_sectors = logical_to_sectors(sdp, dev_max);
 
 	if (sd_validate_min_xfer_size(sdkp))
-		blk_queue_io_min(sdkp->disk->queue,
-				 logical_to_bytes(sdp, sdkp->min_xfer_blocks));
+		lim.io_min = logical_to_bytes(sdp, sdkp->min_xfer_blocks);
 	else
-		blk_queue_io_min(sdkp->disk->queue, 0);
+		lim.io_min = 0;
 
 	/*
 	 * Limit default to SCSI host optimal sector limit if set. There may be
 	 * an impact on performance for when the size of a request exceeds this
 	 * host limit.
 	 */
-	q->limits.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
+	lim.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
 	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
-		q->limits.io_opt = min_not_zero(q->limits.io_opt,
+		lim.io_opt = min_not_zero(lim.io_opt,
 				logical_to_bytes(sdp, sdkp->opt_xfer_blocks));
 	}
 
 	sdkp->first_scan = 0;
 
 	set_capacity_and_notify(disk, logical_to_sectors(sdp, sdkp->capacity));
-	sd_config_write_same(sdkp);
+	sd_config_write_same(sdkp, &lim);
 	kfree(buffer);
 
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
+
 	/*
 	 * For a zoned drive, revalidating the zones can be done only once
 	 * the gendisk capacity is set. So if this fails, set back the gendisk
diff --git a/drivers/scsi/sd.h b/drivers/scsi/sd.h
index 49dd600bfa4825..b4170b17bad47a 100644
--- a/drivers/scsi/sd.h
+++ b/drivers/scsi/sd.h
@@ -239,7 +239,8 @@ static inline int sd_is_zoned(struct scsi_disk *sdkp)
 
 #ifdef CONFIG_BLK_DEV_ZONED
 
-int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE]);
+int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
+		u8 buf[SD_BUF_SIZE]);
 int sd_zbc_revalidate_zones(struct scsi_disk *sdkp);
 blk_status_t sd_zbc_setup_zone_mgmt_cmnd(struct scsi_cmnd *cmd,
 					 unsigned char op, bool all);
@@ -250,7 +251,8 @@ int sd_zbc_report_zones(struct gendisk *disk, sector_t sector,
 
 #else /* CONFIG_BLK_DEV_ZONED */
 
-static inline int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
+static inline int sd_zbc_read_zones(struct scsi_disk *sdkp,
+		struct queue_limits *lim, u8 buf[SD_BUF_SIZE])
 {
 	return 0;
 }
diff --git a/drivers/scsi/sd_zbc.c b/drivers/scsi/sd_zbc.c
index 1c24c844e8d178..f685838d9ed214 100644
--- a/drivers/scsi/sd_zbc.c
+++ b/drivers/scsi/sd_zbc.c
@@ -582,13 +582,15 @@ int sd_zbc_revalidate_zones(struct scsi_disk *sdkp)
 /**
  * sd_zbc_read_zones - Read zone information and update the request queue
  * @sdkp: SCSI disk pointer.
+ * @lim: queue limits to read into
  * @buf: 512 byte buffer used for storing SCSI command output.
  *
  * Read zone information and update the request queue zone characteristics and
  * also the zoned device information in *sdkp. Called by sd_revalidate_disk()
  * before the gendisk capacity has been set.
  */
-int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
+int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
+		u8 buf[SD_BUF_SIZE])
 {
 	struct gendisk *disk = sdkp->disk;
 	struct request_queue *q = disk->queue;
@@ -626,14 +628,13 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	/* The drive satisfies the kernel restrictions: set it up */
 	blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q);
 	if (sdkp->zones_max_open == U32_MAX)
-		disk_set_max_open_zones(disk, 0);
+		lim->max_open_zones = 0;
 	else
-		disk_set_max_open_zones(disk, sdkp->zones_max_open);
-	disk_set_max_active_zones(disk, 0);
-	blk_queue_chunk_sectors(q,
-			logical_to_sectors(sdkp->device, zone_blocks));
+		lim->max_open_zones = sdkp->zones_max_open;
+	lim->max_active_zones = 0;
+	lim->chunk_sectors = logical_to_sectors(sdkp->device, zone_blocks);
 	/* Enable block layer zone append emulation */
-	blk_queue_max_zone_append_sectors(q, 0);
+	lim->max_zone_append_sectors = 0;
 
 	return 0;
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731390.1136902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVR-0005Ly-3o; Wed, 29 May 2024 05:05:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731390.1136902; Wed, 29 May 2024 05:05:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVQ-0005Jw-Tn; Wed, 29 May 2024 05:05:44 +0000
Received: by outflank-mailman (input) for mailman id 731390;
 Wed, 29 May 2024 05:05:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVP-0002IO-NG
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:43 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 192cf4dd-1d79-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 07:05:43 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVJ-00000002pg5-2kL2; Wed, 29 May 2024 05:05:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 192cf4dd-1d79-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=/Z/UqEGIRF/2yhrzEkv4fi5cgTRBRNNGaBlYe7kzoKc=; b=Iq1aE57gCD8tq2J5nqhiiNOVLi
	+9Gv/+7KO17t2hAhGk9p5PeCiJyWZdYMY9koayODJf7+agr2mys30WFPAAA8+njrkjyL303aWHMiW
	TI7STHXoQt+Adq0bJl7LZizfyHd4UwLeOoZ3GkgKnWXZK7sgFTJbgkKQaAyw2vVAvCY+8udFmvYaz
	iiNOfZ6SKFo8bxw+xsRES4+zEPU7qX95BZ17rMF5gaxth7wi2wDzCKE1j7o1UdpFke8cNXQ0Nr/O6
	Q2v5InhPKQ2so789ubxFzqPUkMwAs3EUyf21+GLKe3c3cO2hjaWH+bix77Rr3HW9LSBKviGvOKLoG
	SVTH+agg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 10/12] sr: convert to the atomic queue limits API
Date: Wed, 29 May 2024 07:04:12 +0200
Message-ID: <20240529050507.1392041-11-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Assign all queue limits through a local queue_limits variable and
queue_limits_commit_update so that we can't race updating them from
multiple places, and free the queue when updating them so that
in-progress I/O submissions don't see half-updated limits.

Also use the chance to clean up variable names to standard ones.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/scsi/sr.c | 42 +++++++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 17 deletions(-)

diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c
index 7ab000942b97fc..3f491019103e0c 100644
--- a/drivers/scsi/sr.c
+++ b/drivers/scsi/sr.c
@@ -111,7 +111,7 @@ static struct lock_class_key sr_bio_compl_lkclass;
 static int sr_open(struct cdrom_device_info *, int);
 static void sr_release(struct cdrom_device_info *);
 
-static void get_sectorsize(struct scsi_cd *);
+static int get_sectorsize(struct scsi_cd *);
 static int get_capabilities(struct scsi_cd *);
 
 static unsigned int sr_check_events(struct cdrom_device_info *cdi,
@@ -473,15 +473,15 @@ static blk_status_t sr_init_command(struct scsi_cmnd *SCpnt)
 	return BLK_STS_IOERR;
 }
 
-static void sr_revalidate_disk(struct scsi_cd *cd)
+static int sr_revalidate_disk(struct scsi_cd *cd)
 {
 	struct scsi_sense_hdr sshdr;
 
 	/* if the unit is not ready, nothing more to do */
 	if (scsi_test_unit_ready(cd->device, SR_TIMEOUT, MAX_RETRIES, &sshdr))
-		return;
+		return 0;
 	sr_cd_check(&cd->cdi);
-	get_sectorsize(cd);
+	return get_sectorsize(cd);
 }
 
 static int sr_block_open(struct gendisk *disk, blk_mode_t mode)
@@ -494,13 +494,16 @@ static int sr_block_open(struct gendisk *disk, blk_mode_t mode)
 		return -ENXIO;
 
 	scsi_autopm_get_device(sdev);
-	if (disk_check_media_change(disk))
-		sr_revalidate_disk(cd);
+	if (disk_check_media_change(disk)) {
+		ret = sr_revalidate_disk(cd);
+		if (ret)
+			goto out;
+	}
 
 	mutex_lock(&cd->lock);
 	ret = cdrom_open(&cd->cdi, mode);
 	mutex_unlock(&cd->lock);
-
+out:
 	scsi_autopm_put_device(sdev);
 	if (ret)
 		scsi_device_put(cd->device);
@@ -685,7 +688,9 @@ static int sr_probe(struct device *dev)
 	blk_pm_runtime_init(sdev->request_queue, dev);
 
 	dev_set_drvdata(dev, cd);
-	sr_revalidate_disk(cd);
+	error = sr_revalidate_disk(cd);
+	if (error)
+		goto unregister_cdrom;
 
 	error = device_add_disk(&sdev->sdev_gendev, disk, NULL);
 	if (error)
@@ -714,13 +719,14 @@ static int sr_probe(struct device *dev)
 }
 
 
-static void get_sectorsize(struct scsi_cd *cd)
+static int get_sectorsize(struct scsi_cd *cd)
 {
+	struct request_queue *q = cd->device->request_queue;
 	static const u8 cmd[10] = { READ_CAPACITY };
 	unsigned char buffer[8] = { };
-	int the_result;
+	struct queue_limits lim;
+	int err;
 	int sector_size;
-	struct request_queue *queue;
 	struct scsi_failure failure_defs[] = {
 		{
 			.result = SCMD_FAILURE_RESULT_ANY,
@@ -736,10 +742,10 @@ static void get_sectorsize(struct scsi_cd *cd)
 	};
 
 	/* Do the command and wait.. */
-	the_result = scsi_execute_cmd(cd->device, cmd, REQ_OP_DRV_IN, buffer,
+	err = scsi_execute_cmd(cd->device, cmd, REQ_OP_DRV_IN, buffer,
 				      sizeof(buffer), SR_TIMEOUT, MAX_RETRIES,
 				      &exec_args);
-	if (the_result) {
+	if (err) {
 		cd->capacity = 0x1fffff;
 		sector_size = 2048;	/* A guess, just in case */
 	} else {
@@ -789,10 +795,12 @@ static void get_sectorsize(struct scsi_cd *cd)
 		set_capacity(cd->disk, cd->capacity);
 	}
 
-	queue = cd->device->request_queue;
-	blk_queue_logical_block_size(queue, sector_size);
-
-	return;
+	lim = queue_limits_start_update(q);
+	lim.logical_block_size = sector_size;
+	blk_mq_freeze_queue(q);
+	err = queue_limits_commit_update(q, &lim);
+	blk_mq_unfreeze_queue(q);
+	return err;
 }
 
 static int get_capabilities(struct scsi_cd *cd)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731397.1136918 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVX-0006Mb-DZ; Wed, 29 May 2024 05:05:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731397.1136918; Wed, 29 May 2024 05:05:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVX-0006MO-Am; Wed, 29 May 2024 05:05:51 +0000
Received: by outflank-mailman (input) for mailman id 731397;
 Wed, 29 May 2024 05:05:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVV-0002IP-Uf
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:49 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1b823ee0-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:47 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVM-00000002piI-0kLX; Wed, 29 May 2024 05:05:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b823ee0-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=1TMBumfZQSCLa0EYYGLQXO2sAaRj0OeR2uXq5/vNYqI=; b=4OJ+9uzIel2bBZHKvc7/v0fiNF
	8DWZKq+syaBMlokn+JA3FU2cVK1QNsQXxBzd2kUJNk8evkXYXMw0nzNpxPewABqpvTUVfTZmrfGMF
	XrkCgBOHvXppmKg/AHpBBN2r0goP8PI58Jw3FeKUkYHfR43gmjpUdjtqzA6wbkIMFiEo4HJgJb7qv
	r2PHE8nND+mLmcjyvzv99wmH4ZG3V/+8SyQKtjDvjKEJvFNIgC1H+OaeREzkpuobPkjG8CUOSUeVa
	mZ1wtKRXN7M5JSWshpDhYlk6H+Vi8LJGAVmryoU8+ajrUFG7/Wb5NbUUCv6cb0lreEVd+S69ZWmsA
	oP6yjPjg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 11/12] block: remove unused queue limits API
Date: Wed, 29 May 2024 07:04:13 +0200
Message-ID: <20240529050507.1392041-12-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Remove all APIs that are unused now that sd and sr have been converted
to the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/blk-settings.c   | 190 -----------------------------------------
 include/linux/blkdev.h |  12 ---
 2 files changed, 202 deletions(-)

diff --git a/block/blk-settings.c b/block/blk-settings.c
index a49abdb3554834..0b038729608f4b 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -293,24 +293,6 @@ int queue_limits_set(struct request_queue *q, struct queue_limits *lim)
 }
 EXPORT_SYMBOL_GPL(queue_limits_set);
 
-/**
- * blk_queue_chunk_sectors - set size of the chunk for this queue
- * @q:  the request queue for the device
- * @chunk_sectors:  chunk sectors in the usual 512b unit
- *
- * Description:
- *    If a driver doesn't want IOs to cross a given chunk size, it can set
- *    this limit and prevent merging across chunks. Note that the block layer
- *    must accept a page worth of data at any offset. So if the crossing of
- *    chunks is a hard limitation in the driver, it must still be prepared
- *    to split single page bios.
- **/
-void blk_queue_chunk_sectors(struct request_queue *q, unsigned int chunk_sectors)
-{
-	q->limits.chunk_sectors = chunk_sectors;
-}
-EXPORT_SYMBOL(blk_queue_chunk_sectors);
-
 /**
  * blk_queue_max_discard_sectors - set max sectors for a single discard
  * @q:  the request queue for the device
@@ -352,139 +334,6 @@ void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
 }
 EXPORT_SYMBOL(blk_queue_max_write_zeroes_sectors);
 
-/**
- * blk_queue_max_zone_append_sectors - set max sectors for a single zone append
- * @q:  the request queue for the device
- * @max_zone_append_sectors: maximum number of sectors to write per command
- *
- * Sets the maximum number of sectors allowed for zone append commands. If
- * Specifying 0 for @max_zone_append_sectors indicates that the queue does
- * not natively support zone append operations and that the block layer must
- * emulate these operations using regular writes.
- **/
-void blk_queue_max_zone_append_sectors(struct request_queue *q,
-		unsigned int max_zone_append_sectors)
-{
-	unsigned int max_sectors = 0;
-
-	if (WARN_ON(!blk_queue_is_zoned(q)))
-		return;
-
-	if (max_zone_append_sectors) {
-		max_sectors = min(q->limits.max_hw_sectors,
-				  max_zone_append_sectors);
-		max_sectors = min(q->limits.chunk_sectors, max_sectors);
-
-		/*
-		 * Signal eventual driver bugs resulting in the max_zone_append
-		 * sectors limit being 0 due to the chunk_sectors limit (zone
-		 * size) not set or the max_hw_sectors limit not set.
-		 */
-		WARN_ON_ONCE(!max_sectors);
-	}
-
-	q->limits.max_zone_append_sectors = max_sectors;
-}
-EXPORT_SYMBOL_GPL(blk_queue_max_zone_append_sectors);
-
-/**
- * blk_queue_logical_block_size - set logical block size for the queue
- * @q:  the request queue for the device
- * @size:  the logical block size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible block size that the
- *   storage device can address.  The default of 512 covers most
- *   hardware.
- **/
-void blk_queue_logical_block_size(struct request_queue *q, unsigned int size)
-{
-	struct queue_limits *limits = &q->limits;
-
-	limits->logical_block_size = size;
-
-	if (limits->discard_granularity < limits->logical_block_size)
-		limits->discard_granularity = limits->logical_block_size;
-
-	if (limits->physical_block_size < size)
-		limits->physical_block_size = size;
-
-	if (limits->io_min < limits->physical_block_size)
-		limits->io_min = limits->physical_block_size;
-
-	limits->max_hw_sectors =
-		round_down(limits->max_hw_sectors, size >> SECTOR_SHIFT);
-	limits->max_sectors =
-		round_down(limits->max_sectors, size >> SECTOR_SHIFT);
-}
-EXPORT_SYMBOL(blk_queue_logical_block_size);
-
-/**
- * blk_queue_physical_block_size - set physical block size for the queue
- * @q:  the request queue for the device
- * @size:  the physical block size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible sector size that the
- *   hardware can operate on without reverting to read-modify-write
- *   operations.
- */
-void blk_queue_physical_block_size(struct request_queue *q, unsigned int size)
-{
-	q->limits.physical_block_size = size;
-
-	if (q->limits.physical_block_size < q->limits.logical_block_size)
-		q->limits.physical_block_size = q->limits.logical_block_size;
-
-	if (q->limits.discard_granularity < q->limits.physical_block_size)
-		q->limits.discard_granularity = q->limits.physical_block_size;
-
-	if (q->limits.io_min < q->limits.physical_block_size)
-		q->limits.io_min = q->limits.physical_block_size;
-}
-EXPORT_SYMBOL(blk_queue_physical_block_size);
-
-/**
- * blk_queue_zone_write_granularity - set zone write granularity for the queue
- * @q:  the request queue for the zoned device
- * @size:  the zone write granularity size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible size allowing to write in
- *   sequential zones of a zoned block device.
- */
-void blk_queue_zone_write_granularity(struct request_queue *q,
-				      unsigned int size)
-{
-	if (WARN_ON_ONCE(!blk_queue_is_zoned(q)))
-		return;
-
-	q->limits.zone_write_granularity = size;
-
-	if (q->limits.zone_write_granularity < q->limits.logical_block_size)
-		q->limits.zone_write_granularity = q->limits.logical_block_size;
-}
-EXPORT_SYMBOL_GPL(blk_queue_zone_write_granularity);
-
-/**
- * blk_queue_alignment_offset - set physical block alignment offset
- * @q:	the request queue for the device
- * @offset: alignment offset in bytes
- *
- * Description:
- *   Some devices are naturally misaligned to compensate for things like
- *   the legacy DOS partition table 63-sector offset.  Low-level drivers
- *   should call this function for devices whose first sector is not
- *   naturally aligned.
- */
-void blk_queue_alignment_offset(struct request_queue *q, unsigned int offset)
-{
-	q->limits.alignment_offset =
-		offset & (q->limits.physical_block_size - 1);
-	q->limits.misaligned = 0;
-}
-EXPORT_SYMBOL(blk_queue_alignment_offset);
-
 void disk_update_readahead(struct gendisk *disk)
 {
 	blk_apply_bdi_limits(disk->bdi, &disk->queue->limits);
@@ -514,26 +363,6 @@ void blk_limits_io_min(struct queue_limits *limits, unsigned int min)
 }
 EXPORT_SYMBOL(blk_limits_io_min);
 
-/**
- * blk_queue_io_min - set minimum request size for the queue
- * @q:	the request queue for the device
- * @min:  smallest I/O size in bytes
- *
- * Description:
- *   Storage devices may report a granularity or preferred minimum I/O
- *   size which is the smallest request the device can perform without
- *   incurring a performance penalty.  For disk drives this is often the
- *   physical block size.  For RAID arrays it is often the stripe chunk
- *   size.  A properly aligned multiple of minimum_io_size is the
- *   preferred request size for workloads where a high number of I/O
- *   operations is desired.
- */
-void blk_queue_io_min(struct request_queue *q, unsigned int min)
-{
-	blk_limits_io_min(&q->limits, min);
-}
-EXPORT_SYMBOL(blk_queue_io_min);
-
 /**
  * blk_limits_io_opt - set optimal request size for a device
  * @limits: the queue limits
@@ -841,25 +670,6 @@ void blk_queue_write_cache(struct request_queue *q, bool wc, bool fua)
 }
 EXPORT_SYMBOL_GPL(blk_queue_write_cache);
 
-/**
- * disk_set_zoned - inidicate a zoned device
- * @disk:	gendisk to configure
- */
-void disk_set_zoned(struct gendisk *disk)
-{
-	struct request_queue *q = disk->queue;
-
-	WARN_ON_ONCE(!IS_ENABLED(CONFIG_BLK_DEV_ZONED));
-
-	/*
-	 * Set the zone write granularity to the device logical block
-	 * size by default. The driver can change this value if needed.
-	 */
-	q->limits.zoned = true;
-	blk_queue_zone_write_granularity(q, queue_logical_block_size(q));
-}
-EXPORT_SYMBOL_GPL(disk_set_zoned);
-
 int bdev_alignment_offset(struct block_device *bdev)
 {
 	struct request_queue *q = bdev_get_queue(bdev);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index aefdda9f4ec711..bee71deb8ca066 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -331,8 +331,6 @@ struct queue_limits {
 typedef int (*report_zones_cb)(struct blk_zone *zone, unsigned int idx,
 			       void *data);
 
-void disk_set_zoned(struct gendisk *disk);
-
 #define BLK_ALL_ZONES  ((unsigned int)-1)
 int blkdev_report_zones(struct block_device *bdev, sector_t sector,
 		unsigned int nr_zones, report_zones_cb cb, void *data);
@@ -928,24 +926,14 @@ static inline void queue_limits_cancel_update(struct request_queue *q)
 /*
  * Access functions for manipulating queue properties
  */
-extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int);
 void blk_queue_max_secure_erase_sectors(struct request_queue *q,
 		unsigned int max_sectors);
 extern void blk_queue_max_discard_sectors(struct request_queue *q,
 		unsigned int max_discard_sectors);
 extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
 		unsigned int max_write_same_sectors);
-extern void blk_queue_logical_block_size(struct request_queue *, unsigned int);
-extern void blk_queue_max_zone_append_sectors(struct request_queue *q,
-		unsigned int max_zone_append_sectors);
-extern void blk_queue_physical_block_size(struct request_queue *, unsigned int);
-void blk_queue_zone_write_granularity(struct request_queue *q,
-				      unsigned int size);
-extern void blk_queue_alignment_offset(struct request_queue *q,
-				       unsigned int alignment);
 void disk_update_readahead(struct gendisk *disk);
 extern void blk_limits_io_min(struct queue_limits *limits, unsigned int min);
-extern void blk_queue_io_min(struct request_queue *q, unsigned int min);
 extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
 extern void blk_set_queue_depth(struct request_queue *q, unsigned int depth);
 extern void blk_set_stacking_limits(struct queue_limits *lim);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:05:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:05:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731400.1136924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVY-0006TG-0S; Wed, 29 May 2024 05:05:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731400.1136924; Wed, 29 May 2024 05:05:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCBVX-0006Rw-Pw; Wed, 29 May 2024 05:05:51 +0000
Received: by outflank-mailman (input) for mailman id 731400;
 Wed, 29 May 2024 05:05:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r7nM=NA=bombadil.srs.infradead.org=BATV+c55c297d46ee5715703b+7584+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCBVW-0002IP-Ui
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 05:05:50 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c77ef8d-1d79-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 07:05:48 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-7775-b725-99f7-3344.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:7775:b725:99f7:3344] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCBVO-00000002pk9-3paf; Wed, 29 May 2024 05:05:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c77ef8d-1d79-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=8d6ngVwHNTNauOwhnmSKDDZurLIjQvxFhyRamCIijqE=; b=ReWXkSxrT/T0XfXFgzP7fcQh8P
	LoBLaZcev4afPR23+iF5PXS7kCs2z2TUVpoIyd/78ykilQnjG3jxnGKaz2qygkpvtbwzRtjvrCAYl
	kRalvqMO0n637ZnM+ujlbGRx9Uu7J/K8mx82XsU4sJ6adxwSBGxzWOrKd2C7hHkoCEYhQW7+4GHs4
	8eb0wh5wj73qBWmxsrBOfDOpKI6TsU3lSvJljEqqgtLUrlfbSgX2/FMMZqnCGSthgaaxRI+ADIMzo
	LhS0RyV7JdhFep0OEvQhfZB6LNhKVpnMk+2X11OFk5auX0QSxcd9e8fvmb2iVW/2tkg3CnDFmOfdP
	YoUBvdAw==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 12/12] block: add special APIs for run-time disabling of discard and friends
Date: Wed, 29 May 2024 07:04:14 +0200
Message-ID: <20240529050507.1392041-13-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240529050507.1392041-1-hch@lst.de>
References: <20240529050507.1392041-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

A few drivers optimistically try to support discard, write zeroes and
secure erase and disable the features from the I/O completion handler
if the hardware can't support them.  This disable can't be done using
the atomic queue limits API because the I/O completion handlers can't
take sleeping locks or freezer the queue.  Keep the existing clearing
of the relevant field to zero, but replace the old blk_queue_max_*
APIs with new disable APIs that force the value to 0.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 arch/um/drivers/ubd_kern.c   |  5 ++---
 block/blk-settings.c         | 41 ------------------------------------
 drivers/block/xen-blkfront.c |  4 ++--
 drivers/scsi/sd.c            |  4 ++--
 include/linux/blkdev.h       | 28 ++++++++++++++++++------
 5 files changed, 28 insertions(+), 54 deletions(-)

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index a79a3b7c33a647..7eae1519300fbd 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -475,10 +475,9 @@ static void ubd_handler(void)
 				struct request_queue *q = io_req->req->q;
 
 				if (req_op(io_req->req) == REQ_OP_DISCARD)
-					blk_queue_max_discard_sectors(q, 0);
+					blk_queue_disable_discard(q);
 				if (req_op(io_req->req) == REQ_OP_WRITE_ZEROES)
-					blk_queue_max_write_zeroes_sectors(q,
-							0);
+					blk_queue_disable_write_zeroes(q);
 			}
 			blk_mq_end_request(io_req->req, io_req->error);
 			kfree(io_req);
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 0b038729608f4b..996f247fc98e80 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -293,47 +293,6 @@ int queue_limits_set(struct request_queue *q, struct queue_limits *lim)
 }
 EXPORT_SYMBOL_GPL(queue_limits_set);
 
-/**
- * blk_queue_max_discard_sectors - set max sectors for a single discard
- * @q:  the request queue for the device
- * @max_discard_sectors: maximum number of sectors to discard
- **/
-void blk_queue_max_discard_sectors(struct request_queue *q,
-		unsigned int max_discard_sectors)
-{
-	struct queue_limits *lim = &q->limits;
-
-	lim->max_hw_discard_sectors = max_discard_sectors;
-	lim->max_discard_sectors =
-		min(max_discard_sectors, lim->max_user_discard_sectors);
-}
-EXPORT_SYMBOL(blk_queue_max_discard_sectors);
-
-/**
- * blk_queue_max_secure_erase_sectors - set max sectors for a secure erase
- * @q:  the request queue for the device
- * @max_sectors: maximum number of sectors to secure_erase
- **/
-void blk_queue_max_secure_erase_sectors(struct request_queue *q,
-		unsigned int max_sectors)
-{
-	q->limits.max_secure_erase_sectors = max_sectors;
-}
-EXPORT_SYMBOL(blk_queue_max_secure_erase_sectors);
-
-/**
- * blk_queue_max_write_zeroes_sectors - set max sectors for a single
- *                                      write zeroes
- * @q:  the request queue for the device
- * @max_write_zeroes_sectors: maximum number of sectors to write per command
- **/
-void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
-		unsigned int max_write_zeroes_sectors)
-{
-	q->limits.max_write_zeroes_sectors = max_write_zeroes_sectors;
-}
-EXPORT_SYMBOL(blk_queue_max_write_zeroes_sectors);
-
 void disk_update_readahead(struct gendisk *disk)
 {
 	blk_apply_bdi_limits(disk->bdi, &disk->queue->limits);
diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index fd7c0ff2139cee..9b4ec3e4908cce 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -1605,8 +1605,8 @@ static irqreturn_t blkif_interrupt(int irq, void *dev_id)
 				blkif_req(req)->error = BLK_STS_NOTSUPP;
 				info->feature_discard = 0;
 				info->feature_secdiscard = 0;
-				blk_queue_max_discard_sectors(rq, 0);
-				blk_queue_max_secure_erase_sectors(rq, 0);
+				blk_queue_disable_discard(rq);
+				blk_queue_disable_secure_erase(rq);
 			}
 			break;
 		case BLKIF_OP_FLUSH_DISKCACHE:
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 03e67936b27928..56fd523b3987a5 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -837,7 +837,7 @@ static unsigned char sd_setup_protect_cmnd(struct scsi_cmnd *scmd,
 static void sd_disable_discard(struct scsi_disk *sdkp)
 {
 	sdkp->provisioning_mode = SD_LBP_DISABLE;
-	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
+	blk_queue_disable_discard(sdkp->disk->queue);
 }
 
 static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
@@ -1019,7 +1019,7 @@ static void sd_disable_write_same(struct scsi_disk *sdkp)
 {
 	sdkp->device->no_write_same = 1;
 	sdkp->max_ws_blocks = 0;
-	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
+	blk_queue_disable_write_zeroes(sdkp->disk->queue);
 }
 
 static void sd_config_write_same(struct scsi_disk *sdkp,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index bee71deb8ca066..b83441da12456a 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -923,15 +923,31 @@ static inline void queue_limits_cancel_update(struct request_queue *q)
 	mutex_unlock(&q->limits_lock);
 }
 
+/*
+ * These helpers are for drivers that have sloppy feature negotiation and might
+ * have to disable DISCARD, WRITE_ZEROES or SECURE_DISCARD from the I/O
+ * completion handler when the device returned an indicator that the respective
+ * feature is not actually supported.  They are racy and the driver needs to
+ * cope with that.  Try to avoid this scheme if you can.
+ */
+static inline void blk_queue_disable_discard(struct request_queue *q)
+{
+	q->limits.max_discard_sectors = 0;
+}
+
+static inline void blk_queue_disable_secure_erase(struct request_queue *q)
+{
+	q->limits.max_secure_erase_sectors = 0;
+}
+
+static inline void blk_queue_disable_write_zeroes(struct request_queue *q)
+{
+	q->limits.max_write_zeroes_sectors = 0;
+}
+
 /*
  * Access functions for manipulating queue properties
  */
-void blk_queue_max_secure_erase_sectors(struct request_queue *q,
-		unsigned int max_sectors);
-extern void blk_queue_max_discard_sectors(struct request_queue *q,
-		unsigned int max_discard_sectors);
-extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
-		unsigned int max_write_same_sectors);
 void disk_update_readahead(struct gendisk *disk);
 extern void blk_limits_io_min(struct queue_limits *limits, unsigned int min);
 extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 05:54:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 05:54:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731453.1136939 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCCG7-0006Hs-B3; Wed, 29 May 2024 05:53:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731453.1136939; Wed, 29 May 2024 05:53:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCCG7-0006Hl-7r; Wed, 29 May 2024 05:53:59 +0000
Received: by outflank-mailman (input) for mailman id 731453;
 Wed, 29 May 2024 05:53:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCCG5-0006HY-JR; Wed, 29 May 2024 05:53:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCCG5-0007N2-Fb; Wed, 29 May 2024 05:53:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCCG5-0007xi-3H; Wed, 29 May 2024 05:53:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCCG5-0002nQ-2c; Wed, 29 May 2024 05:53:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=a7niEyZ3pREgmGVJDanW/NY1IrbNwYQnpfE3oxxEXDs=; b=Qzn7CHo1rW8PLneIDIJ+BATt/G
	EfDTrFyDOmQjzCpOAgGmt1QZun+6YfXfOJ88fnRRFiTvwk1vEqmui30fIPoSOVDJQfQDDR4jjs9dW
	OzhcuCpEM4VhmWlBNG9NyxpZyYBiSiSR2wDrJnpRqYZvGj0auLnKSbjHsx7G3Xz5kc+o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186175-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186175: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d93f78bfe25f695d8ffb61d110da9df293ed71b
X-Osstest-Versions-That:
    xen=96af090e33130b0bf0953f3ccab8e7a163392318
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 05:53:57 +0000

flight 186175 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186175/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186168
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186168
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186168
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186168
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186168
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186168
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2d93f78bfe25f695d8ffb61d110da9df293ed71b
baseline version:
 xen                  96af090e33130b0bf0953f3ccab8e7a163392318

Last test of basis   186168  2024-05-28 10:08:43 Z    0 days
Testing same since   186175  2024-05-28 19:08:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jason.andryuk@amd.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   96af090e33..2d93f78bfe  2d93f78bfe25f695d8ffb61d110da9df293ed71b -> master


From xen-devel-bounces@lists.xenproject.org Wed May 29 06:32:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 06:32:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731464.1136957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCCqt-0002id-1O; Wed, 29 May 2024 06:31:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731464.1136957; Wed, 29 May 2024 06:31:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCCqs-0002iW-UE; Wed, 29 May 2024 06:31:58 +0000
Received: by outflank-mailman (input) for mailman id 731464;
 Wed, 29 May 2024 06:31:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCCqr-0002iQ-98
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 06:31:57 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 24ff23fc-1d85-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 08:31:56 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a6269ad9a6fso195478466b.2
 for <xen-devel@lists.xenproject.org>; Tue, 28 May 2024 23:31:56 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda8481sm680873466b.213.2024.05.28.23.31.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 May 2024 23:31:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24ff23fc-1d85-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716964315; x=1717569115; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uRFnEJZv5mdZ6dvigi45RN52WydUBZXB2fLglmEZNLk=;
        b=OLXmgvFFMfN5RZgq6Vg6LsJ/3Mfz1mnhNDsB6UIBJr/7CjH0CxCo4YqGYFJYVnJaA4
         yw1Ix0UnUoBgb56l+Yr9iVH4isYr27zRx7fK0gPz2zJV6iZ7ipPIqyE8UCMiPlG9KiAe
         EwIJ7VTBL1YKJD12p/bZmZyOAWD/Qwqv8AdZoyKK6OW5lTGplz00wvCJhj5UCtXil5WC
         7ZBzKbrloourAWooEV53qbTjvnFK0yJhlGiBdHYbjr6KizHQDgscqsSLHdXRYA6ifbYl
         mEZZfReyELdFQiaNrWA0WlszAvjPt/zZEYqOZA+QVa2vZXWJsyVoj0Kn8/uFOrAUr3hh
         1pGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716964315; x=1717569115;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uRFnEJZv5mdZ6dvigi45RN52WydUBZXB2fLglmEZNLk=;
        b=i09slR0GReaamwTrYsI26QDTJF+0jG50a57ae13vYyFQb7URURlzLbq2t7maOJzk+m
         m7geTh26jN+H1V9QrcLtvY3FZTxlnZnxga5ki7p4TrT8YRooIRUlVqg+gytz0GHPIxBW
         W86MuAHrMxoO8/3V5s+rw5d0n8qSgIGNXGBpt+CJ6ug+6ObYPxQWeqRJp0bzZwYC3OY3
         Kdw4eXoY9C9Z8Yr3whdVfKC0IUCJj+h8OUKOUeY/fBzzdhVn96y2iARxWlitxWuEUaJA
         8mSTXeWQUqSawFphA/D6cyKdJQWniENepJL8znJz4UvykL2EUZZwU0gFRMpIEYbKgOnN
         Xt2g==
X-Forwarded-Encrypted: i=1; AJvYcCVdNiZPh9FvFaazp2TkNMN97o9KEo6IpoWRig32+jzSc3eSFuADwS6MHHPDPiXXOzKIeMt4L/fYzBBL9RgdTt/zZMiv8M9VLKAQNhYN8rY=
X-Gm-Message-State: AOJu0YwWOIow51p+iiCrCzdgMKv+xw0qQ74443MeIMI84ZlaGGSGWyZU
	q/DvHfdJc0PTWPGxTD22lJRmNMB9Rm3kio0Cj7O6l47vV3OUVd3Bpgey46vwbQ==
X-Google-Smtp-Source: AGHT+IEkVbHxiY/FoIu6Z0751qf9xqd6m5i84U0ULG3E8oTMZFoQSaJGZRHecglut0QUTTomUCHZkg==
X-Received: by 2002:a17:906:2484:b0:a59:af4c:c7de with SMTP id a640c23a62f3a-a6264f15de7mr952006666b.58.1716964315578;
        Tue, 28 May 2024 23:31:55 -0700 (PDT)
Message-ID: <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
Date: Wed, 29 May 2024 08:31:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 04:41, Chen, Jiqian wrote:
> Hi,
> On 2024/5/17 19:50, Jan Beulich wrote:
>> On 17.05.2024 13:14, Chen, Jiqian wrote:
>>> On 2024/5/17 18:51, Jan Beulich wrote:
>>>> On 17.05.2024 12:45, Chen, Jiqian wrote:
>>>>> On 2024/5/16 22:01, Jan Beulich wrote:
>>>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>>> +        if ( gsi >= nr_irqs_gsi )
>>>>>>> +        {
>>>>>>> +            ret = -EINVAL;
>>>>>>> +            break;
>>>>>>> +        }
>>>>>>> +
>>>>>>> +        if ( !irq_access_permitted(current->domain, gsi) ||
>>>>>>
>>>>>> I.e. assuming IRQ == GSI? Is that a valid assumption when any number of
>>>>>> source overrides may be surfaced by ACPI?
>>>>> All irqs smaller than nr_irqs_gsi are gsi, aren't they?
>>>>
>>>> They are, but there's not necessarily a 1:1 mapping.
>>> Oh, so do I need to add a new gsi_caps to store granted gsi?
>>
>> Probably not. You ought to be able to translate between GSI and IRQ,
>> and then continue to record in / check against IRQ permissions.
> But I found in function init_irq_data:
>     for ( irq = 0; irq < nr_irqs_gsi; irq++ )
>     {
>         int rc;
> 
>         desc = irq_to_desc(irq);
>         desc->irq = irq;
> 
>         rc = init_one_irq_desc(desc);
>         if ( rc )
>             return rc;
>     }
> Does it mean that when irq < nr_irqs_gsi, the gsi and irq is a 1:1 mapping?

No, as explained before. I also don't see how you would derive that from
the code above. "nr_irqs_gsi" describes what its name says: The number of
IRQs mapping to a (_some_) GSI. That's to tell them from the non-GSI (i.e.
mainly MSI) ones. There's no implication whatsoever on the IRQ <-> GSI
mapping.

> What's more, when using PHYSDEVOP_setup_gsi, it calls mp_register_gsi,
> and in mp_register_gsi, it uses " desc = irq_to_desc(gsi); " to get irq_desc directly.

Which may be wrong, while that wrong-ness may not have hit anyone in
practice (for reasons that would need working out).

> Combining above, can we consider "gsi == irq" when irq < nr_irqs_gsi ?

Again - no.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 06:56:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 06:56:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731471.1136966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDEa-0005bl-0O; Wed, 29 May 2024 06:56:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731471.1136966; Wed, 29 May 2024 06:56:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDEZ-0005be-U4; Wed, 29 May 2024 06:56:27 +0000
Received: by outflank-mailman (input) for mailman id 731471;
 Wed, 29 May 2024 06:56:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9Qc4=NA=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1sCDEY-0005bY-I0
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 06:56:26 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20604.outbound.protection.outlook.com
 [2a01:111:f400:7e88::604])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8fa52fae-1d88-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 08:56:24 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by CYYPR12MB8989.namprd12.prod.outlook.com (2603:10b6:930:c2::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Wed, 29 May
 2024 06:56:19 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7611.030; Wed, 29 May 2024
 06:56:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8fa52fae-1d88-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d271MOobSqiWsr2Fzvp82W1acdsVQvqe5HmcF0hbn2vWPhkZBi8M5rWjX5pw8kADAVvbCEnEjklS/Oeq1tiOYqKkK146b6WmRZZ9fuTKWnztEdKLkdKHWsd02WlxZGAYnUgJZmUlEOiPxo6+gVNrerwCGx6sSovPiuYkVlB4I/kWcMm8d478Vu2vfE93DBNJKx+7kw9osM0tPG7M57dQ13qFXZNthlECivwYQ89m2MAG+2iKAI69dACzsKQwL2NSbmEHiessDYZMyuw3/JnPWD30bn4vwTKgWtig4a8rl4T71vwsaP0f6LuEPRULucgPVG/RntZkQPZZI6KDgyRXRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=47llhVA5fJda5hOa/uPhhG+yP4DLIy+MMKnylrP/iec=;
 b=eM4eqvaTF9UWLfMNGWypJWGLrlLwWbloTzeyg2sF/QgbW8K6hGXA/4Db7GWr90I1XKmx9ooQwzg/6x5ax46fB7AR5uR/iA46+xs3U57Oqc0Cqc69IZDQ1WnF8+nvLhbUOuk08Ky1VII58Hl5/K1c+E42CzqDc8T5yKdKHCZ0NlAhn+n53qgnRXhx2U0T4zxPMLtau2D7HrUVU9XOLTqJzr3zCoqe+n89VNG22Mcd0xZgEXGieWazqGdlMvOJ2aiiV2q1FHwl2TZ0qXdnKcFznGYHKIJWeECC+o1E6yhX2g/hLAPUGWT+T2CHYjwYriU+qJYoC5xxM7NxG6b69SuAqA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=47llhVA5fJda5hOa/uPhhG+yP4DLIy+MMKnylrP/iec=;
 b=t9lAABuGXvPiU7+sHatw4GJY7qockbSfL3rvyVDwqJM/2Ntj/jQenbUe84J0Qt+/hehdbwxO851MzVmwcLQ+uSsxuvLZvBP6xP2dQC6HNEmaq/i2ZQlTAVZWS5mMiECFUKH68/sqCmg5O6QepsT7V0EdnnjqmufB0u5mO9MJCsE=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index:
 AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4D//4e1AIAAis+A//+FmgCAEsTugP//viAAABEhvwA=
Date: Wed, 29 May 2024 06:56:19 +0000
Message-ID:
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
In-Reply-To: <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7611.030)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|CYYPR12MB8989:EE_
x-ms-office365-filtering-correlation-id: 02c9b6b4-904d-42fa-6879-08dc7fac7197
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|376005|7416005|1800799015|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?cGI3ZURqVzl1VDh6TmMyTFk4MEFvNGwybXE4UDVseVRpYnF4ODRPMmdTdjU3?=
 =?utf-8?B?QnNJMmhUZ2RWdG5valJCNExVSjRMbjhoRUZMelN2Y0hUZlpOamJWZlR4dWtT?=
 =?utf-8?B?d0ZZdVQrTm9pRWE5cHUxaUI2VXdBWGM0dTNuSlFpZjNobWE5Z2F5ZjlJb05T?=
 =?utf-8?B?TlNQZk5scmlJOTBUOFNOZGJUaHlIbTBnRWIrMG4xREoyc09TQ3pVcHk1b25r?=
 =?utf-8?B?NnZTOHluWldLRlQwTWp2Y0FMOE1HYWpZa0tNZ1lpU0o3aG0xRElwSGxPUVNB?=
 =?utf-8?B?QUNNVXhJTUJxY3ltb1FITVQrWHlOZlhUaGFnc29JVitiMkhvamdvVVgrQ1ov?=
 =?utf-8?B?UTk2b2M3Vlh1YkN4NHBoNmlldU14SVVWZWd0M003VnYxaGRvam5EVDFtVTVW?=
 =?utf-8?B?OEpGaVNMVFkxVHNMK3J2VHpuYjFnWDBiajBsMHlIWllqRUlEMGtBV2ZPWlI1?=
 =?utf-8?B?S291OVRsRVBSdXRwREUza0VOdTMrckh5Rklyb3FGemZCVzJ6em9tY3JZR1Mx?=
 =?utf-8?B?eWllTTJhQ3JBdTNIVlNZTDBlSEVSVTBUSW9PYW1wK2tsR1h0ajV3U21nOFJp?=
 =?utf-8?B?dVJyenBqQVJXcm1EWkt1RktwdDlqR1ozMFk2aDVyL1VGRTNWM0tBUzVyZWVa?=
 =?utf-8?B?ejQ4aEg0N0dmb2RUYitRKyt6d01VZS9Qc2g5UFJ5WkVsVTU0UnZhV2ZrbzJX?=
 =?utf-8?B?VHpiRlZDTWF5cmNxVDRBVDlmWHNZejF6WkIycE9mWmFNWGV0N2dGSHdJMzhZ?=
 =?utf-8?B?ZXFKK0xwdlA1WGFGWW02RlJBeGdFMFBjOHdXNk5oWHV1ZnA1RTY3ZlY5R2N4?=
 =?utf-8?B?MTJYL21QR1h0Q3VnUDdySTVqNmk4Q2psVWxTNDdzcmJwT2xZZW9RUDVibEJI?=
 =?utf-8?B?QTY1RXZQTE9vOE1ZbHFUWGxUMXZMS1JDWkFFN05PcklkUXo0SDBMSithaWgy?=
 =?utf-8?B?K1JoYTV5V1BON2ZkR3NqcGc4Yy9uVmpVUy9lYlU4N3p1VjhLeGxsWVRITlVt?=
 =?utf-8?B?anRSVEdYc1ZSMjhvM0NtU0RuVjk1WkhST2k5cW9tTFN5RTBOV3ZpbVFVYTZq?=
 =?utf-8?B?c0F6VDEyY29CRTlNcWxTajg4bVZVNy82MWUvcm5TdE01VWFtY28yaVpmOS9B?=
 =?utf-8?B?WUNmT1VxVnd0RUx0VjAzZVFkUGF6N3R5Qk14SHk2YWwveGUwRG1NSVAxQ0lw?=
 =?utf-8?B?S1NIVnBTeWk1TWdHVjloME02d1BoUWord0JWUTlDWktzTnBSdEdGZmpFNGRk?=
 =?utf-8?B?NkxWb1FXd1ZCSk5wdC9KMWNaeUJxR2dVZEZaWlYwZ1NrbnE4VVllZ2FPNzZy?=
 =?utf-8?B?N3pCNktXVnpxSHMyNWtwd3UySGhyVU4vekJHVG56dGRBVEVSWk9rSVp1K3Rm?=
 =?utf-8?B?aHR6aWhpT05RdUpnTy8vcCtwc2U3WENLeW92dEo1cWhhdmN3UWVCL0duOXVC?=
 =?utf-8?B?bEtxbGcrekd1VTNaVWtwcUhlajFONDVQSktJa1gwaXM5ZGlHYjU5aFhmdkNG?=
 =?utf-8?B?NGlwUlV2Z1NkbjNIWDZwWWZDMklaK1RuaFQzbEtqOFRUbTVoL0t5dGI4M2xF?=
 =?utf-8?B?aEdBbS9pT2FIMjB3cWtkNkRlVUJxL0NTbnJYVmUrM3JxaHRhcndOdE5xTkto?=
 =?utf-8?B?U2t5Mkh5cEJuelFsUU9kRXgxelhYVThvU0ZKeElVdmpJaE1xUUJOUmt2eGFI?=
 =?utf-8?B?S1doYVJxUmdybEFxTjh5ZCtMMENRVk9EMTNhYVY3MVVLR1o2eVEyQUFLc3dk?=
 =?utf-8?B?ZFlQS211WjBHdXFPTG5WcFhxbTRWcytSeDV6dUlBR2s0b05POGZSbGtPdnIv?=
 =?utf-8?B?N1FYZHRtTkpWTnR4cUk2QT09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(7416005)(1800799015)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?WVdRb1NsNXNqZ0k3b3B3aWdNNGh4SnlTZ3BEbUdkV3lKbkRadVNBNTZOb1NB?=
 =?utf-8?B?ZklWb2o3RXdOV1pLYlljSVJBMlNMMUkxeFZxcTJncEhRaFg5YUR5MGJWVmJY?=
 =?utf-8?B?eDI4cTdaK0NXa1U5VTZmYVRId2drVkF5M0l6ME9zUG8xOEdPblUyRC9tYUF1?=
 =?utf-8?B?WVVvZ2pHdFQrYS8wWktodlNiL25Bd3JNV0tDMkM1SnNpN0t3Q1B2bjc2UGZ1?=
 =?utf-8?B?VGJ6aTZEVmJSSm9nRWNvSE5sdU5uZFRFYVJ1ZVNFNnZkVExvUC9Ga2VFS3ZK?=
 =?utf-8?B?T3RhVDkxTkVZeHpMTS8xOVRDdkphM25hSWoraDV4Vno5WHFqaDdOR2NIclU5?=
 =?utf-8?B?R0dQZk9neHU0L3dWdmtvU3FZRkJBcGwrVWE3SDlQRFlYeElBblQ3OGRBRUFH?=
 =?utf-8?B?akQwa1VIQVNDNHliWDlMVHg1UFIwdmdhanhROFdheUhnOUpFaWxZMzZMOGxh?=
 =?utf-8?B?UTB6UFlNcGw5ODdyYUZ0TWd2SUxqUHJ3M0xMYytNQ29TT2xTbWtUTFl1NUlG?=
 =?utf-8?B?V1RDYjcrQ0hmZktHczhIUE4vMVBZTEpPYSs4SXZyMFJSYlFCTThldVp6WFQy?=
 =?utf-8?B?WkFqa2tuZFpWSGJKU3dqSTcyR2lRek51N0ZrbnRPTG1lWHVkbjBFTFJIVXpB?=
 =?utf-8?B?bmRhSkNjWC9BMHg3NjNPcFJVcTA3bCtKV0dKd00xNy9qMU5IU08yelBWeU0x?=
 =?utf-8?B?eDMxbUM3eDN0UklqSnExMk8yT0k4WDRJeU52RGszbVpWeTUrQkJ1VEhsTzlx?=
 =?utf-8?B?R0FGc1hxbG1BSDZpeHdHakttL3U1V2M0cU1OM2RtQWJCTzRpcVB3MHRRVG5X?=
 =?utf-8?B?TDJSekpDVnM0dWovNjlCQys0OTJ0NmhKcXJKN1lYN0tFcFQ4ZXo1UmRmOHJW?=
 =?utf-8?B?MVJhazZveDIraFdyY0ZMV0hiU2RIZDFReHFCZFEvdXdnSFBZTytRc2RKdHpo?=
 =?utf-8?B?TEd0dGNqZ21nNXordmVOQkFSR1puQjZZWUFzMkVkWi84dkdQb3RvU3o5VlVO?=
 =?utf-8?B?dkR6ZzIvNThCM3duTjZ0c0lQZm5Hd3RjdUgvWVpzd2RVTjU2QkhOMmlsY2Vq?=
 =?utf-8?B?d2ZRSW01TjByMkpzQkdzUjJCZWt5Y1JjVWhUNXhQTTMrSUlxZzQ5dEgrNm5G?=
 =?utf-8?B?UjVxZCtWSktDbU9oU0NOL2NvYXZmVjNsRjZocE1uNjlIZ21NV0VnS3dlT3ZI?=
 =?utf-8?B?bkpEWXlESXZpK2diL2hoTXRZRGVnK3hTOUgxcFkvMVhqOEZkZGhKd2pYWEwr?=
 =?utf-8?B?VDY4Vkh5WXU5dXRncmV4d25BL1ZENDdaMFlxZFM1OFVQTUtuZDc0bzNOUzd4?=
 =?utf-8?B?akphV1BIdVBxbXBRWUc0YzR2MTloUWoweExZcFdLc3d4TVFOUHJSUy9acGND?=
 =?utf-8?B?QktTZTBxOEpGNGd0WG1TYU5JSEc4V0xQSkV6cUI4Q2Rwd0cydzE5N0hUaVY0?=
 =?utf-8?B?ZUpidHVkOW9GQS9ESXR6cWdJazhoNmxqNFZNWmt4OWpMWjVoYkpJWk5HZWVk?=
 =?utf-8?B?bUVoOTFlZllYWHRnWFV4WlBlTkFzK202bndyejg0OUhGSmZqUUNkclNLQ3pn?=
 =?utf-8?B?Yk81M1VhL0ExTFZNeWQyNkhUSmpuK0tROWVOcGRVMi9DaUlTYlhmTndjL2xi?=
 =?utf-8?B?aTFZakJNU3Z3LzRzMktCVnZTVmx5L2o5OFFJRzRYOEFTUGNTQzF6VVFKMURC?=
 =?utf-8?B?RzRWTEorc0tobnYyTFRLa2daeXI1VThXczdIeVE1ZGR6aUVzSXQ3bUFscWtt?=
 =?utf-8?B?dU5sUFlyYjFBa1FNSkpqM0hQZDRCT3lnREZjUUhwOXdDZ3Q1c21HdVF1b1l2?=
 =?utf-8?B?MnpjLzBwdDFtRWRDS2pWZVk0bGVSR1dOTE1WZ3h6N1duQzg1OE1qTlFZd1hq?=
 =?utf-8?B?M2JjOWRMd2xJR1JMUkFQQjluWVpJaWFmekJpQXRtbjgxM3hUTnhMTG1PUkI0?=
 =?utf-8?B?amdXMmI2dWpBaWR6SzhjYmFXSUNmUER5SFNMeW01bnk3bkxDVmMrZStYT3ZW?=
 =?utf-8?B?ZzJTaWp2aVdjR084QzMvMitCVGVvTUlkQ1F4MjRWa2o4VHZPY293VDdkYmVh?=
 =?utf-8?B?ZVhBOCt0SkRlbkh6aFhsWGJQbE02QnYzaXhWNlJTQldJZm00U2FUcVBVSzRW?=
 =?utf-8?Q?Udgo=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <24906CDB39577B4E8F9CD30A939A50BA@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 02c9b6b4-904d-42fa-6879-08dc7fac7197
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2024 06:56:19.6541
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: x2ZAuuHeHTXH5+oPxCaU3i3Fe3Fj+KvTscSDIK/H83hXPbMpHLy8IU+t90daCknn4RPus1eT9dUjlOajxYUBvw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8989

T24gMjAyNC81LzI5IDE0OjMxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMjkuMDUuMjAyNCAw
NDo0MSwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gSGksDQo+PiBPbiAyMDI0LzUvMTcgMTk6NTAs
IEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDE3LjA1LjIwMjQgMTM6MTQsIENoZW4sIEppcWlh
biB3cm90ZToNCj4+Pj4gT24gMjAyNC81LzE3IDE4OjUxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4+
Pj4+IE9uIDE3LjA1LjIwMjQgMTI6NDUsIENoZW4sIEppcWlhbiB3cm90ZToNCj4+Pj4+PiBPbiAy
MDI0LzUvMTYgMjI6MDEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+Pj4+PiBPbiAxNi4wNS4yMDI0
IDExOjUyLCBKaXFpYW4gQ2hlbiB3cm90ZToNCj4+Pj4+Pj4+ICsgICAgICAgIGlmICggZ3NpID49
IG5yX2lycXNfZ3NpICkNCj4+Pj4+Pj4+ICsgICAgICAgIHsNCj4+Pj4+Pj4+ICsgICAgICAgICAg
ICByZXQgPSAtRUlOVkFMOw0KPj4+Pj4+Pj4gKyAgICAgICAgICAgIGJyZWFrOw0KPj4+Pj4+Pj4g
KyAgICAgICAgfQ0KPj4+Pj4+Pj4gKw0KPj4+Pj4+Pj4gKyAgICAgICAgaWYgKCAhaXJxX2FjY2Vz
c19wZXJtaXR0ZWQoY3VycmVudC0+ZG9tYWluLCBnc2kpIHx8DQo+Pj4+Pj4+DQo+Pj4+Pj4+IEku
ZS4gYXNzdW1pbmcgSVJRID09IEdTST8gSXMgdGhhdCBhIHZhbGlkIGFzc3VtcHRpb24gd2hlbiBh
bnkgbnVtYmVyIG9mDQo+Pj4+Pj4+IHNvdXJjZSBvdmVycmlkZXMgbWF5IGJlIHN1cmZhY2VkIGJ5
IEFDUEk/DQo+Pj4+Pj4gQWxsIGlycXMgc21hbGxlciB0aGFuIG5yX2lycXNfZ3NpIGFyZSBnc2ks
IGFyZW4ndCB0aGV5Pw0KPj4+Pj4NCj4+Pj4+IFRoZXkgYXJlLCBidXQgdGhlcmUncyBub3QgbmVj
ZXNzYXJpbHkgYSAxOjEgbWFwcGluZy4NCj4+Pj4gT2gsIHNvIGRvIEkgbmVlZCB0byBhZGQgYSBu
ZXcgZ3NpX2NhcHMgdG8gc3RvcmUgZ3JhbnRlZCBnc2k/DQo+Pj4NCj4+PiBQcm9iYWJseSBub3Qu
IFlvdSBvdWdodCB0byBiZSBhYmxlIHRvIHRyYW5zbGF0ZSBiZXR3ZWVuIEdTSSBhbmQgSVJRLA0K
Pj4+IGFuZCB0aGVuIGNvbnRpbnVlIHRvIHJlY29yZCBpbiAvIGNoZWNrIGFnYWluc3QgSVJRIHBl
cm1pc3Npb25zLg0KPj4gQnV0IEkgZm91bmQgaW4gZnVuY3Rpb24gaW5pdF9pcnFfZGF0YToNCj4+
ICAgICBmb3IgKCBpcnEgPSAwOyBpcnEgPCBucl9pcnFzX2dzaTsgaXJxKysgKQ0KPj4gICAgIHsN
Cj4+ICAgICAgICAgaW50IHJjOw0KPj4NCj4+ICAgICAgICAgZGVzYyA9IGlycV90b19kZXNjKGly
cSk7DQo+PiAgICAgICAgIGRlc2MtPmlycSA9IGlycTsNCj4+DQo+PiAgICAgICAgIHJjID0gaW5p
dF9vbmVfaXJxX2Rlc2MoZGVzYyk7DQo+PiAgICAgICAgIGlmICggcmMgKQ0KPj4gICAgICAgICAg
ICAgcmV0dXJuIHJjOw0KPj4gICAgIH0NCj4+IERvZXMgaXQgbWVhbiB0aGF0IHdoZW4gaXJxIDwg
bnJfaXJxc19nc2ksIHRoZSBnc2kgYW5kIGlycSBpcyBhIDE6MSBtYXBwaW5nPw0KPiANCj4gTm8s
IGFzIGV4cGxhaW5lZCBiZWZvcmUuIEkgYWxzbyBkb24ndCBzZWUgaG93IHlvdSB3b3VsZCBkZXJp
dmUgdGhhdCBmcm9tIHRoZSBjb2RlIGFib3ZlLg0KQmVjYXVzZSBoZXJlIHNldCBkZXNjLT5pcnEg
PSBpcnEsIGFuZCBpdCBzZWVtcyB0aGVyZSBpcyBubyBvdGhlciBwbGFjZSB0byBjaGFuZ2UgdGhp
cyBkZXNjLT5pcnEsIHNvLCBnc2kgMSBpcyBjb25zaWRlcmVkIHRvIGlycSAxLg0KDQo+ICJucl9p
cnFzX2dzaSIgZGVzY3JpYmVzIHdoYXQgaXRzIG5hbWUgc2F5czogVGhlIG51bWJlciBvZg0KPiBJ
UlFzIG1hcHBpbmcgdG8gYSAoX3NvbWVfKSBHU0kuIFRoYXQncyB0byB0ZWxsIHRoZW0gZnJvbSB0
aGUgbm9uLUdTSSAoaS5lLg0KPiBtYWlubHkgTVNJKSBvbmVzLiBUaGVyZSdzIG5vIGltcGxpY2F0
aW9uIHdoYXRzb2V2ZXIgb24gdGhlIElSUSA8LT4gR1NJDQo+IG1hcHBpbmcuDQo+IA0KPj4gV2hh
dCdzIG1vcmUsIHdoZW4gdXNpbmcgUEhZU0RFVk9QX3NldHVwX2dzaSwgaXQgY2FsbHMgbXBfcmVn
aXN0ZXJfZ3NpLA0KPj4gYW5kIGluIG1wX3JlZ2lzdGVyX2dzaSwgaXQgdXNlcyAiIGRlc2MgPSBp
cnFfdG9fZGVzYyhnc2kpOyAiIHRvIGdldCBpcnFfZGVzYyBkaXJlY3RseS4NCj4gDQo+IFdoaWNo
IG1heSBiZSB3cm9uZywgd2hpbGUgdGhhdCB3cm9uZy1uZXNzIG1heSBub3QgaGF2ZSBoaXQgYW55
b25lIGluDQo+IHByYWN0aWNlIChmb3IgcmVhc29ucyB0aGF0IHdvdWxkIG5lZWQgd29ya2luZyBv
dXQpLg0KPiANCj4+IENvbWJpbmluZyBhYm92ZSwgY2FuIHdlIGNvbnNpZGVyICJnc2kgPT0gaXJx
IiB3aGVuIGlycSA8IG5yX2lycXNfZ3NpID8NCj4gDQo+IEFnYWluIC0gbm8uDQpTaW5jZSB5b3Ug
YXJlIGNlcnRhaW4gdGhhdCB0aGV5IGFyZSBub3QgZXF1YWwsIGNvdWxkIHlvdSB0ZWxsIG1lIHdo
ZXJlIHNob3cgdGhleSBhcmUgbm90IGVxdWFsIG9yIHdoZXJlIGJ1aWxkIHRoZWlyIG1hcHBpbmdz
LA0Kc28gdGhhdCBJIGNhbiBrbm93IGhvdyB0byBkbyBhIGNvbnZlcnNpb24gZ3NpIGZyb20gaXJx
Lg0KDQo+IA0KPiBKYW4NCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQpKaXFpYW4gQ2hlbi4NCg==


From xen-devel-bounces@lists.xenproject.org Wed May 29 07:10:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:10:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731478.1136977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDRo-0008Q4-6Y; Wed, 29 May 2024 07:10:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731478.1136977; Wed, 29 May 2024 07:10:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDRo-0008Px-3b; Wed, 29 May 2024 07:10:08 +0000
Received: by outflank-mailman (input) for mailman id 731478;
 Wed, 29 May 2024 07:10:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCDRn-0008Kk-59
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:10:07 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 77879a61-1d8a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:10:03 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a653972487fso11436466b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:10:03 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a63559d155csm103602966b.11.2024.05.29.00.10.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 00:10:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77879a61-1d8a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716966603; x=1717571403; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=CQaycxxtR0oc9cSmzquAVtKXqxUU6P8ER3m5NLBKLqI=;
        b=JCdt1X3p4ZTWsrQ/4P1kAzhihcv1YwMi9st3EQYc6/HJNsugEi33cd6D7LvMbDtDhf
         nXRjkavIWa9oCGgxWr+1ABbTZs9sk/Tnjucl3Jcwuso9jV0e48FcypC+EnYn5V98NCJo
         Si7xEeyi69hA6L5vOCL/qIPOwpSaBB8dmKIuv/3B6b7syQAb85oA4oop9p0R2aB54tbp
         t88GQ7Ahf1yoOsW0OyqwXzUUhPqDU56o6jFFy2OOblrdKbj+GFEk/WvBpCU0n1xulrEK
         k+MWsOQrzW51Jjx6VdLiIq9qQK9nYPRUclIqxDTH5JsrUTiLQAKrucMjs07G5ErZrkCT
         t8sQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716966603; x=1717571403;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CQaycxxtR0oc9cSmzquAVtKXqxUU6P8ER3m5NLBKLqI=;
        b=YFrxnruv7uG5oX4UsQo/QXdKWJMHV2tXR+YPz8pWGLfsxYeFPFpCbCCUdD/ibq8x7u
         O94MPqv0iVXgOAuSvupjnamhvNLS7fy8JoZAUVczv+JjSk+SIE3COidu3WqKB/hfESco
         i58HmoNZ/xJ8xGRvrW9Q2QtffhPld7SVMa6yO8QoQ56Yebh78/cm/UVCljJfWyB9nkh4
         VyVP1IaYn37OZuti9C3iwyvHvF32L4/1B/J5QCJtcoKKJq08aj/Zif1Y5G0Rz/hZ4MMj
         6Fb1sNCTZuEj+kmmYTJz1coatVwsupYO+JPLgP9M4gGHJajWBR0iDb+nTvwK3pTfPY4g
         Xadg==
X-Forwarded-Encrypted: i=1; AJvYcCUFRPZ3z+1nz9OmjilceHtubX48jAPxFAMlmgftCBrp+XYrcbZLQVBZTB7RtqOxtJeFrVmPMR3tTpAElX1AYPGrGeGxwbetQbOpLmD9p+U=
X-Gm-Message-State: AOJu0YzFPXKoUa/30xtP4PGj8sIRcVnzjF9vicmDqeSkQ1w0d58hJCsn
	OQs2+PPPJljU3wgligj7twP5IXfxddijm9ruKT2dAK5gBYPXkfe+ViG/9ndjQA==
X-Google-Smtp-Source: AGHT+IFjyfM2m9iGVPw4RpPjemlzR0fgkzoilKpt2Qpic4+3MgSCUJZJMvdFAmzuY4q0BpFvPRG4WA==
X-Received: by 2002:a17:907:72c4:b0:a62:c215:9c20 with SMTP id a640c23a62f3a-a62c2159dc6mr700542066b.77.1716966602960;
        Wed, 29 May 2024 00:10:02 -0700 (PDT)
Message-ID: <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
Date: Wed, 29 May 2024 09:10:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 08:56, Chen, Jiqian wrote:
> On 2024/5/29 14:31, Jan Beulich wrote:
>> On 29.05.2024 04:41, Chen, Jiqian wrote:
>>> Hi,
>>> On 2024/5/17 19:50, Jan Beulich wrote:
>>>> On 17.05.2024 13:14, Chen, Jiqian wrote:
>>>>> On 2024/5/17 18:51, Jan Beulich wrote:
>>>>>> On 17.05.2024 12:45, Chen, Jiqian wrote:
>>>>>>> On 2024/5/16 22:01, Jan Beulich wrote:
>>>>>>>> On 16.05.2024 11:52, Jiqian Chen wrote:
>>>>>>>>> +        if ( gsi >= nr_irqs_gsi )
>>>>>>>>> +        {
>>>>>>>>> +            ret = -EINVAL;
>>>>>>>>> +            break;
>>>>>>>>> +        }
>>>>>>>>> +
>>>>>>>>> +        if ( !irq_access_permitted(current->domain, gsi) ||
>>>>>>>>
>>>>>>>> I.e. assuming IRQ == GSI? Is that a valid assumption when any number of
>>>>>>>> source overrides may be surfaced by ACPI?
>>>>>>> All irqs smaller than nr_irqs_gsi are gsi, aren't they?
>>>>>>
>>>>>> They are, but there's not necessarily a 1:1 mapping.
>>>>> Oh, so do I need to add a new gsi_caps to store granted gsi?
>>>>
>>>> Probably not. You ought to be able to translate between GSI and IRQ,
>>>> and then continue to record in / check against IRQ permissions.
>>> But I found in function init_irq_data:
>>>     for ( irq = 0; irq < nr_irqs_gsi; irq++ )
>>>     {
>>>         int rc;
>>>
>>>         desc = irq_to_desc(irq);
>>>         desc->irq = irq;
>>>
>>>         rc = init_one_irq_desc(desc);
>>>         if ( rc )
>>>             return rc;
>>>     }
>>> Does it mean that when irq < nr_irqs_gsi, the gsi and irq is a 1:1 mapping?
>>
>> No, as explained before. I also don't see how you would derive that from the code above.
> Because here set desc->irq = irq, and it seems there is no other place to change this desc->irq, so, gsi 1 is considered to irq 1.

What are you taking this from? The loop bound isn't nr_gsis, and the iteration
variable isn't in GSI space either; it's in IRQ numbering space. In this loop
we're merely leveraging that every GSI has a corresponding IRQ; there are no
assumptions made about the mapping between the two. Afaics at least.

>> "nr_irqs_gsi" describes what its name says: The number of
>> IRQs mapping to a (_some_) GSI. That's to tell them from the non-GSI (i.e.
>> mainly MSI) ones. There's no implication whatsoever on the IRQ <-> GSI
>> mapping.
>>
>>> What's more, when using PHYSDEVOP_setup_gsi, it calls mp_register_gsi,
>>> and in mp_register_gsi, it uses " desc = irq_to_desc(gsi); " to get irq_desc directly.
>>
>> Which may be wrong, while that wrong-ness may not have hit anyone in
>> practice (for reasons that would need working out).
>>
>>> Combining above, can we consider "gsi == irq" when irq < nr_irqs_gsi ?
>>
>> Again - no.
> Since you are certain that they are not equal, could you tell me where show they are not equal or where build their mappings,
> so that I can know how to do a conversion gsi from irq.

I did point you at the ACPI Interrupt Source Override structure before.
We're parsing those in acpi_parse_int_src_ovr(), to give you a place to
start going from.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 07:13:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:13:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731482.1136988 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDUk-0000bO-L2; Wed, 29 May 2024 07:13:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731482.1136988; Wed, 29 May 2024 07:13:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDUk-0000bH-Gl; Wed, 29 May 2024 07:13:10 +0000
Received: by outflank-mailman (input) for mailman id 731482;
 Wed, 29 May 2024 07:13:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCDUj-0000b7-Fh
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:13:09 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e52a47fd-1d8a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:13:06 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so178230066b.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:13:07 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc4ff24sm692535466b.130.2024.05.29.00.13.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 00:13:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e52a47fd-1d8a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716966787; x=1717571587; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Ym8+Kx4FdFlFPWRKcWibUZmDuugOnBJ9TD8u5uTPtag=;
        b=fDCDaogex7N1vuLQAgjdPuvKoDoOSfg2a7U57kL6fJ+Yq7YRIhlcvlVGaDmbAUze3a
         T59ZuVCL3R51i71PZ0Shl7qp0yJ+CuNTRX3jgiAM+coyMqexw4pt6aATFNWiQYOfYP1q
         1DC5BBLaP3sqjv54cGiKWulKx7ZQptlTpuX0OMcsbjAjLHOUFFqxmh/cFN0EF3BTiivH
         UERI9KYlGsgeObAPLYj6tozT6CRWHx0/tZR0AZzgkGNw7MSk/fGU+KsQ2nQpFphmJwKu
         Wu2pl9+6IYUSKKbRHweisu1SNc08KKgD+AdrW80fP/21QZOWSXgV67YKs1h8bi9Is0HO
         JuaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716966787; x=1717571587;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Ym8+Kx4FdFlFPWRKcWibUZmDuugOnBJ9TD8u5uTPtag=;
        b=ilL/f4c6ueL1gWgH2xvMrXYOmE6VSTq7Jf4qo9n/NIopgHqz6dvi+ng+IB6++viCEZ
         hBpA/VC/YHM0vlaDSOApTzwWEtdSQh7g1KVgpiDh6C0KpWvKpGUVk4uqdtX70rs5iB0G
         TZKM2u1CVQXjyWEZz7BSWuDCrHb6zF6Ov/b0YckVTNMkgjW8yeQj7s9zYcQZesVsYqx1
         V1OTn3aqBiz5NI6/ioWY9WgGxtD1OetZXDm1e3/V1+mB/6EMdxkWqHTdKZqHZU39ZU/H
         PCiIAzDEiDEAwcSI3hakf2Jn8JWua+PJ/NTWUNVyMO3mNdDs1Rzvo1ILWN4U63BOUzzk
         ctiQ==
X-Forwarded-Encrypted: i=1; AJvYcCWZY4zsTPYWIl8erDzCGi0fr3edFBhg3JQ1W7aNdo0PDKrufcq//i3zRo+jk8i5eq/mdiFjDlPTUUrWU6z5rNvM2TOMJO0ShjpZJNBzGMk=
X-Gm-Message-State: AOJu0YxJegdJKpTLHpDPGerg9QZxRYtxJRi/eQv+xquIHVp2oLy7OFxL
	FMc+yuIBiDtL0mir7rVQ6AwBgvp/3LgD2FnRse+dS01nxp7QH1vgmy20XR4ycA==
X-Google-Smtp-Source: AGHT+IEif6jGTfTkRxmBNVRO7V3ZOrM6H7KDujrAx/g/YW86O7/9C8M3wTJ/VYJT1S+j3OmCOm6DmQ==
X-Received: by 2002:a17:906:2482:b0:a59:9e01:e788 with SMTP id a640c23a62f3a-a62646d6280mr900924266b.34.1716966786871;
        Wed, 29 May 2024 00:13:06 -0700 (PDT)
Message-ID: <7ba8686b-8d59-4107-b807-0e8397fc0e25@suse.com>
Date: Wed, 29 May 2024 09:13:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 for-4.19 0.5/13] xen: Introduce CONFIG_SELF_TESTS
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240528142238.1340228-1-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240528142238.1340228-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.05.2024 16:22, Andrew Cooper wrote:
> ... and move x86's stub_selftest() under this new option.
> 
> There is value in having these tests included in release builds too.
> 
> It will shortly be used to gate the bitops unit tests on all architectures.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> I've gone with SELF_TESTS rather than BOOT_TESTS, because already in bitops
> we've got compile time tests (which aren't strictly boot time), and the
> livepatching testing wants to be included here and is definitely not boot
> time.

I second this consideration, fwiw.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 07:24:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:24:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731488.1136996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDfb-0002Rw-La; Wed, 29 May 2024 07:24:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731488.1136996; Wed, 29 May 2024 07:24:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDfb-0002Rp-J4; Wed, 29 May 2024 07:24:23 +0000
Received: by outflank-mailman (input) for mailman id 731488;
 Wed, 29 May 2024 07:24:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCDfZ-0002Rj-Qd
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:24:21 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 772d9ea7-1d8c-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 09:24:20 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a635a74e0deso114407166b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:24:20 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc4fedesm678741166b.118.2024.05.29.00.24.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 00:24:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 772d9ea7-1d8c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716967460; x=1717572260; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=iUTtgt2aFMpSmMUgR5accW4qs9bkbOHGw/CHMkfJ1Q8=;
        b=QVajw/zxfxohsxJo3pJbeNKXtSTg5k67F6TpB509qpb1ci0pS4tcGG2BDjbxqu8nzy
         prmCjl+02Dxrr0FglN0mEdZCxli8xY5rQmTq9q4ilTCf2UQmb+xC1UqllY7NpbFFWVpv
         W2HnVVovwCS7cg+z2QNrzkaIFYOWG2HBNO021guj/jz9/3y5U803ggWtYS6hgYp5QoN+
         fFWdIEx57p8qpsyBsmDlk8gOzOjG97uh9QgnZtPNIzAt1+oQyT2LlaZfQgoa9jQwaKP8
         BW5+Dnhn9Zz45gZiCu8fGcCRlwjUPfNoASIwKn8PyRUlnGw1NIu1gRjHxAL81CgfOzYD
         v0Sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967460; x=1717572260;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iUTtgt2aFMpSmMUgR5accW4qs9bkbOHGw/CHMkfJ1Q8=;
        b=M1lg2a/P9v8AthjsmHYAUrrrJU0kfOwAKoMX04FYn+t8Fx2QvLQpmGgunMPvZboMLm
         pOXEq344AxCS56YOdXsXa4svhzuoPHqF2XZUMdQyJa2Nl/5ofnNJ7S/tDrNJUGJ5ocbI
         /sP1S/6N62zi7ERGvGI9fXZoPUrh4YWOUf52LGZ0r3upsS9N8hg66AsShU+JxfYfvOQZ
         9zGQX0G3PYPulckxZe5K5eIj/u1P38ABQxCx/oxPAPT1pDgT4cBnL+pqAAEqiwzqcZ50
         LkeCStMIgh9qWo6f22YMhZyZ/IWbVFTIMB769+y0qvZM1tjX46XpYDbiWFf5f/BJmaOa
         lmCQ==
X-Forwarded-Encrypted: i=1; AJvYcCW7GXMGUGv/zJ//3VWq+Pz2nr8fNiXkwLs7i5vB0Pina4aA1pVT0SudxQ7XtenpWbLdyQz2k2YVexuP49CpCHH79hOO/1lOmOHmD6rgF7c=
X-Gm-Message-State: AOJu0YxGJ5CXgA2cXepC1skUEQAszlkJPuZTw8NHIkpBA8atS+0ksIE4
	OurBCxUau2xg1Mo0EaUh5unD0PndoznjN4X8si+YAdHlER3vs9M7g+p/8W1HIA==
X-Google-Smtp-Source: AGHT+IE0PWBhT7K0kCY7fbVzU19w3zq5nP7KvA65nBv7Rm8vBSqhEzwIRqR+ukse2WvnVt07ycwKZw==
X-Received: by 2002:a17:906:e84:b0:a62:404a:d0d0 with SMTP id a640c23a62f3a-a62643e458cmr923721266b.42.1716967459948;
        Wed, 29 May 2024 00:24:19 -0700 (PDT)
Message-ID: <0c4a9a00-b8de-4472-afb4-0a8d70c7aada@suse.com>
Date: Wed, 29 May 2024 09:24:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 0/3] xen/x86: support foreign mappings for
 HVM/PVH
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Community Manager <community.manager@xenproject.org>,
 Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20240517133352.94347-1-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240517133352.94347-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.05.2024 15:33, Roger Pau Monne wrote:
> Hello,
> 
> The following series attempts to solve a shortcoming of HVM/PVH guests
> with the lack of support for foreign mappings.  Such lack of support
> prevents using PVH based guests as stubdomains for example.
> 
> Add support in a way similar to how it's done on Arm, by iterating over
> the p2m based on the maximum gfn.
> 
> Patch 2 is not strictly needed.  Moving the enablement of altp2m from an
> HVM param to a create domctl flag avoids any possible race with the HVM
> param changing after it's been evaluated.  Note the param can only be
> set by the control domain, and libxl currently sets it at domain
> create.  Also altp2m enablement is different from activation, as
> activation does happen during runtime of the domain.
> 
> Thanks, Roger.
> 
> Roger Pau Monne (3):
>   xen/x86: account number of foreign mappings in the p2m
>   xen/x86: enable altp2m at create domain domctl
>   xen/x86: remove foreign mappings from the p2m on teardown

Here, too, I'd like to ask whether to keep this as a candidate for 4.19, or
whether to postpone. Afaict what's still missing are Arm and tool chain acks
on patch 2.

Jan

>  CHANGELOG.md                        |  1 +
>  tools/libs/light/libxl_create.c     | 23 +++++++++-
>  tools/libs/light/libxl_x86.c        | 26 +++++------
>  tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
>  xen/arch/arm/domain.c               |  6 +++
>  xen/arch/x86/domain.c               | 28 ++++++++++++
>  xen/arch/x86/hvm/hvm.c              | 23 +++++++++-
>  xen/arch/x86/include/asm/p2m.h      | 32 +++++++++-----
>  xen/arch/x86/mm/p2m-basic.c         | 18 ++++++++
>  xen/arch/x86/mm/p2m.c               | 68 +++++++++++++++++++++++++++--
>  xen/include/public/domctl.h         | 20 ++++++++-
>  xen/include/public/hvm/params.h     |  9 +---
>  12 files changed, 215 insertions(+), 41 deletions(-)
> 



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:25:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:25:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731492.1137008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDgu-0002y7-17; Wed, 29 May 2024 07:25:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731492.1137008; Wed, 29 May 2024 07:25:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDgt-0002y0-TN; Wed, 29 May 2024 07:25:43 +0000
Received: by outflank-mailman (input) for mailman id 731492;
 Wed, 29 May 2024 07:25:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCDgs-0002xp-4N
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:25:42 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a5dc5c8d-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:25:38 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a626776cc50so181708766b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:25:40 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a62c501e85dsm444539966b.117.2024.05.29.00.25.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:25:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5dc5c8d-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716967540; x=1717572340; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Cekc9GdArRjBtHvDU3egGvR/lGulkBwka/aFwWcGvjc=;
        b=KT0n7Bmkxfn3DogRARn2snuOFKvRbi/Bq1aCf+0uYW/eppfm1Pj5XYCmNyFzcMMndE
         JkQHg2SHXVz0/tqInWrxDgpl/uXQ6nH3nL80rL/nm3Pob/M3lCkkF6PUumeYPEwzRJVp
         QyQuJyadXGwRInUybies4oCYo5O/3a+UX885KPYUV4I0bdIK63gJNA6+jnID6lc2yiTZ
         Cfa/+4RA0qW8qBDmWKxSyYkSpzaty+bPEny0zB6NQzDHBmSxypTXIl2xxB5SrJUWJHvb
         oJoT38WAfOngInDYZQWCIfkyeIqAN6nbPINIhtgLpsZoVnOaO4kcLeF655IybY6llRL7
         hniw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967540; x=1717572340;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Cekc9GdArRjBtHvDU3egGvR/lGulkBwka/aFwWcGvjc=;
        b=vPxgB4xneUquicLOxfjfXLWhW6LXhwJYtUeYaeN2/lNel7U1ldJ3etwm7YZi1oY+VS
         MFF27fw3uvL1Gt8lomHSkF88bIXV/dC8EMji2/4fdWiGHHt5w/Y6zy2w68yoU640nkTF
         s+zkZ7FX0Cc5mLTrhPbD5zUXdPtjpHICHqzCV1pzB07HX+EI8bN0XtEiixUYJ+JN0+Bn
         MvEGbFA2cTQaWIqd+CgvICYVYSlIPpDwGpic9c4SwaZAGQBsnR1tDgboEp5ZMBRRSVEZ
         5Z7JFxDOAdfpZBgUTnV4vsGS53QDnKTUSMOjIcCf7OQ2DIgPcywoaPHyokV/9cyxaM/D
         LLng==
X-Forwarded-Encrypted: i=1; AJvYcCWDXgSYIEK17Do7/xTdk0pCBYQBYMho/ZsR5zSMRTbwpnc+W955NxqpdySDxUdVzO+hCerHsPdeIxFDF6uGnKh5w0Gdo++SR1dFO2st2DA=
X-Gm-Message-State: AOJu0YzcBUZzO4H5eatzRilJyrcPCUgXHQJ9QVrbI2iNDCNB2zf28CtY
	SJMmZpOOdT1dkEA5L88Mai/z3q8HqqHi0QIC4BAQntxzjWqM7APr
X-Google-Smtp-Source: AGHT+IG+hNAAaqDhe2WTq0a0yMjwvAezCXiCbkWw2nCvGBSayVfBse6QzLtdRgbDKk0Lwn7tOqMuKA==
X-Received: by 2002:a17:906:3896:b0:a61:bd80:7ba6 with SMTP id a640c23a62f3a-a62641a56c3mr812136266b.3.1716967539441;
        Wed, 29 May 2024 00:25:39 -0700 (PDT)
Message-ID: <371b5bd54da8d77f351c986d1a43d042b869402b.camel@gmail.com>
Subject: Re: [XEN PATCH v4 0/3] x86: make Intel/AMD vPMU & MCE support
 configurable
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 09:25:38 +0200
In-Reply-To: <9b5890de-ce5b-424c-80f9-3be83e9b94f0@suse.com>
References: <cover.1716366581.git.Sergiy_Kibrik@epam.com>
	 <9b5890de-ce5b-424c-80f9-3be83e9b94f0@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Mon, 2024-05-27 at 17:47 +0200, Jan Beulich wrote:
> Oleksii,
>=20
> On 22.05.2024 10:37, Sergiy Kibrik wrote:
> > Three remaining patches to separate support of Intel & AMD CPUs in
> > Xen build.
> > Most of related patches from previous series had already been
> > merged.
> > Specific changes since v3 are provided per-patch.
> >=20
> > v3 series here:
> > https://lore.kernel.org/xen-devel/cover.1715673586.git.Sergiy_Kibrik@ep=
am.com/
> >=20
> > =C2=A0 -Sergiy
> >=20
> > Sergiy Kibrik (3):
> > =C2=A0 x86/intel: move vmce_has_lmce() routine to header
> > =C2=A0 x86/MCE: add default switch case in init_nonfatal_mce_checker()
> > =C2=A0 x86/MCE: optional build of AMD/Intel MCE code
>=20
> As I'm apparently confused as to the state 4.19 is in, may I please
> ask
> whether this series is still okay to go in, or whether it should be
> postponed until after branching.
I am okay to go in this release.

Sorry for the confusion with the 4.19 state. I'll send a proper
schedule today.


~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731495.1137016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhH-0003Qv-74; Wed, 29 May 2024 07:26:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731495.1137016; Wed, 29 May 2024 07:26:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhH-0003Qm-4R; Wed, 29 May 2024 07:26:07 +0000
Received: by outflank-mailman (input) for mailman id 731495;
 Wed, 29 May 2024 07:26:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhG-0003Qb-NB
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:06 +0000
Received: from mail-lf1-x143.google.com (mail-lf1-x143.google.com
 [2a00:1450:4864:20::143])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b4688c57-1d8c-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 09:26:03 +0200 (CEST)
Received: by mail-lf1-x143.google.com with SMTP id
 2adb3069b0e04-52b59f77670so93489e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:03 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4688c57-1d8c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967562; x=1717572362; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=ZKvf3N3QBgrkKCjztY5Opirx663UEubBXrA8grWK3w4=;
        b=H5pjecPHkJJncxtn44SfS3b8pu2XVQUaJTMQdLm+PaWgqTFPmFzdDZPslVOLczuF0o
         nPRxjQO90Zy+CvEEfb6PEiuvHlbMR+BIuejD8mr1IjZuy0/drIjtyozMcrBEAaJ28s4/
         Z34nfvRH1ne8Z+vGjr0UdGzxmUt/0g/0AXjRlLSXaeZS1823byI5mdI4SiH8rvz8ztMI
         jqcsuZPMvHleRe8mz3P5z8wGXkNhl41HqtIw8ks/fChDiDhaN63AxxWr3uc5a99F7saj
         V0nODXY9b8VCQDoffClyp35+73zzYOsKvDSWccUPKOD/wHSWHW5WBDVuBRVyaXbBloP4
         qthA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967562; x=1717572362;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ZKvf3N3QBgrkKCjztY5Opirx663UEubBXrA8grWK3w4=;
        b=r2ImCGG9j5cx2gNKFSqCXEU0SWMKQSmdf8cY+oE+R75zmnax0peaQmNzAzgwm7UQCv
         gzciTEq7bs2qY+JqMSZ3/AnOrE5QxQFFZ6LTCVw93+wDNVVyW1a33C13bRXeEIfMHyDL
         wHhxCGt6CN+PvKCKMInuCzeaVCtSJEh70WsYGC+NEyVRgizlhwJxYAiu5Kxfi6LR8QWa
         jpbIyyJUAX2brGXttB7fc6+NksEbetGIzBAXstCjgO0etBy/ghV4DpeK45U01cqqJ7Hr
         m+SwNGi9iMXC9KW0w6N10psTku5zhHizm2R7CJfdhmyWoC8GivynTdUOZ6DOVwxpkTAo
         /DCg==
X-Gm-Message-State: AOJu0Yw8IQxfRItkqhlx0GS9M1vsSdMZCvi3c5tX41XgAmfbeQc/RFFe
	WLicoq7lFbTVduU0pZ7or76LrZAA4Xzc3SWyqLVBFG1yGY5EjviT7D1ZS9Eofp99QxbWosUII2W
	2An/ueQ==
X-Google-Smtp-Source: AGHT+IFKGaop/h2JULMJhTxHRPmMIMatDi4tGed43Uqer6JxWZ74iEewDYiJAxCXeinh3W4WsR/1Dg==
X-Received: by 2002:a05:6512:33d6:b0:51f:4c53:8e4b with SMTP id 2adb3069b0e04-52965c3109emr16550651e87.38.1716967561738;
        Wed, 29 May 2024 00:26:01 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 0/7] FF-A notifications
Date: Wed, 29 May 2024 09:25:52 +0200
Message-Id: <20240529072559.2486986-1-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi,

This patch set adds support for FF-A notifications. We only support
global notifications, per vCPU notifications remain unsupported.

The first three patches are further cleanup and can be merged before the
rest if desired.

A physical SGI is used to make Xen aware of pending FF-A notifications. The
physical SGI is selected by the SPMC in the secure world. Since it must not
already be used by Xen the SPMC is in practice forced to donate one of the
secure SGIs, but that's normally not a problem. The SGI handling in Xen is
updated to support registration of handlers for SGIs that aren't statically
assigned, that is, SGI IDs above GIC_SGI_MAX.

The patch "xen/arm: add and call init_tee_interrupt()" provides a hook for
register the needed per-cpu interrupt handler in "xen/arm: ffa: support
notification".

The patch "xen/arm: add and call tee_free_domain_ctx()" provides a hook for
later freeing of the TEE context. This hook is used in "xen/arm: ffa:
support notification" and avoids the problem with TEE context being freed
while we need to access it when handling a Schedule Receiver interrupt. It
was suggested as an alternative in [1] that the TEE context could be freed
from complete_domain_destroy().

[1] https://lore.kernel.org/all/CAHUa44H4YpoxYT7e6WNH5XJFpitZQjqP9Ng4SmTy4eWhyN+F+w@mail.gmail.com/

Thanks,
Jens

v4->v5:
- Added two new patches "xen/arm: add and call init_tee_interrupt()" and
  "xen/arm: add and call tee_free_domain_ctx()"
- Updated "xen/arm: ffa: support notification", details in the patch

v3->v4:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.

v2->v3:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.

v1->v2:
- "xen/arm: ffa: support notification" and
  "xen/arm: allow dynamically assigned SGI handlers" updated as requestsed,
  details in each patch.
- Added Bertrands R-B for "xen/arm: ffa: refactor ffa_handle_call()",
  "xen/arm: ffa: use ACCESS_ONCE()", and
  "xen/arm: ffa: simplify ffa_handle_mem_share()"

Jens Wiklander (7):
  xen/arm: ffa: refactor ffa_handle_call()
  xen/arm: ffa: use ACCESS_ONCE()
  xen/arm: ffa: simplify ffa_handle_mem_share()
  xen/arm: allow dynamically assigned SGI handlers
  xen/arm: add and call init_tee_interrupt()
  xen/arm: add and call tee_free_domain_ctx()
  xen/arm: ffa: support notification

 xen/arch/arm/domain.c              |   1 +
 xen/arch/arm/gic.c                 |  12 +-
 xen/arch/arm/include/asm/gic.h     |   2 +-
 xen/arch/arm/include/asm/tee/tee.h |  14 +
 xen/arch/arm/irq.c                 |  18 +-
 xen/arch/arm/smpboot.c             |   2 +
 xen/arch/arm/tee/Makefile          |   1 +
 xen/arch/arm/tee/ffa.c             | 100 +++++--
 xen/arch/arm/tee/ffa_notif.c       | 409 +++++++++++++++++++++++++++++
 xen/arch/arm/tee/ffa_partinfo.c    |   9 +-
 xen/arch/arm/tee/ffa_private.h     |  56 +++-
 xen/arch/arm/tee/ffa_shm.c         |  33 +--
 xen/arch/arm/tee/tee.c             |  14 +-
 xen/include/public/arch-arm.h      |  14 +
 14 files changed, 622 insertions(+), 63 deletions(-)
 create mode 100644 xen/arch/arm/tee/ffa_notif.c

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731496.1137027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhI-0003ft-Fx; Wed, 29 May 2024 07:26:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731496.1137027; Wed, 29 May 2024 07:26:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhI-0003fk-Bv; Wed, 29 May 2024 07:26:08 +0000
Received: by outflank-mailman (input) for mailman id 731496;
 Wed, 29 May 2024 07:26:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhH-0002xp-Oo
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:07 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b49b26a9-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:26:03 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-5295eb47b48so2163324e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:04 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b49b26a9-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967564; x=1717572364; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2VzeWK8x82YxZO3FACx+2RlqRVfXhRtSExk4PP34ai4=;
        b=UF1xwjfU11jMAoruWrPSOaPXIXLOW9FovPzSabYrt0ugV5KA0mOb4BQqD3O1DNojl1
         cDYNMvxOUtOjjSYFRDmo6LuyLV0F9lg1/EB7G49yQ86Qu2AdAGoteK32HKlkfuGkHOUc
         bPottUM5iY/qJdBEayPr4KwtF9tJOssTNii/Bvp2j3OFty1YF7ipoXHX3H3MnpXSFwiz
         4cr3kBjMsIGd5wpFlGvxaJ9zk5fxq0JZAM2XuT/Zt9HdajCW0YyKlMRaGZ01vxywhi2a
         rrehQkzlxwT0jTdBfqnuubZTd8TpOCj8fwpWY3W8PlfEe+BHmA+AIdyXpVrYPWUXdQuD
         zALg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967564; x=1717572364;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2VzeWK8x82YxZO3FACx+2RlqRVfXhRtSExk4PP34ai4=;
        b=o6wV/5svJxALnVMTh/ziVE1oM/M3JzCFb7pnS4SAgEiDZEZqTLM/SpVc9aSq2HylKq
         T0Qk7jh8oInSAGcY2gdvQXCxD2tXK782Bx3Mj2USxhquq7yjfQFPYjCYUA1ypCUCvKSJ
         ygKMdfwfOQjXolMh2I0KyqcvhE1U97venPWNTgSFaxDzrIXlN1r5XErv6Bwa5fGSGEsK
         YYo1H9FCwTSsu0wh4ImAFv9kwYQMicBSP7dWKM4fDYT50YXZpGXy4BY/wiJWt6uXsXbx
         iIdK127xilcmISUTTYcDdInI7qUmlahhSmHQ9PLO3RTtm4c55PpjdL6SRbejI1W/tBr/
         9REg==
X-Gm-Message-State: AOJu0YxS9DpYipRdb6/YNMyU1eZY98us/2d+x6+igh+VIfjllph+AFBr
	7uWXdlxKUHzxiBpYzh+0sCJ/U8CUgkwZCUukQhTcUOeZd264mX1Hl4Svdp9G5ByYgq1M18H6OXL
	D0wM=
X-Google-Smtp-Source: AGHT+IHUammB1br6gC/kNEl2MIJXLC3DblISLU4GACqDN+CQ2EGDJncNtdswHtQjYuXcwxxcJqhvRg==
X-Received: by 2002:a19:6a17:0:b0:529:c0c6:faad with SMTP id 2adb3069b0e04-529c0c6fbcamr1133747e87.28.1716967563718;
        Wed, 29 May 2024 00:26:03 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 1/7] xen/arm: ffa: refactor ffa_handle_call()
Date: Wed, 29 May 2024 09:25:53 +0200
Message-Id: <20240529072559.2486986-2-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Refactors the large switch block in ffa_handle_call() to use common code
for the simple case where it's either an error code or success with no
further parameters.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa.c | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 8665201e34a9..5209612963e1 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -273,18 +273,10 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
     case FFA_RXTX_MAP_64:
         e = ffa_handle_rxtx_map(fid, get_user_reg(regs, 1),
 				get_user_reg(regs, 2), get_user_reg(regs, 3));
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_RXTX_UNMAP:
         e = ffa_handle_rxtx_unmap();
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_PARTITION_INFO_GET:
         e = ffa_handle_partition_info_get(get_user_reg(regs, 1),
                                           get_user_reg(regs, 2),
@@ -299,11 +291,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
         return true;
     case FFA_RX_RELEASE:
         e = ffa_handle_rx_release();
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
     case FFA_MSG_SEND_DIRECT_REQ_32:
     case FFA_MSG_SEND_DIRECT_REQ_64:
         handle_msg_send_direct_req(regs, fid);
@@ -316,17 +304,19 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
         e = ffa_handle_mem_reclaim(regpair_to_uint64(get_user_reg(regs, 2),
                                                      get_user_reg(regs, 1)),
                                    get_user_reg(regs, 3));
-        if ( e )
-            ffa_set_regs_error(regs, e);
-        else
-            ffa_set_regs_success(regs, 0, 0);
-        return true;
+        break;
 
     default:
         gprintk(XENLOG_ERR, "ffa: unhandled fid 0x%x\n", fid);
         ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
         return true;
     }
+
+    if ( e )
+        ffa_set_regs_error(regs, e);
+    else
+        ffa_set_regs_success(regs, 0, 0);
+    return true;
 }
 
 static int ffa_domain_init(struct domain *d)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731497.1137037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhJ-0003vP-Mn; Wed, 29 May 2024 07:26:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731497.1137037; Wed, 29 May 2024 07:26:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhJ-0003v9-IL; Wed, 29 May 2024 07:26:09 +0000
Received: by outflank-mailman (input) for mailman id 731497;
 Wed, 29 May 2024 07:26:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhI-0002xp-Or
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:08 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b5c3aa7e-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:26:05 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-52b6ccfdf28so6718e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:06 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5c3aa7e-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967565; x=1717572365; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Injz1eHKMU34CsdRACoEV/3/mGn5muvQCz5MmACq6o4=;
        b=qsR57bksHdfVIS5ZgWF3GPCxGYE8lK0+trTmrCkCSVGiejL6rCS/NttsludEcpQjEF
         PyWJEMKtHvA21IMjCII4vr5eAaIvHbLr8oQ9OaCrt+FSDNSj4dRdA7Fd9LwWACCxnFe9
         l8wAsNRPJcavjHJPBS7lNDZGlo97oTdct0z9kVNeTbdpUqyfXnEhtPr1vUnlNYYs5HuS
         hf0LHzQv0QbDSaiJeahyAa0e+cW4d/lSEtJi2rXyDAoz0lav6CcfUQnVFs5X+xTlihQG
         h0mCsH9xmgYQVi4KTtE6likulUB6bHywx8Hii8WYhhTDWzqJUvur1gJ81MyBhPpFVOEx
         5Gtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967565; x=1717572365;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Injz1eHKMU34CsdRACoEV/3/mGn5muvQCz5MmACq6o4=;
        b=aQxL1JTBCjJ6HymiCQ3XVGRi6XjlE4TFCVS46CxGDg5RTPo4ajx1fDEliR6rF3pxLh
         rAb1HtnpD32TC+7y6fJSdAFLV9UQn4Ut8x+9O6aIODhMlyyh7ntW4tJ3ObECTqYN2K1q
         3WRTmCjpTn0NkbD4+emkmkTZX/qt4nAesgBQR/rqHd78zbUFKEFuiARFxjkNmXmMKuCr
         cYUi9ufur1rPPIDUrLOnY06FHShoTpZ/cYoUooaEacMHrYpxzOmzpcZzl2+Vnr+m6kjS
         GYJcy8qfc+U3kOX80tq0OZ5Jhac22UEJKwy/67EOVKZBp1aW+TRgx3d+JhcNXAasF2L9
         eO8A==
X-Gm-Message-State: AOJu0Ywngn9R1gWxOrbF2s98067GZn+kXTenfmzgnlW3N/pLANSAacX5
	Xqp8KXjzx37FZzjCO6bun3NVqHEY8DZJZUAxfYUIsilDI9f4Jh+bCgMXiQ/dpIXjRWKu9VYvcX1
	18BQ=
X-Google-Smtp-Source: AGHT+IG4us99S6HJMmw1BnNMPCNiTcik3IIzYpyfmcLKgmcqg3Zk8NHoh84cKeLXUs6K75aOamqMZw==
X-Received: by 2002:a05:6512:4cc:b0:51e:f2bb:158a with SMTP id 2adb3069b0e04-52966ca8c0fmr8792474e87.64.1716967565550;
        Wed, 29 May 2024 00:26:05 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 2/7] xen/arm: ffa: use ACCESS_ONCE()
Date: Wed, 29 May 2024 09:25:54 +0200
Message-Id: <20240529072559.2486986-3-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Replace read_atomic() with ACCESS_ONCE() to match the intended use, that
is, to prevent the compiler from (via optimization) reading shared
memory more than once.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa_shm.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/tee/ffa_shm.c b/xen/arch/arm/tee/ffa_shm.c
index eed9ad2d2986..75a5b66aeb4c 100644
--- a/xen/arch/arm/tee/ffa_shm.c
+++ b/xen/arch/arm/tee/ffa_shm.c
@@ -7,6 +7,7 @@
 #include <xen/sizes.h>
 #include <xen/types.h>
 #include <xen/mm.h>
+#include <xen/lib.h>
 #include <xen/list.h>
 #include <xen/spinlock.h>
 
@@ -171,8 +172,8 @@ static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
 
     for ( n = 0; n < range_count; n++ )
     {
-        page_count = read_atomic(&range[n].page_count);
-        addr = read_atomic(&range[n].address);
+        page_count = ACCESS_ONCE(range[n].page_count);
+        addr = ACCESS_ONCE(range[n].address);
         for ( m = 0; m < page_count; m++ )
         {
             if ( pg_idx >= shm->page_count )
@@ -527,13 +528,13 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
 
     mem_access = ctx->tx + trans.mem_access_offs;
-    if ( read_atomic(&mem_access->access_perm.perm) != FFA_MEM_ACC_RW )
+    if ( ACCESS_ONCE(mem_access->access_perm.perm) != FFA_MEM_ACC_RW )
     {
         ret = FFA_RET_NOT_SUPPORTED;
         goto out_unlock;
     }
 
-    region_offs = read_atomic(&mem_access->region_offs);
+    region_offs = ACCESS_ONCE(mem_access->region_offs);
     if ( sizeof(*region_descr) + region_offs > frag_len )
     {
         ret = FFA_RET_NOT_SUPPORTED;
@@ -541,8 +542,8 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
     }
 
     region_descr = ctx->tx + region_offs;
-    range_count = read_atomic(&region_descr->address_range_count);
-    page_count = read_atomic(&region_descr->total_page_count);
+    range_count = ACCESS_ONCE(region_descr->address_range_count);
+    page_count = ACCESS_ONCE(region_descr->total_page_count);
 
     if ( !page_count )
     {
@@ -557,7 +558,7 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
     }
     shm->sender_id = trans.sender_id;
-    shm->ep_id = read_atomic(&mem_access->access_perm.endpoint_id);
+    shm->ep_id = ACCESS_ONCE(mem_access->access_perm.endpoint_id);
 
     /*
      * Check that the Composite memory region descriptor fits.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731498.1137047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhL-0004Cu-3h; Wed, 29 May 2024 07:26:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731498.1137047; Wed, 29 May 2024 07:26:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhK-0004Ch-WE; Wed, 29 May 2024 07:26:11 +0000
Received: by outflank-mailman (input) for mailman id 731498;
 Wed, 29 May 2024 07:26:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhJ-0003Qb-2E
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:09 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b79f8de9-1d8c-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 09:26:08 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e95a1eff78so22859931fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:08 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b79f8de9-1d8c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967567; x=1717572367; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XH0hx5FhOx9XD5Quim9d/dWx8ptFKhc8V8iHDPJ7zTg=;
        b=apGCZuFr314QW5S7oM+Ji2IjCiTkVcm76gRDB0ig0ePZFb2lUj/2Ao+JG3aMpkpTP/
         I4LYBMQbLfNLcFmSKcwrsvkZ5+2KOc6FKbyOL/yhsb1F5F9WiDcPixHSR2LQbvBkii1W
         CY9gw55nFQki/HVz6Te0iZ6z1AngEmxGU85EefXN8sbgsKNq3MWUGnUC4cQKr2malIAp
         1+1krxV6tANNUgllzaj1Oh5qsM9bs+4MRkHTEH2q83X7t/BOIZ+mO0Zn5uR5xDG5Ymfh
         Q7lsSJFxFAFV0HnJqW81P+HlSC2FhjFCKnYn+RVy1XtvnkDNOk2PL53KmkSEAdeMxtny
         yqHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967567; x=1717572367;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XH0hx5FhOx9XD5Quim9d/dWx8ptFKhc8V8iHDPJ7zTg=;
        b=tDbM54x4c9pvDm+z7rt4N6DFrvzLZUvPvAaHQfVYfTdUmHyN1i0A3M6mBTMFm10jj4
         7mAN8FMFL1rYNd37MLrFqAy5FRjKDy6rrOwZT6+Dg1n7XYKPm4sDzlB2VlbaNEfS/56n
         WbJkaDqtpgx5O5bEVUa/9bxbrg9pcw2IplvlvHT7SFtA2UONg7dn4uN9VvNzBjAbqgem
         BKtkieScplpqBLd8ukfYeKZ5vBRFx9oWKLDjjHR94cQFGvKbKnoFyD24UwyEgPpn8tn3
         6IRzEenoXWJ2ylt1kzGkWW2eCENBeLt+aLb7/vv5vfQ9zYVz9QqK69vl+T6outHmBy87
         80vQ==
X-Gm-Message-State: AOJu0YwFqRgYzF6pmdSq+9JodT9KBfJANxMiSvdI1SEo/wCAwHjWEJcn
	L+6wkelQ/BtN0OklDmWncYXcvHeXgDfFvbU/La/8oQ74AOca6XmYGKmKvRVdmSJ9NXkAlPGFchV
	QWhM=
X-Google-Smtp-Source: AGHT+IFIoBrKT/kDs3uPPD+xIx8y8di16Jqchsw2ABXeJEk1y0F6sSFegq5DQBoH4LL32mpv6TxtmQ==
X-Received: by 2002:a19:ad4a:0:b0:529:4864:9c17 with SMTP id 2adb3069b0e04-52967463e64mr12099898e87.57.1716967567404;
        Wed, 29 May 2024 00:26:07 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 3/7] xen/arm: ffa: simplify ffa_handle_mem_share()
Date: Wed, 29 May 2024 09:25:55 +0200
Message-Id: <20240529072559.2486986-4-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Simplify ffa_handle_mem_share() by removing the start_page_idx and
last_page_idx parameters from get_shm_pages() and check that the number
of pages matches expectations at the end of get_shm_pages().

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 xen/arch/arm/tee/ffa_shm.c | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/tee/ffa_shm.c b/xen/arch/arm/tee/ffa_shm.c
index 75a5b66aeb4c..370d83ec5cf8 100644
--- a/xen/arch/arm/tee/ffa_shm.c
+++ b/xen/arch/arm/tee/ffa_shm.c
@@ -159,10 +159,9 @@ static int32_t ffa_mem_reclaim(uint32_t handle_lo, uint32_t handle_hi,
  */
 static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
                          const struct ffa_address_range *range,
-                         uint32_t range_count, unsigned int start_page_idx,
-                         unsigned int *last_page_idx)
+                         uint32_t range_count)
 {
-    unsigned int pg_idx = start_page_idx;
+    unsigned int pg_idx = 0;
     gfn_t gfn;
     unsigned int n;
     unsigned int m;
@@ -191,7 +190,9 @@ static int get_shm_pages(struct domain *d, struct ffa_shm_mem *shm,
         }
     }
 
-    *last_page_idx = pg_idx;
+    /* The ranges must add up */
+    if ( pg_idx < shm->page_count )
+            return FFA_RET_INVALID_PARAMETERS;
 
     return FFA_RET_OK;
 }
@@ -460,7 +461,6 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
     struct domain *d = current->domain;
     struct ffa_ctx *ctx = d->arch.tee;
     struct ffa_shm_mem *shm = NULL;
-    unsigned int last_page_idx = 0;
     register_t handle_hi = 0;
     register_t handle_lo = 0;
     int ret = FFA_RET_DENIED;
@@ -570,15 +570,9 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)
         goto out;
     }
 
-    ret = get_shm_pages(d, shm, region_descr->address_range_array, range_count,
-                        0, &last_page_idx);
+    ret = get_shm_pages(d, shm, region_descr->address_range_array, range_count);
     if ( ret )
         goto out;
-    if ( last_page_idx != shm->page_count )
-    {
-        ret = FFA_RET_INVALID_PARAMETERS;
-        goto out;
-    }
 
     /* Note that share_shm() uses our tx buffer */
     spin_lock(&ffa_tx_buffer_lock);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731499.1137057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhN-0004VO-Bi; Wed, 29 May 2024 07:26:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731499.1137057; Wed, 29 May 2024 07:26:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhN-0004VF-8C; Wed, 29 May 2024 07:26:13 +0000
Received: by outflank-mailman (input) for mailman id 731499;
 Wed, 29 May 2024 07:26:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhL-0002xp-UK
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:11 +0000
Received: from mail-lf1-x141.google.com (mail-lf1-x141.google.com
 [2a00:1450:4864:20::141])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b7b4ac3f-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:26:08 +0200 (CEST)
Received: by mail-lf1-x141.google.com with SMTP id
 2adb3069b0e04-52b59f77670so93660e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:10 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7b4ac3f-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967569; x=1717572369; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SILgfa72VI0ajqqfGUDbvcNqJ8ZZiLfW7kFdd1Zm0SE=;
        b=AuB4qbJ2xqimzqlEorSKI44W2m0Wekva43pyAvzluEBYQO/uksqZgbPHXejPJW+A7A
         A0MWYEnCBvwmnO7KNwEeUjMY0MStM2BfoCy+l7SWT+Gvi11A35jaLtlzzDzllpwcWxRq
         Mo4/WJUN7/fBOL7571xA6onMrSR84WT+JI4+OJGaKLwq2HH8Aj6H0LKdvj6/dHV4STbR
         TyxEk56Qtue/hNwxFOHEmCe2y67j5WpsNbQe9Q5R13J2TVCWvy5fb4VfeHRCOUlxQigb
         vtgZVxsxdNTFd29s6T+GkWNxQqFXk4jtePAlTNrmGIZz86iGsenRlGSeL86Ct4BairIX
         ai1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967569; x=1717572369;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SILgfa72VI0ajqqfGUDbvcNqJ8ZZiLfW7kFdd1Zm0SE=;
        b=Y78kyyTdgmyCjkUxf0blcYMLeUQqQ7vEMfCNnQcfDwlnwXK+TZTbxF9GRfVrL1mlyr
         /RZktFe1vkCbUNXcf2VTN8rmWEPm8WMLZM2SgvEYrUGlzP4tLPA0sloaj0r2FwIRW6hS
         P3FVal98/pp2pbmmc0Mr4aiNTlydZhZdYIfv6+OiWsf6Qf9PEB1nAb/Tv2QIEgM0VOsL
         pfQ1n8pOtLaDbtmeHc9MLjiBFnZN04cHdqFTZ0n37Ot0D1xjAS6KwCXEW0tVnbmwdWlI
         VULYjptIitvzDMW41RLqt6V7rXPkD2G/IZ+U1cXP+QaSHcUP80N/3AKwMJ0E30jks9Om
         xy2g==
X-Gm-Message-State: AOJu0YzBMnXzEG6TZJB2U0jRsGC+o78anRXPJWEvX3Oa/XTWVJVHJltw
	+oZhd1ihYkqUtWPaCxDlMNJQZgSLH9w+3GJtR91DbJfr6UkhN63dD+5x5UR/GV/kL4Po8kSDNFR
	RERFstg==
X-Google-Smtp-Source: AGHT+IGBGpu4uivcFEM720OCgQ74X83zEPu/U5/K/JeSfte5w6XVAwJSrJjdaWEyuMCcyTnyu21+EA==
X-Received: by 2002:a05:6512:344d:b0:51c:c7ad:329b with SMTP id 2adb3069b0e04-52965c30f7fmr11942347e87.30.1716967569170;
        Wed, 29 May 2024 00:26:09 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH v5 4/7] xen/arm: allow dynamically assigned SGI handlers
Date: Wed, 29 May 2024 09:25:56 +0200
Message-Id: <20240529072559.2486986-5-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Updates so request_irq() can be used with a dynamically assigned SGI irq
as input. This prepares for a later patch where an FF-A schedule
receiver interrupt handler is installed for an SGI generated by the
secure world.

>From the Arm Base System Architecture v1.0C [1]:
"The system shall implement at least eight Non-secure SGIs, assigned to
interrupt IDs 0-7."

gic_route_irq_to_xen() don't gic_set_irq_type() for SGIs since they are
always edge triggered.

gic_interrupt() is updated to route the dynamically assigned SGIs to
do_IRQ() instead of do_sgi(). The latter still handles the statically
assigned SGI handlers like for instance GIC_SGI_CALL_FUNCTION.

[1] https://developer.arm.com/documentation/den0094/

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
v3->v4
- Use IRQ_TYPE_EDGE_RISING instead of DT_IRQ_TYPE_EDGE_RISING

v2->v3
- Rename GIC_SGI_MAX to GIC_SGI_STATIC_MAX and rename do_sgi() to
  do_static_sgi()
- Update comment in setup_irq() to mention that SGI irq_desc is banked
- Add ASSERT() in do_IRQ() that the irq isn't an SGI before injecting
  calling vgic_inject_irq()
- Initialize local_irqs_type[] range for SGIs as IRQ_TYPE_EDGE_RISING
- Adding link to the Arm Base System Architecture v1.0C

v1->v2
- Update patch description as requested
---
 xen/arch/arm/gic.c             | 12 +++++++-----
 xen/arch/arm/include/asm/gic.h |  2 +-
 xen/arch/arm/irq.c             | 18 ++++++++++++++----
 3 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index b3467a76ae75..3eaf670fd731 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -38,7 +38,7 @@ const struct gic_hw_operations *gic_hw_ops;
 static void __init __maybe_unused build_assertions(void)
 {
     /* Check our enum gic_sgi only covers SGIs */
-    BUILD_BUG_ON(GIC_SGI_MAX > NR_GIC_SGI);
+    BUILD_BUG_ON(GIC_SGI_STATIC_MAX > NR_GIC_SGI);
 }
 
 void register_gic_ops(const struct gic_hw_operations *ops)
@@ -117,7 +117,9 @@ void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
 
     desc->handler = gic_hw_ops->gic_host_irq_type;
 
-    gic_set_irq_type(desc, desc->arch.type);
+    /* SGIs are always edge-triggered, so there is need to set it */
+    if ( desc->irq >= NR_GIC_SGI)
+        gic_set_irq_type(desc, desc->arch.type);
     gic_set_irq_priority(desc, priority);
 }
 
@@ -322,7 +324,7 @@ void gic_disable_cpu(void)
     gic_hw_ops->disable_interface();
 }
 
-static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
+static void do_static_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi)
 {
     struct irq_desc *desc = irq_to_desc(sgi);
 
@@ -367,7 +369,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         /* Reading IRQ will ACK it */
         irq = gic_hw_ops->read_irq();
 
-        if ( likely(irq >= 16 && irq < 1020) )
+        if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) )
         {
             isb();
             do_IRQ(regs, irq, is_fiq);
@@ -379,7 +381,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         }
         else if ( unlikely(irq < 16) )
         {
-            do_sgi(regs, irq);
+            do_static_sgi(regs, irq);
         }
         else
         {
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index 03f209529b13..541f0eeb808a 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -285,7 +285,7 @@ enum gic_sgi {
     GIC_SGI_EVENT_CHECK,
     GIC_SGI_DUMP_STATE,
     GIC_SGI_CALL_FUNCTION,
-    GIC_SGI_MAX,
+    GIC_SGI_STATIC_MAX,
 };
 
 /* SGI irq mode types */
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index bcce80a4d624..5224898265a5 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -152,7 +152,13 @@ void __init init_IRQ(void)
 
     spin_lock(&local_irqs_type_lock);
     for ( irq = 0; irq < NR_LOCAL_IRQS; irq++ )
-        local_irqs_type[irq] = IRQ_TYPE_INVALID;
+    {
+        /* SGIs are always edge-triggered */
+        if ( irq < NR_GIC_SGI )
+            local_irqs_type[irq] = IRQ_TYPE_EDGE_RISING;
+        else
+            local_irqs_type[irq] = IRQ_TYPE_INVALID;
+    }
     spin_unlock(&local_irqs_type_lock);
 
     BUG_ON(init_local_irq_data(smp_processor_id()) < 0);
@@ -224,9 +230,12 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
 
     perfc_incr(irqs);
 
-    ASSERT(irq >= 16); /* SGIs do not come down this path */
+    /* Statically assigned SGIs do not come down this path */
+    ASSERT(irq >= GIC_SGI_STATIC_MAX);
 
-    if ( irq < 32 )
+    if ( irq < NR_GIC_SGI )
+        perfc_incr(ipis);
+    else if ( irq < NR_GIC_LOCAL_IRQS )
         perfc_incr(ppis);
     else
         perfc_incr(spis);
@@ -260,6 +269,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
          * The irq cannot be a PPI, we only support delivery of SPIs to
          * guests.
          */
+        ASSERT(irq >= NR_GIC_SGI);
         vgic_inject_irq(info->d, NULL, info->virq, true);
         goto out_no_end;
     }
@@ -396,7 +406,7 @@ int setup_irq(unsigned int irq, unsigned int irqflags, struct irqaction *new)
     {
         gic_route_irq_to_xen(desc, GIC_PRI_IRQ);
         /* It's fine to use smp_processor_id() because:
-         * For PPI: irq_desc is banked
+         * For SGI and PPI: irq_desc is banked
          * For SPI: we don't care for now which CPU will receive the
          * interrupt
          * TODO: Handle case where SPI is setup on different CPU than
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731500.1137062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhN-0004YU-NF; Wed, 29 May 2024 07:26:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731500.1137062; Wed, 29 May 2024 07:26:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhN-0004Xs-Gp; Wed, 29 May 2024 07:26:13 +0000
Received: by outflank-mailman (input) for mailman id 731500;
 Wed, 29 May 2024 07:26:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhM-0003Qb-Ar
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:12 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b97cdd2f-1d8c-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 09:26:11 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52ae14e78e5so352601e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:11 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b97cdd2f-1d8c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967571; x=1717572371; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yqcaM/22lmEA4Im1sZdSJGF9/Zga7zG5i6aY5+pENkc=;
        b=JoqR5twlnKnlx641Z91rxxU7T2zsze1jbWPyq7jV7bIcw8vMuN6wJR7Q3SoSsE4Y1Q
         xdzm6RUEM/vpArvjzXs7plobqRFoJ26zqQQ6Kbcn+e1NARTno2gd6YPczuNFYxvsWCpJ
         2fvYGpX0AoOlqatB4YnI+w54CeDVmCCJ1LZo8oiMuuj+GklOb/nJhbD5aWdEjNXKAJ/N
         ioHdR53nLpu0+kjZh+pCe8o5HnJk3vPj59hGIgM8cG43lU/3mUWNH/pyIIzhGHrV3PsY
         X7pUgi/komKUFXjbQR8Iwpt2IQxeUp+RoSkLnya1PpFEke9SXqXfwVk7l+FfRtWt4DMV
         Rx7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967571; x=1717572371;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=yqcaM/22lmEA4Im1sZdSJGF9/Zga7zG5i6aY5+pENkc=;
        b=v2A/XgjRNFAKdaaFu6kZlF40djo64q5chEUGkphmnxh+hD31caEXmCIbEYrLMu1hIa
         PltF7hUaIBJCWkXC9vQTXaX6te3JtSkYfpf2lVD5vEKScjN7P6qEVmYOS8Ki/3Enyom5
         J675lGnu6pSkA/lVkTw8DjIfnB6awx4GAoD0Uz1/mZ3mnwGP2IVwlO6rnU8O2GzGaPU9
         de3PrtDZ/yGXScTFHsVzltu/M73J9JrtiB50VNA8+GzrDbcr2N8Zb5cUX9U2qFGnAoat
         RUvIiPxBa/0LaTqMDXFMYWL3AZ74BybhC0q0mNGEXxw93+ASDI4W+FY6hHncC0JB3v8K
         z+GQ==
X-Gm-Message-State: AOJu0YxdRXJ9dwOFy18PpGS3BGs8K4Agd+W2v8luHbtLe9UfK58fuNsi
	NZwtfkg9WoVKDzKRKN36v08y5PgR6a0FeWfBJiQMaB8wib0at0ifkk0Wlny9pbu1Ij9DdW9fRO7
	IRfo=
X-Google-Smtp-Source: AGHT+IHk4ObidGwCD812vZGe+Z1dmGa50LEp02UVipqGGD3NRtyQPamoN4Tga8ZC5srPW8Wo0VdLlA==
X-Received: by 2002:a05:6512:eaa:b0:51c:d6c9:e9a3 with SMTP id 2adb3069b0e04-52964aba890mr10135809e87.17.1716967570797;
        Wed, 29 May 2024 00:26:10 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 5/7] xen/arm: add and call init_tee_interrupt()
Date: Wed, 29 May 2024 09:25:57 +0200
Message-Id: <20240529072559.2486986-6-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add init_tee_interrupt() to the TEE mediator framework and call it from
start_secondary() late enough that per-cpu interrupts can be configured
on CPUs as they are initialized. This is needed in later patches.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
 xen/arch/arm/include/asm/tee/tee.h | 8 ++++++++
 xen/arch/arm/smpboot.c             | 2 ++
 xen/arch/arm/tee/tee.c             | 6 ++++++
 3 files changed, 16 insertions(+)

diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/asm/tee/tee.h
index da324467e130..eda8a8aa38f2 100644
--- a/xen/arch/arm/include/asm/tee/tee.h
+++ b/xen/arch/arm/include/asm/tee/tee.h
@@ -28,6 +28,9 @@ struct tee_mediator_ops {
      */
     bool (*probe)(void);
 
+    /* Initialize eventual interrupt handlers on the secondary CPUs */
+    void (*init_interrupt)(void);
+
     /*
      * Called during domain construction if toolstack requests to enable
      * TEE support so mediator can inform TEE about new
@@ -66,6 +69,7 @@ int tee_domain_init(struct domain *d, uint16_t tee_type);
 int tee_domain_teardown(struct domain *d);
 int tee_relinquish_resources(struct domain *d);
 uint16_t tee_get_type(void);
+void init_tee_interrupt(void);
 
 #define REGISTER_TEE_MEDIATOR(_name, _namestr, _type, _ops)         \
 static const struct tee_mediator_desc __tee_desc_##_name __used     \
@@ -105,6 +109,10 @@ static inline uint16_t tee_get_type(void)
     return XEN_DOMCTL_CONFIG_TEE_NONE;
 }
 
+static inline void init_tee_interrupt(void)
+{
+}
+
 #endif  /* CONFIG_TEE */
 
 #endif /* __ARCH_ARM_TEE_TEE_H__ */
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 93a10d7721b4..e1c1e20efd98 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -29,6 +29,7 @@
 #include <asm/procinfo.h>
 #include <asm/psci.h>
 #include <asm/acpi.h>
+#include <asm/tee/tee.h>
 
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
@@ -401,6 +402,7 @@ void asmlinkage start_secondary(void)
      */
     init_maintenance_interrupt();
     init_timer_interrupt();
+    init_tee_interrupt();
 
     local_abort_enable();
 
diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
index ddd17506a9ff..8117fd55123e 100644
--- a/xen/arch/arm/tee/tee.c
+++ b/xen/arch/arm/tee/tee.c
@@ -96,6 +96,12 @@ static int __init tee_init(void)
 
 __initcall(tee_init);
 
+void __init init_tee_interrupt(void)
+{
+    if ( cur_mediator && cur_mediator->ops->init_interrupt)
+        cur_mediator->ops->init_interrupt();
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731501.1137076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhQ-000570-S4; Wed, 29 May 2024 07:26:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731501.1137076; Wed, 29 May 2024 07:26:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhQ-00056q-On; Wed, 29 May 2024 07:26:16 +0000
Received: by outflank-mailman (input) for mailman id 731501;
 Wed, 29 May 2024 07:26:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhP-0002xp-7I
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:15 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b99d04ef-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:26:11 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-529682e013dso2045289e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:13 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b99d04ef-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967572; x=1717572372; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Px3daIYvC5d3ihWRjkhDysb/DJOq/aAiZUJCtozW1Ns=;
        b=LA4eHj7skT1HsI4uWzgRhfNAZo7SnBYMQNsx5agvJ7gwsb2J/Vqw42wt77K1J+G4Bu
         fgONsnWw+w7ftjUdfSChH4wqXHGy6q8CPGPkbPi0ii0QtZytxVMWQjHzHnvvkwT1nj2+
         reBb9PO4UmgZIkwW2o4psRdg2RbrrtdAVfeA+AkNyhygb8Kp2y3+31xb3pXAAuFeKpgO
         IQB8/V4DdYcmXNC5SvTGTYbWWj92TPPhjJraWf0FdyPWBZE/gnZuYpiN4UY6MxdzvuwW
         tPH0wt8WmvT5f+Vyv5HzGOebGqARxyT53TU9oXaO/r3yL6SZVdHWNkRNsJwwJ40fqOzl
         pQ7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967572; x=1717572372;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Px3daIYvC5d3ihWRjkhDysb/DJOq/aAiZUJCtozW1Ns=;
        b=uImUovKYjA4c/TLcB5obFRmLO9XJ0nqEpPD9VkoSCzJ3P5XA8P5w3xZSq3HTy2WqsG
         KywJnvLrho4EEpP/tyHBCOIxjJnNDl4KsWAvZaNs8JvpcYZAIo8QLpGJWg+OLI9Lk9qB
         M65OPS2PjkpG5G3elhYipK7RbBkf1ilmwUn2LT1wITmLCT61XxHhOnvXfK/akkgzZKFz
         Yq7Gk/pXBNUkOtUUwdH9cd85cRbJSak0CDJSWoCFiiPfy5D/XRGspY9jNJlL9q63A763
         XNuktXSCKwF/jSI3XxHzkPRls4UyuEydkShzqSWzN0wfHxpbCwFy4UR18Ywwof6cbwLA
         ADOg==
X-Gm-Message-State: AOJu0Yy3LtSophNl45IhMU0Iz941RBumdL3G1DijDCpWFwjG/QEeGEmg
	oOrqh/bd9q1o+6tAFqSH2/MDlY1+4IJAarIEVRriWRiWJQZFi6Z4SCnHHqkrr/fl+nKS5WPeReF
	1ICA=
X-Google-Smtp-Source: AGHT+IHZbnpYtnOOu1mZzARWcnHwDkNNpf8A2L3wrm68sMOOMmvSDs1EjYrgF4UQT+PbkcE6gf4Tyg==
X-Received: by 2002:ac2:5dd3:0:b0:521:a96:bf15 with SMTP id 2adb3069b0e04-52965198d34mr8322912e87.38.1716967572307;
        Wed, 29 May 2024 00:26:12 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH v5 6/7] xen/arm: add and call tee_free_domain_ctx()
Date: Wed, 29 May 2024 09:25:58 +0200
Message-Id: <20240529072559.2486986-7-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add tee_free_domain_ctx() to the TEE mediator framework.
tee_free_domain_ctx() is called from arch_domain_destroy() to allow late
freeing of the d->arch.tee context. This will simplify access to
d->arch.tee for domains retrieved with rcu_lock_domain_by_id().

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
 xen/arch/arm/domain.c              | 1 +
 xen/arch/arm/include/asm/tee/tee.h | 6 ++++++
 xen/arch/arm/tee/tee.c             | 6 ++++++
 3 files changed, 13 insertions(+)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 34cbfe699a68..61e46a157ccc 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -837,6 +837,7 @@ int arch_domain_teardown(struct domain *d)
 
 void arch_domain_destroy(struct domain *d)
 {
+    tee_free_domain_ctx(d);
     /* IOMMU page table is shared with P2M, always call
      * iommu_domain_destroy() before p2m_final_teardown().
      */
diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/asm/tee/tee.h
index eda8a8aa38f2..2e99a38184be 100644
--- a/xen/arch/arm/include/asm/tee/tee.h
+++ b/xen/arch/arm/include/asm/tee/tee.h
@@ -38,6 +38,7 @@ struct tee_mediator_ops {
      */
     int (*domain_init)(struct domain *d);
     int (*domain_teardown)(struct domain *d);
+    void (*free_domain_ctx)(struct domain *d);
 
     /*
      * Called during domain destruction to relinquish resources used
@@ -70,6 +71,7 @@ int tee_domain_teardown(struct domain *d);
 int tee_relinquish_resources(struct domain *d);
 uint16_t tee_get_type(void);
 void init_tee_interrupt(void);
+void tee_free_domain_ctx(struct domain *d);
 
 #define REGISTER_TEE_MEDIATOR(_name, _namestr, _type, _ops)         \
 static const struct tee_mediator_desc __tee_desc_##_name __used     \
@@ -113,6 +115,10 @@ static inline void init_tee_interrupt(void)
 {
 }
 
+static inline void tee_free_domain_ctx(struct domain *d)
+{
+}
+
 #endif  /* CONFIG_TEE */
 
 #endif /* __ARCH_ARM_TEE_TEE_H__ */
diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
index 8117fd55123e..cb65f187f51f 100644
--- a/xen/arch/arm/tee/tee.c
+++ b/xen/arch/arm/tee/tee.c
@@ -102,6 +102,12 @@ void __init init_tee_interrupt(void)
         cur_mediator->ops->init_interrupt();
 }
 
+void tee_free_domain_ctx(struct domain *d)
+{
+    if ( cur_mediator && cur_mediator->ops->free_domain_ctx)
+        cur_mediator->ops->free_domain_ctx(d);
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:26:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:26:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731504.1137087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhT-0005YD-9A; Wed, 29 May 2024 07:26:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731504.1137087; Wed, 29 May 2024 07:26:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDhT-0005XA-34; Wed, 29 May 2024 07:26:19 +0000
Received: by outflank-mailman (input) for mailman id 731504;
 Wed, 29 May 2024 07:26:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T+wO=NA=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1sCDhR-0002xp-K8
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:26:17 +0000
Received: from mail-lf1-x142.google.com (mail-lf1-x142.google.com
 [2a00:1450:4864:20::142])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id baaca1c1-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:26:13 +0200 (CEST)
Received: by mail-lf1-x142.google.com with SMTP id
 2adb3069b0e04-5241b49c0daso2037241e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:26:15 -0700 (PDT)
Received: from localhost.localdomain ([2001:2043:5e37:9d00:b09:3200:dd72:cfc8])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5296ee4a58fsm1182353e87.67.2024.05.29.00.26.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:26:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: baaca1c1-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1716967574; x=1717572374; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KaYVHgDBDb/dcSaigpLjukQO1YXkyiCDeKoH2jDdqdU=;
        b=fPuYR+aCZFeZEX7cxx8imf3UL3TP4JXnZNru7p6LpcAXefpSvHz01nPKdE0dTsYvBn
         v6aOiy8Nc3TjV+eTmWx8xQsrZAeu1uptC5ay8QnzVqkPlZr/oxCjzyyhUgV+IHAImhYS
         ZDbA3o6qaLnCJz0QrDyLv98fNzCGBLTBPIoE3VRrKEXkifidEHsYiF5qXm0J9eYZzARH
         hAIzNq7LB1BLgQXCc4o/gdWCQ+3NjdDZYo6+Z6dFOkowTcmAC3yyBcdAyOaI2mzVyWfY
         nDphTS1WmBA6yQ2mKgWmU/HQDCubsriEPESgnv0V0wDeP0+zqf7yhkq6syJM8/AGqVci
         YyuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967574; x=1717572374;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KaYVHgDBDb/dcSaigpLjukQO1YXkyiCDeKoH2jDdqdU=;
        b=K4gqPMGLsIgZHqDTSk8CtYoku+PVhq213QQN5jfX+eOIzLi7THs1l09qlavyEQzq0b
         cuTxkstsl4rS7JzvDg4UFcQ5REUOQV3RJQW3KohBSM96nG2QRyQGyPp/5tKR3DmmKUnm
         i/WdQ736wvDq8P272uKZXBxYbGEG01X5H4fZhKiU1GgeqrNR9/A8jNQ5MiFVeEju5xIm
         yH4vchZs2l72AYbtacGyThml1fDnypotzD7Boxmj1c+/Ae0kZ0ZlBC1rL4xcE9APeE5D
         l9FOKxyqJczSltz+xGGE90PsljUumEDtQG8p7fv8u5NUox2lAgc/28nAYXCUxRDdlTnh
         FWZQ==
X-Gm-Message-State: AOJu0YyD4CarN72AWWgE2OjHZs3lKkk5ogMTrQFvuDgKdej5FmXrZBAv
	GqMPhYdEz3P2Skxi+MXC+WZwHafJUzc64nQalwaV+bXETuRiiEgXzSUuFDGjtQ6GCdPoj0Dj47G
	bSu4PQA==
X-Google-Smtp-Source: AGHT+IGDjJw1+iUK1GJvCm2xiDFsxGaereF2OKqYlw8CutyGSEtt28lcpc5JDhpdakdMJJ06xROlVQ==
X-Received: by 2002:ac2:4f8a:0:b0:519:6953:2ffc with SMTP id 2adb3069b0e04-52966ca64bbmr8145309e87.42.1716967573859;
        Wed, 29 May 2024 00:26:13 -0700 (PDT)
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [XEN PATCH v5 7/7] xen/arm: ffa: support notification
Date: Wed, 29 May 2024 09:25:59 +0200
Message-Id: <20240529072559.2486986-8-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529072559.2486986-1-jens.wiklander@linaro.org>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add support for FF-A notifications, currently limited to an SP (Secure
Partition) sending an asynchronous notification to a guest.

Guests and Xen itself are made aware of pending notifications with an
interrupt. The interrupt handler triggers a tasklet to retrieve the
notifications using the FF-A ABI and deliver them to their destinations.

Update ffa_partinfo_domain_init() to return error code like
ffa_notif_domain_init().

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---
v4->v5:
- Move the freeing of d->arch.tee to the new TEE mediator free_domain_ctx
  callback to make the context accessible during rcu_lock_domain_by_id()
  from a tasklet
- Initialize interrupt handlers for secondary CPUs from the new TEE mediator
  init_interrupt() callback
- Restore the ffa_probe() from v3, that is, remove the
  presmp_initcall(ffa_init) approach and use ffa_probe() as usual now that we
  have the init_interrupt callback.
- A tasklet is added to handle the Schedule Receiver interrupt. The tasklet
  finds each relevant domain with rcu_lock_domain_by_id() which now is enough
  to guarantee that the FF-A context can be accessed.
- The notification interrupt handler only schedules the notification
  tasklet mentioned above

v3->v4:
- Add another note on FF-A limitations
- Clear secure_pending in ffa_handle_notification_get() if both SP and SPM
  bitmaps are retrieved
- ASSERT that ffa_rcu_lock_domain_by_vm_id() isn't passed the vm_id FF-A
  uses for Xen itself
- Replace the get_domain_by_id() call done via ffa_get_domain_by_vm_id() in
  notif_irq_handler() with a call to rcu_lock_live_remote_domain_by_id() via
  ffa_rcu_lock_domain_by_vm_id()
- Remove spinlock in struct ffa_ctx_notif and use atomic functions as needed
  to access and update the secure_pending field
- In notif_irq_handler(), look for the first online CPU instead of assuming
  that the first CPU is online
- Initialize FF-A via presmp_initcall() before the other CPUs are online,
  use register_cpu_notifier() to install the interrupt handler
  notif_irq_handler()
- Update commit message to reflect recent updates

v2->v3:
- Add a GUEST_ prefix and move FFA_NOTIF_PEND_INTR_ID and
  FFA_SCHEDULE_RECV_INTR_ID to public/arch-arm.h
- Register the Xen SRI handler on each CPU using on_selected_cpus() and
  setup_irq()
- Check that the SGI ID retrieved with FFA_FEATURE_SCHEDULE_RECV_INTR
  doesn't conflict with static SGI handlers

v1->v2:
- Addressing review comments
- Change ffa_handle_notification_{bind,unbind,set}() to take struct
  cpu_user_regs *regs as argument.
- Update ffa_partinfo_domain_init() and ffa_notif_domain_init() to return
  an error code.
- Fixing a bug in handle_features() for FFA_FEATURE_SCHEDULE_RECV_INTR.
---
 xen/arch/arm/tee/Makefile       |   1 +
 xen/arch/arm/tee/ffa.c          |  72 +++++-
 xen/arch/arm/tee/ffa_notif.c    | 409 ++++++++++++++++++++++++++++++++
 xen/arch/arm/tee/ffa_partinfo.c |   9 +-
 xen/arch/arm/tee/ffa_private.h  |  56 ++++-
 xen/arch/arm/tee/tee.c          |   2 +-
 xen/include/public/arch-arm.h   |  14 ++
 7 files changed, 548 insertions(+), 15 deletions(-)
 create mode 100644 xen/arch/arm/tee/ffa_notif.c

diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
index f0112a2f922d..7c0f46f7f446 100644
--- a/xen/arch/arm/tee/Makefile
+++ b/xen/arch/arm/tee/Makefile
@@ -2,5 +2,6 @@ obj-$(CONFIG_FFA) += ffa.o
 obj-$(CONFIG_FFA) += ffa_shm.o
 obj-$(CONFIG_FFA) += ffa_partinfo.o
 obj-$(CONFIG_FFA) += ffa_rxtx.o
+obj-$(CONFIG_FFA) += ffa_notif.o
 obj-y += tee.o
 obj-$(CONFIG_OPTEE) += optee.o
diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 5209612963e1..d644ee97cd5b 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -39,6 +39,12 @@
  *   - at most 32 shared memory regions per guest
  * o FFA_MSG_SEND_DIRECT_REQ:
  *   - only supported from a VM to an SP
+ * o FFA_NOTIFICATION_*:
+ *   - only supports global notifications, that is, per vCPU notifications
+ *     are not supported
+ *   - doesn't support signalling the secondary scheduler of pending
+ *     notification for secure partitions
+ *   - doesn't support notifications for Xen itself
  *
  * There are some large locked sections with ffa_tx_buffer_lock and
  * ffa_rx_buffer_lock. Especially the ffa_tx_buffer_lock spinlock used
@@ -194,6 +200,8 @@ out:
 
 static void handle_features(struct cpu_user_regs *regs)
 {
+    struct domain *d = current->domain;
+    struct ffa_ctx *ctx = d->arch.tee;
     uint32_t a1 = get_user_reg(regs, 1);
     unsigned int n;
 
@@ -240,6 +248,30 @@ static void handle_features(struct cpu_user_regs *regs)
         BUILD_BUG_ON(PAGE_SIZE != FFA_PAGE_SIZE);
         ffa_set_regs_success(regs, 0, 0);
         break;
+    case FFA_FEATURE_NOTIF_PEND_INTR:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, GUEST_FFA_NOTIF_PEND_INTR_ID, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
+    case FFA_FEATURE_SCHEDULE_RECV_INTR:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, GUEST_FFA_SCHEDULE_RECV_INTR_ID, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
+
+    case FFA_NOTIFICATION_BIND:
+    case FFA_NOTIFICATION_UNBIND:
+    case FFA_NOTIFICATION_GET:
+    case FFA_NOTIFICATION_SET:
+    case FFA_NOTIFICATION_INFO_GET_32:
+    case FFA_NOTIFICATION_INFO_GET_64:
+        if ( ctx->notif.enabled )
+            ffa_set_regs_success(regs, 0, 0);
+        else
+            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        break;
     default:
         ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
         break;
@@ -305,6 +337,22 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
                                                      get_user_reg(regs, 1)),
                                    get_user_reg(regs, 3));
         break;
+    case FFA_NOTIFICATION_BIND:
+        e = ffa_handle_notification_bind(regs);
+        break;
+    case FFA_NOTIFICATION_UNBIND:
+        e = ffa_handle_notification_unbind(regs);
+        break;
+    case FFA_NOTIFICATION_INFO_GET_32:
+    case FFA_NOTIFICATION_INFO_GET_64:
+        ffa_handle_notification_info_get(regs);
+        return true;
+    case FFA_NOTIFICATION_GET:
+        ffa_handle_notification_get(regs);
+        return true;
+    case FFA_NOTIFICATION_SET:
+        e = ffa_handle_notification_set(regs);
+        break;
 
     default:
         gprintk(XENLOG_ERR, "ffa: unhandled fid 0x%x\n", fid);
@@ -322,6 +370,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
 static int ffa_domain_init(struct domain *d)
 {
     struct ffa_ctx *ctx;
+    int ret;
 
     if ( !ffa_version )
         return -ENODEV;
@@ -345,10 +394,11 @@ static int ffa_domain_init(struct domain *d)
      * error, so no need for cleanup in this function.
      */
 
-    if ( !ffa_partinfo_domain_init(d) )
-        return -EIO;
+    ret = ffa_partinfo_domain_init(d);
+    if ( ret )
+        return ret;
 
-    return 0;
+    return ffa_notif_domain_init(d);
 }
 
 static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_time)
@@ -376,13 +426,6 @@ static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_time)
     }
     else
     {
-        /*
-         * domain_destroy() might have been called (via put_domain() in
-         * ffa_reclaim_shms()), so we can't touch the domain structure
-         * anymore.
-         */
-        xfree(ctx);
-
         /* Only check if there has been a change to the teardown queue */
         if ( !first_time )
         {
@@ -423,12 +466,18 @@ static int ffa_domain_teardown(struct domain *d)
         return 0;
 
     ffa_rxtx_domain_destroy(d);
+    ffa_notif_domain_destroy(d);
 
     ffa_domain_teardown_continue(ctx, true /* first_time */);
 
     return 0;
 }
 
+static void ffa_free_domain_ctx(struct domain *d)
+{
+    XFREE(d->arch.tee);
+}
+
 static int ffa_relinquish_resources(struct domain *d)
 {
     return 0;
@@ -502,6 +551,7 @@ static bool ffa_probe(void)
     if ( !ffa_partinfo_init() )
         goto err_rxtx_destroy;
 
+    ffa_notif_init();
     INIT_LIST_HEAD(&ffa_teardown_head);
     init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0);
 
@@ -517,8 +567,10 @@ err_rxtx_destroy:
 static const struct tee_mediator_ops ffa_ops =
 {
     .probe = ffa_probe,
+    .init_interrupt = ffa_notif_init_interrupt,
     .domain_init = ffa_domain_init,
     .domain_teardown = ffa_domain_teardown,
+    .free_domain_ctx = ffa_free_domain_ctx,
     .relinquish_resources = ffa_relinquish_resources,
     .handle_call = ffa_handle_call,
 };
diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c
new file mode 100644
index 000000000000..e8e8b62590b3
--- /dev/null
+++ b/xen/arch/arm/tee/ffa_notif.c
@@ -0,0 +1,409 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024  Linaro Limited
+ */
+
+#include <xen/const.h>
+#include <xen/cpu.h>
+#include <xen/list.h>
+#include <xen/notifier.h>
+#include <xen/spinlock.h>
+#include <xen/tasklet.h>
+#include <xen/types.h>
+
+#include <asm/smccc.h>
+#include <asm/regs.h>
+
+#include "ffa_private.h"
+
+static bool __ro_after_init notif_enabled;
+static unsigned int __ro_after_init notif_sri_irq;
+
+int ffa_handle_notification_bind(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst & 0xFFFFU) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    if ( flags )    /* Only global notifications are supported */
+        return FFA_RET_DENIED;
+
+    /*
+     * We only support notifications from SP so no need to check the sender
+     * endpoint ID, the SPMC will take care of that for us.
+     */
+    return ffa_simple_call(FFA_NOTIFICATION_BIND, src_dst, flags, bitmap_hi,
+                           bitmap_lo);
+}
+
+int ffa_handle_notification_unbind(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst & 0xFFFFU) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    /*
+     * We only support notifications from SP so no need to check the
+     * destination endpoint ID, the SPMC will take care of that for us.
+     */
+    return  ffa_simple_call(FFA_NOTIFICATION_UNBIND, src_dst, 0, bitmap_hi,
+                            bitmap_lo);
+}
+
+void ffa_handle_notification_info_get(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( !notif_enabled )
+    {
+        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        return;
+    }
+
+    if ( test_and_clear_bool(ctx->notif.secure_pending) )
+    {
+        /* A pending global notification for the guest */
+        ffa_set_regs(regs, FFA_SUCCESS_64, 0,
+                     1U << FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT, ffa_get_vm_id(d),
+                     0, 0, 0, 0);
+    }
+    else
+    {
+        /* Report an error if there where no pending global notification */
+        ffa_set_regs_error(regs, FFA_RET_NO_DATA);
+    }
+}
+
+void ffa_handle_notification_get(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t recv = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t w2 = 0;
+    uint32_t w3 = 0;
+    uint32_t w4 = 0;
+    uint32_t w5 = 0;
+    uint32_t w6 = 0;
+    uint32_t w7 = 0;
+
+    if ( !notif_enabled )
+    {
+        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
+        return;
+    }
+
+    if ( (recv & 0xFFFFU) != ffa_get_vm_id(d) )
+    {
+        ffa_set_regs_error(regs, FFA_RET_INVALID_PARAMETERS);
+        return;
+    }
+
+    if ( flags & ( FFA_NOTIF_FLAG_BITMAP_SP | FFA_NOTIF_FLAG_BITMAP_SPM ) )
+    {
+        struct arm_smccc_1_2_regs arg = {
+            .a0 = FFA_NOTIFICATION_GET,
+            .a1 = recv,
+            .a2 = flags & ( FFA_NOTIF_FLAG_BITMAP_SP |
+                            FFA_NOTIF_FLAG_BITMAP_SPM ),
+        };
+        struct arm_smccc_1_2_regs resp;
+        int32_t e;
+
+        /*
+         * Clear secure pending if both FFA_NOTIF_FLAG_BITMAP_SP and
+         * FFA_NOTIF_FLAG_BITMAP_SPM are set since secure world can't have
+         * any more pending notifications.
+         */
+        if ( ( flags  & FFA_NOTIF_FLAG_BITMAP_SP ) &&
+             ( flags & FFA_NOTIF_FLAG_BITMAP_SPM ) )
+        {
+                struct ffa_ctx *ctx = d->arch.tee;
+
+                ACCESS_ONCE(ctx->notif.secure_pending) = false;
+        }
+
+        arm_smccc_1_2_smc(&arg, &resp);
+        e = ffa_get_ret_code(&resp);
+        if ( e )
+        {
+            ffa_set_regs_error(regs, e);
+            return;
+        }
+
+        if ( flags & FFA_NOTIF_FLAG_BITMAP_SP )
+        {
+            w2 = resp.a2;
+            w3 = resp.a3;
+        }
+
+        if ( flags & FFA_NOTIF_FLAG_BITMAP_SPM )
+            w6 = resp.a6;
+    }
+
+    ffa_set_regs(regs, FFA_SUCCESS_32, 0, w2, w3, w4, w5, w6, w7);
+}
+
+int ffa_handle_notification_set(struct cpu_user_regs *regs)
+{
+    struct domain *d = current->domain;
+    uint32_t src_dst = get_user_reg(regs, 1);
+    uint32_t flags = get_user_reg(regs, 2);
+    uint32_t bitmap_lo = get_user_reg(regs, 3);
+    uint32_t bitmap_hi = get_user_reg(regs, 4);
+
+    if ( !notif_enabled )
+        return FFA_RET_NOT_SUPPORTED;
+
+    if ( (src_dst >> 16) != ffa_get_vm_id(d) )
+        return FFA_RET_INVALID_PARAMETERS;
+
+    /* Let the SPMC check the destination of the notification */
+    return ffa_simple_call(FFA_NOTIFICATION_SET, src_dst, flags, bitmap_lo,
+                           bitmap_hi);
+}
+
+/*
+ * Extract a 16-bit ID (index n) from the successful return value from
+ * FFA_NOTIFICATION_INFO_GET_64 or FFA_NOTIFICATION_INFO_GET_32. IDs are
+ * returned in registers 3 to 7 with four IDs per register for 64-bit
+ * calling convention and two IDs per register for 32-bit calling
+ * convention.
+ */
+static uint16_t get_id_from_resp(struct arm_smccc_1_2_regs *resp,
+                                 unsigned int n)
+{
+    unsigned int ids_per_reg;
+    unsigned int reg_idx;
+    unsigned int reg_shift;
+
+    if ( smccc_is_conv_64(resp->a0) )
+        ids_per_reg = 4;
+    else
+        ids_per_reg = 2;
+
+    reg_idx = n / ids_per_reg + 3;
+    reg_shift = ( n % ids_per_reg ) * 16;
+
+    switch ( reg_idx )
+    {
+    case 3:
+        return resp->a3 >> reg_shift;
+    case 4:
+        return resp->a4 >> reg_shift;
+    case 5:
+        return resp->a5 >> reg_shift;
+    case 6:
+        return resp->a6 >> reg_shift;
+    case 7:
+        return resp->a7 >> reg_shift;
+    default:
+        ASSERT(0); /* "Can't happen" */
+        return 0;
+    }
+}
+
+static void notif_vm_pend_intr(uint16_t vm_id)
+{
+    struct ffa_ctx *ctx;
+    struct domain *d;
+    struct vcpu *v;
+
+    /*
+     * vm_id == 0 means a notifications pending for Xen itself, but
+     * we don't support that yet.
+     */
+    if ( !vm_id )
+        return;
+
+    d = ffa_rcu_lock_domain_by_vm_id(vm_id);
+    if ( !d )
+        return;
+
+    ctx = d->arch.tee;
+    if ( !ctx )
+        goto out_unlock;
+
+    /*
+     * arch.tee is freed from complete_domain_destroy() so the RCU lock
+     * guarantees that the data structure isn't freed while we're accessing
+     * it.
+     */
+    ACCESS_ONCE(ctx->notif.secure_pending) = true;
+
+    /*
+     * Since we're only delivering global notification, always
+     * deliver to the first online vCPU. It doesn't matter
+     * which we chose, as long as it's available.
+     */
+    for_each_vcpu(d, v)
+    {
+        if ( is_vcpu_online(v) )
+        {
+            vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INTR_ID,
+                            true);
+            break;
+        }
+    }
+    if ( !v )
+        printk(XENLOG_ERR "ffa: can't inject NPI, all vCPUs offline\n");
+
+out_unlock:
+    rcu_unlock_domain(d);
+}
+
+static void notif_sri_action(void *unused)
+{
+    const struct arm_smccc_1_2_regs arg = {
+        .a0 = FFA_NOTIFICATION_INFO_GET_64,
+    };
+    struct arm_smccc_1_2_regs resp;
+    unsigned int id_pos;
+    unsigned int list_count;
+    uint64_t ids_count;
+    unsigned int n;
+    int32_t res;
+
+    do {
+        arm_smccc_1_2_smc(&arg, &resp);
+        res = ffa_get_ret_code(&resp);
+        if ( res )
+        {
+            if ( res != FFA_RET_NO_DATA )
+                printk(XENLOG_ERR "ffa: notification info get failed: error %d\n",
+                       res);
+            return;
+        }
+
+        ids_count = resp.a2 >> FFA_NOTIF_INFO_GET_ID_LIST_SHIFT;
+        list_count = ( resp.a2 >> FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT ) &
+                     FFA_NOTIF_INFO_GET_ID_COUNT_MASK;
+
+        id_pos = 0;
+        for ( n = 0; n < list_count; n++ )
+        {
+            unsigned int count = ((ids_count >> 2 * n) & 0x3) + 1;
+            uint16_t vm_id = get_id_from_resp(&resp, id_pos);
+
+            notif_vm_pend_intr(vm_id);
+
+            id_pos += count;
+        }
+
+    } while (resp.a2 & FFA_NOTIF_INFO_GET_MORE_FLAG);
+}
+
+static DECLARE_TASKLET(notif_sri_tasklet, notif_sri_action, NULL);
+
+static void notif_irq_handler(int irq, void *data)
+{
+    tasklet_schedule(&notif_sri_tasklet);
+}
+
+static int32_t ffa_notification_bitmap_create(uint16_t vm_id,
+                                              uint32_t vcpu_count)
+{
+    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_CREATE, vm_id, vcpu_count,
+                           0, 0);
+}
+
+static int32_t ffa_notification_bitmap_destroy(uint16_t vm_id)
+{
+    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0, 0);
+}
+
+void ffa_notif_init_interrupt(void)
+{
+    int ret;
+
+    if ( notif_enabled && notif_sri_irq < NR_GIC_SGI )
+    {
+        /*
+         * An error here is unlikely since the primary CPU has already
+         * succeeded in installing the interrupt handler. If this fails it
+         * may lead to a problem with notifictaions.
+         *
+         * The CPUs without an notification handler installed will fail to
+         * trigger on the SGI indicating that there are notifications
+         * pending, while the SPMC in the secure world will not notice that
+         * the interrupt was lost.
+         */
+        ret = request_irq(notif_sri_irq, 0, notif_irq_handler, "FF-A notif",
+                          NULL);
+        if ( ret )
+            printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n",
+                   notif_sri_irq, ret);
+    }
+}
+
+void ffa_notif_init(void)
+{
+    const struct arm_smccc_1_2_regs arg = {
+        .a0 = FFA_FEATURES,
+        .a1 = FFA_FEATURE_SCHEDULE_RECV_INTR,
+    };
+    struct arm_smccc_1_2_regs resp;
+    unsigned int irq;
+    int ret;
+
+    arm_smccc_1_2_smc(&arg, &resp);
+    if ( resp.a0 != FFA_SUCCESS_32 )
+        return;
+
+    irq = resp.a2;
+    notif_sri_irq = irq;
+    if ( irq >= NR_GIC_SGI )
+        irq_set_type(irq, IRQ_TYPE_EDGE_RISING);
+    ret = request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL);
+    if ( ret )
+    {
+        printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n",
+               irq, ret);
+        return;
+    }
+
+    notif_enabled = true;
+}
+
+int ffa_notif_domain_init(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+    int32_t res;
+
+    if ( !notif_enabled )
+        return 0;
+
+    res = ffa_notification_bitmap_create(ffa_get_vm_id(d), d->max_vcpus);
+    if ( res )
+        return -ENOMEM;
+
+    ctx->notif.enabled = true;
+
+    return 0;
+}
+
+void ffa_notif_domain_destroy(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( ctx->notif.enabled )
+    {
+        ffa_notification_bitmap_destroy(ffa_get_vm_id(d));
+        ctx->notif.enabled = false;
+    }
+}
diff --git a/xen/arch/arm/tee/ffa_partinfo.c b/xen/arch/arm/tee/ffa_partinfo.c
index dc1059584828..93a03c6bc672 100644
--- a/xen/arch/arm/tee/ffa_partinfo.c
+++ b/xen/arch/arm/tee/ffa_partinfo.c
@@ -306,7 +306,7 @@ static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
     }
 }
 
-bool ffa_partinfo_domain_init(struct domain *d)
+int ffa_partinfo_domain_init(struct domain *d)
 {
     unsigned int count = BITS_TO_LONGS(subscr_vm_destroyed_count);
     struct ffa_ctx *ctx = d->arch.tee;
@@ -315,7 +315,7 @@ bool ffa_partinfo_domain_init(struct domain *d)
 
     ctx->vm_destroy_bitmap = xzalloc_array(unsigned long, count);
     if ( !ctx->vm_destroy_bitmap )
-        return false;
+        return -ENOMEM;
 
     for ( n = 0; n < subscr_vm_created_count; n++ )
     {
@@ -330,7 +330,10 @@ bool ffa_partinfo_domain_init(struct domain *d)
     }
     vm_destroy_bitmap_init(ctx, n);
 
-    return n == subscr_vm_created_count;
+    if ( n != subscr_vm_created_count )
+        return -EIO;
+
+    return 0;
 }
 
 bool ffa_partinfo_domain_destroy(struct domain *d)
diff --git a/xen/arch/arm/tee/ffa_private.h b/xen/arch/arm/tee/ffa_private.h
index 98236cbf14a3..7c6b06f686fc 100644
--- a/xen/arch/arm/tee/ffa_private.h
+++ b/xen/arch/arm/tee/ffa_private.h
@@ -25,6 +25,7 @@
 #define FFA_RET_DENIED                  -6
 #define FFA_RET_RETRY                   -7
 #define FFA_RET_ABORTED                 -8
+#define FFA_RET_NO_DATA                 -9
 
 /* FFA_VERSION helpers */
 #define FFA_VERSION_MAJOR_SHIFT         16U
@@ -175,6 +176,21 @@
  */
 #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U)
 
+/* Flags used in calls to FFA_NOTIFICATION_GET interface  */
+#define FFA_NOTIF_FLAG_BITMAP_SP        BIT(0, U)
+#define FFA_NOTIF_FLAG_BITMAP_VM        BIT(1, U)
+#define FFA_NOTIF_FLAG_BITMAP_SPM       BIT(2, U)
+#define FFA_NOTIF_FLAG_BITMAP_HYP       BIT(3, U)
+
+#define FFA_NOTIF_INFO_GET_MORE_FLAG        BIT(0, U)
+#define FFA_NOTIF_INFO_GET_ID_LIST_SHIFT    12
+#define FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT   7
+#define FFA_NOTIF_INFO_GET_ID_COUNT_MASK    0x1F
+
+/* Feature IDs used with FFA_FEATURES */
+#define FFA_FEATURE_NOTIF_PEND_INTR     0x1U
+#define FFA_FEATURE_SCHEDULE_RECV_INTR  0x2U
+
 /* Function IDs */
 #define FFA_ERROR                       0x84000060U
 #define FFA_SUCCESS_32                  0x84000061U
@@ -213,6 +229,24 @@
 #define FFA_MEM_FRAG_TX                 0x8400007BU
 #define FFA_MSG_SEND                    0x8400006EU
 #define FFA_MSG_POLL                    0x8400006AU
+#define FFA_NOTIFICATION_BITMAP_CREATE  0x8400007DU
+#define FFA_NOTIFICATION_BITMAP_DESTROY 0x8400007EU
+#define FFA_NOTIFICATION_BIND           0x8400007FU
+#define FFA_NOTIFICATION_UNBIND         0x84000080U
+#define FFA_NOTIFICATION_SET            0x84000081U
+#define FFA_NOTIFICATION_GET            0x84000082U
+#define FFA_NOTIFICATION_INFO_GET_32    0x84000083U
+#define FFA_NOTIFICATION_INFO_GET_64    0xC4000083U
+
+struct ffa_ctx_notif {
+    bool enabled;
+
+    /*
+     * True if domain is reported by FFA_NOTIFICATION_INFO_GET to have
+     * pending global notifications.
+     */
+    bool secure_pending;
+};
 
 struct ffa_ctx {
     void *rx;
@@ -228,6 +262,7 @@ struct ffa_ctx {
     struct list_head shm_list;
     /* Number of allocated shared memory object */
     unsigned int shm_count;
+    struct ffa_ctx_notif notif;
     /*
      * tx_lock is used to serialize access to tx
      * rx_lock is used to serialize access to rx
@@ -257,7 +292,7 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs);
 int ffa_handle_mem_reclaim(uint64_t handle, uint32_t flags);
 
 bool ffa_partinfo_init(void);
-bool ffa_partinfo_domain_init(struct domain *d);
+int ffa_partinfo_domain_init(struct domain *d);
 bool ffa_partinfo_domain_destroy(struct domain *d);
 int32_t ffa_handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,
                                       uint32_t w4, uint32_t w5, uint32_t *count,
@@ -271,12 +306,31 @@ uint32_t ffa_handle_rxtx_map(uint32_t fid, register_t tx_addr,
 uint32_t ffa_handle_rxtx_unmap(void);
 int32_t ffa_handle_rx_release(void);
 
+void ffa_notif_init(void);
+void ffa_notif_init_interrupt(void);
+int ffa_notif_domain_init(struct domain *d);
+void ffa_notif_domain_destroy(struct domain *d);
+
+int ffa_handle_notification_bind(struct cpu_user_regs *regs);
+int ffa_handle_notification_unbind(struct cpu_user_regs *regs);
+void ffa_handle_notification_info_get(struct cpu_user_regs *regs);
+void ffa_handle_notification_get(struct cpu_user_regs *regs);
+int ffa_handle_notification_set(struct cpu_user_regs *regs);
+
 static inline uint16_t ffa_get_vm_id(const struct domain *d)
 {
     /* +1 since 0 is reserved for the hypervisor in FF-A */
     return d->domain_id + 1;
 }
 
+static inline struct domain *ffa_rcu_lock_domain_by_vm_id(uint16_t vm_id)
+{
+    ASSERT(vm_id);
+
+    /* -1 to match ffa_get_vm_id() */
+    return rcu_lock_domain_by_id(vm_id - 1);
+}
+
 static inline void ffa_set_regs(struct cpu_user_regs *regs, register_t v0,
                                 register_t v1, register_t v2, register_t v3,
                                 register_t v4, register_t v5, register_t v6,
diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
index cb65f187f51f..1e890ecc2932 100644
--- a/xen/arch/arm/tee/tee.c
+++ b/xen/arch/arm/tee/tee.c
@@ -94,7 +94,7 @@ static int __init tee_init(void)
     return 0;
 }
 
-__initcall(tee_init);
+presmp_initcall(tee_init);
 
 void __init init_tee_interrupt(void)
 {
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 289af81bd69d..e2412a17474e 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -505,6 +505,7 @@ typedef uint64_t xen_callback_t;
 #define GUEST_MAX_VCPUS 128
 
 /* Interrupts */
+
 #define GUEST_TIMER_VIRT_PPI    27
 #define GUEST_TIMER_PHYS_S_PPI  29
 #define GUEST_TIMER_PHYS_NS_PPI 30
@@ -515,6 +516,19 @@ typedef uint64_t xen_callback_t;
 #define GUEST_VIRTIO_MMIO_SPI_FIRST   33
 #define GUEST_VIRTIO_MMIO_SPI_LAST    43
 
+/*
+ * SGI is the preferred delivery mechanism of FF-A pending notifications or
+ * schedule recveive interrupt. SGIs 8-15 are normally not used by a guest
+ * as they in a non-virtualized system typically are assigned to the secure
+ * world. Here we're free to use SGI 8-15 since they are virtual and have
+ * nothing to do with the secure world.
+ *
+ * For partitioning of SGIs see also Arm Base System Architecture v1.0C,
+ * https://developer.arm.com/documentation/den0094/
+ */
+#define GUEST_FFA_NOTIF_PEND_INTR_ID      8
+#define GUEST_FFA_SCHEDULE_RECV_INTR_ID   9
+
 /* PSCI functions */
 #define PSCI_cpu_suspend 0
 #define PSCI_cpu_off     1
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:27:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:27:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731520.1137096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDic-0007kQ-T3; Wed, 29 May 2024 07:27:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731520.1137096; Wed, 29 May 2024 07:27:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDic-0007kJ-QA; Wed, 29 May 2024 07:27:30 +0000
Received: by outflank-mailman (input) for mailman id 731520;
 Wed, 29 May 2024 07:27:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCDic-0007k0-2N
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:27:30 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e5d982c0-1d8c-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:27:26 +0200 (CEST)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-579cd80450fso1018081a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:27:28 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc519a2sm682287666b.126.2024.05.29.00.27.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:27:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5d982c0-1d8c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716967647; x=1717572447; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=GK9NPz5901Lfdu9S6zdeE71DfiXh9d9/sGktk9K7Rc8=;
        b=HqkrqfJ2s46fxDKXzU4vVvlWhdfg6FP/2DZJgQf3+ft6DKLVFZJydnpKJZtrne9dhq
         BzbxBmB2sygwYu1+q4NtLJ/Pl4vAX6UNi4DSYyohe8R8NLOx9LVvxX22wYneR6yFaE3v
         WzEfceteiuKeBqsQ+i2jEuXcmqJHuP5XdjYFwUMEv45Zwvl1dRyUppV0iPBZlTjygVZP
         R3aaOeSXdkDgtyMPjgnTTTpEDJHApnEihJ0tTPZaMENClpheeBn9nVvizowc2zYh2xI7
         ZHTfzrHirPM8bebmw3eg5GcHb2+GL/plp/kASF/jXZaMOYCuZeI4SoTDgcnZAhwli04L
         NETQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967647; x=1717572447;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GK9NPz5901Lfdu9S6zdeE71DfiXh9d9/sGktk9K7Rc8=;
        b=vwrVOzbvm1aqul7DdWFyP03JTz8m4hTdh7W+IcHVJsf19IsIfXjKl1i5zmr4n2gjlR
         i1gDIHDSLHdjscdTIUZ15qPxRRWSgH6HocpBISyBQu8On/cY85f47w0g0SS0bWCvGiK9
         PAK74CWLMInAl55WOvyRb/iPEUmZ3abdXS/6Fn8b7OkFe4J8Qd4faURl0TZ4CvP1fj0g
         UGD9+F2iTr88969yBjmD55CYtrmTD7uNEZCJCqYUH5qGwN1TJFQC4/ZJTozuoMRsjG/j
         xgwrpBzBfVxTFBxO5QPR1hemzQSOjZ1L6OVTfZM1Zv9BAdx7Z7b5fmnPK/eB+RX46bmc
         oXBg==
X-Forwarded-Encrypted: i=1; AJvYcCULPUcoMnInEQbE/zHxujhXANPPAe+U+ErrvHufTEkZnHxZ+4dTEYz3tsBRzhs+LZCngAj74q9OcDu3cNGpYqnI3xB6pu/7ZrjIEOb5D4I=
X-Gm-Message-State: AOJu0Yz2eWXNnSNOwhoD5jAcfU3vWy9EhCgmmPDxIFvNV15Ueu/wMwnW
	9o8MfoiwGLJeBVpLP9WD+RA99snkoMqVzdNAF6f661+7oAqygMui
X-Google-Smtp-Source: AGHT+IHfmSilipFaS9f1+RngjvaFXxbkzlVr9fESZGYP1kenxfqzP2W9WKNQDsOsP9g84iVIswDhAg==
X-Received: by 2002:a17:906:3e4f:b0:a5c:d4b2:6a44 with SMTP id a640c23a62f3a-a642d3846f7mr89494666b.16.1716967646801;
        Wed, 29 May 2024 00:27:26 -0700 (PDT)
Message-ID: <dbc397df840aedacc00613d25a99db7e82dea950.camel@gmail.com>
Subject: Re: [PATCH for-4.19 0/3] xen: Misc MISRA changes
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>,  Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Roberto Bagnara <roberto.bagnara@bugseng.com>, 
 "consulting @ bugseng . com" <consulting@bugseng.com>
Date: Wed, 29 May 2024 09:27:25 +0200
In-Reply-To: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
References: <20240521171546.3650664-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-21 at 18:15 +0100, Andrew Cooper wrote:
> Misc fixes collected during today's call.
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

>=20
> Andrew Cooper (3):
> =C2=A0 xen/lzo: Implement COPY{4,8} using memcpy()
> =C2=A0 xen/x86: Drop useless non-Kconfig CONFIG_* variables
> =C2=A0 xen/x86: Address two misc MISRA 17.7 violations
>=20
> =C2=A0xen/arch/x86/alternative.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 |=C2=A0 4 ++--
> =C2=A0xen/arch/x86/include/asm/config.h |=C2=A0 4 ----
> =C2=A0xen/arch/x86/nmi.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 5 ++---
> =C2=A0xen/common/lzo.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 11 ++---------
> =C2=A0xen/include/xen/acpi.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 9 ---------
> =C2=A0xen/include/xen/watchdog.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 | 13 +------------
> =C2=A06 files changed, 7 insertions(+), 39 deletions(-)
>=20
>=20
> base-commit: 26b122e3bf8f3921d87312fbf5e7e13872ae92b0



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:30:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:30:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731540.1137106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDls-0001Ma-8d; Wed, 29 May 2024 07:30:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731540.1137106; Wed, 29 May 2024 07:30:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDls-0001MT-5n; Wed, 29 May 2024 07:30:52 +0000
Received: by outflank-mailman (input) for mailman id 731540;
 Wed, 29 May 2024 07:30:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCDlr-0001LE-4Z
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:30:51 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5dad0872-1d8d-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:30:47 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a5cec2c2981so80187566b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:30:48 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57862f76dfdsm6624995a12.21.2024.05.29.00.30.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:30:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5dad0872-1d8d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716967848; x=1717572648; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=lMbV4F+yaD2jh4OuPFYX3Dr/e6XNzX4eo758/WuM2j4=;
        b=AQAeRIqjUeFjCCG2xBJ5VkEjYBBIqUfPr2BAq2caL0Ml14ABBKdeVAvXmdMK83rURc
         0vCRNm+JNwag6xsWt4Vlvpllxx2kK6/EYgwwGn1cxLhD3xPP26wTyNCrOTxWg3s6lf9s
         Dah22ZiHSBWtW/t/Ird2TMJEpO2DxD0ZXHnkRthkR9WrF0G9eWfCzQa5wxOWGHmchGWX
         TzdKDlgYF1ynmJ+b8c5GFpC5RclC54zSkY664zM3MFuD8MjOUYLZ/v+ljGrrARlRTYI+
         THQEnJABUqMfNEyxFQpwiLl6rJN6iY0IMBbsmUsz448lLbW8gnTQcYNB7RGExod/TqDl
         8BYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716967848; x=1717572648;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lMbV4F+yaD2jh4OuPFYX3Dr/e6XNzX4eo758/WuM2j4=;
        b=ooLocKVnbF8qXMB2102jamURNJ1wXkw1pUugI2/Yi4g+chWV2tw/I/2gpJy5tVwdqs
         WFEAQvRwyv4oZyUjnUWZSMr/Kuc72b/E2+hfAK6P+QiJRnUwoMrj/v34x0Vb+McmK2IH
         waBQcjVvPP7ALbNjjdKezkkZjiNIRIT7/3h41Tu1hOis2oo6Phl/dQTslAkgl2MuIywr
         FY2GoP2+/0jtJGo2SoL2yOqe9907CvMkPOZVkpCMMxUg117k0OCWYvHbuuWzRgn6czrX
         NQ09kBdAKuZ1qSM44QZnXq1JInAMUCDoeSJ1SW5v5NRnpMFe87/1QPg84i/zh1iprY4w
         ajQg==
X-Forwarded-Encrypted: i=1; AJvYcCVqooQDLPhMvQI5uRLnXZHWKrYOGbhsEmVm1t0UWdWqLrwT3BFKt4tJE4VQ7eV1nAANN5i0psqpjJZEyiwvjpjnL+fFtOQcls8wmCkFNVA=
X-Gm-Message-State: AOJu0YzanNB9V06WPWHxKvPq+NM+CNCfQRzkcJ4VP+icZMaIr5NXMv2f
	YaOTveC1DNN+gCCjqmGiXp5jdleyaOYhngzB0XIZfsTzRafZqeyq
X-Google-Smtp-Source: AGHT+IFHJTNxjC2De6aMZ5GYZZyz8tJ2WSpMlTQWFt9TPX6hbS7WBbMgkt3rHGNaYzAfla/LmQZGiQ==
X-Received: by 2002:a50:85ca:0:b0:574:eb0f:df9b with SMTP id 4fb4d7f45d1cf-5785195096dmr12414152a12.14.1716967847905;
        Wed, 29 May 2024 00:30:47 -0700 (PDT)
Message-ID: <be825b761563b0f427c9c7167e13ce0195e8fb3c.camel@gmail.com>
Subject: Re: [PATCH v2 for-4.19 0.5/13] xen: Introduce CONFIG_SELF_TESTS
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>, 
 Michal Orzel <michal.orzel@amd.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, "consulting @ bugseng . com"
 <consulting@bugseng.com>, Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>, Nicola Vetrini
 <nicola.vetrini@bugseng.com>
Date: Wed, 29 May 2024 09:30:46 +0200
In-Reply-To: <20240528142238.1340228-1-andrew.cooper3@citrix.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
	 <20240528142238.1340228-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-28 at 15:22 +0100, Andrew Cooper wrote:
> ... and move x86's stub_selftest() under this new option.
>=20
> There is value in having these tests included in release builds too.
>=20
> It will shortly be used to gate the bitops unit tests on all
> architectures.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Looks good to me.
We can consider it to be merged to 4.19:
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> CC: Shawn Anastasio <sanastasio@raptorengineering.com>
> CC: consulting@bugseng.com=C2=A0<consulting@bugseng.com>
> CC: Simone Ballarin <simone.ballarin@bugseng.com>
> CC: Federico Serafini <federico.serafini@bugseng.com>
> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
>=20
> v2.5:
> =C2=A0* As suggested in "[PATCH v2 05/13] xen/bitops: Implement
> generic_f?sl() in
> =C2=A0lib/"
>=20
> I've gone with SELF_TESTS rather than BOOT_TESTS, because already in
> bitops
> we've got compile time tests (which aren't strictly boot time), and
> the
> livepatching testing wants to be included here and is definitely not
> boot
> time.
> ---
> =C2=A0xen/Kconfig.debug=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 6 ++++++
> =C2=A0xen/arch/x86/extable.c | 4 ++--
> =C2=A0xen/arch/x86/setup.c=C2=A0=C2=A0 | 2 +-
> =C2=A03 files changed, 9 insertions(+), 3 deletions(-)
>=20
> diff --git a/xen/Kconfig.debug b/xen/Kconfig.debug
> index 61b24ac552cd..07ff7eb7ba83 100644
> --- a/xen/Kconfig.debug
> +++ b/xen/Kconfig.debug
> @@ -29,6 +29,12 @@ config FRAME_POINTER
> =C2=A0	=C2=A0 maybe slower, but it gives very useful debugging
> information
> =C2=A0	=C2=A0 in case of any Xen bugs.
> =C2=A0
> +config SELF_TESTS
> +	bool "Extra self-testing"
> +	default DEBUG
> +	help
> +	=C2=A0 Enable extra unit and functional testing.
> +
> =C2=A0config COVERAGE
> =C2=A0	bool "Code coverage support"
> =C2=A0	depends on !LIVEPATCH
> diff --git a/xen/arch/x86/extable.c b/xen/arch/x86/extable.c
> index 8415cd1fa249..705cf9eb94ca 100644
> --- a/xen/arch/x86/extable.c
> +++ b/xen/arch/x86/extable.c
> @@ -144,7 +144,7 @@ search_exception_table(const struct cpu_user_regs
> *regs, unsigned long *stub_ra)
> =C2=A0=C2=A0=C2=A0=C2=A0 return 0;
> =C2=A0}
> =C2=A0
> -#ifdef CONFIG_DEBUG
> +#ifdef CONFIG_SELF_TESTS
> =C2=A0#include <asm/setup.h>
> =C2=A0#include <asm/traps.h>
> =C2=A0
> @@ -214,7 +214,7 @@ int __init cf_check stub_selftest(void)
> =C2=A0=C2=A0=C2=A0=C2=A0 return 0;
> =C2=A0}
> =C2=A0__initcall(stub_selftest);
> -#endif
> +#endif /* CONFIG_SELF_TESTS */
> =C2=A0
> =C2=A0unsigned long asmlinkage search_pre_exception_table(struct
> cpu_user_regs *regs)
> =C2=A0{
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index b50c9c84af6d..dd51e68dbe5b 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -742,7 +742,7 @@ static void noreturn init_done(void)
> =C2=A0=C2=A0=C2=A0=C2=A0 system_state =3D SYS_STATE_active;
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0 /* Re-run stub recovery self-tests with CET-SS a=
ctive. */
> -=C2=A0=C2=A0=C2=A0 if ( IS_ENABLED(CONFIG_DEBUG) && cpu_has_xen_shstk )
> +=C2=A0=C2=A0=C2=A0 if ( IS_ENABLED(CONFIG_SELF_TESTS) && cpu_has_xen_shs=
tk )
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 stub_selftest();
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0 domain_unpause_by_systemcontroller(dom0);
>=20
> base-commit: 2d93f78bfe25f695d8ffb61d110da9df293ed71b



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:41:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:41:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731560.1137117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDwB-0003O3-73; Wed, 29 May 2024 07:41:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731560.1137117; Wed, 29 May 2024 07:41:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCDwB-0003Nw-4M; Wed, 29 May 2024 07:41:31 +0000
Received: by outflank-mailman (input) for mailman id 731560;
 Wed, 29 May 2024 07:41:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCDw9-0003Nl-Rh
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:41:29 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dadd8a22-1d8e-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:41:26 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-52449b7aa2bso2073515e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:41:28 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52970c33134sm1174378e87.209.2024.05.29.00.41.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:41:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dadd8a22-1d8e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716968487; x=1717573287; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=EmVBLTKuy8j8dtHnds1qPYPeZejP9VO3PNdXIxBa3Wk=;
        b=KJbXrDpvEl8MsJOuodirz0DaePpJf6e6OdKZM9Y7CB7zFcGgxBeQdpOEcebjU3lpkD
         W2S/kmPvAQ855/YStikYY5FPn0j+o18wcvmjcav8y+pO1cI3eXd/cm4/mIFwwxbIAxSH
         drcxNogL9YqFUTt+y62TF1oAUX59XT8mk5OgdD0NvU21sMA9mCoEWDKC9iNsCcSoOte/
         0J7JaDWrC/tr1Jx9ILJH6SUGBvGvuM3hmeSVvvP5kwqL4eXYN5f/U6igdVbZcURLrV5D
         uodxpfqti5iVE+kLfNOE0p/tb7JcLPTFEYGZA9EdtGlSwM1yHxOukXZVzTHBdEP2+oXV
         hXbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716968487; x=1717573287;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EmVBLTKuy8j8dtHnds1qPYPeZejP9VO3PNdXIxBa3Wk=;
        b=G6FT7ePwn//OCQ2hxoGdpjZh6PLCMDcpYDA7ibGqmwDVluB2X6caNMtyeI2/80ygn7
         vge1yc9HRwt5czRn6lIcOT5J3j0nIDKSWZbAn22OPlA26eKuFQRyWUPzukeQzzLOCwYI
         9D6QY26O9wr4oK7GQwi/RISqTQb62Sf4BhVBtqKmGpwR+ewc/WoacET6yJ56k0VZ+i8Q
         kW5vXtx35VHt0RZ/iyNwyERdji1indAqWTWpGWOHsQ4o6l+QPX/v5tDRoOvf3+tGXGaw
         fpgqCLgjh37HdgOrrUD3UuRYSPI9Z2CGp+2ivQLi5w85v7uonzmURg2sRarjuZww3mMM
         nBJg==
X-Forwarded-Encrypted: i=1; AJvYcCVi16dkCO4Gz1v/aCXldzuz6JaMbghoDEOINB0941X3ZV38JWk5pbcfTjLXbMuGYAWbvxTQHnX12aqP0K1J8oBZK2ivkN1K/1hpJeSkHh0=
X-Gm-Message-State: AOJu0YxlzeveX3z7rCwE5zne2BTZE4/M4pt6Ny20vqrsNO01HyCYFMjD
	9zVqoSavs7eDMy2frKFqSOSK4IwmhHHN1V0tjlxZZqQcJAd/aeAS
X-Google-Smtp-Source: AGHT+IFyEe3UWPlnQQCU8RbG8XouxBCGr6j3dVJmpk/qxzpfnxCZHElvGFXzAy4G4sORVHhT33Cf2g==
X-Received: by 2002:a05:6512:3f09:b0:529:ac49:45d1 with SMTP id 2adb3069b0e04-529ac494708mr8471955e87.66.1716968487354;
        Wed, 29 May 2024 00:41:27 -0700 (PDT)
Message-ID: <8e9a563f1e0f45883502bc47c4942081a5d66e1a.camel@gmail.com>
Subject: Re: [PATCH for-4.19 v3 0/3] xen/x86: support foreign mappings for
 HVM/PVH
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Anthony PERARD
 <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Christian Lindig
 <christian.lindig@citrix.com>,  David Scott <dave@recoil.org>, Bertrand
 Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,  "Daniel P. Smith"
 <dpsmith@apertussolutions.com>, Community Manager
 <community.manager@xenproject.org>, Roger Pau Monne <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 09:41:26 +0200
In-Reply-To: <0c4a9a00-b8de-4472-afb4-0a8d70c7aada@suse.com>
References: <20240517133352.94347-1-roger.pau@citrix.com>
	 <0c4a9a00-b8de-4472-afb4-0a8d70c7aada@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 09:24 +0200, Jan Beulich wrote:
> On 17.05.2024 15:33, Roger Pau Monne wrote:
> > Hello,
> >=20
> > The following series attempts to solve a shortcoming of HVM/PVH
> > guests
> > with the lack of support for foreign mappings.=C2=A0 Such lack of
> > support
> > prevents using PVH based guests as stubdomains for example.
> >=20
> > Add support in a way similar to how it's done on Arm, by iterating
> > over
> > the p2m based on the maximum gfn.
> >=20
> > Patch 2 is not strictly needed.=C2=A0 Moving the enablement of altp2m
> > from an
> > HVM param to a create domctl flag avoids any possible race with the
> > HVM
> > param changing after it's been evaluated.=C2=A0 Note the param can only
> > be
> > set by the control domain, and libxl currently sets it at domain
> > create.=C2=A0 Also altp2m enablement is different from activation, as
> > activation does happen during runtime of the domain.
> >=20
> > Thanks, Roger.
> >=20
> > Roger Pau Monne (3):
> > =C2=A0 xen/x86: account number of foreign mappings in the p2m
> > =C2=A0 xen/x86: enable altp2m at create domain domctl
> > =C2=A0 xen/x86: remove foreign mappings from the p2m on teardown
>=20
> Here, too, I'd like to ask whether to keep this as a candidate for
> 4.19, or
> whether to postpone. Afaict what's still missing are Arm and tool
> chain acks
> on patch 2.
We can consider to have this patch series in 4.19:
 Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
>=20
> Jan
>=20
> > =C2=A0CHANGELOG.md=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 |=C2=A0 1 +
> > =C2=A0tools/libs/light/libxl_create.c=C2=A0=C2=A0=C2=A0=C2=A0 | 23 ++++=
+++++-
> > =C2=A0tools/libs/light/libxl_x86.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 | 26 +++++------
> > =C2=A0tools/ocaml/libs/xc/xenctrl_stubs.c |=C2=A0 2 +-
> > =C2=A0xen/arch/arm/domain.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 6 +++
> > =C2=A0xen/arch/x86/domain.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 28 ++++++++++++
> > =C2=A0xen/arch/x86/hvm/hvm.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 23 +++++++++-
> > =C2=A0xen/arch/x86/include/asm/p2m.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 32=
 +++++++++-----
> > =C2=A0xen/arch/x86/mm/p2m-basic.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 | 18 ++++++++
> > =C2=A0xen/arch/x86/mm/p2m.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 68
> > +++++++++++++++++++++++++++--
> > =C2=A0xen/include/public/domctl.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 | 20 ++++++++-
> > =C2=A0xen/include/public/hvm/params.h=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 9=
 +---
> > =C2=A012 files changed, 215 insertions(+), 41 deletions(-)
> >=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed May 29 07:50:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:50:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731566.1137127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCE4l-000646-1K; Wed, 29 May 2024 07:50:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731566.1137127; Wed, 29 May 2024 07:50:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCE4k-00063z-UZ; Wed, 29 May 2024 07:50:22 +0000
Received: by outflank-mailman (input) for mailman id 731566;
 Wed, 29 May 2024 07:50:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCE4i-00063t-Vn
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:50:20 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 18350c66-1d90-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:50:19 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-52ab11ecdbaso603300e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 00:50:19 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-529a50b1d85sm979602e87.113.2024.05.29.00.50.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 00:50:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18350c66-1d90-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716969018; x=1717573818; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=o9wnVc5y1gCAGWJdhNxeWyVR0sVx4jE96Kr/TvXDqTI=;
        b=feWdlt7V8905xYwDwRuxWXWW7vzCV5xG68rLwbHfvAZTsChNFoIL3VgNNh/+Fp1s5s
         2rxORrnBARk3cGGFk2ANb6DasgSQZwcw5aB4fAmLgBshgVEQyiBHTp8dL81Dus/bd6oH
         pD6ojL3o9jRhOxhXOXfChnaKW5s5OLEQn39M2/BA/g/FRfKygTsjq/r0paTLouHR7n6k
         Gokq9/TsAJoTXzoUFkk+6qgwSpakK0pLy6eIDT6LaHhoBb61cUIHzlLXLcBA0Q/CaBDU
         S9q0PprZFZd82vIVmBi17s+3k8tYSDdD0o8D32lERo+fqvQoFF8JN2bpb2xdggojj6yT
         yG3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716969018; x=1717573818;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=o9wnVc5y1gCAGWJdhNxeWyVR0sVx4jE96Kr/TvXDqTI=;
        b=A5eM3V+Qsg+noi81ytLMnwUklZzIPODUBysDUPPCxop8VeQHMC8CisoqdLZyvSMXMQ
         HPSa0yUCygy8uxNgHrQAjtVeEH8q6O7FDTZtPpv61XZVg4+pKBHdeF85ediId+luI/EY
         wh9pg3gwHWi+3kbx4V6gWYcT7sqlBjZ2XGIg6FHMG8bmvQSWBAj0rfDeBpKXaYttZfT/
         qVRIKJZY+Z6m5M7Z33QVWaLDKkTryXdJeQ6GjI25/4wyTywlrXlSc6PHFGBEt8hc/1T9
         22UBAJ8XhsYgfHoduPxLkypwVXreASwZp/FgCdVu5af01rjIGgnIO0Zso4RF/fd/ebt5
         Ec8A==
X-Forwarded-Encrypted: i=1; AJvYcCXmJf1ouu9HpYFx6hyhLHTcGlPDWJRNJEQw+nDrlADYjOIsSSj6LQ8Qvy6AF9wYzL8ittCOw7ZTTE8CoAQGCkV17e2QXnNvqUC2LdefI+w=
X-Gm-Message-State: AOJu0Yw6KKtMFdoFKwr3uEwqmVY5977SmlRL6gesmYjI5MdtSv0AiAXb
	PSJpPS95TJ9YpSrwo7bNb0G/4AJbq0hM9BT8sIz538QXmqwY5AHB
X-Google-Smtp-Source: AGHT+IHOcN9akvGQv2+Y8rbx+k4RAethkWJMRPxEfXmd3H7IhSf1VzlA0o+lbcA+p+VX1mJFpoNDHQ==
X-Received: by 2002:ac2:4db6:0:b0:529:ac38:a160 with SMTP id 2adb3069b0e04-529ac38a434mr5405257e87.34.1716969018501;
        Wed, 29 May 2024 00:50:18 -0700 (PDT)
Message-ID: <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,  Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 09:50:17 +0200
In-Reply-To: <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
	 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
	 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
	 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
	 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Tue, 2024-05-28 at 09:53 +0100, Julien Grall wrote:
> > > > +/**
> > > > + * generic_test_bit - Determine whether a bit is set
> > > > + * @nr: bit number to test
> > > > + * @addr: Address to start counting from
> > > > + *
> > > > + * This operation is non-atomic and can be reordered.
> > > > + * If two examples of this operation race, one can appear to
> > > > succeed
> > > > + * but actually fail.=C2=A0 You must protect multiple accesses wit=
h
> > > > a
> > > > lock.
> > > > + */
> > >=20
> > > You got carried away updating comments - there's no raciness for
> > > simple test_bit(). As is also expressed by its name not having
> > > those
> > > double underscores that the others have.
> > Then it is true for every function in this header. Based on the
> > naming
> > the conclusion can be done if it is atomic/npn-atomic and can/can't
> > be
> > reordered.=20
>=20
> So let me start with that my only request is to keep the existing=20
> comments as you move it. It looks like there were none of test_bit()
> before.
Just to clarify that I understand correctly.

Do we need any comment above functions generic_*()? Based on that they
are implemented in generic way they will be always "non-atomic and can
be reordered.".

Do you find the following comment useful?

" * If two examples of this operation race, one can appear to succeed
 * but actually fail.  You must protect multiple accesses with a lock."

It seems to me that it can dropped as basically "non-atomic and can be
reordered." means that.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 29 07:57:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 07:57:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731572.1137138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEBX-00076y-PY; Wed, 29 May 2024 07:57:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731572.1137138; Wed, 29 May 2024 07:57:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEBX-00076r-L7; Wed, 29 May 2024 07:57:23 +0000
Received: by outflank-mailman (input) for mailman id 731572;
 Wed, 29 May 2024 07:57:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DcHb=NA=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1sCEBX-00076l-4a
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 07:57:23 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060b.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 12a2ec2a-1d91-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 09:57:20 +0200 (CEST)
Received: from PH7PR13CA0004.namprd13.prod.outlook.com (2603:10b6:510:174::25)
 by MN0PR12MB6341.namprd12.prod.outlook.com (2603:10b6:208:3c2::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Wed, 29 May
 2024 07:57:16 +0000
Received: from CY4PEPF0000EE34.namprd05.prod.outlook.com
 (2603:10b6:510:174:cafe::f1) by PH7PR13CA0004.outlook.office365.com
 (2603:10b6:510:174::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.19 via Frontend
 Transport; Wed, 29 May 2024 07:57:16 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EE34.mail.protection.outlook.com (10.167.242.40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7633.15 via Frontend Transport; Wed, 29 May 2024 07:57:15 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 29 May
 2024 02:57:12 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 29 May 2024 02:57:11 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12a2ec2a-1d91-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eAYUNztqgpLnBHm6T2kegJp+p/TRR2xlOHZCacgFj4HiZ3dkZ+tvMa/7maiQewBlLDpyAENwex5sX3Raq8LfVyb8hdUY8WdepMSJYlhDAwF1lg61dsrO9M5eOnLWbtlexsCTM2yMnDitUus8B0TZ31p9bfadVkj0mSoR0WHQ8yOFxnLdkxoRrsNLPTeEY3E6DtfaYMmLg0PGGsf0g4ycuo89z7o8mzL0LoOo3UvIXMgDHevW7CXHSNTGrYHaa1FR48Xpd/gvI/adjEPPCNYo5Yy2NTbUC4fo2vAa/46HK4i3q6PfC24OUAYi9KnDU5X00Mp2A7d0KK0IBWOnQTcgSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BZXHWSmTLEehqwLmMPRdKW925X2CNhmKJtd2q3IMBvs=;
 b=EE4JOaNNFyD3X5WySChIW4hbQ/JgV/jBul2i7xWigsAQZNIT1/QZhM9qnGpDjFaJTZtOz6p3TAta+LsEWVG46uOT+Ny3NMRoMWgmPVwSXeYJwpRrZ4imzfmW+QJau8fGru47fwPFXI8IHY3NKy3+2mfnrD7nJggnqeyYi/lPjRq4eZl6bdWWd8XMTqNXrNuv2OYe+0D5OZAZb0L6SFI8m/AgR7nWOnDgu7eib7jOYe2UOOug5gfmtKN30s85QwUl4S1B5ztKQWzZByf6Tcf4V/aLQvu4pHq0ITB7NNywL0Lh5bFSPYYgFqw9VtGlfDLFpPqfotK7V9QgZR8Ifj+nmA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BZXHWSmTLEehqwLmMPRdKW925X2CNhmKJtd2q3IMBvs=;
 b=gOCUuvkADBBVf3LJWE+jZyiWqOpSwKwpkxtrYHzM+1P4qiQXsSWwHsx/kt2CwpJ5efkjHxoss52hEd8dOVa0NNekhO0c/360UQHa8iLf/NWtMLxvAPaErKLr161oFvlC1pzN0p9hO/N11MQ+TvZdq5/khha5daaCFpSfhKp2nSM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <b30fa6dd-97f4-4249-a1b1-5cbf13e20055@amd.com>
Date: Wed, 29 May 2024 09:57:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 2/4] xen/arm: Alloc XenStore page for Dom0less DomUs
 from hypervisor
To: Stefano Stabellini <stefano.stabellini@amd.com>,
	<xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, "Daniel P . Smith"
	<dpsmith@apertussolutions.com>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
 <20240524225522.2878481-2-stefano.stabellini@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240524225522.2878481-2-stefano.stabellini@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE34:EE_|MN0PR12MB6341:EE_
X-MS-Office365-Filtering-Correlation-Id: 8342e4b0-2bc0-4b0c-521e-08dc7fb4f4b0
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|376005|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?OG9KN2NsRmpOL2pGczhBTDdzMExMbVhYZzFKeDFPZjFreEZzSVNRYTE2ak05?=
 =?utf-8?B?S2M5SGJqRjhpMVVlZEowUnd2NXNGQUJOQmpIaVRibW5pcVlaMkV1dlhVTFRE?=
 =?utf-8?B?L1pYN0w5QkZxRUgwUFFacUZEUDhWcWQ2bUNPVi9SajBUa3pqSUpoeHB6OThN?=
 =?utf-8?B?UzQ1MWM1SWFLT21TZXdFR1FUaUlRQU9mVUVoOWhnT09aaW1WdG5qeEhtRmg3?=
 =?utf-8?B?UmxkV2dVUTdYTkM3aTgzMVpBbHo5dXpPVThnbTBaa0doSnUvdmNKVExWa3c3?=
 =?utf-8?B?eHFoWlVpM3lCdFRPUmIwWXNFa05RY2NsejlURWx2UkFMS1oxYVoxa2FvOTFv?=
 =?utf-8?B?NEJ1cE9qVlkrcENGRmNTVE5wT0RHV09VMHA3cnlyRTBUdlErVkhMNEtpVzJn?=
 =?utf-8?B?SkZMazEvbTNJclVwYy9IUG5DZTlTOHRUR3pEYTBaUjZCdmVSd1FxWWMyUEhI?=
 =?utf-8?B?cEtCK1pUUWxhY3R6UXk3S3RvTlpPdExQWHVIejd1RzU1RGxFQkVwU2NUd200?=
 =?utf-8?B?dmI2clp3T0czUTg2dmc5ejRiSy9wOGJGTGFZZGhXUUIxUTlZZ2xIcVdYQ0NK?=
 =?utf-8?B?N0toUjBmbGFDN2pUaVhJN2lEOERCb2hiZDFBeFRRaFVnYTZZcGFTM0FhZ05o?=
 =?utf-8?B?SjVXR0liaEhDRk9raFBaNEhtNUczY3k2THBwMnArYTZkUHlHOU5iZVpKQ3F5?=
 =?utf-8?B?YkNaK1JNaHdHQTRMMXdOUnhabUVteldtV2xlbW15M2llMW5KTlV1cU5sK0lI?=
 =?utf-8?B?algvc1NPZTgycDZHNjBZYXNMd2FSVUVMTW5BeHVENE1yRWFvMVlRQm9RK1NV?=
 =?utf-8?B?OHJSc1VOWWo5UEhSYkFsdnlqUjJCVHYxR0RjUENIeGVPQytNKzBGR05ucnlN?=
 =?utf-8?B?dWxWLzZXWFVMZS8xUE5VeElzWmlSeTlra0JVWk8xaXd3UU5wRTZvaTRtcjc5?=
 =?utf-8?B?MzloVXNWeklWRTNNaHdEODJFSW5iTlMvVTd6UmUrSFE4Rjl2V2pEYWh6d3kx?=
 =?utf-8?B?R3NmZllvejI2dFBNc256REZOWUF6S0lBbVFwUVd5L3dYUWNWYXpxOGllQ3N2?=
 =?utf-8?B?emVKTUVWZ1BnNFNJNHNxMS9TUTNIWlh6WkRLanplY0c1NE5BTHJFcys2Y21G?=
 =?utf-8?B?UkpybDF4Z3h6TFNsOUcvd0NPRkNZN3BueUxuRzZxRmh3blJKYjZkYm1SbUln?=
 =?utf-8?B?L2YyeW91bS9ZdzdIZEJ6WHB6VnNzaXB0RVRabFJKK3JNVlhuVkQxQzBsTU1N?=
 =?utf-8?B?VHR1TG9JUkp5WDJCeWxzS3c2eVhZUmdCS3BYdXZDVVVxVU5NYkpSZFFoK1pt?=
 =?utf-8?B?RkhRbVVqWlBjZ1E1dllyMm45bmhIWXh0MVBRQ3haK2ozbHJ4eW0weDA1Wk9U?=
 =?utf-8?B?ZzE2UTFRWEFFZHRmcEVxUFZON1o2VTZjbTZuT0lFS0x3eVd0aUlQZHJvbExO?=
 =?utf-8?B?NWxhTVl6QW9uK01BbDZOMHRGYTdOelJrVEtPWmJrMFZPelFFVkMvUlpMYVlw?=
 =?utf-8?B?MjJ6RDZVYko4clpYTHhnNWVaTVlEUERtbHJyaXNJTlBRQkF6SDJWSng5T0I1?=
 =?utf-8?B?MUtqbFNvQkFZQ0VxRzB4c0QzZU55OEZvYTZzRG9KRGxaV2FlR1F5ekczVUE1?=
 =?utf-8?B?WkxFTG1DQjRZRjQvNWF4QjNjM09qR3d2UlRBV3lGb0ZuNG1DRTdLcUlaS0cx?=
 =?utf-8?B?VCt4NnhVbUltM1VpMkhsRFdIY05BbDhqRG1NM2FvQmFzbzFsTHJxdEg1SlVP?=
 =?utf-8?B?QkFKSkVyejhsck11WnlYdm9xSkpaN1JRd3lURTlHU0EwM3o5UjJpVzJCSm53?=
 =?utf-8?B?S1ZFTVdoMFRRSkIxc1FOSlYzL3pTaFd2ODJYMkNOVFJ6UmJSVXZ4akxvcVZP?=
 =?utf-8?Q?Wl+ElD1XowTKu?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 May 2024 07:57:15.4867
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8342e4b0-2bc0-4b0c-521e-08dc7fb4f4b0
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE34.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6341

Hi Stefano,

Wasn't this patch supposed to be folded with patch no.3 to avoid breaking the bisection?

On 25/05/2024 00:55, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> There are use cases (for example using the PV driver) in Dom0less
> setup that require Dom0less DomUs start immediately with Dom0, but
> initialize XenStore later after Dom0's successful boot and call to
> the init-dom0less application.
> 
> An error message can seen from the init-dom0less application on
> 1:1 direct-mapped domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> The "magic page" is a terminology used in the toolstack as reserved
> pages for the VM to have access to virtual platform capabilities.
> Currently the magic pages for Dom0less DomUs are populated by the
> init-dom0less app through populate_physmap(), and populate_physmap()
> automatically assumes gfn == mfn for 1:1 direct mapped domains. This
> cannot be true for the magic pages that are allocated later from the
> init-dom0less application executed in Dom0. For domain using statically
> allocated memory but not 1:1 direct-mapped, similar error "failed to
> retrieve a reserved page" can be seen as the reserved memory list is
> empty at that time.
> 
> Since for init-dom0less, the magic page region is only for XenStore.
> To solve above issue, this commit allocates the XenStore page for
> Dom0less DomUs at the domain construction time. The PFN will be
> noted and communicated to the init-dom0less application executed
> from Dom0. To keep the XenStore late init protocol, set the connection
> status to XENSTORE_RECONNECT.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Suggested-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
>  xen/arch/arm/dom0less-build.c | 55 ++++++++++++++++++++++++++++++++++-
>  1 file changed, 54 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 74f053c242..2963ecc0b3 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -1,5 +1,6 @@
>  /* SPDX-License-Identifier: GPL-2.0-only */
>  #include <xen/device_tree.h>
> +#include <xen/domain_page.h>
>  #include <xen/err.h>
>  #include <xen/event.h>
>  #include <xen/grant_table.h>
> @@ -10,6 +11,8 @@
>  #include <xen/sizes.h>
>  #include <xen/vmap.h>
>  
> +#include <public/io/xs_wire.h>
> +
>  #include <asm/arm64/sve.h>
>  #include <asm/dom0less-build.h>
>  #include <asm/domain_build.h>
> @@ -739,6 +742,53 @@ static int __init alloc_xenstore_evtchn(struct domain *d)
>      return 0;
>  }
>  
> +#define XENSTORE_PFN_OFFSET 1
> +static int __init alloc_xenstore_page(struct domain *d)
> +{
> +    struct page_info *xenstore_pg;
> +    struct xenstore_domain_interface *interface;
> +    mfn_t mfn;
> +    gfn_t gfn;
> +    int rc;
> +
> +    if ( (UINT_MAX - d->max_pages) < 1 )
> +    {
> +        printk(XENLOG_ERR "%pd: Over-allocation for d->max_pages by 1 page.\n",
> +               d);
> +        return -EINVAL;
> +    }
> +    d->max_pages += 1;
NIT: empty line here for readability

> +    xenstore_pg = alloc_domheap_page(d, MEMF_bits(32));
> +    if ( xenstore_pg == NULL && is_64bit_domain(d) )
> +        xenstore_pg = alloc_domheap_page(d, 0);
> +    if ( xenstore_pg == NULL )
> +        return -ENOMEM;
> +
> +    mfn = page_to_mfn(xenstore_pg);
> +    if ( !mfn_x(mfn) )
> +        return -ENOMEM;
I think you should free the allocated page here

Other than that:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal


From xen-devel-bounces@lists.xenproject.org Wed May 29 08:00:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:00:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731585.1137158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEEV-0000oD-JM; Wed, 29 May 2024 08:00:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731585.1137158; Wed, 29 May 2024 08:00:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEEV-0000o6-GU; Wed, 29 May 2024 08:00:27 +0000
Received: by outflank-mailman (input) for mailman id 731585;
 Wed, 29 May 2024 08:00:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DcHb=NA=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1sCEEU-0000ns-LI
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:00:26 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on2060f.outbound.protection.outlook.com
 [2a01:111:f403:2412::60f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8107c3da-1d91-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:00:24 +0200 (CEST)
Received: from CH2PR18CA0002.namprd18.prod.outlook.com (2603:10b6:610:4f::12)
 by IA1PR12MB8190.namprd12.prod.outlook.com (2603:10b6:208:3f2::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Wed, 29 May
 2024 08:00:19 +0000
Received: from CH3PEPF0000000D.namprd04.prod.outlook.com
 (2603:10b6:610:4f:cafe::85) by CH2PR18CA0002.outlook.office365.com
 (2603:10b6:610:4f::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.18 via Frontend
 Transport; Wed, 29 May 2024 08:00:19 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH3PEPF0000000D.mail.protection.outlook.com (10.167.244.43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7633.15 via Frontend Transport; Wed, 29 May 2024 08:00:19 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 29 May
 2024 03:00:18 -0500
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 29 May
 2024 03:00:18 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 29 May 2024 03:00:17 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8107c3da-1d91-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nQ1FOhUSjYYB+jAaEvIMl9aMEkpCJmi42xe3IjFbv8nEzPM4pR/XbNioHZ6SozKJGrSEtYx18A27WgxPyfWolc8rggxS42IcqdAS+/AuCHrNmb4oYqY3D73vg57yYB0xlh/rNLmm17TRiUQxB5R4FDvgXKPIH6F1eVVTp210HZrwGM1Ktbv2S8E4BlofvMbfnzRuPyz/QgR6HyWiBH7tcA/2kM63vyR3P8M0xap0AdHvfwECD01tLCg9zr2XSSn3Wx9T5yVAmwNYMD05o8PkzKp0PeGrTGYupHeSYoa3SKwfwcZWUv71hwOECxui1rJVldB0jFaEtdAx1KyWwGp9mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=k1XdPZSgXGOjtOcpE/ko0av/GDtvu6CmLAGd0/E+Sps=;
 b=MvEh3xRwzvWVW64qkeuM5jVyCfRpP93DrSdId9lmALcghfDt3nRyJJOXPnU1afCCpMu8HQn6jRf5QPdcT+l9oCbpfhUWGYVOv25TkjzxRSbUXP0XNHYEfj2r0bFgQcvP+GRcLjoycdnLM8AMrLt2hsq7ZGA+Yob7+Mry/jm67VdHvB+0x0bYkOq54kgVZF4l3+0qrvR8j8vhBIIWAWktw3xcEX/NfZuhjhf7j8Ha2t06G+W9vuQFuRf4yq0nuuZ9q6dM75T5p+gjj0YmgQJ2dDcH82sHGVAuyKGoT3UJSTIOiZ+Sy/fj/YUiMB3sz3p+pxPTk0Y8WG0KFz7Hw/C1oA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=k1XdPZSgXGOjtOcpE/ko0av/GDtvu6CmLAGd0/E+Sps=;
 b=Dux55XB0BsEZHZQHT1ZgP0dR2Bdxctu6ebcie9+2A9vtjM8sLzQBN6ezSJyvw+kEQvYz7RtblXPD8YNHMG0CDx0f5C7Faeq9/Nh5/GASfGnT0NwReV5ukzwjUfY3toPR2O6SWKvUvccW0P6SGbStNp8BdClbwIspQLR9s1yhD1g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ee3b5c27-c481-4f80-aaf3-9f91ed1c41e6@amd.com>
Date: Wed, 29 May 2024 10:00:16 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 4/4] docs/features/dom0less: Update the late XenStore
 init protocol
To: Stefano Stabellini <stefano.stabellini@amd.com>,
	<xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, <bertrand.marquis@arm.com>,
	<Volodymyr_Babchuk@epam.com>, Henry Wang <xin.wang2@amd.com>
References: <alpine.DEB.2.22.394.2405241552240.2557291@ubuntu-linux-20-04-desktop>
 <20240524225522.2878481-4-stefano.stabellini@amd.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240524225522.2878481-4-stefano.stabellini@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH3PEPF0000000D:EE_|IA1PR12MB8190:EE_
X-MS-Office365-Filtering-Correlation-Id: 584fd66b-28d7-4c83-5daa-08dc7fb56232
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?YjU3Z2wyT29lbGR5YVZQNDl4di9PelNUcVJEOE1MY3VsVWV2U1dVWW15a0Vv?=
 =?utf-8?B?ZHR3YVZvZ1hwc0dLYW1xU0FZa2c1R01ZN3JnR01qbjlPQlQvTjJCRGxrMnVx?=
 =?utf-8?B?bzRoYVI3aVBzMmhzRFhaNzZsdExkU0VWT0J6azlKK3RNQjZsRGNnNEROaW5w?=
 =?utf-8?B?WDVNSWFRSWhBaDZKYUpMU1FOcUFkV1VwY0lBQTV0WGFjZVQzRjdSK0tCOHZ0?=
 =?utf-8?B?Q1A3d1J2NHlvcVBHRi8wWkllbElZS3JOelduZjVqdjUvVi9UcFd2aG0vTmdI?=
 =?utf-8?B?RXRXcVJPUkp6Z2tlUER3RGdHbzI3TU9EZGp0Y0NQVHFPSXNQUi9ock9EUzJr?=
 =?utf-8?B?N0lVbUkyQmpiZXVtOEJ5Z0tsalBPTEc1ZnBXNVp1Y1Z3UXpxK0lQanpxZ2Rq?=
 =?utf-8?B?RCtnWmwvcEdFZk95Qld3TnE1ZXJHTFlUNWtLT01lL2lCNzA0dkFybVFxMzhk?=
 =?utf-8?B?eHMySU5kV0lhWjZWcjFwczJBbHI4dFdYa3hmeEQvZmY0UEpISm5aRjFLM1l5?=
 =?utf-8?B?V1NzSVhLVWlaNnFxdS9XRmpvZlNNeEtjOEhNUTA1T1FBQzB5amx5QXo4RWdU?=
 =?utf-8?B?V1FyVG5NNXVsbFdIUmVUbVhqYXJuNk5lL0dsWnd6M2RVWk9UUkF4MU5NT0hT?=
 =?utf-8?B?NFhVRTFaSEN0U0kyaktPanZZWmQ5ay8vQWRFanI1SEtLZ2IyWU0wbzArQjlO?=
 =?utf-8?B?aTc3c2hQM1czQUZCWkRyKzlZcmNTUXdtR2p0VmkxS3hJOHd3azQ5Vzd2ZHh1?=
 =?utf-8?B?UlJzNE5XWHJVTUgvblgvT0NmYm1jbHNqek1nM0Z2VmJFRnR1VkdYTTUxdk9k?=
 =?utf-8?B?NDNVR3dTbW1Gb0FYR2xPR0tQajFpV0dxWWN0eThTcFlxQzREdzRRMDJnSFRY?=
 =?utf-8?B?UjRYT0NPNHpsUUUzMXNrdWJmOWNRWjgzY21PNzl1dHdJMHZ5K3VhVFl2enVN?=
 =?utf-8?B?MFVqZURZY2lzNDNKellGSHVJZlQwZ2xoTSsvTHErWUErK0c5Q2hpNm5Od2dr?=
 =?utf-8?B?bVpWTGtveTZUc2JvSEZzUlZjSmlnYVJrd0ROR3RLT3BqcGEvTFdTQUpyWXY2?=
 =?utf-8?B?STN0bHkxY0hSS0dxMjZ0SklGV0htQVVnR25FWWlSd0hONXFyRG5aWkJpNXlk?=
 =?utf-8?B?V1FyS3ZuVlBKUHJJZEV4d0FyTjllc3hLS2FHRGFrTWw2NVZvc3ZTUENjZnNs?=
 =?utf-8?B?UkJtZ3dhbVJWbVl2S2o2OWJrVHcxMUN3a1VPTUR3ZW9SMytJK294bXNvSWpC?=
 =?utf-8?B?WjMxTmVHSFc1cVlLWDU0dmdkK2FCTVF4RDROb0tPL1YxbG5YSVBkVjl0UWQw?=
 =?utf-8?B?R29Hb09CUTNjdnpNVlN2dCtlNzczakJ6ZVNuOVpYZVl6ZlpsT0N4OWs0WlR6?=
 =?utf-8?B?NlR4UDBTV25MUXdrcm02amlMakNPZ1E2Smg3MnNQeEZ1YlNaNDZjODdESE90?=
 =?utf-8?B?Ylc1bzhCSkNBY1FFU1RoVEp5UWtFcGxYUHhIMHVLdjIxNFQ5dFpkcThPc1BF?=
 =?utf-8?B?T1JIQXZFaGhLZ21HdHJMekxRc1RUVXNGK2k0bk96TCs0dGd3OEk3ditvdHd6?=
 =?utf-8?B?cncrRUtxamRIMFVqZDJtY1BzS2FVMzlOdHhwdjJwYjYrYzRpY05MQTA1enlm?=
 =?utf-8?B?WUlkRXhrWXhGQ2ZrY1lVZHpKenMvbWNLd01uLzUwbVljS0NKd09aZE5CSE90?=
 =?utf-8?B?L0dXdmpDZlVERml6ZnphZzlSV01XdStRWGNiY1ZSUDJhOUU2NUFBVG9XQzc3?=
 =?utf-8?B?RU5JUXpEMlNmL0FRMVkxakQ1UXdqZm1lK3EzeERSNUhkdGE1dmVYQUZic1JT?=
 =?utf-8?B?K3RCdjQxc1RzS3dreDNqcGxjNlJ4SjFBUUtEUm9kdEUwOFhnckp3UDdZVFNr?=
 =?utf-8?Q?s4YFFl2yHbxW3?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 May 2024 08:00:19.2418
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 584fd66b-28d7-4c83-5daa-08dc7fb56232
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH3PEPF0000000D.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8190


On 25/05/2024 00:55, Stefano Stabellini wrote:
> From: Henry Wang <xin.wang2@amd.com>
> 
> With the new allocation strategy of Dom0less DomUs XenStore page,
> update the doc of the late XenStore init protocol accordingly.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:00:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:00:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731588.1137173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEEn-0001Cx-S4; Wed, 29 May 2024 08:00:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731588.1137173; Wed, 29 May 2024 08:00:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEEn-0001Cq-P6; Wed, 29 May 2024 08:00:45 +0000
Received: by outflank-mailman (input) for mailman id 731588;
 Wed, 29 May 2024 08:00:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEEm-0001Au-JC
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:00:44 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a98e8d9-1d91-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 10:00:43 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 8C2D6CE1676;
 Wed, 29 May 2024 08:00:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D1365C2BD10;
 Wed, 29 May 2024 08:00:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a98e8d9-1d91-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716969637;
	bh=G8ukn4/ipULjaA61nZoKSh0N7cnsV0YBmRdlGxpAHeY=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=SuVzllmYSRfquuJcDcRWnc0GHSPtFNyp26AyBluVbJ0up4tbRtITWjDNuKs1XeKsj
	 9unRV1rYyOv2gxNp3rc+jY9sEkrRoqVZBHFucwYGOw+cZtDcO5G4HtfEKacdVyOA5g
	 6mlURbCvBUcRD04eBHxPfjMS/Igzp8+knm7PD62kn5RYL66jlTq5s1KxPqTq4ABeHL
	 J7gn1nDulWyl0ZYRxhgciUY+N6bzgokG2U4wmuyIXgq6xnWCSKMrB/TqQkHYeFuDg7
	 CGmud4af4+/jP58GUdbsQFpcZukByzTLb1+jCTWsClYve1yVdsTgRE9/fLAmqjluGl
	 t4JqpvDKBOzCw==
Message-ID: <8878dcb7-5f18-4e34-b917-ee5e1ee15cff@kernel.org>
Date: Wed, 29 May 2024 17:00:34 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 01/12] ubd: untagle discard vs write zeroes not support
 handling
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-2-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-2-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Discard and Write Zeroes are different operation and implemented
> by different fallocate opcodes for ubd.  If one fails the other one
> can work and vice versa.
> 
> Split the code to disable the operations in ubd_handler to only
> disable the operation that actually failed.
> 
> Fixes: 50109b5a03b4 ("um: Add support for DISCARD in the UBD Driver")
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  arch/um/drivers/ubd_kern.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
> index ef805eaa9e013d..a79a3b7c33a647 100644
> --- a/arch/um/drivers/ubd_kern.c
> +++ b/arch/um/drivers/ubd_kern.c
> @@ -471,9 +471,14 @@ static void ubd_handler(void)
>  		for (count = 0; count < n/sizeof(struct io_thread_req *); count++) {
>  			struct io_thread_req *io_req = (*irq_req_buffer)[count];
>  
> -			if ((io_req->error == BLK_STS_NOTSUPP) && (req_op(io_req->req) == REQ_OP_DISCARD)) {
> -				blk_queue_max_discard_sectors(io_req->req->q, 0);
> -				blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
> +			if (io_req->error == BLK_STS_NOTSUPP) {
> +				struct request_queue *q = io_req->req->q;
> +
> +				if (req_op(io_req->req) == REQ_OP_DISCARD)
> +					blk_queue_max_discard_sectors(q, 0);
> +				if (req_op(io_req->req) == REQ_OP_WRITE_ZEROES)

Nit: this can be an "else if".

Otherwise, looks OK to me.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

> +					blk_queue_max_write_zeroes_sectors(q,
> +							0);
>  			}
>  			blk_mq_end_request(io_req->req, io_req->error);
>  			kfree(io_req);

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:03:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:03:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731594.1137183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEHj-0001zH-AZ; Wed, 29 May 2024 08:03:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731594.1137183; Wed, 29 May 2024 08:03:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEHj-0001zA-6k; Wed, 29 May 2024 08:03:47 +0000
Received: by outflank-mailman (input) for mailman id 731594;
 Wed, 29 May 2024 08:03:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCEHi-0001z0-CG
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:03:46 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f8295c1b-1d91-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:03:44 +0200 (CEST)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2e716e3030aso15967521fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 01:03:44 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bcd73acsm24683021fa.29.2024.05.29.01.03.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 01:03:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8295c1b-1d91-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716969823; x=1717574623; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Cwlm85y15hvEBwmwyKTMv8ZFXNIJ7y0Stx07S583OOQ=;
        b=f84lZxyc1PlEoCvxvfgh4R7Z2ZLiOmWNaJER9o69AE7vr1OUHQbMxpsGu2ADzOO6vE
         ErHQXttu5YjTgmVpkiHrG8MHph9Ceg7DjwyQSwr1CvkR1dVFkPnpNVny1LOwhBI61puX
         SBx85i1qik3D96mUtrHADBhrYvIX8ppe4OPsyKZOSUUWieTXCTEqYbpLIuCiQPvVWx3s
         pzotZPTRhlwdY6SElVbDS0KFD9onnJAV9j7/KrJS+zmK1PU8T+hOBcp/SZvnX8UPuCJA
         lEGEQ/pIDnfJwyG2/z2XgVvEjkv3zVoMuF8p5u/0pZrnUqrSmp5m7eIB1zvCEmv5Foql
         hrHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716969823; x=1717574623;
        h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from
         :subject:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Cwlm85y15hvEBwmwyKTMv8ZFXNIJ7y0Stx07S583OOQ=;
        b=Pz6gwAFTaAHQGLrAGo1SeYU2GwWYP269yZF8Vx9hDchE8+uKDaoZai5G92hTIxu+ke
         uHkLc2TkymmjNOmis3wqszQy2giCqoxgBjWR/r7hkeLDUCxSAGVA0s/ND8VJisQYDnv0
         NGMVGoYVOSkXvbObUTjOkYs+hej+Zvet6aBFloJKKjCIDMvJG63Cchs4351fhgdOSroj
         /ytxVIfiSEtgfKilX1J633OISGlTy8grcaVT60zV8+pWUFgjl/5/Szp+xM3Rq8IdARTg
         mRwPjdv6JyX823nQUlLKccMp3QJT0TA7i6OhnuhVSHlJRn/Xe5UstN21VLWXiASwEwig
         whNQ==
X-Gm-Message-State: AOJu0YxVearGcy5XfOPBa0g3N+K0A5zgvYuVrTf9M0KK+sLI/LBmW2HD
	1A0hHc0Psk4rcvDWc0gp7A92SjOx/V60emNA6WEkMn90xgrglW8IpjDJkLhC
X-Google-Smtp-Source: AGHT+IEucP+Ht4k1hx3HGvoBTWJctR13u69mYAuMoYihHdUT04IFGF405mIJQJgQVGKeKmWJiQ7UOA==
X-Received: by 2002:a2e:3c18:0:b0:2ea:7db8:7ce6 with SMTP id 38308e7fff4ca-2ea7db87ec8mr3209811fa.38.1716969822089;
        Wed, 29 May 2024 01:03:42 -0700 (PDT)
Message-ID: <e9514fa9fc0a0ea9eae2f0c4accf9b3060e95f9e.camel@gmail.com>
Subject: Updated Xen 4.19 schedule
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Wed, 29 May 2024 10:03:41 +0200
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

Hello everyone,

I would like to announce that I have decided to update the Xen 4.19
schedule due to the extended feature freeze period and the upcoming Xen
Summit next week.

I propose the following updates:
   Code Freeze: from May 31 to June 7
   Hard Code Freeze: from June 21 to June 28
   Final commits: from July 5 to July 12
   Release: July 17
The release date is shifted, but it still remains in July, which seems
acceptable to me.

One more thing:
No release ack is needed before Rc1. Please commit bug fixes at will.

Have a nice day.

Best regards,
Oleksii


From xen-devel-bounces@lists.xenproject.org Wed May 29 08:05:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:05:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731597.1137193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEJE-0002YE-K2; Wed, 29 May 2024 08:05:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731597.1137193; Wed, 29 May 2024 08:05:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEJE-0002Y7-Gc; Wed, 29 May 2024 08:05:20 +0000
Received: by outflank-mailman (input) for mailman id 731597;
 Wed, 29 May 2024 08:05:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEJD-0002Xz-Gw
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:05:19 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2efb1d22-1d92-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:05:17 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 8C59562769;
 Wed, 29 May 2024 08:05:15 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC3FAC2BD10;
 Wed, 29 May 2024 08:05:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2efb1d22-1d92-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716969914;
	bh=blmbzNIoOv7Q3qES0R0/fx18waY7Hk11DTOByTzWHdc=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=SFYbjMJRJ6TIxFXkpgxG4GyzyI/tGBapJQlaGBsNH35uF5q0J/+ufhABOEt5aku5A
	 OGZ/sMeIOSXH403CaXaX972OKhKJYoQKJJDBM5BKOy6U8Nl6vudfsO6HMEQn0f+nwQ
	 EEJNnQcb59MH+bO1MBxpMjGfNAxOzKZEQYEFEGgG3tXhnEMyC0CEKrwsbiSwmPGa0h
	 XlxMBzvwrj6cpKyibCWOGaP3KUxu2jy3ox/5rATAy/QYehwu9ehrekMkeR3fcUmmMZ
	 Z5STO7HCskML/36YIjpBFP0Bm0HiT4KQ6nulMQvLBqHtZ3L7qdm7OgC52ERnL5CdaG
	 wz7IMJehNc1hg==
Message-ID: <f39216c0-f11d-4d4a-ae40-b4cf82e4760e@kernel.org>
Date: Wed, 29 May 2024 17:05:11 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for
 max_sectors
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-3-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-3-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> The soft max_sectors limit is normally capped by the hardware limits and
> an arbitrary upper limit enforced by the kernel, but can be modified by
> the user.  A few drivers want to increase this limit (nbd, rbd) or
> adjust it up or down based on hardware capabilities (sd).
> 
> Change blk_validate_limits to default max_sectors to the optimal I/O
> size, or upgrade it to the preferred minimal I/O size if that is
> larger than the kernel default if no optimal I/O size is provided based
> on the logic in the SD driver.
> 
> This keeps the existing kernel default for drivers that do not provide
> an io_opt or very big io_min value, but picks a much more useful
> default for those who provide these hints, and allows to remove the
> hacks to set the user max_sectors limit in nbd, rbd and sd.
> 
> Note that rd picks a different value for the optimal I/O size vs the
> user max_sectors value, so this is a bit of a behavior change that
> could use careful review from people familiar with rbd.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks OK to me.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:08:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:08:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731609.1137203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCELw-00039Z-Vo; Wed, 29 May 2024 08:08:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731609.1137203; Wed, 29 May 2024 08:08:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCELw-00039S-TI; Wed, 29 May 2024 08:08:08 +0000
Received: by outflank-mailman (input) for mailman id 731609;
 Wed, 29 May 2024 08:08:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCELv-00039M-BM
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:08:07 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9219fb80-1d92-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:08:05 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 34657CE17E7;
 Wed, 29 May 2024 08:07:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 775ACC2BD10;
 Wed, 29 May 2024 08:07:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9219fb80-1d92-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970076;
	bh=03smAZVJ+ZngbkLzugn+0eqk3vVwA0F29enlttGAPng=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=bpShr1IttagkOMnjhsCdo/g2zCZdY/J9hEgnQoOTkwd/TZHIkKASoDSf8+3m+pnEU
	 5dgEnznMnW+pmlqrkv7jTaUFTY8kya1x2kUeM5RGbZFNTb0NPcdTvktgyScpIfQiJz
	 9SYMygauOoY+shIGTFDnwWQBHpng4+D37VR3bFWoOFjpfjsD6ALyH7jGpvSUfQiild
	 6MYzCPhcHg2QsVviU+udRgGt0N3LKskHM5NeZnp90eaxAQpCTzBcjNqXPPgoNsWukV
	 tC4NqjBN6Q6n3ZjWDlVJ+cmaEz8hv6lMBRlGWijOn043mJOWWcO86EQJdQiEIrkwsT
	 sU1UkJ18oQYDg==
Message-ID: <24cad983-a1a0-4983-989d-79bcc9fd8a0a@kernel.org>
Date: Wed, 29 May 2024 17:07:53 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 03/12] sd: simplify the ZBC case in
 provisioning_mode_store
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-4-hch@lst.de>
Content-Language: en-US
From: Damien Le Moal <dlemoal@kernel.org>
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-4-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Don't reset the discard settings to no-op over and over when a user
> writes to the provisioning attribute as that is already the default
> mode for ZBC devices.  In hindsight we should have made writing to
> the attribute fail for ZBC devices, but the code has probably been
> around for far too long to change this now.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  drivers/scsi/sd.c | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 3dff9150ce11e2..15d0035048d902 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -461,14 +461,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
>  	if (!capable(CAP_SYS_ADMIN))
>  		return -EACCES;
>  
> -	if (sd_is_zoned(sdkp)) {
> -		sd_config_discard(sdkp, SD_LBP_DISABLE);
> -		return count;
> -	}
> -
>  	if (sdp->type != TYPE_DISK)
>  		return -EINVAL;
>  
> +	/* ignore the proivisioning mode for ZBB devices */

s/proivisioning/provisioning
s/ZBB/ZBC

With that fixed,

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

> +	if (sd_is_zoned(sdkp))
> +		return count;
> +
>  	mode = sysfs_match_string(lbp_mode, buf);
>  	if (mode < 0)
>  		return -EINVAL;

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:11:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:11:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731614.1137213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEOc-0004lD-CA; Wed, 29 May 2024 08:10:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731614.1137213; Wed, 29 May 2024 08:10:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEOc-0004l6-8w; Wed, 29 May 2024 08:10:54 +0000
Received: by outflank-mailman (input) for mailman id 731614;
 Wed, 29 May 2024 08:10:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEOa-0004jZ-St
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:10:52 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f66e79c7-1d92-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 10:10:51 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 8CFE16280B;
 Wed, 29 May 2024 08:10:50 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 485B6C2BD10;
 Wed, 29 May 2024 08:10:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f66e79c7-1d92-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970250;
	bh=PgFyg9WLnPMWyhLQvcC3XtuqulgVK5WUqskSxNXqJas=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=DZK1VEReps9AHETpZWUhqpsG+MI00Z0ABB6+gsyW4Gi/Z5H2wxRpGys0fUV/CkDBi
	 ekm5UCfe5kEy0dcpfusrq9SB9N1ZwavpRU82ufk8JfWE2lzM+/YXbDY2BXVIIMJY0E
	 T5zQzfJHsOYj5o65HP0rDO9kO+0LU/4axtrqsBUXTVh+zCLdt4IzBnkae9F0/64AST
	 gqMNnuwWzByDW5WKf7i4hE7fr6Q5anX+m2We3adJ81nyxudkw5rtlBVLP6JZkCEX3i
	 Pcy2SlsuFxqd5emx5tb3bXdTCobvVRykH3xWoY1UPtWqs2G9X7g/qXHlmyyZj0p92U
	 NdB9TJbaHuhIA==
Message-ID: <868a031d-a6f5-4f89-b0ee-f74f6ab38dbc@kernel.org>
Date: Wed, 29 May 2024 17:10:47 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 04/12] sd: add a sd_disable_discard helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-5-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-5-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Add helper to disable discard when it is not supported and use it
> instead of sd_config_discard in the I/O completion handler.  This avoids
> touching more fields than required in the I/O completion handler and
> prepares for converting sd to use the atomic queue limits API.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:12:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:12:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731619.1137222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEQ6-0005Z9-OJ; Wed, 29 May 2024 08:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731619.1137222; Wed, 29 May 2024 08:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEQ6-0005Z2-Lo; Wed, 29 May 2024 08:12:26 +0000
Received: by outflank-mailman (input) for mailman id 731619;
 Wed, 29 May 2024 08:12:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEQ4-0005Yu-TO
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:12:24 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ccc38e1-1d93-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:12:23 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id BFBE9623A4;
 Wed, 29 May 2024 08:12:21 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7666DC32786;
 Wed, 29 May 2024 08:12:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ccc38e1-1d93-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970341;
	bh=6hbYxOsQLA742vo6TjIi/DCNFqZKNxvrO5GmFHCpV44=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=R5mS/igRnNYSuaOU/WungJiFXByfPmU52aV3/8lDlnahx3fw8xYGEgm9I0GwjGMLI
	 rxgEHuZC8FN59Nts7D7mlYOVBxUtXbONIbSuFZA/nsChkRReh17QPPy3Nln5IcP01O
	 bsT5Xv1cO9MaQHL1Y0r23CFCdo5I/pmKCJKFQuVPp2FGd5eU3g8CUn1vaagml+SwH/
	 VFaQ6/MQ6XcCtKYl3ZsNtZ6oahfm06jpUk78jThj1RR2tPWitAJRbqdrAgZ8cH96NI
	 HA2Qjk0f15qnfydLy1JSV5a8XNMA+v8I41uKRHJ4B+z8Gf4xJOm2ZPo9UslIKFHhn/
	 xR//bL6vRZCpw==
Message-ID: <dd7a2835-8c38-4b52-aaea-f80064ccd22b@kernel.org>
Date: Wed, 29 May 2024 17:12:18 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 05/12] sd: add a sd_disable_write_same helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-6-hch@lst.de>
Content-Language: en-US
From: Damien Le Moal <dlemoal@kernel.org>
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-6-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Add helper to disable WRITE SAME when it is not supported and use it
> instead of sd_config_write_same in the I/O completion handler.  This
> avoids touching more fields than required in the I/O completion handler
> and  prepares for converting sd to use the atomic queue limits API.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:13:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:13:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731625.1137232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCERO-0006Cb-1W; Wed, 29 May 2024 08:13:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731625.1137232; Wed, 29 May 2024 08:13:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCERN-0006CU-Uo; Wed, 29 May 2024 08:13:45 +0000
Received: by outflank-mailman (input) for mailman id 731625;
 Wed, 29 May 2024 08:13:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCERM-0006Ay-D8
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:13:44 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5c25a63b-1d93-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:13:42 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 3C0E662064;
 Wed, 29 May 2024 08:13:41 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8D31C2BD10;
 Wed, 29 May 2024 08:13:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c25a63b-1d93-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970420;
	bh=QIo4uCV5OL7C/0lr7uj7aSJrY6ALOm2KSU2WDuE+n9Y=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=Vs/JsUarWt+IojFVJmO8nf1xEFuiF80vD56DlthacWxH/uEfpxVkJo1Nrcs6QZxxq
	 Pmn0gMPmiWSbKF+Fz7knbH+rHcLuGSe7ESzyXmY7MGFw0ToZ2p3ayTMoQggCPIYWWr
	 MbIzeTTCJHYhB2LGdxbLZD0TC+R1ByZ434lytDh+Xyz5Ge0KqL5crvujQZtunoVYKi
	 Ams3qK54YyXYWj2unBULezBx87cSmwZtQ1w7qW3/AqQIfpG9VhjpHgpg/JcGjxTlgB
	 qC8UPDqbQ6kQNgqB5asVxdm9rrmo7slKA3lDydzmgHK3fffKyb9ERgIY/7/ky4cbSE
	 bwm1JTmXJj9Eg==
Message-ID: <5248688e-4832-4243-b0d3-04ef6f459aa4@kernel.org>
Date: Wed, 29 May 2024 17:13:37 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 06/12] sd: simplify the disable case in sd_config_discard
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-7-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-7-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Fall through to the main call to blk_queue_max_discard_sectors given that
> max_blocks has been initialized to zero above instead of duplicating the
> call.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:14:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:14:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731629.1137243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCESX-0006tV-Ak; Wed, 29 May 2024 08:14:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731629.1137243; Wed, 29 May 2024 08:14:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCESX-0006tO-7S; Wed, 29 May 2024 08:14:57 +0000
Received: by outflank-mailman (input) for mailman id 731629;
 Wed, 29 May 2024 08:14:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCESW-0006tI-EI
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:14:56 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 871fa197-1d93-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:14:54 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 55AC2626B5;
 Wed, 29 May 2024 08:14:53 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B98BC2BD10;
 Wed, 29 May 2024 08:14:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 871fa197-1d93-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970493;
	bh=807aMJTkC1p+AoxUwaQMdhMlZoRzxAswKTa2dL4juDQ=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=UxzQFdZDojD5zpNCznqhTAYgdt/u91jAQBq4urzSctjoQNdJc95kx61nCycDlwl+Q
	 1JAZAr2BWpd8fVSOv1QtVb8WnA/H3q5yZ/J6rV5BrOS0h/5R7SBN4XhluIsMA1bnkC
	 ZgxW77pqNTlnmVDHqOWJDaloGp+tPXvK5/dfQKe4x5GXGHc718XUnBvEf6dq9tu658
	 4RCg8AKM8Yv0x+3AoBe6WqI19ZjEWYVxMLHJL8shoIDvIkL5J73b1/ED3qKdfnQSgN
	 6cjSWuxuKauqmy2HYUIA1sDs9N0HxdoZvZ2tKGZTyDH71GI3SCMxH44xQHHHqS1bMT
	 OeyWAOUjf5bLQ==
Message-ID: <87cbfdc6-4ba3-4a3c-9d80-24584fa38f27@kernel.org>
Date: Wed, 29 May 2024 17:14:50 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 07/12] sd: factor out a sd_discard_mode helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-8-hch@lst.de>
Content-Language: en-US
From: Damien Le Moal <dlemoal@kernel.org>
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-8-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Split the logic to pick the right discard mode into a little helper
> to prepare for further changes.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:18:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731638.1137253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEW6-0008Ux-P4; Wed, 29 May 2024 08:18:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731638.1137253; Wed, 29 May 2024 08:18:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEW6-0008UC-Li; Wed, 29 May 2024 08:18:38 +0000
Received: by outflank-mailman (input) for mailman id 731638;
 Wed, 29 May 2024 08:18:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEW6-0008Tu-0G
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:18:38 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09e31287-1d94-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:18:35 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id B51DBCE1717;
 Wed, 29 May 2024 08:18:27 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 058B3C32786;
 Wed, 29 May 2024 08:18:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09e31287-1d94-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716970707;
	bh=tdLgkL/G57QWDQYIQEXqBgxXMvBYXtk7Q684oBRA8MI=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=LtR3Oh2agCU9HURoDtBiUh7iTJLGV4B2vVD+qBUmDBqW0QyV3eFxvnbTWLYWNw50G
	 RB5F6E5jmfbwdx4YKjiEpROnk6sbnVvAIqXTJDWEejc7ZNcxGoEWiuHV14ngXeucKK
	 3MdK9HGPHlJYvUXI/8tUaQ0uLgAUaBXFpEhxviGN3usBWl1+GVSFyj7trFfsAdECb8
	 GTGIG7ne+XR/vEI89MNuxclVJdVem1+wTntjOmKx7Ai3Wvm2rt9icq/wfCzCOfhHWj
	 IZ28O7umJrgJMRyP+GPGCXscEvF3FCY+uz4C0cntcL17ACYS8VsyxIYO/y9XSQCsU4
	 hNmwuFFXE3C5w==
Message-ID: <633e78ae-eed4-4437-a6f7-e8c5e22756c8@kernel.org>
Date: Wed, 29 May 2024 17:18:23 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 08/12] sd: cleanup zoned queue limits initialization
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-9-hch@lst.de>
Content-Language: en-US
From: Damien Le Moal <dlemoal@kernel.org>
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-9-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Consolidate setting zone-related queue limits in sd_zbc_read_zones
> instead of splitting them between sd_zbc_revalidate_zones and
> sd_zbc_read_zones, and move the early_zone_information initialization
> in sd_zbc_read_zones above setting up the queue limits.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:19:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:19:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731641.1137263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEWc-0000iv-1l; Wed, 29 May 2024 08:19:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731641.1137263; Wed, 29 May 2024 08:19:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEWb-0000io-Uq; Wed, 29 May 2024 08:19:09 +0000
Received: by outflank-mailman (input) for mailman id 731641;
 Wed, 29 May 2024 08:19:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCEWb-0008Tu-51
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:19:09 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e4f7402-1d94-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:19:07 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-57863a8f4b2so1827896a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 01:19:07 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a629ecfabfbsm545782266b.178.2024.05.29.01.19.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 01:19:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e4f7402-1d94-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716970747; x=1717575547; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=wNjNbYaLsyT5Jwixw5X30ALmJK/jdlxR7zI+m87qCns=;
        b=Q773CwPe8RFSe5fQABnph7NMNIM4pglEoEHrIy+Rlbr5Sk6yAaoWI6RZtiCbKwM7/m
         B40BXMEjwSsXm3Cumf4UKNLk9TSGUa6sHDEPREYUypk334StfAWuh+TLju3sW70XkXZL
         qjNgCZe6HbKak0J1JqMNeqUGohwpmuBUiuD8pdcLMTZ1fIN5ThdeU8/inyCMAkrfZjwy
         BGCuammFhSO57as42+dWvBfynBFu0WSQUb9TdgctivrCIsrsRpg91ZLIU1DY9i+25cTf
         ENpzZ0+3tD1fw3cYeS9IPqYm8kZZwBJDI8sYPrbjkY+zYwB2kQ+tIL6equDmGlEyebkN
         argg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716970747; x=1717575547;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wNjNbYaLsyT5Jwixw5X30ALmJK/jdlxR7zI+m87qCns=;
        b=GshR91820P06pVGhrzLIW1r4DB8VJpbt3/acpYcp5niJ/kMR0YQILY3goRYMoev9+V
         8+yPaHwsRglnpyc9RKuoJHM+3XgKYDgpj6rBogG5apO3Es6zs0QuQY/wM6QBwWR5yVqW
         5SGxYvuyjNieG32rV1gA4/U/R55OS2rwMFH5mRtwxkBsNyCoPBfkv5mUBs1VmXT8C7S9
         JkC8Efha5CtB+V6Zpz0ZQ908MBJ8mhZk26/33WbosAzezGCrJiSjZ/rQ2eqbgeN64tWS
         IzT21B4lTJZtpWzL/z9NJ97YKBqnX5qzYUILKnldbD6WmiUK5Gg78XlzWiWvPbkImP1I
         yX4g==
X-Forwarded-Encrypted: i=1; AJvYcCXAv0IkoKv2cd44+v3lI3iF++XOtdwXNFaWuLpUyxJMVfQdef2typj9x0EL+W6l32H+o6J0sZ5nFzdSNVBvW3G8JpQUTPUOvQQYlTxOFr4=
X-Gm-Message-State: AOJu0Yyc3lpddP0GlVb1UcgkTiwBZ94Rh1OLkX1mVQCsTREhq6xg199I
	sv835ZkkSaaVWWc8Jo9xYZRCLKThqSWnuuFPLo4nn/1t3B03LmIxjWzRfVbM1Q==
X-Google-Smtp-Source: AGHT+IHoSb/Y+bhFzOWre+Eg2rYo0Jgk5qRFk95FWLV4po2FB51JRnNHZcgSVim0z68MvXkyMFLWnQ==
X-Received: by 2002:a17:906:140d:b0:a63:15c1:4334 with SMTP id a640c23a62f3a-a6315c144a4mr450190966b.26.1716970746756;
        Wed, 29 May 2024 01:19:06 -0700 (PDT)
Message-ID: <495de773-bbd6-4a05-b6a4-e00cda301a96@suse.com>
Date: Wed, 29 May 2024 10:19:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Updated Xen 4.19 schedule
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
References: <e9514fa9fc0a0ea9eae2f0c4accf9b3060e95f9e.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e9514fa9fc0a0ea9eae2f0c4accf9b3060e95f9e.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 10:03, Oleksii K. wrote:
> Hello everyone,
> 
> I would like to announce that I have decided to update the Xen 4.19
> schedule due to the extended feature freeze period and the upcoming Xen
> Summit next week.
> 
> I propose the following updates:
>    Code Freeze: from May 31 to June 7

This is almost fully the week of the summit. With ...

>    Hard Code Freeze: from June 21 to June 28

... this, did you perhaps mean May 31 to June 20 there?

>    Final commits: from July 5 to July 12

Somewhat similarly, what's between June 28 and July 5?

Jan

>    Release: July 17
> The release date is shifted, but it still remains in July, which seems
> acceptable to me.
> 
> One more thing:
> No release ack is needed before Rc1. Please commit bug fixes at will.
> 
> Have a nice day.
> 
> Best regards,
> Oleksii



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:23:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:23:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731649.1137273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEaq-0003VY-Ga; Wed, 29 May 2024 08:23:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731649.1137273; Wed, 29 May 2024 08:23:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEaq-0003VR-DJ; Wed, 29 May 2024 08:23:32 +0000
Received: by outflank-mailman (input) for mailman id 731649;
 Wed, 29 May 2024 08:23:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEao-0003VL-Sa
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:23:30 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ba48e395-1d94-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 10:23:29 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 6A932626B5;
 Wed, 29 May 2024 08:23:28 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C398C2BD10;
 Wed, 29 May 2024 08:23:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba48e395-1d94-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716971008;
	bh=UnLzeIpjO2H8DXobwx0iVT/QPf6bQElmv5UNVtEUT0A=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=O6oJdroelTELWIStQcjdZBa8C9t0BA8C42VKn5vNrwvO9ZdR3rUrwH3ZLwjFFK0Oe
	 HhiPBOH2XVUERxnnGQ6s891sk9q1DG/PnFL0LkBDnOMmP+BS9Xidu7ZeyiJobkI7Mc
	 aurytkNHgMhSMIO6KEtPbr+B/Lm3Ypkam7hGJBLyD18GZpSe9Cj5EF1Kb2puX7kRs5
	 KKnc0fqXHQGePuNkSVkdaId1TdYlGbhhiavhXQgTW+r4g12/5y6OeB3rzT9MZESuNJ
	 vPo1fpxXY6E38PJ/CvZgo2QxoMoClT9dhzC5WZl2dJPLzL0U0DaPQ4DmDMx98s5JET
	 LA3uCxbs7kWag==
Message-ID: <5e23755e-2473-46e3-98c4-1a1e3fb8bfdb@kernel.org>
Date: Wed, 29 May 2024 17:23:25 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 09/12] sd: convert to the atomic queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-10-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-10-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Assign all queue limits through a local queue_limits variable and
> queue_limits_commit_update so that we can't race updating them from
> multiple places, and free the queue when updating them so that

s/free/freeze

> in-progress I/O submissions don't see half-updated limits.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

With the above fixed, looks good to me.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:25:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:25:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731652.1137283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEcc-0004Pp-Sg; Wed, 29 May 2024 08:25:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731652.1137283; Wed, 29 May 2024 08:25:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEcc-0004Pi-Od; Wed, 29 May 2024 08:25:22 +0000
Received: by outflank-mailman (input) for mailman id 731652;
 Wed, 29 May 2024 08:25:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEcb-0004PY-At
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:25:21 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa224c6c-1d94-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:25:18 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id ED81CCE1185;
 Wed, 29 May 2024 08:25:12 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34391C2BD10;
 Wed, 29 May 2024 08:25:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa224c6c-1d94-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716971112;
	bh=68h8q5F+j4pjKEcxNZSHuDAUqZpgnW57uKiQqJYjvXk=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=cqyEaNLJFYd4VgaxK4xOgO1fLB7/V2NkEYcZNMj3ccG2Mzvpp61LRxO5H4wNFOXas
	 9i9P5vApYqzNiVftsPqH6+3ivxqZDbQQpXB9SIohRK0TdnoPp833Z44jgkjFhNZuhY
	 BGPG5IuhLO8Vo39srFS1asTLrRf87QTWOJrOiQEM6H9VY7vVFKAL0M8AaZGrySb+3e
	 wtua7SU6aKnlakSQaBr/SFZqCqkbfJyWm4sHhHK/vx2ZqIcj/ebCP9P4hluFoJ78kb
	 xa2RVmjSw0F9dLWXjsdQcxM1ZvFiEoX8BWzuuviifbIpZoK74eW2DiuDsdsaVYZzqk
	 NPfPYWCoLxCZw==
Message-ID: <1048ffa4-36bb-4d72-9abf-a8e2dfc874c2@kernel.org>
Date: Wed, 29 May 2024 17:25:09 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 10/12] sr: convert to the atomic queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-11-hch@lst.de>
Content-Language: en-US
From: Damien Le Moal <dlemoal@kernel.org>
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-11-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Assign all queue limits through a local queue_limits variable and
> queue_limits_commit_update so that we can't race updating them from
> multiple places, and free the queue when updating them so that
> in-progress I/O submissions don't see half-updated limits.
> 
> Also use the chance to clean up variable names to standard ones.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks good to me.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:28:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:28:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731660.1137292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEfU-0004zW-7o; Wed, 29 May 2024 08:28:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731660.1137292; Wed, 29 May 2024 08:28:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEfU-0004zP-5I; Wed, 29 May 2024 08:28:20 +0000
Received: by outflank-mailman (input) for mailman id 731660;
 Wed, 29 May 2024 08:28:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEfS-0004zF-6t
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:28:18 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64fbfd93-1d95-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:28:16 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 0AABA62064;
 Wed, 29 May 2024 08:28:15 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id BF68BC2BD10;
 Wed, 29 May 2024 08:28:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64fbfd93-1d95-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716971294;
	bh=4xLTsI+g+6PM4GZdcp6tkOURF5lktb+4fm2gix3vClQ=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=creCoCfkg1r9P0k8mhbQwfgzqAswZmiHcWn5Bd1MuoZI86SxYC6hCoH0KS7axJpRP
	 T5h8NZUH+144AFLShb1Ye209bRfiGKQfICluAxPCjIl31GxjbCY/ZZO9BCNN2JyPxs
	 KI2ckBHEkR0/WvEE3z6RgUJOVV2nWrAL1xRk0dgjzNMb2/4WydlFDV4Vv9Q9Exms4t
	 zw1nCtAcJ7pSBxyqVgpFzXBd4oJqfsn4Md73ZGniohT92LzP2SC+w4HfaZP9jPfdIk
	 B1KifcBYlefe27lPs+SGjeRu5o3YScEpjkE/6WSWgxZ1cgJ6Uev8aMv3ZeVlm9xQqD
	 86KgS37/4vtFA==
Message-ID: <89f98897-bfb4-48d1-9b60-d793fde9fade@kernel.org>
Date: Wed, 29 May 2024 17:28:11 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 11/12] block: remove unused queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-12-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-12-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> Remove all APIs that are unused now that sd and sr have been converted
> to the atomic queue limits API.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

I think that disk_set_max_open_zones() and disk_set_max_active_zones() can also
go away.

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:31:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:31:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731667.1137302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEi3-0006yL-N9; Wed, 29 May 2024 08:30:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731667.1137302; Wed, 29 May 2024 08:30:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEi3-0006yE-KZ; Wed, 29 May 2024 08:30:59 +0000
Received: by outflank-mailman (input) for mailman id 731667;
 Wed, 29 May 2024 08:30:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZZuD=NA=kernel.org=dlemoal@srs-se1.protection.inumbo.net>)
 id 1sCEi3-0006y8-3S
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:30:59 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c3ba5716-1d95-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:30:56 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 47202CE1665;
 Wed, 29 May 2024 08:30:49 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8AC01C2BD10;
 Wed, 29 May 2024 08:30:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3ba5716-1d95-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1716971448;
	bh=okobbeHOzXq6ivMGxWAQOlHqPIeKQZDwNC64vcjjQ6o=;
	h=Date:Subject:To:Cc:References:From:In-Reply-To:From;
	b=oJHg4EaC6VLoBZLA2tkfhAXWRViSHX90IfPOto1y2u0MBzS2lz8peI4cgrhwJmiR6
	 j9IGX6ydtNHZbJidgJPmMLDbdMXz0lQ2SjJFcgOj/ZzOZSBuyzHMbC9X6txlhFCKjq
	 zicWj7/lpaI7DNz0gaj1s72zKODzpr9MW0uRepDVL9LHZlHfiRvRJLXPYCR5+1kuWl
	 1WzRU4j7uDZm8UVS5ixrUgADkNSKzvq4M0wIXGo6299fOfN++X+wdoS1ukfz0V0jSB
	 OymHRmFQtxJAOdroVyR9aNThRLSfqkLHyDmmUaVQP0WnD4PbxrI2HzlwMAzSkKWaIa
	 Mi/WLM+Dn9sPQ==
Message-ID: <cf5628a4-50e5-4397-9633-c00a3df2a2df@kernel.org>
Date: Wed, 29 May 2024 17:30:45 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 12/12] block: add special APIs for run-time disabling of
 discard and friends
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-13-hch@lst.de>
From: Damien Le Moal <dlemoal@kernel.org>
Content-Language: en-US
Organization: Western Digital Research
In-Reply-To: <20240529050507.1392041-13-hch@lst.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 5/29/24 14:04, Christoph Hellwig wrote:
> A few drivers optimistically try to support discard, write zeroes and
> secure erase and disable the features from the I/O completion handler
> if the hardware can't support them.  This disable can't be done using
> the atomic queue limits API because the I/O completion handlers can't
> take sleeping locks or freezer the queue.  Keep the existing clearing

s/freezer/freeze

> of the relevant field to zero, but replace the old blk_queue_max_*
> APIs with new disable APIs that force the value to 0.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

With the typo fixed, looks OK to me.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research



From xen-devel-bounces@lists.xenproject.org Wed May 29 08:36:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 08:36:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731673.1137312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEnV-0007mR-AO; Wed, 29 May 2024 08:36:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731673.1137312; Wed, 29 May 2024 08:36:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCEnV-0007m4-7M; Wed, 29 May 2024 08:36:37 +0000
Received: by outflank-mailman (input) for mailman id 731673;
 Wed, 29 May 2024 08:36:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCEnU-0007ln-98
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 08:36:36 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8e75296b-1d96-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 10:36:34 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-52b1ba74d39so320012e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 01:36:34 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8e534sm702084066b.170.2024.05.29.01.36.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 01:36:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e75296b-1d96-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716971794; x=1717576594; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=aue1JfAUF2Db+m9WM9eLeyB/E18PJUjmuGIUgDBYp40=;
        b=ae+V03gIOqnkBKKkMUkgHXtMDnlEnvAUvGjogl42ZEhnkkZ6HRZ+o3g62Y8W2wRdkd
         trsJ+hxPzeZEoICYgg+bRERYk05ft7mX0Wo+KVsF7HJ4ZkhXeUlRpUHIU4WN2xZ2PXfr
         id8T9XwnAgfuJNMCnzEYELtD6OfV3aPPJ59cVWjQNudZGa+IlOsd8Mba0+B/2aE/UXvj
         dnrUmeerAc7Z6ZC5PLyz2DIEePOKg7zmTlH4sbBCNnTVVImTsROjz4/tHBQ4wMSPCt14
         mYRQALJ9GCaVCBBO34XqclxZ4RK20JFydc+9yAIcWQWi2LF2tsENzJuzcF6DnN6dVFC8
         nb+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716971794; x=1717576594;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aue1JfAUF2Db+m9WM9eLeyB/E18PJUjmuGIUgDBYp40=;
        b=GtZG1MSRKqf47WlGOWSSQLMA139VrlyL/Vtd+wxrhGWHcEnDXmlGcpZAtYsRRsDlcL
         Bm0ljsMVMhq07yQdBFLHyVwxfIj6JJfh0EVlQCYy3b3RVU12yT8zQDkKAdFGXBLfGQkD
         KNMTKcFMyJtjU1siCSF+JpwBNrzOOu0Z/F36QBZkrWk3dZB8+HyG5GFqK0ZLjYdqosV1
         Pg2ZfWI5ea7y8QCJIwm/HcTcvmPkv1Uqb0LzoMdu5ItCDC4WjE98w27/dDoOkAh4Gxl5
         4rPt9nL7lBOqCovJV5EaMNpId0g8r3mhG8nhVqEOyPuYDgZkkWcUGMffOFGE8keNjpnu
         9U9A==
X-Forwarded-Encrypted: i=1; AJvYcCVo/+fExo6Z9WbT2kZDSRiQupq7uojgamCMZAJ/5RyREL3SXQhlQzptH0a16qtpb73x4Q60OcilJStAAvZzS/OF1Fwht6akoYBHUE86WqE=
X-Gm-Message-State: AOJu0YzhWzn893wHAzDeEEIK4KvYuaRitAReSH4uUNXNRZZL27cGcgHd
	JMnRHa8uOJFvuZxvY/mINQLfNMQTAFtBFbX2p9neNIr34owEb9LcAWa/5sO3fg==
X-Google-Smtp-Source: AGHT+IHHSehhRNBBC08d5X+GZe7hByLCqX7iZuq7R5DVW1LgYx34rYmht5n+89yYXtjsEWpcBSzWRQ==
X-Received: by 2002:a05:6512:902:b0:51d:4af6:bb86 with SMTP id 2adb3069b0e04-529612c529amr9250737e87.0.1716971793843;
        Wed, 29 May 2024 01:36:33 -0700 (PDT)
Message-ID: <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
Date: Wed, 29 May 2024 10:36:32 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Julien Grall <julien@xen.org>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 09:50, Oleksii K. wrote:
> On Tue, 2024-05-28 at 09:53 +0100, Julien Grall wrote:
>>>>> +/**
>>>>> + * generic_test_bit - Determine whether a bit is set
>>>>> + * @nr: bit number to test
>>>>> + * @addr: Address to start counting from
>>>>> + *
>>>>> + * This operation is non-atomic and can be reordered.
>>>>> + * If two examples of this operation race, one can appear to
>>>>> succeed
>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>> a
>>>>> lock.
>>>>> + */
>>>>
>>>> You got carried away updating comments - there's no raciness for
>>>> simple test_bit(). As is also expressed by its name not having
>>>> those
>>>> double underscores that the others have.
>>> Then it is true for every function in this header. Based on the
>>> naming
>>> the conclusion can be done if it is atomic/npn-atomic and can/can't
>>> be
>>> reordered. 
>>
>> So let me start with that my only request is to keep the existing 
>> comments as you move it. It looks like there were none of test_bit()
>> before.
> Just to clarify that I understand correctly.
> 
> Do we need any comment above functions generic_*()? Based on that they
> are implemented in generic way they will be always "non-atomic and can
> be reordered.".

I indicated before that I think reproducing the same comments __test_and_*
already have also for generic_* isn't overly useful. If someone insisted
on them being there as well, I could live with that, though.

> Do you find the following comment useful?
> 
> " * If two examples of this operation race, one can appear to succeed
>  * but actually fail.  You must protect multiple accesses with a lock."
> 
> It seems to me that it can dropped as basically "non-atomic and can be
> reordered." means that.

I agree, or else - as indicated before - the wording would need to further
change. Yet iirc you've added that in response to a comment from Julien,
so you'll primarily want his input as to the presence of something along
these lines.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731690.1137353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCW-0007GR-5b; Wed, 29 May 2024 09:02:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731690.1137353; Wed, 29 May 2024 09:02:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCW-0007GK-1p; Wed, 29 May 2024 09:02:28 +0000
Received: by outflank-mailman (input) for mailman id 731690;
 Wed, 29 May 2024 09:02:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCV-0006nV-3E
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:27 +0000
Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com
 [2607:f8b0:4864:20::82c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b3c2558-1d9a-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:02:26 +0200 (CEST)
Received: by mail-qt1-x82c.google.com with SMTP id
 d75a77b69052e-43f87ba7d3eso8992531cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:26 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb16b9770sm52129441cf.6.2024.05.29.02.02.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b3c2558-1d9a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973345; x=1717578145; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nBJNMx2S48pD7GARHNMhhe8qb1w3vAy8ioD3DvtxaK0=;
        b=h9HQD/CtQY7JTRR314/fx5bxpAqhS/rmFY2phrwXtUnOMsOcrbt+FI+IeRb29Zd+nd
         P6LN6vcwDq5/L9m+8t508co5xltNbfdxtPNykzdw71RBxWCh8dgSKqQE9k9t1ugjGlCn
         Z7VFnCxAYQhA24rpL0USKvu7EBJGXAu2TVVQs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973345; x=1717578145;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nBJNMx2S48pD7GARHNMhhe8qb1w3vAy8ioD3DvtxaK0=;
        b=dsfrday04ntS/rA0twvR5OYLlEGyC4JZKqS+hrnom1JQ5vPCPrhw+cgTSEynRypMtq
         JUXUHMnYh+jb1e5+kkrpw3h4r7p4YNRyjyYHfWvPqepvucM/zZWSV9R4hGK4W46VyBlF
         CzOeTMgJVtaPXltXlOcALuBX30zjVOCMKk+jpkGyR3YRzWsjavLH0vQzJT2ARq4TSMbu
         jEKMGSjSRI0F3VPf3IP6R4ntK/MVoESrM2qio6uSMhvXuQcWz+53roXB3Ti9pq+gG8yQ
         Ed+WDjdaD6v4q6rI7TNS6vZqDqLerTlge+4O2Hf3U3kHuhFllzylmD1hlcwKEyZxvPyk
         9eVw==
X-Gm-Message-State: AOJu0YzltI1JhO+aSzNLo56x7goXzBDzmPNqUJjW52t0hD8YWVAiuqAC
	fXycsdrS5g5W859QY9uCUDQgNqgqtHrV6bNMX/0hYACgjhiNkLxaHi2PK2rdSDLhLkS9HZrZwUw
	7
X-Google-Smtp-Source: AGHT+IFHrl3J3xvu8cJrUJptqFsfoUxoTDgqwa6ABiAHIEPBty/+m2iD8CgPpr1TfF9sKgBxFQzAoQ==
X-Received: by 2002:a05:622a:1811:b0:43e:e6e:21df with SMTP id d75a77b69052e-43fb0e8584emr144478991cf.15.1716973345017;
        Wed, 29 May 2024 02:02:25 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns false if CPU operations are underway
Date: Wed, 29 May 2024 11:01:25 +0200
Message-ID: <20240529090132.59434-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
a cpu_hotplug_{begin,done}() region the function will still return success,
because a CPU taking the rwlock in read mode after having taken it in write
mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
as it should always return false when called with a CPU hot{,un}plug operation
is in progress.  Otherwise the logic in send_IPI_mask() for example is wrong,
as it could decide to use the shorthand even when a CPU operation is in
progress.

Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
already hold in write mode by the current CPU, as read_trylock() would
otherwise return true.

Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/common/cpu.c         | 3 ++-
 xen/include/xen/rwlock.h | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/xen/common/cpu.c b/xen/common/cpu.c
index 6173220e771b..d76f80fe2e99 100644
--- a/xen/common/cpu.c
+++ b/xen/common/cpu.c
@@ -49,7 +49,8 @@ static DEFINE_RWLOCK(cpu_add_remove_lock);
 
 bool get_cpu_maps(void)
 {
-    return read_trylock(&cpu_add_remove_lock);
+    return !rw_is_write_locked_by_me(&cpu_add_remove_lock) &&
+           read_trylock(&cpu_add_remove_lock);
 }
 
 void put_cpu_maps(void)
diff --git a/xen/include/xen/rwlock.h b/xen/include/xen/rwlock.h
index a2e98cad343e..4e7802821859 100644
--- a/xen/include/xen/rwlock.h
+++ b/xen/include/xen/rwlock.h
@@ -316,6 +316,8 @@ static always_inline void write_lock_irq(rwlock_t *l)
 
 #define rw_is_locked(l)               _rw_is_locked(l)
 #define rw_is_write_locked(l)         _rw_is_write_locked(l)
+#define rw_is_write_locked_by_me(l) \
+    lock_evaluate_nospec(_is_write_locked_by_me(atomic_read(&(l)->cnts)))
 
 
 typedef struct percpu_rwlock percpu_rwlock_t;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731687.1137322 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCR-0006YX-BE; Wed, 29 May 2024 09:02:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731687.1137322; Wed, 29 May 2024 09:02:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCR-0006YQ-8b; Wed, 29 May 2024 09:02:23 +0000
Received: by outflank-mailman (input) for mailman id 731687;
 Wed, 29 May 2024 09:02:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCP-0006YK-SU
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:21 +0000
Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com
 [2607:f8b0:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 27297f9d-1d9a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:02:19 +0200 (CEST)
Received: by mail-oi1-x22d.google.com with SMTP id
 5614622812f47-3cabac56b38so898712b6e.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:19 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac070e1460sm52339466d6.40.2024.05.29.02.02.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27297f9d-1d9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973337; x=1717578137; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=yZF32pZ4CySVT63PvMfVFEEjipVnmCoEx5V33tI/850=;
        b=jU8oImCdkqRMkKHakjJAqQSYHbI9BaUKdFyWkNbGqgUxmEli5Q7d2M6FjdkyBRHXWZ
         vXlPK5RVwTnDsmI/zqcobgmN7ByNgQroMWH5izF3w470238gfg/reHsRMsPlhlJb1ruY
         GSiVFcoqEZ37ZHj4LRd9/FNrkR1gfMEit2UGk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973337; x=1717578137;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=yZF32pZ4CySVT63PvMfVFEEjipVnmCoEx5V33tI/850=;
        b=er63QhVQT952kvL22hazu984LfGsR876h9br9X/LWZ9sac28pLlVZEwRsCyaSZgqUN
         wb+rqCPb2/7UwjvaREevX1JGDCT7iIqwxuTiqQhWQ3gJnY3pfGruxJHgghW+EpR/zbwy
         bRknG77DyLghfITEYKBJUUei3L04j71Kk5/jycxa54SmMdWI88sw3cj4qmMjG6QCc69X
         rv45mt91wZC/uQKnJLEpzoXFVQU90Ihkys6GEhn8CvXNrq2KDFzC8OL4SQyP20p8ElzQ
         sh1szKvscnVkbL1qR4Lkew/nh2hd/X40gfX/fjlor/oPHYS9CQcHlxGj2kZwT2hggJpP
         dJyg==
X-Gm-Message-State: AOJu0Yyh4d9o8KsxDCtKgtPyg3yn2X0VoILzE2HwyY7eQaBbJvZgHKIo
	Gt2EkdejnnBDMCtDQyRhr828QE2LyvFQ7uiNaiiKSYrvpTWFBU2tMODB7xq9oWXCVyCPgPejGv6
	8
X-Google-Smtp-Source: AGHT+IH0ZJzmvauilykbAJmHui3TPflhX0w+i/ry24eVRpmVsIVFIJO+Rs/P50zziFBhLR2bEDa1kw==
X-Received: by 2002:a05:6808:3026:b0:3c9:c3ca:e2cb with SMTP id 5614622812f47-3d1a5483e47mr16811764b6e.11.1716973337560;
        Wed, 29 May 2024 02:02:17 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 0/9] x86/irq: fixes for CPU hot{,un}plug
Date: Wed, 29 May 2024 11:01:22 +0200
Message-ID: <20240529090132.59434-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following series aim to fix interrupt handling when doing CPU
plug/unplug operations.  Without this series running:

cpus=`xl info max_cpu_id`
while [ 1 ]; do
    for i in `seq 1 $cpus`; do
        xen-hptool cpu-offline $i;
        xen-hptool cpu-online $i;
    done
done

Quite quickly results in interrupts getting lost and "No irq handler for
vector" messages on the Xen console.  Drivers in dom0 also start getting
interrupt timeouts and the system becomes unusable.

After applying the series running the loop over night still result in a
fully usable system, no  "No irq handler for vector" messages at all, no
interrupt loses reported by dom0.  Test with
x2apic-mode={mixed,cluster}.

I'm tagging this for 4.19 as it's IMO bugfixes, but the series has grown
quite bigger than expected, and hence we need to be careful to not
introduce breakages late in the release cycle.  I've attempted to
document all code as good as I could, interrupt handling has some
unexpected corner cases that are hard to diagnose and reason about.

I'm currently also doing some extra testing with XenRT in case I've
missed something.

Thanks, Roger.

Roger Pau Monne (9):
  x86/irq: remove offline CPUs from old CPU mask when adjusting
    move_cleanup_count
  xen/cpu: do not get the CPU map in stop_machine_run()
  xen/cpu: ensure get_cpu_maps() returns false if CPU operations are
    underway
  x86/irq: describe how the interrupt CPU movement works
  x86/irq: limit interrupt movement done by fixup_irqs()
  x86/irq: restrict CPU movement in set_desc_affinity()
  x86/irq: deal with old_cpu_mask for interrupts in movement in
    fixup_irqs()
  x86/irq: handle moving interrupts in _assign_irq_vector()
  x86/irq: forward pending interrupts to new destination in fixup_irqs()

 xen/arch/x86/apic.c             |   5 +
 xen/arch/x86/include/asm/apic.h |   3 +
 xen/arch/x86/include/asm/irq.h  |  28 +++++-
 xen/arch/x86/irq.c              | 172 +++++++++++++++++++++++++-------
 xen/common/cpu.c                |   8 +-
 xen/common/stop_machine.c       |  15 +--
 xen/include/xen/cpu.h           |   2 +
 xen/include/xen/rwlock.h        |   2 +
 8 files changed, 190 insertions(+), 45 deletions(-)

-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731691.1137363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCa-0007aS-DC; Wed, 29 May 2024 09:02:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731691.1137363; Wed, 29 May 2024 09:02:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCa-0007aB-9z; Wed, 29 May 2024 09:02:32 +0000
Received: by outflank-mailman (input) for mailman id 731691;
 Wed, 29 May 2024 09:02:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCZ-0006YK-BK
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:31 +0000
Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com
 [2607:f8b0:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2cfd3434-1d9a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:02:29 +0200 (CEST)
Received: by mail-oi1-x232.google.com with SMTP id
 5614622812f47-3d1bb1c3a88so1045063b6e.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:29 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abcc0f27sm450462085a.31.2024.05.29.02.02.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cfd3434-1d9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973347; x=1717578147; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=51wZoBHUI8YnkZDZQOJzY42CcaN9TxiSHoaNW33FOv0=;
        b=a9GP9EOSCE9MyHLQzMFUWTVJIFmRByi4NJctgTO6GqhZhTfXgO5XMTfwZT5/AT2Xdy
         wDjWbO+rbibgxjT8Zusy3iO/ntGE33szBsBxE1KxPL2RTqWJBtUSubNOyFFxfipbSZuK
         xVOcrdnzXC4kJmPGjeTktumriXTH+Yb4+Gu4Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973347; x=1717578147;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=51wZoBHUI8YnkZDZQOJzY42CcaN9TxiSHoaNW33FOv0=;
        b=Os2n9ISTbcGzhLtelnL3KKHgp21fiw8QMjmSZYnpkuTjTgHfZpLH8hHaTdIpM82Lq3
         69g/RXvFwV5oEqVpfmt+CFTQMGnGt4GdjJiUEUW7WkqO33+huXvaaDQGo3T0rFgGUBip
         Fs9y6uo5VfJSDeT48KXgT0w+j1icjRCje1/+7CXcwCibJZ18esNJsD+2ckjFHNm+YduX
         jVjyqUsZb6Ll9t10jo2wuWwNM4hp+8iQF929BFr2fwJhDgDH0+5tYuJyHd1vouzCLWT9
         efJasPt6Xkm9OxKUBI9hY33VoP9MgQmi6gVbZ8rqveh5J7MCrlmCQFap0YRc2g0VENP9
         Hx0Q==
X-Gm-Message-State: AOJu0Yx8whlMjgbDJS3eKrWjsFKwPQU12JK3zG+NXWZMmPcMrqFZOlxI
	8B1A1oe61VtycBgXwLpfI4pTqq/7dljQ7y2vQrkOXEekSg0558mUcwZHi779Sd15K31l+Qx5PIr
	y
X-Google-Smtp-Source: AGHT+IFfACwML2Op18jla+kB7L8l2Ckh1hot8lils5Zo/p1hrUkHo+HwbRRecuy/K1vQlG/qE76xBg==
X-Received: by 2002:a05:6870:a54b:b0:24f:df37:75dc with SMTP id 586e51a60fabf-24fdf383398mr11135367fac.51.1716973347325;
        Wed, 29 May 2024 02:02:27 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 4/9] x86/irq: describe how the interrupt CPU movement works
Date: Wed, 29 May 2024 11:01:26 +0200
Message-ID: <20240529090132.59434-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The logic to move interrupts across CPUs is complex, attempt to provide a
comment that describes the expected behavior so users of the interrupt system
have more context about the usage of the arch_irq_desc structure fields.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/include/asm/irq.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index 413994d2133b..80a3aa7a88b9 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -28,6 +28,32 @@ typedef struct {
 
 struct irq_desc;
 
+/*
+ * Xen logic for moving interrupts around CPUs allows manipulating interrupts
+ * that target remote CPUs.  The logic to move an interrupt from CPU(s) is as
+ * follows:
+ *
+ * 1. cpu_mask and vector is copied to old_cpu_mask and old_vector.
+ * 2. New cpu_mask and vector are set, vector is setup at the new destination.
+ * 3. move_in_progress is set.
+ * 4. Interrupt source is updated to target new CPU and vector.
+ * 5. Interrupts arriving at old_cpu_mask are processed normally.
+ * 6. When an interrupt is delivered at the new destination (cpu_mask) as part
+ *    of acking the interrupt move_in_progress is cleared and move_cleanup_count
+ *    is set to the weight of online CPUs in old_cpu_mask.
+ *    IRQ_MOVE_CLEANUP_VECTOR is sent to all CPUs in old_cpu_mask.
+ * 7. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the
+ *    vector entry and decrease the count in move_cleanup_count.  The CPU that
+ *    sets move_cleanup_count to 0 releases the vector.
+ *
+ * Note that when interrupt movement (either move_in_progress or
+ * move_cleanup_count set) is in progress it's not possible to move the
+ * interrupt to yet a different CPU.
+ *
+ * By keeping the vector in the old CPU(s) configured until the interrupt is
+ * acked on the new destination Xen allows draining any pending interrupts at
+ * the old destinations.
+ */
 struct arch_irq_desc {
         s16 vector;                  /* vector itself is only 8 bits, */
         s16 old_vector;              /* but we use -1 for unassigned  */
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731689.1137343 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCU-00071I-QB; Wed, 29 May 2024 09:02:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731689.1137343; Wed, 29 May 2024 09:02:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCU-00071B-ND; Wed, 29 May 2024 09:02:26 +0000
Received: by outflank-mailman (input) for mailman id 731689;
 Wed, 29 May 2024 09:02:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCT-0006nV-T7
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:25 +0000
Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com
 [2607:f8b0:4864:20::f34])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2a42f51a-1d9a-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:02:24 +0200 (CEST)
Received: by mail-qv1-xf34.google.com with SMTP id
 6a1803df08f44-6ab9d406101so10364256d6.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:24 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac163189cbsm52218246d6.125.2024.05.29.02.02.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a42f51a-1d9a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973343; x=1717578143; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=q0Nh3rBmN+pTUZ9mi1QuaHwm6Qdmm1bW1P320o0L9uQ=;
        b=UrCRNhoAdwKdMwnay26lSkyLKktz/PB5H92GLpZxY27JCpJxj/cf5ZewimxjIy28hr
         fV5XTeAa4MyoxbDVvBL9cm17LFMuRO7GsJxeoVM/UGz6OLS7VaMhpVYhtbiQ/CIhfBEo
         mBUKP+Twdv2nzuy9KtzB8y4SXpuXmzmzbicmM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973343; x=1717578143;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=q0Nh3rBmN+pTUZ9mi1QuaHwm6Qdmm1bW1P320o0L9uQ=;
        b=jKCK9rPPQij7Z2sFb3KGSteqLY6q78V5/TOV71rWFIfCbI3XO8DVuJikl7kqAhiIBS
         g/JA5396YceHog8lOwiDgB8XKPYQkMJbYcOsaDbr3DMeLvl0287vnebNW/xM6MjJlnq0
         KDjN9QyFk8Cx1dVTpFyWZMcIXZoIV4z7rvsqyJZJOPG6sgF/iO3G0M4nKUhbT8r2cAL6
         6O/xKXw1tNd0xxDWxbYZ8WqengB6CVyR2ceW1aw0KNu8iG8khEuoFZnNjgW3xbPso4UW
         VEimtzETbXHnZZtI9Q50CUdr+jc+sg4Kt1kmVWm1P0l3zKyl6ETokyVF1jcJV0bs9wnk
         Br0w==
X-Gm-Message-State: AOJu0Yxg+JxPe7mLlFPVUa02Ni7NyzAo6dT9b4/sZKs4DnfiWVqULpEO
	ZT5doBshy5l01CSd4cnByPlgSV5ZglTsTVjAG2iGlYSPmm8+dFS6KMmIgCgkGvkY6cHy7yIqRQc
	m
X-Google-Smtp-Source: AGHT+IFHRIN1eZv4oe0SSy/cz+Iz9pCra+5E0JzLjhkBniawjYKENoDwI5H8li2IB8bRXo70SdnG0g==
X-Received: by 2002:a05:6214:4a07:b0:6a0:b3cc:ee48 with SMTP id 6a1803df08f44-6abc8cd1be4mr175200136d6.37.1716973342641;
        Wed, 29 May 2024 02:02:22 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH for-4.19 2/9] xen/cpu: do not get the CPU map in stop_machine_run()
Date: Wed, 29 May 2024 11:01:24 +0200
Message-ID: <20240529090132.59434-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current callers of stop_machine_run() outside of init code already have the
CPU maps locked, and hence there's no reason for stop_machine_run() to attempt
to lock again.

Replace the get_cpu_maps() call with a suitable unreachable assert.

Further changes will modify the conditions under which get_cpu_maps() returns
success and without the adjustment proposed here the usage of
stop_machine_run() in cpu_down() would then return an error.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/common/cpu.c          |  5 +++++
 xen/common/stop_machine.c | 15 ++++++++-------
 xen/include/xen/cpu.h     |  2 ++
 3 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/xen/common/cpu.c b/xen/common/cpu.c
index 8709db4d2957..6173220e771b 100644
--- a/xen/common/cpu.c
+++ b/xen/common/cpu.c
@@ -68,6 +68,11 @@ void cpu_hotplug_done(void)
     write_unlock(&cpu_add_remove_lock);
 }
 
+bool cpu_map_locked(void)
+{
+    return rw_is_locked(&cpu_add_remove_lock);
+}
+
 static NOTIFIER_HEAD(cpu_chain);
 
 void __init register_cpu_notifier(struct notifier_block *nb)
diff --git a/xen/common/stop_machine.c b/xen/common/stop_machine.c
index 398cfd507c10..7face75648e8 100644
--- a/xen/common/stop_machine.c
+++ b/xen/common/stop_machine.c
@@ -82,9 +82,15 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
     BUG_ON(!local_irq_is_enabled());
     BUG_ON(!is_idle_vcpu(current));
 
-    /* cpu_online_map must not change. */
-    if ( !get_cpu_maps() )
+    /*
+     * cpu_online_map must not change.  The only two callers of
+     * stop_machine_run() outside of init code already have the CPU map locked.
+     */
+    if ( system_state >= SYS_STATE_active && !cpu_map_locked() )
+    {
+        ASSERT_UNREACHABLE();
         return -EBUSY;
+    }
 
     nr_cpus = num_online_cpus();
     if ( cpu_online(this) )
@@ -92,10 +98,7 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
 
     /* Must not spin here as the holder will expect us to be descheduled. */
     if ( !spin_trylock(&stopmachine_lock) )
-    {
-        put_cpu_maps();
         return -EBUSY;
-    }
 
     stopmachine_data.fn = fn;
     stopmachine_data.fn_data = data;
@@ -136,8 +139,6 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
 
     spin_unlock(&stopmachine_lock);
 
-    put_cpu_maps();
-
     return ret;
 }
 
diff --git a/xen/include/xen/cpu.h b/xen/include/xen/cpu.h
index e1d4eb59675c..d8c8264c58b0 100644
--- a/xen/include/xen/cpu.h
+++ b/xen/include/xen/cpu.h
@@ -13,6 +13,8 @@ void put_cpu_maps(void);
 void cpu_hotplug_begin(void);
 void cpu_hotplug_done(void);
 
+bool cpu_map_locked(void);
+
 /* Receive notification of CPU hotplug events. */
 void register_cpu_notifier(struct notifier_block *nb);
 
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731688.1137333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCT-0006mq-Hw; Wed, 29 May 2024 09:02:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731688.1137333; Wed, 29 May 2024 09:02:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCT-0006mj-F4; Wed, 29 May 2024 09:02:25 +0000
Received: by outflank-mailman (input) for mailman id 731688;
 Wed, 29 May 2024 09:02:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCR-0006YK-Vs
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:23 +0000
Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com
 [2607:f8b0:4864:20::731])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 28b94192-1d9a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:02:22 +0200 (CEST)
Received: by mail-qk1-x731.google.com with SMTP id
 af79cd13be357-794c3946468so117511285a.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794db33d9f8sm73278785a.134.2024.05.29.02.02.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28b94192-1d9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973340; x=1717578140; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=erP1nWqmPbjeOHjS7xu18nxyw0w9w/NsNw7WUoH3eXI=;
        b=r+yAnjlX7xkDSh4NSiYWo7CscaXOqMbnAKxpl6+seBmvqNr8U22JRPbik/7UTILPz8
         5Yc5nrxkEF7NpfRqlSeCWi/YqXKAyZuEFnJXtCHHPPw/re/+/1LHm9u3UBfoN9npEWTQ
         pILzOwNLQlM0L6zf9Re4Qypwyc1YGqRPqtU3M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973340; x=1717578140;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=erP1nWqmPbjeOHjS7xu18nxyw0w9w/NsNw7WUoH3eXI=;
        b=AlScULjXJ163QT54JPhrA6FwcuJKakr8yOiRO/Nl2wKG6QjLtpVmpz9L4XGC1KTMH3
         zcFB8d/LdjuHL1GAX7QXYsKcEUWsh+H5lCNBm004opt+NlGHwvnVXcLEGBMhUX1d68qn
         1ovamcA/aOAzGekx8DFPKb1rbjvA8jYlwLfjCE7xwa0GibSDIvoTVG0ANiiWZVT9Id7B
         TgKvjWOwY9wEveV9QEomCx+bIgFvENj1ovY1ZQook9WRlKZ7TjGsXUxVFLmoXDmKU1sZ
         njwzqXoYPi1eDQrZb7mzM4Smhzhenydgn4aUKDdCcVAO9Eu8cXomWSk1rzGMo4bcPgIc
         cP0g==
X-Gm-Message-State: AOJu0YyMWVcIpnped4PS5JyGXOEFrCDBgFGlzpAj/Wor+Q3dyUjM+dcC
	HDpP301R/tbMmtR1PgkL6P4NoIapXTp3vNWonKb3EsemNl4pfvI6NdHE1I+Vji3OY4kYYMirNXa
	3
X-Google-Smtp-Source: AGHT+IGtDPgZsC09q6labm0ZQe0wplK8jSwdnIGkpbIgFSfZy/tLnmb/8TjhjPNyH7JAOytWhLdvRg==
X-Received: by 2002:a05:620a:394d:b0:794:a993:711e with SMTP id af79cd13be357-794ab060b3cmr1999790485a.16.1716973340284;
        Wed, 29 May 2024 02:02:20 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 1/9] x86/irq: remove offline CPUs from old CPU mask when adjusting move_cleanup_count
Date: Wed, 29 May 2024 11:01:23 +0200
Message-ID: <20240529090132.59434-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When adjusting move_cleanup_count to account for CPUs that are offline also
adjust old_cpu_mask, otherwise further calls to fixup_irqs() could subtract
those again creating and create an imbalance in move_cleanup_count.

Fixes: 472e0b74c5c4 ('x86/IRQ: deal with move cleanup count state in fixup_irqs()')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/irq.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index c16205a9beb6..9716e00e873b 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2572,6 +2572,14 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
             desc->arch.move_cleanup_count -= cpumask_weight(affinity);
             if ( !desc->arch.move_cleanup_count )
                 release_old_vec(desc);
+            else
+                /*
+                 * Adjust old_cpu_mask to account for the offline CPUs,
+                 * otherwise further calls to fixup_irqs() could subtract those
+                 * again and possibly underflow the counter.
+                 */
+                cpumask_and(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
+                            &cpu_online_map);
         }
 
         if ( !desc->action || cpumask_subset(desc->affinity, mask) )
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731692.1137373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCb-0007r6-NN; Wed, 29 May 2024 09:02:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731692.1137373; Wed, 29 May 2024 09:02:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCb-0007qt-Ik; Wed, 29 May 2024 09:02:33 +0000
Received: by outflank-mailman (input) for mailman id 731692;
 Wed, 29 May 2024 09:02:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCa-0006nV-A0
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:32 +0000
Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com
 [2607:f8b0:4864:20::72d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2e5a6174-1d9a-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:02:31 +0200 (CEST)
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-794ab0eb68cso51887985a.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:31 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abca595fsm451836885a.20.2024.05.29.02.02.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e5a6174-1d9a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973350; x=1717578150; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eAuY/xXpNim8cPIlMAplQjmAZ+i26Dd77jwyP57jPsU=;
        b=psIvWBPFrvk87102dToGJzui93PeHM2ALt6pJimyhtj9Jqx/gws6tfycUk+Tp6lH9Q
         WRUBd4dj/h+LgIylwqd0IkKyBXutnkxWBbwfBq1+Rdg5wrdtIE+IT2eN9t004Jocwyi3
         37f0IrudiZIkb0kBrrWeMaURMSzYBCBQaqJH8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973350; x=1717578150;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=eAuY/xXpNim8cPIlMAplQjmAZ+i26Dd77jwyP57jPsU=;
        b=Q1n1OfEuGEK1QDlBixzyAzl7YSQSAHmmkNcLmurWwboz4jsrfttg8pTVQtN2oXP+hd
         iyCTm110LRmg5fGe3yk1l9MjYP3NMeNVmUkDf4jRBDp/pTAfOpYsBSYnAWhayI7VxzDR
         ul9fZ/ZxIyR58/sO+RE/PZNIrX222P3jkK+TNV7xZXlGNqBLGv3jjZrwMd1pmVCeL5Lv
         EB+OFK1Ii6FKwpujsQe0f6nIhjRfSfkxyBau0JiBh58Jg7aQpkHIIE8S6BMmVPR5x4Z4
         ZqA3j8UXE1sWuIx175k49NOFHCMDWfQm0yxUVKE6Gze1dEc4XQJywYXu0IjJYs/8qFHS
         t5kw==
X-Gm-Message-State: AOJu0YzvvnKl+p77R9jVDhVfWR8+/AM78VCXnnfOBD5NywQFP4EjjTHR
	uEaW2VWi/z3v1nlXjldIapJ0SKZBV/sqx8cb99CKs6iHHD/QKBrVTbgOCTqQCnCQi4/VQLS+Foc
	Q
X-Google-Smtp-Source: AGHT+IEX7YTYod6TYWAapgRjN+lQ5kEIMzH3l8YDx0WLvT/Q0qT/SVJs+By7i8tud7uDQbMZHbLf+A==
X-Received: by 2002:a05:620a:6112:b0:794:ca6a:2101 with SMTP id af79cd13be357-794dfb9935bmr229992585a.14.1716973349638;
        Wed, 29 May 2024 02:02:29 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 5/9] x86/irq: limit interrupt movement done by fixup_irqs()
Date: Wed, 29 May 2024 11:01:27 +0200
Message-ID: <20240529090132.59434-6-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current check used in fixup_irqs() to decide whether to move around
interrupts is based on the affinity mask, but such mask can have all bits set,
and hence is unlikely to be a subset of the input mask.  For example if an
interrupt has an affinity mask of all 1s, any input to fixup_irqs() that's not
an all set CPU mask would cause that interrupt to be shuffled around
unconditionally.

What fixup_irqs() care about is evacuating interrupts from CPUs not set on the
input CPU mask, and for that purpose it should check whether the interrupt is
assigned to a CPU not present in the input mask.  Assume that ->arch.cpu_mask
is a subset of the ->affinity mask, and keep the current logic that resets the
->affinity mask if the interrupt has to be shuffled around.

While there also adjust the comment as to the purpose of fixup_irqs().

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Do not AND ->arch.cpu_mask with cpu_online_map.
 - Keep using cpumask_subset().
 - Integrate into bigger series.
---
 xen/arch/x86/include/asm/irq.h | 2 +-
 xen/arch/x86/irq.c             | 9 +++++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index 80a3aa7a88b9..b7dc75d0acbd 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -156,7 +156,7 @@ void free_domain_pirqs(struct domain *d);
 int map_domain_emuirq_pirq(struct domain *d, int pirq, int emuirq);
 int unmap_domain_pirq_emuirq(struct domain *d, int pirq);
 
-/* Reset irq affinities to match the given CPU mask. */
+/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
 void fixup_irqs(const cpumask_t *mask, bool verbose);
 void fixup_eoi(void);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 9716e00e873b..1b7127090377 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2525,7 +2525,7 @@ static int __init cf_check setup_dump_irqs(void)
 }
 __initcall(setup_dump_irqs);
 
-/* Reset irq affinities to match the given CPU mask. */
+/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */
 void fixup_irqs(const cpumask_t *mask, bool verbose)
 {
     unsigned int irq;
@@ -2582,7 +2582,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
                             &cpu_online_map);
         }
 
-        if ( !desc->action || cpumask_subset(desc->affinity, mask) )
+        /*
+         * Avoid shuffling the interrupt around as long as current target CPUs
+         * are a subset of the input mask.  What fixup_irqs() cares about is
+         * evacuating interrupts from CPUs not in the input mask.
+         */
+        if ( !desc->action || cpumask_subset(desc->arch.cpu_mask, mask) )
         {
             spin_unlock(&desc->lock);
             continue;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731693.1137383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCd-0008AM-Ui; Wed, 29 May 2024 09:02:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731693.1137383; Wed, 29 May 2024 09:02:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCd-0008AB-RO; Wed, 29 May 2024 09:02:35 +0000
Received: by outflank-mailman (input) for mailman id 731693;
 Wed, 29 May 2024 09:02:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCc-0006nV-Mr
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:34 +0000
Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com
 [2607:f8b0:4864:20::72c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2fae73e0-1d9a-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:02:34 +0200 (CEST)
Received: by mail-qk1-x72c.google.com with SMTP id
 af79cd13be357-794ba2d4579so116587785a.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:34 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd39bf3sm451529785a.114.2024.05.29.02.02.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fae73e0-1d9a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973352; x=1717578152; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ALobPQ5l/DjhjFTulvS4fT4tWBapunVg8uReYEyLy/E=;
        b=NN6sNLWAMpjm5z/6U1xsOe2F8lj6MiTnZTkEQaJY3IM33I70k12LMRMFhBKc4hb18R
         BOu8PQtnSNAsVyDWQ9fYk1GbQUDKLvUII2QegZKsefLNZxRdXG/CVBmgSfoaYa/OEOV1
         /ON4cJqxYar8+MWdfGmYGJW9X9lR+bYrEGs+w=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973352; x=1717578152;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ALobPQ5l/DjhjFTulvS4fT4tWBapunVg8uReYEyLy/E=;
        b=SdFyns2rw/E64vmszjoQAyUdmlEDDccMfL1NNJ5wcmNyTPYmrvu8bnmcY2J4fTNGGP
         BMQ1Zvr1IODDyR8sPwSmj3CW1R0ZSTOd8qBsG1GTCvDP8kksBV9eVsUp3UBf+TOh8lqr
         6MjHpXkyaDhe0hOmjWpCicMPJxot51FNbtMHsskpaXEC79Sjd+JeuXeqWctrOQyIOtin
         JyNRTIekwA7cgVhXRWjGcBeQQYaaSy2bDOf4R37nzDvAJLaVAysM4FJ43ieA70hMTkH4
         OO99af/klQZCdhBluIevnbMPf80s16LKfWxPnNRfLjLHTynbcl++qxuIxRgzoQxVxEl6
         XiNg==
X-Gm-Message-State: AOJu0Yzap0RTor07fLQxNEHqr1Gc+CZyFn03yp3FCGCVfrNWe9QUz+t7
	ratmDoIj4uc4qha0UvIBx0y1eVbIzBFBGAr1wSDqgBsdbHQBucR6yY7QKAgh7cgLlbKDt0+OhC7
	l
X-Google-Smtp-Source: AGHT+IFeh7suErWELrxwwMWXUq8jDXXBH01dea6zwgIMVCoOkjB04GbH9B18swjZG3RrGBpGvWUrUw==
X-Received: by 2002:a05:620a:4141:b0:790:789d:b3d0 with SMTP id af79cd13be357-794ab099dd1mr1642553285a.38.1716973352006;
        Wed, 29 May 2024 02:02:32 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 6/9] x86/irq: restrict CPU movement in set_desc_affinity()
Date: Wed, 29 May 2024 11:01:28 +0200
Message-ID: <20240529090132.59434-7-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

If external interrupts are using logical mode it's possible to have an overlap
between the current ->arch.cpu_mask and the provided mask (or TARGET_CPUS).  If
that's the case avoid assigning a new vector and just move the interrupt to a
member of ->arch.cpu_mask that overlaps with the provided mask and is online.

While there also add an extra assert to ensure the mask containing the possible
destinations is not empty before calling cpu_mask_to_apicid(), as at that point
having an empty mask is not expected.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/irq.c | 34 +++++++++++++++++++++++++++-------
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 1b7127090377..ae8fa574d4e7 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -846,19 +846,38 @@ void cf_check irq_complete_move(struct irq_desc *desc)
 
 unsigned int set_desc_affinity(struct irq_desc *desc, const cpumask_t *mask)
 {
-    int ret;
-    unsigned long flags;
     cpumask_t dest_mask;
 
     if ( mask && !cpumask_intersects(mask, &cpu_online_map) )
         return BAD_APICID;
 
-    spin_lock_irqsave(&vector_lock, flags);
-    ret = _assign_irq_vector(desc, mask ?: TARGET_CPUS);
-    spin_unlock_irqrestore(&vector_lock, flags);
+    /*
+     * mask input set con contain CPUs that are not online.  To decide whether
+     * the interrupt needs to be migrated restrict the input mask to the CPUs
+     * that are online.
+     */
+    if ( mask )
+        cpumask_and(&dest_mask, mask, &cpu_online_map);
+    else
+        cpumask_copy(&dest_mask, TARGET_CPUS);
 
-    if ( ret < 0 )
-        return BAD_APICID;
+    /*
+     * Only move the interrupt if there are no CPUs left in ->arch.cpu_mask
+     * that can handle it, otherwise just shuffle it around ->arch.cpu_mask
+     * to an available destination.
+     */
+    if ( !cpumask_intersects(desc->arch.cpu_mask, &dest_mask) )
+    {
+        int ret;
+        unsigned long flags;
+
+        spin_lock_irqsave(&vector_lock, flags);
+        ret = _assign_irq_vector(desc, mask ?: TARGET_CPUS);
+        spin_unlock_irqrestore(&vector_lock, flags);
+
+        if ( ret < 0 )
+            return BAD_APICID;
+    }
 
     if ( mask )
     {
@@ -871,6 +890,7 @@ unsigned int set_desc_affinity(struct irq_desc *desc, const cpumask_t *mask)
         cpumask_copy(&dest_mask, desc->arch.cpu_mask);
     }
     cpumask_and(&dest_mask, &dest_mask, &cpu_online_map);
+    ASSERT(!cpumask_empty(&dest_mask));
 
     return cpu_mask_to_apicid(&dest_mask);
 }
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731694.1137393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCg-0008Uk-8K; Wed, 29 May 2024 09:02:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731694.1137393; Wed, 29 May 2024 09:02:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCg-0008UO-5B; Wed, 29 May 2024 09:02:38 +0000
Received: by outflank-mailman (input) for mailman id 731694;
 Wed, 29 May 2024 09:02:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCf-0006nV-Gh
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:37 +0000
Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com
 [2607:f8b0:4864:20::f31])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31767148-1d9a-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:02:37 +0200 (CEST)
Received: by mail-qv1-xf31.google.com with SMTP id
 6a1803df08f44-6ad74e5afeaso3776796d6.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:37 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ac17a0a904sm51825196d6.140.2024.05.29.02.02.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31767148-1d9a-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973355; x=1717578155; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UBMxjkuwWl1wII73dXqlBR3D5ru/taoCebE2CrhXMFU=;
        b=VTybKQemf7cHSxbVVTJb3uNNKfTB+KGbuRp3klWjuRwfIe82OLN6JIRMJXv0SElzp8
         wIwFR02fAgDdW6iqa5fcnRnLID6CtQR7aFYMCZD3l8gvufitWms4uSj3/mLbe+Wlh4+s
         gYohS9eJ/KfQW+CH/iOdW3g0sRss1B3kc6F8o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973355; x=1717578155;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UBMxjkuwWl1wII73dXqlBR3D5ru/taoCebE2CrhXMFU=;
        b=U8Fsfvxy5r79F0JT0BsngRaDB9+JIZKUckwqOtxXIDCXi3CRrLmaBJ0+ixF6GZa+Zg
         pu6CWEz0fnm2EMneghYp78h83+hHJt0FzJ0FcFsib1s4O7pmQsHOPl0NMv3RhI8vd8gE
         Cv4GBbhM1YUmnLv2OyDAvwcd0B6APRInWzbZGYncG1z59ciVtcP9k0qOwr8v6TVkOJcg
         Vjbf0D6seRq282k0YVpX6UGo1d1ZCX7JKy3cQ8A48sKy4zN/LPpF5DqLa8vr7vleKcfh
         s1B+ABcGCVsJvxV/VFnnVCPSKp5c3VOg16mFC0uHTstlFohYi+5fexvzzf1wk+UltOKA
         21yg==
X-Gm-Message-State: AOJu0YygypSdB1HgIyriMq0+60CICpOaAd7HIez4Ej/WT7e/GH3G0U4j
	YXxf0I4b/a/ZFdsCXJf7HvTzLesQHe1HbtIIgCQjJjaQKIqtJFgRTF/NCPS08eKbyEOQmMFD+2M
	d
X-Google-Smtp-Source: AGHT+IGDSyevI0ED2KfpIKZoqT7Y+oUaLl05GYOTvdKaWQqSzqfErWgaoruVthSi4UMNX+Z5fAaxJQ==
X-Received: by 2002:a05:6214:20a6:b0:6ad:5cb2:b6c with SMTP id 6a1803df08f44-6ad9f9596e6mr24131236d6.9.1716973354286;
        Wed, 29 May 2024 02:02:34 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 7/9] x86/irq: deal with old_cpu_mask for interrupts in movement in fixup_irqs()
Date: Wed, 29 May 2024 11:01:29 +0200
Message-ID: <20240529090132.59434-8-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Given the current logic it's possible for ->arch.old_cpu_mask to get out of
sync: if a CPU set in old_cpu_mask is offlined and then onlined
again without old_cpu_mask having been updated the data in the mask will no
longer be accurate, as when brought back online the CPU will no longer have
old_vector configured to handle the old interrupt source.

If there's an interrupt movement in progress, and the to be offlined CPU (which
is the call context) is in the old_cpu_mask clear it and update the mask, so it
doesn't contain stale data.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/irq.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index ae8fa574d4e7..8822f382bfe4 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2602,6 +2602,28 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
                             &cpu_online_map);
         }
 
+        if ( desc->arch.move_in_progress &&
+             !cpumask_test_cpu(cpu, &cpu_online_map) &&
+             cpumask_test_cpu(cpu, desc->arch.old_cpu_mask) )
+        {
+            /*
+             * This CPU is going offline, remove it from ->arch.old_cpu_mask
+             * and possibly release the old vector if the old mask becomes
+             * empty.
+             *
+             * Note cleaning ->arch.old_cpu_mask is required if the CPU is
+             * brought offline and then online again, as when re-onlined the
+             * per-cpu vector table will no longer have ->arch.old_vector
+             * setup, and hence ->arch.old_cpu_mask would be stale.
+             */
+            cpumask_clear_cpu(cpu, desc->arch.old_cpu_mask);
+            if ( cpumask_empty(desc->arch.old_cpu_mask) )
+            {
+                desc->arch.move_in_progress = 0;
+                release_old_vec(desc);
+            }
+        }
+
         /*
          * Avoid shuffling the interrupt around as long as current target CPUs
          * are a subset of the input mask.  What fixup_irqs() cares about is
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731696.1137403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCk-0000VQ-Kc; Wed, 29 May 2024 09:02:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731696.1137403; Wed, 29 May 2024 09:02:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCk-0000VH-Gj; Wed, 29 May 2024 09:02:42 +0000
Received: by outflank-mailman (input) for mailman id 731696;
 Wed, 29 May 2024 09:02:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCj-0006YK-8g
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:41 +0000
Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com
 [2607:f8b0:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 329e9554-1d9a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:02:39 +0200 (CEST)
Received: by mail-ot1-x32e.google.com with SMTP id
 46e09a7af769-6f12ed79fdfso977450a34.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:39 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb1800900sm51957851cf.49.2024.05.29.02.02.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 329e9554-1d9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973357; x=1717578157; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vw6o6qcvvPGFZkVuxNsOOgXzV4/8wiApb/K/FBe9Ung=;
        b=Lj4UUHhIye+Rmj3k6w5hYyy1hvn97MWilKKJ5dQZAWYOV0DlfU5iJVRhVeQPs4TELf
         TiWFfFXDc4DlpY7ZbiQdkhE/2DEkC9a7pIB3+bOzAEHeUMv7Sj9/sYHklNJZVuHuGG+o
         NwXP40hVBeXXIXXsfk6sBQRD/teKnpQhREgjo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973357; x=1717578157;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=vw6o6qcvvPGFZkVuxNsOOgXzV4/8wiApb/K/FBe9Ung=;
        b=MB9f6Y/4MjDopF4x2/1i4eK8ZsxCRjk76D1vwktTcxHi4pBQa9hPFk3lp9Af436CYK
         xw6Y+Lk6/MiFwRHrav312MQueX4furrlGNKyHyDHLIfUIEK0QVD+6HndG/yh390lU0Y3
         ih419QHOz9VlRoboeiYIJDqamJG6OmY6haKoEU6bd9NJmwYmE9Nums57mrpDWRx6WZ95
         /1r8YGOU0XGFD4ewvGe87q46BPull9wOCwyXSQlDt3wwLXcmf0bauL8ylTn5GNk1ZfTv
         E/qzrtEDN7i5laCLlecfRcbaftVKBRJuHG+p4C+TDyZMHE0m9vSfw7gaDjwVZpnChRT0
         m5/g==
X-Gm-Message-State: AOJu0YxYpKvD1yRHpUAfY6kJVTCg4GT2uo2GMo97bK8gSk3W2LplMP6e
	ojiLqJOZGLufuLG94F/w57ENbD0kOR7uBayDDp5s0lVBKI1E/6/4uiK1fAitUO7mnG7ML1gp7Py
	b
X-Google-Smtp-Source: AGHT+IFLHUdyzkgrzHVkeQlrYo1dYeqmlu2o9LZO6GV5j4xp6V8azEcXB3lbTGzHXJWxe/5WS1YSzA==
X-Received: by 2002:a9d:6246:0:b0:6f1:2485:6d7e with SMTP id 46e09a7af769-6f8d0b9258cmr13539533a34.36.1716973356660;
        Wed, 29 May 2024 02:02:36 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 8/9] x86/irq: handle moving interrupts in _assign_irq_vector()
Date: Wed, 29 May 2024 11:01:30 +0200
Message-ID: <20240529090132.59434-9-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently there's logic in fixup_irqs() that attempts to prevent
_assign_irq_vector() from failing, as fixup_irqs() is required to evacuate all
interrupts form the CPUs not present in the input mask.  The current logic in
fixup_irqs() is incomplete, as it doesn't deal with interrupts that have
move_cleanup_count > 0 and a non-empty ->arch.old_cpu_mask field.

Instead of attempting to fixup the interrupt descriptor in fixup_irqs() so that
_assign_irq_vector() cannot fail, introduce some logic in _assign_irq_vector()
to deal with interrupts that have either move_{in_progress,cleanup_count} set
and no remaining online CPUs in ->arch.cpu_mask.

If _assign_irq_vector() is requested to move an interrupt in the state
described above, first attempt to see if ->arch.old_cpu_mask contains any valid
CPUs that could be used as fallback, and if that's the case do move the
interrupt back to the previous destination.  Note this is easier because the
vector hasn't been released yet, so there's no need to allocate and setup a new
vector on the destination.

Due to the logic in fixup_irqs() that clears offline CPUs from
->arch.old_cpu_mask (and releases the old vector if the mask becomes empty) it
shouldn't be possible to get into _assign_irq_vector() with
->arch.move_{in_progress,cleanup_count} set but no online CPUs in
->arch.old_cpu_mask.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/irq.c | 66 ++++++++++++++++++++++++++--------------------
 1 file changed, 38 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 8822f382bfe4..33979729257e 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -553,7 +553,44 @@ static int _assign_irq_vector(struct irq_desc *desc, const cpumask_t *mask)
     }
 
     if ( desc->arch.move_in_progress || desc->arch.move_cleanup_count )
-        return -EAGAIN;
+    {
+        /*
+         * If the current destination is online refuse to shuffle.  Retry after
+         * the in-progress movement has finished.
+         */
+        if ( cpumask_intersects(desc->arch.cpu_mask, &cpu_online_map) )
+            return -EAGAIN;
+
+        /*
+         * Fallback to the old destination if moving is in progress and the
+         * current destination is to be offlined.  This is required by
+         * fixup_irqs() to evacuate interrupts from a CPU to be offlined.
+         *
+         * Due to the logic in fixup_irqs() that clears offlined CPUs from
+         * ->arch.old_cpu_mask it shouldn't be possible to get here with
+         * ->arch.move_{in_progress,cleanup_count} set and no online CPUs in
+         * ->arch.old_cpu_mask.
+         */
+        ASSERT(valid_irq_vector(desc->arch.old_vector));
+        ASSERT(cpumask_intersects(desc->arch.old_cpu_mask, mask));
+        ASSERT(cpumask_intersects(desc->arch.old_cpu_mask, &cpu_online_map));
+
+        /* Set the old destination as the current one. */
+        desc->arch.vector = desc->arch.old_vector;
+        cpumask_and(desc->arch.cpu_mask, desc->arch.old_cpu_mask, mask);
+
+        /* Undo any possibly done cleanup. */
+        for_each_cpu(cpu, desc->arch.cpu_mask)
+            per_cpu(vector_irq, cpu)[desc->arch.vector] = irq;
+
+        /* Cancel the pending move. */
+        desc->arch.old_vector = IRQ_VECTOR_UNASSIGNED;
+        cpumask_clear(desc->arch.old_cpu_mask);
+        desc->arch.move_in_progress = 0;
+        desc->arch.move_cleanup_count = 0;
+
+        return 0;
+    }
 
     err = -ENOSPC;
 
@@ -2635,33 +2672,6 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
             continue;
         }
 
-        /*
-         * In order for the affinity adjustment below to be successful, we
-         * need _assign_irq_vector() to succeed. This in particular means
-         * clearing desc->arch.move_in_progress if this would otherwise
-         * prevent the function from succeeding. Since there's no way for the
-         * flag to get cleared anymore when there's no possible destination
-         * left (the only possibility then would be the IRQs enabled window
-         * after this loop), there's then also no race with us doing it here.
-         *
-         * Therefore the logic here and there need to remain in sync.
-         */
-        if ( desc->arch.move_in_progress &&
-             !cpumask_intersects(mask, desc->arch.cpu_mask) )
-        {
-            unsigned int cpu;
-
-            cpumask_and(affinity, desc->arch.old_cpu_mask, &cpu_online_map);
-
-            spin_lock(&vector_lock);
-            for_each_cpu(cpu, affinity)
-                per_cpu(vector_irq, cpu)[desc->arch.old_vector] = ~irq;
-            spin_unlock(&vector_lock);
-
-            release_old_vec(desc);
-            desc->arch.move_in_progress = 0;
-        }
-
         if ( !cpumask_intersects(mask, desc->affinity) )
         {
             break_affinity = true;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:02:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:02:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731698.1137413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCm-0000wx-Uk; Wed, 29 May 2024 09:02:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731698.1137413; Wed, 29 May 2024 09:02:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFCm-0000wb-Pt; Wed, 29 May 2024 09:02:44 +0000
Received: by outflank-mailman (input) for mailman id 731698;
 Wed, 29 May 2024 09:02:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCFCk-0006YK-OZ
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:02:42 +0000
Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com
 [2607:f8b0:4864:20::f29])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 33d8ac75-1d9a-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:02:41 +0200 (CEST)
Received: by mail-qv1-xf29.google.com with SMTP id
 6a1803df08f44-6ab9d9ad0c0so8254406d6.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:02:41 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad96fac09esm12093906d6.32.2024.05.29.02.02.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:02:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33d8ac75-1d9a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716973359; x=1717578159; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0uM4Rhz8t2QnRPi9kZmp+UEqFUepWlQwqYUfgN8dy9c=;
        b=e+xCEhtAQCp3qNhOk2GTGEcioOcrlflSePRuLPytC8M77bXWqazMR3EvNKNnleqpPT
         FW0I9SDB7h9jMqPvWJxW0WbC3z10gSXLf1a+fwbundu71Ybid3Dwmczc2fOdrau9qSfx
         2He6uiR2e/IH6OQ3HBuioJkRMGko9ScYYt2MA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716973359; x=1717578159;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0uM4Rhz8t2QnRPi9kZmp+UEqFUepWlQwqYUfgN8dy9c=;
        b=YzF0yNC3c9zx84sLeqqmeD3nCa2wx5PDxHKHh2bsaGoi1rPGnMfKu/LtKqB2Xmjl3U
         GTHuGlBpj4/KCirArZ6cCZRcbMigCOoHLgpvPgZGwqVsuxtp8aYV7cWVz8HAVs1ZyJDY
         wUT5d9lVaPfvE0HKV7fpqPO8inHmkvS57XNUICcypUKh8OkpTlYfreFLdEY5hqK76Ffq
         r/XuGtyXSndFPMEEE+HuSjBOZQEZwKD7UFGhIrybVl+xdoliooy/lnV8Nb9M3NTHYfx/
         fhpdNPauzrrM81/YugCd6Dzygv3HaXCIC43pTPzRpIWB5EMOMh+Gu4BGFuXtL2TjQPTv
         ddBQ==
X-Gm-Message-State: AOJu0YwQnd1XdBaAmPZ8S5wfbtADgh6Pb4AiukM46rj00leaSrT0pUOT
	NGUrYsXdglJtN1iOuq3OdE7z8SaojuxdH1cUYiIyOmLT9tcxvM7OF25KdYMtY09CXK8jBVoncLN
	C
X-Google-Smtp-Source: AGHT+IFVnbdL74wVKWAekUSIphAYXJzvIYyS86bdPsV32FCEpHOK4om7tjuX8q29tjlykqBQJmATNA==
X-Received: by 2002:a05:6214:3a0a:b0:6ab:71d6:cbd5 with SMTP id 6a1803df08f44-6abcd169caamr175634936d6.53.1716973359026;
        Wed, 29 May 2024 02:02:39 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH for-4.19 9/9] x86/irq: forward pending interrupts to new destination in fixup_irqs()
Date: Wed, 29 May 2024 11:01:31 +0200
Message-ID: <20240529090132.59434-10-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

fixup_irqs() is used to evacuate interrupts from to be offlined CPUs.  Given
the CPU is to become offline, the normal migration logic used by Xen where the
vector in the previous target(s) is left configured until the interrupt is
received on the new destination is not suitable.

Instead attempt to do as much as possible in order to prevent loosing
interrupts.  If fixup_irqs() is called from the CPU to be offlined (as is
currently the case) attempt to forward pending vectors when interrupts that
target the current CPU are migrated to a different destination.

Additionally, for interrupts that have already been moved from the current CPU
prior to the call to fixup_irqs() but that haven't been delivered to the new
destination (iow: interrupts with move_in_progress set and the current CPU set
in ->arch.old_cpu_mask) also check whether the previous vector is pending and
forward it to the new destination.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/apic.c             |  5 +++++
 xen/arch/x86/include/asm/apic.h |  3 +++
 xen/arch/x86/irq.c              | 39 +++++++++++++++++++++++++++++++--
 3 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 6567af685a1b..4d77fba3ed19 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1543,3 +1543,8 @@ void check_for_unexpected_msi(unsigned int vector)
 {
     BUG_ON(apic_isr_read(vector));
 }
+
+bool lapic_check_pending(unsigned int vector)
+{
+    return apic_read(APIC_IRR + (vector / 32 * 0x10)) & (1U << (vector % 32));
+}
diff --git a/xen/arch/x86/include/asm/apic.h b/xen/arch/x86/include/asm/apic.h
index d1cb001fb4ab..7b5a0832c05e 100644
--- a/xen/arch/x86/include/asm/apic.h
+++ b/xen/arch/x86/include/asm/apic.h
@@ -179,6 +179,9 @@ extern void record_boot_APIC_mode(void);
 extern enum apic_mode current_local_apic_mode(void);
 extern void check_for_unexpected_msi(unsigned int vector);
 
+/* Return whether vector is pending in IRR. */
+bool lapic_check_pending(unsigned int vector);
+
 uint64_t calibrate_apic_timer(void);
 uint32_t apic_tmcct_read(void);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 33979729257e..211ad3bd7cf5 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2591,8 +2591,8 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
 
     for ( irq = 0; irq < nr_irqs; irq++ )
     {
-        bool break_affinity = false, set_affinity = true;
-        unsigned int vector;
+        bool break_affinity = false, set_affinity = true, check_irr = false;
+        unsigned int vector, cpu = smp_processor_id();
         cpumask_t *affinity = this_cpu(scratch_cpumask);
 
         if ( irq == 2 )
@@ -2643,6 +2643,25 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
              !cpumask_test_cpu(cpu, &cpu_online_map) &&
              cpumask_test_cpu(cpu, desc->arch.old_cpu_mask) )
         {
+            /*
+             * This to be offlined CPU was the target of an interrupt that's
+             * been moved, and the new destination target hasn't yet
+             * acknowledged any interrupt from it.
+             *
+             * We know the interrupt is configured to target the new CPU at
+             * this point, so we can check IRR for any pending vectors and
+             * forward them to the new destination.
+             *
+             * Note the difference between move_in_progress or
+             * move_cleanup_count being set.  For the later we know the new
+             * destination has already acked at least one interrupt from this
+             * source, and hence there's no need to forward any stale
+             * interrupts.
+             */
+            if ( lapic_check_pending(desc->arch.old_vector) )
+                send_IPI_mask(cpumask_of(cpumask_any(desc->arch.cpu_mask)),
+                              desc->arch.vector);
+
             /*
              * This CPU is going offline, remove it from ->arch.old_cpu_mask
              * and possibly release the old vector if the old mask becomes
@@ -2683,11 +2702,27 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
         if ( desc->handler->disable )
             desc->handler->disable(desc);
 
+        /*
+         * If the current CPU is going offline and is (one of) the target(s) of
+         * the interrupt signal to check whether there are any pending vectors
+         * to be handled in the local APIC after the interrupt has been moved.
+         */
+        if ( !cpu_online(cpu) && cpumask_test_cpu(cpu, desc->arch.cpu_mask) )
+            check_irr = true;
+
         if ( desc->handler->set_affinity )
             desc->handler->set_affinity(desc, affinity);
         else if ( !(warned++) )
             set_affinity = false;
 
+        if ( check_irr && lapic_check_pending(vector) )
+            /*
+             * Forward pending interrupt to the new destination, this CPU is
+             * going offline and otherwise the interrupt would be lost.
+             */
+            send_IPI_mask(cpumask_of(cpumask_any(desc->arch.cpu_mask)),
+                          desc->arch.vector);
+
         if ( desc->handler->enable )
             desc->handler->enable(desc);
 
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:27:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:27:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731749.1137422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFay-0008U1-Re; Wed, 29 May 2024 09:27:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731749.1137422; Wed, 29 May 2024 09:27:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFay-0008Tu-Oq; Wed, 29 May 2024 09:27:44 +0000
Received: by outflank-mailman (input) for mailman id 731749;
 Wed, 29 May 2024 09:27:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCFay-0008Tk-6G; Wed, 29 May 2024 09:27:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCFax-0003Ji-Vy; Wed, 29 May 2024 09:27:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCFax-0006rl-Mu; Wed, 29 May 2024 09:27:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCFax-0001IR-MP; Wed, 29 May 2024 09:27:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=U0Rd9mrsMzsKgp8s5bi0G3tkCQPrCpGUXehM3SHpr9Y=; b=bMLdx4iDlbqBoyyHmwULmBNzXC
	G7LA/DXjeozbSKAbIN8/42Nm9hyDUNEAVWBjZBXM7H7BYW4m86lM9VTUKgpN1xYLf9Odw1n1NsVJn
	+gu8Li+nsynClPzlXSWvPDlEbZzHjTZGuiL1XxO9xigToAnbNv7m5pu92MRecatyj4f0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186179-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186179: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=38c6c364088e3ba2d7e0b18690af6800d338c6a9
X-Osstest-Versions-That:
    libvirt=a4f38f6ffe6a9edc001d18890ccfc3f38e72fb94
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 09:27:43 +0000

flight 186179 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186179/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186165
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              38c6c364088e3ba2d7e0b18690af6800d338c6a9
baseline version:
 libvirt              a4f38f6ffe6a9edc001d18890ccfc3f38e72fb94

Last test of basis   186165  2024-05-28 04:20:40 Z    1 days
Testing same since   186179  2024-05-29 04:18:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Jiri Denemark <jdenemar@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   a4f38f6ffe..38c6c36408  38c6c364088e3ba2d7e0b18690af6800d338c6a9 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed May 29 09:33:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:33:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731756.1137433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFgc-00038x-HQ; Wed, 29 May 2024 09:33:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731756.1137433; Wed, 29 May 2024 09:33:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFgc-00038q-DP; Wed, 29 May 2024 09:33:34 +0000
Received: by outflank-mailman (input) for mailman id 731756;
 Wed, 29 May 2024 09:33:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCFgb-00038k-98
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:33:33 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83552b71-1d9e-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 11:33:32 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a62ef52e837so197980466b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:33:31 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c817b0asm701607466b.3.2024.05.29.02.33.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 02:33:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83552b71-1d9e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716975211; x=1717580011; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Tq91sQ7nMEnbVq0UOMM1nyvnrVt5+YqmvGQDF/9V2UQ=;
        b=f6Q1vi5WdRumZePjXpEdl7AZGNdTOoPfibQjIGLeVAd0cdnNrLaGbdgpRMifH9wcpc
         XZgJOrEZHjgfjIMgBsDglfVC+qdTZWU3r2L8DlH/GH0MlA5OcPOgq41y0sb8v/XOGShX
         ZfsKX/vDQeqXK/jwxYLO1PhwpYvkhQkv7nIh8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716975211; x=1717580011;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tq91sQ7nMEnbVq0UOMM1nyvnrVt5+YqmvGQDF/9V2UQ=;
        b=qmqdBOounqH0t6XXHdNhygTWDyTlpOLERAI4ZXr6S84qtEE5aZn93ATFlUwPolaTm/
         KFDlXaQ8xb4EjFnU5TTCMm7ZZOOJx+0Bwfn0+5VJFblvxfuHRs9x7HpdZO7xSKOqY2tu
         NM4Q0/a3oyqeNWJSt9pmMljTedLXNK6Pu7DdctR2qZh5rQ6cTCEBze4MZF2aUEVyDTbi
         Pc8PDkOjWyjVpTbnhI3zGt2zvE9kCsgZQEq4IUnck2u83NE6sF83PswykKeFxwOrpYRT
         L34a8F5LDWDMwstGVpnzexoyKtOUgiUfPvJOQf35ApsS/s1AiZFnNfC6dK7eSkpcyPyJ
         Fh0Q==
X-Forwarded-Encrypted: i=1; AJvYcCVt5csBIEn4VCEudvgZxkwDrOhUSXzmEs4p0aRqm71zDVwtXkqtDN2d8goI+kXYMuEFcEoRGS55ds2XSjxJl8IYPJ+y95mxfd8VsC/SoBQ=
X-Gm-Message-State: AOJu0YxpvjB7MQFQCSgrAtWu+NAtr6OYGs7h6Lc1vPaSJQy8a3VTN1zK
	EQTBOYxQM8tggaSvhPU3YMwni1EPkuINb1oPg6XZ3IMkA211rkOMwYYwmJGQBNI=
X-Google-Smtp-Source: AGHT+IHEyyNHbJ24rsoHkCw6+E/xiD3ZDWVl4a+VTsiyXAhQb9bZZoRJ8Ip2j14k9ziLuOLJ3T1QvQ==
X-Received: by 2002:a17:906:1945:b0:a59:b807:330a with SMTP id a640c23a62f3a-a62641cf941mr1040170366b.32.1716975211237;
        Wed, 29 May 2024 02:33:31 -0700 (PDT)
Message-ID: <85b946d8-9f14-480e-9c47-20ace4587f4b@citrix.com>
Date: Wed, 29 May 2024 10:33:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 9/9] x86/irq: forward pending interrupts to new
 destination in fixup_irqs()
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-10-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240529090132.59434-10-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 10:01 am, Roger Pau Monne wrote:
> diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
> index 6567af685a1b..4d77fba3ed19 100644
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -1543,3 +1543,8 @@ void check_for_unexpected_msi(unsigned int vector)
>  {
>      BUG_ON(apic_isr_read(vector));
>  }
> +
> +bool lapic_check_pending(unsigned int vector)
> +{
> +    return apic_read(APIC_IRR + (vector / 32 * 0x10)) & (1U << (vector % 32));
> +}
> diff --git a/xen/arch/x86/include/asm/apic.h b/xen/arch/x86/include/asm/apic.h
> index d1cb001fb4ab..7b5a0832c05e 100644
> --- a/xen/arch/x86/include/asm/apic.h
> +++ b/xen/arch/x86/include/asm/apic.h
> @@ -179,6 +179,9 @@ extern void record_boot_APIC_mode(void);
>  extern enum apic_mode current_local_apic_mode(void);
>  extern void check_for_unexpected_msi(unsigned int vector);
>  
> +/* Return whether vector is pending in IRR. */
> +bool lapic_check_pending(unsigned int vector);
> +

As a minor point, we've already got apic_isr_read() as a static inline. 
This wants to be a matching apic_irr_read() IMO.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 29 09:37:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:37:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731762.1137444 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFkb-0003mg-5n; Wed, 29 May 2024 09:37:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731762.1137444; Wed, 29 May 2024 09:37:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCFkb-0003mZ-11; Wed, 29 May 2024 09:37:41 +0000
Received: by outflank-mailman (input) for mailman id 731762;
 Wed, 29 May 2024 09:37:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCFkZ-0003mT-BK
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:37:39 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15b5f5f8-1d9f-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 11:37:37 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso130482466b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 02:37:37 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6590d02e53sm15473266b.119.2024.05.29.02.37.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 02:37:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15b5f5f8-1d9f-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716975457; x=1717580257; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=XcNnbIHkW/U9vIREQElgqUTXZk5+980LQo63v6C3vg4=;
        b=DNyaLGMg7Bur/X8crHNIVg5LReMTqL4Jug1H3t/OfZeR/fdixSWLJStUMYFPB4+4lL
         KltPNfpaMr7qdX16B2tF0ynBGGPWxFnqZx5PHtuY5jI3p3Y8sZxflLQhmEYuNI1A9wUs
         OOHkSmtEtCyeVwx/n5fkTdJo4zYkyEzWqG1JG7qU+d8H6G36JhIVcepGruKqmdl6hCmJ
         mypdDwkLvYUW52Fm3ffS7SrNINLH83QdNv+Qqz7efPqdOOr2CZzQdn5tVAEJf+cf8+6w
         tN67jvPE4PyGZf5g4mdSlpccD5c5U+hX582FSEN37S3In8BDnVhQex3Qt3Tvk1UhCsOn
         ix9w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716975457; x=1717580257;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XcNnbIHkW/U9vIREQElgqUTXZk5+980LQo63v6C3vg4=;
        b=Iz88xn/H2lCIMui6Zc1uItzNTCCelsMRVvAJBVZlQxjoTAAEImTljtMon3mvAi6NPv
         TpyRf3RPk88XzEmv6RRGiVd0yVChrQaDqJrvPB7MtL7fGWvg+O8kVJFeIUjkCe1A7dXn
         iQYI5DSytEosSOalP3G7GCovrd5qvwubw43T1ni80livez156W+5XhFGQ5JTnK+VXBZk
         ZEkCMuFVUPJmxZIwEDlAGkF+4YPWXDvUzVFizYiKKfyIiEGXTKspuJSZxxfmsZnJX8bY
         wv+C4PWYxGnd7tyO4JSv9ASPnNmm4Hw2bi7zwCjIg4InKZ5oLJcaWDIYtM1hp1xuYshc
         nsAQ==
X-Forwarded-Encrypted: i=1; AJvYcCV2JsGH/owLJIVdoADmKp0zWWMFuoMnPRr9XtGrfFTVkWzQXk6C0UgZdfy3lUfH5+BdVpyoxnuL0h6BfuOHKWyIPXjQnp6I0HW6dtnGk7o=
X-Gm-Message-State: AOJu0YzsoHbDbImMdgL0+gjRJqLep2ynhnKRkOMQzd/GHWpcuTqtmfN+
	cKhImS5C8t90SD0wT3OMyt1qSprNkERz+WCE7bs5yH3XCa9y3uRS
X-Google-Smtp-Source: AGHT+IG7PHF2zT8oN+tH8Ybs8ivLzyNjYwo0eMhqKUh2ITdHkdmh54XSGmya+06zH4SCeaqQ+FxClQ==
X-Received: by 2002:a17:906:319b:b0:a59:bf27:5f2e with SMTP id a640c23a62f3a-a642d5a384fmr140282466b.20.1716975456668;
        Wed, 29 May 2024 02:37:36 -0700 (PDT)
Message-ID: <96cbb9df754f35d8df805df0138c942466a8f904.camel@gmail.com>
Subject: Re: [PATCH for-4.19 0/9] x86/irq: fixes for CPU hot{,un}plug
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>,  George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Date: Wed, 29 May 2024 11:37:35 +0200
In-Reply-To: <20240529090132.59434-1-roger.pau@citrix.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 11:01 +0200, Roger Pau Monne wrote:
> Hello,
>=20
> The following series aim to fix interrupt handling when doing CPU
> plug/unplug operations.=C2=A0 Without this series running:
>=20
> cpus=3D`xl info max_cpu_id`
> while [ 1 ]; do
> =C2=A0=C2=A0=C2=A0 for i in `seq 1 $cpus`; do
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xen-hptool cpu-offline $i;
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xen-hptool cpu-online $i;
> =C2=A0=C2=A0=C2=A0 done
> done
>=20
> Quite quickly results in interrupts getting lost and "No irq handler
> for
> vector" messages on the Xen console.=C2=A0 Drivers in dom0 also start
> getting
> interrupt timeouts and the system becomes unusable.
>=20
> After applying the series running the loop over night still result in
> a
> fully usable system, no=C2=A0 "No irq handler for vector" messages at all=
,
> no
> interrupt loses reported by dom0.=C2=A0 Test with
> x2apic-mode=3D{mixed,cluster}.
>=20
> I'm tagging this for 4.19 as it's IMO bugfixes, but the series has
> grown
> quite bigger than expected, and hence we need to be careful to not
> introduce breakages late in the release cycle.=C2=A0 I've attempted to
> document all code as good as I could, interrupt handling has some
> unexpected corner cases that are hard to diagnose and reason about.
Despite of the fact that it can be considered as bugfixes, it seems to
me that this patch series can be risky. Let's wait for maintainers
opinion...

~ Oleksii
>=20
> I'm currently also doing some extra testing with XenRT in case I've
> missed something.
>=20
> Thanks, Roger.
>=20
> Roger Pau Monne (9):
> =C2=A0 x86/irq: remove offline CPUs from old CPU mask when adjusting
> =C2=A0=C2=A0=C2=A0 move_cleanup_count
> =C2=A0 xen/cpu: do not get the CPU map in stop_machine_run()
> =C2=A0 xen/cpu: ensure get_cpu_maps() returns false if CPU operations are
> =C2=A0=C2=A0=C2=A0 underway
> =C2=A0 x86/irq: describe how the interrupt CPU movement works
> =C2=A0 x86/irq: limit interrupt movement done by fixup_irqs()
> =C2=A0 x86/irq: restrict CPU movement in set_desc_affinity()
> =C2=A0 x86/irq: deal with old_cpu_mask for interrupts in movement in
> =C2=A0=C2=A0=C2=A0 fixup_irqs()
> =C2=A0 x86/irq: handle moving interrupts in _assign_irq_vector()
> =C2=A0 x86/irq: forward pending interrupts to new destination in
> fixup_irqs()
>=20
> =C2=A0xen/arch/x86/apic.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 5 +
> =C2=A0xen/arch/x86/include/asm/apic.h |=C2=A0=C2=A0 3 +
> =C2=A0xen/arch/x86/include/asm/irq.h=C2=A0 |=C2=A0 28 +++++-
> =C2=A0xen/arch/x86/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 172 +++++++++++++++++++++++++-----
> --
> =C2=A0xen/common/cpu.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 8 +-
> =C2=A0xen/common/stop_machine.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=
=A0 15 +--
> =C2=A0xen/include/xen/cpu.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 +
> =C2=A0xen/include/xen/rwlock.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
|=C2=A0=C2=A0 2 +
> =C2=A08 files changed, 190 insertions(+), 45 deletions(-)
>=20



From xen-devel-bounces@lists.xenproject.org Wed May 29 09:59:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 09:59:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731770.1137456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCG5o-0007lz-Q1; Wed, 29 May 2024 09:59:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731770.1137456; Wed, 29 May 2024 09:59:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCG5o-0007ls-NN; Wed, 29 May 2024 09:59:36 +0000
Received: by outflank-mailman (input) for mailman id 731770;
 Wed, 29 May 2024 09:59:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCG5m-0007lm-Uw
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 09:59:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCG5g-0003rA-Nr; Wed, 29 May 2024 09:59:28 +0000
Received: from [15.248.2.235] (helo=[10.24.67.23])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCG5f-0008Lt-Gi; Wed, 29 May 2024 09:59:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=b124mvg5qzZDE/KIEcur7k4yY6QifKGxb9xSVpWeEXI=; b=RbvaeSl3B2IAMRvYMtGLwhkxla
	Cvprdxq+3PLWZgAx+UYrrpQSllnNIlQgqPLox2SnMM9u1dVnfvmsVr3XVW2bRns/hOY9ztpT1kLWe
	h0S6v41uw+d17jD/7FCj76pgl/4Ao7IxBrd5wnOBHT0dUj0nwQUKNtuyNQbw4Ay9NuQE=;
Message-ID: <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
Date: Wed, 29 May 2024 10:59:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 29/05/2024 09:36, Jan Beulich wrote:
> On 29.05.2024 09:50, Oleksii K. wrote:
>> On Tue, 2024-05-28 at 09:53 +0100, Julien Grall wrote:
>>>>>> +/**
>>>>>> + * generic_test_bit - Determine whether a bit is set
>>>>>> + * @nr: bit number to test
>>>>>> + * @addr: Address to start counting from
>>>>>> + *
>>>>>> + * This operation is non-atomic and can be reordered.
>>>>>> + * If two examples of this operation race, one can appear to
>>>>>> succeed
>>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>>> a
>>>>>> lock.
>>>>>> + */
>>>>>
>>>>> You got carried away updating comments - there's no raciness for
>>>>> simple test_bit(). As is also expressed by its name not having
>>>>> those
>>>>> double underscores that the others have.
>>>> Then it is true for every function in this header. Based on the
>>>> naming
>>>> the conclusion can be done if it is atomic/npn-atomic and can/can't
>>>> be
>>>> reordered.
>>>
>>> So let me start with that my only request is to keep the existing
>>> comments as you move it. It looks like there were none of test_bit()
>>> before.
>> Just to clarify that I understand correctly.
>>
>> Do we need any comment above functions generic_*()? Based on that they
>> are implemented in generic way they will be always "non-atomic and can
>> be reordered.".
> 
> I indicated before that I think reproducing the same comments __test_and_*
> already have also for generic_* isn't overly useful. If someone insisted
> on them being there as well, I could live with that, though.

Would you be ok if the comment is only on top of the __test_and_* 
version? (So no comments on top of the generic_*)

> 
>> Do you find the following comment useful?
>>
>> " * If two examples of this operation race, one can appear to succeed
>>   * but actually fail.  You must protect multiple accesses with a lock."
>>
>> It seems to me that it can dropped as basically "non-atomic and can be
>> reordered." means that.
> 
> I agree, or else - as indicated before - the wording would need to further
> change. Yet iirc you've added that in response to a comment from Julien,
> so you'll primarily want his input as to the presence of something along
> these lines.

I didn't realise this was an existing comment. I think the suggestion is 
a little bit odd because you could use the atomic version of the helper.

Looking at Linux, the second sentence was dropped. But not the first 
one. I would suggest to do the same. IOW keep:

"
If two examples of this operation race, one can appear to succeed but 
actually fail.
"

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 29 10:06:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 10:06:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731776.1137466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGCH-0001MO-FC; Wed, 29 May 2024 10:06:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731776.1137466; Wed, 29 May 2024 10:06:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGCH-0001MH-Cl; Wed, 29 May 2024 10:06:17 +0000
Received: by outflank-mailman (input) for mailman id 731776;
 Wed, 29 May 2024 10:06:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCGCF-0001M7-OA
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 10:06:15 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1508d9bb-1da3-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 12:06:14 +0200 (CEST)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a626776cc50so200776666b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 03:06:14 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc514e5sm703665466b.103.2024.05.29.03.06.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 03:06:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1508d9bb-1da3-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716977174; x=1717581974; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Szh9A5q61GDC37BYYzt17hODMTfbk8CYAqyaWjpLXH8=;
        b=No9TJwkk3CNljnUdVLPxfxqxriiwjgX0mClzDICsziHrm9drb4gJMyv9za/Ikt1DJ2
         qyZk4u6TyN13P7NL0UsdAOKc/x2qhj5fHOM4uWWqwYjCOYjhykEjFyEq/f3ppsM38B8G
         sLoleD+ATRrkIfsuCdJA3O9ljmD84lDg09alwnQrVNs0SBV10b4MTMkbfRiq/bLW+Pmq
         aDSOLJlEv2mMH1/NEtaO0+/zAMR6R63XF9H3enkYqFJHfGr8Q7hqJ1ZMEisu5W6pJxUT
         CHn2xoywSPdst+V6/ZNZmg/x6D6lFEBNkSHiYH/5udchM/Y+oCdIJAa2q+TlZQn3oUt1
         g+zw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716977174; x=1717581974;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Szh9A5q61GDC37BYYzt17hODMTfbk8CYAqyaWjpLXH8=;
        b=WqJTAsyo/RTXcl8IYgs5vPKno+5j9WIr+FfpwRNIh/ninmnDGtam3vdX1gDZGuKZGh
         YF0VxQ2GvUolfAKLNLeoxqxeVqIcV5dcx2jktEWihJGHD0XCiUsXhBvcVzldBQo7GpTN
         7ZBlmtYKruCbaZ0bjTRs8nYQNbOYdBSbEVoFsQdfsqFKbdyr8sKvlS03/S8FRwU3Jpa1
         EbCKumTRqaq6m1SiSvCRtv1t3nKkdNTqGOn1FGjkoetxcfO8A9zgkf/RkND7KBFJajb7
         rEJdpN4g6VMALdGqn1jTXEwIA93KlM+tJJQyBRc4hRkMZDIfStMHWJ9SSWIONxzZPkuo
         0FQg==
X-Forwarded-Encrypted: i=1; AJvYcCW9hCDxgvYUZhHhYMwHiirOH5GyQkRMC1zeiLweGyyR/jGMMBc5+Btibk1H5p25hGwDz4nEhOJNvk+2351Szm6JoNhVhvD3N+deIxQV7fU=
X-Gm-Message-State: AOJu0Yyq9pNkza9HoOynliOFVh0FjLjj02MEpy4GS72FgCN2cTwFEy81
	U8SWm8+Zf7ljtsf0e//+sVNTMWQjo/XvMk59kl1UUR1xkhQZbl7vI8mHwNeyLA==
X-Google-Smtp-Source: AGHT+IEkmcXk7sMYZ3JnO5rqWwkQ+LuisaiXMRfAFcexLXhIfGwcF2BmdN8KuXlXmk79OamyBTV+Tw==
X-Received: by 2002:a17:906:f353:b0:a55:9dec:355f with SMTP id a640c23a62f3a-a626511604fmr827485966b.70.1716977173615;
        Wed, 29 May 2024 03:06:13 -0700 (PDT)
Message-ID: <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
Date: Wed, 29 May 2024 12:06:12 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
To: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, "Oleksii K." <oleksii.kurochko@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 11:59, Julien Grall wrote:
> Hi,
> 
> On 29/05/2024 09:36, Jan Beulich wrote:
>> On 29.05.2024 09:50, Oleksii K. wrote:
>>> On Tue, 2024-05-28 at 09:53 +0100, Julien Grall wrote:
>>>>>>> +/**
>>>>>>> + * generic_test_bit - Determine whether a bit is set
>>>>>>> + * @nr: bit number to test
>>>>>>> + * @addr: Address to start counting from
>>>>>>> + *
>>>>>>> + * This operation is non-atomic and can be reordered.
>>>>>>> + * If two examples of this operation race, one can appear to
>>>>>>> succeed
>>>>>>> + * but actually fail.  You must protect multiple accesses with
>>>>>>> a
>>>>>>> lock.
>>>>>>> + */
>>>>>>
>>>>>> You got carried away updating comments - there's no raciness for
>>>>>> simple test_bit(). As is also expressed by its name not having
>>>>>> those
>>>>>> double underscores that the others have.
>>>>> Then it is true for every function in this header. Based on the
>>>>> naming
>>>>> the conclusion can be done if it is atomic/npn-atomic and can/can't
>>>>> be
>>>>> reordered.
>>>>
>>>> So let me start with that my only request is to keep the existing
>>>> comments as you move it. It looks like there were none of test_bit()
>>>> before.
>>> Just to clarify that I understand correctly.
>>>
>>> Do we need any comment above functions generic_*()? Based on that they
>>> are implemented in generic way they will be always "non-atomic and can
>>> be reordered.".
>>
>> I indicated before that I think reproducing the same comments __test_and_*
>> already have also for generic_* isn't overly useful. If someone insisted
>> on them being there as well, I could live with that, though.
> 
> Would you be ok if the comment is only on top of the __test_and_* 
> version? (So no comments on top of the generic_*)

That's my preferred variant, actually. The alternative I would also be
okay-ish with is to have the comments also ahead of generic_*.

>>> Do you find the following comment useful?
>>>
>>> " * If two examples of this operation race, one can appear to succeed
>>>   * but actually fail.  You must protect multiple accesses with a lock."
>>>
>>> It seems to me that it can dropped as basically "non-atomic and can be
>>> reordered." means that.
>>
>> I agree, or else - as indicated before - the wording would need to further
>> change. Yet iirc you've added that in response to a comment from Julien,
>> so you'll primarily want his input as to the presence of something along
>> these lines.
> 
> I didn't realise this was an existing comment. I think the suggestion is 
> a little bit odd because you could use the atomic version of the helper.
> 
> Looking at Linux, the second sentence was dropped. But not the first 
> one. I would suggest to do the same. IOW keep:
> 
> "
> If two examples of this operation race, one can appear to succeed but 
> actually fail.
> "

As indicated, I'm okay with that being retained, but only in a form that
actually makes sense. I've explained before (to Oleksii) what I consider
wrong in this way of wording things.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 10:38:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 10:38:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731782.1137477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGgv-0007VF-LG; Wed, 29 May 2024 10:37:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731782.1137477; Wed, 29 May 2024 10:37:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGgv-0007V8-HQ; Wed, 29 May 2024 10:37:57 +0000
Received: by outflank-mailman (input) for mailman id 731782;
 Wed, 29 May 2024 10:37:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCGgv-0007V2-3n
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 10:37:57 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 821e871e-1da7-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 12:37:55 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a6519c6ec7bso55371066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 03:37:55 -0700 (PDT)
Received: from andrewcoop.citrite.net ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6556c5240dsm27935166b.59.2024.05.29.03.37.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 03:37:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 821e871e-1da7-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716979074; x=1717583874; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=viiDxHEW1YNwELQ0IFmBJyKxI1/XuOHmb/mN59Za2XA=;
        b=HOF5y+xkHELEF6A8IMhBD2pNjRMAiPGTvbHnWgmsfgI7qk+a36CEakPEU+LPVq2DFE
         xsSFGzWCvIASx89wjtwsZLeBZSY2mHF2ft7tsiz0J70fqoU37h4+Z3NZCvR8IOMayOax
         oaRFtaoLM+rexXT7qi0wBt0j34d2JsyVcEbx0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716979074; x=1717583874;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=viiDxHEW1YNwELQ0IFmBJyKxI1/XuOHmb/mN59Za2XA=;
        b=q5xHWQzeAThX6KhSRrak3B0x60kGwU9WwwAQPIktHWhuZ1kUBuYF9tEx8IUrahRcwJ
         Npy7J/CqNg/TDNa1n6eUNHhwP2TRGVX+T9aU47pFzMRH7Vi8X3wT0G5DHpqqqs+PXiV1
         IGyyBgRO2GafaP4DltcBkHQ4HDm/hSjGz3fGaTHKQxanKC6OpADUxg020Llc/kl9mwB8
         aa7D+GlqY7UCDxYWucOxidaN2AYparwkjn3asFutmCnij24epFTv0xkVMqjDAEsyMjUN
         UKVadLMHAcS66gZAyfbNr8pblZpIqpFVD0almpLz/xIww/HgMKfrKSwk6rNBFLkJUFt3
         /uFQ==
X-Gm-Message-State: AOJu0Yy/PbyK7UTSpzrEzznHaQIgnNfcpRDn1dTZP+8UVlP9mos8sxYf
	gUfbc/Ph7v0gHsJPKK2/UBIi/HC1wcxWRqclSUKHSnxJlwJrlz+4dbwq+/BAHjBwQUpb5hCz2RO
	aHog=
X-Google-Smtp-Source: AGHT+IGQUTw+UzXBYE2EkuiStKI4dCdxBr0jpT4bP+Fzc7MHiqgxTHRyqcYmFPl934h3emAAqbEMkQ==
X-Received: by 2002:a17:907:cb1b:b0:a5a:1b60:7c9a with SMTP id a640c23a62f3a-a62651565f9mr1284404766b.71.1716979074472;
        Wed, 29 May 2024 03:37:54 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH] Partial revert of "x86/MCE: optional build of AMD/Intel MCE code"
Date: Wed, 29 May 2024 11:37:52 +0100
Message-Id: <20240529103752.1431366-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

{cmci,lmce}_support are written during S3 resume, so cannot live in
__ro_after_init.  Move them back to being __read_mostly, as they were
originally.

Link: https://gitlab.com/xen-project/xen/-/jobs/6966698361
Fixes: 19b6e9f9149f ("x86/MCE: optional build of AMD/Intel MCE code")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>

We're past feature freeze and this was a silent change in a patch, which was
also untested.  A 30s look at mcheck_init() shows clearly that it's not a 30s
job to fix.
---
 xen/arch/x86/cpu/mcheck/mce.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index 1664ca6412ac..32c1b2756b90 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -38,10 +38,10 @@ DEFINE_PER_CPU_READ_MOSTLY(unsigned int, nr_mce_banks);
 unsigned int __read_mostly firstbank;
 unsigned int __read_mostly ppin_msr;
 uint8_t __read_mostly cmci_apic_vector;
-bool __ro_after_init cmci_support;
+bool __read_mostly cmci_support;
 
 /* If mce_force_broadcast == 1, lmce_support will be disabled forcibly. */
-bool __ro_after_init lmce_support;
+bool __read_mostly lmce_support;
 
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, poll_bankmask);
 DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, no_cmci_banks);

base-commit: 0840bc5ea114f536a4bdfb2ca095b79f2069aae6
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 29 10:46:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 10:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731788.1137487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGp4-00010h-E8; Wed, 29 May 2024 10:46:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731788.1137487; Wed, 29 May 2024 10:46:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGp4-00010a-AI; Wed, 29 May 2024 10:46:22 +0000
Received: by outflank-mailman (input) for mailman id 731788;
 Wed, 29 May 2024 10:46:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCGp3-00010Q-5o
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 10:46:21 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id af13c275-1da8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 12:46:20 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57a033c2ec2so709417a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 03:46:20 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579bfc697c6sm5605444a12.70.2024.05.29.03.46.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 03:46:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af13c275-1da8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716979580; x=1717584380; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OrABS7SjAcs27w1VQMkqKccqeT9vVGVOfZ3FokQbsNI=;
        b=b3BBoHnSFDmD6T57/18dC8whkkFnLgxRtgAlMwk3NUj8ic+KNN/LacptkGaRNlFgN0
         H5JmQpLsM9OCORCJhRAPM5JkopUT+OqL9xnrUmbrSc12T3YshteY4m0aIcXT8eCmJs6I
         Xsu1bAwsDK++TVEIX9wKu6v8Zks1gLDip+bJjlP5LWhF/F/XdrcQIrJfRqQBPZQTwUlx
         5K8Fki6X/aG45qNlXdaQBgbzcPLgAmcIs6x6vZRovw1UM73qPW8RQC8gGxHCnbgHbJ+1
         wOP2jHiMgqStyfLyLryK2yISGLz38IGIwKEb7A1Z+yHjx9pkSGeWehNrMb2jHqzd6vvU
         6eUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716979580; x=1717584380;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OrABS7SjAcs27w1VQMkqKccqeT9vVGVOfZ3FokQbsNI=;
        b=cNJe4OcUfO/gcDgcjLuN0BCb9H8t2zRK8hJpiHBUpbwEIxoZ4FdY/a/83thtlh2mxZ
         pjIAJg2Zg6eT6nxK/zC8QCot6gRO0OaslGd51jw24PmwsnIcsSZ4YofpHh7wQuhXEzTG
         QwopfeCvgVT8jf73zvJ593ex70YuU53Vp1KUewa0wDxkvy0iGO9eLdEDukipqpd2AmwL
         u6baNwySpw8qZywb0WUBYNMmIjjBVYpUws9FCyjJmoP60oiMzo7+0TJRsBrMXuF70Su8
         Zz1akX0JGuU5z98KO57xQfIbt715eqqAmhhxElU6932ztqofo98kVYUclUsip+OE5au4
         GPfg==
X-Gm-Message-State: AOJu0Yz2YbjMt4AMJr1cnZ9QZrxM1F1yPwB/+XA7OdZ4w5ItpbblnyhD
	eKmnhs82Y1jx2Q0K7ZuPox7QVNJs0HdwdTqZVbDGdC9n+VXzzZ5rvLEUyb16DDw4fGpUbiHbVlc
	=
X-Google-Smtp-Source: AGHT+IEmHKE6bnkaXLkG2f7doWQitoSbKyfK4Ff031bt/HTsQgez4z0TLxTgfC7BQFCcXfvXo7vlnw==
X-Received: by 2002:a50:cc9b:0:b0:572:d1e1:b4bf with SMTP id 4fb4d7f45d1cf-57851970370mr9171889a12.9.1716979579664;
        Wed, 29 May 2024 03:46:19 -0700 (PDT)
Message-ID: <740c940c-f651-4d7d-954f-78b7d9a31a90@suse.com>
Date: Wed, 29 May 2024 12:46:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/MCE: don't treat S3 resume the same as boot
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Checks for running on CPU0 are bogus; they identify S3 resume the same
as being the BSP while booting. Replace with the same check we use
elsewhere to properly limit things to just the BSP.

Link: https://gitlab.com/xen-project/xen/-/jobs/6966698361
Fixes: 19b6e9f9149f ("x86/MCE: optional build of AMD/Intel MCE code")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -767,7 +767,7 @@ static void intel_init_mca(struct cpuinf
         lmce = intel_enable_lmce();
 
 #define CAP(enabled, name) ((enabled) ? ", " name : "")
-    if ( smp_processor_id() == 0 )
+    if ( c == &boot_cpu_data )
     {
         dprintk(XENLOG_INFO,
                 "MCA Capability: firstbank %d, extended MCE MSR %d%s%s%s%s\n",


From xen-devel-bounces@lists.xenproject.org Wed May 29 10:57:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 10:57:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731794.1137497 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGzU-00035s-Dg; Wed, 29 May 2024 10:57:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731794.1137497; Wed, 29 May 2024 10:57:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCGzU-00035l-Al; Wed, 29 May 2024 10:57:08 +0000
Received: by outflank-mailman (input) for mailman id 731794;
 Wed, 29 May 2024 10:57:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCGzS-00035f-NS
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 10:57:06 +0000
Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com
 [2607:f8b0:4864:20::729])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2f8aee0f-1daa-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 12:57:05 +0200 (CEST)
Received: by mail-qk1-x729.google.com with SMTP id
 af79cd13be357-794ab01e747so127949185a.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 03:57:05 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd368cesm459569185a.119.2024.05.29.03.57.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 03:57:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f8aee0f-1daa-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716980224; x=1717585024; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Bh6Kw7VARAxBwC+mWC9bOvabqHB+rw1no6MVwRvJ5Es=;
        b=RKVLjjfMN4uiJo5aPKnFOqh0DjeXp6NXETFxATe0EAcUSrW6H37mc+e+wfpktenI59
         as8HJ/DbiLJRg0+a4AGeBMYjvwjra6fd+iiz4zQZmD7RTwElNKSLCuK3s6AYOiuCpeSH
         f0B2f5k2tJTUEpfEfvB/kTYw4nKq/1aLAgqhU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716980224; x=1717585024;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Bh6Kw7VARAxBwC+mWC9bOvabqHB+rw1no6MVwRvJ5Es=;
        b=aIYDmY75IyeNHB3YGGSAmP8X+Z1Mg74rLsyyaFmy/fRzNOK515Q1w3NCJgDchEm+TA
         rw3vunqqIurR4eBb9RF1Rjk3jyrr6jCjeE7IyjwdWoniWSVxY57b6pm1O/o+hJ6eWr+d
         3iW/ZIaLjloC4KvGN2kmWkbQ1idkxJXIQWQZWMf8+bT7ojLJEdd9p7wGQQPqsDf62ObX
         p1wfAg4p7QgUxrI+3hPuFKn5UDkkSKV2uxOdmBikVDEp2wOpG7WnJmB8+Ui9aozfMrpi
         AUY1dTGC+JXlf9eh68j8LyHkDXqs5Aoxz8x5h+0SsFvbaNQuk4jSjmSEVMJ4WF/8JDAq
         Io+g==
X-Forwarded-Encrypted: i=1; AJvYcCXZCkQcoZycV/LpgSZgz5TaSqCVI7ZW6ipcSOR49WGLNGb6JlXiP5yFcqpax2ymmMqKFGPs50hcwyZWLG1ncaSZ29oLcjvmxLHoyd3rZTg=
X-Gm-Message-State: AOJu0Yy/peAEhVtAwOljAXG8jV33PuiBh4+imwj6TKcvRO7aa7Cs0cNV
	tjZenz7EvanwXoTCIHd+0vP7V86xtBIhSrtdk76ctq9oC+5jS338h6fyqY+b++s=
X-Google-Smtp-Source: AGHT+IGlFmVxmnWzy374jrSgalYA08W5ivfkJVzZUGOjP+Jyglc74tpsFF3diDYKu1Co0pr+DsyrQw==
X-Received: by 2002:a05:620a:258d:b0:793:18be:39d1 with SMTP id af79cd13be357-794ab08a5ebmr1750921085a.32.1716980224576;
        Wed, 29 May 2024 03:57:04 -0700 (PDT)
Message-ID: <ae0c1428-1894-4362-8ba2-b78670553cc7@citrix.com>
Date: Wed, 29 May 2024 11:57:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/MCE: don't treat S3 resume the same as boot
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <740c940c-f651-4d7d-954f-78b7d9a31a90@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <740c940c-f651-4d7d-954f-78b7d9a31a90@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/05/2024 11:46 am, Jan Beulich wrote:
> Checks for running on CPU0 are bogus; they identify S3 resume the same
> as being the BSP while booting. Replace with the same check we use
> elsewhere to properly limit things to just the BSP.
>
> Link: https://gitlab.com/xen-project/xen/-/jobs/6966698361
> Fixes: 19b6e9f9149f ("x86/MCE: optional build of AMD/Intel MCE code")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>
> --- a/xen/arch/x86/cpu/mcheck/mce_intel.c
> +++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
> @@ -767,7 +767,7 @@ static void intel_init_mca(struct cpuinf
>          lmce = intel_enable_lmce();
>  
>  #define CAP(enabled, name) ((enabled) ? ", " name : "")
> -    if ( smp_processor_id() == 0 )
> +    if ( c == &boot_cpu_data )

Look at the callers of this function.

This doesn't help in the slightest, because mcheck_init() is
extra-specially wonky.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed May 29 10:58:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 10:58:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731797.1137506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCH0m-0003bt-NT; Wed, 29 May 2024 10:58:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731797.1137506; Wed, 29 May 2024 10:58:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCH0m-0003bm-Kg; Wed, 29 May 2024 10:58:28 +0000
Received: by outflank-mailman (input) for mailman id 731797;
 Wed, 29 May 2024 10:58:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XI+4=NA=epam.com=prvs=2879370e9d=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1sCH0l-0003bg-Fh
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 10:58:27 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5e9590a8-1daa-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 12:58:24 +0200 (CEST)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44TAV1vB008526;
 Wed, 29 May 2024 10:58:13 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3ydatgwrg4-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 May 2024 10:58:12 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by AM7PR03MB6546.eurprd03.prod.outlook.com (2603:10a6:20b:1bf::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.19; Wed, 29 May
 2024 10:58:08 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Wed, 29 May 2024
 10:58:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e9590a8-1daa-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZkD5/7oeVDZ9rHxeW6bDTYfDrWv+Jum0kzvmRX+ujoyAUj/rgA8HseLH6dwo1hqp6hjhwZIP7o14rNSjRzqja+7cuvE+RrkUrCsYRWPmYK4fD8X2YxYcrCnoWoAt0oIHSeE19DkdklUrs1orIp8dvwyK+E13vjEvWH4X2gUWFdc8ezn8sWfq36Y713Vo8KQM5o9kO7XcNPYOYWL1lQtGuPbrVemCl3cHOkcWC4bzxJoUWjN04GxmrXVyAvv98lbvyoVy0mwZoIu5Z5Plw5zvqr7kJtQ4Cz5kXABQ3WhoM20R9MvIg+ig8B2VBrA4jRTDNXBEQkfLFEJdtoYJc71v0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=X9ZdFziTF2PFL+iFVJdyZyFKsVyqu0l3iLUiWDYqcVE=;
 b=TrPbaWvigwsta+TbVRokoJbsfY4WEVbjQu+EeYu/Wqbgs1eHzsHgFVa6PoiIFDckwBquxQkcdzvRFk80ZJW1Ku4J0ClGUM6r2tGlcbhmFofirLRnKVylznTDrwjqcpSJCxhi90jdGkCJaIm1Dx58zxDCDtKUJPoKkgyPETqGjL/IAahv9n5o79n1bv02sUvfp7noYwsMku7HALCJHT0mkJNjzXAI+m89YoYpDwFSmABuX4yt9bbdoSbnCa1FXvKSYtF2e0pTiGHnuEOngDrooRfJZVMcONFtMB+o4hkDL8d+iLQod8VXkJGC32ldO5rHR3ryJfzgTWHrfbgQk4T1+w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=X9ZdFziTF2PFL+iFVJdyZyFKsVyqu0l3iLUiWDYqcVE=;
 b=K4dwciVP30nCkdUUFFVFINT27SIx+6MUowaAp3tR1UwS91bef77yTep30Coj9+pwvH4abhdH5deQ/4+IotzPBTZtOA1PxPjY0jZcuoXO07FAI0VvNjWr2NNj3TpTTf5V2to9GSUV5SFN1AMjWDFsDRRYl8niaDmRt4N3bmap4RIWEqft3yqhI9uWtNeJWf9KgzQNDsSp1MI9AHfQV57A9kAM+AWPph2CandYyDX0LnHdbczd3g82IcMs0hU2Mbq95nwccQsO16/dEaulb3Ae5lgoqHkyQMdbZIZkeew6TztO2cG3kFGa3WyibGwjeRxskJWicJiWYwQXih9hyc6XFA==
Message-ID: <8d9fd413-fa35-429d-bc4b-8261671b9c6a@epam.com>
Date: Wed, 29 May 2024 13:58:06 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 12/15] x86/vmx: guard access to cpu_has_vmx_* in
 common code
To: Jan Beulich <jbeulich@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
        Paul Durrant <paul@xen.org>,
        Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
        Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151748080.2544314@ubuntu-linux-20-04-desktop>
 <4f2c1486-1b58-4c8d-a74e-ad83bf9cc4c7@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <4f2c1486-1b58-4c8d-a74e-ad83bf9cc4c7@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA2P291CA0025.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:1f::6) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|AM7PR03MB6546:EE_
X-MS-Office365-Filtering-Correlation-Id: 7279ca61-85ce-4636-f2da-08dc7fce398c
X-LD-Processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|366007;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?K1JKcmM1cWhMR2lNSyt0OG82VTZTeXRJY2NwVEM2TEpjTUFpMmZpQm8vWWtU?=
 =?utf-8?B?QklERk9KRHdNRWx0cWhGSDRwR3AvaVB6Z2QxYTlsZTBPVzE2WlB1UG1jK3dP?=
 =?utf-8?B?amlJOUh4UHBHWUVOZWJzWjdMTnhwOVNPS0xtTk9BQU1SUVlKNXVLaGpHVFFy?=
 =?utf-8?B?UmduYTBXUXVaakZ3by93eFM5T3hmV3lsZThqZG05Y0RhbGExTlFGMUZuc2NQ?=
 =?utf-8?B?U0l0QXY0eE1RMW9vL1JCdWZKTjRiajNYZ0N4UytZR29wQXhFR3oreElGayt1?=
 =?utf-8?B?VXZram14R2dQc2NCakJFaUpnNWpYMXNOd3dEUVBtQ1ZUVHZDeUVneWZsemxI?=
 =?utf-8?B?Mk9qOGNKMVZRb3ZNUzBEZ2Z2YzZPRFFDQVF0Vk1yNTdCT3I1d1ZyeExIL0du?=
 =?utf-8?B?OUNDeHlCcC9XaTBoQ1dIdGpXUzJOWEk4SW9PYWcwdUlNTEtOVm5naEF3ajk1?=
 =?utf-8?B?dVBmUklkRzd3TlV1Q0RwMTg3ek04RGN5VzdmNlpPRjc0U2pEK1FabE50WjVa?=
 =?utf-8?B?ZHlFNURqZEtWQzIxOU9kV3I0eWpLVElMbkJSbEUvczRqY3RyTERWbWZNQUFx?=
 =?utf-8?B?U1BWRWY0ZWNndUt3ckw0cWQyOTFKZ1BLUzRUS2R1QkpqYnozOXhaWFU1NzJS?=
 =?utf-8?B?RkRzNWxLZ2tPbjhWa09mVGZYamhmT1UvK2JnRjl2Q1lzMkVCZmh6R2pjamVU?=
 =?utf-8?B?bTJqaWNodmVXTGJnMmlJMHZGZm9RZlFOcGFmanJBdFA5RXpSaml6THF4cGM3?=
 =?utf-8?B?R1ZFWmNYWHJUWjNMbEl1a0xZQjN2dXpheDVUY0NBdW9hZlpsbTlsUjk0OWpx?=
 =?utf-8?B?L2lNUCtTdW1VdVRQTkd4WE00MHBhVURtL0xaTzJWMHRDSTl6SW1FQ0lQWFVr?=
 =?utf-8?B?WFJQK05CZ0NkYlJyWDRCaFNSODNjRXZxNlNwL1E0cjdrcldnUU8yWUhPa29k?=
 =?utf-8?B?TTJoRmI1bFdYeDhzOStlRU96VU15eVlhL2hQQXpQeloyMkl5ZXdMTElDMksr?=
 =?utf-8?B?TTVXMVk3dWNSMlJVK0FBNzVpb2hBc2J3Z21iaThRb3lSZy9lbWIyeVI4YTZv?=
 =?utf-8?B?NzVxZnNTQ01vblZKc0FTTkQ1UHNFaXdrbHlFSUNCeDNaUVYrejhRalpHdS9z?=
 =?utf-8?B?eGw4L1BlQnpBbW10WlNOV0FNczFTcXd3bGRhK1dTSWIxNzB2NVp5am1OSElO?=
 =?utf-8?B?K0dOOU1DR0ZNYXlPaTBYTHRhb3UxanZGZzB1cHI5a2xRV01PVS9peUc1RDVM?=
 =?utf-8?B?Y1BOSW9UK1NRWW0rdUVLT0FiTUNrWHlmVzUrejV5K1lwNGgwaG90K2pySHV5?=
 =?utf-8?B?dTBUcGhEYlJRZlpGNmFwWTN0RktycXBYMTlzMjg5M05pT0FQUnp4c014Z2t2?=
 =?utf-8?B?UEZOTjRXOUZRMnFLbkRYQ0ZRaFpXSlN6bzE2MnNQUGpaNTBxREpJYjlZSzI4?=
 =?utf-8?B?QjgwdjdiT0V1Y1laV3dDZGgraWFhUWdtc3Bvem54QjZnOWJyV29leFlqMTlV?=
 =?utf-8?B?NWpqWU9QYlNxaFhjY203T0paTXlNZndkS0xRbnJwWWI3d0FMbXlsb1dxUXZV?=
 =?utf-8?B?ay9kRjd3Mk4wR3ViSlpEdjdjSDhCQmEvYWkxZEJoUXYxc05HNSs4bVFtQnph?=
 =?utf-8?B?a2szcVY5WGtYaXh0M085ODY0MU13RzZUYzdFM0xNU1JXbytQeDlQMUgyV2M3?=
 =?utf-8?B?RUNHVUI2cXcycUVoMW4vM0RacWdVakhUQU14TXovdkpQQUd1N201clBBPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?QnVmUXIyMUZDRUhYdG80b3pUSHlFc3NQc1c3aUFwVzJpMElKMHRnajl1Nzdp?=
 =?utf-8?B?K0p3ODRvb3NINExiSmkvdHFPSXpOdFM3eUVvOEJ5UFhoNWhYWHB1U21pWVVl?=
 =?utf-8?B?cGdwbHcwUHpTOUZ1ejRKb0sxckdiOW10dTY2aTVyWmpLTDI3U3lxMnhkNHNC?=
 =?utf-8?B?TnY2cnFHdnNMS29CNDZpMk1OZ2lvSDI4eVpaZ0lhbHljUDFQa3FNK1lrWS9Z?=
 =?utf-8?B?SVp1NElBdWhkVjV5SXl2eW1DcTU4OURSUG02dDdidGc1eWNUMHdJWlhTN0Ru?=
 =?utf-8?B?L3RicVV5dEVtdkd0eE0zQXFiN0JRanJrNVhuZlo4QTQvd0hQTnMybzh1dXB0?=
 =?utf-8?B?TTJpdmxmbjlpYXF1d3dOOGYxZWw2KzE1cG9CTmZhQ1ZoeTVZNGtsalVwUGs0?=
 =?utf-8?B?SkxEVXF2bXE2UldqS2x2bUIxTFhvblU1K0VWYldLWmE5V0JXTEpnU3REeWM3?=
 =?utf-8?B?SmZwWVpVQ056NGlOUVV4MHpBZjVHRFRFSC9PQnJkM2U5SUxYa0E5Y0d1MWVy?=
 =?utf-8?B?V1VmRFhrY0NOUks0VURka2wzMHZSTzY0QmFZMzJUY3dpa1FXdWdBVXJnMTBU?=
 =?utf-8?B?WnNPa2wrZENxS1k5cTlUQk8zd0RwZmE0dU9FSk1aeEp6MXlNS2UwZXNCa3d6?=
 =?utf-8?B?bjFhRWxmSEdvSll1VFFUSWFmcG9Qc2ZCRy9qUTllaWtDVyt1KzNhKzlFb0l4?=
 =?utf-8?B?d2JHc0NyK1JOYTNxVzIzMm9HRDBTS1lQRFZtVjBnVGZhWGE4YWVxcHNZRTV0?=
 =?utf-8?B?MU0wR2loZmVmUGt1YWhWT0pHVXphK1FGWE5yYWR2V0FrdFhtY1lCTWpiUXRj?=
 =?utf-8?B?TU9Sd2JObHk4MkZBbzM1UVR0UlpKalhaVkp3ZkkvNjF4bE9sWTZSRE9vNHRF?=
 =?utf-8?B?dW5IalBETExKWklTRXFReXh5TlVweXVCNkE4TFYrYnIxZ1lCOXNQY0kwRkhj?=
 =?utf-8?B?d2RqR1l3Q0x0T1BWdnhGSkpPZGNXaGxQTUVTYmJNMkFCMDhDbjU5cWlUSy9p?=
 =?utf-8?B?cXlLOVMrdEVQOHFzZTBDY09ia1dmbkcram1PVWpUdW8yV2dXTmRKMjBuaFl6?=
 =?utf-8?B?dENvZTBlVkNJR1dRM3ZxR0haZ1daSGp1Z2ZTQ0wrWmh4enZIVWpTc0Fab0tw?=
 =?utf-8?B?Wmd5R3gwRUFEajQ1M0JoVis1bkJuSkhqeDVReFhOQWtnNGdTamtsU3dGN2ph?=
 =?utf-8?B?M0NaZW5SK1Q4VVhIVElTT1FXWVdDSlJNRVB5VUpET2VXLzB3NWFCSEFBY0VW?=
 =?utf-8?B?eWh2Z0pqbzFyTHBJNWFld3BJc2FiZ3lOdVpTY2cyczVYL2dwVHBwNjRjUHRp?=
 =?utf-8?B?Z2RNajU0V1RISkN3Z01TNkN0RExrMm9JUHNSeGRxemRIUm9jWWFpNlVkcVhx?=
 =?utf-8?B?MkpVekhVTFhRZmk1anBtbDZxcXVmTzIxY3lPZjI4QWZIbFk5d3VoTDJSb1lv?=
 =?utf-8?B?YWF5THovbFlxVG1LcnkySndkVkRzRHJvOTVKQ3BVQlJUMUlUNHNieVJQeE9a?=
 =?utf-8?B?c2NLRUcxa211Rzd5QTkyWWVMSEptb09HU0ZSVVhEYkhSN0gyTnFYUDJUeUFj?=
 =?utf-8?B?RXlvWngxcVlzbGVLcVlTWGVDYXBJY1FIMmozZlBiMmFKREpUNzBFK2hnOFlw?=
 =?utf-8?B?Z3VjVVBVR2ZMVVVvTnJ0K2FNZzBqVWphS2swdW14OTdWVGxnQ0ljRnV6Q3Ir?=
 =?utf-8?B?YnBrcHFXN1ZOMGZBRVBuZGVSbTd0OVRmU2VZd2RxRWNoY3E5MVpaUkU4bFJB?=
 =?utf-8?B?TjViNkZwSWpwcWpPcURzOTBFK05RVzlEYkVTMTI3eWRyaVZkeklkT3JITHVO?=
 =?utf-8?B?aS82a2xOMGpIK0RuV0dOdjlUU2hEcWpMQVgyOEFVU05wUVpVa2FoSnVIdkJw?=
 =?utf-8?B?eThMTGRsRHM5SVczaDZlTFk5anJ2U1d4cHZuT05VczgydVFqV2RtR1FEMGd6?=
 =?utf-8?B?eUVtcXFuMmRPMWZ3UGlFb0VLcEJsUkl1TDJZUDFjSEk3T2toMkVFSHdteWd2?=
 =?utf-8?B?Q251VVE1Y1ZDNjdTS1FFWCttQlkzempCV1UwRUxqeUU4dis0OERzUTNQeThP?=
 =?utf-8?B?ZmxBYk5EYlpaNnA1Z0NscE1Pbm1QZUxlQ1RJeXRybWdDcnFaanNUcUhSNHFQ?=
 =?utf-8?B?ZVFrUlRzNCt0eExZaDBES0dPZEFiUGpub2wyem54MTRhc0JyTGhJTzZWY08z?=
 =?utf-8?B?Wmc9PQ==?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7279ca61-85ce-4636-f2da-08dc7fce398c
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 May 2024 10:58:08.6432
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 628Rkm0rCXSfQhhP0PzdTrPy+2SPNnsAHtI8S/ov2vzYTE+zZ77eEAVUGI3DdyAdOhmJ+JxX3NIM/Txw33ysGw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR03MB6546
X-Proofpoint-ORIG-GUID: HNoUSf3S63o46jq3_dqkQ71qifvvP9G5
X-Proofpoint-GUID: HNoUSf3S63o46jq3_dqkQ71qifvvP9G5
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-29_07,2024-05-28_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 mlxlogscore=999 suspectscore=0 phishscore=0 clxscore=1011 mlxscore=0
 bulkscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0
 impostorscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.19.0-2405170001 definitions=main-2405290073

16.05.24 10:32, Jan Beulich:
> On 16.05.2024 02:50, Stefano Stabellini wrote:
>> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>>> There're several places in common code, outside of arch/x86/hvm/vmx,
>>> where cpu_has_vmx_* get accessed without checking if VMX present first.
>>> We may want to guard these macros, as they read global variables defined
>>> inside vmx-specific files -- so VMX can be made optional later on.
>>>
>>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
>>> CC: Andrew Cooper <andrew.cooper3@citrix.com>
>>> CC: Jan Beulich <jbeulich@suse.com>
>>> ---
>>> Here I've tried a different approach from prev.patches [1,2] -- instead of
>>> modifying whole set of cpu_has_{svm/vmx}_* macros, we can:
>>>   1) do not touch SVM part at all, because just as Andrew pointed out they're
>>> used inside arch/x86/hvm/svm only.
>>>   2) track several places in common code where cpu_has_vmx_* features are
>>> checked out and guard them using cpu_has_vmx condition
>>>   3) two of cpu_has_vmx_* macros being used in common code are checked in a bit
>>> more tricky way, so instead of making complex conditionals even more complicated,
>>> we can instead integrate cpu_has_vmx condition inside these two macros.
>>>
>>> This patch aims to replace [1,2] from v1 series by doing steps above.
>>>
[..]
>>
>> I am missing some of the previous discussions but why can't we just fix
>> all of the cpu_has_vmx_* #defines in vmcs.h to also check for
>> cpu_has_vmx?
>>
>> That seems easier and simpler than to add add-hoc checks at the invocations?
> 
> I'd like to take the question on step further: Following 0b5f149338e3
> ("x86/HVM: hide SVM/VMX when their enabling is prohibited by firmware"),
> is this change needed at all? IOW is there a path left where cpu_has_vmx
> may be false, by any cpu_has_vmx_* may still yield true?
> 

This change is about exec control variables (vmx_secondary_exec_control, 
vmx_pin_based_exec_control etc) not been built, because they're in vmx 
code, but accessed in common code. The description is probably unclear 
about that.
Also build issues related to VMX can be solved differently, without 
touching cpu_has_vmx_* macros and related logic at all.
I can move exec control variables from vmcs.c to common hvm.c, this 
would be simpler change and directly related to problem that I'm having.

   -Sergiy


From xen-devel-bounces@lists.xenproject.org Wed May 29 11:09:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 11:09:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731806.1137517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHAv-0005lU-L5; Wed, 29 May 2024 11:08:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731806.1137517; Wed, 29 May 2024 11:08:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHAv-0005lN-Hk; Wed, 29 May 2024 11:08:57 +0000
Received: by outflank-mailman (input) for mailman id 731806;
 Wed, 29 May 2024 11:08:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCHAu-0005lH-Jz
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 11:08:56 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d6da2eb8-1dab-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 13:08:55 +0200 (CEST)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a626777f74eso228889366b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 04:08:55 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6357bf05f6sm115536666b.206.2024.05.29.04.08.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 04:08:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6da2eb8-1dab-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716980935; x=1717585735; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=CwnYHq/a9hUpOD704HVTgol18BEIUqDzbT4ME56HY+4=;
        b=R2Dwtfz0FmRl37JLi8vP/PVVzsiVm2TqxfXNSs+qqOz67/5e4KvclThFrY1CNUdVdn
         ciim+yzMrvWPLGtH2kkH10aeRNRQBghFyKbv/yIFla1q4P7lE/vNwaYq3Ma7JWat2Q31
         VixUXwZ5LVwnyuPM7bKr33ffKwIgwKx6RSVWB0O5VopbL2OYgeAjsxXeZHOCUc65vUmf
         XKk+nLPiuo0jMbytZXhZslGV8Hklq9sANiOScQdQ1w6W9dy2bAtxUyKsH4UHJgFk+aql
         ibY6jnqmTJ1HpAgSKtjjYWePuFbBsaoL8BQ3IwyStIqKG26HvS6whgYoXIvjaQX7EJsh
         08Mg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716980935; x=1717585735;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CwnYHq/a9hUpOD704HVTgol18BEIUqDzbT4ME56HY+4=;
        b=JY4K5dIsmdUSwo8dr/9SQzhrEE5jurvxbIoaiuWznBbAzWxsDHkNxpejt/H4/GY/6t
         q6U98xSpUYpqVUC1s+kMCbPm01LlzcS435WcMu4q/yOKFwIWkXH39WTRBKfDJsGGYF56
         tX/6evOr6msXbnIU4XZetzDize65C3MTXOG2fJ6IPOScTRB9AhkYQ4ZWP91UFBJlaZ0u
         9o6RoygzERXHbDu/ko/h34ibWLqzBoKYesJh0Ck98129NEQ+ugUzP2njoC12ywBScK9k
         qeIOlWe9zHb+UOxgRm8erGrkKt/YnJsM5UxaqHYJMRJnUvMd9v37i37SA2BqNf3e0FTN
         Aiww==
X-Forwarded-Encrypted: i=1; AJvYcCV9+uGFBb1kOJYraumrecXyh27LeAVeVEn45/7HB66qzTfavUdvUlLjJmOEbZvt2B19USxIbWCSH9AUpWlJAgRe46WFb85zrldOihEdBA0=
X-Gm-Message-State: AOJu0YxYlZ0yBn35k8fSxLPWKKTMQcQpCZHdWxDfKQjGHvibyV2xFtB6
	PFKbFpQE92JLURziRfPiF4mNi978G5I5QwDFEkMC5MHqBcDUyBHP
X-Google-Smtp-Source: AGHT+IEsA1eA3bkTDlQFLjpY0AZGbLds3lg1hTIj+3Fcw1pHqZNlwXC0bnQC3q/ADFvnRFdjmmtC9w==
X-Received: by 2002:a17:906:1547:b0:a65:c143:b3d3 with SMTP id a640c23a62f3a-a65c143caa4mr25062166b.3.1716980934654;
        Wed, 29 May 2024 04:08:54 -0700 (PDT)
Message-ID: <1dde31ff838637bbe414ccf5860f21bfdd8450cd.camel@gmail.com>
Subject: Re: Updated Xen 4.19 schedule
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Wed, 29 May 2024 13:08:53 +0200
In-Reply-To: <495de773-bbd6-4a05-b6a4-e00cda301a96@suse.com>
References: <e9514fa9fc0a0ea9eae2f0c4accf9b3060e95f9e.camel@gmail.com>
	 <495de773-bbd6-4a05-b6a4-e00cda301a96@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 10:19 +0200, Jan Beulich wrote:
> On 29.05.2024 10:03, Oleksii K. wrote:
> > Hello everyone,
> >=20
> > I would like to announce that I have decided to update the Xen 4.19
> > schedule due to the extended feature freeze period and the upcoming
> > Xen
> > Summit next week.
> >=20
> > I propose the following updates:
> > =C2=A0=C2=A0 Code Freeze: from May 31 to June 7
>=20
> This is almost fully the week of the summit. With ...
>=20
> > =C2=A0=C2=A0 Hard Code Freeze: from June 21 to June 28
>=20
> ... this, did you perhaps mean May 31 to June 20 there?
>=20
> > =C2=A0=C2=A0 Final commits: from July 5 to July 12
>=20
> Somewhat similarly, what's between June 28 and July 5?
It should be RCx ( IIUC how it was done previously ) +  bugfixes for
serious bugs (including regressions), and probably low-risk fixes only

Let me share the updated schedule as I made incorrect calculations:

Code Freeze: Fri, June 14 (+2 weeks from Feature Freeze, considering
the week of the summit)

Hard Freeze: Fri, June 28 (+2 weeks from Code Freeze, usually it is +3
weeks, but I think we can go with +2 and extend ( to 3 weeks ) if
necessary)

Final Commits: Fri, July 12 (+2 weeks from Hard Freeze)
Release: Wed, July 17

~ Oleksii
>=20
> Jan
>=20
> > =C2=A0=C2=A0 Release: July 17
> > The release date is shifted, but it still remains in July, which
> > seems
> > acceptable to me.
> >=20
> > One more thing:
> > No release ack is needed before Rc1. Please commit bug fixes at
> > will.
> >=20
> > Have a nice day.
> >=20
> > Best regards,
> > Oleksii
>=20



From xen-devel-bounces@lists.xenproject.org Wed May 29 11:10:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 11:10:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731810.1137528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHCs-0007cV-1v; Wed, 29 May 2024 11:10:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731810.1137528; Wed, 29 May 2024 11:10:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHCr-0007cO-T2; Wed, 29 May 2024 11:10:57 +0000
Received: by outflank-mailman (input) for mailman id 731810;
 Wed, 29 May 2024 11:10:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCHCr-0007cG-7q
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 11:10:57 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1edde8be-1dac-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 13:10:56 +0200 (CEST)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-5785e443cf4so1950915a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 04:10:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1edde8be-1dac-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716981056; x=1717585856; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=gpZKrKra7Iqk+QjOMF+QAPCTFC0C0WSuN63kFGROMUo=;
        b=fsnbH7c0aVyebpPOTaEbBFJuuzT/vjNfdz8uFheBAYK59ghx4Swvo1ZK+TYKj6w+6b
         3VXcFyvEPhh4q+IEHMeEaTdsOaxuuhf9WqJp8zqC2ZNeh85Hs0swCKOf0dOXhiOiX4lD
         5Lz9R8Lb9h2NK/T9tFgOoZj48474Gs7dXnmbWff1BEAhiDv3Q0cpUyMFTiIsyqaFE3a3
         6nGDOK2nCYih8qkMJNS1rC3pO+2F4QvtrWAXvz1KkVv+2hpvPSeiVm5iHjTST1bTM+Sy
         NpQ8sU6x150iQKmli3qTND9XmJbQGIxw2H/ewVKCJbMMLS1nUAe85IM/RBB+GS1nxF5K
         4q/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716981056; x=1717585856;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gpZKrKra7Iqk+QjOMF+QAPCTFC0C0WSuN63kFGROMUo=;
        b=hJ9RRZRVYBVc3A+1zl2UDNQg2y5TFPDlD5y3GQmprhyPrIqY46tFlNhkYDQxnp3Jf/
         +e22oPekVHfgvX4zkDv+YX+yE+wr8+rikPIJqKoMaIpmYp07TpzPfHHXl2acTGrP7IQV
         0a8Laz2D7NEuBTT1mGr8Q+VsSSwNNstVy8Hcvmu2sRFnr7t9+AkxibVEKs2NsqIl/vlA
         HvPmOnDT2oJrbc+NtfvG78uLvhonThiG/LTuf9jwR6oDIz9uNO+HFPEm2n5gN+qxkXEp
         CKJHgVcWOXIzD/4rXs3CQyOE31BpZlN7SdNZMR7pKN4ioLcSZf13r8cANie7SFGDUZqr
         pUTA==
X-Forwarded-Encrypted: i=1; AJvYcCWfFMFzaGYkHrpjqKUAC+I87L1P/6o9CryXNyaTvmY5BWUxFPDdyg7+YnSN9g3YRwlfxGzRBkAH3LUwgX5m+63X15qm4g9p92jz9GbCekI=
X-Gm-Message-State: AOJu0YwDal5swkvjJxiYTlLewdftcjvWlbWsFOYi028iK1O693POAFMl
	qxWquDPd7y1WQPZghkYMO4KwPDEvN8m4WMPHDfBVbdQ+ZnQpkFANP/elCI1Px1hyBB4Xw8tvZk4
	JRVNpv4GNSB97oDX6dxzgO8D8F8G6Hlwhdv0=
X-Google-Smtp-Source: AGHT+IGGjT/BPndtKnYGowXLebktl6eEnPT7PGwOc/WAfZDkZI+bPyxhUgP2QKo0/TNvru3Auo0+hOEzpXMHhwD4LJM=
X-Received: by 2002:a50:8a9c:0:b0:579:fb7e:210d with SMTP id
 4fb4d7f45d1cf-579fb7e2991mr2748072a12.1.1716981055398; Wed, 29 May 2024
 04:10:55 -0700 (PDT)
MIME-Version: 1.0
References: <20240524105152.1301842-1-edgar.iglesias@gmail.com>
 <20240524105152.1301842-4-edgar.iglesias@gmail.com> <4025f82f-242c-4c3e-952e-8ecf7a66838f@linaro.org>
In-Reply-To: <4025f82f-242c-4c3e-952e-8ecf7a66838f@linaro.org>
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
Date: Wed, 29 May 2024 13:10:43 +0200
Message-ID: <CAJy5ezpgANR4VuewnXs=MJwAfipxNq4+1r3LwY-f8-7_R6mwRQ@mail.gmail.com>
Subject: Re: [PATCH v7 3/8] xen: Add xen_mr_is_memory()
To: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>
Cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>, David Hildenbrand <david@redhat.com>, 
	Anthony PERARD <anthony@xenproject.org>, Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000e4a930061995cd5e"

--000000000000e4a930061995cd5e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 27, 2024 at 6:25=E2=80=AFPM Philippe Mathieu-Daud=C3=A9 <philmd=
@linaro.org>
wrote:

> Hi Edgar,
>
> On 24/5/24 12:51, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>
> >
> > Add xen_mr_is_memory() to abstract away tests for the
> > xen_memory MR.
> >
> > No functional changes.
> >
> > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
> > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> > Acked-by: David Hildenbrand <david@redhat.com>
> > ---
> >   hw/xen/xen-hvm-common.c | 10 ++++++++--
> >   include/sysemu/xen.h    |  8 ++++++++
> >   2 files changed, 16 insertions(+), 2 deletions(-)
>
> To consolidate we could add:
>
>    static MemoryRegion xen_memory;
>
>    MemoryRegion *xen_mr_memory_init(uint64_t block_len)
>    {
>       assert(!xen_memory.size);
>       memory_region_init_ram(&xen_memory, NULL, "xen.ram", block_len,
> &error_fatal);
>       return &xen_memory;
>    }
>
> and remove the extern declaration.
>

Thanks,

We have a future patch series in the workings that will add a PVH machine
for x86, I'll keep this in mind for then!


>
> > diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> > index 754ec2e6cb..dc72f83bcb 100644
> > --- a/include/sysemu/xen.h
> > +++ b/include/sysemu/xen.h
> > @@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start,
> ram_addr_t length);
> >   void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
> >                      struct MemoryRegion *mr, Error **errp);
> >
> > +bool xen_mr_is_memory(MemoryRegion *mr);
> > +
> >   #else /* !CONFIG_XEN_IS_POSSIBLE */
> >
> >   #define xen_enabled() 0
> > @@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr=
,
> ram_addr_t size,
> >       g_assert_not_reached();
> >   }
> >
> > +static inline bool xen_mr_is_memory(MemoryRegion *mr)
> > +{
> > +    g_assert_not_reached();
> > +    return false;
>
> No need for the stub, just always declare xen_mr_is_memory() ...
> > +}
> > +
> >   #endif /* CONFIG_XEN_IS_POSSIBLE */
>
> ... here.
>
> >   #endif
>
> Removing the stub:
> Reviewed-by: Philippe Mathieu-Daud=C3=A9 <philmd@linaro.org>
>
>
Thanks Philippe, will remove the stubs in v8.

Cheers,
Edgar

--000000000000e4a930061995cd5e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Mon, May 27, 2024 at 6:25=E2=80=AFPM P=
hilippe Mathieu-Daud=C3=A9 &lt;<a href=3D"mailto:philmd@linaro.org">philmd@=
linaro.org</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px soli=
d rgb(204,204,204);padding-left:1ex">Hi Edgar,<br>
<br>
On 24/5/24 12:51, Edgar E. Iglesias wrote:<br>
&gt; From: &quot;Edgar E. Iglesias&quot; &lt;<a href=3D"mailto:edgar.iglesi=
as@amd.com" target=3D"_blank">edgar.iglesias@amd.com</a>&gt;<br>
&gt; <br>
&gt; Add xen_mr_is_memory() to abstract away tests for the<br>
&gt; xen_memory MR.<br>
&gt; <br>
&gt; No functional changes.<br>
&gt; <br>
&gt; Signed-off-by: Edgar E. Iglesias &lt;<a href=3D"mailto:edgar.iglesias@=
amd.com" target=3D"_blank">edgar.iglesias@amd.com</a>&gt;<br>
&gt; Reviewed-by: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kern=
el.org" target=3D"_blank">sstabellini@kernel.org</a>&gt;<br>
&gt; Acked-by: David Hildenbrand &lt;<a href=3D"mailto:david@redhat.com" ta=
rget=3D"_blank">david@redhat.com</a>&gt;<br>
&gt; ---<br>
&gt;=C2=A0 =C2=A0hw/xen/xen-hvm-common.c | 10 ++++++++--<br>
&gt;=C2=A0 =C2=A0include/sysemu/xen.h=C2=A0 =C2=A0 |=C2=A0 8 ++++++++<br>
&gt;=C2=A0 =C2=A02 files changed, 16 insertions(+), 2 deletions(-)<br>
<br>
To consolidate we could add:<br>
<br>
=C2=A0 =C2=A0static MemoryRegion xen_memory;<br>
<br>
=C2=A0 =C2=A0MemoryRegion *xen_mr_memory_init(uint64_t block_len)<br>
=C2=A0 =C2=A0{<br>
=C2=A0 =C2=A0 =C2=A0 assert(!xen_memory.size);<br>
=C2=A0 =C2=A0 =C2=A0 memory_region_init_ram(&amp;xen_memory, NULL, &quot;xe=
n.ram&quot;, block_len, <br>
&amp;error_fatal);<br>
=C2=A0 =C2=A0 =C2=A0 return &amp;xen_memory;<br>
=C2=A0 =C2=A0}<br>
<br>
and remove the extern declaration.<br></blockquote><div><br></div><div>Than=
ks,</div><div><br></div><div>We have a future patch series in the workings =
that will add a PVH machine for x86, I&#39;ll keep this in mind for then!</=
div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
&gt; diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h<br>
&gt; index 754ec2e6cb..dc72f83bcb 100644<br>
&gt; --- a/include/sysemu/xen.h<br>
&gt; +++ b/include/sysemu/xen.h<br>
&gt; @@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_a=
ddr_t length);<br>
&gt;=C2=A0 =C2=A0void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 struct MemoryRegion *mr, Error **errp);<br>
&gt;=C2=A0 =C2=A0<br>
&gt; +bool xen_mr_is_memory(MemoryRegion *mr);<br>
&gt; +<br>
&gt;=C2=A0 =C2=A0#else /* !CONFIG_XEN_IS_POSSIBLE */<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0#define xen_enabled() 0<br>
&gt; @@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_add=
r, ram_addr_t size,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0g_assert_not_reached();<br>
&gt;=C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0<br>
&gt; +static inline bool xen_mr_is_memory(MemoryRegion *mr)<br>
&gt; +{<br>
&gt; +=C2=A0 =C2=A0 g_assert_not_reached();<br>
&gt; +=C2=A0 =C2=A0 return false;<br>
<br>
No need for the stub, just always declare xen_mr_is_memory() ...<br>
&gt; +}<br>
&gt; +<br>
&gt;=C2=A0 =C2=A0#endif /* CONFIG_XEN_IS_POSSIBLE */<br>
<br>
... here.<br>
<br>
&gt;=C2=A0 =C2=A0#endif<br>
<br>
Removing the stub:<br>
Reviewed-by: Philippe Mathieu-Daud=C3=A9 &lt;<a href=3D"mailto:philmd@linar=
o.org" target=3D"_blank">philmd@linaro.org</a>&gt;<br>
<br></blockquote><div><br></div><div>Thanks Philippe, will remove the stubs=
 in v8.</div><div><br></div><div>Cheers,</div><div>Edgar</div></div></div>

--000000000000e4a930061995cd5e--


From xen-devel-bounces@lists.xenproject.org Wed May 29 11:14:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 11:14:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731819.1137537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHFv-0008PN-Gx; Wed, 29 May 2024 11:14:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731819.1137537; Wed, 29 May 2024 11:14:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHFv-0008PG-EI; Wed, 29 May 2024 11:14:07 +0000
Received: by outflank-mailman (input) for mailman id 731819;
 Wed, 29 May 2024 11:14:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9Qc4=NA=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1sCHFt-0008PA-DU
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 11:14:05 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20620.outbound.protection.outlook.com
 [2a01:111:f403:200a::620])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8d8f7b36-1dac-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 13:14:03 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by LV3PR12MB9259.namprd12.prod.outlook.com (2603:10b6:408:1b0::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Wed, 29 May
 2024 11:13:57 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7611.030; Wed, 29 May 2024
 11:13:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d8f7b36-1dac-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QdLoeC2dZNjczCD01oO8MJDc5dH1XZIxF3BmxP3UbDPsvKrllMs1PGUKs1mV3FjZ/N8hJjK8KxbGEKEUOPzkqkRFjny1WqiN4LWOMypdHePNJ4HUl9/1CASbhf78cKJrMARGs8eA3piYuqY5GllA/MoKJn4Xc+7jmWfL5MD839sVpIPvzcWgDas9dNaHt2CAocs4v/VARg7ZWXPlVIVU1WUQxqvwN6SObxgkyqc77ilqprMMEKeK6NdKSZee5RCTvdFp9RORcXu7I2SRvEesYKodBfdJeLnvfOGsCeQZBEm2qvC2fPKKu7lPGPrCqCNo7nEjX2yZFvd7kiKIg6+sgg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ouve0qwK2OY5i048n9z/9WGma22Qu49JQu6el8IvFSs=;
 b=cpbNvWVLei1q5h4GYNjdQdR/sd+PnmdqA/70NPWVXJPJUc5mXaWHvYP/8k7JL97Sr99mUsrQLeI2P0axaeSBVIZqqFkST0k9y0qHNKsf98T/6lQg9pkJ74iVmXOM84Uzo2e9ti32pXNenvhq6PB/EfdomoNE8hWLB27mTOTljArRwOMQOCEoPMw1ChBFa6i+XNyymEY6b1GvJm8uRGOShZoN29nqnMZvFPKf7gCBIDuRPcyOei5+KAdkIAMBCYInXJzLoa241PCZLihwn/wPxQx1F2Dkj2Z6eX8v1lSGV+NqhbFVjU0SXrh8XKUrTXIMcoaXNCprkV0XE2kX7eJJ1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ouve0qwK2OY5i048n9z/9WGma22Qu49JQu6el8IvFSs=;
 b=qiH/L9VWckqnt3pBFxhQGu4AC6t61Ool9Ns6NnLvomnLjsTNxvDcLSeanKACI/0pjemlCzykmRPubea4vegPAJl17+dxuF4K1EsfnTAOKPuWZORJWPW4RWRBhMaavof2u1u0GUU1aWAmDsH2CiqCizjhKzDlkXYlyXsrQNoMqWs=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index:
 AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4D//4e1AIAAis+A//+FmgCAEsTugP//viAAABEhvwD//4GYgIAAoyuA
Date: Wed, 29 May 2024 11:13:57 +0000
Message-ID:
 <BL1PR12MB58493D3365CC451F36DB554FE7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
In-Reply-To: <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7611.030)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|LV3PR12MB9259:EE_
x-ms-office365-filtering-correlation-id: 1ad8e48d-cc7f-4b05-847b-08dc7fd06ef5
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|7416005|1800799015|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?cFJ5SzAveVhRS3o2SXRnOGJXbjQ3ZC9NRlVXaklnMVdPOVpzek5WQ29RS0t6?=
 =?utf-8?B?bHFLMkFDVUlZM1ViZWZsZDhuQUc0dDdzd1VjWElPYWx6a1NRMXFQMGFTRGVm?=
 =?utf-8?B?K0FPNlYwc1JTT1VTN2RvYmQyanpGMWZlQUk4RENLdXlGUy9TajdyL3V2a25K?=
 =?utf-8?B?NE52WTNwTkZ4STk5TWNmd3lCTnJPRlJCYnpMbUw1eTFCMHpuQkp4dHFZZGw0?=
 =?utf-8?B?V0c2NE9selJoeWRXVmwzcGIwUTJUWmhJY2dpMDd3enpTSzkrVUR0VlZjbTN1?=
 =?utf-8?B?dzZkNlJsWWxMeUZxeTBlbWo1eHgwMUppUVNWUUdCZy9XR3NoVFJIU1RqR1JU?=
 =?utf-8?B?N3o4UDI1WnVNMUwwWlJUZVA0QTk0QTd1UFJCVERGTDkwN1JEeGZRQ2RmK1pi?=
 =?utf-8?B?OFZxZU4yclpsamZkRjFQek9FV0E3QVNqczU1OGIxWi9IY3pUWS91Yll5aExK?=
 =?utf-8?B?Ti9nY1ducUFSRUJLcVUzSlE1S1paREtTdnFMNUFQcDMrcTZqczR6aFdMZ2lv?=
 =?utf-8?B?K2R3d3E2OTVZdFBhckgxWTBRUnc3VzR4cHk2bXJGNU1UVjRPQVhubjJ2WVZO?=
 =?utf-8?B?RmxnbjVEa3NMUmJKMDRES3ZyLytzN3hWK2RKVFFZcHJlaExNVS96R2s0U1cv?=
 =?utf-8?B?YWJNaU9lLzVjYmFuaXIxcWRGWEszbm9BNkRpQXJYTWxaUXFyR0ZtMXdYL3lQ?=
 =?utf-8?B?K09pam94bHJibkhXZVVFUi9YVDA3d0V2cUxoWXZiQjIyS2pQNTNtOCt1RFor?=
 =?utf-8?B?NGhZMjdvd2ppRlRyZ1BxeDh2dEV2dVFXOFRRVzBBYXRaSlhZbllTREpYRWQy?=
 =?utf-8?B?N2piZEQ4Wi9naTFPWlpTRUJzQkl6aFU4SzFaN2dtN0h6cERXbjVZVFlqOFJk?=
 =?utf-8?B?REtnZnNxRFBiNTZxZHJqWmlIUEVCZDgyNkVEMmNFVlduVW03MzdZWHJjWnl4?=
 =?utf-8?B?NEJZTktBMWxjeGh3KzZ1M2NoUkZqYnd4QzRTc0hxNEJxTTdXeEpDekZZZlV0?=
 =?utf-8?B?VUdPYnBJSjBRRHhZUEp1OTUyOGF3NXJ5b2ZldXUxMFdJTXkzWU0yVHlLVDRw?=
 =?utf-8?B?NVR0SUpPWEkyeVV2STBEaXoyODFodnlXV203MUtFU1Bxa1lSazFMbTk2Ty9r?=
 =?utf-8?B?dHdzQzVPTFBTOFFJY0dxS2h4RlZyQkxKbWIzK2FVejM4c1I3ZXYxdlEwNEFi?=
 =?utf-8?B?bW1DK3diM3lKVkd3NmV3ekprUDVYR1V5VHAvQ2g1MkFPSUZzM2tVL3ZUNlFn?=
 =?utf-8?B?UmtGbTBKK09KMDNlcmUzR0c0TEt3ZW1rQjVuOEFaTDN4UDE1bkR0dnBveFA1?=
 =?utf-8?B?MXlnWWZ2b2JUN1E4NE5oLzU1R2t0aHpHNnhYQnhkS2JucnNLZnR5bVNpNDJP?=
 =?utf-8?B?ekxWK1M4UkZPVkloNGFPL2N0NHphWW5YK1NycWhOTXNjVVAwYmVRMHNWWEdW?=
 =?utf-8?B?eVlMSXFvd3VCSG5LRmhkR3Y3RDlzR0RNVzJkbXV3LzZ6L1pLN0RqY21uNm55?=
 =?utf-8?B?cjRrU0V0Qm9yOHVyd2crTXZTTlNJSjhIRmFtQjRKWlN1NHdUQ0hlVXBOeFds?=
 =?utf-8?B?dkU2Y3pnVzNJM3NqRWxMdEN4YUEyc1IvVEF1eEQwNXFiYzVvQXQxUmFjcVpC?=
 =?utf-8?B?MThXdnhZMUFIb1dURE1YcnV5bWswQkJzT09jNG12Q2pWdEYvbkNFMGMvNVJw?=
 =?utf-8?B?WFVjMVBmVDg3MmxoZ0kwZ1owbmh6QzZhS2VqQ1E4ZDlvVVFaWGJuR0RPUlZQ?=
 =?utf-8?Q?02OsxhmFcrOVpQs/YckG07yDYgqoyN5A0ZKbnHC?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?a2F5UjJNRUxHOTV3ay92OU5iQ2lzVzkrdFdpZlQwNUxYRXRCVjVDUHl5NG5z?=
 =?utf-8?B?d3NUQWovenUybUFUNGNabkF5Yk95aklxUDBWZU1XbWFtV2RoaVFpbVpBTGVv?=
 =?utf-8?B?bHNSNkxuNjBvTkZuQ3ZGQ0IwcVpkWEd4ZUNHK0NpdEFWcUZ1RFBJalFsSzdo?=
 =?utf-8?B?em9SSVpoWm1IRXNaU2tMMkNXdXVISWg1SFhxMDRvTmVWVHBMSStjTHJvL1ln?=
 =?utf-8?B?bEdlVklZUG5mU3diNDNxQk1Ib1AxTnMvWGROQzBYeTFkTUFsMm9vSHBRRU91?=
 =?utf-8?B?RW5UUDllbmZJZ0NPRk95RDNQblpRKytYeVhHczlZTDhhNmdjdldkVVRGaThD?=
 =?utf-8?B?dVlKMFc3b2RPY2xHRFpHT1dESUQ0YVFCWkpjVEFKYktsSmNKYXNqNjg5TXdN?=
 =?utf-8?B?dFJnaHJmRmg5WnVYVjBWeDlOWkJ5YnE0RlUxU2Q1NllPZkJMRjVSL1VPbnBT?=
 =?utf-8?B?Rmg5OFdabUpJTmFSL3hzUjZVajNSQjI3eCtvODdjZFhXSkIwYTBwWktBUVht?=
 =?utf-8?B?Vm5wM0NhcEx3ZXl0SWhoeTBUUEVhTFZlQjcvN2Y1R2pXZS9qZFlRbjJmblRr?=
 =?utf-8?B?N1ZQM042NjFUZUlnSUdlS3pqeVRKYkR4cThtWC9YYkYzRWt4VStoRXhlbDZG?=
 =?utf-8?B?NHU0eEVNTmcvazF2c3Rkc2ltT2NXMi9QOGRoM29TZ2wralQ3U09HSy9pUlJQ?=
 =?utf-8?B?eTJUbjByVXpqUGplZEtYSk01NnpLUkJuQjBZT09jT0FiYkllK1dMS2RERFVo?=
 =?utf-8?B?Z2Q2b1lwM0c1MUVUdURsMXB6ZzNkdHVNVVY3blFuclRwZnQwSi9mZklNRDc0?=
 =?utf-8?B?K093OXk5ME5VZFhBS1JEcmJ0ZW9xZHhGRndMRGRpMGtZaDBHQVlxSExmY0xt?=
 =?utf-8?B?WXdZUmR4eURGbnJ5UE43Y1pncS9SN1JtcHlzT1RyVHJHVWdMd0RRZU80dFpN?=
 =?utf-8?B?cFdjYUEzU1lMMWNiQ3kxUkR4Y2Flc0EwRmo3UlVqSjdjUTdBN2REUWwxL1Bw?=
 =?utf-8?B?RjBja05YcWN6SjV6bXZUbEkwUVBJamo3NVN3V1pZcjFaL1VHWVBHZVFSdmJJ?=
 =?utf-8?B?VUxjeVMxMlIxRW1FM2kxZ1drZ0lYSUNkSWtTLysybkZCeVAzK0JwZktmZjdj?=
 =?utf-8?B?VWYrTnNLaW9jdzRIUkdDWHVhd1l4b2JvWWttYkNYeUFWRnY1K2d4OVhpVHBj?=
 =?utf-8?B?KzlqYWVLaFlzNDVCUFZtcEZGMTBNVGw1Q1dkVGJaRDd1Ylo4Qm1hRXhCQ0JT?=
 =?utf-8?B?ZTM4c0dQZ21VajRhWlU0YjR1eVJ2WTlvUmMyZmYweXJ6S3packtaTEJjVVZo?=
 =?utf-8?B?NmYxNjVwdDY2bmlsNTFPSW9qSmdyN2x3ajNqSGxHU0poay9vT2JJcHhtU0Rr?=
 =?utf-8?B?RG96b094WFRzRkxHb2M4UElBSmMveFVaK0VlYk1QeE05dXpUa0xsMFJkRnZ3?=
 =?utf-8?B?OVRFTk52a2Y2WFdLRFNkV1IwQ0JUNkt5UU5xV3I4TXBSZXNZS1hFYklWcGpw?=
 =?utf-8?B?Qm1NN0VWcEgxdFpuSFBrRUxoWU9vTjNTTkVkVmRETUFPS1Z4QUhxUDNWRUd5?=
 =?utf-8?B?NWVrR2p4WmlPbzF0dTk2bXIyTFFwVVh0S0YxTTErM0VHSVpPWDk5Nzh1OExh?=
 =?utf-8?B?b2dCNHVUNkI4cHFSTmdtUFN2N0lwYml4SFlRZ2lLKzZUdTVpNTV2dk5BMEZo?=
 =?utf-8?B?MlhBQWJpMjhoVzJEank4ek1Hckl6c3UzbnFGWWxhVG5CM1FHMExMVWFTYzhq?=
 =?utf-8?B?emt1VWJoYzlIK2hQSkh5WjZXY0R1bnR0aVpQT2NXZnZ1TDZRY0laVjdBUlcy?=
 =?utf-8?B?K1BPYjkwK1oyZCs1LzBLemg3VGNRYWtlSHZxa3JZNjJxME5vSW9xc1BQVVhT?=
 =?utf-8?B?aXVzQ0pYMHJrV3V4OGwzcUpmNzNVM3A4djYwRExPKy9PMXVVc1BhblRKR1Jq?=
 =?utf-8?B?WTJmaHZUNTMzcUZQUG5GUXpDSlZNWXFTTjYvUnZkc0FueVJCMUxvZ0dodmxI?=
 =?utf-8?B?Qm0vMEpSSWhOSFVmNlh0K09LQXltNkFjVDVlMWlFc1FVTUExNWJETlhqYm5R?=
 =?utf-8?B?VHRHLzAycFppbGQ0Nkc5VVdzc3M5OFpFSngrMCtCWk12OXV6SHdxRzFNKzdG?=
 =?utf-8?Q?48rs=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <00EAC22BEDC7CC45AD35FCAA6E14900A@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ad8e48d-cc7f-4b05-847b-08dc7fd06ef5
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2024 11:13:57.0664
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 0Tnjs22SOMI/WYXOHGycthKOHbOMqioKXwdIXt45zbb1XPEyEe22c/ma5a4qQBdZo5CvXxL8gmG9uiWVdWd49A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9259

T24gMjAyNC81LzI5IDE1OjEwLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMjkuMDUuMjAyNCAw
ODo1NiwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzI5IDE0OjMxLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAyOS4wNS4yMDI0IDA0OjQxLCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IEhpLA0KPj4+PiBPbiAyMDI0LzUvMTcgMTk6NTAsIEphbiBCZXVsaWNoIHdyb3RlOg0K
Pj4+Pj4gT24gMTcuMDUuMjAyNCAxMzoxNCwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4+Pj4+IE9u
IDIwMjQvNS8xNyAxODo1MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+Pj4+IE9uIDE3LjA1LjIw
MjQgMTI6NDUsIENoZW4sIEppcWlhbiB3cm90ZToNCj4+Pj4+Pj4+IE9uIDIwMjQvNS8xNiAyMjow
MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+Pj4+Pj4gT24gMTYuMDUuMjAyNCAxMTo1MiwgSmlx
aWFuIENoZW4gd3JvdGU6DQo+Pj4+Pj4+Pj4+ICsgICAgICAgIGlmICggZ3NpID49IG5yX2lycXNf
Z3NpICkNCj4+Pj4+Pj4+Pj4gKyAgICAgICAgew0KPj4+Pj4+Pj4+PiArICAgICAgICAgICAgcmV0
ID0gLUVJTlZBTDsNCj4+Pj4+Pj4+Pj4gKyAgICAgICAgICAgIGJyZWFrOw0KPj4+Pj4+Pj4+PiAr
ICAgICAgICB9DQo+Pj4+Pj4+Pj4+ICsNCj4+Pj4+Pj4+Pj4gKyAgICAgICAgaWYgKCAhaXJxX2Fj
Y2Vzc19wZXJtaXR0ZWQoY3VycmVudC0+ZG9tYWluLCBnc2kpIHx8DQo+Pj4+Pj4+Pj4NCj4+Pj4+
Pj4+PiBJLmUuIGFzc3VtaW5nIElSUSA9PSBHU0k/IElzIHRoYXQgYSB2YWxpZCBhc3N1bXB0aW9u
IHdoZW4gYW55IG51bWJlciBvZg0KPj4+Pj4+Pj4+IHNvdXJjZSBvdmVycmlkZXMgbWF5IGJlIHN1
cmZhY2VkIGJ5IEFDUEk/DQo+Pj4+Pj4+PiBBbGwgaXJxcyBzbWFsbGVyIHRoYW4gbnJfaXJxc19n
c2kgYXJlIGdzaSwgYXJlbid0IHRoZXk/DQo+Pj4+Pj4+DQo+Pj4+Pj4+IFRoZXkgYXJlLCBidXQg
dGhlcmUncyBub3QgbmVjZXNzYXJpbHkgYSAxOjEgbWFwcGluZy4NCj4+Pj4+PiBPaCwgc28gZG8g
SSBuZWVkIHRvIGFkZCBhIG5ldyBnc2lfY2FwcyB0byBzdG9yZSBncmFudGVkIGdzaT8NCj4+Pj4+
DQo+Pj4+PiBQcm9iYWJseSBub3QuIFlvdSBvdWdodCB0byBiZSBhYmxlIHRvIHRyYW5zbGF0ZSBi
ZXR3ZWVuIEdTSSBhbmQgSVJRLA0KPj4+Pj4gYW5kIHRoZW4gY29udGludWUgdG8gcmVjb3JkIGlu
IC8gY2hlY2sgYWdhaW5zdCBJUlEgcGVybWlzc2lvbnMuDQo+Pj4+IEJ1dCBJIGZvdW5kIGluIGZ1
bmN0aW9uIGluaXRfaXJxX2RhdGE6DQo+Pj4+ICAgICBmb3IgKCBpcnEgPSAwOyBpcnEgPCBucl9p
cnFzX2dzaTsgaXJxKysgKQ0KPj4+PiAgICAgew0KPj4+PiAgICAgICAgIGludCByYzsNCj4+Pj4N
Cj4+Pj4gICAgICAgICBkZXNjID0gaXJxX3RvX2Rlc2MoaXJxKTsNCj4+Pj4gICAgICAgICBkZXNj
LT5pcnEgPSBpcnE7DQo+Pj4+DQo+Pj4+ICAgICAgICAgcmMgPSBpbml0X29uZV9pcnFfZGVzYyhk
ZXNjKTsNCj4+Pj4gICAgICAgICBpZiAoIHJjICkNCj4+Pj4gICAgICAgICAgICAgcmV0dXJuIHJj
Ow0KPj4+PiAgICAgfQ0KPj4+PiBEb2VzIGl0IG1lYW4gdGhhdCB3aGVuIGlycSA8IG5yX2lycXNf
Z3NpLCB0aGUgZ3NpIGFuZCBpcnEgaXMgYSAxOjEgbWFwcGluZz8NCj4+Pg0KPj4+IE5vLCBhcyBl
eHBsYWluZWQgYmVmb3JlLiBJIGFsc28gZG9uJ3Qgc2VlIGhvdyB5b3Ugd291bGQgZGVyaXZlIHRo
YXQgZnJvbSB0aGUgY29kZSBhYm92ZS4NCj4+IEJlY2F1c2UgaGVyZSBzZXQgZGVzYy0+aXJxID0g
aXJxLCBhbmQgaXQgc2VlbXMgdGhlcmUgaXMgbm8gb3RoZXIgcGxhY2UgdG8gY2hhbmdlIHRoaXMg
ZGVzYy0+aXJxLCBzbywgZ3NpIDEgaXMgY29uc2lkZXJlZCB0byBpcnEgMS4NCj4gDQo+IFdoYXQg
YXJlIHlvdSB0YWtpbmcgdGhpcyBmcm9tPyBUaGUgbG9vcCBib3VuZCBpc24ndCBucl9nc2lzLCBh
bmQgdGhlIGl0ZXJhdGlvbg0KPiB2YXJpYWJsZSBpc24ndCBpbiBHU0kgc3BhY2UgZWl0aGVyOyBp
dCdzIGluIElSUSBudW1iZXJpbmcgc3BhY2UuIEluIHRoaXMgbG9vcA0KPiB3ZSdyZSBtZXJlbHkg
bGV2ZXJhZ2luZyB0aGF0IGV2ZXJ5IEdTSSBoYXMgYSBjb3JyZXNwb25kaW5nIElSUTsNCj4gdGhl
cmUgYXJlIG5vIGFzc3VtcHRpb25zIG1hZGUgYWJvdXQgdGhlIG1hcHBpbmcgYmV0d2VlbiB0aGUg
dHdvLiBBZmFpY3MgYXQgbGVhc3QuDQo+IA0KPj4+ICJucl9pcnFzX2dzaSIgZGVzY3JpYmVzIHdo
YXQgaXRzIG5hbWUgc2F5czogVGhlIG51bWJlciBvZg0KPj4+IElSUXMgbWFwcGluZyB0byBhIChf
c29tZV8pIEdTSS4gVGhhdCdzIHRvIHRlbGwgdGhlbSBmcm9tIHRoZSBub24tR1NJIChpLmUuDQo+
Pj4gbWFpbmx5IE1TSSkgb25lcy4gVGhlcmUncyBubyBpbXBsaWNhdGlvbiB3aGF0c29ldmVyIG9u
IHRoZSBJUlEgPC0+IEdTSQ0KPj4+IG1hcHBpbmcuDQo+Pj4NCj4+Pj4gV2hhdCdzIG1vcmUsIHdo
ZW4gdXNpbmcgUEhZU0RFVk9QX3NldHVwX2dzaSwgaXQgY2FsbHMgbXBfcmVnaXN0ZXJfZ3NpLA0K
Pj4+PiBhbmQgaW4gbXBfcmVnaXN0ZXJfZ3NpLCBpdCB1c2VzICIgZGVzYyA9IGlycV90b19kZXNj
KGdzaSk7ICIgdG8gZ2V0IGlycV9kZXNjIGRpcmVjdGx5Lg0KPj4+DQo+Pj4gV2hpY2ggbWF5IGJl
IHdyb25nLCB3aGlsZSB0aGF0IHdyb25nLW5lc3MgbWF5IG5vdCBoYXZlIGhpdCBhbnlvbmUgaW4N
Cj4+PiBwcmFjdGljZSAoZm9yIHJlYXNvbnMgdGhhdCB3b3VsZCBuZWVkIHdvcmtpbmcgb3V0KS4N
Cj4+Pg0KPj4+PiBDb21iaW5pbmcgYWJvdmUsIGNhbiB3ZSBjb25zaWRlciAiZ3NpID09IGlycSIg
d2hlbiBpcnEgPCBucl9pcnFzX2dzaSA/DQo+Pj4NCj4+PiBBZ2FpbiAtIG5vLg0KPj4gU2luY2Ug
eW91IGFyZSBjZXJ0YWluIHRoYXQgdGhleSBhcmUgbm90IGVxdWFsLCBjb3VsZCB5b3UgdGVsbCBt
ZSB3aGVyZSBzaG93IHRoZXkgYXJlIG5vdCBlcXVhbCBvciB3aGVyZSBidWlsZCB0aGVpciBtYXBw
aW5ncywNCj4+IHNvIHRoYXQgSSBjYW4ga25vdyBob3cgdG8gZG8gYSBjb252ZXJzaW9uIGdzaSBm
cm9tIGlycS4NCj4gDQo+IEkgZGlkIHBvaW50IHlvdSBhdCB0aGUgQUNQSSBJbnRlcnJ1cHQgU291
cmNlIE92ZXJyaWRlIHN0cnVjdHVyZSBiZWZvcmUuDQo+IFdlJ3JlIHBhcnNpbmcgdGhvc2UgaW4g
YWNwaV9wYXJzZV9pbnRfc3JjX292cigpLCB0byBnaXZlIHlvdSBhIHBsYWNlIHRvDQo+IHN0YXJ0
IGdvaW5nIGZyb20uDQpPaCEgSSB0aGluayBJIGtub3cuDQpJZiBJIHdhbnQgdG8gdHJhbnNmb3Jt
IGdzaSB0byBpcnEsIEkgbmVlZCB0byBkbyBiZWxvdzoNCglpbnQgaXJxLCBlbnRyeSwgaW9hcGlj
LCBwaW47DQoNCglpb2FwaWMgPSBtcF9maW5kX2lvYXBpYyhnc2kpOw0KCXBpbiA9IGdzaSAtIG1w
X2lvYXBpY19yb3V0aW5nW2lvYXBpY10uZ3NpX2Jhc2U7DQoJZW50cnkgPSBmaW5kX2lycV9lbnRy
eShpb2FwaWMsIHBpbiwgbXBfSU5UKTsNCglpcnEgPSBwaW5fMl9pcnEoZW50cnksIGlvYXBpYywg
cGluKTsNCg0KQW0gSSByaWdodD8NCg0KPiANCj4gSmFuDQoNCi0tIA0KQmVzdCByZWdhcmRzLA0K
SmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Wed May 29 11:47:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 11:47:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731826.1137547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHm0-0004tB-TC; Wed, 29 May 2024 11:47:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731826.1137547; Wed, 29 May 2024 11:47:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCHm0-0004t4-PO; Wed, 29 May 2024 11:47:16 +0000
Received: by outflank-mailman (input) for mailman id 731826;
 Wed, 29 May 2024 11:47:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U3iI=NA=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sCHly-0004sy-S0
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 11:47:14 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2fd72bb4-1db1-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 13:47:12 +0200 (CEST)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-57a033c2e9fso1013748a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 04:47:12 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93d38esm715595566b.90.2024.05.29.04.47.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 04:47:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fd72bb4-1db1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716983232; x=1717588032; darn=lists.xenproject.org;
        h=in-reply-to:from:content-language:references:cc:to:subject
         :user-agent:mime-version:date:message-id:from:to:cc:subject:date
         :message-id:reply-to;
        bh=S/Psd25rW2y4rli4LwsJaW87kPft2OszbH0V2yg3gwc=;
        b=E6oCTbHXTSSU77yjoU2yY+PWcZCh1nvrHix8EW89Po5LqJEQarUcjxGU4d/0sSNcH1
         e54nZsx6rjSDjwejVCT2GPg1gL4q/8IZ2jzn4odArY0QIcV1DlZafILg3mFDISenwYqq
         Myb5P5fwGPCkiV/uOOygeM6RKww/FeTs1VU6ZsXR63gsXT9P0XHlN3PyT6j4GZ3gNhMl
         OjrvsGt+40JwufkJf70b/HbYI4+HMnBIwUbNX77hHhtjsugGPL+HB3+rZk+FjP3bBzxW
         q7QSV/1mog+tDuLw4nM77pBh5OR7Xk7XL+rauiGIX/WB5LzEhRj270hiSHaO5vK3xBff
         hRrA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716983232; x=1717588032;
        h=in-reply-to:from:content-language:references:cc:to:subject
         :user-agent:mime-version:date:message-id:x-gm-message-state:from:to
         :cc:subject:date:message-id:reply-to;
        bh=S/Psd25rW2y4rli4LwsJaW87kPft2OszbH0V2yg3gwc=;
        b=XiM9pmp7yTav9/mi0vkvl/omGq0wMxjTzHUayYXyfinDi3vxoVpJr2nzjvihmxxROj
         8V6lJRKV5mByqybex9u+EpvNwfFlxdymR2uOhpm5gwByIPe8umG0/ThN0osLm8VTs88c
         NaavK/NrnokLEbQvSXAZn13iYJcj/PsisJj565Uj+6spjz0wRpZrEmG91X3XJ5hrxA/k
         KC2pTNEXc6PJYyxfRFuNUaTcf8duPNhwRB5AeIvRMpFw6PqjckofkcSTHst946PDFN26
         sb24EKcpBPsjuuQvrHbcS4hozrRwnFe6M7uMljUPWrDC4UtG9DKyzpBnwf6pehtlhgJ3
         IgjQ==
X-Forwarded-Encrypted: i=1; AJvYcCX2t64Sgnplvf7F+TTFmeoOI/0WbI94X1Xe+tIT+/Tg2xZmhoe9IlqYulRTi0frv/T5abCdykOTj96Pu7SJk5M6vP49aw1aWmt4hRSJWYk=
X-Gm-Message-State: AOJu0YzlfdZjn3CKHYyqvSkkhkxZ7gLIZ0A1yNzfKz1Zr50JTmBaNOQF
	2L56VPjy5yEVtAeU7bQwkMnyyhgK1j+1ixr8U/fWk6at0Y/hMjPok7ROCrOYRPU=
X-Google-Smtp-Source: AGHT+IGJWdxHIY+FPGA9gdd5UvM+HwASGUODX3LSDhISFrxbeOG/caoasZJ+wVAGhVT0Ww1I+CijvA==
X-Received: by 2002:a17:906:af92:b0:a61:ac3e:2b4c with SMTP id a640c23a62f3a-a62646d5e73mr1374069566b.40.1716983231582;
        Wed, 29 May 2024 04:47:11 -0700 (PDT)
Content-Type: multipart/mixed; boundary="------------1aFz7oXFQ2CEfG3fkws5c0WR"
Message-ID: <a749c066-0c05-47dd-b499-e712e5454033@suse.com>
Date: Wed, 29 May 2024 13:47:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Cc: Dario Faggioli <dfaggioli@suse.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ZlW-ZyFeAzvh3pGG@macbook>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <ZlW-ZyFeAzvh3pGG@macbook>

This is a multi-part message in MIME format.
--------------1aFz7oXFQ2CEfG3fkws5c0WR
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 28.05.24 13:22, Roger Pau Monné wrote:
> Hello,
> 
> When the stop_machine_run() call in cpu_down() fails and calls the CPU
> notifier CPU_DOWN_FAILED hook the following assert triggers in the
> scheduling code:
> 
> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
> ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
> CPU:    0
> RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> RFLAGS: 0000000000010093   CONTEXT: hypervisor
> rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
> rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
> rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
> r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
> r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
> cr3: 00000000574c2000   cr2: 0000000000000000
> fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
>   fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
> Xen stack trace from rsp=ffff83203ffffd30:
>     ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
>     0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
>     0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
>     ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
>     ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
>     00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
>     ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
>     ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
>     ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
>     ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
>     ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
>     0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
>     ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
>     0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
>     ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
>     0000000000000000 0000000000000000 0000000000000000 0000000000000000
>     0000000000000000 0000000000000000 0000000000000000 0000000000000000
>     0000000000000000 0000000000000000 0000000000000000 0000000000000000
>     0000000000000000 0000000000000000 0000000000000000 0000000000000000
>     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> Xen call trace:
>     [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
>     [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
>     [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
>     [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
>     [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
>     [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
>     [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
>     [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
>     [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
>     [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
>     [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
>     [<ffff82d040324391>] F continue_running+0x5b/0x5d
> 
> 
> ****************************************
> Panic on CPU 0:
> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
> ****************************************
> 
> The issue seems to be that since the CPU hasn't been removed, it's
> still part of prv->initialized and the assert in csched2_free_pdata()
> called as part of free_cpu_rm_data() triggers.
> 
> It's easy to reproduce by substituting the stop_machine_run() call in
> cpu_down() with an error.

Could you please give the attached patch a try?

Only compile tested, though...


Juergen

--------------1aFz7oXFQ2CEfG3fkws5c0WR
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-xen-sched-fix-error-path-of-cpu-removal.patch"
Content-Disposition: attachment;
 filename="0001-xen-sched-fix-error-path-of-cpu-removal.patch"
Content-Transfer-Encoding: base64

RnJvbSA1OTI1ZjE1YWNlOGJlMTg2YzlmNDFjMGNkYTJkNGE2NzViMGY3YmI5IE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+
CkRhdGU6IFdlZCwgMjkgTWF5IDIwMjQgMTM6MjQ6MzYgKzAyMDAKU3ViamVjdDogW1BBVENI
XSB4ZW4vc2NoZWQ6IGZpeCBlcnJvciBwYXRoIG9mIGNwdSByZW1vdmFsCk1JTUUtVmVyc2lv
bjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApDb250ZW50
LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpJbiBjYXNlIHJlbW92YWwgb2YgYSBjcHUgZmFp
bHMgYXQgdGhlIENQVV9EWUlORyBzdGVwLCBhbiBBU1NFUlQoKSBpbgp0aGUgY3JlZGl0MiBz
Y2hlZHVsZXIgbWlnaHQgdHJpZ2dlcjoKCkFzc2VydGlvbiAnIWNwdW1hc2tfdGVzdF9jcHUo
Y3B1LCAmcHJ2LT5pbml0aWFsaXplZCknIGZhaWxlZCBhdCBjb21tb24vc2NoZWQvY3JlZDEK
LS0tLVsgWGVuLTQuMTktdW5zdGFibGUgIHg4Nl82NCAgZGVidWc9eSAgVGFpbnRlZDogICBD
ICAgIF0tLS0tCkNQVTogICAgMApSSVA6ICAgIGUwMDg6WzxmZmZmODJkMDQwMjQ4Mjk5Pl0g
Y29tbW9uL3NjaGVkL2NyZWRpdDIuYyNjc2NoZWQyX2ZyZWVfcGRhdGErMHhjOC8weDE3NwpS
RkxBR1M6IDAwMDAwMDAwMDAwMTAwOTMgICBDT05URVhUOiBoeXBlcnZpc29yCnJheDogMDAw
MDAwMDAwMDAwMDAwMCAgIHJieDogZmZmZjgzMjAyZWNjMmY4MCAgIHJjeDogZmZmZjgzMjAy
ZjNlNjRjMApyZHg6IDAwMDAwMDAwMDAwMDAwMDEgICByc2k6IDAwMDAwMDAwMDAwMDAwMDIg
ICByZGk6IGZmZmY4MzIwMmVjYzJmODgKcmJwOiBmZmZmODMyMDNmZmZmZDU4ICAgcnNwOiBm
ZmZmODMyMDNmZmZmZDMwICAgcjg6ICAwMDAwMDAwMDAwMDAwMDAwCnI5OiAgZmZmZjgzMjAy
ZjNlNmUwMSAgIHIxMDogMDAwMDAwMDAwMDAwMDAwMCAgIHIxMTogMGYwZjBmMGYwZjBmMGYw
ZgpyMTI6IGZmZmY4MzIwMmVjYjgwYjAgICByMTM6IDAwMDAwMDAwMDAwMDAwMDEgICByMTQ6
IDAwMDAwMDAwMDAwMDAyODIKcjE1OiBmZmZmODMyMDJlY2JiZjAwICAgY3IwOiAwMDAwMDAw
MDgwMDUwMDNiICAgY3I0OiAwMDAwMDAwMDAwNzUyNmUwCmNyMzogMDAwMDAwMDA1NzRjMjAw
MCAgIGNyMjogMDAwMDAwMDAwMDAwMDAwMApmc2I6IDAwMDAwMDAwMDAwMDAwMDAgICBnc2I6
IDAwMDAwMDAwMDAwMDAwMDAgICBnc3M6IDAwMDAwMDAwMDAwMDAwMDAKZHM6IDAwMDAgICBl
czogMDAwMCAgIGZzOiAwMDAwICAgZ3M6IDAwMDAgICBzczogMDAwMCAgIGNzOiBlMDA4Clhl
biBjb2RlIGFyb3VuZCA8ZmZmZjgyZDA0MDI0ODI5OT4gKGNvbW1vbi9zY2hlZC9jcmVkaXQy
LmMjY3NjaGVkMl9mcmVlX3BkYXRhKzB4YzgvMHgxNzcpOgogZmUgZmYgZWIgOWEgMGYgMGIg
MGYgMGIgPDBmPiAwYiA0OSA4ZCA0ZiAwOCA0OSA4YiA0NyAwOCA0OCAzYiA0OCAwOCA3NSAy
ZQpYZW4gc3RhY2sgdHJhY2UgZnJvbSByc3A9ZmZmZjgzMjAzZmZmZmQzMDoKICAgZmZmZjgz
MjAyZDc0ZDEwMCAwMDAwMDAwMDAwMDAwMDAxIGZmZmY4MmQwNDA0YzQ0MzAgMDAwMDAwMDAw
MDAwMDAwNgogICAwMDAwMDAwMDAwMDAwMDAwIGZmZmY4MzIwM2ZmZmZkNzggZmZmZjgyZDA0
MDI1NzQ1NCAwMDAwMDAwMDAwMDAwMDAwCiAgIDAwMDAwMDAwMDAwMDAwMDEgZmZmZjgzMjAz
ZmZmZmRhOCBmZmZmODJkMDQwMjFmMzAzIGZmZmY4MmQwNDA0YzQ2MjgKICAgZmZmZjgyZDA0
MDRjNDYyMCBmZmZmODJkMDQwNGM0NDMwIDAwMDAwMDAwMDAwMDAwMDYgZmZmZjgzMjAzZmZm
ZmRmMAogICBmZmZmODJkMDQwMjJiYzRjIGZmZmY4MzIwM2ZmZmZlMTggMDAwMDAwMDAwMDAw
MDAwMSAwMDAwMDAwMDAwMDAwMDAxCiAgIDAwMDAwMDAwZmZmZmZmZjAgMDAwMDAwMDAwMDAw
MDAwMCAwMDAwMDAwMDAwMDAwMDAwIGZmZmY4MmQwNDA1ZTY1MDAKICAgZmZmZjgzMjAzZmZm
ZmUwOCBmZmZmODJkMDQwMjA0ZmQ1IDAwMDAwMDAwMDAwMDAwMDEgZmZmZjgzMjAzZmZmZmUz
MAogICBmZmZmODJkMDQwMjA1NGYwIGZmZmY4MmQwNDA0YzU4NjAgMDAwMDAwMDAwMDAwMDAw
MSBmZmZmODMyMDJlYzc1MDAwCiAgIGZmZmY4MzIwM2ZmZmZlNDggZmZmZjgyZDA0MDM0OGMy
NSBmZmZmODMyMDJkNzRkMGQwIGZmZmY4MzIwM2ZmZmZlNjgKICAgZmZmZjgyZDA0MDIwNzFh
YSBmZmZmODMyMDJlYzc1MWQwIGZmZmY4MmQwNDA1Y2UyMTAgZmZmZjgzMjAzZmZmZmU4MAog
ICBmZmZmODJkMDQwMjM0M2M5IGZmZmY4MmQwNDA1Y2UyMDAgZmZmZjgzMjAzZmZmZmViMCBm
ZmZmODJkMDQwMjM0NjMxCiAgIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwN2ZmZiBm
ZmZmODJkMDQwNWQ1MDgwIGZmZmY4MmQwNDA1Y2UyMTAKICAgZmZmZjgzMjAzZmZmZmVlOCBm
ZmZmODJkMDQwMzIxNDExIGZmZmY4MmQwNDAzMjEzOTkgZmZmZjgzMjAyZjNhOTAwMAogICAw
MDAwMDAwMDAwMDAwMDAwIDAwMDAwMDFkOTFhNmZhMmQgZmZmZjgyZDA0MDVlNjUwMCBmZmZm
ODMyMDNmZmZmZGUwCiAgIGZmZmY4MmQwNDAzMjQzOTEgMDAwMDAwMDAwMDAwMDAwMCAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAKICAgMDAwMDAwMDAwMDAwMDAwMCAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMAogICAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwCiAgIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAKICAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMAogICAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMDAw
MDAwMDAwClhlbiBjYWxsIHRyYWNlOgogICBbPGZmZmY4MmQwNDAyNDgyOTk+XSBSIGNvbW1v
bi9zY2hlZC9jcmVkaXQyLmMjY3NjaGVkMl9mcmVlX3BkYXRhKzB4YzgvMHgxNzcKICAgWzxm
ZmZmODJkMDQwMjU3NDU0Pl0gRiBmcmVlX2NwdV9ybV9kYXRhKzB4NDEvMHg1OAogICBbPGZm
ZmY4MmQwNDAyMWYzMDM+XSBGIGNvbW1vbi9zY2hlZC9jcHVwb29sLmMjY3B1X2NhbGxiYWNr
KzB4ZmIvMHg0NjYKICAgWzxmZmZmODJkMDQwMjJiYzRjPl0gRiBub3RpZmllcl9jYWxsX2No
YWluKzB4NmMvMHg5NgogICBbPGZmZmY4MmQwNDAyMDRmZDU+XSBGIGNvbW1vbi9jcHUuYyNj
cHVfbm90aWZpZXJfY2FsbF9jaGFpbisweDFiLzB4MzYKICAgWzxmZmZmODJkMDQwMjA1NGYw
Pl0gRiBjcHVfZG93bisweGE3LzB4MTQzCiAgIFs8ZmZmZjgyZDA0MDM0OGMyNT5dIEYgY3B1
X2Rvd25faGVscGVyKzB4MTEvMHgyNwogICBbPGZmZmY4MmQwNDAyMDcxYWE+XSBGIGNvbW1v
bi9kb21haW4uYyNjb250aW51ZV9oeXBlcmNhbGxfdGFza2xldF9oYW5kbGVyKzB4NTAvMHhi
ZAogICBbPGZmZmY4MmQwNDAyMzQzYzk+XSBGIGNvbW1vbi90YXNrbGV0LmMjZG9fdGFza2xl
dF93b3JrKzB4NzYvMHhhZgogICBbPGZmZmY4MmQwNDAyMzQ2MzE+XSBGIGRvX3Rhc2tsZXQr
MHg1Yi8weDhkCiAgIFs8ZmZmZjgyZDA0MDMyMTQxMT5dIEYgYXJjaC94ODYvZG9tYWluLmMj
aWRsZV9sb29wKzB4NzgvMHhlNgogICBbPGZmZmY4MmQwNDAzMjQzOTE+XSBGIGNvbnRpbnVl
X3J1bm5pbmcrMHg1Yi8weDVkCgpUaGlzIGNhbiBiZSBmaXhlZCBieSBub3QgZnJlZWluZyB0
aGUgY3B1J3Mgc2NoZWR1bGluZyBkYXRhIGluIGNhc2UgdGhlCkNQVV9EWUlORyBzdGVwIGhh
c24ndCBiZWVuIHBlcmZvcm1lZC4gSW5zdGVhZCB0aGUgbm90IHVzZWQgc3RydWN0CnNjaGVk
X3Jlc291cmNlIGluc3RhbmNlcyBuZWVkIHRvIGJlIGZyZWVkIGluIG9yZGVyIHRvIGF2b2lk
IG1lbW9yeQpsZWFraW5nLgoKRml4ZXM6IGQ4NDQ3MzY4OTYxMSAoInhlbi9zY2hlZDogZml4
IGNwdSBob3RwbHVnIikKUmVwb3J0ZWQtYnk6IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2VyLnBh
dUBjaXRyaXguY29tPgpTaWduZWQtb2ZmLWJ5OiBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3Vz
ZS5jb20+Ci0tLQogeGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMgICAgfCAxOCArKysrKysrKysr
KysrKysrLS0KIHhlbi9jb21tb24vc2NoZWQvcHJpdmF0ZS5oIHwgIDEgKwogMiBmaWxlcyBj
aGFuZ2VkLCAxNyBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBh
L3hlbi9jb21tb24vc2NoZWQvY29yZS5jIGIveGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMKaW5k
ZXggZDg0YjY1ZjE5Ny4uY2JiZWVhZjBjYSAxMDA2NDQKLS0tIGEveGVuL2NvbW1vbi9zY2hl
ZC9jb3JlLmMKKysrIGIveGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMKQEAgLTMyNjQsNiArMzI2
NCw4IEBAIHN0cnVjdCBjcHVfcm1fZGF0YSAqYWxsb2NfY3B1X3JtX2RhdGEodW5zaWduZWQg
aW50IGNwdSwgYm9vbCBhZmZfYWxsb2MpCiAgICAgICAgIGRhdGEtPnNyW2lkeF0tPnNjaGVk
dWxlX2xvY2sgPSBzci0+c2NoZWR1bGVfbG9jazsKICAgICB9CiAKKyAgICBkYXRhLT5uX3Ny
X3VudXNlZCA9IGlkeDsKKwogIG91dDoKICAgICByY3VfcmVhZF91bmxvY2soJnNjaGVkX3Jl
c19yY3Vsb2NrKTsKIApAQCAtMzI3Miw4ICszMjc0LDE4IEBAIHN0cnVjdCBjcHVfcm1fZGF0
YSAqYWxsb2NfY3B1X3JtX2RhdGEodW5zaWduZWQgaW50IGNwdSwgYm9vbCBhZmZfYWxsb2Mp
CiAKIHZvaWQgZnJlZV9jcHVfcm1fZGF0YShzdHJ1Y3QgY3B1X3JtX2RhdGEgKm1lbSwgdW5z
aWduZWQgaW50IGNwdSkKIHsKLSAgICBzY2hlZF9mcmVlX3VkYXRhKG1lbS0+b2xkX29wcywg
bWVtLT52cHJpdl9vbGQpOwotICAgIHNjaGVkX2ZyZWVfcGRhdGEobWVtLT5vbGRfb3BzLCBt
ZW0tPnBwcml2X29sZCwgY3B1KTsKKyAgICB1bnNpZ25lZCBpbnQgaWR4OworCisgICAgaWYg
KCAhbWVtLT5uX3NyX3VudXNlZCApCisgICAgeworICAgICAgICBzY2hlZF9mcmVlX3VkYXRh
KG1lbS0+b2xkX29wcywgbWVtLT52cHJpdl9vbGQpOworICAgICAgICBzY2hlZF9mcmVlX3Bk
YXRhKG1lbS0+b2xkX29wcywgbWVtLT5wcHJpdl9vbGQsIGNwdSk7CisgICAgfQorICAgIGVs
c2UKKyAgICB7CisgICAgICAgIGZvciAoIGlkeCA9IDA7IGlkeCA8IG1lbS0+bl9zcl91bnVz
ZWQ7IGlkeCsrICkKKyAgICAgICAgICAgIHNjaGVkX3Jlc19mcmVlKCZtZW0tPnNyW2lkeF0t
PnJjdSk7CisgICAgfQogICAgIGZyZWVfYWZmaW5pdHlfbWFza3MoJm1lbS0+YWZmaW5pdHkp
OwogCiAgICAgeGZyZWUobWVtKTsKQEAgLTMzMTIsNiArMzMyNCw4IEBAIGludCBzY2hlZHVs
ZV9jcHVfcm0odW5zaWduZWQgaW50IGNwdSwgc3RydWN0IGNwdV9ybV9kYXRhICpkYXRhKQog
ICAgIC8qIFNlZSBjb21tZW50IGluIHNjaGVkdWxlX2NwdV9hZGQoKSByZWdhcmRpbmcgbG9j
ayBzd2l0Y2hpbmcuICovCiAgICAgb2xkX2xvY2sgPSBwY3B1X3NjaGVkdWxlX2xvY2tfaXJx
c2F2ZShjcHUsICZmbGFncyk7CiAKKyAgICBkYXRhLT5uX3NyX3VudXNlZCA9IDA7CisKICAg
ICBmb3JfZWFjaF9jcHUgKCBjcHVfaXRlciwgc3ItPmNwdXMgKQogICAgIHsKICAgICAgICAg
cGVyX2NwdShzY2hlZF9yZXNfaWR4LCBjcHVfaXRlcikgPSAwOwpkaWZmIC0tZ2l0IGEveGVu
L2NvbW1vbi9zY2hlZC9wcml2YXRlLmggYi94ZW4vY29tbW9uL3NjaGVkL3ByaXZhdGUuaApp
bmRleCBjMGU3Yzk2ZDI0Li5mMDEyOWZkOWNjIDEwMDY0NAotLS0gYS94ZW4vY29tbW9uL3Nj
aGVkL3ByaXZhdGUuaAorKysgYi94ZW4vY29tbW9uL3NjaGVkL3ByaXZhdGUuaApAQCAtNjQ1
LDYgKzY0NSw3IEBAIHN0cnVjdCBjcHVfcm1fZGF0YSB7CiAgICAgY29uc3Qgc3RydWN0IHNj
aGVkdWxlciAqb2xkX29wczsKICAgICB2b2lkICpwcHJpdl9vbGQ7CiAgICAgdm9pZCAqdnBy
aXZfb2xkOworICAgIHVuc2lnbmVkIGludCBuX3NyX3VudXNlZDsKICAgICBzdHJ1Y3Qgc2No
ZWRfcmVzb3VyY2UgKnNyW107CiB9OwogCi0tIAoyLjM1LjMKCg==

--------------1aFz7oXFQ2CEfG3fkws5c0WR--


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:13:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:13:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731833.1137557 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIAp-0000vz-VV; Wed, 29 May 2024 12:12:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731833.1137557; Wed, 29 May 2024 12:12:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIAp-0000vs-Sd; Wed, 29 May 2024 12:12:55 +0000
Received: by outflank-mailman (input) for mailman id 731833;
 Wed, 29 May 2024 12:12:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCIAp-0000vm-9a
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:12:55 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c292fa46-1db4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 14:12:46 +0200 (CEST)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-52ab1d7243fso1087326e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:12:46 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8d606sm721799666b.176.2024.05.29.05.12.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 05:12:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c292fa46-1db4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716984766; x=1717589566; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=fmRB6jrOdeLAvItV4kDL65wtkp1iX9xMxEWKsiF7NLc=;
        b=KKpAsr5JcRBdaiSpqjmlY5B1HGr8o+rWWEGr6/wbxZLFHxSDmBRDtmVSEVPmf4NfFo
         BaGTs+ynb0uFu6wW3DWY56ujIE3F2CrXqyPJ05GCxxeL6ADFxw1b3WUcSh8+kkk4m8JX
         oc5nDtiysWbOj2sDQsfeZA/O5fVD2j1bXP8DYLXDMffOqvlIOFbEtibO1uQGI1uFxmME
         5vy27JlhbnhyB1/DCY0STw4OGeP1wSIaDz/hCItLU3SMp4oeApsP5b0Pqwg0gWzt3EqU
         PVT9fPGl4E+IgM2mh6IEFVbqNM08Z6jCoWeMhQULXJKpiK1INMpxYaBzHCRQqq6e06xM
         A/pA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716984766; x=1717589566;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=fmRB6jrOdeLAvItV4kDL65wtkp1iX9xMxEWKsiF7NLc=;
        b=lyrKjm7Fi70PtEZ2+O8dFGMvUZc52TwqYmWSZBsmYZotN6HGnhjASwglpiHTKx7F3S
         zBhyqrKm5lqUEMPVgvqpF/GHbev8qAiPvH0qXpQZu7vC+cebBeelin1f2kGDIMgad+TU
         Lkaet5LT2TobNlRHWFbBbCj8rtUQiMFgZxl0QdgScP2td2q1NbcRg5p1VB3wUJjQIfHx
         qq4KnTZmWtpgCyM0Sk5TCX/tyU+svJJE8lc/+UbH+Fne0LhdNaqxwkd0q6vcKL49DRtO
         /QKtTpKpRBN7ttHF4d0KmDf5Ka+8sz13SEINHdDhGHaQ96rHyeHib34D0Q5BUtArP7JT
         bcvQ==
X-Forwarded-Encrypted: i=1; AJvYcCU29hqRkYOt246F5p32hCL1wYI9u6SZrhGXw7Yb787vcLvUxlusCM3O49ZsL1P3jNu2eIqB9NPrsqurx3lbr1YwIK/sv8zSA5DOzJrg6EQ=
X-Gm-Message-State: AOJu0YzxppiVjPJtjcKBYrwlq7izgvIKh8FJTlzpPKoy8N+rcoYOPA9N
	SIo10r8jFJ/hNgGWHO0uYvS9XUU8fMLwqG5Mo71SpC2uT7hRewcLzFlJOecBhw==
X-Google-Smtp-Source: AGHT+IHc6zNMy6fBtIKY22dplRV0lCweNsI+hKLE0IKrdJxQPIg4XCsVhuU1X1ZlR/SBxjXvpdS9kw==
X-Received: by 2002:a05:6512:ad6:b0:529:b717:2a14 with SMTP id 2adb3069b0e04-529b7172ebdmr5368906e87.14.1716984766206;
        Wed, 29 May 2024 05:12:46 -0700 (PDT)
Message-ID: <8b73306a-cb79-43b6-9da5-bb90c0bf8627@suse.com>
Date: Wed, 29 May 2024 14:12:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Partial revert of "x86/MCE: optional build of AMD/Intel
 MCE code"
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Sergiy Kibrik <Sergiy_Kibrik@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240529103752.1431366-1-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240529103752.1431366-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 12:37, Andrew Cooper wrote:
> {cmci,lmce}_support are written during S3 resume, so cannot live in
> __ro_after_init.  Move them back to being __read_mostly, as they were
> originally.
> 
> Link: https://gitlab.com/xen-project/xen/-/jobs/6966698361
> Fixes: 19b6e9f9149f ("x86/MCE: optional build of AMD/Intel MCE code")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Wed May 29 12:16:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:16:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731837.1137566 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIEY-0001U1-Dd; Wed, 29 May 2024 12:16:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731837.1137566; Wed, 29 May 2024 12:16:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIEY-0001Tu-B1; Wed, 29 May 2024 12:16:46 +0000
Received: by outflank-mailman (input) for mailman id 731837;
 Wed, 29 May 2024 12:16:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCIEX-0001To-1h
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:16:45 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4f910756-1db5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:16:43 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a626919d19dso149639766b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:16:43 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8d70bsm707420166b.180.2024.05.29.05.16.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 05:16:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f910756-1db5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716985003; x=1717589803; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=rG/fvfzzduYmDhNxdJuBdBQWYN0pxrOBGx3ksl4Y5A4=;
        b=Ay7RHCQn5ioXrcML2obpJMUIN9gPulBAUdqJIRQqYRdvB7cBtdsH19srtZDMAZBvip
         Riayk/4uM/qnqZCTMKaXsgJsthlS9zxvncdQF9la4GAUn4cBPrGC6C6qvdIdPveceRYs
         cg99s3rutDwnZ5lf/WiqxJXv0sLQ4aMqU8djm1nPpePK42Dc3+yIigAUhGpQDjoz1T93
         q7i1WJJweE1HQlWjcbyuwLLaeO3Akw2QLcp6It1PH4TZU/NZEOzWVIU9S8sPJSzSyKyi
         h/wbUoE8a3ODlDVjF5M+J/1IHPRVJgDcZhu72L3YYZKDD7MT06UQ5YGuBGFFhY/p7S+O
         0C5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716985003; x=1717589803;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rG/fvfzzduYmDhNxdJuBdBQWYN0pxrOBGx3ksl4Y5A4=;
        b=Ee1z7e8bB6/kk/7O6RxLU7rC22DPa4PwvVpcTmRE85dxX0k5Gt/0JnFtN1UekRh6XG
         5Tr4hSU0XIzP9iWDcDCWZYFqqaHrlYtJZrqOIKlh0eCNGkVd5UjWSAZEEp4GveOVxq00
         BTb22+bvHE0XCuVqwjYujaXVgPgNa0Y4AZNVV+82yfdd35VU9yPEtKPmZxr3zdGNOFkb
         ateqxymz3gCt5EDzpJKJduQr57zbux2POE/9fboNeudID5MEHWgP6SNWTLcVfggK0Hyl
         KEtr/6u2vqTvxXtc6IfFccpud93EXU6EBX1XTOH+ZrwqwzHdRFLS0pI5Lg3lPw1pyTZe
         TvAQ==
X-Gm-Message-State: AOJu0YxeuPE5fRKDtR/Z6debu3ut5c1ZQEbmw1hHO7AWL82khmY5QZQA
	5uCAzsonEIJonmglQrcYxoTOak4e8zX9ehtx7/l9WGnH7xLx6B4rQhi5kRd1cg==
X-Google-Smtp-Source: AGHT+IHR/6lZ/Dx+CVPKGCJswTV2hCRNabPR03yu1XxtMiJVSbZjjVPRlCqTJOwnbV8GcRNOhkB0AA==
X-Received: by 2002:a17:906:a898:b0:a62:2e6e:d28c with SMTP id a640c23a62f3a-a642da8aeedmr167064066b.34.1716985002973;
        Wed, 29 May 2024 05:16:42 -0700 (PDT)
Message-ID: <bfa3a3e0-0cc7-4321-80f1-814a256a054f@suse.com>
Date: Wed, 29 May 2024 14:16:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 12/15] x86/vmx: guard access to cpu_has_vmx_* in
 common code
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: xen-devel@lists.xenproject.org, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <c1f40e8e9a35f7e9ba55adc44a9fe93d48749c92.1715761386.git.Sergiy_Kibrik@epam.com>
 <alpine.DEB.2.22.394.2405151748080.2544314@ubuntu-linux-20-04-desktop>
 <4f2c1486-1b58-4c8d-a74e-ad83bf9cc4c7@suse.com>
 <8d9fd413-fa35-429d-bc4b-8261671b9c6a@epam.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8d9fd413-fa35-429d-bc4b-8261671b9c6a@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 12:58, Sergiy Kibrik wrote:
> 16.05.24 10:32, Jan Beulich:
>> On 16.05.2024 02:50, Stefano Stabellini wrote:
>>> On Wed, 15 May 2024, Sergiy Kibrik wrote:
>>>> There're several places in common code, outside of arch/x86/hvm/vmx,
>>>> where cpu_has_vmx_* get accessed without checking if VMX present first.
>>>> We may want to guard these macros, as they read global variables defined
>>>> inside vmx-specific files -- so VMX can be made optional later on.
>>>>
>>>> Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
>>>> CC: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> CC: Jan Beulich <jbeulich@suse.com>
>>>> ---
>>>> Here I've tried a different approach from prev.patches [1,2] -- instead of
>>>> modifying whole set of cpu_has_{svm/vmx}_* macros, we can:
>>>>   1) do not touch SVM part at all, because just as Andrew pointed out they're
>>>> used inside arch/x86/hvm/svm only.
>>>>   2) track several places in common code where cpu_has_vmx_* features are
>>>> checked out and guard them using cpu_has_vmx condition
>>>>   3) two of cpu_has_vmx_* macros being used in common code are checked in a bit
>>>> more tricky way, so instead of making complex conditionals even more complicated,
>>>> we can instead integrate cpu_has_vmx condition inside these two macros.
>>>>
>>>> This patch aims to replace [1,2] from v1 series by doing steps above.
>>>>
> [..]
>>>
>>> I am missing some of the previous discussions but why can't we just fix
>>> all of the cpu_has_vmx_* #defines in vmcs.h to also check for
>>> cpu_has_vmx?
>>>
>>> That seems easier and simpler than to add add-hoc checks at the invocations?
>>
>> I'd like to take the question on step further: Following 0b5f149338e3
>> ("x86/HVM: hide SVM/VMX when their enabling is prohibited by firmware"),
>> is this change needed at all? IOW is there a path left where cpu_has_vmx
>> may be false, by any cpu_has_vmx_* may still yield true?
>>
> 
> This change is about exec control variables (vmx_secondary_exec_control, 
> vmx_pin_based_exec_control etc) not been built, because they're in vmx 
> code, but accessed in common code. The description is probably unclear 
> about that.
> Also build issues related to VMX can be solved differently, without 
> touching cpu_has_vmx_* macros and related logic at all.
> I can move exec control variables from vmcs.c to common hvm.c, this 
> would be simpler change and directly related to problem that I'm having.

That would be moving them one layer too high. Proper disentangling then
will need to wait until that data is actually part of the (host) CPU
policy. For the time being your change may thus be acceptable, assuming
that we won't be very quick in doing said move.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:21:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:21:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731843.1137576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIJ0-0003IL-U5; Wed, 29 May 2024 12:21:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731843.1137576; Wed, 29 May 2024 12:21:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIJ0-0003IE-RX; Wed, 29 May 2024 12:21:22 +0000
Received: by outflank-mailman (input) for mailman id 731843;
 Wed, 29 May 2024 12:21:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCIIz-0003I8-Cu
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:21:21 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCIIr-0006HU-BV; Wed, 29 May 2024 12:21:13 +0000
Received: from [15.248.2.235] (helo=[10.24.67.23])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCIIr-0001Zw-3M; Wed, 29 May 2024 12:21:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=fa+qKvA3WLWsfIvafx9t6tO6vymn5iXMkbx+AH/aoAM=; b=tb0il0hI86CTPNlMEJXbVBC0ud
	4uPPeA/BPr3B8REYExKoqE6tiqV73h15TO+WeutjF4BUxZoPTuzICvaT/G6ITdYBTvNaW0LYerofo
	TAS8lm50hecvXMbvJeLzCmzCoAU0UJt/pMuWasiQouLz1GedH/2T+GBxah/cERGNVN7w=;
Message-ID: <ff1139a3-4f69-46d4-8f0c-c3aae778be3d@xen.org>
Date: Wed, 29 May 2024 13:21:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 v3 0/3] xen/x86: support foreign mappings for
 HVM/PVH
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Community Manager <community.manager@xenproject.org>,
 Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20240517133352.94347-1-roger.pau@citrix.com>
 <0c4a9a00-b8de-4472-afb4-0a8d70c7aada@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <0c4a9a00-b8de-4472-afb4-0a8d70c7aada@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 29/05/2024 08:24, Jan Beulich wrote:
> On 17.05.2024 15:33, Roger Pau Monne wrote:
>> Hello,
>>
>> The following series attempts to solve a shortcoming of HVM/PVH guests
>> with the lack of support for foreign mappings.  Such lack of support
>> prevents using PVH based guests as stubdomains for example.
>>
>> Add support in a way similar to how it's done on Arm, by iterating over
>> the p2m based on the maximum gfn.
>>
>> Patch 2 is not strictly needed.  Moving the enablement of altp2m from an
>> HVM param to a create domctl flag avoids any possible race with the HVM
>> param changing after it's been evaluated.  Note the param can only be
>> set by the control domain, and libxl currently sets it at domain
>> create.  Also altp2m enablement is different from activation, as
>> activation does happen during runtime of the domain.
>>
>> Thanks, Roger.
>>
>> Roger Pau Monne (3):
>>    xen/x86: account number of foreign mappings in the p2m
>>    xen/x86: enable altp2m at create domain domctl
>>    xen/x86: remove foreign mappings from the p2m on teardown
> 
> Here, too, I'd like to ask whether to keep this as a candidate for 4.19, or
> whether to postpone. Afaict what's still missing are Arm and tool chain acks
> on patch 2.

Stefano already provided an Ack [1], so the Arm part is now covered.

I am also guessing you meant "tools change" rather than "tool chain". 
Anthony is away at the moment. But this was reviewed by Juergen.

Oleksii also release-ack the series, so I think it would be fine to 
merge it.

Cheers,

[1] 
https://lore.kernel.org/xen-devel/8e9a563f1e0f45883502bc47c4942081a5d66e1a.camel@gmail.com/T/#mdcb3e89046f817bd1b98190baa2b6d6b04e1ec05

> 
> Jan
> 
>>   CHANGELOG.md                        |  1 +
>>   tools/libs/light/libxl_create.c     | 23 +++++++++-
>>   tools/libs/light/libxl_x86.c        | 26 +++++------
>>   tools/ocaml/libs/xc/xenctrl_stubs.c |  2 +-
>>   xen/arch/arm/domain.c               |  6 +++
>>   xen/arch/x86/domain.c               | 28 ++++++++++++
>>   xen/arch/x86/hvm/hvm.c              | 23 +++++++++-
>>   xen/arch/x86/include/asm/p2m.h      | 32 +++++++++-----
>>   xen/arch/x86/mm/p2m-basic.c         | 18 ++++++++
>>   xen/arch/x86/mm/p2m.c               | 68 +++++++++++++++++++++++++++--
>>   xen/include/public/domctl.h         | 20 ++++++++-
>>   xen/include/public/hvm/params.h     |  9 +---
>>   12 files changed, 215 insertions(+), 41 deletions(-)
>>
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:22:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:22:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731848.1137587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIJy-0003oY-87; Wed, 29 May 2024 12:22:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731848.1137587; Wed, 29 May 2024 12:22:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIJy-0003oR-59; Wed, 29 May 2024 12:22:22 +0000
Received: by outflank-mailman (input) for mailman id 731848;
 Wed, 29 May 2024 12:22:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCIJw-0003Hx-ED
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:22:20 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17dd407a-1db6-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:22:19 +0200 (CEST)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-5785f861868so2553259a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:22:19 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578524bae15sm8434039a12.86.2024.05.29.05.22.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 05:22:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17dd407a-1db6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716985339; x=1717590139; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=na9fcaKGSlAgX9uq5y39B2dcV5IpquzF8DZjZIqaaOE=;
        b=JapTokb9vQErW6GD0V8luujmfqasoX2rGaYktOtBPl04Flzsk3mv3pM4/mnGYxg1fl
         MoTSQ8cmrelxf132FdEO5DCmNTptQDqJZp4BDH5Moj2lkJ1jDwSYYI8bSS3D27cObYT9
         Q5M/jbtECEFvb5ltJJ6T0WrR/JsrSnVuxwp0XujaqUCER32Bdher+bvjcW63CsY6i7np
         UeBV7OfqIghBIHYxDHQ0IvEOIvJogFjSw4i4nFb00coO1euHgENnItY6Yo9s3glB+2AV
         X1AnqbFvk2J2HyYiWKusoyOwDVSK+xT3MTbgLwJ7RtEx3Gg7NA49M9H0sqx2jjrbW9Zm
         crKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716985339; x=1717590139;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=na9fcaKGSlAgX9uq5y39B2dcV5IpquzF8DZjZIqaaOE=;
        b=fBLsDOJ8FpROWAll3vChnE9iAysydE99eDCeTOH0uz/OFuJejbrxkqoehHbUskuLIJ
         zGfWoQ7ANlRipSSc1lFDSHy/YqiLf/xZ2RCcx/awjnZ7gelqcwVSooqIJhotLBOymC8L
         /XTPFv30n/5MMoDm14OPQbh6GATrmwTKjt1yUvrHTF0474j4UqedFjQ+xTZFXGspTB/P
         STbP6tanW6Z3KnXiCRXqPerY8S8up4nDP7YibZAyaqpvnYoxHR20fOAMn65jAUPp7p98
         DtPVcStzI2jHlZrLe4OChti81jX0qNMhkCVwvmksWGio37PnzYlFmUyHP66Z4wkcKDw+
         gZNA==
X-Forwarded-Encrypted: i=1; AJvYcCWYyX/1fR8Ip1mnDNxS2saY8y3t1E8ubo2OY/M+2Vlrxn5hn+Gtt/Zne/JSMVdWRke1RiRGcpNSD1L/NkaPsOdu9uxkRwmrtvq5ny8v93U=
X-Gm-Message-State: AOJu0Yxc/w9vgNPFTP0eeMOgiXOK5yoDqeGt54nju5//CYJRijccfDCr
	HGGGHGhXV//bM4iKXbUsTggUZSoI3eRR0gEiKiaoNPEIMndmRj822Bs7TWKPJQ==
X-Google-Smtp-Source: AGHT+IHkCwPGgTIU4SBN2CVNqIPZ76XnHdJDcGodJU9OhLVSe+7Lan/03wR+gc3pY3fhjCcRz/3XMg==
X-Received: by 2002:a50:a45d:0:b0:578:63e0:e4f1 with SMTP id 4fb4d7f45d1cf-57863e0e555mr9312388a12.17.1716985339090;
        Wed, 29 May 2024 05:22:19 -0700 (PDT)
Message-ID: <fbaf7086-85d8-4433-91d9-ef8f74512685@suse.com>
Date: Wed, 29 May 2024 14:22:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
 <BL1PR12MB58493D3365CC451F36DB554FE7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB58493D3365CC451F36DB554FE7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 13:13, Chen, Jiqian wrote:
> On 2024/5/29 15:10, Jan Beulich wrote:
>> On 29.05.2024 08:56, Chen, Jiqian wrote:
>>> On 2024/5/29 14:31, Jan Beulich wrote:
>>>> On 29.05.2024 04:41, Chen, Jiqian wrote:
>>>>> But I found in function init_irq_data:
>>>>>     for ( irq = 0; irq < nr_irqs_gsi; irq++ )
>>>>>     {
>>>>>         int rc;
>>>>>
>>>>>         desc = irq_to_desc(irq);
>>>>>         desc->irq = irq;
>>>>>
>>>>>         rc = init_one_irq_desc(desc);
>>>>>         if ( rc )
>>>>>             return rc;
>>>>>     }
>>>>> Does it mean that when irq < nr_irqs_gsi, the gsi and irq is a 1:1 mapping?
>>>>
>>>> No, as explained before. I also don't see how you would derive that from the code above.
>>> Because here set desc->irq = irq, and it seems there is no other place to change this desc->irq, so, gsi 1 is considered to irq 1.
>>
>> What are you taking this from? The loop bound isn't nr_gsis, and the iteration
>> variable isn't in GSI space either; it's in IRQ numbering space. In this loop
>> we're merely leveraging that every GSI has a corresponding IRQ;
>> there are no assumptions made about the mapping between the two. Afaics at least.
>>
>>>> "nr_irqs_gsi" describes what its name says: The number of
>>>> IRQs mapping to a (_some_) GSI. That's to tell them from the non-GSI (i.e.
>>>> mainly MSI) ones. There's no implication whatsoever on the IRQ <-> GSI
>>>> mapping.
>>>>
>>>>> What's more, when using PHYSDEVOP_setup_gsi, it calls mp_register_gsi,
>>>>> and in mp_register_gsi, it uses " desc = irq_to_desc(gsi); " to get irq_desc directly.
>>>>
>>>> Which may be wrong, while that wrong-ness may not have hit anyone in
>>>> practice (for reasons that would need working out).
>>>>
>>>>> Combining above, can we consider "gsi == irq" when irq < nr_irqs_gsi ?
>>>>
>>>> Again - no.
>>> Since you are certain that they are not equal, could you tell me where show they are not equal or where build their mappings,
>>> so that I can know how to do a conversion gsi from irq.
>>
>> I did point you at the ACPI Interrupt Source Override structure before.
>> We're parsing those in acpi_parse_int_src_ovr(), to give you a place to
>> start going from.
> Oh! I think I know.
> If I want to transform gsi to irq, I need to do below:
> 	int irq, entry, ioapic, pin;
> 
> 	ioapic = mp_find_ioapic(gsi);
> 	pin = gsi - mp_ioapic_routing[ioapic].gsi_base;
> 	entry = find_irq_entry(ioapic, pin, mp_INT);
> 	irq = pin_2_irq(entry, ioapic, pin);
> 
> Am I right?

This looks plausible, yes.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:23:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:23:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731853.1137597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIL9-0004Pb-KD; Wed, 29 May 2024 12:23:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731853.1137597; Wed, 29 May 2024 12:23:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIL9-0004PU-HQ; Wed, 29 May 2024 12:23:35 +0000
Received: by outflank-mailman (input) for mailman id 731853;
 Wed, 29 May 2024 12:23:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vA4L=NA=gmail.com=yskelg@srs-se1.protection.inumbo.net>)
 id 1sCIL7-0004PK-Lj
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:23:33 +0000
Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com
 [2607:f8b0:4864:20::1035])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 42e7f4f2-1db6-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:23:32 +0200 (CEST)
Received: by mail-pj1-x1035.google.com with SMTP id
 98e67ed59e1d1-2bdd8968dabso1493451a91.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:23:32 -0700 (PDT)
Received: from paran-QEMU-Virtual-Machine.. ([118.32.98.101])
 by smtp.gmail.com with ESMTPSA id
 98e67ed59e1d1-2bf5f987248sm9399359a91.44.2024.05.29.05.23.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 05:23:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42e7f4f2-1db6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716985411; x=1717590211; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Bu8X2LCUalqzu3Ya3NkSgnwgrwDIZEosx1eIzHQjc28=;
        b=TeFe4NSCMMl017y+MOfd4lylluB+eQdlE54itGLIPe7o3z0PjghW4eWZ/3QVVR08fj
         /PXZ9lKYG8yCTRARFmYTtyrC1ExWBkmaoCnI7A6bJU7QUt+oxcrgQCiOMZm/banIZReU
         57PRVDcjFDjXnCllQe6q7WeXJNJiYQbe2ShzyqeA6jWnn7f05qiNODg8HZCOd7OD3jS/
         PeVz5MS0e0nHPOr4eiOwohFq6t0BRvAGN+YZh4cTpQNeLiutBmoH3zmyqLZsH5apfK0f
         kxrQ64UFheF0b7cNSiHb8Ovnwma0P2JKWPLvc1+pJ2x45kaInahefHGuwCqiJsIXI/vW
         lMgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716985411; x=1717590211;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Bu8X2LCUalqzu3Ya3NkSgnwgrwDIZEosx1eIzHQjc28=;
        b=SC+ep7X0nHvpvs8FU8m7TLGe9zJ+WSHra7yA4YOd6U1BPGv/TyMJdVTJD5KClq2Kmz
         fU7uYkullmnXXrTAwco9LOJIDtRDMAPjD0hZhmgS1pKlrMzx+ApgDqXU1Qy4MblVHr4d
         yo0jGoB09ub8JMwkHuiOQuAksk3ti718roDJ4hfdx6/vzp4d9c3Z2s37m3AgrlBE+ciH
         VTv2wl0BKprQXfyiOwvOwDDE2TgQyIeqbqY1JpMRRAR3Fmw4o0TxKtyS/TXR6Zs/OENC
         gXvzCZ/BMb+ZMV8s+FIg2wVs7iNiysFwJ27o0EtIqu47wW6Wu9Q9hFdowWmtkPOREVkG
         HKsg==
X-Forwarded-Encrypted: i=1; AJvYcCV4IdL4m/nckulyyvcd5Tk+0m9BzP5hbC2PU4Oq04YhY0bMfJihDz4c3ED5BJbGg5EsHiCdPWrDm7Z023BYZc0fspWPr19qLx7OfvtiC8c=
X-Gm-Message-State: AOJu0YxE8SjPFNbgWmosNZPj5G3oPQPTNAzfr5vzswuAtU2YBIv/AhlT
	SrFAS2m6ZL4OswjtN4TuaoEm/ttoBWewbxJHgIhZy/5Ht3bZvt5e
X-Google-Smtp-Source: AGHT+IGpop5Rh0Gl6NPGn+yzSeNeXcyhhaZwMs48bAu4Ha451DIdYSt4ISyatz1BTqz9iqNA8XPlsg==
X-Received: by 2002:a17:90b:46d3:b0:2ae:7f27:82cd with SMTP id 98e67ed59e1d1-2bf5e171f8amr12877376a91.7.1716985411004;
        Wed, 29 May 2024 05:23:31 -0700 (PDT)
From: yskelg@gmail.com
To: Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: skhan@linuxfoundation.org,
	sj@kernel.org,
	Austin Kim <austindh.kim@gmail.com>,
	shjy180909@gmail.com,
	linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-kernel-mentees@lists.linuxfoundation.org,
	Yunseong Kim <yskelg@gmail.com>
Subject: [PATCH] xen/xenbus: handle potential dangling pointer issue in xen_pcibk_xenbus_probe
Date: Wed, 29 May 2024 21:22:33 +0900
Message-Id: <20240529122232.25360-1-yskelg@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Yunseong Kim <yskelg@gmail.com>

If 'xen_pcibk_init_devices()' fails. This ensures that 'pdev->xdev' does
not point to 'xdev' when 'pdev' is freed.

Signed-off-by: Yunseong Kim <yskelg@gmail.com>
---
 drivers/xen/xen-pciback/xenbus.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index b11e401f1b1e..348d6803b8c0 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -54,6 +54,7 @@ static struct xen_pcibk_device *alloc_pdev(struct xenbus_device *xdev)
 	INIT_WORK(&pdev->op_work, xen_pcibk_do_op);
 
 	if (xen_pcibk_init_devices(pdev)) {
+		pdev->xdev = NULL;
 		kfree(pdev);
 		pdev = NULL;
 	}
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 12:28:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:28:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731860.1137606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIPP-00051N-3G; Wed, 29 May 2024 12:27:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731860.1137606; Wed, 29 May 2024 12:27:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIPP-00051G-0b; Wed, 29 May 2024 12:27:59 +0000
Received: by outflank-mailman (input) for mailman id 731860;
 Wed, 29 May 2024 12:27:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U3iI=NA=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sCIPN-00051A-Lk
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:27:57 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0949662-1db6-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:27:56 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4211249fdf4so18314445e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:27:56 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100fad7fasm211038455e9.38.2024.05.29.05.27.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 05:27:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0949662-1db6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716985676; x=1717590476; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=o0im46VMvTgzf4CgpxLpfyG0tO3nmPdi92aMWun6A3U=;
        b=KUb4RRdHLI2vuXJ+h00fC7xsxwTijWMHRUbaYcHW8oOxrRQ2PANDoDaapU6yCKlmjQ
         g/unyPn5n0RRP96rhczt9iCDaKt09Ooi3Gc3sAxzea7xG1hyqzp/WD6vFHRVrPFkKQR9
         BOhIYMihp4EEjVE5T+fT7iXZj6CKG2YlKVtamhU4tC1Byz6wtR4tulVKa9I4EVRLWJvC
         trEfm9WKD+joD0o0uQ3Ih1cMuB23mn48lojM+u9qRghUTOsiSs4wNzG014auPbAVqeDL
         KlR1yUq5H/8iq2ipjBRg58+nMwy/WCMt+sW7a2sivCwHV8o1f4uo968XznGD8NZXJIzJ
         eQ3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716985676; x=1717590476;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=o0im46VMvTgzf4CgpxLpfyG0tO3nmPdi92aMWun6A3U=;
        b=sU+/uOgOk3EZDwkB+PM5aCtxgBqjpPCWMchrLYZ/pcxf0daBh3cXJ5qhElTbi4Aeuw
         lMWN1totWma807BvF77JS6eLRjuUsNcZwfa+aC7QOfoAIlPLprCPtvHib5qKe7ZKiznS
         7bpMMVMC8JfLj1mkIgrxJLxh+HscNohYVeH6KH8IAjy+5ov6KyrK0FoWAXA/qhWabpTB
         BtGk2cecSbbZS+WCg9Cx8mrrjnm1ZbY4HtKkT6spLJNg3u6xB568JnMIW5bPmghkDm8h
         wLl2f6dDrvUqDgN7BVmGfv4Hfbdt5e2tl/hLUzMtpZ8eaNhiY2SRCSWR/4/XJ7a0M3VI
         pWqw==
X-Forwarded-Encrypted: i=1; AJvYcCWQ0zhInG4VHyuCDppKD+RZQ05X0gHDVuIEU2vXElKQWbZqAsyBpc+AoyH6hPFv/UX8oP4wmavDuthCAUa1MvRwosSAGEpoyAhKB8VIJDM=
X-Gm-Message-State: AOJu0Yw4UmmDxzoqYNhuYcZcu9/yMawqqRwStouNHs3OV/4jaAfmZK1o
	39b64K67vWR+b4dSCYlq7KFFdZ1JpncxYxxQxsbHyb2Ok079URn/uhBxeduidvw=
X-Google-Smtp-Source: AGHT+IGh33p4hKsb6CQ/ea4derqiTxXb5sBTzSj5MqjCQx3uSuJyvMs24xDjjXfWO5KMwaEt87X2sA==
X-Received: by 2002:a05:600c:5593:b0:420:2b2e:f6e7 with SMTP id 5b1f17b1804b1-42108a79034mr125180765e9.17.1716985675868;
        Wed, 29 May 2024 05:27:55 -0700 (PDT)
Message-ID: <0c53415d-3394-455a-a6b5-99d3efb92318@suse.com>
Date: Wed, 29 May 2024 14:27:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/xenbus: handle potential dangling pointer issue in
 xen_pcibk_xenbus_probe
To: yskelg@gmail.com, Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: skhan@linuxfoundation.org, sj@kernel.org,
 Austin Kim <austindh.kim@gmail.com>, shjy180909@gmail.com,
 linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
 linux-kernel-mentees@lists.linuxfoundation.org
References: <20240529122232.25360-1-yskelg@gmail.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240529122232.25360-1-yskelg@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 29.05.24 14:22, yskelg@gmail.com wrote:
> From: Yunseong Kim <yskelg@gmail.com>
> 
> If 'xen_pcibk_init_devices()' fails. This ensures that 'pdev->xdev' does
> not point to 'xdev' when 'pdev' is freed.
> 
> Signed-off-by: Yunseong Kim <yskelg@gmail.com>
> ---
>   drivers/xen/xen-pciback/xenbus.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index b11e401f1b1e..348d6803b8c0 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -54,6 +54,7 @@ static struct xen_pcibk_device *alloc_pdev(struct xenbus_device *xdev)
>   	INIT_WORK(&pdev->op_work, xen_pcibk_do_op);
>   
>   	if (xen_pcibk_init_devices(pdev)) {
> +		pdev->xdev = NULL;
>   		kfree(pdev);
>   		pdev = NULL;
>   	}

NAK.

This doesn't make any sense, as pdev is freed.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:41:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:41:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731871.1137617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIbw-00006J-7D; Wed, 29 May 2024 12:40:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731871.1137617; Wed, 29 May 2024 12:40:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIbw-00006C-4A; Wed, 29 May 2024 12:40:56 +0000
Received: by outflank-mailman (input) for mailman id 731871;
 Wed, 29 May 2024 12:40:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCIbv-000066-83
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:40:55 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id afff5eae-1db8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:40:53 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-52a6ef5e731so1178675e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:40:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc500a1sm713337466b.125.2024.05.29.05.40.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 05:40:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afff5eae-1db8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716986453; x=1717591253; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=leLXeNe0XEV2m4n0wwCD4Lh/2K4710UzGF+YXnQAG8M=;
        b=ZKGTuFIYxsEnc5dCtCnIIzS3Wvsgk3d0aL8Bt9UgTjgI+CJVc6+zwQk4yg4a7CiEVe
         LmdSSr8XUDhvxv1qGu6urDM+T3wjjhOVScx1mDpCKsLiw2cQB1bCDI5LkMitq065kDYg
         xQ3dryIWiHvGv3F/1z/oGdleNagabeEBOS5gCbzPgpejNK7ngz+UKnc48bjO/84W7Qir
         hjsQ/edUOlHYEGccl/xlIxQMlSlhZ6cuxDRzmThyFAkqX3kAwm9wbyOm65p7sp0eO6dH
         dgie01URdwc9mPHpdEtMzN2Hms2Q5r09K3nqZYyVhU3hfQ40X3a2bPyKdaMjhTlnyl/u
         Syjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716986453; x=1717591253;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=leLXeNe0XEV2m4n0wwCD4Lh/2K4710UzGF+YXnQAG8M=;
        b=MKSfT0MK8Schy2RvOSDCbY7JujvKKM35AlgWnprPlYfbCtIgp5Jqd5imaiqRcAbsOS
         6zzxNE0dkqDBkhfmLGUTbJ+yKSdoxB7H8uK0jMPSkvpVlSRXPqVqc20g2/WxYaj3Jsc1
         3Gn4T+aorH9F2YH+ODlcD1W8LCU0Q+qzYE7QmaI5eh49S/oWw27ykn643rhIQ8arosY4
         lU/jqUJahMA7W79skJ8SD2KSWUuMfablr5Xw+gneFrmsW3ud+VVVoM+9xS2WAfuU6g7e
         bh4GC5E+zf64lf4AksKBD8DluW79KEVxqHD1DvgBHi8ipa02utoyllJ1EdgLQcbsK1Uy
         O4gg==
X-Forwarded-Encrypted: i=1; AJvYcCUFyBunCksqgktWJlFoZYR4BnoEdmvSY8Zu8bdBF/TV50v0bj8U74CcH6zIw5ORk8l52JTHd8j0FXphCFLi/5dx8Uoetu19qxEEwT0bhLw=
X-Gm-Message-State: AOJu0YzRMdrgvbKRHiM7RpO4JB1HAcLVklr/bRsgc1BJ6KzP+Z3XAomk
	QzCE5CUNAo9Yyo99n912/Fkqlto5bMoLw9hCcv8H25DAT0PZfMqK1DpmgMQDhA==
X-Google-Smtp-Source: AGHT+IGm8PeX2GO9KrCtKsVZD1buqtBvVZxRdsCL5NMokU1Z8WYvyfCMZqEXmlHtphwMnL3v8n8o2A==
X-Received: by 2002:a05:6512:52e:b0:52b:4c20:5cec with SMTP id 2adb3069b0e04-52b4c205d4fmr712196e87.61.1716986453293;
        Wed, 29 May 2024 05:40:53 -0700 (PDT)
Message-ID: <0a2f1f9a-fab1-409c-96f8-399b19077f12@suse.com>
Date: Wed, 29 May 2024 14:40:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 1/9] x86/irq: remove offline CPUs from old CPU
 mask when adjusting move_cleanup_count
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-2-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240529090132.59434-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 11:01, Roger Pau Monne wrote:
> When adjusting move_cleanup_count to account for CPUs that are offline also
> adjust old_cpu_mask, otherwise further calls to fixup_irqs() could subtract
> those again creating and create an imbalance in move_cleanup_count.

I'm in trouble with "creating"; I can't seem to be able to guess what you may
have meant.

> Fixes: 472e0b74c5c4 ('x86/IRQ: deal with move cleanup count state in fixup_irqs()')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

With the above clarified (adjustment can be done while committing)
Reviewed-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/arch/x86/irq.c
> +++ b/xen/arch/x86/irq.c
> @@ -2572,6 +2572,14 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>              desc->arch.move_cleanup_count -= cpumask_weight(affinity);
>              if ( !desc->arch.move_cleanup_count )
>                  release_old_vec(desc);
> +            else
> +                /*
> +                 * Adjust old_cpu_mask to account for the offline CPUs,
> +                 * otherwise further calls to fixup_irqs() could subtract those
> +                 * again and possibly underflow the counter.
> +                 */
> +                cpumask_and(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
> +                            &cpu_online_map);
>          }

While functionality-wise okay, imo it would be slightly better to use
"affinity" here as well, so that even without looking at context beyond
what's shown here there is a direct connection to the cpumask_weight()
call. I.e.

                cpumask_andnot(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
                               affinity);

Thoughts?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 12:46:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 12:46:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731876.1137627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIh1-0000hT-PZ; Wed, 29 May 2024 12:46:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731876.1137627; Wed, 29 May 2024 12:46:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIh1-0000hM-Mq; Wed, 29 May 2024 12:46:11 +0000
Received: by outflank-mailman (input) for mailman id 731876;
 Wed, 29 May 2024 12:46:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCIgz-0000hG-Ta
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 12:46:09 +0000
Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com
 [2607:f8b0:4864:20::735])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 68e40220-1db9-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 14:46:04 +0200 (CEST)
Received: by mail-qk1-x735.google.com with SMTP id
 af79cd13be357-794e5e77b8bso12322685a.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 05:46:04 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd39bf3sm466104785a.114.2024.05.29.05.46.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 05:46:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68e40220-1db9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716986763; x=1717591563; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=aFNf5VydKznM4YREguZ+9UnLyeQJGJfw2gjaHbvAWec=;
        b=n/3lRIYmR7TK9os+XGCRBP9AgFT9c9tL/QYnfZNOHSmkWrRT3d8d8BL5f67FK3nUee
         hJCAUzppbKH95tbHCW0ScxfA3YWuBt9AXg/nkS0FbfEjf3EMQqTD3tcbcRo+vMeAEAYJ
         x6RCryL6B/yfoI56FB5scpycaFvHc8bLOeZnw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716986763; x=1717591563;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aFNf5VydKznM4YREguZ+9UnLyeQJGJfw2gjaHbvAWec=;
        b=hV2IdbQCbcRnY8cDO/RjjehA5QILrNtFYGhMNgN7Bn7o3QDPCB2Wrx1+LTksla5ksD
         0oiuKVdrE/3uaAxjpGo72Nop/CZSMWitOm/++nverAr9yCAzjgrxV37IY7ZFDkJweiYt
         FNDZdzZrGr22qZKyzOQ2g7mm36tMaqroy+367lm1T5W+J6A1RfAAcMdrsKT7h+j7Nm+S
         It7wb5ZGMZwR0Kl+Vul6OJlTlTo7J1FJfXckJm/p4D68Ol0Lxt/oaX9ZpGQ8AA4l3Bv2
         RX8F3+dyASKKrc9u/HRj3eXPkCkTSLAWV/VGGEfh4yPXC+fS0as7sJSchQ2fjAzSwJg/
         lPXw==
X-Gm-Message-State: AOJu0YzBbsVxE/gyWBNgvfGvvr+NTYRpfQiD0e73l+blcUhQuDizN5Vy
	I2/EUBiRuborZKpvsDBfMHyQ+2oXoAcb+aA36LhTQ+XjIG4ifx4hPsW5BbICtQM=
X-Google-Smtp-Source: AGHT+IHm6w8Jp8Ddjpkdcix0hcRoRFtcGRl7rBLCzoq+2sGuYXSHTn5haLHWUyW4PicAza0oeeGj7w==
X-Received: by 2002:a05:620a:4089:b0:794:d524:18fc with SMTP id af79cd13be357-794d5241b8dmr565822485a.56.1716986763370;
        Wed, 29 May 2024 05:46:03 -0700 (PDT)
Date: Wed, 29 May 2024 14:46:00 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Dario Faggioli <dfaggioli@suse.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
Message-ID: <ZlcjiOJyAuvyy48n@macbook>
References: <ZlW-ZyFeAzvh3pGG@macbook>
 <a749c066-0c05-47dd-b499-e712e5454033@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a749c066-0c05-47dd-b499-e712e5454033@suse.com>

On Wed, May 29, 2024 at 01:47:09PM +0200, Jürgen Groß wrote:
> On 28.05.24 13:22, Roger Pau Monné wrote:
> > Hello,
> > 
> > When the stop_machine_run() call in cpu_down() fails and calls the CPU
> > notifier CPU_DOWN_FAILED hook the following assert triggers in the
> > scheduling code:
> > 
> > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
> > ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
> > CPU:    0
> > RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> > RFLAGS: 0000000000010093   CONTEXT: hypervisor
> > rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
> > rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
> > rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
> > r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> > r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
> > r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
> > cr3: 00000000574c2000   cr2: 0000000000000000
> > fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> > ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> > Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
> >   fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
> > Xen stack trace from rsp=ffff83203ffffd30:
> >     ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
> >     0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
> >     0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
> >     ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
> >     ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
> >     00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
> >     ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
> >     ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
> >     ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
> >     ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
> >     ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
> >     0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
> >     ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
> >     0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
> >     ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
> >     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> >     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> >     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> >     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> >     0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > Xen call trace:
> >     [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> >     [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
> >     [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
> >     [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
> >     [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
> >     [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
> >     [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
> >     [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
> >     [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
> >     [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
> >     [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
> >     [<ffff82d040324391>] F continue_running+0x5b/0x5d
> > 
> > 
> > ****************************************
> > Panic on CPU 0:
> > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
> > ****************************************
> > 
> > The issue seems to be that since the CPU hasn't been removed, it's
> > still part of prv->initialized and the assert in csched2_free_pdata()
> > called as part of free_cpu_rm_data() triggers.
> > 
> > It's easy to reproduce by substituting the stop_machine_run() call in
> > cpu_down() with an error.
> 
> Could you please give the attached patch a try?

I still get the following assert:

Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
----[ Xen-4.19-unstable  x86_64  debug=y  Not tainted ]----
CPU:    0
RIP:    e008:[<ffff82d040248173>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
RFLAGS: 0000000000010093   CONTEXT: hypervisor
rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
rbp: ffff83203ffffd50   rsp: ffff83203ffffd28   r8:  0000000000000000
r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
cr3: 00000000574c2000   cr2: 0000000000000000
fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
Xen code around <ffff82d040248173> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
 fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
Xen stack trace from rsp=ffff83203ffffd28:
   0000000000000001 ffff83202e029100 ffff82d0404c4430 0000000000000006
   0000000000000000 ffff83203ffffd70 ffff82d040257338 0000000000000000
   0000000000000001 ffff83203ffffda0 ffff82d04021f1e6 ffff82d0404c4628
   ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffde8
   ffff82d04022bb2f ffff83203ffffe10 0000000000000001 0000000000000001
   0000000000000000 ffff83203ffffe10 0000000000000000 ffff82d0405e6500
   ffff83203ffffe00 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
   ffff82d040205464 ffff82d0404c5860 0000000000000001 ffff83202ec5d000
   0000000000000000 ffff83203ffffe48 ffff82d040348bd0 ffff83202e0290d0
   ffff83203ffffe68 ffff82d04020708d ffff83202ec5d1d0 ffff82d0405ce210
   ffff83203ffffe80 ffff82d0402342a3 ffff82d0405ce200 ffff83203ffffeb0
   ffff82d04023450b 0000000000000000 0000000000007fff ffff82d0405d5080
   ffff82d0405ce210 ffff83203ffffee8 ffff82d040321363 ffff82d0403212eb
   ffff83202f3a9000 0000000000000000 00000014b1552bff ffff82d0405e6500
   ffff83203ffffde0 ffff82d0403242e3 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000000
Xen call trace:
   [<ffff82d040248173>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
   [<ffff82d040257338>] F free_cpu_rm_data+0x48/0x80
   [<ffff82d04021f1e6>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
   [<ffff82d04022bb2f>] F notifier_call_chain+0x6c/0x96
   [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
   [<ffff82d040205464>] F cpu_down+0x60/0x83
   [<ffff82d040348bd0>] F cpu_down_helper+0x11/0x27
   [<ffff82d04020708d>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
   [<ffff82d0402342a3>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
   [<ffff82d04023450b>] F do_tasklet+0x5b/0x8d
   [<ffff82d040321363>] F arch/x86/domain.c#idle_loop+0x78/0xe6
   [<ffff82d0403242e3>] F continue_running+0x5b/0x5d


****************************************
Panic on CPU 0:
Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
****************************************

Reboot in five seconds...
Resetting with ACPI MEMORY or I/O RESET_REG.

I have the following bodge to trigger the failure:

diff --git a/xen/common/cpu.c b/xen/common/cpu.c
index d76f80fe2e99..b38796038e31 100644
--- a/xen/common/cpu.c
+++ b/xen/common/cpu.c
@@ -126,6 +126,7 @@ int cpu_down(unsigned int cpu)
     if ( err )
         goto fail;

+    goto fail;
     if ( system_state < SYS_STATE_active || system_state == SYS_STATE_resume )
         on_selected_cpus(cpumask_of(cpu), _take_cpu_down, NULL, true);
     else if ( (err = stop_machine_run(take_cpu_down, NULL, cpu)) < 0 )

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:03:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731886.1137637 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIy4-0003so-BV; Wed, 29 May 2024 13:03:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731886.1137637; Wed, 29 May 2024 13:03:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIy4-0003sh-7w; Wed, 29 May 2024 13:03:48 +0000
Received: by outflank-mailman (input) for mailman id 731886;
 Wed, 29 May 2024 13:03:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCIy2-0003sX-Vg; Wed, 29 May 2024 13:03:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCIy2-00073X-U0; Wed, 29 May 2024 13:03:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCIy2-0005KN-JC; Wed, 29 May 2024 13:03:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCIy2-00006R-Im; Wed, 29 May 2024 13:03:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MsuOXwE9om0UbZttxCMfL9cpXXAtztT/KLUqkHtFSwo=; b=2B7dnnI0J+QnM9D8WACk+wOXsE
	69te9CPDHZWSlux9l509OghE1Ve8LLSXlGtnbfHvDyRw2KcjY2NM9fU238DBBlCfqe7++WmxHV124
	GM7MIh04h7bi/0Mw0758QpZFdDPDkzBIv/2dLcdRKFzGeyxxy95xuFmeGH28yWM+MJP4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186181-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186181: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2bc462f645824a439879190ed19a231cb5b8034c
X-Osstest-Versions-That:
    xen=d27c2835e0005200d464b57156c76455d46f74bb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 13:03:46 +0000

flight 186181 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186181/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2bc462f645824a439879190ed19a231cb5b8034c
baseline version:
 xen                  d27c2835e0005200d464b57156c76455d46f74bb

Last test of basis   186172  2024-05-28 17:01:57 Z    0 days
Testing same since   186181  2024-05-29 08:00:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Sergiy Kibrik <Sergiy_Kibrik@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d27c2835e0..2bc462f645  2bc462f645824a439879190ed19a231cb5b8034c -> smoke


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:04:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:04:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731891.1137647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIyY-0004IT-K9; Wed, 29 May 2024 13:04:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731891.1137647; Wed, 29 May 2024 13:04:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCIyY-0004IM-H0; Wed, 29 May 2024 13:04:18 +0000
Received: by outflank-mailman (input) for mailman id 731891;
 Wed, 29 May 2024 13:04:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCIyX-0004Fb-Mw
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:04:17 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3cded9b-1dbb-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 15:04:16 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a6266ffdba8so202585966b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:04:15 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda6e1esm712714066b.215.2024.05.29.06.04.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:04:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3cded9b-1dbb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716987855; x=1717592655; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=4SmJBgQj4kkfRJe3YSFEaksOdM/6dp7tHx5/c/VQHWk=;
        b=CFRJqlQKVdv5uA1HWP17udkjKIg3gUje8dbweH09Gh38FYp31c38qfW3ayyIL92v5N
         e2cwBKgQ5gCyWYStJGcqBBce2R0FNGBIl151RzBpWTaoPTAdSuBXeeJwkAluPkmmMWkD
         6Wuqloqc3fSi3UM+sTbZ6yjfBaBZLju7hmy8Pz7ef3T00/aZNOjqsx12PBMDMFjgGXv8
         H7PAtc0JUKBzRrJQg0op8GwVriMzDGWM0FjjaYYZUD+kKmfY8syQGN0t/c3dp8vN8Ilv
         QA/AqWl2iveGy05KutNreEt5aWuqV9CV2s1x+xh6FVEwGXAFt29+Bg8ZUCVznNvpXK1f
         f1zw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716987855; x=1717592655;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4SmJBgQj4kkfRJe3YSFEaksOdM/6dp7tHx5/c/VQHWk=;
        b=mrbpQTBFQDcY8hoZxqyqYHuVFnH2BhAXBqMxl1VQO76m97xlzfYJSU4rXDpO2W8iMU
         emvMwIylVY0bTjwNsdnHOzF44kZMHK9sI+ZGU4x+TK6mWDp4ikxwD4ibYwpDcI/7arhc
         v5+Lr5liJoyr3GwIA9SdZGp9lnvbJofQb2WnddMWaK/B3TdmaFGut2u10ND5O2I2NQDh
         o+hsXaL0Ehp5hhEfdkLYzvkP+ZPu3v9yCV9X3MXLLxxvtJCP51xvmurE/htOBXpFbbiM
         uEkKwc4uxW0BoMYHu8Hwc4y28FNGorY85ktMM4yo0RyoI8OFhtUOIBZx/H0zEIiEX34O
         0hBg==
X-Forwarded-Encrypted: i=1; AJvYcCX+K4vtmwk8EMFZ4vyAD/9M4u3Q0mV6mtbkNg97lxeFznLx3IX5y9nJX288S7xvN3NphCkwrpZkiDBA3xYvFKVShMzZwhiU2EZ1Deh/6D8=
X-Gm-Message-State: AOJu0YxKJag7hRNF+BB7nnyn3fiyOKGdbVdmL+3meKP9Zj/pc81xhWRR
	BsNRNO0QLOufjCkchpoRGQXAiJcrUNA67jIzTXMn+OgAhdda65TICcM83NHd3w==
X-Google-Smtp-Source: AGHT+IE1sVyl/sF+eassBEV2kWTS3dPGbvgm8eHXxo2pIh4lOChr6BRGAJF2+9CO9FOxJ43OUF7i7g==
X-Received: by 2002:a17:906:a404:b0:a60:4a8b:b4a0 with SMTP id a640c23a62f3a-a62641931d5mr1116351966b.4.1716987855144;
        Wed, 29 May 2024 06:04:15 -0700 (PDT)
Message-ID: <7954b3c2-e101-4db0-b317-a6585a6a8689@suse.com>
Date: Wed, 29 May 2024 15:04:13 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 2/9] xen/cpu: do not get the CPU map in
 stop_machine_run()
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-3-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240529090132.59434-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 11:01, Roger Pau Monne wrote:
> The current callers of stop_machine_run() outside of init code already have the
> CPU maps locked, and hence there's no reason for stop_machine_run() to attempt
> to lock again.

While purely from a description perspective this is okay, ...

> --- a/xen/common/stop_machine.c
> +++ b/xen/common/stop_machine.c
> @@ -82,9 +82,15 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
>      BUG_ON(!local_irq_is_enabled());
>      BUG_ON(!is_idle_vcpu(current));
>  
> -    /* cpu_online_map must not change. */
> -    if ( !get_cpu_maps() )
> +    /*
> +     * cpu_online_map must not change.  The only two callers of
> +     * stop_machine_run() outside of init code already have the CPU map locked.
> +     */

... the "two" here is not unlikely to quickly go stale; who knows what PPC
and RISC-V will have as their code becomes more complete?

I'm also unconvinced that requiring ...

> +    if ( system_state >= SYS_STATE_active && !cpu_map_locked() )

... this for all future (post-init) uses of stop_machine_run() is a good
idea. It is quite a bit more natural, to me at least, for the function to
effect this itself, as is the case prior to your change.

In fact I'm puzzled by Arm's (init-time) use: They use it twice, for errata
workaround and feature enabling. They do that after bringing up secondary
CPUs, but still from start_xen(). How's that going to work for CPUs brought
offline and then back online later on? __cpu_up() isn't __init, so there's
no obvious sign that soft-{off,on}lining isn't possible on Arm. Cc-ing Arm
maintainers.

Jan

> +    {
> +        ASSERT_UNREACHABLE();
>          return -EBUSY;
> +    }
>  
>      nr_cpus = num_online_cpus();
>      if ( cpu_online(this) )
> @@ -92,10 +98,7 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
>  
>      /* Must not spin here as the holder will expect us to be descheduled. */
>      if ( !spin_trylock(&stopmachine_lock) )
> -    {
> -        put_cpu_maps();
>          return -EBUSY;
> -    }
>  
>      stopmachine_data.fn = fn;
>      stopmachine_data.fn_data = data;
> @@ -136,8 +139,6 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
>  
>      spin_unlock(&stopmachine_lock);
>  
> -    put_cpu_maps();
> -
>      return ret;
>  }
>  
> diff --git a/xen/include/xen/cpu.h b/xen/include/xen/cpu.h
> index e1d4eb59675c..d8c8264c58b0 100644
> --- a/xen/include/xen/cpu.h
> +++ b/xen/include/xen/cpu.h
> @@ -13,6 +13,8 @@ void put_cpu_maps(void);
>  void cpu_hotplug_begin(void);
>  void cpu_hotplug_done(void);
>  
> +bool cpu_map_locked(void);
> +
>  /* Receive notification of CPU hotplug events. */
>  void register_cpu_notifier(struct notifier_block *nb);
>  



From xen-devel-bounces@lists.xenproject.org Wed May 29 13:08:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:08:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731898.1137658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJ31-0005SW-64; Wed, 29 May 2024 13:08:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731898.1137658; Wed, 29 May 2024 13:08:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJ31-0005SP-22; Wed, 29 May 2024 13:08:55 +0000
Received: by outflank-mailman (input) for mailman id 731898;
 Wed, 29 May 2024 13:08:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U3iI=NA=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sCJ30-0005SJ-8Q
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:08:54 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 98339781-1dbc-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 15:08:51 +0200 (CEST)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-42011507a54so5459265e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:08:51 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-42100ef1207sm210330775e9.10.2024.05.29.06.08.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:08:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98339781-1dbc-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716988131; x=1717592931; darn=lists.xenproject.org;
        h=in-reply-to:from:content-language:references:cc:to:subject
         :user-agent:mime-version:date:message-id:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HN44IP/EPgAWNx8R+O45r0VzJVTWAPmHTxoncaAiZd8=;
        b=eU/62mSddXZo6c7l/XnYMhlTHtJufWcM55jHVYP67K1qO/xq/h3nCwaUCA2y9lZz7r
         W79O9oEMPKZlPP/+mPHOe1z92hn3VbJjc+I9ZOXqPOofXrYRKAqTO0vlZ10UstLILsUZ
         fYUNzbh7rh0tZp7nj2Fu8Cveas/F34d4S0EKEp9mEubCR7Gl3Wa87dZ86k8KG7u9nlsu
         fwUxT7zCSVsKBSd53BY/bQrQ2vpHVwU/8hb8rgiMRYOJqtoSXqfQIUG1R80KKTunXtGX
         5g2Fx1iA/VITkRUGshytAYKI+DUzslYmFBKI3fkH4dQEYoheVwHxRgf/Cmgp7TXSC+Qs
         vrSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716988131; x=1717592931;
        h=in-reply-to:from:content-language:references:cc:to:subject
         :user-agent:mime-version:date:message-id:x-gm-message-state:from:to
         :cc:subject:date:message-id:reply-to;
        bh=HN44IP/EPgAWNx8R+O45r0VzJVTWAPmHTxoncaAiZd8=;
        b=HB4Bu2xIZ2Cduthi5MYG3gWCtsgNMVa3FFoVTOFn84OfQ+vGdvwegtIyykx7xemGAS
         OC7xW6P9TgXFkTGndZsp3pRM4Gl8sUVWYkhGP+WiIguC03F8d2GYXXEEXidpOjLEHCF7
         ykyay2F2ZAa6icSISsxFfjTltpbXuKTwsQiofs271l+8puRx/WngvRleHdrokyEBtUYa
         jL1JYemgqllY9wOPMpInzUNERM3WTtbFxFI4YjZL44bMfeN/fkxwUCwffeYjdEATZfXa
         M301/wrh66oYFyf4bUojmg9AXbv6zHbpieHHN2/EC806WCpvQAgmfWXCL5DdQ5oiBaAL
         1U6g==
X-Gm-Message-State: AOJu0YzGp3YdfQ9K3zSMMOVXn3uLpX40hjsuynB/4Esg6iUF848EU5Qx
	0T0dIsVEuXMxjBag9W0Vk8M573ag0N9ubpQpJZ8WUgU+OVCWOaTVt1ZGdXz7YuA=
X-Google-Smtp-Source: AGHT+IH6L0dHonFoKdM5gCBtBtrmW0k+WnkhPRhOmMeF8pG2RLfTAazcTPM8W75g12Z+/bHkETRa9w==
X-Received: by 2002:a05:600c:5110:b0:41b:9427:562e with SMTP id 5b1f17b1804b1-42122af30c0mr18867355e9.12.1716988131040;
        Wed, 29 May 2024 06:08:51 -0700 (PDT)
Content-Type: multipart/mixed; boundary="------------2V6G5Eh02X4o3VSnr14rCpbS"
Message-ID: <1147a06d-d9ec-4fa9-a1d0-dfecd7a670dc@suse.com>
Date: Wed, 29 May 2024 15:08:49 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Dario Faggioli <dfaggioli@suse.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ZlW-ZyFeAzvh3pGG@macbook>
 <a749c066-0c05-47dd-b499-e712e5454033@suse.com> <ZlcjiOJyAuvyy48n@macbook>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <ZlcjiOJyAuvyy48n@macbook>

This is a multi-part message in MIME format.
--------------2V6G5Eh02X4o3VSnr14rCpbS
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 29.05.24 14:46, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 01:47:09PM +0200, Jürgen Groß wrote:
>> On 28.05.24 13:22, Roger Pau Monné wrote:
>>> Hello,
>>>
>>> When the stop_machine_run() call in cpu_down() fails and calls the CPU
>>> notifier CPU_DOWN_FAILED hook the following assert triggers in the
>>> scheduling code:
>>>
>>> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
>>> ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
>>> CPU:    0
>>> RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
>>> RFLAGS: 0000000000010093   CONTEXT: hypervisor
>>> rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
>>> rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
>>> rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
>>> r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
>>> r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
>>> r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
>>> cr3: 00000000574c2000   cr2: 0000000000000000
>>> fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
>>> ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>>> Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
>>>    fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
>>> Xen stack trace from rsp=ffff83203ffffd30:
>>>      ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
>>>      0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
>>>      0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
>>>      ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
>>>      ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
>>>      00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
>>>      ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
>>>      ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
>>>      ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
>>>      ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
>>>      ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
>>>      0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
>>>      ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
>>>      0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
>>>      ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
>>>      0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>      0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>      0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>      0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>      0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>> Xen call trace:
>>>      [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
>>>      [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
>>>      [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
>>>      [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
>>>      [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
>>>      [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
>>>      [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
>>>      [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
>>>      [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
>>>      [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
>>>      [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
>>>      [<ffff82d040324391>] F continue_running+0x5b/0x5d
>>>
>>>
>>> ****************************************
>>> Panic on CPU 0:
>>> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
>>> ****************************************
>>>
>>> The issue seems to be that since the CPU hasn't been removed, it's
>>> still part of prv->initialized and the assert in csched2_free_pdata()
>>> called as part of free_cpu_rm_data() triggers.
>>>
>>> It's easy to reproduce by substituting the stop_machine_run() call in
>>> cpu_down() with an error.
>>
>> Could you please give the attached patch a try?
> 
> I still get the following assert:

Oh, silly me. Without core scheduling active nr_sr_unused will be 0 all
the time. :-(

Next try.


Juergen

--------------2V6G5Eh02X4o3VSnr14rCpbS
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-xen-sched-fix-error-path-of-cpu-removal.patch"
Content-Disposition: attachment;
 filename="0001-xen-sched-fix-error-path-of-cpu-removal.patch"
Content-Transfer-Encoding: base64

RnJvbSA4YjcwN2IwZTRkNjhiMTYzNmYxYzVmZjBlMmVmNTRiZWE1ZDk5MjFhIE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+
CkRhdGU6IFdlZCwgMjkgTWF5IDIwMjQgMTM6MjQ6MzYgKzAyMDAKU3ViamVjdDogW1BBVENI
XSB4ZW4vc2NoZWQ6IGZpeCBlcnJvciBwYXRoIG9mIGNwdSByZW1vdmFsCk1JTUUtVmVyc2lv
bjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApDb250ZW50
LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpJbiBjYXNlIHJlbW92YWwgb2YgYSBjcHUgZmFp
bHMgYXQgdGhlIENQVV9EWUlORyBzdGVwLCBhbiBBU1NFUlQoKSBpbgp0aGUgY3JlZGl0MiBz
Y2hlZHVsZXIgbWlnaHQgdHJpZ2dlcjoKCkFzc2VydGlvbiAnIWNwdW1hc2tfdGVzdF9jcHUo
Y3B1LCAmcHJ2LT5pbml0aWFsaXplZCknIGZhaWxlZCBhdCBjb21tb24vc2NoZWQvY3JlZDEK
LS0tLVsgWGVuLTQuMTktdW5zdGFibGUgIHg4Nl82NCAgZGVidWc9eSAgVGFpbnRlZDogICBD
ICAgIF0tLS0tCkNQVTogICAgMApSSVA6ICAgIGUwMDg6WzxmZmZmODJkMDQwMjQ4Mjk5Pl0g
Y29tbW9uL3NjaGVkL2NyZWRpdDIuYyNjc2NoZWQyX2ZyZWVfcGRhdGErMHhjOC8weDE3NwpS
RkxBR1M6IDAwMDAwMDAwMDAwMTAwOTMgICBDT05URVhUOiBoeXBlcnZpc29yCnJheDogMDAw
MDAwMDAwMDAwMDAwMCAgIHJieDogZmZmZjgzMjAyZWNjMmY4MCAgIHJjeDogZmZmZjgzMjAy
ZjNlNjRjMApyZHg6IDAwMDAwMDAwMDAwMDAwMDEgICByc2k6IDAwMDAwMDAwMDAwMDAwMDIg
ICByZGk6IGZmZmY4MzIwMmVjYzJmODgKcmJwOiBmZmZmODMyMDNmZmZmZDU4ICAgcnNwOiBm
ZmZmODMyMDNmZmZmZDMwICAgcjg6ICAwMDAwMDAwMDAwMDAwMDAwCnI5OiAgZmZmZjgzMjAy
ZjNlNmUwMSAgIHIxMDogMDAwMDAwMDAwMDAwMDAwMCAgIHIxMTogMGYwZjBmMGYwZjBmMGYw
ZgpyMTI6IGZmZmY4MzIwMmVjYjgwYjAgICByMTM6IDAwMDAwMDAwMDAwMDAwMDEgICByMTQ6
IDAwMDAwMDAwMDAwMDAyODIKcjE1OiBmZmZmODMyMDJlY2JiZjAwICAgY3IwOiAwMDAwMDAw
MDgwMDUwMDNiICAgY3I0OiAwMDAwMDAwMDAwNzUyNmUwCmNyMzogMDAwMDAwMDA1NzRjMjAw
MCAgIGNyMjogMDAwMDAwMDAwMDAwMDAwMApmc2I6IDAwMDAwMDAwMDAwMDAwMDAgICBnc2I6
IDAwMDAwMDAwMDAwMDAwMDAgICBnc3M6IDAwMDAwMDAwMDAwMDAwMDAKZHM6IDAwMDAgICBl
czogMDAwMCAgIGZzOiAwMDAwICAgZ3M6IDAwMDAgICBzczogMDAwMCAgIGNzOiBlMDA4Clhl
biBjb2RlIGFyb3VuZCA8ZmZmZjgyZDA0MDI0ODI5OT4gKGNvbW1vbi9zY2hlZC9jcmVkaXQy
LmMjY3NjaGVkMl9mcmVlX3BkYXRhKzB4YzgvMHgxNzcpOgogZmUgZmYgZWIgOWEgMGYgMGIg
MGYgMGIgPDBmPiAwYiA0OSA4ZCA0ZiAwOCA0OSA4YiA0NyAwOCA0OCAzYiA0OCAwOCA3NSAy
ZQpYZW4gc3RhY2sgdHJhY2UgZnJvbSByc3A9ZmZmZjgzMjAzZmZmZmQzMDoKICAgZmZmZjgz
MjAyZDc0ZDEwMCAwMDAwMDAwMDAwMDAwMDAxIGZmZmY4MmQwNDA0YzQ0MzAgMDAwMDAwMDAw
MDAwMDAwNgogICAwMDAwMDAwMDAwMDAwMDAwIGZmZmY4MzIwM2ZmZmZkNzggZmZmZjgyZDA0
MDI1NzQ1NCAwMDAwMDAwMDAwMDAwMDAwCiAgIDAwMDAwMDAwMDAwMDAwMDEgZmZmZjgzMjAz
ZmZmZmRhOCBmZmZmODJkMDQwMjFmMzAzIGZmZmY4MmQwNDA0YzQ2MjgKICAgZmZmZjgyZDA0
MDRjNDYyMCBmZmZmODJkMDQwNGM0NDMwIDAwMDAwMDAwMDAwMDAwMDYgZmZmZjgzMjAzZmZm
ZmRmMAogICBmZmZmODJkMDQwMjJiYzRjIGZmZmY4MzIwM2ZmZmZlMTggMDAwMDAwMDAwMDAw
MDAwMSAwMDAwMDAwMDAwMDAwMDAxCiAgIDAwMDAwMDAwZmZmZmZmZjAgMDAwMDAwMDAwMDAw
MDAwMCAwMDAwMDAwMDAwMDAwMDAwIGZmZmY4MmQwNDA1ZTY1MDAKICAgZmZmZjgzMjAzZmZm
ZmUwOCBmZmZmODJkMDQwMjA0ZmQ1IDAwMDAwMDAwMDAwMDAwMDEgZmZmZjgzMjAzZmZmZmUz
MAogICBmZmZmODJkMDQwMjA1NGYwIGZmZmY4MmQwNDA0YzU4NjAgMDAwMDAwMDAwMDAwMDAw
MSBmZmZmODMyMDJlYzc1MDAwCiAgIGZmZmY4MzIwM2ZmZmZlNDggZmZmZjgyZDA0MDM0OGMy
NSBmZmZmODMyMDJkNzRkMGQwIGZmZmY4MzIwM2ZmZmZlNjgKICAgZmZmZjgyZDA0MDIwNzFh
YSBmZmZmODMyMDJlYzc1MWQwIGZmZmY4MmQwNDA1Y2UyMTAgZmZmZjgzMjAzZmZmZmU4MAog
ICBmZmZmODJkMDQwMjM0M2M5IGZmZmY4MmQwNDA1Y2UyMDAgZmZmZjgzMjAzZmZmZmViMCBm
ZmZmODJkMDQwMjM0NjMxCiAgIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwN2ZmZiBm
ZmZmODJkMDQwNWQ1MDgwIGZmZmY4MmQwNDA1Y2UyMTAKICAgZmZmZjgzMjAzZmZmZmVlOCBm
ZmZmODJkMDQwMzIxNDExIGZmZmY4MmQwNDAzMjEzOTkgZmZmZjgzMjAyZjNhOTAwMAogICAw
MDAwMDAwMDAwMDAwMDAwIDAwMDAwMDFkOTFhNmZhMmQgZmZmZjgyZDA0MDVlNjUwMCBmZmZm
ODMyMDNmZmZmZGUwCiAgIGZmZmY4MmQwNDAzMjQzOTEgMDAwMDAwMDAwMDAwMDAwMCAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAKICAgMDAwMDAwMDAwMDAwMDAwMCAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMAogICAwMDAw
MDAwMDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwCiAgIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAKICAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMAogICAwMDAwMDAw
MDAwMDAwMDAwIDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMDAw
MDAwMDAwClhlbiBjYWxsIHRyYWNlOgogICBbPGZmZmY4MmQwNDAyNDgyOTk+XSBSIGNvbW1v
bi9zY2hlZC9jcmVkaXQyLmMjY3NjaGVkMl9mcmVlX3BkYXRhKzB4YzgvMHgxNzcKICAgWzxm
ZmZmODJkMDQwMjU3NDU0Pl0gRiBmcmVlX2NwdV9ybV9kYXRhKzB4NDEvMHg1OAogICBbPGZm
ZmY4MmQwNDAyMWYzMDM+XSBGIGNvbW1vbi9zY2hlZC9jcHVwb29sLmMjY3B1X2NhbGxiYWNr
KzB4ZmIvMHg0NjYKICAgWzxmZmZmODJkMDQwMjJiYzRjPl0gRiBub3RpZmllcl9jYWxsX2No
YWluKzB4NmMvMHg5NgogICBbPGZmZmY4MmQwNDAyMDRmZDU+XSBGIGNvbW1vbi9jcHUuYyNj
cHVfbm90aWZpZXJfY2FsbF9jaGFpbisweDFiLzB4MzYKICAgWzxmZmZmODJkMDQwMjA1NGYw
Pl0gRiBjcHVfZG93bisweGE3LzB4MTQzCiAgIFs8ZmZmZjgyZDA0MDM0OGMyNT5dIEYgY3B1
X2Rvd25faGVscGVyKzB4MTEvMHgyNwogICBbPGZmZmY4MmQwNDAyMDcxYWE+XSBGIGNvbW1v
bi9kb21haW4uYyNjb250aW51ZV9oeXBlcmNhbGxfdGFza2xldF9oYW5kbGVyKzB4NTAvMHhi
ZAogICBbPGZmZmY4MmQwNDAyMzQzYzk+XSBGIGNvbW1vbi90YXNrbGV0LmMjZG9fdGFza2xl
dF93b3JrKzB4NzYvMHhhZgogICBbPGZmZmY4MmQwNDAyMzQ2MzE+XSBGIGRvX3Rhc2tsZXQr
MHg1Yi8weDhkCiAgIFs8ZmZmZjgyZDA0MDMyMTQxMT5dIEYgYXJjaC94ODYvZG9tYWluLmMj
aWRsZV9sb29wKzB4NzgvMHhlNgogICBbPGZmZmY4MmQwNDAzMjQzOTE+XSBGIGNvbnRpbnVl
X3J1bm5pbmcrMHg1Yi8weDVkCgpUaGlzIGNhbiBiZSBmaXhlZCBieSBub3QgZnJlZWluZyB0
aGUgY3B1J3Mgc2NoZWR1bGluZyBkYXRhIGluIGNhc2UgdGhlCkNQVV9EWUlORyBzdGVwIGhh
c24ndCBiZWVuIHBlcmZvcm1lZC4gSW5zdGVhZCB0aGUgbm90IHVzZWQgc3RydWN0CnNjaGVk
X3Jlc291cmNlIGluc3RhbmNlcyBuZWVkIHRvIGJlIGZyZWVkIGluIG9yZGVyIHRvIGF2b2lk
IG1lbW9yeQpsZWFraW5nLgoKRml4ZXM6IGQ4NDQ3MzY4OTYxMSAoInhlbi9zY2hlZDogZml4
IGNwdSBob3RwbHVnIikKUmVwb3J0ZWQtYnk6IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2VyLnBh
dUBjaXRyaXguY29tPgpTaWduZWQtb2ZmLWJ5OiBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3Vz
ZS5jb20+Ci0tLQogeGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMgICAgfCAxOCArKysrKysrKysr
KysrKysrLS0KIHhlbi9jb21tb24vc2NoZWQvcHJpdmF0ZS5oIHwgIDEgKwogMiBmaWxlcyBj
aGFuZ2VkLCAxNyBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBh
L3hlbi9jb21tb24vc2NoZWQvY29yZS5jIGIveGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMKaW5k
ZXggZDg0YjY1ZjE5Ny4uMTk5NzkzZmM5ZSAxMDA2NDQKLS0tIGEveGVuL2NvbW1vbi9zY2hl
ZC9jb3JlLmMKKysrIGIveGVuL2NvbW1vbi9zY2hlZC9jb3JlLmMKQEAgLTMyNjQsNiArMzI2
NCw4IEBAIHN0cnVjdCBjcHVfcm1fZGF0YSAqYWxsb2NfY3B1X3JtX2RhdGEodW5zaWduZWQg
aW50IGNwdSwgYm9vbCBhZmZfYWxsb2MpCiAgICAgICAgIGRhdGEtPnNyW2lkeF0tPnNjaGVk
dWxlX2xvY2sgPSBzci0+c2NoZWR1bGVfbG9jazsKICAgICB9CiAKKyAgICBkYXRhLT5uX3Ny
X3VudXNlZCA9IGlkeCArIDE7CisKICBvdXQ6CiAgICAgcmN1X3JlYWRfdW5sb2NrKCZzY2hl
ZF9yZXNfcmN1bG9jayk7CiAKQEAgLTMyNzIsOCArMzI3NCwxOCBAQCBzdHJ1Y3QgY3B1X3Jt
X2RhdGEgKmFsbG9jX2NwdV9ybV9kYXRhKHVuc2lnbmVkIGludCBjcHUsIGJvb2wgYWZmX2Fs
bG9jKQogCiB2b2lkIGZyZWVfY3B1X3JtX2RhdGEoc3RydWN0IGNwdV9ybV9kYXRhICptZW0s
IHVuc2lnbmVkIGludCBjcHUpCiB7Ci0gICAgc2NoZWRfZnJlZV91ZGF0YShtZW0tPm9sZF9v
cHMsIG1lbS0+dnByaXZfb2xkKTsKLSAgICBzY2hlZF9mcmVlX3BkYXRhKG1lbS0+b2xkX29w
cywgbWVtLT5wcHJpdl9vbGQsIGNwdSk7CisgICAgdW5zaWduZWQgaW50IGlkeDsKKworICAg
IGlmICggIW1lbS0+bl9zcl91bnVzZWQgKQorICAgIHsKKyAgICAgICAgc2NoZWRfZnJlZV91
ZGF0YShtZW0tPm9sZF9vcHMsIG1lbS0+dnByaXZfb2xkKTsKKyAgICAgICAgc2NoZWRfZnJl
ZV9wZGF0YShtZW0tPm9sZF9vcHMsIG1lbS0+cHByaXZfb2xkLCBjcHUpOworICAgIH0KKyAg
ICBlbHNlCisgICAgeworICAgICAgICBmb3IgKCBpZHggPSAwOyBpZHggPCBtZW0tPm5fc3Jf
dW51c2VkIC0gMTsgaWR4KysgKQorICAgICAgICAgICAgc2NoZWRfcmVzX2ZyZWUoJm1lbS0+
c3JbaWR4XS0+cmN1KTsKKyAgICB9CiAgICAgZnJlZV9hZmZpbml0eV9tYXNrcygmbWVtLT5h
ZmZpbml0eSk7CiAKICAgICB4ZnJlZShtZW0pOwpAQCAtMzMxMiw2ICszMzI0LDggQEAgaW50
IHNjaGVkdWxlX2NwdV9ybSh1bnNpZ25lZCBpbnQgY3B1LCBzdHJ1Y3QgY3B1X3JtX2RhdGEg
KmRhdGEpCiAgICAgLyogU2VlIGNvbW1lbnQgaW4gc2NoZWR1bGVfY3B1X2FkZCgpIHJlZ2Fy
ZGluZyBsb2NrIHN3aXRjaGluZy4gKi8KICAgICBvbGRfbG9jayA9IHBjcHVfc2NoZWR1bGVf
bG9ja19pcnFzYXZlKGNwdSwgJmZsYWdzKTsKIAorICAgIGRhdGEtPm5fc3JfdW51c2VkID0g
MDsKKwogICAgIGZvcl9lYWNoX2NwdSAoIGNwdV9pdGVyLCBzci0+Y3B1cyApCiAgICAgewog
ICAgICAgICBwZXJfY3B1KHNjaGVkX3Jlc19pZHgsIGNwdV9pdGVyKSA9IDA7CmRpZmYgLS1n
aXQgYS94ZW4vY29tbW9uL3NjaGVkL3ByaXZhdGUuaCBiL3hlbi9jb21tb24vc2NoZWQvcHJp
dmF0ZS5oCmluZGV4IGMwZTdjOTZkMjQuLmYwMTI5ZmQ5Y2MgMTAwNjQ0Ci0tLSBhL3hlbi9j
b21tb24vc2NoZWQvcHJpdmF0ZS5oCisrKyBiL3hlbi9jb21tb24vc2NoZWQvcHJpdmF0ZS5o
CkBAIC02NDUsNiArNjQ1LDcgQEAgc3RydWN0IGNwdV9ybV9kYXRhIHsKICAgICBjb25zdCBz
dHJ1Y3Qgc2NoZWR1bGVyICpvbGRfb3BzOwogICAgIHZvaWQgKnBwcml2X29sZDsKICAgICB2
b2lkICp2cHJpdl9vbGQ7CisgICAgdW5zaWduZWQgaW50IG5fc3JfdW51c2VkOwogICAgIHN0
cnVjdCBzY2hlZF9yZXNvdXJjZSAqc3JbXTsKIH07CiAKLS0gCjIuMzUuMwoK

--------------2V6G5Eh02X4o3VSnr14rCpbS--


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:22:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:22:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731908.1137666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJFs-0000fY-BM; Wed, 29 May 2024 13:22:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731908.1137666; Wed, 29 May 2024 13:22:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJFs-0000fR-8n; Wed, 29 May 2024 13:22:12 +0000
Received: by outflank-mailman (input) for mailman id 731908;
 Wed, 29 May 2024 13:22:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vA4L=NA=gmail.com=yskelg@srs-se1.protection.inumbo.net>)
 id 1sCJFq-0000fL-KT
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:22:10 +0000
Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com
 [2607:f8b0:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 73200855-1dbe-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 15:22:09 +0200 (CEST)
Received: by mail-pl1-x62d.google.com with SMTP id
 d9443c01a7336-1f44b594deeso15478065ad.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:22:09 -0700 (PDT)
Received: from paran-QEMU-Virtual-Machine.. ([118.32.98.101])
 by smtp.gmail.com with ESMTPSA id
 d9443c01a7336-1f44c760a21sm99183205ad.4.2024.05.29.06.22.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 06:22:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73200855-1dbe-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716988928; x=1717593728; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=0Kur8IddoyK7auUB0DpsJfjxMjtBncw932qUbr08xEk=;
        b=UOYXeg3gOzC7/rpJqFKx4f8bohFdAcQSXZRp/ZXQ3qDoWm1cIIbvKIlWZKbCTGt6OD
         MoDR+jH9Snrk49d7p+wanMLPezE/MWg1eCxYuugyRk4zb5aGjO0Qblv/FlyubdMIyeGa
         Wf83s7zPbA/arjuio93z3AOUBr22UzUnYaGsF2PDI4chfW7F78FG0Qvym1GypT3hjVca
         a842IRyEqyj/S7HRJvd8Uy9XSLhk4BvpeioiKM3rORDZqhNbaKuOfNd1nW+Bcv2P5KZq
         d6oCjVJaFTIvWSBQAXFSsZWobjOSK6YLPirvkF+VKvjTRzaN7XvbougFRxi+GHbmr6qt
         LaNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716988928; x=1717593728;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0Kur8IddoyK7auUB0DpsJfjxMjtBncw932qUbr08xEk=;
        b=s7cj++RyKoIIR5oXhK8LsascEaXXCWDzzzN1a+FTyBKPK3VrriN7+QVn+sBaj2wW+y
         P/JcVesE3qk/Emx1Y3XZraonQgssZ8tJtvFtVmp2yKI1pyHRQRI84Meyh6e7YJgDY8zo
         k+PsImiDkqA50zUjDvM+mVDZNxTUeDzqNWUY0W2hZj5P9DMpNweTZOjnMKC2IteIjSmB
         RCuADrf5yjiJpKyeYZs7A+VunSVOEG4a4jPDutyyQeBbefkl7exVaCvROM76Qkt5heFs
         KOfeP2M7a4aM3ZzTrTasm6wnFYMdGgp93+JqqwV8rOTaw9SHFB5GbwoBgl6m7hMEZD08
         Px8w==
X-Forwarded-Encrypted: i=1; AJvYcCXgKW8MjuhrMNXvqpQ9VOls7pkU100/0UMd6uZNr6775mXEvB8Cyc9VI5dJT0W7j3W7xSfm/yBOa3VLYVL2aEWTSY5MQKzecfi65pLX17I=
X-Gm-Message-State: AOJu0YwL5h6OgXp6hCykIpl9/1aZj0UIamJUZtxTTH9bGRqFZRv0T27n
	5F5v57NBAego4pyfwLYW0ncYhq0Ba6Jh5KF6j3gTr8Lkda1eLVZo
X-Google-Smtp-Source: AGHT+IEsBv5bIxgTY/q47JQPTYogoxqv6LI2GlfX+nm0L8UY7lG45NwaKZRAtWholBUpgq2B4dfkIg==
X-Received: by 2002:a17:902:e84f:b0:1f4:990c:5ef1 with SMTP id d9443c01a7336-1f4990c5fecmr96953905ad.31.1716988927615;
        Wed, 29 May 2024 06:22:07 -0700 (PDT)
From: yskelg@gmail.com
To: Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: skhan@linuxfoundation.org,
	sj@kernel.org,
	Austin Kim <austindh.kim@gmail.com>,
	shjy180909@gmail.com,
	linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-kernel-mentees@lists.linuxfoundation.org,
	Yunseong Kim <yskelg@gmail.com>
Subject: [PATCH] xen/xenbus: handle pointer to NULL in alloc/free_pdev
Date: Wed, 29 May 2024 22:19:28 +0900
Message-Id: <20240529131926.29590-1-yskelg@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Yunseong Kim <yskelg@gmail.com>

Modify 'alloc_pdev()' to set 'pdev->xdev' to NULL
if 'xen_pcibk_init_devices()' fails. This ensures that 'pdev->xdev' does
not point to 'xdev' when 'pdev' is freed.
And modify 'free_pdev()' to set 'pdev' to NULL.

Signed-off-by: Yunseong Kim <yskelg@gmail.com>
---
 drivers/xen/xen-pciback/xenbus.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index b11e401f1b1e..f1709b8e284a 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -54,6 +54,7 @@ static struct xen_pcibk_device *alloc_pdev(struct xenbus_device *xdev)
 	INIT_WORK(&pdev->op_work, xen_pcibk_do_op);
 
 	if (xen_pcibk_init_devices(pdev)) {
+		pdev->xdev = NULL;
 		kfree(pdev);
 		pdev = NULL;
 	}
@@ -102,6 +103,7 @@ static void free_pdev(struct xen_pcibk_device *pdev)
 	pdev->xdev = NULL;
 
 	kfree(pdev);
+	pdev = NULL;
 }
 
 static int xen_pcibk_do_attach(struct xen_pcibk_device *pdev, int gnt_ref,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 13:28:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:28:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731913.1137676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJMH-0001GO-0B; Wed, 29 May 2024 13:28:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731913.1137676; Wed, 29 May 2024 13:28:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJMG-0001GH-Tu; Wed, 29 May 2024 13:28:48 +0000
Received: by outflank-mailman (input) for mailman id 731913;
 Wed, 29 May 2024 13:28:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U3iI=NA=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sCJMF-0001GB-4i
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:28:47 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 601acb56-1dbf-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 15:28:46 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-52aea6067c8so802262e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:28:46 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579d7276d95sm3726512a12.78.2024.05.29.06.28.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:28:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 601acb56-1dbf-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716989325; x=1717594125; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gXmr6j/tvonQwkWDxBgodd8VSD2h8lNR4zZeNwuG1Go=;
        b=UTvwRpNuNy/KP6pChpMwQDUORi6+BxJGlj9NpCZ2VW2NxqoCyP3MeZQc4eflWa8a3V
         BXAi1k5q2REgFBLphOCl20x/aee9cqf31iclWDzygS1Rb11Ofpz13C+QlcyMRS6QY3iY
         8cyVXfD89CuJuBDFzkFmqjjDVv9i/4K4PQEKLVcSPMoS6ykOSyBHe0yqizocghwSDCWT
         tMiebrGZ1EuDNnYLdSWF98ePpUnjJQlS2B7c+hZ7bT+hrgMUwWrB5hC1NJRkwNHHuwAm
         kL36ZpCy7sv1/fpzRlT+XfUYabsAsWZuDHWKK5Nmx/S6D25HBxJPUbtWReFJyObM4oxA
         iyuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716989325; x=1717594125;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gXmr6j/tvonQwkWDxBgodd8VSD2h8lNR4zZeNwuG1Go=;
        b=l2UHBm04caYXqRCx7QhxaBaOtLaXJhfg4plS82yKIGlim5e2ZkDFNlVUjCPgGNJzOR
         WoxZh1MfNPYkIP80VvcOFNoxl3RLR9UzFFSThszDLzfXPL3+YrHGuc6TWp/g91YfPFFe
         Uiov1YQQH3cfBPS4kHpgs4FmAifrx2dDod/gNE0eh1D175y/rMtR1jjVFwnBiLXHCl8m
         Yl5g+eIj0OzLUx+S5xlWlI9/9vtkYyVE62j6spCSt47lucdLDdY0tFmmcC12CgfuuP7t
         hskqlq+sQLZ/Va4XpN3RvXd3+GG6rloS8i5kVWfNJan+Vlr9fPZzYcLGjd/wqXDQ9BI5
         x0Dw==
X-Forwarded-Encrypted: i=1; AJvYcCUb8drI2Mf31CdiJ+RfARXOEZaYEahEWp27H0+x2Xfol/yv+A9+QJpcmxApn1fNmPxlzqIkD7XGkRVpogHRs6nWqlFZpW/jk5p4gVOCOPc=
X-Gm-Message-State: AOJu0YyW49By3Wjoz+dl67Q9SRAnAqQWdb8la/OIvLaDColPzYAtdkLt
	YWo41tiUKLL6QNoxNB0I17KvpUekNMQxMnR166syQFDFPH/0ypQQPUvwRjyILz4=
X-Google-Smtp-Source: AGHT+IEq5J35jp5oyjSaT4U2EpNtDbrlLRiwcYvKIzo9K+RuuN07fUdxYFnLzwmPTjwExZKBrEuQig==
X-Received: by 2002:a05:6512:1042:b0:529:a389:f6a6 with SMTP id 2adb3069b0e04-529a389f747mr9715451e87.67.1716989325376;
        Wed, 29 May 2024 06:28:45 -0700 (PDT)
Message-ID: <2095a4a2-7db5-4de6-9609-d8e6a5adf27f@suse.com>
Date: Wed, 29 May 2024 15:28:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/xenbus: handle pointer to NULL in alloc/free_pdev
To: yskelg@gmail.com, Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: skhan@linuxfoundation.org, sj@kernel.org,
 Austin Kim <austindh.kim@gmail.com>, shjy180909@gmail.com,
 linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
 linux-kernel-mentees@lists.linuxfoundation.org
References: <20240529131926.29590-1-yskelg@gmail.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240529131926.29590-1-yskelg@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 29.05.24 15:19, yskelg@gmail.com wrote:
> From: Yunseong Kim <yskelg@gmail.com>
> 
> Modify 'alloc_pdev()' to set 'pdev->xdev' to NULL
> if 'xen_pcibk_init_devices()' fails. This ensures that 'pdev->xdev' does
> not point to 'xdev' when 'pdev' is freed.
> And modify 'free_pdev()' to set 'pdev' to NULL.
> 
> Signed-off-by: Yunseong Kim <yskelg@gmail.com>
> ---
>   drivers/xen/xen-pciback/xenbus.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index b11e401f1b1e..f1709b8e284a 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -54,6 +54,7 @@ static struct xen_pcibk_device *alloc_pdev(struct xenbus_device *xdev)
>   	INIT_WORK(&pdev->op_work, xen_pcibk_do_op);
>   
>   	if (xen_pcibk_init_devices(pdev)) {
> +		pdev->xdev = NULL;
>   		kfree(pdev);
>   		pdev = NULL;
>   	}
> @@ -102,6 +103,7 @@ static void free_pdev(struct xen_pcibk_device *pdev)
>   	pdev->xdev = NULL;
>   
>   	kfree(pdev);
> +	pdev = NULL;
>   }
>   
>   static int xen_pcibk_do_attach(struct xen_pcibk_device *pdev, int gnt_ref,

NAK!

Please stop that nonsense. NULL-ing pointers which have no chance to be used any
more is just adding code for no purpose at all.

Please don't send other iterations of this patch. You are wasting review
bandwidth.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:35:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:35:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731921.1137688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJSP-0003MT-LV; Wed, 29 May 2024 13:35:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731921.1137688; Wed, 29 May 2024 13:35:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJSP-0003MM-H0; Wed, 29 May 2024 13:35:09 +0000
Received: by outflank-mailman (input) for mailman id 731921;
 Wed, 29 May 2024 13:35:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCJSN-0003MG-Lu
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:35:07 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 42c81c58-1dc0-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 15:35:06 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a630ff4ac84so235989166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:35:06 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8b980sm715700266b.154.2024.05.29.06.35.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:35:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42c81c58-1dc0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716989706; x=1717594506; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=EV46/AZY/HKbhP5b1+XxO0OUC3ytXActtv4A1/6Stek=;
        b=TiHmx3MtSpgndvIiIHzQkRLqDovHm+f44qAOLxkLebcXN3CRl3Vu6Zx0i3Ow1uxDfY
         kPZTbBg+Uv27lGYUGaYcjA4TXihdNoxAacPV9ebJuUl0s3DLjG4/W2bGsKkrkd9GLKWU
         R6PepH8IEUkbsb6SmyguaTa0vQMwAtsYCD5A2E9f3VPM1eCVKhqBSD6sTOvhzFC6BBdf
         UlcwQUYRrX1M5ybG/ywtKUfNyW0ppSNTmHm/9h7X4/xcOO/LVZVSrJIgseNwaBMAV1LY
         XMohvrC4m6ia4xd+unRAm7KhYWiBO/q8e/Kdp8tO7q5TJ1S3VwCJQPtmSe8z+26Ox7co
         o8cQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716989706; x=1717594506;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EV46/AZY/HKbhP5b1+XxO0OUC3ytXActtv4A1/6Stek=;
        b=bRva5MXpBAwkN+lUntIbVSmR2H5JRTc0bVxEkiIbbrop+AHZWPjCFL61MNR2+BY+G/
         PcuWU84SIdXX0dJzaIEE0PF0X34NQY5+ih8VatIolfihLQ5baDWn3px7mCqX8zx8PA5L
         GOBP0nOU1j0ubmnU9HzkceymwLCH++GRDgDiAArfc3+08sG+2uILWtH1ADVYzSmh0Ahw
         mtyDNqFQeVlBJ5qj6AAfR1TiUaBRbgkguW2WvBbVC4cziu/+E/XAFPGw41OUEvN+GJVh
         86BhbRgtuPvPsqAV/QYBNoSXukUvBlpbMSANAY2wGzSNsv/xEx4klS4vjom5qTGC8fQ4
         G5Sw==
X-Forwarded-Encrypted: i=1; AJvYcCXGigg2kXdm/bfkFOUroVWo06cx0k1y41gdvdf+g4rtMF3KyJV3Nj6PPJu49tHrYfBUnJ/W9SSOLTfIo0bw1LQAAWb/vTd0piEFxG/fmJE=
X-Gm-Message-State: AOJu0Yxmdcy9PSrV+6MvjN7+0xMGFyxQQNmKxLzfBR383qySxX66z4Zi
	/oAsneNlRDg6AOetrGrG8B8ZyLk0KENVDNG/XqssH4EJO2lmooxiKOQEbyZciA==
X-Google-Smtp-Source: AGHT+IHy4uAKWHKXXq+m7FmTz1cT01vmXB4tmbtDy1RvjWBXyhq0hdfVR9gs4tr+6wQX6D4+UKzVNw==
X-Received: by 2002:a17:906:1c12:b0:a63:359b:e5bf with SMTP id a640c23a62f3a-a63359be69cmr324786666b.42.1716989705772;
        Wed, 29 May 2024 06:35:05 -0700 (PDT)
Message-ID: <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>
Date: Wed, 29 May 2024 15:35:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns false
 if CPU operations are underway
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240529090132.59434-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 11:01, Roger Pau Monne wrote:
> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
> a cpu_hotplug_{begin,done}() region the function will still return success,
> because a CPU taking the rwlock in read mode after having taken it in write
> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
> as it should always return false when called with a CPU hot{,un}plug operation
> is in progress.

I'm not sure I can agree with this. The CPU doing said operation ought to be
aware of what it is itself doing. And all other CPUs will get back false from
get_cpu_maps().

>  Otherwise the logic in send_IPI_mask() for example is wrong,
> as it could decide to use the shorthand even when a CPU operation is in
> progress.

It's also not becoming clear what's wrong there: As long as a CPU isn't
offline enough to not be in cpu_online_map anymore, it may well need to still
be the target of IPIs, and targeting it with a shorthand then is still fine.

In any event this would again affect only the CPU leading the CPU operation,
which should clearly know at which point(s) it is okay to send IPIs. Are we
actually sending any IPIs from within CPU-online or CPU-offline paths?
Together with the earlier paragraph the critical window would be between the
CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
then the question would be what bad, if any, would happen to that CPU if an
IPI was still targeted at it by way of using the shorthand. I'm pretty sure
it runs with IRQs off at that time, so no ordinary IRQ could be delivered.

> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
> already hold in write mode by the current CPU, as read_trylock() would
> otherwise return true.
> 
> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')

I'm puzzled by this as well: Prior to that and the change referenced by its
Fixes: tag, recursive spin locks were used. For the purposes here that's the
same as permitting read locking even when the write lock is already held by
the local CPU.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:36:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:36:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731925.1137697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJTk-0003sG-U2; Wed, 29 May 2024 13:36:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731925.1137697; Wed, 29 May 2024 13:36:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJTk-0003s9-R1; Wed, 29 May 2024 13:36:32 +0000
Received: by outflank-mailman (input) for mailman id 731925;
 Wed, 29 May 2024 13:36:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vA4L=NA=gmail.com=yskelg@srs-se1.protection.inumbo.net>)
 id 1sCJTj-0003rz-Mz
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:36:31 +0000
Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com
 [2607:f8b0:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 74a9d63d-1dc0-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 15:36:30 +0200 (CEST)
Received: by mail-pl1-x634.google.com with SMTP id
 d9443c01a7336-1f4a52b9589so18285965ad.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:36:30 -0700 (PDT)
Received: from [192.168.50.95] ([118.32.98.101])
 by smtp.gmail.com with ESMTPSA id
 d9443c01a7336-1f5e6a0c323sm7448965ad.259.2024.05.29.06.36.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:36:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74a9d63d-1dc0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716989789; x=1717594589; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dicpLSmGeHTxR8brlcEnN0cI1QEPfhfEJAkQtcJYjXg=;
        b=i0IwzJFNGucVXcWsUDt9wk8k21tDhJoykjLHu7HMxRqvuFhExZ7MJWkQ9t7PcNJINI
         G3CON6CgCGHGhmI5fPeLILnTitYHAF5xLd/E0L8EsZFzReStKOjQRCTtH6bAE53RkOYw
         vCQ4J7FRLEwqyjXkD8LRqD+0e0B+k4O0Xdmt7KVc/KgXCAmMFolHMYFuI1hNKHprswKC
         OTh3yGJauxrJvjVxVb9ZZ97P9a24yL2QNbRlm+inNRdx6deTCMCQu+nZKIsrrvsAfSGw
         H+MLBL1f98iJCok9A2+YI42mTB9f8DxsCH39dpLtABzKRTmamw4iHC/LI8Tc/Lh9oZUU
         8BMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716989789; x=1717594589;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dicpLSmGeHTxR8brlcEnN0cI1QEPfhfEJAkQtcJYjXg=;
        b=oH/LDWYq+bFyy+mWy10POuKn/5wXzUywjIXE5wdDyWQqxvd+jIcBdvkuRaqQcdl1gu
         v9hpWKaPrQNb43GQF9cjZ0EaR76c4A19BeA/h1Eb/bvITBYb3OiwXqUf32JIhGg0xpt0
         mJEweO2WlXaUx6SMMDnAADi4nP0j3JohKnMkmnzF1MH9VlqCIIncXXjD8BNrYVnKiB51
         S/5wRu7gdCGSQJ24tWYCym4A2LjrI//hYXGH4QqOxwoTndtDyJ5G5xm+0ah6arqhl9mM
         YQXUgRDZyIaylRwjCsmT3/ssNxicNgEC3p5SDCS+SngE03ISSVFGiGTA/n7arzrqDOZe
         jYPQ==
X-Forwarded-Encrypted: i=1; AJvYcCV0Pl+9cIhGURq4Y386IB7Dp8/TUOir1kcBAINLkaCKmJjjnRE7QG1+GagMoWSxZiRKkzZXuJbdIFadnBLlMNtdoP9US3hM4Ys3Dei02DY=
X-Gm-Message-State: AOJu0Yy/wx6RP0skodSLVcHaK7Kswpq+/qMJCPpns8+DlgpI+t5EhEct
	PYsGOX/CGh41faadDkUHk6afHpMtagOU0Mex4A0wKVdvz39ticF0
X-Google-Smtp-Source: AGHT+IG2bzrds8yi4l23vZDAB9yvZEHtwj8ed98GsFY012BXS9ao17xB0hQkMahe4hP7gPU0TMZLBA==
X-Received: by 2002:a17:902:ce87:b0:1f4:8a83:a2dc with SMTP id d9443c01a7336-1f48a83a580mr95544275ad.10.1716989789220;
        Wed, 29 May 2024 06:36:29 -0700 (PDT)
Message-ID: <12ac8cc2-3117-4cb5-ab3a-2e8add3270a6@gmail.com>
Date: Wed, 29 May 2024 22:36:24 +0900
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/xenbus: handle potential dangling pointer issue in
 xen_pcibk_xenbus_probe
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: skhan@linuxfoundation.org, sj@kernel.org,
 Austin Kim <austindh.kim@gmail.com>, shjy180909@gmail.com,
 linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
 linux-kernel-mentees@lists.linuxfoundation.org
References: <20240529122232.25360-1-yskelg@gmail.com>
 <0c53415d-3394-455a-a6b5-99d3efb92318@suse.com>
Content-Language: en-US
From: Yunseong Kim <yskelg@gmail.com>
In-Reply-To: <0c53415d-3394-455a-a6b5-99d3efb92318@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit



On 5/29/24 9:27 오후, Jürgen Groß wrote:
> On 29.05.24 14:22, yskelg@gmail.com wrote:
>> From: Yunseong Kim <yskelg@gmail.com>
>>
>> If 'xen_pcibk_init_devices()' fails. This ensures that 'pdev->xdev' does
>> not point to 'xdev' when 'pdev' is freed.
>>
>> Signed-off-by: Yunseong Kim <yskelg@gmail.com>
>> ---
>>   drivers/xen/xen-pciback/xenbus.c | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/xen/xen-pciback/xenbus.c
>> b/drivers/xen/xen-pciback/xenbus.c
>> index b11e401f1b1e..348d6803b8c0 100644
>> --- a/drivers/xen/xen-pciback/xenbus.c
>> +++ b/drivers/xen/xen-pciback/xenbus.c
>> @@ -54,6 +54,7 @@ static struct xen_pcibk_device *alloc_pdev(struct
>> xenbus_device *xdev)
>>       INIT_WORK(&pdev->op_work, xen_pcibk_do_op);
>>         if (xen_pcibk_init_devices(pdev)) {
>> +        pdev->xdev = NULL;
>>           kfree(pdev);
>>           pdev = NULL;
>>       }
> 
> NAK.
> 
> This doesn't make any sense, as pdev is freed.
> 
> 
> Juergen

Thanks for the code review Juergen!

I don't know if it's a big deal, but I think that if pdev->xdev don't
replace it with NULL, 'struct xenbus_device *xdev' address value stays
in the kmalloc cached memory.

And most of the kernel code I've seen does this (even when it's
'free_pdev()'), so I sent a patch.

In the very unlikely event that. I was thinking if someone look at the
value without initializing the kmalloced memory, someone might see the
'xenbus_device *xdev address'.

Warm Regards,

Yunseong Kim


From xen-devel-bounces@lists.xenproject.org Wed May 29 13:51:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731933.1137707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJhp-0007Bx-4v; Wed, 29 May 2024 13:51:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731933.1137707; Wed, 29 May 2024 13:51:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJhp-0007Bp-25; Wed, 29 May 2024 13:51:05 +0000
Received: by outflank-mailman (input) for mailman id 731933;
 Wed, 29 May 2024 13:51:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCJho-0007Bi-Cf
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:51:04 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c9671b0-1dc2-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 15:51:02 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a6265d48ec3so227910666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:51:02 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a631e324816sm257782466b.15.2024.05.29.06.51.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 06:51:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c9671b0-1dc2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716990662; x=1717595462; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=lcxfRb9l0SxIIEy8fUc/kJZbB2DUYx1Jm1uQRc2lLas=;
        b=BJE3rrTnEo+nOpWA32UVg/ujZaxiZi8isota30F07bXV47VXm3MY0s05Y7Q5DabCCF
         9bw3fiL8PJELO5L7kxdcmly4TeD41qlyGoXVr3KT3urXg+GD2STN9vo4bhbk0dPnao4Z
         U14mbXZzDUUMt62gJF7mWC0j+M9r9Ifp6jBIy2SUtVIpXaUi9tSeeBs1dKBMlwHc8kMz
         tMdLiLGu4qoXM1LHq/KB+e+9s04ngGisP+W7xZF9HsiDVSeG6BSJ+S0jEpDRhWbfVQ1U
         LcREwCmx2q91sXrd9VfBZJ/xcW0+wwUD1z7xYtf36SiKiBe5+obb4GrR1/QutH6h7E0E
         7Uhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716990662; x=1717595462;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lcxfRb9l0SxIIEy8fUc/kJZbB2DUYx1Jm1uQRc2lLas=;
        b=js6I+ctlPJh5cR5gvQq+sU1ppxjuYnEmvxV54BXkq2Ggq276d3TKCrkwly0w0794FP
         ZCKaRLUhEaPo7amrxSrR/HsS6KxeAzU788BNgbKLjlaewio6mWIwNJe8Xt2br/uE+GA5
         5psYjcpJKQRI1y1opmAqNexqRbpqX83hAw8H2k8Dhy51au0xWEYhGzbzVBU7Lwgytlob
         EjC2O5uhXH/j5y+CrNzy4qn4ZFnsbSZO0wM54YZTa8UsTSmO6din7tgKEdN+r31sc7jj
         gYKnQXMTk7IaekwmP7CP/9bPON2y3OIf7UwIJbNXEwkCOpcxqkVQBjNuz8fOU0TrTrn6
         j0wA==
X-Forwarded-Encrypted: i=1; AJvYcCWqLYGVS5txCUc1RP+9XJEC4CQ0k+V64vVwbtxxWOvp7/W17YI86N+U/s5fPAF9MYvHykMpWIgcwRKqxExPRfAIawyJrHOsTEg4r/ikKw8=
X-Gm-Message-State: AOJu0YzZ7gnZBi/8rfYriVaamwNX8akvViA8mA0Zez5oIS8RzHxQ6khr
	UrvRmROXM+MhtEc22RRPadPFYHm/bczNW4WvW2NAMucJfKOw0PLI0HJ49ou5
X-Google-Smtp-Source: AGHT+IF76O2EbZ/e4NtRXa4J1Q+hGUtBLuQcf+wZpEN/SoLT5PkQGaiBBmEk9R2Y02jVybSnvH8XbA==
X-Received: by 2002:a17:907:82a5:b0:a59:d39a:8d65 with SMTP id a640c23a62f3a-a62641b9de4mr886178466b.21.1716990661645;
        Wed, 29 May 2024 06:51:01 -0700 (PDT)
Message-ID: <2129c69d4d53c666218e1d3fe3ca2cd5af7edf42.camel@gmail.com>
Subject: Re: [PATCH] Partial revert of "x86/MCE: optional build of AMD/Intel
 MCE code"
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>, Sergiy Kibrik <Sergiy_Kibrik@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>
Date: Wed, 29 May 2024 15:51:00 +0200
In-Reply-To: <20240529103752.1431366-1-andrew.cooper3@citrix.com>
References: <20240529103752.1431366-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 11:37 +0100, Andrew Cooper wrote:
> {cmci,lmce}_support are written during S3 resume, so cannot live in
> __ro_after_init.=C2=A0 Move them back to being __read_mostly, as they wer=
e
> originally.
>=20
> Link: https://gitlab.com/xen-project/xen/-/jobs/6966698361
> Fixes: 19b6e9f9149f ("x86/MCE: optional build of AMD/Intel MCE code")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> We're past feature freeze and this was a silent change in a patch,
> which was
> also untested.=C2=A0 A 30s look at mcheck_init() shows clearly that it's
> not a 30s
> job to fix.
> ---
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
> =C2=A0xen/arch/x86/cpu/mcheck/mce.c | 4 ++--
> =C2=A01 file changed, 2 insertions(+), 2 deletions(-)
>=20
> diff --git a/xen/arch/x86/cpu/mcheck/mce.c
> b/xen/arch/x86/cpu/mcheck/mce.c
> index 1664ca6412ac..32c1b2756b90 100644
> --- a/xen/arch/x86/cpu/mcheck/mce.c
> +++ b/xen/arch/x86/cpu/mcheck/mce.c
> @@ -38,10 +38,10 @@ DEFINE_PER_CPU_READ_MOSTLY(unsigned int,
> nr_mce_banks);
> =C2=A0unsigned int __read_mostly firstbank;
> =C2=A0unsigned int __read_mostly ppin_msr;
> =C2=A0uint8_t __read_mostly cmci_apic_vector;
> -bool __ro_after_init cmci_support;
> +bool __read_mostly cmci_support;
> =C2=A0
> =C2=A0/* If mce_force_broadcast =3D=3D 1, lmce_support will be disabled
> forcibly. */
> -bool __ro_after_init lmce_support;
> +bool __read_mostly lmce_support;
> =C2=A0
> =C2=A0DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, poll_bankmask);
> =C2=A0DEFINE_PER_CPU_READ_MOSTLY(struct mca_banks *, no_cmci_banks);
>=20
> base-commit: 0840bc5ea114f536a4bdfb2ca095b79f2069aae6



From xen-devel-bounces@lists.xenproject.org Wed May 29 13:57:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 13:57:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731939.1137716 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJnw-0008FA-Qv; Wed, 29 May 2024 13:57:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731939.1137716; Wed, 29 May 2024 13:57:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJnw-0008F3-OR; Wed, 29 May 2024 13:57:24 +0000
Received: by outflank-mailman (input) for mailman id 731939;
 Wed, 29 May 2024 13:57:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCJnv-0008Es-B2
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 13:57:23 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5e372253-1dc3-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 15:57:21 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a59a609dd3fso177196566b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 06:57:20 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6519c92796sm55866066b.104.2024.05.29.06.57.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 06:57:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e372253-1dc3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716991040; x=1717595840; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xmEslsxWpEHWD6AFlN1vPdTyzF2JumoRvAwOUyq0Ru4=;
        b=GvqAKcmM6Cu9ieRVdvHJRleqQMWdQxamUT0TXDa6N/AFOFKLTCT1DjgwEDreb04Nkt
         BTLgUWBwgZ5ijWjo0Xt0kht5lyHYYVv4IlDtWzzYqUAxIGZUzylH3+jXNdqZ2RL8SIO9
         QhczC2dSmMmtARMt3C5PwBI1QLMdZh3qmoHKuQJfuhz32Yei7U8cLorOSgjK5P21yudY
         Q056GeRecal/xEEfTLsQH1XEks/sLZaL5FZRxDAKnyyp3XC3z4cSIqv+0r7Qx+WjylWD
         ZaEq/EPWMgk9NEqQkmIaPVFQR2HjW1+l29E8NCVnHW6yjuaGDSm/bJ6MHzvxtMN5mQQM
         rYmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991040; x=1717595840;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xmEslsxWpEHWD6AFlN1vPdTyzF2JumoRvAwOUyq0Ru4=;
        b=PSSYQa+qcN3fpnR2svgODvWrqwbx6tdfQs/5+9IdwKTcl16d0mI4cMslkfOxtHk03q
         SbRkUP2v9cBqS5jbIg6j40vKo8A0rlVHVqpe7FtHKzEzBu5oao7zoJev3ebkHlj0q2gk
         j2KzEPOgFAvcVSNSKdAlp6c9jMXcIUVRoC9bQ9UfTiaPEyGuYogFzI1LmNLxfmE5/yVP
         cvd80NxHqMEe1p3IvPY9sYcvQv0y0gTWFNRKbajUQjD1ksFiUgOBeyNCV6eiCIliN4Us
         T28jMiwybNFnnRhkMivKTkWIZwpHEyB0tIH9gMtMC5CSfDy32F9dDiKttT8TC+Bngo4Z
         etVg==
X-Forwarded-Encrypted: i=1; AJvYcCWU1PA9N4LHXyst5qFFURfqIx6sZbDFk4N9zH0n4qUKpKZe6AJR7BZZECJZStuRJZlUbHd1+4yDZtJq0MHlTFVm8GoU/KScpuUfxxKonxg=
X-Gm-Message-State: AOJu0YzQiSeN2/hkmYFiTavXMBGEUuW92Mz3veqJKfuGCHEeaIXjWW0f
	1iNo5KXngnxTpIOjwMvmljWC1W+82z6Z+zeUyXwo5jp3ASZ05DxYFE94BIS6Cg==
X-Google-Smtp-Source: AGHT+IFYlPgOKjtH0HyCJcDYtpUkGShmrz98Xy/y0bgiE2xXv0mO1PttIwdXO0xebnyGPvxAyf1x3w==
X-Received: by 2002:a17:906:af0e:b0:a59:beb4:62b8 with SMTP id a640c23a62f3a-a642d5a388bmr189558266b.21.1716991040376;
        Wed, 29 May 2024 06:57:20 -0700 (PDT)
Message-ID: <24201381-2a43-427e-a3e5-1de316d212b8@suse.com>
Date: Wed, 29 May 2024 15:57:19 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 4/9] x86/irq: describe how the interrupt CPU
 movement works
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-5-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240529090132.59434-5-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 11:01, Roger Pau Monne wrote:
> --- a/xen/arch/x86/include/asm/irq.h
> +++ b/xen/arch/x86/include/asm/irq.h
> @@ -28,6 +28,32 @@ typedef struct {
>  
>  struct irq_desc;
>  
> +/*
> + * Xen logic for moving interrupts around CPUs allows manipulating interrupts
> + * that target remote CPUs.  The logic to move an interrupt from CPU(s) is as
> + * follows:
> + *
> + * 1. cpu_mask and vector is copied to old_cpu_mask and old_vector.
> + * 2. New cpu_mask and vector are set, vector is setup at the new destination.
> + * 3. move_in_progress is set.
> + * 4. Interrupt source is updated to target new CPU and vector.
> + * 5. Interrupts arriving at old_cpu_mask are processed normally.
> + * 6. When an interrupt is delivered at the new destination (cpu_mask) as part
> + *    of acking the interrupt move_in_progress is cleared and move_cleanup_count

Nit: A comma after "interrupt" may help reading.

> + *    is set to the weight of online CPUs in old_cpu_mask.
> + *    IRQ_MOVE_CLEANUP_VECTOR is sent to all CPUs in old_cpu_mask.

These last two steps aren't precise enough, compared to what the code does.
old_cpu_mask is first reduced to online CPUs therein. If the result is non-
empty, what you describe is done. If, however, the result is empty, the
vector is released right away (this code may be there just in case, but I
think it shouldn't be omitted here).

> + * 7. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the
> + *    vector entry and decrease the count in move_cleanup_count.  The CPU that
> + *    sets move_cleanup_count to 0 releases the vector.
> + *
> + * Note that when interrupt movement (either move_in_progress or
> + * move_cleanup_count set) is in progress it's not possible to move the
> + * interrupt to yet a different CPU.
> + *
> + * By keeping the vector in the old CPU(s) configured until the interrupt is
> + * acked on the new destination Xen allows draining any pending interrupts at
> + * the old destinations.
> + */
>  struct arch_irq_desc {
>          s16 vector;                  /* vector itself is only 8 bits, */
>          s16 old_vector;              /* but we use -1 for unassigned  */

I take it that it is not a goal to (also) describe under what conditions
an IRQ move may actually be initiated (IRQ_MOVE_PENDING)? I ask not the
least because the 2nd from last paragraph lightly touches that area.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 14:02:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:02:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731945.1137727 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJt9-0001qH-FJ; Wed, 29 May 2024 14:02:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731945.1137727; Wed, 29 May 2024 14:02:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJt9-0001qA-BE; Wed, 29 May 2024 14:02:47 +0000
Received: by outflank-mailman (input) for mailman id 731945;
 Wed, 29 May 2024 14:02:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tF9l=NA=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1sCJt7-0001pt-HJ
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:02:45 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e35a604-1dc4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:02:43 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-52ac0c4b62cso892771e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:02:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e35a604-1dc4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716991362; x=1717596162; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=ZYzRlzTLzEjtaMe1Xvod3sHOTprCiFQ7vhrNGjOUgJw=;
        b=i+8ISP2umVv4RqNej59Ru6T4bmD+FLZNaSYQdV0rJFl5ROXhW7VCIRUzWyGh2xjVkk
         WgOf9L0OcMYxa7j2uaf8/EKvwSKF6+yiDGKbnaSMpnC+3UbbHB7BXjHmsTcZ5XKK7T7o
         yuiQYio2VlyyCOWA232tE1yN5fqkUfVkm5akw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991362; x=1717596162;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ZYzRlzTLzEjtaMe1Xvod3sHOTprCiFQ7vhrNGjOUgJw=;
        b=vbKc21QJtwv/ho7LjfIskMRsBTarqnevMeZCmfKOshPHU2xv6by2tk7hsKGIJTTOzu
         EVzNbocrHbB9SAgpgRzFgglR/RJ7UX8bmztVjSKfsJVN8x9m6r8jEHigrFZmxa4uWHDM
         gHz3RPttTsjSqwl8TUeJ1eNgXZFCUEcEblIUQvGCagz6c1hYfg8ZdVBWkOOtROZsxLAK
         blDDbW+6eQ6cNYgZyO2u8epaHBFkd0ZJXuzBhCfSX/fkSTdFG57E6UDL7NaWbAOS2Q1+
         O4wNq2OMaxYgOIQ2VXuA4mq4Kgx13Dm+26bn48+EfpL7u0KgzedINQux+I7TYmBe2kOn
         4Upg==
X-Gm-Message-State: AOJu0Ywd0rBG4Eq922Gnuj5+la2hSNmRVs6+3wNmSCQtFNRv/fLqAosC
	/98WW3RZDANQb6V19r6jzLOK6oaKHHVZnA8JvRDs36c+gS125U3X5VhyHw8BOI65vx8pAZ0ukel
	D/7zTOZZUrKt7hxLahD2EU/fojkOIb3AxwcmqNvPXMtB09IjZ
X-Google-Smtp-Source: AGHT+IGaZRiwUIWmLWB9uh5JxA0BzSTXgoQlGWOpGvePkNO4yaTDjiC7u5vIYIE5ALkU9qUOQSWgNqqn54W6h8keMbw=
X-Received: by 2002:ac2:4833:0:b0:51e:1bed:13a8 with SMTP id
 2adb3069b0e04-52964caa1e1mr8378853e87.29.1716991361899; Wed, 29 May 2024
 07:02:41 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 29 May 2024 15:02:05 +0100
Message-ID: <CAO-mL=w0OCCnt+x_8oyv4MvZ-SjcN1M5sGUJzU-buK1vusgyUw@mail.gmail.com>
Subject: [ANNOUNCE] Join us for free virtually at Xen Summit 2024!
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Cc: committers@xenproject.org, advisory-board@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000035af2506199834af"

--00000000000035af2506199834af
Content-Type: text/plain; charset="UTF-8"

Hello Xen Community,

Join us virtually at Xen Summit (4-6th June), for free using Jitsi!

We want to encourage as many of you to participate in the Xen Summit as
possible. Just because you are not physically attending, doesn't mean you
can't get involved.
*(Please note, there will be no professional AV or audio equipment, and it
is a community effort to enable others to get involved.) *

On the day, please find the *schedule of talks listed here.*
<https://events.linuxfoundation.org/xen-project-summit/program/schedule/>

*Instructions:*

   - Our team will help host the main presentation talk sessions using Jitsi

   - Either myself or one attendee volunteer will help host each design
   session using Jitsi
   - Ensure you are using the Lisbon timezone
   - All keynotes and session presentation talks will be streamed using:
      - *SESSION PRESENTATIONS LINK <https://meet.jit.si/XenSummit24Main>*.
   - Two design sessions will happen simultaneously (from Wednesday, June
   5th @ 13:45).
   Please look out on the schedule for the time and whether the session
   you'd like to attend is:
      - *DESIGN SESSION A <https://meet.jit.si/XenDesignSessionsA> *(Liberdade
      Room)
      - *DESIGN SESSION B <https://meet.jit.si/XenDesignSessionsB> *(Augusta
      Room)


   - Thursday design sessions will be finalized on the schedule by the end
   of day on Wednesday
   - The same links will be used throughout talks and sessions
   - (Optional) Join our Xen Summit matrix channel for updates on the day:
   https://matrix.to/#/#xen-project-summit:matrix.org

*Some ground rules to follow:*

   - Enter your full name on Jitsi so everyone knows who you are
   - Please mute yourself upon joining
   - Turning on cameras is optional, but we encourage doing this for design
   sessions
   - Do *not* shout out your questions during session presentations,
   instead ask these on the chat function and we will do our best to ask on
   behalf of you
   - During design sessions, we encourage you to unmute and participate
   freely
   - If multiple people wish to speak, please use the 'raise hand' function
   on Jitsi or chat
   - Should there be a need, moderators will have permission to remove
   anyone who is disruptive in sessions on Jitsi
   - If you face issues on the day, please let us know via Matrix - we will
   do our best to help, but please note this is a community effort

*Jitsi links:*

Session presentation link:

https://meet.jit.si/XenSummit24Main

Design Session A (Liberdade Room) link:
https://meet.jit.si/XenDesignSessionsA

Design Session B (Augusta Room) link: https://meet.jit.si/XenDesignSessionsB

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=XenSummit24Main

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenSummit24Main#config.startSilent=true

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=XenDesignSessionsA

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenDesignSessionsA#config.startSilent=true

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=XenDesignSessionsB

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenDesignSessionsB#config.startSilent=true

Schedule Example:

Tuesday 4th & Wednesday 5th June 2024

(Lisbon timezone)

Schedule Example: Wednesday 5th & Thursday 6th June 2024

(Lisbon timezone)

Schedule Example:  Wednesday 5th & Thursday 6th June 2024

(Lisbon timezone)

09:00

Welcome & Opening Remarks - Kelly Choi, Community Manager, Cloud Software
Group, XenServer

LIBERDADE I

09:10

Xen Project 2024 Weather Report - Kelly Choi, XenServer, Cloud Software
Group.

LIBERDADE I

13:45

The future of Xen Project physical events
<https://design-sessions.xenproject.org/uid/discussion/disc_OlJce1uK3uI0OjS2cL55/view>

LIBERDADE I

14:35

IOMMU paravirtualization and Xen IOMMU subsystem rework
<https://design-sessions.xenproject.org/uid/discussion/disc_PEgBNIXMyEkdJoysE27O/view>

LIBERDADE I

13:45

Downstream working group
<https://design-sessions.xenproject.org/uid/discussion/disc_z78Lt2EIZt2qxaS3FSlQ/view>

AUGUSTA I

14:35

Xen Safety Requirements upstreaming
<https://design-sessions.xenproject.org/uid/discussion/disc_01JE9EOi9zxxAU8lfB6Z/view>

AUGUSTA I



*How to filter the schedule:*



We look forward to seeing you there, both virtually and physically!

Many thanks,
Kelly Choi

Community Manager
Xen Project

--00000000000035af2506199834af
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello Xen Community,=C2=A0<div><br></div><div>Join us virt=
ually at Xen Summit (4-6th June), for free using=C2=A0<span>Jitsi</span>!=
=C2=A0</div><div><br></div><div>We want to encourage as many of you to part=
icipate in the Xen Summit as possible. Just because you are not physically =
attending, doesn&#39;t mean you can&#39;t get involved.=C2=A0</div><div><i>=
(Please note, there will be no professional AV or audio equipment, and it i=
s a community effort to enable others to get involved.)=C2=A0</i></div><div=
><br></div><div>On the day, please find the=C2=A0<a href=3D"https://events.=
linuxfoundation.org/xen-project-summit/program/schedule/" target=3D"_blank"=
><b>schedule=C2=A0of talks listed here.</b></a></div><div><br></div><div><b=
><u>Instructions:</u></b></div><div><ul><li style=3D"margin-left:15px">Our =
team will help host the main presentation talk sessions using=C2=A0<span>Ji=
tsi</span>=C2=A0</li><li style=3D"margin-left:15px">Either myself or one at=
tendee volunteer will help host each design session using=C2=A0<span>Jitsi<=
/span>=C2=A0</li><li style=3D"margin-left:15px">Ensure you are using the Li=
sbon timezone</li><li style=3D"margin-left:15px">All keynotes and session p=
resentation talks will be streamed using:</li><ul><li style=3D"margin-left:=
15px"><b><a href=3D"https://meet.jit.si/XenSummit24Main" target=3D"_blank">=
SESSION PRESENTATIONS LINK</a></b>.=C2=A0</li></ul><li style=3D"margin-left=
:15px">Two design sessions will happen simultaneously (from Wednesday, June=
 5th @ 13:45).<br>Please look out on the schedule for the time and whether =
the session you&#39;d like to attend is:=C2=A0</li><ul><li style=3D"margin-=
left:15px"><b><a href=3D"https://meet.jit.si/XenDesignSessionsA" target=3D"=
_blank">DESIGN SESSION A</a>=C2=A0</b>(Liberdade Room)=C2=A0</li><li style=
=3D"margin-left:15px"><b><a href=3D"https://meet.jit.si/XenDesignSessionsB"=
 target=3D"_blank">DESIGN SESSION=C2=A0B</a>=C2=A0</b>(Augusta Room)</li></=
ul></ul><ul><li style=3D"margin-left:15px">Thursday design sessions will be=
 finalized on the schedule by the end of day on Wednesday</li><li style=3D"=
margin-left:15px">The same links will be used throughout talks and sessions=
</li><li style=3D"margin-left:15px">(Optional) Join our Xen Summit matrix c=
hannel for updates on the day:=C2=A0<a href=3D"https://matrix.to/#/%23xen-p=
roject-summit:matrix.org" target=3D"_blank">https://matrix.to/#/#xen-projec=
t-summit:matrix.org</a>=C2=A0=C2=A0</li></ul><div><div><b><u>Some ground ru=
les to follow:</u></b></div><div><ul><li style=3D"margin-left:15px">Enter y=
our full name on Jitsi so everyone knows who you are</li><li style=3D"margi=
n-left:15px">Please mute yourself upon joining=C2=A0</li><li style=3D"margi=
n-left:15px">Turning on cameras is optional, but we encourage doing this fo=
r design sessions</li><li style=3D"margin-left:15px">Do <u>not</u> shout ou=
t your questions during session presentations, instead ask these on the cha=
t function and we will do our best to ask on behalf of you</li><li style=3D=
"margin-left:15px">During design sessions, we encourage you to unmute and p=
articipate freely</li><li style=3D"margin-left:15px">If multiple people wis=
h to speak, please use the &#39;raise hand&#39; function on=C2=A0<span>Jits=
i or chat</span></li><li style=3D"margin-left:15px">Should there be a need,=
 moderators will have permission to remove anyone who is disruptive in sess=
ions on=C2=A0<span>Jitsi</span></li><li style=3D"margin-left:15px"><span>If=
 you face issues on the day, please let us know via Matrix - we will do our=
 best to help, but please note this is a community effort=C2=A0</span></li>=
</ul><div><b><u>Jitsi links:</u></b></div></div></div></div><div><span id=
=3D"m_3017200869748370048gmail-docs-internal-guid-f9217878-7fff-07f7-1772-7=
8aa156120b2"><div dir=3D"ltr" style=3D"margin-left:0pt" align=3D"left"><tab=
le style=3D"border:none;border-collapse:collapse"><colgroup><col width=3D"2=
08"><col width=3D"208"><col width=3D"208"></colgroup><tbody><tr style=3D"he=
ight:58.5pt"><td style=3D"border-width:0.681818pt;border-style:solid;border=
-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D=
"ltr" style=3D"line-height:1.728;margin-top:0pt;margin-bottom:0pt"><span st=
yle=3D"font-size:11pt;font-family:Arial,sans-serif;color:rgb(0,0,0);backgro=
und-color:transparent;font-weight:700;font-variant-numeric:normal;font-vari=
ant-east-asian:normal;font-variant-alternates:normal;vertical-align:baselin=
e">Session presentation link:</span></p><p dir=3D"ltr" style=3D"line-height=
:1.728;margin-top:0pt;margin-bottom:0pt"><a href=3D"https://meet.jit.si/Xen=
Summit24Main" style=3D"text-decoration-line:none" target=3D"_blank"><span s=
tyle=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transp=
arent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:n=
ormal;font-variant-alternates:normal;text-decoration-line:underline;vertica=
l-align:baseline">https://meet.jit.si/XenSummit24Main</span></a><span style=
=3D"font-size:11pt;font-family:Arial,sans-serif;color:rgb(0,0,0);background=
-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant=
-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline">=
=C2=A0</span></p></td><td style=3D"border-width:0.681818pt;border-style:sol=
id;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden">=
<p dir=3D"ltr" style=3D"line-height:1.728;margin-top:0pt;margin-bottom:0pt"=
><span style=3D"font-size:11pt;font-family:Arial,sans-serif;color:rgb(0,0,0=
);background-color:transparent;font-weight:700;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">Design Session A (Liberdade Room) link: </span><a href=3D"https=
://meet.jit.si/XenDesignSessionsA" style=3D"text-decoration-line:none" targ=
et=3D"_blank"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;ba=
ckground-color:transparent;font-weight:700;font-variant-numeric:normal;font=
-variant-east-asian:normal;font-variant-alternates:normal;text-decoration-l=
ine:underline;vertical-align:baseline">https://meet.jit.si/XenDesignSession=
sA</span></a></p></td><td style=3D"border-width:0.681818pt;border-style:sol=
id;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden">=
<p dir=3D"ltr" style=3D"line-height:1.728;margin-top:0pt;margin-bottom:0pt"=
><span style=3D"font-size:11pt;font-family:Arial,sans-serif;color:rgb(0,0,0=
);background-color:transparent;font-weight:700;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">Design Session B (Augusta Room) link: </span><a href=3D"https:/=
/meet.jit.si/XenDesignSessionsB" style=3D"text-decoration-line:none" target=
=3D"_blank"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;back=
ground-color:transparent;font-weight:700;font-variant-numeric:normal;font-v=
ariant-east-asian:normal;font-variant-alternates:normal;text-decoration-lin=
e:underline;vertical-align:baseline">https://meet.jit.si/XenDesignSessionsB=
</span></a></p></td></tr><tr style=3D"height:171.75pt"><td style=3D"border-=
width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:=
top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;=
margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family=
:Arial,sans-serif;background-color:transparent;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">See meeting dial-in numbers: </span><a href=3D"https://meet.jit=
.si/static/dialInInfo.html?room=3DXenSummit24Main" style=3D"text-decoration=
-line:none" target=3D"_blank"><span style=3D"font-size:11pt;font-family:Ari=
al,sans-serif;background-color:transparent;font-variant-numeric:normal;font=
-variant-east-asian:normal;font-variant-alternates:normal;text-decoration-l=
ine:underline;vertical-align:baseline">https://meet.jit.si/static/dialInInf=
o.html?room=3DXenSummit24Main</span></a><span style=3D"font-size:11pt;font-=
family:Arial,sans-serif;background-color:transparent;font-variant-numeric:n=
ormal;font-variant-east-asian:normal;font-variant-alternates:normal;vertica=
l-align:baseline">=C2=A0</span></p><br><p dir=3D"ltr" style=3D"line-height:=
1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-=
family:Arial,sans-serif;background-color:transparent;font-variant-numeric:n=
ormal;font-variant-east-asian:normal;font-variant-alternates:normal;vertica=
l-align:baseline">If also dialing-in through a room phone, join without con=
necting to audio: </span><a href=3D"https://meet.jit.si/XenSummit24Main#con=
fig.startSilent=3Dtrue" style=3D"text-decoration-line:none" target=3D"_blan=
k"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-co=
lor:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;=
font-variant-alternates:normal;text-decoration-line:underline;vertical-alig=
n:baseline">https://meet.jit.si/XenSummit24Main#config.startSilent=3Dtrue</=
span></a><span style=3D"font-size:11pt;font-family:Arial,sans-serif;backgro=
und-color:transparent;font-variant-numeric:normal;font-variant-east-asian:n=
ormal;font-variant-alternates:normal;vertical-align:baseline">=C2=A0</span>=
</p></td><td style=3D"border-width:0.681818pt;border-style:solid;border-col=
or:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr=
" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=
=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transparen=
t;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-a=
lternates:normal;vertical-align:baseline">See meeting dial-in numbers: </sp=
an><a href=3D"https://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSe=
ssionsA" style=3D"text-decoration-line:none" target=3D"_blank"><span style=
=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transparen=
t;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-a=
lternates:normal;text-decoration-line:underline;vertical-align:baseline">ht=
tps://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsA</span></=
a></p><br><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-b=
ottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;backg=
round-color:transparent;font-variant-numeric:normal;font-variant-east-asian=
:normal;font-variant-alternates:normal;vertical-align:baseline">If also dia=
ling-in through a room phone, join without connecting to audio: </span><a h=
ref=3D"https://meet.jit.si/XenDesignSessionsA#config.startSilent=3Dtrue" st=
yle=3D"text-decoration-line:none" target=3D"_blank"><span style=3D"font-siz=
e:11pt;font-family:Arial,sans-serif;background-color:transparent;font-varia=
nt-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:no=
rmal;text-decoration-line:underline;vertical-align:baseline">https://meet.j=
it.si/XenDesignSessionsA#config.startSilent=3Dtrue</span></a></p></td><td s=
tyle=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);=
vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"lin=
e-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:1=
1pt;font-family:Arial,sans-serif;background-color:transparent;font-variant-=
numeric:normal;font-variant-east-asian:normal;font-variant-alternates:norma=
l;vertical-align:baseline">See meeting dial-in numbers: </span><a href=3D"h=
ttps://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsB" style=
=3D"text-decoration-line:none" target=3D"_blank"><span style=3D"font-size:1=
1pt;font-family:Arial,sans-serif;background-color:transparent;font-variant-=
numeric:normal;font-variant-east-asian:normal;font-variant-alternates:norma=
l;text-decoration-line:underline;vertical-align:baseline">https://meet.jit.=
si/static/dialInInfo.html?room=3DXenDesignSessionsB</span></a></p><br><p di=
r=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><spa=
n style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:tra=
nsparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-va=
riant-alternates:normal;vertical-align:baseline">If also dialing-in through=
 a room phone, join without connecting to audio: </span><a href=3D"https://=
meet.jit.si/XenDesignSessionsB#config.startSilent=3Dtrue" style=3D"text-dec=
oration-line:none" target=3D"_blank"><span style=3D"font-size:11pt;font-fam=
ily:Arial,sans-serif;background-color:transparent;font-variant-numeric:norm=
al;font-variant-east-asian:normal;font-variant-alternates:normal;text-decor=
ation-line:underline;vertical-align:baseline">https://meet.jit.si/XenDesign=
SessionsB#config.startSilent=3Dtrue</span></a></p></td></tr><tr style=3D"he=
ight:82.5pt"><td style=3D"border-width:0.681818pt;border-style:solid;border=
-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D=
"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span st=
yle=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transpa=
rent;font-weight:700;font-style:italic;font-variant-numeric:normal;font-var=
iant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseli=
ne">Schedule Example:=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1=
.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-f=
amily:Arial,sans-serif;background-color:transparent;font-style:italic;font-=
variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternat=
es:normal;vertical-align:baseline">Tuesday 4th &amp; Wednesday 5th June 202=
4</span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin=
-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;bac=
kground-color:transparent;font-style:italic;font-variant-numeric:normal;fon=
t-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:b=
aseline">(Lisbon timezone)</span></p></td><td style=3D"border-width:0.68181=
8pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:5=
pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0p=
t;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-s=
erif;background-color:transparent;font-weight:700;font-style:italic;font-va=
riant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates=
:normal;vertical-align:baseline">Schedule Example: </span><span style=3D"fo=
nt-size:11pt;font-family:Arial,sans-serif;background-color:transparent;font=
-style:italic;font-variant-numeric:normal;font-variant-east-asian:normal;fo=
nt-variant-alternates:normal;vertical-align:baseline">Wednesday 5th &amp; T=
hursday 6th June 2024</span></p><p dir=3D"ltr" style=3D"line-height:1.656;m=
argin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:=
Arial,sans-serif;background-color:transparent;font-style:italic;font-varian=
t-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:nor=
mal;vertical-align:baseline">(Lisbon timezone)</span></p></td><td style=3D"=
border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical=
-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height=
:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font=
-family:Arial,sans-serif;background-color:transparent;font-weight:700;font-=
style:italic;font-variant-numeric:normal;font-variant-east-asian:normal;fon=
t-variant-alternates:normal;vertical-align:baseline">Schedule Example:=C2=
=A0 </span><span style=3D"font-size:11pt;font-family:Arial,sans-serif;backg=
round-color:transparent;font-style:italic;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-variant-alternates:normal;vertical-align:bas=
eline">Wednesday 5th &amp; Thursday 6th June 2024</span></p><p dir=3D"ltr" =
style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D=
"font-size:11pt;font-family:Arial,sans-serif;background-color:transparent;f=
ont-style:italic;font-variant-numeric:normal;font-variant-east-asian:normal=
;font-variant-alternates:normal;vertical-align:baseline">(Lisbon timezone)<=
/span></p><br></td></tr><tr style=3D"height:201.75pt"><td style=3D"border-w=
idth:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:t=
op;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;m=
argin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:=
Arial,sans-serif;background-color:transparent;font-variant-numeric:normal;f=
ont-variant-east-asian:normal;font-variant-alternates:normal;vertical-align=
:baseline">09:00=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1.656;=
margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family=
:Arial,sans-serif;background-color:transparent;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">Welcome &amp; Opening Remarks - Kelly Choi, Community Manager, =
Cloud Software Group, XenServer</span></p><p dir=3D"ltr" style=3D"line-heig=
ht:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;fo=
nt-family:Arial,sans-serif;background-color:transparent;font-variant-numeri=
c:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vert=
ical-align:baseline">LIBERDADE I</span></p><br><p dir=3D"ltr" style=3D"line=
-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11=
pt;font-family:Arial,sans-serif;background-color:transparent;font-variant-n=
umeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal=
;vertical-align:baseline">09:10=C2=A0</span></p><p dir=3D"ltr" style=3D"lin=
e-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:1=
1pt;font-family:Arial,sans-serif;background-color:transparent;font-variant-=
numeric:normal;font-variant-east-asian:normal;font-variant-alternates:norma=
l;vertical-align:baseline">Xen Project 2024 Weather Report - Kelly Choi, Xe=
nServer, Cloud Software Group.</span></p><p dir=3D"ltr" style=3D"line-heigh=
t:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;fon=
t-family:Arial,sans-serif;background-color:transparent;font-variant-numeric=
:normal;font-variant-east-asian:normal;font-variant-alternates:normal;verti=
cal-align:baseline">LIBERDADE I</span></p><br></td><td style=3D"border-widt=
h:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;=
padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;marg=
in-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Ari=
al,sans-serif;background-color:transparent;font-variant-numeric:normal;font=
-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:ba=
seline">13:45=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1.656;mar=
gin-top:0pt;margin-bottom:0pt"><a href=3D"https://design-sessions.xenprojec=
t.org/uid/discussion/disc_OlJce1uK3uI0OjS2cL55/view" style=3D"text-decorati=
on-line:none" target=3D"_blank"><span style=3D"font-size:11pt;font-family:A=
rial,sans-serif;font-variant-numeric:normal;font-variant-east-asian:normal;=
font-variant-alternates:normal;text-decoration-line:underline;vertical-alig=
n:baseline">The future of Xen Project physical events</span></a></p><p dir=
=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span=
 style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:tran=
sparent;font-variant-numeric:normal;font-variant-east-asian:normal;font-var=
iant-alternates:normal;vertical-align:baseline">LIBERDADE I</span></p><br><=
p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt">=
<span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color=
:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;fon=
t-variant-alternates:normal;vertical-align:baseline">14:35=C2=A0</span></p>=
<p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"=
><a href=3D"https://design-sessions.xenproject.org/uid/discussion/disc_PEgB=
NIXMyEkdJoysE27O/view" style=3D"text-decoration-line:none" target=3D"_blank=
"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;font-variant-n=
umeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal=
;text-decoration-line:underline;vertical-align:baseline">IOMMU paravirtuali=
zation and Xen IOMMU subsystem rework</span></a></p><p dir=3D"ltr" style=3D=
"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-si=
ze:11pt;font-family:Arial,sans-serif;background-color:transparent;font-vari=
ant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:n=
ormal;vertical-align:baseline">LIBERDADE I</span></p></td><td style=3D"bord=
er-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-ali=
gn:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.6=
56;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-fam=
ily:Arial,sans-serif;background-color:transparent;font-variant-numeric:norm=
al;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-a=
lign:baseline">13:45=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1.=
656;margin-top:0pt;margin-bottom:0pt"><a href=3D"https://design-sessions.xe=
nproject.org/uid/discussion/disc_z78Lt2EIZt2qxaS3FSlQ/view" style=3D"text-d=
ecoration-line:none" target=3D"_blank"><span style=3D"font-size:11pt;font-f=
amily:Arial,sans-serif;font-variant-numeric:normal;font-variant-east-asian:=
normal;font-variant-alternates:normal;text-decoration-line:underline;vertic=
al-align:baseline">Downstream working group</span></a></p><p dir=3D"ltr" st=
yle=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"f=
ont-size:11pt;font-family:Arial,sans-serif;background-color:transparent;fon=
t-variant-numeric:normal;font-variant-east-asian:normal;font-variant-altern=
ates:normal;vertical-align:baseline">AUGUSTA I</span></p><br><p dir=3D"ltr"=
 style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=
=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transparen=
t;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-a=
lternates:normal;vertical-align:baseline">14:35=C2=A0</span></p><p dir=3D"l=
tr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a href=3D=
"https://design-sessions.xenproject.org/uid/discussion/disc_01JE9EOi9zxxAU8=
lfB6Z/view" style=3D"text-decoration-line:none" target=3D"_blank"><span sty=
le=3D"font-size:11pt;font-family:Arial,sans-serif;font-variant-numeric:norm=
al;font-variant-east-asian:normal;font-variant-alternates:normal;text-decor=
ation-line:underline;vertical-align:baseline">Xen Safety Requirements upstr=
eaming</span></a></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0=
pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-=
serif;background-color:transparent;font-variant-numeric:normal;font-variant=
-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline">=
AUGUSTA I</span></p><br><br></td></tr></tbody></table></div></span></div><d=
iv><span id=3D"m_3017200869748370048m_2927081184909999839m_-837099889567594=
7915gmail-docs-internal-guid-7c024368-7fff-f910-ae50-70b83521ee30"><div><sp=
an><br></span></div><b><u>How to filter the schedule:</u></b><br><span styl=
e=3D"border:none;display:inline-block;overflow:hidden;width:624px;height:41=
1px"><img src=3D"https://lh7-us.googleusercontent.com/nJqdiQUkRhper7xaNfEg5=
kdR7Jsn_7Mb7Muwgz_IaImd4-ZOK0D5o9_jypDXHX2ybCsevBnzCISatQZxOIu5eEkBWWsYjYpX=
NCTw3khSi5qySJoq7hzZ5HPsoLxAJOQ5KjEzJXfvgVnLc4s_53AqXuE" width=3D"624" heig=
ht=3D"411" style=3D"margin-left:0px;margin-top:0px"></span></span><br></div=
><div><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_si=
gnature"><div dir=3D"ltr"><div><br></div><div><br></div><div>We look forwar=
d to seeing you there, both virtually and physically!</div><div><br></div><=
div>Many thanks,</div><div>Kelly Choi</div><div><br></div><div><div style=
=3D"color:rgb(136,136,136)">Community Manager</div><div style=3D"color:rgb(=
136,136,136)">Xen Project=C2=A0<br></div></div></div></div></div></div>

--00000000000035af2506199834af--


From xen-devel-bounces@lists.xenproject.org Wed May 29 14:07:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731994.1137798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy5-00050J-H9; Wed, 29 May 2024 14:07:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731994.1137798; Wed, 29 May 2024 14:07:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy5-000509-EQ; Wed, 29 May 2024 14:07:53 +0000
Received: by outflank-mailman (input) for mailman id 731994;
 Wed, 29 May 2024 14:07:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCJy4-0004Fc-46
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:07:52 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d574fccb-1dc4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:07:50 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-52ae14e78e5so818072e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:07:50 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-529b712e856sm736822e87.197.2024.05.29.07.07.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:07:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d574fccb-1dc4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716991670; x=1717596470; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8Zme5MAc1tBIih0t7wrm5r256IXDIRT3xxgBybd+OME=;
        b=SD1i/pICPsiT+YN882ho54wlay6dFQERXGEeWfrPDo6Bq73nnDx98KqphTWEdscsmW
         aPhqIJ7W4l1Q/GWoI8NruEB+mOmSEm555k1rrtiqH6WmlrHcK51Y4RQ48oNT+k0b9MCH
         9p06J8OiJ4gg4pqh7kUbXQNuCOXT2Ad/UiHQoI8ggyIyfiimbnMRlvNjlPyIFQu4Huvl
         2UM5BeUZTGmRIT64TiSPI8a1kiDIDR9YmHDNMWZRbwTCtRH69hJwAPFVJVSa9GAiyzYa
         l+0PIKLXZbr6LGbNc7E8PJtaXvA6YeBpo/NXmdTJfvYEIWXbUs11Rbjn+JMXpvC3ZQDX
         X8Ng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991670; x=1717596470;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8Zme5MAc1tBIih0t7wrm5r256IXDIRT3xxgBybd+OME=;
        b=fSZqa0K7smf7JyEUYSwSSvDmxz4Vs+nUkP2as/k0VEdC47ioLNVnRSH97uPyEzwdrI
         HGf6WGSP9f+CZWqu1SNNozDM6pq+cSPSqgQwCdfZt7ZQAk6YoiTKccvJRhHtLug552KK
         zcP66f4EuxGPWJd82DFrZVSNADAbLsSuw77XlDkXncUsXpD3FF5+2dj1Ac4MFEJ8rPRy
         zWQewrE87+bW7L900LRIZ4p+Zk/2+MV0nuliAq83EZCmFKtF9RCrFMSzKU1N2qVPJwaj
         6gnSWA/UHkxibk92bHVPcGru2poErNqxbsEwbDyGWtzR+RiJxcFwOP6PYflDODDfLZVU
         LMiQ==
X-Forwarded-Encrypted: i=1; AJvYcCW/Nt0I7sSn7K+i1vS3JKh8mp92Ig8KvVl6yghKDOl0/ouSLP157LMN04M6aIqFK+5Ro/irIRGMdZRlagbH6h3SP8/QVcUUOXhPvVX4wjI=
X-Gm-Message-State: AOJu0YzotxyBWseCUM7dlkGlrEMX40zkcAeU9Rg5dSLvPgsVKIw1W5MH
	4NquYA8U5zz8m7a74sNYGZboBebvauVnq2L76eWDg9Zo+Nxk5EH3
X-Google-Smtp-Source: AGHT+IG4vbZH58aNNCx5WtguQ+wygMZh32hk2g0EZYP53A4we56wYQmUuGwtdGe8zRPUZLd0GfJq8g==
X-Received: by 2002:a05:6512:3194:b0:51f:3fea:cbcf with SMTP id 2adb3069b0e04-529664dac0cmr11284687e87.52.1716991669796;
        Wed, 29 May 2024 07:07:49 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v8 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()
Date: Wed, 29 May 2024 16:07:37 +0200
Message-Id: <20240529140739.1387692-7-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
References: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/xen/xen-mapcache.c         | 16 +++++++++++-----
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c              |  9 +++++----
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
                              void *vaddr,
                              hwaddr size,
                              hwaddr address_index,
-                             bool dummy)
+                             bool dummy,
+                             ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
     xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
+                                       ram_addr_t ram_offset,
                                        uint8_t lock, bool dma, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
     if (!entry) {
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
-        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+        xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                         ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
                 !test_bits(address_offset >> XC_PAGE_SHIFT,
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
-            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+            xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+                             ram_offset);
         }
     }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
                        hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
     uint8_t *p;
 
     mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+                               lock, dma, is_write);
     mapcache_unlock(mapcache);
     return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false);
+                     cache_size, address_index, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
                         void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+                       ram_addr_t ram_addr_offset,
                        uint8_t lock, bool dma,
                        bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
                                      hwaddr phys_addr,
                                      hwaddr size,
+                                     ram_addr_t ram_addr_offset,
                                      uint8_t lock,
                                      bool dma,
                                      bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
          */
         if (xen_mr_is_memory(block->mr)) {
             return xen_map_cache(block->mr, block->offset + addr,
-                                 len, lock, lock,
-                                 is_write);
+                                 len, block->offset,
+                                 lock, lock, is_write);
         }
 
         block->host = xen_map_cache(block->mr, block->offset,
-                                    block->max_length, 1,
-                                    lock, is_write);
+                                    block->max_length,
+                                    block->offset,
+                                    1, lock, is_write);
     }
 
     return ramblock_ptr(block, addr);
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:07:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731993.1137788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy1-0004iY-AK; Wed, 29 May 2024 14:07:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731993.1137788; Wed, 29 May 2024 14:07:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy1-0004iR-7G; Wed, 29 May 2024 14:07:49 +0000
Received: by outflank-mailman (input) for mailman id 731993;
 Wed, 29 May 2024 14:07:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCJy0-0004U5-7I
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:07:48 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d361b662-1dc4-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:07:46 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-52b119986f2so710587e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:07:46 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-529715e57c1sm1333816e87.286.2024.05.29.07.07.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:07:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d361b662-1dc4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716991666; x=1717596466; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yfQU3CIk8oi6JUEGcx9QZf/9+B1Oq0qqW+7JWEaNpUQ=;
        b=QoswkUFCR7FGFb+hWlx6AuRK9tIIKedF4mT5M9Vym6NhCnNt9L/y6LyY6S5NeM6m2F
         blU/eyXywaU32Ekx8sq4+uh43onH5LCL/KLHJ0Vl+npjWnyTqgh3m8Y9B85h0WHpvBZP
         6EkVID6sK79PHiDCm+Fd1JZVnND5UzHJQIRiJF3EmXDXueQhdnYIlveIRaLauEnIYuQE
         Z1AMBdzpU3IP+eicgW2NZnUEgxxFCS+eXtudP1d3klxg1nF11tix1sccKMSxxCuf8QyY
         Dx5QW9N7b/k2oUas267tyxGqPBut8wT7IwYDCmsuDiiMhYIAwy3a6p0boky3uvRT5xfO
         AfPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991666; x=1717596466;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=yfQU3CIk8oi6JUEGcx9QZf/9+B1Oq0qqW+7JWEaNpUQ=;
        b=NFC1lhpBNflk5TRn7V+3ogxm5Z2fWvHdzycE4bvH/w1omJA0EVm+By/fMA2yC+xIBJ
         tA0tlFFWTy3f9bCXK8RTYHS2iBTlKE+Wk4lcmioNa/js4B1eukApeMYu35pkfFg+Xw0V
         aXqfMq8L5uyRfKUfdlACL+dUxFGVcodzMgP+eDcGBaEh6qekSXk7zUWlOSpZztwD5quI
         spOOCPPLNkO63kDA6gdklkKp3mL1q5AH20/+qi3mcfy89t+BTVQ99PKvzrv6nM44XJOr
         +ewZo3mVuCPCUBxUzQCLysuaB4ZPO/viOKl9ZuVpWX1YUrq+AeLictE2OL+/Y5gDSVyY
         acLg==
X-Forwarded-Encrypted: i=1; AJvYcCXRL1mH+PPVfjDvx2s7bQwGmMrAIoJtlGXDIsGmqDc3mOTT/mPNbmW1zTpuR9rabA57IbSL5TEr70NMzHiAR2ex/YAO0u9JK/qUG+4E8KY=
X-Gm-Message-State: AOJu0YwpqxQt5LJDF4CvIgfvJt/8rr6MUeDp+wT9yY/7lezgR3d8QPaa
	y8w8dHbdcW+TQ2NhqSQfEtn2g+zSlsMAXdzts9Gty8mm1cQfl8D0
X-Google-Smtp-Source: AGHT+IGzkXOYSImMOV6N87l6JZQ270Dpu6VtX1s3RWgHcPsFkFEwPwb1/bHSmEc0ivZv+7V/GBf3hw==
X-Received: by 2002:a19:2d48:0:b0:526:92d7:52d4 with SMTP id 2adb3069b0e04-529bea19e84mr2711966e87.61.1716991666190;
        Wed, 29 May 2024 07:07:46 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	David Hildenbrand <david@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v8 3/8] xen: Add xen_mr_is_memory()
Date: Wed, 29 May 2024 16:07:34 +0200
Message-Id: <20240529140739.1387692-4-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
References: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/xen/xen-hvm-common.c | 10 ++++++++--
 include/sysemu/xen.h    |  1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+    return mr == &xen_memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
                    Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
         return;
     }
 
-    if (mr == &xen_memory) {
+    if (xen_mr_is_memory(mr)) {
         return;
     }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
     XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-    if (section->mr == &xen_memory) {
+    if (xen_mr_is_memory(section->mr)) {
         return;
     } else {
         if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..3445888e39 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -49,4 +49,5 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
 
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
+bool xen_mr_is_memory(MemoryRegion *mr);
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:07:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731990.1137768 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJxy-0004Fz-Sp; Wed, 29 May 2024 14:07:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731990.1137768; Wed, 29 May 2024 14:07:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJxy-0004Fs-QD; Wed, 29 May 2024 14:07:46 +0000
Received: by outflank-mailman (input) for mailman id 731990;
 Wed, 29 May 2024 14:07:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCJxx-0004Fc-BY
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:07:45 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d13072ec-1dc4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:07:43 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-52aea6067c8so855752e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:07:43 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52970c3327esm1261419e87.196.2024.05.29.07.07.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:07:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d13072ec-1dc4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716991663; x=1717596463; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=XX98xi2OKo1rMxHP05l7Ai2sBmm2UHSJYclhGJTbPY6hemQcqpyC8gJ1ERp5o/PSAt
         tEPwxxcgUxzAO6iklFeg6zMb+tinc/T1CmAIfuL2tP1DlBayxR3HKgjPTwK5gjdwkWhY
         raHVbt7P8jJ9trw4DLwEt1P8g0BxlKUvbujbes/LDaIDPu6I9/NMLmFYxaYzcNdB2Ldm
         kvkfXETtSIARjRNGf/NqZ/97Utjhya12PTOVY2dN8Y6XvB+7nrv1z9dIBx81hSSgkK+S
         yrxgRydCpUTQKUgtI86nBZFT2/qITe1LqguQzbuCxyIwh8SWdz2lHnf+xWQ/Bf7p+yBj
         5TUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991663; x=1717596463;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oKmmum6z28VcY8ouSg3YJA3bHZvP87XkBXtO/hd8PR8=;
        b=U+fAI+Riu3y+qdXSeL7gDt3hPMpOPwoQ4xS1Yew5UsGWvqUsuYXPEhrzVwfhGD/UL9
         wB2em/Fl/qEBv8AJVWjY4JxogOVN3kX79FZNZX7xuZOF3IAcNtPGQFfuceRcwCo2LOTR
         gCF07Pxa2CQaBQW74jVNxtB1cUjtWHXLEimf0VLMNdckSJPFMjoxd52WnUttWQK54F36
         dCYnKlcZC8sb9Oo8EgQmdQXj+yoVKbxBwusQE8E5blT5hzfy8SgRzyrKfsl79HUtePnj
         TO/Ev5uVqMLeIK+UrsIdio/EE6YuX7P1aOi7Zrzn6vzAScvbELEdrMDKevbT19YugyFi
         eftA==
X-Forwarded-Encrypted: i=1; AJvYcCVJR3HQ8fLNZH6ZQnLxuxES0/b+L8fIX3ZXtoHd9Sx7Hplflj+Es2ACKVVIDEPw9lNPWg6gsBTorteLHiwIfynWXD/xMBuO2ZgZVQzkdEU=
X-Gm-Message-State: AOJu0YyaTlI6Jd5R0gwtX1wOtjRHcfNouhLYANgo6PDIUoX2bInS0L+4
	OByRnvNwXQL6XHNCbJYBdf9AkdQrf/zudAG7B3UuRqGAmOY5UmbZ
X-Google-Smtp-Source: AGHT+IG9X1hyyW5bWOM2Lb/ezgqDrIcsxWZZ6e+vRIDT8WUGaZyhyargSGMsndYosYqEtRb+Y+5fOQ==
X-Received: by 2002:a05:6512:398b:b0:529:a2d8:3009 with SMTP id 2adb3069b0e04-529a2d830e5mr11094033e87.43.1716991662399;
        Wed, 29 May 2024 07:07:42 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v8 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
Date: Wed, 29 May 2024 16:07:32 +0200
Message-Id: <20240529140739.1387692-2-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
References: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 54 ++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE     (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
     /* For most cases (>99.9%), the page address is the same. */
     MapCacheEntry *last_entry;
     unsigned long max_mcache_size;
-    unsigned int mcache_bucket_shift;
+    unsigned int bucket_shift;
+    unsigned long bucket_size;
 
     phys_offset_to_gaddr_t phys_offset_to_gaddr;
     QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
                                            void *opaque,
+                                           unsigned int bucket_shift,
                                            unsigned long max_size)
 {
     unsigned long size;
     MapCache *mc;
 
+    assert(bucket_shift >= XC_PAGE_SHIFT);
+
     mc = g_new0(MapCache, 1);
 
     mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
     QTAILQ_INIT(&mc->locked_entries);
 
+    mc->bucket_shift = bucket_shift;
+    mc->bucket_size = 1UL << bucket_shift;
     mc->max_mcache_size = max_size;
 
     mc->nr_buckets =
         (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-          (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
-         (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+          (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+         (bucket_shift - XC_PAGE_SHIFT));
 
     size = mc->nr_buckets * sizeof(MapCacheEntry);
     size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
 {
     struct rlimit rlimit_as;
     unsigned long max_mcache_size;
+    unsigned int bucket_shift;
+
+    if (HOST_LONG_BITS == 32) {
+        bucket_shift = 16;
+    } else {
+        bucket_shift = 20;
+    }
 
     if (geteuid() == 0) {
         rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
         }
     }
 
-    mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+    mapcache = xen_map_cache_init_single(f, opaque,
+                                         bucket_shift,
+                                         max_mcache_size);
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
     entry->valid_mapping = NULL;
 
     for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
     }
 
     /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
     bool dummy = false;
 
 tryagain:
-    address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = phys_addr >> mc->bucket_shift;
+    address_offset = phys_addr & (mc->bucket_size - 1);
 
     trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
         return mc->last_entry->vaddr_base + address_offset;
     }
 
-    /* size is always a multiple of MCACHE_BUCKET_SIZE */
+    /* size is always a multiple of mc->bucket_size */
     if (size) {
         cache_size = size + address_offset;
-        if (cache_size % MCACHE_BUCKET_SIZE) {
-            cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+        if (cache_size % mc->bucket_size) {
+            cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
         }
     } else {
-        cache_size = MCACHE_BUCKET_SIZE;
+        cache_size = mc->bucket_size;
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
         trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
         raddr = RAM_ADDR_INVALID;
     } else {
-        raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+        raddr = (reventry->paddr_index << mc->bucket_shift) +
              ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
     }
     mapcache_unlock(mc);
@@ -585,8 +597,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
     hwaddr address_index, address_offset;
     hwaddr test_bit_size, cache_size = size;
 
-    address_index  = old_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = old_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = old_phys_addr >> mc->bucket_shift;
+    address_offset = old_phys_addr & (mc->bucket_size - 1);
 
     assert(size);
     /* test_bit_size is always a multiple of XC_PAGE_SIZE */
@@ -595,8 +607,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         test_bit_size += XC_PAGE_SIZE - (test_bit_size % XC_PAGE_SIZE);
     }
     cache_size = size + address_offset;
-    if (cache_size % MCACHE_BUCKET_SIZE) {
-        cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
+    if (cache_size % mc->bucket_size) {
+        cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
     }
 
     entry = &mc->entry[address_index % mc->nr_buckets];
@@ -609,8 +621,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
-    address_index  = new_phys_addr >> MCACHE_BUCKET_SHIFT;
-    address_offset = new_phys_addr & (MCACHE_BUCKET_SIZE - 1);
+    address_index  = new_phys_addr >> mc->bucket_shift;
+    address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:07:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731991.1137778 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy0-0004UG-3y; Wed, 29 May 2024 14:07:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731991.1137778; Wed, 29 May 2024 14:07:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy0-0004U9-0l; Wed, 29 May 2024 14:07:48 +0000
Received: by outflank-mailman (input) for mailman id 731991;
 Wed, 29 May 2024 14:07:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCJxy-0004Fc-6z
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:07:46 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1ef8959-1dc4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:07:44 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-52ab1d7243fso1309798e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:07:44 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b233sm1274418e87.135.2024.05.29.07.07.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:07:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1ef8959-1dc4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716991664; x=1717596464; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=FVt1ngxRLYD1k7hvosEvjumaiPvYRKgMOLSuMGarKcuTVZTzSxEeN8rKR1NUZl+mi5
         /iZY4xs3MHZOQhUCXrXb+ESynSEjxvPHo/IimW7RSSo8lEJn+T67B8opXVMC4zSHYINu
         EsbpDLo4z7j+U0l8vKjOsPs2wCs5qNqIYHgItXZzTABZJeNT3hKDi87Sq7XSAW9QbA5j
         y57EHS3XiQl7ScWMWeaOpfllUYrprMgMrk+EUnw7QumZThyp6W/67ZX/8iu7IYZKovQD
         6xDsH8XruihqB3mGeXuQYuQwoc0TteibaikDYPr2UIJWJd9iJYYIigHIUZwyCsUio5r6
         f1GA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991664; x=1717596464;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=suqLmt7AH8VX+OrzngkGIik/kesngvj14F2vCYL7OFM=;
        b=XUZoH+llZJFkUagjVEvlOGJPCeA/RUVNzIJbcY5McfXT7XauDkXKKDuQvAeHe7CsjW
         YRx6QiXQRCfnM6R4T8kZEbIcUVTfIy6+eWrfWM85Z6Qiua57YsQAo0JwCyilC4ezrsQm
         N4q63+Y3eI3tdP9iu2mq4EOuvCziXn4TW7DtXrGnyG4Kx7kSDQxYdqQwsXV/gzE5I+C9
         cqvRbdAzCyGp3e8SBtzD24TA0+v7pmj+rjWwtD5wdcQqYe3MgnrDh4RLjdfe0uHLXzxc
         KVia15B04xOxkPYQS1nmZC5D4g2j4r07cBk40ASrh1JajFxNyq69Cdr/GoeZUQTrqz6T
         7Z8w==
X-Forwarded-Encrypted: i=1; AJvYcCV36Yczd0UzPednRoF+T9lucLpAtnG62olT0Qh7cuRv9oHJlq/gz8QGKfofP1647k4Z2CXSjs0LzRRjn3DMx7Lvxz9gPQUTbA4Xm8P6vXs=
X-Gm-Message-State: AOJu0YxexOGC/uzxYNQVEBTg5V1VqGN/QonnR6a4NxhaO23mDkwKKc8p
	SdrwWmzw/OKy2TrQhX1fmqRklb/TfgPHifz/Do1BNgBlxaDv/lYk
X-Google-Smtp-Source: AGHT+IGdkBzBgEhHrd1LW9KAGa/cGv5sdiBTlj+GoDqEM5VjT/z9rWbfYzsj/AoEXZVsdTS9qD3Uag==
X-Received: by 2002:a05:6512:3b12:b0:51d:2529:7c4d with SMTP id 2adb3069b0e04-52961a82092mr14632029e87.0.1716991663724;
        Wed, 29 May 2024 07:07:43 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v8 2/8] xen: mapcache: Unmap first entries in buckets
Date: Wed, 29 May 2024 16:07:33 +0200
Message-Id: <20240529140739.1387692-3-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
References: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
         return;
     }
     entry->lock--;
-    if (entry->lock > 0 || pentry == NULL) {
+    if (entry->lock > 0) {
         return;
     }
 
-    pentry->next = entry->next;
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
     if (munmap(entry->vaddr_base, entry->size) != 0) {
         perror("unmap fails");
         exit(-1);
     }
+
     g_free(entry->valid_mapping);
-    g_free(entry);
+    if (pentry) {
+        pentry->next = entry->next;
+        g_free(entry);
+    } else {
+        memset(entry, 0, sizeof *entry);
+    }
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:07:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:07:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.731995.1137808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy7-0005I7-Ut; Wed, 29 May 2024 14:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 731995.1137808; Wed, 29 May 2024 14:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCJy7-0005I0-QV; Wed, 29 May 2024 14:07:55 +0000
Received: by outflank-mailman (input) for mailman id 731995;
 Wed, 29 May 2024 14:07:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9nLQ=NA=gmail.com=edgar.iglesias@srs-se1.protection.inumbo.net>)
 id 1sCJy6-0004U5-F8
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:07:54 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d757e734-1dc4-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:07:53 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e73359b8fbso25209341fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:07:53 -0700 (PDT)
Received: from gmail.com (213-67-3-247-no600.tbcn.telia.com. [213.67.3.247])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bcf47a4sm25715181fa.57.2024.05.29.07.07.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:07:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d757e734-1dc4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716991673; x=1717596473; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6KnYV1+9ySu8xm9Hqqrupes7IMG1mFuyj5Dx0g125FI=;
        b=ZJguEYA/goiqgE71vkjZksCHexlAHJQy0pKwAfN7K+IOtzikM6Qan8s8pxg9hllTP8
         +NKzkcElvVcFMWpesVxiWgApRHKfiHSZNAUWqs5es5zEfUP8JHrI/hke8qbPrCzHc2aH
         FIycn4SJ+lSVxHAgAaPmYIcaXykkYaAOTU56gUBbo7JMuszqfewXGXi0f9zObqNUjr3M
         NAf+/UHLHdPOxewxsr8nPCbru3OwOPYDh1k+KcQNLdXXTfMlZkyLGKqqQpDyJmu916pD
         a1217N3oSWX+AmDc41SNDiVy/MgH9nUdlcueqyEsFza+0zV3aXSjjEJ3pkkvIznmSade
         q9fQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716991673; x=1717596473;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=6KnYV1+9ySu8xm9Hqqrupes7IMG1mFuyj5Dx0g125FI=;
        b=vYe4BNpjEG4sagHtjSbKNOXcB0MU3SL4iyxBw+AE+pe14G8D+qvpB9ZHgMrs7mcnyF
         ehGdZq6E4YvSZ0OsPa+tUfhLYc5Mxu25LS5KFbItBdjrxlfN6bA8YDtoYCNQ0Awa9k8t
         ZmsvtVXHxLiLAKyOjyuk5nJFZhEEzEVfMQTU6MhMXSwvXcI0baO2x/COmNVTOuH0QPTY
         rzc/LbCIhFIV9sV07L+2joFyYqZsV0TvnDrLLthqGvvKJNTlIDXLutv12HPj49cswzEF
         k/Wj2SaaQv83xklFXf0OIl0478/yyAc+JPIpxjTYgkOTLYLCvgt/8eqxD6dXWf7PYh3F
         6njg==
X-Forwarded-Encrypted: i=1; AJvYcCVmd8qv3GS/m1sVDrMlL+3QlDZjYtnPzgtz/161RS4LuErj4g1Sf6gMVe14a4lA6qmh9sTJNKvwQgL9QSkNP6HxGOK4MR4MWrFQlKDz9G8=
X-Gm-Message-State: AOJu0YwVDOiO+mi7CGUMf7LqZtJ9LS3ry/tlFowK8eBuhmZJjmYI7DFA
	L7W2cRHk9Yml1l61f5cGghtZ6MIzEi+lRDNx9wSfJXhDD36LkKu6
X-Google-Smtp-Source: AGHT+IGD9ZwNPNUv35c+UJ5GB2OUukDkocTjVhprwhJ89cISo2uRDy4+PmwLngnnEeG1Una9nQbZqw==
X-Received: by 2002:a2e:a685:0:b0:2ea:80b7:c464 with SMTP id 38308e7fff4ca-2ea80b7c7acmr4569281fa.22.1716991672097;
        Wed, 29 May 2024 07:07:52 -0700 (PDT)
From: "Edgar E. Iglesias" <edgar.iglesias@gmail.com>
To: qemu-devel@nongnu.org
Cc: edgar.iglesias@gmail.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	"Edgar E. Iglesias" <edgar.iglesias@amd.com>,
	Manos Pitsidianakis <manos.pitsidianakis@linaro.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v8 7/8] xen: mapcache: Add support for grant mappings
Date: Wed, 29 May 2024 16:07:38 +0200
Message-Id: <20240529140739.1387692-8-edgar.iglesias@gmail.com>
X-Mailer: git-send-email 2.40.1
In-Reply-To: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
References: <20240529140739.1387692-1-edgar.iglesias@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: "Edgar E. Iglesias" <edgar.iglesias@amd.com>

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

CC: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-hvm-common.c         |  12 ++-
 hw/xen/xen-mapcache.c           | 165 +++++++++++++++++++++++++-------
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h            |   1 +
 4 files changed, 144 insertions(+), 37 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-    return mr == &xen_memory;
+    return mr == &xen_memory || mr == &xen_grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+    return mr == &xen_grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..5f23b0adbe 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include <sys/resource.h>
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include <xenevtchn.h>
+#include <xengnttab.h>
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE     (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
     unsigned long *valid_mapping;
     uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
     uint8_t flags;
     hwaddr size;
     struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     unsigned long max_mcache_size;
     unsigned int bucket_shift;
 
+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
+    if (xen_region_gnttabdev == NULL) {
+        error_report("mapcache: Failed to open gnttab device");
+        exit(EXIT_FAILURE);
+    }
+
     if (HOST_LONG_BITS == 32) {
         bucket_shift = 16;
     } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
     mapcache = xen_map_cache_init_single(f, opaque,
                                          bucket_shift,
                                          max_mcache_size);
+
+    /*
+     * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+     * map anything beyond the number of pages granted to us.
+     */
+    mapcache_grants = xen_map_cache_init_single(f, opaque,
+                                                XC_PAGE_SHIFT,
+                                                max_mcache_size);
+
     setrlimit(RLIMIT_AS, &rlimit_as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
                              hwaddr size,
                              hwaddr address_index,
                              bool dummy,
+                             bool grant,
+                             bool is_write,
                              ram_addr_t ram_offset)
 {
     uint8_t *vaddr_base;
-    xen_pfn_t *pfns;
-    int *err;
+    g_autofree uint32_t *refs = NULL;
+    g_autofree xen_pfn_t *pfns = NULL;
+    g_autofree int *err;
     unsigned int i;
     hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
     trace_xen_remap_bucket(address_index);
 
-    pfns = g_new0(xen_pfn_t, nb_pfn);
+    if (grant) {
+        refs = g_new0(uint32_t, nb_pfn);
+    } else {
+        pfns = g_new0(xen_pfn_t, nb_pfn);
+    }
     err = g_new0(int, nb_pfn);
 
     if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
     g_free(entry->valid_mapping);
     entry->valid_mapping = NULL;
 
-    for (i = 0; i < nb_pfn; i++) {
-        pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+    if (grant) {
+        hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+        for (i = 0; i < nb_pfn; i++) {
+            refs[i] = grant_base + i;
+        }
+    } else {
+        for (i = 0; i < nb_pfn; i++) {
+            pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+        }
     }
 
-    /*
-     * If the caller has requested the mapping at a specific address use
-     * MAP_FIXED to make sure it's honored.
-     */
+    entry->flags &= ~XEN_MAPCACHE_ENTRY_GRANT;
+
     if (!dummy) {
-        vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
-                                           PROT_READ | PROT_WRITE,
-                                           vaddr ? MAP_FIXED : 0,
-                                           nb_pfn, pfns, err);
+        if (grant) {
+            int prot = PROT_READ;
+
+            if (is_write) {
+                prot |= PROT_WRITE;
+            }
+
+            entry->flags |= XEN_MAPCACHE_ENTRY_GRANT;
+            assert(vaddr == NULL);
+            vaddr_base = xengnttab_map_domain_grant_refs(xen_region_gnttabdev,
+                                                         nb_pfn,
+                                                         xen_domid, refs,
+                                                         prot);
+        } else {
+            /*
+             * If the caller has requested the mapping at a specific address use
+             * MAP_FIXED to make sure it's honored.
+             *
+             * We don't yet support upgrading mappings from RO to RW, to handle
+             * models using ordinary address_space_rw(), foreign mappings ignore
+             * is_write and are always mapped RW.
+             */
+            vaddr_base = xenforeignmemory_map2(xen_fmem, xen_domid, vaddr,
+                                               PROT_READ | PROT_WRITE,
+                                               vaddr ? MAP_FIXED : 0,
+                                               nb_pfn, pfns, err);
+        }
         if (vaddr_base == NULL) {
-            perror("xenforeignmemory_map2");
+            perror(grant ? "xengnttab_map_domain_grant_refs"
+                           : "xenforeignmemory_map2");
             exit(-1);
         }
     } else {
@@ -260,15 +318,13 @@ static void xen_remap_bucket(MapCache *mc,
             bitmap_set(entry->valid_mapping, i, 1);
         }
     }
-
-    g_free(pfns);
-    g_free(err);
 }
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
                                        hwaddr phys_addr, hwaddr size,
                                        ram_addr_t ram_offset,
-                                       uint8_t lock, bool dma, bool is_write)
+                                       uint8_t lock, bool dma,
+                                       bool grant, bool is_write)
 {
     MapCacheEntry *entry, *pentry = NULL,
                   *free_entry = NULL, *free_pentry = NULL;
@@ -340,7 +396,7 @@ tryagain:
         entry = g_new0(MapCacheEntry, 1);
         pentry->next = entry;
         xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                         ram_offset);
+                         grant, is_write, ram_offset);
     } else if (!entry->lock) {
         if (!entry->vaddr_base || entry->paddr_index != address_index ||
                 entry->size != cache_size ||
@@ -348,7 +404,7 @@ tryagain:
                     test_bit_size >> XC_PAGE_SHIFT,
                     entry->valid_mapping)) {
             xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
-                             ram_offset);
+                             grant, is_write, ram_offset);
         }
     }
 
@@ -399,12 +455,26 @@ uint8_t *xen_map_cache(MemoryRegion *mr,
                        uint8_t lock, bool dma,
                        bool is_write)
 {
+    bool grant = xen_mr_is_grants(mr);
+    MapCache *mc = grant ? mapcache_grants : mapcache;
     uint8_t *p;
 
-    mapcache_lock(mapcache);
-    p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
-                               lock, dma, is_write);
-    mapcache_unlock(mapcache);
+    if (grant && !lock) {
+        /*
+         * Grants are only supported via address_space_map(). Anything
+         * else is considered a user/guest error.
+         *
+         * QEMU generally doesn't expect these mappings to ever fail, so
+         * if this happens we report an error message and abort().
+         */
+        error_report("Tried to access a grant reference without mapping it.");
+        abort();
+    }
+
+    mapcache_lock(mc);
+    p = xen_map_cache_unlocked(mc, phys_addr, size, ram_addr_offset,
+                               lock, dma, grant, is_write);
+    mapcache_unlock(mc);
     return p;
 }
 
@@ -449,7 +519,14 @@ static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 {
-    return xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    ram_addr_t addr;
+
+    addr = xen_ram_addr_from_mapcache_single(mapcache, ptr);
+    if (addr == RAM_ADDR_INVALID) {
+        addr = xen_ram_addr_from_mapcache_single(mapcache_grants, ptr);
+    }
+
+    return addr;
 }
 
 static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
@@ -460,6 +537,7 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     hwaddr paddr_index;
     hwaddr size;
     int found = 0;
+    int rc;
 
     QTAILQ_FOREACH(reventry, &mc->locked_entries, next) {
         if (reventry->vaddr_req == buffer) {
@@ -502,7 +580,14 @@ static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
     }
 
     ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
-    if (munmap(entry->vaddr_base, entry->size) != 0) {
+    if (entry->flags & XEN_MAPCACHE_ENTRY_GRANT) {
+        rc = xengnttab_unmap(xen_region_gnttabdev, entry->vaddr_base,
+                             entry->size >> mc->bucket_shift);
+    } else {
+        rc = munmap(entry->vaddr_base, entry->size);
+    }
+
+    if (rc) {
         perror("unmap fails");
         exit(-1);
     }
@@ -521,14 +606,24 @@ typedef struct XenMapCacheData {
     uint8_t *buffer;
 } XenMapCacheData;
 
+static void xen_invalidate_map_cache_entry_single(MapCache *mc, uint8_t *buffer)
+{
+    mapcache_lock(mc);
+    xen_invalidate_map_cache_entry_unlocked(mc, buffer);
+    mapcache_unlock(mc);
+}
+
+static void xen_invalidate_map_cache_entry_all(uint8_t *buffer)
+{
+    xen_invalidate_map_cache_entry_single(mapcache, buffer);
+    xen_invalidate_map_cache_entry_single(mapcache_grants, buffer);
+}
+
 static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
     XenMapCacheData *data = opaque;
 
-    mapcache_lock(mapcache);
-    xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
-    mapcache_unlock(mapcache);
-
+    xen_invalidate_map_cache_entry_all(data->buffer);
     aio_co_wake(data->co);
 }
 
@@ -543,9 +638,7 @@ void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
                                 xen_invalidate_map_cache_entry_bh, &data);
         qemu_coroutine_yield();
     } else {
-        mapcache_lock(mapcache);
-        xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
-        mapcache_unlock(mapcache);
+        xen_invalidate_map_cache_entry_all(buffer);
     }
 }
 
@@ -597,6 +690,7 @@ void xen_invalidate_map_cache(void)
     bdrv_drain_all();
 
     xen_invalidate_map_cache_single(mapcache);
+    xen_invalidate_map_cache_single(mapcache_grants);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
@@ -632,13 +726,16 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
         return NULL;
     }
 
+    assert((entry->flags & XEN_MAPCACHE_ENTRY_GRANT) == 0);
+
     address_index  = new_phys_addr >> mc->bucket_shift;
     address_offset = new_phys_addr & (mc->bucket_size - 1);
 
     trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
     xen_remap_bucket(mc, entry, entry->vaddr_base,
-                     cache_size, address_index, false, old_phys_addr);
+                     cache_size, address_index, false,
+                     false, false, old_phys_addr);
     if (!test_bits(address_offset >> XC_PAGE_SHIFT,
                 test_bit_size >> XC_PAGE_SHIFT,
                 entry->valid_mapping)) {
diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 65a51aac2e..3d796235dc 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -16,6 +16,7 @@
 #include <xen/hvm/ioreq.h>
 
 extern MemoryRegion xen_memory;
+extern MemoryRegion xen_grants;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
 
@@ -29,6 +30,8 @@ extern DeviceListener xen_device_listener;
     do { } while (0)
 #endif
 
+#define XEN_GRANT_ADDR_OFF (1ULL << 63)
+
 static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
 {
     return shared_page->vcpu_ioreq[i].vp_eport;
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 3445888e39..d70eacfbe2 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -50,4 +50,5 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 bool xen_mr_is_memory(MemoryRegion *mr);
+bool xen_mr_is_grants(MemoryRegion *mr);
 #endif
-- 
2.40.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:19:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:19:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732037.1137825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000qs-ES; Wed, 29 May 2024 14:19:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732037.1137825; Wed, 29 May 2024 14:19:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000qC-6C; Wed, 29 May 2024 14:19:56 +0000
Received: by outflank-mailman (input) for mailman id 732037;
 Wed, 29 May 2024 14:19:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCK9i-0000nP-Pz
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:19:54 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 844d1903-1dc6-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:19:53 +0200 (CEST)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-57a033c2e9fso1255235a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:19:53 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6389a0b4ccsm101767066b.22.2024.05.29.07.19.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:19:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 844d1903-1dc6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716992392; x=1717597192; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Rc/urejuK2M9KAkefkWK+X4luH5a6jraofBI99vzPZo=;
        b=e//jalF831zaKPqYUYFU5K8wvkCJ0CTEu7sQz/hh5Ru/yZqdhDy+kR03msssUBipsk
         vvpcLOq+INJA66UrPonON+waMSAvJHoGhzTaN7jbSU+Rk4rDqzRtIzEmyFgd29hN6vOX
         2z8M4wSO75ivYYA906e0c53DeonFL5DUzh9LE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716992392; x=1717597192;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Rc/urejuK2M9KAkefkWK+X4luH5a6jraofBI99vzPZo=;
        b=lhFOuPidZx/g9R5ZD91gDZxLvcfOJQQk4iKqygHo/dZbtKSgfmtNHzPwSdW9YTNzWC
         b5/2/g8XlxhkqlWyyTF9FH8BWquo+adLvUp1CeruAxaDzxY9xeP3/mNXjcpPFPuLZo9b
         74IiCi02bJClP+rQzdFCFFTabJwMPGOYkIhYpWslDmxENY2cZre1ArlnAuR96RIRbms8
         CmUeKU81U6JTc/53c8DO9/rpA1dSg3w/rDAj6f83cHr8TD4Oe7sVtw/5Rad1iOnCCWKa
         iCA3Njup0bYWLRoIQNpSaqQEoTBmDrLrN1skNhXI24x5/WEXOFEN3e5LC0WftqgMXjvt
         /MZw==
X-Gm-Message-State: AOJu0YzM+tSp+tbWkh4OgU48kDFbaBYsHMTp81tFsDRkRASgc/CpQ+pk
	OhzNxRg1sAU9EV5UxQCKBSQsImmvjnCFW3f1wY+xqEGdWOWm5MotUSQuGq1v1KZ/4sgJpsporjp
	6TK0=
X-Google-Smtp-Source: AGHT+IEqb5api7YOB5c5JcJp7ScU29OHf161yGSsdq3HrC5QI8Ff3C9+zcTXyrgoIB2eySMmpW8DjQ==
X-Received: by 2002:a17:907:7d9f:b0:a62:b36a:eafa with SMTP id a640c23a62f3a-a62b36b0213mr1634016866b.57.1716992392482;
        Wed, 29 May 2024 07:19:52 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for HW jobs
Date: Wed, 29 May 2024 15:19:43 +0100
Message-Id: <20240529141945.41669-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529141945.41669-1-andrew.cooper3@citrix.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This restriction doesn't provide any security because anyone with suitable
permissions on the HW runners can bypass it with this local patch.

Requiring branches to be protected hampers usability of transient testing
branches (specifically, can't delete branches except via the Gitlab UI).

Drop the requirement.

Fixes: 746774cd1786 ("automation: introduce a dom0less test run on Xilinx hardware")
Fixes: 0ab316e7e15f ("automation: add a smoke and suspend test on an Alder Lake system")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Fixes because this wants backporting, but it also needs acks from both Marek
and Stefano as the owners of the hardware in question.
---
 automation/gitlab-ci/test.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index ad249fa0a5d9..efd3ad46f08e 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -92,7 +92,7 @@
     when: always
   only:
     variables:
-      - $XILINX_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true"
+      - $XILINX_JOBS == "true"
   tags:
     - xilinx
 
@@ -112,7 +112,7 @@
     when: always
   only:
     variables:
-      - $QUBES_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true"
+      - $QUBES_JOBS == "true"
   tags:
     - qubes-hw2
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:19:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:19:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732036.1137818 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000nr-26; Wed, 29 May 2024 14:19:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732036.1137818; Wed, 29 May 2024 14:19:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9j-0000nk-VC; Wed, 29 May 2024 14:19:55 +0000
Received: by outflank-mailman (input) for mailman id 732036;
 Wed, 29 May 2024 14:19:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCK9i-0000nO-Eh
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:19:54 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 837de55b-1dc6-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:19:52 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a63359aaaa6so272480766b.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:19:52 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6389a0b4ccsm101767066b.22.2024.05.29.07.19.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:19:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 837de55b-1dc6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716992391; x=1717597191; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=b7NBtADKKSHisJ5YHk93xXrTguK6Evlu1C7r13rztjA=;
        b=jdYt7QCOa1Hb4yKWZ3HHhU8qqanq1LDljl7dBpMD0eF+Z+RjsqECZ56lrPtBhfdV28
         GQibWURRSp6zHvTrN/ocOObztzU3w+Cem86BwSKcYPYBMv9JZSNAO2wmfjaS5oi+IRoW
         bD1ejK0rF4MthFCzVmRWrDPZt/B7/dOyNWlI0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716992391; x=1717597191;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=b7NBtADKKSHisJ5YHk93xXrTguK6Evlu1C7r13rztjA=;
        b=atNHkYUMc2cmFIpBPllILEU6RzkoA2Qp2Z3W+UOZgjKYiWviz9eAXZj5oyGF23+fUF
         qouE5DCJK9NrVkt0GGPdL32+yDjAnrl1cKq6LqbtktoLU3wrHBi4MNTItQ3ohFB8/BfQ
         MVNYoAcUwv4aOT3xRbqJpMQ+NDZid+lT0qFcHmOFPFxr+Eahlix+6zoMk1rYPHE0fDbd
         OnFjdIpZBl3YD5ubtx29qlW0BBOmep6VNQFYmMcrwjOrwBesI7Zj52YNyACNjRAqAkkT
         1qMEXFAYS5WAbli1MwOo8jLXXVgcm9AQwU+hPGJLzkYg+F3roFHl+BdSXfZDNHhIy4xE
         iOtw==
X-Gm-Message-State: AOJu0Yzf58iNNTJIOO3UI9TGJ846YKzl4apnsT9kqNGg7DZmrwbPr4Cs
	+Z0sNK7nOEMl1mQKN8G85VQUk9AdYjkO/RM0gIxh8AOexfAmB4LlB8Dd9/tsg8YTmmeiWGhnX7D
	130k=
X-Google-Smtp-Source: AGHT+IFSeN1eQXlevEyGCfv4T2C28LpmyR94XuaLMZ86aaY2GVHGI+rA3LWvrFwzRHFBKCmmQlCz5g==
X-Received: by 2002:a17:906:13c1:b0:a59:c367:560c with SMTP id a640c23a62f3a-a6265148b73mr1026065666b.60.1716992391233;
        Wed, 29 May 2024 07:19:51 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH for-4.19 0/3] CI: Misc improvements
Date: Wed, 29 May 2024 15:19:42 +0100
Message-Id: <20240529141945.41669-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

All found while making extensive use of Gitlab CI for the bitops boot testing.

For 4.19.  It's all very low risk, and improves the utility/useability of our
testing.

Andrew Cooper (3):
  CI: Remove CI_COMMIT_REF_PROTECTED requirement for HW jobs
  CI: Use a debug build of Xen for the Xilinx HW tests
  CI: Improve serial handling in qemu-smoke-ppc64le.sh

 automation/gitlab-ci/test.yaml           |  8 ++++----
 automation/scripts/qemu-smoke-ppc64le.sh | 13 +++++++------
 2 files changed, 11 insertions(+), 10 deletions(-)

-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:19:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:19:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732038.1137829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000wl-KL; Wed, 29 May 2024 14:19:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732038.1137829; Wed, 29 May 2024 14:19:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000vR-EA; Wed, 29 May 2024 14:19:56 +0000
Received: by outflank-mailman (input) for mailman id 732038;
 Wed, 29 May 2024 14:19:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCK9j-0000nP-Eq
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:19:55 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 853ebdb4-1dc6-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:19:54 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f464so2789442a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:19:54 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6389a0b4ccsm101767066b.22.2024.05.29.07.19.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:19:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 853ebdb4-1dc6-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716992394; x=1717597194; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dYSXQhJ0PGNDlVpaDWx18D2dK2stcV3qob4/lvY8HMU=;
        b=N8XIv60u30rCL+EVRlh40On8PeAQPP/7JU4m3myfLz6G3x6aHdKSy278rzWtH8lkiQ
         hAeaVmpbTTrUEy30Wl3Mj0uZCEpjqqTcli37heorFOuCfjJfHkPo0rpEvCoEkdai0N19
         HLn65zOe5tb/uTIYWnzLKlceMZN5XbAz8zWHc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716992394; x=1717597194;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dYSXQhJ0PGNDlVpaDWx18D2dK2stcV3qob4/lvY8HMU=;
        b=S1r5qi8AkiU2E67K8itK/fTX4HAQlA8xOswvmghYiZWih/vWEeUSaLYX6ETjJGIqZu
         Iz4G/Rq3kmw3d0PK3Nl8xPUCxEQ7Z9ikUgeUoUATTfBlEBI1mLEx67Eh5avugKzKBd1V
         9D+2ZdW2+DYS4/ul3NSzyn3ggb+yCQdbuFmbsIJIitwpzRRqsX1GjMtxyq3KFnEFJkZA
         hSDVJLmcEKvDTaueh46T9ntraSdRYFcTvWP+G4EksimijB/xFFXdJrfUSYRGyjqrju9d
         71hBoG0GOH5yyMuK9uLb0fT0jphPd/X8JEnZ3b4hdZibG91c+2cXqz3gtLahEXjQPQ4/
         Tk+w==
X-Gm-Message-State: AOJu0YwNiwzGdUHmgyWwxdivlzBLUvMKBF8i3sssXg1Gjc/Yo4qggDg9
	pVA0V8JyHhLR/kgW+1Wl0ZOhl5+mGIRzYAfB3l34tUxEvKRQtVXtRWOOjdSOLPEDZCX3l55N7Yt
	OUnc=
X-Google-Smtp-Source: AGHT+IH8Mm/hWecuku0+y8gQZdQGlDm2/HgQ7UTncEgW22ljdqB4qeA1EPM0HiL6Nz9RQ80UF36lyQ==
X-Received: by 2002:a17:906:2516:b0:a63:535b:b316 with SMTP id a640c23a62f3a-a63535bb4a2mr285160066b.44.1716992394253;
        Wed, 29 May 2024 07:19:54 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 3/3] CI: Improve serial handling in qemu-smoke-ppc64le.sh
Date: Wed, 29 May 2024 15:19:45 +0100
Message-Id: <20240529141945.41669-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529141945.41669-1-andrew.cooper3@citrix.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Have PPC put serial to stdout like all other tests, so it shows up in the main
job log.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 automation/scripts/qemu-smoke-ppc64le.sh | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
index 2adbdac87ef5..701d81107d18 100755
--- a/automation/scripts/qemu-smoke-ppc64le.sh
+++ b/automation/scripts/qemu-smoke-ppc64le.sh
@@ -2,15 +2,15 @@
 
 set -ex
 
+serial_log="$(pwd)/smoke.serial"
+
 # machine type from first arg passed directly to qemu -M
 machine=$1
 
 # Run the test
-rm -f smoke.serial
+rm -f ${serial_log}
 set +e
 
-touch smoke.serial
-
 timeout -k 1 20 \
 binaries/qemu-system-ppc64 \
     -bios binaries/skiboot.lid \
@@ -20,9 +20,10 @@ binaries/qemu-system-ppc64 \
     -vga none \
     -monitor none \
     -nographic \
-    -serial file:smoke.serial \
-    -kernel binaries/xen
+    -serial stdio \
+    -kernel binaries/xen \
+    |& tee ${serial_log} | sed 's/\r//'
 
 set -e
-(grep -q "Hello, ppc64le!" smoke.serial) || exit 1
+(grep -q "Hello, ppc64le!" ${serial_log}) || exit 1
 exit 0
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:19:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:19:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732039.1137836 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000zK-SS; Wed, 29 May 2024 14:19:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732039.1137836; Wed, 29 May 2024 14:19:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCK9k-0000yE-Ks; Wed, 29 May 2024 14:19:56 +0000
Received: by outflank-mailman (input) for mailman id 732039;
 Wed, 29 May 2024 14:19:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCK9j-0000nO-Ln
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:19:55 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 84d5f838-1dc6-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:19:54 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-52ac0c4b62cso918754e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:19:54 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6389a0b4ccsm101767066b.22.2024.05.29.07.19.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:19:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84d5f838-1dc6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716992393; x=1717597193; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=P3GqAy/9EOMNyS8UbjPjYfePETa99PYyhqdiXPHDoPE=;
        b=TXYBGC+OEyDAMbhpRNI13SOTl0DoxbNcOAm0mBzh7ylI3V5SrGRVIrswusBTyhnOqU
         VGJlsP5iiZqxJBRbtd2+T0arSXjiOtq24V3W/UOG+098pnFlIqcWxuuDO4uMD5vc1H7D
         2URQcL0/BTG4Aar90PMNHbmpVReve7RYabzuI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716992393; x=1717597193;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=P3GqAy/9EOMNyS8UbjPjYfePETa99PYyhqdiXPHDoPE=;
        b=KHigry1sX0aBg4h8Ln2rIAfB3KOfnDr2VQFh/ljb37p/jRqmNvC+C5XbmBX37JPduN
         std/MjnnNpP6ScgKvDbLv09mPG9CQBl8tFB1CSYFMohZrDUP+rGnL7CszWNaglULNL6C
         ruTbgvdztB2mcTn2+shMe86ERELRXEO/0W6KilCBL53RXJ53wnvnfyEaxF9VG84zNpKj
         RvT4UM7rqbjIhdrMG0E2bC97jZN2ANOWBPgXi1kcC3+Zdrg/s8Wudr5KUF1/3MhZjeDL
         +jzSg6IRLni+l+rms4NQ0BPDJ31eCogZTuzgFFRN89VZiBqi21cztz7ZYUp9JffrtByu
         59XQ==
X-Gm-Message-State: AOJu0YyP7nCEvHae/AuL58pPY5YqSb3fLN3G3yFSxYGH/XSXprwAKf1r
	1FNptPovPD3XjwiTYTUVebptHV9gf7nlV8NX0Og44v6bfMcU/yg68YmNiEc8qYlP/O6p8z464en
	Ie3w=
X-Google-Smtp-Source: AGHT+IFHdcS1iAmyGtIA/4CVKsINRCA84cKkjw2uKIkYUASYk+PNJqQ3Mz8TSQNwvgUX+M8KtjRFfw==
X-Received: by 2002:ac2:5f65:0:b0:529:b634:bed1 with SMTP id 2adb3069b0e04-529b634bf97mr3873913e87.58.1716992393449;
        Wed, 29 May 2024 07:19:53 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH 2/3] CI: Use a debug build of Xen for the Xilinx HW tests
Date: Wed, 29 May 2024 15:19:44 +0100
Message-Id: <20240529141945.41669-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240529141945.41669-1-andrew.cooper3@citrix.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

... like the other hardware tests.  This gets more value out of the testing.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 automation/gitlab-ci/test.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index efd3ad46f08e..e96ccdfad54c 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
     - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.18-gcc-debug-arm64
 
 xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
   extends: .xilinx-arm64
@@ -157,7 +157,7 @@ xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
     - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh gem-passthrough 2>&1 | tee ${LOGFILE}
   needs:
     - *arm64-test-needs
-    - alpine-3.18-gcc-arm64
+    - alpine-3.18-gcc-debug-arm64
 
 adl-smoke-x86-64-gcc-debug:
   extends: .adl-x86-64
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:30:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732068.1137884 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKC-0006n4-Bj; Wed, 29 May 2024 14:30:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732068.1137884; Wed, 29 May 2024 14:30:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKC-0006mx-8M; Wed, 29 May 2024 14:30:44 +0000
Received: by outflank-mailman (input) for mailman id 732068;
 Wed, 29 May 2024 14:30:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKKB-0006Xm-4R
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:30:43 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07273fc0-1dc8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:30:42 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a62ef52e837so236580566b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:30:42 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937b92sm720180266b.68.2024.05.29.07.30.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:30:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07273fc0-1dc8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993041; x=1717597841; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OLhg3CnTs3yrTCfULMEgcsfOyxcHrWQPcRCfZl8y51o=;
        b=kAfA0mVn4zFL9VjrbVGUTMeFhuNGOafvVMG+d1rYfm6T425ZJiIU3oRNtzvK6p6KTr
         8A1iR43uuqRxxzDoLNq2v9s/uG3xjAsnaz0+50zfx7CCJvL9XWEo94gBpJ75DZrmLM7J
         7IlJqpvZXHRnV+1BblU14d5H++wYRt8vk0qNY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993041; x=1717597841;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OLhg3CnTs3yrTCfULMEgcsfOyxcHrWQPcRCfZl8y51o=;
        b=msXtEEmmMFMjleOO9wH2lIRBtOG3uUJQjnOQhIPBhpYQDKKDVwmmc3XNpx64eDqVoh
         meulTDQGoy5ZVmKfzt+JT0rS/+a+RJfQVJ14D1h6L6Y2PssRYeA4cE0zf1JpesIea8Kc
         nheMdwxlWqTu2Rvu4VddbbnEfVB7qxLMM6dGWwt9MBF/GN5BjVexDoIJpmA/yXN7D8G7
         TIOIRPy70xic8APgiTnzqtffY8dOBzS9p4Ktk9/6Eibh3wBK3AUJsNpHevm5WYRSakfo
         IzLI/qU2UeC/LWj1Q16dAAVGq5A66/pRhVQ35jBBLIV0Pp/TfjNVGtXu+LaIeoy45pjT
         wmRw==
X-Gm-Message-State: AOJu0Yy7DeEyWZl5Eg2aN5HUcbn/hL8ppFMUoMH7wAJMjfePGz/D9nU8
	qbch+YEntg2mUcr5vbuq4DrSKEA0sJlNPqqYla1E95jaQhXzNybxVXQkz+0zkt3fiZ+oXDOhWkm
	u
X-Google-Smtp-Source: AGHT+IGwXxQgnWzo5aeNTixO7+ynUQSRC4UA4W07qr2Zdq6yTTCP6CWsteUJQjhk8q1Ru8XYg2mhxg==
X-Received: by 2002:a17:906:f813:b0:a59:a85d:31c6 with SMTP id a640c23a62f3a-a62651123b4mr1050704066b.66.1716993041280;
        Wed, 29 May 2024 07:30:41 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v4 1/2] tools/xg: Streamline cpu policy serialise/deserialise calls
Date: Wed, 29 May 2024 15:30:37 +0100
Message-Id: <f456bfb8996bb1fd4b965755622cda6fcb61b297.1716992707.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716992707.git.alejandro.vallejo@cloud.com>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The idea is to use xc_cpu_policy_t as a single object containing both the
serialised and deserialised forms of the policy. Note that we need lengths
for the arrays, as the serialised policies may be shorter than the array
capacities.

* Add the serialised lengths to the struct so we can distinguish
  between length and capacity of the serialisation buffers.
* Remove explicit buffer+lengths in serialise/deserialise calls
  and use the internal buffer inside xc_cpu_policy_t instead.
* Refactor everything to use the new serialisation functions.
* Remove redundant serialization calls and avoid allocating dynamic
  memory aside from the policy objects in xen-cpuid. Also minor cleanup
  in the policy print call sites.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
v4:
  * Clarify with a comment the semantics of the serialised buffer accessors
  * Remove local variables in policy serialiser
---
 tools/include/xenguest.h            | 14 ++++-
 tools/libs/guest/xg_cpuid_x86.c     | 94 +++++++++++++++++++----------
 tools/libs/guest/xg_private.h       |  2 +
 tools/libs/guest/xg_sr_common_x86.c | 56 ++++++-----------
 tools/misc/xen-cpuid.c              | 41 ++++---------
 5 files changed, 108 insertions(+), 99 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index e01f494b772a..85d56f26537b 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -799,15 +799,23 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy);
 
 /* Manipulate a policy via architectural representations. */
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
 int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                                const xen_cpuid_leaf_t *leaves,
                                uint32_t nr);
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
                               const xen_msr_entry_t *msrs, uint32_t nr);
 
+/*
+ * Accessors for the serialised forms of the policy. The outputs are pointers
+ * into the policy object and not fresh allocations, so their lifetimes are tied
+ * to the policy object itself.
+ */
+int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
+int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs, uint32_t *nr);
+
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest);
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..6cab5c60bb41 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
-                              unsigned int nr_leaves, unsigned int nr_entries)
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
     int rc;
 
     rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    nr_leaves, &err_leaf, &err_subleaf);
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
     if ( rc )
     {
         if ( err_leaf != -1 )
@@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
     }
 
     rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  nr_entries, &err_msr);
+                                  policy->nr_msrs, &err_msr);
     if ( rc )
     {
         if ( err_msr != -1 )
@@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
-    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
     int rc;
 
-    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
-                                 policy->msrs, &nr_msrs);
+    rc = xc_cpu_policy_serialise(xch, policy);
     if ( rc )
         return rc;
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
-                                  nr_msrs, policy->msrs,
+    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
+                                  policy->nr_msrs, policy->msrs,
                                   &err_leaf, &err_subleaf, &err_msr);
     if ( rc )
     {
@@ -942,32 +944,26 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
     return rc;
 }
 
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
 {
     int rc;
+    p->nr_leaves = ARRAY_SIZE(p->leaves);
+    p->nr_msrs = ARRAY_SIZE(p->msrs);
 
-    if ( leaves )
+    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &p->nr_leaves);
+    if ( rc )
     {
-        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
-        if ( rc )
-        {
-            ERROR("Failed to serialize CPUID policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize CPUID policy");
+        errno = -rc;
+        return -1;
     }
 
-    if ( msrs )
+    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &p->nr_msrs);
+    if ( rc )
     {
-        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
-        if ( rc )
-        {
-            ERROR("Failed to serialize MSR policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize MSR policy");
+        errno = -rc;
+        return -1;
     }
 
     errno = 0;
@@ -1012,6 +1008,42 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
     return rc;
 }
 
+int xc_cpu_policy_get_leaves(xc_interface *xch,
+                             const xc_cpu_policy_t *policy,
+                             const xen_cpuid_leaf_t **leaves,
+                             uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch CPUID leaves from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *leaves = policy->leaves;
+    *nr = policy->nr_leaves;
+
+    return 0;
+}
+
+int xc_cpu_policy_get_msrs(xc_interface *xch,
+                           const xc_cpu_policy_t *policy,
+                           const xen_msr_entry_t **msrs,
+                           uint32_t *nr)
+{
+    if ( !policy )
+    {
+        ERROR("Failed to fetch MSRs from policy object");
+        errno = -EINVAL;
+        return -1;
+    }
+
+    *msrs = policy->msrs;
+    *nr = policy->nr_msrs;
+
+    return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest)
 {
diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
index d73947094f2e..a65dae818f3d 100644
--- a/tools/libs/guest/xg_private.h
+++ b/tools/libs/guest/xg_private.h
@@ -177,6 +177,8 @@ struct xc_cpu_policy {
     struct cpu_policy policy;
     xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
     xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
+    uint32_t nr_leaves;
+    uint32_t nr_msrs;
 };
 #endif /* x86 */
 
diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
index 563b4f016877..a0d67c3211c6 100644
--- a/tools/libs/guest/xg_sr_common_x86.c
+++ b/tools/libs/guest/xg_sr_common_x86.c
@@ -1,4 +1,5 @@
 #include "xg_sr_common_x86.h"
+#include "xg_sr_stream_format.h"
 
 int write_x86_tsc_info(struct xc_sr_context *ctx)
 {
@@ -45,54 +46,39 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
-    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
-    uint32_t nr_leaves = 0, nr_msrs = 0;
-    xc_cpu_policy_t *policy = NULL;
+    xc_cpu_policy_t *policy = xc_cpu_policy_init();
     int rc;
 
-    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
-    {
-        PERROR("Unable to get CPU Policy size");
-        return -1;
-    }
-
-    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
-    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
-    policy = xc_cpu_policy_init();
-    if ( !cpuid.data || !msrs.data || !policy )
-    {
-        ERROR("Cannot allocate memory for CPU Policy");
-        rc = -1;
-        goto out;
-    }
-
-    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
+    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
     {
         PERROR("Unable to get d%d CPU Policy", ctx->domid);
         rc = -1;
         goto out;
     }
-    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
-                                 msrs.data, &nr_msrs) )
-    {
-        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
-        rc = -1;
-        goto out;
-    }
 
-    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
-    if ( cpuid.length )
+
+    if ( policy->nr_leaves )
     {
-        rc = write_record(ctx, &cpuid);
+        struct xc_sr_record record = {
+            .type = REC_TYPE_X86_CPUID_POLICY,
+            .data = policy->leaves,
+            .length = policy->nr_leaves * sizeof(*policy->leaves),
+        };
+
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
 
-    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
-    if ( msrs.length )
+    if ( policy->nr_msrs )
     {
-        rc = write_record(ctx, &msrs);
+        struct xc_sr_record record = {
+            .type = REC_TYPE_X86_MSR_POLICY,
+            .data = policy->msrs,
+            .length = policy->nr_msrs * sizeof(*policy->msrs),
+        };
+
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
@@ -100,8 +86,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     rc = 0;
 
  out:
-    free(cpuid.data);
-    free(msrs.data);
     xc_cpu_policy_destroy(policy);
 
     return rc;
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 4c4593528dfe..488f43378406 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -156,12 +156,18 @@ static void dump_info(xc_interface *xch, bool detail)
 
     free(fs);
 }
-
-static void print_policy(const char *name,
-                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
-                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
+static void print_policy(xc_interface *xch, const char *name,
+                         const xc_cpu_policy_t *policy)
 {
     unsigned int l;
+    const xen_cpuid_leaf_t *leaves;
+    const xen_msr_entry_t *msrs;
+    uint32_t nr_leaves, nr_msrs;
+
+    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) )
+        err(1, "xc_cpu_policy_get_leaves()");
+    if ( xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
+        err(1, "xc_cpu_policy_get_msrs()");
 
     printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
     printf(" CPUID:\n");
@@ -287,8 +293,6 @@ int main(int argc, char **argv)
             [ XEN_SYSCTL_cpu_policy_pv_default ]   = "PV Default",
             [ XEN_SYSCTL_cpu_policy_hvm_default ]  = "HVM Default",
         };
-        xen_cpuid_leaf_t *leaves;
-        xen_msr_entry_t *msrs;
         uint32_t i, max_leaves, max_msrs;
 
         xc_interface *xch = xc_interface_open(0, 0, 0);
@@ -305,36 +309,21 @@ int main(int argc, char **argv)
             printf("Xen reports there are maximum %u leaves and %u MSRs\n",
                    max_leaves, max_msrs);
 
-        leaves = calloc(max_leaves, sizeof(xen_cpuid_leaf_t));
-        if ( !leaves )
-            err(1, "calloc(max_leaves)");
-        msrs = calloc(max_msrs, sizeof(xen_msr_entry_t));
-        if ( !msrs )
-            err(1, "calloc(max_msrs)");
-
         if ( domid != -1 )
         {
             char name[20];
-            uint32_t nr_leaves = max_leaves;
-            uint32_t nr_msrs = max_msrs;
 
             if ( xc_cpu_policy_get_domain(xch, domid, policy) )
                 err(1, "xc_cpu_policy_get_domain(, %d, )", domid);
-            if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                         msrs, &nr_msrs) )
-                err(1, "xc_cpu_policy_serialise");
 
             snprintf(name, sizeof(name), "Domain %d", domid);
-            print_policy(name, leaves, nr_leaves, msrs, nr_msrs);
+            print_policy(xch, name, policy);
         }
         else
         {
             /* Get system policies */
             for ( i = 0; i < ARRAY_SIZE(sys_policies); ++i )
             {
-                uint32_t nr_leaves = max_leaves;
-                uint32_t nr_msrs = max_msrs;
-
                 if ( xc_cpu_policy_get_system(xch, i, policy) )
                 {
                     if ( errno == EOPNOTSUPP )
@@ -346,18 +335,12 @@ int main(int argc, char **argv)
 
                     err(1, "xc_cpu_policy_get_system(, %s, )", sys_policies[i]);
                 }
-                if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                             msrs, &nr_msrs) )
-                    err(1, "xc_cpu_policy_serialise");
 
-                print_policy(sys_policies[i], leaves, nr_leaves,
-                             msrs, nr_msrs);
+                print_policy(xch, sys_policies[i], policy);
             }
         }
 
         xc_cpu_policy_destroy(policy);
-        free(leaves);
-        free(msrs);
         xc_interface_close(xch);
     }
     else if ( mode == MODE_INFO || mode == MODE_DETAIL )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:30:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732069.1137894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKE-00073c-Lp; Wed, 29 May 2024 14:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732069.1137894; Wed, 29 May 2024 14:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKE-00073V-Iy; Wed, 29 May 2024 14:30:46 +0000
Received: by outflank-mailman (input) for mailman id 732069;
 Wed, 29 May 2024 14:30:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKKD-00071F-K3
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:30:45 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0789a1e1-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:30:43 +0200 (CEST)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a6341cf2c99so225960566b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:30:43 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937b92sm720180266b.68.2024.05.29.07.30.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:30:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0789a1e1-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993042; x=1717597842; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ACDNCT9wciuPyxlqGAwOtlGDI3MKxbnMcgfQ+hZoPRk=;
        b=R6twMvs5QlWwQPwcQ1SiYP/aRjqQhY5umhorzvNzWqJYxXYVwLRcQ7G/koD8ft8SAX
         rAeXldHe9pfDfZdxW2ndLW5ErBMWUvu5CXBzkUIcZTFyR16/csVB2nZUKb+TPEmAI3UJ
         KzXVNy4Zrk0a7P4ThGbJIjj+wZUhlAxJXu+YY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993042; x=1717597842;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ACDNCT9wciuPyxlqGAwOtlGDI3MKxbnMcgfQ+hZoPRk=;
        b=wYPf3E4BxGALTYLl37qM1WeUS2WLxfRyy0b4qshiPm0cGJKe7eWkfhCZdZp1C0bc0n
         oRdEkvYP6EwhjDpgReL2rO4AapONADeslzvSnPxPJOmBpblJ7wgOi1wJGYdwEX2dKgfG
         5eYRjUElaswXk0WTvLPs+jQb2hiZs791ZOpWAFD3T6BSufcXq7I0+TOPLPM3cjTni+LI
         9DmzPgGZA+Br5FPQwtPedzxSCWbehjeSEe2hanmOGR/YgEJOzNZGj+xIg9rFSauwi43X
         WixEFLIxRpsWqzOKgZ5wYORcoVD7SbxR6305Ohab/KWtkZcSGvT/9/tnQ5Sza7xvNZLG
         TgqQ==
X-Gm-Message-State: AOJu0Yy0Bu9iGxwaHeYbnkztlp1sYl/YoGX1uGqBoOeK0qn8tVdYbTsW
	LbF5t7gi7OzrJcWbBvbicGk0Y5tXMU3yK/d7Akk+z2A5ssVOhi93WnQyGqZRj8wUUqTZlyfhgHi
	q
X-Google-Smtp-Source: AGHT+IEQpWFvsL1H2BKO/eU+9sH/Muc2mBce4oKTznbn9kp74NNLaZnaQfm805NjowAt1d5vQ+SeQA==
X-Received: by 2002:a17:906:c113:b0:a62:a9ca:2111 with SMTP id a640c23a62f3a-a62a9ca25admr1011211866b.11.1716993042076;
        Wed, 29 May 2024 07:30:42 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v4 2/2] tools/xg: Clean up xend-style overrides for CPU policies
Date: Wed, 29 May 2024 15:30:38 +0100
Message-Id: <ce6dc0221bbab353b6f47cf4bbbbcdf3aacc0cb8.1716992707.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716992707.git.alejandro.vallejo@cloud.com>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Factor out policy getters/setters from both (CPUID and MSR) policy override
functions. Additionally, use host policy rather than featureset when
preparing the cur policy, saving one hypercall and several lines of
boilerplate.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v4:
  * Indentation adjustment.
  * Fix unhandled corner case using bsearch() with MSR and leaf buffers
---
 tools/libs/guest/xg_cpuid_x86.c | 437 ++++++++++----------------------
 1 file changed, 130 insertions(+), 307 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 6cab5c60bb41..552ec2ab7312 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -36,6 +36,34 @@ enum {
 #define bitmaskof(idx)      (1u << ((idx) & 31))
 #define featureword_of(idx) ((idx) >> 5)
 
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
+{
+    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    int rc;
+
+    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
+    if ( rc )
+    {
+        if ( err_leaf != -1 )
+            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
+                  err_leaf, err_subleaf, -rc, strerror(-rc));
+        return rc;
+    }
+
+    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
+                                  policy->nr_msrs, &err_msr);
+    if ( rc )
+    {
+        if ( err_msr != -1 )
+            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
+                  err_msr, -rc, strerror(-rc));
+        return rc;
+    }
+
+    return 0;
+}
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
 {
     struct xen_sysctl sysctl = {};
@@ -260,102 +288,37 @@ static int compare_leaves(const void *l, const void *r)
     return 0;
 }
 
-static xen_cpuid_leaf_t *find_leaf(
-    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-    const struct xc_xend_cpuid *xend)
+static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
+                                   const struct xc_xend_cpuid *xend)
 {
     const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
 
-    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
+    return bsearch(&key, p->leaves, p->nr_leaves,
+                   sizeof(*p->leaves), compare_leaves);
 }
 
-static int xc_cpuid_xend_policy(
-    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
+static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
+                                const struct xc_xend_cpuid *xend,
+                                xc_cpu_policy_t *host,
+                                xc_cpu_policy_t *def,
+                                xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto fail;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -ENOMEM;
-    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        goto fail;
-    }
-
-    /* Get the domain's current policy. */
-    nr_msrs = 0;
-    nr_cur = nr_leaves;
-    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto fail;
-    }
+    if ( !xend )
+        return 0;
 
-    /* Get the domain type's default policy. */
-    nr_msrs = 0;
-    nr_def = nr_leaves;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_def, def, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto fail;
-    }
+    if ( !host || !def || !cur )
+        return -EINVAL;
 
-    /* Get the host policy. */
-    nr_msrs = 0;
-    nr_host = nr_leaves;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_host, host, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -EINVAL;
     for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
     {
-        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
-        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
-        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
+        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
+        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
+        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
 
         if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
         {
             ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
-            goto fail;
+            return -EINVAL;
         }
 
         for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
@@ -384,7 +347,7 @@ static int xc_cpuid_xend_policy(
                 {
                     ERROR("Bad character '%c' in policy[%d] string '%s'",
                           xend->policy[i][j], i, xend->policy[i]);
-                    goto fail;
+                    return -EINVAL;
                 }
 
                 clear_bit(31 - j, cur_reg);
@@ -394,25 +357,7 @@ static int xc_cpuid_xend_policy(
         }
     }
 
-    /* Feed the transformed currrent policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Success! */
-
- fail:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 static int compare_msr(const void *l, const void *r)
@@ -426,104 +371,37 @@ static int compare_msr(const void *l, const void *r)
     return lhs->idx < rhs->idx ? -1 : 1;
 }
 
-static xen_msr_entry_t *find_msr(
-    xen_msr_entry_t *msrs, unsigned int nr_msrs,
-    uint32_t index)
+static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
+                                 uint32_t index)
 {
     const xen_msr_entry_t key = { .idx = index };
 
-    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
+    return bsearch(&key, p->msrs, p->nr_msrs, sizeof(*p->msrs), compare_msr);
 }
 
-
 static int xc_msr_policy(xc_interface *xch, domid_t domid,
-                         const struct xc_msr *msr)
+                         const struct xc_msr *msr,
+                         xc_cpu_policy_t *host,
+                         xc_cpu_policy_t *def,
+                         xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto out;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto out;
-    }
-
-    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        rc = -ENOMEM;
-        goto out;
-    }
-
-    /* Get the domain's current policy. */
-    nr_leaves = 0;
-    nr_cur = nr_msrs;
-    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_leaves = 0;
-    nr_def = nr_msrs;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, NULL, &nr_def, def);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto out;
-    }
+    if ( !msr )
+        return 0;
 
-    /* Get the host policy. */
-    nr_leaves = 0;
-    nr_host = nr_msrs;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_leaves, NULL, &nr_host, host);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto out;
-    }
+    if ( !host || !def || !cur )
+        return -EINVAL;
 
     for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
     {
-        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
-        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
-        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
         unsigned int i;
+        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
+        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
+        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
 
         if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
         {
             ERROR("Missing MSR %#x", msr->index);
-            rc = -ENOENT;
-            goto out;
+            return -ENOENT;
         }
 
         for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
@@ -542,8 +420,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
             {
                 ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
                       msr->index, msr->policy[i], msr->policy);
-                rc = -EINVAL;
-                goto out;
+                return -EINVAL;
             }
 
             if ( val )
@@ -553,25 +430,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
         }
     }
 
-    /* Feed the transformed policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Success! */
-
- out:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
@@ -583,14 +442,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     int rc;
     bool hvm;
     xc_domaininfo_t di;
-    struct xc_cpu_policy *p = xc_cpu_policy_init();
-    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-    uint32_t len = ARRAY_SIZE(host_featureset);
+    unsigned int i;
 
-    if ( !p )
-        return -ENOMEM;
+    struct xc_cpu_policy *host = xc_cpu_policy_init();
+    struct xc_cpu_policy *def = xc_cpu_policy_init();
+    struct xc_cpu_policy *cur = xc_cpu_policy_init();
+
+    if ( !host || !def || !cur )
+    {
+        rc = -ENOMEM;
+        goto out;
+    }
 
     if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
     {
@@ -600,21 +462,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-    /* Get the host policy. */
-    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-                               &len, host_featureset);
-    /* Tolerate "buffer too small", as we've got the bits we need. */
-    if ( rc && errno != ENOBUFS )
+    /* Get the raw host policy */
+    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+    if ( rc )
     {
-        PERROR("Failed to obtain host featureset");
+        PERROR("Failed to obtain host policy");
         rc = -errno;
         goto out;
     }
 
     /* Get the domain's default policy. */
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, p->leaves, &nr_msrs, NULL);
+    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                                  def);
     if ( rc )
     {
         PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
@@ -622,14 +482,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
-                                    &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-              err_leaf, err_subleaf, -rc, strerror(-rc));
-        goto out;
-    }
+    /* Copy the deserialised default policy to modify it */
+    memcpy(cur, def, sizeof(*cur));
 
     if ( restore )
     {
@@ -647,18 +501,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * - Re-enable features which have become (possibly) off by default.
          */
 
-        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+        cur->policy.basic.rdrand = host->policy.basic.rdrand;
+        cur->policy.feat.hle = host->policy.feat.hle;
+        cur->policy.feat.rtm = host->policy.feat.rtm;
 
         if ( hvm )
         {
-            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+            cur->policy.feat.mpx = host->policy.feat.mpx;
         }
 
-        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
-        p->policy.feat.max_subleaf = 0;
-        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
+        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
+        cur->policy.feat.max_subleaf = 0;
+        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
     }
 
     if ( featureset )
@@ -702,17 +556,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
             }
         }
 
-        x86_cpu_featureset_to_policy(feat, &p->policy);
+        x86_cpu_featureset_to_policy(feat, &cur->policy);
     }
     else
     {
-        p->policy.extd.itsc = itsc;
+        cur->policy.extd.itsc = itsc;
 
         if ( hvm )
         {
-            p->policy.basic.pae = pae;
-            p->policy.basic.vmx = nested_virt;
-            p->policy.extd.svm = nested_virt;
+            cur->policy.basic.pae = pae;
+            cur->policy.basic.vmx = nested_virt;
+            cur->policy.extd.svm = nested_virt;
         }
     }
 
@@ -722,8 +576,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * On hardware without CPUID Faulting, PV guests see real topology.
          * As a consequence, they also need to see the host htt/cmp fields.
          */
-        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
-        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+        cur->policy.basic.htt       = host->policy.basic.htt;
+        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
     }
     else
     {
@@ -731,28 +585,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * Topology for HVM guests is entirely controlled by Xen.  For now, we
          * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
          */
-        p->policy.basic.htt = true;
-        p->policy.extd.cmp_legacy = false;
+        cur->policy.basic.htt = true;
+        cur->policy.extd.cmp_legacy = false;
 
         /*
          * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
          * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
          * overflow.
          */
-        if ( !p->policy.basic.lppp )
-            p->policy.basic.lppp = 2;
-        else if ( !(p->policy.basic.lppp & 0x80) )
-            p->policy.basic.lppp *= 2;
+        if ( !cur->policy.basic.lppp )
+            cur->policy.basic.lppp = 2;
+        else if ( !(cur->policy.basic.lppp & 0x80) )
+            cur->policy.basic.lppp *= 2;
 
-        switch ( p->policy.x86_vendor )
+        switch ( cur->policy.x86_vendor )
         {
         case X86_VENDOR_INTEL:
-            for ( i = 0; (p->policy.cache.subleaf[i].type &&
-                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
+            for ( i = 0; cur->policy.cache.subleaf[i].type &&
+                         i < ARRAY_SIZE(cur->policy.cache.raw); ++i )
             {
-                p->policy.cache.subleaf[i].cores_per_package =
-                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
-                p->policy.cache.subleaf[i].threads_per_cache = 0;
+                cur->policy.cache.subleaf[i].cores_per_package =
+                    (cur->policy.cache.subleaf[i].cores_per_package << 1) | 1;
+                cur->policy.cache.subleaf[i].threads_per_cache = 0;
             }
             break;
 
@@ -772,49 +626,46 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
              * apic_id_size values greater than 7.  Limit the value to
              * 7 for now.
              */
-            if ( p->policy.extd.nc < 0x7f )
+            if ( cur->policy.extd.nc < 0x7f )
             {
-                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
-                    p->policy.extd.apic_id_size++;
+                if ( cur->policy.extd.apic_id_size != 0 &&
+                     cur->policy.extd.apic_id_size < 0x7 )
+                    cur->policy.extd.apic_id_size++;
 
-                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
+                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
             }
             break;
         }
     }
 
-    nr_leaves = ARRAY_SIZE(p->leaves);
-    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
-    if ( rc )
+    if ( xend || msr )
     {
-        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-        goto out;
+        /* The overrides are over the serialised form of the policy */
+        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
+            goto out;
+
+        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
+            goto out;
+        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
+            goto out;
+
+        if ( (rc = deserialize_policy(xch, cur)) )
+            goto out;
     }
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, p->leaves, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
+    rc = xc_cpu_policy_set_domain(xch, domid, cur);
     if ( rc )
     {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
         rc = -errno;
         goto out;
     }
 
-    if ( xend && (rc = xc_cpuid_xend_policy(xch, domid, xend)) )
-        goto out;
-
-    if ( msr )
-    {
-        rc = xc_msr_policy(xch, domid, msr);
-        if ( rc )
-            goto out;
-    }
-
     rc = 0;
 
 out:
-    xc_cpu_policy_destroy(p);
+    xc_cpu_policy_destroy(def);
+    xc_cpu_policy_destroy(host);
+    xc_cpu_policy_destroy(cur);
 
     return rc;
 }
@@ -834,34 +685,6 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
-{
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    int rc;
-
-    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    policy->nr_leaves, &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        if ( err_leaf != -1 )
-            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-                  err_leaf, err_subleaf, -rc, strerror(-rc));
-        return rc;
-    }
-
-    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  policy->nr_msrs, &err_msr);
-    if ( rc )
-    {
-        if ( err_msr != -1 )
-            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
-                  err_msr, -rc, strerror(-rc));
-        return rc;
-    }
-
-    return 0;
-}
-
 int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t *policy)
 {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:30:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732067.1137874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKB-0006Yn-4A; Wed, 29 May 2024 14:30:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732067.1137874; Wed, 29 May 2024 14:30:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKKB-0006Yg-1W; Wed, 29 May 2024 14:30:43 +0000
Received: by outflank-mailman (input) for mailman id 732067;
 Wed, 29 May 2024 14:30:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKKA-0006Xm-FV
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:30:42 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06850f46-1dc8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:30:41 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a62972c88a9so272487466b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:30:41 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c937b92sm720180266b.68.2024.05.29.07.30.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:30:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06850f46-1dc8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993040; x=1717597840; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+yUT538RuJE2ZDH/10+V5jqjUG50quYq2RZl3j1igMg=;
        b=kBV4ppiFZbRuz9I8ER25BPGmPHFz8B9Hxi17jBr92dqML4ftUyi/LCA0NgPI+5hcgU
         oyG0mU0+C9A2kWcqZj1S3VUD2l/J3ZfHUyrAODaL6Ss6w3qTPuigDS8UqfGJ23P51mUN
         lNLAaf9B5YqFlPEBFHdVJmidANkm6sp7OdZLM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993040; x=1717597840;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=+yUT538RuJE2ZDH/10+V5jqjUG50quYq2RZl3j1igMg=;
        b=V78+JpqWf5uBGE1glrj7g35eGBI9HT2bhKwUrxS4jc7VypWvHGOHXiF//EjbFQS+lG
         bRrlgo1BD16GXlUZI/UVJ2eR0oqNp7eH6GQHoNxT6A3qMxfOfiSW4blR/b2TQoKCyL3i
         n+wLVYE6nvgioELM9wOfBidzfeaxePwJWekq0zkdT89UvRe+4tc/9wJT4c48ri0iHAhW
         EM4LxO2fBae/HboxeU1jcLY7x6xIFDpGHHNJywNVxbWyhFCU4ssYEGj6zFNy184PFTpQ
         BpAlJbJejOlJUZKIrZpioAiwb47M/h0tQ3bMnSAFuWM2I1ohvLz6pFE6NRu2++X2G+Uc
         C8wA==
X-Gm-Message-State: AOJu0Yzz6rdZaIxN/S2Pjl1SLPRFpKmYCpPWy+dpHKTCfvCuvJ8ZcCYr
	eIFdd4hppJ8pIxHqu+yUZAk8kv7sHEM2swAZf0DPfp/xIs7Mt2pc+qYGIMFckYvkaYArKgaiKPQ
	n
X-Google-Smtp-Source: AGHT+IFsXtn8pr9gBJIq4zfxnVRojgg1TjyaV7rfzONLW9FO5US8h8doGmzs5PcIryRjhidABCM8Yw==
X-Received: by 2002:a17:906:3695:b0:a5a:7cfe:30e9 with SMTP id a640c23a62f3a-a6264f01e40mr1054978566b.49.1716993040272;
        Wed, 29 May 2024 07:30:40 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v4 0/2] Clean the policy manipulation path in domain creation
Date: Wed, 29 May 2024 15:30:36 +0100
Message-Id: <cover.1716992707.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

v3 -> v4:
  * More changes in style and comments
  * Removal of unneeded local variables
  * Fixed unhandled corner cases in bsearch()

v2 -> v3:
  * Style adjustments
  * Revert of loop index scope refactors

v1 -> v2:
  * Removed xc_cpu_policy from xenguest.h (dropped v1/patch1)
  * Added accessors for xc_cpu_policy so the serialised form can be extracted.
  * Modified xen-cpuid to use accessors.

==== Original cover letter ====

In the context of creating a domain, we currently issue a lot of hypercalls
redundantly while populating its CPU policy; likely a side effect of
organic growth more than anything else.

However, the worst part is not the overhead (this is a glacially cold
path), but the insane amounts of boilerplate that make it really hard to
pick apart what's going on. One major contributor to this situation is the
fact that what's effectively "setup" and "teardown" phases in policy
manipulation are not factored out from the functions that perform said
manipulations, leading to the same getters and setter being invoked many
times, when once each would do.

Another big contributor is the code being unaware of when a policy is
serialised and when it's not.

This patch attempts to alleviate this situation, yielding over 200 LoC
reduction.

Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
         from clients of libxc/libxg.
Patch 2: Changes the (de)serialization wrappers in xenguest so they always
         serialise to/from the internal buffers of xc_cpu_policy_t. The
         struct is suitably expanded to hold extra information required.
Patch 3: Performs the refactor of the policy manipulation code so that it
         follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.

Alejandro Vallejo (2):
  tools/xg: Streamline cpu policy serialise/deserialise calls
  tools/xg: Clean up xend-style overrides for CPU policies

 tools/include/xenguest.h            |  14 +-
 tools/libs/guest/xg_cpuid_x86.c     | 525 ++++++++++------------------
 tools/libs/guest/xg_private.h       |   2 +
 tools/libs/guest/xg_sr_common_x86.c |  56 ++-
 tools/misc/xen-cpuid.c              |  41 +--
 5 files changed, 235 insertions(+), 403 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732082.1137904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKM9-0000FE-0u; Wed, 29 May 2024 14:32:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732082.1137904; Wed, 29 May 2024 14:32:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKM8-0000F5-U7; Wed, 29 May 2024 14:32:44 +0000
Received: by outflank-mailman (input) for mailman id 732082;
 Wed, 29 May 2024 14:32:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKM7-0000Ev-6T
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:43 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4eaa57e2-1dc8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:32:42 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-57a034697fbso1230661a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:42 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4eaa57e2-1dc8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993161; x=1717597961; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=9C8upn0+uPtzReijnF8q6aOxyYqZqHWB7OMGoHj4hy0=;
        b=LW4EvxFXPMO3E/5VReeEOIAXi5C5d+gXzoh7G9oKYM3pRDTOG7tKTIB5NCeBHaQWxH
         oPelFBuqApzHI/m/SA/sbNZVoZA/wSzC0gr3ticgaST8u9AjlVX0291c9vhS00lyTaPv
         HK8s9A+B8m+pWl7xX5xMjFpYBdUJ4uwrp3+a0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993161; x=1717597961;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=9C8upn0+uPtzReijnF8q6aOxyYqZqHWB7OMGoHj4hy0=;
        b=m/TlDy4BrQYyl9ZAD5LsGUjD4K4NIuoH533oRwvlvT0OAYqojN5RMrDNUvZ1zXCDit
         yawvSBflFH3EPcKKF9bhCgxrh+NH60l2ij0fZy+6rThy+sZ4Fkw+7BgGknblVQOR74fJ
         8CpSPb2HeghVAPqx7mDczOwU7IHiDnYsBKvJQzbCCJvR5s/w7sdqmClkObWGgSgIAWi9
         r7sVHJKiZsWZY0rEWFc4laPptfl98Vbq1S+gKppy6WqP+okqYF0Z2WY22Bme2ZDGr/h3
         PxsuiyePj7NfHK16upOp03c65YcL4oWOEDL1kPr0HybY35sf8gEfSN385rJ1yLEw13vt
         v/SQ==
X-Gm-Message-State: AOJu0YxaT9O2z2eWV7MW4L9qw+9oqz5XHbwgIDC1s3VNJScdh3OjC3RA
	pemm3UGoqeqdZV0iyoQ1Rjo4eYpPgD5u6cgrEHgCiF1REXAtO6IlG+BSn0eY2deGdsfphOJCX0t
	6
X-Google-Smtp-Source: AGHT+IEuF2nSH4c+npKYbSL026MzljXIKW9ismFoMwursBiPX1MK1SBRR3VOxnvzSvaSBhHniKsB/g==
X-Received: by 2002:a17:906:3614:b0:a62:8116:cb59 with SMTP id a640c23a62f3a-a628116cbf3mr894840166b.30.1716993161417;
        Wed, 29 May 2024 07:32:41 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3 0/6] x86: Expose consistent topology to guests
Date: Wed, 29 May 2024 15:32:29 +0100
Message-Id: <cover.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

v2 -> v3:

  (v2/patch2 and v2/patch4 are already committed)

  * Moved the vlapic check hook addition to v3/patch1
    * And created a check hook for the architectural state too for consistency.
  * Fixed migrations from Xen <= 4.13 by reconstructing the previous topology.
  * Correctly set the APIC ID after a policy change when vlapic is already in
    x2APIC mode.
  * Removed bogus assumption introduced in v1 and v2 on hvmloader about which
    8bit APIC IDs represent ids > 254. (it's "id % 0xff", not "min(id, 0xff)".
        * Used an x2apic flag check instead.
  * Various formatting adjustments.

v1 -> v2:

  * v1/patch 4 replaced by a different strategy (See patches 4 and 5 in v2):
      * Have hvmloader populate MADT with the real APIC IDs as read by the APs
        themselves rather than giving it knowledge on how to derive them.
  * Removed patches 2 and 3 in v1, as no longer relevant.
  * Split v1/patch6 in two parts ((a) creating the generator and (b) plugging it
    in) and use the generator in the unit tests of the vcpuid->apicid mapping
    function. Becomes patches 6 and 8 in v2.

  Patch 1: Same as v1/patch1.
  Patch 2: Header dependency cleanup in preparation for patch3.
  Patch 3: Adds vlapic_hidden check for the newly introduced reserved area.
  Patch 4: [hvmloader] Replaces INIT+SIPI+SIPI sequences with hypercalls.
  Patch 5: [hvmloader] Retrieve the per-CPU APIC IDs from the APs themselves.
  Patch 6: Split from v1/patch6.
  Patch 7: Logically matching v1/patch5, but using v2/patch6 for testing.
  Patch 8: Split from v1/patch6.

=== Original cover letter ===

Current topology handling is close to non-existent. As things stand, APIC
IDs are allocated through the apic_id=vcpu_id*2 relation without giving any
hints to the OS on how to parse the x2APIC ID of a given CPU and assuming
the guest will assume 2 threads per core.

This series involves bringing x2APIC IDs into the migration stream, so
older guests keep operating as they used to and enhancing Xen+toolstack so
new guests get topology information consistent with their x2APIC IDs. As a
side effect of this, x2APIC IDs are now packed and don't have (unless under
a pathological case) gaps.

Further work ought to allow combining this topology configurations with
gang-scheduling of guest hyperthreads into affine physical hyperthreads.
For the time being it purposefully keeps the configuration of "1 socket" +
"1 thread per core" + "1 core per vCPU".

Patch 1: Includes x2APIC IDs in the migration stream. This allows Xen to
         reconstruct the right x2APIC IDs on migrated-in guests, and
         future-proofs itself in the face of x2APIC ID derivation changes.
Patch 2: Minor refactor to expose xc_cpu_policy in libxl
Patch 3: Refactors xen/lib/x86 to work on non-Xen freestanding environments
         (e.g: hvmloader)
Patch 4: Remove old assumptions about vcpu_id<->apic_id relationship in hvmloader
Patch 5: Add logic to derive x2APIC IDs given a CPU policy and vCPU IDs
Patch 6: Includes a simple topology generator for toolstack so new guests
         have topologically consistent information in CPUID


Alejandro Vallejo (6):
  x86/vlapic: Move lapic migration checks to the check hooks
  xen/x86: Add initial x2APIC ID to the per-vLAPIC save area
  tools/hvmloader: Retrieve (x2)APIC IDs from the APs themselves
  xen/lib: Add topology generator for x86
  xen/x86: Derive topologically correct x2APIC IDs from the policy
  xen/x86: Synthesise domain topologies

 tools/firmware/hvmloader/config.h        |   6 +-
 tools/firmware/hvmloader/hvmloader.c     |   4 +-
 tools/firmware/hvmloader/smp.c           |  54 ++++--
 tools/include/xen-tools/common-macros.h  |   5 +
 tools/libs/guest/xg_cpuid_x86.c          |  24 ++-
 tools/tests/cpu-policy/test-cpu-policy.c | 201 +++++++++++++++++++++++
 xen/arch/x86/cpu-policy.c                |   9 +-
 xen/arch/x86/cpuid.c                     |  14 +-
 xen/arch/x86/hvm/vlapic.c                | 122 ++++++++++----
 xen/arch/x86/include/asm/hvm/hvm.h       |   2 +
 xen/arch/x86/include/asm/hvm/vlapic.h    |   2 +
 xen/include/public/arch-x86/hvm/save.h   |   2 +
 xen/include/xen/lib/x86/cpu-policy.h     |  27 +++
 xen/lib/x86/policy.c                     | 167 +++++++++++++++++++
 xen/lib/x86/private.h                    |   4 +
 15 files changed, 587 insertions(+), 56 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732083.1137914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMA-0000Uq-Ds; Wed, 29 May 2024 14:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732083.1137914; Wed, 29 May 2024 14:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMA-0000Uj-A9; Wed, 29 May 2024 14:32:46 +0000
Received: by outflank-mailman (input) for mailman id 732083;
 Wed, 29 May 2024 14:32:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKM9-0000Ev-LF
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:45 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 503a8b59-1dc8-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 16:32:45 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so2743066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:44 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 503a8b59-1dc8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993164; x=1717597964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gvIO42hHDCq3AD6mYH6mrssGWU1hiwJplrDubSM8t74=;
        b=Lg+yacAcZPg5aPhU8PbBBA7eD6VdZuHVfSZjTmE8ZVinA4YdELZ0HYjP8mjv/XqX/9
         gcIC1nf6wETlau5OoQDvwzI/yJoJP5F+jRD/haPipriCjSY/hXKh8yiOa+zUdBYtNsNZ
         tshlBGhUFOvZEL4NNU1g8sKDITOLrtPBD1FxE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993164; x=1717597964;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gvIO42hHDCq3AD6mYH6mrssGWU1hiwJplrDubSM8t74=;
        b=Pis9V+6qduTT+RUIVfP2AnEljj4Z/SYN5ij1MEiseCX0misqINkkfvMZFhy2GUjPET
         tZIVhahyt8k5AvygKwphz90QLaaac2+Q6Z1AJP0vsMWX21zMTt7owJ8bm0RDA7cmO9w9
         HwoC+7KXVOVab/veG1AeEweH/WX00v9fZvmzyZaqFdPVjb6g2yscr4JqM9r96iWDOqE8
         rvNRCgYc2PsTITfEydYJGiIEv9oAb3AiwrrVIZkJwCtoFyNMbVgO37Wr9IpXM/kxcgn2
         X31+MAfx094Ge9a63Vx7i+A9rXwH9hzhWacg6zraRHdMZvHbs0vkOYC0jjHTFe1IjKry
         /19Q==
X-Gm-Message-State: AOJu0YzMhiJPo4NUchhg+Yf1dOJufsx2P0sDcDR8leoSv15GLFYnbky8
	fXL0ZUVLDH2mtSDhG40/lm/qpRlBe6AWPko1jAuubiXJFevv8YKnESeHp17JmIzvp7JSiHU79Uk
	t
X-Google-Smtp-Source: AGHT+IGCDJIqKAXnBevXhyvx7XtqD/jOuMkVyGaO6Fhb1dyuw1DrLN5lXGDXAkLJhnzLaQ+P4T4SrQ==
X-Received: by 2002:a17:906:40c8:b0:a62:a48c:1123 with SMTP id a640c23a62f3a-a642d2775bamr207614566b.5.1716993164217;
        Wed, 29 May 2024 07:32:44 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC save area
Date: Wed, 29 May 2024 15:32:31 +0100
Message-Id: <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This allows the initial x2APIC ID to be sent on the migration stream. The
hardcoded mapping x2apic_id=2*vcpu_id is maintained for the time being.
Given the vlapic data is zero-extended on restore, fix up migrations from
hosts without the field by setting it to the old convention if zero.

x2APIC IDs are calculated from the CPU policy where the guest topology is
defined. For the time being, the function simply returns the old
relationship, but will eventually return results consistent with the
topology.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Added rsvd_zero check to the check hook (introduced in v3/patch1).
  * Set APIC ID properly during policy update if the APIC is already
    in x2apic mode, ensuring its LDR is updated too in that case.
  * Fixed typo in variable for x86_x2apic_id_from_vcpu_id().
    * Missed due to being mid-series.
  * Rewrote the comment on CPUID leaf 0xb.
  * Rewrote the comment on x86_x2apic_id_from_vcpu_id()
---
 xen/arch/x86/cpuid.c                   | 14 ++++-----
 xen/arch/x86/hvm/vlapic.c              | 41 ++++++++++++++++++++++++--
 xen/arch/x86/include/asm/hvm/hvm.h     |  2 ++
 xen/arch/x86/include/asm/hvm/vlapic.h  |  2 ++
 xen/include/public/arch-x86/hvm/save.h |  2 ++
 xen/include/xen/lib/x86/cpu-policy.h   |  9 ++++++
 xen/lib/x86/policy.c                   | 11 +++++++
 7 files changed, 70 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 7a38e032146a..ebcdbc5cbc5d 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -139,10 +139,9 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
         const struct cpu_user_regs *regs;
 
     case 0x1:
-        /* TODO: Rework topology logic. */
         res->b &= 0x00ffffffu;
         if ( is_hvm_domain(d) )
-            res->b |= (v->vcpu_id * 2) << 24;
+            res->b |= vlapic_x2apic_id(vcpu_vlapic(v)) << 24;
 
         /* TODO: Rework vPMU control in terms of toolstack choices. */
         if ( vpmu_available(v) &&
@@ -312,18 +311,15 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
 
     case 0xb:
         /*
-         * In principle, this leaf is Intel-only.  In practice, it is tightly
-         * coupled with x2apic, and we offer an x2apic-capable APIC emulation
-         * to guests on AMD hardware as well.
-         *
-         * TODO: Rework topology logic.
+         * Don't expose topology information to PV guests. Exposed on HVM
+         * along with x2APIC because they are tightly coupled.
          */
-        if ( p->basic.x2apic )
+        if ( is_hvm_domain(d) && p->basic.x2apic )
         {
             *(uint8_t *)&res->c = subleaf;
 
             /* Fix the x2APIC identifier. */
-            res->d = v->vcpu_id * 2;
+            res->d = vlapic_x2apic_id(vcpu_vlapic(v));
         }
         break;
 
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index a0df62b5ec0a..626a6258a4d4 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1072,7 +1072,7 @@ static uint32_t x2apic_ldr_from_id(uint32_t id)
 static void set_x2apic_id(struct vlapic *vlapic)
 {
     const struct vcpu *v = vlapic_vcpu(vlapic);
-    uint32_t apic_id = v->vcpu_id * 2;
+    uint32_t apic_id = vlapic->hw.x2apic_id;
     uint32_t apic_ldr = x2apic_ldr_from_id(apic_id);
 
     /*
@@ -1086,6 +1086,26 @@ static void set_x2apic_id(struct vlapic *vlapic)
     vlapic_set_reg(vlapic, APIC_LDR, apic_ldr);
 }
 
+void vlapic_cpu_policy_changed(struct vcpu *v)
+{
+    struct vlapic *vlapic = vcpu_vlapic(v);
+    const struct cpu_policy *cp = v->domain->arch.cpu_policy;
+
+    /*
+     * Don't override the initial x2APIC ID if we have migrated it or
+     * if the domain doesn't have vLAPIC at all.
+     */
+    if ( !has_vlapic(v->domain) || vlapic->loaded.hw )
+        return;
+
+    vlapic->hw.x2apic_id = x86_x2apic_id_from_vcpu_id(cp, v->vcpu_id);
+
+    if ( vlapic_x2apic_mode(vlapic) )
+        set_x2apic_id(vlapic); /* Set the APIC ID _and_ the LDR */
+    else
+        vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
+}
+
 int guest_wrmsr_apic_base(struct vcpu *v, uint64_t val)
 {
     const struct cpu_policy *cp = v->domain->arch.cpu_policy;
@@ -1452,7 +1472,7 @@ void vlapic_reset(struct vlapic *vlapic)
     if ( v->vcpu_id == 0 )
         vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
 
-    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
+    vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id));
     vlapic_do_init(vlapic);
 }
 
@@ -1520,6 +1540,16 @@ static void lapic_load_fixup(struct vlapic *vlapic)
     const struct vcpu *v = vlapic_vcpu(vlapic);
     uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id);
 
+    /*
+     * Loading record without hw.x2apic_id in the save stream, calculate using
+     * the traditional "vcpu_id * 2" relation. There's an implicit assumption
+     * that vCPU0 always has x2APIC0, which is true for the old relation, and
+     * still holds under the new x2APIC generation algorithm. While that case
+     * goes through the conditional it's benign because it still maps to zero.
+     */
+    if ( !vlapic->hw.x2apic_id )
+        vlapic->hw.x2apic_id = v->vcpu_id * 2;
+
     /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */
     if ( !vlapic_x2apic_mode(vlapic) ||
          (vlapic->loaded.ldr == good_ldr) )
@@ -1588,6 +1618,13 @@ static int cf_check lapic_check_hidden(const struct domain *d,
          APIC_BASE_EXTD )
         return -EINVAL;
 
+    /*
+     * Fail migrations from newer versions of Xen where
+     * rsvd_zero is interpreted as something else.
+     */
+    if ( s.rsvd_zero )
+        return -EINVAL;
+
     return 0;
 }
 
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index 1c01e22c8e62..746b4739f53f 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -16,6 +16,7 @@
 #include <asm/current.h>
 #include <asm/x86_emulate.h>
 #include <asm/hvm/asid.h>
+#include <asm/hvm/vlapic.h>
 
 struct pirq; /* needed by pi_update_irte */
 
@@ -448,6 +449,7 @@ static inline void hvm_update_guest_efer(struct vcpu *v)
 static inline void hvm_cpuid_policy_changed(struct vcpu *v)
 {
     alternative_vcall(hvm_funcs.cpuid_policy_changed, v);
+    vlapic_cpu_policy_changed(v);
 }
 
 static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset,
diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h
index 2c4ff94ae7a8..34f23cd38a20 100644
--- a/xen/arch/x86/include/asm/hvm/vlapic.h
+++ b/xen/arch/x86/include/asm/hvm/vlapic.h
@@ -44,6 +44,7 @@
 #define vlapic_xapic_mode(vlapic)                               \
     (!vlapic_hw_disabled(vlapic) && \
      !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD))
+#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id)
 
 /*
  * Generic APIC bitmap vector update & search routines.
@@ -107,6 +108,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack);
 
 int  vlapic_init(struct vcpu *v);
 void vlapic_destroy(struct vcpu *v);
+void vlapic_cpu_policy_changed(struct vcpu *v);
 
 void vlapic_reset(struct vlapic *vlapic);
 
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 7ecacadde165..1c2ec669ffc9 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -394,6 +394,8 @@ struct hvm_hw_lapic {
     uint32_t             disabled; /* VLAPIC_xx_DISABLED */
     uint32_t             timer_divisor;
     uint64_t             tdt_msr;
+    uint32_t             x2apic_id;
+    uint32_t             rsvd_zero;
 };
 
 DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic);
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index d5e447e9dc06..392320b9adbe 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -542,6 +542,15 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err);
 
+/**
+ * Calculates the x2APIC ID of a vCPU given a CPU policy
+ *
+ * @param p          CPU policy of the domain.
+ * @param id         vCPU ID of the vCPU.
+ * @returns x2APIC ID of the vCPU.
+ */
+uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index f033d22785be..b70b22d55fcf 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -2,6 +2,17 @@
 
 #include <xen/lib/x86/cpu-policy.h>
 
+uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
+{
+    /*
+     * TODO: Derive x2APIC ID from the topology information inside `p`
+     *       rather than from the vCPU ID alone. This bodge is a temporary
+     *       measure until all infra is in place to retrieve or derive the
+     *       initial x2APIC ID from migrated domains.
+     */
+    return id * 2;
+}
+
 int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732084.1137924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMB-0000lX-Mh; Wed, 29 May 2024 14:32:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732084.1137924; Wed, 29 May 2024 14:32:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMB-0000lM-HJ; Wed, 29 May 2024 14:32:47 +0000
Received: by outflank-mailman (input) for mailman id 732084;
 Wed, 29 May 2024 14:32:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKMA-0000Tu-6P
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:46 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4fceb73a-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:32:44 +0200 (CEST)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2e95a74d51fso4463131fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:44 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fceb73a-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993163; x=1717597963; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JQ0Z2MuZPai5+x8nYCnYhzQNJt6RhMWelwC5TWM5Ogs=;
        b=Cp8KFdxIfSXTeR3eYrG/dK5Z5vA92t89293nTSOlIEcgjvoenjQbdUY3S+TQOzyj5J
         ViDO4tT9JaODmNi8AOllIwSi74+v9GuJC9BR7nm2iTjwgkNvOWUGtsURj5gEwRQ4rOzh
         ls8z6EcQ/W/5dm639oxBD2Ax2Yuqup/GqMM9E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993163; x=1717597963;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JQ0Z2MuZPai5+x8nYCnYhzQNJt6RhMWelwC5TWM5Ogs=;
        b=lVyqGRGt8P9gw8JFXBHV+HUvZaeOzAv+UYnqMiQEHZTaYOXdJAmqHo3TRbrY8/IVkW
         Exbf044EQD4T3FdS3e1r3CvTpex8SJ0E5P9DzyHTk2tjz88Avwj96NeSVEW9iaEnJ6Lm
         ut+M7lI0LdJg5iKBhbzcFRkCuku3GeqQ6IwSMoaT7w/EjHgpeHq+N6Q0axNR9ouna59n
         kyYnXCa6yWQtw0IFRqm0JmLfRPbrgbi9VfrAx/kNAbjA11hdrbzK+vR38JTrrtQTtkvR
         t1l2o3o9BTDcRW5ZCI4xgTxOywRIxQlok2kNIqG7cIeuQ262FpWJUV1ItOOLbpKvsOdQ
         aH5g==
X-Gm-Message-State: AOJu0YzYet9Zbjb5hCRTfK2W+Ha5xo6UboGxHGd48QNdvVQFVyp+Zun9
	QVyzSGZjjII/pb9J8Ryv6kwta7UTBHd4Z/5rZ9FCS+HOIYORBp42/c3ZDZ73SxzmeY1hx9ojI7T
	U
X-Google-Smtp-Source: AGHT+IHDUsIhMRGeVQpyBeVL9vJxvadAtJqOlw/C7yhkmpleEPvJsfTSGOUs7kr2FkkLuVGGYY6P2Q==
X-Received: by 2002:a05:6512:23a0:b0:529:be5c:8de0 with SMTP id 2adb3069b0e04-529be5c8f07mr4543915e87.49.1716993162398;
        Wed, 29 May 2024 07:32:42 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 1/6] x86/vlapic: Move lapic migration checks to the check hooks
Date: Wed, 29 May 2024 15:32:30 +0100
Message-Id: <f87f27de6fadf896bf4d50c8a37e2b9b7201afb2.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

While doing this, factor out checks common to architectural and hidden state.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Moved from v2/patch3.
  * Added check hook for the architectural state as well.
  * Use domain_vcpu() rather than the previous open coded checks for vcpu range.
---
 xen/arch/x86/hvm/vlapic.c | 81 +++++++++++++++++++++++++--------------
 1 file changed, 53 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 9cfc82666ae5..a0df62b5ec0a 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1553,60 +1553,85 @@ static void lapic_load_fixup(struct vlapic *vlapic)
                v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
 }
 
-static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
-{
-    unsigned int vcpuid = hvm_load_instance(h);
-    struct vcpu *v;
-    struct vlapic *s;
 
+static int lapic_check_common(const struct domain *d, unsigned int vcpuid)
+{
     if ( !has_vlapic(d) )
         return -ENODEV;
 
     /* Which vlapic to load? */
-    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
+    if ( !domain_vcpu(d, vcpuid) )
     {
         dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
                 d->domain_id, vcpuid);
         return -EINVAL;
     }
-    s = vcpu_vlapic(v);
 
-    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
+    return 0;
+}
+
+static int cf_check lapic_check_hidden(const struct domain *d,
+                                       hvm_domain_context_t *h)
+{
+    unsigned int vcpuid = hvm_load_instance(h);
+    struct hvm_hw_lapic s;
+    int rc;
+
+    if ( (rc = lapic_check_common(d, vcpuid)) )
+        return rc;
+
+    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) != 0 )
+        return -ENODATA;
+
+    /* EN=0 with EXTD=1 is illegal */
+    if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) ==
+         APIC_BASE_EXTD )
         return -EINVAL;
 
+    return 0;
+}
+
+static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
+{
+    unsigned int vcpuid = hvm_load_instance(h);
+    struct vcpu *v = d->vcpu[vcpuid];
+    struct vlapic *s = vcpu_vlapic(v);
+
+    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
+        BUG();
+
     s->loaded.hw = 1;
     if ( s->loaded.regs )
         lapic_load_fixup(s);
 
-    if ( !(s->hw.apic_base_msr & APIC_BASE_ENABLE) &&
-         unlikely(vlapic_x2apic_mode(s)) )
-        return -EINVAL;
-
     hvm_update_vlapic_mode(v);
 
     return 0;
 }
 
-static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
+static int cf_check lapic_check_regs(const struct domain *d,
+                                     hvm_domain_context_t *h)
 {
     unsigned int vcpuid = hvm_load_instance(h);
-    struct vcpu *v;
-    struct vlapic *s;
+    int rc;
 
-    if ( !has_vlapic(d) )
-        return -ENODEV;
+    if ( (rc = lapic_check_common(d, vcpuid)) )
+        return rc;
 
-    /* Which vlapic to load? */
-    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
-    {
-        dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
-                d->domain_id, vcpuid);
-        return -EINVAL;
-    }
-    s = vcpu_vlapic(v);
+    if ( !hvm_get_entry(LAPIC_REGS, h) )
+        return -ENODATA;
+
+    return 0;
+}
+
+static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
+{
+    unsigned int vcpuid = hvm_load_instance(h);
+    struct vcpu *v = d->vcpu[vcpuid];
+    struct vlapic *s = vcpu_vlapic(v);
 
     if ( hvm_load_entry(LAPIC_REGS, h, s->regs) != 0 )
-        return -EINVAL;
+        BUG();
 
     s->loaded.id = vlapic_get_reg(s, APIC_ID);
     s->loaded.ldr = vlapic_get_reg(s, APIC_LDR);
@@ -1623,9 +1648,9 @@ static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
     return 0;
 }
 
-HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, NULL,
+HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, lapic_check_hidden,
                           lapic_load_hidden, 1, HVMSR_PER_VCPU);
-HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs, NULL,
+HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs, lapic_check_regs,
                           lapic_load_regs, 1, HVMSR_PER_VCPU);
 
 int vlapic_init(struct vcpu *v)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732085.1137934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMC-00012t-Tl; Wed, 29 May 2024 14:32:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732085.1137934; Wed, 29 May 2024 14:32:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMC-00012c-PB; Wed, 29 May 2024 14:32:48 +0000
Received: by outflank-mailman (input) for mailman id 732085;
 Wed, 29 May 2024 14:32:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKMB-0000Tu-FY
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:47 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 50b6373a-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:32:45 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-5789733769dso24214a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:45 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50b6373a-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993165; x=1717597965; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Ej2rPGHuJHoDIQ9G5DQv2lvCsMSRzdoop6CVxyhmmpE=;
        b=kfb3ILmihg9O58dmrYwLURACOw1LnUiC0e3/dSUNC4iGKQROEX1f9Xz9U9evB4QVHc
         7/N1Kc7Ud3xsduISzqeFRwVJsSzGfQrTuL92dyXQoPiOXTM1/Li/k+PRj+GZFmnlip2Y
         fcWRwA+jUr0opqKpRt4/foreLtCuIakE4YopY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993165; x=1717597965;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Ej2rPGHuJHoDIQ9G5DQv2lvCsMSRzdoop6CVxyhmmpE=;
        b=P1UYqOEOCPCLmiCU8/0mjXK20I/L1iHCOORnHBj2/itm2VzvSCUUykLdrSvzyGdqOY
         gPEZEq8laDpQPS0FwGUUD0KAjkJG8t4Pu5ZtBsdXGArKi6EDAQ80NVa/HHrc/bs7TtlC
         Lz44VaJjaTHRtQHdddFBXTHscchIAuztppqB6bfkHE/JWXsKSAq8ocqh8Io5EVeF+r9D
         qoYBYCeRLSS22xmxWnqSCI5/M7clqMiK5DzZ4mPu9inUbsnioM4TEI+7SpggXMbG0XaD
         o4bK96/WdldwMbEB5PMzL1bMFjPfJBDUrQHON821ich2L+KNJ1m8le7tYWuO/JXXPM4k
         haSA==
X-Gm-Message-State: AOJu0YyYI4QiI6BjuPm5l0HmQvu8vXgQ9hdpsaK36IKQT1C87QC5QUB7
	DWEHu9kokwlttuqdF6r1cAh5vHde4xyVthzcYMILofvPz4AwYVYCBqQ98rcSV6Vl5WatxFBRP18
	/
X-Google-Smtp-Source: AGHT+IHeS6/oDzvSmtImY2NDIfAASj8WslOmXwjKcdgQTIesJwXIWNp5FDLSzbKkBCsZx8a/GcTN4w==
X-Received: by 2002:a17:906:1555:b0:a5a:24ab:f5e with SMTP id a640c23a62f3a-a642d7d7473mr185870466b.25.1716993164998;
        Wed, 29 May 2024 07:32:44 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v3 3/6] tools/hvmloader: Retrieve (x2)APIC IDs from the APs themselves
Date: Wed, 29 May 2024 15:32:32 +0100
Message-Id: <eb7438639cc4755d3c959029d809451ee000ccbc.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to
populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs
from hvmloader.

While at this also remove ap_callin, as writing the APIC ID may serve the same
purpose.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Moved ACCESS_ONCE() to common-macros.h
  * 8bit APIC IDs clipping at 255 was a bogus assumption. Stop relying on it.
    * APIC ID taken from leaf 0xb instead when the x2apic feature is supported.
  * Assert APIC IDs read by the APs are never zero, as that's always the BSP.
  * Added comment about how CPU_TO_X2APICID serves as cross-vcpu synchronizer.
---
 tools/firmware/hvmloader/config.h       |  6 ++-
 tools/firmware/hvmloader/hvmloader.c    |  4 +-
 tools/firmware/hvmloader/smp.c          | 54 ++++++++++++++++++++-----
 tools/include/xen-tools/common-macros.h |  5 +++
 4 files changed, 56 insertions(+), 13 deletions(-)

diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
index cd716bf39245..213ac1f28e17 100644
--- a/tools/firmware/hvmloader/config.h
+++ b/tools/firmware/hvmloader/config.h
@@ -4,6 +4,8 @@
 #include <stdint.h>
 #include <stdbool.h>
 
+#include <xen/hvm/hvm_info_table.h>
+
 enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt };
 extern enum virtual_vga virtual_vga;
 
@@ -48,8 +50,10 @@ extern uint8_t ioapic_version;
 
 #define IOAPIC_ID           0x01
 
+extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
+
 #define LAPIC_BASE_ADDRESS  0xfee00000
-#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
+#define LAPIC_ID(vcpu_id)   (CPU_TO_X2APICID[(vcpu_id)])
 
 #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
 #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
index f8af88fabf24..5c02e8fc226a 100644
--- a/tools/firmware/hvmloader/hvmloader.c
+++ b/tools/firmware/hvmloader/hvmloader.c
@@ -341,11 +341,11 @@ int main(void)
 
     printf("CPU speed is %u MHz\n", get_cpu_mhz());
 
+    smp_initialise();
+
     apic_setup();
     pci_setup();
 
-    smp_initialise();
-
     perform_tests();
 
     if ( bios->bios_info_setup )
diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
index 5d46eee1c5f4..f878d91898bf 100644
--- a/tools/firmware/hvmloader/smp.c
+++ b/tools/firmware/hvmloader/smp.c
@@ -29,7 +29,34 @@
 
 #include <xen/vcpu.h>
 
-static int ap_callin;
+/**
+ * Lookup table of x2APIC IDs.
+ *
+ * Each entry is populated its respective CPU as they come online. This is required
+ * for generating the MADT with minimal assumptions about ID relationships.
+ */
+uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS];
+
+/** Tristate about x2apic being supported. -1=unknown */
+static int has_x2apic = -1;
+
+static uint32_t read_apic_id(void)
+{
+    uint32_t apic_id;
+
+    if ( has_x2apic )
+        cpuid(0xb, NULL, NULL, NULL, &apic_id);
+    else
+    {
+        cpuid(1, NULL, &apic_id, NULL, NULL);
+        apic_id >>= 24;
+    }
+
+    /* Never called by cpu0, so should never return 0 */
+    ASSERT(!apic_id);
+
+    return apic_id;
+}
 
 static void __attribute__((regparm(1))) cpu_setup(unsigned int cpu)
 {
@@ -37,13 +64,17 @@ static void __attribute__((regparm(1))) cpu_setup(unsigned int cpu)
     cacheattr_init();
     printf("done.\n");
 
-    if ( !cpu ) /* Used on the BSP too */
+    /* The BSP exits early because its APIC ID is known to be zero */
+    if ( !cpu )
         return;
 
     wmb();
-    ap_callin = 1;
+    ACCESS_ONCE(CPU_TO_X2APICID[cpu]) = read_apic_id();
 
-    /* After this point, the BSP will shut us down. */
+    /*
+     * After this point the BSP will shut us down. A write to
+     * CPU_TO_X2APICID[cpu] signals the BSP to bring down `cpu`.
+     */
 
     for ( ;; )
         asm volatile ( "hlt" );
@@ -54,10 +85,6 @@ static void boot_cpu(unsigned int cpu)
     static uint8_t ap_stack[PAGE_SIZE] __attribute__ ((aligned (16)));
     static struct vcpu_hvm_context ap;
 
-    /* Initialise shared variables. */
-    ap_callin = 0;
-    wmb();
-
     /* Wake up the secondary processor */
     ap = (struct vcpu_hvm_context) {
         .mode = VCPU_HVM_MODE_32B,
@@ -90,10 +117,11 @@ static void boot_cpu(unsigned int cpu)
         BUG();
 
     /*
-     * Wait for the secondary processor to complete initialisation.
+     * Wait for the secondary processor to complete initialisation,
+     * which is signaled by its x2APIC ID being written to the LUT.
      * Do not touch shared resources meanwhile.
      */
-    while ( !ap_callin )
+    while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) )
         cpu_relax();
 
     /* Take the secondary processor offline. */
@@ -104,6 +132,12 @@ static void boot_cpu(unsigned int cpu)
 void smp_initialise(void)
 {
     unsigned int i, nr_cpus = hvm_info->nr_vcpus;
+    uint32_t ecx;
+
+    cpuid(1, NULL, NULL, &ecx, NULL);
+    has_x2apic = (ecx >> 21) & 1;
+    if ( has_x2apic )
+        printf("x2APIC supported\n");
 
     printf("Multiprocessor initialisation:\n");
     cpu_setup(0);
diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
index 60912225cb7a..336c6309d96e 100644
--- a/tools/include/xen-tools/common-macros.h
+++ b/tools/include/xen-tools/common-macros.h
@@ -108,4 +108,9 @@
 #define get_unaligned(ptr)      get_unaligned_t(typeof(*(ptr)), ptr)
 #define put_unaligned(val, ptr) put_unaligned_t(typeof(*(ptr)), val, ptr)
 
+#define __ACCESS_ONCE(x) ({                             \
+            (void)(typeof(x))0; /* Scalar typecheck. */ \
+            (volatile typeof(x) *)&(x); })
+#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
+
 #endif	/* __XEN_TOOLS_COMMON_MACROS__ */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732086.1137944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKME-0001K4-A9; Wed, 29 May 2024 14:32:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732086.1137944; Wed, 29 May 2024 14:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKME-0001Jl-6j; Wed, 29 May 2024 14:32:50 +0000
Received: by outflank-mailman (input) for mailman id 732086;
 Wed, 29 May 2024 14:32:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKMC-0000Tu-OY
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:48 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5143fb92-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:32:46 +0200 (CEST)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a626919d19dso6782666b.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:46 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5143fb92-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993166; x=1717597966; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AqAozDSX51jFkv79KS5nld9q87yWir7rtgXqZSiRO/Y=;
        b=jwjrHmgT3SvP8MH5Ab4BbsudgDqgJ0MRPvzKJ38usk5t3grbSMg6OWm+5ubmXmK9e5
         c7vWafgVOyHm/kT2S0NiwUYpBavvJNM3YBMwOEcBzjQDDytidkb5cher0jypO1X8tefR
         iYzgIr4/9Uzy7aWsYha92iTTYQUPSWN/mvaow=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993166; x=1717597966;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=AqAozDSX51jFkv79KS5nld9q87yWir7rtgXqZSiRO/Y=;
        b=HArz5FSaChZFWqyUIOGcLsx+WI5iC0Pis8mYGTT+zMX+1oXhrHmR9V2lDNJRRRD5L3
         QBr0WPFXthH3F1NBQ8kr27B8gMfgDd3HumQrpkFYIPDAZyu5f4kNtf3cAaT/Q+Ty9PAA
         uNddU1yen0IS7M8zj4rnmwejHZno1OrjCdPN1lQQjSEBtn+sIOnTEUPKfrlt713WG3Uu
         QV2ndZk7tCnvXBsaaOStBGmQvcxIb5bFyoBBAbrHw4/38bAUPRBHRNMnDLu/WyekBMMo
         Fb7T+dW0FjDhlSfqRHo6Rb/kKDjOb1WZq5m9LE87Z7IMfkO/F7OcmjzcXY5W7KbZoxMT
         qbvA==
X-Gm-Message-State: AOJu0YxxaVeIgvJqdhd1HZjk6sEk3+dWAPSpGJzuFbdTe6DtDp3InE3r
	B40yHCgBWiJKkaALKbxNq38wXKBVGohYgpcyv1A8Hze5OkInSDS7imovCapR+m8UGWQhhPMOa7N
	0
X-Google-Smtp-Source: AGHT+IEOThwnXaeq9fshTzbk9nrW6TnAafCr7FaExi1BI368bZLSekGyXbBXb69LLRWXbZvmyqDphQ==
X-Received: by 2002:a17:906:348d:b0:a59:c62c:344d with SMTP id a640c23a62f3a-a642d37e41emr200072366b.9.1716993165836;
        Wed, 29 May 2024 07:32:45 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v3 4/6] xen/lib: Add topology generator for x86
Date: Wed, 29 May 2024 15:32:33 +0100
Message-Id: <22c291ff33d2fe88b92e24946304a73064cb247c.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a helper to populate topology leaves in the cpu policy from
threads/core and cores/package counts. It's unit-tested in test-cpu-policy.c,
but it's not connected to the rest of the code yet.

Adds the ASSERT() macro to xen/lib/x86/private.h, as it was missing.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Style adjustments (linewraps, newlines...)
  * Slight refactor of the TOPO() macro in unit tests.
  * Reduce indentation of x86_topo_from_parts().
  * Remove "no functional change" from commit message.
  * Assert n!=0 in clz(n)
    * Which implied adding the ASSERT() macro to private.h
---
 tools/tests/cpu-policy/test-cpu-policy.c | 133 +++++++++++++++++++++++
 xen/include/xen/lib/x86/cpu-policy.h     |  16 +++
 xen/lib/x86/policy.c                     |  90 +++++++++++++++
 xen/lib/x86/private.h                    |   4 +
 4 files changed, 243 insertions(+)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
index 301df2c00285..849d7cebaa7c 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -650,6 +650,137 @@ static void test_is_compatible_failure(void)
     }
 }
 
+static void test_topo_from_parts(void)
+{
+    static const struct test {
+        unsigned int threads_per_core;
+        unsigned int cores_per_pkg;
+        struct cpu_policy policy;
+    } tests[] = {
+        {
+            .threads_per_core = 3, .cores_per_pkg = 1,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
+                    { .nr_logical = 1, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 1, .cores_per_pkg = 3,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
+                    { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 7, .cores_per_pkg = 5,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
+                    { .nr_logical = 5, .level = 1, .type = 2, .id_shift = 6, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 2, .cores_per_pkg = 128,
+            .policy = {
+                .x86_vendor = X86_VENDOR_AMD,
+                .topo.subleaf = {
+                    { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
+                    { .nr_logical = 128, .level = 1, .type = 2,
+                      .id_shift = 8, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 3, .cores_per_pkg = 1,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, },
+                    { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 1, .cores_per_pkg = 3,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, },
+                    { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 7, .cores_per_pkg = 5,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, },
+                    { .nr_logical = 35, .level = 1, .type = 2, .id_shift = 6, },
+                },
+            },
+        },
+        {
+            .threads_per_core = 2, .cores_per_pkg = 128,
+            .policy = {
+                .x86_vendor = X86_VENDOR_INTEL,
+                .topo.subleaf = {
+                    { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, },
+                    { .nr_logical = 256, .level = 1, .type = 2,
+                      .id_shift = 8, },
+                },
+            },
+        },
+    };
+
+    printf("Testing topology synthesis from parts:\n");
+
+    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+    {
+        const struct test *t = &tests[i];
+        struct cpu_policy actual = { .x86_vendor = t->policy.x86_vendor };
+        int rc = x86_topo_from_parts(&actual, t->threads_per_core,
+                                     t->cores_per_pkg);
+
+        if ( rc || memcmp(&actual.topo, &t->policy.topo, sizeof(actual.topo)) )
+        {
+#define TOPO(n, f)  t->policy.topo.subleaf[(n)].f, actual.topo.subleaf[(n)].f
+            fail("FAIL[%d] - '%s %u t/c, %u c/p'\n",
+                 rc,
+                 x86_cpuid_vendor_to_str(t->policy.x86_vendor),
+                 t->threads_per_core, t->cores_per_pkg);
+            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
+                   "             expected_lvl=%u actual_lvl=%u\n"
+                   "             expected_type=%u actual_type=%u\n"
+                   "             expected_shift=%u actual_shift=%u\n",
+                   0,
+                   TOPO(0, nr_logical),
+                   TOPO(0, level),
+                   TOPO(0, type),
+                   TOPO(0, id_shift));
+
+            printf("  subleaf=%u  expected_n=%u actual_n=%u\n"
+                   "             expected_lvl=%u actual_lvl=%u\n"
+                   "             expected_type=%u actual_type=%u\n"
+                   "             expected_shift=%u actual_shift=%u\n",
+                   1,
+                   TOPO(1, nr_logical),
+                   TOPO(1, level),
+                   TOPO(1, type),
+                   TOPO(1, id_shift));
+#undef TOPO
+        }
+    }
+}
+
 int main(int argc, char **argv)
 {
     printf("CPU Policy unit tests\n");
@@ -667,6 +798,8 @@ int main(int argc, char **argv)
     test_is_compatible_success();
     test_is_compatible_failure();
 
+    test_topo_from_parts();
+
     if ( nr_failures )
         printf("Done: %u failures\n", nr_failures);
     else
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index 392320b9adbe..f5df18e9f77c 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -551,6 +551,22 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
  */
 uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id);
 
+/**
+ * Synthesise topology information in `p` given high-level constraints
+ *
+ * Topology is given in various fields accross several leaves, some of
+ * which are vendor-specific. This function uses the policy itself to
+ * derive such leaves from threads/core and cores/package.
+ *
+ * @param p                   CPU policy of the domain.
+ * @param threads_per_core    threads/core. Doesn't need to be a power of 2.
+ * @param cores_per_package   cores/package. Doesn't need to be a power of 2.
+ * @return                    0 on success; -errno on failure
+ */
+int x86_topo_from_parts(struct cpu_policy *p,
+                        unsigned int threads_per_core,
+                        unsigned int cores_per_pkg);
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index b70b22d55fcf..7709736a2812 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -13,6 +13,96 @@ uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
     return id * 2;
 }
 
+static unsigned int order(unsigned int n)
+{
+    ASSERT(n); /* clz(0) is UB */
+
+    return 8 * sizeof(n) - __builtin_clz(n);
+}
+
+int x86_topo_from_parts(struct cpu_policy *p,
+                        unsigned int threads_per_core,
+                        unsigned int cores_per_pkg)
+{
+    unsigned int threads_per_pkg = threads_per_core * cores_per_pkg;
+    unsigned int apic_id_size;
+
+    if ( !p || !threads_per_core || !cores_per_pkg )
+        return -EINVAL;
+
+    p->basic.max_leaf = MAX(0xb, p->basic.max_leaf);
+
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
+
+    /* thread level */
+    p->topo.subleaf[0].nr_logical = threads_per_core;
+    p->topo.subleaf[0].id_shift = 0;
+    p->topo.subleaf[0].level = 0;
+    p->topo.subleaf[0].type = 1;
+    if ( threads_per_core > 1 )
+        p->topo.subleaf[0].id_shift = order(threads_per_core - 1);
+
+    /* core level */
+    p->topo.subleaf[1].nr_logical = cores_per_pkg;
+    if ( p->x86_vendor == X86_VENDOR_INTEL )
+        p->topo.subleaf[1].nr_logical = threads_per_pkg;
+    p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift;
+    p->topo.subleaf[1].level = 1;
+    p->topo.subleaf[1].type = 2;
+    if ( cores_per_pkg > 1 )
+        p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1);
+
+    apic_id_size = p->topo.subleaf[1].id_shift;
+
+    /*
+     * Contrary to what the name might seem to imply. HTT is an enabler for
+     * SMP and there's no harm in setting it even with a single vCPU.
+     */
+    p->basic.htt = true;
+    p->basic.lppp = MIN(0xff, p->basic.lppp);
+
+    switch ( p->x86_vendor )
+    {
+    case X86_VENDOR_INTEL: {
+        struct cpuid_cache_leaf *sl = p->cache.subleaf;
+
+        for ( size_t i = 0; sl->type &&
+                            i < ARRAY_SIZE(p->cache.raw); i++, sl++ )
+        {
+            sl->cores_per_package = cores_per_pkg - 1;
+            sl->threads_per_cache = threads_per_core - 1;
+            if ( sl->type == 3 /* unified cache */ )
+                sl->threads_per_cache = threads_per_pkg - 1;
+        }
+        break;
+    }
+
+    case X86_VENDOR_AMD:
+    case X86_VENDOR_HYGON:
+        /* Expose p->basic.lppp */
+        p->extd.cmp_legacy = true;
+
+        /* Clip NC to the maximum value it can hold */
+        p->extd.nc = 0xff;
+        if ( threads_per_pkg <= 0xff )
+            p->extd.nc = threads_per_pkg - 1;
+
+        /* TODO: Expose leaf e1E */
+        p->extd.topoext = false;
+
+        /*
+         * Clip APIC ID to 8 bits, as that's what high core-count machines do.
+         *
+         * That's what AMD EPYC 9654 does with >256 CPUs.
+         */
+        p->extd.apic_id_size = MIN(8, apic_id_size);
+
+        break;
+    }
+
+    return 0;
+}
+
 int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err)
diff --git a/xen/lib/x86/private.h b/xen/lib/x86/private.h
index 60bb82a400b7..2ec9dbee33c2 100644
--- a/xen/lib/x86/private.h
+++ b/xen/lib/x86/private.h
@@ -4,6 +4,7 @@
 #ifdef __XEN__
 
 #include <xen/bitops.h>
+#include <xen/bug.h>
 #include <xen/guest_access.h>
 #include <xen/kernel.h>
 #include <xen/lib.h>
@@ -17,6 +18,7 @@
 
 #else
 
+#include <assert.h>
 #include <errno.h>
 #include <inttypes.h>
 #include <stdbool.h>
@@ -28,6 +30,8 @@
 
 #include <xen-tools/common-macros.h>
 
+#define ASSERT(x) assert(x)
+
 static inline bool test_bit(unsigned int bit, const void *vaddr)
 {
     const char *addr = vaddr;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732087.1137949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKME-0001MR-Le; Wed, 29 May 2024 14:32:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732087.1137949; Wed, 29 May 2024 14:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKME-0001LK-Fa; Wed, 29 May 2024 14:32:50 +0000
Received: by outflank-mailman (input) for mailman id 732087;
 Wed, 29 May 2024 14:32:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKMD-0000Tu-GG
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:49 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51ceb76e-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:32:47 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a63359aaaa6so274373666b.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:47 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51ceb76e-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993167; x=1717597967; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hgRshC8PIUKcDrs5c2zLkCCg7umtJyxW/HyRPKfHKPk=;
        b=OJkv0CUzDkTO2zcH2V3p4wXROqFdWmjNnd/tcAg3wzx5XrwIVaF31S2tKAp88xrCDM
         XckDpv4J2JQZTpvY+9kXcQ3ErfAzC2i8zafdcSN8itjCEoVmV/aZsH4B9wnIDsNLrCJF
         IPdmsdgLJf2ZmeqVPcYGVgFY2Kv9F0n9iq6Ck=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993167; x=1717597967;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=hgRshC8PIUKcDrs5c2zLkCCg7umtJyxW/HyRPKfHKPk=;
        b=OK+iFZ4wQ6NyTvKPCkaTm27zOj0zEHszbxdwXMRggIlVZkNk1D2aIHZ+ZqI7L+ToyS
         2+BQ3fkBQ5ukApKe4n6PEHRLVG53NSWYFTXkfgE2s2VTFykLPqrjQ9DmQLoh0Ax6Eg6y
         lxvykrRRKjMMNWYoR786xZgi2LNJ3ey/w8r94tNabg7aG/qPicVkps4cfKra3v2kgMbS
         mlRF53a20mhWkTJPfzeGp9BjSgjGvL0c8zxet+DnVNvyXqlBF98lZ1ORrP9BxZbklhD7
         kDQYJpFKV/SYWMjNwpm8pOg/HvvHtsN8TghGdT2Aj2vCY654G+y49VUIGRYTDICquZ2a
         aMlw==
X-Gm-Message-State: AOJu0YwdnGsnNEkDJVfMBJE/edB41FS5ob7jW6TaazKuh7alurzrNDyO
	uSsLUXsnzwY8wKslXEZ86NE7hO6sNA0/3x2lNCv/X+CM8xYPAjWk0s9ClTDAtdLcQTvlKrxjL/d
	x
X-Google-Smtp-Source: AGHT+IGjD11cLXk8jwlNmuLmPdqqildXHoRCJVxhX1JxaCN4sTVT/mV9v7sxuuC1d7V1axehTvoTqg==
X-Received: by 2002:a17:906:888:b0:a59:bc75:5000 with SMTP id a640c23a62f3a-a62642da520mr1056775166b.12.1716993166772;
        Wed, 29 May 2024 07:32:46 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>
Subject: [PATCH v3 5/6] xen/x86: Derive topologically correct x2APIC IDs from the policy
Date: Wed, 29 May 2024 15:32:34 +0100
Message-Id: <d3f016ebd656402f5b40cf55a51a7f970e6e69ec.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Implements the helper for mapping vcpu_id to x2apic_id given a valid
topology in a policy. The algo is written with the intention of extending
it to leaves 0x1f and extended 0x26 in the future.

Toolstack doesn't set leaf 0xb and the HVM default policy has it cleared,
so the leaf is not implemented. In that case, the new helper just returns
the legacy mapping.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Formatting adjustments.
  * Replace a conditional in x86_topo_from_parts() with MIN()
    * Was meant to happen in v2, but fell between the cracks.
  * Moved the `policy` variable to the inner scope so it's clean for every test.
  * Rewrote commit message to say "extended 0x26" rather than e26.
---
 tools/tests/cpu-policy/test-cpu-policy.c | 68 ++++++++++++++++++++++
 xen/include/xen/lib/x86/cpu-policy.h     |  2 +
 xen/lib/x86/policy.c                     | 73 ++++++++++++++++++++++--
 3 files changed, 138 insertions(+), 5 deletions(-)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c
index 849d7cebaa7c..e5f9b8f7ee39 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -781,6 +781,73 @@ static void test_topo_from_parts(void)
     }
 }
 
+static void test_x2apic_id_from_vcpu_id_success(void)
+{
+    static const struct test {
+        unsigned int vcpu_id;
+        unsigned int threads_per_core;
+        unsigned int cores_per_pkg;
+        uint32_t x2apic_id;
+        uint8_t x86_vendor;
+    } tests[] = {
+        {
+            .vcpu_id = 3, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 1 << 2,
+        },
+        {
+            .vcpu_id = 6, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 2 << 2,
+        },
+        {
+            .vcpu_id = 24, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = 1 << 5,
+        },
+        {
+            .vcpu_id = 35, .threads_per_core = 3, .cores_per_pkg = 8,
+            .x2apic_id = (35 % 3) | (((35 / 3) % 8) << 2) | ((35 / 24) << 5),
+        },
+        {
+            .vcpu_id = 96, .threads_per_core = 7, .cores_per_pkg = 3,
+            .x2apic_id = (96 % 7) | (((96 / 7) % 3) << 3) | ((96 / 21) << 5),
+        },
+    };
+
+    const uint8_t vendors[] = {
+        X86_VENDOR_INTEL,
+        X86_VENDOR_AMD,
+        X86_VENDOR_CENTAUR,
+        X86_VENDOR_SHANGHAI,
+        X86_VENDOR_HYGON,
+    };
+
+    printf("Testing x2apic id from vcpu id success:\n");
+
+    /* Perform the test run on every vendor we know about */
+    for ( size_t i = 0; i < ARRAY_SIZE(vendors); ++i )
+    {
+        for ( size_t j = 0; j < ARRAY_SIZE(tests); ++j )
+        {
+            struct cpu_policy policy = { .x86_vendor = vendors[i] };
+            const struct test *t = &tests[j];
+            uint32_t x2apic_id;
+            int rc = x86_topo_from_parts(&policy, t->threads_per_core,
+                                         t->cores_per_pkg);
+
+            if ( rc ) {
+                fail("FAIL[%d] - 'x86_topo_from_parts() failed", rc);
+                continue;
+            }
+
+            x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id);
+            if ( x2apic_id != t->x2apic_id )
+                fail("FAIL - '%s cpu%u %u t/c %u c/p'. bad x2apic_id: expected=%u actual=%u\n",
+                     x86_cpuid_vendor_to_str(policy.x86_vendor),
+                     t->vcpu_id, t->threads_per_core, t->cores_per_pkg,
+                     t->x2apic_id, x2apic_id);
+        }
+    }
+}
+
 int main(int argc, char **argv)
 {
     printf("CPU Policy unit tests\n");
@@ -799,6 +866,7 @@ int main(int argc, char **argv)
     test_is_compatible_failure();
 
     test_topo_from_parts();
+    test_x2apic_id_from_vcpu_id_success();
 
     if ( nr_failures )
         printf("Done: %u failures\n", nr_failures);
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index f5df18e9f77c..2cbc2726a861 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -545,6 +545,8 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
 /**
  * Calculates the x2APIC ID of a vCPU given a CPU policy
  *
+ * If the policy lacks leaf 0xb falls back to legacy mapping of apic_id=cpu*2
+ *
  * @param p          CPU policy of the domain.
  * @param id         vCPU ID of the vCPU.
  * @returns x2APIC ID of the vCPU.
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index 7709736a2812..239386b71769 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -2,15 +2,78 @@
 
 #include <xen/lib/x86/cpu-policy.h>
 
+static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
+{
+    /*
+     * `nr_logical` reported by Intel is the number of THREADS contained in
+     * the next topological scope. For example, assuming a system with 2
+     * threads/core and 3 cores/module in a fully symmetric topology,
+     * `nr_logical` at the core level will report 6. Because it's reporting
+     * the number of threads in a module.
+     *
+     * On AMD/Hygon, nr_logical is already normalized by the higher scoped
+     * level (cores/complex, etc) so we can return it as-is.
+     */
+    if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
+        return p->topo.subleaf[lvl].nr_logical;
+
+    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;
+}
+
 uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
 {
+    uint32_t shift = 0, x2apic_id = 0;
+
+    /* In the absence of topology leaves, fallback to traditional mapping */
+    if ( !p->topo.subleaf[0].type )
+        return id * 2;
+
     /*
-     * TODO: Derive x2APIC ID from the topology information inside `p`
-     *       rather than from the vCPU ID alone. This bodge is a temporary
-     *       measure until all infra is in place to retrieve or derive the
-     *       initial x2APIC ID from migrated domains.
+     * `id` means different things at different points of the algo
+     *
+     * At lvl=0: global thread_id (same as vcpu_id)
+     * At lvl=1: global core_id
+     * At lvl=2: global socket_id (actually complex_id in AMD, module_id
+     *                             in Intel, but the name is inconsequential)
+     *
+     *                 +--+
+     *            ____ |#0| ______           <= 1 socket
+     *           /     +--+       \+--+
+     *       __#0__              __|#1|__    <= 2 cores/socket
+     *      /   |  \        +--+/  +-|+  \
+     *    #0   #1   #2      |#3|    #4    #5 <= 3 threads/core
+     *                      +--+
+     *
+     * ... and so on. Global in this context means that it's a unique
+     * identifier for the whole topology, and not relative to the level
+     * it's in. For example, in the diagram shown above, we're looking at
+     * thread #3 in the global sense, though it's #0 within its core.
+     *
+     * Note that dividing a global thread_id by the number of threads per
+     * core returns the global core id that contains it. e.g: 0, 1 or 2
+     * divided by 3 returns core_id=0. 3, 4 or 5 divided by 3 returns core
+     * 1, and so on. An analogous argument holds for higher levels. This is
+     * the property we exploit to derive x2apic_id from vcpu_id.
+     *
+     * NOTE: `topo` is currently derived from leaf 0xb, which is bound to two
+     * levels, but once we track leaves 0x1f (or extended 0x26) there will be a
+     * few more. The algorithm is written to cope with that case.
      */
-    return id * 2;
+    for ( uint32_t i = 0; i < ARRAY_SIZE(p->topo.raw); i++ )
+    {
+        uint32_t nr_parts;
+
+        if ( !p->topo.subleaf[i].type )
+            /* sentinel subleaf */
+            break;
+
+        nr_parts = parts_per_higher_scoped_level(p, i);
+        x2apic_id |= (id % nr_parts) << shift;
+        id /= nr_parts;
+        shift = p->topo.subleaf[i].id_shift;
+    }
+
+    return (id << shift) | x2apic_id;
 }
 
 static unsigned int order(unsigned int n)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:32:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:32:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732088.1137953 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKMF-0001T8-0s; Wed, 29 May 2024 14:32:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732088.1137953; Wed, 29 May 2024 14:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKME-0001Sk-QU; Wed, 29 May 2024 14:32:50 +0000
Received: by outflank-mailman (input) for mailman id 732088;
 Wed, 29 May 2024 14:32:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCKMD-0000Tu-TP
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:32:49 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5223fbe6-1dc8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:32:48 +0200 (CEST)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a5a89787ea4so230993566b.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:32:48 -0700 (PDT)
Received: from EMEAENGAAD19049.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a647b827400sm74614166b.69.2024.05.29.07.32.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:32:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5223fbe6-1dc8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716993167; x=1717597967; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=V5Itkas4jSxEAne1qNgyzUKuJE5iH1oRoQ5V5cxab3I=;
        b=D/p9ABnMHMSmnyXHpGToXGTmtf2KeovJ2ytBtWPyoo2zTjwIEBsq2hWbyWZ8CBAHMR
         dJLX/g4A1sOG2PnOiN8t8t7LC+7/x84sBBGHExzU7gZ0hdJEhR1rRWPpvsH3XeDe2GFa
         ZUAgKp3dVT+iJEKsjszcvCLOyAP8QyqcDiZwk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716993167; x=1717597967;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=V5Itkas4jSxEAne1qNgyzUKuJE5iH1oRoQ5V5cxab3I=;
        b=Iq/cVy6TrMsAYHZVaWWJfgFX7ojptSDWOQ8xr6R4i0ZLutw+OMOkHTQkDGgM4zX8l+
         4/iZRAJocSFZ88JImsOJugMBxuzUKKd9axgXcBJEyYmfeG8jINAGVf6r6l4uHjMspgSx
         JZB4DERiogv4CnYqVh4Z/4GZY4PIlSkfMcANuVWI8MSO/Rns6aEjmNRUPEXWKsrxO80T
         RgL0kFQjI8d+90j9n5frW9MlOpZT9TA21jktuoJpxiBpsl6zIPPqP0o9FOuy8zrA5yQ4
         a8KnNMnb6NxXahXmBsrnE2EhYuZRkB+U6UPjyq6fpVVEj29MzcURRZUNuE3wbROV/I4I
         8X6w==
X-Gm-Message-State: AOJu0YyIa0375UDFQjWb+DV6GKKOCtjftuDY+lgAZgMBTgnjJhHAmxh3
	WLyDK4nXs3RxGp05lxKXhVc7fEtCim+KOW8BefXdtWjYp6iL7C6EnxJWnUE/Qu6WIpnSZwUfi1w
	c
X-Google-Smtp-Source: AGHT+IHPnuAu70v7eX7J2gWR++BS4a0v2Gol7RrzKimIuNA8uGHadRzfXiOzDpkwtstDZCJyuaoerg==
X-Received: by 2002:a17:906:3c4b:b0:a59:c52b:9933 with SMTP id a640c23a62f3a-a62643e0792mr936602766b.30.1716993167521;
        Wed, 29 May 2024 07:32:47 -0700 (PDT)
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 6/6] xen/x86: Synthesise domain topologies
Date: Wed, 29 May 2024 15:32:35 +0100
Message-Id: <b2d6a6e64c0a84ca272deffce952798a33b53097.1716976271.git.alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1716976271.git.alejandro.vallejo@cloud.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Expose sensible topologies in leaf 0xb. At the moment it synthesises non-HT
systems, in line with the previous code intent.

Leaf 0xb in the host policy is no longer zapped and the guest {max,def} policies
have their topology leaves zapped instead. The intent is for toolstack to
populate them. There's no current use for the topology information in the host
policy, but it makes no harm.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
v3:
  * Formatting adjustments.
  * Restored previous topology logic and gated it through the "restore" variable
  * Print return code on topo generation failures.
  * Adjusted comment on wiping the topology in the guest policies.
  * Described the changes in topology zero-out in the commit message.
---
 tools/libs/guest/xg_cpuid_x86.c | 24 +++++++++++++++++++++++-
 xen/arch/x86/cpu-policy.c       |  9 ++++++---
 xen/lib/x86/policy.c            |  9 ++++++---
 3 files changed, 35 insertions(+), 7 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..6062dcab01ce 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -725,8 +725,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
         p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
     }
-    else
+    else if ( restore )
     {
+        /*
+         * Reconstruct the topology exposed on Xen <= 4.13. It makes very little
+         * sense, but it's what those guests saw so it's set in stone now.
+         *
+         * Guests from Xen 4.14 onwards carry their own CPUID leaves in the
+         * migration stream so they don't need special treatment.
+         */
+
         /*
          * Topology for HVM guests is entirely controlled by Xen.  For now, we
          * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
@@ -782,6 +790,20 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
             break;
         }
     }
+    else
+    {
+        /* TODO: Expose the ability to choose a custom topology for HVM/PVH */
+        unsigned int threads_per_core = 1;
+        unsigned int cores_per_pkg = di.max_vcpu_id + 1;
+
+        rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg);
+        if ( rc )
+        {
+            ERROR("Failed to generate topology: rc=%d t/c=%u c/p=%u",
+                  rc, threads_per_core, cores_per_pkg);
+            goto out;
+        }
+    }
 
     nr_leaves = ARRAY_SIZE(p->leaves);
     rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index b96f4ee55cc4..ecbe98302df2 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -278,9 +278,6 @@ static void recalculate_misc(struct cpu_policy *p)
 
     p->basic.raw[0x8] = EMPTY_LEAF;
 
-    /* TODO: Rework topology logic. */
-    memset(p->topo.raw, 0, sizeof(p->topo.raw));
-
     p->basic.raw[0xc] = EMPTY_LEAF;
 
     p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES;
@@ -628,6 +625,9 @@ static void __init calculate_pv_max_policy(void)
     recalculate_xstate(p);
 
     p->extd.raw[0xa] = EMPTY_LEAF; /* No SVM for PV guests. */
+
+    /* Wipe host topology. Populated by toolstack */
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
 }
 
 static void __init calculate_pv_def_policy(void)
@@ -791,6 +791,9 @@ static void __init calculate_hvm_max_policy(void)
 
     /* It's always possible to emulate CPUID faulting for HVM guests */
     p->platform_info.cpuid_faulting = true;
+
+    /* Wipe host topology. Populated by toolstack */
+    memset(p->topo.raw, 0, sizeof(p->topo.raw));
 }
 
 static void __init calculate_hvm_def_policy(void)
diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
index 239386b71769..01b9ed39d597 100644
--- a/xen/lib/x86/policy.c
+++ b/xen/lib/x86/policy.c
@@ -2,7 +2,8 @@
 
 #include <xen/lib/x86/cpu-policy.h>
 
-static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t lvl)
+static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p,
+                                              size_t lvl)
 {
     /*
      * `nr_logical` reported by Intel is the number of THREADS contained in
@@ -17,10 +18,12 @@ static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, size_t
     if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl )
         return p->topo.subleaf[lvl].nr_logical;
 
-    return p->topo.subleaf[lvl].nr_logical / p->topo.subleaf[lvl - 1].nr_logical;
+    return p->topo.subleaf[lvl].nr_logical /
+           p->topo.subleaf[lvl - 1].nr_logical;
 }
 
-uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
+uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p,
+                                    uint32_t id)
 {
     uint32_t shift = 0, x2apic_id = 0;
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed May 29 14:38:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:38:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732122.1137973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKRo-0004Y8-1s; Wed, 29 May 2024 14:38:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732122.1137973; Wed, 29 May 2024 14:38:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKRn-0004Y1-VN; Wed, 29 May 2024 14:38:35 +0000
Received: by outflank-mailman (input) for mailman id 732122;
 Wed, 29 May 2024 14:38:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCKRn-0004Xp-EA; Wed, 29 May 2024 14:38:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCKRn-0000Ti-C0; Wed, 29 May 2024 14:38:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCKRn-0007XJ-0O; Wed, 29 May 2024 14:38:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCKRm-00077n-W2; Wed, 29 May 2024 14:38:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Avqw/djpYXcHBScX23cMUjPmPnnpbi1v0hp7wzJFAFY=; b=mmkIHS87cCa/o4QKxq7gdABXPG
	56HVjJdZDOQscjHpOsD9I9JnpXGMSObV7wnmYSDQV7ZYH0bRrElgjlfkfa1+eFhD/pvABNVgDnooP
	jyiIGz5gxVYx+SFmwkJfbSZ5jHoigdPqWAlJonbZo5CXwMak+1z4f1trP9u/x+WG0KC0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186180-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186180: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d27c2835e0005200d464b57156c76455d46f74bb
X-Osstest-Versions-That:
    xen=2d93f78bfe25f695d8ffb61d110da9df293ed71b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 14:38:34 +0000

flight 186180 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186180/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186175
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186175
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186175
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186175
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186175
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186175
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d27c2835e0005200d464b57156c76455d46f74bb
baseline version:
 xen                  2d93f78bfe25f695d8ffb61d110da9df293ed71b

Last test of basis   186175  2024-05-28 19:08:59 Z    0 days
Testing same since   186180  2024-05-29 05:58:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2d93f78bfe..d27c2835e0  d27c2835e0005200d464b57156c76455d46f74bb -> master


From xen-devel-bounces@lists.xenproject.org Wed May 29 14:58:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 14:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732146.1137999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKl5-0002Jx-OF; Wed, 29 May 2024 14:58:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732146.1137999; Wed, 29 May 2024 14:58:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKl5-0002Jq-LC; Wed, 29 May 2024 14:58:31 +0000
Received: by outflank-mailman (input) for mailman id 732146;
 Wed, 29 May 2024 14:58:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCKl4-0002Jk-MD
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 14:58:30 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e83abd71-1dcb-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 16:58:28 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a62ef52e837so239979966b.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 07:58:28 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a65ab5c9009sm34339766b.116.2024.05.29.07.58.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 07:58:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e83abd71-1dcb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1716994708; x=1717599508; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=BNViD2QjGbbdDZhkRCgcqFaqgTYj5buD4YzpkiaWovc=;
        b=f92myGlXfNyZ9ZAazv4Un8OvqoLONPNjsJkYXDt5St2+3hQwT01cUZT9R0YBpuuQ0R
         5Rm8U0th6o6eU6HrBu55mArtwvvHVNzhtEZRGHcUgu4vDiq5NMPPsDv8zfYaSuyayCtv
         ZYXH0F0JpWX2NgguSIKVaf7tGYeZLgAZmw+23ePBsDtCB/WOy/8r5iXD7UNWhKavFZBS
         A90vzuEiBVwbZo8CpGnzd/L/OivdOJgE0kN7ELYj1IcMAIhh6U2Gls5ZJoUnjpzqRnAI
         OOCpIjkWRPxPHfrWi+CaJ/KjXcC6q4NGeBvLhkBqP8cSgeWvzBuRfmj4Xl3ZDVn6CaC7
         jTew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716994708; x=1717599508;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BNViD2QjGbbdDZhkRCgcqFaqgTYj5buD4YzpkiaWovc=;
        b=cRjfppmElCyUmzP+MS3Po3yDZrSXSVEEMuFy9kyOTHK84ISsoneX7MOOmcRIhA8Nub
         iy28X8VkmAWpx+ohc+av2ssD/QGzNeooRzLjaRCIB+/H3pgZV7Sb2uwC+2NyLpFKPRYl
         rermErJLim1MYPk7o5GozoiPo5b7mt5yCtICnBvyt9OZ0JZPfjM7Orhl50SabqlYVSd1
         0/F1w2McI22d8xlHuIeYbaCGhHKc00Mdp8/YPMMoPCJTFJ8VXpqSKVcHctABxXPZk+MP
         p3+n23gp9OZ0BVGIvcoIi0oFvZtvf02FzX/8imkOKtpMoWciKoR/cWV+i0QwjZyejCr1
         4tkA==
X-Forwarded-Encrypted: i=1; AJvYcCXlDrVTG5NLgf4F1V+TLGV/3TICXE+NtPPiFa9vGkN/HN3LG6Ki83VPDz+DlawC/Y5rdGkwqmVFSV0O6mHTtf1NTCcagZ2hH8hJeQdpPKE=
X-Gm-Message-State: AOJu0YwGlL4W2ZmQgNXIf2dtbzuC2EpmCy36MEmw/VUHzCvBwrHgthpY
	5HovLWTAuenaGgRK+EBTWHOInuX9xExPdC4M33RwpC2k6Ui3G03Y
X-Google-Smtp-Source: AGHT+IG+BJpUK57O/ezY+NRQ1ijJsptJOVug6YwQgG0HEd07BzlFFao2QiSbkMUYJiYCj3dd68/3QA==
X-Received: by 2002:a17:906:2890:b0:a5a:a2b6:ba8f with SMTP id a640c23a62f3a-a6262f87932mr1158371866b.0.1716994707639;
        Wed, 29 May 2024 07:58:27 -0700 (PDT)
Message-ID: <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,  Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 16:58:26 +0200
In-Reply-To: <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
	 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
	 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
	 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
	 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
	 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
	 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
	 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
	 <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

static always_inline bool test_bit(int nr, const volatile void *addr)On
Wed, 2024-05-29 at 12:06 +0200, Jan Beulich wrote:
> On 29.05.2024 11:59, Julien Grall wrote:
> > Hi,
> >=20
> > On 29/05/2024 09:36, Jan Beulich wrote:
> > > On 29.05.2024 09:50, Oleksii K. wrote:
> > > > On Tue, 2024-05-28 at 09:53 +0100, Julien Grall wrote:
> > > > > > > > +/**
> > > > > > > > + * generic_test_bit - Determine whether a bit is set
> > > > > > > > + * @nr: bit number to test
> > > > > > > > + * @addr: Address to start counting from
> > > > > > > > + *
> > > > > > > > + * This operation is non-atomic and can be reordered.
> > > > > > > > + * If two examples of this operation race, one can
> > > > > > > > appear to
> > > > > > > > succeed
> > > > > > > > + * but actually fail.=C2=A0 You must protect multiple
> > > > > > > > accesses with
> > > > > > > > a
> > > > > > > > lock.
> > > > > > > > + */
> > > > > > >=20
> > > > > > > You got carried away updating comments - there's no
> > > > > > > raciness for
> > > > > > > simple test_bit(). As is also expressed by its name not
> > > > > > > having
> > > > > > > those
> > > > > > > double underscores that the others have.
> > > > > > Then it is true for every function in this header. Based on
> > > > > > the
> > > > > > naming
> > > > > > the conclusion can be done if it is atomic/npn-atomic and
> > > > > > can/can't
> > > > > > be
> > > > > > reordered.
> > > > >=20
> > > > > So let me start with that my only request is to keep the
> > > > > existing
> > > > > comments as you move it. It looks like there were none of
> > > > > test_bit()
> > > > > before.
> > > > Just to clarify that I understand correctly.
> > > >=20
> > > > Do we need any comment above functions generic_*()? Based on
> > > > that they
> > > > are implemented in generic way they will be always "non-atomic
> > > > and can
> > > > be reordered.".
> > >=20
> > > I indicated before that I think reproducing the same comments
> > > __test_and_*
> > > already have also for generic_* isn't overly useful. If someone
> > > insisted
> > > on them being there as well, I could live with that, though.
> >=20
> > Would you be ok if the comment is only on top of the __test_and_*=20
> > version? (So no comments on top of the generic_*)
>=20
> That's my preferred variant, actually. The alternative I would also
> be
> okay-ish with is to have the comments also ahead of generic_*.
>=20
> > > > Do you find the following comment useful?
> > > >=20
> > > > " * If two examples of this operation race, one can appear to
> > > > succeed
> > > > =C2=A0 * but actually fail.=C2=A0 You must protect multiple accesse=
s with
> > > > a lock."
> > > >=20
> > > > It seems to me that it can dropped as basically "non-atomic and
> > > > can be
> > > > reordered." means that.
> > >=20
> > > I agree, or else - as indicated before - the wording would need
> > > to further
> > > change. Yet iirc you've added that in response to a comment from
> > > Julien,
> > > so you'll primarily want his input as to the presence of
> > > something along
> > > these lines.
> >=20
> > I didn't realise this was an existing comment. I think the
> > suggestion is=20
> > a little bit odd because you could use the atomic version of the
> > helper.
> >=20
> > Looking at Linux, the second sentence was dropped. But not the
> > first=20
> > one. I would suggest to do the same. IOW keep:
> >=20
> > "
> > If two examples of this operation race, one can appear to succeed
> > but=20
> > actually fail.
> > "
>=20
> As indicated, I'm okay with that being retained, but only in a form
> that
> actually makes sense. I've explained before (to Oleksii) what I
> consider
> wrong in this way of wording things.

Would it be suitable to rephrase it in the following way:
     * This operation is non-atomic and can be reordered.
   - * If two examples of this operation race, one can appear to
   succeed
   - * but actually fail.  You must protect multiple accesses with a
   lock.
   + * If two instances of this operation race, one may succeed in
   updating
   + * the bit in memory, but actually fail. It should be protected
   from
   + * potentially racy behavior.
     */
    static always_inline bool
    __test_and_set_bit(int nr, volatile void *addr)
   @@ -228,8 +191,9 @@ __test_and_set_bit(int nr, volatile void *addr)
     * @addr: Address to count from
     *
     * This operation is non-atomic and can be reordered.
   - * If two examples of this operation race, one can appear to
   succeed
   - * but actually fail.  You must protect multiple accesses with a
   lock.
   + * If two instances of this operation race, one may succeed in
   clearing
   + * the bit in memory, but actually fail. It should be protected
   from
   + * potentially racy behavior.
     */
    static always_inline bool
    __test_and_clear_bit(int nr, volatile void *addr)
   @@ -251,8 +215,9 @@ __test_and_clear_bit(int nr, volatile void
   *addr)
     * @addr: Address to count from
     *
     * This operation is non-atomic and can be reordered.
   - * If two examples of this operation race, one can appear to
   succeed
   - * but actually fail.  You must protect multiple accesses with a
   lock.
   + * If two instances of this operation race, one may succeed in
   changing
   + * the bit in memory, but actually fail. It should be protected
   from
   + * potentially racy behavior.
     */
    static always_inline bool
    __test_and_change_bit(int nr, volatile void *addr)
   @@ -274,8 +239,9 @@ __test_and_change_bit(int nr, volatile void
   *addr)
     * @addr: Address to start counting from
     *
     * This operation is non-atomic and can be reordered.
   - * If two examples of this operation race, one can appear to
   succeed
   - * but actually fail.  You must protect multiple accesses with a
   lock.
   + * If two instances of this operation race, one may succeed in
   testing
   + * the bit in memory, but actually fail. It should be protected
   from
   + * potentially racy behavior.
     */
    static always_inline bool test_bit(int nr, const volatile void
   *addr)

~ Oleksii
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Wed May 29 15:03:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:03:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732152.1138009 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKps-0004ND-8r; Wed, 29 May 2024 15:03:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732152.1138009; Wed, 29 May 2024 15:03:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCKps-0004N6-61; Wed, 29 May 2024 15:03:28 +0000
Received: by outflank-mailman (input) for mailman id 732152;
 Wed, 29 May 2024 15:03:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCKpr-0004N0-9n
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:03:27 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 99acefbf-1dcc-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 17:03:26 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-52b6ccfdf28so565395e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:03:26 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a08abd8sm15023448f8f.32.2024.05.29.08.03.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:03:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99acefbf-1dcc-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716995006; x=1717599806; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=B15XUFlmFFShNmtqs74WHz9M7v0pbt+dJmTE0D0fce0=;
        b=arhX60sFD2HJ/L98aREawqjeD3S53Sm0KNQsD71RGn4Z9/U/hcGfaf49AynALIMFkd
         TVR+LQZ0DrG3n1AkHeh0qaNVOmGlqnnSVFwi9hjf/UmkxZx+D9KaxOgf5V8Cs7pApy1w
         3AaZVJ+rHiHOJefCHyucqy85OuZ4isW8Sf230=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716995006; x=1717599806;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=B15XUFlmFFShNmtqs74WHz9M7v0pbt+dJmTE0D0fce0=;
        b=gIDScNA5f+mj7G0/l1zPOH9VL1ErDGT/0pQF+6me0hUG49oltUdRe48woJulfwVSkl
         gSBDXBL4ORNBY4LEUYfEaecod+3ajJhZutwacGJWqM+xmK0xuekhR4NJ+s9UDdCxWB8+
         Yq5N1rCaNKNBxM5RSfSTdRFPLE/JfUTNRz2R6UuwqSlqrDbEPVTAa7pRJ7beJ7t/HEkn
         Pmy8MibISIBg+iOnoG8rNlZnlNSm5GWuUtD9CusrsMRHGDC5xxksbShAx+vsYXqtb1QJ
         Rx5BrJf51bR9sX6TCyaRr7UulnyrSJj+nYyEia+YWnl/SufX9A65TobIF08EL3cP+NGU
         w8xQ==
X-Forwarded-Encrypted: i=1; AJvYcCUJFkxuRd1uDTViWHL5OanPJkViR8OYBY8GO3Gjx6bkEVHhvsPStbO+3slJvdDq2Y+iXFbI8pL/Js8a+urXfuqLkHTvaAwdgcSt3OF1lKQ=
X-Gm-Message-State: AOJu0YxLK0L30EFKTzgcZwnsaGRy4xgjLdjtNP3M86P358AdvhEfV6hc
	1PSKd0dIfKV4ruHvR70HdJde4AcleG1gETuNaSLvvvF84P7HXsBBrbO6ObMNRUE=
X-Google-Smtp-Source: AGHT+IED122ZGSEUteWAaWI2hzBfQzDzxqJyxuQHPvDWYjR041fJuao15xFdMQdk/W3sKWrUL3T06w==
X-Received: by 2002:a2e:9483:0:b0:2e6:f556:48a5 with SMTP id 38308e7fff4ca-2e95b08f95dmr106981761fa.19.1716995005455;
        Wed, 29 May 2024 08:03:25 -0700 (PDT)
Date: Wed, 29 May 2024 17:03:24 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns
 false if CPU operations are underway
Message-ID: <ZldDvH8GhhQcu5NX@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>

On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
> On 29.05.2024 11:01, Roger Pau Monne wrote:
> > Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
> > a cpu_hotplug_{begin,done}() region the function will still return success,
> > because a CPU taking the rwlock in read mode after having taken it in write
> > mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
> > as it should always return false when called with a CPU hot{,un}plug operation
> > is in progress.
> 
> I'm not sure I can agree with this. The CPU doing said operation ought to be
> aware of what it is itself doing. And all other CPUs will get back false from
> get_cpu_maps().

Well, the CPU is aware in the context of cpu_{up,down}(), but not in
the interrupts that might be handled while that operation is in
progress, see below for a concrete example.

> >  Otherwise the logic in send_IPI_mask() for example is wrong,
> > as it could decide to use the shorthand even when a CPU operation is in
> > progress.
> 
> It's also not becoming clear what's wrong there: As long as a CPU isn't
> offline enough to not be in cpu_online_map anymore, it may well need to still
> be the target of IPIs, and targeting it with a shorthand then is still fine.

The issue is in the online path: there's a window where the CPU is
online (and the lapic active), but cpu_online_map hasn't been updated
yet.  A specific example would be time_calibration() being executed on
the CPU that is running cpu_up().  That could result in a shorthand
IPI being used, but the mask in r.cpu_calibration_map not containing
the CPU that's being brought up online because it's not yet added to
cpu_online_map.  Then the number of CPUs actually running
time_calibration_rendezvous_fn won't match the weight of the cpumask
in r.cpu_calibration_map.

> In any event this would again affect only the CPU leading the CPU operation,
> which should clearly know at which point(s) it is okay to send IPIs. Are we
> actually sending any IPIs from within CPU-online or CPU-offline paths?

Yes, I've seen the time rendezvous happening while in the middle of a
hotplug operation, and the CPU coordinating the rendezvous being the
one doing the CPU hotplug operation, so get_cpu_maps() returning true.

> Together with the earlier paragraph the critical window would be between the
> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
> then the question would be what bad, if any, would happen to that CPU if an
> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
> 
> > Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
> > already hold in write mode by the current CPU, as read_trylock() would
> > otherwise return true.
> > 
> > Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
> 
> I'm puzzled by this as well: Prior to that and the change referenced by its
> Fixes: tag, recursive spin locks were used. For the purposes here that's the
> same as permitting read locking even when the write lock is already held by
> the local CPU.

I see, so the Fixes should be:

x86/smp: use APIC ALLBUT destination shorthand when possible

Instead, which is the commit that started using get_cpu_maps() in
send_IPI_mask().

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:15:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:15:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732160.1138020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL1F-0007sL-BE; Wed, 29 May 2024 15:15:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732160.1138020; Wed, 29 May 2024 15:15:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL1F-0007sE-7t; Wed, 29 May 2024 15:15:13 +0000
Received: by outflank-mailman (input) for mailman id 732160;
 Wed, 29 May 2024 15:15:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCL1D-0007s8-Ga
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:15:11 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3c668b34-1dce-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:15:08 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-421124a0b37so12366655e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:15:08 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421089667fbsm183477005e9.9.2024.05.29.08.15.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:15:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c668b34-1dce-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716995708; x=1717600508; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=qSy0T2PouEWbwJsRaQlmCKlqzvnnh2V+hB726PPoFjE=;
        b=KfHlz280EFp682AZJ/t6X1lLgr1z28TGSoLAf3KZKieYHBwrKzj9AgJNvcfVfG4yK4
         PR512bOlbC763GsmSGty/tQM3ANfQGk9MQ6E2Kzuc3WgmQWkfKEITFJck0P31Ac2OdEK
         6l20rTdMic6JWWs2w9P+0NeugtoANBoIrY2mU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716995708; x=1717600508;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qSy0T2PouEWbwJsRaQlmCKlqzvnnh2V+hB726PPoFjE=;
        b=OMTskQY2/WfXvhjFFI9Sr9/irpOmzk6S86yk8e+aeqOMWcZ/AlAl4frirNKy8JPuTS
         Q7vn5GCP8r8mRyAwRcslPIpTiRc7SyLNBFks61qBuuY4/VZcNXD6mF85OmOf+eWZLjY7
         xk3boxfPqkXDAOzldRAC/lwsySeoJgNBehM96CD2J96+J16w3LLNcUi8l/Jq/evY7/OB
         uzUtThC+qusd4GhxRnZ1Cx1MgpUDzTfk2+sw0bXux33p+QdEYj8UeV7ENljux7p+bYlQ
         DCDfldDP83H3zscpCM7O5xotld6PIN55Cz67RNok2x3g45K1JRff1SL8y9XVzTCOB8Qa
         utKA==
X-Forwarded-Encrypted: i=1; AJvYcCVatwLLsjnG0c+t/vHem9GI7ojKTUZhVY040HoudKEVa1mvIcHvdGQtLwnEta9rJTRvGlqee3xxxGdpCm1qRnmPM4Y+pWJbMFkggv8NBJk=
X-Gm-Message-State: AOJu0YwvM+mBCSe96srhMW+N52zNp9rGUhQAPB39MfxxfF9WNkounXqa
	l7hYJ9jP813qughANSO/kAOIGocjnlljLqrs8dWWSsG6Tr38AAwvrjk958tg6R8=
X-Google-Smtp-Source: AGHT+IGp5hmAE9khhn1k4Gr5pKrXNFzd8/UjnwMq9U1HbYZP8O5xtRk1YqUCLHmZ/LMfaOh+uppQ6Q==
X-Received: by 2002:a7b:cc96:0:b0:41b:9e4f:d2b2 with SMTP id 5b1f17b1804b1-421089d50c1mr152042025e9.2.1716995708119;
        Wed, 29 May 2024 08:15:08 -0700 (PDT)
Date: Wed, 29 May 2024 17:15:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 1/9] x86/irq: remove offline CPUs from old CPU
 mask when adjusting move_cleanup_count
Message-ID: <ZldGe6z70R0CD32Z@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-2-roger.pau@citrix.com>
 <0a2f1f9a-fab1-409c-96f8-399b19077f12@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0a2f1f9a-fab1-409c-96f8-399b19077f12@suse.com>

On Wed, May 29, 2024 at 02:40:51PM +0200, Jan Beulich wrote:
> On 29.05.2024 11:01, Roger Pau Monne wrote:
> > When adjusting move_cleanup_count to account for CPUs that are offline also
> > adjust old_cpu_mask, otherwise further calls to fixup_irqs() could subtract
> > those again creating and create an imbalance in move_cleanup_count.
> 
> I'm in trouble with "creating"; I can't seem to be able to guess what you may
> have meant.

Oh, sorry, that's a typo.

I was meaning to point out that not removing the already subtracted
CPUs from the mask can lead to further calls to fixup_irqs()
subtracting them again and move_cleanup_count possibly underflowing.

Would you prefer to write it as:

"... could subtract those again and possibly underflow move_cleanup_count."

> > Fixes: 472e0b74c5c4 ('x86/IRQ: deal with move cleanup count state in fixup_irqs()')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> With the above clarified (adjustment can be done while committing)
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> > --- a/xen/arch/x86/irq.c
> > +++ b/xen/arch/x86/irq.c
> > @@ -2572,6 +2572,14 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
> >              desc->arch.move_cleanup_count -= cpumask_weight(affinity);
> >              if ( !desc->arch.move_cleanup_count )
> >                  release_old_vec(desc);
> > +            else
> > +                /*
> > +                 * Adjust old_cpu_mask to account for the offline CPUs,
> > +                 * otherwise further calls to fixup_irqs() could subtract those
> > +                 * again and possibly underflow the counter.
> > +                 */
> > +                cpumask_and(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
> > +                            &cpu_online_map);
> >          }
> 
> While functionality-wise okay, imo it would be slightly better to use
> "affinity" here as well, so that even without looking at context beyond
> what's shown here there is a direct connection to the cpumask_weight()
> call. I.e.
> 
>                 cpumask_andnot(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
>                                affinity);
> 
> Thoughts?

It was more straightforward for me to reason that removing the offline
CPUs is OK, but I can see that you might prefer to use 'affinity',
because that's the weight that's subtracted from move_cleanup_count.
Using either should lead to the same result if my understanding is
correct.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:20:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:20:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732166.1138029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL6O-0001oO-Sl; Wed, 29 May 2024 15:20:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732166.1138029; Wed, 29 May 2024 15:20:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL6O-0001oH-Pv; Wed, 29 May 2024 15:20:32 +0000
Received: by outflank-mailman (input) for mailman id 732166;
 Wed, 29 May 2024 15:20:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCL6N-0001nJ-6m
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:20:31 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbdea2a0-1dce-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 17:20:29 +0200 (CEST)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-354c5b7fafaso1837521f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:20:30 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a1c930esm14934098f8f.84.2024.05.29.08.20.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:20:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbdea2a0-1dce-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716996029; x=1717600829; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=8ZngnzdVwUETCzNF4W0WI6nHjkGn2lJLP9Y8fUsjVkM=;
        b=nlV19uGNivcEuV4CU1ZZwxWhjoqToojwwLryfAuK22HRLhB9C88R6KEgoiFZptf7ew
         36o1F5bss+egHhrJ9BYXzGtNHppIoc7lbwOtyMGDkLEWi8dZmdEFxENIKeRDF/kMoOa6
         KwWaFicmkPGNwZ5c8rzuIpSYWQVqVAbeRG/rc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996029; x=1717600829;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8ZngnzdVwUETCzNF4W0WI6nHjkGn2lJLP9Y8fUsjVkM=;
        b=dSJy+4SXJvr4R9+ucPsoCmyYlg5/VhO4HrJv4RSdcWFm4TMYlOPuh4qvHWpohJbNPo
         HengI/wR4C1hRn1XXP8GOnljTbpOLQkIkpLMZnFmfzpA4zw3q6DlvwqtTDG2uodcWsRH
         P/hgS5eAu/X26mTydp2YC3qTyy23pH94kQr4NTgOsbDGJ0SqPxLT6Pndg4I+xr3YMS1K
         ClOEhDKBbeRA8YciX9IdOvhfoTs5M/BS/39CrJyo4Lpszu1qrq53u9gRIfehac1JHTal
         CilstBroYdx4DfrBoFliGnMgw3yD5fKT1hRgJPdy07IMI92+K/G9CzFQDC/ZbDaGqZn8
         mwHg==
X-Forwarded-Encrypted: i=1; AJvYcCVx6WktvrwYG+SwhEKD4fmy3l11lNxbWXBiEXO4OhDd+b076qXIZrYi6/ikffuR1jwvMDUKyjwyfZ0ujxS+urx0skO1/EkNUFuSlE9o68A=
X-Gm-Message-State: AOJu0YzlWWffVPoHtOjiMIa6O96LCMU/O6yZQ2JYwzxbB1YENXiigfkd
	oTiC8/Ld9F0N6aizDaJrEhDrXIKUUNOp4bRmYx+X1eQVZa5pLrOwNHWrV9v9cT0=
X-Google-Smtp-Source: AGHT+IGg4nxxVTkPZuL3F8YUOV6oczXJrhH85Qfkwhqm3yRfXpCC9vdY2G2LU2F5xavIlL1l1KOZEw==
X-Received: by 2002:a5d:58ed:0:b0:355:7ab:2a84 with SMTP id ffacd0b85a97d-3552fde1980mr10303633f8f.49.1716996029334;
        Wed, 29 May 2024 08:20:29 -0700 (PDT)
Date: Wed, 29 May 2024 17:20:28 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org, Michal Orzel <michal.orzel@amd.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>
Subject: Re: [PATCH for-4.19 2/9] xen/cpu: do not get the CPU map in
 stop_machine_run()
Message-ID: <ZldHvAW_2nqAuWcq@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-3-roger.pau@citrix.com>
 <7954b3c2-e101-4db0-b317-a6585a6a8689@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <7954b3c2-e101-4db0-b317-a6585a6a8689@suse.com>

On Wed, May 29, 2024 at 03:04:13PM +0200, Jan Beulich wrote:
> On 29.05.2024 11:01, Roger Pau Monne wrote:
> > The current callers of stop_machine_run() outside of init code already have the
> > CPU maps locked, and hence there's no reason for stop_machine_run() to attempt
> > to lock again.
> 
> While purely from a description perspective this is okay, ...
> 
> > --- a/xen/common/stop_machine.c
> > +++ b/xen/common/stop_machine.c
> > @@ -82,9 +82,15 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
> >      BUG_ON(!local_irq_is_enabled());
> >      BUG_ON(!is_idle_vcpu(current));
> >  
> > -    /* cpu_online_map must not change. */
> > -    if ( !get_cpu_maps() )
> > +    /*
> > +     * cpu_online_map must not change.  The only two callers of
> > +     * stop_machine_run() outside of init code already have the CPU map locked.
> > +     */
> 
> ... the "two" here is not unlikely to quickly go stale; who knows what PPC
> and RISC-V will have as their code becomes more complete?
> 
> I'm also unconvinced that requiring ...
> 
> > +    if ( system_state >= SYS_STATE_active && !cpu_map_locked() )
> 
> ... this for all future (post-init) uses of stop_machine_run() is a good
> idea. It is quite a bit more natural, to me at least, for the function to
> effect this itself, as is the case prior to your change.

This is mostly a pre-req for the next change that switches
get_cpu_maps() to return false if the current CPU is holding the CPU
maps lock in write mode.

IF we don't want to go this route we need a way to signal
send_IPI_mask() when a CPU hot{,un}plug operation is taking place,
because get_cpu_maps() enough is not suitable.

Overall I don't like the corner case where get_cpu_maps() returns true
if a CPU hot{,un}plug operation is taking place in the current CPU
context.  The guarantee of get_cpu_maps() is that no CPU hot{,un}plug
operations can be in progress if it returns true.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:22:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:22:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732170.1138040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL7t-0002to-7R; Wed, 29 May 2024 15:22:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732170.1138040; Wed, 29 May 2024 15:22:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCL7t-0002th-3e; Wed, 29 May 2024 15:22:05 +0000
Received: by outflank-mailman (input) for mailman id 732170;
 Wed, 29 May 2024 15:22:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCL7s-0002sI-KV
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:22:04 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 339bee64-1dcf-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 17:22:03 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a621cb07d8fso250072466b.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:22:03 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cc8dcbdsm741528166b.175.2024.05.29.08.22.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 08:22:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 339bee64-1dcf-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716996123; x=1717600923; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=tdxYn410jYiX71+OkYGeRpynQ7LAu99vUNbAzovPB3g=;
        b=gJLy4z4GnpYKXJGkw7OUKql3G6SwcmhBS5qZwWxuX4cETe+YG5PkQ+WFY4K/n5TxhX
         nODLES2hUusutoSfC20LV09taFavzxFGMeBI75sc2pkwsORpyUKBy5ZB5cBWSuzbKXGV
         jzZPNzrgUMEqNIAkQeNBPlHcZNt1+rcBCKMx2xvHHXu2a/Eh6GD0X2ZWJUxdyFOadXuH
         V7uAegaUdUiC08Tk5/H4jDn+ufa4cyxq1iIg1LVGrlS0bzrIwaaDN/g+4sxJZVSv89Ix
         YM8Rh6JT2vkA1DINl/qmY+uAV1AAAO9Yd7uJwCoy32jU9Vo6LUTCKdj9ps6/ZQoBfbKq
         r0Yw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996123; x=1717600923;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tdxYn410jYiX71+OkYGeRpynQ7LAu99vUNbAzovPB3g=;
        b=TklcYug8BJXHK2BL9aZ9gAKOuGvnRhNiwMJFpmIiEYwy8JqNM2QWfx3MOfZ4ciLC+F
         sa7tQ0MAaSAVdLxTbsYnqMWIxCKl0xh7oKMMMk/TN1ExPQCuoQwuF2oRHYXIb/x7eBfD
         Pbzj1mtfymPrvW0oQ2TEzXvyc5f0fLv66VdW1DdxsoLYxJcOtm5rq3Qh9iDsPQ3ewqam
         hpHjN7pu89f2gD/6g4LSaWJSj6kvwoJzn8k8DTh8426O5JBusjb4qKpArArdyAEHkeb5
         4ohbRAKvSPbhecpK4uye4k9xblNz2l7TfoD2TulNXhmvFS5CBudfjFHgsvqwdMg1FiJk
         7v3Q==
X-Forwarded-Encrypted: i=1; AJvYcCVdqdoI2fjFo+2aEWGXiMVlsirG2Lp8ED75qQCNLly1BnsAcENk1ATUyKdCMRUctUUp0QNybl01fh+oECOaHlqgKHylb5tvpSEaKlrAhfM=
X-Gm-Message-State: AOJu0Yxg/BGbYMFBYxyKtwdimbXXB0Nm6x0rUrUBGbIkcblTFf5bAzYX
	g8jSw0+QLqo7V68yHCIOe/FuzwVDM5H19fMBHvedy8B3e9x7Nw4DMgWgNmZEcw==
X-Google-Smtp-Source: AGHT+IFdsCupvYhSyVJvwfh2MXpfu9RkSBdAsD0GB8snKmg/jUgZ0JBK3sD92cdE7lZNTFGCA3u8lg==
X-Received: by 2002:a17:907:76d6:b0:a63:3c57:5806 with SMTP id a640c23a62f3a-a633c575ccbmr360740266b.32.1716996122927;
        Wed, 29 May 2024 08:22:02 -0700 (PDT)
Message-ID: <23790bd6-e06a-4adc-90a9-517f947be00a@suse.com>
Date: Wed, 29 May 2024 17:22:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
To: "Oleksii K." <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Julien Grall <julien@xen.org>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
 <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
 <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.05.2024 16:58, Oleksii K. wrote:
> static always_inline bool test_bit(int nr, const volatile void *addr)On
> Wed, 2024-05-29 at 12:06 +0200, Jan Beulich wrote:
>> On 29.05.2024 11:59, Julien Grall wrote:
>>> I didn't realise this was an existing comment. I think the
>>> suggestion is 
>>> a little bit odd because you could use the atomic version of the
>>> helper.
>>>
>>> Looking at Linux, the second sentence was dropped. But not the
>>> first 
>>> one. I would suggest to do the same. IOW keep:
>>>
>>> "
>>> If two examples of this operation race, one can appear to succeed
>>> but 
>>> actually fail.
>>> "
>>
>> As indicated, I'm okay with that being retained, but only in a form
>> that
>> actually makes sense. I've explained before (to Oleksii) what I
>> consider
>> wrong in this way of wording things.
> 
> Would it be suitable to rephrase it in the following way:
>      * This operation is non-atomic and can be reordered.
>    - * If two examples of this operation race, one can appear to
>    succeed
>    - * but actually fail.  You must protect multiple accesses with a
>    lock.
>    + * If two instances of this operation race, one may succeed in
>    updating
>    + * the bit in memory, but actually fail. It should be protected
>    from
>    + * potentially racy behavior.
>      */
>     static always_inline bool
>     __test_and_set_bit(int nr, volatile void *addr)

You've lost the "appear to" ahead of "succeed". Yet even with the adjustment
I still don't see what the "appear to succeed" really is supposed to mean
here. The issue (imo) isn't with success or failure, but with the write of
one CPU potentially being immediately overwritten by another CPU, not
observing the bit change that the first CPU did. IOW both will succeed (or
appear to succeed), but one update may end up being lost. Maybe "..., both
may update memory with their view of the new value, not taking into account
the update the respectively other one did"? Or "..., both will appear to
succeed, but one of the updates may be lost"?

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:27:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:27:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732177.1138049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLCp-0003hM-P1; Wed, 29 May 2024 15:27:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732177.1138049; Wed, 29 May 2024 15:27:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLCp-0003hF-MN; Wed, 29 May 2024 15:27:11 +0000
Received: by outflank-mailman (input) for mailman id 732177;
 Wed, 29 May 2024 15:27:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCLCo-0003h9-Sj
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:27:10 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e98ab62e-1dcf-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:27:08 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-572c65cea55so160697a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:27:08 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57861b4ddefsm7274275a12.60.2024.05.29.08.27.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 08:27:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e98ab62e-1dcf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716996428; x=1717601228; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Q6MsRT6qQ4rmyScI+Jz3Lny4rVSNKZa/8YroPYI+NZM=;
        b=fEUDq2uVxpNZo//1hgPyJgKzj/ARQSUA+J6pmAF0jvbvaQ+1iW2Wwkx9ZbzoiOAtOu
         0DGP5FFYKUX2LIquWgA4+Cr06p37M28Koq7OmGB5DLH4kKALUBcoorGLYe4WzhwFXRgl
         VsX9KxjLESCGOC5x3JTgLhqj/Y0O7QZmKfz5/um+ggAo51eFpUPsZ9jS5y9xqjQOAk7M
         eHjC2DhKg0HGqyWvJw4ciZG4KCo9YmYEH+MHlfFRPaaPfBN5uPIKw6uobed93iBjTQSv
         /XDCCv4V2i6jz1koGifiRqY6Sn3QfuH2RMvuDbnU9hih+2veWqnhWfmkfzGsEZ8nxbwm
         6Ohg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996428; x=1717601228;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Q6MsRT6qQ4rmyScI+Jz3Lny4rVSNKZa/8YroPYI+NZM=;
        b=WMCc0jKUnR73B5O2SgZt87QjORC0iCQt/QfzI/rD+6PQ3AzWRG7ByF10rep+hMfKEO
         63XZ1Nmx3lFwJPYa/TYrE2Xo79Omyn7RcOjQeLeBc3Wr6vbRf9f0mA4qlWcFnfEdCjVe
         XRL5bK8s6NbiwYhzRkpMrL80zAcxipjg7Plydu+quit8123H6/dm8rVwOyzcTMb7Sp4m
         4s9mT5tOrTd6+ACIXk+4ECwx4H90amCqWqXxpgWiPxYm8FhChELIxhMpBYr1uzBns2CA
         hiY+nSOezJpT82wr/krx2cz/Xcs3DgWAsYMO3QFuDKiWLuYSs+2XGFdwAPhdvs6YxLaS
         ALEQ==
X-Forwarded-Encrypted: i=1; AJvYcCXFeVMv1VTqs87z4Vt4i+xaoLGGRL4V4AFq7Q+8LKl/m/tJ6AyCuA/pK6rRC//r5KGzv8+auF9arTxGRAme2saWLaKnwV76Wzj1WHCke/0=
X-Gm-Message-State: AOJu0YyX601SfriWk6xy0qAXPZcQsipwDuC7UOD406a8iW01sJEYuGHv
	ts65yYBSbiMTU4xYNXRhkn6ichYF9kEKz/dOvu6Mex3KHbcdDe1WN6/x/5SHzA==
X-Google-Smtp-Source: AGHT+IF0FWIqeSqjU2L8e6lfQioqXAc59wKYjgFXUDQW/tMeP8jl6Sstt/qyaXWyU6/A+XW1Y6blIg==
X-Received: by 2002:a50:955c:0:b0:574:ebf4:f786 with SMTP id 4fb4d7f45d1cf-57a03fb3bfcmr2271955a12.16.1716996428014;
        Wed, 29 May 2024 08:27:08 -0700 (PDT)
Message-ID: <b9a667ea-45a6-45d1-8dda-0ec3aa00b9cf@suse.com>
Date: Wed, 29 May 2024 17:27:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 1/9] x86/irq: remove offline CPUs from old CPU
 mask when adjusting move_cleanup_count
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-2-roger.pau@citrix.com>
 <0a2f1f9a-fab1-409c-96f8-399b19077f12@suse.com> <ZldGe6z70R0CD32Z@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZldGe6z70R0CD32Z@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 17:15, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 02:40:51PM +0200, Jan Beulich wrote:
>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>> When adjusting move_cleanup_count to account for CPUs that are offline also
>>> adjust old_cpu_mask, otherwise further calls to fixup_irqs() could subtract
>>> those again creating and create an imbalance in move_cleanup_count.
>>
>> I'm in trouble with "creating"; I can't seem to be able to guess what you may
>> have meant.
> 
> Oh, sorry, that's a typo.
> 
> I was meaning to point out that not removing the already subtracted
> CPUs from the mask can lead to further calls to fixup_irqs()
> subtracting them again and move_cleanup_count possibly underflowing.
> 
> Would you prefer to write it as:
> 
> "... could subtract those again and possibly underflow move_cleanup_count."

Fine with me. Looks like simply deleting "creating" and keeping the rest
as it was would be okay too? Whatever you prefer in the end.

>>> --- a/xen/arch/x86/irq.c
>>> +++ b/xen/arch/x86/irq.c
>>> @@ -2572,6 +2572,14 @@ void fixup_irqs(const cpumask_t *mask, bool verbose)
>>>              desc->arch.move_cleanup_count -= cpumask_weight(affinity);
>>>              if ( !desc->arch.move_cleanup_count )
>>>                  release_old_vec(desc);
>>> +            else
>>> +                /*
>>> +                 * Adjust old_cpu_mask to account for the offline CPUs,
>>> +                 * otherwise further calls to fixup_irqs() could subtract those
>>> +                 * again and possibly underflow the counter.
>>> +                 */
>>> +                cpumask_and(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
>>> +                            &cpu_online_map);
>>>          }
>>
>> While functionality-wise okay, imo it would be slightly better to use
>> "affinity" here as well, so that even without looking at context beyond
>> what's shown here there is a direct connection to the cpumask_weight()
>> call. I.e.
>>
>>                 cpumask_andnot(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask,
>>                                affinity);
>>
>> Thoughts?
> 
> It was more straightforward for me to reason that removing the offline
> CPUs is OK, but I can see that you might prefer to use 'affinity',
> because that's the weight that's subtracted from move_cleanup_count.
> Using either should lead to the same result if my understanding is
> correct.

That was the conclusion I came to, or else I wouldn't have made the
suggestion. Unless you have a strong preference for the as-is form, I'd
indeed prefer the suggested alternative.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:28:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:28:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732183.1138059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLDl-0004GC-5g; Wed, 29 May 2024 15:28:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732183.1138059; Wed, 29 May 2024 15:28:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLDl-0004G5-3A; Wed, 29 May 2024 15:28:09 +0000
Received: by outflank-mailman (input) for mailman id 732183;
 Wed, 29 May 2024 15:28:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCLDj-0003h9-Be
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:28:07 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0b5b4007-1dd0-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:28:05 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-42108856c33so513935e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:28:05 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35579d7dd8esm15013488f8f.10.2024.05.29.08.28.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:28:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b5b4007-1dd0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716996485; x=1717601285; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vgf6iO5UWB16AaiuEruVh71mcKykYduHrqyGEP27qgQ=;
        b=ek65V3zFx2r5cRVE2J4nOwj1we8yvi9PsYOg+0U4lhf24r50DSNWqeywzPoL5ZX4x4
         1AW71XMZaLOhTxUOMb0AZhu/gLiCe+k48StontqdwF6Kv3d4II9ZK77+QAi4XdzB++4t
         i9wwykdTWVYM4qrZ/6sUCe9OB1w6XFOs2lmYM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996485; x=1717601285;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vgf6iO5UWB16AaiuEruVh71mcKykYduHrqyGEP27qgQ=;
        b=gI3IJN+xZoa6eDOUNVqaC5zXzZSWL9IiSay/4hOQlPzptnEMVwnk5yCjHsNdSpkTbO
         MZnz2Zic5wnqRt1H2RRkN4ZZnuDR/aaBaQPQvduI+YYb+8kIkBlPULTVSCkVcCIKeFuz
         QzNO2qTcPWMxIlbRzTkJFm+XFRS4q36zYeoXcNP30B51lwj4STySmBuY2tPGuWoLFXU+
         FSL/D6/VeYdITtC4zsl50J04kh6b+0vsHRmfQksh9YWOINxKbOdXZcQw3muvVGRYoZPb
         Ilb9nKh5HpFPe8MikUhFecykRtWTKaZseyWGmme+I3M2nhsO3Ctwu8Ef+z7yWDYIVcZs
         EgwA==
X-Forwarded-Encrypted: i=1; AJvYcCURyZcuMnAcSOJjoK0oBmUv1Qs4h5BgyFosdqJ4JWcSAw9RacjEszXhIJv/DJcChZxyCQ3ceRgGfiTFEgMTNnRPzrcG+Mv6aEUMFhZQfxk=
X-Gm-Message-State: AOJu0Yxh/fNmPCtWv0rMmXhB7g5Fbe4Y1Ax05TwxSvts5O2IGsI+12oV
	yvfQgaz/TnyUDVxpAYigg4wo8i2VTlOjyxCGEs65i7+Lo6Hz5sd2SYsZLg4eE1A=
X-Google-Smtp-Source: AGHT+IGRV4rt6jmRJeV7HWcXdePj5WDRmBcz9L5WQ9WZM+MdBl9XEb/Dxss5ZvWRyOfE6pH9kTlGpQ==
X-Received: by 2002:a05:600c:3548:b0:41f:c5c5:c9df with SMTP id 5b1f17b1804b1-42122b00e42mr24253635e9.14.1716996484837;
        Wed, 29 May 2024 08:28:04 -0700 (PDT)
Date: Wed, 29 May 2024 17:28:03 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 4/9] x86/irq: describe how the interrupt CPU
 movement works
Message-ID: <ZldJgwu2nDGv2HJP@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-5-roger.pau@citrix.com>
 <24201381-2a43-427e-a3e5-1de316d212b8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <24201381-2a43-427e-a3e5-1de316d212b8@suse.com>

On Wed, May 29, 2024 at 03:57:19PM +0200, Jan Beulich wrote:
> On 29.05.2024 11:01, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/include/asm/irq.h
> > +++ b/xen/arch/x86/include/asm/irq.h
> > @@ -28,6 +28,32 @@ typedef struct {
> >  
> >  struct irq_desc;
> >  
> > +/*
> > + * Xen logic for moving interrupts around CPUs allows manipulating interrupts
> > + * that target remote CPUs.  The logic to move an interrupt from CPU(s) is as
> > + * follows:
> > + *
> > + * 1. cpu_mask and vector is copied to old_cpu_mask and old_vector.
> > + * 2. New cpu_mask and vector are set, vector is setup at the new destination.
> > + * 3. move_in_progress is set.
> > + * 4. Interrupt source is updated to target new CPU and vector.
> > + * 5. Interrupts arriving at old_cpu_mask are processed normally.
> > + * 6. When an interrupt is delivered at the new destination (cpu_mask) as part
> > + *    of acking the interrupt move_in_progress is cleared and move_cleanup_count
> 
> Nit: A comma after "interrupt" may help reading.
> 
> > + *    is set to the weight of online CPUs in old_cpu_mask.
> > + *    IRQ_MOVE_CLEANUP_VECTOR is sent to all CPUs in old_cpu_mask.
> 
> These last two steps aren't precise enough, compared to what the code does.
> old_cpu_mask is first reduced to online CPUs therein. If the result is non-
> empty, what you describe is done. If, however, the result is empty, the
> vector is released right away (this code may be there just in case, but I
> think it shouldn't be omitted here).

I've left that out because I got the impression it made the text more
complex to follow (with the extra branch) for no real benefit, but I'm
happy to attempt to add it.

> 
> > + * 7. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the
> > + *    vector entry and decrease the count in move_cleanup_count.  The CPU that
> > + *    sets move_cleanup_count to 0 releases the vector.
> > + *
> > + * Note that when interrupt movement (either move_in_progress or
> > + * move_cleanup_count set) is in progress it's not possible to move the
> > + * interrupt to yet a different CPU.
> > + *
> > + * By keeping the vector in the old CPU(s) configured until the interrupt is
> > + * acked on the new destination Xen allows draining any pending interrupts at
> > + * the old destinations.
> > + */
> >  struct arch_irq_desc {
> >          s16 vector;                  /* vector itself is only 8 bits, */
> >          s16 old_vector;              /* but we use -1 for unassigned  */
> 
> I take it that it is not a goal to (also) describe under what conditions
> an IRQ move may actually be initiated (IRQ_MOVE_PENDING)? I ask not the
> least because the 2nd from last paragraph lightly touches that area.

Right, I was mostly focused on moves (forcefully) initiated from
fixup_irqs(), which is different from the opportunistic affinity
changes signaled by IRQ_MOVE_PENDING.

Not sure whether I want to mention this ahead of the list in a
paragraph, or just add it as a step.  Do you have any preference?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:30:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:30:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732188.1138070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLG6-0005pJ-Hz; Wed, 29 May 2024 15:30:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732188.1138070; Wed, 29 May 2024 15:30:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLG6-0005pC-FK; Wed, 29 May 2024 15:30:34 +0000
Received: by outflank-mailman (input) for mailman id 732188;
 Wed, 29 May 2024 15:30:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DcHb=NA=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1sCLG5-0005p6-D6
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:30:33 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61ecdb89-1dd0-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 17:30:32 +0200 (CEST)
Received: from PH8PR20CA0004.namprd20.prod.outlook.com (2603:10b6:510:23c::12)
 by BL3PR12MB6378.namprd12.prod.outlook.com (2603:10b6:208:3b1::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.28; Wed, 29 May
 2024 15:30:26 +0000
Received: from CY4PEPF0000E9D8.namprd05.prod.outlook.com
 (2603:10b6:510:23c:cafe::fc) by PH8PR20CA0004.outlook.office365.com
 (2603:10b6:510:23c::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30 via Frontend
 Transport; Wed, 29 May 2024 15:30:25 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000E9D8.mail.protection.outlook.com (10.167.241.71) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7633.15 via Frontend Transport; Wed, 29 May 2024 15:30:25 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 29 May
 2024 10:30:24 -0500
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 29 May 2024 10:30:23 -0500
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61ecdb89-1dd0-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IuPoDDsLIDCEwwp5ieUxidhXdZIDkath53V/79jXX/5P5l9jiY8pSxyiW7LDWSGwjtd8L9+J+vvgn3HfB6Oe77ESZa1kvCtxMHCEJr+xp8tFDiPrscpWkDHDNK7NiHUwXbXhdqyMw8kiX9dPRt4iVYIrkyoUzyzhx2DyeZ8lGb5rBT+rIOBi6w8X1+HLo6OG3++MdQr/B49keJXnMvZZ9/diX9OnsuQPy5DBlH13OQ77F00djQFicwZqhSh3v941IEdq+vjeCnbHWGgAIPeAS/vZnK470TPweMCsyA49jDJzYgSvyzkv/auGaHCRUabcHLsnn81kbkeYNwA4JGfu3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wTUBGfr3d7J2x5WWwui99WN/K/I5ilvznelX0mozrms=;
 b=fe4YPKXrmi1exRq1+V+ni1nSsDxN+inD2STUv50T398aa9E4VajuD2RqWdMH/y8nUazmFmf4Hl7zGuJj29jawvv9QOSeMEFquv9D0FZX5WdNB9QuLD1WG2tBNBbezEZsvbSZS8ClF806TDDFUB8d35Kj5y75Ymy8IYAbATli3k7DtHuu/CkynwAJaZ4Cn4vO4vYwtLIdYcJcR+sb+pcpqW9Pw6IS1L2ZSCNJjsyDhckSP/SHP7FeY4WTPV8jLxo1CgvO2j0dDeRS/oCW14Wami0tf9WDsfVStzg4rggfhJOFDqP3jW6fKj/ISp2H02bQ5eByULIkFJpMcuXD65+KXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=citrix.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wTUBGfr3d7J2x5WWwui99WN/K/I5ilvznelX0mozrms=;
 b=sJ+f78gcVI13e70nagRQSTRYZCFJ3rqvMJ2NIm+AwGXBGFcJHjKla+cjIHNIRyj60q4D7fTxaO43ObwYpDj75EgpCcjFIwJbhACEQCGjGPMDotwp2C6pz99KQxxmkb5g9d+dhic0+D8LShNggeyNeyXXgY5Mqsk8HYGNjiFwVAQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <10676879-e0e4-4332-b8cb-732d56cda818@amd.com>
Date: Wed, 29 May 2024 17:30:22 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] CI: Use a debug build of Xen for the Xilinx HW tests
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>
CC: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Stefano
 Stabellini <sstabellini@kernel.org>,
	=?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
	<marmarek@invisiblethingslab.com>, Oleksii Kurochko
	<oleksii.kurochko@gmail.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
 <20240529141945.41669-3-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240529141945.41669-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Received-SPF: None (SATLEXMB04.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D8:EE_|BL3PR12MB6378:EE_
X-MS-Office365-Filtering-Correlation-Id: a26c6f6d-97d2-49ad-e048-08dc7ff44347
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?eDhNYWxBNXlGSlY0VnY1UUw1akdCSno3MG5FNlZUUXhoRmo5ekN3RGJ4UFdn?=
 =?utf-8?B?MlcxQ0NrNER1QmxiS0g3VjlZNDk5dElPU2crQWhEK2RxMVAxYlpHWlAwRnFu?=
 =?utf-8?B?R3U4d3ZFclJkVXFna3Myb3lweHNSV0hvU3RNVlVnanRHbmswTkFQVFdiK05s?=
 =?utf-8?B?NGFLUmtnd2VJQUtCQU4rWExNZlpTWUJwRzA2RXVKN2NHZ3ZLbWtUTnp3SGsr?=
 =?utf-8?B?NDI1NFA3eDNCekRXald5aVBzbyt0a3VCTEtrUTFkRCtnVTJOZG9aVExlbllH?=
 =?utf-8?B?WHpqa3FYekZoQlBqeTBKTGMvbkM0SU5aaVdGQzAyNm9ZMzhwRzJQdWhGT3Qr?=
 =?utf-8?B?TjFhSFh1clg0aVkwZWUzanlDcGFXdEgwUGZhUUdMTVVKM3UveW1xUlBMY1dz?=
 =?utf-8?B?YjBBcGpsdHVDOUNYUml4V1JkNmJtMzlobEd1TmFEdGhSdndQUlVWSUVCV3JH?=
 =?utf-8?B?N1dkdVFjdW5zQUNqZ2R0SmloNnRDRjNuZm16VFdtOFRxeHlTMEptNmdDc0gz?=
 =?utf-8?B?d1B6eGMrSUluc2Z5RkRCVCthT1Rydyt1ZzZ4T2RTc0FRM3k2MzdQdDFhdmJW?=
 =?utf-8?B?c3hyNTgvNnRmTE1pSnd2YU9ZYStGTlMzL0krRVdTc09aQXZBaytnbDY5QkUx?=
 =?utf-8?B?YVRFMS9iSHBNaGQrMEFMQW00Y1pHR2JhOStUNGd4aExRRzRQMk00Zmd2WnRl?=
 =?utf-8?B?NHZuRnZ2YS9Vanl0cXNxcGJXOEZYVWtFeldkNno3SjBZYjYyNWVzMlBDM2RI?=
 =?utf-8?B?aVFBV2VqSmVZb0p5UVV1bjd4NndwMGx0VDRRVjREbURWVnZiWlBGbXVuSjFh?=
 =?utf-8?B?OUJ5ejhqR252RWJsK2pUQVRkSlpZd1B0YzlOSFl2bUlCNGp3NTB3SUNhWmVC?=
 =?utf-8?B?RCsycEE3dkNrVkpnODlNWTZZYStzb2orcDVQVk82bU5veHlHUkgyV21abENM?=
 =?utf-8?B?WllxZXFJSjhTNTgxK3ZVTEZUMzV5VFJGWmFYRVN4QkptMVZTeEpvUEViVzFX?=
 =?utf-8?B?OWZ6cDdiRjJhT2JJVzVNYmg1V1Y5Z0RkcEJnWERBMzB3UlM3b0FuUWVoY1Jw?=
 =?utf-8?B?TkVVekt4ekpLWlB3WWZUTFE1YS9YOFV2Zm5YcDhhK3FGd1ZWRXhXaWhWaFhK?=
 =?utf-8?B?Rks2VTFZeGdUL2p0L0FBWlhyY05VNE1Wd3VIb2FHUEE1azlJWS8rT1BURTE1?=
 =?utf-8?B?Q0JQZitadkx5cWdSd3Q3aVFWVjRSTEpZUGlTbGcvbFhUWDRrOWJHUCsvbDVB?=
 =?utf-8?B?eDBBQjFGWWFMNFNuYjAxdEgxMmhrVWVhelJyTDRleEJ5L0p5QlowVUFFYnAw?=
 =?utf-8?B?Ulp3UmJyQWFaaU96a0xIVXJEY0FubHA2bEQ2bU81Umk5aVg4U0wvaXZhMEQz?=
 =?utf-8?B?aEF6ZmJlL0NPQ3ZqeTJFNXlaZGhqTE15RVVVcGQ3SzdtUnpHMUhuRFljZ00v?=
 =?utf-8?B?dzJPR2Q2UlovVVF5RlV2UmRCWllWdlMwSm1JSDkyVEVjTDVHUmZKMG5MVk0z?=
 =?utf-8?B?MUpqUEVBSGpGNUZsMWpHcDRGVWV3aUxpWXNGNHMrckJvVGZOT244ZmZuQmJo?=
 =?utf-8?B?ZWtSSVBBQU9lQTQ2L2lSQjF2QUdmUXRCUW13SGZMRkRuYVYxL3hublBYZ1lw?=
 =?utf-8?B?S3A5eVZCVFVmc1hhTWx3K0dDZ0VESWNGWkkzOFVLRktUbDRNRklNMUYzNzFs?=
 =?utf-8?B?ZW5XNWZxWElDSDEzN2x0K1Evcnk3REI5SVltZUIzd0N5OGE0VjVRcFM5TkNp?=
 =?utf-8?B?bWxtTlEzRnJPeDk4eVR0TW1oVGNqRTRjUFdZZGxPNFRkdWVXNit3R0s5NWNB?=
 =?utf-8?B?cGlvTVVoZENvd1hJWlpac1prbWIyK096VjdRY0U0cmNZL1NXampVclFrUytI?=
 =?utf-8?Q?9M+Weic+7Ceux?=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 May 2024 15:30:25.6205
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a26c6f6d-97d2-49ad-e048-08dc7ff44347
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6378

Hi Andrew,

On 29/05/2024 16:19, Andrew Cooper wrote:
> 
> 
> ... like the other hardware tests.  This gets more value out of the testing.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  automation/gitlab-ci/test.yaml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index efd3ad46f08e..e96ccdfad54c 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
>      - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
>    needs:
>      - *arm64-test-needs
> -    - alpine-3.18-gcc-arm64
> +    - alpine-3.18-gcc-debug-arm64
This change should be reflected in the name of the test changed (here and below), so that it contains -debug suffix just like every other debug job.
With that done:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:31:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:31:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732192.1138080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLGd-0006OD-QY; Wed, 29 May 2024 15:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732192.1138080; Wed, 29 May 2024 15:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLGd-0006O6-Mp; Wed, 29 May 2024 15:31:07 +0000
Received: by outflank-mailman (input) for mailman id 732192;
 Wed, 29 May 2024 15:31:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCLGc-0006Nv-Go
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:31:06 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 761f1814-1dd0-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:31:04 +0200 (CEST)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-52a54d664e3so1217773e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:31:04 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a6590d02e53sm45253766b.119.2024.05.29.08.31.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 08:31:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 761f1814-1dd0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716996664; x=1717601464; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=M0ApkL0EDfx0AVoficEZQyK5XZidmmaFxKpqjcjDPTg=;
        b=Xnd6snh+13PhyTrxchSgPKU0rVnniAeShrq4OihajYF0Pv3IsnsMdscZTEqVdf9jVH
         5lwhq40C+C2Mvkz+gw2f9pt5aQM1jqSXVl8FDm/1stUydRsJKQs/TnaiCtAR4RRntMrp
         Gb89o/OnZt4FyluZ6lPqezhQ8A0HlLQKcuEuiFYF33oXALk4z2WCxijCf99Po/dAeowh
         Ao0zaw4Dy7tobw3TXHv3YmEklf2f+1bYcjfgWI0uPjMs3UcHdw4ZjxWiQg2EthMp0bF5
         HlM2FMSphRywfWnLMkTdgnbaOHk/l2ydN/UyCuTuxXLu+L4Ny+bIf5hcJrgkKUytjo7b
         juDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996664; x=1717601464;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=M0ApkL0EDfx0AVoficEZQyK5XZidmmaFxKpqjcjDPTg=;
        b=EmaSrKkN8ishBdidD15Nn8QuY3fFjxqCMPCE8dJk+eLGaZxQqc3FfictUrdbQI7767
         /eWxR9iX88iRUHCBhy24bkee9dU34J4XOl4pXUSmC4dgIl4ezK9bod1vkrMpaElfMzuP
         EQqbkuOMdpt06IDGI33yv4DlrYR1U7roz4PwV/v2xbZ11k4xWtWnSSY9TkOx87rOG0Pm
         zldCLyooWHSfkM0FXXXv+rJA451dpr0CyZy2fqS28LTqwdsFqSm0KzFqRzS3gecM5YYt
         Enl/+A2E3rmWHNOmLgcO/IWWqkRKubj/du2vbMyAtrb2C7ej7LLJAFuNrMQsgiZVPE4A
         v3WQ==
X-Forwarded-Encrypted: i=1; AJvYcCWAb98N0Q4JFaWF3/dQT6BT6tJ4vCxkcssXumEMInxr1+LCa+C4rm55EpBPP8abNywwEyEcp6jxpQWb6Fpncveclfp7CbatkltXfT+DTI8=
X-Gm-Message-State: AOJu0YyT1ktC9JatwqOWC2cwV53mxEf4yw0LVO0vZOdQVHlq62cQ9aig
	mJCaSvHAVaUifn4a0cCQE/+DpLEF6H0nGcaWDT78nUrFjwWnK7P9UHljtJqH9g==
X-Google-Smtp-Source: AGHT+IF0uzGLQlLjUHzcntWU8AcBe0p2JBn7yQXrVfpE49iOBGndG5TFByqo/L0+xHTAWidMJ3Mxhw==
X-Received: by 2002:a05:6512:214f:b0:52b:660a:666 with SMTP id 2adb3069b0e04-52b660a0821mr858544e87.38.1716996663888;
        Wed, 29 May 2024 08:31:03 -0700 (PDT)
Message-ID: <b427a74b-60e4-4bdf-96dd-164ea2506ba1@suse.com>
Date: Wed, 29 May 2024 17:31:02 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 2/9] xen/cpu: do not get the CPU map in
 stop_machine_run()
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-3-roger.pau@citrix.com>
 <7954b3c2-e101-4db0-b317-a6585a6a8689@suse.com> <ZldHvAW_2nqAuWcq@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZldHvAW_2nqAuWcq@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 17:20, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 03:04:13PM +0200, Jan Beulich wrote:
>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>> The current callers of stop_machine_run() outside of init code already have the
>>> CPU maps locked, and hence there's no reason for stop_machine_run() to attempt
>>> to lock again.
>>
>> While purely from a description perspective this is okay, ...
>>
>>> --- a/xen/common/stop_machine.c
>>> +++ b/xen/common/stop_machine.c
>>> @@ -82,9 +82,15 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
>>>      BUG_ON(!local_irq_is_enabled());
>>>      BUG_ON(!is_idle_vcpu(current));
>>>  
>>> -    /* cpu_online_map must not change. */
>>> -    if ( !get_cpu_maps() )
>>> +    /*
>>> +     * cpu_online_map must not change.  The only two callers of
>>> +     * stop_machine_run() outside of init code already have the CPU map locked.
>>> +     */
>>
>> ... the "two" here is not unlikely to quickly go stale; who knows what PPC
>> and RISC-V will have as their code becomes more complete?
>>
>> I'm also unconvinced that requiring ...
>>
>>> +    if ( system_state >= SYS_STATE_active && !cpu_map_locked() )
>>
>> ... this for all future (post-init) uses of stop_machine_run() is a good
>> idea. It is quite a bit more natural, to me at least, for the function to
>> effect this itself, as is the case prior to your change.
> 
> This is mostly a pre-req for the next change that switches
> get_cpu_maps() to return false if the current CPU is holding the CPU
> maps lock in write mode.
> 
> IF we don't want to go this route we need a way to signal
> send_IPI_mask() when a CPU hot{,un}plug operation is taking place,
> because get_cpu_maps() enough is not suitable.
> 
> Overall I don't like the corner case where get_cpu_maps() returns true
> if a CPU hot{,un}plug operation is taking place in the current CPU
> context.  The guarantee of get_cpu_maps() is that no CPU hot{,un}plug
> operations can be in progress if it returns true.

I'm not convinced of looking at it this way. To me the guarantee is
merely that no CPU operation is taking place _elsewhere_. As indicated,
imo the local CPU should be well aware of what context it's actually in,
and hence what is (or is not) appropriate to do at a particular point in
time.

I guess what I'm missing is an example of a concrete code path where
things presently go wrong.

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:34:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:34:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732205.1138090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLKC-0007iN-7w; Wed, 29 May 2024 15:34:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732205.1138090; Wed, 29 May 2024 15:34:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLKC-0007iG-5L; Wed, 29 May 2024 15:34:48 +0000
Received: by outflank-mailman (input) for mailman id 732205;
 Wed, 29 May 2024 15:34:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCLKB-0007cn-17
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:34:47 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9370afa-1dd0-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:34:45 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-354dfe54738so1427846f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:34:45 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3579354abd1sm11795489f8f.59.2024.05.29.08.34.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:34:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9370afa-1dd0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716996884; x=1717601684; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=pCCA05Bk3lERzojsDOglqz/kiT65TlZvr8I1ZvB6rL8=;
        b=SKtTevj5bEBe/+oa0tqzu7hKyCC3w1gK2jg4XfmP7UHfmVEIW97Y/BNQkw6PFoe7YH
         3Pst1lAdK5rnYWOxFeEcUXisviwcviPmZRCwpJvwGEPwLiG9j5Mcze5vca2IkNrOQvhT
         9er5XwxUqtC7WWlRDNNDGwHBMUtq8tgEbshAQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716996884; x=1717601684;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pCCA05Bk3lERzojsDOglqz/kiT65TlZvr8I1ZvB6rL8=;
        b=WLIAnoWE8Jkb+XoGv0rj3jj0g0lFeypuuB7VtZi9xvdELewHL4bQ9Ef/8FlOULRBEL
         q0/DqO+/iz9ZtVn8KJOcMvoE1iHgmScFsx3lg4FygIixOfxHwjXBwz0NcXM0g0vDjPTw
         jK9GSQwqw+TCDylopm7XmcuCQor9tXRYlzdVnateKIPlP7rrOu/mGr2Rt2YKpO1d7+Io
         wDQsd7Y6WW5Y6CTQtGCtJgkn/tR6UulM1UJgDU1ZLeZaRFnFLXA6qMLIJRFR/dIlvfJ2
         Y1BFphe/aGDJGwP3vzUfJu7KFdGcHGvCdmYH+P8Uvo4a+r7fOMYj5LKHKDuLSaJPFhOE
         72PA==
X-Forwarded-Encrypted: i=1; AJvYcCVm56P6aajOSZnkTt5nwZeoCM8stNXAE6JhlPO/7DEXnotj0BZ5kMzq6mW/BQ1JePSEE7XPYAmInInStDqBVCb84T04geLEmGc/OVDxXwk=
X-Gm-Message-State: AOJu0YyTZN3kGA4LjwlI20uulEbSvOYXR2b+HXXxQ+QVtMIWzrGFAGSO
	679uUZpGP5M/vnni20WanCXu4lIq/+Y5Y3piu2jo5JNwcgpqhwumIFnRAFDnPbCETj8amqqDYlT
	x
X-Google-Smtp-Source: AGHT+IHlnj+ZlxIbrLBaWfVr90N1X8dzZ2rSo1XF0EhFYRGUIOsRr7xWrsu3gmVQ7gipje0ubacDjA==
X-Received: by 2002:adf:f883:0:b0:354:f7b9:75c1 with SMTP id ffacd0b85a97d-355221819ddmr10648025f8f.25.1716996883847;
        Wed, 29 May 2024 08:34:43 -0700 (PDT)
Date: Wed, 29 May 2024 17:34:42 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 1/9] x86/irq: remove offline CPUs from old CPU
 mask when adjusting move_cleanup_count
Message-ID: <ZldLEp0xBQo90uZD@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-2-roger.pau@citrix.com>
 <0a2f1f9a-fab1-409c-96f8-399b19077f12@suse.com>
 <ZldGe6z70R0CD32Z@macbook>
 <b9a667ea-45a6-45d1-8dda-0ec3aa00b9cf@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b9a667ea-45a6-45d1-8dda-0ec3aa00b9cf@suse.com>

On Wed, May 29, 2024 at 05:27:06PM +0200, Jan Beulich wrote:
> On 29.05.2024 17:15, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 02:40:51PM +0200, Jan Beulich wrote:
> >> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>> When adjusting move_cleanup_count to account for CPUs that are offline also
> >>> adjust old_cpu_mask, otherwise further calls to fixup_irqs() could subtract
> >>> those again creating and create an imbalance in move_cleanup_count.
> >>
> >> I'm in trouble with "creating"; I can't seem to be able to guess what you may
> >> have meant.
> > 
> > Oh, sorry, that's a typo.
> > 
> > I was meaning to point out that not removing the already subtracted
> > CPUs from the mask can lead to further calls to fixup_irqs()
> > subtracting them again and move_cleanup_count possibly underflowing.
> > 
> > Would you prefer to write it as:
> > 
> > "... could subtract those again and possibly underflow move_cleanup_count."
> 
> Fine with me. Looks like simply deleting "creating" and keeping the rest
> as it was would be okay too? Whatever you prefer in the end.

Yes, whatever you think it's clearer TBH, I don't really have a
preference.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:48:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:48:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732212.1138101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLXq-0002ew-IL; Wed, 29 May 2024 15:48:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732212.1138101; Wed, 29 May 2024 15:48:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLXq-0002ep-DU; Wed, 29 May 2024 15:48:54 +0000
Received: by outflank-mailman (input) for mailman id 732212;
 Wed, 29 May 2024 15:48:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCLXp-0002ej-Iw
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:48:53 +0000
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com
 [2607:f8b0:4864:20::f35])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f23f3977-1dd2-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 17:48:52 +0200 (CEST)
Received: by mail-qv1-xf35.google.com with SMTP id
 6a1803df08f44-6ad7d743147so11467796d6.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:48:52 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad766b860bsm38247616d6.30.2024.05.29.08.48.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 08:48:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f23f3977-1dd2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716997731; x=1717602531; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=PgRNgiYIQHb6g+NzeBggleQEhdVe/MdIKlY4mjx5K8A=;
        b=auWgE/i/CXcQd1Iwe2lbrpn2sB08KmqpZMOuxf02omjyHryVdpZ+sX0OifTkoS9bNj
         0UpUOibFjgZAXN+ObO9toxUK2XsoWbyTzk/YbSNEcyRmdWWcExSBSinDNQSst7sVba+O
         5FmSvR3gJUbmUCvhRQKIfsxfzD6etJVgl9c3g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716997731; x=1717602531;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PgRNgiYIQHb6g+NzeBggleQEhdVe/MdIKlY4mjx5K8A=;
        b=doxOocdehWwCFj1BekNtFbQV37mjJFMdn8fuPdQINc56zqN1K2cXnaELiHODI+SONI
         CRjyJQca+yyYd6M61vLO9yZGMAC+Y6Q4G8ZVVyPmyKHeNsaAC3/nOBPDoZlJln+Cll5U
         1VTkm8iN7kcT/riJ54yYe9c29W8ZRM2U4OSIMO1GKke6X1QQkM5o5ZHbdV1gL3AWOtAc
         44q+b+74TFdOHolLEz47iU/m0n9b6GXkLxOgz+QlMQMFr6KAbVxtclY/6aTS9v4QUQyR
         jorhPBV+LhBXNqcxYP1yxgmKryoVsdXPFt5ozH3vidF3tPigozX6t6FZsfvnx+bE/Gng
         oQqA==
X-Forwarded-Encrypted: i=1; AJvYcCXAQJPmsz+5pRiaIuQX+IesU35kjfInWmIjCjTZx4kmeMJpqhK8MIFGh+JKgagc1xO2X4tDQEDMukbYY/efbsFQij0VgezB98J17HBQ1FQ=
X-Gm-Message-State: AOJu0Yy41LOlU/PhytjK4bfFUk+uhCVKXJp5T6bF9rZfc6dsJWj2MZxY
	VrbixI/6tQotL4L/aBZSx7DecoYEL/RmN2zI1hAmAPtcuTMtvRWZBG8e5tcZtmA=
X-Google-Smtp-Source: AGHT+IETdG0QZ/AxtnexQjkf3YpxKM1RQB43AjgQshkIFB9F55dvSvLibJ1Ylz9/P9IG/qetI4zMFg==
X-Received: by 2002:a05:6214:519a:b0:6ad:5dbd:7452 with SMTP id 6a1803df08f44-6ad5dbd7b32mr152170496d6.19.1716997731109;
        Wed, 29 May 2024 08:48:51 -0700 (PDT)
Date: Wed, 29 May 2024 17:48:48 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org, Michal Orzel <michal.orzel@amd.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>
Subject: Re: [PATCH for-4.19 2/9] xen/cpu: do not get the CPU map in
 stop_machine_run()
Message-ID: <ZldOYC4lE7eFFJOR@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-3-roger.pau@citrix.com>
 <7954b3c2-e101-4db0-b317-a6585a6a8689@suse.com>
 <ZldHvAW_2nqAuWcq@macbook>
 <b427a74b-60e4-4bdf-96dd-164ea2506ba1@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b427a74b-60e4-4bdf-96dd-164ea2506ba1@suse.com>

On Wed, May 29, 2024 at 05:31:02PM +0200, Jan Beulich wrote:
> On 29.05.2024 17:20, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 03:04:13PM +0200, Jan Beulich wrote:
> >> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>> The current callers of stop_machine_run() outside of init code already have the
> >>> CPU maps locked, and hence there's no reason for stop_machine_run() to attempt
> >>> to lock again.
> >>
> >> While purely from a description perspective this is okay, ...
> >>
> >>> --- a/xen/common/stop_machine.c
> >>> +++ b/xen/common/stop_machine.c
> >>> @@ -82,9 +82,15 @@ int stop_machine_run(int (*fn)(void *data), void *data, unsigned int cpu)
> >>>      BUG_ON(!local_irq_is_enabled());
> >>>      BUG_ON(!is_idle_vcpu(current));
> >>>  
> >>> -    /* cpu_online_map must not change. */
> >>> -    if ( !get_cpu_maps() )
> >>> +    /*
> >>> +     * cpu_online_map must not change.  The only two callers of
> >>> +     * stop_machine_run() outside of init code already have the CPU map locked.
> >>> +     */
> >>
> >> ... the "two" here is not unlikely to quickly go stale; who knows what PPC
> >> and RISC-V will have as their code becomes more complete?
> >>
> >> I'm also unconvinced that requiring ...
> >>
> >>> +    if ( system_state >= SYS_STATE_active && !cpu_map_locked() )
> >>
> >> ... this for all future (post-init) uses of stop_machine_run() is a good
> >> idea. It is quite a bit more natural, to me at least, for the function to
> >> effect this itself, as is the case prior to your change.
> > 
> > This is mostly a pre-req for the next change that switches
> > get_cpu_maps() to return false if the current CPU is holding the CPU
> > maps lock in write mode.
> > 
> > IF we don't want to go this route we need a way to signal
> > send_IPI_mask() when a CPU hot{,un}plug operation is taking place,
> > because get_cpu_maps() enough is not suitable.
> > 
> > Overall I don't like the corner case where get_cpu_maps() returns true
> > if a CPU hot{,un}plug operation is taking place in the current CPU
> > context.  The guarantee of get_cpu_maps() is that no CPU hot{,un}plug
> > operations can be in progress if it returns true.
> 
> I'm not convinced of looking at it this way. To me the guarantee is
> merely that no CPU operation is taking place _elsewhere_. As indicated,
> imo the local CPU should be well aware of what context it's actually in,
> and hence what is (or is not) appropriate to do at a particular point in
> time.
> 
> I guess what I'm missing is an example of a concrete code path where
> things presently go wrong.

See the specific example in patch 3/9 with time_calibration() and it's
usage of send_IPI_mask() when called from a CPU executing in cpu_up()
context.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 15:49:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 15:49:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732216.1138109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLYo-0003OO-P9; Wed, 29 May 2024 15:49:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732216.1138109; Wed, 29 May 2024 15:49:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLYo-0003OH-M4; Wed, 29 May 2024 15:49:54 +0000
Received: by outflank-mailman (input) for mailman id 732216;
 Wed, 29 May 2024 15:49:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RuQ4=NA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCLYn-0003Mj-6E
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 15:49:53 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1547850b-1dd3-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 17:49:50 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a61b70394c0so262353366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 08:49:50 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a633e46509dsm202197866b.98.2024.05.29.08.49.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 08:49:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1547850b-1dd3-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1716997790; x=1717602590; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=FBxhl+/jWPboBiSQPER9GviILA7VLdyEBPTQQv2/cco=;
        b=d4R6Rkia++t/QUw2KsBKdkUPMV9NWcsl1mjE8GVTAiK5OuqdAGXJcrxXjObkUlvYTp
         mqa9CDYCIarIgMESJs5IM4+s1mN083hxzIRywiJqBa3xGMl/6dueMIe/8wk7uDr1yTTJ
         Y3w6V7rjSJnILczke9BbEcbdXp8r2XW2Of+12/gE4E9vFcngKYcKF8HwUeWYjfzaoRnj
         qq0pc6w12QKPysl4FohqQxqOp9eP9dFcq+lSmgaKuKVrXg1CQMnMGCGaDgfpffm3B2cK
         Z+nwSZeP3FnkMIAiInvpjNLQo4iq441e0HQqsQhwRQmoZsqpEebruVFb2vYm9NX0FO9U
         dALg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716997790; x=1717602590;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=FBxhl+/jWPboBiSQPER9GviILA7VLdyEBPTQQv2/cco=;
        b=uwVLZqOCzV9Wlg6SYB8JfCiWAjCUL8Rj3RY7surs0RqPPC5vg5IhL0id8twsvS9qPc
         yApJZqwka3+w4e2qxw4itNankbVEAd7z1h3T+gA0tQ1sSzo1wSQv3Scr4ywhpEYhkIse
         mEUOkcrMMPEkkEN3XNz6HUhUdO5Hpj4Ve+P9dN5uFfdD3d0FSPCPYQFVfkaWt6yiRZPv
         ohPkAvO1n7io4QLtg2FNX0CQuJPu/Tf9qcCWNG55IIUkgcivew6yyqy159GZkc/xbOJY
         J2XUNGzggP2Sgia7zfS511Sj16D7ekD0mGoObH0q6cJk6bGvvwWy6Ar9zctEtn7krHFF
         0wqQ==
X-Forwarded-Encrypted: i=1; AJvYcCViB6qYaEjNEOkUWcNzp/HtsE9ZoYnmIzNyFaURiQnZOR6dhAi6sSPQ7GgGlHSPaM6/jf6+kSevdcNNMOpzVw28wzVKUFF4CJOFK1Wwy0Y=
X-Gm-Message-State: AOJu0Yx0QGKazh4tpPOIV7twwCWOf0QZMYk+KFmjYCK9JzM2sV+/15+x
	zD3ZmWyIO+ecPyKN2EcZx54k0XV+UdjlwFavcPBGyCyZBc2ME1spd5mbNxmFXQ==
X-Google-Smtp-Source: AGHT+IGwmotPgDbZs8ZtMEEXx3sjLnyD0J+H0L/yqp9+ke0KJFcQeAUqbwxB35qMrGEnDLQrkLJp6A==
X-Received: by 2002:a17:906:6549:b0:a59:a857:85ce with SMTP id a640c23a62f3a-a6264f12ae0mr981771266b.52.1716997789910;
        Wed, 29 May 2024 08:49:49 -0700 (PDT)
Message-ID: <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com>
Date: Wed, 29 May 2024 17:49:48 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns false
 if CPU operations are underway
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com> <ZldDvH8GhhQcu5NX@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZldDvH8GhhQcu5NX@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 17:03, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
>>> a cpu_hotplug_{begin,done}() region the function will still return success,
>>> because a CPU taking the rwlock in read mode after having taken it in write
>>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
>>> as it should always return false when called with a CPU hot{,un}plug operation
>>> is in progress.
>>
>> I'm not sure I can agree with this. The CPU doing said operation ought to be
>> aware of what it is itself doing. And all other CPUs will get back false from
>> get_cpu_maps().
> 
> Well, the CPU is aware in the context of cpu_{up,down}(), but not in
> the interrupts that might be handled while that operation is in
> progress, see below for a concrete example.
> 
>>>  Otherwise the logic in send_IPI_mask() for example is wrong,
>>> as it could decide to use the shorthand even when a CPU operation is in
>>> progress.
>>
>> It's also not becoming clear what's wrong there: As long as a CPU isn't
>> offline enough to not be in cpu_online_map anymore, it may well need to still
>> be the target of IPIs, and targeting it with a shorthand then is still fine.
> 
> The issue is in the online path: there's a window where the CPU is
> online (and the lapic active), but cpu_online_map hasn't been updated
> yet.  A specific example would be time_calibration() being executed on
> the CPU that is running cpu_up().  That could result in a shorthand
> IPI being used, but the mask in r.cpu_calibration_map not containing
> the CPU that's being brought up online because it's not yet added to
> cpu_online_map.  Then the number of CPUs actually running
> time_calibration_rendezvous_fn won't match the weight of the cpumask
> in r.cpu_calibration_map.

I see, but maybe only partly. Prior to the CPU having its bit set in
cpu_online_map, can it really take interrupts already? Shouldn't it be
running with IRQs off until later, thus preventing it from making it
into the rendezvous function in the first place? But yes, I can see
how the IRQ (IPI) then being delivered later (once IRQs are enabled)
might cause problems, too.

Plus, with how the rendezvous function is invoked (via
on_selected_cpus() with the mask copied from cpu_online_map), the
first check in smp_call_function_interrupt() ought to prevent the
function from being called on the CPU being onlined. A problem would
arise though if the IPI arrived later and call_data was already
(partly or fully) overwritten with the next request.

>> In any event this would again affect only the CPU leading the CPU operation,
>> which should clearly know at which point(s) it is okay to send IPIs. Are we
>> actually sending any IPIs from within CPU-online or CPU-offline paths?
> 
> Yes, I've seen the time rendezvous happening while in the middle of a
> hotplug operation, and the CPU coordinating the rendezvous being the
> one doing the CPU hotplug operation, so get_cpu_maps() returning true.

Right, yet together with ...

>> Together with the earlier paragraph the critical window would be between the
>> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
>> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
>> then the question would be what bad, if any, would happen to that CPU if an
>> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
>> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
>>
>>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
>>> already hold in write mode by the current CPU, as read_trylock() would
>>> otherwise return true.
>>>
>>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
>>
>> I'm puzzled by this as well: Prior to that and the change referenced by its
>> Fixes: tag, recursive spin locks were used. For the purposes here that's the
>> same as permitting read locking even when the write lock is already held by
>> the local CPU.
> 
> I see, so the Fixes should be:
> 
> x86/smp: use APIC ALLBUT destination shorthand when possible
> 
> Instead, which is the commit that started using get_cpu_maps() in
> send_IPI_mask().

... this I then wonder whether it's really only the condition in
send_IPI_mask() which needs further amending, rather than fiddling with
get_cpu_maps().

Jan


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:03:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:03:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732224.1138120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLlh-0007Zb-RC; Wed, 29 May 2024 16:03:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732224.1138120; Wed, 29 May 2024 16:03:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLlh-0007ZU-NO; Wed, 29 May 2024 16:03:13 +0000
Received: by outflank-mailman (input) for mailman id 732224;
 Wed, 29 May 2024 16:03:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCLlg-0007ZO-Ga
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:03:12 +0000
Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com
 [2607:f8b0:4864:20::72e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f17de2d0-1dd4-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 18:03:10 +0200 (CEST)
Received: by mail-qk1-x72e.google.com with SMTP id
 af79cd13be357-794e5e77ca2so32479885a.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:03:10 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abcc0c0fsm481958085a.49.2024.05.29.09.03.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 09:03:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f17de2d0-1dd4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716998589; x=1717603389; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=9KShG6rduuKgiTh52KMZseWBuFaT08ch1UyxawXl2sg=;
        b=BJSACBpjDUlzGW+XYwQ5WoSn+Q06wgRr4x244VRkNoNosWOyFZsGGvow/WwoTZzyru
         kKaHTjqyRn9F7aD3+U5ZxwmX4ecLckFW25ZtOvWCnf//g42YDVJtGK/xTg+ksJ3Axl2J
         VdyzQ/21PCosIP4bbe+RSco4Nm7Qd7C3IHXTs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716998589; x=1717603389;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9KShG6rduuKgiTh52KMZseWBuFaT08ch1UyxawXl2sg=;
        b=ueaVHWBXwcENll+sCzRKabt0sgooPn1paErQJa+/s1XI6sGFrkr9l42NLA3nM/xAAS
         tQ/wRW2moLGI++DpXQj05u3TaXTulpa3auFzTlX48m08omjoioen+3AbcWHtNImpJ1sm
         xWCf48l+8lJQiJgF93ACXexh3xHpbND4bw786R7lu9xd2XAIMfqilYl1quNMzb6aOBfn
         Fz2biZPm6G9LFfNl5DsG6WFsvVQhI96GRKDilA3W+USW7nSUBNQuNhsZyE6njRmOhccS
         tXtEegF+ZQFSxiyIjJ/I0ii9QBJefST1fVPWtBkxliKdv5XEaP9851ajtodbXTzWDr2m
         N0sw==
X-Gm-Message-State: AOJu0YzKXGMP607RpwFtWZSg+sDNX1I3mCmpf+0aiTGS//ocf7/OIFn4
	5/mJzeFQHtkaFACXc2e+6YiIIWy8P/LjHEK/Po0XC8c+NeIhwniFPiXEFRokE4Y=
X-Google-Smtp-Source: AGHT+IEQHfHuEA36kApM6r2/jvPlfsCNtEE0R+8wJYreAscgWUtRziIYc+Erl2zTQpZIrgojwpf4hw==
X-Received: by 2002:a05:620a:3722:b0:792:c708:601c with SMTP id af79cd13be357-794ab0f4961mr1926559685a.57.1716998588723;
        Wed, 29 May 2024 09:03:08 -0700 (PDT)
Date: Wed, 29 May 2024 18:03:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Dario Faggioli <dfaggioli@suse.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
Message-ID: <ZldRusciaXuwTC9u@macbook>
References: <ZlW-ZyFeAzvh3pGG@macbook>
 <a749c066-0c05-47dd-b499-e712e5454033@suse.com>
 <ZlcjiOJyAuvyy48n@macbook>
 <1147a06d-d9ec-4fa9-a1d0-dfecd7a670dc@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1147a06d-d9ec-4fa9-a1d0-dfecd7a670dc@suse.com>

On Wed, May 29, 2024 at 03:08:49PM +0200, Jürgen Groß wrote:
> On 29.05.24 14:46, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 01:47:09PM +0200, Jürgen Groß wrote:
> > > On 28.05.24 13:22, Roger Pau Monné wrote:
> > > > Hello,
> > > > 
> > > > When the stop_machine_run() call in cpu_down() fails and calls the CPU
> > > > notifier CPU_DOWN_FAILED hook the following assert triggers in the
> > > > scheduling code:
> > > > 
> > > > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
> > > > ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
> > > > CPU:    0
> > > > RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> > > > RFLAGS: 0000000000010093   CONTEXT: hypervisor
> > > > rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
> > > > rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
> > > > rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
> > > > r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> > > > r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
> > > > r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
> > > > cr3: 00000000574c2000   cr2: 0000000000000000
> > > > fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> > > > ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> > > > Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
> > > >    fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
> > > > Xen stack trace from rsp=ffff83203ffffd30:
> > > >      ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
> > > >      0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
> > > >      0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
> > > >      ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
> > > >      ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
> > > >      00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
> > > >      ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
> > > >      ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
> > > >      ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
> > > >      ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
> > > >      ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
> > > >      0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
> > > >      ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
> > > >      0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
> > > >      ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
> > > >      0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > >      0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > >      0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > >      0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > >      0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > Xen call trace:
> > > >      [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> > > >      [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
> > > >      [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
> > > >      [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
> > > >      [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
> > > >      [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
> > > >      [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
> > > >      [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
> > > >      [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
> > > >      [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
> > > >      [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
> > > >      [<ffff82d040324391>] F continue_running+0x5b/0x5d
> > > > 
> > > > 
> > > > ****************************************
> > > > Panic on CPU 0:
> > > > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
> > > > ****************************************
> > > > 
> > > > The issue seems to be that since the CPU hasn't been removed, it's
> > > > still part of prv->initialized and the assert in csched2_free_pdata()
> > > > called as part of free_cpu_rm_data() triggers.
> > > > 
> > > > It's easy to reproduce by substituting the stop_machine_run() call in
> > > > cpu_down() with an error.
> > > 
> > > Could you please give the attached patch a try?
> > 
> > I still get the following assert:
> 
> Oh, silly me. Without core scheduling active nr_sr_unused will be 0 all
> the time. :-(
> 
> Next try.

I'm afraid I have a new trace for you:

Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
----[ Xen-4.19-unstable  x86_64  debug=y  Not tainted ]----
CPU:    0
RIP:    e008:[<ffff82d040247d27>] common/sched/credit2.c#csched2_switch_sched+0x115/0x339
RFLAGS: 0000000000010093   CONTEXT: hypervisor
rax: 000000000000c000   rbx: 0000000000000001   rcx: ffff82d0405e6500
rdx: 0000004feee13000   rsi: 0000000000000004   rdi: ffff83202ecc2f88
rbp: ffff83203ffffc80   rsp: ffff83203ffffc38   r8:  0000000000000000
r9:  ffff83202ecbbf01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
r12: ffff83202ecc2f80   r13: ffff83402ca50100   r14: ffff83402ca50140
r15: ffff83202ecc2f88   cr0: 000000008005003b   cr4: 00000000007526e0
cr3: 00000000574c2000   cr2: 0000000000000000
fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
Xen code around <ffff82d040247d27> (common/sched/credit2.c#csched2_switch_sched+0x115/0x339):
 7c ff ff ff 0f 0b 0f 0b <0f> 0b 0f 0b 41 8b 56 30 89 de 48 8d 3d e8 00 1a
Xen stack trace from rsp=ffff83203ffffc38:
   ffff83203ffffc48 ffff82d0402332ba ffff83203ffffc68 ffff82d04023343d
   0000000000000001 ffff82d0405cf398 ffff83402ca50100 ffff82d0405e6500
   ffff83202ecbbdb0 ffff83203ffffd18 ffff82d040256e1a ffff83203fff386c
   ffff83203fff2000 0000000000000005 ffff83202ecbbf00 ffff83402ca50140
   ffff83203fff3868 0000000000000282 0000000040233509 ffff83202ecbbdb0
   ffff83402ca50100 ffff83202f3e6d80 ffff83202ecc2ec0 ffff83202ecc2ec0
   0000000000000001 ffff82d0403da460 0000000000000048 0000000000000000
   ffff83203ffffd48 ffff82d0402414b7 0000000000000001 0000000000000000
   ffff82d0403da460 0000000000000006 ffff83203ffffd70 ffff82d04024173d
   0000000000000000 0000000000000001 ffff82d0404c4430 ffff83203ffffda0
   ffff82d04021f1f9 ffff82d0404c4628 ffff82d0404c4620 ffff82d0404c4430
   0000000000000006 ffff83203ffffde8 ffff82d04022bb2f ffff83203ffffe10
   0000000000000001 0000000000000001 0000000000000000 ffff83203ffffe10
   0000000000000000 ffff82d0405e6500 ffff83203ffffe00 ffff82d040204fd5
   0000000000000001 ffff83203ffffe30 ffff82d040205464 ffff82d0404c5860
   0000000000000001 ffff83202ec86000 0000000000000000 ffff83203ffffe48
   ffff82d040348c32 ffff83402ca500d0 ffff83203ffffe68 ffff82d04020708d
   ffff83202ec861d0 ffff82d0405ce210 ffff83203ffffe80 ffff82d0402342a3
   ffff82d0405ce200 ffff83203ffffeb0 ffff82d04023450b 0000000000000000
   0000000000007fff ffff82d0405d5080 ffff82d0405ce210 ffff83203ffffee8
Xen call trace:
   [<ffff82d040247d27>] R common/sched/credit2.c#csched2_switch_sched+0x115/0x339
   [<ffff82d040256e1a>] F schedule_cpu_add+0x1a4/0x463
   [<ffff82d0402414b7>] F common/sched/cpupool.c#cpupool_assign_cpu_locked+0x5a/0x17e
   [<ffff82d04024173d>] F common/sched/cpupool.c#cpupool_cpu_add+0x162/0x16c
   [<ffff82d04021f1f9>] F common/sched/cpupool.c#cpu_callback+0x10e/0x466
   [<ffff82d04022bb2f>] F notifier_call_chain+0x6c/0x96
   [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
   [<ffff82d040205464>] F cpu_down+0x60/0x83
   [<ffff82d040348c32>] F cpu_down_helper+0x11/0x27
   [<ffff82d04020708d>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
   [<ffff82d0402342a3>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
   [<ffff82d04023450b>] F do_tasklet+0x5b/0x8d
   [<ffff82d040321372>] F arch/x86/domain.c#idle_loop+0x78/0xe6
   [<ffff82d0403242f2>] F continue_running+0x5b/0x5d


****************************************
Panic on CPU 0:
Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
****************************************

This time is one of the asserts in init_pdata().

Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:15:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:15:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732231.1138129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLxH-0001u0-VT; Wed, 29 May 2024 16:15:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732231.1138129; Wed, 29 May 2024 16:15:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLxH-0001tt-Su; Wed, 29 May 2024 16:15:11 +0000
Received: by outflank-mailman (input) for mailman id 732231;
 Wed, 29 May 2024 16:15:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FJ4/=NA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCLxH-0001tn-Fl
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:15:11 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e2c68f0-1dd6-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 18:15:09 +0200 (CEST)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-6ab9dacc38fso11255186d6.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:15:09 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad78aaca11sm36663656d6.22.2024.05.29.09.14.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 09:14:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e2c68f0-1dd6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1716999308; x=1717604108; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=OYlk3/tTSyjPQV57UqgUOb99llBbT/CYPx7FTSIdvvU=;
        b=wPQVchr90NCze0iGFt/PFy/P+CQ3cdn76xQTE8t4a5N4VhRNMolYOj7Qjb+5NSAF6Y
         ULJ8ck7GpEg7gVG/w3qvp8/76ydkoQpO54KqL/FqLI5icIJ32TB/J8mC+pcoJ6wrcsjO
         WMl+pfU1Jenevyn9Hb0A4D+xpML68DhdV7DDc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716999308; x=1717604108;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OYlk3/tTSyjPQV57UqgUOb99llBbT/CYPx7FTSIdvvU=;
        b=Q6jUXC3gcmVBpPUbm9ODs6Q9bgmlMfaMqe43q4Kf0FWUL6lBbA8xVgFY5dlBKxv+H2
         tXRFBYiYXuHk0OgIP3LioYYrxDutabwjCZA8Z8ZwePOm1IjzqLiwOG8eX8LPZB1e63vt
         xRGMTP3SQ/qKb7OgJ6imiVYI0i1B8fNlI7VSQZ0bynYzKdCWAbmyux00HUbCTd8Cczqb
         u39+dqYHhZkPRmNw0Lf0ibV2nHxQQloXRKa0uCcRYqV/HNEmX2PVsEX6QmeYOlzsOokZ
         2El2VF6aE/0pomsJtuFL9MDZC35gGuBhTJo6DkBUyFgCUB2sKQFOidxbrk6lIDwTFkFt
         GMTQ==
X-Forwarded-Encrypted: i=1; AJvYcCV3FK36V2mfBdFPs1jZSADUCvRmt5JtnuQkRRdTDpTsnSwXNhERA5FArkCZPWGdauuXTOmcO55t+TxaM9uy/SrZqoI7VXrJXE0byaov8XA=
X-Gm-Message-State: AOJu0YznAXgMIzKajQcNiC34JfhYzeMBaWZG20x2fWE24TMfNNCjjZnZ
	yq3OlyNa3xzZ/v+67INnSR705OHULi8JX3t4v6vB4JdxvaLf7CSvy3HhF6e8dPc=
X-Google-Smtp-Source: AGHT+IGOgmQlqiTgJH5OV7NzBiyz75sHSeV8hMSb68pIIrS+e9Q5J30WGnyvRuXhm6svKKPMraV6kA==
X-Received: by 2002:a05:6214:5541:b0:6ad:a1d8:6e56 with SMTP id 6a1803df08f44-6ada1d86fa5mr23717636d6.6.1716999307777;
        Wed, 29 May 2024 09:15:07 -0700 (PDT)
Date: Wed, 29 May 2024 18:14:56 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns
 false if CPU operations are underway
Message-ID: <ZldUgJRJvytRsjhu@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>
 <ZldDvH8GhhQcu5NX@macbook>
 <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com>

On Wed, May 29, 2024 at 05:49:48PM +0200, Jan Beulich wrote:
> On 29.05.2024 17:03, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
> >> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
> >>> a cpu_hotplug_{begin,done}() region the function will still return success,
> >>> because a CPU taking the rwlock in read mode after having taken it in write
> >>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
> >>> as it should always return false when called with a CPU hot{,un}plug operation
> >>> is in progress.
> >>
> >> I'm not sure I can agree with this. The CPU doing said operation ought to be
> >> aware of what it is itself doing. And all other CPUs will get back false from
> >> get_cpu_maps().
> > 
> > Well, the CPU is aware in the context of cpu_{up,down}(), but not in
> > the interrupts that might be handled while that operation is in
> > progress, see below for a concrete example.
> > 
> >>>  Otherwise the logic in send_IPI_mask() for example is wrong,
> >>> as it could decide to use the shorthand even when a CPU operation is in
> >>> progress.
> >>
> >> It's also not becoming clear what's wrong there: As long as a CPU isn't
> >> offline enough to not be in cpu_online_map anymore, it may well need to still
> >> be the target of IPIs, and targeting it with a shorthand then is still fine.
> > 
> > The issue is in the online path: there's a window where the CPU is
> > online (and the lapic active), but cpu_online_map hasn't been updated
> > yet.  A specific example would be time_calibration() being executed on
> > the CPU that is running cpu_up().  That could result in a shorthand
> > IPI being used, but the mask in r.cpu_calibration_map not containing
> > the CPU that's being brought up online because it's not yet added to
> > cpu_online_map.  Then the number of CPUs actually running
> > time_calibration_rendezvous_fn won't match the weight of the cpumask
> > in r.cpu_calibration_map.
> 
> I see, but maybe only partly. Prior to the CPU having its bit set in
> cpu_online_map, can it really take interrupts already? Shouldn't it be
> running with IRQs off until later, thus preventing it from making it
> into the rendezvous function in the first place? But yes, I can see
> how the IRQ (IPI) then being delivered later (once IRQs are enabled)
> might cause problems, too.

The interrupt will get set in IRR and handled when interrupts are
enabled.

> 
> Plus, with how the rendezvous function is invoked (via
> on_selected_cpus() with the mask copied from cpu_online_map), the
> first check in smp_call_function_interrupt() ought to prevent the
> function from being called on the CPU being onlined. A problem would
> arise though if the IPI arrived later and call_data was already
> (partly or fully) overwritten with the next request.

Yeah, there's a small window where the fields in call_data are out of
sync.

> >> In any event this would again affect only the CPU leading the CPU operation,
> >> which should clearly know at which point(s) it is okay to send IPIs. Are we
> >> actually sending any IPIs from within CPU-online or CPU-offline paths?
> > 
> > Yes, I've seen the time rendezvous happening while in the middle of a
> > hotplug operation, and the CPU coordinating the rendezvous being the
> > one doing the CPU hotplug operation, so get_cpu_maps() returning true.
> 
> Right, yet together with ...
> 
> >> Together with the earlier paragraph the critical window would be between the
> >> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
> >> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
> >> then the question would be what bad, if any, would happen to that CPU if an
> >> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
> >> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
> >>
> >>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
> >>> already hold in write mode by the current CPU, as read_trylock() would
> >>> otherwise return true.
> >>>
> >>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
> >>
> >> I'm puzzled by this as well: Prior to that and the change referenced by its
> >> Fixes: tag, recursive spin locks were used. For the purposes here that's the
> >> same as permitting read locking even when the write lock is already held by
> >> the local CPU.
> > 
> > I see, so the Fixes should be:
> > 
> > x86/smp: use APIC ALLBUT destination shorthand when possible
> > 
> > Instead, which is the commit that started using get_cpu_maps() in
> > send_IPI_mask().
> 
> ... this I then wonder whether it's really only the condition in
> send_IPI_mask() which needs further amending, rather than fiddling with
> get_cpu_maps().

That the other option, but I have impression it's more fragile to
adjust the condition in send_IPI_mask() rather than fiddle with
get_cpu_maps().

However if that's the preference I can adjust.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:16:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:16:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732235.1138140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLyy-0002WA-9o; Wed, 29 May 2024 16:16:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732235.1138140; Wed, 29 May 2024 16:16:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCLyy-0002W3-6m; Wed, 29 May 2024 16:16:56 +0000
Received: by outflank-mailman (input) for mailman id 732235;
 Wed, 29 May 2024 16:16:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCLyx-0002Vx-LE
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:16:55 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc3b7df4-1dd6-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 18:16:52 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5785f861868so2858914a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:16:52 -0700 (PDT)
Received: from [10.80.5.21] ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626c93d38esm738158266b.90.2024.05.29.09.16.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 09:16:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc3b7df4-1dd6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1716999412; x=1717604212; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=uSaFLAXaJOhnhik3iHCg3RZxOkBFRX4xe9SATGr9Lfc=;
        b=VbS8tQ/H8Xc1ocDBhLgq/wAwbC02jROk7Ba401IB4qRiKUJUg+NkBBYivARJXMT9a8
         K3+KQ0UFD4awfdy2aHGtDTOEKmbI+1U1qIvfMyrQLptiHu50SLFyTZs+FyEVml+dkkUO
         cHopFXQnZwydLiHE3jbnAVSB8Ie/IUFJJ3KAg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1716999412; x=1717604212;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=uSaFLAXaJOhnhik3iHCg3RZxOkBFRX4xe9SATGr9Lfc=;
        b=VNSMDhlZliqgAVjFfBw/KbGO6zqSpIfcarsy5i48T3BEzGWpGkDAQCxq2BqakMDNVa
         9eGbZ2mM339rEAyl5lHJz4gK4Sb149Yln7AtcNRtqq63lkNmq9mdW3kUXG3+By1NDcQg
         uYGLoTuWvONkzZiTilGY5krtEmeKEytq3aCJ7hONYffeedteUz67Zw7OgSrWLzAQJNZh
         pgFk6jPjGvqf/yhdORPo+p/02VokXyV67tWtqr+belqt3NPF1zdFIUNo8alrAgH5njxb
         RUtPm2GnhuVV5CWCFaoal0ZfFrHkB9sC2eINRYgRjpdZjmEtQq0PTd8K8ub92JdZm1C1
         LXKQ==
X-Gm-Message-State: AOJu0Ywv0CGe4yG1gFXwyI79n4LIgkdFCrhuIgbBijn8qrLTWrULz83E
	PjvR1Q3xabAi498ECftdQMg0BO2hNVU3Z2RFBvIVcS42nm0lqhUNzBj6oB4hekLGXl1nYEWNDGu
	z
X-Google-Smtp-Source: AGHT+IFFxyxzsg9OyiN4hyIsTNVdhvWQ+UictNf335WOqAAsGG2PXqST1pyX/TzB9GZPMTf1iKL+Gw==
X-Received: by 2002:a17:906:184d:b0:a5a:8c59:d6d with SMTP id a640c23a62f3a-a62641c7f50mr1040014266b.17.1716999411884;
        Wed, 29 May 2024 09:16:51 -0700 (PDT)
Message-ID: <e554d88d-2164-4dde-9083-8e8db5a2712f@cloud.com>
Date: Wed, 29 May 2024 17:16:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/6] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Anthony PERARD <anthony@xenproject.org>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <eb7438639cc4755d3c959029d809451ee000ccbc.1716976271.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <eb7438639cc4755d3c959029d809451ee000ccbc.1716976271.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/05/2024 15:32, Alejandro Vallejo wrote:
> +static uint32_t read_apic_id(void)
> +{
> +    uint32_t apic_id;
> +
> +    if ( has_x2apic )
> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
> +    else
> +    {
> +        cpuid(1, NULL, &apic_id, NULL, NULL);
> +        apic_id >>= 24;
> +    }
> +
> +    /* Never called by cpu0, so should never return 0 */
> +    ASSERT(!apic_id);

Bah! Typo. That's meant to be ASSERT(apic_id).

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:29:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:29:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732242.1138149 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMAl-0004we-9F; Wed, 29 May 2024 16:29:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732242.1138149; Wed, 29 May 2024 16:29:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMAl-0004wX-6f; Wed, 29 May 2024 16:29:07 +0000
Received: by outflank-mailman (input) for mailman id 732242;
 Wed, 29 May 2024 16:29:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCMAk-0004wR-DY
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:29:06 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9048a480-1dd8-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 18:29:04 +0200 (CEST)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-529614b8c29so3716482e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:29:04 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52970e1ee18sm1309041e87.232.2024.05.29.09.29.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 09:29:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9048a480-1dd8-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717000144; x=1717604944; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=FJaU8tLMW6SaZCgrZmBW0/o64XVc5CcWTPBqhIdX+/s=;
        b=Id/2d4WIBVa1fBZ+yR5E7HkCuozUbRwTYfB179wtyEusuHth6SssShNiBGoVhhv2kz
         vvMQNZr435vkUufL+vL4XvrBv9JzPXPxLCcEhgJDk7m1hfd1Dp/uRBpLIx7L6+IZ1Bdh
         3ZbZ/P0AgpCGIzjGlF759KWQ7CQVoqBlKhtcAgDslukjC9RrrUYhXr5izLVK31bHA/Yb
         +yoGQrXhP3vPBNNl6RQAiGbEf6pwKmHfaB6GJJJVI6C51jCveO/9pCMirbtDcFywVQtv
         FaZgNeM3vvb3Xngq1uv2p9ThuSHpJdXGql3FU7dLoEeIxZdC0gFta2fiIF+zU28B5Mi9
         6OBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717000144; x=1717604944;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FJaU8tLMW6SaZCgrZmBW0/o64XVc5CcWTPBqhIdX+/s=;
        b=klvLA9Iv9MSOhRMYIidbi+d4wTbOorczH4Yp8nstCkM4B4jQEbJesaKJn/0DhzfVXq
         z2gdOp+fNJsPUNcemcVHOmWdi139qpMr6eX5eMyjzcrjLjJrcMHfKcRg11ELNIR2c4nX
         bHRbCWhqOxZKtpgE510Oy0TBZ7HrxHaGKiKvn8BI6DkwgHd8yvB2jFFbkg6jI3HHtgaq
         xJUCk22ERWjagyfg0YmHulcaNqUYuFnNnTgmmN0UwJJgahJXNcL2IJ2k3nvtuPNAAHZb
         d/cU1rT7TN9c5jqZ+v2juqYro2wXKf/LaipINzrZNOZL2Z2cQ14LdVbuVP928OJAxlvX
         H2rA==
X-Forwarded-Encrypted: i=1; AJvYcCV7M+WS30W1F4vFVTrBrAZ61C8ItoB+BnboUrxeXJqcXraOhSNp0EmjtTk4dpffFM8YL3xBGGBjj6FkZ3lSUcS/Sqfrv0juAMpJJfx6xRc=
X-Gm-Message-State: AOJu0YzfND3vCm5iPHZyjd+SGlx7mUtjNXEUcooOhWKjrmS4AKnSoRFJ
	0x/GNPVYKN3jpXcnuchIpwQwyjfZ+MISYlh3fyFixCWh8+b4E9nW
X-Google-Smtp-Source: AGHT+IGpsJ6MbZ1ydNDz4Wn74sLgTSA4Ovs9CeTRsL+E0PWrN58GFqmMRJTG3xV7Ro05SPh2IoPLQA==
X-Received: by 2002:a19:f716:0:b0:51f:36ed:ee43 with SMTP id 2adb3069b0e04-52965f10d17mr9895087e87.31.1717000143463;
        Wed, 29 May 2024 09:29:03 -0700 (PDT)
Message-ID: <1b301099b71c61c8e537ec6eb8fe064c3c2348f3.camel@gmail.com>
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,  Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 18:29:02 +0200
In-Reply-To: <23790bd6-e06a-4adc-90a9-517f947be00a@suse.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
	 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
	 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
	 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
	 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
	 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
	 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
	 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
	 <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
	 <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
	 <23790bd6-e06a-4adc-90a9-517f947be00a@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 17:22 +0200, Jan Beulich wrote:
> On 29.05.2024 16:58, Oleksii K. wrote:
> > static always_inline bool test_bit(int nr, const volatile void
> > *addr)On
> > Wed, 2024-05-29 at 12:06 +0200, Jan Beulich wrote:
> > > On 29.05.2024 11:59, Julien Grall wrote:
> > > > I didn't realise this was an existing comment. I think the
> > > > suggestion is=20
> > > > a little bit odd because you could use the atomic version of
> > > > the
> > > > helper.
> > > >=20
> > > > Looking at Linux, the second sentence was dropped. But not the
> > > > first=20
> > > > one. I would suggest to do the same. IOW keep:
> > > >=20
> > > > "
> > > > If two examples of this operation race, one can appear to
> > > > succeed
> > > > but=20
> > > > actually fail.
> > > > "
> > >=20
> > > As indicated, I'm okay with that being retained, but only in a
> > > form
> > > that
> > > actually makes sense. I've explained before (to Oleksii) what I
> > > consider
> > > wrong in this way of wording things.
> >=20
> > Would it be suitable to rephrase it in the following way:
> > =C2=A0=C2=A0=C2=A0=C2=A0 * This operation is non-atomic and can be reor=
dered.
> > =C2=A0=C2=A0 - * If two examples of this operation race, one can appear=
 to
> > =C2=A0=C2=A0 succeed
> > =C2=A0=C2=A0 - * but actually fail.=C2=A0 You must protect multiple acc=
esses with
> > a
> > =C2=A0=C2=A0 lock.
> > =C2=A0=C2=A0 + * If two instances of this operation race, one may succe=
ed in
> > =C2=A0=C2=A0 updating
> > =C2=A0=C2=A0 + * the bit in memory, but actually fail. It should be pro=
tected
> > =C2=A0=C2=A0 from
> > =C2=A0=C2=A0 + * potentially racy behavior.
> > =C2=A0=C2=A0=C2=A0=C2=A0 */
> > =C2=A0=C2=A0=C2=A0 static always_inline bool
> > =C2=A0=C2=A0=C2=A0 __test_and_set_bit(int nr, volatile void *addr)
>=20
> You've lost the "appear to" ahead of "succeed". Yet even with the
> adjustment
> I still don't see what the "appear to succeed" really is supposed to
> mean
> here. The issue (imo) isn't with success or failure, but with the
> write of
> one CPU potentially being immediately overwritten by another CPU, not
> observing the bit change that the first CPU did. IOW both will
> succeed (or
> appear to succeed), but one update may end up being lost. Maybe "...,
> both
> may update memory with their view of the new value, not taking into
> account
> the update the respectively other one did"? Or "..., both will appear
> to
> succeed, but one of the updates may be lost"?
For me, the first one is clearer.

Julien, would that be okay for you?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Wed May 29 16:36:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:36:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732248.1138159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMHY-0007Aq-Vt; Wed, 29 May 2024 16:36:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732248.1138159; Wed, 29 May 2024 16:36:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMHY-0007Aj-Sz; Wed, 29 May 2024 16:36:08 +0000
Received: by outflank-mailman (input) for mailman id 732248;
 Wed, 29 May 2024 16:36:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCMHX-0007Ad-Og
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:36:07 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8c1d9dbf-1dd9-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 18:36:06 +0200 (CEST)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e6f2534e41so23365431fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:36:06 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2e95bf2cd3bsm25322251fa.140.2024.05.29.09.36.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 09:36:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c1d9dbf-1dd9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717000566; x=1717605366; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=T70BVXJlLVIQvl4XQvyZ5Z7U/u96RL4MJT8N1NZdq5k=;
        b=KDSb5TZwULl006yTugLsrab6xDOI5NTdVuFxA0jGilWRc6sZFEt4HtnIbgOl+wuYPy
         vlC8kHD/vYQRRJnBPeRkStHeQYePcXJJ/f5zdAPc5aLjF/NlojhWDMiP25nXOd+vDCKy
         6ldZuZ4jfo5i7WBbNp/Q8eXR6Z5YF+dhwO488DvJKCFWzy7NMmJK6mYVzFkvIYEDwjPO
         3YxWt5ReWteoGQt2M6JOC5O+/984SrLmmV0Nycoy30/SPuFHkNHp+MT7RxxdqdbZm2So
         cUZr6+0dRRWIpZryHblHaE4AJczIK1HBAVEvswsy6j0h/kp/T9vadwocIfpzBCXAva7s
         AhJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717000566; x=1717605366;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T70BVXJlLVIQvl4XQvyZ5Z7U/u96RL4MJT8N1NZdq5k=;
        b=HQvQGdRX2s6UYRg6U38we81/hmNwPQrjoLe/ny4LYWKlNptguGx4o8DpZqQb+sD4dU
         00sjQTKs9tZlKDEc07FUby2U2ciBqbsvslMVe+G0SuqKOWOLtiXX9cRiNjgGzFCtKCbL
         RnvHSs7pq8uRlfFtk7MFc4kEKiV0t46Yvg5svRJkmmDFNe3xwW4CIqGfBCoo3D/Qc2db
         36QBzm3k6EJAkDTOiGuuKn8tzQxY0hHzmXZJ+KDPSnGcS4Fd9qg8U41gl0zh9zRoNM2A
         W0DukzKKK2upr1oCyzaHqbtkHl/LGkW8yspTTn74QQ0YKZwovLhiG4lnZDPaTlEOflhi
         x8ow==
X-Forwarded-Encrypted: i=1; AJvYcCWcADMee+XiC56AsjhRQ2wvn7JeZ997vJWsAuGK5Wj4Gwn0A9EpyifTmhLbg8CRTz1Udb1l1cLSPwfYcoXmEE9sxY4v2sOXRIe1GqFO5E0=
X-Gm-Message-State: AOJu0YwvIfXDUadh65itJAfMRxjk2rIrjwnyyNt3TJ+25TblmRVhGNlu
	446eZ/sspR5L6mr9E0fJQ86w6BHSxr//uc30mWO0ZG3/2ceWa7Yy
X-Google-Smtp-Source: AGHT+IFov/sg0n7n/D64osxKiEyFqQyXiOZE6fNPicZtHRZtPgIMD+F/8xmlyur2pGBYQH25HfNqWw==
X-Received: by 2002:a2e:7212:0:b0:2ea:79ab:6709 with SMTP id 38308e7fff4ca-2ea79ab6889mr16763201fa.31.1717000566102;
        Wed, 29 May 2024 09:36:06 -0700 (PDT)
Message-ID: <3c33c0cc45a8c3b21e5cf8addf2cd34ae017a648.camel@gmail.com>
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,  Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Date: Wed, 29 May 2024 18:36:04 +0200
In-Reply-To: <1b301099b71c61c8e537ec6eb8fe064c3c2348f3.camel@gmail.com>
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
	 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
	 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
	 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
	 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
	 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
	 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
	 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
	 <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
	 <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
	 <23790bd6-e06a-4adc-90a9-517f947be00a@suse.com>
	 <1b301099b71c61c8e537ec6eb8fe064c3c2348f3.camel@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 18:29 +0200, Oleksii K. wrote:
> On Wed, 2024-05-29 at 17:22 +0200, Jan Beulich wrote:
> > On 29.05.2024 16:58, Oleksii K. wrote:
> > > static always_inline bool test_bit(int nr, const volatile void
> > > *addr)On
> > > Wed, 2024-05-29 at 12:06 +0200, Jan Beulich wrote:
> > > > On 29.05.2024 11:59, Julien Grall wrote:
> > > > > I didn't realise this was an existing comment. I think the
> > > > > suggestion is=20
> > > > > a little bit odd because you could use the atomic version of
> > > > > the
> > > > > helper.
> > > > >=20
> > > > > Looking at Linux, the second sentence was dropped. But not
> > > > > the
> > > > > first=20
> > > > > one. I would suggest to do the same. IOW keep:
> > > > >=20
> > > > > "
> > > > > If two examples of this operation race, one can appear to
> > > > > succeed
> > > > > but=20
> > > > > actually fail.
> > > > > "
> > > >=20
> > > > As indicated, I'm okay with that being retained, but only in a
> > > > form
> > > > that
> > > > actually makes sense. I've explained before (to Oleksii) what I
> > > > consider
> > > > wrong in this way of wording things.
> > >=20
> > > Would it be suitable to rephrase it in the following way:
> > > =C2=A0=C2=A0=C2=A0=C2=A0 * This operation is non-atomic and can be re=
ordered.
> > > =C2=A0=C2=A0 - * If two examples of this operation race, one can appe=
ar to
> > > =C2=A0=C2=A0 succeed
> > > =C2=A0=C2=A0 - * but actually fail.=C2=A0 You must protect multiple a=
ccesses
> > > with
> > > a
> > > =C2=A0=C2=A0 lock.
> > > =C2=A0=C2=A0 + * If two instances of this operation race, one may suc=
ceed
> > > in
> > > =C2=A0=C2=A0 updating
> > > =C2=A0=C2=A0 + * the bit in memory, but actually fail. It should be
> > > protected
> > > =C2=A0=C2=A0 from
> > > =C2=A0=C2=A0 + * potentially racy behavior.
> > > =C2=A0=C2=A0=C2=A0=C2=A0 */
> > > =C2=A0=C2=A0=C2=A0 static always_inline bool
> > > =C2=A0=C2=A0=C2=A0 __test_and_set_bit(int nr, volatile void *addr)
> >=20
> > You've lost the "appear to" ahead of "succeed". Yet even with the
> > adjustment
> > I still don't see what the "appear to succeed" really is supposed
> > to
> > mean
> > here. The issue (imo) isn't with success or failure, but with the
> > write of
> > one CPU potentially being immediately overwritten by another CPU,
> > not
> > observing the bit change that the first CPU did. IOW both will
> > succeed (or
> > appear to succeed), but one update may end up being lost. Maybe
> > "...,
> > both
> > may update memory with their view of the new value, not taking into
> > account
> > the update the respectively other one did"? Or "..., both will
> > appear
> > to
> > succeed, but one of the updates may be lost"?
> For me, the first one is clearer.
If then this part of the comment is needed for test_bit() as it is
doing only reading?

> Julien, would that be okay for you?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Wed May 29 16:43:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:43:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732253.1138171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMO8-0000cK-NI; Wed, 29 May 2024 16:42:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732253.1138171; Wed, 29 May 2024 16:42:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMO8-0000cD-Ig; Wed, 29 May 2024 16:42:56 +0000
Received: by outflank-mailman (input) for mailman id 732253;
 Wed, 29 May 2024 16:42:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygCr=NA=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCMO7-0000c7-28
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:42:55 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7ecea0d3-1dda-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 18:42:54 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-57857e0f45cso2634434a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:42:54 -0700 (PDT)
Received: from [10.80.5.21] ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda8bb3sm732696066b.223.2024.05.29.09.42.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 09:42:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ecea0d3-1dda-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1717000973; x=1717605773; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to
         :cc:subject:date:message-id:reply-to;
        bh=S6EOVyZktDMpdICGYNJ8jso8s2pDMedSOIV5YlgQ6wo=;
        b=diglAESMqX6hjjPE72ag6ZtmK+ETDGFL870EaxLnS3vkTtXss0t/T5GuxmbU/frEjm
         RKNytV1nC/83krnYXGDNPZjgd2iAYR1ubL07nWMfKpwFWIYUIb0cFY4jd7/qITVfuvZb
         NWokXvJs+0ciJxcF/kRsi1H0A7NmEyaDInIYc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717000973; x=1717605773;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:from:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=S6EOVyZktDMpdICGYNJ8jso8s2pDMedSOIV5YlgQ6wo=;
        b=IR4SexDsjIFPYMUUx7sK3ccaUewNYXgpNKv9cN6MdoRqosLyPW2PzMLNmrJkl9px3T
         amN0xotEZmJnT0Aam9nkpD6PK+kK3AuG7saR9WZnK5ThyzmJdBtCcbRIFSAHXJnwOBAj
         OIGbablSP++h2M5yfmbReIHsth8W9XOPMKeFWxRERJbN9/K6+lmVWgU4CpoO5ilXGzqM
         EJgFcryc7CL8wkoMktoV9ku2hE1JXZIcnkvJkmf5acam3hjagWDp6SmaWS5VK7VU1dPG
         wSOA9bmQ6cNImaYA8CX3KZoUr0qE8lcbJ8FwYpYaTDnwxyeG9579yUXB1c25fBuDjrvP
         HgSQ==
X-Gm-Message-State: AOJu0YxHglS//AwtOIGEq40tV8veaLmP2s6rB9Dvi8AhpnEH8cr8e+Fw
	urTVDg7C/HNNOBOssErz9m2RraH71/Kk+kW9IxaUtD39mjlT78n6U2XfpB+uWpybdekDQmYJpgt
	fbvE=
X-Google-Smtp-Source: AGHT+IGoopiv/+wn4+20sMzy2oHAxCrF8jnOmo/8eJfLfajmg8UQILBgV4jERa9VaSSjayqzyy1QuQ==
X-Received: by 2002:a17:906:5a83:b0:a5a:1579:9033 with SMTP id a640c23a62f3a-a62641aa499mr950499966b.1.1717000973201;
        Wed, 29 May 2024 09:42:53 -0700 (PDT)
Message-ID: <f2eab8cc-e434-4a51-aee3-9e8344ffd4f8@cloud.com>
Date: Wed, 29 May 2024 17:42:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/6] tools/hvmloader: Retrieve (x2)APIC IDs from the
 APs themselves
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Anthony PERARD <anthony@xenproject.org>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <eb7438639cc4755d3c959029d809451ee000ccbc.1716976271.git.alejandro.vallejo@cloud.com>
 <e554d88d-2164-4dde-9083-8e8db5a2712f@cloud.com>
Content-Language: en-GB
In-Reply-To: <e554d88d-2164-4dde-9083-8e8db5a2712f@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/05/2024 17:16, Alejandro Vallejo wrote:
> On 29/05/2024 15:32, Alejandro Vallejo wrote:
>> +static uint32_t read_apic_id(void)
>> +{
>> +    uint32_t apic_id;
>> +
>> +    if ( has_x2apic )
>> +        cpuid(0xb, NULL, NULL, NULL, &apic_id);
>> +    else
>> +    {
>> +        cpuid(1, NULL, &apic_id, NULL, NULL);
>> +        apic_id >>= 24;
>> +    }
>> +
>> +    /* Never called by cpu0, so should never return 0 */
>> +    ASSERT(!apic_id);
> 
> Bah! Typo. That's meant to be ASSERT(apic_id).
> 
> Cheers,
> Alejandro

And something else is broke, because I can't boot HVM guests anymore.
Thought we had HVM tests in gitlab, but apparently we don't.

https://gitlab.com/xen-project/people/agvallejo/xen/-/pipelines/1309033234

Closest I see is booting alpine, but that's a 1-vCPU PV case, which
wouldn't expose this.

I'll resend the series after weeding out the bugs.

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:50:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:50:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732261.1138181 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMV6-0002na-HS; Wed, 29 May 2024 16:50:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732261.1138181; Wed, 29 May 2024 16:50:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMV6-0002nT-Cy; Wed, 29 May 2024 16:50:08 +0000
Received: by outflank-mailman (input) for mailman id 732261;
 Wed, 29 May 2024 16:50:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCMV6-0002nN-1E
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 16:50:08 +0000
Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com
 [2607:f8b0:4864:20::82e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8096ee38-1ddb-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 18:50:07 +0200 (CEST)
Received: by mail-qt1-x82e.google.com with SMTP id
 d75a77b69052e-43fdf8a6437so8374021cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 09:50:07 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb17fbebdsm55679141cf.46.2024.05.29.09.50.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 09:50:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8096ee38-1ddb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717001406; x=1717606206; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=apUfwLDUrvnO2geTFFcQGuCY0nhi+UFcWu3hendT4M0=;
        b=jZ1hE1BGwqnTMdptv9yNR2pqVGwP/WXA9Vzzcdn2Pp75b5RehXDhuBNYpcURjU+QU7
         5DuXo4W8/l2Ghfxc11RcvoRNTCni6IHT1xQ4lpKHKjwtwzlBWfgRX+LufbjaDphbvdev
         XYE8EQlJkE9bhghqh9vacdIdS/WjP8e5yflWw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717001406; x=1717606206;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=apUfwLDUrvnO2geTFFcQGuCY0nhi+UFcWu3hendT4M0=;
        b=oqVq/5TgUjP4NurypuZe6FfuB3vK6AtnG3jMpHFODINXf4e5LDGS5/JRcrZETk9CZp
         YGxBOHa6H41j4/KBbKomA8ab9gsbhIcK8puC/vrecMvy5/k2Fn9KPFb9nZ+sCl0S4jCg
         Zngfo1HLEnxqVMbLyOv7PWQ3zADWofPHInLHZ/sodGqzk9kP7QKTdv70pIfE/R4WpcHs
         il02LfAhWRv0iAz4P+x1ZY3ErApgInJH8jwOU+lA2p4qCmgm54ek/oPh+sK80+vbai/S
         59aMlFrhGoSBuNIVwj5kXg811Cs7xpO+ROl5vgN7ygSxy8OVAwSQKPxvD6bohOOUxV08
         Nq4w==
X-Forwarded-Encrypted: i=1; AJvYcCXvY2x6yNZzGios0NFnjBNFosvqdJpcmbcPAfT/imJXnAtfwxNCs6zHHVFl8Fp6H70pNnZL3UE+VojWvEJYoE3J46p2DxUNQwFrAddcCik=
X-Gm-Message-State: AOJu0Yx4UFWQDh5/FgZUmufJ7drEIX8PG00I9eOYNBcfJ0MbkMAXgo3l
	st+f+dXsFlU2XTcyMjZm53yRCVGvpGvmcTmjucTEWpXSajapddC7mvbe9jsy678=
X-Google-Smtp-Source: AGHT+IHzPCHWrqu/cwLI9d7GJ525lDaWEii2Qo/86hOu/9XSlgVo1ofx59fYWp2DCF2nBmqkdm+V3A==
X-Received: by 2002:ac8:5dd2:0:b0:43c:56c6:b489 with SMTP id d75a77b69052e-43fb0e89777mr159747171cf.59.1717001405867;
        Wed, 29 May 2024 09:50:05 -0700 (PDT)
Message-ID: <c0114094-c312-4d4f-a1a9-73b4432a9885@citrix.com>
Date: Wed, 29 May 2024 17:50:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] CI: Use a debug build of Xen for the Xilinx HW tests
To: Michal Orzel <michal.orzel@amd.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
 <20240529141945.41669-3-andrew.cooper3@citrix.com>
 <10676879-e0e4-4332-b8cb-732d56cda818@amd.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <10676879-e0e4-4332-b8cb-732d56cda818@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 4:30 pm, Michal Orzel wrote:
> Hi Andrew,
>
> On 29/05/2024 16:19, Andrew Cooper wrote:
>>
>> ... like the other hardware tests.  This gets more value out of the testing.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Stefano Stabellini <sstabellini@kernel.org>
>> CC: Michal Orzel <michal.orzel@amd.com>
>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>>  automation/gitlab-ci/test.yaml | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
>> index efd3ad46f08e..e96ccdfad54c 100644
>> --- a/automation/gitlab-ci/test.yaml
>> +++ b/automation/gitlab-ci/test.yaml
>> @@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
>>      - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
>>    needs:
>>      - *arm64-test-needs
>> -    - alpine-3.18-gcc-arm64
>> +    - alpine-3.18-gcc-debug-arm64
> This change should be reflected in the name of the test changed (here and below), so that it contains -debug suffix just like every other debug job.
> With that done:
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Wow, this is a giant mess.  There's no consistency anywhere.

I've got a debug suffix for this one, and a debug infix for the other
one, but I'm seriously tempted to sed over the whole config and clean
this up for once and for all.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 29 16:57:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 16:57:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732268.1138189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMcX-0003rW-6Q; Wed, 29 May 2024 16:57:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732268.1138189; Wed, 29 May 2024 16:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMcX-0003rP-3s; Wed, 29 May 2024 16:57:49 +0000
Received: by outflank-mailman (input) for mailman id 732268;
 Wed, 29 May 2024 16:57:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCMcW-0003rF-8L; Wed, 29 May 2024 16:57:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCMcW-0003MR-6l; Wed, 29 May 2024 16:57:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCMcV-00044u-Ve; Wed, 29 May 2024 16:57:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCMcV-0007Gl-V2; Wed, 29 May 2024 16:57:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cwFMxdMOYNlTO0O6NQg/hHKjt6Z7IKo5L8/Kxk8x8zo=; b=ahf2D3CERf50s4oIkf1z9dStkb
	/WpGOMeJnaofXGhrjGKYySHV8qkQ5OuP4tS7SbpyM1plasmeivQnh2SbF6CfIJfYAKIRMWclCZFan
	Vhf2bJZsjBngMUi8PiWzOZRwbUiEFaPYb7QskAJDPN/27C/uJOfwzwRS/+xYXst2kQbM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186182-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186182: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0840bc5ea114f536a4bdfb2ca095b79f2069aae6
X-Osstest-Versions-That:
    xen=2bc462f645824a439879190ed19a231cb5b8034c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 16:57:47 +0000

flight 186182 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186182/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  0840bc5ea114f536a4bdfb2ca095b79f2069aae6
baseline version:
 xen                  2bc462f645824a439879190ed19a231cb5b8034c

Last test of basis   186181  2024-05-29 08:00:24 Z    0 days
Testing same since   186182  2024-05-29 14:03:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2bc462f645..0840bc5ea1  0840bc5ea114f536a4bdfb2ca095b79f2069aae6 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed May 29 17:15:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 17:15:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732279.1138200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMtg-0007Ku-JS; Wed, 29 May 2024 17:15:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732279.1138200; Wed, 29 May 2024 17:15:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCMtg-0007Kn-Ga; Wed, 29 May 2024 17:15:32 +0000
Received: by outflank-mailman (input) for mailman id 732279;
 Wed, 29 May 2024 17:15:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCMtf-0007Kh-P0
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 17:15:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCMta-0003gg-Vh; Wed, 29 May 2024 17:15:26 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCMta-0006Qf-Nm; Wed, 29 May 2024 17:15:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=9vBy5Q27dCX+E0y7NfJbQrgKRcT+F44lMaupc9twbPc=; b=EbNOnVuEqNPTgoFop2xDPFUZgD
	iFMRO6Zykr9yByKc30DCd8vRvnpsxjBEMGf5g6tPxIof6qNtmqfFt+TP2B1KgRyZuj3CD6GIsIza1
	d9UGiGwfmScOEoY4th398J9hy/dVmEF/Zs4nl22JsPvV94lYbgU0lyviIVO/B4GgfEwA=;
Message-ID: <c53db088-d829-476d-b1cb-e8c8a1cdca63@xen.org>
Date: Wed, 29 May 2024 18:15:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v11 2/9] xen: introduce generic non-atomic test_*bit()
Content-Language: en-GB
To: "Oleksii K." <oleksii.kurochko@gmail.com>, Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1716547693.git.oleksii.kurochko@gmail.com>
 <79c3c31f0032a79c25d0a458b6091904457c8939.1716547693.git.oleksii.kurochko@gmail.com>
 <ab275ed4-29c3-4473-b1ee-2a9cda63eeaf@suse.com>
 <d8fd70469a1ac8d8cc291dddd0496f6bfabf6720.camel@gmail.com>
 <a39c3c03-ef54-4329-833d-03b60f162234@xen.org>
 <56b97916d1c36040a0be547759d5d10d311c9ed3.camel@gmail.com>
 <5438a9b1-d13a-415b-95e4-af520c228e01@suse.com>
 <c72784d5-20d2-4583-9e8a-f8b1cbf31aa6@xen.org>
 <ded13a36-b790-4989-a952-9a4130293b50@suse.com>
 <2bc05407cc62c829d63da757e071db51a003eb79.camel@gmail.com>
 <23790bd6-e06a-4adc-90a9-517f947be00a@suse.com>
 <1b301099b71c61c8e537ec6eb8fe064c3c2348f3.camel@gmail.com>
 <3c33c0cc45a8c3b21e5cf8addf2cd34ae017a648.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <3c33c0cc45a8c3b21e5cf8addf2cd34ae017a648.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 29/05/2024 17:36, Oleksii K. wrote:
> On Wed, 2024-05-29 at 18:29 +0200, Oleksii K. wrote:
>> On Wed, 2024-05-29 at 17:22 +0200, Jan Beulich wrote:
>>> On 29.05.2024 16:58, Oleksii K. wrote:
>>>> static always_inline bool test_bit(int nr, const volatile void
>>>> *addr)On
>>>> Wed, 2024-05-29 at 12:06 +0200, Jan Beulich wrote:
>>>>> On 29.05.2024 11:59, Julien Grall wrote:
>>>>>> I didn't realise this was an existing comment. I think the
>>>>>> suggestion is
>>>>>> a little bit odd because you could use the atomic version of
>>>>>> the
>>>>>> helper.
>>>>>>
>>>>>> Looking at Linux, the second sentence was dropped. But not
>>>>>> the
>>>>>> first
>>>>>> one. I would suggest to do the same. IOW keep:
>>>>>>
>>>>>> "
>>>>>> If two examples of this operation race, one can appear to
>>>>>> succeed
>>>>>> but
>>>>>> actually fail.
>>>>>> "
>>>>>
>>>>> As indicated, I'm okay with that being retained, but only in a
>>>>> form
>>>>> that
>>>>> actually makes sense. I've explained before (to Oleksii) what I
>>>>> consider
>>>>> wrong in this way of wording things.
>>>>
>>>> Would it be suitable to rephrase it in the following way:
>>>>       * This operation is non-atomic and can be reordered.
>>>>     - * If two examples of this operation race, one can appear to
>>>>     succeed
>>>>     - * but actually fail.  You must protect multiple accesses
>>>> with
>>>> a
>>>>     lock.
>>>>     + * If two instances of this operation race, one may succeed
>>>> in
>>>>     updating
>>>>     + * the bit in memory, but actually fail. It should be
>>>> protected
>>>>     from
>>>>     + * potentially racy behavior.
>>>>       */
>>>>      static always_inline bool
>>>>      __test_and_set_bit(int nr, volatile void *addr)
>>>
>>> You've lost the "appear to" ahead of "succeed". Yet even with the
>>> adjustment
>>> I still don't see what the "appear to succeed" really is supposed
>>> to
>>> mean
>>> here. The issue (imo) isn't with success or failure, but with the
>>> write of
>>> one CPU potentially being immediately overwritten by another CPU,
>>> not
>>> observing the bit change that the first CPU did. IOW both will
>>> succeed (or
>>> appear to succeed), but one update may end up being lost. Maybe
>>> "...,
>>> both
>>> may update memory with their view of the new value, not taking into
>>> account
>>> the update the respectively other one did"? Or "..., both will
>>> appear
>>> to
>>> succeed, but one of the updates may be lost"?
>> For me, the first one is clearer.

I actually find the second better because it explicitely spell out the 
issue. I can live with the first one though.

> If then this part of the comment is needed for test_bit() as it is
> doing only reading?

I don't think so. As Jan said, test_bit() is not a read-modify-write 
operation.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 29 19:07:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:07:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732291.1138210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOdd-0003xQ-AW; Wed, 29 May 2024 19:07:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732291.1138210; Wed, 29 May 2024 19:07:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOdd-0003xJ-7I; Wed, 29 May 2024 19:07:05 +0000
Received: by outflank-mailman (input) for mailman id 732291;
 Wed, 29 May 2024 19:07:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCOdb-0003vg-G1
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:07:03 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1a66bf2-1dee-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:07:02 +0200 (CEST)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-35507dfe221so30089f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:07:02 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35dbdb6f34fsm480917f8f.73.2024.05.29.12.07.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 12:07:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1a66bf2-1dee-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717009622; x=1717614422; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=f9GbTqm3z2RY4p8Voqy2YdjfaWsjsOlp7FSpcTeGhzk=;
        b=IXwbnFidMVGc7oCSuHuvyO5pAVra3P7uusXNyHrL9OWIadQLiXep0Z/unqhl0VnDTu
         xq89eI7/vda8JYQ1lJ60Er9PLZPa/e1endlEIVUXs85rL+Ktf6pTGAeuGamclREv42/r
         JPJ8WB0LuqvvcLkWwQDSlTxl1HCo96yKnjYLI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717009622; x=1717614422;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=f9GbTqm3z2RY4p8Voqy2YdjfaWsjsOlp7FSpcTeGhzk=;
        b=nqAWw/lGF9sQqFgLUNTf0wcJCrEqHkY0KRWbfy8nBmGshMoHyjZA+BImLSkcEblqMC
         vXU61VdwE6+6l/bG1auQU0OihJSu1QocVB39r5IZ6lM7z9bq9tldkQgA3RK0TpDosQTP
         bn5Qq5UnTk1PQxei0oJF/ltcntlcZM0RKtqoCAV5PIe9YWHdrSSV0Kc8By62BuudvYP/
         4ExC2ohoPcJcZr8avPrpRcrG2xCnpa54aeNzd3ioWpsEEaCFYo2C7PurT0KmREPGYH7h
         a8E47VNC1xKwbepdsDni1qSQRczUMmqXLapy/JvXmFqp4pCVij2T8gS3VHIL+g56GQ/A
         NWqg==
X-Forwarded-Encrypted: i=1; AJvYcCU62XGE2dD58dByNoARGoUVzTJKzUl7b6rBKkOvQjKAvwJlsj2qOnrghdZA2DKjUrYhC9YR4fj4o5prsQ2BbwNYZXBPKKzz9PceHnkvCaE=
X-Gm-Message-State: AOJu0YyebFBGp5uENzMtAGD6TgEbl9oBoF5n7hteWjjvar40db89CrR/
	NYiSHe9qB5V4oTsL2qmti7fb+gQkg2nv5x9k9YVhID4LiFjEkqLTdnPNgRwL0Tg=
X-Google-Smtp-Source: AGHT+IEC7p/6dhFINNLc1zsCCbJF4y0fd5IL9efqXOTp6aPIinADJAyVnBh76V1CZbDxmZKM3uwKOw==
X-Received: by 2002:a05:6000:1188:b0:354:fa5b:62ab with SMTP id ffacd0b85a97d-35dc00924e7mr100017f8f.18.1717009621717;
        Wed, 29 May 2024 12:07:01 -0700 (PDT)
Message-ID: <57432bd5-2799-4358-b78e-b294c9a7f9a6@citrix.com>
Date: Wed, 29 May 2024 20:07:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 04/13] xen/page_alloc: Coerce min(flsl(), foo)
 expressions to being unsigned
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-5-andrew.cooper3@citrix.com>
 <d94f3c36-66f1-4db0-bc57-477e48c61240@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d94f3c36-66f1-4db0-bc57-477e48c61240@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27/05/2024 7:26 am, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
>> This is in order to maintain bisectability through the subsequent changes,
>> where flsl() changes sign-ness non-atomically by architecture.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
>

Actually, by coercing the min() in pvh_populate_memory_range() from
patch 9 in this patch, the bisection-complexity of this series drops
massively, and in particular I can merge patch 3 into 10.

As you've given R-by on both, I'm going to go ahead and do this in order
to make some headway on the series, given the deadlines, and that the
RISC-V series is still pending this one.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 29 19:19:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:19:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732300.1138220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOpH-0005Yo-Ba; Wed, 29 May 2024 19:19:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732300.1138220; Wed, 29 May 2024 19:19:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOpH-0005Yh-86; Wed, 29 May 2024 19:19:07 +0000
Received: by outflank-mailman (input) for mailman id 732300;
 Wed, 29 May 2024 19:19:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCOpF-0005Y4-IJ
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:19:05 +0000
Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com
 [2607:f8b0:4864:20::f2e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4ff99782-1df0-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:19:05 +0200 (CEST)
Received: by mail-qv1-xf2e.google.com with SMTP id
 6a1803df08f44-6ad881cce98so324516d6.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:19:04 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad79daf087sm36998316d6.135.2024.05.29.12.19.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 12:19:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ff99782-1df0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717010344; x=1717615144; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=A451K2FZ5ONsAR56Fa33xMjJytr1GTI3tyRijCHx0ZA=;
        b=I9UmQYgJHfP/H2e1rgiHwQFH0zcDv3IRYTXj7PjKOGlONgYLIXFpAgqkblxeFa2FcB
         ZGOmUY7Jov4zNKKEoLzWMr7MlSKWoDMQEDSZ/5mRtFspV4rTeLAIGLFim6fPFNzYNl4f
         stvhF/4MgCoNtTaE+XtfmrFThJxemkCkOv8ik=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717010344; x=1717615144;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=A451K2FZ5ONsAR56Fa33xMjJytr1GTI3tyRijCHx0ZA=;
        b=M/BByvSZlKLzSOleHNf9rvHOBwaHGLHeDnCSORmQfNOeKLYXJ20dz3+fpqXOVOGGL+
         n32HoVfxhCPajOUo6th0O8mWohwWgAXKdnODycNqgq1xSlATW4Op9nSlyk+j5ZotOVw8
         QvQiA6QgNAj8P3nxx7XE8yJ5Kb3SzZiv4LVgrq+4wu8alLb+1+xilgIgRzXgP4xIr6Nr
         EDFGq2M+Fsy0IEsqEBd3fA2xEywN0qI0s5Q0qMI2oQJroR6zsrqRj+ofzV+ckudDoaIZ
         4gsFET+Q8q5uRKGdDeaDw4CEoRWcSdAYX7fzjxedVfYPipdAuC8C5kCJkkCVi8HCCZ+V
         2Lww==
X-Forwarded-Encrypted: i=1; AJvYcCU45T4jdgKXDxVzO3Fjo2R5stMDFcQ98PnQNDF4Dun6PSTjXfubaqCJLCkMNi8xV32JqvOKXHl8dXgwbHy9Tt8vM21E8hjWq6nQ3f/e86Y=
X-Gm-Message-State: AOJu0YwEUMyABlOkpGEsXvXS0RxzzwnNR94jZJ/VXPRgvmYOt2YRi1Oz
	RXIKZ0abC2fpAj/uwztEt1DTsbCAGEsJ+SXzNg7mawwp5nae59Z2twPPduFkdgc=
X-Google-Smtp-Source: AGHT+IGStWpjlx88wZTazUGbv+1bMxVCGPneySaqZNjuqAPvkRRc2PuRFRqeP4z2nP8nR6kV8eFKMw==
X-Received: by 2002:a05:6214:4285:b0:6ae:ab:f875 with SMTP id 6a1803df08f44-6ae0cb22733mr1283086d6.21.1717010343706;
        Wed, 29 May 2024 12:19:03 -0700 (PDT)
Message-ID: <52ff651b-ddc4-4537-a176-742b19e9dfca@citrix.com>
Date: Wed, 29 May 2024 20:19:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 04/13] xen/page_alloc: Coerce min(flsl(), foo)
 expressions to being unsigned
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-5-andrew.cooper3@citrix.com>
 <d94f3c36-66f1-4db0-bc57-477e48c61240@suse.com>
 <57432bd5-2799-4358-b78e-b294c9a7f9a6@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <57432bd5-2799-4358-b78e-b294c9a7f9a6@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:07 pm, Andrew Cooper wrote:
> On 27/05/2024 7:26 am, Jan Beulich wrote:
>> On 24.05.2024 22:03, Andrew Cooper wrote:
>>> This is in order to maintain bisectability through the subsequent changes,
>>> where flsl() changes sign-ness non-atomically by architecture.
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>
>>
> Actually, by coercing the min() in pvh_populate_memory_range() from
> patch 9 in this patch, the bisection-complexity of this series drops
> massively, and in particular I can merge patch 3 into 10.
>
> As you've given R-by on both, I'm going to go ahead and do this in order
> to make some headway on the series, given the deadlines, and that the
> RISC-V series is still pending this one.

Actually not quite.  It's even more simple.  This patch stays the same,
and the coercion gets added as find_first_set_bit() turns into ffsl().

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 29 19:20:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:20:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732304.1138231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOqR-00074I-PA; Wed, 29 May 2024 19:20:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732304.1138231; Wed, 29 May 2024 19:20:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCOqR-00074B-L3; Wed, 29 May 2024 19:20:19 +0000
Received: by outflank-mailman (input) for mailman id 732304;
 Wed, 29 May 2024 19:20:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xAhx=NA=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1sCOqR-000745-AD
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:20:19 +0000
Received: from mail-oo1-xc2e.google.com (mail-oo1-xc2e.google.com
 [2607:f8b0:4864:20::c2e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7b043fac-1df0-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 21:20:17 +0200 (CEST)
Received: by mail-oo1-xc2e.google.com with SMTP id
 006d021491bc7-5b9776123a3so22305eaf.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:20:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b043fac-1df0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717010416; x=1717615216; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nYEZ3QdPOBVtKLMGxphyZXjG3jXLrbQxZILjLBd0uT8=;
        b=cKFi4iq26jWEDOk4ZDVZEWDEGfOKFZ2yu0bK+ZYFI9F60QAyg0nWL6PMPode6w00rQ
         5NCy7mvvsTU0e6tO8H28siHblF1cd7BJ5q9lecUl4jQY+RPXAeL2YCnLM/81gY3w5piS
         P7G6e/IlN8EbRy7txN3Fh5gc2/ey3mKTopK5Udufx/1mj53JCmS6CGbWEtE5POcFFw9Z
         KeIP6iReqCyGgbO+v8yeDZf8AH/RmstMCB6nZ0i1Z3o6jrZRv575w0DGuMeuZkghmaiQ
         vGYJk/2pnmsd9dPleZqvn/MpIErC91fP0NPKERosQB8dOJAu0ESg4YUWFNcyKAimdUkp
         8YaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717010416; x=1717615216;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nYEZ3QdPOBVtKLMGxphyZXjG3jXLrbQxZILjLBd0uT8=;
        b=uVK3jrYS3WEjQMz9fPurjFmvga8B6WLS1WB7FSgvtnuIQ8gzpCZ/pd3qtMIG8d+MDb
         fMc8wqD3mt6S0bVmi4JFQw30KzTxSZdXQPHKKBk4K9genGUv79dI4icgojWWPTn01xId
         etgCrN4ljorBGTNGVQ3HOrKLpgngkJQC6zmRHdDWdjhUVKQYsqheoiUNDRAQbgf/6W81
         62ihlvDG+PKF1YB10HZAqw/lnJTFf5ZMTW5WzIvjMlekl/uU+gW+1qi1UGwU8Qq1dA0j
         VkKyeNgTKbcW0o5YOpjLZB4tdd9rnwnDc/ynBieA0bsgU3NDNbX0pFukCWydjdcsVHfx
         Az1g==
X-Forwarded-Encrypted: i=1; AJvYcCWYHkaJcm1YSoO5Hqduc3OihkznIiBAL6Pr1gk+cuu1LVZ68s63ahqvbm+SUYHsiMG6TxXwS88+seIOlSIxvB/Ctz/4rOWB+cMN9Kg+U/8=
X-Gm-Message-State: AOJu0YxTpvz5medRmyxL1Tk4mru8AjXkEwnO///69A35a0j035E178kT
	WYBoettvGMhWBrqIh1M3dyO0kkNWJomh+ggTiRdt07tiemhA1hOiKbQLXffZsJ04jb3+1gG3ND3
	B2+acwa2GqDvIWaV1RinXPvlkMcU=
X-Google-Smtp-Source: AGHT+IH7dzzBsRBUM6NjiNPEqPePk3LBlVotkKy36aTOkYG7k8Mdi271TA5C51uZ1qI7DV9hYpXkCeMN4l0lHFIg46E=
X-Received: by 2002:a05:6871:5210:b0:24f:dad3:97c with SMTP id
 586e51a60fabf-25060dd25aemr162328fac.46.1717010415890; Wed, 29 May 2024
 12:20:15 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1716029860.git.w1benny@gmail.com> <ba5b81fdaf174a236c3963fcfd29ae3b19aff13d.1716029860.git.w1benny@gmail.com>
 <21196eb3-82ca-470b-8dc9-1a95b427f3b9@suse.com> <CAKBKdXj7xUyuy-wPCjR0a+kNwqCmGXOoO=-pMQa7=Yiyaom_mg@mail.gmail.com>
 <12335f4d-adde-42d3-aa25-b662e701b96d@suse.com>
In-Reply-To: <12335f4d-adde-42d3-aa25-b662e701b96d@suse.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Wed, 29 May 2024 21:20:04 +0200
Message-ID: <CAKBKdXjuf3T1ozYFm3njU5pq0bwZnHjAoHjD0LsvcDnCRxdTBw@mail.gmail.com>
Subject: Re: [PATCH for-4.19? v4 4/6] x86: Make the maximum number of altp2m
 views configurable
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Tamas K Lengyel <tamas@tklengyel.com>, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 27, 2024 at 8:19=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
>
> This is suspicious: You compare against one value but log another. This
> isn't EPT-specific, so shouldn't use MAX_EPTP.

Sorry, I copy-pasted a snippet and didn't edit it correctly. Of
course, it should have been:

if ( config->nr_altp2m > MAX_NR_ALTP2M )
{
    dprintk(XENLOG_INFO, "nr_altp2m must be <=3D %lu\n", MAX_NR_ALTP2M);
    return -EINVAL;
}

> > ... should I be consistent and also replace these accesses with
> > altp2m_get_eptp/altp2m_get_p2m (which will internally use
> > array_index_nospec), or should I leave them as they are?
>
> Perhaps leave them as they are, unless you can technically justify the
> adjustment.

If we can avoid speculative execution, why just not do it? The
performance overhead array_index_nospec is negligible compared to the
whole VMEXIT handling. It will also serve as future-proofing, since
nobody will be confused whether they should directly access the array,
but instead use the accessor function.

Currently, the idea seems to be that array_index_nospec() is used when
the index is user-controlled, and not used when the index is under
xen's control (i.e. in loops). But I found at least 2 instances where
the index _is_ user controlled and the nospec access is not used -
further proving my previous point.

That being said, if there are no protests, I would replace all array
index accesses with the newly introduced accessor functions, which
will unconditionally use array_index_nospec().

P.


From xen-devel-bounces@lists.xenproject.org Wed May 29 19:35:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:35:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732312.1138239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCP4u-0000Qz-Ve; Wed, 29 May 2024 19:35:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732312.1138239; Wed, 29 May 2024 19:35:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCP4u-0000Qs-Sg; Wed, 29 May 2024 19:35:16 +0000
Received: by outflank-mailman (input) for mailman id 732312;
 Wed, 29 May 2024 19:35:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0akS=NA=raptorengineering.com=sanastasio@srs-se1.protection.inumbo.net>)
 id 1sCP4t-0000Qm-M1
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:35:15 +0000
Received: from raptorengineering.com (mail.raptorengineering.com
 [23.155.224.40]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 909ac36f-1df2-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:35:13 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 092A58287045;
 Wed, 29 May 2024 14:35:11 -0500 (CDT)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id WiR5U2434Tuw; Wed, 29 May 2024 14:35:10 -0500 (CDT)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id CDC43828718A;
 Wed, 29 May 2024 14:35:09 -0500 (CDT)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id u-IVwvgOBNcD; Wed, 29 May 2024 14:35:09 -0500 (CDT)
Received: from [10.11.0.2] (5.edge.rptsys.com [23.155.224.38])
 by mail.rptsys.com (Postfix) with ESMTPSA id C6B9F8287045;
 Wed, 29 May 2024 14:35:08 -0500 (CDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 909ac36f-1df2-11ef-90a1-e314d9c70b13
DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com CDC43828718A
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD;
	t=1717011309; bh=dwXIRi2j9fE09Pzdoge9/pLUywiTgX1ILFFSyE1O6Gc=;
	h=Message-ID:Date:MIME-Version:To:From;
	b=sxOxTfzuKi6BmoGGNt0fvda0Yucgiwi5lGbwWNMggm8D+uIPYcYdvP86udtWnBrnk
	 5b1tNqz/dkYRZjcLgVqSFmNK75RpqMGiK4nl8vDeU5ijbSVOB3B2bmrzWhmWdjgj2y
	 K48MZlgUCjwOzgvXVlTQ2jwc/z1DOpupAJvN2QsI=
X-Virus-Scanned: amavisd-new at rptsys.com
Message-ID: <2e845397-5d0f-4905-baef-cbf4402abd55@raptorengineering.com>
Date: Wed, 29 May 2024 14:35:08 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 01/13] ppc/boot: Run constructors on boot
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-2-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Shawn Anastasio <sanastasio@raptorengineering.com>
In-Reply-To: <20240524200338.1232391-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi Andrew,

On 5/24/24 3:03 PM, Andrew Cooper wrote:
> PPC collects constructors, but doesn't run them yet.  Do so.
> 
> They'll shortly be used to confirm correct behaviour of the bitops primitives.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---

Looks good to me.

Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

Thanks,
Shawn



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732321.1138269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOI-0003r2-Vh; Wed, 29 May 2024 19:55:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732321.1138269; Wed, 29 May 2024 19:55:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOI-0003qv-So; Wed, 29 May 2024 19:55:18 +0000
Received: by outflank-mailman (input) for mailman id 732321;
 Wed, 29 May 2024 19:55:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOH-0003Nf-1G
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:17 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5e3be7a4-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:15 +0200 (CEST)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-52b7b829bc7so171602e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:15 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e3be7a4-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012515; x=1717617315; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YVBXN5m3ZDHo1lfuHzyltxfOnJFU0pSyHgrlkIAnckU=;
        b=VkJ8aBeqhPAZRb9xwykDdEuUB4rbf6kjOkRcirCWCsp4BZ3ghATBXhqvnH90nca+g5
         DifOsL/aYoIeD6+NV1sANXb/Fj73BWJSe49uYJCAMzEMzuPhUZ48opNbQn1CA+X599iL
         5qbgl06f8/Rj6TJICFvgb7QLZBhjaUf7ca9sdxjaFU6T3okhdfVv3JPEphJRVzD1Bmnc
         EVVioO+rkbVoQwagIgrNMFJPC4KcS3+EbeN9XhDVTX6gl7VxvkDm1UwsI4Swm9DOLDcn
         +uLZRLtedlCSWBTqSAIwp8jeN/XwrunPwIHj5gqq++zQmhUDWKE7Vu+v1bN+3j7P9HO8
         3E5g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012515; x=1717617315;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YVBXN5m3ZDHo1lfuHzyltxfOnJFU0pSyHgrlkIAnckU=;
        b=AJKtaYlN7J0Gxsil+NxQ3zA01V+mv5FQHhKRLdmM70Imo7J5oPr3CzYfvtOQIVnd0i
         lWY+MEZgMQLeAArLTaoOgCyT3QtqKWZ4zTyCVCYejg7VEpBIZCR8qFWbrfyeD28pQDxy
         kQAgamQCCkntdW1LtGybbPBkYGompWfTAfw0aeTAta+MG4lAkw4kb8kWQri6OglCmSLh
         UdrEF6bCCK+IhMDjBzxazD784VF5kcTbOU0Crj1CcNbT3omjpACU8O8Z9xkwb4ovH78n
         8NiMFPEdUAHNGdO4t+jHj1FGkPRmDwyEADPzKWfBu/hct+OQRY0vlFbPDXugvfOG0+h3
         Qi3Q==
X-Gm-Message-State: AOJu0YxStiB55mIq0s7IyCIe/X2I7HutEBybS+fXZuMvEa215zUtt8eo
	3O7b4J8e+K9ttBitWcbJldiTLTUuMIubMCX+kqAzHYKYL6T64Po7v2sXq1qB
X-Google-Smtp-Source: AGHT+IGxVp6idhC7Wfi+drTwdCPMybv1eYUhKHQinDOkohJRTyyuhIhTpio2EZiatCVKYOV4qkkf7w==
X-Received: by 2002:ac2:4839:0:b0:529:b3c9:7261 with SMTP id 2adb3069b0e04-52b7d40f4c4mr106781e87.5.1717012514495;
        Wed, 29 May 2024 12:55:14 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v12 2/8] xen: introduce generic non-atomic test_*bit()
Date: Wed, 29 May 2024 21:55:03 +0200
Message-ID: <526d2a5a76f03aa0e3cc7ee3192b1c87834f0e9e.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

The following approach was chosen for generic*() and arch*() bit
operation functions:
If the bit operation function that is going to be generic starts
with the prefix "__", then the corresponding generic/arch function
will also contain the "__" prefix. For example:
 * test_bit() will be defined using arch_test_bit() and
   generic_test_bit().
 * __test_and_set_bit() will be defined using
   arch__test_and_set_bit() and generic__test_and_set_bit().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V12:
 - revert change of moving the definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h.
   ( a separate patch will be provided to put BITS_PER_BYTE to proper place )
 - drop comments on top of generic_*() functions and update the comments above __test_*() and test_bit().
 - update how static inline __test_*() are defined ( drop pointless fallback #define ) and test_bit().
 - drop the footer after Signed-off-by.
---
Changes in V11:
 - fix identation in generic_test_bit() function.
 - move definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h
 - drop the changes in arm64/livepatch.c.
 - update the the comments on top of functions: generic__test_and_set_bit(), generic__test_and_clear_bit(),  generic__test_and_change_bit(),
   generic_test_bit().
 - Update footer after Signed-off section.
 - Rebase the patch on top of staging branch, so it can be merged when necessary approves will be given.
 - ? add Reviewed-by: Jan Beulich <jbeulich@suse.com>.
---
Changes in V10:
 - update the commit message. ( re-order paragraphs and add explanation usage of prefix "__" in bit
   operation function names )
 - add  parentheses around the whole expression of bitop_bad_size() macros.
 - move macros bitop_bad_size() above asm/bitops.h as it is not arch-specific anymore and there is
   no need for overriding it.
 - drop macros check_bitop_size() and use "if ( bitop_bad_size(addr) ) __bitop_bad_size();" implictly
   where it is needed.
 - in <xen/bitops.h> use 'int' as a first parameter for __test_and_*(), generic__test_and_*() to be
   consistent with how the mentioned functions were declared in the original per-arch functions.
 - add 'const' to p variable in generic_test_bit().
 - move definition of BITOP_BITS_PER_WORD and bitop_uint_t to xen/bitops.h as we don't allow for arch
   overrides these definitions anymore.
---
Changes in V9:
  - move up xen/bitops.h in ppc/asm/page.h.
  - update defintion of arch_check_bitop_size.
    And drop correspondent macros from x86/asm/bitops.h
  - drop parentheses in generic__test_and_set_bit() for definition of
    local variable p.
  - fix indentation inside #ifndef BITOP_TYPE...#endif
  - update the commit message.
---
 Changes in V8:
  - drop __pure for function which uses volatile.
  - drop unnessary () in generic__test_and_change_bit() for addr casting.
  - update prototype of generic_test_bit() and test_bit(): now it returns bool
    instead of int.
  - update generic_test_bit() to use BITOP_MASK().
  - Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
  - add a footer with explanation of dependency on an uncommitted patch after Signed-off.
  - abstract bitop_size().
  - move BITOP_TYPE define to <xen/types.h>.
---
 Changes in V7:
  - move everything to xen/bitops.h to follow the same approach for all generic
    bit ops.
  - put together BITOP_BITS_PER_WORD and bitops_uint_t.
  - make BITOP_MASK more generic.
  - drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
    enough.
  - drop "_" for generic__{test_and_set_bit,...}().
  - drop " != 0" for functions which return bool.
  - add volatile during the cast for generic__{...}().
  - update the commit message.
  - update arch related code to follow the proposed generic approach.
---
 Changes in V6:
  - Nothing changed ( only rebase )
---
 Changes in V5:
   - new patch
---
 xen/arch/arm/include/asm/bitops.h |  67 --------------
 xen/arch/ppc/include/asm/bitops.h |  52 -----------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  31 ++-----
 xen/include/xen/bitops.h          | 146 ++++++++++++++++++++++++++++++
 6 files changed, 157 insertions(+), 143 deletions(-)

diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 8f4bdc09d1..3c023103f7 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -22,9 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 #define ADDR (*(volatile int *) addr)
@@ -76,70 +73,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 #define arch_fls(x)  ((x) ? 32 - __builtin_clz(x) : 0)
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 8119b5ace8..eb3355812e 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -15,9 +15,6 @@
 #define __set_bit(n, p)         set_bit(n, p)
 #define __clear_bit(n, p)       clear_bit(n, p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 /* PPC bit number conversion */
@@ -69,17 +66,6 @@ static inline void clear_bit(int nr, volatile void *addr)
     clear_bits(BITOP_MASK(nr), (volatile unsigned int *)addr + BITOP_WORD(nr));
 }
 
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-    const volatile unsigned int *p = addr;
-    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
-}
-
 static inline unsigned int test_and_clear_bits(
     unsigned int mask,
     volatile unsigned int *p)
@@ -133,44 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old | mask;
-    return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old & ~mask;
-    return (old & mask) != 0;
-}
-
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 #define arch_fls(x)  ((x) ? 32 - __builtin_clz(x) : 0)
diff --git a/xen/arch/ppc/include/asm/page.h b/xen/arch/ppc/include/asm/page.h
index 890e285051..6d4cd2611c 100644
--- a/xen/arch/ppc/include/asm/page.h
+++ b/xen/arch/ppc/include/asm/page.h
@@ -2,9 +2,9 @@
 #ifndef _ASM_PPC_PAGE_H
 #define _ASM_PPC_PAGE_H
 
+#include <xen/bitops.h>
 #include <xen/types.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
diff --git a/xen/arch/ppc/mm-radix.c b/xen/arch/ppc/mm-radix.c
index ab5a10695c..9055730997 100644
--- a/xen/arch/ppc/mm-radix.c
+++ b/xen/arch/ppc/mm-radix.c
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index fc9fe73ad5..8ee5cbf127 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -19,9 +19,6 @@
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
-extern void __bitop_bad_size(void);
-#define bitop_bad_size(addr) (sizeof(*(addr)) < 4)
-
 /**
  * set_bit - Atomically set a bit in memory
  * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 6a5e28730a..be54122556 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -4,6 +4,17 @@
 #include <xen/compiler.h>
 #include <xen/types.h>
 
+#define BITOP_BITS_PER_WORD 32
+typedef uint32_t bitop_uint_t;
+
+#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
+
+#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
+
+extern void __bitop_bad_size(void);
+
+#define bitop_bad_size(addr) (sizeof(*(addr)) < sizeof(bitop_uint_t))
+
 #include <asm/bitops.h>
 
 /*
@@ -94,6 +105,141 @@ static always_inline __pure unsigned int fls64(uint64_t x)
 
 /* --------------------- Please tidy below here --------------------- */
 
+static always_inline bool
+generic__test_and_set_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask);
+}
+
+static always_inline bool
+generic__test_and_clear_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask);
+}
+
+static always_inline bool
+generic__test_and_change_bit(int nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask);
+}
+
+static always_inline bool generic_test_bit(int nr, const volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    const volatile bitop_uint_t *p =
+        (const volatile bitop_uint_t *)addr + BITOP_WORD(nr);
+
+    return (*p & mask);
+}
+
+/**
+ * __test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two instances of this operation race, both may update memory with
+ * their view of the new value, not taking into account the update the
+ * respectively other one did. It should be protected from potentially
+ * racy behavior.
+ */
+static always_inline bool
+__test_and_set_bit(int nr, volatile void *addr)
+{
+#ifdef arch__test_and_set_bit
+    return arch__test_and_set_bit(nr, addr);
+#else
+    return generic__test_and_set_bit(nr, addr);
+#endif
+}
+#define __test_and_set_bit(nr, addr) ({             \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_set_bit(nr, addr);                   \
+})
+
+/**
+ * __test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two instances of this operation race, both may update memory with
+ * their view of the new value, not taking into account the update the
+ * respectively other one did. It should be protected from potentially
+ * racy behavior.
+ */
+static always_inline bool
+__test_and_clear_bit(int nr, volatile void *addr)
+{
+#ifdef arch__test_and_clear_bit
+    return arch__test_and_clear_bit(nr, addr);
+#else
+    return generic__test_and_clear_bit(nr, addr);
+#endif
+}
+#define __test_and_clear_bit(nr, addr) ({           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_clear_bit(nr, addr);                 \
+})
+
+/**
+ * __test_and_change_bit - Change a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two instances of this operation race, both may update memory with
+ * their view of the new value, not taking into account the update the
+ * respectively other one did. It should be protected from potentially
+ * racy behavior.
+ */
+static always_inline bool
+__test_and_change_bit(int nr, volatile void *addr)
+{
+#ifdef arch__test_and_change_bit
+    return arch__test_and_change_bit(nr, addr);
+#else
+    return generic__test_and_change_bit(nr, addr);
+#endif
+}
+#define __test_and_change_bit(nr, addr) ({              \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    __test_and_change_bit(nr, addr);                    \
+})
+
+/**
+ * test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static always_inline bool test_bit(int nr, const volatile void *addr)
+{
+#ifdef arch_test_bit
+    return arch_test_bit(nr, addr);
+#else
+    return generic_test_bit(nr, addr);
+#endif
+}
+#define test_bit(nr, addr) ({                           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    test_bit(nr, addr);                                 \
+})
+
 #ifndef find_next_bit
 /**
  * find_next_bit - find the next set bit in a memory region
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732319.1138250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOG-0003Ns-CC; Wed, 29 May 2024 19:55:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732319.1138250; Wed, 29 May 2024 19:55:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOG-0003Nl-9D; Wed, 29 May 2024 19:55:16 +0000
Received: by outflank-mailman (input) for mailman id 732319;
 Wed, 29 May 2024 19:55:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOF-0003Nf-4S
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:15 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5d3c7546-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:14 +0200 (CEST)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-529614b8c29so216246e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:14 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d3c7546-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012513; x=1717617313; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=pAzpqmChn/xM5jNHeWdfIx1rNGhRNDTsnTQUp2N3w0c=;
        b=I2t0OkCHFl8rSoZPn8ZRHNqYHKWWp2EIQv2LyQ13V393SpKpOyzoHOMbhcW4p8amAR
         KWq2x7UNpPNQ7Os/cHlUT1MMhhWH/sGDDDpfD4yNMc5+i5FvJm7YLNIQLsGwBOHoN+8M
         ChA8Bo+uOVZJ0fu07pAoyUu/fZr74UjZLQiFMSqJ0ksfoME1WsXZG9438RJiCKdDrGTw
         HjOK4VBeENpsz7iBkeMowiyTOFaoATT2r9qCc+ElMlcIXgUWuVubgQcXNJfU3NuOE9kW
         SS1XCD0lxbZ4bO5ia8O0x71OqkSEq7VA420/kp1Z0SvMPsJVm8OsiipAtFrmoo4mhzBJ
         2dxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012513; x=1717617313;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pAzpqmChn/xM5jNHeWdfIx1rNGhRNDTsnTQUp2N3w0c=;
        b=Zj3tXmioLBG6m/HhQQEy9lYPfZMh/U6eJxFvHpi6TtVuMDKNpCigbRx/4jh8cXRD6W
         DFs7PO8EmrU6iGFD6uAkfDpTV+jGJbmRV3zUEtmuiRd+pj3qfVbXWgdG0LkaniBkvBt6
         T0mJJgiW6DDh/vU8tRjd/WuPbhGJOt7vWAE6QkJJ2ceCHt1t87YrPrhzd0VDGlEVjV90
         ezS4r1kFhCIMdmF+ppbIZBFpEXjtrCfOsPMDaS2V9VfQ8R2iNXv6XH9FymC4gV8bC0LN
         Q9ZHyzyRu1epCK4wMTs9li9h8q5YU77SOzOFkwAIgR9nAeq3FAMZe82BhwGSFLAWsS6D
         XRsA==
X-Gm-Message-State: AOJu0YzSE4GDOCaLNLzUsbsGIDhcvcg2HuKzpESyh2HG0MdbICLSPxS5
	Mj3AaqKZnSaAOXggKwtSvafnQw7SOLcjnCMdqnmt19asuI8Y9kxvsgnVjBht
X-Google-Smtp-Source: AGHT+IEhwUhr5SI6KtlUMzXiIkwF4VeWu3DSGpjuardRodBn6EAmuWCRKQgr6QpcNaG3tPdKcFtcjA==
X-Received: by 2002:ac2:42c7:0:b0:51d:d630:365c with SMTP id 2adb3069b0e04-52b7d419a32mr132874e87.4.1717012512944;
        Wed, 29 May 2024 12:55:12 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH for-4.19? v12 0/8] *Enable build of full Xen for RISC-V
Date: Wed, 29 May 2024 21:55:01 +0200
Message-ID: <cover.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH v2 for-4.19 00/13] xen/bitops: Untangle ffs()/fls() infrastructure [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.
As an option, it can be considered to merge arch-specific patch and then just
rebase [2] and drop arch-specific changes.

The link to the branch with rebased patches on top of [1] could be found here:
  https://gitlab.com/xen-project/people/olkur/xen/-/commits/riscv-full-xen-build-v12

[1] https://lore.kernel.org/xen-devel/20240524200338.1232391-1-andrew.cooper3@citrix.com/
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

***
I think that I need to write a separate email requesting approval for this patch
series to be included in Xen 4.19. Most of the patches are RISC-V specific,
so there is a low risk of breaking anything else.
There is only one patch that touches common or arch-specific files, and it
doesn't introduce any functional changes.
Since I can't approve it myself, I am asking for someone else to do so.
***

---
Changes in V12:
 - Rebase the patch series on top of [1] mentioned above.
 - Update the cover letter message.
 - "[PATCH v11 3/9] xen/bitops: implement fls{l}() in common logic" was droped
   as it is a part of patch series [1] mentioned above.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V11:
  - Patch was merged to staging:
    - [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
	  - [PATCH v10 06/14] xen/riscv: introduce atomic.h
	  - [PATCH v10 07/14] xen/riscv: introduce monitor.h
	  - [PATCH v10 09/14] xen/riscv: add required things to current.h
	  - [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V10:
  - Patch was merged to staging:
    - [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V9:
 - Patch was merged to staging:
    - [PATCH v8 07/17] xen/riscv: introduce io.h
  	- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V8:
 - Patch was merged to staging:
    - [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
    - [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
 - Update the commit message:
     - drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
     - Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (8):
  xen/riscv: disable unnecessary configs
  xen: introduce generic non-atomic test_*bit()
  xen/riscv: introduce bitops.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                  |   4 +
 automation/gitlab-ci/build.yaml         |   4 +
 xen/arch/arm/include/asm/bitops.h       |  67 ----
 xen/arch/ppc/include/asm/bitops.h       |  52 ---
 xen/arch/ppc/include/asm/page.h         |   2 +-
 xen/arch/ppc/mm-radix.c                 |   2 +-
 xen/arch/riscv/Makefile                 |  17 +-
 xen/arch/riscv/arch.mk                  |   4 -
 xen/arch/riscv/configs/tiny64_defconfig |  12 +-
 xen/arch/riscv/early_printk.c           | 168 ----------
 xen/arch/riscv/include/asm/bitops.h     | 137 ++++++++
 xen/arch/riscv/include/asm/cache.h      |   2 +
 xen/arch/riscv/include/asm/domain.h     |   2 +
 xen/arch/riscv/include/asm/mm.h         | 240 ++++++++++++++
 xen/arch/riscv/include/asm/p2m.h        |   2 +
 xen/arch/riscv/mm.c                     |  52 ++-
 xen/arch/riscv/setup.c                  |  10 +-
 xen/arch/riscv/stubs.c                  | 415 ++++++++++++++++++++++++
 xen/arch/riscv/traps.c                  |  25 ++
 xen/arch/x86/include/asm/bitops.h       |  31 +-
 xen/include/xen/bitops.h                | 146 +++++++++
 21 files changed, 1069 insertions(+), 325 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/stubs.c

-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732322.1138274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOJ-0003tX-9V; Wed, 29 May 2024 19:55:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732322.1138274; Wed, 29 May 2024 19:55:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOJ-0003sM-42; Wed, 29 May 2024 19:55:19 +0000
Received: by outflank-mailman (input) for mailman id 732322;
 Wed, 29 May 2024 19:55:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOH-0003Nf-DS
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:17 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ef92ff7-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:16 +0200 (CEST)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-529b79609cbso173149e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:16 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ef92ff7-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012516; x=1717617316; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=h3OBfOym5W7zP/NOzLdy/wTYIWbuyzij9mGpUhq8jn8=;
        b=FXOMKagBdb7WBn+WKoaoOCl3NTDtgwaYgcP/t2jk/vaIqJx0mgssV5tsMsOHSNj5+r
         6hKjE4djiOUkCVb3ZcL8/K9vdZWtZAKmmQTEz6DpjNusjCvAoerw61Hbl19ytQRGNvRQ
         nOsyzWt4KA4feEQf3W9Sze3v5v3CBVjSRkcZAKDxDoJ6iOQB9et0Rq+v4ZS/EkSKZl3l
         c5Vgbo9+8GVVVejgGqxBKShA4c4tM2AbhOQZfMbdrlxStcZ5+S0NLEPFv570TY5EsSvs
         v5GkEDgPnnLl7yItr6Ds7JshotbqkN1UBSnLncPh3dyj02qoN9+xZgWWAUqZ5vBqP73Q
         XLiQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012516; x=1717617316;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=h3OBfOym5W7zP/NOzLdy/wTYIWbuyzij9mGpUhq8jn8=;
        b=Z0ZZdRN50W+xs5/y8py/Meh0yfcvs7H7lRSKgFcd3W8YI60klAbvksdXjixZLQsFKs
         uKu6kvPMQz/aVwIH55OnhL8MIXWU0a20Npj6cwyNjjlMOORS518k8A0WtRRyHSrnubbM
         Qp52pqAO3NVpj/hIL9/7ol//wpi/PEqkT66KZblClUfFoa8RbDsI9qF7Tw8i+rI2ivS0
         U+4TdeniShfSZ6/ykV7qXcZJoKZC+wKB4anS0Fmhf8CkS1ssCJZiuxd1KbmCG4jDCNjZ
         H3Gyo3ZfUUvmEGYnbwRm3rjlKGPDh6QyM8cU++g7Cc9N0sBFEicffXRajNspoNpQpo++
         euiw==
X-Gm-Message-State: AOJu0YwyyDeKXVgxnsL+mIxkcybcCjhfnAomUcKetK+FMCQLNtjo+m1x
	9vhcIoq3a/F6qQbN1bDudeXPYdnJkfVdmJgrZxnFgww/4IDqSgHA9Da/NmkP
X-Google-Smtp-Source: AGHT+IF8TRHBMLnG7vbLddBJSm+lXJuXkNyuIx+55aDyvR4lh964SVdEQEN1swQiiL5CcERQAR3igQ==
X-Received: by 2002:ac2:5f08:0:b0:52b:797f:b21f with SMTP id 2adb3069b0e04-52b7d47a923mr117416e87.51.1717012516270;
        Wed, 29 May 2024 12:55:16 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 4/8] xen/riscv: add definition of __read_mostly
Date: Wed, 29 May 2024 21:55:05 +0200
Message-ID: <b93a0eab949f49d2977a3358b6c6c940b8028f4d.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

Considering that there is still no still final decision regarding patch [2] my suggestion
is to merge RISC-V specific patch and just drop the changes in patch [2].

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V9-V12:
 - Only rebase was done.
---
Change in V8:
 - update the footer after Signed-off.
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732323.1138282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOJ-000409-Om; Wed, 29 May 2024 19:55:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732323.1138282; Wed, 29 May 2024 19:55:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOJ-0003yY-Fp; Wed, 29 May 2024 19:55:19 +0000
Received: by outflank-mailman (input) for mailman id 732323;
 Wed, 29 May 2024 19:55:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOH-0003Q0-PQ
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:17 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5e624e07-1df5-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 21:55:16 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-52ac99930c6so270039e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:16 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e624e07-1df5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012515; x=1717617315; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5j/RmSAlhCXJU1vUCQMT/ujGfnXfs53M7nbtzjes1TI=;
        b=fEUhk5AA8ukCLPi7XQ2/NiqZL+ep3PgY6SLKzfRFoN1Ujeyophk2ftajYp5oyr/FUB
         KaUQsk6i2vEWk2Q1CJ2J6jx/V/ESUXfVEAav7eSpNFaNKKVgvpfNrVKZGwMRzh6OPWE3
         VRFUQasY09XWkdfdq0iC78BTWlZDNAGdNf5m2Oy+WQryFb5IosO/xkHu6M2LoDRvycQY
         FfJ2CQQOM1UM/v1iDd54C1/dRBeq87X0Jqz02yLF+xTwCC7DWsZYSwZ5D2mQGGH6MCLV
         YSxNjBAH9Zq4IZrIt+Ctlo98jUiAMRvoCu/VtPO6KfaQCzDOHS/yejHr9xoNe1boBpqp
         Ljtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012515; x=1717617315;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5j/RmSAlhCXJU1vUCQMT/ujGfnXfs53M7nbtzjes1TI=;
        b=kBQxKH8TZKI8yuefj7eIHjF4MpnOqi6tCj1X8TNLZx/jSDoAEXpT0BZm2JYrVC35gL
         ZtXG9SNSovJHjgDT0TsLRVX+0HlTo5CNm3rwPfnsmLGtbsD/9ZAVCF3eu4vzvtz0TW5m
         F85LMKBHvGjgWo0YVvyXL59+2mhdcwX5PrDSuRWel105srLzPBBfQiEDUFbQpAGPYd/l
         03b8YbK8Edb0xyHXoufed4oQnzZ319fePGDep50clSiZV/ZDQ64W+dg/bp+M5/0SI+2z
         kbn1AG9wGEqcXp4KO46IN8vN3g1BpXpiMqVXSsUBDRy3Vt0VYXhOoTrSA8wkNs5LER75
         WFEg==
X-Gm-Message-State: AOJu0Yz7gmJnk4ougoN37KtlgI7JV9B9v7Do2vKKUVcBxree66niXr0V
	fYPRJt0MM2JMGZYUBg69Mfjpdn1BWZfrGQN0ALiA1yJgZxX3c+CRbiJNqJNh
X-Google-Smtp-Source: AGHT+IHsKzwdLJDunzsII+Oo8Y+RSm0pjg/FSCmVV8H2Jruqwk6TXnOHVvqxpsKkabpyMwtmcSKuRg==
X-Received: by 2002:a19:ca06:0:b0:518:dae6:d0ec with SMTP id 2adb3069b0e04-52b7d418d7amr115370e87.4.1717012515204;
        Wed, 29 May 2024 12:55:15 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 3/8] xen/riscv: introduce bitops.h
Date: Wed, 29 May 2024 21:55:04 +0200
Message-ID: <b70eda27ec40fff2fdd1225f8b4f17c036b550b7.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V11-V12:
 - Nothing changed. Only rebase was done.
---
Changes in V10:
 - update the error message BITS_PER_LONG -> BITOP_BITS_PER_WORD
---
Changes in V9:
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects
   to work with 32-bit quantities.
 - s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
 - define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of
   "introduce generic non-atomic test_*bit()".
 - drop duplicated __set_bit() and __clear_bit().
 - drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
 - update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
 - drop 1 padding blank in test_and_op_bit_ord().
 - update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
   change return type to bool.
 - change addr argument type of test_and_change_bit(): unsigned long * -> void *.
 - move test_and_change_bit() closer to other test_and-s function.
 - Code style fixes: tabs -> space.
 - s/#undef __op_bit/#undef op_bit.
 - update the commit message: delete information about generic-non-atomic.h changes as now
   it is a separate patch.
---
Changes in V7:
 - Update the commit message.
 - Drop "__" for __op_bit and __op_bit_ord as they are atomic.
 - add comment above __set_bit and __clear_bit about why they are defined as atomic.
 - align bitops_uint_t with __AMO().
 - make changes after  generic non-atomic test_*bit() were changed.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
 - rebase clean ups were done: drop unused asm-generic includes
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..7f7af3fda1
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline bool test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline bool test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline bool test_and_change_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732320.1138259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOH-0003cF-Kc; Wed, 29 May 2024 19:55:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732320.1138259; Wed, 29 May 2024 19:55:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOH-0003c8-HT; Wed, 29 May 2024 19:55:17 +0000
Received: by outflank-mailman (input) for mailman id 732320;
 Wed, 29 May 2024 19:55:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOG-0003Q0-Pp
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:16 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5da20ca6-1df5-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 21:55:14 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-52ac0c4b62cso164194e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:14 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5da20ca6-1df5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012514; x=1717617314; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aXNTVAvwjq58BlgUtIZuszRpR8ogKV+d9WBU1hktqSI=;
        b=irzYxgrAplbrCWbeVGSGntb5IqYNxAKcb620gIwu44wZIubkESataaJJM3oi1tCxiW
         46tT0H0AyHSr5q1SMfJjLYwomp3cwoXnuUX14A//Wdi3BlTSHj3tXbBAdw8vH1I3x2sb
         XXnRV3SrP4yTBmv/SqBgX1TmVYJCF4vtO3foxk3ppH1gF3ZIh4Pv9/CmIctuY3scPrq6
         gBkzj5CHjDODV/mTTyQaSfOnl7ZndyvWMCsO6D6qqvoSZ94i6/4IXn1kQongjfzEQQNy
         jC+bm3MSU6hqfdGpZnQJ+zE+BjHevG0BnDO7FQWD9Tc84hPiiG74LqLUeCxD03EP5gV0
         G/vw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012514; x=1717617314;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=aXNTVAvwjq58BlgUtIZuszRpR8ogKV+d9WBU1hktqSI=;
        b=rFrkn1DUv9OWs47FUUKLcrimtloTtYoYtQcgpzAuXlCi7grLGUWc5iLiJ7x6535BGG
         3W4OfcIbVfTP9ejjAV8YeszC3q821FDYDNW3XqyS043ObrpXffCRDRvBVGjbAeNeXrBt
         XcrF1RI/AJU3BP9pWbgjg2jMpy3y7njjDB8SwaUvyKZ+IMzAB0dH9aWflIIBvBBJ30dL
         yYIh2uuF9V1F6vxCGHycYaJL3B8PoJ6IAoo8UZBaJ8nMG+9mxpAZqPy2Bjv5QFOVdy+7
         QsARQ4BIHxs+hJHZz/3hWPMZ57ntPUy8SmLbm32V40U78lRRGHlgCDCrQskK9Rkq/aDk
         32rg==
X-Gm-Message-State: AOJu0Yy/1k4kViE8gz7IITqU/FWR84cCVEZ6NAAGyzRZH4PxhLddswJR
	qG3Fy452t9quyUnXcFwiiXUcI3ttzn86yqaRYtOHZRlsD1p30WWbi2+7sZJH
X-Google-Smtp-Source: AGHT+IFFRNU83IuW9JRf9yB0tc8xEPXk5YxlD+5CPw8HDXR54dZcgidi8PHSzdNDVUTsmrweHZsZxw==
X-Received: by 2002:a19:8c50:0:b0:51d:d78:4952 with SMTP id 2adb3069b0e04-52b7d49058cmr93764e87.65.1717012513693;
        Wed, 29 May 2024 12:55:13 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
Date: Wed, 29 May 2024 21:55:02 +0200
Message-ID: <35cf9d52e538aab964a3ecc050260abb3f27c60d.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Only configs which lead to compilation issues were disabled.

Remove lines related to disablement of configs which aren't affected
compilation:
 -# CONFIG_SCHED_CREDIT is not set
 -# CONFIG_SCHED_RTDS is not set
 -# CONFIG_SCHED_NULL is not set
 -# CONFIG_SCHED_ARINC653 is not set
 -# CONFIG_TRACEBUFFER is not set
 -# CONFIG_HYPFS is not set
 -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set

To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
check_get_page_from_gfn() from xen/p2m-common.h.

Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was
included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
aren't declared started to occur.

CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.

CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
acpi_disabled, so it is disabled for now.

PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
also disabled for now, as RISC-V hasn't introduced this headers yet.

LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V10-V12:
 - Nothing changed. Only rebase.
---
Changes in V9:
 - update the commit message: add info about LIVEPATCH and PERF_COUNTERS.
---
Changes in V8:
 - disabled CPU_BOOT_TIME_CPUPOOLS as it requires an introduction of cpu_physical_id() and acpi_disabled.
 - leave XSM disabled, add explanation in the commit message.
 - drop HYPFS as the patch was provided to resolve compilation issue when this condif is enabled for RISC-V.
 - include asm/p2m.h to asm/domain.h, and xen/errno.h to asm/p2m.h to drop ARGO config from
   tiny64_defconfing and build.yaml.
 - update the commit message.
---
Changes in V7:
 - Disable only configs which cause compilation issues.
 - Update the commit message.
---
Changes in V6:
 - Nothing changed. Only rebase.
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         |  4 ++++
 xen/arch/riscv/configs/tiny64_defconfig | 12 +++++-------
 xen/arch/riscv/include/asm/domain.h     |  2 ++
 xen/arch/riscv/include/asm/p2m.h        |  2 ++
 4 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 5985be9378..3290a36dca 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -379,10 +379,14 @@ alpine-3.18-gcc-debug-arm64:
 .riscv-fixed-randconfig:
   variables: &riscv-fixed-randconfig
     EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_BOOT_TIME_CPUPOOLS=n
       CONFIG_COVERAGE=n
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64-debug:
   extends: .gcc-riscv64-cross-build-debug
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..fc7a04872f 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -1,12 +1,10 @@
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
+# CONFIG_BOOT_TIME_CPUPOOLS is not set
 # CONFIG_GRANT_TABLE is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_XSM is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 027bfa8a93..16a9dd57aa 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -5,6 +5,8 @@
 #include <xen/xmalloc.h>
 #include <public/hvm/params.h>
 
+#include <asm/p2m.h>
+
 struct hvm_domain
 {
     uint64_t              params[HVM_NR_PARAMS];
diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
index 387f372b5d..26860c0ae7 100644
--- a/xen/arch/riscv/include/asm/p2m.h
+++ b/xen/arch/riscv/include/asm/p2m.h
@@ -2,6 +2,8 @@
 #ifndef __ASM_RISCV_P2M_H__
 #define __ASM_RISCV_P2M_H__
 
+#include <xen/errno.h>
+
 #include <asm/page-bits.h>
 
 #define paddr_bits PADDR_BITS
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732324.1138288 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOK-00048k-7h; Wed, 29 May 2024 19:55:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732324.1138288; Wed, 29 May 2024 19:55:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOJ-00045Y-Ua; Wed, 29 May 2024 19:55:19 +0000
Received: by outflank-mailman (input) for mailman id 732324;
 Wed, 29 May 2024 19:55:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOI-0003Nf-Py
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:18 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f9e4398-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:18 +0200 (CEST)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-52388d9ca98so210960e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:18 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f9e4398-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012517; x=1717617317; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DMQeYwEU6AHLj7P2XHDLxfUMvE/iSYQ4z0ULAUHzwVk=;
        b=bPCDrhJ0xPO12T7s02CCkp/fR74AlWxlqT40oVnkv8ZaX+kJeAS1GAX96g2w5civh1
         aZF0MXVr4hUAv1DKRKwLtCnGzR7+aIbmowxggE0kHH7MtmXEORizIR+do5DBmZhjZW22
         kvQLmDHv5cfvwPUrY7VVDiN4tKtYV054f46k9LBCS0+Af9fGanSrUVzDv30Krp9s0oNF
         mUrYEcHjSZGiJwDLh7Q9/HHe/ZQ7mNYzR2ARRBUDele5mE0QLhVjACJB3g3EJMZPBdA+
         2hzLCio50Zm/7+rT1r4Y3XxswAa8fT/B1ww9Jggzu47dI1dw6zseUB6HtjCQTuGJpeqh
         2LPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012517; x=1717617317;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DMQeYwEU6AHLj7P2XHDLxfUMvE/iSYQ4z0ULAUHzwVk=;
        b=U+tkmFrsxYoeYW9OdIFSF6kuXPbCAiSrZ2vZR46IhBNMxTLrtIsaqnhc4k9dtD8ial
         evcvTmABXTu4gxsLFVEQiN5F0biyelblfVK4tYavrulKiMRmUh8IJNoE9ossxik10zLE
         x0EFi6SgldO3Uaj6GECdfHu/+SCXbN/skb0EQ7tRKWBg4F4w9dr4ZwxAfJZbYCA66EgA
         3zqQsCs7SYUyRtxu26NWz8eB/cbCiNqyVwXMajAUO3EiUk/HbdLFsOJWYguiBagM611n
         yvu1GAaF/O5fv7GvmURC0x2QERLlwAjEjMm2O6yCgEB/Ab2eynkEYscxl8Ioojn41442
         jtEw==
X-Gm-Message-State: AOJu0YyufUe5e2hiDyATrxj6wId53PDu+JNOo9BlCd1Hisjd0TopeR0R
	OU/BX3xAUv0P/SAqw6RYm/+7lvVnH+4nzpHcvyRnlJ+eHAJ9QtAaFI+831B6
X-Google-Smtp-Source: AGHT+IEH4C5lILkalV+nWFz2iyaFhC6D1bbYewSPaIJ87E/cD2PmkOAHAOjVu3nf9oU3j06qC1SGcA==
X-Received: by 2002:a05:6512:3619:b0:529:be5c:8de0 with SMTP id 2adb3069b0e04-52b7d499b84mr117290e87.49.1717012517035;
        Wed, 29 May 2024 12:55:17 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Wed, 29 May 2024 21:55:06 +0200
Message-ID: <d00b86f41ef2c7d928a28dadd8c34fb845f23d0a.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V8-V12:
 - Nothing changed only rebase.
---
Changes in V7:
 - update argument type of maddr_to_virt() function: unsigned long -> paddr_t
 - rename argument of PFN_ORDER(): pfn -> pg.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V6:
 - drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
 - parenthesize va in definition of vmap_to_mfn().
 - Code style fixes.
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 242 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..cc4a07a71c 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,246 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732325.1138304 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOL-0004br-Eh; Wed, 29 May 2024 19:55:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732325.1138304; Wed, 29 May 2024 19:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOL-0004aT-7t; Wed, 29 May 2024 19:55:21 +0000
Received: by outflank-mailman (input) for mailman id 732325;
 Wed, 29 May 2024 19:55:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOJ-0003Nf-DU
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:19 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ff10d06-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:18 +0200 (CEST)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-52a6ef5e731so236714e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:18 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ff10d06-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012518; x=1717617318; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=HT5eQZFG3kOfjlltqo/sjfVASYxX5jfFvb3QP7WAnA4=;
        b=j8M6/thJwD6aBd8mAkZQP6aKlla6gkO38+sWFUhn5FkbvpBcF7CVDpNcXrlQvRlL4t
         dWlnN8gMX6FN1RE1wQAylp6BNRY5/8tSeRaKoc1MrQqlQAylPcLkN2u5JMLsbHxD3XD2
         9hSt1CjhZO5I5/f8gTXEoASwUL9spknkOcDimjaqNLce/z0egmZe/DJwnKV2Ycb37HbI
         29/WuvKgwbunptpTHTTiNiBNjT6qnkvujRgsK6BYzUOXIxrHtnwFt8vrjxKQqh/qk4tc
         em+YVEsBMdOVLFZGuvQe44u2Z6xN6bPDq9V2TDEfYDZlMM7GJJUgGgCBEIVtcAjv5uX7
         /XDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012518; x=1717617318;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=HT5eQZFG3kOfjlltqo/sjfVASYxX5jfFvb3QP7WAnA4=;
        b=F9Wf8OsduInVIXZc/W4GR/3pycUH2aCzObD66NLVH0/2UBGrtmKY6eIO+NLhswUWNN
         lWyFQtYW54M9DBeDUGOIbYvLyUOnN2+wzSNHqtmcRV6oq1kzL2NzUZThDuTfebWzY+3m
         D6hymlha7/rzJtkfEAPTwEs4e8jVF0p/t+DT6B5nyO1UUO2GpYf7sjpmNXoi4fmf/Xak
         y6B0a1AaYtMA865U2+h838VnHUExFyKerVODuBGwYakZgna1Gs0S2kK4mdtH5b0FeFaw
         BM4mBZhxw1oV5uXi5HsvLeL1Myj6wDBw9gHILaumRVS5KctHAUGlO/ZR/rqeAXCwUKI1
         TeKQ==
X-Gm-Message-State: AOJu0YyoZ21ZBil0TROmyE28SKBdqm4Wal+5lmGwBGpbFK9qK1/2bU9L
	TUQGPfJ5VgGU4K14cxeDt0CuaI9E1hQylXj0IQYKqgskh3H2B04Ar1P3XVup
X-Google-Smtp-Source: AGHT+IENbAEXIatG6RzIfpPoL19Qek9oosir4ctTUoquQPNkD+5xNxItPVUgu7SlQtOFp7OajD8J+A==
X-Received: by 2002:a05:6512:3b06:b0:52a:d87f:60e3 with SMTP id 2adb3069b0e04-52b7d48c622mr148339e87.57.1717012517792;
        Wed, 29 May 2024 12:55:17 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 6/8] xen/riscv: add minimal amount of stubs to build full Xen
Date: Wed, 29 May 2024 21:55:07 +0200
Message-ID: <4e5b814c3f73bd4ae6bb80296e17ac80bd8d224a.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7-V12:
 - Only rebase was done.
---
Changes in V6:
 - update the commit in stubs.c around /* ... common/irq.c ... */
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 439 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 523 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..8285bcffef
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, but common/irq.c isn't
+ * built for now. These changes will be removed there when common/irq.c is
+ * ready.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732326.1138320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPON-0005Bn-UN; Wed, 29 May 2024 19:55:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732326.1138320; Wed, 29 May 2024 19:55:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPON-0005B5-Q0; Wed, 29 May 2024 19:55:23 +0000
Received: by outflank-mailman (input) for mailman id 732326;
 Wed, 29 May 2024 19:55:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOK-0003Nf-77
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:20 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60881d7a-1df5-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 21:55:19 +0200 (CEST)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-52a6ef5e731so236760e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:19 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60881d7a-1df5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012519; x=1717617319; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RB69PkHLC0P/c7KJEa2nAcuiiLx+QigdAnc7qWzg1l4=;
        b=ZdoiPYve5Ig14vbzHhUPSfvKDnbmzOTLlhgqdc1YrraXzJPtpQS+5GPILCstLUNt86
         cgn3RgnwNicgPagmprQQPqZzBZ1Uv8iVaMwX9o5yoBgEdaaQUmt0cy2NLg4+GGp4xaZW
         POJIYT15Y8jzWwBzc9XgOvRXGCIvCqxun+6BPG0AhM9QEzWf2KHDDUxpDiJWquGfkOnY
         g9HCIlvRAsnvkuuajWz5ZhIIYdoGkZ9oHIV/wxzyvBOQMANbW+LmdjnWhUcPsf8TJAAp
         iBUSTn0XD7c6uH3W0fCKi5GzkYD6xg5CJ67t5QdxQy0QwzxCeMWITS60YmNx9IQBZfvl
         +5Tg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012519; x=1717617319;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=RB69PkHLC0P/c7KJEa2nAcuiiLx+QigdAnc7qWzg1l4=;
        b=KrGQfEyzqiMKNiMI0bMd8ySwK+JHX7y5GZh2s8eQReW/GrSyFcRpFzq53nARJ46LoQ
         lIF10gpnABILlhIYUodtxmZ3Bc9AyjcF6uJAX62WWn7A+KAybT2BNz1eM00WblePQIoD
         AUPMsMmbg/OFUkF+sXKPTAYcA6OP6WVr5uC7hp4JCoVyuwWfM3O5DgpLgMqdAWuETJ0g
         uX9eiTIjsyBDneREGVzWxwhyEB4d55r6lm2cbm4uMFcvzKxdfvM9eUli8vTMo8FGnZR6
         nZvYgaF8YHeY/VivaDlngviTj6nPT2xfk5tq+5IOg1TOU90GGxFSaUAadtbZ7na/3XCF
         FCjQ==
X-Gm-Message-State: AOJu0YzJA4gDaCXYZ4RqhBfPNtSVnSebHxpSsJZ++2H6A+XjlDW89RJV
	D7ahh1cWYpD9R9EcP5SGw1/54YwUFYUoByUuZm7FRriYqwS4HVPDg3hC/B8U
X-Google-Smtp-Source: AGHT+IEdszj2ogMZI1dkrwdXUnmCfQoc81L+BYRLNYf+UmK0Mq/P+Qp3C8Sbi01ORtNa1IAloSaJmg==
X-Received: by 2002:a05:6512:358f:b0:51d:1002:520d with SMTP id 2adb3069b0e04-52b7d4905b8mr101793e87.64.1717012518832;
        Wed, 29 May 2024 12:55:18 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 7/8] xen/riscv: enable full Xen build
Date: Wed, 29 May 2024 21:55:08 +0200
Message-ID: <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 At least this patch cann't be merged w/o Andrew's patch series is merged as ffs related
 functions are used from that patch series:
  https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
---
Changes in V5-V12:
 - Nothing changed. Only rebase.
 - Add the footer after Signed-off section.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  24 -----
 4 files changed, 15 insertions(+), 197 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8c071aff65..17827c302c 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 8285bcffef..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, but common/irq.c isn't
- * built for now. These changes will be removed there when common/irq.c is
- * ready.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 19:55:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 19:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732327.1138325 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOO-0005GH-CO; Wed, 29 May 2024 19:55:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732327.1138325; Wed, 29 May 2024 19:55:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCPOO-0005FX-5u; Wed, 29 May 2024 19:55:24 +0000
Received: by outflank-mailman (input) for mailman id 732327;
 Wed, 29 May 2024 19:55:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DI5e=NA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCPOM-0003Q0-2p
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 19:55:22 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6115c601-1df5-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 21:55:20 +0200 (CEST)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-52ae38957e8so180184e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 12:55:20 -0700 (PDT)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b249sm1344203e87.178.2024.05.29.12.55.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 29 May 2024 12:55:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6115c601-1df5-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717012520; x=1717617320; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hr0rCeOkEj/7qjuywq9V+WOZcGXF4MF4JxLpkyei6eA=;
        b=cEmnx+D3ywjqDiQ61TqSFWgsXFMvhEbYnI/hRZEw8VaE8AEDKhpM2CW7mNyuIYjKya
         YTWqefzvZRF7/t7Ip+0l5WO5po5RT7mRuvyOL9IL6IGHp4U/3IQSKbQ8imhZI1zmf6pb
         3g/7cKLnCa03a5u/aqvKL0efgsaSeSzrKQOXVNr06o44+ldfZhnsCv93UAR/CdIjv7YJ
         LWy5nfyWpJ718esvrHeBwKdwX1qS5o3l2krweiFz5nIjGjJ/pMQ7pxqbgrxu5tl3t0ZM
         ahiqdLG8kVb4Wdw4B4R3Uww4F08lkBWJcT4jWadlTN3SIHJr7BuHAi1ysCI5rLe7zJkF
         vD1Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717012520; x=1717617320;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=hr0rCeOkEj/7qjuywq9V+WOZcGXF4MF4JxLpkyei6eA=;
        b=rX4lmIOo87FzgGPJ+4n2W3oTC1Fa2k6e9uWPegTvePDCFwGeuZkfY+xVAF7YMQj34A
         z78e3WZej2K9SuRTf1872jmtEfqC54USl/WZUJxU5roS17M5fjdPwY/2HmoIHtsMlBjq
         uBak8n48sLvHxJYVX/TB5iQLD6qmiKiS0k/VgnNIcpFTRs8EG1uPXLMKxDO5Cbrxnf3v
         qE0S7Y4UXfmz5p2sWcjEOuzJLstXCH6LgEE50AtX+rgNxawUxJr0vR++z6VOs6JbCUq9
         wmnQcZziRDthkJlTb6K6rK58pt6/48JfiY2r7aLMD3z9ZSJCqxIUutLIX2yYOQn/uV3U
         nopQ==
X-Gm-Message-State: AOJu0YyRiAvqDZ/6EkIWOINDsFwL8dty0934uUF/zQaFlGwmAy4UZqwZ
	jQDgo05jYQds2WAwg8OuNSpBdj+gAFHO1sRSw4T377iacblFauylF51V4x2v
X-Google-Smtp-Source: AGHT+IHZb5bjx6as4NkrWWXbmTHBBqMPT5aoF6PiwOXbVfFYazz9oHCuIHZn3+RaT6A7TSxMotIgCg==
X-Received: by 2002:a05:6512:40b:b0:528:4841:e4a1 with SMTP id 2adb3069b0e04-52b7d432e94mr105642e87.40.1717012519811;
        Wed, 29 May 2024 12:55:19 -0700 (PDT)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
Date: Wed, 29 May 2024 21:55:09 +0200
Message-ID: <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <cover.1717008161.git.oleksii.kurochko@gmail.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
--
Changes in V5-V12:
 - Nothing changed. Only rebase.
---
 Changes in V6:
  - update the message in README.
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README b/README
index c8a108449e..30da5ff9c0 100644
--- a/README
+++ b/README
@@ -48,6 +48,10 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+          Older GCC and GNU Binutils would work, but this is not a guarantee.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.45.0



From xen-devel-bounces@lists.xenproject.org Wed May 29 20:41:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 20:41:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732367.1138341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQ6l-0007Pe-Pm; Wed, 29 May 2024 20:41:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732367.1138341; Wed, 29 May 2024 20:41:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQ6l-0007PX-L8; Wed, 29 May 2024 20:41:15 +0000
Received: by outflank-mailman (input) for mailman id 732367;
 Wed, 29 May 2024 20:41:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCQ6k-0007PN-6b; Wed, 29 May 2024 20:41:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCQ6k-0007GL-1K; Wed, 29 May 2024 20:41:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCQ6j-0003Hd-N9; Wed, 29 May 2024 20:41:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCQ6j-0004oy-Mf; Wed, 29 May 2024 20:41:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YI+glZokiknySNkNFm0lZcc5Qhx+xQwwChEySMTsR80=; b=Zg7hSGsjdrTdQewdMLkFVRRiIh
	jAUI/yC/G2HLMytKE4DIY95ErrikRcA1j8S79Tv2KTOw0+SMsDXtss7CrLpvUNS9V7RvgUTC4Mmoj
	F1xtzrQ3UKa0zLXSo5Fd2D8c2rp7mpCoFDj3o1iFDD70kmgw/XH6DaYd+WbZ6feQLf/Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186184-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186184: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
X-Osstest-Versions-That:
    xen=0840bc5ea114f536a4bdfb2ca095b79f2069aae6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 20:41:13 +0000

flight 186184 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186184/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d
baseline version:
 xen                  0840bc5ea114f536a4bdfb2ca095b79f2069aae6

Last test of basis   186182  2024-05-29 14:03:46 Z    0 days
Testing same since   186184  2024-05-29 17:04:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0840bc5ea1..12f9599a3b  12f9599a3b135afa270ccf0795f3e9665059955d -> smoke


From xen-devel-bounces@lists.xenproject.org Wed May 29 20:43:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 20:43:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732374.1138349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQ98-0007xq-58; Wed, 29 May 2024 20:43:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732374.1138349; Wed, 29 May 2024 20:43:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQ98-0007xj-24; Wed, 29 May 2024 20:43:42 +0000
Received: by outflank-mailman (input) for mailman id 732374;
 Wed, 29 May 2024 20:43:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Yx1n=NA=epam.com=prvs=28793c6931=volodymyr_babchuk@srs-se1.protection.inumbo.net>)
 id 1sCQ96-0007xd-9q
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 20:43:40 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 201b8e73-1dfc-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 22:43:38 +0200 (CEST)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44TG8Nd6029630;
 Wed, 29 May 2024 20:43:31 GMT
Received: from eur02-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3ydbacyqww-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 May 2024 20:43:31 +0000 (GMT)
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 (2603:10a6:150:16a::21) by GV1PR03MB8688.eurprd03.prod.outlook.com
 (2603:10a6:150:90::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.16; Wed, 29 May
 2024 20:43:24 +0000
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e]) by GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e%3]) with mapi id 15.20.7633.001; Wed, 29 May 2024
 20:43:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 201b8e73-1dfc-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L+vT4JwdRlt+IgaCvEerKJs4RbHA4YLqGLdQcFWsI/D9ZrqAkOi1s2wnpWlLw25Bv7MDWeBj1Ha1zTDYCv5QHqe5sQp4oG0fnDTpqcjxa3EGg+kUyG+vXfF9TvTt11kAXiQEN08SYVrJ88BGe8CW4F/EmHtmlaTkPaIYl5zZttfyisVEUKA/XSjbWUnCPNSU/uNvepJYWRYAxpsA+498elSSTTchU8TDaYFpdoveaaJfjCfxVlXQ8/SmjbFuxT2dpNMIcwVOgDZhatMCrdW2yYC+A1s8jJ5ENduIQQX9AIiipJ1alpNnMkCHaql9OI6cJmBhr/EGm019oVft3gOFiA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=poiSLI/4mVXatWy9jxBVhfK1NYsiz2gi7xQ01hdZmZs=;
 b=LcffK7QhsLyCT48rYoYFt2Hro2GB99upE8AflUK25u97/jz3vwvknDyzRQZhpVAjnccwXd6zArTit8Wiv7P05KtH+/jdIJMP5gClCSCGdJjGh2Gzf5M0EWEIb5nbSEX8AtNz4AImQH/cxm6Zu+Qrp5SM9efruqNU+wD91G0/3Oho7L0Pqh3gvQxWht911G9UpKirevPPpCWu+mNRfcZ7M4lIge6dGUD6+8Imw8s24mTgDTBqCqDAocnCZKjD/7mKuym19VCnrcwK3mN4xZz19kAx4otxL3mCcz8UNXuXe/dqY0YsyDuewjpKPQgKG8zO9rqCz/nqu/zfdbWMe6K4qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=poiSLI/4mVXatWy9jxBVhfK1NYsiz2gi7xQ01hdZmZs=;
 b=P3fjA+z177sX2zwnSoHbGAypEmqwTKJrUr1hTREOnRMBXnSGb0DDd+m2pMj4hW5qAj3A8XcLvANC+UC74D6j343ERvOqDTUb879WG51vxhputHM+9TjVl69TwbdlQJwAC9LjRClhuHp6LLYwJZW8w7LS2Y5MzAPbuZcNczxknYbfTfPOqQjWKXm+dkv/nGG5FD4R+WRpzY74+GlrJhVUfpkhNCdnfGf35RNCb0PvLIpRxbAPvclpdAMxnJX8kH5UqRwGqDxxG5GdOkQP7JbtkDsHXvQrVQSiL6cXCNDaDeHYwm6q4Z5n7grnvUWwqFh6SkNkfwZo73OAHvoHmsnU1w==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Michal Orzel <michal.orzel@amd.com>,
        Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH] arm: dom0less: add TEE support
Thread-Topic: [PATCH] arm: dom0less: add TEE support
Thread-Index: AQHasgjZKwS4pdCyJUe9LewXYMLHeA==
Date: Wed, 29 May 2024 20:43:23 +0000
Message-ID: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-mailer: git-send-email 2.45.1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: GV1PR03MB10456:EE_|GV1PR03MB8688:EE_
x-ms-office365-filtering-correlation-id: faa4e910-2f81-4403-6bb7-08dc801ffbe6
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|376005|1800799015|38070700009;
x-microsoft-antispam-message-info: 
 =?iso-8859-1?Q?7h/KCmUP12knRe+R1IdPyAKKDh8UCArIMj/Vo5H8p+FCk+7FOO3pPAf4kV?=
 =?iso-8859-1?Q?Gp9aomIcAK0E0UOSX1ld/6hMk6u3jKMeAQ3FR3hQEBmgQJgfua2yrItMVX?=
 =?iso-8859-1?Q?00CNoGQdX+WvV+7utMTyvVORTyyA2T0amiCuw6ynU72hzsUy9WLMIZSSrk?=
 =?iso-8859-1?Q?W577ECtCWG6xiPW79MYrC+5/+WzSDaIcD7Y+A19isF3Tr8P1d4m/ss64tY?=
 =?iso-8859-1?Q?i1CAZQNtKU8GFSNh/MJzHLB7yGyXRMHYtNV+9+3eER/B1m2gKvnzUG3A0a?=
 =?iso-8859-1?Q?NOzeaBoxcP+a4hn8F6NKRx8op3ePTFmEfatyqVBKHFE9MsHjoeOZWXpKct?=
 =?iso-8859-1?Q?72qIXP5ICEEhqwutKXxEe2lYJ+vLsXzJPnNwsI3ZEdDhsHqRhrRQQsGBTH?=
 =?iso-8859-1?Q?3GQ0x4xPQwCVd29DzjCzULqM8/oRPxPbuxPrf1WtK2hefUcJNVb6wswHm4?=
 =?iso-8859-1?Q?hdsEHE9MIuglWIRYzj+aA/yu4HDU8QFJzEU9U43DVbyqzgKdMS04LpN50X?=
 =?iso-8859-1?Q?xR0RiZHMrTtnjdX1jVHBwFvb1eO9kh0mdn/gE7TSCq1V9BPHWQgltPHnmG?=
 =?iso-8859-1?Q?tP+k6UzrX4gkSeB3d0dTxjqBjpVUlOSADAKykm5hAGN4iiSkuhvUTC2fGK?=
 =?iso-8859-1?Q?mQ2pJ9A7vSE2N1zJBjVLXCy+NVwsJlzPVb+OTK0jaQHhPVzwLE9+kTFBQn?=
 =?iso-8859-1?Q?HKHeLMLQBBXPBcayi9Xs5OIrBVIcRR+Hkv3z+Yg137ohwrBRCKMcfISI6H?=
 =?iso-8859-1?Q?6J0uXXfssB5vUb/7Lb2gXqvzLlFPQ3TuxZ8dUlbtTIcPbS5HAMqDCZjPnQ?=
 =?iso-8859-1?Q?J5T9hUosBM4uZUdfDjFT2vTrI3h6uSEoHsiTOXT7jTn62lBYA4T6ba3LLF?=
 =?iso-8859-1?Q?R9Ou4vpBpCu1Q8Q0N1cXcQmVtDduZJ8ClF+DJriYFxD046Mk62KVpOd073?=
 =?iso-8859-1?Q?X+J9A3D4Aij9dJkrMn1xJJOdiM6jJk0GugOeGNUrX4RxWSsQW0aQow51KT?=
 =?iso-8859-1?Q?shXy/PCC4sac2zRc62e4MT7Qb0ZQw6F8mvHytYbW+mQWmhx9O4tWPKt27G?=
 =?iso-8859-1?Q?uABtOcJSzpw+7rkYmdY4e0sYN/qZCqfl9LMX0Dg0YBEAhSyNVb2N6MTZDI?=
 =?iso-8859-1?Q?ZkjIspa62vYa7xBcCmK4cgglNts4GAIOOMEf79TlnlKAB12aXkz8uayj0L?=
 =?iso-8859-1?Q?iltj3gq7i5uGOuFd6ssyaKeOAakTqH1+fdEZcQBWACkNQwDbCBuK3cAwgS?=
 =?iso-8859-1?Q?TpMWA4MORbEufNkp13pkX67fcZmBOy2BLYHcz4qd+Reuh4OJiN/15+MXMi?=
 =?iso-8859-1?Q?TkzxJI44dxlkVxiJF15rfO3Gt++f+Azq5De/aGFYAluszrTHkItlCRM0pb?=
 =?iso-8859-1?Q?59okONbZhIAOlyYMBwWbixr7ydxsWMaA=3D=3D?=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR03MB10456.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?DHlJ864yJLphoECHgqn/ki2/AsdElq5aaZ9JiQKFdGpsjECfa5s/BQP8Dh?=
 =?iso-8859-1?Q?9lgdegKXlW+cL+SHBC0J2P7aYy41198BhagaSr9wllaS0/do0/bVlQVyQK?=
 =?iso-8859-1?Q?oVswbzH6Z7JBsAVL2Ra3Xh9U6iC7DXOBr5Vaoiw1MeE4OJ53QoMaKYcC1f?=
 =?iso-8859-1?Q?vrdkz35sCWkuIVMJWLzv0b85et+e7FK+tnMWFmLa0vWxq5PVHFVhkvgMFB?=
 =?iso-8859-1?Q?BzwNilJbJ7/FuNcm7WMRzWGni96wX8lzeO5Q7OBUUyEXJOEcqauWqZ1X1k?=
 =?iso-8859-1?Q?Sexjy/l9q3Z4djy3ELcyRusxcjuKDAtlXyLYnf6OwN/KxRlXaqaBKuJpPH?=
 =?iso-8859-1?Q?I+RbDaXFr7jXQt5fdPwG0W8zwP1+hFviDRaPcsmJH8EJEybuLf4WQw2ELF?=
 =?iso-8859-1?Q?xf4SDvUcs7/oPXTpnzZq1PwaiBjPAt9P4Do8rj8B8Ug2pvqlVs3ccQuGkR?=
 =?iso-8859-1?Q?wmQltzxblvfFJ/XiTS9HVfxxubPMns5dFrdfVq1ZXz0bWgNzKJKCgjWII5?=
 =?iso-8859-1?Q?4BviHflKM6zspEa/C50aAc/vCsy8BNtePPy8N3sj8rgLux8Kuuz5TIacNk?=
 =?iso-8859-1?Q?Cu2gyhyhfSISUQx1H0Xvtyc4aax2pm11zu9yaOZ5eRDNDJ0KfwYRbma+ko?=
 =?iso-8859-1?Q?fPIIJOKsmDWyUtIfFGRIAmUrloQCmHKhR4xjQ+HIiudafDvu+oK+rpVNRA?=
 =?iso-8859-1?Q?DUih44JlKAFvdY3k5LijlUm/qjoBO+D+dUsVPlJwRRhBKvBz1M31IyLEwF?=
 =?iso-8859-1?Q?RpJ6YuknIUqqRwTN3KLN8dvLwep3QGngtVEFSY3/aGDAXEgTCnFvlilGYA?=
 =?iso-8859-1?Q?KprdMNqAv7VzbbiML7IcR0CWBG/y3OM91AkXpn0V1irk+sC4++l6w35Jt/?=
 =?iso-8859-1?Q?O3EZjOqCa/ys3qAl7oiqqvo6j+hCXjRd9PQ81GNC17Ykf+CDN6nkSlN6Om?=
 =?iso-8859-1?Q?+7AYMtxm5GYTPROLTvXUzyMzNEV012sg3TY9pl50kwCxU14lwFbt+KMpzc?=
 =?iso-8859-1?Q?ncXnJpmxPYcE+YE76zwJoc5VTxcwaENyicGd1dzA5jy/VgLgX+fikGEEE0?=
 =?iso-8859-1?Q?SbBWDkctcPwRN6djnNuL/zAnWCYLddoOKRmYGVHAtC0xD09dqk12AtOdmd?=
 =?iso-8859-1?Q?2RqMzL8UlruTAM/htXs/pbPfkNUDagaPQWCBmEBz6OkcqSLGXA8XnFcu9H?=
 =?iso-8859-1?Q?Xv2TJ+0JkjB1tqR+XGJW1T1IQ8pqzINs+qvphenok4DT5b/xw+NEGOuyE5?=
 =?iso-8859-1?Q?79IQPSrtmIyNzWmwl0fa+qtUdc6xIVvBBr3Sk0DsDeR1WC0bVWJ/tSGYH/?=
 =?iso-8859-1?Q?dVpPZPTamRhUtfumEq7xV/ssfAIvHxiX4GXbxxY4h6aDyIHO8sQaiEtNpQ?=
 =?iso-8859-1?Q?mhdQNwjF7iChQ085U3Pbs/cKNmnwFTLm7yZlxzKbX3Qr+MUwyHnr/f/utZ?=
 =?iso-8859-1?Q?JyUpVuK6bvOUYZsiVC92rNPV6HmkZQTv18xk8PEC/Egn75E8LSQA2v2vPi?=
 =?iso-8859-1?Q?UyLOZUkNq3xG9ihMO4DxXxYbPeCZF8X1GAe7tpywTPOESNYVwBpaWruus7?=
 =?iso-8859-1?Q?D50gZTNf6Oychcx7o2nIl6ro9ZDBGmQwG0JGUwFDvMjzDy/AXC955+hFbx?=
 =?iso-8859-1?Q?qJuHapmbbrTNWtU2nAHWtplI5Mv2D2TW3hsCX5K3Hm/kOIITqL1RxiXg?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: GV1PR03MB10456.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: faa4e910-2f81-4403-6bb7-08dc801ffbe6
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2024 20:43:23.7975
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ySONSQcVqxDddaS4mSobLiHxW987QJwLOwKvTY+RT01MwypE6DIxjsPs6Ys8NxpFdnxQbeoHwO8Zt3dlfRkvlc4sBl7H8jvWbJDA7nhhI6Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR03MB8688
X-Proofpoint-GUID: a6KU1NCBuiD7i22QR48HX3VEhhig8Eac
X-Proofpoint-ORIG-GUID: a6KU1NCBuiD7i22QR48HX3VEhhig8Eac
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-29_16,2024-05-28_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 phishscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 malwarescore=0
 mlxlogscore=999 clxscore=1011 bulkscore=0 suspectscore=0 spamscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405170001 definitions=main-2405290146

Allow to provide TEE type for a Dom0less guest via "xen,tee"
property. Create appropriate nodes in the guests' device tree and
initialize tee subsystem for it.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
---
 xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++++
 xen/arch/arm/include/asm/kernel.h |  3 ++
 2 files changed, 72 insertions(+)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index fb63ec6fd1..1ea3ecc45c 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -15,6 +15,7 @@
 #include <asm/domain_build.h>
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
+#include <asm/tee/tee.h>
=20
 bool __init is_dom0less_mode(void)
 {
@@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct kernel_=
info *kinfo)
 }
 #endif
=20
+#ifdef CONFIG_OPTEE
+static int __init make_optee_node(struct kernel_info *kinfo)
+{
+    void *fdt =3D kinfo->fdt;
+    int res;
+
+    res =3D fdt_begin_node(fdt, "firmware");
+    if ( res )
+        return res;
+
+    res =3D fdt_begin_node(fdt, "optee");
+    if ( res )
+        return res;
+
+    res =3D fdt_property_string(fdt, "compatible", "linaro,optee-tz");
+    if ( res )
+        return res;
+
+    res =3D fdt_property_string(fdt, "method", "hvc");
+    if ( res )
+        return res;
+
+    /* end of "optee" */
+    res =3D fdt_end_node(fdt);
+    if ( res )
+        return res;
+
+    /* end of "firmware" */
+    res =3D fdt_end_node(fdt);
+    if ( res )
+        return res;
+
+    return 0;
+}
+#endif
+
 /*
  * Scan device tree properties for passthrough specific information.
  * Returns < 0 on error
@@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain *d, s=
truct kernel_info *kinfo)
     if ( ret )
         goto err;
=20
+#ifdef CONFIG_OPTEE
+    if ( kinfo->tee_type =3D=3D XEN_DOMCTL_CONFIG_TEE_OPTEE)
+    {
+        ret =3D make_optee_node(kinfo);
+        if ( ret )
+            goto err;
+    }
+#endif
+
     /*
      * domain_handle_dtb_bootmodule has to be called before the rest of
      * the device tree is generated because it depends on the value of
@@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
 {
     struct kernel_info kinfo =3D {};
     const char *dom0less_enhanced;
+#ifdef CONFIG_TEE
+    const char *tee;
+#endif
     int rc;
     u64 mem;
     u32 p2m_mem_mb;
@@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d,
     else if ( rc =3D=3D 0 && !strcmp(dom0less_enhanced, "no-xenstore") )
         kinfo.dom0less_feature =3D DOM0LESS_ENHANCED_NO_XS;
=20
+#ifdef CONFIG_TEE
+    rc =3D dt_property_read_string(node, "xen,tee", &tee);
+    if ( rc =3D=3D -EILSEQ ||
+         rc =3D=3D -ENODATA ||
+         (rc =3D=3D 0 && !strcmp(tee, "none")) )
+    {
+        if ( !hardware_domain )
+            kinfo.tee_type =3D XEN_DOMCTL_CONFIG_TEE_NONE;
+    }
+    else if ( rc =3D=3D 0 && !strcmp(tee, "optee") )
+        kinfo.tee_type =3D XEN_DOMCTL_CONFIG_TEE_OPTEE;
+#endif
     if ( vcpu_create(d, 0) =3D=3D NULL )
         return -ENOMEM;
=20
@@ -824,6 +885,14 @@ static int __init construct_domU(struct domain *d,
             return rc;
     }
=20
+#ifdef CONFIG_TEE
+    if ( kinfo.tee_type )
+    {
+        rc =3D tee_domain_init(d, kinfo.tee_type);
+        if ( rc < 0 )
+            return rc;
+    }
+#endif
     rc =3D prepare_dtb_domU(d, &kinfo);
     if ( rc < 0 )
         return rc;
diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/k=
ernel.h
index 0a23e86c2d..7e7b3f4d56 100644
--- a/xen/arch/arm/include/asm/kernel.h
+++ b/xen/arch/arm/include/asm/kernel.h
@@ -57,6 +57,9 @@ struct kernel_info {
     /* Enable pl011 emulation */
     bool vpl011;
=20
+    /* TEE type */
+    uint16_t tee_type;
+
     /* Enable/Disable PV drivers interfaces */
     uint16_t dom0less_feature;
=20
--=20
2.45.1


From xen-devel-bounces@lists.xenproject.org Wed May 29 21:11:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 21:11:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732385.1138360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQaF-000477-D2; Wed, 29 May 2024 21:11:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732385.1138360; Wed, 29 May 2024 21:11:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQaF-000470-A9; Wed, 29 May 2024 21:11:43 +0000
Received: by outflank-mailman (input) for mailman id 732385;
 Wed, 29 May 2024 21:11:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EvTx=NA=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sCQaE-00046u-EK
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 21:11:42 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0a4659fb-1e00-11ef-90a1-e314d9c70b13;
 Wed, 29 May 2024 23:11:41 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4VqMWZ58Ltz6Cnk97;
 Wed, 29 May 2024 21:11:38 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id MByd4wIRut8R; Wed, 29 May 2024 21:11:34 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4VqMWR0L4Mz6Cnk95;
 Wed, 29 May 2024 21:11:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a4659fb-1e00-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717017094; x=1719609095; bh=wVUy7o3bXf/noy6YSu0RsfBf
	kRiwLQrgb5cMP2Pa7q8=; b=QFjHmHY3/zEesfTR56+Y2SAfx/9kF8USAaPfodaa
	1NlbISeQtFhMQo5eKeT71tNXYkxOsczf07HR2Jg7kDECPZHbMqyg4jYdehcmpc6U
	pDScRpKKu8Rjyhm7pbf60HZtmxriYQ66+aBloWTpP6m/aGYjU5d8dR4eSVUqQ/OA
	xYtc6lzNLpfolCh3T1Mfx+tpCeFXkuhAlul/aC6d93xzuIXsbbqvGhKw9HLqQ8Hv
	yZpoz1hZaaNwZnzsSClMVINGA6pWnoN6MCYirTHscroKGo27eDVHIhGrEeOJHh7Z
	7W5OQwUFrkBQjwlsjPeC4wUGmMgkfdxis25NEBJvc3gvXQ==
X-Virus-Scanned: by MailRoute
Message-ID: <7828c48b-2a37-4033-a634-7a4fad4f94fd@acm.org>
Date: Wed, 29 May 2024 14:11:28 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 07/12] sd: factor out a sd_discard_mode helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-8-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-8-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Split the logic to pick the right discard mode into a little helper
> to prepare for further changes.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>


From xen-devel-bounces@lists.xenproject.org Wed May 29 21:12:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 21:12:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732388.1138370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQae-0004XV-LN; Wed, 29 May 2024 21:12:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732388.1138370; Wed, 29 May 2024 21:12:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQae-0004XO-Ha; Wed, 29 May 2024 21:12:08 +0000
Received: by outflank-mailman (input) for mailman id 732388;
 Wed, 29 May 2024 21:12:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCQad-0004Vv-6H
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 21:12:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCQac-0007ov-RU; Wed, 29 May 2024 21:12:06 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCQac-0002qN-Jx; Wed, 29 May 2024 21:12:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=THmYiT9zAxdWi1+vL4P+XbWF6tAahHUIHfjQWlhZ2ic=; b=ia16PbdKvSXROAGscJhwOwK40l
	5J6zfCkh/Td3jyxD94v3IyKiF8NFnoys4aCpn218kBhCUqe6AfY2gQ7+hZhuLlCGdzFKFrOPFzZRp
	MrNTWU8mwNRs4MhZxehImljW3dC0dZiVee9nElJDraD69z+Zms51PT8C1iGclHOKzMCs=;
Message-ID: <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org>
Date: Wed, 29 May 2024 22:12:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arm: dom0less: add TEE support
Content-Language: en-GB
To: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Volodymyr,

Can you clarify whether this is intended for the next release cycle?

On 29/05/2024 21:43, Volodymyr Babchuk wrote:
> Allow to provide TEE type for a Dom0less guest via "xen,tee"
> property. Create appropriate nodes in the guests' device tree and
> initialize tee subsystem for it.

The new property needs to be documented in 
docs/misc/arm/device-tree/booting.txt.

> 
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> ---
>   xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++++
>   xen/arch/arm/include/asm/kernel.h |  3 ++
>   2 files changed, 72 insertions(+)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index fb63ec6fd1..1ea3ecc45c 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -15,6 +15,7 @@
>   #include <asm/domain_build.h>
>   #include <asm/static-memory.h>
>   #include <asm/static-shmem.h>
> +#include <asm/tee/tee.h>
>   
>   bool __init is_dom0less_mode(void)
>   {
> @@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
>   }
>   #endif
>   
> +#ifdef CONFIG_OPTEE
> +static int __init make_optee_node(struct kernel_info *kinfo)

Please introduce a callback in the TEE framework that will create the 
OPTEE node.

[...]

>   /*
>    * Scan device tree properties for passthrough specific information.
>    * Returns < 0 on error
> @@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
>       if ( ret )
>           goto err;
>   
> +#ifdef CONFIG_OPTEE
> +    if ( kinfo->tee_type == XEN_DOMCTL_CONFIG_TEE_OPTEE)
> +    {
> +        ret = make_optee_node(kinfo);
> +        if ( ret )
> +            goto err;
> +    }
> +#endif
> +
>       /*
>        * domain_handle_dtb_bootmodule has to be called before the rest of
>        * the device tree is generated because it depends on the value of
> @@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
>   {
>       struct kernel_info kinfo = {};
>       const char *dom0less_enhanced;
> +#ifdef CONFIG_TEE
> +    const char *tee;
> +#endif
>       int rc;
>       u64 mem;
>       u32 p2m_mem_mb;
> @@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d,
>       else if ( rc == 0 && !strcmp(dom0less_enhanced, "no-xenstore") )
>           kinfo.dom0less_feature = DOM0LESS_ENHANCED_NO_XS;
>   
> +#ifdef CONFIG_TEE

I would rather prefer if this code is implemented in tee.c. We also...

> +    rc = dt_property_read_string(node, "xen,tee", &tee);

... want to return an error if "xen,tee" exists because CONFIG_TEE is 
not set.

> +    if ( rc == -EILSEQ ||
> +         rc == -ENODATA ||
> +         (rc == 0 && !strcmp(tee, "none")) )
> +    {
> +        if ( !hardware_domain )


I don't understand this check. Why would we require dom0 for OP-TEE? In 
any case we should avoid to hide a feature behind the user back. At 
minimum, we should print a message, but I would rather prefer a panic() 
because the user asks for a feature and we denied it.

> +            kinfo.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
Why don't we have a else case? Are you assuming that tee_type is 
initialized to TEE_NONE (which luckily happens to be 0)? If so, why do 
we need the check above?

> +    }
> +    else if ( rc == 0 && !strcmp(tee, "optee") )
> +        kinfo.tee_type = XEN_DOMCTL_CONFIG_TEE_OPTEE;

All the other values should be rejected.

> +#endif
>       if ( vcpu_create(d, 0) == NULL )
>           return -ENOMEM;
>   
> @@ -824,6 +885,14 @@ static int __init construct_domU(struct domain *d,
>               return rc;
>       }
>   
> +#ifdef CONFIG_TEE
> +    if ( kinfo.tee_type )
> +    {
> +        rc = tee_domain_init(d, kinfo.tee_type);

Can you explain why do you need to call tee_domain_init() rather than 
setting d_cfg.arch.tee_type just before domain_create() is called and 
rely on the latter to call the former?

> +        if ( rc < 0 )
> +            return rc;
> +    }
> +#endif
>       rc = prepare_dtb_domU(d, &kinfo);
>       if ( rc < 0 )
>           return rc;
> diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/kernel.h
> index 0a23e86c2d..7e7b3f4d56 100644
> --- a/xen/arch/arm/include/asm/kernel.h
> +++ b/xen/arch/arm/include/asm/kernel.h
> @@ -57,6 +57,9 @@ struct kernel_info {
>       /* Enable pl011 emulation */
>       bool vpl011;
>   
> +    /* TEE type */
> +    uint16_t tee_type;
> +
>       /* Enable/Disable PV drivers interfaces */
>       uint16_t dom0less_feature;
>   

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed May 29 21:35:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 21:35:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732396.1138381 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQws-0007uf-8k; Wed, 29 May 2024 21:35:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732396.1138381; Wed, 29 May 2024 21:35:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCQws-0007uY-4Y; Wed, 29 May 2024 21:35:06 +0000
Received: by outflank-mailman (input) for mailman id 732396;
 Wed, 29 May 2024 21:35:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Yx1n=NA=epam.com=prvs=28793c6931=volodymyr_babchuk@srs-se1.protection.inumbo.net>)
 id 1sCQwq-0007uS-Qi
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 21:35:04 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4ddcb97c-1e03-11ef-b4bb-af5377834399;
 Wed, 29 May 2024 23:35:02 +0200 (CEST)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44TFleUn029438;
 Wed, 29 May 2024 21:34:53 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3ydbacyu00-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 May 2024 21:34:53 +0000 (GMT)
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 (2603:10a6:150:16a::21) by PAWPR03MB9130.eurprd03.prod.outlook.com
 (2603:10a6:102:33e::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.16; Wed, 29 May
 2024 21:34:49 +0000
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e]) by GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e%3]) with mapi id 15.20.7633.001; Wed, 29 May 2024
 21:34:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ddcb97c-1e03-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JemPXmWXoVtQqTSEYEyEiSO+GBrMh0imp+tNFeEGTd4erC5emFDj1T1H9Dg53Gq/2hzAtlCHPjbwZ02zPgqpFqe8ghTGqyp928CwnLWeyC0jfuvtRsv/3T7iQaeOXrynYh71MZT1OaNb7Ge8jVW8xYhZqp/xsCmLxfT8bKaE/qd0nwmc+d0I92/7nlbwUvgEnxDNw1HjzL25AalsN0FtooPuT4eL220jP83ryXJ0ZVXsv8osTaXErKd559CO6E074hu0+Gqu5U9Q2B7RX1Fp+wOZaRw4cRpaqol9MWJY7iHjAEyd1S4qNhQOBi5rXljQ6nlLqhqUlgs+EjAqncAq1Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Vv87pxFTF0IfVJy5Y0SjWbBBW76pu6lBtaPyNGrGEe8=;
 b=UqS74zlB00nbynvd+UJxmYjSylmtEBRDF0xH17lcXQHFuVET0pRoXeN7De1NT1lqdxeaa8fnNfUBeCcjsDAKTS9OP+bQBpQ1ibR1eGEilZn2Ysw41nAZ6XrNBrAj3LNwOKNppwD43AZu5mVcMiXuEqhToW/FoBPC2rwB3g4TT8A5vJvvML1xAQekJe9vwxCkzDJg7+bmUNSvjZMx5b0choByh0Z5rZnvxIWBc13oYVsYXIE/rqQ+DFZcShwQJyup3jc3q1hF5UAjr6qQulEc7rZYVusALnIdQHsp+kaAwcE9ndKC7mxTlsBVHRM3Ivy/QvRxpDYWstbF+dAiD715aQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Vv87pxFTF0IfVJy5Y0SjWbBBW76pu6lBtaPyNGrGEe8=;
 b=UkVZ23Sacg/Dl7rUtrCVlVrJden3PmIj2JJnM4NoiWdiAyWO5t58loyTDDBWOb8dvSzOeP27fQIlz4eLp8I3U8cr+ytKo2qRte0NwwGeYht6TbYFSfITWFBEYlDzHpdcnoTzNlMiBegbuTCEDhUhAfT51RbXRdlc4ZIr0qYaOMqo389Qx+CGRfhxC27zr+CFVpOWxmaX7DtWcchdK/W6jAO+IjDNSue6HiXl7RLve9aiaHvRIkJuyubp8Kl3JYSEWA0e9/V88jIUlquI/q9zcY3vj5OYBR+BTJ53HQnShB3C7ryxh2P/OyuDDcv5jEP3ncpyiCBGoF1N61b7PtFwaw==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: Julien Grall <julien@xen.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Stefano
 Stabellini <sstabellini@kernel.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] arm: dom0less: add TEE support
Thread-Topic: [PATCH] arm: dom0less: add TEE support
Thread-Index: AQHasgjZKwS4pdCyJUe9LewXYMLHeLGutaSAgAAAkIA=
Date: Wed, 29 May 2024 21:34:48 +0000
Message-ID: <87bk4oxpxk.fsf@epam.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org>
In-Reply-To: <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: mu4e 1.10.7; emacs 29.3
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: GV1PR03MB10456:EE_|PAWPR03MB9130:EE_
x-ms-office365-filtering-correlation-id: eb966ea8-c62e-4423-f936-08dc80272a9b
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
x-microsoft-antispam-message-info: 
 =?iso-8859-1?Q?ZJAisddI+hjMqwe2fbi3RoZtK88N79L5sut3ozPJqRY5jA+OlxI35oPsMN?=
 =?iso-8859-1?Q?TJnPTOQ2xY2knmOfUDdOrJCYFfAnncmhfY9lRPwyoJ4YV8vpFp5G8WEX9V?=
 =?iso-8859-1?Q?kfj66XprwTMHKKr/AK9PGKgSArc6/fzmgHtkITgM9lqjU7/9A2mGXLdE4P?=
 =?iso-8859-1?Q?cw28j07aQLEHn1pwQYnV0xxbNS0ZkImC8XaFKPtHu/lbw3eOKDtz2rPlAV?=
 =?iso-8859-1?Q?zoo0WI68ZBUSWqM/VczkBKLpnxHHsUd9Wk+ETzKpI1e8Bk5FoDXH+Jcdry?=
 =?iso-8859-1?Q?yHH9TOwan0t0cemBG+oFgApnum/p8yq+f5Q9Xrar0y7l1BE6TBK9aFoSx6?=
 =?iso-8859-1?Q?zZ6JemAet6cLxOuCLfpB3mOOo4boN1hBjXIQtH7fN4qWrc6UwPBbvNIj2F?=
 =?iso-8859-1?Q?TdmRJxpClb0IQjCWA7gamDxlNkk7eUpb2qEkhAl3R+DRDLKXkf2VvImAJB?=
 =?iso-8859-1?Q?dJQV5NK3JuJpWTH2h4WY1ewLSKahJJP1A7K1zDWjF4xrL89DwOlClQlZY+?=
 =?iso-8859-1?Q?XqE1IMaXucB4ZheTL0JqmFxECYTRKviMYTwZmUbiAUK+nBiUHY3mV6ONuT?=
 =?iso-8859-1?Q?Jcl4YxldxFRSg5zHfYJp4rzcIrjhDzrsYilosgcgcV3Tkk2Us2cMrm7btm?=
 =?iso-8859-1?Q?M5xhq5WwS4Qfdvsxrnc839v2WZYgWrEH2DX5eZaRumySJD4ntw1ZeFn84v?=
 =?iso-8859-1?Q?nCjUbDfjULgmVMzO3O6j6vLIX6RWn2PBgPKhK40i3kDLSw5IraEJTCqCVK?=
 =?iso-8859-1?Q?ZGocsufHFxxaoV7FxiC3EqoT9UWmKAbkYD5WtOahbP0KXzRb/eTrvE1Zul?=
 =?iso-8859-1?Q?k28U4yL3vVnHjBSbPtJ1SXTi+isU431U+TGAEqecjYBFlG0oaafvP6hlwF?=
 =?iso-8859-1?Q?8J0d33dWSw/q5VmPMbvM/TXkFwGmXNAdmykW8+o/bFyPrmAGjOVtbEaELQ?=
 =?iso-8859-1?Q?C2deNAO7jP53ikRXOrW0s4Bl/39z1f1mmITqpMdgyU6SOyXWfaezrlNbcM?=
 =?iso-8859-1?Q?Czvj3vfeyO1KEibQ6HXNFXKNehOc7pAFyAjsWYgeI2JJBYb7K7ztvKF2V1?=
 =?iso-8859-1?Q?RQlxp9ncoigNjctTCgcsgVIUCI9q30xBjJXtUEirUCJ2/+hy8z/dIIsA4k?=
 =?iso-8859-1?Q?wRL/SdiuFlbVwfDnrLxuHXXvwKi9NvymKw7d7i0NjUgkCVF2Q5GgZUeJAs?=
 =?iso-8859-1?Q?z1fVUpBEV95B8539Vztm3NgXsmJ/pK1BEKXIoVaRVMq5osv8td/yjoyar9?=
 =?iso-8859-1?Q?9Zc+enjT4jAVNCvU4MNqeco9pCCcZmMymjumCYy4sAyIQyI+tdMEJZXGf8?=
 =?iso-8859-1?Q?zHtAT6wy5jyct5zHmd0vtThJQCBHJewCy7IAh122sTHSwrHKDSrd+Zgh3c?=
 =?iso-8859-1?Q?qXS7XUUghEIGWtoeMFRrVC5R2CecMvfQ=3D=3D?=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR03MB10456.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?Dv5Q5LQfCc5VDxfYRWKGQyeVvM70fNjC2G3S9TqfqM8df87Cq29FCt/FCi?=
 =?iso-8859-1?Q?6modDfVOk4jmEEFMlTg3ZESGpDbBV3uIrx9Y2ATngsTWSh+4nGFlSt/C4H?=
 =?iso-8859-1?Q?xheCtQOwroYF7rxdBDSQc4F1z0Jd7Ow9vi7yD2m3Oosr3utjh8FnfSHQQr?=
 =?iso-8859-1?Q?pyyY8S+PDzwuPBvnUIJU2SXxPFYfyDtdm6TBsX9Zg/kLVrsozK6IDt5DSC?=
 =?iso-8859-1?Q?TGjIeQ/5nsl3T6QbE4RoEWaft8xoPBdskemXZRFgOR7c4A0hnMffshx2aa?=
 =?iso-8859-1?Q?rw20N66vm3ynF/h8Z5DuBN1skaXh0Zc21avbt3QaJgdSmIF651xOyvc/dR?=
 =?iso-8859-1?Q?aOJ6CZHsbHwkb+ZaJJ1V2Bztxm/oaX8LnmwDIGkqBuX1cN5P4KJFd+nByD?=
 =?iso-8859-1?Q?nPIrRofCqEQWAw/c3gLxxl/35XC7KBEEOJrn0RGVffgFdAS7abaNivuQlK?=
 =?iso-8859-1?Q?9jfSeWyog1sjBPy+sOQEmWth8cqLazytlQ5Z8O21XjqGr7Ful+n98ijjfI?=
 =?iso-8859-1?Q?H9+NVCnUpdMVzkpncyWhhOSVECKNUywJpkvmgOSA+DRXPiuW+x5Zyy9YU6?=
 =?iso-8859-1?Q?ph3GDHoOhDXaJ2P9gFMZLP2oOlNWJ4wiNnRV8lSCyeLxSn3csxxBN4hUEu?=
 =?iso-8859-1?Q?+fpPx0NzTvL5BIFCqVgDlEy6f2lPOfiiQYDqq4lMeVtNAd3H1i03FIih3T?=
 =?iso-8859-1?Q?TqsX/ILd+NX+8z6ljqDL2XHXremQnt4JGwQl4NbMV6oQIeWCLaBRB59zfu?=
 =?iso-8859-1?Q?QoStncPPsOawuN8p0+he8YQKbbcwAZ/Kr5+dEdGm88boKqVZLKl226cl7W?=
 =?iso-8859-1?Q?giAYOljEBmj1maTfvtDlvMBrRdbIeDFluZBSHHZJlPE7eFR2p/0THKuXex?=
 =?iso-8859-1?Q?wvTUfXFIrP3eg6KlzFXLskyYVuj+ePvT2r5NshWB1mv6U/RRvUSvX/wDDe?=
 =?iso-8859-1?Q?BLgGRj8HfvAo1ztpFIc5miUnHo9TC6UoYagVUSPTSBhOt4ET1RVuSgW++m?=
 =?iso-8859-1?Q?SvWfrcIRZiwGCyJEAnsngMhE+/r8KmJvNmgXiqYcsbT3m6XepRnO9E9pOP?=
 =?iso-8859-1?Q?vwiNJR1oHPtqaC4kab43u9QQoDYC/yN4B0NGD8vBeWvKGIPSYyHq5jDv4H?=
 =?iso-8859-1?Q?4tWx68y/E5siIygT9msrU4/M/UhXg57fOyyVsMOXDewZ4Xk/+fQMIf1xXS?=
 =?iso-8859-1?Q?SDV72bAnNYuuI7n5skB3cS1pdm9nnppKtzvyCtvjXEfrY7PHkKqOkoHc5y?=
 =?iso-8859-1?Q?HYR9/Cl9Cs8OG2O8d03jeGhbCSI6gi19ktAKh+dzFaPFZYAO5kd6OkGokA?=
 =?iso-8859-1?Q?bD575TIyRM/Tj4410thhBGoiUrsFEvJL6fiZfSPWoMIWHr5Gb64iQSjtdU?=
 =?iso-8859-1?Q?HZV0uXhBAktvUsPRH2wGxvNri0+E5XMl2X2oekg6qtrMuMWANPCDexyafC?=
 =?iso-8859-1?Q?ASMP0NmLfRz/VT+OC1+1h183q2kdvGgXTfRq9/Q5W5I7OB9kwL2BMO0LcJ?=
 =?iso-8859-1?Q?3KMAjbtQJZeFeRi3A5T1X8A7i/vL0yQIdyNSTswsVSAC5jjbk3T258X7an?=
 =?iso-8859-1?Q?WOdfTaCGopGgvQXWJvzmZkKvZ1Bl8z+M4t5n0K7cWpzHyTLfqUBRKEx4rQ?=
 =?iso-8859-1?Q?zy4vLf2RedFawb6DL57QoGwzFHXAT/GOQcJUj0S/NdvQlwdAZZ5bBp+Q?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: GV1PR03MB10456.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eb966ea8-c62e-4423-f936-08dc80272a9b
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2024 21:34:48.6243
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 99f46Z6/Fv/vE+dDebm/hd94sImD91vZKN0T+E+vw4eYdzUkbwGtnd68Cs4bcESreh9C0BA8WgxaXoFCHpvRWFLItSXasvXmCk4Zdbx9hMs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR03MB9130
X-Proofpoint-GUID: ZPCxRbIvSlClq-HouQqH1dTzn6z34BB3
X-Proofpoint-ORIG-GUID: ZPCxRbIvSlClq-HouQqH1dTzn6z34BB3
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-29_16,2024-05-28_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 phishscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 malwarescore=0
 mlxlogscore=999 clxscore=1015 bulkscore=0 suspectscore=0 spamscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405170001 definitions=main-2405290153


Hi Julien,

Julien Grall <julien@xen.org> writes:

> Hi Volodymyr,
>
> Can you clarify whether this is intended for the next release cycle?

Well, I don't think that this patch should be committed ASAP if this is
what you are asking about.

> On 29/05/2024 21:43, Volodymyr Babchuk wrote:
>> Allow to provide TEE type for a Dom0less guest via "xen,tee"
>> property. Create appropriate nodes in the guests' device tree and
>> initialize tee subsystem for it.
>
> The new property needs to be documented in
> docs/misc/arm/device-tree/booting.txt.
>

Yes, missed that.

>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>> ---
>>   xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++++
>>   xen/arch/arm/include/asm/kernel.h |  3 ++
>>   2 files changed, 72 insertions(+)
>> diff --git a/xen/arch/arm/dom0less-build.c
>> b/xen/arch/arm/dom0less-build.c
>> index fb63ec6fd1..1ea3ecc45c 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -15,6 +15,7 @@
>>   #include <asm/domain_build.h>
>>   #include <asm/static-memory.h>
>>   #include <asm/static-shmem.h>
>> +#include <asm/tee/tee.h>
>>     bool __init is_dom0less_mode(void)
>>   {
>> @@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct kern=
el_info *kinfo)
>>   }
>>   #endif
>>   +#ifdef CONFIG_OPTEE
>> +static int __init make_optee_node(struct kernel_info *kinfo)
>
> Please introduce a callback in the TEE framework that will create the
> OPTEE node.

This is the reason why this is RFC. I wanted to discuss the right method
of doing this. "optee" node should reside in "/firmware/" node as per
device tree bindings. But "/firmware/" node can contain additional
entries, for example linux device tree bindings also define
"/firmware/sdei". So, probably correct solution is to implement function
"make_firmware_node()" in this file, which in turn will call TEE
framework.

But we are making assumption that all TEE implementation will have its
node inside "/firmware/". I am not 100% sure that this is correct. For
example I saw that Google Trusty uses "/trusty" node (directly inside
the DTS root). On other hand, it is not defined in dts bindings, as far
as I know.

>>   /*
>>    * Scan device tree properties for passthrough specific information.
>>    * Returns < 0 on error
>> @@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain *d=
, struct kernel_info *kinfo)
>>       if ( ret )
>>           goto err;
>>   +#ifdef CONFIG_OPTEE
>> +    if ( kinfo->tee_type =3D=3D XEN_DOMCTL_CONFIG_TEE_OPTEE)
>> +    {
>> +        ret =3D make_optee_node(kinfo);
>> +        if ( ret )
>> +            goto err;
>> +    }
>> +#endif
>> +
>>       /*
>>        * domain_handle_dtb_bootmodule has to be called before the rest o=
f
>>        * the device tree is generated because it depends on the value of
>> @@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
>>   {
>>       struct kernel_info kinfo =3D {};
>>       const char *dom0less_enhanced;
>> +#ifdef CONFIG_TEE
>> +    const char *tee;
>> +#endif
>>       int rc;
>>       u64 mem;
>>       u32 p2m_mem_mb;
>> @@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d,
>>       else if ( rc =3D=3D 0 && !strcmp(dom0less_enhanced, "no-xenstore")=
 )
>>           kinfo.dom0less_feature =3D DOM0LESS_ENHANCED_NO_XS;
>>   +#ifdef CONFIG_TEE
>
> I would rather prefer if this code is implemented in tee.c. We also...
>
>> +    rc =3D dt_property_read_string(node, "xen,tee", &tee);
>
> ... want to return an error if "xen,tee" exists because CONFIG_TEE is
> not set.
>
>> +    if ( rc =3D=3D -EILSEQ ||
>> +         rc =3D=3D -ENODATA ||
>> +         (rc =3D=3D 0 && !strcmp(tee, "none")) )
>> +    {
>> +        if ( !hardware_domain )
>
>
> I don't understand this check. Why would we require dom0 for OP-TEE?

OP-TEE is enabled for Dom0 unconditionally in create_dom0(void);

This is another topic I wanted to discuss, actually, Should we use the
same "xen,tee" for Dom0? In this case we might want to alter Dom0 DTB to
remove TEE entry if user wants it to be disabled.

> In any case we should avoid to hide a feature behind the user back. At
> minimum, we should print a message, but I would rather prefer a
> panic() because the user asks for a feature and we denied it.
>
>> +            kinfo.tee_type =3D XEN_DOMCTL_CONFIG_TEE_NONE;
> Why don't we have a else case? Are you assuming that tee_type is
> initialized to TEE_NONE (which luckily happens to be 0)? If so, why do
> we need the check above?

Yes, you are right, I'll rework this part.

[...]

>> +    if ( kinfo.tee_type )
>> +    {
>> +        rc =3D tee_domain_init(d, kinfo.tee_type);
>
> Can you explain why do you need to call tee_domain_init() rather than
> setting d_cfg.arch.tee_type just before domain_create() is called and
> rely on the latter to call the former?
>

Because I was not familiar with dom0less code good enough. Your proposal
is much better, I'll rework this.

--=20
WBR, Volodymyr=


From xen-devel-bounces@lists.xenproject.org Wed May 29 22:18:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 22:18:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732403.1138390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCRcQ-0005ep-Cp; Wed, 29 May 2024 22:18:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732403.1138390; Wed, 29 May 2024 22:18:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCRcQ-0005ei-A0; Wed, 29 May 2024 22:18:02 +0000
Received: by outflank-mailman (input) for mailman id 732403;
 Wed, 29 May 2024 22:18:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mTS/=NA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCRcP-0005ea-IA
 for xen-devel@lists.xenproject.org; Wed, 29 May 2024 22:18:01 +0000
Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com
 [2607:f8b0:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e108749-1e09-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 00:17:59 +0200 (CEST)
Received: by mail-ot1-x32b.google.com with SMTP id
 46e09a7af769-6f8ea563a46so163467a34.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 May 2024 15:17:59 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fcb486bc5sm36774151cf.62.2024.05.29.15.17.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 May 2024 15:17:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e108749-1e09-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717021078; x=1717625878; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xmP//5DMcNNnHbKeTdY3Oug/2yWmOU3CIASIGw7CGA4=;
        b=ngzm9aLyTWbGs6gdni0xL5BKphkt/OyKY1xZxbCQUQA4O5OAc2mA9Rz/gjP71kXnQ/
         jJNSNfWB/By4//u74qX3AB6VL5I1G1R3t095/Ty9GjhP/bWVt2ZrLOeSZZA7IpVYHFsx
         wqkpVlklCUHvfiVEryIoqMXLbJw99K6BnzL8w=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717021078; x=1717625878;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xmP//5DMcNNnHbKeTdY3Oug/2yWmOU3CIASIGw7CGA4=;
        b=u1UYs/U5GC346+iufvB/3xxPhWcVJU7GFtuJr5bPCHeGpOucF2vgGRY1Wi7qwX5Abb
         No6oWirtbFzr3xXPvW1JkmcX7sPWMD1lFGH8yqm313W7OCOe0hQFRPmeXk6WhSGTwiL4
         hI+QcV05zlQt8dc92Yh0IVYyxbsVwJW9rjkSx/QH2BzzikpcrnAVashKUKsQ/EDJSOCv
         tT5OxAgbQCX2dg+aiJovtQNO3hSZaUu5IDOeOussfyQS3HKC9wpzAnNgwlBkXBz6pIyl
         r79bUS4KSG2l80A9MfUbQCx+dra3dDCyBzJaK4BIHvChJAdmCMvCNfRZ3g7Xe9lP0rd1
         DOOg==
X-Forwarded-Encrypted: i=1; AJvYcCX4jEC9yagpSkxeQKNQ0oZ66f1wHGigYr9KFv8rQvmKKc9nzkWfHF7LXS772FpenRvS6uXcjPlSvEfqFTwCKGYIUoGOoVMOc/YBc6Sk/rw=
X-Gm-Message-State: AOJu0YxgWB10Qr8YVbYJkCdjZgGlA6iXEqhFyfBeZDxhWlgfWtysNYtE
	tzdMStoF53s4nyRlSjzR9GVSyFDoAOPc2qa9A5bJCdDczaxCKkxQzmyK2/ZTwg4=
X-Google-Smtp-Source: AGHT+IEuOmjUVXFf6i/HTma0O3mfyQ5tTcM7lvFvLU6hoieJ+/nV6jGK1RxEeZwkS2uSlwLzrzDk+g==
X-Received: by 2002:a05:6830:4490:b0:6f0:47a2:c1ab with SMTP id 46e09a7af769-6f90aeaf329mr543147a34.16.1717021077898;
        Wed, 29 May 2024 15:17:57 -0700 (PDT)
Message-ID: <34015fae-ddb1-44ec-b87d-d24683b34557@citrix.com>
Date: Wed, 29 May 2024 23:17:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 10/13] xen/bitops: Delete find_first_set_bit()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-11-andrew.cooper3@citrix.com>
 <a9b7ddb7-a485-4d0f-b2cc-7207093514c1@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <a9b7ddb7-a485-4d0f-b2cc-7207093514c1@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27/05/2024 1:58 pm, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
>> No more users.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
>
>

FYI, with the rearrangements I've done, this safely merges into and
supersedes patch 4.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed May 29 22:33:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 May 2024 22:33:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732412.1138409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCRr7-0000Xz-UH; Wed, 29 May 2024 22:33:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732412.1138409; Wed, 29 May 2024 22:33:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCRr7-0000Xr-OG; Wed, 29 May 2024 22:33:13 +0000
Received: by outflank-mailman (input) for mailman id 732412;
 Wed, 29 May 2024 22:33:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCRr5-0000XK-VA; Wed, 29 May 2024 22:33:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCRr5-0000hh-TX; Wed, 29 May 2024 22:33:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCRr5-00064V-JA; Wed, 29 May 2024 22:33:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCRr5-0004mk-Ih; Wed, 29 May 2024 22:33:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=G3tpKffLVFXAubnGo0TEMrd/jIa1+S16Pt47qIArXuc=; b=kxENJVv7l0rfvkYNsKvrA4QxL3
	aCniTHIb8dYr+AW+5hIpbXLfRfbGSg4dFKOo1Fv9Ktv47LV1TxepNI8fXh8XflaqN11PPnM02nZCX
	cHUFGu/JpU2QvXGP8jHfxQHE58xYxYZNE92k6n6dwegqZn1+TqX89QySW1gM9pUjkxfs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186183-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186183: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2bc462f645824a439879190ed19a231cb5b8034c
X-Osstest-Versions-That:
    xen=d27c2835e0005200d464b57156c76455d46f74bb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 May 2024 22:33:11 +0000

flight 186183 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186183/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186180
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186180
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186180
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186180
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186180
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186180
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2bc462f645824a439879190ed19a231cb5b8034c
baseline version:
 xen                  d27c2835e0005200d464b57156c76455d46f74bb

Last test of basis   186180  2024-05-29 05:58:09 Z    0 days
Testing same since   186183  2024-05-29 15:08:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Sergiy Kibrik <Sergiy_Kibrik@epam.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d27c2835e0..2bc462f645  2bc462f645824a439879190ed19a231cb5b8034c -> master


From xen-devel-bounces@lists.xenproject.org Thu May 30 00:26:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 00:26:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732425.1138425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCTcD-0005Zz-MW; Thu, 30 May 2024 00:25:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732425.1138425; Thu, 30 May 2024 00:25:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCTcD-0005Zs-Jj; Thu, 30 May 2024 00:25:57 +0000
Received: by outflank-mailman (input) for mailman id 732425;
 Thu, 30 May 2024 00:25:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KshN=NB=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1sCTcB-0005Zk-BB
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 00:25:55 +0000
Received: from wfhigh5-smtp.messagingengine.com
 (wfhigh5-smtp.messagingengine.com [64.147.123.156])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2a75ca94-1e1b-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 02:25:51 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.west.internal (Postfix) with ESMTP id 057D51800134;
 Wed, 29 May 2024 20:25:47 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Wed, 29 May 2024 20:25:48 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 29 May 2024 20:25:45 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a75ca94-1e1b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm1; t=1717028747;
	 x=1717115147; bh=08citzDiN2uS6821pFoEZp+AjfcQSFyt3aPtXMU2tXw=; b=
	qefYF7RX/Ms780axJo1mDqebme6G2o8Z0mCrp2+2ZGYzqaCme939fMdaFQiHl+RT
	I3o9WJA+ZY8EqU8frsfll6wW7RrO96Xx6A2IKftXsxb9UpVtI0QmIAj9cwi3Y61e
	NiNN1VsY7H41qDQF9uouRmv4oDSD/ik92iyu5iBiOlMQLsm+cRN14pQvEu8b9J/V
	L8ouzZBvA/RIsB20wYasI3bp0miyw45T6D0X5wtCijp29hKMguGYWbkzYDd3sBNa
	35qUW5oa9r6ubGdEyVzrKP8ybKE4gWvpc4hzC31V0vxaKo0oECSm/CeRfnhapJP/
	sZSZzVAmTPFr1bFhP3YUSg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1717028747; x=1717115147; bh=08citzDiN2uS6821pFoEZp+AjfcQ
	SFyt3aPtXMU2tXw=; b=PeskwC0Ir1aqrpRDkZJNLZh6gb/zbmSL3tvGRnk6k8qN
	0S/5DYB/Jw5+cTmUteIbNMNsjjAE54G57Z7eGYMfLg2tYa94l7qS81d8Qc4QuoYN
	uB7lCXzcuWfBlbOKC9PUpw3w53D5o5aQ+76uXAtLFnKlbdG8a3jFk7q5aAtELbp1
	RUgAxZ/O1QW78fBRVnnjwo5U4ML+U6Hd14ij8twkEhO5BzNCu7zwXR0HfWbXiQfZ
	I9nISROI7DPXjoc4Wq3c3S4T07IYQl9bYc1jisUfMx0VV/IobsUR+QRS1LAl5yX1
	rfX3o/4gjSUlRrC8qetBhgYRCibAeSyziNYuYvLGHw==
X-ME-Sender: <xms:i8dXZhwK7BO3NVupsqVWMtWwxyWz9ovOS0eaVu84PTxXyyFV3enMEQ>
    <xme:i8dXZhSSW26LVQeOYNMoTUJ4cAVSX4o4KYgAqn59j4gXqF0fXCwXSLHlvjI5JYlFW
    q4OGfZ-CtWUQQ>
X-ME-Received: <xmr:i8dXZrWCCzxXFE7WR-i22HbJB1swEFgXl7qfk9CAwPbI0DPhaBCM71lwEFc>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekvddgfedvucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:i8dXZjixFKKx8qooXas9d9oUE7UGW0YQWo2m5szSCz_dJgowqvo9vw>
    <xmx:i8dXZjCJ68Ud51I7q5xNnZd5dHX0icpDUOrpAyPCOhi4LlEmRdygYg>
    <xmx:i8dXZsIlsXudyJwzvmWl8JPOVRy3bpweNXzg6YK3kiXs_PWkPV8dEg>
    <xmx:i8dXZiDxzk8rT0heeiwjcxThyaMffbwXRsUi5psUNFcKdkzyyg6llw>
    <xmx:i8dXZu7cytMgdmflU3fhnip21oSqJUmD7mlaCI5S4j9Y52Sf86XDnl9c>
Feedback-ID: i1568416f:Fastmail
Date: Thu, 30 May 2024 02:25:41 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for
 HW jobs
Message-ID: <ZlfHh_blx1i0eE7O@mail-itl>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
 <20240529141945.41669-2-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="2bIh9cXtMpJeLTbF"
Content-Disposition: inline
In-Reply-To: <20240529141945.41669-2-andrew.cooper3@citrix.com>


--2bIh9cXtMpJeLTbF
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Thu, 30 May 2024 02:25:41 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for
 HW jobs

On Wed, May 29, 2024 at 03:19:43PM +0100, Andrew Cooper wrote:
> This restriction doesn't provide any security because anyone with suitable
> permissions on the HW runners can bypass it with this local patch.
>=20
> Requiring branches to be protected hampers usability of transient testing
> branches (specifically, can't delete branches except via the Gitlab UI).
>
> Drop the requirement.
>=20
> Fixes: 746774cd1786 ("automation: introduce a dom0less test run on Xilinx=
 hardware")
> Fixes: 0ab316e7e15f ("automation: add a smoke and suspend test on an Alde=
r Lake system")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Runners used to be set to run only on protected branches. I think it
isn't the case anymore from what I see, but it needs checking (I don't
see specific settings in all the projects). If it were still the case,
removing variable check would result in jobs forever pending.

Other than that, I'm okay with this change, since the hw runners are
added only to select projects. You can interpret this as Acked-by, if
you verify if indeed runners are not limited to protected branches only.

I will need to adjust setting of my project, to set "QUBES_JOBS" only
to some branches - I used to use branch protection rules as a proxy to
selecting on which branch to run hw tests...

> ---
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Fixes because this wants backporting, but it also needs acks from both Ma=
rek
> and Stefano as the owners of the hardware in question.
> ---
>  automation/gitlab-ci/test.yaml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>=20
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.y=
aml
> index ad249fa0a5d9..efd3ad46f08e 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -92,7 +92,7 @@
>      when: always
>    only:
>      variables:
> -      - $XILINX_JOBS =3D=3D "true" && $CI_COMMIT_REF_PROTECTED =3D=3D "t=
rue"
> +      - $XILINX_JOBS =3D=3D "true"
>    tags:
>      - xilinx
> =20
> @@ -112,7 +112,7 @@
>      when: always
>    only:
>      variables:
> -      - $QUBES_JOBS =3D=3D "true" && $CI_COMMIT_REF_PROTECTED =3D=3D "tr=
ue"
> +      - $QUBES_JOBS =3D=3D "true"
>    tags:
>      - qubes-hw2
> =20
> --=20
> 2.30.2
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--2bIh9cXtMpJeLTbF
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZXx4cACgkQ24/THMrX
1yx+Nwf+MYUlCbZHeAuvtaVkSdCOEbvMNu31TleFCnDnHrxQkX8Uj/nzONqkz42I
Wk/zzS3fbkHi1lrTWjKJYmxSZ+nukeYbfDnxxIaT4vygoDvBelFF82Oes6xqES01
eScEzHQPGz9UIK07k5gpe7LrZyNIqKEGfnILd77L34siUufOVSetydRHviJXSOse
+z9AuBL8SA1GsTNxfKjRyRcFBV/hTjvQV2w+pKTfwwii5AfVw08souYMUOhrhFgS
vgt10SiQ0aAgQBmukjsLHDo0zpDcwucTYSBFn1G4X5DLK9wsTQ64WDxOhnDTo3BB
KuZwn9I0bECW8vkKeGHP8TeIZ/C/Dw==
=vAra
-----END PGP SIGNATURE-----

--2bIh9cXtMpJeLTbF--


From xen-devel-bounces@lists.xenproject.org Thu May 30 02:30:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 02:30:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732436.1138435 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCVYA-0001GQ-2a; Thu, 30 May 2024 02:29:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732436.1138435; Thu, 30 May 2024 02:29:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCVY9-0001GJ-WF; Thu, 30 May 2024 02:29:54 +0000
Received: by outflank-mailman (input) for mailman id 732436;
 Thu, 30 May 2024 02:29:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVY9-0001G9-CT; Thu, 30 May 2024 02:29:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVY9-0004Yq-1h; Thu, 30 May 2024 02:29:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVY8-0006o9-M4; Thu, 30 May 2024 02:29:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVY8-0003qZ-Lc; Thu, 30 May 2024 02:29:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BdMY+ghUta+KZ9MqOWP+KW4CI0BSTrK9gqnZMQZl47A=; b=6jRokYR4+hFd2KtliOf/JyTWbR
	FD1+4Xe3Ci4EJc9ugCSS62u/qCQx5ntfdfBQIjo3r1AOIwxuDFNVDVC9/W3/lxV8uuo0oJRZXWhpZ
	H9cS8WNAyADfv2atzMBXnWd276/ijfjtTEIStjOv4F/fVEIMI/JSsO7rJd0Bzdd4VZ1I=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186187-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186187: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=79655e27686031326eda3ea290225db6aea3b53f
X-Osstest-Versions-That:
    ovmf=55f8bddade205c9cbe3583d5d31d0048cdf26ed4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 02:29:52 +0000

flight 186187 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186187/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 79655e27686031326eda3ea290225db6aea3b53f
baseline version:
 ovmf                 55f8bddade205c9cbe3583d5d31d0048cdf26ed4

Last test of basis   186178  2024-05-29 00:41:09 Z    1 days
Testing same since   186187  2024-05-30 00:42:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   55f8bddade..79655e2768  79655e27686031326eda3ea290225db6aea3b53f -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 02:36:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 02:36:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732444.1138446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCVe7-0002qb-RA; Thu, 30 May 2024 02:36:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732444.1138446; Thu, 30 May 2024 02:36:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCVe7-0002qU-NS; Thu, 30 May 2024 02:36:03 +0000
Received: by outflank-mailman (input) for mailman id 732444;
 Thu, 30 May 2024 02:36:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVe6-0002qK-NP; Thu, 30 May 2024 02:36:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVe6-0004eq-Ij; Thu, 30 May 2024 02:36:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVe6-00071G-83; Thu, 30 May 2024 02:36:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCVe6-0007bM-7X; Thu, 30 May 2024 02:36:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OaSmiby+iJXVpubQOBgeSTqQHKduviMYWbwgIZOzyi0=; b=wnlaFfQhS1FlmGCKU3aJSgRFzn
	lF2H1Z0GpqNwC275hms3D5rrovn6adzMAmRJHiduLhjtoEeOnCkXLFgjBSMBwT2FYiPtdwnQKBZxA
	KN8tIcdZi94PNQ6s3ss9RFcFxVwqZEdHlxJ8nxQZIrCJZ8BKSGavtytJYskdZLA/gDW4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186185-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186185: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-amd64-xsm:xen-build:fail:regression
    linux-linus:build-amd64:xen-build:fail:regression
    linux-linus:build-i386:xen-build:fail:regression
    linux-linus:build-i386-xsm:xen-build:fail:regression
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:build-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:build-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-bios:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-uefi:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4a4be1ad3a6efea16c56615f31117590fd881358
X-Osstest-Versions-That:
    linux=e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 02:36:02 +0000

flight 186185 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186185/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 186174
 build-amd64                   6 xen-build                fail REGR. vs. 186174
 build-i386                    6 xen-build                fail REGR. vs. 186174
 build-i386-xsm                6 xen-build                fail REGR. vs. 186174
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 186174
 test-armhf-armhf-xl-multivcpu  8 xen-boot                fail REGR. vs. 186174

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-vhd       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-raw       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine      1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-bios  1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186174
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4a4be1ad3a6efea16c56615f31117590fd881358
baseline version:
 linux                e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc

Last test of basis   186174  2024-05-28 18:10:31 Z    1 days
Testing same since   186185  2024-05-29 17:42:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dominique Martinet <asmadeus@codewreck.org>
  Herbert Xu <herbert@gondor.apana.org.au>
  Linus Torvalds <torvalds@linux-foundation.org>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                blocked 
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-amd64-examine                                     blocked 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 blocked 
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      blocked 
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      blocked 
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 4a4be1ad3a6efea16c56615f31117590fd881358
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:39:34 2024 -0700

    Revert "vfs: Delete the associated dentry when deleting a file"
    
    This reverts commit 681ce8623567ba7e7333908e9826b77145312dda.
    
    We gave it a try, but it turns out the kernel test robot did in fact
    find performance regressions for it, so we'll have to look at the more
    involved alternative fixes for Yafang Shao's Elasticsearch load issue.
    
    There were several alternatives discussed, they just weren't as simple
    as this first attempt.
    
    The report is of a -7.4% regression of filebench.sum_operations/s, which
    appears significant enough to trigger my "this patch may get reverted if
    somebody finds a performance regression on some other load" rule.
    
    So it's still the case that we should end up deleting dentries more
    aggressively - or just be better at pruning them later - but it needs a
    bit more finesse than this simple thing.
    
    Link: https://lore.kernel.org/all/202405291318.4dfbb352-oliver.sang@intel.com/
    Cc: Yafang Shao <laoar.shao@gmail.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 397a83ab978553ca2970ad1ccdbac0cdc732efd9
Merge: db163660b02a c898afdc1564
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:25:15 2024 -0700

    Merge tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux
    
    Pull 9p fixes from Dominique Martinet:
     "Two fixes headed to stable trees:
    
       - a trace event was dumping uninitialized values
    
       - a missing lock that was thought to have exclusive access, and it
         turned out not to"
    
    * tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux:
      9p: add missing locking around taking dentry fid list
      net/9p: fix uninit-value in p9_client_rpc()

commit db163660b02abbffebfad1bcd6dbce1201c72731
Merge: e0cce98fe279 67ec8cdf2997
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:12:58 2024 -0700

    Merge tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
    
    Pull crypto fix from Herbert Xu:
     "This fixes a new run-time warning triggered by tpm"
    
    * tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      hwrng: core - Remove add_early_randomness

commit 67ec8cdf29971677b2fb4b6d92871eb5d5e95597
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date:   Wed May 22 13:37:54 2024 +0800

    hwrng: core - Remove add_early_randomness
    
    A potential deadlock was reported with the config file at
    
    https://web.archive.org/web/20240522052129/https://0x0.st/XPN_.txt
    
    In this particular configuration, the deadlock doesn't exist because
    the warning triggered at a point before modules were even available.
    However, the deadlock can be real because any module loaded would
    invoke async_synchronize_full.
    
    The issue is spurious for software crypto algorithms which aren't
    themselves involved in async probing.  However, it would be hard to
    avoid for a PCI crypto driver using async probing.
    
    In this particular call trace, the problem is easily avoided because
    the only reason the module is being requested during probing is the
    add_early_randomness call in the hwrng core.  This feature is
    vestigial since there is now a kernel thread dedicated to doing
    exactly this.
    
    So remove add_early_randomness as it is no longer needed.
    
    Reported-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Reported-by: Eric Biggers <ebiggers@kernel.org>
    Fixes: 1b6d7f9eb150 ("tpm: add session encryption protection to tpm2_get_random()")
    Link: https://lore.kernel.org/r/119dc5ed-f159-41be-9dda-1a056f29888d@notapiano/
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
    Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
    Tested-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

commit c898afdc15645efb555acb6d85b484eb40a45409
Author: Dominique Martinet <asmadeus@codewreck.org>
Date:   Tue May 21 21:13:36 2024 +0900

    9p: add missing locking around taking dentry fid list
    
    Fix a use-after-free on dentry's d_fsdata fid list when a thread
    looks up a fid through dentry while another thread unlinks it:
    
    UAF thread:
    refcount_t: addition on 0; use-after-free.
     p9_fid_get linux/./include/net/9p/client.h:262
     v9fs_fid_find+0x236/0x280 linux/fs/9p/fid.c:129
     v9fs_fid_lookup_with_uid linux/fs/9p/fid.c:181
     v9fs_fid_lookup+0xbf/0xc20 linux/fs/9p/fid.c:314
     v9fs_vfs_getattr_dotl+0xf9/0x360 linux/fs/9p/vfs_inode_dotl.c:400
     vfs_statx+0xdd/0x4d0 linux/fs/stat.c:248
    
    Freed by:
     p9_fid_destroy (inlined)
     p9_client_clunk+0xb0/0xe0 linux/net/9p/client.c:1456
     p9_fid_put linux/./include/net/9p/client.h:278
     v9fs_dentry_release+0xb5/0x140 linux/fs/9p/vfs_dentry.c:55
     v9fs_remove+0x38f/0x620 linux/fs/9p/vfs_inode.c:518
     vfs_unlink+0x29a/0x810 linux/fs/namei.c:4335
    
    The problem is that d_fsdata was not accessed under d_lock, because
    d_release() normally is only called once the dentry is otherwise no
    longer accessible but since we also call it explicitly in v9fs_remove
    that lock is required:
    move the hlist out of the dentry under lock then unref its fids once
    they are no longer accessible.
    
    Fixes: 154372e67d40 ("fs/9p: fix create-unlink-getattr idiom")
    Cc: stable@vger.kernel.org
    Reported-by: Meysam Firouzi
    Reported-by: Amirmohammad Eftekhar
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Message-ID: <20240521122947.1080227-1-asmadeus@codewreck.org>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>

commit 25460d6f39024cc3b8241b14c7ccf0d6f11a736a
Author: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
Date:   Mon Apr 8 07:10:39 2024 -0700

    net/9p: fix uninit-value in p9_client_rpc()
    
    Syzbot with the help of KMSAN reported the following error:
    
    BUG: KMSAN: uninit-value in trace_9p_client_res include/trace/events/9p.h:146 [inline]
    BUG: KMSAN: uninit-value in p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     trace_9p_client_res include/trace/events/9p.h:146 [inline]
     p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    Uninit was created at:
     __alloc_pages+0x9d6/0xe70 mm/page_alloc.c:4598
     __alloc_pages_node include/linux/gfp.h:238 [inline]
     alloc_pages_node include/linux/gfp.h:261 [inline]
     alloc_slab_page mm/slub.c:2175 [inline]
     allocate_slab mm/slub.c:2338 [inline]
     new_slab+0x2de/0x1400 mm/slub.c:2391
     ___slab_alloc+0x1184/0x33d0 mm/slub.c:3525
     __slab_alloc mm/slub.c:3610 [inline]
     __slab_alloc_node mm/slub.c:3663 [inline]
     slab_alloc_node mm/slub.c:3835 [inline]
     kmem_cache_alloc+0x6d3/0xbe0 mm/slub.c:3852
     p9_tag_alloc net/9p/client.c:278 [inline]
     p9_client_prepare_req+0x20a/0x1770 net/9p/client.c:641
     p9_client_rpc+0x27e/0x1340 net/9p/client.c:688
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    If p9_check_errors() fails early in p9_client_rpc(), req->rc.tag
    will not be properly initialized. However, trace_9p_client_res()
    ends up trying to print it out anyway before p9_client_rpc()
    finishes.
    
    Fix this issue by assigning default values to p9_fcall fields
    such as 'tag' and (just in case KMSAN unearths something new) 'id'
    during the tag allocation stage.
    
    Reported-and-tested-by: syzbot+ff14db38f56329ef68df@syzkaller.appspotmail.com
    Fixes: 348b59012e5c ("net/9p: Convert net/9p protocol dumps to tracepoints")
    Signed-off-by: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Cc: stable@vger.kernel.org
    Message-ID: <20240408141039.30428-1-n.zhandarovich@fintech.ru>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>


From xen-devel-bounces@lists.xenproject.org Thu May 30 03:59:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 03:59:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732457.1138456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCWwO-0003DM-F9; Thu, 30 May 2024 03:59:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732457.1138456; Thu, 30 May 2024 03:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCWwO-0003DF-BV; Thu, 30 May 2024 03:59:00 +0000
Received: by outflank-mailman (input) for mailman id 732457;
 Thu, 30 May 2024 03:58:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCWwN-0003D5-Gr; Thu, 30 May 2024 03:58:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCWwN-0005xp-FP; Thu, 30 May 2024 03:58:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCWwN-0001PO-6E; Thu, 30 May 2024 03:58:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCWwN-0006Oh-5d; Thu, 30 May 2024 03:58:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uuzLXX9HztuseY2wql01v2MXzMHOTqG2pdABBbsIVyg=; b=63TthiBVovfFl1sBTDCujZvWN1
	OzAfAWrBq1PWPL4jzew3ACzndckaOzOBpXeS7gYbbPUPIwaSdf9muP6uewyxXeBV3MjxpVVAPRSj3
	rasAcPbRK6sRHphKWXePXRGNomtEEld9pzd/h/vF9kvF23ePDcSuAhCkZ7eirY/dr+cU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186188-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186188: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=30b6d08e27c767ba9756a244099d73c826abcc8d
X-Osstest-Versions-That:
    ovmf=79655e27686031326eda3ea290225db6aea3b53f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 03:58:59 +0000

flight 186188 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186188/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 30b6d08e27c767ba9756a244099d73c826abcc8d
baseline version:
 ovmf                 79655e27686031326eda3ea290225db6aea3b53f

Last test of basis   186187  2024-05-30 00:42:59 Z    0 days
Testing same since   186188  2024-05-30 02:30:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Zhiguang Liu <zhiguang.liu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   79655e2768..30b6d08e27  30b6d08e27c767ba9756a244099d73c826abcc8d -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 07:04:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 07:04:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732482.1138467 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCZpt-0006dy-Ag; Thu, 30 May 2024 07:04:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732482.1138467; Thu, 30 May 2024 07:04:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCZpt-0006dr-5j; Thu, 30 May 2024 07:04:29 +0000
Received: by outflank-mailman (input) for mailman id 732482;
 Thu, 30 May 2024 07:04:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCZpr-0006dh-Sd; Thu, 30 May 2024 07:04:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCZpr-00017j-Qk; Thu, 30 May 2024 07:04:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCZpr-0001Dc-AO; Thu, 30 May 2024 07:04:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCZpr-0000Ys-9t; Thu, 30 May 2024 07:04:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NzT/L3YNx7mJPsPSPoclm3ulW4yQgJh/3hS9Akuglf8=; b=bGqpKwy7lzbNLqsLyFNxz3TI/R
	VSlssCAguHuY+U5hRHrjVthDWUQTUkf+DVw9BAEOJiqyayuoJQzXhDfA5xGn7E6h2x7Ru2F9JRWV1
	ZuXOYCQkmDwcEix7Cm9atnI4IZAkmbjiBQpRehfoYACp4OUaZQvLcwH61GJshkcM8vP8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186191-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186191: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=843f2d0964bd0444fa6bdbb1ee79dc838cfa4452
X-Osstest-Versions-That:
    ovmf=30b6d08e27c767ba9756a244099d73c826abcc8d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 07:04:27 +0000

flight 186191 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186191/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 843f2d0964bd0444fa6bdbb1ee79dc838cfa4452
baseline version:
 ovmf                 30b6d08e27c767ba9756a244099d73c826abcc8d

Last test of basis   186188  2024-05-30 02:30:18 Z    0 days
Testing same since   186191  2024-05-30 04:41:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   30b6d08e27..843f2d0964  843f2d0964bd0444fa6bdbb1ee79dc838cfa4452 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 07:53:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 07:53:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732498.1138476 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCabb-0003iD-GW; Thu, 30 May 2024 07:53:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732498.1138476; Thu, 30 May 2024 07:53:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCabb-0003i6-Dg; Thu, 30 May 2024 07:53:47 +0000
Received: by outflank-mailman (input) for mailman id 732498;
 Thu, 30 May 2024 07:53:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCabZ-0003hy-Pb
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 07:53:45 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bcf37c54-1e59-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 09:53:44 +0200 (CEST)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-354ba5663c9so652058f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 00:53:44 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35579d7db97sm16555229f8f.24.2024.05.30.00.53.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 00:53:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcf37c54-1e59-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717055623; x=1717660423; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=li5PB+SVoxtb4ME8F/9PRui3RwKWJvCvwTggjmT/4Xc=;
        b=T8W25XZv0yVjrVjpsHgbM7eKCwedRQWNsemubFoFA/0mU0aWF7g3e6rqzBS3DSZ1kL
         HryDspJ6ZpXHDsLJ6w0Vwc6ejupzG9Mbo/zGkpXYnE6M0kkY9RWHV1VKciyob4kk29eg
         8aj0UBLNGWGhLYtyNXVVbQpUXS3X0foosXoAg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717055623; x=1717660423;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=li5PB+SVoxtb4ME8F/9PRui3RwKWJvCvwTggjmT/4Xc=;
        b=SlQntR7xK9eNaRJ3bXqgX4tlyFXXfgZXAEbb7yOrNJtaqyfUucrU7vEqYujhgb2GOc
         4p7hOvhcu+RF/Tip2mvMbyWmDHqCEC/H4Rz2OuCUxye1l3D53jyUUXuXjIX1e0wt8vC+
         G/vKNDuGOibi3+GyGTI58DwjsERTCSBQjMQqNC9ClYXyh72K2ypccXDb9Mxfk1KM7wOi
         T4Ddm5LyJ87lOTaY03/2UupqUIOJGq/We1UkhVKdbHODqijTwcWNbiBGxyl6YFjJ2MOz
         WpEJUrFLW4xcTvaZRyytQJeOQobULNi2grrrsvX5+YdiTt5aORjIiT0oXFtTosBe6Coi
         174A==
X-Gm-Message-State: AOJu0Ywnrf1ke/XldWHvoqW3zBR3ahssIbvlosjvLo0B9tUdV7L7dnr+
	acm1w6XpUTl8GjlfZz3UjQWdIC0t83XhbeVewih3OxtNmUc5sM1roT7Cy9LhY8LrEVyZdHDTeYl
	p
X-Google-Smtp-Source: AGHT+IEYFUcZuJopx1tsUk/xX1jaKtL8IvQTQ4cwdFMDlC5+LAT/CxxX6AETi7v2js7CfgUTXXSspg==
X-Received: by 2002:a05:6000:1751:b0:354:f622:fc5b with SMTP id ffacd0b85a97d-35dc00908fbmr726685f8f.23.1717055623085;
        Thu, 30 May 2024 00:53:43 -0700 (PDT)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH] x86/hvm: allow XENMEM_machine_memory_map
Date: Thu, 30 May 2024 09:53:18 +0200
Message-ID: <20240530075318.67491-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

For HVM based control domains XENMEM_machine_memory_map must be available so
that the `e820_host` xl.cfg option can be used.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/hvm/hypercall.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 14679dd82971..7fb3136f0c7c 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -25,7 +25,6 @@ long hvm_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 
     switch ( cmd & MEMOP_CMD_MASK )
     {
-    case XENMEM_machine_memory_map:
     case XENMEM_machphys_mapping:
         return -ENOSYS;
     }
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu May 30 07:59:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 07:59:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732504.1138487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCagz-0004K3-4E; Thu, 30 May 2024 07:59:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732504.1138487; Thu, 30 May 2024 07:59:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCagy-0004Jw-Vg; Thu, 30 May 2024 07:59:20 +0000
Received: by outflank-mailman (input) for mailman id 732504;
 Thu, 30 May 2024 07:59:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCagy-0004Jq-Fg
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 07:59:20 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCagy-0001yV-6y; Thu, 30 May 2024 07:59:20 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCagy-0006Gv-0E; Thu, 30 May 2024 07:59:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=2JdIZsVrMP+NTjDl00GDom5j3GLWLAtOS7VtkMSJoDA=; b=VP3kkuSW78lkNBGtobjBR//BdR
	4NKllrx1EExRvTOwv8IV8eolnzUg4o1rlcQYz9O1uyX/fcgvkn9j5dNlqCvj4EgQC/+V0GjHj7ISI
	eePCrsuyFY1UzCyIwaANvC/LaL6LZaEuyGyXoGAoBlL+21JMWAmSQkXcgGPpl3kbujJE=;
Message-ID: <eff6e2e6-5c58-4b19-99f4-3754f655210d@xen.org>
Date: Thu, 30 May 2024 08:59:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arm: dom0less: add TEE support
Content-Language: en-GB
To: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <87bk4oxpxk.fsf@epam.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 29/05/2024 22:34, Volodymyr Babchuk wrote:
> 
> Hi Julien,

Hi Volodymyr,

> 
> Julien Grall <julien@xen.org> writes:
> 
>> Hi Volodymyr,
>>
>> Can you clarify whether this is intended for the next release cycle?
> 
> Well, I don't think that this patch should be committed ASAP if this is
> what you are asking about.
> 
>> On 29/05/2024 21:43, Volodymyr Babchuk wrote:
>>> Allow to provide TEE type for a Dom0less guest via "xen,tee"
>>> property. Create appropriate nodes in the guests' device tree and
>>> initialize tee subsystem for it.
>>
>> The new property needs to be documented in
>> docs/misc/arm/device-tree/booting.txt.
>>
> 
> Yes, missed that.
> 
>>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>> ---
>>>    xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++++
>>>    xen/arch/arm/include/asm/kernel.h |  3 ++
>>>    2 files changed, 72 insertions(+)
>>> diff --git a/xen/arch/arm/dom0less-build.c
>>> b/xen/arch/arm/dom0less-build.c
>>> index fb63ec6fd1..1ea3ecc45c 100644
>>> --- a/xen/arch/arm/dom0less-build.c
>>> +++ b/xen/arch/arm/dom0less-build.c
>>> @@ -15,6 +15,7 @@
>>>    #include <asm/domain_build.h>
>>>    #include <asm/static-memory.h>
>>>    #include <asm/static-shmem.h>
>>> +#include <asm/tee/tee.h>
>>>      bool __init is_dom0less_mode(void)
>>>    {
>>> @@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
>>>    }
>>>    #endif
>>>    +#ifdef CONFIG_OPTEE
>>> +static int __init make_optee_node(struct kernel_info *kinfo)
>>
>> Please introduce a callback in the TEE framework that will create the
>> OPTEE node.
> 
> This is the reason why this is RFC.

If this is meant an RFC, then I would recommend to tag your series with 
RFC. Similarly...


> I wanted to discuss the right method
> of doing this.

... if you have any open questions. Then please write them down after 
the "---" (so they are not committed). So this is not a guessing game 
for the reviewer.

For instance, if you hadn't asked the question, I wouldn't have realized 
you were not entirely happy with your solution. To me it looked fine 
because this is self-contained in an OP-TEE specific function.

> "optee" node should reside in "/firmware/" node as per
> device tree bindings. But "/firmware/" node can contain additional
> entries, for example linux device tree bindings also define
> "/firmware/sdei". So, probably correct solution is to implement function
> "make_firmware_node()" in this file, which in turn will call TEE
> framework.

Longer term possibly. But at the moment, we have no implementation of 
the "sdei" node and I am not aware of any future plan. So I don't think 
it is necessary to split the work in two functions.

> 
> But we are making assumption that all TEE implementation will have its
> node inside "/firmware/". I am not 100% sure that this is correct. For
> example I saw that Google Trusty uses "/trusty" node (directly inside
> the DTS root). On other hand, it is not defined in dts bindings, as far
> as I know.

TBH, if there is no official binding documentation, then Xen cannot 
sensibly create those nodes because they are not "stable". So the first 
step would be to have official binding.


Bertrand, I couldn't find any documentation for the FFA binding. Do you 
know if they will be created under /firmware?

> 
>>>    /*
>>>     * Scan device tree properties for passthrough specific information.
>>>     * Returns < 0 on error
>>> @@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
>>>        if ( ret )
>>>            goto err;
>>>    +#ifdef CONFIG_OPTEE
>>> +    if ( kinfo->tee_type == XEN_DOMCTL_CONFIG_TEE_OPTEE)
>>> +    {
>>> +        ret = make_optee_node(kinfo);
>>> +        if ( ret )
>>> +            goto err;
>>> +    }
>>> +#endif
>>> +
>>>        /*
>>>         * domain_handle_dtb_bootmodule has to be called before the rest of
>>>         * the device tree is generated because it depends on the value of
>>> @@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
>>>    {
>>>        struct kernel_info kinfo = {};
>>>        const char *dom0less_enhanced;
>>> +#ifdef CONFIG_TEE
>>> +    const char *tee;
>>> +#endif
>>>        int rc;
>>>        u64 mem;
>>>        u32 p2m_mem_mb;
>>> @@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d,
>>>        else if ( rc == 0 && !strcmp(dom0less_enhanced, "no-xenstore") )
>>>            kinfo.dom0less_feature = DOM0LESS_ENHANCED_NO_XS;
>>>    +#ifdef CONFIG_TEE
>>
>> I would rather prefer if this code is implemented in tee.c. We also...
>>
>>> +    rc = dt_property_read_string(node, "xen,tee", &tee);
>>
>> ... want to return an error if "xen,tee" exists because CONFIG_TEE is
>> not set.
>>
>>> +    if ( rc == -EILSEQ ||
>>> +         rc == -ENODATA ||
>>> +         (rc == 0 && !strcmp(tee, "none")) )
>>> +    {
>>> +        if ( !hardware_domain )
>>
>>
>> I don't understand this check. Why would we require dom0 for OP-TEE?
> 
> OP-TEE is enabled for Dom0 unconditionally in create_dom0(void);

I am sorry but this still doesn't make sense. AFAICT, this path is only 
used by domU. In some dom0less setup, we may not have dom0 at all. So 
why do you want to prevent OP-TEE for such case?

Or are you intending to check that "d" is not the hardware domain? If 
so, you have the wrong check (you want to check is_hardware_domain(d) 
and AFAIK this path is not called for dom0.

> 
> This is another topic I wanted to discuss, actually, Should we use the
> same "xen,tee" for Dom0? In this case we might want to alter Dom0 DTB to
> remove TEE entry if user wants it to be disabled.
Is there any existing use case to disable OP-TEE in dom0? I am asking 
because removing the nodes will make the code a bit more complicated. So 
if there is no need, then my preference is to not do it.

> 
>> In any case we should avoid to hide a feature behind the user back. At
>> minimum, we should print a message, but I would rather prefer a
>> panic() because the user asks for a feature and we denied it.
>>
>>> +            kinfo.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
>> Why don't we have a else case? Are you assuming that tee_type is
>> initialized to TEE_NONE (which luckily happens to be 0)? If so, why do
>> we need the check above?
> 
> Yes, you are right, I'll rework this part.
> 
> [...]
> 
>>> +    if ( kinfo.tee_type )
>>> +    {
>>> +        rc = tee_domain_init(d, kinfo.tee_type);
>>
>> Can you explain why do you need to call tee_domain_init() rather than
>> setting d_cfg.arch.tee_type just before domain_create() is called and
>> rely on the latter to call the former?
>>
> 
> Because I was not familiar with dom0less code good enough. 

The only difference between a domain created from Xen and the tools and 
where the parsing of the configuration is done. Other than they the 
creation happens exactly the same way (e.g. domain_create() is called)

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 30 08:04:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 08:04:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732515.1138496 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCali-0006RZ-QL; Thu, 30 May 2024 08:04:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732515.1138496; Thu, 30 May 2024 08:04:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCali-0006RS-Mv; Thu, 30 May 2024 08:04:14 +0000
Received: by outflank-mailman (input) for mailman id 732515;
 Thu, 30 May 2024 08:04:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCalh-0006RM-4m
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 08:04:13 +0000
Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com
 [2607:f8b0:4864:20::22a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 32c0031b-1e5b-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 10:04:12 +0200 (CEST)
Received: by mail-oi1-x22a.google.com with SMTP id
 5614622812f47-3d1bf6d156cso280356b6e.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 01:04:12 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb18325bcsm63747821cf.53.2024.05.30.01.04.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 01:04:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32c0031b-1e5b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717056251; x=1717661051; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zWLHlds/PNmILzywgRlVCbLx5sv8F+I6CGnf141A0qo=;
        b=SN6qW1cE1d7E43531QPYEkyNdIBisjAcyYkePSfAQwP1y3p4SkpuIFvhg0vFX634th
         PbTo991XsEzxZdxz5OQu9bKUUb0rGMrszgK4/A2eh+UaGjD9bpa/vI/uEGN7yY5Tzzh1
         IX/oPxqABjs6R1N2Wgndx/XsfpPux5NKrross=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717056251; x=1717661051;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zWLHlds/PNmILzywgRlVCbLx5sv8F+I6CGnf141A0qo=;
        b=Q7NBI2Fuve/BNxks9BDXgd27WnyqUGqYeiZ3Bq8PKsGl/OaPyCr5SO+khYlD053pOx
         fIK40v2Nhq5XMFuMTE6EUUwgKTE4MHzROqcTQMUfyGiYUoflLttCp8VRa8xN2MFt2Coz
         MafSOKWoit0O1otjHuat8m5Qbvj8CS0WsM2i/uitEGpUp11IY8f04ygW0p8fBMUdfcLA
         9A+QWa7HSIVT0sbEiXQarxVpA6IenwZnvjMCNb6o74oQQFoAfFjGY/X/PD61DeIrdW5X
         hXvILXrg3CKoaA97gjP1p2QN/jA61auOs9awu8CT8b0amH/ey1te4zj/yDEl3vjHEqmx
         SRRA==
X-Forwarded-Encrypted: i=1; AJvYcCU5Y9lB5TRjWJ4WsDNCoVAJjDBFo3vseBB9nT3O7HqOQtt+kNHz3OFp0+i5vuFPw0mrfDqXDHiCzcWQBg4wE0sBOtEfPwRa872+P9hScT4=
X-Gm-Message-State: AOJu0YyA8qJhnrM3UjVxIE0JZ4GsjYjdcucdHvqb9Li2H7cn8ba7hE71
	0kzDiY0ei+568LERm4Qtrz6KgP3k2GiJbUih0Y2WZwPZW9kHTSWiR/sGWowEuhU=
X-Google-Smtp-Source: AGHT+IHBGEVyMNrpeCV5dHwTnXhrLj8zOX1rydecYxruDO82P84vwUSTm5HMiVFOqKQ9p39hNb2EWQ==
X-Received: by 2002:aca:d0f:0:b0:3c9:6ea5:f591 with SMTP id 5614622812f47-3d1dccf0717mr1381595b6e.44.1717056250875;
        Thu, 30 May 2024 01:04:10 -0700 (PDT)
Message-ID: <0768b842-719e-4736-a941-dfa4d50c173d@citrix.com>
Date: Thu, 30 May 2024 09:04:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/hvm: allow XENMEM_machine_memory_map
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>
References: <20240530075318.67491-1-roger.pau@citrix.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240530075318.67491-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 8:53 am, Roger Pau Monne wrote:
> For HVM based control domains XENMEM_machine_memory_map must be available so
> that the `e820_host` xl.cfg option can be used.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Seems safe enough to allow.

Does this want a reported-by, or some further discussion about how it
was found?

Also, as it's mostly PVH Dom0 bugfixing, shouldn't we want it in 4.19?

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 08:07:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 08:07:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732524.1138506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCaoP-00073F-C6; Thu, 30 May 2024 08:07:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732524.1138506; Thu, 30 May 2024 08:07:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCaoP-000738-8F; Thu, 30 May 2024 08:07:01 +0000
Received: by outflank-mailman (input) for mailman id 732524;
 Thu, 30 May 2024 08:07:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCaoO-00072y-1d; Thu, 30 May 2024 08:07:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCaoN-0002fE-VH; Thu, 30 May 2024 08:06:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCaoN-0002ks-Jo; Thu, 30 May 2024 08:06:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCaoN-0006bH-JJ; Thu, 30 May 2024 08:06:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3u2wqj1hJJs+BwOUeO6lYzXJrIIKwKjQzIQBQrjHf58=; b=SEDv9Uk/3KmtkTZm2I0agk9dNW
	nuPa6qIm8K3Ut2zodY/qT+F2sBwTACtySV/HSiHIcNOBwpbjVEqP62GhqYu4rHUurJb9UNWIIonei
	ARvLo0DCojY33Fp1WGXfwOvu2mLPjAcg/cyIxQTBle769xbkGBc2qh44Kw8yVWMyvN/M=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186186-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186186: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
X-Osstest-Versions-That:
    xen=2bc462f645824a439879190ed19a231cb5b8034c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 08:06:59 +0000

flight 186186 xen-unstable real [real]
flight 186192 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186186/
http://logs.test-lab.xenproject.org/osstest/logs/186192/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 186192-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186183
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186183
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186183
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186183
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186183
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186183
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d
baseline version:
 xen                  2bc462f645824a439879190ed19a231cb5b8034c

Last test of basis   186183  2024-05-29 15:08:45 Z    0 days
Testing same since   186186  2024-05-29 22:38:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2bc462f645..12f9599a3b  12f9599a3b135afa270ccf0795f3e9665059955d -> master


From xen-devel-bounces@lists.xenproject.org Thu May 30 08:14:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 08:14:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732543.1138536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCavb-0000af-Ba; Thu, 30 May 2024 08:14:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732543.1138536; Thu, 30 May 2024 08:14:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCavb-0000aY-7s; Thu, 30 May 2024 08:14:27 +0000
Received: by outflank-mailman (input) for mailman id 732543;
 Thu, 30 May 2024 08:14:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCava-0000aS-F1
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 08:14:26 +0000
Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com
 [2607:f8b0:4864:20::82d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0752cfa-1e5c-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 10:14:25 +0200 (CEST)
Received: by mail-qt1-x82d.google.com with SMTP id
 d75a77b69052e-43fb909d45cso3108991cf.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 01:14:25 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ad860c9fdesm32734876d6.34.2024.05.30.01.14.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 01:14:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0752cfa-1e5c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717056864; x=1717661664; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=AB3C2fbZzXkeRXi10wONWuuMDzrUvbNythpE1EQ3R7I=;
        b=HWEZinL6IQdpTVZAAtHk3LjgbfqYijhBQF5NLMxGZhkDJznq06Lmrfxo+ByXYY08tQ
         oQVjrta2ZBZu/aivvC5pp3XcmaAqORilBGeb6JxPRYL6agOZQJBYVEXkS4xOLK3Os2+g
         pfFi69ky5H1NzWeKHgb2Sis83PGu/0b4fjxl0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717056864; x=1717661664;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=AB3C2fbZzXkeRXi10wONWuuMDzrUvbNythpE1EQ3R7I=;
        b=reoIMfeqMiQUUX0uWQdfsZ32zRL7xY9/xvpw4Z39sbbsF/75oXdOfRq1Hl7Sv/Ewi9
         I3SKOnUkxDmKDObZcTlxO5dsl3GtaZ2808K2CPrnnQCQx3103LJeltv2TqwS6k69Bbqb
         nNeGUY+ycT/yU9FNRluGDDtO0rnI7QwH4nmtqClNcCyqPxdyZIP1j4jLiH4IAY8+DElb
         oK2EnCN0r/Lviy9zZmVxCruG3+NDZpHBv8XzEW+6TiBeRl6f352jo3H8jhzNADE9yw9c
         suqoYPkiH8+BgnkR/ATPQnkwZBc17nyQpgZBv9q3uavlfX6WHKb80hEgm81nFf6ZAUdG
         ed0g==
X-Gm-Message-State: AOJu0YxWxxLwBfxjitYRxq8kcC7R70q/RX6hl2GO5PERCzQGKHOaRqAf
	qVPSwWQRuStZxTfbuqu5jHeHZjc2+MvnzL7wWG2A3C06V+sAZm5KEG27ozj16LsZXqO4opiwjjS
	n
X-Google-Smtp-Source: AGHT+IG1BWCAhoXAMgzSCSERzv+mVXMuvxiNmddmuDXRO7NPrUPAUFBAnZFStlhC/x4Lb/usqgx5mw==
X-Received: by 2002:ac8:7d54:0:b0:43e:3d29:4312 with SMTP id d75a77b69052e-43fe92bba06mr17241281cf.22.1717056864437;
        Thu, 30 May 2024 01:14:24 -0700 (PDT)
Date: Thu, 30 May 2024 10:14:22 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH] x86/hvm: allow XENMEM_machine_memory_map
Message-ID: <Zlg1XiUFR0sF3KCg@macbook>
References: <20240530075318.67491-1-roger.pau@citrix.com>
 <0768b842-719e-4736-a941-dfa4d50c173d@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0768b842-719e-4736-a941-dfa4d50c173d@citrix.com>

On Thu, May 30, 2024 at 09:04:08AM +0100, Andrew Cooper wrote:
> On 30/05/2024 8:53 am, Roger Pau Monne wrote:
> > For HVM based control domains XENMEM_machine_memory_map must be available so
> > that the `e820_host` xl.cfg option can be used.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Seems safe enough to allow.
> 
> Does this want a reported-by, or some further discussion about how it
> was found?

I've found it while attempting to repro an issue with e820_host
reported by Marek, but the issue he reported is not related to this.
It's just that I have most of my test systems set as PVH dom0.

> Also, as it's mostly PVH Dom0 bugfixing, shouldn't we want it in 4.19?

Yeah, forgot to add the for-4.19 line and Oleksii, adding him now for
consideration for 4.19.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 08:15:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 08:15:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732548.1138546 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCax1-00016i-LI; Thu, 30 May 2024 08:15:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732548.1138546; Thu, 30 May 2024 08:15:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCax1-00016b-Id; Thu, 30 May 2024 08:15:55 +0000
Received: by outflank-mailman (input) for mailman id 732548;
 Thu, 30 May 2024 08:15:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCax0-00016V-1U
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 08:15:54 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d4aaf3c6-1e5c-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 10:15:53 +0200 (CEST)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-794ba2d4601so46431385a.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 01:15:53 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd083a9sm538276385a.77.2024.05.30.01.15.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 01:15:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4aaf3c6-1e5c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717056952; x=1717661752; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=X2hd4avSIK5bX5EiqwjantdwrJQHKTlw2wHLS5/HiIk=;
        b=aWRVu7zaCwrW8BB4iiWBN/rxr8GFTu2sksC4dk5dFteNmthKeKKYVRwd/z/tvF2fS6
         73RJj7BeeZ1S4P7SVs8bGDWTrK0m+swar5rieJ8pRATxWm7feizb7ODKkL1TFNlfgEOE
         Q3Wvcojd7TyoriPjLpl0Y4oyF/Vb+gHlZ4Hfs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717056952; x=1717661752;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=X2hd4avSIK5bX5EiqwjantdwrJQHKTlw2wHLS5/HiIk=;
        b=tWOTsRBZNFJPOamSnYE/bbyZLC3M8yM6LE26QpH/YUwtSerXvnXHuopcJFqHNwIXfb
         9Z1a7u5TC3+1RB2xDFIm0jz1uRBGQDbYEauxExNvLSragpEskwdsMUERql100clEoH4U
         UffaxqbM62GGy+tynmDGplk5c2hd/fYKhP8AbQB/KTjcGd9sd+f058o6eelShwOmgVyk
         kpq5X6R++dhdBTALmOxLuvC2FSozINkssMVvhrtu54g61+bvKYfau73QpFUVWwUpnxVT
         UNnNSJ8rFnV6ojfKCr+/hBptNS+kP59kA28VL+4ERhVATZQh/bN3P37RygTokCs98U6L
         9RbA==
X-Gm-Message-State: AOJu0YynIzkWzq/D0Ea+OTpu+lycXTvZE2y8snCGAq5xBNzyZlrWPb2D
	mHNwA6Q1T0nWxd4YHbyKAznaTPdj5GjevrCnzmJKJM7+Yu7zKgYQdIojTpL00AI=
X-Google-Smtp-Source: AGHT+IFqkk64lklx4BqpmT7xWMNve9pzAFAGf0YnkJg5e+nevMU4Js+ee20fep0BD3jlqOE8I2eF1Q==
X-Received: by 2002:a05:620a:22b6:b0:793:822:d737 with SMTP id af79cd13be357-794e9e3a3a8mr140449685a.69.1717056951835;
        Thu, 30 May 2024 01:15:51 -0700 (PDT)
Message-ID: <3f160e36-9e7f-409c-bac5-5de2b13d0ae9@citrix.com>
Date: Thu, 30 May 2024 09:15:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/hvm: allow XENMEM_machine_memory_map
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <20240530075318.67491-1-roger.pau@citrix.com>
 <0768b842-719e-4736-a941-dfa4d50c173d@citrix.com> <Zlg1XiUFR0sF3KCg@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zlg1XiUFR0sF3KCg@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 9:14 am, Roger Pau Monné wrote:
> On Thu, May 30, 2024 at 09:04:08AM +0100, Andrew Cooper wrote:
>> On 30/05/2024 8:53 am, Roger Pau Monne wrote:
>>> For HVM based control domains XENMEM_machine_memory_map must be available so
>>> that the `e820_host` xl.cfg option can be used.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Seems safe enough to allow.
>>
>> Does this want a reported-by, or some further discussion about how it
>> was found?
> I've found it while attempting to repro an issue with e820_host
> reported by Marek, but the issue he reported is not related to this.
> It's just that I have most of my test systems set as PVH dom0.
>
>> Also, as it's mostly PVH Dom0 bugfixing, shouldn't we want it in 4.19?
> Yeah, forgot to add the for-4.19 line and Oleksii, adding him now for
> consideration for 4.19.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Thu May 30 09:04:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:04:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732571.1138556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCbhR-0007kf-82; Thu, 30 May 2024 09:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732571.1138556; Thu, 30 May 2024 09:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCbhR-0007kR-5F; Thu, 30 May 2024 09:03:53 +0000
Received: by outflank-mailman (input) for mailman id 732571;
 Thu, 30 May 2024 09:03:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lthE=NB=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sCbhP-0007jh-IM
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 09:03:51 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 (mail-dbaeur03on20601.outbound.protection.outlook.com
 [2a01:111:f403:260d::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87bdd136-1e63-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 11:03:50 +0200 (CEST)
Received: from AM8P191CA0003.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::8)
 by DB8PR08MB5499.eurprd08.prod.outlook.com (2603:10a6:10:fa::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Thu, 30 May
 2024 09:03:47 +0000
Received: from AM4PEPF00025F9A.EURPRD83.prod.outlook.com
 (2603:10a6:20b:21a:cafe::d1) by AM8P191CA0003.outlook.office365.com
 (2603:10a6:20b:21a::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21 via Frontend
 Transport; Thu, 30 May 2024 09:03:47 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00025F9A.mail.protection.outlook.com (10.167.16.9) with
 Microsoft SMTP
 Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7656.0 via
 Frontend Transport; Thu, 30 May 2024 09:03:45 +0000
Received: ("Tessian outbound a0025cf119c5:v327");
 Thu, 30 May 2024 09:03:45 +0000
Received: from 2bc1eb3bc0f0.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 69D9C1F4-9622-4E9D-9857-0D0B855F1120.1; 
 Thu, 30 May 2024 09:03:39 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2bc1eb3bc0f0.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 May 2024 09:03:39 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by FRZPR08MB11190.eurprd08.prod.outlook.com (2603:10a6:d10:139::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Thu, 30 May
 2024 09:03:37 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Thu, 30 May 2024
 09:03:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87bdd136-1e63-11ef-90a1-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=JYqwl4xl0KhAUQMUblpOGdPch8MJf7oMSG7e16o7jhrxNT3D1oCKsBKieyoSOYKB1psNhc9DUfA8V2yCGOfhVRJb73mK2vq4DYFBLvFM5aiYsc8ENRAZIeveuIbUjU377TAHCkssmgu2En/uXevkfbFnWTunZ/EN+Tkh56/cyuei7TugjVgKcwC6fl/n+iIVJD1HC7vKWY4FeFS2sjpRoiS8u7jAE4BrFl/mQg4YQZ0nJOdQEw9chQxYOiaan4p/oyvYnsxf5sEZbqCRs153+ZdX/tzJWU3CtCCFdfIHY/bvKtiJtnmILaW7Vd5qYLBK9FR2K5qs6Sr0qXnzXYTcig==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pBf+gGNTpvlyPFsg4FGV3jSyPfiHCbGdaEQksFAO3Dw=;
 b=blpvcJCtYdqeiY8YiMlk3aDmxfyWxAMN0sWUoWNZSQUpe5Msg5aJI+7rqPqrA+Lw+4mPA557i+CZmIwFsZm9kq61/xTlcenGcraZ2Ec0kD+2yWBdUPLeeCAmzeGjd1awuuGHMYypJR+K+CUpHKioxOCSlPjLO2mcl3fxcNlhrPgJ0z7OSo0nYlMsLzSLrFyn7MQPUbKCG1GaFSwnvP+t3Ifyz1HGTS+zomP3P3TrKRYcXEl/6QscBfN9bkm81LpZnmQzVSWoGqBHQfmBgtGR0TRs7UKcucHQcjtpz6jIe5JNOEIh1coQ7ImSNcVGDi0aVB++L55u8HywBN9B0S+Iww==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pBf+gGNTpvlyPFsg4FGV3jSyPfiHCbGdaEQksFAO3Dw=;
 b=EleRQWOY+wNZM7dVQLK/8v7OeZL9Dm5kwfDKo8kfZqXgU+B3hdfu2jP1WPEflw7ur+67xr726a3yVeEAP0uvniwZSh9k6fh1JeHZwlU7x9PR1RkTZI6TGwQbigAY9RE1nAFbAnTun1eBZ59ZNUqEydmhWAd3ZwxDgBXsYh1UPlM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: f2f7685686976c73
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NIk4va0MznYfibV2SzVm3OuGpH1BHWIM3ppvrSNOzdrj3ZRpAxmCwMP74wpBrtGV0j1Vky3tooO04yjIub1xPpEkxWy1w/2Lep3S0b2TAigC35SZhZ54qG4vvJ7IKpIs4jwBMFotljTO+tnBadcbu0ZZwXpjrIcXuRg0gPA7qtwUEPb/XoLb9wCMOGXcIdANo8kS8+40SoRSUHsXV/KNMh84KfuaBgIqFJUBJ41y+MSt4kemMK7HcwaYGEK0tt+P2HXn3VjemRlxkUVeB0MO6GqU3q9p7VjePi1tHb9GrObCeEuDnDi+zTj1bnc0/alUoWkiQEghBtim5j7HpgjeKQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pBf+gGNTpvlyPFsg4FGV3jSyPfiHCbGdaEQksFAO3Dw=;
 b=V6YSO6IsGSL8NI8381qZpcCv73U2Cp9J6VlR1+rXCQX4o2IbIz8fiR5Hd27xFsc5Sx2k2G/j2cxHQL8XYoRvc/WrZ/nlnS5OUjVacoqke2+/LJ879zogE6Z6sZ4hIDma48Tx25+gdUNdsMOR0mONDvI+vHGQkb4MSqHWZVWLyhubq8J8aR2x8KmPtZyo9WnPE+GUr324lFLXIjvaJNkuytsOiIzvPujNrimHCPPd2D8TdTL6a6fsbiTcS2xBrWeEgSJBpRW0P9SoWOmCtvU2FNo7nYIbngkE/xxZRzgmtsUMn7of2MmSa+B2Rw+FpWvxmZNv2BlUFZu/+lftRc3dpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pBf+gGNTpvlyPFsg4FGV3jSyPfiHCbGdaEQksFAO3Dw=;
 b=EleRQWOY+wNZM7dVQLK/8v7OeZL9Dm5kwfDKo8kfZqXgU+B3hdfu2jP1WPEflw7ur+67xr726a3yVeEAP0uvniwZSh9k6fh1JeHZwlU7x9PR1RkTZI6TGwQbigAY9RE1nAFbAnTun1eBZ59ZNUqEydmhWAd3ZwxDgBXsYh1UPlM=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] arm: dom0less: add TEE support
Thread-Topic: [PATCH] arm: dom0less: add TEE support
Thread-Index: AQHasgjZKwS4pdCyJUe9LewXYMLHeLGutaSAgAAAkICAALRFAIAAEesA
Date: Thu, 30 May 2024 09:03:36 +0000
Message-ID: <91FCE951-780D-40A3-BF67-4788FA80DCE8@arm.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
 <eff6e2e6-5c58-4b19-99f4-3754f655210d@xen.org>
In-Reply-To: <eff6e2e6-5c58-4b19-99f4-3754f655210d@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|FRZPR08MB11190:EE_|AM4PEPF00025F9A:EE_|DB8PR08MB5499:EE_
X-MS-Office365-Filtering-Correlation-Id: 23374a66-34d1-43b1-8714-08dc8087696a
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|366007|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?gZrZBjD9MswbsJB+QeWmP0Vqj79/RO+ThCLQNdW3S5udYSW6fAmitL2GUEOm?=
 =?us-ascii?Q?UWz8XySrA7p5IUvEtcQnjzqn5LPozlZBhygizZxdfwAn/uBYtvjycq9W9LyF?=
 =?us-ascii?Q?Fohx80g7z9msuY6U86RrAisr9ndlwKESu63fGzF+UTwcEou24NwCntd9vW7Q?=
 =?us-ascii?Q?b5fMzTxgcXroJ9l1tfK9A21nu9g8UGcmi5Mm2xjWIospFYm1n2CZOw0jl43M?=
 =?us-ascii?Q?pkimdfrqgOYkcKSPxuzFFcg/VXN626yuCd5QauyIuMxvFCjv3OKVSA1lmjfT?=
 =?us-ascii?Q?tA6DBSFTYc5xnzz6iq4UrepRhkf0dOz0d1EVlym++RoB75NsVWu5dvtODNL/?=
 =?us-ascii?Q?AcwFDJD6TQtRlPvYW9h0CIOaJFY2770KEoSEkci8y66NDJtmVHu9mrwoDz/+?=
 =?us-ascii?Q?xF3Z0AZ6rOgnm4heAsV5d71T43fzMq3kGg3GHSMwq8hK1WCGhJNAoz5FaAjL?=
 =?us-ascii?Q?4ATE/xkj4ugwmePDKYtYGuqwz68NTmdAtpaQYIapzMf3NLQejOMHq6QiFjBX?=
 =?us-ascii?Q?eQQxijBkL+UCxEZYJSJhUseb3iX+nNv1HSxkAhWnehZeVbwpajSbJEIC0JVw?=
 =?us-ascii?Q?Vqn+7FDztdzOU7VbfW1AJJ86yALlU5U0D5T1lc4JpBuNw2sGQJ7HKBg3+lbm?=
 =?us-ascii?Q?XeriAYQ+laLu0akAX7VH9MebzmBOlnlJAWHNS6q3TnoL2pOv/mcZKfGbm3ut?=
 =?us-ascii?Q?lgAenPoqeWp6+OTuYPBPX6caMRhoyVE5g3oS05nb3OP0jIxSiXkURvA7jyOH?=
 =?us-ascii?Q?KjcUhitDO4jt6MwIbb9F/qum28V7P9G/k9fUK/uoKxaXzcih17nAWWxSHYLl?=
 =?us-ascii?Q?iGpmtdlZ8lCkQ/BIQduk2UUWC1km+wPA/awhlxI9huFch0ECc78RmjrirqHW?=
 =?us-ascii?Q?FgNzsMh4p++ONaXzNJUPHA/VP4KlYIAOhKSn02mEzjGGXlkLPNJfholXAJz3?=
 =?us-ascii?Q?TNQ8GvmGKY4rEMlX7UKxNDza8YiF9NGLqjgE9iJ1AqcFeTwKeyV8CxgxHkWc?=
 =?us-ascii?Q?DJvXqKuEkhEqHw4HPU5NAXUPZVVRo9znvYM9SvaqSXafxz7zFz7P4w5ytm9F?=
 =?us-ascii?Q?IfmrzTwk2y9fXx5CMQPaa9Rh03iYl/gSx2HjQCoH7rXkeueBrO/7XKENpRof?=
 =?us-ascii?Q?Bhk9rnoiWbS+ajaS3JReIYEyL+h+YgO/wwr/Cf9PHm67ZMEvahs6WqE1hEg6?=
 =?us-ascii?Q?4JeqfBCgx0oBJpudxm4qNPNk66P+DYBZPfjx+GUYPXT+23BZdqpQEhEqQlN8?=
 =?us-ascii?Q?4Nfh5xH5aaGNv5NNcSBIKKkDYImaOlBYa4Az6wVVGGekXvBPSRXN4IcorgZE?=
 =?us-ascii?Q?el9BwZfuwRWMywfBms8ouQ8bFCdchWpNKwkAwFg9qHl5Gg=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F5E7A21EE3C5CA48AF09995F75C68F33@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRZPR08MB11190
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00025F9A.EURPRD83.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	fd2f1c10-4442-4cb8-9ab3-08dc80876427
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|82310400017|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?PC9rnt+LPrdxWmprW7oyZ1VJzXrdtK17x7YaTMiRTtqh3bzf2Z6DMYQp68bg?=
 =?us-ascii?Q?t3MMvrn+5PvvhhXguPpy7NBxZVO6XETK2Ggha5c6UumqvbtkLoah0Y4g4T9l?=
 =?us-ascii?Q?g1mVJUWhvsB1pJi3RhNfL2i0YT+9ZCG6CihrfzW/KwoZ0Mjem5sxZavs2GgL?=
 =?us-ascii?Q?h2BmTm8B1SGGEfnkQziJQ7/FveZx2i04QVHcN8MY9WWldviZKk4M0dVRAXp0?=
 =?us-ascii?Q?fDCGTFYjhc6KoIbip790SLngfKFJIoVl8IW2bx/ZGzH/gp2td6eU1+0EhVGI?=
 =?us-ascii?Q?fXLpTJCqTFuWyIqTFxHiuGqJC7Dd514G+h5HHbODOeBz/wfdQlKFCKOIL++g?=
 =?us-ascii?Q?nYxFVfRKRDX1Iz/w9XIAObkxBpKESRyJb4ZOtyCXxFS7ltHr07nVihZBt1ad?=
 =?us-ascii?Q?g2QurSPB7ELo2FOmBGRgL4u946LpOdkFkDKkRznBIQZ18DupFA9OxURS9sd+?=
 =?us-ascii?Q?GnzC9SyRRAFsL9ql8B1zmgiPCYHaCbGvGohwRKC/G5LsqI5qAycyHXLxkbts?=
 =?us-ascii?Q?duFJ2uFAo01JNDx/AiKGbmV8xDEczI6ES1ZuwsT4gwa0KLARh2YSkc/68RlZ?=
 =?us-ascii?Q?nRl2Dp/0bwPW9uMTOpHT07kpepFxtcizYi6y1S4fu+ZDihY0lDfbdZlYsqxt?=
 =?us-ascii?Q?03wIaV1oUVTUajMXfN4K4Mn9AmGERpcWuiefHBdOkK25326zg0oZMo8ey3O/?=
 =?us-ascii?Q?FhMzcoz8xHRr36jDpntXKSrDTXaPCBhNjuOOcAUZvv5OGNQrhzFD3vIDlv7x?=
 =?us-ascii?Q?xi1RctbpkHsyKE5x8Id9CqKTZW2IVcl0hXJDqk/YN+Peswh+2L8cwWQMo16U?=
 =?us-ascii?Q?3wWnzoQA2IfzORp6N96mUuaIe8apC6ILFwpXrwp06njnHGMiBAipNyfHUYD0?=
 =?us-ascii?Q?+DyePzabaev9gIHDOTtUcLcn3pAsFv5b6IIMDXXGVDI2PRoNaIu3Z6fN+wyx?=
 =?us-ascii?Q?1AvHizGRlVvJS7bLfpHGZSeXrzYQDo30TcWZZkmUvsCoMUZdms0LfAb7cO15?=
 =?us-ascii?Q?30UBS+U3OZ/OmsPAHGfrbppKh6jNLG0cYrVzBG8qzL2bzPxDia2+nF6grnzl?=
 =?us-ascii?Q?Kn7hmnqUiUHqNT8YEW+B/uKImOboOV3KwAVk2YnI2yjb8TJyZ2c7YJQqye2Q?=
 =?us-ascii?Q?Ng2iriBpoGQYCBpKuJtL57IeFp0pqCblj3U+DhMZa9VdHQPoytNkVGhpn/In?=
 =?us-ascii?Q?dbTeKeya4euN1PhEGCSQ2lmltRF3U2QBtFQLUvLu5WARuAd5YcvNAJK58AT/?=
 =?us-ascii?Q?DCmDxd5A1OTCjFNBMsK7MqgVGljStq/JMtKvz/jS9kti7WkkVvNFAs+egWBN?=
 =?us-ascii?Q?XZ6g7s8Nh1NExwI7jo9SX6BbsSdigzbYe7VHKtnR53aoP67ZyUUMqZ7EAIno?=
 =?us-ascii?Q?C6H+d1C34OrM/D8SPWprDw/OVSs9?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 09:03:45.6672
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 23374a66-34d1-43b1-8714-08dc8087696a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00025F9A.EURPRD83.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5499

Hi Julien,

> On 30 May 2024, at 09:59, Julien Grall <julien@xen.org> wrote:
>=20
>=20
>=20
> On 29/05/2024 22:34, Volodymyr Babchuk wrote:
>> Hi Julien,
>=20
> Hi Volodymyr,
>=20
>> Julien Grall <julien@xen.org> writes:
>>> Hi Volodymyr,
>>>=20
>>> Can you clarify whether this is intended for the next release cycle?
>> Well, I don't think that this patch should be committed ASAP if this is
>> what you are asking about.
>>> On 29/05/2024 21:43, Volodymyr Babchuk wrote:
>>>> Allow to provide TEE type for a Dom0less guest via "xen,tee"
>>>> property. Create appropriate nodes in the guests' device tree and
>>>> initialize tee subsystem for it.
>>>=20
>>> The new property needs to be documented in
>>> docs/misc/arm/device-tree/booting.txt.
>>>=20
>> Yes, missed that.
>>>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>>> ---
>>>>   xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++=
++
>>>>   xen/arch/arm/include/asm/kernel.h |  3 ++
>>>>   2 files changed, 72 insertions(+)
>>>> diff --git a/xen/arch/arm/dom0less-build.c
>>>> b/xen/arch/arm/dom0less-build.c
>>>> index fb63ec6fd1..1ea3ecc45c 100644
>>>> --- a/xen/arch/arm/dom0less-build.c
>>>> +++ b/xen/arch/arm/dom0less-build.c
>>>> @@ -15,6 +15,7 @@
>>>>   #include <asm/domain_build.h>
>>>>   #include <asm/static-memory.h>
>>>>   #include <asm/static-shmem.h>
>>>> +#include <asm/tee/tee.h>
>>>>     bool __init is_dom0less_mode(void)
>>>>   {
>>>> @@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct ke=
rnel_info *kinfo)
>>>>   }
>>>>   #endif
>>>>   +#ifdef CONFIG_OPTEE
>>>> +static int __init make_optee_node(struct kernel_info *kinfo)
>>>=20
>>> Please introduce a callback in the TEE framework that will create the
>>> OPTEE node.
>> This is the reason why this is RFC.
>=20
> If this is meant an RFC, then I would recommend to tag your series with R=
FC. Similarly...
>=20
>=20
>> I wanted to discuss the right method
>> of doing this.
>=20
> ... if you have any open questions. Then please write them down after the=
 "---" (so they are not committed). So this is not a guessing game for the =
reviewer.
>=20
> For instance, if you hadn't asked the question, I wouldn't have realized =
you were not entirely happy with your solution. To me it looked fine becaus=
e this is self-contained in an OP-TEE specific function.
>=20
>> "optee" node should reside in "/firmware/" node as per
>> device tree bindings. But "/firmware/" node can contain additional
>> entries, for example linux device tree bindings also define
>> "/firmware/sdei". So, probably correct solution is to implement function
>> "make_firmware_node()" in this file, which in turn will call TEE
>> framework.
>=20
> Longer term possibly. But at the moment, we have no implementation of the=
 "sdei" node and I am not aware of any future plan. So I don't think it is =
necessary to split the work in two functions.
>=20
>> But we are making assumption that all TEE implementation will have its
>> node inside "/firmware/". I am not 100% sure that this is correct. For
>> example I saw that Google Trusty uses "/trusty" node (directly inside
>> the DTS root). On other hand, it is not defined in dts bindings, as far
>> as I know.
>=20
> TBH, if there is no official binding documentation, then Xen cannot sensi=
bly create those nodes because they are not "stable". So the first step wou=
ld be to have official binding.
>=20
>=20
> Bertrand, I couldn't find any documentation for the FFA binding. Do you k=
now if they will be created under /firmware?

There is not device tree entry needed for FF-A because it is detected throu=
gh a FF-A call.

>=20
>>>>   /*
>>>>    * Scan device tree properties for passthrough specific information.
>>>>    * Returns < 0 on error
>>>> @@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain =
*d, struct kernel_info *kinfo)
>>>>       if ( ret )
>>>>           goto err;
>>>>   +#ifdef CONFIG_OPTEE
>>>> +    if ( kinfo->tee_type =3D=3D XEN_DOMCTL_CONFIG_TEE_OPTEE)
>>>> +    {
>>>> +        ret =3D make_optee_node(kinfo);
>>>> +        if ( ret )
>>>> +            goto err;
>>>> +    }
>>>> +#endif
>>>> +
>>>>       /*
>>>>        * domain_handle_dtb_bootmodule has to be called before the rest=
 of
>>>>        * the device tree is generated because it depends on the value =
of
>>>> @@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
>>>>   {
>>>>       struct kernel_info kinfo =3D {};
>>>>       const char *dom0less_enhanced;
>>>> +#ifdef CONFIG_TEE
>>>> +    const char *tee;
>>>> +#endif
>>>>       int rc;
>>>>       u64 mem;
>>>>       u32 p2m_mem_mb;
>>>> @@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d=
,
>>>>       else if ( rc =3D=3D 0 && !strcmp(dom0less_enhanced, "no-xenstore=
") )
>>>>           kinfo.dom0less_feature =3D DOM0LESS_ENHANCED_NO_XS;
>>>>   +#ifdef CONFIG_TEE
>>>=20
>>> I would rather prefer if this code is implemented in tee.c. We also...
>>>=20
>>>> +    rc =3D dt_property_read_string(node, "xen,tee", &tee);
>>>=20
>>> ... want to return an error if "xen,tee" exists because CONFIG_TEE is
>>> not set.
>>>=20
>>>> +    if ( rc =3D=3D -EILSEQ ||
>>>> +         rc =3D=3D -ENODATA ||
>>>> +         (rc =3D=3D 0 && !strcmp(tee, "none")) )
>>>> +    {
>>>> +        if ( !hardware_domain )
>>>=20
>>>=20
>>> I don't understand this check. Why would we require dom0 for OP-TEE?
>> OP-TEE is enabled for Dom0 unconditionally in create_dom0(void);
>=20
> I am sorry but this still doesn't make sense. AFAICT, this path is only u=
sed by domU. In some dom0less setup, we may not have dom0 at all. So why do=
 you want to prevent OP-TEE for such case?
>=20
> Or are you intending to check that "d" is not the hardware domain? If so,=
 you have the wrong check (you want to check is_hardware_domain(d) and AFAI=
K this path is not called for dom0.
>=20
>> This is another topic I wanted to discuss, actually, Should we use the
>> same "xen,tee" for Dom0? In this case we might want to alter Dom0 DTB to
>> remove TEE entry if user wants it to be disabled.
> Is there any existing use case to disable OP-TEE in dom0? I am asking bec=
ause removing the nodes will make the code a bit more complicated. So if th=
ere is no need, then my preference is to not do it.

I would say there are several:
- optee not supported in Xen (dom0 cannot access it anyway)
- optee to be used in a guest instead of dom0
- ff-a used to communicate with optee (in this case optee support is not us=
ed but ff-a is).

On this subject, I will not ask you to add support for FF-A for this but wh=
atever you do please keep in mind
that we will probably add the same for FF-A so that we end up with somethin=
g coherent where the tee can
be selected by configuration for guests or by device tree for dom0 or dom0l=
ess guests.

I will make a path on the next version of the patch for this.

Cheers
Bertrand




From xen-devel-bounces@lists.xenproject.org Thu May 30 09:20:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:20:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732580.1138566 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCbxW-00025E-Jx; Thu, 30 May 2024 09:20:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732580.1138566; Thu, 30 May 2024 09:20:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCbxW-000257-HB; Thu, 30 May 2024 09:20:30 +0000
Received: by outflank-mailman (input) for mailman id 732580;
 Thu, 30 May 2024 09:17:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/zMW=NB=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sCbuC-00013j-Ff
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 09:17:04 +0000
Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com
 [205.220.177.32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f8e7cae-1e65-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 11:17:02 +0200 (CEST)
Received: from pps.filterd (m0246632.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44U7n5hV002784; Thu, 30 May 2024 09:16:43 GMT
Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta03.appoci.oracle.com [138.1.37.129])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8fcghay-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 30 May 2024 09:16:43 +0000
Received: from pps.filterd
 (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44U7Jqpe006215; Thu, 30 May 2024 09:16:42 GMT
Received: from nam10-mw2-obe.outbound.protection.outlook.com
 (mail-mw2nam10lp2042.outbound.protection.outlook.com [104.47.55.42])
 by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yd7c6pax4-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 30 May 2024 09:16:42 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Thu, 30 May
 2024 09:16:39 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Thu, 30 May 2024
 09:16:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f8e7cae-1e65-11ef-90a1-e314d9c70b13
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3DBdPYxC5USHVULUTy7ZfZqhM+RCgM6BcNA8YOCr9E+nE=3D;_b?=
 =?UTF-8?Q?=3DiEXCWqw7ljieCh6HQcv6moF6g6zlOtYTs6l/bQyZBV7mrEjbMzJiceT6F0TR?=
 =?UTF-8?Q?mJYApcHD_rrmIiuOIR//EoyWR5w5CSCCTXNENc+IUNFx3eivp4gh377HOwdhGXi?=
 =?UTF-8?Q?LP5i7sN6kjGLHM_LDBfpz+1caJFy25YoJCcGZ/fD/OU2M+YZLB4kMTCD69tUswW?=
 =?UTF-8?Q?3N8vMnLx5eV8OkcTv0Xc_vNISwufG4aQ5sr3GKcT/Xq7FCrvNjhtKsIEv90+n7P?=
 =?UTF-8?Q?+gACPMVFAPtt9crp60YwPRs3dr_smafucRx/zlmlFgn9uu1Q+J/CM2vjEQXkAQ3?=
 =?UTF-8?Q?uxyo1jJ1rqG7DC7kE0Kqc8IyI4bl7JCo_Mw=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZW4sdwt7nQbdjT0+fvotSJ75hJ6npAhUXI5NdHvh5g2RfO7Hwze0n6s4G7cAHt5sM7X2WtDBBIjEVtGgby77A+ClUiEEERrhNwsldrqxryazhuHk5ou7HAB5cNLPT9RuywPSPtBJxKEfr2b19fM2xMP911XbMN9HPtFlE8saewhwqJtkhvVSbPAg8DCxN3hXyf84ug3gHb0sz+fEuR3KELd3qgrluqMkrcjbpCSvNpTKIvZAl8hJYsIULC7JIvXVKG9eEzs9OUInDVTmSWxH9lvDF7uTRgPeBdEI4Vx+xN3+sihlxpPD1k/ioz6RTGWx7AGkO/97mo0nYRPoKptH0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BdPYxC5USHVULUTy7ZfZqhM+RCgM6BcNA8YOCr9E+nE=;
 b=MxU1uyBe/4XWuZlHRz6ZGJ//6QpvoPjDgaSd3O0hpy4VHW1FKCqSfE04+mAGBtdbxoYWqCUHOyGsTS2P/9xlDtT8XnZ+LhvY0R5Vd95y2NAnjwGhp4xgVLBXB3sbAsBonL5WRbbXn0DjH9NSfjgUnnTwIEhEs4cvGQRqyUtu4dJ2HGLMSYWC/iPr63WwiOmOhrxrCtgmsx5x64f+6e9/9zVXTFWxMNhGyg93UvvaLsue55s/d2FjpgKWvjnh8wJy3jrDClGkKIJyV57+rifk4KX+Pm9/9bI/KQtbCB2kUqIIUt3SuMxwaxr8QlFdg57WNNJiDjxXgGEZyxMIXlNwhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BdPYxC5USHVULUTy7ZfZqhM+RCgM6BcNA8YOCr9E+nE=;
 b=G/7QTVDgO+k85ObUI8TepKa4ed7WFzUGZgaPA/T1NlpuXHcU5g6u/DK/lJFVobL9IyH3DZgHV7oNkueIPT1RrT/AuDv25KcA5Mk5siotZ7nswU82hU2RiKjTMiHDBmwJmkT3sO/vg+heGPGmcBgVVb9ScoszC+PtHlQMwESHSG8=
Message-ID: <1a1854bb-1f28-44d1-a4ac-30872bd6c3c8@oracle.com>
Date: Thu, 30 May 2024 10:16:33 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 09/12] sd: convert to the atomic queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-10-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240529050507.1392041-10-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM9P193CA0024.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:20b:21e::29) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|PH0PR10MB5706:EE_
X-MS-Office365-Filtering-Correlation-Id: 58489bb1-ca80-451c-ae61-08dc80893636
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|7416005|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?Zk0xKzhCaGd6TXRoNUJzU21JbVY5VEkrRVA4TndzUW1Ycm5kUnRnWmozM3Fs?=
 =?utf-8?B?UGpOYTVxdXYrdlI3bHlUSTY3c3Q3SHBVODBFRWYra25zaDdXbEJUUnV6ME5w?=
 =?utf-8?B?OWFldVYvMHhRdVkvNmFQcHRnSjFlWDdLT0szV1RDanpKQWxPMURhbmJUSjlo?=
 =?utf-8?B?NEMrMzZzWHlKeE9SRTNESGN4YmE0ekIxWHlIZHZvdWdaaVpIZXRFRFREQktw?=
 =?utf-8?B?R2RocTEvczQ4ZUhJZ3p0RmZDaEpzWGkxREtBejJVWjlyN05yc1Rtak5YUWpF?=
 =?utf-8?B?dk54SVlSaURrQ3lFZmtEV3NZWjVRNG5KamtqRzFHNFlVdTBmZDFHRFFWbjBi?=
 =?utf-8?B?Mmh2bmtTYnZHTnpEOVRTcVRzSUJvQmpLQUNsV0xaK2o5aklIdzUvZ3ZGVXlT?=
 =?utf-8?B?SGxHdk1hbmNxdklPQnp2VVovYUhWRE0xK1lHaGhhbGo1cVlrLzg4VDZyMDVu?=
 =?utf-8?B?U1JjZVlrYW5QK2tKZnRPV25VQlhTZFE0am5hbnlXbWoxdlE4cWpuU2ZjUGR2?=
 =?utf-8?B?MmlYMmN0UlRpYVF4VDRNTVBTMlVzVURhcytXSmtXRnU0T1J5Q1VXK2FwSTNK?=
 =?utf-8?B?QzRwemsxbGxTeWNhZ3Z0bUZ1V250YXN5NEU2VjZxbXFqZGVGRUgzYVIyNVN0?=
 =?utf-8?B?TW82K05MM3hYQUlQQ2xNMUlUWmpDNTRtWVRTZlI3ZGJ0RGJvYS9wOFdHTjh4?=
 =?utf-8?B?c1ZXUkkxdnluYk1yUjJPbmoxRkxNM3MvaElUSjNDTm5pTFMzWWNMVWhxSkQ3?=
 =?utf-8?B?Umlwb01mMU16a3VRQWxMRHlpNk9jcGxWMDVuTCthQkVSQ3FEdW4vcVJLY3lB?=
 =?utf-8?B?RS9Ddk9FZTV5MHpMdy8vRTRSaXp4T0JHaU1pNjRObVFzaU1GenFwWWhGRGNO?=
 =?utf-8?B?WmtMbmw1UXFoMVd6dXlyM0dJbjM0cVFRaXVjdmRCbG1DSHZNL2JYZXZsTi9l?=
 =?utf-8?B?VzMrSzVETUFFUGpFZitUWVJOSlNDMXJFQlEzQTdNYjlXZUwrZjN2SDh0cENM?=
 =?utf-8?B?cVpvcFRuTEhVQ0pCUWs4NGpnN1ZiN3I4Zkw4K0NvSi9yWnlIT0Y4RktLTDVl?=
 =?utf-8?B?cUFpeklwdDdXM29WaGx1VXNjak9KVkF2eDFpRTNsenVaMWZIL3ZKSW94OTc2?=
 =?utf-8?B?TUcxRU1HNWV2QWx3Y1FrWTVYWDJSWDVSMi8rVGhtSi96OGdhTTRja2N6TW1y?=
 =?utf-8?B?cHd5U3Z0TFZHUDM2K2dIZG9rN3JqK2xWY0RFVVVqVWpTZTRvbnM2KytZNnNF?=
 =?utf-8?B?UmFiQUdCaUw1UFYrSnRzWSt5dEJycW1sUjI3N0hXVEdGcDlBVHVhZWZTa2pK?=
 =?utf-8?B?UHJYbXhJOTdzaGFzSEd0VktpTXIzaSt2Y2RtR3NHbUdaRUI5RXRnWkRGOVRP?=
 =?utf-8?B?WXpXbndBTGp1aCtBRlc2dFloNFFMVFRQbEZwb3RIMnZXalVDYnBWdVBjOENa?=
 =?utf-8?B?eW5qa0xKVjllZXVjNHdoanhOYk5CZ1hrdFJGNFNxOURoTnlhdkxYYUZuRWxN?=
 =?utf-8?B?aGU3YzVLMEhFajByTXlQWTYxYUZFSERETmdIbTNIYjdMTnR2a0V3Y1J0QXJL?=
 =?utf-8?B?elRTNG1nbVJkQTRMOG4vbHl5TjBqaDNKMGFzclRITXVsUndVbU1uMTYwTXpN?=
 =?utf-8?B?aTI2VFRhbmpmQnYxVU9ieXcwNnAraStPTjBaaVpTRFpSM0hUenl6anZLdU45?=
 =?utf-8?B?dktJcEszckovaVRpSnVPSnFtYXFkTXBmZXNkSzkxNkJXeEY2MFVJN3JBPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(7416005)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?amJLcStFaDB3aGZZZndOMzkyRGQvc3BFeUNZNUtkN25iNVoxUnJlWkZzam81?=
 =?utf-8?B?VXcyU3dXYU5sRjl0Ylh3TlBqWXo5VnFCR3B4UEtvNDRIeER3WmNPRXRScmRx?=
 =?utf-8?B?REZKY3hIRDhjaStkRExVd3I2eGRmYTk2Smd0NnBMcGh6d1dtTEhxVHIwa0dY?=
 =?utf-8?B?UEZ6OHhnMm9MN2NLQ0svUjE4TTN6ZHF6dHBwRUd1cVdQRHp5RXNXbWlaeTdC?=
 =?utf-8?B?V2ZsNFNzY2x5RzkxNEhuZExaZkhJMTh3SkdwdG43TFdUWkI5SWVSTmN5d1ZI?=
 =?utf-8?B?NndIS0Z6VUpaMVAzMjE5NzVWSE5Ib05FaVBsZTA0SVVqVDlGc3hMNVBUajJL?=
 =?utf-8?B?SDROVTlRV2ZwaUxWNWQ4Q1hrOGtCOXNjckovNmZreUxiRVlJRWR2QnJmVGJL?=
 =?utf-8?B?UWtzV1hadmd1ckc0ZGQ5eU1uRGVWb0kzK25HcGp5eitYN1NGTWZCdURWTUNW?=
 =?utf-8?B?NkdvMS8xVWpBL3FDZWdNWktLQW9xR2R6RW5PK1NidTI2Q2JsUGFFVjVHOWFM?=
 =?utf-8?B?NGd5SXBxekZ1cEZCNVRjSEkyQ3U5Tlp0M0ZkQUJWRGxvWXgvSGZOM3RPMTlz?=
 =?utf-8?B?U0VMb1hOanF4NzlqTC9ibjI2ZXFKZGFsZTJDb3dQL3UxZ3BOSVhoamlEcFhi?=
 =?utf-8?B?WS80SkZWZ2o2SHdVSEx3ZWcrUWpHUm1JM0dkTVdnZGxqRjVDRS9MTUxlSjdR?=
 =?utf-8?B?SUkvcVQzSmx3ZFVoSDVia1crYXdQMk9IWlFmbDFMRG5GMkhzTTNaRnpwbGp3?=
 =?utf-8?B?WUx0bURzeHBhTU1mTzlkbDExcEZ3dm1VRDVDb0xTZ0pSZWxSNjIwVzJxKzJt?=
 =?utf-8?B?MHZhL1ZPaEpydXhKaDAxUi9PeEFZZXF5TjJKekx1Y2hCb0FnMklFaUxVWmF6?=
 =?utf-8?B?K3NiVmZxZ0VpUjlUZjA1d3RFcXlYaFk1VEMxd3gxdTlHTExqR1hJc0dFajRq?=
 =?utf-8?B?TzMzbG5UMWFoMnJaMVkrQm01cDZ0L1M4a2wyMlp5WXlMWWhYVjVvN2UrNnRY?=
 =?utf-8?B?bCtjcDNCVG0wTkRqM0l2bGxPVFp3WDlUdHdmcmJxUTd1TE5RRm41dU9BaTFY?=
 =?utf-8?B?aWgwTnFyeFMvcUJ6Kzc5OVZDY1FXMkZKOTF2TWdhT3djMUtudVNETzQxeDd2?=
 =?utf-8?B?c0s5TkE3bFhVd2NEWVBxMk1Pd3J4MThKTmhLWllZRnYvUDBxMzlCZHdpa2FJ?=
 =?utf-8?B?MUxJVHdEUGE4SSt0Z2U5cmhyVHRGenp3RHk5bUNEYnBpazlnbVJWKzJGRUQ2?=
 =?utf-8?B?ZUNmVnJtaExlZTFONGFyZDgvUkxMZENzNzhLOStYSzZ2TGRyanpjUFB5enlj?=
 =?utf-8?B?anN6ekFNM2wrM2RzS3lRdkwvRFM4UHB6NXVzeWVRQ3JWTW93ejNMS01YZ3JH?=
 =?utf-8?B?Y0haMWh3a3ZaZ0ljaTBLOGFQNFVlQ1NhMHJXVUlHT1FRNGpKQjFWSDBtQkFt?=
 =?utf-8?B?OHRXQldRaTZaTnpFY0h4TmFqQlRVUjA4aFRVUWxCN29zOCtlajZ3YVR2aTB0?=
 =?utf-8?B?MTJyeTNqR3g5YjkwMTRrcHRzeGhqZDBLN1lKNjZpd29FNjZKd1ZWSE5TZEhO?=
 =?utf-8?B?VzlUQUlKajRSU1N4czhWdWdxK1llU2RoNS9KT0VFa3RLWkFFRENRSjAzU1Vo?=
 =?utf-8?B?Y01zOFFMODJPaEQ4bTkrbkdzd2ZjRGEvTVBYZlN3RXdWN1V1U25MWG1sN0NM?=
 =?utf-8?B?bmtQYm40c3JmZVowL0xnYmg2VWd3VEcxTzMyQzREWE9kRm5kT2plczExS1Fm?=
 =?utf-8?B?RDVrbzkzcG1odGZSZWE0TlZsSzU4bGZpM3FDQXBFYlVibitaRVEwMDlqd0xZ?=
 =?utf-8?B?RWpETGp2L1VQRVlHQ0RJT0NxTWltZWxwU1hhSlNmbFBSZTFCNUppb2hFMTJn?=
 =?utf-8?B?Y0tPVVpRL0w1U21EU2FIQVhkVEphdGdjOGJYcEdFc1RjVnZ2S1FnT1ZFamdR?=
 =?utf-8?B?eWdBbmhyYTcxK09QSFk1T3BCUWVsTDhUSVVMSzIyTlVmRXV0T1UrQTJSRW5P?=
 =?utf-8?B?bTJrZnluNm1vNTkyaW5aS1psTFV4QWN2RnZuNHE3Z0RFK1MwalV4RXVaeWt3?=
 =?utf-8?B?U1RvMWxYYjh0QW9QR3B4c0dsVmVPRFlJR3pITFhCWFJTaFNEcWgvK1lNbFdx?=
 =?utf-8?B?b3R2a0l6YktNWmRMcU9jYndBWFUvWHNzenk0TEVOUEs3QWlMcFU2RnZtaGVD?=
 =?utf-8?B?c0E9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	UKTJAUO3q0sghuFUKdNK7rSr3+26ZMaipkdPiZvhJNc6wEVZhwno+clU5y4O/Pp2HdYW9iY9QMLMMu16TGOXKV3oMqO6rCXomRyHdIh/WZfnTafxZz4bl4UmIM1aGNnza2BXXAKWMZhOy89w23kxxkLdAvcbSne+mnYN+gYyu7tBvhFK8HSJ0LyLN22ccrn8smNq6J3SS6TQxzSGOnTYS6o7//CFbHNaxMIx5yrdTbVTyM7juYmbzAKLezaOGOmFtu14XqszW5g5ilg3xVRAKAtNPlQ1UVvjqyIbahZr71mY+wKwbf0jJSiIyb3sQPinnCkfuOvXDXSGFWGk7WdKk6Kg2elkAL0MtyuzKyp8WFRxFURwuezmn8qYPRAVucqDnqEPXAVEdNx1UOd3F7bpYuRtRa2xDXL7LN8lY4qFNaaFTk2rnxKAhk7pgfiAPoaQtLchXox5YiNwAymXCHJi9qo4iFJIjwEDrTiPRcLepaAWdUPt4qYGcDZXXMW8XftWdeFHEbUV7EDIED/sBj8hfOAGscwNl2qU6E7IYjlsm9EBer516R1/bQotVqeWKIlKz5QLaotXqC9hIEHT5GXx1HmqlLAFlMZe5/AytMHkTeg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 58489bb1-ca80-451c-ae61-08dc80893636
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 09:16:39.0204
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: B5GSa65PHgdF5oTKzy44z/QFhMi5Un1nMJTu2pndw9ivSAwVhBMG/01UeobuMRTKqOVi2hMHLSMrwmVHN7oHSQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB5706
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-30_06,2024-05-28_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 bulkscore=0
 suspectscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405300069
X-Proofpoint-GUID: j10R_a81KfEQxABw76oXQWImEvLQnw5t
X-Proofpoint-ORIG-GUID: j10R_a81KfEQxABw76oXQWImEvLQnw5t

On 29/05/2024 06:04, Christoph Hellwig wrote:
> Assign all queue limits through a local queue_limits variable and
> queue_limits_commit_update so that we can't race updating them from
> multiple places, and free the queue when updating them so that
> in-progress I/O submissions don't see half-updated limits.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>   drivers/scsi/sd.c     | 126 ++++++++++++++++++++++++------------------
>   drivers/scsi/sd.h     |   6 +-
>   drivers/scsi/sd_zbc.c |  15 ++---
>   3 files changed, 84 insertions(+), 63 deletions(-)
> 
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 2d08b69154b995..03e67936b27928 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -101,12 +101,13 @@ MODULE_ALIAS_SCSI_DEVICE(TYPE_ZBC);
>   
>   #define SD_MINORS	16
>   
> -static void sd_config_discard(struct scsi_disk *, unsigned int);
> -static void sd_config_write_same(struct scsi_disk *);
> +static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
> +		unsigned int mode);

Are there any reasons why we keep forward declarations like this? 
AFAICS, this sd_config_discard forward declaration could be removed.

> +static void sd_config_write_same(struct scsi_disk *sdkp,
> +		struct queue_limits *lim);
>   static int  sd_revalidate_disk(struct gendisk *);
>   static void sd_unlock_native_capacity(struct gendisk *disk);
>   static void sd_shutdown(struct device *);
> -static void sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer);
>   static void scsi_disk_release(struct device *cdev);
>   
>   static DEFINE_IDA(sd_index_ida);
> @@ -456,7 +457,8 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
>   {
>   	struct scsi_disk *sdkp = to_scsi_disk(dev);
>   	struct scsi_device *sdp = sdkp->device;
> -	int mode;
> +	struct queue_limits lim;
> +	int mode, err;
>   
>   	if (!capable(CAP_SYS_ADMIN))
>   		return -EACCES;
> @@ -472,8 +474,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
>   	if (mode < 0)
>   		return -EINVAL;
>   
> -	sd_config_discard(sdkp, mode);
> -
> +	lim = queue_limits_start_update(sdkp->disk->queue);
> +	sd_config_discard(sdkp, &lim, mode);
> +	blk_mq_freeze_queue(sdkp->disk->queue);
> +	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
> +	blk_mq_unfreeze_queue(sdkp->disk->queue);
> +	if (err)
> +		return err;
>   	return count;
>   }
>   static DEVICE_ATTR_RW(provisioning_mode);
> @@ -556,6 +563,7 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
>   {
>   	struct scsi_disk *sdkp = to_scsi_disk(dev);
>   	struct scsi_device *sdp = sdkp->device;
> +	struct queue_limits lim;
>   	unsigned long max;
>   	int err;
>   
> @@ -577,8 +585,13 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
>   		sdkp->max_ws_blocks = max;
>   	}
>   
> -	sd_config_write_same(sdkp);
> -
> +	lim = queue_limits_start_update(sdkp->disk->queue);
> +	sd_config_write_same(sdkp, &lim);
> +	blk_mq_freeze_queue(sdkp->disk->queue);
> +	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
> +	blk_mq_unfreeze_queue(sdkp->disk->queue);
> +	if (err)
> +		return err;
>   	return count;
>   }
>   static DEVICE_ATTR_RW(max_write_same_blocks);
> @@ -827,17 +840,15 @@ static void sd_disable_discard(struct scsi_disk *sdkp)
>   	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
>   }
>   
> -static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
> +static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
> +		unsigned int mode)
>   {
> -	struct request_queue *q = sdkp->disk->queue;
>   	unsigned int logical_block_size = sdkp->device->sector_size;
>   	unsigned int max_blocks = 0;
>   
> -	q->limits.discard_alignment =
> -		sdkp->unmap_alignment * logical_block_size;
> -	q->limits.discard_granularity =
> -		max(sdkp->physical_block_size,
> -		    sdkp->unmap_granularity * logical_block_size);
> +	lim->discard_alignment = sdkp->unmap_alignment * logical_block_size;
> +	lim->discard_granularity = max(sdkp->physical_block_size,
> +			sdkp->unmap_granularity * logical_block_size);
>   	sdkp->provisioning_mode = mode;
>   
>   	switch (mode) {
> @@ -875,7 +886,8 @@ static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
>   		break;
>   	}
>   
> -	blk_queue_max_discard_sectors(q, max_blocks * (logical_block_size >> 9));
> +	lim->max_hw_discard_sectors = max_blocks *
> +		(logical_block_size >> SECTOR_SHIFT);
>   }
>   
>   static void *sd_set_special_bvec(struct request *rq, unsigned int data_len)
> @@ -1010,9 +1022,9 @@ static void sd_disable_write_same(struct scsi_disk *sdkp)
>   	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
>   }
>   
> -static void sd_config_write_same(struct scsi_disk *sdkp)
> +static void sd_config_write_same(struct scsi_disk *sdkp,
> +		struct queue_limits *lim)
>   {
> -	struct request_queue *q = sdkp->disk->queue;
>   	unsigned int logical_block_size = sdkp->device->sector_size;
>   
>   	if (sdkp->device->no_write_same) {
> @@ -1066,8 +1078,8 @@ static void sd_config_write_same(struct scsi_disk *sdkp)
>   	}
>   
>   out:
> -	blk_queue_max_write_zeroes_sectors(q, sdkp->max_ws_blocks *
> -					 (logical_block_size >> 9));
> +	lim->max_write_zeroes_sectors =
> +		sdkp->max_ws_blocks * (logical_block_size >> 9);

Would it be ok to use SECTOR_SHIFT here? A similar change is made in 
sd_config_discard(), above

>   }
>   
>   static blk_status_t sd_setup_flush_cmnd(struct scsi_cmnd *cmd)
> @@ -2523,7 +2535,7 @@ static void read_capacity_error(struct scsi_disk *sdkp, struct scsi_device *sdp,
>   #define READ_CAPACITY_RETRIES_ON_RESET	10
>   
>   static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
> -						unsigned char *buffer)
> +		struct queue_limits *lim, unsigned char *buffer)
>   {
>   	unsigned char cmd[16];
>   	struct scsi_sense_hdr sshdr;
> @@ -2597,7 +2609,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
>   
>   	/* Lowest aligned logical block */
>   	alignment = ((buffer[14] & 0x3f) << 8 | buffer[15]) * sector_size;
> -	blk_queue_alignment_offset(sdp->request_queue, alignment);
> +	lim->alignment_offset = alignment;
>   	if (alignment && sdkp->first_scan)
>   		sd_printk(KERN_NOTICE, sdkp,
>   			  "physical block alignment offset: %u\n", alignment);
> @@ -2608,7 +2620,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
>   		if (buffer[14] & 0x40) /* LBPRZ */
>   			sdkp->lbprz = 1;
>   
> -		sd_config_discard(sdkp, SD_LBP_WS16);
> +		sd_config_discard(sdkp, lim, SD_LBP_WS16);
>   	}
>   
>   	sdkp->capacity = lba + 1;
> @@ -2711,13 +2723,14 @@ static int sd_try_rc16_first(struct scsi_device *sdp)
>    * read disk capacity
>    */
>   static void
> -sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
> +sd_read_capacity(struct scsi_disk *sdkp, struct queue_limits *lim,
> +		unsigned char *buffer)
>   {
>   	int sector_size;
>   	struct scsi_device *sdp = sdkp->device;
>   
>   	if (sd_try_rc16_first(sdp)) {
> -		sector_size = read_capacity_16(sdkp, sdp, buffer);
> +		sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
>   		if (sector_size == -EOVERFLOW)
>   			goto got_data;
>   		if (sector_size == -ENODEV)
> @@ -2737,7 +2750,7 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
>   			int old_sector_size = sector_size;
>   			sd_printk(KERN_NOTICE, sdkp, "Very big device. "
>   					"Trying to use READ CAPACITY(16).\n");
> -			sector_size = read_capacity_16(sdkp, sdp, buffer);
> +			sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
>   			if (sector_size < 0) {
>   				sd_printk(KERN_NOTICE, sdkp,
>   					"Using 0xffffffff as device size\n");
> @@ -2796,9 +2809,8 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
>   		 */
>   		sector_size = 512;
>   	}
> -	blk_queue_logical_block_size(sdp->request_queue, sector_size);
> -	blk_queue_physical_block_size(sdp->request_queue,
> -				      sdkp->physical_block_size);
> +	lim->logical_block_size = sector_size;
> +	lim->physical_block_size = sdkp->physical_block_size;
>   	sdkp->device->sector_size = sector_size;
>   
>   	if (sdkp->capacity > 0xffffffff)
> @@ -3220,11 +3232,11 @@ static unsigned int sd_discard_mode(struct scsi_disk *sdkp)
>   	return SD_LBP_DISABLE;
>   }
>   
> -/**
> - * sd_read_block_limits - Query disk device for preferred I/O sizes.
> - * @sdkp: disk to query
> +/*
> + * Query disk device for preferred I/O sizes.
>    */
> -static void sd_read_block_limits(struct scsi_disk *sdkp)
> +static void sd_read_block_limits(struct scsi_disk *sdkp,
> +		struct queue_limits *lim)
>   {
>   	struct scsi_vpd *vpd;
>   
> @@ -3258,7 +3270,7 @@ static void sd_read_block_limits(struct scsi_disk *sdkp)
>   			sdkp->unmap_alignment =
>   				get_unaligned_be32(&vpd->data[32]) & ~(1 << 31);
>   
> -		sd_config_discard(sdkp, sd_discard_mode(sdkp));
> +		sd_config_discard(sdkp, lim, sd_discard_mode(sdkp));
>   	}
>   
>    out:
> @@ -3278,10 +3290,10 @@ static void sd_read_block_limits_ext(struct scsi_disk *sdkp)
>   }
>   
>   /**

below is not a kernel doc comment

> - * sd_read_block_characteristics - Query block dev. characteristics
> - * @sdkp: disk to query
> + * Query block dev. characteristics
>    */
> -static void sd_read_block_characteristics(struct scsi_disk *sdkp)
> +static void sd_read_block_characteristics(struct scsi_disk *sdkp,
> +		struct queue_limits *lim)
>   {
>   	struct request_queue *q = sdkp->disk->queue;
>   	struct scsi_vpd *vpd;
> @@ -3307,29 +3319,26 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
>   
>   #ifdef CONFIG_BLK_DEV_ZONED /* sd_probe rejects ZBD devices early otherwise */
>   	if (sdkp->device->type == TYPE_ZBC) {
> -		/*
> -		 * Host-managed.
> -		 */
> -		disk_set_zoned(sdkp->disk);
> +		lim->zoned = true;
>   
>   		/*
>   		 * Per ZBC and ZAC specifications, writes in sequential write
>   		 * required zones of host-managed devices must be aligned to
>   		 * the device physical block size.
>   		 */
> -		blk_queue_zone_write_granularity(q, sdkp->physical_block_size);
> +		lim->zone_write_granularity = sdkp->physical_block_size;
>   	} else {
>   		/*
>   		 * Host-aware devices are treated as conventional.
>   		 */
> -		WARN_ON_ONCE(blk_queue_is_zoned(q));
> +		lim->zoned = false;
>   	}
>   #endif /* CONFIG_BLK_DEV_ZONED */
>   
>   	if (!sdkp->first_scan)
>   		return;
>   
> -	if (blk_queue_is_zoned(q))
> +	if (lim->zoned)
>   		sd_printk(KERN_NOTICE, sdkp, "Host-managed zoned block device\n");
>   	else if (sdkp->zoned == 1)
>   		sd_printk(KERN_NOTICE, sdkp, "Host-aware SMR disk used as regular disk\n");
> @@ -3605,8 +3614,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
>   	struct scsi_device *sdp = sdkp->device;
>   	struct request_queue *q = sdkp->disk->queue;
>   	sector_t old_capacity = sdkp->capacity;
> +	struct queue_limits lim;
>   	unsigned char *buffer;
>   	unsigned int dev_max;
> +	int err;
>   
>   	SCSI_LOG_HLQUEUE(3, sd_printk(KERN_INFO, sdkp,
>   				      "sd_revalidate_disk\n"));
> @@ -3627,12 +3638,14 @@ static int sd_revalidate_disk(struct gendisk *disk)
>   
>   	sd_spinup_disk(sdkp);
>   
> +	lim = queue_limits_start_update(sdkp->disk->queue);
> +
>   	/*
>   	 * Without media there is no reason to ask; moreover, some devices
>   	 * react badly if we do.
>   	 */
>   	if (sdkp->media_present) {
> -		sd_read_capacity(sdkp, buffer);
> +		sd_read_capacity(sdkp, &lim, buffer);
>   		/*
>   		 * Some USB/UAS devices return generic values for mode pages
>   		 * until the media has been accessed. Trigger a READ operation
> @@ -3651,10 +3664,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
>   
>   		if (scsi_device_supports_vpd(sdp)) {
>   			sd_read_block_provisioning(sdkp);
> -			sd_read_block_limits(sdkp);
> +			sd_read_block_limits(sdkp, &lim);
>   			sd_read_block_limits_ext(sdkp);
> -			sd_read_block_characteristics(sdkp);
> -			sd_zbc_read_zones(sdkp, buffer);
> +			sd_read_block_characteristics(sdkp, &lim);
> +			sd_zbc_read_zones(sdkp, &lim, buffer);
>   			sd_read_cpr(sdkp);
>   		}
>   
> @@ -3683,28 +3696,33 @@ static int sd_revalidate_disk(struct gendisk *disk)
>   	q->limits.max_dev_sectors = logical_to_sectors(sdp, dev_max);


is setting q->limits.max_dev_sectors directly proper?

>   
>   	if (sd_validate_min_xfer_size(sdkp))
> -		blk_queue_io_min(sdkp->disk->queue,
> -				 logical_to_bytes(sdp, sdkp->min_xfer_blocks));
> +		lim.io_min = logical_to_bytes(sdp, sdkp->min_xfer_blocks);
>   	else
> -		blk_queue_io_min(sdkp->disk->queue, 0);
> +		lim.io_min = 0;
>   
>   	/*
>   	 * Limit default to SCSI host optimal sector limit if set. There may be
>   	 * an impact on performance for when the size of a request exceeds this
>   	 * host limit.
>   	 */
> -	q->limits.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
> +	lim.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
>   	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
> -		q->limits.io_opt = min_not_zero(q->limits.io_opt,
> +		lim.io_opt = min_not_zero(lim.io_opt,
>   				logical_to_bytes(sdp, sdkp->opt_xfer_blocks));
>   	}
>   
>   	sdkp->first_scan = 0;
>   
>   	set_capacity_and_notify(disk, logical_to_sectors(sdp, sdkp->capacity));
> -	sd_config_write_same(sdkp);
> +	sd_config_write_same(sdkp, &lim);
>   	kfree(buffer);
>   



From xen-devel-bounces@lists.xenproject.org Thu May 30 09:29:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:29:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732589.1138576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCc6K-0002o6-Gy; Thu, 30 May 2024 09:29:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732589.1138576; Thu, 30 May 2024 09:29:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCc6K-0002nz-EO; Thu, 30 May 2024 09:29:36 +0000
Received: by outflank-mailman (input) for mailman id 732589;
 Thu, 30 May 2024 09:29:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCc6J-0002np-JJ; Thu, 30 May 2024 09:29:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCc6J-000423-ET; Thu, 30 May 2024 09:29:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCc6J-0005cA-7Y; Thu, 30 May 2024 09:29:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCc6J-0002C3-7F; Thu, 30 May 2024 09:29:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0ofYnzuvhnSLzV2/s9+A7Og4rysFWZWtQSWU79dp/fs=; b=vdbPjlIR209udAjwf32MJvjU62
	oJYq2f4owA7qUy2s/Nvw1caIUf5fUUCnjNdkDI+N9PMoL0a2xhOrHXSp/XehB9RHPRDlPLdk8vy8q
	uYz+QIeEZfzOsSr4B5wIGoGE9PM66Dc8oSxgoAVSMonOb4prbA/HWvroOOyUtxbhzypU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186193-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186193: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=cd4cebabf5834c403c9d9ac4f162e8336024bbcd
X-Osstest-Versions-That:
    ovmf=843f2d0964bd0444fa6bdbb1ee79dc838cfa4452
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 09:29:35 +0000

flight 186193 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186193/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 cd4cebabf5834c403c9d9ac4f162e8336024bbcd
baseline version:
 ovmf                 843f2d0964bd0444fa6bdbb1ee79dc838cfa4452

Last test of basis   186191  2024-05-30 04:41:08 Z    0 days
Testing same since   186193  2024-05-30 07:11:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gua Guo <gua.guo@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   843f2d0964..cd4cebabf5  cd4cebabf5834c403c9d9ac4f162e8336024bbcd -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 09:41:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:41:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732599.1138586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcHg-0005Tk-H2; Thu, 30 May 2024 09:41:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732599.1138586; Thu, 30 May 2024 09:41:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcHg-0005Tc-Dx; Thu, 30 May 2024 09:41:20 +0000
Received: by outflank-mailman (input) for mailman id 732599;
 Thu, 30 May 2024 09:41:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lthE=NB=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sCcHe-0005TW-SF
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 09:41:18 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04on2062.outbound.protection.outlook.com [40.107.7.62])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c26ed9a8-1e68-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 11:41:16 +0200 (CEST)
Received: from DB8PR09CA0001.eurprd09.prod.outlook.com (2603:10a6:10:a0::14)
 by GV1PR08MB7777.eurprd08.prod.outlook.com (2603:10a6:150:55::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Thu, 30 May
 2024 09:40:43 +0000
Received: from DU2PEPF00028D00.eurprd03.prod.outlook.com
 (2603:10a6:10:a0:cafe::35) by DB8PR09CA0001.outlook.office365.com
 (2603:10a6:10:a0::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21 via Frontend
 Transport; Thu, 30 May 2024 09:40:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028D00.mail.protection.outlook.com (10.167.242.184) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15
 via Frontend Transport; Thu, 30 May 2024 09:40:42 +0000
Received: ("Tessian outbound 7e38650a3f1f:v327");
 Thu, 30 May 2024 09:40:42 +0000
Received: from 4d74ecbe6518.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D9D66A29-D9D7-47A0-B000-7BC43094032B.1; 
 Thu, 30 May 2024 09:40:35 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4d74ecbe6518.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 May 2024 09:40:35 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by PAWPR08MB8789.eurprd08.prod.outlook.com (2603:10a6:102:332::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Thu, 30 May
 2024 09:40:33 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Thu, 30 May 2024
 09:40:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c26ed9a8-1e68-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=EX8AXS8dlzrpgaEfTKCePlXy+HEwrTPcXFB09Wkl2+hDVnYt5e3kTUrhI8EuC3f6FfM9LTrLwaeNA/01YjsRFxA3nZr935gZE0FnzEa7PYbTDpOPmLpZP9DUuGbHgjBAGmZgMhA+A3drnMntHNOemly2aJdy0yrlVhXOZJuiJnWi0atimPHEcTllqwjP3iTRJ0mD3YlPoIXtyxL7NNiF4q1s+APlM4h2Xukl9nSuXe9LkT4P9RjwNHw2zLn6OfphpwC5Dj0vLQC7HJxK+mFEAg8is91m7RE22WJowJ7gwDHiXqc4NMspOjQ5sfPRb2M8kq4pB76YaSwDqEYNLuOSJA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RERca++Jkv+bFKsXzSH4ddrRDHhcxvb/w+O8l/W0HLM=;
 b=LhLeH4Cy4WZgDo54HA9c4NCt8qWdnAqAVolRaQGCOK9xWkWhcgTbpgdwG2ovyg6HrmI/29ebS/cMnGqiYVCw3QNI/o6s71FFe2b3zNWXJQCcC75Ov3BQf97AJsplF8YchsJFLPgsu0rCyZhQqJGIzGs+tE3WPvgtmA1PYV3s3OBEyp4MvTAFvFwyzIt2PvUQmbiLxG0ZQuWdD0rh/JXdV8AJ6dvTndAj7zoO7YbU1vJGFhaulvOE2OLN3PAmj+ynd3sKQt4mtVmvTxi1zvAtrTxB3UzwwAJuM8wxRpM8RJoNN3QVeUQbtmR90a8v7/wjN8oUVVzCeB04YEtgeuDzNA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RERca++Jkv+bFKsXzSH4ddrRDHhcxvb/w+O8l/W0HLM=;
 b=HW8aXt9yRHt67d6LUS2QcWoKu1uzMbs+YcV/yLem7IHsDwFXCqdWs+Up61n7iFYn8x/oo0hNHaDzZYsAE/hfQO5v6zR5hCP6b5Kf2iG9qKOZkQUaPxPO0/ywGiZfZzJxTqbQG/96gj7jjjVN0MbAeVF+BGJti9Ta51IMY8YlD0g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8f4ae92c69ae20ca
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FuzZGisy3/XVMCNDMTNUFaEhAXiwoR6/M8Ed1GvskTmnDttqnKWKF4ty+GtUbk33c3JLGkolwLvU9OFCcDlkHhOjT8UozZGCV3e9tpTPRm06eoDMBHn5VaWf2D6ATpAYP7g8eUXIvOj7S4hag2+K3vFPcOKytWcKybKuBurJD/B+dTYxtrjB7N6jVoLkjqhSHPzhEPE74vAtlGCa3pMNNWKCXhxwqgm6lbiww5NaReiEoMH95iAsn/mO+IkY4ouKxop2YLpH9E+D2vhBkZ0RbOqaei7xBZyzpLsWFLI6rtFYS+lPlsvMEma9owRA4guR5GrDcuzRRq3wCJXtVNnJgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RERca++Jkv+bFKsXzSH4ddrRDHhcxvb/w+O8l/W0HLM=;
 b=FhDKbGctvvHsUA79WVdtpAFAHnCVOrB+cWBDImnUSaNqsb+NHpEBjEH88pdk92z8HXp/gtMSqviuPaf1vCO3MS9Pb/Ksa0FbRqHXDciRDzDbaWrxecSYOrnVF9LY9GKbgjAVak9feqan9VXN92lEu1slYAahDzw5BOob6/jrBySp6LFOfwcrFp8ROgjUj8Qa4r6uVYjl83fA2RVFBEKpXDCvUEE9G2hdZA45JX1q4tZgzmt62aJewdd6jfMdZaT0nmLiCKxC71gNNdYBvVD8GXRwYNc8++pxy2SY9pE7OzduHekXRpBOtj7Qqv1azZqlfD6zeCkgX3l0AEu9o3a5yw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RERca++Jkv+bFKsXzSH4ddrRDHhcxvb/w+O8l/W0HLM=;
 b=HW8aXt9yRHt67d6LUS2QcWoKu1uzMbs+YcV/yLem7IHsDwFXCqdWs+Up61n7iFYn8x/oo0hNHaDzZYsAE/hfQO5v6zR5hCP6b5Kf2iG9qKOZkQUaPxPO0/ywGiZfZzJxTqbQG/96gj7jjjVN0MbAeVF+BGJti9Ta51IMY8YlD0g=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Julien Grall <julien@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] arm: dom0less: add TEE support
Thread-Topic: [PATCH] arm: dom0less: add TEE support
Thread-Index: AQHasgjZKwS4pdCyJUe9LewXYMLHeLGutaSAgAAAkICAANCCAA==
Date: Thu, 30 May 2024 09:40:33 +0000
Message-ID: <CBB42DF2-94AE-4566-B83D-3AB3CB2E01DE@arm.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
In-Reply-To: <87bk4oxpxk.fsf@epam.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|PAWPR08MB8789:EE_|DU2PEPF00028D00:EE_|GV1PR08MB7777:EE_
X-MS-Office365-Filtering-Correlation-Id: 08d9ea9a-847a-4708-718a-08dc808c92b0
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|366007|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?00KzUZtDkSDB27//bZRoxGA64zHlyNkS455319MKBup9eM3zEU3jASYvHgzG?=
 =?us-ascii?Q?0RwK+joQ5r+DfFpKAni5XfOXDyXIC4BaU072Y5aCaKXhWNlfNjVHn+5ht3Lw?=
 =?us-ascii?Q?EImUtMGzwwnNCtDCZ7vRswGqkwBUZvPXDratnquKVfDTv5s9YxfF4dbqWHaJ?=
 =?us-ascii?Q?ELZg+Eod1FFgU3ZG7uhHCmO/GfdJxBEi3ps+kQfB7wENLqRyeE1SYCuV1gbz?=
 =?us-ascii?Q?oCoYoTDtfswSel84gQEFuCtEgGkZRGJKw+5vRlvWcFhN/CnmAJ24AO2UtHkY?=
 =?us-ascii?Q?Xkbknxyp1YYBm9pYI15kOUpOkadTFhg56rVf3PvieFiF3z3jsHR67gyOKiGt?=
 =?us-ascii?Q?OwOEGH1nAjfEgkgqQ5/r3+VgsT8+AXgufaWUE5Q3tJJwEJDtNm6GuR6ef2J1?=
 =?us-ascii?Q?McbKevhIaq9PGcAIftSkcxE61yhNn6HETnR8Hjfopv66+8kWTsiOx/2d/1RA?=
 =?us-ascii?Q?PRtliqvGNM3dqbQc6DcqEjsHEBkZz+8EAwSgOaENIROvKZy8l46gmxN3R6dx?=
 =?us-ascii?Q?lSx0OfWDMW7WnyaqfYXxNQz8aK6Rj/9kiOlI0NXNFaEuB+MVEcFIg7cQtx2F?=
 =?us-ascii?Q?0NHINMKJeIdq9lzogPDi2lAI7mqGWdOSQOeESu/PxN05dCImWAQ3SJa5dDBr?=
 =?us-ascii?Q?MNTjvjm+HSd9Pu5i8k8tfGT88YVJmsXC3qE4QZgdS6kubtvE7KuRDxZlBKl7?=
 =?us-ascii?Q?FpKuwc+eLA0jKcPLoIEzfE1ozRsvmvYmBI7ZmyHOVng50XXgf8uFz/J8/ZcO?=
 =?us-ascii?Q?1H6IclYsAtWbMHY/v9FmFaTHYzkQ2aDXNT+PZti29V96mDbuTvNs80MioRxm?=
 =?us-ascii?Q?8veVANpCU0KvQhj8az1/hgjCq+lqPSUh93aHGcWH24mpegJqCU0h5/htbpnY?=
 =?us-ascii?Q?8qEz65xYiFk0V0vVzxgmbXhnRTuydhl9lJNKbCKz4jpMvmOI+OrGcuwfzQ0M?=
 =?us-ascii?Q?UMaaUHr3SDm1CPl+uXbnSSUupD7thbr4sDbCeoypG3RZ7NhqzznShRs9o4qf?=
 =?us-ascii?Q?ttL4VrsVBh7DF738pEtXfMlQrLFD9U98L6xwG1nwLSCnIgMPK02meaZe9+sj?=
 =?us-ascii?Q?//qqVFCfUQegHuB2XG//lCEFSWrVWXqPzjXSWc3BykS2j86SmsdTv/VSAmUq?=
 =?us-ascii?Q?j8ZpF1L9STKusuZWih/6P4HuRatUsB4Wh3I8Pf4DX1GHv/Z5BWjtZi5p8miV?=
 =?us-ascii?Q?qFAmtC96DKEfcjdjC54LyiUlGCNqP1R0IbPuKA1jTE6pKHF/MDeqBdSjuXOa?=
 =?us-ascii?Q?SFkfLLhuoKGetoAOCaxgS2ACvCcF9YBHo1ABDN1VlnzkP0ul2ShwyO7Ei3Yg?=
 =?us-ascii?Q?4MXvl7m7zlTPaxxseKgfzH94vB2V/fEojBHvLvOr1NLnDA=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <C0CA106A256ECD4386490A024D408DD8@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR08MB8789
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028D00.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d2be8740-873d-421e-8dc6-08dc808c8d24
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|35042699013|82310400017|376005|36860700004|1800799015;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?5AEb8Di9GVGDMNQYdtuun+1Vs3Hc7eN8W+HVblMUpnnINuH7zhivrz3jdd8q?=
 =?us-ascii?Q?OjXgcljkUCXN8pNGRZ8b1DJlpZM0rp/LzQ1cMQ3vBXBtn0ga6iKmgHeLNy3p?=
 =?us-ascii?Q?Zz2gCNh/dQ+IedCOZ/+wfbKzEqQNHHYQGiPBwENs4GdIn/bDHUQtkyTj8Cac?=
 =?us-ascii?Q?gYT3wM0TbBQPCHn47cPhPGmn7YmOFdVimvdtOtcIwXKNq0CvxOLGsOgm+Y1d?=
 =?us-ascii?Q?6kpjgbV0yfRHz48QcpGHOc40bb1wgpVkZGXuyAUG8AeEdz2JEflQjmVtrVkC?=
 =?us-ascii?Q?iaQN6dfk8moqLXSvQR7vvh7c8kpjVcNEv51NJtJQt//C0cZImrfzPabVjtRy?=
 =?us-ascii?Q?j7mFy8eMAbaESxoXqkJvyUb70MInzyqbt0xkjdUIJhh2kQaIhK3sUznAetcD?=
 =?us-ascii?Q?UbI8Llec92/1sR1p2NBNPqw5Ydlff2hqzsJBj5usnzRPJ+zKtvZeoQfjLf3S?=
 =?us-ascii?Q?bR6ZkiVtpzlc9XscIY1oOOzltGn7iieS3VOq6HkbBU4XFbfYLCwDy+LuBXZ4?=
 =?us-ascii?Q?C2oI3oLvSuy99EFlMuHgCqO0DUKn7HOdrM6EXvtgGsJsFEC0IiE3N01+i8/p?=
 =?us-ascii?Q?hPSKlINHcbTolCOMhJDk/4mGfzaO0/R40F3oam9FoiXXZp67oB6oWNpnxlBZ?=
 =?us-ascii?Q?NmMA17XmeMmGIqooCqfO9uFAoloxuCSMyhyj+rMaQPR53/prQQt7QYIK1vSO?=
 =?us-ascii?Q?E+Q7R6Wzh+js4RrmjZIB0ZDghC8wbSHHzCVpuqopOxwmn5sCu9KmedqMrCL9?=
 =?us-ascii?Q?2pViD3B8rVMsuyP0KDGt6b3dfoKVQO+TnorO0363eX+nwzvfBXUQ/RW6VOo0?=
 =?us-ascii?Q?+6O79lQ/27kjuh7aHHk8G9L3Yn0wy9HNdxPHQAA5BLsN3UaZQzah5z9ONR0G?=
 =?us-ascii?Q?LDt4kHuV3BERmiQbTnggg+MB1gLxzYSshnMdu9eNJxBwYfPkjYAMShBY/f6M?=
 =?us-ascii?Q?GNfjTYlfqZpUeVetsj6xIULrll/33q7YdWhotUirya77ohw/pIIFSVE9xzKo?=
 =?us-ascii?Q?ce8iD+LKCLz/Ak6NCTCOREPe9xprMlp1Go8Ta8cIGixjXJzRjC6olhi5rowM?=
 =?us-ascii?Q?KR3LDukz4DKkHu8WyfU4uBKziUnEk0x8LvBwy1aIjXLeCdzg7hcD+9D4h/Cr?=
 =?us-ascii?Q?Jqiybv17qDQSYcOtcEEjHNw4QVvU5wjK+1O8SAFvGC9gKFKxpG/rmUWXRICl?=
 =?us-ascii?Q?XRh6MglrNz1/SjRwX5T1JkqKvdZUX+PbsIcFVprpog7gStyGzyG98u2405HS?=
 =?us-ascii?Q?q6eLiMhWnbrcNOYJyG+2Z8LjYc0gPB60wE0UT6bIkg7kO6NgNkMSH5S96Vfx?=
 =?us-ascii?Q?6iMDaa0S4YDaqC8+gHG4wGq+MX8K731XRW+B7gzbWxXI7StNzX9pjWVkXhil?=
 =?us-ascii?Q?ygos+5O622d6WiQwW+LSrrBboDCz?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(35042699013)(82310400017)(376005)(36860700004)(1800799015);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 09:40:42.4581
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 08d9ea9a-847a-4708-718a-08dc808c92b0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028D00.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7777

Hi Volodomyr,

First: thanks a lot for this as having a solution for selecting the tee
for guests on a dom0less configuration was something needed.

Let me answer a bit more on the rest of the patch,

> On 29 May 2024, at 23:34, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com> =
wrote:
>=20
>=20
> Hi Julien,
>=20
> Julien Grall <julien@xen.org> writes:
>=20
>> Hi Volodymyr,
>>=20
>> Can you clarify whether this is intended for the next release cycle?
>=20
> Well, I don't think that this patch should be committed ASAP if this is
> what you are asking about.
>=20
>> On 29/05/2024 21:43, Volodymyr Babchuk wrote:
>>> Allow to provide TEE type for a Dom0less guest via "xen,tee"
>>> property. Create appropriate nodes in the guests' device tree and
>>> initialize tee subsystem for it.
>>=20
>> The new property needs to be documented in
>> docs/misc/arm/device-tree/booting.txt.
>>=20
>=20
> Yes, missed that.
>=20
>>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>> ---
>>>  xen/arch/arm/dom0less-build.c     | 69 +++++++++++++++++++++++++++++++
>>>  xen/arch/arm/include/asm/kernel.h |  3 ++
>>>  2 files changed, 72 insertions(+)
>>> diff --git a/xen/arch/arm/dom0less-build.c
>>> b/xen/arch/arm/dom0less-build.c
>>> index fb63ec6fd1..1ea3ecc45c 100644
>>> --- a/xen/arch/arm/dom0less-build.c
>>> +++ b/xen/arch/arm/dom0less-build.c
>>> @@ -15,6 +15,7 @@
>>>  #include <asm/domain_build.h>
>>>  #include <asm/static-memory.h>
>>>  #include <asm/static-shmem.h>
>>> +#include <asm/tee/tee.h>
>>>    bool __init is_dom0less_mode(void)
>>>  {
>>> @@ -277,6 +278,42 @@ static int __init make_vpl011_uart_node(struct ker=
nel_info *kinfo)
>>>  }
>>>  #endif
>>>  +#ifdef CONFIG_OPTEE
>>> +static int __init make_optee_node(struct kernel_info *kinfo)
>>=20
>> Please introduce a callback in the TEE framework that will create the
>> OPTEE node.
>=20
> This is the reason why this is RFC. I wanted to discuss the right method
> of doing this. "optee" node should reside in "/firmware/" node as per
> device tree bindings. But "/firmware/" node can contain additional
> entries, for example linux device tree bindings also define
> "/firmware/sdei". So, probably correct solution is to implement function
> "make_firmware_node()" in this file, which in turn will call TEE
> framework.
>=20

I think we need something in tee.c calling on the right tee implementation
depending on what is enabled for the guest as Julien suggested.


> But we are making assumption that all TEE implementation will have its
> node inside "/firmware/". I am not 100% sure that this is correct. For
> example I saw that Google Trusty uses "/trusty" node (directly inside
> the DTS root). On other hand, it is not defined in dts bindings, as far
> as I know.


Regarding the firmware part you can easily handle that by looking for /firm=
ware
and create it if it does not exist before creating your sub-node and this s=
hould
be node in the optee node creation function not in tee.c.

>=20
>>>  /*
>>>   * Scan device tree properties for passthrough specific information.
>>>   * Returns < 0 on error
>>> @@ -650,6 +687,15 @@ static int __init prepare_dtb_domU(struct domain *=
d, struct kernel_info *kinfo)
>>>      if ( ret )
>>>          goto err;
>>>  +#ifdef CONFIG_OPTEE
>>> +    if ( kinfo->tee_type =3D=3D XEN_DOMCTL_CONFIG_TEE_OPTEE)
>>> +    {
>>> +        ret =3D make_optee_node(kinfo);
>>> +        if ( ret )
>>> +            goto err;
>>> +    }
>>> +#endif
>>> +
>>>      /*
>>>       * domain_handle_dtb_bootmodule has to be called before the rest o=
f
>>>       * the device tree is generated because it depends on the value of
>>> @@ -743,6 +789,9 @@ static int __init construct_domU(struct domain *d,
>>>  {
>>>      struct kernel_info kinfo =3D {};
>>>      const char *dom0less_enhanced;
>>> +#ifdef CONFIG_TEE
>>> +    const char *tee;
>>> +#endif
>>>      int rc;
>>>      u64 mem;
>>>      u32 p2m_mem_mb;
>>> @@ -786,6 +835,18 @@ static int __init construct_domU(struct domain *d,
>>>      else if ( rc =3D=3D 0 && !strcmp(dom0less_enhanced, "no-xenstore")=
 )
>>>          kinfo.dom0less_feature =3D DOM0LESS_ENHANCED_NO_XS;
>>>  +#ifdef CONFIG_TEE
>>=20
>> I would rather prefer if this code is implemented in tee.c. We also...

ack

>>=20
>>> +    rc =3D dt_property_read_string(node, "xen,tee", &tee);
>>=20
>> ... want to return an error if "xen,tee" exists because CONFIG_TEE is
>> not set.

ack

>>=20
>>> +    if ( rc =3D=3D -EILSEQ ||
>>> +         rc =3D=3D -ENODATA ||
>>> +         (rc =3D=3D 0 && !strcmp(tee, "none")) )
>>> +    {
>>> +        if ( !hardware_domain )
>>=20
>>=20
>> I don't understand this check. Why would we require dom0 for OP-TEE?
>=20
> OP-TEE is enabled for Dom0 unconditionally in create_dom0(void);
>=20
> This is another topic I wanted to discuss, actually, Should we use the
> same "xen,tee" for Dom0? In this case we might want to alter Dom0 DTB to
> remove TEE entry if user wants it to be disabled.

This is only called for domU anyway so i do not think this check makes sens=
e.

More than that, the fact that we are enabling optee support by default in d=
om0
might need shortly to be revisited as one might want to have a Xen compiled
with both optee and FF-A so we should have a solution to select which one
(if any) is enabled in dom0 (command line argument or device tree entry).

>=20
>> In any case we should avoid to hide a feature behind the user back. At
>> minimum, we should print a message, but I would rather prefer a
>> panic() because the user asks for a feature and we denied it.

ack for the panic.

>>=20
>>> +            kinfo.tee_type =3D XEN_DOMCTL_CONFIG_TEE_NONE;
>> Why don't we have a else case? Are you assuming that tee_type is
>> initialized to TEE_NONE (which luckily happens to be 0)? If so, why do
>> we need the check above?
>=20
> Yes, you are right, I'll rework this part.
>=20
> [...]
>=20
>>> +    if ( kinfo.tee_type )
>>> +    {
>>> +        rc =3D tee_domain_init(d, kinfo.tee_type);
>>=20
>> Can you explain why do you need to call tee_domain_init() rather than
>> setting d_cfg.arch.tee_type just before domain_create() is called and
>> rely on the latter to call the former?
>>=20
>=20
> Because I was not familiar with dom0less code good enough. Your proposal
> is much better, I'll rework this.
>=20
> --=20
> WBR, Volodymyr

Cheers
Bertrand



From xen-devel-bounces@lists.xenproject.org Thu May 30 09:41:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:41:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732600.1138595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcHl-0005id-NW; Thu, 30 May 2024 09:41:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732600.1138595; Thu, 30 May 2024 09:41:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcHl-0005iW-Ky; Thu, 30 May 2024 09:41:25 +0000
Received: by outflank-mailman (input) for mailman id 732600;
 Thu, 30 May 2024 09:41:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCcHk-0005TW-AF
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 09:41:24 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5f44948-1e68-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 11:41:22 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-354fb2d9026so749330f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 02:41:22 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35dbf0b2a93sm1683419f8f.22.2024.05.30.02.41.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 02:41:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5f44948-1e68-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717062081; x=1717666881; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=9n3bIz0tcEFlZ832RlwXwEdMi2z/RMwB8j9S8/acoRU=;
        b=jBLHSQgoBl4JWX9dDXCI2abT1X3hcbGlc2uKkZDlyD7Be94vcvrSDPK8AEGJgWVzk7
         t0JWtA/IjhRaTOSDHO7PCJvfiutH+c4e4CXoTcnLGiX71V6KabunB2TyxpCqImtEKs+H
         YYB1tsIB6zRGffyhtVVeanUXnl03C2xpjUKr4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717062081; x=1717666881;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9n3bIz0tcEFlZ832RlwXwEdMi2z/RMwB8j9S8/acoRU=;
        b=rBLhEOU9pyEzEZ51dgcuTzrnbBJRry7VPpvd7GauMafPDFGJPeKvAoxsNcTHvxg3cY
         YFs2S40tt9awea+9Vpm/ZoxKfZE+EIPcv5vB+G6GOerJKB04U3gGUZvEVi9DzRqUv2Sw
         F9RGy3S6+wrIeVZjOmoAdFz7XvPm1nRAplpiNMEDWo16VaDxwJCrG6aEcQdcVHOxUdUm
         /V0qp3p1FSFSm6JcHXKtWj+h3hA/3JDVZJcUHDsIAtRg6/glaF8u3EOQIipmSt2YvEt0
         ghNPi4XmiD7vC7snG3LzlgRGFin7LI1kg571WvaYNfp/v5OOnlvRYJo+/GnE+lllnuGU
         /MSA==
X-Gm-Message-State: AOJu0YxSdafb4OivkqDsVI3Oktz1xRQn0DMBauEwWQmRUbYMpjgeElpf
	2kYJmscZH3FQspaPQT4o9ldZnNWkJ/cyFIO9Ea6indqyazB8bYQtTou6M3vBiII=
X-Google-Smtp-Source: AGHT+IGLHnVDII//Q3Ehl4zvoK+vnH/GyuSHrQuR2RfVzh45Uyno3MJzxqkz6LALn6RU5sAkQ0skSQ==
X-Received: by 2002:adf:ec0c:0:b0:354:f724:641d with SMTP id ffacd0b85a97d-35dc009a9dfmr850784f8f.42.1717062081195;
        Thu, 30 May 2024 02:41:21 -0700 (PDT)
Date: Thu, 30 May 2024 11:41:20 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v4 2/2] tools/xg: Clean up xend-style overrides for CPU
 policies
Message-ID: <ZlhJwB3ke5HN2VKL@macbook>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
 <ce6dc0221bbab353b6f47cf4bbbbcdf3aacc0cb8.1716992707.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ce6dc0221bbab353b6f47cf4bbbbcdf3aacc0cb8.1716992707.git.alejandro.vallejo@cloud.com>

On Wed, May 29, 2024 at 03:30:38PM +0100, Alejandro Vallejo wrote:
> Factor out policy getters/setters from both (CPUID and MSR) policy override
> functions. Additionally, use host policy rather than featureset when
> preparing the cur policy, saving one hypercall and several lines of
> boilerplate.
> 
> No functional change intended.

One change that's worth mentioning is that now the policy gets set
only once per domain, as the whole policy is prepared and uploaded to
the hypervisor, rather than uploading a partial policy which is then
further adjusted by xc_{cpuid_xend,msr}_policy() calls.

> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

> ---
> v4:
>   * Indentation adjustment.
>   * Fix unhandled corner case using bsearch() with MSR and leaf buffers
> ---
>  tools/libs/guest/xg_cpuid_x86.c | 437 ++++++++++----------------------
>  1 file changed, 130 insertions(+), 307 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 6cab5c60bb41..552ec2ab7312 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -36,6 +36,34 @@ enum {
>  #define bitmaskof(idx)      (1u << ((idx) & 31))
>  #define featureword_of(idx) ((idx) >> 5)
>  
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
> +{
> +    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> +    int rc;
> +
> +    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
> +    if ( rc )
> +    {
> +        if ( err_leaf != -1 )
> +            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> +                  err_leaf, err_subleaf, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> +                                  policy->nr_msrs, &err_msr);
> +    if ( rc )
> +    {
> +        if ( err_msr != -1 )
> +            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
> +                  err_msr, -rc, strerror(-rc));
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
>  int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
>  {
>      struct xen_sysctl sysctl = {};
> @@ -260,102 +288,37 @@ static int compare_leaves(const void *l, const void *r)
>      return 0;
>  }
>  
> -static xen_cpuid_leaf_t *find_leaf(
> -    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
> -    const struct xc_xend_cpuid *xend)
> +static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
> +                                   const struct xc_xend_cpuid *xend)
>  {
>      const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
>  
> -    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
> +    return bsearch(&key, p->leaves, p->nr_leaves,
> +                   sizeof(*p->leaves), compare_leaves);
>  }
>  
> -static int xc_cpuid_xend_policy(
> -    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
> +static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
> +                                const struct xc_xend_cpuid *xend,
> +                                xc_cpu_policy_t *host,
> +                                xc_cpu_policy_t *def,
> +                                xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    rc = -ENOMEM;
> -    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        goto fail;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_msrs = 0;
> -    nr_cur = nr_leaves;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto fail;
> -    }
> +    if ( !xend )
> +        return 0;
>  
> -    /* Get the domain type's default policy. */
> -    nr_msrs = 0;
> -    nr_def = nr_leaves;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_def, def, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto fail;
> -    }
> +    if ( !host || !def || !cur )
> +        return -EINVAL;
>  
> -    /* Get the host policy. */
> -    nr_msrs = 0;
> -    nr_host = nr_leaves;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_host, host, &nr_msrs, NULL);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    rc = -EINVAL;
>      for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
>      {
> -        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
> -        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
> -        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
> +        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
> +        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
> +        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
>  
>          if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
>          {
>              ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
> -            goto fail;
> +            return -EINVAL;
>          }
>  
>          for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
> @@ -384,7 +347,7 @@ static int xc_cpuid_xend_policy(
>                  {
>                      ERROR("Bad character '%c' in policy[%d] string '%s'",
>                            xend->policy[i][j], i, xend->policy[i]);
> -                    goto fail;
> +                    return -EINVAL;
>                  }
>  
>                  clear_bit(31 - j, cur_reg);
> @@ -394,25 +357,7 @@ static int xc_cpuid_xend_policy(
>          }
>      }
>  
> -    /* Feed the transformed currrent policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto fail;
> -    }
> -
> -    /* Success! */
> -
> - fail:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  static int compare_msr(const void *l, const void *r)
> @@ -426,104 +371,37 @@ static int compare_msr(const void *l, const void *r)
>      return lhs->idx < rhs->idx ? -1 : 1;
>  }
>  
> -static xen_msr_entry_t *find_msr(
> -    xen_msr_entry_t *msrs, unsigned int nr_msrs,
> -    uint32_t index)
> +static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
> +                                 uint32_t index)
>  {
>      const xen_msr_entry_t key = { .idx = index };
>  
> -    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
> +    return bsearch(&key, p->msrs, p->nr_msrs, sizeof(*p->msrs), compare_msr);
>  }
>  
> -
>  static int xc_msr_policy(xc_interface *xch, domid_t domid,
> -                         const struct xc_msr *msr)
> +                         const struct xc_msr *msr,
> +                         xc_cpu_policy_t *host,
> +                         xc_cpu_policy_t *def,
> +                         xc_cpu_policy_t *cur)
>  {
> -    int rc;
> -    bool hvm;
> -    xc_domaininfo_t di;
> -    unsigned int nr_leaves, nr_msrs;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    /*
> -     * Three full policies.  The host, default for the domain type,
> -     * and domain current.
> -     */
> -    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
> -    unsigned int nr_host, nr_def, nr_cur;
> -
> -    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
> -    {
> -        PERROR("Failed to obtain d%d info", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -    hvm = di.flags & XEN_DOMINF_hvm_guest;
> -
> -    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain policy info size");
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
> -         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
> -         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
> -    {
> -        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
> -        rc = -ENOMEM;
> -        goto out;
> -    }
> -
> -    /* Get the domain's current policy. */
> -    nr_leaves = 0;
> -    nr_cur = nr_msrs;
> -    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain d%d current policy", domid);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Get the domain type's default policy. */
> -    nr_leaves = 0;
> -    nr_def = nr_msrs;
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, NULL, &nr_def, def);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
> -        rc = -errno;
> -        goto out;
> -    }
> +    if ( !msr )
> +        return 0;
>  
> -    /* Get the host policy. */
> -    nr_leaves = 0;
> -    nr_host = nr_msrs;
> -    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
> -                               &nr_leaves, NULL, &nr_host, host);
> -    if ( rc )
> -    {
> -        PERROR("Failed to obtain host policy");
> -        rc = -errno;
> -        goto out;
> -    }
> +    if ( !host || !def || !cur )
> +        return -EINVAL;
>  
>      for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
>      {
> -        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
> -        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
> -        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
>          unsigned int i;
> +        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
> +        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
> +        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
>  
>          if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
>          {
>              ERROR("Missing MSR %#x", msr->index);
> -            rc = -ENOENT;
> -            goto out;
> +            return -ENOENT;
>          }
>  
>          for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
> @@ -542,8 +420,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>              {
>                  ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
>                        msr->index, msr->policy[i], msr->policy);
> -                rc = -EINVAL;
> -                goto out;
> +                return -EINVAL;
>              }
>  
>              if ( val )
> @@ -553,25 +430,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
>          }
>      }
>  
> -    /* Feed the transformed policy back up to Xen. */
> -    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
> -                                  &err_leaf, &err_subleaf, &err_msr);
> -    if ( rc )
> -    {
> -        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
> -               domid, err_leaf, err_subleaf, err_msr);
> -        rc = -errno;
> -        goto out;
> -    }
> -
> -    /* Success! */
> -
> - out:
> -    free(cur);
> -    free(def);
> -    free(host);
> -
> -    return rc;
> +    return 0;
>  }
>  
>  int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
> @@ -583,14 +442,17 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      int rc;
>      bool hvm;
>      xc_domaininfo_t di;
> -    struct xc_cpu_policy *p = xc_cpu_policy_init();
> -    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
> -    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
> -    uint32_t len = ARRAY_SIZE(host_featureset);
> +    unsigned int i;
>  
> -    if ( !p )
> -        return -ENOMEM;
> +    struct xc_cpu_policy *host = xc_cpu_policy_init();
> +    struct xc_cpu_policy *def = xc_cpu_policy_init();
> +    struct xc_cpu_policy *cur = xc_cpu_policy_init();
> +
> +    if ( !host || !def || !cur )
> +    {
> +        rc = -ENOMEM;
> +        goto out;
> +    }
>  
>      if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
>      {
> @@ -600,21 +462,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>      }
>      hvm = di.flags & XEN_DOMINF_hvm_guest;
>  
> -    /* Get the host policy. */
> -    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
> -                               &len, host_featureset);
> -    /* Tolerate "buffer too small", as we've got the bits we need. */
> -    if ( rc && errno != ENOBUFS )
> +    /* Get the raw host policy */
> +    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
> +    if ( rc )
>      {
> -        PERROR("Failed to obtain host featureset");
> +        PERROR("Failed to obtain host policy");
>          rc = -errno;
>          goto out;
>      }
>  
>      /* Get the domain's default policy. */
> -    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> -                                        : XEN_SYSCTL_cpu_policy_pv_default,
> -                               &nr_leaves, p->leaves, &nr_msrs, NULL);
> +    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
> +                                           : XEN_SYSCTL_cpu_policy_pv_default,
> +                                  def);
>      if ( rc )
>      {
>          PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
> @@ -622,14 +482,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
>          goto out;
>      }
>  
> -    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
> -                                    &err_leaf, &err_subleaf);
> -    if ( rc )
> -    {
> -        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
> -              err_leaf, err_subleaf, -rc, strerror(-rc));
> -        goto out;
> -    }
> +    /* Copy the deserialised default policy to modify it */
> +    memcpy(cur, def, sizeof(*cur));

That memcpy() worries me a bit if in the future we have dynamically
allocated fields inside the policy.  I guess we will remember to
modify this.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 09:44:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 09:44:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732609.1138605 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcL6-0006d2-AK; Thu, 30 May 2024 09:44:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732609.1138605; Thu, 30 May 2024 09:44:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcL6-0006cv-7b; Thu, 30 May 2024 09:44:52 +0000
Received: by outflank-mailman (input) for mailman id 732609;
 Thu, 30 May 2024 09:44:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCcL4-0006co-C9
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 09:44:50 +0000
Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com
 [2607:f8b0:4864:20::735])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 415ad686-1e69-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 11:44:49 +0200 (CEST)
Received: by mail-qk1-x735.google.com with SMTP id
 af79cd13be357-7948b50225bso50019085a.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 02:44:49 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd0666fsm544506885a.79.2024.05.30.02.44.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 02:44:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 415ad686-1e69-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717062288; x=1717667088; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=PTf+B5WtwnWHAd6RGmeFdfVlo8mS5DXfB1OSZeEIUzE=;
        b=orkPFcVC1KYU797FVh1FoX+zzoOROSlLzBgAHhkpUN1U5Hn2Q/cacZAkuJJ5qCl0k5
         S0FN3E89CF2IIeA2tUKXIBW/WrlpRUFpRZuELIlbtEZWpIVBxRz/9DT4/AJkRokjBlkI
         4Ohh7uj0N1caq9scYGK6ksrSEyUYvycRMX3LQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717062288; x=1717667088;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PTf+B5WtwnWHAd6RGmeFdfVlo8mS5DXfB1OSZeEIUzE=;
        b=ROuDxOtzMXnlaPuU8NYtdedyDV5n1GtIuT1WSI5KXk645HTs6I3AQuBqfNPWdSPvvs
         s1aAutTfUEKMOos28Q7RYn53z5vBWFARXVPpzV/r5D7GXJdJgd2wK1xv7SvUeQKsfYAN
         cCsgEcqbCs3UdbztpoE++XdgAsYf+iOQ0FZc8JcanlPCmpnkQw3Yp7Bvv08dP6Vi/tQ/
         pAZMYMV/zNtptjUlSGD78mD+/PmIMZxmgB21pKWn4ZnQZkAgwmxWApJSrOqM9swlVI7S
         ZXFQ+kEq4jHwdilOgAJt+To+DYOag0xjB3vHGeRUIYuYq2NB8zmTriXx0pVQw7XJeqKr
         +m7w==
X-Forwarded-Encrypted: i=1; AJvYcCWLIfpRIZd4FL3/pv4ugnWViqKmEP28nyAawX08WxxkheiyPmVQqc5oVoHWuO1YtR5I7IW/+uvG0+ziRJB+wL4SwlEkoOWS4ZYdRK+cGtA=
X-Gm-Message-State: AOJu0Yw2PmLbzR/VB1uChoPzN2N3XTUiEgOqD3hRJ4wFlOlSEM0c25VN
	NOAn1cSoaqssdnlSEGuC4SPqy130//5By2EbfHU+I8qYvX46mAOrrK4PiS1MJ4sTcJgeJzxPSy9
	KeZk=
X-Google-Smtp-Source: AGHT+IEy1b/Y7Ab1MtP0QK53j9OkEj9PZXSOZySLWo2g3NjIvDhD9sfSJsddFwzV1uIfLyASjkIzww==
X-Received: by 2002:a05:620a:1088:b0:792:e88e:f7b1 with SMTP id af79cd13be357-794e9e181d9mr151742585a.51.1717062288403;
        Thu, 30 May 2024 02:44:48 -0700 (PDT)
Message-ID: <19e240ca-6606-482c-9c0b-3c0a6c2c4e89@citrix.com>
Date: Thu, 30 May 2024 10:44:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 0/2] Clean the policy manipulation path in domain
 creation
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <cover.1716992707.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/05/2024 3:30 pm, Alejandro Vallejo wrote:
> Alejandro Vallejo (2):
>   tools/xg: Streamline cpu policy serialise/deserialise calls
>   tools/xg: Clean up xend-style overrides for CPU policies

Oleksii: Please consider for 4.19.

This is internal clean-up to CPUID handling which has been trying (one
way or another) to land for more than 3 years now.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 10:07:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 10:07:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732618.1138615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcgw-0001VJ-1i; Thu, 30 May 2024 10:07:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732618.1138615; Thu, 30 May 2024 10:07:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcgv-0001VC-VN; Thu, 30 May 2024 10:07:25 +0000
Received: by outflank-mailman (input) for mailman id 732618;
 Thu, 30 May 2024 10:07:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCcgu-0001V1-O0
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 10:07:24 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 675e577b-1e6c-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 12:07:21 +0200 (CEST)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-42121d27861so7038525e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 03:07:21 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421270ad931sm20198485e9.47.2024.05.30.03.07.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 03:07:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 675e577b-1e6c-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717063641; x=1717668441; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=2qgRoi1UruKHEcIoyHWg6Le1Z1y6p+xfsDeU7FtjGEE=;
        b=KrS/s1ddaZ7P/D5EJbVO7P/hJyrM5tguxKQaz3cRX4X88J6phxSeIefTyH7gm09i9k
         yPBZPy75frIskXYtetvq3rLRDNMBBHf0SGfsmnrNwiy2gEUXM5IoQ+TocdZhgs9YQRTy
         +xViD8Zpm5YuSdGf17Gio4zqcTK5QusGF/lcg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717063641; x=1717668441;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2qgRoi1UruKHEcIoyHWg6Le1Z1y6p+xfsDeU7FtjGEE=;
        b=sdeVKXWzxtk5shTvzto9kPXFeN22bg6hMk2RywV9PrzddW1mFeWalHSVO9dTkJg95/
         iOYD8OgBzdqLePZjE18WalRRT8dqFp5Syql6OI7mot/jY7cPzmjxKJLAahlErneWqyfv
         GWGkHmGOXLGgS169q2bUk2hIW1xw9s738V4EoQaMThlUuebVVxciDYo8D8/6lRfglOao
         Y4LscbDYx923EMy8BPTtL2FOG6oY5xdvnyZC21u4U9FvnA4RMifd8muq6AOk9q+/W/wC
         JNPAEPmkYBBPVMNqKASaApu933q+LXpOPwmPYgJ0fD/hVxMbW7RLoCQZHJddOjHrS4cm
         si+Q==
X-Gm-Message-State: AOJu0YyMbIbk3KrHRHOAqwNkimCu8xxd/kOJdvLYCpCMKAOY751uoFrB
	jGU+Fl+zJyiVIp/L78QdZqb9Q6QHHHuwhRigILDqvDX7uDcZSlw5aEBTVAIM4BU=
X-Google-Smtp-Source: AGHT+IFBD9Zjdf7eZFhoxfP6yz58jrTNxhhlpIfXv3Vc/WBsCcal9SFYt3KOi3Y2iTNj6JonFGaoIA==
X-Received: by 2002:a05:600c:4f8f:b0:418:f991:713f with SMTP id 5b1f17b1804b1-4212781b9bbmr17113555e9.23.1717063640612;
        Thu, 30 May 2024 03:07:20 -0700 (PDT)
Date: Thu, 30 May 2024 12:07:19 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v3 1/6] x86/vlapic: Move lapic migration checks to the
 check hooks
Message-ID: <ZlhP11Vvk6c1Ix36@macbook>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <f87f27de6fadf896bf4d50c8a37e2b9b7201afb2.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <f87f27de6fadf896bf4d50c8a37e2b9b7201afb2.1716976271.git.alejandro.vallejo@cloud.com>

On Wed, May 29, 2024 at 03:32:30PM +0100, Alejandro Vallejo wrote:
> While doing this, factor out checks common to architectural and hidden state.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

Reviewed-by: Roger PAu Monné <roger.pau@citrix.com>

With the BUG() possibly replaced with ASSERT_UNREACHABLE(),

> ---
> v3:
>   * Moved from v2/patch3.
>   * Added check hook for the architectural state as well.
>   * Use domain_vcpu() rather than the previous open coded checks for vcpu range.
> ---
>  xen/arch/x86/hvm/vlapic.c | 81 +++++++++++++++++++++++++--------------
>  1 file changed, 53 insertions(+), 28 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
> index 9cfc82666ae5..a0df62b5ec0a 100644
> --- a/xen/arch/x86/hvm/vlapic.c
> +++ b/xen/arch/x86/hvm/vlapic.c
> @@ -1553,60 +1553,85 @@ static void lapic_load_fixup(struct vlapic *vlapic)
>                 v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr);
>  }
>  
> -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> -{
> -    unsigned int vcpuid = hvm_load_instance(h);
> -    struct vcpu *v;
> -    struct vlapic *s;
>  
> +static int lapic_check_common(const struct domain *d, unsigned int vcpuid)
> +{
>      if ( !has_vlapic(d) )
>          return -ENODEV;
>  
>      /* Which vlapic to load? */
> -    if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
> +    if ( !domain_vcpu(d, vcpuid) )
>      {
>          dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n",
>                  d->domain_id, vcpuid);

The message here is kind of misleading as printing apic%u makes it
look like it's an APIC ID, but it's a vCPU ID.  It would be best to
just print: "HVM restore: dom%d has no vCPU %u\n".

>          return -EINVAL;
>      }
> -    s = vcpu_vlapic(v);
>  
> -    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> +    return 0;
> +}
> +
> +static int cf_check lapic_check_hidden(const struct domain *d,
> +                                       hvm_domain_context_t *h)
> +{
> +    unsigned int vcpuid = hvm_load_instance(h);
> +    struct hvm_hw_lapic s;
> +    int rc;
> +
> +    if ( (rc = lapic_check_common(d, vcpuid)) )
> +        return rc;

Nit: I don't like much to assign values inside of conditions, I would
rather do:

int rc = lapic_check_common(d, vcpuid);

if ( rc )
    return rc;

> +
> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s) != 0 )
> +        return -ENODATA;
> +
> +    /* EN=0 with EXTD=1 is illegal */
> +    if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) ==
> +         APIC_BASE_EXTD )
>          return -EINVAL;
>  
> +    return 0;
> +}
> +
> +static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
> +{
> +    unsigned int vcpuid = hvm_load_instance(h);
> +    struct vcpu *v = d->vcpu[vcpuid];

Not sure whether it's worth using domain_vcpu() here.  We have already
checked the vCPU is valid.

> +    struct vlapic *s = vcpu_vlapic(v);
> +
> +    if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 )
> +        BUG();

I would use { ASSERT_UNREACHABLE(); return -EINVAL; } here, there's
IMO no strict reason to panic on non-debug builds.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 10:14:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 10:14:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732625.1138626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcnt-0003Gj-Pa; Thu, 30 May 2024 10:14:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732625.1138626; Thu, 30 May 2024 10:14:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCcnt-0003Gc-M5; Thu, 30 May 2024 10:14:37 +0000
Received: by outflank-mailman (input) for mailman id 732625;
 Thu, 30 May 2024 10:14:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCcns-0003GW-5F
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 10:14:36 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 69602dd9-1e6d-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 12:14:34 +0200 (CEST)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-35dc04717a1so480982f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 03:14:34 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-3557a0909a7sm16887391f8f.55.2024.05.30.03.14.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 03:14:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69602dd9-1e6d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717064073; x=1717668873; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=7fFZBq18WqVghcHtWuAgoj32tpEX347GF3vAtMzqrng=;
        b=X98nEzFUl9SePVuvfkRxmdaR8CnHBnaz+EQ+HiqLQ+6wq3+CSziWYWkuEYjZdSOPja
         APwQK4bSQBme0reoQNkfxdg96HcQ5t2vdTAH74UhCqH7wA48Z+CgWWMAe+ZxMUrkg7LL
         BKGXKv6R1pp2/o/3frkz9wuHTW8tjncZOUWJ8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717064073; x=1717668873;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7fFZBq18WqVghcHtWuAgoj32tpEX347GF3vAtMzqrng=;
        b=eA10Bh0z3qUPXj2OcgRVEaVfgE5Bw2SeMZ4PakTHEypw5+jVHsGPhQS5m/cd/QkYvq
         F9UbRhPThb/bNAoGC+5SfnHP1+IGWnCJfzTOBZRTRqW+n/2z1yQb+Xnqv70UbGqncivJ
         XHz2Eq14KD2KSWA2ycYv1w9H4r/Lq042bxFnDHtWrinR1XspAGa3TFu0EfYPW7q6geLZ
         NqPygnVoL6S0+gV5Vwh5TnchXApy/TgwJTacwahYJFuSmdPgAua0ZjUrMIyTHOQnvorw
         4xm7IU6HyEwHvGMamAofTlo04JqUJAsq4gFKHnklvHY+Fpn7Y6OurP3+UHFT2Kr7eC4T
         1NQA==
X-Gm-Message-State: AOJu0Yyk00Jz4pk6hqHYHstKTWjqkiPD/J2ebw4CqrKbOK1QorpjKnic
	v9nxeVsFpXE/X02GEVXoo10LyieIlRUpZcpq6EkgaKa1KwVl/wzGqEc9/JKgEYA=
X-Google-Smtp-Source: AGHT+IHDALesgu7Kz/Z9C2Lx+ykM6bC6I76joVU+11Cv1qWrj790FpRKaF4RD/l9GMskEbJ5UfnF8A==
X-Received: by 2002:a5d:4acb:0:b0:354:db90:6df1 with SMTP id ffacd0b85a97d-35dc009cc6emr2187749f8f.38.1717064073549;
        Thu, 30 May 2024 03:14:33 -0700 (PDT)
Date: Thu, 30 May 2024 12:14:32 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
Message-ID: <ZlhRiEOvIZ9W6RoD@macbook>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>

On Wed, May 29, 2024 at 03:32:31PM +0100, Alejandro Vallejo wrote:
> This allows the initial x2APIC ID to be sent on the migration stream. The
> hardcoded mapping x2apic_id=2*vcpu_id is maintained for the time being.
> Given the vlapic data is zero-extended on restore, fix up migrations from
> hosts without the field by setting it to the old convention if zero.
> 
> x2APIC IDs are calculated from the CPU policy where the guest topology is
> defined. For the time being, the function simply returns the old
> relationship, but will eventually return results consistent with the
> topology.
> 
> Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 10:14:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 10:14:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732626.1138636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCco4-0003YB-0R; Thu, 30 May 2024 10:14:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732626.1138636; Thu, 30 May 2024 10:14:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCco3-0003Y4-Tk; Thu, 30 May 2024 10:14:47 +0000
Received: by outflank-mailman (input) for mailman id 732626;
 Thu, 30 May 2024 10:14:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCco2-0003XJ-9A
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 10:14:46 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6fee0bac-1e6d-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 12:14:45 +0200 (CEST)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-57a033c2ed4so358272a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 03:14:45 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578524bc547sm9655170a12.96.2024.05.30.03.14.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 03:14:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fee0bac-1e6d-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717064084; x=1717668884; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=tTfIIWc+srFYonFV65OGtACpB5k1PlG9WcTO4urHy48=;
        b=MA4+UFBj+s7jmKx2vOw8fCZm7oWHRetjw6et/S9VeVyRLUGDpR3laDpHmZbCS5izDo
         VDNlqMxw1P5V/XqBJfrow5FieM01vthBPMt4pbWR9YyME53pafjcu08YMdxz6znf2xTB
         rBSkPHUwwHexSn8N4WgbJs3WYBvSVMyWh84GU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717064084; x=1717668884;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=tTfIIWc+srFYonFV65OGtACpB5k1PlG9WcTO4urHy48=;
        b=gV0kixXBRSoTDOiVuWroa5642zGHWnWlkXqSW45vbwbyseRUuaAF5+vQI6ZjabSITN
         w//MpErtPUKDE2PPK/6uOKJ8L5X9Ra74tGZ3uOs1rWNYm8u3j8mndW8nzBcMuI5g66XX
         EDquCnBXd4c17KjLsqjv1nENT2wVohqTHUAX9F4TXkBhWoB4p8XrGgFvzs+VmFi/EGeF
         yTNRV17EhQlknSUvktyBy0O/i2P6cfN06W/T6xW0Al5HT/LOWZp7HZzTpiu3r9S/st3l
         NoI087VCBhuB5I7EPi2SsvrAoeR/UKlnRjAbsgPsQz3fS26bda4zF9n3ASLtHFNTZwU4
         jl+A==
X-Gm-Message-State: AOJu0YwU3MyyA8tY4kPcyf7mkPnXgujIOjp8fdoVVkIwbWpGHW1kgLM/
	N8NneMlaPL+bu/YWpdEhsUtw5zpnOtXqiZ5JNwcwGp2ozwjma1r5BAlIdOsGQkV+Km5/O5kI9zp
	aTnA=
X-Google-Smtp-Source: AGHT+IFVXhseUmJR3bV1SH3Zn0hZ9L628OT9ocnmo1k6r49pZduu4EhkKgafMhmeA+VALegv1zYZEg==
X-Received: by 2002:a50:d6d4:0:b0:574:eb0f:df9b with SMTP id 4fb4d7f45d1cf-57a17833cb5mr1503255a12.14.1717064084283;
        Thu, 30 May 2024 03:14:44 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] tools: (Actually) drop libsystemd as a dependency
Date: Thu, 30 May 2024 11:14:39 +0100
Message-Id: <20240530101439.16730-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When reinstating some of systemd.m4 between v1 and v2, I reintroduced a little
too much.  While {c,o}xenstored are indeed no longer linked against
libsystemd, ./configure still looks for it.

Drop this too.

Fixes: ae26101f6bfc ("tools: Drop libsystemd as a dependency")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony@xenproject.org>
CC: Juergen Gross <jgross@suse.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Roger Pau Monné <roger.pau@citrix.com>

Found when trying to build Xen in XenServer with libsystemd absent from the
chroot.
---
 m4/systemd.m4   |   8 --
 tools/configure | 229 +-----------------------------------------------
 2 files changed, 1 insertion(+), 236 deletions(-)

diff --git a/m4/systemd.m4 b/m4/systemd.m4
index e4fe51a8ba..ab12ea313d 100644
--- a/m4/systemd.m4
+++ b/m4/systemd.m4
@@ -86,13 +86,6 @@ AC_DEFUN([AX_CHECK_SYSTEMD], [
 	],[systemd=n])
 ])
 
-AC_DEFUN([AX_CHECK_SYSTEMD_ENABLE_AVAILABLE], [
-	PKG_CHECK_MODULES([SYSTEMD], [libsystemd-daemon], [systemd="y"],[
-		PKG_CHECK_MODULES([SYSTEMD], [libsystemd >= 209],
-				  [systemd="y"],[systemd="n"])
-	])
-])
-
 dnl Enables systemd by default and requires a --disable-systemd option flag
 dnl to configure if you want to disable.
 AC_DEFUN([AX_ENABLE_SYSTEMD], [
@@ -112,6 +105,5 @@ dnl to have systemd build libraries it will be enabled. You can always force
 dnl disable with --disable-systemd
 AC_DEFUN([AX_AVAILABLE_SYSTEMD], [
 	AX_ALLOW_SYSTEMD_OPTS()
-	AX_CHECK_SYSTEMD_ENABLE_AVAILABLE()
 	AX_CHECK_SYSTEMD()
 ])
diff --git a/tools/configure b/tools/configure
index b8faa1d520..459bfb5652 100755
--- a/tools/configure
+++ b/tools/configure
@@ -626,8 +626,6 @@ ac_subst_vars='LTLIBOBJS
 LIBOBJS
 pvshim
 ninepfs
-SYSTEMD_LIBS
-SYSTEMD_CFLAGS
 SYSTEMD_MODULES_LOAD
 SYSTEMD_DIR
 systemd
@@ -864,9 +862,7 @@ pixman_LIBS
 libzstd_CFLAGS
 libzstd_LIBS
 LIBNL3_CFLAGS
-LIBNL3_LIBS
-SYSTEMD_CFLAGS
-SYSTEMD_LIBS'
+LIBNL3_LIBS'
 
 
 # Initialize some variables set by options.
@@ -1621,10 +1617,6 @@ Some influential environment variables:
   LIBNL3_CFLAGS
               C compiler flags for LIBNL3, overriding pkg-config
   LIBNL3_LIBS linker flags for LIBNL3, overriding pkg-config
-  SYSTEMD_CFLAGS
-              C compiler flags for SYSTEMD, overriding pkg-config
-  SYSTEMD_LIBS
-              linker flags for SYSTEMD, overriding pkg-config
 
 Use these variables to override the choices made by `configure' or to help
 it to find libraries and programs with nonstandard names/locations.
@@ -3889,8 +3881,6 @@ esac
 
 
 
-
-
 
 
 
@@ -9540,223 +9530,6 @@ fi
 
 
 
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd-daemon\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd-daemon" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd-daemon\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd-daemon" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd-daemon" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd-daemon" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-	systemd="n"
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-	systemd="n"
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-	systemd="y"
-fi
-
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
-$as_echo_n "checking for SYSTEMD... " >&6; }
-
-if test -n "$SYSTEMD_CFLAGS"; then
-    pkg_cv_SYSTEMD_CFLAGS="$SYSTEMD_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_CFLAGS=`$PKG_CONFIG --cflags "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-if test -n "$SYSTEMD_LIBS"; then
-    pkg_cv_SYSTEMD_LIBS="$SYSTEMD_LIBS"
- elif test -n "$PKG_CONFIG"; then
-    if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libsystemd >= 209\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libsystemd >= 209") 2>&5
-  ac_status=$?
-  $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
-  test $ac_status = 0; }; then
-  pkg_cv_SYSTEMD_LIBS=`$PKG_CONFIG --libs "libsystemd >= 209" 2>/dev/null`
-		      test "x$?" != "x0" && pkg_failed=yes
-else
-  pkg_failed=yes
-fi
- else
-    pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
-   	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
-        _pkg_short_errors_supported=yes
-else
-        _pkg_short_errors_supported=no
-fi
-        if test $_pkg_short_errors_supported = yes; then
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        else
-	        SYSTEMD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libsystemd >= 209" 2>&1`
-        fi
-	# Put the nasty error message in config.log where it belongs
-	echo "$SYSTEMD_PKG_ERRORS" >&5
-
-	systemd="n"
-elif test $pkg_failed = untried; then
-     	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-	systemd="n"
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-	systemd="y"
-fi
-
-else
-	SYSTEMD_CFLAGS=$pkg_cv_SYSTEMD_CFLAGS
-	SYSTEMD_LIBS=$pkg_cv_SYSTEMD_LIBS
-        { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-	systemd="y"
-fi
-
-
 		if test "x$enable_systemd" != "xno"; then :
 
 	     if test "x$systemd" = "xy" ; then :
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu May 30 10:35:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 10:35:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732642.1138647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCd7x-0007Ek-OL; Thu, 30 May 2024 10:35:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732642.1138647; Thu, 30 May 2024 10:35:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCd7x-0007Ed-Je; Thu, 30 May 2024 10:35:21 +0000
Received: by outflank-mailman (input) for mailman id 732642;
 Thu, 30 May 2024 10:35:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCd7w-0007EX-EI
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 10:35:20 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCd7u-0005BO-Mr; Thu, 30 May 2024 10:35:18 +0000
Received: from [15.248.2.26] (helo=[10.24.67.18])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCd7u-00075Z-Cz; Thu, 30 May 2024 10:35:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=xCpCPIdJ11U9KOAdkbscgoHanCLOoti5gX3qByTVS7o=; b=AkMI7fD7MGIK3anj819bCjsEfN
	LW5lEZ4o3nKgfaQ26SEEtFwgP+6fLgZPummyCCJ/4LBwm5xJXkY2jrKqFvVtksYc+XRxE+TycXR9P
	YArSoUTesEIsldcBcmQcyiuvH5225T29dpWcV0bF49v7M52GLs27KjTkFM6RtgO5iiqI=;
Message-ID: <047ca670-cd9e-4803-8083-7770b7230042@xen.org>
Date: Thu, 30 May 2024 11:35:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arm: dom0less: add TEE support
Content-Language: en-GB
To: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
 <CBB42DF2-94AE-4566-B83D-3AB3CB2E01DE@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CBB42DF2-94AE-4566-B83D-3AB3CB2E01DE@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Bertrand,

On 30/05/2024 10:40, Bertrand Marquis wrote:
>> But we are making assumption that all TEE implementation will have its
>> node inside "/firmware/". I am not 100% sure that this is correct. For
>> example I saw that Google Trusty uses "/trusty" node (directly inside
>> the DTS root). On other hand, it is not defined in dts bindings, as far
>> as I know.
> 
> 
> Regarding the firmware part you can easily handle that by looking for /firmware
> and create it if it does not exist before creating your sub-node and this should
> be node in the optee node creation function not in tee.c.

This would work if the node /firmware doesn't exist. But how would you 
handle the case where it is already present?

I looked at the libfdt API and AFAICT the DTB creation needs to be 
linear. IOW, you can't add a subnode to an already created node.

There is an helper to create a placeholder, but AFAIK this is only for a 
property. You also need to know the size in advance.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 30 10:57:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 10:57:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732649.1138655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdT3-00026F-86; Thu, 30 May 2024 10:57:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732649.1138655; Thu, 30 May 2024 10:57:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdT3-000268-5P; Thu, 30 May 2024 10:57:09 +0000
Received: by outflank-mailman (input) for mailman id 732649;
 Thu, 30 May 2024 10:57:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCdT1-00025y-RY; Thu, 30 May 2024 10:57:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCdT1-0005Vq-NF; Thu, 30 May 2024 10:57:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCdT1-000262-Eu; Thu, 30 May 2024 10:57:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCdT1-0004Cs-EM; Thu, 30 May 2024 10:57:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LkacN8eRHtn8csVr92RJndKlNDZWQgEavn96dxuo0QU=; b=gbdtGJUO1bLGUL3wYJMIMJ8+ya
	/rD2z0ovnXy67c9vbN5+2ETW3Fbvb8cT1v7DQDtuUR8+FV+FM6ZyY1HJE9qORGXCdMB+26KinzSYM
	lxA24yU6vQMinBnXhLg+vo/66bzlyGRj40swLEglHVgB/zYcGn3Snp8yfUcdDrJ0h3JY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186189-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186189: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:build-amd64-xsm:xen-build:fail:regression
    linux-linus:build-amd64:xen-build:fail:regression
    linux-linus:build-i386:xen-build:fail:regression
    linux-linus:build-i386-xsm:xen-build:fail:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-uefi:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:build-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:build-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-bios:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4a4be1ad3a6efea16c56615f31117590fd881358
X-Osstest-Versions-That:
    linux=e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 10:57:07 +0000

flight 186189 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186189/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 186174
 build-amd64-xsm               6 xen-build      fail in 186185 REGR. vs. 186174
 build-amd64                   6 xen-build      fail in 186185 REGR. vs. 186174
 build-i386                    6 xen-build      fail in 186185 REGR. vs. 186174
 build-i386-xsm                6 xen-build      fail in 186185 REGR. vs. 186174

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  8 xen-boot        fail in 186185 pass in 186189
 test-armhf-armhf-libvirt      8 xen-boot                   fail pass in 186185

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl           1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-raw       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-pair         1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-examine      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)      blocked in 186185 n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)    blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)        blocked in 186185 n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)           blocked in 186185 n/a
 build-i386-libvirt            1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)           blocked in 186185 n/a
 build-amd64-libvirt           1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-pygrub       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)         blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)        blocked in 186185 n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)      blocked in 186185 n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-vhd       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-examine-bios  1 build-check(1)          blocked in 186185 n/a
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 186185 like 186174
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 186185 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 186185 never pass
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 186185 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 186185 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 186185 never pass
 test-armhf-armhf-xl-rtds      8 xen-boot                     fail  like 186174
 test-armhf-armhf-xl-credit1   8 xen-boot                     fail  like 186174
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186174
 test-armhf-armhf-examine      8 reboot                       fail  like 186174
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186174
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4a4be1ad3a6efea16c56615f31117590fd881358
baseline version:
 linux                e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc

Last test of basis   186174  2024-05-28 18:10:31 Z    1 days
Testing same since   186185  2024-05-29 17:42:32 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dominique Martinet <asmadeus@codewreck.org>
  Herbert Xu <herbert@gondor.apana.org.au>
  Linus Torvalds <torvalds@linux-foundation.org>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 4a4be1ad3a6efea16c56615f31117590fd881358
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:39:34 2024 -0700

    Revert "vfs: Delete the associated dentry when deleting a file"
    
    This reverts commit 681ce8623567ba7e7333908e9826b77145312dda.
    
    We gave it a try, but it turns out the kernel test robot did in fact
    find performance regressions for it, so we'll have to look at the more
    involved alternative fixes for Yafang Shao's Elasticsearch load issue.
    
    There were several alternatives discussed, they just weren't as simple
    as this first attempt.
    
    The report is of a -7.4% regression of filebench.sum_operations/s, which
    appears significant enough to trigger my "this patch may get reverted if
    somebody finds a performance regression on some other load" rule.
    
    So it's still the case that we should end up deleting dentries more
    aggressively - or just be better at pruning them later - but it needs a
    bit more finesse than this simple thing.
    
    Link: https://lore.kernel.org/all/202405291318.4dfbb352-oliver.sang@intel.com/
    Cc: Yafang Shao <laoar.shao@gmail.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 397a83ab978553ca2970ad1ccdbac0cdc732efd9
Merge: db163660b02a c898afdc1564
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:25:15 2024 -0700

    Merge tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux
    
    Pull 9p fixes from Dominique Martinet:
     "Two fixes headed to stable trees:
    
       - a trace event was dumping uninitialized values
    
       - a missing lock that was thought to have exclusive access, and it
         turned out not to"
    
    * tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux:
      9p: add missing locking around taking dentry fid list
      net/9p: fix uninit-value in p9_client_rpc()

commit db163660b02abbffebfad1bcd6dbce1201c72731
Merge: e0cce98fe279 67ec8cdf2997
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:12:58 2024 -0700

    Merge tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
    
    Pull crypto fix from Herbert Xu:
     "This fixes a new run-time warning triggered by tpm"
    
    * tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      hwrng: core - Remove add_early_randomness

commit 67ec8cdf29971677b2fb4b6d92871eb5d5e95597
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date:   Wed May 22 13:37:54 2024 +0800

    hwrng: core - Remove add_early_randomness
    
    A potential deadlock was reported with the config file at
    
    https://web.archive.org/web/20240522052129/https://0x0.st/XPN_.txt
    
    In this particular configuration, the deadlock doesn't exist because
    the warning triggered at a point before modules were even available.
    However, the deadlock can be real because any module loaded would
    invoke async_synchronize_full.
    
    The issue is spurious for software crypto algorithms which aren't
    themselves involved in async probing.  However, it would be hard to
    avoid for a PCI crypto driver using async probing.
    
    In this particular call trace, the problem is easily avoided because
    the only reason the module is being requested during probing is the
    add_early_randomness call in the hwrng core.  This feature is
    vestigial since there is now a kernel thread dedicated to doing
    exactly this.
    
    So remove add_early_randomness as it is no longer needed.
    
    Reported-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Reported-by: Eric Biggers <ebiggers@kernel.org>
    Fixes: 1b6d7f9eb150 ("tpm: add session encryption protection to tpm2_get_random()")
    Link: https://lore.kernel.org/r/119dc5ed-f159-41be-9dda-1a056f29888d@notapiano/
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
    Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
    Tested-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

commit c898afdc15645efb555acb6d85b484eb40a45409
Author: Dominique Martinet <asmadeus@codewreck.org>
Date:   Tue May 21 21:13:36 2024 +0900

    9p: add missing locking around taking dentry fid list
    
    Fix a use-after-free on dentry's d_fsdata fid list when a thread
    looks up a fid through dentry while another thread unlinks it:
    
    UAF thread:
    refcount_t: addition on 0; use-after-free.
     p9_fid_get linux/./include/net/9p/client.h:262
     v9fs_fid_find+0x236/0x280 linux/fs/9p/fid.c:129
     v9fs_fid_lookup_with_uid linux/fs/9p/fid.c:181
     v9fs_fid_lookup+0xbf/0xc20 linux/fs/9p/fid.c:314
     v9fs_vfs_getattr_dotl+0xf9/0x360 linux/fs/9p/vfs_inode_dotl.c:400
     vfs_statx+0xdd/0x4d0 linux/fs/stat.c:248
    
    Freed by:
     p9_fid_destroy (inlined)
     p9_client_clunk+0xb0/0xe0 linux/net/9p/client.c:1456
     p9_fid_put linux/./include/net/9p/client.h:278
     v9fs_dentry_release+0xb5/0x140 linux/fs/9p/vfs_dentry.c:55
     v9fs_remove+0x38f/0x620 linux/fs/9p/vfs_inode.c:518
     vfs_unlink+0x29a/0x810 linux/fs/namei.c:4335
    
    The problem is that d_fsdata was not accessed under d_lock, because
    d_release() normally is only called once the dentry is otherwise no
    longer accessible but since we also call it explicitly in v9fs_remove
    that lock is required:
    move the hlist out of the dentry under lock then unref its fids once
    they are no longer accessible.
    
    Fixes: 154372e67d40 ("fs/9p: fix create-unlink-getattr idiom")
    Cc: stable@vger.kernel.org
    Reported-by: Meysam Firouzi
    Reported-by: Amirmohammad Eftekhar
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Message-ID: <20240521122947.1080227-1-asmadeus@codewreck.org>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>

commit 25460d6f39024cc3b8241b14c7ccf0d6f11a736a
Author: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
Date:   Mon Apr 8 07:10:39 2024 -0700

    net/9p: fix uninit-value in p9_client_rpc()
    
    Syzbot with the help of KMSAN reported the following error:
    
    BUG: KMSAN: uninit-value in trace_9p_client_res include/trace/events/9p.h:146 [inline]
    BUG: KMSAN: uninit-value in p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     trace_9p_client_res include/trace/events/9p.h:146 [inline]
     p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    Uninit was created at:
     __alloc_pages+0x9d6/0xe70 mm/page_alloc.c:4598
     __alloc_pages_node include/linux/gfp.h:238 [inline]
     alloc_pages_node include/linux/gfp.h:261 [inline]
     alloc_slab_page mm/slub.c:2175 [inline]
     allocate_slab mm/slub.c:2338 [inline]
     new_slab+0x2de/0x1400 mm/slub.c:2391
     ___slab_alloc+0x1184/0x33d0 mm/slub.c:3525
     __slab_alloc mm/slub.c:3610 [inline]
     __slab_alloc_node mm/slub.c:3663 [inline]
     slab_alloc_node mm/slub.c:3835 [inline]
     kmem_cache_alloc+0x6d3/0xbe0 mm/slub.c:3852
     p9_tag_alloc net/9p/client.c:278 [inline]
     p9_client_prepare_req+0x20a/0x1770 net/9p/client.c:641
     p9_client_rpc+0x27e/0x1340 net/9p/client.c:688
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    If p9_check_errors() fails early in p9_client_rpc(), req->rc.tag
    will not be properly initialized. However, trace_9p_client_res()
    ends up trying to print it out anyway before p9_client_rpc()
    finishes.
    
    Fix this issue by assigning default values to p9_fcall fields
    such as 'tag' and (just in case KMSAN unearths something new) 'id'
    during the tag allocation stage.
    
    Reported-and-tested-by: syzbot+ff14db38f56329ef68df@syzkaller.appspotmail.com
    Fixes: 348b59012e5c ("net/9p: Convert net/9p protocol dumps to tracepoints")
    Signed-off-by: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Cc: stable@vger.kernel.org
    Message-ID: <20240408141039.30428-1-n.zhandarovich@fintech.ru>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>


From xen-devel-bounces@lists.xenproject.org Thu May 30 11:02:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 11:02:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732657.1138666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdYN-0003uQ-0Q; Thu, 30 May 2024 11:02:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732657.1138666; Thu, 30 May 2024 11:02:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdYM-0003uJ-T6; Thu, 30 May 2024 11:02:38 +0000
Received: by outflank-mailman (input) for mailman id 732657;
 Thu, 30 May 2024 11:02:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCdYM-0003uD-1U
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 11:02:38 +0000
Received: from mail-oa1-x30.google.com (mail-oa1-x30.google.com
 [2001:4860:4864:20::30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1f30d0aa-1e74-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 13:02:36 +0200 (CEST)
Received: by mail-oa1-x30.google.com with SMTP id
 586e51a60fabf-24c9f628e71so369593fac.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 04:02:36 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd39f03sm545342985a.123.2024.05.30.04.02.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 04:02:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f30d0aa-1e74-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717066955; x=1717671755; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+MY9kowjPktpaukRFVgqUmVbbj7WrzsZj/2n8vR/SgQ=;
        b=qvhD5z+8wjMECKylnJ6f9BaVCPZauGAepObvMcMFm8cInSGfc6s8B0cB2xo1Dl793u
         yjyN6F10EnjbmcXbOCElChKvsSw4tPeArxVPHRdscADidBUX1HVjBe8rjUqgT7zF4XBB
         qkcXa2h0+yZfVchfjAd1e4Ct03z+Z7xb6zD4U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717066955; x=1717671755;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+MY9kowjPktpaukRFVgqUmVbbj7WrzsZj/2n8vR/SgQ=;
        b=GuqZ6fbQJL6jl5bdWWAsCyMhNmq+ulIfY8xwb0jRnTlAnB2N/6MDj958rkZAHN/6bF
         pK5MiTCSIoxezO8nKlmbFd5RsubdGZM7OzHJv7oHom5C7WmQ239jTKTve+Tvxz8aTgAr
         vhnYZYPAkRGCjcQ+dqaWVtGDxQZkFlEnp74OQlwchMinIDROtuzZrnriGzCNb3Ce+Kqu
         x5V/gzKsrjD3I4pQQs9ql3u6tn457ydu83aKegGvG9Qe/7pOKGFh+k52zbx6XdetXwrM
         n6ueFOS8z7YVBbrgiVmjOFCsb1lVr8kQ3FFILs3Mhq4kwbjqqCFV91hLeAeOmtKMcept
         2PgQ==
X-Gm-Message-State: AOJu0Yx6x46frKRiIaYn70NpbGVFLQFY8UIIjhxsMK0mAKR5LOTE9Hyj
	xpBSjo34zxVMSrdQoC9QM5EM8gB8PUrUktne8Ezi2l6b+r1xI8QFYHq/m2Va6aFLzwRYyD86tVk
	6
X-Google-Smtp-Source: AGHT+IGqG4hrRmq7ZsrUUzCmAL7M8TPeSQHtOxD7cumclVXEImEY0j/t0w7P/IrXFsUH/+cuaqbDJQ==
X-Received: by 2002:a05:6870:32c9:b0:24c:ac96:ac72 with SMTP id 586e51a60fabf-25060b76234mr1886703fac.19.1717066955343;
        Thu, 30 May 2024 04:02:35 -0700 (PDT)
Date: Thu, 30 May 2024 13:02:32 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: Re: [PATCH] tools: (Actually) drop libsystemd as a dependency
Message-ID: <ZlhcyOmu_Cc92pP5@macbook>
References: <20240530101439.16730-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240530101439.16730-1-andrew.cooper3@citrix.com>

On Thu, May 30, 2024 at 11:14:39AM +0100, Andrew Cooper wrote:
> When reinstating some of systemd.m4 between v1 and v2, I reintroduced a little
> too much.  While {c,o}xenstored are indeed no longer linked against
> libsystemd, ./configure still looks for it.
> 
> Drop this too.
> 
> Fixes: ae26101f6bfc ("tools: Drop libsystemd as a dependency")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

LGTM, but my knowledge of systemd is very limited.

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 11:08:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 11:08:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732663.1138676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCde1-0004W7-K8; Thu, 30 May 2024 11:08:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732663.1138676; Thu, 30 May 2024 11:08:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCde1-0004W0-GX; Thu, 30 May 2024 11:08:29 +0000
Received: by outflank-mailman (input) for mailman id 732663;
 Thu, 30 May 2024 11:08:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCde0-0004Vu-Jz
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 11:08:28 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f0c18f6d-1e74-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 13:08:27 +0200 (CEST)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a635a74e031so88883566b.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 04:08:27 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a66b9a876c2sm16307666b.196.2024.05.30.04.08.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 04:08:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0c18f6d-1e74-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717067307; x=1717672107; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=cYj/4GpETTQq5gzUVSXQKkVw4LGNcmrbU/zLmYHMHP8=;
        b=J7o9lwu5EvSkd6QGNWyHB4vwbbBa30WF7bONBKND/Edap/hz3Rp3D5zDUZE5cmPqOp
         6ZiBcyobLd2UyPgDPpFo39/71kJqtB0+E4wn/t8WVADY7SThgdigxFs+rOqqtefjl//x
         mxI9bm138U3zGLofyyeh1MQAxvcMweQqSoCWA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717067307; x=1717672107;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=cYj/4GpETTQq5gzUVSXQKkVw4LGNcmrbU/zLmYHMHP8=;
        b=J3t0ChfTe/6Ta4wRGTyjBgEd4l9s37GxWERJ2cCOazUN6kFEO4Oa+Q08O0d/OUjxhJ
         QvGOO4AabrQ76Yxp3n0MtpC10ZOY0Rfa1gAjnSHRiXOKaA73AEcw71KgwWzHScQPcmrQ
         bMrYfktfP5DW9Np1PJ8z1WXC+J1Urdq8chKXwFcMxyH81wWlqR8P9qwqOviPyiEGTkq9
         PFqcEsHgzbPV8Rb4pKOz6UPVycy/JylBGqe1UH7tVQ0hibX7ybZCn+IhUheIZUwYhEyJ
         aF7ewuM68Kv11h2NbIl2Vq7/uyd1OzGbGPCzdE0c6HQM3FEb65GaPOkhyJETax53Pr8W
         ZNpw==
X-Forwarded-Encrypted: i=1; AJvYcCVd1nMW2OCKgavyxuKPMmPPr+O2SHwbG1pwfweEpS6U57jeclcLX4dBrVSmq+zy438en2xtFLCB+ObWTRpRZGImhOZn2Et4jKV0Kn6e3Es=
X-Gm-Message-State: AOJu0Yz/jO08Ljt6UIHvrdUSVFmHNZfm+OqdtLKd/3Tuj2Ew7QNB2LaW
	4ASoUX5VaoKC79kjB/+mK9MMw3PurpNKtfyN7m7NNG8yxSfqppFXlVaSLgQbJ4A=
X-Google-Smtp-Source: AGHT+IFcVVO5KFEiGJMwiCmjs/Okz70R8fcylmlMjDG+mj4RquN/Dr6YOOKhQPHUkJ4IcmzFShT83Q==
X-Received: by 2002:a17:906:3958:b0:a5c:daf2:1cfb with SMTP id a640c23a62f3a-a65e923c323mr222182366b.63.1717067307086;
        Thu, 30 May 2024 04:08:27 -0700 (PDT)
Message-ID: <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>
Date: Thu, 30 May 2024 12:08:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 3:32 pm, Alejandro Vallejo wrote:
> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
> index f033d22785be..b70b22d55fcf 100644
> --- a/xen/lib/x86/policy.c
> +++ b/xen/lib/x86/policy.c
> @@ -2,6 +2,17 @@
>  
>  #include <xen/lib/x86/cpu-policy.h>
>  
> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
> +{
> +    /*
> +     * TODO: Derive x2APIC ID from the topology information inside `p`
> +     *       rather than from the vCPU ID alone. This bodge is a temporary
> +     *       measure until all infra is in place to retrieve or derive the
> +     *       initial x2APIC ID from migrated domains.
> +     */
> +    return id * 2;
> +}
> +

I'm afraid it's nonsensical to try and derive x2APIC ID from a
policy+vcpu_id.

Take a step back, and think the data through.

A VM has:
* A unique APIC_ID for each vCPU
* Info in CPUID describing how to decompose the APIC_ID into topology

Right now, because this is all completely broken, we have:
* Hardcoded APIC_ID = vCPU_ID * 2
* Total nonsense in CPUID


When constructing a VM, the toolstack (given suitable admin
guidance/defaults) *must* choose both:
* The APIC_ID themselves
* The CPUID topo data to match

i.e. this series should be editing the toolstack's call to
xc_domain_hvm_setcontext().

It's not, because AFAICT you're depending on the migration compatibility
logic and inserting a new hardcoded assumption about symmetry of the layout.


The data flows we need are:

(New) create:
* Toolstack chooses both parts of topo information
* Xen needs a default, which reasonably can be APIC_ID=vCPU_ID when the
rest of the data flow has been cleaned up.  But this is needs to be
explicit in vcpu_create() and without reference to the policy.

And to be clear, it's fine for now for the toolstack to choose a
symmetric layout and pick appropriate APIC_IDs+CPUID for this, but it
needs to be the toolstack making this decision, not Xen inventing state
out of thin air based on the toolstack only giving half the information.

(New) migrate:
* Data from the stream, exactly as presented

(Compat) migrate:
* Synthesize the missing xapic_id field in LAPIC_REGs as APIC_ID=vCPU_ID
* 2.

I'm pretty sure this will be a net reduction in complexity in this
series.  It definitely reduces the Xen complexity.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 11:12:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 11:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732669.1138685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdho-0006FM-2L; Thu, 30 May 2024 11:12:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732669.1138685; Thu, 30 May 2024 11:12:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdhn-0006FF-Vo; Thu, 30 May 2024 11:12:23 +0000
Received: by outflank-mailman (input) for mailman id 732669;
 Thu, 30 May 2024 11:12:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCdhm-0006F9-VL
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 11:12:22 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7bd0ff95-1e75-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 13:12:21 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a635a74e0deso75117466b.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 04:12:21 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a626cda2e60sm810341166b.189.2024.05.30.04.12.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 04:12:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7bd0ff95-1e75-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717067540; x=1717672340; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=9zQF3Nyb+SVEpU/uIHxnQvGzPyaWiFBmCLVh5gHC2Rw=;
        b=gH8byyepyV5ltPR7qWHZZnAWCG7Uk58r7VX5EJh5UOgcC/dHr3YxaNJATii+oxG35t
         vamEyRwRSbYNJa1Hnwp18V8Z518AYbl/qiXgZmUXnYZypmmZ1Dm9GCje+GOlrvQl5nze
         SSQD23UplPpyqMYCbGu+etkd80zNq0gycS/WQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717067540; x=1717672340;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9zQF3Nyb+SVEpU/uIHxnQvGzPyaWiFBmCLVh5gHC2Rw=;
        b=bZ1/ZDHsZjp633VRssIz+fP5/0LdtDuzaG2tk+LlWPohgNdeMGbt/iYuM1KSzk2zcj
         mHvYzgHV5RUygqJeQZ/kXeYBK+uxdHgjAPVVlD8U0+4FQmnHoFxrvE+xy+HfnkMR3w6r
         AUlBcS5UMKJ8lWmKRGQ8z+D+0K/NwmhrGCL32G0lIfqGruuqnyDH2KTD0On0iSJXwxlz
         RwlIzUFvHvD1GA95GMrJtheFm7AEXtN13VXIivLiSWMqWPgwx+pK/vg1RJ660JD8Mjam
         HHhXIVeEsjDI6/1OHNHeKd+RNqjb+RAGNYBQGhkmZjmhpVjNppTAcHxeI5ErdObJV9sE
         3GJw==
X-Gm-Message-State: AOJu0YxUSJzWnefioOKCZY2NatfU8eo5p9hkRYBizZxV43bPKjsqIYpO
	k1MYLScKmOdW5OOnhrU+geJ866nttWPp4OhebKqQGEfa+6YLXqXCR+RcPVJ/BYzBpd2hlWhT0Bt
	0
X-Google-Smtp-Source: AGHT+IEBFHm3hSRt93Xk0S6pfM6xCy4gOrh3g+PzdSbwkIOMZydGP/djUIJz+pQ969QAPj4EOn5J0A==
X-Received: by 2002:a17:906:755:b0:a64:3518:f904 with SMTP id a640c23a62f3a-a65e8d12597mr140407566b.7.1717067540471;
        Thu, 30 May 2024 04:12:20 -0700 (PDT)
Message-ID: <71bbc310-1093-44d1-a0b0-e5a8ab10a0ea@citrix.com>
Date: Thu, 30 May 2024 12:12:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools: (Actually) drop libsystemd as a dependency
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 George Dunlap <George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
References: <20240530101439.16730-1-andrew.cooper3@citrix.com>
 <ZlhcyOmu_Cc92pP5@macbook>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZlhcyOmu_Cc92pP5@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 12:02 pm, Roger Pau Monné wrote:
> On Thu, May 30, 2024 at 11:14:39AM +0100, Andrew Cooper wrote:
>> When reinstating some of systemd.m4 between v1 and v2, I reintroduced a little
>> too much.  While {c,o}xenstored are indeed no longer linked against
>> libsystemd, ./configure still looks for it.
>>
>> Drop this too.
>>
>> Fixes: ae26101f6bfc ("tools: Drop libsystemd as a dependency")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> LGTM, but my knowledge of systemd is very limited.
>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.  TBH, this is all M4/autoconf, rather than systemd.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 11:19:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 11:19:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732674.1138696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdoj-0006rY-Ot; Thu, 30 May 2024 11:19:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732674.1138696; Thu, 30 May 2024 11:19:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCdoj-0006rR-M6; Thu, 30 May 2024 11:19:33 +0000
Received: by outflank-mailman (input) for mailman id 732674;
 Thu, 30 May 2024 11:19:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UPWJ=NB=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1sCdoi-0006rL-3O
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 11:19:32 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on20607.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::607])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7ad562d3-1e76-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 13:19:29 +0200 (CEST)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DS0PR12MB7899.namprd12.prod.outlook.com (2603:10b6:8:149::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Thu, 30 May
 2024 11:19:23 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::b77f:9333:3a5a:d285%6]) with mapi id 15.20.7633.021; Thu, 30 May 2024
 11:19:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ad562d3-1e76-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YIrxbhBXVsmZr2JjFEcTbi5BMaThgc7COYQrn80UVTWgjFCy5m8Cla9CR3v5ocEjTRe+2WTTgt/yfkd7fMqdvSo7HcCGiPMPmWBqtSILjYVBmHUmBJKxBcJu9E9sKnj1lVg0Bgo328gcrdZPB3JnOV6JKQsfCWRuNi2pAwf2gaVtNJlMO2b4RSgB0jE0AhNbO55FQJdRFUbI+aUXNgaJKStftKsr5Pi4ZaZRdHgo9kkBUsSrhyuO5xM04TJLS/f6T/79fiQ8zsXvm3tLHQ7p4CN5YwgNSO9Te7Ew0Hi577ZQ2oEgrE0YKeVA0bqTQOs31lHPJbdd44tVNY6oBN82ww==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2/bsaiOlmnOP24VqNb9Ma4aPIn679EPHR0zSu94KM2w=;
 b=kXWSi8bVXLQK/iZYgaAnbutwTEyQJz3Lkl0YyUx4iP8xXtFCj6QDSn9MtgGMpO5wPfdZwXB39ETNcDBqdscT8oQCYLQyaFzDDYvo7Lpd0RLM9rndeYGw22AqqImul6FQC+BpZM4KG75HSyU+RmrRFSw9adVJDQgHqTTH+08srp9g3Q/wdEG2rdYKeMhoPikKpjylAlp3xELiOrupecf/y4pjKUyYHcy6y6bslguTzvNEQF7fxHObWTG+Pg7ZztqgEDAtUvvKXfI9Eq6a8fth3AXBJ6E3Pp+7C52G6X+jLyFLtPsRxHu9lrSctieTy8WbVR90TN5aGyksUshmFTF2ow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2/bsaiOlmnOP24VqNb9Ma4aPIn679EPHR0zSu94KM2w=;
 b=h2sMPUQK5T602Coh+tWJTqm9Huv4wbh0GjsoffbYEVEkeAfc+ldhZbZyxoQKy976U4/iINB9WBiNRTTWbVutDDRBUAI3oDKPbc1tUccFsX3Psxknr7v2sf8oTYyCgXhnnyyVM5Ge4P96CYSoiEAvr3ig906sHvvdXoTyc6pfjQk=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony@xenproject.org>, Juergen Gross <jgross@suse.com>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Daniel P
 . Smith" <dpsmith@apertussolutions.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index:
 AQHap3bgiER4vYjwvk2+R5oTa8V63LGZ5EYAgAHVa4D//4e1AIAAis+A//+FmgCAEsTugP//viAAABEhvwD//4GYgIAAoyuA//+0FICAAgSCAA==
Date: Thu, 30 May 2024 11:19:22 +0000
Message-ID:
 <BL1PR12MB58494B521CB40BAEA30CB412E7F32@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
 <BL1PR12MB58493D3365CC451F36DB554FE7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <fbaf7086-85d8-4433-91d9-ef8f74512685@suse.com>
In-Reply-To: <fbaf7086-85d8-4433-91d9-ef8f74512685@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL1PR12MB5849.namprd12.prod.outlook.com
 (15.20.7633.017)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DS0PR12MB7899:EE_
x-ms-office365-filtering-correlation-id: e566d5c5-d7e5-4bf3-6362-08dc809a5b91
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam:
 BCL:0;ARA:13230031|366007|1800799015|7416005|376005|38070700009;
x-microsoft-antispam-message-info:
 =?utf-8?B?UUhOZkNqUElFSnUxQmdUZ2I1ZlVjZ2RZbU0zbTUvOW9TcEhYNVd4WkJ2M3lx?=
 =?utf-8?B?Vm10Z0JjWGxRSGN3a3BjUFNWSGRGUGh2Z0ZhczZJclNsTmVIcnYyZG5rcThO?=
 =?utf-8?B?TlRxMnh1aUNLSXl6eXlOWmlJZThCVTZXRUkwaUZTaGZPWkRIWnRnZk5UK3NN?=
 =?utf-8?B?eTRFeXFiQ3hyaXphZnVvVEhHZU04aU1hRndTQVVYelZQVzBtRkl5TFBUd3lR?=
 =?utf-8?B?ZDlBNTBmQXlkNFRNajNrWTVUWkV3d1lGTzMxYVo4T3hmc3Z2VU8yRzdkSmJr?=
 =?utf-8?B?bjlCZXBkK0NmTmw5aW9WY2YzRU5EdWc2YndocnhDQU5VM1FWMzFsL2pUVzF0?=
 =?utf-8?B?VVpMRmtHZHRlZEhwbk44RC9VVUZmTXhJOHZBZkJKS2pkNi8vY20rVXNERFhy?=
 =?utf-8?B?YVR2ZW9yZHJwNHQwZGJZVExPdkVScmE2VXBKNThpSlMrb0Z3bUxndFlnc2xw?=
 =?utf-8?B?aFUwSlZ0cWhjcEZ5bEdLQTBUMnVwRFBwaDM0bkdyYVh1RVNqcEtJYURHWlcv?=
 =?utf-8?B?SXhsQXdHM2E3T29xWUtBcDA2TmhlR2pSZEtteUEwamZZZEhiLzBnd2FWejVC?=
 =?utf-8?B?VldRbzk2MWpFZDVONU4zbG9FV0FzM2lFQ2taYjZjR0RKUHBiZlA3VjY4U1RV?=
 =?utf-8?B?dXZWYXBDdXdhM0ptaHluVkRXV3ZwaElOa0pRYnc5R2VzOEhETWF4U2IyMnlr?=
 =?utf-8?B?TER3QjBGeFYyWVZHSUJKWFloQkhsY3dPeXBQVlhxdk1KSG00VHFEbitFQzIx?=
 =?utf-8?B?TkJkUUY5aFB6azRCeGl1QTBWVU5BVTVmdSt2QWxFMnF4bXNMci9acGdCbGNX?=
 =?utf-8?B?K0NwMTZKYjh1S3grcU5wT0hYUUk2WFlQT0d6UFhYMDdNUTdzTndwV2tWUW11?=
 =?utf-8?B?dnpwcEF3RGp5RENvZ1Q3MFplSlNuUTE1Z3RWZ0U5N2pKQTZUYzNvbHNpUEg1?=
 =?utf-8?B?L0ZQbHNJQ2cyRURzUmhONzllWTR5REEyMXZmUS9KZjdiWFZHZVJERVljMGtu?=
 =?utf-8?B?WUUzY2VOTDJ5MjE4d2xZZVdRNzhONzR1R2JYM2p6dkk2VnlsQkEvWjdDZHpZ?=
 =?utf-8?B?b242bkY3QklvOW55UnFGYUpaNU1YSUdTQTczeWNRYnl1S0haZFF3bjg2M1R3?=
 =?utf-8?B?WExyVFpMUEl3a3IzQ09qY3plekV6VTd2NkM1U3h3V2xySFczYUh5L0thTzNv?=
 =?utf-8?B?THp6S01xamYrYkl1cDBEZmxwcGJpdlUxSXVtcXJRcEtrQk5CVDNJR1I4NVZE?=
 =?utf-8?B?R25LY0NGT1ZvMkJoaXdCaXNNWEplV2c4Zk4yWWxzVWFGcC9LbkZDWWV2Y2NF?=
 =?utf-8?B?Qi9xTGpyTGl2MmgwV0hyemtNRTdGRklZTTFBdnNsYk9XbFhXajZOcXdaUS9M?=
 =?utf-8?B?Qm50bndZQ0gvWnFFcWtVVXYycDFlSFc5L0ozUU1BQTh5MmhtOEk3bysreHJF?=
 =?utf-8?B?S0FqWWxjdVFFS1p6dDVXUWpOZ1F6WHVIUFpLYXdkZHhqVFFZUnRPQUlsRjJM?=
 =?utf-8?B?Q0ZMTUtReDh1MCszRStyUWh5cW5QS3haZXFLYklBSzAyV0pBWWVYT2RJZjg2?=
 =?utf-8?B?NkFXcEczU2pCeDBOSEFXemgxNmp2SlhRdnZRV0xyeGtLTFcxUWVKckFQRk4z?=
 =?utf-8?B?UEkwUEQzME1qZXpyV1NPSkZNbXNqSUU1TTF2aGsvZHAxSE9Pa3k0bU9kUGpR?=
 =?utf-8?B?T3haM3BsSXJ0REU1bVBCTmYwSGlGZGJaSUYybWM0V2tzbHdmSXZENE1HZStX?=
 =?utf-8?B?ZGl2TVNlemtyK0xFUTNkM2dMbHJVMVlycEVBMnRsYVI0SmliblhNTklKVmtt?=
 =?utf-8?B?bWRtUVlZeTR6Q2FkbzVaZz09?=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(7416005)(376005)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?eXZ3d3Bhd3JpVnpjWFhYVFZTc3plQWkyYzVnTko4OEp0NmFWMUJ4SGRkOVpF?=
 =?utf-8?B?cU9yQVQydWNaZmhCZ3pUS3Z2Rk56T2RDVUptQmFrN3F1UU56WnJjUzA1SzNw?=
 =?utf-8?B?bE9icDFaR2t4TE9VWkExc0h0MzAxZjZTNVJTSTdNT2laRHJuUEZFL1RBNmlR?=
 =?utf-8?B?RkR5bEk5NDVFSUoxeUN2YVlCMlpyWG5nM1JaV2RXNlhTeUY5dHRROVYxTXVB?=
 =?utf-8?B?U1J0WkU3MWNVS2ZEQ3Y1WGlPeVlKTFY1U2w1bzY1cWJWN1BGOGJnUGtIVS92?=
 =?utf-8?B?Um5wNVJUZlQvY0dXUm1aWXdLbUIreXB5Y3NnSE14WnY4K2dDSjFrOFptMTJX?=
 =?utf-8?B?c0lTck94K3NsTGZ6a2ZBRnZNZDRiTnE5RjlUTDNkRkREd1d1N1pRcy9rN3do?=
 =?utf-8?B?S00vbjFzYXo4MHg4SFQ2Z2h0QnU2ZkZjSVE0ZXFnTktBM1lBYjRuYXcrSWll?=
 =?utf-8?B?emd2OTNOVXVibVljdEEvUXNieE1rK0d2aEVZQ0JBQmdhNHI3a1F0clc0dTE1?=
 =?utf-8?B?YjlsUHRodEp1b0Jqamk0TDBMMzE4SnNGeTRsOGFJa0JSTWI0ZUNsQ2R0ZDVT?=
 =?utf-8?B?czgyMHBWcWZZRGdkcmZNV0dQdEtwWjFSVlViUTRGYnppRkRWbWZ4WWwvWjRs?=
 =?utf-8?B?b0UyVi9xb09mbUxwZ0lXbXN1YXJUaG1QTGVxK2dYQmVuUVY3NmlWNEx4VWhP?=
 =?utf-8?B?QnA3TEtMMGM3VEw0Rmg1eXpCV1YxRHdCVjJQaE51VTlwMnhwcmRqenB6SUdM?=
 =?utf-8?B?ZEs5Q1dxK1c4UHQzdXhMYmtDRit6azlOblczMlZCVlpmamtaV3pLbnlUOHFR?=
 =?utf-8?B?NGMzWERoQ1Vsak4wREppY2F4ajVZSkZnY3d0bkd0amlGLzMveXZGUWo3a0wy?=
 =?utf-8?B?Qk93emNsZU1rYmx1R3hDVUtUVTlhRXB5OTNDWlY0VWZ3clc1a0J2UzZUZGky?=
 =?utf-8?B?Mk1ibkpYLzdwV2VmTEJkMktqdjFSVW92TmdId2tFTUEvR1d6YmIyQ0FJd1VC?=
 =?utf-8?B?ZFU5YlluREtkUGVYOUNxZUlhNEtmWGYyK2ZJelNLSERNMFY3UjM2dk42RTlT?=
 =?utf-8?B?Y3d3bGlZOTRyMUNpTEhrTndqVFQ2Q3pMMENVZEYxM3RmbHBVNFREQndIVlh1?=
 =?utf-8?B?NGZ0bUpEQmVzTkdyaDFieTJNN2l1RDZscktuWEFDbnpiRDJ5Z2JXZ3BlQ1JI?=
 =?utf-8?B?cjUzMXBhNTl2RUNGdGhzd2FZanpSeFVwcDlGYXRzWGRPeTluanJMaC9aRW1h?=
 =?utf-8?B?Mi9LZkJWRTJiWmNKREhSUFhBL1FLUjNuQThQQ1NwZjZRZUNMM20yZzNOL3Bx?=
 =?utf-8?B?ZzRNMnVKNFVnTHZ0MnVta2lDWklaV2Y5UTNUdVBjbitmb0wzQlRldGttQVNJ?=
 =?utf-8?B?amk1dVFSR3hOS2dNT0w4YzdVL0I4OVh3ZnBNTkZRcGxHNU1ONVNQTmN5ZlpI?=
 =?utf-8?B?bmNNZE1TTmxsYkx4dEt0RzBYMnNoRE0zMXhHaTRCdWdNKzlZL1JhRlBRTHNj?=
 =?utf-8?B?S3F1R0ZRZHJyRThBdzhaU255OHlwTG42TVNEUVh4alFyaDRnNVZHUDBFbGJ1?=
 =?utf-8?B?Q3lHN1p2OGxtd0wrM05nUTVMK2M3MURwMklxNSttNzZkcG5mNTF0aUJocHdO?=
 =?utf-8?B?cGUyQUZRcUs0OEwwQW5neENoM0NLRU5yYVY0SUluWWNrVmh1Q1NtZFo3WUsz?=
 =?utf-8?B?Wk9ZK3Jsa1NYQ1R6ZGltRGtuc3M4WVh3dkcwWjJZUjFucmdiYzV3NzloM0N0?=
 =?utf-8?B?TUxLeUFkM0NoT0JGUXJ2RVcxT01Nc3g0MzhRaStDQU5ROVNoV1BRNnVYVExD?=
 =?utf-8?B?bXpqdGNPcWU5NWlxa1NuQm5ib0JlOEJ6Wk9PelA5TG1lZjMzdlVGcW1Wa0hL?=
 =?utf-8?B?QVk1THZwWFJ5djY0L2laalNYWDM4ajRDcGdsbnQ1cnVqb0NWNXdRa1d3ZStD?=
 =?utf-8?B?Yi9TOHhoVU1hRHpHS3l2N3ZGNzJWbFIvQm94dHBJSUE2ZlhTVmtYeHFYRDVT?=
 =?utf-8?B?ZjdJRER4QU0rdjUwaWJRVGlKa0NsaGdoZXkxNmY1dzJCQlJlVnB5WloyaGlr?=
 =?utf-8?B?OE9QTmxzbEJIMmJzM2lUVVB6Z21UQ2NiaFllYTdqZXFUeHk4Qmg4VnVnYzZh?=
 =?utf-8?Q?7djk=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <047F22B4717A79479AB8B3DD4E816F5F@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e566d5c5-d7e5-4bf3-6362-08dc809a5b91
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 May 2024 11:19:22.8903
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Kj2S3RmLRpRWBVQ2sAR2sQshquuRVaM3WDg6DS1yeOwToTnWjiXco4pCMcofVgsp0lmJOxeobzJVFSWUXqYKOA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7899

T24gMjAyNC81LzI5IDIwOjIyLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMjkuMDUuMjAyNCAx
MzoxMywgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24gMjAyNC81LzI5IDE1OjEwLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAyOS4wNS4yMDI0IDA4OjU2LCBDaGVuLCBKaXFpYW4gd3JvdGU6
DQo+Pj4+IE9uIDIwMjQvNS8yOSAxNDozMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+PiBPbiAy
OS4wNS4yMDI0IDA0OjQxLCBDaGVuLCBKaXFpYW4gd3JvdGU6DQo+Pj4+Pj4gQnV0IEkgZm91bmQg
aW4gZnVuY3Rpb24gaW5pdF9pcnFfZGF0YToNCj4+Pj4+PiAgICAgZm9yICggaXJxID0gMDsgaXJx
IDwgbnJfaXJxc19nc2k7IGlycSsrICkNCj4+Pj4+PiAgICAgew0KPj4+Pj4+ICAgICAgICAgaW50
IHJjOw0KPj4+Pj4+DQo+Pj4+Pj4gICAgICAgICBkZXNjID0gaXJxX3RvX2Rlc2MoaXJxKTsNCj4+
Pj4+PiAgICAgICAgIGRlc2MtPmlycSA9IGlycTsNCj4+Pj4+Pg0KPj4+Pj4+ICAgICAgICAgcmMg
PSBpbml0X29uZV9pcnFfZGVzYyhkZXNjKTsNCj4+Pj4+PiAgICAgICAgIGlmICggcmMgKQ0KPj4+
Pj4+ICAgICAgICAgICAgIHJldHVybiByYzsNCj4+Pj4+PiAgICAgfQ0KPj4+Pj4+IERvZXMgaXQg
bWVhbiB0aGF0IHdoZW4gaXJxIDwgbnJfaXJxc19nc2ksIHRoZSBnc2kgYW5kIGlycSBpcyBhIDE6
MSBtYXBwaW5nPw0KPj4+Pj4NCj4+Pj4+IE5vLCBhcyBleHBsYWluZWQgYmVmb3JlLiBJIGFsc28g
ZG9uJ3Qgc2VlIGhvdyB5b3Ugd291bGQgZGVyaXZlIHRoYXQgZnJvbSB0aGUgY29kZSBhYm92ZS4N
Cj4+Pj4gQmVjYXVzZSBoZXJlIHNldCBkZXNjLT5pcnEgPSBpcnEsIGFuZCBpdCBzZWVtcyB0aGVy
ZSBpcyBubyBvdGhlciBwbGFjZSB0byBjaGFuZ2UgdGhpcyBkZXNjLT5pcnEsIHNvLCBnc2kgMSBp
cyBjb25zaWRlcmVkIHRvIGlycSAxLg0KPj4+DQo+Pj4gV2hhdCBhcmUgeW91IHRha2luZyB0aGlz
IGZyb20/IFRoZSBsb29wIGJvdW5kIGlzbid0IG5yX2dzaXMsIGFuZCB0aGUgaXRlcmF0aW9uDQo+
Pj4gdmFyaWFibGUgaXNuJ3QgaW4gR1NJIHNwYWNlIGVpdGhlcjsgaXQncyBpbiBJUlEgbnVtYmVy
aW5nIHNwYWNlLiBJbiB0aGlzIGxvb3ANCj4+PiB3ZSdyZSBtZXJlbHkgbGV2ZXJhZ2luZyB0aGF0
IGV2ZXJ5IEdTSSBoYXMgYSBjb3JyZXNwb25kaW5nIElSUTsNCj4+PiB0aGVyZSBhcmUgbm8gYXNz
dW1wdGlvbnMgbWFkZSBhYm91dCB0aGUgbWFwcGluZyBiZXR3ZWVuIHRoZSB0d28uIEFmYWljcyBh
dCBsZWFzdC4NCj4+Pg0KPj4+Pj4gIm5yX2lycXNfZ3NpIiBkZXNjcmliZXMgd2hhdCBpdHMgbmFt
ZSBzYXlzOiBUaGUgbnVtYmVyIG9mDQo+Pj4+PiBJUlFzIG1hcHBpbmcgdG8gYSAoX3NvbWVfKSBH
U0kuIFRoYXQncyB0byB0ZWxsIHRoZW0gZnJvbSB0aGUgbm9uLUdTSSAoaS5lLg0KPj4+Pj4gbWFp
bmx5IE1TSSkgb25lcy4gVGhlcmUncyBubyBpbXBsaWNhdGlvbiB3aGF0c29ldmVyIG9uIHRoZSBJ
UlEgPC0+IEdTSQ0KPj4+Pj4gbWFwcGluZy4NCj4+Pj4+DQo+Pj4+Pj4gV2hhdCdzIG1vcmUsIHdo
ZW4gdXNpbmcgUEhZU0RFVk9QX3NldHVwX2dzaSwgaXQgY2FsbHMgbXBfcmVnaXN0ZXJfZ3NpLA0K
Pj4+Pj4+IGFuZCBpbiBtcF9yZWdpc3Rlcl9nc2ksIGl0IHVzZXMgIiBkZXNjID0gaXJxX3RvX2Rl
c2MoZ3NpKTsgIiB0byBnZXQgaXJxX2Rlc2MgZGlyZWN0bHkuDQo+Pj4+Pg0KPj4+Pj4gV2hpY2gg
bWF5IGJlIHdyb25nLCB3aGlsZSB0aGF0IHdyb25nLW5lc3MgbWF5IG5vdCBoYXZlIGhpdCBhbnlv
bmUgaW4NCj4+Pj4+IHByYWN0aWNlIChmb3IgcmVhc29ucyB0aGF0IHdvdWxkIG5lZWQgd29ya2lu
ZyBvdXQpLg0KPj4+Pj4NCj4+Pj4+PiBDb21iaW5pbmcgYWJvdmUsIGNhbiB3ZSBjb25zaWRlciAi
Z3NpID09IGlycSIgd2hlbiBpcnEgPCBucl9pcnFzX2dzaSA/DQo+Pj4+Pg0KPj4+Pj4gQWdhaW4g
LSBuby4NCj4+Pj4gU2luY2UgeW91IGFyZSBjZXJ0YWluIHRoYXQgdGhleSBhcmUgbm90IGVxdWFs
LCBjb3VsZCB5b3UgdGVsbCBtZSB3aGVyZSBzaG93IHRoZXkgYXJlIG5vdCBlcXVhbCBvciB3aGVy
ZSBidWlsZCB0aGVpciBtYXBwaW5ncywNCj4+Pj4gc28gdGhhdCBJIGNhbiBrbm93IGhvdyB0byBk
byBhIGNvbnZlcnNpb24gZ3NpIGZyb20gaXJxLg0KPj4+DQo+Pj4gSSBkaWQgcG9pbnQgeW91IGF0
IHRoZSBBQ1BJIEludGVycnVwdCBTb3VyY2UgT3ZlcnJpZGUgc3RydWN0dXJlIGJlZm9yZS4NCj4+
PiBXZSdyZSBwYXJzaW5nIHRob3NlIGluIGFjcGlfcGFyc2VfaW50X3NyY19vdnIoKSwgdG8gZ2l2
ZSB5b3UgYSBwbGFjZSB0bw0KPj4+IHN0YXJ0IGdvaW5nIGZyb20uDQo+PiBPaCEgSSB0aGluayBJ
IGtub3cuDQo+PiBJZiBJIHdhbnQgdG8gdHJhbnNmb3JtIGdzaSB0byBpcnEsIEkgbmVlZCB0byBk
byBiZWxvdzoNCj4+IAlpbnQgaXJxLCBlbnRyeSwgaW9hcGljLCBwaW47DQo+Pg0KPj4gCWlvYXBp
YyA9IG1wX2ZpbmRfaW9hcGljKGdzaSk7DQo+PiAJcGluID0gZ3NpIC0gbXBfaW9hcGljX3JvdXRp
bmdbaW9hcGljXS5nc2lfYmFzZTsNCj4+IAllbnRyeSA9IGZpbmRfaXJxX2VudHJ5KGlvYXBpYywg
cGluLCBtcF9JTlQpOw0KPj4gCWlycSA9IHBpbl8yX2lycShlbnRyeSwgaW9hcGljLCBwaW4pOw0K
Pj4NCj4+IEFtIEkgcmlnaHQ/DQo+IA0KPiBUaGlzIGxvb2tzIHBsYXVzaWJsZSwgeWVzLg0KSSBk
dW1wIGFsbCBtcGNfY29uZmlnX2ludHNyYyBvZiBhcnJheSBtcF9pcnFzLCBpdCBzaG93czoNCihY
RU4pIGZpbmRfaXJxX2VudHJ5IHR5cGUgMyBpcnF0eXBlIDAgaXJxZmxhZyAwIHNyY2J1cyAwIHNy
Y2J1c2lycSAwIGRzdGFwaWMgMzMgZHN0aXJxIDINCihYRU4pIGZpbmRfaXJxX2VudHJ5IHR5cGUg
MyBpcnF0eXBlIDAgaXJxZmxhZyAxNSBzcmNidXMgMCBzcmNidXNpcnEgOSBkc3RhcGljIDMzIGRz
dGlycSA5DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBz
cmNidXMgMCBzcmNidXNpcnEgMSBkc3RhcGljIDMzIGRzdGlycSAxDQooWEVOKSBmaW5kX2lycV9l
bnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBzcmNidXMgMCBzcmNidXNpcnEgMyBkc3Rh
cGljIDMzIGRzdGlycSAzDQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGly
cWZsYWcgMCBzcmNidXMgMCBzcmNidXNpcnEgNCBkc3RhcGljIDMzIGRzdGlycSA0DQooWEVOKSBm
aW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBzcmNidXMgMCBzcmNidXNp
cnEgNSBkc3RhcGljIDMzIGRzdGlycSA1DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJx
dHlwZSAwIGlycWZsYWcgMCBzcmNidXMgMCBzcmNidXNpcnEgNiBkc3RhcGljIDMzIGRzdGlycSA2
DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBzcmNidXMg
MCBzcmNidXNpcnEgNyBkc3RhcGljIDMzIGRzdGlycSA3DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0
eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBzcmNidXMgMCBzcmNidXNpcnEgOCBkc3RhcGljIDMz
IGRzdGlycSA4DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcg
MCBzcmNidXMgMCBzcmNidXNpcnEgMTAgZHN0YXBpYyAzMyBkc3RpcnEgMTANCihYRU4pIGZpbmRf
aXJxX2VudHJ5IHR5cGUgMyBpcnF0eXBlIDAgaXJxZmxhZyAwIHNyY2J1cyAwIHNyY2J1c2lycSAx
MSBkc3RhcGljIDMzIGRzdGlycSAxMQ0KKFhFTikgZmluZF9pcnFfZW50cnkgdHlwZSAzIGlycXR5
cGUgMCBpcnFmbGFnIDAgc3JjYnVzIDAgc3JjYnVzaXJxIDEyIGRzdGFwaWMgMzMgZHN0aXJxIDEy
DQooWEVOKSBmaW5kX2lycV9lbnRyeSB0eXBlIDMgaXJxdHlwZSAwIGlycWZsYWcgMCBzcmNidXMg
MCBzcmNidXNpcnEgMTMgZHN0YXBpYyAzMyBkc3RpcnEgMTMNCihYRU4pIGZpbmRfaXJxX2VudHJ5
IHR5cGUgMyBpcnF0eXBlIDAgaXJxZmxhZyAwIHNyY2J1cyAwIHNyY2J1c2lycSAxNCBkc3RhcGlj
IDMzIGRzdGlycSAxNA0KKFhFTikgZmluZF9pcnFfZW50cnkgdHlwZSAzIGlycXR5cGUgMCBpcnFm
bGFnIDAgc3JjYnVzIDAgc3JjYnVzaXJxIDE1IGRzdGFwaWMgMzMgZHN0aXJxIDE1DQoNCkl0IHNl
ZW1zIG9ubHkgTGVnYWN5IGlycSBhbmQgZ3NpWzA6MTVdIGhhcyBhIG1hcHBpbmcgaW4gbXBfaXJx
cy4NCk90aGVyIGdzaSBjYW4gYmUgY29uc2lkZXJlZCAxOjEgbWFwcGluZyB3aXRoIGlycT8gT3Ig
YXJlIHRoZXJlIG90aGVyIHBsYWNlcyByZWZsZWN0IHRoZSBtYXBwaW5nIGJldHdlZW4gaXJxIGFu
ZCBnc2k/DQpBbmQgbXkgY29kZSB3aWxsIGJlOg0KICAgIGNhc2UgWEVOX0RPTUNUTF9nc2lfcGVy
bWlzc2lvbjoNCiAgICB7DQogICAgICAgIHVuc2lnbmVkIGludCBnc2kgPSBkb21jdGwtPnUuZ3Np
X3Blcm1pc3Npb24uZ3NpOw0KICAgICAgICBpbnQgaXJxOw0KICAgICAgICBib29sIGFsbG93ID0g
ZG9tY3RsLT51LmdzaV9wZXJtaXNzaW9uLmFsbG93X2FjY2VzczsNCg0KICAgICAgICAvKg0KICAg
ICAgICAgKiBnc2lbMCwxNV0gaXMgbm90IDE6MSBtYXBwaW5nIHRvIGxlZ2FjeSBpcnFbMCwxNV0s
IGl0IG5lZWQgYQ0KICAgICAgICAgKiB0cmFuc2Zvcm1hdGlvbi4gT3RoZXIgZ3NpIGlzIGNvbnNp
ZGVyZWQgYmUgMToxIG1hcHBpbmcgdG8gaXJxDQogICAgICAgICAqLw0KICAgICAgICBpZiAoIGdz
aSA8IDE2ICkNCiAgICAgICAgICAgIGlycSA9IGdzaV8yX2lycShnc2kpOw0KICAgICAgICBlbHNl
DQogICAgICAgICAgICBpcnEgPSBnc2k7DQoNCiAgICAgICAgLyoNCiAgICAgICAgICogSWYgY3Vy
cmVudCBkb21haW4gaXMgUFYgb3IgaXQgaGFzIFBJUlEgZmxhZywgaXQgaGFzIGEgbWFwcGluZw0K
ICAgICAgICAgKiBvZiBnc2ksIHBpcnEgYW5kIGlycSwgc28gaXQgc2hvdWxkIHVzZSBYRU5fRE9N
Q1RMX2lycV9wZXJtaXNzaW9uDQogICAgICAgICAqIHRvIGdyYW50IGlycSBwZXJtaXNzaW9uLg0K
ICAgICAgICAgKi8NCiAgICAgICAgaWYgKCBpc19wdl9kb21haW4oY3VycmVudC0+ZG9tYWluKSB8
fCBoYXNfcGlycShjdXJyZW50LT5kb21haW4pICkNCiAgICAgICAgew0KICAgICAgICAgICAgcmV0
ID0gLUVPUE5PVFNVUFA7DQogICAgICAgICAgICBicmVhazsNCiAgICAgICAgfQ0KDQogICAgICAg
IGlmICggZ3NpID49IG5yX2lycXNfZ3NpIHx8IGlycSA8IDAgKQ0KICAgICAgICB7DQogICAgICAg
ICAgICByZXQgPSAtRUlOVkFMOw0KICAgICAgICAgICAgYnJlYWs7DQogICAgICAgIH0NCg0KICAg
ICAgICBpZiAoICFpcnFfYWNjZXNzX3Blcm1pdHRlZChjdXJyZW50LT5kb21haW4sIGlycSkgfHwN
CiAgICAgICAgICAgICB4c21faXJxX3Blcm1pc3Npb24oWFNNX0hPT0ssIGQsIGlycSwgYWxsb3cp
ICkNCiAgICAgICAgew0KICAgICAgICAgICAgcmV0ID0gLUVQRVJNOw0KICAgICAgICAgICAgYnJl
YWs7DQogICAgICAgIH0NCg0KICAgICAgICBpZiAoIGFsbG93ICkNCiAgICAgICAgICAgIHJldCA9
IGlycV9wZXJtaXRfYWNjZXNzKGQsIGlycSk7DQogICAgICAgIGVsc2UNCiAgICAgICAgICAgIHJl
dCA9IGlycV9kZW55X2FjY2VzcyhkLCBpcnEpOw0KICAgICAgICBicmVhazsNCiAgICB9DQoNCklz
IGFib3ZlIGFjY2VwdGFibGU/DQoNCj4gDQo+IEphbg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkpp
cWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Thu May 30 11:43:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 11:43:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732685.1138722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCeBq-00035K-Q6; Thu, 30 May 2024 11:43:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732685.1138722; Thu, 30 May 2024 11:43:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCeBq-00035D-M4; Thu, 30 May 2024 11:43:26 +0000
Received: by outflank-mailman (input) for mailman id 732685;
 Thu, 30 May 2024 11:43:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCeBp-000353-Nz; Thu, 30 May 2024 11:43:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCeBp-0006HM-LA; Thu, 30 May 2024 11:43:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCeBp-0003ZS-DB; Thu, 30 May 2024 11:43:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCeBp-0007TR-Cc; Thu, 30 May 2024 11:43:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nbr3VhcRmHb3kXSxKpJ1MPsh/agb+c2sEcAAQFgA1aM=; b=FV5bL2iDyL1U3AboIg48crMaHS
	631m/CE7ExVAy2okdTHU6AzhU1Xl7QOCeiamQUIo9yLkdhJr83ZWwOktEgaoz2PO4cf4APF/7RpOt
	au+XiR84UEEOMuLW+0fqupCBmeH1Pp1a7rgqD3FwkCDsAZlX50HCuo8W/yJWUju8+zi8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186195-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186195: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=c695e3182aa7497833f1b0fc69f6776fec8cb8cf
X-Osstest-Versions-That:
    ovmf=cd4cebabf5834c403c9d9ac4f162e8336024bbcd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 11:43:25 +0000

flight 186195 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186195/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 c695e3182aa7497833f1b0fc69f6776fec8cb8cf
baseline version:
 ovmf                 cd4cebabf5834c403c9d9ac4f162e8336024bbcd

Last test of basis   186193  2024-05-30 07:11:07 Z    0 days
Testing same since   186195  2024-05-30 09:41:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Nickle Wang <nicklew@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   cd4cebabf5..c695e3182a  c695e3182aa7497833f1b0fc69f6776fec8cb8cf -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 12:45:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 12:45:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732703.1138732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCf9n-0002PO-Cu; Thu, 30 May 2024 12:45:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732703.1138732; Thu, 30 May 2024 12:45:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCf9n-0002PH-90; Thu, 30 May 2024 12:45:23 +0000
Received: by outflank-mailman (input) for mailman id 732703;
 Thu, 30 May 2024 12:45:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XHBK=NB=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1sCf9l-0002PB-Lx
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 12:45:21 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 78fe07ea-1e82-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 14:45:20 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-5785f861868so1023797a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 05:45:19 -0700 (PDT)
Received: from ?IPV6:2003:e5:8729:4000:29eb:6d9d:3214:39d2?
 (p200300e58729400029eb6d9d321439d2.dip0.t-ipconnect.de.
 [2003:e5:8729:4000:29eb:6d9d:3214:39d2])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578f47126f0sm7401739a12.91.2024.05.30.05.45.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 05:45:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78fe07ea-1e82-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717073119; x=1717677919; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wwL9VC6bw4mSsExmP/I8aaPL6juThXBiKzt+hu5ZDtY=;
        b=acqxpm4045fiUTfYPMITqjT+Uryj8O7XaMqljWG0GodqYvZ/CyNFISejM1wh55a2Ag
         ZbGpBULnIeBVWUq3BC23kkX9sctfmjrOAim3Gcxi3mBGmjZcvkH9a9hYcNW+0X/4FQPa
         VArTKNT0SIenWAMPS3RohqGtSKoNiZon8J5r9ibcegr1dOEPUbyJTJCc+TLbzya7IMpn
         sKEHO0nKFAGNJDDegRSHQkASTA7fL+JQrojegH3MgGl67v4alZ/XCW0CLI5lxm917XPv
         DOKQZ01/Pj4tjOJzRxydS2vNkD688Oyn23JT08cCVHgnfRroqnu//uP1BHvc3zVc8XYe
         WY8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717073119; x=1717677919;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wwL9VC6bw4mSsExmP/I8aaPL6juThXBiKzt+hu5ZDtY=;
        b=FqpCEBpDZCu9qDsH5sHMuVCMFRj0+jqQMU5gb6e3FOyVmJ0S7J2pfcU7KYL1pwEiSb
         DM3TsWcFQ9TIburH5FKMYhtiL5a9RN4R+Zrr33qe9i6BYv1ih7bafoRfp1aYE8MzOF3S
         BCXJLG6RazXHig1RC17xdwz2aPq+P+KmqGSnQW2ZWj//RxlXo06mN7ZOEk00byTFA+LG
         h2tU4Li5HCmYLTMSMySPI4PcaFLwLvckTKjdjOhVy7TfoXUAwkn6zKsCkoYxTwDOMNlw
         f3RUYorJOeLzbQ0udjXJ6wlOHwAqYEEvMeR7aRiTKTM5F4/QBUNlp2S4u2dnOK+N4VGv
         bdUw==
X-Gm-Message-State: AOJu0YxQ6Epj4TJbm62iqBGuUy/VTYRVrO8oy0fM6mb21L4lEuWeZLFE
	eJpPy6WOv8mboMedFk47k74uEyUIBNeuDx5cP7EbYNxg02P/aKRyIjWhYi0EpRI=
X-Google-Smtp-Source: AGHT+IHyNKid3eB/fKP3NQqMI5VnTEBQyzcnBRyQtsdGDgH+OxSqAvFHZwSsMStY3wI8w9ZhF22bYg==
X-Received: by 2002:a50:9ec8:0:b0:572:9b20:fa with SMTP id 4fb4d7f45d1cf-57a17952d26mr1490565a12.33.1717073119076;
        Thu, 30 May 2024 05:45:19 -0700 (PDT)
Message-ID: <bba693b3-ad73-4fb8-ae9c-dba025f6f5c1@suse.com>
Date: Thu, 30 May 2024 14:45:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Dario Faggioli <dfaggioli@suse.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ZlW-ZyFeAzvh3pGG@macbook>
 <a749c066-0c05-47dd-b499-e712e5454033@suse.com> <ZlcjiOJyAuvyy48n@macbook>
 <1147a06d-d9ec-4fa9-a1d0-dfecd7a670dc@suse.com> <ZldRusciaXuwTC9u@macbook>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <ZldRusciaXuwTC9u@macbook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 29.05.24 18:03, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 03:08:49PM +0200, Jürgen Groß wrote:
>> On 29.05.24 14:46, Roger Pau Monné wrote:
>>> On Wed, May 29, 2024 at 01:47:09PM +0200, Jürgen Groß wrote:
>>>> On 28.05.24 13:22, Roger Pau Monné wrote:
>>>>> Hello,
>>>>>
>>>>> When the stop_machine_run() call in cpu_down() fails and calls the CPU
>>>>> notifier CPU_DOWN_FAILED hook the following assert triggers in the
>>>>> scheduling code:
>>>>>
>>>>> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
>>>>> ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
>>>>> CPU:    0
>>>>> RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
>>>>> RFLAGS: 0000000000010093   CONTEXT: hypervisor
>>>>> rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
>>>>> rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
>>>>> rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
>>>>> r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
>>>>> r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
>>>>> r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
>>>>> cr3: 00000000574c2000   cr2: 0000000000000000
>>>>> fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
>>>>> ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
>>>>> Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
>>>>>     fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
>>>>> Xen stack trace from rsp=ffff83203ffffd30:
>>>>>       ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
>>>>>       0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
>>>>>       0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
>>>>>       ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
>>>>>       ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
>>>>>       00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
>>>>>       ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
>>>>>       ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
>>>>>       ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
>>>>>       ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
>>>>>       ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
>>>>>       0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
>>>>>       ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
>>>>>       0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
>>>>>       ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
>>>>>       0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>>>       0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>>>       0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>>>       0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>>>       0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>>>> Xen call trace:
>>>>>       [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
>>>>>       [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
>>>>>       [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
>>>>>       [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
>>>>>       [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
>>>>>       [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
>>>>>       [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
>>>>>       [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
>>>>>       [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
>>>>>       [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
>>>>>       [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
>>>>>       [<ffff82d040324391>] F continue_running+0x5b/0x5d
>>>>>
>>>>>
>>>>> ****************************************
>>>>> Panic on CPU 0:
>>>>> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
>>>>> ****************************************
>>>>>
>>>>> The issue seems to be that since the CPU hasn't been removed, it's
>>>>> still part of prv->initialized and the assert in csched2_free_pdata()
>>>>> called as part of free_cpu_rm_data() triggers.
>>>>>
>>>>> It's easy to reproduce by substituting the stop_machine_run() call in
>>>>> cpu_down() with an error.
>>>>
>>>> Could you please give the attached patch a try?
>>>
>>> I still get the following assert:
>>
>> Oh, silly me. Without core scheduling active nr_sr_unused will be 0 all
>> the time. :-(
>>
>> Next try.
> 
> I'm afraid I have a new trace for you:
> 
> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
> ----[ Xen-4.19-unstable  x86_64  debug=y  Not tainted ]----
> CPU:    0
> RIP:    e008:[<ffff82d040247d27>] common/sched/credit2.c#csched2_switch_sched+0x115/0x339
> RFLAGS: 0000000000010093   CONTEXT: hypervisor
> rax: 000000000000c000   rbx: 0000000000000001   rcx: ffff82d0405e6500
> rdx: 0000004feee13000   rsi: 0000000000000004   rdi: ffff83202ecc2f88
> rbp: ffff83203ffffc80   rsp: ffff83203ffffc38   r8:  0000000000000000
> r9:  ffff83202ecbbf01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> r12: ffff83202ecc2f80   r13: ffff83402ca50100   r14: ffff83402ca50140
> r15: ffff83202ecc2f88   cr0: 000000008005003b   cr4: 00000000007526e0
> cr3: 00000000574c2000   cr2: 0000000000000000
> fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> Xen code around <ffff82d040247d27> (common/sched/credit2.c#csched2_switch_sched+0x115/0x339):
>   7c ff ff ff 0f 0b 0f 0b <0f> 0b 0f 0b 41 8b 56 30 89 de 48 8d 3d e8 00 1a
> Xen stack trace from rsp=ffff83203ffffc38:
>     ffff83203ffffc48 ffff82d0402332ba ffff83203ffffc68 ffff82d04023343d
>     0000000000000001 ffff82d0405cf398 ffff83402ca50100 ffff82d0405e6500
>     ffff83202ecbbdb0 ffff83203ffffd18 ffff82d040256e1a ffff83203fff386c
>     ffff83203fff2000 0000000000000005 ffff83202ecbbf00 ffff83402ca50140
>     ffff83203fff3868 0000000000000282 0000000040233509 ffff83202ecbbdb0
>     ffff83402ca50100 ffff83202f3e6d80 ffff83202ecc2ec0 ffff83202ecc2ec0
>     0000000000000001 ffff82d0403da460 0000000000000048 0000000000000000
>     ffff83203ffffd48 ffff82d0402414b7 0000000000000001 0000000000000000
>     ffff82d0403da460 0000000000000006 ffff83203ffffd70 ffff82d04024173d
>     0000000000000000 0000000000000001 ffff82d0404c4430 ffff83203ffffda0
>     ffff82d04021f1f9 ffff82d0404c4628 ffff82d0404c4620 ffff82d0404c4430
>     0000000000000006 ffff83203ffffde8 ffff82d04022bb2f ffff83203ffffe10
>     0000000000000001 0000000000000001 0000000000000000 ffff83203ffffe10
>     0000000000000000 ffff82d0405e6500 ffff83203ffffe00 ffff82d040204fd5
>     0000000000000001 ffff83203ffffe30 ffff82d040205464 ffff82d0404c5860
>     0000000000000001 ffff83202ec86000 0000000000000000 ffff83203ffffe48
>     ffff82d040348c32 ffff83402ca500d0 ffff83203ffffe68 ffff82d04020708d
>     ffff83202ec861d0 ffff82d0405ce210 ffff83203ffffe80 ffff82d0402342a3
>     ffff82d0405ce200 ffff83203ffffeb0 ffff82d04023450b 0000000000000000
>     0000000000007fff ffff82d0405d5080 ffff82d0405ce210 ffff83203ffffee8
> Xen call trace:
>     [<ffff82d040247d27>] R common/sched/credit2.c#csched2_switch_sched+0x115/0x339
>     [<ffff82d040256e1a>] F schedule_cpu_add+0x1a4/0x463
>     [<ffff82d0402414b7>] F common/sched/cpupool.c#cpupool_assign_cpu_locked+0x5a/0x17e
>     [<ffff82d04024173d>] F common/sched/cpupool.c#cpupool_cpu_add+0x162/0x16c
>     [<ffff82d04021f1f9>] F common/sched/cpupool.c#cpu_callback+0x10e/0x466
>     [<ffff82d04022bb2f>] F notifier_call_chain+0x6c/0x96
>     [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
>     [<ffff82d040205464>] F cpu_down+0x60/0x83
>     [<ffff82d040348c32>] F cpu_down_helper+0x11/0x27
>     [<ffff82d04020708d>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
>     [<ffff82d0402342a3>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
>     [<ffff82d04023450b>] F do_tasklet+0x5b/0x8d
>     [<ffff82d040321372>] F arch/x86/domain.c#idle_loop+0x78/0xe6
>     [<ffff82d0403242f2>] F continue_running+0x5b/0x5d
> 
> 
> ****************************************
> Panic on CPU 0:
> Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
> ****************************************
> 
> This time is one of the asserts in init_pdata().

Yeah, the reason is similar, but fixing this is a little bit more work
than the other patch.

Not sure I'll manage to do this before Xen Summit.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu May 30 13:14:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 13:14:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732713.1138741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfcC-0006Ma-Hr; Thu, 30 May 2024 13:14:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732713.1138741; Thu, 30 May 2024 13:14:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfcC-0006MT-FM; Thu, 30 May 2024 13:14:44 +0000
Received: by outflank-mailman (input) for mailman id 732713;
 Thu, 30 May 2024 13:14:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCfcA-0006MN-Md
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 13:14:42 +0000
Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com
 [2607:f8b0:4864:20::729])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9297fae5-1e86-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 15:14:41 +0200 (CEST)
Received: by mail-qk1-x729.google.com with SMTP id
 af79cd13be357-794ab0eb68cso92588285a.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 06:14:41 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43fb18b027fsm65531381cf.76.2024.05.30.06.14.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 06:14:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9297fae5-1e86-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717074880; x=1717679680; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Xbga/+JxbiPkiiuMYw3s9l83Twi+4Hqsqx9Nq475CQE=;
        b=n7yy8E0eXoou5q3o3G/4yB1CPizBZgUjpQfKf2dLfCs+SYcj7XQgzBgzAB1EOIev5I
         dVWzf1JzKzgRR1Y2NCh9ifv4fP0hpL3pD149sIfqHnybyp6b43rDumLSPWRvTbepD7Qn
         0CLilJpvYDCftipOekdzYC6bvT+HcyH4OHccU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717074880; x=1717679680;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Xbga/+JxbiPkiiuMYw3s9l83Twi+4Hqsqx9Nq475CQE=;
        b=Y8zoBi3CH0NKrehZDwLsEy1YqTisa5UGNVAAbNO8/HrxhyeUm42/aHAol9vBCsSee/
         ulaweZ47Ck5GcvUK9w97tOsOVvpzzzAk+928JAmmD/DBnvoQ1XoyBMkJSGF88IZcB1qB
         swReazKbHvytZ/qyFfVdt8idBJ2PTwKA7vJd1hOOyHPglNx7rDtvYWhVTkEC08pngOwm
         1ubOmJBnYgunzKbNUOs0I5F7U1Y5b82JPjuBK04pPU/S0P667iY609HibTLeDc+5Luu0
         SF9F+MIoD5dq00G/jD9Y3fAQS/78fNwoJ4sxzGRPyZ3UAr3NyFyAcKdwohb0VfzfTWiI
         dd4g==
X-Gm-Message-State: AOJu0YyCUDArL9miVagZsUn2ged9dl0nHM5TdWUjAi2x9CEln4T1YCA3
	1vit8yFHRgiGqlvdyQOF1GuPh5DzewhAcIm2Gw46KF/+futyOFQr9aRANHusrAI=
X-Google-Smtp-Source: AGHT+IHIgaB7UKhhdbvoRc5d1psxZQ5SKNxQT1S/t9pW8iWAFFgunQOihZvr30j2U2Bz42sMR085IQ==
X-Received: by 2002:a05:6214:48d:b0:6ab:6d2c:ceee with SMTP id 6a1803df08f44-6ae0faf57a6mr26698856d6.21.1717074879875;
        Thu, 30 May 2024 06:14:39 -0700 (PDT)
Date: Thu, 30 May 2024 15:14:37 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Anthony PERARD <anthony@xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: Re: [PATCH] tools: (Actually) drop libsystemd as a dependency
Message-ID: <Zlh7vRwvICxVg06f@macbook>
References: <20240530101439.16730-1-andrew.cooper3@citrix.com>
 <ZlhcyOmu_Cc92pP5@macbook>
 <71bbc310-1093-44d1-a0b0-e5a8ab10a0ea@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <71bbc310-1093-44d1-a0b0-e5a8ab10a0ea@citrix.com>

On Thu, May 30, 2024 at 12:12:19PM +0100, Andrew Cooper wrote:
> On 30/05/2024 12:02 pm, Roger Pau Monné wrote:
> > On Thu, May 30, 2024 at 11:14:39AM +0100, Andrew Cooper wrote:
> >> When reinstating some of systemd.m4 between v1 and v2, I reintroduced a little
> >> too much.  While {c,o}xenstored are indeed no longer linked against
> >> libsystemd, ./configure still looks for it.
> >>
> >> Drop this too.
> >>
> >> Fixes: ae26101f6bfc ("tools: Drop libsystemd as a dependency")
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > LGTM, but my knowledge of systemd is very limited.
> >
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.  TBH, this is all M4/autoconf, rather than systemd.

Right, but it's about systemd dependencies which is what I don't know
about.  The m4 stuff LGTM, whether it's appropriate to drop the
dependency is what I can't be sure about.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 13:15:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 13:15:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732716.1138752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfdF-0006r8-Rs; Thu, 30 May 2024 13:15:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732716.1138752; Thu, 30 May 2024 13:15:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfdF-0006r1-OW; Thu, 30 May 2024 13:15:49 +0000
Received: by outflank-mailman (input) for mailman id 732716;
 Thu, 30 May 2024 13:15:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCfdE-0006qr-ND
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 13:15:48 +0000
Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com
 [2607:f8b0:4864:20::733])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ba12cf6b-1e86-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 15:15:47 +0200 (CEST)
Received: by mail-qk1-x733.google.com with SMTP id
 af79cd13be357-794ab12341aso88920485a.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 06:15:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abcdca88sm552340385a.66.2024.05.30.06.15.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 06:15:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba12cf6b-1e86-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717074946; x=1717679746; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=9+qqFQq3i9fpO4By1QcUnaKmtZDkBza7jyHqUqOSYUQ=;
        b=tX0lr+N6YI/pTKoLIATLWxIpa+BfgaF4yI1JAIMvnpRrc86W2Vtzcns1lwITC1To1O
         R1J1JqSMOrbleYDcHS8iLwmLObV6NKGZcjEwNiePhCvmqooFYzb0U3fI9fVgWfDTRlt+
         FKU40fBf9syCFd8CoQblaPKj2+XagwO1dz8CU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717074946; x=1717679746;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9+qqFQq3i9fpO4By1QcUnaKmtZDkBza7jyHqUqOSYUQ=;
        b=JPRIvimqr5vYfKxohKsda7y1WOYtVMmWQTZlqCOzN7P/O6krNpu+0c5dHsCadDdOBj
         1fK3RRMoCfp/0YjFO65Ze9JgZjfUl9o1ra270gH4kZN7on6n7bGPRdv8sVga5OCtoTWf
         z+fkdC6uvaOMz8chTyvV0zmy/6bcH1UdmYLTJHfDclgHQszqIyGuttf6K2x+3nNmZqzg
         T5D2xxY1PXpJVMjre5Yv/yZBiOCmFe2rZkzjyCoGT4ViDW420kqxnr91Rk+D4x5OVj02
         YttSd7twIng22mkYplA6urPQELlxaD2+OCxczydQfn+5Yqshp5XzTy6VjxRfn2L01q4z
         9uRw==
X-Gm-Message-State: AOJu0YzlcTSp2SzwSrGT5eLfaqe3160qUOnQ4N03TbxebIcYtjIEf/59
	+hxGdpUoRKUHlE3GvkBh6QSs0VfZoNXAD9HuU6HvAKt0yAiesxWrFzxVRRxXbZM=
X-Google-Smtp-Source: AGHT+IFsmOvsoe0XMJa9f4G/2/j2L1HfkUjQfPzr6qOAXFu6m/dFtUDSbMZnR//tpZ5JwNRbTxDxmA==
X-Received: by 2002:a05:620a:4506:b0:794:e68c:260a with SMTP id af79cd13be357-794eaeb14aemr314917085a.4.1717074946225;
        Thu, 30 May 2024 06:15:46 -0700 (PDT)
Date: Thu, 30 May 2024 15:15:43 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Dario Faggioli <dfaggioli@suse.com>,
	George Dunlap <george.dunlap@citrix.com>
Subject: Re: CPU_DOWN_FAILED hits ASSERTs in scheduling logic
Message-ID: <Zlh7_zYCV0Pc8ZUl@macbook>
References: <ZlW-ZyFeAzvh3pGG@macbook>
 <a749c066-0c05-47dd-b499-e712e5454033@suse.com>
 <ZlcjiOJyAuvyy48n@macbook>
 <1147a06d-d9ec-4fa9-a1d0-dfecd7a670dc@suse.com>
 <ZldRusciaXuwTC9u@macbook>
 <bba693b3-ad73-4fb8-ae9c-dba025f6f5c1@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bba693b3-ad73-4fb8-ae9c-dba025f6f5c1@suse.com>

On Thu, May 30, 2024 at 02:45:18PM +0200, Jürgen Groß wrote:
> On 29.05.24 18:03, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 03:08:49PM +0200, Jürgen Groß wrote:
> > > On 29.05.24 14:46, Roger Pau Monné wrote:
> > > > On Wed, May 29, 2024 at 01:47:09PM +0200, Jürgen Groß wrote:
> > > > > On 28.05.24 13:22, Roger Pau Monné wrote:
> > > > > > Hello,
> > > > > > 
> > > > > > When the stop_machine_run() call in cpu_down() fails and calls the CPU
> > > > > > notifier CPU_DOWN_FAILED hook the following assert triggers in the
> > > > > > scheduling code:
> > > > > > 
> > > > > > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/cred1
> > > > > > ----[ Xen-4.19-unstable  x86_64  debug=y  Tainted:   C    ]----
> > > > > > CPU:    0
> > > > > > RIP:    e008:[<ffff82d040248299>] common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> > > > > > RFLAGS: 0000000000010093   CONTEXT: hypervisor
> > > > > > rax: 0000000000000000   rbx: ffff83202ecc2f80   rcx: ffff83202f3e64c0
> > > > > > rdx: 0000000000000001   rsi: 0000000000000002   rdi: ffff83202ecc2f88
> > > > > > rbp: ffff83203ffffd58   rsp: ffff83203ffffd30   r8:  0000000000000000
> > > > > > r9:  ffff83202f3e6e01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> > > > > > r12: ffff83202ecb80b0   r13: 0000000000000001   r14: 0000000000000282
> > > > > > r15: ffff83202ecbbf00   cr0: 000000008005003b   cr4: 00000000007526e0
> > > > > > cr3: 00000000574c2000   cr2: 0000000000000000
> > > > > > fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> > > > > > ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> > > > > > Xen code around <ffff82d040248299> (common/sched/credit2.c#csched2_free_pdata+0xc8/0x177):
> > > > > >     fe ff eb 9a 0f 0b 0f 0b <0f> 0b 49 8d 4f 08 49 8b 47 08 48 3b 48 08 75 2e
> > > > > > Xen stack trace from rsp=ffff83203ffffd30:
> > > > > >       ffff83202d74d100 0000000000000001 ffff82d0404c4430 0000000000000006
> > > > > >       0000000000000000 ffff83203ffffd78 ffff82d040257454 0000000000000000
> > > > > >       0000000000000001 ffff83203ffffda8 ffff82d04021f303 ffff82d0404c4628
> > > > > >       ffff82d0404c4620 ffff82d0404c4430 0000000000000006 ffff83203ffffdf0
> > > > > >       ffff82d04022bc4c ffff83203ffffe18 0000000000000001 0000000000000001
> > > > > >       00000000fffffff0 0000000000000000 0000000000000000 ffff82d0405e6500
> > > > > >       ffff83203ffffe08 ffff82d040204fd5 0000000000000001 ffff83203ffffe30
> > > > > >       ffff82d0402054f0 ffff82d0404c5860 0000000000000001 ffff83202ec75000
> > > > > >       ffff83203ffffe48 ffff82d040348c25 ffff83202d74d0d0 ffff83203ffffe68
> > > > > >       ffff82d0402071aa ffff83202ec751d0 ffff82d0405ce210 ffff83203ffffe80
> > > > > >       ffff82d0402343c9 ffff82d0405ce200 ffff83203ffffeb0 ffff82d040234631
> > > > > >       0000000000000000 0000000000007fff ffff82d0405d5080 ffff82d0405ce210
> > > > > >       ffff83203ffffee8 ffff82d040321411 ffff82d040321399 ffff83202f3a9000
> > > > > >       0000000000000000 0000001d91a6fa2d ffff82d0405e6500 ffff83203ffffde0
> > > > > >       ffff82d040324391 0000000000000000 0000000000000000 0000000000000000
> > > > > >       0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > > >       0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > > >       0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > > >       0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > > >       0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > > > > > Xen call trace:
> > > > > >       [<ffff82d040248299>] R common/sched/credit2.c#csched2_free_pdata+0xc8/0x177
> > > > > >       [<ffff82d040257454>] F free_cpu_rm_data+0x41/0x58
> > > > > >       [<ffff82d04021f303>] F common/sched/cpupool.c#cpu_callback+0xfb/0x466
> > > > > >       [<ffff82d04022bc4c>] F notifier_call_chain+0x6c/0x96
> > > > > >       [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
> > > > > >       [<ffff82d0402054f0>] F cpu_down+0xa7/0x143
> > > > > >       [<ffff82d040348c25>] F cpu_down_helper+0x11/0x27
> > > > > >       [<ffff82d0402071aa>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
> > > > > >       [<ffff82d0402343c9>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
> > > > > >       [<ffff82d040234631>] F do_tasklet+0x5b/0x8d
> > > > > >       [<ffff82d040321411>] F arch/x86/domain.c#idle_loop+0x78/0xe6
> > > > > >       [<ffff82d040324391>] F continue_running+0x5b/0x5d
> > > > > > 
> > > > > > 
> > > > > > ****************************************
> > > > > > Panic on CPU 0:
> > > > > > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:4111
> > > > > > ****************************************
> > > > > > 
> > > > > > The issue seems to be that since the CPU hasn't been removed, it's
> > > > > > still part of prv->initialized and the assert in csched2_free_pdata()
> > > > > > called as part of free_cpu_rm_data() triggers.
> > > > > > 
> > > > > > It's easy to reproduce by substituting the stop_machine_run() call in
> > > > > > cpu_down() with an error.
> > > > > 
> > > > > Could you please give the attached patch a try?
> > > > 
> > > > I still get the following assert:
> > > 
> > > Oh, silly me. Without core scheduling active nr_sr_unused will be 0 all
> > > the time. :-(
> > > 
> > > Next try.
> > 
> > I'm afraid I have a new trace for you:
> > 
> > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
> > ----[ Xen-4.19-unstable  x86_64  debug=y  Not tainted ]----
> > CPU:    0
> > RIP:    e008:[<ffff82d040247d27>] common/sched/credit2.c#csched2_switch_sched+0x115/0x339
> > RFLAGS: 0000000000010093   CONTEXT: hypervisor
> > rax: 000000000000c000   rbx: 0000000000000001   rcx: ffff82d0405e6500
> > rdx: 0000004feee13000   rsi: 0000000000000004   rdi: ffff83202ecc2f88
> > rbp: ffff83203ffffc80   rsp: ffff83203ffffc38   r8:  0000000000000000
> > r9:  ffff83202ecbbf01   r10: 0000000000000000   r11: 0f0f0f0f0f0f0f0f
> > r12: ffff83202ecc2f80   r13: ffff83402ca50100   r14: ffff83402ca50140
> > r15: ffff83202ecc2f88   cr0: 000000008005003b   cr4: 00000000007526e0
> > cr3: 00000000574c2000   cr2: 0000000000000000
> > fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> > ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> > Xen code around <ffff82d040247d27> (common/sched/credit2.c#csched2_switch_sched+0x115/0x339):
> >   7c ff ff ff 0f 0b 0f 0b <0f> 0b 0f 0b 41 8b 56 30 89 de 48 8d 3d e8 00 1a
> > Xen stack trace from rsp=ffff83203ffffc38:
> >     ffff83203ffffc48 ffff82d0402332ba ffff83203ffffc68 ffff82d04023343d
> >     0000000000000001 ffff82d0405cf398 ffff83402ca50100 ffff82d0405e6500
> >     ffff83202ecbbdb0 ffff83203ffffd18 ffff82d040256e1a ffff83203fff386c
> >     ffff83203fff2000 0000000000000005 ffff83202ecbbf00 ffff83402ca50140
> >     ffff83203fff3868 0000000000000282 0000000040233509 ffff83202ecbbdb0
> >     ffff83402ca50100 ffff83202f3e6d80 ffff83202ecc2ec0 ffff83202ecc2ec0
> >     0000000000000001 ffff82d0403da460 0000000000000048 0000000000000000
> >     ffff83203ffffd48 ffff82d0402414b7 0000000000000001 0000000000000000
> >     ffff82d0403da460 0000000000000006 ffff83203ffffd70 ffff82d04024173d
> >     0000000000000000 0000000000000001 ffff82d0404c4430 ffff83203ffffda0
> >     ffff82d04021f1f9 ffff82d0404c4628 ffff82d0404c4620 ffff82d0404c4430
> >     0000000000000006 ffff83203ffffde8 ffff82d04022bb2f ffff83203ffffe10
> >     0000000000000001 0000000000000001 0000000000000000 ffff83203ffffe10
> >     0000000000000000 ffff82d0405e6500 ffff83203ffffe00 ffff82d040204fd5
> >     0000000000000001 ffff83203ffffe30 ffff82d040205464 ffff82d0404c5860
> >     0000000000000001 ffff83202ec86000 0000000000000000 ffff83203ffffe48
> >     ffff82d040348c32 ffff83402ca500d0 ffff83203ffffe68 ffff82d04020708d
> >     ffff83202ec861d0 ffff82d0405ce210 ffff83203ffffe80 ffff82d0402342a3
> >     ffff82d0405ce200 ffff83203ffffeb0 ffff82d04023450b 0000000000000000
> >     0000000000007fff ffff82d0405d5080 ffff82d0405ce210 ffff83203ffffee8
> > Xen call trace:
> >     [<ffff82d040247d27>] R common/sched/credit2.c#csched2_switch_sched+0x115/0x339
> >     [<ffff82d040256e1a>] F schedule_cpu_add+0x1a4/0x463
> >     [<ffff82d0402414b7>] F common/sched/cpupool.c#cpupool_assign_cpu_locked+0x5a/0x17e
> >     [<ffff82d04024173d>] F common/sched/cpupool.c#cpupool_cpu_add+0x162/0x16c
> >     [<ffff82d04021f1f9>] F common/sched/cpupool.c#cpu_callback+0x10e/0x466
> >     [<ffff82d04022bb2f>] F notifier_call_chain+0x6c/0x96
> >     [<ffff82d040204fd5>] F common/cpu.c#cpu_notifier_call_chain+0x1b/0x36
> >     [<ffff82d040205464>] F cpu_down+0x60/0x83
> >     [<ffff82d040348c32>] F cpu_down_helper+0x11/0x27
> >     [<ffff82d04020708d>] F common/domain.c#continue_hypercall_tasklet_handler+0x50/0xbd
> >     [<ffff82d0402342a3>] F common/tasklet.c#do_tasklet_work+0x76/0xaf
> >     [<ffff82d04023450b>] F do_tasklet+0x5b/0x8d
> >     [<ffff82d040321372>] F arch/x86/domain.c#idle_loop+0x78/0xe6
> >     [<ffff82d0403242f2>] F continue_running+0x5b/0x5d
> > 
> > 
> > ****************************************
> > Panic on CPU 0:
> > Assertion '!cpumask_test_cpu(cpu, &prv->initialized)' failed at common/sched/credit2.c:3987
> > ****************************************
> > 
> > This time is one of the asserts in init_pdata().
> 
> Yeah, the reason is similar, but fixing this is a little bit more work
> than the other patch.
> 
> Not sure I'll manage to do this before Xen Summit.

No worries, I'm not in a rush.  I'm happy as long as it's on your
plate and not mine :).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 13:23:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 13:23:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732725.1138762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfkK-0000GE-OZ; Thu, 30 May 2024 13:23:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732725.1138762; Thu, 30 May 2024 13:23:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCfkK-0000G7-KZ; Thu, 30 May 2024 13:23:08 +0000
Received: by outflank-mailman (input) for mailman id 732725;
 Thu, 30 May 2024 13:23:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lthE=NB=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sCfkJ-0000G1-Ec
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 13:23:07 +0000
Received: from EUR03-DBA-obe.outbound.protection.outlook.com
 (mail-dbaeur03on20601.outbound.protection.outlook.com
 [2a01:111:f403:260d::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bf9e4918-1e87-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 15:23:06 +0200 (CEST)
Received: from AS4P189CA0060.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:659::15)
 by AS8PR08MB7864.eurprd08.prod.outlook.com (2603:10a6:20b:52f::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.30; Thu, 30 May
 2024 13:23:00 +0000
Received: from AM1PEPF000252DB.eurprd07.prod.outlook.com
 (2603:10a6:20b:659:cafe::c6) by AS4P189CA0060.outlook.office365.com
 (2603:10a6:20b:659::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21 via Frontend
 Transport; Thu, 30 May 2024 13:23:00 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM1PEPF000252DB.mail.protection.outlook.com (10.167.16.53) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15
 via Frontend Transport; Thu, 30 May 2024 13:23:00 +0000
Received: ("Tessian outbound a0025cf119c5:v327");
 Thu, 30 May 2024 13:22:59 +0000
Received: from 7a062bfab231.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6F2FFDA0-1461-4C6A-A8D9-0EEF65A1F537.1; 
 Thu, 30 May 2024 13:22:53 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7a062bfab231.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 May 2024 13:22:53 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by AS8PR08MB7887.eurprd08.prod.outlook.com (2603:10a6:20b:52b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Thu, 30 May
 2024 13:22:50 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Thu, 30 May 2024
 13:22:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf9e4918-1e87-11ef-90a1-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=TowmUU8xO316RJ7dL9LKEFjVUnBj7gTIOJRV/JwDmGzBU9cwN+jJHFJtYASYfoXIQPMP/pn6ZnNcwIimyLmsiMLa3tLg1YtryjXjRXXOahkMffeQWd6H+UJD3MMHT4BPvcgugo4ksq7a4L8T9BpcWqOWoyHPt5qkuYpVDcl8H7Au6o2YMftwKEnbZfAZDONqCzcsb19UfL/E+DIeUVNUNkmwp72/Pl8hCwt+Zsc7XVqd2qm40zNO+T4ZrVficooYu+I0LxiIzPRqfGT9MWInGiPrLrLhX1rLHIbP8kquDsfPGX0FJbxKJydJ+uV7XBeWq21dvbwOmXbhw/htx86n/g==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xvybumCqWgkYIZD6Th8cbehhH1VZ5yOfQbP5Ck7p3bU=;
 b=FkmpZ7iv/YO9eZiItsRKl9FhoR9nFpgY6D5esr8SZN8qMn++KinGADW8ooSxeHWkeYuQOsWF/d89VTd9TvlerNcxzcNlO4psXTsaqV3IhL2o3wiyxulvBH9L7bxkHncd6NvH0MjSEmltKnZ3F6ojBy1p20mCJQe4hffACF+M8ptjeCeWXBV6N5gFd1NA4CyXc7r5319xQqrjU8aifSeSb/8ktIdP8/lQVwXhc9/zuzQuT1bf/Qv/OqpqIOfdzQKgIEovunKKc0Udr4XD39W57FIX48aNFTlA9fuZD+zxQ8uDRlzfsO0flv07VzR3tvnDwMcwWuRMHzCtDFvc5pkH1w==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xvybumCqWgkYIZD6Th8cbehhH1VZ5yOfQbP5Ck7p3bU=;
 b=jz/ysq173zQtZnvCCrKjxUFV+wKzAVQGWy5+XHDW66IjZ/dTfY7u5/g8/zrwW/+j1CmEaJo0C1Vnq0aWV5xx1QgCIGCOJ/FR+uTMdTZNLbaRAWkiFsMbBJ9dDZqGvmvSUA1YORDeHRHIpLJ/tUtcb3Zh6055oWEEipsUSS0ycTk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 38323b2e27ca1ca1
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UNVsBOsGqqIia0frtU04RjPQDJJTjqpZDVqfiI01idOnN/1KO0BC4C6oeMxVXEHIDn+Ext/aDCsN0WIpZG+PTmhvxknRs0uMlAfFuz1qbkCoYUEmB/ksMJfDW8gDn0yFbSxOGVpEGZlzs2/iQun1iOvQzuEg5nkNVkFAdKf3qfC73E7/NkyXdOTxeCnRg8AeCR60vvuD3P6OX+bvvvWdxxvUCisSBKkrdD2Q+UhxPNVsgUG9oMUItO4lShMGRqhuFdimbllktAoMwBCifle9S5iTHDuLaLcafA7AL8zu+ea9QFo82X5zEcy9HVBRgGZSxCxJEJJhCKxc0Lw714x/5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xvybumCqWgkYIZD6Th8cbehhH1VZ5yOfQbP5Ck7p3bU=;
 b=JxG/rt/x+ucDP4QVGeHMwCdBWEiqxsx2VethMvl2rL9/NaEUn/tjSPKaH8s7LYyZjZenrqPqDucyTngHtKC2xta0IZtl9vv1EjfTRMaHSPFSiBDRostRXsRm9CkiqinmoOYsx1dw01TS6/YASNuLfVx8OxP5PELp/0ZVd6EkAvcejrRg6Zn02Q0s2VLhhHRfBJ1QyJpnuTCGQSlh1MJszDp9786QbdrNqDnicsBoUwDDV7hArDx243toCsU+CltuI99dSntoIuCuoyI0PrwC4vsYhJ4cE01+7OA9XxSW7B67LfkZZN1dGFAhGObQxzVCGLazwHelPtH0+Rc0du1Giw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xvybumCqWgkYIZD6Th8cbehhH1VZ5yOfQbP5Ck7p3bU=;
 b=jz/ysq173zQtZnvCCrKjxUFV+wKzAVQGWy5+XHDW66IjZ/dTfY7u5/g8/zrwW/+j1CmEaJo0C1Vnq0aWV5xx1QgCIGCOJ/FR+uTMdTZNLbaRAWkiFsMbBJ9dDZqGvmvSUA1YORDeHRHIpLJ/tUtcb3Zh6055oWEEipsUSS0ycTk=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Michal Orzel <michal.orzel@amd.com>
Subject: Re: [PATCH] arm: dom0less: add TEE support
Thread-Topic: [PATCH] arm: dom0less: add TEE support
Thread-Index: AQHasgjZKwS4pdCyJUe9LewXYMLHeLGutaSAgAAAkICAANCCAIAAD1cAgAAuxIA=
Date: Thu, 30 May 2024 13:22:50 +0000
Message-ID: <E337CF92-42DE-424C-95D5-97FF096885B1@arm.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
 <CBB42DF2-94AE-4566-B83D-3AB3CB2E01DE@arm.com>
 <047ca670-cd9e-4803-8083-7770b7230042@xen.org>
In-Reply-To: <047ca670-cd9e-4803-8083-7770b7230042@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|AS8PR08MB7887:EE_|AM1PEPF000252DB:EE_|AS8PR08MB7864:EE_
X-MS-Office365-Filtering-Correlation-Id: 6e4c6108-b476-4edd-a8ac-08dc80aba0bc
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?kdS/2RUbUNW0R4jGkJ10mXHxtMZ0XWEIRdUVpBi4DFpFcwc6YfGd2vHX12Kf?=
 =?us-ascii?Q?5re2sFKXNG6gU+YxiMwN9d4M5IiPps7F9a1XIhNgpza0R1N07QgBFKcW1ycD?=
 =?us-ascii?Q?mj6teShBRBPbAyi0Ro1RzLEqGvMsaXg1/hTGMe27RxokOtZLMNzjQ8koidEj?=
 =?us-ascii?Q?hOhU7fjZ9IFtiyqO1dssd9SlE/FmwpitI1Yet0RbIDlzTernbT5bxTqFkwkf?=
 =?us-ascii?Q?wWU6L8zmeQ8DaMYeYY/LEG/Q4bM1MgwgHrswUTlGeEZPb8CWDXdzGbsw3y5h?=
 =?us-ascii?Q?ZQ1vRotfTRdSvmiW3HB3s3SNnJURF4hw2xNw2M4YzMmTPlFk9D+DkXFh1jxY?=
 =?us-ascii?Q?rbJfgSNrlnyaYuVn2kYDYfDO8IzeC+h+G3G5coYfxWT5A2YFOfBLUamTv6iy?=
 =?us-ascii?Q?A+v6686Lfzd8N2uAk+2GltvFakIqY2DIeAknBOSyKGImxwrEG22vayH3FI2m?=
 =?us-ascii?Q?NW/58O33khc1TCRyiv+CJtH6is1z0KNG1QQZXK/Ucqfzi+nT8ulbcQeq/d6p?=
 =?us-ascii?Q?+DtbKjAg4YFjh7QiVxqcsQgGYOEvLik2OwEKZJDx+MOK0XEorB71xtaBag41?=
 =?us-ascii?Q?96aDHNB6LrVIMqaJfPFC7MWoZVFEb/FpNH/Dek8sxcuZONLqgXBNlC+f20XI?=
 =?us-ascii?Q?LMpAtH+luxlOY9jIoskBaGxPOf9sPWuZ1Up+V4kEQ6obB9VA3w9nPnmJaVbs?=
 =?us-ascii?Q?CfIxaW1UwBOgm+DPj/cCXoRQexVG7Aajgtl5J31zLFowWGzvJtSbGYjwtN2t?=
 =?us-ascii?Q?IkSoITRGJ+bE64ZnjHOF4wUacxwpXP/S1IEoB+2A3UC7o2uI4ulBz8CCt8ez?=
 =?us-ascii?Q?mk1h+QpRjdgKVbtUvxHqyvSGhu10DvhlTD7l6wJ8co81E22TJse8sRF3UpJE?=
 =?us-ascii?Q?hpCD7GSyUoaEECOVWQYcWrF//TDfqdeDO1jMCLQ4i2+Su4gV8mpJ6tPt0B7L?=
 =?us-ascii?Q?I/d3zqeRB4yRMHTqXFp9MgZSbv4KjW9FU/CBFTBnMJDqmM+qENMDbWgmL4Un?=
 =?us-ascii?Q?VSgnTyfVigQrdCvPf+uSmVV0L4S7pk1J/1Ps7DYuMFhMwj7bhqxDnQUJ38S6?=
 =?us-ascii?Q?9TNX+mppE52RKnHw4XCfEBN+JzMS5aHDgRKRx/dcfwnY0jWgMpW5cvoP3481?=
 =?us-ascii?Q?3QWYNtquxZ1rxx6a3MeQla70omB7dIWCVjwd+YtX+Uw8su+alokg5wExAkCJ?=
 =?us-ascii?Q?U5Pey7I2aKHVElaAiPKMMfrWOzM3T+tfaHXw6clvfmxUD4y3iPd+tiKu0iUC?=
 =?us-ascii?Q?TXZ46l1STS6rO3FF1dTWVxHxvDcNnkW8a/DzbraHAkRgNJ2qz9JMlMsspanc?=
 =?us-ascii?Q?V0y275bFccECsecG0li2cIhG4JdRu24W6hh4ipWqiBM7/g=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8019A8B62F3DF64FAB43FA7E37E20A49@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7887
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM1PEPF000252DB.eurprd07.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d2047446-bb4a-409e-4e10-08dc80ab9aa1
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|36860700004|82310400017|1800799015|35042699013|376005;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?Y79kZ13BaDQOLFzbG24mHVWiRKFSQOng48rh6R5GO75mOS10DpEa2pYZALJH?=
 =?us-ascii?Q?Pv0fCYEiLKOJVuDxJ9cYjDmru3A7nGuTcrnR4cY+WuDQgQ9x6RJF1L74fdKT?=
 =?us-ascii?Q?Nk64SS08oKiH7TCKE93byc0T1xB+QF3mkPri+uPfZwdxiGP9jvc4IRJJfGk0?=
 =?us-ascii?Q?LzuHexstM8s/7V6rD+zcVPYh2xRi/x6rc1hWYPW9x6M7ETfb5xItqvzratAz?=
 =?us-ascii?Q?xrFFc1Y0QBW8p60MDNlB+qVb/RorHq9LLu+MUQ0E2lODvgjbR3lNrxepD/U0?=
 =?us-ascii?Q?hAPIJ3OUIWzavZFYMUetEsPa7FMF/fLgzEs2NcWPvK0jFUAytf+y+0bDOMOt?=
 =?us-ascii?Q?7XrKDaWqB9C1E45Jl8tzv7DLtDBDBrhVmIzHyosc8CKtwfwnD8idu1mX6j3y?=
 =?us-ascii?Q?gISaAYABC7g1lPI7REoYIQnswB5JSGBon4cZJAEbB786bRHpe0RPkPOGWguE?=
 =?us-ascii?Q?Z3UAxU8oLvkUTj7S6naSPW3dQX50B6anuaOiTeokQSEy9RBTMwNuS74175ab?=
 =?us-ascii?Q?yq+PUhA5rCTdOYH4gPHejDxEtHWNrKeZD5VYabPhKbWWfMeZDEdI7pu6dBax?=
 =?us-ascii?Q?QOUVlsiFp1Bs3UyKf8SQXmyP1Ykqn8sM/GZBkDJ7lw+5eygARizlLH6BudFs?=
 =?us-ascii?Q?efh1mVP5DM/jWznXlBh9my6Z2yzzfLaDtM45/vNurEWV5BqbostwOk7PTwuy?=
 =?us-ascii?Q?k0YFlozY8TYVDENTtkdn4H0WfWsRuSy7wVjAGqU899KOCg1BSBALb2dqCmgv?=
 =?us-ascii?Q?DUl0P8s10sTy6q6Zx271gnTzoigoG9VDwpB3IHB/eyxmQ9eqBfVk4CSQX7h8?=
 =?us-ascii?Q?PK4IzijjFJwxQWzO/W367L4CiTltgB8z3ywooQA88fGlAyE/dBBSPzW4gXxD?=
 =?us-ascii?Q?U93SyrqxhZ3DGHGzPbjcqMSmLx74DxjExahwrKApTSxciZ3iJLr1M0rohiyu?=
 =?us-ascii?Q?s4BXoiX2UboBZ4YZF9hNK5LvDXJCgpb4yuEG+hRwwodQm+Z/1AHtkFyb+4a/?=
 =?us-ascii?Q?eqV8fXAMoNOSAe6B2oaFFiSsb+NTMB+5/gbYP+pYmwHhxS2FTz9XGbxuzpqg?=
 =?us-ascii?Q?rH+P++H1u+T0qihPJCAB1SPABX33adVfRMLE7d2UeLUsEYwZEM4LwFtwTwvO?=
 =?us-ascii?Q?F2n3gFfkX70NDF+k2BZR/NrDzRwvS02Dsnxj3NS53ngG393VUmq1DVDbqGMc?=
 =?us-ascii?Q?wzKwEUufC8uhgOt+nsn3rUMS90+OSNnWqvCz9ppP8JtCGFSnZINXK2s5ro3z?=
 =?us-ascii?Q?IwHskwOEOLMRjDkRsGfHxBlScmo2BTF0wr29cA2j0C8RKMgrpxTvWJ6Ou4Ur?=
 =?us-ascii?Q?Z6Mtl6+nFK/6Zy0/1Isymwqmj/H3kg3HEINWX8x75P4m8MRq3FkcgYcoRzEq?=
 =?us-ascii?Q?CtUeCuiwQ4xVwh5DrMEGwJMjLFh4?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(35042699013)(376005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 13:23:00.3780
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e4c6108-b476-4edd-a8ac-08dc80aba0bc
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM1PEPF000252DB.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7864

Hi Julien,

> On 30 May 2024, at 12:35, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Bertrand,
>=20
> On 30/05/2024 10:40, Bertrand Marquis wrote:
>>> But we are making assumption that all TEE implementation will have its
>>> node inside "/firmware/". I am not 100% sure that this is correct. For
>>> example I saw that Google Trusty uses "/trusty" node (directly inside
>>> the DTS root). On other hand, it is not defined in dts bindings, as far
>>> as I know.
>> Regarding the firmware part you can easily handle that by looking for /f=
irmware
>> and create it if it does not exist before creating your sub-node and thi=
s should
>> be node in the optee node creation function not in tee.c.
>=20
> This would work if the node /firmware doesn't exist. But how would you ha=
ndle the case where it is already present?
>=20
> I looked at the libfdt API and AFAICT the DTB creation needs to be linear=
. IOW, you can't add a subnode to an already created node.
>=20
> There is an helper to create a placeholder, but AFAIK this is only for a =
property. You also need to know the size in advance.

I thought it was possible but i definitely can be wrong.

As right now we have only one need for the node, we could delay a possible =
solution and just create it in the optee driver.
Designing a solution for a possible future case right now seems a bit compl=
ex without a use case.

Cheers
Bertrand

>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Thu May 30 13:48:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 13:48:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732735.1138772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCg8i-0003MP-L2; Thu, 30 May 2024 13:48:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732735.1138772; Thu, 30 May 2024 13:48:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCg8i-0003MI-Hx; Thu, 30 May 2024 13:48:20 +0000
Received: by outflank-mailman (input) for mailman id 732735;
 Thu, 30 May 2024 13:48:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=za+F=NB=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1sCg8h-0003MC-Et
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 13:48:19 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 449adc4f-1e8b-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 15:48:17 +0200 (CEST)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2e72b8931caso9268351fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 06:48:17 -0700 (PDT)
Received: from [10.80.5.21] ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-579d65934ddsm5250799a12.38.2024.05.30.06.48.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 06:48:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 449adc4f-1e8b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1717076897; x=1717681697; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=K5U4QsW9eP5bbrjen53u+l8DlvxSzZvlDhAIrTvwM3Q=;
        b=V3JORDLJ6DTAlGrpz/pddCFSiJHP6IvOdFQSeKTvk9zPwb8P0E2gexazaMqp1+s346
         PBOSWIrC5pHOZOhSfRkYF4SUt/a6g+begbqH+uviFvib13Dgw43RG2jtYZn9++TEVesu
         TruL89fW0ao8iydnYPzFe/+pTqg9DH9dp3jlk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717076897; x=1717681697;
        h=content-transfer-encoding:in-reply-to:from:content-language
         :references:cc:to:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=K5U4QsW9eP5bbrjen53u+l8DlvxSzZvlDhAIrTvwM3Q=;
        b=cOVlhbXfUMkkt6d8x6Y9jUuGuDUrkJA7rSLYpOjbRipUZjx8xeuwxAeacUJa0scaR8
         Nr0NRYNJqg6REzMslD0uWspmvddq0RzrRr0poOCp9N5AFToKrH/QnzOHTUxcx+6MgG5g
         Hooib7I4+9Bezm740FAqUBWRAOjZlrlS1MvAcr5SzVMsYXwd/0t3d3dvMConpj5eNifi
         /bcQgxk2Lr9Q/b09xmKSqcICG7jcdA/7OWRRoS6Tnxj96xNtJkKG0BheNoCgLNFqlsVy
         jadmHEH8ajU1MA61ZEWzwN+Eam0sy91WL1Fb0WIDfWMXS+QwwE8RpdhvjUTi4AKPXj3O
         L65A==
X-Forwarded-Encrypted: i=1; AJvYcCX8T7JORmuNo6eOk0pxpPU6byDribKhvyiKqaKGc7qk30jKnfArniJ2YUa8WEQR2zCUamYe1VlEKEaxn4k5jFb0nK01OgI9opIQuaZkgpo=
X-Gm-Message-State: AOJu0Yw60eU4IZtOPWK9hV9hncWUjsDOmyX8HoimDn7Q5vIEfGD4aOu5
	Sd/quKK24+jlwm93dIxZsRpt0oG3c2pBEhS5/Rb9IvH1IaB3e23tjuevjiVM1io=
X-Google-Smtp-Source: AGHT+IHyxnbdapBD3o3qDyHk96NDfdqHc30OIhVvFlOaBfN2C6DCWzPG825yuBpYNV0FBpMt/jIR1A==
X-Received: by 2002:a2e:8096:0:b0:2ea:7e50:6c94 with SMTP id 38308e7fff4ca-2ea8479e99amr10751361fa.16.1717076896645;
        Thu, 30 May 2024 06:48:16 -0700 (PDT)
Message-ID: <e6b9fc65-365b-4a38-8181-a76d576939e4@cloud.com>
Date: Thu, 30 May 2024 14:48:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
 <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>
Content-Language: en-GB
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
In-Reply-To: <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 12:08, Andrew Cooper wrote:
> On 29/05/2024 3:32 pm, Alejandro Vallejo wrote:
>> diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
>> index f033d22785be..b70b22d55fcf 100644
>> --- a/xen/lib/x86/policy.c
>> +++ b/xen/lib/x86/policy.c
>> @@ -2,6 +2,17 @@
>>  
>>  #include <xen/lib/x86/cpu-policy.h>
>>  
>> +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
>> +{
>> +    /*
>> +     * TODO: Derive x2APIC ID from the topology information inside `p`
>> +     *       rather than from the vCPU ID alone. This bodge is a temporary
>> +     *       measure until all infra is in place to retrieve or derive the
>> +     *       initial x2APIC ID from migrated domains.
>> +     */
>> +    return id * 2;
>> +}
>> +
> 
> I'm afraid it's nonsensical to try and derive x2APIC ID from a
> policy+vcpu_id.

That's debatable, and we clearly have different views, however...

> 
> Take a step back, and think the data through.
> 
> A VM has:
> * A unique APIC_ID for each vCPU
> * Info in CPUID describing how to decompose the APIC_ID into topology
> 
> Right now, because this is all completely broken, we have:
> * Hardcoded APIC_ID = vCPU_ID * 2
> * Total nonsense in CPUID
> 
> 
> When constructing a VM, the toolstack (given suitable admin
> guidance/defaults) *must* choose both:
> * The APIC_ID themselves
> * The CPUID topo data to match
> 
> i.e. this series should be editing the toolstack's call to
> xc_domain_hvm_setcontext().
> 
> It's not, because AFAICT you're depending on the migration compatibility
> logic and inserting a new hardcoded assumption about symmetry of the layout.
> 
> 
> The data flows we need are:
> 
> (New) create:
> * Toolstack chooses both parts of topo information
> * Xen needs a default, which reasonably can be APIC_ID=vCPU_ID when the
> rest of the data flow has been cleaned up.  But this is needs to be
> explicit in vcpu_create() and without reference to the policy.
> 
> And to be clear, it's fine for now for the toolstack to choose a
> symmetric layout and pick appropriate APIC_IDs+CPUID for this, but it
> needs to be the toolstack making this decision, not Xen inventing state
> out of thin air based on the toolstack only giving half the information.
> 
> (New) migrate:
> * Data from the stream, exactly as presented
> 
> (Compat) migrate:
> * Synthesize the missing xapic_id field in LAPIC_REGs as APIC_ID=vCPU_ID
> * 2.
> 
> I'm pretty sure this will be a net reduction in complexity in this
> series.  It definitely reduces the Xen complexity.
> 
> ~Andrew

... I didn't know toolstack could send hvmcontexts during non-migrated
domain creation. That's neat!

I was going to defend my approach (because it does make sense), but
there's an extra benefit from yours you didn't seem to notice. With the
x2apicid in the migration stream (patches 1 and parts of 2) it's not
only possible to set the APIC ID from toolstack per vCPU with the
contexts, but it would also allow toolstack to be responsible to
preinitialize all the APICs in x2apic mode when any of them is 255 or more.

I'll try to do that soon-ish. I suspect the pain points are going to be
making it work nicely as well on 1vCPU systems with no APIC (are those
expected to work?).

I'm not looking forward to re-testing all of this again...

Cheers,
Alejandro


From xen-devel-bounces@lists.xenproject.org Thu May 30 14:22:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 14:22:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732749.1138782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgg2-0000QL-5m; Thu, 30 May 2024 14:22:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732749.1138782; Thu, 30 May 2024 14:22:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgg2-0000QE-2m; Thu, 30 May 2024 14:22:46 +0000
Received: by outflank-mailman (input) for mailman id 732749;
 Thu, 30 May 2024 14:22:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgg0-0000Q4-Pg; Thu, 30 May 2024 14:22:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgg0-0000UQ-Mo; Thu, 30 May 2024 14:22:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgg0-0002CD-Dr; Thu, 30 May 2024 14:22:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgg0-0000E8-DL; Thu, 30 May 2024 14:22:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tEcqv9bbN5aufUk6pWsqWUBBHGjpbnzGp7px5quZ9E8=; b=1IAvLfLiE8JKhm2w+SwHHt01i8
	tnhl7zQwntfAhk1NnIe5345QVVyGtPHvxJWcC2PmfypXSQC06uvU+1cel8ksMCYArIFRixp/n1NUt
	8suF3SpnaRDd+8R/x/N1eyRj+D7fcZobvsL3XUOZyNSW0AD0B2/28RrD2IrgrSUVofrA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186198-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186198: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=9518d77eb869034a141799b3d28cac20ecb60fe0
X-Osstest-Versions-That:
    ovmf=c695e3182aa7497833f1b0fc69f6776fec8cb8cf
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 14:22:44 +0000

flight 186198 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186198/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 9518d77eb869034a141799b3d28cac20ecb60fe0
baseline version:
 ovmf                 c695e3182aa7497833f1b0fc69f6776fec8cb8cf

Last test of basis   186195  2024-05-30 09:41:11 Z    0 days
Testing same since   186198  2024-05-30 12:41:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Du Lin <du.lin@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   c695e3182a..9518d77eb8  9518d77eb869034a141799b3d28cac20ecb60fe0 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 14:43:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 14:43:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732765.1138792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgzs-00039G-PM; Thu, 30 May 2024 14:43:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732765.1138792; Thu, 30 May 2024 14:43:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgzs-000399-Mj; Thu, 30 May 2024 14:43:16 +0000
Received: by outflank-mailman (input) for mailman id 732765;
 Thu, 30 May 2024 14:43:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCgzr-000393-OI
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 14:43:15 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f1f79677-1e92-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 16:43:14 +0200 (CEST)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-52b119986f2so1129097e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 07:43:14 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-5297066b89bsm1563053e87.175.2024.05.30.07.43.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 07:43:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1f79677-1e92-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717080194; x=1717684994; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=tZ+9rqcEtwFOPm0ZOsQRoFuxkJ0lnj3tyqVGcJWh0IQ=;
        b=LZX86YSYiteeSUUrMOblxr5mORfg4WKtaxu3XSObWr8uS0lXO5N7r6OVGYhx81tOv+
         1smzfn8YXli1IWCx/bBJPz9pb3JlD8RnCIQaE7KPZR6DTPbmIzexeca9hVg3jSNLLjUP
         yOCaR/NhITmElxj9YL+7eWC3uL5h259t4d6qBHgaenKlpwixvEdvX68O70cV3uzo/fOE
         O8AxONDU1RNW4uDrDhf+wPsB+4X4pCxd5Flh/15kE2cspt1lc1Excgc8WrJtfMREpHxr
         N046lpUvdVmC/Y9yeDjv+9GqCn1JtQb4fMAAWcfAH+4UM46QvdlB/yetPjLhg032dIGb
         DV1Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717080194; x=1717684994;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tZ+9rqcEtwFOPm0ZOsQRoFuxkJ0lnj3tyqVGcJWh0IQ=;
        b=CBwkmvq9Ga0mqu+k2awiT98y2u6WveqvsjYXWbsGxQnakeKGBIsO/NJBe5WVNETEyB
         ovPFTVEiajSOS2DtFnq3UJaHCHWKPdfTqb29ahsK2LK/imFFb7fQWmVyIKm0G/i33WFr
         tNsHhEMfmhluSftY7f8in7QAjeAJqoKwEKI2YG39gffnEKHxixyiuysJc50q7YN8VSkC
         i+pwo+6NTDyt7wj7ZtDq9CkpyPrK3TU3aPAiQgBPqOKEqbr3yMu7uEsb3THP+dgsTDym
         RiRh0qwGQ0eug9kpdDIy2sZJ2EXUyp2nhLUKy2nIHaqT1V5awEEYYEPod3VhHMyS6fHG
         neIQ==
X-Forwarded-Encrypted: i=1; AJvYcCVMWoFru0boYV1TRk7r5qqfWp3dt4dxIdQrB7bRALYxKJM0i66X2NAnBu7Bp+dRzWLC2rjZD9ggcKQxjqbv20KboE2bP98IbYV84vvBNeU=
X-Gm-Message-State: AOJu0Yzzw1lxVPcCArFF4GLWHOr0O6rtC7s3gjsECElJQzvvRKYYKOUD
	mru9Pakvhxhu4EuGwW04a1XrFkLsjjt0/5wWt0ZW4F8STJSjCv3C
X-Google-Smtp-Source: AGHT+IFgC7WcTwoibTB69tnsOMxQGi4DOm+Yj58Ag7dpTkqVRAaja+qwf+VoVYWgjlHfv4n16irGwQ==
X-Received: by 2002:a19:ac08:0:b0:51c:a0e1:2a44 with SMTP id 2adb3069b0e04-52b7d427180mr1726513e87.26.1717080193641;
        Thu, 30 May 2024 07:43:13 -0700 (PDT)
Message-ID: <7f25c5a034083b88547398ff94fe23ec92ca2779.camel@gmail.com>
Subject: Re: [PATCH] tools: (Actually) drop libsystemd as a dependency
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross
 <jgross@suse.com>,  George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>
Date: Thu, 30 May 2024 16:43:12 +0200
In-Reply-To: <20240530101439.16730-1-andrew.cooper3@citrix.com>
References: <20240530101439.16730-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 11:14 +0100, Andrew Cooper wrote:
> When reinstating some of systemd.m4 between v1 and v2, I reintroduced
> a little
> too much.=C2=A0 While {c,o}xenstored are indeed no longer linked against
> libsystemd, ./configure still looks for it.
>=20
> Drop this too.
>=20
> Fixes: ae26101f6bfc ("tools: Drop libsystemd as a dependency")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
> ---
> CC: Anthony PERARD <anthony@xenproject.org>
> CC: Juergen Gross <jgross@suse.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>=20
> Found when trying to build Xen in XenServer with libsystemd absent
> from the
> chroot.
> ---
> =C2=A0m4/systemd.m4=C2=A0=C2=A0 |=C2=A0=C2=A0 8 --
> =C2=A0tools/configure | 229 +--------------------------------------------=
-
> --
> =C2=A02 files changed, 1 insertion(+), 236 deletions(-)
>=20
> diff --git a/m4/systemd.m4 b/m4/systemd.m4
> index e4fe51a8ba..ab12ea313d 100644
> --- a/m4/systemd.m4
> +++ b/m4/systemd.m4
> @@ -86,13 +86,6 @@ AC_DEFUN([AX_CHECK_SYSTEMD], [
> =C2=A0	],[systemd=3Dn])
> =C2=A0])
> =C2=A0
> -AC_DEFUN([AX_CHECK_SYSTEMD_ENABLE_AVAILABLE], [
> -	PKG_CHECK_MODULES([SYSTEMD], [libsystemd-daemon],
> [systemd=3D"y"],[
> -		PKG_CHECK_MODULES([SYSTEMD], [libsystemd >=3D 209],
> -				=C2=A0 [systemd=3D"y"],[systemd=3D"n"])
> -	])
> -])
> -
> =C2=A0dnl Enables systemd by default and requires a --disable-systemd
> option flag
> =C2=A0dnl to configure if you want to disable.
> =C2=A0AC_DEFUN([AX_ENABLE_SYSTEMD], [
> @@ -112,6 +105,5 @@ dnl to have systemd build libraries it will be
> enabled. You can always force
> =C2=A0dnl disable with --disable-systemd
> =C2=A0AC_DEFUN([AX_AVAILABLE_SYSTEMD], [
> =C2=A0	AX_ALLOW_SYSTEMD_OPTS()
> -	AX_CHECK_SYSTEMD_ENABLE_AVAILABLE()
> =C2=A0	AX_CHECK_SYSTEMD()
> =C2=A0])
> diff --git a/tools/configure b/tools/configure
> index b8faa1d520..459bfb5652 100755
> --- a/tools/configure
> +++ b/tools/configure
> @@ -626,8 +626,6 @@ ac_subst_vars=3D'LTLIBOBJS
> =C2=A0LIBOBJS
> =C2=A0pvshim
> =C2=A0ninepfs
> -SYSTEMD_LIBS
> -SYSTEMD_CFLAGS
> =C2=A0SYSTEMD_MODULES_LOAD
> =C2=A0SYSTEMD_DIR
> =C2=A0systemd
> @@ -864,9 +862,7 @@ pixman_LIBS
> =C2=A0libzstd_CFLAGS
> =C2=A0libzstd_LIBS
> =C2=A0LIBNL3_CFLAGS
> -LIBNL3_LIBS
> -SYSTEMD_CFLAGS
> -SYSTEMD_LIBS'
> +LIBNL3_LIBS'
> =C2=A0
> =C2=A0
> =C2=A0# Initialize some variables set by options.
> @@ -1621,10 +1617,6 @@ Some influential environment variables:
> =C2=A0=C2=A0 LIBNL3_CFLAGS
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 C compiler flags for LIBNL3, overriding pkg-config
> =C2=A0=C2=A0 LIBNL3_LIBS linker flags for LIBNL3, overriding pkg-config
> -=C2=A0 SYSTEMD_CFLAGS
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 C compiler flags for SYSTEMD, overriding pkg-config
> -=C2=A0 SYSTEMD_LIBS
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 linker flags for SYSTEMD, overriding pkg-config
> =C2=A0
> =C2=A0Use these variables to override the choices made by `configure' or
> to help
> =C2=A0it to find libraries and programs with nonstandard names/locations.
> @@ -3889,8 +3881,6 @@ esac
> =C2=A0
> =C2=A0
> =C2=A0
> -
> -
> =C2=A0
> =C2=A0
> =C2=A0
> @@ -9540,223 +9530,6 @@ fi
> =C2=A0
> =C2=A0
> =C2=A0
> -
> -pkg_failed=3Dno
> -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
> -$as_echo_n "checking for SYSTEMD... " >&6; }
> -
> -if test -n "$SYSTEMD_CFLAGS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D"$SYSTEMD_CFLAGS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd-daemon\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D`$PKG_CONFIG --cflags "libsystemd-daemon"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -if test -n "$SYSTEMD_LIBS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_LIBS=3D"$SYSTEMD_LIBS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd-daemon\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd-daemon") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_LIBS=3D`$PKG_CONFIG --libs "libsystemd-daemon"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -
> -
> -
> -if test $pkg_failed =3D yes; then
> -=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" =
>&5
> -$as_echo "no" >&6; }
> -
> -if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dyes
> -else
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dno
> -fi
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if test $_pkg_short_errors_su=
pported =3D yes; then
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --short-errors --
> print-errors --cflags --libs "libsystemd-daemon" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --print-errors --
> cflags --libs "libsystemd-daemon" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fi
> -	# Put the nasty error message in config.log where it belongs
> -	echo "$SYSTEMD_PKG_ERRORS" >&5
> -
> -
> -
> -pkg_failed=3Dno
> -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
> -$as_echo_n "checking for SYSTEMD... " >&6; }
> -
> -if test -n "$SYSTEMD_CFLAGS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D"$SYSTEMD_CFLAGS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd >=3D 209\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd >=3D 209") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D`$PKG_CONFIG --cflags "libsystemd >=3D 20=
9"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -if test -n "$SYSTEMD_LIBS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_LIBS=3D"$SYSTEMD_LIBS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd >=3D 209\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd >=3D 209") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_LIBS=3D`$PKG_CONFIG --libs "libsystemd >=3D 209"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -
> -
> -
> -if test $pkg_failed =3D yes; then
> -=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" =
>&5
> -$as_echo "no" >&6; }
> -
> -if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dyes
> -else
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dno
> -fi
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if test $_pkg_short_errors_su=
pported =3D yes; then
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --short-errors --
> print-errors --cflags --libs "libsystemd >=3D 209" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --print-errors --
> cflags --libs "libsystemd >=3D 209" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fi
> -	# Put the nasty error message in config.log where it belongs
> -	echo "$SYSTEMD_PKG_ERRORS" >&5
> -
> -	systemd=3D"n"
> -elif test $pkg_failed =3D untried; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: =
result: no" >&5
> -$as_echo "no" >&6; }
> -	systemd=3D"n"
> -else
> -	SYSTEMD_CFLAGS=3D$pkg_cv_SYSTEMD_CFLAGS
> -	SYSTEMD_LIBS=3D$pkg_cv_SYSTEMD_LIBS
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { $as_echo "$as_me:${as_linen=
o-$LINENO}: result: yes" >&5
> -$as_echo "yes" >&6; }
> -	systemd=3D"y"
> -fi
> -
> -elif test $pkg_failed =3D untried; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: =
result: no" >&5
> -$as_echo "no" >&6; }
> -
> -
> -pkg_failed=3Dno
> -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for SYSTEMD" >&5
> -$as_echo_n "checking for SYSTEMD... " >&6; }
> -
> -if test -n "$SYSTEMD_CFLAGS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D"$SYSTEMD_CFLAGS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd >=3D 209\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd >=3D 209") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_CFLAGS=3D`$PKG_CONFIG --cflags "libsystemd >=3D 20=
9"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -if test -n "$SYSTEMD_LIBS"; then
> -=C2=A0=C2=A0=C2=A0 pkg_cv_SYSTEMD_LIBS=3D"$SYSTEMD_LIBS"
> - elif test -n "$PKG_CONFIG"; then
> -=C2=A0=C2=A0=C2=A0 if test -n "$PKG_CONFIG" && \
> -=C2=A0=C2=A0=C2=A0 { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONF=
IG --exists
> --print-errors \"libsystemd >=3D 209\""; } >&5
> -=C2=A0 ($PKG_CONFIG --exists --print-errors "libsystemd >=3D 209") 2>&5
> -=C2=A0 ac_status=3D$?
> -=C2=A0 $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5
> -=C2=A0 test $ac_status =3D 0; }; then
> -=C2=A0 pkg_cv_SYSTEMD_LIBS=3D`$PKG_CONFIG --libs "libsystemd >=3D 209"
> 2>/dev/null`
> -		=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 test "x$?" !=3D "x0" && pkg_failed=3Dye=
s
> -else
> -=C2=A0 pkg_failed=3Dyes
> -fi
> - else
> -=C2=A0=C2=A0=C2=A0 pkg_failed=3Duntried
> -fi
> -
> -
> -
> -if test $pkg_failed =3D yes; then
> -=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" =
>&5
> -$as_echo "no" >&6; }
> -
> -if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dyes
> -else
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _pkg_short_errors_supported=
=3Dno
> -fi
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if test $_pkg_short_errors_su=
pported =3D yes; then
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --short-errors --
> print-errors --cflags --libs "libsystemd >=3D 209" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else
> -	=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSTEMD_PKG_ERRORS=3D`$PKG_C=
ONFIG --print-errors --
> cflags --libs "libsystemd >=3D 209" 2>&1`
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fi
> -	# Put the nasty error message in config.log where it belongs
> -	echo "$SYSTEMD_PKG_ERRORS" >&5
> -
> -	systemd=3D"n"
> -elif test $pkg_failed =3D untried; then
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0	{ $as_echo "$as_me:${as_lineno-$LINENO}: =
result: no" >&5
> -$as_echo "no" >&6; }
> -	systemd=3D"n"
> -else
> -	SYSTEMD_CFLAGS=3D$pkg_cv_SYSTEMD_CFLAGS
> -	SYSTEMD_LIBS=3D$pkg_cv_SYSTEMD_LIBS
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { $as_echo "$as_me:${as_linen=
o-$LINENO}: result: yes" >&5
> -$as_echo "yes" >&6; }
> -	systemd=3D"y"
> -fi
> -
> -else
> -	SYSTEMD_CFLAGS=3D$pkg_cv_SYSTEMD_CFLAGS
> -	SYSTEMD_LIBS=3D$pkg_cv_SYSTEMD_LIBS
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { $as_echo "$as_me:${as_linen=
o-$LINENO}: result: yes" >&5
> -$as_echo "yes" >&6; }
> -	systemd=3D"y"
> -fi
> -
> -
> =C2=A0		if test "x$enable_systemd" !=3D "xno"; then :
> =C2=A0
> =C2=A0	=C2=A0=C2=A0=C2=A0=C2=A0 if test "x$systemd" =3D "xy" ; then :



From xen-devel-bounces@lists.xenproject.org Thu May 30 14:43:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 14:43:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732766.1138802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgzx-0003OS-4e; Thu, 30 May 2024 14:43:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732766.1138802; Thu, 30 May 2024 14:43:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCgzx-0003OL-16; Thu, 30 May 2024 14:43:21 +0000
Received: by outflank-mailman (input) for mailman id 732766;
 Thu, 30 May 2024 14:43:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgzv-0003Nm-F4; Thu, 30 May 2024 14:43:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgzv-0000pB-Dy; Thu, 30 May 2024 14:43:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgzv-0002sF-0J; Thu, 30 May 2024 14:43:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCgzu-0000z2-W3; Thu, 30 May 2024 14:43:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KXtadldSmf7F9D42XP5p/tzc+cqMvWqQ+DtweProt8o=; b=bv2sfCRGdclz7aW26rBcZnsOWX
	U/loFHQDFMzxApm3ZDqUcPHE5REa1Fit54qxTysGLSsKy12onWvZtTAw1WBO87rJeXaUy+ab4d49x
	hETd6eAiW+uN+SSIfNtmbJ2ZtRV+MT2hvO3gpXKZOUJk0Jlp0Ac5L8O43kOYhhaMHrKo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186190-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186190: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=2ea493598fd5a3efe4f25d842b2f09bc5c74c6c1
X-Osstest-Versions-That:
    libvirt=38c6c364088e3ba2d7e0b18690af6800d338c6a9
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 14:43:18 +0000

flight 186190 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186190/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186179
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              2ea493598fd5a3efe4f25d842b2f09bc5c74c6c1
baseline version:
 libvirt              38c6c364088e3ba2d7e0b18690af6800d338c6a9

Last test of basis   186179  2024-05-29 04:18:48 Z    1 days
Testing same since   186190  2024-05-30 04:22:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Pavel Hrdina <phrdina@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   38c6c36408..2ea493598f  2ea493598fd5a3efe4f25d842b2f09bc5c74c6c1 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 14:49:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 14:49:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732782.1138812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCh5M-0004Lf-MP; Thu, 30 May 2024 14:48:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732782.1138812; Thu, 30 May 2024 14:48:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCh5M-0004LY-Jh; Thu, 30 May 2024 14:48:56 +0000
Received: by outflank-mailman (input) for mailman id 732782;
 Thu, 30 May 2024 14:48:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCh5K-0004LS-KA
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 14:48:54 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bc05ecd6-1e93-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 16:48:53 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-52ad8230bb9so1092241e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 07:48:53 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b7d42911fsm243645e87.281.2024.05.30.07.48.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 07:48:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc05ecd6-1e93-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717080533; x=1717685333; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=/aH8GhKdYYOAQ9pcnE2jsIowDMws/vgnHyfVWQgneqM=;
        b=M/JHjPzUK5bfRnBYSWKNu8Xsh8xwl0aliEdml3fztqzFmKTlGgI5EtC7k2TerrgOQh
         gVLp1SezHDF9uWkD2h1LoGxXzet7xa2CxhjBFZ8D7ZZqMBlobVmz/talbAGKYrnJyBsn
         T3EVI2jkVrcG4jIZds0roUZ10X+DfA3wspQrN2/OnZCIUvnccLyElqn0dihw0+RJSl9X
         ajt3SF8nVmWKz8hRGnsqSlLw7L5tZEsBjwQyBaF+FPXflNqCU7fYuPV1cQTjDxtUkIc5
         O2fXpu172Lvj3ChYduf0RFVP0Op9ijV5pVSWGkR4LIjjPZssvyVaNQCUf7i/4IDZsI1P
         On9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717080533; x=1717685333;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/aH8GhKdYYOAQ9pcnE2jsIowDMws/vgnHyfVWQgneqM=;
        b=EsNg2evrjPnbsaKCAgFDRZ+sDZc1VTW42V3wKG022oGeagfdt7ZfH5aqPYObT1b1AR
         fDIhArlcuBVtHCe7iAme7MCyGlzG6rjyt2JB/mfmJzRiA2CYd/rIHakeK+LpmQ4SKDss
         JaLH70ykoeR1F59XXg45Ks/5mAOesP4x9SDs/0q77fugjYZc8dDz/Dt+k2ZP5CnbIJo1
         zxHQSOPcafIGRYalES6NyB5Vy+Cj3TlFBBUIwnahNIna4PDTEPjXk3fhfL8XwXpD+hBd
         S86r1tWL4VS4O5is3TVEkGLPzJt7UM+f3XIMyRPKPtPkpkzcw2N/ZbYRuUTcKK93ND+Z
         ss8A==
X-Forwarded-Encrypted: i=1; AJvYcCVKaZha1oySIXu1As5HR3lyxsvUDirG48n2klOknlPyxPppvCyunfGxXmjQ/6mrVFL4RFEj2/x3v47ar2+m49U3z1C8k6dptOv4wwYWU+o=
X-Gm-Message-State: AOJu0YxPohQ8BbtuY3u1ZYT14ho9GrLhNhhornhOMVfvD/QXBMfyr/2Q
	DI8pHOnuUEk/Srmh4O6sk6DZ595xWVvznoinKNlAeTJ4Xw/gIXRy
X-Google-Smtp-Source: AGHT+IGTwvIJwrmRudii8xHUhVchPDfTGVMguqLBG2sasuFKwsn9a3grBAP9ICOAk+Vg6RP3hZhfbA==
X-Received: by 2002:a05:6512:701:b0:516:d219:3779 with SMTP id 2adb3069b0e04-52b7d48922bmr1449038e87.58.1717080532696;
        Thu, 30 May 2024 07:48:52 -0700 (PDT)
Message-ID: <ce7409498d3e8620e873d854caf3c61828e34d15.camel@gmail.com>
Subject: Re: [PATCH v4 0/2] Clean the policy manipulation path in domain
 creation
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Alejandro Vallejo
	 <alejandro.vallejo@cloud.com>, Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross
 <jgross@suse.com>,  Jan Beulich <jbeulich@suse.com>, Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Date: Thu, 30 May 2024 16:48:51 +0200
In-Reply-To: <19e240ca-6606-482c-9c0b-3c0a6c2c4e89@citrix.com>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
	 <19e240ca-6606-482c-9c0b-3c0a6c2c4e89@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 10:44 +0100, Andrew Cooper wrote:
> On 29/05/2024 3:30 pm, Alejandro Vallejo wrote:
> > Alejandro Vallejo (2):
> > =C2=A0 tools/xg: Streamline cpu policy serialise/deserialise calls
> > =C2=A0 tools/xg: Clean up xend-style overrides for CPU policies
>=20
> Oleksii: Please consider for 4.19.
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
>=20
> This is internal clean-up to CPUID handling which has been trying
> (one
> way or another) to land for more than 3 years now.
>=20
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Thu May 30 15:48:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 15:48:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732807.1138822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCi0c-0003LD-Gz; Thu, 30 May 2024 15:48:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732807.1138822; Thu, 30 May 2024 15:48:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCi0c-0003L6-DG; Thu, 30 May 2024 15:48:06 +0000
Received: by outflank-mailman (input) for mailman id 732807;
 Thu, 30 May 2024 15:48:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MghO=NB=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCi0b-0003L0-1J
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 15:48:05 +0000
Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com
 [2607:f8b0:4864:20::f2f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id feaa4aed-1e9b-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 17:48:02 +0200 (CEST)
Received: by mail-qv1-xf2f.google.com with SMTP id
 6a1803df08f44-6ae0adfba9cso5624466d6.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 08:48:02 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ae3747dabfsm1590926d6.80.2024.05.30.08.47.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 08:48:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: feaa4aed-1e9b-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717084081; x=1717688881; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=oxqf3xs1970Nf5zEjfxi/reh434J2MVMUjSk7tO9XVk=;
        b=grlr1wTWe6rXInwLhhL9i7Q0zAafTYM5pvnYWJf+nglTOMtGQm3i0SAGwmERpyyDqD
         23wNBCIsmPIEcIwXbDl9wnbtKfccvK0UApKV9zzgxp5TaOaCEunFMTZ0Qw3AwFZTWCQM
         rN4IVMzPplfqTK231Th/O9fhjVaBiZTKSOgdg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717084081; x=1717688881;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=oxqf3xs1970Nf5zEjfxi/reh434J2MVMUjSk7tO9XVk=;
        b=pvn3seU9MqS91t/rT1D7pnZmQAk1AIGrNQ2zNIjPfh+nyg9LKDn+ZSOThHjhUgLfew
         j5lASdJRG8d0d2siSfKw7fcD5/aoYDpDg/8bE0/J+1zMpkChOKnO0EdC7PybrJzZgrU7
         gZedbkMt+7xVoCHxdWvZcCJN3UnYg2k2trhrEUE6vnGNTOlCYirjBWY8zp4NCr1/hmtM
         B3sdBYqcr/IsIdRWBXWoUxuYi3M64jOc9l7OyacLUhUyKGN6TXPxudXQa1kEZZAiXzAL
         E8wCvkr3qFF4rMkNc/ChspQa1pjfiq5Ddfl2eBTwCCQcQjgPXuj644k/GmgYBjfyB02W
         qPyw==
X-Forwarded-Encrypted: i=1; AJvYcCWx0YeOI8+HjMWmTEpc8e7P0tF2bLN36ZyTc42ZDdt0IzaqjZYbehRLYyeD59HfKFAGz3hxIkPJUVPHXMEq0lUtENCYPjerGedMo1K1bx4=
X-Gm-Message-State: AOJu0YyIaH7VQFwqf6Hm6plVZhGUEEWNq5eSOe3+OzYk9iDUxh7x3cYO
	9/hfTzDPN3+KsJZOrkWPA66RTnIwFQdHo3YMx7eZwUvzi7GHZr/VABbGsqTSwRw=
X-Google-Smtp-Source: AGHT+IHE9QQL0YCf+/J5BYCzknuSYnGYXbQ6CM3je54Kz7kkHtTY+j+Jg+sXGzp1GTOoWYKJVt35Eg==
X-Received: by 2002:a05:6214:3c8a:b0:6ae:20:3723 with SMTP id 6a1803df08f44-6ae0cafc262mr30183836d6.5.1717084080634;
        Thu, 30 May 2024 08:48:00 -0700 (PDT)
Date: Thu, 30 May 2024 17:47:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
Message-ID: <Zlifrqp5NfKu8xTw@macbook>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
 <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>
 <e6b9fc65-365b-4a38-8181-a76d576939e4@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <e6b9fc65-365b-4a38-8181-a76d576939e4@cloud.com>

On Thu, May 30, 2024 at 02:48:10PM +0100, Alejandro Vallejo wrote:
> I'll try to do that soon-ish. I suspect the pain points are going to be
> making it work nicely as well on 1vCPU systems with no APIC (are
> those expected to work?).

We do not allow creation of PVH/HVM domains without an emulated local
APIC, and I don't think we ever want to allow doing so (see
emulation_flags_ok()).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu May 30 15:51:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 15:51:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732812.1138831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCi3X-0004y6-Ss; Thu, 30 May 2024 15:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732812.1138831; Thu, 30 May 2024 15:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCi3X-0004xz-Q9; Thu, 30 May 2024 15:51:07 +0000
Received: by outflank-mailman (input) for mailman id 732812;
 Thu, 30 May 2024 15:51:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RCh7=NB=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCi3W-0004xt-PM
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 15:51:06 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ab6d9e1-1e9c-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 17:51:02 +0200 (CEST)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2ea82db9a1fso12117171fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 08:51:02 -0700 (PDT)
Received: from ?IPV6:2003:ca:b724:491c:251a:48e:75b1:a320?
 (p200300cab724491c251a048e75b1a320.dip0.t-ipconnect.de.
 [2003:ca:b724:491c:251a:48e:75b1:a320])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-578524bb86bsm9603759a12.97.2024.05.30.08.51.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 08:51:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ab6d9e1-1e9c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717084262; x=1717689062; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=M0mOBzAZp90VOVLlxifP8OSrPoz+pnEc2VAGr0EWOe0=;
        b=GuI+3bw7QlFyCP3BJJIp7fZ8lc3kAc5Z3CprMSCrh0J9ave0cPOSBDng+E2rA6IIkq
         EeZYJggjRO04bvZ3U45ZNrxfT3+fogtkXmP8KLGa7ms4FEIvIu3P7sDzpoTc4RuJwvJU
         cHa7P3nEFeloMBacj61BDpSUoPdAF6L+6Hcp7eFKAfMSl6jXzt594beYY5FU2+H8Y5YI
         zzz1QNa0pZmecgNGCofLBNNICiFwmVVwds5KhFWhdkvjaGUUZFKXnvhfmhsmH7OdDRRN
         ZcKtOqa17FDMIvI6Ndo4OwM2aS5G1zNSASuvVLEUCh2YsQcJQKQsRXNis1WxXyxWvdxU
         JLqg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717084262; x=1717689062;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=M0mOBzAZp90VOVLlxifP8OSrPoz+pnEc2VAGr0EWOe0=;
        b=ZaP2YZ1Kto4myD7LPTSayhzS4kKd1cXlDnfjGbUTq0wMLH/DBVAOnxPFCfCave4xys
         hbMJA7eL592UFFgde4ZyTJUDLn2ACoensrh7OPNvyC3B7Ggk1Q+YzPqxQoUee140C0Ka
         5H4wsP/37PhMq/R9oK2ZajtxzIs8073RuHyn54IbRbOnJwGjbjVUdrEsLJ3AFAU9ES2c
         pYlfNC2/kTNome5RWZxGCqq+6pis1NmfcC427SofTg+F6QKFhFl0jE1XTipt0nx0+XJ1
         CEpv0GPOeQADJ/V3AcTkYfsCVeNclvWQK5JFAwkPa0/aheHfwTeh20DqcVnFzccSDzmy
         neUA==
X-Forwarded-Encrypted: i=1; AJvYcCWIideQaOJvSC/ZvJRgUCYH+J7iSzMqZumfK1yZnwCJfq63OomtYQFlbRgpia33S1Xk7U8DYLwAQQ8vk4y5pRgoGxcV5IatrCxQZcMiYcA=
X-Gm-Message-State: AOJu0YxoyHmuBOV1nuO8SV70O1H0hL/ju7iKeLQJKnokJQpt1B2ufEnH
	HWm4x/QAlLBA9nKwQXpxZ/WU4JrFchqDE/GE5Y1ZicRW1XRftndfZDDJRcfoGA==
X-Google-Smtp-Source: AGHT+IFZLpzoZIyXErnvVdHQSnEEyhOoh4f9oGCYrW7Xhi5VQzOGDXA9VB3dEVl9g4xK1LxTZfegZQ==
X-Received: by 2002:a2e:be23:0:b0:2ea:8217:214a with SMTP id 38308e7fff4ca-2ea84826aa3mr16438861fa.32.1717084262073;
        Thu, 30 May 2024 08:51:02 -0700 (PDT)
Message-ID: <677e564e-4702-4a37-83df-8d47135b62ff@suse.com>
Date: Thu, 30 May 2024 17:51:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v8 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
 "Huang, Ray" <Ray.Huang@amd.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240516095235.64128-1-Jiqian.Chen@amd.com>
 <20240516095235.64128-6-Jiqian.Chen@amd.com>
 <9652011f-3f24-43f8-b91e-88bd3982a4c4@suse.com>
 <BL1PR12MB5849EB5EE20B1A6C647F5717E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <b10e68e2-3279-471d-a089-c40934050737@suse.com>
 <BL1PR12MB58491A32C32C33545AC71AB7E7EE2@BL1PR12MB5849.namprd12.prod.outlook.com>
 <4b311c82-b252-413a-bb64-0a36aa97680a@suse.com>
 <BL1PR12MB5849333D416160492A7475E2E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <70c86c74-3ed6-4b22-9ba6-3f927f81bcd0@suse.com>
 <BL1PR12MB584922B0352AA2F4A359FD66E7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <7cdff236-bb7d-4dad-9a83-47faaa6dc15f@suse.com>
 <BL1PR12MB58493D3365CC451F36DB554FE7F22@BL1PR12MB5849.namprd12.prod.outlook.com>
 <fbaf7086-85d8-4433-91d9-ef8f74512685@suse.com>
 <BL1PR12MB58494B521CB40BAEA30CB412E7F32@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <BL1PR12MB58494B521CB40BAEA30CB412E7F32@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.05.2024 13:19, Chen, Jiqian wrote:
> On 2024/5/29 20:22, Jan Beulich wrote:
>> On 29.05.2024 13:13, Chen, Jiqian wrote:
>>> On 2024/5/29 15:10, Jan Beulich wrote:
>>>> On 29.05.2024 08:56, Chen, Jiqian wrote:
>>>>> On 2024/5/29 14:31, Jan Beulich wrote:
>>>>>> On 29.05.2024 04:41, Chen, Jiqian wrote:
>>>>>>> But I found in function init_irq_data:
>>>>>>>     for ( irq = 0; irq < nr_irqs_gsi; irq++ )
>>>>>>>     {
>>>>>>>         int rc;
>>>>>>>
>>>>>>>         desc = irq_to_desc(irq);
>>>>>>>         desc->irq = irq;
>>>>>>>
>>>>>>>         rc = init_one_irq_desc(desc);
>>>>>>>         if ( rc )
>>>>>>>             return rc;
>>>>>>>     }
>>>>>>> Does it mean that when irq < nr_irqs_gsi, the gsi and irq is a 1:1 mapping?
>>>>>>
>>>>>> No, as explained before. I also don't see how you would derive that from the code above.
>>>>> Because here set desc->irq = irq, and it seems there is no other place to change this desc->irq, so, gsi 1 is considered to irq 1.
>>>>
>>>> What are you taking this from? The loop bound isn't nr_gsis, and the iteration
>>>> variable isn't in GSI space either; it's in IRQ numbering space. In this loop
>>>> we're merely leveraging that every GSI has a corresponding IRQ;
>>>> there are no assumptions made about the mapping between the two. Afaics at least.
>>>>
>>>>>> "nr_irqs_gsi" describes what its name says: The number of
>>>>>> IRQs mapping to a (_some_) GSI. That's to tell them from the non-GSI (i.e.
>>>>>> mainly MSI) ones. There's no implication whatsoever on the IRQ <-> GSI
>>>>>> mapping.
>>>>>>
>>>>>>> What's more, when using PHYSDEVOP_setup_gsi, it calls mp_register_gsi,
>>>>>>> and in mp_register_gsi, it uses " desc = irq_to_desc(gsi); " to get irq_desc directly.
>>>>>>
>>>>>> Which may be wrong, while that wrong-ness may not have hit anyone in
>>>>>> practice (for reasons that would need working out).
>>>>>>
>>>>>>> Combining above, can we consider "gsi == irq" when irq < nr_irqs_gsi ?
>>>>>>
>>>>>> Again - no.
>>>>> Since you are certain that they are not equal, could you tell me where show they are not equal or where build their mappings,
>>>>> so that I can know how to do a conversion gsi from irq.
>>>>
>>>> I did point you at the ACPI Interrupt Source Override structure before.
>>>> We're parsing those in acpi_parse_int_src_ovr(), to give you a place to
>>>> start going from.
>>> Oh! I think I know.
>>> If I want to transform gsi to irq, I need to do below:
>>> 	int irq, entry, ioapic, pin;
>>>
>>> 	ioapic = mp_find_ioapic(gsi);
>>> 	pin = gsi - mp_ioapic_routing[ioapic].gsi_base;
>>> 	entry = find_irq_entry(ioapic, pin, mp_INT);
>>> 	irq = pin_2_irq(entry, ioapic, pin);
>>>
>>> Am I right?
>>
>> This looks plausible, yes.
> I dump all mpc_config_intsrc of array mp_irqs, it shows:
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 0 dstapic 33 dstirq 2
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 15 srcbus 0 srcbusirq 9 dstapic 33 dstirq 9
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 1 dstapic 33 dstirq 1
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 3 dstapic 33 dstirq 3
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 4 dstapic 33 dstirq 4
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 5 dstapic 33 dstirq 5
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 6 dstapic 33 dstirq 6
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 7 dstapic 33 dstirq 7
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 8 dstapic 33 dstirq 8
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 10 dstapic 33 dstirq 10
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 11 dstapic 33 dstirq 11
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 12 dstapic 33 dstirq 12
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 13 dstapic 33 dstirq 13
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 14 dstapic 33 dstirq 14
> (XEN) find_irq_entry type 3 irqtype 0 irqflag 0 srcbus 0 srcbusirq 15 dstapic 33 dstirq 15
> 
> It seems only Legacy irq and gsi[0:15] has a mapping in mp_irqs.
> Other gsi can be considered 1:1 mapping with irq? Or are there other places reflect the mapping between irq and gsi?

It may be uncommon to have overrides for higher GSIs, but I don't think ACPI
disallows that.

Jan


From xen-devel-bounces@lists.xenproject.org Thu May 30 16:44:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:44:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732828.1138841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCisx-0003lV-MX; Thu, 30 May 2024 16:44:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732828.1138841; Thu, 30 May 2024 16:44:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCisx-0003lO-Jd; Thu, 30 May 2024 16:44:15 +0000
Received: by outflank-mailman (input) for mailman id 732828;
 Thu, 30 May 2024 16:44:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCisw-0003lI-Do
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:44:14 +0000
Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com
 [2607:f8b0:4864:20::72a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8093f9d-1ea3-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 18:44:13 +0200 (CEST)
Received: by mail-qk1-x72a.google.com with SMTP id
 af79cd13be357-794ab20699cso7061785a.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:44:13 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd32207sm565507585a.112.2024.05.30.09.44.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 09:44:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8093f9d-1ea3-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717087452; x=1717692252; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=vT2fk1rgF/oHjVjPLloIashUDDheN/bTYNz4G/vtus0=;
        b=ILJMUUC0h3ShcnQvub0rnOx/vR2dD2KaMkf5r8qIrKBHdN2y/3doy5Pe3Xw6TGR6d/
         Q6aj6ND7retn+JHvJ5AIpED8R0d74Rx1tJHAwwK8CRnKCptd9a6Mfn3Ed3+/nTVxpN26
         J6UEyoIk0sTPTgpk7nq6EEpN1JE/I9NgQ9aoo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717087452; x=1717692252;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vT2fk1rgF/oHjVjPLloIashUDDheN/bTYNz4G/vtus0=;
        b=gqy//wo+MdsG3HuMw5kGcNC/OcXDnF7RqypDVW/NU+11b+VUZ0DqKbU9uKP9SM6AKY
         aai4VlRRve85/O7spN7zKDhbuQiSu0wwGlmdtBuufo14l16btuwHltO+0748HkzfAjyc
         h02OWWpcDGxIj7I21UeK3j3PkwY9LjErg3kahgGX83jRv+L7ATDXtij8Izma0L+JvGSs
         Du+o5RAoQ/LEJDGHc4BzPjQsJUYOhdrqAjQC0ju41pw4faavVwJzaQ7i6LegO0J1XfiP
         iXyhA/OivkFV+t7Q5MWH6SyQO+vKCHzFxktfxZUF3SVevy5EapHbnA1oB7yKP4hWeUbi
         hvMA==
X-Forwarded-Encrypted: i=1; AJvYcCWtR0R7jC2hTW0QG5cXnnt9FSO3+8vnThlbRWnPE2y8w2f/KkEsIrajqH2haJF2moBs6OR0iekGzCOwlFGBK/Lp7AvrWIJpQd+h7JVobGM=
X-Gm-Message-State: AOJu0Yz1DESxIG50nm7pT3wbM1giyqKDwGIv72sQlykkUs49Pp+8h7Dt
	BSSGtobxu0VJShacZsKIMP7Qk+zSsEhH/col/yjn7Z8eSOLQ5GgyVkj/TuKvtOY=
X-Google-Smtp-Source: AGHT+IHo3hmRL/CFEP2hQrB5W1//G4XauQ0LntrUNwnRnDZulWc/lTZLPy2AdXcuCQW/M6IvRCNL6Q==
X-Received: by 2002:a05:620a:1a1f:b0:794:c275:c9a1 with SMTP id af79cd13be357-794e9d95ce0mr326934085a.1.1717087451827;
        Thu, 30 May 2024 09:44:11 -0700 (PDT)
Message-ID: <0f12ce69-b139-4d97-8e36-23dd7c6bea41@citrix.com>
Date: Thu, 30 May 2024 17:44:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <35cf9d52e538aab964a3ecc050260abb3f27c60d.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <35cf9d52e538aab964a3ecc050260abb3f27c60d.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit


The subject should say "update Kconfig", because you're not (only)
disabling.

I'd suggest "xen/riscv: Update Kconfig in preparation for a full Xen build".

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> Disables unnecessary configs for two cases:
> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
> 2. By using tiny64_defconfig for non-randconfig builds.
>
> Only configs which lead to compilation issues were disabled.
>
> Remove lines related to disablement of configs which aren't affected
> compilation:
>  -# CONFIG_SCHED_CREDIT is not set
>  -# CONFIG_SCHED_RTDS is not set
>  -# CONFIG_SCHED_NULL is not set
>  -# CONFIG_SCHED_ARINC653 is not set
>  -# CONFIG_TRACEBUFFER is not set
>  -# CONFIG_HYPFS is not set
>  -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
>
> To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
> as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
> check_get_page_from_gfn() from xen/p2m-common.h.
>
> Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was
> included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
> aren't declared started to occur.
>
> CONFIG_XSM=n as it requires an introduction of:
> * boot_module_find_by_kind()
> * BOOTMOD_XSM
> * struct bootmodule
> * copy_from_paddr()
> The mentioned things aren't introduced now.
>
> CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
> acpi_disabled, so it is disabled for now.

CONFIG_BOOT_TIME_CPUPOOLS

Also the "depends on DT" isn't good enough as a restriction IMO.  It's
very ARM-dom0less specific.

> PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
> also disabled for now, as RISC-V hasn't introduced this headers yet.
> LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
> so to avoid compilation errors for randconfig it is disabled for now.

PERF_COUNTERS is x86-only, and both LIVEPATCH really should be guarded
by have HAVE_$FOO selected by ARCH.

However, that's not work to get stuck into now.

It's quite unreasonable how much stuff doesn't work in simple builds...

> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

I'm happy to fix up the two minor issues on commit.


From xen-devel-bounces@lists.xenproject.org Thu May 30 16:48:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:48:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732835.1138852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCixV-0004QF-Bz; Thu, 30 May 2024 16:48:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732835.1138852; Thu, 30 May 2024 16:48:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCixV-0004Q8-8u; Thu, 30 May 2024 16:48:57 +0000
Received: by outflank-mailman (input) for mailman id 732835;
 Thu, 30 May 2024 16:48:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCixT-0004Q2-Mb
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:48:55 +0000
Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com
 [2607:f8b0:4864:20::e2a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7fe227a4-1ea4-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 18:48:54 +0200 (CEST)
Received: by mail-vs1-xe2a.google.com with SMTP id
 ada2fe7eead31-48a39e0c441so18692137.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:48:54 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd0845dsm566765285a.91.2024.05.30.09.47.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 09:48:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fe227a4-1ea4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717087733; x=1717692533; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=L+KqWGZTZGpOULZC/Hv+9sZLxHgeNukXsnVk4PzK7m4=;
        b=XAUPxEjOrd/jxgHqS1fiOULrv/j0UlQZmJ+zu8cfitF2D4NcdkXE9fmaDpnJG0DuU1
         JXyjcg/c+kW7O51mrEox6+d4lvOmSQdxD+XYAthy70dSW7qR1woTFtdzuT+lwkwxEpcC
         8UiKmcd71VKVyZvkn1saSgumkUCczl3PdHDEM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717087733; x=1717692533;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=L+KqWGZTZGpOULZC/Hv+9sZLxHgeNukXsnVk4PzK7m4=;
        b=EmLR5YvSrpWn6otKMHjUtJaaclcAfx4vFT5GGPKm496SZSAe2D0EqOFeN8ObSJrAZQ
         R+5Yvw+/vY5TMoLd/hBTkjNFUhXmmR+ao+79GLT+AzeQRcdW1/L0i/6s8bUzzZ+n5xdT
         2x2t0QfKA3PR4YBT16uYjwZ3fMnW3dwJeEzl78IUIfUYHu/3WchnRGuYSTpJnzT5xw92
         i2dXpeG6EW+ue1ox6cQCq5QttOlksG8H5yzjrFRWSRaV+cJJIb5UK6OLw9Rl5g1USgFO
         h0UF/cObekbSogslS9Qr6Sxh+D0hZD5LgXW8Hu9k6RRYAP1kqerK61nsqYcJP+FiANMa
         oyyA==
X-Forwarded-Encrypted: i=1; AJvYcCWLBBQFkOxGGHbqw/B+ywL/oVpWtP4yYvENhS44IvRuUGd6NpiHydLmhbrIdj7U2RfHltJUKR1qC/XXkOZNlyplPFJWAFjNggA0+WgKhfo=
X-Gm-Message-State: AOJu0YyJ1wduGc4ZrDKCqk81UFTOJiWCwS+50H7yCA3c2fMH5xMX2qU/
	6977p6hrO9R1N/ABgpW0mAFQdCnx78Jpw7PXr0paNh3zmJQqeA5Xp6EhoxCyf+E=
X-Google-Smtp-Source: AGHT+IGEu6NItf5t7ZoJdXMLpswN1Ejx0s5HXCP7DuFzFEI+nVhvVIwvp8QMGk3qHF6IYNf1CEhz1g==
X-Received: by 2002:a05:6102:21ab:b0:47f:684:a3d with SMTP id ada2fe7eead31-48baeab1b70mr2790458137.27.1717087733444;
        Thu, 30 May 2024 09:48:53 -0700 (PDT)
Message-ID: <065c4b2f-93b6-428f-beca-b77e6f98cecc@citrix.com>
Date: Thu, 30 May 2024 17:47:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>, Jan Beulich
 <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> diff --git a/README b/README
> index c8a108449e..30da5ff9c0 100644
> --- a/README
> +++ b/README
> @@ -48,6 +48,10 @@ provided by your OS distributor:
>        - For ARM 64-bit:
>          - GCC 5.1 or later
>          - GNU Binutils 2.24 or later
> +      - For RISC-V 64-bit:
> +        - GCC 12.2 or later
> +        - GNU Binutils 2.39 or later
> +          Older GCC and GNU Binutils would work, but this is not a guarantee.

This sentence isn't appropriate to live here.

The commit message saying "this is what we run in CI" is perfectly good
enough.

With this dropped, Reviewed-by: Andrew Cooper
<andrew.cooper3@citrix.com>.  Can fix on commit.


From xen-devel-bounces@lists.xenproject.org Thu May 30 16:49:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:49:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732839.1138861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCiyK-0004uL-KO; Thu, 30 May 2024 16:49:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732839.1138861; Thu, 30 May 2024 16:49:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCiyK-0004uE-H4; Thu, 30 May 2024 16:49:48 +0000
Received: by outflank-mailman (input) for mailman id 732839;
 Thu, 30 May 2024 16:49:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCiyI-0004pF-Js
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:49:46 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9de464cc-1ea4-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 18:49:45 +0200 (CEST)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-79476195696so73398585a.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:49:45 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794abd0845dsm566765285a.91.2024.05.30.09.48.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 09:49:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9de464cc-1ea4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717087784; x=1717692584; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=D06sgf/A4q21cvquEasbm7EXnFj7RPSW5uAPC2GLecA=;
        b=q0sdOSnTvqH4YUw03m2Qi0IGQUsrYvdhsuibpFqImur3RdijcrMG47uN4zahmYne4K
         oyuJxy3EIwWGPjSmcEAYxQDxmN/9HXu1gUqfAmNL6M6+aAEJ2bS3E8LbjmIylM/IKeoC
         6qp9HqV4bDGKbhDUmaMXb166tnu7K/oyStbTs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717087784; x=1717692584;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=D06sgf/A4q21cvquEasbm7EXnFj7RPSW5uAPC2GLecA=;
        b=nwaf1+4V9ZEcuK1v1ESN5KGPSYHuRRA1eFJuBf/346YqazzbZhqrep2+AVjEGRQcce
         gniV0gxOCZlNcckQkMfhsiWwEVOUqx2UW9condBEoimgRI/X8OEIFvJ1mEVtfABzN3no
         2hxRUtpSrmBWK0ZZwUfGCpGQI8+ptzaUrLOTcOicGycjLDfyALkPg2amDsLqNBVeF2V0
         17odpK7eIizf/J4xY8xV8Te/4X7t5nZS9P20JnTDCUzgu0OV4I8+9fI7LvyUPrZB9ZeL
         BihHENnwQVBf1LcSe1AzwhEvRaThzc90zTyN+D0irgqz05BCzXKSVFp5nphfaZiovCQK
         ldxQ==
X-Forwarded-Encrypted: i=1; AJvYcCUQ5wVougEO48Llxj8712eQz2egbQb6H3fdE+RPN58qk5WKDlSsjXa4QZ5okrcYp6btHwRi6eQd8W2qcMR7EtNaHjjBRBUEiz0VnYPg+hY=
X-Gm-Message-State: AOJu0YyzMo5tWXAjktSHkJT982s8h7/5gIqpMIfEtnUEKG6OTS+fNF+6
	dUlj9xVhFOzl1BCrPp57/BjKYVNA1dHX1AsqjKBk8K0+TdHsS0J+ql2JuUgkPU0=
X-Google-Smtp-Source: AGHT+IHMby/7clvFJcqhMCFATWdGoNWQIlt5oEnOTS06iaB6XpSRX+868/szTkNH/IGeyIdqVLgRdQ==
X-Received: by 2002:a05:620a:60f4:b0:794:e8af:69c9 with SMTP id af79cd13be357-794e9e4c079mr261938085a.75.1717087783890;
        Thu, 30 May 2024 09:49:43 -0700 (PDT)
Message-ID: <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
Date: Thu, 30 May 2024 17:48:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
> index 8285bcffef..bda35fc347 100644
> --- a/xen/arch/riscv/stubs.c
> +++ b/xen/arch/riscv/stubs.c
> @@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
>  
>  nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
>  
> -/*
> - * max_page is defined in page_alloc.c which isn't complied for now.
> - * definition of max_page will be remove as soon as page_alloc is built.
> - */
> -unsigned long __read_mostly max_page;
> -
>  /* time.c */
>  
>  unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
> @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
>  {
>      BUG_ON("unimplemented");
>  }
> -
> -/*
> - * The following functions are defined in common/irq.c, but common/irq.c isn't
> - * built for now. These changes will be removed there when common/irq.c is
> - * ready.
> - */
> -
> -void cf_check irq_actor_none(struct irq_desc *desc)
> -{
> -    BUG_ON("unimplemented");
> -}
> -
> -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
> -{
> -    BUG_ON("unimplemented");
> -
> -    return 0;
> -}

All 3 of these are introduced in the previous patch and deleted again
here.  Looks like a rebasing accident.

(This patch has to be the final one touching build related things, so I
can't simply fix it on commit)

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 16:52:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:52:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732845.1138873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj0P-0006dU-1s; Thu, 30 May 2024 16:51:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732845.1138873; Thu, 30 May 2024 16:51:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj0O-0006dN-TX; Thu, 30 May 2024 16:51:56 +0000
Received: by outflank-mailman (input) for mailman id 732845;
 Thu, 30 May 2024 16:51:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCj0N-0006d6-FT
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:51:55 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eb9bf29a-1ea4-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 18:51:54 +0200 (CEST)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-57a2ed9af7dso344886a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:51:54 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57a31c6d44fsm4979a12.61.2024.05.30.09.51.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 09:51:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb9bf29a-1ea4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717087914; x=1717692714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=q1iXChzx3I0tb/MZycIp9NIeTxN3s9RHuHsOh+AQOu8=;
        b=pak9d7QxkCSTnmQvMXf11EpbzpFhaf9NZZYQwS3Hz3ohyjSbJZfZmhgqw9R1rj/7jJ
         tZzNd6wz1d+kTvJzaB1Ed6hS3sEF7AxPurfQjD8ECftrKf8RW4iGGnSaXW8b7NYlCsIn
         2eatbYUGxGAYTW6ijDbxc6Zdc1n/SNEtWYOH0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717087914; x=1717692714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=q1iXChzx3I0tb/MZycIp9NIeTxN3s9RHuHsOh+AQOu8=;
        b=rEt7zghdK/1UTgTbNr4nWcE5bNac0xqeA/92eIE992m1Yjdtbmjtl1Y7Fi0yXse8xW
         YVBh21mpErCYo86x1vz/yXA1CBNBwocM6+aX4VKzWQCSJ5vAW8DaKOjI9rf1YSR1DSiI
         G0onOfMtcW5mJ/d+XwNo/9NAxUGl7JGWnPhHCFrAhLIqSzNoynlTXw3tGB3G7smfuRW4
         GcDqCJUuu3AxmaUi8xS9qCkVxXHDJ1tQvCXyq5/dZeqlHeYTNHHtGAInW6dok8hbKSHG
         eOwyy+8+HZhC1KrEeecMbqZOOyvON05TmUBYov2tNPOzVL9bywxPVgUJn5ANvZNYMwpH
         mchA==
X-Forwarded-Encrypted: i=1; AJvYcCUAhLbtmu2jt4ALFCuLohmyYn3A0AezkuR4xLQwBW8pWZYTW8kbnGnNBGZ9cGB+W5P0gaVN4vVzSaXxi0O7nEune94Nyc3+qZgkAL39XD0=
X-Gm-Message-State: AOJu0YyUUNKF5qbE+He8C39KUKpzVXz8X+3BN6T18yK3lCuUY7RNfoLu
	acZFjpHdYygFbDM5g8ctOy/5lvKfC1UdFqhflJQvIS7d/djnvjjlec0WmOTcfyQ=
X-Google-Smtp-Source: AGHT+IFo54zFzKbmCCHj/8bzC2qy9sQ4S1+IoyIfa4G+0HIRfVaVpZ4iN+k1vkD+Y1x2HmmAI/WuIw==
X-Received: by 2002:a50:8e5d:0:b0:57a:242e:806b with SMTP id 4fb4d7f45d1cf-57a242e81d1mr1515517a12.18.1717087914380;
        Thu, 30 May 2024 09:51:54 -0700 (PDT)
Message-ID: <02564ec6-1f44-465f-9e3f-9ab92a08ff9d@citrix.com>
Date: Thu, 30 May 2024 17:51:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 6/8] xen/riscv: add minimal amount of stubs to build
 full Xen
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <4e5b814c3f73bd4ae6bb80296e17ac80bd8d224a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <4e5b814c3f73bd4ae6bb80296e17ac80bd8d224a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
> new file mode 100644
> index 0000000000..8285bcffef
> --- /dev/null
> +++ b/xen/arch/riscv/stubs.c
> @@ -0,0 +1,439 @@
> <snip>
> +void udelay(unsigned long usecs)
> +{
> +    BUG_ON("unimplemented");
> +}
> +
> +/* guest_access.h */ 

You've got trailing whitespace here.

Also, this patch is (still) a shocking indictment on how bad our common
vs arch split really is.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 16:56:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:56:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732852.1138883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj4L-0007Dk-HP; Thu, 30 May 2024 16:56:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732852.1138883; Thu, 30 May 2024 16:56:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj4L-0007Dd-Cw; Thu, 30 May 2024 16:56:01 +0000
Received: by outflank-mailman (input) for mailman id 732852;
 Thu, 30 May 2024 16:56:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCj4K-0007DX-TO
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:56:00 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7db3fbf7-1ea5-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 18:55:59 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-52b83225088so400851e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:55:59 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d34500sm16955e87.54.2024.05.30.09.55.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 09:55:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7db3fbf7-1ea5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717088159; x=1717692959; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=YGHVeE5O/iLTCsbWg7npMOS8KTfQhrI/JPwNfdlyenI=;
        b=FSOzoo89/xtcwkSBII3zt+reyu9xKwCT6RrR0m3v4Arpk/ME1/9JJxy9ZoNagUCZDZ
         YFnEuAN79dMphfWKO8qsE+yV7LVpK9qaZItwXFNISeb0D9Z90E3nVdY8UqZt4IL0sjqt
         VzmB3W5BPas0lzJY8MffkSjbczA4DjLKAe3Rm2fJJjinHy0ek8UUAYlmrheJSDuoalVU
         AFvTLWUSxDpYcapharcF+4yOOI5vHcYvD94QS70Cj3kg1Thllu6jCGgy0R2rB6BDaif9
         jAi5cbOYnM7roiLxYGnIjXhWY7t2xyAaFv9WuyY/QIjVHplE0lmgceNfI8x/Hs4xN+ps
         DAGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717088159; x=1717692959;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YGHVeE5O/iLTCsbWg7npMOS8KTfQhrI/JPwNfdlyenI=;
        b=hWxIITr+dPpQIqva1tOWcrPKbMsVVsBP+sVWqG1SeJxpBq7t20SdhtymGiTJyAASst
         x9Tw8t3XZsqx8F0jfyiXt2C68QL6yE1n/eER4JmH81dmTqZljG9vjMs7VpcU77MEfZKJ
         8LnZg7a6ynAaxUHxiGbRlicu+YPYWhrTNx5gbbG9AtMtu+SAWd0vjcbjkNTJvekMnBWJ
         STJsAjiVPPfxTtO8uTYPK70EISbFZsEq3AXo1o7FlabvCbWchy2z8Q9HMD2AVmXRWYbl
         +y3cNY0UiSOuf2cYX23MHBA3DHjfzDFQmNqZp5HKoQfoXJXWgo+tnjuF4GvknJthR4M2
         r2Kg==
X-Forwarded-Encrypted: i=1; AJvYcCXcgwm2QdmiAYw5XVq9q8Nyy3/gSrQJmBAiZLa3zu7g8sp9+9b59B5img34GFPCPMTInWo8IM7EkPEJ2w5meDev7qxNi7sOkDuziA5K0j0=
X-Gm-Message-State: AOJu0YyeKIAKmYZAxPIgioVQkgQMWce+V5EARrigZ4o3AI5lBufwx7o7
	8jiRgRBPwOHaVTBdFF9WLGLk1w3HXuEqkucambG1Dh2khqfkbH2k
X-Google-Smtp-Source: AGHT+IGfn1AuIvmFx4weRdCBLGL14uKkUhrDaGlkdYpXW/GSKV+9ZefApbmFShGAE+MisgcaRCrVrQ==
X-Received: by 2002:a19:770d:0:b0:51f:4c53:8e4c with SMTP id 2adb3069b0e04-52b7d43aee4mr1725419e87.33.1717088159135;
        Thu, 30 May 2024 09:55:59 -0700 (PDT)
Message-ID: <840dacc2b79260c31761a6fadd95fc5d4bb2b50d.camel@gmail.com>
Subject: Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Alistair Francis <alistair.francis@wdc.com>, Bob
 Eshleman <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
Date: Thu, 30 May 2024 18:55:58 +0200
In-Reply-To: <0f12ce69-b139-4d97-8e36-23dd7c6bea41@citrix.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
	 <35cf9d52e538aab964a3ecc050260abb3f27c60d.1717008161.git.oleksii.kurochko@gmail.com>
	 <0f12ce69-b139-4d97-8e36-23dd7c6bea41@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 17:44 +0100, Andrew Cooper wrote:
>=20
> The subject should say "update Kconfig", because you're not (only)
> disabling.
>=20
> I'd suggest "xen/riscv: Update Kconfig in preparation for a full Xen
> build".
>=20
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> > Disables unnecessary configs for two cases:
> > 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
> > (GitLab CI jobs).
> > 2. By using tiny64_defconfig for non-randconfig builds.
> >=20
> > Only configs which lead to compilation issues were disabled.
> >=20
> > Remove lines related to disablement of configs which aren't
> > affected
> > compilation:
> > =C2=A0-# CONFIG_SCHED_CREDIT is not set
> > =C2=A0-# CONFIG_SCHED_RTDS is not set
> > =C2=A0-# CONFIG_SCHED_NULL is not set
> > =C2=A0-# CONFIG_SCHED_ARINC653 is not set
> > =C2=A0-# CONFIG_TRACEBUFFER is not set
> > =C2=A0-# CONFIG_HYPFS is not set
> > =C2=A0-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
> >=20
> > To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to
> > <asm/domain.h>
> > as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
> > check_get_page_from_gfn() from xen/p2m-common.h.
> >=20
> > Also, it was included <xen/errno.h> to asm/p2m.h as after the
> > latter was
> > included to <asm/domain.h> the compilation error that EINVAL,
> > EOPNOTSUPP
> > aren't declared started to occur.
> >=20
> > CONFIG_XSM=3Dn as it requires an introduction of:
> > * boot_module_find_by_kind()
> > * BOOTMOD_XSM
> > * struct bootmodule
> > * copy_from_paddr()
> > The mentioned things aren't introduced now.
> >=20
> > CPU_BOOT_TIME_CPUPOOLS requires an introduction of
> > cpu_physical_id() and
> > acpi_disabled, so it is disabled for now.
>=20
> CONFIG_BOOT_TIME_CPUPOOLS
>=20
> Also the "depends on DT" isn't good enough as a restriction IMO.=C2=A0
> It's
> very ARM-dom0less specific.
>=20
> > PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
> > also disabled for now, as RISC-V hasn't introduced this headers
> > yet.
> > LIVEPATCH isn't ready for RISC-V too and it can be overriden by
> > randconfig,
> > so to avoid compilation errors for randconfig it is disabled for
> > now.
>=20
> PERF_COUNTERS is x86-only, and both LIVEPATCH really should be
> guarded
> by have HAVE_$FOO selected by ARCH.
>=20
> However, that's not work to get stuck into now.
>=20
> It's quite unreasonable how much stuff doesn't work in simple
> builds...
>=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
>=20
> I'm happy to fix up the two minor issues on commit.
Thanks. I Would appreciate that.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Thu May 30 16:57:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 16:57:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732860.1138891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj5S-0007pL-RA; Thu, 30 May 2024 16:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732860.1138891; Thu, 30 May 2024 16:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCj5S-0007pE-Oh; Thu, 30 May 2024 16:57:10 +0000
Received: by outflank-mailman (input) for mailman id 732860;
 Thu, 30 May 2024 16:57:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCj5R-0007if-G7
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 16:57:09 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a6dd481c-1ea5-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 18:57:09 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52ad8230bb9so1251578e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 09:57:09 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d89490sm13652e87.247.2024.05.30.09.57.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 09:57:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6dd481c-1ea5-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717088228; x=1717693028; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=OE0fnyDYoxBJGk4RvrZhMNcCSm1o0tvSnGpWEbaKIPE=;
        b=PZn8J9uG80yvpfCiw5iAs/7WkmclJS/Yx3z5OH2kZypePP+z9eHUunKEsDd2YrTAfB
         cJjYak4EjdV4bHl4MKhDNyD+3ZOvPWFau/4uiU57sW8BQffxeCA9lRTmUbB9R8b7P+AX
         u3SZ/UrUbMnAoHEg7JG0RddZ3gSkE/wZg544HSMaOKsrd9KsQvVuoLLkEJVjg0B68eGN
         5Iw77+csJjzOpKnmlpFNsn1rMejhUrVH8XwEthFnT5ip4VeliwYxtJKyCOXos6im4Ne/
         Imq9k8HYJ+Q1TCXcK9ObqI7FF8uqrk89cJ3Zxwy7mkDHNOjGrzgzDCWdtJrR2PEZEUig
         /F2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717088228; x=1717693028;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OE0fnyDYoxBJGk4RvrZhMNcCSm1o0tvSnGpWEbaKIPE=;
        b=DMLDGfdBrfHkN7VgK19HeVCPXdVHCd7N3qL6ZiakYssOqNWuC/8EKWsbIZrAFUe7NP
         SCbajOc9k6vGRe958R+EC1UW5N/PayTCiAB3pMYNYLwITmJ4F0BCkWJ0dRq8SqfcGKCc
         kUJd3LwgSFTCN8w5kX0ae3em/+QR0ZctwfWzf1WlKDEo0QCqduoQ7GPQWYEB2FzMGRDh
         2VA7ddFXQlXFRgmu5Bqn8sh9Fs5d+fWujJzLBSfvT7oJDQ0wgJX1CaW4FZa48mIEhGmB
         hfOfFlQcB0FzS1fqVYZ5uTVuSN8QF5Wdgql0OD531hrJbBKu8g2724BvTntTvG9/ZKMz
         7x9A==
X-Gm-Message-State: AOJu0YzufBW8rIM9FOYprARM7QTPwW2+VgugLu/ArgaLd80z6BuLidTp
	37WGlaReGZff7IvNDndVNSxVfN/zQDOKEURSfLxxCR4l0Tn5h0Qs
X-Google-Smtp-Source: AGHT+IHIxvth94CNUYqycPZFdRPatiagN4A/J8O3o9wp0JPvvwFXwvwmtwwwCVkqQ/d8JtUEVSzUDw==
X-Received: by 2002:a05:6512:32b5:b0:51e:150e:2c45 with SMTP id 2adb3069b0e04-52b7d489296mr1750482e87.63.1717088228276;
        Thu, 30 May 2024 09:57:08 -0700 (PDT)
Message-ID: <2d8785a6f44e3877ee198683f55022eed2599224.camel@gmail.com>
Subject: Re: [PATCH] x86/hvm: allow XENMEM_machine_memory_map
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Andrew Cooper
	 <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>
Date: Thu, 30 May 2024 18:57:07 +0200
In-Reply-To: <Zlg1XiUFR0sF3KCg@macbook>
References: <20240530075318.67491-1-roger.pau@citrix.com>
	 <0768b842-719e-4736-a941-dfa4d50c173d@citrix.com>
	 <Zlg1XiUFR0sF3KCg@macbook>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 10:14 +0200, Roger Pau Monn=C3=A9 wrote:
> On Thu, May 30, 2024 at 09:04:08AM +0100, Andrew Cooper wrote:
> > On 30/05/2024 8:53 am, Roger Pau Monne wrote:
> > > For HVM based control domains XENMEM_machine_memory_map must be
> > > available so
> > > that the `e820_host` xl.cfg option can be used.
> > >=20
> > > Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> >=20
> > Seems safe enough to allow.
> >=20
> > Does this want a reported-by, or some further discussion about how
> > it
> > was found?
>=20
> I've found it while attempting to repro an issue with e820_host
> reported by Marek, but the issue he reported is not related to this.
> It's just that I have most of my test systems set as PVH dom0.
>=20
> > Also, as it's mostly PVH Dom0 bugfixing, shouldn't we want it in
> > 4.19?
>=20
> Yeah, forgot to add the for-4.19 line and Oleksii, adding him now for
> consideration for 4.19.
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:12:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:12:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732871.1138905 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjKL-0002T3-5z; Thu, 30 May 2024 17:12:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732871.1138905; Thu, 30 May 2024 17:12:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjKL-0002Sw-2J; Thu, 30 May 2024 17:12:33 +0000
Received: by outflank-mailman (input) for mailman id 732871;
 Thu, 30 May 2024 17:12:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCjKJ-0002Sq-9i
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:12:31 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cbe210da-1ea7-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 19:12:30 +0200 (CEST)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2e95abc7259so12279211fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:12:30 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2ea91cc7cd3sm192111fa.92.2024.05.30.10.12.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 10:12:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbe210da-1ea7-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717089150; x=1717693950; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=HP/kXBIbTyyxkwl6LnojllrDDcfzOpLmHuatAwn4qkM=;
        b=hZb54Iyyln5+kyAX7C4KyKWEjwQBs83ZhJeoEa+ihgjGnt6GwNKl08tY4vz9Kx/A7D
         vGLLM0axMFuRfupitZduolRkJKGqmwGO75PHmN6TVOSNVP48562HwLIveWmbKT8wROM+
         Rnu5FatZ4kDgjxUIzDSmF2BEzabDBCiXsL6SS30Sh4XIQQwAG8bmndRcL1qyLNXzz3zk
         X9rJ00yTQRGaOyDK/0kyH0/J36YbkXDJYnWZle1JoWcwnt7RlZ7vd/R7QrvNmFVtfGAv
         LQwLDo3Pe80dZ66Aza3d/pR3XLxJh+CsZjbGMBukfziVMeoEJKGeoZfF3AxOc9z1KW8E
         c5lg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717089150; x=1717693950;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HP/kXBIbTyyxkwl6LnojllrDDcfzOpLmHuatAwn4qkM=;
        b=pW9pW76yEIF0g8ypbxVTSK0NCUQrMzf6dyxJe0fjToG4kysIEXHqtuOLya29lEOuIG
         PeMSHdxrTZhKh6PW9xdP3oWNL5FH9Aw8BDoYym1kFnr2/6NOeiW5QdQ//bEtT0y4xy+9
         QsS4/r9Tt9TSXo/LXCRqhS8/oqmRISAuERF0RULFWh4QVMHIy7kRfKLPgWx/dNuMTdUI
         81BkSiQ7I1MUxm/n7Y1vUcgWNTFTYhC6CNM7XiUuTlVEGGcWPco1yNGhe3Oh2i5wM66c
         g3R69pcqHRtkAFU6mZ6fjIthoM2wTCnIVSM3SZD0gk3MeZtbXttq+31v7ys/ipD6cA4v
         rUXw==
X-Forwarded-Encrypted: i=1; AJvYcCUR5EA2WMvQ+1rMCWcdIYX1+B3P2GYwRWlpi2n/7lFpyq4cSSeyGNoSZTmxKXg3dzY+kwQJPO3CgTbAhodG0moNB73uop7z51I4AsrCObA=
X-Gm-Message-State: AOJu0YyPYq760UcqaqkmMdx+edZtMdqjqY5WIJJePScEOfbSgS9YWvBZ
	PvPB4Jf7V7BUJgrSx7s/STWEYiXUwl+leeMPKV/aXhNBx1R+VU6W
X-Google-Smtp-Source: AGHT+IGoB0OFErXOjruUTHJOg7SsXp0E49xsEN4Kya0FnmUfSarzL16ACc+pqzveGsrWKwRZb9WjGQ==
X-Received: by 2002:a2e:9951:0:b0:2ea:59a1:2810 with SMTP id 38308e7fff4ca-2ea84849961mr16816421fa.48.1717089149385;
        Thu, 30 May 2024 10:12:29 -0700 (PDT)
Message-ID: <2747a20d490252c279dd051c41a9b8dc69fb4c3e.camel@gmail.com>
Subject: Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Date: Thu, 30 May 2024 19:12:28 +0200
In-Reply-To: <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
	 <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
	 <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 17:48 +0100, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> > diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
> > index 8285bcffef..bda35fc347 100644
> > --- a/xen/arch/riscv/stubs.c
> > +++ b/xen/arch/riscv/stubs.c
> > @@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t,
> > cpu_core_mask);
> > =C2=A0
> > =C2=A0nodemask_t __read_mostly node_online_map =3D { { [0] =3D 1UL } };
> > =C2=A0
> > -/*
> > - * max_page is defined in page_alloc.c which isn't complied for
> > now.
> > - * definition of max_page will be remove as soon as page_alloc is
> > built.
> > - */
> > -unsigned long __read_mostly max_page;
> > -
> > =C2=A0/* time.c */
> > =C2=A0
> > =C2=A0unsigned long __ro_after_init cpu_khz;=C2=A0 /* CPU clock frequen=
cy in
> > kHz. */
> > @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
> > =C2=A0{
> > =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > =C2=A0}
> > -
> > -/*
> > - * The following functions are defined in common/irq.c, but
> > common/irq.c isn't
> > - * built for now. These changes will be removed there when
> > common/irq.c is
> > - * ready.
> > - */
> > -
> > -void cf_check irq_actor_none(struct irq_desc *desc)
> > -{
> > -=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > -}
> > -
> > -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
> > -{
> > -=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > -
> > -=C2=A0=C2=A0=C2=A0 return 0;
> > -}
>=20
> All 3 of these are introduced in the previous patch and deleted again
> here.=C2=A0 Looks like a rebasing accident.
Not really.

This was done to avoid build failures for RISC-V. If the HEAD is on the
previous patch where these changes are introduced and then we just drop
them, it will lead to a linkage error because these functions are
defined in common/irq.c, which isn't built for RISC-V if the HEAD is on
the previous patch:
   /build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'
   riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to
   `irq_startup_none'
   riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to
   `irq_actor_none'
   riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to
   `irq_actor_none'
   riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to
   `irq_actor_none'
   riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't
   defined

That is why these stubs were introduced in the previous patch (because
common/irq.c isn't built at that moment) and are removed in this patch
(since at the moment of this patch, common/irq.c is now being built).

~ Oleksii

>=20
> (This patch has to be the final one touching build related things, so
> I
> can't simply fix it on commit)
>=20
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Thu May 30 17:16:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:16:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732876.1138915 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjNi-0003GL-K3; Thu, 30 May 2024 17:16:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732876.1138915; Thu, 30 May 2024 17:16:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjNi-0003GE-Gc; Thu, 30 May 2024 17:16:02 +0000
Received: by outflank-mailman (input) for mailman id 732876;
 Thu, 30 May 2024 17:16:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCjNh-0003G4-5g; Thu, 30 May 2024 17:16:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCjNh-0003q7-3N; Thu, 30 May 2024 17:16:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCjNg-0007A6-RU; Thu, 30 May 2024 17:16:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCjNg-00087V-Qw; Thu, 30 May 2024 17:16:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9uyzcQUUXX/z+Z/sPXAkbkdti951Ip7p9df+V0Iq2kU=; b=Exonm7mgDVCri0QHbpNdEP8sFV
	pl3dIqOAXKlD+FHsaQx71Z7v4QpQae96qDeiwFVLEVTEIWqO/e8kAg+CvXydjDpDWxj0Ukae39Ext
	qbfpN0yOd02HJ0+RUv7cB6h+ARGDQ0U8IGONRZi5naJaJdj0FGd7Mslwx5fICWobau64=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186197-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186197: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9a905d7dc65883af082532b4dc91ce0131e54047
X-Osstest-Versions-That:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 17:16:00 +0000

flight 186197 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186197/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9a905d7dc65883af082532b4dc91ce0131e54047
baseline version:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d

Last test of basis   186184  2024-05-29 17:04:00 Z    1 days
Testing same since   186197  2024-05-30 12:00:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   12f9599a3b..9a905d7dc6  9a905d7dc65883af082532b4dc91ce0131e54047 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:16:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:16:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732882.1138925 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjOb-0003lk-SE; Thu, 30 May 2024 17:16:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732882.1138925; Thu, 30 May 2024 17:16:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjOb-0003lb-PO; Thu, 30 May 2024 17:16:57 +0000
Received: by outflank-mailman (input) for mailman id 732882;
 Thu, 30 May 2024 17:16:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCjOa-0003gj-2n
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:16:56 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a1bac40-1ea8-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 19:16:55 +0200 (CEST)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-52a54d664e3so309863e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:16:55 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d75f4bsm18250e87.161.2024.05.30.10.16.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 10:16:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a1bac40-1ea8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717089415; x=1717694215; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=3KyvK7NuDDcsd/qpYPAYUyHPjbobIFeX5lcnSbEZTks=;
        b=HMpG5pic5tKX+nFTVvv+ZjE5YwNzvq2w21rlWFqmxPLrhSRWQnTDaYQZM5c785JVGJ
         ob34sniieUzPvbRgDFgbBhuXHgEomE+0WzQdE+i4SkEN6WgPTWYAYJI2nmnyU4UQ5I1I
         fKX9C5EOUGHwjbchVAczeKi7/N7esgNiHikK31Zqad0zo74neOiHKThJ/mgFmqsC+HLQ
         J2Rlc5t1JYRXxqKeW/FNyMURgwfAfqkkE+6Ug6EWDxf34WwXWbFqina2JbqrkDArHCx6
         H6m28lYsk7OV/Dyme23iN4SizeanKaai014YgOvQfMcNGxOljQ5kSCYmFUPc/+/addTe
         dyYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717089415; x=1717694215;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3KyvK7NuDDcsd/qpYPAYUyHPjbobIFeX5lcnSbEZTks=;
        b=e6TbP4J0tn1JXlAV9ONdIfpFip5Wfz9g7qYYiFjdaOPldC7NrytnHlwarsf6GZ7p60
         uaNw4OLYA/y0rzz0lTVrF1R/CWP2hi9z+03CJeUdkVCCQuRSmaV1ZWm1Q+eKtYL2Mp/L
         JqCezyILNstcC1FeYAQYNc5s3+PZsImW2rNhBT+GuOErT+c/cWd33GbEgS6CRIE7X049
         vvo39rmwXA1YQObG23A25Lx1fWHLV+uAp5hx5JsQ289pQwGGq1HEGU71CJJK/16kDVkp
         9+IMQLeGXOwQmxsMyK182TsDXG7NL+GvlRnY+YJeVTTGi63k65t7QYrSQiGcGRbPDh/L
         mdXA==
X-Forwarded-Encrypted: i=1; AJvYcCWrsoaRl+N9fMpskHPl1eMu7OSpXlxoahM7ZaoeqWJJVXYujKqHvIyd+lOCnn6eRjYcc8f03idLivrwj1roNEg5dBHDlpYv0FW9W5+3LZ4=
X-Gm-Message-State: AOJu0YxAD3oe5UqvEJ4VsHNnxiZP0t/qh1sWvwpoWUo/HL9oU7+l1xMQ
	cql2Dj1OIDd+V2HHY62sAow8rIezdDy/mKygdyQ5NW1J1qV9okLp
X-Google-Smtp-Source: AGHT+IF+CJtFue6WSdCAjfghn2Y0V1T5+JFeYSBPaGWTWZt5ajAqE0KZhKzvk6esrBMR/9MSmPGXYA==
X-Received: by 2002:ac2:518d:0:b0:52b:8612:eddd with SMTP id 2adb3069b0e04-52b8612eebemr30778e87.17.1717089414947;
        Thu, 30 May 2024 10:16:54 -0700 (PDT)
Message-ID: <27148c63318ef78d542db7cc20373e3cb3836df3.camel@gmail.com>
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org,  Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Stefano Stabellini
	 <sstabellini@kernel.org>
Date: Thu, 30 May 2024 19:16:54 +0200
In-Reply-To: <065c4b2f-93b6-428f-beca-b77e6f98cecc@citrix.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
	 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
	 <065c4b2f-93b6-428f-beca-b77e6f98cecc@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 17:47 +0100, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> > diff --git a/README b/README
> > index c8a108449e..30da5ff9c0 100644
> > --- a/README
> > +++ b/README
> > @@ -48,6 +48,10 @@ provided by your OS distributor:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For ARM 64-bit:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 5.1 or later
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.24 or=
 later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For RISC-V 64-bit:
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 12.2 or later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.39 or late=
r
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Older GCC and G=
NU Binutils would work, but this is not a
> > guarantee.
>=20
> This sentence isn't appropriate to live here.
>=20
> The commit message saying "this is what we run in CI" is perfectly
> good
> enough.
>=20
> With this dropped, Reviewed-by: Andrew Cooper
> <andrew.cooper3@citrix.com>.=C2=A0 Can fix on commt.
I am okay with dropping this sentence, but someone ( unfortunately I
don't remember who Jan? Julien? ) requested it, and I think it would be
nice to hear their opinion before doing so.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:22:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:22:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732896.1138934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjTg-0005U9-Dx; Thu, 30 May 2024 17:22:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732896.1138934; Thu, 30 May 2024 17:22:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjTg-0005U2-BJ; Thu, 30 May 2024 17:22:12 +0000
Received: by outflank-mailman (input) for mailman id 732896;
 Thu, 30 May 2024 17:22:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCjTf-0005Tw-9X
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:22:11 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 25a3ab56-1ea9-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 19:22:10 +0200 (CEST)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-57a23997da3so956254a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:22:10 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57a31b99445sm35146a12.18.2024.05.30.10.22.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 10:22:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25a3ab56-1ea9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717089730; x=1717694530; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=JB2maaoYCqR6n0ZV4f5ax1waFqmhG7Zo47ceb3HW61M=;
        b=L7DWXisPX/T1KZjwCQDFsdmd8G+GMSdwY4dnwvxuA5usuocvKRdc9V82xLhmDkHv7v
         saC0Fznn4B95kOawbVOAk8uA1Snr5Si1JWfdGfPNfy0yaoJNF1SmaY2K4JZblpQsPon8
         QzrrAd1DaSHRmLuCBcoKTaBdKG/DS0KRh5A6g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717089730; x=1717694530;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JB2maaoYCqR6n0ZV4f5ax1waFqmhG7Zo47ceb3HW61M=;
        b=SSKWyIaE/5fzOdhJspmka+Vaj86vvBMbzv2x0l0mRHkalc3elJwgUEoYq3UcO/L1+b
         ZuiyDLpeG7LUnWcHF0bBntFBUQO53us2uFC0Iw68W11VRDS/u54Qb61wR7qNI52PxHcQ
         hjpRDeK+nghjOqirJjiQXU2YvbKQfSJX3eOYzILThSQMhwse2B5WexCzO638AJVc5a1m
         t8Ujowtdld0FpDjOMC5iG/M8jJcbPGFSu5orZevayUg//cijPLEE/Ol/vfn8gDSP6mn6
         odMeGXT6qKIg/ziEl8KDHHrr6fcHYHBx1AkTkpLfwVfkNyp2IihjdQZ3pgbfex59C3/u
         LSZA==
X-Forwarded-Encrypted: i=1; AJvYcCXt9yKK3uJQdpJ0El9/AhOUlbdb0Y75TOiKYGnumkU1+jaJnsyS7hrmEGM58LqJMtHfuoe28NFwiEPHqwoJU9mnhr/66uDkMskCABXCXKw=
X-Gm-Message-State: AOJu0Yyg3H+NICwLuMD9FRVghiLTgKyQuhTxVrh4vEKStcvqrWHpH8IR
	0fh7vqt/PGsXmhHev2ZqoQ5kprjFB5EzXj1xU1xrML83Fi8oRqXdZFo8ifpD/VC48caZvXGSr9H
	g
X-Google-Smtp-Source: AGHT+IFqD4g9gWrWBuQlAjhg3EVC8h6c95E84upKFYvRL8oiayZjf8iEtuFyZRTP+qWZE6fGLTyjSA==
X-Received: by 2002:a50:f69a:0:b0:57a:1e0a:379f with SMTP id 4fb4d7f45d1cf-57a1e0a381bmr1422422a12.16.1717089729646;
        Thu, 30 May 2024 10:22:09 -0700 (PDT)
Message-ID: <8f37cb67-32ae-4e01-af60-f94af136698f@citrix.com>
Date: Thu, 30 May 2024 18:22:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
To: "Oleksii K." <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
 <065c4b2f-93b6-428f-beca-b77e6f98cecc@citrix.com>
 <27148c63318ef78d542db7cc20373e3cb3836df3.camel@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <27148c63318ef78d542db7cc20373e3cb3836df3.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 6:16 pm, Oleksii K. wrote:
> On Thu, 2024-05-30 at 17:47 +0100, Andrew Cooper wrote:
>> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>>> diff --git a/README b/README
>>> index c8a108449e..30da5ff9c0 100644
>>> --- a/README
>>> +++ b/README
>>> @@ -48,6 +48,10 @@ provided by your OS distributor:
>>>        - For ARM 64-bit:
>>>          - GCC 5.1 or later
>>>          - GNU Binutils 2.24 or later
>>> +      - For RISC-V 64-bit:
>>> +        - GCC 12.2 or later
>>> +        - GNU Binutils 2.39 or later
>>> +          Older GCC and GNU Binutils would work, but this is not a
>>> guarantee.
>> This sentence isn't appropriate to live here.
>>
>> The commit message saying "this is what we run in CI" is perfectly
>> good
>> enough.
>>
>> With this dropped, Reviewed-by: Andrew Cooper
>> <andrew.cooper3@citrix.com>.  Can fix on commt.
> I am okay with dropping this sentence, but someone ( unfortunately I
> don't remember who Jan? Julien? ) requested it, and I think it would be
> nice to hear their opinion before doing so.

It's line noise, and literally a redundant statement.  The same is true
of every other line in the file, and we don't write it because that
would be incredibly stupid.

Anyone who wants to see why this was chosen can read the commit message.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu May 30 17:23:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:23:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732900.1138945 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjUY-00061Q-Sn; Thu, 30 May 2024 17:23:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732900.1138945; Thu, 30 May 2024 17:23:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjUY-00061J-On; Thu, 30 May 2024 17:23:06 +0000
Received: by outflank-mailman (input) for mailman id 732900;
 Thu, 30 May 2024 17:23:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCjUX-000615-8f
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:23:05 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 45de0072-1ea9-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 19:23:04 +0200 (CEST)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-57a196134d1so1179822a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:23:04 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a653a15a60dsm170248166b.132.2024.05.30.10.23.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 10:23:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45de0072-1ea9-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717089784; x=1717694584; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Gxtv7RcWMMnRJ6POixtScnnqoGKFMSUUVO44xQ2i5/0=;
        b=ldNKa4x0puWBwkybF7X4scE9jlkjuspCMZBkZFfigIdibaaIb79DKamIMeU02viWh4
         SjzmdytBlynZ7NTvmV+mPeeG/dz0uNcWCjzUZS/tuFDRI13ICY++RxPxxB24F9ERGWni
         9Q/SFUPqEk7mWcmT2dNEwmeathSM08dpQL5xA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717089784; x=1717694584;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Gxtv7RcWMMnRJ6POixtScnnqoGKFMSUUVO44xQ2i5/0=;
        b=D29rxYiXqts5paLQbpXoVuNjoK8I60VgxGEMldX4+8fQbwtFKutWJwMGz8OPpdA4/s
         IrzLTE0EZzoBwjBEpHwWVc7ucth9v3uSrAxLXGC3MJK2xQvjamc13lo9AA4D8hzbySRP
         t7LInMvW1wDOhdqUqhTjqQzWhA+Q6ZzxwYIF5d5zckrgXREBy30O6Y1IsDZ29N7CAs9F
         lJU8AMwUu/ao3eMFf1ZEgt4sbkyRoXbZy65HkvdS2DEulKhy4qQREBTqBplBaVnmJMeL
         Buo4PPn07UdLxHt/Isg0rfgkDNrBKsQ/9cl1ze6+lTfqcYs1s+TXFkEV1OO13dOf/9yy
         8/iA==
X-Forwarded-Encrypted: i=1; AJvYcCWhWP1ut4cKDMtXK3Y/qtqTN8xbUSH0xCR9UGtTVeLlyQbo6nkVwtX6+dcBfEJWofZRGpskxWadbS0sn6OnN1xRajwtaIWKx9oRSyhyvA4=
X-Gm-Message-State: AOJu0YxBv3psbHSOOsziMXR5VZ7wBkthgfwVTKlYSH9YLmXlscNprqB1
	wd2AqQCBFUCObdEyuEo8gWNkDNfFI+61ZsX1P/WIH1qLPR/z5tABKXuOks2MKG8=
X-Google-Smtp-Source: AGHT+IFTpQItYLTPlIE3ApLJWAcfaNC/piKgXc05qVsSYxeVj7bWM/R/LiF8mD4a+hxVZxu+JXXkfw==
X-Received: by 2002:a17:906:d195:b0:a59:aa68:99a0 with SMTP id a640c23a62f3a-a65e8e5472emr153925966b.18.1717089783756;
        Thu, 30 May 2024 10:23:03 -0700 (PDT)
Message-ID: <70128dba-498f-4d85-8507-bb1621182754@citrix.com>
Date: Thu, 30 May 2024 18:23:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build
 full Xen
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <d00b86f41ef2c7d928a28dadd8c34fb845f23d0a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d00b86f41ef2c7d928a28dadd8c34fb845f23d0a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

This patch looks like it can go in independently?  Or does it depend on
having bitops.h working in practice?

However, one very strong suggestion...


> diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
> index 07c7a0abba..cc4a07a71c 100644
> --- a/xen/arch/riscv/include/asm/mm.h
> +++ b/xen/arch/riscv/include/asm/mm.h
> @@ -3,11 +3,246 @@
> <snip>
> +/* PDX of the first page in the frame table. */
> +extern unsigned long frametable_base_pdx;
> +
> +/* Convert between machine frame numbers and page-info structures. */
> +#define mfn_to_page(mfn)                                            \
> +    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
> +#define page_to_mfn(pg)                                             \
> +    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)

Do yourself a favour and not introduce frametable_base_pdx to begin with.

Every RISC-V board I can find has things starting from 0 in physical
address space, with RAM starting immediately after.

Taking the microchip board as an example, RAM actually starts at
0x8000000, which means that having frametable_base_pdx and assuming it
does get set to 0x8000 (which isn't even a certainty, given that I think
you'll need struct pages covering the PLICs), then what you are trading
off is:

* Saving 32k of virtual address space only (no need to even allocate
memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn
conversion, which is a screaming hotpath all over Xen.

It's a terribly short-sighted tradeoff.

32k of VA space might be worth saving in a 32bit build (I personally
wouldn't - especially as there's no need to share Xen's VA space with
guests, given no PV guests on ARM/RISC-V), but it's absolutely not at
all in an a 64bit build with TB of VA space available.

Even if we do find a board with the first interesting thing in the
frametable starting sufficiently away from 0 that it might be worth
considering this slide, then it should still be Kconfig-able in a
similar way to PDX_COMPRESSION.

You don't want to be penalising everyone because of a theoretical/weird
corner case.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:29:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:29:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732909.1138955 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjam-0006u8-I5; Thu, 30 May 2024 17:29:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732909.1138955; Thu, 30 May 2024 17:29:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjam-0006u1-FU; Thu, 30 May 2024 17:29:32 +0000
Received: by outflank-mailman (input) for mailman id 732909;
 Thu, 30 May 2024 17:29:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sCjal-0006tv-BU
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:29:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCjal-00043d-6p; Thu, 30 May 2024 17:29:31 +0000
Received: from [15.248.2.26] (helo=[10.24.67.18])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sCjal-00010w-0L; Thu, 30 May 2024 17:29:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=qXxktVINme67eJbU5IwzWEPPmpeaXo2Dihp5Mrl7cJo=; b=xUPA+ZHrgZcgp6EBAUQ3duPx/q
	vXxkNZc2aX4KSfUN9Xvkw29ALHfV4Xw33ipn95FSjvtL5z1Bsc2CRnj3dyi84BjiLJRGhMAY5RhZK
	ooxXoKdqp054Im401SJo8DP+E8QHA97SAsXYzkW+u9OzyOR+fesIAA38NONrtWB9S+fY=;
Message-ID: <e246d628-83c9-4021-b32b-3a30e0dfcf6a@xen.org>
Date: Thu, 30 May 2024 18:29:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org,
 Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
 <065c4b2f-93b6-428f-beca-b77e6f98cecc@citrix.com>
 <27148c63318ef78d542db7cc20373e3cb3836df3.camel@gmail.com>
 <8f37cb67-32ae-4e01-af60-f94af136698f@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <8f37cb67-32ae-4e01-af60-f94af136698f@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 30/05/2024 18:22, Andrew Cooper wrote:
> On 30/05/2024 6:16 pm, Oleksii K. wrote:
>> On Thu, 2024-05-30 at 17:47 +0100, Andrew Cooper wrote:
>>> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>>>> diff --git a/README b/README
>>>> index c8a108449e..30da5ff9c0 100644
>>>> --- a/README
>>>> +++ b/README
>>>> @@ -48,6 +48,10 @@ provided by your OS distributor:
>>>>         - For ARM 64-bit:
>>>>           - GCC 5.1 or later
>>>>           - GNU Binutils 2.24 or later
>>>> +      - For RISC-V 64-bit:
>>>> +        - GCC 12.2 or later
>>>> +        - GNU Binutils 2.39 or later
>>>> +          Older GCC and GNU Binutils would work, but this is not a
>>>> guarantee.
>>> This sentence isn't appropriate to live here.
>>>
>>> The commit message saying "this is what we run in CI" is perfectly
>>> good
>>> enough.
>>>
>>> With this dropped, Reviewed-by: Andrew Cooper
>>> <andrew.cooper3@citrix.com>.  Can fix on commt.
>> I am okay with dropping this sentence, but someone ( unfortunately I
>> don't remember who Jan? Julien? ) requested it, and I think it would be
>> nice to hear their opinion before doing so.

I don't think it was me :). I have no issue with dropping the line.

> 
> It's line noise, and literally a redundant statement.  The same is true
> of every other line in the file,

+1. If anyone wanted extra clarify, we could spell out at the beginning 
of the file that this is the versions are confident with.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:46:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:46:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732917.1138965 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjqp-0001vS-Pv; Thu, 30 May 2024 17:46:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732917.1138965; Thu, 30 May 2024 17:46:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjqp-0001vL-NK; Thu, 30 May 2024 17:46:07 +0000
Received: by outflank-mailman (input) for mailman id 732917;
 Thu, 30 May 2024 17:46:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCjqp-0001vF-7a
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:46:07 +0000
Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com
 [2607:f8b0:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7abd8beb-1eac-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 19:46:02 +0200 (CEST)
Received: by mail-oi1-x234.google.com with SMTP id
 5614622812f47-3c9cc681ee0so621606b6e.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:46:02 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ae4a7462a6sm455936d6.40.2024.05.30.10.45.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 10:45:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7abd8beb-1eac-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717091161; x=1717695961; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=8EyIqUxtnOINUrW9rFJ3KihtzaOkt2Vxdi5/JFaCyqk=;
        b=Ut7mwNtWcokbTdbZT0cQdKWGlfh/79o11wrpwYHQNFmcCTT3P1rFVzIdtoB37ZeRnM
         NusCXoWN3KloCiCRUvhKQhOriJM1br0H+/pTkdRIasih7T8hwKEtVBdXu2SoxSQmSHir
         xZ3rFPKAn22ok2j0JZztKqOGM5ZXDyKZFFAaI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717091161; x=1717695961;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8EyIqUxtnOINUrW9rFJ3KihtzaOkt2Vxdi5/JFaCyqk=;
        b=wylaC5D4JNxG9wEMFvTZbKN4uWy1WNY/5qiL8v2CTpcMOd4U0pZfSfciaRGhLL2tFR
         J10eW4QmHkZpZQLWq6g7FLTERaQgR0jYDTb3ki1IK61bFTVjRoBS7ZPtM3mLJcgLsUrq
         jtyEGe393MgP3bxHkSkr2Ex36AcUH+89g8joMCEBI1lnJD+1oxZ79KodNGQiNlaUCVSo
         aU2Lej9kKbipFASRFJmHcw0d+N96pT160vxDNfTdvXiSqSBeVrWoRsGTogjCsK0igfzJ
         NCLnk9lw8sv3vn3J8C+08/xNtWNU36IgYeSlw5cQfpBTCl80ULXxZon2YoqmsRKxsOXQ
         x1tQ==
X-Forwarded-Encrypted: i=1; AJvYcCU3hnVvUyXlWQAcaV2lHBNKSatij/1nqstuFJnVFW+DABH7nAcvTeYjBTI+pwUSK3o+j/Rislsyh4OZW6Bl0lRmes+3dEkjU79Ppj97i74=
X-Gm-Message-State: AOJu0YwXMnqd8xAf6oc/JiSqKFf7YulDOYQPQRt4BJ7lAwzS8CdoTavg
	9SCFG9sD3ixnp70GEp6Ufp31PTUjBFeoYQskCk5BkE6Fx2cYqNtOaU5rOuMC8rc=
X-Google-Smtp-Source: AGHT+IFxeKXYq48pYd3SIZXu8Mx/AM4laNIxC/SFNNFXYx1M4EibGPLEo/iX1PDG0PLIMDMD+BuS5w==
X-Received: by 2002:a05:6808:3010:b0:3c9:9473:4300 with SMTP id 5614622812f47-3d1dcd20f26mr3505299b6e.48.1717091160796;
        Thu, 30 May 2024 10:46:00 -0700 (PDT)
Message-ID: <61c227f6-9515-4bfb-932b-aa0a5d33bc66@citrix.com>
Date: Thu, 30 May 2024 18:45:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
 <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
 <2747a20d490252c279dd051c41a9b8dc69fb4c3e.camel@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <2747a20d490252c279dd051c41a9b8dc69fb4c3e.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 6:12 pm, Oleksii K. wrote:
> On Thu, 2024-05-30 at 17:48 +0100, Andrew Cooper wrote:
>> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>>> diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
>>> index 8285bcffef..bda35fc347 100644
>>> --- a/xen/arch/riscv/stubs.c
>>> +++ b/xen/arch/riscv/stubs.c
>>> @@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t,
>>> cpu_core_mask);
>>>  
>>>  nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
>>>  
>>> -/*
>>> - * max_page is defined in page_alloc.c which isn't complied for
>>> now.
>>> - * definition of max_page will be remove as soon as page_alloc is
>>> built.
>>> - */
>>> -unsigned long __read_mostly max_page;
>>> -
>>>  /* time.c */
>>>  
>>>  unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in
>>> kHz. */
>>> @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
>>>  {
>>>      BUG_ON("unimplemented");
>>>  }
>>> -
>>> -/*
>>> - * The following functions are defined in common/irq.c, but
>>> common/irq.c isn't
>>> - * built for now. These changes will be removed there when
>>> common/irq.c is
>>> - * ready.
>>> - */
>>> -
>>> -void cf_check irq_actor_none(struct irq_desc *desc)
>>> -{
>>> -    BUG_ON("unimplemented");
>>> -}
>>> -
>>> -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
>>> -{
>>> -    BUG_ON("unimplemented");
>>> -
>>> -    return 0;
>>> -}
>> All 3 of these are introduced in the previous patch and deleted again
>> here.  Looks like a rebasing accident.
> Not really.
>
> This was done to avoid build failures for RISC-V. If the HEAD is on the
> previous patch where these changes are introduced and then we just drop
> them, it will lead to a linkage error because these functions are
> defined in common/irq.c, which isn't built for RISC-V if the HEAD is on
> the previous patch:
>    /build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'

Nothing in the series touches entry.S, so I'm not sure what this is
complaining about.

The stub for get_upper_mfn_bound() references max_page, but it's only
used in a SYSCTL so you can avoid the problem with a BUG_ON().

BTW, you also don't need a return after a BUG_ON(). 
__builtin_unreachable() takes care of everything properly for you.


>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to
>    `irq_startup_none'
>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to
>    `irq_actor_none'
>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to
>    `irq_actor_none'
>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to
>    `irq_actor_none'
>    riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't
>    defined
>
> That is why these stubs were introduced in the previous patch (because
> common/irq.c isn't built at that moment) and are removed in this patch
> (since at the moment of this patch, common/irq.c is now being built).

These OTOH are a side effect of how no_irq_type works, which is
horrifying, and not something I can unsee.

I'll see about fixing it, because I really can't bare to leave it like
this...

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 17:47:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 17:47:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732921.1138975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjsW-0002St-5I; Thu, 30 May 2024 17:47:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732921.1138975; Thu, 30 May 2024 17:47:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCjsW-0002Sm-1J; Thu, 30 May 2024 17:47:52 +0000
Received: by outflank-mailman (input) for mailman id 732921;
 Thu, 30 May 2024 17:47:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCjsU-0002Sg-Qw
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 17:47:50 +0000
Received: from mail-oa1-x2a.google.com (mail-oa1-x2a.google.com
 [2001:4860:4864:20::2a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bae77918-1eac-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 19:47:49 +0200 (CEST)
Received: by mail-oa1-x2a.google.com with SMTP id
 586e51a60fabf-24d8b372911so617672fac.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 10:47:49 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43ff258039fsm256311cf.79.2024.05.30.10.47.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 10:47:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bae77918-1eac-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717091268; x=1717696068; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=8Ria0bPgRndThepF5tccflUZu7ASqbAJHvzyfzmhvuQ=;
        b=TRFDtOPULrVLIXvenbLRqYl3aVMyZjwKZd5t5gIWb3tOFhYhgVN8u0HIcmRHA7rcrq
         IC2A/HDk5t+9ZRFWFrODLd4aeYGJkvaZPtcrqzlrFp15piNV/WtuSr/foyYwYLeW1f9n
         xolcwDIofBl/Klx/IWpoO41kvZo4KhknBTG5A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717091268; x=1717696068;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8Ria0bPgRndThepF5tccflUZu7ASqbAJHvzyfzmhvuQ=;
        b=O9iUGj2+werfnw3UPApvIzAeB97FmL/bkfUSX/NPEWn1i0kWMYBEWnbjFKHjveFWy+
         cvbYuvc01oOi+R0L/kDS2yAO/MtwewtNL8nWNAtsatWdr7Rt6GLXVjYgJdxNQKoNw6YH
         DoIUbUGU2lMb80LsTDH//pkCg6ylaOdXrYcgdOE/26v38drwL6Gs9KV8H4LNdMJYEXm3
         y6o7kIxvjY/+xJTQys2h5ph70tulzt4BhND7bE/7T870N1h7UTASTILe0t2xuZVrf7ex
         rSyEqhX3YkC36DyK5IIdkLUYWpGcDZ1uB3yZ86LOju2z32RbhXUY0qZWbKb11NSzzTp0
         p9/w==
X-Forwarded-Encrypted: i=1; AJvYcCW26N1DQgDi6DwoFHv3uJnJMiyObPmpkej7rEN1bkTlFwuGdjao+uzGaN0aKTwj3mRaFQPbiZE1Gi5lF0KYWq93Wu/YBJ/pzYAzyzb45oU=
X-Gm-Message-State: AOJu0YzhzaVhLAXCQNlrnhjs5oNrEh6nDdKk5NsyNIvfYrhsxHrgiunQ
	4UVCey8f42JwxX/BTZ5rE5FOKL2147ZEAeCHmx01qr6Per4LuqdRfd1/HvgE+Vc=
X-Google-Smtp-Source: AGHT+IFHYuzj514kDmVKOdpM4ERsPRcRWRYcWbNG4R4lYTwq5eA1Vvep9lW5KdOdMugMCuEONJUM0A==
X-Received: by 2002:a05:6870:d68f:b0:24f:d57f:86ad with SMTP id 586e51a60fabf-25060b015c3mr3065166fac.8.1717091268426;
        Thu, 30 May 2024 10:47:48 -0700 (PDT)
Message-ID: <0c667b0f-7469-4c38-a16b-f8d62e894a98@citrix.com>
Date: Thu, 30 May 2024 18:47:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 6/8] xen/riscv: add minimal amount of stubs to build
 full Xen
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <4e5b814c3f73bd4ae6bb80296e17ac80bd8d224a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <4e5b814c3f73bd4ae6bb80296e17ac80bd8d224a.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
> index fe3a43be20..2c3fb7d72e 100644
> --- a/xen/arch/riscv/mm.c
> +++ b/xen/arch/riscv/mm.c
> @@ -1,5 +1,6 @@
>  /* SPDX-License-Identifier: GPL-2.0-only */
>  
> +#include <xen/bug.h>
>  #include <xen/cache.h>
>  #include <xen/compiler.h>
>  #include <xen/init.h>
> @@ -14,6 +15,9 @@
>  #include <asm/page.h>
>  #include <asm/processor.h>
>  
> +unsigned long __ro_after_init frametable_base_pdx;
> +unsigned long __ro_after_init frametable_virt_end;

Along with removing base_pdx, virt_end appears to be unique to ARM
before and after the series.  I'd recommend dropping it too.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 18:08:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:08:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732935.1138985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkC5-000695-Pp; Thu, 30 May 2024 18:08:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732935.1138985; Thu, 30 May 2024 18:08:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkC5-00068y-NG; Thu, 30 May 2024 18:08:05 +0000
Received: by outflank-mailman (input) for mailman id 732935;
 Thu, 30 May 2024 18:08:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCkC4-00068o-Uy; Thu, 30 May 2024 18:08:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCkC4-0004lQ-QV; Thu, 30 May 2024 18:08:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCkC4-0008Ob-Ec; Thu, 30 May 2024 18:08:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCkC4-0003Uq-E9; Thu, 30 May 2024 18:08:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BOLkw+gxkvO0g2d5oEpMV3M3H6auMFwy7trDeVt63iI=; b=uI9tvISnObmcCKsxGMn9uFg668
	qAq5HJ1PZmARv5yO8JhgXC0rkvMzREB3DYzUhT1AXV/RIzgbtpFxKr9M9H6X7C5xPjMUSq+iYsR4b
	mtaLsW5RkhMWz86zXN5IWCocBEV6sBcFPKK9FWckR8+GL31amZYGn29zYftJUNwjQgLw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186194-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186194: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
X-Osstest-Versions-That:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 18:08:04 +0000

flight 186194 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186194/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186186
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186186
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186186
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186186
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186186
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186186
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d
baseline version:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d

Last test of basis   186194  2024-05-30 08:11:21 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu May 30 18:22:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:22:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732944.1138994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkPq-0000zV-UX; Thu, 30 May 2024 18:22:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732944.1138994; Thu, 30 May 2024 18:22:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkPq-0000zO-S3; Thu, 30 May 2024 18:22:18 +0000
Received: by outflank-mailman (input) for mailman id 732944;
 Thu, 30 May 2024 18:22:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCkPp-0000zI-96
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:22:17 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8a1a5363-1eb1-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 20:22:14 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e95a883101so15651731fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:22:14 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d34b44sm34711e87.47.2024.05.30.11.22.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 11:22:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a1a5363-1eb1-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717093334; x=1717698134; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Gbgfy0IeTG3caifn22RgpWb2FS+8d3sBHn20PALzr2Y=;
        b=BcIQW3R+N8kmQMeuhete06E/VMLi8mQPM96ee1efhusEhuR5oPtRrfv1y1GkOC0Jnw
         ibUKERCI5pQe2SeX/kiFVdu09iu0GKw3PvRkNSeNLLmHk5mG4UZxYZaVfp/8sFsSmU3v
         jINl24bL3CiJUloEmUnJwVbQtAt8Gl1m9Q9P03HGb6K3zxALL3sYCTLojfXdsrxfJjRd
         JtGcH8cl9pO3Ub+WA76tNAaujCPZ5P+PttoyUBtz+ngehsgQJJ77IlCCVjjw1VkC2DGm
         BFBfMKlql5GqgCY0lGNH8AxKz8zWG0loz3PhcJV27ppqypX/koEhYnW7X8m9/vrs6GBW
         jDkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717093334; x=1717698134;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Gbgfy0IeTG3caifn22RgpWb2FS+8d3sBHn20PALzr2Y=;
        b=HXkD9k6b21DNrTaN3NetdRZsfifU1NU4mEAIIVPZwUu1lIFWSEmew2MeeCUcNhw1s6
         /D1wpx8lZsCJqT1sDInaxc5TNT9mWxZwDnkl/vXelTkgFFHqwpEWnTzD/aKU/OuZpur8
         gUlc1OALFNFIwArKCOvJRjPcJjfR2bX862ctm579V7uCVuYhHSZ4K6gu1Gp5xCTGL7A8
         pEyOVkaad2GiF1RebROOArB7t6huy75IUWL4/dr8h9ASjACwg2fJ+jlgdFMGce/SC4uV
         wi8Xhnn1J7BcF23v1CtfbvQ5AFr0+wG/asEyW8Ozmzr2upQKjV+iQG0BdSUxJFxyxgKh
         6Mxg==
X-Forwarded-Encrypted: i=1; AJvYcCVPaQS3GNdp30KpJXJNcf3YPJdGomOdUr79L15QiVlM9Im2hrx4pHA2P4oHZUcB3UMm8EnC1ez9ADZRXIKLCdj/uPi/04m6AqNgaI0aKkI=
X-Gm-Message-State: AOJu0YyJ7zRP/AZl+Wuk/3CcsMX/ixzT65BFIeVXBXG6FYH5yFZoSk+H
	z9wtFTnk6unc1g32TsnsEDShk/dsZ5/m7UiGlOJRaODBBjRp4EjD
X-Google-Smtp-Source: AGHT+IH+Jv246nTz3BmjhX4ngAvAPZgSFMIFXNCD4MCgktBeKti0UCEVEkgUjnZDxWP8059wx1/8vA==
X-Received: by 2002:ac2:5217:0:b0:52b:7a10:3537 with SMTP id 2adb3069b0e04-52b7d47a0e8mr1783402e87.54.1717093334008;
        Thu, 30 May 2024 11:22:14 -0700 (PDT)
Message-ID: <7721c1b4eb0ea76cca5460264954d40d639499b7.camel@gmail.com>
Subject: Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build
 full Xen
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Date: Thu, 30 May 2024 20:22:12 +0200
In-Reply-To: <70128dba-498f-4d85-8507-bb1621182754@citrix.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
	 <d00b86f41ef2c7d928a28dadd8c34fb845f23d0a.1717008161.git.oleksii.kurochko@gmail.com>
	 <70128dba-498f-4d85-8507-bb1621182754@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 18:23 +0100, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > Acked-by: Jan Beulich <jbeulich@suse.com>
>=20
> This patch looks like it can go in independently?=C2=A0 Or does it depend
> on
> having bitops.h working in practice?
>=20
> However, one very strong suggestion...
>=20
>=20
> > diff --git a/xen/arch/riscv/include/asm/mm.h
> > b/xen/arch/riscv/include/asm/mm.h
> > index 07c7a0abba..cc4a07a71c 100644
> > --- a/xen/arch/riscv/include/asm/mm.h
> > +++ b/xen/arch/riscv/include/asm/mm.h
> > @@ -3,11 +3,246 @@
> > <snip>
> > +/* PDX of the first page in the frame table. */
> > +extern unsigned long frametable_base_pdx;
> > +
> > +/* Convert between machine frame numbers and page-info structures.
> > */
> > +#define
> > mfn_to_page(mfn)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 (frame_table + (mfn_to_pdx(mfn) - frametable_base_p=
dx))
> > +#define
> > page_to_mfn(pg)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 pdx_to_mfn((unsigned long)((pg) - frame_table) +
> > frametable_base_pdx)
>=20
> Do yourself a favour and not introduce frametable_base_pdx to begin
> with.
>=20
> Every RISC-V board I can find has things starting from 0 in physical
> address space, with RAM starting immediately after.
I checked Linux kernel and grep there:
   [ok@fedora linux-aia]$ grep -Rni "memory@" arch/riscv/boot/dts/ --
   exclude "*.tmp" -I
   arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33:   =
=20
   memory@40000000 {
   arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28:     memory@80000000
   {
   arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49:      ddrc_cache:
   memory@1000000000 {
   arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33:   ddrc_cache_lo:
   memory@80000000 {
   arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37:   ddrc_cache_hi:
   memory@1040000000 {
   arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34:      ddrc_cache_lo:
   memory@80000000 {
   arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40:      ddrc_cache_hi:
   memory@1000000000 {
   arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22:   ddrc_cache_lo:
   memory@80000000 {
   arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27:   ddrc_cache_hi:
   memory@1000000000 {
   arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57:   ddrc_cache_lo:
   memory@80000000 {
   arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63:   ddrc_cache_hi:
   memory@1040000000 {
   arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32:  memory@0 {
   arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14:    =20
   memory@0 {
   arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26:    memory@80000000
   {
   arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12:     memory@80000000 {
   arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26: memory@80000000
   {
   arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25: memory@80000000
   {
   arch/riscv/boot/dts/canaan/k210.dtsi:82:        sram: memory@80000000 {
  =20
And based on  that majority of supported by Linux kernel boards has RAM
starting not from 0 in physical address space. Am I confusing
something?

>=20
> Taking the microchip board as an example, RAM actually starts at
> 0x8000000,
Today we had conversation with the guy from SiFive in xen-devel channel
and he mentioned that they are using "starfive visionfive2 and sifive
unleashed platforms." which based on the grep above has RAM not at 0
address.

Also, QEMU uses 0x8000000.

>  which means that having frametable_base_pdx and assuming it
> does get set to 0x8000 (which isn't even a certainty, given that I
> think
> you'll need struct pages covering the PLICs), then what you are
> trading
> off is:
>=20
> * Saving 32k of virtual address space only (no need to even allocate
> memory for this range of the framtable), by
> * Having an extra memory load and add/sub in every page <-> mfn
> conversion, which is a screaming hotpath all over Xen.
>=20
> It's a terribly short-sighted tradeoff.
>=20
> 32k of VA space might be worth saving in a 32bit build (I personally
> wouldn't - especially as there's no need to share Xen's VA space with
> guests, given no PV guests on ARM/RISC-V), but it's absolutely not at
> all in an a 64bit build with TB of VA space available.
>=20
> Even if we do find a board with the first interesting thing in the
> frametable starting sufficiently away from 0 that it might be worth
> considering this slide, then it should still be Kconfig-able in a
> similar way to PDX_COMPRESSION.
I found your tradeoffs reasonable, but I don't understand how it will
work if RAM does not start from 0, as the frametable address and RAM
address are linked.
I tried to look at the PDX_COMPRESSION config and couldn't find any
"slide" there. Could you please clarify this for me?
If to use this "slide" would it help to avoid the mentioned above
tradeoffs?

One more question: if we decide to go without frametable_base_pdx,
would it be sufficient to simply remove mentions of it from the code (
at least, for now )?

~ Oleksii
>=20
> You don't want to be penalising everyone because of a
> theoretical/weird
> corner case.


>=20
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Thu May 30 18:27:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:27:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732949.1139005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkUS-0001ZT-G8; Thu, 30 May 2024 18:27:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732949.1139005; Thu, 30 May 2024 18:27:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkUS-0001ZM-DS; Thu, 30 May 2024 18:27:04 +0000
Received: by outflank-mailman (input) for mailman id 732949;
 Thu, 30 May 2024 18:27:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCkUR-0001Xj-BD
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:27:03 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 357c51f4-1eb2-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 20:27:02 +0200 (CEST)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2e72224c395so10996641fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:27:02 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d89485sm32279e87.254.2024.05.30.11.27.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 11:27:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 357c51f4-1eb2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717093622; x=1717698422; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=JO7Vprcr74OLOAx5MFxE/wYQ+52Z1NDEYUh3yhUtyds=;
        b=XH5VEJ9QfFpXqhOkewZ9/4wWPthqri+tl4jGPaJs5I5+/FKmp3JghNcZZOaNIX2C5D
         DER0EzAPO9pIj6XSRD4V+Wb7/4HrzWVm1Q7Eppfkp83fUpICjtJj+v2HB5aqTfN4yJJ9
         vYjPaw25IINfLQlHx6bq8U9rVxRKB1hRYNIkVj5Kv5WqDbwEIPBzw4sy6vdg0O5EU7oH
         YEhdKlk4H9lus512qQt9+qJp3Lfizr+t02E7UhOLT+Cw/ObArI4/YY4PGx9rpi2WPEwS
         1SC0MUpE7lwGyiGZpesm1AJvhpUM0annCspQT+QxQ+mfH6qZoSc392XqHgJq9Il76n9m
         gcFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717093622; x=1717698422;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JO7Vprcr74OLOAx5MFxE/wYQ+52Z1NDEYUh3yhUtyds=;
        b=M5/4YPjnH4R6xwbKUCPu2zE4XX4DV/pVG97pUz1d48ZYKuD40+NNIPrnD7TJHiExys
         Db6akv6x2wqPGsFM8jNB5vtKq3kEgQ36zk4DLWMJIMKCiUH/STgs1IGJwEWlo24AIqRv
         kwJWb5PwnfnnpXaooc7qATlx2tJW1nDV2GKgdB2f06z69PplTfNIxOvzgrZ3b4/itWy/
         49FBQImPMdEiIu9ZokOEAe4w78pv/rzjncclv2kYhJh9gNNknR2+0CyAFG1+63PlWo7R
         c3i+Qk8WJVRtzbQTDMX9YwGlDYzoXfFdSJaexqYwM7InAoRnZeY29sgfvNFC28ZFDtDh
         7rIQ==
X-Forwarded-Encrypted: i=1; AJvYcCWoEbBAfAbEaUQvPVmAqJvA8YROFY0ngrrTHzfEbmNe0jXq4xlhhmZsUDnVc/r+EvEPFb3DtLIH748Y/L+/zG8DuSaIF4q/lg9xQtedDGY=
X-Gm-Message-State: AOJu0Yww+6Bh67XLsQg4pArsyDk6TJ0wKYxjqOvFp28dJOXf2Ftf2f58
	6Cd9dpLf0JiKQKu6GbrxFEETlIWaiWW/pF7BPEK0dFBhdnAXowTb
X-Google-Smtp-Source: AGHT+IELFMT8Jpq+yczGXk2pdKBr0KHXOEFr2vdN1VNKLGjt4omHsnA3LtdOZyXLNztdwi/sh4PorA==
X-Received: by 2002:ac2:4c2f:0:b0:51e:2a1d:6654 with SMTP id 2adb3069b0e04-52b7d43aeefmr1648061e87.31.1717093621584;
        Thu, 30 May 2024 11:27:01 -0700 (PDT)
Message-ID: <934dbe6381e646d551e7f815aacb415e87f0210d.camel@gmail.com>
Subject: Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Date: Thu, 30 May 2024 20:27:00 +0200
In-Reply-To: <61c227f6-9515-4bfb-932b-aa0a5d33bc66@citrix.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
	 <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
	 <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
	 <2747a20d490252c279dd051c41a9b8dc69fb4c3e.camel@gmail.com>
	 <61c227f6-9515-4bfb-932b-aa0a5d33bc66@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 18:45 +0100, Andrew Cooper wrote:
> On 30/05/2024 6:12 pm, Oleksii K. wrote:
> > On Thu, 2024-05-30 at 17:48 +0100, Andrew Cooper wrote:
> > > On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> > > > diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
> > > > index 8285bcffef..bda35fc347 100644
> > > > --- a/xen/arch/riscv/stubs.c
> > > > +++ b/xen/arch/riscv/stubs.c
> > > > @@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t,
> > > > cpu_core_mask);
> > > > =C2=A0
> > > > =C2=A0nodemask_t __read_mostly node_online_map =3D { { [0] =3D 1UL =
} };
> > > > =C2=A0
> > > > -/*
> > > > - * max_page is defined in page_alloc.c which isn't complied
> > > > for
> > > > now.
> > > > - * definition of max_page will be remove as soon as page_alloc
> > > > is
> > > > built.
> > > > - */
> > > > -unsigned long __read_mostly max_page;
> > > > -
> > > > =C2=A0/* time.c */
> > > > =C2=A0
> > > > =C2=A0unsigned long __ro_after_init cpu_khz;=C2=A0 /* CPU clock fre=
quency
> > > > in
> > > > kHz. */
> > > > @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
> > > > =C2=A0{
> > > > =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > > > =C2=A0}
> > > > -
> > > > -/*
> > > > - * The following functions are defined in common/irq.c, but
> > > > common/irq.c isn't
> > > > - * built for now. These changes will be removed there when
> > > > common/irq.c is
> > > > - * ready.
> > > > - */
> > > > -
> > > > -void cf_check irq_actor_none(struct irq_desc *desc)
> > > > -{
> > > > -=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > > > -}
> > > > -
> > > > -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
> > > > -{
> > > > -=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > > > -
> > > > -=C2=A0=C2=A0=C2=A0 return 0;
> > > > -}
> > > All 3 of these are introduced in the previous patch and deleted
> > > again
> > > here.=C2=A0 Looks like a rebasing accident.
> > Not really.
> >=20
> > This was done to avoid build failures for RISC-V. If the HEAD is on
> > the
> > previous patch where these changes are introduced and then we just
> > drop
> > them, it will lead to a linkage error because these functions are
> > defined in common/irq.c, which isn't built for RISC-V if the HEAD
> > is on
> > the previous patch:
> > =C2=A0=C2=A0 /build/xen/arch/riscv/entry.S:86: undefined reference to
> > `max_page'
>=20
> Nothing in the series touches entry.S, so I'm not sure what this is
> complaining about.
>=20
> The stub for get_upper_mfn_bound() references max_page, but it's only
> used in a SYSCTL so you can avoid the problem with a BUG_ON().
I didn't get how I can use BUG_ON() with declaration of variable to
avoid the compilation issue the undefined reference to max_page?

>=20
> BTW, you also don't need a return after a BUG_ON().=C2=A0
> __builtin_unreachable() takes care of everything properly for you.
>=20
>=20
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined
> > reference to
> > =C2=A0=C2=A0 `irq_startup_none'
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined
> > reference to
> > =C2=A0=C2=A0 `irq_actor_none'
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined
> > reference to
> > =C2=A0=C2=A0 `irq_actor_none'
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined
> > reference to
> > =C2=A0=C2=A0 `irq_actor_none'
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_n=
one'
> > isn't
> > =C2=A0=C2=A0 defined
> >=20
> > That is why these stubs were introduced in the previous patch
> > (because
> > common/irq.c isn't built at that moment) and are removed in this
> > patch
> > (since at the moment of this patch, common/irq.c is now being
> > built).
>=20
> These OTOH are a side effect of how no_irq_type works, which is
> horrifying, and not something I can unsee.
>=20
> I'll see about fixing it, because I really can't bare to leave it
> like
> this...
I am really sorry, but I don't understand should I deal with this
somehow now or the provided changes are okay?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 30 18:40:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:40:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732957.1139015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhW-0004bY-Q4; Thu, 30 May 2024 18:40:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732957.1139015; Thu, 30 May 2024 18:40:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhW-0004bR-MC; Thu, 30 May 2024 18:40:34 +0000
Received: by outflank-mailman (input) for mailman id 732957;
 Thu, 30 May 2024 18:40:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCkhV-0004bL-4Y
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:40:33 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1826dad5-1eb4-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 20:40:32 +0200 (CEST)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5789733769dso2315600a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:40:32 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67e6f02d29sm4086666b.14.2024.05.30.11.40.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 11:40:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1826dad5-1eb4-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717094431; x=1717699231; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=JJuNoSq0OuQqzdgRdtc7jFxhjSPmfyd/fXrf/kWjNhQ=;
        b=Sag8l9CZTqN8yLky+ThHtbuWbWuKheSZAG59ey80tQCGT+ZQ48u5QD+Rh3isGE8/2Z
         sM8jKYjqnj3ue2WfUC9BLI5BL+QFjGSrh4l0oM6uTX06veRE2qMZpENd5lXKjfh6Ps2q
         ycRD176PW3/w/VA5QZ3HnzmdYimxzS/V/cWfU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717094431; x=1717699231;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=JJuNoSq0OuQqzdgRdtc7jFxhjSPmfyd/fXrf/kWjNhQ=;
        b=anMg3vf7i83lwoIx5PKYKIP/WwiuTot58QMOSgGkG3qikyZR5P9stZwm7Z58KQeq6C
         5eZj+SJXKSRHlWgPnDQkpXW16cHrG6GgOkii8Gy94+jzcEFZHOfJCJYFf464Fapry0L9
         1AGN+zgPEeWmX3mNPG1nbj2obrAi7m90KLf7kUemUQV989Nk0vi64W3PJfI/m/60++n2
         uOHbr9EqAARtRv7SApBlt7Xmg0DZjdZBm7y76/JTnMkQo2gbHlQmkSyOqIxQFM7j8aVR
         3Gxf1HToLNDzurywVPTMk8zJVdmUJ72Ra+P2aHcXVvz0Bq5qQUxngxPeyszgtOEBFDNB
         fbvw==
X-Gm-Message-State: AOJu0Yxbbq5DYw+WVeWREkGNPBpzjzZKUMi28VYa6oLsKqTv4NTnfAE/
	7RsVUCeuHavcEos4bjkeBNQPlMcB8eCnBtWbN/b/x7xKMSDaRjK0CFI3KTJDrC1nmtWF0TqlLsX
	i
X-Google-Smtp-Source: AGHT+IFM4sUCoWuD618J0Ddv1WsGtE5ssj5fZDr9xCzIqkDsGm94uJZcQrb+haYSG/2G2rsH7I82sQ==
X-Received: by 2002:a17:906:1c41:b0:a5a:8cc0:8c23 with SMTP id a640c23a62f3a-a65f0bd69cemr233518966b.27.1717094431322;
        Thu, 30 May 2024 11:40:31 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH for-4.19 0/2] arch/irq: Untangle no_irq_type
Date: Thu, 30 May 2024 19:40:25 +0100
Message-Id: <20240530184027.44609-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Found when reviewing Oleksii's series to enable the RISC-V build.

The way no_irq_type works is horrifying.  Make it less-so.

Andrew Cooper (2):
  arch/irq: Make irq_ack_none() mandatory
  arch/irq: Centralise no_irq_type

 xen/arch/arm/include/asm/irq.h |  3 +++
 xen/arch/arm/irq.c             | 14 ++------------
 xen/arch/ppc/stubs.c           | 11 ++++-------
 xen/arch/x86/irq.c             | 11 +----------
 xen/common/irq.c               | 13 +++++++++++++
 xen/include/xen/irq.h          |  8 +++++++-
 6 files changed, 30 insertions(+), 30 deletions(-)


base-commit: 9a905d7dc65883af082532b4dc91ce0131e54047
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 30 18:40:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:40:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732958.1139025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhZ-0004rK-1q; Thu, 30 May 2024 18:40:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732958.1139025; Thu, 30 May 2024 18:40:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhY-0004rD-Ty; Thu, 30 May 2024 18:40:36 +0000
Received: by outflank-mailman (input) for mailman id 732958;
 Thu, 30 May 2024 18:40:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCkhX-0004ea-6x
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:40:35 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 18b2f88d-1eb4-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 20:40:33 +0200 (CEST)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-57a30dbdb7fso218522a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:40:33 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67e6f02d29sm4086666b.14.2024.05.30.11.40.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 11:40:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18b2f88d-1eb4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717094432; x=1717699232; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/5QMwuJU8AJ5lyqy+M0BkxbHCvu9t37ti0og1vGBBFw=;
        b=RPST3wXfWYZSMbmtGkpA+4O4NhPDK+Z3gVW/vdGO3z7JQEonVDnKO5IprFrQtLFphD
         neve3yJnhkQtxRHMbcNhLNdy69OaULi1QDXIhMqjHqJbWkNIZUgrKEiqdQ+IpV9oxikS
         fzM//Xs8iuUr9gaEkfEQGzqn9tv1IJuwpJALA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717094432; x=1717699232;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/5QMwuJU8AJ5lyqy+M0BkxbHCvu9t37ti0og1vGBBFw=;
        b=PUJ39ZjnySZx7F3WSbsvlLqAR/K2BGFcUHl84C21HQM27/4aRvBG+xCDfHTXprla7E
         AUGHxGkL1veELJTjTJ+avw65cxVW0pyyfPz9nTDTQIctrCHSA8oNn1ro5DDhQ0xQYFmy
         lqpn4Pijudf5UriSA+a8uru8Tf/NBUtZ2qJKsxkImWL+FO/ZPRjsThFjRYwG+Lvvljda
         c8xzHsFjuAcMLIssOxDf1wp4Se9/n1txKnNbEz9VrMNz9RYlmQFQ96FA5HruJT8XH1HW
         27y+fOUdAAAD0+5iDNQB3R8pX/02MlYen3ml9kDpJKq8+8SWIwwZCqXworf/8sLgzjKD
         lnjg==
X-Gm-Message-State: AOJu0YzvS2mLynBZyE3INJstX7i0vYor1Ava9dkVo0FEP6RW3yPpXdzI
	kR9olxlPqeOGbjN4L9At77l57VcOzLfvuTefdpAxhwI0exAf7ZAbkfC8A/iGDqej86qbgQxkAzk
	d
X-Google-Smtp-Source: AGHT+IETGLUBFNDb8N54iW+kGM7XqF5hjLpboxoNZIpw3z1Xr11N+iJwjq9qdxtheDVhcDAuuhaTOA==
X-Received: by 2002:a17:906:4f8c:b0:a59:a18e:3fda with SMTP id a640c23a62f3a-a65e92152e1mr208527366b.68.1717094432361;
        Thu, 30 May 2024 11:40:32 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH 1/2] arch/irq: Make irq_ack_none() mandatory
Date: Thu, 30 May 2024 19:40:26 +0100
Message-Id: <20240530184027.44609-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240530184027.44609-1-andrew.cooper3@citrix.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Any non-stub implementation of these is going to have to do something here.

irq_end_none() is more complicated and has arch-specific interactions with
irq_ack_none(), so make it optional.

For PPC, introduce a stub irq_ack_none().

For ARM and x86, export the existing {ack,end}_none() helpers, gaining an irq_
prefix for consisntency with everything else in no_irq_type.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>
---
 xen/arch/arm/include/asm/irq.h | 3 +++
 xen/arch/arm/irq.c             | 8 ++++----
 xen/arch/ppc/stubs.c           | 6 ++++++
 xen/arch/x86/irq.c             | 4 ++--
 xen/include/xen/irq.h          | 6 ++++++
 5 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
index 1bae5388878e..ec437add0971 100644
--- a/xen/arch/arm/include/asm/irq.h
+++ b/xen/arch/arm/include/asm/irq.h
@@ -98,6 +98,9 @@ void irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask);
  */
 bool irq_type_set_by_domain(const struct domain *d);
 
+void irq_end_none(struct irq_desc *irq);
+#define irq_end_none irq_end_none
+
 #endif /* _ASM_HW_IRQ_H */
 /*
  * Local variables:
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index bcce80a4d624..7138f9e7c283 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,12 +31,12 @@ struct irq_guest
     unsigned int virq;
 };
 
-static void ack_none(struct irq_desc *irq)
+void irq_ack_none(struct irq_desc *irq)
 {
     printk("unexpected IRQ trap at irq %02x\n", irq->irq);
 }
 
-static void end_none(struct irq_desc *irq)
+void irq_end_none(struct irq_desc *irq)
 {
     /*
      * Still allow a CPU to end an interrupt if we receive a spurious
@@ -51,8 +51,8 @@ hw_irq_controller no_irq_type = {
     .shutdown = irq_shutdown_none,
     .enable = irq_enable_none,
     .disable = irq_disable_none,
-    .ack = ack_none,
-    .end = end_none
+    .ack = irq_ack_none,
+    .end = irq_end_none
 };
 
 static irq_desc_t irq_desc[NR_IRQS];
diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
index da193839bd09..4e03428e071a 100644
--- a/xen/arch/ppc/stubs.c
+++ b/xen/arch/ppc/stubs.c
@@ -134,12 +134,18 @@ void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
     BUG_ON("unimplemented");
 }
 
+void irq_ack_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
 hw_irq_controller no_irq_type = {
     .typename = "none",
     .startup = irq_startup_none,
     .shutdown = irq_shutdown_none,
     .enable = irq_enable_none,
     .disable = irq_disable_none,
+    .ack = irq_ack_none,
 };
 
 int arch_init_one_irq_desc(struct irq_desc *desc)
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index c16205a9beb6..cfd7a08479d2 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -467,7 +467,7 @@ int __init init_irq_data(void)
     return 0;
 }
 
-static void cf_check ack_none(struct irq_desc *desc)
+void cf_check irq_ack_none(struct irq_desc *desc)
 {
     ack_bad_irq(desc->irq);
 }
@@ -478,7 +478,7 @@ hw_irq_controller no_irq_type = {
     irq_shutdown_none,
     irq_enable_none,
     irq_disable_none,
-    ack_none,
+    irq_ack_none,
 };
 
 static vmask_t *irq_get_used_vector_mask(int irq)
diff --git a/xen/include/xen/irq.h b/xen/include/xen/irq.h
index 89f7a8317a87..b71f65db8621 100644
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -130,6 +130,12 @@ void cf_check irq_actor_none(struct irq_desc *desc);
 #define irq_disable_none irq_actor_none
 #define irq_enable_none irq_actor_none
 
+/*
+ * irq_ack_none() must be provided by the architecture.
+ * irq_end_none() is optional, and opted into using a define.
+ */
+void irq_ack_none(struct irq_desc *irq);
+
 /*
  * Per-cpu interrupted context register state - the inner-most interrupt frame
  * on the stack.
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 30 18:40:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:40:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732959.1139032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhZ-0004v6-C0; Thu, 30 May 2024 18:40:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732959.1139032; Thu, 30 May 2024 18:40:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkhZ-0004tc-57; Thu, 30 May 2024 18:40:37 +0000
Received: by outflank-mailman (input) for mailman id 732959;
 Thu, 30 May 2024 18:40:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCkhX-0004ea-TO
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:40:35 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 195f3364-1eb4-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 20:40:34 +0200 (CEST)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a6266ffdba8so94041966b.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:40:34 -0700 (PDT)
Received: from andrew-laptop.citrite.net ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67e6f02d29sm4086666b.14.2024.05.30.11.40.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 11:40:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 195f3364-1eb4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717094433; x=1717699233; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JuATIEzMZcMO0sma4DN6PPyKFK9rWyN48A84jE2N8B4=;
        b=dVMD198jeziMqpU+HVNRBi0zdyMUDQ/QNvF11a0Pi/xmoTFpXHcy13BZdMwjToKyP/
         ITiJ9AxlN3Beub9z4ZMv384DzTuqls09lAFw5ydb8vawHnrk/ct6ZPjK+Y71iM7BgN1j
         hB5SrCLNCtgFCdvbupmH1tF4orj7znBaEUaRA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717094433; x=1717699233;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JuATIEzMZcMO0sma4DN6PPyKFK9rWyN48A84jE2N8B4=;
        b=H37o9fCFKDHEuRErTBIhpDKDrNQhukbW3UWe3zUrSCjMTfEfmvNgp6JTaw3GfADMqi
         SrOfrTqa343BhA8zW52pBB0ME1Fcj+tdZ4yavdwdrvnEq8QPgTwxhb/D05KDDGwT/MQT
         VLPvjClc+uBBrCJd0xNFtaE8qjUYR3uEdWa8FJj37LvQV9aBJVw7tQX4tgqBGQgPWGB9
         Ad449JJxiH6MVS/cqYBAArLI+hMlkICEk4Fo/1AfI91A7r5EHbuwx8GZLR5GhOF6v8bU
         Hd1lgEi+yEEMvqXyli8UsoCCLnCmdQ0LvAd9xJ6uHbnFespUWA9Ova5+cJMVt/8/ux0x
         SbPw==
X-Gm-Message-State: AOJu0YzxupP51+uTbcgrPBEYTWOU/q7WtqYbyNpo4Nsk3c6EzrFBDRc8
	1f809OjIZWUW2lNE2mZQGrNHsoDn1qEXips5mzdyBIDlivtrPC8JsgUl6T6rKC+5NLPvP1KrCzY
	T
X-Google-Smtp-Source: AGHT+IHI7hSinh5KOPtR5MaVfNKKi6KsCTPQWGoxfcps63uJaoA2hCPyH6LCOpUNr5QWhTyaxvVRbw==
X-Received: by 2002:a17:906:7f8e:b0:a62:a63c:18f0 with SMTP id a640c23a62f3a-a65e8d12070mr192357266b.1.1717094433363;
        Thu, 30 May 2024 11:40:33 -0700 (PDT)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH 2/2] arch/irq: Centralise no_irq_type
Date: Thu, 30 May 2024 19:40:27 +0100
Message-Id: <20240530184027.44609-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240530184027.44609-1-andrew.cooper3@citrix.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Having no_irq_type defined per arch, but using common callbacks is a mess, and
particualrly hard to bootstrap a new architecture with.

Now that the ack()/end() hooks have been exported suitably, move the
definition of no_irq_type into common/irq.c, and into .rodata for good
measure.

No functional change, but a whole lot less tangled.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>

Oleksii: For RISC-V, you should only need to provide a irq_ack_none() stub now.
---
 xen/arch/arm/irq.c    | 10 ----------
 xen/arch/ppc/stubs.c  |  9 ---------
 xen/arch/x86/irq.c    |  9 ---------
 xen/common/irq.c      | 13 +++++++++++++
 xen/include/xen/irq.h |  2 +-
 5 files changed, 14 insertions(+), 29 deletions(-)

diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 7138f9e7c283..e5fb26a3de2d 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -45,16 +45,6 @@ void irq_end_none(struct irq_desc *irq)
     gic_hw_ops->gic_host_irq_type->end(irq);
 }
 
-hw_irq_controller no_irq_type = {
-    .typename = "none",
-    .startup = irq_startup_none,
-    .shutdown = irq_shutdown_none,
-    .enable = irq_enable_none,
-    .disable = irq_disable_none,
-    .ack = irq_ack_none,
-    .end = irq_end_none
-};
-
 static irq_desc_t irq_desc[NR_IRQS];
 static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
 
diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
index 4e03428e071a..923f0e7b2095 100644
--- a/xen/arch/ppc/stubs.c
+++ b/xen/arch/ppc/stubs.c
@@ -139,15 +139,6 @@ void irq_ack_none(struct irq_desc *desc)
     BUG_ON("unimplemented");
 }
 
-hw_irq_controller no_irq_type = {
-    .typename = "none",
-    .startup = irq_startup_none,
-    .shutdown = irq_shutdown_none,
-    .enable = irq_enable_none,
-    .disable = irq_disable_none,
-    .ack = irq_ack_none,
-};
-
 int arch_init_one_irq_desc(struct irq_desc *desc)
 {
     BUG_ON("unimplemented");
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index cfd7a08479d2..e36e06deaa68 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -472,15 +472,6 @@ void cf_check irq_ack_none(struct irq_desc *desc)
     ack_bad_irq(desc->irq);
 }
 
-hw_irq_controller no_irq_type = {
-    "none",
-    irq_startup_none,
-    irq_shutdown_none,
-    irq_enable_none,
-    irq_disable_none,
-    irq_ack_none,
-};
-
 static vmask_t *irq_get_used_vector_mask(int irq)
 {
     vmask_t *ret = NULL;
diff --git a/xen/common/irq.c b/xen/common/irq.c
index 7225b4637486..29729349a6f2 100644
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -3,6 +3,19 @@
 
 DEFINE_PER_CPU(const struct cpu_user_regs *, irq_regs);
 
+const hw_irq_controller no_irq_type = {
+    .typename  = "none",
+    .startup   = irq_startup_none,
+    .shutdown  = irq_shutdown_none,
+    .enable    = irq_enable_none,
+    .disable   = irq_disable_none,
+    .ack       = irq_ack_none,
+
+#ifdef irq_end_none /* Hook is optional per arch */
+    .end       = irq_end_none,
+#endif
+};
+
 int init_one_irq_desc(struct irq_desc *desc)
 {
     int err;
diff --git a/xen/include/xen/irq.h b/xen/include/xen/irq.h
index b71f65db8621..327cd2217c7e 100644
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -122,7 +122,7 @@ extern int request_irq(unsigned int irq, unsigned int irqflags,
                void (*handler)(int irq, void *dev_id),
                const char *devname, void *dev_id);
 
-extern hw_irq_controller no_irq_type;
+extern const hw_irq_controller no_irq_type;
 void cf_check no_action(int cpl, void *dev_id);
 unsigned int cf_check irq_startup_none(struct irq_desc *desc);
 void cf_check irq_actor_none(struct irq_desc *desc);
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu May 30 18:44:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:44:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732973.1139045 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkkw-0006JJ-Mx; Thu, 30 May 2024 18:44:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732973.1139045; Thu, 30 May 2024 18:44:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCkkw-0006JC-K7; Thu, 30 May 2024 18:44:06 +0000
Received: by outflank-mailman (input) for mailman id 732973;
 Thu, 30 May 2024 18:44:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCkkw-0006If-3P
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 18:44:06 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9688e79a-1eb4-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 20:44:04 +0200 (CEST)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a6269ad9a6fso22992766b.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 11:44:04 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67eb344426sm3417866b.215.2024.05.30.11.44.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 11:44:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9688e79a-1eb4-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717094644; x=1717699444; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=L6NWe3kCd+GNZm6t1FJuEgUue9UdCcMzBrBat6t+Zow=;
        b=LQ6uQLpIubjsWMfbGbKB64W34kXw5Wr+OGsIT1wbjN3sDpDiAPkI/EomDFSCMU8LuW
         TNQXkD1QY65rds9MyVGoJcQzAnJMNU0zv4ITm7gn1Yo7XFFtenHhf6Kwm6nJ1SCh3HMF
         1R2j1a/ANRpIqTpe3YfqgeFKbJuC/d6nmgTzA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717094644; x=1717699444;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=L6NWe3kCd+GNZm6t1FJuEgUue9UdCcMzBrBat6t+Zow=;
        b=nSAUtzWB1ZWE6B1To7pgKcbcKy0HX8JOSEavl9WyeiaZEXlspOMeYF+wOtCj/kvZ00
         pnjSLXUwSXYPfw8xMICT4FVMzHhaM1/B5in7qXxzzDLckk0b1a619RG29T1wVB1Q1juN
         HC5Q3R5EbbYrbQbVMaOR+J98npjhhCn/x3G7T4JsHr5UlExVvorbCSltMx7uMg4t0Bxn
         KwhgnC9gEO4uC+2INzhsTF+YHiydse0YMZgP5BJhW+Z0k+s8E0jIj2IFuxrFH5xe8bb/
         jCOBqmzcYDC/wxEicciUN3VAhv/fRRzG0lDeGJEJhsn7w3KS8FyiPJdDB62zNHxdyQfM
         2imQ==
X-Forwarded-Encrypted: i=1; AJvYcCX+JRaAJKeuQ/GE0tCs0WNHts6Sf0L22RA4LiFI4L8GTEVA9cqSpZD+wRfIo/tIM72Mctu/l5turnjAojCNhZ3fK6sQJW8QiL1GQJ97ebg=
X-Gm-Message-State: AOJu0YxRko442x8SDL2hS5iS9AsuLC8c7jNE09TH6vD6DjrZWwyWAI1m
	nqyhr6OEVMlNtyNpLsv2XOQY3tTMpj0DZPAUT9jltP5IFOKRSoPEXBZbLopJAIg=
X-Google-Smtp-Source: AGHT+IFi+rh5zccMyhLoSjo1NENBaIhyHTI64FwwUlasxSvWHKTzZFGDu76ckAYw+G0W2nEugV+ZKg==
X-Received: by 2002:a17:906:c05a:b0:a65:cea9:dd59 with SMTP id a640c23a62f3a-a65e8e6ae3fmr164053166b.33.1717094643651;
        Thu, 30 May 2024 11:44:03 -0700 (PDT)
Message-ID: <a1572f97-98f1-48d7-9533-4daa4fa944b8@citrix.com>
Date: Thu, 30 May 2024 19:44:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
To: "Oleksii K." <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <1cea913117f771a5f3b4404d7bfb7e1329f3f38e.1717008161.git.oleksii.kurochko@gmail.com>
 <1ecd7a2c-b7da-4c59-8ebd-c3841d5991f8@citrix.com>
 <2747a20d490252c279dd051c41a9b8dc69fb4c3e.camel@gmail.com>
 <61c227f6-9515-4bfb-932b-aa0a5d33bc66@citrix.com>
 <934dbe6381e646d551e7f815aacb415e87f0210d.camel@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <934dbe6381e646d551e7f815aacb415e87f0210d.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30/05/2024 7:27 pm, Oleksii K. wrote:
> On Thu, 2024-05-30 at 18:45 +0100, Andrew Cooper wrote:
>> On 30/05/2024 6:12 pm, Oleksii K. wrote:
>>> On Thu, 2024-05-30 at 17:48 +0100, Andrew Cooper wrote:
>>>> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>>>>> diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
>>>>> index 8285bcffef..bda35fc347 100644
>>>>> --- a/xen/arch/riscv/stubs.c
>>>>> +++ b/xen/arch/riscv/stubs.c
>>>>> @@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t,
>>>>> cpu_core_mask);
>>>>>  
>>>>>  nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
>>>>>  
>>>>> -/*
>>>>> - * max_page is defined in page_alloc.c which isn't complied
>>>>> for
>>>>> now.
>>>>> - * definition of max_page will be remove as soon as page_alloc
>>>>> is
>>>>> built.
>>>>> - */
>>>>> -unsigned long __read_mostly max_page;
>>>>> -
>>>>>  /* time.c */
>>>>>  
>>>>>  unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency
>>>>> in
>>>>> kHz. */
>>>>> @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
>>>>>  {
>>>>>      BUG_ON("unimplemented");
>>>>>  }
>>>>> -
>>>>> -/*
>>>>> - * The following functions are defined in common/irq.c, but
>>>>> common/irq.c isn't
>>>>> - * built for now. These changes will be removed there when
>>>>> common/irq.c is
>>>>> - * ready.
>>>>> - */
>>>>> -
>>>>> -void cf_check irq_actor_none(struct irq_desc *desc)
>>>>> -{
>>>>> -    BUG_ON("unimplemented");
>>>>> -}
>>>>> -
>>>>> -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
>>>>> -{
>>>>> -    BUG_ON("unimplemented");
>>>>> -
>>>>> -    return 0;
>>>>> -}
>>>> All 3 of these are introduced in the previous patch and deleted
>>>> again
>>>> here.  Looks like a rebasing accident.
>>> Not really.
>>>
>>> This was done to avoid build failures for RISC-V. If the HEAD is on
>>> the
>>> previous patch where these changes are introduced and then we just
>>> drop
>>> them, it will lead to a linkage error because these functions are
>>> defined in common/irq.c, which isn't built for RISC-V if the HEAD
>>> is on
>>> the previous patch:
>>>    /build/xen/arch/riscv/entry.S:86: undefined reference to
>>> `max_page'
>> Nothing in the series touches entry.S, so I'm not sure what this is
>> complaining about.
>>
>> The stub for get_upper_mfn_bound() references max_page, but it's only
>> used in a SYSCTL so you can avoid the problem with a BUG_ON().
> I didn't get how I can use BUG_ON() with declaration of variable to
> avoid the compilation issue the undefined reference to max_page?

You don't need to implement get_upper_mfn_bound() yet.

If you implement it as:

unsigned long get_upper_mfn_bound(void)
{
    BUG_ON("unimplemented");
}

in the previous patch, then you also don't have any problems with
max_page between these two patches either.

>
>> BTW, you also don't need a return after a BUG_ON(). 
>> __builtin_unreachable() takes care of everything properly for you.
>>
>>
>>>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined
>>> reference to
>>>    `irq_startup_none'
>>>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined
>>> reference to
>>>    `irq_actor_none'
>>>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined
>>> reference to
>>>    `irq_actor_none'
>>>    riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined
>>> reference to
>>>    `irq_actor_none'
>>>    riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none'
>>> isn't
>>>    defined
>>>
>>> That is why these stubs were introduced in the previous patch
>>> (because
>>> common/irq.c isn't built at that moment) and are removed in this
>>> patch
>>> (since at the moment of this patch, common/irq.c is now being
>>> built).
>> These OTOH are a side effect of how no_irq_type works, which is
>> horrifying, and not something I can unsee.
>>
>> I'll see about fixing it, because I really can't bare to leave it
>> like
>> this...
> I am really sorry, but I don't understand should I deal with this
> somehow now or the provided changes are okay?

I've emailed out a 2-patch series to unbreak this.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 18:44:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 18:44:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732976.1139054 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCklJ-0006jF-1D; Thu, 30 May 2024 18:44:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732976.1139054; Thu, 30 May 2024 18:44:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCklI-0006j8-Uz; Thu, 30 May 2024 18:44:28 +0000
Received: by outflank-mailman (input) for mailman id 732976;
 Thu, 30 May 2024 18:44:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCklI-0006ij-Ap; Thu, 30 May 2024 18:44:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCklI-0005cY-9C; Thu, 30 May 2024 18:44:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCklI-0000oM-02; Thu, 30 May 2024 18:44:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCklH-0001fy-Vs; Thu, 30 May 2024 18:44:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=q9yDaERmEMOH1yhX1uwdFx+vQ72Xo5QZ8bUZANGtRfI=; b=MgghIaa8peuPaVnWoP1FBzHY/u
	sdfVNmQOpIG6nHLoBfoqOMeDuGys6UJJ4G+yilitRtRFvK1rM16fYrlN2GU50XocbAqO2d35xpGQA
	g6+83SgzTfKFKOMNrVgwhF1L+99KkCQ1o4qCAfDySlcHSNUsFZqdeq1UtNL3w2+jHeKg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186199-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186199: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=e7848481160b270ebc59d68ecbc8d2722e3aed8c
X-Osstest-Versions-That:
    ovmf=9518d77eb869034a141799b3d28cac20ecb60fe0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 18:44:27 +0000

flight 186199 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186199/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 e7848481160b270ebc59d68ecbc8d2722e3aed8c
baseline version:
 ovmf                 9518d77eb869034a141799b3d28cac20ecb60fe0

Last test of basis   186198  2024-05-30 12:41:14 Z    0 days
Testing same since   186199  2024-05-30 17:14:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  dependabot[bot] <support@github.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   9518d77eb8..e784848116  e7848481160b270ebc59d68ecbc8d2722e3aed8c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 19:39:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:39:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.732994.1139066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClcU-00065p-31; Thu, 30 May 2024 19:39:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 732994.1139066; Thu, 30 May 2024 19:39:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClcT-00065i-VD; Thu, 30 May 2024 19:39:25 +0000
Received: by outflank-mailman (input) for mailman id 732994;
 Thu, 30 May 2024 19:39:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sClcS-00065c-Mv
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:39:24 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 50d53f1f-1ebc-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:39:23 +0200 (CEST)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2e95a883101so16615601fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 12:39:23 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 38308e7fff4ca-2ea91ce8773sm502891fa.102.2024.05.30.12.39.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 12:39:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50d53f1f-1ebc-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717097963; x=1717702763; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=+nrLzN7m2dk4bnSZ+3NSo2S7ZYGMRvpTxgRNUYuc4/Y=;
        b=CEHRECgrfqIMxpoOlLZqW5fZFkmxpsuFXGahe3CeLpLokaRrNCsn9qp56SK6/I6ldu
         bX8YvMCF3JTxvS07NsfywV+fYan6VYKmR1KfsoJQwp0TCvNp7eD74sBNrpceNGul9AC6
         oV1UUGOPkMVUxX7OEmj8bpp2jFQj8TfTt1PBeN9+0+x2PDqM6CMbUMVHt5Ieec0UGC+V
         1dwBydaJpHoQ/arvvqxOKJCgN3yeIqqLVCY7kwy9N21iVulrzPAit3udBeLntZ5FSRiw
         Y3mdW57T0j6aqeHYwvrboPzd9LQiiF5I+k/sTQrVr3RhrEtXmdSrxTPCp44eKnvLjAf6
         qyxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717097963; x=1717702763;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+nrLzN7m2dk4bnSZ+3NSo2S7ZYGMRvpTxgRNUYuc4/Y=;
        b=gTRm7Aw6i/Q6cGnxPac+PVNYr42FR75p34310WZJtHXUJUTS+w1h8nsjq2I8rPMDAe
         SIJ8u/EJUIdq5HXOUGu0LO7PSGef01J4Fv/Lk57wfs75Kcsn3uQHCFvxqstkVYAxUn0D
         KNLKvhgXTT7m+4SB+en5/CDh/dYjZYjCuXIAV3SV1VJn+6T4rcrAubRB7dFypJgnaSnf
         AubOW1Vf+LUql3YBiHh7dq/XrWhpJYG1HDYtPrJSI0jRIw28fqkA365tRu4581XWAyYL
         Jp6KmzW+ib51rJgncsfqdVoTT+QT6sA89kHCZh6r8IHgkhn5PNvOcYBHkAo4i792nGto
         c+ww==
X-Forwarded-Encrypted: i=1; AJvYcCWfphNmcK95XnTt2/pkdOfhaevHIGQqx4TgyHfU2Z5yXd9MclVRzUYAk07ASQZvQP4Dn6wkEGB/IwqMRY1LA8areDl6k/HT49s9AqTeHrQ=
X-Gm-Message-State: AOJu0YxDElIb+UsG3UfbYJKnaj79VkHfuD6/Wq+jHYKec9l3+6trz5YP
	jZg3NT+d7m9UlBUu6lIL8IruCXkCEiirHIV3hpsXqy4mRuRISCoi
X-Google-Smtp-Source: AGHT+IFvZ6E7VIR1loWl5lNDeAIlA3+tQEMoEMJpUbEaBHSY07oLpZeoGlNno/mrIcv1dXk9nkmB4g==
X-Received: by 2002:a2e:7a09:0:b0:2e9:5689:6fde with SMTP id 38308e7fff4ca-2ea8479f906mr18058261fa.14.1717097962518;
        Thu, 30 May 2024 12:39:22 -0700 (PDT)
Message-ID: <a925b579a3a7309532248f9d647e7ffcfc2d30af.camel@gmail.com>
Subject: Re: [PATCH for-4.19 0/2] arch/irq: Untangle no_irq_type
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>
Date: Thu, 30 May 2024 21:39:21 +0200
In-Reply-To: <20240530184027.44609-1-andrew.cooper3@citrix.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 19:40 +0100, Andrew Cooper wrote:
> Found when reviewing Oleksii's series to enable the RISC-V build.
>=20
> The way no_irq_type works is horrifying.=C2=A0 Make it less-so.
>=20
> Andrew Cooper (2):
> =C2=A0 arch/irq: Make irq_ack_none() mandatory
> =C2=A0 arch/irq: Centralise no_irq_type
>=20
> =C2=A0xen/arch/arm/include/asm/irq.h |=C2=A0 3 +++
> =C2=A0xen/arch/arm/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 | 14 ++------------
> =C2=A0xen/arch/ppc/stubs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 | 11 ++++-------
> =C2=A0xen/arch/x86/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 | 11 +----------
> =C2=A0xen/common/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 13 +++++++++++++
> =C2=A0xen/include/xen/irq.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 |=C2=A0 8 +++++++-
> =C2=A06 files changed, 30 insertions(+), 30 deletions(-)
>=20
>=20
> base-commit: 9a905d7dc65883af082532b4dc91ce0131e54047
I am okay to have it in release if the necessary acks will be recieved.

Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu May 30 19:44:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:44:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733000.1139075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClhL-0007hW-JK; Thu, 30 May 2024 19:44:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733000.1139075; Thu, 30 May 2024 19:44:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClhL-0007hP-Gd; Thu, 30 May 2024 19:44:27 +0000
Received: by outflank-mailman (input) for mailman id 733000;
 Thu, 30 May 2024 19:44:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sClhJ-0007hJ-D2
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:44:25 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 03bcea86-1ebd-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 21:44:23 +0200 (CEST)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-52b7ffd9f6eso462647e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 12:44:23 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d34ac4sm58001e87.46.2024.05.30.12.44.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 12:44:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03bcea86-1ebd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717098263; x=1717703063; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=sukSV7LXz8sqSpeex4HAzcY4n5xGT54JRUeqy+8Pje0=;
        b=iwq8d7WEfkTNGxhyDBISpNZwN9EpKq7d8RC8KzMUtaGIgscoHr9a0v/XOa/TuOn8bI
         9Cqjx4il16FG/mRAmLpW4X2LXm6UuoheXzDqhyHaebHHNkiYegJCa13AUNWrqBceb4IN
         Cobs1e3uuK8J3jVyoB2V9IX/m0T9wXluUTFaanWaIIAuCsDJDVjjQJ+zROFQPGGk6nzc
         VDoSpd5QN8HekG+bL7X7VeL9y/PeQVDNG4NIbX9tS1F6QsYHfxAJtH17HBupNb+bVJRB
         4EEvMM8qKm4zQ12ZnVXSilHSzxkDhC1dYjHSDVhITobi3K7cHCE8+K3678Vge0xtcFTa
         Cgzw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717098263; x=1717703063;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sukSV7LXz8sqSpeex4HAzcY4n5xGT54JRUeqy+8Pje0=;
        b=lnR6HtWBt1moIpjpkVUi8RXabJFIoa4MFlSt7Pki6kkiXrkawlQA6/cgrLIiwvMBNa
         R9F7WTfHAbLNAoQc6BY8/RPZl7wA4CP15+E9munkbTWcpcycjctVMMj5beEw+lSf7MNR
         3P4d5z6eNcFif2uxRSmwZkE/GDSbCjVjLNoWvIS420WjwYQOtE8hkmvKe3N7I2C2DpVq
         yozqguettpUk7JfrFTvhbUmzkEN9Vuq3iMMwhOYrYtlCTXT8+/o2xukP6nYl51rwD0eL
         E1WUrJsMyzy7sH2taz7z+tgE/V25ueNkYZyrOf/EN/UDAKkijeNRbPU2Me1XM0OHSbxI
         ADoA==
X-Forwarded-Encrypted: i=1; AJvYcCV/cpL85S+ytBkueUIIMhSZpBos0iO+3s8Lj8z34Y9+UEllJvCnlI0wacDYhgLX59hJ71RjoJ2hC/21kEyVBpAPZ8YX6h6E9cLm9eBXG+M=
X-Gm-Message-State: AOJu0Yzn5xdt9C12NLaUX+X0/mvY8A8MHOkrpQ7AnG+1aCbKLB8Tc/m6
	k0Mq3/uAx9frmVAt+k/5CK4pDkZ6uVs7QJx49Klu/649BNVcj4VU
X-Google-Smtp-Source: AGHT+IGjNszMiSTTc5sW2iEMeJSCX3mRUbzdRx1edzyfxTjKsEzmG7mZXN+ncQUdZ+Bj2Yu7sZBF/A==
X-Received: by 2002:a19:ac08:0:b0:529:593f:3f3c with SMTP id 2adb3069b0e04-52b7d47ac02mr2051476e87.53.1717098262444;
        Thu, 30 May 2024 12:44:22 -0700 (PDT)
Message-ID: <078d34627c96d923e473717f59789489fa199ea4.camel@gmail.com>
Subject: Re: [PATCH 1/2] arch/irq: Make irq_ack_none() mandatory
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>
Date: Thu, 30 May 2024 21:44:21 +0200
In-Reply-To: <20240530184027.44609-2-andrew.cooper3@citrix.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
	 <20240530184027.44609-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 19:40 +0100, Andrew Cooper wrote:
> Any non-stub implementation of these is going to have to do something
> here.
>=20
> irq_end_none() is more complicated and has arch-specific interactions
> with
> irq_ack_none(), so make it optional.
>=20
> For PPC, introduce a stub irq_ack_none().
>=20
> For ARM and x86, export the existing {ack,end}_none() helpers,
> gaining an irq_
> prefix for consisntency with everything else in no_irq_type.
>=20
> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
LGTM: Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> CC: Shawn Anastasio <sanastasio@raptorengineering.com>
> ---
> =C2=A0xen/arch/arm/include/asm/irq.h | 3 +++
> =C2=A0xen/arch/arm/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 | 8 ++++----
> =C2=A0xen/arch/ppc/stubs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 | 6 ++++++
> =C2=A0xen/arch/x86/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 | 4 ++--
> =C2=A0xen/include/xen/irq.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 | 6 ++++++
> =C2=A05 files changed, 21 insertions(+), 6 deletions(-)
>=20
> diff --git a/xen/arch/arm/include/asm/irq.h
> b/xen/arch/arm/include/asm/irq.h
> index 1bae5388878e..ec437add0971 100644
> --- a/xen/arch/arm/include/asm/irq.h
> +++ b/xen/arch/arm/include/asm/irq.h
> @@ -98,6 +98,9 @@ void irq_set_affinity(struct irq_desc *desc, const
> cpumask_t *mask);
> =C2=A0 */
> =C2=A0bool irq_type_set_by_domain(const struct domain *d);
> =C2=A0
> +void irq_end_none(struct irq_desc *irq);
> +#define irq_end_none irq_end_none
> +
> =C2=A0#endif /* _ASM_HW_IRQ_H */
> =C2=A0/*
> =C2=A0 * Local variables:
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index bcce80a4d624..7138f9e7c283 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,12 +31,12 @@ struct irq_guest
> =C2=A0=C2=A0=C2=A0=C2=A0 unsigned int virq;
> =C2=A0};
> =C2=A0
> -static void ack_none(struct irq_desc *irq)
> +void irq_ack_none(struct irq_desc *irq)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 printk("unexpected IRQ trap at irq %02x\n", irq-=
>irq);
> =C2=A0}
> =C2=A0
> -static void end_none(struct irq_desc *irq)
> +void irq_end_none(struct irq_desc *irq)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 /*
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Still allow a CPU to end an interrupt if=
 we receive a
> spurious
> @@ -51,8 +51,8 @@ hw_irq_controller no_irq_type =3D {
> =C2=A0=C2=A0=C2=A0=C2=A0 .shutdown =3D irq_shutdown_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 .enable =3D irq_enable_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 .disable =3D irq_disable_none,
> -=C2=A0=C2=A0=C2=A0 .ack =3D ack_none,
> -=C2=A0=C2=A0=C2=A0 .end =3D end_none
> +=C2=A0=C2=A0=C2=A0 .ack =3D irq_ack_none,
> +=C2=A0=C2=A0=C2=A0 .end =3D irq_end_none
> =C2=A0};
> =C2=A0
> =C2=A0static irq_desc_t irq_desc[NR_IRQS];
> diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
> index da193839bd09..4e03428e071a 100644
> --- a/xen/arch/ppc/stubs.c
> +++ b/xen/arch/ppc/stubs.c
> @@ -134,12 +134,18 @@ void pirq_set_affinity(struct domain *d, int
> pirq, const cpumask_t *mask)
> =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> =C2=A0}
> =C2=A0
> +void irq_ack_none(struct irq_desc *desc)
> +{
> +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> +}
> +
> =C2=A0hw_irq_controller no_irq_type =3D {
> =C2=A0=C2=A0=C2=A0=C2=A0 .typename =3D "none",
> =C2=A0=C2=A0=C2=A0=C2=A0 .startup =3D irq_startup_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 .shutdown =3D irq_shutdown_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 .enable =3D irq_enable_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 .disable =3D irq_disable_none,
> +=C2=A0=C2=A0=C2=A0 .ack =3D irq_ack_none,
> =C2=A0};
> =C2=A0
> =C2=A0int arch_init_one_irq_desc(struct irq_desc *desc)
> diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
> index c16205a9beb6..cfd7a08479d2 100644
> --- a/xen/arch/x86/irq.c
> +++ b/xen/arch/x86/irq.c
> @@ -467,7 +467,7 @@ int __init init_irq_data(void)
> =C2=A0=C2=A0=C2=A0=C2=A0 return 0;
> =C2=A0}
> =C2=A0
> -static void cf_check ack_none(struct irq_desc *desc)
> +void cf_check irq_ack_none(struct irq_desc *desc)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 ack_bad_irq(desc->irq);
> =C2=A0}
> @@ -478,7 +478,7 @@ hw_irq_controller no_irq_type =3D {
> =C2=A0=C2=A0=C2=A0=C2=A0 irq_shutdown_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 irq_enable_none,
> =C2=A0=C2=A0=C2=A0=C2=A0 irq_disable_none,
> -=C2=A0=C2=A0=C2=A0 ack_none,
> +=C2=A0=C2=A0=C2=A0 irq_ack_none,
> =C2=A0};
> =C2=A0
> =C2=A0static vmask_t *irq_get_used_vector_mask(int irq)
> diff --git a/xen/include/xen/irq.h b/xen/include/xen/irq.h
> index 89f7a8317a87..b71f65db8621 100644
> --- a/xen/include/xen/irq.h
> +++ b/xen/include/xen/irq.h
> @@ -130,6 +130,12 @@ void cf_check irq_actor_none(struct irq_desc
> *desc);
> =C2=A0#define irq_disable_none irq_actor_none
> =C2=A0#define irq_enable_none irq_actor_none
> =C2=A0
> +/*
> + * irq_ack_none() must be provided by the architecture.
> + * irq_end_none() is optional, and opted into using a define.
> + */
> +void irq_ack_none(struct irq_desc *irq);
> +
> =C2=A0/*
> =C2=A0 * Per-cpu interrupted context register state - the inner-most
> interrupt frame
> =C2=A0 * on the stack.



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:45:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:45:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733003.1139085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClhp-00089e-Rh; Thu, 30 May 2024 19:44:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733003.1139085; Thu, 30 May 2024 19:44:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClhp-00089X-Nu; Thu, 30 May 2024 19:44:57 +0000
Received: by outflank-mailman (input) for mailman id 733003;
 Thu, 30 May 2024 19:44:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sClhp-0007hJ-15
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:44:57 +0000
Received: from 009.lax.mailroute.net (009.lax.mailroute.net [199.89.1.12])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15b3a2a2-1ebd-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 21:44:54 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 009.lax.mailroute.net (Postfix) with ESMTP id 4VqxY061N4zlgMVV;
 Thu, 30 May 2024 19:44:52 +0000 (UTC)
Received: from 009.lax.mailroute.net ([127.0.0.1])
 by localhost (009.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id XhrHqHFE5lIy; Thu, 30 May 2024 19:44:48 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 009.lax.mailroute.net (Postfix) with ESMTPSA id 4VqxXv0mMXzlgMVR;
 Thu, 30 May 2024 19:44:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15b3a2a2-1ebd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717098288; x=1719690289; bh=1qawYRnNtHLyKkfk2KIuL7Vw
	Fij1fGlSBD1rE6Kec4k=; b=O+dyHa1DO+LxwuvN37caN3rc79Al5B59aVt872Yy
	myvPBRd3qJOpB/ssbZf9DD6jeQwG3oky5nPyGHmKtOjLfDnHlE+e3wVIfqzpWjOs
	6hfPZNndbOYIdhdiprqYgfQpwZxp2KIDZmAsJBYG5LuLkM6flrrsn+Z7CP3aCzDV
	pwN4+gbFQD4bp6z4s36qcFNV+1waUaRjvrfL+xtNhs1hivVR8LlUKLXvQ4CEJ/sF
	RymJznQ8+fPT5LcsiI4IQupBmhHjJpPce8ad1C01w5LQVIcRKBZRcPzwxcXH/hff
	wu+Gqo4QOptIYFBfw/rmjL36jeH02rPcp9YADbGYnOeaDg==
X-Virus-Scanned: by MailRoute
Message-ID: <2384c3c9-5629-4438-a87b-0e05e1b6f5cc@acm.org>
Date: Thu, 30 May 2024 12:44:46 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 01/12] ubd: untagle discard vs write zeroes not support
 handling
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-2-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-2-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Discard and Write Zeroes are different operation and implemented
> by different fallocate opcodes for ubd.  If one fails the other one
> can work and vice versa.
> 
> Split the code to disable the operations in ubd_handler to only
> disable the operation that actually failed.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:45:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:45:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733005.1139094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCli6-00005E-2F; Thu, 30 May 2024 19:45:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733005.1139094; Thu, 30 May 2024 19:45:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCli5-000054-Vn; Thu, 30 May 2024 19:45:13 +0000
Received: by outflank-mailman (input) for mailman id 733005;
 Thu, 30 May 2024 19:45:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LA40=NB=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCli5-0007hJ-Bx
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:45:13 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 209a75a0-1ebd-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 21:45:11 +0200 (CEST)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-52b87505f01so73458e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 12:45:11 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d89279sm55002e87.244.2024.05.30.12.45.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 May 2024 12:45:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 209a75a0-1ebd-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717098311; x=1717703111; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=9mHYZgWE9lLreAEm31XHRh2mF44QYu4c7ycfIU+9ZIk=;
        b=PAnPFSMenajWMNN6NbRYD+fz1FE+GGZOcJDWBm7QyRDXtVOJ/VoJDJNxOzeBkqzct6
         k4KM++En7bWFDYlMoq3ecZqRO/aRnVLxXPPD85J9HddH3RdTvKNXC7gOBmUEgMgbBHGs
         hehtSJgsLD0Znpkq0gFwrotRLCIUfww6V1j2xFFcnWZn/nM0xj7GZ1MQ1yYF+PJNOm8+
         LGxLyDRqmpan07cj2E3BNkqmrCNkJASj6zijBUHAn5w4SltfI8ijK04rPwkJL1nGYiiw
         Fg66Xs4PtZQsuJEEOb/cfFfKPOyDoVvCLJl7igZKGWCBIHzQJDZep7ITVxzbqe6QYKO7
         F8XQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717098311; x=1717703111;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9mHYZgWE9lLreAEm31XHRh2mF44QYu4c7ycfIU+9ZIk=;
        b=iFQpq94jtPrwTm3s5j9ILSBbtLp/sEjrzOOoQup8sRb7bBGIBWi2mJrr4kZ7GtFw2P
         NvZo5FLO4py1yPS5XdwNYhLZm/iS9A/DFwJaHoq91p8c9px64MvNT2O9Fz4/z46geIy1
         QymnNZmXfk6q0RqeqGxh8Une8feKu/h8P8MCBBEUDHhtX9Bqn5e70ezFhdV0+FZNgZE9
         GudJULyg7OV61RD0xEyi7MYCFhSJWg6smUZX8SoyLDuy0GaqhWMoftGzs9f7630dOUmO
         7jgGrCDVCtaQinFqPrE4qkTq75+tEMigd2NTxhCDzISqaii4gqY3nqlggA4uXxeLWbs7
         bBcQ==
X-Forwarded-Encrypted: i=1; AJvYcCUsmH9R0F2YXHbHUreih2FvMclQjdEbSDWK3SOgZwvhReujgnbFsdzZSJ0/6tcGTev/tKJXRLEp9QI+lfH4KtU4ldMTtxq3Lio5zqJF1lk=
X-Gm-Message-State: AOJu0YxDckSHi4s3Kfg2aOXM47zDA+FhVJZ7qFzvKsWdks2vUqUCivSP
	5u2G1DzgMWUcke9xJFT+lZfh+d0A//Ylredsr+gdhsCfdHEmdZl4
X-Google-Smtp-Source: AGHT+IHBBM3NzlNbOhsvPnamA9T6IP2Gv1nTl6C3lCz3LlkMqYDYfbHqzjNr9bmiEN1PAuSSQ45g9A==
X-Received: by 2002:a05:6512:3a5:b0:52b:7917:9966 with SMTP id 2adb3069b0e04-52b7d418fc2mr1717709e87.8.1717098311121;
        Thu, 30 May 2024 12:45:11 -0700 (PDT)
Message-ID: <67e54efb6c8a4bee1d244df08b6e48b17c79a84d.camel@gmail.com>
Subject: Re: [PATCH 2/2] arch/irq: Centralise no_irq_type
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>
Date: Thu, 30 May 2024 21:45:10 +0200
In-Reply-To: <20240530184027.44609-3-andrew.cooper3@citrix.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
	 <20240530184027.44609-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Thu, 2024-05-30 at 19:40 +0100, Andrew Cooper wrote:
> Having no_irq_type defined per arch, but using common callbacks is a
> mess, and
> particualrly hard to bootstrap a new architecture with.
>=20
> Now that the ack()/end() hooks have been exported suitably, move the
> definition of no_irq_type into common/irq.c, and into .rodata for
> good
> measure.
>=20
> No functional change, but a whole lot less tangled.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
LGTM: Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>


> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> CC: Shawn Anastasio <sanastasio@raptorengineering.com>
>=20
> Oleksii: For RISC-V, you should only need to provide a irq_ack_none()
> stub now.
Sure, I will update my patch series during rebase.

~ Oleksii
> ---
> =C2=A0xen/arch/arm/irq.c=C2=A0=C2=A0=C2=A0 | 10 ----------
> =C2=A0xen/arch/ppc/stubs.c=C2=A0 |=C2=A0 9 ---------
> =C2=A0xen/arch/x86/irq.c=C2=A0=C2=A0=C2=A0 |=C2=A0 9 ---------
> =C2=A0xen/common/irq.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 13 +++++++++++++
> =C2=A0xen/include/xen/irq.h |=C2=A0 2 +-
> =C2=A05 files changed, 14 insertions(+), 29 deletions(-)
>=20
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 7138f9e7c283..e5fb26a3de2d 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -45,16 +45,6 @@ void irq_end_none(struct irq_desc *irq)
> =C2=A0=C2=A0=C2=A0=C2=A0 gic_hw_ops->gic_host_irq_type->end(irq);
> =C2=A0}
> =C2=A0
> -hw_irq_controller no_irq_type =3D {
> -=C2=A0=C2=A0=C2=A0 .typename =3D "none",
> -=C2=A0=C2=A0=C2=A0 .startup =3D irq_startup_none,
> -=C2=A0=C2=A0=C2=A0 .shutdown =3D irq_shutdown_none,
> -=C2=A0=C2=A0=C2=A0 .enable =3D irq_enable_none,
> -=C2=A0=C2=A0=C2=A0 .disable =3D irq_disable_none,
> -=C2=A0=C2=A0=C2=A0 .ack =3D irq_ack_none,
> -=C2=A0=C2=A0=C2=A0 .end =3D irq_end_none
> -};
> -
> =C2=A0static irq_desc_t irq_desc[NR_IRQS];
> =C2=A0static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
> =C2=A0
> diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
> index 4e03428e071a..923f0e7b2095 100644
> --- a/xen/arch/ppc/stubs.c
> +++ b/xen/arch/ppc/stubs.c
> @@ -139,15 +139,6 @@ void irq_ack_none(struct irq_desc *desc)
> =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> =C2=A0}
> =C2=A0
> -hw_irq_controller no_irq_type =3D {
> -=C2=A0=C2=A0=C2=A0 .typename =3D "none",
> -=C2=A0=C2=A0=C2=A0 .startup =3D irq_startup_none,
> -=C2=A0=C2=A0=C2=A0 .shutdown =3D irq_shutdown_none,
> -=C2=A0=C2=A0=C2=A0 .enable =3D irq_enable_none,
> -=C2=A0=C2=A0=C2=A0 .disable =3D irq_disable_none,
> -=C2=A0=C2=A0=C2=A0 .ack =3D irq_ack_none,
> -};
> -
> =C2=A0int arch_init_one_irq_desc(struct irq_desc *desc)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
> index cfd7a08479d2..e36e06deaa68 100644
> --- a/xen/arch/x86/irq.c
> +++ b/xen/arch/x86/irq.c
> @@ -472,15 +472,6 @@ void cf_check irq_ack_none(struct irq_desc
> *desc)
> =C2=A0=C2=A0=C2=A0=C2=A0 ack_bad_irq(desc->irq);
> =C2=A0}
> =C2=A0
> -hw_irq_controller no_irq_type =3D {
> -=C2=A0=C2=A0=C2=A0 "none",
> -=C2=A0=C2=A0=C2=A0 irq_startup_none,
> -=C2=A0=C2=A0=C2=A0 irq_shutdown_none,
> -=C2=A0=C2=A0=C2=A0 irq_enable_none,
> -=C2=A0=C2=A0=C2=A0 irq_disable_none,
> -=C2=A0=C2=A0=C2=A0 irq_ack_none,
> -};
> -
> =C2=A0static vmask_t *irq_get_used_vector_mask(int irq)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 vmask_t *ret =3D NULL;
> diff --git a/xen/common/irq.c b/xen/common/irq.c
> index 7225b4637486..29729349a6f2 100644
> --- a/xen/common/irq.c
> +++ b/xen/common/irq.c
> @@ -3,6 +3,19 @@
> =C2=A0
> =C2=A0DEFINE_PER_CPU(const struct cpu_user_regs *, irq_regs);
> =C2=A0
> +const hw_irq_controller no_irq_type =3D {
> +=C2=A0=C2=A0=C2=A0 .typename=C2=A0 =3D "none",
> +=C2=A0=C2=A0=C2=A0 .startup=C2=A0=C2=A0 =3D irq_startup_none,
> +=C2=A0=C2=A0=C2=A0 .shutdown=C2=A0 =3D irq_shutdown_none,
> +=C2=A0=C2=A0=C2=A0 .enable=C2=A0=C2=A0=C2=A0 =3D irq_enable_none,
> +=C2=A0=C2=A0=C2=A0 .disable=C2=A0=C2=A0 =3D irq_disable_none,
> +=C2=A0=C2=A0=C2=A0 .ack=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D irq_ack_=
none,
> +
> +#ifdef irq_end_none /* Hook is optional per arch */
> +=C2=A0=C2=A0=C2=A0 .end=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D irq_end_=
none,
> +#endif
> +};
> +
> =C2=A0int init_one_irq_desc(struct irq_desc *desc)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 int err;
> diff --git a/xen/include/xen/irq.h b/xen/include/xen/irq.h
> index b71f65db8621..327cd2217c7e 100644
> --- a/xen/include/xen/irq.h
> +++ b/xen/include/xen/irq.h
> @@ -122,7 +122,7 @@ extern int request_irq(unsigned int irq, unsigned
> int irqflags,
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 void (*handler)(int irq, void *dev_id),
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 const char *devname, void *dev_id);
> =C2=A0
> -extern hw_irq_controller no_irq_type;
> +extern const hw_irq_controller no_irq_type;
> =C2=A0void cf_check no_action(int cpl, void *dev_id);
> =C2=A0unsigned int cf_check irq_startup_none(struct irq_desc *desc);
> =C2=A0void cf_check irq_actor_none(struct irq_desc *desc);



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:47:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:47:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733015.1139105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClk7-0000su-Gb; Thu, 30 May 2024 19:47:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733015.1139105; Thu, 30 May 2024 19:47:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClk7-0000sn-DS; Thu, 30 May 2024 19:47:19 +0000
Received: by outflank-mailman (input) for mailman id 733015;
 Thu, 30 May 2024 19:47:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sClk6-0000sh-E4
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:47:18 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a3a6d97-1ebd-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:47:16 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4Vqxbk6tSqz6Cnk9T;
 Thu, 30 May 2024 19:47:14 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id tO0K_eu1GhdP; Thu, 30 May 2024 19:47:08 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4VqxbZ4pbBz6Cnk9F;
 Thu, 30 May 2024 19:47:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a3a6d97-1ebd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717098428; x=1719690429; bh=gIZD+XBfeoiXCs+vsIOauc0J
	AFtw0ViQpf0RwIfdqeI=; b=LeBK1nTDoaUU8KAvyoiidxeqI1H8VC7yltJTPW4y
	XkYIFoLA0Xsv871K8AMKv1pruRb7L5Vojxd3TXpVjOmpzCKnxzsAzPwXavUCLmiG
	5Io9SoOjP5YKnml8f0lVeIWDw6umtR8N2JBzgwVMlGiuo36RH4kWKMRKIp7SrKHq
	49OVo1TUglVJnZ/FHnf1DulkWr6IflF9OncnRDnJq9pnqGyM16QVtbsNy1ahrI3q
	GKMmq9vzmNV1PmpyA8YGpgIcN378oHvXWXKo5o6AkOKWk1xSeOKUZJjERmaxIpao
	1jHLbCjdnG+qM70N4r3Tjg99XIgFTQPKTwcpUYGYirs6RQ==
X-Virus-Scanned: by MailRoute
Message-ID: <8d927897-0436-4f47-9b03-c3e9adb90f40@acm.org>
Date: Thu, 30 May 2024 12:47:06 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for
 max_sectors
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-3-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-3-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> The soft max_sectors limit is normally capped by the hardware limits and
> an arbitrary upper limit enforced by the kernel, but can be modified by
> the user.  A few drivers want to increase this limit (nbd, rbd) or
> adjust it up or down based on hardware capabilities (sd).
> 
> Change blk_validate_limits to default max_sectors to the optimal I/O
> size, or upgrade it to the preferred minimal I/O size if that is
> larger than the kernel default if no optimal I/O size is provided based
> on the logic in the SD driver.
> 
> This keeps the existing kernel default for drivers that do not provide
> an io_opt or very big io_min value, but picks a much more useful
> default for those who provide these hints, and allows to remove the
> hacks to set the user max_sectors limit in nbd, rbd and sd.
> 
> Note that rd picks a different value for the optimal I/O size vs the
             ^^
             rbd?

> user max_sectors value, so this is a bit of a behavior change that
> could use careful review from people familiar with rbd.

Anyway:

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:48:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:48:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733022.1139114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCll3-0001Rs-Ny; Thu, 30 May 2024 19:48:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733022.1139114; Thu, 30 May 2024 19:48:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCll3-0001Rl-LY; Thu, 30 May 2024 19:48:17 +0000
Received: by outflank-mailman (input) for mailman id 733022;
 Thu, 30 May 2024 19:48:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sCll2-0001RU-47
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:48:16 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d4247be-1ebd-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:48:15 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4Vqxcs6krTz6Cnk9T;
 Thu, 30 May 2024 19:48:13 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id ZPBeIfQpf7PL; Thu, 30 May 2024 19:48:09 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqxcl3Zhlz6Cnk9F;
 Thu, 30 May 2024 19:48:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d4247be-1ebd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717098489; x=1719690490; bh=dyGzZSWLsyackA37XVq9nl02
	3b/Svu5aWeAkdF5uPBU=; b=rGDN10KGvDcmbfFgFHYGEoAseaR0fagJcQ3Kgqep
	Lubtq290OW2C91K9XeG9czurQf/wRzgWunDsFsAMxwo9wkZ+qvmKm4R0a1IhEFZs
	mDWx9FVqmwxJ1uh0jSOQryGjPbUDq3am1SqtPcwOYoYJeBhb7NS+0SA896mRwdax
	46is36KI4H8l2gh+0pU4iA/zQI9dYtDmKkBX014QQbhNpPugXqH9dZ3vHZGJc29x
	QNtcdSGsqF3KHp+AUYXr1LTxQON8ZUdXzrzE864yWEALccGzPTaLHZ74IyVMr54h
	4/Z8+E6et0bHB2aWboLglX7RhsaK/9E7/5fftwo1zhVw2w==
X-Virus-Scanned: by MailRoute
Message-ID: <55cbf9eb-a52c-4d1f-aaac-5d426b60d436@acm.org>
Date: Thu, 30 May 2024 12:48:07 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 03/12] sd: simplify the ZBC case in
 provisioning_mode_store
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-4-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-4-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Don't reset the discard settings to no-op over and over when a user
> writes to the provisioning attribute as that is already the default
> mode for ZBC devices.  In hindsight we should have made writing to
> the attribute fail for ZBC devices, but the code has probably been
> around for far too long to change this now.

If Damien's feedback gets addressed, feel free to add:

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:48:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:48:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733023.1139124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCll7-0001i0-VK; Thu, 30 May 2024 19:48:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733023.1139124; Thu, 30 May 2024 19:48:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCll7-0001hr-Sf; Thu, 30 May 2024 19:48:21 +0000
Received: by outflank-mailman (input) for mailman id 733023;
 Thu, 30 May 2024 19:48:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=csea=NB=gmail.com=idryomov@srs-se1.protection.inumbo.net>)
 id 1sCll6-0001RU-EY
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:48:20 +0000
Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com
 [2001:4860:4864:20::29])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90242bff-1ebd-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:48:19 +0200 (CEST)
Received: by mail-oa1-x29.google.com with SMTP id
 586e51a60fabf-24c9f73ccaaso721319fac.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 12:48:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90242bff-1ebd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717098498; x=1717703298; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=41y1G1yWB6PRjXPqqWESihB4DpKvKPtS6LjA52lk1aM=;
        b=OIZ+wTtEi+sATpwCjdmYQf9r7LANBeAjWEEiDlm8qcJTj6chROqfgCMyLyl8LBAOsy
         lE90EStB9lpJZyWzf/WsZqe2cVhQIepyrtN/Zpn9Bfeka24LHc5Y+2PFvOMc4JK3+Efm
         8QPqjUusCe6r+St/Jz6Mlkf+9qwOeqbCAnnbl319vPZhxXAAshKkdfK2FS+jRgSu/3g9
         8a+XIDkM1EkTz97783v3UuFRGvUZduOzSZKtywWXw9QcYHqg5KOw7yBhRi2BSzpLr+VB
         1r87unwrdR12Rq7mmR1A9L7h3xr+rfFGl4kTdxUc8kb0oo6+hqgYQBQLJu8DCceZvL5T
         uAvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717098498; x=1717703298;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=41y1G1yWB6PRjXPqqWESihB4DpKvKPtS6LjA52lk1aM=;
        b=NIeo7a+PfFrb8UJQaj7deG19JQwryMlY2lt9hydKZEsYhF4Dd1K0jkQLY53RUkLO3J
         jkuaQ0t5yTzEf2QCOspKefcBxlyp2DRvXAOxGUZv5jljr76zu8eqNjc1rA0xHBF5F2Et
         +EwYZPCnBXNYBFiOGmkepR/cQx5xrlcWVE89LbTxqEh5PgRdQv4TYgQUpR0iVJEvwD5q
         46LVVBI0RN5Y6QekM0wuDiNKNLYcf5n9Lto9gCjDEDzOfuCSVr1St9PCRfU7VR5rA11h
         bo1JCkVAFedmvjTjPGuuNo+4LerJGOU0bwNNX6a2I8XctBgcJEsG1w7sVWNIsMqWG5va
         YOdA==
X-Forwarded-Encrypted: i=1; AJvYcCWbzg6sjX03hcU8aapmL9YEfQiZL4tcKuxasYeRSsYSphiXgefHZ6IlCtD77qKy+2YJuAEhyGKgxNo+sv8Eo4EiQROAT24qVKhVSa6yKzw=
X-Gm-Message-State: AOJu0YxE4tBcofKoiEDP4OnkiYTAwqgPnFSoNLKba9vBJiF7pnWsvjTg
	IsF0y0pU38HFqe/63RRpUshbjpT0lAgUvv+CzZXQamMf+FBFYvqTwG59AI7RH6zW+K1nfFw1doa
	Zb847FnVyrK8c+vcZ0LB1pM16t7o=
X-Google-Smtp-Source: AGHT+IGxcLZtvGZ8VsT8cGFEuW/elxJjHWgoM8RDpdS5UpADe97i5sefwnR8gwVFsz1jHFTWagn8cx64x4raaIJwd/s=
X-Received: by 2002:a05:6870:b492:b0:250:6f7c:495 with SMTP id
 586e51a60fabf-2506f7c0a24mr2683166fac.9.1717098498330; Thu, 30 May 2024
 12:48:18 -0700 (PDT)
MIME-Version: 1.0
References: <20240529050507.1392041-1-hch@lst.de> <20240529050507.1392041-3-hch@lst.de>
In-Reply-To: <20240529050507.1392041-3-hch@lst.de>
From: Ilya Dryomov <idryomov@gmail.com>
Date: Thu, 30 May 2024 21:48:06 +0200
Message-ID: <CAOi1vP-F0FO4WTnrEt7FC-uu2C8NTbejvJQQGdZqT475c2G1jA@mail.gmail.com>
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for max_sectors
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, "Martin K. Petersen" <martin.petersen@oracle.com>, 
	Richard Weinberger <richard@nod.at>, Anton Ivanov <anton.ivanov@cambridgegreys.com>, 
	Johannes Berg <johannes@sipsolutions.net>, Josef Bacik <josef@toxicpanda.com>, 
	Dongsheng Yang <dongsheng.yang@easystack.cn>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	linux-um@lists.infradead.org, linux-block@vger.kernel.org, 
	nbd@other.debian.org, ceph-devel@vger.kernel.org, 
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, May 29, 2024 at 7:05=E2=80=AFAM Christoph Hellwig <hch@lst.de> wrot=
e:
>
> The soft max_sectors limit is normally capped by the hardware limits and
> an arbitrary upper limit enforced by the kernel, but can be modified by
> the user.  A few drivers want to increase this limit (nbd, rbd) or
> adjust it up or down based on hardware capabilities (sd).
>
> Change blk_validate_limits to default max_sectors to the optimal I/O
> size, or upgrade it to the preferred minimal I/O size if that is
> larger than the kernel default if no optimal I/O size is provided based
> on the logic in the SD driver.
>
> This keeps the existing kernel default for drivers that do not provide
> an io_opt or very big io_min value, but picks a much more useful
> default for those who provide these hints, and allows to remove the
> hacks to set the user max_sectors limit in nbd, rbd and sd.
>
> Note that rd picks a different value for the optimal I/O size vs the
> user max_sectors value, so this is a bit of a behavior change that
> could use careful review from people familiar with rbd.

Hi Christoph,

For rbd, this change effectively lowers max_sectors from 4M to 64K or
less and that is definitely not desirable.  From previous interactions
with users we want max_sectors to match max_hw_sectors -- this has come
up a quite a few times over the years.  Some people just aren't aware
of the soft cap and the fact that it's adjustable and get frustrated
over the time poured into debugging their iostat numbers for workloads
that can send object (set) size I/Os.

Looking at the git history, we lowered io_opt from objset_bytes to
opts->alloc_size in commit [1], but I guess io_opt was lowered just
along for the ride.  What that commit was concerned with is really
discard_granularity and to a smaller extent io_min.

How much difference does io_opt make in the real world?  If what rbd
does stands in the way of a tree-wide cleanup, I would much rather bump
io_opt back to objset_bytes (i.e. what max_user_sectors is currently
set to).

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/comm=
it/?id=3D16d80c54ad42c573a897ae7bcf5a9816be54e6fe

Thanks,

                Ilya


From xen-devel-bounces@lists.xenproject.org Thu May 30 19:50:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:50:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733034.1139134 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClnN-0003td-BW; Thu, 30 May 2024 19:50:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733034.1139134; Thu, 30 May 2024 19:50:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClnN-0003tW-8w; Thu, 30 May 2024 19:50:41 +0000
Received: by outflank-mailman (input) for mailman id 733034;
 Thu, 30 May 2024 19:50:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sClnM-0003tQ-1B
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:50:40 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2e70403-1ebd-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:50:38 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4Vqxgd2Y61z6Cnk9V;
 Thu, 30 May 2024 19:50:37 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id NvbqtULo1Ahy; Thu, 30 May 2024 19:50:32 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4VqxgW0TsNz6Cnk9T;
 Thu, 30 May 2024 19:50:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2e70403-1ebd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717098632; x=1719690633; bh=7QFmifQMChBXa8KST+DsqdnF
	Epu7aV4gNbQITRSzRXY=; b=WUjlX1PDIQyMgUb/0/yO+Fvzm6H2r3VbR/ytQbyG
	ja1ZtUlCegyS9BGCsSsID9x1/+kDjEs1NxZDIrjljFC2ZUpQ8nSD6H6RckeDAaV/
	8NybhiHj4Sye5m61h+RgA1L76P3LsN9DZgjtG7WVblDA+TCSZ/ARfrUBW1SRcvIq
	CUnB7FlcYqzV2NUiwaoz3LW53sA83Z9iucHomhpMAkPcCpOBKOEtiyurze8Qo3I0
	N3az9ReZGqTkXsyoXq0i+KzjVMS+igSHckcZ9ZyYDlURgNiKKLrMqPh/lqphOWEQ
	PLQr+L12hPXk4p6dZ8EbtkCXzU85zv+TAPmk7Obq/DjhLg==
X-Virus-Scanned: by MailRoute
Message-ID: <bb168ec3-b3e8-4e40-8f8c-eb0a1dd00f16@acm.org>
Date: Thu, 30 May 2024 12:50:30 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 04/12] sd: add a sd_disable_discard helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-5-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-5-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Add helper to disable discard when it is not supported and use it
> instead of sd_config_discard in the I/O completion handler.  This avoids
> touching more fields than required in the I/O completion handler and
> prepares for converting sd to use the atomic queue limits API.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:51:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:51:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733039.1139144 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClnr-0004Se-J1; Thu, 30 May 2024 19:51:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733039.1139144; Thu, 30 May 2024 19:51:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClnr-0004SX-GQ; Thu, 30 May 2024 19:51:11 +0000
Received: by outflank-mailman (input) for mailman id 733039;
 Thu, 30 May 2024 19:51:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sClnq-0003tQ-0a
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:51:10 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f5043a6e-1ebd-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:51:09 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4VqxhD06zdz6Cnk9V;
 Thu, 30 May 2024 19:51:08 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id x4rVbmHKH9zP; Thu, 30 May 2024 19:51:03 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqxh56rNhz6Cnk9T;
 Thu, 30 May 2024 19:51:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5043a6e-1ebd-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717098663; x=1719690664; bh=BLwZbOo2VuEjeawIOT3dZywp
	LeD1IzBghu5o+yp6hiA=; b=BMa7rd7XppGue5oaVmK2KHUY3MIK85AfzEEn+EKp
	bcNLOjbFEai52dS82B0yhSPKs5RyqQXlGiN4QD+YUDbrcAZHAjNBuw86VOjnbcWc
	KpSflEQS63VRNlvjxLG6uywRVDTYgOMSrCxedOS87bGL09MYLZfeGA3T5ub8rgQI
	sS/meOXbeO/ZoOUiTMgwb/aGcVJUK8awiaLnzIKJ7y1nfc5R9zHMSdwMeuPfjcqP
	3xL2my5/vEmEuShKNLcDV1pHPpTg6z1N0uk5Ah0GFOd0IfgaFouOWdGCAH9KTqO3
	3vxrR40G8smQRaSEyQHqccCit4O/AMIRLYUJfU2LODky0g==
X-Virus-Scanned: by MailRoute
Message-ID: <391fad91-7a65-4b2d-aa5d-58a5bc341d5f@acm.org>
Date: Thu, 30 May 2024 12:51:01 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 05/12] sd: add a sd_disable_write_same helper
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-6-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-6-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Add helper to disable WRITE SAME when it is not supported and use it
> instead of sd_config_write_same in the I/O completion handler.  This
> avoids touching more fields than required in the I/O completion handler
> and  prepares for converting sd to use the atomic queue limits API.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 19:52:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 19:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733042.1139156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClpV-0005GB-Ud; Thu, 30 May 2024 19:52:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733042.1139156; Thu, 30 May 2024 19:52:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClpV-0005G4-QY; Thu, 30 May 2024 19:52:53 +0000
Received: by outflank-mailman (input) for mailman id 733042;
 Thu, 30 May 2024 19:52:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sClpU-0005Fw-7M
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 19:52:52 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3253247d-1ebe-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 21:52:51 +0200 (CEST)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a59a352bbd9so204601966b.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 12:52:51 -0700 (PDT)
Received: from [10.125.231.30] ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57a31bb842asm153358a12.36.2024.05.30.12.52.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 12:52:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3253247d-1ebe-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717098771; x=1717703571; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=66sYQHl4BCSEPgMTbi+xgnu8w6fvK5zkPBldVjlOX6M=;
        b=ghWLoeKcqWURWfHiMdYyWuyn8NXPFqIBocHvb+0b9Q1pe8xIxaF+K2CoMbIQJGTtv/
         bd+2nRwLZLaCSB9MrYB+YqSTG/NujDDBKtXRLZXv8A+L1CEdiTic4zaeWED7FRpRJZoD
         AfgOwHNOZuWiZVIysg0SinOhFgE66vP1EDDF4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717098771; x=1717703571;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=66sYQHl4BCSEPgMTbi+xgnu8w6fvK5zkPBldVjlOX6M=;
        b=Scndn+bsTwO81oTOZe2DhviRWQEcn3V/IXWLB2JGMYLr7FIJw2xdgvRR5yW93Gs7fm
         r7/7u9wnKG1B+Z2dIYekNWWMQPWSiYEW0JyXa+lHp+V3UNW9fKYf0AOpor91w956jJpq
         /tCdq14oJWA8Q29UZdhdZrAkbU0ZK/U/PrY1Yw35+ly+pj4Wofj+QA+VZKcinKnuJgmX
         5OIHH1PU9NYfJUo/uwkHcr3zLAj1XOe9jiJKnsth3CcH728k8HpMw9ywrNb22Q1ivjsd
         Boq3Y84IiHvUbP6R4yETXuwf/QpceSAY4Jz6QaS5G+rBeyGtcbmFLZcq7HP/U9aia6kV
         XPYQ==
X-Forwarded-Encrypted: i=1; AJvYcCVqzmMun2+lRH+8J0WwH0D1Wx2/zV9wNVqiLAPQd9Nfv5NKjIQlP3HQCIXMMWxPipmUZ0nlFiBTTs2Enwx3PP+76yo1KqA5S6Yev1xmBFQ=
X-Gm-Message-State: AOJu0YzlIhGSCzTeGd+tzEZkKp6IHqBchbPM1L6iVn3UHdDnb6BpdTIt
	+BzP1Xgcn1IGPRdGTNzKOoRLjG71t5DFZbWRKFtqouy75KtcK+jkTtC3OjVgBpI=
X-Google-Smtp-Source: AGHT+IGgkzz7Rx7rkwy6I6mOUC3zd9eHKsyLg3uCoIC8shlqg6maXpZBnBdS+omtHnnbC462p/zuxQ==
X-Received: by 2002:a17:906:c2d8:b0:a59:bf27:5f2e with SMTP id a640c23a62f3a-a65f0a56cb7mr192344466b.20.1717098770602;
        Thu, 30 May 2024 12:52:50 -0700 (PDT)
Message-ID: <d4e5b4c8-d494-440b-8970-488b49bee12e@citrix.com>
Date: Thu, 30 May 2024 20:52:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>, Jan Beulich
 <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
> diff --git a/README b/README
> index c8a108449e..30da5ff9c0 100644
> --- a/README
> +++ b/README
> @@ -48,6 +48,10 @@ provided by your OS distributor:
>        - For ARM 64-bit:
>          - GCC 5.1 or later
>          - GNU Binutils 2.24 or later
> +      - For RISC-V 64-bit:
> +        - GCC 12.2 or later
> +        - GNU Binutils 2.39 or later

I would like to petition for GCC 10 and Binutils 2.35.

These are the versions provided as cross-compilers by Debian, and
therefore are the versions I would prefer to do smoke testing with.

One issue is in cpu_relax(), needing this diff to fix:

diff --git a/xen/arch/riscv/include/asm/processor.h
b/xen/arch/riscv/include/asm/processor.h
index 6846151717f7..830a05dd8e3a 100644
--- a/xen/arch/riscv/include/asm/processor.h
+++ b/xen/arch/riscv/include/asm/processor.h
@@ -67,7 +67,7 @@ static inline void cpu_relax(void)
     __asm__ __volatile__ ( "pause" );
 #else
     /* Encoding of the pause instruction */
-    __asm__ __volatile__ ( ".insn 0x0100000F" );
+    __asm__ __volatile__ ( ".4byte 0x0100000F" );
 #endif
 
     barrier();

The .insn directive appears to check that the byte pattern is a known
extension, where .4byte doesn't.  AFAICT, this makes .insn pretty
useless for what I'd expect is it's main purpose...


The other problem is a real issue in cmpxchg.h, already committed to
staging (51dabd6312c).

RISC-V does a conditional toolchain for the Zbb extension
(xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction
in emulate_xchg_1_2().

Nevertheless, this is also quite easy to fix:

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h
b/xen/arch/riscv/include/asm/cmpxchg.h
index d5e678c03678..12ecb0950701 100644
--- a/xen/arch/riscv/include/asm/cmpxchg.h
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -18,6 +18,20 @@
         : "r" (new) \
         : "memory" );
 
+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
old, form
+ * it of a NOT+AND pair
+ */
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "not " rd ", " rs2 "\n"                     \
+    "and " rd ", " rs1 ", " rd "\n"
+#endif
+
+
 /*
  * For LR and SC, the A extension requires that the address held in rs1 be
  * naturally aligned to the size of the operand (i.e., eight-byte aligned
@@ -48,7 +62,7 @@
     \
     asm volatile ( \
         "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
-        "   andn  %[scratch], %[old], %[mask]\n" \
+        ANDN_INSN("%[scratch]", "%[old]", "%[mask]") \
         "   or   %[scratch], %[scratch], %z[new_]\n" \
         "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
         "   bnez %[scratch], 0b\n" \



And with that, everything builds... but doesn't link.  I've got this:

  LDS     arch/riscv/xen.lds
riscv64-linux-gnu-ld      -T arch/riscv/xen.lds -N prelink.o \
    ./common/symbols-dummy.o -o ./.xen-syms.0
riscv64-linux-gnu-ld: prelink.o: in function `keyhandler_crash_action':
/local/xen.git/xen/common/keyhandler.c:552: undefined reference to
`guest_physmap_remove_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol
`guest_physmap_remove_page' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value

which is completely bizarre.

keyhandler_crash_action() has no actual reference to
guest_physmap_remove_page(), and keyhandler.o has no such relocation.

I'm still investigating this one.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu May 30 20:02:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 20:02:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733064.1139164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClyc-00072c-Sy; Thu, 30 May 2024 20:02:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733064.1139164; Thu, 30 May 2024 20:02:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sClyc-00072V-QP; Thu, 30 May 2024 20:02:18 +0000
Received: by outflank-mailman (input) for mailman id 733064;
 Thu, 30 May 2024 20:02:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sClya-00072P-W6
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 20:02:17 +0000
Received: from 008.lax.mailroute.net (008.lax.mailroute.net [199.89.1.11])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 809cc256-1ebf-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 22:02:13 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 008.lax.mailroute.net (Postfix) with ESMTP id 4Vqxwz1Qt3z6Cnk9T;
 Thu, 30 May 2024 20:02:11 +0000 (UTC)
Received: from 008.lax.mailroute.net ([127.0.0.1])
 by localhost (008.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id TbhOJyciqa7X; Thu, 30 May 2024 20:02:06 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 008.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqxwr3ZGLz6Cnk9F;
 Thu, 30 May 2024 20:02:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 809cc256-1ebf-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717099326; x=1719691327; bh=mbOtu3q9zjV5nBmJEGmUM2Ch
	lXZYTtjXht6IjdlxuYo=; b=F/EUZ7ZoluH2NApSjB6LS4U+IseVIwCjj1+dB0GQ
	Hzd/jksY/Fw5Zro8gliSKkstCF1FjokCnbsfexxZKJdkTxEAEPIYDNL8Qr6xEnV9
	rJ7hsYcQWK7x55L8pxb/1Sxg7HUnHdJTyGXoCBAKmGZJnKsfm9hEfmMr40tWPhVi
	H6OveqevfRhZsr+gmDFCxlYpIB9o9xQP2SQ9BKfuKoYQ2KM6iZ+h17N6LqEDIWcJ
	VpoHnGJGnHY0dmLgFjDoWjP/wCgsZwF0k5CVqOvxoLdxfHwd8lS4d6WQoZn/kYPz
	/dr2Nfj8gmo+hFuTTKGCcdGS7oOPWkNQF2/83ZmILBkUCA==
X-Virus-Scanned: by MailRoute
Message-ID: <6d52dd83-0cad-448e-8a22-b6f0e1bcac46@acm.org>
Date: Thu, 30 May 2024 13:02:02 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 06/12] sd: simplify the disable case in sd_config_discard
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-7-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-7-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Fall through to the main call to blk_queue_max_discard_sectors given that
> max_blocks has been initialized to zero above instead of duplicating the
> call.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 20:07:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 20:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733075.1139175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm3h-00088A-FG; Thu, 30 May 2024 20:07:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733075.1139175; Thu, 30 May 2024 20:07:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm3h-000882-Ch; Thu, 30 May 2024 20:07:33 +0000
Received: by outflank-mailman (input) for mailman id 733075;
 Thu, 30 May 2024 20:07:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sCm3g-00086D-4P
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 20:07:32 +0000
Received: from 009.lax.mailroute.net (009.lax.mailroute.net [199.89.1.12])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3e3de31e-1ec0-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 22:07:31 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 009.lax.mailroute.net (Postfix) with ESMTP id 4Vqy352wXTzlgMVW;
 Thu, 30 May 2024 20:07:29 +0000 (UTC)
Received: from 009.lax.mailroute.net ([127.0.0.1])
 by localhost (009.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id rlAwti_y4elY; Thu, 30 May 2024 20:07:24 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 009.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqy2z2TfbzlgMVV;
 Thu, 30 May 2024 20:07:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e3de31e-1ec0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717099644; x=1719691645; bh=+yfFxI5vXGgBtiGU+D99/ljE
	G0kpvZXmxQcSkudrr9s=; b=SqjKoZZdpT4MNXYDyRUM4JuIL15+lNbQrcfJuZsH
	5i+fsrz5hnUl5DlaKW9QovdJ8SbgPABGz/lXk+zTUX3IvVSOZF4hWm47ReZABvHU
	1s1mscyI3oBlHP5Trs1lvEnLVrL3MR78iVb0m7+89cOwsGpuvycL4ADEymuw7yOK
	9MD9bMpEl08HInq9MOQgt8+vi3M9lBRE3UqeLb9J4kdp+Fh7/qcGLKjRlZCsb8Bx
	RHiqCZS/DM7HHcWymvBmLE2Q9l8U7sARF6UgjfX76g9qvw1SryWswVnTVCdSLau+
	ynch4c8cn09wBIf16ogwWgDF+qVJWH9byKOSFXu+7Qieag==
X-Virus-Scanned: by MailRoute
Message-ID: <53fc8442-e27b-4e05-b93e-7b39b9146cce@acm.org>
Date: Thu, 30 May 2024 13:07:22 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 08/12] sd: cleanup zoned queue limits initialization
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-9-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-9-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Consolidate setting zone-related queue limits in sd_zbc_read_zones
> instead of splitting them between sd_zbc_revalidate_zones and
> sd_zbc_read_zones, and move the early_zone_information initialization
> in sd_zbc_read_zones above setting up the queue limits.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 20:08:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 20:08:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733079.1139185 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm4S-0000Fv-Oq; Thu, 30 May 2024 20:08:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733079.1139185; Thu, 30 May 2024 20:08:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm4S-0000Fo-L2; Thu, 30 May 2024 20:08:20 +0000
Received: by outflank-mailman (input) for mailman id 733079;
 Thu, 30 May 2024 20:08:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sCm4R-0000Fe-DQ
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 20:08:19 +0000
Received: from 009.lax.mailroute.net (009.lax.mailroute.net [199.89.1.12])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5a99fc40-1ec0-11ef-90a1-e314d9c70b13;
 Thu, 30 May 2024 22:08:18 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 009.lax.mailroute.net (Postfix) with ESMTP id 4Vqy411LtgzlgMVW;
 Thu, 30 May 2024 20:08:17 +0000 (UTC)
Received: from 009.lax.mailroute.net ([127.0.0.1])
 by localhost (009.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id 1HkePWDZ3j6e; Thu, 30 May 2024 20:08:13 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 009.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqy3w21PZzlgMVV;
 Thu, 30 May 2024 20:08:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a99fc40-1ec0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717099693; x=1719691694; bh=HAP2WXDhccKrBY2e7nb7eVhK
	KMZLSAfPEEOn8QVjvBM=; b=obeHdmp9YsUXuOZQLqScMEmH6Ih7ZfY/4iMCkRPI
	LYigt/zOg1B/psGhSXL1A6Nun9Xb4ACnTmneav70Y/d2lCEMW1iyAULS0JOi7N+M
	4D0Z++5JAW5CvV9F2QsCWiTxD25HQJrjl8jRDIIicgxUnyiQBaH4qocTpaXWsRpQ
	NFHwx9+r/3FZrmIkDJAMiSIYHaIZ4Dz1EiaeUipwbed35LdTnRzrbsLX4Btd/RQ/
	RHjxP9YfAad0XG4VPmEW2MotDp1hU4Cf4txtLNGvfUWjW59v/jqig6nN1/pf0pbD
	Jb5Rn6a1EQX7UKGspz9POkgxHq+rMXXDAO1vlh1wffB6ow==
X-Virus-Scanned: by MailRoute
Message-ID: <f4bf4ccd-04fe-4067-a658-270887abb50d@acm.org>
Date: Thu, 30 May 2024 13:08:11 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 11/12] block: remove unused queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-12-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-12-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> Remove all APIs that are unused now that sd and sr have been converted
> to the atomic queue limits API.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 20:09:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 20:09:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733082.1139194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm5W-0000vL-1B; Thu, 30 May 2024 20:09:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733082.1139194; Thu, 30 May 2024 20:09:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCm5V-0000vE-Uq; Thu, 30 May 2024 20:09:25 +0000
Received: by outflank-mailman (input) for mailman id 733082;
 Thu, 30 May 2024 20:09:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/CNF=NB=acm.org=bvanassche@srs-se1.protection.inumbo.net>)
 id 1sCm5V-0000v8-1D
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 20:09:25 +0000
Received: from 009.lax.mailroute.net (009.lax.mailroute.net [199.89.1.12])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 80c01328-1ec0-11ef-b4bb-af5377834399;
 Thu, 30 May 2024 22:09:22 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by 009.lax.mailroute.net (Postfix) with ESMTP id 4Vqy5F1Q97zlgMVW;
 Thu, 30 May 2024 20:09:21 +0000 (UTC)
Received: from 009.lax.mailroute.net ([127.0.0.1])
 by localhost (009.lax [127.0.0.1]) (mroute_mailscanner, port 10029) with LMTP
 id AYBsZ1kn-wfP; Thu, 30 May 2024 20:09:17 +0000 (UTC)
Received: from [100.96.154.26] (unknown [104.132.0.90])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 (Authenticated sender: bvanassche@acm.org)
 by 009.lax.mailroute.net (Postfix) with ESMTPSA id 4Vqy5720mhzlgMVV;
 Thu, 30 May 2024 20:09:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80c01328-1ec0-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acm.org; h=
	content-transfer-encoding:content-type:content-type:in-reply-to
	:from:from:content-language:references:subject:subject
	:user-agent:mime-version:date:date:message-id:received:received;
	 s=mr01; t=1717099757; x=1719691758; bh=72TYEp9qG+JFxDJOhc8vL98i
	94kXdGccHRpnAQFkti8=; b=nQibsLkJj+e718qU89zq/w3b+Xj7bZcJX1t4S5Ib
	6Y2TqQaSrn/m9KkYDg8nG47BrzEZi5y6e/fkqJH0jKclSqaBOiW4YJTNeGb1Sp0q
	ZkAl4cqRGtJ9BhVg3y0q9c8DIYnTeW34UN3c12ZZq8UlXTpwStkplQ+/oORhUCu1
	CTLLln6gMBNxvppMEcu/G6wtlrVMp25vVLoBMfI4CsCDWdrrCAO59XEhKrT6cbhE
	Cef818A+XIhDPVWZrIoeq54kVP/vsyNbked97rhR5i8C7RP7qmtB77Ne3Id9P9eD
	p014OI2YgEP5j/ozLgrC3mtdb8oHumiTveZFFAH+BafSnw==
X-Virus-Scanned: by MailRoute
Message-ID: <12bdbf52-d97c-4c47-8e8b-bfc10bbdc985@acm.org>
Date: Thu, 30 May 2024 13:09:14 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 12/12] block: add special APIs for run-time disabling of
 discard and friends
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
 "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
 Anton Ivanov <anton.ivanov@cambridgegreys.com>,
 Johannes Berg <johannes@sipsolutions.net>, Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
 Dongsheng Yang <dongsheng.yang@easystack.cn>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 linux-um@lists.infradead.org, linux-block@vger.kernel.org,
 nbd@other.debian.org, ceph-devel@vger.kernel.org,
 xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-13-hch@lst.de>
Content-Language: en-US
From: Bart Van Assche <bvanassche@acm.org>
In-Reply-To: <20240529050507.1392041-13-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/28/24 22:04, Christoph Hellwig wrote:
> A few drivers optimistically try to support discard, write zeroes and
> secure erase and disable the features from the I/O completion handler
> if the hardware can't support them.  This disable can't be done using
> the atomic queue limits API because the I/O completion handlers can't
> take sleeping locks or freezer the queue.  Keep the existing clearing
> of the relevant field to zero, but replace the old blk_queue_max_*
> APIs with new disable APIs that force the value to 0.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>



From xen-devel-bounces@lists.xenproject.org Thu May 30 22:23:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 22:23:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733106.1139204 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCoAW-0004Hc-AB; Thu, 30 May 2024 22:22:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733106.1139204; Thu, 30 May 2024 22:22:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCoAW-0004HV-7F; Thu, 30 May 2024 22:22:44 +0000
Received: by outflank-mailman (input) for mailman id 733106;
 Thu, 30 May 2024 22:22:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCoAU-0004HL-Ea; Thu, 30 May 2024 22:22:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCoAU-0000r7-AN; Thu, 30 May 2024 22:22:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCoAT-00018V-Ti; Thu, 30 May 2024 22:22:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCoAT-00040A-TF; Thu, 30 May 2024 22:22:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wQlDOYsx0biTMABJ4qUXDAQQ6iOLIp8ap4ulqGZSj5U=; b=t1S6KHKLqL4VzQi+V/P9jfUYdp
	fFiwnMjI2OhxpJvJVGD0G1f5wFI/bsD0MHqKVXSvJ0Qpj0vAAQLgUFvI60mHfmunNXdM0KrO+giYW
	1CcfIgN3DGNTQjHlGLJbfL2/+Oa1a1LzpcbRJEEwHh6Du6LQsNAQHzn0xkQ7H/qQsH9s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186196-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186196: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-vhd:xen-boot:fail:regression
    linux-linus:build-amd64-xsm:xen-build:fail:regression
    linux-linus:build-amd64:xen-build:fail:regression
    linux-linus:build-i386:xen-build:fail:regression
    linux-linus:build-i386-xsm:xen-build:fail:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-bios:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:build-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:build-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine-uefi:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-examine:reboot:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4a4be1ad3a6efea16c56615f31117590fd881358
X-Osstest-Versions-That:
    linux=e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 22:22:41 +0000

flight 186196 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186196/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-vhd  8 xen-boot                 fail REGR. vs. 186174
 build-amd64-xsm               6 xen-build      fail in 186185 REGR. vs. 186174
 build-amd64                   6 xen-build      fail in 186185 REGR. vs. 186174
 build-i386                    6 xen-build      fail in 186185 REGR. vs. 186174
 build-i386-xsm                6 xen-build      fail in 186185 REGR. vs. 186174

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  8 xen-boot        fail in 186185 pass in 186196
 test-armhf-armhf-libvirt      8 xen-boot                   fail pass in 186185
 test-armhf-armhf-xl           8 xen-boot                   fail pass in 186189
 test-amd64-amd64-xl-pvhv2-amd 22 guest-start/debian.repeat fail pass in 186189

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-vhd  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)    blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-examine      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-pair         1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)      blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)         blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-examine-bios  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-xl-vhd       1 build-check(1)           blocked in 186185 n/a
 build-i386-libvirt            1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)   blocked in 186185 n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)        blocked in 186185 n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)      blocked in 186185 n/a
 test-amd64-amd64-pygrub       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-xl           1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-libvirt      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-raw       1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 186185 n/a
 build-amd64-libvirt           1 build-check(1)           blocked in 186185 n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)        blocked in 186185 n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)          blocked in 186185 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 1 build-check(1) blocked in 186185 n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked in 186185 n/a
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 186185 like 186174
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 186185 never pass
 test-armhf-armhf-xl-rtds      8 xen-boot            fail in 186189 like 186174
 test-armhf-armhf-xl-credit1   8 xen-boot            fail in 186189 like 186174
 test-armhf-armhf-xl         15 migrate-support-check fail in 186189 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 186189 never pass
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 186189 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 186189 never pass
 test-armhf-armhf-xl-raw     14 migrate-support-check fail in 186189 never pass
 test-armhf-armhf-xl-raw 15 saverestore-support-check fail in 186189 never pass
 test-armhf-armhf-xl-raw       8 xen-boot                     fail  like 186174
 test-armhf-armhf-xl-arndale   8 xen-boot                     fail  like 186174
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186174
 test-armhf-armhf-examine      8 reboot                       fail  like 186174
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186174
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4a4be1ad3a6efea16c56615f31117590fd881358
baseline version:
 linux                e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc

Last test of basis   186174  2024-05-28 18:10:31 Z    2 days
Testing same since   186185  2024-05-29 17:42:32 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dominique Martinet <asmadeus@codewreck.org>
  Herbert Xu <herbert@gondor.apana.org.au>
  Linus Torvalds <torvalds@linux-foundation.org>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 4a4be1ad3a6efea16c56615f31117590fd881358
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:39:34 2024 -0700

    Revert "vfs: Delete the associated dentry when deleting a file"
    
    This reverts commit 681ce8623567ba7e7333908e9826b77145312dda.
    
    We gave it a try, but it turns out the kernel test robot did in fact
    find performance regressions for it, so we'll have to look at the more
    involved alternative fixes for Yafang Shao's Elasticsearch load issue.
    
    There were several alternatives discussed, they just weren't as simple
    as this first attempt.
    
    The report is of a -7.4% regression of filebench.sum_operations/s, which
    appears significant enough to trigger my "this patch may get reverted if
    somebody finds a performance regression on some other load" rule.
    
    So it's still the case that we should end up deleting dentries more
    aggressively - or just be better at pruning them later - but it needs a
    bit more finesse than this simple thing.
    
    Link: https://lore.kernel.org/all/202405291318.4dfbb352-oliver.sang@intel.com/
    Cc: Yafang Shao <laoar.shao@gmail.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: Christian Brauner <brauner@kernel.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 397a83ab978553ca2970ad1ccdbac0cdc732efd9
Merge: db163660b02a c898afdc1564
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:25:15 2024 -0700

    Merge tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux
    
    Pull 9p fixes from Dominique Martinet:
     "Two fixes headed to stable trees:
    
       - a trace event was dumping uninitialized values
    
       - a missing lock that was thought to have exclusive access, and it
         turned out not to"
    
    * tag '9p-for-6.10-rc2' of https://github.com/martinetd/linux:
      9p: add missing locking around taking dentry fid list
      net/9p: fix uninit-value in p9_client_rpc()

commit db163660b02abbffebfad1bcd6dbce1201c72731
Merge: e0cce98fe279 67ec8cdf2997
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed May 29 09:12:58 2024 -0700

    Merge tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
    
    Pull crypto fix from Herbert Xu:
     "This fixes a new run-time warning triggered by tpm"
    
    * tag 'v6.10-p3' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      hwrng: core - Remove add_early_randomness

commit 67ec8cdf29971677b2fb4b6d92871eb5d5e95597
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date:   Wed May 22 13:37:54 2024 +0800

    hwrng: core - Remove add_early_randomness
    
    A potential deadlock was reported with the config file at
    
    https://web.archive.org/web/20240522052129/https://0x0.st/XPN_.txt
    
    In this particular configuration, the deadlock doesn't exist because
    the warning triggered at a point before modules were even available.
    However, the deadlock can be real because any module loaded would
    invoke async_synchronize_full.
    
    The issue is spurious for software crypto algorithms which aren't
    themselves involved in async probing.  However, it would be hard to
    avoid for a PCI crypto driver using async probing.
    
    In this particular call trace, the problem is easily avoided because
    the only reason the module is being requested during probing is the
    add_early_randomness call in the hwrng core.  This feature is
    vestigial since there is now a kernel thread dedicated to doing
    exactly this.
    
    So remove add_early_randomness as it is no longer needed.
    
    Reported-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Reported-by: Eric Biggers <ebiggers@kernel.org>
    Fixes: 1b6d7f9eb150 ("tpm: add session encryption protection to tpm2_get_random()")
    Link: https://lore.kernel.org/r/119dc5ed-f159-41be-9dda-1a056f29888d@notapiano/
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
    Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
    Tested-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

commit c898afdc15645efb555acb6d85b484eb40a45409
Author: Dominique Martinet <asmadeus@codewreck.org>
Date:   Tue May 21 21:13:36 2024 +0900

    9p: add missing locking around taking dentry fid list
    
    Fix a use-after-free on dentry's d_fsdata fid list when a thread
    looks up a fid through dentry while another thread unlinks it:
    
    UAF thread:
    refcount_t: addition on 0; use-after-free.
     p9_fid_get linux/./include/net/9p/client.h:262
     v9fs_fid_find+0x236/0x280 linux/fs/9p/fid.c:129
     v9fs_fid_lookup_with_uid linux/fs/9p/fid.c:181
     v9fs_fid_lookup+0xbf/0xc20 linux/fs/9p/fid.c:314
     v9fs_vfs_getattr_dotl+0xf9/0x360 linux/fs/9p/vfs_inode_dotl.c:400
     vfs_statx+0xdd/0x4d0 linux/fs/stat.c:248
    
    Freed by:
     p9_fid_destroy (inlined)
     p9_client_clunk+0xb0/0xe0 linux/net/9p/client.c:1456
     p9_fid_put linux/./include/net/9p/client.h:278
     v9fs_dentry_release+0xb5/0x140 linux/fs/9p/vfs_dentry.c:55
     v9fs_remove+0x38f/0x620 linux/fs/9p/vfs_inode.c:518
     vfs_unlink+0x29a/0x810 linux/fs/namei.c:4335
    
    The problem is that d_fsdata was not accessed under d_lock, because
    d_release() normally is only called once the dentry is otherwise no
    longer accessible but since we also call it explicitly in v9fs_remove
    that lock is required:
    move the hlist out of the dentry under lock then unref its fids once
    they are no longer accessible.
    
    Fixes: 154372e67d40 ("fs/9p: fix create-unlink-getattr idiom")
    Cc: stable@vger.kernel.org
    Reported-by: Meysam Firouzi
    Reported-by: Amirmohammad Eftekhar
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Message-ID: <20240521122947.1080227-1-asmadeus@codewreck.org>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>

commit 25460d6f39024cc3b8241b14c7ccf0d6f11a736a
Author: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
Date:   Mon Apr 8 07:10:39 2024 -0700

    net/9p: fix uninit-value in p9_client_rpc()
    
    Syzbot with the help of KMSAN reported the following error:
    
    BUG: KMSAN: uninit-value in trace_9p_client_res include/trace/events/9p.h:146 [inline]
    BUG: KMSAN: uninit-value in p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     trace_9p_client_res include/trace/events/9p.h:146 [inline]
     p9_client_rpc+0x1314/0x1340 net/9p/client.c:754
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    Uninit was created at:
     __alloc_pages+0x9d6/0xe70 mm/page_alloc.c:4598
     __alloc_pages_node include/linux/gfp.h:238 [inline]
     alloc_pages_node include/linux/gfp.h:261 [inline]
     alloc_slab_page mm/slub.c:2175 [inline]
     allocate_slab mm/slub.c:2338 [inline]
     new_slab+0x2de/0x1400 mm/slub.c:2391
     ___slab_alloc+0x1184/0x33d0 mm/slub.c:3525
     __slab_alloc mm/slub.c:3610 [inline]
     __slab_alloc_node mm/slub.c:3663 [inline]
     slab_alloc_node mm/slub.c:3835 [inline]
     kmem_cache_alloc+0x6d3/0xbe0 mm/slub.c:3852
     p9_tag_alloc net/9p/client.c:278 [inline]
     p9_client_prepare_req+0x20a/0x1770 net/9p/client.c:641
     p9_client_rpc+0x27e/0x1340 net/9p/client.c:688
     p9_client_create+0x1551/0x1ff0 net/9p/client.c:1031
     v9fs_session_init+0x1b9/0x28e0 fs/9p/v9fs.c:410
     v9fs_mount+0xe2/0x12b0 fs/9p/vfs_super.c:122
     legacy_get_tree+0x114/0x290 fs/fs_context.c:662
     vfs_get_tree+0xa7/0x570 fs/super.c:1797
     do_new_mount+0x71f/0x15e0 fs/namespace.c:3352
     path_mount+0x742/0x1f20 fs/namespace.c:3679
     do_mount fs/namespace.c:3692 [inline]
     __do_sys_mount fs/namespace.c:3898 [inline]
     __se_sys_mount+0x725/0x810 fs/namespace.c:3875
     __x64_sys_mount+0xe4/0x150 fs/namespace.c:3875
     do_syscall_64+0xd5/0x1f0
     entry_SYSCALL_64_after_hwframe+0x6d/0x75
    
    If p9_check_errors() fails early in p9_client_rpc(), req->rc.tag
    will not be properly initialized. However, trace_9p_client_res()
    ends up trying to print it out anyway before p9_client_rpc()
    finishes.
    
    Fix this issue by assigning default values to p9_fcall fields
    such as 'tag' and (just in case KMSAN unearths something new) 'id'
    during the tag allocation stage.
    
    Reported-and-tested-by: syzbot+ff14db38f56329ef68df@syzkaller.appspotmail.com
    Fixes: 348b59012e5c ("net/9p: Convert net/9p protocol dumps to tracepoints")
    Signed-off-by: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
    Reviewed-by: Christian Schoenebeck <linux_oss@crudebyte.com>
    Cc: stable@vger.kernel.org
    Message-ID: <20240408141039.30428-1-n.zhandarovich@fintech.ru>
    Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>


From xen-devel-bounces@lists.xenproject.org Thu May 30 22:52:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 22:52:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733118.1139215 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCod9-000102-PI; Thu, 30 May 2024 22:52:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733118.1139215; Thu, 30 May 2024 22:52:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCod9-0000zv-Ln; Thu, 30 May 2024 22:52:19 +0000
Received: by outflank-mailman (input) for mailman id 733118;
 Thu, 30 May 2024 22:52:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCod8-0000zl-LL; Thu, 30 May 2024 22:52:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCod8-0001Ja-GK; Thu, 30 May 2024 22:52:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCod8-00025h-7i; Thu, 30 May 2024 22:52:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCod8-0006Tu-7K; Thu, 30 May 2024 22:52:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=npukHvQg+zMfnNXDn08u99FUhQPxlxvrlwF7AR1efvY=; b=AFBS18kjND7d7zjFG/C+d7F7pR
	+YZBMsrIvh1UqxQSqh6CX53juNNdasl+bNt5C+2oRryc7cFU/dJcGnwC5UNVvX/72DzbBHymJ6vF+
	zL1yKqLQy9HKCjyXgVasqFi1VFSItEagQ5rvBVpWccpAKUDtW1nHjc4IPV0POstU/A6k=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186201-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186201: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=ced13b93afea87a8a1fe6ddbb67240a84cb2e3d3
X-Osstest-Versions-That:
    ovmf=e7848481160b270ebc59d68ecbc8d2722e3aed8c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 May 2024 22:52:18 +0000

flight 186201 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186201/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 ced13b93afea87a8a1fe6ddbb67240a84cb2e3d3
baseline version:
 ovmf                 e7848481160b270ebc59d68ecbc8d2722e3aed8c

Last test of basis   186199  2024-05-30 17:14:30 Z    0 days
Testing same since   186201  2024-05-30 21:13:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Sam <Sam_Tsai@wiwynn.com>
  Sam Tsai [Wiwynn] <sam_tsai@wiwynn.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   e784848116..ced13b93af  ced13b93afea87a8a1fe6ddbb67240a84cb2e3d3 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu May 30 23:59:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 May 2024 23:59:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733130.1139224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCpfq-00017I-1f; Thu, 30 May 2024 23:59:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733130.1139224; Thu, 30 May 2024 23:59:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCpfp-00017B-Uw; Thu, 30 May 2024 23:59:09 +0000
Received: by outflank-mailman (input) for mailman id 733130;
 Thu, 30 May 2024 23:59:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8pxM=NB=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCpfp-000175-Dk
 for xen-devel@lists.xenproject.org; Thu, 30 May 2024 23:59:09 +0000
Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com
 [2607:f8b0:4864:20::831])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 99c18c1e-1ee0-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 01:59:08 +0200 (CEST)
Received: by mail-qt1-x831.google.com with SMTP id
 d75a77b69052e-43fdf8a6437so2289171cf.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 16:59:07 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 d75a77b69052e-43ff23bfa2dsm2872701cf.22.2024.05.30.16.59.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 16:59:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99c18c1e-1ee0-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717113547; x=1717718347; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=AT2rc5LODno5TFOto2QjQ8ousrDMjzLxgKW1mhEZVuo=;
        b=io0c44K4d5jcWMTVA4zqbZj/te0n2nNr4zakVi9Q/K3LZ3+0bMStUDYKVdclcoho1i
         2uhdHu6nn1tL+ldU/68igsDR2VqUBp8q6ba3OQUmf+5iZ4D5zWsChpvsXiKVIUPqJmVl
         we6Ncmj224vZixLIEH8wzzJ0IPCzXlfLMN9oI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717113547; x=1717718347;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=AT2rc5LODno5TFOto2QjQ8ousrDMjzLxgKW1mhEZVuo=;
        b=laTeyB3dP2ctLn5TvM4IzXci7NogKDuJimgtB8wSfIGzXJ3Mnoyg6dIJ3Yuv5rRp3A
         FxYwd/+ObqHpMcJVfJgQCa6c2vacwlzeLuDstXZac6ivEF5PuqiNHdpy0MBGyE4WpJYI
         y1FsuoEg05SpiiCzp/iJAF3BPUb+FyThZzqulh8nL1a6kNDsPvn98gVmyFSzmIISZQ+M
         EZVEzRGbV8DZVQsnpXcTjVGrAIvTe2wxI66ogISysDPCjq7uh5164qHWPcGoNsclQ38G
         yHDFUmGFHMubljtHHgRLdzdDbmi1oExhV3WHHqCsXl0Mjjt6IVnUeokw9LJLplWHVnzg
         lJZA==
X-Forwarded-Encrypted: i=1; AJvYcCVIwSVx6Eg/v6wwXYEhBgOCRzBPPWj4WbZsPsWvYmqgeh5xcMsvhx0dyp7b3L+S6BDvmPcGG6XrLxWewBlO4Qix1LwbKvOnrwwIMbEOzRw=
X-Gm-Message-State: AOJu0YzcZOjoVn5gEHy/urXNwxl7wVp6F2W7UGt1eR4qlaAtmQnTj+T9
	/rEUBAwT9WjEGJb8eaMiLp3hrfaWC3Swj6+8ofNHDnvzkP+bcJkmOmtqMJc7wuc=
X-Google-Smtp-Source: AGHT+IGWztHCZH+bZwACsZuBGkg0Kz9R39QrzM2O7itMqQF8NTIKbFpFbmro2+iLP7zJEIj5Cg8snw==
X-Received: by 2002:a05:622a:164c:b0:43f:e7c7:a4e2 with SMTP id d75a77b69052e-43ff522d242mr3280341cf.5.1717113546244;
        Thu, 30 May 2024 16:59:06 -0700 (PDT)
Message-ID: <497624e9-053a-407e-86f0-e3d2c8883cd7@citrix.com>
Date: Fri, 31 May 2024 00:59:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 1/2] tools/xg: Streamline cpu policy
 serialise/deserialise calls
To: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Anthony PERARD <anthony@xenproject.org>, Juergen Gross <jgross@suse.com>,
 Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1716992707.git.alejandro.vallejo@cloud.com>
 <f456bfb8996bb1fd4b965755622cda6fcb61b297.1716992707.git.alejandro.vallejo@cloud.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <f456bfb8996bb1fd4b965755622cda6fcb61b297.1716992707.git.alejandro.vallejo@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/05/2024 3:30 pm, Alejandro Vallejo wrote:
> diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
> index e01f494b772a..85d56f26537b 100644
> --- a/tools/include/xenguest.h
> +++ b/tools/include/xenguest.h
> @@ -799,15 +799,23 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy);
>  
>  /* Manipulate a policy via architectural representations. */
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
>  int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
>                                 const xen_cpuid_leaf_t *leaves,
>                                 uint32_t nr);
>  int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>                                const xen_msr_entry_t *msrs, uint32_t nr);
>  
> +/*
> + * Accessors for the serialised forms of the policy. The outputs are pointers
> + * into the policy object and not fresh allocations, so their lifetimes are tied
> + * to the policy object itself.

This is far more complicated.  See below.

> + */
> +int xc_cpu_policy_get_leaves(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                             const xen_cpuid_leaf_t **leaves, uint32_t *nr);
> +int xc_cpu_policy_get_msrs(xc_interface *xch, const xc_cpu_policy_t *policy,
> +                           const xen_msr_entry_t **msrs, uint32_t *nr);
> +
>  /* Compatibility calculations. */
>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
>                                   xc_cpu_policy_t *guest);
> diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
> index 4453178100ad..6cab5c60bb41 100644
> --- a/tools/libs/guest/xg_cpuid_x86.c
> +++ b/tools/libs/guest/xg_cpuid_x86.c
> @@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
>      }
>  }
>  
> -static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
> -                              unsigned int nr_leaves, unsigned int nr_entries)
> +static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
>      int rc;
>  
>      rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
> -                                    nr_leaves, &err_leaf, &err_subleaf);
> +                                    policy->nr_leaves, &err_leaf, &err_subleaf);
>      if ( rc )
>      {
>          if ( err_leaf != -1 )

Urgh - this is a mess.  (Not your fault, but we really need to think
twice before continuing.)

xc_cpu_policy_serialise() is an exported function and, prior to this
series, used by external entities to get at the content inside the
opaque object.

deserialize_policy()  (Clearly not written by me - Roger?) is a local
helper.  Also it looks wonky in the next patch, although I think that's
just code movement to avoid a forward declaration?

By the end of the series, xc_cpu_policy_serialise() isn't used
externally, but it's still exported.

But, besides the visibility, there's a second difference...


> @@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
>      }
>  
>      rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
> -                                  nr_entries, &err_msr);
> +                                  policy->nr_msrs, &err_msr);
>      if ( rc )
>      {
>          if ( err_msr != -1 )
> @@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);

... they're asymmetric as to whether the caller or the callee preloads
policy->nr_*.

Both of these need rationalising, one way or another.


But, there's a related problem.

Previously there was only one canonical form (the deserialised form),
and anything operating on state was responsible for getting it back to
being the deserialised form.

Now, there are two forms which are coexist side by side.  The buffer
exposed by get_{cpuid,msr}() is only good until the next operation which
uses what were (previously) the internal staging buffer(s).

And that makes it a fragile and error prone interface.

>      if ( rc )
>      {
>          errno = -rc;
> @@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
>          return rc;
>      }
>  
> -    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
> +    policy->nr_leaves = nr_leaves;
> +    policy->nr_msrs = nr_msrs;
> +
> +    rc = deserialize_policy(xch, policy);
>      if ( rc )
>      {
>          errno = -rc;
> @@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>                               xc_cpu_policy_t *policy)
>  {
>      uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
> -    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
> -    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
>      int rc;
>  
> -    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
> -                                 policy->msrs, &nr_msrs);
> +    rc = xc_cpu_policy_serialise(xch, policy);
>      if ( rc )
>          return rc;
>  
> -    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
> -                                  nr_msrs, policy->msrs,
> +    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
> +                                  policy->nr_msrs, policy->msrs,
>                                    &err_leaf, &err_subleaf, &err_msr);
>      if ( rc )
>      {
> @@ -942,32 +944,26 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
>      return rc;
>  }
>  
> -int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
> -                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
> -                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
> +int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
>  {
>      int rc;
> +    p->nr_leaves = ARRAY_SIZE(p->leaves);
> +    p->nr_msrs = ARRAY_SIZE(p->msrs);
>  
> -    if ( leaves )
> +    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &p->nr_leaves);
> +    if ( rc )
>      {
> -        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
> -        if ( rc )
> -        {
> -            ERROR("Failed to serialize CPUID policy");
> -            errno = -rc;
> -            return -1;
> -        }
> +        ERROR("Failed to serialize CPUID policy");
> +        errno = -rc;
> +        return -1;
>      }
>  
> -    if ( msrs )
> +    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &p->nr_msrs);
> +    if ( rc )
>      {
> -        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
> -        if ( rc )
> -        {
> -            ERROR("Failed to serialize MSR policy");
> -            errno = -rc;
> -            return -1;
> -        }
> +        ERROR("Failed to serialize MSR policy");
> +        errno = -rc;
> +        return -1;
>      }
>  
>      errno = 0;
> @@ -1012,6 +1008,42 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
>      return rc;
>  }
>  
> +int xc_cpu_policy_get_leaves(xc_interface *xch,
> +                             const xc_cpu_policy_t *policy,
> +                             const xen_cpuid_leaf_t **leaves,
> +                             uint32_t *nr)
> +{
> +    if ( !policy )
> +    {
> +        ERROR("Failed to fetch CPUID leaves from policy object");
> +        errno = -EINVAL;
> +        return -1;
> +    }

This check isn't useful, and it's making the interface inconsistent. 
There's no case ever where a NULL policy is meaningful, except for the
very initial failure to allocate, and there it's the return value not an
input parameter.

More importantly however, the error message is misleading as a consequence.

> +
> +    *leaves = policy->leaves;
> +    *nr = policy->nr_leaves;
> +
> +    return 0;
> +}
> +
> +int xc_cpu_policy_get_msrs(xc_interface *xch,
> +                           const xc_cpu_policy_t *policy,
> +                           const xen_msr_entry_t **msrs,
> +                           uint32_t *nr)
> +{
> +    if ( !policy )
> +    {
> +        ERROR("Failed to fetch MSRs from policy object");
> +        errno = -EINVAL;
> +        return -1;
> +    }
> +
> +    *msrs = policy->msrs;
> +    *nr = policy->nr_msrs;
> +
> +    return 0;
> +}
> +
>  bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
>                                   xc_cpu_policy_t *guest)
>  {
> diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
> index d73947094f2e..a65dae818f3d 100644
> --- a/tools/libs/guest/xg_private.h
> +++ b/tools/libs/guest/xg_private.h
> @@ -177,6 +177,8 @@ struct xc_cpu_policy {
>      struct cpu_policy policy;
>      xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
>      xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
> +    uint32_t nr_leaves;
> +    uint32_t nr_msrs;

These need a comment explaining how they're used, and sadly they have no
relationship to the lengths of the array.  There's a corner case where
they can end up larger.

>  };
>  #endif /* x86 */
>  
> diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
> index 563b4f016877..a0d67c3211c6 100644
> --- a/tools/libs/guest/xg_sr_common_x86.c
> +++ b/tools/libs/guest/xg_sr_common_x86.c
> @@ -1,4 +1,5 @@
>  #include "xg_sr_common_x86.h"
> +#include "xg_sr_stream_format.h"

I'm pretty sure this shouldn't be necessary.  Is it?

>  
>  int write_x86_tsc_info(struct xc_sr_context *ctx)
>  {
> @@ -45,54 +46,39 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
>  int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
>  {
>      xc_interface *xch = ctx->xch;
> -    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
> -    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
> -    uint32_t nr_leaves = 0, nr_msrs = 0;
> -    xc_cpu_policy_t *policy = NULL;
> +    xc_cpu_policy_t *policy = xc_cpu_policy_init();
>      int rc;
>  
> -    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
> -    {
> -        PERROR("Unable to get CPU Policy size");
> -        return -1;
> -    }
> -
> -    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
> -    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
> -    policy = xc_cpu_policy_init();
> -    if ( !cpuid.data || !msrs.data || !policy )
> -    {
> -        ERROR("Cannot allocate memory for CPU Policy");
> -        rc = -1;
> -        goto out;
> -    }
> -
> -    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
> +    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
>      {
>          PERROR("Unable to get d%d CPU Policy", ctx->domid);
>          rc = -1;
>          goto out;
>      }
> -    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
> -                                 msrs.data, &nr_msrs) )
> -    {
> -        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
> -        rc = -1;
> -        goto out;
> -    }

Wow, the old code here was especially daft.

We're having Xen serialise the policy, copying (double buffering) into
the policy object then desensitising.  And vs the old copy, we've got
rid of the re-serialise into yet another buffer.

But we should still be using a plain XEN_DOMCTL_get_cpu_policy here. 
Literally all we want to do is take the array(s) Xen gave us and feed
them straight into the fd.

deserialising is already a reasonably expensive operation (every
individual leaf coordinate needs re-range checking), and is only ever
going to get worse.

It will probably help to split the changes to
write_x86_cpu_policy_records() out into a separate patch.  It's more
clear cut and also addresses one of the local vs external issues
discussed above.


>  
> -    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
> -    if ( cpuid.length )
> +
> +    if ( policy->nr_leaves )
>      {
> -        rc = write_record(ctx, &cpuid);
> +        struct xc_sr_record record = {
> +            .type = REC_TYPE_X86_CPUID_POLICY,
> +            .data = policy->leaves,
> +            .length = policy->nr_leaves * sizeof(*policy->leaves),
> +        };
> +
> +        rc = write_record(ctx, &record);

Please keep this name being cpuid.  It's more helpful when grepping, and
it also shrinks the diff.

>          if ( rc )
>              goto out;
>      }
>  
> -    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
> -    if ( msrs.length )
> +    if ( policy->nr_msrs )
>      {
> -        rc = write_record(ctx, &msrs);
> +        struct xc_sr_record record = {
> +            .type = REC_TYPE_X86_MSR_POLICY,
> +            .data = policy->msrs,
> +            .length = policy->nr_msrs * sizeof(*policy->msrs),
> +        };
> +
> +        rc = write_record(ctx, &record);
>          if ( rc )
>              goto out;
>      }
> @@ -100,8 +86,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
>      rc = 0;
>  
>   out:
> -    free(cpuid.data);
> -    free(msrs.data);
>      xc_cpu_policy_destroy(policy);
>  
>      return rc;
> diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
> index 4c4593528dfe..488f43378406 100644
> --- a/tools/misc/xen-cpuid.c
> +++ b/tools/misc/xen-cpuid.c
> @@ -156,12 +156,18 @@ static void dump_info(xc_interface *xch, bool detail)
>  
>      free(fs);
>  }
> -

Stray (deleted) whitespace.

> -static void print_policy(const char *name,
> -                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
> -                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
> +static void print_policy(xc_interface *xch, const char *name,
> +                         const xc_cpu_policy_t *policy)
>  {
>      unsigned int l;
> +    const xen_cpuid_leaf_t *leaves;
> +    const xen_msr_entry_t *msrs;
> +    uint32_t nr_leaves, nr_msrs;
> +
> +    if ( xc_cpu_policy_get_leaves(xch, policy, &leaves, &nr_leaves) )
> +        err(1, "xc_cpu_policy_get_leaves()");
> +    if ( xc_cpu_policy_get_msrs(xch, policy, &msrs, &nr_msrs) )
> +        err(1, "xc_cpu_policy_get_msrs()");

Not an issue with here per say, but to drive home the main problem.

This doesn't return the current leaves/msrs.  It gives you whatever's
stale in the staging buffer, which happens to be ok in xen-cpuid because
it only ever reads a policy...

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:11:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:11:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733142.1139234 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprG-0004cl-BE; Fri, 31 May 2024 00:10:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733142.1139234; Fri, 31 May 2024 00:10:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprG-0004ce-7w; Fri, 31 May 2024 00:10:58 +0000
Received: by outflank-mailman (input) for mailman id 733142;
 Fri, 31 May 2024 00:10:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCprE-0004cW-Qr
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:10:56 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3da61046-1ee2-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 02:10:54 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 9D2FCCE1B96;
 Fri, 31 May 2024 00:10:50 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F37A3C2BBFC;
 Fri, 31 May 2024 00:10:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3da61046-1ee2-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717114249;
	bh=zu0XXiDpdoXFuaeqYoN3SExvsNMHcQwNXY3Dk1hEiFw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=AZoVeQmHhI1c29CMa9rdmO8BO54Pz3IqGRATLZOyg0KCdosBnuVIVSWFU0IDe4sLr
	 77VS783KOMmhMlm9TcFi3vZlCPpG/4dAtS9OouSFyEvIM/tzWtkK93Q2ntyrkfHaN6
	 sE86+fGD9i5Giqho5vcdehMbO6IvApq4rLBnFuP1vVcwDUy5ERrgIJ5fMVV4KlaGgJ
	 +ZPQJxV4sKbGvo+8dWp/Cq0r0oCSv10ojKYfnpzm3tgWR9TOGKS3vfuQ4w7nY5FY6z
	 ADCIIjDW4TxDDq/Uir4VmfBKehrNDmI18YA1P6acC53Ud73usXZn50pn0Gm2feLvqZ
	 EYrkrr8MMVUSg==
Date: Thu, 30 May 2024 17:10:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Michal Orzel <michal.orzel@amd.com>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 2/3] CI: Use a debug build of Xen for the Xilinx HW
 tests
In-Reply-To: <20240529141945.41669-3-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405301710380.2557291@ubuntu-linux-20-04-desktop>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com> <20240529141945.41669-3-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1953463651-1717114248=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1953463651-1717114248=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 29 May 2024, Andrew Cooper wrote:
> ... like the other hardware tests.  This gets more value out of the testing.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  automation/gitlab-ci/test.yaml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index efd3ad46f08e..e96ccdfad54c 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
>      - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
>    needs:
>      - *arm64-test-needs
> -    - alpine-3.18-gcc-arm64
> +    - alpine-3.18-gcc-debug-arm64
>  
>  xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
>    extends: .xilinx-arm64
> @@ -157,7 +157,7 @@ xilinx-smoke-dom0less-arm64-gcc-gem-passthrough:
>      - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh gem-passthrough 2>&1 | tee ${LOGFILE}
>    needs:
>      - *arm64-test-needs
> -    - alpine-3.18-gcc-arm64
> +    - alpine-3.18-gcc-debug-arm64
>  
>  adl-smoke-x86-64-gcc-debug:
>    extends: .adl-x86-64
> -- 
> 2.30.2
> 
--8323329-1953463651-1717114248=:2557291--


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:11:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:11:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733143.1139245 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprT-0004wO-Lw; Fri, 31 May 2024 00:11:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733143.1139245; Fri, 31 May 2024 00:11:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprT-0004wH-JK; Fri, 31 May 2024 00:11:11 +0000
Received: by outflank-mailman (input) for mailman id 733143;
 Fri, 31 May 2024 00:11:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tcd8=NC=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCprS-0004vF-Ro
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:11:10 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4824f835-1ee2-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 02:11:09 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-4210aa012e5so16181745e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 17:11:09 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212e4b1ba4sm927335e9.39.2024.05.30.17.11.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 17:11:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4824f835-1ee2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717114269; x=1717719069; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=i2pAKhjleKNGk/T56K69zZ4GNIwwq9AiC1D8esE9Tfw=;
        b=GJI1oYZa71LVB/x8GioSRHpngdxjqa2nm7ALgQ+mAqJIyZatlUJV/ySgpKuQ29BNpE
         dB0E7zALTqRKVLg+yKqkl3X8lsHlqwcAo5ySRhj1Xzdu5jq8PZD+guehwRIiQZt1gzAm
         AtA/0lTLqtK5E/KA1Ub518TqQPrSMzRYFEJsA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717114269; x=1717719069;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=i2pAKhjleKNGk/T56K69zZ4GNIwwq9AiC1D8esE9Tfw=;
        b=mC2CEiT48y8Xlhzd/fT2GCjgTLINRjRdP/i5gqsyB27FH4oyK3ppZ+fnwIJ5CiW9WW
         x9m8lQyDI3bcOkpLJHC3PXYKqRH49+TtXdYc9DrCyPc/aHiuYn2ErhBwIlFC+e3qLFtn
         XfwFAxcdOkjRfCGUzbLzQSdTfiTQbg2EBW4Wl5QaGg6COkve4x2YM9eInFJuSNhCz5g/
         hjW8lowHc6MQGgoHdqdCXM5Xc4UIYDgiO8UrTvtc9Oy6TCLjcfZw+Jt+p7g4zN71uPUH
         RAOl41wnnAgSIolTvXz6fOacB1sg526E6dIeD3aYlaH0XvfB60pUcg7zUviav0R4AcDN
         +INg==
X-Forwarded-Encrypted: i=1; AJvYcCXJuQoAIvNYANnR+xX6zXJ5VrRU3Liat357dF/GwyDMqcaza82dDwR7/4unQf19Pl0TY9vUJQ9EVK6E1tBsRtuw+RfmmLFHXl9so+RjD9k=
X-Gm-Message-State: AOJu0YyvBLVJF5NX5vUgEZBJy/CZYl3QTaoDYlwYy52eyBQ9cYHyki5y
	/51gDNuj/X35bCyT70tJKoeUgwpDcyuGnbw0g/Ffh2ZgaEgvYPdmE/29HpkurRM=
X-Google-Smtp-Source: AGHT+IGnKMLnG9o6dLTi5nBVaDAPo1tRuPcVXjAz9/xrvlHs9JX1MWr1k6FDf7weCaxTnhsqG5qMMA==
X-Received: by 2002:a05:600c:8a8:b0:418:df5a:3fba with SMTP id 5b1f17b1804b1-4212e09cca8mr2290385e9.32.1717114268900;
        Thu, 30 May 2024 17:11:08 -0700 (PDT)
Message-ID: <40f9e04b-1238-477b-81b4-7c880b363797@citrix.com>
Date: Fri, 31 May 2024 01:11:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <35cf9d52e538aab964a3ecc050260abb3f27c60d.1717008161.git.oleksii.kurochko@gmail.com>
 <0f12ce69-b139-4d97-8e36-23dd7c6bea41@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <0f12ce69-b139-4d97-8e36-23dd7c6bea41@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30/05/2024 5:44 pm, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>> To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h>
>> as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of
>> check_get_page_from_gfn() from xen/p2m-common.h.

Actually, this is an ARGO bug not a RISC-V bug.

It only builds in other architectures because of transitive dependencies
through asm/domain.h

I've fixed up argo.c to include the right header, which also prevents
(for now at least) adding to general tangle which is asm/domain.h

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:11:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:11:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733148.1139255 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprj-0005V2-Th; Fri, 31 May 2024 00:11:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733148.1139255; Fri, 31 May 2024 00:11:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCprj-0005Uv-QQ; Fri, 31 May 2024 00:11:27 +0000
Received: by outflank-mailman (input) for mailman id 733148;
 Fri, 31 May 2024 00:11:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCpri-0004vF-Vw
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:11:26 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4f8bfce7-1ee2-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 02:11:25 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 14C04CE1B96;
 Fri, 31 May 2024 00:11:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 749F4C2BBFC;
 Fri, 31 May 2024 00:11:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f8bfce7-1ee2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717114279;
	bh=o05O8CFGXuLnVFy1jNXuWtwoboOu9eskAY6ksrj3EbY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fiKGXBImtUdx2vB0apWlielJ6LmtnEnn5Tw6+nOxXxHPt7WZBGshqI4p+v+0jqmqV
	 OBofDjYoBVi00ukcq1XcRoCANM/QHtvvxYl7CU7HGKGZQ71971tlxXWLx24GWQ/yfs
	 /iPP6fthxzsRL2uxLJfj/R99ev7aSvJiXhq0HvMQVSL/yt1EoWduZsrh8NrzOIBwmI
	 XIVGcYMooGkP91oCqJ33+Lbk3ZqsfV3VsYI5keaOXAo1Bl9rnCFLTu0iT5IdcD0vmX
	 W+lB2pcQg39I/WHvxuLkO+96JM+/o6c8o8vixyQCy2L2Nkc57YlWrAYLN6ol8OnLWr
	 vDQWCiey4VPyg==
Date: Thu, 30 May 2024 17:11:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    Xen-devel <xen-devel@lists.xenproject.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 2/3] CI: Use a debug build of Xen for the Xilinx HW
 tests
In-Reply-To: <10676879-e0e4-4332-b8cb-732d56cda818@amd.com>
Message-ID: <alpine.DEB.2.22.394.2405301711050.2557291@ubuntu-linux-20-04-desktop>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com> <20240529141945.41669-3-andrew.cooper3@citrix.com> <10676879-e0e4-4332-b8cb-732d56cda818@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1169749750-1717114278=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1169749750-1717114278=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 29 May 2024, Michal Orzel wrote:
> Hi Andrew,
> 
> On 29/05/2024 16:19, Andrew Cooper wrote:
> > 
> > 
> > ... like the other hardware tests.  This gets more value out of the testing.
> > 
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > ---
> > CC: Roger Pau Monné <roger.pau@citrix.com>
> > CC: Stefano Stabellini <sstabellini@kernel.org>
> > CC: Michal Orzel <michal.orzel@amd.com>
> > CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> >  automation/gitlab-ci/test.yaml | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index efd3ad46f08e..e96ccdfad54c 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -149,7 +149,7 @@ xilinx-smoke-dom0less-arm64-gcc:
> >      - ./automation/scripts/xilinx-smoke-dom0less-arm64.sh 2>&1 | tee ${LOGFILE}
> >    needs:
> >      - *arm64-test-needs
> > -    - alpine-3.18-gcc-arm64
> > +    - alpine-3.18-gcc-debug-arm64
> This change should be reflected in the name of the test changed (here and below), so that it contains -debug suffix just like every other debug job.
> With that done:
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Ops, good catch! I missed it
--8323329-1169749750-1717114278=:2557291--


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:13:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:13:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733157.1139264 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCptH-0006H6-6d; Fri, 31 May 2024 00:13:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733157.1139264; Fri, 31 May 2024 00:13:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCptH-0006Gz-42; Fri, 31 May 2024 00:13:03 +0000
Received: by outflank-mailman (input) for mailman id 733157;
 Fri, 31 May 2024 00:13:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCptG-0006GX-7B
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:13:02 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8906db72-1ee2-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 02:13:00 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 97919CE1B92;
 Fri, 31 May 2024 00:12:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8D89DC2BBFC;
 Fri, 31 May 2024 00:12:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8906db72-1ee2-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717114376;
	bh=DX/ZJW+AbNYC5W7lCQnC5G1JyZNmuQtKiqpllAciZwc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ZSt25Whc2XL1HVbF7Add4f1cKME/xXDoK+Qth6Ufc8nOWjHrdj2DI05fm6GjoriKd
	 VQbYy2Awhqy1+a8gkySRWiKUcW4Lsg2gZk9M65AUJkjIWkZkQXB5/g5V5F4ttNhdVN
	 oc1ZD0dP+9IXHxowlQ8aTnQRVtDwC+LoBj2FEI8euWeFRbNIlAuyU+h3rXfmHKPaNg
	 G60Or1hBFyTzHpYQqUZB0VwCPOitvy/IOT3X9dDgxYfCFysCqznFH9kaKsHd/mBYyz
	 eD931aJN3hGO7T1OLOWOyFDa5CxsfjjOUQVBRk9wV1QL3ZQfCVf7uZ3GDxQ37fr8zx
	 sKvioYCVMqSRA==
Date: Thu, 30 May 2024 17:12:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Michal Orzel <michal.orzel@amd.com>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 3/3] CI: Improve serial handling in
 qemu-smoke-ppc64le.sh
In-Reply-To: <20240529141945.41669-4-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405301712470.2557291@ubuntu-linux-20-04-desktop>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com> <20240529141945.41669-4-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1971276845-1717114375=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1971276845-1717114375=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 29 May 2024, Andrew Cooper wrote:
> Have PPC put serial to stdout like all other tests, so it shows up in the main
> job log.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  automation/scripts/qemu-smoke-ppc64le.sh | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
> index 2adbdac87ef5..701d81107d18 100755
> --- a/automation/scripts/qemu-smoke-ppc64le.sh
> +++ b/automation/scripts/qemu-smoke-ppc64le.sh
> @@ -2,15 +2,15 @@
>  
>  set -ex
>  
> +serial_log="$(pwd)/smoke.serial"
> +
>  # machine type from first arg passed directly to qemu -M
>  machine=$1
>  
>  # Run the test
> -rm -f smoke.serial
> +rm -f ${serial_log}
>  set +e
>  
> -touch smoke.serial
> -
>  timeout -k 1 20 \
>  binaries/qemu-system-ppc64 \
>      -bios binaries/skiboot.lid \
> @@ -20,9 +20,10 @@ binaries/qemu-system-ppc64 \
>      -vga none \
>      -monitor none \
>      -nographic \
> -    -serial file:smoke.serial \
> -    -kernel binaries/xen
> +    -serial stdio \
> +    -kernel binaries/xen \
> +    |& tee ${serial_log} | sed 's/\r//'
>  
>  set -e
> -(grep -q "Hello, ppc64le!" smoke.serial) || exit 1
> +(grep -q "Hello, ppc64le!" ${serial_log}) || exit 1
>  exit 0
> -- 
> 2.30.2
> 
--8323329-1971276845-1717114375=:2557291--


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:43:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:43:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733170.1139282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqMb-0002tQ-DP; Fri, 31 May 2024 00:43:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733170.1139282; Fri, 31 May 2024 00:43:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqMb-0002tJ-Ay; Fri, 31 May 2024 00:43:21 +0000
Received: by outflank-mailman (input) for mailman id 733170;
 Fri, 31 May 2024 00:43:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCqMZ-0002tD-Fu
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:43:19 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c444246f-1ee6-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 02:43:16 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2D3D66290C;
 Fri, 31 May 2024 00:43:15 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E649BC2BBFC;
 Fri, 31 May 2024 00:43:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c444246f-1ee6-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717116194;
	bh=fYW4LEG0d2v6XHYpihKqCQJDdPWeRaWNXDK+gNPIzQw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=B1ELs8QvgPH11h4MJqJyoa1zyX2/uJftZpy3B9FYK+SKG87p+MM96TOjuJzBrI/O8
	 T5XSLh2gdmjqt19SW/HjuIG0kkhUDerCBNP4Wf1tPwi51pDDoMjyOD4aOBxkxdDBqE
	 yzHTvGFT9jfrKqbWGrzAuvI6qT7ZHkKz8/nO7G5iB9jy/WjT6Ao/mk5hQeSy6/O1gT
	 OxvKe07fZJAh6Taw/5z9Nuuq+MOiUh6qf8jjfWdq7ymB4wZtPy6RBmTiw0J8Q+QPEl
	 beXWZira/poxiWmx8aV4Rj8TRHwBikOU5G8L/taEk5jv/wIXleDrAzyml0QQ+62wDP
	 yWPlH09dlngAQ==
Date: Thu, 30 May 2024 17:43:12 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    Xen-devel <xen-devel@lists.xenproject.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for
 HW jobs
In-Reply-To: <ZlfHh_blx1i0eE7O@mail-itl>
Message-ID: <alpine.DEB.2.22.394.2405301715010.2557291@ubuntu-linux-20-04-desktop>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com> <20240529141945.41669-2-andrew.cooper3@citrix.com> <ZlfHh_blx1i0eE7O@mail-itl>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1773755421-1717115846=:2557291"
Content-ID: <alpine.DEB.2.22.394.2405301738011.2557291@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1773755421-1717115846=:2557291
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2405301738012.2557291@ubuntu-linux-20-04-desktop>

On Thu, 30 May 2024, Marek Marczykowski-Górecki wrote:
> On Wed, May 29, 2024 at 03:19:43PM +0100, Andrew Cooper wrote:
> > This restriction doesn't provide any security because anyone with suitable
> > permissions on the HW runners can bypass it with this local patch.
> > 
> > Requiring branches to be protected hampers usability of transient testing
> > branches (specifically, can't delete branches except via the Gitlab UI).
> >
> > Drop the requirement.
> > 
> > Fixes: 746774cd1786 ("automation: introduce a dom0less test run on Xilinx hardware")
> > Fixes: 0ab316e7e15f ("automation: add a smoke and suspend test on an Alder Lake system")
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Runners used to be set to run only on protected branches. I think it
> isn't the case anymore from what I see, but it needs checking (I don't
> see specific settings in all the projects). If it were still the case,
> removing variable check would result in jobs forever pending.

Andrew, thank you so much for pointing this out.

I think the idea was that we can specify the individual users with
access to protected branches. We cannot add restrictions for unprotected
branches. So if we set the gitlab runner to only run protected jobs,
then the $CI_COMMIT_REF_PROTECTED check makes sense. Not for security,
but to prevent the jobs from getting stuck waiting for a runner that
will never arrive.

However, like Marek said, now the gitlab runners don't have the
"Protected" check set, so it is all useless :-(

I would prefer to set "Protected" in the gitlab runners settings so that
it becomes easier to specify users that can and cannot trigger the jobs.

Then, we'll need the $CI_COMMIT_REF_PROTECTED check, not for security,
but to avoid pipelines getting stuck for unprotected branches.

It is really difficult to restrict users from triggering jobs in other
way because they are all automatically added to all subprojects.


Would you guys be OK if I set "Protected" in the Xilinx and Qubes gitlab
runners as soon as possible?
--8323329-1773755421-1717115846=:2557291--


From xen-devel-bounces@lists.xenproject.org Fri May 31 00:57:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 00:57:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733175.1139292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqaT-0004rk-ID; Fri, 31 May 2024 00:57:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733175.1139292; Fri, 31 May 2024 00:57:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqaT-0004rd-FO; Fri, 31 May 2024 00:57:41 +0000
Received: by outflank-mailman (input) for mailman id 733175;
 Fri, 31 May 2024 00:57:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCqaR-0004rX-T6
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 00:57:39 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c6326d5f-1ee8-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 02:57:38 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 47F4862A2B;
 Fri, 31 May 2024 00:57:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 16F58C2BBFC;
 Fri, 31 May 2024 00:57:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6326d5f-1ee8-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717117057;
	bh=/eEABicd2wC3kYUMST9f2jcve2jqMmLFTvCQaWYiGE0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pfK43IXskjZS7Z+xDEEqvqItg1tIc+dkpVENHxzVcCsXpXhE21lkZNJ1OaUCogzxB
	 Q7zgU0KlhB9UJNxTPH/5CtBomvgii//L1FAIL6CDhPSJNSzwrtSDD0+QPqE4VcxZLh
	 D8Hbo79PBE7eegi1OZVwzcSo/ygiQ4isdTwRi4pT8IIsy3DHJdMxFwaA7I8jL+DxAB
	 zGXHKN9N3naybwmdnpTW2Z2vnZuUhZHI9ltJZz2F6CFNGsafsKhR/0WiYbt6AUYsx9
	 RokDTrTiWYnLiQAAhUVBgjZFnKG2daZwB8SCHGn1lHNLaqayxFpA1Vnk5hDf8wD1EV
	 BunqxwUT3u6tw==
Date: Thu, 30 May 2024 17:57:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Jan Beulich <JBeulich@suse.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>, 
    Shawn Anastasio <sanastasio@raptorengineering.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Federico Serafini <federico.serafini@bugseng.com>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: Re: [PATCH v2 03/13] ARM/bitops: Change find_first_set_bit() to be
 a define
In-Reply-To: <20240524200338.1232391-4-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405301757220.2557291@ubuntu-linux-20-04-desktop>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com> <20240524200338.1232391-4-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 May 2024, Andrew Cooper wrote:
> This is in order to maintain bisectability through the subsequent changes, as
> the order of definitions is altered.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 01:03:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 01:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733180.1139303 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqgG-0005gD-5C; Fri, 31 May 2024 01:03:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733180.1139303; Fri, 31 May 2024 01:03:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqgG-0005g6-2P; Fri, 31 May 2024 01:03:40 +0000
Received: by outflank-mailman (input) for mailman id 733180;
 Fri, 31 May 2024 01:03:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCqgF-0005fz-44
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 01:03:39 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9b0786d0-1ee9-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 03:03:35 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 349F162A3F;
 Fri, 31 May 2024 01:03:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F24F5C2BBFC;
 Fri, 31 May 2024 01:03:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b0786d0-1ee9-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717117413;
	bh=aVd5LuN6M3XAeK8UOhR8JfGPaq/mAscPNc2cSB/MUdo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gOeiix9t5lSPUwppW8w5GC4F3X9sq9EF7R/SKLBfomCF8Gcuptl6ELMJ24cFle74u
	 wDIwQL//hmA5qfT8Rqv+/iknnjZBADB89DSSGWzgnP64lbo/tDLaqRQVln1HFnaatA
	 c4g0GUMyjfqgCxLOVQ63Giu1WMZIrilIDh8i5I6H1KwArWErR5uv0fFImhQ++1O1GT
	 gSQGnSbwhv7Re+lNUR4uXdIR/mzre27J1/Vr27bsRfSIsoU6g6adhJGV2RM9VtHYcD
	 MSj9qCPOIpb2+g9KEe1dzgC0nK24wY3G5OZ3LK5aPHOq2sa/ezfaf67W1heGce1yTt
	 vwz700xjVgyzw==
Date: Thu, 30 May 2024 18:03:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>, 
    Shawn Anastasio <sanastasio@raptorengineering.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Federico Serafini <federico.serafini@bugseng.com>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 05/13] xen/bitops: Implement generic_f?sl() in lib/
In-Reply-To: <7305a8b9-5f43-47e4-987b-c5ca76bcc312@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405301803050.2557291@ubuntu-linux-20-04-desktop>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com> <20240524200338.1232391-6-andrew.cooper3@citrix.com> <7305a8b9-5f43-47e4-987b-c5ca76bcc312@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 May 2024, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
> > generic_f?s() being static inline is the cause of lots of the complexity
> > between the common and arch-specific bitops.h
> > 
> > They appear to be static inline for constant-folding reasons (ARM uses them
> > for this), but there are better ways to achieve the same effect.
> > 
> > It is presumptuous that an unrolled binary search is the right algorithm to
> > use on all microarchitectures.  Indeed, it's not for the eventual users, but
> > that can be addressed at a later point.
> > 
> > It is also nonsense to implement the int form as the base primitive and
> > construct the long form from 2x int in 64-bit builds, when it's just one extra
> > step to operate at the native register width.
> > 
> > Therefore, implement generic_f?sl() in lib/.  They're not actually needed in
> > x86/ARM/PPC by the end of the cleanup (i.e. the functions will be dropped by
> > the linker), and they're only expected be needed by RISC-V on hardware which
> > lacks the Zbb extension.
> > 
> > Implement generic_fls() in terms of generic_flsl() for now, but this will be
> > cleaned up in due course.
> > 
> > Provide basic runtime testing using __constructor inside the lib/ file.  This
> > is important, as it means testing runs if and only if generic_f?sl() are used
> > elsewhere in Xen.
> > 
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> with a suggestion and a question.

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> > I suspect we want to swap CONFIG_DEBUG for CONFIG_BOOT_UNIT_TESTS in due
> > course.  These ought to be able to be used in a release build too.
> 
> +1

+1


From xen-devel-bounces@lists.xenproject.org Fri May 31 01:12:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 01:12:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733187.1139312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqoZ-0007lw-28; Fri, 31 May 2024 01:12:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733187.1139312; Fri, 31 May 2024 01:12:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqoY-0007lp-Vt; Fri, 31 May 2024 01:12:14 +0000
Received: by outflank-mailman (input) for mailman id 733187;
 Fri, 31 May 2024 01:12:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCqoY-0007lf-B0; Fri, 31 May 2024 01:12:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCqoY-000395-6d; Fri, 31 May 2024 01:12:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCqoX-0008CS-Tq; Fri, 31 May 2024 01:12:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCqoX-0003x1-TW; Fri, 31 May 2024 01:12:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TvH0Mnc970mV4Y3vmfb0SIIVpgYStQqHaE/IxfRnohw=; b=kX19XUnzH/k1hhDLIIZU7vnTn1
	8Z+6Ho2pMymSzAn5UjmlB/9/bXjhE7rc25/ISeNFJggXxLoeiBcxmbSejkTR3JRIHyxT/Vr+5CKiR
	ZptIwVvlYbw3D2tLDOid5/xiEKS1TuNkdcRy/fHItks8/bphwZLTG8Chr5aC3SY/3JAE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186203-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186203: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=a8dc6bf73f789f38f2930641b20dfd6d9e38f411
X-Osstest-Versions-That:
    ovmf=ced13b93afea87a8a1fe6ddbb67240a84cb2e3d3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 01:12:13 +0000

flight 186203 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186203/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 a8dc6bf73f789f38f2930641b20dfd6d9e38f411
baseline version:
 ovmf                 ced13b93afea87a8a1fe6ddbb67240a84cb2e3d3

Last test of basis   186201  2024-05-30 21:13:00 Z    0 days
Testing same since   186203  2024-05-30 23:12:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  dependabot[bot] <support@github.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   ced13b93af..a8dc6bf73f  a8dc6bf73f789f38f2930641b20dfd6d9e38f411 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 01:15:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 01:15:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733193.1139324 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqrC-0008Kd-HC; Fri, 31 May 2024 01:14:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733193.1139324; Fri, 31 May 2024 01:14:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqrC-0008KW-D0; Fri, 31 May 2024 01:14:58 +0000
Received: by outflank-mailman (input) for mailman id 733193;
 Fri, 31 May 2024 01:14:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCqrB-0008KP-Jc
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 01:14:57 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ff8389a-1eeb-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 03:14:55 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 9956F62A69;
 Fri, 31 May 2024 01:14:53 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FE78C32786;
 Fri, 31 May 2024 01:14:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ff8389a-1eeb-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717118093;
	bh=G9ICLlFDLIkdzXmV0iXMa+kIxih68zobAVPFmpeXTEo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bwh6WucFJvkGXMNapVCspS/zI2wGgAk20gOFb/+cJ6hxj1bupX8gXEwqtTw+Fjus1
	 InQD2SdcP04duwNNvVMAr0nDMJhcxzZ2J0DY896aUjdb0d++ZVJmkucaMsb8bl/tvq
	 Pa1XNTrNDlZKsW/laEK85aTO7LolkluzS9AS6KAb26IC2ZmfYiWl0GgxWza6B4Gd9i
	 vR5gwNINkAuy8LxICbZAzak1Iy6WaJRopqe3/uStfnzFbpAqHPdsle9pVsMUOPD6bq
	 JUN0n1BAE9qC1/jdWsUDYVn/Jgc6C0j+s261eXvaXpwtNNOrNjhxSihqCTIO7jqFnn
	 ispWlatpkbU6w==
Date: Thu, 30 May 2024 18:14:49 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: Xen-devel <xen-devel@lists.xenproject.org>, 
    Jan Beulich <JBeulich@suse.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>, 
    Shawn Anastasio <sanastasio@raptorengineering.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Federico Serafini <federico.serafini@bugseng.com>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>
Subject: Re: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
In-Reply-To: <20240524200338.1232391-7-andrew.cooper3@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2405301809170.2557291@ubuntu-linux-20-04-desktop>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com> <20240524200338.1232391-7-andrew.cooper3@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-254836557-1717118093=:2557291"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-254836557-1717118093=:2557291
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 May 2024, Andrew Cooper wrote:
> Perform constant-folding unconditionally, rather than having it implemented
> inconsistency between architectures.
> 
> Confirm the expected behaviour with compile time and boot time tests.
> 
> For non-constant inputs, use arch_ffs() if provided but fall back to
> generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin that
> works in all configurations.
> 
> For x86, rename ffs() to arch_ffs() and adjust the prototype.
> 
> For PPC, __builtin_ctz() is 1/3 of the size of size of the transform to
> generic_fls().  Drop the definition entirely.  ARM too benefits in the general
> case by using __builtin_ctz(), but less dramatically because it using
> optimised asm().
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

This patch made me realize that we should add __builtin_ctz,
__builtin_constant_p and always_inline to
docs/misra/C-language-toolchain.rst as they don't seem to be currently
documented and they are not part of the C standard

Patch welcome :-)

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> CC: Shawn Anastasio <sanastasio@raptorengineering.com>
> CC: consulting@bugseng.com <consulting@bugseng.com>
> CC: Simone Ballarin <simone.ballarin@bugseng.com>
> CC: Federico Serafini <federico.serafini@bugseng.com>
> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> v2:
>  * Fall back to generic, not builtin.
>  * Extend the testing with multi-bit values.
>  * Use always_inline for x86
>  * Defer x86 optimisation to a later change
> ---
>  xen/arch/arm/include/asm/bitops.h |  2 +-
>  xen/arch/ppc/include/asm/bitops.h |  2 +-
>  xen/arch/x86/include/asm/bitops.h |  3 ++-
>  xen/common/Makefile               |  1 +
>  xen/common/bitops.c               | 19 +++++++++++++++++++
>  xen/include/xen/bitops.h          | 17 +++++++++++++++++
>  6 files changed, 41 insertions(+), 3 deletions(-)
>  create mode 100644 xen/common/bitops.c
> 
> diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
> index ec1cf7b9b323..a88ec2612e16 100644
> --- a/xen/arch/arm/include/asm/bitops.h
> +++ b/xen/arch/arm/include/asm/bitops.h
> @@ -157,7 +157,7 @@ static inline int fls(unsigned int x)
>  }
>  
>  
> -#define ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
> +#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
>  #define ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
>  
>  /**
> diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
> index ab692d01717b..5c36a6cc0ce3 100644
> --- a/xen/arch/ppc/include/asm/bitops.h
> +++ b/xen/arch/ppc/include/asm/bitops.h
> @@ -173,7 +173,7 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)
>  
>  #define flsl(x) generic_flsl(x)
>  #define fls(x) generic_flsl(x)
> -#define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
> +#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
>  #define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
>  
>  /**
> diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
> index 5a71afbc89d5..122767fc0d10 100644
> --- a/xen/arch/x86/include/asm/bitops.h
> +++ b/xen/arch/x86/include/asm/bitops.h
> @@ -430,7 +430,7 @@ static inline int ffsl(unsigned long x)
>      return (int)r+1;
>  }
>  
> -static inline int ffs(unsigned int x)
> +static always_inline unsigned int arch_ffs(unsigned int x)
>  {
>      int r;
>  
> @@ -440,6 +440,7 @@ static inline int ffs(unsigned int x)
>            "1:" : "=r" (r) : "rm" (x));
>      return r + 1;
>  }
> +#define arch_ffs arch_ffs
>  
>  /**
>   * fls - find last bit set
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index d512cad5243f..21a4fb4c7166 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -1,5 +1,6 @@
>  obj-$(CONFIG_ARGO) += argo.o
>  obj-y += bitmap.o
> +obj-bin-$(CONFIG_DEBUG) += bitops.init.o
>  obj-$(CONFIG_GENERIC_BUG_FRAME) += bug.o
>  obj-$(CONFIG_HYPFS_CONFIG) += config_data.o
>  obj-$(CONFIG_CORE_PARKING) += core_parking.o
> diff --git a/xen/common/bitops.c b/xen/common/bitops.c
> new file mode 100644
> index 000000000000..8c161b8ea7fa
> --- /dev/null
> +++ b/xen/common/bitops.c
> @@ -0,0 +1,19 @@
> +#include <xen/bitops.h>
> +#include <xen/boot-check.h>
> +#include <xen/init.h>
> +
> +static void __init test_ffs(void)
> +{
> +    /* unsigned int ffs(unsigned int) */
> +    CHECK(ffs, 0, 0);
> +    CHECK(ffs, 1, 1);
> +    CHECK(ffs, 3, 1);
> +    CHECK(ffs, 7, 1);
> +    CHECK(ffs, 6, 2);
> +    CHECK(ffs, 0x80000000U, 32);
> +}
> +
> +static void __init __constructor test_bitops(void)
> +{
> +    test_ffs();
> +}
> diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
> index cd405df96180..f7e90a2893a5 100644
> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -31,6 +31,23 @@ unsigned int __pure generic_flsl(unsigned long x);
>  
>  #include <asm/bitops.h>
>  
> +/*
> + * Find First/Last Set bit (all forms).
> + *
> + * Bits are labelled from 1.  Returns 0 if given 0.
> + */
> +static always_inline __pure unsigned int ffs(unsigned int x)
> +{
> +    if ( __builtin_constant_p(x) )
> +        return __builtin_ffs(x);
> +
> +#ifdef arch_ffs
> +    return arch_ffs(x);
> +#else
> +    return generic_ffsl(x);
> +#endif
> +}
> +
>  /* --------------------- Please tidy below here --------------------- */
>  
>  #ifndef find_next_bit
> -- 
> 2.30.2
> 
--8323329-254836557-1717118093=:2557291--


From xen-devel-bounces@lists.xenproject.org Fri May 31 01:16:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 01:16:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733198.1139333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqsF-0000QD-Pu; Fri, 31 May 2024 01:16:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733198.1139333; Fri, 31 May 2024 01:16:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCqsF-0000Q6-MP; Fri, 31 May 2024 01:16:03 +0000
Received: by outflank-mailman (input) for mailman id 733198;
 Fri, 31 May 2024 01:16:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h8zp=NC=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1sCqsE-0000Pt-1e
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 01:16:02 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 573df8b7-1eeb-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 03:16:01 +0200 (CEST)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C8BE261628;
 Fri, 31 May 2024 01:15:59 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 71C59C2BBFC;
 Fri, 31 May 2024 01:15:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 573df8b7-1eeb-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1717118159;
	bh=0ciOzSyMXNlpzwdsYHjf2865CDzJmea3wO2lRnTSFYY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Dja9Cx0xNZm/b9dEn2JRfpB45TmKRkjKzR0lwMZ5RVYfaoLQa7ASqTMYM75V0orLz
	 oM/V+OR2Jb5gC1gLg/iYZBpqRdQ1sGgf8au+NiDpKASCyr58vv202CbmeWqShahNHi
	 +ybKtpo9K1Pk4Ka76pv9UGkJY6n6NoovubiRbn2PzQkiC0jYDPO2oMs45Wuxpaifm2
	 XdlrDksV8T6CtOmVSeD/V1CgGg83IWng0hY2NJNojzRs6U0Pc04quJxm0a1chckT6O
	 xGY7Q8UFj7ihC8RCZkanZ1wufBzHFNiOSjLczeMYVwEVXVCkgEvnhiluKwlM9Src4E
	 cS15RVWvdzq0A==
Date: Thu, 30 May 2024 18:15:55 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>, 
    Shawn Anastasio <sanastasio@raptorengineering.com>, 
    "consulting @ bugseng . com" <consulting@bugseng.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Federico Serafini <federico.serafini@bugseng.com>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 08/13] xen/bitops: Implement ffsl() in common logic
In-Reply-To: <db5f4449-e333-426f-bbe8-e5b77daba16d@suse.com>
Message-ID: <alpine.DEB.2.22.394.2405301815490.2557291@ubuntu-linux-20-04-desktop>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com> <20240524200338.1232391-9-andrew.cooper3@citrix.com> <db5f4449-e333-426f-bbe8-e5b77daba16d@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 May 2024, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
> > Just like ffs() in the previous changes.  Express the upper bound of the
> > testing in terms of BITS_PER_LONG as it varies between architectures.
> > 
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Fri May 31 04:10:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 04:10:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733224.1139343 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCtbD-0004x3-Lg; Fri, 31 May 2024 04:10:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733224.1139343; Fri, 31 May 2024 04:10:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCtbD-0004ww-Ho; Fri, 31 May 2024 04:10:39 +0000
Received: by outflank-mailman (input) for mailman id 733224;
 Fri, 31 May 2024 04:10:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtbC-0004wm-Ci; Fri, 31 May 2024 04:10:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtbC-0006Vu-9U; Fri, 31 May 2024 04:10:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtbB-0005cW-Ux; Fri, 31 May 2024 04:10:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtbB-0002pS-UK; Fri, 31 May 2024 04:10:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PvbGnXC7+2ebYeWHR8EmOL5NHxT+Hb1cJyXAqlR1uVQ=; b=1//p1yg5aKthFmDG8u/CcrLtsm
	vvdqIhBL7zUaOKuBtXJp7WILo3k2Fn4/mPhtc378p+HgvV+Qp4Yw9oa1mqWXQPJ1QcvkB9M//T3FZ
	uUxaGpfZ/imAC1VXR1qdYLBI4suzATsZerA7DIgLhy/n/5ZH5j09HeE1m38//recu/I4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186205-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186205: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=5f68a363d0d95bd0d383861ae21886d9824a8cd4
X-Osstest-Versions-That:
    ovmf=a8dc6bf73f789f38f2930641b20dfd6d9e38f411
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 04:10:37 +0000

flight 186205 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186205/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 5f68a363d0d95bd0d383861ae21886d9824a8cd4
baseline version:
 ovmf                 a8dc6bf73f789f38f2930641b20dfd6d9e38f411

Last test of basis   186203  2024-05-30 23:12:41 Z    0 days
Testing same since   186205  2024-05-31 02:11:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  dependabot[bot] <support@github.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   a8dc6bf73f..5f68a363d0  5f68a363d0d95bd0d383861ae21886d9824a8cd4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 04:19:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 04:19:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733231.1139353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCtjY-0005ao-Ei; Fri, 31 May 2024 04:19:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733231.1139353; Fri, 31 May 2024 04:19:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCtjY-0005ah-Br; Fri, 31 May 2024 04:19:16 +0000
Received: by outflank-mailman (input) for mailman id 733231;
 Fri, 31 May 2024 04:19:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtjX-0005aX-R4; Fri, 31 May 2024 04:19:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtjX-0006dH-Oq; Fri, 31 May 2024 04:19:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtjX-0005pM-DD; Fri, 31 May 2024 04:19:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCtjX-0005vW-CZ; Fri, 31 May 2024 04:19:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xHPLY0I6wN15/OEkTMh+nNVz65TZSEjWq6n5rjTPJ7o=; b=okFIJPCkMVANHHmVtCJgrc6DNN
	NHm1cY8+OlGEosPXV2bt2PM87KbLF/eYgZQ1yJW0ubvdSHWYPDdS1IiJRvbYQNey2CBOf18UxV87m
	dl9mNLSTZ0/C4R24qBgKA6P9aOtTnwI3Z8QsgKL1PEh0Hk/0xsOSh3WpG9T9L3sfBIro=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186204-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 186204: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1250c73c1ae2eec0308b4efe3e345127e9dbdb2b
X-Osstest-Versions-That:
    xen=9a905d7dc65883af082532b4dc91ce0131e54047
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 04:19:15 +0000

flight 186204 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186204/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1250c73c1ae2eec0308b4efe3e345127e9dbdb2b
baseline version:
 xen                  9a905d7dc65883af082532b4dc91ce0131e54047

Last test of basis   186197  2024-05-30 12:00:22 Z    0 days
Testing same since   186204  2024-05-31 01:02:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9a905d7dc6..1250c73c1a  1250c73c1ae2eec0308b4efe3e345127e9dbdb2b -> smoke


From xen-devel-bounces@lists.xenproject.org Fri May 31 05:21:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 05:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733245.1139362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCuhQ-0005au-3l; Fri, 31 May 2024 05:21:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733245.1139362; Fri, 31 May 2024 05:21:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCuhQ-0005an-0x; Fri, 31 May 2024 05:21:08 +0000
Received: by outflank-mailman (input) for mailman id 733245;
 Fri, 31 May 2024 05:21:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCuhO-0005aN-Gr; Fri, 31 May 2024 05:21:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCuhO-00085v-D8; Fri, 31 May 2024 05:21:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCuhO-0007Md-2P; Fri, 31 May 2024 05:21:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCuhO-0007p6-21; Fri, 31 May 2024 05:21:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rk//T4eaOXcu1zN+t54511Ug3ya8U9tg+SmQhLQthbg=; b=IXMtf693kq+BLJyGune8/zk2Kn
	o4G/dZlwUkB3DW7zY5YnqjEJsfBKqcLR8B9gY2qDo6tFNExLY9DsOQrma2XWu0hphbe+xs9dDftR2
	KXTEdVD4yvHOev6zqjIp9ws6sDarQrqtIEJgdxah60HCuw0UlgzFwVVe/QqY8X5a5Bug=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186200-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186200: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:guest-localmigrate/x10:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9a905d7dc65883af082532b4dc91ce0131e54047
X-Osstest-Versions-That:
    xen=12f9599a3b135afa270ccf0795f3e9665059955d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 05:21:06 +0000

flight 186200 xen-unstable real [real]
flight 186207 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/186200/
http://logs.test-lab.xenproject.org/osstest/logs/186207/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 19 guest-localmigrate/x10 fail pass in 186207-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186194
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186194
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186194
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186194
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186194
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186194
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9a905d7dc65883af082532b4dc91ce0131e54047
baseline version:
 xen                  12f9599a3b135afa270ccf0795f3e9665059955d

Last test of basis   186194  2024-05-30 08:11:21 Z    0 days
Testing same since   186200  2024-05-30 18:38:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   12f9599a3b..9a905d7dc6  9a905d7dc65883af082532b4dc91ce0131e54047 -> master


From xen-devel-bounces@lists.xenproject.org Fri May 31 05:48:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 05:48:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733262.1139388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCv84-0008SV-Jb; Fri, 31 May 2024 05:48:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733262.1139388; Fri, 31 May 2024 05:48:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCv84-0008SO-Gx; Fri, 31 May 2024 05:48:40 +0000
Received: by outflank-mailman (input) for mailman id 733262;
 Fri, 31 May 2024 05:48:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ULw3=NC=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1sCv83-0008SI-00
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 05:48:39 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6c22bc1e-1f11-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 07:48:38 +0200 (CEST)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 0721C68BEB; Fri, 31 May 2024 07:48:33 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c22bc1e-1f11-11ef-90a1-e314d9c70b13
Date: Fri, 31 May 2024 07:48:32 +0200
From: Christoph Hellwig <hch@lst.de>
To: John Garry <john.g.garry@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	Roger Pau Monn?? <roger.pau@citrix.com>,
	linux-um@lists.infradead.org, linux-block@vger.kernel.org,
	nbd@other.debian.org, ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Subject: Re: [PATCH 09/12] sd: convert to the atomic queue limits API
Message-ID: <20240531054832.GB17396@lst.de>
References: <20240529050507.1392041-1-hch@lst.de> <20240529050507.1392041-10-hch@lst.de> <1a1854bb-1f28-44d1-a4ac-30872bd6c3c8@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1a1854bb-1f28-44d1-a4ac-30872bd6c3c8@oracle.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Thu, May 30, 2024 at 10:16:33AM +0100, John Garry wrote:
>> -static void sd_config_write_same(struct scsi_disk *);
>> +static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
>> +		unsigned int mode);
>
> Are there any reasons why we keep forward declarations like this? AFAICS, 
> this sd_config_discard forward declaration could be removed.

Mostly to avoid churn.  This is a series that needs to feed into the
block tree, so I don't want major churn in sd.c.  Maybe after the dust
has settled it would be nice to bring sd.c into a natural order.

>> -	blk_queue_max_write_zeroes_sectors(q, sdkp->max_ws_blocks *
>> -					 (logical_block_size >> 9));
>> +	lim->max_write_zeroes_sectors =
>> +		sdkp->max_ws_blocks * (logical_block_size >> 9);
>
> Would it be ok to use SECTOR_SHIFT here? A similar change is made in 
> sd_config_discard(), above

Sure.

>> +		sd_config_discard(sdkp, lim, sd_discard_mode(sdkp));
>>   	}
>>      out:
>> @@ -3278,10 +3290,10 @@ static void sd_read_block_limits_ext(struct scsi_disk *sdkp)
>>   }
>>     /**
>
> below is not a kernel doc comment

And that is on the one hand intentional to avoid documenting all the
obvious paramters in a local function, but on the other hand requires
removing the double *. Fixed.

>>   @@ -3683,28 +3696,33 @@ static int sd_revalidate_disk(struct gendisk 
>> *disk)
>>   	q->limits.max_dev_sectors = logical_to_sectors(sdp, dev_max);
>
>
> is setting q->limits.max_dev_sectors directly proper?

No, and I've already fixed it in my local tree.



From xen-devel-bounces@lists.xenproject.org Fri May 31 05:55:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 05:55:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733269.1139398 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvEF-0001Ux-7L; Fri, 31 May 2024 05:55:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733269.1139398; Fri, 31 May 2024 05:55:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvEF-0001Uq-4b; Fri, 31 May 2024 05:55:03 +0000
Received: by outflank-mailman (input) for mailman id 733269;
 Fri, 31 May 2024 05:55:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ULw3=NC=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1sCvEE-0001Uk-DO
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 05:55:02 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 50f0194f-1f12-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 07:55:00 +0200 (CEST)
Received: by verein.lst.de (Postfix, from userid 2407)
 id D3CD568BFE; Fri, 31 May 2024 07:54:56 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50f0194f-1f12-11ef-b4bb-af5377834399
Date: Fri, 31 May 2024 07:54:56 +0200
From: Christoph Hellwig <hch@lst.de>
To: Ilya Dryomov <idryomov@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	Roger Pau Monn?? <roger.pau@citrix.com>,
	linux-um@lists.infradead.org, linux-block@vger.kernel.org,
	nbd@other.debian.org, ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for
 max_sectors
Message-ID: <20240531055456.GC17396@lst.de>
References: <20240529050507.1392041-1-hch@lst.de> <20240529050507.1392041-3-hch@lst.de> <CAOi1vP-F0FO4WTnrEt7FC-uu2C8NTbejvJQQGdZqT475c2G1jA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAOi1vP-F0FO4WTnrEt7FC-uu2C8NTbejvJQQGdZqT475c2G1jA@mail.gmail.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Thu, May 30, 2024 at 09:48:06PM +0200, Ilya Dryomov wrote:
> For rbd, this change effectively lowers max_sectors from 4M to 64K or
> less and that is definitely not desirable.  From previous interactions
> with users we want max_sectors to match max_hw_sectors -- this has come
> up a quite a few times over the years.  Some people just aren't aware
> of the soft cap and the fact that it's adjustable and get frustrated
> over the time poured into debugging their iostat numbers for workloads
> that can send object (set) size I/Os.
> 
> Looking at the git history, we lowered io_opt from objset_bytes to
> opts->alloc_size in commit [1], but I guess io_opt was lowered just
> along for the ride.  What that commit was concerned with is really
> discard_granularity and to a smaller extent io_min.
> 
> How much difference does io_opt make in the real world?  If what rbd
> does stands in the way of a tree-wide cleanup, I would much rather bump
> io_opt back to objset_bytes (i.e. what max_user_sectors is currently
> set to).

The only existing in-kernel usage is to set the readahead size.
Based on your comments I seems like we should revert io_opt to
objset to ->alloc_size in a prep patch?



From xen-devel-bounces@lists.xenproject.org Fri May 31 06:18:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:18:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733274.1139409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvaj-0004Ax-18; Fri, 31 May 2024 06:18:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733274.1139409; Fri, 31 May 2024 06:18:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvai-0004Aq-U3; Fri, 31 May 2024 06:18:16 +0000
Received: by outflank-mailman (input) for mailman id 733274;
 Fri, 31 May 2024 06:18:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCvah-0004Ak-A5
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:18:15 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8e956857-1f15-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 08:18:12 +0200 (CEST)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a673a60f544so83860766b.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 23:18:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67e6f038b5sm51499566b.34.2024.05.30.23.18.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 23:18:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e956857-1f15-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717136291; x=1717741091; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=nn/e8lquZXF1nnUIFfv4gc/2j2gaSh7SKplJqVn88Wo=;
        b=NaRGy3lc7J5qFSwSlmNqgkTAc1EwHNife8FVAEeK7FAG2NHhhTUnJtwhQjx14vFOah
         zAPCe9MFvmBRE9kUlDPsTah4pZopYWDNNNOFuDDbMWGPaOZA0STwKgvV/UCocf5M7SV1
         aFmQa7ioOdLAdFQRAvNfhlCQ8i5Zcr053dKQDZt4vYMAFX194BCPJXe7SqnDobqIJscT
         P42gqzcSeq7Zf+D0hJczxFYrHCmRaRwAcifjGujQP63ZcSS0E4NhVJ7gdzM7/5iTFVg7
         1+KjkJNou2lbO5GrbPQcB0zqdfAPD3Ye6a5q3vo8YcvgBxYHT5BPDSSDAKambHuZmNrx
         LYLA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717136291; x=1717741091;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nn/e8lquZXF1nnUIFfv4gc/2j2gaSh7SKplJqVn88Wo=;
        b=D7L8+OC/s4Uao4tE/amkPpRQt+BROrwAETP5vVC9Z3GHVCtaJ9b3GFn7vO6Xlkb54Y
         KhYZmdZhqzUEClYMCgmlTRnVjuKOFduWX/DuB3rPtlemOYBQkwDB2/mC51wQPBeVw1sa
         wxrxHEn/Th8va+FgrEr1cOUprQIW0lwOvFnOWBkMrKQLJxfYXFz0QaTUfUtghf64Tl2o
         MOipz1qtPd9kWkOdibXL7/393XWxAtUl4y/ZYIDMHSYuWgpEbFmSuCIeqCPsYFsRsxsx
         jZjtV0Rt6Gu1dcckBCY3hVeifoPGSqsbCTdCm1L1qSBM2M1LrURJuIGhb0ckTJDpqZ29
         NtOg==
X-Forwarded-Encrypted: i=1; AJvYcCVpR2OCp38DErTG1Q3OfZ2hTkfJhj/5hm7gVEpXjH80ClNoyp3dmIoOsw+Kj3auMMhkX8KUhplUjEX8F32rwpGQd14bE4ln9ZigDDVgHdU=
X-Gm-Message-State: AOJu0YzDaKoiMIiYWdbnHDH4svuPPXWC8OBlkbfEHFEQbbcJ7mUDWpso
	NgEY7HjYJqW2vopS8ebDfqyf979OCSSmH4C9HQZkthrYVs7P6aliSJEo395uXQ==
X-Google-Smtp-Source: AGHT+IGAKp6X3H2N43hNuNomACQu9vmApamfOd034G7EHuW0NbRvSs+WYYUBbKgyQ8Bj7osI+sC15g==
X-Received: by 2002:a17:906:fd55:b0:a68:5f08:2e2a with SMTP id a640c23a62f3a-a685f082eefmr27374466b.28.1717136291390;
        Thu, 30 May 2024 23:18:11 -0700 (PDT)
Message-ID: <79a2d936-62f1-4749-9e75-0be019cd3d99@suse.com>
Date: Fri, 31 May 2024 08:18:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions
 for RISC-V64
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <c6ff49af9a107965f8121862e6b32c24548956e6.1717008161.git.oleksii.kurochko@gmail.com>
 <d4e5b4c8-d494-440b-8970-488b49bee12e@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d4e5b4c8-d494-440b-8970-488b49bee12e@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.05.2024 21:52, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>> diff --git a/README b/README
>> index c8a108449e..30da5ff9c0 100644
>> --- a/README
>> +++ b/README
>> @@ -48,6 +48,10 @@ provided by your OS distributor:
>>        - For ARM 64-bit:
>>          - GCC 5.1 or later
>>          - GNU Binutils 2.24 or later
>> +      - For RISC-V 64-bit:
>> +        - GCC 12.2 or later
>> +        - GNU Binutils 2.39 or later
> 
> I would like to petition for GCC 10 and Binutils 2.35.
> 
> These are the versions provided as cross-compilers by Debian, and
> therefore are the versions I would prefer to do smoke testing with.

See why I asked to amend the specified versions by a softening sentence that
you (only now) said you dislike? The "this is what we use in CI" makes it a
very random choice, entirely unrelated to the compiler's abilities.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 06:27:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:27:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733280.1139418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvj7-0005kY-PZ; Fri, 31 May 2024 06:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733280.1139418; Fri, 31 May 2024 06:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvj7-0005kR-Ms; Fri, 31 May 2024 06:26:57 +0000
Received: by outflank-mailman (input) for mailman id 733280;
 Fri, 31 May 2024 06:26:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCvj5-0005kL-UW
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:26:55 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5aa587e-1f16-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 08:26:53 +0200 (CEST)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a673a60f544so84480066b.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 23:26:53 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67eb3444bfsm51371266b.201.2024.05.30.23.26.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 23:26:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5aa587e-1f16-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717136813; x=1717741613; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=/BnfkwZulJdfDdkH/BzNnva6jmSvd1VIYFrx5/7OMk4=;
        b=cxEUTE9Wegl1ytxZbCGRclNSy5dVzYFxkuMo1O9vIsFcryfhqb4o4UxZR3xp4Gs3ZR
         b0rdPJm83fHmTbrtTSCHBanYLGd9M2N0Wq3fGhhd/+jRgNdfHnKgaY3dA7TfbHdCWYok
         wHyz2Mc4oTcxOsDasiuL5PZqzGpK3UlOJuM8Oo1j+3hkurrZtyvcL5NhIFN0lyDg0lZu
         IrxSYcOytxs2TeI3rMXGbkwMLDSPh2EktwVbq9hn6/tBKJhuF1uhu5cHvaZxf5GV1dk3
         CYA/0IS7K7WF2vrQNrSWNavAKX0BAW7IfBrKc3fm+zeq5DfEieHqOIw78QU6BbTvDKe5
         tHsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717136813; x=1717741613;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/BnfkwZulJdfDdkH/BzNnva6jmSvd1VIYFrx5/7OMk4=;
        b=tV3CQqHTXjXxKM9ujhXtg/my1KvK6+/upq7Q5PYbulsgfaRo8fWq0+w82efTdcXNVd
         I2nS/N2zJjJeXqDSqCC+W3QZNJZ2tD+Xu31IyLVMWa52Y+Rg3YBIrCIUCSIRdI8vFYs/
         ZZZvjW69ydF/2crIun9Zk+tuQp1CAoeo4Qpj94yDyZkvl/3a3jsSsLj8IxRC98nRrqpJ
         b5x7AVsuv9lta1GqeflC+fuDkr+Dk9iEcqLIJW8Sk9d4IKPswHS0mjnz3gGyms++9WUK
         /WXfX+QiBkXt7MGYnbss4SOHjm/gQ0dZ4VBgDhOjtPjcGQs/u3BX0/lkREtzhctKECrg
         Di7g==
X-Forwarded-Encrypted: i=1; AJvYcCV+7NsPPb17pR986JGUCwzUrBohggFqgMMyXxnkruK3VsVoB2Rr4hvl/cudLu1E0ssPfr6bVWXf/XjlT+4WgkhUlzHBiv14zB1qsy5NzhE=
X-Gm-Message-State: AOJu0Yz0RjsRz8w6VVwGFO5ZpAUnoUrL+hx6kTzjWNLxsysnDOv8as+X
	/+ldR7Go8MOoWe5syRhHAnu5xRCVITC+u8jg1bhUXwgPV53YYQt2oVLdyF7JcA==
X-Google-Smtp-Source: AGHT+IGk6XJ2jrMyGYDevOOAdcERz8ZVEpgahSbolsE3uYVFhVD+bapaGGYNrlQvhmT9jTAx5Bwbow==
X-Received: by 2002:a17:906:43c7:b0:a59:bacc:b082 with SMTP id a640c23a62f3a-a6821d646a5mr52337966b.60.1717136813296;
        Thu, 30 May 2024 23:26:53 -0700 (PDT)
Message-ID: <b24681c6-fbd3-4b01-beb4-560eabf3ad49@suse.com>
Date: Fri, 31 May 2024 08:26:51 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build
 full Xen
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1717008161.git.oleksii.kurochko@gmail.com>
 <d00b86f41ef2c7d928a28dadd8c34fb845f23d0a.1717008161.git.oleksii.kurochko@gmail.com>
 <70128dba-498f-4d85-8507-bb1621182754@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <70128dba-498f-4d85-8507-bb1621182754@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 30.05.2024 19:23, Andrew Cooper wrote:
> On 29/05/2024 8:55 pm, Oleksii Kurochko wrote:
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> This patch looks like it can go in independently?  Or does it depend on
> having bitops.h working in practice?

The latter, iirc. In fact I had already tried at least twice to pull this ahead.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 06:43:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:43:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733286.1139428 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvyZ-00007P-0k; Fri, 31 May 2024 06:42:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733286.1139428; Fri, 31 May 2024 06:42:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCvyY-00007I-U3; Fri, 31 May 2024 06:42:54 +0000
Received: by outflank-mailman (input) for mailman id 733286;
 Fri, 31 May 2024 06:42:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCvyX-00007C-U7
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:42:53 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 01388ec3-1f19-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 08:42:52 +0200 (CEST)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-57a30dbdb7fso678039a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 23:42:52 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67e73fab00sm53348066b.67.2024.05.30.23.42.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 May 2024 23:42:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01388ec3-1f19-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717137772; x=1717742572; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=FxhN/r2VrG94jWLE0WWGU0xxbhVXur51NZsN++2qHdw=;
        b=RK4HEm9oXqh6oXfeRAQbpDIIO95i97AYWJA/TobLGBj6hiO8T4LcYxeIGoM5Zs/ME6
         9sz/hqMX0/AN57a8OU7XZW6IqrQmlUqDfZZdBONsEhjW8JTK+N4uwOf877s03UmXSMDj
         Rr8Gr2lr0SVRR0LAfpJX90ZYG6+RNzJoezY5VdDn840wWmGI/YJ5LmEnkvgG3AIVYqOP
         7ZsoIcp9BuxA8a+rG1X/43702X9dxId8lkgjLCcsfr+LVkWZaApY79F0MwQnWEI7OXZn
         h3quQLPJdZXghvssGG5tOI8vSyNzH8LIDOoS11EGl3a3BOURfLGC5dXRMZCXt2Fjoo0x
         IGEg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717137772; x=1717742572;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=FxhN/r2VrG94jWLE0WWGU0xxbhVXur51NZsN++2qHdw=;
        b=wuTWyR4seZBT/35+Tg9r39EePiR77X4cCF54joNXL8JyzQTH2yZtrDvYy4qTMhwC/5
         34xkzM3PevIlDbXiE8S3s3p/KS/CqYyARFPif1nOfIn7vIKCsyMlTIbxWD78L/ir4DAw
         oAWEqo5Ap0ApRYQT+0b4NiePbPQs1JceNnVwSoVCVzbsJZxd8u/sV/oWUBIhlizKKR57
         gHjDSHzOoG1aMr5Ju3ry5uuDeM7PsMgQpoA7PtkQnP68oEmY2gzSaIbqPlYOToz41XXp
         8sWQqgBKdACMm0lT044dgb60cpGkUJmy1u2Fi3Yqq1DIrHnYBsrY3pytH5T5M/axjRiZ
         8DYw==
X-Forwarded-Encrypted: i=1; AJvYcCWA64+b7HZLNRb5OUzfTE0gUUJXGqLlcYw0xWn/+gDrgqWEtXfBtT90A6Ib1qFyxcdDaQktpgaR/QxDJExxtFC6AWICWvSyYfMjh7/NPos=
X-Gm-Message-State: AOJu0YxI+xxZU9rsOZsC4X5IqfTET+HVMOykpSFQK213dsF5UFsTWKWv
	saiCL1Mif6P8tC7/oys0zLgqM5ZfHt8dd2LlXJp33wZBdtwhx3KKtmLJJaavLA==
X-Google-Smtp-Source: AGHT+IEjokOS0mtq2igit/GyX2pqTJ7AdmL/cF7zNw3hk3ZEkK35GX0M9dzMb8LkDUVqC5MRHNXaYw==
X-Received: by 2002:a17:906:1718:b0:a5a:f16:32b1 with SMTP id a640c23a62f3a-a68205179d0mr58711966b.31.1717137772230;
        Thu, 30 May 2024 23:42:52 -0700 (PDT)
Message-ID: <e5741eb1-1708-463f-8b3a-2d34d6fd4fbc@suse.com>
Date: Fri, 31 May 2024 08:42:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] arch/irq: Make irq_ack_none() mandatory
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
 <20240530184027.44609-2-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240530184027.44609-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.05.2024 20:40, Andrew Cooper wrote:
> Any non-stub implementation of these is going to have to do something here.

For whatever definition of "something", seeing ...

> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,12 +31,12 @@ struct irq_guest
>      unsigned int virq;
>  };
>  
> -static void ack_none(struct irq_desc *irq)
> +void irq_ack_none(struct irq_desc *irq)
>  {
>      printk("unexpected IRQ trap at irq %02x\n", irq->irq);
>  }

... this, which - perhaps apart from the word "trap" - is entirely Arm-
independent, and could hence quite well live in a common code fallback
implementation. Nevertheless with patch 2 clearly being an improvement,
both patches:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 06:48:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:48:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733290.1139440 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCw3w-0000jC-LX; Fri, 31 May 2024 06:48:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733290.1139440; Fri, 31 May 2024 06:48:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCw3w-0000j5-GN; Fri, 31 May 2024 06:48:28 +0000
Received: by outflank-mailman (input) for mailman id 733290;
 Fri, 31 May 2024 06:48:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kOju=NC=gmail.com=idryomov@srs-se1.protection.inumbo.net>)
 id 1sCw3v-0000iz-2m
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:48:27 +0000
Received: from mail-oo1-xc2e.google.com (mail-oo1-xc2e.google.com
 [2607:f8b0:4864:20::c2e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c7512c0a-1f19-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 08:48:26 +0200 (CEST)
Received: by mail-oo1-xc2e.google.com with SMTP id
 006d021491bc7-5b970e90ab8so1030275eaf.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 May 2024 23:48:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7512c0a-1f19-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717138104; x=1717742904; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UHfXsuY+pdYsu6JkPOmIMiIuAZ41w4yugEvYSYmSsZg=;
        b=gH2lbifJgnQb7tvPGqJY2YSPGDFzKlIsDe78WWauG21PwJ/YqOG2eFrGjXkcRA9zHw
         Yau3I3yH4NzY97Rck7ojb9nSe3zEbfwC54GtSkUIbYSZSnDJydqkf3mwzy8UlFmcjePC
         ffRqMnnfL6NbyudJ4bzyXKJF17Qkg8gw3WXHmfLliOCDchWmBwDg5g74BjLObNJj5HdE
         zF1tpsw+qCjmbpb2UumpaEYypTLIuzgKeA7Wv0yeKd8nmh2tcNFLWQ37NtbzfwfiSfnl
         up31vYtJs8H+UMzbOGXlLkaYaJ7nPL2YOhg8ufqE2ZqRZsLgyejFlwFX66LQiRtd3SZ+
         /oeg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717138104; x=1717742904;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UHfXsuY+pdYsu6JkPOmIMiIuAZ41w4yugEvYSYmSsZg=;
        b=gDZoL9M1jMZaV7P0NF9V6DKpqhlmim4jPI2+vxY1SS5onprnri4YyX8xWjgq2aQYWI
         YF2BgCh9ph+3CynHhAt733egEyT9kFjPq0D4tlxY6gqYZwF8dlN5Usvjnurl8AZMqDkH
         1pMIPYEGsX8ock7Pzh/J/Jn+MVgAgq83Lk6FgmYOuTC8FksrRMU7CeJxsRGh5K8By4Hx
         pQ5dZZZgLLO80SScBU4rTB9GvEH+Uf3kAWoiIcHfRjfPXlG4I7SFRWEVU+GJq8P2h3EC
         P2P9vdrVGYAk0u/dzIR+dPT7MJ2+Tfj4lIqxzMP8fiCt14l0Y+j+jbsVFPQFvYixNdjW
         g3BA==
X-Forwarded-Encrypted: i=1; AJvYcCW8MBvUVyighyvONgiquhlsCCEwqixfC7NlMlYc1r7JjvL0xrNw0rrCZQM4VAyvRlrsUJ2QnhD1hjeYSgqU4l+AULr8kO0bjgwKaWAe1gE=
X-Gm-Message-State: AOJu0YwuK/qUBBFw/04KXgfxpO4q/yn6AMH0shy+vIv7GuDzI/3irZPY
	en/vGGVpid1nGqtSGrgPtUMVaEXSaMXkM/dwniRHNOd97/+UHAeez4QH6uJLKUFljG2wv2NTTk8
	B49y57UoxuAB/R5WsHKf+zGJe2SU=
X-Google-Smtp-Source: AGHT+IEkgVU5yVpc+xeOYwUOJd29bp6vThPd4yP1XUDfgboWzFw/FVhH/FxeC96crk9g7iXBffgH6P0YPBoJ35FyjnE=
X-Received: by 2002:a05:6820:151a:b0:5b9:70f8:4b82 with SMTP id
 006d021491bc7-5ba05e1f5a1mr1238669eaf.9.1717138104409; Thu, 30 May 2024
 23:48:24 -0700 (PDT)
MIME-Version: 1.0
References: <20240529050507.1392041-1-hch@lst.de> <20240529050507.1392041-3-hch@lst.de>
 <CAOi1vP-F0FO4WTnrEt7FC-uu2C8NTbejvJQQGdZqT475c2G1jA@mail.gmail.com> <20240531055456.GC17396@lst.de>
In-Reply-To: <20240531055456.GC17396@lst.de>
From: Ilya Dryomov <idryomov@gmail.com>
Date: Fri, 31 May 2024 08:48:12 +0200
Message-ID: <CAOi1vP-VXeOH-kShRKv-b=id1zN9tLiqOo8EKpOWoJuQp_Pm1g@mail.gmail.com>
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for max_sectors
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, "Martin K. Petersen" <martin.petersen@oracle.com>, 
	Richard Weinberger <richard@nod.at>, Anton Ivanov <anton.ivanov@cambridgegreys.com>, 
	Johannes Berg <johannes@sipsolutions.net>, Josef Bacik <josef@toxicpanda.com>, 
	Dongsheng Yang <dongsheng.yang@easystack.cn>, "Roger Pau Monn??" <roger.pau@citrix.com>, 
	linux-um@lists.infradead.org, linux-block@vger.kernel.org, 
	nbd@other.debian.org, ceph-devel@vger.kernel.org, 
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 31, 2024 at 7:54=E2=80=AFAM Christoph Hellwig <hch@lst.de> wrot=
e:
>
> On Thu, May 30, 2024 at 09:48:06PM +0200, Ilya Dryomov wrote:
> > For rbd, this change effectively lowers max_sectors from 4M to 64K or
> > less and that is definitely not desirable.  From previous interactions
> > with users we want max_sectors to match max_hw_sectors -- this has come
> > up a quite a few times over the years.  Some people just aren't aware
> > of the soft cap and the fact that it's adjustable and get frustrated
> > over the time poured into debugging their iostat numbers for workloads
> > that can send object (set) size I/Os.
> >
> > Looking at the git history, we lowered io_opt from objset_bytes to
> > opts->alloc_size in commit [1], but I guess io_opt was lowered just
> > along for the ride.  What that commit was concerned with is really
> > discard_granularity and to a smaller extent io_min.
> >
> > How much difference does io_opt make in the real world?  If what rbd
> > does stands in the way of a tree-wide cleanup, I would much rather bump
> > io_opt back to objset_bytes (i.e. what max_user_sectors is currently
> > set to).
>
> The only existing in-kernel usage is to set the readahead size.
> Based on your comments I seems like we should revert io_opt to
> objset to ->alloc_size in a prep patch?

We should revert io_opt from opts->alloc_size to objset_bytes (I think
it's what you meant to say but typoed).

How do you want to handle it?  I can put together a patch, send it to
ceph-devel and it will be picked by linux-next sometime next week.  Then
this patch would grow a contextual conflict and the description would
need to be updated to not mention a behavior change for rbd anymore.

Thanks,

                Ilya


From xen-devel-bounces@lists.xenproject.org Fri May 31 06:56:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:56:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733298.1139449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwBM-0002Yg-Dt; Fri, 31 May 2024 06:56:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733298.1139449; Fri, 31 May 2024 06:56:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwBM-0002YX-BO; Fri, 31 May 2024 06:56:08 +0000
Received: by outflank-mailman (input) for mailman id 733298;
 Fri, 31 May 2024 06:56:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ULw3=NC=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1sCwBK-0002YA-PP
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:56:06 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8bbd908-1f1a-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 08:56:03 +0200 (CEST)
Received: by verein.lst.de (Postfix, from userid 2407)
 id CC83168BEB; Fri, 31 May 2024 08:56:00 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8bbd908-1f1a-11ef-90a1-e314d9c70b13
Date: Fri, 31 May 2024 08:56:00 +0200
From: Christoph Hellwig <hch@lst.de>
To: Ilya Dryomov <idryomov@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	Roger Pau Monn?? <roger.pau@citrix.com>,
	linux-um@lists.infradead.org, linux-block@vger.kernel.org,
	nbd@other.debian.org, ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Subject: Re: [PATCH 02/12] block: take io_opt and io_min into account for
 max_sectors
Message-ID: <20240531065600.GA18681@lst.de>
References: <20240529050507.1392041-1-hch@lst.de> <20240529050507.1392041-3-hch@lst.de> <CAOi1vP-F0FO4WTnrEt7FC-uu2C8NTbejvJQQGdZqT475c2G1jA@mail.gmail.com> <20240531055456.GC17396@lst.de> <CAOi1vP-VXeOH-kShRKv-b=id1zN9tLiqOo8EKpOWoJuQp_Pm1g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAOi1vP-VXeOH-kShRKv-b=id1zN9tLiqOo8EKpOWoJuQp_Pm1g@mail.gmail.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Fri, May 31, 2024 at 08:48:12AM +0200, Ilya Dryomov wrote:
> We should revert io_opt from opts->alloc_size to objset_bytes (I think
> it's what you meant to say but typoed).

Yes, sorry.

> How do you want to handle it?  I can put together a patch, send it to
> ceph-devel and it will be picked by linux-next sometime next week.  Then
> this patch would grow a contextual conflict and the description would
> need to be updated to not mention a behavior change for rbd anymore.

If that works for you I'd much prefer to include it with this series.
I'd be happy to write it myself.



From xen-devel-bounces@lists.xenproject.org Fri May 31 06:56:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 06:56:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733301.1139459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwBy-00032v-Lo; Fri, 31 May 2024 06:56:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733301.1139459; Fri, 31 May 2024 06:56:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwBy-00032o-JB; Fri, 31 May 2024 06:56:46 +0000
Received: by outflank-mailman (input) for mailman id 733301;
 Fri, 31 May 2024 06:56:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSYE=NC=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1sCwBx-0002YA-2o
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 06:56:45 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f08d1780-1f1a-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 08:56:44 +0200 (CEST)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id E62F64EE0743;
 Fri, 31 May 2024 08:56:42 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f08d1780-1f1a-11ef-90a1-e314d9c70b13
MIME-Version: 1.0
Date: Fri, 31 May 2024 08:56:42 +0200
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
 <xen-devel@lists.xenproject.org>, Jan Beulich <JBeulich@suse.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>, "consulting @ bugseng . com"
 <consulting@bugseng.com>, Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>
Subject: Re: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
In-Reply-To: <alpine.DEB.2.22.394.2405301809170.2557291@ubuntu-linux-20-04-desktop>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-7-andrew.cooper3@citrix.com>
 <alpine.DEB.2.22.394.2405301809170.2557291@ubuntu-linux-20-04-desktop>
Message-ID: <7b974b36b89c216379b86170af9de451@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-05-31 03:14, Stefano Stabellini wrote:
> On Fri, 24 May 2024, Andrew Cooper wrote:
>> Perform constant-folding unconditionally, rather than having it 
>> implemented
>> inconsistency between architectures.
>> 
>> Confirm the expected behaviour with compile time and boot time tests.
>> 
>> For non-constant inputs, use arch_ffs() if provided but fall back to
>> generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin 
>> that
>> works in all configurations.
>> 
>> For x86, rename ffs() to arch_ffs() and adjust the prototype.
>> 
>> For PPC, __builtin_ctz() is 1/3 of the size of size of the transform 
>> to
>> generic_fls().  Drop the definition entirely.  ARM too benefits in the 
>> general
>> case by using __builtin_ctz(), but less dramatically because it using
>> optimised asm().
>> 
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> This patch made me realize that we should add __builtin_ctz,
> __builtin_constant_p and always_inline to
> docs/misra/C-language-toolchain.rst as they don't seem to be currently
> documented and they are not part of the C standard
> 
> Patch welcome :-)
> 

I can send a patch for the builtins. I think that for attributes it was 
decided to document the use of the __attribute__ token, rather than 
listing all the attributes used by Xen

> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> 
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Wei Liu <wl@xen.org>
>> CC: Stefano Stabellini <sstabellini@kernel.org>
>> CC: Julien Grall <julien@xen.org>
>> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
>> CC: Bertrand Marquis <bertrand.marquis@arm.com>
>> CC: Michal Orzel <michal.orzel@amd.com>
>> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> CC: Shawn Anastasio <sanastasio@raptorengineering.com>
>> CC: consulting@bugseng.com <consulting@bugseng.com>
>> CC: Simone Ballarin <simone.ballarin@bugseng.com>
>> CC: Federico Serafini <federico.serafini@bugseng.com>
>> CC: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> 
>> v2:
>>  * Fall back to generic, not builtin.
>>  * Extend the testing with multi-bit values.
>>  * Use always_inline for x86
>>  * Defer x86 optimisation to a later change
>> ---
>>  xen/arch/arm/include/asm/bitops.h |  2 +-
>>  xen/arch/ppc/include/asm/bitops.h |  2 +-
>>  xen/arch/x86/include/asm/bitops.h |  3 ++-
>>  xen/common/Makefile               |  1 +
>>  xen/common/bitops.c               | 19 +++++++++++++++++++
>>  xen/include/xen/bitops.h          | 17 +++++++++++++++++
>>  6 files changed, 41 insertions(+), 3 deletions(-)
>>  create mode 100644 xen/common/bitops.c
>> 
>> diff --git a/xen/arch/arm/include/asm/bitops.h 
>> b/xen/arch/arm/include/asm/bitops.h
>> index ec1cf7b9b323..a88ec2612e16 100644
>> --- a/xen/arch/arm/include/asm/bitops.h
>> +++ b/xen/arch/arm/include/asm/bitops.h
>> @@ -157,7 +157,7 @@ static inline int fls(unsigned int x)
>>  }
>> 
>> 
>> -#define ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
>> +#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
>>  #define ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); 
>> })
>> 
>>  /**
>> diff --git a/xen/arch/ppc/include/asm/bitops.h 
>> b/xen/arch/ppc/include/asm/bitops.h
>> index ab692d01717b..5c36a6cc0ce3 100644
>> --- a/xen/arch/ppc/include/asm/bitops.h
>> +++ b/xen/arch/ppc/include/asm/bitops.h
>> @@ -173,7 +173,7 @@ static inline int __test_and_clear_bit(int nr, 
>> volatile void *addr)
>> 
>>  #define flsl(x) generic_flsl(x)
>>  #define fls(x) generic_flsl(x)
>> -#define ffs(x) ({ unsigned int t_ = (x); fls(t_ & -t_); })
>> +#define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
>>  #define ffsl(x) ({ unsigned long t_ = (x); flsl(t_ & -t_); })
>> 
>>  /**
>> diff --git a/xen/arch/x86/include/asm/bitops.h 
>> b/xen/arch/x86/include/asm/bitops.h
>> index 5a71afbc89d5..122767fc0d10 100644
>> --- a/xen/arch/x86/include/asm/bitops.h
>> +++ b/xen/arch/x86/include/asm/bitops.h
>> @@ -430,7 +430,7 @@ static inline int ffsl(unsigned long x)
>>      return (int)r+1;
>>  }
>> 
>> -static inline int ffs(unsigned int x)
>> +static always_inline unsigned int arch_ffs(unsigned int x)
>>  {
>>      int r;
>> 
>> @@ -440,6 +440,7 @@ static inline int ffs(unsigned int x)
>>            "1:" : "=r" (r) : "rm" (x));
>>      return r + 1;
>>  }
>> +#define arch_ffs arch_ffs
>> 
>>  /**
>>   * fls - find last bit set
>> diff --git a/xen/common/Makefile b/xen/common/Makefile
>> index d512cad5243f..21a4fb4c7166 100644
>> --- a/xen/common/Makefile
>> +++ b/xen/common/Makefile
>> @@ -1,5 +1,6 @@
>>  obj-$(CONFIG_ARGO) += argo.o
>>  obj-y += bitmap.o
>> +obj-bin-$(CONFIG_DEBUG) += bitops.init.o
>>  obj-$(CONFIG_GENERIC_BUG_FRAME) += bug.o
>>  obj-$(CONFIG_HYPFS_CONFIG) += config_data.o
>>  obj-$(CONFIG_CORE_PARKING) += core_parking.o
>> diff --git a/xen/common/bitops.c b/xen/common/bitops.c
>> new file mode 100644
>> index 000000000000..8c161b8ea7fa
>> --- /dev/null
>> +++ b/xen/common/bitops.c
>> @@ -0,0 +1,19 @@
>> +#include <xen/bitops.h>
>> +#include <xen/boot-check.h>
>> +#include <xen/init.h>
>> +
>> +static void __init test_ffs(void)
>> +{
>> +    /* unsigned int ffs(unsigned int) */
>> +    CHECK(ffs, 0, 0);
>> +    CHECK(ffs, 1, 1);
>> +    CHECK(ffs, 3, 1);
>> +    CHECK(ffs, 7, 1);
>> +    CHECK(ffs, 6, 2);
>> +    CHECK(ffs, 0x80000000U, 32);
>> +}
>> +
>> +static void __init __constructor test_bitops(void)
>> +{
>> +    test_ffs();
>> +}
>> diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
>> index cd405df96180..f7e90a2893a5 100644
>> --- a/xen/include/xen/bitops.h
>> +++ b/xen/include/xen/bitops.h
>> @@ -31,6 +31,23 @@ unsigned int __pure generic_flsl(unsigned long x);
>> 
>>  #include <asm/bitops.h>
>> 
>> +/*
>> + * Find First/Last Set bit (all forms).
>> + *
>> + * Bits are labelled from 1.  Returns 0 if given 0.
>> + */
>> +static always_inline __pure unsigned int ffs(unsigned int x)
>> +{
>> +    if ( __builtin_constant_p(x) )
>> +        return __builtin_ffs(x);
>> +
>> +#ifdef arch_ffs
>> +    return arch_ffs(x);
>> +#else
>> +    return generic_ffsl(x);
>> +#endif
>> +}
>> +
>>  /* --------------------- Please tidy below here --------------------- 
>> */
>> 
>>  #ifndef find_next_bit
>> --
>> 2.30.2
>> 

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:02:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:02:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733310.1139469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwHT-0004ru-7y; Fri, 31 May 2024 07:02:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733310.1139469; Fri, 31 May 2024 07:02:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwHT-0004rn-4d; Fri, 31 May 2024 07:02:27 +0000
Received: by outflank-mailman (input) for mailman id 733310;
 Fri, 31 May 2024 07:02:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCwHR-0004rg-O0
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:02:25 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bab90099-1f1b-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:02:22 +0200 (CEST)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2ea9386cde0so5926801fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 00:02:22 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57a31c6d4a0sm660261a12.77.2024.05.31.00.02.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 00:02:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bab90099-1f1b-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717138942; x=1717743742; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Bxmq4PUQqJEOSiOM3m0L/AXci6h8bSFL7LysBvNsQx8=;
        b=cFJu5GIV8DipEKoTJ4LgfCnr6WiRTCbtp1NzCCTONOlJTV3xrY6PDV+sCSkzbZ+Xw7
         cHC8KwIgFFm+Hx0RN4KeYhnHhAnRyjl0bgtZUKcCZoFlLtWROV9XB+hYIvTkb8yqfnAS
         SRMPslX3WdlfPFcd408ooSmpc1zgGDk4uN2WIT1e629a2RUC4dMMok7IyeYdTPP0d5ES
         15Krxmd1EO/BMPWhgxdSrlANQTPxzyjVoMliVCLCLjU6EbmcIFMFmoFjEtJb3/byE3hb
         SlvbX6wNkRwgiNs20SC2WllMvTmLtcH/EFQb3sICmyVhX8fOJc2fxBWtyuU7GizDy2E5
         u+VA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717138942; x=1717743742;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Bxmq4PUQqJEOSiOM3m0L/AXci6h8bSFL7LysBvNsQx8=;
        b=sC8rgIA7f61LqNhN6nOs7XyQvatjcT1akypo1LnuNqbbPj5NfSFV/L3Mrt02CwC84B
         j+ijbwTCZR5knUJoq03Es6xNP8XMGc5j1kkr2ayz2Q0pYpXl1uGrP6qb643XJqbvcBv0
         2+aG/FZZmp7yqlkT8bCTtaz7dUeyJwmrjSN4LhlR1iBPG4ecLT+mboLb0fhXarYiQr4y
         ugh0c34V5sMndTAb7xz9SP4vCZozJGWio1azErueQ/hhMdi+7druEP6tgJ6wKDWgBvDu
         G8KM6xlKBtSZv01Te7U05WDJrHO8nHtUoZcb0ez4ZWLnb5R/DNEVdRLT2HkWgOwhEt8C
         vxAw==
X-Forwarded-Encrypted: i=1; AJvYcCVOZ2OOteFLYYo6P0rkNZhCQqny6MszVcsY1f5HRF9jyUTXQyJUxVkVz+Dfs/Y+R+zfUW1HpO6CEP6ZPCL582jwIL8fQxJf67SbOrFspiQ=
X-Gm-Message-State: AOJu0Yz3HhamdodkwugdtvZdzzEsD9rzdHomzjc8l0Xb25gr0jukBvp6
	3VzhLujqypbFLX7KvEJNkH1qVfa4xzOK8j3EEbKKmH696nOLe2y2a0nJTH4Mrw==
X-Google-Smtp-Source: AGHT+IGwC+tSg9suj5VY3MS7ejlCHPdm5Nysm2Sjcr8LbL4XrQtprP00IoX+ksiP/h6BjXPazTyIgw==
X-Received: by 2002:a19:5e5d:0:b0:52a:622c:dae8 with SMTP id 2adb3069b0e04-52b89596635mr504443e87.20.1717138942252;
        Fri, 31 May 2024 00:02:22 -0700 (PDT)
Message-ID: <075168de-1a3f-4f23-a445-8ba74082fb8f@suse.com>
Date: Fri, 31 May 2024 09:02:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns false
 if CPU operations are underway
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com> <ZldDvH8GhhQcu5NX@macbook>
 <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com> <ZldUgJRJvytRsjhu@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZldUgJRJvytRsjhu@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 18:14, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 05:49:48PM +0200, Jan Beulich wrote:
>> On 29.05.2024 17:03, Roger Pau Monné wrote:
>>> On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
>>>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>>>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
>>>>> a cpu_hotplug_{begin,done}() region the function will still return success,
>>>>> because a CPU taking the rwlock in read mode after having taken it in write
>>>>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
>>>>> as it should always return false when called with a CPU hot{,un}plug operation
>>>>> is in progress.
>>>>
>>>> I'm not sure I can agree with this. The CPU doing said operation ought to be
>>>> aware of what it is itself doing. And all other CPUs will get back false from
>>>> get_cpu_maps().
>>>
>>> Well, the CPU is aware in the context of cpu_{up,down}(), but not in
>>> the interrupts that might be handled while that operation is in
>>> progress, see below for a concrete example.
>>>
>>>>>  Otherwise the logic in send_IPI_mask() for example is wrong,
>>>>> as it could decide to use the shorthand even when a CPU operation is in
>>>>> progress.
>>>>
>>>> It's also not becoming clear what's wrong there: As long as a CPU isn't
>>>> offline enough to not be in cpu_online_map anymore, it may well need to still
>>>> be the target of IPIs, and targeting it with a shorthand then is still fine.
>>>
>>> The issue is in the online path: there's a window where the CPU is
>>> online (and the lapic active), but cpu_online_map hasn't been updated
>>> yet.  A specific example would be time_calibration() being executed on
>>> the CPU that is running cpu_up().  That could result in a shorthand
>>> IPI being used, but the mask in r.cpu_calibration_map not containing
>>> the CPU that's being brought up online because it's not yet added to
>>> cpu_online_map.  Then the number of CPUs actually running
>>> time_calibration_rendezvous_fn won't match the weight of the cpumask
>>> in r.cpu_calibration_map.
>>
>> I see, but maybe only partly. Prior to the CPU having its bit set in
>> cpu_online_map, can it really take interrupts already? Shouldn't it be
>> running with IRQs off until later, thus preventing it from making it
>> into the rendezvous function in the first place? But yes, I can see
>> how the IRQ (IPI) then being delivered later (once IRQs are enabled)
>> might cause problems, too.
> 
> The interrupt will get set in IRR and handled when interrupts are
> enabled.
> 
>>
>> Plus, with how the rendezvous function is invoked (via
>> on_selected_cpus() with the mask copied from cpu_online_map), the
>> first check in smp_call_function_interrupt() ought to prevent the
>> function from being called on the CPU being onlined. A problem would
>> arise though if the IPI arrived later and call_data was already
>> (partly or fully) overwritten with the next request.
> 
> Yeah, there's a small window where the fields in call_data are out of
> sync.
> 
>>>> In any event this would again affect only the CPU leading the CPU operation,
>>>> which should clearly know at which point(s) it is okay to send IPIs. Are we
>>>> actually sending any IPIs from within CPU-online or CPU-offline paths?
>>>
>>> Yes, I've seen the time rendezvous happening while in the middle of a
>>> hotplug operation, and the CPU coordinating the rendezvous being the
>>> one doing the CPU hotplug operation, so get_cpu_maps() returning true.
>>
>> Right, yet together with ...
>>
>>>> Together with the earlier paragraph the critical window would be between the
>>>> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
>>>> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
>>>> then the question would be what bad, if any, would happen to that CPU if an
>>>> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
>>>> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
>>>>
>>>>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
>>>>> already hold in write mode by the current CPU, as read_trylock() would
>>>>> otherwise return true.
>>>>>
>>>>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
>>>>
>>>> I'm puzzled by this as well: Prior to that and the change referenced by its
>>>> Fixes: tag, recursive spin locks were used. For the purposes here that's the
>>>> same as permitting read locking even when the write lock is already held by
>>>> the local CPU.
>>>
>>> I see, so the Fixes should be:
>>>
>>> x86/smp: use APIC ALLBUT destination shorthand when possible
>>>
>>> Instead, which is the commit that started using get_cpu_maps() in
>>> send_IPI_mask().
>>
>> ... this I then wonder whether it's really only the condition in
>> send_IPI_mask() which needs further amending, rather than fiddling with
>> get_cpu_maps().
> 
> That the other option, but I have impression it's more fragile to
> adjust the condition in send_IPI_mask() rather than fiddle with
> get_cpu_maps().
> 
> However if that's the preference I can adjust.

I guess we need other REST input here then. The two of us clearly disagree on
what use of get_cpu_maps() is meant to guarantee. And I deem fiddling with
common code here more risky (and more intrusive - the other change would be
a single-line code change afaict, plus extending the related comment).

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:06:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:06:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733316.1139478 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwL9-0005Ti-LZ; Fri, 31 May 2024 07:06:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733316.1139478; Fri, 31 May 2024 07:06:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwL9-0005Tb-J5; Fri, 31 May 2024 07:06:15 +0000
Received: by outflank-mailman (input) for mailman id 733316;
 Fri, 31 May 2024 07:06:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCwL7-0005TV-Ot
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:06:13 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 43bbeb89-1f1c-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:06:12 +0200 (CEST)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a63359aaaa6so194841366b.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 00:06:12 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67eab84ebcsm54252166b.150.2024.05.31.00.06.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 00:06:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43bbeb89-1f1c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717139172; x=1717743972; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xc35MN7Ki9JfS9cT2TThfFPfvWqpsYztpVrHRcx1pkY=;
        b=L5QPTcyDwo0LbpP0iz1783/3LPZOgSoIesqCQmBCLmAtGQ49cgIWPzrvoUaFkMU5wt
         LBKzlFlUtRpkSPN2S4m2fM9Hgq8Ooyn3bWoNkVlH5xV4QVnVOiMtHnOx98fAnSTakAYR
         9qpYMxl7ra1WcT4o1IMsI2tJ2ussxHi71xMIx9o1Czm7p6NIjxjH3VPcY/RTnn+nVLm5
         rW1lyf+Wnp3SAyMLlSyEpagozktV7NITYIrWh2oD2DEGgndnjpv4I8gfjhHOcpn56g7W
         8Q90FlSQHjzJb21o9rKUnGnHyXUjrd0dyM3sBMPGQ0+3m5YvppMwad6fMhYdt6FEqrOA
         nOgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717139172; x=1717743972;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xc35MN7Ki9JfS9cT2TThfFPfvWqpsYztpVrHRcx1pkY=;
        b=S2CWIpoNC/5G36Rc6stQ54dDxZU5eTC1k1HVkx3lYqycDP4FbZL1RYztShsGOKYCy0
         a8xAkZqfXYNeSLMMVOUE1YdjOWtvIGElqulGKsYCI5YmzMj38cF7OaJQaBw0Shlnin1k
         Cl2W27edjMELoDK8qYZlfZgwE7xTNQ7n+z/zUomaU2PsqsuFDvio3TZ44ROmz09JJQAr
         Nod0z9klquXKQBEkDG90dcjiRCoZwmJyOhOrlTPyv4BZGRBHTt4MPe5lXC3ar7FMhZ8/
         Wjm2Khy7/ZNkfqEW31jJ3AXUnIgZ2YErMFHH5qcdaTFyu6vIgkLHqxLymxFDTbFCYYMx
         EYjg==
X-Forwarded-Encrypted: i=1; AJvYcCXc+e9kQ8OjUDbfL4cXUP/CSeSq/DbIBroPtd2AC3YXAiYyl2nK66Ktumm3Ulm7UNuRTwca+d7Djwl19ajM8QjGV8wFYi6DBpcAua2MY7E=
X-Gm-Message-State: AOJu0Yzvsit9xorpFaThHTfld3yFqKjrJzVT2VDWcBZ8Jd6bK1q1MFP8
	ZwsPNXSkfqbqoK4icAykWOkufcVk2sKWaq7VKLVnFAJzUcdRFTG74Bm3TYGHwQ==
X-Google-Smtp-Source: AGHT+IGUN27DhbOrV6/mgfgehwQDzVP2rQrMhqZfXiFC5o+Y3sbDL/JZL94mkaTmP1jsrZ6wdnR5gg==
X-Received: by 2002:a17:906:d8c5:b0:a68:137b:949 with SMTP id a640c23a62f3a-a6820516e7bmr55450366b.29.1717139172056;
        Fri, 31 May 2024 00:06:12 -0700 (PDT)
Message-ID: <011d96d4-8778-430f-bb93-4a48f7ddd32c@suse.com>
Date: Fri, 31 May 2024 09:06:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 4/9] x86/irq: describe how the interrupt CPU
 movement works
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-5-roger.pau@citrix.com>
 <24201381-2a43-427e-a3e5-1de316d212b8@suse.com> <ZldJgwu2nDGv2HJP@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZldJgwu2nDGv2HJP@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.05.2024 17:28, Roger Pau Monné wrote:
> On Wed, May 29, 2024 at 03:57:19PM +0200, Jan Beulich wrote:
>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/include/asm/irq.h
>>> +++ b/xen/arch/x86/include/asm/irq.h
>>> @@ -28,6 +28,32 @@ typedef struct {
>>>  
>>>  struct irq_desc;
>>>  
>>> +/*
>>> + * Xen logic for moving interrupts around CPUs allows manipulating interrupts
>>> + * that target remote CPUs.  The logic to move an interrupt from CPU(s) is as
>>> + * follows:
>>> + *
>>> + * 1. cpu_mask and vector is copied to old_cpu_mask and old_vector.
>>> + * 2. New cpu_mask and vector are set, vector is setup at the new destination.
>>> + * 3. move_in_progress is set.
>>> + * 4. Interrupt source is updated to target new CPU and vector.
>>> + * 5. Interrupts arriving at old_cpu_mask are processed normally.
>>> + * 6. When an interrupt is delivered at the new destination (cpu_mask) as part
>>> + *    of acking the interrupt move_in_progress is cleared and move_cleanup_count
>>
>> Nit: A comma after "interrupt" may help reading.
>>
>>> + *    is set to the weight of online CPUs in old_cpu_mask.
>>> + *    IRQ_MOVE_CLEANUP_VECTOR is sent to all CPUs in old_cpu_mask.
>>
>> These last two steps aren't precise enough, compared to what the code does.
>> old_cpu_mask is first reduced to online CPUs therein. If the result is non-
>> empty, what you describe is done. If, however, the result is empty, the
>> vector is released right away (this code may be there just in case, but I
>> think it shouldn't be omitted here).
> 
> I've left that out because I got the impression it made the text more
> complex to follow (with the extra branch) for no real benefit, but I'm
> happy to attempt to add it.

Why "no real benefit"? Isn't the goal to accurately describe what code does
(in various places)? If the result isn't an accurate description in one
specific regard, how reliable would the rest be from a reader's perspective?

>>> + * 7. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the
>>> + *    vector entry and decrease the count in move_cleanup_count.  The CPU that
>>> + *    sets move_cleanup_count to 0 releases the vector.
>>> + *
>>> + * Note that when interrupt movement (either move_in_progress or
>>> + * move_cleanup_count set) is in progress it's not possible to move the
>>> + * interrupt to yet a different CPU.
>>> + *
>>> + * By keeping the vector in the old CPU(s) configured until the interrupt is
>>> + * acked on the new destination Xen allows draining any pending interrupts at
>>> + * the old destinations.
>>> + */
>>>  struct arch_irq_desc {
>>>          s16 vector;                  /* vector itself is only 8 bits, */
>>>          s16 old_vector;              /* but we use -1 for unassigned  */
>>
>> I take it that it is not a goal to (also) describe under what conditions
>> an IRQ move may actually be initiated (IRQ_MOVE_PENDING)? I ask not the
>> least because the 2nd from last paragraph lightly touches that area.
> 
> Right, I was mostly focused on moves (forcefully) initiated from
> fixup_irqs(), which is different from the opportunistic affinity
> changes signaled by IRQ_MOVE_PENDING.
> 
> Not sure whether I want to mention this ahead of the list in a
> paragraph, or just add it as a step.  Do you have any preference?

I think ahead might be better. But I also won't insist on it being added.
Just if you don't, perhaps mention in the description that leaving that
out is intentional.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:10:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:10:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733326.1139489 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwPA-0007Eq-9i; Fri, 31 May 2024 07:10:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733326.1139489; Fri, 31 May 2024 07:10:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwPA-0007Ej-6a; Fri, 31 May 2024 07:10:24 +0000
Received: by outflank-mailman (input) for mailman id 733326;
 Fri, 31 May 2024 07:10:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iDPo=NC=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1sCwP9-0007DT-0g
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:10:23 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8393343-1f1c-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:10:22 +0200 (CEST)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-52b7b829bc7so1979639e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 00:10:21 -0700 (PDT)
Received: from [192.168.219.221] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 2adb3069b0e04-52b84d75f0asm231003e87.181.2024.05.31.00.10.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 31 May 2024 00:10:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8393343-1f1c-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717139421; x=1717744221; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=asD06tKeYOLm9DU4GcU4NryDvwgLkhMMNNwedXmQIS0=;
        b=GiQNEIAj+EM6ObHp5zY2Iisp0jS1UmSpZhwQEC7hoi2EaHpgTqwjGdAPf4YOR3WHc5
         cxxZVFYDDLbK94iLIjObCLZhRuEIrj0bnK7tdRiDQLiDn3THavFbVMaYbREEIiay7g/H
         ueqv+yD+ovqHaRFzxNHAYdfp1cl7a5jITw79WFkVBM2aEBpTk2VWwVmglvDwBnezQ3Rv
         6HSZijk3lty9KiZFK47mCCK3c07RGk7R2KgqPa2C4S1XOAHkLky5lGqZknOIyG7BUjg0
         B5Ret+9xjHRsludKEeeH6b8RojJpZwU21kYvv/qNfwHhnvqC61M/oprBpui+PiLLaQtR
         Hr6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717139421; x=1717744221;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=asD06tKeYOLm9DU4GcU4NryDvwgLkhMMNNwedXmQIS0=;
        b=vw8Za9g+9jOFDXUVZ2T2DGttopgZV0QGjl1y4ewF8s7eavMJ04iXAlsHkFIQZ0Kt9c
         saE0+DCmCSep0tzF4yQ7A9dPDkYK2UtC+9oOYMb1JB2Ovtvr2JlyZshXwoAUKbnmKJdB
         GGD89d1qu8J8JVb3LObQB8JLYOfmo+hP7qT7v0z2xu87y4ogTRSHG5enqKrdUH1+F1KC
         6Fx68ll3ggA8vTnkNWcTXqttic/IBjZ0SZA++IgYru9rlVbHyDkG1JN6HSeYoxUg/2L0
         4Sg067TDqdRQbQyBO3ZpSJKYUZmLqgfKx1epGEXO5Eb/AiyhdE4TEhszIAxDlp7j6Iub
         R13Q==
X-Forwarded-Encrypted: i=1; AJvYcCXeVDeBPSeF9sUTjCPtMFYPScInkp3pSjCaaJI7aGfcTqFGGYpBC7tqn/UTIgyQjfZy16N/6XkTvzlcXrLd/UIUiPGeaXru0TQ+zNBDxKE=
X-Gm-Message-State: AOJu0YzNqLx3xJ/9+dDD1biBrA3a5aDV5DQOjyce/2v2hhiBvi5au3K8
	N8cfdC60Fx3xqEIrrV0lPdSrQs05B5jhDcjBZS0mTYhUn/xvk9Ek
X-Google-Smtp-Source: AGHT+IGLkYNkMpXu5uTi5c283Ga1Vd77e13m6jsBgozEB6pKOjR9Nyvyhg6kB+g0D7a8yQDUxb3TnA==
X-Received: by 2002:ac2:5222:0:b0:52b:6d5c:bf68 with SMTP id 2adb3069b0e04-52b896cbe85mr454221e87.57.1717139421108;
        Fri, 31 May 2024 00:10:21 -0700 (PDT)
Message-ID: <a0e350d7f15c36025624e1ed9328062e6be3a752.camel@gmail.com>
Subject: Re: [PATCH for-4.19 0/3] CI: Misc improvements
From: "Oleksii K." <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	 <xen-devel@lists.xenproject.org>
Cc: Roger Pau =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Michal Orzel <michal.orzel@amd.com>,
 Marek =?ISO-8859-1?Q?Marczykowski-G=F3recki?=
 <marmarek@invisiblethingslab.com>
Date: Fri, 31 May 2024 09:10:20 +0200
In-Reply-To: <20240529141945.41669-1-andrew.cooper3@citrix.com>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 (3.52.1-1.fc40) 
MIME-Version: 1.0

On Wed, 2024-05-29 at 15:19 +0100, Andrew Cooper wrote:
> All found while making extensive use of Gitlab CI for the bitops boot
> testing.
>=20
> For 4.19.=C2=A0 It's all very low risk, and improves the
> utility/useability of our
> testing.
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii
>=20
> Andrew Cooper (3):
> =C2=A0 CI: Remove CI_COMMIT_REF_PROTECTED requirement for HW jobs
> =C2=A0 CI: Use a debug build of Xen for the Xilinx HW tests
> =C2=A0 CI: Improve serial handling in qemu-smoke-ppc64le.sh
>=20
> =C2=A0automation/gitlab-ci/test.yaml=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 8 ++++----
> =C2=A0automation/scripts/qemu-smoke-ppc64le.sh | 13 +++++++------
> =C2=A02 files changed, 11 insertions(+), 10 deletions(-)
>=20



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:31:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:31:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733330.1139498 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwjS-0002GL-RC; Fri, 31 May 2024 07:31:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733330.1139498; Fri, 31 May 2024 07:31:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwjS-0002GE-Oa; Fri, 31 May 2024 07:31:22 +0000
Received: by outflank-mailman (input) for mailman id 733330;
 Fri, 31 May 2024 07:31:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hFcc=NC=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCwjR-0002G4-BE
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:31:21 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5e48751-1f1f-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:31:19 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-420180b59b7so18984895e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 00:31:19 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212b858f09sm16316055e9.22.2024.05.31.00.31.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 31 May 2024 00:31:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5e48751-1f1f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717140679; x=1717745479; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=loOuozFLfXQWM3tGBsR1neCzIFT2++j29Qc+Dp3UfL4=;
        b=aqF40UOyGh+/qNbECjfnXukOCjeqqmeOHo6hJZ1ELP+My3lC5Eisy8o3ybuXPXFXix
         sbhsGdmJ4nZZOLCqm47vhcp1PxG5tkNgoZuqxxrSpoR/40stHxh2+dPg29IjeqH0maef
         in02srQ0HwPDk09YLpHvMByTQVGvCHJcR+ATc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717140679; x=1717745479;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=loOuozFLfXQWM3tGBsR1neCzIFT2++j29Qc+Dp3UfL4=;
        b=SMFu4wh0nlJHaJSceFOc9OS5A/dYAJHE/IhUU7pp/rjKpp6nSDfQuOJjrPRXgeoa1c
         ojuVcRslDojMohi5AJayvT9FOdnk9bKTf3GSUGrejjhmQjw4UvKM4ANsf4cwtTni3z0V
         J3AJTKEETZqClDxrWGm/fazgurleNtYr4QWanOFRKSyArI+yHxEIAeIVhd/BKTBqISrI
         P7ET2wbBFtri1WDg1EeF8htsQsGu8dg+1m6RN3H0xqyAhL7fgN0XrXd14Vclcqe4tAiO
         odQHfbg37p2sEHb+kq9swYjMmMgWlBd+o+j9o94Lu6Yn0xoP8wDdQHGtOUq+MZFdLEZW
         na7g==
X-Forwarded-Encrypted: i=1; AJvYcCUpQzSBMtA8wedCpzFOiOzvfNgHekk6eMeXUiFGj6kKMX/aGchwXghMaItgqi4Hv5wt1QUtrmnCZN52kRVoaL2qCchkG0MJ2jR+OZ7nTDY=
X-Gm-Message-State: AOJu0Yy+NbRaq4i3DGOAl2YWqv2N+bSUZgexWkXl26R32kAAVMUzx/I7
	0A6nfzAFiiEfvsKSljeOCMP2rOmroEzaZFN3C402x765497Cp0r84FBzSBUYPS0=
X-Google-Smtp-Source: AGHT+IGoM6O0GmM5dunanaQI/oHbqlPeFtNk3GGgfcS+uaH6VIPO9MqTj1nVZgpTcwGe7ZEfGCQoGw==
X-Received: by 2002:a05:600c:1d21:b0:421:2b33:2522 with SMTP id 5b1f17b1804b1-4212e0bd913mr8729715e9.34.1717140679075;
        Fri, 31 May 2024 00:31:19 -0700 (PDT)
Date: Fri, 31 May 2024 09:31:17 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns
 false if CPU operations are underway
Message-ID: <Zll8xQJiM83EZhTd@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>
 <ZldDvH8GhhQcu5NX@macbook>
 <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com>
 <ZldUgJRJvytRsjhu@macbook>
 <075168de-1a3f-4f23-a445-8ba74082fb8f@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <075168de-1a3f-4f23-a445-8ba74082fb8f@suse.com>

On Fri, May 31, 2024 at 09:02:20AM +0200, Jan Beulich wrote:
> On 29.05.2024 18:14, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 05:49:48PM +0200, Jan Beulich wrote:
> >> On 29.05.2024 17:03, Roger Pau Monné wrote:
> >>> On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
> >>>> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>>>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
> >>>>> a cpu_hotplug_{begin,done}() region the function will still return success,
> >>>>> because a CPU taking the rwlock in read mode after having taken it in write
> >>>>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
> >>>>> as it should always return false when called with a CPU hot{,un}plug operation
> >>>>> is in progress.
> >>>>
> >>>> I'm not sure I can agree with this. The CPU doing said operation ought to be
> >>>> aware of what it is itself doing. And all other CPUs will get back false from
> >>>> get_cpu_maps().
> >>>
> >>> Well, the CPU is aware in the context of cpu_{up,down}(), but not in
> >>> the interrupts that might be handled while that operation is in
> >>> progress, see below for a concrete example.
> >>>
> >>>>>  Otherwise the logic in send_IPI_mask() for example is wrong,
> >>>>> as it could decide to use the shorthand even when a CPU operation is in
> >>>>> progress.
> >>>>
> >>>> It's also not becoming clear what's wrong there: As long as a CPU isn't
> >>>> offline enough to not be in cpu_online_map anymore, it may well need to still
> >>>> be the target of IPIs, and targeting it with a shorthand then is still fine.
> >>>
> >>> The issue is in the online path: there's a window where the CPU is
> >>> online (and the lapic active), but cpu_online_map hasn't been updated
> >>> yet.  A specific example would be time_calibration() being executed on
> >>> the CPU that is running cpu_up().  That could result in a shorthand
> >>> IPI being used, but the mask in r.cpu_calibration_map not containing
> >>> the CPU that's being brought up online because it's not yet added to
> >>> cpu_online_map.  Then the number of CPUs actually running
> >>> time_calibration_rendezvous_fn won't match the weight of the cpumask
> >>> in r.cpu_calibration_map.
> >>
> >> I see, but maybe only partly. Prior to the CPU having its bit set in
> >> cpu_online_map, can it really take interrupts already? Shouldn't it be
> >> running with IRQs off until later, thus preventing it from making it
> >> into the rendezvous function in the first place? But yes, I can see
> >> how the IRQ (IPI) then being delivered later (once IRQs are enabled)
> >> might cause problems, too.
> > 
> > The interrupt will get set in IRR and handled when interrupts are
> > enabled.
> > 
> >>
> >> Plus, with how the rendezvous function is invoked (via
> >> on_selected_cpus() with the mask copied from cpu_online_map), the
> >> first check in smp_call_function_interrupt() ought to prevent the
> >> function from being called on the CPU being onlined. A problem would
> >> arise though if the IPI arrived later and call_data was already
> >> (partly or fully) overwritten with the next request.
> > 
> > Yeah, there's a small window where the fields in call_data are out of
> > sync.
> > 
> >>>> In any event this would again affect only the CPU leading the CPU operation,
> >>>> which should clearly know at which point(s) it is okay to send IPIs. Are we
> >>>> actually sending any IPIs from within CPU-online or CPU-offline paths?
> >>>
> >>> Yes, I've seen the time rendezvous happening while in the middle of a
> >>> hotplug operation, and the CPU coordinating the rendezvous being the
> >>> one doing the CPU hotplug operation, so get_cpu_maps() returning true.
> >>
> >> Right, yet together with ...
> >>
> >>>> Together with the earlier paragraph the critical window would be between the
> >>>> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
> >>>> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
> >>>> then the question would be what bad, if any, would happen to that CPU if an
> >>>> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
> >>>> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
> >>>>
> >>>>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
> >>>>> already hold in write mode by the current CPU, as read_trylock() would
> >>>>> otherwise return true.
> >>>>>
> >>>>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
> >>>>
> >>>> I'm puzzled by this as well: Prior to that and the change referenced by its
> >>>> Fixes: tag, recursive spin locks were used. For the purposes here that's the
> >>>> same as permitting read locking even when the write lock is already held by
> >>>> the local CPU.
> >>>
> >>> I see, so the Fixes should be:
> >>>
> >>> x86/smp: use APIC ALLBUT destination shorthand when possible
> >>>
> >>> Instead, which is the commit that started using get_cpu_maps() in
> >>> send_IPI_mask().
> >>
> >> ... this I then wonder whether it's really only the condition in
> >> send_IPI_mask() which needs further amending, rather than fiddling with
> >> get_cpu_maps().
> > 
> > That the other option, but I have impression it's more fragile to
> > adjust the condition in send_IPI_mask() rather than fiddle with
> > get_cpu_maps().
> > 
> > However if that's the preference I can adjust.
> 
> I guess we need other REST input here then. The two of us clearly disagree on
> what use of get_cpu_maps() is meant to guarantee. And I deem fiddling with
> common code here more risky (and more intrusive - the other change would be
> a single-line code change afaict, plus extending the related comment).

How do you envision that other change to be done?  Adding an extra
variable and toggling it in cpu_hotplug_{begin,done}() to signal
whether a CPU hotplug is in progress?

If I go this route I would like to add a comment to get_cpu_maps() in
order to notice this IMO weird property of succeeding when calling
from a CPU that's performing a hotplug operation.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:39:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:39:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733335.1139508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwr5-0002uU-IZ; Fri, 31 May 2024 07:39:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733335.1139508; Fri, 31 May 2024 07:39:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCwr5-0002uN-G0; Fri, 31 May 2024 07:39:15 +0000
Received: by outflank-mailman (input) for mailman id 733335;
 Fri, 31 May 2024 07:39:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hFcc=NC=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCwr4-0002uH-JI
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:39:14 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df19e9f9-1f20-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:39:12 +0200 (CEST)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4211e42e362so18835515e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 00:39:11 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421270ad52dsm45890645e9.42.2024.05.31.00.39.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 31 May 2024 00:39:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df19e9f9-1f20-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717141151; x=1717745951; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=cDc024rXtosex59CMa4/lBxCrn3pW6jBZBTVPYFmpqI=;
        b=WyUT4zFDZBEb7W1pU4RJchCwSX11zENtTwBJpO0X7b6/cSd0EAF4pA9n36435Y57g7
         PitQ91wFJ7x6XGtejGkQujbNdt1WXHmi0iB5pkri9qe4L2cejxn9d+zRqPuOhZE7kb1B
         vRRa9Zh1NDiyqC394M/G71/wWAkzrJA6Vl6j4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717141151; x=1717745951;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cDc024rXtosex59CMa4/lBxCrn3pW6jBZBTVPYFmpqI=;
        b=ZTQ2NTeJpj/D6goEd+xeR8KWuyhGbPS9tMi0KJG2ZZsXUaEfCjstYVlYx0n9xINk22
         r+2qEfx2AJBLxi9ZDM7DJxhQqJyu0TxvXdjOtrip28WnFkHx7ybmESslWw/LAqBh1PC+
         0gNZhze6Yx8Tru4zy6+vKnBXrBlTz3cFo2Eidr4Ba5jw+q604FgtwpO5V33Hf+S/XCLQ
         a/Ia0ETiKeCv5c7PiY1Z2/E5aE0wPEjmLkiFKeQQBUqtAdKmUW9LlQpKKwSzZDRWTLIv
         tw1XsRXqiLpC83W+xVV1Hphpo5NIv8FljG/oMSXLyBkpX7ulIWsMt9LRf/uP5NFO/T/7
         FiZg==
X-Forwarded-Encrypted: i=1; AJvYcCWJVEwm+Vf9hRrWH1e0u1oKBp4sC1CZcuH1ctLzj94dhCKXjWihiQfEdKktJ7trhQ+vOjpOr15XKrEKFWdB+vHipO1SupirRM4oLekcQ/8=
X-Gm-Message-State: AOJu0Yx+BvlKgB6z4oybA7dpWeisQkadLmHHsKbg0HpDN8/N1D44EWxF
	c1l0wKX230HX4froNXEKQR6GSOxwyBiEqSEBEoTRgWkxbF5BVkAwU5pQ9Wn2AJI=
X-Google-Smtp-Source: AGHT+IGtWhplZak+P2xZbJ5xQnjQ1blltKczKDPJirQKSrAYXHU9PD/YUY00ruNvL7VgR8Rv6A2yug==
X-Received: by 2002:a05:600c:450b:b0:421:2990:7f8b with SMTP id 5b1f17b1804b1-4212e044c39mr9594985e9.3.1717141150893;
        Fri, 31 May 2024 00:39:10 -0700 (PDT)
Date: Fri, 31 May 2024 09:39:09 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 4/9] x86/irq: describe how the interrupt CPU
 movement works
Message-ID: <Zll-nd7hdZDZy-gn@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-5-roger.pau@citrix.com>
 <24201381-2a43-427e-a3e5-1de316d212b8@suse.com>
 <ZldJgwu2nDGv2HJP@macbook>
 <011d96d4-8778-430f-bb93-4a48f7ddd32c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <011d96d4-8778-430f-bb93-4a48f7ddd32c@suse.com>

On Fri, May 31, 2024 at 09:06:10AM +0200, Jan Beulich wrote:
> On 29.05.2024 17:28, Roger Pau Monné wrote:
> > On Wed, May 29, 2024 at 03:57:19PM +0200, Jan Beulich wrote:
> >> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>> --- a/xen/arch/x86/include/asm/irq.h
> >>> +++ b/xen/arch/x86/include/asm/irq.h
> >>> @@ -28,6 +28,32 @@ typedef struct {
> >>>  
> >>>  struct irq_desc;
> >>>  
> >>> +/*
> >>> + * Xen logic for moving interrupts around CPUs allows manipulating interrupts
> >>> + * that target remote CPUs.  The logic to move an interrupt from CPU(s) is as
> >>> + * follows:
> >>> + *
> >>> + * 1. cpu_mask and vector is copied to old_cpu_mask and old_vector.
> >>> + * 2. New cpu_mask and vector are set, vector is setup at the new destination.
> >>> + * 3. move_in_progress is set.
> >>> + * 4. Interrupt source is updated to target new CPU and vector.
> >>> + * 5. Interrupts arriving at old_cpu_mask are processed normally.
> >>> + * 6. When an interrupt is delivered at the new destination (cpu_mask) as part
> >>> + *    of acking the interrupt move_in_progress is cleared and move_cleanup_count
> >>
> >> Nit: A comma after "interrupt" may help reading.
> >>
> >>> + *    is set to the weight of online CPUs in old_cpu_mask.
> >>> + *    IRQ_MOVE_CLEANUP_VECTOR is sent to all CPUs in old_cpu_mask.
> >>
> >> These last two steps aren't precise enough, compared to what the code does.
> >> old_cpu_mask is first reduced to online CPUs therein. If the result is non-
> >> empty, what you describe is done. If, however, the result is empty, the
> >> vector is released right away (this code may be there just in case, but I
> >> think it shouldn't be omitted here).
> > 
> > I've left that out because I got the impression it made the text more
> > complex to follow (with the extra branch) for no real benefit, but I'm
> > happy to attempt to add it.
> 
> Why "no real benefit"? Isn't the goal to accurately describe what code does
> (in various places)? If the result isn't an accurate description in one
> specific regard, how reliable would the rest be from a reader's perspective?

FWIW, it seemed to me the reduction of old_cpu_mask was (kind of) a
shortcut to what the normal path does, by releasing the vector early
if there are no online CPUs in old_cpu_mask.

Now that you made me look into it, I think after this series the
old_cpu_mask should never contain offline CPUs, as fixup_irqs() will
take care of removing offliend CPUs from old_cpu_mask, and freeing the
vector if the set becomes empty.

I will expand the comment to mention this case, and consider adjusting
it if this series get merged.

> >>> + * 7. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the
> >>> + *    vector entry and decrease the count in move_cleanup_count.  The CPU that
> >>> + *    sets move_cleanup_count to 0 releases the vector.
> >>> + *
> >>> + * Note that when interrupt movement (either move_in_progress or
> >>> + * move_cleanup_count set) is in progress it's not possible to move the
> >>> + * interrupt to yet a different CPU.
> >>> + *
> >>> + * By keeping the vector in the old CPU(s) configured until the interrupt is
> >>> + * acked on the new destination Xen allows draining any pending interrupts at
> >>> + * the old destinations.
> >>> + */
> >>>  struct arch_irq_desc {
> >>>          s16 vector;                  /* vector itself is only 8 bits, */
> >>>          s16 old_vector;              /* but we use -1 for unassigned  */
> >>
> >> I take it that it is not a goal to (also) describe under what conditions
> >> an IRQ move may actually be initiated (IRQ_MOVE_PENDING)? I ask not the
> >> least because the 2nd from last paragraph lightly touches that area.
> > 
> > Right, I was mostly focused on moves (forcefully) initiated from
> > fixup_irqs(), which is different from the opportunistic affinity
> > changes signaled by IRQ_MOVE_PENDING.
> > 
> > Not sure whether I want to mention this ahead of the list in a
> > paragraph, or just add it as a step.  Do you have any preference?
> 
> I think ahead might be better. But I also won't insist on it being added.
> Just if you don't, perhaps mention in the description that leaving that
> out is intentional.

No, I'm fine with adding it.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733343.1139534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0W-00054w-UE; Fri, 31 May 2024 07:49:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733343.1139534; Fri, 31 May 2024 07:49:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0W-00054B-OQ; Fri, 31 May 2024 07:49:00 +0000
Received: by outflank-mailman (input) for mailman id 733343;
 Fri, 31 May 2024 07:48:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0V-0004oC-DK
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:48:59 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3bede7cd-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:48:57 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0K-00000009XWY-3uLP; Fri, 31 May 2024 07:48:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bede7cd-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=rhxveVy4ScfquPw+Ix55dD12V3sv6PgWUOdWXrb/Dgc=; b=J4nlNlkbb/6E8DoWjrWzBe17s+
	CJVqEecG19yjleGxex9L/rhFT9qasHn6qna9Ox4xdCtfT2NL2WybUr+Btt1JBtMiN63u2AYIWSWYp
	x/b5RauYpwQbW6TG2qy5VU8YmAs4LK1RJ0azUWoICI7ChyFsp1PntmHJb1zjTQ1qbjAApinNKsCX3
	3UxuUS6VXFWwGUdhPdyE9jBuRrwWRfNrp862wHcecAY3+bCIcTlA9yjGFxVf6N9I6LrqtpjuKoH1w
	/hMhiL+hKd/CwPMEmeWIWF8VMYlcztrGWuM8vJw6GkXqKBTc9aMxPq8nTUApUw6ocrxKXVdwFsOXs
	sp7B+rOg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 02/14] ubd: untagle discard vs write zeroes not support handling
Date: Fri, 31 May 2024 09:47:57 +0200
Message-ID: <20240531074837.1648501-3-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Discard and Write Zeroes are different operation and implemented
by different fallocate opcodes for ubd.  If one fails the other one
can work and vice versa.

Split the code to disable the operations in ubd_handler to only
disable the operation that actually failed.

Fixes: 50109b5a03b4 ("um: Add support for DISCARD in the UBD Driver")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 arch/um/drivers/ubd_kern.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 0c9542d58c01b7..093c87879d08ba 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -449,10 +449,11 @@ static int bulk_req_safe_read(
 
 static void ubd_end_request(struct io_thread_req *io_req)
 {
-	if (io_req->error == BLK_STS_NOTSUPP &&
-	    req_op(io_req->req) == REQ_OP_DISCARD) {
-		blk_queue_max_discard_sectors(io_req->req->q, 0);
-		blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
+	if (io_req->error == BLK_STS_NOTSUPP) {
+		if (req_op(io_req->req) == REQ_OP_DISCARD)
+			blk_queue_max_discard_sectors(io_req->req->q, 0);
+		else if (req_op(io_req->req) == REQ_OP_WRITE_ZEROES)
+			blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
 	}
 	blk_mq_end_request(io_req->req, io_req->error);
 	kfree(io_req);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733342.1139529 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0W-00052f-LX; Fri, 31 May 2024 07:49:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733342.1139529; Fri, 31 May 2024 07:49:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0W-00052Y-Hl; Fri, 31 May 2024 07:49:00 +0000
Received: by outflank-mailman (input) for mailman id 733342;
 Fri, 31 May 2024 07:48:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0U-0004oC-O4
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:48:58 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3af473f0-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:48:56 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0I-00000009XVz-1ERf; Fri, 31 May 2024 07:48:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3af473f0-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=8qWYYD/WE7sZdc6vKaSwzAcVLajLtvsC2W0WllIXaIM=; b=1A1yb09khg0zzo88GBpLvTFsHc
	tV8ef2UQ1VXidXvG732ApqmyCupKhGkL5G59GgO8xU8oyRy3a7wLIj5Fqsb7ozie1sIBv/ltIsHYx
	+btWJXOahq4wAEa2PPIJxwFXAazEivQE/BryJ8mbFTKha/lhrUy32leI2VIH6j6MGV9GQOA2CaiHz
	ExUMMEXqbTiHF//iRBxcXt5wvKwfJDb8BxMSI0kYUpaS1XbArUgDpnhBV5zsm3WXP9abkponn0rgr
	g9AmlPK83Uxe92IoHb7Jc1v2oHu4zV0TiIRlrabbLStJXmWX3Wjs0EvPEaMCe032BaHlcz+c/JPQo
	uvqey7Tw==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 01/14] ubd: refactor the interrupt handler
Date: Fri, 31 May 2024 09:47:56 +0200
Message-ID: <20240531074837.1648501-2-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Instead of a separate handler function that leaves no work in the
interrupt hanler itself, split out a per-request end I/O helper and
clean up the coding style and variable naming while we're at it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 arch/um/drivers/ubd_kern.c | 49 ++++++++++++++------------------------
 1 file changed, 18 insertions(+), 31 deletions(-)

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index ef805eaa9e013d..0c9542d58c01b7 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -447,43 +447,30 @@ static int bulk_req_safe_read(
 	return n;
 }
 
-/* Called without dev->lock held, and only in interrupt context. */
-static void ubd_handler(void)
+static void ubd_end_request(struct io_thread_req *io_req)
 {
-	int n;
-	int count;
-
-	while(1){
-		n = bulk_req_safe_read(
-			thread_fd,
-			irq_req_buffer,
-			&irq_remainder,
-			&irq_remainder_size,
-			UBD_REQ_BUFFER_SIZE
-		);
-		if (n < 0) {
-			if(n == -EAGAIN)
-				break;
-			printk(KERN_ERR "spurious interrupt in ubd_handler, "
-			       "err = %d\n", -n);
-			return;
-		}
-		for (count = 0; count < n/sizeof(struct io_thread_req *); count++) {
-			struct io_thread_req *io_req = (*irq_req_buffer)[count];
-
-			if ((io_req->error == BLK_STS_NOTSUPP) && (req_op(io_req->req) == REQ_OP_DISCARD)) {
-				blk_queue_max_discard_sectors(io_req->req->q, 0);
-				blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
-			}
-			blk_mq_end_request(io_req->req, io_req->error);
-			kfree(io_req);
-		}
+	if (io_req->error == BLK_STS_NOTSUPP &&
+	    req_op(io_req->req) == REQ_OP_DISCARD) {
+		blk_queue_max_discard_sectors(io_req->req->q, 0);
+		blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
 	}
+	blk_mq_end_request(io_req->req, io_req->error);
+	kfree(io_req);
 }
 
 static irqreturn_t ubd_intr(int irq, void *dev)
 {
-	ubd_handler();
+	int len, i;
+
+	while ((len = bulk_req_safe_read(thread_fd, irq_req_buffer,
+			&irq_remainder, &irq_remainder_size,
+			UBD_REQ_BUFFER_SIZE)) >= 0) {
+		for (i = 0; i < len / sizeof(struct io_thread_req *); i++)
+			ubd_end_request((*irq_req_buffer)[i]);
+	}
+
+	if (len < 0 && len != -EAGAIN)
+		pr_err("spurious interrupt in %s, err = %d\n", __func__, len);
 	return IRQ_HANDLED;
 }
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733345.1139559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0Z-0005lU-Gq; Fri, 31 May 2024 07:49:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733345.1139559; Fri, 31 May 2024 07:49:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0Z-0005lK-D9; Fri, 31 May 2024 07:49:03 +0000
Received: by outflank-mailman (input) for mailman id 733345;
 Fri, 31 May 2024 07:49:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0Y-0004o6-7G
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:02 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3e4dbefc-1f22-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:49:01 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0P-00000009XZp-47jk; Fri, 31 May 2024 07:48:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e4dbefc-1f22-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=Hmvd8CugAGeadU1twTfeSXY4eccQybGsIHGkHL/IkB0=; b=my2INQcFxeQCNvgjmcsLc6Fb75
	+EFy+S+L2zWGIt2g4BBPXT+zKxxSbEKSxb3tBRS+43v7OGTOfpAaW1jjrvVXj8p6heokqg/3QgPvL
	cgEBbCQix9cOwyUKFuYg+h5o9MEKJotPtCESJKIwP3xUkAsfSsX/y9a+v4vJDSw3AsL5pT6hLH7Lu
	GuIXPs5ZW+9cBS0QF0sROYtRVuRTwsPOFFLgXpIrSa2T3CEX1IdFPbiV5seheDqGCLuYFDc6Yd5EF
	vffvu0LIjnfyd9A6B8PL+fYm3X9czzdyrgWKqx6Sa5+9qR0j+vzU7k6HmdEMljdnNzNPqRFDODR5A
	UiRrYXYA==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 04/14] block: take io_opt and io_min into account for max_sectors
Date: Fri, 31 May 2024 09:47:59 +0200
Message-ID: <20240531074837.1648501-5-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

The soft max_sectors limit is normally capped by the hardware limits and
an arbitrary upper limit enforced by the kernel, but can be modified by
the user.  A few drivers want to increase this limit (nbd, rbd) or
adjust it up or down based on hardware capabilities (sd).

Change blk_validate_limits to default max_sectors to the optimal I/O
size, or upgrade it to the preferred minimal I/O size if that is
larger than the kernel default if no optimal I/O size is provided based
on the logic in the SD driver.

This keeps the existing kernel default for drivers that do not provide
an io_opt or very big io_min value, but picks a much more useful
default for those who provide these hints, and allows to remove the
hacks to set the user max_sectors limit in nbd, rbd and sd.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 block/blk-settings.c |  7 +++++++
 drivers/block/nbd.c  |  2 +-
 drivers/block/rbd.c  |  1 -
 drivers/scsi/sd.c    | 29 +++++------------------------
 4 files changed, 13 insertions(+), 26 deletions(-)

diff --git a/block/blk-settings.c b/block/blk-settings.c
index effeb9a639bb45..a49abdb3554834 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -153,6 +153,13 @@ static int blk_validate_limits(struct queue_limits *lim)
 		if (lim->max_user_sectors < PAGE_SIZE / SECTOR_SIZE)
 			return -EINVAL;
 		lim->max_sectors = min(max_hw_sectors, lim->max_user_sectors);
+	} else if (lim->io_opt) {
+		lim->max_sectors =
+			min(max_hw_sectors, lim->io_opt >> SECTOR_SHIFT);
+	} else if (lim->io_min &&
+		   lim->io_min > (BLK_DEF_MAX_SECTORS_CAP << SECTOR_SHIFT)) {
+		lim->max_sectors =
+			min(max_hw_sectors, lim->io_min >> SECTOR_SHIFT);
 	} else {
 		lim->max_sectors = min(max_hw_sectors, BLK_DEF_MAX_SECTORS_CAP);
 	}
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 22a79a62cc4eab..ad887d614d5b3f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1808,7 +1808,7 @@ static struct nbd_device *nbd_dev_add(int index, unsigned int refs)
 {
 	struct queue_limits lim = {
 		.max_hw_sectors		= 65536,
-		.max_user_sectors	= 256,
+		.io_opt			= 256 << SECTOR_SHIFT,
 		.max_segments		= USHRT_MAX,
 		.max_segment_size	= UINT_MAX,
 	};
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 46dc487ccc17eb..22ad704f81d8b9 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -4954,7 +4954,6 @@ static int rbd_init_disk(struct rbd_device *rbd_dev)
 	    rbd_dev->layout.object_size * rbd_dev->layout.stripe_count;
 	struct queue_limits lim = {
 		.max_hw_sectors		= objset_bytes >> SECTOR_SHIFT,
-		.max_user_sectors	= objset_bytes >> SECTOR_SHIFT,
 		.io_opt			= objset_bytes,
 		.io_min			= rbd_dev->opts->alloc_size,
 		.max_segments		= USHRT_MAX,
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index f6c822c9cbd2d3..3dff9150ce11e2 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -3593,7 +3593,7 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	struct request_queue *q = sdkp->disk->queue;
 	sector_t old_capacity = sdkp->capacity;
 	unsigned char *buffer;
-	unsigned int dev_max, rw_max;
+	unsigned int dev_max;
 
 	SCSI_LOG_HLQUEUE(3, sd_printk(KERN_INFO, sdkp,
 				      "sd_revalidate_disk\n"));
@@ -3675,34 +3675,15 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	else
 		blk_queue_io_min(sdkp->disk->queue, 0);
 
-	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
-		q->limits.io_opt = logical_to_bytes(sdp, sdkp->opt_xfer_blocks);
-		rw_max = logical_to_sectors(sdp, sdkp->opt_xfer_blocks);
-	} else {
-		q->limits.io_opt = 0;
-		rw_max = min_not_zero(logical_to_sectors(sdp, dev_max),
-				      (sector_t)BLK_DEF_MAX_SECTORS_CAP);
-	}
-
 	/*
 	 * Limit default to SCSI host optimal sector limit if set. There may be
 	 * an impact on performance for when the size of a request exceeds this
 	 * host limit.
 	 */
-	rw_max = min_not_zero(rw_max, sdp->host->opt_sectors);
-
-	/* Do not exceed controller limit */
-	rw_max = min(rw_max, queue_max_hw_sectors(q));
-
-	/*
-	 * Only update max_sectors if previously unset or if the current value
-	 * exceeds the capabilities of the hardware.
-	 */
-	if (sdkp->first_scan ||
-	    q->limits.max_sectors > q->limits.max_dev_sectors ||
-	    q->limits.max_sectors > q->limits.max_hw_sectors) {
-		q->limits.max_sectors = rw_max;
-		q->limits.max_user_sectors = rw_max;
+	q->limits.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
+	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
+		q->limits.io_opt = min_not_zero(q->limits.io_opt,
+				logical_to_bytes(sdp, sdkp->opt_xfer_blocks));
 	}
 
 	sdkp->first_scan = 0;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733344.1139548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0Y-0005Vd-7R; Fri, 31 May 2024 07:49:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733344.1139548; Fri, 31 May 2024 07:49:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0Y-0005VW-4o; Fri, 31 May 2024 07:49:02 +0000
Received: by outflank-mailman (input) for mailman id 733344;
 Fri, 31 May 2024 07:49:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0X-0004oC-9h
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:01 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d3a3e68-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:48:59 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0N-00000009XXW-22Ka; Fri, 31 May 2024 07:48:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d3a3e68-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=0q2p+u53R9YY1ujQIkkIYHdGljvaoYfmRjVYZa1pGUw=; b=NQeQhA2UOUYImH9TH8f84PjtdO
	ObERM3qoC8ul3CKlxRBe0dwRdXaSSfDKIuJPDJKIXddbCk4DuIEOabi0xeClPM4iSB5KZs7KRVBke
	QXuG41cYK3YbO5bkABYuBrfzr2b20SayGqug5sRun1euqWYw8yIMYgwVe7df1qO/MfVZyOgF1Y4jH
	8S8LuTfmFWj0aSfI2cvZCNBbal2QtT+CQ6/gS37bIRRARF26B6AGUCKpsAA2d+30Zq8U6sLyU8dBv
	Bk3tXLY8MqgLz5OO3tA8AK0iTNwDc6TpStdHEqtPrSKHpQTid5SoU4/Ug1OFPywaZgXzR2J5gCky/
	tKVHxQ0Q==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: [PATCH 03/14] rbd: increase io_opt again
Date: Fri, 31 May 2024 09:47:58 +0200
Message-ID: <20240531074837.1648501-4-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Commit 16d80c54ad42 ("rbd: set io_min, io_opt and discard_granularity to
alloc_size") lowered the io_opt size for rbd from objset_bytes which is
4MB for typical setup to alloc_size which is typically 64KB.

The commit mostly talks about discard behavior and does mention io_min
in passing.  Reducing io_opt means reducing the readahead size, which
seems counter-intuitive given that rbd currently abuses the user
max_sectors setting to actually increase the I/O size.  Switch back
to the old setting to allow larger reads (the readahead size despite it's
name actually limits the size of any buffered read) and to prepare
for using io_opt in the max_sectors calculation and getting drivers out
of the business of overriding the max_user_sectors value.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/block/rbd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 26ff5cd2bf0abc..46dc487ccc17eb 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -4955,8 +4955,8 @@ static int rbd_init_disk(struct rbd_device *rbd_dev)
 	struct queue_limits lim = {
 		.max_hw_sectors		= objset_bytes >> SECTOR_SHIFT,
 		.max_user_sectors	= objset_bytes >> SECTOR_SHIFT,
+		.io_opt			= objset_bytes,
 		.io_min			= rbd_dev->opts->alloc_size,
-		.io_opt			= rbd_dev->opts->alloc_size,
 		.max_segments		= USHRT_MAX,
 		.max_segment_size	= UINT_MAX,
 	};
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733341.1139519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0V-0004oS-Ew; Fri, 31 May 2024 07:48:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733341.1139519; Fri, 31 May 2024 07:48:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0V-0004oL-B7; Fri, 31 May 2024 07:48:59 +0000
Received: by outflank-mailman (input) for mailman id 733341;
 Fri, 31 May 2024 07:48:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0T-0004o6-5E
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:48:58 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3afb0e0c-1f22-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:48:56 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0C-00000009XUr-41fW; Fri, 31 May 2024 07:48:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3afb0e0c-1f22-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:
	Content-ID:Content-Description:In-Reply-To:References;
	bh=X6ZSIyh1cBG1UFDOkvLStzuDyEP+AFeCGGHYkP7yPXE=; b=wh98NvxtuA1xZGSKFRioyMm51I
	vrfovyzZnajBqXQQrJzm7vrjL319N5CmN22SqQEH5AWCG3vSYC2Ehtl5dcKGB5JTVnjGnZ5BKPqLM
	k/8mUprcZcrR0sLsTDn+NROurZ//Aujt8btpgMENtK0/To9YThW/cFbdJDIRSieOZxuCL8t6W/I55
	BtAv6L1mDU06KnafTVhy4T18Em8h0Ba2Y70K4JL0T5UE87wHrD0U6qXXRuxbTCgayrCNZ2qZ6PffF
	YrCVgO4qoHmBcSzLnvh4ZzHMZzUuHyvAU4HVYk4J4d95GDSA618ThURvQ6uULdXsoDERjoAsBWfP6
	qbtPlb0Q==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org
Subject: convert the SCSI ULDs to the atomic queue limits API v2
Date: Fri, 31 May 2024 09:47:55 +0200
Message-ID: <20240531074837.1648501-1-hch@lst.de>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Hi all,

this series converts the SCSI upper level drivers to the atomic queue
limits API.

The first patch is a bug fix for ubd that later patches depend on and
might be worth picking up for 6.10.

The second patch changes the max_sectors calculation to take the optimal
I/O size into account so that sd, nbd and rbd don't have to mess with
the user max_sector value.  I'd love to see a careful review from the
nbd and rbd maintainers for this one!

The following patches clean up a few lose ends in the sd driver, and
then convert sd and sr to the atomic queue limits API.  The final
patches remove the now unused block APIs, and convert a few to be
specific to their now more narrow use case.

The patches are against Jens' block-6.10 tree.  Due to the amount of
block layer changes in here, and other that will depend on it, it
would be good if this could eventually be merged through the block
tree, or at least a shared branch between the SCSI and block trees.

Changes since v1:
 - change the io_opt value for rbd
 - fix a left-over direct assignent to q->limits
 - add a new patch to refactor the ubd interrupt handler
 - use an else if to micro-optimize the ubd error handling
 - also remove disk_set_max_open_zones and disk_set_max_active_zones
 - use SECTOR_SHIFT in one more place
 - various spelling fixes
 - comment formating fix

Diffstat:
 arch/um/drivers/ubd_kern.c   |   50 +++------
 block/blk-settings.c         |  238 +------------------------------------------
 drivers/block/nbd.c          |    2 
 drivers/block/rbd.c          |    3 
 drivers/block/xen-blkfront.c |    4 
 drivers/scsi/sd.c            |  222 ++++++++++++++++++++--------------------
 drivers/scsi/sd.h            |    6 -
 drivers/scsi/sd_zbc.c        |   27 ++--
 drivers/scsi/sr.c            |   42 ++++---
 include/linux/blkdev.h       |   52 +++------
 10 files changed, 210 insertions(+), 436 deletions(-)


From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733346.1139569 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0a-00062a-P0; Fri, 31 May 2024 07:49:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733346.1139569; Fri, 31 May 2024 07:49:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0a-00062M-KK; Fri, 31 May 2024 07:49:04 +0000
Received: by outflank-mailman (input) for mailman id 733346;
 Fri, 31 May 2024 07:49:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0Z-0004o6-DZ
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:03 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f190ec8-1f22-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:49:03 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0S-00000009XbL-2AyO; Fri, 31 May 2024 07:48:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f190ec8-1f22-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=FAwSfn6ckfnQfuPEUHGpbbgoT8gFgpbbTIk4PRDlm6U=; b=fbXoGRqkGTVOs0QpSZx1za9SJZ
	wqVDSjCy1no2SJW1DLELgUUweO0zbCp7XqisGNgydQGJuEz4rVDUKXUgRM5AmJuAPPjYJlYUvwtkh
	TWFfhZlnAQj3Rw/BSfNpRA6U1AHETgyqHOaeb5qOgQAV5sGO0cu1Qce5CKmhis721gFK7f6ewt+Hl
	mzjO2pXvW3XVfYoA51SiP+wqtSR6lDKtbcJiyvM1VF7YK5J/zJk0XCfyiKO92WAR3AjBFDLtodw43
	uZ6lQl/D07TmW80lgSIq98oUc4YJVGrlTrsAfUYYrzsd9K7Sfua52KNVwv31wAcbpsDwQ18f37u4H
	QmsvUDdw==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 05/14] sd: simplify the ZBC case in provisioning_mode_store
Date: Fri, 31 May 2024 09:48:00 +0200
Message-ID: <20240531074837.1648501-6-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Don't reset the discard settings to no-op over and over when a user
writes to the provisioning attribute as that is already the default
mode for ZBC devices.  In hindsight we should have made writing to
the attribute fail for ZBC devices, but the code has probably been
around for far too long to change this now.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 3dff9150ce11e2..83aa17fea39d39 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -461,14 +461,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 	if (!capable(CAP_SYS_ADMIN))
 		return -EACCES;
 
-	if (sd_is_zoned(sdkp)) {
-		sd_config_discard(sdkp, SD_LBP_DISABLE);
-		return count;
-	}
-
 	if (sdp->type != TYPE_DISK)
 		return -EINVAL;
 
+	/* ignore the provisioning mode for ZBC devices */
+	if (sd_is_zoned(sdkp))
+		return count;
+
 	mode = sysfs_match_string(lbp_mode, buf);
 	if (mode < 0)
 		return -EINVAL;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733347.1139579 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0f-0006PE-0N; Fri, 31 May 2024 07:49:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733347.1139579; Fri, 31 May 2024 07:49:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0e-0006On-St; Fri, 31 May 2024 07:49:08 +0000
Received: by outflank-mailman (input) for mailman id 733347;
 Fri, 31 May 2024 07:49:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0d-0004oC-Jg
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:07 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 41033a96-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:06 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0V-00000009Xcv-0vGx; Fri, 31 May 2024 07:48:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41033a96-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=G254IYRsUd/F6t4PlJahixfTtyWkS+KjkuNozPFW2uM=; b=mCfAB2Q3IG7ARRB/1pICP3EKhx
	BtsvlSjEu+mUGJHdLHTvoJOmfRKjXxZbNxLy4KOfOIoTiP37YS600VEXpgV/uP9orX00qlWer8q/4
	MqGvCypDXaIwvVXmRrJ1K8o1tyNVFri/wRN+r8xhPtEgtnttXcI2/uR1po1i3iqcBuzQdmttvNtJF
	oChxsdFiMIHdgU8Fvaju7qPUGHzeaRTfRR1LK2zbdzzOelGNuR2nbzEhvJ15Xap/+vOuczm4IhNaA
	4KPN2ftKI7ksG9v6J9jDdoyHPHRrn7UXV85lXH0GCoodOce35RcaAaP3IEmY28xn6qGm+n9XQh5zR
	gLnZ0o+w==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 06/14] sd: add a sd_disable_discard helper
Date: Fri, 31 May 2024 09:48:01 +0200
Message-ID: <20240531074837.1648501-7-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Add helper to disable discard when it is not supported and use it
instead of sd_config_discard in the I/O completion handler.  This avoids
touching more fields than required in the I/O completion handler and
prepares for converting sd to use the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 83aa17fea39d39..f07d90474e682b 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -821,6 +821,12 @@ static unsigned char sd_setup_protect_cmnd(struct scsi_cmnd *scmd,
 	return protect;
 }
 
+static void sd_disable_discard(struct scsi_disk *sdkp)
+{
+	sdkp->provisioning_mode = SD_LBP_DISABLE;
+	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
+}
+
 static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 {
 	struct request_queue *q = sdkp->disk->queue;
@@ -2245,12 +2251,12 @@ static int sd_done(struct scsi_cmnd *SCpnt)
 		case 0x24:	/* INVALID FIELD IN CDB */
 			switch (SCpnt->cmnd[0]) {
 			case UNMAP:
-				sd_config_discard(sdkp, SD_LBP_DISABLE);
+				sd_disable_discard(sdkp);
 				break;
 			case WRITE_SAME_16:
 			case WRITE_SAME:
 				if (SCpnt->cmnd[1] & 8) { /* UNMAP */
-					sd_config_discard(sdkp, SD_LBP_DISABLE);
+					sd_disable_discard(sdkp);
 				} else {
 					sdkp->device->no_write_same = 1;
 					sd_config_write_same(sdkp);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733348.1139589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0h-0006jJ-98; Fri, 31 May 2024 07:49:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733348.1139589; Fri, 31 May 2024 07:49:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0h-0006j6-5K; Fri, 31 May 2024 07:49:11 +0000
Received: by outflank-mailman (input) for mailman id 733348;
 Fri, 31 May 2024 07:49:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0g-0004oC-Lb
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:10 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 42ce95de-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:09 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0X-00000009XeI-2flr; Fri, 31 May 2024 07:49:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42ce95de-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=NIYU24POaVEtCWxh2Uz76sasJLK5fExY/uxdAFrpLn0=; b=nvWGxGR5UCqtJLt0MzfGEBXnRL
	vxsmtMLHS1hp0z0eBbJsI51GWBRhrA2+Lck1DFcYdERBAx3KcneUDzBnOglm5G+IfC+X2MADP07Jj
	saCga2YJOejXuJjmKaPvmIpDS+wNLQHuYkF228yXr/q9S5QKvtOfgkc6aqVAvRZNiuGM8/6ioOixR
	fa78mHFjrLtwDVtZ2ckhco4l97C5UC7UdJtwC8yE9zPhXtzLPRBv0UkgWdg/HF9Z0W/SxVp/BhuNx
	IZUiOmTk9V8IWo2G70I7pqb0wHbZ9WCexXrsHNoo4DmXAZa+i22XFuQMxwVHwGHCSRlu4soLwwS6O
	e4SUs9tw==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 07/14] sd: add a sd_disable_write_same helper
Date: Fri, 31 May 2024 09:48:02 +0200
Message-ID: <20240531074837.1648501-8-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Add helper to disable WRITE SAME when it is not supported and use it
instead of sd_config_write_same in the I/O completion handler.  This
avoids touching more fields than required in the I/O completion handler
and  prepares for converting sd to use the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index f07d90474e682b..70211d0b187652 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -1004,6 +1004,13 @@ static blk_status_t sd_setup_write_zeroes_cmnd(struct scsi_cmnd *cmd)
 	return sd_setup_write_same10_cmnd(cmd, false);
 }
 
+static void sd_disable_write_same(struct scsi_disk *sdkp)
+{
+	sdkp->device->no_write_same = 1;
+	sdkp->max_ws_blocks = 0;
+	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
+}
+
 static void sd_config_write_same(struct scsi_disk *sdkp)
 {
 	struct request_queue *q = sdkp->disk->queue;
@@ -2258,8 +2265,7 @@ static int sd_done(struct scsi_cmnd *SCpnt)
 				if (SCpnt->cmnd[1] & 8) { /* UNMAP */
 					sd_disable_discard(sdkp);
 				} else {
-					sdkp->device->no_write_same = 1;
-					sd_config_write_same(sdkp);
+					sd_disable_write_same(sdkp);
 					req->rq_flags |= RQF_QUIET;
 				}
 				break;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733351.1139599 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0k-0007D0-Qj; Fri, 31 May 2024 07:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733351.1139599; Fri, 31 May 2024 07:49:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0k-0007Cl-Mo; Fri, 31 May 2024 07:49:14 +0000
Received: by outflank-mailman (input) for mailman id 733351;
 Fri, 31 May 2024 07:49:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0j-0004oC-72
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:13 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4460cb50-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:11 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0a-00000009Xfo-0bb2; Fri, 31 May 2024 07:49:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4460cb50-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=HN2Hktsuqrp8T8GVou6AHcXHJfFLm1CDuoHLTArZtbQ=; b=lzSvXRRwzld/pMYWiFEKRDRG2R
	S8H/9KAsLWG+7rCB0xOZNV350iYhUSHRfAKBFICCqHErh6IeHq2cVZLjPLGTNwt0sKNOjOyQMdSDV
	AEJE2W/7wU/JPARqknHB/teDENsrfXzvflDt8grlnj+PbL6uFlGnmyIPKa9bJ37oUrbUpR+ZsEMve
	N/VNYOyU0GCIc2LMMuiQtuHRwpXnbqjHrYK53jvCzl6pWVmFQXJCKXQHsMvt3zWuBGJhUyOSUlMbY
	CkQ2uUkan33bspD1IDNqzU7wQgTUkNAjBNx4O8h/zxKzZMwt7T8O1npKByu3eerVKPrQQXprz/j9N
	t+/ZgFRw==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 08/14] sd: simplify the disable case in sd_config_discard
Date: Fri, 31 May 2024 09:48:03 +0200
Message-ID: <20240531074837.1648501-9-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Fall through to the main call to blk_queue_max_discard_sectors given that
max_blocks has been initialized to zero above instead of duplicating the
call.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 70211d0b187652..0dbc6eb7a7cac3 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -844,8 +844,7 @@ static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 
 	case SD_LBP_FULL:
 	case SD_LBP_DISABLE:
-		blk_queue_max_discard_sectors(q, 0);
-		return;
+		break;
 
 	case SD_LBP_UNMAP:
 		max_blocks = min_not_zero(sdkp->max_unmap_blocks,
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733354.1139609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0o-0007dz-45; Fri, 31 May 2024 07:49:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733354.1139609; Fri, 31 May 2024 07:49:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0n-0007dr-W4; Fri, 31 May 2024 07:49:17 +0000
Received: by outflank-mailman (input) for mailman id 733354;
 Fri, 31 May 2024 07:49:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0m-0004oC-4X
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:16 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4614d04f-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:14 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0c-00000009Xgz-3ZvO; Fri, 31 May 2024 07:49:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4614d04f-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=rb1OF73CoqNaiVO8CqTHVc8xH4T7Xk6a2c9CxacCVxo=; b=ixc7sU02vm0CYaeNZLVqmJqr1F
	w3t3CHukX6Y/WrwEBPmir631IGBTKvdXkr3Gx7sPqVqokxqFFVlywpqEW1Ike10ParnnBt1UYNr5g
	oxfWZdjBwiqVCxtSkH5LykBrDs6zXobZzMgztC7hMbwDnMSYlJqmNgh8Q9kC05ZfeL1SBTCNAFVWE
	+8sLD6+qG9QyI/eL8OKGy0oMy9FUSqpJQy2NiOWqYRZmyBUvVUNO6Lvocvevg3GFFr1HQvo+FKyWO
	zjfg3D4GZ3IuWhl7HByairPmkUxGxODsK90DtnOBsC9OwFCM9RkfiY/ajKZ3KzidolhxQDvM0BqsD
	2Z40FzGQ==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 09/14] sd: factor out a sd_discard_mode helper
Date: Fri, 31 May 2024 09:48:04 +0200
Message-ID: <20240531074837.1648501-10-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Split the logic to pick the right discard mode into a little helper
to prepare for further changes.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 0dbc6eb7a7cac3..39eddfac09ef8f 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -3201,6 +3201,25 @@ static void sd_read_app_tag_own(struct scsi_disk *sdkp, unsigned char *buffer)
 	return;
 }
 
+static unsigned int sd_discard_mode(struct scsi_disk *sdkp)
+{
+	if (!sdkp->lbpvpd) {
+		/* LBP VPD page not provided */
+		if (sdkp->max_unmap_blocks)
+			return SD_LBP_UNMAP;
+		return SD_LBP_WS16;
+	}
+
+	/* LBP VPD page tells us what to use */
+	if (sdkp->lbpu && sdkp->max_unmap_blocks)
+		return SD_LBP_UNMAP;
+	if (sdkp->lbpws)
+		return SD_LBP_WS16;
+	if (sdkp->lbpws10)
+		return SD_LBP_WS10;
+	return SD_LBP_DISABLE;
+}
+
 /**
  * sd_read_block_limits - Query disk device for preferred I/O sizes.
  * @sdkp: disk to query
@@ -3239,23 +3258,7 @@ static void sd_read_block_limits(struct scsi_disk *sdkp)
 			sdkp->unmap_alignment =
 				get_unaligned_be32(&vpd->data[32]) & ~(1 << 31);
 
-		if (!sdkp->lbpvpd) { /* LBP VPD page not provided */
-
-			if (sdkp->max_unmap_blocks)
-				sd_config_discard(sdkp, SD_LBP_UNMAP);
-			else
-				sd_config_discard(sdkp, SD_LBP_WS16);
-
-		} else {	/* LBP VPD page tells us what to use */
-			if (sdkp->lbpu && sdkp->max_unmap_blocks)
-				sd_config_discard(sdkp, SD_LBP_UNMAP);
-			else if (sdkp->lbpws)
-				sd_config_discard(sdkp, SD_LBP_WS16);
-			else if (sdkp->lbpws10)
-				sd_config_discard(sdkp, SD_LBP_WS10);
-			else
-				sd_config_discard(sdkp, SD_LBP_DISABLE);
-		}
+		sd_config_discard(sdkp, sd_discard_mode(sdkp));
 	}
 
  out:
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733358.1139619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0p-0007xy-FN; Fri, 31 May 2024 07:49:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733358.1139619; Fri, 31 May 2024 07:49:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0p-0007xa-AC; Fri, 31 May 2024 07:49:19 +0000
Received: by outflank-mailman (input) for mailman id 733358;
 Fri, 31 May 2024 07:49:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0o-0004oC-Ld
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:18 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 479c1633-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:17 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0g-00000009Xih-13mT; Fri, 31 May 2024 07:49:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 479c1633-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=wVVN+ho4qP+ZR7D7NuzXuwmQjlkhywzU3nSc/hJClAc=; b=QZrkdfl3DDyuwh2a7Q4n0400qr
	04KjE55KXAnyTNn+jHpVvd09LViptfhuL2m6pO+SqoYzByvZ1uF6g3hbEvHVQuscZYGiv9qH9dKXI
	7F+/G+I9owuYVx+yMsBAYl6nRr4jTXOPQ+EOU6xXNTsTgwkw0BnnxmoX271AreMLl9v4vf9/OjuKt
	bLMly1lAU6mCENiG10UTLoTmXL4lVxHMQ/KPsUQQQFv71494Ap5HBXNBWltj1sb2kw9kQscDv6dTO
	Le0XTwBOV6AoUM3S00jTLpkzkDVSA1w3wBDuxfSX71QSRVMI2L4W9NtLQARIGXjDUkr0iRaK1i/Y0
	SDmnPCyg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 10/14] sd: cleanup zoned queue limits initialization
Date: Fri, 31 May 2024 09:48:05 +0200
Message-ID: <20240531074837.1648501-11-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Consolidate setting zone-related queue limits in sd_zbc_read_zones
instead of splitting them between sd_zbc_revalidate_zones and
sd_zbc_read_zones, and move the early_zone_information initialization
in sd_zbc_read_zones above setting up the queue limits.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd_zbc.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/sd_zbc.c b/drivers/scsi/sd_zbc.c
index 806036e48abeda..1c24c844e8d178 100644
--- a/drivers/scsi/sd_zbc.c
+++ b/drivers/scsi/sd_zbc.c
@@ -565,12 +565,6 @@ int sd_zbc_revalidate_zones(struct scsi_disk *sdkp)
 	sdkp->zone_info.zone_blocks = zone_blocks;
 	sdkp->zone_info.nr_zones = nr_zones;
 
-	blk_queue_chunk_sectors(q,
-			logical_to_sectors(sdkp->device, zone_blocks));
-
-	/* Enable block layer zone append emulation */
-	blk_queue_max_zone_append_sectors(q, 0);
-
 	flags = memalloc_noio_save();
 	ret = blk_revalidate_disk_zones(disk);
 	memalloc_noio_restore(flags);
@@ -625,6 +619,10 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	if (ret != 0)
 		goto err;
 
+	nr_zones = round_up(sdkp->capacity, zone_blocks) >> ilog2(zone_blocks);
+	sdkp->early_zone_info.nr_zones = nr_zones;
+	sdkp->early_zone_info.zone_blocks = zone_blocks;
+
 	/* The drive satisfies the kernel restrictions: set it up */
 	blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q);
 	if (sdkp->zones_max_open == U32_MAX)
@@ -632,10 +630,10 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	else
 		disk_set_max_open_zones(disk, sdkp->zones_max_open);
 	disk_set_max_active_zones(disk, 0);
-	nr_zones = round_up(sdkp->capacity, zone_blocks) >> ilog2(zone_blocks);
-
-	sdkp->early_zone_info.nr_zones = nr_zones;
-	sdkp->early_zone_info.zone_blocks = zone_blocks;
+	blk_queue_chunk_sectors(q,
+			logical_to_sectors(sdkp->device, zone_blocks));
+	/* Enable block layer zone append emulation */
+	blk_queue_max_zone_append_sectors(q, 0);
 
 	return 0;
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:49:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:49:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733367.1139629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0z-0000iJ-Pp; Fri, 31 May 2024 07:49:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733367.1139629; Fri, 31 May 2024 07:49:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx0z-0000i8-M7; Fri, 31 May 2024 07:49:29 +0000
Received: by outflank-mailman (input) for mailman id 733367;
 Fri, 31 May 2024 07:49:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0y-0004o6-CH
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:28 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4ddf27ff-1f22-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 09:49:27 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0q-00000009XpF-495X; Fri, 31 May 2024 07:49:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ddf27ff-1f22-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=puwKRK0RcHItIRFrYIY/fDmsBtElHLDF9wXWafbLcHw=; b=X0VqmD7VHuCNDs94Elv2YloGby
	p5+jzBSrlgnlZnALN2myqc19yIqGUbxATwdTIB+h+Oym2URU0toacKR0d4Ppc/0DS03DVNLrJR/JN
	C0oEYlZlZaXdI7CfyuNl+mRUzoVf6pmjadJVEWgRdvSHT99utfWmtBwMcKGiPPrFiDu5UYBU0AJTT
	bf8QszdcpR2GXbNackCU5hnE6pc8UUMxryRLdq12mSDIJJq0EfwCw9eywAFGNliDSf5mo+yP4eS4k
	EUChub1u9DFsYyI4C013eed0SbY3fXmcR7jEUJDVEc5q22Vd0+RSQtqNiHgElauAc4ssidf+0okIP
	pkpsRF1w==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 14/14] block: add special APIs for run-time disabling of discard and friends
Date: Fri, 31 May 2024 09:48:09 +0200
Message-ID: <20240531074837.1648501-15-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

A few drivers optimistically try to support discard, write zeroes and
secure erase and disable the features from the I/O completion handler
if the hardware can't support them.  This disable can't be done using
the atomic queue limits API because the I/O completion handlers can't
take sleeping locks or freeze the queue.  Keep the existing clearing
of the relevant field to zero, but replace the old blk_queue_max_*
APIs with new disable APIs that force the value to 0.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 arch/um/drivers/ubd_kern.c   |  4 ++--
 block/blk-settings.c         | 41 ------------------------------------
 drivers/block/xen-blkfront.c |  4 ++--
 drivers/scsi/sd.c            |  4 ++--
 include/linux/blkdev.h       | 28 ++++++++++++++++++------
 5 files changed, 28 insertions(+), 53 deletions(-)

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 093c87879d08ba..cdcb75a68989dd 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -451,9 +451,9 @@ static void ubd_end_request(struct io_thread_req *io_req)
 {
 	if (io_req->error == BLK_STS_NOTSUPP) {
 		if (req_op(io_req->req) == REQ_OP_DISCARD)
-			blk_queue_max_discard_sectors(io_req->req->q, 0);
+			blk_queue_disable_discard(io_req->req->q);
 		else if (req_op(io_req->req) == REQ_OP_WRITE_ZEROES)
-			blk_queue_max_write_zeroes_sectors(io_req->req->q, 0);
+			blk_queue_disable_write_zeroes(io_req->req->q);
 	}
 	blk_mq_end_request(io_req->req, io_req->error);
 	kfree(io_req);
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 0b038729608f4b..996f247fc98e80 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -293,47 +293,6 @@ int queue_limits_set(struct request_queue *q, struct queue_limits *lim)
 }
 EXPORT_SYMBOL_GPL(queue_limits_set);
 
-/**
- * blk_queue_max_discard_sectors - set max sectors for a single discard
- * @q:  the request queue for the device
- * @max_discard_sectors: maximum number of sectors to discard
- **/
-void blk_queue_max_discard_sectors(struct request_queue *q,
-		unsigned int max_discard_sectors)
-{
-	struct queue_limits *lim = &q->limits;
-
-	lim->max_hw_discard_sectors = max_discard_sectors;
-	lim->max_discard_sectors =
-		min(max_discard_sectors, lim->max_user_discard_sectors);
-}
-EXPORT_SYMBOL(blk_queue_max_discard_sectors);
-
-/**
- * blk_queue_max_secure_erase_sectors - set max sectors for a secure erase
- * @q:  the request queue for the device
- * @max_sectors: maximum number of sectors to secure_erase
- **/
-void blk_queue_max_secure_erase_sectors(struct request_queue *q,
-		unsigned int max_sectors)
-{
-	q->limits.max_secure_erase_sectors = max_sectors;
-}
-EXPORT_SYMBOL(blk_queue_max_secure_erase_sectors);
-
-/**
- * blk_queue_max_write_zeroes_sectors - set max sectors for a single
- *                                      write zeroes
- * @q:  the request queue for the device
- * @max_write_zeroes_sectors: maximum number of sectors to write per command
- **/
-void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
-		unsigned int max_write_zeroes_sectors)
-{
-	q->limits.max_write_zeroes_sectors = max_write_zeroes_sectors;
-}
-EXPORT_SYMBOL(blk_queue_max_write_zeroes_sectors);
-
 void disk_update_readahead(struct gendisk *disk)
 {
 	blk_apply_bdi_limits(disk->bdi, &disk->queue->limits);
diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index fd7c0ff2139cee..9b4ec3e4908cce 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -1605,8 +1605,8 @@ static irqreturn_t blkif_interrupt(int irq, void *dev_id)
 				blkif_req(req)->error = BLK_STS_NOTSUPP;
 				info->feature_discard = 0;
 				info->feature_secdiscard = 0;
-				blk_queue_max_discard_sectors(rq, 0);
-				blk_queue_max_secure_erase_sectors(rq, 0);
+				blk_queue_disable_discard(rq);
+				blk_queue_disable_secure_erase(rq);
 			}
 			break;
 		case BLKIF_OP_FLUSH_DISKCACHE:
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 049071b5681989..d957e29b17a98a 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -837,7 +837,7 @@ static unsigned char sd_setup_protect_cmnd(struct scsi_cmnd *scmd,
 static void sd_disable_discard(struct scsi_disk *sdkp)
 {
 	sdkp->provisioning_mode = SD_LBP_DISABLE;
-	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
+	blk_queue_disable_discard(sdkp->disk->queue);
 }
 
 static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
@@ -1019,7 +1019,7 @@ static void sd_disable_write_same(struct scsi_disk *sdkp)
 {
 	sdkp->device->no_write_same = 1;
 	sdkp->max_ws_blocks = 0;
-	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
+	blk_queue_disable_write_zeroes(sdkp->disk->queue);
 }
 
 static void sd_config_write_same(struct scsi_disk *sdkp,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index ad995e7a769811..ac8e0cb2353a0e 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -912,15 +912,31 @@ static inline void queue_limits_cancel_update(struct request_queue *q)
 	mutex_unlock(&q->limits_lock);
 }
 
+/*
+ * These helpers are for drivers that have sloppy feature negotiation and might
+ * have to disable DISCARD, WRITE_ZEROES or SECURE_DISCARD from the I/O
+ * completion handler when the device returned an indicator that the respective
+ * feature is not actually supported.  They are racy and the driver needs to
+ * cope with that.  Try to avoid this scheme if you can.
+ */
+static inline void blk_queue_disable_discard(struct request_queue *q)
+{
+	q->limits.max_discard_sectors = 0;
+}
+
+static inline void blk_queue_disable_secure_erase(struct request_queue *q)
+{
+	q->limits.max_secure_erase_sectors = 0;
+}
+
+static inline void blk_queue_disable_write_zeroes(struct request_queue *q)
+{
+	q->limits.max_write_zeroes_sectors = 0;
+}
+
 /*
  * Access functions for manipulating queue properties
  */
-void blk_queue_max_secure_erase_sectors(struct request_queue *q,
-		unsigned int max_sectors);
-extern void blk_queue_max_discard_sectors(struct request_queue *q,
-		unsigned int max_discard_sectors);
-extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
-		unsigned int max_write_same_sectors);
 void disk_update_readahead(struct gendisk *disk);
 extern void blk_limits_io_min(struct queue_limits *limits, unsigned int min);
 extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 07:50:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 07:50:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733373.1139639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx28-0003o5-2G; Fri, 31 May 2024 07:50:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733373.1139639; Fri, 31 May 2024 07:50:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCx27-0003ny-Va; Fri, 31 May 2024 07:50:39 +0000
Received: by outflank-mailman (input) for mailman id 733373;
 Fri, 31 May 2024 07:50:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0s-0004oC-Ks
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:22 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 49859518-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:20 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0j-00000009Xkp-05jS; Fri, 31 May 2024 07:49:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49859518-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=4/nz/WYb/Zb/tIFnKGf5HamatZXHIwD043qHeq+0qpw=; b=AYRwtnDJjcn1Q7UeYo5d6GeEQx
	LaFV50ZR3TRtTUPo8wntoO06zmq4gAGnJ4ANVns7K10iL11t+zsZvnxbVY/HHhcvd2HdDnEWt6M3I
	KS5qNdot9Yv78noRBAaqTrX/ncfwca4zubufWhJKQGUPXFvenQRFZ0KXmeDfJL+VjW82wIqW8tud1
	HjM+o0kSZkpMgbeCnqFY1QrLbfyBjmPJZpNx5f8ORMO5QrdP+A9P4pN4Mb8wwDb/UPnDb7PsSBtdi
	81r6DYNBSIm0KtezndCLhQPbtYZeuKW2SoxK392+JhzCXfVQZZs0XHHxuwx2JgLFimAQ2/ZkXNkOR
	n83SSfyA==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 11/14] sd: convert to the atomic queue limits API
Date: Fri, 31 May 2024 09:48:06 +0200
Message-ID: <20240531074837.1648501-12-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Assign all queue limits through a local queue_limits variable and
queue_limits_commit_update so that we can't race updating them from
multiple places, and freeze the queue when updating them so that
in-progress I/O submissions don't see half-updated limits.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sd.c     | 130 ++++++++++++++++++++++++------------------
 drivers/scsi/sd.h     |   6 +-
 drivers/scsi/sd_zbc.c |  15 ++---
 3 files changed, 85 insertions(+), 66 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 39eddfac09ef8f..049071b5681989 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -101,12 +101,13 @@ MODULE_ALIAS_SCSI_DEVICE(TYPE_ZBC);
 
 #define SD_MINORS	16
 
-static void sd_config_discard(struct scsi_disk *, unsigned int);
-static void sd_config_write_same(struct scsi_disk *);
+static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned int mode);
+static void sd_config_write_same(struct scsi_disk *sdkp,
+		struct queue_limits *lim);
 static int  sd_revalidate_disk(struct gendisk *);
 static void sd_unlock_native_capacity(struct gendisk *disk);
 static void sd_shutdown(struct device *);
-static void sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer);
 static void scsi_disk_release(struct device *cdev);
 
 static DEFINE_IDA(sd_index_ida);
@@ -456,7 +457,8 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 {
 	struct scsi_disk *sdkp = to_scsi_disk(dev);
 	struct scsi_device *sdp = sdkp->device;
-	int mode;
+	struct queue_limits lim;
+	int mode, err;
 
 	if (!capable(CAP_SYS_ADMIN))
 		return -EACCES;
@@ -472,8 +474,13 @@ provisioning_mode_store(struct device *dev, struct device_attribute *attr,
 	if (mode < 0)
 		return -EINVAL;
 
-	sd_config_discard(sdkp, mode);
-
+	lim = queue_limits_start_update(sdkp->disk->queue);
+	sd_config_discard(sdkp, &lim, mode);
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
 	return count;
 }
 static DEVICE_ATTR_RW(provisioning_mode);
@@ -556,6 +563,7 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
 {
 	struct scsi_disk *sdkp = to_scsi_disk(dev);
 	struct scsi_device *sdp = sdkp->device;
+	struct queue_limits lim;
 	unsigned long max;
 	int err;
 
@@ -577,8 +585,13 @@ max_write_same_blocks_store(struct device *dev, struct device_attribute *attr,
 		sdkp->max_ws_blocks = max;
 	}
 
-	sd_config_write_same(sdkp);
-
+	lim = queue_limits_start_update(sdkp->disk->queue);
+	sd_config_write_same(sdkp, &lim);
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
 	return count;
 }
 static DEVICE_ATTR_RW(max_write_same_blocks);
@@ -827,17 +840,15 @@ static void sd_disable_discard(struct scsi_disk *sdkp)
 	blk_queue_max_discard_sectors(sdkp->disk->queue, 0);
 }
 
-static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
+static void sd_config_discard(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned int mode)
 {
-	struct request_queue *q = sdkp->disk->queue;
 	unsigned int logical_block_size = sdkp->device->sector_size;
 	unsigned int max_blocks = 0;
 
-	q->limits.discard_alignment =
-		sdkp->unmap_alignment * logical_block_size;
-	q->limits.discard_granularity =
-		max(sdkp->physical_block_size,
-		    sdkp->unmap_granularity * logical_block_size);
+	lim->discard_alignment = sdkp->unmap_alignment * logical_block_size;
+	lim->discard_granularity = max(sdkp->physical_block_size,
+			sdkp->unmap_granularity * logical_block_size);
 	sdkp->provisioning_mode = mode;
 
 	switch (mode) {
@@ -875,7 +886,8 @@ static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
 		break;
 	}
 
-	blk_queue_max_discard_sectors(q, max_blocks * (logical_block_size >> 9));
+	lim->max_hw_discard_sectors = max_blocks *
+		(logical_block_size >> SECTOR_SHIFT);
 }
 
 static void *sd_set_special_bvec(struct request *rq, unsigned int data_len)
@@ -1010,9 +1022,9 @@ static void sd_disable_write_same(struct scsi_disk *sdkp)
 	blk_queue_max_write_zeroes_sectors(sdkp->disk->queue, 0);
 }
 
-static void sd_config_write_same(struct scsi_disk *sdkp)
+static void sd_config_write_same(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
-	struct request_queue *q = sdkp->disk->queue;
 	unsigned int logical_block_size = sdkp->device->sector_size;
 
 	if (sdkp->device->no_write_same) {
@@ -1066,8 +1078,8 @@ static void sd_config_write_same(struct scsi_disk *sdkp)
 	}
 
 out:
-	blk_queue_max_write_zeroes_sectors(q, sdkp->max_ws_blocks *
-					 (logical_block_size >> 9));
+	lim->max_write_zeroes_sectors =
+		sdkp->max_ws_blocks * (logical_block_size >> SECTOR_SHIFT);
 }
 
 static blk_status_t sd_setup_flush_cmnd(struct scsi_cmnd *cmd)
@@ -2523,7 +2535,7 @@ static void read_capacity_error(struct scsi_disk *sdkp, struct scsi_device *sdp,
 #define READ_CAPACITY_RETRIES_ON_RESET	10
 
 static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
-						unsigned char *buffer)
+		struct queue_limits *lim, unsigned char *buffer)
 {
 	unsigned char cmd[16];
 	struct scsi_sense_hdr sshdr;
@@ -2597,7 +2609,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
 
 	/* Lowest aligned logical block */
 	alignment = ((buffer[14] & 0x3f) << 8 | buffer[15]) * sector_size;
-	blk_queue_alignment_offset(sdp->request_queue, alignment);
+	lim->alignment_offset = alignment;
 	if (alignment && sdkp->first_scan)
 		sd_printk(KERN_NOTICE, sdkp,
 			  "physical block alignment offset: %u\n", alignment);
@@ -2608,7 +2620,7 @@ static int read_capacity_16(struct scsi_disk *sdkp, struct scsi_device *sdp,
 		if (buffer[14] & 0x40) /* LBPRZ */
 			sdkp->lbprz = 1;
 
-		sd_config_discard(sdkp, SD_LBP_WS16);
+		sd_config_discard(sdkp, lim, SD_LBP_WS16);
 	}
 
 	sdkp->capacity = lba + 1;
@@ -2711,13 +2723,14 @@ static int sd_try_rc16_first(struct scsi_device *sdp)
  * read disk capacity
  */
 static void
-sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
+sd_read_capacity(struct scsi_disk *sdkp, struct queue_limits *lim,
+		unsigned char *buffer)
 {
 	int sector_size;
 	struct scsi_device *sdp = sdkp->device;
 
 	if (sd_try_rc16_first(sdp)) {
-		sector_size = read_capacity_16(sdkp, sdp, buffer);
+		sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
 		if (sector_size == -EOVERFLOW)
 			goto got_data;
 		if (sector_size == -ENODEV)
@@ -2737,7 +2750,7 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
 			int old_sector_size = sector_size;
 			sd_printk(KERN_NOTICE, sdkp, "Very big device. "
 					"Trying to use READ CAPACITY(16).\n");
-			sector_size = read_capacity_16(sdkp, sdp, buffer);
+			sector_size = read_capacity_16(sdkp, sdp, lim, buffer);
 			if (sector_size < 0) {
 				sd_printk(KERN_NOTICE, sdkp,
 					"Using 0xffffffff as device size\n");
@@ -2796,9 +2809,8 @@ sd_read_capacity(struct scsi_disk *sdkp, unsigned char *buffer)
 		 */
 		sector_size = 512;
 	}
-	blk_queue_logical_block_size(sdp->request_queue, sector_size);
-	blk_queue_physical_block_size(sdp->request_queue,
-				      sdkp->physical_block_size);
+	lim->logical_block_size = sector_size;
+	lim->physical_block_size = sdkp->physical_block_size;
 	sdkp->device->sector_size = sector_size;
 
 	if (sdkp->capacity > 0xffffffff)
@@ -3220,11 +3232,11 @@ static unsigned int sd_discard_mode(struct scsi_disk *sdkp)
 	return SD_LBP_DISABLE;
 }
 
-/**
- * sd_read_block_limits - Query disk device for preferred I/O sizes.
- * @sdkp: disk to query
+/*
+ * Query disk device for preferred I/O sizes.
  */
-static void sd_read_block_limits(struct scsi_disk *sdkp)
+static void sd_read_block_limits(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
 	struct scsi_vpd *vpd;
 
@@ -3258,7 +3270,7 @@ static void sd_read_block_limits(struct scsi_disk *sdkp)
 			sdkp->unmap_alignment =
 				get_unaligned_be32(&vpd->data[32]) & ~(1 << 31);
 
-		sd_config_discard(sdkp, sd_discard_mode(sdkp));
+		sd_config_discard(sdkp, lim, sd_discard_mode(sdkp));
 	}
 
  out:
@@ -3277,11 +3289,9 @@ static void sd_read_block_limits_ext(struct scsi_disk *sdkp)
 	rcu_read_unlock();
 }
 
-/**
- * sd_read_block_characteristics - Query block dev. characteristics
- * @sdkp: disk to query
- */
-static void sd_read_block_characteristics(struct scsi_disk *sdkp)
+/* Query block device characteristics */
+static void sd_read_block_characteristics(struct scsi_disk *sdkp,
+		struct queue_limits *lim)
 {
 	struct request_queue *q = sdkp->disk->queue;
 	struct scsi_vpd *vpd;
@@ -3307,29 +3317,26 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 
 #ifdef CONFIG_BLK_DEV_ZONED /* sd_probe rejects ZBD devices early otherwise */
 	if (sdkp->device->type == TYPE_ZBC) {
-		/*
-		 * Host-managed.
-		 */
-		disk_set_zoned(sdkp->disk);
+		lim->zoned = true;
 
 		/*
 		 * Per ZBC and ZAC specifications, writes in sequential write
 		 * required zones of host-managed devices must be aligned to
 		 * the device physical block size.
 		 */
-		blk_queue_zone_write_granularity(q, sdkp->physical_block_size);
+		lim->zone_write_granularity = sdkp->physical_block_size;
 	} else {
 		/*
 		 * Host-aware devices are treated as conventional.
 		 */
-		WARN_ON_ONCE(blk_queue_is_zoned(q));
+		lim->zoned = false;
 	}
 #endif /* CONFIG_BLK_DEV_ZONED */
 
 	if (!sdkp->first_scan)
 		return;
 
-	if (blk_queue_is_zoned(q))
+	if (lim->zoned)
 		sd_printk(KERN_NOTICE, sdkp, "Host-managed zoned block device\n");
 	else if (sdkp->zoned == 1)
 		sd_printk(KERN_NOTICE, sdkp, "Host-aware SMR disk used as regular disk\n");
@@ -3605,8 +3612,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
 	struct scsi_device *sdp = sdkp->device;
 	struct request_queue *q = sdkp->disk->queue;
 	sector_t old_capacity = sdkp->capacity;
+	struct queue_limits lim;
 	unsigned char *buffer;
 	unsigned int dev_max;
+	int err;
 
 	SCSI_LOG_HLQUEUE(3, sd_printk(KERN_INFO, sdkp,
 				      "sd_revalidate_disk\n"));
@@ -3627,12 +3636,14 @@ static int sd_revalidate_disk(struct gendisk *disk)
 
 	sd_spinup_disk(sdkp);
 
+	lim = queue_limits_start_update(sdkp->disk->queue);
+
 	/*
 	 * Without media there is no reason to ask; moreover, some devices
 	 * react badly if we do.
 	 */
 	if (sdkp->media_present) {
-		sd_read_capacity(sdkp, buffer);
+		sd_read_capacity(sdkp, &lim, buffer);
 		/*
 		 * Some USB/UAS devices return generic values for mode pages
 		 * until the media has been accessed. Trigger a READ operation
@@ -3651,10 +3662,10 @@ static int sd_revalidate_disk(struct gendisk *disk)
 
 		if (scsi_device_supports_vpd(sdp)) {
 			sd_read_block_provisioning(sdkp);
-			sd_read_block_limits(sdkp);
+			sd_read_block_limits(sdkp, &lim);
 			sd_read_block_limits_ext(sdkp);
-			sd_read_block_characteristics(sdkp);
-			sd_zbc_read_zones(sdkp, buffer);
+			sd_read_block_characteristics(sdkp, &lim);
+			sd_zbc_read_zones(sdkp, &lim, buffer);
 			sd_read_cpr(sdkp);
 		}
 
@@ -3680,31 +3691,36 @@ static int sd_revalidate_disk(struct gendisk *disk)
 
 	/* Some devices report a maximum block count for READ/WRITE requests. */
 	dev_max = min_not_zero(dev_max, sdkp->max_xfer_blocks);
-	q->limits.max_dev_sectors = logical_to_sectors(sdp, dev_max);
+	lim.max_dev_sectors = logical_to_sectors(sdp, dev_max);
 
 	if (sd_validate_min_xfer_size(sdkp))
-		blk_queue_io_min(sdkp->disk->queue,
-				 logical_to_bytes(sdp, sdkp->min_xfer_blocks));
+		lim.io_min = logical_to_bytes(sdp, sdkp->min_xfer_blocks);
 	else
-		blk_queue_io_min(sdkp->disk->queue, 0);
+		lim.io_min = 0;
 
 	/*
 	 * Limit default to SCSI host optimal sector limit if set. There may be
 	 * an impact on performance for when the size of a request exceeds this
 	 * host limit.
 	 */
-	q->limits.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
+	lim.io_opt = sdp->host->opt_sectors << SECTOR_SHIFT;
 	if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
-		q->limits.io_opt = min_not_zero(q->limits.io_opt,
+		lim.io_opt = min_not_zero(lim.io_opt,
 				logical_to_bytes(sdp, sdkp->opt_xfer_blocks));
 	}
 
 	sdkp->first_scan = 0;
 
 	set_capacity_and_notify(disk, logical_to_sectors(sdp, sdkp->capacity));
-	sd_config_write_same(sdkp);
+	sd_config_write_same(sdkp, &lim);
 	kfree(buffer);
 
+	blk_mq_freeze_queue(sdkp->disk->queue);
+	err = queue_limits_commit_update(sdkp->disk->queue, &lim);
+	blk_mq_unfreeze_queue(sdkp->disk->queue);
+	if (err)
+		return err;
+
 	/*
 	 * For a zoned drive, revalidating the zones can be done only once
 	 * the gendisk capacity is set. So if this fails, set back the gendisk
diff --git a/drivers/scsi/sd.h b/drivers/scsi/sd.h
index 49dd600bfa4825..b4170b17bad47a 100644
--- a/drivers/scsi/sd.h
+++ b/drivers/scsi/sd.h
@@ -239,7 +239,8 @@ static inline int sd_is_zoned(struct scsi_disk *sdkp)
 
 #ifdef CONFIG_BLK_DEV_ZONED
 
-int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE]);
+int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
+		u8 buf[SD_BUF_SIZE]);
 int sd_zbc_revalidate_zones(struct scsi_disk *sdkp);
 blk_status_t sd_zbc_setup_zone_mgmt_cmnd(struct scsi_cmnd *cmd,
 					 unsigned char op, bool all);
@@ -250,7 +251,8 @@ int sd_zbc_report_zones(struct gendisk *disk, sector_t sector,
 
 #else /* CONFIG_BLK_DEV_ZONED */
 
-static inline int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
+static inline int sd_zbc_read_zones(struct scsi_disk *sdkp,
+		struct queue_limits *lim, u8 buf[SD_BUF_SIZE])
 {
 	return 0;
 }
diff --git a/drivers/scsi/sd_zbc.c b/drivers/scsi/sd_zbc.c
index 1c24c844e8d178..f685838d9ed214 100644
--- a/drivers/scsi/sd_zbc.c
+++ b/drivers/scsi/sd_zbc.c
@@ -582,13 +582,15 @@ int sd_zbc_revalidate_zones(struct scsi_disk *sdkp)
 /**
  * sd_zbc_read_zones - Read zone information and update the request queue
  * @sdkp: SCSI disk pointer.
+ * @lim: queue limits to read into
  * @buf: 512 byte buffer used for storing SCSI command output.
  *
  * Read zone information and update the request queue zone characteristics and
  * also the zoned device information in *sdkp. Called by sd_revalidate_disk()
  * before the gendisk capacity has been set.
  */
-int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
+int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
+		u8 buf[SD_BUF_SIZE])
 {
 	struct gendisk *disk = sdkp->disk;
 	struct request_queue *q = disk->queue;
@@ -626,14 +628,13 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, u8 buf[SD_BUF_SIZE])
 	/* The drive satisfies the kernel restrictions: set it up */
 	blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q);
 	if (sdkp->zones_max_open == U32_MAX)
-		disk_set_max_open_zones(disk, 0);
+		lim->max_open_zones = 0;
 	else
-		disk_set_max_open_zones(disk, sdkp->zones_max_open);
-	disk_set_max_active_zones(disk, 0);
-	blk_queue_chunk_sectors(q,
-			logical_to_sectors(sdkp->device, zone_blocks));
+		lim->max_open_zones = sdkp->zones_max_open;
+	lim->max_active_zones = 0;
+	lim->chunk_sectors = logical_to_sectors(sdkp->device, zone_blocks);
 	/* Enable block layer zone append emulation */
-	blk_queue_max_zone_append_sectors(q, 0);
+	lim->max_zone_append_sectors = 0;
 
 	return 0;
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 08:00:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:00:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733402.1139648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxBi-0006JX-8j; Fri, 31 May 2024 08:00:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733402.1139648; Fri, 31 May 2024 08:00:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxBi-0006JQ-53; Fri, 31 May 2024 08:00:34 +0000
Received: by outflank-mailman (input) for mailman id 733402;
 Fri, 31 May 2024 08:00:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0x-0004oC-OS
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4cbecf75-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:25 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0o-00000009Xnc-0qaG; Fri, 31 May 2024 07:49:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cbecf75-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=VM6XeFxtmVkC4/0tlWYKENrMzCOxD79VPgH4x5Iftu4=; b=TtAcIWToSwgYE7/eKbvt1JeGnu
	K6jCaQ+oOF4MOabiDrdS1DpByF7HLTL42Nada3MVLJuvOEnWzuv49rlFMrSITmS7JtQQmEQIv1IpL
	EhyShbLrrbw2TQkfscjrx3n/PyiZ4XYWP6uPHhhI57QlkLxRBZ8boqJFrE07vukKhLiIdONr5Uo97
	MTXqd5ka1xFxWja47jYedCvUzLs+2w6DqqGB5zB2gqkC4eXuntv9pCvhhP3wbBnuYxVrY6hzA3kpg
	G2skJjdOBPQH//TNcCeTXixNjq0TQG0YsabZ3cdno+GaeBtkLfP5jc6QzjJXKGmhhODxSQHkHmFQt
	gWimIxvQ==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>
Subject: [PATCH 13/14] block: remove unused queue limits API
Date: Fri, 31 May 2024 09:48:08 +0200
Message-ID: <20240531074837.1648501-14-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Remove all APIs that are unused now that sd and sr have been converted
to the atomic queue limits API.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
---
 block/blk-settings.c   | 190 -----------------------------------------
 include/linux/blkdev.h |  24 ------
 2 files changed, 214 deletions(-)

diff --git a/block/blk-settings.c b/block/blk-settings.c
index a49abdb3554834..0b038729608f4b 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -293,24 +293,6 @@ int queue_limits_set(struct request_queue *q, struct queue_limits *lim)
 }
 EXPORT_SYMBOL_GPL(queue_limits_set);
 
-/**
- * blk_queue_chunk_sectors - set size of the chunk for this queue
- * @q:  the request queue for the device
- * @chunk_sectors:  chunk sectors in the usual 512b unit
- *
- * Description:
- *    If a driver doesn't want IOs to cross a given chunk size, it can set
- *    this limit and prevent merging across chunks. Note that the block layer
- *    must accept a page worth of data at any offset. So if the crossing of
- *    chunks is a hard limitation in the driver, it must still be prepared
- *    to split single page bios.
- **/
-void blk_queue_chunk_sectors(struct request_queue *q, unsigned int chunk_sectors)
-{
-	q->limits.chunk_sectors = chunk_sectors;
-}
-EXPORT_SYMBOL(blk_queue_chunk_sectors);
-
 /**
  * blk_queue_max_discard_sectors - set max sectors for a single discard
  * @q:  the request queue for the device
@@ -352,139 +334,6 @@ void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
 }
 EXPORT_SYMBOL(blk_queue_max_write_zeroes_sectors);
 
-/**
- * blk_queue_max_zone_append_sectors - set max sectors for a single zone append
- * @q:  the request queue for the device
- * @max_zone_append_sectors: maximum number of sectors to write per command
- *
- * Sets the maximum number of sectors allowed for zone append commands. If
- * Specifying 0 for @max_zone_append_sectors indicates that the queue does
- * not natively support zone append operations and that the block layer must
- * emulate these operations using regular writes.
- **/
-void blk_queue_max_zone_append_sectors(struct request_queue *q,
-		unsigned int max_zone_append_sectors)
-{
-	unsigned int max_sectors = 0;
-
-	if (WARN_ON(!blk_queue_is_zoned(q)))
-		return;
-
-	if (max_zone_append_sectors) {
-		max_sectors = min(q->limits.max_hw_sectors,
-				  max_zone_append_sectors);
-		max_sectors = min(q->limits.chunk_sectors, max_sectors);
-
-		/*
-		 * Signal eventual driver bugs resulting in the max_zone_append
-		 * sectors limit being 0 due to the chunk_sectors limit (zone
-		 * size) not set or the max_hw_sectors limit not set.
-		 */
-		WARN_ON_ONCE(!max_sectors);
-	}
-
-	q->limits.max_zone_append_sectors = max_sectors;
-}
-EXPORT_SYMBOL_GPL(blk_queue_max_zone_append_sectors);
-
-/**
- * blk_queue_logical_block_size - set logical block size for the queue
- * @q:  the request queue for the device
- * @size:  the logical block size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible block size that the
- *   storage device can address.  The default of 512 covers most
- *   hardware.
- **/
-void blk_queue_logical_block_size(struct request_queue *q, unsigned int size)
-{
-	struct queue_limits *limits = &q->limits;
-
-	limits->logical_block_size = size;
-
-	if (limits->discard_granularity < limits->logical_block_size)
-		limits->discard_granularity = limits->logical_block_size;
-
-	if (limits->physical_block_size < size)
-		limits->physical_block_size = size;
-
-	if (limits->io_min < limits->physical_block_size)
-		limits->io_min = limits->physical_block_size;
-
-	limits->max_hw_sectors =
-		round_down(limits->max_hw_sectors, size >> SECTOR_SHIFT);
-	limits->max_sectors =
-		round_down(limits->max_sectors, size >> SECTOR_SHIFT);
-}
-EXPORT_SYMBOL(blk_queue_logical_block_size);
-
-/**
- * blk_queue_physical_block_size - set physical block size for the queue
- * @q:  the request queue for the device
- * @size:  the physical block size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible sector size that the
- *   hardware can operate on without reverting to read-modify-write
- *   operations.
- */
-void blk_queue_physical_block_size(struct request_queue *q, unsigned int size)
-{
-	q->limits.physical_block_size = size;
-
-	if (q->limits.physical_block_size < q->limits.logical_block_size)
-		q->limits.physical_block_size = q->limits.logical_block_size;
-
-	if (q->limits.discard_granularity < q->limits.physical_block_size)
-		q->limits.discard_granularity = q->limits.physical_block_size;
-
-	if (q->limits.io_min < q->limits.physical_block_size)
-		q->limits.io_min = q->limits.physical_block_size;
-}
-EXPORT_SYMBOL(blk_queue_physical_block_size);
-
-/**
- * blk_queue_zone_write_granularity - set zone write granularity for the queue
- * @q:  the request queue for the zoned device
- * @size:  the zone write granularity size, in bytes
- *
- * Description:
- *   This should be set to the lowest possible size allowing to write in
- *   sequential zones of a zoned block device.
- */
-void blk_queue_zone_write_granularity(struct request_queue *q,
-				      unsigned int size)
-{
-	if (WARN_ON_ONCE(!blk_queue_is_zoned(q)))
-		return;
-
-	q->limits.zone_write_granularity = size;
-
-	if (q->limits.zone_write_granularity < q->limits.logical_block_size)
-		q->limits.zone_write_granularity = q->limits.logical_block_size;
-}
-EXPORT_SYMBOL_GPL(blk_queue_zone_write_granularity);
-
-/**
- * blk_queue_alignment_offset - set physical block alignment offset
- * @q:	the request queue for the device
- * @offset: alignment offset in bytes
- *
- * Description:
- *   Some devices are naturally misaligned to compensate for things like
- *   the legacy DOS partition table 63-sector offset.  Low-level drivers
- *   should call this function for devices whose first sector is not
- *   naturally aligned.
- */
-void blk_queue_alignment_offset(struct request_queue *q, unsigned int offset)
-{
-	q->limits.alignment_offset =
-		offset & (q->limits.physical_block_size - 1);
-	q->limits.misaligned = 0;
-}
-EXPORT_SYMBOL(blk_queue_alignment_offset);
-
 void disk_update_readahead(struct gendisk *disk)
 {
 	blk_apply_bdi_limits(disk->bdi, &disk->queue->limits);
@@ -514,26 +363,6 @@ void blk_limits_io_min(struct queue_limits *limits, unsigned int min)
 }
 EXPORT_SYMBOL(blk_limits_io_min);
 
-/**
- * blk_queue_io_min - set minimum request size for the queue
- * @q:	the request queue for the device
- * @min:  smallest I/O size in bytes
- *
- * Description:
- *   Storage devices may report a granularity or preferred minimum I/O
- *   size which is the smallest request the device can perform without
- *   incurring a performance penalty.  For disk drives this is often the
- *   physical block size.  For RAID arrays it is often the stripe chunk
- *   size.  A properly aligned multiple of minimum_io_size is the
- *   preferred request size for workloads where a high number of I/O
- *   operations is desired.
- */
-void blk_queue_io_min(struct request_queue *q, unsigned int min)
-{
-	blk_limits_io_min(&q->limits, min);
-}
-EXPORT_SYMBOL(blk_queue_io_min);
-
 /**
  * blk_limits_io_opt - set optimal request size for a device
  * @limits: the queue limits
@@ -841,25 +670,6 @@ void blk_queue_write_cache(struct request_queue *q, bool wc, bool fua)
 }
 EXPORT_SYMBOL_GPL(blk_queue_write_cache);
 
-/**
- * disk_set_zoned - inidicate a zoned device
- * @disk:	gendisk to configure
- */
-void disk_set_zoned(struct gendisk *disk)
-{
-	struct request_queue *q = disk->queue;
-
-	WARN_ON_ONCE(!IS_ENABLED(CONFIG_BLK_DEV_ZONED));
-
-	/*
-	 * Set the zone write granularity to the device logical block
-	 * size by default. The driver can change this value if needed.
-	 */
-	q->limits.zoned = true;
-	blk_queue_zone_write_granularity(q, queue_logical_block_size(q));
-}
-EXPORT_SYMBOL_GPL(disk_set_zoned);
-
 int bdev_alignment_offset(struct block_device *bdev)
 {
 	struct request_queue *q = bdev_get_queue(bdev);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 24c36929920b76..ad995e7a769811 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -332,8 +332,6 @@ struct queue_limits {
 typedef int (*report_zones_cb)(struct blk_zone *zone, unsigned int idx,
 			       void *data);
 
-void disk_set_zoned(struct gendisk *disk);
-
 #define BLK_ALL_ZONES  ((unsigned int)-1)
 int blkdev_report_zones(struct block_device *bdev, sector_t sector,
 		unsigned int nr_zones, report_zones_cb cb, void *data);
@@ -638,18 +636,6 @@ static inline unsigned int disk_zone_no(struct gendisk *disk, sector_t sector)
 	return sector >> ilog2(disk->queue->limits.chunk_sectors);
 }
 
-static inline void disk_set_max_open_zones(struct gendisk *disk,
-		unsigned int max_open_zones)
-{
-	disk->queue->limits.max_open_zones = max_open_zones;
-}
-
-static inline void disk_set_max_active_zones(struct gendisk *disk,
-		unsigned int max_active_zones)
-{
-	disk->queue->limits.max_active_zones = max_active_zones;
-}
-
 static inline unsigned int bdev_max_open_zones(struct block_device *bdev)
 {
 	return bdev->bd_disk->queue->limits.max_open_zones;
@@ -929,24 +915,14 @@ static inline void queue_limits_cancel_update(struct request_queue *q)
 /*
  * Access functions for manipulating queue properties
  */
-extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int);
 void blk_queue_max_secure_erase_sectors(struct request_queue *q,
 		unsigned int max_sectors);
 extern void blk_queue_max_discard_sectors(struct request_queue *q,
 		unsigned int max_discard_sectors);
 extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
 		unsigned int max_write_same_sectors);
-extern void blk_queue_logical_block_size(struct request_queue *, unsigned int);
-extern void blk_queue_max_zone_append_sectors(struct request_queue *q,
-		unsigned int max_zone_append_sectors);
-extern void blk_queue_physical_block_size(struct request_queue *, unsigned int);
-void blk_queue_zone_write_granularity(struct request_queue *q,
-				      unsigned int size);
-extern void blk_queue_alignment_offset(struct request_queue *q,
-				       unsigned int alignment);
 void disk_update_readahead(struct gendisk *disk);
 extern void blk_limits_io_min(struct queue_limits *limits, unsigned int min);
-extern void blk_queue_io_min(struct request_queue *q, unsigned int min);
 extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
 extern void blk_set_queue_depth(struct request_queue *q, unsigned int depth);
 extern void blk_set_stacking_limits(struct queue_limits *lim);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 08:03:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:03:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733417.1139659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxE5-00075A-LH; Fri, 31 May 2024 08:03:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733417.1139659; Fri, 31 May 2024 08:03:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxE5-000753-HO; Fri, 31 May 2024 08:03:01 +0000
Received: by outflank-mailman (input) for mailman id 733417;
 Fri, 31 May 2024 08:03:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rv5j=NC=bombadil.srs.infradead.org=BATV+4872a67a45e6d46d0c49+7586+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1sCx0u-0004oC-43
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 07:49:24 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4ad0f821-1f22-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 09:49:22 +0200 (CEST)
Received: from
 2a02-8389-2341-5b80-5ba9-f4da-76fa-44a9.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:5ba9:f4da:76fa:44a9] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1sCx0l-00000009XmW-2kNJ; Fri, 31 May 2024 07:49:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ad0f821-1f22-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender
	:Reply-To:Content-Type:Content-ID:Content-Description;
	bh=iMhM+qb16rWeYhTsrEp/EIKofeiSWWDmAm8PBvcX+R4=; b=WCl9PeXI0Uir7+ImFplX/mJBQW
	PadlppOJVLv1H1hcpEXK7QPq3XeF2IRiVoWUgG06Bh9FnczQtyVExyMIqaOnNpTw7P9Qo5r7UQf7k
	x5VPZ3bxf0mISNX3yuCLpzW2M7W83yLsnfawyqzrxdNRxcGrvghcagqikIplPccHgDyHAPvaOPDsT
	DoqTeon8mAhFTfO+PnoCohD4y2G9mYvaOBqH16GvMM9OIcnCL2KYbiEGUYtjFOolTa87gmoiBMDX5
	/Rynbgk2G8Gy/kQrHMItE34V5vqmXOweozZLBXWJGVWctwcegjzZ3GIsVcGs9nKQcYFWw/jkzPHqw
	NCqH0Xxg==;
From: Christoph Hellwig <hch@lst.de>
To: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org,
	linux-block@vger.kernel.org,
	nbd@other.debian.org,
	ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-scsi@vger.kernel.org,
	Damien Le Moal <dlemoal@kernel.org>
Subject: [PATCH 12/14] sr: convert to the atomic queue limits API
Date: Fri, 31 May 2024 09:48:07 +0200
Message-ID: <20240531074837.1648501-13-hch@lst.de>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
References: <20240531074837.1648501-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Assign all queue limits through a local queue_limits variable and
queue_limits_commit_update so that we can't race updating them from
multiple places, and free the queue when updating them so that
in-progress I/O submissions don't see half-updated limits.

Also use the chance to clean up variable names to standard ones.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
---
 drivers/scsi/sr.c | 42 +++++++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 17 deletions(-)

diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c
index 7ab000942b97fc..3f491019103e0c 100644
--- a/drivers/scsi/sr.c
+++ b/drivers/scsi/sr.c
@@ -111,7 +111,7 @@ static struct lock_class_key sr_bio_compl_lkclass;
 static int sr_open(struct cdrom_device_info *, int);
 static void sr_release(struct cdrom_device_info *);
 
-static void get_sectorsize(struct scsi_cd *);
+static int get_sectorsize(struct scsi_cd *);
 static int get_capabilities(struct scsi_cd *);
 
 static unsigned int sr_check_events(struct cdrom_device_info *cdi,
@@ -473,15 +473,15 @@ static blk_status_t sr_init_command(struct scsi_cmnd *SCpnt)
 	return BLK_STS_IOERR;
 }
 
-static void sr_revalidate_disk(struct scsi_cd *cd)
+static int sr_revalidate_disk(struct scsi_cd *cd)
 {
 	struct scsi_sense_hdr sshdr;
 
 	/* if the unit is not ready, nothing more to do */
 	if (scsi_test_unit_ready(cd->device, SR_TIMEOUT, MAX_RETRIES, &sshdr))
-		return;
+		return 0;
 	sr_cd_check(&cd->cdi);
-	get_sectorsize(cd);
+	return get_sectorsize(cd);
 }
 
 static int sr_block_open(struct gendisk *disk, blk_mode_t mode)
@@ -494,13 +494,16 @@ static int sr_block_open(struct gendisk *disk, blk_mode_t mode)
 		return -ENXIO;
 
 	scsi_autopm_get_device(sdev);
-	if (disk_check_media_change(disk))
-		sr_revalidate_disk(cd);
+	if (disk_check_media_change(disk)) {
+		ret = sr_revalidate_disk(cd);
+		if (ret)
+			goto out;
+	}
 
 	mutex_lock(&cd->lock);
 	ret = cdrom_open(&cd->cdi, mode);
 	mutex_unlock(&cd->lock);
-
+out:
 	scsi_autopm_put_device(sdev);
 	if (ret)
 		scsi_device_put(cd->device);
@@ -685,7 +688,9 @@ static int sr_probe(struct device *dev)
 	blk_pm_runtime_init(sdev->request_queue, dev);
 
 	dev_set_drvdata(dev, cd);
-	sr_revalidate_disk(cd);
+	error = sr_revalidate_disk(cd);
+	if (error)
+		goto unregister_cdrom;
 
 	error = device_add_disk(&sdev->sdev_gendev, disk, NULL);
 	if (error)
@@ -714,13 +719,14 @@ static int sr_probe(struct device *dev)
 }
 
 
-static void get_sectorsize(struct scsi_cd *cd)
+static int get_sectorsize(struct scsi_cd *cd)
 {
+	struct request_queue *q = cd->device->request_queue;
 	static const u8 cmd[10] = { READ_CAPACITY };
 	unsigned char buffer[8] = { };
-	int the_result;
+	struct queue_limits lim;
+	int err;
 	int sector_size;
-	struct request_queue *queue;
 	struct scsi_failure failure_defs[] = {
 		{
 			.result = SCMD_FAILURE_RESULT_ANY,
@@ -736,10 +742,10 @@ static void get_sectorsize(struct scsi_cd *cd)
 	};
 
 	/* Do the command and wait.. */
-	the_result = scsi_execute_cmd(cd->device, cmd, REQ_OP_DRV_IN, buffer,
+	err = scsi_execute_cmd(cd->device, cmd, REQ_OP_DRV_IN, buffer,
 				      sizeof(buffer), SR_TIMEOUT, MAX_RETRIES,
 				      &exec_args);
-	if (the_result) {
+	if (err) {
 		cd->capacity = 0x1fffff;
 		sector_size = 2048;	/* A guess, just in case */
 	} else {
@@ -789,10 +795,12 @@ static void get_sectorsize(struct scsi_cd *cd)
 		set_capacity(cd->disk, cd->capacity);
 	}
 
-	queue = cd->device->request_queue;
-	blk_queue_logical_block_size(queue, sector_size);
-
-	return;
+	lim = queue_limits_start_update(q);
+	lim.logical_block_size = sector_size;
+	blk_mq_freeze_queue(q);
+	err = queue_limits_commit_update(q, &lim);
+	blk_mq_unfreeze_queue(q);
+	return err;
 }
 
 static int get_capabilities(struct scsi_cd *cd)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri May 31 08:05:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:05:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733423.1139669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxGK-0007gF-1b; Fri, 31 May 2024 08:05:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733423.1139669; Fri, 31 May 2024 08:05:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxGJ-0007g8-Tk; Fri, 31 May 2024 08:05:19 +0000
Received: by outflank-mailman (input) for mailman id 733423;
 Fri, 31 May 2024 08:05:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=59kU=NC=epam.com=prvs=2881f902b4=sergiy_kibrik@srs-se1.protection.inumbo.net>)
 id 1sCxGI-0007g2-Au
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:05:18 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83b65c3a-1f24-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 10:05:17 +0200 (CEST)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44V1UdlF013884;
 Fri, 31 May 2024 08:05:06 GMT
Received: from eur03-am7-obe.outbound.protection.outlook.com
 (mail-am7eur03lp2233.outbound.protection.outlook.com [104.47.51.233])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3yev5nt87p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 31 May 2024 08:05:06 +0000 (GMT)
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com (2603:10a6:20b:5c0::11)
 by AM9PR03MB7044.eurprd03.prod.outlook.com (2603:10a6:20b:2d9::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.17; Fri, 31 May
 2024 08:05:02 +0000
Received: from AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d]) by AS8PR03MB9192.eurprd03.prod.outlook.com
 ([fe80::baa9:29b3:908:ed7d%6]) with mapi id 15.20.7611.013; Fri, 31 May 2024
 08:05:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83b65c3a-1f24-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vp6yV8MsnM3nl3DbwOMEN9f5mS8CZc8AZSDoroDmiMuEMwOfys1ubxU8TlGP9+dkxTa2qUgUMUIpahnq+IHZDDhV+9sAQudOgBfNhsprgCojt5qD72EwJhZzpYiV0p4pe3bHmj16m4H80owEc/NM7b/5OTJJW50kXAkp1y6fmpIc6jFen1TrsDBx56sIsd5v7HJC9vMoCn7NZIqBA0wBhdKHjjpWwYJGJUyfVcWvdpB/G5USQPrbA53sJY1T3cxSSAa413RS7R/2H8nImgkGjBr1O1cMFBSdJYZhGomSZelHSWsnQmgs/reit+aAKTm59vwIN+cz4BkYlfk3GGTKTQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=s9tF0rt+SyqLrQlBLDjA2AhwTEJNKlTC9BNsmMaHhNA=;
 b=cH3/eg4OonlmShqQTfG9C0DZldr4NNFyNwAFC5rGTsVza06f+OJnjhOBCB2PT98enTHhadD8Y/XTcUJRrwwNNTIaBjHFMAufuOVy+ERZW6I3JirwZBYpaT+Bxci0AdvsTocTya19aJT84Y0R0Rzw5TbVYjGHH8a7hBMq+eppw5oAUhd8DWWRtfJf1xeTpzRiKcuLoqdYFZ9weBiT5opnoYmQymj65+oUeghcMPEhKmO09n97BBcdMqQ3HofC7cPzw4t/GVDtpo5b7m9RyYG+DVS4Tx9Fl6Gy/7zm+Slc9KSDR8Kc+YutVbkwT5BdW2bkfheajOxOg86QWTn8+v92pg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=s9tF0rt+SyqLrQlBLDjA2AhwTEJNKlTC9BNsmMaHhNA=;
 b=qMxadjsl3Jb+w8aSWG1URX1ROQsiqL9b5518AVK13iHMcm1FvxGsMkwPDhYn+Zk5+n0cRCdnx07zN6Bi4F+JwznfW/19hLtafa8xG0QK9a3JPNh0MUjASSiG/RMUScRMpYaz+vIegR/diK32sXWKLJa/xtY9BDW2Kn6490bhpLWeCPAUNHdHk+IfCmKRKy9gJ8drKcEodOoaEFjgY2t2kHQVuMiQpFmYnBA16AxEvo5bWBmZOpzvUNQYlsqL2swlq8RGCnKv00K/erF54QHYIlX+c3u8o4u4KFKEmFM4hQkQuz1tXkBNsP83QGXDuSD2xGZo56HF8m/jfeavdri9LQ==
Message-ID: <a38ac7e4-cf47-40a4-b76c-6752237b1ccc@epam.com>
Date: Fri, 31 May 2024 11:05:00 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 13/15] x86/ioreq: guard VIO_realmode_completion
 with CONFIG_VMX
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        George Dunlap <george.dunlap@citrix.com>,
        Julien Grall <julien@xen.org>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Xenia Ragiadakou <xenia.ragiadakou@amd.com>,
        xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
 <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
Content-Language: en-US
From: Sergiy Kibrik <sergiy_kibrik@epam.com>
In-Reply-To: <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: WA1P291CA0024.POLP291.PROD.OUTLOOK.COM
 (2603:10a6:1d0:19::18) To AS8PR03MB9192.eurprd03.prod.outlook.com
 (2603:10a6:20b:5c0::11)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: AS8PR03MB9192:EE_|AM9PR03MB7044:EE_
X-MS-Office365-Filtering-Correlation-Id: d48f7cc0-35ef-4097-b925-08dc81485fb2
X-LD-Processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|366007|1800799015;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?RTNtMFFndnlPcW0zREhEWUkvMXlJN0tFdTlQaGlhRE9pdVo4U0tOWG4rTUJJ?=
 =?utf-8?B?SzhaSFFkR29NSlFoK0Z1T0hIbWFkaElSVTVSWElrY2hJMlVBYWFyaXcrTHM4?=
 =?utf-8?B?WHc3YXJtQkEwOURiNnBWYW9uVUJjN2FFZFZsZmNkYkpvMUtkeEtUR1liZGxl?=
 =?utf-8?B?NmYwd1I3YW9QdmI4eXF4dm9pdVdDTTVrSkpLaUlPcVQ4MkdvSjlhbjM1N00v?=
 =?utf-8?B?SlR4eUVXb200VGYvQlNlUHV2OFZhTkNlTlVBc2pPOXNaMVhNYjJTK0ppQ2gx?=
 =?utf-8?B?S3FJZGRNLzdrTVZsTWJuTjdDZ1RJUktzdzJSUndMdVdTMElEbkZNVmhJcmlE?=
 =?utf-8?B?WnNHWS9vc1VrOGN3NWFsZitGcE1kcmlmTHg2YjY5eERDTFZiOHlxY2ZwbU13?=
 =?utf-8?B?UHhVTXU1eTRyQlMvWmRGMDRJMFFJTENFU25lOFdGOXFqU0ZsSmZybkdGZTdN?=
 =?utf-8?B?UmcvekJjcVE4d0RLMTBZNmxCUWREc1owWm1DSTlnc0R0eTdtZTJkNDJsZ0t6?=
 =?utf-8?B?RUZEek1WMWorYWhGNnV4dmtmZi9NU2ZoVXBPdGY4QVJGcUdQYnNZeFpRZGZZ?=
 =?utf-8?B?dDJHMWx2RWViS0oySTYxbjExcG9vNUYwUUJ0MHdzWHFYS3M1bW8zdHdMN2Nx?=
 =?utf-8?B?YnFtMlpJWXpPTGhoOE1kUmhYSVhjK29aSWhVN1pWUTJ6YXl5cEFFU0w5ZXdr?=
 =?utf-8?B?ckNEVWlTZ0hXMUtFakk0dGpOUXVtL3NwVVppNXN3YWppVWtmcksrRHk3cnE3?=
 =?utf-8?B?b0xucnJOV0kxcEZVS0JJL2kvNTg2NkVGcFpaVFN5WDlkTWVOdm1vV0kxU0Fi?=
 =?utf-8?B?bGMyVDVKajdSS3B3d1FNUkRkYnNjWEVPeUVLMDNIWXVwTzhhVVJxWm5ic2VB?=
 =?utf-8?B?VlNPYjN4NEpLSDFJVnJmTnoyemUzYldRejVrZ3BNMDVmRGpvd01YOUQ5ckQv?=
 =?utf-8?B?Ziszb1AyUnJYRHZUdjdDWCtrcmVpYUZqRDZaYXFIU1dDaU5LRFRSdGV6blha?=
 =?utf-8?B?am8yQ2RYdU1pazZTWWZUTEFPSjZvZHZ5TnJNc05OcUw0NTlsSjNLNTZURG1h?=
 =?utf-8?B?WmdUaDd3S1JaVmNodDFmTG9mRktKNTFmaVo2NUpUNHR0bGQxTVFva3FIaytX?=
 =?utf-8?B?clJ4dmRTamhZWmthR1NVcXh4TExCVE01anZIU2RlOG1hR3MyV2xOOHN1aVRj?=
 =?utf-8?B?dGdtdUQrNmFsZXpGYXA3UGFnaDBQaGErY2dnSVg2NUhJZ21wRE0ycVVLYmRa?=
 =?utf-8?B?ZzY3QTZFOWNaYytlU2JnR0NIL0VrZ0NMb2tNOHl1RStRTm9qL3VNRzlnbzVQ?=
 =?utf-8?B?cmNEUkRqOUlkMjFSMGlEcDUrcmg3VEF3RVZUakxQZHlKYkNGVWhsT0FBRU84?=
 =?utf-8?B?WFM3bHhBVzVQL2lzcUFReDVkdDhPYlFoRFg4aGtTYVpNc0lxQWp4czBuVmt5?=
 =?utf-8?B?bmgrNVM3NkhOUkxOaktsbUU1dFB3VGRjUWdGZ3NSRTI2djlKQ3hoUmdHZFp3?=
 =?utf-8?B?SnRWRmJiUTZmQlRycEkrRTZHSEF1SDhwalNFMnhVNDhVdDA2eXJHVHl2Y3ZC?=
 =?utf-8?B?bks3RXlJdXZqUEpESm40OWhDTDlYS3RpWk1MSWtOaG1xS0ZucDM1V2M0UnZx?=
 =?utf-8?B?NnZoa25DWUx0RG4wZXpCdzRnVFM1TExQRzh3S3M0a2JaSzN2UlJ5a2ZzZHJx?=
 =?utf-8?B?dzNvdDJxOXVOWXdLRTNsM3ZoN2JTNTBOTlNGNGtGV3NxcnFFZEpQeGdnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR03MB9192.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(366007)(1800799015);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ZXlmZjQzZ0FUbVBCRnVlR3dkRy96SHgyTTJaSjlHUHZUOHFtdXZqeXZnTlN0?=
 =?utf-8?B?ZzlMai9aY0JjRXA5aHo5VjhiYUxCMGFwcXEvZFdXV0U1U1JrUE05YVBudVN5?=
 =?utf-8?B?MXAzZjhyNU1vbmhWS3MxWUNSSXhoQmJPMktDZnBveC9FaDk0ZEpCQ01DYktv?=
 =?utf-8?B?b2Z4VGFjUVBnNnVsYzJibkFNcTdKV3Nmd2VoSVBTaXh4b05PZnZ3cW9lUkZj?=
 =?utf-8?B?SmRCYnI3ekY0djdJa2VMQ2VZZ1NLa0dmUTI5emRVT3lORVpMempOZW82RDc0?=
 =?utf-8?B?eXZ4eHVjVnUrcnpWYVRVaHJOaHNta2UrekZBYS80VXpFdDEwcUxHOGFoOXVG?=
 =?utf-8?B?aUJ5TVdMZjhCMjRKekhyV1RXeUZFZ1dnV2poVFRySmxPdFhUOUFFa1d4d1ZT?=
 =?utf-8?B?RUpRWkg3dTgyaTkvK1BiWDQyN28vUTdnblNaeGJRTmtsajYzdEpaSWxwRjc1?=
 =?utf-8?B?MU44S1pISlkzdGN0RFlYOHphaW1vZXViQkhxV0ZuMHVLMDVscHZxSFNnRTRM?=
 =?utf-8?B?MlJrVDNacjhFYnlxVHdFWlIyWDFxQ1BIUHVxOS9iOXFhVjVXdTRCeEQ1ZUxt?=
 =?utf-8?B?OEdzOE9lSUwzNlJ4c1ZNeU9FMjRlSzdSYXYxbUcvOU93MmNFUXdxZm5DMEkv?=
 =?utf-8?B?SmdwaVk1RnVtVnRMckNVTWQvUWlMVjY2STh5WmZnZjdyVHNnMEU2TVMydVFF?=
 =?utf-8?B?OWZIR3hiYVFERks0Q2ZnYmR0UFRCRnIyTk10cnRMQURvUzIvL29SMnZZVEJ6?=
 =?utf-8?B?WG45c2pEc2J6eVplMWhZekdwdi8yaGZKYURGVGUrUDlSN0RuQWY3cHZ1Z2dO?=
 =?utf-8?B?NkVjSmFaUUUyaVBLQy9IN25keE5sdGVsa0ErYnBVSk5hbDBhRVFraU1OSGYr?=
 =?utf-8?B?YVkreFR6UzN6TVZpc2tHd1loRXkwQlhaRVBFY2hvd3poWlFyK3VyVmp4M0pF?=
 =?utf-8?B?ZGxhV01YNjYvczdFczFzMGR2RnpidFJmYUxpUlVZSCtJK0EyY2Y0SEI3RENF?=
 =?utf-8?B?L0RJS0pvRFpaRWdiZHRXOGVjSURBUFA4M2lDMFJvTTRxNHRtL2xnRFBOS1B2?=
 =?utf-8?B?Umc0bXRhait2YnkrUW1qNEZJN0plZnhDMXpUV0RqOTkzWnRiNHVwNWduZ2lx?=
 =?utf-8?B?d0FhclVOcHJ5NzdDWG5VVTlCcVk4Sys1QklTSkdPUlowRmEyZDQvWHRiVHh4?=
 =?utf-8?B?Uk1lRHlSbXhFbWRFY1c0cDlhN2NOZ1VpL0lZMExlRC9MTXFFU0RoT1dmd3M2?=
 =?utf-8?B?WUpMbXpjNGpVWmkrOHBFUEI2SUEzUCtrYk5BVU9tTVRCU0Q5bTBSRTIrVUds?=
 =?utf-8?B?dkE1bHRKUmRDcVBTbE9JcXBLWEdVT1JBTjQ0dmp6RkpIMUtJQWtpVEE3NVM0?=
 =?utf-8?B?VzFRbC9wM2JJWlc3d2JKeXoxcFljeWlIYXAzcDFoMFdiejBxS1dra2ZoOXhv?=
 =?utf-8?B?eVF6bDRtQTUyTmovdXIyYkxYMXJDOFdYM29xREovSXorUkNtWkRtSFRLT0JN?=
 =?utf-8?B?SlNwU0hlbitQc2JSWlRvOTRicVpBWHZ4MXp0eE1HL00xdmxwRXJmNmUrQzRM?=
 =?utf-8?B?eldRNFdhQnU0VlVKc3IvclpqVVJWS3ltb083R1pvNzZCT1RrSHpuVG14M0ln?=
 =?utf-8?B?bTdON2JneG92dUFGL2hhME9RRDI0TlJLMEtLQnl6VUQzRzNRTnZIdi82a0xy?=
 =?utf-8?B?SENYb2JYUjhyY2VuNFAxdU8zOFJYSkpSQ2h6cjEydVF3dDFwVEFQeEIxdTFa?=
 =?utf-8?B?bG9qbjFiZGNReGV3bXp6SzNDckxFMVVRc1l5cDVmMlhXR2M4NzJOemt4bUhs?=
 =?utf-8?B?UE5TcmZadCttaEx5SEM0K0JBRXF0Zkg2UG82OWg5dEV2cS9lQWpIV1VvcSsr?=
 =?utf-8?B?aW5OVUhRV2VHMStoMlpFYTZPS0NTQ2NGSkJJa0V1TVlWUWJ5Qk9ZNjI1SXNK?=
 =?utf-8?B?a0d2Smt5alI3WlpnSVZab2lIYm90YXJVVTgxRHU5eWRUTDJJcWpEajBGbW5H?=
 =?utf-8?B?SGV0SzZWM1VIVEEwbG5yekhaM2hNdFdmejk4dk1CZlhCUUNWSElJUkpGajZX?=
 =?utf-8?B?bkdSWVd2RDFkTnhtc0t2d1dNMVFLRWV4UjBmL2UzUTZicGVtalhWVVNaM1h5?=
 =?utf-8?Q?l9hsjQQMrZ/IvuRc7/X0D75xP?=
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d48f7cc0-35ef-4097-b925-08dc81485fb2
X-MS-Exchange-CrossTenant-AuthSource: AS8PR03MB9192.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 08:05:02.4293
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4g7h997xKovxXBB9leyQgBOSEZ/FQhhvrL+EIUc/jWmme68Yk+JlEMKCdu3wQUpy8wuWfVQ8fiktzJwn8nEVgA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7044
X-Proofpoint-GUID: OFwW0wZ9w_2CmkTD_SutCZ4ELe3zU_oC
X-Proofpoint-ORIG-GUID: OFwW0wZ9w_2CmkTD_SutCZ4ELe3zU_oC
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_04,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 spamscore=0 impostorscore=0 suspectscore=0 bulkscore=0 phishscore=0
 mlxscore=0 lowpriorityscore=0 malwarescore=0 mlxlogscore=999 adultscore=0
 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405170001 definitions=main-2405310060

16.05.24 15:11, Jan Beulich:
> On 15.05.2024 11:24, Sergiy Kibrik wrote:
>> --- a/xen/arch/x86/hvm/emulate.c
>> +++ b/xen/arch/x86/hvm/emulate.c
>> @@ -2667,7 +2667,9 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
>>           break;
>>   
>>       case VIO_mmio_completion:
>> +#ifdef CONFIG_VMX
>>       case VIO_realmode_completion:
>> +#endif
>>           BUILD_BUG_ON(sizeof(hvio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
>>           hvio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
>>           memcpy(hvio->mmio_insn, hvmemul_ctxt->insn_buf, hvio->mmio_insn_bytes);
> 
> This change doesn't buy us anything, does it?

why not? Code won't compile w/o it.
Or do you mean hiding the whole VIO_realmode_completion enum under 
CONFIG_VMX wasn't really useful?

>> --- a/xen/arch/x86/hvm/ioreq.c
>> +++ b/xen/arch/x86/hvm/ioreq.c
>> @@ -33,6 +33,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion completion)
>>   {
>>       switch ( completion )
>>       {
>> +#ifdef CONFIG_VMX
>>       case VIO_realmode_completion:
>>       {
>>           struct hvm_emulate_ctxt ctxt;
>> @@ -43,6 +44,7 @@ bool arch_vcpu_ioreq_completion(enum vio_completion completion)
>>   
>>           break;
>>       }
>> +#endif
>>   
>>       default:
>>           ASSERT_UNREACHABLE();
> 
> And while this change is needed for the goal of the series, I wonder whether
> it wouldn't better be arch_vcpu_ioreq_completion() as whole that then gets
> stubbed out.
> 

I'll post a patch to this thread to confirm whether I got your point 
correctly.

  -Sergiy


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:07:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:07:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733428.1139678 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxIK-0008J6-Ex; Fri, 31 May 2024 08:07:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733428.1139678; Fri, 31 May 2024 08:07:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxIK-0008Iz-CD; Fri, 31 May 2024 08:07:24 +0000
Received: by outflank-mailman (input) for mailman id 733428;
 Fri, 31 May 2024 08:07:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=svJw=NC=darkstar.site=sakib@srs-se1.protection.inumbo.net>)
 id 1sCxII-0008Io-EW
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:07:22 +0000
Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cbe250f6-1f24-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:07:20 +0200 (CEST)
Received: from pb-smtp1.pobox.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id A41A0189B2;
 Fri, 31 May 2024 04:07:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1])
 by pb-smtp1.pobox.com (Postfix) with ESMTP id 9AB34189B1;
 Fri, 31 May 2024 04:07:16 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
Received: from localhost (unknown [185.130.54.107])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 6179A189AE;
 Fri, 31 May 2024 04:07:15 -0400 (EDT)
 (envelope-from sakib@darkstar.site)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbe250f6-1f24-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=pobox.com; h=from:to:cc
	:subject:date:message-id:in-reply-to:references:mime-version
	:content-transfer-encoding; s=sasl; bh=yRmQ1XO4PP/T3eUSUNUUBjRlL
	OC4anerKWGpFiIymSM=; b=K/1zvroDJQilAzAKdKV0SA6ShmwQf0khKuzDfjRYV
	hSKesk4LaXFSyVtDX1fB24xt6W8gNukM0T44yd015AgVT69ZVgXsD7H9p7aDwtTo
	tBxc4gPtPcNflf0VoJnKt0z3hoWtFoIdRuynnflCoAvIlILMxF2CIbmoa1O5KbXZ
	Tg=
From: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
To: jbeulich@suse.com
Cc: andrew.cooper3@citrix.com,
	george.dunlap@citrix.com,
	julien@xen.org,
	roger.pau@citrix.com,
	sstabellini@kernel.org,
	xen-devel@lists.xenproject.org,
	xenia.ragiadakou@amd.com,
	Sergiy Kibrik <Sergiy_Kibrik@epam.com>
Subject: [XEN PATCH RFC] ioreq: make arch_vcpu_ioreq_completion() an optional callback
Date: Fri, 31 May 2024 11:06:56 +0300
Message-Id: <20240531080656.2270399-1-Sergiy_Kibrik@epam.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
References: <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
MIME-Version: 1.0
X-Pobox-Relay-ID:
 CAD7A984-1F24-11EF-8941-B84BEB2EC81B-90055647!pb-smtp1.pobox.com
Content-Transfer-Encoding: quoted-printable

For the most cases arch_vcpu_ioreq_completion() routine is just an empty =
stub,
except when handling VIO_realmode_completion, which only happens on HVM
domains running on VT-x machine. When VT-x is disabled in build configura=
tion,
both x86 & arm version of routine become empty stubs.
To dispose of these useless stubs we can do optional call to arch-specifi=
c
ioreq completion handler, if it's present, and drop arm and generic x86 h=
andlers.
Actual handling of VIO_realmore_completion can be done by VMX code then.

Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
---
 xen/arch/arm/ioreq.c       |  6 ------
 xen/arch/x86/hvm/ioreq.c   | 25 -------------------------
 xen/arch/x86/hvm/vmx/vmx.c | 15 +++++++++++++++
 xen/common/ioreq.c         |  5 ++++-
 xen/include/xen/ioreq.h    |  2 +-
 5 files changed, 20 insertions(+), 33 deletions(-)

diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
index 5df755b48b..2e829d2e7f 100644
--- a/xen/arch/arm/ioreq.c
+++ b/xen/arch/arm/ioreq.c
@@ -135,12 +135,6 @@ bool arch_ioreq_complete_mmio(void)
     return false;
 }
=20
-bool arch_vcpu_ioreq_completion(enum vio_completion completion)
-{
-    ASSERT_UNREACHABLE();
-    return true;
-}
-
 /*
  * The "legacy" mechanism of mapping magic pages for the IOREQ servers
  * is x86 specific, so the following hooks don't need to be implemented =
on Arm:
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index b37bbd660b..088650e007 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -29,31 +29,6 @@ bool arch_ioreq_complete_mmio(void)
     return handle_mmio();
 }
=20
-bool arch_vcpu_ioreq_completion(enum vio_completion completion)
-{
-    switch ( completion )
-    {
-#ifdef CONFIG_VMX
-    case VIO_realmode_completion:
-    {
-        struct hvm_emulate_ctxt ctxt;
-
-        hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
-        vmx_realmode_emulate_one(&ctxt);
-        hvm_emulate_writeback(&ctxt);
-
-        break;
-    }
-#endif
-
-    default:
-        ASSERT_UNREACHABLE();
-        break;
-    }
-
-    return true;
-}
-
 static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
 {
     struct domain *d =3D s->target;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 8ba996546f..4f9be50fe7 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2749,6 +2749,20 @@ static void cf_check vmx_set_reg(struct vcpu *v, u=
nsigned int reg, uint64_t val)
     vmx_vmcs_exit(v);
 }
=20
+bool realmode_vcpu_ioreq_completion(enum vio_completion completion)
+{
+    struct hvm_emulate_ctxt ctxt;
+
+    if ( completion !=3D VIO_realmode_completion )
+        ASSERT_UNREACHABLE();
+
+    hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
+    vmx_realmode_emulate_one(&ctxt);
+    hvm_emulate_writeback(&ctxt);
+
+    return true;
+}
+
 static struct hvm_function_table __initdata_cf_clobber vmx_function_tabl=
e =3D {
     .name                 =3D "VMX",
     .cpu_up_prepare       =3D vmx_cpu_up_prepare,
@@ -3070,6 +3084,7 @@ const struct hvm_function_table * __init start_vmx(=
void)
     lbr_tsx_fixup_check();
     ler_to_fixup_check();
=20
+    arch_vcpu_ioreq_completion =3D realmode_vcpu_ioreq_completion;
     return &vmx_function_table;
 }
=20
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 1257a3d972..94fde97ece 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -33,6 +33,8 @@
 #include <public/hvm/ioreq.h>
 #include <public/hvm/params.h>
=20
+bool (*arch_vcpu_ioreq_completion)(enum vio_completion completion) =3D N=
ULL;
+
 void ioreq_request_mapcache_invalidate(const struct domain *d)
 {
     struct vcpu *v =3D current;
@@ -244,7 +246,8 @@ bool vcpu_ioreq_handle_completion(struct vcpu *v)
         break;
=20
     default:
-        res =3D arch_vcpu_ioreq_completion(completion);
+        if ( arch_vcpu_ioreq_completion )
+            res =3D arch_vcpu_ioreq_completion(completion);
         break;
     }
=20
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index cd399adf17..880214ec41 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -111,7 +111,7 @@ void ioreq_domain_init(struct domain *d);
 int ioreq_server_dm_op(struct xen_dm_op *op, struct domain *d, bool *con=
st_op);
=20
 bool arch_ioreq_complete_mmio(void);
-bool arch_vcpu_ioreq_completion(enum vio_completion completion);
+extern bool (*arch_vcpu_ioreq_completion)(enum vio_completion completion=
);
 int arch_ioreq_server_map_pages(struct ioreq_server *s);
 void arch_ioreq_server_unmap_pages(struct ioreq_server *s);
 void arch_ioreq_server_enable(struct ioreq_server *s);
--=20
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri May 31 08:26:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:26:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733441.1139689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxau-0003Ic-0V; Fri, 31 May 2024 08:26:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733441.1139689; Fri, 31 May 2024 08:26:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxat-0003IV-Sc; Fri, 31 May 2024 08:26:35 +0000
Received: by outflank-mailman (input) for mailman id 733441;
 Fri, 31 May 2024 08:26:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tcd8=NC=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCxar-0003IP-SQ
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:26:33 +0000
Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com
 [2607:f8b0:4864:20::b31])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7ba7a1cb-1f27-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:26:31 +0200 (CEST)
Received: by mail-yb1-xb31.google.com with SMTP id
 3f1490d57ef6-dfa7ab78ef2so214450276.0
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 01:26:31 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794f317073fsm42967785a.112.2024.05.31.01.26.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 01:26:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ba7a1cb-1f27-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717143990; x=1717748790; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=1a1ryheQen9WxY4LMMuEhi4hk6Cgc8e7s9lJfWj8R7w=;
        b=pXIXmEswpStYk95hwrp27gvsB+h75t2TdW86+8WqLuFiq6qCMRMygcX46qjjvlFAFW
         JtKeZRO1XCTzqWRs1FHeoJ5ujrMPEK8LOX1TBGn9G++zhq887JMK9+4pjTMeUhe1h17H
         XbdIKB4g3iuyRHhWctXGhfIkUepmkpa8Bz+Zk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717143990; x=1717748790;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1a1ryheQen9WxY4LMMuEhi4hk6Cgc8e7s9lJfWj8R7w=;
        b=BrpLdx+DMxGDSU/u4wM+H5g2/n8BoZSlx7hPStQ3cVMiyzjJK6xHgAtCdFYIu0cNhR
         Nf6hwF598DDSxktrwRjei7Us9XNWj9/0PeoXciT8jzMj0xvTe1+E8a6LSG0RFcH9hmLS
         ZUQ36OKpzxMOHfI0K9/46YvSi3gXq8so1+AhmnEARoQGlYSLVJePdP+NxY51A35s8dvp
         6Mj0kRWOUFCnrLdaOwORydxEV6OkoA6LOqE3QalNe6WOpYEMaxcPetXJkoEm18fen9nQ
         KcMBrhM2o9fL6KirOSOIN2RiRC4l6XhVhAjRhcr9VkIWULfcgjMkmQNGHOCviySvSKVc
         TN5w==
X-Forwarded-Encrypted: i=1; AJvYcCXatPyHk1ASxbDvWng6kEGzdCNL1jlLOhclxM6gHRS/Szu1D05SZNd2T7DzM03J3cRWRgwVpa1D4gU6sTe2HGBvQYsxYn8xmWWaCQHvXM4=
X-Gm-Message-State: AOJu0Yw2U28XfMJevj5zary8WN+dcI/MRX9cIEf+POhPgY/3itf6FOcS
	G1S10zi7pITAahrd6vXJnXquTwKRqrhXmPRUshvT+yvWeGFMuLpQEq2URLZ6u3U=
X-Google-Smtp-Source: AGHT+IGpZvp+kzq6US03HRa3BV/hi+1y6p2t+X4cxz9RWf29hfY2+yaYgUWi+NLHAbBgTSDp+h9KOg==
X-Received: by 2002:a25:874e:0:b0:dcd:d94:3db5 with SMTP id 3f1490d57ef6-dfa73dbaf9fmr1212168276.52.1717143990510;
        Fri, 31 May 2024 01:26:30 -0700 (PDT)
Message-ID: <732f5239-c45e-4439-b28a-265e43d46faa@citrix.com>
Date: Fri, 31 May 2024 09:26:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] arch/irq: Make irq_ack_none() mandatory
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
 <20240530184027.44609-2-andrew.cooper3@citrix.com>
 <e5741eb1-1708-463f-8b3a-2d34d6fd4fbc@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <e5741eb1-1708-463f-8b3a-2d34d6fd4fbc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 31/05/2024 7:42 am, Jan Beulich wrote:
> On 30.05.2024 20:40, Andrew Cooper wrote:
>> Any non-stub implementation of these is going to have to do something here.
> For whatever definition of "something", seeing ...
>
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -31,12 +31,12 @@ struct irq_guest
>>      unsigned int virq;
>>  };
>>  
>> -static void ack_none(struct irq_desc *irq)
>> +void irq_ack_none(struct irq_desc *irq)
>>  {
>>      printk("unexpected IRQ trap at irq %02x\n", irq->irq);
>>  }
> ... this, which - perhaps apart from the word "trap" - is entirely Arm-
> independent, and could hence quite well live in a common code fallback
> implementation.

Not really.

On ARM, ack()+end() are both mandatory and it's end() which is taking
the useful action.

On x86, ack() has the effect and end() is optional (and has a different
prototype even!)


>  Nevertheless with patch 2 clearly being an improvement,
> both patches:
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:31:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:31:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733446.1139698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxfX-0004zc-Gh; Fri, 31 May 2024 08:31:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733446.1139698; Fri, 31 May 2024 08:31:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxfX-0004zV-E0; Fri, 31 May 2024 08:31:23 +0000
Received: by outflank-mailman (input) for mailman id 733446;
 Fri, 31 May 2024 08:31:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCxfW-0004zP-JR
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:31:22 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2811b04e-1f28-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:31:20 +0200 (CEST)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-5751bcb3139so2049117a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 01:31:20 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4fb4d7f45d1cf-57a31b9919asm752375a12.8.2024.05.31.01.31.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 01:31:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2811b04e-1f28-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717144280; x=1717749080; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YcLzLE4Z3E7kIRjSBw2I5srCpX7gYlmZMZ7dNTS8rFo=;
        b=L0RY7sKeizT+jTGQUVgNIf5czWI1Y7X7Ei6HPrPXoMp8hgmzVPHu2xYwAmcJd2cELn
         9Sbfr/OpCljxBq/1AtUA1hEy4+KDEDI5ADRot8BZNRa4R2RNHW2hhJl6t2Eg7Gkkm2YC
         K111SrbGIgkyCUME7CSVkwbnLNDg3nJn2CHRBkjYUxvtsbIoa6/SnjczAdZZAKBmsZKN
         HjZZTtQaUrJd7sKq6PnBiZUI+qTR/I53zeKAUX/j62/dRvKbLbOZRDx/mgM1X2ozfma/
         ojR8N+Id5GDyb1U07rX+WZKwxvRh/cZIerdUg2RPOx8S6HX5h0JdmzhvlFdnZi9vLF17
         8IYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717144280; x=1717749080;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YcLzLE4Z3E7kIRjSBw2I5srCpX7gYlmZMZ7dNTS8rFo=;
        b=fzRoGKxIvMeVnkOZXr9kB9ToBuactIULVTGVn24aFqgZg/dttzPY+2UU8Am/j8DVo3
         JdQj6f2/QguP/3YFUnN0Y33niKU/fs0CJydP07y6/aUhc5asrOYoagHAnYDzjjogu08c
         U6GbNVDFoXSBuapj3lXv5RiPFJ3xclnybzaSqNxWvQmGA2WjwLkHTkS8f/jgAEy1AXnN
         3I3b11+XdjvRN0TCBpZD7//RksG/TJSkhVB0csDXwDxvuA4dVHRNzhMlkQ/IINjvWbLZ
         tn8Le1OSTRX5FmUsN9KALUwp0cfaOlVmJpUiuB88uLnVlD7yZ+Mp9u1cqvnPiSNPmaJI
         uxWw==
X-Forwarded-Encrypted: i=1; AJvYcCXBHEW3xCu1MqY8rQyscnx/RPowPTTDSbFJYr3si2vengE8zNJuMur1g7OnlcESv0bRXZb4NpMILwf9EsO2o7XxDNNNkXTWde5in/0vdDg=
X-Gm-Message-State: AOJu0Yxfgrpo6ndhAOomzfwmwppKdSDoPcnSIDzingzQn74nxrTxLYXa
	gO2oZ7p4GD9QasWaiGPg/wNMU2ssAJLOJrJvokY7gyfRTRFHWrzzqU345ESXQA==
X-Google-Smtp-Source: AGHT+IGe1H042Qk81yAjhw0hO6pCk0VdNukNepwUtcnMubcwkZgxMNZMfV8M3GTOX6KYJGCGONHtnw==
X-Received: by 2002:a50:f687:0:b0:57a:2e7c:1d88 with SMTP id 4fb4d7f45d1cf-57a3638fb05mr916867a12.20.1717144279894;
        Fri, 31 May 2024 01:31:19 -0700 (PDT)
Message-ID: <e9e11f6c-5da0-4121-b716-041fcda9d3b2@suse.com>
Date: Fri, 31 May 2024 10:31:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 13/15] x86/ioreq: guard VIO_realmode_completion
 with CONFIG_VMX
To: Sergiy Kibrik <sergiy_kibrik@epam.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xenia Ragiadakou <xenia.ragiadakou@amd.com>, xen-devel@lists.xenproject.org
References: <cover.1715761386.git.Sergiy_Kibrik@epam.com>
 <9e64fa33b298f789d8340cf1046a9fbf683dd2b7.1715761386.git.Sergiy_Kibrik@epam.com>
 <376d2e89-da6d-49c2-90aa-aab4af6eb0d1@suse.com>
 <a38ac7e4-cf47-40a4-b76c-6752237b1ccc@epam.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a38ac7e4-cf47-40a4-b76c-6752237b1ccc@epam.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 31.05.2024 10:05, Sergiy Kibrik wrote:
> 16.05.24 15:11, Jan Beulich:
>> On 15.05.2024 11:24, Sergiy Kibrik wrote:
>>> --- a/xen/arch/x86/hvm/emulate.c
>>> +++ b/xen/arch/x86/hvm/emulate.c
>>> @@ -2667,7 +2667,9 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
>>>           break;
>>>   
>>>       case VIO_mmio_completion:
>>> +#ifdef CONFIG_VMX
>>>       case VIO_realmode_completion:
>>> +#endif
>>>           BUILD_BUG_ON(sizeof(hvio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
>>>           hvio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
>>>           memcpy(hvio->mmio_insn, hvmemul_ctxt->insn_buf, hvio->mmio_insn_bytes);
>>
>> This change doesn't buy us anything, does it?
> 
> why not? Code won't compile w/o it.
> Or do you mean hiding the whole VIO_realmode_completion enum under 
> CONFIG_VMX wasn't really useful?

That's what I meant, by implication. To me it's extra #ifdef-ary without
real gain.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:34:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:34:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733449.1139708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxi7-0005ko-Sp; Fri, 31 May 2024 08:34:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733449.1139708; Fri, 31 May 2024 08:34:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxi7-0005kh-QG; Fri, 31 May 2024 08:34:03 +0000
Received: by outflank-mailman (input) for mailman id 733449;
 Fri, 31 May 2024 08:34:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fjOV=NC=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1sCxi7-0005kb-6e
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:34:03 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 87c6d5f5-1f28-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:34:01 +0200 (CEST)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a5dcb5a0db4so187108666b.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 01:34:01 -0700 (PDT)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a67eb6214fcsm61579066b.224.2024.05.31.01.33.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 01:34:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87c6d5f5-1f28-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1717144440; x=1717749240; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Fv7P1b8LgUkd769Ez3yba/ErGOX7JkgtzCjTpafeiRU=;
        b=VGsN0GHrWjfvaTHxlowGBeZwy8kpbxUmvNuyy3vqs+uVu0LAnjM3wxlMp40aIw861l
         hrM/A+kMXvpH0xgzm1yfAMTDN3QhFnoeYLBNpamLo/0Hv+EnpDSJ/y4rqjkTEOlgRbIi
         +Zr5nWLPUnMVXVSUnT4XONIU+IcxEK8FJqNs0eno+WOtm1r8h90izbMCpvWdokZ0IwBH
         5KdY9KBBd2cKeGbBZLa6ar0R6EGeFY8GpsPVGb+jAbhFCOvKKBlERFa2gJwePOz8VR3w
         cDspOSszAVKy4eLmsZ+GV0vMRwcVtYFAu5W5g0gCXcIxOoOZf2z097ZhNwkjrdo7Vutk
         b9JA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717144440; x=1717749240;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Fv7P1b8LgUkd769Ez3yba/ErGOX7JkgtzCjTpafeiRU=;
        b=eHTY1TMO3XKiPhr2b83d366GrpLrXCGu+8LC8Id9A8ZK/kxuzjO655rLONU3XpY87P
         NkKGVbo3Y6fFwDLsG6hZgqF70xZlJjZ6UUYZS+m/g+87LB4/BzfLvuFx3ab9/BdHFbXc
         CpcCv6OH+e0kJ7mg4jTwyT2cfhmj3kRPR5SuAjY/mJfkMDTX5/BaWpVY4hiMbLK7ANIS
         nePLI7qHMwXvnvzGN2xxwYh/CDqJ64KcjE9otXt82MmFPlFDaD0pz4Ry8f4c+61CQNig
         GQKuPzgoES8XB7da8fm4x8rppq5eWP2fWnQgLtbskvYFVwgsmYyU7cqo+WyV6VV0UuHi
         asGw==
X-Forwarded-Encrypted: i=1; AJvYcCVlwEeyfQOlAhAeLJVlqhU1X1g51/9UNX4ySD1lNyVz1noICmnkuiAFiB+CFG+Hmb61nn3I+yCz+kpIA/Q0q8s1zQvPj1FYSAiII8m7ILg=
X-Gm-Message-State: AOJu0YzanB14sSVftOoCDcM95M7NKTWkCMeRTQVVDD8G6J4URjjJNuYF
	Xlhd9iBxP9KKvji++zb+zVAQ0LQRMmt7nLnphro0FkT5/VR++PCt28xn2kpnsg==
X-Google-Smtp-Source: AGHT+IEM4v+DjbOY8sH4CpBxHv0Ogs9ljDq3RaZTRruHWVxL/qOdiwzM80y+HIOp1mcerEDi+/1CZQ==
X-Received: by 2002:a17:906:600a:b0:a66:7b79:3572 with SMTP id a640c23a62f3a-a681fc5bfb9mr68101466b.15.1717144440367;
        Fri, 31 May 2024 01:34:00 -0700 (PDT)
Message-ID: <e39c9e0c-2596-483f-a82d-86d424717081@suse.com>
Date: Fri, 31 May 2024 10:33:58 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns false
 if CPU operations are underway
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com> <ZldDvH8GhhQcu5NX@macbook>
 <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com> <ZldUgJRJvytRsjhu@macbook>
 <075168de-1a3f-4f23-a445-8ba74082fb8f@suse.com> <Zll8xQJiM83EZhTd@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zll8xQJiM83EZhTd@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 31.05.2024 09:31, Roger Pau Monné wrote:
> On Fri, May 31, 2024 at 09:02:20AM +0200, Jan Beulich wrote:
>> On 29.05.2024 18:14, Roger Pau Monné wrote:
>>> On Wed, May 29, 2024 at 05:49:48PM +0200, Jan Beulich wrote:
>>>> On 29.05.2024 17:03, Roger Pau Monné wrote:
>>>>> On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
>>>>>> On 29.05.2024 11:01, Roger Pau Monne wrote:
>>>>>>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
>>>>>>> a cpu_hotplug_{begin,done}() region the function will still return success,
>>>>>>> because a CPU taking the rwlock in read mode after having taken it in write
>>>>>>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
>>>>>>> as it should always return false when called with a CPU hot{,un}plug operation
>>>>>>> is in progress.
>>>>>>
>>>>>> I'm not sure I can agree with this. The CPU doing said operation ought to be
>>>>>> aware of what it is itself doing. And all other CPUs will get back false from
>>>>>> get_cpu_maps().
>>>>>
>>>>> Well, the CPU is aware in the context of cpu_{up,down}(), but not in
>>>>> the interrupts that might be handled while that operation is in
>>>>> progress, see below for a concrete example.
>>>>>
>>>>>>>  Otherwise the logic in send_IPI_mask() for example is wrong,
>>>>>>> as it could decide to use the shorthand even when a CPU operation is in
>>>>>>> progress.
>>>>>>
>>>>>> It's also not becoming clear what's wrong there: As long as a CPU isn't
>>>>>> offline enough to not be in cpu_online_map anymore, it may well need to still
>>>>>> be the target of IPIs, and targeting it with a shorthand then is still fine.
>>>>>
>>>>> The issue is in the online path: there's a window where the CPU is
>>>>> online (and the lapic active), but cpu_online_map hasn't been updated
>>>>> yet.  A specific example would be time_calibration() being executed on
>>>>> the CPU that is running cpu_up().  That could result in a shorthand
>>>>> IPI being used, but the mask in r.cpu_calibration_map not containing
>>>>> the CPU that's being brought up online because it's not yet added to
>>>>> cpu_online_map.  Then the number of CPUs actually running
>>>>> time_calibration_rendezvous_fn won't match the weight of the cpumask
>>>>> in r.cpu_calibration_map.
>>>>
>>>> I see, but maybe only partly. Prior to the CPU having its bit set in
>>>> cpu_online_map, can it really take interrupts already? Shouldn't it be
>>>> running with IRQs off until later, thus preventing it from making it
>>>> into the rendezvous function in the first place? But yes, I can see
>>>> how the IRQ (IPI) then being delivered later (once IRQs are enabled)
>>>> might cause problems, too.
>>>
>>> The interrupt will get set in IRR and handled when interrupts are
>>> enabled.
>>>
>>>>
>>>> Plus, with how the rendezvous function is invoked (via
>>>> on_selected_cpus() with the mask copied from cpu_online_map), the
>>>> first check in smp_call_function_interrupt() ought to prevent the
>>>> function from being called on the CPU being onlined. A problem would
>>>> arise though if the IPI arrived later and call_data was already
>>>> (partly or fully) overwritten with the next request.
>>>
>>> Yeah, there's a small window where the fields in call_data are out of
>>> sync.
>>>
>>>>>> In any event this would again affect only the CPU leading the CPU operation,
>>>>>> which should clearly know at which point(s) it is okay to send IPIs. Are we
>>>>>> actually sending any IPIs from within CPU-online or CPU-offline paths?
>>>>>
>>>>> Yes, I've seen the time rendezvous happening while in the middle of a
>>>>> hotplug operation, and the CPU coordinating the rendezvous being the
>>>>> one doing the CPU hotplug operation, so get_cpu_maps() returning true.
>>>>
>>>> Right, yet together with ...
>>>>
>>>>>> Together with the earlier paragraph the critical window would be between the
>>>>>> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
>>>>>> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
>>>>>> then the question would be what bad, if any, would happen to that CPU if an
>>>>>> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
>>>>>> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
>>>>>>
>>>>>>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
>>>>>>> already hold in write mode by the current CPU, as read_trylock() would
>>>>>>> otherwise return true.
>>>>>>>
>>>>>>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
>>>>>>
>>>>>> I'm puzzled by this as well: Prior to that and the change referenced by its
>>>>>> Fixes: tag, recursive spin locks were used. For the purposes here that's the
>>>>>> same as permitting read locking even when the write lock is already held by
>>>>>> the local CPU.
>>>>>
>>>>> I see, so the Fixes should be:
>>>>>
>>>>> x86/smp: use APIC ALLBUT destination shorthand when possible
>>>>>
>>>>> Instead, which is the commit that started using get_cpu_maps() in
>>>>> send_IPI_mask().
>>>>
>>>> ... this I then wonder whether it's really only the condition in
>>>> send_IPI_mask() which needs further amending, rather than fiddling with
>>>> get_cpu_maps().
>>>
>>> That the other option, but I have impression it's more fragile to
>>> adjust the condition in send_IPI_mask() rather than fiddle with
>>> get_cpu_maps().
>>>
>>> However if that's the preference I can adjust.
>>
>> I guess we need other REST input here then. The two of us clearly disagree on
>> what use of get_cpu_maps() is meant to guarantee. And I deem fiddling with
>> common code here more risky (and more intrusive - the other change would be
>> a single-line code change afaict, plus extending the related comment).
> 
> How do you envision that other change to be done?  Adding an extra
> variable and toggling it in cpu_hotplug_{begin,done}() to signal
> whether a CPU hotplug is in progress?

I was thinking of an is-write-locked-by-me check on cpu_add_remove_lock.

Jan


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:34:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:34:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733454.1139719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxik-0006Hm-7T; Fri, 31 May 2024 08:34:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733454.1139719; Fri, 31 May 2024 08:34:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxik-0006Hf-4V; Fri, 31 May 2024 08:34:42 +0000
Received: by outflank-mailman (input) for mailman id 733454;
 Fri, 31 May 2024 08:34:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tcd8=NC=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCxij-0005kb-BP
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:34:41 +0000
Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com
 [2607:f8b0:4864:20::735])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e852ede-1f28-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:34:39 +0200 (CEST)
Received: by mail-qk1-x735.google.com with SMTP id
 af79cd13be357-794ba2d4579so120960885a.1
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 01:34:39 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6ae4b406549sm4912126d6.100.2024.05.31.01.34.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 01:34:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e852ede-1f28-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717144478; x=1717749278; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=LKiAmvQOzZQsFP46IA5w5fikv5CZ3YmraqhDhLdCOpA=;
        b=FC3Wntfx+3ZSZZnGC1zJ5r+nmwU7bIU7SSHe9uO/hta05OORiUWMDmyu/wwMfBtmKA
         p6mDKAwOQzp32ETPF+lGqzSR/NeMLSMzvIdw9sjgUnoUcIuuug5xo4paBpe53rTRRm58
         8aNnvoGYUWprYL8h70W2X8ZuPzEH60iBLIe8I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717144478; x=1717749278;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=LKiAmvQOzZQsFP46IA5w5fikv5CZ3YmraqhDhLdCOpA=;
        b=cidwgu9T4kaUX+8VNuIQvDPXfX3bBMtw+HN6AsEvvmXbOyAfX3C2AoYFl1rql0b/jB
         S5O5InDaksyL7Z0og4xk+6UD8NaChOwECP+I9Noau7eOfTyqv2Oasz8LeulpMAA9b62D
         SiZhlviLa7O+sov6bC9l4LSn8PfQCfl2ij1hkbFlPyJcHVRbk618xea6097MIeC65KZA
         jo5EXAuxJolA3TmN1cGn+ZSdbFvLqE5DzjsgF2pkNWzs9tc7BeGAeZ3ZwKKW2Lc5PqZU
         jS0fqMdyVQieIMxmI+nViiyCFRbYJkZ1Ordc6s0gnVP9xm4Q0oJO11t7bMRNNIzLE1GV
         auWA==
X-Gm-Message-State: AOJu0Yx450AbUqnHxCcGTG9Ce1ShtBfLb4jfrXtU+AXDczSVyo/mZn+i
	slUqQF5A1bjfpOXvUpL+ePASLoirj1yDPXhetbcgVyXO7fg7hOsEL+g6YIVgFao=
X-Google-Smtp-Source: AGHT+IEp9JXxcYkL+cimxVDP84hYyA+XZHmhbqDGZmF0iKYR+DvfQkRt3PirLZbxciH03wf6Kw2wTA==
X-Received: by 2002:a05:6214:449b:b0:6ae:4dea:6c69 with SMTP id 6a1803df08f44-6aecd6ffdbbmr13062246d6.63.1717144478524;
        Fri, 31 May 2024 01:34:38 -0700 (PDT)
Message-ID: <2917e122-51d8-4caf-ba70-52da70d1342a@citrix.com>
Date: Fri, 31 May 2024 09:34:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-7-andrew.cooper3@citrix.com>
 <alpine.DEB.2.22.394.2405301809170.2557291@ubuntu-linux-20-04-desktop>
 <7b974b36b89c216379b86170af9de451@bugseng.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7b974b36b89c216379b86170af9de451@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 31/05/2024 7:56 am, Nicola Vetrini wrote:
> On 2024-05-31 03:14, Stefano Stabellini wrote:
>> On Fri, 24 May 2024, Andrew Cooper wrote:
>>> Perform constant-folding unconditionally, rather than having it
>>> implemented
>>> inconsistency between architectures.
>>>
>>> Confirm the expected behaviour with compile time and boot time tests.
>>>
>>> For non-constant inputs, use arch_ffs() if provided but fall back to
>>> generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin
>>> that
>>> works in all configurations.
>>>
>>> For x86, rename ffs() to arch_ffs() and adjust the prototype.
>>>
>>> For PPC, __builtin_ctz() is 1/3 of the size of size of the transform to
>>> generic_fls().  Drop the definition entirely.  ARM too benefits in
>>> the general
>>> case by using __builtin_ctz(), but less dramatically because it using
>>> optimised asm().
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>
>> This patch made me realize that we should add __builtin_ctz,
>> __builtin_constant_p and always_inline to
>> docs/misra/C-language-toolchain.rst as they don't seem to be currently
>> documented and they are not part of the C standard
>>
>> Patch welcome :-)
>>
>
> I can send a patch for the builtins.

That's very kind of you.

In total by the end of this series, we've got __builtin_constant_p() 
(definitely used elsewhere already), and __builtin_{ffs,ctz,clz}{,l}() 
(3x primitives, 2x input types).

If we're going for a list of the primitive operations, lets add
__builtin_popcnt{,l}() too right away, because if it weren't for 4.19
code freeze, I'd have cleaned up the hweight() helpers too.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:47:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:47:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733463.1139730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxvR-0008Tr-Cq; Fri, 31 May 2024 08:47:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733463.1139730; Fri, 31 May 2024 08:47:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxvR-0008Tk-8U; Fri, 31 May 2024 08:47:49 +0000
Received: by outflank-mailman (input) for mailman id 733463;
 Fri, 31 May 2024 08:47:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCxvP-0008Ta-EG; Fri, 31 May 2024 08:47:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCxvO-0003jd-Sc; Fri, 31 May 2024 08:47:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCxvO-0007dX-Gn; Fri, 31 May 2024 08:47:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCxvO-0006KB-GK; Fri, 31 May 2024 08:47:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=62C/O5z5hrZIzMzfuOKZRYV3Yy2QJziBbVzikL1yOGE=; b=dqRf9ISBuQVaEU3kuBXpeSkYMv
	s051l879rOisNKri5p6UDUmfbBp72tpfpjeJjhJFMFMzKrtWnT8nSLFVS9la2pIizPYG3BCHb1Nuz
	cjKeg8K/Yvf4dm3Wryd0TRTIMFwZ4TinthmJOXFS0OqTfr8Vc9xnvkCpGdsTy0NgQnSI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186202-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 186202: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4a4be1ad3a6efea16c56615f31117590fd881358
X-Osstest-Versions-That:
    linux=e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 08:47:46 +0000

flight 186202 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186202/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-raw       8 xen-boot                     fail  like 186174
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186174
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186174
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186174
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4a4be1ad3a6efea16c56615f31117590fd881358
baseline version:
 linux                e0cce98fe279b64f4a7d81b7f5c3a23d80b92fbc

Last test of basis   186174  2024-05-28 18:10:31 Z    2 days
Testing same since   186185  2024-05-29 17:42:32 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dominique Martinet <asmadeus@codewreck.org>
  Herbert Xu <herbert@gondor.apana.org.au>
  Linus Torvalds <torvalds@linux-foundation.org>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   e0cce98fe279..4a4be1ad3a6e  4a4be1ad3a6efea16c56615f31117590fd881358 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri May 31 08:48:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 08:48:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733467.1139739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxvo-0000VI-L1; Fri, 31 May 2024 08:48:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733467.1139739; Fri, 31 May 2024 08:48:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCxvo-0000VB-Gx; Fri, 31 May 2024 08:48:12 +0000
Received: by outflank-mailman (input) for mailman id 733467;
 Fri, 31 May 2024 08:48:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tcd8=NC=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sCxvm-0000TX-Px
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 08:48:10 +0000
Received: from mail-vk1-xa32.google.com (mail-vk1-xa32.google.com
 [2607:f8b0:4864:20::a32])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 803e95be-1f2a-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 10:48:07 +0200 (CEST)
Received: by mail-vk1-xa32.google.com with SMTP id
 71dfb90a1353d-4e1c721c040so632905e0c.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 01:48:07 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794f2e519bdsm45059785a.0.2024.05.31.01.48.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 01:48:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 803e95be-1f2a-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717145287; x=1717750087; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=iyBk0SEk1iT3MmDI59PUcjBu+s31PdxzRAU73P5bf2U=;
        b=tn+6hOc1bmkz6ih8kYQDiInYqeWqw2e383MylOOKKcj7uCZz0zJKzvp87HMDqRDq8x
         pDI70eXhzXaevKXFJicB+oDqheB3F9Tz/fj0CcwgmbcnVMDdwzl6zey/j7q8BKuf3UJ8
         aTzk5T0x5Sok1bEVN3380YPPp9oH+tWb+B+Sg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717145287; x=1717750087;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=iyBk0SEk1iT3MmDI59PUcjBu+s31PdxzRAU73P5bf2U=;
        b=cCRqwNrre7/Z3HxakOAKqInR8D9uVaaWwuPVbzsk+ytt15KfM4696EYG3i60jz9ITN
         iC/tgKdJN/6HX8OArSfoe7JPhvozj/8dMnzwc+y8tqxXxhYLj9nn6MBrTO5ssX+57dIs
         DrdqdRQTz3XOwBpnA8pZtKbx9TVEh2V4wwi7QG3d29O9r3uzJYbS5Afsh6sPpKNV9vcM
         YHtbg9LOi3PsbXaYef/Cl7ZEKM1Sb7mW6lni41ib9ZAJ0IaprxzY5U5iNRnyyUJ3Zqeh
         qbTHT3i4H283984sTsizt2RIBpwlgbtAYTxAWCle7CWo8dcf+sYt5LMSLQryPAkoR4wE
         HJeQ==
X-Gm-Message-State: AOJu0YwqTfgjp3+VPM4ao92ryCK9E1mDaK+mH9ilz2ODxZEXXoYognSi
	VPlzN5WU6nsNtK5OXB1e24m7F/+8h8Q1HHX0bdgCAef/3IJ3zzJ4+I43aac2ufY=
X-Google-Smtp-Source: AGHT+IGOdwtoan576X8wy5jkS+8Seuy1wz66gax1o6n3YYLQjdPD4LH7phll1qqxFdR96Ms8lFrZbw==
X-Received: by 2002:a1f:f24b:0:b0:4e4:e8a7:2620 with SMTP id 71dfb90a1353d-4eb02ede2aemr1211079e0c.14.1717145285071;
        Fri, 31 May 2024 01:48:05 -0700 (PDT)
Message-ID: <f7ea72c3-45ef-43cb-ab57-b375a4fbc683@citrix.com>
Date: Fri, 31 May 2024 09:48:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 06/13] xen/bitops: Implement ffs() in common logic
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-7-andrew.cooper3@citrix.com>
 <alpine.DEB.2.22.394.2405301809170.2557291@ubuntu-linux-20-04-desktop>
 <7b974b36b89c216379b86170af9de451@bugseng.com>
 <2917e122-51d8-4caf-ba70-52da70d1342a@citrix.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <2917e122-51d8-4caf-ba70-52da70d1342a@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 31/05/2024 9:34 am, Andrew Cooper wrote:
> On 31/05/2024 7:56 am, Nicola Vetrini wrote:
>> On 2024-05-31 03:14, Stefano Stabellini wrote:
>>> On Fri, 24 May 2024, Andrew Cooper wrote:
>>>> Perform constant-folding unconditionally, rather than having it
>>>> implemented
>>>> inconsistency between architectures.
>>>>
>>>> Confirm the expected behaviour with compile time and boot time tests.
>>>>
>>>> For non-constant inputs, use arch_ffs() if provided but fall back to
>>>> generic_ffsl() if not.  In particular, RISC-V doesn't have a builtin
>>>> that
>>>> works in all configurations.
>>>>
>>>> For x86, rename ffs() to arch_ffs() and adjust the prototype.
>>>>
>>>> For PPC, __builtin_ctz() is 1/3 of the size of size of the transform to
>>>> generic_fls().  Drop the definition entirely.  ARM too benefits in
>>>> the general
>>>> case by using __builtin_ctz(), but less dramatically because it using
>>>> optimised asm().
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> This patch made me realize that we should add __builtin_ctz,
>>> __builtin_constant_p and always_inline to
>>> docs/misra/C-language-toolchain.rst as they don't seem to be currently
>>> documented and they are not part of the C standard
>>>
>>> Patch welcome :-)
>>>
>> I can send a patch for the builtins.
> That's very kind of you.
>
> In total by the end of this series, we've got __builtin_constant_p() 
> (definitely used elsewhere already), and __builtin_{ffs,ctz,clz}{,l}() 
> (3x primitives, 2x input types).
>
> If we're going for a list of the primitive operations, lets add
> __builtin_popcnt{,l}() too right away, because if it weren't for 4.19
> code freeze, I'd have cleaned up the hweight() helpers too.

Oh, and it's worth noting that __builtin_{ctz,clz}{,l}() have explicit
UB if given an input of 0.  (Sadly, even on architectures where the
underlying instruction emitted is safe with a 0 input. [0])

This is why every patch in the series using them checks for nonzero input.

UBSAN (with an adequate compiler) will instrument this, and Xen has
__ubsan_handle_invalid_builtin() to diagnose these.

~Andrew

[0] It turns out that Clang has a 2-argument form of the builtin with
the second being the "value forwarded" in case the first is 0.  I've not
investigated whether GCC has the same.


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:09:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:09:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733484.1139748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyFx-0004Eo-EC; Fri, 31 May 2024 09:09:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733484.1139748; Fri, 31 May 2024 09:09:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyFx-0004Eh-B9; Fri, 31 May 2024 09:09:01 +0000
Received: by outflank-mailman (input) for mailman id 733484;
 Fri, 31 May 2024 09:09:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCyFw-0004Eb-D0
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:09:00 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a8dcdfe-1f2d-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:08:59 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-420180b5922so23362575e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:08:59 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212b85c628sm18926505e9.25.2024.05.31.02.08.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:08:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a8dcdfe-1f2d-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717146539; x=1717751339; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=X7W3xI7fAUh+NEJTD6wipxw0E8s8MOWy9REBP0vU0L0=;
        b=OJERIiF7+Z/CCnyFtoAuXgDF8DUrpd1QHwsvwSfewNSO37PPDxWvul7Jt/r2zXkSvj
         tImuDTTEXAFlBTlvdoEg5tubnP2ilKbUq1yvya4yagg8cL4389BtQVKI3Bq4N5awT0qi
         D+Tu2pUNmk68sYabwAMd8Da3bzImJ7Ru2Gm3HY+Z/Y5n7NaeYA2Rb75fK7mzD4XCni/v
         aCMKQ+rmLv+AhGBBdVDV+gnMEgjgt98N6CbJBhB3ImkS3WQIzrkSCynVTkqfPQvLqnnQ
         J8BdZMTKUibQP/mzDkByaD4w/rSHZhp5yHy0t/EaXZ44CCB2z4Ufd6mH42fg6fMGaNkU
         ejog==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146539; x=1717751339;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=X7W3xI7fAUh+NEJTD6wipxw0E8s8MOWy9REBP0vU0L0=;
        b=eeJjnrr3uPwlkL8TUsV9/uxcHQ+bayr57Mo3Mhq0x25zA0HVnFbNfXDetjoLy2uYIS
         NkXDm2oIGCP38FhpUe+cxqQNe5Xw0hNFDDN+r4cjp5Z6YRYu0VXGrPFHQQxCpwX5Pmpx
         ISYmumy4eHENxSaJmAr9M9bQncLSuZlwdfp861zkB6dc9aXBO0Q9JauYI3s81EZPVBmm
         s6WvQtQBAvQZbmMi4KWH5ck5y7f97nUqbbqiglwkuO97ZGPJj38ngkr4CubZZJYljgdR
         33gSBP233sIYYiT1GGNj4iizGynKjULdKtE2sGGMNbZFg37G//tOn3INaOo4vn8qX3Di
         pGEQ==
X-Forwarded-Encrypted: i=1; AJvYcCX5O4u8UdFOG7N7iyphjEYO8p7UehwqWmeq4KhxCj4ySlvpnEPTXjUWzwzSfmhOt9yCkqu6dMLPJ8o+FV6OopBFYbpzjG2YqCWJQiPCFCs=
X-Gm-Message-State: AOJu0YxAEYZ0uwc8lBFfI6zhgg7q+bqeW5EfA/Sq4hdOOJd9c2YOFoHu
	nRjonG8ahbWxMNYn602k7GwsRMEjgIS40aFZjWop3/NIhciBCib3
X-Google-Smtp-Source: AGHT+IHbCCui6ZKB/GXsi4frRAPCbIqN2s04w/9+nqMtxHLIvrzvqQlWWxzQN+FYwKi/P3fUX1a+oQ==
X-Received: by 2002:a05:600c:3106:b0:420:28c7:7260 with SMTP id 5b1f17b1804b1-4212e075761mr12383245e9.21.1717146538559;
        Fri, 31 May 2024 02:08:58 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <fb3dca59-bacf-4457-b1cc-d751750de161@xen.org>
Date: Fri, 31 May 2024 10:08:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 1/7] hw/xen: Remove declarations left over in
 'xen-legacy-backend.h'
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-2-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-2-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> 'xen_blkdev_ops' was removed in commit 19f87870ba ("xen: remove
> the legacy 'xen_disk' backend"), 'xen_netdev_ops' in commit
> 25967ff69f ("hw/xen: update Xen PV NIC to XenDevice model") and
> 'xen_console_ops' in commit 9b77374690 ("hw/xen: update Xen
> console to XenDevice model"). Remove them.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 3 ---
>   1 file changed, 3 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:09:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:09:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733485.1139758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyFz-0004T5-Kk; Fri, 31 May 2024 09:09:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733485.1139758; Fri, 31 May 2024 09:09:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyFz-0004Sy-Hs; Fri, 31 May 2024 09:09:03 +0000
Received: by outflank-mailman (input) for mailman id 733485;
 Fri, 31 May 2024 09:09:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kOju=NC=gmail.com=idryomov@srs-se1.protection.inumbo.net>)
 id 1sCyFy-0004Eb-QW
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:09:02 +0000
Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com
 [2001:4860:4864:20::29])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6b8bc4cd-1f2d-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:09:02 +0200 (CEST)
Received: by mail-oa1-x29.google.com with SMTP id
 586e51a60fabf-25076dde95eso532008fac.0
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:09:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b8bc4cd-1f2d-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717146540; x=1717751340; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=V7TiTdx/nEWXXgHHbHN78B+YsjlKDnX0w9LWVG0G2x4=;
        b=Olb7esvQJJdgWc1w+jurUljBIZwYghFM0dXmiSAiXDrmpFc1FNFW6+rsNIV5P3G38U
         3tIwJ3u4jm6VnzUORWtA2VthKxsHtYsNlj+lgwtvIAxZ+99asQ8IuexrYTCLPHnoBcVF
         6uGy7jtE9qblLc8Xwu3JCG7s1kNrHkQcaTwc39ksp0eVPgsx7+mKxi2P9gYy7m3+pv8t
         /o3mVXmzWQk6i/dOsd+yatAleBuLkzJvN/oRHkX/AiGgdYjmoen1CTlqAWKRI0qjBSkr
         70SHcL4ZhDsTgmdpO+ZMG/xHf6/YsS4S9GxMYrfyVCnRzB/O0Wlxo8zjHy4W5iG4ZQXS
         9otg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146540; x=1717751340;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=V7TiTdx/nEWXXgHHbHN78B+YsjlKDnX0w9LWVG0G2x4=;
        b=cIXHJ4W2y4Gl8eDil0eCqFO8MpUNoe7+k9NldqBA2Q1G00e9yp75m+3Dz62zCdulme
         G/nqkO4Sgyi9tgQLAJlhlsCcqu0M9UpFu/EMuYawetdZedgYtyJAMij8/yR4OK/kJkut
         vqZ11eD/VkMDZx8gnnZHswBEibdO6yJB4EEzP+/aWavi/rfifZxRITYYHhWvsUOHuVwL
         57KAOZ9JDnxFVxZcHbEc6J3JdKnIZ7uJgPLlo1dK0ekWoCumVR5/Fv+Dt2FqCsjJaHg+
         xL4zLB7qhm4MzwJwJzGemS0FbWm3FV9+bBt5fdNChqagysDMGEZXOLKJLaHmq+R+ft0H
         avuQ==
X-Forwarded-Encrypted: i=1; AJvYcCXsAYQYKMaVcI+XmCSjV5/wzz7QWD7uTje/gsTqIHI4XUq1ZFZYYIWRtz9nbtltmlJJf+gGDTqL09sAoedqBaIgWdaQx92pkpqLaUf906M=
X-Gm-Message-State: AOJu0Yx6HADRbyU9ZvHsdUL7FSmc5n+2oITbcrUSSzI6ZkxbiMUe0LHZ
	xZ50vSTgRtdGz32Yo1Mz169TPCVpSyhAjyq10s+lAbvtOHDdQ85fQYD+L/lR1f1wJfLMrz1MLUi
	6wuBtYCWfbV+UCJI+sbPiOfhFmSs=
X-Google-Smtp-Source: AGHT+IHoKiJiwn0RUfGlBR1LOXIQjJEAP/TieA174CPldpRPuDqgZrspO+vxp9UsllxKnQz9RUjusbGg9aknaFlS5Ec=
X-Received: by 2002:a05:6870:9346:b0:250:6422:86a with SMTP id
 586e51a60fabf-25065b70997mr1985845fac.10.1717146540364; Fri, 31 May 2024
 02:09:00 -0700 (PDT)
MIME-Version: 1.0
References: <20240531074837.1648501-1-hch@lst.de> <20240531074837.1648501-4-hch@lst.de>
In-Reply-To: <20240531074837.1648501-4-hch@lst.de>
From: Ilya Dryomov <idryomov@gmail.com>
Date: Fri, 31 May 2024 11:08:48 +0200
Message-ID: <CAOi1vP_mY-a7aiWod-eVz8xuhGz4mHBBoZZgr2FoxS5wVUym3w@mail.gmail.com>
Subject: Re: [PATCH 03/14] rbd: increase io_opt again
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, "Martin K. Petersen" <martin.petersen@oracle.com>, 
	Richard Weinberger <richard@nod.at>, Anton Ivanov <anton.ivanov@cambridgegreys.com>, 
	Johannes Berg <johannes@sipsolutions.net>, Josef Bacik <josef@toxicpanda.com>, 
	Dongsheng Yang <dongsheng.yang@easystack.cn>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	linux-um@lists.infradead.org, linux-block@vger.kernel.org, 
	nbd@other.debian.org, ceph-devel@vger.kernel.org, 
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 31, 2024 at 9:48=E2=80=AFAM Christoph Hellwig <hch@lst.de> wrot=
e:
>
> Commit 16d80c54ad42 ("rbd: set io_min, io_opt and discard_granularity to
> alloc_size") lowered the io_opt size for rbd from objset_bytes which is
> 4MB for typical setup to alloc_size which is typically 64KB.
>
> The commit mostly talks about discard behavior and does mention io_min
> in passing.  Reducing io_opt means reducing the readahead size, which
> seems counter-intuitive given that rbd currently abuses the user
> max_sectors setting to actually increase the I/O size.  Switch back
> to the old setting to allow larger reads (the readahead size despite it's
> name actually limits the size of any buffered read) and to prepare
> for using io_opt in the max_sectors calculation and getting drivers out
> of the business of overriding the max_user_sectors value.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  drivers/block/rbd.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
> index 26ff5cd2bf0abc..46dc487ccc17eb 100644
> --- a/drivers/block/rbd.c
> +++ b/drivers/block/rbd.c
> @@ -4955,8 +4955,8 @@ static int rbd_init_disk(struct rbd_device *rbd_dev=
)
>         struct queue_limits lim =3D {
>                 .max_hw_sectors         =3D objset_bytes >> SECTOR_SHIFT,
>                 .max_user_sectors       =3D objset_bytes >> SECTOR_SHIFT,
> +               .io_opt                 =3D objset_bytes,
>                 .io_min                 =3D rbd_dev->opts->alloc_size,
> -               .io_opt                 =3D rbd_dev->opts->alloc_size,
>                 .max_segments           =3D USHRT_MAX,
>                 .max_segment_size       =3D UINT_MAX,
>         };
> --
> 2.43.0
>

Acked-by: Ilya Dryomov <idryomov@gmail.com>

Thanks,

                Ilya


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:09:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:09:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733491.1139769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyGb-0005QI-TH; Fri, 31 May 2024 09:09:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733491.1139769; Fri, 31 May 2024 09:09:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyGb-0005QB-Qe; Fri, 31 May 2024 09:09:41 +0000
Received: by outflank-mailman (input) for mailman id 733491;
 Fri, 31 May 2024 09:09:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCyGa-0004Eb-1c
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:09:40 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 82833119-1f2d-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:09:39 +0200 (CEST)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2e95a60dfcdso21579541fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:09:39 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212b83d5c7sm18847855e9.8.2024.05.31.02.09.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:09:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82833119-1f2d-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717146579; x=1717751379; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=9nHLqWeneXgssP+00AwHJM3tHuDqX/2jS/REuWoSHIw=;
        b=mlDNHnC4Gji218IjrB5oOQ73chGvvlVNtAIaI8v+okPGK0fy7o7UQLn1175JQSCGmH
         YrQ0KTk1Bjwst/ggO4G9E/k0ktzpKhdaBTBTujG0R9rNrOeiFJrkSNPrRud067atjROc
         0lybD/1SyYKfAXUQjJh6+NavxEulR6hfEV+qniAWzbZgatRYOMgn/5LgLdNE2FowCVb5
         qXI2DL4gWLsYwjCP/JseYTtm7N7WfgIUFUDbXnaYWZs5YB+B9CbdxEo7XBGZxHc28qrR
         Z/q6akl2irKgLkCnT8ceEef9o3FtDe2AsZVgITYzqcCtKFQC0nT9uUAZk35ty1RPqNPh
         PPeA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146579; x=1717751379;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9nHLqWeneXgssP+00AwHJM3tHuDqX/2jS/REuWoSHIw=;
        b=NhPSUrnRL871/e+PlniZ6CX3wg6NhpMikbi/VhWkYcnJCJKplJYXhdDen2YvbBEqJe
         H2VPQvnE6AB7lyof2XFaCv1+20c0ndTOU/gpYhMOu0fTJBA37dDagMvPVZjJ8+Y3c7Pg
         +duPq9opmo79aqmfJXO77VO4BP8449rGJ5JhPcOyUOPDy27MRBCowIqdcjnj7STbwQoU
         oHWoNVa9PUHpSMr6A1L1Ig9RGkCXVU1dpWCaB9r3n0WWXxvk0YQZmN58aGdvXxcDVy8o
         ZkhPLNKzGzI1GODrHj6K6E76n7JDCM5DkN+KvfIIPpNukk8lgzSrNVDswcc47SN8K6g4
         UKQQ==
X-Forwarded-Encrypted: i=1; AJvYcCXP0mcQpVs7KJWRXYbfmQIMvlWSSf3YK8QGzPDFAGkvwHbR4Gw9qFxahv+c6SjO+dj3whzSxeYLeAP+77my+SlTWRIyOx62HhafEfmE7lE=
X-Gm-Message-State: AOJu0YwPLlXzCz4oTULw++CWQIWfQQCo9a3eFYNjlixDVaoEPPC1+C1W
	KKFFR/dKERfHjbF2iSq8Q1oZIXPDOf55mGQfx719zAjhPg2Ji747
X-Google-Smtp-Source: AGHT+IH88NdbdF6pEzhBNUimRkWJPQqdY0qzX+TwohQvgXS4DJe6CtPrF9PW//10pZ/crdnVcStzaw==
X-Received: by 2002:a05:6512:45b:b0:52a:daba:f7f0 with SMTP id 2adb3069b0e04-52b896dc4abmr679253e87.62.1717146578471;
        Fri, 31 May 2024 02:09:38 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <e1451535-b62a-473d-bb6c-00e91274aace@xen.org>
Date: Fri, 31 May 2024 10:09:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 2/7] hw/xen: Constify XenLegacyDevice::XenDevOps
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-3-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-3-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> XenDevOps @ops is not updated, mark it const.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen_pvdev.h  | 2 +-
>   hw/xen/xen-legacy-backend.c | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:10:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:10:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733497.1139779 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyHM-00075h-6b; Fri, 31 May 2024 09:10:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733497.1139779; Fri, 31 May 2024 09:10:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyHM-00075a-2h; Fri, 31 May 2024 09:10:28 +0000
Received: by outflank-mailman (input) for mailman id 733497;
 Fri, 31 May 2024 09:10:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCyHK-00073w-Bf
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:10:26 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9d7f7cc7-1f2d-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:10:24 +0200 (CEST)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-52b7ffd9f6eso1051111e87.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:10:24 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421270ad598sm48564345e9.41.2024.05.31.02.10.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:10:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d7f7cc7-1f2d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717146624; x=1717751424; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=kipgqKIxTWjiY1J4diBph0ORkE9vgc6ZqFETK6Cxncw=;
        b=EImHCa05JtLaZL3prsnaLyNaFm6QZBPa8xIKNMUYpYyWoNNjPMTXzg/REGjFlQsXKO
         d/yep5LtnIIBEQBS/CR2SjlVyWkRVusxTpjFDmT5beruP6lDsK8/6XI9KS5Iq/oMN4yd
         KzbeNOHNPykLpj22fp/l/iN68QWHtusAn7Cj+nGmXjW3dcDxt5DIbWvC+4euecNUr8s1
         LI1fN2k70kMytOD5l4fUrAxbLBCnU7ksL9wQ6z2CCBlqU84antl/+sHz7fJyX5aO4YKv
         t+YnpKr0r2qXcPaNir5qncxlnPv4iG0pWRLFGrRsZ4rUatB8d990eD7ZJPvJIh9O/6so
         XXDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146624; x=1717751424;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kipgqKIxTWjiY1J4diBph0ORkE9vgc6ZqFETK6Cxncw=;
        b=iAPYOgmeiCKnOD2bOtc/H670FwQqiaTzEfJ6XiB8Y51+uPPTfepofNB8kxZLmJLE2A
         F/qcmJT4TE1/EyezPyznieiAO9ZnYaRo9HaUVVA9JDoa+Flx9xniYWKvy1E+NHHneYIs
         NL6Z1o8EYq7RKNOX9fZnFKV0tG2ceXUKvpiVB8IPBnNx+SoDMQZHQKPCJMXfX389gDKn
         ak63s1V1yWQOBMIvoCSky2lOGyad8j0dig65Yw4dscd1R4ymcO7PbUjscyMp2RR/vVev
         3keeLNj93MxvSGZLUL5/c1VEdeI90QyugdVATFVVv7CXfpyae974gUv9r6Bt/urnNqxk
         viWA==
X-Forwarded-Encrypted: i=1; AJvYcCUQFEwMeWidTMZEnAtMlY1g20eOCt26Y5ZCprobkLBx5O4Od50kB/Eyau0GWiVURz7S7lft4sWt9Kme9yAf9nOxwnzaigML3/JWNkVgB54=
X-Gm-Message-State: AOJu0YwQewp1W8rfo1wxPwwz1/+cG6LHTeu/64+uOmMZlhDT0bys/rmX
	datmRLUMle8ztNQXCi7+lBESG163MwX3JgznR9R+6e5TJDQUjjM+
X-Google-Smtp-Source: AGHT+IG7vTACSFJneASAY6oepgxD0uTbsNdLpJPk2wq3sK792UuWxkKyL5fSGa5cotd0sdJt1GNwgA==
X-Received: by 2002:ac2:4a72:0:b0:519:1047:7eac with SMTP id 2adb3069b0e04-52b89571166mr724613e87.23.1717146624171;
        Fri, 31 May 2024 02:10:24 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <98ef7ab5-5c8f-45e8-9bd0-50e014b981ea@xen.org>
Date: Fri, 31 May 2024 10:10:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 3/7] hw/xen: Constify xenstore_be::XenDevOps
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-4-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-4-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> XenDevOps @ops is not updated, mark it const.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 2 +-
>   hw/xen/xen-legacy-backend.c         | 6 +++---
>   2 files changed, 4 insertions(+), 4 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:11:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:11:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733500.1139788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyI8-0007dA-ER; Fri, 31 May 2024 09:11:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733500.1139788; Fri, 31 May 2024 09:11:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyI8-0007d1-Bq; Fri, 31 May 2024 09:11:16 +0000
Received: by outflank-mailman (input) for mailman id 733500;
 Fri, 31 May 2024 09:11:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kOju=NC=gmail.com=idryomov@srs-se1.protection.inumbo.net>)
 id 1sCyI7-00073w-Ar
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:11:15 +0000
Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com
 [2607:f8b0:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ba31fc09-1f2d-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:11:13 +0200 (CEST)
Received: by mail-ot1-x330.google.com with SMTP id
 46e09a7af769-6f8d0a1e500so1357430a34.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:11:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba31fc09-1f2d-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717146672; x=1717751472; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JHySJ52Pl0hABImbEf8CzUryCpcO6Vzdug5aWyPJAWM=;
        b=MVbCRikAHGVs7UB3vplpRi6p55OJ4HfwYY+ATDZQKDXUj+66DJHLlspJI86wjJFFrs
         v6mVgT4tPih85ggRVTMSflTnQqnLkxrTUnKVRtaS7quxPJQ3ZdhWj1Lj3QxHSqgnklGy
         YkanKcD+05weuzE3y+Kfo4RFWorMD0Wo5nCSkXwsCuKYst9xVCpOwAbHtPEiOwHZXMSK
         WfSEMxYGEmNHORa+KjkSA5YtK+D4/i10qgedJYX8qGHSVqqxMrQOfCs7zcNtvXHwux8D
         7hPWzkCOX/wirv+UdOED87Hlte1PZOFdOqXubC9MgSj7uf0uzhKgw4FvJs/TtlwHGRIX
         XZNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146672; x=1717751472;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JHySJ52Pl0hABImbEf8CzUryCpcO6Vzdug5aWyPJAWM=;
        b=toe+lYLWN1i21H2dDNK0xT1Vl2gSGQBVbyZ+mYc+vKm67kilgUQ2muHtRdjUv8RWj8
         D7Waczu4eeBOAA1w8vpJFQAvpyVfdWve85obvkVHzvaztK0AE9oWeccIkKlSWuMqF3CQ
         W3b6rF1esuRfJipujWGCZuIZhkTEvnJJwFtX5jc+6dtytsTc1X8G0PuMMLsEUxIOyy+Y
         R+csfJ4SaWxARBkUwNppEq020eZToHo8Q9tnaXp2aiEUCFWwE039z/GpLfezYQDwFEbx
         XnMjEyzgzO85JMbnSj4fL5vsYtpJeG+FGactkW3sZxAhk3toKaQFq+/wZhIoVELMrzE1
         y9YA==
X-Forwarded-Encrypted: i=1; AJvYcCU2w7EPDXoI6r4QdsrLeaijZJNnAUNFIJ5AK/E1bd2mVYA2I/SAJZk58WvqqA29E5fdoviJHE/MjiVpBJJgUdcGWuWOhsgiAlGI3frrDjQ=
X-Gm-Message-State: AOJu0YxjjwpqunxzkN0xfCIs/7q/UwkqcgbEpYlrDa1Pz7wJc8MA3kL5
	WmVv98McIWtXUC5ZJwo1gCVIOSqUUgP693KNOTW4G733GbwfZFlUl7cS6/FsgS4kYZ0xbpx1Km3
	Qk/DqoxgqhwilktVAFPIIneyKIEU=
X-Google-Smtp-Source: AGHT+IGvsAs1mcJwvgWnNqWcEhu7b6x8mgHTIAZ0GzBV17r0EcNcgsUewUXES03iA8vPcRMr9oo8aonYpNodLH6h808=
X-Received: by 2002:a05:6871:339a:b0:250:75cd:34f2 with SMTP id
 586e51a60fabf-2508b99ababmr1494585fac.22.1717146672346; Fri, 31 May 2024
 02:11:12 -0700 (PDT)
MIME-Version: 1.0
References: <20240531074837.1648501-1-hch@lst.de> <20240531074837.1648501-5-hch@lst.de>
In-Reply-To: <20240531074837.1648501-5-hch@lst.de>
From: Ilya Dryomov <idryomov@gmail.com>
Date: Fri, 31 May 2024 11:11:00 +0200
Message-ID: <CAOi1vP-+j-RHLmnDLpsZREnYb_f7QGGhRC9YOgctsFNuE7yM3Q@mail.gmail.com>
Subject: Re: [PATCH 04/14] block: take io_opt and io_min into account for max_sectors
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, "Martin K. Petersen" <martin.petersen@oracle.com>, 
	Richard Weinberger <richard@nod.at>, Anton Ivanov <anton.ivanov@cambridgegreys.com>, 
	Johannes Berg <johannes@sipsolutions.net>, Josef Bacik <josef@toxicpanda.com>, 
	Dongsheng Yang <dongsheng.yang@easystack.cn>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	linux-um@lists.infradead.org, linux-block@vger.kernel.org, 
	nbd@other.debian.org, ceph-devel@vger.kernel.org, 
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org, 
	Bart Van Assche <bvanassche@acm.org>, Damien Le Moal <dlemoal@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, May 31, 2024 at 9:48=E2=80=AFAM Christoph Hellwig <hch@lst.de> wrot=
e:
>
> The soft max_sectors limit is normally capped by the hardware limits and
> an arbitrary upper limit enforced by the kernel, but can be modified by
> the user.  A few drivers want to increase this limit (nbd, rbd) or
> adjust it up or down based on hardware capabilities (sd).
>
> Change blk_validate_limits to default max_sectors to the optimal I/O
> size, or upgrade it to the preferred minimal I/O size if that is
> larger than the kernel default if no optimal I/O size is provided based
> on the logic in the SD driver.
>
> This keeps the existing kernel default for drivers that do not provide
> an io_opt or very big io_min value, but picks a much more useful
> default for those who provide these hints, and allows to remove the
> hacks to set the user max_sectors limit in nbd, rbd and sd.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Bart Van Assche <bvanassche@acm.org>
> Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
> ---
>  block/blk-settings.c |  7 +++++++
>  drivers/block/nbd.c  |  2 +-

For rbd

>  drivers/block/rbd.c  |  1 -

Acked-by: Ilya Dryomov <idryomov@gmail.com>

Thanks,

                Ilya


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:14:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:14:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733504.1139799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyLF-0008Fh-Se; Fri, 31 May 2024 09:14:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733504.1139799; Fri, 31 May 2024 09:14:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyLF-0008Fa-P3; Fri, 31 May 2024 09:14:29 +0000
Received: by outflank-mailman (input) for mailman id 733504;
 Fri, 31 May 2024 09:14:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sCyLE-0008FU-SJ
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:14:28 +0000
Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com
 [205.220.177.32]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2cf79836-1f2e-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:14:26 +0200 (CEST)
Received: from pps.filterd (m0246630.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V93r1E027556; Fri, 31 May 2024 09:14:08 GMT
Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com
 (iadpaimrmta03.appoci.oracle.com [130.35.103.27])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8j8amy6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 09:14:07 +0000
Received: from pps.filterd
 (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1])
 by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44V8xOFi010688; Fri, 31 May 2024 09:13:52 GMT
Received: from nam12-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam12lp2168.outbound.protection.outlook.com [104.47.59.168])
 by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc511qa0g-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 09:13:52 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by CH0PR10MB7462.namprd10.prod.outlook.com (2603:10b6:610:188::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Fri, 31 May
 2024 09:13:50 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 09:13:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cf79836-1f2e-11ef-b4bb-af5377834399
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3DcHHrRtBpQo6HarUUVxEbRkQ1sXtnA7t664mRopKjiWY=3D;_b?=
 =?UTF-8?Q?=3DUg4UTyMQnf8D4yn1+sak5TU0zscr1/FFBw+MCJoKKstxuU1TKc1R0/YNRYn6?=
 =?UTF-8?Q?483+quwo_ZMhCEmARYOYdrJxJbwr62nUuo3UVNQ68jWn7GKJ5EbqDdVeISo3Jg9?=
 =?UTF-8?Q?1M17teypia0Ygu_rbiR/r7s+rHwnqfaYcUW+s2QoVJd4yBI76zhfW0nyJvBzT33?=
 =?UTF-8?Q?9S09GB1A+7FpkWTbRsI3_tyAvXc9hTz8o4QVnxQtgXzQ5h5MOIKZ61KajO93Li/?=
 =?UTF-8?Q?kUHWPZnHpRjee7b/KtaNze01cd_OMhB06ROrTuQQcnId8/G8sQUC+8QvplLJtMk?=
 =?UTF-8?Q?HSdh2i2JDVd/HXxwGSo1/cJd/wIu4+Cb_+Q=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DgVZVOUWa3jO0pvA89TlU/2ZjbRkxB+g2bnaXF3d7/iyusPf3K06ieO7a7CbNpeTk3Hntx9W1gfFmwKXXg6LeNz9Hw+4OSTUzeuJE0SxhEE7PWGpJtT/D1n1ooqJ5vCMsEyorb1quGP0w1eX7Y5qrd+K0ez82kAXSb2QVmNVLBHLa0O+I6GHmlbhXQbz57z4v6k0+O90HOZ8X7lEyTiJg0vxhC7klHZDPgu1AjqZFmEcLTwx1MjVKrK6HkLjBGv8F5PkrWVRMqLoXJqwdSd9u4x+LybOUsPrXsqOdcttYXFhLMlBpnFDE/drD2cYtdDGPTGDGt4bsh3rPI5YzkbeIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cHHrRtBpQo6HarUUVxEbRkQ1sXtnA7t664mRopKjiWY=;
 b=SYI7jPGug0dM8HIM8Nsv5t2XcvB2xEGC4IVWQDJ3Rv9eYUFrpbZEwb+N/kHNKWKM/iX5zg1L7s98EeQR/Xb5PtslyA+EmBCYRu4cTeHykW3fuuPz82OnfoV01/IoAxZjHr2zNbAfVbF93bfRZAMX6doEE48/ak4XCtkxtp0opcojdVXRBknFX18iSa8frHKIpQuD9kds4TGhPWFwH5uaf3We+Ich8KzYfE7M7p8/TF2PZ1fLDXbS3d/q6zR6ViRJTO2h4tExHLniSC5Viz9oCiix3DvrdWq9KShBTg4dGgsa2TofkR3wouxrHgUEiMjGydrsrQp5ELtSOUq7aFFrOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cHHrRtBpQo6HarUUVxEbRkQ1sXtnA7t664mRopKjiWY=;
 b=fy/WJg0YV8qYk5TyUZ8FYkjYje5ALZsvMeaHpD1W5cPIh6QVhv57e+f5r0+lDIWV3ChfxJry027spIB1eTUZ7KLBr4NyAXg3FWonhitRv+lYFFJ9AotGPvZ3sYDRAX2ve6liiMEZJTXNW4owQL/FocBA2TEEGfqALlzPlMIh58I=
Message-ID: <6f608f8b-2eb8-4ad4-8e8e-d210b9ffd712@oracle.com>
Date: Fri, 31 May 2024 10:13:47 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 12/12] block: add special APIs for run-time disabling of
 discard and friends
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-13-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240529050507.1392041-13-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P123CA0442.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a9::15) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|CH0PR10MB7462:EE_
X-MS-Office365-Filtering-Correlation-Id: c335975d-60ea-4776-ccf8-08dc8151fc4a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005|7416005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?d1dobHJDZzZiWlNRdHA5SlNZS1paN09EV3p5S3lJMmQ3OTk0Nmh3Q3oxdCt3?=
 =?utf-8?B?SzJGMVZQR0VQTnRIY2x4NWlJS0FCNythd2JxRXdHVXRVWlJPc3hRQ0pWRTBn?=
 =?utf-8?B?VXludFkrMnVHdUV1KzFDbVp3ODBiQTI2eHNqaGFDL1JFN0xlazFkVkZvRFNI?=
 =?utf-8?B?WHpOWUNnT3RsMEcvazcwci90M0lVMHgxaUN5VWRaZStlcWd3QmJ4by9IakZm?=
 =?utf-8?B?RU1lUXd2UHpjbUJwbGxDM0hIeVBQV0VsMWVBeVZzRFFrV2E3OS9CVjgzVGtz?=
 =?utf-8?B?dHNLREdLVlVubDZCMkV5ZGJrYktZZTRadWhvbjcyR2F6ekRzdGdmRGFaNG04?=
 =?utf-8?B?aHJYUTRIalZCWGU0RFdDZnF1YVRoMEJ6ckRNQnZ5My9wcTc2V05INnBmZE02?=
 =?utf-8?B?dFpZVlljT05jbWhsVnJqS3dLUmxoT2dqMXJmbnZIeG9wc1FNVW9hYUM3Y3A3?=
 =?utf-8?B?cHB5MzEyN28vYkdkZzRvemN4VFY0eDJzaS9wZS8weE8rUm9FeFNPZW9LaFlS?=
 =?utf-8?B?S0x1Y0xHcVdaeDMzVWJIMEoxSkxUQzlMMVBxTytRbDBmOExub0JwMWJRblZ0?=
 =?utf-8?B?elM4Um81b3hwT0lETXlwaHB6akJoL1ZqbzZuTjFIcEtRT2RScEpkM2Z4OFBR?=
 =?utf-8?B?R3pMZVFlR1AyVzE1dU9CekF5S0VZOHp0dEU1RGg1TSsvNTlpaWlxWDkxMm1u?=
 =?utf-8?B?aW5VWjhVb2dzdXkrdTRIT1dEM1FmdTlaQk1jRjAxamJvQUZPMTh1akN2aEwr?=
 =?utf-8?B?VVVjMFFkNkRIZlZrbXQwU0NUZm03V2FhRlZCQ2poeTBnTFFvVHFva05kNllh?=
 =?utf-8?B?YUtYVmwxS25zREFSaW1xSERnaENpajU5bzRPWjBQblNORzJVamd2VWl1bHRr?=
 =?utf-8?B?eHNQZWZKK1pVL1l1NFAwTTFRMDQ3bit4MVVVV1pPVlhhYTBQS29IWVltUUV1?=
 =?utf-8?B?K2Qzd2l5aUIzKzVacktPOFpHSVBhNXFhNUJsRk9vZHVNZ3ZOTm9JaUlFYlMy?=
 =?utf-8?B?VENPVmxETjdlVzdoMEhGdTJOYVhJNzE1MERrN3d0T0xWUksydGEweU5SK3FM?=
 =?utf-8?B?SjIyZG5vZElnUldnUTRCd256M09lRkI5a1lLejBtREp6V2tYVko3SERyK1NT?=
 =?utf-8?B?RGE2bGMyS28rRWw0V1ZTVEVqTEo0QVV1QmhPUE0xaU54eDIvempSNUo4UVAz?=
 =?utf-8?B?Z3NFSjhVZDFjZ3o2THV5Q2VDQk9Hd09odnRwYTErT1FkN0ZJaURlZDEvVCtY?=
 =?utf-8?B?NUNmQ0Vsb3A5TUhVQzlyM0VjU3JEeC80M3I1STYxcW1IOGFtcy9USXBEWHpI?=
 =?utf-8?B?ekY5aUQvN211VW1UYUxVek52dWthc0NlQWRnem9Ha3UyZEh6b0pDLysyb1dH?=
 =?utf-8?B?WVJLZDNGV2FOZDR6ZUNTN3oxTlQ4YjE5Y0RUSHFhTGY5YVNVUVdHMUQwRlNV?=
 =?utf-8?B?OExoZDV3K2YxNW1ESDk0UTAySzVyaEhSNzdWWURnc3R5Zno4SUlJZlMxRllm?=
 =?utf-8?B?dDl6ZVR6enlhT2s2U1J4YjM0ZFVWdkZCZlcwS3VCOFR4QUV1ckV1cXdQMnJQ?=
 =?utf-8?B?ZlNYelBYUDdLK0NjYVpPelI1R1FRdGRXRGRKajZxM2k5ZUpnNXdUalZKd1Y2?=
 =?utf-8?B?aGdXNDBEeXZZdnBQTDVLR2VsME9rdndBdVRMZnc2MDhqV2V5Yk9TcXVXNzFO?=
 =?utf-8?B?U0NtUkV6R3FjVXA2VDhBbERNTlpRWXEzQjk0QjNmUWtlVFZtdjhsSmhBPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(7416005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?OEYyR0xzb2ZNN3lhN3lTU3lWU2o4bU1PMm1NQW1ZS3dzZS9nMVpvNEJFLzkw?=
 =?utf-8?B?eTUyVnBpRXRJeHdDR1dNeEg0ci8zV3JiQm1odzlYeWQyZGNqZEtCQW9kb1N2?=
 =?utf-8?B?U1RsNUZxSUJPSVhtTDRpSy9nNnUvU1pPQVBGN2VrSEJUNUVKWlZ4cDhYUmhw?=
 =?utf-8?B?V0UwbFB2dWN2SXh4NjBIQnhMeHRFYnk3aUNMMW9zK0ViYXFZNDlYV3Rhc2hR?=
 =?utf-8?B?NmpucHlrWm5pZFFwNjdpWDlETS9XY0pxYVU1ZGZmR1YrbUFUY21aNzcvR280?=
 =?utf-8?B?R1hmWHRXelo4dWJtb2x2Um10ZWFPTmRWdWdRdm8xWXU5V0tCM2d4V0czd2Yx?=
 =?utf-8?B?cTFtaW90VFNUK0tkT3E0MlFXbkhGMTAvZEVsZnFoM1V6c0pqejNCdVhlc0Fu?=
 =?utf-8?B?clZjcC81TzZVN1VhU21CdkV0TWdFNlFiczJqYkF6bFMzMldBL01nRWRHRzN6?=
 =?utf-8?B?Wjc1TjhGaWZBajJXSHhVMEUvbDFvN2pGazZ0Y0c1U21lU1JhSWxZbEsyRUQ0?=
 =?utf-8?B?ajFGTFNQaDhTNXhiRWtrcmtsTHM0VVUySHdwVnZXZ3h2Q3BtVHVDNjI0T2Z5?=
 =?utf-8?B?YzdXTWhSUys4eVJLWVBIMUdRclBOSm1kd3B6TjJXV3EwRXdOK1psOUtzM1p1?=
 =?utf-8?B?Y29aOWs0d0RHZ3VjaGhXWFFuVTkwSG1hQUxVeThrdjV1R3N2SVBwVXJaalps?=
 =?utf-8?B?T1hFbUJNeXRja2Y4TGpzL3ZEMCtuVEhFUnlySXU1ZktNa2xLM1BDTGcrWWNp?=
 =?utf-8?B?QVVYb3lrWG1xcklwYlNFRkpLaGVhd0p4UG1lZmZGQjBHN3hpcmFLNDgwMVlw?=
 =?utf-8?B?WXdXdnpPZzdpdTYyV2E3eWZGaVZwSWVsOFh1TFFkS1hDa3V3VVkvUnM3cnBC?=
 =?utf-8?B?SjdOM2FRY2VNY3VUejdrTFpoWXRTRG54S3dRY3FSVUlBUzR2UE0remttZE9k?=
 =?utf-8?B?YzFzbFNmbCtObEhOTkFBSm5lcnRtQ2kwNStvR1BkMkFCVFBLcnNUT0VaN2Ey?=
 =?utf-8?B?ZEx4Z1RjVWN0QVBYWWRaSUFvUkJ1ZTVGc2s2aDBMVCtlNndRSHRFZ3J1Y1Ba?=
 =?utf-8?B?YTVLK0lqT01lSU9nZEhJaEVER0dKMUtlYWlqdWpDcThubGJoWDBvRllYVGVk?=
 =?utf-8?B?c2piaG9abDkwakUxTnJJa3d4UDhYcSt4UE5sTWtGTU5CZzRRbFQxb2V2bklw?=
 =?utf-8?B?WHpvRHV2aGF2MmJvNGhSNmdkUWtYTE4vR2MxdXF0UFM5MXlsNTdKSXFzekNX?=
 =?utf-8?B?aTBtd2ViMVJlWVZDNy9ET1FTakQvTkVyM3BaS2NsMnY2VHJpdmJLdkFIQVky?=
 =?utf-8?B?Q01XNGhpd09aWE00NldpM1NKRlpkdG1qcFRVcVFHSXpZTWtDNENvVWFkOU80?=
 =?utf-8?B?bXc1eXZOVzkvamFUU2xBclFucDZqUmVCc09sblNpZXJrbDd3ekI4d0szOVFj?=
 =?utf-8?B?SEgyRmlRN0U1WXZEdXpOTk5RaXJsWk9BcGsxdzA5Vnh0eVhQWUFwTWg3aWFY?=
 =?utf-8?B?QUp6eEoyUHJYZE5YRTdESnd6L082K3orMVBuc1JWZ21TQWd5RkN4UTBUdnF4?=
 =?utf-8?B?UGZxY3FuTjVBdEQ1TTBPTzZpcUU0ZlJ1N1Q0dkJaY2NBMFR6TlhlRUVST2lP?=
 =?utf-8?B?MmF2K29QZUp1U1hZOG9USzhxaTlnbWMvRFQydHZkWlp6MmgrMEI2dkErUUQ1?=
 =?utf-8?B?d0lRSFVpYkozY1gxWEhYSGRFY3ZTbUJ3TWRsSEVSNkxVd1FVZ1RDY016RGc3?=
 =?utf-8?B?SGRxaHVoeWx2QmRtQ2RJMVh4TXkwMjZzREcrWVhCNmVJb2ZSZ2lKOUlTTzJU?=
 =?utf-8?B?cnF5NEs2TUsvSm9JKzNCTG9CU1FYYTd5bjFpNmRDS2R0SEoyZjljV1FPT1g0?=
 =?utf-8?B?cytkeWtPNGN3MUNPaTEzaHpDVll4SWc4a2RlYlJwMEtiVkU4Z0E0UFVpZklK?=
 =?utf-8?B?cjNkeElpdEI2L0dsTHdCVXR3VEVEWXFNTkYzRXQxK3IxSnpZSzFJTW1PdXly?=
 =?utf-8?B?VENEenprNmRLVFhsNmFZZ1ZVakhUdlYzeFhhZllDVC9pZXozb2JHT3dsRmhF?=
 =?utf-8?B?YXcrU2F6d2dEUVVOdTJET3d6dFBiLzEyc1J0UzdGb09Ha0Z0YWM0WHEzaGZq?=
 =?utf-8?B?K1FmS0lPbUJqMlFYUGtGY04ybnZiYzg5UTkrSkczUHNvVUxjMDNyamxlUVVl?=
 =?utf-8?B?U1E9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	4kHKVAtjuMQDK+/qAH+z0zm2kfSe83x1rjw3ZVWCKciRgMUdhu9TLPKS8PgYHjmuJvg0eA4VYvi9kxj7uwJl4bGhlSebXLCx6J69M0Z5VNmGoGnlHzd9KI0rIkyFhUPwelJBc24OdFNZQQNmGDzLMmt4gZBaZLRZ1tLSQXZeEuAQu/5bxw/9UmRSyrrP3OxvoGhVGP9B8TXFmnW/hARnk9yzaoMsAmJqACeKOpIf9M+eP/eYI7hZAXLkJs93vRzgNemuKFxvxeXdXKsRXpDEY8GzmxY/dJecczt0aukd9MmXq2OfPOGrMlJrZt8n0c5654dhSAoNzWIr0j6LqZVzEV8nWZ4YmLcqjw3sgfbsVNGnSsVAL1w1v+RGQF9pWZOKPLtIt2JntkENLFABjUNNXb8i/sfSXipU9dBfqpTtofrhMrXJdUZnJUBuhv9s2zZuZxstSEppCP+s4VVzmmEt3nOLM617QTDy/PwKU5BQlQzFTY8m2X76yxc/OtrTXsKzO5dUxI59BLhi00NX0KdMVr5op4JghpUzgvCHU0bsmisZYEq8g3BApJHXfTPpBKEZrL1QWQR0t+0j8/DGOhJtpHqAq1TB1G4DYv8wQW72Gzc=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c335975d-60ea-4776-ccf8-08dc8151fc4a
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 09:13:50.7276
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 44FhA22J2PAWJbljg8/bBf/DYqpIHIdMyy22MJhlFMcozT8qGnBB5qHSu74ZtCCQg97bro3vQtcM3ya6oZNH6A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB7462
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_05,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0
 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 suspectscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2405310068
X-Proofpoint-ORIG-GUID: 0rcWkHplyRDCatLlB3VGhVObKVFEMQWm
X-Proofpoint-GUID: 0rcWkHplyRDCatLlB3VGhVObKVFEMQWm

On 29/05/2024 06:04, Christoph Hellwig wrote:
> A few drivers optimistically try to support discard, write zeroes and
> secure erase and disable the features from the I/O completion handler
> if the hardware can't support them.  This disable can't be done using
> the atomic queue limits API because the I/O completion handlers can't
> take sleeping locks or freezer the queue.  Keep the existing clearing
> of the relevant field to zero, but replace the old blk_queue_max_*
> APIs with new disable APIs that force the value to 0.
> 
> Signed-off-by: Christoph Hellwig<hch@lst.de>

It would be nice to improve this eventually (to use the atomic queue 
limits API). Anyway:
Reviewed-by: John Garry <john.g.garry@oracle.com>


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:14:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:14:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733505.1139809 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyLM-0008WF-5x; Fri, 31 May 2024 09:14:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733505.1139809; Fri, 31 May 2024 09:14:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyLM-0008W4-2z; Fri, 31 May 2024 09:14:36 +0000
Received: by outflank-mailman (input) for mailman id 733505;
 Fri, 31 May 2024 09:14:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sCyLK-0008VD-Ds
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:14:34 +0000
Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com
 [205.220.177.32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3067c957-1f2e-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:14:32 +0200 (CEST)
Received: from pps.filterd (m0246632.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V93kQq020927; Fri, 31 May 2024 09:14:18 GMT
Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com
 (iadpaimrmta03.appoci.oracle.com [130.35.103.27])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8fcjs81-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 09:14:18 +0000
Received: from pps.filterd
 (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1])
 by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44V90U0R010651; Fri, 31 May 2024 09:14:17 GMT
Received: from nam11-bn8-obe.outbound.protection.outlook.com
 (mail-bn8nam11lp2173.outbound.protection.outlook.com [104.47.58.173])
 by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc511qac0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 09:14:17 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by CH0PR10MB7462.namprd10.prod.outlook.com (2603:10b6:610:188::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Fri, 31 May
 2024 09:14:16 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 09:14:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3067c957-1f2e-11ef-90a1-e314d9c70b13
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3D86HW1+PMNrYdS47amrU0JoQ3+mprOzG+m1WSAURzBUg=3D;_b?=
 =?UTF-8?Q?=3DTFbl1bsW447/vgYSK6crHkEHip3JQmIgutevB5tHOEfZiLhZbeh/RuKiH0L2?=
 =?UTF-8?Q?mnW/ohzC_IsAJVdcPKyZJjYT1rVtrmeth36uSQU4kRSBu0s1ntlnYcHs8AEDB2M?=
 =?UTF-8?Q?fHHU1LJiEuvy6C_FZP+l8wQmKqZ+SUJpjobABefvGGluCxlSTLWxzloZqtF2qOr?=
 =?UTF-8?Q?UAHR6Ginj8qJsLqiHol8_B+VqeWFYkM3ClyYWstru1SK0HCo9nSyG0pIWdgKWSP?=
 =?UTF-8?Q?2bRFeJ63/0jpoZAc8daaLrYP2I_0VgN1C/l5QJothy2oY+R4DFttiMYG2lNgCUX?=
 =?UTF-8?Q?+YOmVfNbINb5KuKvQJSeprbZQzkL9zw8_JA=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BYDCRzMnSwXX+gU2UAeeDbPufthB19AYd5mXt1hCec4KZiqpy+Uezg3b+mNctS0mmS4iYqHQjIDks6mcSbceiL4tSU6F7cdhzsw72GsMwvwBzj2rDRYyYkpWBdyIlNAG3FGkLe8n+qlEkJY1g6TjDApL8Hj7X28Hnnn1WIiphCFb5L4JufmO2C2QVmANuECcRRbhrHOP7MG+vktxxInEbXa26F3Qex5YDzie4ndVDlA+UAt8YooXZiBBW7SG6D/dKS4zY8pHZPQXh72gaSTtPktDiMmwyikbBcxk4V9+0pl3c+Qu8jkV1HmrJe5rVp5W0HScKdqcWVZ16OzeBvY1mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=86HW1+PMNrYdS47amrU0JoQ3+mprOzG+m1WSAURzBUg=;
 b=eCIBg/Msu3PFMjzlob20+WSWIQZ7ZS4SBOSIbBXVfZJK5GcUUkGnF7q7Uzg60KtJy5VVtBdSAXmFf+eDh2WUFyvysy1OcMY3weYtTEY5ieey34dsoOc9t/cjHQIdt5+Nu0/7k38t5jCZaDwP7+OrRSKTDnbgeAqmhuVxSm0IFB7mbJK1SfkPhfljKbR1QP4TjP8EF7ngMh1aHndZSCpUJxfV2vWb+rvG/i66FQQADFNyBITxmS3M+466D7RejwW4WfYdmBQWdvU8wyIGHdUy3bXLe7acQ1z1cbcGLzqGd6lwF/AMlNCPW2HIg7avpxGx+gpBqoSU7soPrRpuvlUDBw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=86HW1+PMNrYdS47amrU0JoQ3+mprOzG+m1WSAURzBUg=;
 b=s6kTuVp+HtMTevXWkzET2r4vM2fNG+jBVetZHodlChVV6pMQgoJzXUr1C0B9RLzZa3YdgalbychlRFFfih0VK2Azsf+vuoEdMtQou29l5X3bSLbPcmjJxu45BXkqbjz/xVwxGizGA+4ZevHo/jtWGaGv8l47TroAohpr8k/pM1Y=
Message-ID: <945070d6-b2ef-4be4-b85b-138a41641f43@oracle.com>
Date: Fri, 31 May 2024 10:14:13 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 11/12] block: remove unused queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240529050507.1392041-1-hch@lst.de>
 <20240529050507.1392041-12-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240529050507.1392041-12-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P123CA0442.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a9::15) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|CH0PR10MB7462:EE_
X-MS-Office365-Filtering-Correlation-Id: 45f090b4-c36c-4784-2f2c-08dc81520b64
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005|7416005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?M2lrcHFxZ0t0S3ZCNm9HdUlHb3dPWGtXa0J2ZGJ1L1F3cWYrbjJGUTVTZklj?=
 =?utf-8?B?bTB0U010R2IzcUxrZGZiYnZVTmdyMmQ5Y0I2QnlzdDhHTWlYVndsR0pnUlU5?=
 =?utf-8?B?emdnUzZoQzZybmlXbmFsZFZ0YisyK1BjY1JOUzVXS1RyTFpOUVVjLytqRmJU?=
 =?utf-8?B?SkxsSUN3bnRuRzdwWEsvcW05TkVCUWdaY21oMnFvdFAveko0Qi95THlBei8y?=
 =?utf-8?B?a3pyN2NpMldQc0hEQWxrOEFoSGpXNDF2aWp5ZE1OZE9PSmVNOHptK1dTVE83?=
 =?utf-8?B?N3JDVXN1UmdBVEhpL1hPbE02Nlo5TEpNZ1YxQko3UkdQaGsyR1pTd2xqSEdq?=
 =?utf-8?B?ektMMkw5Z3Z6RDBHeU5ycnhXelBVVmdHVVVLKzZaT2dYNlpFTEFMVzZKSG96?=
 =?utf-8?B?ZXdmTUg4TFU1RE1GYkdsSmJhRXN4QWdNOEswRS83S1pWY253NmczbnJpQ09I?=
 =?utf-8?B?UnkvZzZRTzAxRVFZMHlvZG0xdzYrVyt6eVpnRnBkbWg0UkVGc0FSaFZwNExP?=
 =?utf-8?B?K1pQczNyajBIbFNXWFlaWXRMQm1LU2N2TVNwQXFUcnh4ek44OWJxeTYxOXJG?=
 =?utf-8?B?cEhYdDZtNzJ1N0VWM3RobE5ybW9PbEc1R3JLSnAyemg4QUtKRGljblNGRTlm?=
 =?utf-8?B?UUk5bkQwdG5rYThoSDZ4VHY2OTUwTEtZMDZyRmszNVZOc2NvYXNwY1pTbDFn?=
 =?utf-8?B?ZjVGTlFCeWpZbC9BRGllYTFYWUN3WVByVXVFWWdpdU1ybTFEZlZJcUc2cDQ1?=
 =?utf-8?B?UjNwdVdIVWlsOTJnaWc1MTZoUWtvWVlqNjdnUHk4a3owb3hLTHV1eUN3d3ho?=
 =?utf-8?B?VkNTR0V1SWtod2dTbzVpR25BdFVzbE1oZFVyRXQrZmdJYW9nQ215NlhGT2JG?=
 =?utf-8?B?UkhEUnEyZnlXampiQlk1cWJDTEdvZmpWTGE1ci85UDd6LzVNWUlyNjlzNzdm?=
 =?utf-8?B?bFl5dUo5SjI4VHNFdGJzNjdnaTNqa1VVMG4vRU1Mdk9yeXBwSlJSTHBjcko2?=
 =?utf-8?B?Qk42bkc3Q0dKZUIvU3JjQU5vSEpCU0dMblZUUmZxVkk0bXk2WWd6cGpoRmh3?=
 =?utf-8?B?c3hMNlVqYUpBbFZuVWZqeldVYnlkQjRLZG9pTFByb1g0eERpYjNWZDNaQWZY?=
 =?utf-8?B?VHN2ZFBGUlVMRjlRQk1USlUxTU1HOCtMd1ZQWUZRMFo3M1Rhc1h6ZUdSVkh1?=
 =?utf-8?B?c2Zma3VBdFUxOFFBVTRkakFDb3pucmZMMWNNNEtHdE91MWZXc3ZLNXpTVW1K?=
 =?utf-8?B?SWQzMnJlWG5SdDcrakVLUG5ndjlkWG12QlEzd1lvMElZSUxkNW9DNVVhckhz?=
 =?utf-8?B?OVlSeDFXeDRmcGZuNm83WjVKMjZXTG8vR1JMZUFRQ29pV003bkdTWUZjRXBl?=
 =?utf-8?B?Nm85M2RFSHBKY1FITXRaa3pxdHZBNEpYRjZaSStRRHB1YXJicDlibUtxNDdH?=
 =?utf-8?B?T2xIYk9Hb3hteXowYWxaREZzQm1rUmNsRVNzVWlmWktUelBxNmMrSE1tOTRD?=
 =?utf-8?B?ZFhpNjV3dzlwb2s0L2YyTURRY0FGK0JuRFlneEZMdkJ2bGdqRHpPNE8zc01Z?=
 =?utf-8?B?RzhycSsvTkczK1FhNy9keXQzL000Q1dscjVvOXJVM1V4UG9xZ2J5SWM0U3F0?=
 =?utf-8?B?akFXeDBMR21McG9DNTkzTWpNbXVJR2RGNzJLaGIzSlBzaGd5K2lBbTZoMkJC?=
 =?utf-8?B?NlBaamwxM1QybmUvcm1wTGRIbGRkNjRMV3VPWEFJOUd0dXB5L0FvOWNnPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(7416005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?TXRPamRuUENxUE1welBYLzVFTWJnVnRUMVRMZFk0anR1RStVOWJSNFhSaVA2?=
 =?utf-8?B?elN2MGhBM1Jkd3pwSnhVWDhiUkdZaGQvbUtNeVVTZTgrcDcydjdCSm9zSEpC?=
 =?utf-8?B?OWh4QzBhMTF5QUg1K0xEODZaQVJObTF2OTBQdm1QSHdTTVJpalZqUDREd1BG?=
 =?utf-8?B?K01DdlRSUVJZMlkzZXhYbkw0czJqKytJQ2dSZVdGeTRGdUlLajF1aTdnQ1F6?=
 =?utf-8?B?eHNCM09CdVljTDFIWkVTdWgra1Mrc2ZVQlg0RGpqdC9VN0VFcWUvK2ZPVmlV?=
 =?utf-8?B?KzUxeG5HTTZKS0ZmMlJsclZpZmVja1k1RkpxUExhcFZYQS85RzlxMFFidkxo?=
 =?utf-8?B?cWZDTWdiZjZRZU15dDlkSHhTTDNSY1RkR1ozY2pXSzY5WVgxZWgxREtsK1ZH?=
 =?utf-8?B?eUtOYkFHWHFiRDNMTkZiMjVGL2xoa2FQYldGL3BtUjBxcFBjSGRIOUxYVlBP?=
 =?utf-8?B?a0ZpZitMTHBwYkxaa0Y0Kzh0OWU0VE5kdCtLRmFuM1NvRFlyNkVZK01wWFFy?=
 =?utf-8?B?OTlCYTF1bEUzcDhBN1E1MHFYOEV6OEFESkVXTFNHSHZOajhhNWR3aC8xQjln?=
 =?utf-8?B?VHFnRHh5WXk4YzdQUU1CY25Xck9xUUo3VFJsSjUwVU9kbGxjRzZrTE90Smdn?=
 =?utf-8?B?UDVNUFdHbmFVRUc3Rk54T0ZET3RBYXdIUUJhTWRXZkVNSGNOdUlmRVpoSk8z?=
 =?utf-8?B?OUN4bVRlemcwc2wxTUNFMTdXeG01L1ErUjlGVUYwOURPRVFOY2tBN2VEWUtN?=
 =?utf-8?B?d1c5T25IWTRkdFVKa3VKUzl3N0ljT3hzUUozVlRza0UwNXNOc3dzT3czMkZq?=
 =?utf-8?B?OVhXMENKOU9rY3NTdlU2ckRMbUJhelVXbWc4Q0Z4NS9CY3ljUDNnNFlnblBw?=
 =?utf-8?B?d0tjeVBXYlpFTnlYWVplWFY1MFBGRXZJT0dseUEvTTAvM2EyK2paY3VzVFRY?=
 =?utf-8?B?Rkg0MGYwN1AvL2ptOTZpUHdLNk8zSlN3U0pkaWRnSHRWdEJGUm1MY2xhTVZP?=
 =?utf-8?B?NTZpVG1uRnl3NU96MlV6OGpDekNINDZaVnc2ZDNvdHpOUDIwWlR3RnJhT25Q?=
 =?utf-8?B?LysvMUxoaWpXcFJrSHpSeEhKVDhwNGVtN3BBTkl5UWNlQmJlWHZTeUxOaHQ1?=
 =?utf-8?B?N3VCY2ozVHRiODU2a2NiNkcrZkc5V2pzaGZvMFlVaUYwRUlsamVvTlI1dXFn?=
 =?utf-8?B?elpqUTA0M0psSG9nalQvenRkU3VKV3ZzL0l1Qkl1Q0lMRHZ5UlJpVFM0cDlN?=
 =?utf-8?B?VEx2amh6bmJSaTBpaUFMbFg0M2tmVyt4VXBBVkZPdGlSRzl3blFNVGNLeWY3?=
 =?utf-8?B?Z0orM2hYaml2K2xRdEk4eFZFNXNVNkVOdEZOeTVlM0lXNE42dTIySFV3L3RK?=
 =?utf-8?B?SW9KbVV2STlsU3NSSDF0RkI4Uml1amV5YzRKcFFqNkZNdDl5a1NBZTB0WlR0?=
 =?utf-8?B?NFdzWlpRZFVwY0QwQlBmc3FrbFNMM2x0Ty9SZndZWVduUEQ5V0lrT0ZxMXlG?=
 =?utf-8?B?TEw2Q0d3QUVYU3huYS8rZFBJWnFCazRuajRTU2hkRjYyTFNPbkJJUTVkRjFh?=
 =?utf-8?B?RHRCMFNtRElVU0ticEtwTmQvODdsa2FITXFGcURRd3UvWndRRTFGODE2Yk9O?=
 =?utf-8?B?WG01dW54UGg4ZUdNNGVFSkRJVENkbEZLZUVlbHpEbEFHeWF6aElzZks5RHhE?=
 =?utf-8?B?eVk4TElYd0tNY0tTT04zWUFLcnhLdHNzNy9tRGFsSTJyT3RkV2lwZU83V3p2?=
 =?utf-8?B?VVVLSkV0WjJlK3l6ZExrVm1DVnhTY04rK1UvNTBHcW92eVEzT0tHNXFNNlpJ?=
 =?utf-8?B?RHdoWFJwWnBHWkNJaSs1WmV4NG1PS1JveW1uaDNDS1J2eXovSDJ1dmRydjRL?=
 =?utf-8?B?a3lGSXliTWZLTWxKemxQK2U4VkZRRGV6MitMU3hZdWdPeFEzaElka21acjdK?=
 =?utf-8?B?SEswWHR3TEF5QVFobmE5R1pLRmRnSm43TmlPTDFwVUV3bTFLYm1NRURjMUlM?=
 =?utf-8?B?Y3MxVjllNGhnQ01peWRWcDhNcDhPNmZkQVlOaXRhUGVrdW1HQU9VSkxLc0hw?=
 =?utf-8?B?Y3cremp6a0paOTNLTm1nUjh4RFNFL2cyQnNWbktDanRLYWI3dGVrWGN3b1Vp?=
 =?utf-8?B?UG9UYkw0UXR4bEtOZXlIN05vZE5hV0xJckt2OWVId1c3enlmaTVySXZUcVVX?=
 =?utf-8?B?Y1E9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	1U76y2NM4Gt8xyfUG13Msu/esMoDFBF/8+SMImqdra82cDLF9vJCrD3ArcSOlg4JKEJWKwdLv+pjgrC+fzmgIGHmMfbcBk2dmyPiANHuL7u0OhHI4YwLt3l+vIqJqCe9YR6RTdECTY39RzINtL1mWVcRva5k3kyJHh5CBoEYVnQLor4UDPR5XXkr7b8VoUwYay0+DkdaZ9dB04kW7fAQNRRBj+Tujmjif3q2dyC/unzCbZT9aNeOBrRJzD3si9zF1B/QZ5w/IU5/2GH2LWXhgW8IOGUg99dulFi163RzPgFil3XEUZnX3uaBHUQqrWdVC7TkwN9TdGMRi+Vuci4kHWmltLa17Tak0Ql7eFqhgFDQjOTmAgXIj83Qxmmhs+4D7sEIK/OjUxtzXEMp4+ZxOsFl5f+/x2Xk2AR3HHoi74u424tcwrCgkB6OFJU15RO1gtSEW65aZuZj74sE3HQkS7Min8XuGbloTQX3vhMzDR2Ds40OP6NUGB9rnddcG70Ssoy6yiHGaEu/uVbxkJLbPpXoa1BmXtjtOGt+ObNhO+k8J+bPnc0bTLAWpB/8Xi93GJejzE4QTKHlvNzH6Qe7fnyul1pcoIyzlBfPeXMHsdY=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 45f090b4-c36c-4784-2f2c-08dc81520b64
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 09:14:16.0164
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LIH/+ypdASzb5nw6yBMR1FZceOd2NaufGssVTJGzMIjpGtiTlDLyBMx5iHGijewB4yxNPpuMNvtdWRUIQFoJzA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB7462
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_05,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0
 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 suspectscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2405310068
X-Proofpoint-GUID: pHqA_BRyI_DnMPAOiUm1T0iBZo7dujbO
X-Proofpoint-ORIG-GUID: pHqA_BRyI_DnMPAOiUm1T0iBZo7dujbO

On 29/05/2024 06:04, Christoph Hellwig wrote:
> Remove all APIs that are unused now that sd and sr have been converted
> to the atomic queue limits API.
> 
> Signed-off-by: Christoph Hellwig<hch@lst.de>
> ---


Reviewed-by: John Garry <john.g.garry@oracle.com>


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:15:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:15:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733511.1139819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyMZ-0000th-HV; Fri, 31 May 2024 09:15:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733511.1139819; Fri, 31 May 2024 09:15:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyMZ-0000ta-Dv; Fri, 31 May 2024 09:15:51 +0000
Received: by outflank-mailman (input) for mailman id 733511;
 Fri, 31 May 2024 09:15:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hFcc=NC=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCyMX-0000tU-NL
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:15:49 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5dd67fae-1f2e-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:15:47 +0200 (CEST)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-35dcc681f70so662605f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:15:47 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35dd04ca9bbsm1419317f8f.31.2024.05.31.02.15.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 31 May 2024 02:15:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5dd67fae-1f2e-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717146947; x=1717751747; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=paCibT1IHYwx84XcJTJRVtmEMs2s09+dbMnVZuM2qLY=;
        b=d4SUI8bsx7Gui9TXMneywMrt61Qxx3nygzSwcUXoDIT11r0ERHYdKehlTtvTTBS/5O
         Yctql75izk1YUWey1DUpfqLP0pgrimMPcnpYUrqMsnFOsaqIFLZPIP8Ubfgc/4P5KIGD
         iHQhbj7hByzzshVhb0+vRrKzFTsAdUK0HD6xs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717146947; x=1717751747;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=paCibT1IHYwx84XcJTJRVtmEMs2s09+dbMnVZuM2qLY=;
        b=QD08DCPwSQ4trYbn8uq/E4YakdUkbSXtJcqfcWifiNqDjOIMY7jarD/4YIrxZcndpT
         J7nxcntfcs11dRVJejiQbdcKZRl8qW423yvOR7v9GIHZya3dm//dL08AYXC5DPUQamip
         4JuHxGYWOALBsCKS5RxksOnrHZ8Dtn2q1dG6xZ1ASOEa2djKY++V0VUjuz1EyqjNKzL8
         JlOwj+FVkBoO02KKAUc+vnxwrZov7yVYNEot3jORrmOVMCYF0fk1/vYWj40/Cr6/S+Ft
         XUohTukSx7q7THewPjin/O9VeSkZnGaa6MuSg5TtjqJQ6KTKZTY2qGNAA9U+TmA90UM3
         /+jg==
X-Forwarded-Encrypted: i=1; AJvYcCV2XW9rgRhsyyfNtWdcAY7aXsdFlUfajjXco+rUynKC4ixW/TRWzJWFWU/iyghQ3fPBA5ZnX8OA4jIq0/aqJQuQrkL6Pl/b9ndSTdT9i70=
X-Gm-Message-State: AOJu0Yx+7xjRp38DtlhvOdizr7K9whz+1WD1SM5EZRQVeWX+sd5lLQnm
	qcX37zi1plMgd0BTkRDJvmNE3FpzEgFMuXYFUCt7DWb+hXmH2U9PZJ2F1iQR8g8=
X-Google-Smtp-Source: AGHT+IG9h5ZvRnitqCSpawB+uRqQmxw24ER3YfXL0mYTtp0mnoSO9u6/ApbJalQT7Tk+y2gHOXCkUg==
X-Received: by 2002:a5d:68c6:0:b0:34f:f540:bdbc with SMTP id ffacd0b85a97d-35e0f2892aamr807792f8f.38.1717146946895;
        Fri, 31 May 2024 02:15:46 -0700 (PDT)
Date: Fri, 31 May 2024 11:15:45 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH for-4.19 3/9] xen/cpu: ensure get_cpu_maps() returns
 false if CPU operations are underway
Message-ID: <ZlmVQW_zptkm_n1d@macbook>
References: <20240529090132.59434-1-roger.pau@citrix.com>
 <20240529090132.59434-4-roger.pau@citrix.com>
 <597b0bf9-8949-41f3-a630-18937c219d8b@suse.com>
 <ZldDvH8GhhQcu5NX@macbook>
 <3ccdfec3-924a-4ef0-bf8d-653b6019aecb@suse.com>
 <ZldUgJRJvytRsjhu@macbook>
 <075168de-1a3f-4f23-a445-8ba74082fb8f@suse.com>
 <Zll8xQJiM83EZhTd@macbook>
 <e39c9e0c-2596-483f-a82d-86d424717081@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <e39c9e0c-2596-483f-a82d-86d424717081@suse.com>

On Fri, May 31, 2024 at 10:33:58AM +0200, Jan Beulich wrote:
> On 31.05.2024 09:31, Roger Pau Monné wrote:
> > On Fri, May 31, 2024 at 09:02:20AM +0200, Jan Beulich wrote:
> >> On 29.05.2024 18:14, Roger Pau Monné wrote:
> >>> On Wed, May 29, 2024 at 05:49:48PM +0200, Jan Beulich wrote:
> >>>> On 29.05.2024 17:03, Roger Pau Monné wrote:
> >>>>> On Wed, May 29, 2024 at 03:35:04PM +0200, Jan Beulich wrote:
> >>>>>> On 29.05.2024 11:01, Roger Pau Monne wrote:
> >>>>>>> Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from
> >>>>>>> a cpu_hotplug_{begin,done}() region the function will still return success,
> >>>>>>> because a CPU taking the rwlock in read mode after having taken it in write
> >>>>>>> mode is allowed.  Such behavior however defeats the purpose of get_cpu_maps(),
> >>>>>>> as it should always return false when called with a CPU hot{,un}plug operation
> >>>>>>> is in progress.
> >>>>>>
> >>>>>> I'm not sure I can agree with this. The CPU doing said operation ought to be
> >>>>>> aware of what it is itself doing. And all other CPUs will get back false from
> >>>>>> get_cpu_maps().
> >>>>>
> >>>>> Well, the CPU is aware in the context of cpu_{up,down}(), but not in
> >>>>> the interrupts that might be handled while that operation is in
> >>>>> progress, see below for a concrete example.
> >>>>>
> >>>>>>>  Otherwise the logic in send_IPI_mask() for example is wrong,
> >>>>>>> as it could decide to use the shorthand even when a CPU operation is in
> >>>>>>> progress.
> >>>>>>
> >>>>>> It's also not becoming clear what's wrong there: As long as a CPU isn't
> >>>>>> offline enough to not be in cpu_online_map anymore, it may well need to still
> >>>>>> be the target of IPIs, and targeting it with a shorthand then is still fine.
> >>>>>
> >>>>> The issue is in the online path: there's a window where the CPU is
> >>>>> online (and the lapic active), but cpu_online_map hasn't been updated
> >>>>> yet.  A specific example would be time_calibration() being executed on
> >>>>> the CPU that is running cpu_up().  That could result in a shorthand
> >>>>> IPI being used, but the mask in r.cpu_calibration_map not containing
> >>>>> the CPU that's being brought up online because it's not yet added to
> >>>>> cpu_online_map.  Then the number of CPUs actually running
> >>>>> time_calibration_rendezvous_fn won't match the weight of the cpumask
> >>>>> in r.cpu_calibration_map.
> >>>>
> >>>> I see, but maybe only partly. Prior to the CPU having its bit set in
> >>>> cpu_online_map, can it really take interrupts already? Shouldn't it be
> >>>> running with IRQs off until later, thus preventing it from making it
> >>>> into the rendezvous function in the first place? But yes, I can see
> >>>> how the IRQ (IPI) then being delivered later (once IRQs are enabled)
> >>>> might cause problems, too.
> >>>
> >>> The interrupt will get set in IRR and handled when interrupts are
> >>> enabled.
> >>>
> >>>>
> >>>> Plus, with how the rendezvous function is invoked (via
> >>>> on_selected_cpus() with the mask copied from cpu_online_map), the
> >>>> first check in smp_call_function_interrupt() ought to prevent the
> >>>> function from being called on the CPU being onlined. A problem would
> >>>> arise though if the IPI arrived later and call_data was already
> >>>> (partly or fully) overwritten with the next request.
> >>>
> >>> Yeah, there's a small window where the fields in call_data are out of
> >>> sync.
> >>>
> >>>>>> In any event this would again affect only the CPU leading the CPU operation,
> >>>>>> which should clearly know at which point(s) it is okay to send IPIs. Are we
> >>>>>> actually sending any IPIs from within CPU-online or CPU-offline paths?
> >>>>>
> >>>>> Yes, I've seen the time rendezvous happening while in the middle of a
> >>>>> hotplug operation, and the CPU coordinating the rendezvous being the
> >>>>> one doing the CPU hotplug operation, so get_cpu_maps() returning true.
> >>>>
> >>>> Right, yet together with ...
> >>>>
> >>>>>> Together with the earlier paragraph the critical window would be between the
> >>>>>> CPU being taken off of cpu_online_map and the CPU actually going "dead" (i.e.
> >>>>>> on x86: its LAPIC becoming unresponsive to other than INIT/SIPI). And even
> >>>>>> then the question would be what bad, if any, would happen to that CPU if an
> >>>>>> IPI was still targeted at it by way of using the shorthand. I'm pretty sure
> >>>>>> it runs with IRQs off at that time, so no ordinary IRQ could be delivered.
> >>>>>>
> >>>>>>> Adjust the logic in get_cpu_maps() to return false when the CPUs lock is
> >>>>>>> already hold in write mode by the current CPU, as read_trylock() would
> >>>>>>> otherwise return true.
> >>>>>>>
> >>>>>>> Fixes: 868a01021c6f ('rwlock: allow recursive read locking when already locked in write mode')
> >>>>>>
> >>>>>> I'm puzzled by this as well: Prior to that and the change referenced by its
> >>>>>> Fixes: tag, recursive spin locks were used. For the purposes here that's the
> >>>>>> same as permitting read locking even when the write lock is already held by
> >>>>>> the local CPU.
> >>>>>
> >>>>> I see, so the Fixes should be:
> >>>>>
> >>>>> x86/smp: use APIC ALLBUT destination shorthand when possible
> >>>>>
> >>>>> Instead, which is the commit that started using get_cpu_maps() in
> >>>>> send_IPI_mask().
> >>>>
> >>>> ... this I then wonder whether it's really only the condition in
> >>>> send_IPI_mask() which needs further amending, rather than fiddling with
> >>>> get_cpu_maps().
> >>>
> >>> That the other option, but I have impression it's more fragile to
> >>> adjust the condition in send_IPI_mask() rather than fiddle with
> >>> get_cpu_maps().
> >>>
> >>> However if that's the preference I can adjust.
> >>
> >> I guess we need other REST input here then. The two of us clearly disagree on
> >> what use of get_cpu_maps() is meant to guarantee. And I deem fiddling with
> >> common code here more risky (and more intrusive - the other change would be
> >> a single-line code change afaict, plus extending the related comment).
> > 
> > How do you envision that other change to be done?  Adding an extra
> > variable and toggling it in cpu_hotplug_{begin,done}() to signal
> > whether a CPU hotplug is in progress?
> 
> I was thinking of an is-write-locked-by-me check on cpu_add_remove_lock.

Oh, so basically open-coding what I proposed here as get_cpu_maps() in
send_IPI_mask().  Unless anyone else expresses interest in my current
proposal I would switch to that.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:41:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:41:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733527.1139829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyl7-0006RU-IK; Fri, 31 May 2024 09:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733527.1139829; Fri, 31 May 2024 09:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCyl7-0006RN-FZ; Fri, 31 May 2024 09:41:13 +0000
Received: by outflank-mailman (input) for mailman id 733527;
 Fri, 31 May 2024 09:41:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCyl5-0006RB-Pd; Fri, 31 May 2024 09:41:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCyl5-0004ev-LL; Fri, 31 May 2024 09:41:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCyl5-0000eN-EZ; Fri, 31 May 2024 09:41:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCyl5-0007l3-E3; Fri, 31 May 2024 09:41:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=WSu+Xy1B0+MZjupwPUj0sTeMgbMzYOmiuifM1izT6WM=; b=esNJS5BhVesG4fsKxO5LuKl4hD
	GcvV3GaTRJp5DFtfi6YRfje9EU+IGRmVqxTHJ2H6FiVO+qnX7KNuEajKgPDSAXrxqRP8wfULlM8hq
	XGJp0Pxeu5DXF3cnm/bk3Bu6Y8ssJ1emBFSZFcJzHr79aZ1WZQy7mU3dRashm/6iPVmw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186209-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186209: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=746cc5cc40bef22d606cd22d1feb10d73a7b3d11
X-Osstest-Versions-That:
    ovmf=5f68a363d0d95bd0d383861ae21886d9824a8cd4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 09:41:11 +0000

flight 186209 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186209/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 746cc5cc40bef22d606cd22d1feb10d73a7b3d11
baseline version:
 ovmf                 5f68a363d0d95bd0d383861ae21886d9824a8cd4

Last test of basis   186205  2024-05-31 02:11:12 Z    0 days
Testing same since   186209  2024-05-31 08:13:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Shang Qingyu <qingyu.shang@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   5f68a363d0..746cc5cc40  746cc5cc40bef22d606cd22d1feb10d73a7b3d11 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 09:53:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:53:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733535.1139838 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCywN-0000mm-G7; Fri, 31 May 2024 09:52:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733535.1139838; Fri, 31 May 2024 09:52:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCywN-0000mf-DT; Fri, 31 May 2024 09:52:51 +0000
Received: by outflank-mailman (input) for mailman id 733535;
 Fri, 31 May 2024 09:52:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCywM-0000mZ-Ct
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:52:50 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a20d834-1f33-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:52:49 +0200 (CEST)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-35dd0c06577so760864f8f.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:52:49 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35dd04caf05sm1456441f8f.42.2024.05.31.02.52.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:52:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a20d834-1f33-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717149169; x=1717753969; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=hf6kJtuim518JzfO4xCI8JR2wSGNLAwFB/Bm4BXuuKY=;
        b=hMUU6SVIH+4Lz9sj0cqHV1NmjzbaaQKWsbcJzx3l52IB3KJl+OdtTMsPd/5y8vkEYj
         xt9n90m0rUz/qSd4XT478TRzu85Q1NZX2QPC3nV0O/6zhsHD2+3BlTPwrDdhY75RaDoh
         +8AMf1S/7IqfQkpROSAw+M7+r1IgXHE9doN92lblwRpQMXqWDxQ/6npuDrrASJZ9lYH+
         PqAIysBUqbGITrYp49pkx0VcJMdIEC0CaNoCF2rgHpKSoX86QrychXiyVpjyU42p3vHt
         /jap7BeAt3RPJOx6u+3UWz9qwkNFdPEsQWl5EuZ68CWfFUbph1yFq1JLqWAPBIcdipHV
         FvEg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717149169; x=1717753969;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hf6kJtuim518JzfO4xCI8JR2wSGNLAwFB/Bm4BXuuKY=;
        b=B3ytWNiIeoLCy7+7OTv62Vrnoom27hVb2ESW8w8G2s6GaGicg3gpqTRDhJSihVDvis
         hj6HQToIfqBCzQuHEa0+47+euI/DIg8ePEluKtPbz8Hri+8amQuiRxFbYzbneCX5ixRw
         k+y6Agag/2A518I7DJSZh77Sv3VAC0uEmV/KfAvV6ZBI2GU1EKVMEaPs5Tu29uhtVuCO
         Wqd0LFBlFTtCOaO1terVL4SUiNc4XoOtkjD67aNk1bE2RKQoyJcgyE50l61caY3cShhO
         2jjo6pmRAV7zsFNTZWRLi2pFBqm3QmYfleWnLELaiqxcXhgd3iTtCvistEYvYZIEdqh2
         rXdw==
X-Forwarded-Encrypted: i=1; AJvYcCUJKlCltih6dOEkaBgGpBVHXa2DrLr2tC3UoX+I1FyxrC5f3Nz2OO1RbIVaFCdV1CYoJj8mu5zAXlMn63FJHcpsQIbge63GFh3DRpb7RkY=
X-Gm-Message-State: AOJu0YwNSxtQpPO+qYwvjewTx5LhzUn8CVOSeD7CPS63B9myz87yo3vk
	n/xruFmxp/T2RwCICVNku4A2pggKgizYEVAxhDO3Inh2pfKm1Xog
X-Google-Smtp-Source: AGHT+IEmAqd0XCpRFy8zmJwHnoCNgGkDW2xaUrnVWSz6NQR69vAe7JyIAXDN4o/nuWE0i9HYxGE4Aw==
X-Received: by 2002:adf:face:0:b0:34c:d5e8:faad with SMTP id ffacd0b85a97d-35e0f337d0dmr844739f8f.58.1717149168693;
        Fri, 31 May 2024 02:52:48 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <3c1d9624-90d4-4248-97ac-4bfea2ee273c@xen.org>
Date: Fri, 31 May 2024 10:52:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 4/7] hw/xen: Make XenDevOps structures const
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-5-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-5-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> Keep XenDevOps structures in .rodata.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 8 ++++----
>   hw/9pfs/xen-9p-backend.c            | 2 +-
>   hw/display/xenfb.c                  | 4 ++--
>   hw/usb/xen-usb.c                    | 4 ++--
>   4 files changed, 9 insertions(+), 9 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:57:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:57:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733540.1139848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz0Q-0001OW-3G; Fri, 31 May 2024 09:57:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733540.1139848; Fri, 31 May 2024 09:57:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz0Q-0001OP-0B; Fri, 31 May 2024 09:57:02 +0000
Received: by outflank-mailman (input) for mailman id 733540;
 Fri, 31 May 2024 09:57:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCz0P-0001OJ-Av
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:57:01 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1f32fa44-1f34-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:56:59 +0200 (CEST)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2e95abc7259so19961621fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:56:59 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-35dd04c0d98sm1473887f8f.24.2024.05.31.02.56.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:56:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f32fa44-1f34-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717149419; x=1717754219; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=eTxotFDg5HC5VufmvEJCZwC0zHWN9oWX0NtnsJNidHA=;
        b=klU4urjHDGHfnBKASdxynv2vI7Fsit405utplv6jJCHDinIgjRXLjR/X/CinyIFxad
         Y7x4dLdQFAY6Us67qHOOmznMLgXXuJSyjidYSGo4S+OCwycTCxBnetTUMjz1Iji7eiQu
         Zn4eTcjlMsP0pZ+XewDpoQOCoziua9DTzEXI0gs9ItzbYQfbSgmR7XZmeqE4f8oyyW1i
         8LP/mf/3iHdo/dTNPR8KaVGIzBrENSw/YFV3n0QRaHhh7xoBt3EqI+JIkXVqkcihHP5V
         HYc9kCJKUsAV/2k0Gjito3UODzZB5bcSy6bNR3tojwMPmJeprxO2N3IQgSyuJyCZMLp9
         OB4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717149419; x=1717754219;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eTxotFDg5HC5VufmvEJCZwC0zHWN9oWX0NtnsJNidHA=;
        b=ENDGUgUxc8MqyXPR4gchurQmYSHrS8cNRAMzd2tV5EW9hI4nDnFZguH2QGje+CmL8C
         f1WV3J4YmsjdqgGJU1DbopoIOspsSjmVITWOEpA2m/h+Pf36yCwAgGN2iEunyCNa2Tfs
         eWM4+M6Ko0d3TEOJUpNXrgBdCA3uYKp1ml0JVxgO8XcGJL8YaRd4uG9DJr5TBhZggXhP
         vYJsILwqb6CihQVamWKjq/9TRnkHofUfGw+6LPh12lbVIvOG1+VBdSED/NaF7gNMrWzm
         aHN8oB6HMg7lWy6dmp82ViVDByJzJMSIZCJWUynO15QoEheuRxdSI4AX7w+3az61ihGV
         HFNA==
X-Forwarded-Encrypted: i=1; AJvYcCUDYZTIcQk13/+BUS4QhzqCX0ySphlQYon5PzyzJl43uFcbHM4Rr1Ky0VlCE5YyLnre+lMKMZJuFZEOwaFsVSEc90oOQoArr00XdDN6EDw=
X-Gm-Message-State: AOJu0YwWHfxTlNxCpAVx8rcuGeqYPeqyu0REkVvTDYNwL+CRbVzgSe33
	AqQJmfFyY7qNGWnNhNGEZQ6rCsn85+EW+Y7APU9O14xGbJGDlbuU
X-Google-Smtp-Source: AGHT+IEeGf9pw8BCYVzjYWzmpiH+1eGH7JhqRd4dZK/8oxCqOjugFt64RvUpuwxlKzw/gO+6K7zG9A==
X-Received: by 2002:a2e:9903:0:b0:2e9:794c:19ae with SMTP id 38308e7fff4ca-2ea95122772mr10466311fa.23.1717149418688;
        Fri, 31 May 2024 02:56:58 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <cd6fec85-c5da-4b89-9ceb-cb10ef4048e4@xen.org>
Date: Fri, 31 May 2024 10:56:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 5/7] hw/xen: initialize legacy backends from
 xen_bus_init()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-6-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-6-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
> 
> Prepare for moving the calls to xen_be_register() under the
> control of xen_bus_init(), using the normal xen_backend_init()
> method that is used by the "modern" backends.
> 
> This requires the xenstore global variable to be initialized,
> which is done by xen_be_init().  To ensure that everything is
> ready at the time the xen_backend_init() functions are called,
> remove the xen_be_init() function from all the boards and
> place it directly in xen_bus_init().
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Message-ID: <20240509170044.190795-7-pbonzini@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   hw/i386/pc.c              | 1 -
>   hw/xen/xen-bus.c          | 4 ++++
>   hw/xen/xen-hvm-common.c   | 2 --
>   hw/xenpv/xen_machine_pv.c | 5 +----
>   4 files changed, 5 insertions(+), 7 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:57:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:57:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733544.1139859 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz10-0001u2-Ae; Fri, 31 May 2024 09:57:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733544.1139859; Fri, 31 May 2024 09:57:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz10-0001tv-7s; Fri, 31 May 2024 09:57:38 +0000
Received: by outflank-mailman (input) for mailman id 733544;
 Fri, 31 May 2024 09:57:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCz0y-0001tl-Ia
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:57:36 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3423b2e7-1f34-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 11:57:34 +0200 (CEST)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-4211e42e362so19715555e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:57:34 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-421270697b6sm49927995e9.24.2024.05.31.02.57.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:57:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3423b2e7-1f34-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717149454; x=1717754254; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=uu9NBKSJTgs+S9TqmVtiCE86u3gD06b0P6Ww5JBGrEU=;
        b=YZ82u2wNAwjkmpYEGuI9NaVj1e8puatmRe2rPdujgig2EeYCeJ0angHS3+Pv3ewFkz
         wPpg9weC4rYnvDYp/Uq4o9T8do+W86GlQVt1IwvLfJzhVjOdtP8JkL6iBYt4oFtxtBZU
         4PUlMMhCa3nO99vdt7OFG/ucOXlunyg/Pb2cZMjovMN0iuxq40TIlVch7mWe2gjs4V2E
         HGawfn/fRW8kaY/HS44/FL9q6xikWIG4ijMOh6e834fgBJunU5IDDm/Up8jkDlw26Q+A
         cj6kLDrkOWICRl3Nk3cCG6fJZ4wheCEE9h4Z+kbYwu71FD/48cXAybXhIFDfTG/kD088
         m1+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717149454; x=1717754254;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uu9NBKSJTgs+S9TqmVtiCE86u3gD06b0P6Ww5JBGrEU=;
        b=mpdxGwYcDALV3Yi31wqq22bYvr8X/B7GzUE/N4n2i6KBx2bw/r8OlY5V2bNu9cOiHo
         6nmHw1p07Drgqv7ZimY9R/cKpKpg93PGGA9xCG/HakiJ618yMgZI8HB9LIEG9/5AdL0q
         vk1OmuWFY9gKUqS+5te7dsxaAGU/4Umhv/6GR/KC2+Jzbqdue9qD/toYpoZyEhmI5ssV
         cxw2AVQMlth9yasTPnwTjhJ6KQwZbQvIsTGB8Yw6LeF2G0MrXm3AedgV7hKHknH0oqXC
         P9CH03if24xAlRml8IgAJ3UxN1MElxiWzqD1ff7gUvkjv6DFvStFczzgRzLXIoOCQN8p
         EgBg==
X-Forwarded-Encrypted: i=1; AJvYcCVEwSLPXAtD+AvRYVwDz5a4Ft06cfauDv46GI0fVcbDEpZpzwmW9PnsyVBN54hV990621p4Mh/TpbVpDa/7NG+iRbYFCKf+Oq29+T1LJYg=
X-Gm-Message-State: AOJu0YxSzlHp1CjLNlMQvqMoISBowRm0/6bJKknIziH4T3XFnaqSilMa
	zLKbMwZzVXIMLGTDlOBWXPwHW8ItpegEZL3VylW+9lRyAzQUknXf
X-Google-Smtp-Source: AGHT+IHf4eM0iKAXjhQyPe69MDAOzotT91YOZAZUcca/8FhBKouvh0wHz3U7tpeuUdH7yKkwcX0cHQ==
X-Received: by 2002:a05:600c:3547:b0:416:bc18:2a00 with SMTP id 5b1f17b1804b1-4212e0b5066mr12944045e9.38.1717149453760;
        Fri, 31 May 2024 02:57:33 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <660203c1-b6d9-4978-8336-1988247fd543@xen.org>
Date: Fri, 31 May 2024 10:57:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 6/7] hw/xen: register legacy backends via
 xen_backend_init
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-7-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-7-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
> 
> It is okay to register legacy backends in the middle of xen_bus_init().
> All that the registration does is record the existence of the backend
> in xenstore.
> 
> This makes it possible to remove them from the build without introducing
> undefined symbols in xen_be_init().  It also removes the need for the
> backend_register callback, whose only purpose is to avoid registering
> nonfunctional backends.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-ID: <20240509170044.190795-8-pbonzini@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 11 ++---------
>   include/hw/xen/xen_pvdev.h          |  1 -
>   hw/9pfs/xen-9p-backend.c            |  8 +++++++-
>   hw/display/xenfb.c                  |  8 +++++++-
>   hw/usb/xen-usb.c                    | 14 ++++----------
>   hw/xen/xen-legacy-backend.c         | 16 ----------------
>   6 files changed, 20 insertions(+), 38 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 09:59:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 09:59:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733547.1139868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz2h-0002Ta-L5; Fri, 31 May 2024 09:59:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733547.1139868; Fri, 31 May 2024 09:59:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCz2h-0002TT-I5; Fri, 31 May 2024 09:59:23 +0000
Received: by outflank-mailman (input) for mailman id 733547;
 Fri, 31 May 2024 09:59:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Umgu=NC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1sCz2f-0002TL-Qq
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 09:59:21 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 736b0d27-1f34-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 11:59:20 +0200 (CEST)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-421124a04d6so19730785e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 02:59:20 -0700 (PDT)
Received: from [192.168.3.251] (54-240-197-236.amazon.com. [54.240.197.236])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212b84ba40sm20025015e9.19.2024.05.31.02.59.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 02:59:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 736b0d27-1f34-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1717149560; x=1717754360; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=NWtVkOLDgF1rKwf3NbyJWvRkovccoP5yn6Z5++pjXzA=;
        b=NEe1LlvLvp8PqCHz0BRqviwU5Apul6r/Cz3BXaFPcAg0WXZltz8M62FxvUGbRx69mo
         gBDMRjYZ+mjwUakjG7e7dOmZGQp08VB2gFs4VktFB7DBPq41rrbffUj1XU0zXhg8rqU+
         Qbv/G5mcn/H7N6SVDCTPihku0JzDi0wbw2omhQhXqCLge7j3hwI79fuo8iG+ww7D6pnS
         oqvb18BP5Bdp0BTLZcGDkphN4zmBzVMNH1kx59tU2T1TsPGeQqeXSKeYh5RAVWA61o4c
         ur4Gq4K4TAh6QJGPiR12dKSm0M2jRsp6wsIi+oO4Iw8FZtSBS6XulDW+ynmSgyvWLhsP
         +eiQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717149560; x=1717754360;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=NWtVkOLDgF1rKwf3NbyJWvRkovccoP5yn6Z5++pjXzA=;
        b=qMeQf4cUHTW/0KmvI0XT8+tNkcIs2cPrG+mWzK/JtqYpByjKQ3UHiZq3gE0P0p39Ri
         V23ppvUnFCEyzXbR3AfUltPM/Jl31NdI9dBsOOoQfRsCmqdEXMRk/obkHmg+iySc6lAN
         pV+/FPXiNfWkY3yJCmTyZ9YfsnKSZTE3MTq/Thm2ni8uyKF204p0Wlhvk9xso99Ibaxu
         J3IfLYcLUTDJ+sm6cN3O1c3YwBugsUwWh9VH/E9fp03WlCytpmCBO8/jXUYKXZdjPpYm
         gikpd8ApnsDARAOB05VzraGX0lUMKO0DqVrVpdeGC6WxO+OLa9nFn/O4Wm2BzpurqbJL
         CKQQ==
X-Forwarded-Encrypted: i=1; AJvYcCXnEsJ4Qm2BU+pZC5Sfshkux+oygktxJaAFQEjEI3+K2wODwc95VmCyYIXevpmeFbOBIxuMkQVsvGFR9qebedfvcShu5Xqd3UI0jJeMiWE=
X-Gm-Message-State: AOJu0YwNBGaEYdCj3kdbYO9aKK8+jn1aeuAwnk+rF7Fyttb9qiYhCFWz
	1DpjKjK433f0KfkxSOOSqIOAsQFYJhgGJuNDxFvEs7Yn2gZCQrYZ
X-Google-Smtp-Source: AGHT+IFV51hSYt8NK9PiJAKTrsoXn7LryxkGS6g5ElhZz8ENQt9V721ZCsqMK+nfys/TO6a5gG7gkg==
X-Received: by 2002:a05:600c:4e15:b0:41f:f053:edb4 with SMTP id 5b1f17b1804b1-4212e076511mr9570605e9.23.1717149560202;
        Fri, 31 May 2024 02:59:20 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <0fe5773f-1524-4a9d-bdd8-606b18b4ced4@xen.org>
Date: Fri, 31 May 2024 10:59:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH v2 7/7] hw/xen: Register framebuffer backend via
 xen_backend_init()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Anthony PERARD <anthony@xenproject.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost
 <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Gerd Hoffmann <kraxel@redhat.com>
References: <20240510104908.76908-1-philmd@linaro.org>
 <20240510104908.76908-8-philmd@linaro.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240510104908.76908-8-philmd@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 10/05/2024 11:49, Philippe Mathieu-Daudé wrote:
> Align the framebuffer backend with the other legacy ones,
> register it via xen_backend_init() when '-vga xenfb' is
> used. It is safe because MODULE_INIT_XEN_BACKEND is called
> in xen_bus_realize(), long after CLI processing initialized
> the vga_interface_type variable.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/xen/xen-legacy-backend.h | 3 ---
>   hw/display/xenfb.c                  | 9 +++++++--
>   hw/xenpv/xen_machine_pv.c           | 2 --
>   3 files changed, 7 insertions(+), 7 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>



From xen-devel-bounces@lists.xenproject.org Fri May 31 10:16:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 10:16:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733555.1139880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzJD-0006gv-1Q; Fri, 31 May 2024 10:16:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733555.1139880; Fri, 31 May 2024 10:16:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzJC-0006go-TF; Fri, 31 May 2024 10:16:26 +0000
Received: by outflank-mailman (input) for mailman id 733555;
 Fri, 31 May 2024 10:16:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6UXI=NC=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1sCzJA-0006gX-TZ
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 10:16:25 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5044e86-1f36-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 12:16:23 +0200 (CEST)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a621cb07d8fso198895366b.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 03:16:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5044e86-1f36-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1717150583; x=1717755383; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=r1zQbci+pBPtGfS2ZlOjlBggHLolg5OzHHjzXdDK/fk=;
        b=aA4Nu6wKOWxgGgk5DMyWUKyuYWX1OVXFbbgIW/tJgDSpKYOpNaIXPjtnn8Fccni0Kd
         NlXFAJiKGnxDADeX/hcn0I5COCf0f1bNhA/RWQZ8taLJyePPt+rnfEiQgQufyGaPEgL6
         /+qfGxIXftjnP6y4Sah/okiAxyDJw5Ckz7+EU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717150583; x=1717755383;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=r1zQbci+pBPtGfS2ZlOjlBggHLolg5OzHHjzXdDK/fk=;
        b=HU6nc433Ei8BSabiMbmSiUP1KWQvQ6PXcJGySwBUSTompCUxCrdez8inCEJbTRnsVK
         468GNSMFXQgSJC8rBV7YMDctOs+OkSvYFctuXHGrtH05T8uxFzjdOBxPgReL6Eti2+fe
         WzbrutTQcwTWuysk3EgGC9DnUCwBVC5PeDXuGSkvY1l3ooxjqzE0k4Lkc2nzgeru+NAi
         U0+4UOqzWSDFnADzgGbMEWDT0U3FCR6+SsinppYP9npubzn2dIKj3LucyU2tjaITB6+S
         L3ZyXG55hLdY+S2zRrQ/Esrs8aPmojjP0660y0PMk/IduWl4Wytc6bG25z2CegR6I1+C
         UYdA==
X-Gm-Message-State: AOJu0Yy3pvFeGlCzxT7DVwCab0T8burWymB4X1asGOvLst9j17E90u2m
	1BtwHbfESxyvLTsFuMyYAY4Fy/0H3V6UoSCtBIRZijDsFF385gSSBzGN+soIa7sNSUw51XpP6FE
	isaHDe8VMgl5SpnNqfJw4jOWUoADXpaAQZhS2tCPr8d4zSDA+oRY=
X-Google-Smtp-Source: AGHT+IFhbwaTJkbhiQyrp23HfVhvmGwAxDUup0p5pnzcDhRiG00LVxqhMHWsuWs4/cuTuxLxv3ygF4NjqNSY3EYmvWk=
X-Received: by 2002:a17:906:63d1:b0:a59:aa69:9791 with SMTP id
 a640c23a62f3a-a68208fe2ecmr93247566b.34.1717150582498; Fri, 31 May 2024
 03:16:22 -0700 (PDT)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Fri, 31 May 2024 11:15:46 +0100
Message-ID: <CAO-mL=y+q2iUw-OHkHO96FSg1jfm8aQV-dFsMg4R0VS4+maOXg@mail.gmail.com>
Subject: [ANNOUNCE] Join us for free virtually at Xen Summit 2024 + FAQs!
To: xen-devel <xen-devel@lists.xenproject.org>, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Cc: committers@xenproject.org
Content-Type: multipart/alternative; boundary="0000000000007f0a4f0619bd4676"

--0000000000007f0a4f0619bd4676
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Xen Community,

Join us virtually at Xen Summit (4-6th June), for free using Jitsi!

We want to encourage as many of you to participate in the Xen Summit as
possible. Just because you are not physically attending, doesn't mean you
can't get involved.
*(Please note, there will be no professional AV or audio equipment, and it
is a community effort to enable others to get involved.) *

On the day, please find the *schedule of talks listed here.*
<https://events.linuxfoundation.org/xen-project-summit/program/schedule/>
I've also included some FAQs at the bottom of this email.

*Instructions:*

   - Our team will help host the main presentation talk sessions
   using Jitsi
   - Either myself or one attendee volunteer will help host each design
   session using Jitsi
   - Ensure you are using the Lisbon timezone
   - All sessions on Tuesday 4th June will be streamed using:
      - *SESSION PRESENTATIONS LINK <https://meet.jit.si/XenSummit24Main>*.
   - All sessions on Wednesday 5th June and Thursday 6th June will use the
   following links:
   - *DESIGN SESSION A <https://meet.jit.si/XenDesignSessionsA> *(Liberdade
      Room)
      - *DESIGN SESSION B <https://meet.jit.si/XenDesignSessionsB> *(August=
a
      Room)
      - Please look out on the schedule for the time and which session room
      it takes place in


   - Thursday design sessions will be finalized on the schedule by the end
   of day on Wednesday
   - The same links will be used throughout talks and sessions
   - (Optional) Join our Xen Summit matrix channel for updates on the day:
   https://matrix.to/#/#xen-project-summit:matrix.org

*Some ground rules to follow:*

   - Enter your full name on Jitsi so everyone knows who you are
   - Please mute yourself upon joining
   - Turning on cameras is optional, but we encourage doing this for design
   sessions
   - Do *not* shout out your questions during session presentations,
   instead ask these on the chat function and we will do our best to ask on
   behalf of you
   - During design sessions, we encourage you to unmute and participate
   freely
   - If multiple people wish to speak, please use the 'raise hand' function
   on Jitsi or chat
   - Should there be a need, moderators will have permission to remove
   anyone who is disruptive in sessions on Jitsi
   - If you face issues on the day, please let us know via Matrix - we will
   do our best to help, but please note this is a community effort

*Jitsi links:*

Session presentation link:

https://meet.jit.si/XenSummit24Main

Design Session A (Liberdade Room) link:
https://meet.jit.si/XenDesignSessionsA

Design Session B (Augusta Room) link: https://meet.jit.si/XenDesignSessions=
B

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=3DXenSummit24Main

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenSummit24Main#config.startSilent=3Dtrue

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsA

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenDesignSessionsA#config.startSilent=3Dtrue

See meeting dial-in numbers:
https://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsB

If also dialing-in through a room phone, join without connecting to audio:
https://meet.jit.si/XenDesignSessionsB#config.startSilent=3Dtrue

Schedule Example:

Tuesday 4th & Wednesday 5th June 2024

(Lisbon timezone)

Schedule Example: Wednesday 5th & Thursday 6th June 2024

(Lisbon timezone)

Schedule Example:  Wednesday 5th & Thursday 6th June 2024

(Lisbon timezone)

09:00
Welcome & Opening Remarks - Kelly Choi, Community Manager, Cloud Software
Group, XenServer

09:10

Xen Project 2024 Weather Report - Kelly Choi, XenServer, Cloud Software
Group.

LIBERDADE I

09:10

Challenges and Status of Enabling TrenchBoot in Xen Hypervisor - Micha=C5=
=82
=C5=BBygowski & Piotr Kr=C3=B3l, 3mdeb

13:45

The future of Xen Project physical events
<https://design-sessions.xenproject.org/uid/discussion/disc_OlJce1uK3uI0OjS=
2cL55/view>

LIBERDADE I

14:35

IOMMU paravirtualization and Xen IOMMU subsystem rework
<https://design-sessions.xenproject.org/uid/discussion/disc_PEgBNIXMyEkdJoy=
sE27O/view>

LIBERDADE I
09:10

Using Xenalyze for Performance Analysis - George Dunlap, Xen ServerAUGUSTA =
I





13:45

Downstream working group
<https://design-sessions.xenproject.org/uid/discussion/disc_z78Lt2EIZt2qxaS=
3FSlQ/view>

AUGUSTA I

14:35

Xen Safety Requirements upstreaming
<https://design-sessions.xenproject.org/uid/discussion/disc_01JE9EOi9zxxAU8=
lfB6Z/view>

AUGUSTA I



*How to filter the schedule:*


*FAQs:*

   - *My company would like to sponsor the next Xen Summit, how do I get
   involved?*
      - *Please email *@communitymanager
<community.manager@xenproject.org>* with
      your interest *
   - *Are sessions recorded?*
      - *Yes, all talks are recorded and will be available on YouTube after
      the event. Design sessions on Day 3 are not recorded. *
   - *Can I write about my experience at Xen Summit?*
      - *Yes! We encourage the community to spread the word through social
      media. Please tag us on X or LinkedIn so that we can reshare. *
   -
*Can we see the presenter's slides? *
      - *Some presenters may have uploaded their slides in advance. If
      available, you can view these by clicking on each session in the
screenshot
      above. *

We look forward to seeing you there, both virtually and physically!

Many thanks,
Kelly Choi

Community Manager
Xen Project

--0000000000007f0a4f0619bd4676
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello Xen Community,=C2=A0<div><br></div><div>Join us virt=
ually at Xen Summit (4-6th June), for free using=C2=A0Jitsi!=C2=A0</div><di=
v><br></div><div>We want to encourage as many of you to participate in the =
Xen Summit as possible. Just because you are not physically attending, does=
n&#39;t mean you can&#39;t get involved.=C2=A0</div><div><i>(Please note, t=
here will be no professional AV or audio equipment, and it is a community e=
ffort to enable others to get involved.)=C2=A0</i></div><div><br></div><div=
>On the day, please find the=C2=A0<a href=3D"https://events.linuxfoundation=
.org/xen-project-summit/program/schedule/" target=3D"_blank"><b>schedule=C2=
=A0of talks listed here.</b></a></div><div>I&#39;ve also included some FAQs=
 at the bottom of this email.</div><div><br></div><div><b><u>Instructions:<=
/u></b></div><div><ul><li style=3D"margin-left:15px">Our team will help hos=
t the main presentation talk sessions using=C2=A0Jitsi=C2=A0</li><li style=
=3D"margin-left:15px">Either myself or one attendee volunteer will help hos=
t each design session using=C2=A0Jitsi=C2=A0</li><li style=3D"margin-left:1=
5px">Ensure you are using the Lisbon timezone</li><li style=3D"margin-left:=
15px">All sessions on Tuesday 4th June will be streamed using:</li><ul><li =
style=3D"margin-left:15px"><b><a href=3D"https://meet.jit.si/XenSummit24Mai=
n" target=3D"_blank">SESSION PRESENTATIONS LINK</a></b>.=C2=A0</li></ul><li=
 style=3D"margin-left:15px">All sessions on Wednesday 5th June and Thursday=
 6th June will use the following links:<br></li><ul><li style=3D"margin-lef=
t:15px"><b><a href=3D"https://meet.jit.si/XenDesignSessionsA" target=3D"_bl=
ank">DESIGN SESSION A</a>=C2=A0</b>(Liberdade Room)=C2=A0</li><li style=3D"=
margin-left:15px"><b><a href=3D"https://meet.jit.si/XenDesignSessionsB" tar=
get=3D"_blank">DESIGN SESSION=C2=A0B</a>=C2=A0</b>(Augusta Room)</li><li st=
yle=3D"margin-left:15px">Please look out on the schedule for the time and w=
hich session room it takes place in<br></li></ul></ul><ul><li style=3D"marg=
in-left:15px">Thursday design sessions will be finalized on the schedule by=
 the end of day on Wednesday</li><li style=3D"margin-left:15px">The same li=
nks will be used throughout talks and sessions</li><li style=3D"margin-left=
:15px">(Optional) Join our Xen Summit matrix channel for updates on the day=
:=C2=A0<a href=3D"https://matrix.to/#/%23xen-project-summit:matrix.org" tar=
get=3D"_blank">https://matrix.to/#/#xen-project-summit:matrix.org</a>=C2=A0=
=C2=A0</li></ul><div><div><b><u>Some ground rules to follow:</u></b></div><=
div><ul><li style=3D"margin-left:15px">Enter your full name on Jitsi so eve=
ryone knows who you are</li><li style=3D"margin-left:15px">Please mute your=
self upon joining=C2=A0</li><li style=3D"margin-left:15px">Turning on camer=
as is optional, but we encourage doing this for design sessions</li><li sty=
le=3D"margin-left:15px">Do=C2=A0<u>not</u>=C2=A0shout out your questions du=
ring session presentations, instead ask these on the chat function and we w=
ill do our best to ask on behalf of you</li><li style=3D"margin-left:15px">=
During design sessions, we encourage you to unmute and participate freely</=
li><li style=3D"margin-left:15px">If multiple people wish to speak, please =
use the &#39;raise hand&#39; function on=C2=A0Jitsi or chat</li><li style=
=3D"margin-left:15px">Should there be a need, moderators will have permissi=
on to remove anyone who is disruptive in sessions on=C2=A0Jitsi</li><li sty=
le=3D"margin-left:15px">If you face issues on the day, please let us know v=
ia Matrix - we will do our best to help, but please note this is a communit=
y effort=C2=A0</li></ul><div><b><u>Jitsi links:</u></b></div></div></div></=
div><div><span id=3D"m_-3807238046455862930m_3017200869748370048gmail-docs-=
internal-guid-f9217878-7fff-07f7-1772-78aa156120b2"><div dir=3D"ltr" align=
=3D"left" style=3D"margin-left:0pt"><table style=3D"border:none;border-coll=
apse:collapse"><colgroup><col width=3D"208"><col width=3D"208"><col width=
=3D"208"></colgroup><tbody><tr style=3D"height:58.5pt"><td style=3D"border-=
width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:=
top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.728;=
margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family=
:Arial,sans-serif;color:rgb(0,0,0);background-color:transparent;font-weight=
:700;font-variant-numeric:normal;font-variant-east-asian:normal;font-varian=
t-alternates:normal;vertical-align:baseline">Session presentation link:</sp=
an></p><p dir=3D"ltr" style=3D"line-height:1.728;margin-top:0pt;margin-bott=
om:0pt"><a href=3D"https://meet.jit.si/XenSummit24Main" target=3D"_blank" s=
tyle=3D"text-decoration-line:none"><span style=3D"font-size:11pt;font-famil=
y:Arial,sans-serif;background-color:transparent;font-weight:700;font-varian=
t-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:nor=
mal;text-decoration-line:underline;vertical-align:baseline">https://meet.ji=
t.si/XenSummit24Main</span></a><span style=3D"font-size:11pt;font-family:Ar=
ial,sans-serif;color:rgb(0,0,0);background-color:transparent;font-weight:70=
0;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-a=
lternates:normal;vertical-align:baseline">=C2=A0</span></p></td><td style=
=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vert=
ical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-he=
ight:1.728;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;=
font-family:Arial,sans-serif;color:rgb(0,0,0);background-color:transparent;=
font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;=
font-variant-alternates:normal;vertical-align:baseline">Design Session A (L=
iberdade Room) link:=C2=A0</span><a href=3D"https://meet.jit.si/XenDesignSe=
ssionsA" target=3D"_blank" style=3D"text-decoration-line:none"><span style=
=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transparen=
t;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:norma=
l;font-variant-alternates:normal;text-decoration-line:underline;vertical-al=
ign:baseline">https://meet.jit.si/XenDesignSessionsA</span></a></p></td><td=
 style=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0=
);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"l=
ine-height:1.728;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size=
:11pt;font-family:Arial,sans-serif;color:rgb(0,0,0);background-color:transp=
arent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:n=
ormal;font-variant-alternates:normal;vertical-align:baseline">Design Sessio=
n B (Augusta Room) link:=C2=A0</span><a href=3D"https://meet.jit.si/XenDesi=
gnSessionsB" target=3D"_blank" style=3D"text-decoration-line:none"><span st=
yle=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transpa=
rent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:no=
rmal;font-variant-alternates:normal;text-decoration-line:underline;vertical=
-align:baseline">https://meet.jit.si/XenDesignSessionsB</span></a></p></td>=
</tr><tr style=3D"height:171.75pt"><td style=3D"border-width:0.681818pt;bor=
der-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;over=
flow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margi=
n-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;ba=
ckground-color:transparent;font-variant-numeric:normal;font-variant-east-as=
ian:normal;font-variant-alternates:normal;vertical-align:baseline">See meet=
ing dial-in numbers:=C2=A0</span><a href=3D"https://meet.jit.si/static/dial=
InInfo.html?room=3DXenSummit24Main" target=3D"_blank" style=3D"text-decorat=
ion-line:none"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;b=
ackground-color:transparent;font-variant-numeric:normal;font-variant-east-a=
sian:normal;font-variant-alternates:normal;text-decoration-line:underline;v=
ertical-align:baseline">https://meet.jit.si/static/dialInInfo.html?room=3DX=
enSummit24Main</span></a><span style=3D"font-size:11pt;font-family:Arial,sa=
ns-serif;background-color:transparent;font-variant-numeric:normal;font-vari=
ant-east-asian:normal;font-variant-alternates:normal;vertical-align:baselin=
e">=C2=A0</span></p><br><p dir=3D"ltr" style=3D"line-height:1.656;margin-to=
p:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sa=
ns-serif;background-color:transparent;font-variant-numeric:normal;font-vari=
ant-east-asian:normal;font-variant-alternates:normal;vertical-align:baselin=
e">If also dialing-in through a room phone, join without connecting to audi=
o:=C2=A0</span><a href=3D"https://meet.jit.si/XenSummit24Main#config.startS=
ilent=3Dtrue" target=3D"_blank" style=3D"text-decoration-line:none"><span s=
tyle=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:transp=
arent;font-variant-numeric:normal;font-variant-east-asian:normal;font-varia=
nt-alternates:normal;text-decoration-line:underline;vertical-align:baseline=
">https://meet.jit.si/XenSummit24Main#config.startSilent=3Dtrue</span></a><=
span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:=
transparent;font-variant-numeric:normal;font-variant-east-asian:normal;font=
-variant-alternates:normal;vertical-align:baseline">=C2=A0</span></p></td><=
td style=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0=
,0);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D=
"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-si=
ze:11pt;font-family:Arial,sans-serif;background-color:transparent;font-vari=
ant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:n=
ormal;vertical-align:baseline">See meeting dial-in numbers:=C2=A0</span><a =
href=3D"https://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessions=
A" target=3D"_blank" style=3D"text-decoration-line:none"><span style=3D"fon=
t-size:11pt;font-family:Arial,sans-serif;background-color:transparent;font-=
variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternat=
es:normal;text-decoration-line:underline;vertical-align:baseline">https://m=
eet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsA</span></a></p><=
br><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0=
pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-c=
olor:transparent;font-variant-numeric:normal;font-variant-east-asian:normal=
;font-variant-alternates:normal;vertical-align:baseline">If also dialing-in=
 through a room phone, join without connecting to audio:=C2=A0</span><a hre=
f=3D"https://meet.jit.si/XenDesignSessionsA#config.startSilent=3Dtrue" targ=
et=3D"_blank" style=3D"text-decoration-line:none"><span style=3D"font-size:=
11pt;font-family:Arial,sans-serif;background-color:transparent;font-variant=
-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:norm=
al;text-decoration-line:underline;vertical-align:baseline">https://meet.jit=
.si/XenDesignSessionsA#config.startSilent=3Dtrue</span></a></p></td><td sty=
le=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,0);ve=
rtical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-=
height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11p=
t;font-family:Arial,sans-serif;background-color:transparent;font-variant-nu=
meric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;=
vertical-align:baseline">See meeting dial-in numbers:=C2=A0</span><a href=
=3D"https://meet.jit.si/static/dialInInfo.html?room=3DXenDesignSessionsB" t=
arget=3D"_blank" style=3D"text-decoration-line:none"><span style=3D"font-si=
ze:11pt;font-family:Arial,sans-serif;background-color:transparent;font-vari=
ant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:n=
ormal;text-decoration-line:underline;vertical-align:baseline">https://meet.=
jit.si/static/dialInInfo.html?room=3DXenDesignSessionsB</span></a></p><br><=
p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt">=
<span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color=
:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;fon=
t-variant-alternates:normal;vertical-align:baseline">If also dialing-in thr=
ough a room phone, join without connecting to audio:=C2=A0</span><a href=3D=
"https://meet.jit.si/XenDesignSessionsB#config.startSilent=3Dtrue" target=
=3D"_blank" style=3D"text-decoration-line:none"><span style=3D"font-size:11=
pt;font-family:Arial,sans-serif;background-color:transparent;font-variant-n=
umeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal=
;text-decoration-line:underline;vertical-align:baseline">https://meet.jit.s=
i/XenDesignSessionsB#config.startSilent=3Dtrue</span></a></p></td></tr><tr =
style=3D"height:82.5pt"><td style=3D"border-width:0.681818pt;border-style:s=
olid;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:hidden=
"><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:0p=
t"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-co=
lor:transparent;font-weight:700;font-style:italic;font-variant-numeric:norm=
al;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-a=
lign:baseline">Schedule Example:=C2=A0</span></p><p dir=3D"ltr" style=3D"li=
ne-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:=
11pt;font-family:Arial,sans-serif;background-color:transparent;font-style:i=
talic;font-variant-numeric:normal;font-variant-east-asian:normal;font-varia=
nt-alternates:normal;vertical-align:baseline">Tuesday 4th &amp; Wednesday 5=
th June 2024</span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top=
:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,san=
s-serif;background-color:transparent;font-style:italic;font-variant-numeric=
:normal;font-variant-east-asian:normal;font-variant-alternates:normal;verti=
cal-align:baseline">(Lisbon timezone)</span></p></td><td style=3D"border-wi=
dth:0.681818pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:to=
p;padding:5pt;overflow:hidden"><p dir=3D"ltr" style=3D"line-height:1.656;ma=
rgin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:A=
rial,sans-serif;background-color:transparent;font-weight:700;font-style:ita=
lic;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant=
-alternates:normal;vertical-align:baseline">Schedule Example:=C2=A0</span><=
span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-color:=
transparent;font-style:italic;font-variant-numeric:normal;font-variant-east=
-asian:normal;font-variant-alternates:normal;vertical-align:baseline">Wedne=
sday 5th &amp; Thursday 6th June 2024</span></p><p dir=3D"ltr" style=3D"lin=
e-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:1=
1pt;font-family:Arial,sans-serif;background-color:transparent;font-style:it=
alic;font-variant-numeric:normal;font-variant-east-asian:normal;font-varian=
t-alternates:normal;vertical-align:baseline">(Lisbon timezone)</span></p></=
td><td style=3D"border-width:0.681818pt;border-style:solid;border-color:rgb=
(0,0,0);vertical-align:top;padding:5pt;overflow:hidden"><p dir=3D"ltr" styl=
e=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"fon=
t-size:11pt;font-family:Arial,sans-serif;background-color:transparent;font-=
weight:700;font-style:italic;font-variant-numeric:normal;font-variant-east-=
asian:normal;font-variant-alternates:normal;vertical-align:baseline">Schedu=
le Example:=C2=A0=C2=A0</span><span style=3D"font-size:11pt;font-family:Ari=
al,sans-serif;background-color:transparent;font-style:italic;font-variant-n=
umeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal=
;vertical-align:baseline">Wednesday 5th &amp; Thursday 6th June 2024</span>=
</p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-bottom:=
0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;background-=
color:transparent;font-style:italic;font-variant-numeric:normal;font-varian=
t-east-asian:normal;font-variant-alternates:normal;vertical-align:baseline"=
>(Lisbon timezone)</span></p><br></td></tr><tr style=3D"height:201.75pt"><t=
d style=3D"border-width:0.681818pt;border-style:solid;border-color:rgb(0,0,=
0);vertical-align:top;padding:5pt;overflow:hidden"><h3 style=3D"box-sizing:=
border-box;font-family:Inter,&quot;Avenir Next&quot;,&quot;Helvetica Neue&q=
uot;,&quot;Lucida Grande&quot;,Helvetica,Arial,sans-serif;font-weight:inher=
it;line-height:1.1;margin:17px 0px 0px;padding:0px;border:0px;outline:0px;f=
ont-size:14px;vertical-align:baseline;clear:left;color:rgb(153,153,153);flo=
at:left;text-align:right;width:90px;zoom:1">09:00</h3><div class=3D"gmail-s=
ched-container" style=3D"box-sizing:border-box;padding:0px;border-width:0px=
 0px 0px 1px;border-top-style:initial;border-right-style:initial;border-bot=
tom-style:initial;border-left-style:solid;border-top-color:initial;border-r=
ight-color:initial;border-bottom-color:initial;border-left-color:rgba(0,0,0=
,0.1);outline:0px;font-size:16px;font-family:Inter,&quot;Avenir Next&quot;,=
&quot;Helvetica Neue&quot;,&quot;Lucida Grande&quot;,Helvetica,Arial,sans-s=
erif;vertical-align:baseline;color:rgb(51,51,51)"><div class=3D"gmail-sched=
-container-inner" style=3D"box-sizing:border-box;margin:0px;padding:10px 0p=
x 5px 15px;border:0px;outline:0px;font-weight:inherit;font-family:inherit;v=
ertical-align:baseline"><span class=3D"event ev_3" style=3D"box-sizing:bord=
er-box;margin:0px 12px 12px 0px;padding:0px;border-width:1px;border-style:s=
olid;border-color:transparent rgba(0,0,0,0.15) rgba(0,0,0,0.15) transparent=
;outline:0px;font-weight:inherit;font-family:inherit;vertical-align:baselin=
e;background-color:rgb(166,237,231);border-radius:4px;display:block;float:l=
eft"><a href=3D"https://xenprojectsummit2024.sched.com/event/1bCKF/welcome-=
opening-remarks-kelly-choi-community-manager-cloud-software-group-xenserver=
?iframe=3Dyes&amp;w=3D100%&amp;sidebar=3Dyes&amp;bg=3Dno" class=3D"gmail-na=
me" id=3D"gmail-69e373000a4da4d8e181cc54aa83e898" style=3D"box-sizing:borde=
r-box;text-decoration-line:none;margin:0px;padding:6px 10px 7px;border:0px;=
outline:0px;font-size:15px;font-family:inherit;vertical-align:baseline;colo=
r:rgb(0,0,0);border-bottom-right-radius:3px;border-top-right-radius:3px;flo=
at:left;line-height:1.2;min-height:32px">Welcome &amp; Opening Remarks - Ke=
lly Choi, Community Manager, Cloud Software Group, XenServer<span class=3D"=
gmail-vs" style=3D"box-sizing:border-box;margin:5px 0px 0px;padding:0px;bor=
der:0px;outline:0px;font-size:12px;font-family:inherit;vertical-align:basel=
ine;text-transform:uppercase;clear:left;display:none;opacity:0.8"></span></=
a></span></div></div><br><p dir=3D"ltr" style=3D"line-height:1.656;margin-t=
op:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,s=
ans-serif;background-color:transparent;font-variant-numeric:normal;font-var=
iant-east-asian:normal;font-variant-alternates:normal;vertical-align:baseli=
ne">09:10=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-=
top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,=
sans-serif;background-color:transparent;font-variant-numeric:normal;font-va=
riant-east-asian:normal;font-variant-alternates:normal;vertical-align:basel=
ine">Xen Project 2024 Weather Report - Kelly Choi, XenServer, Cloud Softwar=
e Group.</span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt=
;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-se=
rif;background-color:transparent;font-variant-numeric:normal;font-variant-e=
ast-asian:normal;font-variant-alternates:normal;vertical-align:baseline">LI=
BERDADE I</span></p><br></td><td style=3D"border-width:0.681818pt;border-st=
yle:solid;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;overflow:h=
idden"><h3 style=3D"box-sizing:border-box;font-family:Inter,&quot;Avenir Ne=
xt&quot;,&quot;Helvetica Neue&quot;,&quot;Lucida Grande&quot;,Helvetica,Ari=
al,sans-serif;font-weight:inherit;line-height:1.1;margin:17px 0px 0px;paddi=
ng:0px;border:0px;outline:0px;font-size:14px;vertical-align:baseline;clear:=
left;color:rgb(153,153,153);float:left;text-align:right;width:90px;zoom:1">=
09:10=C2=A0</h3><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;ma=
rgin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif=
;background-color:transparent;font-variant-numeric:normal;font-variant-east=
-asian:normal;font-variant-alternates:normal;vertical-align:baseline"></spa=
n></p><div class=3D"gmail-sched-container" style=3D"box-sizing:border-box;p=
adding:0px;border-width:0px 0px 0px 1px;border-top-style:initial;border-rig=
ht-style:initial;border-bottom-style:initial;border-left-style:solid;border=
-top-color:initial;border-right-color:initial;border-bottom-color:initial;b=
order-left-color:rgba(0,0,0,0.1);outline:0px;font-size:16px;font-family:Int=
er,&quot;Avenir Next&quot;,&quot;Helvetica Neue&quot;,&quot;Lucida Grande&q=
uot;,Helvetica,Arial,sans-serif;vertical-align:baseline;color:rgb(51,51,51)=
"><div class=3D"gmail-sched-container-inner" style=3D"box-sizing:border-box=
;margin:0px;padding:10px 0px 5px 15px;border:0px;outline:0px;font-weight:in=
herit;font-family:inherit;vertical-align:baseline"><span class=3D"event ev_=
5" style=3D"box-sizing:border-box;margin:0px 12px 12px 0px;padding:0px;bord=
er-width:1px;border-style:solid;border-color:transparent rgba(0,0,0,0.15) r=
gba(0,0,0,0.15) transparent;outline:0px;font-weight:inherit;font-family:inh=
erit;vertical-align:baseline;background-color:rgb(109,209,61);border-radius=
:4px;display:block;float:left"><a href=3D"https://xenprojectsummit2024.sche=
d.com/event/1bCFO/challenges-and-status-of-enabling-trenchboot-in-xen-hyper=
visor-michal-zygowski-piotr-krol-3mdeb?iframe=3Dyes&amp;w=3D100%&amp;sideba=
r=3Dyes&amp;bg=3Dno" class=3D"gmail-name" id=3D"gmail-875c64b7afbdbd049318b=
5585b9120f8" style=3D"box-sizing:border-box;text-decoration-line:none;margi=
n:0px;padding:6px 10px 7px;border:0px;outline:0px;font-size:15px;font-famil=
y:inherit;vertical-align:baseline;color:rgb(0,0,0);border-bottom-right-radi=
us:3px;border-top-right-radius:3px;float:left;line-height:1.2;min-height:32=
px">Challenges and Status of Enabling TrenchBoot in Xen Hypervisor - Micha=
=C5=82 =C5=BBygowski &amp; Piotr Kr=C3=B3l, 3mdeb<span class=3D"gmail-vs" s=
tyle=3D"box-sizing:border-box;margin:5px 0px 0px;padding:0px;border:0px;out=
line:0px;font-size:12px;font-family:inherit;vertical-align:baseline;text-tr=
ansform:uppercase;clear:left;display:none;opacity:0.8"></span></a></span></=
div></div><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-b=
ottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;backg=
round-color:transparent;font-variant-numeric:normal;font-variant-east-asian=
:normal;font-variant-alternates:normal;vertical-align:baseline">13:45=C2=A0=
</span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;margin-=
bottom:0pt"><a href=3D"https://design-sessions.xenproject.org/uid/discussio=
n/disc_OlJce1uK3uI0OjS2cL55/view" target=3D"_blank" style=3D"text-decoratio=
n-line:none"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;fon=
t-variant-numeric:normal;font-variant-east-asian:normal;font-variant-altern=
ates:normal;text-decoration-line:underline;vertical-align:baseline">The fut=
ure of Xen Project physical events</span></a></p><p dir=3D"ltr" style=3D"li=
ne-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:=
11pt;font-family:Arial,sans-serif;background-color:transparent;font-variant=
-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:norm=
al;vertical-align:baseline">LIBERDADE I</span></p><br><p dir=3D"ltr" style=
=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font=
-size:11pt;font-family:Arial,sans-serif;background-color:transparent;font-v=
ariant-numeric:normal;font-variant-east-asian:normal;font-variant-alternate=
s:normal;vertical-align:baseline">14:35=C2=A0</span></p><p dir=3D"ltr" styl=
e=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a href=3D"https:/=
/design-sessions.xenproject.org/uid/discussion/disc_PEgBNIXMyEkdJoysE27O/vi=
ew" target=3D"_blank" style=3D"text-decoration-line:none"><span style=3D"fo=
nt-size:11pt;font-family:Arial,sans-serif;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-variant-alternates:normal;text-decoration-li=
ne:underline;vertical-align:baseline">IOMMU paravirtualization and Xen IOMM=
U subsystem rework</span></a></p><p dir=3D"ltr" style=3D"line-height:1.656;=
margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family=
:Arial,sans-serif;background-color:transparent;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">LIBERDADE I</span></p></td><td style=3D"border-width:0.681818pt=
;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:5pt;=
overflow:hidden"><h3 style=3D"box-sizing:border-box;font-family:Inter,&quot=
;Avenir Next&quot;,&quot;Helvetica Neue&quot;,&quot;Lucida Grande&quot;,Hel=
vetica,Arial,sans-serif;font-weight:inherit;line-height:1.1;margin:17px 0px=
 0px;padding:0px;border:0px;outline:0px;font-size:14px;vertical-align:basel=
ine;clear:left;color:rgb(153,153,153);text-align:right;width:90px;zoom:1">0=
9:10=C2=A0</h3><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;mar=
gin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif;=
background-color:transparent;font-variant-numeric:normal;font-variant-east-=
asian:normal;font-variant-alternates:normal;vertical-align:baseline"><span =
class=3D"event ev_5" style=3D"box-sizing:border-box;margin:0px 12px 12px 0p=
x;padding:0px;border-width:1px;border-style:solid;border-color:transparent =
rgba(0,0,0,0.15) rgba(0,0,0,0.15) transparent;outline:0px;font-size:16px;fo=
nt-family:Inter,&quot;Avenir Next&quot;,&quot;Helvetica Neue&quot;,&quot;Lu=
cida Grande&quot;,Helvetica,Arial,sans-serif;vertical-align:baseline;backgr=
ound-color:rgb(109,209,61);border-radius:4px;display:block;float:left;color=
:rgb(51,51,51)"><a href=3D"https://xenprojectsummit2024.sched.com/event/1bC=
FR/using-xenalyze-for-performance-analysis-george-dunlap-xen-server?iframe=
=3Dyes&amp;w=3D100%&amp;sidebar=3Dyes&amp;bg=3Dno" class=3D"gmail-name" id=
=3D"gmail-0cf68d8b5cd7b1b5c490b568c76b39f5" style=3D"box-sizing:border-box;=
text-decoration-line:none;margin:0px;padding:6px 10px 7px;border:0px;outlin=
e:0px;font-size:15px;font-family:inherit;vertical-align:baseline;color:rgb(=
0,0,0);border-bottom-right-radius:3px;border-top-right-radius:3px;float:lef=
t;line-height:1.2;min-height:32px">Using Xenalyze for Performance Analysis =
- George Dunlap, Xen Server<span class=3D"gmail-vs" style=3D"box-sizing:bor=
der-box;margin:5px 0px 0px;padding:0px;border:0px;outline:0px;font-size:12p=
x;font-family:inherit;vertical-align:baseline;text-transform:uppercase;clea=
r:left;display:none;opacity:0.8">AUGUSTA I</span></a></span><br class=3D"gm=
ail-Apple-interchange-newline"></span></p><p dir=3D"ltr" style=3D"line-heig=
ht:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;fo=
nt-family:Arial,sans-serif;background-color:transparent;font-variant-numeri=
c:normal;font-variant-east-asian:normal;font-variant-alternates:normal;vert=
ical-align:baseline"><br></span></p><p dir=3D"ltr" style=3D"line-height:1.6=
56;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-fam=
ily:Arial,sans-serif;background-color:transparent;font-variant-numeric:norm=
al;font-variant-east-asian:normal;font-variant-alternates:normal;vertical-a=
lign:baseline"><br></span></p><p dir=3D"ltr" style=3D"line-height:1.656;mar=
gin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Ar=
ial,sans-serif;background-color:transparent;font-variant-numeric:normal;fon=
t-variant-east-asian:normal;font-variant-alternates:normal;vertical-align:b=
aseline"><br></span></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-to=
p:0pt;margin-bottom:0pt"><br></p><p dir=3D"ltr" style=3D"line-height:1.656;=
margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family=
:Arial,sans-serif;background-color:transparent;font-variant-numeric:normal;=
font-variant-east-asian:normal;font-variant-alternates:normal;vertical-alig=
n:baseline">13:45=C2=A0</span></p><p dir=3D"ltr" style=3D"line-height:1.656=
;margin-top:0pt;margin-bottom:0pt"><a href=3D"https://design-sessions.xenpr=
oject.org/uid/discussion/disc_z78Lt2EIZt2qxaS3FSlQ/view" target=3D"_blank" =
style=3D"text-decoration-line:none"><span style=3D"font-size:11pt;font-fami=
ly:Arial,sans-serif;font-variant-numeric:normal;font-variant-east-asian:nor=
mal;font-variant-alternates:normal;text-decoration-line:underline;vertical-=
align:baseline">Downstream working group</span></a></p><p dir=3D"ltr" style=
=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"font=
-size:11pt;font-family:Arial,sans-serif;background-color:transparent;font-v=
ariant-numeric:normal;font-variant-east-asian:normal;font-variant-alternate=
s:normal;vertical-align:baseline">AUGUSTA I</span></p><br><p dir=3D"ltr" st=
yle=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style=3D"f=
ont-size:11pt;font-family:Arial,sans-serif;background-color:transparent;fon=
t-variant-numeric:normal;font-variant-east-asian:normal;font-variant-altern=
ates:normal;vertical-align:baseline">14:35=C2=A0</span></p><p dir=3D"ltr" s=
tyle=3D"line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a href=3D"http=
s://design-sessions.xenproject.org/uid/discussion/disc_01JE9EOi9zxxAU8lfB6Z=
/view" target=3D"_blank" style=3D"text-decoration-line:none"><span style=3D=
"font-size:11pt;font-family:Arial,sans-serif;font-variant-numeric:normal;fo=
nt-variant-east-asian:normal;font-variant-alternates:normal;text-decoration=
-line:underline;vertical-align:baseline">Xen Safety Requirements upstreamin=
g</span></a></p><p dir=3D"ltr" style=3D"line-height:1.656;margin-top:0pt;ma=
rgin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Arial,sans-serif=
;background-color:transparent;font-variant-numeric:normal;font-variant-east=
-asian:normal;font-variant-alternates:normal;vertical-align:baseline">AUGUS=
TA I</span></p><br><br></td></tr></tbody></table></div></span></div><div><s=
pan id=3D"m_-3807238046455862930m_3017200869748370048m_2927081184909999839m=
_-8370998895675947915gmail-docs-internal-guid-7c024368-7fff-f910-ae50-70b83=
521ee30"><div><br></div><b><u>How to filter the schedule:</u></b><br><span =
style=3D"border:none;display:inline-block;overflow:hidden;width:624px;heigh=
t:411px"><img src=3D"https://lh7-us.googleusercontent.com/nJqdiQUkRhper7xaN=
fEg5kdR7Jsn_7Mb7Muwgz_IaImd4-ZOK0D5o9_jypDXHX2ybCsevBnzCISatQZxOIu5eEkBWWsY=
jYpXNCTw3khSi5qySJoq7hzZ5HPsoLxAJOQ5KjEzJXfvgVnLc4s_53AqXuE" width=3D"624" =
height=3D"411" class=3D"gmail-CToWUd gmail-a6T" tabindex=3D"0" style=3D"cur=
sor: pointer; outline: 0px; margin-left: 0px; margin-top: 0px;"></span></sp=
an><br></div><div><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"lt=
r"><div><br></div><div><b><u>FAQs:</u></b></div><div><ul><li><b>My company =
would like to sponsor the next Xen Summit, how do I get involved?</b></li><=
ul><li><i>Please email=C2=A0</i><a class=3D"gmail_plusreply" id=3D"m_825124=
0063996215973plusReplyChip-1" href=3D"mailto:community.manager@xenproject.o=
rg" target=3D"_blank">@communitymanager</a><i>=C2=A0with your interest=C2=
=A0</i></li></ul><li><b>Are sessions recorded?</b></li><ul><li><i>Yes, all =
talks are recorded and will be available=C2=A0on YouTube after the event. D=
esign sessions on Day 3 are not recorded.=C2=A0</i></li></ul><li><b>Can I w=
rite about my experience at Xen Summit?</b></li><ul><li><i>Yes! We encourag=
e the community to spread the word through social media. Please tag us on X=
 or LinkedIn so that we can reshare.=C2=A0</i></li></ul><li><i><b>Can we se=
e the presenter&#39;s slides?</b><br></i></li><ul><li><i>Some presenters ma=
y have uploaded their slides in advance. If available, you can view these b=
y clicking on each session in the screenshot above.=C2=A0</i></li></ul></ul=
></div><div>We look forward to seeing you there, both virtually and physica=
lly!</div><div><br></div></div></div></div><div><div dir=3D"ltr" class=3D"g=
mail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Ma=
ny thanks,</div><div>Kelly Choi</div><div><br></div><div><div style=3D"colo=
r:rgb(136,136,136)">Community Manager</div><div style=3D"color:rgb(136,136,=
136)">Xen Project=C2=A0<br></div></div></div></div></div></div>

--0000000000007f0a4f0619bd4676--


From xen-devel-bounces@lists.xenproject.org Fri May 31 10:32:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 10:32:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733617.1139917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzYJ-0004Cy-Gb; Fri, 31 May 2024 10:32:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733617.1139917; Fri, 31 May 2024 10:32:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzYJ-0004Cr-DI; Fri, 31 May 2024 10:32:03 +0000
Received: by outflank-mailman (input) for mailman id 733617;
 Fri, 31 May 2024 10:32:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hFcc=NC=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1sCzYI-0004C0-Kf
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 10:32:02 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 037f3f0a-1f39-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 12:32:00 +0200 (CEST)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-420180b5897so8719285e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 03:32:00 -0700 (PDT)
Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-4212278b24dsm40556385e9.0.2024.05.31.03.31.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 31 May 2024 03:31:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 037f3f0a-1f39-11ef-b4bb-af5377834399
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717151520; x=1717756320; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+hVjKyR2L2RSS9i/nt6ABD4p23g93I9Yei3VnU4r534=;
        b=EZ2GATAguwFBJ1bNOoYD4C9EVaeh0ShAeJYGI9pUsKbK1WO4+nvcZRRF3RZe4/eI2K
         SQi4rwt7AzXSQFZyT1GcnQmX+A+zRD4L5o/omP8KaNw5+/HXKr7uMPZaMYZh5bjPIkKG
         T1S7YDfHx/PYbQ/dUiGcuZjdvb/pN5EdSlUWU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717151520; x=1717756320;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+hVjKyR2L2RSS9i/nt6ABD4p23g93I9Yei3VnU4r534=;
        b=g1DJrmMCG0+ZUMRjLOVnim4h1XmbwfNoPMQfdcRkoKzGRkH8SoANErmXDu1x9VNGlP
         Gk2gUE7N4q7Zd6pqp9XXGiuroIiBZL0phObomZE6kmlfCdnhbmClX5/wlg/IJcghwidZ
         y3gyPvBuEjmkbiHLEjpxIVrSt0svCYxsc4MxY3C4v/VPeItZcVW034/MxywXWE2PL2jF
         W62SI4XB5/K9UpiViRHa43CJmI2IcFZY7G3Yuh1bubAbK1LEsUFpKHsl7bRAP6ViqN71
         ntWWWJPAcJGJW+9sFWy5q9utnPd+aXLeWa4Nq3EGFakEh9/Jbcl1E8Qbdf4ZLYlPGjPd
         3v6Q==
X-Forwarded-Encrypted: i=1; AJvYcCV72MpVBTifD/UxF3CHrf+oD11CXXr/SGS5TuziGtnXOpsGA3KDLITOVR8JPg+gvR8SZnFJOkfojeJWA5rzbUWtccjQcgwfHdZrDKtbumo=
X-Gm-Message-State: AOJu0YxbJVPdLMTBRXKVmupqpWdB7FC8PF4uHRBB9qbhehtOVruc4kgT
	neOfYWyW3x6vDPPjUP9CJDSDUYJj1IvUSkIl7wIBA8AvBa5n6uaibW8I8l1K27k=
X-Google-Smtp-Source: AGHT+IHIwawhCwmBBXhQPNTm5AUwRjpTeOHTwEpbkar9LLKV7/XruSjqZeZgaK8GLbPXtx3R30rdFQ==
X-Received: by 2002:a05:600c:1c97:b0:41a:e995:b924 with SMTP id 5b1f17b1804b1-4212e0c222cmr12203965e9.33.1717151519944;
        Fri, 31 May 2024 03:31:59 -0700 (PDT)
Date: Fri, 31 May 2024 12:31:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v3 2/6] xen/x86: Add initial x2APIC ID to the per-vLAPIC
 save area
Message-ID: <ZlmnHnHVbaxLjfjO@macbook>
References: <cover.1716976271.git.alejandro.vallejo@cloud.com>
 <9912423b866ed696c375e0a51954d363c3706470.1716976271.git.alejandro.vallejo@cloud.com>
 <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0d3101bc-624e-43b0-95e7-cc89de3bb259@citrix.com>

On Thu, May 30, 2024 at 12:08:26PM +0100, Andrew Cooper wrote:
> On 29/05/2024 3:32 pm, Alejandro Vallejo wrote:
> > diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c
> > index f033d22785be..b70b22d55fcf 100644
> > --- a/xen/lib/x86/policy.c
> > +++ b/xen/lib/x86/policy.c
> > @@ -2,6 +2,17 @@
> >  
> >  #include <xen/lib/x86/cpu-policy.h>
> >  
> > +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id)
> > +{
> > +    /*
> > +     * TODO: Derive x2APIC ID from the topology information inside `p`
> > +     *       rather than from the vCPU ID alone. This bodge is a temporary
> > +     *       measure until all infra is in place to retrieve or derive the
> > +     *       initial x2APIC ID from migrated domains.
> > +     */
> > +    return id * 2;
> > +}
> > +
> 
> I'm afraid it's nonsensical to try and derive x2APIC ID from a
> policy+vcpu_id.
> 
> Take a step back, and think the data through.
> 
> A VM has:
> * A unique APIC_ID for each vCPU
> * Info in CPUID describing how to decompose the APIC_ID into topology
> 
> Right now, because this is all completely broken, we have:
> * Hardcoded APIC_ID = vCPU_ID * 2
> * Total nonsense in CPUID
> 
> 
> When constructing a VM, the toolstack (given suitable admin
> guidance/defaults) *must* choose both:
> * The APIC_ID themselves
> * The CPUID topo data to match
> 
> i.e. this series should be editing the toolstack's call to
> xc_domain_hvm_setcontext().
> 
> It's not, because AFAICT you're depending on the migration compatibility
> logic and inserting a new hardcoded assumption about symmetry of the layout.
> 
> 
> The data flows we need are:
> 
> (New) create:
> * Toolstack chooses both parts of topo information
> * Xen needs a default, which reasonably can be APIC_ID=vCPU_ID when the
> rest of the data flow has been cleaned up.  But this is needs to be
> explicit in vcpu_create() and without reference to the policy.

Doesn't using APIC_ID=vCPU_ID limits us to only being able to expose
certain typologies? (as vCPU IDs are contiguous).  For example
exposing a topology with 3 cores per package won't be possible?

Not saying it's a bad move to start this way, but if we want to
support exposing more exotic topology sooner or later we will need
some kind of logic that assigns the APIC IDs based on the knowledge of
the expected topology.  Whether is gets such knowledge from the CPU
policy or directly from the toolstack is another question.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri May 31 10:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 10:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733627.1139928 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzjh-0006cz-Iy; Fri, 31 May 2024 10:43:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733627.1139928; Fri, 31 May 2024 10:43:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sCzjh-0006cs-Dq; Fri, 31 May 2024 10:43:49 +0000
Received: by outflank-mailman (input) for mailman id 733627;
 Fri, 31 May 2024 10:43:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCzjg-0006ci-MD; Fri, 31 May 2024 10:43:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCzjg-0005kJ-Ji; Fri, 31 May 2024 10:43:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sCzjg-0002Jo-Ci; Fri, 31 May 2024 10:43:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sCzjg-00041I-CG; Fri, 31 May 2024 10:43:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0Wius/mHdbyomOBraw2vo5uwnYONcmSusBlD0ILmnlI=; b=egnra0GHlGMVkkuE5n0fvgm7HQ
	0n4LMaBlUD78gQJZcXkuuWCavk2T8+B7dlNKziAr64BO7DgRmplXQIiJI++YEcK8CyWLZdLKcsSLx
	xfqzbnDgzDcRiNpR/CEepLfqlNMkBLI922LFAHSJKHmoq2H0teDC54+kwbwvz1QGlNwE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186206-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 186206: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=6f293f1fadc08660ba470d2cd3a91fde58cef617
X-Osstest-Versions-That:
    libvirt=2ea493598fd5a3efe4f25d842b2f09bc5c74c6c1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 10:43:48 +0000

flight 186206 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186206/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186190
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 libvirt              6f293f1fadc08660ba470d2cd3a91fde58cef617
baseline version:
 libvirt              2ea493598fd5a3efe4f25d842b2f09bc5c74c6c1

Last test of basis   186190  2024-05-30 04:22:11 Z    1 days
Testing same since   186206  2024-05-31 04:22:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  Göran Uddeborg <goeran@uddeborg.se>
  Michal Privoznik <mprivozn@redhat.com>
  Weblate <noreply@weblate.org>
  Yuri Chornoivan <yurchor@ukr.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   2ea493598f..6f293f1fad  6f293f1fadc08660ba470d2cd3a91fde58cef617 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 11:12:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:12:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733637.1139937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0Bo-0004Qk-Ng; Fri, 31 May 2024 11:12:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733637.1139937; Fri, 31 May 2024 11:12:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0Bo-0004Qd-Ju; Fri, 31 May 2024 11:12:52 +0000
Received: by outflank-mailman (input) for mailman id 733637;
 Fri, 31 May 2024 11:12:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sD0Bn-0004QX-7t
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:12:51 +0000
Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com
 [205.220.165.32]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b5b7fb7e-1f3e-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 13:12:48 +0200 (CEST)
Received: from pps.filterd (m0333521.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V9VkQ2005534; Fri, 31 May 2024 11:12:21 GMT
Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta02.appoci.oracle.com [147.154.114.232])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8g4ayta-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:12:21 +0000
Received: from pps.filterd
 (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44VAqSDC016232; Fri, 31 May 2024 11:12:20 GMT
Received: from nam10-mw2-obe.outbound.protection.outlook.com
 (mail-mw2nam10lp2043.outbound.protection.outlook.com [104.47.55.43])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc50tqwv5-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:12:20 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by PH0PR10MB7007.namprd10.prod.outlook.com (2603:10b6:510:282::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 11:12:18 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 11:12:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5b7fb7e-1f3e-11ef-b4bb-af5377834399
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3DNrlvPCyMcyqWI/tJUTwItb4RxylbxvVIA4qhSg7km+Q=3D;_b?=
 =?UTF-8?Q?=3DnUW4xs7f5PO9r50nppwk98Hp+BJj1PlsdTaQvIsCNMcqLEa+xtCjOxGr+EBG?=
 =?UTF-8?Q?wI45Z6PL_p7tkWy0NBBSMfau3hUwoDrlwXvjC5XgBCrfPgfY8ADSQX7nI2dk3lP?=
 =?UTF-8?Q?1TaM7EmWql5KJp_oV7S03klv1qPuK0PUsQBgwBsyyyc6uptOCGJNJ90ZQrYxc5I?=
 =?UTF-8?Q?6y7YOPtF6udQIdrm1Fqt_zPwADo5Zb6buQP+cKHHFX/kYOPL/WRpzfh8br7U5qx?=
 =?UTF-8?Q?sj+faNbF0cyPxAyaE/aMRc5yEf_G4VoqMsVTo0j/qykmw3qnOIQYfXxnUkDTi6q?=
 =?UTF-8?Q?YIDQkhh1yrAuGgct9Co5t5Ia45vqu6QF_4A=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Q+CvvaW8eTd/F26GHjAiUYiRLmA6oc0CSqS+l7ebABkbtA0JPaGDgSrsO6G+7U7cK1A9O9h8h3Tu/SjLBRziRFo6TVty5qSuXTnhaqnejvQhKqJfTHJOuetcpWesvou9pTDWlcEYgVxNQXKfOcKbLHweaSKSEU0gcQb+afQHqKGCcKqIqSKfBOYBElVqFz1lyqPg4oTJh8leZ/SRSXchEZgBFOOzOCenDVMOvw9jhqYw3xahlB8N8T7ssU0B1bZouHMoa/YqT3J0AcjnYbVlcgQrIM0tmrSc+3XQ/GI9QRXH0rS0OSghLs+wV/7eA31IYfDYAU+KakRUqS2nP1xOJg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NrlvPCyMcyqWI/tJUTwItb4RxylbxvVIA4qhSg7km+Q=;
 b=RXfln5wCwBpRnBmQEJ5cVyCvquVS/TMATf6NpC5YBAC7YhFS6Nmoc++AU9NiHBD1A3JUQxbcfvXU3x4QoAlg5FbTS701Trm0429Kxnx0QMce2bJDcXhKV6gl4KNjUKrdZW60rfu2xXPP12mOXvZ3R5xZiWrZWRPDN3FotaR5MEVgARFJWgn+xcd9C3QtuVgnI1005/0hLn/tg+X8XgyaOkMWpir/Cgg+GmL+ATmzyKGXeeX7tj3NzJ2w3V/SF01GsPtIibdOeFyNCKeIctTI8VcFpEgIubFAHSz79JedqAE5ixdeQx6sTrCORqMSSPZp5GDWPqA0iz34q2Gm/LFB6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NrlvPCyMcyqWI/tJUTwItb4RxylbxvVIA4qhSg7km+Q=;
 b=xR8kwZJbV+OzrYtU58cUZKd+kpGDvJ3MsUwtBHrb2elNBp/d+z6QlKM1ngfKYSbF53hw/ZH9iyhIwRto4B+PMw7t5nwyqMkoqI7s9JoPNSYsSogNA9FV2Vudp5U00/Zz1Bwg9xLlbe+MULxxeoTx81yLxBlNPBX9+IWSaR/uv5E=
Message-ID: <3aed8f85-890c-4c4f-85eb-fdfaeb65b6fd@oracle.com>
Date: Fri, 31 May 2024 12:12:12 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 13/14] block: remove unused queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org,
        Bart Van Assche <bvanassche@acm.org>
References: <20240531074837.1648501-1-hch@lst.de>
 <20240531074837.1648501-14-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240531074837.1648501-14-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P265CA0193.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:318::7) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|PH0PR10MB7007:EE_
X-MS-Office365-Filtering-Correlation-Id: 6ecb6b76-b14a-4033-6418-08dc81628897
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005|7416005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?bEQ5b1FKQ0F3TDBMMHRrN0FNN25zZDJiUGY0a3Brc3UrZi9BWVRuLy9xTkNJ?=
 =?utf-8?B?eENkK2pXZkh4Y28vY3RDeGpUUndoRWNqSjMwdE02TWoyNXNicVZORHR5alcr?=
 =?utf-8?B?YkcrekRSV3lNSllmMjg1dzUrZlB1c3pJOUcwRldiMS9sSUgxSjVlUWtrTmpu?=
 =?utf-8?B?dUZnOHlEbkUyaC9FbUtRbU9lbUxOeEo5Ky9KWFhPR2JaVmVNemgvMU80Z0dS?=
 =?utf-8?B?bk1oemhUOVNiajZPZ3Y5eVoyY3JTUzM0QkpJREc4VC9OTkJnSkVOM0ZoeGE1?=
 =?utf-8?B?R09Rc211MmJtUnB1NzkxUGFDa3BiVmovTUxBaHZkRktGeDFVS3d2U2svcGFo?=
 =?utf-8?B?VFg5WW4vQmh6dEk0Z0ZEN1JrQ0RKMFlPYXdMejBmTlV2ckx6Z25zOUlRQjV0?=
 =?utf-8?B?YjhOQ3FmR2Jzb05Ja2o5Vm5qd0V4RkdSWlpxVkxhaFdoSnVBNlZYeTlYenZU?=
 =?utf-8?B?ZW5SaDJrMS9QWUFFQnhwb0ZLVWVKMHV2aFNjVk5VdmlwYnNKZklvc1l1cXcr?=
 =?utf-8?B?bDBNN21KZndzaDdBVkFkb1NRNEJIaS9YQzZMUkg1U3NDZUpzL3d3YTF4ZVJM?=
 =?utf-8?B?Vy9jSXZrbzhKeHJpb0RZSlJib0xiU1MxRkNCTU01TDN1czFyOWQ1ZXExbEh0?=
 =?utf-8?B?S2dsWlFVc1N6a1hZR1hRa2hxWmFpU1I5OTVTNlltZTRJZkU3YmtlQ3ZBY3NB?=
 =?utf-8?B?Y0F6SUpOSjJ6cDFZOFdyQ3I4b3QxcFYyTmRxSTVna0FYdnBLVnVZK1cvZVNn?=
 =?utf-8?B?S1BBLy83UjJJZUplUlZGekE3c0hwWVhuYUl2UXhLSnVOelE3eWFwTnM5MHcy?=
 =?utf-8?B?b1l3eDl1SnZ4T0ViWjNmWUxKeFh5NTJqL2xsTlU0UTFPQ2JvVUJrdVFRejV5?=
 =?utf-8?B?Y1RIemx5TXVDdGFQNkZtRktmaEp6dHl3N1EzVVlZRlI4bGlkbHdKT0ZMeGti?=
 =?utf-8?B?UnpXcVllcm9NWFZzUTZ6OTZWZTBaUmpNWnhUVW5HSEx6dEdrQ2YzTkQzVEhH?=
 =?utf-8?B?K0FmNlJXc0R2NnBrNHEwY25JV0dOZmVCZWhVWXp0UjhEZU00eHM1RlNEek9F?=
 =?utf-8?B?N2RBRHRaZXF2OUV4eG9JdFNzLzJTc3orb2poMmhneEwzeVMzdlN4emhDTHZW?=
 =?utf-8?B?a2U3bWU4UlowUVkzMkN1NHp3d01ndlkzWmhnTHJ4T3p0Mmp4bnB3NEdpQlRY?=
 =?utf-8?B?NnF5bmtBeU54dFJxQkR0LzRlWDhibjROTnJJNG9JMEptbkhHVnZLbmJldFFT?=
 =?utf-8?B?NjVpUEp1WHQ3dHZnTTE3TjBtV0NBZm9pVG1xcEwxTU5QU2lpMzB1cVMwczNL?=
 =?utf-8?B?R2lpOENJTEUxYXc3VVQvK3E0U2MvSnBnMEIxTnMrdEdRZStkakwwUS9IaVRl?=
 =?utf-8?B?KysrN3dYUXVTYitrcUV5by9rKzhvdjVhOTU3aTZYaWJpYWtOQ2s0WlJHVnl0?=
 =?utf-8?B?Z3dTUnNlaVBPeGpsdG9CaXYzYmIxOFdETXZ2UGRHYkZhQnBRM1pqYjR4LzE5?=
 =?utf-8?B?cHFlcnI3U1VRdUpUMDk3SWVDUU9ScXd2OXdId3NnU2dsTER5K3pOTGdtSmg3?=
 =?utf-8?B?NG9ib2EySWozL01NK2NUbGdBck9kU3VjNDU3by8vZGErUDlJMHRrdnFXbFlp?=
 =?utf-8?B?Z3VXalpVSGc3T3pOV1AvR2pEUXVSWnpLS2FVZHFMN1JEVGlsQUNJS2ZydHBz?=
 =?utf-8?B?emJ5KzFiZXZoM04zNGZJUlVnWUcwYkhodDlPNUREZUcySXh3ZC9LdGR3PT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(7416005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?cFZ5R0x3eGRGZGRPZjNUc1hKd1M2R3ZjWmFTVlc2a2VDQnBvWUJqTkFmRjAw?=
 =?utf-8?B?Y0ZSalFCY1BvVW5QVEFlT0YwVzdsYmhXT0ExbVhuLzVmR3gwOVNPZytkWjY4?=
 =?utf-8?B?cXVuc2NURE0yaTh2UkRjRDJLSDNma0pld1hQdVphTFM2S0U4OHoyTFBhQ3lT?=
 =?utf-8?B?VzU0NDZNY0ZyVFVDOGFuRlkweG1IWTU2Z1ZUcUJPOUJIQ3dTOUhnRHhodGxZ?=
 =?utf-8?B?eEVmeGZxUTJENWF4OW1PV1hLY1lLYVZjS1RwMWR0SEw5Q3lLZUlvZFByejM5?=
 =?utf-8?B?a0M3M0ZSNjU4ZVZZNzlSbUl5T0xNeHpGd05RWG9UOWhqb3pCV28xV0NTNGZh?=
 =?utf-8?B?T1lqMmtzOHVtZlFQRVhEcDlQbkRjMmFWVXpTMjkzL29sUTQ1WEFBOFE5WU9p?=
 =?utf-8?B?cTdqeFpvVXdpQmRNbkVOSFNBTWJQQ0l6Z2hQeFcza0FhTTQ4SUM4SS9PREhq?=
 =?utf-8?B?ampycnZ2Qnpsd2lTeHF3azRWbHQxUy9ZYlI0eGNtT09wd0VEMmo3Tk1wRFh3?=
 =?utf-8?B?dnBtd01idkJrUkVDYTBid091UFFwTWxPZytWRlVwelIvc0dkVFRFT0dpSWFl?=
 =?utf-8?B?WDZCN3p5M2lqVmcxOHBVYlBORDZ6cmRTK0tIbVBONCs2YmpsN1IvL2daQngv?=
 =?utf-8?B?MjhKSzNIbUhFNTc2dlJjajlKcloyWWFKV1QzVllIWnU1REZsK0FtOVRYU2l2?=
 =?utf-8?B?UnRFem9Cb3gxMmtXV1FjTjV0RXR1NU1XVlZ1Wi9qWW5pNWVnNzVSaEp4WjZ1?=
 =?utf-8?B?K2wzZFVvSjZrM0NaRXRTTnVpendXdU1lM2cvaWJ0aGRyZXhUdEFiVDlqUGNr?=
 =?utf-8?B?QlJuRm1qbWF3RHk5NENXRHpXZmhOWStrS0JIODFPOE9RN3JxaXF1V3dsKzJv?=
 =?utf-8?B?RHBvc2NxZkNtSHZWR0xMaGZnWE9vKzhqT3VtRDFnYWQ3OUwra0xZYVRZMDl3?=
 =?utf-8?B?OEZkeEh5ejdvQmYwWHFBOXpKeXplSEsyWUdVOVYrTnUyaTd0T25Rb1EzZmVt?=
 =?utf-8?B?QS8wWjQ2b2ZLT3d6Zy84YUxFaXFmTVFxRU9LckNpM3NCU29FVWlaWUxhakU3?=
 =?utf-8?B?eEExTk9rTldHMXBadktIeDRnZkJsaklhZDhDbStWalFZajlVdmNoRGZablhk?=
 =?utf-8?B?NUxGV1RnbFB0YkYraHdVa2ZOVzErLzg0ZFB6cTgyQVFmM2ErMHpMeGdremNh?=
 =?utf-8?B?N0JvU1lpc3pRWHlHc2dxQzk3SWN0eWJLb3BKZmwwSDRxQVI3ZGtTOXk3ZUgv?=
 =?utf-8?B?Z3FMYWF0UGxweGdxTDZnS0pkOUNmV0svMjFPdmFHYWsvYmNYT0piUS90Z0Fl?=
 =?utf-8?B?b0xsNEcrUU5sNnRRWWkvZWQ3TFBkMW1IN0YvQWErVEFoYWsxNlB1WnR4TFJU?=
 =?utf-8?B?SU5qM1RZZXpIL1VsQUZDMDkvYnViemhFU0x1SUtYVSswNmM4NkZlMDFkMmxW?=
 =?utf-8?B?WjlvQUx5bVpFWkVheVczTlNMT2xTWUd5QzFFM0RQbkJiS25CQ1ZoRVRVV0lj?=
 =?utf-8?B?dWlCK3pIeU5GN2xYTkE4dXRNcXdKamg1TXlJaDVSUCtGYzQ4M01jUGl3MUxl?=
 =?utf-8?B?OXNxSUhsamE2Z3hoMCsrK2hHNmVtQitxVEVIWU9zU3czMGsyZ2ExeDgzN25Q?=
 =?utf-8?B?MVdzTklHZE90amdKN3VhK3RXNENjbUJVcUpuMm1wUVYvVXMyRUgxWGp2RCs4?=
 =?utf-8?B?OVlmOVFZb0FxRGp3VWhPR2RsVnR6VGlvQ2RISDZEeC84TkxWRENZc2REUmNG?=
 =?utf-8?B?d2Zjd2dpYlRzY2NBMlZ0UEwvWm9lSFVIMHMyNm1wZy9GY0RjSW11THBzRGdW?=
 =?utf-8?B?QlJGSGEraEhaTXlramFQN0dBalZPSkdjaXU1M3pYY09WcWxYdHJsOVVzam8v?=
 =?utf-8?B?TlR5VlFSam9XNGFKUVE5QXZjbWFIYkwwYzBiZStDeHVUa0dKL1lWekRqRnYx?=
 =?utf-8?B?Z3dHYlYrcTdJQnNsSWlkd3NvOExhQjlGdFdzUjdBbXdOZ3FDSEJmY241Z1ZQ?=
 =?utf-8?B?SjB6L1BPRHNCWHlqUERQd2IxeDJySlU1OExKdWNjb1RIUWZXWllRRHkzYjdr?=
 =?utf-8?B?THkvN05zZWRBV3RGTVBJZ3J0MUx0N2dUYUlmdktqbTNqRjVGOWlQd3Fsem9P?=
 =?utf-8?B?ZTgzTmdHWlNFaVErdjJuU2ZwejhPTDdpR3hiVGxSQVV2ei9vc1VxWUsxVTRk?=
 =?utf-8?B?R3c9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	P7Dw/WwJQMlT6ivAC+aLcn7Q+Yuk+suSTGd/hhslbA2c8vBGvav2bE+zbPphFfvgQuTuLOdxu6ZaOc/t0smRGQnEDgbR8zhbr2r/6eY8ke0g+a+ow45JCSV2y47keXN935fJxCmMujGp0arOHdADicW9ExEL+/xSvoqAMTqTYXPrth+su1k+7r6F9vJKvq+nTBdnZxoOu+XMZGXKQdenmHUxFO6SXHV1cfBV/bHT8fS9rkQfyAFhGK9HeaIAK7mJbjnZPL4XYYtZ+NRSFQ/YQnjXQdcah1V6J0k8n5kMqRUFRZL6mDs1dxcw3qOn5i6kOJghFrMgIKgMWVYmVWHk3RBafdbHJ9wGcujRBjcMmOHj27DZwV6L172pKU7HVyFEgHATf7iMnN5ptPnEt7M8AuWPoJ0Zo6ZUoLMgs2RDehQr5kFy4nkJJQFFlQkCeXYvrMajoJChsqcvvXZxG619s06TJdZ9gpsgRdi8zMx9s+W7WO/1lhfpj/ch4qvKUlustM0vlR+OMT/h3+PZ4FYh+YjNeE0uEXFERQkBkGOJGoaKH+ATCtNk3gSFvfF8NziXurZAY/8VABDQdvEIUiLSqluF1tj+1I0CxFCjOvgqFxs=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ecb6b76-b14a-4033-6418-08dc81628897
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 11:12:18.0389
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FlS8g9FNHD10ZFzs2aPWqEsBc5RPGKLVmv73mO+5QeEGqvAUB4gEc9FfIs3zDDz+t1lyykZ1F0Al6MN2AHIO9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB7007
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_07,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405310082
X-Proofpoint-GUID: HWzHE3q1ri3FSbJIqskY_Tg5EiZe8LwJ
X-Proofpoint-ORIG-GUID: HWzHE3q1ri3FSbJIqskY_Tg5EiZe8LwJ

On 31/05/2024 08:48, Christoph Hellwig wrote:
> Remove all APIs that are unused now that sd and sr have been converted
> to the atomic queue limits API.
> 
> Signed-off-by: Christoph Hellwig<hch@lst.de>
> Reviewed-by: Bart Van Assche<bvanassche@acm.org>

Reviewed-by: John Garry <john.g.garry@oracle.com>


From xen-devel-bounces@lists.xenproject.org Fri May 31 11:13:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:13:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733640.1139947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0CI-0004v2-WA; Fri, 31 May 2024 11:13:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733640.1139947; Fri, 31 May 2024 11:13:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0CI-0004ut-TU; Fri, 31 May 2024 11:13:22 +0000
Received: by outflank-mailman (input) for mailman id 733640;
 Fri, 31 May 2024 11:13:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sD0CH-0004QX-1I
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:13:21 +0000
Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com
 [205.220.165.32]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c83df09c-1f3e-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 13:13:19 +0200 (CEST)
Received: from pps.filterd (m0333521.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V9W0Fi005630; Fri, 31 May 2024 11:12:52 GMT
Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta01.appoci.oracle.com [138.1.114.2])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8g4ayu7-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:12:52 +0000
Received: from pps.filterd
 (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44V9DwAm026633; Fri, 31 May 2024 11:12:51 GMT
Received: from nam10-mw2-obe.outbound.protection.outlook.com
 (mail-mw2nam10lp2040.outbound.protection.outlook.com [104.47.55.40])
 by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc509rb36-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:12:51 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by PH0PR10MB7007.namprd10.prod.outlook.com (2603:10b6:510:282::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 11:12:47 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 11:12:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c83df09c-1f3e-11ef-b4bb-af5377834399
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3DhEGrEfvhl50DirHQTj6h8OIju65uZeJ9y0w/EBJbnic=3D;_b?=
 =?UTF-8?Q?=3DDVj+nGnEzLF30QcqobJE5Ev7eYz0a39GXIlU75c46kTt+g0ahtUaeZ3JLhpn?=
 =?UTF-8?Q?gvHIyDoS_8xtM7fJJFjZ208m1LrnJ8O7GNZiYZIbzxZpO6CohXRDwl3nVZOVFTW?=
 =?UTF-8?Q?iYBcoM1jH6haOJ_ixgY91L7RyfIN0WTIrz4VQ2xSyUBjtJlHcUc8QfyR81rA4cn?=
 =?UTF-8?Q?fUzB+66gLjO6VAx2/Owq_Zh/Og7UhzDrgSsF6sC6HnquIh3h/ACd6n0BgyHJtnM?=
 =?UTF-8?Q?c09KP0ctjt+8cAAKIGbZ/5fdeF_fn/wutE7XMDrdi7T/MFuSKKbTqLKSPX8fb7c?=
 =?UTF-8?Q?0FfCG0IuS3LygnrOPnqmzHmR447aP7jo_Bw=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Iy98gaGldBLOmAn8NHkKdJgq17abyV5ejgCXLioS2fFA9Q/Suh5NkBl048hKOge9whUXpCB2yq6bDXjA3/FZy4pb811/swsIfFkI8cH9YfVrBIJ+6OXzhW8b3mZrDZt8qOmFNXH5GtK8yw8SlGd3KQnMejUsg9B3GdenpqI99uPHgVz/lqfnaAFvnaq3FJ7lzoxBnnx28sCTwH2AuKCUKVlYs07YwPn9BsaJVdChBPWgzASaFZpmQJ61Gj7xdUMU3rPYjf1M31YKOPa4VWB2x+GS1xORwfPP3Cm+2W5AuGmn1XjPNIjNtVmq0t/2WppfTO8v4cEKnaK42tF9UdPZCw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hEGrEfvhl50DirHQTj6h8OIju65uZeJ9y0w/EBJbnic=;
 b=IHf5dYDTEIbd10gb7sdUQqeKxy+SYj+RHb2FegGLx9a9dY785ka8YnL4N8lp6lC2YU6A8GsijASP+AV1iCk2g7skHCsUwDg4zxGuWQgFhvZiDzT9Dwftkf59zaX9wU86DAB7qoxvNFpKcC5BzqM7Aj+SLqNxPGp2D6c83+DQGFDc1WA/b+gDRR7DEP7BKErjQPc3uZufS4bvrKtKY64eJNd+KTX1VUPEmOtGlSeYbwHfgOUN5ZfbJNpzkXAK/XjWzMgirGHKamMDZIUTAxQ4RZSBIf8WcqA+Cvx2uKoGYT/wFPvg8DdwbclpC0NXtDtLJkglYuZfBnfAfA72k+UqnQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hEGrEfvhl50DirHQTj6h8OIju65uZeJ9y0w/EBJbnic=;
 b=N4/tMwWqyrIucBTeITGDvFl4GGL10JrZYH76v39bCmHIU679tPpOszlCft3ZJAbFbIOv5kRc5jznTr5ksAo7c6Ka6H1DRbt/7keEaBAfEqDqzYB3h2vYvCy1a+VpcIn02xqscdCJ2dqcR9qJ70qL8XerFoIG0FcvRp/MRVJV4NQ=
Message-ID: <7a7cbb99-23b9-4ea0-855f-8f4db8113806@oracle.com>
Date: Fri, 31 May 2024 12:12:42 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 14/14] block: add special APIs for run-time disabling of
 discard and friends
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org,
        Bart Van Assche <bvanassche@acm.org>,
        Damien Le Moal <dlemoal@kernel.org>
References: <20240531074837.1648501-1-hch@lst.de>
 <20240531074837.1648501-15-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240531074837.1648501-15-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P265CA0195.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:318::12) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|PH0PR10MB7007:EE_
X-MS-Office365-Filtering-Correlation-Id: 379abc61-9b81-48d3-0181-08dc81629a18
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|376005|7416005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?QS9ZUU04Z2RIRXpCZmdoK3N6M0VoRHdWTVdnbkw0ZUx2TlpsNjNrZlFYQnJi?=
 =?utf-8?B?RXFQdVBIcDFyb050Q0YyRHlNSXJsT2dZK2RieGYxVmk1dFRjUlBkeHphYmR6?=
 =?utf-8?B?elpvRUNYampwQnZoWTBhWlcwMHA2UUdmcDQ5NDh5VTNrc2ZPcDdPcVhuSmUv?=
 =?utf-8?B?MUJOSllNTWh1SytQYUtucGpjMlZnSzBvUVpZMFdBV3laZ016c29TbUxoc0d6?=
 =?utf-8?B?YVlhUHg2M1V6V1k2Y1ZVVDk2UjNqckdnSWo0elRZeFBKTVl5aTdrUCs3Q2o0?=
 =?utf-8?B?amJXa0FGTE92OGZqRUdGMVlNM20wZkdlTlgvbHJDaWZjQzhjU284T3oyU2JE?=
 =?utf-8?B?dWxiT1J4QmJZcmo0ZkNTMWRQbUM3U0tSOVkweUV4V21wenl1TEJ0Qm9JSjl1?=
 =?utf-8?B?WGVaYzdnTloyZ3lYTWR1WC9BWTJHWGN0TjV0NEgwTDBPZTdpYWhyS09Tb1VD?=
 =?utf-8?B?OFpUNjh2dWIyQktTazlMSFpxcHBXQ2Q5blVQLzBMREcrdUdxUllPRzFDL3dJ?=
 =?utf-8?B?K21HV0ZoVVo3eDhFM0sza0k1U0tGU1Qya0dDRjVubDhFUzBxR1h4N0sweUFY?=
 =?utf-8?B?Z0RLbmRvdHFhcG43bGI5M3FyRFpVQVVpSSs2MmZMQmwyT25XNjdNM3F2SDY0?=
 =?utf-8?B?TE1GSmszTkw0MkRUSHFpWTYvTVZvMU9uVjBQai96b2tFMHdvSW5qV2U5dkRv?=
 =?utf-8?B?T2dUOWp5dDhRcHRLeFRFM0Y3ZkRQYSt1b2pPY3EvZlY1eG9qWHo0SXZ0bEI4?=
 =?utf-8?B?NzFXeFdUQXFxK0toQmJLcnQ2RmQxQktaYVgwS0lmdWQ1dmhyUGFjbEpwZGha?=
 =?utf-8?B?SmFSaWllREdXdGl6czZJTVh3MEhMU01kNVh5ZjhCbGwyZzVpRVFVeXB4OXM0?=
 =?utf-8?B?cU9kamxob0VaMHUwR1hyUklaWW0zbm1BYjV5V2RIR2xGZnUxamhCUkM0WHd6?=
 =?utf-8?B?N28yS2FPY1d2VDVaYnR0d2VvaWFyaXo3blNkTzNwV0dHZWNxTlVRblh3NTY5?=
 =?utf-8?B?bCtGWWJMQmNkcnlpS3IwVk9JUWZ4Z2hnZXNBaW5Mbm1YQndSc2lPUVZja285?=
 =?utf-8?B?RTAxamRnNzNRd004TFJremZXdjFmMmNYUXZ6QW13Wnhsc1N5MmZWS242cENu?=
 =?utf-8?B?eVQycGRqZkxpZGpIWlpCRG5HZmRvNXlTTFJFYWlvcEJnWjdiUnQ5L2hpQUwx?=
 =?utf-8?B?dkp2RnJNU2ZFOFhJSnBNT1h0RW9jWFJaWVRxTVVsTHFzYytWWkdGeDl4cCtT?=
 =?utf-8?B?Mkh2QklvVW82Zmh3ZUhpSHBJQW9uWHpaTGdkbnowNzEvSFFQbVR1MVpDZXhS?=
 =?utf-8?B?RFVSV0h6d29sUjdyc0dZTWpWVlJ2YWJpTldCcTdWWVQ1NWZQZ1FFTTJXMTNG?=
 =?utf-8?B?ZEJlbTUxamNZZ0tQaHlqL25Bcko0RTlQZC9yQ05JVVpwOGRoenBlZ0xGSVBs?=
 =?utf-8?B?NW96Ui9TZTBNSWNlVmc5Vit4eW40aFdVOVQxam9Gd2I3d2dLZ0FVM2dyT2FN?=
 =?utf-8?B?SUYwWDNkUjBUdk1JVFlZSW0rajhGZFBydVBpYWpOdjdnNlhnU2lZakNqZHpi?=
 =?utf-8?B?ajhTMGFPMTBySlJmMC9JcXNsSGhLSGpwZUZsbG1aMWRXdXFsTDFHRVpHdFls?=
 =?utf-8?B?YmhDUnM4bjI3eENMUHBCZFZWK2lNL2VZcS93R1JDdkF2Y2JkT3RRU2hKaGVV?=
 =?utf-8?B?cnRCUFMyQWVSdThkV1dYcFoyVHVONmJON2lEblVUVVZScjI3b3A4RU5BPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(7416005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?cnNOaUgxRmpIUTdhNnpUbXQ2Q1cwZ29mM1JRZ3QzTUVvM0l6QWxDYUlUdUw4?=
 =?utf-8?B?WjZxYlA0c0U2L3hOd0hEeDdaR3lhMWpPSXBZS2JnTmltZHRLeFNZMFYzbFhY?=
 =?utf-8?B?NHEwMUJSaEI2QVhjR0xnUTFHbHdGMU9rd3VmemtmTURUdkdSdTNwOGo4c0pk?=
 =?utf-8?B?Q2RsSVN3MXdJTHpiMkZ4TUE5UU54azBnN0FBcllHWDVrNzA1Y1V4czhJdW84?=
 =?utf-8?B?SXdjaGl3Y2dGWGNZbDVVQ2gvbXdTdjBva0QxSnIrMHNhbVdnVG4waXNaZUgz?=
 =?utf-8?B?SzFhNmJVMldFNllKUjJJb0lkK0JBNGhhNkx1T3NTbDM5eUtvS0VTVXYrQ1Nv?=
 =?utf-8?B?SW9VbnlveHIydHZGRkFtNy9GM3ZmMWNkWUJ3VGpQK2RKYi82UTZOTWgyR2Zw?=
 =?utf-8?B?UHczdjJKNjh3UlludnpxbmpwMTh3RllhVnc3MHY0YkQya1ZVckFIU1VGV2U5?=
 =?utf-8?B?WTZZbDEwN2cxbXo0Q3E5VVVWOTU3aFRhT0doRTRMNVJYVmRxcGZyOUk0WERZ?=
 =?utf-8?B?dnBheDNhUjFvQ1p6dDlpZVlSSHhLbzhvVnYza1NpZitmN0sweDZLcXF5bTJR?=
 =?utf-8?B?Wjd3L3h0cExjbDZ0V0VkYWlmaEJYS0h0TDh1RG5RRzhwS3BPd1RkL1JMNGJO?=
 =?utf-8?B?cXNqM2F2cjJmMWJlZEJqRklkSzBiVUFMdmkvbHA4QmdsUkwzQlFNdE9nMzgv?=
 =?utf-8?B?ZG1Sc0o3R3o0ZWlrMmhZeGlQZGlsUVEwUlVMUCtFMGxOT3NlVDFzN3QrTFFz?=
 =?utf-8?B?U25mUWhVcUlKNnZlMUZnUDMyS0ViSDVsSWZlMFMwSTNQM0JRaWdOazJZSzcx?=
 =?utf-8?B?VytZRjZQYUVCOUpyeFprRzFzVjNKTndab2hGM2MvbGRDaHFNUUVzWnJVS3JD?=
 =?utf-8?B?UXNLZll3QTk2TjIvOW1BSnJ0ZHVUU0kvdDZ1cW81Vk1pcWJlSU5zNVB5YVQ5?=
 =?utf-8?B?VXRzaysxUzF2ZnZMQVFsVHlITVpmNXJaWmFUZjVFazIvMWJJd3k1eWhHcDUz?=
 =?utf-8?B?N3VReUN0c2tIck9WaFl3TXNmSEFsTkRiWjNCQWM3UEw1cFpaYldtNHlHSVhH?=
 =?utf-8?B?aGxlUFJqMzlDc1V4M3diWXdFeUdzcjBreEc0SnlEUkloeWxpdWhNV3dkdlR2?=
 =?utf-8?B?MGNvNG5OcWk5S0huZjFBbmMwVm8xenFuOG11czVnZE8xNWIzaVlzbzMzamh1?=
 =?utf-8?B?eHVrSE9IT0hzejBERmNkeUV2K3lVYjFkVXp5Rzk2U0RhYzN5SVppZy9jcUVZ?=
 =?utf-8?B?Q21qKzJlaFA5Z0kvZVIwb3creVBYNktNdDNWUlZCVjh4c1dPVzV0dElBdDhM?=
 =?utf-8?B?T1ptNVV3RmY2eU52Sm8xS05GYitBK0tHQnp4bkgxYXAvVmtrdjc1cmdZL1NR?=
 =?utf-8?B?T0N2Z0Z0c1gweDlXZ2xlbUJJM2FuVHMrTldNTnkyVXlUR1JNTkVrTjhkRjVX?=
 =?utf-8?B?cUhMTXU3d0FDcFpvNm9YWHZRZC9ab2p2Z1p2alZIekVzVDl1M01PbkpUOVNv?=
 =?utf-8?B?eitBdWI5UzJFZGRHbXpQWnhoTlNZUWo5Q3gvd1IzSGdWMit0bk1RN1lWLzM0?=
 =?utf-8?B?V2ZWY1QvWWlOYk5JT0tLeW51YVJsVmlRZnlBTElieHAxcUlvWUtmTDhSc3VM?=
 =?utf-8?B?UFJmZzJ0aXp1czJRbzY5dHVMN1l2MDV4QUc3YVZEdFNlbXJaZEVpaXFKY2Zi?=
 =?utf-8?B?bGxEd0FrZjlqMWZxNUJGWXA3Qlc3aDYxSDJ4eXZNeFdLQ29sejA2K0lsTyt3?=
 =?utf-8?B?c0FLMUxEQTdocjdCb0I1UTY2OGR4Ky9kWEtYaXR3UDQza1llZW8reDBydGJ4?=
 =?utf-8?B?TmdOVUFGeEpJeXVaV3V5Z2t3aWVSREFLdURvMnpyKzdGVXQ0cFNmNy9sN2Jk?=
 =?utf-8?B?VEVGWWpwT29UUlk0Wm9pQjN6WDVUMnluOXV6bmVOQmR3MDZtTE1SVXBvR1l1?=
 =?utf-8?B?bUJJOXRPcGFGejl3NHhlaVIrazI0eXJGU001UGYzVlBkS0lBQ0RkOGhYV0JH?=
 =?utf-8?B?dFAzQTg3MTJNLzJYVmttV3BtL1FTbzBRUlIwV21RdGJ6UEV3ZHJxQ3g3YjZC?=
 =?utf-8?B?UVZ3Y0lYdjN0UjREMGlIL04rU2ZXQVB3NW03UlMrVEpHa1ZzL3AzMjRCSXZB?=
 =?utf-8?B?elRIS1BPUDE0S0lwTFE3dys3a2VPK1VFc3hQNGpXeFAyYmpqRVF4WGZJV3pJ?=
 =?utf-8?B?MEE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	mM75Ky+Rp0XOED22ih7gFGm+CyVi5YobrgaH0XtpuwKfZEtPIgKoOD5uJLJkRfq2pw1djSWjtTixukLKmYeLlaabsNAs3X0v3I2GTWTtHneQ7zTYrTF/9icECo7LrNcpORovcUkyyvGsLMe5v55258NFqjR7Pv+sewckvkBs8aI0oGYx/V/SkUfDvUMPWYP4R14WJn/pGPJDjR7aX/EB8Y8IFVZq5tdnxs5B5RhcUR9Yp6IKZvirh0i1UYKckuzPwx2L/utBNpwjZ5+5j4Fv9UPWeCwgANYAjfIFH/sgRMCaoeNms/yv1n8looGGWYfULnBVaibNr0Zt3VDK5aj3ud6i88cEg4tWRL99HjO3IcLaK0BBGhFd9q82xfQovW9No5YSZi7WJjiJrMp39saPvo35T92vgd9pdonG0NluBtuxViwsoCqm0q8/6lDdtHVzHJUhiO+lqQBC7fb+PNp/iYEviLYA+bE+S6Iyi3Z5KF8txfnWr7sUSSeoBpL/94y17sIwN8NaTe/EbCjv4QLno+UID7cTbmnLfg42hiJfW2jmEDniotXnktEvPUFszMs6FF13to8dT1bGDScTSO5UBqkKo8k5x9FK+eflDODVUeI=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 379abc61-9b81-48d3-0181-08dc81629a18
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 11:12:47.3759
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pqfGGOoC0qzZfTyQVuj2z0cVmEfBswvtC0H3ozBDp1gIT30ScZV5SmQ3JrvEQL1imqxLF7V51UbjiJj+LpInXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB7007
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_07,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0
 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405310082
X-Proofpoint-GUID: arDQOMF-4_0TP-qZw765mF9c812Q52b6
X-Proofpoint-ORIG-GUID: arDQOMF-4_0TP-qZw765mF9c812Q52b6

On 31/05/2024 08:48, Christoph Hellwig wrote:
> A few drivers optimistically try to support discard, write zeroes and
> secure erase and disable the features from the I/O completion handler
> if the hardware can't support them.  This disable can't be done using
> the atomic queue limits API because the I/O completion handlers can't
> take sleeping locks or freeze the queue.  Keep the existing clearing
> of the relevant field to zero, but replace the old blk_queue_max_*
> APIs with new disable APIs that force the value to 0.
> 
> Signed-off-by: Christoph Hellwig<hch@lst.de>
> Reviewed-by: Bart Van Assche<bvanassche@acm.org>
> Reviewed-by: Damien Le Moal<dlemoal@kernel.org>


Reviewed-by: John Garry <john.g.garry@oracle.com>


From xen-devel-bounces@lists.xenproject.org Fri May 31 11:30:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:30:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733658.1139957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0Sl-00008W-FH; Fri, 31 May 2024 11:30:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733658.1139957; Fri, 31 May 2024 11:30:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0Sl-00008K-Bz; Fri, 31 May 2024 11:30:23 +0000
Received: by outflank-mailman (input) for mailman id 733658;
 Fri, 31 May 2024 11:30:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sD0Sk-00008E-Pk
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:30:22 +0000
Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com
 [205.220.165.32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 293cae58-1f41-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 13:30:21 +0200 (CEST)
Received: from pps.filterd (m0246629.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V9VaLL025201; Fri, 31 May 2024 11:29:59 GMT
Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta02.appoci.oracle.com [147.154.114.232])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8p7tvk2-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:29:56 +0000
Received: from pps.filterd
 (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44V9GmV8016216; Fri, 31 May 2024 11:29:55 GMT
Received: from nam10-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam10lp2101.outbound.protection.outlook.com [104.47.58.101])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc50trbga-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:29:55 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by DM4PR10MB6839.namprd10.prod.outlook.com (2603:10b6:8:105::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 11:29:53 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 11:29:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 293cae58-1f41-11ef-90a1-e314d9c70b13
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3D53RMh66kGVP5vOMpWKeU8OvlS7xRk8mEwnifqa/5hZw=3D;_b?=
 =?UTF-8?Q?=3DgCalQMmKP5DejF41N+fxkoDCX0bAOAUkY0eP2iPvARo+PJGiMsvx1wC54nj/?=
 =?UTF-8?Q?deW5ketn_D8JIa90MyOVk5sFmENKfCKNwUOtQf/IFxAg9Ooht8p6TTMVRH10tK3?=
 =?UTF-8?Q?d99bSRHj0i97VI_gbU+GmSMLPmIBQBcQto/0l2mhmaezJZYEKdd5ZYMUa82bVM2?=
 =?UTF-8?Q?uhizSqPN8+gYI+s6WOEM_vM8GDPw6DTcNwx8ES4Xs0suGHmXuLlD7yxmBbBUSFz?=
 =?UTF-8?Q?/KyPjMHr5r5qOmZi8dFfPV/1jt_s2mbkrYV3WqJvO6nbsn9gmGojtwhpTDSPqT2?=
 =?UTF-8?Q?OftFhL3VI26WvczXJqyPFhbOggowu66t_Hw=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UJJscuQ8VOFaGQ8NupPAVAd2oLsIcSLGn9osd7i7949nIth1PbLVnh0tX+QJdV4bF0FZn1Kf56VzRXY7Qe3S05kdAJQhiXHHUlHGzYrhq1bKzB+bdSoMPPiILWm4gQu5an3XwZfT7WycL/75nq+kvlaU/Swmr86jNR62erMO5jhBtIqBc4kE+YavpPjVARtNmr4YANcnMihUvmlL5ZHuIC1Dq2233Up5FxEqGokvwmf+3PDgHY+/ZzCWRrBXbVxT0GiFQvPm5heZFiB7v6IDE/aAqk4IO5uyt0KuWofDERoiXgGqQmZFzpxOx5DEMLR/I0QR192BeF0jMG44mJbmOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=53RMh66kGVP5vOMpWKeU8OvlS7xRk8mEwnifqa/5hZw=;
 b=TnyzRR2W4I20u0hesJhi0ncQaGu+MdAs1g71Kw5sMhbPLSOIWSUWqXcRmWF/R3N8JL9lJ53BaLK8eLKRihbGKUlLz2KGvThj3IkrmALD2nhzUtuEQUVYgAe0d4LXDLvGk2q6YWbFV+oivyVOJLG8z5c4KiUKqvslxPuLP3V4k8eBlre0uy0f00uiHHR/af2+OgJm7IaKmhsqgSs2EYzeOe0wC4aVkqgM7r2Q+QfRJTqOUMUvPd+U9gGnJNgN39Muurzuj9o3F9qggLyPo0jFLEOTxJJVGTjwvkOSE++/N9K0jGmvp9m0WEcAOOVUK/QN6xAEM5RabrKf3A488efJPw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=53RMh66kGVP5vOMpWKeU8OvlS7xRk8mEwnifqa/5hZw=;
 b=HG1hPZ8IWoekwxbUwGOnBsh67EDBXxw4BDtSIGZ3WYA3CsSnAgqc7MXzAvIUTQSBd0YFBkZkmeV1SzfbS/tAGa82WTdssGZJKLmTsDhNCYzKH8dLUYIToezZauyM94cKqkDENDrLz+OHd49VsYx1EWSK7i+yDM0PLGBzb32S+U4=
Message-ID: <63bc4953-fa3a-491d-9f8c-180e4e85b46a@oracle.com>
Date: Fri, 31 May 2024 12:29:48 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: convert the SCSI ULDs to the atomic queue limits API v2
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
References: <20240531074837.1648501-1-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240531074837.1648501-1-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P265CA0126.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2c6::17) To DM6PR10MB4313.namprd10.prod.outlook.com
 (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|DM4PR10MB6839:EE_
X-MS-Office365-Filtering-Correlation-Id: d476e1a7-bd03-4c37-c4e4-08dc8164fd90
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?Y1l4K00vVjVpV3MvWTJtbFVmY2tZVnJiUXdweXdodE9RN2ppV3phNjJxNlll?=
 =?utf-8?B?bWg4Vnc5TnNkc29uN0xIMjdFVmloSjA5N2dJVlZLNy9RWncwK2dmNVFvdm5i?=
 =?utf-8?B?Z1RXYmV6ek9xalBQd1ZXWWo2czVad1RlTUZITWVBK1JwUFJ6SHJRcEZLQTRX?=
 =?utf-8?B?TVd6MXduZWM3L3JpQ3FEMG5GSi83R090MW1kdnl1aC9vbUdlTUdlQ0VRaGow?=
 =?utf-8?B?UG13cVIwYVA4R3pTUzRRWGIzMGUyTzNjSE1yaXF3SURuWS9rdWJ6Q0szSTVx?=
 =?utf-8?B?NTlpUlhWZGJJTHMyZjZ1VWtKeUl2M3dBbjZkZ2N4OFF1T2hYZGtaOVRKR3dC?=
 =?utf-8?B?Q2JKdUhrTzQ3VGVjb1dEenp3R3psdXJ1SXBTdlZncXpDLzVDZUpCT2JWVUVT?=
 =?utf-8?B?QUV4M3NJVCtSSlJFTjFLdmFTVHdBWmFtNkZqZUFNUXNIbyswRUt2b2ltUkg2?=
 =?utf-8?B?UnNlLzhNQWo2bGtXRVVIWjR6eHBhZnV0ZXk2VVNjL09Gbnl6Y2pTWEZ0QTN3?=
 =?utf-8?B?ZFVvNjQrdWxWNEFBdVNpdHA0RmN1MURHZkl2Q0F3N2VUdEc4U0lhZGNFNXdH?=
 =?utf-8?B?UWpHQVRNWThpZ1RaZElpeVVMR1JwL091NHVpbG92MmZLelR3eUdMSkJNVGlU?=
 =?utf-8?B?R2JWY1htd1RpZVZubHd3OHNEL1F4NVdhQ09PSG50QXRtTnByWDVQbmlXOHhY?=
 =?utf-8?B?K2VIcDlkQTk3V2E5ZjdVcEdETkUyTmRnaU5CNDJqMUVwc29mOVVyQS9UL3F4?=
 =?utf-8?B?UEh2MW8ySTVYZ2p2TjlPZUJicGpDcHlOam8zdHc1WmhWQWNiNSs5UXpDRkY5?=
 =?utf-8?B?TGVLbmVvZEcrbzJjOFZ5aDdwa0lkVkhUUEE5elMxcHZ3cFNld2hjeUhYWUFn?=
 =?utf-8?B?SlIvOXllUVNDcWorMmJEakpDU0FCQUN6cHNpNUpDVkFMb2hjMnVoU1dUUmlS?=
 =?utf-8?B?RUVuRC9SSWdjNzRYbCt4Y2dobEhxYUhKbnduR1dLMGNONDM0ZTB1UkY3bmVC?=
 =?utf-8?B?MHBsOEo5ZStLc0lyU3NPUzNzbWpyZG5YS2FQYjJndkZ3QWtHY2wxeHJoMWhW?=
 =?utf-8?B?a0x3VGVYcW41OHhFWm40cEU4SnZuQnl0M1FreFJUZWU0bWpGb2RPYVlHb1pM?=
 =?utf-8?B?ZjgxR0hrbXVJR29zWWhzVXdZaDNrRnhjc05LWU1jbzhyY1V0aFBMZjc0bXdy?=
 =?utf-8?B?a2ExL0dhMFFPUkxLTGZRUWhDSW1ob0ZZUE51bkp2dHhMTGFKVzM5UWxVSlVq?=
 =?utf-8?B?UXhqY0p3TGpxaEtHeTFpY2txY1lEN29IdkhZbmlTTXVxSXZEMU1CZHRremhr?=
 =?utf-8?B?TjF2VXBHSjVnekRLdVkrb1RLSVJ2eFZMRUxzOFVodnNvN205SE9aVUhoSDk3?=
 =?utf-8?B?aFhrZ0hscEtLUDZ2QmF3RGozczNadDYrTVFKNjlPTnJ3dktXZEIvbnVDRjNG?=
 =?utf-8?B?NnMraUVpT0RQRGlVUStmU1l6ZmpBNndBSUdhRVVRQmpMSTdmL2lDcnRjZkR3?=
 =?utf-8?B?N3M1b252c3BuNldmMHVhL0c1aEtlOHgwNVFDdlFJOWFmSzRQUG5tRXFaQjk0?=
 =?utf-8?B?cmtScW1aeE1VaEF6WklTajBPbmxmSTBqN1dlRnJoQTNsR3lHcHdIS0s2dG1M?=
 =?utf-8?B?L2Nhd05sbzRMZU5FVHVvQnV1ZDhJd05OV2lLMHRwU1BaMmZ6Y0h1cjA5emoy?=
 =?utf-8?B?SVcvdVVsaEdRNm5MM3BCUkRrV2F6ZDlHcHJzcllvQXhhQUQwbDVYbWN3PT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?TkVaV0g1VzNvVDg0YUVRQmpGUFRGQU5MVkMrQlBqUmVNSmlnRHZuNk9xcUZN?=
 =?utf-8?B?eS9Ga1ZvNjFPRUppVlpwSStadDVjZ1NLWmxzSUxhNlY3MWxZTmRuQ2tpb1VZ?=
 =?utf-8?B?V2hQVTR5NXcvMVVNbTk3Y2VaYXFtTHhQTUlkYWViT1MvWjhqT1hidk42eEQz?=
 =?utf-8?B?aHRzSTBEd29tRUY1akxhMEdQRGlocFhMVU8wWjBicXQ5NmEvZmFtWlpQMi9p?=
 =?utf-8?B?cGFEVk90YVRLcDcxSGVTbGtnWDAyckRadDlpbVZKSzQ1c1FJS3l3WUJtR3ZP?=
 =?utf-8?B?NXBiQ1I5cGdmd204U3RXL0dSTlhpWXNjUVJkRDVKVTdyZGNqNXFzN3VwVVJ0?=
 =?utf-8?B?R3RUQjNhSFZldFpHNTAxVURVNFVTOTNTT001emVseGNLKzIyYnpKaUwwQlRn?=
 =?utf-8?B?b3I0Z3RVRnRwVzZQWkM3NjBBY2VGZ25DeWRCTnF5N3lvOGdCa2Y5Q2NYLzhq?=
 =?utf-8?B?TDRUSVZNcmQvbGhKbWVsc1lLYjEzaGRjbUdqWlIzN3o4WVdzUXA3SFZRdTh5?=
 =?utf-8?B?MWtkbndETU92ZUVkaWM4cUZRcXc3cFpMMy9XU1BsVlRhTjlpWTJ4cElzWElZ?=
 =?utf-8?B?bUVSaHVlQ2JESDdIM210ZDNRYU5jclZ5eUJJTmR6aWIrY3E1cXlCSDlOYmJE?=
 =?utf-8?B?ZzBFUkZhTnhlSEVjeVRNWUQrbnUycGJCa0RpK0hnYlRFbW9lY0luOW9NcEx2?=
 =?utf-8?B?OUV6YWxYK21vR0tWemxYSW9WVU1ieDYvdk40STVEZmNmNkUxSFlZejNtVnlr?=
 =?utf-8?B?OWJ2ZnhnUnQrKysrczYzWXcyOUZJaXB6c3hYbm1uY20rOU5LTWJYS3AyenZU?=
 =?utf-8?B?anBhdnFpVHFGendJcE14dFlrcGZSZnFWOUdITTVxeFdFVTdoQjFsZFN3Q0ZZ?=
 =?utf-8?B?MmpHYWZQeHp1OWwxQUt1M1Awb0sydWpwZ3ZodmI3eEY3ZytScnlFSVlkSEkv?=
 =?utf-8?B?UUplbWdxR011UldIZDNRWWI0cUZuTUJjVHlsSkcrd1pydTJKaUtoQ0FZUXEv?=
 =?utf-8?B?d09VWnhrRnl5KzRQOGhpSDNTQ2EyTCtzZUY2Q3RlNWhaNjYyYzhLdXNkQnRP?=
 =?utf-8?B?MTlBUW9BajR3ZElCakxkNFFFVW9UOG1ydmNaRlh2dlVDVUZwQ3BhTzhBWkg2?=
 =?utf-8?B?MjByaG1sZ0xWR0VQb1N6MDdBbjg5QWprZ3QvNDBsbzhMZSttc1hIMk1GbGd6?=
 =?utf-8?B?cHhHbzRBQ3hOSDRiQ213M1d4cVh6TUtwdnh4SytOZHBQUW5TVDJWYk1NbGZD?=
 =?utf-8?B?dDZZcHBFbm1RLytqdUdZamROSWp2MUJGZWp6ajhXeHJkODFJRnpnbzhUTkkx?=
 =?utf-8?B?R2JneE9GYnFaSk9paDJNQmV3Z1V3eUR5L3o1R2FBb3VzUXFmbzdxQ3Z1YlJ1?=
 =?utf-8?B?U0w3UE1aN0xLd0MrUWR5b1cvR0dHdGpHNTQ1YmVyRUNodTVxNEt6WXpidUth?=
 =?utf-8?B?SVhvTzlWVkc4a2t3WHJxRXdndC81VzkwTnpidXpEMC9CaWtPeE1ybHZWMFFx?=
 =?utf-8?B?SWR0UzZXZFFhMG1lOUp5TGk4U3RNK0hnRkRIY2cwK2ZQTUJwd0M1dDNkUzVF?=
 =?utf-8?B?RVM5M1VKSkhBTmR3L3hLQzRzUGZyZHc0QW5yb3RUZzJLVkNsU0ZHZExRWGpR?=
 =?utf-8?B?T3IwVzlLZ21tamU2SGdLdzRHVm9lRXd2MW54UG1rNWtOVnRtVEhDc3BQZW9L?=
 =?utf-8?B?cWVUR285ckJ1QlllTGtHWVphdWFlL2QydzBHRjQrL1I3TU9VN3V1M1JpWHlX?=
 =?utf-8?B?NlhzV2o1WkFVK3lRSTVIRUNkWTRXNmNEQmtYSmlSNWdmR3lrMGVtNUw4ajN4?=
 =?utf-8?B?UDlUSTJmRnJQRFdNVWpIYk8wSThneVJpSlpKUFVhOHBKdDNtajdycVZiWmIw?=
 =?utf-8?B?UUZkbVFJUHRRZ01lMjVjeWJ1THUreXZSck1ESFVheHAyZXcwS3pVdHpNOElj?=
 =?utf-8?B?RFdXQmgrU1V2Ynh6UEVqdFVuaitud2NNRi95N2tpbTVBOSt3ZElIUWdmd2JU?=
 =?utf-8?B?NFBRVGhGdFFPaGFUbXIrQURMMm1NT2NvWUxML04yNlVmOHFxbndINWhZeFh1?=
 =?utf-8?B?Um1ZZUVFNW96Mk5mc2ljNlpoSkJNK1JmV2tKdEZ5T21QMXFub2pOaXFSem5s?=
 =?utf-8?Q?zKm9hZA9vyG7EnX80FmOrs42v?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	Yzs+ibjoXeGV1hHsRX4/hs/QULq6qJwTPm6uaQF1ymsXdfYWuUOI5JN8kr/uchI6/2PvoinBNhvYODTyJnzCJf5SCdEZpKzwePiNYjyleXiKcTgvoHz+IdxL1Gwx0vGB+ciu73Uzav/HmjCFHJqTKv9+5pnLuCjxc5QtxzJjnJMct2v/LPv83KSDSAuG1ph/U1UJPS1iXyNubvn07T/UDxuiFS+ip5Rsq7Mc2+9k33us7qr6cuXUZVVdsVf0ZCrBm/x6EWIFtp9Q+Tg2i7WrasJ086fZSAazoycICNuQyAmJF39t+WhiJF1MqDRECgA0i9c8jaC9U8Ef9OnEQmWKRStALfrbJK/BZpi6VlJBwYEgA6q6RGR9rFEDg9mCs8IHmxcYdPlv7htEoTYWfme0GmcVboOhRWWW4FNZ8NgjL6nLMP1XlH0TlHrthtNTV5vB77v0dvtdmBI3f4ugrf+HVMGXNLIDm4eQIH/L3HztdltWKR4n47G8pNdMMMhLzocl/DiTfwRtix8NchxhF976RggrgcIgd5GqSyHt3OxC1IwP4eh1YQlHR+HxGs94CDgNXuYnz4O7ttdO4Bh3n+AcrEmvnN+R8yCZ9+Qh9tBuofs=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d476e1a7-bd03-4c37-c4e4-08dc8164fd90
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 11:29:53.2859
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Db5dSHdl7T0cz6ANtAgG2RQtGwsE+3n5Fp5M1A2lWCt5TM9Txz38OTtDo1X1QaSFsyZN0k/KXNkE8O+LT3ftuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB6839
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_08,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405310085
X-Proofpoint-ORIG-GUID: M4WNfROpa8qZMhhhv8K42vQilSAILYRt
X-Proofpoint-GUID: M4WNfROpa8qZMhhhv8K42vQilSAILYRt

On 31/05/2024 08:47, Christoph Hellwig wrote:
> Hi all,

Just saying that b4 for some reason does not create a .mbox to apply for 
this series when using "b4 am". It also no longer does for v1 (when it 
did previously). I guess that the series versioning confused it.

> 
> this series converts the SCSI upper level drivers to the atomic queue
> limits API.
> 
> The first patch is a bug fix for ubd that later patches depend on and
> might be worth picking up for 6.10.
> 
> The second patch changes the max_sectors calculation to take the optimal
> I/O size into account so that sd, nbd and rbd don't have to mess with
> the user max_sector value.  I'd love to see a careful review from the
> nbd and rbd maintainers for this one!
> 
> The following patches clean up a few lose ends in the sd driver, and
> then convert sd and sr to the atomic queue limits API.  The final
> patches remove the now unused block APIs, and convert a few to be
> specific to their now more narrow use case.
> 
> The patches are against Jens' block-6.10 tree.  Due to the amount of
> block layer changes in here, and other that will depend on it, it
> would be good if this could eventually be merged through the block
> tree, or at least a shared branch between the SCSI and block trees.
> 
> Changes since v1:
>   - change the io_opt value for rbd
>   - fix a left-over direct assignent to q->limits
>   - add a new patch to refactor the ubd interrupt handler
>   - use an else if to micro-optimize the ubd error handling
>   - also remove disk_set_max_open_zones and disk_set_max_active_zones
>   - use SECTOR_SHIFT in one more place
>   - various spelling fixes
>   - comment formating fix
> 
> Diffstat:
>   arch/um/drivers/ubd_kern.c   |   50 +++------
>   block/blk-settings.c         |  238 +------------------------------------------
>   drivers/block/nbd.c          |    2
>   drivers/block/rbd.c          |    3
>   drivers/block/xen-blkfront.c |    4
>   drivers/scsi/sd.c            |  222 ++++++++++++++++++++--------------------
>   drivers/scsi/sd.h            |    6 -
>   drivers/scsi/sd_zbc.c        |   27 ++--
>   drivers/scsi/sr.c            |   42 ++++---
>   include/linux/blkdev.h       |   52 +++------
>   10 files changed, 210 insertions(+), 436 deletions(-)
> 



From xen-devel-bounces@lists.xenproject.org Fri May 31 11:35:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:35:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733663.1139966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0XS-0000xB-Vx; Fri, 31 May 2024 11:35:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733663.1139966; Fri, 31 May 2024 11:35:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0XS-0000x4-TL; Fri, 31 May 2024 11:35:14 +0000
Received: by outflank-mailman (input) for mailman id 733663;
 Fri, 31 May 2024 11:35:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i5xk=NC=oracle.com=john.g.garry@srs-se1.protection.inumbo.net>)
 id 1sD0XS-0000wy-A7
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:35:14 +0000
Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com
 [205.220.165.32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d73b2ccf-1f41-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 13:35:13 +0200 (CEST)
Received: from pps.filterd (m0246629.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V9Vfrq025637; Fri, 31 May 2024 11:34:55 GMT
Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta02.appoci.oracle.com [147.154.114.232])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8p7tvr1-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:34:55 +0000
Received: from pps.filterd
 (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44VAwbVT016267; Fri, 31 May 2024 11:34:54 GMT
Received: from nam10-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100])
 by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yc50trgj6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 11:34:54 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
 by DM4PR10MB6839.namprd10.prod.outlook.com (2603:10b6:8:105::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 11:34:52 +0000
Received: from DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088]) by DM6PR10MB4313.namprd10.prod.outlook.com
 ([fe80::4f45:f4ab:121:e088%5]) with mapi id 15.20.7633.018; Fri, 31 May 2024
 11:34:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d73b2ccf-1f41-11ef-90a1-e314d9c70b13
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-transfer-encoding:content-type:date:from:in-reply?=
 =?UTF-8?Q?-to:message-id:mime-version:references:subject:to;_s=3Dcorp-202?=
 =?UTF-8?Q?3-11-20;_bh=3Dc/o75S0jRnXqE1AiLDzyv0X6Ye3yWpS9b0ZDev3AVfU=3D;_b?=
 =?UTF-8?Q?=3DIL9BxH26tZCBICwB/a1j5GfUhpMYot80v1T0zsUENhUpNOs6eCHxdZQ77bXR?=
 =?UTF-8?Q?Bffd8+Qs_u72qmmrUY3+vGCyh11qkVMlqVZxWihTeyVRj/2OzGqNXnytJZrDk+7?=
 =?UTF-8?Q?P9M945tCAN+4Br_pwh8ao/8mgsbLMkZ9eJ5F9n1ABC27NyxMyBgvO7ZwFHKNoh3?=
 =?UTF-8?Q?SCjhCynaB7uDUybNU6ZA_o4Wf6ibavedYR6muOYQVzVoL8SVRIU+jv+L1FtuJ5J?=
 =?UTF-8?Q?AWhh+ESBdkyVnATmhx5LTpCeaD_C9HN9PeNgjtYvDQnOGNBLDJJpyQ5jwCd2ZPK?=
 =?UTF-8?Q?TmuelkxPcatcWUx4QWEe0EITQ3iQ5XmK_NQ=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VGXenzG5sY1u3vF5ni28A7xCbOpAw2Ce0WIeDCB1exdkr8rCW8Ady6jK/iOsX5u6UCTYPjmBbN5ViowEwT41P6p+5NdGBl2oPD6jcNBzDoq8w15baoMrRLRoe4541sC3gJ+UW7GxkaPiHjEgJIj5RKkOvjiMnMOC1ZpmQrBsk1cNYRelaDkjwxqgGA2LDz+8dYhrFBYH6/yr6hJ7eVs7Q8k2fzoON7DB9fpBO4PNEX4bth3obCIlx5NqzYuglK1nQPu+MUMhYWNgLHLuGVMNI3GinatZmnr5x8o7oe34+UxSRpiD8HZD59KvKgtCVvsobxJoyNEgQuXEbjUoPvs2jQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=c/o75S0jRnXqE1AiLDzyv0X6Ye3yWpS9b0ZDev3AVfU=;
 b=RoQH/N+orWBJoto1HBvT7zOunFD0bFQggsyawMW9ARVzBztoSP5YQL4KfA942J2Awfyp6405Zb7ZMYMuMF3AqAJZ2cZOMr22m0JdLLbf5AHmGqAN7AWpB761EXylmZegdDB0GgzjQ1tRn3HqPsb7/9df1C883M/lAQZo/0spwLhAl0sF4XCueCc1ypJfbkjwPDHGZhX6rwoHMqu3U6HfJ/UJwdrObTaZqEq+RLftf79Cu4eVfnK5/SzeWS1evsLOTaz2lHfelG/2ZJsZ9zICwfQDWMo1bWxmILBf8/oeC6H/kXgPc/QKBuwzl2Ry+KvGaJkga9NlxbmZfIm98pR6AQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=c/o75S0jRnXqE1AiLDzyv0X6Ye3yWpS9b0ZDev3AVfU=;
 b=euNCWObatSp1hnPcV4yvVokXZMyRM8Fzqn6N3jFQdKPgOTv8zcNN/x/kvNyuyReor1+00UUpeQyAPcr5UlOERnrs6cyJE4ngxcbLWUehQqMUy6VjGJaNpHLc+58Rflg7vn3ifjia5B5MxxT5d5xxxsA6Vemi/fJAkfgbqLtF7yA=
Message-ID: <a6768ddb-54c5-456e-abcc-39ed99509c87@oracle.com>
Date: Fri, 31 May 2024 12:34:47 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 11/14] sd: convert to the atomic queue limits API
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Richard Weinberger <richard@nod.at>,
        Anton Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Josef Bacik
 <josef@toxicpanda.com>, Ilya Dryomov <idryomov@gmail.com>,
        Dongsheng Yang <dongsheng.yang@easystack.cn>,
        =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org,
        Damien Le Moal <dlemoal@kernel.org>
References: <20240531074837.1648501-1-hch@lst.de>
 <20240531074837.1648501-12-hch@lst.de>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240531074837.1648501-12-hch@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO2P123CA0040.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::28)
 To DM6PR10MB4313.namprd10.prod.outlook.com (2603:10b6:5:212::20)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DM6PR10MB4313:EE_|DM4PR10MB6839:EE_
X-MS-Office365-Filtering-Correlation-Id: 936c4abd-e5d7-42dd-cf8d-08dc8165afc0
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005;
X-Microsoft-Antispam-Message-Info: 
	=?utf-8?B?N3ByRFNFU2tOcHg2QTNDR2VrV0p6SUFzRHpMQVVTZ2lrK0tvV1k4UVRtOHN4?=
 =?utf-8?B?aU93ZEE4YTN2N1ZUODlBMElwM1lCYUJQWHJwQjhpRmMrYTI2UktsNERVcllj?=
 =?utf-8?B?QUc0eXRQeEVTUTVQMEdYaWdNRFBmc0grcDhBZGVkdnFWZExBZEtLYnAzUHVY?=
 =?utf-8?B?Ti9KWEMyTml6Qjh3eGFjY0cvbndmR2tqQVZBT1B2ODJ4WHZZRUFjWHZRdEMy?=
 =?utf-8?B?ZkxrV3A0dnFQVTdHN0dvaHJ5TTFKQ3F2MGZka2V3VThkcVhSOXg0SGNFeklH?=
 =?utf-8?B?bVhCSzA5dHNUekJ4Z3hGVnQxZVlJYUxqUFV0VVhTZVFSSC8weUZweHF6N3pZ?=
 =?utf-8?B?Mjd3SHozWHhXRFRNOU5xNCtzZENmYzVXVXZZcW9uVTR2bEo1aGQ3VytsVkk3?=
 =?utf-8?B?M28vOWxPOGFIR3U2MTg4TDhPZ3N5UmlkbmVLdmN0TG1lb0svenJZaWlJREI4?=
 =?utf-8?B?NzlKam0razF1UVJaRFV4MlZBbHBKc2xIczYzYlJ1Rm9yU2NXNGtaeEl5VUtm?=
 =?utf-8?B?QW1aUlZCcUgrWk84QlNVNXJ3eTZuNi9idkJ1QVNVcmdjZ3FXM2ZMQW1SenNZ?=
 =?utf-8?B?bjJkK2pPREpRN0dCQ0lsVlk0Z25McFBvTTVLYzhGeWN4cG1aSktzUlhQdzlT?=
 =?utf-8?B?WGtIWHVHNkxGcmVnYVZHb24rRGFJdTZQUFMxd2t5bDdoWWlud0lCL1dsa203?=
 =?utf-8?B?QnI5M0J5TlM4QlBrLzdEejZncCtxSWsyOG9WcTFZelFsSUoxME9GenUzelNu?=
 =?utf-8?B?azJuOWxJK3AvTG85UlRxTEx2SmdkSGh1Wk1hSm04M0RFVG1BQ0JmOWdiMWJ4?=
 =?utf-8?B?VUdIUmV5cDVHSWRPeGJjOGRHRTVJb3RTRFVXcHRNT1VsOUhBaWQ3Y0FDMU9U?=
 =?utf-8?B?Mkc4R0U3clJCVzZUd3QzTGNIR1libi9ReElVUE9aNFdpdlZJb3g4T2wveWhF?=
 =?utf-8?B?ZXhhOTN0QXlhYWhNRXhMSXI4eXRnQjBza0lYcGd2Y0p0VUhNN0dtcE5KaDZo?=
 =?utf-8?B?cU90NUpzNlVPcDJ3aEt5SDkrY05qMklzV1hURXk1bGNnL2Z2T05UNEhvVldW?=
 =?utf-8?B?TlVMY1J5K0JiK0FxT3RwQmsyWjJHdk1zNmcyNHI5eTFPS1RLNWZkNHdtTjhS?=
 =?utf-8?B?VlZaY2xTVVczSVA5Z21Rd3NQemdpeGJ0SXlkenJrYWlQbWdtZUFkSHRpSXNF?=
 =?utf-8?B?YUJzVnc3WG5QN21reUhFRmJzUXVTdm81TjFNMmpwb0xKS3pNU2dONFpCU2Qx?=
 =?utf-8?B?amhMYzdxSW1YQnBFNlJ4M2dhOWVXSGFZR3RySit5Wm8zYTd6aVlJSjFqQUNo?=
 =?utf-8?B?QU8vU2Yvbmh6aFlTMjRnbGxNSGVycCtubTB3SnhnN1MxMXJJZjFYVm5pQllY?=
 =?utf-8?B?Sm1TTVlaTXd6ZmVmaUtGOU5nWmdBVU5HUmNLSm5jaXQrcElwZ1JWeDY5SUdz?=
 =?utf-8?B?U0QxWDR4c2dBdnFDRytzYzN4TDhlbTlaQ3pMU3l5TE9SUUZIWVVtSWUzVFNM?=
 =?utf-8?B?cmpIN2NtYkNmM0IxTlV3UlhFT0dBWnF3Z3V1SnVpZWUxUTFxZkR3MXkzdjEy?=
 =?utf-8?B?SjdST2luMjdLL3Z2UDd0NWIyQ0N3QTR1bjRkM3grUHg1MzBOMXRYZ0VTL3BT?=
 =?utf-8?B?b1JHYzgzcS96eGNLR2owL010dStlckErTGZCRUx0dmlOb0g0ZzhmV2grWEl0?=
 =?utf-8?B?OTc2anZqUmJPNW1OK0E1N2NKQXZRV1l1WkYwKys3V0l5b2E5S1ZBbTFBPT0=?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB4313.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?bElBSENtZnIxdnErVnYvc2pkaGdSWi91ZDk0ZG5vZTJjUVZXaU1QeDh5RFQ1?=
 =?utf-8?B?cDBndkNZYnB1OFNtZENuS0p1eE9NK04wd3JhYVhGamUyNDhiWUZYQmhVUUFJ?=
 =?utf-8?B?MUJ4dUxTNWJhdXdrKzh5SnVsZVBURkdyU3pOZWFESmJsSXNyM1BGdmJEcnVm?=
 =?utf-8?B?cU5QVTZDNXlUM1RJTlN5blRDQVpzcXBYdzZNZUJOcWw2ZEFzM0JYaS9rNkQz?=
 =?utf-8?B?ZVJjWjdadHRZWlprcnpDMTNGZG1VNzVZV2dlS3A0dVJxdU1iMXFuSmtEWjJC?=
 =?utf-8?B?dEVPdXlteHdhcDZ2ZGZydWVIL1Vvd0ZrdDA3eDkzdldlUk9rdUhHOWpwdElR?=
 =?utf-8?B?ZlBBSFNrclB6ZDczVHMreGl1Q24wU2FaWTdHajVlc0VMOU9icFpGZFQ4amRo?=
 =?utf-8?B?SWhjb0l3d0xMVnorejN1L2svWmtVNXFSKy9DeGxWeU1iN1hlZEkyaWFvQnJ5?=
 =?utf-8?B?RDhkL2FldjBzK2lmYS9nYUhDQ2ZsS1JQeHRVZ2ovb3BMUmkwMFozclNGMURP?=
 =?utf-8?B?M0tPQjMwa3l4WkgzbFU3Zk9JYVVuZis2K2tJQXE2N0JEMTJwcmcyc0N2QjFS?=
 =?utf-8?B?UThRYVM3ZHVHUFMwdXY1UE1idlZ3RVAxekxlYzZ1S3FaYWlBUDRHZjVxWitF?=
 =?utf-8?B?dm9pcUpzY3orM0YvOXFnaWZwcjcrWVdMdnJkVWQwZ3JIdldIOTk3M1BseWwz?=
 =?utf-8?B?Z2xrRFNDWjZjMjVTSkVxd0JOM3BPRUQzWWwyTEdBbDhuVnJPS3BLcGEvT0xh?=
 =?utf-8?B?eEkzUmxsTlhBTUx3aWkzekZ5YlREOTVkZDVzaUEwazNhTnBxeTk0QzNSU3Mr?=
 =?utf-8?B?WWZmbUxjbEo3QndXYXROdmV5eWNzaXoxakkxWXZvRHBlWGc5MzJJZCtFK2w5?=
 =?utf-8?B?NUlhN2t5czQxUXk0M1Y4R2F5Nld4K3Bxdlpld1p6Q1pXalh2a2lBK0hJczNI?=
 =?utf-8?B?TnpSeGs1VG9zVHZMeVMvNUVQNmRPUWlBTVhNSE5KaUdMRnVXdHh3TWpPakt1?=
 =?utf-8?B?eWFJRGw3cndSRzFMSzlIMFgweE5zWllOTTVkQ1FSR1dLVUs0NHdKY3QrS0Rs?=
 =?utf-8?B?L0NJSEdEM2dZUEJscE9MTkpNYitzYVdaWlp3Y1hHeCszWnRQTGR6SWl2azJT?=
 =?utf-8?B?RlBZeXptNXljaFQzQmwrZldrQWFGVytmZDBNSTJES0NsQkhZR09GaHdkN0hK?=
 =?utf-8?B?S01KZmZOV0dYYVhQOGFYMGVGcGNQSWgvcVFvZS9FQ1p4b1FYaHU1UHZubHdZ?=
 =?utf-8?B?V1FXQjVtVzdNenMyZENRbTdEajdkWlNqclRncWEzUnQ3ZE1jZzRFSkZQUTF4?=
 =?utf-8?B?WjIwbWpHNDZqTy9sejNXUDBvbUhMOXlXS1Bmc2pRZ0o3d2tYQ0I4czJicnQ0?=
 =?utf-8?B?SzlGQW1YYWxrOTFQazVJS3JwY3lCc0hjSlhsbkE0OGM2VzB1b2lkRVBUekVp?=
 =?utf-8?B?VlkwTlRlRDltT1hpL3h4U2pvVUtFbkxoUFRGNEUzTWpPdXc5VENIV0NXUEV4?=
 =?utf-8?B?cDZ0ck5GUlpNMm1mZ2hlWHdTUmcyclRnK0Q1MFN4eDNVbFd4TlNLRTVLRUFm?=
 =?utf-8?B?WGpWOW02dm84NmNLd2ZBanVGaWlFZlpLZ0RRd3pDS0NqMzNpWVdVOWxXOTdt?=
 =?utf-8?B?UnZrcVBGbm5EVDhpakhGRjhxL3EvT2NSRHpGQ0doRWtEaXNWSlc2MDVWMkkw?=
 =?utf-8?B?WU9CalFHS1pJSUppNFNNQjZ6ZXc0ekZKVkY2NHJpcUdBZTcvcjI5aVNmUEgz?=
 =?utf-8?B?RzJNV3EvS1ZkdGtFMDJxMEtDQkh0NWo2RHNGeW1GOVdoWTFzUW9GVVdhWmRY?=
 =?utf-8?B?WXhMcVpkYlJIR2Q0aEx1U0E5anZLaDZGelE5RWpRajVteE9CTVVZMXVlZGJU?=
 =?utf-8?B?VUdLVEw4UEEwSTBjdFdCZGJsb2xnMFBjTU9XRDZ4OFhpZjhJM2FSeDE3RlBQ?=
 =?utf-8?B?SUZ1VE5FQmNUZ3lCWmJwMENaN1JidzlQT0RFSWN1UnVoU29HbWZ2Qy83Q0w5?=
 =?utf-8?B?ZHNoZWdBczBiM0YyZnlidS9jOGJQNGN1R1lSWUd3a0tVck1NWHpMUkx5eUtG?=
 =?utf-8?B?bVEvU2hQbjR3TnVSRjExaG1GNzVoVXFPY0pvN1FrVEpqU2NMVnNFVGliUUtJ?=
 =?utf-8?Q?prqHOvwO8DUgdKPiBbHn6gl5T?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	tw5B1QTBDcjNGsNwzT+rV48POHjqkUOzwQa/L6a67uIkMBWPsZQWPZRx+IAJK8aTkaFLJF2O+RG5xANZpqxZ+943zvz+ITaAAuVSXVmiQzb+lZV9MXlsXEwvYUl0M0u2Gbc2kt+mFJ5zyQdcOAPlCx2Xz0ZHVhdKlwX6WK+6k62mFdB0dTcUJOt6cFMTRcs2Y8yG5xfLQSqG2Kg0yF+8ZE+y0HG4VBUkLQ4DCe2EC2nZPNnn9pCrLR8C922b+RE1mqvzKhvTNazr/ih/+m2QEuw29b/hZEj5zOBUvO8K2EfjnSHSK5hGWdu1q3OR6SAPiiz1MNrKD8v4jVm9O4LtlsA9OefIlYchgKaPUe+hjRgCvMlS8+WmGcVwClrGJ+J4MCS3k84O0zqdmx7eJeWvMQwSt1Q7MTNIdORXvszjylA4sH1QTufsBXtnUEWy87i47BfsD77XkqwVMzmlVvkPrjjwScwuXKGVDQ6Sy2TkMkOuZEoLsxlOGr9naX7s5XGAWq9d9UrFh/uy+vdjABzY4EFHXpMOHxHpxTJA9Sb41oivrqm3oFeq75gOft2Jxj6PyFitBTyr6BkOkEGtdGHNLYKeO5fCtSw6v4bm//YoIZU=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 936c4abd-e5d7-42dd-cf8d-08dc8165afc0
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 11:34:52.2013
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DkTYbYrC9k6wM9GxKt1HVQ1FEA244/KCZCU8tRgwERMgmMgjG3Sp8nWedFBwBrfMiZde7B+x5cFa+DR9NRNSaw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB6839
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_08,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405310086
X-Proofpoint-ORIG-GUID: 4VOGg0iTfSa4Hi-AE7demkKzZNFEXj5v
X-Proofpoint-GUID: 4VOGg0iTfSa4Hi-AE7demkKzZNFEXj5v

On 31/05/2024 08:48, Christoph Hellwig wrote:
> Assign all queue limits through a local queue_limits variable and
> queue_limits_commit_update so that we can't race updating them from
> multiple places, and freeze the queue when updating them so that
> in-progress I/O submissions don't see half-updated limits.
> 
> Signed-off-by: Christoph Hellwig<hch@lst.de>
> Reviewed-by: Damien Le Moal<dlemoal@kernel.org>

FWIW,

Reviewed-by: John Garry <john.g.garry@oracle.com>


From xen-devel-bounces@lists.xenproject.org Fri May 31 11:40:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:40:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733669.1139977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0c9-0002Yp-LF; Fri, 31 May 2024 11:40:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733669.1139977; Fri, 31 May 2024 11:40:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0c9-0002YM-I9; Fri, 31 May 2024 11:40:05 +0000
Received: by outflank-mailman (input) for mailman id 733669;
 Fri, 31 May 2024 11:40:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xPZo=NC=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1sD0c8-0002Ae-0n
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:40:04 +0000
Received: from wfout1-smtp.messagingengine.com
 (wfout1-smtp.messagingengine.com [64.147.123.144])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 837edcfa-1f42-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 13:40:02 +0200 (CEST)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailfout.west.internal (Postfix) with ESMTP id 18F651C00224;
 Fri, 31 May 2024 07:39:59 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute3.internal (MEProxy); Fri, 31 May 2024 07:39:59 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 31 May 2024 07:39:56 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 837edcfa-1f42-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm1; t=1717155598;
	 x=1717241998; bh=PtyDoQ5hXIUw/iYJcJ0Fd/M4Ak5wsByW/SqcYzEaX/Y=; b=
	FjH2Uz3gFWVXntIWh/tFZBVsB0yZQpEIbnBJhHlm1tquzjcFOWQZ8GhzHLnZwWZl
	dsDpue7nhFXqpRYtUro9P+UsNWaRqOkwqi/a2ZCKySflULqAEFVgc8yqEi6+ylAL
	OfTrpBk0zC4h/suma/SjgbZ68T3S8luD+K4H5+bvWb9wEgZf85/4eOrU0ny1FyTo
	hL/Ey8gMbRl0XpbWVf3yUuOlxOJ8o0zpA1uWCGssynrELZYev0Mv54ly3X2UGqcN
	FC1AXz1pz/hCjQGzV3n6TShkNzD9VCwLowPRzLMEu8JpieHy2TvuhjTQUxOHR+74
	7UBnD+l5Kl9ReUBmN6rc0w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1717155598; x=1717241998; bh=PtyDoQ5hXIUw/iYJcJ0Fd/M4Ak5w
	sByW/SqcYzEaX/Y=; b=kfJ8r3d/HPvq4TDMlUmxkVw9c2uiVJI0N2aPLS9bhGP0
	YWpu92UeJ1P1AXC+bPM+OjO93kzWjiGZAgq2gcY/X8PDbZ3nwxFTgTs+WxccKI7S
	uIESJ4vlDgqgyGuDcmxhAj+QIJ0Zo9m1ok/8qOLfwohjtWmeeTLLBYvYgmWhINly
	7PJEm016mXft6bsFhvQc0j52y9oNYKZxPovYusbcDxq7jZCioguDGYFpiceB8W4n
	D9cnoKAo/tZGnIXQcepg0zhGmfEDUvMPtnNzVvgtnT7QqPlrSgCRt/XWgsItY4IP
	/P4cTD9gMgxluTG0IAHtyojvDldnwZqGL4BHxlsJQA==
X-ME-Sender: <xms:DrdZZmSoj_HAULYl5hfZn-cZwWNceuMbp88tqPlvwzGpjyfW-l_7uQ>
    <xme:DrdZZrymmic_iJ2jENfSlgp-mptBh-lXj6RAAd61moqF36DCmPbHTDecELUjOlvS4
    5guYjNs6hpPTQ>
X-ME-Received: <xmr:DrdZZj0-VoP8NyDw0aAF23RwhRukJDplFiQZ35yD2NKQae61IGlEz5erO0I>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekiedggeduucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:DrdZZiC6wvK-z-wSibhxhty8VYeGSdt2xiozSxmRdGvzxuUX59ONwA>
    <xmx:DrdZZvjD5eGZLkfgS-Ecl8J5vsZe3v6sRxx7wiVUz0u-ZdeJ3IiXTA>
    <xmx:DrdZZurca9YnRObjruZwF6g5yu5ZfeCuIZxdBfj8i3yi8Fu8ML4tdA>
    <xmx:DrdZZijdvDrdFXLtRZLZ6WwPXrY4WEIAN1DuNCmAJEWzHRvfotztoA>
    <xmx:DrdZZlYRF1_gY_Arfhaojkz3wMRa2xma2m8EBRLbdbePdu9ADLc44Y8f>
Feedback-ID: i1568416f:Fastmail
Date: Fri, 31 May 2024 13:39:54 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for
 HW jobs
Message-ID: <Zlm3CwgwccpPU4f6@mail-itl>
References: <20240529141945.41669-1-andrew.cooper3@citrix.com>
 <20240529141945.41669-2-andrew.cooper3@citrix.com>
 <ZlfHh_blx1i0eE7O@mail-itl>
 <alpine.DEB.2.22.394.2405301715010.2557291@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="0qbZMnV+fSlTWEii"
Content-Disposition: inline
In-Reply-To: <alpine.DEB.2.22.394.2405301715010.2557291@ubuntu-linux-20-04-desktop>


--0qbZMnV+fSlTWEii
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Fri, 31 May 2024 13:39:54 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: Re: [PATCH 1/3] CI: Remove CI_COMMIT_REF_PROTECTED requirement for
 HW jobs

On Thu, May 30, 2024 at 05:43:12PM -0700, Stefano Stabellini wrote:
> On Thu, 30 May 2024, Marek Marczykowski-G=C3=B3recki wrote:
> > On Wed, May 29, 2024 at 03:19:43PM +0100, Andrew Cooper wrote:
> > > This restriction doesn't provide any security because anyone with sui=
table
> > > permissions on the HW runners can bypass it with this local patch.
> > >=20
> > > Requiring branches to be protected hampers usability of transient tes=
ting
> > > branches (specifically, can't delete branches except via the Gitlab U=
I).
> > >
> > > Drop the requirement.
> > >=20
> > > Fixes: 746774cd1786 ("automation: introduce a dom0less test run on Xi=
linx hardware")
> > > Fixes: 0ab316e7e15f ("automation: add a smoke and suspend test on an =
Alder Lake system")
> > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >=20
> > Runners used to be set to run only on protected branches. I think it
> > isn't the case anymore from what I see, but it needs checking (I don't
> > see specific settings in all the projects). If it were still the case,
> > removing variable check would result in jobs forever pending.
>=20
> Andrew, thank you so much for pointing this out.
>=20
> I think the idea was that we can specify the individual users with
> access to protected branches. We cannot add restrictions for unprotected
> branches. So if we set the gitlab runner to only run protected jobs,
> then the $CI_COMMIT_REF_PROTECTED check makes sense. Not for security,
> but to prevent the jobs from getting stuck waiting for a runner that
> will never arrive.
>=20
> However, like Marek said, now the gitlab runners don't have the
> "Protected" check set, so it is all useless :-(
>=20
> I would prefer to set "Protected" in the gitlab runners settings so that
> it becomes easier to specify users that can and cannot trigger the jobs.

Owners of subprojects can control branch protection rules, so this
feature doesn't help with limiting access to runners added to the whole
group. Qubes runners are not group runners, they are project runners
added only to select projects.

I don't remember why exactly runners got "protected" disabled, but AFAIR
there was some issue with that setting.

> Then, we'll need the $CI_COMMIT_REF_PROTECTED check, not for security,
> but to avoid pipelines getting stuck for unprotected branches.
>=20
> It is really difficult to restrict users from triggering jobs in other
> way because they are all automatically added to all subprojects.
>=20
>=20
> Would you guys be OK if I set "Protected" in the Xilinx and Qubes gitlab
> runners as soon as possible?


--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--0qbZMnV+fSlTWEii
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZZtwsACgkQ24/THMrX
1yy5+Qf+ICtQHPgXu0IXrYCdZYqHJNc18ypMaNIJKR4Ltj0OF40ylFkEcVSBswnX
Z6y0ZlJH9D6R8CFgIIpR5rQ0YxIExslDUj2E1KFnluqt6zg3fFN23eDxflYvmI8C
4O7tEcdDLlB765NAWyCY+CvBvO1yG9xCW0WrkWkKOAwajqYNBUcpVOQ/syhftUnZ
JWb+ePoPrlwq1bDKqHO7jidmjtGIMdfzt2bbJbR4m70VRb5Zftai0uyjr54p+km0
omXfL+T5SJRctnUpcH+xS7Z38gjCKaLW29CXXKJiKFJibyhizm345ZD26nXz5ycm
6tB+eUu7RLxgPCj4JxywV8gi0NVV3A==
=162x
-----END PGP SIGNATURE-----

--0qbZMnV+fSlTWEii--


From xen-devel-bounces@lists.xenproject.org Fri May 31 11:55:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 11:55:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733667.1139987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0r1-0005Eb-Td; Fri, 31 May 2024 11:55:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733667.1139987; Fri, 31 May 2024 11:55:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD0r1-0005EU-QS; Fri, 31 May 2024 11:55:27 +0000
Received: by outflank-mailman (input) for mailman id 733667;
 Fri, 31 May 2024 11:38:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FQNi=NC=samsung.com=nj.shetty@srs-se1.protection.inumbo.net>)
 id 1sD0aE-0001ZQ-06
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 11:38:06 +0000
Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39a5e14f-1f42-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 13:37:59 +0200 (CEST)
Received: from epcas5p3.samsung.com (unknown [182.195.41.41])
 by mailout3.samsung.com (KnoxPortal) with ESMTP id
 20240531113755epoutp03fc728a1c21586f57d87e19a419a7b2e9~Uj86mlAlA2987829878epoutp034
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 11:37:55 +0000 (GMT)
Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by
 epcas5p2.samsung.com (KnoxPortal) with ESMTP id
 20240531113755epcas5p271b8ac6d7e7f1402241b70b4a7ee73b2~Uj86NVzG_2148421484epcas5p2O;
 Fri, 31 May 2024 11:37:55 +0000 (GMT)
Received: from epsmges5p2new.samsung.com (unknown [182.195.38.179]) by
 epsnrtp1.localdomain (Postfix) with ESMTP id 4VrLhd3fPBz4x9Q0; Fri, 31 May
 2024 11:37:53 +0000 (GMT)
Received: from epcas5p2.samsung.com ( [182.195.41.40]) by
 epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id
 CD.E9.09989.196B9566; Fri, 31 May 2024 20:37:53 +0900 (KST)
Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by
 epcas5p1.samsung.com (KnoxPortal) with ESMTPA id
 20240531110559epcas5p1f41d7d904ed46260837c38a57fa32d86~UjhCSqWw21511815118epcas5p1x;
 Fri, 31 May 2024 11:05:59 +0000 (GMT)
Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by
 epsmtrp2.samsung.com (KnoxPortal) with ESMTP id
 20240531110559epsmtrp2e32c4ef6effff2d97c7dd8e4c00c8f46~UjhCRz5O50896008960epsmtrp2m;
 Fri, 31 May 2024 11:05:59 +0000 (GMT)
Received: from epsmtip1.samsung.com ( [182.195.34.30]) by
 epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id
 C0.22.07412.71FA9566; Fri, 31 May 2024 20:05:59 +0900 (KST)
Received: from nj.shetty?samsung.com (unknown [107.99.41.245]) by
 epsmtip1.samsung.com (KnoxPortal) with ESMTPA id
 20240531110557epsmtip1d7d4153c068ec15012f898a1f6bb80b2~Ujg-zJJlx1968819688epsmtip1t;
 Fri, 31 May 2024 11:05:56 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39a5e14f-1f42-11ef-b4bb-af5377834399
DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240531113755epoutp03fc728a1c21586f57d87e19a419a7b2e9~Uj86mlAlA2987829878epoutp034
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com;
	s=mail20170921; t=1717155475;
	bh=Y0VkJe20fMinBlmQK31o7n10A9UqotHuj7Gyw5qxaTQ=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=sRleW/KoTf36lPqWsVssBbgWRUwgJKB+m6Dbwqh+uMUhgrg0M4BXwNNDnk/cUHi8I
	 iVmL+eUijzT1lGesNcZ4YRq1L/gz8j6q81Q/1LsogkscxitQG3Bi/mzIOAm2DtjXXu
	 lU0IBsv3w3WhomYH3Tr5N8omlXy2FkNMIAVlKOsQ=
X-AuditID: b6c32a4a-e57f970000002705-ed-6659b69163db
Date: Fri, 31 May 2024 10:58:56 +0000
From: Nitesh Shetty <nj.shetty@samsung.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, "Martin K. Petersen"
	<martin.petersen@oracle.com>, Richard Weinberger <richard@nod.at>, Anton
	Ivanov <anton.ivanov@cambridgegreys.com>, Johannes Berg
	<johannes@sipsolutions.net>, Josef Bacik <josef@toxicpanda.com>, Ilya
	Dryomov <idryomov@gmail.com>, Dongsheng Yang <dongsheng.yang@easystack.cn>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	linux-um@lists.infradead.org, linux-block@vger.kernel.org,
	nbd@other.debian.org, ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org, Bart Van Assche
	<bvanassche@acm.org>, Damien Le Moal <dlemoal@kernel.org>
Subject: Re: [PATCH 14/14] block: add special APIs for run-time disabling of
 discard and friends
Message-ID: <20240531105856.nkr2gbyhwbzoqycs@nj.shetty@samsung.com>
MIME-Version: 1.0
In-Reply-To: <20240531074837.1648501-15-hch@lst.de>
X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTdxT2d297ubCUXSrLfmBk7CrJxAAtj3oxPOYg5G4QJFM3N3R4pZe2
	AdquD3FbNmDYOFh4yQbYMKaB8aiCWh6BkYYNRcTBWATH6CaCkaDAttYyCI/BWloW//vOd74v
	J985OTjKX8J8cZlcw6rkTBaJeXA6b+57Lai8870MQdcjb2qh+gaHuvKgFKMqLSsoZZm4gFBT
	vbFUZe0YoJqv9CPUw9rzGGW9/oxDrU8LKZN5P/XleBdGFRuSqMaBDYRam2lBqIruBUBVj9Rx
	qeX2CuR1Pj06lkjPNjeg9HxFGaB1HS/Q3foHbvTosJY2Ggoxuq0+l75pGePQPRN5GD1R+yFt
	nTFz6PtLBShdWtzJocvabnBpm9EvhXg/M0rKMmJW5c/K0xVimVwSTSYeSYtLixAJhEHCSOoA
	6S9nstloMj4pJShBlmXPSvqfYbK0diqFUavJkJgolUKrYf2lCrUmmmSV4ixluDJYzWSrtXJJ
	sJzVHBQKBKERduGpTOm9tUU3ZQd+1tzyI8gDdW5FwB2HRDhsGOzhFgEPnE/0AJjf0w+cxTMA
	68yDbs5iCcDvikeQbYt1sszVMAF4a7KF4yxsAG7038YcKg4RAKe6yu0NHMeI/fCnTdxBexMk
	nJkb3hqBEjoufLo4t6XfSZyG1k0T16HnEXGwseqQg+YRXnDw4mOOA7sTYdBQv7g1CxLn3eG5
	yT9dIeJhd9VVrhPvhHMD7S7eF9r+MmFOnAObv2rCnOZzAOrH9cDZiIW6u6WoA6OEFFavXXaZ
	d8Ov77YiTt4TFq89dsXnwa7abbwHXr12yTXAB/66nO/CNLTMr6LOrbQCeN1QgZQBP/1zifTP
	zXPig7DQ8jlXb18ASuyCjRu4E+6D174PuQS4BuDDKtXZElYdoQyVszn/nzldkW0EWw8Q+FYX
	mJ6yBPcBBAd9AOIo6c1blRzP4PPEzEcfsypFmkqbxar7QIT9QOWo70vpCvsHyTVpwvBIQbhI
	JAqPDBMJyZd587pvxHxCwmjYTJZVsqptH4K7++YhHSer3wno4e0eHD42qes13DF278DiUh8N
	FwR85hXvY17dADtqPrgvMY28KZAFno2KzIz9Q2+NIzMSjtj89giDE6QPiW9tu4zJMo+qA9MT
	fUdVw/+shsyOHq3R1jwdvNcEBfxjo0MliYrb3XkrQqlR2LhUkJBSWKFV0L/s1Yz//vfpIpnn
	cVHUiaaYn9enSwrMVP18v2IZv3PSo58lLlcmLOTWVVvnRg79+8OpTzdtQ8FjI7Nvn3j1iwut
	mDwsp3DtE8+2lVtuTKpgInkqKWAgn3fxTC5oafCDEbNPyqPEpld6X0z+bTT2yfpsUGP74VDZ
	u/mrlqCSN1K1GUO6wJgNr0WSo5YywkBUpWb+Axp6BGqJBAAA
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrCIsWRmVeSWpSXmKPExsWy7bCSnK74+sg0g/P7lS3ezNjIYrH6bj+b
	xbQPP5ktPtycxGTxYL+9xbR5VxgtVq4+ymRxf147m8XHDZ9YLP48NLTYe0vbovv6DjaL3lU+
	FsuP/2Oy+P10LZPF5J1vGC1mnF/MavF9y2QmByGPy1e8PZ6vXMbs8XryBEaP1q3cHjtn3WX3
	uHy21GPTqk42j81L6j0Of7jC4rH7ZgObx815hR4fn95i8bj6rZnZo793G4vHhM0bWT0+b5IL
	EIjisklJzcksSy3St0vgyni1eC1jwXPWiua1U5kbGN+zdDFyckgImEh8vDeBvYuRi0NIYDej
	xKknx9khEpISy/4eYYawhSVW/nsOVfSRUaJ54XuwBIuAqsSDHROBJnFwsAloS5z+zwESFhFQ
	knj66iwjSD2zQC+rxOIfP9hAEsICSRIf/+9lBannFXCWWD7dEcQUEoiSmHKID6SCV0BQ4uTM
	J2C3MQuYSczb/JAZpIRZQFpi+T+w6ZwCxhKrlnxhmcAoMAtJxywkHbMQOhYwMq9ilEwtKM5N
	z002LDDMSy3XK07MLS7NS9dLzs/dxAiOWC2NHYz35v/TO8TIxMF4iFGCg1lJhPdXekSaEG9K
	YmVValF+fFFpTmrxIUZpDhYlcV7DGbNThATSE0tSs1NTC1KLYLJMHJxSDUyKXO06JsGVEbsN
	/lRpNHncfb3f3UvXvv7y/cfXXtyZrC3Q9fFlgfhNZy1ds+drd0Zrs5s/62esPHHEVoJ1Edu7
	DxGTz4mfKVtYVp1fNneh4IZtJW9WV76yneTVHmvScyg41//yK91jFUwRYo2mYamCkuxKZuoW
	Ly3mMSquWMZR8CnJvZz3AEfBbNZix1bD/XEpwYLT+p4Xm788eOvjNO4Fisb3+Tj2vSvOYCgr
	Lz9iLqTsdJE/hq+q6m3zx7LYVakVfPU56iVxyql7Jnn/3yKkf0a65FdYWJtTw8m7Jxdav9bJ
	Pfpo9+b7x9N/z3Sr4Ez1zOebG9xpzS6n4tXvfshMLlHSif/fmSeqZRxKLMUZiYZazEXFiQDh
	esZKRwMAAA==
X-CMS-MailID: 20240531110559epcas5p1f41d7d904ed46260837c38a57fa32d86
X-Msg-Generator: CA
Content-Type: multipart/mixed;
	boundary="----Fualeei1.f5fhWGYL679EBd0hH5-OLgtfrOtH6wInDZGwDEe=_482db_"
X-Sendblock-Type: REQ_APPROVE
CMS-TYPE: 105P
DLP-Filter: Pass
X-CFilter-Loop: Reflected
X-CMS-RootMailID: 20240531110559epcas5p1f41d7d904ed46260837c38a57fa32d86
References: <20240531074837.1648501-1-hch@lst.de>
	<20240531074837.1648501-15-hch@lst.de>
	<CGME20240531110559epcas5p1f41d7d904ed46260837c38a57fa32d86@epcas5p1.samsung.com>

------Fualeei1.f5fhWGYL679EBd0hH5-OLgtfrOtH6wInDZGwDEe=_482db_
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Disposition: inline

On 31/05/24 09:48AM, Christoph Hellwig wrote:
>A few drivers optimistically try to support discard, write zeroes and
>secure erase and disable the features from the I/O completion handler
>if the hardware can't support them.  This disable can't be done using
>the atomic queue limits API because the I/O completion handlers can't
>take sleeping locks or freeze the queue.  Keep the existing clearing
>of the relevant field to zero, but replace the old blk_queue_max_*
>APIs with new disable APIs that force the value to 0.
>
>Signed-off-by: Christoph Hellwig <hch@lst.de>
>Reviewed-by: Bart Van Assche <bvanassche@acm.org>
>Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
>---
Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com>

------Fualeei1.f5fhWGYL679EBd0hH5-OLgtfrOtH6wInDZGwDEe=_482db_
Content-Type: text/plain; charset="utf-8"


------Fualeei1.f5fhWGYL679EBd0hH5-OLgtfrOtH6wInDZGwDEe=_482db_--


From xen-devel-bounces@lists.xenproject.org Fri May 31 12:08:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 12:08:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733684.1139996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD13e-0007ks-9D; Fri, 31 May 2024 12:08:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733684.1139996; Fri, 31 May 2024 12:08:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD13e-0007kl-6g; Fri, 31 May 2024 12:08:30 +0000
Received: by outflank-mailman (input) for mailman id 733684;
 Fri, 31 May 2024 12:08:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAP5=NC=oracle.com=martin.petersen@srs-se1.protection.inumbo.net>)
 id 1sD13d-0007ke-Eh
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 12:08:29 +0000
Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com
 [205.220.165.32]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7b89db31-1f46-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 14:08:26 +0200 (CEST)
Received: from pps.filterd (m0246629.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
 44V9VaPq025198; Fri, 31 May 2024 12:08:09 GMT
Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com
 (phxpaimrmta03.appoci.oracle.com [138.1.37.129])
 by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3yb8p7tx8x-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 12:08:08 +0000
Received: from pps.filterd
 (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
 by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19)
 with ESMTP id 44VAB7eR006340; Fri, 31 May 2024 12:08:07 GMT
Received: from nam11-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam11lp2169.outbound.protection.outlook.com [104.47.57.169])
 by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
 3yd7c89dp9-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 31 May 2024 12:08:07 +0000
Received: from PH0PR10MB4759.namprd10.prod.outlook.com (2603:10b6:510:3d::12)
 by SA1PR10MB5866.namprd10.prod.outlook.com (2603:10b6:806:22b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 31 May
 2024 12:08:01 +0000
Received: from PH0PR10MB4759.namprd10.prod.outlook.com
 ([fe80::5c74:6a24:843e:e8f7]) by PH0PR10MB4759.namprd10.prod.outlook.com
 ([fe80::5c74:6a24:843e:e8f7%4]) with mapi id 15.20.7633.021; Fri, 31 May 2024
 12:08:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b89db31-1f46-11ef-b4bb-af5377834399
DKIM-Signature: =?UTF-8?Q?v=3D1;_a=3Drsa-sha256;_c=3Drelaxed/relaxed;_d=3Doracle.com;_h?=
 =?UTF-8?Q?=3Dcc:content-type:date:from:in-reply-to:message-id:mime-versio?=
 =?UTF-8?Q?n:references:subject:to;_s=3Dcorp-2023-11-20;_bh=3DXUmnzuXbv30y?=
 =?UTF-8?Q?x5U2wKdOOuX0y4eJ9Z+Fx4zUNeqmrA8=3D;_b=3DbLik8BL1kIRcvUaO5/XX8V9?=
 =?UTF-8?Q?auvdrZNNiQ1QpwjXBt7TJhhA/sQNCAG33ljPmHc7Me5G+_rhBJMNiOG8zxys8fP?=
 =?UTF-8?Q?ShUoCMKM65Sfz7+dV1hZiRQ06txwmaDpBK9O4PYDwOK2bMkbN6E_1PJZWk1mEIB?=
 =?UTF-8?Q?mi5nwgDqHhmnCavEi4wTlRcKoDfoJ7JcQj6yKtrWD202t2BcF4xTqVUsm_Ooyi0?=
 =?UTF-8?Q?CeOo0g/zbwersyHrysPfVpapLj4B2VqAXRl2B9Mx1tLOTHbmkPMZBPGUng/4yEl?=
 =?UTF-8?Q?_62e2e3sewab+EUKNDiPligSKKLhVNsKBKF6OeKY3fL8wHHERAyG+mEv+JRL+Tw?=
 =?UTF-8?Q?0nM1tp_iA=3D=3D_?=
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=D8Qs0KgGt5BbEcDA7zEdaSCCWaMiB1jiFB4DBE+5MABoYB/YRaG8yzzpzL6KODm4m3BtsQj8+2ddAC/We+EsqMr1oiLxua1K2uXwcZq712xPNEe14RAOwY6SCIsvBki8QdYal7zoDWazK1XMbgI1QbvEbY8XpSfV7M+Ydt1VJoim1bmYkTCS6YsLfy++can89EU6/W+Bx/8RoeKMsW1rTwKjr5CagdEFxn2i3uPSWsEqGFCmIYz15bYCUMpKn1hvgASw52fQMCbju8k800XmwYDJ7EGt4KYsQ//zgnYtVdaJ6btcrypqTEGtR2vc8wSCm5T1HqNQfEB4VuApY2aBtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=XUmnzuXbv30yx5U2wKdOOuX0y4eJ9Z+Fx4zUNeqmrA8=;
 b=a7bmNc1VaUDsa2Wan323773RDqh75STrNUy/Ai3fVokW0bVZLgDuAh1+bpfTT3gg0WlMkxDD08QlXAjAaoa9fvcHlnaVjZ827hpIND2KsuKgTKhBQZPEHhLtA4wZ5Mhhb9z9AgPkDlGiu3P+6soYlU5V++J1Qhj0Rywm/39dDJg8dHAmO6liKpjhWVb7cEY5Jc11dPAHrA6RcMsZ3y4PIuDuL/fRwVhFU8J33mn1AOPA9XgrY4qdc8MWF/7Li33BE9UAn9d3emFweeGgtwwBWpfNOWT3fjczGEVAQ9oZBzmzmuFVoZPlvSEmwVQzXTpxAtEXjPn+wirgcMABKS1y9Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XUmnzuXbv30yx5U2wKdOOuX0y4eJ9Z+Fx4zUNeqmrA8=;
 b=s4mzQLWgmqgVGzkyplSKscdbfLWAsDBRdSqWbV31dFtbw1XqMsHo+CExP5E0Aqjy4430twSFlRjHMnu/JXfrQ8fubLGCYpoFHNWTzL0QJVhWVDopUGqKvPX1VnGGZkvppNH5eZWoP5UyaCCHQsY467OYoKFv3NYIC9eFyugjwpo=
To: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>,
        "Martin K. Petersen"
 <martin.petersen@oracle.com>,
        Richard Weinberger <richard@nod.at>,
        Anton
 Ivanov <anton.ivanov@cambridgegreys.com>,
        Johannes Berg
 <johannes@sipsolutions.net>,
        Josef Bacik <josef@toxicpanda.com>, Ilya
 Dryomov <idryomov@gmail.com>,
        Dongsheng Yang
 <dongsheng.yang@easystack.cn>,
        Roger Pau =?utf-8?Q?Monn=C3=A9?=
 <roger.pau@citrix.com>,
        linux-um@lists.infradead.org, linux-block@vger.kernel.org,
        nbd@other.debian.org, ceph-devel@vger.kernel.org,
        xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Subject: Re: convert the SCSI ULDs to the atomic queue limits API v2
From: "Martin K. Petersen" <martin.petersen@oracle.com>
In-Reply-To: <20240531074837.1648501-1-hch@lst.de> (Christoph Hellwig's
	message of "Fri, 31 May 2024 09:47:55 +0200")
Organization: Oracle Corporation
Message-ID: <yq17cfadwd0.fsf@ca-mkp.ca.oracle.com>
References: <20240531074837.1648501-1-hch@lst.de>
Date: Fri, 31 May 2024 08:07:54 -0400
Content-Type: text/plain
X-ClientProxiedBy: LO4P123CA0232.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a6::21) To PH0PR10MB4759.namprd10.prod.outlook.com
 (2603:10b6:510:3d::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH0PR10MB4759:EE_|SA1PR10MB5866:EE_
X-MS-Office365-Filtering-Correlation-Id: 15231c24-f371-4dc1-49b9-08dc816a5123
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|7416005|1800799015|366007|376005;
X-Microsoft-Antispam-Message-Info: 
	=?us-ascii?Q?KGyupYCU+JuKBi4UuVZYwz9OLjYKekpeYkYcQo/cjZHH7Kuz4PUQyhEzPk7N?=
 =?us-ascii?Q?l24tI/1dCvAMiIc1xsIcTWpz93ph9ygSHtZYuQsALfFTDd5l3esC4ekZUCqV?=
 =?us-ascii?Q?WBpMIwFIBod021ZlfCCw4O8SUgKgDqcaiIzd8LLwkCvHKxPyLOfTtcyHo3UV?=
 =?us-ascii?Q?rb2U3sUKZTjgEm3iNLxCjvU2qzhi6tum3B9P6aY5C0ZdRRYxgXzIk68Ix1Lr?=
 =?us-ascii?Q?fhxhdHYpUz41KhWATzkCaLHEnNcre7833Yt48vqDRDvzDoONmpWIzIX0iLSL?=
 =?us-ascii?Q?qufkgdADy8d3LrB6sCMRAsFbS2cPX/W2jAgJR6Hgn5j/hEoAuFf75TnOYe6N?=
 =?us-ascii?Q?nYESmNVmJARlTgoCd8vxabgcPH1MlTx9+oFmS8whNRWmOo4xgQuY/BmJFv2G?=
 =?us-ascii?Q?iEJttOCUvl5tquX14Ys+gc85j4CFgFCYUAe0p1JumX+w9ReyH+oeBzL3/kYc?=
 =?us-ascii?Q?Jysnnpv/MTJMPSFxLWt0a1dvDLOezXUUi26b0vAZZmAkBMp3eM6rj8l0JaZC?=
 =?us-ascii?Q?QXk1Ok9PdtA1vrh/3X8Lobh6JoUv8AMoFO465u59yyL4NVosgI3i+RVPz7Ax?=
 =?us-ascii?Q?6VNqQ2A5ID4hdJCpLpqeJ+eJe+vddUHBCw0a3DDrsO5WByXHWO630N7zIEL7?=
 =?us-ascii?Q?P8VZtrKD15nZKogvMS//gUmvTsh96nsAKP38sq6hAEnFFSHEk8ua+qHL0gJg?=
 =?us-ascii?Q?OxPvEnDjOX8SmHyExo3I0ndprLUUhONvjDYxRj206TS6OraXzwQuh1OgDQCq?=
 =?us-ascii?Q?nFdMktnlmjIGfEUhkOgblel8E11Ew0pyPIN8N9QjkprHTT/uB9ttlfO6WLQT?=
 =?us-ascii?Q?58IzJvrcxxhGS7/krA1VRG2/R56r4pHRws0xtCOKL4HjFYCXI3JFZyeLnUpk?=
 =?us-ascii?Q?dib20rXzdVbygFJhjy4v+pgiW3JjrfnGGJd0s2w7Y8usMOYN0FguIC9pvRyU?=
 =?us-ascii?Q?oe+quvPlk+0QXwLXoO2C315ZDA37wOkMHH2NIwaOfg6K8/tD9wbHH/lAMdXP?=
 =?us-ascii?Q?8u1fAMcDiBb5zMIMnvat7ZRq2bmw+u3mJx5YvrkPLaNlOBHnGbq0sBFXrsTq?=
 =?us-ascii?Q?dRbMVSL1dlvcx2exNyagFF1utPjgkPMFTvqJcFHHuyH5lZC9z5v4ppl0CFpE?=
 =?us-ascii?Q?5EqEvICwS3k9gPSSzFmziywHgHpDe5CyK71XHSHs6jdxD5tOL5Ggga4+Z2Bu?=
 =?us-ascii?Q?RtSdKe8ahoyvfngWM2Jw1GkBQmpDNcooUk4b81i9CtmNV2Fuuy4h316dXB/1?=
 =?us-ascii?Q?BjhMKetb2ZyZi3tI54UnhFux5O6DV4l6HuWMB3HiJQ=3D=3D?=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR10MB4759.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(7416005)(1800799015)(366007)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?iVzbOLNXq4lTV/BnQAAh0CCD/RpGP+lD4E47LNbEGKNd8R44TGNPsgkSEwWG?=
 =?us-ascii?Q?VwjViG8vklwn1fX5t9N99TuNtPe/ETCHLCGyNHjg7ZOwxgvc1u/PyRv0QSFZ?=
 =?us-ascii?Q?pvFDLTouoK/KPHvp1LgIxPfgb00mpG7qBSYGQ8OphG1fFmhanR3nkedSwCGN?=
 =?us-ascii?Q?Y4Ozu9Z2AyV58qlLM7n+5wp2oqJounr+nKgeZTECy+svTK+RmBYPUXU13Go+?=
 =?us-ascii?Q?wNZshNXhZYS/PRvQlA2nzYlIMpc4+5tamZoFKN1nO+lpKJfoFgRoHVXNxwoF?=
 =?us-ascii?Q?Ds23YiRAcPFRGSC1xX2YVBBDAJcV+tlULpQlKIcWdoNOoTZR7ZunwK3xK6i7?=
 =?us-ascii?Q?NNn9tqEXuKVPUPy3Zl95zKkaKznjX08KXCs7H8+0CXsUIaRcbyN3CKM59jDc?=
 =?us-ascii?Q?3b3QXZsKYmlxyjT59VlzttlhcUX2X/VFWidhzASGaVzWYu/Q1ocpgtNdDJak?=
 =?us-ascii?Q?JFxQXkMXK6a+GMphJw0ih9DFIC07pxJ5sMD8GyxNwDHuETiLVrCQFNlQyrzL?=
 =?us-ascii?Q?IKAq2qG/B1QeBmqAddjALuppMVZVXHDSHKajgDbg8KtU6VT66BM35zzPyvOJ?=
 =?us-ascii?Q?1JikWTCuY8HZXeto229WBA5bLxXo5GW9CZZwMJbO7UdNfY38MKbFmfoKvLLe?=
 =?us-ascii?Q?hGLUYO9q9xNbHPP9uiQBw85FWwSvd6e0fGDK2qxLPvY7HXPIiWEBOvwvCvHl?=
 =?us-ascii?Q?U7ER08nN9yvvT2FNTjT7RNpKe9G4mxIGXisI2YMdam6hQWLMYBrz1/KNpqhx?=
 =?us-ascii?Q?0kD15tbTjPOrXmb8KCai4bmr1XRtBMGOSd77hVCoVnnYM0p/dLEs8ELtmm9J?=
 =?us-ascii?Q?/qnI8R2W4dNQvSwEmWkIc5KwYLdGRzWmWFnBR/sK85F1k19UWoUtau4FuMWQ?=
 =?us-ascii?Q?66oLUfokPSGrq8HOpaEB2NbC3ZHuCd4NP00WEJvF2Q2Me694G7W5+P1uzA27?=
 =?us-ascii?Q?nZ/A9/r+FzS01UhWmjd8mbSMFD9wDr18MS7iP1yNyYl23TCr6B2GBJpCRSt9?=
 =?us-ascii?Q?gQ8mcINFj6gV6X+ixNk3IeOXWcuoXd2D/a0P41dXPDnfbWC8qRASLqMvogIL?=
 =?us-ascii?Q?jS+IJjZt76I6MiLcIR4hbsT+Ykr9ugsOK2wPb7Iu8P9PKDpea/yTChHUQ+qf?=
 =?us-ascii?Q?nMg7daxYnSQNsyOf9ushMhKeMzLh4go+R/VC+xWGB6FzxW2r77O8NlLoIhKm?=
 =?us-ascii?Q?p+tjypG4cW0ZzZWH4+qBV2jVphahthEdVfxflshGS+t5+riIUnQB3F2aqLtF?=
 =?us-ascii?Q?fhxv65iEHO1zNK0yIPWtSohdxmwIjh2D30gkn5nre6jPGvQQYARDo3GqaoWB?=
 =?us-ascii?Q?YK464ZqYg+sfskO8Hdx/ydAvrxzj7gx3r8k6giVeelf+bmoSpBn9MMRfBOqQ?=
 =?us-ascii?Q?ASZWa9dS8XpJDNy1hu70pMbANDc+ni87Mm9N5jHiiVpITRMMNcRSoN+F7lSx?=
 =?us-ascii?Q?z9YFvj2p4hBb/6ZleS5H0nOA9NSqhvpP9Y1VZwE0Kdroimuc/BkPf41VCDPf?=
 =?us-ascii?Q?D5cRgu+7OYqwavsYqjK+dYqK9YI/N4+oAnx9Sa93CEc5DqZnXg1VJvbXitdn?=
 =?us-ascii?Q?wmDAmLWhLXbTljcZJKMdrtbRXYlbMEiXjgOf0AC2MlDlufOn4CRGMEx15L7P?=
 =?us-ascii?Q?QA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	lK8IfBr1SHHVS9FF5MD6gx/516EVDlcMO/iBC6yKlIZiINYPf85/ZRz1VS2hbm1Ou63LclIjZIDbzjnj8dxekW2nTtXLZ9RI5PUOF8kR9T/O8nhE3Kduqgd120Bs5xjdtenS18qKb1r8gq8RIbeufgXOvseUvcQUx1utpCpfoLvhzf0LRd8SxUN5xfLdsppahpvF574hV5JmVFZnN3s0nZ+eST9oH5WJDZr0JyoolfrGdcdXv+riG/CTg0pfdbPLXL0loXGndYqjERFUS5Z2foaNya6R03SopQDw8oy2IiXuE6QSWVy+dIkEDxDNji6EMTzuvKn2xfXAH3XpDwpcn5TjJRYt7RtVZPDCxcSUp35S4PUfnsj3vzAWG88roVeFKu1u0xoaqz4Xxwkvwl8jhDkUlBtMn6pHqGfGpj0kTbaRtQco30hIf2aWjLJ2dyCoZkAjs6GleH2QvAPpatNGJbT25sHZ144U3syrukMvw7pBE2G02NsHdAVEp1d5KIYPq12GHH0uFbwV0QcG+WGTlOsXhTmYlPoc8W2tv8iitK1de9PgZju7IjPy6AqFhHFvktbVLmwH/drpWv9e86p7mJJctskVKYYS4Vb847ESKGw=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 15231c24-f371-4dc1-49b9-08dc816a5123
X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB4759.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 12:08:00.9782
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zKFa9LSK9YuRCEEy2m6s2lnhPchFGR//EIwv63PomuICtEESvhe6cwDAuTJL7NFj0VHKV1vov6dfKgk6pxM7aSbPKewh4HNplkS6uYmumoE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB5866
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_08,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 bulkscore=0
 suspectscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=923
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405310089
X-Proofpoint-ORIG-GUID: kqo-xGx4FEW29fmZBsPEc76KzvlEFz3Q
X-Proofpoint-GUID: kqo-xGx4FEW29fmZBsPEc76KzvlEFz3Q


Christoph,

> The patches are against Jens' block-6.10 tree.  Due to the amount of
> block layer changes in here, and other that will depend on it, it
> would be good if this could eventually be merged through the block
> tree, or at least a shared branch between the SCSI and block trees.

If you have other block layer changes depending on this series we'll
probably need a shared branch. I'll need to make several changes to sd.c
to fix reported issues, including a couple in the zeroing/discard
department.

No objections to your series so far. Just trying to reconcile your
changes with mine...

-- 
Martin K. Petersen	Oracle Linux Engineering


From xen-devel-bounces@lists.xenproject.org Fri May 31 12:24:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 12:24:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733688.1140007 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1Ii-0003Aa-IA; Fri, 31 May 2024 12:24:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733688.1140007; Fri, 31 May 2024 12:24:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1Ii-0003AT-FB; Fri, 31 May 2024 12:24:04 +0000
Received: by outflank-mailman (input) for mailman id 733688;
 Fri, 31 May 2024 12:24:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ULw3=NC=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1sD1Ih-0003AN-Ea
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 12:24:03 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a893a0cc-1f48-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 14:24:00 +0200 (CEST)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 5085768BFE; Fri, 31 May 2024 14:23:56 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a893a0cc-1f48-11ef-b4bb-af5377834399
Date: Fri, 31 May 2024 14:23:56 +0200
From: Christoph Hellwig <hch@lst.de>
To: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>,
	Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Josef Bacik <josef@toxicpanda.com>,
	Ilya Dryomov <idryomov@gmail.com>,
	Dongsheng Yang <dongsheng.yang@easystack.cn>,
	Roger Pau Monn?? <roger.pau@citrix.com>,
	linux-um@lists.infradead.org, linux-block@vger.kernel.org,
	nbd@other.debian.org, ceph-devel@vger.kernel.org,
	xen-devel@lists.xenproject.org, linux-scsi@vger.kernel.org
Subject: Re: convert the SCSI ULDs to the atomic queue limits API v2
Message-ID: <20240531122356.GA24343@lst.de>
References: <20240531074837.1648501-1-hch@lst.de> <yq17cfadwd0.fsf@ca-mkp.ca.oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <yq17cfadwd0.fsf@ca-mkp.ca.oracle.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Fri, May 31, 2024 at 08:07:54AM -0400, Martin K. Petersen wrote:
> If you have other block layer changes depending on this series we'll
> probably need a shared branch. I'll need to make several changes to sd.c
> to fix reported issues, including a couple in the zeroing/discard
> department.

Yes, I also want to move the various feature flags that currently
abususe queue->flags into the queue limits.  In the worst case I could
delay that for another merge window, but a shared branch would be
a lot nicer.



From xen-devel-bounces@lists.xenproject.org Fri May 31 12:28:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 12:28:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733692.1140018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1ND-0003kP-3h; Fri, 31 May 2024 12:28:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733692.1140018; Fri, 31 May 2024 12:28:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1NC-0003kI-Vi; Fri, 31 May 2024 12:28:42 +0000
Received: by outflank-mailman (input) for mailman id 733692;
 Fri, 31 May 2024 12:28:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sD1NC-0003kC-3t
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 12:28:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1NB-0007Sw-Ih; Fri, 31 May 2024 12:28:41 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1NB-0002p5-AT; Fri, 31 May 2024 12:28:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=yqvgxQqn4c1Q3YCCWVk16WRDZPJOqFFjQTBiRPKpCqU=; b=sVfTs3L//p1TZx9uXLiBGCiVWv
	TAbloCNn3nRO9JWm+bmABHRhaQfaYi0OI33mCKiG/IXAYRH2PhVu/GldacFlD6OcXkLYCnI5rytzm
	pLjnhZz9sQKcyXxo+MmZnkPcou2SOnktBRaGq1xVB8Cj1ZOP9NxFtZsUvdSn247D6/qs=;
Message-ID: <cc7bb358-d9a1-4284-b51a-07499e225252@xen.org>
Date: Fri, 31 May 2024 13:28:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] arch/irq: Make irq_ack_none() mandatory
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
 <20240530184027.44609-2-andrew.cooper3@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240530184027.44609-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 30/05/2024 19:40, Andrew Cooper wrote:
> Any non-stub implementation of these is going to have to do something here.
> 
> irq_end_none() is more complicated and has arch-specific interactions with
> irq_ack_none(), so make it optional.
> 
> For PPC, introduce a stub irq_ack_none().
> 
> For ARM and x86, export the existing {ack,end}_none() helpers, gaining an irq_
> prefix for consisntency with everything else in no_irq_type.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 31 12:29:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 12:29:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733695.1140027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1OM-0004Mq-DZ; Fri, 31 May 2024 12:29:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733695.1140027; Fri, 31 May 2024 12:29:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1OM-0004Mj-9G; Fri, 31 May 2024 12:29:54 +0000
Received: by outflank-mailman (input) for mailman id 733695;
 Fri, 31 May 2024 12:29:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sD1OL-0004Mb-2E
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 12:29:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1OK-0007U9-Ev; Fri, 31 May 2024 12:29:52 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1OK-0002sI-9E; Fri, 31 May 2024 12:29:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VfsZDEwUbCdUOLX6uw2OtqmZ9wf9n6BQe/Wg76+pZFY=; b=bomvM5a9mmcwRohJPa+C4dfg4t
	fODXt5tUTRyR9IKcnXTQz19dIy15eN7X/tD8IxR4S3gMdgUBkeOmg6clzlNT27s3TJZh/QiF8yScG
	ZFrK3CmVH0VKOYZclrr/CdcGQbdVq2J1rAixEZt093cPL2YCfgDOrd8Mbw/LZPr5UCMs=;
Message-ID: <e9409a25-e650-470e-8b9e-b34300ad8d26@xen.org>
Date: Fri, 31 May 2024 13:29:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] arch/irq: Centralise no_irq_type
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Jan Beulich <JBeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <20240530184027.44609-1-andrew.cooper3@citrix.com>
 <20240530184027.44609-3-andrew.cooper3@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240530184027.44609-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Andrew,

On 30/05/2024 19:40, Andrew Cooper wrote:
> Having no_irq_type defined per arch, but using common callbacks is a mess, and
> particualrly hard to bootstrap a new architecture with.
> 
> Now that the ack()/end() hooks have been exported suitably, move the
> definition of no_irq_type into common/irq.c, and into .rodata for good
> measure.
> 
> No functional change, but a whole lot less tangled.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 31 12:31:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 12:31:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733699.1140036 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1Po-0006Dw-Lt; Fri, 31 May 2024 12:31:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733699.1140036; Fri, 31 May 2024 12:31:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD1Po-0006Dp-Im; Fri, 31 May 2024 12:31:24 +0000
Received: by outflank-mailman (input) for mailman id 733699;
 Fri, 31 May 2024 12:31:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1sD1Pm-0006Dh-T0
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 12:31:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1Pk-0007X7-S2; Fri, 31 May 2024 12:31:20 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.0.211])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1sD1Pk-0002y9-NI; Fri, 31 May 2024 12:31:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=IDlSFw8YoXovb/WzuVZv5/QASY0oVrdI3Sw9GrIo/dA=; b=bn1MmBH1qNyArXzOeH3IidxKYP
	l8X+SzaG33Uume66u8c36Rhqoogvk8VDG9ugKU9G55R0Q08NqK80y8YwfpGKJSeXgy0hofPIitLFT
	JPvAFele/j0gK1SFhS7AANouoBOp24/ili25CDev47j6R7nz1Jj4wH7Cm8GYl3yej880=;
Message-ID: <9af0eac8-08ac-419f-a31c-a95940913f75@xen.org>
Date: Fri, 31 May 2024 13:31:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arm: dom0less: add TEE support
Content-Language: en-GB
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240529204305.1402036-1-volodymyr_babchuk@epam.com>
 <be133b0b-5b22-4b82-b5f6-3c257b45553f@xen.org> <87bk4oxpxk.fsf@epam.com>
 <CBB42DF2-94AE-4566-B83D-3AB3CB2E01DE@arm.com>
 <047ca670-cd9e-4803-8083-7770b7230042@xen.org>
 <E337CF92-42DE-424C-95D5-97FF096885B1@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <E337CF92-42DE-424C-95D5-97FF096885B1@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Bertrand,

On 30/05/2024 14:22, Bertrand Marquis wrote:
>> On 30 May 2024, at 12:35, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Bertrand,
>>
>> On 30/05/2024 10:40, Bertrand Marquis wrote:
>>>> But we are making assumption that all TEE implementation will have its
>>>> node inside "/firmware/". I am not 100% sure that this is correct. For
>>>> example I saw that Google Trusty uses "/trusty" node (directly inside
>>>> the DTS root). On other hand, it is not defined in dts bindings, as far
>>>> as I know.
>>> Regarding the firmware part you can easily handle that by looking for /firmware
>>> and create it if it does not exist before creating your sub-node and this should
>>> be node in the optee node creation function not in tee.c.
>>
>> This would work if the node /firmware doesn't exist. But how would you handle the case where it is already present?
>>
>> I looked at the libfdt API and AFAICT the DTB creation needs to be linear. IOW, you can't add a subnode to an already created node.
>>
>> There is an helper to create a placeholder, but AFAIK this is only for a property. You also need to know the size in advance.
> 
> I thought it was possible but i definitely can be wrong.
> 
> As right now we have only one need for the node, we could delay a possible solution and just create it in the optee driver.

I am fine with that.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri May 31 14:12:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 14:12:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733730.1140046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD2zY-0004Yz-Kc; Fri, 31 May 2024 14:12:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733730.1140046; Fri, 31 May 2024 14:12:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD2zY-0004Ys-Hs; Fri, 31 May 2024 14:12:24 +0000
Received: by outflank-mailman (input) for mailman id 733730;
 Fri, 31 May 2024 14:12:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tx2a=NC=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sD2zW-0004Yh-Sq
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 14:12:23 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on2062f.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cad66a19-1f57-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 16:12:20 +0200 (CEST)
Received: from DU7PR01CA0030.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:50e::19) by GV2PR08MB8485.eurprd08.prod.outlook.com
 (2603:10a6:150:c3::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Fri, 31 May
 2024 14:12:16 +0000
Received: from DU2PEPF00028CFF.eurprd03.prod.outlook.com
 (2603:10a6:10:50e:cafe::43) by DU7PR01CA0030.outlook.office365.com
 (2603:10a6:10:50e::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24 via Frontend
 Transport; Fri, 31 May 2024 14:12:15 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028CFF.mail.protection.outlook.com (10.167.242.183) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15
 via Frontend Transport; Fri, 31 May 2024 14:12:15 +0000
Received: ("Tessian outbound 6a5ce4665eb8:v327");
 Fri, 31 May 2024 14:12:14 +0000
Received: from bdf609efa8fe.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3410EFDA-39EE-431F-84F1-55ABE1C838CB.1; 
 Fri, 31 May 2024 14:12:09 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bdf609efa8fe.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 31 May 2024 14:12:09 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by DB4PR08MB9215.eurprd08.prod.outlook.com (2603:10a6:10:3f9::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 14:12:06 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Fri, 31 May 2024
 14:12:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cad66a19-1f57-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=V8N7X9cXprRFcQgodZTZWm/0hNoL6I9VkJ4J4o/TCOD47QIZNhudypPbmuSPzLhcpXAFwsdDI0AQzsMEWSXR54vZp+QnPfj+pAJpYiP98F0rrAVbbOjrjCjKJQuQMx62ScV6VKB26drG+/+Bo9kK76MhE8uy7Mj0SY1s+uF+P6aWybW94SUVEpTbUSaXQvxyP7PrqbwP/FzrQQc2DQWA22EYdJUnEqP76FDfcng0G8xoZ7VvH6DhooygvUkayuIeznmMLmSk94KsxhGgOU+q82qcmfoWC2VrmTtx6Tw3/fHCwMNEZIViZwA21kKzqD8DOoXXOv66sA3mADtsbaYQLg==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=jInNIhdvRlovQ+I9As8WX1J+NR9g5ZKOmf2Zxb/tb9c=;
 b=Yy/Li4RyH2H2RrN8TRbwa6eWfiG8uiiiLHEaQGzN7j03N/u6XFq7MdlXHyhSDYHsEEI0oaXJgyRZczGoXiUaqK/0cLnz7Q1sFDOJoDuiNiD8TVVP6jt8liqlv++soVeuRmGsFaqHsdbczGp7eqhq2/8rpE7ifh15j6RBfDDB+0zz4373+Aovb4xDLfE7jHCdswOd1w05P+4MzJfwlEBYBPIXkZsvEYT4QBXNxR+EEQoi7ralB/PYSUc4orTLWuIWGRM9FGUXsfZ1nbEry2MJNC/oDe0yk+QgITKpAWPXKMADuJ+lmnXi1c7rOOjj+q5hNIH2yhOvLfAkoG1qJzfGZw==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jInNIhdvRlovQ+I9As8WX1J+NR9g5ZKOmf2Zxb/tb9c=;
 b=dgNA22nBo0TeTJGwXmmKVxW0atW2ZWQ93cYHtOJJlkMOB4OrkhHgRQGlP7iUNmBaNOwJCfqLNz8HS4T+yS2+/yeMIYOOiqgbeuRBBclJt/7IJ634nRhYFUZaE0+sDKiFq87PubWYSICWOyxjWP1yZ/ytMZ2a9u73Um7P2qoRcC0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: c4f40f3c8131a210
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NbhycE7XLPSn4NkZeG+yQQDP/U9FGBViiIz+QtT5uSwepzVoC9dPvIEUsMys8mSbEQ3hq5r9K/MoGngIDVGVwn00MMIuIntv5i3DESfHpnwalkI/CBO+88u85qvhr/OFNPw0HhFB5YmgbWC2RfNZFfM1f8oeW0wVjzxXAKj2xo+NOetOUUVmxjhRJ46HX3/sLGxRse7t9hQZEW7CBGBoTz2O1gG0a9oz2nkPuV7sz73aN6PDpLv2yNAuontGR2mntfaQO64gUmsaGYGfcswU6kOa/CZSRKAtpD3u0dSee/Id7mLWdcKs7vKW9OXvUkBQeUnxGkXFLmf0QqKdoTwK9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=jInNIhdvRlovQ+I9As8WX1J+NR9g5ZKOmf2Zxb/tb9c=;
 b=eGr9aataHBGBnt/LNaQ1IzeDpxrItuNYnm7p1I4u+sYaXVFRkJPUvbAF1gXQvJB4M+uIW/zmHqP5PG2pP3H3czMStcySnecWAYWd10RuzY8jjmVagMT3gKf8R0IqZdjK2/mxUZfO/kTY8j7/6kLkQPa7AlyQ7pg7QwTqBIB9hF2FWk0Crs8DbiAD1DKkFz20GI4gjmIGbVzvwbOFu3GXUbNiU/O7/BRM13URs/jlK2NwjX3ECtThWN9mTydF3AgCdOedPUp0zP3g5ULfFODIDIFROB5+m396l9KQZyhZW9Zd06JJinFjN12avf6WQnn5Yb+crpSSI4aOUsfQXFtknQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jInNIhdvRlovQ+I9As8WX1J+NR9g5ZKOmf2Zxb/tb9c=;
 b=dgNA22nBo0TeTJGwXmmKVxW0atW2ZWQ93cYHtOJJlkMOB4OrkhHgRQGlP7iUNmBaNOwJCfqLNz8HS4T+yS2+/yeMIYOOiqgbeuRBBclJt/7IJ634nRhYFUZaE0+sDKiFq87PubWYSICWOyxjWP1yZ/ytMZ2a9u73Um7P2qoRcC0=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH v5 4/7] xen/arm: allow dynamically assigned SGI
 handlers
Thread-Topic: [XEN PATCH v5 4/7] xen/arm: allow dynamically assigned SGI
 handlers
Thread-Index: AQHasZmDYpRVB5hNBU2Zi0KwxSKfZbGxZckA
Date: Fri, 31 May 2024 14:12:06 +0000
Message-ID: <04870ABA-2208-401A-B1CE-2FC9A9D75F04@arm.com>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
 <20240529072559.2486986-5-jens.wiklander@linaro.org>
In-Reply-To: <20240529072559.2486986-5-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|DB4PR08MB9215:EE_|DU2PEPF00028CFF:EE_|GV2PR08MB8485:EE_
X-MS-Office365-Filtering-Correlation-Id: db9285d8-d86b-4fef-b4fa-08dc817bac51
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?0QvAAnRsz2lZbdqRT4JNuigSpulSXQdX1GgslRi2zvCL6MunMGBqEREvf/yA?=
 =?us-ascii?Q?jKaeadz3EhWidRjNVWOR9DbHyy9TM1EURzZenQ2aKg8X/aWBrnFQXlI+PA/e?=
 =?us-ascii?Q?Gjt7MTQBFucuNK78Rnm+BFW5oge7XDiam38HR2Zout1Xn1lUNLyLEFYRBogp?=
 =?us-ascii?Q?PzVNVnVNvppdxi/0XU6xeH+uhlZeLBxcbNKJanT530YvQwpeUU/EyZLDPQrR?=
 =?us-ascii?Q?9xOy8Xmxk7ZJAwJsSa98SXBe0voWj1fyVq+8EQI2c7YWkLy+XGA4gPGUWapX?=
 =?us-ascii?Q?YbUqLCW98h/W2P55vt4INw/lltAsumQDQgMLcC0BEpAv+NbysDTqs+sL55N/?=
 =?us-ascii?Q?lcyM8ieV8txMgyviOLr/3hSus1nhQVrdyyOc+BY/Wv7015cgytThI2iBly/R?=
 =?us-ascii?Q?En6jH0W4aODLzlYuSKaxH62W7QDu+VZ9JskYAG2TU0FF6S2OwqHVP1K5wJtl?=
 =?us-ascii?Q?0nVV7A8I558qxMzpvP98kOkK8iQwRPGG5zPd6k42VWXcUZqxfkcJPfPFz8vR?=
 =?us-ascii?Q?HkTE483hktIQuX00QVPDWR/FeL0+Kn1KsyfW9+2NqDjb6UR71L6Fyiivxhpz?=
 =?us-ascii?Q?5v6GXy290rQb41s3/yD1fr6c13W5WVDNreVOk51KXL7sKM2fufOULpcKdBRk?=
 =?us-ascii?Q?TP/VyjAeVo7VEDQDYN2rkIWWFg3hiqczNH1IkVVz9/V3alz04h6Z7gE2j9kU?=
 =?us-ascii?Q?ZKMvr5bOkvp8j/mzzMCheIWA0ft5aMXdbKVK8289WdSDQHuhYK7k9kreJIfo?=
 =?us-ascii?Q?0q4MSvMmuZo6RKfoEOlFRkBSMLebn7i+5hJG1ZJLfDhVi7obZffAyW6fLMSt?=
 =?us-ascii?Q?VrZelRscRCgUg8rv5DB/Nyxt8j1vWJKDlgfvJ+BPbtOrWtM5k6g/bOI/gPtR?=
 =?us-ascii?Q?xDJ01Lg99M5+GEWk/sbQUB40v7zRHd8msAjtgnf5lWCInaUOJqpQeTiyrKIK?=
 =?us-ascii?Q?Qjm74lt7cslIv04cOmCJoSXo8hxXBySxLfliTEFuRYsVGruLVOlLfJsbsGoX?=
 =?us-ascii?Q?u6c7Ngbz73apVI0k0CqnXZD7EeAuKU8T2FccfP1Bjxy6jkLW3Koq1egl+WIa?=
 =?us-ascii?Q?pjXQJrQlVeZhNyhLyEBB/gKR06BqL/F5168VzhxKHNYQ1u9IdJxxFAN2JHMF?=
 =?us-ascii?Q?pMIsk8woS+P7/eoHzgFb4SRGHNvpbEBk4hL2LJrzD6pWbIh8V1E9CzleQsfJ?=
 =?us-ascii?Q?60KiSBP4Mxz6dvJD1UpS6f4F9U15kqgCQkj/Wrop+S4cztconp9i2CE2ZrfT?=
 =?us-ascii?Q?mqDWw7cVZvg8heR3vCSvyactlI/vs30TWjQbcvRrgQ=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <44787716E97C1048A24E75E12A658AEF@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB9215
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028CFF.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	08d6c476-839e-4fbd-a0c7-08dc817ba742
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017|35042699013;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?kFeI0sowbtz9ldk9h23N9D8KZLdQVqHBpkF8J3ZCgDc4MjYp1XANKaDNWHm8?=
 =?us-ascii?Q?EWUvsQR5cDhXfz1Bs7c+ofhUjs2jLN0yqhja/5B1o2B4PgZCsCJvs3sd0vDW?=
 =?us-ascii?Q?75wNjxfueRbmci5ns1f7yQQvkYSYz5tfaxnioMS3jVfb9eon3ljA6wgzUJfc?=
 =?us-ascii?Q?ppLNz9+IkOl93b321cCtwodh2G+ZahwYTxag+jL0hoDvOXJeXm1Sj9M2jTU1?=
 =?us-ascii?Q?0l5lJMJOJ1Wi+Vz6i0EZ7r1Q2gdNeQVgTbC+fByVPAQF/KWygp5qitRHgVTh?=
 =?us-ascii?Q?SwPHMqo4y87yxutXFFwZojTBbWAU8D4L7cLbIBLnA/l0w5+EOPjwYMUs/a8Q?=
 =?us-ascii?Q?/m1gWhGC9dz4zxN7qAFc+Z7Tn1ZRxnrSGmBAwsXjGRR3J//GONr9pfXucV58?=
 =?us-ascii?Q?XbcUkgRnMFUD+eAbtN2JVwaL38LvVy8K7AdJHqfJEsbVQfFOP2OC6cQVU+Vq?=
 =?us-ascii?Q?3WbpkzCMzbqdWEYEiBhwd0hSWP/2+OPsJsl2apJm+Z2XTM7pDeail9SaNxL1?=
 =?us-ascii?Q?YYBMk+cgcuzsP1vKL9tR53yp309BxNbt3tCFc7DOiTdFk29bztEICFGFwtWZ?=
 =?us-ascii?Q?C0h6IYDoaxzE4H41CxQ2Ll3+ks2OJp+878hn8hKwHV+t6ZPKQhF9K1mmoLUp?=
 =?us-ascii?Q?4/6En19ZZZS3Dpxxm1PS5Hi/Herpxaz3YFaqvS8Qrod9XMp3CVYn89TBh5mI?=
 =?us-ascii?Q?rx4dviF20nWfkwcwwHwRjGy//xVMrH+lXDn7ZxLRiEQDWkPlPqFqCgiwxy7a?=
 =?us-ascii?Q?7N/zBuMprCTTfU2AN8oOj/OtvGUrvqN4o+GCeWIqdzHjX4I1VKLdop7+gt/T?=
 =?us-ascii?Q?20ynWWzbJPzDFqKxm2Ykxo31ctmEkS+t4A/SmFAvAdUTwiXgekcH5gHTuuk/?=
 =?us-ascii?Q?Rrmzr/jsV+jI03u+zAqzAyJOIUy2Eh1BriTpZWHVzt8O9Le6F5YOcWthSIjj?=
 =?us-ascii?Q?IMRieWWYBJ9r0mryHWWNQ5USwwUJHZl3IBnF+lxI7foCHg2ADu0KmTVg/x2U?=
 =?us-ascii?Q?56zbK2oV6ew+EDDUqDI6NFcsOp3cnCZWg8HapQQ/nB/aZ4RcwZTTKnae8+ze?=
 =?us-ascii?Q?9IIoY5BamtZlZrbLxIwLr4PIekuZ2JSmSRjx9j2GTWO3RVJFrlxelK21Tl9x?=
 =?us-ascii?Q?KXotXCvX2/NLgOZDrpvBtGxaoYirJ61K67OQt7OYWdTdk73RzRDrxa47Jkn7?=
 =?us-ascii?Q?asOSiUZQzjBIZrNF6lcxpWCmFDvl7TBkDsj7uL6/6W4UyLQrWZd7y6G/6yrd?=
 =?us-ascii?Q?AWe83aWAomG3CX/xCOl26LHoNQAy3oEYe3n/3f2xBVZjYT6ZXj/5VlMPzosi?=
 =?us-ascii?Q?RVq4qAA0hZ1SG+//kGc+7MFA?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017)(35042699013);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 14:12:15.1764
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: db9285d8-d86b-4fef-b4fa-08dc817bac51
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028CFF.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB8485

Hi Jens,

> On 29 May 2024, at 09:25, Jens Wiklander <jens.wiklander@linaro.org> wrot=
e:
>=20
> Updates so request_irq() can be used with a dynamically assigned SGI irq
> as input. This prepares for a later patch where an FF-A schedule
> receiver interrupt handler is installed for an SGI generated by the
> secure world.
>=20
> From the Arm Base System Architecture v1.0C [1]:
> "The system shall implement at least eight Non-secure SGIs, assigned to
> interrupt IDs 0-7."
>=20
> gic_route_irq_to_xen() don't gic_set_irq_type() for SGIs since they are
> always edge triggered.
>=20
> gic_interrupt() is updated to route the dynamically assigned SGIs to
> do_IRQ() instead of do_sgi(). The latter still handles the statically
> assigned SGI handlers like for instance GIC_SGI_CALL_FUNCTION.
>=20
> [1] https://developer.arm.com/documentation/den0094/
>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand



From xen-devel-bounces@lists.xenproject.org Fri May 31 14:16:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 14:16:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733735.1140057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD33v-0005Aq-9x; Fri, 31 May 2024 14:16:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733735.1140057; Fri, 31 May 2024 14:16:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD33v-0005Aj-5j; Fri, 31 May 2024 14:16:55 +0000
Received: by outflank-mailman (input) for mailman id 733735;
 Fri, 31 May 2024 14:16:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tx2a=NC=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sD33t-0005Ad-QD
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 14:16:53 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2607::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d80d3e2-1f58-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 16:16:53 +0200 (CEST)
Received: from AM8P251CA0023.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:21b::28)
 by AS8PR08MB7791.eurprd08.prod.outlook.com (2603:10a6:20b:52d::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Fri, 31 May
 2024 14:16:47 +0000
Received: from AM3PEPF00009BA2.eurprd04.prod.outlook.com
 (2603:10a6:20b:21b:cafe::98) by AM8P251CA0023.outlook.office365.com
 (2603:10a6:20b:21b::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21 via Frontend
 Transport; Fri, 31 May 2024 14:16:47 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF00009BA2.mail.protection.outlook.com (10.167.16.27) with
 Microsoft
 SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15
 via Frontend Transport; Fri, 31 May 2024 14:16:47 +0000
Received: ("Tessian outbound 53b276c4c668:v327");
 Fri, 31 May 2024 14:16:46 +0000
Received: from 12d92602d80b.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 328B485C-F748-4017-BF46-D1A0EDE65C66.1; 
 Fri, 31 May 2024 14:16:40 +0000
Received: from EUR02-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 12d92602d80b.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 31 May 2024 14:16:40 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by DU0PR08MB9749.eurprd08.prod.outlook.com (2603:10a6:10:447::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Fri, 31 May
 2024 14:16:37 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Fri, 31 May 2024
 14:16:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d80d3e2-1f58-11ef-90a1-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=aJ2WSLqrUUc9Ao0RmsVmWk3yFf31t+QvP7lYy/02PxJ1cyF+BEwbs09QfyHmUVI9UABJp6hW4sdc45vLJzEHSrhx72E6yWtvDuRxLQQ8IyfzRxujIpl4OWOmXiI1iLuBpOkUI+xtL61MISrKhduK9aSXZM5fJ5kcU9DOB0/kcJVodz//AZbtBcHrTMLJAUc2/XjD1WFHXaKtaBg8Fp+o7LSySppANH4dyLH3p7kmjrNEGPKikMSbD5CvkfCbTLhvBPjR5LsfFN7vZVxZmAw/+eb5F0NJM58IBERwP2XMDHT4/7XFx4bSBkQJNPBO6EsdP4NepkpYalXBbed3WgIlTw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=j6qopxekm9rB9IanXP9gFZ9QrXGNKkARbXTOYOzxXa8=;
 b=LkE3ZF8EdvATkLRKzaxRk1e29EFBGfwpAEnOoJMks3hbsOR2DFVbHdAvBlG6uUaKSyHgQYKFdgXJF8dc9DTm63C1G99h80n5QSEiUXoBwV3zZ3t9g9/eLVAiwnKGrVdq9HBLViBs1ZfHzn294ETQYadPLImpk8E2RMbEToWrs63qFAUI92Sd8Swt5/8HMOj9yHYdGlGR/XYuJMLyN7bnPyg+9vYvWi4GsXa1Q/eTTQDILzrHcyF7nCBttA6DqCCFtnUxCqLJLVY/aEDxhZSdOTqt0/gpVNkFoiQ56BdBRAPKAasIaLd4WFO17rm6eZqzuesw6nZUXlzlxfFhKm3NtQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=j6qopxekm9rB9IanXP9gFZ9QrXGNKkARbXTOYOzxXa8=;
 b=NFOyWP5JMH3wWta0uZWytfrz03p6cH7uNqb1HOkumDa9Lfvtp8zniU/KlvpO2GAQAv+XOijSsKj0sHjj1Fw56ndqahcW5uOQ62vKUzmoZcPy9HrRMdqY6LzMobKF+5if2koWkJNvN4Az9AqOOUoDGk5FuLX9xQaPd0F2OPfNcDs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: a3fe3f95b2d0976a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IUk1RMImh6HyOtKAEiUneZs02SbgF4WgdGlp8kxRlh5SAa1iqbLYgk2i7bSIUOLG08wOUvHHqAok+bUm3wE6xP6T7wLc4ydIc2Z6GRKALi63JyVVR+HvnzTjXx4V/4sqYeBCFBSEmqTzjgN8Ex8qV97YLYkTDMhGBZVAYKDVJMDOPY3vvJeIjU3JTV/EJXMZ8LzKpizkHTKrxg1uVVAbU4iqOWqWFY3kGC9kkdjOrzr/RGv26J62emLm9nKqr2rSd7JeMkYcYGmFKrcH2tpvaRWtW4DxncFz8miCwlCCE1WJacq70CIplVcyabLl6pKAqPKQhjrTRirLhPtPyig+gA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=j6qopxekm9rB9IanXP9gFZ9QrXGNKkARbXTOYOzxXa8=;
 b=mEUOxyzuhZxhFPbrWsh4A+UDBHXXe7KjpOyFyffgSn90Pdf2WwrYQxrXY7kVdJMJipT8cT/80UaHPkai8f9tChEii20AZH020Vqwaji+dPpbhA7FU/LZx0bVQ/T/BPnGPk7lyOg00KTfpHvHQuelwSblgHpzG1phmGzInLJ7OKqgcTYH0jBNH7JES3qv0LDvLNWbqEsCAJvg6Bx1RouqAfCVtYeeBAlI6Qic2mza5DSMmt6hLperWnkjOv5KhvSLhma2YPIUIt0eegAMJTy15R5gNCXW9KCVErcdbTuAah6Ld4STg2KHgRm6fEm+4I8vUP0jtVKGLGo19mCWglwTfw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=j6qopxekm9rB9IanXP9gFZ9QrXGNKkARbXTOYOzxXa8=;
 b=NFOyWP5JMH3wWta0uZWytfrz03p6cH7uNqb1HOkumDa9Lfvtp8zniU/KlvpO2GAQAv+XOijSsKj0sHjj1Fw56ndqahcW5uOQ62vKUzmoZcPy9HrRMdqY6LzMobKF+5if2koWkJNvN4Az9AqOOUoDGk5FuLX9xQaPd0F2OPfNcDs=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal
 Orzel <michal.orzel@amd.com>
Subject: Re: [XEN PATCH v5 5/7] xen/arm: add and call init_tee_interrupt()
Thread-Topic: [XEN PATCH v5 5/7] xen/arm: add and call init_tee_interrupt()
Thread-Index: AQHasZmEe4CnQsBnEUWYbv68lN21P7GxZwyA
Date: Fri, 31 May 2024 14:16:37 +0000
Message-ID: <C69A15E1-AD34-4F4F-AE30-B90C861AB77C@arm.com>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
 <20240529072559.2486986-6-jens.wiklander@linaro.org>
In-Reply-To: <20240529072559.2486986-6-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|DU0PR08MB9749:EE_|AM3PEPF00009BA2:EE_|AS8PR08MB7791:EE_
X-MS-Office365-Filtering-Correlation-Id: e6f88842-0eb7-4c0e-aad5-08dc817c4e63
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?hpL6UHXUHPH5xo1FGFS5Gbct2r/FpwaZ18Vi++K/v2YztysViwk1hhvZR3+q?=
 =?us-ascii?Q?x8tG59MAXpB0xR7U/KLNytRcGKg22FVtlc5eSSEpPW6fUsiDjvBJlK++7IGP?=
 =?us-ascii?Q?v9GspOYkhvpAB7DUJFP4sKAI79Bbu8pVtBUa6Ge1SDBC8YyfvP4WSq2/70CK?=
 =?us-ascii?Q?Sj1fTEXG1+QneEgGKKnqMmPSfPs5+PN3AUdi9uS3OMO8bN6eYrmqMcdMEbUY?=
 =?us-ascii?Q?lbwdQH+tTQOKDAkQ56GWenz8gSUh/SHnngzGlSy3/lUzHx6zemDzx0kGqDCB?=
 =?us-ascii?Q?dqe1L+Ux914rp/KqL+UO0jL1aqD3T6rbesWR1fjZ9IOL2x6ZF8dYBcv5Km2c?=
 =?us-ascii?Q?Xonfz1H2dyzqZVlfam4dbJNkZRi+bnctcgaoxm2PJFHZK5Bq6QAkR9mgRG7h?=
 =?us-ascii?Q?tpVMIKuFut6nrX8NrXMS0aVoz+/MDBJOzZS2TI1OZbPWFuTvIxrrNCeFvVpi?=
 =?us-ascii?Q?Hjo3dgUq3X+uhBapnY+ZMUeivapzGQu0MxeduDU4G09QFqIRL6OyW6Uio0xq?=
 =?us-ascii?Q?4rtL7G/3bRvC1h3bf0fSVeluQn4zqWDwNyHOLRUj7oFrO7+ABfUp91aCGmYl?=
 =?us-ascii?Q?rn+BHFCKFd8kUiOGqAmePw8WXhONjjhpHccR4vy54/PruJ6anqyzTpOUPagW?=
 =?us-ascii?Q?4E0vdpyMJfoEUG69KtAOquQ13b/gx/K3L0dTKpvl5u8ISiqfWXLdp9itiKRW?=
 =?us-ascii?Q?NLxX/hw9Gof4HQRf6oKLskjObzncdUCTgdzfUujqj5WiCECs+dlyBij3lJOn?=
 =?us-ascii?Q?EnZ5dAzWy9njcQSh/rXSc5qR2VXpQ+6lkUocoGMlRqHqOA1m0WJ0Q0s1/H8f?=
 =?us-ascii?Q?Z/PyV4R/5DWUEKCxf6JIgwvznAJWSXrvfHOn12o/WNt01v3GzjxnfZg5Y3aL?=
 =?us-ascii?Q?qlv+a9Cb1uZhDb4SOR7Gr0RouvtethvgA7/9W52EYOBinRMUJL2Bi6TZzBq6?=
 =?us-ascii?Q?js/3heEZpsQAXnMMGlcr9nmo3L7rt1WBmQ6LtTTfO3608sX+V0V8Drte7849?=
 =?us-ascii?Q?d4tDVi0lBqXSOxuEL6PoKCVyXHbtyzv8Kv7IwT/D2g0emCoyapNFYrKr9/Mp?=
 =?us-ascii?Q?EpQwLNFd8Qv3uLgmFI5KLDcHHBY4BgBiPikmW86wH/IOR+3DXHpcFSngqaZE?=
 =?us-ascii?Q?hTJ2/c9CRoLkVMOhYpExgnrb+nYvp/MlCQhr3lHnKpZiRwpY3sV8vXfV4E1p?=
 =?us-ascii?Q?L01RNQqacfcCSIyxhyn0+CCLskKwn0Yjb6vNFgG6+kNH7zvI5jMd86yVxMye?=
 =?us-ascii?Q?zGW+sP/kIm0QtZNcFmSh/9j7D3knbtWurXnWwA3/xCllqG6e2X+uDxOcGzur?=
 =?us-ascii?Q?ok6Ok3ghLk1j7ilYnzz5hSQZnf8VyUDj3qb8khaTCYdr+w=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7C9CA1BF09453847B5CEA9E0754F365B@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9749
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF00009BA2.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	eef1ddfe-ea9c-4139-25b3-08dc817c48db
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|376005|36860700004|35042699013|1800799015|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?oIVb5+KjWvNPjTQllYU258pq6R8veSaf6iqAuAiK6vvA4+lIoryM57mQxV3k?=
 =?us-ascii?Q?oDUxSchqgMqVPxdAYQIwzjDdv9RODscMyDUJtxxgHHbrdPdXWRMoihuUmJDL?=
 =?us-ascii?Q?kWijJ8NVdgfpZQ29peu8yWSoOFNJYcGyPFzQ7el/zCiIQmw6Ltga1vE2zVKB?=
 =?us-ascii?Q?wM/cFrqHiKSBJLNV2f4Na/O+H1lr2dD6vWVu8TgWjxfHMrV0JXWdBgZc5OwE?=
 =?us-ascii?Q?OBw7oA9/QsKHkHYUVM14mltglwJ3Q7EjiBfHTb1jTFov8qb2/N+ttzHuCRsR?=
 =?us-ascii?Q?wMpyrzkuH8TouX4W2UXKaufwYQucr4Z1tSvpJfZZFZNSdPdT3OiIDNRuyNGU?=
 =?us-ascii?Q?nY0z7y9Fjdg/afTIELPm5Cb2HHRQ630jAW2f4ohqIZTtNxMEZieMoUuSDHdT?=
 =?us-ascii?Q?J1FpELSsOmITlJVaWfB5CLkaNEvO/vJcSwHuBi0WBqfULm35+8DoZnpwk+tu?=
 =?us-ascii?Q?n/pL8lRAwtlN9oa+X4eBgSmp/CSgP1GxxZBtSJTRVf2cZmomwPQyyjn26vPk?=
 =?us-ascii?Q?ymMBDReQ0Qww/It4ROJH4djzJmbB5owQOwlUsKiXjC6si68WA2YGTt1gsIta?=
 =?us-ascii?Q?fB+cLTj+Lv0Uj1ulUDUi99sZkNNi1Imq8xC+YWK7FfynmZD4yMakAFhcRPDd?=
 =?us-ascii?Q?JIUfn63i6A+nis3kDUS5H306ZgyK5wblXJ8grcmJrkrFbPRlw/aWj3oirTUv?=
 =?us-ascii?Q?vO6kQvZyU9LR/h3V652cJTOJBY97qfKLUR3Lfr+zHdy5TfUUjwUmjQXUp2J6?=
 =?us-ascii?Q?DBzQ8S9HuwZRttIIRZxaojmlNm9vut1yQwf8PTJkAQqD7aAg32x8vzTqfcXq?=
 =?us-ascii?Q?GlagOGWWotGtnmam0Hg1JrT6iUBOc6NYB4jwYR35/fThk24qiI+VBMX71xzC?=
 =?us-ascii?Q?gZU2n9wmTU5NF/YqfzRGczXzw3WVhrwqsE1cEfq28Us7C+8Tjo7PBy5OsnRK?=
 =?us-ascii?Q?uUjnoKcKN7PXTGBahVt9u0z3iiDH+QF1jIhrEmI3WQvlQQMoHJVtuXjXoA8Q?=
 =?us-ascii?Q?3GyS5sHsimDIyRmRHg+r/CMHNgpqdPVDzh+9b2rpIS6OeAexYkTt72G/8t/4?=
 =?us-ascii?Q?8HyBBkcH3A5O9TRkhJyo7AAOwd75CIpfC6BEwBPAKCN+I8NFfgvozZXp28pK?=
 =?us-ascii?Q?87q3go3LVsK3AeWv5gJNh7eJApUXScILXUe626fR57FpdoFdCPXdaup+OHU5?=
 =?us-ascii?Q?RH/bF3MH3RX+0wOufsStV0RtcoafmaxgfkldKE4qmchXFTZIvErWY3XKwBCL?=
 =?us-ascii?Q?zZENt0mwALI/BHUoLdohcJb2Rq8O2+PLqaxiG6LFEvP6uQDZynyuwcDfCmT+?=
 =?us-ascii?Q?pSC+365RRYhYUPJlnleISGag9P9Ix1m8MzYTHBxtNkqmByx9HinhY0INffzP?=
 =?us-ascii?Q?WiTpVeJpKOMFRxAzm+6vM9aSAG9i?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(35042699013)(1800799015)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 14:16:47.0068
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e6f88842-0eb7-4c0e-aad5-08dc817c4e63
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF00009BA2.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7791

Hi Jens,

> On 29 May 2024, at 09:25, Jens Wiklander <jens.wiklander@linaro.org> wrot=
e:
>=20
> Add init_tee_interrupt() to the TEE mediator framework and call it from
> start_secondary() late enough that per-cpu interrupts can be configured
> on CPUs as they are initialized. This is needed in later patches.

Just a NIT:
The function you are adding is in fact just an init on secondary cores and
wether you are using to initialise interrupts or something does not really
matter so I would suggest to rename it to "init_secondary" instead to allow
future use cases that are possible.

With that done, feel free to add my R-b on the next version.

Cheers
Bertrand

>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
> ---
> xen/arch/arm/include/asm/tee/tee.h | 8 ++++++++
> xen/arch/arm/smpboot.c             | 2 ++
> xen/arch/arm/tee/tee.c             | 6 ++++++
> 3 files changed, 16 insertions(+)
>=20
> diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/as=
m/tee/tee.h
> index da324467e130..eda8a8aa38f2 100644
> --- a/xen/arch/arm/include/asm/tee/tee.h
> +++ b/xen/arch/arm/include/asm/tee/tee.h
> @@ -28,6 +28,9 @@ struct tee_mediator_ops {
>      */
>     bool (*probe)(void);
>=20
> +    /* Initialize eventual interrupt handlers on the secondary CPUs */
> +    void (*init_interrupt)(void);
> +
>     /*
>      * Called during domain construction if toolstack requests to enable
>      * TEE support so mediator can inform TEE about new
> @@ -66,6 +69,7 @@ int tee_domain_init(struct domain *d, uint16_t tee_type=
);
> int tee_domain_teardown(struct domain *d);
> int tee_relinquish_resources(struct domain *d);
> uint16_t tee_get_type(void);
> +void init_tee_interrupt(void);
>=20
> #define REGISTER_TEE_MEDIATOR(_name, _namestr, _type, _ops)         \
> static const struct tee_mediator_desc __tee_desc_##_name __used     \
> @@ -105,6 +109,10 @@ static inline uint16_t tee_get_type(void)
>     return XEN_DOMCTL_CONFIG_TEE_NONE;
> }
>=20
> +static inline void init_tee_interrupt(void)
> +{
> +}
> +
> #endif  /* CONFIG_TEE */
>=20
> #endif /* __ARCH_ARM_TEE_TEE_H__ */
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 93a10d7721b4..e1c1e20efd98 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -29,6 +29,7 @@
> #include <asm/procinfo.h>
> #include <asm/psci.h>
> #include <asm/acpi.h>
> +#include <asm/tee/tee.h>
>=20
> /* Override macros from asm/page.h to make them work with mfn_t */
> #undef virt_to_mfn
> @@ -401,6 +402,7 @@ void asmlinkage start_secondary(void)
>      */
>     init_maintenance_interrupt();
>     init_timer_interrupt();
> +    init_tee_interrupt();
>=20
>     local_abort_enable();
>=20
> diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
> index ddd17506a9ff..8117fd55123e 100644
> --- a/xen/arch/arm/tee/tee.c
> +++ b/xen/arch/arm/tee/tee.c
> @@ -96,6 +96,12 @@ static int __init tee_init(void)
>=20
> __initcall(tee_init);
>=20
> +void __init init_tee_interrupt(void)
> +{
> +    if ( cur_mediator && cur_mediator->ops->init_interrupt)
> +        cur_mediator->ops->init_interrupt();
> +}
> +
> /*
>  * Local variables:
>  * mode: C
> --=20
> 2.34.1
>=20



From xen-devel-bounces@lists.xenproject.org Fri May 31 14:17:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 14:17:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733739.1140067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD34d-0005ft-HT; Fri, 31 May 2024 14:17:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733739.1140067; Fri, 31 May 2024 14:17:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD34d-0005fm-E2; Fri, 31 May 2024 14:17:39 +0000
Received: by outflank-mailman (input) for mailman id 733739;
 Fri, 31 May 2024 14:17:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tx2a=NC=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sD34c-0005Ad-F3
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 14:17:38 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on20609.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::609])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 88515dd7-1f58-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 16:17:38 +0200 (CEST)
Received: from DU6P191CA0047.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:53f::12)
 by AS2PR08MB9023.eurprd08.prod.outlook.com (2603:10a6:20b:5ff::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 31 May
 2024 14:17:29 +0000
Received: from DU2PEPF0001E9BF.eurprd03.prod.outlook.com
 (2603:10a6:10:53f:cafe::de) by DU6P191CA0047.outlook.office365.com
 (2603:10a6:10:53f::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.23 via Frontend
 Transport; Fri, 31 May 2024 14:17:29 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF0001E9BF.mail.protection.outlook.com (10.167.8.68) with
 Microsoft SMTP
 Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15 via
 Frontend Transport; Fri, 31 May 2024 14:17:28 +0000
Received: ("Tessian outbound 7e38650a3f1f:v327");
 Fri, 31 May 2024 14:17:28 +0000
Received: from dc7523588bca.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 51C0A907-C550-4B67-B67A-4C3D4AD7D21D.1; 
 Fri, 31 May 2024 14:17:21 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id dc7523588bca.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 31 May 2024 14:17:21 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by DU0PR08MB9749.eurprd08.prod.outlook.com (2603:10a6:10:447::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Fri, 31 May
 2024 14:17:20 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Fri, 31 May 2024
 14:17:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88515dd7-1f58-11ef-90a1-e314d9c70b13
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=GNJZbCGIgDIGDdAj6CqWZRAyhG8ggKWFC0d1tCidm/orur47Ut2C8FYusq5KtUiX14/MIwRx4SV71fKZTPAYMC/o8jo0OKdIt9Fmu9Pf/phVCh+ZDchSYZrl2PiDdVNn6KFVHXyL4BBmN2Il/5400RHXBmNsk9cPDMej0l0fwKND5JN0m2rTr/wtGVo1fedTmwD2mab/pmUm45fWdG74/0CT7S9COwuWxQcbM8aHh4Ybb9k94wEwqz2EXDZ04cRXAJ0vzrm4boQz0w4L37YhZ/OBv64kzaPhOBamYZt5g6a2Q0Uc+G2nYUs+1v/oq6W2KEYQ0YDrh6evtxAC0BcYUw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dITFB1iRQXWXAw/4BmCnEF2oLlLW8hTxmBdl72Izwns=;
 b=FdqhbyWSl0BKnV+nB/87IWkgOEnV2hVGsBrd37ELqCSNAX41wQEdGPr6wtkBV+xQDAqwguFohpaFvExSod/tj58QjsiIjTXnGgfW4CD/un0+L9YXOjNW6pz3/C5I3ZmrmYNmfTO0HdPA20fcxSLTis3jDxD1BdhTlBtAyVDslz2iQN/0EgFADkH7aCv5Gev3wlIXapZBSodCr19Vkm/WMgpL+GKHwbUGwsNZYL2GCtZPfGb3xQXnzGngcoPt0ruSiROCXEHpvTU5bcVzn2XMu8dQEryLLSq9f67LErEbMWdihPr4Sflmj855wIMAH8Adqrip2ROEhOPPvbFwT5iiyA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dITFB1iRQXWXAw/4BmCnEF2oLlLW8hTxmBdl72Izwns=;
 b=eexWKA8g0su/5D+hZl/vFl9A4qn/5s2CS9lukKkG4RV/fk2gbilgz7NpzZ+fYuYzY27Ff0oeFNJLL3/yZPAv0tbyEYVzFZCBOli+0MB1o8bbpU6Y/0l8PVHGqBA3fx/MamdHEOd5QOmD3pzjES3FCZxt76KvHtd3ySuqPkgM9Hg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: e582799644bf5fc1
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XrHWc6/GB+vsdX0ua8y90pAuBOkxHjG2Qn10ZatW9ZaY9jH4Nr/fTGv2+J4iuchOv7d1vZd4GMy2ByMGx4guhWS4LKbwYBLjSirvqgM1vbkPDHaVnZWweoir9GFN1qVawUcYkLF98IKTop1EQIQPOeeQrqn4+yRMdeGdRYiakjA/D1i5qVSgQb1TsbLehqfPh8d6V1fnY5ln49pZwrSG4HXEgAEb5/bIwrWHKYi46ujdQ1zr8hryNE1rHQm6yWt/YCxJz30NHX9HrWaxhSjrlNb4TyIjZRTa8V4tSlRV/folX5Dw4EzCEJeCyo1GdySmwccjDSh5TrXMMJtUJn0OTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=dITFB1iRQXWXAw/4BmCnEF2oLlLW8hTxmBdl72Izwns=;
 b=IMjpDJ+jCyhv37PonIsePur9b8UMYhtv/hGXybgBF93iXwLgfIC85Ayrtpxnmjg8JIGSol6tyoHqsuS9I49Xq8PKXyJzX5B/NZW7XL23+OxCy8QRsgKVui4UY8rGU68XQTkWvPIkNqSM0HBimsU4uFXijL57l4sXcNKKDYDRyEpNiEQpuvjfaIeSHulyG/rneyyjOefBQPuNgn6xG/PTQB0k4fQWyW8k3oWkYnZ9VPeDBV+Kft6/f+Teu9Hqm3+574DWsfVs6T4mwJ8SRSnwALSH8C5qsPNzQnULXDgraIYKtqMvDXGCMIO0EK6gFakNcQ0sby+GX+fWuHxBg7N30g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dITFB1iRQXWXAw/4BmCnEF2oLlLW8hTxmBdl72Izwns=;
 b=eexWKA8g0su/5D+hZl/vFl9A4qn/5s2CS9lukKkG4RV/fk2gbilgz7NpzZ+fYuYzY27Ff0oeFNJLL3/yZPAv0tbyEYVzFZCBOli+0MB1o8bbpU6Y/0l8PVHGqBA3fx/MamdHEOd5QOmD3pzjES3FCZxt76KvHtd3ySuqPkgM9Hg=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH v5 6/7] xen/arm: add and call tee_free_domain_ctx()
Thread-Topic: [XEN PATCH v5 6/7] xen/arm: add and call tee_free_domain_ctx()
Thread-Index: AQHasZmGvrbSer/b/kmljSykDWQao7GxZz6A
Date: Fri, 31 May 2024 14:17:20 +0000
Message-ID: <958BBE21-DF91-4D48-93E2-E49E733EE7A0@arm.com>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
 <20240529072559.2486986-7-jens.wiklander@linaro.org>
In-Reply-To: <20240529072559.2486986-7-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|DU0PR08MB9749:EE_|DU2PEPF0001E9BF:EE_|AS2PR08MB9023:EE_
X-MS-Office365-Filtering-Correlation-Id: 4fc5b3cf-c21f-4b5b-4cd4-08dc817c6725
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?erxfUy4ZBTv60Ckcl9gw6QElcW+6rR0bm+3nDVa0EEBc3KNN2SzlCTfOUVkl?=
 =?us-ascii?Q?npBXq1ejWcRvMAB4STFBW34fMRF4fd3yV20O8r6CPOitBLh1UHL7jBWinuCL?=
 =?us-ascii?Q?yx4Qn3tlFd01q9g1q9mkIC3Pmg60dsFMGU7fWM8JttE0i7BOnwYsI61ZA8gU?=
 =?us-ascii?Q?jsifCGHggqGJsCWZ+RBMp7VQp9IwklJWOi2tv2Esg1+SAYfa914MlHpbBy6D?=
 =?us-ascii?Q?AgGh03Inct8F6bMLnGObr2xxqzW6dncrFk3UH/1cvGBMOlb82+vesYAUG1Lg?=
 =?us-ascii?Q?nHKOBaqvQZJeY2PJtv5OlT2vXA9wktFDC1wwVUI5h0sgl0O9MQe2G9oC9a8r?=
 =?us-ascii?Q?J3mWQZ+n1BPYlkZ39DpoaJqYI4+e1A3NQnR0jAVhpqN9hnf7uGcgk3sv748p?=
 =?us-ascii?Q?9FM4pEa4AQaOUcy/jJtb7mVpm5zHD0B0aLOfdDnDLP6Rf7B8sziJGHAI+Yfa?=
 =?us-ascii?Q?Ke53r+yu34pM8Yap2+J/B4HK0dLi7eOavmD4cGH6tTS6Luy45gBZLXT0wgzA?=
 =?us-ascii?Q?AxuZgg8HEFVxCrE0aijseTZEl9ClEZv0/cxHfA/mm2h1gkKz+PaUAVaSR21e?=
 =?us-ascii?Q?V2f5BFqwtbiHaejZ2YMKUDNpwd6EX2JAwbrWLHBN11IGPqwNZWprGx0FKe2n?=
 =?us-ascii?Q?/xXypoXsoWrhmkjzSrugobiVG1VN5EyOsxle7wW+xasbioJyFPuF8g8ReDSM?=
 =?us-ascii?Q?JJV9dHYfn6l1f6wDlM3ShMYFbgAwCd0IFj5C8jlTa/QYe4Es5icHO1c6FjDU?=
 =?us-ascii?Q?Ye+pBB5AJnVb5nHBxxKLUBPns3nP+DQNzUiiVNhLWus6YBILkBY+Cmel10qc?=
 =?us-ascii?Q?q83cKvb+eLN9WmlKmgdgdVlKoicx/D+rZUppWD50UqVX9m73D0xit/oZP9ZF?=
 =?us-ascii?Q?gE2M4zPkyTVi1iprhYe7XPUQeBk35n06W6OJ0Auy8p6ge3InAhLOWrKDD1Ar?=
 =?us-ascii?Q?h9h+jGtPTzVSjSkaHtaRUHXs9Asbk3z+l8vWAeHCB2fpBZS0eDUCMeyyQL/2?=
 =?us-ascii?Q?uoBf6mz38UCNkEd4T9kVDCJPgkaubIW5m2LoiTR1OGYEZZiPxUOIzdWfF8Bu?=
 =?us-ascii?Q?U68nBAwJvyW8aA7yuKGTQaR8OMsAdO5KlqBU9V+hKLsvlmWzqQ22kaLFXxnV?=
 =?us-ascii?Q?5B/LiOfAos4CzNSzp4EkV3MAFX5qUtc3MlKAwqR+6Z6lDBdOUSmQwhiz3jrO?=
 =?us-ascii?Q?FTqkuAdqi7HWuCSglZmTIFdlGqexg0foA6e60QxBlwIDK4pSWCEdr62lVRrc?=
 =?us-ascii?Q?y1FsIiB8hfOVRPrWB31RtNpi7QxHu6ZLA0HJn0YSmy8J3Gc5fWPbzFPwnLz3?=
 =?us-ascii?Q?m7SGVro2gFYNXEnQmvBeTvMm2RhU3YJUx28CZtqs9wOweA=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <13D9C7C2D661B34B8BDB06C4ECAB3E08@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9749
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF0001E9BF.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d0083712-e882-4d7d-8be3-08dc817c621f
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|82310400017|35042699013|376005|36860700004;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?tBDD6fRNI6IRwlofEXcWZXV3wnm2c5NllgaM8cXbpI8zvw+3tH5j7HPn2Uy4?=
 =?us-ascii?Q?G27GyA0C5y8ppzLCpwK5su7oNCFoHHjhCmNkZeg+hE0xMDwH9pfwxSOYJgCg?=
 =?us-ascii?Q?5905iR1IJO/vCo46Ezo3ercu47iaTRG7kQjonTetQUj1KflZRUZmYrmA6v6P?=
 =?us-ascii?Q?ntP/yJ15WvhhJoRiDVubiLPNeBSxO/l21GBfL9dgnpCaEk+XcEXqy6cs/fkG?=
 =?us-ascii?Q?jB5LMTX4WKeh5RSgoOn7yRRHte8ie+9zIBlXRKmAioFvtfpfFCnVL7zkpk2t?=
 =?us-ascii?Q?z9hM6fEpimfsG0wHnzaTw7P8z8YBcw0t0NhXhzWV/MATqd8oxfrSZubeEll6?=
 =?us-ascii?Q?hUQ1n1EYGNp2oGeDIh6O2oF5o7idUtDpdCqY1ogn5sOS/jDZLM6dmQiFARzT?=
 =?us-ascii?Q?1zg5AVAnQXAHjBJ9zQudEEzbCk+zhv5N/f5mPXtEt4tbMD07zM5JvbnjeIHZ?=
 =?us-ascii?Q?WSGTJ1Qvpl/A2w2uWdstGbv+D2qXb0pvkUeFnmFFQ9SRPk8tiG/ugPvMWiyE?=
 =?us-ascii?Q?hgD3w+/N5i9VRAMKnpHyEDTY8MHVsbtnTRWONxjIIHZDGvHijEZtRuyB8r4L?=
 =?us-ascii?Q?/rC0RCLq7P9YUV19/UNYJwSbeswsrB2yIyHE8KmnXXksvaKNHk7FKQpvWC1U?=
 =?us-ascii?Q?SPaF0iiOASvpin73/hiQjiIF2wSLNIURdU1oCiFX4krt77tHEouJJk2BC37X?=
 =?us-ascii?Q?eevf7r0vttB71CaOl6vekBuKjS+Wb8nWkOGu2MnKa6ancas0XPwAk01DojvZ?=
 =?us-ascii?Q?7kWrGqnSXVa+DnsUvm1v+jGVii2f7mxDI5UIhQ5RxM9a3ktFrhmW5aKoqYSW?=
 =?us-ascii?Q?NvlAzV8z7BIkP4RN3Rz2MxC3vkhbQLw5IEBbBbc/oBFNevbvXjyPIHBkgbYd?=
 =?us-ascii?Q?8fP577dc7Zav47VXVTyqlJRjVrmbg4HSs76Lhkj8U8zA+biNWyotludDuPEv?=
 =?us-ascii?Q?Sm2cNsFOFc+U9fuQ2UlGF4cpSt/6K4Nlfb1QdZaT0kO3R/Jk2EQhy+UHej8G?=
 =?us-ascii?Q?y/f7bOtW3AkPNHeoKhaIDsakpfmQ+epvZmjM8F/7fON8LnWYcSuMzVXltWuw?=
 =?us-ascii?Q?Qf97iD9QpwiYOrrKe7y+dsO58MAlRTBPflE+Loc+F5DInPqk+7S+yCwDnPOP?=
 =?us-ascii?Q?cJH/jdghFnf8X/jK1Lnpe3NGoFN9jbuNr30K5RrniQIK1Cz/9FbI3k2Sj81T?=
 =?us-ascii?Q?0/fYSP5WI1NNjNJAJTdWn2A8SLI7hDk/xLvSFGaxaFODU7f2i0CG7T2KsdYY?=
 =?us-ascii?Q?d4aAKSPykVaI9F/o3PLMt9kNFBgbFJa4Mp/6R29Atc2frUupOY+OEPqR9fti?=
 =?us-ascii?Q?p4lPwJ6kke3+yYkFZ9svz4u9E9wIFsBVS1Bom55tV1aOqm6mlf+meVbkgIP8?=
 =?us-ascii?Q?7MIDnz5uPXnUw2u9tT+kFkq5EoEW?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(35042699013)(376005)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 14:17:28.6193
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4fc5b3cf-c21f-4b5b-4cd4-08dc817c6725
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF0001E9BF.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9023

Hi Jens,

> On 29 May 2024, at 09:25, Jens Wiklander <jens.wiklander@linaro.org> wrot=
e:
>=20
> Add tee_free_domain_ctx() to the TEE mediator framework.
> tee_free_domain_ctx() is called from arch_domain_destroy() to allow late
> freeing of the d->arch.tee context. This will simplify access to
> d->arch.tee for domains retrieved with rcu_lock_domain_by_id().
>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> xen/arch/arm/domain.c              | 1 +
> xen/arch/arm/include/asm/tee/tee.h | 6 ++++++
> xen/arch/arm/tee/tee.c             | 6 ++++++
> 3 files changed, 13 insertions(+)
>=20
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 34cbfe699a68..61e46a157ccc 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -837,6 +837,7 @@ int arch_domain_teardown(struct domain *d)
>=20
> void arch_domain_destroy(struct domain *d)
> {
> +    tee_free_domain_ctx(d);
>     /* IOMMU page table is shared with P2M, always call
>      * iommu_domain_destroy() before p2m_final_teardown().
>      */
> diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/as=
m/tee/tee.h
> index eda8a8aa38f2..2e99a38184be 100644
> --- a/xen/arch/arm/include/asm/tee/tee.h
> +++ b/xen/arch/arm/include/asm/tee/tee.h
> @@ -38,6 +38,7 @@ struct tee_mediator_ops {
>      */
>     int (*domain_init)(struct domain *d);
>     int (*domain_teardown)(struct domain *d);
> +    void (*free_domain_ctx)(struct domain *d);
>=20
>     /*
>      * Called during domain destruction to relinquish resources used
> @@ -70,6 +71,7 @@ int tee_domain_teardown(struct domain *d);
> int tee_relinquish_resources(struct domain *d);
> uint16_t tee_get_type(void);
> void init_tee_interrupt(void);
> +void tee_free_domain_ctx(struct domain *d);
>=20
> #define REGISTER_TEE_MEDIATOR(_name, _namestr, _type, _ops)         \
> static const struct tee_mediator_desc __tee_desc_##_name __used     \
> @@ -113,6 +115,10 @@ static inline void init_tee_interrupt(void)
> {
> }
>=20
> +static inline void tee_free_domain_ctx(struct domain *d)
> +{
> +}
> +
> #endif  /* CONFIG_TEE */
>=20
> #endif /* __ARCH_ARM_TEE_TEE_H__ */
> diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
> index 8117fd55123e..cb65f187f51f 100644
> --- a/xen/arch/arm/tee/tee.c
> +++ b/xen/arch/arm/tee/tee.c
> @@ -102,6 +102,12 @@ void __init init_tee_interrupt(void)
>         cur_mediator->ops->init_interrupt();
> }
>=20
> +void tee_free_domain_ctx(struct domain *d)
> +{
> +    if ( cur_mediator && cur_mediator->ops->free_domain_ctx)
> +        cur_mediator->ops->free_domain_ctx(d);
> +}
> +
> /*
>  * Local variables:
>  * mode: C
> --=20
> 2.34.1
>=20



From xen-devel-bounces@lists.xenproject.org Fri May 31 14:20:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 14:20:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733744.1140077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD37p-0007bq-49; Fri, 31 May 2024 14:20:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733744.1140077; Fri, 31 May 2024 14:20:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD37p-0007bj-0s; Fri, 31 May 2024 14:20:57 +0000
Received: by outflank-mailman (input) for mailman id 733744;
 Fri, 31 May 2024 14:20:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD37n-0007bW-4w; Fri, 31 May 2024 14:20:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD37n-0000x3-24; Fri, 31 May 2024 14:20:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD37m-0007jW-Pt; Fri, 31 May 2024 14:20:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sD37m-0000SJ-Pa; Fri, 31 May 2024 14:20:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Wsnz4fBfL5NjkcajVv/MFfBdUJJ8OxkNKgLeapRl8RY=; b=leS25ISK4l2CkOvE0ilnZWJK8v
	Ow+O/rusaXJtFSQF3lv3ITP9dFvOubygXOCkrIr5bPHjIQ7jk/yPb6bv+4k0gK3gHgY8Lmdqis14t
	RDLnjwbUfE8UbZ0zxoNRjfElokmDLUYdey4YPDH8iOPJ5RdMP2sVBh1HwjmvMZVZrMpU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186210-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186210: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=7c584bb04874bb5bad16fcf3996f5a893cc81a1c
X-Osstest-Versions-That:
    ovmf=746cc5cc40bef22d606cd22d1feb10d73a7b3d11
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 14:20:54 +0000

flight 186210 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186210/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 7c584bb04874bb5bad16fcf3996f5a893cc81a1c
baseline version:
 ovmf                 746cc5cc40bef22d606cd22d1feb10d73a7b3d11

Last test of basis   186209  2024-05-31 08:13:05 Z    0 days
Testing same since   186210  2024-05-31 12:44:34 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Qingyu <qingyu.shang@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   746cc5cc40..7c584bb048  7c584bb04874bb5bad16fcf3996f5a893cc81a1c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 14:29:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 14:29:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733756.1140086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD3G6-0008KI-Po; Fri, 31 May 2024 14:29:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733756.1140086; Fri, 31 May 2024 14:29:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD3G6-0008KB-N9; Fri, 31 May 2024 14:29:30 +0000
Received: by outflank-mailman (input) for mailman id 733756;
 Fri, 31 May 2024 14:29:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tx2a=NC=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1sD3G5-0008Hp-0i
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 14:29:29 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04on2043.outbound.protection.outlook.com [40.107.7.43])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2df94626-1f5a-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 16:29:25 +0200 (CEST)
Received: from DU6P191CA0030.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:53f::9) by
 DB3PR08MB10333.eurprd08.prod.outlook.com (2603:10a6:10:436::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May
 2024 14:28:48 +0000
Received: from DU6PEPF00009528.eurprd02.prod.outlook.com
 (2603:10a6:10:53f:cafe::69) by DU6P191CA0030.outlook.office365.com
 (2603:10a6:10:53f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24 via Frontend
 Transport; Fri, 31 May 2024 14:28:48 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU6PEPF00009528.mail.protection.outlook.com (10.167.8.9) with
 Microsoft SMTP
 Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7633.15 via
 Frontend Transport; Fri, 31 May 2024 14:28:48 +0000
Received: ("Tessian outbound 53b276c4c668:v327");
 Fri, 31 May 2024 14:28:47 +0000
Received: from 5763db377f5c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5747244C-149C-48BE-8E98-4E92799A1A78.1; 
 Fri, 31 May 2024 14:28:41 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5763db377f5c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 31 May 2024 14:28:41 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by AS8PR08MB8681.eurprd08.prod.outlook.com (2603:10a6:20b:565::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.23; Fri, 31 May
 2024 14:28:39 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::a8fc:ea0d:baf1:23a%5]) with mapi id 15.20.7611.030; Fri, 31 May 2024
 14:28:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2df94626-1f5a-11ef-b4bb-af5377834399
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=X8zAoCVLAsnkt4E1yMb4o8JNzd3C4MhdUrsc0OcxZNWbMkaI98hqKDUrcDk9w95+FNE0sxHJzS3J8/2mBQSRopMVX+ul/dXoD8ZMfBjRrC+a76FOldS6k268p6HEzUjOU76YAnziYPPDFZhfF24MPKyed1xmihl8PMv7BfjtS4R53FVvVfglbCfUnPGVCFLiN8vKeQzq0H9cU12F1q9+6a9gwhdgznVE8/OLqOtvqIJntyRXm/tCvfEEMlVuUV/iHkPfzkBH0RFMzvrsiVsbwkzKL7qbRwjTwiK78VpHY2phNdtAWbBQD+1KwT5g8UDUiKHwC0oLaIha+op3U/8KSw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=0DtbrsJxqKrugAtIsyvqPvAb/eDAOPvoEkuEbUb1ltI=;
 b=n+UdFUqlhNdZwx3EOqWySGZXly3yGXxt3Y94T+ZudMMfzGVoVB2Af6Oi/x+0lq7299KVQ9wkrMc+BqFRsvFUwJLMFCKW4ttOacmme89Ez3Mg4TWJstFKWaK5yvOQFofKnzAaN2UHPCSOQtHfZzCIOd6p3wyU7ItF3i9fJlPzf0QsHsbgbTvp7rjgwfM7Ee6WIyLXjhPoBYny/P5KnzKCVNciuRgycQCFtxwvDTOoALiu0c7VX/eLwf+kVK9j7/OkjMXhpNk5ito+CQIn965qelpCg8y8d2xP2+kCQmbmSnByuUQUqq7ameISBdGWd9h6NgOZw7shtsve62kXs5xA+A==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1
 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0DtbrsJxqKrugAtIsyvqPvAb/eDAOPvoEkuEbUb1ltI=;
 b=XgwCPX2zfHSynm4nun54fjyuQUimS2S9O4gmBS/oE3HS0OWifjMiUuwaCZexiESoDONV7GfBDn1nsVHgMSbtf03PIBnYiTCywdDLVnZhP1BJPPt1slxDzaj3lJsvkYBJTSkFOw8lr1A0aOTtqeDat31ZvqlQXKEdUfaCEZSAzho=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=arm.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 253bd17835a0ff02
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H7lr/Be/PlcAKE9HdTEPWGPAiL/Qzkku/S0hi6VxKXlehXhvEqm8exEMuu3MOkP5o2K5gs2H01zkB4siUpt7Hw1L5so/v71fx8brcH9EO/XLA0Q6SVyGY3H4U6NOCGn8N4GYpmqEb6d1pRYshN6iHwAcwXHfdp/LagT2dIEFib7LKKFpHSTLJXXR8JqICgGsiKoExkNjH3o29Ezz7PBrZDh+yyCXFd7eyKHaehK/BAlWl4f4TcvWRvausOZMv0raLGCB77NgjG0Wrs01GruzDujcJfG2y+bWIeZxaR0Amzvc74DQXKNC9XKjayILfT7BRnmcYQA5wWtMnpzV3oISKA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=0DtbrsJxqKrugAtIsyvqPvAb/eDAOPvoEkuEbUb1ltI=;
 b=NLn3Piav+1wiYEmKYXuDhjSAxLWQ0E2I5ReGrBtlTacvK8n43Ea/vWFw/4rlMIYsYlnpjnxI+cairUKByoQPCFqaGgUz+BnrAQyE6ZWrD6ULOzZMNAzxEexS+DpkYCYJmNzFHabDyHm9n+BWtxt33vughctrwnj0ymWHc83XqKNpJDVTri/2HIZDyyn3OdGFGrowwPTdsMEiaH1uEh5abJ4+vZOalwVBvLXAkY93Zk+O8WFcbCwp7CzFBG+gWA3z+OmdK9Jq/c7CLFCkcM58NYRbuNjH7gNTK3ie3ViQLojNF1wkqPMrcLCcv3ZFv24++GvRwPGWouGPki42jgZFlA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0DtbrsJxqKrugAtIsyvqPvAb/eDAOPvoEkuEbUb1ltI=;
 b=XgwCPX2zfHSynm4nun54fjyuQUimS2S9O4gmBS/oE3HS0OWifjMiUuwaCZexiESoDONV7GfBDn1nsVHgMSbtf03PIBnYiTCywdDLVnZhP1BJPPt1slxDzaj3lJsvkYBJTSkFOw8lr1A0aOTtqeDat31ZvqlQXKEdUfaCEZSAzho=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal
 Orzel <michal.orzel@amd.com>
Subject: Re: [XEN PATCH v5 7/7] xen/arm: ffa: support notification
Thread-Topic: [XEN PATCH v5 7/7] xen/arm: ffa: support notification
Thread-Index: AQHasZmLIIIAu+VRvk26HUXX7ib+FLGxammA
Date: Fri, 31 May 2024 14:28:39 +0000
Message-ID: <C52D6A7C-1136-4BF1-9060-600157F641F5@arm.com>
References: <20240529072559.2486986-1-jens.wiklander@linaro.org>
 <20240529072559.2486986-8-jens.wiklander@linaro.org>
In-Reply-To: <20240529072559.2486986-8-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|AS8PR08MB8681:EE_|DU6PEPF00009528:EE_|DB3PR08MB10333:EE_
X-MS-Office365-Filtering-Correlation-Id: 81361f87-126e-4594-38bc-08dc817dfc1e
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted:
 BCL:0;ARA:13230031|1800799015|376005|366007|38070700009;
X-Microsoft-Antispam-Message-Info-Original:
 =?us-ascii?Q?YOLIDN5xoq4MzrzCtPj/BKGxI5tF97/WWZVr1vSkAQUYYq5ZIIT1VPHlAWj6?=
 =?us-ascii?Q?9e1qS/s7TfoO4534SkRBF2ZOZ66f4sYnc25RbdUPDlmkt4awkPFd49D5ZIeR?=
 =?us-ascii?Q?ZgMA/qRNqBRSvh9OoA6lR5Ot0TXHMvAWPyRH3OZ5in58M+xT4Yu/bNI1B9uB?=
 =?us-ascii?Q?odf/G2fmpHLytujl4MT0+exSLfr/FrVgqdu7+vXnF8BuAReaNCUn0eNdJW6+?=
 =?us-ascii?Q?bbWRzGaFEyWUZ+LU0Nh2tyQ0PwG0p/oR1l105RQaWHKBIi4fxxKrviPc76T/?=
 =?us-ascii?Q?caUwaDjaTt5bvjpuJzHZy4TYMZdewH79lMKE85JM4/U14uPAj4w7/gyLO9zU?=
 =?us-ascii?Q?aCyTmj0qcGdGT5dNBw3yDgETJe23WFn3xaA/l6IyIvle4py19L9Cyao6JRBo?=
 =?us-ascii?Q?JcaxxIAi3TNI3xyT2Bl2UktIborwWU+L4qU++G6K0cxuLMPn7vkZEDBStHpb?=
 =?us-ascii?Q?DpOApfJnMnmI3sKpQj860gJA1eq52b9ZXeyYM3V2HtfKqtwcHTA966/xGd6l?=
 =?us-ascii?Q?EkjH6uc/3tMDT+AI1/5GvkEWg/SIbf5DVJR7m+afhqEmj5obxOLh1MS62fAE?=
 =?us-ascii?Q?YORg/zTaByjh6dU1sQIMOO+Bl7+zizE1Zi1g9FvTIirrsSRHvaUvNio5/dLc?=
 =?us-ascii?Q?1DLY25lQcJ4R4Wpn9nBaSfUBN8NFxk1GrSn+PWREg/eGs124zixryUAjpSFW?=
 =?us-ascii?Q?pJqsqvr9CIUscVXeKEdnKLMeOkulUUijtWlg4jM6uiPc8unr5pWD+KNxiQrr?=
 =?us-ascii?Q?BE3XTabeMj1bIOU8+Zr2RagBDABQeQfoWg0aKn2268MkVxhBy2Z6XCwRifae?=
 =?us-ascii?Q?eGP0yEmchOwdHGrFRn0OmrA81kLM2WRN6srcLYQZdYGN4yU2mNsPwd7dgdjv?=
 =?us-ascii?Q?+Het2vRYmOwMNe1R5wvyf3jalMq2Eejou5C5sByFV2lrBPaEZ7ytLZicH+fF?=
 =?us-ascii?Q?6S3+wDI+wXFIQ5CGUUFuGjpjrX6GoAoc0KTpL9hG75LcshX+nPWU6QX6AXId?=
 =?us-ascii?Q?tKtrqReq1SbI98wu+QcbVJGQELkrtaOCgqqpenFdG3KPcit5Yqmjk6J/ak6E?=
 =?us-ascii?Q?BOhvnWVdyQFQfkPRT2KbGl0GQp5t9pmDi++J3avXfzZTM963g7vMGUhY243R?=
 =?us-ascii?Q?EsgIe2jjx0gNZYi4jJn/tq7tUKgqH+w9p3b3kMDTVa1fw2VVqAKcK0M0/fez?=
 =?us-ascii?Q?noRyo6oXFGErjSLwdyFOVC3qbHkOmgU4335ovXkmx7Ee0XC1xlRGgbgGbPsI?=
 =?us-ascii?Q?GBS2P8hOxCfO1uaKODAm3DH7TWNBuUVscUpJszTMVQ=3D=3D?=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <AF6D4881E5DB1E4CBB3FF0E9772257BC@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8681
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU6PEPF00009528.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a0c55e5b-a51a-42ca-efe3-08dc817df713
X-Microsoft-Antispam:
	BCL:0;ARA:13230031|1800799015|376005|36860700004|35042699013|82310400017;
X-Microsoft-Antispam-Message-Info:
	=?us-ascii?Q?q2m/3PZghvBmI43vetMRmQtX6KDV/QxagpUFEvPiD+JppaomMH/ZZz+k+a6l?=
 =?us-ascii?Q?V1Rc5ycqUK+e+XY1B8kuqZ6KhcCOqoL9AgqtyEk4mzF8yk9/lnlIDelVvEXC?=
 =?us-ascii?Q?2p5t0fmY+8QXQjkD0oDN2oSEcqzt3eMs/zTkbmtMuOpa7k3teIUKcqEbBnmO?=
 =?us-ascii?Q?ei0RDa50PJIC/dOJyEjGY8rLnwV6sqMbqteQu0VDEwHrWB1l9hMsdQoEd9eP?=
 =?us-ascii?Q?b8/7LBRU5mX9as228AEABfroo+H/T1R1IBIVlsQ212MvdlwujZAdXwwlyTTd?=
 =?us-ascii?Q?oOZC2rRPbbOqVrcxUBNXz/oZ2FmtWAotfojeRd/dYiil7MLCPSKyU+nvRH7A?=
 =?us-ascii?Q?Rnz91DswrJAwx7fO9rUPofAObQqlkXOZcZ7fido6bVD0oC+Wf5Q5xqX0Iin4?=
 =?us-ascii?Q?QSQkl3TzJXHzj0SScWwSgo1O9yh6EyB6nEncDxCEDA6pEpxKoL42UTa9rUqK?=
 =?us-ascii?Q?rJauL1HEzRyo9xihz1LeUUTSVQykAKSE6XK+G9Z3VQVF1YzD1ID49J0dJeCv?=
 =?us-ascii?Q?2m4+AZAXvQZ1Wv9OLzU5dbZbCQs4PzXvNhdJtaa5O+lKVcQN9KuA7JJxaS/B?=
 =?us-ascii?Q?QECUlj6tNyVn8w559FMLpFuFUDgrhhsQJLVHfw0O/GNUG2+m/d+Zfp1hVnTN?=
 =?us-ascii?Q?PmeerURmbpFgM2KstdpUwpvS4ZhL5Pel1qlUo+HQysLsWPXB10ZTCKb5Nan/?=
 =?us-ascii?Q?ES1FOwNVO/ni1wjk7ZrvJJAcuHK+VZfouVmPg4sEuix/xlfr4ajkPLlAHI47?=
 =?us-ascii?Q?Xg2J7MrtnoP4645PdzT9qEGRV9wMIFZZxRtdXFV67vJVSG1ScJiXKkuX5MpV?=
 =?us-ascii?Q?+eb+pcviHuq51f+6IhwfdZ/QSa3Rf5C1+rannQGzzQ5c7toALJ4yMtBrWRGs?=
 =?us-ascii?Q?iglMnpk5PcC4Y/8agLj1bFvww+JmuN4htO6lJIg1EDpX30rgFjI6f+3X2XF2?=
 =?us-ascii?Q?o38WlxKYKC3kHP4ZIpP0CnDYcE2bghXKP0QZUx74J2XA7LbryqJP26gnf2a5?=
 =?us-ascii?Q?VAbCpEljPE94wqOQdxCQveizkgI8FBeOxWb95fV/ktYxbI24/dS+l25LIvnL?=
 =?us-ascii?Q?2/fUFhHPuUEDE04dmRZv5CLnmAt7eavSuNY9xneiZZ8R/q6z2fXID8AXmHsg?=
 =?us-ascii?Q?993HHaEJe1F5eA29/RmVm8lm7K0x/w5yIF1x/tjyOSjZrhYz8aOW/QG5KTbx?=
 =?us-ascii?Q?RRvaKXlUwvOUoBY7joMtnYA3O+7HzNUSscJo+FUgQrcWTOEehAZz+NCvIPTR?=
 =?us-ascii?Q?zAPYw+ImlHvBlv/GjnnkTWvAvaju0kIR+pPyfC1pSy5XrCz1z8W328rkwxyK?=
 =?us-ascii?Q?8QZre7pqZ4xxSoEQj5SBCuMf?=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(35042699013)(82310400017);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 14:28:48.0501
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 81361f87-126e-4594-38bc-08dc817dfc1e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU6PEPF00009528.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR08MB10333

Hi Jens,

> On 29 May 2024, at 09:25, Jens Wiklander <jens.wiklander@linaro.org> wrot=
e:
>=20
> Add support for FF-A notifications, currently limited to an SP (Secure
> Partition) sending an asynchronous notification to a guest.
>=20
> Guests and Xen itself are made aware of pending notifications with an
> interrupt. The interrupt handler triggers a tasklet to retrieve the
> notifications using the FF-A ABI and deliver them to their destinations.
>=20
> Update ffa_partinfo_domain_init() to return error code like
> ffa_notif_domain_init().
>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
> ---
> v4->v5:
> - Move the freeing of d->arch.tee to the new TEE mediator free_domain_ctx
>  callback to make the context accessible during rcu_lock_domain_by_id()
>  from a tasklet
> - Initialize interrupt handlers for secondary CPUs from the new TEE media=
tor
>  init_interrupt() callback
> - Restore the ffa_probe() from v3, that is, remove the
>  presmp_initcall(ffa_init) approach and use ffa_probe() as usual now that=
 we
>  have the init_interrupt callback.
> - A tasklet is added to handle the Schedule Receiver interrupt. The taskl=
et
>  finds each relevant domain with rcu_lock_domain_by_id() which now is eno=
ugh
>  to guarantee that the FF-A context can be accessed.
> - The notification interrupt handler only schedules the notification
>  tasklet mentioned above
>=20
> v3->v4:
> - Add another note on FF-A limitations
> - Clear secure_pending in ffa_handle_notification_get() if both SP and SP=
M
>  bitmaps are retrieved
> - ASSERT that ffa_rcu_lock_domain_by_vm_id() isn't passed the vm_id FF-A
>  uses for Xen itself
> - Replace the get_domain_by_id() call done via ffa_get_domain_by_vm_id() =
in
>  notif_irq_handler() with a call to rcu_lock_live_remote_domain_by_id() v=
ia
>  ffa_rcu_lock_domain_by_vm_id()
> - Remove spinlock in struct ffa_ctx_notif and use atomic functions as nee=
ded
>  to access and update the secure_pending field
> - In notif_irq_handler(), look for the first online CPU instead of assumi=
ng
>  that the first CPU is online
> - Initialize FF-A via presmp_initcall() before the other CPUs are online,
>  use register_cpu_notifier() to install the interrupt handler
>  notif_irq_handler()
> - Update commit message to reflect recent updates
>=20
> v2->v3:
> - Add a GUEST_ prefix and move FFA_NOTIF_PEND_INTR_ID and
>  FFA_SCHEDULE_RECV_INTR_ID to public/arch-arm.h
> - Register the Xen SRI handler on each CPU using on_selected_cpus() and
>  setup_irq()
> - Check that the SGI ID retrieved with FFA_FEATURE_SCHEDULE_RECV_INTR
>  doesn't conflict with static SGI handlers
>=20
> v1->v2:
> - Addressing review comments
> - Change ffa_handle_notification_{bind,unbind,set}() to take struct
>  cpu_user_regs *regs as argument.
> - Update ffa_partinfo_domain_init() and ffa_notif_domain_init() to return
>  an error code.
> - Fixing a bug in handle_features() for FFA_FEATURE_SCHEDULE_RECV_INTR.
> ---
> xen/arch/arm/tee/Makefile       |   1 +
> xen/arch/arm/tee/ffa.c          |  72 +++++-
> xen/arch/arm/tee/ffa_notif.c    | 409 ++++++++++++++++++++++++++++++++
> xen/arch/arm/tee/ffa_partinfo.c |   9 +-
> xen/arch/arm/tee/ffa_private.h  |  56 ++++-
> xen/arch/arm/tee/tee.c          |   2 +-
> xen/include/public/arch-arm.h   |  14 ++
> 7 files changed, 548 insertions(+), 15 deletions(-)
> create mode 100644 xen/arch/arm/tee/ffa_notif.c
>=20
> diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
> index f0112a2f922d..7c0f46f7f446 100644
> --- a/xen/arch/arm/tee/Makefile
> +++ b/xen/arch/arm/tee/Makefile
> @@ -2,5 +2,6 @@ obj-$(CONFIG_FFA) +=3D ffa.o
> obj-$(CONFIG_FFA) +=3D ffa_shm.o
> obj-$(CONFIG_FFA) +=3D ffa_partinfo.o
> obj-$(CONFIG_FFA) +=3D ffa_rxtx.o
> +obj-$(CONFIG_FFA) +=3D ffa_notif.o
> obj-y +=3D tee.o
> obj-$(CONFIG_OPTEE) +=3D optee.o
> diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> index 5209612963e1..d644ee97cd5b 100644
> --- a/xen/arch/arm/tee/ffa.c
> +++ b/xen/arch/arm/tee/ffa.c
> @@ -39,6 +39,12 @@
>  *   - at most 32 shared memory regions per guest
>  * o FFA_MSG_SEND_DIRECT_REQ:
>  *   - only supported from a VM to an SP
> + * o FFA_NOTIFICATION_*:
> + *   - only supports global notifications, that is, per vCPU notificatio=
ns
> + *     are not supported
> + *   - doesn't support signalling the secondary scheduler of pending
> + *     notification for secure partitions
> + *   - doesn't support notifications for Xen itself
>  *
>  * There are some large locked sections with ffa_tx_buffer_lock and
>  * ffa_rx_buffer_lock. Especially the ffa_tx_buffer_lock spinlock used
> @@ -194,6 +200,8 @@ out:
>=20
> static void handle_features(struct cpu_user_regs *regs)
> {
> +    struct domain *d =3D current->domain;
> +    struct ffa_ctx *ctx =3D d->arch.tee;
>     uint32_t a1 =3D get_user_reg(regs, 1);
>     unsigned int n;
>=20
> @@ -240,6 +248,30 @@ static void handle_features(struct cpu_user_regs *re=
gs)
>         BUILD_BUG_ON(PAGE_SIZE !=3D FFA_PAGE_SIZE);
>         ffa_set_regs_success(regs, 0, 0);
>         break;
> +    case FFA_FEATURE_NOTIF_PEND_INTR:
> +        if ( ctx->notif.enabled )
> +            ffa_set_regs_success(regs, GUEST_FFA_NOTIF_PEND_INTR_ID, 0);
> +        else
> +            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
> +        break;
> +    case FFA_FEATURE_SCHEDULE_RECV_INTR:
> +        if ( ctx->notif.enabled )
> +            ffa_set_regs_success(regs, GUEST_FFA_SCHEDULE_RECV_INTR_ID, =
0);
> +        else
> +            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
> +        break;
> +
> +    case FFA_NOTIFICATION_BIND:
> +    case FFA_NOTIFICATION_UNBIND:
> +    case FFA_NOTIFICATION_GET:
> +    case FFA_NOTIFICATION_SET:
> +    case FFA_NOTIFICATION_INFO_GET_32:
> +    case FFA_NOTIFICATION_INFO_GET_64:
> +        if ( ctx->notif.enabled )
> +            ffa_set_regs_success(regs, 0, 0);
> +        else
> +            ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
> +        break;
>     default:
>         ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
>         break;
> @@ -305,6 +337,22 @@ static bool ffa_handle_call(struct cpu_user_regs *re=
gs)
>                                                      get_user_reg(regs, 1=
)),
>                                    get_user_reg(regs, 3));
>         break;
> +    case FFA_NOTIFICATION_BIND:
> +        e =3D ffa_handle_notification_bind(regs);
> +        break;
> +    case FFA_NOTIFICATION_UNBIND:
> +        e =3D ffa_handle_notification_unbind(regs);
> +        break;
> +    case FFA_NOTIFICATION_INFO_GET_32:
> +    case FFA_NOTIFICATION_INFO_GET_64:
> +        ffa_handle_notification_info_get(regs);
> +        return true;
> +    case FFA_NOTIFICATION_GET:
> +        ffa_handle_notification_get(regs);
> +        return true;
> +    case FFA_NOTIFICATION_SET:
> +        e =3D ffa_handle_notification_set(regs);
> +        break;
>=20
>     default:
>         gprintk(XENLOG_ERR, "ffa: unhandled fid 0x%x\n", fid);
> @@ -322,6 +370,7 @@ static bool ffa_handle_call(struct cpu_user_regs *reg=
s)
> static int ffa_domain_init(struct domain *d)
> {
>     struct ffa_ctx *ctx;
> +    int ret;
>=20
>     if ( !ffa_version )
>         return -ENODEV;
> @@ -345,10 +394,11 @@ static int ffa_domain_init(struct domain *d)
>      * error, so no need for cleanup in this function.
>      */
>=20
> -    if ( !ffa_partinfo_domain_init(d) )
> -        return -EIO;
> +    ret =3D ffa_partinfo_domain_init(d);
> +    if ( ret )
> +        return ret;
>=20
> -    return 0;
> +    return ffa_notif_domain_init(d);
> }
>=20
> static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_=
time)
> @@ -376,13 +426,6 @@ static void ffa_domain_teardown_continue(struct ffa_=
ctx *ctx, bool first_time)
>     }
>     else
>     {
> -        /*
> -         * domain_destroy() might have been called (via put_domain() in
> -         * ffa_reclaim_shms()), so we can't touch the domain structure
> -         * anymore.
> -         */
> -        xfree(ctx);
> -
>         /* Only check if there has been a change to the teardown queue */
>         if ( !first_time )
>         {
> @@ -423,12 +466,18 @@ static int ffa_domain_teardown(struct domain *d)
>         return 0;
>=20
>     ffa_rxtx_domain_destroy(d);
> +    ffa_notif_domain_destroy(d);
>=20
>     ffa_domain_teardown_continue(ctx, true /* first_time */);
>=20
>     return 0;
> }
>=20
> +static void ffa_free_domain_ctx(struct domain *d)
> +{
> +    XFREE(d->arch.tee);
> +}
> +
> static int ffa_relinquish_resources(struct domain *d)
> {
>     return 0;
> @@ -502,6 +551,7 @@ static bool ffa_probe(void)
>     if ( !ffa_partinfo_init() )
>         goto err_rxtx_destroy;
>=20
> +    ffa_notif_init();
>     INIT_LIST_HEAD(&ffa_teardown_head);
>     init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0)=
;
>=20
> @@ -517,8 +567,10 @@ err_rxtx_destroy:
> static const struct tee_mediator_ops ffa_ops =3D
> {
>     .probe =3D ffa_probe,
> +    .init_interrupt =3D ffa_notif_init_interrupt,

With the previous change on init secondary i would suggest to
have a ffa_init_secondary here actually calling the ffa_notif_init_interrup=
t.

>     .domain_init =3D ffa_domain_init,
>     .domain_teardown =3D ffa_domain_teardown,
> +    .free_domain_ctx =3D ffa_free_domain_ctx,
>     .relinquish_resources =3D ffa_relinquish_resources,
>     .handle_call =3D ffa_handle_call,
> };
> diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c
> new file mode 100644
> index 000000000000..e8e8b62590b3
> --- /dev/null
> +++ b/xen/arch/arm/tee/ffa_notif.c
> @@ -0,0 +1,409 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2024  Linaro Limited
> + */
> +
> +#include <xen/const.h>
> +#include <xen/cpu.h>
> +#include <xen/list.h>
> +#include <xen/notifier.h>
> +#include <xen/spinlock.h>
> +#include <xen/tasklet.h>
> +#include <xen/types.h>
> +
> +#include <asm/smccc.h>
> +#include <asm/regs.h>
> +
> +#include "ffa_private.h"
> +
> +static bool __ro_after_init notif_enabled;
> +static unsigned int __ro_after_init notif_sri_irq;
> +
> +int ffa_handle_notification_bind(struct cpu_user_regs *regs)
> +{
> +    struct domain *d =3D current->domain;
> +    uint32_t src_dst =3D get_user_reg(regs, 1);
> +    uint32_t flags =3D get_user_reg(regs, 2);
> +    uint32_t bitmap_lo =3D get_user_reg(regs, 3);
> +    uint32_t bitmap_hi =3D get_user_reg(regs, 4);
> +
> +    if ( !notif_enabled )
> +        return FFA_RET_NOT_SUPPORTED;
> +
> +    if ( (src_dst & 0xFFFFU) !=3D ffa_get_vm_id(d) )
> +        return FFA_RET_INVALID_PARAMETERS;
> +
> +    if ( flags )    /* Only global notifications are supported */
> +        return FFA_RET_DENIED;
> +
> +    /*
> +     * We only support notifications from SP so no need to check the sen=
der
> +     * endpoint ID, the SPMC will take care of that for us.
> +     */
> +    return ffa_simple_call(FFA_NOTIFICATION_BIND, src_dst, flags, bitmap=
_hi,
> +                           bitmap_lo);
> +}
> +
> +int ffa_handle_notification_unbind(struct cpu_user_regs *regs)
> +{
> +    struct domain *d =3D current->domain;
> +    uint32_t src_dst =3D get_user_reg(regs, 1);
> +    uint32_t bitmap_lo =3D get_user_reg(regs, 3);
> +    uint32_t bitmap_hi =3D get_user_reg(regs, 4);
> +
> +    if ( !notif_enabled )
> +        return FFA_RET_NOT_SUPPORTED;
> +
> +    if ( (src_dst & 0xFFFFU) !=3D ffa_get_vm_id(d) )
> +        return FFA_RET_INVALID_PARAMETERS;
> +
> +    /*
> +     * We only support notifications from SP so no need to check the
> +     * destination endpoint ID, the SPMC will take care of that for us.
> +     */
> +    return  ffa_simple_call(FFA_NOTIFICATION_UNBIND, src_dst, 0, bitmap_=
hi,
> +                            bitmap_lo);
> +}
> +
> +void ffa_handle_notification_info_get(struct cpu_user_regs *regs)
> +{
> +    struct domain *d =3D current->domain;
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
> +    if ( !notif_enabled )
> +    {
> +        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
> +        return;
> +    }
> +
> +    if ( test_and_clear_bool(ctx->notif.secure_pending) )
> +    {
> +        /* A pending global notification for the guest */
> +        ffa_set_regs(regs, FFA_SUCCESS_64, 0,
> +                     1U << FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT, ffa_get_vm=
_id(d),
> +                     0, 0, 0, 0);
> +    }
> +    else
> +    {
> +        /* Report an error if there where no pending global notification=
 */
> +        ffa_set_regs_error(regs, FFA_RET_NO_DATA);
> +    }
> +}
> +
> +void ffa_handle_notification_get(struct cpu_user_regs *regs)
> +{
> +    struct domain *d =3D current->domain;
> +    uint32_t recv =3D get_user_reg(regs, 1);
> +    uint32_t flags =3D get_user_reg(regs, 2);
> +    uint32_t w2 =3D 0;
> +    uint32_t w3 =3D 0;
> +    uint32_t w4 =3D 0;
> +    uint32_t w5 =3D 0;
> +    uint32_t w6 =3D 0;
> +    uint32_t w7 =3D 0;
> +
> +    if ( !notif_enabled )
> +    {
> +        ffa_set_regs_error(regs, FFA_RET_NOT_SUPPORTED);
> +        return;
> +    }
> +
> +    if ( (recv & 0xFFFFU) !=3D ffa_get_vm_id(d) )
> +    {
> +        ffa_set_regs_error(regs, FFA_RET_INVALID_PARAMETERS);
> +        return;
> +    }
> +
> +    if ( flags & ( FFA_NOTIF_FLAG_BITMAP_SP | FFA_NOTIF_FLAG_BITMAP_SPM =
) )
> +    {
> +        struct arm_smccc_1_2_regs arg =3D {
> +            .a0 =3D FFA_NOTIFICATION_GET,
> +            .a1 =3D recv,
> +            .a2 =3D flags & ( FFA_NOTIF_FLAG_BITMAP_SP |
> +                            FFA_NOTIF_FLAG_BITMAP_SPM ),
> +        };
> +        struct arm_smccc_1_2_regs resp;
> +        int32_t e;
> +
> +        /*
> +         * Clear secure pending if both FFA_NOTIF_FLAG_BITMAP_SP and
> +         * FFA_NOTIF_FLAG_BITMAP_SPM are set since secure world can't ha=
ve
> +         * any more pending notifications.
> +         */
> +        if ( ( flags  & FFA_NOTIF_FLAG_BITMAP_SP ) &&
> +             ( flags & FFA_NOTIF_FLAG_BITMAP_SPM ) )
> +        {
> +                struct ffa_ctx *ctx =3D d->arch.tee;
> +
> +                ACCESS_ONCE(ctx->notif.secure_pending) =3D false;
> +        }
> +
> +        arm_smccc_1_2_smc(&arg, &resp);
> +        e =3D ffa_get_ret_code(&resp);
> +        if ( e )
> +        {
> +            ffa_set_regs_error(regs, e);
> +            return;
> +        }
> +
> +        if ( flags & FFA_NOTIF_FLAG_BITMAP_SP )
> +        {
> +            w2 =3D resp.a2;
> +            w3 =3D resp.a3;
> +        }
> +
> +        if ( flags & FFA_NOTIF_FLAG_BITMAP_SPM )
> +            w6 =3D resp.a6;
> +    }
> +
> +    ffa_set_regs(regs, FFA_SUCCESS_32, 0, w2, w3, w4, w5, w6, w7);
> +}
> +
> +int ffa_handle_notification_set(struct cpu_user_regs *regs)
> +{
> +    struct domain *d =3D current->domain;
> +    uint32_t src_dst =3D get_user_reg(regs, 1);
> +    uint32_t flags =3D get_user_reg(regs, 2);
> +    uint32_t bitmap_lo =3D get_user_reg(regs, 3);
> +    uint32_t bitmap_hi =3D get_user_reg(regs, 4);
> +
> +    if ( !notif_enabled )
> +        return FFA_RET_NOT_SUPPORTED;
> +
> +    if ( (src_dst >> 16) !=3D ffa_get_vm_id(d) )
> +        return FFA_RET_INVALID_PARAMETERS;
> +
> +    /* Let the SPMC check the destination of the notification */
> +    return ffa_simple_call(FFA_NOTIFICATION_SET, src_dst, flags, bitmap_=
lo,
> +                           bitmap_hi);
> +}
> +
> +/*
> + * Extract a 16-bit ID (index n) from the successful return value from
> + * FFA_NOTIFICATION_INFO_GET_64 or FFA_NOTIFICATION_INFO_GET_32. IDs are
> + * returned in registers 3 to 7 with four IDs per register for 64-bit
> + * calling convention and two IDs per register for 32-bit calling
> + * convention.
> + */
> +static uint16_t get_id_from_resp(struct arm_smccc_1_2_regs *resp,
> +                                 unsigned int n)
> +{
> +    unsigned int ids_per_reg;
> +    unsigned int reg_idx;
> +    unsigned int reg_shift;
> +
> +    if ( smccc_is_conv_64(resp->a0) )
> +        ids_per_reg =3D 4;
> +    else
> +        ids_per_reg =3D 2;
> +
> +    reg_idx =3D n / ids_per_reg + 3;
> +    reg_shift =3D ( n % ids_per_reg ) * 16;
> +
> +    switch ( reg_idx )
> +    {
> +    case 3:
> +        return resp->a3 >> reg_shift;
> +    case 4:
> +        return resp->a4 >> reg_shift;
> +    case 5:
> +        return resp->a5 >> reg_shift;
> +    case 6:
> +        return resp->a6 >> reg_shift;
> +    case 7:
> +        return resp->a7 >> reg_shift;
> +    default:
> +        ASSERT(0); /* "Can't happen" */
> +        return 0;
> +    }
> +}
> +
> +static void notif_vm_pend_intr(uint16_t vm_id)
> +{
> +    struct ffa_ctx *ctx;
> +    struct domain *d;
> +    struct vcpu *v;
> +
> +    /*
> +     * vm_id =3D=3D 0 means a notifications pending for Xen itself, but
> +     * we don't support that yet.
> +     */
> +    if ( !vm_id )
> +        return;
> +
> +    d =3D ffa_rcu_lock_domain_by_vm_id(vm_id);
> +    if ( !d )
> +        return;
> +
> +    ctx =3D d->arch.tee;
> +    if ( !ctx )
> +        goto out_unlock;

In both previous cases you are silently ignoring an interrupt
due to an internal error.
Is this something that we should trace ? maybe just debug ?

Could you add a comment to explain why this could happen
(when possible) or not and the possible side effects ?

The second one would be a notification for a domain without
FF-A enabled which is ok but i am a bit more wondering on
the first one.

> +
> +    /*
> +     * arch.tee is freed from complete_domain_destroy() so the RCU lock
> +     * guarantees that the data structure isn't freed while we're access=
ing
> +     * it.
> +     */
> +    ACCESS_ONCE(ctx->notif.secure_pending) =3D true;
> +
> +    /*
> +     * Since we're only delivering global notification, always
> +     * deliver to the first online vCPU. It doesn't matter
> +     * which we chose, as long as it's available.
> +     */
> +    for_each_vcpu(d, v)
> +    {
> +        if ( is_vcpu_online(v) )
> +        {
> +            vgic_inject_irq(d, v, GUEST_FFA_NOTIF_PEND_INTR_ID,
> +                            true);
> +            break;
> +        }
> +    }
> +    if ( !v )
> +        printk(XENLOG_ERR "ffa: can't inject NPI, all vCPUs offline\n");
> +
> +out_unlock:
> +    rcu_unlock_domain(d);
> +}
> +
> +static void notif_sri_action(void *unused)
> +{
> +    const struct arm_smccc_1_2_regs arg =3D {
> +        .a0 =3D FFA_NOTIFICATION_INFO_GET_64,
> +    };
> +    struct arm_smccc_1_2_regs resp;
> +    unsigned int id_pos;
> +    unsigned int list_count;
> +    uint64_t ids_count;
> +    unsigned int n;
> +    int32_t res;
> +
> +    do {
> +        arm_smccc_1_2_smc(&arg, &resp);
> +        res =3D ffa_get_ret_code(&resp);
> +        if ( res )
> +        {
> +            if ( res !=3D FFA_RET_NO_DATA )
> +                printk(XENLOG_ERR "ffa: notification info get failed: er=
ror %d\n",
> +                       res);
> +            return;
> +        }
> +
> +        ids_count =3D resp.a2 >> FFA_NOTIF_INFO_GET_ID_LIST_SHIFT;
> +        list_count =3D ( resp.a2 >> FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT ) =
&
> +                     FFA_NOTIF_INFO_GET_ID_COUNT_MASK;
> +
> +        id_pos =3D 0;
> +        for ( n =3D 0; n < list_count; n++ )
> +        {
> +            unsigned int count =3D ((ids_count >> 2 * n) & 0x3) + 1;
> +            uint16_t vm_id =3D get_id_from_resp(&resp, id_pos);
> +
> +            notif_vm_pend_intr(vm_id);
> +
> +            id_pos +=3D count;
> +        }
> +
> +    } while (resp.a2 & FFA_NOTIF_INFO_GET_MORE_FLAG);
> +}
> +
> +static DECLARE_TASKLET(notif_sri_tasklet, notif_sri_action, NULL);
> +
> +static void notif_irq_handler(int irq, void *data)
> +{
> +    tasklet_schedule(&notif_sri_tasklet);
> +}
> +
> +static int32_t ffa_notification_bitmap_create(uint16_t vm_id,
> +                                              uint32_t vcpu_count)
> +{
> +    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_CREATE, vm_id, vcpu_c=
ount,
> +                           0, 0);
> +}
> +
> +static int32_t ffa_notification_bitmap_destroy(uint16_t vm_id)
> +{
> +    return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0,=
 0);
> +}
> +
> +void ffa_notif_init_interrupt(void)
> +{
> +    int ret;
> +
> +    if ( notif_enabled && notif_sri_irq < NR_GIC_SGI )
> +    {
> +        /*
> +         * An error here is unlikely since the primary CPU has already
> +         * succeeded in installing the interrupt handler. If this fails =
it
> +         * may lead to a problem with notifictaions.
> +         *
> +         * The CPUs without an notification handler installed will fail =
to
> +         * trigger on the SGI indicating that there are notifications
> +         * pending, while the SPMC in the secure world will not notice t=
hat
> +         * the interrupt was lost.
> +         */
> +        ret =3D request_irq(notif_sri_irq, 0, notif_irq_handler, "FF-A n=
otif",
> +                          NULL);
> +        if ( ret )
> +            printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\=
n",
> +                   notif_sri_irq, ret);
> +    }
> +}
> +
> +void ffa_notif_init(void)
> +{
> +    const struct arm_smccc_1_2_regs arg =3D {
> +        .a0 =3D FFA_FEATURES,
> +        .a1 =3D FFA_FEATURE_SCHEDULE_RECV_INTR,
> +    };
> +    struct arm_smccc_1_2_regs resp;
> +    unsigned int irq;
> +    int ret;
> +
> +    arm_smccc_1_2_smc(&arg, &resp);
> +    if ( resp.a0 !=3D FFA_SUCCESS_32 )
> +        return;
> +
> +    irq =3D resp.a2;
> +    notif_sri_irq =3D irq;
> +    if ( irq >=3D NR_GIC_SGI )
> +        irq_set_type(irq, IRQ_TYPE_EDGE_RISING);
> +    ret =3D request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL);
> +    if ( ret )
> +    {
> +        printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n",
> +               irq, ret);
> +        return;
> +    }
> +
> +    notif_enabled =3D true;
> +}
> +
> +int ffa_notif_domain_init(struct domain *d)
> +{
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +    int32_t res;
> +
> +    if ( !notif_enabled )
> +        return 0;
> +
> +    res =3D ffa_notification_bitmap_create(ffa_get_vm_id(d), d->max_vcpu=
s);
> +    if ( res )
> +        return -ENOMEM;
> +
> +    ctx->notif.enabled =3D true;
> +
> +    return 0;
> +}
> +
> +void ffa_notif_domain_destroy(struct domain *d)
> +{
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
> +    if ( ctx->notif.enabled )
> +    {
> +        ffa_notification_bitmap_destroy(ffa_get_vm_id(d));
> +        ctx->notif.enabled =3D false;
> +    }
> +}
> diff --git a/xen/arch/arm/tee/ffa_partinfo.c b/xen/arch/arm/tee/ffa_parti=
nfo.c
> index dc1059584828..93a03c6bc672 100644
> --- a/xen/arch/arm/tee/ffa_partinfo.c
> +++ b/xen/arch/arm/tee/ffa_partinfo.c
> @@ -306,7 +306,7 @@ static void vm_destroy_bitmap_init(struct ffa_ctx *ct=
x,
>     }
> }
>=20
> -bool ffa_partinfo_domain_init(struct domain *d)
> +int ffa_partinfo_domain_init(struct domain *d)
> {
>     unsigned int count =3D BITS_TO_LONGS(subscr_vm_destroyed_count);
>     struct ffa_ctx *ctx =3D d->arch.tee;
> @@ -315,7 +315,7 @@ bool ffa_partinfo_domain_init(struct domain *d)
>=20
>     ctx->vm_destroy_bitmap =3D xzalloc_array(unsigned long, count);
>     if ( !ctx->vm_destroy_bitmap )
> -        return false;
> +        return -ENOMEM;
>=20
>     for ( n =3D 0; n < subscr_vm_created_count; n++ )
>     {
> @@ -330,7 +330,10 @@ bool ffa_partinfo_domain_init(struct domain *d)
>     }
>     vm_destroy_bitmap_init(ctx, n);
>=20
> -    return n =3D=3D subscr_vm_created_count;
> +    if ( n !=3D subscr_vm_created_count )
> +        return -EIO;
> +
> +    return 0;
> }
>=20
> bool ffa_partinfo_domain_destroy(struct domain *d)
> diff --git a/xen/arch/arm/tee/ffa_private.h b/xen/arch/arm/tee/ffa_privat=
e.h
> index 98236cbf14a3..7c6b06f686fc 100644
> --- a/xen/arch/arm/tee/ffa_private.h
> +++ b/xen/arch/arm/tee/ffa_private.h
> @@ -25,6 +25,7 @@
> #define FFA_RET_DENIED                  -6
> #define FFA_RET_RETRY                   -7
> #define FFA_RET_ABORTED                 -8
> +#define FFA_RET_NO_DATA                 -9
>=20
> /* FFA_VERSION helpers */
> #define FFA_VERSION_MAJOR_SHIFT         16U
> @@ -175,6 +176,21 @@
>  */
> #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U)
>=20
> +/* Flags used in calls to FFA_NOTIFICATION_GET interface  */
> +#define FFA_NOTIF_FLAG_BITMAP_SP        BIT(0, U)
> +#define FFA_NOTIF_FLAG_BITMAP_VM        BIT(1, U)
> +#define FFA_NOTIF_FLAG_BITMAP_SPM       BIT(2, U)
> +#define FFA_NOTIF_FLAG_BITMAP_HYP       BIT(3, U)
> +
> +#define FFA_NOTIF_INFO_GET_MORE_FLAG        BIT(0, U)
> +#define FFA_NOTIF_INFO_GET_ID_LIST_SHIFT    12
> +#define FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT   7
> +#define FFA_NOTIF_INFO_GET_ID_COUNT_MASK    0x1F
> +
> +/* Feature IDs used with FFA_FEATURES */
> +#define FFA_FEATURE_NOTIF_PEND_INTR     0x1U
> +#define FFA_FEATURE_SCHEDULE_RECV_INTR  0x2U
> +
> /* Function IDs */
> #define FFA_ERROR                       0x84000060U
> #define FFA_SUCCESS_32                  0x84000061U
> @@ -213,6 +229,24 @@
> #define FFA_MEM_FRAG_TX                 0x8400007BU
> #define FFA_MSG_SEND                    0x8400006EU
> #define FFA_MSG_POLL                    0x8400006AU
> +#define FFA_NOTIFICATION_BITMAP_CREATE  0x8400007DU
> +#define FFA_NOTIFICATION_BITMAP_DESTROY 0x8400007EU
> +#define FFA_NOTIFICATION_BIND           0x8400007FU
> +#define FFA_NOTIFICATION_UNBIND         0x84000080U
> +#define FFA_NOTIFICATION_SET            0x84000081U
> +#define FFA_NOTIFICATION_GET            0x84000082U
> +#define FFA_NOTIFICATION_INFO_GET_32    0x84000083U
> +#define FFA_NOTIFICATION_INFO_GET_64    0xC4000083U
> +
> +struct ffa_ctx_notif {
> +    bool enabled;
> +
> +    /*
> +     * True if domain is reported by FFA_NOTIFICATION_INFO_GET to have
> +     * pending global notifications.
> +     */
> +    bool secure_pending;
> +};
>=20
> struct ffa_ctx {
>     void *rx;
> @@ -228,6 +262,7 @@ struct ffa_ctx {
>     struct list_head shm_list;
>     /* Number of allocated shared memory object */
>     unsigned int shm_count;
> +    struct ffa_ctx_notif notif;
>     /*
>      * tx_lock is used to serialize access to tx
>      * rx_lock is used to serialize access to rx
> @@ -257,7 +292,7 @@ void ffa_handle_mem_share(struct cpu_user_regs *regs)=
;
> int ffa_handle_mem_reclaim(uint64_t handle, uint32_t flags);
>=20
> bool ffa_partinfo_init(void);
> -bool ffa_partinfo_domain_init(struct domain *d);
> +int ffa_partinfo_domain_init(struct domain *d);
> bool ffa_partinfo_domain_destroy(struct domain *d);
> int32_t ffa_handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t =
w3,
>                                       uint32_t w4, uint32_t w5, uint32_t =
*count,
> @@ -271,12 +306,31 @@ uint32_t ffa_handle_rxtx_map(uint32_t fid, register=
_t tx_addr,
> uint32_t ffa_handle_rxtx_unmap(void);
> int32_t ffa_handle_rx_release(void);
>=20
> +void ffa_notif_init(void);
> +void ffa_notif_init_interrupt(void);
> +int ffa_notif_domain_init(struct domain *d);
> +void ffa_notif_domain_destroy(struct domain *d);
> +
> +int ffa_handle_notification_bind(struct cpu_user_regs *regs);
> +int ffa_handle_notification_unbind(struct cpu_user_regs *regs);
> +void ffa_handle_notification_info_get(struct cpu_user_regs *regs);
> +void ffa_handle_notification_get(struct cpu_user_regs *regs);
> +int ffa_handle_notification_set(struct cpu_user_regs *regs);
> +
> static inline uint16_t ffa_get_vm_id(const struct domain *d)
> {
>     /* +1 since 0 is reserved for the hypervisor in FF-A */
>     return d->domain_id + 1;
> }
>=20
> +static inline struct domain *ffa_rcu_lock_domain_by_vm_id(uint16_t vm_id=
)
> +{
> +    ASSERT(vm_id);
> +
> +    /* -1 to match ffa_get_vm_id() */
> +    return rcu_lock_domain_by_id(vm_id - 1);
> +}
> +
> static inline void ffa_set_regs(struct cpu_user_regs *regs, register_t v0=
,
>                                 register_t v1, register_t v2, register_t =
v3,
>                                 register_t v4, register_t v5, register_t =
v6,
> diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
> index cb65f187f51f..1e890ecc2932 100644
> --- a/xen/arch/arm/tee/tee.c
> +++ b/xen/arch/arm/tee/tee.c
> @@ -94,7 +94,7 @@ static int __init tee_init(void)
>     return 0;
> }
>=20
> -__initcall(tee_init);
> +presmp_initcall(tee_init);
>=20
> void __init init_tee_interrupt(void)
> {
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.=
h
> index 289af81bd69d..e2412a17474e 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -505,6 +505,7 @@ typedef uint64_t xen_callback_t;
> #define GUEST_MAX_VCPUS 128
>=20
> /* Interrupts */
> +
> #define GUEST_TIMER_VIRT_PPI    27
> #define GUEST_TIMER_PHYS_S_PPI  29
> #define GUEST_TIMER_PHYS_NS_PPI 30
> @@ -515,6 +516,19 @@ typedef uint64_t xen_callback_t;
> #define GUEST_VIRTIO_MMIO_SPI_FIRST   33
> #define GUEST_VIRTIO_MMIO_SPI_LAST    43
>=20
> +/*
> + * SGI is the preferred delivery mechanism of FF-A pending notifications=
 or
> + * schedule recveive interrupt. SGIs 8-15 are normally not used by a gue=
st
> + * as they in a non-virtualized system typically are assigned to the sec=
ure
> + * world. Here we're free to use SGI 8-15 since they are virtual and hav=
e
> + * nothing to do with the secure world.
> + *
> + * For partitioning of SGIs see also Arm Base System Architecture v1.0C,
> + * https://developer.arm.com/documentation/den0094/
> + */
> +#define GUEST_FFA_NOTIF_PEND_INTR_ID      8
> +#define GUEST_FFA_SCHEDULE_RECV_INTR_ID   9
> +
> /* PSCI functions */
> #define PSCI_cpu_suspend 0
> #define PSCI_cpu_off     1
> --=20
> 2.34.1
>=20

Cheers
Bertrand




From xen-devel-bounces@lists.xenproject.org Fri May 31 15:01:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 15:01:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733768.1140105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD3kv-0005mT-BO; Fri, 31 May 2024 15:01:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733768.1140105; Fri, 31 May 2024 15:01:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD3kv-0005mL-8i; Fri, 31 May 2024 15:01:21 +0000
Received: by outflank-mailman (input) for mailman id 733768;
 Fri, 31 May 2024 15:01:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD3ku-0005m8-Ph; Fri, 31 May 2024 15:01:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD3ku-0001eH-N2; Fri, 31 May 2024 15:01:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD3ku-0000Ba-9k; Fri, 31 May 2024 15:01:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sD3ku-0004br-9I; Fri, 31 May 2024 15:01:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5QEsKHQ3GUxMoMnnF4H9C9bp5LOHbnDKV/bV7VNj3Qg=; b=WHiTW7hhm2U3fylwN/RuhqWI0P
	kK6qEgVWiCgCWH2aJVJQHqPRubFsnOQVS0ot2TSAFwXfdVgjaHM9Adx52YT3h4XSgWQjQCSYZHSsD
	/v13wGAG0vwYPyX6diw1BIPQDebKjr6gDT0NYiRqT5G95jxUXAeGIp+DenVJAjv9QdWc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186208-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 186208: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1250c73c1ae2eec0308b4efe3e345127e9dbdb2b
X-Osstest-Versions-That:
    xen=9a905d7dc65883af082532b4dc91ce0131e54047
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 15:01:20 +0000

flight 186208 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186208/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 186200
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 186200
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 186200
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 186200
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 186200
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186200
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-raw      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-raw      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-qcow2    14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-qcow2    15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-vhd 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1250c73c1ae2eec0308b4efe3e345127e9dbdb2b
baseline version:
 xen                  9a905d7dc65883af082532b4dc91ce0131e54047

Last test of basis   186200  2024-05-30 18:38:49 Z    0 days
Testing same since   186208  2024-05-31 05:23:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-xl-qcow2                                    pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-amd64-amd64-xl-raw                                      pass    
 test-armhf-armhf-xl-raw                                      pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-libvirt-vhd                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9a905d7dc6..1250c73c1a  1250c73c1ae2eec0308b4efe3e345127e9dbdb2b -> master


From xen-devel-bounces@lists.xenproject.org Fri May 31 17:50:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 17:50:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733814.1140115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6O8-0007nY-0N; Fri, 31 May 2024 17:50:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733814.1140115; Fri, 31 May 2024 17:49:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6O7-0007nR-U3; Fri, 31 May 2024 17:49:59 +0000
Received: by outflank-mailman (input) for mailman id 733814;
 Fri, 31 May 2024 17:49:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bCkY=NC=epam.com=prvs=288140c18d=volodymyr_babchuk@srs-se1.protection.inumbo.net>)
 id 1sD6O6-0007nL-8X
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 17:49:58 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 301e7011-1f76-11ef-b4bb-af5377834399;
 Fri, 31 May 2024 19:49:55 +0200 (CEST)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44VDM4JX026259;
 Fri, 31 May 2024 17:49:40 GMT
Received: from eur02-db5-obe.outbound.protection.outlook.com
 (mail-db5eur02lp2104.outbound.protection.outlook.com [104.47.11.104])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3yffce8vpd-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 31 May 2024 17:49:39 +0000 (GMT)
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 (2603:10a6:150:16a::21) by DU2PR03MB9970.eurprd03.prod.outlook.com
 (2603:10a6:10:46c::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.19; Fri, 31 May
 2024 17:49:36 +0000
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e]) by GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e%3]) with mapi id 15.20.7633.001; Fri, 31 May 2024
 17:49:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 301e7011-1f76-11ef-b4bb-af5377834399
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eL+HlJy51J6ROJzPKFw/eK/1v50bmA9Aqzbir+NkX8xxWvWQbNA2EgOM6q9o9Oh0fqaxloAERE/yaDDmlzfd0N5wDv47AlZ4RdA83JwdwkbUZ2mtpud+c0E48H4wli/PgXS0kx+daM61k/j7SdGHc/0DoajRw0v1pthM0L+BORrndkCggX39myX3c369DTtsKuBh3SKt8gzVkmzeApIt2SNBJa9CbdJAJdW8UMCHe5cST33tL4zd6xFwYjfyS9/GqDJiwz3bNxtsbDv37xsoyUlvV2Ce/e7wpqupDNZataTgMmviuKqVRUg91JnbeDr80xxh4o7uuky5qhpaNAvkvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2CL11dFNt49tELnrbAYGB/LEe0N9IS3YfQ7Qsks3K5Y=;
 b=nU1X970wB78GzEyOMvlv+9C6T214iI8/y5IdenEHj97Aj6zYt0dQ+AEWu/tQ/OVZB4Yta/P3NgcnZyImHr9Ht/Avf8tTHiwc5prcZpkL5G4IOhoIq9HbssNeI/AqcaPZEzJ0vEPE1fFLca7GIkOXv+G17klt19c5muKflz+Zk/cgNucwYuFl4jNbU9Go96inPXOflqPqwP2Yj49LLj3ifgkbbSNjnDQyl1BN0RM2fxKs1qoc7r8TiO7489U/TIEZnmKohdTYfFiVHFrG8lIMb5HWslHKbVCQB3woYbX0AxUeC1MkkarVjHJMsj1lpRZvMD81GyBaBgo6TQfKSdz9Aw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2CL11dFNt49tELnrbAYGB/LEe0N9IS3YfQ7Qsks3K5Y=;
 b=X7H6PHnZAD8FF8tKgMbMxN00RI8Avhlr9LOhCGg8kMp36VlS8JTf0KNdA4D794roR02ZuQaVEycRWO2bnM2WfsPzr0l7t3c3wNkCpjYuFePISRdnWB4Z8ZI+oXuyKWaVSrzDk2tEPoBAvI5tTWZMH/dHAt4Fzs36yjhBv0IZNBFHnDTx5pLCe7nrtn6cgLowyklMYQCUdTQUnBkEId8p6an4KboYbUdDrdZcErjJqCgEMh13qO/SEenLwIn278YB79MgFJcCvfkPBi/HxV6qrZVvoUWAwBJFEm6UuxpTJWJxoJEngCLId0hwv5lcrjDfINah+DQDrXzeCD7YaCmC8w==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Michal Orzel <michal.orzel@amd.com>,
        Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [RFC PATCH v2] arm: dom0less: add TEE support
Thread-Topic: [RFC PATCH v2] arm: dom0less: add TEE support
Thread-Index: AQHas4LmA3+MKr53jE6Kc+pdYgdQGA==
Date: Fri, 31 May 2024 17:49:36 +0000
Message-ID: <20240531174915.1679443-1-volodymyr_babchuk@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-mailer: git-send-email 2.45.1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: GV1PR03MB10456:EE_|DU2PR03MB9970:EE_
x-ms-office365-filtering-correlation-id: c41824e4-2e25-47f9-cf7b-08dc819a0953
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|376005|1800799015|366007|38070700009;
x-microsoft-antispam-message-info: 
 =?iso-8859-1?Q?v6Fbscfp+joX2Gs0rp++8Vmzhj3lZ/+UycxfQ1FKkhuk8D3s99tr1dm/sn?=
 =?iso-8859-1?Q?bfJ7UdWg8A+NRAGJ6lB0PheTO5OyJhwiCZa7lhaDujmqEAcaUXuxCyBF/V?=
 =?iso-8859-1?Q?rCOFM0IJGsu2pycTe9Ex3DRPB+PhUBWAMvShd6K4/isBLC8ysv0E6ETl2n?=
 =?iso-8859-1?Q?+YTRvUYpBiUGDmSOIYluHeOl4GrtklLKH8k4t/DVKF1Gf1QP55qfsidLeK?=
 =?iso-8859-1?Q?NE73R6a5fBZ5XKFtyzBm3qsHP9ssyr8rmkGdP9ITen0IgRhvG5OVc/bKRs?=
 =?iso-8859-1?Q?tam2LyuKNpAROfn0OOVO9VvsnRbyD6h9iVKgqY/tpPiQ0G74sBVv87AtPK?=
 =?iso-8859-1?Q?PO2ztuqH5F4pOlq2e1Jf2BreEA/wONLpxr3FTfKEPfLZup68SYTIGVvhuw?=
 =?iso-8859-1?Q?mzHNbCRQZd57o0clF3pSe6xYhX3R1dNAwFNBwPYcNfH2LcvKfQg1uNIwAQ?=
 =?iso-8859-1?Q?2jxg8cz/Fh6rPM2cy6duYpVgs3FzqnM4MhVAYfH/+HP4uz+TFqAASS0UOP?=
 =?iso-8859-1?Q?ul+6hKlEHh8IdGoCxZUTp3asGIX26OtMREwwxKyWP3Tqrlc9vIAY/DrJs1?=
 =?iso-8859-1?Q?uM5lXHZyI3jtdJz85ryXqJw8KRAq+gAXczkWsWWOd5PpD6c1Gl16VMAFmP?=
 =?iso-8859-1?Q?lN33a1UJ728flY2x5+mw9YFqW+usKkW9OYMHsMpynGbzpQGVl61TjXLSBx?=
 =?iso-8859-1?Q?UA+gOj17GLIK9MYsbP5lsOxoWgUmy6spdW2wZVYzRSCVRwiyE5J2w1Mhxg?=
 =?iso-8859-1?Q?XOHwDQWNwVFV9NFlUlf30nymXryCO7YdpHRVsD03T6UT5WZJOlTAJBkYer?=
 =?iso-8859-1?Q?PK6Q5P2YYsVgB/yQg/IzkIWDbKM6fLDQuoBkB+n7byq4JPjAefEGSZTVBW?=
 =?iso-8859-1?Q?nwsUpKasHQLp+9AoqysPqfbl0CIwQeIel9gk9hEmMbhpOK5m/ql6f768Cj?=
 =?iso-8859-1?Q?4yVgaC39JWPEST3TD5wVjAgY93Zv2Rk7XjWc41XoJ/38CuysbCv5K5bKHb?=
 =?iso-8859-1?Q?s2vsPf8q4UGYuT75+0huJi0kPOgKYhkrZ7vnvFD8SSuYwK1c0HvFkeo142?=
 =?iso-8859-1?Q?jC9HtaTMuaItg9SlqJJ2eFD1eHt2lkqwxCWqFpyQPEYT/es73wwmm5yONK?=
 =?iso-8859-1?Q?98Mw9HcmlwDnnGRywG0uxEFDQQIgHrO+sJf9742h/QYwtGlPGTOgiWbBTO?=
 =?iso-8859-1?Q?BnujCyHsEXyHEzAbDlY5LvnGegKMi5ULDHlAxJ+Zxk/C7YtathmM18wZ7T?=
 =?iso-8859-1?Q?pQsZPRXo/90HGl/5E3Ywjxt2AVVjUPEgjBSdEWQLZgy4uXibh6JOFcXfNL?=
 =?iso-8859-1?Q?qZPOcRfIvtdDJjCv45DCZJTk5758NJaFDWtny34NtK8q/AojSdcZzvNUQp?=
 =?iso-8859-1?Q?uRZsOo2b1oANKmOOiXrgthQ996JvMVmA=3D=3D?=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR03MB10456.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?IFQz9aCHIug/BIpKTZH2LDBDh19hnzHRlqSYNUrYxF2y/sZH3LpNcBnXNV?=
 =?iso-8859-1?Q?KzvtMRLLL6RPNEPfcGSvD68uHBt5tAyv/Ua6L187zsZGe3/+mrSsCZtKEs?=
 =?iso-8859-1?Q?ikFSbtuqqyrRXCW9Ult8xF3T/kQUYGmjndTRVXQukvbryc8N6aIAfXnu/h?=
 =?iso-8859-1?Q?HRjHaepErIiXX+IIQrJNDOrSR8IdTnsj8/5YnhNzwr22nNdqese9HsQoAT?=
 =?iso-8859-1?Q?gXDFESolPdBPLhbldxIGyBUlRxnVZ4GthiNtpBaWzcMOLL4TEmYobIn9pE?=
 =?iso-8859-1?Q?ZSYsdbhev7Fu+l92c5B6M8LoJiDuJkl0Q8RtrgQSm9qYm/lj/yx8Rx7cxG?=
 =?iso-8859-1?Q?jruYyjaver0bzPVg0PtoQuxWNnBEU/TKY1TFFSWr3ndC+v5OInK/SdrVJE?=
 =?iso-8859-1?Q?dGN0Rr4TZOk1leArEera3c5yJc9uNGociD5KOyWFJ7r2KzBLLw8IX73kNB?=
 =?iso-8859-1?Q?ikCDuNh2aX8UaHd78QU8iyfzvfWQsZNW/a977EVWd+kOpwaf6wZNHM+GOY?=
 =?iso-8859-1?Q?M0OTvEFy74wvvzcNfefPGLFzDxowQ9sbyi6NaMwwKhLNq1OZxbWThT42z+?=
 =?iso-8859-1?Q?vtNE2Yy/Mdrz+5kxdxG9O24pB1UHtWYLFCkDL/iqaApHKcVqw5ZCVeC/7p?=
 =?iso-8859-1?Q?3xErViSIpY7SignvM6CtGonaxAQBfHeV1qPhljCWvkcZFGJ8PvDcWPkmqd?=
 =?iso-8859-1?Q?FXKT1UYloPVwUCH5wKSClOVdqxqZrVZemoYIUq9J16lcCvNx41YBcVqQUR?=
 =?iso-8859-1?Q?M0hiigo3p3D+hVipkaAI/StzFZm+x8bbyYLICD5SlYGDA5bNMLuDnbz6uS?=
 =?iso-8859-1?Q?k/uDM3I4WyfhhH4YUZKn9CU4lNZ8G1FtyBUF/lqaVtK6VSqx+V2VlSBUUs?=
 =?iso-8859-1?Q?AenMSWvWm5KQxuZ/eno+9xs39x1xYtrgArb0w+VFOy6+bVwA5JSxsXCbR7?=
 =?iso-8859-1?Q?GANDQ7kRi1qy7bnDySCXxTXMDXsdoHkfIDFc/wSyAiA4VDVbRG0EHPHjXy?=
 =?iso-8859-1?Q?2Ifb6J2pCTPSyRTM9a7tKrSbYA3n2syl6IZ/ne5AvN7qsQRViJYH3m+zg7?=
 =?iso-8859-1?Q?Ri2RVO6xMsahdlINOiO9NRFCjKfaEJOzvdK4Q17Iri73m8bqs0UjhCTxrg?=
 =?iso-8859-1?Q?pRDfC1aCURa1uqs9kem5iDaYu5ySt9RQZSK2prdxM148njCksZCAhisHKw?=
 =?iso-8859-1?Q?Jukf0MJsWTvgEIHpgcwf0wFznra/xuZfNEmc5SwPlQFeNTfJQ+QOC1GdJb?=
 =?iso-8859-1?Q?XoiSI5Sc76PLyFwWJXyiX3QZDy91SfclNceETa/xIFLQB1TUswHU3yTv6W?=
 =?iso-8859-1?Q?XaWHw9R/2EKVi8Q+lKWMoRtN0GpFYGsnP/iLvzBPRHuHImntcgmkn7Kv33?=
 =?iso-8859-1?Q?MCuSDaYEEbuioVKcnWd4OLXsxzZkgWNP1SE1RUCD1+RklvsI2i1NZmK5ES?=
 =?iso-8859-1?Q?dOhdCeXAQ8EBobYTxYSvIgOKfYf2/vGXQXvenzIXq7IRlyWajR1UlETGcz?=
 =?iso-8859-1?Q?z9FYXg695XLzvLpaRhMfpvmfCwEwEwd5tZZU9IHJ9sqDyL6jFDSqCtk3xk?=
 =?iso-8859-1?Q?Ea/Ulnl71N135k2Xq1yTSTuOmFkAiqlLNlgVkov7hRlij7l3NP09yjAa/I?=
 =?iso-8859-1?Q?S3cb8NrbEuSO60rYJbahsbLxhWeZOg2JAGw2L4g4yVTBbwS0cvY7qlOg?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: GV1PR03MB10456.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c41824e4-2e25-47f9-cf7b-08dc819a0953
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 May 2024 17:49:36.0781
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 8NAts/c48ICppE65oL3AhLdvSHBwXVD4b/caml5xu6uDj5WlUgoCtlXYtd9yA/sARrnwt+O2GvZwA278Fms+x2kizh0PQaRQGmsnM7tO85g=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR03MB9970
X-Proofpoint-ORIG-GUID: FKuM9b5En1rrGkDo6Wd6YHpdg1cJrxEy
X-Proofpoint-GUID: FKuM9b5En1rrGkDo6Wd6YHpdg1cJrxEy
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_12,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0
 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 impostorscore=0
 mlxlogscore=999 clxscore=1015 bulkscore=0 lowpriorityscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2405170001 definitions=main-2405310135

Extend TEE mediator interface with two functions :

 - tee_get_type_from_dts() returns TEE type based on input string
 - tee_make_dtb_node() creates a DTB entry for the selected
   TEE mediator

Use those new functions to parse "xen,tee" DTS property for dom0less
guests and enable appropriate TEE mediator.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---

This is still RFC because I am not happy how I decide if
tee_make_dtb_node() needs to be called.

TEE type is stored in d_cfg, but d_cfg is not passed to
construct_domU()->prepare_dtb_domU(). So right now I am relying on
fact that every TEE mediator initilizes d->arch.tee.

Also I am sorry about previous completely botched version of this
patch. I really messed it up, including absence of [RFC] tag :(

---
 docs/misc/arm/device-tree/booting.txt | 17 +++++++++++++
 xen/arch/arm/dom0less-build.c         | 19 +++++++++++++++
 xen/arch/arm/include/asm/tee/tee.h    | 13 ++++++++++
 xen/arch/arm/tee/ffa.c                |  8 ++++++
 xen/arch/arm/tee/optee.c              | 35 +++++++++++++++++++++++++++
 xen/arch/arm/tee/tee.c                | 21 ++++++++++++++++
 6 files changed, 113 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-t=
ree/booting.txt
index bbd955e9c2..711a6080b5 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -231,6 +231,23 @@ with the following properties:
     In the future other possible property values might be added to
     enable only selected interfaces.
=20
+- xen,tee
+
+    A string property that describes what TEE mediator should be enabled
+    for the domain. Possible property values are:
+
+    - "none" (or missing property value)
+    No TEE will be available in the VM.
+
+    - "OP-TEE"
+    VM will have access to the OP-TEE using classic OP-TEE SMC interface.
+
+    - "FF-A"
+    VM will have access to a TEE using generic FF-A interface.
+
+    In the future other TEE mediators may be added, extending possible
+    values for this property.
+
 - xen,domain-p2m-mem-mb
=20
     Optional. A 32-bit integer specifying the amount of megabytes of RAM
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index fb63ec6fd1..13fdd44eef 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -15,6 +15,7 @@
 #include <asm/domain_build.h>
 #include <asm/static-memory.h>
 #include <asm/static-shmem.h>
+#include <asm/tee/tee.h>
=20
 bool __init is_dom0less_mode(void)
 {
@@ -650,6 +651,10 @@ static int __init prepare_dtb_domU(struct domain *d, s=
truct kernel_info *kinfo)
     if ( ret )
         goto err;
=20
+    /* We are making assumption that every mediator sets d->arch.tee */
+    if ( d->arch.tee )
+        tee_make_dtb_node(kinfo->fdt);
+
     /*
      * domain_handle_dtb_bootmodule has to be called before the rest of
      * the device tree is generated because it depends on the value of
@@ -871,6 +876,7 @@ void __init create_domUs(void)
         unsigned int flags =3D 0U;
         uint32_t val;
         int rc;
+        const char *tee_name;
=20
         if ( !dt_device_is_compatible(node, "xen,domain") )
             continue;
@@ -881,6 +887,19 @@ void __init create_domUs(void)
         if ( dt_find_property(node, "xen,static-mem", NULL) )
             flags |=3D CDF_staticmem;
=20
+        tee_name =3D dt_get_property(node, "xen,tee", NULL);
+        if ( tee_name )
+        {
+            rc =3D tee_get_type_from_dts(tee_name);
+            if ( rc < 0)
+                panic("Can't enable requested TEE for domain: %d\n", rc);
+            d_cfg.arch.tee_type =3D rc;
+        }
+        else
+        {
+            d_cfg.arch.tee_type =3D XEN_DOMCTL_CONFIG_TEE_NONE;
+        }
+
         if ( dt_property_read_bool(node, "direct-map") )
         {
             if ( !(flags & CDF_staticmem) )
diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/asm/=
tee/tee.h
index da324467e1..9626667545 100644
--- a/xen/arch/arm/include/asm/tee/tee.h
+++ b/xen/arch/arm/include/asm/tee/tee.h
@@ -36,6 +36,9 @@ struct tee_mediator_ops {
     int (*domain_init)(struct domain *d);
     int (*domain_teardown)(struct domain *d);
=20
+    /* Make DTB node that describes TEE. Used when creating a dom0less dom=
ain */
+    int (*make_dtb_node)(void *fdt);
+
     /*
      * Called during domain destruction to relinquish resources used
      * by mediator itself. This function can return -ERESTART to indicate
@@ -65,7 +68,9 @@ bool tee_handle_call(struct cpu_user_regs *regs);
 int tee_domain_init(struct domain *d, uint16_t tee_type);
 int tee_domain_teardown(struct domain *d);
 int tee_relinquish_resources(struct domain *d);
+int tee_make_dtb_node(void *fdt);
 uint16_t tee_get_type(void);
+int tee_get_type_from_dts(const char* prop_value);
=20
 #define REGISTER_TEE_MEDIATOR(_name, _namestr, _type, _ops)         \
 static const struct tee_mediator_desc __tee_desc_##_name __used     \
@@ -105,6 +110,14 @@ static inline uint16_t tee_get_type(void)
     return XEN_DOMCTL_CONFIG_TEE_NONE;
 }
=20
+static inline int tee_get_type_from_dts(const char* prop_value)
+{
+    if ( !strcmp(prop_value, "none") )
+        return XEN_DOMCTL_CONFIG_TEE_NONE;
+
+    return -ENODEV;
+}
+
 #endif  /* CONFIG_TEE */
=20
 #endif /* __ARCH_ARM_TEE_TEE_H__ */
diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 0793c1c758..f315d6eef6 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -1752,6 +1752,13 @@ err_free_ffa_rx:
     return false;
 }
=20
+static int ffa_make_dtb_node(void *fdt)
+{
+	/* FF-A is not configured via dtb */
+
+	return 0;
+}
+
 static const struct tee_mediator_ops ffa_ops =3D
 {
     .probe =3D ffa_probe,
@@ -1759,6 +1766,7 @@ static const struct tee_mediator_ops ffa_ops =3D
     .domain_teardown =3D ffa_domain_teardown,
     .relinquish_resources =3D ffa_relinquish_resources,
     .handle_call =3D ffa_handle_call,
+    .make_dtb_node =3D ffa_make_dtb_node,
 };
=20
 REGISTER_TEE_MEDIATOR(ffa, "FF-A", XEN_DOMCTL_CONFIG_TEE_FFA, &ffa_ops);
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 5151bd90ed..5eb5d01813 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -28,6 +28,7 @@
 #include <xen/domain_page.h>
 #include <xen/err.h>
 #include <xen/guest_access.h>
+#include <xen/libfdt/libfdt.h>
 #include <xen/mm.h>
 #include <xen/sched.h>
=20
@@ -1722,6 +1723,39 @@ static bool optee_handle_call(struct cpu_user_regs *=
regs)
     }
 }
=20
+static int __init optee_make_dtb_node(void *fdt)
+{
+    int res;
+
+    res =3D fdt_begin_node(fdt, "firmware");
+    if ( res )
+        return res;
+
+    res =3D fdt_begin_node(fdt, "optee");
+    if ( res )
+        return res;
+
+    res =3D fdt_property_string(fdt, "compatible", "linaro,optee-tz");
+    if ( res )
+        return res;
+
+    res =3D fdt_property_string(fdt, "method", "hvc");
+    if ( res )
+        return res;
+
+    /* end of "optee" */
+    res =3D fdt_end_node(fdt);
+    if ( res )
+        return res;
+
+    /* end of "firmware" */
+    res =3D fdt_end_node(fdt);
+    if ( res )
+        return res;
+
+    return 0;
+}
+
 static const struct tee_mediator_ops optee_ops =3D
 {
     .probe =3D optee_probe,
@@ -1729,6 +1763,7 @@ static const struct tee_mediator_ops optee_ops =3D
     .domain_teardown =3D optee_domain_teardown,
     .relinquish_resources =3D optee_relinquish_resources,
     .handle_call =3D optee_handle_call,
+    .make_dtb_node =3D optee_make_dtb_node,
 };
=20
 REGISTER_TEE_MEDIATOR(optee, "OP-TEE", XEN_DOMCTL_CONFIG_TEE_OPTEE, &optee=
_ops);
diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
index ddd17506a9..6388166e17 100644
--- a/xen/arch/arm/tee/tee.c
+++ b/xen/arch/arm/tee/tee.c
@@ -68,6 +68,14 @@ int tee_relinquish_resources(struct domain *d)
     return cur_mediator->ops->relinquish_resources(d);
 }
=20
+int tee_make_dtb_node(void *fdt)
+{
+    if ( !cur_mediator )
+        return -ENODEV;
+
+    return cur_mediator->ops->make_dtb_node(fdt);
+}
+
 uint16_t tee_get_type(void)
 {
     if ( !cur_mediator )
@@ -76,6 +84,19 @@ uint16_t tee_get_type(void)
     return cur_mediator->tee_type;
 }
=20
+int tee_get_type_from_dts(const char* prop_value)
+{
+    if ( !strcmp(prop_value, "none") )
+        return XEN_DOMCTL_CONFIG_TEE_NONE;
+
+    if ( !cur_mediator )
+        return -ENODEV;
+
+    if ( !strcmp(prop_value, cur_mediator->name) )
+        return cur_mediator->tee_type;
+
+    return -ENODEV;
+}
=20
 static int __init tee_init(void)
 {
--=20
2.45.1


From xen-devel-bounces@lists.xenproject.org Fri May 31 17:56:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 17:56:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733819.1140125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6UT-0000wr-Pr; Fri, 31 May 2024 17:56:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733819.1140125; Fri, 31 May 2024 17:56:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6UT-0000wk-Mj; Fri, 31 May 2024 17:56:33 +0000
Received: by outflank-mailman (input) for mailman id 733819;
 Fri, 31 May 2024 17:56:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bCkY=NC=epam.com=prvs=288140c18d=volodymyr_babchuk@srs-se1.protection.inumbo.net>)
 id 1sD6US-0000wY-Ip
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 17:56:32 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1bce38e9-1f77-11ef-90a1-e314d9c70b13;
 Fri, 31 May 2024 19:56:31 +0200 (CEST)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 44VHuK2s029588;
 Fri, 31 May 2024 17:56:20 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2044.outbound.protection.outlook.com [104.47.14.44])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3yfkd0g001-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 31 May 2024 17:56:20 +0000 (GMT)
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 (2603:10a6:150:16a::21) by VI2PR03MB10738.eurprd03.prod.outlook.com
 (2603:10a6:800:272::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.16; Fri, 31 May
 2024 17:56:16 +0000
Received: from GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e]) by GV1PR03MB10456.eurprd03.prod.outlook.com
 ([fe80::a41e:5aa8:e298:757e%3]) with mapi id 15.20.7633.001; Fri, 31 May 2024
 17:56:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bce38e9-1f77-11ef-90a1-e314d9c70b13
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=c9Ox6HNUuaq/U+vnoC/ewPWvfe4c1iRmXQGPvXGL7RpR0ubmJjHnVTDyzH17dIZ6kPVsTAFejGPPCgqYGvqSR5EDG8kUjaUVS4NZN+6kigRQ1sLcCn4076Er1HJLOyNprOHYl3WuqWnhQFBNGTJ/Ziiyw7NkLQjLD+ZX5PQy97gp8fHkjPdtdPqMEr9rUFNzthlnhX9Zr43Ppsyy0AmJqHAQzoiqUvyQfWwRqAuM6/KaNzebPLlKt6I1Vb/f6cyOf7RVmT6qPMUcDH2cxL2nV8+TvchutIZO/CHja+YYJqAnHsrPf94RiTIDmvfHbpZgxH/iEvsN6lMiopCZRDzW9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=S8snQEhOQFqU11fm4e3rbBUwlkvRc+kZu7Ez7QA0yKg=;
 b=CUssqmKU1v1zlvVSXIPdW2i2KLBkIYgv3HZlDD2bugDkvBnZI5gMPju8wjqdhDA0puYTIRA8jVYB/6XXFMVDysqdKNCb71D9k/mDwX4dD81r7qVU819NbzRSelzGA/ShEiKDgQusMUoeUWfwbHiXmMP3y+wvmxNyH5uXW7YkraxvRKP7HGAT7GkWLRbRV9mFJhYmtM64R2ryIhtZHKjsxQWJnmgKJqgwqrbBTw03um57Q84fkPUYxsjF4dBFrXSHpDDh5IpaULlTFnJ0myCYOHpTMxkv8JSaNy0aMmceQ+9lsE+BPDOJcRc7ND+Wcd3xV87NrFqCpA2pwBZbMB2niA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=S8snQEhOQFqU11fm4e3rbBUwlkvRc+kZu7Ez7QA0yKg=;
 b=hk9Y5U+A9D3En5BlXTqWpoXhzO+8ttPfDeME0nDO/rZXPGDBFTSNqY90Bmn1qX+HMwTdYtlZQ3hHRR1KWPF14ZBqjkARVCfTxPUcyAmZ1Yuy/fX3ee/dVOSTjg5sUknl+5hqS9dWdqg3rz/fgSYYR/ASSU48vGLwVCVx86i7a3VO7407c1SrgKNh4fe99l77bNNlCJQ4Yafy5l3VHZV6nUWNcAmZnmoloJ+qf7AkGL5S9TP67s/1NCwaecwot8pRrBjh8VqwrJ0/unmLjGa7jJv5f9sgAyFeyVJp/C++tO6OlirUzjWOKAUVRi6Wqlx9txHZVgcKjr24YRBIhX9pjA==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Stefano
 Stabellini <sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Bertrand
 Marquis <bertrand.marquis@arm.com>,
        Michal Orzel <michal.orzel@amd.com>
Subject: Re: [RFC PATCH v2] arm: dom0less: add TEE support
Thread-Topic: [RFC PATCH v2] arm: dom0less: add TEE support
Thread-Index: AQHas4LmA3+MKr53jE6Kc+pdYgdQGLGxoCWA
Date: Fri, 31 May 2024 17:56:16 +0000
Message-ID: <871q5hyif4.fsf@epam.com>
References: <20240531174915.1679443-1-volodymyr_babchuk@epam.com>
In-Reply-To: <20240531174915.1679443-1-volodymyr_babchuk@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: mu4e 1.10.7; emacs 29.3
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: GV1PR03MB10456:EE_|VI2PR03MB10738:EE_
x-ms-office365-filtering-correlation-id: b94ced4d-bd77-4646-1aea-08dc819af82d
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;ARA:13230031|366007|1800799015|376005|38070700009;
x-microsoft-antispam-message-info: 
 =?iso-8859-1?Q?c/BxCJI+Fjl75/Aex0fpEfL6jcyIPJXbX+wnJfA7aVG6fukUkhV8ASGOtv?=
 =?iso-8859-1?Q?J3her1vCSeGEdRKI7FOaKJ4Gb2G1bJO6qdn/xxBPRT2EwS/iCT7Uvi1+q3?=
 =?iso-8859-1?Q?BRGw/HS9Wdc8TY7w+mGxCAHRyUcJhf9knAZkEun2Ae8nBHvjxQ4HnYyeXH?=
 =?iso-8859-1?Q?soosTYfKi/xfCkWasen8QQMUVs2AZIcnbyDpm/FBf2w7LSAsXxm7sditKc?=
 =?iso-8859-1?Q?gn8cT4kXaIYA8hzGsj+tAA83TiZeEUz1GbPkga3pXsZyTT4c/nBSSWEQeU?=
 =?iso-8859-1?Q?ut25Rm/iLBsZ4fprtQgbA02bc7c2J/hy1dYVtkBxGgP0CKOBG7f7wnkkgd?=
 =?iso-8859-1?Q?Gp3frf4FvdCceUghR3NAgi6NCJJsEhKvqadxVWzdAPRKymGI4GuybI1hs4?=
 =?iso-8859-1?Q?L3kuYNm3a5rpsmx8gm78IK5jibH4k3UTls+JRqHce082d4K6LMYiSOTFlC?=
 =?iso-8859-1?Q?NuDS8paYa6hZI/UhFgO+Zs17xMTMCEEVyOIyJ5Y3o1ZFsHcx3cw9pjYDVj?=
 =?iso-8859-1?Q?VLRBZw8eKbrewRjoR9bMHBmuIIjY67wHRs+G6V6Unr7c6rFBIvC7uFVcMH?=
 =?iso-8859-1?Q?f1Kqow8cxeou/g5zyuoxuFksVsQR3x0CkER1+6cZgPcJfng9j++/bkhX/z?=
 =?iso-8859-1?Q?NmjZq8cAEAcfMQF5VC3Fa/5ueUZdeYemI+PXQp2xzZ5dObg16PFcLg2DyB?=
 =?iso-8859-1?Q?6ndWrozfK6QORA7OEn7xE5TZk3f4DbmaGxkPH/JkSVOkPAr0QUQWpQ2PfI?=
 =?iso-8859-1?Q?Ko/sMEV0ukO5IQj51PBkn8bRX/kyCKTatdckGtXUywkjw2C7IaAKdBoEEh?=
 =?iso-8859-1?Q?FfT+mUxAYjXGJppluL1IOPgW5YEOANhFxlgCiWpsAMjp52KduP8SvgfxM5?=
 =?iso-8859-1?Q?ln6xhSdXwUvRanShCYeqStXJJ1jGbTHaOj7EedfZIICZulxytIioZBVEXy?=
 =?iso-8859-1?Q?OKvahGIeGYw9QJ7LEOu/ISOzF3qSvrz87KXAzWoap8902Apx7ChR5eWfUR?=
 =?iso-8859-1?Q?lVeLSnlC96BFWRQnhpGLxYZNRRTvkmSMRi6hnBnTRYVt+IUEce734aatdu?=
 =?iso-8859-1?Q?fwLWq6AkY1+KqD+jhTwTHGmuZAYYtbbd4fbWMfIMPpmT7xKIQfjYo7hllW?=
 =?iso-8859-1?Q?+T4uekgKYdGi4sklVKLaTYL0sdb2cBOYYxX5CRab46TzUKnhXmthW+YrqU?=
 =?iso-8859-1?Q?g5j3thXo8ipBr2CE+XahZsJsHiZuDPBtcVcHl1QteUtFYq/lmOUuPzGtrG?=
 =?iso-8859-1?Q?9WbRhvLSDpp1FU8HMTQ0+Qn+/sv1qojwtj23YEF5BurYxIm1bK7t7dCXld?=
 =?iso-8859-1?Q?L0wa3qqOKDm5GMbzixKVDnNLFdH33LM98rio//yA2xLqdKbIUHHBPIKv0l?=
 =?iso-8859-1?Q?iQ4PykMkItsOEKxO1o2+zvtcE8BZ2FSQ=3D=3D?=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR03MB10456.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?EH1VhRUYp8AbJF4Eny1/u54Wu/J5PvLKG6BHG68FtAWw/uU2fjgFncwFkp?=
 =?iso-8859-1?Q?w//++pXUeONOmmqIA9Yu9YWwVLLufmYH9XRR+F4FuLPpYuFOapc1NZY2E1?=
 =?iso-8859-1?Q?g5wP6A514x/PLbNrCZWhcWhCYvEfuni3geWdBAMIP/LLI8kS96RytNp/iS?=
 =?iso-8859-1?Q?//7OVeCHfrq/eCFRpW7UyC9/PtVYwunnZM9gVYp2Xyj82HeMgF8DLMDi70?=
 =?iso-8859-1?Q?uHIbqDDep75vfosDPcqtB0RYmHInvDxwgeJ4dZowIQ1bMTWK3CzOWLyyMB?=
 =?iso-8859-1?Q?DZXy93+7HSwcUbe6+qaDEKf8CwctkVDmY2PTb2S78isPpCYY/lkxNCwlNl?=
 =?iso-8859-1?Q?nC0FJdU4FQNsIbM1oDCj2C1jqalIJ78f2uWA3rjm5xf2D8mIgJVf1ne/D1?=
 =?iso-8859-1?Q?JK4hDlIJFXOjyVCVkfCBTzm+0CYvYlXwkfm9qG0osccoS6JihwYnMYL5Nd?=
 =?iso-8859-1?Q?57EGkh0Ov1sawMqP1SmX6DiNC1eDN2hXTlj5lAj/yTqxAEU8HAFOb9OTY4?=
 =?iso-8859-1?Q?UeVBnzXJzWWWDezGqNQtdrTgWL/OEwod2TnmY0+8dU1yY+X9QSVkE6T5Xq?=
 =?iso-8859-1?Q?nfJfOg4BEiw/sB7WAVKOwrqyk+CG079+7Qh2GvMIW1dLQaBQkBkQQafbjB?=
 =?iso-8859-1?Q?zmK2jgALTdJsaSFybDWyS1G389eVq5Hpid78pKPK7Hw17jXRbbAJzdelUO?=
 =?iso-8859-1?Q?aITLpTJNzQ1IhD0lPasExNtFzyHBs9DKYcVwWAZhByhKBoULFuGSoSTtGl?=
 =?iso-8859-1?Q?e/WFNmuKWG/H58l0wkgEHg0oMqPJKWaI6zkExJiqun1lPAdR8rqrz2ZdGH?=
 =?iso-8859-1?Q?fP2TYtMXdhvUEeMCQgeReS/QGQsYtTZunDUDd7IH1rsFcmRC9LgNtsLZaM?=
 =?iso-8859-1?Q?PdykIhkUFPhFtPXDlAFXWd7Pu9DPYzn6DXD4jTwbhhqlFvaMj8PYqRoI8z?=
 =?iso-8859-1?Q?hJGbOQ1IWXSEutRc4jMmJ2LDwjRCiRiX/cbeh3QmFfH1v8XSGCLFZwr0PN?=
 =?iso-8859-1?Q?Fr6hU/eRc3vrhrHyKvTYj/zmIes6bHBTr2sKQFtSyUghmjUabAyu0+gKwR?=
 =?iso-8859-1?Q?H/3p4/6+yLEufuOrscEwk6PrT4aYCUKj7cuzcoPXoku7mpfU5jmjoRiFFx?=
 =?iso-8859-1?Q?V39DgTtK9/oEvmVVT4+1Q2TeuL9TwDULkvu85/N2oO76hq1x3kwDDl5Xxj?=
 =?iso-8859-1?Q?o4alA6b6g6R83oFjZVXr1kDH8UmDxo4r9b1ednj3ou144Ly2dQUYPZP/gL?=
 =?iso-8859-1?Q?g8/Cgh1rMN1Tt4s1Iri1LO+og1kBbSPc1jyCLyCi6QlYVRE8Dwpa/0Osph?=
 =?iso-8859-1?Q?dvoQMPnrvipwSHSeuLyNSg6vZ1kEfAE8+jov8k+xOpTL183agIabHJrNsv?=
 =?iso-8859-1?Q?I6KKpWrOQZgMERvlKvs6wTOsVwhrPBNJ/tEOIcP2wVQLOigiMuOJACRy1Y?=
 =?iso-8859-1?Q?D2g5UvBLZ8b8cn7cAE7p+CIp65LJaiMIm6PQvsq4URICvXoqYB3zZ0Q2tS?=
 =?iso-8859-1?Q?IgWtAIuScldletnhwQUCa3L15l4VsnyVRYWwUUYCkEv2KEVRB5YUukwUFB?=
 =?iso-8859-1?Q?Na01K1/4VufpCnxOj/aHM2O3Hx/PbKyhygJT2WyyFYUoTVS2nuU/ek4YDS?=
 =?iso-8859-1?Q?D7DyBcQbT87YchX5r/v2EhSCTKSMxhioJvONmuSRp7TWBiEO1Rh/XSOg?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: GV1PR03MB10456.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b94ced4d-bd77-4646-1aea-08dc819af82d
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 May 2024 17:56:16.8239
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2UOGBnowdRduewB9fZhgy5L0h96H/N2hm64b035AwBSYufCxCBiX70KxJcYbNfSGpa3oWWUnVWJH85hyxDvU25qQ57adfUFisjU1TW8IVlU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI2PR03MB10738
X-Proofpoint-GUID: nqLg42PsRG1SOBJnYjtXKtz8qHiAGHqR
X-Proofpoint-ORIG-GUID: nqLg42PsRG1SOBJnYjtXKtz8qHiAGHqR
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-31_12,2024-05-30_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999
 lowpriorityscore=0 bulkscore=0 spamscore=0 clxscore=1015 suspectscore=0
 impostorscore=0 malwarescore=0 priorityscore=1501 phishscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001
 definitions=main-2405310136


Hello all,

Volodymyr Babchuk <Volodymyr_Babchuk@epam.com> writes:

> Extend TEE mediator interface with two functions :
>
>  - tee_get_type_from_dts() returns TEE type based on input string
>  - tee_make_dtb_node() creates a DTB entry for the selected
>    TEE mediator
>
[..]

>  bool __init is_dom0less_mode(void)
>  {
> @@ -650,6 +651,10 @@ static int __init prepare_dtb_domU(struct domain *d,=
 struct kernel_info *kinfo)
>      if ( ret )
>          goto err;
>

I forgot to add

#ifdef CONFIG_TEE

> +    /* We are making assumption that every mediator sets d->arch.tee */
> +    if ( d->arch.tee )
> +        tee_make_dtb_node(kinfo->fdt);
> +
#endif

So build fails if TEE is disabled.

I'll fix this in the next version. Anyways, this is RFC.

>      /*
>       * domain_handle_dtb_bootmodule has to be called before the rest of
>       * the device tree is generated because it depends on the value of
> @@ -871,6 +876,7 @@ void __init create_domUs(void)
>          unsigned int flags =3D 0U;
>          uint32_t val;
>          int rc;
> +        const char *tee_name;

[...]

--=20
WBR, Volodymyr=


From xen-devel-bounces@lists.xenproject.org Fri May 31 18:14:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 18:14:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733827.1140134 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6ln-0003vE-6w; Fri, 31 May 2024 18:14:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733827.1140134; Fri, 31 May 2024 18:14:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sD6ln-0003v7-44; Fri, 31 May 2024 18:14:27 +0000
Received: by outflank-mailman (input) for mailman id 733827;
 Fri, 31 May 2024 18:14:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD6lm-0003ux-9V; Fri, 31 May 2024 18:14:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD6lm-0005Kc-7m; Fri, 31 May 2024 18:14:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1sD6ll-0004Ta-WF; Fri, 31 May 2024 18:14:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1sD6ll-0005jy-Vk; Fri, 31 May 2024 18:14:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Qv2oqzRQqnIuPXyEvEPY4/RrIlabjolXt7BFmTGseQ4=; b=yHxfdYy5PoL25wlAVHm06Lr1Eg
	KX2eHpYXUYpUqA3tUkTuAK++1JWeX6rjlrK4qzXe0jfjgyvpI5Ich/NcWrHHTNe+XKNHb2UWCK7Rk
	MTagSqB0EBIBKixwWSuQHxY+wuZjsqJtnmwQpziHARwlvMK41/NfJRZ0t/E7zpSdwYcU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-186211-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 186211: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3b36aa96de1d5f7a4660bec5c0cbad2616183dd6
X-Osstest-Versions-That:
    ovmf=7c584bb04874bb5bad16fcf3996f5a893cc81a1c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 31 May 2024 18:14:25 +0000

flight 186211 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186211/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3b36aa96de1d5f7a4660bec5c0cbad2616183dd6
baseline version:
 ovmf                 7c584bb04874bb5bad16fcf3996f5a893cc81a1c

Last test of basis   186210  2024-05-31 12:44:34 Z    0 days
Testing same since   186211  2024-05-31 16:14:27 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Shang Qingyu <qingyu.shang@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   7c584bb048..3b36aa96de  3b36aa96de1d5f7a4660bec5c0cbad2616183dd6 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri May 31 22:42:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 22:42:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733888.1140162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sDAwj-0000Qh-2B; Fri, 31 May 2024 22:42:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733888.1140162; Fri, 31 May 2024 22:42:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sDAwi-0000Qa-TI; Fri, 31 May 2024 22:42:00 +0000
Received: by outflank-mailman (input) for mailman id 733888;
 Fri, 31 May 2024 22:41:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Tcd8=NC=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1sDAwh-0000QU-0C
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 22:41:59 +0000
Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com
 [2607:f8b0:4864:20::1136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbfa2b07-1f9e-11ef-90a1-e314d9c70b13;
 Sat, 01 Jun 2024 00:41:57 +0200 (CEST)
Received: by mail-yw1-x1136.google.com with SMTP id
 00721157ae682-62a0849f5a8so24754727b3.2
 for <xen-devel@lists.xenproject.org>; Fri, 31 May 2024 15:41:57 -0700 (PDT)
Received: from [192.168.1.10] (host-92-26-98-202.as13285.net. [92.26.98.202])
 by smtp.gmail.com with ESMTPSA id
 af79cd13be357-794f3184388sm91021085a.125.2024.05.31.15.41.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 31 May 2024 15:41:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbfa2b07-1f9e-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1717195316; x=1717800116; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=nsT8LH63XtmTzmQcx0YFZfAbK7mznoH1MaRFA0+o4OU=;
        b=nMMUcGhB1Uoz9qRSFK1LTeFk0V6B/HOTd0d4MXktIjjuYmt/ruLeUfYVcY+bhNtiew
         BoiNEebgZToEXGe3oReTgQTt9AMWCGXGOhLlTyXYpeAjylXW1qIGX3eegykmKcpPeAFW
         MH2VKHFIZFWbl8zdZ9Jw8DPIJ7dpIz+SlPB2c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1717195316; x=1717800116;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nsT8LH63XtmTzmQcx0YFZfAbK7mznoH1MaRFA0+o4OU=;
        b=eSrdLhTItDTaxTaJxUIYO4//wQh3TqWVU0PRFal77esZW7gsUBpFL60vow97K7Rxx9
         Ri3SbkRJ871pNynQiAJfO141pI4n3KBw2In1E87Ul/buFJqlAFrhawJXRi+e5ucdGcAG
         JPN3pFlu8Hi0tgHr1Y4kOF/72jnoeyIk/f2nhm8o2sgBxWeclfuBkcTDdtJKh2kae833
         IzcW0S/OL0ln56b+vfYz+iqZXzo56YwAEO3Wm2kFdP0n9nj2P5ed+Y/iEKYM+TJeE/r8
         bDU7nqwkfXG16ccFpCcRbOecSSk2AQ49zU/o6lm/NsI19n65i/YiAbq9+cJ3dM0W60Nx
         Kk5g==
X-Forwarded-Encrypted: i=1; AJvYcCUq4vDxVRdqWzrhqhEolbkGV2WZmx0u/oo3OxFrnq2PIqZBab8S7MWUEmj2hCN2ywpjcqui+dScSSmTkpl5h3Tf8ar+QQEgq4oylOuk/Xk=
X-Gm-Message-State: AOJu0YxAr5GY8SnQw7ivE4Q2fF9P+nYeb293+5Z8xQZ8Sx733/WL+g4R
	045SZaQCSeiGxlDZAZa7DLuAwccV10B32qf/mWnAwd1JNddAf7w7NKa4HBvME+4=
X-Google-Smtp-Source: AGHT+IGplGr+YkdzvmcNPa32xsVdbr+IKqOMzTxGKfcHYWlerHx1lyK30RLd5pdN9JA/+HmVQ9QhGA==
X-Received: by 2002:a81:c907:0:b0:620:255a:a5c5 with SMTP id 00721157ae682-62c79852ef9mr32532317b3.32.1717195315461;
        Fri, 31 May 2024 15:41:55 -0700 (PDT)
Message-ID: <275fbd2a-9460-4836-8dd1-30f36cf137ae@citrix.com>
Date: Fri, 31 May 2024 23:41:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 02/13] xen/bitops: Cleanup ahead of rearrangements
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 "consulting @ bugseng . com" <consulting@bugseng.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Federico Serafini <federico.serafini@bugseng.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240524200338.1232391-1-andrew.cooper3@citrix.com>
 <20240524200338.1232391-3-andrew.cooper3@citrix.com>
 <c70f769b-57b9-4f71-9c31-8052b773d28a@suse.com>
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c70f769b-57b9-4f71-9c31-8052b773d28a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/05/2024 9:24 am, Jan Beulich wrote:
> On 24.05.2024 22:03, Andrew Cooper wrote:
>>  * Rename __attribute_pure__ to just __pure before it gains users.
>>  * Introduce __constructor which is going to be used in lib/, and is
>>    unconditionally cf_check.
>>  * Identify the areas of xen/bitops.h which are a mess.
>>  * Introduce xen/boot-check.h as helpers for compile and boot time testing.
>>    This provides a statement of the ABI, and a confirmation that arch-specific
>>    implementations behave as expected.
>>
>> Sadly Clang 7 and older isn't happy with the compile time checks.  Skip them,
>> and just rely on the runtime checks.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

>
> Further remarks, though:
>
>> ---
>>  xen/include/xen/bitops.h     | 13 ++++++--
>>  xen/include/xen/boot-check.h | 60 ++++++++++++++++++++++++++++++++++++
>>  xen/include/xen/compiler.h   |  3 +-
>>  3 files changed, 72 insertions(+), 4 deletions(-)
>>  create mode 100644 xen/include/xen/boot-check.h
> The bulk of the changes isn't about bitops; it's just that you're intending
> to first use it for testing there. The subject prefix therefore is somewhat
> misleading.

I'll change to "Cleanup and infrastructure ahead ..." but the bitops
aspect is still reasonably important.
>> --- /dev/null
>> +++ b/xen/include/xen/boot-check.h
>> @@ -0,0 +1,60 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +
>> +/*
>> + * Helpers for boot-time checks of basic logic, including confirming that
>> + * examples which should be calculated by the compiler are.
>> + */
>> +#ifndef XEN_BOOT_CHECK_H
>> +#define XEN_BOOT_CHECK_H

Given that CONFIG_SELF_TESTS was subsequently approved, I've renamed
this file to match.

>> +
>> +#include <xen/lib.h>
>> +
>> +/* Hide a value from the optimiser. */
>> +#define HIDE(x)                                                         \
>> +    ({ typeof(x) _x = (x); asm volatile ( "" : "+r" (_x) ); _x; })
> In principle this is a macro that could be of use elsewhere. That's also
> reflected in its entirely generic name. It therefore feels mis-placed in
> this header.

I'd forgotten that we several variations of this already.  compiler.h
has both OPTIMIZER_HIDE_VAR() and RELOC_HIDE().

>  Otoh though the use of "+r" is more restricting than truly
> necessary: While I'm not sure if "+g" would work, i.e. if that wouldn't
> cause issues with literals,

OPTIMIZER_HIDE_VAR() is indeed buggy using "+g", and RELOC_HIDE() even
explains how "g" tickles a bug in a compiler we probably don't care
about any more.

[Slightly out of order] the use of OPTIMIZER_HIDE_VAR() in gsi_vioapic()
is bogus AFAICT, and is actively creating the problem the commit message
says it was trying to avoid.

>  pretty surely "+rm" ought to work, removing
> the strict requirement for the compiler to put a certain value in a
> register.

"+rm" would be ideal in theory, we can't use it in practice because
Clang will (still!) interpret it as "+m" and force a spill.

While that's not necessarily a problem for the SELF_TESTS, it really is
a problem in array_index_mask_nospec(), which is latently buggy even now.

If the compiler really uses the flexibility offered by
OPTIMIZER_HIDE_VAR() to spill the value, array_index_mask_nospec() has
entirely failed at its purpose.

> Assuming you may have reservations against "+g" / "+rm" (and hence the
> construct wants keeping here), maybe rename to e.g. BOOT_CHECK_HIDE()?
> Alternatively, if generalized, moving to xen/macros.h would seem
> appropriate to me.

I've moved it to macros.h (because we should consolidate around it), but
kept as "+r" for both Clang and array_index_mask_nospec() reasons.

I don't expect HIDE() is ever actually going to be used in a case where
letting the value stay in memory is a useful thing overall.  But if you
still feel strongly about it, we can debate further when consolidating
the other users.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri May 31 22:48:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 31 May 2024 22:48:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.733893.1140170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sDB3F-00013Q-Kt; Fri, 31 May 2024 22:48:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 733893.1140170; Fri, 31 May 2024 22:48:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1sDB3F-00013J-IM; Fri, 31 May 2024 22:48:45 +0000
Received: by outflank-mailman (input) for mailman id 733893;
 Fri, 31 May 2024 22:48:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xPZo=NC=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1sDB3E-00013B-KH
 for xen-devel@lists.xenproject.org; Fri, 31 May 2024 22:48:44 +0000
Received: from wfhigh2-smtp.messagingengine.com
 (wfhigh2-smtp.messagingengine.com [64.147.123.153])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ec5b1f42-1f9f-11ef-90a1-e314d9c70b13;
 Sat, 01 Jun 2024 00:48:42 +0200 (CEST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailfhigh.west.internal (Postfix) with ESMTP id 7095718000BA;
 Fri, 31 May 2024 18:48:38 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Fri, 31 May 2024 18:48:38 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 31 May 2024 18:48:36 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec5b1f42-1f9f-11ef-90a1-e314d9c70b13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm1; t=1717195718;
	 x=1717282118; bh=xBqG5KeyHw6loZw0/cyADHLe7k/DxHhcD0woNr8hcJM=; b=
	rx9u5Qp02zYTx6ZIMxPXsbNxEH6kV5FDLGJ+kpgezKK3bf/3WXtF205FFj4sChwG
	dhzSis//Zktvmiie9lDbiHaCafj1UXUJMWM3EyuQ82jhzjx2c7vaC8O/dxZdfjG8
	5msK8j56il9t4jc0iP/EQ41Wt659h9bssY7vsKi05GHaXrJ3YpkNUs9fp5vtxfLw
	Kwfh3kdEbgu4pO/R/G3HO/YLaQkClWXQOAbs3FEqAVnmGvFeRJU2dnkNWEio0XWP
	aiBimquILN4flQ7Xf56zln3/pvkklm8hd9S1Y6hv1Ho6xJUHHb2MMFpjlMv4rU8x
	FgpNDkb96/GoOHvpaXRxuA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1717195718; x=1717282118; bh=xBqG5KeyHw6loZw0/cyADHLe7k/D
	xHhcD0woNr8hcJM=; b=oU2Qun8Wz3C+AVTKd/Kx0pvKv+XFaWc1n/fobD2akB6Y
	UEZNHy2n0/CVgIuXtKZUur4+3KPW8hVJujbbmE/GXhNEtKhtudUOrBCmQHo0UvD2
	grw3sVONIvhl18h3+DGqy85e7dcnbG4B56Mbl1fdyE1mLBPvSftbSJY2q+X+9sBC
	wXDREWoqmcjuKUWwq9vcSk9sHzw0bHVh9X3AqTjkN+rdj0FgVonf3PnP3D+Y6QkV
	gjADxry3b0tW4Z7mByUB72mTuR3qweTJjn8KKeNAwX5FN/SKJQX1RY3CCQjykxfV
	Oscm21Lv1xxQfnfSse7Z0MMHnf1pwXxkueafQU5XjQ==
X-ME-Sender: <xms:xVNaZuqUeSGShb2KukS9kjKwHP9-ZfVSlyk4ZPB4zua9AOwhic1fmg>
    <xme:xVNaZsq_PPhi5u0Iy_4jWZTQFfo0Q7rvZCVNt1HbElnT20lWRBvKr5Lb-XbIXcZFQ
    lPEHCeMcgUB3g>
X-ME-Received: <xmr:xVNaZjN9xTcEvAkS0cUrSfUpk94mStUZ6tu_qW-R6cRA9QZ4tovhvfUuuh8>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekjedgudegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtroertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepheeh
    ueektedufeelvddvffeghedtjeeludehffeuuddugfettdelvdffjeduheevnecuffhomh
    grihhnpehophgvnhhsuhhsvgdrohhrghdpqhhusggvshdqohhsrdhorhhgpdhgihhthhhu
    sgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh
    hmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:xVNaZt6MeRd7BmAMHdrDT4gwwgfdXWJrddEgo6dS7RDYoMfccDqLfg>
    <xmx:xVNaZt4yPNABCzf5cI8gsyctWE6w6IJUX_RxeaY2PsbB60wnIBw-og>
    <xmx:xVNaZti446jogTuRrPrg6OIpKRvBzOA4Z9pFe4CXzmmsq5NXkQKNrg>
    <xmx:xVNaZn5VxapMr8BMG2di-BVVm6HuWMb3YmJOkYDij5GNm9SXDscpzw>
    <xmx:xlNaZtF9OzQnnI6oAkjg0Z4j11NWOVQiKdeypRocfaYWFhA2YnBzlqXl>
Feedback-ID: i1568416f:Fastmail
Date: Sat, 1 Jun 2024 00:48:33 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: NULL pointer dereference in xenbus_thread->...
Message-ID: <ZlpTwbmDjNLkCNgH@mail-itl>
References: <ZO0WrR5J0xuwDIxW@mail-itl>
 <ZTUuRj6e5x5xFVqb@mail-itl>
 <ZgGjf3hpLHXXtb8z@mail-itl>
 <0f8c0e27-e60d-4e64-bc8a-6cb407c67ab2@xen.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="4ybz0eHqAMeE41R+"
Content-Disposition: inline
In-Reply-To: <0f8c0e27-e60d-4e64-bc8a-6cb407c67ab2@xen.org>


--4ybz0eHqAMeE41R+
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sat, 1 Jun 2024 00:48:33 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: NULL pointer dereference in xenbus_thread->...

On Tue, Mar 26, 2024 at 11:00:50AM +0000, Julien Grall wrote:
> Hi Marek,
>=20
> +Juergen for visibility
>=20
> When sending a bug report, I would suggest to CC relevant people as
> otherwise it can get lost (not may people monitors Xen devel if they are =
not
> CCed).
>=20
> Cheers,
>=20
> On 25/03/2024 16:17, Marek Marczykowski-G=C3=B3recki wrote:
> > On Sun, Oct 22, 2023 at 04:14:30PM +0200, Marek Marczykowski-G=C3=B3rec=
ki wrote:
> > > On Mon, Aug 28, 2023 at 11:50:36PM +0200, Marek Marczykowski-G=C3=B3r=
ecki wrote:
> > > > Hi,
> > > >=20
> > > > I've noticed in Qubes's CI failure like this:
> > > >=20
> > > > [  871.271292] BUG: kernel NULL pointer dereference, address: 00000=
00000000000
> > > > [  871.275290] #PF: supervisor read access in kernel mode
> > > > [  871.277282] #PF: error_code(0x0000) - not-present page
> > > > [  871.279182] PGD 106fdb067 P4D 106fdb067 PUD 106fdc067 PMD 0
> > > > [  871.281071] Oops: 0000 [#1] PREEMPT SMP NOPTI
> > > > [  871.282698] CPU: 1 PID: 28 Comm: xenbus Not tainted 6.1.43-1.qub=
es.fc37.x86_64 #1
> > > > [  871.285222] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996=
), BIOS rel-1.16.0-0-gd239552-rebuilt.opensuse.org 04/01/2014
> > > > [  871.288883] RIP: e030:__wake_up_common+0x4c/0x180
> > > > [  871.292838] Code: 24 0c 89 4c 24 08 4d 85 c9 74 0a 41 f6 01 04 0=
f 85 a3 00 00 00 48 8b 43 08 4c 8d 40 e8 48 83 c3 08 49 8d 40 18 48 39 c3 7=
4 5b <49> 8b 40 18 31 ed 4c 8d 70 e8 45 8b 28 41 f6 c5 04 75 5f 49 8b 40
> > > > [  871.299776] RSP: e02b:ffffc900400f7e10 EFLAGS: 00010082
> > > > [  871.301656] RAX: 0000000000000000 RBX: ffff88810541ce98 RCX: 000=
0000000000000
> > > > [  871.304255] RDX: 0000000000000001 RSI: 0000000000000003 RDI: fff=
f88810541ce90
> > > > [  871.306714] RBP: ffffc900400f0280 R08: ffffffffffffffe8 R09: fff=
fc900400f7e68
> > > > [  871.309937] R10: 0000000000007ff0 R11: ffff888100ad3000 R12: fff=
fc900400f7e68
> > > > [  871.312326] R13: 0000000000000000 R14: 0000000000000000 R15: 000=
0000000000000
> > > > [  871.314647] FS:  0000000000000000(0000) GS:ffff88813ff00000(0000=
) knlGS:0000000000000000
> > > > [  871.317677] CS:  10000e030 DS: 0000 ES: 0000 CR0: 00000000800500=
33
> > > > [  871.319644] CR2: 0000000000000000 CR3: 00000001067fe000 CR4: 000=
0000000040660
> > > > [  871.321973] Call Trace:
> > > > [  871.322782]  <TASK>
> > > > [  871.323494]  ? show_trace_log_lvl+0x1d3/0x2ef
> > > > [  871.324901]  ? show_trace_log_lvl+0x1d3/0x2ef
> > > > [  871.326310]  ? show_trace_log_lvl+0x1d3/0x2ef
> > > > [  871.327721]  ? __wake_up_common_lock+0x82/0xd0
> > > > [  871.329147]  ? __die_body.cold+0x8/0xd
> > > > [  871.330378]  ? page_fault_oops+0x163/0x1a0
> > > > [  871.331691]  ? exc_page_fault+0x70/0x170
> > > > [  871.332946]  ? asm_exc_page_fault+0x22/0x30
> > > > [  871.334454]  ? __wake_up_common+0x4c/0x180
> > > > [  871.335777]  __wake_up_common_lock+0x82/0xd0
> > > > [  871.337183]  ? process_writes+0x240/0x240
> > > > [  871.338461]  process_msg+0x18e/0x2f0
> > > > [  871.339627]  xenbus_thread+0x165/0x1c0
> > > > [  871.340830]  ? cpuusage_read+0x10/0x10
> > > > [  871.342032]  kthread+0xe9/0x110
> > > > [  871.343317]  ? kthread_complete_and_exit+0x20/0x20
> > > > [  871.345020]  ret_from_fork+0x22/0x30
> > > > [  871.346239]  </TASK>
> > > > [  871.347060] Modules linked in: snd_hda_codec_generic ledtrig_aud=
io snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec snd_hda_=
core snd_hwdep snd_seq snd_seq_device joydev snd_pcm intel_rapl_msr ppdev i=
ntel_rapl_common snd_timer pcspkr e1000e snd soundcore i2c_piix4 parport_pc=
 parport loop fuse xenfs dm_crypt crct10dif_pclmul crc32_pclmul crc32c_inte=
l polyval_clmulni polyval_generic floppy ghash_clmulni_intel sha512_ssse3 s=
erio_raw virtio_scsi virtio_console bochs xhci_pci xhci_pci_renesas xhci_hc=
d qemu_fw_cfg drm_vram_helper drm_ttm_helper ttm ata_generic pata_acpi xen_=
privcmd xen_pciback xen_blkback xen_gntalloc xen_gntdev xen_evtchn scsi_dh_=
rdac scsi_dh_emc scsi_dh_alua uinput dm_multipath
> > > > [  871.368892] CR2: 0000000000000000
> > > > [  871.370160] ---[ end trace 0000000000000000 ]---
> > > > [  871.371719] RIP: e030:__wake_up_common+0x4c/0x180
> > > > [  871.373273] Code: 24 0c 89 4c 24 08 4d 85 c9 74 0a 41 f6 01 04 0=
f 85 a3 00 00 00 48 8b 43 08 4c 8d 40 e8 48 83 c3 08 49 8d 40 18 48 39 c3 7=
4 5b <49> 8b 40 18 31 ed 4c 8d 70 e8 45 8b 28 41 f6 c5 04 75 5f 49 8b 40
> > > > [  871.379866] RSP: e02b:ffffc900400f7e10 EFLAGS: 00010082
> > > > [  871.381689] RAX: 0000000000000000 RBX: ffff88810541ce98 RCX: 000=
0000000000000
> > > > [  871.383971] RDX: 0000000000000001 RSI: 0000000000000003 RDI: fff=
f88810541ce90
> > > > [  871.386235] RBP: ffffc900400f0280 R08: ffffffffffffffe8 R09: fff=
fc900400f7e68
> > > > [  871.388521] R10: 0000000000007ff0 R11: ffff888100ad3000 R12: fff=
fc900400f7e68
> > > > [  871.390789] R13: 0000000000000000 R14: 0000000000000000 R15: 000=
0000000000000
> > > > [  871.393101] FS:  0000000000000000(0000) GS:ffff88813ff00000(0000=
) knlGS:0000000000000000
> > > > [  871.395671] CS:  10000e030 DS: 0000 ES: 0000 CR0: 00000000800500=
33
> > > > [  871.397863] CR2: 0000000000000000 CR3: 00000001067fe000 CR4: 000=
0000000040660
> > > > [  871.400441] Kernel panic - not syncing: Fatal exception
> > > > [  871.402171] Kernel Offset: disabled
> > > > (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
> > > >=20
> > > > It isn't the first time I see similar crash, but I can't really
> > > > reproduce it reliably. Restarted test usually passes.
> > > > Note this is Xen nested in KVM, so it could very well be some oddity
> > > > about nested virt, although looking at the stack trace, it's unlike=
ly
> > > > and more likely some race condition hit only on slower system.
> > >=20
> > > Recently I've got the same crash on a real system in domU too. And al=
so
> > > on nested on newer kernel 6.1.57 (here it happened in dom0). So, this=
 is
> > > still an issue and affects not only nested case :/
> > >=20
> > > > Unfortunately I don't have symbols for this kernel handy, but there=
 is a
> > > > single wake_up() call in process_writes(), so it shouldn't be an is=
sue.
> > > >=20
> > > > Any ideas?
> > > >=20
> > > > Full log at https://openqa.qubes-os.org/tests/80779/logfile?filenam=
e=3Dserial0.txt
> > >=20
> > > More links at https://github.com/QubesOS/qubes-issues/issues/8638,
> > > including more recent stack trace.
> >=20
> > Happens on 6.1.75 too (new stack trace I've added to the issue above,
> > but it's pretty similar).

Recently I've got a report from another user about similar issue, on
6.6.29 this time. I also still encounter this issue once a month or so,
but the user claims they get it much more often:
https://github.com/QubesOS/qubes-issues/issues/8638#issuecomment-2135419896
The extra conditions reported by the user are:
- old AMD system (KGPE-D16 with Opteron 6282 SE) requiring
  `spec-ctrl=3Dibpb-entry=3Dno-pv` to remain usable=20
- Whonix domU, which has a bunch of sysctl parameters changed, listed
  at:
  - https://github.com/Kicksecure/security-misc
  - https://github.com/Kicksecure/security-misc/blob/master/usr/lib/sysctl.=
d/990-security-misc.conf
  (unsure which are relevant, maybe `vm.swappiness=3D1`?)


--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--4ybz0eHqAMeE41R+
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZaU8EACgkQ24/THMrX
1yzlBQf/U9j0tGuo4A2afz3xwZ7iglL8ZWb6RE7/XS33u15Be96Zpbi4LkzVEXAl
lz9Qyl1wbS3OHMvXeuv+HerZWNXIWOe6lqIonlQBlrFkJn/aC7QHhh0BGb1+S1i9
0DcPMjtxVTfzBDBD6qSi3WlkQgr+GKSChxUqYCAQiQTBzQOrZj9V5AlK4uXSTj+T
E4hxGJRQO229qfFPe81C1SSTUhJ72Y+kcfFvagKRveHxRoJLas1sq82G9v42w5dt
I3rxLAQwrWfTuAQtnl3LrPYUvqIK473AS3J/TGlWv4fUJsyWr+F6Sg+jw21sIXoZ
GrQNSzqTh6+FubfuZmLmZw/64yX4TQ==
=l/kf
-----END PGP SIGNATURE-----

--4ybz0eHqAMeE41R+--